2023-10-09T06:16:24.926 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-09T06:16:24.926 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2023-10-09_05:36:44-fs-wip-yuri-testing-2023-10-06-0949-quincy-testing-default-smithi/7418377 --name vshankar-2023-10-09_05:36:44-fs-wip-yuri-testing-2023-10-06-0949-quincy-testing-default-smithi --description fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} --verbose -- /home/teuthworker/archive/vshankar-2023-10-09_05:36:44-fs-wip-yuri-testing-2023-10-06-0949-quincy-testing-default-smithi/7418377/orig.config.yaml 2023-10-09T06:16:24.980 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-09T06:16:25.058 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2023-10-09_05:36:44-fs-wip-yuri-testing-2023-10-06-0949-quincy-testing-default-smithi/7418377 branch: wip-yuri-testing-2023-10-06-0949-quincy description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} email: vshankar@redhat.com first_in_suite: false job_id: '7418377' kernel: client: sha1: distro flavor: default kdb: true sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2023-10-09_05:36:44-fs-wip-yuri-testing-2023-10-06-0949-quincy-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri-testing-2023-10-06-0949-quincy ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20231009.053123 sha1: 62a27f0d8453e2104f759ed08ba781e24c5ed056 owner: scheduled_vshankar@teuthology priority: 30 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 8174 sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-vshankar-testing-20231009.053123 suite_path: /home/teuthworker/src/github.com_vshankar_ceph_62a27f0d8453e2104f759ed08ba781e24c5ed056/qa suite_relpath: qa suite_repo: https://github.com/vshankar/ceph.git suite_sha1: 62a27f0d8453e2104f759ed08ba781e24c5ed056 targets: smithi163.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN9GGg0ezV7qRelQs7pwRE+MZtSu6jseUyPPtZsCZIeNOgmh5+zpfPVNBkjVEpqTmyPSjfzG3lVrFWglAlauRCs= smithi181.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFYiqfrDjs8ACW9S/FUJkeTfDNLufZAP2urZo97qL+CxrOpH2qyVALjYHlaRDJXbPBxrqZmj0/7St9RDFNpBooo= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestVolumes - tasks.cephfs.test_volumes.TestSubvolumeGroups - tasks.cephfs.test_volumes.TestSubvolumes teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-09_05:36:44 tube: smithi use_shaman: true user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-09T06:16:25.059 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_vshankar_ceph_62a27f0d8453e2104f759ed08ba781e24c5ed056/qa; will attempt to use it 2023-10-09T06:16:25.059 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_vshankar_ceph_62a27f0d8453e2104f759ed08ba781e24c5ed056/qa/tasks 2023-10-09T06:16:25.060 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-09T06:16:25.061 INFO:teuthology.task.internal:Checking packages... 2023-10-09T06:16:25.089 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '08cac0187919fc3a247efdf445f652e2fb30a647' 2023-10-09T06:16:25.090 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-09T06:16:25.090 INFO:teuthology.packaging:ref: None 2023-10-09T06:16:25.090 INFO:teuthology.packaging:tag: None 2023-10-09T06:16:25.090 INFO:teuthology.packaging:branch: wip-yuri-testing-2023-10-06-0949-quincy 2023-10-09T06:16:25.090 INFO:teuthology.packaging:sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-09T06:16:25.090 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri-testing-2023-10-06-0949-quincy 2023-10-09T06:16:25.262 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1320.g08cac018 2023-10-09T06:16:25.263 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-09T06:16:25.273 INFO:teuthology.task.internal:no buildpackages task found 2023-10-09T06:16:25.273 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-09T06:16:25.318 INFO:teuthology.task.internal:Saving configuration 2023-10-09T06:16:25.337 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-09T06:16:25.395 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-09T06:16:25.421 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi163.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-10-09_05:36:44-fs-wip-yuri-testing-2023-10-06-0949-quincy-testing-default-smithi/7418377', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-09 06:10:37.184420', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN9GGg0ezV7qRelQs7pwRE+MZtSu6jseUyPPtZsCZIeNOgmh5+zpfPVNBkjVEpqTmyPSjfzG3lVrFWglAlauRCs='} 2023-10-09T06:16:25.451 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-10-09_05:36:44-fs-wip-yuri-testing-2023-10-06-0949-quincy-testing-default-smithi/7418377', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-09 06:10:37.185594', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFYiqfrDjs8ACW9S/FUJkeTfDNLufZAP2urZo97qL+CxrOpH2qyVALjYHlaRDJXbPBxrqZmj0/7St9RDFNpBooo='} 2023-10-09T06:16:25.451 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-09T06:16:25.468 INFO:teuthology.task.internal:roles: ubuntu@smithi163.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-09T06:16:25.469 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-09T06:16:25.469 INFO:teuthology.run_tasks:Running task console_log... 2023-10-09T06:16:25.704 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9faf874b80>, signals=[15]) 2023-10-09T06:16:25.705 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-09T06:16:25.715 INFO:teuthology.task.internal:Opening connections... 2023-10-09T06:16:25.716 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi163.front.sepia.ceph.com 2023-10-09T06:16:25.718 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-09T06:16:25.794 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi181.front.sepia.ceph.com 2023-10-09T06:16:25.795 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-09T06:16:25.864 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-09T06:16:25.931 DEBUG:teuthology.orchestra.run.smithi163:> uname -m 2023-10-09T06:16:25.949 INFO:teuthology.orchestra.run.smithi163.stdout:x86_64 2023-10-09T06:16:25.949 DEBUG:teuthology.orchestra.run.smithi163:> cat /etc/os-release 2023-10-09T06:16:26.007 INFO:teuthology.orchestra.run.smithi163.stdout:NAME="Red Hat Enterprise Linux" 2023-10-09T06:16:26.007 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION="8.4 (Ootpa)" 2023-10-09T06:16:26.007 INFO:teuthology.orchestra.run.smithi163.stdout:ID="rhel" 2023-10-09T06:16:26.007 INFO:teuthology.orchestra.run.smithi163.stdout:ID_LIKE="fedora" 2023-10-09T06:16:26.008 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_ID="8.4" 2023-10-09T06:16:26.008 INFO:teuthology.orchestra.run.smithi163.stdout:PLATFORM_ID="platform:el8" 2023-10-09T06:16:26.008 INFO:teuthology.orchestra.run.smithi163.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-09T06:16:26.008 INFO:teuthology.orchestra.run.smithi163.stdout:ANSI_COLOR="0;31" 2023-10-09T06:16:26.008 INFO:teuthology.orchestra.run.smithi163.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-09T06:16:26.008 INFO:teuthology.orchestra.run.smithi163.stdout:HOME_URL="https://www.redhat.com/" 2023-10-09T06:16:26.008 INFO:teuthology.orchestra.run.smithi163.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-09T06:16:26.008 INFO:teuthology.orchestra.run.smithi163.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-09T06:16:26.008 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:16:26.008 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-09T06:16:26.008 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-09T06:16:26.009 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-09T06:16:26.009 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-09T06:16:26.009 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server 2023-10-09T06:16:26.030 DEBUG:teuthology.orchestra.run.smithi181:> uname -m 2023-10-09T06:16:26.047 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2023-10-09T06:16:26.048 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2023-10-09T06:16:26.107 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="Red Hat Enterprise Linux" 2023-10-09T06:16:26.107 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="8.4 (Ootpa)" 2023-10-09T06:16:26.107 INFO:teuthology.orchestra.run.smithi181.stdout:ID="rhel" 2023-10-09T06:16:26.107 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="fedora" 2023-10-09T06:16:26.107 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="8.4" 2023-10-09T06:16:26.107 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el8" 2023-10-09T06:16:26.107 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-09T06:16:26.107 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2023-10-09T06:16:26.108 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-09T06:16:26.108 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://www.redhat.com/" 2023-10-09T06:16:26.108 INFO:teuthology.orchestra.run.smithi181.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-09T06:16:26.108 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-09T06:16:26.108 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:16:26.108 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-09T06:16:26.108 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-09T06:16:26.108 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-09T06:16:26.108 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-09T06:16:26.108 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2023-10-09T06:16:26.130 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-09T06:16:26.168 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-09T06:16:26.254 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-09T06:16:26.254 DEBUG:teuthology.orchestra.run.smithi163:> test '!' -e /home/ubuntu/cephtest 2023-10-09T06:16:26.261 DEBUG:teuthology.orchestra.run.smithi181:> test '!' -e /home/ubuntu/cephtest 2023-10-09T06:16:26.276 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-09T06:16:26.332 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-09T06:16:26.332 DEBUG:teuthology.orchestra.run.smithi163:> test -z $(ls -A /var/lib/ceph) 2023-10-09T06:16:26.335 DEBUG:teuthology.orchestra.run.smithi181:> test -z $(ls -A /var/lib/ceph) 2023-10-09T06:16:26.349 INFO:teuthology.orchestra.run.smithi163.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-09T06:16:26.351 INFO:teuthology.orchestra.run.smithi181.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-09T06:16:26.352 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-09T06:16:26.489 INFO:teuthology.run_tasks:Running task kernel... 2023-10-09T06:16:26.709 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c'} 2023-10-09T06:16:26.710 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-10-09T06:16:26.710 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-10-09T06:16:26.710 DEBUG:teuthology.orchestra.run.smithi163:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-09T06:16:26.710 DEBUG:teuthology.orchestra.run.smithi181:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-09T06:16:26.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-09T06:16:26.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-09T06:16:26.727 DEBUG:teuthology.orchestra.run.smithi181:> uname -r 2023-10-09T06:16:26.728 DEBUG:teuthology.orchestra.run.smithi163:> uname -r 2023-10-09T06:16:26.787 INFO:teuthology.orchestra.run.smithi181.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-09T06:16:26.787 INFO:teuthology.orchestra.run.smithi163.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-09T06:16:26.787 INFO:teuthology.task.kernel:Running kernel on smithi181: 4.18.0-305.25.1.el8_4.x86_64 2023-10-09T06:16:26.787 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum install -y kernel 2023-10-09T06:16:26.788 INFO:teuthology.task.kernel:Running kernel on smithi163: 4.18.0-305.25.1.el8_4.x86_64 2023-10-09T06:16:26.788 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum install -y kernel 2023-10-09T06:16:28.178 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2023-10-09T06:16:28.178 INFO:teuthology.orchestra.run.smithi181.stdout:Unable to read consumer identity 2023-10-09T06:16:28.276 INFO:teuthology.orchestra.run.smithi163.stdout:Updating Subscription Management repositories. 2023-10-09T06:16:28.277 INFO:teuthology.orchestra.run.smithi163.stdout:Unable to read consumer identity 2023-10-09T06:16:29.582 INFO:teuthology.orchestra.run.smithi181.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-10-09T06:16:29.682 INFO:teuthology.orchestra.run.smithi163.stdout:Copr repo for python3-asyncssh owned by ceph 29 kB/s | 3.5 kB 00:00 2023-10-09T06:16:30.146 INFO:teuthology.orchestra.run.smithi181.stdout:Extra Packages for Enterprise Linux 30 MB/s | 16 MB 00:00 2023-10-09T06:16:30.403 INFO:teuthology.orchestra.run.smithi163.stdout:Extra Packages for Enterprise Linux 23 MB/s | 16 MB 00:00 2023-10-09T06:16:33.434 INFO:teuthology.orchestra.run.smithi181.stdout:lab-extras 4.7 MB/s | 24 kB 00:00 2023-10-09T06:16:33.751 INFO:teuthology.orchestra.run.smithi163.stdout:lab-extras 4.4 MB/s | 24 kB 00:00 2023-10-09T06:16:34.178 INFO:teuthology.orchestra.run.smithi181.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-10-09T06:16:34.178 INFO:teuthology.orchestra.run.smithi181.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-10-09T06:16:34.178 INFO:teuthology.orchestra.run.smithi181.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-10-09T06:16:34.339 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-10-09T06:16:34.340 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2023-10-09T06:16:34.340 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-10-09T06:16:34.449 DEBUG:teuthology.orchestra.run.smithi181:> echo no | sudo yum reinstall kernel || true 2023-10-09T06:16:34.497 INFO:teuthology.orchestra.run.smithi163.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-10-09T06:16:34.497 INFO:teuthology.orchestra.run.smithi163.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-10-09T06:16:34.497 INFO:teuthology.orchestra.run.smithi163.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-10-09T06:16:34.660 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2023-10-09T06:16:34.661 INFO:teuthology.orchestra.run.smithi163.stdout:Nothing to do. 2023-10-09T06:16:34.661 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2023-10-09T06:16:34.777 DEBUG:teuthology.orchestra.run.smithi163:> echo no | sudo yum reinstall kernel || true 2023-10-09T06:16:34.793 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2023-10-09T06:16:34.793 INFO:teuthology.orchestra.run.smithi181.stdout:Unable to read consumer identity 2023-10-09T06:16:35.086 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:00:02 ago on Mon 09 Oct 2023 06:16:33 AM UTC. 2023-10-09T06:16:35.126 INFO:teuthology.orchestra.run.smithi163.stdout:Updating Subscription Management repositories. 2023-10-09T06:16:35.127 INFO:teuthology.orchestra.run.smithi163.stdout:Unable to read consumer identity 2023-10-09T06:16:35.210 INFO:teuthology.orchestra.run.smithi181.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-09T06:16:35.210 INFO:teuthology.orchestra.run.smithi181.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-09T06:16:35.210 INFO:teuthology.orchestra.run.smithi181.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-09T06:16:35.234 INFO:teuthology.orchestra.run.smithi181.stderr:Error: No packages marked for reinstall. 2023-10-09T06:16:35.284 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum reinstall -y kernel || true 2023-10-09T06:16:35.421 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:00:02 ago on Mon 09 Oct 2023 06:16:33 AM UTC. 2023-10-09T06:16:35.545 INFO:teuthology.orchestra.run.smithi163.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-09T06:16:35.546 INFO:teuthology.orchestra.run.smithi163.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-09T06:16:35.546 INFO:teuthology.orchestra.run.smithi163.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-09T06:16:35.586 INFO:teuthology.orchestra.run.smithi163.stderr:Error: No packages marked for reinstall. 2023-10-09T06:16:35.627 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2023-10-09T06:16:35.627 INFO:teuthology.orchestra.run.smithi181.stdout:Unable to read consumer identity 2023-10-09T06:16:35.642 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum reinstall -y kernel || true 2023-10-09T06:16:35.914 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:00:02 ago on Mon 09 Oct 2023 06:16:33 AM UTC. 2023-10-09T06:16:35.984 INFO:teuthology.orchestra.run.smithi163.stdout:Updating Subscription Management repositories. 2023-10-09T06:16:35.984 INFO:teuthology.orchestra.run.smithi163.stdout:Unable to read consumer identity 2023-10-09T06:16:36.068 INFO:teuthology.orchestra.run.smithi181.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-09T06:16:36.068 INFO:teuthology.orchestra.run.smithi181.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-09T06:16:36.068 INFO:teuthology.orchestra.run.smithi181.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-09T06:16:36.082 INFO:teuthology.orchestra.run.smithi181.stderr:Error: No packages marked for reinstall. 2023-10-09T06:16:36.134 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q kernel | sort -rV | head -n 1 2023-10-09T06:16:36.258 INFO:teuthology.orchestra.run.smithi181.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-09T06:16:36.258 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-09T06:16:36.259 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-10-09T06:16:36.259 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-09T06:16:36.259 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-09T06:16:36.259 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2023-10-09T06:16:36.259 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi181.front.sepia.ceph.com, path=None, version=distro) 2023-10-09T06:16:36.259 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q kernel | sort -rV | head -n 1 2023-10-09T06:16:36.274 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:00:03 ago on Mon 09 Oct 2023 06:16:33 AM UTC. 2023-10-09T06:16:36.383 INFO:teuthology.orchestra.run.smithi181.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-09T06:16:36.384 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-09T06:16:36.384 DEBUG:teuthology.orchestra.run.smithi181:> sudo rpm -qi grub2-tools 2023-10-09T06:16:36.410 INFO:teuthology.orchestra.run.smithi181.stdout:Name : grub2-tools 2023-10-09T06:16:36.410 INFO:teuthology.orchestra.run.smithi181.stdout:Epoch : 1 2023-10-09T06:16:36.410 INFO:teuthology.orchestra.run.smithi181.stdout:Version : 2.02 2023-10-09T06:16:36.411 INFO:teuthology.orchestra.run.smithi181.stdout:Release : 99.el8_4.1 2023-10-09T06:16:36.411 INFO:teuthology.orchestra.run.smithi181.stdout:Architecture: x86_64 2023-10-09T06:16:36.411 INFO:teuthology.orchestra.run.smithi181.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-09T06:16:36.411 INFO:teuthology.orchestra.run.smithi181.stdout:Group : System Environment/Base 2023-10-09T06:16:36.411 INFO:teuthology.orchestra.run.smithi181.stdout:Size : 9264631 2023-10-09T06:16:36.411 INFO:teuthology.orchestra.run.smithi181.stdout:License : GPLv3+ 2023-10-09T06:16:36.411 INFO:teuthology.orchestra.run.smithi181.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-09T06:16:36.411 INFO:teuthology.orchestra.run.smithi181.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-09T06:16:36.411 INFO:teuthology.orchestra.run.smithi181.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-09T06:16:36.411 INFO:teuthology.orchestra.run.smithi181.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-09T06:16:36.411 INFO:teuthology.orchestra.run.smithi181.stdout:Relocations : (not relocatable) 2023-10-09T06:16:36.411 INFO:teuthology.orchestra.run.smithi181.stdout:Packager : Red Hat, Inc. 2023-10-09T06:16:36.412 INFO:teuthology.orchestra.run.smithi181.stdout:Vendor : Red Hat, Inc. 2023-10-09T06:16:36.412 INFO:teuthology.orchestra.run.smithi181.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-09T06:16:36.412 INFO:teuthology.orchestra.run.smithi181.stdout:Summary : Support tools for GRUB. 2023-10-09T06:16:36.412 INFO:teuthology.orchestra.run.smithi181.stdout:Description : 2023-10-09T06:16:36.412 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:16:36.412 INFO:teuthology.orchestra.run.smithi181.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-09T06:16:36.412 INFO:teuthology.orchestra.run.smithi181.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-09T06:16:36.412 INFO:teuthology.orchestra.run.smithi181.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-09T06:16:36.412 INFO:teuthology.orchestra.run.smithi181.stdout:hardware devices. 2023-10-09T06:16:36.412 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:16:36.412 INFO:teuthology.orchestra.run.smithi181.stdout:This subpackage provides tools for support of all platforms. 2023-10-09T06:16:36.413 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-09T06:16:36.413 INFO:teuthology.task.kernel:Updating grub on smithi181 to boot 4.18.0-425.3.1.el8.x86_64 2023-10-09T06:16:36.413 DEBUG:teuthology.orchestra.run.smithi181:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-09T06:16:36.419 INFO:teuthology.orchestra.run.smithi163.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-09T06:16:36.419 INFO:teuthology.orchestra.run.smithi163.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-09T06:16:36.419 INFO:teuthology.orchestra.run.smithi163.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-09T06:16:36.436 INFO:teuthology.orchestra.run.smithi163.stderr:Error: No packages marked for reinstall. 2023-10-09T06:16:36.486 DEBUG:teuthology.orchestra.run.smithi163:> rpm -q kernel | sort -rV | head -n 1 2023-10-09T06:16:36.611 INFO:teuthology.orchestra.run.smithi163.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-09T06:16:36.611 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-09T06:16:36.611 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-10-09T06:16:36.611 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-09T06:16:36.612 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-09T06:16:36.612 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2023-10-09T06:16:36.612 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi163.front.sepia.ceph.com, path=None, version=distro) 2023-10-09T06:16:36.612 DEBUG:teuthology.orchestra.run.smithi163:> rpm -q kernel | sort -rV | head -n 1 2023-10-09T06:16:36.737 INFO:teuthology.orchestra.run.smithi163.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-09T06:16:36.737 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-09T06:16:36.738 DEBUG:teuthology.orchestra.run.smithi163:> sudo rpm -qi grub2-tools 2023-10-09T06:16:36.765 INFO:teuthology.orchestra.run.smithi163.stdout:Name : grub2-tools 2023-10-09T06:16:36.765 INFO:teuthology.orchestra.run.smithi163.stdout:Epoch : 1 2023-10-09T06:16:36.765 INFO:teuthology.orchestra.run.smithi163.stdout:Version : 2.02 2023-10-09T06:16:36.765 INFO:teuthology.orchestra.run.smithi163.stdout:Release : 99.el8_4.1 2023-10-09T06:16:36.765 INFO:teuthology.orchestra.run.smithi163.stdout:Architecture: x86_64 2023-10-09T06:16:36.765 INFO:teuthology.orchestra.run.smithi163.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-09T06:16:36.766 INFO:teuthology.orchestra.run.smithi163.stdout:Group : System Environment/Base 2023-10-09T06:16:36.766 INFO:teuthology.orchestra.run.smithi163.stdout:Size : 9264631 2023-10-09T06:16:36.766 INFO:teuthology.orchestra.run.smithi163.stdout:License : GPLv3+ 2023-10-09T06:16:36.766 INFO:teuthology.orchestra.run.smithi163.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-09T06:16:36.766 INFO:teuthology.orchestra.run.smithi163.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-09T06:16:36.766 INFO:teuthology.orchestra.run.smithi163.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-09T06:16:36.766 INFO:teuthology.orchestra.run.smithi163.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-09T06:16:36.766 INFO:teuthology.orchestra.run.smithi163.stdout:Relocations : (not relocatable) 2023-10-09T06:16:36.766 INFO:teuthology.orchestra.run.smithi163.stdout:Packager : Red Hat, Inc. 2023-10-09T06:16:36.766 INFO:teuthology.orchestra.run.smithi163.stdout:Vendor : Red Hat, Inc. 2023-10-09T06:16:36.766 INFO:teuthology.orchestra.run.smithi163.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-09T06:16:36.766 INFO:teuthology.orchestra.run.smithi163.stdout:Summary : Support tools for GRUB. 2023-10-09T06:16:36.767 INFO:teuthology.orchestra.run.smithi163.stdout:Description : 2023-10-09T06:16:36.767 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:16:36.767 INFO:teuthology.orchestra.run.smithi163.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-09T06:16:36.767 INFO:teuthology.orchestra.run.smithi163.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-09T06:16:36.767 INFO:teuthology.orchestra.run.smithi163.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-09T06:16:36.767 INFO:teuthology.orchestra.run.smithi163.stdout:hardware devices. 2023-10-09T06:16:36.767 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:16:36.767 INFO:teuthology.orchestra.run.smithi163.stdout:This subpackage provides tools for support of all platforms. 2023-10-09T06:16:36.768 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-09T06:16:36.768 INFO:teuthology.task.kernel:Updating grub on smithi163 to boot 4.18.0-425.3.1.el8.x86_64 2023-10-09T06:16:36.768 DEBUG:teuthology.orchestra.run.smithi163:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-09T06:16:37.591 INFO:teuthology.orchestra.run.smithi181.stderr:Generating grub configuration file ... 2023-10-09T06:16:38.166 INFO:teuthology.orchestra.run.smithi163.stderr:Generating grub configuration file ... 2023-10-09T06:16:39.377 INFO:teuthology.orchestra.run.smithi181.stderr:done 2023-10-09T06:16:39.378 DEBUG:teuthology.orchestra.run.smithi181:> mktemp 2023-10-09T06:16:39.392 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.bzXVwP52A5 2023-10-09T06:16:39.392 DEBUG:teuthology.orchestra.run.smithi181:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.bzXVwP52A5 2023-10-09T06:16:39.463 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.bzXVwP52A5 2023-10-09T06:16:39.626 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.bzXVwP52A5 is 6KB 2023-10-09T06:16:39.674 DEBUG:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.bzXVwP52A5 2023-10-09T06:16:39.688 DEBUG:teuthology.orchestra.run.smithi181:> sudo /bin/ls /boot/loader/entries || true 2023-10-09T06:16:39.755 INFO:teuthology.orchestra.run.smithi181.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-09T06:16:39.755 INFO:teuthology.orchestra.run.smithi181.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-09T06:16:39.755 INFO:teuthology.orchestra.run.smithi181.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-09T06:16:39.755 INFO:teuthology.orchestra.run.smithi181.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-09T06:16:39.756 DEBUG:teuthology.orchestra.run.smithi181:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-10-09T06:16:39.917 DEBUG:teuthology.orchestra.run.smithi181:> sudo shutdown -r now 2023-10-09T06:16:39.991 INFO:teuthology.orchestra.run.smithi163.stderr:done 2023-10-09T06:16:39.993 DEBUG:teuthology.orchestra.run.smithi163:> mktemp 2023-10-09T06:16:40.007 INFO:teuthology.orchestra.run.smithi163.stdout:/tmp/tmp.3Kh9zHq8HP 2023-10-09T06:16:40.007 DEBUG:teuthology.orchestra.run.smithi163:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.3Kh9zHq8HP 2023-10-09T06:16:40.087 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 0666 /tmp/tmp.3Kh9zHq8HP 2023-10-09T06:16:40.244 DEBUG:teuthology.orchestra.remote:smithi163:/tmp/tmp.3Kh9zHq8HP is 6KB 2023-10-09T06:16:40.292 DEBUG:teuthology.orchestra.run.smithi163:> rm -fr /tmp/tmp.3Kh9zHq8HP 2023-10-09T06:16:40.305 DEBUG:teuthology.orchestra.run.smithi163:> sudo /bin/ls /boot/loader/entries || true 2023-10-09T06:16:40.376 INFO:teuthology.orchestra.run.smithi163.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-09T06:16:40.377 INFO:teuthology.orchestra.run.smithi163.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-09T06:16:40.377 INFO:teuthology.orchestra.run.smithi163.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-09T06:16:40.377 INFO:teuthology.orchestra.run.smithi163.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-09T06:16:40.378 DEBUG:teuthology.orchestra.run.smithi163:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-10-09T06:16:40.519 DEBUG:teuthology.orchestra.run.smithi163:> sudo shutdown -r now 2023-10-09T06:17:09.946 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-10-09T06:17:09.947 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-10-09T06:17:09.947 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-09T06:17:10.521 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-09T06:17:10.522 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2023-10-09T06:17:10.522 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-09T06:17:45.082 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.163 2023-10-09T06:17:54.089 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2023-10-09T06:17:54.090 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-09T06:17:57.146 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.163 2023-10-09T06:18:09.159 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2023-10-09T06:18:09.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-09T06:18:09.950 DEBUG:teuthology.orchestra.remote:timed out 2023-10-09T06:18:18.959 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-10-09T06:18:18.960 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-09T06:18:20.589 DEBUG:teuthology.orchestra.run.smithi181:> true 2023-10-09T06:18:20.872 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-10-09T06:18:20.872 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "4.18.0-425.3.1.el8.x86_64"... 2023-10-09T06:18:20.872 DEBUG:teuthology.orchestra.run.smithi181:> uname -r 2023-10-09T06:18:20.886 INFO:teuthology.orchestra.run.smithi181.stdout:4.18.0-425.3.1.el8.x86_64 2023-10-09T06:18:20.887 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-10-09T06:18:20.887 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-09T06:18:20.887 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-09T06:18:21.888 DEBUG:teuthology.parallel:result is None 2023-10-09T06:19:09.162 DEBUG:teuthology.orchestra.remote:timed out 2023-10-09T06:19:24.163 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2023-10-09T06:19:24.164 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-09T06:19:24.486 DEBUG:teuthology.orchestra.run.smithi163:> true 2023-10-09T06:19:24.728 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi163.front.sepia.ceph.com' 2023-10-09T06:19:24.728 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "4.18.0-425.3.1.el8.x86_64"... 2023-10-09T06:19:24.728 DEBUG:teuthology.orchestra.run.smithi163:> uname -r 2023-10-09T06:19:24.743 INFO:teuthology.orchestra.run.smithi163.stdout:4.18.0-425.3.1.el8.x86_64 2023-10-09T06:19:24.744 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-10-09T06:19:24.744 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-09T06:19:24.744 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-09T06:19:25.745 DEBUG:teuthology.parallel:result is None 2023-10-09T06:19:25.745 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-09T06:19:25.752 INFO:teuthology.task.internal:Creating test directory... 2023-10-09T06:19:25.752 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-09T06:19:25.755 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-09T06:19:25.774 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-09T06:19:25.780 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-09T06:19:25.833 INFO:teuthology.task.internal:Creating archive directory... 2023-10-09T06:19:25.833 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-09T06:19:25.836 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-09T06:19:25.869 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-09T06:19:25.875 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-09T06:19:25.875 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-09T06:19:25.906 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-09T06:19:25.994 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-09T06:19:26.011 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-09T06:19:26.013 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-09T06:19:26.028 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-09T06:19:26.029 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-09T06:19:26.036 INFO:teuthology.task.internal:Configuring sudo... 2023-10-09T06:19:26.036 DEBUG:teuthology.orchestra.run.smithi163:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-09T06:19:26.038 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-09T06:19:26.064 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-09T06:19:26.124 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-09T06:19:26.124 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-09T06:19:26.126 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-09T06:19:26.141 DEBUG:teuthology.orchestra.run.smithi163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-09T06:19:26.225 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-09T06:19:26.289 DEBUG:teuthology.orchestra.run.smithi163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-09T06:19:26.331 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-09T06:19:26.373 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-09T06:19:26.373 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-09T06:19:26.438 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-09T06:19:26.482 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-09T06:19:26.549 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-09T06:19:26.587 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-09T06:19:26.631 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-09T06:19:26.631 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-09T06:19:26.696 DEBUG:teuthology.orchestra.run.smithi163:> sudo service rsyslog restart 2023-10-09T06:19:26.698 DEBUG:teuthology.orchestra.run.smithi181:> sudo service rsyslog restart 2023-10-09T06:19:26.749 INFO:teuthology.orchestra.run.smithi163.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-09T06:19:26.795 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-09T06:19:27.160 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-09T06:19:27.168 INFO:teuthology.task.internal:Starting timer... 2023-10-09T06:19:27.168 INFO:teuthology.run_tasks:Running task pcp... 2023-10-09T06:19:27.178 INFO:teuthology.run_tasks:Running task selinux... 2023-10-09T06:19:27.185 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-10-09T06:19:27.186 DEBUG:teuthology.orchestra.run.smithi163:> sudo service auditd rotate 2023-10-09T06:19:27.273 INFO:teuthology.orchestra.run.smithi163.stdout:Rotating logs: [ OK ] 2023-10-09T06:19:27.275 DEBUG:teuthology.orchestra.run.smithi181:> sudo service auditd rotate 2023-10-09T06:19:27.339 INFO:teuthology.orchestra.run.smithi181.stdout:Rotating logs: [ OK ] 2023-10-09T06:19:27.341 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-09T06:19:27.341 DEBUG:teuthology.orchestra.run.smithi163:> /usr/sbin/getenforce 2023-10-09T06:19:27.367 INFO:teuthology.orchestra.run.smithi163.stdout:Permissive 2023-10-09T06:19:27.367 DEBUG:teuthology.orchestra.run.smithi181:> /usr/sbin/getenforce 2023-10-09T06:19:27.384 INFO:teuthology.orchestra.run.smithi181.stdout:Permissive 2023-10-09T06:19:27.384 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi163.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi181.front.sepia.ceph.com': 'permissive'} 2023-10-09T06:19:27.384 DEBUG:teuthology.orchestra.run.smithi163:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-09T06:19:27.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-09T06:19:27.432 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-09T06:19:27.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-09T06:19:27.458 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-09T06:19:27.458 DEBUG:teuthology.orchestra.run.smithi163:> sudo /usr/sbin/setenforce permissive 2023-10-09T06:19:27.498 DEBUG:teuthology.orchestra.run.smithi181:> sudo /usr/sbin/setenforce permissive 2023-10-09T06:19:27.523 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-09T06:19:27.532 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-09T06:19:27.536 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-09T06:19:27.655 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-09T06:19:27.671 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-09T06:19:27.672 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi163.front.sepia.ceph.com,smithi181.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-09T06:27:36.344 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi163.front.sepia.ceph.com'), Remote(name='ubuntu@smithi181.front.sepia.ceph.com')] 2023-10-09T06:27:36.374 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2023-10-09T06:27:36.375 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-09T06:27:36.446 DEBUG:teuthology.orchestra.run.smithi163:> true 2023-10-09T06:27:36.513 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi163.front.sepia.ceph.com' 2023-10-09T06:27:36.513 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-10-09T06:27:36.514 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-09T06:27:36.582 DEBUG:teuthology.orchestra.run.smithi181:> true 2023-10-09T06:27:36.650 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-10-09T06:27:36.650 INFO:teuthology.run_tasks:Running task clock... 2023-10-09T06:27:36.662 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-09T06:27:36.663 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-09T06:27:36.663 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-09T06:27:36.666 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-09T06:27:36.666 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-09T06:27:36.692 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-09T06:27:36.706 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-09T06:27:36.721 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-09T06:27:36.736 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-09T06:27:36.739 INFO:teuthology.orchestra.run.smithi163.stderr:sudo: ntpd: command not found 2023-10-09T06:27:36.749 INFO:teuthology.orchestra.run.smithi163.stdout:506 Cannot talk to daemon 2023-10-09T06:27:36.763 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-09T06:27:36.771 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: ntpd: command not found 2023-10-09T06:27:36.777 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-09T06:27:36.782 INFO:teuthology.orchestra.run.smithi181.stdout:506 Cannot talk to daemon 2023-10-09T06:27:36.796 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-09T06:27:36.809 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-09T06:27:36.859 INFO:teuthology.orchestra.run.smithi163.stderr:bash: ntpq: command not found 2023-10-09T06:27:36.861 INFO:teuthology.orchestra.run.smithi163.stdout:210 Number of sources = 4 2023-10-09T06:27:36.861 INFO:teuthology.orchestra.run.smithi163.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-09T06:27:36.862 INFO:teuthology.orchestra.run.smithi163.stdout:=============================================================================== 2023-10-09T06:27:36.862 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-09T06:27:36.862 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-09T06:27:36.862 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-09T06:27:36.862 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-09T06:27:36.871 INFO:teuthology.orchestra.run.smithi181.stderr:bash: ntpq: command not found 2023-10-09T06:27:36.874 INFO:teuthology.orchestra.run.smithi181.stdout:210 Number of sources = 4 2023-10-09T06:27:36.874 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-09T06:27:36.874 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2023-10-09T06:27:36.874 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-09T06:27:36.874 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-09T06:27:36.874 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-09T06:27:36.874 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-09T06:27:36.875 INFO:teuthology.run_tasks:Running task install... 2023-10-09T06:27:36.885 DEBUG:teuthology.task.install:project ceph 2023-10-09T06:27:36.885 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}} 2023-10-09T06:27:36.885 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'} 2023-10-09T06:27:36.886 INFO:teuthology.task.install:Using flavor: default 2023-10-09T06:27:36.893 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-09T06:27:36.893 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-09T06:27:36.893 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647', 'tag': None, 'wait_for_package': False} 2023-10-09T06:27:36.894 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-09T06:27:36.896 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647', 'tag': None, 'wait_for_package': False} 2023-10-09T06:27:36.896 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-09T06:27:37.118 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/ 2023-10-09T06:27:37.119 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1320.g08cac018 2023-10-09T06:27:37.123 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/ 2023-10-09T06:27:37.124 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1320.g08cac018 2023-10-09T06:27:37.236 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-09T06:27:37.237 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-09T06:27:37.237 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-09T06:27:37.248 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-09T06:27:37.248 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-09T06:27:37.248 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-09T06:27:37.262 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-09T06:27:37.262 DEBUG:teuthology.orchestra.run.smithi181:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/08cac0187919fc3a247efdf445f652e2fb30a647/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-09T06:27:37.274 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-09T06:27:37.274 DEBUG:teuthology.orchestra.run.smithi163:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/08cac0187919fc3a247efdf445f652e2fb30a647/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-09T06:27:37.328 DEBUG:teuthology.orchestra.run.smithi181:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-09T06:27:37.340 DEBUG:teuthology.orchestra.run.smithi163:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-09T06:27:37.402 DEBUG:teuthology.orchestra.run.smithi181:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-09T06:27:37.414 DEBUG:teuthology.orchestra.run.smithi163:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-09T06:27:37.466 INFO:teuthology.orchestra.run.smithi181.stdout:check_obsoletes = 1 2023-10-09T06:27:37.467 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum clean all 2023-10-09T06:27:37.477 INFO:teuthology.orchestra.run.smithi163.stdout:check_obsoletes = 1 2023-10-09T06:27:37.479 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum clean all 2023-10-09T06:27:37.855 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2023-10-09T06:27:37.871 INFO:teuthology.orchestra.run.smithi163.stdout:Updating Subscription Management repositories. 2023-10-09T06:27:38.157 INFO:teuthology.orchestra.run.smithi181.stdout:44 files removed 2023-10-09T06:27:38.165 INFO:teuthology.orchestra.run.smithi163.stdout:44 files removed 2023-10-09T06:27:38.208 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-09T06:27:38.215 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-09T06:27:38.558 INFO:teuthology.orchestra.run.smithi181.stdout:Updating Subscription Management repositories. 2023-10-09T06:27:38.564 INFO:teuthology.orchestra.run.smithi163.stdout:Updating Subscription Management repositories. 2023-10-09T06:27:39.384 INFO:teuthology.orchestra.run.smithi181.stdout:ceph packages for x86_64 209 kB/s | 78 kB 00:00 2023-10-09T06:27:39.393 INFO:teuthology.orchestra.run.smithi163.stdout:ceph packages for x86_64 210 kB/s | 78 kB 00:00 2023-10-09T06:27:39.674 INFO:teuthology.orchestra.run.smithi163.stdout:ceph noarch packages 38 kB/s | 9.2 kB 00:00 2023-10-09T06:27:39.724 INFO:teuthology.orchestra.run.smithi181.stdout:ceph noarch packages 31 kB/s | 9.2 kB 00:00 2023-10-09T06:27:40.009 INFO:teuthology.orchestra.run.smithi163.stdout:ceph source packages 4.8 kB/s | 1.5 kB 00:00 2023-10-09T06:27:40.056 INFO:teuthology.orchestra.run.smithi181.stdout:ceph source packages 4.8 kB/s | 1.5 kB 00:00 2023-10-09T06:27:40.161 INFO:teuthology.orchestra.run.smithi163.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-10-09T06:27:40.162 INFO:teuthology.orchestra.run.smithi181.stdout:Copr repo for python3-asyncssh owned by ceph 45 kB/s | 3.5 kB 00:00 2023-10-09T06:27:41.132 INFO:teuthology.orchestra.run.smithi181.stdout:Extra Packages for Enterprise Linux 17 MB/s | 16 MB 00:00 2023-10-09T06:27:41.181 INFO:teuthology.orchestra.run.smithi163.stdout:Extra Packages for Enterprise Linux 16 MB/s | 16 MB 00:00 2023-10-09T06:27:44.439 INFO:teuthology.orchestra.run.smithi181.stdout:lab-extras 4.6 MB/s | 24 kB 00:00 2023-10-09T06:27:44.451 INFO:teuthology.orchestra.run.smithi163.stdout:lab-extras 4.4 MB/s | 24 kB 00:00 2023-10-09T06:27:44.824 INFO:teuthology.orchestra.run.smithi163.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 113 MB/s | 33 MB 00:00 2023-10-09T06:27:44.826 INFO:teuthology.orchestra.run.smithi181.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 125 MB/s | 38 MB 00:00 2023-10-09T06:27:49.953 INFO:teuthology.orchestra.run.smithi163.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 122 MB/s | 38 MB 00:00 2023-10-09T06:27:50.327 INFO:teuthology.orchestra.run.smithi181.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 123 MB/s | 33 MB 00:00 2023-10-09T06:27:58.193 INFO:teuthology.orchestra.run.smithi163.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-09T06:27:58.193 INFO:teuthology.orchestra.run.smithi163.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-09T06:27:58.204 INFO:teuthology.orchestra.run.smithi163.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-09T06:27:58.205 INFO:teuthology.orchestra.run.smithi163.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-09T06:27:58.205 INFO:teuthology.orchestra.run.smithi163.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-09T06:27:58.206 INFO:teuthology.orchestra.run.smithi163.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-09T06:27:58.208 INFO:teuthology.orchestra.run.smithi163.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-09T06:27:58.210 INFO:teuthology.orchestra.run.smithi163.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-09T06:27:58.211 INFO:teuthology.orchestra.run.smithi163.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-09T06:27:58.483 INFO:teuthology.orchestra.run.smithi181.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-09T06:27:58.483 INFO:teuthology.orchestra.run.smithi181.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-09T06:27:58.494 INFO:teuthology.orchestra.run.smithi181.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-09T06:27:58.495 INFO:teuthology.orchestra.run.smithi181.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-09T06:27:58.496 INFO:teuthology.orchestra.run.smithi181.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-09T06:27:58.497 INFO:teuthology.orchestra.run.smithi181.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-09T06:27:58.499 INFO:teuthology.orchestra.run.smithi181.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-09T06:27:58.501 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-09T06:27:58.502 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-09T06:27:58.704 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2023-10-09T06:27:58.715 INFO:teuthology.orchestra.run.smithi163.stdout:====================================================================================================================== 2023-10-09T06:27:58.715 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2023-10-09T06:27:58.715 INFO:teuthology.orchestra.run.smithi163.stdout:====================================================================================================================== 2023-10-09T06:27:58.715 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2023-10-09T06:27:58.716 INFO:teuthology.orchestra.run.smithi163.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-09T06:27:58.716 INFO:teuthology.orchestra.run.smithi163.stdout: ceph x86_64 2:17.2.6-1320.g08cac018.el8 ceph 6.4 k 2023-10-09T06:27:58.716 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-base x86_64 2:17.2.6-1320.g08cac018.el8 ceph 5.2 M 2023-10-09T06:27:58.716 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-fuse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 876 k 2023-10-09T06:27:58.716 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1320.g08cac018.el8 ceph 162 k 2023-10-09T06:27:58.716 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr x86_64 2:17.2.6-1320.g08cac018.el8 ceph 1.6 M 2023-10-09T06:27:58.716 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 121 k 2023-10-09T06:27:58.716 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 1.7 M 2023-10-09T06:27:58.716 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 7.4 M 2023-10-09T06:27:58.716 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-rook noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 47 k 2023-10-09T06:27:58.716 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-radosgw x86_64 2:17.2.6-1320.g08cac018.el8 ceph 17 M 2023-10-09T06:27:58.717 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-test x86_64 2:17.2.6-1320.g08cac018.el8 ceph 52 M 2023-10-09T06:27:58.717 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-volume noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 267 k 2023-10-09T06:27:58.717 INFO:teuthology.orchestra.run.smithi163.stdout: cephadm noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 80 k 2023-10-09T06:27:58.717 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-mirror x86_64 2:17.2.6-1320.g08cac018.el8 ceph 255 k 2023-10-09T06:27:58.717 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-top noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 18 k 2023-10-09T06:27:58.717 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-09T06:27:58.717 INFO:teuthology.orchestra.run.smithi163.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-09T06:27:58.717 INFO:teuthology.orchestra.run.smithi163.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-09T06:27:58.717 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs-devel x86_64 2:17.2.6-1320.g08cac018.el8 ceph 25 k 2023-10-09T06:27:58.717 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 797 k 2023-10-09T06:27:58.717 INFO:teuthology.orchestra.run.smithi163.stdout: librados-devel x86_64 2:17.2.6-1320.g08cac018.el8 ceph 130 k 2023-10-09T06:27:58.718 INFO:teuthology.orchestra.run.smithi163.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-09T06:27:58.718 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-09T06:27:58.718 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-09T06:27:58.718 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cephfs x86_64 2:17.2.6-1320.g08cac018.el8 ceph 190 k 2023-10-09T06:27:58.718 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rados x86_64 2:17.2.6-1320.g08cac018.el8 ceph 371 k 2023-10-09T06:27:58.718 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rbd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 358 k 2023-10-09T06:27:58.718 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rgw x86_64 2:17.2.6-1320.g08cac018.el8 ceph 107 k 2023-10-09T06:27:58.718 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-fuse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 90 k 2023-10-09T06:27:58.718 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-mirror x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.5 M 2023-10-09T06:27:58.718 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-nbd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 173 k 2023-10-09T06:27:58.718 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-09T06:27:58.719 INFO:teuthology.orchestra.run.smithi163.stdout:Upgrading: 2023-10-09T06:27:58.719 INFO:teuthology.orchestra.run.smithi163.stdout: librados2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.5 M 2023-10-09T06:27:58.719 INFO:teuthology.orchestra.run.smithi163.stdout: librbd1 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.6 M 2023-10-09T06:27:58.719 INFO:teuthology.orchestra.run.smithi163.stdout:Installing dependencies: 2023-10-09T06:27:58.719 INFO:teuthology.orchestra.run.smithi163.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-09T06:27:58.719 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-common x86_64 2:17.2.6-1320.g08cac018.el8 ceph 23 M 2023-10-09T06:27:58.719 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 23 k 2023-10-09T06:27:58.719 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mds x86_64 2:17.2.6-1320.g08cac018.el8 ceph 2.3 M 2023-10-09T06:27:58.719 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 238 k 2023-10-09T06:27:58.719 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mon x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.9 M 2023-10-09T06:27:58.719 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-osd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 17 M 2023-10-09T06:27:58.720 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 14 k 2023-10-09T06:27:58.720 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-selinux x86_64 2:17.2.6-1320.g08cac018.el8 ceph 24 k 2023-10-09T06:27:58.720 INFO:teuthology.orchestra.run.smithi163.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-09T06:27:58.720 INFO:teuthology.orchestra.run.smithi163.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-09T06:27:58.720 INFO:teuthology.orchestra.run.smithi163.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-09T06:27:58.720 INFO:teuthology.orchestra.run.smithi163.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-09T06:27:58.720 INFO:teuthology.orchestra.run.smithi163.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-09T06:27:58.720 INFO:teuthology.orchestra.run.smithi163.stdout: libcephsqlite x86_64 2:17.2.6-1320.g08cac018.el8 ceph 174 k 2023-10-09T06:27:58.720 INFO:teuthology.orchestra.run.smithi163.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-09T06:27:58.720 INFO:teuthology.orchestra.run.smithi163.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-09T06:27:58.720 INFO:teuthology.orchestra.run.smithi163.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-09T06:27:58.720 INFO:teuthology.orchestra.run.smithi163.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-09T06:27:58.721 INFO:teuthology.orchestra.run.smithi163.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-09T06:27:58.721 INFO:teuthology.orchestra.run.smithi163.stdout: libradosstriper1 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 507 k 2023-10-09T06:27:58.721 INFO:teuthology.orchestra.run.smithi163.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-09T06:27:58.721 INFO:teuthology.orchestra.run.smithi163.stdout: librgw2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 5.6 M 2023-10-09T06:27:58.721 INFO:teuthology.orchestra.run.smithi163.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-09T06:27:58.721 INFO:teuthology.orchestra.run.smithi163.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-09T06:27:58.721 INFO:teuthology.orchestra.run.smithi163.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-09T06:27:58.721 INFO:teuthology.orchestra.run.smithi163.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-09T06:27:58.721 INFO:teuthology.orchestra.run.smithi163.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-09T06:27:58.721 INFO:teuthology.orchestra.run.smithi163.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-09T06:27:58.722 INFO:teuthology.orchestra.run.smithi163.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-09T06:27:58.722 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-09T06:27:58.722 INFO:teuthology.orchestra.run.smithi163.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-09T06:27:58.722 INFO:teuthology.orchestra.run.smithi163.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-09T06:27:58.722 INFO:teuthology.orchestra.run.smithi163.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-09T06:27:58.722 INFO:teuthology.orchestra.run.smithi163.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-09T06:27:58.722 INFO:teuthology.orchestra.run.smithi163.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-09T06:27:58.722 INFO:teuthology.orchestra.run.smithi163.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-09T06:27:58.722 INFO:teuthology.orchestra.run.smithi163.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-09T06:27:58.722 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-09T06:27:58.722 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-argparse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 45 k 2023-10-09T06:27:58.723 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-common x86_64 2:17.2.6-1320.g08cac018.el8 ceph 115 k 2023-10-09T06:27:58.723 INFO:teuthology.orchestra.run.smithi163.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-09T06:27:58.723 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-09T06:27:58.723 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-09T06:27:58.723 INFO:teuthology.orchestra.run.smithi163.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-09T06:27:58.723 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-09T06:27:58.723 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-09T06:27:58.723 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-09T06:27:58.723 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-09T06:27:58.723 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-09T06:27:58.723 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-09T06:27:58.724 INFO:teuthology.orchestra.run.smithi163.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-09T06:27:58.724 INFO:teuthology.orchestra.run.smithi163.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-09T06:27:58.724 INFO:teuthology.orchestra.run.smithi163.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-09T06:27:58.724 INFO:teuthology.orchestra.run.smithi163.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-09T06:27:58.724 INFO:teuthology.orchestra.run.smithi163.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-09T06:27:58.724 INFO:teuthology.orchestra.run.smithi163.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-09T06:27:58.724 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-09T06:27:58.724 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-09T06:27:58.724 INFO:teuthology.orchestra.run.smithi163.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-09T06:27:58.724 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-09T06:27:58.724 INFO:teuthology.orchestra.run.smithi163.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-09T06:27:58.725 INFO:teuthology.orchestra.run.smithi163.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-09T06:27:58.725 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-09T06:27:58.725 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-09T06:27:58.725 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-09T06:27:58.725 INFO:teuthology.orchestra.run.smithi163.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-09T06:27:58.725 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-09T06:27:58.725 INFO:teuthology.orchestra.run.smithi163.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-09T06:27:58.725 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-09T06:27:58.725 INFO:teuthology.orchestra.run.smithi163.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-09T06:27:58.725 INFO:teuthology.orchestra.run.smithi163.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-09T06:27:58.725 INFO:teuthology.orchestra.run.smithi163.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-09T06:27:58.725 INFO:teuthology.orchestra.run.smithi163.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-09T06:27:58.726 INFO:teuthology.orchestra.run.smithi163.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-09T06:27:58.726 INFO:teuthology.orchestra.run.smithi163.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-09T06:27:58.726 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-09T06:27:58.726 INFO:teuthology.orchestra.run.smithi163.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-09T06:27:58.726 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-09T06:27:58.726 INFO:teuthology.orchestra.run.smithi163.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-09T06:27:58.726 INFO:teuthology.orchestra.run.smithi163.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-09T06:27:58.726 INFO:teuthology.orchestra.run.smithi163.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-09T06:27:58.726 INFO:teuthology.orchestra.run.smithi163.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-09T06:27:58.726 INFO:teuthology.orchestra.run.smithi163.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-09T06:27:58.726 INFO:teuthology.orchestra.run.smithi163.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-09T06:27:58.726 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:27:58.727 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2023-10-09T06:27:58.727 INFO:teuthology.orchestra.run.smithi163.stdout:====================================================================================================================== 2023-10-09T06:27:58.727 INFO:teuthology.orchestra.run.smithi163.stdout:Install 118 Packages 2023-10-09T06:27:58.727 INFO:teuthology.orchestra.run.smithi163.stdout:Upgrade 2 Packages 2023-10-09T06:27:58.727 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:27:58.727 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 202 M 2023-10-09T06:27:58.727 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2023-10-09T06:27:58.911 INFO:teuthology.orchestra.run.smithi163.stdout:(1/120): ceph-17.2.6-1320.g08cac018.el8.x86_64. 43 kB/s | 6.4 kB 00:00 2023-10-09T06:27:58.987 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-10-09T06:27:58.998 INFO:teuthology.orchestra.run.smithi181.stdout:====================================================================================================================== 2023-10-09T06:27:58.999 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2023-10-09T06:27:58.999 INFO:teuthology.orchestra.run.smithi181.stdout:====================================================================================================================== 2023-10-09T06:27:58.999 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2023-10-09T06:27:58.999 INFO:teuthology.orchestra.run.smithi181.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-09T06:27:58.999 INFO:teuthology.orchestra.run.smithi181.stdout: ceph x86_64 2:17.2.6-1320.g08cac018.el8 ceph 6.4 k 2023-10-09T06:27:58.999 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base x86_64 2:17.2.6-1320.g08cac018.el8 ceph 5.2 M 2023-10-09T06:27:58.999 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 876 k 2023-10-09T06:27:58.999 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1320.g08cac018.el8 ceph 162 k 2023-10-09T06:27:58.999 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr x86_64 2:17.2.6-1320.g08cac018.el8 ceph 1.6 M 2023-10-09T06:27:58.999 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 121 k 2023-10-09T06:27:58.999 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 1.7 M 2023-10-09T06:27:59.000 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 7.4 M 2023-10-09T06:27:59.000 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 47 k 2023-10-09T06:27:59.000 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw x86_64 2:17.2.6-1320.g08cac018.el8 ceph 17 M 2023-10-09T06:27:59.000 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test x86_64 2:17.2.6-1320.g08cac018.el8 ceph 52 M 2023-10-09T06:27:59.000 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 267 k 2023-10-09T06:27:59.000 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 80 k 2023-10-09T06:27:59.000 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror x86_64 2:17.2.6-1320.g08cac018.el8 ceph 255 k 2023-10-09T06:27:59.000 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 18 k 2023-10-09T06:27:59.000 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-09T06:27:59.000 INFO:teuthology.orchestra.run.smithi181.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-09T06:27:59.000 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-09T06:27:59.000 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel x86_64 2:17.2.6-1320.g08cac018.el8 ceph 25 k 2023-10-09T06:27:59.001 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 797 k 2023-10-09T06:27:59.001 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel x86_64 2:17.2.6-1320.g08cac018.el8 ceph 130 k 2023-10-09T06:27:59.001 INFO:teuthology.orchestra.run.smithi181.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-09T06:27:59.001 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-09T06:27:59.001 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-09T06:27:59.001 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs x86_64 2:17.2.6-1320.g08cac018.el8 ceph 190 k 2023-10-09T06:27:59.001 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados x86_64 2:17.2.6-1320.g08cac018.el8 ceph 371 k 2023-10-09T06:27:59.001 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 358 k 2023-10-09T06:27:59.001 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw x86_64 2:17.2.6-1320.g08cac018.el8 ceph 107 k 2023-10-09T06:27:59.001 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 90 k 2023-10-09T06:27:59.001 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.5 M 2023-10-09T06:27:59.001 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 173 k 2023-10-09T06:27:59.001 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-09T06:27:59.002 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrading: 2023-10-09T06:27:59.002 INFO:teuthology.orchestra.run.smithi181.stdout: librados2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.5 M 2023-10-09T06:27:59.002 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.6 M 2023-10-09T06:27:59.002 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2023-10-09T06:27:59.002 INFO:teuthology.orchestra.run.smithi181.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-09T06:27:59.002 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common x86_64 2:17.2.6-1320.g08cac018.el8 ceph 23 M 2023-10-09T06:27:59.002 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 23 k 2023-10-09T06:27:59.002 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds x86_64 2:17.2.6-1320.g08cac018.el8 ceph 2.3 M 2023-10-09T06:27:59.002 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 238 k 2023-10-09T06:27:59.002 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.9 M 2023-10-09T06:27:59.002 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 17 M 2023-10-09T06:27:59.002 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 14 k 2023-10-09T06:27:59.003 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux x86_64 2:17.2.6-1320.g08cac018.el8 ceph 24 k 2023-10-09T06:27:59.003 INFO:teuthology.orchestra.run.smithi181.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-09T06:27:59.003 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-09T06:27:59.003 INFO:teuthology.orchestra.run.smithi181.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-09T06:27:59.003 INFO:teuthology.orchestra.run.smithi181.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-09T06:27:59.003 INFO:teuthology.orchestra.run.smithi181.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-09T06:27:59.003 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite x86_64 2:17.2.6-1320.g08cac018.el8 ceph 174 k 2023-10-09T06:27:59.003 INFO:teuthology.orchestra.run.smithi181.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-09T06:27:59.003 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-09T06:27:59.003 INFO:teuthology.orchestra.run.smithi181.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-09T06:27:59.003 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-09T06:27:59.003 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-09T06:27:59.004 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 507 k 2023-10-09T06:27:59.004 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-09T06:27:59.004 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 5.6 M 2023-10-09T06:27:59.004 INFO:teuthology.orchestra.run.smithi181.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-09T06:27:59.004 INFO:teuthology.orchestra.run.smithi181.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-09T06:27:59.004 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-09T06:27:59.004 INFO:teuthology.orchestra.run.smithi181.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-09T06:27:59.004 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-09T06:27:59.004 INFO:teuthology.orchestra.run.smithi181.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-09T06:27:59.004 INFO:teuthology.orchestra.run.smithi181.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-09T06:27:59.004 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-09T06:27:59.004 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-09T06:27:59.005 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-09T06:27:59.005 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-09T06:27:59.005 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-09T06:27:59.005 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-09T06:27:59.005 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-09T06:27:59.005 INFO:teuthology.orchestra.run.smithi181.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-09T06:27:59.005 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-09T06:27:59.005 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 45 k 2023-10-09T06:27:59.005 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common x86_64 2:17.2.6-1320.g08cac018.el8 ceph 115 k 2023-10-09T06:27:59.005 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-09T06:27:59.005 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-09T06:27:59.005 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-09T06:27:59.006 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-09T06:27:59.006 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-09T06:27:59.006 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-09T06:27:59.006 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-09T06:27:59.006 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-09T06:27:59.006 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-09T06:27:59.006 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-09T06:27:59.006 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-09T06:27:59.006 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-09T06:27:59.006 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-09T06:27:59.006 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-09T06:27:59.006 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-09T06:27:59.007 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-09T06:27:59.007 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-09T06:27:59.007 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-09T06:27:59.007 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-09T06:27:59.007 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-09T06:27:59.007 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-09T06:27:59.007 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-09T06:27:59.007 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-09T06:27:59.007 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-09T06:27:59.007 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-09T06:27:59.007 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-09T06:27:59.007 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-09T06:27:59.008 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-09T06:27:59.008 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-09T06:27:59.008 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-09T06:27:59.008 INFO:teuthology.orchestra.run.smithi181.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-09T06:27:59.008 INFO:teuthology.orchestra.run.smithi181.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-09T06:27:59.008 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-09T06:27:59.008 INFO:teuthology.orchestra.run.smithi181.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-09T06:27:59.008 INFO:teuthology.orchestra.run.smithi181.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-09T06:27:59.008 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-09T06:27:59.008 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-09T06:27:59.008 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-09T06:27:59.008 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-09T06:27:59.009 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-09T06:27:59.009 INFO:teuthology.orchestra.run.smithi181.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-09T06:27:59.009 INFO:teuthology.orchestra.run.smithi181.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-09T06:27:59.009 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-09T06:27:59.009 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-09T06:27:59.009 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:27:59.009 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-10-09T06:27:59.009 INFO:teuthology.orchestra.run.smithi181.stdout:====================================================================================================================== 2023-10-09T06:27:59.009 INFO:teuthology.orchestra.run.smithi181.stdout:Install 118 Packages 2023-10-09T06:27:59.009 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrade 2 Packages 2023-10-09T06:27:59.009 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:27:59.010 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 202 M 2023-10-09T06:27:59.010 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2023-10-09T06:27:59.179 INFO:teuthology.orchestra.run.smithi163.stdout:(2/120): ceph-fuse-17.2.6-1320.g08cac018.el8.x8 3.2 MB/s | 876 kB 00:00 2023-10-09T06:27:59.228 INFO:teuthology.orchestra.run.smithi163.stdout:(3/120): ceph-immutable-object-cache-17.2.6-132 3.2 MB/s | 162 kB 00:00 2023-10-09T06:27:59.380 INFO:teuthology.orchestra.run.smithi181.stdout:(1/120): ceph-17.2.6-1320.g08cac018.el8.x86_64. 29 kB/s | 6.4 kB 00:00 2023-10-09T06:27:59.618 INFO:teuthology.orchestra.run.smithi163.stdout:(4/120): ceph-mds-17.2.6-1320.g08cac018.el8.x86 5.9 MB/s | 2.3 MB 00:00 2023-10-09T06:27:59.737 INFO:teuthology.orchestra.run.smithi163.stdout:(5/120): ceph-base-17.2.6-1320.g08cac018.el8.x8 5.3 MB/s | 5.2 MB 00:00 2023-10-09T06:27:59.930 INFO:teuthology.orchestra.run.smithi163.stdout:(6/120): ceph-mgr-17.2.6-1320.g08cac018.el8.x86 5.0 MB/s | 1.6 MB 00:00 2023-10-09T06:28:00.253 INFO:teuthology.orchestra.run.smithi163.stdout:(7/120): ceph-mon-17.2.6-1320.g08cac018.el8.x86 7.6 MB/s | 3.9 MB 00:00 2023-10-09T06:28:00.638 INFO:teuthology.orchestra.run.smithi181.stdout:(2/120): ceph-fuse-17.2.6-1320.g08cac018.el8.x8 697 kB/s | 876 kB 00:01 2023-10-09T06:28:00.837 INFO:teuthology.orchestra.run.smithi163.stdout:(8/120): ceph-common-17.2.6-1320.g08cac018.el8. 11 MB/s | 23 MB 00:02 2023-10-09T06:28:00.841 INFO:teuthology.orchestra.run.smithi181.stdout:(3/120): ceph-immutable-object-cache-17.2.6-132 796 kB/s | 162 kB 00:00 2023-10-09T06:28:00.883 INFO:teuthology.orchestra.run.smithi163.stdout:(9/120): ceph-selinux-17.2.6-1320.g08cac018.el8 538 kB/s | 24 kB 00:00 2023-10-09T06:28:01.703 INFO:teuthology.orchestra.run.smithi163.stdout:(10/120): ceph-radosgw-17.2.6-1320.g08cac018.el 12 MB/s | 17 MB 00:01 2023-10-09T06:28:01.768 INFO:teuthology.orchestra.run.smithi163.stdout:(11/120): cephfs-mirror-17.2.6-1320.g08cac018.e 3.9 MB/s | 255 kB 00:00 2023-10-09T06:28:01.819 INFO:teuthology.orchestra.run.smithi163.stdout:(12/120): libcephfs-devel-17.2.6-1320.g08cac018 494 kB/s | 25 kB 00:00 2023-10-09T06:28:01.898 INFO:teuthology.orchestra.run.smithi163.stdout:(13/120): libcephfs2-17.2.6-1320.g08cac018.el8. 9.8 MB/s | 797 kB 00:00 2023-10-09T06:28:01.954 INFO:teuthology.orchestra.run.smithi163.stdout:(14/120): libcephsqlite-17.2.6-1320.g08cac018.e 3.0 MB/s | 174 kB 00:00 2023-10-09T06:28:02.011 INFO:teuthology.orchestra.run.smithi163.stdout:(15/120): librados-devel-17.2.6-1320.g08cac018. 2.3 MB/s | 130 kB 00:00 2023-10-09T06:28:02.064 INFO:teuthology.orchestra.run.smithi181.stdout:(4/120): ceph-base-17.2.6-1320.g08cac018.el8.x8 1.8 MB/s | 5.2 MB 00:02 2023-10-09T06:28:02.078 INFO:teuthology.orchestra.run.smithi163.stdout:(16/120): libradosstriper1-17.2.6-1320.g08cac01 7.3 MB/s | 507 kB 00:00 2023-10-09T06:28:02.469 INFO:teuthology.orchestra.run.smithi181.stdout:(5/120): ceph-mgr-17.2.6-1320.g08cac018.el8.x86 3.9 MB/s | 1.6 MB 00:00 2023-10-09T06:28:02.810 INFO:teuthology.orchestra.run.smithi163.stdout:(17/120): librgw2-17.2.6-1320.g08cac018.el8.x86 11 MB/s | 5.6 MB 00:00 2023-10-09T06:28:02.811 INFO:teuthology.orchestra.run.smithi163.stdout:(18/120): ceph-osd-17.2.6-1320.g08cac018.el8.x8 6.0 MB/s | 17 MB 00:02 2023-10-09T06:28:02.811 INFO:teuthology.orchestra.run.smithi163.stdout:(19/120): python3-ceph-argparse-17.2.6-1320.g08 288 kB/s | 45 kB 00:00 2023-10-09T06:28:02.811 INFO:teuthology.orchestra.run.smithi163.stdout:(20/120): python3-ceph-common-17.2.6-1320.g08ca 2.6 MB/s | 115 kB 00:00 2023-10-09T06:28:02.811 INFO:teuthology.orchestra.run.smithi163.stdout:(21/120): python3-cephfs-17.2.6-1320.g08cac018. 3.6 MB/s | 190 kB 00:00 2023-10-09T06:28:02.843 INFO:teuthology.orchestra.run.smithi163.stdout:(22/120): python3-rados-17.2.6-1320.g08cac018.e 5.8 MB/s | 371 kB 00:00 2023-10-09T06:28:02.864 INFO:teuthology.orchestra.run.smithi163.stdout:(23/120): python3-rbd-17.2.6-1320.g08cac018.el8 4.7 MB/s | 358 kB 00:00 2023-10-09T06:28:02.905 INFO:teuthology.orchestra.run.smithi163.stdout:(24/120): python3-rgw-17.2.6-1320.g08cac018.el8 1.7 MB/s | 107 kB 00:00 2023-10-09T06:28:02.921 INFO:teuthology.orchestra.run.smithi163.stdout:(25/120): rbd-fuse-17.2.6-1320.g08cac018.el8.x8 1.6 MB/s | 90 kB 00:00 2023-10-09T06:28:02.954 INFO:teuthology.orchestra.run.smithi181.stdout:(6/120): ceph-mon-17.2.6-1320.g08cac018.el8.x86 8.1 MB/s | 3.9 MB 00:00 2023-10-09T06:28:02.982 INFO:teuthology.orchestra.run.smithi163.stdout:(26/120): rbd-nbd-17.2.6-1320.g08cac018.el8.x86 2.8 MB/s | 173 kB 00:00 2023-10-09T06:28:03.051 INFO:teuthology.orchestra.run.smithi163.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1320.g 330 kB/s | 23 kB 00:00 2023-10-09T06:28:03.118 INFO:teuthology.orchestra.run.smithi163.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1320.g08cac01 1.8 MB/s | 121 kB 00:00 2023-10-09T06:28:03.118 INFO:teuthology.orchestra.run.smithi181.stdout:(7/120): ceph-mds-17.2.6-1320.g08cac018.el8.x86 1.0 MB/s | 2.3 MB 00:02 2023-10-09T06:28:03.337 INFO:teuthology.orchestra.run.smithi163.stdout:(29/120): ceph-mgr-dashboard-17.2.6-1320.g08cac 7.7 MB/s | 1.7 MB 00:00 2023-10-09T06:28:03.595 INFO:teuthology.orchestra.run.smithi163.stdout:(30/120): rbd-mirror-17.2.6-1320.g08cac018.el8. 5.0 MB/s | 3.5 MB 00:00 2023-10-09T06:28:03.670 INFO:teuthology.orchestra.run.smithi163.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1320.g08 3.1 MB/s | 238 kB 00:00 2023-10-09T06:28:03.739 INFO:teuthology.orchestra.run.smithi163.stdout:(32/120): ceph-mgr-rook-17.2.6-1320.g08cac018.e 697 kB/s | 47 kB 00:00 2023-10-09T06:28:03.806 INFO:teuthology.orchestra.run.smithi163.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1320.g0 215 kB/s | 14 kB 00:00 2023-10-09T06:28:03.884 INFO:teuthology.orchestra.run.smithi163.stdout:(34/120): ceph-volume-17.2.6-1320.g08cac018.el8 3.3 MB/s | 267 kB 00:00 2023-10-09T06:28:03.955 INFO:teuthology.orchestra.run.smithi163.stdout:(35/120): cephadm-17.2.6-1320.g08cac018.el8.noa 1.1 MB/s | 80 kB 00:00 2023-10-09T06:28:04.024 INFO:teuthology.orchestra.run.smithi163.stdout:(36/120): cephfs-top-17.2.6-1320.g08cac018.el8. 263 kB/s | 18 kB 00:00 2023-10-09T06:28:04.045 INFO:teuthology.orchestra.run.smithi163.stdout:(37/120): ceph-mgr-diskprediction-local-17.2.6- 10 MB/s | 7.4 MB 00:00 2023-10-09T06:28:04.175 INFO:teuthology.orchestra.run.smithi163.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 452 kB/s | 59 kB 00:00 2023-10-09T06:28:04.235 INFO:teuthology.orchestra.run.smithi163.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 1.2 MB/s | 75 kB 00:00 2023-10-09T06:28:04.256 INFO:teuthology.orchestra.run.smithi163.stdout:(40/120): gperftools-libs-2.7-9.el8.x86_64.rpm 1.3 MB/s | 306 kB 00:00 2023-10-09T06:28:04.291 INFO:teuthology.orchestra.run.smithi163.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.2 MB/s | 44 kB 00:00 2023-10-09T06:28:04.336 INFO:teuthology.orchestra.run.smithi163.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 4.1 MB/s | 185 kB 00:00 2023-10-09T06:28:04.339 INFO:teuthology.orchestra.run.smithi163.stdout:(43/120): python3-asyncssh-2.7.0-2.el8.noarch.r 4.5 MB/s | 476 kB 00:00 2023-10-09T06:28:04.369 INFO:teuthology.orchestra.run.smithi163.stdout:(44/120): python3-cachetools-3.1.1-4.el8.noarch 992 kB/s | 33 kB 00:00 2023-10-09T06:28:04.369 INFO:teuthology.orchestra.run.smithi163.stdout:(45/120): python3-certifi-2018.10.15-7.el8.noar 542 kB/s | 16 kB 00:00 2023-10-09T06:28:04.412 INFO:teuthology.orchestra.run.smithi163.stdout:(46/120): python3-cheroot-8.5.2-1.el8.noarch.rp 3.9 MB/s | 173 kB 00:00 2023-10-09T06:28:04.414 INFO:teuthology.orchestra.run.smithi163.stdout:(47/120): python3-cherrypy-18.4.0-1.el8.noarch. 8.6 MB/s | 384 kB 00:00 2023-10-09T06:28:04.443 INFO:teuthology.orchestra.run.smithi163.stdout:(48/120): python3-jaraco-6.2-6.el8.noarch.rpm 375 kB/s | 11 kB 00:00 2023-10-09T06:28:04.448 INFO:teuthology.orchestra.run.smithi163.stdout:(49/120): python3-google-auth-1.1.1-10.el8.noar 2.2 MB/s | 81 kB 00:00 2023-10-09T06:28:04.473 INFO:teuthology.orchestra.run.smithi163.stdout:(50/120): python3-jaraco-functools-2.0-4.el8.no 589 kB/s | 18 kB 00:00 2023-10-09T06:28:04.504 INFO:teuthology.orchestra.run.smithi163.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 1.6 MB/s | 49 kB 00:00 2023-10-09T06:28:04.536 INFO:teuthology.orchestra.run.smithi163.stdout:(52/120): python3-more-itertools-7.2.0-3.el8.no 1.8 MB/s | 59 kB 00:00 2023-10-09T06:28:04.552 INFO:teuthology.orchestra.run.smithi163.stdout:(53/120): python3-kubernetes-11.0.0-6.el8.noarc 11 MB/s | 1.2 MB 00:00 2023-10-09T06:28:04.567 INFO:teuthology.orchestra.run.smithi163.stdout:(54/120): python3-natsort-7.1.1-2.el8.noarch.rp 1.9 MB/s | 60 kB 00:00 2023-10-09T06:28:04.588 INFO:teuthology.orchestra.run.smithi163.stdout:(55/120): python3-pecan-1.3.2-9.el8.noarch.rpm 7.6 MB/s | 283 kB 00:00 2023-10-09T06:28:04.597 INFO:teuthology.orchestra.run.smithi163.stdout:(56/120): python3-portend-2.6-1.el8.noarch.rpm 523 kB/s | 16 kB 00:00 2023-10-09T06:28:04.620 INFO:teuthology.orchestra.run.smithi163.stdout:(57/120): python3-repoze-lru-0.7-6.el8.noarch.r 1.0 MB/s | 34 kB 00:00 2023-10-09T06:28:04.634 INFO:teuthology.orchestra.run.smithi163.stdout:(58/120): python3-routes-2.4.1-12.el8.noarch.rp 5.2 MB/s | 196 kB 00:00 2023-10-09T06:28:04.652 INFO:teuthology.orchestra.run.smithi163.stdout:(59/120): python3-rsa-4.9-2.el8.noarch.rpm 2.0 MB/s | 64 kB 00:00 2023-10-09T06:28:04.665 INFO:teuthology.orchestra.run.smithi163.stdout:(60/120): python3-simplegeneric-0.8.1-17.el8.no 617 kB/s | 19 kB 00:00 2023-10-09T06:28:04.683 INFO:teuthology.orchestra.run.smithi163.stdout:(61/120): python3-singledispatch-3.4.0.3-18.el8 757 kB/s | 24 kB 00:00 2023-10-09T06:28:04.695 INFO:teuthology.orchestra.run.smithi163.stdout:(62/120): python3-tempora-1.14.1-5.el8.noarch.r 967 kB/s | 29 kB 00:00 2023-10-09T06:28:04.715 INFO:teuthology.orchestra.run.smithi163.stdout:(63/120): python3-trustme-0.6.0-4.el8.noarch.rp 879 kB/s | 27 kB 00:00 2023-10-09T06:28:04.734 INFO:teuthology.orchestra.run.smithi163.stdout:(64/120): python3-waitress-1.4.3-1.el8.noarch.r 6.5 MB/s | 255 kB 00:00 2023-10-09T06:28:04.750 INFO:teuthology.orchestra.run.smithi163.stdout:(65/120): python3-webob-1.8.5-1.el8.1.noarch.rp 7.0 MB/s | 251 kB 00:00 2023-10-09T06:28:04.765 INFO:teuthology.orchestra.run.smithi163.stdout:(66/120): python3-websocket-client-0.56.0-5.el8 1.9 MB/s | 61 kB 00:00 2023-10-09T06:28:04.783 INFO:teuthology.orchestra.run.smithi163.stdout:(67/120): python3-webtest-2.0.33-1.el8.noarch.r 2.6 MB/s | 85 kB 00:00 2023-10-09T06:28:04.795 INFO:teuthology.orchestra.run.smithi163.stdout:(68/120): python3-zc-lockfile-2.0-2.el8.noarch. 768 kB/s | 23 kB 00:00 2023-10-09T06:28:04.799 INFO:teuthology.orchestra.run.smithi163.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 17 MB/s | 70 kB 00:00 2023-10-09T06:28:04.847 INFO:teuthology.orchestra.run.smithi163.stdout:(70/120): thrift-0.13.0-2.el8.x86_64.rpm 27 MB/s | 1.7 MB 00:00 2023-10-09T06:28:04.866 INFO:teuthology.orchestra.run.smithi181.stdout:(8/120): ceph-osd-17.2.6-1320.g08cac018.el8.x86 8.7 MB/s | 17 MB 00:01 2023-10-09T06:28:04.879 INFO:teuthology.orchestra.run.smithi163.stdout:(71/120): python3-prettytable-0.7.2-14.el8.noar 1.4 MB/s | 44 kB 00:00 2023-10-09T06:28:04.894 INFO:teuthology.orchestra.run.smithi163.stdout:(72/120): python3-jsonpatch-1.21-2.el8.noarch.r 1.8 MB/s | 27 kB 00:00 2023-10-09T06:28:04.903 INFO:teuthology.orchestra.run.smithi181.stdout:(9/120): ceph-selinux-17.2.6-1320.g08cac018.el8 669 kB/s | 24 kB 00:00 2023-10-09T06:28:04.910 INFO:teuthology.orchestra.run.smithi163.stdout:(73/120): jq-1.5-12.el8.x86_64.rpm 10 MB/s | 161 kB 00:00 2023-10-09T06:28:04.928 INFO:teuthology.orchestra.run.smithi163.stdout:(74/120): flex-2.6.1-9.el8.x86_64.rpm 18 MB/s | 320 kB 00:00 2023-10-09T06:28:04.947 INFO:teuthology.orchestra.run.smithi163.stdout:(75/120): python3-werkzeug-0.12.2-4.el8.noarch. 23 MB/s | 457 kB 00:00 2023-10-09T06:28:05.102 INFO:teuthology.orchestra.run.smithi163.stdout:(76/120): python3-scipy-1.0.0-20.module+el8.1.0 48 MB/s | 15 MB 00:00 2023-10-09T06:28:05.115 INFO:teuthology.orchestra.run.smithi163.stdout:(77/120): python3-markupsafe-0.23-19.el8.x86_64 2.9 MB/s | 39 kB 00:00 2023-10-09T06:28:05.131 INFO:teuthology.orchestra.run.smithi163.stdout:(78/120): python3-babel-2.5.1-5.el8.noarch.rpm 26 MB/s | 4.8 MB 00:00 2023-10-09T06:28:05.140 INFO:teuthology.orchestra.run.smithi163.stdout:(79/120): bison-3.0.4-10.el8.x86_64.rpm 27 MB/s | 688 kB 00:00 2023-10-09T06:28:05.147 INFO:teuthology.orchestra.run.smithi163.stdout:(80/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 16 MB/s | 259 kB 00:00 2023-10-09T06:28:05.155 INFO:teuthology.orchestra.run.smithi163.stdout:(81/120): python3-numpy-f2py-1.14.3-9.el8.x86_6 14 MB/s | 225 kB 00:00 2023-10-09T06:28:05.161 INFO:teuthology.orchestra.run.smithi163.stdout:(82/120): python3-pyasn1-0.3.7-6.el8.noarch.rpm 8.7 MB/s | 126 kB 00:00 2023-10-09T06:28:05.176 INFO:teuthology.orchestra.run.smithi163.stdout:(83/120): oniguruma-6.8.2-2.el8.x86_64.rpm 13 MB/s | 187 kB 00:00 2023-10-09T06:28:05.196 INFO:teuthology.orchestra.run.smithi163.stdout:(84/120): python3-jinja2-2.10.1-2.el8_0.noarch. 26 MB/s | 538 kB 00:00 2023-10-09T06:28:05.217 INFO:teuthology.orchestra.run.smithi163.stdout:(85/120): python3-numpy-1.14.3-9.el8.x86_64.rpm 61 MB/s | 3.7 MB 00:00 2023-10-09T06:28:05.221 INFO:teuthology.orchestra.run.smithi163.stdout:(86/120): socat-1.7.3.3-2.el8.x86_64.rpm 12 MB/s | 302 kB 00:00 2023-10-09T06:28:05.231 INFO:teuthology.orchestra.run.smithi163.stdout:(87/120): python3-mako-1.0.6-13.el8.noarch.rpm 11 MB/s | 157 kB 00:00 2023-10-09T06:28:05.234 INFO:teuthology.orchestra.run.smithi163.stdout:(88/120): python3-pyasn1-modules-0.3.7-6.el8.no 7.9 MB/s | 110 kB 00:00 2023-10-09T06:28:05.248 INFO:teuthology.orchestra.run.smithi163.stdout:(89/120): librdkafka-0.11.4-1.el8.x86_64.rpm 21 MB/s | 353 kB 00:00 2023-10-09T06:28:05.248 INFO:teuthology.orchestra.run.smithi163.stdout:(90/120): python3-jsonpointer-1.10-11.el8.noarc 1.4 MB/s | 20 kB 00:00 2023-10-09T06:28:05.264 INFO:teuthology.orchestra.run.smithi163.stdout:(91/120): python3-pytz-2017.2-9.el8.noarch.rpm 3.5 MB/s | 54 kB 00:00 2023-10-09T06:28:05.271 INFO:teuthology.orchestra.run.smithi163.stdout:(92/120): libtool-2.4.6-25.el8.x86_64.rpm 30 MB/s | 709 kB 00:00 2023-10-09T06:28:05.344 INFO:teuthology.orchestra.run.smithi163.stdout:(93/120): openblas-threads-0.3.12-1.el8.x86_64. 59 MB/s | 4.7 MB 00:00 2023-10-09T06:28:05.362 INFO:teuthology.orchestra.run.smithi163.stdout:(94/120): openblas-0.3.12-1.el8.x86_64.rpm 51 MB/s | 4.6 MB 00:00 2023-10-09T06:28:05.371 INFO:teuthology.orchestra.run.smithi163.stdout:(95/120): automake-1.16.1-7.el8.noarch.rpm 27 MB/s | 713 kB 00:00 2023-10-09T06:28:05.375 INFO:teuthology.orchestra.run.smithi163.stdout:(96/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 1.5 MB/s | 19 kB 00:00 2023-10-09T06:28:05.384 INFO:teuthology.orchestra.run.smithi163.stdout:(97/120): python3-jwt-1.6.1-2.el8.noarch.rpm 3.2 MB/s | 43 kB 00:00 2023-10-09T06:28:05.389 INFO:teuthology.orchestra.run.smithi163.stdout:(98/120): python3-oauthlib-2.1.0-1.el8.noarch.r 11 MB/s | 155 kB 00:00 2023-10-09T06:28:05.398 INFO:teuthology.orchestra.run.smithi163.stdout:(99/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 15 MB/s | 220 kB 00:00 2023-10-09T06:28:05.401 INFO:teuthology.orchestra.run.smithi163.stdout:(100/120): keyutils-libs-devel-1.5.10-6.el8.x86 3.8 MB/s | 48 kB 00:00 2023-10-09T06:28:05.413 INFO:teuthology.orchestra.run.smithi163.stdout:(101/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 15 MB/s | 229 kB 00:00 2023-10-09T06:28:05.414 INFO:teuthology.orchestra.run.smithi163.stdout:(102/120): python3-requests-oauthlib-1.0.0-1.el 3.3 MB/s | 43 kB 00:00 2023-10-09T06:28:05.428 INFO:teuthology.orchestra.run.smithi163.stdout:(103/120): libverto-devel-0.3.0-5.el8.x86_64.rp 1.3 MB/s | 18 kB 00:00 2023-10-09T06:28:05.449 INFO:teuthology.orchestra.run.smithi163.stdout:(104/120): pcre2-devel-10.32-2.el8.x86_64.rpm 28 MB/s | 605 kB 00:00 2023-10-09T06:28:05.456 INFO:teuthology.orchestra.run.smithi163.stdout:(105/120): openssl-devel-1.1.1g-15.el8_3.x86_64 54 MB/s | 2.3 MB 00:00 2023-10-09T06:28:05.463 INFO:teuthology.orchestra.run.smithi163.stdout:(106/120): fuse-2.9.7-12.el8.x86_64.rpm 5.7 MB/s | 83 kB 00:00 2023-10-09T06:28:05.471 INFO:teuthology.orchestra.run.smithi163.stdout:(107/120): elfutils-libelf-devel-0.182-3.el8.x8 3.8 MB/s | 59 kB 00:00 2023-10-09T06:28:05.478 INFO:teuthology.orchestra.run.smithi163.stdout:(108/120): libquadmath-8.4.1-1.el8.x86_64.rpm 11 MB/s | 169 kB 00:00 2023-10-09T06:28:05.485 INFO:teuthology.orchestra.run.smithi163.stdout:(109/120): libcom_err-devel-1.45.6-1.el8.x86_64 2.9 MB/s | 38 kB 00:00 2023-10-09T06:28:05.492 INFO:teuthology.orchestra.run.smithi163.stdout:(110/120): libsepol-devel-2.9-2.el8.x86_64.rpm 6.1 MB/s | 86 kB 00:00 2023-10-09T06:28:05.499 INFO:teuthology.orchestra.run.smithi163.stdout:(111/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 4.1 MB/s | 58 kB 00:00 2023-10-09T06:28:05.506 INFO:teuthology.orchestra.run.smithi163.stdout:(112/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 3.3 MB/s | 47 kB 00:00 2023-10-09T06:28:05.518 INFO:teuthology.orchestra.run.smithi163.stdout:(113/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.r 28 MB/s | 559 kB 00:00 2023-10-09T06:28:05.526 INFO:teuthology.orchestra.run.smithi163.stdout:(114/120): libgfortran-8.4.1-1.el8.x86_64.rpm 33 MB/s | 639 kB 00:00 2023-10-09T06:28:05.532 INFO:teuthology.orchestra.run.smithi163.stdout:(115/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rp 5.1 MB/s | 70 kB 00:00 2023-10-09T06:28:05.542 INFO:teuthology.orchestra.run.smithi163.stdout:(116/120): libselinux-devel-2.9-5.el8.x86_64.rp 13 MB/s | 200 kB 00:00 2023-10-09T06:28:05.545 INFO:teuthology.orchestra.run.smithi163.stdout:(117/120): libuuid-devel-2.32.1-27.el8.x86_64.r 7.0 MB/s | 97 kB 00:00 2023-10-09T06:28:06.168 INFO:teuthology.orchestra.run.smithi163.stdout:(118/120): librbd1-17.2.6-1320.g08cac018.el8.x8 5.8 MB/s | 3.6 MB 00:00 2023-10-09T06:28:06.509 INFO:teuthology.orchestra.run.smithi163.stdout:(119/120): librados2-17.2.6-1320.g08cac018.el8. 3.6 MB/s | 3.5 MB 00:00 2023-10-09T06:28:07.086 INFO:teuthology.orchestra.run.smithi163.stdout:(120/120): ceph-test-17.2.6-1320.g08cac018.el8. 8.3 MB/s | 52 MB 00:06 2023-10-09T06:28:07.089 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2023-10-09T06:28:07.089 INFO:teuthology.orchestra.run.smithi163.stdout:Total 24 MB/s | 202 MB 00:08 2023-10-09T06:28:07.141 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2023-10-09T06:28:07.255 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2023-10-09T06:28:07.255 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2023-10-09T06:28:07.584 INFO:teuthology.orchestra.run.smithi181.stdout:(10/120): ceph-common-17.2.6-1320.g08cac018.el8 2.8 MB/s | 23 MB 00:08 2023-10-09T06:28:07.631 INFO:teuthology.orchestra.run.smithi181.stdout:(11/120): cephfs-mirror-17.2.6-1320.g08cac018.e 5.4 MB/s | 255 kB 00:00 2023-10-09T06:28:07.651 INFO:teuthology.orchestra.run.smithi181.stdout:(12/120): libcephfs-devel-17.2.6-1320.g08cac018 1.2 MB/s | 25 kB 00:00 2023-10-09T06:28:07.756 INFO:teuthology.orchestra.run.smithi181.stdout:(13/120): libcephfs2-17.2.6-1320.g08cac018.el8. 7.5 MB/s | 797 kB 00:00 2023-10-09T06:28:07.777 INFO:teuthology.orchestra.run.smithi181.stdout:(14/120): libcephsqlite-17.2.6-1320.g08cac018.e 7.9 MB/s | 174 kB 00:00 2023-10-09T06:28:07.799 INFO:teuthology.orchestra.run.smithi181.stdout:(15/120): librados-devel-17.2.6-1320.g08cac018. 5.9 MB/s | 130 kB 00:00 2023-10-09T06:28:07.863 INFO:teuthology.orchestra.run.smithi181.stdout:(16/120): libradosstriper1-17.2.6-1320.g08cac01 7.8 MB/s | 507 kB 00:00 2023-10-09T06:28:08.533 INFO:teuthology.orchestra.run.smithi181.stdout:(17/120): librgw2-17.2.6-1320.g08cac018.el8.x86 8.3 MB/s | 5.6 MB 00:00 2023-10-09T06:28:08.554 INFO:teuthology.orchestra.run.smithi181.stdout:(18/120): python3-ceph-argparse-17.2.6-1320.g08 2.1 MB/s | 45 kB 00:00 2023-10-09T06:28:08.978 INFO:teuthology.orchestra.run.smithi181.stdout:(19/120): ceph-test-17.2.6-1320.g08cac018.el8.x 13 MB/s | 52 MB 00:04 2023-10-09T06:28:08.979 INFO:teuthology.orchestra.run.smithi181.stdout:(20/120): python3-ceph-common-17.2.6-1320.g08ca 271 kB/s | 115 kB 00:00 2023-10-09T06:28:09.001 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2023-10-09T06:28:09.005 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2023-10-09T06:28:09.016 INFO:teuthology.orchestra.run.smithi181.stdout:(21/120): python3-cephfs-17.2.6-1320.g08cac018. 4.9 MB/s | 190 kB 00:00 2023-10-09T06:28:09.044 INFO:teuthology.orchestra.run.smithi181.stdout:(22/120): python3-rados-17.2.6-1320.g08cac018.e 5.7 MB/s | 371 kB 00:00 2023-10-09T06:28:09.059 INFO:teuthology.orchestra.run.smithi181.stdout:(23/120): python3-rbd-17.2.6-1320.g08cac018.el8 8.4 MB/s | 358 kB 00:00 2023-10-09T06:28:09.065 INFO:teuthology.orchestra.run.smithi181.stdout:(24/120): python3-rgw-17.2.6-1320.g08cac018.el8 4.9 MB/s | 107 kB 00:00 2023-10-09T06:28:09.095 INFO:teuthology.orchestra.run.smithi181.stdout:(25/120): rbd-fuse-17.2.6-1320.g08cac018.el8.x8 2.4 MB/s | 90 kB 00:00 2023-10-09T06:28:09.134 INFO:teuthology.orchestra.run.smithi181.stdout:(26/120): rbd-nbd-17.2.6-1320.g08cac018.el8.x86 4.5 MB/s | 173 kB 00:00 2023-10-09T06:28:09.170 INFO:teuthology.orchestra.run.smithi181.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1320.g 639 kB/s | 23 kB 00:00 2023-10-09T06:28:09.210 INFO:teuthology.orchestra.run.smithi181.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1320.g08cac01 3.0 MB/s | 121 kB 00:00 2023-10-09T06:28:09.346 INFO:teuthology.orchestra.run.smithi181.stdout:(29/120): ceph-mgr-dashboard-17.2.6-1320.g08cac 12 MB/s | 1.7 MB 00:00 2023-10-09T06:28:09.437 INFO:teuthology.orchestra.run.smithi181.stdout:(30/120): rbd-mirror-17.2.6-1320.g08cac018.el8. 9.3 MB/s | 3.5 MB 00:00 2023-10-09T06:28:09.479 INFO:teuthology.orchestra.run.smithi181.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1320.g08 5.6 MB/s | 238 kB 00:00 2023-10-09T06:28:09.500 INFO:teuthology.orchestra.run.smithi181.stdout:(32/120): ceph-mgr-rook-17.2.6-1320.g08cac018.e 2.2 MB/s | 47 kB 00:00 2023-10-09T06:28:09.521 INFO:teuthology.orchestra.run.smithi181.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1320.g0 704 kB/s | 14 kB 00:00 2023-10-09T06:28:09.563 INFO:teuthology.orchestra.run.smithi181.stdout:(34/120): ceph-volume-17.2.6-1320.g08cac018.el8 6.3 MB/s | 267 kB 00:00 2023-10-09T06:28:09.585 INFO:teuthology.orchestra.run.smithi181.stdout:(35/120): cephadm-17.2.6-1320.g08cac018.el8.noa 3.7 MB/s | 80 kB 00:00 2023-10-09T06:28:09.606 INFO:teuthology.orchestra.run.smithi181.stdout:(36/120): cephfs-top-17.2.6-1320.g08cac018.el8. 877 kB/s | 18 kB 00:00 2023-10-09T06:28:09.802 INFO:teuthology.orchestra.run.smithi181.stdout:(37/120): ceph-mgr-diskprediction-local-17.2.6- 16 MB/s | 7.4 MB 00:00 2023-10-09T06:28:09.990 INFO:teuthology.orchestra.run.smithi181.stdout:(38/120): gperftools-libs-2.7-9.el8.x86_64.rpm 796 kB/s | 306 kB 00:00 2023-10-09T06:28:10.055 INFO:teuthology.orchestra.run.smithi181.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 1.1 MB/s | 75 kB 00:00 2023-10-09T06:28:10.055 INFO:teuthology.orchestra.run.smithi181.stdout:(40/120): liboath-2.6.2-3.el8.x86_64.rpm 232 kB/s | 59 kB 00:00 2023-10-09T06:28:10.124 INFO:teuthology.orchestra.run.smithi181.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 638 kB/s | 44 kB 00:00 2023-10-09T06:28:10.168 INFO:teuthology.orchestra.run.smithi181.stdout:(42/120): python3-asyncssh-2.7.0-2.el8.noarch.r 4.1 MB/s | 476 kB 00:00 2023-10-09T06:28:10.218 INFO:teuthology.orchestra.run.smithi181.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 671 kB/s | 33 kB 00:00 2023-10-09T06:28:10.256 INFO:teuthology.orchestra.run.smithi181.stdout:(44/120): python3-beautifulsoup4-4.6.3-2.el8.1. 1.4 MB/s | 185 kB 00:00 2023-10-09T06:28:10.268 INFO:teuthology.orchestra.run.smithi181.stdout:(45/120): python3-certifi-2018.10.15-7.el8.noar 326 kB/s | 16 kB 00:00 2023-10-09T06:28:10.332 INFO:teuthology.orchestra.run.smithi181.stdout:(46/120): python3-cheroot-8.5.2-1.el8.noarch.rp 2.2 MB/s | 173 kB 00:00 2023-10-09T06:28:10.336 INFO:teuthology.orchestra.run.smithi181.stdout:(47/120): python3-cherrypy-18.4.0-1.el8.noarch. 5.5 MB/s | 384 kB 00:00 2023-10-09T06:28:10.385 INFO:teuthology.orchestra.run.smithi181.stdout:(48/120): python3-jaraco-6.2-6.el8.noarch.rpm 227 kB/s | 11 kB 00:00 2023-10-09T06:28:10.393 INFO:teuthology.orchestra.run.smithi181.stdout:(49/120): python3-google-auth-1.1.1-10.el8.noar 1.3 MB/s | 81 kB 00:00 2023-10-09T06:28:10.437 INFO:teuthology.orchestra.run.smithi181.stdout:(50/120): python3-jaraco-functools-2.0-4.el8.no 335 kB/s | 18 kB 00:00 2023-10-09T06:28:10.545 INFO:teuthology.orchestra.run.smithi181.stdout:(51/120): ceph-radosgw-17.2.6-1320.g08cac018.el 2.3 MB/s | 17 MB 00:07 2023-10-09T06:28:10.546 INFO:teuthology.orchestra.run.smithi181.stdout:(52/120): python3-logutils-0.3.5-11.el8.noarch. 449 kB/s | 49 kB 00:00 2023-10-09T06:28:10.603 INFO:teuthology.orchestra.run.smithi181.stdout:(53/120): python3-kubernetes-11.0.0-6.el8.noarc 5.6 MB/s | 1.2 MB 00:00 2023-10-09T06:28:10.603 INFO:teuthology.orchestra.run.smithi181.stdout:(54/120): python3-natsort-7.1.1-2.el8.noarch.rp 1.0 MB/s | 60 kB 00:00 2023-10-09T06:28:10.652 INFO:teuthology.orchestra.run.smithi181.stdout:(55/120): python3-portend-2.6-1.el8.noarch.rpm 315 kB/s | 16 kB 00:00 2023-10-09T06:28:10.664 INFO:teuthology.orchestra.run.smithi181.stdout:(56/120): python3-pecan-1.3.2-9.el8.noarch.rpm 4.5 MB/s | 283 kB 00:00 2023-10-09T06:28:10.702 INFO:teuthology.orchestra.run.smithi181.stdout:(57/120): python3-repoze-lru-0.7-6.el8.noarch.r 675 kB/s | 34 kB 00:00 2023-10-09T06:28:10.722 INFO:teuthology.orchestra.run.smithi181.stdout:(58/120): python3-routes-2.4.1-12.el8.noarch.rp 3.3 MB/s | 196 kB 00:00 2023-10-09T06:28:10.753 INFO:teuthology.orchestra.run.smithi181.stdout:(59/120): python3-rsa-4.9-2.el8.noarch.rpm 1.2 MB/s | 64 kB 00:00 2023-10-09T06:28:10.772 INFO:teuthology.orchestra.run.smithi181.stdout:(60/120): python3-simplegeneric-0.8.1-17.el8.no 379 kB/s | 19 kB 00:00 2023-10-09T06:28:10.796 INFO:teuthology.orchestra.run.smithi181.stdout:(61/120): python3-more-itertools-7.2.0-3.el8.no 234 kB/s | 59 kB 00:00 2023-10-09T06:28:10.803 INFO:teuthology.orchestra.run.smithi181.stdout:(62/120): python3-singledispatch-3.4.0.3-18.el8 485 kB/s | 24 kB 00:00 2023-10-09T06:28:10.821 INFO:teuthology.orchestra.run.smithi181.stdout:(63/120): python3-tempora-1.14.1-5.el8.noarch.r 594 kB/s | 29 kB 00:00 2023-10-09T06:28:10.857 INFO:teuthology.orchestra.run.smithi181.stdout:(64/120): python3-trustme-0.6.0-4.el8.noarch.rp 455 kB/s | 27 kB 00:00 2023-10-09T06:28:10.865 INFO:teuthology.orchestra.run.smithi181.stdout:(65/120): python3-waitress-1.4.3-1.el8.noarch.r 4.0 MB/s | 255 kB 00:00 2023-10-09T06:28:10.887 INFO:teuthology.orchestra.run.smithi181.stdout:(66/120): python3-webob-1.8.5-1.el8.1.noarch.rp 3.8 MB/s | 251 kB 00:00 2023-10-09T06:28:10.917 INFO:teuthology.orchestra.run.smithi181.stdout:(67/120): python3-webtest-2.0.33-1.el8.noarch.r 1.6 MB/s | 85 kB 00:00 2023-10-09T06:28:10.931 INFO:teuthology.orchestra.run.smithi181.stdout:(68/120): python3-websocket-client-0.56.0-5.el8 820 kB/s | 61 kB 00:00 2023-10-09T06:28:10.936 INFO:teuthology.orchestra.run.smithi181.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 15 MB/s | 70 kB 00:00 2023-10-09T06:28:10.937 INFO:teuthology.orchestra.run.smithi181.stdout:(70/120): python3-zc-lockfile-2.0-2.el8.noarch. 460 kB/s | 23 kB 00:00 2023-10-09T06:28:10.967 INFO:teuthology.orchestra.run.smithi181.stdout:(71/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 617 kB/s | 19 kB 00:00 2023-10-09T06:28:10.972 INFO:teuthology.orchestra.run.smithi181.stdout:(72/120): python3-jwt-1.6.1-2.el8.noarch.rpm 1.2 MB/s | 43 kB 00:00 2023-10-09T06:28:10.983 INFO:teuthology.orchestra.run.smithi181.stdout:(73/120): python3-oauthlib-2.1.0-1.el8.noarch.r 9.2 MB/s | 155 kB 00:00 2023-10-09T06:28:10.989 INFO:teuthology.orchestra.run.smithi181.stdout:(74/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 13 MB/s | 220 kB 00:00 2023-10-09T06:28:10.999 INFO:teuthology.orchestra.run.smithi181.stdout:(75/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 3.1 MB/s | 48 kB 00:00 2023-10-09T06:28:11.006 INFO:teuthology.orchestra.run.smithi181.stdout:(76/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 13 MB/s | 229 kB 00:00 2023-10-09T06:28:11.015 INFO:teuthology.orchestra.run.smithi181.stdout:(77/120): python3-requests-oauthlib-1.0.0-1.el8 2.5 MB/s | 43 kB 00:00 2023-10-09T06:28:11.030 INFO:teuthology.orchestra.run.smithi181.stdout:(78/120): libverto-devel-0.3.0-5.el8.x86_64.rpm 1.2 MB/s | 18 kB 00:00 2023-10-09T06:28:11.050 INFO:teuthology.orchestra.run.smithi181.stdout:(79/120): thrift-0.13.0-2.el8.x86_64.rpm 13 MB/s | 1.7 MB 00:00 2023-10-09T06:28:11.058 INFO:teuthology.orchestra.run.smithi181.stdout:(80/120): openssl-devel-1.1.1g-15.el8_3.x86_64. 45 MB/s | 2.3 MB 00:00 2023-10-09T06:28:11.068 INFO:teuthology.orchestra.run.smithi181.stdout:(81/120): pcre2-devel-10.32-2.el8.x86_64.rpm 16 MB/s | 605 kB 00:00 2023-10-09T06:28:11.073 INFO:teuthology.orchestra.run.smithi181.stdout:(82/120): elfutils-libelf-devel-0.182-3.el8.x86 3.9 MB/s | 59 kB 00:00 2023-10-09T06:28:11.081 INFO:teuthology.orchestra.run.smithi181.stdout:(83/120): fuse-2.9.7-12.el8.x86_64.rpm 2.7 MB/s | 83 kB 00:00 2023-10-09T06:28:11.085 INFO:teuthology.orchestra.run.smithi181.stdout:(84/120): libquadmath-8.4.1-1.el8.x86_64.rpm 9.9 MB/s | 169 kB 00:00 2023-10-09T06:28:11.087 INFO:teuthology.orchestra.run.smithi181.stdout:(85/120): libcom_err-devel-1.45.6-1.el8.x86_64. 2.6 MB/s | 38 kB 00:00 2023-10-09T06:28:11.096 INFO:teuthology.orchestra.run.smithi181.stdout:(86/120): libsepol-devel-2.9-2.el8.x86_64.rpm 5.6 MB/s | 86 kB 00:00 2023-10-09T06:28:11.101 INFO:teuthology.orchestra.run.smithi181.stdout:(87/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 3.5 MB/s | 58 kB 00:00 2023-10-09T06:28:11.102 INFO:teuthology.orchestra.run.smithi181.stdout:(88/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 3.2 MB/s | 47 kB 00:00 2023-10-09T06:28:11.116 INFO:teuthology.orchestra.run.smithi181.stdout:(89/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.rp 27 MB/s | 559 kB 00:00 2023-10-09T06:28:11.117 INFO:teuthology.orchestra.run.smithi181.stdout:(90/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rpm 4.4 MB/s | 70 kB 00:00 2023-10-09T06:28:11.123 INFO:teuthology.orchestra.run.smithi181.stdout:(91/120): libgfortran-8.4.1-1.el8.x86_64.rpm 28 MB/s | 639 kB 00:00 2023-10-09T06:28:11.132 INFO:teuthology.orchestra.run.smithi181.stdout:(92/120): libselinux-devel-2.9-5.el8.x86_64.rpm 13 MB/s | 200 kB 00:00 2023-10-09T06:28:11.133 INFO:teuthology.orchestra.run.smithi181.stdout:(93/120): libuuid-devel-2.32.1-27.el8.x86_64.rp 6.2 MB/s | 97 kB 00:00 2023-10-09T06:28:11.147 INFO:teuthology.orchestra.run.smithi181.stdout:(94/120): python3-jsonpatch-1.21-2.el8.noarch.r 1.8 MB/s | 27 kB 00:00 2023-10-09T06:28:11.149 INFO:teuthology.orchestra.run.smithi181.stdout:(95/120): python3-prettytable-0.7.2-14.el8.noar 2.5 MB/s | 44 kB 00:00 2023-10-09T06:28:11.162 INFO:teuthology.orchestra.run.smithi181.stdout:(96/120): jq-1.5-12.el8.x86_64.rpm 11 MB/s | 161 kB 00:00 2023-10-09T06:28:11.166 INFO:teuthology.orchestra.run.smithi181.stdout:(97/120): flex-2.6.1-9.el8.x86_64.rpm 18 MB/s | 320 kB 00:00 2023-10-09T06:28:11.180 INFO:teuthology.orchestra.run.smithi181.stdout:(98/120): python3-werkzeug-0.12.2-4.el8.noarch. 25 MB/s | 457 kB 00:00 2023-10-09T06:28:11.195 INFO:teuthology.orchestra.run.smithi181.stdout:(99/120): python3-markupsafe-0.23-19.el8.x86_64 2.6 MB/s | 39 kB 00:00 2023-10-09T06:28:11.217 INFO:teuthology.orchestra.run.smithi181.stdout:(100/120): bison-3.0.4-10.el8.x86_64.rpm 31 MB/s | 688 kB 00:00 2023-10-09T06:28:11.234 INFO:teuthology.orchestra.run.smithi181.stdout:(101/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 15 MB/s | 259 kB 00:00 2023-10-09T06:28:11.251 INFO:teuthology.orchestra.run.smithi181.stdout:(102/120): python3-babel-2.5.1-5.el8.noarch.rpm 57 MB/s | 4.8 MB 00:00 2023-10-09T06:28:11.255 INFO:teuthology.orchestra.run.smithi181.stdout:(103/120): python3-numpy-f2py-1.14.3-9.el8.x86_ 11 MB/s | 225 kB 00:00 2023-10-09T06:28:11.270 INFO:teuthology.orchestra.run.smithi181.stdout:(104/120): python3-pyasn1-0.3.7-6.el8.noarch.rp 6.8 MB/s | 126 kB 00:00 2023-10-09T06:28:11.285 INFO:teuthology.orchestra.run.smithi181.stdout:(105/120): oniguruma-6.8.2-2.el8.x86_64.rpm 12 MB/s | 187 kB 00:00 2023-10-09T06:28:11.307 INFO:teuthology.orchestra.run.smithi181.stdout:(106/120): python3-jinja2-2.10.1-2.el8_0.noarch 25 MB/s | 538 kB 00:00 2023-10-09T06:28:11.411 INFO:teuthology.orchestra.run.smithi181.stdout:(107/120): python3-scipy-1.0.0-20.module+el8.1. 51 MB/s | 15 MB 00:00 2023-10-09T06:28:11.416 INFO:teuthology.orchestra.run.smithi181.stdout:(108/120): socat-1.7.3.3-2.el8.x86_64.rpm 2.7 MB/s | 302 kB 00:00 2023-10-09T06:28:11.429 INFO:teuthology.orchestra.run.smithi181.stdout:(109/120): python3-numpy-1.14.3-9.el8.x86_64.rp 21 MB/s | 3.7 MB 00:00 2023-10-09T06:28:11.431 INFO:teuthology.orchestra.run.smithi181.stdout:(110/120): python3-pyasn1-modules-0.3.7-6.el8.n 7.2 MB/s | 110 kB 00:00 2023-10-09T06:28:11.432 INFO:teuthology.orchestra.run.smithi181.stdout:(111/120): python3-mako-1.0.6-13.el8.noarch.rpm 7.5 MB/s | 157 kB 00:00 2023-10-09T06:28:11.444 INFO:teuthology.orchestra.run.smithi181.stdout:(112/120): python3-jsonpointer-1.10-11.el8.noar 1.5 MB/s | 20 kB 00:00 2023-10-09T06:28:11.449 INFO:teuthology.orchestra.run.smithi181.stdout:(113/120): librdkafka-0.11.4-1.el8.x86_64.rpm 18 MB/s | 353 kB 00:00 2023-10-09T06:28:11.457 INFO:teuthology.orchestra.run.smithi181.stdout:(114/120): python3-pytz-2017.2-9.el8.noarch.rpm 4.1 MB/s | 54 kB 00:00 2023-10-09T06:28:11.463 INFO:teuthology.orchestra.run.smithi181.stdout:(115/120): libtool-2.4.6-25.el8.x86_64.rpm 23 MB/s | 709 kB 00:00 2023-10-09T06:28:11.493 INFO:teuthology.orchestra.run.smithi181.stdout:(116/120): automake-1.16.1-7.el8.noarch.rpm 23 MB/s | 713 kB 00:00 2023-10-09T06:28:11.528 INFO:teuthology.orchestra.run.smithi181.stdout:(117/120): openblas-threads-0.3.12-1.el8.x86_64 60 MB/s | 4.7 MB 00:00 2023-10-09T06:28:11.548 INFO:teuthology.orchestra.run.smithi181.stdout:(118/120): openblas-0.3.12-1.el8.x86_64.rpm 51 MB/s | 4.6 MB 00:00 2023-10-09T06:28:11.935 INFO:teuthology.orchestra.run.smithi181.stdout:(119/120): librbd1-17.2.6-1320.g08cac018.el8.x8 8.8 MB/s | 3.6 MB 00:00 2023-10-09T06:28:12.352 INFO:teuthology.orchestra.run.smithi181.stdout:(120/120): librados2-17.2.6-1320.g08cac018.el8. 4.0 MB/s | 3.5 MB 00:00 2023-10-09T06:28:12.356 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2023-10-09T06:28:12.357 INFO:teuthology.orchestra.run.smithi181.stdout:Total 15 MB/s | 202 MB 00:13 2023-10-09T06:28:12.403 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2023-10-09T06:28:12.515 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2023-10-09T06:28:12.515 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2023-10-09T06:28:13.757 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2023-10-09T06:28:13.947 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-09T06:28:14.191 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 2/122 2023-10-09T06:28:14.255 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2023-10-09T06:28:14.259 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2023-10-09T06:28:14.372 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-09T06:28:14.499 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-09T06:28:14.713 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-09T06:28:15.086 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-09T06:28:15.313 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : librados2-2:17.2.6-1320.g08cac018.el8.x86_64 5/122 2023-10-09T06:28:15.703 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librados2-2:17.2.6-1320.g08cac018.el8.x86_64 5/122 2023-10-09T06:28:15.913 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-09T06:28:16.046 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-09T06:28:16.246 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 7/122 2023-10-09T06:28:16.349 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 7/122 2023-10-09T06:28:16.755 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 8/122 2023-10-09T06:28:17.401 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-09T06:28:17.763 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-09T06:28:17.859 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-09T06:28:18.091 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-09T06:28:18.278 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-09T06:28:18.471 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-09T06:28:18.574 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-09T06:28:18.762 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 14/122 2023-10-09T06:28:18.828 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 14/122 2023-10-09T06:28:18.975 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 15/122 2023-10-09T06:28:19.063 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2023-10-09T06:28:19.121 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-09T06:28:19.258 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-09T06:28:19.291 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-09T06:28:19.470 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-09T06:28:19.491 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-09T06:28:19.640 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-09T06:28:19.875 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-09T06:28:20.009 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-09T06:28:20.179 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-09T06:28:20.260 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-09T06:28:20.467 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librados2-2:17.2.6-1320.g08cac018.el8.x86_64 4/122 2023-10-09T06:28:20.700 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-09T06:28:20.873 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-2:17.2.6-1320.g08cac018.el8.x86_64 4/122 2023-10-09T06:28:20.887 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openblas-0.3.12-1.el8.x86_64 19/122 2023-10-09T06:28:21.083 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 5/122 2023-10-09T06:28:21.216 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 5/122 2023-10-09T06:28:21.383 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-09T06:28:21.485 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-09T06:28:21.624 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 19/122 2023-10-09T06:28:21.663 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 7/122 2023-10-09T06:28:21.882 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 20/122 2023-10-09T06:28:22.022 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/122 2023-10-09T06:28:22.052 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 21/122 2023-10-09T06:28:22.250 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 22/122 2023-10-09T06:28:22.501 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/122 2023-10-09T06:28:22.716 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-09T06:28:22.810 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 24/122 2023-10-09T06:28:22.908 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-09T06:28:23.002 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-09T06:28:23.253 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-09T06:28:23.431 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-09T06:28:23.607 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-09T06:28:23.625 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-09T06:28:23.753 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-09T06:28:23.786 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ceph-common-2:17.2.6-1320.g08cac018.el8. 26/122 2023-10-09T06:28:23.907 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 14/122 2023-10-09T06:28:23.915 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ceph-argparse-2:17.2.6-1320.g08cac018.el 27/122 2023-10-09T06:28:24.022 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 14/122 2023-10-09T06:28:24.171 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 15/122 2023-10-09T06:28:24.225 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_6 28/122 2023-10-09T06:28:24.457 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 16/122 2023-10-09T06:28:24.698 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/122 2023-10-09T06:28:24.872 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 18/122 2023-10-09T06:28:25.032 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-09T06:28:25.203 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-09T06:28:25.254 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 29/122 2023-10-09T06:28:25.399 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-09T06:28:25.583 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 30/122 2023-10-09T06:28:25.904 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 31/122 2023-10-09T06:28:25.955 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-09T06:28:26.107 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-09T06:28:26.134 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 32/122 2023-10-09T06:28:26.410 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 33/122 2023-10-09T06:28:26.661 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-09T06:28:26.671 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-09T06:28:26.873 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-0.3.12-1.el8.x86_64 22/122 2023-10-09T06:28:27.619 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 22/122 2023-10-09T06:28:28.414 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 23/122 2023-10-09T06:28:28.695 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 24/122 2023-10-09T06:28:28.754 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-09T06:28:28.961 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-09T06:28:29.097 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 35/122 2023-10-09T06:28:29.163 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-common-2:17.2.6-1320.g08cac018.el8. 26/122 2023-10-09T06:28:29.327 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-argparse-2:17.2.6-1320.g08cac018.el 27/122 2023-10-09T06:28:29.349 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-09T06:28:29.543 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-09T06:28:29.624 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_6 28/122 2023-10-09T06:28:29.733 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 38/122 2023-10-09T06:28:29.888 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 29/122 2023-10-09T06:28:29.924 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 39/122 2023-10-09T06:28:30.104 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librados-devel-2:17.2.6-1320.g08cac018.el8.x86_6 40/122 2023-10-09T06:28:30.140 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 30/122 2023-10-09T06:28:30.258 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 41/122 2023-10-09T06:28:30.331 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-09T06:28:30.441 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 42/122 2023-10-09T06:28:30.629 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 43/122 2023-10-09T06:28:30.788 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : fuse-2.9.7-12.el8.x86_64 44/122 2023-10-09T06:28:30.955 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 45/122 2023-10-09T06:28:31.134 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 46/122 2023-10-09T06:28:31.297 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 47/122 2023-10-09T06:28:31.585 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 48/122 2023-10-09T06:28:31.840 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 49/122 2023-10-09T06:28:32.094 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 50/122 2023-10-09T06:28:32.434 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-09T06:28:32.435 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 51/122 2023-10-09T06:28:32.657 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : automake-1.16.1-7.el8.noarch 52/122 2023-10-09T06:28:32.830 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 32/122 2023-10-09T06:28:32.842 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 53/122 2023-10-09T06:28:33.025 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 54/122 2023-10-09T06:28:33.256 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 55/122 2023-10-09T06:28:33.402 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-09T06:28:33.477 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-09T06:28:33.652 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : jq-1.5-12.el8.x86_64 57/122 2023-10-09T06:28:33.840 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/122 2023-10-09T06:28:33.887 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 58/122 2023-10-09T06:28:34.036 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-09T06:28:34.170 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/122 2023-10-09T06:28:34.190 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-09T06:28:34.418 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/122 2023-10-09T06:28:34.459 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-09T06:28:34.660 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-09T06:28:34.690 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-09T06:28:34.842 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-09T06:28:34.917 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-09T06:28:35.030 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-09T06:28:35.150 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 38/122 2023-10-09T06:28:35.187 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-09T06:28:35.361 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 39/122 2023-10-09T06:28:35.369 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-09T06:28:35.613 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-09T06:28:35.644 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librados-devel-2:17.2.6-1320.g08cac018.el8.x86_6 40/122 2023-10-09T06:28:35.829 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-09T06:28:35.888 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : automake-1.16.1-7.el8.noarch 41/122 2023-10-09T06:28:36.005 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-09T06:28:36.054 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 42/122 2023-10-09T06:28:36.182 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-09T06:28:36.270 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 43/122 2023-10-09T06:28:36.415 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-09T06:28:36.484 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 44/122 2023-10-09T06:28:36.630 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-09T06:28:36.631 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-09T06:28:36.736 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-09T06:28:36.875 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-09T06:28:36.880 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : jq-1.5-12.el8.x86_64 46/122 2023-10-09T06:28:37.060 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 47/122 2023-10-09T06:28:37.158 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-09T06:28:37.279 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 48/122 2023-10-09T06:28:37.408 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-09T06:28:37.442 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 49/122 2023-10-09T06:28:37.591 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-09T06:28:37.628 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 50/122 2023-10-09T06:28:37.816 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : fuse-2.9.7-12.el8.x86_64 51/122 2023-10-09T06:28:37.832 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-09T06:28:37.951 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 52/122 2023-10-09T06:28:38.170 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 53/122 2023-10-09T06:28:38.350 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 54/122 2023-10-09T06:28:38.658 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/122 2023-10-09T06:28:38.903 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/122 2023-10-09T06:28:38.990 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-09T06:28:39.185 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/122 2023-10-09T06:28:39.290 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-09T06:28:39.510 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 58/122 2023-10-09T06:28:39.536 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-09T06:28:39.664 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-09T06:28:39.867 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-09T06:28:39.883 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-09T06:28:40.143 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-09T06:28:40.171 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-09T06:28:40.372 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-09T06:28:40.515 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-09T06:28:40.612 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-09T06:28:40.747 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-09T06:28:40.785 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-09T06:28:40.924 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-09T06:28:40.967 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-09T06:28:41.106 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-09T06:28:41.176 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-09T06:28:41.308 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-09T06:28:41.531 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-09T06:28:41.758 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-09T06:28:41.919 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-09T06:28:42.174 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-09T06:28:42.424 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-09T06:28:42.660 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-09T06:28:42.983 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-09T06:28:43.228 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-09T06:28:43.453 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-09T06:28:43.770 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-09T06:28:44.969 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-09T06:28:45.294 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-09T06:28:45.522 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-09T06:28:45.881 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-09T06:28:46.162 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-09T06:28:46.250 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-09T06:28:46.471 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-09T06:28:46.492 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-09T06:28:46.718 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-09T06:28:46.896 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-09T06:28:47.088 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-09T06:28:47.129 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-09T06:28:47.375 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 88/122 2023-10-09T06:28:47.466 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 88/122 2023-10-09T06:28:47.633 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 89/122 2023-10-09T06:28:52.472 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-09T06:28:52.720 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-09T06:28:53.307 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-09T06:28:53.580 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 88/122 2023-10-09T06:28:53.670 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 88/122 2023-10-09T06:28:53.853 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 89/122 2023-10-09T06:29:05.293 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 89/122 2023-10-09T06:29:05.294 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /sys 2023-10-09T06:29:05.294 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /proc 2023-10-09T06:29:05.294 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /dev 2023-10-09T06:29:05.294 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /run 2023-10-09T06:29:05.294 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /mnt 2023-10-09T06:29:05.294 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /var/tmp 2023-10-09T06:29:05.294 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /home 2023-10-09T06:29:05.294 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /tmp 2023-10-09T06:29:05.294 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /dev 2023-10-09T06:29:05.294 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:05.696 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 90/122 2023-10-09T06:29:07.214 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 90/122 2023-10-09T06:29:07.355 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 91/122 2023-10-09T06:29:07.580 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 91/122 2023-10-09T06:29:07.800 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 92/122 2023-10-09T06:29:08.192 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 92/122 2023-10-09T06:29:08.371 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 93/122 2023-10-09T06:29:08.388 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 93/122 2023-10-09T06:29:11.321 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: cephadm-2:17.2.6-1320.g08cac018.el8.noarch 94/122 2023-10-09T06:29:11.499 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephadm-2:17.2.6-1320.g08cac018.el8.noarch 94/122 2023-10-09T06:29:11.629 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 89/122 2023-10-09T06:29:11.629 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /sys 2023-10-09T06:29:11.629 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /proc 2023-10-09T06:29:11.630 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /dev 2023-10-09T06:29:11.630 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /run 2023-10-09T06:29:11.630 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /mnt 2023-10-09T06:29:11.630 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /var/tmp 2023-10-09T06:29:11.630 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /home 2023-10-09T06:29:11.630 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /tmp 2023-10-09T06:29:11.630 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /dev 2023-10-09T06:29:11.630 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:11.668 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.e 95/122 2023-10-09T06:29:11.916 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1320.g08cac018. 96/122 2023-10-09T06:29:12.033 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 90/122 2023-10-09T06:29:12.084 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 97/122 2023-10-09T06:29:12.300 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 97/122 2023-10-09T06:29:12.546 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 98/122 2023-10-09T06:29:13.529 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 98/122 2023-10-09T06:29:13.546 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 90/122 2023-10-09T06:29:13.695 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 99/122 2023-10-09T06:29:13.735 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 91/122 2023-10-09T06:29:13.792 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 99/122 2023-10-09T06:29:13.961 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 91/122 2023-10-09T06:29:14.127 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 92/122 2023-10-09T06:29:14.168 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el 100/122 2023-10-09T06:29:14.362 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 101/122 2023-10-09T06:29:14.399 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 101/122 2023-10-09T06:29:14.522 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 92/122 2023-10-09T06:29:14.569 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 102/122 2023-10-09T06:29:14.581 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 102/122 2023-10-09T06:29:14.666 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 93/122 2023-10-09T06:29:14.684 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 93/122 2023-10-09T06:29:16.216 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-2:17.2.6-1320.g08cac018.el8.x86_64 103/122 2023-10-09T06:29:16.471 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 104/122 2023-10-09T06:29:16.567 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 104/122 2023-10-09T06:29:16.720 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1320.g08cac 105/122 2023-10-09T06:29:16.763 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1320.g08cac 105/122 2023-10-09T06:29:16.936 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 106/122 2023-10-09T06:29:17.285 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 106/122 2023-10-09T06:29:17.453 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 107/122 2023-10-09T06:29:17.697 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephadm-2:17.2.6-1320.g08cac018.el8.noarch 94/122 2023-10-09T06:29:17.931 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephadm-2:17.2.6-1320.g08cac018.el8.noarch 94/122 2023-10-09T06:29:18.114 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.e 95/122 2023-10-09T06:29:18.301 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1320.g08cac018. 96/122 2023-10-09T06:29:18.499 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 97/122 2023-10-09T06:29:18.720 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 97/122 2023-10-09T06:29:18.976 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 98/122 2023-10-09T06:29:19.959 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 98/122 2023-10-09T06:29:20.148 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 99/122 2023-10-09T06:29:20.248 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 99/122 2023-10-09T06:29:20.625 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el 100/122 2023-10-09T06:29:20.793 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 101/122 2023-10-09T06:29:20.830 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 101/122 2023-10-09T06:29:20.998 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 102/122 2023-10-09T06:29:21.009 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 102/122 2023-10-09T06:29:22.392 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 107/122 2023-10-09T06:29:22.611 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-2:17.2.6-1320.g08cac018.el8.x86_64 103/122 2023-10-09T06:29:22.714 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 108/122 2023-10-09T06:29:22.884 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 104/122 2023-10-09T06:29:22.910 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libtool-2.4.6-25.el8.x86_64 109/122 2023-10-09T06:29:22.977 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 104/122 2023-10-09T06:29:23.149 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1320.g08cac 105/122 2023-10-09T06:29:23.192 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1320.g08cac 105/122 2023-10-09T06:29:23.399 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 106/122 2023-10-09T06:29:23.629 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 109/122 2023-10-09T06:29:23.751 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 106/122 2023-10-09T06:29:23.940 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 107/122 2023-10-09T06:29:23.992 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 110/122 2023-10-09T06:29:24.182 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 111/122 2023-10-09T06:29:24.338 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 112/122 2023-10-09T06:29:24.514 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_ 113/122 2023-10-09T06:29:24.664 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-09T06:29:24.826 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 115/122 2023-10-09T06:29:25.001 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 116/122 2023-10-09T06:29:25.175 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 117/122 2023-10-09T06:29:25.380 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 118/122 2023-10-09T06:29:25.576 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : bison-3.0.4-10.el8.x86_64 119/122 2023-10-09T06:29:25.649 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 119/122 2023-10-09T06:29:25.834 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flex-2.6.1-9.el8.x86_64 120/122 2023-10-09T06:29:25.844 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 120/122 2023-10-09T06:29:25.844 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-09T06:29:26.071 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-09T06:29:26.071 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-09T06:29:26.987 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-09T06:29:26.987 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-2:17.2.6-1320.g08cac018.el8.x86_64 1/122 2023-10-09T06:29:26.987 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 2/122 2023-10-09T06:29:26.987 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 3/122 2023-10-09T06:29:26.987 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 4/122 2023-10-09T06:29:26.987 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1320.g08cac 5/122 2023-10-09T06:29:26.987 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-09T06:29:26.987 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 7/122 2023-10-09T06:29:26.988 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 8/122 2023-10-09T06:29:26.988 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 9/122 2023-10-09T06:29:26.988 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-09T06:29:26.988 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 11/122 2023-10-09T06:29:26.988 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 12/122 2023-10-09T06:29:26.988 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-09T06:29:26.988 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_ 14/122 2023-10-09T06:29:26.988 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 15/122 2023-10-09T06:29:26.988 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 16/122 2023-10-09T06:29:26.988 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados-devel-2:17.2.6-1320.g08cac018.el8.x86_6 17/122 2023-10-09T06:29:26.988 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 18/122 2023-10-09T06:29:26.988 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 19/122 2023-10-09T06:29:26.989 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1320.g08cac018.el 20/122 2023-10-09T06:29:26.989 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ceph-common-2:17.2.6-1320.g08cac018.el8. 21/122 2023-10-09T06:29:26.989 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_6 22/122 2023-10-09T06:29:26.989 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 23/122 2023-10-09T06:29:26.989 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 24/122 2023-10-09T06:29:26.989 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 25/122 2023-10-09T06:29:26.989 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 26/122 2023-10-09T06:29:26.989 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 27/122 2023-10-09T06:29:26.997 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 28/122 2023-10-09T06:29:26.998 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1320.g08cac018. 29/122 2023-10-09T06:29:26.998 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 30/122 2023-10-09T06:29:26.998 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 31/122 2023-10-09T06:29:26.998 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 32/122 2023-10-09T06:29:26.998 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el 33/122 2023-10-09T06:29:26.998 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 34/122 2023-10-09T06:29:26.998 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.e 35/122 2023-10-09T06:29:26.998 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 36/122 2023-10-09T06:29:26.998 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephadm-2:17.2.6-1320.g08cac018.el8.noarch 37/122 2023-10-09T06:29:26.999 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 38/122 2023-10-09T06:29:26.999 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-09T06:29:26.999 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-09T06:29:26.999 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-09T06:29:26.999 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-09T06:29:26.999 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-09T06:29:26.999 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-09T06:29:26.999 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-09T06:29:26.999 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-09T06:29:26.999 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-09T06:29:26.999 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-09T06:29:26.999 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-09T06:29:27.000 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-09T06:29:27.000 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-09T06:29:27.000 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-09T06:29:27.000 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-09T06:29:27.000 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-09T06:29:27.000 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-09T06:29:27.000 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-09T06:29:27.000 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-09T06:29:27.001 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-09T06:29:27.001 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-09T06:29:27.001 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-09T06:29:27.001 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-09T06:29:27.001 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-09T06:29:27.001 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-09T06:29:27.001 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-09T06:29:27.001 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-09T06:29:27.001 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-09T06:29:27.002 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-09T06:29:27.002 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-09T06:29:27.002 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-09T06:29:27.002 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-09T06:29:27.002 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-09T06:29:27.002 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 72/122 2023-10-09T06:29:27.002 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/122 2023-10-09T06:29:27.002 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 74/122 2023-10-09T06:29:27.002 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : jq-1.5-12.el8.x86_64 75/122 2023-10-09T06:29:27.003 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flex-2.6.1-9.el8.x86_64 76/122 2023-10-09T06:29:27.003 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/122 2023-10-09T06:29:27.003 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 78/122 2023-10-09T06:29:27.003 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 79/122 2023-10-09T06:29:27.003 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : bison-3.0.4-10.el8.x86_64 80/122 2023-10-09T06:29:27.003 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 81/122 2023-10-09T06:29:27.003 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 82/122 2023-10-09T06:29:27.003 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 83/122 2023-10-09T06:29:27.003 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 84/122 2023-10-09T06:29:27.003 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 85/122 2023-10-09T06:29:27.004 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 86/122 2023-10-09T06:29:27.004 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 87/122 2023-10-09T06:29:27.004 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 88/122 2023-10-09T06:29:27.004 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 89/122 2023-10-09T06:29:27.004 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 90/122 2023-10-09T06:29:27.004 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 91/122 2023-10-09T06:29:27.004 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 92/122 2023-10-09T06:29:27.004 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 93/122 2023-10-09T06:29:27.005 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 94/122 2023-10-09T06:29:27.005 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 95/122 2023-10-09T06:29:27.005 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : automake-1.16.1-7.el8.noarch 96/122 2023-10-09T06:29:27.005 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 97/122 2023-10-09T06:29:27.005 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 98/122 2023-10-09T06:29:27.005 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 99/122 2023-10-09T06:29:27.005 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 100/122 2023-10-09T06:29:27.005 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 101/122 2023-10-09T06:29:27.006 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 102/122 2023-10-09T06:29:27.006 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 103/122 2023-10-09T06:29:27.006 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 104/122 2023-10-09T06:29:27.006 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 105/122 2023-10-09T06:29:27.006 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 106/122 2023-10-09T06:29:27.006 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 107/122 2023-10-09T06:29:27.006 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 108/122 2023-10-09T06:29:27.006 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 109/122 2023-10-09T06:29:27.007 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 110/122 2023-10-09T06:29:27.007 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 111/122 2023-10-09T06:29:27.007 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 112/122 2023-10-09T06:29:27.007 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 113/122 2023-10-09T06:29:27.007 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 114/122 2023-10-09T06:29:27.007 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 115/122 2023-10-09T06:29:27.007 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 116/122 2023-10-09T06:29:27.007 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 117/122 2023-10-09T06:29:27.007 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 118/122 2023-10-09T06:29:27.007 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados2-2:17.2.6-1320.g08cac018.el8.x86_64 119/122 2023-10-09T06:29:27.008 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-09T06:29:27.008 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 121/122 2023-10-09T06:29:28.933 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 107/122 2023-10-09T06:29:29.237 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-09T06:29:29.237 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:01:40 ago on Mon 09 Oct 2023 06:27:49 AM UTC. 2023-10-09T06:29:29.906 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 108/122 2023-10-09T06:29:30.233 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 109/122 2023-10-09T06:29:30.414 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 110/122 2023-10-09T06:29:30.688 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 111/122 2023-10-09T06:29:30.872 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libtool-2.4.6-25.el8.x86_64 112/122 2023-10-09T06:29:30.884 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 112/122 2023-10-09T06:29:31.077 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_ 113/122 2023-10-09T06:29:31.227 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-09T06:29:31.263 INFO:teuthology.orchestra.run.smithi163.stdout:Installed products updated. 2023-10-09T06:29:31.405 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 115/122 2023-10-09T06:29:31.408 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:31.408 INFO:teuthology.orchestra.run.smithi163.stdout:Upgraded: 2023-10-09T06:29:31.409 INFO:teuthology.orchestra.run.smithi163.stdout: librados2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.409 INFO:teuthology.orchestra.run.smithi163.stdout: librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.409 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2023-10-09T06:29:31.409 INFO:teuthology.orchestra.run.smithi163.stdout: automake-1.16.1-7.el8.noarch 2023-10-09T06:29:31.409 INFO:teuthology.orchestra.run.smithi163.stdout: bison-3.0.4-10.el8.x86_64 2023-10-09T06:29:31.409 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.409 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.410 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.410 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.410 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-grafana-dashboards-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-09T06:29:31.410 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-immutable-object-cache-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.410 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.410 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.411 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-09T06:29:31.411 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-09T06:29:31.411 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-09T06:29:31.411 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-09T06:29:31.411 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-09T06:29:31.411 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.411 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.411 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-09T06:29:31.412 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.412 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.412 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.412 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-09T06:29:31.412 INFO:teuthology.orchestra.run.smithi163.stdout: cephadm-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-09T06:29:31.412 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.412 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-09T06:29:31.412 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-09T06:29:31.412 INFO:teuthology.orchestra.run.smithi163.stdout: flex-2.6.1-9.el8.x86_64 2023-10-09T06:29:31.413 INFO:teuthology.orchestra.run.smithi163.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-09T06:29:31.413 INFO:teuthology.orchestra.run.smithi163.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-09T06:29:31.413 INFO:teuthology.orchestra.run.smithi163.stdout: jq-1.5-12.el8.x86_64 2023-10-09T06:29:31.413 INFO:teuthology.orchestra.run.smithi163.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-09T06:29:31.413 INFO:teuthology.orchestra.run.smithi163.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-09T06:29:31.413 INFO:teuthology.orchestra.run.smithi163.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-09T06:29:31.413 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.413 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.414 INFO:teuthology.orchestra.run.smithi163.stdout: libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.414 INFO:teuthology.orchestra.run.smithi163.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-09T06:29:31.414 INFO:teuthology.orchestra.run.smithi163.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-09T06:29:31.414 INFO:teuthology.orchestra.run.smithi163.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-09T06:29:31.414 INFO:teuthology.orchestra.run.smithi163.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-09T06:29:31.414 INFO:teuthology.orchestra.run.smithi163.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-09T06:29:31.414 INFO:teuthology.orchestra.run.smithi163.stdout: librados-devel-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.414 INFO:teuthology.orchestra.run.smithi163.stdout: libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.414 INFO:teuthology.orchestra.run.smithi163.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-09T06:29:31.414 INFO:teuthology.orchestra.run.smithi163.stdout: librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.415 INFO:teuthology.orchestra.run.smithi163.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-09T06:29:31.415 INFO:teuthology.orchestra.run.smithi163.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-09T06:29:31.415 INFO:teuthology.orchestra.run.smithi163.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-09T06:29:31.415 INFO:teuthology.orchestra.run.smithi163.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-09T06:29:31.415 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-09T06:29:31.415 INFO:teuthology.orchestra.run.smithi163.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-09T06:29:31.415 INFO:teuthology.orchestra.run.smithi163.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-09T06:29:31.415 INFO:teuthology.orchestra.run.smithi163.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-09T06:29:31.415 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-09T06:29:31.416 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-09T06:29:31.416 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-09T06:29:31.416 INFO:teuthology.orchestra.run.smithi163.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-09T06:29:31.416 INFO:teuthology.orchestra.run.smithi163.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-09T06:29:31.416 INFO:teuthology.orchestra.run.smithi163.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-09T06:29:31.416 INFO:teuthology.orchestra.run.smithi163.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-09T06:29:31.416 INFO:teuthology.orchestra.run.smithi163.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-09T06:29:31.416 INFO:teuthology.orchestra.run.smithi163.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-09T06:29:31.416 INFO:teuthology.orchestra.run.smithi163.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-09T06:29:31.416 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-09T06:29:31.417 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-argparse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.417 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.417 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.417 INFO:teuthology.orchestra.run.smithi163.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-09T06:29:31.417 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-09T06:29:31.417 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-09T06:29:31.417 INFO:teuthology.orchestra.run.smithi163.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-09T06:29:31.417 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-09T06:29:31.418 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-09T06:29:31.418 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-09T06:29:31.418 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-09T06:29:31.418 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-09T06:29:31.418 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-09T06:29:31.418 INFO:teuthology.orchestra.run.smithi163.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-09T06:29:31.419 INFO:teuthology.orchestra.run.smithi163.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-09T06:29:31.419 INFO:teuthology.orchestra.run.smithi163.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-09T06:29:31.419 INFO:teuthology.orchestra.run.smithi163.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-09T06:29:31.419 INFO:teuthology.orchestra.run.smithi163.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-09T06:29:31.419 INFO:teuthology.orchestra.run.smithi163.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-09T06:29:31.419 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-09T06:29:31.419 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-09T06:29:31.419 INFO:teuthology.orchestra.run.smithi163.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-09T06:29:31.419 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-09T06:29:31.419 INFO:teuthology.orchestra.run.smithi163.stdout: python3-portend-2.6-1.el8.noarch 2023-10-09T06:29:31.419 INFO:teuthology.orchestra.run.smithi163.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-09T06:29:31.420 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-09T06:29:31.420 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-09T06:29:31.420 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-09T06:29:31.420 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.420 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.420 INFO:teuthology.orchestra.run.smithi163.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-09T06:29:31.420 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-09T06:29:31.420 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.421 INFO:teuthology.orchestra.run.smithi163.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-09T06:29:31.421 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-09T06:29:31.421 INFO:teuthology.orchestra.run.smithi163.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-09T06:29:31.421 INFO:teuthology.orchestra.run.smithi163.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-09T06:29:31.421 INFO:teuthology.orchestra.run.smithi163.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-09T06:29:31.421 INFO:teuthology.orchestra.run.smithi163.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-09T06:29:31.421 INFO:teuthology.orchestra.run.smithi163.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-09T06:29:31.421 INFO:teuthology.orchestra.run.smithi163.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-09T06:29:31.422 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-09T06:29:31.422 INFO:teuthology.orchestra.run.smithi163.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-09T06:29:31.422 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-09T06:29:31.422 INFO:teuthology.orchestra.run.smithi163.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-09T06:29:31.422 INFO:teuthology.orchestra.run.smithi163.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-09T06:29:31.422 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.422 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.422 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:31.422 INFO:teuthology.orchestra.run.smithi163.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-09T06:29:31.423 INFO:teuthology.orchestra.run.smithi163.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-09T06:29:31.423 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-09T06:29:31.423 INFO:teuthology.orchestra.run.smithi163.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-09T06:29:31.423 INFO:teuthology.orchestra.run.smithi163.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-09T06:29:31.423 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:31.424 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2023-10-09T06:29:31.564 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 116/122 2023-10-09T06:29:31.660 DEBUG:teuthology.parallel:result is None 2023-10-09T06:29:31.824 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 117/122 2023-10-09T06:29:32.047 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : bison-3.0.4-10.el8.x86_64 118/122 2023-10-09T06:29:32.101 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 118/122 2023-10-09T06:29:32.297 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flex-2.6.1-9.el8.x86_64 119/122 2023-10-09T06:29:32.306 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 119/122 2023-10-09T06:29:32.416 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 120/122 2023-10-09T06:29:32.416 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-09T06:29:32.633 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-09T06:29:32.633 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-09T06:29:33.554 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-09T06:29:33.554 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-2:17.2.6-1320.g08cac018.el8.x86_64 1/122 2023-10-09T06:29:33.554 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 2/122 2023-10-09T06:29:33.554 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 3/122 2023-10-09T06:29:33.554 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 4/122 2023-10-09T06:29:33.554 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1320.g08cac 5/122 2023-10-09T06:29:33.554 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-09T06:29:33.555 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 7/122 2023-10-09T06:29:33.555 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 8/122 2023-10-09T06:29:33.555 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 9/122 2023-10-09T06:29:33.555 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-09T06:29:33.555 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 11/122 2023-10-09T06:29:33.555 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 12/122 2023-10-09T06:29:33.555 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-09T06:29:33.555 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_ 14/122 2023-10-09T06:29:33.555 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 15/122 2023-10-09T06:29:33.555 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 16/122 2023-10-09T06:29:33.555 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados-devel-2:17.2.6-1320.g08cac018.el8.x86_6 17/122 2023-10-09T06:29:33.555 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 18/122 2023-10-09T06:29:33.556 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 19/122 2023-10-09T06:29:33.556 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1320.g08cac018.el 20/122 2023-10-09T06:29:33.556 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-common-2:17.2.6-1320.g08cac018.el8. 21/122 2023-10-09T06:29:33.556 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_6 22/122 2023-10-09T06:29:33.556 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 23/122 2023-10-09T06:29:33.556 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 24/122 2023-10-09T06:29:33.556 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 25/122 2023-10-09T06:29:33.556 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 26/122 2023-10-09T06:29:33.556 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 27/122 2023-10-09T06:29:33.556 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 28/122 2023-10-09T06:29:33.556 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1320.g08cac018. 29/122 2023-10-09T06:29:33.564 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 30/122 2023-10-09T06:29:33.564 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 31/122 2023-10-09T06:29:33.564 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 32/122 2023-10-09T06:29:33.564 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el 33/122 2023-10-09T06:29:33.564 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 34/122 2023-10-09T06:29:33.564 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.e 35/122 2023-10-09T06:29:33.565 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 36/122 2023-10-09T06:29:33.565 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephadm-2:17.2.6-1320.g08cac018.el8.noarch 37/122 2023-10-09T06:29:33.565 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 38/122 2023-10-09T06:29:33.565 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-09T06:29:33.565 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-09T06:29:33.565 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-09T06:29:33.565 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-09T06:29:33.565 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-09T06:29:33.565 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-09T06:29:33.565 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-09T06:29:33.565 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-09T06:29:33.565 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-09T06:29:33.566 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-09T06:29:33.566 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-09T06:29:33.566 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-09T06:29:33.566 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-09T06:29:33.566 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-09T06:29:33.566 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-09T06:29:33.566 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-09T06:29:33.566 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-09T06:29:33.566 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-09T06:29:33.566 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-09T06:29:33.566 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-09T06:29:33.566 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-09T06:29:33.567 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-09T06:29:33.567 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-09T06:29:33.567 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-09T06:29:33.567 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-09T06:29:33.567 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-09T06:29:33.567 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-09T06:29:33.567 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-09T06:29:33.567 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-09T06:29:33.567 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-09T06:29:33.567 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-09T06:29:33.567 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-09T06:29:33.567 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-09T06:29:33.567 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 72/122 2023-10-09T06:29:33.568 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/122 2023-10-09T06:29:33.568 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/122 2023-10-09T06:29:33.568 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/122 2023-10-09T06:29:33.568 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/122 2023-10-09T06:29:33.568 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 77/122 2023-10-09T06:29:33.568 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/122 2023-10-09T06:29:33.568 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 79/122 2023-10-09T06:29:33.568 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 80/122 2023-10-09T06:29:33.568 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 81/122 2023-10-09T06:29:33.568 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 82/122 2023-10-09T06:29:33.568 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 83/122 2023-10-09T06:29:33.568 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 84/122 2023-10-09T06:29:33.569 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 85/122 2023-10-09T06:29:33.569 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 86/122 2023-10-09T06:29:33.569 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 87/122 2023-10-09T06:29:33.569 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 88/122 2023-10-09T06:29:33.569 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 89/122 2023-10-09T06:29:33.569 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 90/122 2023-10-09T06:29:33.569 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 91/122 2023-10-09T06:29:33.569 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 92/122 2023-10-09T06:29:33.569 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 93/122 2023-10-09T06:29:33.569 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 94/122 2023-10-09T06:29:33.569 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 95/122 2023-10-09T06:29:33.569 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 96/122 2023-10-09T06:29:33.570 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : jq-1.5-12.el8.x86_64 97/122 2023-10-09T06:29:33.570 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flex-2.6.1-9.el8.x86_64 98/122 2023-10-09T06:29:33.570 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 99/122 2023-10-09T06:29:33.570 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 100/122 2023-10-09T06:29:33.570 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 101/122 2023-10-09T06:29:33.570 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : bison-3.0.4-10.el8.x86_64 102/122 2023-10-09T06:29:33.570 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 103/122 2023-10-09T06:29:33.570 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 104/122 2023-10-09T06:29:33.570 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/122 2023-10-09T06:29:33.570 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 106/122 2023-10-09T06:29:33.570 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/122 2023-10-09T06:29:33.570 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 108/122 2023-10-09T06:29:33.571 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 109/122 2023-10-09T06:29:33.571 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 110/122 2023-10-09T06:29:33.571 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 111/122 2023-10-09T06:29:33.571 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 112/122 2023-10-09T06:29:33.571 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 113/122 2023-10-09T06:29:33.571 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 114/122 2023-10-09T06:29:33.571 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 115/122 2023-10-09T06:29:33.571 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 116/122 2023-10-09T06:29:33.571 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 117/122 2023-10-09T06:29:33.571 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : automake-1.16.1-7.el8.noarch 118/122 2023-10-09T06:29:33.571 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-2:17.2.6-1320.g08cac018.el8.x86_64 119/122 2023-10-09T06:29:33.571 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-09T06:29:33.571 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 121/122 2023-10-09T06:29:35.809 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-09T06:29:35.809 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:01:45 ago on Mon 09 Oct 2023 06:27:50 AM UTC. 2023-10-09T06:29:37.902 INFO:teuthology.orchestra.run.smithi181.stdout:Installed products updated. 2023-10-09T06:29:38.040 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:38.040 INFO:teuthology.orchestra.run.smithi181.stdout:Upgraded: 2023-10-09T06:29:38.040 INFO:teuthology.orchestra.run.smithi181.stdout: librados2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.040 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.040 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2023-10-09T06:29:38.040 INFO:teuthology.orchestra.run.smithi181.stdout: automake-1.16.1-7.el8.noarch 2023-10-09T06:29:38.040 INFO:teuthology.orchestra.run.smithi181.stdout: bison-3.0.4-10.el8.x86_64 2023-10-09T06:29:38.040 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.041 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.041 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.041 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.041 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-09T06:29:38.041 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.041 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.041 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.041 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-09T06:29:38.041 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-09T06:29:38.041 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-09T06:29:38.041 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-09T06:29:38.041 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-09T06:29:38.042 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.042 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.042 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-09T06:29:38.042 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.042 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.042 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.042 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-09T06:29:38.042 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-09T06:29:38.042 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.042 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-09T06:29:38.042 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-09T06:29:38.042 INFO:teuthology.orchestra.run.smithi181.stdout: flex-2.6.1-9.el8.x86_64 2023-10-09T06:29:38.043 INFO:teuthology.orchestra.run.smithi181.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-09T06:29:38.043 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-09T06:29:38.043 INFO:teuthology.orchestra.run.smithi181.stdout: jq-1.5-12.el8.x86_64 2023-10-09T06:29:38.043 INFO:teuthology.orchestra.run.smithi181.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-09T06:29:38.043 INFO:teuthology.orchestra.run.smithi181.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-09T06:29:38.043 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-09T06:29:38.043 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.043 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.043 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.043 INFO:teuthology.orchestra.run.smithi181.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-09T06:29:38.043 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-09T06:29:38.043 INFO:teuthology.orchestra.run.smithi181.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-09T06:29:38.044 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-09T06:29:38.044 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-09T06:29:38.044 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.044 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.044 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-09T06:29:38.044 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.044 INFO:teuthology.orchestra.run.smithi181.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-09T06:29:38.044 INFO:teuthology.orchestra.run.smithi181.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-09T06:29:38.044 INFO:teuthology.orchestra.run.smithi181.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-09T06:29:38.044 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-09T06:29:38.044 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-09T06:29:38.044 INFO:teuthology.orchestra.run.smithi181.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-09T06:29:38.044 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-09T06:29:38.045 INFO:teuthology.orchestra.run.smithi181.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-09T06:29:38.045 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-09T06:29:38.045 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-09T06:29:38.045 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-09T06:29:38.045 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-09T06:29:38.045 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-09T06:29:38.045 INFO:teuthology.orchestra.run.smithi181.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-09T06:29:38.045 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-09T06:29:38.045 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-09T06:29:38.045 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-09T06:29:38.045 INFO:teuthology.orchestra.run.smithi181.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-09T06:29:38.045 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-09T06:29:38.046 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.046 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.046 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.046 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-09T06:29:38.046 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-09T06:29:38.046 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-09T06:29:38.046 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-09T06:29:38.046 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-09T06:29:38.046 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-09T06:29:38.046 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-09T06:29:38.046 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-09T06:29:38.046 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-09T06:29:38.047 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-09T06:29:38.047 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-09T06:29:38.047 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-09T06:29:38.047 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-09T06:29:38.047 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-09T06:29:38.047 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-09T06:29:38.047 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-09T06:29:38.047 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-09T06:29:38.047 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-09T06:29:38.047 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-09T06:29:38.047 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-09T06:29:38.047 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend-2.6-1.el8.noarch 2023-10-09T06:29:38.047 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-09T06:29:38.048 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-09T06:29:38.048 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-09T06:29:38.048 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-09T06:29:38.048 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.048 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.048 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-09T06:29:38.048 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-09T06:29:38.048 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.048 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-09T06:29:38.048 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-09T06:29:38.048 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-09T06:29:38.048 INFO:teuthology.orchestra.run.smithi181.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-09T06:29:38.049 INFO:teuthology.orchestra.run.smithi181.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-09T06:29:38.049 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-09T06:29:38.049 INFO:teuthology.orchestra.run.smithi181.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-09T06:29:38.049 INFO:teuthology.orchestra.run.smithi181.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-09T06:29:38.049 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-09T06:29:38.049 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-09T06:29:38.049 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-09T06:29:38.049 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-09T06:29:38.049 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-09T06:29:38.049 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.049 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.049 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-09T06:29:38.050 INFO:teuthology.orchestra.run.smithi181.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-09T06:29:38.050 INFO:teuthology.orchestra.run.smithi181.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-09T06:29:38.050 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-09T06:29:38.050 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-09T06:29:38.050 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-09T06:29:38.050 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:38.050 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-10-09T06:29:38.394 DEBUG:teuthology.parallel:result is None 2023-10-09T06:29:38.394 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-09T06:29:38.539 DEBUG:teuthology.orchestra.run.smithi163:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-09T06:29:38.565 INFO:teuthology.orchestra.run.smithi163.stdout:17.2.6-1320.g08cac018.el8 2023-10-09T06:29:38.565 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1320.g08cac018.el8 2023-10-09T06:29:38.566 INFO:teuthology.task.install:The correct ceph version 17.2.6-1320.g08cac018 is installed. 2023-10-09T06:29:38.566 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-09T06:29:38.810 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-09T06:29:38.831 INFO:teuthology.orchestra.run.smithi181.stdout:17.2.6-1320.g08cac018.el8 2023-10-09T06:29:38.831 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1320.g08cac018.el8 2023-10-09T06:29:38.832 INFO:teuthology.task.install:The correct ceph version 17.2.6-1320.g08cac018 is installed. 2023-10-09T06:29:38.832 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-09T06:29:38.832 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-09T06:29:38.832 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-09T06:29:38.875 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-09T06:29:38.875 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-09T06:29:38.913 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-09T06:29:38.914 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-09T06:29:38.914 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/daemon-helper 2023-10-09T06:29:38.942 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-09T06:29:39.008 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-09T06:29:39.060 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/daemon-helper 2023-10-09T06:29:39.085 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-09T06:29:39.157 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-09T06:29:39.157 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-09T06:29:39.158 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-09T06:29:39.192 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-09T06:29:39.258 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-09T06:29:39.259 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-09T06:29:39.284 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-09T06:29:39.360 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-09T06:29:39.361 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-09T06:29:39.361 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/stdin-killer 2023-10-09T06:29:39.386 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-09T06:29:39.451 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-09T06:29:39.451 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/stdin-killer 2023-10-09T06:29:39.476 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-09T06:29:39.542 INFO:teuthology.run_tasks:Running task ceph... 2023-10-09T06:29:39.643 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-09T06:29:39.643 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 777 /var/log/ceph 2023-10-09T06:29:39.647 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 777 /var/log/ceph 2023-10-09T06:29:39.671 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-09T06:29:39.671 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-09T06:29:39.712 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-09T06:29:39.736 INFO:tasks.ceph:Creating extra log directories... 2023-10-09T06:29:39.736 DEBUG:teuthology.orchestra.run.smithi163:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-09T06:29:39.777 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-09T06:29:42.350 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-09T06:29:42.350 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-09T06:29:42.350 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-09T06:29:42.351 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-09T06:29:42.351 DEBUG:teuthology.orchestra.run.smithi163:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-09T06:29:42.368 DEBUG:teuthology.orchestra.run.smithi163:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-09T06:29:42.488 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-09T06:29:42.515 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-09T06:29:42.515 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-09T06:29:42.533 DEBUG:teuthology.orchestra.run.smithi181:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-09T06:29:42.656 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-09T06:29:42.690 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-09T06:29:42.690 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-09T06:29:42.690 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2023-10-09_05:36:44-fs-wip-yuri-testing-2023-10-06-0949-quincy-testing-default-smithi/7418377', 'branch': 'wip-yuri-testing-2023-10-06-0949-quincy', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7418377', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'vshankar-2023-10-09_05:36:44-fs-wip-yuri-testing-2023-10-06-0949-quincy-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.4', 'overrides': {'admin_socket': {'branch': 'wip-yuri-testing-2023-10-06-0949-quincy'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing-20231009.053123', 'sha1': '62a27f0d8453e2104f759ed08ba781e24c5ed056'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 30, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 8174, 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing-20231009.053123', 'suite_path': '/home/teuthworker/src/github.com_vshankar_ceph_62a27f0d8453e2104f759ed08ba781e24c5ed056/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/vshankar/ceph.git', 'suite_sha1': '62a27f0d8453e2104f759ed08ba781e24c5ed056', 'targets': {'smithi163.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN9GGg0ezV7qRelQs7pwRE+MZtSu6jseUyPPtZsCZIeNOgmh5+zpfPVNBkjVEpqTmyPSjfzG3lVrFWglAlauRCs=', 'smithi181.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFYiqfrDjs8ACW9S/FUJkeTfDNLufZAP2urZo97qL+CxrOpH2qyVALjYHlaRDJXbPBxrqZmj0/7St9RDFNpBooo='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestVolumes', 'tasks.cephfs.test_volumes.TestSubvolumeGroups', 'tasks.cephfs.test_volumes.TestSubvolumes']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-09_05:36:44', 'tube': 'smithi', 'use_shaman': True, 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-09T06:29:42.691 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-09T06:29:42.694 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-09T06:29:42.740 DEBUG:teuthology.orchestra.run.smithi163:> sudo install -d -m0777 -- /var/run/ceph 2023-10-09T06:29:42.750 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m0777 -- /var/run/ceph 2023-10-09T06:29:42.806 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-09T06:29:42.806 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/scratch_devs of=/dev/stdout 2023-10-09T06:29:42.830 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-09T06:29:42.830 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_1 2023-10-09T06:29:42.886 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-09T06:29:42.886 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-09T06:29:42.886 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 6h/6d Inode: 217845 Links: 1 2023-10-09T06:29:42.887 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-09T06:29:42.887 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2023-10-09T06:29:42.887 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2023-10-09 06:29:26.568539426 +0000 2023-10-09T06:29:42.887 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2023-10-09 06:27:06.795455012 +0000 2023-10-09T06:29:42.887 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2023-10-09 06:27:06.795455012 +0000 2023-10-09T06:29:42.887 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: - 2023-10-09T06:29:42.887 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-09T06:29:42.952 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2023-10-09T06:29:42.952 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2023-10-09T06:29:42.952 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000330745 s, 1.5 MB/s 2023-10-09T06:29:42.953 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-09T06:29:43.010 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_2 2023-10-09T06:29:43.065 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-09T06:29:43.066 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-09T06:29:43.066 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 6h/6d Inode: 221665 Links: 1 2023-10-09T06:29:43.066 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-09T06:29:43.066 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2023-10-09T06:29:43.066 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2023-10-09 06:29:26.568539426 +0000 2023-10-09T06:29:43.066 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2023-10-09 06:27:07.294443609 +0000 2023-10-09T06:29:43.066 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2023-10-09 06:27:07.294443609 +0000 2023-10-09T06:29:43.066 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: - 2023-10-09T06:29:43.066 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-09T06:29:43.129 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2023-10-09T06:29:43.129 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2023-10-09T06:29:43.130 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000232478 s, 2.2 MB/s 2023-10-09T06:29:43.130 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-09T06:29:43.186 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_3 2023-10-09T06:29:43.241 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-09T06:29:43.241 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-09T06:29:43.241 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 6h/6d Inode: 215925 Links: 1 2023-10-09T06:29:43.241 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-09T06:29:43.241 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2023-10-09T06:29:43.241 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2023-10-09 06:29:26.568539426 +0000 2023-10-09T06:29:43.242 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2023-10-09 06:27:07.830431359 +0000 2023-10-09T06:29:43.242 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2023-10-09 06:27:07.830431359 +0000 2023-10-09T06:29:43.242 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: - 2023-10-09T06:29:43.242 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-09T06:29:43.304 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2023-10-09T06:29:43.304 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2023-10-09T06:29:43.304 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000239576 s, 2.1 MB/s 2023-10-09T06:29:43.305 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-09T06:29:43.361 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_4 2023-10-09T06:29:43.416 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-09T06:29:43.416 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-09T06:29:43.416 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 6h/6d Inode: 218042 Links: 1 2023-10-09T06:29:43.416 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-09T06:29:43.416 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2023-10-09T06:29:43.416 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2023-10-09 06:29:26.568539426 +0000 2023-10-09T06:29:43.416 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2023-10-09 06:27:08.322420116 +0000 2023-10-09T06:29:43.416 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2023-10-09 06:27:08.322420116 +0000 2023-10-09T06:29:43.417 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: - 2023-10-09T06:29:43.417 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-09T06:29:43.479 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2023-10-09T06:29:43.479 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2023-10-09T06:29:43.479 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000197059 s, 2.6 MB/s 2023-10-09T06:29:43.480 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-09T06:29:43.535 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-09T06:29:43.536 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-09T06:29:43.536 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/scratch_devs of=/dev/stdout 2023-10-09T06:29:43.552 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-09T06:29:43.552 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_1 2023-10-09T06:29:43.608 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-09T06:29:43.608 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-09T06:29:43.609 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 218765 Links: 1 2023-10-09T06:29:43.609 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-09T06:29:43.609 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-10-09T06:29:43.609 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-10-09 06:29:33.093863387 +0000 2023-10-09T06:29:43.609 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-10-09 06:27:06.807137011 +0000 2023-10-09T06:29:43.609 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-10-09 06:27:06.807137011 +0000 2023-10-09T06:29:43.609 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2023-10-09T06:29:43.609 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-09T06:29:43.673 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-10-09T06:29:43.674 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-10-09T06:29:43.674 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000240694 s, 2.1 MB/s 2023-10-09T06:29:43.675 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-09T06:29:43.731 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_2 2023-10-09T06:29:43.787 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-09T06:29:43.787 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-09T06:29:43.787 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 218800 Links: 1 2023-10-09T06:29:43.788 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-09T06:29:43.788 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-10-09T06:29:43.788 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-10-09 06:29:33.093863387 +0000 2023-10-09T06:29:43.788 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-10-09 06:27:07.299127199 +0000 2023-10-09T06:29:43.789 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-10-09 06:27:07.299127199 +0000 2023-10-09T06:29:43.789 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2023-10-09T06:29:43.789 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-09T06:29:43.850 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-10-09T06:29:43.850 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-10-09T06:29:43.850 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000255214 s, 2.0 MB/s 2023-10-09T06:29:43.851 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-09T06:29:43.907 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_3 2023-10-09T06:29:43.962 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-09T06:29:43.962 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-09T06:29:43.963 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 220674 Links: 1 2023-10-09T06:29:43.963 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-09T06:29:43.963 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-10-09T06:29:43.963 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-10-09 06:29:33.093863387 +0000 2023-10-09T06:29:43.963 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-10-09 06:27:07.840116410 +0000 2023-10-09T06:29:43.963 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-10-09 06:27:07.840116410 +0000 2023-10-09T06:29:43.963 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2023-10-09T06:29:43.963 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-09T06:29:44.026 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-10-09T06:29:44.026 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-10-09T06:29:44.026 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000241873 s, 2.1 MB/s 2023-10-09T06:29:44.027 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-09T06:29:44.083 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_4 2023-10-09T06:29:44.138 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-09T06:29:44.138 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-09T06:29:44.138 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 222587 Links: 1 2023-10-09T06:29:44.138 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-09T06:29:44.139 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-10-09T06:29:44.139 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-10-09 06:29:33.094863367 +0000 2023-10-09T06:29:44.139 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-10-09 06:27:08.342106398 +0000 2023-10-09T06:29:44.139 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-10-09 06:27:08.342106398 +0000 2023-10-09T06:29:44.139 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2023-10-09T06:29:44.139 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-09T06:29:44.201 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-10-09T06:29:44.201 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-10-09T06:29:44.201 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000195861 s, 2.6 MB/s 2023-10-09T06:29:44.202 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-09T06:29:44.258 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-09T06:29:44.258 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi163.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi181.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-09T06:29:44.258 INFO:tasks.ceph:Generating config... 2023-10-09T06:29:44.259 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-09T06:29:44.259 INFO:tasks.ceph:[client] debug client = 20 2023-10-09T06:29:44.259 INFO:tasks.ceph:[client] debug ms = 1 2023-10-09T06:29:44.259 INFO:tasks.ceph:[client] pid file = 2023-10-09T06:29:44.260 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-09T06:29:44.260 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-09T06:29:44.260 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-10-09T06:29:44.260 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-09T06:29:44.260 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-09T06:29:44.260 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-09T06:29:44.260 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-09T06:29:44.260 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-09T06:29:44.260 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-09T06:29:44.260 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-09T06:29:44.260 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-09T06:29:44.260 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-09T06:29:44.261 INFO:tasks.ceph:[mgr] debug client = 20 2023-10-09T06:29:44.261 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-10-09T06:29:44.261 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-09T06:29:44.261 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-09T06:29:44.261 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-09T06:29:44.261 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-09T06:29:44.261 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-09T06:29:44.261 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-09T06:29:44.261 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-09T06:29:44.261 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-09T06:29:44.261 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-09T06:29:44.262 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-09T06:29:44.262 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-09T06:29:44.262 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-09T06:29:44.262 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-09T06:29:44.265 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-09T06:29:44.265 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-09T06:29:44.265 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-09T06:29:44.265 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-09T06:29:44.265 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-09T06:29:44.265 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-09T06:29:44.265 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-09T06:29:44.266 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-09T06:29:44.266 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-09T06:29:44.266 INFO:tasks.ceph:Setting up mon.a... 2023-10-09T06:29:44.266 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-09T06:29:44.317 INFO:teuthology.orchestra.run.smithi163.stdout:creating /etc/ceph/ceph.keyring 2023-10-09T06:29:44.319 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-09T06:29:44.359 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-09T06:29:44.424 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.163'), ('mon.b', '172.21.15.181'), ('mon.c', '[v2:172.21.15.181:3301,v1:172.21.15.181:6790]')] 2023-10-09T06:29:44.424 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.163,172.21.15.181,[v2:172.21.15.181:3301,v1:172.21.15.181:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-09T06:29:44.425 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-09T06:29:44.425 DEBUG:teuthology.orchestra.run.smithi163:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-09T06:29:44.478 DEBUG:teuthology.orchestra.run.smithi163:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.163 --add b 172.21.15.181 --addv c '[v2:172.21.15.181:3301,v1:172.21.15.181:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-09T06:29:44.560 INFO:teuthology.orchestra.run.smithi163.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-09T06:29:44.560 INFO:teuthology.orchestra.run.smithi163.stdout:monmaptool: generated fsid 2520b8f7-539f-4bae-9f17-1fce66f66ddd 2023-10-09T06:29:44.560 INFO:teuthology.orchestra.run.smithi163.stdout:setting min_mon_release = octopus 2023-10-09T06:29:44.561 INFO:teuthology.orchestra.run.smithi163.stdout:epoch 0 2023-10-09T06:29:44.561 INFO:teuthology.orchestra.run.smithi163.stdout:fsid 2520b8f7-539f-4bae-9f17-1fce66f66ddd 2023-10-09T06:29:44.561 INFO:teuthology.orchestra.run.smithi163.stdout:last_changed 2023-10-09T06:29:44.560991+0000 2023-10-09T06:29:44.561 INFO:teuthology.orchestra.run.smithi163.stdout:created 2023-10-09T06:29:44.560991+0000 2023-10-09T06:29:44.561 INFO:teuthology.orchestra.run.smithi163.stdout:min_mon_release 15 (octopus) 2023-10-09T06:29:44.561 INFO:teuthology.orchestra.run.smithi163.stdout:election_strategy: 1 2023-10-09T06:29:44.561 INFO:teuthology.orchestra.run.smithi163.stdout:0: [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon.a 2023-10-09T06:29:44.561 INFO:teuthology.orchestra.run.smithi163.stdout:1: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.b 2023-10-09T06:29:44.561 INFO:teuthology.orchestra.run.smithi163.stdout:2: [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] mon.c 2023-10-09T06:29:44.561 INFO:teuthology.orchestra.run.smithi163.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-09T06:29:44.562 DEBUG:teuthology.orchestra.run.smithi163:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-09T06:29:44.617 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 2520b8f7-539f-4bae-9f17-1fce66f66ddd... 2023-10-09T06:29:44.618 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-09T06:29:44.659 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-09T06:29:44.699 INFO:teuthology.orchestra.run.smithi163.stdout:[global] 2023-10-09T06:29:44.700 INFO:teuthology.orchestra.run.smithi163.stdout: chdir = "" 2023-10-09T06:29:44.700 INFO:teuthology.orchestra.run.smithi163.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-09T06:29:44.700 INFO:teuthology.orchestra.run.smithi163.stdout: auth supported = cephx 2023-10-09T06:29:44.700 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:44.700 INFO:teuthology.orchestra.run.smithi163.stdout: filestore xattr use omap = true 2023-10-09T06:29:44.700 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:44.700 INFO:teuthology.orchestra.run.smithi163.stdout: mon clock drift allowed = 1.000 2023-10-09T06:29:44.700 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:44.700 INFO:teuthology.orchestra.run.smithi163.stdout: osd crush chooseleaf type = 0 2023-10-09T06:29:44.700 INFO:teuthology.orchestra.run.smithi163.stdout: auth debug = true 2023-10-09T06:29:44.700 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:44.701 INFO:teuthology.orchestra.run.smithi163.stdout: ms die on old message = true 2023-10-09T06:29:44.701 INFO:teuthology.orchestra.run.smithi163.stdout: ms die on bug = true 2023-10-09T06:29:44.701 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:44.701 INFO:teuthology.orchestra.run.smithi163.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-09T06:29:44.701 INFO:teuthology.orchestra.run.smithi163.stdout: mon pg warn max object skew = 0 2023-10-09T06:29:44.701 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:44.701 INFO:teuthology.orchestra.run.smithi163.stdout: # disable pg_autoscaler by default for new pools 2023-10-09T06:29:44.701 INFO:teuthology.orchestra.run.smithi163.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-09T06:29:44.701 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:44.701 INFO:teuthology.orchestra.run.smithi163.stdout: osd pool default size = 2 2023-10-09T06:29:44.701 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:44.701 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd allow primary affinity = true 2023-10-09T06:29:44.702 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd allow pg remap = true 2023-10-09T06:29:44.702 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on legacy crush tunables = false 2023-10-09T06:29:44.702 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on crush straw calc version zero = false 2023-10-09T06:29:44.702 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on no sortbitwise = false 2023-10-09T06:29:44.702 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on osd down out interval zero = false 2023-10-09T06:29:44.702 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on too few osds = false 2023-10-09T06:29:44.702 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-09T06:29:44.702 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-09T06:29:44.702 INFO:teuthology.orchestra.run.smithi163.stdout: mon_allow_pool_size_one = true 2023-10-09T06:29:44.702 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:44.702 INFO:teuthology.orchestra.run.smithi163.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-09T06:29:44.703 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:44.703 INFO:teuthology.orchestra.run.smithi163.stdout: osd default data pool replay window = 5 2023-10-09T06:29:44.703 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:44.703 INFO:teuthology.orchestra.run.smithi163.stdout: mon allow pool delete = true 2023-10-09T06:29:44.703 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:44.703 INFO:teuthology.orchestra.run.smithi163.stdout: mon cluster log file level = debug 2023-10-09T06:29:44.703 INFO:teuthology.orchestra.run.smithi163.stdout: debug asserts on shutdown = true 2023-10-09T06:29:44.703 INFO:teuthology.orchestra.run.smithi163.stdout: mon health detail to clog = false 2023-10-09T06:29:44.703 INFO:teuthology.orchestra.run.smithi163.stdout: mon host = "172.21.15.163,172.21.15.181,[v2:172.21.15.181:3301,v1:172.21.15.181:6790]" 2023-10-09T06:29:44.703 INFO:teuthology.orchestra.run.smithi163.stdout: ms die on skipped message = False 2023-10-09T06:29:44.703 INFO:teuthology.orchestra.run.smithi163.stdout: fsid = 2520b8f7-539f-4bae-9f17-1fce66f66ddd 2023-10-09T06:29:44.704 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:44.704 INFO:teuthology.orchestra.run.smithi163.stdout:[osd] 2023-10-09T06:29:44.704 INFO:teuthology.orchestra.run.smithi163.stdout: osd journal size = 100 2023-10-09T06:29:44.704 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:44.704 INFO:teuthology.orchestra.run.smithi163.stdout: osd scrub load threshold = 5.0 2023-10-09T06:29:44.704 INFO:teuthology.orchestra.run.smithi163.stdout: osd scrub max interval = 600 2023-10-09T06:29:44.704 INFO:teuthology.orchestra.run.smithi163.stdout: osd mclock profile = high_recovery_ops 2023-10-09T06:29:44.704 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:44.704 INFO:teuthology.orchestra.run.smithi163.stdout: osd recover clone overlap = true 2023-10-09T06:29:44.704 INFO:teuthology.orchestra.run.smithi163.stdout: osd recovery max chunk = 1048576 2023-10-09T06:29:44.704 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:44.704 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug shutdown = true 2023-10-09T06:29:44.705 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug op order = true 2023-10-09T06:29:44.705 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug verify stray on activate = true 2023-10-09T06:29:44.705 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:44.705 INFO:teuthology.orchestra.run.smithi163.stdout: osd open classes on start = true 2023-10-09T06:29:44.705 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug pg log writeout = true 2023-10-09T06:29:44.705 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:44.705 INFO:teuthology.orchestra.run.smithi163.stdout: osd deep scrub update digest min age = 30 2023-10-09T06:29:44.705 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:44.705 INFO:teuthology.orchestra.run.smithi163.stdout: osd map max advance = 10 2023-10-09T06:29:44.705 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:44.705 INFO:teuthology.orchestra.run.smithi163.stdout: journal zero on create = true 2023-10-09T06:29:44.705 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:44.706 INFO:teuthology.orchestra.run.smithi163.stdout: filestore ondisk finisher threads = 3 2023-10-09T06:29:44.706 INFO:teuthology.orchestra.run.smithi163.stdout: filestore apply finisher threads = 3 2023-10-09T06:29:44.706 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:44.706 INFO:teuthology.orchestra.run.smithi163.stdout: bdev debug aio = true 2023-10-09T06:29:44.706 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug misdirected ops = true 2023-10-09T06:29:44.706 INFO:teuthology.orchestra.run.smithi163.stdout: bdev async discard = True 2023-10-09T06:29:44.706 INFO:teuthology.orchestra.run.smithi163.stdout: bdev enable discard = True 2023-10-09T06:29:44.706 INFO:teuthology.orchestra.run.smithi163.stdout: bluestore allocator = bitmap 2023-10-09T06:29:44.706 INFO:teuthology.orchestra.run.smithi163.stdout: bluestore block size = 96636764160 2023-10-09T06:29:44.706 INFO:teuthology.orchestra.run.smithi163.stdout: bluestore fsck on mount = True 2023-10-09T06:29:44.706 INFO:teuthology.orchestra.run.smithi163.stdout: debug bluefs = 1/20 2023-10-09T06:29:44.707 INFO:teuthology.orchestra.run.smithi163.stdout: debug bluestore = 1/20 2023-10-09T06:29:44.707 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2023-10-09T06:29:44.707 INFO:teuthology.orchestra.run.smithi163.stdout: debug osd = 20 2023-10-09T06:29:44.707 INFO:teuthology.orchestra.run.smithi163.stdout: debug rocksdb = 4/10 2023-10-09T06:29:44.707 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd backfillfull_ratio = 0.85 2023-10-09T06:29:44.707 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd full ratio = 0.9 2023-10-09T06:29:44.707 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd nearfull ratio = 0.8 2023-10-09T06:29:44.707 INFO:teuthology.orchestra.run.smithi163.stdout: osd failsafe full ratio = 0.95 2023-10-09T06:29:44.707 INFO:teuthology.orchestra.run.smithi163.stdout: osd objectstore = bluestore 2023-10-09T06:29:44.707 INFO:teuthology.orchestra.run.smithi163.stdout: osd op complaint time = 180 2023-10-09T06:29:44.707 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:44.707 INFO:teuthology.orchestra.run.smithi163.stdout:[mgr] 2023-10-09T06:29:44.708 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2023-10-09T06:29:44.708 INFO:teuthology.orchestra.run.smithi163.stdout: debug mgr = 20 2023-10-09T06:29:44.708 INFO:teuthology.orchestra.run.smithi163.stdout: debug mon = 20 2023-10-09T06:29:44.708 INFO:teuthology.orchestra.run.smithi163.stdout: debug auth = 20 2023-10-09T06:29:44.708 INFO:teuthology.orchestra.run.smithi163.stdout: mon reweight min pgs per osd = 4 2023-10-09T06:29:44.708 INFO:teuthology.orchestra.run.smithi163.stdout: mon reweight min bytes per osd = 10 2023-10-09T06:29:44.708 INFO:teuthology.orchestra.run.smithi163.stdout: mgr/telemetry/nag = false 2023-10-09T06:29:44.708 INFO:teuthology.orchestra.run.smithi163.stdout: debug client = 20 2023-10-09T06:29:44.708 INFO:teuthology.orchestra.run.smithi163.stdout: debug finisher = 20 2023-10-09T06:29:44.709 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:44.709 INFO:teuthology.orchestra.run.smithi163.stdout:[mon] 2023-10-09T06:29:44.709 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2023-10-09T06:29:44.709 INFO:teuthology.orchestra.run.smithi163.stdout: debug mon = 20 2023-10-09T06:29:44.709 INFO:teuthology.orchestra.run.smithi163.stdout: debug paxos = 20 2023-10-09T06:29:44.709 INFO:teuthology.orchestra.run.smithi163.stdout: debug auth = 20 2023-10-09T06:29:44.709 INFO:teuthology.orchestra.run.smithi163.stdout: mon data avail warn = 5 2023-10-09T06:29:44.709 INFO:teuthology.orchestra.run.smithi163.stdout: mon mgr mkfs grace = 240 2023-10-09T06:29:44.709 INFO:teuthology.orchestra.run.smithi163.stdout: mon reweight min pgs per osd = 4 2023-10-09T06:29:44.709 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd reporter subtree level = osd 2023-10-09T06:29:44.709 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd prime pg temp = true 2023-10-09T06:29:44.709 INFO:teuthology.orchestra.run.smithi163.stdout: mon reweight min bytes per osd = 10 2023-10-09T06:29:44.710 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:44.710 INFO:teuthology.orchestra.run.smithi163.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-09T06:29:44.710 INFO:teuthology.orchestra.run.smithi163.stdout: auth mon ticket ttl = 660 # 11m 2023-10-09T06:29:44.710 INFO:teuthology.orchestra.run.smithi163.stdout: auth service ticket ttl = 240 # 4m 2023-10-09T06:29:44.710 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:44.710 INFO:teuthology.orchestra.run.smithi163.stdout: # don't complain about insecure global_id in the test suite 2023-10-09T06:29:44.710 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-09T06:29:44.710 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-09T06:29:44.711 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:44.711 INFO:teuthology.orchestra.run.smithi163.stdout: # 1m isn't quite enough 2023-10-09T06:29:44.711 INFO:teuthology.orchestra.run.smithi163.stdout: mon_down_mkfs_grace = 2m 2023-10-09T06:29:44.711 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:44.711 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_filestore_osds = false 2023-10-09T06:29:44.711 INFO:teuthology.orchestra.run.smithi163.stdout: mon op complaint time = 120 2023-10-09T06:29:44.711 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:29:44.711 INFO:teuthology.orchestra.run.smithi163.stdout:[client] 2023-10-09T06:29:44.711 INFO:teuthology.orchestra.run.smithi163.stdout: rgw cache enabled = true 2023-10-09T06:29:44.711 INFO:teuthology.orchestra.run.smithi163.stdout: rgw enable ops log = true 2023-10-09T06:29:44.711 INFO:teuthology.orchestra.run.smithi163.stdout: rgw enable usage log = true 2023-10-09T06:29:44.711 INFO:teuthology.orchestra.run.smithi163.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-09T06:29:44.712 INFO:teuthology.orchestra.run.smithi163.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-09T06:29:44.712 INFO:teuthology.orchestra.run.smithi163.stdout: client mount timeout = 600 2023-10-09T06:29:44.712 INFO:teuthology.orchestra.run.smithi163.stdout: debug client = 20 2023-10-09T06:29:44.712 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2023-10-09T06:29:44.712 INFO:teuthology.orchestra.run.smithi163.stdout: pid file = "" 2023-10-09T06:29:44.712 INFO:teuthology.orchestra.run.smithi163.stdout: rados mon op timeout = 900 2023-10-09T06:29:44.712 INFO:teuthology.orchestra.run.smithi163.stdout: rados osd op timeout = 900 2023-10-09T06:29:44.712 INFO:teuthology.orchestra.run.smithi163.stdout:[mon.a] 2023-10-09T06:29:44.712 INFO:teuthology.orchestra.run.smithi163.stdout:[mon.b] 2023-10-09T06:29:44.712 INFO:teuthology.orchestra.run.smithi163.stdout:[mon.c] 2023-10-09T06:29:44.712 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.a] 2023-10-09T06:29:44.712 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.c] 2023-10-09T06:29:44.713 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.b] 2023-10-09T06:29:44.713 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.d] 2023-10-09T06:29:44.713 INFO:teuthology.orchestra.run.smithi163.stdout:[mds] 2023-10-09T06:29:44.713 INFO:teuthology.orchestra.run.smithi163.stdout: debug mds = 20 2023-10-09T06:29:44.713 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2023-10-09T06:29:44.713 INFO:teuthology.orchestra.run.smithi163.stdout: mds debug frag = True 2023-10-09T06:29:44.713 INFO:teuthology.orchestra.run.smithi163.stdout: mds debug scatterstat = True 2023-10-09T06:29:44.713 INFO:teuthology.orchestra.run.smithi163.stdout: mds op complaint time = 180 2023-10-09T06:29:44.713 INFO:teuthology.orchestra.run.smithi163.stdout: mds verify scatter = True 2023-10-09T06:29:44.713 INFO:teuthology.orchestra.run.smithi163.stdout: osd op complaint time = 180 2023-10-09T06:29:44.713 INFO:teuthology.orchestra.run.smithi163.stdout: rados mon op timeout = 900 2023-10-09T06:29:44.713 INFO:teuthology.orchestra.run.smithi163.stdout: rados osd op timeout = 900 2023-10-09T06:29:44.715 INFO:teuthology.orchestra.run.smithi181.stdout:[global] 2023-10-09T06:29:44.715 INFO:teuthology.orchestra.run.smithi181.stdout: chdir = "" 2023-10-09T06:29:44.715 INFO:teuthology.orchestra.run.smithi181.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-09T06:29:44.715 INFO:teuthology.orchestra.run.smithi181.stdout: auth supported = cephx 2023-10-09T06:29:44.715 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:44.715 INFO:teuthology.orchestra.run.smithi181.stdout: filestore xattr use omap = true 2023-10-09T06:29:44.715 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:44.716 INFO:teuthology.orchestra.run.smithi181.stdout: mon clock drift allowed = 1.000 2023-10-09T06:29:44.716 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:44.716 INFO:teuthology.orchestra.run.smithi181.stdout: osd crush chooseleaf type = 0 2023-10-09T06:29:44.716 INFO:teuthology.orchestra.run.smithi181.stdout: auth debug = true 2023-10-09T06:29:44.716 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:44.716 INFO:teuthology.orchestra.run.smithi181.stdout: ms die on old message = true 2023-10-09T06:29:44.716 INFO:teuthology.orchestra.run.smithi181.stdout: ms die on bug = true 2023-10-09T06:29:44.716 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:44.716 INFO:teuthology.orchestra.run.smithi181.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-09T06:29:44.716 INFO:teuthology.orchestra.run.smithi181.stdout: mon pg warn max object skew = 0 2023-10-09T06:29:44.716 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:44.717 INFO:teuthology.orchestra.run.smithi181.stdout: # disable pg_autoscaler by default for new pools 2023-10-09T06:29:44.717 INFO:teuthology.orchestra.run.smithi181.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-09T06:29:44.717 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:44.717 INFO:teuthology.orchestra.run.smithi181.stdout: osd pool default size = 2 2023-10-09T06:29:44.717 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:44.717 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd allow primary affinity = true 2023-10-09T06:29:44.717 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd allow pg remap = true 2023-10-09T06:29:44.717 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on legacy crush tunables = false 2023-10-09T06:29:44.717 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on crush straw calc version zero = false 2023-10-09T06:29:44.717 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on no sortbitwise = false 2023-10-09T06:29:44.717 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on osd down out interval zero = false 2023-10-09T06:29:44.717 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on too few osds = false 2023-10-09T06:29:44.718 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-09T06:29:44.718 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-09T06:29:44.718 INFO:teuthology.orchestra.run.smithi181.stdout: mon_allow_pool_size_one = true 2023-10-09T06:29:44.718 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:44.718 INFO:teuthology.orchestra.run.smithi181.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-09T06:29:44.718 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:44.718 INFO:teuthology.orchestra.run.smithi181.stdout: osd default data pool replay window = 5 2023-10-09T06:29:44.718 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:44.719 INFO:teuthology.orchestra.run.smithi181.stdout: mon allow pool delete = true 2023-10-09T06:29:44.719 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:44.719 INFO:teuthology.orchestra.run.smithi181.stdout: mon cluster log file level = debug 2023-10-09T06:29:44.719 INFO:teuthology.orchestra.run.smithi181.stdout: debug asserts on shutdown = true 2023-10-09T06:29:44.719 INFO:teuthology.orchestra.run.smithi181.stdout: mon health detail to clog = false 2023-10-09T06:29:44.719 INFO:teuthology.orchestra.run.smithi181.stdout: mon host = "172.21.15.163,172.21.15.181,[v2:172.21.15.181:3301,v1:172.21.15.181:6790]" 2023-10-09T06:29:44.719 INFO:teuthology.orchestra.run.smithi181.stdout: ms die on skipped message = False 2023-10-09T06:29:44.719 INFO:teuthology.orchestra.run.smithi181.stdout: fsid = 2520b8f7-539f-4bae-9f17-1fce66f66ddd 2023-10-09T06:29:44.719 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:44.720 INFO:teuthology.orchestra.run.smithi181.stdout:[osd] 2023-10-09T06:29:44.720 INFO:teuthology.orchestra.run.smithi181.stdout: osd journal size = 100 2023-10-09T06:29:44.720 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:44.720 INFO:teuthology.orchestra.run.smithi181.stdout: osd scrub load threshold = 5.0 2023-10-09T06:29:44.720 INFO:teuthology.orchestra.run.smithi181.stdout: osd scrub max interval = 600 2023-10-09T06:29:44.720 INFO:teuthology.orchestra.run.smithi181.stdout: osd mclock profile = high_recovery_ops 2023-10-09T06:29:44.720 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:44.720 INFO:teuthology.orchestra.run.smithi181.stdout: osd recover clone overlap = true 2023-10-09T06:29:44.720 INFO:teuthology.orchestra.run.smithi181.stdout: osd recovery max chunk = 1048576 2023-10-09T06:29:44.720 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:44.720 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug shutdown = true 2023-10-09T06:29:44.720 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug op order = true 2023-10-09T06:29:44.721 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug verify stray on activate = true 2023-10-09T06:29:44.721 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:44.721 INFO:teuthology.orchestra.run.smithi181.stdout: osd open classes on start = true 2023-10-09T06:29:44.721 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug pg log writeout = true 2023-10-09T06:29:44.721 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:44.721 INFO:teuthology.orchestra.run.smithi181.stdout: osd deep scrub update digest min age = 30 2023-10-09T06:29:44.721 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:44.721 INFO:teuthology.orchestra.run.smithi181.stdout: osd map max advance = 10 2023-10-09T06:29:44.721 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:44.721 INFO:teuthology.orchestra.run.smithi181.stdout: journal zero on create = true 2023-10-09T06:29:44.721 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:44.721 INFO:teuthology.orchestra.run.smithi181.stdout: filestore ondisk finisher threads = 3 2023-10-09T06:29:44.722 INFO:teuthology.orchestra.run.smithi181.stdout: filestore apply finisher threads = 3 2023-10-09T06:29:44.722 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:44.722 INFO:teuthology.orchestra.run.smithi181.stdout: bdev debug aio = true 2023-10-09T06:29:44.722 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug misdirected ops = true 2023-10-09T06:29:44.722 INFO:teuthology.orchestra.run.smithi181.stdout: bdev async discard = True 2023-10-09T06:29:44.722 INFO:teuthology.orchestra.run.smithi181.stdout: bdev enable discard = True 2023-10-09T06:29:44.722 INFO:teuthology.orchestra.run.smithi181.stdout: bluestore allocator = bitmap 2023-10-09T06:29:44.722 INFO:teuthology.orchestra.run.smithi181.stdout: bluestore block size = 96636764160 2023-10-09T06:29:44.722 INFO:teuthology.orchestra.run.smithi181.stdout: bluestore fsck on mount = True 2023-10-09T06:29:44.722 INFO:teuthology.orchestra.run.smithi181.stdout: debug bluefs = 1/20 2023-10-09T06:29:44.722 INFO:teuthology.orchestra.run.smithi181.stdout: debug bluestore = 1/20 2023-10-09T06:29:44.723 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2023-10-09T06:29:44.723 INFO:teuthology.orchestra.run.smithi181.stdout: debug osd = 20 2023-10-09T06:29:44.723 INFO:teuthology.orchestra.run.smithi181.stdout: debug rocksdb = 4/10 2023-10-09T06:29:44.723 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd backfillfull_ratio = 0.85 2023-10-09T06:29:44.723 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd full ratio = 0.9 2023-10-09T06:29:44.723 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd nearfull ratio = 0.8 2023-10-09T06:29:44.723 INFO:teuthology.orchestra.run.smithi181.stdout: osd failsafe full ratio = 0.95 2023-10-09T06:29:44.723 INFO:teuthology.orchestra.run.smithi181.stdout: osd objectstore = bluestore 2023-10-09T06:29:44.723 INFO:teuthology.orchestra.run.smithi181.stdout: osd op complaint time = 180 2023-10-09T06:29:44.723 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:44.723 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr] 2023-10-09T06:29:44.723 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2023-10-09T06:29:44.724 INFO:teuthology.orchestra.run.smithi181.stdout: debug mgr = 20 2023-10-09T06:29:44.724 INFO:teuthology.orchestra.run.smithi181.stdout: debug mon = 20 2023-10-09T06:29:44.724 INFO:teuthology.orchestra.run.smithi181.stdout: debug auth = 20 2023-10-09T06:29:44.724 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min pgs per osd = 4 2023-10-09T06:29:44.724 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min bytes per osd = 10 2023-10-09T06:29:44.724 INFO:teuthology.orchestra.run.smithi181.stdout: mgr/telemetry/nag = false 2023-10-09T06:29:44.724 INFO:teuthology.orchestra.run.smithi181.stdout: debug client = 20 2023-10-09T06:29:44.724 INFO:teuthology.orchestra.run.smithi181.stdout: debug finisher = 20 2023-10-09T06:29:44.724 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:44.724 INFO:teuthology.orchestra.run.smithi181.stdout:[mon] 2023-10-09T06:29:44.724 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2023-10-09T06:29:44.724 INFO:teuthology.orchestra.run.smithi181.stdout: debug mon = 20 2023-10-09T06:29:44.725 INFO:teuthology.orchestra.run.smithi181.stdout: debug paxos = 20 2023-10-09T06:29:44.725 INFO:teuthology.orchestra.run.smithi181.stdout: debug auth = 20 2023-10-09T06:29:44.725 INFO:teuthology.orchestra.run.smithi181.stdout: mon data avail warn = 5 2023-10-09T06:29:44.725 INFO:teuthology.orchestra.run.smithi181.stdout: mon mgr mkfs grace = 240 2023-10-09T06:29:44.725 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min pgs per osd = 4 2023-10-09T06:29:44.725 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd reporter subtree level = osd 2023-10-09T06:29:44.725 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd prime pg temp = true 2023-10-09T06:29:44.725 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min bytes per osd = 10 2023-10-09T06:29:44.725 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:44.725 INFO:teuthology.orchestra.run.smithi181.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-09T06:29:44.725 INFO:teuthology.orchestra.run.smithi181.stdout: auth mon ticket ttl = 660 # 11m 2023-10-09T06:29:44.725 INFO:teuthology.orchestra.run.smithi181.stdout: auth service ticket ttl = 240 # 4m 2023-10-09T06:29:44.726 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:44.726 INFO:teuthology.orchestra.run.smithi181.stdout: # don't complain about insecure global_id in the test suite 2023-10-09T06:29:44.726 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-09T06:29:44.726 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-09T06:29:44.726 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:44.726 INFO:teuthology.orchestra.run.smithi181.stdout: # 1m isn't quite enough 2023-10-09T06:29:44.726 INFO:teuthology.orchestra.run.smithi181.stdout: mon_down_mkfs_grace = 2m 2023-10-09T06:29:44.726 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:44.726 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_filestore_osds = false 2023-10-09T06:29:44.726 INFO:teuthology.orchestra.run.smithi181.stdout: mon op complaint time = 120 2023-10-09T06:29:44.726 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-09T06:29:44.726 INFO:teuthology.orchestra.run.smithi181.stdout:[client] 2023-10-09T06:29:44.727 INFO:teuthology.orchestra.run.smithi181.stdout: rgw cache enabled = true 2023-10-09T06:29:44.727 INFO:teuthology.orchestra.run.smithi181.stdout: rgw enable ops log = true 2023-10-09T06:29:44.727 INFO:teuthology.orchestra.run.smithi181.stdout: rgw enable usage log = true 2023-10-09T06:29:44.727 INFO:teuthology.orchestra.run.smithi181.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-09T06:29:44.727 INFO:teuthology.orchestra.run.smithi181.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-09T06:29:44.727 INFO:teuthology.orchestra.run.smithi181.stdout: client mount timeout = 600 2023-10-09T06:29:44.727 INFO:teuthology.orchestra.run.smithi181.stdout: debug client = 20 2023-10-09T06:29:44.727 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2023-10-09T06:29:44.727 INFO:teuthology.orchestra.run.smithi181.stdout: pid file = "" 2023-10-09T06:29:44.727 INFO:teuthology.orchestra.run.smithi181.stdout: rados mon op timeout = 900 2023-10-09T06:29:44.727 INFO:teuthology.orchestra.run.smithi181.stdout: rados osd op timeout = 900 2023-10-09T06:29:44.727 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.a] 2023-10-09T06:29:44.728 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.b] 2023-10-09T06:29:44.728 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.c] 2023-10-09T06:29:44.728 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.a] 2023-10-09T06:29:44.728 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.c] 2023-10-09T06:29:44.728 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.b] 2023-10-09T06:29:44.728 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.d] 2023-10-09T06:29:44.728 INFO:teuthology.orchestra.run.smithi181.stdout:[mds] 2023-10-09T06:29:44.728 INFO:teuthology.orchestra.run.smithi181.stdout: debug mds = 20 2023-10-09T06:29:44.728 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2023-10-09T06:29:44.728 INFO:teuthology.orchestra.run.smithi181.stdout: mds debug frag = True 2023-10-09T06:29:44.729 INFO:teuthology.orchestra.run.smithi181.stdout: mds debug scatterstat = True 2023-10-09T06:29:44.729 INFO:teuthology.orchestra.run.smithi181.stdout: mds op complaint time = 180 2023-10-09T06:29:44.729 INFO:teuthology.orchestra.run.smithi181.stdout: mds verify scatter = True 2023-10-09T06:29:44.729 INFO:teuthology.orchestra.run.smithi181.stdout: osd op complaint time = 180 2023-10-09T06:29:44.729 INFO:teuthology.orchestra.run.smithi181.stdout: rados mon op timeout = 900 2023-10-09T06:29:44.729 INFO:teuthology.orchestra.run.smithi181.stdout: rados osd op timeout = 900 2023-10-09T06:29:44.729 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-09T06:29:44.729 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-09T06:29:44.806 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-09T06:29:44.806 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-09T06:29:44.807 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-09T06:29:44.821 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-09T06:29:44.821 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-09T06:29:44.876 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi163.front.sepia.ceph.com 2023-10-09T06:29:44.876 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-09T06:29:44.876 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-09T06:29:44.876 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-09T06:29:44.950 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-09T06:29:44.950 DEBUG:teuthology.orchestra.run.smithi163:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-09T06:29:45.005 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi181.front.sepia.ceph.com 2023-10-09T06:29:45.005 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-09T06:29:45.005 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-09T06:29:45.005 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-09T06:29:45.041 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-09T06:29:45.041 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-09T06:29:45.095 INFO:tasks.ceph:Setting up mon nodes... 2023-10-09T06:29:45.095 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-09T06:29:45.096 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-09T06:29:45.143 INFO:teuthology.orchestra.run.smithi163.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-09T06:29:45.145 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-09T06:29:45.207 INFO:teuthology.orchestra.run.smithi181.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-09T06:29:45.210 INFO:tasks.ceph:Setting up mds nodes... 2023-10-09T06:29:45.210 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-09T06:29:45.258 INFO:teuthology.orchestra.run.smithi163.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-09T06:29:45.260 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-09T06:29:45.284 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-09T06:29:45.372 INFO:teuthology.orchestra.run.smithi163.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-09T06:29:45.374 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-09T06:29:45.398 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-09T06:29:45.446 INFO:teuthology.orchestra.run.smithi181.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-09T06:29:45.448 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-09T06:29:45.472 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-09T06:29:45.560 INFO:teuthology.orchestra.run.smithi181.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-09T06:29:45.562 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-09T06:29:45.586 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-09T06:29:45.586 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-09T06:29:45.623 INFO:teuthology.orchestra.run.smithi163.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-09T06:29:45.636 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-09T06:29:45.715 INFO:teuthology.orchestra.run.smithi163.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-09T06:29:45.727 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-09T06:29:45.764 INFO:teuthology.orchestra.run.smithi181.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-09T06:29:45.776 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-09T06:29:45.854 INFO:teuthology.orchestra.run.smithi181.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-09T06:29:45.866 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-09T06:29:45.866 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi163.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi181.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-09T06:29:45.867 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-09T06:29:45.890 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-09T06:29:45.890 INFO:tasks.ceph:role: osd.0 2023-10-09T06:29:45.890 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi163.front.sepia.ceph.com 2023-10-09T06:29:45.890 DEBUG:teuthology.orchestra.run.smithi163:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-09T06:29:45.962 INFO:teuthology.orchestra.run.smithi163.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-09T06:29:45.962 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-09T06:29:45.962 INFO:teuthology.orchestra.run.smithi163.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-09T06:29:45.962 INFO:teuthology.orchestra.run.smithi163.stdout: = reflink=1 2023-10-09T06:29:45.962 INFO:teuthology.orchestra.run.smithi163.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-09T06:29:45.962 INFO:teuthology.orchestra.run.smithi163.stdout: = sunit=0 swidth=0 blks 2023-10-09T06:29:45.962 INFO:teuthology.orchestra.run.smithi163.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-09T06:29:45.962 INFO:teuthology.orchestra.run.smithi163.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-09T06:29:45.963 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-09T06:29:45.963 INFO:teuthology.orchestra.run.smithi163.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-09T06:29:46.389 INFO:teuthology.orchestra.run.smithi163.stdout:Discarding blocks...Done. 2023-10-09T06:29:46.392 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi163.front.sepia.ceph.com -o noatime 2023-10-09T06:29:46.392 DEBUG:teuthology.orchestra.run.smithi163:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-09T06:29:46.572 DEBUG:teuthology.orchestra.run.smithi163:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-09T06:29:46.605 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-09T06:29:46.671 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-09T06:29:46.671 INFO:tasks.ceph:role: osd.1 2023-10-09T06:29:46.671 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi163.front.sepia.ceph.com 2023-10-09T06:29:46.672 DEBUG:teuthology.orchestra.run.smithi163:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-09T06:29:46.745 INFO:teuthology.orchestra.run.smithi163.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-09T06:29:46.745 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-09T06:29:46.745 INFO:teuthology.orchestra.run.smithi163.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-09T06:29:46.745 INFO:teuthology.orchestra.run.smithi163.stdout: = reflink=1 2023-10-09T06:29:46.745 INFO:teuthology.orchestra.run.smithi163.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-09T06:29:46.745 INFO:teuthology.orchestra.run.smithi163.stdout: = sunit=0 swidth=0 blks 2023-10-09T06:29:46.745 INFO:teuthology.orchestra.run.smithi163.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-09T06:29:46.745 INFO:teuthology.orchestra.run.smithi163.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-09T06:29:46.745 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-09T06:29:46.745 INFO:teuthology.orchestra.run.smithi163.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-09T06:29:47.171 INFO:teuthology.orchestra.run.smithi163.stdout:Discarding blocks...Done. 2023-10-09T06:29:47.173 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi163.front.sepia.ceph.com -o noatime 2023-10-09T06:29:47.173 DEBUG:teuthology.orchestra.run.smithi163:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-09T06:29:47.281 DEBUG:teuthology.orchestra.run.smithi163:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-09T06:29:47.313 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-09T06:29:47.380 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-09T06:29:47.381 INFO:tasks.ceph:role: osd.2 2023-10-09T06:29:47.381 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi163.front.sepia.ceph.com 2023-10-09T06:29:47.381 DEBUG:teuthology.orchestra.run.smithi163:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-09T06:29:47.453 INFO:teuthology.orchestra.run.smithi163.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-09T06:29:47.454 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-09T06:29:47.454 INFO:teuthology.orchestra.run.smithi163.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-09T06:29:47.454 INFO:teuthology.orchestra.run.smithi163.stdout: = reflink=1 2023-10-09T06:29:47.454 INFO:teuthology.orchestra.run.smithi163.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-09T06:29:47.454 INFO:teuthology.orchestra.run.smithi163.stdout: = sunit=0 swidth=0 blks 2023-10-09T06:29:47.454 INFO:teuthology.orchestra.run.smithi163.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-09T06:29:47.454 INFO:teuthology.orchestra.run.smithi163.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-09T06:29:47.454 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-09T06:29:47.454 INFO:teuthology.orchestra.run.smithi163.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-09T06:29:47.897 INFO:teuthology.orchestra.run.smithi163.stdout:Discarding blocks...Done. 2023-10-09T06:29:47.899 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi163.front.sepia.ceph.com -o noatime 2023-10-09T06:29:47.899 DEBUG:teuthology.orchestra.run.smithi163:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-09T06:29:48.075 DEBUG:teuthology.orchestra.run.smithi163:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-09T06:29:48.109 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-09T06:29:48.175 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-09T06:29:48.175 INFO:tasks.ceph:role: osd.3 2023-10-09T06:29:48.175 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi163.front.sepia.ceph.com 2023-10-09T06:29:48.175 DEBUG:teuthology.orchestra.run.smithi163:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-09T06:29:48.248 INFO:teuthology.orchestra.run.smithi163.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-09T06:29:48.248 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-09T06:29:48.249 INFO:teuthology.orchestra.run.smithi163.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-09T06:29:48.249 INFO:teuthology.orchestra.run.smithi163.stdout: = reflink=1 2023-10-09T06:29:48.249 INFO:teuthology.orchestra.run.smithi163.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-09T06:29:48.249 INFO:teuthology.orchestra.run.smithi163.stdout: = sunit=0 swidth=0 blks 2023-10-09T06:29:48.249 INFO:teuthology.orchestra.run.smithi163.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-09T06:29:48.249 INFO:teuthology.orchestra.run.smithi163.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-09T06:29:48.249 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-09T06:29:48.249 INFO:teuthology.orchestra.run.smithi163.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-09T06:29:48.692 INFO:teuthology.orchestra.run.smithi163.stdout:Discarding blocks...Done. 2023-10-09T06:29:48.694 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi163.front.sepia.ceph.com -o noatime 2023-10-09T06:29:48.694 DEBUG:teuthology.orchestra.run.smithi163:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-09T06:29:48.871 DEBUG:teuthology.orchestra.run.smithi163:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-09T06:29:48.903 DEBUG:teuthology.orchestra.run.smithi163:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-09T06:29:48.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:48.998+0000 7ff410567540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-09T06:29:48.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:48.998+0000 7ff410567540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-09T06:29:48.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:48.998+0000 7ff410567540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-09T06:29:48.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:48.998+0000 7ff410567540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-09T06:29:48.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:48.998+0000 7ff410567540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-09T06:29:48.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:48.999+0000 7ff410567540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-09T06:29:49.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:49.523+0000 7ff410567540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-09T06:29:50.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:50.287+0000 7ff410567540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-09T06:29:51.049 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:51.048+0000 7ff410567540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-09T06:29:51.846 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-09T06:29:51.874 DEBUG:teuthology.orchestra.run.smithi163:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-09T06:29:51.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:51.956+0000 7ff632eaa540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-09T06:29:51.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:51.957+0000 7ff632eaa540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-09T06:29:51.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:51.957+0000 7ff632eaa540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-09T06:29:51.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:51.957+0000 7ff632eaa540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-09T06:29:51.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:51.957+0000 7ff632eaa540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-09T06:29:51.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:51.957+0000 7ff632eaa540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-09T06:29:52.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:52.482+0000 7ff632eaa540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-09T06:29:53.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:53.246+0000 7ff632eaa540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-09T06:29:54.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:54.009+0000 7ff632eaa540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-09T06:29:54.815 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-09T06:29:54.844 DEBUG:teuthology.orchestra.run.smithi163:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-09T06:29:54.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:54.929+0000 7ff11d256540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-09T06:29:54.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:54.929+0000 7ff11d256540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-09T06:29:54.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:54.929+0000 7ff11d256540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-09T06:29:54.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:54.929+0000 7ff11d256540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-09T06:29:54.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:54.929+0000 7ff11d256540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-09T06:29:54.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:54.930+0000 7ff11d256540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-09T06:29:55.460 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:55.459+0000 7ff11d256540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-09T06:29:56.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:56.228+0000 7ff11d256540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-09T06:29:56.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:56.992+0000 7ff11d256540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-09T06:29:57.794 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-09T06:29:57.823 DEBUG:teuthology.orchestra.run.smithi163:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-09T06:29:57.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:57.908+0000 7f23bc6fd540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-09T06:29:57.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:57.908+0000 7f23bc6fd540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-09T06:29:57.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:57.908+0000 7f23bc6fd540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-09T06:29:57.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:57.908+0000 7f23bc6fd540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-09T06:29:57.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:57.908+0000 7f23bc6fd540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-09T06:29:57.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:57.909+0000 7f23bc6fd540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-09T06:29:58.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:58.442+0000 7f23bc6fd540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-09T06:29:59.211 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:59.210+0000 7f23bc6fd540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-09T06:29:59.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:29:59.974+0000 7f23bc6fd540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-09T06:30:00.768 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-09T06:30:00.798 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-09T06:30:00.828 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-09T06:30:00.828 INFO:tasks.ceph:role: osd.4 2023-10-09T06:30:00.828 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi181.front.sepia.ceph.com 2023-10-09T06:30:00.828 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-09T06:30:00.904 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-09T06:30:00.905 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-09T06:30:00.905 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-09T06:30:00.905 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 2023-10-09T06:30:00.905 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-09T06:30:00.905 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2023-10-09T06:30:00.905 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-09T06:30:00.905 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-09T06:30:00.905 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-09T06:30:00.905 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-09T06:30:01.359 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2023-10-09T06:30:01.360 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2023-10-09T06:30:01.360 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-09T06:30:01.553 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-09T06:30:01.587 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-09T06:30:01.652 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-09T06:30:01.652 INFO:tasks.ceph:role: osd.5 2023-10-09T06:30:01.653 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi181.front.sepia.ceph.com 2023-10-09T06:30:01.653 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-09T06:30:01.725 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-09T06:30:01.725 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-09T06:30:01.725 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-09T06:30:01.725 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 2023-10-09T06:30:01.726 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-09T06:30:01.726 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2023-10-09T06:30:01.726 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-09T06:30:01.726 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-09T06:30:01.726 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-09T06:30:01.726 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-09T06:30:02.129 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2023-10-09T06:30:02.131 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2023-10-09T06:30:02.131 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-09T06:30:02.243 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-09T06:30:02.315 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-09T06:30:02.388 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-09T06:30:02.388 INFO:tasks.ceph:role: osd.6 2023-10-09T06:30:02.388 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi181.front.sepia.ceph.com 2023-10-09T06:30:02.388 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-09T06:30:02.469 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-09T06:30:02.469 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-09T06:30:02.470 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-09T06:30:02.470 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 2023-10-09T06:30:02.470 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-09T06:30:02.470 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2023-10-09T06:30:02.470 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-09T06:30:02.470 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-09T06:30:02.470 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-09T06:30:02.471 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-09T06:30:02.919 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2023-10-09T06:30:02.921 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2023-10-09T06:30:02.921 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-09T06:30:03.058 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-09T06:30:03.096 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-09T06:30:03.168 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-09T06:30:03.168 INFO:tasks.ceph:role: osd.7 2023-10-09T06:30:03.168 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi181.front.sepia.ceph.com 2023-10-09T06:30:03.169 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-09T06:30:03.240 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-09T06:30:03.240 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-09T06:30:03.240 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-09T06:30:03.241 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 2023-10-09T06:30:03.241 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-09T06:30:03.241 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2023-10-09T06:30:03.241 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-09T06:30:03.241 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-09T06:30:03.241 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-09T06:30:03.241 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-09T06:30:03.691 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2023-10-09T06:30:03.693 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2023-10-09T06:30:03.693 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-09T06:30:03.865 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-09T06:30:03.894 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-09T06:30:03.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:03.992+0000 7fed0789e540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-09T06:30:03.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:03.992+0000 7fed0789e540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-09T06:30:03.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:03.992+0000 7fed0789e540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-09T06:30:03.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:03.992+0000 7fed0789e540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-09T06:30:03.993 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:03.992+0000 7fed0789e540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-09T06:30:03.994 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:03.993+0000 7fed0789e540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-09T06:30:04.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:04.517+0000 7fed0789e540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-09T06:30:05.284 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:05.284+0000 7fed0789e540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-09T06:30:06.046 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:06.047+0000 7fed0789e540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-09T06:30:06.845 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-09T06:30:06.870 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-09T06:30:06.951 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:06.951+0000 7ff1983b2540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-09T06:30:06.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:06.951+0000 7ff1983b2540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-09T06:30:06.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:06.951+0000 7ff1983b2540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-09T06:30:06.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:06.951+0000 7ff1983b2540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-09T06:30:06.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:06.951+0000 7ff1983b2540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-09T06:30:06.952 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:06.952+0000 7ff1983b2540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-09T06:30:07.474 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:07.474+0000 7ff1983b2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-09T06:30:08.241 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:08.241+0000 7ff1983b2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-09T06:30:09.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:09.004+0000 7ff1983b2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-09T06:30:09.797 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-09T06:30:09.864 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-09T06:30:09.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:09.948+0000 7f550541b540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-09T06:30:09.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:09.948+0000 7f550541b540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-09T06:30:09.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:09.948+0000 7f550541b540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-09T06:30:09.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:09.948+0000 7f550541b540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-09T06:30:09.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:09.948+0000 7f550541b540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-09T06:30:09.949 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:09.949+0000 7f550541b540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-09T06:30:10.478 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:10.478+0000 7f550541b540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-09T06:30:11.243 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:11.243+0000 7f550541b540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-09T06:30:12.005 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:12.006+0000 7f550541b540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-09T06:30:12.692 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:30:12.696 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:30:12.810 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-09T06:30:12.878 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-09T06:30:12.960 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:12.960+0000 7fac6a6f9540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-09T06:30:12.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:12.960+0000 7fac6a6f9540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-09T06:30:12.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:12.960+0000 7fac6a6f9540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-09T06:30:12.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:12.960+0000 7fac6a6f9540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-09T06:30:12.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:12.960+0000 7fac6a6f9540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-09T06:30:12.961 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:12.961+0000 7fac6a6f9540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-09T06:30:13.487 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:13.487+0000 7fac6a6f9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-09T06:30:14.249 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:14.249+0000 7fac6a6f9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-09T06:30:15.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:15.013+0000 7fac6a6f9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-09T06:30:15.808 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-09T06:30:15.873 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-09T06:30:15.873 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-09T06:30:15.873 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-09T06:30:15.902 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-09T06:30:15.902 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-09T06:30:15.968 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-09T06:30:15.968 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-09T06:30:16.034 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-09T06:30:16.035 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-09T06:30:16.099 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-09T06:30:16.099 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-09T06:30:16.165 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-09T06:30:16.165 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-09T06:30:16.229 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-09T06:30:16.230 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-09T06:30:16.294 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-09T06:30:16.294 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-09T06:30:16.319 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-09T06:30:16.319 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-09T06:30:16.384 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-09T06:30:16.384 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-09T06:30:16.449 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-09T06:30:16.449 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-09T06:30:16.512 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-09T06:30:16.512 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-09T06:30:16.576 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-09T06:30:16.576 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-09T06:30:16.640 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-09T06:30:16.641 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-09T06:30:16.704 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-09T06:30:16.704 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-09T06:30:16.719 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-09T06:30:16.719 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-09T06:30:16.774 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-09T06:30:16.774 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-09T06:30:16.788 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-09T06:30:16.788 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-09T06:30:16.842 INFO:tasks.ceph:Adding keys to all mons... 2023-10-09T06:30:16.843 DEBUG:teuthology.orchestra.run.smithi163:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-09T06:30:16.845 DEBUG:teuthology.orchestra.run.smithi181:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-09T06:30:16.885 INFO:teuthology.orchestra.run.smithi163.stdout:[mgr.y] 2023-10-09T06:30:16.885 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDZnSNlcFWVCBAAR2dKPg01ciTiNpSX0JIKFg== 2023-10-09T06:30:16.886 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.a] 2023-10-09T06:30:16.886 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDZnSNltXtwDxAAlnrWRsfnZDLEmsJVowh3Mg== 2023-10-09T06:30:16.886 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.c] 2023-10-09T06:30:16.886 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDZnSNlGis7FhAAYG76TiOFX8QkTwjNX2+Ncw== 2023-10-09T06:30:16.886 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.0] 2023-10-09T06:30:16.886 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDcnSNlrQeFOxAA+s6UitrTn5TK1N7kgySoXw== 2023-10-09T06:30:16.886 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.1] 2023-10-09T06:30:16.886 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDfnSNlleccORAAnB+lNUFblivdoY44JfELxA== 2023-10-09T06:30:16.886 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.2] 2023-10-09T06:30:16.886 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDinSNlFoB4NxAAT0+trRBij5rqgc+F0eWUfQ== 2023-10-09T06:30:16.886 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.3] 2023-10-09T06:30:16.886 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDlnSNlRRQ6NhAABoBtrFjSvQujolq0Pxrd1Q== 2023-10-09T06:30:16.887 INFO:teuthology.orchestra.run.smithi163.stdout:[mgr.x] 2023-10-09T06:30:16.887 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDZnSNl2sloDBAA0FoWN9iovCl6783o3/btiA== 2023-10-09T06:30:16.887 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.b] 2023-10-09T06:30:16.887 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDZnSNlTdGhGhAAXuldy0HrFd+44sSu/EV3qA== 2023-10-09T06:30:16.887 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.d] 2023-10-09T06:30:16.887 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDZnSNlKhRuIRAAfhG5Gpc7klXJloYXepOhsg== 2023-10-09T06:30:16.887 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.4] 2023-10-09T06:30:16.887 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDrnSNl308uOxAA5OJyPLGQmVSJqha5d+nKKQ== 2023-10-09T06:30:16.887 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.5] 2023-10-09T06:30:16.887 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDunSNlHnK+OBAAU7l4dCJ82SbGSzBEm8Axzw== 2023-10-09T06:30:16.887 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.6] 2023-10-09T06:30:16.887 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDxnSNlwUSLOBAAN5N+F5ho+Co8pBh4Rz+lXQ== 2023-10-09T06:30:16.888 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.7] 2023-10-09T06:30:16.888 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQD0nSNl2gxGORAAoE8OLGEifcEx/km8rfSV9g== 2023-10-09T06:30:16.888 INFO:teuthology.orchestra.run.smithi163.stdout:[client.0] 2023-10-09T06:30:16.888 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDZnSNlnxY5JRAApr6czcpMdQYYtXQ9yg2q6A== 2023-10-09T06:30:16.888 INFO:teuthology.orchestra.run.smithi163.stdout:[client.2] 2023-10-09T06:30:16.888 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDZnSNlYyOmKhAAQmpr+mGfJH4GUa28dtlJwA== 2023-10-09T06:30:16.888 INFO:teuthology.orchestra.run.smithi163.stdout:[client.1] 2023-10-09T06:30:16.888 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDZnSNlk6ybLRAAuShqTDAtymGdxN8KBdZDDA== 2023-10-09T06:30:16.888 INFO:teuthology.orchestra.run.smithi163.stdout:[client.3] 2023-10-09T06:30:16.888 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDZnSNlD2H1MhAA4zvaXY/Gd0kndzi+QSwp7w== 2023-10-09T06:30:16.905 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr.y] 2023-10-09T06:30:16.905 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDZnSNlcFWVCBAAR2dKPg01ciTiNpSX0JIKFg== 2023-10-09T06:30:16.905 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.a] 2023-10-09T06:30:16.905 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDZnSNltXtwDxAAlnrWRsfnZDLEmsJVowh3Mg== 2023-10-09T06:30:16.905 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.c] 2023-10-09T06:30:16.905 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDZnSNlGis7FhAAYG76TiOFX8QkTwjNX2+Ncw== 2023-10-09T06:30:16.905 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.0] 2023-10-09T06:30:16.906 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDcnSNlrQeFOxAA+s6UitrTn5TK1N7kgySoXw== 2023-10-09T06:30:16.906 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.1] 2023-10-09T06:30:16.906 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDfnSNlleccORAAnB+lNUFblivdoY44JfELxA== 2023-10-09T06:30:16.906 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.2] 2023-10-09T06:30:16.906 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDinSNlFoB4NxAAT0+trRBij5rqgc+F0eWUfQ== 2023-10-09T06:30:16.906 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.3] 2023-10-09T06:30:16.906 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDlnSNlRRQ6NhAABoBtrFjSvQujolq0Pxrd1Q== 2023-10-09T06:30:16.906 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr.x] 2023-10-09T06:30:16.907 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDZnSNl2sloDBAA0FoWN9iovCl6783o3/btiA== 2023-10-09T06:30:16.907 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.b] 2023-10-09T06:30:16.907 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDZnSNlTdGhGhAAXuldy0HrFd+44sSu/EV3qA== 2023-10-09T06:30:16.907 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.d] 2023-10-09T06:30:16.907 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDZnSNlKhRuIRAAfhG5Gpc7klXJloYXepOhsg== 2023-10-09T06:30:16.907 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.4] 2023-10-09T06:30:16.907 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDrnSNl308uOxAA5OJyPLGQmVSJqha5d+nKKQ== 2023-10-09T06:30:16.907 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.5] 2023-10-09T06:30:16.907 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDunSNlHnK+OBAAU7l4dCJ82SbGSzBEm8Axzw== 2023-10-09T06:30:16.908 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.6] 2023-10-09T06:30:16.908 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDxnSNlwUSLOBAAN5N+F5ho+Co8pBh4Rz+lXQ== 2023-10-09T06:30:16.908 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.7] 2023-10-09T06:30:16.908 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQD0nSNl2gxGORAAoE8OLGEifcEx/km8rfSV9g== 2023-10-09T06:30:16.908 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2023-10-09T06:30:16.908 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDZnSNlnxY5JRAApr6czcpMdQYYtXQ9yg2q6A== 2023-10-09T06:30:16.909 INFO:teuthology.orchestra.run.smithi181.stdout:[client.2] 2023-10-09T06:30:16.909 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDZnSNlYyOmKhAAQmpr+mGfJH4GUa28dtlJwA== 2023-10-09T06:30:16.909 INFO:teuthology.orchestra.run.smithi181.stdout:[client.1] 2023-10-09T06:30:16.909 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDZnSNlk6ybLRAAuShqTDAtymGdxN8KBdZDDA== 2023-10-09T06:30:16.909 INFO:teuthology.orchestra.run.smithi181.stdout:[client.3] 2023-10-09T06:30:16.909 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDZnSNlD2H1MhAA4zvaXY/Gd0kndzi+QSwp7w== 2023-10-09T06:30:16.910 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-09T06:30:16.932 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-09T06:30:16.996 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-09T06:30:16.998 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-09T06:30:17.048 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-09T06:30:17.050 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-09T06:30:17.100 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-09T06:30:17.103 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-09T06:30:17.150 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-09T06:30:17.153 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-09T06:30:17.200 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-09T06:30:17.202 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-09T06:30:17.248 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-09T06:30:17.251 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-09T06:30:17.298 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-09T06:30:17.300 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-09T06:30:17.348 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-09T06:30:17.350 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-09T06:30:17.401 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-09T06:30:17.403 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-09T06:30:17.453 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-09T06:30:17.455 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-09T06:30:17.503 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-09T06:30:17.505 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-09T06:30:17.553 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-09T06:30:17.555 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-09T06:30:17.602 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-09T06:30:17.604 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-09T06:30:17.651 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-09T06:30:17.654 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-09T06:30:17.703 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-09T06:30:17.706 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-09T06:30:17.756 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-09T06:30:17.758 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-09T06:30:17.808 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-09T06:30:17.810 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-09T06:30:17.860 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-09T06:30:17.860 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-09T06:30:17.885 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-09T06:30:17.976 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-09T06:30:18.000 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-09T06:30:18.025 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-09T06:30:18.117 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-09T06:30:18.141 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-09T06:30:18.204 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-09T06:30:18.295 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-09T06:30:18.320 DEBUG:teuthology.orchestra.run.smithi163:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-09T06:30:18.322 DEBUG:teuthology.orchestra.run.smithi181:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-09T06:30:18.375 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-09T06:30:18.376 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-09T06:30:18.376 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-09T06:30:18.379 INFO:tasks.ceph.mon.a:Started 2023-10-09T06:30:18.379 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-09T06:30:18.379 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-09T06:30:18.418 INFO:tasks.ceph.mon.b:Started 2023-10-09T06:30:18.418 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-09T06:30:18.418 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-09T06:30:18.420 INFO:tasks.ceph.mon.c:Started 2023-10-09T06:30:18.420 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-09T06:30:18.420 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-09T06:30:18.420 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-09T06:30:18.422 INFO:tasks.ceph.mgr.y:Started 2023-10-09T06:30:18.422 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-09T06:30:18.423 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-09T06:30:18.424 INFO:tasks.ceph.mgr.x:Started 2023-10-09T06:30:18.424 INFO:tasks.ceph:Setting crush tunables to default 2023-10-09T06:30:18.425 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd crush tunables default 2023-10-09T06:30:18.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.541+0000 7f1d0b8a6700 1 Processor -- start 2023-10-09T06:30:18.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.541+0000 7f1d0b8a6700 1 -- start start 2023-10-09T06:30:18.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.542+0000 7f1d0b8a6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d04069d60 0x7f1d040c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:18.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.542+0000 7f1d0b8a6700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d0406bdb0 con 0x7f1d040aca80 2023-10-09T06:30:18.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.542+0000 7f1d0b8a6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d040cb370 con 0x7f1d040c81f0 2023-10-09T06:30:18.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.542+0000 7f1d0b8a6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d040c8670 con 0x7f1d04069d60 2023-10-09T06:30:18.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.542+0000 7f1d09642700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f1d040aca80 0x7f1d0406aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:37782/0 (socket says 172.21.15.163:37782) 2023-10-09T06:30:18.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.542+0000 7f1d09642700 1 -- 172.21.15.163:0/2771690004 learned_addr learned my addr 172.21.15.163:0/2771690004 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:18.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.542+0000 7f1d08e41700 1 --2- 172.21.15.163:0/2771690004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d04069d60 0x7f1d040c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:18.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.543+0000 7f1d08e41700 1 -- 172.21.15.163:0/2771690004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d04069d60 msgr2=0x7f1d040c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-09T06:30:18.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.543+0000 7f1d08e41700 1 -- 172.21.15.163:0/2771690004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d04069d60 msgr2=0x7f1d040c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-09T06:30:18.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.543+0000 7f1d08e41700 1 --2- 172.21.15.163:0/2771690004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d04069d60 0x7f1d040c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-09T06:30:18.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.543+0000 7f1d08e41700 1 --2- 172.21.15.163:0/2771690004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d04069d60 0x7f1d040c7cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-09T06:30:18.730 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.729+0000 7f1cfbfff700 1 -- 172.21.15.163:0/2771690004 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3962016450 0 0) 0x7f1d0406bdb0 con 0x7f1d040aca80 2023-10-09T06:30:18.731 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.730+0000 7f1cfbfff700 1 -- 172.21.15.163:0/2771690004 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cf0003570 con 0x7f1d040aca80 2023-10-09T06:30:18.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.730+0000 7f1cfbfff700 1 -- 172.21.15.163:0/2771690004 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3294319729 0 0) 0x7f1cf0003570 con 0x7f1d040aca80 2023-10-09T06:30:18.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.730+0000 7f1cfbfff700 1 -- 172.21.15.163:0/2771690004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d04069d60 msgr2=0x7f1d040c7cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-09T06:30:18.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.730+0000 7f1cfbfff700 1 --2- 172.21.15.163:0/2771690004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d04069d60 0x7f1d040c7cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:18.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.730+0000 7f1cfbfff700 1 -- 172.21.15.163:0/2771690004 >> v1:172.21.15.181:6789/0 conn(0x7f1d040c81f0 legacy=0x7f1d040cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:18.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.730+0000 7f1cfbfff700 1 -- 172.21.15.163:0/2771690004 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d04159f60 con 0x7f1d040aca80 2023-10-09T06:30:18.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.730+0000 7f1cfbfff700 1 -- 172.21.15.163:0/2771690004 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f1cf40015f0 con 0x7f1d040aca80 2023-10-09T06:30:18.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.730+0000 7f1cfbfff700 1 -- 172.21.15.163:0/2771690004 >> v1:172.21.15.163:6789/0 conn(0x7f1d040aca80 legacy=0x7f1d0406aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:18.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.730+0000 7f1cfbfff700 1 --2- 172.21.15.163:0/2771690004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cf0003a90 0x7f1cf0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:18.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.730+0000 7f1cfbfff700 1 --2- 172.21.15.163:0/2771690004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d04069d60 0x7f1cf0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:18.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.730+0000 7f1cfbfff700 1 --2- 172.21.15.163:0/2771690004 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cf0004da0 0x7f1cf00051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:18.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.730+0000 7f1cfbfff700 1 -- 172.21.15.163:0/2771690004 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d040c8670 con 0x7f1cf0004da0 2023-10-09T06:30:18.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.730+0000 7f1cfbfff700 1 -- 172.21.15.163:0/2771690004 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf000bc20 con 0x7f1cf0003a90 2023-10-09T06:30:18.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.730+0000 7f1cfbfff700 1 -- 172.21.15.163:0/2771690004 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf000bde0 con 0x7f1d04069d60 2023-10-09T06:30:18.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.731+0000 7f1d09642700 1 --2- 172.21.15.163:0/2771690004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cf0003a90 0x7f1cf0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:18.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.731+0000 7f1d09e43700 1 --2- 172.21.15.163:0/2771690004 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cf0004da0 0x7f1cf00051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:18.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.731+0000 7f1d08e41700 1 --2- 172.21.15.163:0/2771690004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d04069d60 0x7f1cf0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:18.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.731+0000 7f1d09e43700 1 -- 172.21.15.163:0/2771690004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d04069d60 msgr2=0x7f1cf0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:18.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.731+0000 7f1d09e43700 1 --2- 172.21.15.163:0/2771690004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d04069d60 0x7f1cf0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:18.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.731+0000 7f1d09e43700 1 -- 172.21.15.163:0/2771690004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cf0003a90 msgr2=0x7f1cf0003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:18.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.731+0000 7f1d09e43700 1 --2- 172.21.15.163:0/2771690004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cf0003a90 0x7f1cf0003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:18.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.731+0000 7f1d09e43700 1 -- 172.21.15.163:0/2771690004 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf0003570 con 0x7f1cf0004da0 2023-10-09T06:30:18.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.731+0000 7f1d09e43700 1 --2- 172.21.15.163:0/2771690004 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cf0004da0 0x7f1cf00051f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc001fc0 tx=0x7f1cfc001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be923ede6c74d87c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:18.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.731+0000 7f1cfbfff700 1 -- 172.21.15.163:0/2771690004 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc001980 con 0x7f1cf0004da0 2023-10-09T06:30:18.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.731+0000 7f1cfbfff700 1 -- 172.21.15.163:0/2771690004 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cfc00c8b0 con 0x7f1cf0004da0 2023-10-09T06:30:18.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.731+0000 7f1cfbfff700 1 -- 172.21.15.163:0/2771690004 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc00cc70 con 0x7f1cf0004da0 2023-10-09T06:30:18.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.732+0000 7f1d0b8a6700 1 -- 172.21.15.163:0/2771690004 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cf0004da0 msgr2=0x7f1cf00051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:18.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.732+0000 7f1d0b8a6700 1 --2- 172.21.15.163:0/2771690004 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cf0004da0 0x7f1cf00051f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc001fc0 tx=0x7f1cfc001670 comp rx=0 tx=0).stop 2023-10-09T06:30:18.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.732+0000 7f1d09e43700 1 -- 172.21.15.163:0/2771690004 reap_dead start 2023-10-09T06:30:18.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.732+0000 7f1d0b8a6700 1 -- 172.21.15.163:0/2771690004 shutdown_connections 2023-10-09T06:30:18.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.732+0000 7f1d0b8a6700 1 -- 172.21.15.163:0/2771690004 >> 172.21.15.163:0/2771690004 conn(0x7f1d040cd670 msgr2=0x7f1d040cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:18.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.732+0000 7f1d0b8a6700 1 -- 172.21.15.163:0/2771690004 shutdown_connections 2023-10-09T06:30:18.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.732+0000 7f1d0b8a6700 1 -- 172.21.15.163:0/2771690004 wait complete. 2023-10-09T06:30:18.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.733+0000 7f1d0b8a6700 1 Processor -- start 2023-10-09T06:30:18.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.733+0000 7f1d0b8a6700 1 -- start start 2023-10-09T06:30:18.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.734+0000 7f1d0b8a6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cf0004da0 0x7f1d0415cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:18.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.734+0000 7f1d0b8a6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d0415a120 0x7f1d040c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:18.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.734+0000 7f1d0b8a6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d040ab300 0x7f1d040ab750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:18.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.734+0000 7f1d0b8a6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0415a8c0 con 0x7f1d0415a120 2023-10-09T06:30:18.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.734+0000 7f1d0b8a6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d040abc90 con 0x7f1cf0004da0 2023-10-09T06:30:18.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.734+0000 7f1d0b8a6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0416a480 con 0x7f1d040ab300 2023-10-09T06:30:18.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.734+0000 7f1d08e41700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d0415a120 0x7f1d040c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:18.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.734+0000 7f1d09642700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cf0004da0 0x7f1d0415cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:18.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.734+0000 7f1d09642700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cf0004da0 0x7f1d0415cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:52334/0 (socket says 172.21.15.163:52334) 2023-10-09T06:30:18.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.734+0000 7f1d09642700 1 -- 172.21.15.163:0/4155945516 learned_addr learned my addr 172.21.15.163:0/4155945516 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:18.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.734+0000 7f1d08e41700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d0415a120 0x7f1d040c6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:50628/0 (socket says 172.21.15.163:50628) 2023-10-09T06:30:18.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.734+0000 7f1d09e43700 1 --2- 172.21.15.163:0/4155945516 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d040ab300 0x7f1d040ab750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:18.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.734+0000 7f1d08e41700 1 -- 172.21.15.163:0/4155945516 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d040ab300 msgr2=0x7f1d040ab750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:18.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.734+0000 7f1d08e41700 1 --2- 172.21.15.163:0/4155945516 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d040ab300 0x7f1d040ab750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:18.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.734+0000 7f1d08e41700 1 -- 172.21.15.163:0/4155945516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cf0004da0 msgr2=0x7f1d0415cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:18.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.734+0000 7f1d08e41700 1 --2- 172.21.15.163:0/4155945516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cf0004da0 0x7f1d0415cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:18.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.734+0000 7f1d08e41700 1 -- 172.21.15.163:0/4155945516 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d040b18c0 con 0x7f1d0415a120 2023-10-09T06:30:18.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.734+0000 7f1d09642700 1 --2- 172.21.15.163:0/4155945516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cf0004da0 0x7f1d0415cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:30:18.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.735+0000 7f1d08e41700 1 --2- 172.21.15.163:0/4155945516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d0415a120 0x7f1d040c6d60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000f6b0 tx=0x7f1d00008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:18.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.735+0000 7f1cf97fa700 1 -- 172.21.15.163:0/4155945516 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d00014670 con 0x7f1d0415a120 2023-10-09T06:30:18.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.735+0000 7f1cf97fa700 1 -- 172.21.15.163:0/4155945516 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d000147d0 con 0x7f1d0415a120 2023-10-09T06:30:18.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.735+0000 7f1cf97fa700 1 -- 172.21.15.163:0/4155945516 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d00014b70 con 0x7f1d0415a120 2023-10-09T06:30:18.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.735+0000 7f1d0b8a6700 1 -- 172.21.15.163:0/4155945516 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d041696c0 con 0x7f1d0415a120 2023-10-09T06:30:18.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.735+0000 7f1d0b8a6700 1 -- 172.21.15.163:0/4155945516 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d04169c80 con 0x7f1d0415a120 2023-10-09T06:30:18.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.736+0000 7f1d0b8a6700 1 -- 172.21.15.163:0/4155945516 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d0404c3c0 con 0x7f1d0415a120 2023-10-09T06:30:18.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.737+0000 7f1cf97fa700 1 -- 172.21.15.163:0/4155945516 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f1d00027440 con 0x7f1d0415a120 2023-10-09T06:30:18.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.737+0000 7f1cf97fa700 1 -- 172.21.15.163:0/4155945516 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f1d0001d770 con 0x7f1d0415a120 2023-10-09T06:30:18.746 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.739+0000 7f1cf97fa700 1 -- 172.21.15.163:0/4155945516 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f1d0003c6d0 con 0x7f1d0415a120 2023-10-09T06:30:18.815 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.814+0000 7f1d0b8a6700 1 -- 172.21.15.163:0/4155945516 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f1d040678d0 con 0x7f1d0415a120 2023-10-09T06:30:18.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.822+0000 7f1cf97fa700 1 -- 172.21.15.163:0/4155945516 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f1d0002b020 con 0x7f1d0415a120 2023-10-09T06:30:18.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.822+0000 7f1d0b8a6700 1 -- 172.21.15.163:0/4155945516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d0415a120 msgr2=0x7f1d040c6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:18.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.822+0000 7f1d0b8a6700 1 --2- 172.21.15.163:0/4155945516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d0415a120 0x7f1d040c6d60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000f6b0 tx=0x7f1d00008040 comp rx=0 tx=0).stop 2023-10-09T06:30:18.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.823+0000 7f1d0b8a6700 1 -- 172.21.15.163:0/4155945516 shutdown_connections 2023-10-09T06:30:18.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.823+0000 7f1d0b8a6700 1 --2- 172.21.15.163:0/4155945516 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d040ab300 0x7f1d040ab750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:18.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.823+0000 7f1d0b8a6700 1 --2- 172.21.15.163:0/4155945516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cf0004da0 0x7f1d0415cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:18.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.823+0000 7f1d0b8a6700 1 --2- 172.21.15.163:0/4155945516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d0415a120 0x7f1d040c6d60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:18.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.823+0000 7f1d0b8a6700 1 -- 172.21.15.163:0/4155945516 >> 172.21.15.163:0/4155945516 conn(0x7f1d040cd670 msgr2=0x7f1d04159530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:18.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.823+0000 7f1d0b8a6700 1 -- 172.21.15.163:0/4155945516 shutdown_connections 2023-10-09T06:30:18.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:18.823+0000 7f1d0b8a6700 1 -- 172.21.15.163:0/4155945516 wait complete. 2023-10-09T06:30:18.826 INFO:teuthology.orchestra.run.smithi163.stderr:adjusted tunables profile to default 2023-10-09T06:30:18.836 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-09T06:30:18.836 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-09T06:30:18.837 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-09T06:30:18.862 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-09T06:30:18.862 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-09T06:30:18.927 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-09T06:30:18.927 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-09T06:30:18.992 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-09T06:30:18.992 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-09T06:30:19.057 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-09T06:30:19.057 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-09T06:30:19.082 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-09T06:30:19.082 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-09T06:30:19.147 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-09T06:30:19.147 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-09T06:30:19.212 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-09T06:30:19.213 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-09T06:30:19.277 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 51bfc982-10fe-4bfd-9541-07653c290e84 0 2023-10-09T06:30:19.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.433+0000 7f4dca614700 1 Processor -- start 2023-10-09T06:30:19.434 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.433+0000 7f4dca614700 1 -- start start 2023-10-09T06:30:19.436 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.434+0000 7f4dca614700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4dc4157680 0x7f4dc40cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:19.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.434+0000 7f4dca614700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4dc4157a50 0x7f4dc40cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:19.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.434+0000 7f4dca614700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dc40cb900 con 0x7f4dc40b1450 2023-10-09T06:30:19.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.434+0000 7f4dca614700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dc40cc700 con 0x7f4dc4157680 2023-10-09T06:30:19.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.434+0000 7f4dca614700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dc40cd240 con 0x7f4dc4157a50 2023-10-09T06:30:19.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.434+0000 7f4dc8e11700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4dc4157a50 0x7f4dc40cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:19.437 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.434+0000 7f4dc9e13700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4dc40b1450 0x7f4dc40c8cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58328/0 (socket says 172.21.15.181:58328) 2023-10-09T06:30:19.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.434+0000 7f4dc9e13700 1 -- 172.21.15.181:0/2823751319 learned_addr learned my addr 172.21.15.181:0/2823751319 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-09T06:30:19.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.434+0000 7f4dc9612700 1 --2- 172.21.15.181:0/2823751319 >> v2:172.21.15.163:3300/0 conn(0x7f4dc4157680 0x7f4dc40cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:19.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.434+0000 7f4dc8e11700 1 -- 172.21.15.181:0/2823751319 >> v2:172.21.15.163:3300/0 conn(0x7f4dc4157680 msgr2=0x7f4dc40cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:19.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.434+0000 7f4dc8e11700 1 --2- 172.21.15.181:0/2823751319 >> v2:172.21.15.163:3300/0 conn(0x7f4dc4157680 0x7f4dc40cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:19.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.434+0000 7f4dc8e11700 1 -- 172.21.15.181:0/2823751319 >> v1:172.21.15.181:6789/0 conn(0x7f4dc40b1450 legacy=0x7f4dc40c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:19.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.434+0000 7f4dbbfff700 1 -- 172.21.15.181:0/2823751319 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1786405216 0 0) 0x7f4dbc001490 con 0x7f4dc40b1450 2023-10-09T06:30:19.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.434+0000 7f4dc8e11700 1 -- 172.21.15.181:0/2823751319 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc40c9a20 con 0x7f4dc4157a50 2023-10-09T06:30:19.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.435+0000 7f4dc8e11700 1 --2- 172.21.15.181:0/2823751319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4dc4157a50 0x7f4dc40cb3c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4dc40cbc70 tx=0x7f4dc00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27ff24de1f9eee7f server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:19.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.435+0000 7f4dbbfff700 1 -- 172.21.15.181:0/2823751319 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0011630 con 0x7f4dc4157a50 2023-10-09T06:30:19.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.435+0000 7f4dbbfff700 1 -- 172.21.15.181:0/2823751319 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc0011790 con 0x7f4dc4157a50 2023-10-09T06:30:19.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.435+0000 7f4dbbfff700 1 -- 172.21.15.181:0/2823751319 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0011b30 con 0x7f4dc4157a50 2023-10-09T06:30:19.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.435+0000 7f4dca614700 1 -- 172.21.15.181:0/2823751319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4dc4157a50 msgr2=0x7f4dc40cb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:19.439 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.435+0000 7f4dca614700 1 --2- 172.21.15.181:0/2823751319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4dc4157a50 0x7f4dc40cb3c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4dc40cbc70 tx=0x7f4dc00091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:19.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.435+0000 7f4dca614700 1 -- 172.21.15.181:0/2823751319 shutdown_connections 2023-10-09T06:30:19.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.435+0000 7f4dca614700 1 --2- 172.21.15.181:0/2823751319 >> v2:172.21.15.163:3300/0 conn(0x7f4dc4157680 0x7f4dc40cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:19.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.435+0000 7f4dca614700 1 --2- 172.21.15.181:0/2823751319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4dc4157a50 0x7f4dc40cb3c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:19.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.435+0000 7f4dca614700 1 -- 172.21.15.181:0/2823751319 >> 172.21.15.181:0/2823751319 conn(0x7f4dc406baa0 msgr2=0x7f4dc406bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:19.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.435+0000 7f4dca614700 1 -- 172.21.15.181:0/2823751319 shutdown_connections 2023-10-09T06:30:19.440 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.435+0000 7f4dca614700 1 -- 172.21.15.181:0/2823751319 wait complete. 2023-10-09T06:30:19.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.436+0000 7f4dca614700 1 Processor -- start 2023-10-09T06:30:19.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.437+0000 7f4dca614700 1 -- start start 2023-10-09T06:30:19.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.437+0000 7f4dca614700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4dc40b1450 0x7f4dc4157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:19.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.437+0000 7f4dca614700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4dc4157680 0x7f4dc4168ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:19.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.437+0000 7f4dca614700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4dc4157a50 0x7f4dc4168720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:19.443 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.437+0000 7f4dca614700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc40b18e0 con 0x7f4dc40b1450 2023-10-09T06:30:19.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.437+0000 7f4dca614700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc4168c60 con 0x7f4dc4157a50 2023-10-09T06:30:19.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.437+0000 7f4dca614700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc41584d0 con 0x7f4dc4157680 2023-10-09T06:30:19.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.438+0000 7f4dc9e13700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4dc4157a50 0x7f4dc4168720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:19.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.438+0000 7f4dc8e11700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4dc4157680 0x7f4dc4168ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:19.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.438+0000 7f4dc8e11700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4dc4157680 0x7f4dc4168ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:52268/0 (socket says 172.21.15.181:52268) 2023-10-09T06:30:19.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.438+0000 7f4dc9e13700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4dc4157a50 0x7f4dc4168720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59442/0 (socket says 172.21.15.181:59442) 2023-10-09T06:30:19.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.438+0000 7f4dc8e11700 1 -- 172.21.15.181:0/1078379042 learned_addr learned my addr 172.21.15.181:0/1078379042 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-09T06:30:19.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.438+0000 7f4dc9612700 1 --2- 172.21.15.181:0/1078379042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4dc40b1450 0x7f4dc4157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:19.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.438+0000 7f4dc8e11700 1 -- 172.21.15.181:0/1078379042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4dc4157a50 msgr2=0x7f4dc4168720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:19.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.438+0000 7f4dc8e11700 1 --2- 172.21.15.181:0/1078379042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4dc4157a50 0x7f4dc4168720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:19.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.438+0000 7f4dc8e11700 1 -- 172.21.15.181:0/1078379042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4dc40b1450 msgr2=0x7f4dc4157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:19.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.438+0000 7f4dc8e11700 1 --2- 172.21.15.181:0/1078379042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4dc40b1450 0x7f4dc4157ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:19.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.438+0000 7f4dc8e11700 1 -- 172.21.15.181:0/1078379042 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc0009420 con 0x7f4dc4157680 2023-10-09T06:30:19.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.438+0000 7f4dc9e13700 1 --2- 172.21.15.181:0/1078379042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4dc4157a50 0x7f4dc4168720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:30:19.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.438+0000 7f4dc9612700 1 --2- 172.21.15.181:0/1078379042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4dc40b1450 0x7f4dc4157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:30:19.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.438+0000 7f4dc8e11700 1 --2- 172.21.15.181:0/1078379042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4dc4157680 0x7f4dc4168ec0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0013000 tx=0x7f4dc0017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:19.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.438+0000 7f4db97fa700 1 -- 172.21.15.181:0/1078379042 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0011b10 con 0x7f4dc4157680 2023-10-09T06:30:19.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.438+0000 7f4db97fa700 1 -- 172.21.15.181:0/1078379042 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc00233f0 con 0x7f4dc4157680 2023-10-09T06:30:19.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.438+0000 7f4db97fa700 1 -- 172.21.15.181:0/1078379042 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0023790 con 0x7f4dc4157680 2023-10-09T06:30:19.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.438+0000 7f4dca614700 1 -- 172.21.15.181:0/1078379042 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dc41587b0 con 0x7f4dc4157680 2023-10-09T06:30:19.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.438+0000 7f4dca614700 1 -- 172.21.15.181:0/1078379042 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc4158d70 con 0x7f4dc4157680 2023-10-09T06:30:19.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.438+0000 7f4db97fa700 1 -- 172.21.15.181:0/1078379042 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f4dc0023b80 con 0x7f4dc4157680 2023-10-09T06:30:19.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.438+0000 7f4db97fa700 1 -- 172.21.15.181:0/1078379042 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f4dc002daa0 con 0x7f4dc4157680 2023-10-09T06:30:19.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.439+0000 7f4dca614700 1 -- 172.21.15.181:0/1078379042 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dc40b1a50 con 0x7f4dc4157680 2023-10-09T06:30:19.447 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.441+0000 7f4db97fa700 1 -- 172.21.15.181:0/1078379042 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f4dc0028070 con 0x7f4dc4157680 2023-10-09T06:30:19.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.524+0000 7f4dca614700 1 -- 172.21.15.181:0/1078379042 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd new", "uuid": "51bfc982-10fe-4bfd-9541-07653c290e84", "id": 0} v 0) v1 -- 0x7f4dc40cc910 con 0x7f4dc4157680 2023-10-09T06:30:19.530 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.529+0000 7f4db97fa700 1 -- 172.21.15.181:0/1078379042 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "51bfc982-10fe-4bfd-9541-07653c290e84", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4dc0015020 con 0x7f4dc4157680 2023-10-09T06:30:19.530 INFO:teuthology.orchestra.run.smithi181.stdout:0 2023-10-09T06:30:19.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.530+0000 7f4dca614700 1 -- 172.21.15.181:0/1078379042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4dc4157680 msgr2=0x7f4dc4168ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:19.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.530+0000 7f4dca614700 1 --2- 172.21.15.181:0/1078379042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4dc4157680 0x7f4dc4168ec0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0013000 tx=0x7f4dc0017000 comp rx=0 tx=0).stop 2023-10-09T06:30:19.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.530+0000 7f4dca614700 1 -- 172.21.15.181:0/1078379042 shutdown_connections 2023-10-09T06:30:19.531 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.530+0000 7f4dca614700 1 --2- 172.21.15.181:0/1078379042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4dc40b1450 0x7f4dc4157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:19.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.530+0000 7f4dca614700 1 --2- 172.21.15.181:0/1078379042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4dc4157a50 0x7f4dc4168720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:19.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.530+0000 7f4dca614700 1 --2- 172.21.15.181:0/1078379042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4dc4157680 0x7f4dc4168ec0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:19.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.530+0000 7f4dca614700 1 -- 172.21.15.181:0/1078379042 >> 172.21.15.181:0/1078379042 conn(0x7f4dc406baa0 msgr2=0x7f4dc41566e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:19.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.530+0000 7f4dca614700 1 -- 172.21.15.181:0/1078379042 shutdown_connections 2023-10-09T06:30:19.532 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.530+0000 7f4dca614700 1 -- 172.21.15.181:0/1078379042 wait complete. 2023-10-09T06:30:19.543 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new ae88b544-638e-469d-87a3-9646eed2b725 1 2023-10-09T06:30:19.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.656+0000 7fba211a8700 1 Processor -- start 2023-10-09T06:30:19.657 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.656+0000 7fba211a8700 1 -- start start 2023-10-09T06:30:19.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.657+0000 7fba211a8700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fba1c159c60 0x7fba1c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:19.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.657+0000 7fba211a8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fba1c15db10 0x7fba1c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:19.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.657+0000 7fba211a8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba1c15a160 con 0x7fba1c159890 2023-10-09T06:30:19.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.657+0000 7fba211a8700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba1c15df50 con 0x7fba1c159c60 2023-10-09T06:30:19.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.657+0000 7fba211a8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba1c0d13a0 con 0x7fba1c15db10 2023-10-09T06:30:19.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.657+0000 7fba1b59e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fba1c15db10 0x7fba1c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:19.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.657+0000 7fba1ad9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fba1c159890 0x7fba1c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58330/0 (socket says 172.21.15.181:58330) 2023-10-09T06:30:19.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.657+0000 7fba1ad9d700 1 -- 172.21.15.181:0/1978572373 learned_addr learned my addr 172.21.15.181:0/1978572373 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-09T06:30:19.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.657+0000 7fba1a59c700 1 --2- 172.21.15.181:0/1978572373 >> v2:172.21.15.163:3300/0 conn(0x7fba1c159c60 0x7fba1c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:19.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.658+0000 7fba19d9b700 1 -- 172.21.15.181:0/1978572373 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1426699168 0 0) 0x7fba1c15a160 con 0x7fba1c159890 2023-10-09T06:30:19.660 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.658+0000 7fba19d9b700 1 -- 172.21.15.181:0/1978572373 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba08003570 con 0x7fba1c159890 2023-10-09T06:30:19.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.658+0000 7fba1b59e700 1 -- 172.21.15.181:0/1978572373 >> v2:172.21.15.163:3300/0 conn(0x7fba1c159c60 msgr2=0x7fba1c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:19.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.658+0000 7fba1b59e700 1 --2- 172.21.15.181:0/1978572373 >> v2:172.21.15.163:3300/0 conn(0x7fba1c159c60 0x7fba1c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:19.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.658+0000 7fba1b59e700 1 -- 172.21.15.181:0/1978572373 >> v1:172.21.15.181:6789/0 conn(0x7fba1c159890 legacy=0x7fba1c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:19.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.658+0000 7fba1b59e700 1 -- 172.21.15.181:0/1978572373 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1c15e140 con 0x7fba1c15db10 2023-10-09T06:30:19.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.658+0000 7fba1b59e700 1 --2- 172.21.15.181:0/1978572373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba1c15db10 0x7fba1c069560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fba1c06a890 tx=0x7fba14009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=985197474801b8f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:19.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.658+0000 7fba19d9b700 1 -- 172.21.15.181:0/1978572373 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1400e070 con 0x7fba1c15db10 2023-10-09T06:30:19.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.658+0000 7fba19d9b700 1 -- 172.21.15.181:0/1978572373 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1400bab0 con 0x7fba1c15db10 2023-10-09T06:30:19.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.658+0000 7fba19d9b700 1 -- 172.21.15.181:0/1978572373 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1400be70 con 0x7fba1c15db10 2023-10-09T06:30:19.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.659+0000 7fba211a8700 1 -- 172.21.15.181:0/1978572373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba1c15db10 msgr2=0x7fba1c069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:19.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.659+0000 7fba211a8700 1 --2- 172.21.15.181:0/1978572373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba1c15db10 0x7fba1c069560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fba1c06a890 tx=0x7fba14009780 comp rx=0 tx=0).stop 2023-10-09T06:30:19.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.659+0000 7fba211a8700 1 -- 172.21.15.181:0/1978572373 shutdown_connections 2023-10-09T06:30:19.661 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.659+0000 7fba211a8700 1 --2- 172.21.15.181:0/1978572373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba1c15db10 0x7fba1c069560 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:19.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.659+0000 7fba211a8700 1 --2- 172.21.15.181:0/1978572373 >> v2:172.21.15.163:3300/0 conn(0x7fba1c159c60 0x7fba1c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:19.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.659+0000 7fba211a8700 1 -- 172.21.15.181:0/1978572373 >> 172.21.15.181:0/1978572373 conn(0x7fba1c068e20 msgr2=0x7fba1c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:19.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.659+0000 7fba211a8700 1 -- 172.21.15.181:0/1978572373 shutdown_connections 2023-10-09T06:30:19.662 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.659+0000 7fba211a8700 1 -- 172.21.15.181:0/1978572373 wait complete. 2023-10-09T06:30:19.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.660+0000 7fba211a8700 1 Processor -- start 2023-10-09T06:30:19.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.660+0000 7fba211a8700 1 -- start start 2023-10-09T06:30:19.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.661+0000 7fba211a8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba1c159890 0x7fba1c151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:19.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.661+0000 7fba211a8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba1c159c60 0x7fba1c0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:19.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.661+0000 7fba211a8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba1c15db10 0x7fba1c0c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:19.664 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.661+0000 7fba211a8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c0b4a70 con 0x7fba1c159c60 2023-10-09T06:30:19.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.661+0000 7fba211a8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c0cd0c0 con 0x7fba1c159890 2023-10-09T06:30:19.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.661+0000 7fba211a8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c0b14a0 con 0x7fba1c15db10 2023-10-09T06:30:19.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.661+0000 7fba1ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba1c159890 0x7fba1c151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:19.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.661+0000 7fba1ad9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba1c159890 0x7fba1c151b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59450/0 (socket says 172.21.15.181:59450) 2023-10-09T06:30:19.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.661+0000 7fba1ad9d700 1 -- 172.21.15.181:0/2260820599 learned_addr learned my addr 172.21.15.181:0/2260820599 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-09T06:30:19.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.661+0000 7fba1b59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba1c15db10 0x7fba1c0c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:19.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.661+0000 7fba1a59c700 1 --2- 172.21.15.181:0/2260820599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba1c159c60 0x7fba1c0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:19.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.661+0000 7fba1b59e700 1 -- 172.21.15.181:0/2260820599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba1c159890 msgr2=0x7fba1c151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:19.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.661+0000 7fba1b59e700 1 --2- 172.21.15.181:0/2260820599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba1c159890 0x7fba1c151b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:19.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.661+0000 7fba1b59e700 1 -- 172.21.15.181:0/2260820599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba1c159c60 msgr2=0x7fba1c0b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:19.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.661+0000 7fba1b59e700 1 --2- 172.21.15.181:0/2260820599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba1c159c60 0x7fba1c0b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:19.665 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.661+0000 7fba1b59e700 1 -- 172.21.15.181:0/2260820599 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba14009a20 con 0x7fba1c15db10 2023-10-09T06:30:19.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.661+0000 7fba1ad9d700 1 --2- 172.21.15.181:0/2260820599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba1c159890 0x7fba1c151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:30:19.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.661+0000 7fba1a59c700 1 --2- 172.21.15.181:0/2260820599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba1c159c60 0x7fba1c0b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:30:19.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.661+0000 7fba1b59e700 1 --2- 172.21.15.181:0/2260820599 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba1c15db10 0x7fba1c0c6640 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fba14016000 tx=0x7fba14019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:19.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.661+0000 7fba12ffd700 1 -- 172.21.15.181:0/2260820599 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba14012dd0 con 0x7fba1c15db10 2023-10-09T06:30:19.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.661+0000 7fba12ffd700 1 -- 172.21.15.181:0/2260820599 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba14027470 con 0x7fba1c15db10 2023-10-09T06:30:19.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.661+0000 7fba211a8700 1 -- 172.21.15.181:0/2260820599 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba1c0b0890 con 0x7fba1c15db10 2023-10-09T06:30:19.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.661+0000 7fba12ffd700 1 -- 172.21.15.181:0/2260820599 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba14027830 con 0x7fba1c15db10 2023-10-09T06:30:19.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.661+0000 7fba211a8700 1 -- 172.21.15.181:0/2260820599 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba1c0b0ed0 con 0x7fba1c15db10 2023-10-09T06:30:19.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.662+0000 7fba12ffd700 1 -- 172.21.15.181:0/2260820599 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fba14024070 con 0x7fba1c15db10 2023-10-09T06:30:19.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.662+0000 7fba12ffd700 1 -- 172.21.15.181:0/2260820599 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7fba1401dcd0 con 0x7fba1c15db10 2023-10-09T06:30:19.666 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.662+0000 7fba211a8700 1 -- 172.21.15.181:0/2260820599 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba1c0b1610 con 0x7fba1c15db10 2023-10-09T06:30:19.667 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.664+0000 7fba12ffd700 1 -- 172.21.15.181:0/2260820599 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fba140466d0 con 0x7fba1c15db10 2023-10-09T06:30:19.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.748+0000 7fba211a8700 1 -- 172.21.15.181:0/2260820599 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ae88b544-638e-469d-87a3-9646eed2b725", "id": 1} v 0) v1 -- 0x7fba1c04c3c0 con 0x7fba1c15db10 2023-10-09T06:30:19.754 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.753+0000 7fba12ffd700 1 -- 172.21.15.181:0/2260820599 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ae88b544-638e-469d-87a3-9646eed2b725", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fba14035030 con 0x7fba1c15db10 2023-10-09T06:30:19.754 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-09T06:30:19.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.754+0000 7fba211a8700 1 -- 172.21.15.181:0/2260820599 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba1c15db10 msgr2=0x7fba1c0c6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:19.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.754+0000 7fba211a8700 1 --2- 172.21.15.181:0/2260820599 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba1c15db10 0x7fba1c0c6640 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fba14016000 tx=0x7fba14019000 comp rx=0 tx=0).stop 2023-10-09T06:30:19.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.754+0000 7fba211a8700 1 -- 172.21.15.181:0/2260820599 shutdown_connections 2023-10-09T06:30:19.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.754+0000 7fba211a8700 1 --2- 172.21.15.181:0/2260820599 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba1c15db10 0x7fba1c0c6640 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:19.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.754+0000 7fba211a8700 1 --2- 172.21.15.181:0/2260820599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba1c159890 0x7fba1c151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:19.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.754+0000 7fba211a8700 1 --2- 172.21.15.181:0/2260820599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba1c159c60 0x7fba1c0b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:19.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.754+0000 7fba211a8700 1 -- 172.21.15.181:0/2260820599 >> 172.21.15.181:0/2260820599 conn(0x7fba1c068e20 msgr2=0x7fba1c15b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:19.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.754+0000 7fba211a8700 1 -- 172.21.15.181:0/2260820599 shutdown_connections 2023-10-09T06:30:19.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.754+0000 7fba211a8700 1 -- 172.21.15.181:0/2260820599 wait complete. 2023-10-09T06:30:19.766 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new e5d77274-5154-4ca7-8201-2cbf958775b3 2 2023-10-09T06:30:19.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.882+0000 7f37f6f88700 1 Processor -- start 2023-10-09T06:30:19.883 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.882+0000 7f37f6f88700 1 -- start start 2023-10-09T06:30:19.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.883+0000 7f37f6f88700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37f0159c60 0x7f37f015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:19.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.883+0000 7f37f6f88700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f37f015db10 0x7f37f0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:19.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.883+0000 7f37f6f88700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37f015a160 con 0x7f37f0159890 2023-10-09T06:30:19.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.883+0000 7f37f6f88700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37f00d13a0 con 0x7f37f015db10 2023-10-09T06:30:19.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.883+0000 7f37f6f88700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f015e140 con 0x7f37f0159c60 2023-10-09T06:30:19.886 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.883+0000 7f37effff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37f0159c60 0x7f37f015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:19.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.883+0000 7f37f5525700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f37f015db10 0x7f37f0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:19.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.883+0000 7f37effff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37f0159c60 0x7f37f015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:52280/0 (socket says 172.21.15.181:52280) 2023-10-09T06:30:19.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.883+0000 7f37effff700 1 -- 172.21.15.181:0/3772412967 learned_addr learned my addr 172.21.15.181:0/3772412967 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-09T06:30:19.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.884+0000 7f37effff700 1 -- 172.21.15.181:0/3772412967 >> v2:172.21.15.181:3300/0 conn(0x7f37f015db10 msgr2=0x7f37f0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:19.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.884+0000 7f37effff700 1 --2- 172.21.15.181:0/3772412967 >> v2:172.21.15.181:3300/0 conn(0x7f37f015db10 0x7f37f0069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:19.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.884+0000 7f37effff700 1 -- 172.21.15.181:0/3772412967 >> v1:172.21.15.181:6789/0 conn(0x7f37f0159890 legacy=0x7f37f015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:19.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.884+0000 7f37ef7fe700 1 -- 172.21.15.181:0/3772412967 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2557959171 0 0) 0x7f37f015a160 con 0x7f37f0159890 2023-10-09T06:30:19.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.884+0000 7f37effff700 1 -- 172.21.15.181:0/3772412967 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f015a3d0 con 0x7f37f0159c60 2023-10-09T06:30:19.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.884+0000 7f37f5525700 1 --2- 172.21.15.181:0/3772412967 >> v2:172.21.15.181:3300/0 conn(0x7f37f015db10 0x7f37f0069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:30:19.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.884+0000 7f37effff700 1 --2- 172.21.15.181:0/3772412967 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37f0159c60 0x7f37f015d5d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0099b0 tx=0x7f37dc009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58f813a7a9c6086c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:19.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.884+0000 7f37ef7fe700 1 -- 172.21.15.181:0/3772412967 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc009ea0 con 0x7f37f0159c60 2023-10-09T06:30:19.887 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.884+0000 7f37ef7fe700 1 -- 172.21.15.181:0/3772412967 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37dc00b860 con 0x7f37f0159c60 2023-10-09T06:30:19.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.884+0000 7f37ef7fe700 1 -- 172.21.15.181:0/3772412967 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc00bc00 con 0x7f37f0159c60 2023-10-09T06:30:19.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.885+0000 7f37f6f88700 1 -- 172.21.15.181:0/3772412967 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37f0159c60 msgr2=0x7f37f015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:19.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.885+0000 7f37f6f88700 1 --2- 172.21.15.181:0/3772412967 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37f0159c60 0x7f37f015d5d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0099b0 tx=0x7f37dc009250 comp rx=0 tx=0).stop 2023-10-09T06:30:19.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.885+0000 7f37f6f88700 1 -- 172.21.15.181:0/3772412967 shutdown_connections 2023-10-09T06:30:19.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.885+0000 7f37f6f88700 1 --2- 172.21.15.181:0/3772412967 >> v2:172.21.15.181:3300/0 conn(0x7f37f015db10 0x7f37f0069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:19.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.885+0000 7f37f6f88700 1 --2- 172.21.15.181:0/3772412967 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37f0159c60 0x7f37f015d5d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:19.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.885+0000 7f37f6f88700 1 -- 172.21.15.181:0/3772412967 >> 172.21.15.181:0/3772412967 conn(0x7f37f0068e20 msgr2=0x7f37f00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:19.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.885+0000 7f37f6f88700 1 -- 172.21.15.181:0/3772412967 shutdown_connections 2023-10-09T06:30:19.888 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.885+0000 7f37f6f88700 1 -- 172.21.15.181:0/3772412967 wait complete. 2023-10-09T06:30:19.890 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.886+0000 7f37f6f88700 1 Processor -- start 2023-10-09T06:30:19.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.886+0000 7f37f6f88700 1 -- start start 2023-10-09T06:30:19.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.887+0000 7f37f6f88700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37f0159890 0x7f37f00b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:19.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.887+0000 7f37f6f88700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37f0159c60 0x7f37f0140b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:19.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.887+0000 7f37f6f88700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37f015db10 0x7f37f00cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:19.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.887+0000 7f37f6f88700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f00b3c90 con 0x7f37f0159890 2023-10-09T06:30:19.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.887+0000 7f37f6f88700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f01410d0 con 0x7f37f015db10 2023-10-09T06:30:19.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.887+0000 7f37f6f88700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f0152820 con 0x7f37f0159c60 2023-10-09T06:30:19.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.887+0000 7f37effff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37f0159c60 0x7f37f0140b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:19.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.887+0000 7f37effff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37f0159c60 0x7f37f0140b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:52284/0 (socket says 172.21.15.181:52284) 2023-10-09T06:30:19.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.887+0000 7f37effff700 1 -- 172.21.15.181:0/3738554748 learned_addr learned my addr 172.21.15.181:0/3738554748 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-09T06:30:19.891 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.887+0000 7f37f5525700 1 --2- 172.21.15.181:0/3738554748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37f015db10 0x7f37f00cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:19.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.887+0000 7f37f4d24700 1 --2- 172.21.15.181:0/3738554748 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37f0159890 0x7f37f00b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:19.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.887+0000 7f37f5525700 1 -- 172.21.15.181:0/3738554748 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37f0159c60 msgr2=0x7f37f0140b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:19.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.887+0000 7f37f5525700 1 --2- 172.21.15.181:0/3738554748 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37f0159c60 0x7f37f0140b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:19.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.887+0000 7f37f5525700 1 -- 172.21.15.181:0/3738554748 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37f0159890 msgr2=0x7f37f00b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:19.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.887+0000 7f37f5525700 1 --2- 172.21.15.181:0/3738554748 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37f0159890 0x7f37f00b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:19.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.887+0000 7f37f5525700 1 -- 172.21.15.181:0/3738554748 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e8009a20 con 0x7f37f015db10 2023-10-09T06:30:19.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.887+0000 7f37effff700 1 --2- 172.21.15.181:0/3738554748 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37f0159c60 0x7f37f0140b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:30:19.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.887+0000 7f37f5525700 1 --2- 172.21.15.181:0/3738554748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37f015db10 0x7f37f00cd2d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f37e8009e70 tx=0x7f37e800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:19.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.888+0000 7f37ecff9700 1 -- 172.21.15.181:0/3738554748 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e800caa0 con 0x7f37f015db10 2023-10-09T06:30:19.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.888+0000 7f37ecff9700 1 -- 172.21.15.181:0/3738554748 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e800f040 con 0x7f37f015db10 2023-10-09T06:30:19.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.888+0000 7f37f6f88700 1 -- 172.21.15.181:0/3738554748 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37dc0094f0 con 0x7f37f015db10 2023-10-09T06:30:19.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.888+0000 7f37ecff9700 1 -- 172.21.15.181:0/3738554748 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e800ce90 con 0x7f37f015db10 2023-10-09T06:30:19.892 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.888+0000 7f37f6f88700 1 -- 172.21.15.181:0/3738554748 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f00b1100 con 0x7f37f015db10 2023-10-09T06:30:19.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.888+0000 7f37ecff9700 1 -- 172.21.15.181:0/3738554748 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f37e8003020 con 0x7f37f015db10 2023-10-09T06:30:19.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.888+0000 7f37ecff9700 1 -- 172.21.15.181:0/3738554748 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f37e8013810 con 0x7f37f015db10 2023-10-09T06:30:19.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.889+0000 7f37f6f88700 1 -- 172.21.15.181:0/3738554748 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f004c3c0 con 0x7f37f015db10 2023-10-09T06:30:19.893 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.891+0000 7f37ecff9700 1 -- 172.21.15.181:0/3738554748 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f37e8018030 con 0x7f37f015db10 2023-10-09T06:30:19.915 INFO:tasks.ceph.mgr.y.smithi163.stderr:2023-10-09T06:30:19.915+0000 7fc41d766180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-09T06:30:19.932 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:30:19.932+0000 7ff9dfa21180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-09T06:30:19.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.973+0000 7f37f6f88700 1 -- 172.21.15.181:0/3738554748 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e5d77274-5154-4ca7-8201-2cbf958775b3", "id": 2} v 0) v1 -- 0x7f37f006a5b0 con 0x7f37f015db10 2023-10-09T06:30:19.977 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.977+0000 7f37ecff9700 1 -- 172.21.15.181:0/3738554748 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e5d77274-5154-4ca7-8201-2cbf958775b3", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f37e8014870 con 0x7f37f015db10 2023-10-09T06:30:19.978 INFO:teuthology.orchestra.run.smithi181.stdout:2 2023-10-09T06:30:19.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.978+0000 7f37f6f88700 1 -- 172.21.15.181:0/3738554748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37f015db10 msgr2=0x7f37f00cd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:19.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.978+0000 7f37f6f88700 1 --2- 172.21.15.181:0/3738554748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37f015db10 0x7f37f00cd2d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f37e8009e70 tx=0x7f37e800b040 comp rx=0 tx=0).stop 2023-10-09T06:30:19.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.978+0000 7f37f6f88700 1 -- 172.21.15.181:0/3738554748 shutdown_connections 2023-10-09T06:30:19.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.978+0000 7f37f6f88700 1 --2- 172.21.15.181:0/3738554748 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37f0159890 0x7f37f00b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:19.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.978+0000 7f37f6f88700 1 --2- 172.21.15.181:0/3738554748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37f015db10 0x7f37f00cd2d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:19.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.978+0000 7f37f6f88700 1 --2- 172.21.15.181:0/3738554748 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37f0159c60 0x7f37f0140b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:19.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.978+0000 7f37f6f88700 1 -- 172.21.15.181:0/3738554748 >> 172.21.15.181:0/3738554748 conn(0x7f37f0068e20 msgr2=0x7f37f01573f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:19.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.978+0000 7f37f6f88700 1 -- 172.21.15.181:0/3738554748 shutdown_connections 2023-10-09T06:30:19.979 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:19.978+0000 7f37f6f88700 1 -- 172.21.15.181:0/3738554748 wait complete. 2023-10-09T06:30:19.990 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b 3 2023-10-09T06:30:20.050 INFO:tasks.ceph.mgr.y.smithi163.stderr:2023-10-09T06:30:20.050+0000 7fc41d766180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-09T06:30:20.071 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:30:20.071+0000 7ff9dfa21180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-09T06:30:20.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.104+0000 7f22792c5700 1 Processor -- start 2023-10-09T06:30:20.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.105+0000 7f22792c5700 1 -- start start 2023-10-09T06:30:20.108 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.105+0000 7f22792c5700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f227406c0a0 0x7f22740caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:20.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.105+0000 7f22792c5700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f227415d7e0 con 0x7f227406bcd0 2023-10-09T06:30:20.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.105+0000 7f22792c5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f227415dac0 con 0x7f227415df50 2023-10-09T06:30:20.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.105+0000 7f22792c5700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f227415aa30 con 0x7f227406c0a0 2023-10-09T06:30:20.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.105+0000 7f22737fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f227415df50 0x7f22740c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58360/0 (socket says 172.21.15.181:58360) 2023-10-09T06:30:20.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.105+0000 7f22737fe700 1 -- 172.21.15.181:0/1353570514 learned_addr learned my addr 172.21.15.181:0/1353570514 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-09T06:30:20.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.106+0000 7f22727fc700 1 --2- 172.21.15.181:0/1353570514 >> v2:172.21.15.163:3300/0 conn(0x7f227406c0a0 0x7f22740caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:20.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.106+0000 7f2271ffb700 1 -- 172.21.15.181:0/1353570514 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2928928440 0 0) 0x7f227415dac0 con 0x7f227415df50 2023-10-09T06:30:20.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.106+0000 7f2271ffb700 1 -- 172.21.15.181:0/1353570514 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f225c003570 con 0x7f227415df50 2023-10-09T06:30:20.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.106+0000 7f22727fc700 1 -- 172.21.15.181:0/1353570514 >> v1:172.21.15.181:6789/0 conn(0x7f227415df50 legacy=0x7f22740c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:20.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.106+0000 7f22727fc700 1 -- 172.21.15.181:0/1353570514 >> v1:172.21.15.163:6789/0 conn(0x7f227406bcd0 legacy=0x7f227415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:20.109 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.106+0000 7f22727fc700 1 -- 172.21.15.181:0/1353570514 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227415ad10 con 0x7f227406c0a0 2023-10-09T06:30:20.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.106+0000 7f2271ffb700 1 -- 172.21.15.181:0/1353570514 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1427771889 0 0) 0x7f227415d7e0 con 0x7f227406bcd0 2023-10-09T06:30:20.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.106+0000 7f22727fc700 1 --2- 172.21.15.181:0/1353570514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f227406c0a0 0x7f22740caa90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f22640098e0 tx=0x7f22640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f18252bd800a444 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:20.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.106+0000 7f2271ffb700 1 -- 172.21.15.181:0/1353570514 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226400b640 con 0x7f227406c0a0 2023-10-09T06:30:20.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.106+0000 7f2271ffb700 1 -- 172.21.15.181:0/1353570514 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226400b7a0 con 0x7f227406c0a0 2023-10-09T06:30:20.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.106+0000 7f2271ffb700 1 -- 172.21.15.181:0/1353570514 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226400bb40 con 0x7f227406c0a0 2023-10-09T06:30:20.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.107+0000 7f22792c5700 1 -- 172.21.15.181:0/1353570514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f227406c0a0 msgr2=0x7f22740caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:20.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.107+0000 7f22792c5700 1 --2- 172.21.15.181:0/1353570514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f227406c0a0 0x7f22740caa90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f22640098e0 tx=0x7f22640091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:20.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.107+0000 7f22792c5700 1 -- 172.21.15.181:0/1353570514 shutdown_connections 2023-10-09T06:30:20.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.107+0000 7f22792c5700 1 --2- 172.21.15.181:0/1353570514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f227406c0a0 0x7f22740caa90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:20.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.107+0000 7f22792c5700 1 -- 172.21.15.181:0/1353570514 >> 172.21.15.181:0/1353570514 conn(0x7f22740ce600 msgr2=0x7f22740cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:20.110 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.107+0000 7f22792c5700 1 -- 172.21.15.181:0/1353570514 shutdown_connections 2023-10-09T06:30:20.111 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.107+0000 7f22792c5700 1 -- 172.21.15.181:0/1353570514 wait complete. 2023-10-09T06:30:20.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.108+0000 7f22792c5700 1 Processor -- start 2023-10-09T06:30:20.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.108+0000 7f22792c5700 1 -- start start 2023-10-09T06:30:20.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.109+0000 7f22792c5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f227406bcd0 0x7f2274144ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:20.113 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.109+0000 7f22792c5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f227406c0a0 0x7f2274153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:20.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.109+0000 7f22792c5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f227415df50 0x7f2274155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:20.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.109+0000 7f22792c5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22740b2270 con 0x7f227406c0a0 2023-10-09T06:30:20.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.109+0000 7f22792c5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22740b24c0 con 0x7f227415df50 2023-10-09T06:30:20.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.109+0000 7f22792c5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22740b27a0 con 0x7f227406bcd0 2023-10-09T06:30:20.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.109+0000 7f22737fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f227415df50 0x7f2274155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:20.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.109+0000 7f2272ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f227406bcd0 0x7f2274144ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:20.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.109+0000 7f22737fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f227415df50 0x7f2274155830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59480/0 (socket says 172.21.15.181:59480) 2023-10-09T06:30:20.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.109+0000 7f2272ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f227406bcd0 0x7f2274144ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:52292/0 (socket says 172.21.15.181:52292) 2023-10-09T06:30:20.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.109+0000 7f22737fe700 1 -- 172.21.15.181:0/1693534622 learned_addr learned my addr 172.21.15.181:0/1693534622 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-09T06:30:20.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.109+0000 7f22727fc700 1 --2- 172.21.15.181:0/1693534622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f227406c0a0 0x7f2274153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:20.114 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.109+0000 7f2272ffd700 1 -- 172.21.15.181:0/1693534622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f227415df50 msgr2=0x7f2274155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:20.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.109+0000 7f2272ffd700 1 --2- 172.21.15.181:0/1693534622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f227415df50 0x7f2274155830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:20.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.109+0000 7f2272ffd700 1 -- 172.21.15.181:0/1693534622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f227406c0a0 msgr2=0x7f2274153b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:20.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.109+0000 7f2272ffd700 1 --2- 172.21.15.181:0/1693534622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f227406c0a0 0x7f2274153b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:20.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.109+0000 7f2272ffd700 1 -- 172.21.15.181:0/1693534622 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2264009420 con 0x7f227406bcd0 2023-10-09T06:30:20.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.109+0000 7f22737fe700 1 --2- 172.21.15.181:0/1693534622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f227415df50 0x7f2274155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:30:20.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.109+0000 7f2272ffd700 1 --2- 172.21.15.181:0/1693534622 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f227406bcd0 0x7f2274144ac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f226800c510 tx=0x7f2268007ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:20.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.109+0000 7f225affd700 1 -- 172.21.15.181:0/1693534622 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226800ce80 con 0x7f227406bcd0 2023-10-09T06:30:20.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.109+0000 7f225affd700 1 -- 172.21.15.181:0/1693534622 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2268011590 con 0x7f227406bcd0 2023-10-09T06:30:20.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.109+0000 7f225affd700 1 -- 172.21.15.181:0/1693534622 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2268011950 con 0x7f227406bcd0 2023-10-09T06:30:20.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.109+0000 7f22792c5700 1 -- 172.21.15.181:0/1693534622 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22740b2a80 con 0x7f227406bcd0 2023-10-09T06:30:20.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.110+0000 7f22792c5700 1 -- 172.21.15.181:0/1693534622 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22740b3040 con 0x7f227406bcd0 2023-10-09T06:30:20.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.110+0000 7f225affd700 1 -- 172.21.15.181:0/1693534622 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f226801c440 con 0x7f227406bcd0 2023-10-09T06:30:20.115 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.110+0000 7f225affd700 1 -- 172.21.15.181:0/1693534622 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f2268001010 con 0x7f227406bcd0 2023-10-09T06:30:20.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.110+0000 7f22792c5700 1 -- 172.21.15.181:0/1693534622 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f227404c3c0 con 0x7f227406bcd0 2023-10-09T06:30:20.116 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.112+0000 7f225affd700 1 -- 172.21.15.181:0/1693534622 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f2268017070 con 0x7f227406bcd0 2023-10-09T06:30:20.129 INFO:tasks.ceph.mgr.y.smithi163.stderr:2023-10-09T06:30:20.128+0000 7fc41d766180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-09T06:30:20.150 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:30:20.150+0000 7ff9dfa21180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-09T06:30:20.195 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.195+0000 7f22792c5700 1 -- 172.21.15.181:0/1693534622 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b", "id": 3} v 0) v1 -- 0x7f227415aff0 con 0x7f227406bcd0 2023-10-09T06:30:20.199 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.198+0000 7f225affd700 1 -- 172.21.15.181:0/1693534622 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f226801c9c0 con 0x7f227406bcd0 2023-10-09T06:30:20.199 INFO:teuthology.orchestra.run.smithi181.stdout:3 2023-10-09T06:30:20.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.200+0000 7f22792c5700 1 -- 172.21.15.181:0/1693534622 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f227406bcd0 msgr2=0x7f2274144ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:20.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.200+0000 7f22792c5700 1 --2- 172.21.15.181:0/1693534622 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f227406bcd0 0x7f2274144ac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f226800c510 tx=0x7f2268007ec0 comp rx=0 tx=0).stop 2023-10-09T06:30:20.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.200+0000 7f22792c5700 1 -- 172.21.15.181:0/1693534622 shutdown_connections 2023-10-09T06:30:20.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.200+0000 7f22792c5700 1 --2- 172.21.15.181:0/1693534622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f227415df50 0x7f2274155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:20.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.200+0000 7f22792c5700 1 --2- 172.21.15.181:0/1693534622 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f227406bcd0 0x7f2274144ac0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:20.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.200+0000 7f22792c5700 1 --2- 172.21.15.181:0/1693534622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f227406c0a0 0x7f2274153b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:20.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.200+0000 7f22792c5700 1 -- 172.21.15.181:0/1693534622 >> 172.21.15.181:0/1693534622 conn(0x7f22740ce600 msgr2=0x7f227415bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:20.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.200+0000 7f22792c5700 1 -- 172.21.15.181:0/1693534622 shutdown_connections 2023-10-09T06:30:20.201 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.200+0000 7f22792c5700 1 -- 172.21.15.181:0/1693534622 wait complete. 2023-10-09T06:30:20.213 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 4440f08d-6323-44ad-a0db-fd7e5ddf7424 4 2023-10-09T06:30:20.318 INFO:tasks.ceph.mgr.y.smithi163.stderr:2023-10-09T06:30:20.316+0000 7fc41d766180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-09T06:30:20.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.328+0000 7f9c18dcc700 1 Processor -- start 2023-10-09T06:30:20.329 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.328+0000 7f9c18dcc700 1 -- start start 2023-10-09T06:30:20.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.329+0000 7f9c18dcc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9c1406aa40 0x7f9c140cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:20.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.329+0000 7f9c18dcc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9c140c91e0 0x7f9c14160390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:20.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.329+0000 7f9c18dcc700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c140cc210 con 0x7f9c1406bac0 2023-10-09T06:30:20.338 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.329+0000 7f9c18dcc700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c140cc4f0 con 0x7f9c1406aa40 2023-10-09T06:30:20.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.329+0000 7f9c18dcc700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c140cc7d0 con 0x7f9c140c91e0 2023-10-09T06:30:20.339 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.329+0000 7f9c12d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9c140c91e0 0x7f9c14160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:20.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.329+0000 7f9c12d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9c140c91e0 0x7f9c14160390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59488/0 (socket says 172.21.15.181:59488) 2023-10-09T06:30:20.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.329+0000 7f9c12d9d700 1 -- 172.21.15.181:0/4036287651 learned_addr learned my addr 172.21.15.181:0/4036287651 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-09T06:30:20.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.329+0000 7f9c1259c700 1 --2- 172.21.15.181:0/4036287651 >> v2:172.21.15.163:3300/0 conn(0x7f9c1406aa40 0x7f9c140cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:20.340 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.329+0000 7f9c12d9d700 1 -- 172.21.15.181:0/4036287651 >> v2:172.21.15.163:3300/0 conn(0x7f9c1406aa40 msgr2=0x7f9c140cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:20.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.329+0000 7f9c12d9d700 1 --2- 172.21.15.181:0/4036287651 >> v2:172.21.15.163:3300/0 conn(0x7f9c1406aa40 0x7f9c140cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:20.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.329+0000 7f9c12d9d700 1 -- 172.21.15.181:0/4036287651 >> v1:172.21.15.163:6789/0 conn(0x7f9c1406bac0 legacy=0x7f9c1406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:20.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.329+0000 7f9c12d9d700 1 -- 172.21.15.181:0/4036287651 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c140ccab0 con 0x7f9c140c91e0 2023-10-09T06:30:20.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.329+0000 7f9c12d9d700 1 --2- 172.21.15.181:0/4036287651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c140c91e0 0x7f9c14160390 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9c1406e510 tx=0x7f9c0c00ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4b1a7828414e959 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:20.341 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.330+0000 7f9c1159a700 1 -- 172.21.15.181:0/4036287651 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c010070 con 0x7f9c140c91e0 2023-10-09T06:30:20.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.330+0000 7f9c1159a700 1 -- 172.21.15.181:0/4036287651 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0c00dde0 con 0x7f9c140c91e0 2023-10-09T06:30:20.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.330+0000 7f9c1159a700 1 -- 172.21.15.181:0/4036287651 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c00b320 con 0x7f9c140c91e0 2023-10-09T06:30:20.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.330+0000 7f9c18dcc700 1 -- 172.21.15.181:0/4036287651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c140c91e0 msgr2=0x7f9c14160390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:20.342 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.330+0000 7f9c18dcc700 1 --2- 172.21.15.181:0/4036287651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c140c91e0 0x7f9c14160390 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9c1406e510 tx=0x7f9c0c00ba50 comp rx=0 tx=0).stop 2023-10-09T06:30:20.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.330+0000 7f9c18dcc700 1 -- 172.21.15.181:0/4036287651 shutdown_connections 2023-10-09T06:30:20.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.330+0000 7f9c18dcc700 1 --2- 172.21.15.181:0/4036287651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c140c91e0 0x7f9c14160390 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:20.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.330+0000 7f9c18dcc700 1 --2- 172.21.15.181:0/4036287651 >> v2:172.21.15.163:3300/0 conn(0x7f9c1406aa40 0x7f9c140cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:20.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.330+0000 7f9c18dcc700 1 -- 172.21.15.181:0/4036287651 >> 172.21.15.181:0/4036287651 conn(0x7f9c140cd670 msgr2=0x7f9c140cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:20.343 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.330+0000 7f9c18dcc700 1 -- 172.21.15.181:0/4036287651 shutdown_connections 2023-10-09T06:30:20.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.330+0000 7f9c18dcc700 1 -- 172.21.15.181:0/4036287651 wait complete. 2023-10-09T06:30:20.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.331+0000 7f9c18dcc700 1 Processor -- start 2023-10-09T06:30:20.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.331+0000 7f9c18dcc700 1 -- start start 2023-10-09T06:30:20.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.332+0000 7f9c18dcc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c1406aa40 0x7f9c14159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:20.344 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.332+0000 7f9c18dcc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c1406bac0 0x7f9c14165b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:20.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.332+0000 7f9c18dcc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c140c91e0 0x7f9c141660c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:20.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.332+0000 7f9c18dcc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c141598d0 con 0x7f9c1406aa40 2023-10-09T06:30:20.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.332+0000 7f9c18dcc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c14159bb0 con 0x7f9c140c91e0 2023-10-09T06:30:20.345 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.332+0000 7f9c18dcc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c14159e90 con 0x7f9c1406bac0 2023-10-09T06:30:20.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.332+0000 7f9c12d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c140c91e0 0x7f9c141660c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:20.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.332+0000 7f9c12d9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c140c91e0 0x7f9c141660c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59492/0 (socket says 172.21.15.181:59492) 2023-10-09T06:30:20.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.332+0000 7f9c11d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c1406bac0 0x7f9c14165b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:20.346 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.332+0000 7f9c12d9d700 1 -- 172.21.15.181:0/4210885270 learned_addr learned my addr 172.21.15.181:0/4210885270 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-09T06:30:20.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.332+0000 7f9c1259c700 1 --2- 172.21.15.181:0/4210885270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c1406aa40 0x7f9c14159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:20.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.333+0000 7f9c12d9d700 1 -- 172.21.15.181:0/4210885270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c1406bac0 msgr2=0x7f9c14165b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:20.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.333+0000 7f9c12d9d700 1 --2- 172.21.15.181:0/4210885270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c1406bac0 0x7f9c14165b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:20.347 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.333+0000 7f9c12d9d700 1 -- 172.21.15.181:0/4210885270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c1406aa40 msgr2=0x7f9c14159080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:20.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.333+0000 7f9c12d9d700 1 --2- 172.21.15.181:0/4210885270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c1406aa40 0x7f9c14159080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:20.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.333+0000 7f9c12d9d700 1 -- 172.21.15.181:0/4210885270 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0c00bcf0 con 0x7f9c140c91e0 2023-10-09T06:30:20.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.333+0000 7f9c12d9d700 1 --2- 172.21.15.181:0/4210885270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c140c91e0 0x7f9c141660c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9c140cc910 tx=0x7f9c0c01a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:20.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.333+0000 7f9c027fc700 1 -- 172.21.15.181:0/4210885270 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c00b4d0 con 0x7f9c140c91e0 2023-10-09T06:30:20.348 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.333+0000 7f9c027fc700 1 -- 172.21.15.181:0/4210885270 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0c01e410 con 0x7f9c140c91e0 2023-10-09T06:30:20.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.333+0000 7f9c18dcc700 1 -- 172.21.15.181:0/4210885270 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c1415a170 con 0x7f9c140c91e0 2023-10-09T06:30:20.349 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.333+0000 7f9c027fc700 1 -- 172.21.15.181:0/4210885270 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c01e7d0 con 0x7f9c140c91e0 2023-10-09T06:30:20.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.333+0000 7f9c18dcc700 1 -- 172.21.15.181:0/4210885270 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c141595c0 con 0x7f9c140c91e0 2023-10-09T06:30:20.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.333+0000 7f9c027fc700 1 -- 172.21.15.181:0/4210885270 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f9c0c003040 con 0x7f9c140c91e0 2023-10-09T06:30:20.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.333+0000 7f9c027fc700 1 -- 172.21.15.181:0/4210885270 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f9c0c028960 con 0x7f9c140c91e0 2023-10-09T06:30:20.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.334+0000 7f9c18dcc700 1 -- 172.21.15.181:0/4210885270 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c1404c3c0 con 0x7f9c140c91e0 2023-10-09T06:30:20.350 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.336+0000 7f9c027fc700 1 -- 172.21.15.181:0/4210885270 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f9c0c02d060 con 0x7f9c140c91e0 2023-10-09T06:30:20.351 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:30:20.344+0000 7ff9dfa21180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-09T06:30:20.420 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.419+0000 7f9c18dcc700 1 -- 172.21.15.181:0/4210885270 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4440f08d-6323-44ad-a0db-fd7e5ddf7424", "id": 4} v 0) v1 -- 0x7f9c140c7cb0 con 0x7f9c140c91e0 2023-10-09T06:30:20.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.423+0000 7f9c027fc700 1 -- 172.21.15.181:0/4210885270 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4440f08d-6323-44ad-a0db-fd7e5ddf7424", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9c0c023070 con 0x7f9c140c91e0 2023-10-09T06:30:20.424 INFO:teuthology.orchestra.run.smithi181.stdout:4 2023-10-09T06:30:20.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.424+0000 7f9c18dcc700 1 -- 172.21.15.181:0/4210885270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c140c91e0 msgr2=0x7f9c141660c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:20.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.424+0000 7f9c18dcc700 1 --2- 172.21.15.181:0/4210885270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c140c91e0 0x7f9c141660c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9c140cc910 tx=0x7f9c0c01a000 comp rx=0 tx=0).stop 2023-10-09T06:30:20.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.424+0000 7f9c18dcc700 1 -- 172.21.15.181:0/4210885270 shutdown_connections 2023-10-09T06:30:20.425 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.424+0000 7f9c18dcc700 1 --2- 172.21.15.181:0/4210885270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c1406aa40 0x7f9c14159080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:20.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.424+0000 7f9c18dcc700 1 --2- 172.21.15.181:0/4210885270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c140c91e0 0x7f9c141660c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:20.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.424+0000 7f9c18dcc700 1 --2- 172.21.15.181:0/4210885270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c1406bac0 0x7f9c14165b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:20.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.424+0000 7f9c18dcc700 1 -- 172.21.15.181:0/4210885270 >> 172.21.15.181:0/4210885270 conn(0x7f9c140cd670 msgr2=0x7f9c140c3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:20.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.424+0000 7f9c18dcc700 1 -- 172.21.15.181:0/4210885270 shutdown_connections 2023-10-09T06:30:20.426 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.424+0000 7f9c18dcc700 1 -- 172.21.15.181:0/4210885270 wait complete. 2023-10-09T06:30:20.436 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 38a04aeb-2406-4cbd-953c-f7a762102715 5 2023-10-09T06:30:20.494 INFO:tasks.ceph.mgr.y.smithi163.stderr:2023-10-09T06:30:20.493+0000 7fc41d766180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-09T06:30:20.526 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:30:20.525+0000 7ff9dfa21180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-09T06:30:20.553 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.552+0000 7ff7bd082700 1 Processor -- start 2023-10-09T06:30:20.554 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.552+0000 7ff7bd082700 1 -- start start 2023-10-09T06:30:20.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.553+0000 7ff7bd082700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7b8068aa0 0x7ff7b8162820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:20.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.553+0000 7ff7bd082700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7b80c7de0 con 0x7ff7b8140350 2023-10-09T06:30:20.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.553+0000 7ff7bd082700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7b8140c10 con 0x7ff7b8140720 2023-10-09T06:30:20.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.553+0000 7ff7bd082700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b8141490 con 0x7ff7b8068aa0 2023-10-09T06:30:20.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.553+0000 7ff7b759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7b8068aa0 0x7ff7b8162820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:20.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.553+0000 7ff7b659c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff7b8140720 0x7ff7b8068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58372/0 (socket says 172.21.15.181:58372) 2023-10-09T06:30:20.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.553+0000 7ff7b659c700 1 -- 172.21.15.181:0/1235179714 learned_addr learned my addr 172.21.15.181:0/1235179714 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-09T06:30:20.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.554+0000 7ff7b759e700 1 -- 172.21.15.181:0/1235179714 >> v1:172.21.15.181:6789/0 conn(0x7ff7b8140720 legacy=0x7ff7b8068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:20.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.554+0000 7ff7b759e700 1 -- 172.21.15.181:0/1235179714 >> v1:172.21.15.163:6789/0 conn(0x7ff7b8140350 legacy=0x7ff7b8069c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:20.555 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.554+0000 7ff7b5d9b700 1 -- 172.21.15.181:0/1235179714 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1614543351 0 0) 0x7ff7b8140c10 con 0x7ff7b8140720 2023-10-09T06:30:20.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.554+0000 7ff7b759e700 1 -- 172.21.15.181:0/1235179714 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b8140ed0 con 0x7ff7b8068aa0 2023-10-09T06:30:20.556 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.554+0000 7ff7b759e700 1 --2- 172.21.15.181:0/1235179714 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7b8068aa0 0x7ff7b8162820 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0009e90 tx=0x7ff7b0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=333f1fa69d72117e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:20.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.554+0000 7ff7b5d9b700 1 -- 172.21.15.181:0/1235179714 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b000e070 con 0x7ff7b8068aa0 2023-10-09T06:30:20.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.554+0000 7ff7b5d9b700 1 -- 172.21.15.181:0/1235179714 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b000bc30 con 0x7ff7b8068aa0 2023-10-09T06:30:20.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.554+0000 7ff7b5d9b700 1 -- 172.21.15.181:0/1235179714 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b0012430 con 0x7ff7b8068aa0 2023-10-09T06:30:20.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.554+0000 7ff7bd082700 1 -- 172.21.15.181:0/1235179714 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7b8068aa0 msgr2=0x7ff7b8162820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:20.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.554+0000 7ff7bd082700 1 --2- 172.21.15.181:0/1235179714 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7b8068aa0 0x7ff7b8162820 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0009e90 tx=0x7ff7b0009780 comp rx=0 tx=0).stop 2023-10-09T06:30:20.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.554+0000 7ff7bd082700 1 -- 172.21.15.181:0/1235179714 shutdown_connections 2023-10-09T06:30:20.557 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.554+0000 7ff7bd082700 1 --2- 172.21.15.181:0/1235179714 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7b8068aa0 0x7ff7b8162820 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:20.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.555+0000 7ff7bd082700 1 -- 172.21.15.181:0/1235179714 >> 172.21.15.181:0/1235179714 conn(0x7ff7b80cd670 msgr2=0x7ff7b80cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:20.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.555+0000 7ff7bd082700 1 -- 172.21.15.181:0/1235179714 shutdown_connections 2023-10-09T06:30:20.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.555+0000 7ff7bd082700 1 -- 172.21.15.181:0/1235179714 wait complete. 2023-10-09T06:30:20.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.556+0000 7ff7bd082700 1 Processor -- start 2023-10-09T06:30:20.558 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.556+0000 7ff7bd082700 1 -- start start 2023-10-09T06:30:20.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.557+0000 7ff7bd082700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7b8068aa0 0x7ff7b816a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:20.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.557+0000 7ff7bd082700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7b8140350 0x7ff7b81690e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:20.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.557+0000 7ff7bd082700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7b8140720 0x7ff7b813cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:20.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.557+0000 7ff7bd082700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b80ae590 con 0x7ff7b8068aa0 2023-10-09T06:30:20.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.557+0000 7ff7bd082700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b816a7c0 con 0x7ff7b8140350 2023-10-09T06:30:20.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.557+0000 7ff7bd082700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b80ae7a0 con 0x7ff7b8140720 2023-10-09T06:30:20.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.557+0000 7ff7b659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7b8140350 0x7ff7b81690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:20.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.557+0000 7ff7b759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7b8140720 0x7ff7b813cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:20.561 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.557+0000 7ff7b659c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7b8140350 0x7ff7b81690e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59502/0 (socket says 172.21.15.181:59502) 2023-10-09T06:30:20.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.557+0000 7ff7b759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7b8140720 0x7ff7b813cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:52316/0 (socket says 172.21.15.181:52316) 2023-10-09T06:30:20.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.557+0000 7ff7b659c700 1 -- 172.21.15.181:0/21761899 learned_addr learned my addr 172.21.15.181:0/21761899 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-09T06:30:20.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.557+0000 7ff7b6d9d700 1 --2- 172.21.15.181:0/21761899 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7b8068aa0 0x7ff7b816a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:20.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.557+0000 7ff7b759e700 1 -- 172.21.15.181:0/21761899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7b8140350 msgr2=0x7ff7b81690e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:20.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.557+0000 7ff7b759e700 1 --2- 172.21.15.181:0/21761899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7b8140350 0x7ff7b81690e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:20.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.557+0000 7ff7b759e700 1 -- 172.21.15.181:0/21761899 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7b8068aa0 msgr2=0x7ff7b816a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:20.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.557+0000 7ff7b759e700 1 --2- 172.21.15.181:0/21761899 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7b8068aa0 0x7ff7b816a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:20.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.557+0000 7ff7b759e700 1 -- 172.21.15.181:0/21761899 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b0009a20 con 0x7ff7b8140720 2023-10-09T06:30:20.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.557+0000 7ff7b759e700 1 --2- 172.21.15.181:0/21761899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7b8140720 0x7ff7b813cdb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0016000 tx=0x7ff7b0019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:20.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.557+0000 7ff7aeffd700 1 -- 172.21.15.181:0/21761899 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b001b420 con 0x7ff7b8140720 2023-10-09T06:30:20.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.557+0000 7ff7aeffd700 1 -- 172.21.15.181:0/21761899 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b001b580 con 0x7ff7b8140720 2023-10-09T06:30:20.562 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.557+0000 7ff7aeffd700 1 -- 172.21.15.181:0/21761899 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b001b920 con 0x7ff7b8140720 2023-10-09T06:30:20.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.557+0000 7ff7bd082700 1 -- 172.21.15.181:0/21761899 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b80aea80 con 0x7ff7b8140720 2023-10-09T06:30:20.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.557+0000 7ff7bd082700 1 -- 172.21.15.181:0/21761899 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b80af040 con 0x7ff7b8140720 2023-10-09T06:30:20.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.557+0000 7ff7aeffd700 1 -- 172.21.15.181:0/21761899 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7ff7b001bd50 con 0x7ff7b8140720 2023-10-09T06:30:20.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.558+0000 7ff7aeffd700 1 -- 172.21.15.181:0/21761899 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7ff7b00253f0 con 0x7ff7b8140720 2023-10-09T06:30:20.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.558+0000 7ff7bd082700 1 -- 172.21.15.181:0/21761899 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b804c3c0 con 0x7ff7b8140720 2023-10-09T06:30:20.563 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.562+0000 7ff7aeffd700 1 -- 172.21.15.181:0/21761899 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7ff7b002a070 con 0x7ff7b8140720 2023-10-09T06:30:20.572 INFO:tasks.ceph.mgr.y.smithi163.stderr:2023-10-09T06:30:20.571+0000 7fc41d766180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-09T06:30:20.608 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:30:20.607+0000 7ff9dfa21180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-09T06:30:20.645 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.644+0000 7ff7bd082700 1 -- 172.21.15.181:0/21761899 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd new", "uuid": "38a04aeb-2406-4cbd-953c-f7a762102715", "id": 5} v 0) v1 -- 0x7ff7b81411b0 con 0x7ff7b8140720 2023-10-09T06:30:20.648 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.647+0000 7ff7aeffd700 1 -- 172.21.15.181:0/21761899 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "38a04aeb-2406-4cbd-953c-f7a762102715", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff7b001c400 con 0x7ff7b8140720 2023-10-09T06:30:20.648 INFO:teuthology.orchestra.run.smithi181.stdout:5 2023-10-09T06:30:20.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.648+0000 7ff7bd082700 1 -- 172.21.15.181:0/21761899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7b8140720 msgr2=0x7ff7b813cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:20.649 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.648+0000 7ff7bd082700 1 --2- 172.21.15.181:0/21761899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7b8140720 0x7ff7b813cdb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0016000 tx=0x7ff7b0019000 comp rx=0 tx=0).stop 2023-10-09T06:30:20.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.649+0000 7ff7bd082700 1 -- 172.21.15.181:0/21761899 shutdown_connections 2023-10-09T06:30:20.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.649+0000 7ff7bd082700 1 --2- 172.21.15.181:0/21761899 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7b8068aa0 0x7ff7b816a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:20.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.649+0000 7ff7bd082700 1 --2- 172.21.15.181:0/21761899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7b8140720 0x7ff7b813cdb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:20.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.649+0000 7ff7bd082700 1 --2- 172.21.15.181:0/21761899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7b8140350 0x7ff7b81690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:20.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.649+0000 7ff7bd082700 1 -- 172.21.15.181:0/21761899 >> 172.21.15.181:0/21761899 conn(0x7ff7b80cd670 msgr2=0x7ff7b80cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:20.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.649+0000 7ff7bd082700 1 -- 172.21.15.181:0/21761899 shutdown_connections 2023-10-09T06:30:20.650 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.649+0000 7ff7bd082700 1 -- 172.21.15.181:0/21761899 wait complete. 2023-10-09T06:30:20.662 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 4023515b-10ff-428d-ab92-acad13005e9f 6 2023-10-09T06:30:20.709 INFO:tasks.ceph.mgr.y.smithi163.stderr:2023-10-09T06:30:20.708+0000 7fc41d766180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-09T06:30:20.750 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:30:20.750+0000 7ff9dfa21180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-09T06:30:20.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.778+0000 7f59033c6700 1 Processor -- start 2023-10-09T06:30:20.779 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.778+0000 7f59033c6700 1 -- start start 2023-10-09T06:30:20.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.779+0000 7f59033c6700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f58fc0b16d0 0x7f58fc0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:20.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.779+0000 7f59033c6700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58fc0b2080 con 0x7f58fc0b4e70 2023-10-09T06:30:20.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.779+0000 7f59033c6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58fc0cfb00 con 0x7f58fc0b33b0 2023-10-09T06:30:20.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.779+0000 7f59033c6700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58fc0b5c40 con 0x7f58fc0b16d0 2023-10-09T06:30:20.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.779+0000 7f5902bc5700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f58fc0b33b0 0x7f58fc0b37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58374/0 (socket says 172.21.15.181:58374) 2023-10-09T06:30:20.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.779+0000 7f5902bc5700 1 -- 172.21.15.181:0/1007580217 learned_addr learned my addr 172.21.15.181:0/1007580217 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-09T06:30:20.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.779+0000 7f5901bc3700 1 --2- 172.21.15.181:0/1007580217 >> v2:172.21.15.163:3300/0 conn(0x7f58fc0b16d0 0x7f58fc0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:20.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.779+0000 7f59013c2700 1 -- 172.21.15.181:0/1007580217 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1394007800 0 0) 0x7f58fc0cfb00 con 0x7f58fc0b33b0 2023-10-09T06:30:20.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.779+0000 7f59013c2700 1 -- 172.21.15.181:0/1007580217 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58e8003570 con 0x7f58fc0b33b0 2023-10-09T06:30:20.781 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.779+0000 7f5901bc3700 1 -- 172.21.15.181:0/1007580217 >> v1:172.21.15.181:6789/0 conn(0x7f58fc0b33b0 legacy=0x7f58fc0b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:20.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.780+0000 7f5901bc3700 1 -- 172.21.15.181:0/1007580217 >> v1:172.21.15.163:6789/0 conn(0x7f58fc0b4e70 legacy=0x7f58fc0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:20.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.780+0000 7f5901bc3700 1 -- 172.21.15.181:0/1007580217 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58fc0b5f20 con 0x7f58fc0b16d0 2023-10-09T06:30:20.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.780+0000 7f5901bc3700 1 --2- 172.21.15.181:0/1007580217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58fc0b16d0 0x7f58fc0b1b40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f58f80098e0 tx=0x7f58f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96effa28ee63b138 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:20.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.780+0000 7f59013c2700 1 -- 172.21.15.181:0/1007580217 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f8011630 con 0x7f58fc0b16d0 2023-10-09T06:30:20.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.780+0000 7f59013c2700 1 -- 172.21.15.181:0/1007580217 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f8011790 con 0x7f58fc0b16d0 2023-10-09T06:30:20.782 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.780+0000 7f59013c2700 1 -- 172.21.15.181:0/1007580217 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f8011b30 con 0x7f58fc0b16d0 2023-10-09T06:30:20.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.780+0000 7f59033c6700 1 -- 172.21.15.181:0/1007580217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58fc0b16d0 msgr2=0x7f58fc0b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:20.784 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.780+0000 7f59033c6700 1 --2- 172.21.15.181:0/1007580217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58fc0b16d0 0x7f58fc0b1b40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f58f80098e0 tx=0x7f58f80091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:20.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.781+0000 7f59033c6700 1 -- 172.21.15.181:0/1007580217 shutdown_connections 2023-10-09T06:30:20.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.781+0000 7f59033c6700 1 --2- 172.21.15.181:0/1007580217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58fc0b16d0 0x7f58fc0b1b40 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:20.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.781+0000 7f59033c6700 1 -- 172.21.15.181:0/1007580217 >> 172.21.15.181:0/1007580217 conn(0x7f58fc0d10c0 msgr2=0x7f58fc0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:20.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.781+0000 7f59033c6700 1 -- 172.21.15.181:0/1007580217 shutdown_connections 2023-10-09T06:30:20.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.781+0000 7f59033c6700 1 -- 172.21.15.181:0/1007580217 wait complete. 2023-10-09T06:30:20.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.782+0000 7f59033c6700 1 Processor -- start 2023-10-09T06:30:20.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.782+0000 7f59033c6700 1 -- start start 2023-10-09T06:30:20.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.782+0000 7f59033c6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58fc0b16d0 0x7f58fc0a7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:20.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.782+0000 7f59033c6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58fc0b33b0 0x7f58fc0a8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:20.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.782+0000 7f59033c6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58fc0b4e70 0x7f58fc169b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:20.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.782+0000 7f59033c6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58fc0cafa0 con 0x7f58fc0b4e70 2023-10-09T06:30:20.785 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.782+0000 7f59033c6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58fc0a8cc0 con 0x7f58fc0b33b0 2023-10-09T06:30:20.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.782+0000 7f59033c6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58fc0a8fa0 con 0x7f58fc0b16d0 2023-10-09T06:30:20.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.782+0000 7f5901bc3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58fc0b33b0 0x7f58fc0a8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:20.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.782+0000 7f5901bc3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58fc0b33b0 0x7f58fc0a8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59516/0 (socket says 172.21.15.181:59516) 2023-10-09T06:30:20.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.782+0000 7f5901bc3700 1 -- 172.21.15.181:0/4278680473 learned_addr learned my addr 172.21.15.181:0/4278680473 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-09T06:30:20.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.782+0000 7f59023c4700 1 --2- 172.21.15.181:0/4278680473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58fc0b16d0 0x7f58fc0a7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:20.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.782+0000 7f5902bc5700 1 --2- 172.21.15.181:0/4278680473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58fc0b4e70 0x7f58fc169b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:20.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.783+0000 7f5901bc3700 1 -- 172.21.15.181:0/4278680473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58fc0b16d0 msgr2=0x7f58fc0a7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:20.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.783+0000 7f5901bc3700 1 --2- 172.21.15.181:0/4278680473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58fc0b16d0 0x7f58fc0a7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:20.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.783+0000 7f5901bc3700 1 -- 172.21.15.181:0/4278680473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58fc0b4e70 msgr2=0x7f58fc169b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:20.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.783+0000 7f5901bc3700 1 --2- 172.21.15.181:0/4278680473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58fc0b4e70 0x7f58fc169b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:20.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.783+0000 7f5901bc3700 1 -- 172.21.15.181:0/4278680473 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f8009420 con 0x7f58fc0b33b0 2023-10-09T06:30:20.786 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.783+0000 7f5901bc3700 1 --2- 172.21.15.181:0/4278680473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58fc0b33b0 0x7f58fc0a8490 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f58fc070dd0 tx=0x7f58f8016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:20.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.783+0000 7f58f27fc700 1 -- 172.21.15.181:0/4278680473 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f801cb80 con 0x7f58fc0b33b0 2023-10-09T06:30:20.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.783+0000 7f58f27fc700 1 -- 172.21.15.181:0/4278680473 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f801cce0 con 0x7f58fc0b33b0 2023-10-09T06:30:20.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.783+0000 7f59033c6700 1 -- 172.21.15.181:0/4278680473 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58fc0a9280 con 0x7f58fc0b33b0 2023-10-09T06:30:20.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.783+0000 7f58f27fc700 1 -- 172.21.15.181:0/4278680473 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f801a660 con 0x7f58fc0b33b0 2023-10-09T06:30:20.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.783+0000 7f59033c6700 1 -- 172.21.15.181:0/4278680473 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58fc0a89d0 con 0x7f58fc0b33b0 2023-10-09T06:30:20.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.783+0000 7f58f27fc700 1 -- 172.21.15.181:0/4278680473 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f58f801ad10 con 0x7f58fc0b33b0 2023-10-09T06:30:20.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.783+0000 7f58f27fc700 1 -- 172.21.15.181:0/4278680473 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7f58f802edb0 con 0x7f58fc0b33b0 2023-10-09T06:30:20.787 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.784+0000 7f59033c6700 1 -- 172.21.15.181:0/4278680473 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58fc088990 con 0x7f58fc0b33b0 2023-10-09T06:30:20.788 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.786+0000 7f58f27fc700 1 -- 172.21.15.181:0/4278680473 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f58f8033060 con 0x7f58fc0b33b0 2023-10-09T06:30:20.807 INFO:tasks.ceph.mgr.y.smithi163.stderr:2023-10-09T06:30:20.807+0000 7fc41d766180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-09T06:30:20.850 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:30:20.849+0000 7ff9dfa21180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-09T06:30:20.868 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.867+0000 7f59033c6700 1 -- 172.21.15.181:0/4278680473 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4023515b-10ff-428d-ab92-acad13005e9f", "id": 6} v 0) v1 -- 0x7f58fc0cb0e0 con 0x7f58fc0b33b0 2023-10-09T06:30:20.876 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.876+0000 7f58f27fc700 1 -- 172.21.15.181:0/4278680473 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4023515b-10ff-428d-ab92-acad13005e9f", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f58f8029070 con 0x7f58fc0b33b0 2023-10-09T06:30:20.877 INFO:teuthology.orchestra.run.smithi181.stdout:6 2023-10-09T06:30:20.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.876+0000 7f59033c6700 1 -- 172.21.15.181:0/4278680473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58fc0b33b0 msgr2=0x7f58fc0a8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:20.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.876+0000 7f59033c6700 1 --2- 172.21.15.181:0/4278680473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58fc0b33b0 0x7f58fc0a8490 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f58fc070dd0 tx=0x7f58f8016000 comp rx=0 tx=0).stop 2023-10-09T06:30:20.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.876+0000 7f59033c6700 1 -- 172.21.15.181:0/4278680473 shutdown_connections 2023-10-09T06:30:20.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.876+0000 7f59033c6700 1 --2- 172.21.15.181:0/4278680473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58fc0b4e70 0x7f58fc169b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:20.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.876+0000 7f59033c6700 1 --2- 172.21.15.181:0/4278680473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58fc0b16d0 0x7f58fc0a7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:20.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.876+0000 7f59033c6700 1 --2- 172.21.15.181:0/4278680473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58fc0b33b0 0x7f58fc0a8490 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:20.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.877+0000 7f59033c6700 1 -- 172.21.15.181:0/4278680473 >> 172.21.15.181:0/4278680473 conn(0x7f58fc0d10c0 msgr2=0x7f58fc069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:20.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.877+0000 7f59033c6700 1 -- 172.21.15.181:0/4278680473 shutdown_connections 2023-10-09T06:30:20.878 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:20.877+0000 7f59033c6700 1 -- 172.21.15.181:0/4278680473 wait complete. 2023-10-09T06:30:20.889 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 82d7b464-b236-4762-b546-511dfc0ac7ed 7 2023-10-09T06:30:21.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.003+0000 7f3851149700 1 Processor -- start 2023-10-09T06:30:21.004 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.003+0000 7f3851149700 1 -- start start 2023-10-09T06:30:21.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.004+0000 7f3851149700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f384c140350 0x7f384c069c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:21.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.004+0000 7f3851149700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f384c068aa0 0x7f384c0c8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:21.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.004+0000 7f3851149700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f384c0c7de0 con 0x7f384c06a1c0 2023-10-09T06:30:21.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.004+0000 7f3851149700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f384c140c10 con 0x7f384c140350 2023-10-09T06:30:21.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.004+0000 7f3851149700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f384c141490 con 0x7f384c068aa0 2023-10-09T06:30:21.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.004+0000 7f384b59e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f384c068aa0 0x7f384c0c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:21.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.004+0000 7f384a59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f384c06a1c0 0x7f384c068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:58380/0 (socket says 172.21.15.181:58380) 2023-10-09T06:30:21.007 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.004+0000 7f384a59c700 1 -- 172.21.15.181:0/2145129090 learned_addr learned my addr 172.21.15.181:0/2145129090 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-09T06:30:21.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.005+0000 7f384b59e700 1 -- 172.21.15.181:0/2145129090 >> v2:172.21.15.163:3300/0 conn(0x7f384c140350 msgr2=0x7f384c069c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:21.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.005+0000 7f384ad9d700 1 --2- 172.21.15.181:0/2145129090 >> v2:172.21.15.163:3300/0 conn(0x7f384c140350 0x7f384c069c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:21.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.005+0000 7f3849d9b700 1 -- 172.21.15.181:0/2145129090 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3228431595 0 0) 0x7f384c0c7de0 con 0x7f384c06a1c0 2023-10-09T06:30:21.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.005+0000 7f384b59e700 1 --2- 172.21.15.181:0/2145129090 >> v2:172.21.15.163:3300/0 conn(0x7f384c140350 0x7f384c069c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:21.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.005+0000 7f384b59e700 1 -- 172.21.15.181:0/2145129090 >> v1:172.21.15.181:6789/0 conn(0x7f384c06a1c0 legacy=0x7f384c068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:21.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.005+0000 7f384b59e700 1 -- 172.21.15.181:0/2145129090 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384c140ed0 con 0x7f384c068aa0 2023-10-09T06:30:21.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.005+0000 7f384b59e700 1 --2- 172.21.15.181:0/2145129090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f384c068aa0 0x7f384c0c8270 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f384c0c7c40 tx=0x7f3844009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf431f0b424cff44 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:21.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.005+0000 7f3849d9b700 1 -- 172.21.15.181:0/2145129090 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384400e070 con 0x7f384c068aa0 2023-10-09T06:30:21.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.005+0000 7f3849d9b700 1 -- 172.21.15.181:0/2145129090 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384400bc30 con 0x7f384c068aa0 2023-10-09T06:30:21.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.005+0000 7f3849d9b700 1 -- 172.21.15.181:0/2145129090 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3844012430 con 0x7f384c068aa0 2023-10-09T06:30:21.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.005+0000 7f3851149700 1 -- 172.21.15.181:0/2145129090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f384c068aa0 msgr2=0x7f384c0c8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:21.008 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.005+0000 7f3851149700 1 --2- 172.21.15.181:0/2145129090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f384c068aa0 0x7f384c0c8270 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f384c0c7c40 tx=0x7f3844009780 comp rx=0 tx=0).stop 2023-10-09T06:30:21.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.005+0000 7f3851149700 1 -- 172.21.15.181:0/2145129090 shutdown_connections 2023-10-09T06:30:21.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.005+0000 7f3851149700 1 --2- 172.21.15.181:0/2145129090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f384c068aa0 0x7f384c0c8270 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:21.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.005+0000 7f3851149700 1 --2- 172.21.15.181:0/2145129090 >> v2:172.21.15.163:3300/0 conn(0x7f384c140350 0x7f384c069c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:21.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.005+0000 7f3851149700 1 -- 172.21.15.181:0/2145129090 >> 172.21.15.181:0/2145129090 conn(0x7f384c0cd670 msgr2=0x7f384c0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:21.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.006+0000 7f3851149700 1 -- 172.21.15.181:0/2145129090 shutdown_connections 2023-10-09T06:30:21.009 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.006+0000 7f3851149700 1 -- 172.21.15.181:0/2145129090 wait complete. 2023-10-09T06:30:21.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.007+0000 7f3851149700 1 Processor -- start 2023-10-09T06:30:21.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.007+0000 7f3851149700 1 -- start start 2023-10-09T06:30:21.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.007+0000 7f3851149700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f384c068aa0 0x7f384c13c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:21.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.007+0000 7f3851149700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f384c06a1c0 0x7f384c158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:21.011 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.007+0000 7f3851149700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f384c140350 0x7f384c0c2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:21.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.007+0000 7f3851149700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384c0b1960 con 0x7f384c06a1c0 2023-10-09T06:30:21.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.007+0000 7f3851149700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384c0b1c40 con 0x7f384c140350 2023-10-09T06:30:21.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.007+0000 7f3851149700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384c0b1f20 con 0x7f384c068aa0 2023-10-09T06:30:21.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.008+0000 7f384b59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f384c140350 0x7f384c0c2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:21.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.008+0000 7f384b59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f384c140350 0x7f384c0c2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:59542/0 (socket says 172.21.15.181:59542) 2023-10-09T06:30:21.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.008+0000 7f384b59e700 1 -- 172.21.15.181:0/4005719711 learned_addr learned my addr 172.21.15.181:0/4005719711 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-09T06:30:21.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.008+0000 7f384a59c700 1 --2- 172.21.15.181:0/4005719711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f384c06a1c0 0x7f384c158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:21.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.008+0000 7f384ad9d700 1 --2- 172.21.15.181:0/4005719711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f384c068aa0 0x7f384c13c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:21.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.008+0000 7f384b59e700 1 -- 172.21.15.181:0/4005719711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f384c068aa0 msgr2=0x7f384c13c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:21.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.008+0000 7f384b59e700 1 --2- 172.21.15.181:0/4005719711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f384c068aa0 0x7f384c13c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:21.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.008+0000 7f384b59e700 1 -- 172.21.15.181:0/4005719711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f384c06a1c0 msgr2=0x7f384c158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:21.012 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.008+0000 7f384b59e700 1 --2- 172.21.15.181:0/4005719711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f384c06a1c0 0x7f384c158bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:21.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.008+0000 7f384b59e700 1 -- 172.21.15.181:0/4005719711 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3844009a20 con 0x7f384c140350 2023-10-09T06:30:21.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.008+0000 7f384b59e700 1 --2- 172.21.15.181:0/4005719711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f384c140350 0x7f384c0c2ea0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3844012580 tx=0x7f3844019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:21.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.008+0000 7f3832ffd700 1 -- 172.21.15.181:0/4005719711 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3844012e10 con 0x7f384c140350 2023-10-09T06:30:21.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.008+0000 7f3832ffd700 1 -- 172.21.15.181:0/4005719711 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384401e630 con 0x7f384c140350 2023-10-09T06:30:21.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.008+0000 7f3851149700 1 -- 172.21.15.181:0/4005719711 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f384c0b2200 con 0x7f384c140350 2023-10-09T06:30:21.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.008+0000 7f3832ffd700 1 -- 172.21.15.181:0/4005719711 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384401e9f0 con 0x7f384c140350 2023-10-09T06:30:21.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.008+0000 7f3851149700 1 -- 172.21.15.181:0/4005719711 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f384c13c5b0 con 0x7f384c140350 2023-10-09T06:30:21.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.008+0000 7f3832ffd700 1 -- 172.21.15.181:0/4005719711 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f3844024070 con 0x7f384c140350 2023-10-09T06:30:21.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.008+0000 7f3832ffd700 1 -- 172.21.15.181:0/4005719711 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7f3844027d10 con 0x7f384c140350 2023-10-09T06:30:21.013 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.009+0000 7f3851149700 1 -- 172.21.15.181:0/4005719711 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f384c04c3c0 con 0x7f384c140350 2023-10-09T06:30:21.014 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.011+0000 7f3832ffd700 1 -- 172.21.15.181:0/4005719711 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f384401ee20 con 0x7f384c140350 2023-10-09T06:30:21.096 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.095+0000 7f3851149700 1 -- 172.21.15.181:0/4005719711 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd new", "uuid": "82d7b464-b236-4762-b546-511dfc0ac7ed", "id": 7} v 0) v1 -- 0x7f384c0b1d80 con 0x7f384c140350 2023-10-09T06:30:21.101 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.100+0000 7f3832ffd700 1 -- 172.21.15.181:0/4005719711 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "82d7b464-b236-4762-b546-511dfc0ac7ed", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f384c0b1d80 con 0x7f384c140350 2023-10-09T06:30:21.101 INFO:teuthology.orchestra.run.smithi181.stdout:7 2023-10-09T06:30:21.102 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.101+0000 7f3851149700 1 -- 172.21.15.181:0/4005719711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f384c140350 msgr2=0x7f384c0c2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:21.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.101+0000 7f3851149700 1 --2- 172.21.15.181:0/4005719711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f384c140350 0x7f384c0c2ea0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3844012580 tx=0x7f3844019000 comp rx=0 tx=0).stop 2023-10-09T06:30:21.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.101+0000 7f3851149700 1 -- 172.21.15.181:0/4005719711 shutdown_connections 2023-10-09T06:30:21.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.101+0000 7f3851149700 1 --2- 172.21.15.181:0/4005719711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f384c140350 0x7f384c0c2ea0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:21.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.101+0000 7f3851149700 1 --2- 172.21.15.181:0/4005719711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f384c068aa0 0x7f384c13c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:21.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.101+0000 7f3851149700 1 --2- 172.21.15.181:0/4005719711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f384c06a1c0 0x7f384c158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:21.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.101+0000 7f3851149700 1 -- 172.21.15.181:0/4005719711 >> 172.21.15.181:0/4005719711 conn(0x7f384c0cd670 msgr2=0x7f384c156d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:21.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.101+0000 7f3851149700 1 -- 172.21.15.181:0/4005719711 shutdown_connections 2023-10-09T06:30:21.103 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-09T06:30:21.101+0000 7f3851149700 1 -- 172.21.15.181:0/4005719711 wait complete. 2023-10-09T06:30:21.114 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-09T06:30:21.114 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-09T06:30:21.116 INFO:tasks.ceph.osd.0:Started 2023-10-09T06:30:21.116 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-09T06:30:21.117 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-09T06:30:21.119 INFO:tasks.ceph.osd.1:Started 2023-10-09T06:30:21.119 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-09T06:30:21.119 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-09T06:30:21.120 INFO:tasks.ceph.osd.2:Started 2023-10-09T06:30:21.121 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-09T06:30:21.121 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-09T06:30:21.123 INFO:tasks.ceph.osd.3:Started 2023-10-09T06:30:21.123 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-09T06:30:21.123 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-09T06:30:21.125 INFO:tasks.ceph.osd.4:Started 2023-10-09T06:30:21.125 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-09T06:30:21.125 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-09T06:30:21.130 INFO:tasks.ceph.osd.5:Started 2023-10-09T06:30:21.130 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-09T06:30:21.130 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-09T06:30:21.132 INFO:tasks.ceph.osd.6:Started 2023-10-09T06:30:21.132 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-09T06:30:21.132 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-09T06:30:21.134 INFO:tasks.ceph.osd.7:Started 2023-10-09T06:30:21.134 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:30:21.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.276+0000 7f2d09f56700 1 Processor -- start 2023-10-09T06:30:21.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.276+0000 7f2d09f56700 1 -- start start 2023-10-09T06:30:21.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.276+0000 7f2d09f56700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2d040c91e0 0x7f2d040cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:21.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.276+0000 7f2d09f56700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d0406ae10 0x7f2d0406b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:21.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.276+0000 7f2d09f56700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d040cc210 con 0x7f2d0406aa40 2023-10-09T06:30:21.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.276+0000 7f2d09f56700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d040cc4f0 con 0x7f2d040c91e0 2023-10-09T06:30:21.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.277+0000 7f2d09f56700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d040cc7d0 con 0x7f2d0406ae10 2023-10-09T06:30:21.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.277+0000 7f2d02ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2d040c91e0 0x7f2d040cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:21.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.277+0000 7f2d02ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2d040c91e0 0x7f2d040cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:50666/0 (socket says 172.21.15.163:50666) 2023-10-09T06:30:21.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.277+0000 7f2d02ffd700 1 -- 172.21.15.163:0/3889715475 learned_addr learned my addr 172.21.15.163:0/3889715475 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:21.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.277+0000 7f2d037fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f2d0406aa40 0x7f2d0406bac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:37808/0 (socket says 172.21.15.163:37808) 2023-10-09T06:30:21.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.277+0000 7f2d03fff700 1 --2- 172.21.15.163:0/3889715475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d0406ae10 0x7f2d0406b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:21.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.277+0000 7f2d02ffd700 1 -- 172.21.15.163:0/3889715475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d0406ae10 msgr2=0x7f2d0406b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:21.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.277+0000 7f2d02ffd700 1 --2- 172.21.15.163:0/3889715475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d0406ae10 0x7f2d0406b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:21.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.277+0000 7f2d02ffd700 1 -- 172.21.15.163:0/3889715475 >> v1:172.21.15.163:6789/0 conn(0x7f2d0406aa40 legacy=0x7f2d0406bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:21.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.277+0000 7f2d02ffd700 1 -- 172.21.15.163:0/3889715475 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d040ccab0 con 0x7f2d040c91e0 2023-10-09T06:30:21.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.277+0000 7f2d027fc700 1 -- 172.21.15.163:0/3889715475 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 186570643 0 0) 0x7f2d040cc210 con 0x7f2d0406aa40 2023-10-09T06:30:21.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.277+0000 7f2d02ffd700 1 --2- 172.21.15.163:0/3889715475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d040c91e0 0x7f2d040cbc00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2cf80099e0 tx=0x7f2cf8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619cf9d88b3bae74 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:21.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.277+0000 7f2d027fc700 1 -- 172.21.15.163:0/3889715475 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf800b820 con 0x7f2d040c91e0 2023-10-09T06:30:21.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.277+0000 7f2d027fc700 1 -- 172.21.15.163:0/3889715475 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf800b980 con 0x7f2d040c91e0 2023-10-09T06:30:21.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.277+0000 7f2d027fc700 1 -- 172.21.15.163:0/3889715475 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf800bd20 con 0x7f2d040c91e0 2023-10-09T06:30:21.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.278+0000 7f2d09f56700 1 -- 172.21.15.163:0/3889715475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d040c91e0 msgr2=0x7f2d040cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:21.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.278+0000 7f2d09f56700 1 --2- 172.21.15.163:0/3889715475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d040c91e0 0x7f2d040cbc00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2cf80099e0 tx=0x7f2cf8009250 comp rx=0 tx=0).stop 2023-10-09T06:30:21.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.278+0000 7f2d09f56700 1 -- 172.21.15.163:0/3889715475 shutdown_connections 2023-10-09T06:30:21.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.278+0000 7f2d09f56700 1 --2- 172.21.15.163:0/3889715475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d0406ae10 0x7f2d0406b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:21.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.278+0000 7f2d09f56700 1 --2- 172.21.15.163:0/3889715475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d040c91e0 0x7f2d040cbc00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:21.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.278+0000 7f2d09f56700 1 -- 172.21.15.163:0/3889715475 >> 172.21.15.163:0/3889715475 conn(0x7f2d040cd670 msgr2=0x7f2d040cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:21.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.278+0000 7f2d09f56700 1 -- 172.21.15.163:0/3889715475 shutdown_connections 2023-10-09T06:30:21.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.278+0000 7f2d09f56700 1 -- 172.21.15.163:0/3889715475 wait complete. 2023-10-09T06:30:21.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.279+0000 7f2d09f56700 1 Processor -- start 2023-10-09T06:30:21.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.279+0000 7f2d09f56700 1 -- start start 2023-10-09T06:30:21.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.280+0000 7f2d09f56700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d0406aa40 0x7f2d040b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:21.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.280+0000 7f2d09f56700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d0406ae10 0x7f2d041462b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:21.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.280+0000 7f2d09f56700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d040c91e0 0x7f2d040b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:21.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.280+0000 7f2d09f56700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d040b1870 con 0x7f2d040c91e0 2023-10-09T06:30:21.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.280+0000 7f2d09f56700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d041467f0 con 0x7f2d0406aa40 2023-10-09T06:30:21.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.280+0000 7f2d09f56700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d040b0c60 con 0x7f2d0406ae10 2023-10-09T06:30:21.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.280+0000 7f2d03fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d040c91e0 0x7f2d040b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:21.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.280+0000 7f2d03fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d040c91e0 0x7f2d040b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:50674/0 (socket says 172.21.15.163:50674) 2023-10-09T06:30:21.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.280+0000 7f2d03fff700 1 -- 172.21.15.163:0/3628257670 learned_addr learned my addr 172.21.15.163:0/3628257670 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:21.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.280+0000 7f2d02ffd700 1 --2- 172.21.15.163:0/3628257670 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d0406ae10 0x7f2d041462b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:21.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.280+0000 7f2d037fe700 1 --2- 172.21.15.163:0/3628257670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d0406aa40 0x7f2d040b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:21.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.280+0000 7f2d03fff700 1 -- 172.21.15.163:0/3628257670 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d0406ae10 msgr2=0x7f2d041462b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:21.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.280+0000 7f2d03fff700 1 --2- 172.21.15.163:0/3628257670 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d0406ae10 0x7f2d041462b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:21.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.280+0000 7f2d03fff700 1 -- 172.21.15.163:0/3628257670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d0406aa40 msgr2=0x7f2d040b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:21.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.280+0000 7f2d03fff700 1 --2- 172.21.15.163:0/3628257670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d0406aa40 0x7f2d040b2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:21.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.280+0000 7f2d03fff700 1 -- 172.21.15.163:0/3628257670 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf80094f0 con 0x7f2d040c91e0 2023-10-09T06:30:21.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.280+0000 7f2d03fff700 1 --2- 172.21.15.163:0/3628257670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d040c91e0 0x7f2d040b0410 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009fa0 tx=0x7f2cfc009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:21.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.281+0000 7f2ceb7fe700 1 -- 172.21.15.163:0/3628257670 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc00ebc0 con 0x7f2d040c91e0 2023-10-09T06:30:21.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.281+0000 7f2ceb7fe700 1 -- 172.21.15.163:0/3628257670 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cfc00ed20 con 0x7f2d040c91e0 2023-10-09T06:30:21.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.281+0000 7f2d09f56700 1 -- 172.21.15.163:0/3628257670 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d040b0f40 con 0x7f2d040c91e0 2023-10-09T06:30:21.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.281+0000 7f2ceb7fe700 1 -- 172.21.15.163:0/3628257670 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc014670 con 0x7f2d040c91e0 2023-10-09T06:30:21.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.281+0000 7f2d09f56700 1 -- 172.21.15.163:0/3628257670 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d040b1500 con 0x7f2d040c91e0 2023-10-09T06:30:21.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.281+0000 7f2ceb7fe700 1 -- 172.21.15.163:0/3628257670 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f2cfc0149e0 con 0x7f2d040c91e0 2023-10-09T06:30:21.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.281+0000 7f2ceb7fe700 1 -- 172.21.15.163:0/3628257670 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f2cfc01ed70 con 0x7f2d040c91e0 2023-10-09T06:30:21.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.282+0000 7f2d09f56700 1 -- 172.21.15.163:0/3628257670 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d040b19e0 con 0x7f2d040c91e0 2023-10-09T06:30:21.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.284+0000 7f2ceb7fe700 1 -- 172.21.15.163:0/3628257670 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f2cfc01e690 con 0x7f2d040c91e0 2023-10-09T06:30:21.354 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.353+0000 7f2d09f56700 1 -- 172.21.15.163:0/3628257670 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d0404c3c0 con 0x7f2d040c91e0 2023-10-09T06:30:21.354 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.354+0000 7f2ceb7fe700 1 -- 172.21.15.163:0/3628257670 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f2cfc02d090 con 0x7f2d040c91e0 2023-10-09T06:30:21.355 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:30:21.355 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":10,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:30:21.097413+0000","last_up_change":"0.000000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:30:21.356 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.355+0000 7f2d09f56700 1 -- 172.21.15.163:0/3628257670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d040c91e0 msgr2=0x7f2d040b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:21.356 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.355+0000 7f2d09f56700 1 --2- 172.21.15.163:0/3628257670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d040c91e0 0x7f2d040b0410 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009fa0 tx=0x7f2cfc009900 comp rx=0 tx=0).stop 2023-10-09T06:30:21.356 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.355+0000 7f2d09f56700 1 -- 172.21.15.163:0/3628257670 shutdown_connections 2023-10-09T06:30:21.356 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.355+0000 7f2d09f56700 1 --2- 172.21.15.163:0/3628257670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d040c91e0 0x7f2d040b0410 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:21.356 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.355+0000 7f2d09f56700 1 --2- 172.21.15.163:0/3628257670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d0406aa40 0x7f2d040b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:21.356 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.355+0000 7f2d09f56700 1 --2- 172.21.15.163:0/3628257670 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d0406ae10 0x7f2d041462b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:21.356 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.355+0000 7f2d09f56700 1 -- 172.21.15.163:0/3628257670 >> 172.21.15.163:0/3628257670 conn(0x7f2d040cd670 msgr2=0x7f2d0415c950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:21.356 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.355+0000 7f2d09f56700 1 -- 172.21.15.163:0/3628257670 shutdown_connections 2023-10-09T06:30:21.357 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:21.355+0000 7f2d09f56700 1 -- 172.21.15.163:0/3628257670 wait complete. 2023-10-09T06:30:21.358 INFO:tasks.ceph.mgr.y.smithi163.stderr:2023-10-09T06:30:21.358+0000 7fc41d766180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-09T06:30:21.367 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-09T06:30:21.368 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-09T06:30:21.395 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:30:21.394+0000 7ff9dfa21180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-09T06:30:21.440 INFO:tasks.ceph.mgr.y.smithi163.stderr:2023-10-09T06:30:21.439+0000 7fc41d766180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-09T06:30:21.472 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:30:21.471+0000 7ff9dfa21180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-09T06:30:21.520 INFO:tasks.ceph.mgr.y.smithi163.stderr:2023-10-09T06:30:21.519+0000 7fc41d766180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-09T06:30:21.551 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:30:21.551+0000 7ff9dfa21180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-09T06:30:21.720 INFO:tasks.ceph.osd.2.smithi163.stderr:2023-10-09T06:30:21.719+0000 7fdcaf00f540 -1 Falling back to public interface 2023-10-09T06:30:21.723 INFO:tasks.ceph.osd.5.smithi181.stderr:2023-10-09T06:30:21.722+0000 7fd931337540 -1 Falling back to public interface 2023-10-09T06:30:21.731 INFO:tasks.ceph.osd.6.smithi181.stderr:2023-10-09T06:30:21.730+0000 7f8a626f0540 -1 Falling back to public interface 2023-10-09T06:30:21.735 INFO:tasks.ceph.osd.1.smithi163.stderr:2023-10-09T06:30:21.734+0000 7f1a8e5fb540 -1 Falling back to public interface 2023-10-09T06:30:21.735 INFO:tasks.ceph.osd.7.smithi181.stderr:2023-10-09T06:30:21.735+0000 7f2d26d6d540 -1 Falling back to public interface 2023-10-09T06:30:21.736 INFO:tasks.ceph.osd.3.smithi163.stderr:2023-10-09T06:30:21.735+0000 7f939d40f540 -1 Falling back to public interface 2023-10-09T06:30:21.740 INFO:tasks.ceph.osd.0.smithi163.stderr:2023-10-09T06:30:21.739+0000 7f0112ab3540 -1 Falling back to public interface 2023-10-09T06:30:21.746 INFO:tasks.ceph.osd.4.smithi181.stderr:2023-10-09T06:30:21.743+0000 7f6d47d66540 -1 Falling back to public interface 2023-10-09T06:30:22.105 INFO:tasks.ceph.mgr.y.smithi163.stderr:2023-10-09T06:30:22.104+0000 7fc41d766180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-09T06:30:22.123 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:30:22.123+0000 7ff9dfa21180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-09T06:30:22.173 DEBUG:teuthology.orchestra.run.smithi163:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-09T06:30:22.247 INFO:tasks.ceph.mgr.y.smithi163.stderr:2023-10-09T06:30:22.247+0000 7fc41d766180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-09T06:30:22.260 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:30:22.260+0000 7ff9dfa21180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-09T06:30:22.282 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.281+0000 7f2a676e9700 1 Processor -- start 2023-10-09T06:30:22.282 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.281+0000 7f2a676e9700 1 -- start start 2023-10-09T06:30:22.284 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.282+0000 7f2a676e9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2a6015d1d0 0x7f2a600cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:22.284 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.282+0000 7f2a676e9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a600cbfa0 0x7f2a600ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:22.284 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.282+0000 7f2a676e9700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a600cf230 con 0x7f2a6015ce00 2023-10-09T06:30:22.284 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.282+0000 7f2a676e9700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a600cc3c0 con 0x7f2a6015d1d0 2023-10-09T06:30:22.284 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.282+0000 7f2a676e9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a600adfe0 con 0x7f2a600cbfa0 2023-10-09T06:30:22.284 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.282+0000 7f2a64c84700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2a6015d1d0 0x7f2a600cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:22.285 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.282+0000 7f2a65485700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f2a6015ce00 0x7f2a600ad8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:37810/0 (socket says 172.21.15.163:37810) 2023-10-09T06:30:22.285 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.282+0000 7f2a64c84700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2a6015d1d0 0x7f2a600cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:50690/0 (socket says 172.21.15.163:50690) 2023-10-09T06:30:22.285 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.282+0000 7f2a65485700 1 -- 172.21.15.163:0/222895331 learned_addr learned my addr 172.21.15.163:0/222895331 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:22.285 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.282+0000 7f2a65c86700 1 --2- 172.21.15.163:0/222895331 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a600cbfa0 0x7f2a600ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:22.285 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.282+0000 7f2a64c84700 1 -- 172.21.15.163:0/222895331 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a600cbfa0 msgr2=0x7f2a600ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:22.285 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.282+0000 7f2a64c84700 1 --2- 172.21.15.163:0/222895331 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a600cbfa0 0x7f2a600ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:22.285 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.282+0000 7f2a64c84700 1 -- 172.21.15.163:0/222895331 >> v1:172.21.15.163:6789/0 conn(0x7f2a6015ce00 legacy=0x7f2a600ad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:22.285 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.282+0000 7f2a57fff700 1 -- 172.21.15.163:0/222895331 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 393851800 0 0) 0x7f2a600cf230 con 0x7f2a6015ce00 2023-10-09T06:30:22.285 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.282+0000 7f2a64c84700 1 -- 172.21.15.163:0/222895331 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a600ce380 con 0x7f2a6015d1d0 2023-10-09T06:30:22.285 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.283+0000 7f2a64c84700 1 --2- 172.21.15.163:0/222895331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a6015d1d0 0x7f2a600cba60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0098e0 tx=0x7f2a5c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52253e894dc5cb1b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:22.285 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.283+0000 7f2a57fff700 1 -- 172.21.15.163:0/222895331 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c00b640 con 0x7f2a6015d1d0 2023-10-09T06:30:22.286 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.283+0000 7f2a57fff700 1 -- 172.21.15.163:0/222895331 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5c00b7a0 con 0x7f2a6015d1d0 2023-10-09T06:30:22.286 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.283+0000 7f2a57fff700 1 -- 172.21.15.163:0/222895331 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c00bb40 con 0x7f2a6015d1d0 2023-10-09T06:30:22.286 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.283+0000 7f2a676e9700 1 -- 172.21.15.163:0/222895331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a6015d1d0 msgr2=0x7f2a600cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:22.286 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.283+0000 7f2a676e9700 1 --2- 172.21.15.163:0/222895331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a6015d1d0 0x7f2a600cba60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0098e0 tx=0x7f2a5c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:22.286 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.283+0000 7f2a676e9700 1 -- 172.21.15.163:0/222895331 shutdown_connections 2023-10-09T06:30:22.286 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.283+0000 7f2a676e9700 1 --2- 172.21.15.163:0/222895331 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a600cbfa0 0x7f2a600ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:22.286 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.283+0000 7f2a676e9700 1 --2- 172.21.15.163:0/222895331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a6015d1d0 0x7f2a600cba60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:22.286 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.283+0000 7f2a676e9700 1 -- 172.21.15.163:0/222895331 >> 172.21.15.163:0/222895331 conn(0x7f2a60099b10 msgr2=0x7f2a60087ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:22.286 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.283+0000 7f2a676e9700 1 -- 172.21.15.163:0/222895331 shutdown_connections 2023-10-09T06:30:22.286 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.283+0000 7f2a676e9700 1 -- 172.21.15.163:0/222895331 wait complete. 2023-10-09T06:30:22.289 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.284+0000 7f2a676e9700 1 Processor -- start 2023-10-09T06:30:22.289 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.285+0000 7f2a676e9700 1 -- start start 2023-10-09T06:30:22.289 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.285+0000 7f2a676e9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a600cbfa0 0x7f2a600c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:22.289 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.285+0000 7f2a676e9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a6015ce00 0x7f2a6015b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:22.289 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.285+0000 7f2a676e9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a6015d1d0 0x7f2a6015bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:22.289 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.285+0000 7f2a676e9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a600c9ed0 con 0x7f2a6015ce00 2023-10-09T06:30:22.289 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.285+0000 7f2a676e9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a600c8220 con 0x7f2a600cbfa0 2023-10-09T06:30:22.289 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.285+0000 7f2a676e9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a600c8d40 con 0x7f2a6015d1d0 2023-10-09T06:30:22.289 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.286+0000 7f2a64c84700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a6015ce00 0x7f2a6015b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:22.290 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.286+0000 7f2a64c84700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a6015ce00 0x7f2a6015b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:50700/0 (socket says 172.21.15.163:50700) 2023-10-09T06:30:22.290 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.286+0000 7f2a64c84700 1 -- 172.21.15.163:0/2191782283 learned_addr learned my addr 172.21.15.163:0/2191782283 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:22.290 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.286+0000 7f2a65c86700 1 --2- 172.21.15.163:0/2191782283 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a6015d1d0 0x7f2a6015bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:22.290 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.286+0000 7f2a65485700 1 --2- 172.21.15.163:0/2191782283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a600cbfa0 0x7f2a600c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:22.290 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.286+0000 7f2a64c84700 1 -- 172.21.15.163:0/2191782283 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a6015d1d0 msgr2=0x7f2a6015bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:22.290 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.286+0000 7f2a64c84700 1 --2- 172.21.15.163:0/2191782283 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a6015d1d0 0x7f2a6015bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:22.290 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.286+0000 7f2a64c84700 1 -- 172.21.15.163:0/2191782283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a600cbfa0 msgr2=0x7f2a600c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:22.290 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.286+0000 7f2a64c84700 1 --2- 172.21.15.163:0/2191782283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a600cbfa0 0x7f2a600c9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:22.290 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.286+0000 7f2a64c84700 1 -- 172.21.15.163:0/2191782283 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5c009420 con 0x7f2a6015ce00 2023-10-09T06:30:22.290 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.286+0000 7f2a64c84700 1 --2- 172.21.15.163:0/2191782283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a6015ce00 0x7f2a6015b570 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2a60089270 tx=0x7f2a5c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:22.291 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.286+0000 7f2a557fa700 1 -- 172.21.15.163:0/2191782283 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c00bb20 con 0x7f2a6015ce00 2023-10-09T06:30:22.291 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.286+0000 7f2a557fa700 1 -- 172.21.15.163:0/2191782283 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5c000c90 con 0x7f2a6015ce00 2023-10-09T06:30:22.291 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.286+0000 7f2a676e9700 1 -- 172.21.15.163:0/2191782283 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a600c8410 con 0x7f2a6015ce00 2023-10-09T06:30:22.291 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.286+0000 7f2a557fa700 1 -- 172.21.15.163:0/2191782283 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c011630 con 0x7f2a6015ce00 2023-10-09T06:30:22.291 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.286+0000 7f2a676e9700 1 -- 172.21.15.163:0/2191782283 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a600c89d0 con 0x7f2a6015ce00 2023-10-09T06:30:22.291 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.286+0000 7f2a557fa700 1 -- 172.21.15.163:0/2191782283 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f2a5c0119a0 con 0x7f2a6015ce00 2023-10-09T06:30:22.291 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.286+0000 7f2a557fa700 1 -- 172.21.15.163:0/2191782283 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f2a5c02cc60 con 0x7f2a6015ce00 2023-10-09T06:30:22.291 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.287+0000 7f2a676e9700 1 -- 172.21.15.163:0/2191782283 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a600ad4b0 con 0x7f2a6015ce00 2023-10-09T06:30:22.292 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.289+0000 7f2a557fa700 1 -- 172.21.15.163:0/2191782283 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f2a5c031060 con 0x7f2a6015ce00 2023-10-09T06:30:22.336 INFO:tasks.ceph.mgr.y.smithi163.stderr:2023-10-09T06:30:22.335+0000 7fc41d766180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-09T06:30:22.356 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:30:22.354+0000 7ff9dfa21180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-09T06:30:22.359 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.358+0000 7f2a676e9700 1 -- 172.21.15.163:0/2191782283 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a600caa10 con 0x7f2a6015ce00 2023-10-09T06:30:22.360 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.359+0000 7f2a557fa700 1 -- 172.21.15.163:0/2191782283 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f2a5c015020 con 0x7f2a6015ce00 2023-10-09T06:30:22.360 INFO:teuthology.misc.health.smithi163.stdout: 2023-10-09T06:30:22.360 INFO:teuthology.misc.health.smithi163.stdout:{"epoch":10,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:30:21.097413+0000","last_up_change":"0.000000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:30:22.362 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.360+0000 7f2a676e9700 1 -- 172.21.15.163:0/2191782283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a6015ce00 msgr2=0x7f2a6015b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:22.362 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.360+0000 7f2a676e9700 1 --2- 172.21.15.163:0/2191782283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a6015ce00 0x7f2a6015b570 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2a60089270 tx=0x7f2a5c017000 comp rx=0 tx=0).stop 2023-10-09T06:30:22.362 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.360+0000 7f2a676e9700 1 -- 172.21.15.163:0/2191782283 shutdown_connections 2023-10-09T06:30:22.362 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.360+0000 7f2a676e9700 1 --2- 172.21.15.163:0/2191782283 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a6015d1d0 0x7f2a6015bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:22.362 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.360+0000 7f2a676e9700 1 --2- 172.21.15.163:0/2191782283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a600cbfa0 0x7f2a600c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:22.362 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.360+0000 7f2a676e9700 1 --2- 172.21.15.163:0/2191782283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a6015ce00 0x7f2a6015b570 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:22.362 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.360+0000 7f2a676e9700 1 -- 172.21.15.163:0/2191782283 >> 172.21.15.163:0/2191782283 conn(0x7f2a60099b10 msgr2=0x7f2a600d05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:22.362 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.360+0000 7f2a676e9700 1 -- 172.21.15.163:0/2191782283 shutdown_connections 2023-10-09T06:30:22.363 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:22.360+0000 7f2a676e9700 1 -- 172.21.15.163:0/2191782283 wait complete. 2023-10-09T06:30:22.372 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-09T06:30:22.591 INFO:tasks.ceph.mgr.y.smithi163.stderr:2023-10-09T06:30:22.591+0000 7fc41d766180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-09T06:30:22.608 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:30:22.604+0000 7ff9dfa21180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-09T06:30:22.668 INFO:tasks.ceph.mgr.y.smithi163.stderr:2023-10-09T06:30:22.667+0000 7fc41d766180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-09T06:30:22.682 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:30:22.680+0000 7ff9dfa21180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-09T06:30:23.504 INFO:tasks.ceph.mgr.y.smithi163.stderr:2023-10-09T06:30:23.503+0000 7fc41d766180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-09T06:30:23.504 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:30:23.503+0000 7ff9dfa21180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-09T06:30:23.578 INFO:tasks.ceph.mgr.y.smithi163.stderr:2023-10-09T06:30:23.577+0000 7fc41d766180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-09T06:30:23.578 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:30:23.577+0000 7ff9dfa21180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-09T06:30:23.654 INFO:tasks.ceph.mgr.y.smithi163.stderr:2023-10-09T06:30:23.652+0000 7fc41d766180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-09T06:30:23.654 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:30:23.652+0000 7ff9dfa21180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-09T06:30:23.786 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:30:23.785+0000 7ff9dfa21180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-09T06:30:23.788 INFO:tasks.ceph.mgr.y.smithi163.stderr:2023-10-09T06:30:23.787+0000 7fc41d766180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-09T06:30:23.872 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:30:23.870+0000 7ff9dfa21180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-09T06:30:23.873 INFO:tasks.ceph.mgr.y.smithi163.stderr:2023-10-09T06:30:23.872+0000 7fc41d766180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-09T06:30:24.025 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:30:24.023+0000 7ff9dfa21180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-09T06:30:24.032 INFO:tasks.ceph.mgr.y.smithi163.stderr:2023-10-09T06:30:24.031+0000 7fc41d766180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-09T06:30:24.100 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:30:24.099+0000 7ff9dfa21180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-09T06:30:24.108 INFO:tasks.ceph.mgr.y.smithi163.stderr:2023-10-09T06:30:24.107+0000 7fc41d766180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-09T06:30:24.184 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:30:24.176+0000 7ff9dfa21180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-09T06:30:24.193 INFO:tasks.ceph.mgr.y.smithi163.stderr:2023-10-09T06:30:24.189+0000 7fc41d766180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-09T06:30:24.269 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:30:24.268+0000 7ff9dfa21180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-09T06:30:24.271 INFO:tasks.ceph.mgr.y.smithi163.stderr:2023-10-09T06:30:24.270+0000 7fc41d766180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-09T06:30:24.970 INFO:tasks.ceph.osd.5.smithi181.stderr:2023-10-09T06:30:24.969+0000 7fd931337540 -1 osd.5 0 log_to_monitors true 2023-10-09T06:30:25.209 INFO:tasks.ceph.osd.6.smithi181.stderr:2023-10-09T06:30:25.209+0000 7f8a626f0540 -1 osd.6 0 log_to_monitors true 2023-10-09T06:30:25.224 INFO:tasks.ceph.osd.3.smithi163.stderr:2023-10-09T06:30:25.224+0000 7f939d40f540 -1 osd.3 0 log_to_monitors true 2023-10-09T06:30:25.225 INFO:tasks.ceph.osd.7.smithi181.stderr:2023-10-09T06:30:25.224+0000 7f2d26d6d540 -1 osd.7 0 log_to_monitors true 2023-10-09T06:30:25.249 INFO:tasks.ceph.osd.4.smithi181.stderr:2023-10-09T06:30:25.248+0000 7f6d47d66540 -1 osd.4 0 log_to_monitors true 2023-10-09T06:30:25.252 INFO:tasks.ceph.osd.2.smithi163.stderr:2023-10-09T06:30:25.252+0000 7fdcaf00f540 -1 osd.2 0 log_to_monitors true 2023-10-09T06:30:25.295 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:30:25.294+0000 7ff998e22700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-09T06:30:25.295 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:30:25.295+0000 7ff998e22700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-09T06:30:25.305 INFO:tasks.ceph.osd.0.smithi163.stderr:2023-10-09T06:30:25.305+0000 7f0112ab3540 -1 osd.0 0 log_to_monitors true 2023-10-09T06:30:25.316 INFO:tasks.ceph.osd.1.smithi163.stderr:2023-10-09T06:30:25.317+0000 7f1a8e5fb540 -1 osd.1 0 log_to_monitors true 2023-10-09T06:30:28.505 INFO:tasks.ceph.osd.3.smithi163.stderr:2023-10-09T06:30:28.505+0000 7f9394031700 -1 osd.3 0 waiting for initial osdmap 2023-10-09T06:30:28.510 INFO:tasks.ceph.osd.3.smithi163.stderr:2023-10-09T06:30:28.511+0000 7f938ce28700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-09T06:30:28.511 INFO:tasks.ceph.osd.0.smithi163.stderr:2023-10-09T06:30:28.512+0000 7f01096d5700 -1 osd.0 0 waiting for initial osdmap 2023-10-09T06:30:28.518 INFO:tasks.ceph.osd.0.smithi163.stderr:2023-10-09T06:30:28.519+0000 7f01024cc700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-09T06:30:28.527 INFO:tasks.ceph.osd.1.smithi163.stderr:2023-10-09T06:30:28.527+0000 7f1a8521d700 -1 osd.1 0 waiting for initial osdmap 2023-10-09T06:30:28.528 INFO:tasks.ceph.osd.2.smithi163.stderr:2023-10-09T06:30:28.528+0000 7fdca7434700 -1 osd.2 0 waiting for initial osdmap 2023-10-09T06:30:28.530 INFO:tasks.ceph.osd.7.smithi181.stderr:2023-10-09T06:30:28.530+0000 7f2d1d98f700 -1 osd.7 0 waiting for initial osdmap 2023-10-09T06:30:28.535 INFO:tasks.ceph.osd.2.smithi163.stderr:2023-10-09T06:30:28.535+0000 7fdc9ea28700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-09T06:30:28.537 INFO:tasks.ceph.osd.1.smithi163.stderr:2023-10-09T06:30:28.538+0000 7f1a7e014700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-09T06:30:28.538 INFO:tasks.ceph.osd.7.smithi181.stderr:2023-10-09T06:30:28.537+0000 7f2d16786700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-09T06:30:28.544 INFO:tasks.ceph.osd.5.smithi181.stderr:2023-10-09T06:30:28.543+0000 7fd927f59700 -1 osd.5 0 waiting for initial osdmap 2023-10-09T06:30:28.549 INFO:tasks.ceph.osd.5.smithi181.stderr:2023-10-09T06:30:28.547+0000 7fd920d50700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-09T06:30:28.551 INFO:tasks.ceph.osd.6.smithi181.stderr:2023-10-09T06:30:28.550+0000 7f8a59312700 -1 osd.6 0 waiting for initial osdmap 2023-10-09T06:30:28.555 INFO:tasks.ceph.osd.6.smithi181.stderr:2023-10-09T06:30:28.555+0000 7f8a52109700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-09T06:30:28.560 INFO:tasks.ceph.osd.4.smithi181.stderr:2023-10-09T06:30:28.560+0000 7f6d3e988700 -1 osd.4 0 waiting for initial osdmap 2023-10-09T06:30:28.564 INFO:tasks.ceph.osd.4.smithi181.stderr:2023-10-09T06:30:28.563+0000 7f6d3777f700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-09T06:30:29.178 DEBUG:teuthology.orchestra.run.smithi163:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-09T06:30:29.286 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.286+0000 7f3d1b354700 1 Processor -- start 2023-10-09T06:30:29.286 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.286+0000 7f3d1b354700 1 -- start start 2023-10-09T06:30:29.295 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.287+0000 7f3d1b354700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d1415ce50 0x7f3d1415d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:29.295 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.287+0000 7f3d1b354700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3d140b4730 0x7f3d140b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:29.295 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.287+0000 7f3d1b354700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d14159ef0 con 0x7f3d1415daf0 2023-10-09T06:30:29.296 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.287+0000 7f3d1b354700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d140b6090 con 0x7f3d140b4730 2023-10-09T06:30:29.296 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.287+0000 7f3d1b354700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1415df40 con 0x7f3d1415ce50 2023-10-09T06:30:29.296 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.287+0000 7f3d1ab53700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3d140b4730 0x7f3d140b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:29.296 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.287+0000 7f3d1ab53700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3d140b4730 0x7f3d140b4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47584/0 (socket says 172.21.15.163:47584) 2023-10-09T06:30:29.296 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.287+0000 7f3d1ab53700 1 -- 172.21.15.163:0/3047786887 learned_addr learned my addr 172.21.15.163:0/3047786887 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:29.296 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.287+0000 7f3d1a352700 1 --2- 172.21.15.163:0/3047786887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d1415ce50 0x7f3d1415d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:29.297 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.287+0000 7f3d1ab53700 1 -- 172.21.15.163:0/3047786887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d1415ce50 msgr2=0x7f3d1415d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:29.297 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.287+0000 7f3d1ab53700 1 --2- 172.21.15.163:0/3047786887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d1415ce50 0x7f3d1415d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:29.297 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.287+0000 7f3d1ab53700 1 -- 172.21.15.163:0/3047786887 >> v1:172.21.15.181:6789/0 conn(0x7f3d1415daf0 legacy=0x7f3d1415a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:29.297 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.288+0000 7f3d1ab53700 1 -- 172.21.15.163:0/3047786887 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d140b5480 con 0x7f3d140b4730 2023-10-09T06:30:29.297 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.288+0000 7f3d1ab53700 1 --2- 172.21.15.163:0/3047786887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d140b4730 0x7f3d140b4b90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c009f20 tx=0x7f3d0c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9a891d22d721a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:29.297 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.288+0000 7f3d19350700 1 -- 172.21.15.163:0/3047786887 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c00e070 con 0x7f3d140b4730 2023-10-09T06:30:29.298 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.288+0000 7f3d19350700 1 -- 172.21.15.163:0/3047786887 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0c00bb10 con 0x7f3d140b4730 2023-10-09T06:30:29.298 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.288+0000 7f3d19350700 1 -- 172.21.15.163:0/3047786887 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c0092e0 con 0x7f3d140b4730 2023-10-09T06:30:29.298 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.288+0000 7f3d1b354700 1 -- 172.21.15.163:0/3047786887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d140b4730 msgr2=0x7f3d140b4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:29.298 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.288+0000 7f3d1b354700 1 --2- 172.21.15.163:0/3047786887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d140b4730 0x7f3d140b4b90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c009f20 tx=0x7f3d0c009840 comp rx=0 tx=0).stop 2023-10-09T06:30:29.298 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.288+0000 7f3d1b354700 1 -- 172.21.15.163:0/3047786887 shutdown_connections 2023-10-09T06:30:29.298 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.288+0000 7f3d1b354700 1 --2- 172.21.15.163:0/3047786887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d140b4730 0x7f3d140b4b90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:29.299 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.288+0000 7f3d1b354700 1 --2- 172.21.15.163:0/3047786887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d1415ce50 0x7f3d1415d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:29.299 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.288+0000 7f3d1b354700 1 -- 172.21.15.163:0/3047786887 >> 172.21.15.163:0/3047786887 conn(0x7f3d14087ff0 msgr2=0x7f3d1406d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:29.299 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.288+0000 7f3d1b354700 1 -- 172.21.15.163:0/3047786887 shutdown_connections 2023-10-09T06:30:29.299 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.288+0000 7f3d1b354700 1 -- 172.21.15.163:0/3047786887 wait complete. 2023-10-09T06:30:29.299 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.290+0000 7f3d1b354700 1 Processor -- start 2023-10-09T06:30:29.300 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.290+0000 7f3d1b354700 1 -- start start 2023-10-09T06:30:29.300 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.290+0000 7f3d1b354700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d140b4730 0x7f3d1409a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:29.300 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.290+0000 7f3d1b354700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d1415ce50 0x7f3d140cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:29.300 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.290+0000 7f3d1b354700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d1415daf0 0x7f3d140cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:29.300 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.290+0000 7f3d1b354700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d140cce90 con 0x7f3d1415daf0 2023-10-09T06:30:29.300 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.290+0000 7f3d1b354700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d140cc130 con 0x7f3d1415ce50 2023-10-09T06:30:29.301 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.290+0000 7f3d1b354700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1416afb0 con 0x7f3d140b4730 2023-10-09T06:30:29.301 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.290+0000 7f3d1ab53700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d1415daf0 0x7f3d140cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:29.301 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.290+0000 7f3d1ab53700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d1415daf0 0x7f3d140cebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47594/0 (socket says 172.21.15.163:47594) 2023-10-09T06:30:29.301 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.290+0000 7f3d1ab53700 1 -- 172.21.15.163:0/598406003 learned_addr learned my addr 172.21.15.163:0/598406003 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:29.301 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.290+0000 7f3d19b51700 1 --2- 172.21.15.163:0/598406003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d1415ce50 0x7f3d140cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:29.301 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.291+0000 7f3d1a352700 1 --2- 172.21.15.163:0/598406003 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d140b4730 0x7f3d1409a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:29.302 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.291+0000 7f3d1ab53700 1 -- 172.21.15.163:0/598406003 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d140b4730 msgr2=0x7f3d1409a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:29.302 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.291+0000 7f3d1ab53700 1 --2- 172.21.15.163:0/598406003 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d140b4730 0x7f3d1409a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:29.302 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.291+0000 7f3d1ab53700 1 -- 172.21.15.163:0/598406003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d1415ce50 msgr2=0x7f3d140cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:29.302 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.291+0000 7f3d1ab53700 1 --2- 172.21.15.163:0/598406003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d1415ce50 0x7f3d140cbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:29.302 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.291+0000 7f3d1ab53700 1 -- 172.21.15.163:0/598406003 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d0c009ae0 con 0x7f3d1415daf0 2023-10-09T06:30:29.303 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.291+0000 7f3d1ab53700 1 --2- 172.21.15.163:0/598406003 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d1415daf0 0x7f3d140cebe0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c009430 tx=0x7f3d0c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:29.303 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.291+0000 7f3d0a7fc700 1 -- 172.21.15.163:0/598406003 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c0032f0 con 0x7f3d1415daf0 2023-10-09T06:30:29.303 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.291+0000 7f3d0a7fc700 1 -- 172.21.15.163:0/598406003 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0c008e70 con 0x7f3d1415daf0 2023-10-09T06:30:29.303 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.291+0000 7f3d1b354700 1 -- 172.21.15.163:0/598406003 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d1416acd0 con 0x7f3d1415daf0 2023-10-09T06:30:29.303 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.291+0000 7f3d0a7fc700 1 -- 172.21.15.163:0/598406003 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c026670 con 0x7f3d1415daf0 2023-10-09T06:30:29.304 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.291+0000 7f3d1b354700 1 -- 172.21.15.163:0/598406003 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d140ccb20 con 0x7f3d1415daf0 2023-10-09T06:30:29.304 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.292+0000 7f3d1b354700 1 -- 172.21.15.163:0/598406003 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d14064b50 con 0x7f3d1415daf0 2023-10-09T06:30:29.304 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.292+0000 7f3d0a7fc700 1 -- 172.21.15.163:0/598406003 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f3d0c003040 con 0x7f3d1415daf0 2023-10-09T06:30:29.304 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.293+0000 7f3d0a7fc700 1 --2- 172.21.15.163:0/598406003 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3cf8066ea0 0x7f3cf8069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:29.304 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.293+0000 7f3d0a7fc700 1 -- 172.21.15.163:0/598406003 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2227+0+0 (secure 0 0 0) 0x7f3d0c085700 con 0x7f3d1415daf0 2023-10-09T06:30:29.305 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.293+0000 7f3d1a352700 1 --2- 172.21.15.163:0/598406003 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3cf8066ea0 0x7f3cf8069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:29.305 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.293+0000 7f3d1a352700 1 --2- 172.21.15.163:0/598406003 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3cf8066ea0 0x7f3cf8069350 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3d10009970 tx=0x7f3d10008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:29.305 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.297+0000 7f3d0a7fc700 1 -- 172.21.15.163:0/598406003 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d0c0537e0 con 0x7f3d1415daf0 2023-10-09T06:30:29.481 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.481+0000 7f3d1b354700 1 -- 172.21.15.163:0/598406003 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d140b1d20 con 0x7f3d1415daf0 2023-10-09T06:30:29.482 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.482+0000 7f3d0a7fc700 1 -- 172.21.15.163:0/598406003 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f3d0c056e00 con 0x7f3d1415daf0 2023-10-09T06:30:29.482 INFO:teuthology.misc.health.smithi163.stdout: 2023-10-09T06:30:29.482 INFO:teuthology.misc.health.smithi163.stdout:{"epoch":12,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:30:26.464837+0000","last_up_change":"0.000000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:30:29.484 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.484+0000 7f3d1b354700 1 -- 172.21.15.163:0/598406003 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3cf8066ea0 msgr2=0x7f3cf8069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:29.484 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.484+0000 7f3d1b354700 1 --2- 172.21.15.163:0/598406003 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3cf8066ea0 0x7f3cf8069350 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3d10009970 tx=0x7f3d10008040 comp rx=0 tx=0).stop 2023-10-09T06:30:29.484 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.484+0000 7f3d1b354700 1 -- 172.21.15.163:0/598406003 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d1415daf0 msgr2=0x7f3d140cebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:29.484 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.484+0000 7f3d1b354700 1 --2- 172.21.15.163:0/598406003 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d1415daf0 0x7f3d140cebe0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c009430 tx=0x7f3d0c018000 comp rx=0 tx=0).stop 2023-10-09T06:30:29.485 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.484+0000 7f3d1b354700 1 -- 172.21.15.163:0/598406003 shutdown_connections 2023-10-09T06:30:29.485 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.484+0000 7f3d1b354700 1 --2- 172.21.15.163:0/598406003 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3cf8066ea0 0x7f3cf8069350 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:29.485 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.484+0000 7f3d1b354700 1 --2- 172.21.15.163:0/598406003 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d1415daf0 0x7f3d140cebe0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:29.485 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.484+0000 7f3d1b354700 1 --2- 172.21.15.163:0/598406003 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d140b4730 0x7f3d1409a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:29.485 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.484+0000 7f3d1b354700 1 --2- 172.21.15.163:0/598406003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d1415ce50 0x7f3d140cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:29.485 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.484+0000 7f3d1b354700 1 -- 172.21.15.163:0/598406003 >> 172.21.15.163:0/598406003 conn(0x7f3d14087ff0 msgr2=0x7f3d1415a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:29.485 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.484+0000 7f3d1b354700 1 -- 172.21.15.163:0/598406003 shutdown_connections 2023-10-09T06:30:29.485 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:29.484+0000 7f3d1b354700 1 -- 172.21.15.163:0/598406003 wait complete. 2023-10-09T06:30:29.495 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-09T06:30:36.304 DEBUG:teuthology.orchestra.run.smithi163:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-09T06:30:36.421 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.420+0000 7f8134b5d700 1 Processor -- start 2023-10-09T06:30:36.421 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.420+0000 7f8134b5d700 1 -- start start 2023-10-09T06:30:36.423 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.421+0000 7f8134b5d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f813015dcc0 0x7f81300b3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:36.423 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.421+0000 7f8134b5d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8130159ef0 con 0x7f813015ce50 2023-10-09T06:30:36.423 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.421+0000 7f8134b5d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81300b6090 con 0x7f813015b140 2023-10-09T06:30:36.423 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.421+0000 7f8134b5d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81300b5480 con 0x7f813015dcc0 2023-10-09T06:30:36.423 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.421+0000 7f812f7fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f813015ce50 0x7f813015d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:35032/0 (socket says 172.21.15.163:35032) 2023-10-09T06:30:36.424 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.421+0000 7f812effd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f813015dcc0 0x7f81300b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:36.424 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.421+0000 7f812f7fe700 1 -- 172.21.15.163:0/3031725103 learned_addr learned my addr 172.21.15.163:0/3031725103 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:36.424 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.421+0000 7f812e7fc700 1 -- 172.21.15.163:0/3031725103 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4229580077 0 0) 0x7f8130159ef0 con 0x7f813015ce50 2023-10-09T06:30:36.424 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.421+0000 7f812e7fc700 1 -- 172.21.15.163:0/3031725103 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8118003570 con 0x7f813015ce50 2023-10-09T06:30:36.424 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.421+0000 7f812effd700 1 -- 172.21.15.163:0/3031725103 >> v1:172.21.15.181:6789/0 conn(0x7f813015b140 legacy=0x7f81300b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:36.424 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.421+0000 7f812effd700 1 -- 172.21.15.163:0/3031725103 >> v1:172.21.15.163:6789/0 conn(0x7f813015ce50 legacy=0x7f813015d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:36.424 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.422+0000 7f812effd700 1 -- 172.21.15.163:0/3031725103 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81300b5760 con 0x7f813015dcc0 2023-10-09T06:30:36.424 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.422+0000 7f812effd700 1 --2- 172.21.15.163:0/3031725103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f813015dcc0 0x7f81300b3720 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f81240098e0 tx=0x7f81240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf61c25cf8f0820 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:36.424 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.422+0000 7f812e7fc700 1 -- 172.21.15.163:0/3031725103 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812400b690 con 0x7f813015dcc0 2023-10-09T06:30:36.424 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.422+0000 7f812e7fc700 1 -- 172.21.15.163:0/3031725103 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f812400b7f0 con 0x7f813015dcc0 2023-10-09T06:30:36.424 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.422+0000 7f812e7fc700 1 -- 172.21.15.163:0/3031725103 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812400bb90 con 0x7f813015dcc0 2023-10-09T06:30:36.425 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.422+0000 7f8134b5d700 1 -- 172.21.15.163:0/3031725103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f813015dcc0 msgr2=0x7f81300b3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:36.425 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.422+0000 7f8134b5d700 1 --2- 172.21.15.163:0/3031725103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f813015dcc0 0x7f81300b3720 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f81240098e0 tx=0x7f81240091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:36.425 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.422+0000 7f8134b5d700 1 -- 172.21.15.163:0/3031725103 shutdown_connections 2023-10-09T06:30:36.425 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.422+0000 7f8134b5d700 1 --2- 172.21.15.163:0/3031725103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f813015dcc0 0x7f81300b3720 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:36.425 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.422+0000 7f8134b5d700 1 -- 172.21.15.163:0/3031725103 >> 172.21.15.163:0/3031725103 conn(0x7f8130087ff0 msgr2=0x7f813006d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:36.425 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.422+0000 7f8134b5d700 1 -- 172.21.15.163:0/3031725103 shutdown_connections 2023-10-09T06:30:36.425 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.422+0000 7f8134b5d700 1 -- 172.21.15.163:0/3031725103 wait complete. 2023-10-09T06:30:36.427 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.424+0000 7f8134b5d700 1 Processor -- start 2023-10-09T06:30:36.428 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.424+0000 7f8134b5d700 1 -- start start 2023-10-09T06:30:36.428 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.424+0000 7f8134b5d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f813015b140 0x7f813016b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:36.428 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.424+0000 7f8134b5d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f813015ce50 0x7f81300af290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:36.428 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.424+0000 7f8134b5d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f813015dcc0 0x7f8130155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:36.428 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.424+0000 7f8134b5d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8130167ee0 con 0x7f813015ce50 2023-10-09T06:30:36.428 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.424+0000 7f8134b5d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8130153f90 con 0x7f813015b140 2023-10-09T06:30:36.428 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.424+0000 7f8134b5d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8130154270 con 0x7f813015dcc0 2023-10-09T06:30:36.428 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.424+0000 7f812effd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f813015ce50 0x7f81300af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:36.428 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.425+0000 7f812effd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f813015ce50 0x7f81300af290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47612/0 (socket says 172.21.15.163:47612) 2023-10-09T06:30:36.428 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.425+0000 7f812effd700 1 -- 172.21.15.163:0/2655715745 learned_addr learned my addr 172.21.15.163:0/2655715745 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:36.429 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.425+0000 7f812ffff700 1 --2- 172.21.15.163:0/2655715745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f813015dcc0 0x7f8130155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:36.429 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.425+0000 7f812f7fe700 1 --2- 172.21.15.163:0/2655715745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f813015b140 0x7f813016b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:36.429 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.425+0000 7f812effd700 1 -- 172.21.15.163:0/2655715745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f813015dcc0 msgr2=0x7f8130155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:36.429 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.425+0000 7f812effd700 1 --2- 172.21.15.163:0/2655715745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f813015dcc0 0x7f8130155030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:36.429 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.425+0000 7f812effd700 1 -- 172.21.15.163:0/2655715745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f813015b140 msgr2=0x7f813016b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:36.429 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.425+0000 7f812effd700 1 --2- 172.21.15.163:0/2655715745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f813015b140 0x7f813016b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:36.429 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.425+0000 7f812effd700 1 -- 172.21.15.163:0/2655715745 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8124009420 con 0x7f813015ce50 2023-10-09T06:30:36.429 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.425+0000 7f812f7fe700 1 --2- 172.21.15.163:0/2655715745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f813015b140 0x7f813016b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:30:36.429 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.425+0000 7f812effd700 1 --2- 172.21.15.163:0/2655715745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f813015ce50 0x7f81300af290 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f812400a010 tx=0x7f8124016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:36.429 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.425+0000 7f81177fe700 1 -- 172.21.15.163:0/2655715745 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8124000c40 con 0x7f813015ce50 2023-10-09T06:30:36.429 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.425+0000 7f81177fe700 1 -- 172.21.15.163:0/2655715745 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8124020070 con 0x7f813015ce50 2023-10-09T06:30:36.429 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.425+0000 7f8134b5d700 1 -- 172.21.15.163:0/2655715745 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8130154550 con 0x7f813015ce50 2023-10-09T06:30:36.430 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.425+0000 7f81177fe700 1 -- 172.21.15.163:0/2655715745 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812401b6a0 con 0x7f813015ce50 2023-10-09T06:30:36.430 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.425+0000 7f8134b5d700 1 -- 172.21.15.163:0/2655715745 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8130154b10 con 0x7f813015ce50 2023-10-09T06:30:36.430 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.426+0000 7f8134b5d700 1 -- 172.21.15.163:0/2655715745 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8130154d00 con 0x7f813015ce50 2023-10-09T06:30:36.430 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.427+0000 7f81177fe700 1 -- 172.21.15.163:0/2655715745 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8124022030 con 0x7f813015ce50 2023-10-09T06:30:36.430 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.427+0000 7f81177fe700 1 --2- 172.21.15.163:0/2655715745 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f811c066f30 0x7f811c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:36.430 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.427+0000 7f81177fe700 1 -- 172.21.15.163:0/2655715745 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8124085c20 con 0x7f813015ce50 2023-10-09T06:30:36.430 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.427+0000 7f812f7fe700 1 --2- 172.21.15.163:0/2655715745 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f811c066f30 0x7f811c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:36.430 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.428+0000 7f812f7fe700 1 --2- 172.21.15.163:0/2655715745 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f811c066f30 0x7f811c0693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8120001360 tx=0x7f8120009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:36.432 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.431+0000 7f81177fe700 1 -- 172.21.15.163:0/2655715745 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f81240531e0 con 0x7f813015ce50 2023-10-09T06:30:36.617 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.617+0000 7f8134b5d700 1 -- 172.21.15.163:0/2655715745 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8130064b50 con 0x7f813015ce50 2023-10-09T06:30:36.618 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.618+0000 7f81177fe700 1 -- 172.21.15.163:0/2655715745 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12175 (secure 0 0 0) 0x7f8124056800 con 0x7f813015ce50 2023-10-09T06:30:36.618 INFO:teuthology.misc.health.smithi163.stdout: 2023-10-09T06:30:36.619 INFO:teuthology.misc.health.smithi163.stdout:{"epoch":16,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:30:32.782320+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:30:36.621 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.619+0000 7f8134b5d700 1 -- 172.21.15.163:0/2655715745 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f811c066f30 msgr2=0x7f811c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:36.621 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.619+0000 7f8134b5d700 1 --2- 172.21.15.163:0/2655715745 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f811c066f30 0x7f811c0693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8120001360 tx=0x7f8120009040 comp rx=0 tx=0).stop 2023-10-09T06:30:36.621 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.619+0000 7f8134b5d700 1 -- 172.21.15.163:0/2655715745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f813015ce50 msgr2=0x7f81300af290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:36.621 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.619+0000 7f8134b5d700 1 --2- 172.21.15.163:0/2655715745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f813015ce50 0x7f81300af290 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f812400a010 tx=0x7f8124016000 comp rx=0 tx=0).stop 2023-10-09T06:30:36.621 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.619+0000 7f8134b5d700 1 -- 172.21.15.163:0/2655715745 shutdown_connections 2023-10-09T06:30:36.621 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.619+0000 7f8134b5d700 1 --2- 172.21.15.163:0/2655715745 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f811c066f30 0x7f811c0693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:36.622 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.619+0000 7f8134b5d700 1 --2- 172.21.15.163:0/2655715745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f813015dcc0 0x7f8130155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:36.622 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.619+0000 7f8134b5d700 1 --2- 172.21.15.163:0/2655715745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f813015b140 0x7f813016b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:36.622 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.619+0000 7f8134b5d700 1 --2- 172.21.15.163:0/2655715745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f813015ce50 0x7f81300af290 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:36.622 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.619+0000 7f8134b5d700 1 -- 172.21.15.163:0/2655715745 >> 172.21.15.163:0/2655715745 conn(0x7f8130087ff0 msgr2=0x7f813015b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:36.622 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.620+0000 7f8134b5d700 1 -- 172.21.15.163:0/2655715745 shutdown_connections 2023-10-09T06:30:36.622 INFO:teuthology.misc.health.smithi163.stderr:2023-10-09T06:30:36.620+0000 7f8134b5d700 1 -- 172.21.15.163:0/2655715745 wait complete. 2023-10-09T06:30:36.631 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-09T06:30:36.632 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-09T06:30:36.632 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-09T06:30:36.632 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-09T06:30:36.635 INFO:tasks.ceph.mds.a:Started 2023-10-09T06:30:36.635 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-09T06:30:36.635 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-09T06:30:36.637 INFO:tasks.ceph.mds.c:Started 2023-10-09T06:30:36.637 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-09T06:30:36.637 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-09T06:30:36.640 INFO:tasks.ceph.mds.b:Started 2023-10-09T06:30:36.640 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-09T06:30:36.640 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-09T06:30:36.642 INFO:tasks.ceph.mds.d:Started 2023-10-09T06:30:36.642 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-09T06:30:36.642 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:30:36.708 INFO:tasks.ceph.mds.a.smithi163.stdout:starting mds.a at 2023-10-09T06:30:36.709 INFO:tasks.ceph.mds.c.smithi163.stdout:starting mds.c at 2023-10-09T06:30:36.722 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-09T06:30:36.722 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-09T06:30:36.723 INFO:tasks.ceph.mds.d.smithi181.stderr:2023-10-09T06:30:36.722+0000 7f84e7b5c700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-09T06:30:36.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.763+0000 7f88c2f05700 1 Processor -- start 2023-10-09T06:30:36.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.763+0000 7f88c2f05700 1 -- start start 2023-10-09T06:30:36.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.764+0000 7f88c2f05700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88bc157960 0x7f88bc0af250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:36.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.764+0000 7f88c2f05700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f88bc06b4d0 0x7f88bc068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:36.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.764+0000 7f88c2f05700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f88bc06b940 0x7f88bc0688d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:36.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.764+0000 7f88c2f05700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88bc0ac760 con 0x7f88bc06b940 2023-10-09T06:30:36.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.764+0000 7f88c2f05700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88bc0af830 con 0x7f88bc06b4d0 2023-10-09T06:30:36.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.764+0000 7f88c2f05700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc068e10 con 0x7f88bc157960 2023-10-09T06:30:36.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.764+0000 7f88c14a2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f88bc06b940 0x7f88bc0688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:36.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.764+0000 7f88c14a2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f88bc06b940 0x7f88bc0688d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47654/0 (socket says 172.21.15.163:47654) 2023-10-09T06:30:36.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.764+0000 7f88c14a2700 1 -- 172.21.15.163:0/2211295592 learned_addr learned my addr 172.21.15.163:0/2211295592 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:36.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.764+0000 7f88c0ca1700 1 --2- 172.21.15.163:0/2211295592 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88bc157960 0x7f88bc0af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:36.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.764+0000 7f88bbfff700 1 --2- 172.21.15.163:0/2211295592 >> v2:172.21.15.181:3300/0 conn(0x7f88bc06b4d0 0x7f88bc068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:36.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.764+0000 7f88c14a2700 1 -- 172.21.15.163:0/2211295592 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88bc157960 msgr2=0x7f88bc0af250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:36.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.764+0000 7f88c14a2700 1 --2- 172.21.15.163:0/2211295592 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88bc157960 0x7f88bc0af250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:36.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.764+0000 7f88c14a2700 1 -- 172.21.15.163:0/2211295592 >> v2:172.21.15.181:3300/0 conn(0x7f88bc06b4d0 msgr2=0x7f88bc068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:36.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.764+0000 7f88c14a2700 1 --2- 172.21.15.163:0/2211295592 >> v2:172.21.15.181:3300/0 conn(0x7f88bc06b4d0 0x7f88bc068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:36.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.764+0000 7f88c14a2700 1 -- 172.21.15.163:0/2211295592 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88bc06b030 con 0x7f88bc06b940 2023-10-09T06:30:36.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.764+0000 7f88c14a2700 1 --2- 172.21.15.163:0/2211295592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88bc06b940 0x7f88bc0688d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0ccc70 tx=0x7f88b4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c02438ee13b6b5f server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:36.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.764+0000 7f88bb7fe700 1 -- 172.21.15.163:0/2211295592 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b400e070 con 0x7f88bc06b940 2023-10-09T06:30:36.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.764+0000 7f88bb7fe700 1 -- 172.21.15.163:0/2211295592 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b400ba50 con 0x7f88bc06b940 2023-10-09T06:30:36.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.764+0000 7f88bb7fe700 1 -- 172.21.15.163:0/2211295592 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b40092e0 con 0x7f88bc06b940 2023-10-09T06:30:36.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.765+0000 7f88c2f05700 1 -- 172.21.15.163:0/2211295592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88bc06b940 msgr2=0x7f88bc0688d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:36.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.765+0000 7f88c2f05700 1 --2- 172.21.15.163:0/2211295592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88bc06b940 0x7f88bc0688d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0ccc70 tx=0x7f88b4009840 comp rx=0 tx=0).stop 2023-10-09T06:30:36.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.765+0000 7f88c2f05700 1 -- 172.21.15.163:0/2211295592 shutdown_connections 2023-10-09T06:30:36.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.765+0000 7f88c2f05700 1 --2- 172.21.15.163:0/2211295592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88bc06b940 0x7f88bc0688d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:36.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.765+0000 7f88c2f05700 1 --2- 172.21.15.163:0/2211295592 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88bc157960 0x7f88bc0af250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:36.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.765+0000 7f88c2f05700 1 --2- 172.21.15.163:0/2211295592 >> v2:172.21.15.181:3300/0 conn(0x7f88bc06b4d0 0x7f88bc068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:36.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.765+0000 7f88c2f05700 1 -- 172.21.15.163:0/2211295592 >> 172.21.15.163:0/2211295592 conn(0x7f88bc0c9870 msgr2=0x7f88bc0c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:36.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.765+0000 7f88c2f05700 1 -- 172.21.15.163:0/2211295592 shutdown_connections 2023-10-09T06:30:36.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.765+0000 7f88c2f05700 1 -- 172.21.15.163:0/2211295592 wait complete. 2023-10-09T06:30:36.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.766+0000 7f88c2f05700 1 Processor -- start 2023-10-09T06:30:36.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.766+0000 7f88c2f05700 1 -- start start 2023-10-09T06:30:36.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.767+0000 7f88c2f05700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88bc06b4d0 0x7f88bc1584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:36.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.767+0000 7f88c2f05700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88bc06b940 0x7f88bc0c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:36.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.767+0000 7f88c2f05700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88bc157960 0x7f88bc0afba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:36.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.767+0000 7f88c2f05700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc158af0 con 0x7f88bc157960 2023-10-09T06:30:36.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.767+0000 7f88c2f05700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc0ae2b0 con 0x7f88bc06b940 2023-10-09T06:30:36.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.767+0000 7f88c2f05700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc0c4980 con 0x7f88bc06b4d0 2023-10-09T06:30:36.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.767+0000 7f88c14a2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88bc157960 0x7f88bc0afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:36.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.767+0000 7f88c14a2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88bc157960 0x7f88bc0afba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47670/0 (socket says 172.21.15.163:47670) 2023-10-09T06:30:36.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.767+0000 7f88c14a2700 1 -- 172.21.15.163:0/1563335472 learned_addr learned my addr 172.21.15.163:0/1563335472 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:36.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.767+0000 7f88c0ca1700 1 --2- 172.21.15.163:0/1563335472 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88bc06b4d0 0x7f88bc1584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:36.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.767+0000 7f88bbfff700 1 --2- 172.21.15.163:0/1563335472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88bc06b940 0x7f88bc0c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:36.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.767+0000 7f88c14a2700 1 -- 172.21.15.163:0/1563335472 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88bc06b4d0 msgr2=0x7f88bc1584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:36.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.767+0000 7f88c14a2700 1 --2- 172.21.15.163:0/1563335472 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88bc06b4d0 0x7f88bc1584a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:36.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.767+0000 7f88c14a2700 1 -- 172.21.15.163:0/1563335472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88bc06b940 msgr2=0x7f88bc0c6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:36.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.767+0000 7f88c14a2700 1 --2- 172.21.15.163:0/1563335472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88bc06b940 0x7f88bc0c6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:36.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.767+0000 7f88c14a2700 1 -- 172.21.15.163:0/1563335472 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b4009ae0 con 0x7f88bc157960 2023-10-09T06:30:36.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.767+0000 7f88c14a2700 1 --2- 172.21.15.163:0/1563335472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88bc157960 0x7f88bc0afba0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f88bc06cac0 tx=0x7f88b4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:36.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.768+0000 7f88b8ff9700 1 -- 172.21.15.163:0/1563335472 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b40032f0 con 0x7f88bc157960 2023-10-09T06:30:36.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.768+0000 7f88b8ff9700 1 -- 172.21.15.163:0/1563335472 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b4008e80 con 0x7f88bc157960 2023-10-09T06:30:36.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.768+0000 7f88c2f05700 1 -- 172.21.15.163:0/1563335472 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88bc0b00e0 con 0x7f88bc157960 2023-10-09T06:30:36.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.768+0000 7f88b8ff9700 1 -- 172.21.15.163:0/1563335472 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b40266b0 con 0x7f88bc157960 2023-10-09T06:30:36.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.768+0000 7f88c2f05700 1 -- 172.21.15.163:0/1563335472 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88bc0c4330 con 0x7f88bc157960 2023-10-09T06:30:36.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.769+0000 7f88c2f05700 1 -- 172.21.15.163:0/1563335472 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88bc04c3c0 con 0x7f88bc157960 2023-10-09T06:30:36.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.769+0000 7f88b8ff9700 1 -- 172.21.15.163:0/1563335472 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f88b4003040 con 0x7f88bc157960 2023-10-09T06:30:36.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.769+0000 7f88b8ff9700 1 --2- 172.21.15.163:0/1563335472 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f88a0066f30 0x7f88a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:36.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.769+0000 7f88b8ff9700 1 -- 172.21.15.163:0/1563335472 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f88b4085cc0 con 0x7f88bc157960 2023-10-09T06:30:36.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.769+0000 7f88c0ca1700 1 --2- 172.21.15.163:0/1563335472 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f88a0066f30 0x7f88a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:36.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.770+0000 7f88c0ca1700 1 --2- 172.21.15.163:0/1563335472 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f88a0066f30 0x7f88a00693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f88b00098e0 tx=0x7f88b0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:36.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.773+0000 7f88b8ff9700 1 -- 172.21.15.163:0/1563335472 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f88b4053330 con 0x7f88bc157960 2023-10-09T06:30:36.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.960+0000 7f88c2f05700 1 -- 172.21.15.163:0/1563335472 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88bc0c5cb0 con 0x7f88bc157960 2023-10-09T06:30:36.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.961+0000 7f88b8ff9700 1 -- 172.21.15.163:0/1563335472 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12175 (secure 0 0 0) 0x7f88b4009940 con 0x7f88bc157960 2023-10-09T06:30:36.962 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:30:36.962 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":16,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:30:32.782320+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:30:36.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.963+0000 7f88c2f05700 1 -- 172.21.15.163:0/1563335472 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f88a0066f30 msgr2=0x7f88a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:36.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.963+0000 7f88c2f05700 1 --2- 172.21.15.163:0/1563335472 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f88a0066f30 0x7f88a00693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f88b00098e0 tx=0x7f88b0008040 comp rx=0 tx=0).stop 2023-10-09T06:30:36.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.963+0000 7f88c2f05700 1 -- 172.21.15.163:0/1563335472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88bc157960 msgr2=0x7f88bc0afba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:36.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.963+0000 7f88c2f05700 1 --2- 172.21.15.163:0/1563335472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88bc157960 0x7f88bc0afba0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f88bc06cac0 tx=0x7f88b4018000 comp rx=0 tx=0).stop 2023-10-09T06:30:36.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.963+0000 7f88c2f05700 1 -- 172.21.15.163:0/1563335472 shutdown_connections 2023-10-09T06:30:36.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.963+0000 7f88c2f05700 1 --2- 172.21.15.163:0/1563335472 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f88a0066f30 0x7f88a00693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:36.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.963+0000 7f88c2f05700 1 --2- 172.21.15.163:0/1563335472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88bc157960 0x7f88bc0afba0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:36.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.963+0000 7f88c2f05700 1 --2- 172.21.15.163:0/1563335472 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88bc06b4d0 0x7f88bc1584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:36.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.963+0000 7f88c2f05700 1 --2- 172.21.15.163:0/1563335472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88bc06b940 0x7f88bc0c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:36.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.963+0000 7f88c2f05700 1 -- 172.21.15.163:0/1563335472 >> 172.21.15.163:0/1563335472 conn(0x7f88bc0c9870 msgr2=0x7f88bc068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:36.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.963+0000 7f88c2f05700 1 -- 172.21.15.163:0/1563335472 shutdown_connections 2023-10-09T06:30:36.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:36.963+0000 7f88c2f05700 1 -- 172.21.15.163:0/1563335472 wait complete. 2023-10-09T06:30:36.976 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-09T06:30:36.977 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:30:37.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.095+0000 7f1a9ede5700 1 Processor -- start 2023-10-09T06:30:37.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.095+0000 7f1a9ede5700 1 -- start start 2023-10-09T06:30:37.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.096+0000 7f1a9ede5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1a98069e80 0x7f1a980c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:37.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.096+0000 7f1a9ede5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a98157bd0 0x7f1a980c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:37.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.096+0000 7f1a9ede5700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a980c8e00 con 0x7f1a98157800 2023-10-09T06:30:37.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.096+0000 7f1a9ede5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a980c8600 con 0x7f1a98069e80 2023-10-09T06:30:37.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.096+0000 7f1a9ede5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a980af800 con 0x7f1a98157bd0 2023-10-09T06:30:37.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.096+0000 7f1a9cb81700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f1a98157800 0x7f1a98069770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:35058/0 (socket says 172.21.15.163:35058) 2023-10-09T06:30:37.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.096+0000 7f1a9cb81700 1 -- 172.21.15.163:0/3165867733 learned_addr learned my addr 172.21.15.163:0/3165867733 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:37.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.096+0000 7f1a9d382700 1 --2- 172.21.15.163:0/3165867733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a98157bd0 0x7f1a980c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:37.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.096+0000 7f1a97fff700 1 --2- 172.21.15.163:0/3165867733 >> v2:172.21.15.181:3300/0 conn(0x7f1a98069e80 0x7f1a980c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:37.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.096+0000 7f1a977fe700 1 -- 172.21.15.163:0/3165867733 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2264127620 0 0) 0x7f1a8c001110 con 0x7f1a98157800 2023-10-09T06:30:37.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.096+0000 7f1a977fe700 1 -- 172.21.15.163:0/3165867733 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a88003570 con 0x7f1a98157800 2023-10-09T06:30:37.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.097+0000 7f1a977fe700 1 -- 172.21.15.163:0/3165867733 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1521874027 0 0) 0x7f1a88003570 con 0x7f1a98157800 2023-10-09T06:30:37.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.097+0000 7f1a977fe700 1 -- 172.21.15.163:0/3165867733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a98157bd0 msgr2=0x7f1a980c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:37.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.097+0000 7f1a977fe700 1 --2- 172.21.15.163:0/3165867733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a98157bd0 0x7f1a980c70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:37.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.097+0000 7f1a977fe700 1 -- 172.21.15.163:0/3165867733 >> v2:172.21.15.181:3300/0 conn(0x7f1a98069e80 msgr2=0x7f1a980c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:37.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.097+0000 7f1a977fe700 1 --2- 172.21.15.163:0/3165867733 >> v2:172.21.15.181:3300/0 conn(0x7f1a98069e80 0x7f1a980c6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:37.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.097+0000 7f1a977fe700 1 -- 172.21.15.163:0/3165867733 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a980c8840 con 0x7f1a98157800 2023-10-09T06:30:37.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.097+0000 7f1a977fe700 1 -- 172.21.15.163:0/3165867733 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f1a8c001600 con 0x7f1a98157800 2023-10-09T06:30:37.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.097+0000 7f1a977fe700 1 -- 172.21.15.163:0/3165867733 >> v1:172.21.15.163:6789/0 conn(0x7f1a98157800 legacy=0x7f1a98069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:37.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.097+0000 7f1a977fe700 1 --2- 172.21.15.163:0/3165867733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a88003a60 0x7f1a88003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:37.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.097+0000 7f1a977fe700 1 --2- 172.21.15.163:0/3165867733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a88004810 0x7f1a88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:37.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.097+0000 7f1a977fe700 1 --2- 172.21.15.163:0/3165867733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a98157bd0 0x7f1a880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:37.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.097+0000 7f1a977fe700 1 -- 172.21.15.163:0/3165867733 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a980c8600 con 0x7f1a88003a60 2023-10-09T06:30:37.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.097+0000 7f1a977fe700 1 -- 172.21.15.163:0/3165867733 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a980af800 con 0x7f1a88004810 2023-10-09T06:30:37.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.097+0000 7f1a977fe700 1 -- 172.21.15.163:0/3165867733 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8800ba60 con 0x7f1a98157bd0 2023-10-09T06:30:37.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.097+0000 7f1a9cb81700 1 --2- 172.21.15.163:0/3165867733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a88003a60 0x7f1a88003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:37.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.097+0000 7f1a97fff700 1 --2- 172.21.15.163:0/3165867733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a88004810 0x7f1a88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:37.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.097+0000 7f1a9d382700 1 --2- 172.21.15.163:0/3165867733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a98157bd0 0x7f1a880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:37.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.097+0000 7f1a9cb81700 1 -- 172.21.15.163:0/3165867733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a98157bd0 msgr2=0x7f1a880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:37.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.097+0000 7f1a9cb81700 1 --2- 172.21.15.163:0/3165867733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a98157bd0 0x7f1a880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:37.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.097+0000 7f1a9cb81700 1 -- 172.21.15.163:0/3165867733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a88004810 msgr2=0x7f1a88004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:37.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.097+0000 7f1a9cb81700 1 --2- 172.21.15.163:0/3165867733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a88004810 0x7f1a88004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:37.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.097+0000 7f1a9cb81700 1 -- 172.21.15.163:0/3165867733 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a88003570 con 0x7f1a88003a60 2023-10-09T06:30:37.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.097+0000 7f1a9d382700 1 --2- 172.21.15.163:0/3165867733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a98157bd0 0x7f1a880051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:30:37.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.097+0000 7f1a9cb81700 1 --2- 172.21.15.163:0/3165867733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a88003a60 0x7f1a88003e70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00c620 tx=0x7f1a8c007a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fba9b3d0963759b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:37.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.097+0000 7f1a977fe700 1 -- 172.21.15.163:0/3165867733 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c00cde0 con 0x7f1a88003a60 2023-10-09T06:30:37.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.097+0000 7f1a977fe700 1 -- 172.21.15.163:0/3165867733 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8c005920 con 0x7f1a88003a60 2023-10-09T06:30:37.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.097+0000 7f1a977fe700 1 -- 172.21.15.163:0/3165867733 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c005ce0 con 0x7f1a88003a60 2023-10-09T06:30:37.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.098+0000 7f1a9ede5700 1 -- 172.21.15.163:0/3165867733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a88003a60 msgr2=0x7f1a88003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:37.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.098+0000 7f1a9ede5700 1 --2- 172.21.15.163:0/3165867733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a88003a60 0x7f1a88003e70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00c620 tx=0x7f1a8c007a80 comp rx=0 tx=0).stop 2023-10-09T06:30:37.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.098+0000 7f1a9d382700 1 -- 172.21.15.163:0/3165867733 reap_dead start 2023-10-09T06:30:37.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.098+0000 7f1a9ede5700 1 -- 172.21.15.163:0/3165867733 shutdown_connections 2023-10-09T06:30:37.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.098+0000 7f1a9ede5700 1 -- 172.21.15.163:0/3165867733 >> 172.21.15.163:0/3165867733 conn(0x7f1a98068e10 msgr2=0x7f1a980cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:37.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.098+0000 7f1a9ede5700 1 -- 172.21.15.163:0/3165867733 shutdown_connections 2023-10-09T06:30:37.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.098+0000 7f1a9ede5700 1 -- 172.21.15.163:0/3165867733 wait complete. 2023-10-09T06:30:37.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.099+0000 7f1a9ede5700 1 Processor -- start 2023-10-09T06:30:37.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.099+0000 7f1a9ede5700 1 -- start start 2023-10-09T06:30:37.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.100+0000 7f1a9ede5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a88003a60 0x7f1a98149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:37.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.100+0000 7f1a9ede5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a9816b470 0x7f1a98155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:37.111 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.100+0000 7f1a9ede5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a981562e0 0x7f1a981505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:37.111 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.100+0000 7f1a9ede5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a98156780 con 0x7f1a9816b470 2023-10-09T06:30:37.111 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.100+0000 7f1a9ede5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a98152760 con 0x7f1a981562e0 2023-10-09T06:30:37.111 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.100+0000 7f1a9ede5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a980c61c0 con 0x7f1a88003a60 2023-10-09T06:30:37.111 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.100+0000 7f1a97fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a9816b470 0x7f1a98155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:37.111 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.100+0000 7f1a97fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a9816b470 0x7f1a98155da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47688/0 (socket says 172.21.15.163:47688) 2023-10-09T06:30:37.111 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.100+0000 7f1a97fff700 1 -- 172.21.15.163:0/329656913 learned_addr learned my addr 172.21.15.163:0/329656913 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:37.111 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.100+0000 7f1a9d382700 1 --2- 172.21.15.163:0/329656913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a981562e0 0x7f1a981505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:37.111 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.100+0000 7f1a9cb81700 1 --2- 172.21.15.163:0/329656913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a88003a60 0x7f1a98149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:37.111 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.100+0000 7f1a97fff700 1 -- 172.21.15.163:0/329656913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a88003a60 msgr2=0x7f1a98149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:37.112 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.100+0000 7f1a97fff700 1 --2- 172.21.15.163:0/329656913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a88003a60 0x7f1a98149240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:37.112 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.100+0000 7f1a97fff700 1 -- 172.21.15.163:0/329656913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a981562e0 msgr2=0x7f1a981505a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:37.112 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.100+0000 7f1a97fff700 1 --2- 172.21.15.163:0/329656913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a981562e0 0x7f1a981505a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:37.112 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.100+0000 7f1a97fff700 1 -- 172.21.15.163:0/329656913 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a980c55b0 con 0x7f1a9816b470 2023-10-09T06:30:37.112 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.100+0000 7f1a97fff700 1 --2- 172.21.15.163:0/329656913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a9816b470 0x7f1a98155da0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1a84011b70 tx=0x7f1a840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:37.112 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.100+0000 7f1a94ff9700 1 -- 172.21.15.163:0/329656913 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8400f9f0 con 0x7f1a9816b470 2023-10-09T06:30:37.112 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.100+0000 7f1a94ff9700 1 -- 172.21.15.163:0/329656913 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8400fb50 con 0x7f1a9816b470 2023-10-09T06:30:37.112 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.101+0000 7f1a9ede5700 1 -- 172.21.15.163:0/329656913 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a980c5890 con 0x7f1a9816b470 2023-10-09T06:30:37.112 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.101+0000 7f1a94ff9700 1 -- 172.21.15.163:0/329656913 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8400fed0 con 0x7f1a9816b470 2023-10-09T06:30:37.112 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.101+0000 7f1a9ede5700 1 -- 172.21.15.163:0/329656913 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a980c5e50 con 0x7f1a9816b470 2023-10-09T06:30:37.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.102+0000 7f1a9ede5700 1 -- 172.21.15.163:0/329656913 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a9804c3c0 con 0x7f1a9816b470 2023-10-09T06:30:37.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.102+0000 7f1a94ff9700 1 -- 172.21.15.163:0/329656913 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1a84014650 con 0x7f1a9816b470 2023-10-09T06:30:37.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.102+0000 7f1a94ff9700 1 --2- 172.21.15.163:0/329656913 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a7c066f30 0x7f1a7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:37.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.102+0000 7f1a94ff9700 1 -- 172.21.15.163:0/329656913 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f1a84018030 con 0x7f1a9816b470 2023-10-09T06:30:37.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.103+0000 7f1a9cb81700 1 --2- 172.21.15.163:0/329656913 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a7c066f30 0x7f1a7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:37.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.103+0000 7f1a9cb81700 1 --2- 172.21.15.163:0/329656913 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a7c066f30 0x7f1a7c0693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c008fd0 tx=0x7f1a8c015000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:37.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.107+0000 7f1a94ff9700 1 -- 172.21.15.163:0/329656913 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a840521a0 con 0x7f1a9816b470 2023-10-09T06:30:37.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.286+0000 7f1a9ede5700 1 -- 172.21.15.163:0/329656913 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1a980c6300 con 0x7f1a9816b470 2023-10-09T06:30:37.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.286+0000 7f1a94ff9700 1 -- 172.21.15.163:0/329656913 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1a840220a0 con 0x7f1a9816b470 2023-10-09T06:30:37.288 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:30:37.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.288+0000 7f1a9ede5700 1 -- 172.21.15.163:0/329656913 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a7c066f30 msgr2=0x7f1a7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:37.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.288+0000 7f1a9ede5700 1 --2- 172.21.15.163:0/329656913 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a7c066f30 0x7f1a7c0693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c008fd0 tx=0x7f1a8c015000 comp rx=0 tx=0).stop 2023-10-09T06:30:37.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.288+0000 7f1a9ede5700 1 -- 172.21.15.163:0/329656913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a9816b470 msgr2=0x7f1a98155da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:37.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.288+0000 7f1a9ede5700 1 --2- 172.21.15.163:0/329656913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a9816b470 0x7f1a98155da0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1a84011b70 tx=0x7f1a840091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:37.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.288+0000 7f1a9ede5700 1 -- 172.21.15.163:0/329656913 shutdown_connections 2023-10-09T06:30:37.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.288+0000 7f1a9ede5700 1 --2- 172.21.15.163:0/329656913 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a7c066f30 0x7f1a7c0693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:37.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.288+0000 7f1a9ede5700 1 --2- 172.21.15.163:0/329656913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a981562e0 0x7f1a981505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:37.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.288+0000 7f1a9ede5700 1 --2- 172.21.15.163:0/329656913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a88003a60 0x7f1a98149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:37.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.288+0000 7f1a9ede5700 1 --2- 172.21.15.163:0/329656913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a9816b470 0x7f1a98155da0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:37.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.288+0000 7f1a9ede5700 1 -- 172.21.15.163:0/329656913 >> 172.21.15.163:0/329656913 conn(0x7f1a98068e10 msgr2=0x7f1a98154f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:37.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.288+0000 7f1a9ede5700 1 -- 172.21.15.163:0/329656913 shutdown_connections 2023-10-09T06:30:37.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.289+0000 7f1a9ede5700 1 -- 172.21.15.163:0/329656913 wait complete. 2023-10-09T06:30:37.301 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:30:37.416 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.415+0000 7fecca6fb700 1 Processor -- start 2023-10-09T06:30:37.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.416+0000 7fecca6fb700 1 -- start start 2023-10-09T06:30:37.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.416+0000 7fecca6fb700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fecc406c0a0 0x7fecc40caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:37.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.416+0000 7fecca6fb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fecc415df50 0x7fecc415e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:37.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.416+0000 7fecca6fb700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecc415d7e0 con 0x7fecc406bcd0 2023-10-09T06:30:37.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.416+0000 7fecca6fb700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecc415dac0 con 0x7fecc406c0a0 2023-10-09T06:30:37.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.416+0000 7fecca6fb700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecc415aa30 con 0x7fecc415df50 2023-10-09T06:30:37.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.416+0000 7fecc37fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fecc406c0a0 0x7fecc40caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:37.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.416+0000 7fecc37fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fecc406c0a0 0x7fecc40caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47700/0 (socket says 172.21.15.163:47700) 2023-10-09T06:30:37.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.417+0000 7fecc37fe700 1 -- 172.21.15.163:0/4006019317 learned_addr learned my addr 172.21.15.163:0/4006019317 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:37.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.417+0000 7fecc8c98700 1 --2- 172.21.15.163:0/4006019317 >> v2:172.21.15.181:3300/0 conn(0x7fecc415df50 0x7fecc415e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:37.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.417+0000 7fecc37fe700 1 -- 172.21.15.163:0/4006019317 >> v2:172.21.15.181:3300/0 conn(0x7fecc415df50 msgr2=0x7fecc415e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:37.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.417+0000 7fecc37fe700 1 --2- 172.21.15.163:0/4006019317 >> v2:172.21.15.181:3300/0 conn(0x7fecc415df50 0x7fecc415e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:37.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.417+0000 7fecc37fe700 1 -- 172.21.15.163:0/4006019317 >> v1:172.21.15.181:6789/0 conn(0x7fecc406bcd0 legacy=0x7fecc415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:37.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.417+0000 7fecc37fe700 1 -- 172.21.15.163:0/4006019317 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc415ad10 con 0x7fecc406c0a0 2023-10-09T06:30:37.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.417+0000 7fecc37fe700 1 --2- 172.21.15.163:0/4006019317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecc406c0a0 0x7fecc40caa90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fecc415d720 tx=0x7fecb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727f6703ca8c90d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:37.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.417+0000 7fecc2ffd700 1 -- 172.21.15.163:0/4006019317 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb800b860 con 0x7fecc406c0a0 2023-10-09T06:30:37.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.417+0000 7fecc2ffd700 1 -- 172.21.15.163:0/4006019317 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb800b9c0 con 0x7fecc406c0a0 2023-10-09T06:30:37.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.417+0000 7fecc2ffd700 1 -- 172.21.15.163:0/4006019317 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb800bd80 con 0x7fecc406c0a0 2023-10-09T06:30:37.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.418+0000 7fecca6fb700 1 -- 172.21.15.163:0/4006019317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecc406c0a0 msgr2=0x7fecc40caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:37.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.418+0000 7fecca6fb700 1 --2- 172.21.15.163:0/4006019317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecc406c0a0 0x7fecc40caa90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fecc415d720 tx=0x7fecb8009250 comp rx=0 tx=0).stop 2023-10-09T06:30:37.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.418+0000 7fecca6fb700 1 -- 172.21.15.163:0/4006019317 shutdown_connections 2023-10-09T06:30:37.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.418+0000 7fecca6fb700 1 --2- 172.21.15.163:0/4006019317 >> v2:172.21.15.181:3300/0 conn(0x7fecc415df50 0x7fecc415e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:37.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.418+0000 7fecca6fb700 1 --2- 172.21.15.163:0/4006019317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecc406c0a0 0x7fecc40caa90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:37.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.418+0000 7fecca6fb700 1 -- 172.21.15.163:0/4006019317 >> 172.21.15.163:0/4006019317 conn(0x7fecc40ce600 msgr2=0x7fecc40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:37.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.418+0000 7fecca6fb700 1 -- 172.21.15.163:0/4006019317 shutdown_connections 2023-10-09T06:30:37.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.418+0000 7fecca6fb700 1 -- 172.21.15.163:0/4006019317 wait complete. 2023-10-09T06:30:37.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.419+0000 7fecca6fb700 1 Processor -- start 2023-10-09T06:30:37.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.419+0000 7fecca6fb700 1 -- start start 2023-10-09T06:30:37.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.420+0000 7fecca6fb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecc406bcd0 0x7fecc4157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:37.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.420+0000 7fecca6fb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecc415df50 0x7fecc40b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:37.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.420+0000 7fecca6fb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecc40b3c00 0x7fecc415b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:37.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.420+0000 7fecca6fb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc416a880 con 0x7fecc40b3c00 2023-10-09T06:30:37.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.420+0000 7fecca6fb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc40b4180 con 0x7fecc415df50 2023-10-09T06:30:37.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.420+0000 7fecca6fb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc416a230 con 0x7fecc406bcd0 2023-10-09T06:30:37.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.420+0000 7fecc8c98700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecc40b3c00 0x7fecc415b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:37.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.420+0000 7fecc8c98700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecc40b3c00 0x7fecc415b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47706/0 (socket says 172.21.15.163:47706) 2023-10-09T06:30:37.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.420+0000 7fecc8c98700 1 -- 172.21.15.163:0/3118306663 learned_addr learned my addr 172.21.15.163:0/3118306663 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:37.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.420+0000 7fecc37fe700 1 --2- 172.21.15.163:0/3118306663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecc415df50 0x7fecc40b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:37.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.420+0000 7fecc3fff700 1 --2- 172.21.15.163:0/3118306663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecc406bcd0 0x7fecc4157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:37.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.420+0000 7fecc8c98700 1 -- 172.21.15.163:0/3118306663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecc406bcd0 msgr2=0x7fecc4157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:37.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.420+0000 7fecc8c98700 1 --2- 172.21.15.163:0/3118306663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecc406bcd0 0x7fecc4157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:37.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.420+0000 7fecc8c98700 1 -- 172.21.15.163:0/3118306663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecc415df50 msgr2=0x7fecc40b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:37.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.420+0000 7fecc8c98700 1 --2- 172.21.15.163:0/3118306663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecc415df50 0x7fecc40b36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:37.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.420+0000 7fecc8c98700 1 -- 172.21.15.163:0/3118306663 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb80094f0 con 0x7fecc40b3c00 2023-10-09T06:30:37.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.420+0000 7fecc8c98700 1 --2- 172.21.15.163:0/3118306663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecc40b3c00 0x7fecc415b360 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fecbc007f20 tx=0x7fecbc009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:37.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.420+0000 7fecabfff700 1 -- 172.21.15.163:0/3118306663 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecbc00eb90 con 0x7fecc40b3c00 2023-10-09T06:30:37.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.420+0000 7fecabfff700 1 -- 172.21.15.163:0/3118306663 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecbc00ed10 con 0x7fecc40b3c00 2023-10-09T06:30:37.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.420+0000 7fecca6fb700 1 -- 172.21.15.163:0/3118306663 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecc416a510 con 0x7fecc40b3c00 2023-10-09T06:30:37.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.420+0000 7fecabfff700 1 -- 172.21.15.163:0/3118306663 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecbc014670 con 0x7fecc40b3c00 2023-10-09T06:30:37.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.420+0000 7fecca6fb700 1 -- 172.21.15.163:0/3118306663 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecc416e3b0 con 0x7fecc40b3c00 2023-10-09T06:30:37.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.421+0000 7fecca6fb700 1 -- 172.21.15.163:0/3118306663 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecc416a9f0 con 0x7fecc40b3c00 2023-10-09T06:30:37.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.422+0000 7fecabfff700 1 -- 172.21.15.163:0/3118306663 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fecbc0149e0 con 0x7fecc40b3c00 2023-10-09T06:30:37.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.422+0000 7fecabfff700 1 --2- 172.21.15.163:0/3118306663 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fecb0066f30 0x7fecb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:37.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.422+0000 7fecabfff700 1 -- 172.21.15.163:0/3118306663 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fecbc084760 con 0x7fecc40b3c00 2023-10-09T06:30:37.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.422+0000 7fecc3fff700 1 --2- 172.21.15.163:0/3118306663 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fecb0066f30 0x7fecb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:37.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.423+0000 7fecc3fff700 1 --2- 172.21.15.163:0/3118306663 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fecb0066f30 0x7fecb00693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fecb40039e0 tx=0x7fecb4007ec0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:37.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.426+0000 7fecabfff700 1 -- 172.21.15.163:0/3118306663 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fecbc051d20 con 0x7fecc40b3c00 2023-10-09T06:30:37.651 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.650+0000 7fecca6fb700 1 -- 172.21.15.163:0/3118306663 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fecc404c3c0 con 0x7fecc40b3c00 2023-10-09T06:30:37.651 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.650+0000 7fecabfff700 1 -- 172.21.15.163:0/3118306663 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1259 (secure 0 0 0) 0x7fecbc055340 con 0x7fecc40b3c00 2023-10-09T06:30:37.652 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:30:37.652 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2270076605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2270076605},{"type":"v1","addr":"172.21.15.163:6834","nonce":2270076605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-10-09T06:30:37.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.653+0000 7fecca6fb700 1 -- 172.21.15.163:0/3118306663 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fecb0066f30 msgr2=0x7fecb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:37.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.653+0000 7fecca6fb700 1 --2- 172.21.15.163:0/3118306663 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fecb0066f30 0x7fecb00693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fecb40039e0 tx=0x7fecb4007ec0 comp rx=0 tx=0).stop 2023-10-09T06:30:37.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.653+0000 7fecca6fb700 1 -- 172.21.15.163:0/3118306663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecc40b3c00 msgr2=0x7fecc415b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:37.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.653+0000 7fecca6fb700 1 --2- 172.21.15.163:0/3118306663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecc40b3c00 0x7fecc415b360 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fecbc007f20 tx=0x7fecbc009860 comp rx=0 tx=0).stop 2023-10-09T06:30:37.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.653+0000 7fecca6fb700 1 -- 172.21.15.163:0/3118306663 shutdown_connections 2023-10-09T06:30:37.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.653+0000 7fecca6fb700 1 --2- 172.21.15.163:0/3118306663 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fecb0066f30 0x7fecb00693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:37.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.653+0000 7fecca6fb700 1 --2- 172.21.15.163:0/3118306663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecc40b3c00 0x7fecc415b360 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:37.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.653+0000 7fecca6fb700 1 --2- 172.21.15.163:0/3118306663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecc406bcd0 0x7fecc4157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:37.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.653+0000 7fecca6fb700 1 --2- 172.21.15.163:0/3118306663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecc415df50 0x7fecc40b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:37.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.653+0000 7fecca6fb700 1 -- 172.21.15.163:0/3118306663 >> 172.21.15.163:0/3118306663 conn(0x7fecc40ce600 msgr2=0x7fecc406cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:37.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.653+0000 7fecca6fb700 1 -- 172.21.15.163:0/3118306663 shutdown_connections 2023-10-09T06:30:37.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:37.653+0000 7fecca6fb700 1 -- 172.21.15.163:0/3118306663 wait complete. 2023-10-09T06:30:37.655 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 2 2023-10-09T06:30:39.666 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:30:39.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.792+0000 7f0058316700 1 Processor -- start 2023-10-09T06:30:39.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.792+0000 7f0058316700 1 -- start start 2023-10-09T06:30:39.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.793+0000 7f0058316700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f005006bcd0 0x7f0050140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:39.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.793+0000 7f0058316700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00500c9510 0x7f0050141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:39.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.793+0000 7f0058316700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f005015c2a0 con 0x7f005006c0a0 2023-10-09T06:30:39.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.793+0000 7f0058316700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f005015b690 con 0x7f005006bcd0 2023-10-09T06:30:39.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.793+0000 7f0058316700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f005015b970 con 0x7f00500c9510 2023-10-09T06:30:39.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.793+0000 7f00558b1700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f005006c0a0 0x7f00500c8e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:52346/0 (socket says 172.21.15.163:52346) 2023-10-09T06:30:39.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.793+0000 7f00558b1700 1 -- 172.21.15.163:0/3865077206 learned_addr learned my addr 172.21.15.163:0/3865077206 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:39.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.794+0000 7f00568b3700 1 --2- 172.21.15.163:0/3865077206 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00500c9510 0x7f0050141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:39.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.794+0000 7f00560b2700 1 --2- 172.21.15.163:0/3865077206 >> v2:172.21.15.181:3300/0 conn(0x7f005006bcd0 0x7f0050140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:39.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.794+0000 7f00550b0700 1 -- 172.21.15.163:0/3865077206 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3782684296 0 0) 0x7f005015c2a0 con 0x7f005006c0a0 2023-10-09T06:30:39.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.794+0000 7f00550b0700 1 -- 172.21.15.163:0/3865077206 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f003c003570 con 0x7f005006c0a0 2023-10-09T06:30:39.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.794+0000 7f00550b0700 1 -- 172.21.15.163:0/3865077206 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3273248961 0 0) 0x7f003c003570 con 0x7f005006c0a0 2023-10-09T06:30:39.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.794+0000 7f00550b0700 1 -- 172.21.15.163:0/3865077206 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00500c9510 msgr2=0x7f0050141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:39.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.794+0000 7f00550b0700 1 --2- 172.21.15.163:0/3865077206 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00500c9510 0x7f0050141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:39.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.794+0000 7f00550b0700 1 -- 172.21.15.163:0/3865077206 >> v2:172.21.15.181:3300/0 conn(0x7f005006bcd0 msgr2=0x7f0050140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:39.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.794+0000 7f00550b0700 1 --2- 172.21.15.163:0/3865077206 >> v2:172.21.15.181:3300/0 conn(0x7f005006bcd0 0x7f0050140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:39.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.794+0000 7f00550b0700 1 -- 172.21.15.163:0/3865077206 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f005015bc50 con 0x7f005006c0a0 2023-10-09T06:30:39.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.794+0000 7f00550b0700 1 -- 172.21.15.163:0/3865077206 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f0040001470 con 0x7f005006c0a0 2023-10-09T06:30:39.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.794+0000 7f00550b0700 1 -- 172.21.15.163:0/3865077206 >> v1:172.21.15.163:6789/0 conn(0x7f005006c0a0 legacy=0x7f00500c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:39.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.794+0000 7f00550b0700 1 --2- 172.21.15.163:0/3865077206 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00500c9510 0x7f003c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:39.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.794+0000 7f00550b0700 1 --2- 172.21.15.163:0/3865077206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f003c003fe0 0x7f003c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:39.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.794+0000 7f00550b0700 1 --2- 172.21.15.163:0/3865077206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f003c004d50 0x7f003c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:39.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.794+0000 7f00550b0700 1 -- 172.21.15.163:0/3865077206 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005015b690 con 0x7f003c003fe0 2023-10-09T06:30:39.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.794+0000 7f00550b0700 1 -- 172.21.15.163:0/3865077206 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005015b970 con 0x7f003c004d50 2023-10-09T06:30:39.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.794+0000 7f00550b0700 1 -- 172.21.15.163:0/3865077206 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f003c00ba90 con 0x7f00500c9510 2023-10-09T06:30:39.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.794+0000 7f00558b1700 1 --2- 172.21.15.163:0/3865077206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f003c003fe0 0x7f003c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:39.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.794+0000 7f00560b2700 1 --2- 172.21.15.163:0/3865077206 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00500c9510 0x7f003c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:39.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.794+0000 7f00568b3700 1 --2- 172.21.15.163:0/3865077206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f003c004d50 0x7f003c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:39.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.794+0000 7f00558b1700 1 -- 172.21.15.163:0/3865077206 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00500c9510 msgr2=0x7f003c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:39.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.794+0000 7f00558b1700 1 --2- 172.21.15.163:0/3865077206 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00500c9510 0x7f003c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:39.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.794+0000 7f00558b1700 1 -- 172.21.15.163:0/3865077206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f003c004d50 msgr2=0x7f003c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:39.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.794+0000 7f00558b1700 1 --2- 172.21.15.163:0/3865077206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f003c004d50 0x7f003c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:39.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.794+0000 7f00558b1700 1 -- 172.21.15.163:0/3865077206 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003c003570 con 0x7f003c003fe0 2023-10-09T06:30:39.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.795+0000 7f00558b1700 1 --2- 172.21.15.163:0/3865077206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f003c003fe0 0x7f003c004810 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f004000c3c0 tx=0x7f005006e470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510c838bd4f00434 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:39.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.795+0000 7f00550b0700 1 -- 172.21.15.163:0/3865077206 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004000cc30 con 0x7f003c003fe0 2023-10-09T06:30:39.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.795+0000 7f00550b0700 1 -- 172.21.15.163:0/3865077206 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f004000cdd0 con 0x7f003c003fe0 2023-10-09T06:30:39.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.795+0000 7f00550b0700 1 -- 172.21.15.163:0/3865077206 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0040005b70 con 0x7f003c003fe0 2023-10-09T06:30:39.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.795+0000 7f0058316700 1 -- 172.21.15.163:0/3865077206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f003c003fe0 msgr2=0x7f003c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:39.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.795+0000 7f0058316700 1 --2- 172.21.15.163:0/3865077206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f003c003fe0 0x7f003c004810 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f004000c3c0 tx=0x7f005006e470 comp rx=0 tx=0).stop 2023-10-09T06:30:39.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.795+0000 7f00568b3700 1 -- 172.21.15.163:0/3865077206 reap_dead start 2023-10-09T06:30:39.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.795+0000 7f0058316700 1 -- 172.21.15.163:0/3865077206 shutdown_connections 2023-10-09T06:30:39.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.795+0000 7f0058316700 1 -- 172.21.15.163:0/3865077206 >> 172.21.15.163:0/3865077206 conn(0x7f00500ce600 msgr2=0x7f00500cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:39.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.795+0000 7f0058316700 1 -- 172.21.15.163:0/3865077206 shutdown_connections 2023-10-09T06:30:39.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.795+0000 7f0058316700 1 -- 172.21.15.163:0/3865077206 wait complete. 2023-10-09T06:30:39.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.796+0000 7f0058316700 1 Processor -- start 2023-10-09T06:30:39.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.796+0000 7f0058316700 1 -- start start 2023-10-09T06:30:39.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.797+0000 7f0058316700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f003c003fe0 0x7f00500c8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:39.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.797+0000 7f0058316700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00500c77f0 0x7f00500a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:39.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.797+0000 7f0058316700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f005013e960 0x7f0050167f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:39.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.797+0000 7f0058316700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00500c7360 con 0x7f003c003fe0 2023-10-09T06:30:39.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.797+0000 7f0058316700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00500c6750 con 0x7f00500c77f0 2023-10-09T06:30:39.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.797+0000 7f0058316700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00500c6a30 con 0x7f005013e960 2023-10-09T06:30:39.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.797+0000 7f00560b2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f003c003fe0 0x7f00500c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:39.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.797+0000 7f00560b2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f003c003fe0 0x7f00500c8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36608/0 (socket says 172.21.15.163:36608) 2023-10-09T06:30:39.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.797+0000 7f00560b2700 1 -- 172.21.15.163:0/2751984934 learned_addr learned my addr 172.21.15.163:0/2751984934 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:39.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.797+0000 7f00568b3700 1 --2- 172.21.15.163:0/2751984934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f005013e960 0x7f0050167f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:39.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.797+0000 7f00558b1700 1 --2- 172.21.15.163:0/2751984934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00500c77f0 0x7f00500a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:39.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.798+0000 7f00560b2700 1 -- 172.21.15.163:0/2751984934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f005013e960 msgr2=0x7f0050167f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:39.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.798+0000 7f00560b2700 1 --2- 172.21.15.163:0/2751984934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f005013e960 0x7f0050167f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:39.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.798+0000 7f00560b2700 1 -- 172.21.15.163:0/2751984934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00500c77f0 msgr2=0x7f00500a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:39.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.798+0000 7f00560b2700 1 --2- 172.21.15.163:0/2751984934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00500c77f0 0x7f00500a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:39.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.798+0000 7f00560b2700 1 -- 172.21.15.163:0/2751984934 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00500c6d10 con 0x7f003c003fe0 2023-10-09T06:30:39.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.798+0000 7f00558b1700 1 --2- 172.21.15.163:0/2751984934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00500c77f0 0x7f00500a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:30:39.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.798+0000 7f00560b2700 1 --2- 172.21.15.163:0/2751984934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f003c003fe0 0x7f00500c8610 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f004c011f80 tx=0x7f004c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:39.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.798+0000 7f00467fc700 1 -- 172.21.15.163:0/2751984934 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c00e040 con 0x7f003c003fe0 2023-10-09T06:30:39.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.798+0000 7f00467fc700 1 -- 172.21.15.163:0/2751984934 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f004c00f040 con 0x7f003c003fe0 2023-10-09T06:30:39.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.798+0000 7f00467fc700 1 -- 172.21.15.163:0/2751984934 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c015430 con 0x7f003c003fe0 2023-10-09T06:30:39.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.798+0000 7f0058316700 1 -- 172.21.15.163:0/2751984934 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00500c6ff0 con 0x7f003c003fe0 2023-10-09T06:30:39.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.798+0000 7f0058316700 1 -- 172.21.15.163:0/2751984934 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0050168440 con 0x7f003c003fe0 2023-10-09T06:30:39.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.799+0000 7f0058316700 1 -- 172.21.15.163:0/2751984934 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f005004c3c0 con 0x7f003c003fe0 2023-10-09T06:30:39.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.799+0000 7f00467fc700 1 -- 172.21.15.163:0/2751984934 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f004c015820 con 0x7f003c003fe0 2023-10-09T06:30:39.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.800+0000 7f00467fc700 1 --2- 172.21.15.163:0/2751984934 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0034066f30 0x7f00340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:39.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.800+0000 7f00467fc700 1 -- 172.21.15.163:0/2751984934 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f004c084e40 con 0x7f003c003fe0 2023-10-09T06:30:39.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.800+0000 7f00558b1700 1 --2- 172.21.15.163:0/2751984934 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0034066f30 0x7f00340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:39.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.800+0000 7f00558b1700 1 --2- 172.21.15.163:0/2751984934 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0034066f30 0x7f00340693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f004000c3c0 tx=0x7f0040015000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:39.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:39.804+0000 7f00467fc700 1 -- 172.21.15.163:0/2751984934 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f004c052460 con 0x7f003c003fe0 2023-10-09T06:30:40.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.028+0000 7f0058316700 1 -- 172.21.15.163:0/2751984934 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f005006c9d0 con 0x7f003c003fe0 2023-10-09T06:30:40.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.029+0000 7f00467fc700 1 -- 172.21.15.163:0/2751984934 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3536 (secure 0 0 0) 0x7f004c055a80 con 0x7f003c003fe0 2023-10-09T06:30:40.030 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:30:40.030 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3440080308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3440080308},{"type":"v1","addr":"172.21.15.181:6837","nonce":3440080308}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1990962358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1990962358},{"type":"v1","addr":"172.21.15.163:6835","nonce":1990962358}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2270076605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2270076605},{"type":"v1","addr":"172.21.15.163:6834","nonce":2270076605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1149296546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1149296546},{"type":"v1","addr":"172.21.15.181:6836","nonce":1149296546}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-10-09T06:30:40.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.031+0000 7f0058316700 1 -- 172.21.15.163:0/2751984934 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0034066f30 msgr2=0x7f00340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:40.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.031+0000 7f0058316700 1 --2- 172.21.15.163:0/2751984934 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0034066f30 0x7f00340693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f004000c3c0 tx=0x7f0040015000 comp rx=0 tx=0).stop 2023-10-09T06:30:40.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.031+0000 7f0058316700 1 -- 172.21.15.163:0/2751984934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f003c003fe0 msgr2=0x7f00500c8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:40.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.031+0000 7f0058316700 1 --2- 172.21.15.163:0/2751984934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f003c003fe0 0x7f00500c8610 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f004c011f80 tx=0x7f004c00b040 comp rx=0 tx=0).stop 2023-10-09T06:30:40.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.031+0000 7f0058316700 1 -- 172.21.15.163:0/2751984934 shutdown_connections 2023-10-09T06:30:40.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.031+0000 7f0058316700 1 --2- 172.21.15.163:0/2751984934 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0034066f30 0x7f00340693e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:40.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.031+0000 7f0058316700 1 --2- 172.21.15.163:0/2751984934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f003c003fe0 0x7f00500c8610 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:40.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.031+0000 7f0058316700 1 --2- 172.21.15.163:0/2751984934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f005013e960 0x7f0050167f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:40.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.031+0000 7f0058316700 1 --2- 172.21.15.163:0/2751984934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00500c77f0 0x7f00500a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:40.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.031+0000 7f0058316700 1 -- 172.21.15.163:0/2751984934 >> 172.21.15.163:0/2751984934 conn(0x7f00500ce600 msgr2=0x7f00500cad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:40.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.031+0000 7f0058316700 1 -- 172.21.15.163:0/2751984934 shutdown_connections 2023-10-09T06:30:40.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.031+0000 7f0058316700 1 -- 172.21.15.163:0/2751984934 wait complete. 2023-10-09T06:30:40.034 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 3 2023-10-09T06:30:40.044 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:30:40.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.157+0000 7f18eb140700 1 Processor -- start 2023-10-09T06:30:40.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.157+0000 7f18eb140700 1 -- start start 2023-10-09T06:30:40.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.158+0000 7f18eb140700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f18e406bcd0 0x7f18e40caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:40.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.158+0000 7f18eb140700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18e415d7e0 con 0x7f18e415df50 2023-10-09T06:30:40.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.158+0000 7f18eb140700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18e415dac0 con 0x7f18e406c0a0 2023-10-09T06:30:40.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.158+0000 7f18eb140700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e415aa30 con 0x7f18e406bcd0 2023-10-09T06:30:40.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.158+0000 7f18e96dd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f18e415df50 0x7f18e40c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:52362/0 (socket says 172.21.15.163:52362) 2023-10-09T06:30:40.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.158+0000 7f18e96dd700 1 -- 172.21.15.163:0/546300534 learned_addr learned my addr 172.21.15.163:0/546300534 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:40.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.158+0000 7f18e8edc700 1 --2- 172.21.15.163:0/546300534 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f18e406bcd0 0x7f18e40caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:40.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.158+0000 7f18e37fe700 1 -- 172.21.15.163:0/546300534 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 198615554 0 0) 0x7f18e415d7e0 con 0x7f18e415df50 2023-10-09T06:30:40.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.158+0000 7f18e37fe700 1 -- 172.21.15.163:0/546300534 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18d0003570 con 0x7f18e415df50 2023-10-09T06:30:40.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.158+0000 7f18e37fe700 1 -- 172.21.15.163:0/546300534 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1073151829 0 0) 0x7f18e415dac0 con 0x7f18e406c0a0 2023-10-09T06:30:40.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.158+0000 7f18e37fe700 1 -- 172.21.15.163:0/546300534 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18e415d7e0 con 0x7f18e406c0a0 2023-10-09T06:30:40.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.158+0000 7f18e8edc700 1 -- 172.21.15.163:0/546300534 >> v1:172.21.15.181:6789/0 conn(0x7f18e406c0a0 legacy=0x7f18e415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:40.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.158+0000 7f18e37fe700 1 -- 172.21.15.163:0/546300534 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1274903203 0 0) 0x7f18d0003570 con 0x7f18e415df50 2023-10-09T06:30:40.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.158+0000 7f18e8edc700 1 -- 172.21.15.163:0/546300534 >> v1:172.21.15.163:6789/0 conn(0x7f18e415df50 legacy=0x7f18e40c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:40.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.158+0000 7f18e8edc700 1 -- 172.21.15.163:0/546300534 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e415ad10 con 0x7f18e406bcd0 2023-10-09T06:30:40.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.159+0000 7f18e8edc700 1 --2- 172.21.15.163:0/546300534 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f18e406bcd0 0x7f18e40caa90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009910 tx=0x7f18dc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=216875e49c86a71e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:40.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.159+0000 7f18e37fe700 1 -- 172.21.15.163:0/546300534 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18cc001470 con 0x7f18e406bcd0 2023-10-09T06:30:40.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.159+0000 7f18e37fe700 1 -- 172.21.15.163:0/546300534 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18dc011420 con 0x7f18e406bcd0 2023-10-09T06:30:40.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.159+0000 7f18e37fe700 1 -- 172.21.15.163:0/546300534 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc0117a0 con 0x7f18e406bcd0 2023-10-09T06:30:40.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.159+0000 7f18eb140700 1 -- 172.21.15.163:0/546300534 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f18e406bcd0 msgr2=0x7f18e40caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:40.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.159+0000 7f18eb140700 1 --2- 172.21.15.163:0/546300534 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f18e406bcd0 0x7f18e40caa90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009910 tx=0x7f18dc0091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:40.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.159+0000 7f18eb140700 1 -- 172.21.15.163:0/546300534 shutdown_connections 2023-10-09T06:30:40.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.159+0000 7f18eb140700 1 --2- 172.21.15.163:0/546300534 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f18e406bcd0 0x7f18e40caa90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:40.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.159+0000 7f18eb140700 1 -- 172.21.15.163:0/546300534 >> 172.21.15.163:0/546300534 conn(0x7f18e40ce600 msgr2=0x7f18e40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:40.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.159+0000 7f18eb140700 1 -- 172.21.15.163:0/546300534 shutdown_connections 2023-10-09T06:30:40.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.159+0000 7f18eb140700 1 -- 172.21.15.163:0/546300534 wait complete. 2023-10-09T06:30:40.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.160+0000 7f18eb140700 1 Processor -- start 2023-10-09T06:30:40.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.161+0000 7f18eb140700 1 -- start start 2023-10-09T06:30:40.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.161+0000 7f18eb140700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f18e406bcd0 0x7f18e40b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:40.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.161+0000 7f18eb140700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18e406c0a0 0x7f18e41532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:40.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.161+0000 7f18eb140700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18e415df50 0x7f18e4153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:40.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.161+0000 7f18eb140700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e4154a40 con 0x7f18e415df50 2023-10-09T06:30:40.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.161+0000 7f18eb140700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e4158250 con 0x7f18e406c0a0 2023-10-09T06:30:40.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.161+0000 7f18eb140700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e40b22d0 con 0x7f18e406bcd0 2023-10-09T06:30:40.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.161+0000 7f18e96dd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18e415df50 0x7f18e4153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:40.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.161+0000 7f18e96dd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18e415df50 0x7f18e4153800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36624/0 (socket says 172.21.15.163:36624) 2023-10-09T06:30:40.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.161+0000 7f18e96dd700 1 -- 172.21.15.163:0/494698834 learned_addr learned my addr 172.21.15.163:0/494698834 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:40.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.162+0000 7f18e8edc700 1 --2- 172.21.15.163:0/494698834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f18e406bcd0 0x7f18e40b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:40.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.162+0000 7f18e3fff700 1 --2- 172.21.15.163:0/494698834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18e406c0a0 0x7f18e41532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:40.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.162+0000 7f18e96dd700 1 -- 172.21.15.163:0/494698834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f18e406bcd0 msgr2=0x7f18e40b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:40.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.162+0000 7f18e96dd700 1 --2- 172.21.15.163:0/494698834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f18e406bcd0 0x7f18e40b1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:40.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.162+0000 7f18e96dd700 1 -- 172.21.15.163:0/494698834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18e406c0a0 msgr2=0x7f18e41532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:40.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.162+0000 7f18e96dd700 1 --2- 172.21.15.163:0/494698834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18e406c0a0 0x7f18e41532c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:40.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.162+0000 7f18e96dd700 1 -- 172.21.15.163:0/494698834 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc009420 con 0x7f18e415df50 2023-10-09T06:30:40.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.162+0000 7f18e3fff700 1 --2- 172.21.15.163:0/494698834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18e406c0a0 0x7f18e41532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:30:40.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.162+0000 7f18e96dd700 1 --2- 172.21.15.163:0/494698834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18e415df50 0x7f18e4153800 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00cb70 tx=0x7f18cc001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:40.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.162+0000 7f18e0ff9700 1 -- 172.21.15.163:0/494698834 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18cc007150 con 0x7f18e415df50 2023-10-09T06:30:40.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.162+0000 7f18e0ff9700 1 -- 172.21.15.163:0/494698834 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18cc0072b0 con 0x7f18e415df50 2023-10-09T06:30:40.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.162+0000 7f18eb140700 1 -- 172.21.15.163:0/494698834 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e4153e30 con 0x7f18e415df50 2023-10-09T06:30:40.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.162+0000 7f18e0ff9700 1 -- 172.21.15.163:0/494698834 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18cc007670 con 0x7f18e415df50 2023-10-09T06:30:40.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.162+0000 7f18eb140700 1 -- 172.21.15.163:0/494698834 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e41543f0 con 0x7f18e415df50 2023-10-09T06:30:40.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.163+0000 7f18eb140700 1 -- 172.21.15.163:0/494698834 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18e4154bb0 con 0x7f18e415df50 2023-10-09T06:30:40.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.163+0000 7f18e0ff9700 1 -- 172.21.15.163:0/494698834 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f18cc004a40 con 0x7f18e415df50 2023-10-09T06:30:40.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.164+0000 7f18e0ff9700 1 --2- 172.21.15.163:0/494698834 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f18c4066f30 0x7f18c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:40.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.164+0000 7f18e0ff9700 1 -- 172.21.15.163:0/494698834 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f18cc083480 con 0x7f18e415df50 2023-10-09T06:30:40.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.164+0000 7f18e8edc700 1 --2- 172.21.15.163:0/494698834 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f18c4066f30 0x7f18c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:40.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.165+0000 7f18e8edc700 1 --2- 172.21.15.163:0/494698834 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f18c4066f30 0x7f18c40693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f18dc003080 tx=0x7f18dc016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:40.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.168+0000 7f18e0ff9700 1 -- 172.21.15.163:0/494698834 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18cc050ac0 con 0x7f18e415df50 2023-10-09T06:30:40.355 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.354+0000 7f18eb140700 1 -- 172.21.15.163:0/494698834 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18e404c3f0 con 0x7f18e415df50 2023-10-09T06:30:40.356 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.355+0000 7f18e0ff9700 1 -- 172.21.15.163:0/494698834 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12175 (secure 0 0 0) 0x7f18cc018090 con 0x7f18e415df50 2023-10-09T06:30:40.356 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:30:40.356 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":16,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:30:32.782320+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:30:40.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.357+0000 7f18eb140700 1 -- 172.21.15.163:0/494698834 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f18c4066f30 msgr2=0x7f18c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:40.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.357+0000 7f18eb140700 1 --2- 172.21.15.163:0/494698834 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f18c4066f30 0x7f18c40693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f18dc003080 tx=0x7f18dc016000 comp rx=0 tx=0).stop 2023-10-09T06:30:40.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.357+0000 7f18eb140700 1 -- 172.21.15.163:0/494698834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18e415df50 msgr2=0x7f18e4153800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:40.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.357+0000 7f18eb140700 1 --2- 172.21.15.163:0/494698834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18e415df50 0x7f18e4153800 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00cb70 tx=0x7f18cc001a20 comp rx=0 tx=0).stop 2023-10-09T06:30:40.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.357+0000 7f18eb140700 1 -- 172.21.15.163:0/494698834 shutdown_connections 2023-10-09T06:30:40.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.357+0000 7f18eb140700 1 --2- 172.21.15.163:0/494698834 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f18c4066f30 0x7f18c40693e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:40.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.357+0000 7f18eb140700 1 --2- 172.21.15.163:0/494698834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18e415df50 0x7f18e4153800 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:40.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.357+0000 7f18eb140700 1 --2- 172.21.15.163:0/494698834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f18e406bcd0 0x7f18e40b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:40.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.357+0000 7f18eb140700 1 --2- 172.21.15.163:0/494698834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18e406c0a0 0x7f18e41532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:40.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.357+0000 7f18eb140700 1 -- 172.21.15.163:0/494698834 >> 172.21.15.163:0/494698834 conn(0x7f18e40ce600 msgr2=0x7f18e40c8810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:40.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.357+0000 7f18eb140700 1 -- 172.21.15.163:0/494698834 shutdown_connections 2023-10-09T06:30:40.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.357+0000 7f18eb140700 1 -- 172.21.15.163:0/494698834 wait complete. 2023-10-09T06:30:40.373 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-09T06:30:40.374 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:30:40.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.491+0000 7fa818926700 1 Processor -- start 2023-10-09T06:30:40.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.491+0000 7fa818926700 1 -- start start 2023-10-09T06:30:40.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.492+0000 7fa818926700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa814157680 0x7fa8140cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:40.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.492+0000 7fa818926700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa814157a50 0x7fa8140cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:40.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.492+0000 7fa818926700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8140b1450 0x7fa8140c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:40.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.492+0000 7fa818926700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8140cc700 con 0x7fa814157680 2023-10-09T06:30:40.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.492+0000 7fa818926700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8140cb900 con 0x7fa814157a50 2023-10-09T06:30:40.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.492+0000 7fa818926700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8140cd240 con 0x7fa8140b1450 2023-10-09T06:30:40.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.492+0000 7fa812d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa814157680 0x7fa8140cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:40.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.492+0000 7fa812d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa814157680 0x7fa8140cae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36626/0 (socket says 172.21.15.163:36626) 2023-10-09T06:30:40.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.492+0000 7fa812d9d700 1 -- 172.21.15.163:0/3819730578 learned_addr learned my addr 172.21.15.163:0/3819730578 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:40.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.492+0000 7fa81359e700 1 --2- 172.21.15.163:0/3819730578 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8140b1450 0x7fa8140c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:40.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.492+0000 7fa812d9d700 1 -- 172.21.15.163:0/3819730578 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8140b1450 msgr2=0x7fa8140c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:40.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.492+0000 7fa812d9d700 1 --2- 172.21.15.163:0/3819730578 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8140b1450 0x7fa8140c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:40.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.492+0000 7fa812d9d700 1 -- 172.21.15.163:0/3819730578 >> v2:172.21.15.181:3300/0 conn(0x7fa814157a50 msgr2=0x7fa8140cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:40.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.492+0000 7fa81259c700 1 --2- 172.21.15.163:0/3819730578 >> v2:172.21.15.181:3300/0 conn(0x7fa814157a50 0x7fa8140cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:40.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.492+0000 7fa812d9d700 1 --2- 172.21.15.163:0/3819730578 >> v2:172.21.15.181:3300/0 conn(0x7fa814157a50 0x7fa8140cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:40.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.492+0000 7fa812d9d700 1 -- 172.21.15.163:0/3819730578 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8140c9a20 con 0x7fa814157680 2023-10-09T06:30:40.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.492+0000 7fa81359e700 1 --2- 172.21.15.163:0/3819730578 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8140b1450 0x7fa8140c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:30:40.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.492+0000 7fa81259c700 1 --2- 172.21.15.163:0/3819730578 >> v2:172.21.15.181:3300/0 conn(0x7fa814157a50 0x7fa8140cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:30:40.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.492+0000 7fa812d9d700 1 --2- 172.21.15.163:0/3819730578 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa814157680 0x7fa8140cae80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa8140cbc70 tx=0x7fa804009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=720360b2af77084a server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:40.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.493+0000 7fa811d9b700 1 -- 172.21.15.163:0/3819730578 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80400e070 con 0x7fa814157680 2023-10-09T06:30:40.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.493+0000 7fa811d9b700 1 -- 172.21.15.163:0/3819730578 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa80400ba50 con 0x7fa814157680 2023-10-09T06:30:40.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.493+0000 7fa811d9b700 1 -- 172.21.15.163:0/3819730578 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8040092e0 con 0x7fa814157680 2023-10-09T06:30:40.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.493+0000 7fa818926700 1 -- 172.21.15.163:0/3819730578 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa814157680 msgr2=0x7fa8140cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:40.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.493+0000 7fa818926700 1 --2- 172.21.15.163:0/3819730578 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa814157680 0x7fa8140cae80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa8140cbc70 tx=0x7fa804009840 comp rx=0 tx=0).stop 2023-10-09T06:30:40.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.493+0000 7fa818926700 1 -- 172.21.15.163:0/3819730578 shutdown_connections 2023-10-09T06:30:40.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.493+0000 7fa818926700 1 --2- 172.21.15.163:0/3819730578 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8140b1450 0x7fa8140c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:40.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.493+0000 7fa818926700 1 --2- 172.21.15.163:0/3819730578 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa814157680 0x7fa8140cae80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:40.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.493+0000 7fa818926700 1 --2- 172.21.15.163:0/3819730578 >> v2:172.21.15.181:3300/0 conn(0x7fa814157a50 0x7fa8140cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:40.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.493+0000 7fa818926700 1 -- 172.21.15.163:0/3819730578 >> 172.21.15.163:0/3819730578 conn(0x7fa81406baa0 msgr2=0x7fa81406bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:40.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.493+0000 7fa818926700 1 -- 172.21.15.163:0/3819730578 shutdown_connections 2023-10-09T06:30:40.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.493+0000 7fa818926700 1 -- 172.21.15.163:0/3819730578 wait complete. 2023-10-09T06:30:40.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.494+0000 7fa818926700 1 Processor -- start 2023-10-09T06:30:40.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.494+0000 7fa818926700 1 -- start start 2023-10-09T06:30:40.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.495+0000 7fa818926700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8140b1450 0x7fa8141590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:40.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.495+0000 7fa818926700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa814157680 0x7fa814145220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:40.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.495+0000 7fa818926700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa814157a50 0x7fa814155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:40.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.495+0000 7fa818926700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa814158020 con 0x7fa8140b1450 2023-10-09T06:30:40.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.495+0000 7fa818926700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa814158300 con 0x7fa814157680 2023-10-09T06:30:40.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.495+0000 7fa818926700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8141585e0 con 0x7fa814157a50 2023-10-09T06:30:40.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.495+0000 7fa812d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8140b1450 0x7fa8141590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:40.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.495+0000 7fa812d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8140b1450 0x7fa8141590c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36632/0 (socket says 172.21.15.163:36632) 2023-10-09T06:30:40.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.495+0000 7fa812d9d700 1 -- 172.21.15.163:0/658507642 learned_addr learned my addr 172.21.15.163:0/658507642 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:40.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.495+0000 7fa81259c700 1 --2- 172.21.15.163:0/658507642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa814157680 0x7fa814145220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:40.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.495+0000 7fa81359e700 1 --2- 172.21.15.163:0/658507642 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa814157a50 0x7fa814155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:40.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.495+0000 7fa812d9d700 1 -- 172.21.15.163:0/658507642 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa814157a50 msgr2=0x7fa814155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:40.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.495+0000 7fa812d9d700 1 --2- 172.21.15.163:0/658507642 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa814157a50 0x7fa814155670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:40.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.495+0000 7fa812d9d700 1 -- 172.21.15.163:0/658507642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa814157680 msgr2=0x7fa814145220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:40.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.495+0000 7fa812d9d700 1 --2- 172.21.15.163:0/658507642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa814157680 0x7fa814145220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:40.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.495+0000 7fa812d9d700 1 -- 172.21.15.163:0/658507642 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa804009ae0 con 0x7fa8140b1450 2023-10-09T06:30:40.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.496+0000 7fa812d9d700 1 --2- 172.21.15.163:0/658507642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8140b1450 0x7fa8141590c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa804009880 tx=0x7fa804018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:40.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.496+0000 7fa80affd700 1 -- 172.21.15.163:0/658507642 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa804008bb0 con 0x7fa8140b1450 2023-10-09T06:30:40.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.496+0000 7fa80affd700 1 -- 172.21.15.163:0/658507642 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa804008d10 con 0x7fa8140b1450 2023-10-09T06:30:40.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.496+0000 7fa818926700 1 -- 172.21.15.163:0/658507642 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8141588c0 con 0x7fa8140b1450 2023-10-09T06:30:40.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.496+0000 7fa80affd700 1 -- 172.21.15.163:0/658507642 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8040266b0 con 0x7fa8140b1450 2023-10-09T06:30:40.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.496+0000 7fa818926700 1 -- 172.21.15.163:0/658507642 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa814145760 con 0x7fa8140b1450 2023-10-09T06:30:40.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.497+0000 7fa818926700 1 -- 172.21.15.163:0/658507642 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa814158de0 con 0x7fa8140b1450 2023-10-09T06:30:40.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.497+0000 7fa80affd700 1 -- 172.21.15.163:0/658507642 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa804003040 con 0x7fa8140b1450 2023-10-09T06:30:40.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.497+0000 7fa80affd700 1 --2- 172.21.15.163:0/658507642 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa7f4066f60 0x7fa7f4069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:40.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.498+0000 7fa80affd700 1 -- 172.21.15.163:0/658507642 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fa804086120 con 0x7fa8140b1450 2023-10-09T06:30:40.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.498+0000 7fa81259c700 1 --2- 172.21.15.163:0/658507642 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa7f4066f60 0x7fa7f4069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:40.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.498+0000 7fa81259c700 1 --2- 172.21.15.163:0/658507642 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa7f4066f60 0x7fa7f4069410 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa814156320 tx=0x7fa7fc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:40.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.502+0000 7fa80affd700 1 -- 172.21.15.163:0/658507642 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa814158de0 con 0x7fa8140b1450 2023-10-09T06:30:40.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.684+0000 7fa818926700 1 -- 172.21.15.163:0/658507642 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa814158de0 con 0x7fa8140b1450 2023-10-09T06:30:40.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.684+0000 7fa80affd700 1 -- 172.21.15.163:0/658507642 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa804053790 con 0x7fa8140b1450 2023-10-09T06:30:40.686 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:30:40.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.686+0000 7fa818926700 1 -- 172.21.15.163:0/658507642 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa7f4066f60 msgr2=0x7fa7f4069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:40.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.686+0000 7fa818926700 1 --2- 172.21.15.163:0/658507642 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa7f4066f60 0x7fa7f4069410 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa814156320 tx=0x7fa7fc008040 comp rx=0 tx=0).stop 2023-10-09T06:30:40.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.686+0000 7fa818926700 1 -- 172.21.15.163:0/658507642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8140b1450 msgr2=0x7fa8141590c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:40.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.686+0000 7fa818926700 1 --2- 172.21.15.163:0/658507642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8140b1450 0x7fa8141590c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa804009880 tx=0x7fa804018000 comp rx=0 tx=0).stop 2023-10-09T06:30:40.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.686+0000 7fa818926700 1 -- 172.21.15.163:0/658507642 shutdown_connections 2023-10-09T06:30:40.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.686+0000 7fa818926700 1 --2- 172.21.15.163:0/658507642 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa7f4066f60 0x7fa7f4069410 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:40.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.686+0000 7fa818926700 1 --2- 172.21.15.163:0/658507642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8140b1450 0x7fa8141590c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:40.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.686+0000 7fa818926700 1 --2- 172.21.15.163:0/658507642 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa814157a50 0x7fa814155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:40.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.686+0000 7fa818926700 1 --2- 172.21.15.163:0/658507642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa814157680 0x7fa814145220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:40.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.686+0000 7fa818926700 1 -- 172.21.15.163:0/658507642 >> 172.21.15.163:0/658507642 conn(0x7fa81406baa0 msgr2=0x7fa81406d130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:40.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.686+0000 7fa818926700 1 -- 172.21.15.163:0/658507642 shutdown_connections 2023-10-09T06:30:40.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.686+0000 7fa818926700 1 -- 172.21.15.163:0/658507642 wait complete. 2023-10-09T06:30:40.699 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-09T06:30:40.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.829+0000 7fd749598700 1 Processor -- start 2023-10-09T06:30:40.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.830+0000 7fd749598700 1 -- start start 2023-10-09T06:30:40.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.830+0000 7fd749598700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd74406bcd0 0x7fd744140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:40.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.830+0000 7fd749598700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd74406c0a0 0x7fd744141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:40.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.830+0000 7fd749598700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd74415c2a0 con 0x7fd7440c8e00 2023-10-09T06:30:40.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.830+0000 7fd749598700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7440c9900 con 0x7fd74406bcd0 2023-10-09T06:30:40.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.830+0000 7fd749598700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd74415b690 con 0x7fd74406c0a0 2023-10-09T06:30:40.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.830+0000 7fd742ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd74406bcd0 0x7fd744140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:40.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.830+0000 7fd742ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd74406bcd0 0x7fd744140df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36640/0 (socket says 172.21.15.163:36640) 2023-10-09T06:30:40.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.830+0000 7fd742ffd700 1 -- 172.21.15.163:0/2997989877 learned_addr learned my addr 172.21.15.163:0/2997989877 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:40.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.831+0000 7fd73a7fc700 1 --2- 172.21.15.163:0/2997989877 >> v2:172.21.15.181:3300/0 conn(0x7fd74406c0a0 0x7fd744141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:40.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.831+0000 7fd742ffd700 1 -- 172.21.15.163:0/2997989877 >> v2:172.21.15.181:3300/0 conn(0x7fd74406c0a0 msgr2=0x7fd744141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:40.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.831+0000 7fd742ffd700 1 --2- 172.21.15.163:0/2997989877 >> v2:172.21.15.181:3300/0 conn(0x7fd74406c0a0 0x7fd744141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:40.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.831+0000 7fd742ffd700 1 -- 172.21.15.163:0/2997989877 >> v1:172.21.15.181:6789/0 conn(0x7fd7440c8e00 legacy=0x7fd7440c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:40.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.831+0000 7fd742ffd700 1 -- 172.21.15.163:0/2997989877 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74415b970 con 0x7fd74406bcd0 2023-10-09T06:30:40.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.831+0000 7fd742ffd700 1 --2- 172.21.15.163:0/2997989877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd74406bcd0 0x7fd744140df0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd74406c6e0 tx=0x7fd734009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36ed87476d0e402 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:40.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.831+0000 7fd7427fc700 1 -- 172.21.15.163:0/2997989877 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd734009e70 con 0x7fd74406bcd0 2023-10-09T06:30:40.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.831+0000 7fd7427fc700 1 -- 172.21.15.163:0/2997989877 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd73400b860 con 0x7fd74406bcd0 2023-10-09T06:30:40.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.831+0000 7fd7427fc700 1 -- 172.21.15.163:0/2997989877 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73400bc00 con 0x7fd74406bcd0 2023-10-09T06:30:40.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.831+0000 7fd749598700 1 -- 172.21.15.163:0/2997989877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd74406bcd0 msgr2=0x7fd744140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:40.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.831+0000 7fd749598700 1 --2- 172.21.15.163:0/2997989877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd74406bcd0 0x7fd744140df0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd74406c6e0 tx=0x7fd734009250 comp rx=0 tx=0).stop 2023-10-09T06:30:40.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.832+0000 7fd749598700 1 -- 172.21.15.163:0/2997989877 shutdown_connections 2023-10-09T06:30:40.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.832+0000 7fd749598700 1 --2- 172.21.15.163:0/2997989877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd74406bcd0 0x7fd744140df0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:40.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.832+0000 7fd749598700 1 --2- 172.21.15.163:0/2997989877 >> v2:172.21.15.181:3300/0 conn(0x7fd74406c0a0 0x7fd744141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:40.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.832+0000 7fd749598700 1 -- 172.21.15.163:0/2997989877 >> 172.21.15.163:0/2997989877 conn(0x7fd7440ce600 msgr2=0x7fd7440cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:40.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.832+0000 7fd749598700 1 -- 172.21.15.163:0/2997989877 shutdown_connections 2023-10-09T06:30:40.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.832+0000 7fd749598700 1 -- 172.21.15.163:0/2997989877 wait complete. 2023-10-09T06:30:40.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.833+0000 7fd749598700 1 Processor -- start 2023-10-09T06:30:40.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.833+0000 7fd749598700 1 -- start start 2023-10-09T06:30:40.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.834+0000 7fd749598700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd74406bcd0 0x7fd74413f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:40.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.834+0000 7fd749598700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd74406c0a0 0x7fd7440b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:40.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.834+0000 7fd749598700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7440c8e00 0x7fd744157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:40.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.834+0000 7fd749598700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7440c56b0 con 0x7fd74406bcd0 2023-10-09T06:30:40.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.834+0000 7fd749598700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd744168830 con 0x7fd74406c0a0 2023-10-09T06:30:40.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.834+0000 7fd749598700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7440b3510 con 0x7fd7440c8e00 2023-10-09T06:30:40.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.834+0000 7fd742ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd74406bcd0 0x7fd74413f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:40.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.834+0000 7fd742ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd74406bcd0 0x7fd74413f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36654/0 (socket says 172.21.15.163:36654) 2023-10-09T06:30:40.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.834+0000 7fd742ffd700 1 -- 172.21.15.163:0/1366603825 learned_addr learned my addr 172.21.15.163:0/1366603825 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:40.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.834+0000 7fd73a7fc700 1 --2- 172.21.15.163:0/1366603825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd74406c0a0 0x7fd7440b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:40.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.834+0000 7fd7437fe700 1 --2- 172.21.15.163:0/1366603825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7440c8e00 0x7fd744157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:40.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.834+0000 7fd742ffd700 1 -- 172.21.15.163:0/1366603825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7440c8e00 msgr2=0x7fd744157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:40.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.834+0000 7fd742ffd700 1 --2- 172.21.15.163:0/1366603825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7440c8e00 0x7fd744157270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:40.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.834+0000 7fd742ffd700 1 -- 172.21.15.163:0/1366603825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd74406c0a0 msgr2=0x7fd7440b2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:40.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.834+0000 7fd742ffd700 1 --2- 172.21.15.163:0/1366603825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd74406c0a0 0x7fd7440b2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:40.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.834+0000 7fd742ffd700 1 -- 172.21.15.163:0/1366603825 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7340094f0 con 0x7fd74406bcd0 2023-10-09T06:30:40.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.834+0000 7fd742ffd700 1 --2- 172.21.15.163:0/1366603825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd74406bcd0 0x7fd74413f090 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd744069220 tx=0x7fd734017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:40.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.834+0000 7fd73b7fe700 1 -- 172.21.15.163:0/1366603825 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd734011420 con 0x7fd74406bcd0 2023-10-09T06:30:40.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.834+0000 7fd73b7fe700 1 -- 172.21.15.163:0/1366603825 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7340115a0 con 0x7fd74406bcd0 2023-10-09T06:30:40.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.834+0000 7fd73b7fe700 1 -- 172.21.15.163:0/1366603825 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd734011960 con 0x7fd74406bcd0 2023-10-09T06:30:40.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.834+0000 7fd749598700 1 -- 172.21.15.163:0/1366603825 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd744167f00 con 0x7fd74406bcd0 2023-10-09T06:30:40.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.834+0000 7fd749598700 1 -- 172.21.15.163:0/1366603825 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7441684c0 con 0x7fd74406bcd0 2023-10-09T06:30:40.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.835+0000 7fd749598700 1 -- 172.21.15.163:0/1366603825 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd744168bc0 con 0x7fd74406bcd0 2023-10-09T06:30:40.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.835+0000 7fd73b7fe700 1 -- 172.21.15.163:0/1366603825 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd7340253f0 con 0x7fd74406bcd0 2023-10-09T06:30:40.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.836+0000 7fd73b7fe700 1 --2- 172.21.15.163:0/1366603825 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd724066f30 0x7fd7240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:40.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.836+0000 7fd73b7fe700 1 -- 172.21.15.163:0/1366603825 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fd734085ba0 con 0x7fd74406bcd0 2023-10-09T06:30:40.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.836+0000 7fd73a7fc700 1 --2- 172.21.15.163:0/1366603825 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd724066f30 0x7fd7240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:40.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.837+0000 7fd73a7fc700 1 --2- 172.21.15.163:0/1366603825 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd724066f30 0x7fd7240693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd74416b1a0 tx=0x7fd72c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:40.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:40.840+0000 7fd73b7fe700 1 -- 172.21.15.163:0/1366603825 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd734053160 con 0x7fd74406bcd0 2023-10-09T06:30:41.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.025+0000 7fd749598700 1 -- 172.21.15.163:0/1366603825 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd74404c3c0 con 0x7fd74406bcd0 2023-10-09T06:30:41.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.026+0000 7fd73b7fe700 1 -- 172.21.15.163:0/1366603825 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd734015090 con 0x7fd74406bcd0 2023-10-09T06:30:41.028 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:30:41.028 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:30:41.028 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:30:41.028 INFO:teuthology.orchestra.run.smithi163.stdout: "poolnum": 1, 2023-10-09T06:30:41.028 INFO:teuthology.orchestra.run.smithi163.stdout: "poolname": ".mgr" 2023-10-09T06:30:41.028 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:30:41.028 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:30:41.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.028+0000 7fd749598700 1 -- 172.21.15.163:0/1366603825 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd724066f30 msgr2=0x7fd7240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:41.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.028+0000 7fd749598700 1 --2- 172.21.15.163:0/1366603825 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd724066f30 0x7fd7240693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd74416b1a0 tx=0x7fd72c008040 comp rx=0 tx=0).stop 2023-10-09T06:30:41.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.028+0000 7fd749598700 1 -- 172.21.15.163:0/1366603825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd74406bcd0 msgr2=0x7fd74413f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:41.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.028+0000 7fd749598700 1 --2- 172.21.15.163:0/1366603825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd74406bcd0 0x7fd74413f090 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd744069220 tx=0x7fd734017000 comp rx=0 tx=0).stop 2023-10-09T06:30:41.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.028+0000 7fd749598700 1 -- 172.21.15.163:0/1366603825 shutdown_connections 2023-10-09T06:30:41.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.028+0000 7fd749598700 1 --2- 172.21.15.163:0/1366603825 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd724066f30 0x7fd7240693e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:41.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.028+0000 7fd749598700 1 --2- 172.21.15.163:0/1366603825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd74406bcd0 0x7fd74413f090 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:41.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.028+0000 7fd749598700 1 --2- 172.21.15.163:0/1366603825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7440c8e00 0x7fd744157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:41.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.028+0000 7fd749598700 1 --2- 172.21.15.163:0/1366603825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd74406c0a0 0x7fd7440b2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:41.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.028+0000 7fd749598700 1 -- 172.21.15.163:0/1366603825 >> 172.21.15.163:0/1366603825 conn(0x7fd7440ce600 msgr2=0x7fd7440c3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:41.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.028+0000 7fd749598700 1 -- 172.21.15.163:0/1366603825 shutdown_connections 2023-10-09T06:30:41.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.028+0000 7fd749598700 1 -- 172.21.15.163:0/1366603825 wait complete. 2023-10-09T06:30:41.041 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-09T06:30:41.042 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-09T06:30:41.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.159+0000 7eff2f42c700 1 Processor -- start 2023-10-09T06:30:41.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.159+0000 7eff2f42c700 1 -- start start 2023-10-09T06:30:41.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.160+0000 7eff2f42c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7eff280cd910 0x7eff280ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:41.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.160+0000 7eff2f42c700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff280b5b00 con 0x7eff280b55a0 2023-10-09T06:30:41.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.160+0000 7eff2f42c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff280b5da0 con 0x7eff28159ec0 2023-10-09T06:30:41.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.160+0000 7eff2f42c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff280b6080 con 0x7eff280cd910 2023-10-09T06:30:41.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.160+0000 7eff2d9c9700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7eff280b55a0 0x7eff280ce880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:52368/0 (socket says 172.21.15.163:52368) 2023-10-09T06:30:41.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.160+0000 7eff2d9c9700 1 -- 172.21.15.163:0/1320970810 learned_addr learned my addr 172.21.15.163:0/1320970810 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:41.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.160+0000 7eff2c9c7700 1 --2- 172.21.15.163:0/1320970810 >> v2:172.21.15.181:3300/0 conn(0x7eff280cd910 0x7eff280ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:41.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.160+0000 7eff1ffff700 1 -- 172.21.15.163:0/1320970810 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 362904937 0 0) 0x7eff280b5b00 con 0x7eff280b55a0 2023-10-09T06:30:41.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.160+0000 7eff1ffff700 1 -- 172.21.15.163:0/1320970810 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff14003570 con 0x7eff280b55a0 2023-10-09T06:30:41.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff1ffff700 1 -- 172.21.15.163:0/1320970810 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 472574558 0 0) 0x7eff280b5da0 con 0x7eff28159ec0 2023-10-09T06:30:41.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff1ffff700 1 -- 172.21.15.163:0/1320970810 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff280b5b00 con 0x7eff28159ec0 2023-10-09T06:30:41.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff1ffff700 1 -- 172.21.15.163:0/1320970810 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 586898889 0 0) 0x7eff14003570 con 0x7eff280b55a0 2023-10-09T06:30:41.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff1ffff700 1 -- 172.21.15.163:0/1320970810 >> v2:172.21.15.181:3300/0 conn(0x7eff280cd910 msgr2=0x7eff280ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:41.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff1ffff700 1 --2- 172.21.15.163:0/1320970810 >> v2:172.21.15.181:3300/0 conn(0x7eff280cd910 0x7eff280ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:41.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff1ffff700 1 -- 172.21.15.163:0/1320970810 >> v1:172.21.15.181:6789/0 conn(0x7eff28159ec0 legacy=0x7eff280b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:41.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff1ffff700 1 -- 172.21.15.163:0/1320970810 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff280cc100 con 0x7eff280b55a0 2023-10-09T06:30:41.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff1ffff700 1 -- 172.21.15.163:0/1320970810 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7eff200017f0 con 0x7eff280b55a0 2023-10-09T06:30:41.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff1ffff700 1 -- 172.21.15.163:0/1320970810 >> v1:172.21.15.163:6789/0 conn(0x7eff280b55a0 legacy=0x7eff280ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:41.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff1ffff700 1 --2- 172.21.15.163:0/1320970810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff14003c20 0x7eff14004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:41.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff1ffff700 1 --2- 172.21.15.163:0/1320970810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff14004810 0x7eff14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:41.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff1ffff700 1 --2- 172.21.15.163:0/1320970810 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff140051c0 0x7eff1400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:41.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff1ffff700 1 -- 172.21.15.163:0/1320970810 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff280b6080 con 0x7eff14004810 2023-10-09T06:30:41.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff1ffff700 1 -- 172.21.15.163:0/1320970810 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1400c130 con 0x7eff14003c20 2023-10-09T06:30:41.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff1ffff700 1 -- 172.21.15.163:0/1320970810 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff1400c2f0 con 0x7eff140051c0 2023-10-09T06:30:41.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff2c9c7700 1 --2- 172.21.15.163:0/1320970810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff14004810 0x7eff14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:41.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff2d1c8700 1 --2- 172.21.15.163:0/1320970810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff14003c20 0x7eff14004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:41.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff2d9c9700 1 --2- 172.21.15.163:0/1320970810 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff140051c0 0x7eff1400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:41.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff2c9c7700 1 -- 172.21.15.163:0/1320970810 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff140051c0 msgr2=0x7eff1400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:41.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff2c9c7700 1 --2- 172.21.15.163:0/1320970810 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff140051c0 0x7eff1400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:41.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff2c9c7700 1 -- 172.21.15.163:0/1320970810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff14003c20 msgr2=0x7eff14004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:41.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff2c9c7700 1 --2- 172.21.15.163:0/1320970810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff14003c20 0x7eff14004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:41.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff2c9c7700 1 -- 172.21.15.163:0/1320970810 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff14003570 con 0x7eff14004810 2023-10-09T06:30:41.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff2d9c9700 1 --2- 172.21.15.163:0/1320970810 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff140051c0 0x7eff1400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:30:41.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff2d1c8700 1 --2- 172.21.15.163:0/1320970810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff14003c20 0x7eff14004010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:30:41.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff2d9c9700 1 -- 172.21.15.163:0/1320970810 reap_dead start 2023-10-09T06:30:41.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff2c9c7700 1 --2- 172.21.15.163:0/1320970810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff14004810 0x7eff14004c80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7eff280c75d0 tx=0x7eff180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f1e3a0b36789c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:41.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff1ffff700 1 -- 172.21.15.163:0/1320970810 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1800e710 con 0x7eff14004810 2023-10-09T06:30:41.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff1ffff700 1 -- 172.21.15.163:0/1320970810 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff1800e870 con 0x7eff14004810 2023-10-09T06:30:41.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.161+0000 7eff1ffff700 1 -- 172.21.15.163:0/1320970810 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1800ec30 con 0x7eff14004810 2023-10-09T06:30:41.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.162+0000 7eff2f42c700 1 -- 172.21.15.163:0/1320970810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff14004810 msgr2=0x7eff14004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:41.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.162+0000 7eff2f42c700 1 --2- 172.21.15.163:0/1320970810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff14004810 0x7eff14004c80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7eff280c75d0 tx=0x7eff180091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:41.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.162+0000 7eff2f42c700 1 -- 172.21.15.163:0/1320970810 shutdown_connections 2023-10-09T06:30:41.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.162+0000 7eff2f42c700 1 --2- 172.21.15.163:0/1320970810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff14004810 0x7eff14004c80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:41.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.162+0000 7eff2f42c700 1 -- 172.21.15.163:0/1320970810 >> 172.21.15.163:0/1320970810 conn(0x7eff280999f0 msgr2=0x7eff280b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:41.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.162+0000 7eff2f42c700 1 -- 172.21.15.163:0/1320970810 shutdown_connections 2023-10-09T06:30:41.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.162+0000 7eff2f42c700 1 -- 172.21.15.163:0/1320970810 wait complete. 2023-10-09T06:30:41.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.163+0000 7eff2f42c700 1 Processor -- start 2023-10-09T06:30:41.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.163+0000 7eff2f42c700 1 -- start start 2023-10-09T06:30:41.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.164+0000 7eff2f42c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff14004810 0x7eff28069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:41.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.164+0000 7eff2f42c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff28155e60 0x7eff28155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:41.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.164+0000 7eff2f42c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff281586d0 0x7eff28158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:41.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.164+0000 7eff2f42c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff280699b0 con 0x7eff14004810 2023-10-09T06:30:41.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.164+0000 7eff2f42c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff281579c0 con 0x7eff281586d0 2023-10-09T06:30:41.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.164+0000 7eff2f42c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff28069ea0 con 0x7eff28155e60 2023-10-09T06:30:41.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.164+0000 7eff2d1c8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff14004810 0x7eff28069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:41.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.164+0000 7eff2d1c8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff14004810 0x7eff28069470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36668/0 (socket says 172.21.15.163:36668) 2023-10-09T06:30:41.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.164+0000 7eff2d1c8700 1 -- 172.21.15.163:0/405070134 learned_addr learned my addr 172.21.15.163:0/405070134 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:41.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.164+0000 7eff2c9c7700 1 --2- 172.21.15.163:0/405070134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff28155e60 0x7eff28155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:41.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.164+0000 7eff2d9c9700 1 --2- 172.21.15.163:0/405070134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff281586d0 0x7eff28158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:41.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.164+0000 7eff2d1c8700 1 -- 172.21.15.163:0/405070134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff28155e60 msgr2=0x7eff28155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:41.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.164+0000 7eff2d1c8700 1 --2- 172.21.15.163:0/405070134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff28155e60 0x7eff28155670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:41.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.164+0000 7eff2d1c8700 1 -- 172.21.15.163:0/405070134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff281586d0 msgr2=0x7eff28158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:41.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.164+0000 7eff2d1c8700 1 --2- 172.21.15.163:0/405070134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff281586d0 0x7eff28158b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:41.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.164+0000 7eff2d1c8700 1 -- 172.21.15.163:0/405070134 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff28158240 con 0x7eff14004810 2023-10-09T06:30:41.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.164+0000 7eff2d9c9700 1 --2- 172.21.15.163:0/405070134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff281586d0 0x7eff28158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:30:41.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.165+0000 7eff2d1c8700 1 --2- 172.21.15.163:0/405070134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff14004810 0x7eff28069470 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7eff2400e870 tx=0x7eff24009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:41.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.165+0000 7eff1d7fa700 1 -- 172.21.15.163:0/405070134 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24006870 con 0x7eff14004810 2023-10-09T06:30:41.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.165+0000 7eff1d7fa700 1 -- 172.21.15.163:0/405070134 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff240069d0 con 0x7eff14004810 2023-10-09T06:30:41.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.165+0000 7eff1d7fa700 1 -- 172.21.15.163:0/405070134 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24006d70 con 0x7eff14004810 2023-10-09T06:30:41.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.165+0000 7eff2f42c700 1 -- 172.21.15.163:0/405070134 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff28068850 con 0x7eff14004810 2023-10-09T06:30:41.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.165+0000 7eff2f42c700 1 -- 172.21.15.163:0/405070134 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff28157bf0 con 0x7eff14004810 2023-10-09T06:30:41.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.166+0000 7eff2f42c700 1 -- 172.21.15.163:0/405070134 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff28063c90 con 0x7eff14004810 2023-10-09T06:30:41.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.166+0000 7eff1d7fa700 1 -- 172.21.15.163:0/405070134 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7eff24007240 con 0x7eff14004810 2023-10-09T06:30:41.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.166+0000 7eff1d7fa700 1 --2- 172.21.15.163:0/405070134 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7eff0c066f30 0x7eff0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:41.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.166+0000 7eff1d7fa700 1 -- 172.21.15.163:0/405070134 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7eff24084520 con 0x7eff14004810 2023-10-09T06:30:41.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.166+0000 7eff2c9c7700 1 --2- 172.21.15.163:0/405070134 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7eff0c066f30 0x7eff0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:41.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.167+0000 7eff2c9c7700 1 --2- 172.21.15.163:0/405070134 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7eff0c066f30 0x7eff0c0693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7eff180091e0 tx=0x7eff1801a000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:41.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.170+0000 7eff1d7fa700 1 -- 172.21.15.163:0/405070134 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff24051ae0 con 0x7eff14004810 2023-10-09T06:30:41.352 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.351+0000 7eff2f42c700 1 -- 172.21.15.163:0/405070134 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7eff280718b0 con 0x7eff14004810 2023-10-09T06:30:41.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.737+0000 7eff1d7fa700 1 -- 172.21.15.163:0/405070134 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7eff24055100 con 0x7eff14004810 2023-10-09T06:30:41.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.740+0000 7eff2f42c700 1 -- 172.21.15.163:0/405070134 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7eff0c066f30 msgr2=0x7eff0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:41.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.740+0000 7eff2f42c700 1 --2- 172.21.15.163:0/405070134 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7eff0c066f30 0x7eff0c0693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7eff180091e0 tx=0x7eff1801a000 comp rx=0 tx=0).stop 2023-10-09T06:30:41.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.740+0000 7eff2f42c700 1 -- 172.21.15.163:0/405070134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff14004810 msgr2=0x7eff28069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:41.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.740+0000 7eff2f42c700 1 --2- 172.21.15.163:0/405070134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff14004810 0x7eff28069470 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7eff2400e870 tx=0x7eff24009040 comp rx=0 tx=0).stop 2023-10-09T06:30:41.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.740+0000 7eff2f42c700 1 -- 172.21.15.163:0/405070134 shutdown_connections 2023-10-09T06:30:41.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.740+0000 7eff2f42c700 1 --2- 172.21.15.163:0/405070134 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7eff0c066f30 0x7eff0c0693e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:41.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.740+0000 7eff2f42c700 1 --2- 172.21.15.163:0/405070134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff14004810 0x7eff28069470 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:41.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.740+0000 7eff2f42c700 1 --2- 172.21.15.163:0/405070134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff281586d0 0x7eff28158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:41.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.740+0000 7eff2f42c700 1 --2- 172.21.15.163:0/405070134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff28155e60 0x7eff28155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:41.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.740+0000 7eff2f42c700 1 -- 172.21.15.163:0/405070134 >> 172.21.15.163:0/405070134 conn(0x7eff280999f0 msgr2=0x7eff280c63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:41.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.740+0000 7eff2f42c700 1 -- 172.21.15.163:0/405070134 shutdown_connections 2023-10-09T06:30:41.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.740+0000 7eff2f42c700 1 -- 172.21.15.163:0/405070134 wait complete. 2023-10-09T06:30:41.743 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_metadata' created 2023-10-09T06:30:41.756 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-09T06:30:41.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.889+0000 7f57d1ddd700 1 Processor -- start 2023-10-09T06:30:41.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.889+0000 7f57d1ddd700 1 -- start start 2023-10-09T06:30:41.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.890+0000 7f57d1ddd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f57cc0cca50 0x7f57cc0caab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:41.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.890+0000 7f57d1ddd700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57cc157c50 con 0x7f57cc0cce20 2023-10-09T06:30:41.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.890+0000 7f57d1ddd700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57cc15d7d0 con 0x7f57cc15df40 2023-10-09T06:30:41.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.890+0000 7f57d1ddd700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57cc15aa20 con 0x7f57cc0cca50 2023-10-09T06:30:41.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.890+0000 7f57caffd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f57cc0cce20 0x7f57cc15a1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:52382/0 (socket says 172.21.15.163:52382) 2023-10-09T06:30:41.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.890+0000 7f57caffd700 1 -- 172.21.15.163:0/2808434789 learned_addr learned my addr 172.21.15.163:0/2808434789 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:41.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.890+0000 7f57cb7fe700 1 --2- 172.21.15.163:0/2808434789 >> v2:172.21.15.181:3300/0 conn(0x7f57cc0cca50 0x7f57cc0caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:41.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.890+0000 7f57ca7fc700 1 -- 172.21.15.163:0/2808434789 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2154177901 0 0) 0x7f57c0001110 con 0x7f57cc0cce20 2023-10-09T06:30:41.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.890+0000 7f57ca7fc700 1 -- 172.21.15.163:0/2808434789 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57b4003570 con 0x7f57cc0cce20 2023-10-09T06:30:41.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.891+0000 7f57ca7fc700 1 -- 172.21.15.163:0/2808434789 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3427505770 0 0) 0x7f57b4003570 con 0x7f57cc0cce20 2023-10-09T06:30:41.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.891+0000 7f57ca7fc700 1 -- 172.21.15.163:0/2808434789 >> v2:172.21.15.181:3300/0 conn(0x7f57cc0cca50 msgr2=0x7f57cc0caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:41.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.891+0000 7f57ca7fc700 1 --2- 172.21.15.163:0/2808434789 >> v2:172.21.15.181:3300/0 conn(0x7f57cc0cca50 0x7f57cc0caab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:41.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.891+0000 7f57ca7fc700 1 -- 172.21.15.163:0/2808434789 >> v1:172.21.15.181:6789/0 conn(0x7f57cc15df40 legacy=0x7f57cc06ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:41.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.891+0000 7f57ca7fc700 1 -- 172.21.15.163:0/2808434789 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57cc15ad00 con 0x7f57cc0cce20 2023-10-09T06:30:41.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.891+0000 7f57ca7fc700 1 -- 172.21.15.163:0/2808434789 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f57c0001600 con 0x7f57cc0cce20 2023-10-09T06:30:41.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.891+0000 7f57ca7fc700 1 -- 172.21.15.163:0/2808434789 >> v1:172.21.15.163:6789/0 conn(0x7f57cc0cce20 legacy=0x7f57cc15a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:41.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.891+0000 7f57ca7fc700 1 --2- 172.21.15.163:0/2808434789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57b4003a60 0x7f57b4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:41.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.891+0000 7f57ca7fc700 1 --2- 172.21.15.163:0/2808434789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57b4004810 0x7f57b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:41.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.891+0000 7f57ca7fc700 1 --2- 172.21.15.163:0/2808434789 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57b40052e0 0x7f57b400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:41.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.891+0000 7f57ca7fc700 1 -- 172.21.15.163:0/2808434789 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc15aa20 con 0x7f57b4003a60 2023-10-09T06:30:41.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.891+0000 7f57ca7fc700 1 -- 172.21.15.163:0/2808434789 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b400c230 con 0x7f57b4004810 2023-10-09T06:30:41.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.891+0000 7f57ca7fc700 1 -- 172.21.15.163:0/2808434789 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b400c3f0 con 0x7f57b40052e0 2023-10-09T06:30:41.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.891+0000 7f57cb7fe700 1 --2- 172.21.15.163:0/2808434789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57b4003a60 0x7f57b4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:41.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.891+0000 7f57caffd700 1 --2- 172.21.15.163:0/2808434789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57b4004810 0x7f57b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:41.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.891+0000 7f57cbfff700 1 --2- 172.21.15.163:0/2808434789 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57b40052e0 0x7f57b400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:41.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.891+0000 7f57cb7fe700 1 -- 172.21.15.163:0/2808434789 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57b40052e0 msgr2=0x7f57b400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:41.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.891+0000 7f57cb7fe700 1 --2- 172.21.15.163:0/2808434789 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57b40052e0 0x7f57b400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:41.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.891+0000 7f57cb7fe700 1 -- 172.21.15.163:0/2808434789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57b4004810 msgr2=0x7f57b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:41.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.891+0000 7f57cb7fe700 1 --2- 172.21.15.163:0/2808434789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57b4004810 0x7f57b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:41.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.891+0000 7f57cb7fe700 1 -- 172.21.15.163:0/2808434789 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b4003570 con 0x7f57b4003a60 2023-10-09T06:30:41.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.891+0000 7f57cbfff700 1 -- 172.21.15.163:0/2808434789 reap_dead start 2023-10-09T06:30:41.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.891+0000 7f57cb7fe700 1 --2- 172.21.15.163:0/2808434789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57b4003a60 0x7f57b4003e70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00bb90 tx=0x7f57bc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c1b337f6b3c907 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:41.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.891+0000 7f57ca7fc700 1 -- 172.21.15.163:0/2808434789 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc00d5d0 con 0x7f57b4003a60 2023-10-09T06:30:41.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.892+0000 7f57ca7fc700 1 -- 172.21.15.163:0/2808434789 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57bc00d730 con 0x7f57b4003a60 2023-10-09T06:30:41.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.892+0000 7f57ca7fc700 1 -- 172.21.15.163:0/2808434789 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc00daf0 con 0x7f57b4003a60 2023-10-09T06:30:41.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.892+0000 7f57d1ddd700 1 -- 172.21.15.163:0/2808434789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57b4003a60 msgr2=0x7f57b4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:41.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.892+0000 7f57d1ddd700 1 --2- 172.21.15.163:0/2808434789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57b4003a60 0x7f57b4003e70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00bb90 tx=0x7f57bc008040 comp rx=0 tx=0).stop 2023-10-09T06:30:41.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.892+0000 7f57d1ddd700 1 -- 172.21.15.163:0/2808434789 shutdown_connections 2023-10-09T06:30:41.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.892+0000 7f57d1ddd700 1 --2- 172.21.15.163:0/2808434789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57b4003a60 0x7f57b4003e70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:41.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.892+0000 7f57d1ddd700 1 -- 172.21.15.163:0/2808434789 >> 172.21.15.163:0/2808434789 conn(0x7f57cc0ce5c0 msgr2=0x7f57cc0ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:41.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.892+0000 7f57d1ddd700 1 -- 172.21.15.163:0/2808434789 shutdown_connections 2023-10-09T06:30:41.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.892+0000 7f57d1ddd700 1 -- 172.21.15.163:0/2808434789 wait complete. 2023-10-09T06:30:41.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.893+0000 7f57d1ddd700 1 Processor -- start 2023-10-09T06:30:41.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.893+0000 7f57d1ddd700 1 -- start start 2023-10-09T06:30:41.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.894+0000 7f57d1ddd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57b4003a60 0x7f57cc0c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:41.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.894+0000 7f57d1ddd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57cc14a340 0x7f57cc0c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:41.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.894+0000 7f57d1ddd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57cc0caa90 0x7f57cc0caf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:41.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.894+0000 7f57d1ddd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc0c95b0 con 0x7f57cc0caa90 2023-10-09T06:30:41.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.894+0000 7f57d1ddd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc16b170 con 0x7f57cc14a340 2023-10-09T06:30:41.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.894+0000 7f57d1ddd700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc0cb440 con 0x7f57b4003a60 2023-10-09T06:30:41.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.894+0000 7f57cbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57cc0caa90 0x7f57cc0caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:41.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.894+0000 7f57cbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57cc0caa90 0x7f57cc0caf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36688/0 (socket says 172.21.15.163:36688) 2023-10-09T06:30:41.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.894+0000 7f57cbfff700 1 -- 172.21.15.163:0/1157109916 learned_addr learned my addr 172.21.15.163:0/1157109916 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:41.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.894+0000 7f57cb7fe700 1 --2- 172.21.15.163:0/1157109916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57b4003a60 0x7f57cc0c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:41.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.894+0000 7f57caffd700 1 --2- 172.21.15.163:0/1157109916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57cc14a340 0x7f57cc0c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:41.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.894+0000 7f57cbfff700 1 -- 172.21.15.163:0/1157109916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57b4003a60 msgr2=0x7f57cc0c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:41.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.894+0000 7f57cbfff700 1 --2- 172.21.15.163:0/1157109916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57b4003a60 0x7f57cc0c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:41.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.894+0000 7f57cbfff700 1 -- 172.21.15.163:0/1157109916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57cc14a340 msgr2=0x7f57cc0c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:41.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.894+0000 7f57cbfff700 1 --2- 172.21.15.163:0/1157109916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57cc14a340 0x7f57cc0c75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:41.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.894+0000 7f57cbfff700 1 -- 172.21.15.163:0/1157109916 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57cc0c7af0 con 0x7f57cc0caa90 2023-10-09T06:30:41.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.895+0000 7f57cbfff700 1 --2- 172.21.15.163:0/1157109916 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57cc0caa90 0x7f57cc0caf00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f57c4000f80 tx=0x7f57c4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:41.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.895+0000 7f57b37fe700 1 -- 172.21.15.163:0/1157109916 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c40032e0 con 0x7f57cc0caa90 2023-10-09T06:30:41.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.895+0000 7f57b37fe700 1 -- 172.21.15.163:0/1157109916 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c4001580 con 0x7f57cc0caa90 2023-10-09T06:30:41.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.895+0000 7f57d1ddd700 1 -- 172.21.15.163:0/1157109916 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57cc14a7b0 con 0x7f57cc0caa90 2023-10-09T06:30:41.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.895+0000 7f57b37fe700 1 -- 172.21.15.163:0/1157109916 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c4001940 con 0x7f57cc0caa90 2023-10-09T06:30:41.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.895+0000 7f57d1ddd700 1 -- 172.21.15.163:0/1157109916 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57cc16a840 con 0x7f57cc0caa90 2023-10-09T06:30:41.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.896+0000 7f57d1ddd700 1 -- 172.21.15.163:0/1157109916 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57cc0c5670 con 0x7f57cc0caa90 2023-10-09T06:30:41.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.896+0000 7f57b37fe700 1 -- 172.21.15.163:0/1157109916 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f57c4001cb0 con 0x7f57cc0caa90 2023-10-09T06:30:41.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.897+0000 7f57b37fe700 1 --2- 172.21.15.163:0/1157109916 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f57b8066f30 0x7f57b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:41.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.897+0000 7f57b37fe700 1 -- 172.21.15.163:0/1157109916 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f57c4083bf0 con 0x7f57cc0caa90 2023-10-09T06:30:41.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.897+0000 7f57cb7fe700 1 --2- 172.21.15.163:0/1157109916 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f57b8066f30 0x7f57b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:41.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.897+0000 7f57cb7fe700 1 --2- 172.21.15.163:0/1157109916 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f57b8066f30 0x7f57b80693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00bb90 tx=0x7f57bc01b000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:41.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:41.901+0000 7f57b37fe700 1 -- 172.21.15.163:0/1157109916 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f57cc0c5670 con 0x7f57cc0caa90 2023-10-09T06:30:42.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.085+0000 7f57d1ddd700 1 -- 172.21.15.163:0/1157109916 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f57cc0c5670 con 0x7f57cc0caa90 2023-10-09T06:30:42.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.744+0000 7f57b37fe700 1 -- 172.21.15.163:0/1157109916 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f57cc0c5670 con 0x7f57cc0caa90 2023-10-09T06:30:42.751 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.750+0000 7f57d1ddd700 1 -- 172.21.15.163:0/1157109916 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f57b8066f30 msgr2=0x7f57b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:42.751 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.750+0000 7f57d1ddd700 1 --2- 172.21.15.163:0/1157109916 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f57b8066f30 0x7f57b80693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00bb90 tx=0x7f57bc01b000 comp rx=0 tx=0).stop 2023-10-09T06:30:42.751 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.750+0000 7f57d1ddd700 1 -- 172.21.15.163:0/1157109916 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57cc0caa90 msgr2=0x7f57cc0caf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:42.751 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.750+0000 7f57d1ddd700 1 --2- 172.21.15.163:0/1157109916 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57cc0caa90 0x7f57cc0caf00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f57c4000f80 tx=0x7f57c4009040 comp rx=0 tx=0).stop 2023-10-09T06:30:42.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.752+0000 7f57d1ddd700 1 -- 172.21.15.163:0/1157109916 shutdown_connections 2023-10-09T06:30:42.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.752+0000 7f57d1ddd700 1 --2- 172.21.15.163:0/1157109916 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f57b8066f30 0x7f57b80693e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:42.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.752+0000 7f57d1ddd700 1 --2- 172.21.15.163:0/1157109916 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57cc0caa90 0x7f57cc0caf00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:42.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.752+0000 7f57d1ddd700 1 --2- 172.21.15.163:0/1157109916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57b4003a60 0x7f57cc0c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:42.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.752+0000 7f57d1ddd700 1 --2- 172.21.15.163:0/1157109916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57cc14a340 0x7f57cc0c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:42.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.752+0000 7f57d1ddd700 1 -- 172.21.15.163:0/1157109916 >> 172.21.15.163:0/1157109916 conn(0x7f57cc0ce5c0 msgr2=0x7f57cc0c3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:42.754 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.753+0000 7f57d1ddd700 1 -- 172.21.15.163:0/1157109916 shutdown_connections 2023-10-09T06:30:42.754 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.753+0000 7f57d1ddd700 1 -- 172.21.15.163:0/1157109916 wait complete. 2023-10-09T06:30:42.757 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_data' created 2023-10-09T06:30:42.771 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-09T06:30:42.785 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:30:42.819 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:30:42.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.891+0000 7fb3eae15700 1 Processor -- start 2023-10-09T06:30:42.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.891+0000 7fb3eae15700 1 -- start start 2023-10-09T06:30:42.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.892+0000 7fb3eae15700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb3e40c7fd0 0x7fb3e40c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:42.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.892+0000 7fb3eae15700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e40cb5d0 con 0x7fb3e40c9770 2023-10-09T06:30:42.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.892+0000 7fb3eae15700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e40af490 con 0x7fb3e40c93a0 2023-10-09T06:30:42.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.892+0000 7fb3eae15700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e40af770 con 0x7fb3e40c7fd0 2023-10-09T06:30:42.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.892+0000 7fb3e9612700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb3e40c9770 0x7fb3e40caec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:52396/0 (socket says 172.21.15.163:52396) 2023-10-09T06:30:42.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.892+0000 7fb3e9612700 1 -- 172.21.15.163:0/1097412877 learned_addr learned my addr 172.21.15.163:0/1097412877 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:42.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.892+0000 7fb3ea614700 1 --2- 172.21.15.163:0/1097412877 >> v2:172.21.15.181:3300/0 conn(0x7fb3e40c7fd0 0x7fb3e40c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:42.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.892+0000 7fb3e8e11700 1 -- 172.21.15.163:0/1097412877 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 510999847 0 0) 0x7fb3e40cb5d0 con 0x7fb3e40c9770 2023-10-09T06:30:42.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.892+0000 7fb3e8e11700 1 -- 172.21.15.163:0/1097412877 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3cc003570 con 0x7fb3e40c9770 2023-10-09T06:30:42.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.893+0000 7fb3e8e11700 1 -- 172.21.15.163:0/1097412877 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3655515328 0 0) 0x7fb3e40af490 con 0x7fb3e40c93a0 2023-10-09T06:30:42.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.893+0000 7fb3e8e11700 1 -- 172.21.15.163:0/1097412877 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3e40cb5d0 con 0x7fb3e40c93a0 2023-10-09T06:30:42.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.893+0000 7fb3e8e11700 1 -- 172.21.15.163:0/1097412877 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2673974308 0 0) 0x7fb3cc003570 con 0x7fb3e40c9770 2023-10-09T06:30:42.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.893+0000 7fb3ea614700 1 -- 172.21.15.163:0/1097412877 >> v1:172.21.15.181:6789/0 conn(0x7fb3e40c93a0 legacy=0x7fb3e406c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:42.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.893+0000 7fb3ea614700 1 -- 172.21.15.163:0/1097412877 >> v1:172.21.15.163:6789/0 conn(0x7fb3e40c9770 legacy=0x7fb3e40caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:42.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.893+0000 7fb3ea614700 1 -- 172.21.15.163:0/1097412877 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e40afa50 con 0x7fb3e40c7fd0 2023-10-09T06:30:42.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.893+0000 7fb3ea614700 1 --2- 172.21.15.163:0/1097412877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3e40c7fd0 0x7fb3e40c83f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40098e0 tx=0x7fb3d40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c824670f7b4616c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:42.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.893+0000 7fb3e8e11700 1 -- 172.21.15.163:0/1097412877 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d80015f0 con 0x7fb3e40c7fd0 2023-10-09T06:30:42.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.893+0000 7fb3e8e11700 1 -- 172.21.15.163:0/1097412877 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d4011420 con 0x7fb3e40c7fd0 2023-10-09T06:30:42.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.893+0000 7fb3e8e11700 1 -- 172.21.15.163:0/1097412877 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d40117c0 con 0x7fb3e40c7fd0 2023-10-09T06:30:42.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.893+0000 7fb3eae15700 1 -- 172.21.15.163:0/1097412877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3e40c7fd0 msgr2=0x7fb3e40c83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:42.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.893+0000 7fb3eae15700 1 --2- 172.21.15.163:0/1097412877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3e40c7fd0 0x7fb3e40c83f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40098e0 tx=0x7fb3d40091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:42.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.893+0000 7fb3eae15700 1 -- 172.21.15.163:0/1097412877 shutdown_connections 2023-10-09T06:30:42.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.893+0000 7fb3eae15700 1 --2- 172.21.15.163:0/1097412877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3e40c7fd0 0x7fb3e40c83f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:42.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.893+0000 7fb3eae15700 1 -- 172.21.15.163:0/1097412877 >> 172.21.15.163:0/1097412877 conn(0x7fb3e40cddc0 msgr2=0x7fb3e40c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:42.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.894+0000 7fb3eae15700 1 -- 172.21.15.163:0/1097412877 shutdown_connections 2023-10-09T06:30:42.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.894+0000 7fb3eae15700 1 -- 172.21.15.163:0/1097412877 wait complete. 2023-10-09T06:30:42.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.895+0000 7fb3eae15700 1 Processor -- start 2023-10-09T06:30:42.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.895+0000 7fb3eae15700 1 -- start start 2023-10-09T06:30:42.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.895+0000 7fb3eae15700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3e40c7fd0 0x7fb3e4154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:42.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.895+0000 7fb3eae15700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3e40c93a0 0x7fb3e4155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:42.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.895+0000 7fb3eae15700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3e40c9770 0x7fb3e40b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:42.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.895+0000 7fb3eae15700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e4156550 con 0x7fb3e40c9770 2023-10-09T06:30:42.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.895+0000 7fb3eae15700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e4156340 con 0x7fb3e40c93a0 2023-10-09T06:30:42.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.895+0000 7fb3eae15700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e40b1b10 con 0x7fb3e40c7fd0 2023-10-09T06:30:42.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.895+0000 7fb3ea614700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3e40c9770 0x7fb3e40b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:42.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.895+0000 7fb3ea614700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3e40c9770 0x7fb3e40b28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36702/0 (socket says 172.21.15.163:36702) 2023-10-09T06:30:42.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.895+0000 7fb3ea614700 1 -- 172.21.15.163:0/2614187928 learned_addr learned my addr 172.21.15.163:0/2614187928 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:42.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.896+0000 7fb3e9612700 1 --2- 172.21.15.163:0/2614187928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3e40c93a0 0x7fb3e4155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:42.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.896+0000 7fb3e9e13700 1 --2- 172.21.15.163:0/2614187928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3e40c7fd0 0x7fb3e4154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:42.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.896+0000 7fb3ea614700 1 -- 172.21.15.163:0/2614187928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3e40c7fd0 msgr2=0x7fb3e4154de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:42.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.896+0000 7fb3ea614700 1 --2- 172.21.15.163:0/2614187928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3e40c7fd0 0x7fb3e4154de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:42.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.896+0000 7fb3ea614700 1 -- 172.21.15.163:0/2614187928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3e40c93a0 msgr2=0x7fb3e4155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:42.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.896+0000 7fb3ea614700 1 --2- 172.21.15.163:0/2614187928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3e40c93a0 0x7fb3e4155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:42.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.896+0000 7fb3ea614700 1 -- 172.21.15.163:0/2614187928 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d4009420 con 0x7fb3e40c9770 2023-10-09T06:30:42.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.896+0000 7fb3e9e13700 1 --2- 172.21.15.163:0/2614187928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3e40c7fd0 0x7fb3e4154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:30:42.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.896+0000 7fb3ea614700 1 --2- 172.21.15.163:0/2614187928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3e40c9770 0x7fb3e40b28d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40093f0 tx=0x7fb3d4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:42.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.896+0000 7fb3e1ffb700 1 -- 172.21.15.163:0/2614187928 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d401cd50 con 0x7fb3e40c9770 2023-10-09T06:30:42.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.896+0000 7fb3e1ffb700 1 -- 172.21.15.163:0/2614187928 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d401a450 con 0x7fb3e40c9770 2023-10-09T06:30:42.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.896+0000 7fb3eae15700 1 -- 172.21.15.163:0/2614187928 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e40b1df0 con 0x7fb3e40c9770 2023-10-09T06:30:42.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.896+0000 7fb3e1ffb700 1 -- 172.21.15.163:0/2614187928 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d401a810 con 0x7fb3e40c9770 2023-10-09T06:30:42.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.896+0000 7fb3eae15700 1 -- 172.21.15.163:0/2614187928 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e40b23b0 con 0x7fb3e40c9770 2023-10-09T06:30:42.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.897+0000 7fb3eae15700 1 -- 172.21.15.163:0/2614187928 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e40c8f60 con 0x7fb3e40c9770 2023-10-09T06:30:42.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.898+0000 7fb3e1ffb700 1 -- 172.21.15.163:0/2614187928 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb3d401ab80 con 0x7fb3e40c9770 2023-10-09T06:30:42.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.898+0000 7fb3e1ffb700 1 --2- 172.21.15.163:0/2614187928 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb3d0066f30 0x7fb3d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:42.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.898+0000 7fb3e1ffb700 1 -- 172.21.15.163:0/2614187928 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fb3d4085ec0 con 0x7fb3e40c9770 2023-10-09T06:30:42.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.898+0000 7fb3e9e13700 1 --2- 172.21.15.163:0/2614187928 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb3d0066f30 0x7fb3d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:42.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.899+0000 7fb3e9e13700 1 --2- 172.21.15.163:0/2614187928 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb3d0066f30 0x7fb3d00693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc004040 tx=0x7fb3dc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:42.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:42.902+0000 7fb3e1ffb700 1 -- 172.21.15.163:0/2614187928 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3d4053190 con 0x7fb3e40c9770 2023-10-09T06:30:43.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.131+0000 7fb3eae15700 1 -- 172.21.15.163:0/2614187928 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb3e40b25d0 con 0x7fb3e40c9770 2023-10-09T06:30:43.752 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.751+0000 7fb3e1ffb700 1 -- 172.21.15.163:0/2614187928 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb3d40567b0 con 0x7fb3e40c9770 2023-10-09T06:30:43.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.753+0000 7fb3eae15700 1 -- 172.21.15.163:0/2614187928 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb3d0066f30 msgr2=0x7fb3d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:43.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.753+0000 7fb3eae15700 1 --2- 172.21.15.163:0/2614187928 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb3d0066f30 0x7fb3d00693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc004040 tx=0x7fb3dc009040 comp rx=0 tx=0).stop 2023-10-09T06:30:43.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.754+0000 7fb3eae15700 1 -- 172.21.15.163:0/2614187928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3e40c9770 msgr2=0x7fb3e40b28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:43.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.754+0000 7fb3eae15700 1 --2- 172.21.15.163:0/2614187928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3e40c9770 0x7fb3e40b28d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40093f0 tx=0x7fb3d4016000 comp rx=0 tx=0).stop 2023-10-09T06:30:43.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.754+0000 7fb3eae15700 1 -- 172.21.15.163:0/2614187928 shutdown_connections 2023-10-09T06:30:43.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.754+0000 7fb3eae15700 1 --2- 172.21.15.163:0/2614187928 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb3d0066f30 0x7fb3d00693e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:43.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.754+0000 7fb3eae15700 1 --2- 172.21.15.163:0/2614187928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3e40c9770 0x7fb3e40b28d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:43.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.754+0000 7fb3eae15700 1 --2- 172.21.15.163:0/2614187928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3e40c7fd0 0x7fb3e4154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:43.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.754+0000 7fb3eae15700 1 --2- 172.21.15.163:0/2614187928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3e40c93a0 0x7fb3e4155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:43.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.754+0000 7fb3eae15700 1 -- 172.21.15.163:0/2614187928 >> 172.21.15.163:0/2614187928 conn(0x7fb3e40cddc0 msgr2=0x7fb3e4068c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:43.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.754+0000 7fb3eae15700 1 -- 172.21.15.163:0/2614187928 shutdown_connections 2023-10-09T06:30:43.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.754+0000 7fb3eae15700 1 -- 172.21.15.163:0/2614187928 wait complete. 2023-10-09T06:30:43.759 INFO:teuthology.orchestra.run.smithi163.stderr:new fs with metadata pool 2 and data pool 3 2023-10-09T06:30:43.770 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:30:43.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.925+0000 7ff2a98f4700 1 Processor -- start 2023-10-09T06:30:43.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.925+0000 7ff2a98f4700 1 -- start start 2023-10-09T06:30:43.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.926+0000 7ff2a98f4700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff2a40c8400 0x7ff2a406ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:43.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.926+0000 7ff2a98f4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2a406f2b0 0x7ff2a4068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:43.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.926+0000 7ff2a98f4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a406e8e0 con 0x7ff2a41597d0 2023-10-09T06:30:43.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.926+0000 7ff2a98f4700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a415a8f0 con 0x7ff2a40c8400 2023-10-09T06:30:43.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.926+0000 7ff2a98f4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a415abd0 con 0x7ff2a406f2b0 2023-10-09T06:30:43.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.926+0000 7ff29bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff2a40c8400 0x7ff2a406ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:43.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.926+0000 7ff29bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff2a40c8400 0x7ff2a406ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36716/0 (socket says 172.21.15.163:36716) 2023-10-09T06:30:43.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.926+0000 7ff29bfff700 1 -- 172.21.15.163:0/908938722 learned_addr learned my addr 172.21.15.163:0/908938722 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:43.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.926+0000 7ff2a90f3700 1 --2- 172.21.15.163:0/908938722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2a406f2b0 0x7ff2a4068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:43.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.926+0000 7ff29bfff700 1 -- 172.21.15.163:0/908938722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2a406f2b0 msgr2=0x7ff2a4068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:43.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.926+0000 7ff29bfff700 1 --2- 172.21.15.163:0/908938722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2a406f2b0 0x7ff2a4068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:43.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.926+0000 7ff29bfff700 1 -- 172.21.15.163:0/908938722 >> v1:172.21.15.181:6789/0 conn(0x7ff2a41597d0 legacy=0x7ff2a406bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:43.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.926+0000 7ff29bfff700 1 -- 172.21.15.163:0/908938722 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a415aeb0 con 0x7ff2a40c8400 2023-10-09T06:30:43.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.927+0000 7ff29bfff700 1 --2- 172.21.15.163:0/908938722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2a40c8400 0x7ff2a406ed70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff294009990 tx=0x7ff294009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117fb0b566564891 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:43.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.927+0000 7ff29b7fe700 1 -- 172.21.15.163:0/908938722 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff294005c10 con 0x7ff2a40c8400 2023-10-09T06:30:43.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.927+0000 7ff29b7fe700 1 -- 172.21.15.163:0/908938722 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff294005d70 con 0x7ff2a40c8400 2023-10-09T06:30:43.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.927+0000 7ff29b7fe700 1 -- 172.21.15.163:0/908938722 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29400b840 con 0x7ff2a40c8400 2023-10-09T06:30:43.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.927+0000 7ff2a98f4700 1 -- 172.21.15.163:0/908938722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2a40c8400 msgr2=0x7ff2a406ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:43.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.927+0000 7ff2a98f4700 1 --2- 172.21.15.163:0/908938722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2a40c8400 0x7ff2a406ed70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff294009990 tx=0x7ff294009220 comp rx=0 tx=0).stop 2023-10-09T06:30:43.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.927+0000 7ff2a98f4700 1 -- 172.21.15.163:0/908938722 shutdown_connections 2023-10-09T06:30:43.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.927+0000 7ff2a98f4700 1 --2- 172.21.15.163:0/908938722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2a406f2b0 0x7ff2a4068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:43.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.927+0000 7ff2a98f4700 1 --2- 172.21.15.163:0/908938722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2a40c8400 0x7ff2a406ed70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:43.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.927+0000 7ff2a98f4700 1 -- 172.21.15.163:0/908938722 >> 172.21.15.163:0/908938722 conn(0x7ff2a406b450 msgr2=0x7ff2a40cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:43.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.927+0000 7ff2a98f4700 1 -- 172.21.15.163:0/908938722 shutdown_connections 2023-10-09T06:30:43.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.927+0000 7ff2a98f4700 1 -- 172.21.15.163:0/908938722 wait complete. 2023-10-09T06:30:43.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.928+0000 7ff2a98f4700 1 Processor -- start 2023-10-09T06:30:43.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.929+0000 7ff2a98f4700 1 -- start start 2023-10-09T06:30:43.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.929+0000 7ff2a98f4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2a406f2b0 0x7ff2a4159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:43.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.929+0000 7ff2a98f4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2a40c8400 0x7ff2a4155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:43.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.929+0000 7ff2a98f4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2a41597d0 0x7ff2a4155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:43.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.929+0000 7ff2a98f4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a416a210 con 0x7ff2a41597d0 2023-10-09T06:30:43.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.929+0000 7ff2a98f4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a416a4f0 con 0x7ff2a40c8400 2023-10-09T06:30:43.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.929+0000 7ff2a98f4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a416a7d0 con 0x7ff2a406f2b0 2023-10-09T06:30:43.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.929+0000 7ff2a90f3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2a41597d0 0x7ff2a4155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:43.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.929+0000 7ff2a90f3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2a41597d0 0x7ff2a4155d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36724/0 (socket says 172.21.15.163:36724) 2023-10-09T06:30:43.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.929+0000 7ff2a90f3700 1 -- 172.21.15.163:0/197361988 learned_addr learned my addr 172.21.15.163:0/197361988 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:43.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.930+0000 7ff2a88f2700 1 --2- 172.21.15.163:0/197361988 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2a406f2b0 0x7ff2a4159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:43.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.930+0000 7ff29bfff700 1 --2- 172.21.15.163:0/197361988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2a40c8400 0x7ff2a4155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:43.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.930+0000 7ff2a90f3700 1 -- 172.21.15.163:0/197361988 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2a406f2b0 msgr2=0x7ff2a4159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:43.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.930+0000 7ff2a90f3700 1 --2- 172.21.15.163:0/197361988 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2a406f2b0 0x7ff2a4159df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:43.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.930+0000 7ff2a90f3700 1 -- 172.21.15.163:0/197361988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2a40c8400 msgr2=0x7ff2a4155800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:43.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.930+0000 7ff2a90f3700 1 --2- 172.21.15.163:0/197361988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2a40c8400 0x7ff2a4155800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:43.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.930+0000 7ff2a90f3700 1 -- 172.21.15.163:0/197361988 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff294009480 con 0x7ff2a41597d0 2023-10-09T06:30:43.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.930+0000 7ff2a90f3700 1 --2- 172.21.15.163:0/197361988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2a41597d0 0x7ff2a4155d40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff29c009fa0 tx=0x7ff29c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:43.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.930+0000 7ff298ff9700 1 -- 172.21.15.163:0/197361988 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c00ed20 con 0x7ff2a41597d0 2023-10-09T06:30:43.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.930+0000 7ff298ff9700 1 -- 172.21.15.163:0/197361988 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29c00ee80 con 0x7ff2a41597d0 2023-10-09T06:30:43.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.930+0000 7ff2a98f4700 1 -- 172.21.15.163:0/197361988 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a416aab0 con 0x7ff2a41597d0 2023-10-09T06:30:43.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.930+0000 7ff298ff9700 1 -- 172.21.15.163:0/197361988 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c014670 con 0x7ff2a41597d0 2023-10-09T06:30:43.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.930+0000 7ff2a98f4700 1 -- 172.21.15.163:0/197361988 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a415a330 con 0x7ff2a41597d0 2023-10-09T06:30:43.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.931+0000 7ff2a98f4700 1 -- 172.21.15.163:0/197361988 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a415a550 con 0x7ff2a41597d0 2023-10-09T06:30:43.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.931+0000 7ff298ff9700 1 -- 172.21.15.163:0/197361988 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff29c0149e0 con 0x7ff2a41597d0 2023-10-09T06:30:43.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.931+0000 7ff298ff9700 1 --2- 172.21.15.163:0/197361988 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff290066f30 0x7ff2900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:43.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.932+0000 7ff298ff9700 1 -- 172.21.15.163:0/197361988 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff29c084d70 con 0x7ff2a41597d0 2023-10-09T06:30:43.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.932+0000 7ff2a88f2700 1 --2- 172.21.15.163:0/197361988 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff290066f30 0x7ff2900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:43.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.932+0000 7ff2a88f2700 1 --2- 172.21.15.163:0/197361988 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff290066f30 0x7ff2900693e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000a8a0 tx=0x7ff2a0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:43.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:43.936+0000 7ff298ff9700 1 -- 172.21.15.163:0/197361988 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff29c051fe0 con 0x7ff2a41597d0 2023-10-09T06:30:44.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.118+0000 7ff2a98f4700 1 -- 172.21.15.163:0/197361988 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff2a416a350 con 0x7ff2a41597d0 2023-10-09T06:30:44.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.119+0000 7ff298ff9700 1 -- 172.21.15.163:0/197361988 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+15411 (secure 0 0 0) 0x7ff2a416a350 con 0x7ff2a41597d0 2023-10-09T06:30:44.119 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:30:44.120 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":19,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:30:43.132806+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:30:41.369014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-09T06:30:42.103001+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:30:44.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.121+0000 7ff2a98f4700 1 -- 172.21.15.163:0/197361988 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff290066f30 msgr2=0x7ff2900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:44.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.121+0000 7ff2a98f4700 1 --2- 172.21.15.163:0/197361988 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff290066f30 0x7ff2900693e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000a8a0 tx=0x7ff2a0008040 comp rx=0 tx=0).stop 2023-10-09T06:30:44.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.121+0000 7ff2a98f4700 1 -- 172.21.15.163:0/197361988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2a41597d0 msgr2=0x7ff2a4155d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:44.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.121+0000 7ff2a98f4700 1 --2- 172.21.15.163:0/197361988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2a41597d0 0x7ff2a4155d40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff29c009fa0 tx=0x7ff29c009900 comp rx=0 tx=0).stop 2023-10-09T06:30:44.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.121+0000 7ff2a98f4700 1 -- 172.21.15.163:0/197361988 shutdown_connections 2023-10-09T06:30:44.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.121+0000 7ff2a98f4700 1 --2- 172.21.15.163:0/197361988 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff290066f30 0x7ff2900693e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:44.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.121+0000 7ff2a98f4700 1 --2- 172.21.15.163:0/197361988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2a41597d0 0x7ff2a4155d40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:44.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.121+0000 7ff2a98f4700 1 --2- 172.21.15.163:0/197361988 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2a406f2b0 0x7ff2a4159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:44.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.121+0000 7ff2a98f4700 1 --2- 172.21.15.163:0/197361988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2a40c8400 0x7ff2a4155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:44.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.121+0000 7ff2a98f4700 1 -- 172.21.15.163:0/197361988 >> 172.21.15.163:0/197361988 conn(0x7ff2a406b450 msgr2=0x7ff2a41564b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:44.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.121+0000 7ff2a98f4700 1 -- 172.21.15.163:0/197361988 shutdown_connections 2023-10-09T06:30:44.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.121+0000 7ff2a98f4700 1 -- 172.21.15.163:0/197361988 wait complete. 2023-10-09T06:30:44.134 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:30:44.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.247+0000 7fd8ef474700 1 Processor -- start 2023-10-09T06:30:44.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.248+0000 7fd8ef474700 1 -- start start 2023-10-09T06:30:44.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.248+0000 7fd8ef474700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd8e8159c60 0x7fd8e815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:44.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.248+0000 7fd8ef474700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd8e815db10 0x7fd8e8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:44.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.248+0000 7fd8ef474700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8e815a160 con 0x7fd8e8159890 2023-10-09T06:30:44.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.248+0000 7fd8ef474700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8e815df50 con 0x7fd8e8159c60 2023-10-09T06:30:44.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.248+0000 7fd8ef474700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8e80d13a0 con 0x7fd8e815db10 2023-10-09T06:30:44.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.249+0000 7fd8eca0f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd8e8159c60 0x7fd8e815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:44.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.249+0000 7fd8ed210700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fd8e8159890 0x7fd8e815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:52406/0 (socket says 172.21.15.163:52406) 2023-10-09T06:30:44.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.249+0000 7fd8ed210700 1 -- 172.21.15.163:0/2727609531 learned_addr learned my addr 172.21.15.163:0/2727609531 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:44.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.249+0000 7fd8eda11700 1 --2- 172.21.15.163:0/2727609531 >> v2:172.21.15.181:3300/0 conn(0x7fd8e815db10 0x7fd8e8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:44.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.249+0000 7fd8eca0f700 1 -- 172.21.15.163:0/2727609531 >> v2:172.21.15.181:3300/0 conn(0x7fd8e815db10 msgr2=0x7fd8e8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:44.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.249+0000 7fd8eca0f700 1 --2- 172.21.15.163:0/2727609531 >> v2:172.21.15.181:3300/0 conn(0x7fd8e815db10 0x7fd8e8069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:44.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.249+0000 7fd8eca0f700 1 -- 172.21.15.163:0/2727609531 >> v1:172.21.15.163:6789/0 conn(0x7fd8e8159890 legacy=0x7fd8e815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:44.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.249+0000 7fd8dffff700 1 -- 172.21.15.163:0/2727609531 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3249183681 0 0) 0x7fd8e815a160 con 0x7fd8e8159890 2023-10-09T06:30:44.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.249+0000 7fd8eca0f700 1 -- 172.21.15.163:0/2727609531 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e815e140 con 0x7fd8e8159c60 2023-10-09T06:30:44.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.249+0000 7fd8eca0f700 1 --2- 172.21.15.163:0/2727609531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8e8159c60 0x7fd8e815d5d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd8e806a890 tx=0x7fd8d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713da124c5484019 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:44.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.249+0000 7fd8dffff700 1 -- 172.21.15.163:0/2727609531 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d800b6b0 con 0x7fd8e8159c60 2023-10-09T06:30:44.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.249+0000 7fd8dffff700 1 -- 172.21.15.163:0/2727609531 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d800b810 con 0x7fd8e8159c60 2023-10-09T06:30:44.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.249+0000 7fd8dffff700 1 -- 172.21.15.163:0/2727609531 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d800bbb0 con 0x7fd8e8159c60 2023-10-09T06:30:44.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.250+0000 7fd8ef474700 1 -- 172.21.15.163:0/2727609531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8e8159c60 msgr2=0x7fd8e815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:44.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.250+0000 7fd8ef474700 1 --2- 172.21.15.163:0/2727609531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8e8159c60 0x7fd8e815d5d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd8e806a890 tx=0x7fd8d80091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:44.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.250+0000 7fd8ef474700 1 -- 172.21.15.163:0/2727609531 shutdown_connections 2023-10-09T06:30:44.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.250+0000 7fd8ef474700 1 --2- 172.21.15.163:0/2727609531 >> v2:172.21.15.181:3300/0 conn(0x7fd8e815db10 0x7fd8e8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:44.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.250+0000 7fd8ef474700 1 --2- 172.21.15.163:0/2727609531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8e8159c60 0x7fd8e815d5d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:44.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.250+0000 7fd8ef474700 1 -- 172.21.15.163:0/2727609531 >> 172.21.15.163:0/2727609531 conn(0x7fd8e8068e20 msgr2=0x7fd8e80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:44.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.250+0000 7fd8ef474700 1 -- 172.21.15.163:0/2727609531 shutdown_connections 2023-10-09T06:30:44.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.250+0000 7fd8ef474700 1 -- 172.21.15.163:0/2727609531 wait complete. 2023-10-09T06:30:44.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.251+0000 7fd8ef474700 1 Processor -- start 2023-10-09T06:30:44.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.251+0000 7fd8ef474700 1 -- start start 2023-10-09T06:30:44.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.252+0000 7fd8ef474700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8e8159890 0x7fd8e8151670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:44.256 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.252+0000 7fd8ef474700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8e8159c60 0x7fd8e816dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:44.256 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.252+0000 7fd8ef474700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8e815db10 0x7fd8e814a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:44.256 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.252+0000 7fd8ef474700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e80b1190 con 0x7fd8e8159890 2023-10-09T06:30:44.256 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.252+0000 7fd8ef474700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e816e260 con 0x7fd8e8159c60 2023-10-09T06:30:44.256 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.252+0000 7fd8ef474700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e8151c70 con 0x7fd8e815db10 2023-10-09T06:30:44.256 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.252+0000 7fd8ed210700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8e8159890 0x7fd8e8151670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:44.256 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.252+0000 7fd8ed210700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8e8159890 0x7fd8e8151670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36742/0 (socket says 172.21.15.163:36742) 2023-10-09T06:30:44.256 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.252+0000 7fd8ed210700 1 -- 172.21.15.163:0/3671194019 learned_addr learned my addr 172.21.15.163:0/3671194019 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:44.256 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.252+0000 7fd8eca0f700 1 --2- 172.21.15.163:0/3671194019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8e8159c60 0x7fd8e816dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:44.256 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.252+0000 7fd8eda11700 1 --2- 172.21.15.163:0/3671194019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8e815db10 0x7fd8e814a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:44.256 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.252+0000 7fd8ed210700 1 -- 172.21.15.163:0/3671194019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8e815db10 msgr2=0x7fd8e814a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:44.256 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.252+0000 7fd8ed210700 1 --2- 172.21.15.163:0/3671194019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8e815db10 0x7fd8e814a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:44.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.252+0000 7fd8ed210700 1 -- 172.21.15.163:0/3671194019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8e8159c60 msgr2=0x7fd8e816dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:44.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.252+0000 7fd8ed210700 1 --2- 172.21.15.163:0/3671194019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8e8159c60 0x7fd8e816dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:44.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.252+0000 7fd8ed210700 1 -- 172.21.15.163:0/3671194019 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d8009420 con 0x7fd8e8159890 2023-10-09T06:30:44.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.252+0000 7fd8eca0f700 1 --2- 172.21.15.163:0/3671194019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8e8159c60 0x7fd8e816dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:30:44.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.252+0000 7fd8ed210700 1 --2- 172.21.15.163:0/3671194019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8e8159890 0x7fd8e8151670 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400c7e0 tx=0x7fd8e4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:44.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.252+0000 7fd8dd7fa700 1 -- 172.21.15.163:0/3671194019 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4005b30 con 0x7fd8e8159890 2023-10-09T06:30:44.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.252+0000 7fd8dd7fa700 1 -- 172.21.15.163:0/3671194019 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e4005c90 con 0x7fd8e8159890 2023-10-09T06:30:44.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.252+0000 7fd8ef474700 1 -- 172.21.15.163:0/3671194019 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e8151f50 con 0x7fd8e8159890 2023-10-09T06:30:44.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.252+0000 7fd8dd7fa700 1 -- 172.21.15.163:0/3671194019 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4007300 con 0x7fd8e8159890 2023-10-09T06:30:44.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.252+0000 7fd8ef474700 1 -- 172.21.15.163:0/3671194019 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e8152510 con 0x7fd8e8159890 2023-10-09T06:30:44.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.253+0000 7fd8ef474700 1 -- 172.21.15.163:0/3671194019 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8e804c3c0 con 0x7fd8e8159890 2023-10-09T06:30:44.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.254+0000 7fd8dd7fa700 1 -- 172.21.15.163:0/3671194019 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd8e401b400 con 0x7fd8e8159890 2023-10-09T06:30:44.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.254+0000 7fd8dd7fa700 1 --2- 172.21.15.163:0/3671194019 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd8cc066f30 0x7fd8cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:44.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.254+0000 7fd8dd7fa700 1 -- 172.21.15.163:0/3671194019 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd8e4084090 con 0x7fd8e8159890 2023-10-09T06:30:44.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.254+0000 7fd8eca0f700 1 --2- 172.21.15.163:0/3671194019 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd8cc066f30 0x7fd8cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:44.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.255+0000 7fd8eca0f700 1 --2- 172.21.15.163:0/3671194019 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd8cc066f30 0x7fd8cc0693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd8d80093f0 tx=0x7fd8d8017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:44.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.258+0000 7fd8dd7fa700 1 -- 172.21.15.163:0/3671194019 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd8e4051380 con 0x7fd8e8159890 2023-10-09T06:30:44.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.443+0000 7fd8ef474700 1 -- 172.21.15.163:0/3671194019 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8e80caba0 con 0x7fd8e8159890 2023-10-09T06:30:44.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.444+0000 7fd8dd7fa700 1 -- 172.21.15.163:0/3671194019 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+15411 (secure 0 0 0) 0x7fd8e40549a0 con 0x7fd8e8159890 2023-10-09T06:30:44.445 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:30:44.445 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":19,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:30:43.132806+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:30:41.369014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-09T06:30:42.103001+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:30:44.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.446+0000 7fd8ef474700 1 -- 172.21.15.163:0/3671194019 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd8cc066f30 msgr2=0x7fd8cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:44.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.446+0000 7fd8ef474700 1 --2- 172.21.15.163:0/3671194019 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd8cc066f30 0x7fd8cc0693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd8d80093f0 tx=0x7fd8d8017000 comp rx=0 tx=0).stop 2023-10-09T06:30:44.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.446+0000 7fd8ef474700 1 -- 172.21.15.163:0/3671194019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8e8159890 msgr2=0x7fd8e8151670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:44.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.446+0000 7fd8ef474700 1 --2- 172.21.15.163:0/3671194019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8e8159890 0x7fd8e8151670 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400c7e0 tx=0x7fd8e4007df0 comp rx=0 tx=0).stop 2023-10-09T06:30:44.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.446+0000 7fd8ef474700 1 -- 172.21.15.163:0/3671194019 shutdown_connections 2023-10-09T06:30:44.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.446+0000 7fd8ef474700 1 --2- 172.21.15.163:0/3671194019 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd8cc066f30 0x7fd8cc0693e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:44.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.446+0000 7fd8ef474700 1 --2- 172.21.15.163:0/3671194019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8e8159890 0x7fd8e8151670 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:44.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.446+0000 7fd8ef474700 1 --2- 172.21.15.163:0/3671194019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8e815db10 0x7fd8e814a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:44.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.446+0000 7fd8ef474700 1 --2- 172.21.15.163:0/3671194019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8e8159c60 0x7fd8e816dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:44.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.446+0000 7fd8ef474700 1 -- 172.21.15.163:0/3671194019 >> 172.21.15.163:0/3671194019 conn(0x7fd8e8068e20 msgr2=0x7fd8e814d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:44.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.446+0000 7fd8ef474700 1 -- 172.21.15.163:0/3671194019 shutdown_connections 2023-10-09T06:30:44.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.446+0000 7fd8ef474700 1 -- 172.21.15.163:0/3671194019 wait complete. 2023-10-09T06:30:44.459 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-09T06:30:44.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.575+0000 7fe9b6ca2700 1 Processor -- start 2023-10-09T06:30:44.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.575+0000 7fe9b6ca2700 1 -- start start 2023-10-09T06:30:44.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.576+0000 7fe9b6ca2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe9b00caa90 0x7fe9b00c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:44.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.576+0000 7fe9b6ca2700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b015d7e0 con 0x7fe9b006c0a0 2023-10-09T06:30:44.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.576+0000 7fe9b6ca2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b015dac0 con 0x7fe9b006bcd0 2023-10-09T06:30:44.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.576+0000 7fe9b6ca2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b00caef0 con 0x7fe9b00caa90 2023-10-09T06:30:44.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.576+0000 7fe9a7fff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fe9b006c0a0 0x7fe9b015df50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:52412/0 (socket says 172.21.15.163:52412) 2023-10-09T06:30:44.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.576+0000 7fe9a7fff700 1 -- 172.21.15.163:0/2564814229 learned_addr learned my addr 172.21.15.163:0/2564814229 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:44.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.576+0000 7fe9b523f700 1 --2- 172.21.15.163:0/2564814229 >> v2:172.21.15.181:3300/0 conn(0x7fe9b00caa90 0x7fe9b00c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:44.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.577+0000 7fe9a77fe700 1 -- 172.21.15.163:0/2564814229 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 869355812 0 0) 0x7fe9b015d7e0 con 0x7fe9b006c0a0 2023-10-09T06:30:44.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.577+0000 7fe9a77fe700 1 -- 172.21.15.163:0/2564814229 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe998003570 con 0x7fe9b006c0a0 2023-10-09T06:30:44.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.577+0000 7fe9a77fe700 1 -- 172.21.15.163:0/2564814229 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 240638494 0 0) 0x7fe9b015dac0 con 0x7fe9b006bcd0 2023-10-09T06:30:44.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.577+0000 7fe9a77fe700 1 -- 172.21.15.163:0/2564814229 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9b015d7e0 con 0x7fe9b006bcd0 2023-10-09T06:30:44.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.577+0000 7fe9b523f700 1 -- 172.21.15.163:0/2564814229 >> v1:172.21.15.181:6789/0 conn(0x7fe9b006bcd0 legacy=0x7fe9b015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:44.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.577+0000 7fe9b523f700 1 -- 172.21.15.163:0/2564814229 >> v1:172.21.15.163:6789/0 conn(0x7fe9b006c0a0 legacy=0x7fe9b015df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:44.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.577+0000 7fe9b523f700 1 -- 172.21.15.163:0/2564814229 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b015aa30 con 0x7fe9b00caa90 2023-10-09T06:30:44.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.577+0000 7fe9b523f700 1 --2- 172.21.15.163:0/2564814229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9b00caa90 0x7fe9b00c8c40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe9a80098e0 tx=0x7fe9a80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22081954fb4e77c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:44.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.577+0000 7fe9a77fe700 1 -- 172.21.15.163:0/2564814229 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8011630 con 0x7fe9b00caa90 2023-10-09T06:30:44.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.577+0000 7fe9a77fe700 1 -- 172.21.15.163:0/2564814229 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a8011790 con 0x7fe9b00caa90 2023-10-09T06:30:44.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.577+0000 7fe9a77fe700 1 -- 172.21.15.163:0/2564814229 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8011b30 con 0x7fe9b00caa90 2023-10-09T06:30:44.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.578+0000 7fe9b6ca2700 1 -- 172.21.15.163:0/2564814229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9b00caa90 msgr2=0x7fe9b00c8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:44.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.578+0000 7fe9b6ca2700 1 --2- 172.21.15.163:0/2564814229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9b00caa90 0x7fe9b00c8c40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe9a80098e0 tx=0x7fe9a80091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:44.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.578+0000 7fe9b6ca2700 1 -- 172.21.15.163:0/2564814229 shutdown_connections 2023-10-09T06:30:44.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.578+0000 7fe9b6ca2700 1 --2- 172.21.15.163:0/2564814229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9b00caa90 0x7fe9b00c8c40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:44.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.578+0000 7fe9b6ca2700 1 -- 172.21.15.163:0/2564814229 >> 172.21.15.163:0/2564814229 conn(0x7fe9b00ce600 msgr2=0x7fe9b00cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:44.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.578+0000 7fe9b6ca2700 1 -- 172.21.15.163:0/2564814229 shutdown_connections 2023-10-09T06:30:44.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.578+0000 7fe9b6ca2700 1 -- 172.21.15.163:0/2564814229 wait complete. 2023-10-09T06:30:44.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.579+0000 7fe9b6ca2700 1 Processor -- start 2023-10-09T06:30:44.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.579+0000 7fe9b6ca2700 1 -- start start 2023-10-09T06:30:44.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.580+0000 7fe9b6ca2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9b006bcd0 0x7fe9b0155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:44.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.580+0000 7fe9b6ca2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9b006c0a0 0x7fe9b00b0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:44.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.580+0000 7fe9b6ca2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9b00caa90 0x7fe9b00c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:44.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.580+0000 7fe9b6ca2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b0155950 con 0x7fe9b006c0a0 2023-10-09T06:30:44.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.580+0000 7fe9b6ca2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b00c8200 con 0x7fe9b006bcd0 2023-10-09T06:30:44.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.580+0000 7fe9b6ca2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b00b1910 con 0x7fe9b00caa90 2023-10-09T06:30:44.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.580+0000 7fe9a7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9b006c0a0 0x7fe9b00b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:44.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.580+0000 7fe9a7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9b006c0a0 0x7fe9b00b0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36750/0 (socket says 172.21.15.163:36750) 2023-10-09T06:30:44.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.580+0000 7fe9a7fff700 1 -- 172.21.15.163:0/2782014017 learned_addr learned my addr 172.21.15.163:0/2782014017 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:44.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.580+0000 7fe9b4a3e700 1 --2- 172.21.15.163:0/2782014017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9b006bcd0 0x7fe9b0155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:44.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.580+0000 7fe9b523f700 1 --2- 172.21.15.163:0/2782014017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9b00caa90 0x7fe9b00c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:44.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.580+0000 7fe9a7fff700 1 -- 172.21.15.163:0/2782014017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9b00caa90 msgr2=0x7fe9b00c46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:44.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.580+0000 7fe9a7fff700 1 --2- 172.21.15.163:0/2782014017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9b00caa90 0x7fe9b00c46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:44.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.580+0000 7fe9a7fff700 1 -- 172.21.15.163:0/2782014017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9b006bcd0 msgr2=0x7fe9b0155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:44.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.580+0000 7fe9a7fff700 1 --2- 172.21.15.163:0/2782014017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9b006bcd0 0x7fe9b0155410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:44.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.580+0000 7fe9a7fff700 1 -- 172.21.15.163:0/2782014017 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a8009420 con 0x7fe9b006c0a0 2023-10-09T06:30:44.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.580+0000 7fe9b4a3e700 1 --2- 172.21.15.163:0/2782014017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9b006bcd0 0x7fe9b0155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:30:44.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.580+0000 7fe9b523f700 1 --2- 172.21.15.163:0/2782014017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9b00caa90 0x7fe9b00c46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:30:44.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.580+0000 7fe9a7fff700 1 --2- 172.21.15.163:0/2782014017 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9b006c0a0 0x7fe9b00b0110 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000c790 tx=0x7fe9a0007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:44.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.580+0000 7fe9a4ff9700 1 -- 172.21.15.163:0/2782014017 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0005b30 con 0x7fe9b006c0a0 2023-10-09T06:30:44.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.580+0000 7fe9a4ff9700 1 -- 172.21.15.163:0/2782014017 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a0005c90 con 0x7fe9b006c0a0 2023-10-09T06:30:44.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.580+0000 7fe9b6ca2700 1 -- 172.21.15.163:0/2782014017 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b00b1bf0 con 0x7fe9b006c0a0 2023-10-09T06:30:44.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.580+0000 7fe9a4ff9700 1 -- 172.21.15.163:0/2782014017 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0007300 con 0x7fe9b006c0a0 2023-10-09T06:30:44.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.580+0000 7fe9b6ca2700 1 -- 172.21.15.163:0/2782014017 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b00b21b0 con 0x7fe9b006c0a0 2023-10-09T06:30:44.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.581+0000 7fe9b6ca2700 1 -- 172.21.15.163:0/2782014017 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9b00c83c0 con 0x7fe9b006c0a0 2023-10-09T06:30:44.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.582+0000 7fe9a4ff9700 1 -- 172.21.15.163:0/2782014017 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe9a001b3b0 con 0x7fe9b006c0a0 2023-10-09T06:30:44.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.582+0000 7fe9a4ff9700 1 --2- 172.21.15.163:0/2782014017 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe99c066f30 0x7fe99c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:44.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.582+0000 7fe9a4ff9700 1 -- 172.21.15.163:0/2782014017 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe9a0083b60 con 0x7fe9b006c0a0 2023-10-09T06:30:44.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.582+0000 7fe9b4a3e700 1 --2- 172.21.15.163:0/2782014017 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe99c066f30 0x7fe99c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:44.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.583+0000 7fe9b4a3e700 1 --2- 172.21.15.163:0/2782014017 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe99c066f30 0x7fe99c0693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac003fa0 tx=0x7fe9ac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:44.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.586+0000 7fe9a4ff9700 1 -- 172.21.15.163:0/2782014017 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe9a0050dd0 con 0x7fe9b006c0a0 2023-10-09T06:30:44.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:44.811+0000 7fe9b6ca2700 1 -- 172.21.15.163:0/2782014017 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe9b004c3c0 con 0x7fe9b006c0a0 2023-10-09T06:30:45.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.760+0000 7fe9a4ff9700 1 -- 172.21.15.163:0/2782014017 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe9a0017070 con 0x7fe9b006c0a0 2023-10-09T06:30:45.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.762+0000 7fe9b6ca2700 1 -- 172.21.15.163:0/2782014017 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe99c066f30 msgr2=0x7fe99c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:45.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.762+0000 7fe9b6ca2700 1 --2- 172.21.15.163:0/2782014017 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe99c066f30 0x7fe99c0693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac003fa0 tx=0x7fe9ac009040 comp rx=0 tx=0).stop 2023-10-09T06:30:45.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.762+0000 7fe9b6ca2700 1 -- 172.21.15.163:0/2782014017 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9b006c0a0 msgr2=0x7fe9b00b0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:45.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.762+0000 7fe9b6ca2700 1 --2- 172.21.15.163:0/2782014017 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9b006c0a0 0x7fe9b00b0110 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000c790 tx=0x7fe9a0007ea0 comp rx=0 tx=0).stop 2023-10-09T06:30:45.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.762+0000 7fe9b6ca2700 1 -- 172.21.15.163:0/2782014017 shutdown_connections 2023-10-09T06:30:45.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.762+0000 7fe9b6ca2700 1 --2- 172.21.15.163:0/2782014017 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe99c066f30 0x7fe99c0693e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:45.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.762+0000 7fe9b6ca2700 1 --2- 172.21.15.163:0/2782014017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9b00caa90 0x7fe9b00c46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:45.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.762+0000 7fe9b6ca2700 1 --2- 172.21.15.163:0/2782014017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9b006bcd0 0x7fe9b0155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:45.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.762+0000 7fe9b6ca2700 1 --2- 172.21.15.163:0/2782014017 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9b006c0a0 0x7fe9b00b0110 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:45.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.762+0000 7fe9b6ca2700 1 -- 172.21.15.163:0/2782014017 >> 172.21.15.163:0/2782014017 conn(0x7fe9b00ce600 msgr2=0x7fe9b0069480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:45.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.762+0000 7fe9b6ca2700 1 -- 172.21.15.163:0/2782014017 shutdown_connections 2023-10-09T06:30:45.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.762+0000 7fe9b6ca2700 1 -- 172.21.15.163:0/2782014017 wait complete. 2023-10-09T06:30:45.777 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-09T06:30:45.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.895+0000 7f017d8ff700 1 Processor -- start 2023-10-09T06:30:45.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.895+0000 7f017d8ff700 1 -- start start 2023-10-09T06:30:45.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.896+0000 7f017d8ff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f017806b520 0x7f01780cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:45.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.896+0000 7f017d8ff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01780cc210 con 0x7f017806aa40 2023-10-09T06:30:45.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.896+0000 7f017d8ff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01780cc4f0 con 0x7f01780c91e0 2023-10-09T06:30:45.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.896+0000 7f017d8ff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01780cc7d0 con 0x7f017806b520 2023-10-09T06:30:45.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.896+0000 7f0176ffd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f017806aa40 0x7f017806bac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:52428/0 (socket says 172.21.15.163:52428) 2023-10-09T06:30:45.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.896+0000 7f0176ffd700 1 -- 172.21.15.163:0/1449088418 learned_addr learned my addr 172.21.15.163:0/1449088418 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:45.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.897+0000 7f01767fc700 1 -- 172.21.15.163:0/1449088418 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1831632806 0 0) 0x7f01780cc210 con 0x7f017806aa40 2023-10-09T06:30:45.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.897+0000 7f01767fc700 1 -- 172.21.15.163:0/1449088418 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0164003570 con 0x7f017806aa40 2023-10-09T06:30:45.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.897+0000 7f01777fe700 1 --2- 172.21.15.163:0/1449088418 >> v2:172.21.15.181:3300/0 conn(0x7f017806b520 0x7f01780cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:45.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.897+0000 7f01767fc700 1 -- 172.21.15.163:0/1449088418 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4034300501 0 0) 0x7f0164003570 con 0x7f017806aa40 2023-10-09T06:30:45.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.897+0000 7f01767fc700 1 -- 172.21.15.163:0/1449088418 >> v2:172.21.15.181:3300/0 conn(0x7f017806b520 msgr2=0x7f01780cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:45.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.897+0000 7f01767fc700 1 --2- 172.21.15.163:0/1449088418 >> v2:172.21.15.181:3300/0 conn(0x7f017806b520 0x7f01780cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:45.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.897+0000 7f01767fc700 1 -- 172.21.15.163:0/1449088418 >> v1:172.21.15.181:6789/0 conn(0x7f01780c91e0 legacy=0x7f017806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:45.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.897+0000 7f01767fc700 1 -- 172.21.15.163:0/1449088418 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01780ccab0 con 0x7f017806aa40 2023-10-09T06:30:45.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.897+0000 7f01767fc700 1 -- 172.21.15.163:0/1449088418 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f01680015f0 con 0x7f017806aa40 2023-10-09T06:30:45.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.897+0000 7f01767fc700 1 -- 172.21.15.163:0/1449088418 >> v1:172.21.15.163:6789/0 conn(0x7f017806aa40 legacy=0x7f017806bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:45.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.897+0000 7f01767fc700 1 --2- 172.21.15.163:0/1449088418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0164003a60 0x7f0164003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:45.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.897+0000 7f01767fc700 1 --2- 172.21.15.163:0/1449088418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0164004810 0x7f0164004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:45.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.897+0000 7f01767fc700 1 --2- 172.21.15.163:0/1449088418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01640052e0 0x7f016400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:45.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.897+0000 7f01767fc700 1 -- 172.21.15.163:0/1449088418 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01780cc7d0 con 0x7f0164003a60 2023-10-09T06:30:45.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.897+0000 7f01767fc700 1 -- 172.21.15.163:0/1449088418 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016400c190 con 0x7f01640052e0 2023-10-09T06:30:45.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.897+0000 7f01767fc700 1 -- 172.21.15.163:0/1449088418 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016400c350 con 0x7f0164004810 2023-10-09T06:30:45.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.897+0000 7f0176ffd700 1 --2- 172.21.15.163:0/1449088418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0164003a60 0x7f0164003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:45.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.897+0000 7f016e7fc700 1 --2- 172.21.15.163:0/1449088418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0164004810 0x7f0164004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:45.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.897+0000 7f01777fe700 1 --2- 172.21.15.163:0/1449088418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01640052e0 0x7f016400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:45.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.897+0000 7f0176ffd700 1 -- 172.21.15.163:0/1449088418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0164004810 msgr2=0x7f0164004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:45.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.897+0000 7f0176ffd700 1 --2- 172.21.15.163:0/1449088418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0164004810 0x7f0164004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:45.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.897+0000 7f0176ffd700 1 -- 172.21.15.163:0/1449088418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01640052e0 msgr2=0x7f016400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:45.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.897+0000 7f0176ffd700 1 --2- 172.21.15.163:0/1449088418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01640052e0 0x7f016400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:45.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.897+0000 7f01777fe700 1 -- 172.21.15.163:0/1449088418 reap_dead start 2023-10-09T06:30:45.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.897+0000 7f0176ffd700 1 -- 172.21.15.163:0/1449088418 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0164003570 con 0x7f0164003a60 2023-10-09T06:30:45.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.897+0000 7f0176ffd700 1 --2- 172.21.15.163:0/1449088418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0164003a60 0x7f0164003e70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f016800c3c0 tx=0x7f0168007920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a6b3903e133c6a server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:45.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.897+0000 7f01767fc700 1 -- 172.21.15.163:0/1449088418 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016800cc90 con 0x7f0164003a60 2023-10-09T06:30:45.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.898+0000 7f01767fc700 1 -- 172.21.15.163:0/1449088418 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f016800ce10 con 0x7f0164003a60 2023-10-09T06:30:45.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.898+0000 7f01767fc700 1 -- 172.21.15.163:0/1449088418 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0168007320 con 0x7f0164003a60 2023-10-09T06:30:45.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.898+0000 7f017d8ff700 1 -- 172.21.15.163:0/1449088418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0164003a60 msgr2=0x7f0164003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:45.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.898+0000 7f017d8ff700 1 --2- 172.21.15.163:0/1449088418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0164003a60 0x7f0164003e70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f016800c3c0 tx=0x7f0168007920 comp rx=0 tx=0).stop 2023-10-09T06:30:45.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.898+0000 7f017d8ff700 1 -- 172.21.15.163:0/1449088418 shutdown_connections 2023-10-09T06:30:45.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.898+0000 7f017d8ff700 1 --2- 172.21.15.163:0/1449088418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0164003a60 0x7f0164003e70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:45.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.898+0000 7f017d8ff700 1 -- 172.21.15.163:0/1449088418 >> 172.21.15.163:0/1449088418 conn(0x7f01780cd670 msgr2=0x7f01780cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:45.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.898+0000 7f017d8ff700 1 -- 172.21.15.163:0/1449088418 shutdown_connections 2023-10-09T06:30:45.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.898+0000 7f017d8ff700 1 -- 172.21.15.163:0/1449088418 wait complete. 2023-10-09T06:30:45.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.899+0000 7f017d8ff700 1 Processor -- start 2023-10-09T06:30:45.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.899+0000 7f017d8ff700 1 -- start start 2023-10-09T06:30:45.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.900+0000 7f017d8ff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0164003a60 0x7f01780b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:45.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.900+0000 7f017d8ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0178149ad0 0x7f0178147090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:45.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.900+0000 7f017d8ff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01780b1b20 0x7f01780cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:45.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.900+0000 7f017d8ff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01780b1690 con 0x7f0178149ad0 2023-10-09T06:30:45.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.900+0000 7f017d8ff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01780b0a80 con 0x7f01780b1b20 2023-10-09T06:30:45.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.900+0000 7f017d8ff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01780b0d60 con 0x7f0164003a60 2023-10-09T06:30:45.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.900+0000 7f016e7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0178149ad0 0x7f0178147090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:45.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.900+0000 7f016e7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0178149ad0 0x7f0178147090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36764/0 (socket says 172.21.15.163:36764) 2023-10-09T06:30:45.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.900+0000 7f016e7fc700 1 -- 172.21.15.163:0/3136980429 learned_addr learned my addr 172.21.15.163:0/3136980429 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:45.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.900+0000 7f0176ffd700 1 --2- 172.21.15.163:0/3136980429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0164003a60 0x7f01780b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:45.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.900+0000 7f01777fe700 1 --2- 172.21.15.163:0/3136980429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01780b1b20 0x7f01780cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:45.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.900+0000 7f016e7fc700 1 -- 172.21.15.163:0/3136980429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0164003a60 msgr2=0x7f01780b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:45.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.900+0000 7f016e7fc700 1 --2- 172.21.15.163:0/3136980429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0164003a60 0x7f01780b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:45.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.900+0000 7f016e7fc700 1 -- 172.21.15.163:0/3136980429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01780b1b20 msgr2=0x7f01780cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:45.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.900+0000 7f016e7fc700 1 --2- 172.21.15.163:0/3136980429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01780b1b20 0x7f01780cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:45.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.900+0000 7f016e7fc700 1 -- 172.21.15.163:0/3136980429 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01780b1040 con 0x7f0178149ad0 2023-10-09T06:30:45.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.901+0000 7f016e7fc700 1 --2- 172.21.15.163:0/3136980429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0178149ad0 0x7f0178147090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f016000e5e0 tx=0x7f0160009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:45.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.901+0000 7f016f7fe700 1 -- 172.21.15.163:0/3136980429 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0160012590 con 0x7f0178149ad0 2023-10-09T06:30:45.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.901+0000 7f016f7fe700 1 -- 172.21.15.163:0/3136980429 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01600126f0 con 0x7f0178149ad0 2023-10-09T06:30:45.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.901+0000 7f016f7fe700 1 -- 172.21.15.163:0/3136980429 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0160012a90 con 0x7f0178149ad0 2023-10-09T06:30:45.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.901+0000 7f017d8ff700 1 -- 172.21.15.163:0/3136980429 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01780b1320 con 0x7f0178149ad0 2023-10-09T06:30:45.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.901+0000 7f017d8ff700 1 -- 172.21.15.163:0/3136980429 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01781475d0 con 0x7f0178149ad0 2023-10-09T06:30:45.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.902+0000 7f017d8ff700 1 -- 172.21.15.163:0/3136980429 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01780b1840 con 0x7f0178149ad0 2023-10-09T06:30:45.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.902+0000 7f016f7fe700 1 -- 172.21.15.163:0/3136980429 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0160013420 con 0x7f0178149ad0 2023-10-09T06:30:45.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.902+0000 7f016f7fe700 1 --2- 172.21.15.163:0/3136980429 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0158066f30 0x7f01580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:45.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.903+0000 7f016f7fe700 1 -- 172.21.15.163:0/3136980429 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0160084420 con 0x7f0178149ad0 2023-10-09T06:30:45.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.903+0000 7f0176ffd700 1 --2- 172.21.15.163:0/3136980429 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0158066f30 0x7f01580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:45.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.903+0000 7f0176ffd700 1 --2- 172.21.15.163:0/3136980429 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0158066f30 0x7f01580693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0168008fd0 tx=0x7f0168015000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:45.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:45.907+0000 7f016f7fe700 1 -- 172.21.15.163:0/3136980429 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01780b1840 con 0x7f0178149ad0 2023-10-09T06:30:46.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.132+0000 7f017d8ff700 1 -- 172.21.15.163:0/3136980429 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f017804c3c0 con 0x7f0178149ad0 2023-10-09T06:30:46.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.767+0000 7f016f7fe700 1 -- 172.21.15.163:0/3136980429 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0160051690 con 0x7f0178149ad0 2023-10-09T06:30:46.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.769+0000 7f017d8ff700 1 -- 172.21.15.163:0/3136980429 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0158066f30 msgr2=0x7f01580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:46.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.769+0000 7f017d8ff700 1 --2- 172.21.15.163:0/3136980429 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0158066f30 0x7f01580693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0168008fd0 tx=0x7f0168015000 comp rx=0 tx=0).stop 2023-10-09T06:30:46.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.769+0000 7f017d8ff700 1 -- 172.21.15.163:0/3136980429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0178149ad0 msgr2=0x7f0178147090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:46.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.769+0000 7f017d8ff700 1 --2- 172.21.15.163:0/3136980429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0178149ad0 0x7f0178147090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f016000e5e0 tx=0x7f0160009040 comp rx=0 tx=0).stop 2023-10-09T06:30:46.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.769+0000 7f017d8ff700 1 -- 172.21.15.163:0/3136980429 shutdown_connections 2023-10-09T06:30:46.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.769+0000 7f017d8ff700 1 --2- 172.21.15.163:0/3136980429 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0158066f30 0x7f01580693e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:46.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.769+0000 7f017d8ff700 1 --2- 172.21.15.163:0/3136980429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01780b1b20 0x7f01780cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:46.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.769+0000 7f017d8ff700 1 --2- 172.21.15.163:0/3136980429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0164003a60 0x7f01780b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:46.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.769+0000 7f017d8ff700 1 --2- 172.21.15.163:0/3136980429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0178149ad0 0x7f0178147090 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:46.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.769+0000 7f017d8ff700 1 -- 172.21.15.163:0/3136980429 >> 172.21.15.163:0/3136980429 conn(0x7f01780cd670 msgr2=0x7f0178068ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:46.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.770+0000 7f017d8ff700 1 -- 172.21.15.163:0/3136980429 shutdown_connections 2023-10-09T06:30:46.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.770+0000 7f017d8ff700 1 -- 172.21.15.163:0/3136980429 wait complete. 2023-10-09T06:30:46.783 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:30:46.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.898+0000 7f53e88a5700 1 Processor -- start 2023-10-09T06:30:46.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.898+0000 7f53e88a5700 1 -- start start 2023-10-09T06:30:46.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.899+0000 7f53e88a5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f53e4159420 0x7f53e40b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:46.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.899+0000 7f53e88a5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53e41597f0 0x7f53e40b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:46.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.899+0000 7f53e88a5700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e40b11d0 con 0x7f53e415d230 2023-10-09T06:30:46.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.899+0000 7f53e88a5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e40b14b0 con 0x7f53e4159420 2023-10-09T06:30:46.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.899+0000 7f53e88a5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e40b1790 con 0x7f53e41597f0 2023-10-09T06:30:46.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.899+0000 7f53e359e700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f53e415d230 0x7f53e40b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:52430/0 (socket says 172.21.15.163:52430) 2023-10-09T06:30:46.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.899+0000 7f53e359e700 1 -- 172.21.15.163:0/2709143789 learned_addr learned my addr 172.21.15.163:0/2709143789 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:46.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.899+0000 7f53e2d9d700 1 --2- 172.21.15.163:0/2709143789 >> v2:172.21.15.181:3300/0 conn(0x7f53e4159420 0x7f53e40b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:46.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.899+0000 7f53dbfff700 1 --2- 172.21.15.163:0/2709143789 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53e41597f0 0x7f53e40b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:46.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.899+0000 7f53e259c700 1 -- 172.21.15.163:0/2709143789 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 541176833 0 0) 0x7f53e40b11d0 con 0x7f53e415d230 2023-10-09T06:30:46.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.899+0000 7f53e259c700 1 -- 172.21.15.163:0/2709143789 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53d0003570 con 0x7f53e415d230 2023-10-09T06:30:46.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.899+0000 7f53e259c700 1 -- 172.21.15.163:0/2709143789 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2493946617 0 0) 0x7f53d0003570 con 0x7f53e415d230 2023-10-09T06:30:46.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.899+0000 7f53e259c700 1 -- 172.21.15.163:0/2709143789 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53e41597f0 msgr2=0x7f53e40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:46.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.899+0000 7f53e259c700 1 --2- 172.21.15.163:0/2709143789 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53e41597f0 0x7f53e40b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:46.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.899+0000 7f53e259c700 1 -- 172.21.15.163:0/2709143789 >> v2:172.21.15.181:3300/0 conn(0x7f53e4159420 msgr2=0x7f53e40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:46.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.899+0000 7f53e259c700 1 --2- 172.21.15.163:0/2709143789 >> v2:172.21.15.181:3300/0 conn(0x7f53e4159420 0x7f53e40b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:46.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.899+0000 7f53e259c700 1 -- 172.21.15.163:0/2709143789 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e40b1a70 con 0x7f53e415d230 2023-10-09T06:30:46.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.899+0000 7f53e259c700 1 -- 172.21.15.163:0/2709143789 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f53dc0017f0 con 0x7f53e415d230 2023-10-09T06:30:46.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.900+0000 7f53e259c700 1 -- 172.21.15.163:0/2709143789 >> v1:172.21.15.163:6789/0 conn(0x7f53e415d230 legacy=0x7f53e40b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:46.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.900+0000 7f53e259c700 1 --2- 172.21.15.163:0/2709143789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53d0003a60 0x7f53d0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:46.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.900+0000 7f53e259c700 1 --2- 172.21.15.163:0/2709143789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53d0004810 0x7f53d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:46.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.900+0000 7f53e259c700 1 --2- 172.21.15.163:0/2709143789 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53e41597f0 0x7f53d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:46.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.900+0000 7f53e259c700 1 -- 172.21.15.163:0/2709143789 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e40b14b0 con 0x7f53d0004810 2023-10-09T06:30:46.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.900+0000 7f53e259c700 1 -- 172.21.15.163:0/2709143789 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e40b1790 con 0x7f53d0003a60 2023-10-09T06:30:46.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.900+0000 7f53e259c700 1 -- 172.21.15.163:0/2709143789 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d000ba60 con 0x7f53e41597f0 2023-10-09T06:30:46.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.900+0000 7f53dbfff700 1 --2- 172.21.15.163:0/2709143789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53d0004810 0x7f53d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:46.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.900+0000 7f53e2d9d700 1 --2- 172.21.15.163:0/2709143789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53d0003a60 0x7f53d0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:46.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.900+0000 7f53e359e700 1 --2- 172.21.15.163:0/2709143789 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53e41597f0 0x7f53d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:46.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.900+0000 7f53dbfff700 1 -- 172.21.15.163:0/2709143789 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53e41597f0 msgr2=0x7f53d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:46.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.900+0000 7f53dbfff700 1 --2- 172.21.15.163:0/2709143789 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53e41597f0 0x7f53d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:46.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.900+0000 7f53dbfff700 1 -- 172.21.15.163:0/2709143789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53d0003a60 msgr2=0x7f53d0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:46.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.900+0000 7f53dbfff700 1 --2- 172.21.15.163:0/2709143789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53d0003a60 0x7f53d0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:46.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.900+0000 7f53dbfff700 1 -- 172.21.15.163:0/2709143789 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d0003570 con 0x7f53d0004810 2023-10-09T06:30:46.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.900+0000 7f53dbfff700 1 --2- 172.21.15.163:0/2709143789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53d0004810 0x7f53d0004c80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f53cc009c30 tx=0x7f53cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2bf967f1568386f server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:46.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.900+0000 7f53e259c700 1 -- 172.21.15.163:0/2709143789 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc00e710 con 0x7f53d0004810 2023-10-09T06:30:46.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.900+0000 7f53e259c700 1 -- 172.21.15.163:0/2709143789 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53cc00e870 con 0x7f53d0004810 2023-10-09T06:30:46.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.900+0000 7f53e259c700 1 -- 172.21.15.163:0/2709143789 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc00ec30 con 0x7f53d0004810 2023-10-09T06:30:46.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.901+0000 7f53e88a5700 1 -- 172.21.15.163:0/2709143789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53d0004810 msgr2=0x7f53d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:46.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.901+0000 7f53e88a5700 1 --2- 172.21.15.163:0/2709143789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53d0004810 0x7f53d0004c80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f53cc009c30 tx=0x7f53cc0091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:46.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.901+0000 7f53e359e700 1 -- 172.21.15.163:0/2709143789 reap_dead start 2023-10-09T06:30:46.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.901+0000 7f53e88a5700 1 -- 172.21.15.163:0/2709143789 shutdown_connections 2023-10-09T06:30:46.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.901+0000 7f53e88a5700 1 -- 172.21.15.163:0/2709143789 >> 172.21.15.163:0/2709143789 conn(0x7f53e40d0660 msgr2=0x7f53e40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:46.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.901+0000 7f53e88a5700 1 -- 172.21.15.163:0/2709143789 shutdown_connections 2023-10-09T06:30:46.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.901+0000 7f53e88a5700 1 -- 172.21.15.163:0/2709143789 wait complete. 2023-10-09T06:30:46.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.902+0000 7f53e88a5700 1 Processor -- start 2023-10-09T06:30:46.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.902+0000 7f53e88a5700 1 -- start start 2023-10-09T06:30:46.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.902+0000 7f53e88a5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53d0004810 0x7f53e415b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:46.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.902+0000 7f53e88a5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53e414ffc0 0x7f53e40b5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:46.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.903+0000 7f53e88a5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53e4150430 0x7f53e40b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:46.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.903+0000 7f53e88a5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e4151d70 con 0x7f53d0004810 2023-10-09T06:30:46.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.903+0000 7f53e88a5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e415c4c0 con 0x7f53e414ffc0 2023-10-09T06:30:46.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.903+0000 7f53e88a5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e4151160 con 0x7f53e4150430 2023-10-09T06:30:46.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.903+0000 7f53e2d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53d0004810 0x7f53e415b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:46.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.903+0000 7f53e2d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53d0004810 0x7f53e415b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36790/0 (socket says 172.21.15.163:36790) 2023-10-09T06:30:46.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.903+0000 7f53e2d9d700 1 -- 172.21.15.163:0/1154480397 learned_addr learned my addr 172.21.15.163:0/1154480397 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:46.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.903+0000 7f53e359e700 1 --2- 172.21.15.163:0/1154480397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53e4150430 0x7f53e40b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:46.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.903+0000 7f53dbfff700 1 --2- 172.21.15.163:0/1154480397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53e414ffc0 0x7f53e40b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:46.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.903+0000 7f53e2d9d700 1 -- 172.21.15.163:0/1154480397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53e4150430 msgr2=0x7f53e40b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:46.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.903+0000 7f53e2d9d700 1 --2- 172.21.15.163:0/1154480397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53e4150430 0x7f53e40b5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:46.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.903+0000 7f53e2d9d700 1 -- 172.21.15.163:0/1154480397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53e414ffc0 msgr2=0x7f53e40b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:46.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.903+0000 7f53e2d9d700 1 --2- 172.21.15.163:0/1154480397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53e414ffc0 0x7f53e40b5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:46.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.903+0000 7f53e2d9d700 1 -- 172.21.15.163:0/1154480397 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e4151440 con 0x7f53d0004810 2023-10-09T06:30:46.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.903+0000 7f53e359e700 1 --2- 172.21.15.163:0/1154480397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53e4150430 0x7f53e40b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:30:46.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.903+0000 7f53dbfff700 1 --2- 172.21.15.163:0/1154480397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53e414ffc0 0x7f53e40b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:30:46.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.903+0000 7f53e2d9d700 1 --2- 172.21.15.163:0/1154480397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53d0004810 0x7f53e415b5a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f53d4011d40 tx=0x7f53d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:46.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.903+0000 7f53daffd700 1 -- 172.21.15.163:0/1154480397 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d400fac0 con 0x7f53d0004810 2023-10-09T06:30:46.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.903+0000 7f53daffd700 1 -- 172.21.15.163:0/1154480397 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d400fc20 con 0x7f53d0004810 2023-10-09T06:30:46.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.903+0000 7f53e88a5700 1 -- 172.21.15.163:0/1154480397 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e4151720 con 0x7f53d0004810 2023-10-09T06:30:46.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.903+0000 7f53daffd700 1 -- 172.21.15.163:0/1154480397 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d4014440 con 0x7f53d0004810 2023-10-09T06:30:46.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.903+0000 7f53e88a5700 1 -- 172.21.15.163:0/1154480397 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e415bae0 con 0x7f53d0004810 2023-10-09T06:30:46.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.904+0000 7f53e88a5700 1 -- 172.21.15.163:0/1154480397 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e4152050 con 0x7f53d0004810 2023-10-09T06:30:46.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.904+0000 7f53daffd700 1 -- 172.21.15.163:0/1154480397 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f53d4014a80 con 0x7f53d0004810 2023-10-09T06:30:46.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.905+0000 7f53daffd700 1 --2- 172.21.15.163:0/1154480397 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f53c4066f30 0x7f53c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:46.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.905+0000 7f53daffd700 1 -- 172.21.15.163:0/1154480397 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f53d4018030 con 0x7f53d0004810 2023-10-09T06:30:46.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.905+0000 7f53dbfff700 1 --2- 172.21.15.163:0/1154480397 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f53c4066f30 0x7f53c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:46.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.906+0000 7f53dbfff700 1 --2- 172.21.15.163:0/1154480397 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f53c4066f30 0x7f53c40693e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0091e0 tx=0x7f53cc01a000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:46.919 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:46.909+0000 7f53daffd700 1 -- 172.21.15.163:0/1154480397 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f53d4052500 con 0x7f53d0004810 2023-10-09T06:30:47.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.131+0000 7f53e88a5700 1 -- 172.21.15.163:0/1154480397 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53e404c3c0 con 0x7f53d0004810 2023-10-09T06:30:47.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.132+0000 7f53daffd700 1 -- 172.21.15.163:0/1154480397 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4603 (secure 0 0 0) 0x7f53d4055b20 con 0x7f53d0004810 2023-10-09T06:30:47.133 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:30:47.133 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3440080308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3440080308},{"type":"v1","addr":"172.21.15.181:6837","nonce":3440080308}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1990962358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1990962358},{"type":"v1","addr":"172.21.15.163:6835","nonce":1990962358}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2270076605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2270076605},{"type":"v1","addr":"172.21.15.163:6834","nonce":2270076605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:30:43.133124+0000","modified":"2023-10-09T06:30:46.760523+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/1149296546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1149296546},{"type":"v1","addr":"172.21.15.181:6836","nonce":1149296546}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-09T06:30:47.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.134+0000 7f53e88a5700 1 -- 172.21.15.163:0/1154480397 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f53c4066f30 msgr2=0x7f53c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:47.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.134+0000 7f53e88a5700 1 --2- 172.21.15.163:0/1154480397 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f53c4066f30 0x7f53c40693e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0091e0 tx=0x7f53cc01a000 comp rx=0 tx=0).stop 2023-10-09T06:30:47.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.134+0000 7f53e88a5700 1 -- 172.21.15.163:0/1154480397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53d0004810 msgr2=0x7f53e415b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:47.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.134+0000 7f53e88a5700 1 --2- 172.21.15.163:0/1154480397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53d0004810 0x7f53e415b5a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f53d4011d40 tx=0x7f53d40091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:47.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.134+0000 7f53e88a5700 1 -- 172.21.15.163:0/1154480397 shutdown_connections 2023-10-09T06:30:47.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.134+0000 7f53e88a5700 1 --2- 172.21.15.163:0/1154480397 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f53c4066f30 0x7f53c40693e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:47.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.134+0000 7f53e88a5700 1 --2- 172.21.15.163:0/1154480397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53d0004810 0x7f53e415b5a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:47.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.134+0000 7f53e88a5700 1 --2- 172.21.15.163:0/1154480397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53e4150430 0x7f53e40b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:47.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.134+0000 7f53e88a5700 1 --2- 172.21.15.163:0/1154480397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53e414ffc0 0x7f53e40b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:47.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.134+0000 7f53e88a5700 1 -- 172.21.15.163:0/1154480397 >> 172.21.15.163:0/1154480397 conn(0x7f53e40d0660 msgr2=0x7f53e40b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:47.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.134+0000 7f53e88a5700 1 -- 172.21.15.163:0/1154480397 shutdown_connections 2023-10-09T06:30:47.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.134+0000 7f53e88a5700 1 -- 172.21.15.163:0/1154480397 wait complete. 2023-10-09T06:30:47.137 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 8 2023-10-09T06:30:47.147 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:30:47.261 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.260+0000 7f11944f4700 1 Processor -- start 2023-10-09T06:30:47.261 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.260+0000 7f11944f4700 1 -- start start 2023-10-09T06:30:47.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.261+0000 7f11944f4700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f118c159c60 0x7f118c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:47.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.261+0000 7f11944f4700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f118c15a160 con 0x7f118c159890 2023-10-09T06:30:47.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.261+0000 7f11944f4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f118c0d13a0 con 0x7f118c15db10 2023-10-09T06:30:47.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.261+0000 7f11944f4700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f118c15df50 con 0x7f118c159c60 2023-10-09T06:30:47.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.261+0000 7f1192290700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f118c159890 0x7f118c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:52432/0 (socket says 172.21.15.163:52432) 2023-10-09T06:30:47.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.261+0000 7f1192290700 1 -- 172.21.15.163:0/1088246592 learned_addr learned my addr 172.21.15.163:0/1088246592 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:47.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.261+0000 7f1191a8f700 1 --2- 172.21.15.163:0/1088246592 >> v2:172.21.15.163:3300/0 conn(0x7f118c159c60 0x7f118c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:47.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.261+0000 7f119128e700 1 -- 172.21.15.163:0/1088246592 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3669911791 0 0) 0x7f118c15a160 con 0x7f118c159890 2023-10-09T06:30:47.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.262+0000 7f119128e700 1 -- 172.21.15.163:0/1088246592 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1178003570 con 0x7f118c159890 2023-10-09T06:30:47.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.262+0000 7f1191a8f700 1 -- 172.21.15.163:0/1088246592 >> v1:172.21.15.181:6789/0 conn(0x7f118c15db10 legacy=0x7f118c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:47.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.262+0000 7f1191a8f700 1 -- 172.21.15.163:0/1088246592 >> v1:172.21.15.163:6789/0 conn(0x7f118c159890 legacy=0x7f118c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:47.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.262+0000 7f1191a8f700 1 -- 172.21.15.163:0/1088246592 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f118c15e140 con 0x7f118c159c60 2023-10-09T06:30:47.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.262+0000 7f1191a8f700 1 --2- 172.21.15.163:0/1088246592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f118c159c60 0x7f118c15d5d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f117c0098e0 tx=0x7f117c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28674948085bb38e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:47.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.262+0000 7f119128e700 1 -- 172.21.15.163:0/1088246592 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c00b640 con 0x7f118c159c60 2023-10-09T06:30:47.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.262+0000 7f119128e700 1 -- 172.21.15.163:0/1088246592 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117c00b7a0 con 0x7f118c159c60 2023-10-09T06:30:47.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.262+0000 7f119128e700 1 -- 172.21.15.163:0/1088246592 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c00bb40 con 0x7f118c159c60 2023-10-09T06:30:47.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.262+0000 7f11944f4700 1 -- 172.21.15.163:0/1088246592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f118c159c60 msgr2=0x7f118c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:47.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.262+0000 7f11944f4700 1 --2- 172.21.15.163:0/1088246592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f118c159c60 0x7f118c15d5d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f117c0098e0 tx=0x7f117c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:47.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.262+0000 7f11944f4700 1 -- 172.21.15.163:0/1088246592 shutdown_connections 2023-10-09T06:30:47.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.262+0000 7f11944f4700 1 --2- 172.21.15.163:0/1088246592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f118c159c60 0x7f118c15d5d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:47.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.262+0000 7f11944f4700 1 -- 172.21.15.163:0/1088246592 >> 172.21.15.163:0/1088246592 conn(0x7f118c068e20 msgr2=0x7f118c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:47.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.262+0000 7f11944f4700 1 -- 172.21.15.163:0/1088246592 shutdown_connections 2023-10-09T06:30:47.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.262+0000 7f11944f4700 1 -- 172.21.15.163:0/1088246592 wait complete. 2023-10-09T06:30:47.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.264+0000 7f11944f4700 1 Processor -- start 2023-10-09T06:30:47.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.264+0000 7f11944f4700 1 -- start start 2023-10-09T06:30:47.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.264+0000 7f11944f4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f118c159890 0x7f118c145110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:47.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.264+0000 7f11944f4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f118c159c60 0x7f118c154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:47.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.264+0000 7f11944f4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f118c15db10 0x7f118c15f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:47.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.264+0000 7f11944f4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c152620 con 0x7f118c159c60 2023-10-09T06:30:47.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.264+0000 7f11944f4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c16d080 con 0x7f118c15db10 2023-10-09T06:30:47.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.264+0000 7f11944f4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f118c0b23a0 con 0x7f118c159890 2023-10-09T06:30:47.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.264+0000 7f1191a8f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f118c159c60 0x7f118c154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:47.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.264+0000 7f1191a8f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f118c159c60 0x7f118c154970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36808/0 (socket says 172.21.15.163:36808) 2023-10-09T06:30:47.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.264+0000 7f1191a8f700 1 -- 172.21.15.163:0/4290200445 learned_addr learned my addr 172.21.15.163:0/4290200445 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:47.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.264+0000 7f1192290700 1 --2- 172.21.15.163:0/4290200445 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f118c159890 0x7f118c145110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:47.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.264+0000 7f1192a91700 1 --2- 172.21.15.163:0/4290200445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f118c15db10 0x7f118c15f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:47.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.265+0000 7f1191a8f700 1 -- 172.21.15.163:0/4290200445 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f118c159890 msgr2=0x7f118c145110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:47.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.265+0000 7f1191a8f700 1 --2- 172.21.15.163:0/4290200445 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f118c159890 0x7f118c145110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:47.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.265+0000 7f1191a8f700 1 -- 172.21.15.163:0/4290200445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f118c15db10 msgr2=0x7f118c15f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:47.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.265+0000 7f1191a8f700 1 --2- 172.21.15.163:0/4290200445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f118c15db10 0x7f118c15f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:47.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.265+0000 7f1191a8f700 1 -- 172.21.15.163:0/4290200445 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c009420 con 0x7f118c159c60 2023-10-09T06:30:47.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.265+0000 7f1191a8f700 1 --2- 172.21.15.163:0/4290200445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f118c159c60 0x7f118c154970 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f118c070e20 tx=0x7f117c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:47.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.265+0000 7f11827fc700 1 -- 172.21.15.163:0/4290200445 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c000d10 con 0x7f118c159c60 2023-10-09T06:30:47.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.265+0000 7f11827fc700 1 -- 172.21.15.163:0/4290200445 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117c00bb20 con 0x7f118c159c60 2023-10-09T06:30:47.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.265+0000 7f11944f4700 1 -- 172.21.15.163:0/4290200445 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f118c16d320 con 0x7f118c159c60 2023-10-09T06:30:47.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.265+0000 7f11827fc700 1 -- 172.21.15.163:0/4290200445 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c011650 con 0x7f118c159c60 2023-10-09T06:30:47.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.265+0000 7f11944f4700 1 -- 172.21.15.163:0/4290200445 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f118c145650 con 0x7f118c159c60 2023-10-09T06:30:47.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.266+0000 7f11944f4700 1 -- 172.21.15.163:0/4290200445 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f118c16d530 con 0x7f118c159c60 2023-10-09T06:30:47.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.266+0000 7f11827fc700 1 -- 172.21.15.163:0/4290200445 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f117c0119c0 con 0x7f118c159c60 2023-10-09T06:30:47.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.266+0000 7f11827fc700 1 --2- 172.21.15.163:0/4290200445 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1170066f30 0x7f11700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:47.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.267+0000 7f11827fc700 1 -- 172.21.15.163:0/4290200445 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f117c085b60 con 0x7f118c159c60 2023-10-09T06:30:47.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.267+0000 7f1192290700 1 --2- 172.21.15.163:0/4290200445 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1170066f30 0x7f11700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:47.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.267+0000 7f1192290700 1 --2- 172.21.15.163:0/4290200445 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1170066f30 0x7f11700693e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1188005c50 tx=0x7f1188009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:47.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.271+0000 7f11827fc700 1 -- 172.21.15.163:0/4290200445 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f117c052dd0 con 0x7f118c159c60 2023-10-09T06:30:47.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.455+0000 7f11944f4700 1 -- 172.21.15.163:0/4290200445 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f118c04c3c0 con 0x7f118c159c60 2023-10-09T06:30:47.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.455+0000 7f11827fc700 1 -- 172.21.15.163:0/4290200445 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f117c0563f0 con 0x7f118c159c60 2023-10-09T06:30:47.456 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:30:47.457 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":20,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:30:44.466923+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:30:41.369014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-09T06:30:42.103001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:30:47.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.457+0000 7f11944f4700 1 -- 172.21.15.163:0/4290200445 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1170066f30 msgr2=0x7f11700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:47.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.457+0000 7f11944f4700 1 --2- 172.21.15.163:0/4290200445 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1170066f30 0x7f11700693e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1188005c50 tx=0x7f1188009040 comp rx=0 tx=0).stop 2023-10-09T06:30:47.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.457+0000 7f11944f4700 1 -- 172.21.15.163:0/4290200445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f118c159c60 msgr2=0x7f118c154970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:47.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.457+0000 7f11944f4700 1 --2- 172.21.15.163:0/4290200445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f118c159c60 0x7f118c154970 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f118c070e20 tx=0x7f117c017000 comp rx=0 tx=0).stop 2023-10-09T06:30:47.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.458+0000 7f11944f4700 1 -- 172.21.15.163:0/4290200445 shutdown_connections 2023-10-09T06:30:47.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.458+0000 7f11944f4700 1 --2- 172.21.15.163:0/4290200445 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1170066f30 0x7f11700693e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:47.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.458+0000 7f11944f4700 1 --2- 172.21.15.163:0/4290200445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f118c15db10 0x7f118c15f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:47.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.458+0000 7f11944f4700 1 --2- 172.21.15.163:0/4290200445 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f118c159890 0x7f118c145110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:47.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.458+0000 7f11944f4700 1 --2- 172.21.15.163:0/4290200445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f118c159c60 0x7f118c154970 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:47.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.458+0000 7f11944f4700 1 -- 172.21.15.163:0/4290200445 >> 172.21.15.163:0/4290200445 conn(0x7f118c068e20 msgr2=0x7f118c1553b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:47.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.458+0000 7f11944f4700 1 -- 172.21.15.163:0/4290200445 shutdown_connections 2023-10-09T06:30:47.460 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.458+0000 7f11944f4700 1 -- 172.21.15.163:0/4290200445 wait complete. 2023-10-09T06:30:47.471 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-09T06:30:47.471 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:30:47.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.584+0000 7fbcc3ad2700 1 Processor -- start 2023-10-09T06:30:47.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.584+0000 7fbcc3ad2700 1 -- start start 2023-10-09T06:30:47.588 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.585+0000 7fbcc3ad2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcbc159db0 0x7fbcbc069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:47.588 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.585+0000 7fbcc3ad2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbcbc06a2b0 0x7fbcbc06a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:47.588 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.585+0000 7fbcc3ad2700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcbc0c96a0 con 0x7fbcbc1599e0 2023-10-09T06:30:47.588 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.585+0000 7fbcc3ad2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcbc0cb9e0 con 0x7fbcbc06a2b0 2023-10-09T06:30:47.588 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.585+0000 7fbcc3ad2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc0cab60 con 0x7fbcbc159db0 2023-10-09T06:30:47.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.585+0000 7fbcc186e700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fbcbc1599e0 0x7fbcbc069080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:52442/0 (socket says 172.21.15.163:52442) 2023-10-09T06:30:47.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.585+0000 7fbcc186e700 1 -- 172.21.15.163:0/475748052 learned_addr learned my addr 172.21.15.163:0/475748052 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:47.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.586+0000 7fbcc206f700 1 --2- 172.21.15.163:0/475748052 >> v2:172.21.15.181:3300/0 conn(0x7fbcbc06a2b0 0x7fbcbc06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:47.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.586+0000 7fbcc106d700 1 --2- 172.21.15.163:0/475748052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcbc159db0 0x7fbcbc069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:47.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.586+0000 7fbcc086c700 1 -- 172.21.15.163:0/475748052 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3309938928 0 0) 0x7fbcbc0c96a0 con 0x7fbcbc1599e0 2023-10-09T06:30:47.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.586+0000 7fbcc086c700 1 -- 172.21.15.163:0/475748052 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbca8003570 con 0x7fbcbc1599e0 2023-10-09T06:30:47.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.586+0000 7fbcc206f700 1 -- 172.21.15.163:0/475748052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcbc159db0 msgr2=0x7fbcbc069ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:47.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.586+0000 7fbcc206f700 1 --2- 172.21.15.163:0/475748052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcbc159db0 0x7fbcbc069ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:47.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.586+0000 7fbcc206f700 1 -- 172.21.15.163:0/475748052 >> v1:172.21.15.163:6789/0 conn(0x7fbcbc1599e0 legacy=0x7fbcbc069080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:47.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.586+0000 7fbcc206f700 1 -- 172.21.15.163:0/475748052 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcbc0cb420 con 0x7fbcbc06a2b0 2023-10-09T06:30:47.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.586+0000 7fbcc106d700 1 --2- 172.21.15.163:0/475748052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcbc159db0 0x7fbcbc069ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:30:47.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.586+0000 7fbcc206f700 1 --2- 172.21.15.163:0/475748052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcbc06a2b0 0x7fbcbc06a710 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4009e60 tx=0x7fbcb4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afac1abbd970afbb server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:47.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.586+0000 7fbcc086c700 1 -- 172.21.15.163:0/475748052 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb400e070 con 0x7fbcbc06a2b0 2023-10-09T06:30:47.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.586+0000 7fbcc086c700 1 -- 172.21.15.163:0/475748052 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb400bab0 con 0x7fbcbc06a2b0 2023-10-09T06:30:47.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.586+0000 7fbcc086c700 1 -- 172.21.15.163:0/475748052 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb400be30 con 0x7fbcbc06a2b0 2023-10-09T06:30:47.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.587+0000 7fbcc3ad2700 1 -- 172.21.15.163:0/475748052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcbc06a2b0 msgr2=0x7fbcbc06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:47.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.587+0000 7fbcc3ad2700 1 --2- 172.21.15.163:0/475748052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcbc06a2b0 0x7fbcbc06a710 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4009e60 tx=0x7fbcb4009780 comp rx=0 tx=0).stop 2023-10-09T06:30:47.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.587+0000 7fbcc3ad2700 1 -- 172.21.15.163:0/475748052 shutdown_connections 2023-10-09T06:30:47.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.587+0000 7fbcc3ad2700 1 --2- 172.21.15.163:0/475748052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcbc06a2b0 0x7fbcbc06a710 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:47.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.587+0000 7fbcc3ad2700 1 --2- 172.21.15.163:0/475748052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcbc159db0 0x7fbcbc069ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:47.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.587+0000 7fbcc3ad2700 1 -- 172.21.15.163:0/475748052 >> 172.21.15.163:0/475748052 conn(0x7fbcbc0c9860 msgr2=0x7fbcbc0c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:47.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.587+0000 7fbcc3ad2700 1 -- 172.21.15.163:0/475748052 shutdown_connections 2023-10-09T06:30:47.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.587+0000 7fbcc3ad2700 1 -- 172.21.15.163:0/475748052 wait complete. 2023-10-09T06:30:47.593 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.588+0000 7fbcc3ad2700 1 Processor -- start 2023-10-09T06:30:47.593 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.588+0000 7fbcc3ad2700 1 -- start start 2023-10-09T06:30:47.593 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.589+0000 7fbcc3ad2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcbc06a2b0 0x7fbcbc151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:47.593 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.589+0000 7fbcc3ad2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcbc1599e0 0x7fbcbc15c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:47.593 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.589+0000 7fbcc3ad2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcbc159db0 0x7fbcbc15c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:47.593 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.589+0000 7fbcc3ad2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc151fd0 con 0x7fbcbc159db0 2023-10-09T06:30:47.594 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.589+0000 7fbcc3ad2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc169cb0 con 0x7fbcbc06a2b0 2023-10-09T06:30:47.594 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.589+0000 7fbcc3ad2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc16a7f0 con 0x7fbcbc1599e0 2023-10-09T06:30:47.594 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.589+0000 7fbcc206f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcbc159db0 0x7fbcbc15c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:47.594 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.589+0000 7fbcc206f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcbc159db0 0x7fbcbc15c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36818/0 (socket says 172.21.15.163:36818) 2023-10-09T06:30:47.594 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.589+0000 7fbcc206f700 1 -- 172.21.15.163:0/3969201461 learned_addr learned my addr 172.21.15.163:0/3969201461 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:47.594 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.589+0000 7fbcc106d700 1 --2- 172.21.15.163:0/3969201461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcbc1599e0 0x7fbcbc15c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:47.594 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.589+0000 7fbcc186e700 1 --2- 172.21.15.163:0/3969201461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcbc06a2b0 0x7fbcbc151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:47.594 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.589+0000 7fbcc206f700 1 -- 172.21.15.163:0/3969201461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcbc1599e0 msgr2=0x7fbcbc15c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:47.594 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.589+0000 7fbcc206f700 1 --2- 172.21.15.163:0/3969201461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcbc1599e0 0x7fbcbc15c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:47.594 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.589+0000 7fbcc206f700 1 -- 172.21.15.163:0/3969201461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcbc06a2b0 msgr2=0x7fbcbc151a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:47.594 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.589+0000 7fbcc206f700 1 --2- 172.21.15.163:0/3969201461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcbc06a2b0 0x7fbcbc151a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:47.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.589+0000 7fbcc206f700 1 -- 172.21.15.163:0/3969201461 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcac009420 con 0x7fbcbc159db0 2023-10-09T06:30:47.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.589+0000 7fbcc206f700 1 --2- 172.21.15.163:0/3969201461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcbc159db0 0x7fbcbc15c820 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc06d6b0 tx=0x7fbcb4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:47.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.589+0000 7fbcb1ffb700 1 -- 172.21.15.163:0/3969201461 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4012e10 con 0x7fbcbc159db0 2023-10-09T06:30:47.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.589+0000 7fbcb1ffb700 1 -- 172.21.15.163:0/3969201461 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb4027470 con 0x7fbcbc159db0 2023-10-09T06:30:47.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.589+0000 7fbcc3ad2700 1 -- 172.21.15.163:0/3969201461 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb4009a20 con 0x7fbcbc159db0 2023-10-09T06:30:47.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.589+0000 7fbcb1ffb700 1 -- 172.21.15.163:0/3969201461 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4027830 con 0x7fbcbc159db0 2023-10-09T06:30:47.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.590+0000 7fbcc3ad2700 1 -- 172.21.15.163:0/3969201461 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcbc06ae80 con 0x7fbcbc159db0 2023-10-09T06:30:47.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.590+0000 7fbcc3ad2700 1 -- 172.21.15.163:0/3969201461 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcbc16a9a0 con 0x7fbcbc159db0 2023-10-09T06:30:47.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.591+0000 7fbcb1ffb700 1 -- 172.21.15.163:0/3969201461 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbcb4024070 con 0x7fbcbc159db0 2023-10-09T06:30:47.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.591+0000 7fbcb1ffb700 1 --2- 172.21.15.163:0/3969201461 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbca0066f30 0x7fbca00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:47.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.591+0000 7fbcb1ffb700 1 -- 172.21.15.163:0/3969201461 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbcb4086ad0 con 0x7fbcbc159db0 2023-10-09T06:30:47.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.591+0000 7fbcc186e700 1 --2- 172.21.15.163:0/3969201461 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbca0066f30 0x7fbca00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:47.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.592+0000 7fbcc186e700 1 --2- 172.21.15.163:0/3969201461 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbca0066f30 0x7fbca00693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8007fd0 tx=0x7fbcb8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:47.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.595+0000 7fbcb1ffb700 1 -- 172.21.15.163:0/3969201461 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbcbc16a9a0 con 0x7fbcbc159db0 2023-10-09T06:30:47.732 INFO:tasks.ceph.mds.b.smithi181.stderr:2023-10-09T06:30:47.731+0000 7fef6a0a8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-09T06:30:47.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.776+0000 7fbcc3ad2700 1 -- 172.21.15.163:0/3969201461 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbcbc16a960 con 0x7fbca0066f30 2023-10-09T06:30:47.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.785+0000 7fbcb1ffb700 1 -- 172.21.15.163:0/3969201461 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264535 (secure 0 0 0) 0x7fbcbc16a960 con 0x7fbca0066f30 2023-10-09T06:30:47.794 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:30:47.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.788+0000 7fbcc3ad2700 1 -- 172.21.15.163:0/3969201461 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbca0066f30 msgr2=0x7fbca00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:47.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.788+0000 7fbcc3ad2700 1 --2- 172.21.15.163:0/3969201461 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbca0066f30 0x7fbca00693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8007fd0 tx=0x7fbcb8009040 comp rx=0 tx=0).stop 2023-10-09T06:30:47.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.788+0000 7fbcc3ad2700 1 -- 172.21.15.163:0/3969201461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcbc159db0 msgr2=0x7fbcbc15c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:47.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.788+0000 7fbcc3ad2700 1 --2- 172.21.15.163:0/3969201461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcbc159db0 0x7fbcbc15c820 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc06d6b0 tx=0x7fbcb4019000 comp rx=0 tx=0).stop 2023-10-09T06:30:47.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.788+0000 7fbcc3ad2700 1 -- 172.21.15.163:0/3969201461 shutdown_connections 2023-10-09T06:30:47.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.788+0000 7fbcc3ad2700 1 --2- 172.21.15.163:0/3969201461 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbca0066f30 0x7fbca00693e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:47.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.788+0000 7fbcc3ad2700 1 --2- 172.21.15.163:0/3969201461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcbc159db0 0x7fbcbc15c820 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:47.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.788+0000 7fbcc3ad2700 1 --2- 172.21.15.163:0/3969201461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcbc06a2b0 0x7fbcbc151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:47.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.788+0000 7fbcc3ad2700 1 --2- 172.21.15.163:0/3969201461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcbc1599e0 0x7fbcbc15c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:47.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.788+0000 7fbcc3ad2700 1 -- 172.21.15.163:0/3969201461 >> 172.21.15.163:0/3969201461 conn(0x7fbcbc0c9860 msgr2=0x7fbcbc06dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:47.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.788+0000 7fbcc3ad2700 1 -- 172.21.15.163:0/3969201461 shutdown_connections 2023-10-09T06:30:47.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.788+0000 7fbcc3ad2700 1 -- 172.21.15.163:0/3969201461 wait complete. 2023-10-09T06:30:47.798 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:30:47.801 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-09T06:30:46.295024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":122,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381092,"kb_used_data":2788,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593628,"statfs":{"total":773094113280,"available":770655875072,"internally_reserved":0,"allocated":2854912,"data_stored":1731716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.556757"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.492500+0000","last_change":"2023-10-09T06:30:43.487077+0000","last_active":"2023-10-09T06:30:43.492500+0000","last_peered":"2023-10-09T06:30:43.492500+0000","last_clean":"2023-10-09T06:30:43.492500+0000","last_became_active":"2023-10-09T06:30:43.486817+0000","last_became_peered":"2023-10-09T06:30:43.486817+0000","last_unstale":"2023-10-09T06:30:43.492500+0000","last_undegraded":"2023-10-09T06:30:43.492500+0000","last_fullsized":"2023-10-09T06:30:43.492500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:59:57.434765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.486972+0000","last_change":"2023-10-09T06:30:43.484119+0000","last_active":"2023-10-09T06:30:43.486972+0000","last_peered":"2023-10-09T06:30:43.486972+0000","last_clean":"2023-10-09T06:30:43.486972+0000","last_became_active":"2023-10-09T06:30:43.483865+0000","last_became_peered":"2023-10-09T06:30:43.483865+0000","last_unstale":"2023-10-09T06:30:43.486972+0000","last_undegraded":"2023-10-09T06:30:43.486972+0000","last_fullsized":"2023-10-09T06:30:43.486972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:06:29.986154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.484893+0000","last_change":"2023-10-09T06:30:43.480754+0000","last_active":"2023-10-09T06:30:43.484893+0000","last_peered":"2023-10-09T06:30:43.484893+0000","last_clean":"2023-10-09T06:30:43.484893+0000","last_became_active":"2023-10-09T06:30:43.480474+0000","last_became_peered":"2023-10-09T06:30:43.480474+0000","last_unstale":"2023-10-09T06:30:43.484893+0000","last_undegraded":"2023-10-09T06:30:43.484893+0000","last_fullsized":"2023-10-09T06:30:43.484893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:56:01.192585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493176+0000","last_change":"2023-10-09T06:30:43.489012+0000","last_active":"2023-10-09T06:30:43.493176+0000","last_peered":"2023-10-09T06:30:43.493176+0000","last_clean":"2023-10-09T06:30:43.493176+0000","last_became_active":"2023-10-09T06:30:43.488614+0000","last_became_peered":"2023-10-09T06:30:43.488614+0000","last_unstale":"2023-10-09T06:30:43.493176+0000","last_undegraded":"2023-10-09T06:30:43.493176+0000","last_fullsized":"2023-10-09T06:30:43.493176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:41:00.854052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.486903+0000","last_change":"2023-10-09T06:30:43.483013+0000","last_active":"2023-10-09T06:30:43.486903+0000","last_peered":"2023-10-09T06:30:43.486903+0000","last_clean":"2023-10-09T06:30:43.486903+0000","last_became_active":"2023-10-09T06:30:43.482373+0000","last_became_peered":"2023-10-09T06:30:43.482373+0000","last_unstale":"2023-10-09T06:30:43.486903+0000","last_undegraded":"2023-10-09T06:30:43.486903+0000","last_fullsized":"2023-10-09T06:30:43.486903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:46:57.531379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.492270+0000","last_change":"2023-10-09T06:30:43.488813+0000","last_active":"2023-10-09T06:30:43.492270+0000","last_peered":"2023-10-09T06:30:43.492270+0000","last_clean":"2023-10-09T06:30:43.492270+0000","last_became_active":"2023-10-09T06:30:43.488532+0000","last_became_peered":"2023-10-09T06:30:43.488532+0000","last_unstale":"2023-10-09T06:30:43.492270+0000","last_undegraded":"2023-10-09T06:30:43.492270+0000","last_fullsized":"2023-10-09T06:30:43.492270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:57:57.207775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.491984+0000","last_change":"2023-10-09T06:30:43.486418+0000","last_active":"2023-10-09T06:30:43.491984+0000","last_peered":"2023-10-09T06:30:43.491984+0000","last_clean":"2023-10-09T06:30:43.491984+0000","last_became_active":"2023-10-09T06:30:43.486128+0000","last_became_peered":"2023-10-09T06:30:43.486128+0000","last_unstale":"2023-10-09T06:30:43.491984+0000","last_undegraded":"2023-10-09T06:30:43.491984+0000","last_fullsized":"2023-10-09T06:30:43.491984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:57:57.207775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493719+0000","last_change":"2023-10-09T06:30:43.490024+0000","last_active":"2023-10-09T06:30:43.493719+0000","last_peered":"2023-10-09T06:30:43.493719+0000","last_clean":"2023-10-09T06:30:43.493719+0000","last_became_active":"2023-10-09T06:30:43.489785+0000","last_became_peered":"2023-10-09T06:30:43.489785+0000","last_unstale":"2023-10-09T06:30:43.493719+0000","last_undegraded":"2023-10-09T06:30:43.493719+0000","last_fullsized":"2023-10-09T06:30:43.493719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:02:03.930820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496624+0000","last_change":"2023-10-09T06:30:43.492372+0000","last_active":"2023-10-09T06:30:43.496624+0000","last_peered":"2023-10-09T06:30:43.496624+0000","last_clean":"2023-10-09T06:30:43.496624+0000","last_became_active":"2023-10-09T06:30:43.492120+0000","last_became_peered":"2023-10-09T06:30:43.492120+0000","last_unstale":"2023-10-09T06:30:43.496624+0000","last_undegraded":"2023-10-09T06:30:43.496624+0000","last_fullsized":"2023-10-09T06:30:43.496624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:20:42.701396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493788+0000","last_change":"2023-10-09T06:30:43.489507+0000","last_active":"2023-10-09T06:30:43.493788+0000","last_peered":"2023-10-09T06:30:43.493788+0000","last_clean":"2023-10-09T06:30:43.493788+0000","last_became_active":"2023-10-09T06:30:43.489279+0000","last_became_peered":"2023-10-09T06:30:43.489279+0000","last_unstale":"2023-10-09T06:30:43.493788+0000","last_undegraded":"2023-10-09T06:30:43.493788+0000","last_fullsized":"2023-10-09T06:30:43.493788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:05:59.526736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496502+0000","last_change":"2023-10-09T06:30:43.491554+0000","last_active":"2023-10-09T06:30:43.496502+0000","last_peered":"2023-10-09T06:30:43.496502+0000","last_clean":"2023-10-09T06:30:43.496502+0000","last_became_active":"2023-10-09T06:30:43.491305+0000","last_became_peered":"2023-10-09T06:30:43.491305+0000","last_unstale":"2023-10-09T06:30:43.496502+0000","last_undegraded":"2023-10-09T06:30:43.496502+0000","last_fullsized":"2023-10-09T06:30:43.496502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:15:54.706732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.483269+0000","last_change":"2023-10-09T06:30:43.478613+0000","last_active":"2023-10-09T06:30:43.483269+0000","last_peered":"2023-10-09T06:30:43.483269+0000","last_clean":"2023-10-09T06:30:43.483269+0000","last_became_active":"2023-10-09T06:30:43.477580+0000","last_became_peered":"2023-10-09T06:30:43.477580+0000","last_unstale":"2023-10-09T06:30:43.483269+0000","last_undegraded":"2023-10-09T06:30:43.483269+0000","last_fullsized":"2023-10-09T06:30:43.483269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:15:34.625915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.491028+0000","last_change":"2023-10-09T06:30:43.485665+0000","last_active":"2023-10-09T06:30:43.491028+0000","last_peered":"2023-10-09T06:30:43.491028+0000","last_clean":"2023-10-09T06:30:43.491028+0000","last_became_active":"2023-10-09T06:30:43.485346+0000","last_became_peered":"2023-10-09T06:30:43.485346+0000","last_unstale":"2023-10-09T06:30:43.491028+0000","last_undegraded":"2023-10-09T06:30:43.491028+0000","last_fullsized":"2023-10-09T06:30:43.491028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:56:01.192585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.487663+0000","last_change":"2023-10-09T06:30:43.483174+0000","last_active":"2023-10-09T06:30:43.487663+0000","last_peered":"2023-10-09T06:30:43.487663+0000","last_clean":"2023-10-09T06:30:43.487663+0000","last_became_active":"2023-10-09T06:30:43.482518+0000","last_became_peered":"2023-10-09T06:30:43.482518+0000","last_unstale":"2023-10-09T06:30:43.487663+0000","last_undegraded":"2023-10-09T06:30:43.487663+0000","last_fullsized":"2023-10-09T06:30:43.487663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:56:48.342402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493620+0000","last_change":"2023-10-09T06:30:43.489829+0000","last_active":"2023-10-09T06:30:43.493620+0000","last_peered":"2023-10-09T06:30:43.493620+0000","last_clean":"2023-10-09T06:30:43.493620+0000","last_became_active":"2023-10-09T06:30:43.489549+0000","last_became_peered":"2023-10-09T06:30:43.489549+0000","last_unstale":"2023-10-09T06:30:43.493620+0000","last_undegraded":"2023-10-09T06:30:43.493620+0000","last_fullsized":"2023-10-09T06:30:43.493620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:56:01.192585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496629+0000","last_change":"2023-10-09T06:30:43.491356+0000","last_active":"2023-10-09T06:30:43.496629+0000","last_peered":"2023-10-09T06:30:43.496629+0000","last_clean":"2023-10-09T06:30:43.496629+0000","last_became_active":"2023-10-09T06:30:43.491058+0000","last_became_peered":"2023-10-09T06:30:43.491058+0000","last_unstale":"2023-10-09T06:30:43.496629+0000","last_undegraded":"2023-10-09T06:30:43.496629+0000","last_fullsized":"2023-10-09T06:30:43.496629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:03:07.803820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.492196+0000","last_change":"2023-10-09T06:30:43.489230+0000","last_active":"2023-10-09T06:30:43.492196+0000","last_peered":"2023-10-09T06:30:43.492196+0000","last_clean":"2023-10-09T06:30:43.492196+0000","last_became_active":"2023-10-09T06:30:43.488094+0000","last_became_peered":"2023-10-09T06:30:43.488094+0000","last_unstale":"2023-10-09T06:30:43.492196+0000","last_undegraded":"2023-10-09T06:30:43.492196+0000","last_fullsized":"2023-10-09T06:30:43.492196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:36:01.020241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.488118+0000","last_change":"2023-10-09T06:30:43.486270+0000","last_active":"2023-10-09T06:30:43.488118+0000","last_peered":"2023-10-09T06:30:43.488118+0000","last_clean":"2023-10-09T06:30:43.488118+0000","last_became_active":"2023-10-09T06:30:43.486003+0000","last_became_peered":"2023-10-09T06:30:43.486003+0000","last_unstale":"2023-10-09T06:30:43.488118+0000","last_undegraded":"2023-10-09T06:30:43.488118+0000","last_fullsized":"2023-10-09T06:30:43.488118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:55:20.971921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.485740+0000","last_change":"2023-10-09T06:30:43.482479+0000","last_active":"2023-10-09T06:30:43.485740+0000","last_peered":"2023-10-09T06:30:43.485740+0000","last_clean":"2023-10-09T06:30:43.485740+0000","last_became_active":"2023-10-09T06:30:43.482203+0000","last_became_peered":"2023-10-09T06:30:43.482203+0000","last_unstale":"2023-10-09T06:30:43.485740+0000","last_undegraded":"2023-10-09T06:30:43.485740+0000","last_fullsized":"2023-10-09T06:30:43.485740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:41:39.976781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.761744+0000","last_change":"2023-10-09T06:30:42.761358+0000","last_active":"2023-10-09T06:30:43.761744+0000","last_peered":"2023-10-09T06:30:43.761744+0000","last_clean":"2023-10-09T06:30:43.761744+0000","last_became_active":"2023-10-09T06:30:42.761139+0000","last_became_peered":"2023-10-09T06:30:42.761139+0000","last_unstale":"2023-10-09T06:30:43.761744+0000","last_undegraded":"2023-10-09T06:30:43.761744+0000","last_fullsized":"2023-10-09T06:30:43.761744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:27:28.658955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.490991+0000","last_change":"2023-10-09T06:30:43.485519+0000","last_active":"2023-10-09T06:30:43.490991+0000","last_peered":"2023-10-09T06:30:43.490991+0000","last_clean":"2023-10-09T06:30:43.490991+0000","last_became_active":"2023-10-09T06:30:43.485289+0000","last_became_peered":"2023-10-09T06:30:43.485289+0000","last_unstale":"2023-10-09T06:30:43.490991+0000","last_undegraded":"2023-10-09T06:30:43.490991+0000","last_fullsized":"2023-10-09T06:30:43.490991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:41:39.976781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.764141+0000","last_change":"2023-10-09T06:30:42.758771+0000","last_active":"2023-10-09T06:30:43.764141+0000","last_peered":"2023-10-09T06:30:43.764141+0000","last_clean":"2023-10-09T06:30:43.764141+0000","last_became_active":"2023-10-09T06:30:42.756764+0000","last_became_peered":"2023-10-09T06:30:42.756764+0000","last_unstale":"2023-10-09T06:30:43.764141+0000","last_undegraded":"2023-10-09T06:30:43.764141+0000","last_fullsized":"2023-10-09T06:30:43.764141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:26:03.347834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493888+0000","last_change":"2023-10-09T06:30:43.491484+0000","last_active":"2023-10-09T06:30:43.493888+0000","last_peered":"2023-10-09T06:30:43.493888+0000","last_clean":"2023-10-09T06:30:43.493888+0000","last_became_active":"2023-10-09T06:30:43.491319+0000","last_became_peered":"2023-10-09T06:30:43.491319+0000","last_unstale":"2023-10-09T06:30:43.493888+0000","last_undegraded":"2023-10-09T06:30:43.493888+0000","last_fullsized":"2023-10-09T06:30:43.493888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:12:09.812942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.761120+0000","last_change":"2023-10-09T06:30:42.763648+0000","last_active":"2023-10-09T06:30:43.761120+0000","last_peered":"2023-10-09T06:30:43.761120+0000","last_clean":"2023-10-09T06:30:43.761120+0000","last_became_active":"2023-10-09T06:30:42.763348+0000","last_became_peered":"2023-10-09T06:30:42.763348+0000","last_unstale":"2023-10-09T06:30:43.761120+0000","last_undegraded":"2023-10-09T06:30:43.761120+0000","last_fullsized":"2023-10-09T06:30:43.761120+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:55:19.967211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496719+0000","last_change":"2023-10-09T06:30:43.491778+0000","last_active":"2023-10-09T06:30:43.496719+0000","last_peered":"2023-10-09T06:30:43.496719+0000","last_clean":"2023-10-09T06:30:43.496719+0000","last_became_active":"2023-10-09T06:30:43.491137+0000","last_became_peered":"2023-10-09T06:30:43.491137+0000","last_unstale":"2023-10-09T06:30:43.496719+0000","last_undegraded":"2023-10-09T06:30:43.496719+0000","last_fullsized":"2023-10-09T06:30:43.496719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:46:57.531379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.761164+0000","last_change":"2023-10-09T06:30:42.761346+0000","last_active":"2023-10-09T06:30:43.761164+0000","last_peered":"2023-10-09T06:30:43.761164+0000","last_clean":"2023-10-09T06:30:43.761164+0000","last_became_active":"2023-10-09T06:30:42.761028+0000","last_became_peered":"2023-10-09T06:30:42.761028+0000","last_unstale":"2023-10-09T06:30:43.761164+0000","last_undegraded":"2023-10-09T06:30:43.761164+0000","last_fullsized":"2023-10-09T06:30:43.761164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:20:58.653921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.492949+0000","last_change":"2023-10-09T06:30:43.489975+0000","last_active":"2023-10-09T06:30:43.492949+0000","last_peered":"2023-10-09T06:30:43.492949+0000","last_clean":"2023-10-09T06:30:43.492949+0000","last_became_active":"2023-10-09T06:30:43.489712+0000","last_became_peered":"2023-10-09T06:30:43.489712+0000","last_unstale":"2023-10-09T06:30:43.492949+0000","last_undegraded":"2023-10-09T06:30:43.492949+0000","last_fullsized":"2023-10-09T06:30:43.492949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:41:39.976781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.762311+0000","last_change":"2023-10-09T06:30:42.757689+0000","last_active":"2023-10-09T06:30:43.762311+0000","last_peered":"2023-10-09T06:30:43.762311+0000","last_clean":"2023-10-09T06:30:43.762311+0000","last_became_active":"2023-10-09T06:30:42.757270+0000","last_became_peered":"2023-10-09T06:30:42.757270+0000","last_unstale":"2023-10-09T06:30:43.762311+0000","last_undegraded":"2023-10-09T06:30:43.762311+0000","last_fullsized":"2023-10-09T06:30:43.762311+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:52:48.682306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.490754+0000","last_change":"2023-10-09T06:30:43.486187+0000","last_active":"2023-10-09T06:30:43.490754+0000","last_peered":"2023-10-09T06:30:43.490754+0000","last_clean":"2023-10-09T06:30:43.490754+0000","last_became_active":"2023-10-09T06:30:43.485917+0000","last_became_peered":"2023-10-09T06:30:43.485917+0000","last_unstale":"2023-10-09T06:30:43.490754+0000","last_undegraded":"2023-10-09T06:30:43.490754+0000","last_fullsized":"2023-10-09T06:30:43.490754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:20:59.658630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.763447+0000","last_change":"2023-10-09T06:30:42.758939+0000","last_active":"2023-10-09T06:30:43.763447+0000","last_peered":"2023-10-09T06:30:43.763447+0000","last_clean":"2023-10-09T06:30:43.763447+0000","last_became_active":"2023-10-09T06:30:42.758667+0000","last_became_peered":"2023-10-09T06:30:42.758667+0000","last_unstale":"2023-10-09T06:30:43.763447+0000","last_undegraded":"2023-10-09T06:30:43.763447+0000","last_fullsized":"2023-10-09T06:30:43.763447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:59:42.424421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.488120+0000","last_change":"2023-10-09T06:30:43.484481+0000","last_active":"2023-10-09T06:30:43.488120+0000","last_peered":"2023-10-09T06:30:43.488120+0000","last_clean":"2023-10-09T06:30:43.488120+0000","last_became_active":"2023-10-09T06:30:43.483640+0000","last_became_peered":"2023-10-09T06:30:43.483640+0000","last_unstale":"2023-10-09T06:30:43.488120+0000","last_undegraded":"2023-10-09T06:30:43.488120+0000","last_fullsized":"2023-10-09T06:30:43.488120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:38:10.342140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.474106+0000","last_change":"2023-10-09T06:30:42.758834+0000","last_active":"2023-10-09T06:30:43.474106+0000","last_peered":"2023-10-09T06:30:43.474106+0000","last_clean":"2023-10-09T06:30:43.474106+0000","last_became_active":"2023-10-09T06:30:42.758561+0000","last_became_peered":"2023-10-09T06:30:42.758561+0000","last_unstale":"2023-10-09T06:30:43.474106+0000","last_undegraded":"2023-10-09T06:30:43.474106+0000","last_fullsized":"2023-10-09T06:30:43.474106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:46:54.709506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.490866+0000","last_change":"2023-10-09T06:30:43.483623+0000","last_active":"2023-10-09T06:30:43.490866+0000","last_peered":"2023-10-09T06:30:43.490866+0000","last_clean":"2023-10-09T06:30:43.490866+0000","last_became_active":"2023-10-09T06:30:43.483435+0000","last_became_peered":"2023-10-09T06:30:43.483435+0000","last_unstale":"2023-10-09T06:30:43.490866+0000","last_undegraded":"2023-10-09T06:30:43.490866+0000","last_fullsized":"2023-10-09T06:30:43.490866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:57:57.207775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.485588+0000","last_change":"2023-10-09T06:30:42.761685+0000","last_active":"2023-10-09T06:30:43.485588+0000","last_peered":"2023-10-09T06:30:43.485588+0000","last_clean":"2023-10-09T06:30:43.485588+0000","last_became_active":"2023-10-09T06:30:42.761432+0000","last_became_peered":"2023-10-09T06:30:42.761432+0000","last_unstale":"2023-10-09T06:30:43.485588+0000","last_undegraded":"2023-10-09T06:30:43.485588+0000","last_fullsized":"2023-10-09T06:30:43.485588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:11:08.967445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.486800+0000","last_change":"2023-10-09T06:30:43.483621+0000","last_active":"2023-10-09T06:30:43.486800+0000","last_peered":"2023-10-09T06:30:43.486800+0000","last_clean":"2023-10-09T06:30:43.486800+0000","last_became_active":"2023-10-09T06:30:43.483365+0000","last_became_peered":"2023-10-09T06:30:43.483365+0000","last_unstale":"2023-10-09T06:30:43.486800+0000","last_undegraded":"2023-10-09T06:30:43.486800+0000","last_fullsized":"2023-10-09T06:30:43.486800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:57:59.094337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.470554+0000","last_change":"2023-10-09T06:30:42.757148+0000","last_active":"2023-10-09T06:30:43.470554+0000","last_peered":"2023-10-09T06:30:43.470554+0000","last_clean":"2023-10-09T06:30:43.470554+0000","last_became_active":"2023-10-09T06:30:42.756630+0000","last_became_peered":"2023-10-09T06:30:42.756630+0000","last_unstale":"2023-10-09T06:30:43.470554+0000","last_undegraded":"2023-10-09T06:30:43.470554+0000","last_fullsized":"2023-10-09T06:30:43.470554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:11:08.967445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496741+0000","last_change":"2023-10-09T06:30:43.492800+0000","last_active":"2023-10-09T06:30:43.496741+0000","last_peered":"2023-10-09T06:30:43.496741+0000","last_clean":"2023-10-09T06:30:43.496741+0000","last_became_active":"2023-10-09T06:30:43.492567+0000","last_became_peered":"2023-10-09T06:30:43.492567+0000","last_unstale":"2023-10-09T06:30:43.496741+0000","last_undegraded":"2023-10-09T06:30:43.496741+0000","last_fullsized":"2023-10-09T06:30:43.496741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:05:59.526736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.482776+0000","last_change":"2023-10-09T06:30:42.759966+0000","last_active":"2023-10-09T06:30:43.482776+0000","last_peered":"2023-10-09T06:30:43.482776+0000","last_clean":"2023-10-09T06:30:43.482776+0000","last_became_active":"2023-10-09T06:30:42.759686+0000","last_became_peered":"2023-10-09T06:30:42.759686+0000","last_unstale":"2023-10-09T06:30:43.482776+0000","last_undegraded":"2023-10-09T06:30:43.482776+0000","last_fullsized":"2023-10-09T06:30:43.482776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:46:54.709506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493906+0000","last_change":"2023-10-09T06:30:43.491580+0000","last_active":"2023-10-09T06:30:43.493906+0000","last_peered":"2023-10-09T06:30:43.493906+0000","last_clean":"2023-10-09T06:30:43.493906+0000","last_became_active":"2023-10-09T06:30:43.491323+0000","last_became_peered":"2023-10-09T06:30:43.491323+0000","last_unstale":"2023-10-09T06:30:43.493906+0000","last_undegraded":"2023-10-09T06:30:43.493906+0000","last_fullsized":"2023-10-09T06:30:43.493906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:02:03.930820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.761837+0000","last_change":"2023-10-09T06:30:42.758290+0000","last_active":"2023-10-09T06:30:43.761837+0000","last_peered":"2023-10-09T06:30:43.761837+0000","last_clean":"2023-10-09T06:30:43.761837+0000","last_became_active":"2023-10-09T06:30:42.758008+0000","last_became_peered":"2023-10-09T06:30:42.758008+0000","last_unstale":"2023-10-09T06:30:43.761837+0000","last_undegraded":"2023-10-09T06:30:43.761837+0000","last_fullsized":"2023-10-09T06:30:43.761837+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:20:58.653921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.483249+0000","last_change":"2023-10-09T06:30:43.478610+0000","last_active":"2023-10-09T06:30:43.483249+0000","last_peered":"2023-10-09T06:30:43.483249+0000","last_clean":"2023-10-09T06:30:43.483249+0000","last_became_active":"2023-10-09T06:30:43.477942+0000","last_became_peered":"2023-10-09T06:30:43.477942+0000","last_unstale":"2023-10-09T06:30:43.483249+0000","last_undegraded":"2023-10-09T06:30:43.483249+0000","last_fullsized":"2023-10-09T06:30:43.483249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:29:12.176820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.766388+0000","last_change":"2023-10-09T06:30:42.759743+0000","last_active":"2023-10-09T06:30:43.766388+0000","last_peered":"2023-10-09T06:30:43.766388+0000","last_clean":"2023-10-09T06:30:43.766388+0000","last_became_active":"2023-10-09T06:30:42.757647+0000","last_became_peered":"2023-10-09T06:30:42.757647+0000","last_unstale":"2023-10-09T06:30:43.766388+0000","last_undegraded":"2023-10-09T06:30:43.766388+0000","last_fullsized":"2023-10-09T06:30:43.766388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:46:54.709506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.485759+0000","last_change":"2023-10-09T06:30:43.484192+0000","last_active":"2023-10-09T06:30:43.485759+0000","last_peered":"2023-10-09T06:30:43.485759+0000","last_clean":"2023-10-09T06:30:43.485759+0000","last_became_active":"2023-10-09T06:30:43.483935+0000","last_became_peered":"2023-10-09T06:30:43.483935+0000","last_unstale":"2023-10-09T06:30:43.485759+0000","last_undegraded":"2023-10-09T06:30:43.485759+0000","last_fullsized":"2023-10-09T06:30:43.485759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:36:01.020241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.763949+0000","last_change":"2023-10-09T06:30:42.760604+0000","last_active":"2023-10-09T06:30:43.763949+0000","last_peered":"2023-10-09T06:30:43.763949+0000","last_clean":"2023-10-09T06:30:43.763949+0000","last_became_active":"2023-10-09T06:30:42.760363+0000","last_became_peered":"2023-10-09T06:30:42.760363+0000","last_unstale":"2023-10-09T06:30:43.763949+0000","last_undegraded":"2023-10-09T06:30:43.763949+0000","last_fullsized":"2023-10-09T06:30:43.763949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:26:03.347834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.484683+0000","last_change":"2023-10-09T06:30:43.479684+0000","last_active":"2023-10-09T06:30:43.484683+0000","last_peered":"2023-10-09T06:30:43.484683+0000","last_clean":"2023-10-09T06:30:43.484683+0000","last_became_active":"2023-10-09T06:30:43.479262+0000","last_became_peered":"2023-10-09T06:30:43.479262+0000","last_unstale":"2023-10-09T06:30:43.484683+0000","last_undegraded":"2023-10-09T06:30:43.484683+0000","last_fullsized":"2023-10-09T06:30:43.484683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:09:10.695002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.482211+0000","last_change":"2023-10-09T06:30:42.762087+0000","last_active":"2023-10-09T06:30:43.482211+0000","last_peered":"2023-10-09T06:30:43.482211+0000","last_clean":"2023-10-09T06:30:43.482211+0000","last_became_active":"2023-10-09T06:30:42.761818+0000","last_became_peered":"2023-10-09T06:30:42.761818+0000","last_unstale":"2023-10-09T06:30:43.482211+0000","last_undegraded":"2023-10-09T06:30:43.482211+0000","last_fullsized":"2023-10-09T06:30:43.482211+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:45:19.887880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.486923+0000","last_change":"2023-10-09T06:30:43.483220+0000","last_active":"2023-10-09T06:30:43.486923+0000","last_peered":"2023-10-09T06:30:43.486923+0000","last_clean":"2023-10-09T06:30:43.486923+0000","last_became_active":"2023-10-09T06:30:43.482733+0000","last_became_peered":"2023-10-09T06:30:43.482733+0000","last_unstale":"2023-10-09T06:30:43.486923+0000","last_undegraded":"2023-10-09T06:30:43.486923+0000","last_fullsized":"2023-10-09T06:30:43.486923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:43:44.457796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.470076+0000","last_change":"2023-10-09T06:30:42.759270+0000","last_active":"2023-10-09T06:30:43.470076+0000","last_peered":"2023-10-09T06:30:43.470076+0000","last_clean":"2023-10-09T06:30:43.470076+0000","last_became_active":"2023-10-09T06:30:42.758836+0000","last_became_peered":"2023-10-09T06:30:42.758836+0000","last_unstale":"2023-10-09T06:30:43.470076+0000","last_undegraded":"2023-10-09T06:30:43.470076+0000","last_fullsized":"2023-10-09T06:30:43.470076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:36:40.691450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.484511+0000","last_change":"2023-10-09T06:30:43.480359+0000","last_active":"2023-10-09T06:30:43.484511+0000","last_peered":"2023-10-09T06:30:43.484511+0000","last_clean":"2023-10-09T06:30:43.484511+0000","last_became_active":"2023-10-09T06:30:43.480040+0000","last_became_peered":"2023-10-09T06:30:43.480040+0000","last_unstale":"2023-10-09T06:30:43.484511+0000","last_undegraded":"2023-10-09T06:30:43.484511+0000","last_fullsized":"2023-10-09T06:30:43.484511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:06:29.986154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.764519+0000","last_change":"2023-10-09T06:30:42.756126+0000","last_active":"2023-10-09T06:30:43.764519+0000","last_peered":"2023-10-09T06:30:43.764519+0000","last_clean":"2023-10-09T06:30:43.764519+0000","last_became_active":"2023-10-09T06:30:42.755132+0000","last_became_peered":"2023-10-09T06:30:42.755132+0000","last_unstale":"2023-10-09T06:30:43.764519+0000","last_undegraded":"2023-10-09T06:30:43.764519+0000","last_fullsized":"2023-10-09T06:30:43.764519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:45:19.887880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.492004+0000","last_change":"2023-10-09T06:30:43.487909+0000","last_active":"2023-10-09T06:30:43.492004+0000","last_peered":"2023-10-09T06:30:43.492004+0000","last_clean":"2023-10-09T06:30:43.492004+0000","last_became_active":"2023-10-09T06:30:43.487634+0000","last_became_peered":"2023-10-09T06:30:43.487634+0000","last_unstale":"2023-10-09T06:30:43.492004+0000","last_undegraded":"2023-10-09T06:30:43.492004+0000","last_fullsized":"2023-10-09T06:30:43.492004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:41:00.854052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.760383+0000","last_change":"2023-10-09T06:30:42.756538+0000","last_active":"2023-10-09T06:30:43.760383+0000","last_peered":"2023-10-09T06:30:43.760383+0000","last_clean":"2023-10-09T06:30:43.760383+0000","last_became_active":"2023-10-09T06:30:42.756109+0000","last_became_peered":"2023-10-09T06:30:42.756109+0000","last_unstale":"2023-10-09T06:30:43.760383+0000","last_undegraded":"2023-10-09T06:30:43.760383+0000","last_fullsized":"2023-10-09T06:30:43.760383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:30:02.609828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.485735+0000","last_change":"2023-10-09T06:30:43.483554+0000","last_active":"2023-10-09T06:30:43.485735+0000","last_peered":"2023-10-09T06:30:43.485735+0000","last_clean":"2023-10-09T06:30:43.485735+0000","last_became_active":"2023-10-09T06:30:43.483342+0000","last_became_peered":"2023-10-09T06:30:43.483342+0000","last_unstale":"2023-10-09T06:30:43.485735+0000","last_undegraded":"2023-10-09T06:30:43.485735+0000","last_fullsized":"2023-10-09T06:30:43.485735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:06:04.546054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.478131+0000","last_change":"2023-10-09T06:30:42.754490+0000","last_active":"2023-10-09T06:30:43.478131+0000","last_peered":"2023-10-09T06:30:43.478131+0000","last_clean":"2023-10-09T06:30:43.478131+0000","last_became_active":"2023-10-09T06:30:42.754238+0000","last_became_peered":"2023-10-09T06:30:42.754238+0000","last_unstale":"2023-10-09T06:30:43.478131+0000","last_undegraded":"2023-10-09T06:30:43.478131+0000","last_fullsized":"2023-10-09T06:30:43.478131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:45:19.887880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496827+0000","last_change":"2023-10-09T06:30:43.493269+0000","last_active":"2023-10-09T06:30:43.496827+0000","last_peered":"2023-10-09T06:30:43.496827+0000","last_clean":"2023-10-09T06:30:43.496827+0000","last_became_active":"2023-10-09T06:30:43.491762+0000","last_became_peered":"2023-10-09T06:30:43.491762+0000","last_unstale":"2023-10-09T06:30:43.496827+0000","last_undegraded":"2023-10-09T06:30:43.496827+0000","last_fullsized":"2023-10-09T06:30:43.496827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:10:42.930778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.482114+0000","last_change":"2023-10-09T06:30:42.763709+0000","last_active":"2023-10-09T06:30:43.482114+0000","last_peered":"2023-10-09T06:30:43.482114+0000","last_clean":"2023-10-09T06:30:43.482114+0000","last_became_active":"2023-10-09T06:30:42.763455+0000","last_became_peered":"2023-10-09T06:30:42.763455+0000","last_unstale":"2023-10-09T06:30:43.482114+0000","last_undegraded":"2023-10-09T06:30:43.482114+0000","last_fullsized":"2023-10-09T06:30:43.482114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:59:42.424421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.490783+0000","last_change":"2023-10-09T06:30:43.486278+0000","last_active":"2023-10-09T06:30:43.490783+0000","last_peered":"2023-10-09T06:30:43.490783+0000","last_clean":"2023-10-09T06:30:43.490783+0000","last_became_active":"2023-10-09T06:30:43.485877+0000","last_became_peered":"2023-10-09T06:30:43.485877+0000","last_unstale":"2023-10-09T06:30:43.490783+0000","last_undegraded":"2023-10-09T06:30:43.490783+0000","last_fullsized":"2023-10-09T06:30:43.490783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:55:20.971921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.486699+0000","last_change":"2023-10-09T06:30:43.482444+0000","last_active":"2023-10-09T06:30:43.486699+0000","last_peered":"2023-10-09T06:30:43.486699+0000","last_clean":"2023-10-09T06:30:43.486699+0000","last_became_active":"2023-10-09T06:30:43.482168+0000","last_became_peered":"2023-10-09T06:30:43.482168+0000","last_unstale":"2023-10-09T06:30:43.486699+0000","last_undegraded":"2023-10-09T06:30:43.486699+0000","last_fullsized":"2023-10-09T06:30:43.486699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:56:48.342402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.476414+0000","last_change":"2023-10-09T06:30:42.755534+0000","last_active":"2023-10-09T06:30:43.476414+0000","last_peered":"2023-10-09T06:30:43.476414+0000","last_clean":"2023-10-09T06:30:43.476414+0000","last_became_active":"2023-10-09T06:30:42.755246+0000","last_became_peered":"2023-10-09T06:30:42.755246+0000","last_unstale":"2023-10-09T06:30:43.476414+0000","last_undegraded":"2023-10-09T06:30:43.476414+0000","last_fullsized":"2023-10-09T06:30:43.476414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:52:48.682306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493672+0000","last_change":"2023-10-09T06:30:43.488993+0000","last_active":"2023-10-09T06:30:43.493672+0000","last_peered":"2023-10-09T06:30:43.493672+0000","last_clean":"2023-10-09T06:30:43.493672+0000","last_became_active":"2023-10-09T06:30:43.488701+0000","last_became_peered":"2023-10-09T06:30:43.488701+0000","last_unstale":"2023-10-09T06:30:43.493672+0000","last_undegraded":"2023-10-09T06:30:43.493672+0000","last_fullsized":"2023-10-09T06:30:43.493672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:05:59.526736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.479498+0000","last_change":"2023-10-09T06:30:42.757897+0000","last_active":"2023-10-09T06:30:43.479498+0000","last_peered":"2023-10-09T06:30:43.479498+0000","last_clean":"2023-10-09T06:30:43.479498+0000","last_became_active":"2023-10-09T06:30:42.757432+0000","last_became_peered":"2023-10-09T06:30:42.757432+0000","last_unstale":"2023-10-09T06:30:43.479498+0000","last_undegraded":"2023-10-09T06:30:43.479498+0000","last_fullsized":"2023-10-09T06:30:43.479498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:36:40.691450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":163,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.474256+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:30:43.474256+0000","last_peered":"2023-10-09T06:30:43.474256+0000","last_clean":"2023-10-09T06:30:43.474256+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:30:43.474256+0000","last_undegraded":"2023-10-09T06:30:43.474256+0000","last_fullsized":"2023-10-09T06:30:43.474256+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.484921+0000","last_change":"2023-10-09T06:30:43.479970+0000","last_active":"2023-10-09T06:30:43.484921+0000","last_peered":"2023-10-09T06:30:43.484921+0000","last_clean":"2023-10-09T06:30:43.484921+0000","last_became_active":"2023-10-09T06:30:43.479731+0000","last_became_peered":"2023-10-09T06:30:43.479731+0000","last_unstale":"2023-10-09T06:30:43.484921+0000","last_undegraded":"2023-10-09T06:30:43.484921+0000","last_fullsized":"2023-10-09T06:30:43.484921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:11:59.504301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.475172+0000","last_change":"2023-10-09T06:30:42.763408+0000","last_active":"2023-10-09T06:30:43.475172+0000","last_peered":"2023-10-09T06:30:43.475172+0000","last_clean":"2023-10-09T06:30:43.475172+0000","last_became_active":"2023-10-09T06:30:42.763129+0000","last_became_peered":"2023-10-09T06:30:42.763129+0000","last_unstale":"2023-10-09T06:30:43.475172+0000","last_undegraded":"2023-10-09T06:30:43.475172+0000","last_fullsized":"2023-10-09T06:30:43.475172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:29:11.172110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.475746+0000","last_change":"2023-10-09T06:30:42.766229+0000","last_active":"2023-10-09T06:30:43.475746+0000","last_peered":"2023-10-09T06:30:43.475746+0000","last_clean":"2023-10-09T06:30:43.475746+0000","last_became_active":"2023-10-09T06:30:42.765968+0000","last_became_peered":"2023-10-09T06:30:42.765968+0000","last_unstale":"2023-10-09T06:30:43.475746+0000","last_undegraded":"2023-10-09T06:30:43.475746+0000","last_fullsized":"2023-10-09T06:30:43.475746+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:59:42.424421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.486657+0000","last_change":"2023-10-09T06:30:43.482450+0000","last_active":"2023-10-09T06:30:43.486657+0000","last_peered":"2023-10-09T06:30:43.486657+0000","last_clean":"2023-10-09T06:30:43.486657+0000","last_became_active":"2023-10-09T06:30:43.482187+0000","last_became_peered":"2023-10-09T06:30:43.482187+0000","last_unstale":"2023-10-09T06:30:43.486657+0000","last_undegraded":"2023-10-09T06:30:43.486657+0000","last_fullsized":"2023-10-09T06:30:43.486657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:09:10.695002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.478234+0000","last_change":"2023-10-09T06:30:42.759163+0000","last_active":"2023-10-09T06:30:43.478234+0000","last_peered":"2023-10-09T06:30:43.478234+0000","last_clean":"2023-10-09T06:30:43.478234+0000","last_became_active":"2023-10-09T06:30:42.758583+0000","last_became_peered":"2023-10-09T06:30:42.758583+0000","last_unstale":"2023-10-09T06:30:43.478234+0000","last_undegraded":"2023-10-09T06:30:43.478234+0000","last_fullsized":"2023-10-09T06:30:43.478234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:36:40.691450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.487871+0000","last_change":"2023-10-09T06:30:43.480222+0000","last_active":"2023-10-09T06:30:43.487871+0000","last_peered":"2023-10-09T06:30:43.487871+0000","last_clean":"2023-10-09T06:30:43.487871+0000","last_became_active":"2023-10-09T06:30:43.479958+0000","last_became_peered":"2023-10-09T06:30:43.479958+0000","last_unstale":"2023-10-09T06:30:43.487871+0000","last_undegraded":"2023-10-09T06:30:43.487871+0000","last_fullsized":"2023-10-09T06:30:43.487871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:57:59.094337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.761656+0000","last_change":"2023-10-09T06:30:42.757526+0000","last_active":"2023-10-09T06:30:43.761656+0000","last_peered":"2023-10-09T06:30:43.761656+0000","last_clean":"2023-10-09T06:30:43.761656+0000","last_became_active":"2023-10-09T06:30:42.756934+0000","last_became_peered":"2023-10-09T06:30:42.756934+0000","last_unstale":"2023-10-09T06:30:43.761656+0000","last_undegraded":"2023-10-09T06:30:43.761656+0000","last_fullsized":"2023-10-09T06:30:43.761656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:45:19.887880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.487633+0000","last_change":"2023-10-09T06:30:43.482622+0000","last_active":"2023-10-09T06:30:43.487633+0000","last_peered":"2023-10-09T06:30:43.487633+0000","last_clean":"2023-10-09T06:30:43.487633+0000","last_became_active":"2023-10-09T06:30:43.482362+0000","last_became_peered":"2023-10-09T06:30:43.482362+0000","last_unstale":"2023-10-09T06:30:43.487633+0000","last_undegraded":"2023-10-09T06:30:43.487633+0000","last_fullsized":"2023-10-09T06:30:43.487633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:15:54.706732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.474220+0000","last_change":"2023-10-09T06:30:42.764161+0000","last_active":"2023-10-09T06:30:43.474220+0000","last_peered":"2023-10-09T06:30:43.474220+0000","last_clean":"2023-10-09T06:30:43.474220+0000","last_became_active":"2023-10-09T06:30:42.763872+0000","last_became_peered":"2023-10-09T06:30:42.763872+0000","last_unstale":"2023-10-09T06:30:43.474220+0000","last_undegraded":"2023-10-09T06:30:43.474220+0000","last_fullsized":"2023-10-09T06:30:43.474220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:36:40.691450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.491559+0000","last_change":"2023-10-09T06:30:43.487127+0000","last_active":"2023-10-09T06:30:43.491559+0000","last_peered":"2023-10-09T06:30:43.491559+0000","last_clean":"2023-10-09T06:30:43.491559+0000","last_became_active":"2023-10-09T06:30:43.486428+0000","last_became_peered":"2023-10-09T06:30:43.486428+0000","last_unstale":"2023-10-09T06:30:43.491559+0000","last_undegraded":"2023-10-09T06:30:43.491559+0000","last_fullsized":"2023-10-09T06:30:43.491559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:59:43.429131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.478013+0000","last_change":"2023-10-09T06:30:42.758491+0000","last_active":"2023-10-09T06:30:43.478013+0000","last_peered":"2023-10-09T06:30:43.478013+0000","last_clean":"2023-10-09T06:30:43.478013+0000","last_became_active":"2023-10-09T06:30:42.757552+0000","last_became_peered":"2023-10-09T06:30:42.757552+0000","last_unstale":"2023-10-09T06:30:43.478013+0000","last_undegraded":"2023-10-09T06:30:43.478013+0000","last_fullsized":"2023-10-09T06:30:43.478013+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:11:08.967445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.490847+0000","last_change":"2023-10-09T06:30:43.483055+0000","last_active":"2023-10-09T06:30:43.490847+0000","last_peered":"2023-10-09T06:30:43.490847+0000","last_clean":"2023-10-09T06:30:43.490847+0000","last_became_active":"2023-10-09T06:30:43.482794+0000","last_became_peered":"2023-10-09T06:30:43.482794+0000","last_unstale":"2023-10-09T06:30:43.490847+0000","last_undegraded":"2023-10-09T06:30:43.490847+0000","last_fullsized":"2023-10-09T06:30:43.490847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:12:09.812942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.478093+0000","last_change":"2023-10-09T06:30:42.755096+0000","last_active":"2023-10-09T06:30:43.478093+0000","last_peered":"2023-10-09T06:30:43.478093+0000","last_clean":"2023-10-09T06:30:43.478093+0000","last_became_active":"2023-10-09T06:30:42.754834+0000","last_became_peered":"2023-10-09T06:30:42.754834+0000","last_unstale":"2023-10-09T06:30:43.478093+0000","last_undegraded":"2023-10-09T06:30:43.478093+0000","last_fullsized":"2023-10-09T06:30:43.478093+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:52:48.682306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.491004+0000","last_change":"2023-10-09T06:30:43.485563+0000","last_active":"2023-10-09T06:30:43.491004+0000","last_peered":"2023-10-09T06:30:43.491004+0000","last_clean":"2023-10-09T06:30:43.491004+0000","last_became_active":"2023-10-09T06:30:43.485051+0000","last_became_peered":"2023-10-09T06:30:43.485051+0000","last_unstale":"2023-10-09T06:30:43.491004+0000","last_undegraded":"2023-10-09T06:30:43.491004+0000","last_fullsized":"2023-10-09T06:30:43.491004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:36:01.020241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.763572+0000","last_change":"2023-10-09T06:30:42.762583+0000","last_active":"2023-10-09T06:30:43.763572+0000","last_peered":"2023-10-09T06:30:43.763572+0000","last_clean":"2023-10-09T06:30:43.763572+0000","last_became_active":"2023-10-09T06:30:42.762319+0000","last_became_peered":"2023-10-09T06:30:42.762319+0000","last_unstale":"2023-10-09T06:30:43.763572+0000","last_undegraded":"2023-10-09T06:30:43.763572+0000","last_fullsized":"2023-10-09T06:30:43.763572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:15:33.621205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.484629+0000","last_change":"2023-10-09T06:30:43.481138+0000","last_active":"2023-10-09T06:30:43.484629+0000","last_peered":"2023-10-09T06:30:43.484629+0000","last_clean":"2023-10-09T06:30:43.484629+0000","last_became_active":"2023-10-09T06:30:43.480789+0000","last_became_peered":"2023-10-09T06:30:43.480789+0000","last_unstale":"2023-10-09T06:30:43.484629+0000","last_undegraded":"2023-10-09T06:30:43.484629+0000","last_fullsized":"2023-10-09T06:30:43.484629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:12:09.812942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.475749+0000","last_change":"2023-10-09T06:30:42.754630+0000","last_active":"2023-10-09T06:30:43.475749+0000","last_peered":"2023-10-09T06:30:43.475749+0000","last_clean":"2023-10-09T06:30:43.475749+0000","last_became_active":"2023-10-09T06:30:42.754309+0000","last_became_peered":"2023-10-09T06:30:42.754309+0000","last_unstale":"2023-10-09T06:30:43.475749+0000","last_undegraded":"2023-10-09T06:30:43.475749+0000","last_fullsized":"2023-10-09T06:30:43.475749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:46:54.709506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496592+0000","last_change":"2023-10-09T06:30:43.492044+0000","last_active":"2023-10-09T06:30:43.496592+0000","last_peered":"2023-10-09T06:30:43.496592+0000","last_clean":"2023-10-09T06:30:43.496592+0000","last_became_active":"2023-10-09T06:30:43.491715+0000","last_became_peered":"2023-10-09T06:30:43.491715+0000","last_unstale":"2023-10-09T06:30:43.496592+0000","last_undegraded":"2023-10-09T06:30:43.496592+0000","last_fullsized":"2023-10-09T06:30:43.496592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:10:42.930778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.764217+0000","last_change":"2023-10-09T06:30:42.759144+0000","last_active":"2023-10-09T06:30:43.764217+0000","last_peered":"2023-10-09T06:30:43.764217+0000","last_clean":"2023-10-09T06:30:43.764217+0000","last_became_active":"2023-10-09T06:30:42.758729+0000","last_became_peered":"2023-10-09T06:30:42.758729+0000","last_unstale":"2023-10-09T06:30:43.764217+0000","last_undegraded":"2023-10-09T06:30:43.764217+0000","last_fullsized":"2023-10-09T06:30:43.764217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:46:54.709506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496783+0000","last_change":"2023-10-09T06:30:43.493008+0000","last_active":"2023-10-09T06:30:43.496783+0000","last_peered":"2023-10-09T06:30:43.496783+0000","last_clean":"2023-10-09T06:30:43.496783+0000","last_became_active":"2023-10-09T06:30:43.491584+0000","last_became_peered":"2023-10-09T06:30:43.491584+0000","last_unstale":"2023-10-09T06:30:43.496783+0000","last_undegraded":"2023-10-09T06:30:43.496783+0000","last_fullsized":"2023-10-09T06:30:43.496783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:43:44.457796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.762576+0000","last_change":"2023-10-09T06:30:42.756415+0000","last_active":"2023-10-09T06:30:43.762576+0000","last_peered":"2023-10-09T06:30:43.762576+0000","last_clean":"2023-10-09T06:30:43.762576+0000","last_became_active":"2023-10-09T06:30:42.755726+0000","last_became_peered":"2023-10-09T06:30:42.755726+0000","last_unstale":"2023-10-09T06:30:43.762576+0000","last_undegraded":"2023-10-09T06:30:43.762576+0000","last_fullsized":"2023-10-09T06:30:43.762576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:26:03.347834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.485808+0000","last_change":"2023-10-09T06:30:43.482852+0000","last_active":"2023-10-09T06:30:43.485808+0000","last_peered":"2023-10-09T06:30:43.485808+0000","last_clean":"2023-10-09T06:30:43.485808+0000","last_became_active":"2023-10-09T06:30:43.482581+0000","last_became_peered":"2023-10-09T06:30:43.482581+0000","last_unstale":"2023-10-09T06:30:43.485808+0000","last_undegraded":"2023-10-09T06:30:43.485808+0000","last_fullsized":"2023-10-09T06:30:43.485808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:03:07.803820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.490571+0000","last_change":"2023-10-09T06:30:43.485037+0000","last_active":"2023-10-09T06:30:43.490571+0000","last_peered":"2023-10-09T06:30:43.490571+0000","last_clean":"2023-10-09T06:30:43.490571+0000","last_became_active":"2023-10-09T06:30:43.484747+0000","last_became_peered":"2023-10-09T06:30:43.484747+0000","last_unstale":"2023-10-09T06:30:43.490571+0000","last_undegraded":"2023-10-09T06:30:43.490571+0000","last_fullsized":"2023-10-09T06:30:43.490571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:57:59.094337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493668+0000","last_change":"2023-10-09T06:30:43.488771+0000","last_active":"2023-10-09T06:30:43.493668+0000","last_peered":"2023-10-09T06:30:43.493668+0000","last_clean":"2023-10-09T06:30:43.493668+0000","last_became_active":"2023-10-09T06:30:43.488505+0000","last_became_peered":"2023-10-09T06:30:43.488505+0000","last_unstale":"2023-10-09T06:30:43.493668+0000","last_undegraded":"2023-10-09T06:30:43.493668+0000","last_fullsized":"2023-10-09T06:30:43.493668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:20:59.658630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496662+0000","last_change":"2023-10-09T06:30:43.492057+0000","last_active":"2023-10-09T06:30:43.496662+0000","last_peered":"2023-10-09T06:30:43.496662+0000","last_clean":"2023-10-09T06:30:43.496662+0000","last_became_active":"2023-10-09T06:30:43.491469+0000","last_became_peered":"2023-10-09T06:30:43.491469+0000","last_unstale":"2023-10-09T06:30:43.496662+0000","last_undegraded":"2023-10-09T06:30:43.496662+0000","last_fullsized":"2023-10-09T06:30:43.496662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:59:57.434765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.488149+0000","last_change":"2023-10-09T06:30:43.484938+0000","last_active":"2023-10-09T06:30:43.488149+0000","last_peered":"2023-10-09T06:30:43.488149+0000","last_clean":"2023-10-09T06:30:43.488149+0000","last_became_active":"2023-10-09T06:30:43.484188+0000","last_became_peered":"2023-10-09T06:30:43.484188+0000","last_unstale":"2023-10-09T06:30:43.488149+0000","last_undegraded":"2023-10-09T06:30:43.488149+0000","last_fullsized":"2023-10-09T06:30:43.488149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:57:59.094337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.488046+0000","last_change":"2023-10-09T06:30:43.481165+0000","last_active":"2023-10-09T06:30:43.488046+0000","last_peered":"2023-10-09T06:30:43.488046+0000","last_clean":"2023-10-09T06:30:43.488046+0000","last_became_active":"2023-10-09T06:30:43.480896+0000","last_became_peered":"2023-10-09T06:30:43.480896+0000","last_unstale":"2023-10-09T06:30:43.488046+0000","last_undegraded":"2023-10-09T06:30:43.488046+0000","last_fullsized":"2023-10-09T06:30:43.488046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:56:48.342402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.486824+0000","last_change":"2023-10-09T06:30:43.483125+0000","last_active":"2023-10-09T06:30:43.486824+0000","last_peered":"2023-10-09T06:30:43.486824+0000","last_clean":"2023-10-09T06:30:43.486824+0000","last_became_active":"2023-10-09T06:30:43.482886+0000","last_became_peered":"2023-10-09T06:30:43.482886+0000","last_unstale":"2023-10-09T06:30:43.486824+0000","last_undegraded":"2023-10-09T06:30:43.486824+0000","last_fullsized":"2023-10-09T06:30:43.486824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:15:54.706732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.484766+0000","last_change":"2023-10-09T06:30:43.479321+0000","last_active":"2023-10-09T06:30:43.484766+0000","last_peered":"2023-10-09T06:30:43.484766+0000","last_clean":"2023-10-09T06:30:43.484766+0000","last_became_active":"2023-10-09T06:30:43.477877+0000","last_became_peered":"2023-10-09T06:30:43.477877+0000","last_unstale":"2023-10-09T06:30:43.484766+0000","last_undegraded":"2023-10-09T06:30:43.484766+0000","last_fullsized":"2023-10-09T06:30:43.484766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:09:10.695002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.487894+0000","last_change":"2023-10-09T06:30:43.480511+0000","last_active":"2023-10-09T06:30:43.487894+0000","last_peered":"2023-10-09T06:30:43.487894+0000","last_clean":"2023-10-09T06:30:43.487894+0000","last_became_active":"2023-10-09T06:30:43.480250+0000","last_became_peered":"2023-10-09T06:30:43.480250+0000","last_unstale":"2023-10-09T06:30:43.487894+0000","last_undegraded":"2023-10-09T06:30:43.487894+0000","last_fullsized":"2023-10-09T06:30:43.487894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:15:54.706732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.491696+0000","last_change":"2023-10-09T06:30:43.488975+0000","last_active":"2023-10-09T06:30:43.491696+0000","last_peered":"2023-10-09T06:30:43.491696+0000","last_clean":"2023-10-09T06:30:43.491696+0000","last_became_active":"2023-10-09T06:30:43.488639+0000","last_became_peered":"2023-10-09T06:30:43.488639+0000","last_unstale":"2023-10-09T06:30:43.491696+0000","last_undegraded":"2023-10-09T06:30:43.491696+0000","last_fullsized":"2023-10-09T06:30:43.491696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:06:04.546054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.487823+0000","last_change":"2023-10-09T06:30:43.479862+0000","last_active":"2023-10-09T06:30:43.487823+0000","last_peered":"2023-10-09T06:30:43.487823+0000","last_clean":"2023-10-09T06:30:43.487823+0000","last_became_active":"2023-10-09T06:30:43.479597+0000","last_became_peered":"2023-10-09T06:30:43.479597+0000","last_unstale":"2023-10-09T06:30:43.487823+0000","last_undegraded":"2023-10-09T06:30:43.487823+0000","last_fullsized":"2023-10-09T06:30:43.487823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:09:10.695002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496763+0000","last_change":"2023-10-09T06:30:43.492872+0000","last_active":"2023-10-09T06:30:43.496763+0000","last_peered":"2023-10-09T06:30:43.496763+0000","last_clean":"2023-10-09T06:30:43.496763+0000","last_became_active":"2023-10-09T06:30:43.492634+0000","last_became_peered":"2023-10-09T06:30:43.492634+0000","last_unstale":"2023-10-09T06:30:43.496763+0000","last_undegraded":"2023-10-09T06:30:43.496763+0000","last_fullsized":"2023-10-09T06:30:43.496763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:02:03.930820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.488131+0000","last_change":"2023-10-09T06:30:43.484753+0000","last_active":"2023-10-09T06:30:43.488131+0000","last_peered":"2023-10-09T06:30:43.488131+0000","last_clean":"2023-10-09T06:30:43.488131+0000","last_became_active":"2023-10-09T06:30:43.483916+0000","last_became_peered":"2023-10-09T06:30:43.483916+0000","last_unstale":"2023-10-09T06:30:43.488131+0000","last_undegraded":"2023-10-09T06:30:43.488131+0000","last_fullsized":"2023-10-09T06:30:43.488131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:06:29.986154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.484622+0000","last_change":"2023-10-09T06:30:43.481252+0000","last_active":"2023-10-09T06:30:43.484622+0000","last_peered":"2023-10-09T06:30:43.484622+0000","last_clean":"2023-10-09T06:30:43.484622+0000","last_became_active":"2023-10-09T06:30:43.481008+0000","last_became_peered":"2023-10-09T06:30:43.481008+0000","last_unstale":"2023-10-09T06:30:43.484622+0000","last_undegraded":"2023-10-09T06:30:43.484622+0000","last_fullsized":"2023-10-09T06:30:43.484622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:02:03.930820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298060,"kb_used_data":772,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073780,"statfs":{"total":96636764160,"available":96331550720,"internally_reserved":0,"allocated":790528,"data_stored":655288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":72190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":212992,"data_stored":70814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.247}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298080,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073760,"statfs":{"total":96636764160,"available":96331530240,"internally_reserved":0,"allocated":811008,"data_stored":661632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.254}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297476,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074364,"statfs":{"total":96636764160,"available":96332148736,"internally_reserved":0,"allocated":192512,"data_stored":64366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":72190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":70780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297480,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074360,"statfs":{"total":96636764160,"available":96332144640,"internally_reserved":0,"allocated":196608,"data_stored":64456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:30:47.810 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:30:47.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.955+0000 7faaee89e700 1 Processor -- start 2023-10-09T06:30:47.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.955+0000 7faaee89e700 1 -- start start 2023-10-09T06:30:47.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.956+0000 7faaee89e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faae81574d0 0x7faae80cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:47.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.956+0000 7faaee89e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7faae806c0f0 0x7faae80b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:47.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.956+0000 7faaee89e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faae80b16a0 con 0x7faae80cc3c0 2023-10-09T06:30:47.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.956+0000 7faaee89e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faae80b1c60 con 0x7faae806c0f0 2023-10-09T06:30:47.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.956+0000 7faaee89e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faae81578a0 con 0x7faae81574d0 2023-10-09T06:30:47.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.956+0000 7faaee09d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7faae806c0f0 0x7faae80b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:47.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.956+0000 7faaed09b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7faae80cc3c0 0x7faae806b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:52444/0 (socket says 172.21.15.163:52444) 2023-10-09T06:30:47.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.956+0000 7faaed09b700 1 -- 172.21.15.163:0/782263199 learned_addr learned my addr 172.21.15.163:0/782263199 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:47.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.956+0000 7faaed89c700 1 --2- 172.21.15.163:0/782263199 >> v2:172.21.15.181:3300/0 conn(0x7faae81574d0 0x7faae80cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:47.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.956+0000 7faaee09d700 1 -- 172.21.15.163:0/782263199 >> v2:172.21.15.181:3300/0 conn(0x7faae81574d0 msgr2=0x7faae80cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:47.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.956+0000 7faaee09d700 1 --2- 172.21.15.163:0/782263199 >> v2:172.21.15.181:3300/0 conn(0x7faae81574d0 0x7faae80cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:47.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.956+0000 7faaee09d700 1 -- 172.21.15.163:0/782263199 >> v1:172.21.15.163:6789/0 conn(0x7faae80cc3c0 legacy=0x7faae806b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:47.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.956+0000 7faaee09d700 1 -- 172.21.15.163:0/782263199 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae80cdb70 con 0x7faae806c0f0 2023-10-09T06:30:47.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.956+0000 7faaec89a700 1 -- 172.21.15.163:0/782263199 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3649446997 0 0) 0x7faae80b16a0 con 0x7faae80cc3c0 2023-10-09T06:30:47.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.957+0000 7faaee09d700 1 --2- 172.21.15.163:0/782263199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faae806c0f0 0x7faae80b20f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faae8071330 tx=0x7faae0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a87a436543f212f server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:47.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.957+0000 7faaec89a700 1 -- 172.21.15.163:0/782263199 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae000e070 con 0x7faae806c0f0 2023-10-09T06:30:47.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.957+0000 7faaec89a700 1 -- 172.21.15.163:0/782263199 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae000bb60 con 0x7faae806c0f0 2023-10-09T06:30:47.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.957+0000 7faaec89a700 1 -- 172.21.15.163:0/782263199 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae0012430 con 0x7faae806c0f0 2023-10-09T06:30:47.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.957+0000 7faaee89e700 1 -- 172.21.15.163:0/782263199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faae806c0f0 msgr2=0x7faae80b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:47.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.957+0000 7faaee89e700 1 --2- 172.21.15.163:0/782263199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faae806c0f0 0x7faae80b20f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faae8071330 tx=0x7faae0009780 comp rx=0 tx=0).stop 2023-10-09T06:30:47.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.957+0000 7faaee89e700 1 -- 172.21.15.163:0/782263199 shutdown_connections 2023-10-09T06:30:47.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.957+0000 7faaee89e700 1 --2- 172.21.15.163:0/782263199 >> v2:172.21.15.181:3300/0 conn(0x7faae81574d0 0x7faae80cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:47.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.957+0000 7faaee89e700 1 --2- 172.21.15.163:0/782263199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faae806c0f0 0x7faae80b20f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:47.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.957+0000 7faaee89e700 1 -- 172.21.15.163:0/782263199 >> 172.21.15.163:0/782263199 conn(0x7faae80ce430 msgr2=0x7faae80ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:47.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.957+0000 7faaee89e700 1 -- 172.21.15.163:0/782263199 shutdown_connections 2023-10-09T06:30:47.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.957+0000 7faaee89e700 1 -- 172.21.15.163:0/782263199 wait complete. 2023-10-09T06:30:47.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.958+0000 7faaee89e700 1 Processor -- start 2023-10-09T06:30:47.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.958+0000 7faaee89e700 1 -- start start 2023-10-09T06:30:47.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.959+0000 7faaee89e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faae806c0f0 0x7faae8152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:47.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.959+0000 7faaee89e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faae80cc3c0 0x7faae80c9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:47.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.959+0000 7faaee89e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faae81574d0 0x7faae80af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:47.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.959+0000 7faaee89e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae8069e60 con 0x7faae81574d0 2023-10-09T06:30:47.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.959+0000 7faaee89e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae806a610 con 0x7faae806c0f0 2023-10-09T06:30:47.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.959+0000 7faaee89e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faae806a050 con 0x7faae80cc3c0 2023-10-09T06:30:47.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.959+0000 7faaee09d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faae81574d0 0x7faae80af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:47.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.959+0000 7faaee09d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faae81574d0 0x7faae80af540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36840/0 (socket says 172.21.15.163:36840) 2023-10-09T06:30:47.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.959+0000 7faaee09d700 1 -- 172.21.15.163:0/4173071588 learned_addr learned my addr 172.21.15.163:0/4173071588 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:47.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.959+0000 7faaed89c700 1 --2- 172.21.15.163:0/4173071588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faae806c0f0 0x7faae8152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:47.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.959+0000 7faaed09b700 1 --2- 172.21.15.163:0/4173071588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faae80cc3c0 0x7faae80c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:47.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.959+0000 7faaee09d700 1 -- 172.21.15.163:0/4173071588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faae80cc3c0 msgr2=0x7faae80c9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:47.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.959+0000 7faaee09d700 1 --2- 172.21.15.163:0/4173071588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faae80cc3c0 0x7faae80c9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:47.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.959+0000 7faaee09d700 1 -- 172.21.15.163:0/4173071588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faae806c0f0 msgr2=0x7faae8152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:47.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.959+0000 7faaee09d700 1 --2- 172.21.15.163:0/4173071588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faae806c0f0 0x7faae8152b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:47.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.959+0000 7faaee09d700 1 -- 172.21.15.163:0/4173071588 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae0009a20 con 0x7faae81574d0 2023-10-09T06:30:47.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.960+0000 7faaee09d700 1 --2- 172.21.15.163:0/4173071588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faae81574d0 0x7faae80af540 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faae00099f0 tx=0x7faae0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:47.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.960+0000 7faaddffb700 1 -- 172.21.15.163:0/4173071588 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae0027470 con 0x7faae81574d0 2023-10-09T06:30:47.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.960+0000 7faaddffb700 1 -- 172.21.15.163:0/4173071588 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae0027610 con 0x7faae81574d0 2023-10-09T06:30:47.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.960+0000 7faaee89e700 1 -- 172.21.15.163:0/4173071588 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae806a330 con 0x7faae81574d0 2023-10-09T06:30:47.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.960+0000 7faaddffb700 1 -- 172.21.15.163:0/4173071588 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae00279d0 con 0x7faae81574d0 2023-10-09T06:30:47.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.960+0000 7faaee89e700 1 -- 172.21.15.163:0/4173071588 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae80c79f0 con 0x7faae81574d0 2023-10-09T06:30:47.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.961+0000 7faaee89e700 1 -- 172.21.15.163:0/4173071588 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faae8153090 con 0x7faae81574d0 2023-10-09T06:30:47.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.961+0000 7faaddffb700 1 -- 172.21.15.163:0/4173071588 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faae0024070 con 0x7faae81574d0 2023-10-09T06:30:47.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.962+0000 7faaddffb700 1 --2- 172.21.15.163:0/4173071588 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faacc066f30 0x7faacc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:47.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.962+0000 7faaddffb700 1 -- 172.21.15.163:0/4173071588 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7faae0086af0 con 0x7faae81574d0 2023-10-09T06:30:47.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.962+0000 7faaed89c700 1 --2- 172.21.15.163:0/4173071588 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faacc066f30 0x7faacc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:47.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.962+0000 7faaed89c700 1 --2- 172.21.15.163:0/4173071588 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faacc066f30 0x7faacc0693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faae4009d00 tx=0x7faae4009120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:47.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:47.966+0000 7faaddffb700 1 -- 172.21.15.163:0/4173071588 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faae0053d60 con 0x7faae81574d0 2023-10-09T06:30:48.145 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.144+0000 7faaee89e700 1 -- 172.21.15.163:0/4173071588 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faae804c3c0 con 0x7faacc066f30 2023-10-09T06:30:48.152 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.152+0000 7faaddffb700 1 -- 172.21.15.163:0/4173071588 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264535 (secure 0 0 0) 0x7faae804c3c0 con 0x7faacc066f30 2023-10-09T06:30:48.156 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:30:48.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.155+0000 7faaee89e700 1 -- 172.21.15.163:0/4173071588 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faacc066f30 msgr2=0x7faacc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:48.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.155+0000 7faaee89e700 1 --2- 172.21.15.163:0/4173071588 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faacc066f30 0x7faacc0693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faae4009d00 tx=0x7faae4009120 comp rx=0 tx=0).stop 2023-10-09T06:30:48.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.155+0000 7faaee89e700 1 -- 172.21.15.163:0/4173071588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faae81574d0 msgr2=0x7faae80af540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:48.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.155+0000 7faaee89e700 1 --2- 172.21.15.163:0/4173071588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faae81574d0 0x7faae80af540 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faae00099f0 tx=0x7faae0019000 comp rx=0 tx=0).stop 2023-10-09T06:30:48.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.155+0000 7faaee89e700 1 -- 172.21.15.163:0/4173071588 shutdown_connections 2023-10-09T06:30:48.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.155+0000 7faaee89e700 1 --2- 172.21.15.163:0/4173071588 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faacc066f30 0x7faacc0693e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:48.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.155+0000 7faaee89e700 1 --2- 172.21.15.163:0/4173071588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faae81574d0 0x7faae80af540 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:48.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.155+0000 7faaee89e700 1 --2- 172.21.15.163:0/4173071588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faae806c0f0 0x7faae8152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:48.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.155+0000 7faaee89e700 1 --2- 172.21.15.163:0/4173071588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faae80cc3c0 0x7faae80c9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:48.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.155+0000 7faaee89e700 1 -- 172.21.15.163:0/4173071588 >> 172.21.15.163:0/4173071588 conn(0x7faae80ce430 msgr2=0x7faae8154990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:48.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.155+0000 7faaee89e700 1 -- 172.21.15.163:0/4173071588 shutdown_connections 2023-10-09T06:30:48.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.155+0000 7faaee89e700 1 -- 172.21.15.163:0/4173071588 wait complete. 2023-10-09T06:30:48.160 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:30:48.167 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-09T06:30:46.295024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":122,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381092,"kb_used_data":2788,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593628,"statfs":{"total":773094113280,"available":770655875072,"internally_reserved":0,"allocated":2854912,"data_stored":1731716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.556757"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.492500+0000","last_change":"2023-10-09T06:30:43.487077+0000","last_active":"2023-10-09T06:30:43.492500+0000","last_peered":"2023-10-09T06:30:43.492500+0000","last_clean":"2023-10-09T06:30:43.492500+0000","last_became_active":"2023-10-09T06:30:43.486817+0000","last_became_peered":"2023-10-09T06:30:43.486817+0000","last_unstale":"2023-10-09T06:30:43.492500+0000","last_undegraded":"2023-10-09T06:30:43.492500+0000","last_fullsized":"2023-10-09T06:30:43.492500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:59:57.434765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.486972+0000","last_change":"2023-10-09T06:30:43.484119+0000","last_active":"2023-10-09T06:30:43.486972+0000","last_peered":"2023-10-09T06:30:43.486972+0000","last_clean":"2023-10-09T06:30:43.486972+0000","last_became_active":"2023-10-09T06:30:43.483865+0000","last_became_peered":"2023-10-09T06:30:43.483865+0000","last_unstale":"2023-10-09T06:30:43.486972+0000","last_undegraded":"2023-10-09T06:30:43.486972+0000","last_fullsized":"2023-10-09T06:30:43.486972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:06:29.986154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.484893+0000","last_change":"2023-10-09T06:30:43.480754+0000","last_active":"2023-10-09T06:30:43.484893+0000","last_peered":"2023-10-09T06:30:43.484893+0000","last_clean":"2023-10-09T06:30:43.484893+0000","last_became_active":"2023-10-09T06:30:43.480474+0000","last_became_peered":"2023-10-09T06:30:43.480474+0000","last_unstale":"2023-10-09T06:30:43.484893+0000","last_undegraded":"2023-10-09T06:30:43.484893+0000","last_fullsized":"2023-10-09T06:30:43.484893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:56:01.192585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493176+0000","last_change":"2023-10-09T06:30:43.489012+0000","last_active":"2023-10-09T06:30:43.493176+0000","last_peered":"2023-10-09T06:30:43.493176+0000","last_clean":"2023-10-09T06:30:43.493176+0000","last_became_active":"2023-10-09T06:30:43.488614+0000","last_became_peered":"2023-10-09T06:30:43.488614+0000","last_unstale":"2023-10-09T06:30:43.493176+0000","last_undegraded":"2023-10-09T06:30:43.493176+0000","last_fullsized":"2023-10-09T06:30:43.493176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:41:00.854052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.486903+0000","last_change":"2023-10-09T06:30:43.483013+0000","last_active":"2023-10-09T06:30:43.486903+0000","last_peered":"2023-10-09T06:30:43.486903+0000","last_clean":"2023-10-09T06:30:43.486903+0000","last_became_active":"2023-10-09T06:30:43.482373+0000","last_became_peered":"2023-10-09T06:30:43.482373+0000","last_unstale":"2023-10-09T06:30:43.486903+0000","last_undegraded":"2023-10-09T06:30:43.486903+0000","last_fullsized":"2023-10-09T06:30:43.486903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:46:57.531379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.492270+0000","last_change":"2023-10-09T06:30:43.488813+0000","last_active":"2023-10-09T06:30:43.492270+0000","last_peered":"2023-10-09T06:30:43.492270+0000","last_clean":"2023-10-09T06:30:43.492270+0000","last_became_active":"2023-10-09T06:30:43.488532+0000","last_became_peered":"2023-10-09T06:30:43.488532+0000","last_unstale":"2023-10-09T06:30:43.492270+0000","last_undegraded":"2023-10-09T06:30:43.492270+0000","last_fullsized":"2023-10-09T06:30:43.492270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:57:57.207775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.491984+0000","last_change":"2023-10-09T06:30:43.486418+0000","last_active":"2023-10-09T06:30:43.491984+0000","last_peered":"2023-10-09T06:30:43.491984+0000","last_clean":"2023-10-09T06:30:43.491984+0000","last_became_active":"2023-10-09T06:30:43.486128+0000","last_became_peered":"2023-10-09T06:30:43.486128+0000","last_unstale":"2023-10-09T06:30:43.491984+0000","last_undegraded":"2023-10-09T06:30:43.491984+0000","last_fullsized":"2023-10-09T06:30:43.491984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:57:57.207775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493719+0000","last_change":"2023-10-09T06:30:43.490024+0000","last_active":"2023-10-09T06:30:43.493719+0000","last_peered":"2023-10-09T06:30:43.493719+0000","last_clean":"2023-10-09T06:30:43.493719+0000","last_became_active":"2023-10-09T06:30:43.489785+0000","last_became_peered":"2023-10-09T06:30:43.489785+0000","last_unstale":"2023-10-09T06:30:43.493719+0000","last_undegraded":"2023-10-09T06:30:43.493719+0000","last_fullsized":"2023-10-09T06:30:43.493719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:02:03.930820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496624+0000","last_change":"2023-10-09T06:30:43.492372+0000","last_active":"2023-10-09T06:30:43.496624+0000","last_peered":"2023-10-09T06:30:43.496624+0000","last_clean":"2023-10-09T06:30:43.496624+0000","last_became_active":"2023-10-09T06:30:43.492120+0000","last_became_peered":"2023-10-09T06:30:43.492120+0000","last_unstale":"2023-10-09T06:30:43.496624+0000","last_undegraded":"2023-10-09T06:30:43.496624+0000","last_fullsized":"2023-10-09T06:30:43.496624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:20:42.701396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493788+0000","last_change":"2023-10-09T06:30:43.489507+0000","last_active":"2023-10-09T06:30:43.493788+0000","last_peered":"2023-10-09T06:30:43.493788+0000","last_clean":"2023-10-09T06:30:43.493788+0000","last_became_active":"2023-10-09T06:30:43.489279+0000","last_became_peered":"2023-10-09T06:30:43.489279+0000","last_unstale":"2023-10-09T06:30:43.493788+0000","last_undegraded":"2023-10-09T06:30:43.493788+0000","last_fullsized":"2023-10-09T06:30:43.493788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:05:59.526736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496502+0000","last_change":"2023-10-09T06:30:43.491554+0000","last_active":"2023-10-09T06:30:43.496502+0000","last_peered":"2023-10-09T06:30:43.496502+0000","last_clean":"2023-10-09T06:30:43.496502+0000","last_became_active":"2023-10-09T06:30:43.491305+0000","last_became_peered":"2023-10-09T06:30:43.491305+0000","last_unstale":"2023-10-09T06:30:43.496502+0000","last_undegraded":"2023-10-09T06:30:43.496502+0000","last_fullsized":"2023-10-09T06:30:43.496502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:15:54.706732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.483269+0000","last_change":"2023-10-09T06:30:43.478613+0000","last_active":"2023-10-09T06:30:43.483269+0000","last_peered":"2023-10-09T06:30:43.483269+0000","last_clean":"2023-10-09T06:30:43.483269+0000","last_became_active":"2023-10-09T06:30:43.477580+0000","last_became_peered":"2023-10-09T06:30:43.477580+0000","last_unstale":"2023-10-09T06:30:43.483269+0000","last_undegraded":"2023-10-09T06:30:43.483269+0000","last_fullsized":"2023-10-09T06:30:43.483269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:15:34.625915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.491028+0000","last_change":"2023-10-09T06:30:43.485665+0000","last_active":"2023-10-09T06:30:43.491028+0000","last_peered":"2023-10-09T06:30:43.491028+0000","last_clean":"2023-10-09T06:30:43.491028+0000","last_became_active":"2023-10-09T06:30:43.485346+0000","last_became_peered":"2023-10-09T06:30:43.485346+0000","last_unstale":"2023-10-09T06:30:43.491028+0000","last_undegraded":"2023-10-09T06:30:43.491028+0000","last_fullsized":"2023-10-09T06:30:43.491028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:56:01.192585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.487663+0000","last_change":"2023-10-09T06:30:43.483174+0000","last_active":"2023-10-09T06:30:43.487663+0000","last_peered":"2023-10-09T06:30:43.487663+0000","last_clean":"2023-10-09T06:30:43.487663+0000","last_became_active":"2023-10-09T06:30:43.482518+0000","last_became_peered":"2023-10-09T06:30:43.482518+0000","last_unstale":"2023-10-09T06:30:43.487663+0000","last_undegraded":"2023-10-09T06:30:43.487663+0000","last_fullsized":"2023-10-09T06:30:43.487663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:56:48.342402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493620+0000","last_change":"2023-10-09T06:30:43.489829+0000","last_active":"2023-10-09T06:30:43.493620+0000","last_peered":"2023-10-09T06:30:43.493620+0000","last_clean":"2023-10-09T06:30:43.493620+0000","last_became_active":"2023-10-09T06:30:43.489549+0000","last_became_peered":"2023-10-09T06:30:43.489549+0000","last_unstale":"2023-10-09T06:30:43.493620+0000","last_undegraded":"2023-10-09T06:30:43.493620+0000","last_fullsized":"2023-10-09T06:30:43.493620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:56:01.192585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496629+0000","last_change":"2023-10-09T06:30:43.491356+0000","last_active":"2023-10-09T06:30:43.496629+0000","last_peered":"2023-10-09T06:30:43.496629+0000","last_clean":"2023-10-09T06:30:43.496629+0000","last_became_active":"2023-10-09T06:30:43.491058+0000","last_became_peered":"2023-10-09T06:30:43.491058+0000","last_unstale":"2023-10-09T06:30:43.496629+0000","last_undegraded":"2023-10-09T06:30:43.496629+0000","last_fullsized":"2023-10-09T06:30:43.496629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:03:07.803820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.492196+0000","last_change":"2023-10-09T06:30:43.489230+0000","last_active":"2023-10-09T06:30:43.492196+0000","last_peered":"2023-10-09T06:30:43.492196+0000","last_clean":"2023-10-09T06:30:43.492196+0000","last_became_active":"2023-10-09T06:30:43.488094+0000","last_became_peered":"2023-10-09T06:30:43.488094+0000","last_unstale":"2023-10-09T06:30:43.492196+0000","last_undegraded":"2023-10-09T06:30:43.492196+0000","last_fullsized":"2023-10-09T06:30:43.492196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:36:01.020241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.488118+0000","last_change":"2023-10-09T06:30:43.486270+0000","last_active":"2023-10-09T06:30:43.488118+0000","last_peered":"2023-10-09T06:30:43.488118+0000","last_clean":"2023-10-09T06:30:43.488118+0000","last_became_active":"2023-10-09T06:30:43.486003+0000","last_became_peered":"2023-10-09T06:30:43.486003+0000","last_unstale":"2023-10-09T06:30:43.488118+0000","last_undegraded":"2023-10-09T06:30:43.488118+0000","last_fullsized":"2023-10-09T06:30:43.488118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:55:20.971921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.485740+0000","last_change":"2023-10-09T06:30:43.482479+0000","last_active":"2023-10-09T06:30:43.485740+0000","last_peered":"2023-10-09T06:30:43.485740+0000","last_clean":"2023-10-09T06:30:43.485740+0000","last_became_active":"2023-10-09T06:30:43.482203+0000","last_became_peered":"2023-10-09T06:30:43.482203+0000","last_unstale":"2023-10-09T06:30:43.485740+0000","last_undegraded":"2023-10-09T06:30:43.485740+0000","last_fullsized":"2023-10-09T06:30:43.485740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:41:39.976781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.761744+0000","last_change":"2023-10-09T06:30:42.761358+0000","last_active":"2023-10-09T06:30:43.761744+0000","last_peered":"2023-10-09T06:30:43.761744+0000","last_clean":"2023-10-09T06:30:43.761744+0000","last_became_active":"2023-10-09T06:30:42.761139+0000","last_became_peered":"2023-10-09T06:30:42.761139+0000","last_unstale":"2023-10-09T06:30:43.761744+0000","last_undegraded":"2023-10-09T06:30:43.761744+0000","last_fullsized":"2023-10-09T06:30:43.761744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:27:28.658955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.490991+0000","last_change":"2023-10-09T06:30:43.485519+0000","last_active":"2023-10-09T06:30:43.490991+0000","last_peered":"2023-10-09T06:30:43.490991+0000","last_clean":"2023-10-09T06:30:43.490991+0000","last_became_active":"2023-10-09T06:30:43.485289+0000","last_became_peered":"2023-10-09T06:30:43.485289+0000","last_unstale":"2023-10-09T06:30:43.490991+0000","last_undegraded":"2023-10-09T06:30:43.490991+0000","last_fullsized":"2023-10-09T06:30:43.490991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:41:39.976781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.764141+0000","last_change":"2023-10-09T06:30:42.758771+0000","last_active":"2023-10-09T06:30:43.764141+0000","last_peered":"2023-10-09T06:30:43.764141+0000","last_clean":"2023-10-09T06:30:43.764141+0000","last_became_active":"2023-10-09T06:30:42.756764+0000","last_became_peered":"2023-10-09T06:30:42.756764+0000","last_unstale":"2023-10-09T06:30:43.764141+0000","last_undegraded":"2023-10-09T06:30:43.764141+0000","last_fullsized":"2023-10-09T06:30:43.764141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:26:03.347834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493888+0000","last_change":"2023-10-09T06:30:43.491484+0000","last_active":"2023-10-09T06:30:43.493888+0000","last_peered":"2023-10-09T06:30:43.493888+0000","last_clean":"2023-10-09T06:30:43.493888+0000","last_became_active":"2023-10-09T06:30:43.491319+0000","last_became_peered":"2023-10-09T06:30:43.491319+0000","last_unstale":"2023-10-09T06:30:43.493888+0000","last_undegraded":"2023-10-09T06:30:43.493888+0000","last_fullsized":"2023-10-09T06:30:43.493888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:12:09.812942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.761120+0000","last_change":"2023-10-09T06:30:42.763648+0000","last_active":"2023-10-09T06:30:43.761120+0000","last_peered":"2023-10-09T06:30:43.761120+0000","last_clean":"2023-10-09T06:30:43.761120+0000","last_became_active":"2023-10-09T06:30:42.763348+0000","last_became_peered":"2023-10-09T06:30:42.763348+0000","last_unstale":"2023-10-09T06:30:43.761120+0000","last_undegraded":"2023-10-09T06:30:43.761120+0000","last_fullsized":"2023-10-09T06:30:43.761120+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:55:19.967211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496719+0000","last_change":"2023-10-09T06:30:43.491778+0000","last_active":"2023-10-09T06:30:43.496719+0000","last_peered":"2023-10-09T06:30:43.496719+0000","last_clean":"2023-10-09T06:30:43.496719+0000","last_became_active":"2023-10-09T06:30:43.491137+0000","last_became_peered":"2023-10-09T06:30:43.491137+0000","last_unstale":"2023-10-09T06:30:43.496719+0000","last_undegraded":"2023-10-09T06:30:43.496719+0000","last_fullsized":"2023-10-09T06:30:43.496719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:46:57.531379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.761164+0000","last_change":"2023-10-09T06:30:42.761346+0000","last_active":"2023-10-09T06:30:43.761164+0000","last_peered":"2023-10-09T06:30:43.761164+0000","last_clean":"2023-10-09T06:30:43.761164+0000","last_became_active":"2023-10-09T06:30:42.761028+0000","last_became_peered":"2023-10-09T06:30:42.761028+0000","last_unstale":"2023-10-09T06:30:43.761164+0000","last_undegraded":"2023-10-09T06:30:43.761164+0000","last_fullsized":"2023-10-09T06:30:43.761164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:20:58.653921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.492949+0000","last_change":"2023-10-09T06:30:43.489975+0000","last_active":"2023-10-09T06:30:43.492949+0000","last_peered":"2023-10-09T06:30:43.492949+0000","last_clean":"2023-10-09T06:30:43.492949+0000","last_became_active":"2023-10-09T06:30:43.489712+0000","last_became_peered":"2023-10-09T06:30:43.489712+0000","last_unstale":"2023-10-09T06:30:43.492949+0000","last_undegraded":"2023-10-09T06:30:43.492949+0000","last_fullsized":"2023-10-09T06:30:43.492949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:41:39.976781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.762311+0000","last_change":"2023-10-09T06:30:42.757689+0000","last_active":"2023-10-09T06:30:43.762311+0000","last_peered":"2023-10-09T06:30:43.762311+0000","last_clean":"2023-10-09T06:30:43.762311+0000","last_became_active":"2023-10-09T06:30:42.757270+0000","last_became_peered":"2023-10-09T06:30:42.757270+0000","last_unstale":"2023-10-09T06:30:43.762311+0000","last_undegraded":"2023-10-09T06:30:43.762311+0000","last_fullsized":"2023-10-09T06:30:43.762311+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:52:48.682306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.490754+0000","last_change":"2023-10-09T06:30:43.486187+0000","last_active":"2023-10-09T06:30:43.490754+0000","last_peered":"2023-10-09T06:30:43.490754+0000","last_clean":"2023-10-09T06:30:43.490754+0000","last_became_active":"2023-10-09T06:30:43.485917+0000","last_became_peered":"2023-10-09T06:30:43.485917+0000","last_unstale":"2023-10-09T06:30:43.490754+0000","last_undegraded":"2023-10-09T06:30:43.490754+0000","last_fullsized":"2023-10-09T06:30:43.490754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:20:59.658630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.763447+0000","last_change":"2023-10-09T06:30:42.758939+0000","last_active":"2023-10-09T06:30:43.763447+0000","last_peered":"2023-10-09T06:30:43.763447+0000","last_clean":"2023-10-09T06:30:43.763447+0000","last_became_active":"2023-10-09T06:30:42.758667+0000","last_became_peered":"2023-10-09T06:30:42.758667+0000","last_unstale":"2023-10-09T06:30:43.763447+0000","last_undegraded":"2023-10-09T06:30:43.763447+0000","last_fullsized":"2023-10-09T06:30:43.763447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:59:42.424421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.488120+0000","last_change":"2023-10-09T06:30:43.484481+0000","last_active":"2023-10-09T06:30:43.488120+0000","last_peered":"2023-10-09T06:30:43.488120+0000","last_clean":"2023-10-09T06:30:43.488120+0000","last_became_active":"2023-10-09T06:30:43.483640+0000","last_became_peered":"2023-10-09T06:30:43.483640+0000","last_unstale":"2023-10-09T06:30:43.488120+0000","last_undegraded":"2023-10-09T06:30:43.488120+0000","last_fullsized":"2023-10-09T06:30:43.488120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:38:10.342140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.474106+0000","last_change":"2023-10-09T06:30:42.758834+0000","last_active":"2023-10-09T06:30:43.474106+0000","last_peered":"2023-10-09T06:30:43.474106+0000","last_clean":"2023-10-09T06:30:43.474106+0000","last_became_active":"2023-10-09T06:30:42.758561+0000","last_became_peered":"2023-10-09T06:30:42.758561+0000","last_unstale":"2023-10-09T06:30:43.474106+0000","last_undegraded":"2023-10-09T06:30:43.474106+0000","last_fullsized":"2023-10-09T06:30:43.474106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:46:54.709506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.490866+0000","last_change":"2023-10-09T06:30:43.483623+0000","last_active":"2023-10-09T06:30:43.490866+0000","last_peered":"2023-10-09T06:30:43.490866+0000","last_clean":"2023-10-09T06:30:43.490866+0000","last_became_active":"2023-10-09T06:30:43.483435+0000","last_became_peered":"2023-10-09T06:30:43.483435+0000","last_unstale":"2023-10-09T06:30:43.490866+0000","last_undegraded":"2023-10-09T06:30:43.490866+0000","last_fullsized":"2023-10-09T06:30:43.490866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:57:57.207775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.485588+0000","last_change":"2023-10-09T06:30:42.761685+0000","last_active":"2023-10-09T06:30:43.485588+0000","last_peered":"2023-10-09T06:30:43.485588+0000","last_clean":"2023-10-09T06:30:43.485588+0000","last_became_active":"2023-10-09T06:30:42.761432+0000","last_became_peered":"2023-10-09T06:30:42.761432+0000","last_unstale":"2023-10-09T06:30:43.485588+0000","last_undegraded":"2023-10-09T06:30:43.485588+0000","last_fullsized":"2023-10-09T06:30:43.485588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:11:08.967445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.486800+0000","last_change":"2023-10-09T06:30:43.483621+0000","last_active":"2023-10-09T06:30:43.486800+0000","last_peered":"2023-10-09T06:30:43.486800+0000","last_clean":"2023-10-09T06:30:43.486800+0000","last_became_active":"2023-10-09T06:30:43.483365+0000","last_became_peered":"2023-10-09T06:30:43.483365+0000","last_unstale":"2023-10-09T06:30:43.486800+0000","last_undegraded":"2023-10-09T06:30:43.486800+0000","last_fullsized":"2023-10-09T06:30:43.486800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:57:59.094337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.470554+0000","last_change":"2023-10-09T06:30:42.757148+0000","last_active":"2023-10-09T06:30:43.470554+0000","last_peered":"2023-10-09T06:30:43.470554+0000","last_clean":"2023-10-09T06:30:43.470554+0000","last_became_active":"2023-10-09T06:30:42.756630+0000","last_became_peered":"2023-10-09T06:30:42.756630+0000","last_unstale":"2023-10-09T06:30:43.470554+0000","last_undegraded":"2023-10-09T06:30:43.470554+0000","last_fullsized":"2023-10-09T06:30:43.470554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:11:08.967445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496741+0000","last_change":"2023-10-09T06:30:43.492800+0000","last_active":"2023-10-09T06:30:43.496741+0000","last_peered":"2023-10-09T06:30:43.496741+0000","last_clean":"2023-10-09T06:30:43.496741+0000","last_became_active":"2023-10-09T06:30:43.492567+0000","last_became_peered":"2023-10-09T06:30:43.492567+0000","last_unstale":"2023-10-09T06:30:43.496741+0000","last_undegraded":"2023-10-09T06:30:43.496741+0000","last_fullsized":"2023-10-09T06:30:43.496741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:05:59.526736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.482776+0000","last_change":"2023-10-09T06:30:42.759966+0000","last_active":"2023-10-09T06:30:43.482776+0000","last_peered":"2023-10-09T06:30:43.482776+0000","last_clean":"2023-10-09T06:30:43.482776+0000","last_became_active":"2023-10-09T06:30:42.759686+0000","last_became_peered":"2023-10-09T06:30:42.759686+0000","last_unstale":"2023-10-09T06:30:43.482776+0000","last_undegraded":"2023-10-09T06:30:43.482776+0000","last_fullsized":"2023-10-09T06:30:43.482776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:46:54.709506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493906+0000","last_change":"2023-10-09T06:30:43.491580+0000","last_active":"2023-10-09T06:30:43.493906+0000","last_peered":"2023-10-09T06:30:43.493906+0000","last_clean":"2023-10-09T06:30:43.493906+0000","last_became_active":"2023-10-09T06:30:43.491323+0000","last_became_peered":"2023-10-09T06:30:43.491323+0000","last_unstale":"2023-10-09T06:30:43.493906+0000","last_undegraded":"2023-10-09T06:30:43.493906+0000","last_fullsized":"2023-10-09T06:30:43.493906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:02:03.930820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.761837+0000","last_change":"2023-10-09T06:30:42.758290+0000","last_active":"2023-10-09T06:30:43.761837+0000","last_peered":"2023-10-09T06:30:43.761837+0000","last_clean":"2023-10-09T06:30:43.761837+0000","last_became_active":"2023-10-09T06:30:42.758008+0000","last_became_peered":"2023-10-09T06:30:42.758008+0000","last_unstale":"2023-10-09T06:30:43.761837+0000","last_undegraded":"2023-10-09T06:30:43.761837+0000","last_fullsized":"2023-10-09T06:30:43.761837+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:20:58.653921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.483249+0000","last_change":"2023-10-09T06:30:43.478610+0000","last_active":"2023-10-09T06:30:43.483249+0000","last_peered":"2023-10-09T06:30:43.483249+0000","last_clean":"2023-10-09T06:30:43.483249+0000","last_became_active":"2023-10-09T06:30:43.477942+0000","last_became_peered":"2023-10-09T06:30:43.477942+0000","last_unstale":"2023-10-09T06:30:43.483249+0000","last_undegraded":"2023-10-09T06:30:43.483249+0000","last_fullsized":"2023-10-09T06:30:43.483249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:29:12.176820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.766388+0000","last_change":"2023-10-09T06:30:42.759743+0000","last_active":"2023-10-09T06:30:43.766388+0000","last_peered":"2023-10-09T06:30:43.766388+0000","last_clean":"2023-10-09T06:30:43.766388+0000","last_became_active":"2023-10-09T06:30:42.757647+0000","last_became_peered":"2023-10-09T06:30:42.757647+0000","last_unstale":"2023-10-09T06:30:43.766388+0000","last_undegraded":"2023-10-09T06:30:43.766388+0000","last_fullsized":"2023-10-09T06:30:43.766388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:46:54.709506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.485759+0000","last_change":"2023-10-09T06:30:43.484192+0000","last_active":"2023-10-09T06:30:43.485759+0000","last_peered":"2023-10-09T06:30:43.485759+0000","last_clean":"2023-10-09T06:30:43.485759+0000","last_became_active":"2023-10-09T06:30:43.483935+0000","last_became_peered":"2023-10-09T06:30:43.483935+0000","last_unstale":"2023-10-09T06:30:43.485759+0000","last_undegraded":"2023-10-09T06:30:43.485759+0000","last_fullsized":"2023-10-09T06:30:43.485759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:36:01.020241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.763949+0000","last_change":"2023-10-09T06:30:42.760604+0000","last_active":"2023-10-09T06:30:43.763949+0000","last_peered":"2023-10-09T06:30:43.763949+0000","last_clean":"2023-10-09T06:30:43.763949+0000","last_became_active":"2023-10-09T06:30:42.760363+0000","last_became_peered":"2023-10-09T06:30:42.760363+0000","last_unstale":"2023-10-09T06:30:43.763949+0000","last_undegraded":"2023-10-09T06:30:43.763949+0000","last_fullsized":"2023-10-09T06:30:43.763949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:26:03.347834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.484683+0000","last_change":"2023-10-09T06:30:43.479684+0000","last_active":"2023-10-09T06:30:43.484683+0000","last_peered":"2023-10-09T06:30:43.484683+0000","last_clean":"2023-10-09T06:30:43.484683+0000","last_became_active":"2023-10-09T06:30:43.479262+0000","last_became_peered":"2023-10-09T06:30:43.479262+0000","last_unstale":"2023-10-09T06:30:43.484683+0000","last_undegraded":"2023-10-09T06:30:43.484683+0000","last_fullsized":"2023-10-09T06:30:43.484683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:09:10.695002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.482211+0000","last_change":"2023-10-09T06:30:42.762087+0000","last_active":"2023-10-09T06:30:43.482211+0000","last_peered":"2023-10-09T06:30:43.482211+0000","last_clean":"2023-10-09T06:30:43.482211+0000","last_became_active":"2023-10-09T06:30:42.761818+0000","last_became_peered":"2023-10-09T06:30:42.761818+0000","last_unstale":"2023-10-09T06:30:43.482211+0000","last_undegraded":"2023-10-09T06:30:43.482211+0000","last_fullsized":"2023-10-09T06:30:43.482211+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:45:19.887880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.486923+0000","last_change":"2023-10-09T06:30:43.483220+0000","last_active":"2023-10-09T06:30:43.486923+0000","last_peered":"2023-10-09T06:30:43.486923+0000","last_clean":"2023-10-09T06:30:43.486923+0000","last_became_active":"2023-10-09T06:30:43.482733+0000","last_became_peered":"2023-10-09T06:30:43.482733+0000","last_unstale":"2023-10-09T06:30:43.486923+0000","last_undegraded":"2023-10-09T06:30:43.486923+0000","last_fullsized":"2023-10-09T06:30:43.486923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:43:44.457796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.470076+0000","last_change":"2023-10-09T06:30:42.759270+0000","last_active":"2023-10-09T06:30:43.470076+0000","last_peered":"2023-10-09T06:30:43.470076+0000","last_clean":"2023-10-09T06:30:43.470076+0000","last_became_active":"2023-10-09T06:30:42.758836+0000","last_became_peered":"2023-10-09T06:30:42.758836+0000","last_unstale":"2023-10-09T06:30:43.470076+0000","last_undegraded":"2023-10-09T06:30:43.470076+0000","last_fullsized":"2023-10-09T06:30:43.470076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:36:40.691450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.484511+0000","last_change":"2023-10-09T06:30:43.480359+0000","last_active":"2023-10-09T06:30:43.484511+0000","last_peered":"2023-10-09T06:30:43.484511+0000","last_clean":"2023-10-09T06:30:43.484511+0000","last_became_active":"2023-10-09T06:30:43.480040+0000","last_became_peered":"2023-10-09T06:30:43.480040+0000","last_unstale":"2023-10-09T06:30:43.484511+0000","last_undegraded":"2023-10-09T06:30:43.484511+0000","last_fullsized":"2023-10-09T06:30:43.484511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:06:29.986154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.764519+0000","last_change":"2023-10-09T06:30:42.756126+0000","last_active":"2023-10-09T06:30:43.764519+0000","last_peered":"2023-10-09T06:30:43.764519+0000","last_clean":"2023-10-09T06:30:43.764519+0000","last_became_active":"2023-10-09T06:30:42.755132+0000","last_became_peered":"2023-10-09T06:30:42.755132+0000","last_unstale":"2023-10-09T06:30:43.764519+0000","last_undegraded":"2023-10-09T06:30:43.764519+0000","last_fullsized":"2023-10-09T06:30:43.764519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:45:19.887880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.492004+0000","last_change":"2023-10-09T06:30:43.487909+0000","last_active":"2023-10-09T06:30:43.492004+0000","last_peered":"2023-10-09T06:30:43.492004+0000","last_clean":"2023-10-09T06:30:43.492004+0000","last_became_active":"2023-10-09T06:30:43.487634+0000","last_became_peered":"2023-10-09T06:30:43.487634+0000","last_unstale":"2023-10-09T06:30:43.492004+0000","last_undegraded":"2023-10-09T06:30:43.492004+0000","last_fullsized":"2023-10-09T06:30:43.492004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:41:00.854052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.760383+0000","last_change":"2023-10-09T06:30:42.756538+0000","last_active":"2023-10-09T06:30:43.760383+0000","last_peered":"2023-10-09T06:30:43.760383+0000","last_clean":"2023-10-09T06:30:43.760383+0000","last_became_active":"2023-10-09T06:30:42.756109+0000","last_became_peered":"2023-10-09T06:30:42.756109+0000","last_unstale":"2023-10-09T06:30:43.760383+0000","last_undegraded":"2023-10-09T06:30:43.760383+0000","last_fullsized":"2023-10-09T06:30:43.760383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:30:02.609828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.485735+0000","last_change":"2023-10-09T06:30:43.483554+0000","last_active":"2023-10-09T06:30:43.485735+0000","last_peered":"2023-10-09T06:30:43.485735+0000","last_clean":"2023-10-09T06:30:43.485735+0000","last_became_active":"2023-10-09T06:30:43.483342+0000","last_became_peered":"2023-10-09T06:30:43.483342+0000","last_unstale":"2023-10-09T06:30:43.485735+0000","last_undegraded":"2023-10-09T06:30:43.485735+0000","last_fullsized":"2023-10-09T06:30:43.485735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:06:04.546054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.478131+0000","last_change":"2023-10-09T06:30:42.754490+0000","last_active":"2023-10-09T06:30:43.478131+0000","last_peered":"2023-10-09T06:30:43.478131+0000","last_clean":"2023-10-09T06:30:43.478131+0000","last_became_active":"2023-10-09T06:30:42.754238+0000","last_became_peered":"2023-10-09T06:30:42.754238+0000","last_unstale":"2023-10-09T06:30:43.478131+0000","last_undegraded":"2023-10-09T06:30:43.478131+0000","last_fullsized":"2023-10-09T06:30:43.478131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:45:19.887880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496827+0000","last_change":"2023-10-09T06:30:43.493269+0000","last_active":"2023-10-09T06:30:43.496827+0000","last_peered":"2023-10-09T06:30:43.496827+0000","last_clean":"2023-10-09T06:30:43.496827+0000","last_became_active":"2023-10-09T06:30:43.491762+0000","last_became_peered":"2023-10-09T06:30:43.491762+0000","last_unstale":"2023-10-09T06:30:43.496827+0000","last_undegraded":"2023-10-09T06:30:43.496827+0000","last_fullsized":"2023-10-09T06:30:43.496827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:10:42.930778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.482114+0000","last_change":"2023-10-09T06:30:42.763709+0000","last_active":"2023-10-09T06:30:43.482114+0000","last_peered":"2023-10-09T06:30:43.482114+0000","last_clean":"2023-10-09T06:30:43.482114+0000","last_became_active":"2023-10-09T06:30:42.763455+0000","last_became_peered":"2023-10-09T06:30:42.763455+0000","last_unstale":"2023-10-09T06:30:43.482114+0000","last_undegraded":"2023-10-09T06:30:43.482114+0000","last_fullsized":"2023-10-09T06:30:43.482114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:59:42.424421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.490783+0000","last_change":"2023-10-09T06:30:43.486278+0000","last_active":"2023-10-09T06:30:43.490783+0000","last_peered":"2023-10-09T06:30:43.490783+0000","last_clean":"2023-10-09T06:30:43.490783+0000","last_became_active":"2023-10-09T06:30:43.485877+0000","last_became_peered":"2023-10-09T06:30:43.485877+0000","last_unstale":"2023-10-09T06:30:43.490783+0000","last_undegraded":"2023-10-09T06:30:43.490783+0000","last_fullsized":"2023-10-09T06:30:43.490783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:55:20.971921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.486699+0000","last_change":"2023-10-09T06:30:43.482444+0000","last_active":"2023-10-09T06:30:43.486699+0000","last_peered":"2023-10-09T06:30:43.486699+0000","last_clean":"2023-10-09T06:30:43.486699+0000","last_became_active":"2023-10-09T06:30:43.482168+0000","last_became_peered":"2023-10-09T06:30:43.482168+0000","last_unstale":"2023-10-09T06:30:43.486699+0000","last_undegraded":"2023-10-09T06:30:43.486699+0000","last_fullsized":"2023-10-09T06:30:43.486699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:56:48.342402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.476414+0000","last_change":"2023-10-09T06:30:42.755534+0000","last_active":"2023-10-09T06:30:43.476414+0000","last_peered":"2023-10-09T06:30:43.476414+0000","last_clean":"2023-10-09T06:30:43.476414+0000","last_became_active":"2023-10-09T06:30:42.755246+0000","last_became_peered":"2023-10-09T06:30:42.755246+0000","last_unstale":"2023-10-09T06:30:43.476414+0000","last_undegraded":"2023-10-09T06:30:43.476414+0000","last_fullsized":"2023-10-09T06:30:43.476414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:52:48.682306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493672+0000","last_change":"2023-10-09T06:30:43.488993+0000","last_active":"2023-10-09T06:30:43.493672+0000","last_peered":"2023-10-09T06:30:43.493672+0000","last_clean":"2023-10-09T06:30:43.493672+0000","last_became_active":"2023-10-09T06:30:43.488701+0000","last_became_peered":"2023-10-09T06:30:43.488701+0000","last_unstale":"2023-10-09T06:30:43.493672+0000","last_undegraded":"2023-10-09T06:30:43.493672+0000","last_fullsized":"2023-10-09T06:30:43.493672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:05:59.526736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.479498+0000","last_change":"2023-10-09T06:30:42.757897+0000","last_active":"2023-10-09T06:30:43.479498+0000","last_peered":"2023-10-09T06:30:43.479498+0000","last_clean":"2023-10-09T06:30:43.479498+0000","last_became_active":"2023-10-09T06:30:42.757432+0000","last_became_peered":"2023-10-09T06:30:42.757432+0000","last_unstale":"2023-10-09T06:30:43.479498+0000","last_undegraded":"2023-10-09T06:30:43.479498+0000","last_fullsized":"2023-10-09T06:30:43.479498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:36:40.691450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":163,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.474256+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:30:43.474256+0000","last_peered":"2023-10-09T06:30:43.474256+0000","last_clean":"2023-10-09T06:30:43.474256+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:30:43.474256+0000","last_undegraded":"2023-10-09T06:30:43.474256+0000","last_fullsized":"2023-10-09T06:30:43.474256+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.484921+0000","last_change":"2023-10-09T06:30:43.479970+0000","last_active":"2023-10-09T06:30:43.484921+0000","last_peered":"2023-10-09T06:30:43.484921+0000","last_clean":"2023-10-09T06:30:43.484921+0000","last_became_active":"2023-10-09T06:30:43.479731+0000","last_became_peered":"2023-10-09T06:30:43.479731+0000","last_unstale":"2023-10-09T06:30:43.484921+0000","last_undegraded":"2023-10-09T06:30:43.484921+0000","last_fullsized":"2023-10-09T06:30:43.484921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:11:59.504301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.475172+0000","last_change":"2023-10-09T06:30:42.763408+0000","last_active":"2023-10-09T06:30:43.475172+0000","last_peered":"2023-10-09T06:30:43.475172+0000","last_clean":"2023-10-09T06:30:43.475172+0000","last_became_active":"2023-10-09T06:30:42.763129+0000","last_became_peered":"2023-10-09T06:30:42.763129+0000","last_unstale":"2023-10-09T06:30:43.475172+0000","last_undegraded":"2023-10-09T06:30:43.475172+0000","last_fullsized":"2023-10-09T06:30:43.475172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:29:11.172110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.475746+0000","last_change":"2023-10-09T06:30:42.766229+0000","last_active":"2023-10-09T06:30:43.475746+0000","last_peered":"2023-10-09T06:30:43.475746+0000","last_clean":"2023-10-09T06:30:43.475746+0000","last_became_active":"2023-10-09T06:30:42.765968+0000","last_became_peered":"2023-10-09T06:30:42.765968+0000","last_unstale":"2023-10-09T06:30:43.475746+0000","last_undegraded":"2023-10-09T06:30:43.475746+0000","last_fullsized":"2023-10-09T06:30:43.475746+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:59:42.424421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.486657+0000","last_change":"2023-10-09T06:30:43.482450+0000","last_active":"2023-10-09T06:30:43.486657+0000","last_peered":"2023-10-09T06:30:43.486657+0000","last_clean":"2023-10-09T06:30:43.486657+0000","last_became_active":"2023-10-09T06:30:43.482187+0000","last_became_peered":"2023-10-09T06:30:43.482187+0000","last_unstale":"2023-10-09T06:30:43.486657+0000","last_undegraded":"2023-10-09T06:30:43.486657+0000","last_fullsized":"2023-10-09T06:30:43.486657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:09:10.695002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.478234+0000","last_change":"2023-10-09T06:30:42.759163+0000","last_active":"2023-10-09T06:30:43.478234+0000","last_peered":"2023-10-09T06:30:43.478234+0000","last_clean":"2023-10-09T06:30:43.478234+0000","last_became_active":"2023-10-09T06:30:42.758583+0000","last_became_peered":"2023-10-09T06:30:42.758583+0000","last_unstale":"2023-10-09T06:30:43.478234+0000","last_undegraded":"2023-10-09T06:30:43.478234+0000","last_fullsized":"2023-10-09T06:30:43.478234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:36:40.691450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.487871+0000","last_change":"2023-10-09T06:30:43.480222+0000","last_active":"2023-10-09T06:30:43.487871+0000","last_peered":"2023-10-09T06:30:43.487871+0000","last_clean":"2023-10-09T06:30:43.487871+0000","last_became_active":"2023-10-09T06:30:43.479958+0000","last_became_peered":"2023-10-09T06:30:43.479958+0000","last_unstale":"2023-10-09T06:30:43.487871+0000","last_undegraded":"2023-10-09T06:30:43.487871+0000","last_fullsized":"2023-10-09T06:30:43.487871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:57:59.094337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.761656+0000","last_change":"2023-10-09T06:30:42.757526+0000","last_active":"2023-10-09T06:30:43.761656+0000","last_peered":"2023-10-09T06:30:43.761656+0000","last_clean":"2023-10-09T06:30:43.761656+0000","last_became_active":"2023-10-09T06:30:42.756934+0000","last_became_peered":"2023-10-09T06:30:42.756934+0000","last_unstale":"2023-10-09T06:30:43.761656+0000","last_undegraded":"2023-10-09T06:30:43.761656+0000","last_fullsized":"2023-10-09T06:30:43.761656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:45:19.887880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.487633+0000","last_change":"2023-10-09T06:30:43.482622+0000","last_active":"2023-10-09T06:30:43.487633+0000","last_peered":"2023-10-09T06:30:43.487633+0000","last_clean":"2023-10-09T06:30:43.487633+0000","last_became_active":"2023-10-09T06:30:43.482362+0000","last_became_peered":"2023-10-09T06:30:43.482362+0000","last_unstale":"2023-10-09T06:30:43.487633+0000","last_undegraded":"2023-10-09T06:30:43.487633+0000","last_fullsized":"2023-10-09T06:30:43.487633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:15:54.706732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.474220+0000","last_change":"2023-10-09T06:30:42.764161+0000","last_active":"2023-10-09T06:30:43.474220+0000","last_peered":"2023-10-09T06:30:43.474220+0000","last_clean":"2023-10-09T06:30:43.474220+0000","last_became_active":"2023-10-09T06:30:42.763872+0000","last_became_peered":"2023-10-09T06:30:42.763872+0000","last_unstale":"2023-10-09T06:30:43.474220+0000","last_undegraded":"2023-10-09T06:30:43.474220+0000","last_fullsized":"2023-10-09T06:30:43.474220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:36:40.691450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.491559+0000","last_change":"2023-10-09T06:30:43.487127+0000","last_active":"2023-10-09T06:30:43.491559+0000","last_peered":"2023-10-09T06:30:43.491559+0000","last_clean":"2023-10-09T06:30:43.491559+0000","last_became_active":"2023-10-09T06:30:43.486428+0000","last_became_peered":"2023-10-09T06:30:43.486428+0000","last_unstale":"2023-10-09T06:30:43.491559+0000","last_undegraded":"2023-10-09T06:30:43.491559+0000","last_fullsized":"2023-10-09T06:30:43.491559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:59:43.429131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.478013+0000","last_change":"2023-10-09T06:30:42.758491+0000","last_active":"2023-10-09T06:30:43.478013+0000","last_peered":"2023-10-09T06:30:43.478013+0000","last_clean":"2023-10-09T06:30:43.478013+0000","last_became_active":"2023-10-09T06:30:42.757552+0000","last_became_peered":"2023-10-09T06:30:42.757552+0000","last_unstale":"2023-10-09T06:30:43.478013+0000","last_undegraded":"2023-10-09T06:30:43.478013+0000","last_fullsized":"2023-10-09T06:30:43.478013+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:11:08.967445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.490847+0000","last_change":"2023-10-09T06:30:43.483055+0000","last_active":"2023-10-09T06:30:43.490847+0000","last_peered":"2023-10-09T06:30:43.490847+0000","last_clean":"2023-10-09T06:30:43.490847+0000","last_became_active":"2023-10-09T06:30:43.482794+0000","last_became_peered":"2023-10-09T06:30:43.482794+0000","last_unstale":"2023-10-09T06:30:43.490847+0000","last_undegraded":"2023-10-09T06:30:43.490847+0000","last_fullsized":"2023-10-09T06:30:43.490847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:12:09.812942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.478093+0000","last_change":"2023-10-09T06:30:42.755096+0000","last_active":"2023-10-09T06:30:43.478093+0000","last_peered":"2023-10-09T06:30:43.478093+0000","last_clean":"2023-10-09T06:30:43.478093+0000","last_became_active":"2023-10-09T06:30:42.754834+0000","last_became_peered":"2023-10-09T06:30:42.754834+0000","last_unstale":"2023-10-09T06:30:43.478093+0000","last_undegraded":"2023-10-09T06:30:43.478093+0000","last_fullsized":"2023-10-09T06:30:43.478093+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:52:48.682306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.491004+0000","last_change":"2023-10-09T06:30:43.485563+0000","last_active":"2023-10-09T06:30:43.491004+0000","last_peered":"2023-10-09T06:30:43.491004+0000","last_clean":"2023-10-09T06:30:43.491004+0000","last_became_active":"2023-10-09T06:30:43.485051+0000","last_became_peered":"2023-10-09T06:30:43.485051+0000","last_unstale":"2023-10-09T06:30:43.491004+0000","last_undegraded":"2023-10-09T06:30:43.491004+0000","last_fullsized":"2023-10-09T06:30:43.491004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:36:01.020241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.763572+0000","last_change":"2023-10-09T06:30:42.762583+0000","last_active":"2023-10-09T06:30:43.763572+0000","last_peered":"2023-10-09T06:30:43.763572+0000","last_clean":"2023-10-09T06:30:43.763572+0000","last_became_active":"2023-10-09T06:30:42.762319+0000","last_became_peered":"2023-10-09T06:30:42.762319+0000","last_unstale":"2023-10-09T06:30:43.763572+0000","last_undegraded":"2023-10-09T06:30:43.763572+0000","last_fullsized":"2023-10-09T06:30:43.763572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:15:33.621205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.484629+0000","last_change":"2023-10-09T06:30:43.481138+0000","last_active":"2023-10-09T06:30:43.484629+0000","last_peered":"2023-10-09T06:30:43.484629+0000","last_clean":"2023-10-09T06:30:43.484629+0000","last_became_active":"2023-10-09T06:30:43.480789+0000","last_became_peered":"2023-10-09T06:30:43.480789+0000","last_unstale":"2023-10-09T06:30:43.484629+0000","last_undegraded":"2023-10-09T06:30:43.484629+0000","last_fullsized":"2023-10-09T06:30:43.484629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:12:09.812942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.475749+0000","last_change":"2023-10-09T06:30:42.754630+0000","last_active":"2023-10-09T06:30:43.475749+0000","last_peered":"2023-10-09T06:30:43.475749+0000","last_clean":"2023-10-09T06:30:43.475749+0000","last_became_active":"2023-10-09T06:30:42.754309+0000","last_became_peered":"2023-10-09T06:30:42.754309+0000","last_unstale":"2023-10-09T06:30:43.475749+0000","last_undegraded":"2023-10-09T06:30:43.475749+0000","last_fullsized":"2023-10-09T06:30:43.475749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:46:54.709506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496592+0000","last_change":"2023-10-09T06:30:43.492044+0000","last_active":"2023-10-09T06:30:43.496592+0000","last_peered":"2023-10-09T06:30:43.496592+0000","last_clean":"2023-10-09T06:30:43.496592+0000","last_became_active":"2023-10-09T06:30:43.491715+0000","last_became_peered":"2023-10-09T06:30:43.491715+0000","last_unstale":"2023-10-09T06:30:43.496592+0000","last_undegraded":"2023-10-09T06:30:43.496592+0000","last_fullsized":"2023-10-09T06:30:43.496592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:10:42.930778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.764217+0000","last_change":"2023-10-09T06:30:42.759144+0000","last_active":"2023-10-09T06:30:43.764217+0000","last_peered":"2023-10-09T06:30:43.764217+0000","last_clean":"2023-10-09T06:30:43.764217+0000","last_became_active":"2023-10-09T06:30:42.758729+0000","last_became_peered":"2023-10-09T06:30:42.758729+0000","last_unstale":"2023-10-09T06:30:43.764217+0000","last_undegraded":"2023-10-09T06:30:43.764217+0000","last_fullsized":"2023-10-09T06:30:43.764217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:46:54.709506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496783+0000","last_change":"2023-10-09T06:30:43.493008+0000","last_active":"2023-10-09T06:30:43.496783+0000","last_peered":"2023-10-09T06:30:43.496783+0000","last_clean":"2023-10-09T06:30:43.496783+0000","last_became_active":"2023-10-09T06:30:43.491584+0000","last_became_peered":"2023-10-09T06:30:43.491584+0000","last_unstale":"2023-10-09T06:30:43.496783+0000","last_undegraded":"2023-10-09T06:30:43.496783+0000","last_fullsized":"2023-10-09T06:30:43.496783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:43:44.457796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.762576+0000","last_change":"2023-10-09T06:30:42.756415+0000","last_active":"2023-10-09T06:30:43.762576+0000","last_peered":"2023-10-09T06:30:43.762576+0000","last_clean":"2023-10-09T06:30:43.762576+0000","last_became_active":"2023-10-09T06:30:42.755726+0000","last_became_peered":"2023-10-09T06:30:42.755726+0000","last_unstale":"2023-10-09T06:30:43.762576+0000","last_undegraded":"2023-10-09T06:30:43.762576+0000","last_fullsized":"2023-10-09T06:30:43.762576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:26:03.347834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.485808+0000","last_change":"2023-10-09T06:30:43.482852+0000","last_active":"2023-10-09T06:30:43.485808+0000","last_peered":"2023-10-09T06:30:43.485808+0000","last_clean":"2023-10-09T06:30:43.485808+0000","last_became_active":"2023-10-09T06:30:43.482581+0000","last_became_peered":"2023-10-09T06:30:43.482581+0000","last_unstale":"2023-10-09T06:30:43.485808+0000","last_undegraded":"2023-10-09T06:30:43.485808+0000","last_fullsized":"2023-10-09T06:30:43.485808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:03:07.803820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.490571+0000","last_change":"2023-10-09T06:30:43.485037+0000","last_active":"2023-10-09T06:30:43.490571+0000","last_peered":"2023-10-09T06:30:43.490571+0000","last_clean":"2023-10-09T06:30:43.490571+0000","last_became_active":"2023-10-09T06:30:43.484747+0000","last_became_peered":"2023-10-09T06:30:43.484747+0000","last_unstale":"2023-10-09T06:30:43.490571+0000","last_undegraded":"2023-10-09T06:30:43.490571+0000","last_fullsized":"2023-10-09T06:30:43.490571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:57:59.094337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493668+0000","last_change":"2023-10-09T06:30:43.488771+0000","last_active":"2023-10-09T06:30:43.493668+0000","last_peered":"2023-10-09T06:30:43.493668+0000","last_clean":"2023-10-09T06:30:43.493668+0000","last_became_active":"2023-10-09T06:30:43.488505+0000","last_became_peered":"2023-10-09T06:30:43.488505+0000","last_unstale":"2023-10-09T06:30:43.493668+0000","last_undegraded":"2023-10-09T06:30:43.493668+0000","last_fullsized":"2023-10-09T06:30:43.493668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:20:59.658630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496662+0000","last_change":"2023-10-09T06:30:43.492057+0000","last_active":"2023-10-09T06:30:43.496662+0000","last_peered":"2023-10-09T06:30:43.496662+0000","last_clean":"2023-10-09T06:30:43.496662+0000","last_became_active":"2023-10-09T06:30:43.491469+0000","last_became_peered":"2023-10-09T06:30:43.491469+0000","last_unstale":"2023-10-09T06:30:43.496662+0000","last_undegraded":"2023-10-09T06:30:43.496662+0000","last_fullsized":"2023-10-09T06:30:43.496662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:59:57.434765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.488149+0000","last_change":"2023-10-09T06:30:43.484938+0000","last_active":"2023-10-09T06:30:43.488149+0000","last_peered":"2023-10-09T06:30:43.488149+0000","last_clean":"2023-10-09T06:30:43.488149+0000","last_became_active":"2023-10-09T06:30:43.484188+0000","last_became_peered":"2023-10-09T06:30:43.484188+0000","last_unstale":"2023-10-09T06:30:43.488149+0000","last_undegraded":"2023-10-09T06:30:43.488149+0000","last_fullsized":"2023-10-09T06:30:43.488149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:57:59.094337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.488046+0000","last_change":"2023-10-09T06:30:43.481165+0000","last_active":"2023-10-09T06:30:43.488046+0000","last_peered":"2023-10-09T06:30:43.488046+0000","last_clean":"2023-10-09T06:30:43.488046+0000","last_became_active":"2023-10-09T06:30:43.480896+0000","last_became_peered":"2023-10-09T06:30:43.480896+0000","last_unstale":"2023-10-09T06:30:43.488046+0000","last_undegraded":"2023-10-09T06:30:43.488046+0000","last_fullsized":"2023-10-09T06:30:43.488046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:56:48.342402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.486824+0000","last_change":"2023-10-09T06:30:43.483125+0000","last_active":"2023-10-09T06:30:43.486824+0000","last_peered":"2023-10-09T06:30:43.486824+0000","last_clean":"2023-10-09T06:30:43.486824+0000","last_became_active":"2023-10-09T06:30:43.482886+0000","last_became_peered":"2023-10-09T06:30:43.482886+0000","last_unstale":"2023-10-09T06:30:43.486824+0000","last_undegraded":"2023-10-09T06:30:43.486824+0000","last_fullsized":"2023-10-09T06:30:43.486824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:15:54.706732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.484766+0000","last_change":"2023-10-09T06:30:43.479321+0000","last_active":"2023-10-09T06:30:43.484766+0000","last_peered":"2023-10-09T06:30:43.484766+0000","last_clean":"2023-10-09T06:30:43.484766+0000","last_became_active":"2023-10-09T06:30:43.477877+0000","last_became_peered":"2023-10-09T06:30:43.477877+0000","last_unstale":"2023-10-09T06:30:43.484766+0000","last_undegraded":"2023-10-09T06:30:43.484766+0000","last_fullsized":"2023-10-09T06:30:43.484766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:09:10.695002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.487894+0000","last_change":"2023-10-09T06:30:43.480511+0000","last_active":"2023-10-09T06:30:43.487894+0000","last_peered":"2023-10-09T06:30:43.487894+0000","last_clean":"2023-10-09T06:30:43.487894+0000","last_became_active":"2023-10-09T06:30:43.480250+0000","last_became_peered":"2023-10-09T06:30:43.480250+0000","last_unstale":"2023-10-09T06:30:43.487894+0000","last_undegraded":"2023-10-09T06:30:43.487894+0000","last_fullsized":"2023-10-09T06:30:43.487894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:15:54.706732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.491696+0000","last_change":"2023-10-09T06:30:43.488975+0000","last_active":"2023-10-09T06:30:43.491696+0000","last_peered":"2023-10-09T06:30:43.491696+0000","last_clean":"2023-10-09T06:30:43.491696+0000","last_became_active":"2023-10-09T06:30:43.488639+0000","last_became_peered":"2023-10-09T06:30:43.488639+0000","last_unstale":"2023-10-09T06:30:43.491696+0000","last_undegraded":"2023-10-09T06:30:43.491696+0000","last_fullsized":"2023-10-09T06:30:43.491696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:06:04.546054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.487823+0000","last_change":"2023-10-09T06:30:43.479862+0000","last_active":"2023-10-09T06:30:43.487823+0000","last_peered":"2023-10-09T06:30:43.487823+0000","last_clean":"2023-10-09T06:30:43.487823+0000","last_became_active":"2023-10-09T06:30:43.479597+0000","last_became_peered":"2023-10-09T06:30:43.479597+0000","last_unstale":"2023-10-09T06:30:43.487823+0000","last_undegraded":"2023-10-09T06:30:43.487823+0000","last_fullsized":"2023-10-09T06:30:43.487823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:09:10.695002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496763+0000","last_change":"2023-10-09T06:30:43.492872+0000","last_active":"2023-10-09T06:30:43.496763+0000","last_peered":"2023-10-09T06:30:43.496763+0000","last_clean":"2023-10-09T06:30:43.496763+0000","last_became_active":"2023-10-09T06:30:43.492634+0000","last_became_peered":"2023-10-09T06:30:43.492634+0000","last_unstale":"2023-10-09T06:30:43.496763+0000","last_undegraded":"2023-10-09T06:30:43.496763+0000","last_fullsized":"2023-10-09T06:30:43.496763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:02:03.930820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.488131+0000","last_change":"2023-10-09T06:30:43.484753+0000","last_active":"2023-10-09T06:30:43.488131+0000","last_peered":"2023-10-09T06:30:43.488131+0000","last_clean":"2023-10-09T06:30:43.488131+0000","last_became_active":"2023-10-09T06:30:43.483916+0000","last_became_peered":"2023-10-09T06:30:43.483916+0000","last_unstale":"2023-10-09T06:30:43.488131+0000","last_undegraded":"2023-10-09T06:30:43.488131+0000","last_fullsized":"2023-10-09T06:30:43.488131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:06:29.986154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.484622+0000","last_change":"2023-10-09T06:30:43.481252+0000","last_active":"2023-10-09T06:30:43.484622+0000","last_peered":"2023-10-09T06:30:43.484622+0000","last_clean":"2023-10-09T06:30:43.484622+0000","last_became_active":"2023-10-09T06:30:43.481008+0000","last_became_peered":"2023-10-09T06:30:43.481008+0000","last_unstale":"2023-10-09T06:30:43.484622+0000","last_undegraded":"2023-10-09T06:30:43.484622+0000","last_fullsized":"2023-10-09T06:30:43.484622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:02:03.930820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298060,"kb_used_data":772,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073780,"statfs":{"total":96636764160,"available":96331550720,"internally_reserved":0,"allocated":790528,"data_stored":655288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":72190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":212992,"data_stored":70814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.247}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298080,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073760,"statfs":{"total":96636764160,"available":96331530240,"internally_reserved":0,"allocated":811008,"data_stored":661632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.254}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297476,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074364,"statfs":{"total":96636764160,"available":96332148736,"internally_reserved":0,"allocated":192512,"data_stored":64366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":72190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":70780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297480,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074360,"statfs":{"total":96636764160,"available":96332144640,"internally_reserved":0,"allocated":196608,"data_stored":64456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:30:48.173 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-09T06:30:48.173 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-09T06:30:48.173 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-09T06:30:48.173 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-09T06:30:48.174 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-09T06:30:48.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.322+0000 7fcee738e700 1 Processor -- start 2023-10-09T06:30:48.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.322+0000 7fcee738e700 1 -- start start 2023-10-09T06:30:48.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.323+0000 7fcee738e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcee01574d0 0x7fcee00cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:48.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.323+0000 7fcee738e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcee006c0f0 0x7fcee00b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:48.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.323+0000 7fcee738e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcee00b16a0 con 0x7fcee00cc3c0 2023-10-09T06:30:48.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.323+0000 7fcee738e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee00b1c60 con 0x7fcee006c0f0 2023-10-09T06:30:48.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.323+0000 7fcee738e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee01578a0 con 0x7fcee01574d0 2023-10-09T06:30:48.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.323+0000 7fcee6b8d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcee006c0f0 0x7fcee00b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:48.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.323+0000 7fcee5b8b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fcee00cc3c0 0x7fcee006b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:52450/0 (socket says 172.21.15.163:52450) 2023-10-09T06:30:48.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.323+0000 7fcee5b8b700 1 -- 172.21.15.163:0/1648081498 learned_addr learned my addr 172.21.15.163:0/1648081498 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:48.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.323+0000 7fcee638c700 1 --2- 172.21.15.163:0/1648081498 >> v2:172.21.15.181:3300/0 conn(0x7fcee01574d0 0x7fcee00cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:48.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.323+0000 7fcee6b8d700 1 -- 172.21.15.163:0/1648081498 >> v2:172.21.15.181:3300/0 conn(0x7fcee01574d0 msgr2=0x7fcee00cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:48.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.323+0000 7fcee6b8d700 1 --2- 172.21.15.163:0/1648081498 >> v2:172.21.15.181:3300/0 conn(0x7fcee01574d0 0x7fcee00cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:48.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.323+0000 7fcee6b8d700 1 -- 172.21.15.163:0/1648081498 >> v1:172.21.15.163:6789/0 conn(0x7fcee00cc3c0 legacy=0x7fcee006b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:48.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.323+0000 7fcee6b8d700 1 -- 172.21.15.163:0/1648081498 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee00cdb70 con 0x7fcee006c0f0 2023-10-09T06:30:48.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.323+0000 7fcee538a700 1 -- 172.21.15.163:0/1648081498 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1232145002 0 0) 0x7fcee00b16a0 con 0x7fcee00cc3c0 2023-10-09T06:30:48.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.324+0000 7fcee6b8d700 1 --2- 172.21.15.163:0/1648081498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcee006c0f0 0x7fcee00b20f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcee0071330 tx=0x7fced8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d32edcde0856a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:48.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.324+0000 7fcee538a700 1 -- 172.21.15.163:0/1648081498 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced800e070 con 0x7fcee006c0f0 2023-10-09T06:30:48.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.324+0000 7fcee538a700 1 -- 172.21.15.163:0/1648081498 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced800bc30 con 0x7fcee006c0f0 2023-10-09T06:30:48.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.324+0000 7fcee538a700 1 -- 172.21.15.163:0/1648081498 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced8012430 con 0x7fcee006c0f0 2023-10-09T06:30:48.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.324+0000 7fcee738e700 1 -- 172.21.15.163:0/1648081498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcee006c0f0 msgr2=0x7fcee00b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:48.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.324+0000 7fcee738e700 1 --2- 172.21.15.163:0/1648081498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcee006c0f0 0x7fcee00b20f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcee0071330 tx=0x7fced8009780 comp rx=0 tx=0).stop 2023-10-09T06:30:48.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.324+0000 7fcee738e700 1 -- 172.21.15.163:0/1648081498 shutdown_connections 2023-10-09T06:30:48.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.324+0000 7fcee738e700 1 --2- 172.21.15.163:0/1648081498 >> v2:172.21.15.181:3300/0 conn(0x7fcee01574d0 0x7fcee00cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:48.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.324+0000 7fcee738e700 1 --2- 172.21.15.163:0/1648081498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcee006c0f0 0x7fcee00b20f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:48.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.324+0000 7fcee738e700 1 -- 172.21.15.163:0/1648081498 >> 172.21.15.163:0/1648081498 conn(0x7fcee00ce430 msgr2=0x7fcee00ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:48.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.324+0000 7fcee738e700 1 -- 172.21.15.163:0/1648081498 shutdown_connections 2023-10-09T06:30:48.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.324+0000 7fcee738e700 1 -- 172.21.15.163:0/1648081498 wait complete. 2023-10-09T06:30:48.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.325+0000 7fcee738e700 1 Processor -- start 2023-10-09T06:30:48.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.325+0000 7fcee738e700 1 -- start start 2023-10-09T06:30:48.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.326+0000 7fcee738e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcee006c0f0 0x7fcee0140e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:48.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.326+0000 7fcee738e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcee00cc3c0 0x7fcee00adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:48.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.326+0000 7fcee738e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee01574d0 0x7fcee00ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:48.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.326+0000 7fcee738e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee00af550 con 0x7fcee00cc3c0 2023-10-09T06:30:48.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.326+0000 7fcee738e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee00b0980 con 0x7fcee01574d0 2023-10-09T06:30:48.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.326+0000 7fcee738e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee00ae8d0 con 0x7fcee006c0f0 2023-10-09T06:30:48.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.326+0000 7fcee5b8b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcee00cc3c0 0x7fcee00adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:48.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.326+0000 7fcee5b8b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcee00cc3c0 0x7fcee00adcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36858/0 (socket says 172.21.15.163:36858) 2023-10-09T06:30:48.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.326+0000 7fcee5b8b700 1 -- 172.21.15.163:0/71182739 learned_addr learned my addr 172.21.15.163:0/71182739 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:48.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.326+0000 7fcee6b8d700 1 --2- 172.21.15.163:0/71182739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee01574d0 0x7fcee00ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:48.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.326+0000 7fcee638c700 1 --2- 172.21.15.163:0/71182739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcee006c0f0 0x7fcee0140e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:48.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.326+0000 7fcee5b8b700 1 -- 172.21.15.163:0/71182739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcee006c0f0 msgr2=0x7fcee0140e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:48.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.326+0000 7fcee5b8b700 1 --2- 172.21.15.163:0/71182739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcee006c0f0 0x7fcee0140e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:48.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.327+0000 7fcee5b8b700 1 -- 172.21.15.163:0/71182739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee01574d0 msgr2=0x7fcee00ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:48.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.327+0000 7fcee5b8b700 1 --2- 172.21.15.163:0/71182739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee01574d0 0x7fcee00ae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:48.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.327+0000 7fcee5b8b700 1 -- 172.21.15.163:0/71182739 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced8009a20 con 0x7fcee00cc3c0 2023-10-09T06:30:48.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.327+0000 7fcee5b8b700 1 --2- 172.21.15.163:0/71182739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcee00cc3c0 0x7fcee00adcf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fced000c810 tx=0x7fced0007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:48.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.327+0000 7fced67fc700 1 -- 172.21.15.163:0/71182739 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced0005c00 con 0x7fcee00cc3c0 2023-10-09T06:30:48.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.327+0000 7fced67fc700 1 -- 172.21.15.163:0/71182739 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced0005d60 con 0x7fcee00cc3c0 2023-10-09T06:30:48.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.327+0000 7fcee738e700 1 -- 172.21.15.163:0/71182739 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee00aebb0 con 0x7fcee00cc3c0 2023-10-09T06:30:48.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.327+0000 7fced67fc700 1 -- 172.21.15.163:0/71182739 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced0007300 con 0x7fcee00cc3c0 2023-10-09T06:30:48.335 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.327+0000 7fcee738e700 1 -- 172.21.15.163:0/71182739 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee00af170 con 0x7fcee00cc3c0 2023-10-09T06:30:48.335 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.328+0000 7fcee738e700 1 -- 172.21.15.163:0/71182739 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee014e5c0 con 0x7fcee00cc3c0 2023-10-09T06:30:48.335 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.328+0000 7fced67fc700 1 -- 172.21.15.163:0/71182739 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fced001b3b0 con 0x7fcee00cc3c0 2023-10-09T06:30:48.335 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.328+0000 7fced67fc700 1 --2- 172.21.15.163:0/71182739 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcec4066f30 0x7fcec40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:48.335 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.329+0000 7fced67fc700 1 -- 172.21.15.163:0/71182739 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fced00839f0 con 0x7fcee00cc3c0 2023-10-09T06:30:48.335 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.329+0000 7fcee638c700 1 --2- 172.21.15.163:0/71182739 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcec4066f30 0x7fcec40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:48.335 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.329+0000 7fcee638c700 1 --2- 172.21.15.163:0/71182739 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcec4066f30 0x7fcec40693e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcee015d040 tx=0x7fcedc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:48.335 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.333+0000 7fced67fc700 1 -- 172.21.15.163:0/71182739 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fced0050c60 con 0x7fcee00cc3c0 2023-10-09T06:30:48.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.571+0000 7fcee738e700 1 -- 172.21.15.163:0/71182739 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fcee004c3c0 con 0x7fcee00cc3c0 2023-10-09T06:30:48.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.574+0000 7fced67fc700 1 -- 172.21.15.163:0/71182739 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+163706 (secure 0 0 0) 0x7fced0017020 con 0x7fcee00cc3c0 2023-10-09T06:30:48.576 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:30:48.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.576+0000 7fcee738e700 1 -- 172.21.15.163:0/71182739 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcec4066f30 msgr2=0x7fcec40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:48.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.576+0000 7fcee738e700 1 --2- 172.21.15.163:0/71182739 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcec4066f30 0x7fcec40693e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcee015d040 tx=0x7fcedc008040 comp rx=0 tx=0).stop 2023-10-09T06:30:48.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.576+0000 7fcee738e700 1 -- 172.21.15.163:0/71182739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcee00cc3c0 msgr2=0x7fcee00adcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:48.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.576+0000 7fcee738e700 1 --2- 172.21.15.163:0/71182739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcee00cc3c0 0x7fcee00adcf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fced000c810 tx=0x7fced0007db0 comp rx=0 tx=0).stop 2023-10-09T06:30:48.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.576+0000 7fcee738e700 1 -- 172.21.15.163:0/71182739 shutdown_connections 2023-10-09T06:30:48.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.576+0000 7fcee738e700 1 --2- 172.21.15.163:0/71182739 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcec4066f30 0x7fcec40693e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:48.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.576+0000 7fcee738e700 1 --2- 172.21.15.163:0/71182739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee01574d0 0x7fcee00ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:48.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.576+0000 7fcee738e700 1 --2- 172.21.15.163:0/71182739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcee006c0f0 0x7fcee0140e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:48.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.576+0000 7fcee738e700 1 --2- 172.21.15.163:0/71182739 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcee00cc3c0 0x7fcee00adcf0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:48.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.576+0000 7fcee738e700 1 -- 172.21.15.163:0/71182739 >> 172.21.15.163:0/71182739 conn(0x7fcee00ce430 msgr2=0x7fcee00b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:48.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.576+0000 7fcee738e700 1 -- 172.21.15.163:0/71182739 shutdown_connections 2023-10-09T06:30:48.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.576+0000 7fcee738e700 1 -- 172.21.15.163:0/71182739 wait complete. 2023-10-09T06:30:48.589 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":5,"active_gid":4103,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":90545},{"type":"v1","addr":"172.21.15.181:6833","nonce":90545}]},"active_addr":"172.21.15.181:6833/90545","active_change":"2023-10-09T06:30:24.284090+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4105,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":1590864936}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":2683786647}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":3907802465}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":3325649324}]}]} 2023-10-09T06:30:48.594 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-09T06:30:48.594 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-09T06:30:48.594 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:30:48.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.708+0000 7f0a7b2c6700 1 Processor -- start 2023-10-09T06:30:48.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.709+0000 7f0a7b2c6700 1 -- start start 2023-10-09T06:30:48.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.709+0000 7f0a7b2c6700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0a74159890 0x7f0a7415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:48.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.709+0000 7f0a7b2c6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0a74159c60 0x7f0a7415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:48.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.709+0000 7f0a7b2c6700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a7415a160 con 0x7f0a7415db10 2023-10-09T06:30:48.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.709+0000 7f0a7b2c6700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a7415df50 con 0x7f0a74159890 2023-10-09T06:30:48.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.709+0000 7f0a7b2c6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a740d13a0 con 0x7f0a74159c60 2023-10-09T06:30:48.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.710+0000 7f0a79062700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0a74159890 0x7f0a7415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:48.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.710+0000 7f0a79863700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f0a7415db10 0x7f0a74069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60836/0 (socket says 172.21.15.163:60836) 2023-10-09T06:30:48.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.710+0000 7f0a79863700 1 -- 172.21.15.163:0/3440700130 learned_addr learned my addr 172.21.15.163:0/3440700130 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:48.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.710+0000 7f0a78861700 1 --2- 172.21.15.163:0/3440700130 >> v2:172.21.15.181:3300/0 conn(0x7f0a74159c60 0x7f0a7415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:48.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.710+0000 7f0a79062700 1 -- 172.21.15.163:0/3440700130 >> v2:172.21.15.181:3300/0 conn(0x7f0a74159c60 msgr2=0x7f0a7415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:48.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.710+0000 7f0a79062700 1 --2- 172.21.15.163:0/3440700130 >> v2:172.21.15.181:3300/0 conn(0x7f0a74159c60 0x7f0a7415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:48.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.710+0000 7f0a79062700 1 -- 172.21.15.163:0/3440700130 >> v1:172.21.15.163:6789/0 conn(0x7f0a7415db10 legacy=0x7f0a74069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:48.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.710+0000 7f0a79062700 1 -- 172.21.15.163:0/3440700130 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7415e140 con 0x7f0a74159890 2023-10-09T06:30:48.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.710+0000 7f0a6bfff700 1 -- 172.21.15.163:0/3440700130 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1932300187 0 0) 0x7f0a7415a160 con 0x7f0a7415db10 2023-10-09T06:30:48.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.710+0000 7f0a78861700 1 --2- 172.21.15.163:0/3440700130 >> v2:172.21.15.181:3300/0 conn(0x7f0a74159c60 0x7f0a7415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:30:48.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.710+0000 7f0a79062700 1 --2- 172.21.15.163:0/3440700130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a74159890 0x7f0a7415abd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0a740b67f0 tx=0x7f0a70009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d309611bcd6acf0e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:48.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.710+0000 7f0a6bfff700 1 -- 172.21.15.163:0/3440700130 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7000e070 con 0x7f0a74159890 2023-10-09T06:30:48.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.710+0000 7f0a6bfff700 1 -- 172.21.15.163:0/3440700130 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7000bc30 con 0x7f0a74159890 2023-10-09T06:30:48.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.710+0000 7f0a6bfff700 1 -- 172.21.15.163:0/3440700130 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a70012430 con 0x7f0a74159890 2023-10-09T06:30:48.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.711+0000 7f0a7b2c6700 1 -- 172.21.15.163:0/3440700130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a74159890 msgr2=0x7f0a7415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:48.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.711+0000 7f0a7b2c6700 1 --2- 172.21.15.163:0/3440700130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a74159890 0x7f0a7415abd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0a740b67f0 tx=0x7f0a70009780 comp rx=0 tx=0).stop 2023-10-09T06:30:48.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.711+0000 7f0a7b2c6700 1 -- 172.21.15.163:0/3440700130 shutdown_connections 2023-10-09T06:30:48.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.711+0000 7f0a7b2c6700 1 --2- 172.21.15.163:0/3440700130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a74159890 0x7f0a7415abd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:48.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.711+0000 7f0a7b2c6700 1 --2- 172.21.15.163:0/3440700130 >> v2:172.21.15.181:3300/0 conn(0x7f0a74159c60 0x7f0a7415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:48.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.711+0000 7f0a7b2c6700 1 -- 172.21.15.163:0/3440700130 >> 172.21.15.163:0/3440700130 conn(0x7f0a74068e20 msgr2=0x7f0a740d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:48.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.711+0000 7f0a7b2c6700 1 -- 172.21.15.163:0/3440700130 shutdown_connections 2023-10-09T06:30:48.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.711+0000 7f0a7b2c6700 1 -- 172.21.15.163:0/3440700130 wait complete. 2023-10-09T06:30:48.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.712+0000 7f0a7b2c6700 1 Processor -- start 2023-10-09T06:30:48.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.712+0000 7f0a7b2c6700 1 -- start start 2023-10-09T06:30:48.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.713+0000 7f0a7b2c6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a74159890 0x7f0a740b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:48.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.713+0000 7f0a7b2c6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a74159c60 0x7f0a74147260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:48.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.713+0000 7f0a7b2c6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a7415db10 0x7f0a740ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:48.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.713+0000 7f0a7b2c6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a740b4df0 con 0x7f0a7415db10 2023-10-09T06:30:48.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.713+0000 7f0a7b2c6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a741477a0 con 0x7f0a74159890 2023-10-09T06:30:48.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.713+0000 7f0a7b2c6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a74169f90 con 0x7f0a74159c60 2023-10-09T06:30:48.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.713+0000 7f0a79863700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a7415db10 0x7f0a740ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:48.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.713+0000 7f0a79863700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a7415db10 0x7f0a740ca4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43694/0 (socket says 172.21.15.163:43694) 2023-10-09T06:30:48.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.713+0000 7f0a79863700 1 -- 172.21.15.163:0/4280974655 learned_addr learned my addr 172.21.15.163:0/4280974655 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:48.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.713+0000 7f0a78861700 1 --2- 172.21.15.163:0/4280974655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a74159c60 0x7f0a74147260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:48.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.713+0000 7f0a79062700 1 --2- 172.21.15.163:0/4280974655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a74159890 0x7f0a740b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:48.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.713+0000 7f0a79863700 1 -- 172.21.15.163:0/4280974655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a74159c60 msgr2=0x7f0a74147260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:48.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.713+0000 7f0a79863700 1 --2- 172.21.15.163:0/4280974655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a74159c60 0x7f0a74147260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:48.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.713+0000 7f0a79863700 1 -- 172.21.15.163:0/4280974655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a74159890 msgr2=0x7f0a740b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:48.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.713+0000 7f0a79863700 1 --2- 172.21.15.163:0/4280974655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a74159890 0x7f0a740b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:48.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.713+0000 7f0a79863700 1 -- 172.21.15.163:0/4280974655 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a70009a20 con 0x7f0a7415db10 2023-10-09T06:30:48.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.713+0000 7f0a79062700 1 --2- 172.21.15.163:0/4280974655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a74159890 0x7f0a740b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:30:48.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.713+0000 7f0a79863700 1 --2- 172.21.15.163:0/4280974655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a7415db10 0x7f0a740ca4a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00c530 tx=0x7f0a6c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:48.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.713+0000 7f0a697fa700 1 -- 172.21.15.163:0/4280974655 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c005b30 con 0x7f0a7415db10 2023-10-09T06:30:48.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.713+0000 7f0a697fa700 1 -- 172.21.15.163:0/4280974655 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a6c005c90 con 0x7f0a7415db10 2023-10-09T06:30:48.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.713+0000 7f0a7b2c6700 1 -- 172.21.15.163:0/4280974655 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a7416a270 con 0x7f0a7415db10 2023-10-09T06:30:48.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.713+0000 7f0a697fa700 1 -- 172.21.15.163:0/4280974655 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c007320 con 0x7f0a7415db10 2023-10-09T06:30:48.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.713+0000 7f0a7b2c6700 1 -- 172.21.15.163:0/4280974655 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a7416a830 con 0x7f0a7415db10 2023-10-09T06:30:48.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.714+0000 7f0a7b2c6700 1 -- 172.21.15.163:0/4280974655 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a7404c3c0 con 0x7f0a7415db10 2023-10-09T06:30:48.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.715+0000 7f0a697fa700 1 -- 172.21.15.163:0/4280974655 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0a6c01b3b0 con 0x7f0a7415db10 2023-10-09T06:30:48.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.715+0000 7f0a697fa700 1 --2- 172.21.15.163:0/4280974655 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0a58066f30 0x7f0a580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:48.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.715+0000 7f0a697fa700 1 -- 172.21.15.163:0/4280974655 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0a6c0839a0 con 0x7f0a7415db10 2023-10-09T06:30:48.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.715+0000 7f0a79062700 1 --2- 172.21.15.163:0/4280974655 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0a58066f30 0x7f0a580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:48.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.716+0000 7f0a79062700 1 --2- 172.21.15.163:0/4280974655 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0a58066f30 0x7f0a580693e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0a740b67f0 tx=0x7f0a70019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:48.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.719+0000 7f0a697fa700 1 -- 172.21.15.163:0/4280974655 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a6c050c10 con 0x7f0a7415db10 2023-10-09T06:30:48.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.905+0000 7f0a7b2c6700 1 -- 172.21.15.163:0/4280974655 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a7406a5b0 con 0x7f0a7415db10 2023-10-09T06:30:48.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.905+0000 7f0a697fa700 1 -- 172.21.15.163:0/4280974655 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f0a6c017090 con 0x7f0a7415db10 2023-10-09T06:30:48.906 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:30:48.907 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":20,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:30:44.466923+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:30:41.369014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-09T06:30:42.103001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:30:48.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.908+0000 7f0a7b2c6700 1 -- 172.21.15.163:0/4280974655 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0a58066f30 msgr2=0x7f0a580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:48.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.908+0000 7f0a7b2c6700 1 --2- 172.21.15.163:0/4280974655 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0a58066f30 0x7f0a580693e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0a740b67f0 tx=0x7f0a70019000 comp rx=0 tx=0).stop 2023-10-09T06:30:48.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.908+0000 7f0a7b2c6700 1 -- 172.21.15.163:0/4280974655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a7415db10 msgr2=0x7f0a740ca4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:48.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.908+0000 7f0a7b2c6700 1 --2- 172.21.15.163:0/4280974655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a7415db10 0x7f0a740ca4a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00c530 tx=0x7f0a6c007f90 comp rx=0 tx=0).stop 2023-10-09T06:30:48.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.908+0000 7f0a7b2c6700 1 -- 172.21.15.163:0/4280974655 shutdown_connections 2023-10-09T06:30:48.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.908+0000 7f0a7b2c6700 1 --2- 172.21.15.163:0/4280974655 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0a58066f30 0x7f0a580693e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:48.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.908+0000 7f0a7b2c6700 1 --2- 172.21.15.163:0/4280974655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a7415db10 0x7f0a740ca4a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:48.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.908+0000 7f0a7b2c6700 1 --2- 172.21.15.163:0/4280974655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a74159890 0x7f0a740b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:48.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.908+0000 7f0a7b2c6700 1 --2- 172.21.15.163:0/4280974655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a74159c60 0x7f0a74147260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:48.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.908+0000 7f0a7b2c6700 1 -- 172.21.15.163:0/4280974655 >> 172.21.15.163:0/4280974655 conn(0x7f0a74068e20 msgr2=0x7f0a740b1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:48.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.908+0000 7f0a7b2c6700 1 -- 172.21.15.163:0/4280974655 shutdown_connections 2023-10-09T06:30:48.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:48.908+0000 7f0a7b2c6700 1 -- 172.21.15.163:0/4280974655 wait complete. 2023-10-09T06:30:48.921 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-09T06:30:48.921 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:30:49.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.034+0000 7f643dedc700 1 Processor -- start 2023-10-09T06:30:49.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.034+0000 7f643dedc700 1 -- start start 2023-10-09T06:30:49.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.035+0000 7f643dedc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6438159420 0x7f64380b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.035+0000 7f643dedc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f643815d230 0x7f643815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.035+0000 7f643dedc700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f643815dbd0 con 0x7f64381597f0 2023-10-09T06:30:49.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.035+0000 7f643dedc700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64380b2980 con 0x7f643815d230 2023-10-09T06:30:49.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.035+0000 7f643dedc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64380b11d0 con 0x7f6438159420 2023-10-09T06:30:49.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.035+0000 7f643ceda700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6438159420 0x7f64380b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.035+0000 7f643d6db700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f643815d230 0x7f643815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.035+0000 7f642ffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f64381597f0 0x7f64380b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:47772/0 (socket says 172.21.15.163:47772) 2023-10-09T06:30:49.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.035+0000 7f643ceda700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6438159420 0x7f64380b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:44560/0 (socket says 172.21.15.163:44560) 2023-10-09T06:30:49.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.035+0000 7f643d6db700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f643815d230 0x7f643815d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:50270/0 (socket says 172.21.15.163:50270) 2023-10-09T06:30:49.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.035+0000 7f642ffff700 1 -- 172.21.15.163:0/206342861 learned_addr learned my addr 172.21.15.163:0/206342861 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:49.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.036+0000 7f643ceda700 1 -- 172.21.15.163:0/206342861 >> v2:172.21.15.181:3300/0 conn(0x7f643815d230 msgr2=0x7f643815d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:49.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.036+0000 7f643ceda700 1 --2- 172.21.15.163:0/206342861 >> v2:172.21.15.181:3300/0 conn(0x7f643815d230 0x7f643815d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.036+0000 7f643ceda700 1 -- 172.21.15.163:0/206342861 >> v1:172.21.15.181:6789/0 conn(0x7f64381597f0 legacy=0x7f64380b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.036+0000 7f643ceda700 1 -- 172.21.15.163:0/206342861 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64380b14b0 con 0x7f6438159420 2023-10-09T06:30:49.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.036+0000 7f643d6db700 1 --2- 172.21.15.163:0/206342861 >> v2:172.21.15.181:3300/0 conn(0x7f643815d230 0x7f643815d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:30:49.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.036+0000 7f643ceda700 1 --2- 172.21.15.163:0/206342861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6438159420 0x7f64380b5f40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6434009e60 tx=0x7f6434009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62bfa9979f3139d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.036+0000 7f642f7fe700 1 -- 172.21.15.163:0/206342861 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643400e070 con 0x7f6438159420 2023-10-09T06:30:49.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.036+0000 7f642f7fe700 1 -- 172.21.15.163:0/206342861 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f643400bb60 con 0x7f6438159420 2023-10-09T06:30:49.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.036+0000 7f642f7fe700 1 -- 172.21.15.163:0/206342861 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6434012430 con 0x7f6438159420 2023-10-09T06:30:49.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.037+0000 7f643dedc700 1 -- 172.21.15.163:0/206342861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6438159420 msgr2=0x7f64380b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.037+0000 7f643dedc700 1 --2- 172.21.15.163:0/206342861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6438159420 0x7f64380b5f40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6434009e60 tx=0x7f6434009780 comp rx=0 tx=0).stop 2023-10-09T06:30:49.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.037+0000 7f643dedc700 1 -- 172.21.15.163:0/206342861 shutdown_connections 2023-10-09T06:30:49.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.037+0000 7f643dedc700 1 --2- 172.21.15.163:0/206342861 >> v2:172.21.15.181:3300/0 conn(0x7f643815d230 0x7f643815d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.037+0000 7f643dedc700 1 --2- 172.21.15.163:0/206342861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6438159420 0x7f64380b5f40 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.037+0000 7f643dedc700 1 -- 172.21.15.163:0/206342861 >> 172.21.15.163:0/206342861 conn(0x7f64380d0660 msgr2=0x7f64380d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:49.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.037+0000 7f643dedc700 1 -- 172.21.15.163:0/206342861 shutdown_connections 2023-10-09T06:30:49.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.037+0000 7f643dedc700 1 -- 172.21.15.163:0/206342861 wait complete. 2023-10-09T06:30:49.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.038+0000 7f643dedc700 1 Processor -- start 2023-10-09T06:30:49.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.038+0000 7f643dedc700 1 -- start start 2023-10-09T06:30:49.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.039+0000 7f643dedc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6438159420 0x7f643815c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.039+0000 7f643dedc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64381597f0 0x7f64381698c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.039+0000 7f643dedc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f643815d230 0x7f643814ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.039+0000 7f643dedc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6438151720 con 0x7f6438159420 2023-10-09T06:30:49.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.039+0000 7f643dedc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643814d230 con 0x7f64381597f0 2023-10-09T06:30:49.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.039+0000 7f643dedc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643814d6a0 con 0x7f643815d230 2023-10-09T06:30:49.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.039+0000 7f643ceda700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6438159420 0x7f643815c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.039+0000 7f643ceda700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6438159420 0x7f643815c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43700/0 (socket says 172.21.15.163:43700) 2023-10-09T06:30:49.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.039+0000 7f642ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64381597f0 0x7f64381698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.039+0000 7f643ceda700 1 -- 172.21.15.163:0/3033313644 learned_addr learned my addr 172.21.15.163:0/3033313644 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:49.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.039+0000 7f643d6db700 1 --2- 172.21.15.163:0/3033313644 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f643815d230 0x7f643814ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.039+0000 7f643ceda700 1 -- 172.21.15.163:0/3033313644 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f643815d230 msgr2=0x7f643814ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.039+0000 7f643ceda700 1 --2- 172.21.15.163:0/3033313644 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f643815d230 0x7f643814ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.039+0000 7f643ceda700 1 -- 172.21.15.163:0/3033313644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64381597f0 msgr2=0x7f64381698c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.039+0000 7f643ceda700 1 --2- 172.21.15.163:0/3033313644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64381597f0 0x7f64381698c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.039+0000 7f643ceda700 1 -- 172.21.15.163:0/3033313644 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6430009420 con 0x7f6438159420 2023-10-09T06:30:49.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.039+0000 7f643ceda700 1 --2- 172.21.15.163:0/3033313644 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6438159420 0x7f643815c3c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f64340097c0 tx=0x7f6434019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.039+0000 7f642cff9700 1 -- 172.21.15.163:0/3033313644 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6434012cf0 con 0x7f6438159420 2023-10-09T06:30:49.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.039+0000 7f642cff9700 1 -- 172.21.15.163:0/3033313644 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6434012e50 con 0x7f6438159420 2023-10-09T06:30:49.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.039+0000 7f642cff9700 1 -- 172.21.15.163:0/3033313644 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64340276c0 con 0x7f6438159420 2023-10-09T06:30:49.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.039+0000 7f643dedc700 1 -- 172.21.15.163:0/3033313644 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6434009a20 con 0x7f6438159420 2023-10-09T06:30:49.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.039+0000 7f643dedc700 1 -- 172.21.15.163:0/3033313644 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643816a1f0 con 0x7f6438159420 2023-10-09T06:30:49.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.040+0000 7f643dedc700 1 -- 172.21.15.163:0/3033313644 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f643804c3c0 con 0x7f6438159420 2023-10-09T06:30:49.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.041+0000 7f642cff9700 1 -- 172.21.15.163:0/3033313644 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6434024070 con 0x7f6438159420 2023-10-09T06:30:49.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.041+0000 7f642cff9700 1 --2- 172.21.15.163:0/3033313644 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6424066f30 0x7f64240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.041+0000 7f642cff9700 1 -- 172.21.15.163:0/3033313644 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6434086a50 con 0x7f6438159420 2023-10-09T06:30:49.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.041+0000 7f642ffff700 1 --2- 172.21.15.163:0/3033313644 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6424066f30 0x7f64240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.042+0000 7f642ffff700 1 --2- 172.21.15.163:0/3033313644 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6424066f30 0x7f64240693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6428000c00 tx=0x7f6428009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.045+0000 7f642cff9700 1 -- 172.21.15.163:0/3033313644 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6434053d80 con 0x7f6438159420 2023-10-09T06:30:49.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.229+0000 7f643dedc700 1 -- 172.21.15.163:0/3033313644 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64380b1790 con 0x7f6438159420 2023-10-09T06:30:49.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.230+0000 7f642cff9700 1 -- 172.21.15.163:0/3033313644 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f64340573a0 con 0x7f6438159420 2023-10-09T06:30:49.231 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:30:49.231 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":20,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:30:44.466923+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:30:41.369014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-09T06:30:42.103001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:30:49.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.232+0000 7f643dedc700 1 -- 172.21.15.163:0/3033313644 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6424066f30 msgr2=0x7f64240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.232+0000 7f643dedc700 1 --2- 172.21.15.163:0/3033313644 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6424066f30 0x7f64240693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6428000c00 tx=0x7f6428009040 comp rx=0 tx=0).stop 2023-10-09T06:30:49.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.232+0000 7f643dedc700 1 -- 172.21.15.163:0/3033313644 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6438159420 msgr2=0x7f643815c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.232+0000 7f643dedc700 1 --2- 172.21.15.163:0/3033313644 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6438159420 0x7f643815c3c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f64340097c0 tx=0x7f6434019000 comp rx=0 tx=0).stop 2023-10-09T06:30:49.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.232+0000 7f643dedc700 1 -- 172.21.15.163:0/3033313644 shutdown_connections 2023-10-09T06:30:49.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.232+0000 7f643dedc700 1 --2- 172.21.15.163:0/3033313644 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6424066f30 0x7f64240693e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.232+0000 7f643dedc700 1 --2- 172.21.15.163:0/3033313644 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6438159420 0x7f643815c3c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.232+0000 7f643dedc700 1 --2- 172.21.15.163:0/3033313644 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f643815d230 0x7f643814ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.232+0000 7f643dedc700 1 --2- 172.21.15.163:0/3033313644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64381597f0 0x7f64381698c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.232+0000 7f643dedc700 1 -- 172.21.15.163:0/3033313644 >> 172.21.15.163:0/3033313644 conn(0x7f64380d0660 msgr2=0x7f64380cfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:49.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.233+0000 7f643dedc700 1 -- 172.21.15.163:0/3033313644 shutdown_connections 2023-10-09T06:30:49.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.233+0000 7f643dedc700 1 -- 172.21.15.163:0/3033313644 wait complete. 2023-10-09T06:30:49.246 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-09T06:30:49.246 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-09T06:30:49.247 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-09T06:30:49.247 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-09T06:30:49.247 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-09T06:30:49.248 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-09T06:30:49.248 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-09T06:30:49.248 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-09T06:30:49.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.436+0000 7f106ba6f700 1 Processor -- start 2023-10-09T06:30:49.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.436+0000 7f106ba6f700 1 -- start start 2023-10-09T06:30:49.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.437+0000 7f106ba6f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f10640b0010 0x7f106415d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.437+0000 7f106ba6f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f106415dd90 0x7f10640adb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.437+0000 7f106ba6f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10640b0750 con 0x7f10640afc40 2023-10-09T06:30:49.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.437+0000 7f106ba6f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10640aca70 con 0x7f10640b0010 2023-10-09T06:30:49.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.437+0000 7f106ba6f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10640acd50 con 0x7f106415dd90 2023-10-09T06:30:49.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.437+0000 7f106a00c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f106415dd90 0x7f10640adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.437+0000 7f106900a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f10640b0010 0x7f106415d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.437+0000 7f106a00c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f106415dd90 0x7f10640adb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:44576/0 (socket says 172.21.15.163:44576) 2023-10-09T06:30:49.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.437+0000 7f106a00c700 1 -- 172.21.15.163:0/1632922641 learned_addr learned my addr 172.21.15.163:0/1632922641 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:49.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.437+0000 7f106900a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f10640b0010 0x7f106415d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:50286/0 (socket says 172.21.15.163:50286) 2023-10-09T06:30:49.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.438+0000 7f106900a700 1 -- 172.21.15.163:0/1632922641 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f106415dd90 msgr2=0x7f10640adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:49.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.438+0000 7f106900a700 1 --2- 172.21.15.163:0/1632922641 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f106415dd90 0x7f10640adb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.438+0000 7f106900a700 1 -- 172.21.15.163:0/1632922641 >> v1:172.21.15.163:6789/0 conn(0x7f10640afc40 legacy=0x7f10640b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:49.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.438+0000 7f106900a700 1 -- 172.21.15.163:0/1632922641 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10640ad030 con 0x7f10640b0010 2023-10-09T06:30:49.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.438+0000 7f106900a700 1 --2- 172.21.15.163:0/1632922641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10640b0010 0x7f106415d850 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f10600098e0 tx=0x7f10600091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a2699098f79dec9 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.439+0000 7f1068809700 1 -- 172.21.15.163:0/1632922641 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060011630 con 0x7f10640b0010 2023-10-09T06:30:49.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.439+0000 7f1068809700 1 -- 172.21.15.163:0/1632922641 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10600117b0 con 0x7f10640b0010 2023-10-09T06:30:49.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.439+0000 7f1068809700 1 -- 172.21.15.163:0/1632922641 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060011b70 con 0x7f10640b0010 2023-10-09T06:30:49.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.440+0000 7f106ba6f700 1 -- 172.21.15.163:0/1632922641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10640b0010 msgr2=0x7f106415d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.440+0000 7f106ba6f700 1 --2- 172.21.15.163:0/1632922641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10640b0010 0x7f106415d850 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f10600098e0 tx=0x7f10600091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.440+0000 7f106ba6f700 1 -- 172.21.15.163:0/1632922641 shutdown_connections 2023-10-09T06:30:49.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.440+0000 7f106ba6f700 1 --2- 172.21.15.163:0/1632922641 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f106415dd90 0x7f10640adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.440+0000 7f106ba6f700 1 --2- 172.21.15.163:0/1632922641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10640b0010 0x7f106415d850 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.440+0000 7f106ba6f700 1 -- 172.21.15.163:0/1632922641 >> 172.21.15.163:0/1632922641 conn(0x7f106406a320 msgr2=0x7f10640cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:49.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.440+0000 7f106ba6f700 1 -- 172.21.15.163:0/1632922641 shutdown_connections 2023-10-09T06:30:49.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.440+0000 7f106ba6f700 1 -- 172.21.15.163:0/1632922641 wait complete. 2023-10-09T06:30:49.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.442+0000 7f106ba6f700 1 Processor -- start 2023-10-09T06:30:49.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.442+0000 7f106ba6f700 1 -- start start 2023-10-09T06:30:49.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.443+0000 7f106ba6f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10640afc40 0x7f106414f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.443+0000 7f106ba6f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10640b0010 0x7f1064152330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.443+0000 7f106ba6f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f106415dd90 0x7f1064153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.443+0000 7f106ba6f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1064151ea0 con 0x7f10640b0010 2023-10-09T06:30:49.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.443+0000 7f106ba6f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1064151290 con 0x7f106415dd90 2023-10-09T06:30:49.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.443+0000 7f106ba6f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1064151570 con 0x7f10640afc40 2023-10-09T06:30:49.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.444+0000 7f106900a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10640b0010 0x7f1064152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.444+0000 7f106900a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10640b0010 0x7f1064152330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43714/0 (socket says 172.21.15.163:43714) 2023-10-09T06:30:49.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.444+0000 7f106900a700 1 -- 172.21.15.163:0/1490128315 learned_addr learned my addr 172.21.15.163:0/1490128315 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:49.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.444+0000 7f106a00c700 1 --2- 172.21.15.163:0/1490128315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f106415dd90 0x7f1064153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.444+0000 7f106980b700 1 --2- 172.21.15.163:0/1490128315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10640afc40 0x7f106414f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.444+0000 7f106900a700 1 -- 172.21.15.163:0/1490128315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10640afc40 msgr2=0x7f106414f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.444+0000 7f106900a700 1 --2- 172.21.15.163:0/1490128315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10640afc40 0x7f106414f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.444+0000 7f106900a700 1 -- 172.21.15.163:0/1490128315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f106415dd90 msgr2=0x7f1064153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.444+0000 7f106900a700 1 --2- 172.21.15.163:0/1490128315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f106415dd90 0x7f1064153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.444+0000 7f106900a700 1 -- 172.21.15.163:0/1490128315 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1060009420 con 0x7f10640b0010 2023-10-09T06:30:49.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.444+0000 7f106a00c700 1 --2- 172.21.15.163:0/1490128315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f106415dd90 0x7f1064153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:30:49.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.444+0000 7fa755cf0700 1 Processor -- start 2023-10-09T06:30:49.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.444+0000 7fdf55914700 1 Processor -- start 2023-10-09T06:30:49.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.445+0000 7fdf55914700 1 -- start start 2023-10-09T06:30:49.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.445+0000 7f106900a700 1 --2- 172.21.15.163:0/1490128315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10640b0010 0x7f1064152330 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1064064290 tx=0x7f1060016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.445+0000 7fa84206a700 1 Processor -- start 2023-10-09T06:30:49.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.445+0000 7f1059ffb700 1 -- 172.21.15.163:0/1490128315 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106001bd90 con 0x7f10640b0010 2023-10-09T06:30:49.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.445+0000 7f1059ffb700 1 -- 172.21.15.163:0/1490128315 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1060020070 con 0x7f10640b0010 2023-10-09T06:30:49.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.445+0000 7f1059ffb700 1 -- 172.21.15.163:0/1490128315 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10600276d0 con 0x7f10640b0010 2023-10-09T06:30:49.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.445+0000 7fa755cf0700 1 -- start start 2023-10-09T06:30:49.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.446+0000 7fa755cf0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa75006bcd0 0x7fa750140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.446+0000 7fa755cf0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa7500c9510 0x7fa750141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.446+0000 7fa755cf0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa75015c2a0 con 0x7fa75006c0a0 2023-10-09T06:30:49.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.446+0000 7fa755cf0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa75015b690 con 0x7fa7500c9510 2023-10-09T06:30:49.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.446+0000 7fa755cf0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa75015b970 con 0x7fa75006bcd0 2023-10-09T06:30:49.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.446+0000 7fa74effd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa75006c0a0 0x7fa7500c8e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:47786/0 (socket says 172.21.15.163:47786) 2023-10-09T06:30:49.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.446+0000 7fa74effd700 1 -- 172.21.15.163:0/2572152356 learned_addr learned my addr 172.21.15.163:0/2572152356 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:49.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.445+0000 7f106ba6f700 1 -- 172.21.15.163:0/1490128315 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1064151850 con 0x7f10640b0010 2023-10-09T06:30:49.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.445+0000 7f106ba6f700 1 -- 172.21.15.163:0/1490128315 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1064152870 con 0x7f10640b0010 2023-10-09T06:30:49.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.445+0000 7fa84206a700 1 -- start start 2023-10-09T06:30:49.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.446+0000 7fa84206a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa83c0afc40 0x7fa83c0b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.446+0000 7fa84206a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa83c0b0010 0x7fa83c15df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.446+0000 7fa84206a700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa83c0b0750 con 0x7fa83c0b2140 2023-10-09T06:30:49.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.446+0000 7fa84206a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa83c0aca70 con 0x7fa83c0afc40 2023-10-09T06:30:49.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.446+0000 7fa84206a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c0acd50 con 0x7fa83c0b0010 2023-10-09T06:30:49.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.446+0000 7fa74ffff700 1 --2- 172.21.15.163:0/2572152356 >> v2:172.21.15.181:3300/0 conn(0x7fa7500c9510 0x7fa750141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.446+0000 7fdf55914700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdf50158440 0x7fdf50068aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.446+0000 7fdf55914700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdf500b21b0 0x7fdf500ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.446+0000 7fdf55914700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf5015f260 con 0x7fdf50068fe0 2023-10-09T06:30:49.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.446+0000 7fdf55914700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf500aef20 con 0x7fdf500b21b0 2023-10-09T06:30:49.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.446+0000 7fdf55914700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf50069430 con 0x7fdf50158440 2023-10-09T06:30:49.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.447+0000 7fdf4e7fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fdf50068fe0 0x7fdf500b1aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:47788/0 (socket says 172.21.15.163:47788) 2023-10-09T06:30:49.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.447+0000 7fdf4e7fc700 1 -- 172.21.15.163:0/510119492 learned_addr learned my addr 172.21.15.163:0/510119492 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:49.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.447+0000 7fdf4effd700 1 --2- 172.21.15.163:0/510119492 >> v2:172.21.15.181:3300/0 conn(0x7fdf50158440 0x7fdf50068aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.447+0000 7fa83bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa83c0b0010 0x7fa83c15df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.447+0000 7fa83bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa83c0b0010 0x7fa83c15df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:44590/0 (socket says 172.21.15.163:44590) 2023-10-09T06:30:49.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.447+0000 7fa83bfff700 1 -- 172.21.15.163:0/3144537032 learned_addr learned my addr 172.21.15.163:0/3144537032 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:49.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.447+0000 7fdf4dffb700 1 -- 172.21.15.163:0/510119492 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 896981036 0 0) 0x7fdf5015f260 con 0x7fdf50068fe0 2023-10-09T06:30:49.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.447+0000 7fdf4dffb700 1 -- 172.21.15.163:0/510119492 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf38003570 con 0x7fdf50068fe0 2023-10-09T06:30:49.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.447+0000 7fdf4f7fe700 1 --2- 172.21.15.163:0/510119492 >> v2:172.21.15.163:3300/0 conn(0x7fdf500b21b0 0x7fdf500ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.447+0000 7f106ba6f700 1 -- 172.21.15.163:0/1490128315 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f106404c3c0 con 0x7f10640b0010 2023-10-09T06:30:49.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.447+0000 7fa74ffff700 1 -- 172.21.15.163:0/2572152356 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa75006bcd0 msgr2=0x7fa750140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:49.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.447+0000 7fa74ffff700 1 --2- 172.21.15.163:0/2572152356 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa75006bcd0 0x7fa750140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.447+0000 7fdf4effd700 1 -- 172.21.15.163:0/510119492 >> v2:172.21.15.163:3300/0 conn(0x7fdf500b21b0 msgr2=0x7fdf500ab8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:49.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.447+0000 7fdf4effd700 1 --2- 172.21.15.163:0/510119492 >> v2:172.21.15.163:3300/0 conn(0x7fdf500b21b0 0x7fdf500ab8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.447+0000 7fdf4effd700 1 -- 172.21.15.163:0/510119492 >> v1:172.21.15.181:6789/0 conn(0x7fdf50068fe0 legacy=0x7fdf500b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.447+0000 7fdf4effd700 1 -- 172.21.15.163:0/510119492 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5015e380 con 0x7fdf50158440 2023-10-09T06:30:49.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.447+0000 7fa74ffff700 1 -- 172.21.15.163:0/2572152356 >> v1:172.21.15.181:6789/0 conn(0x7fa75006c0a0 legacy=0x7fa7500c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.447+0000 7fa74ffff700 1 -- 172.21.15.163:0/2572152356 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75015bc50 con 0x7fa7500c9510 2023-10-09T06:30:49.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.447+0000 7fa83a7fc700 1 -- 172.21.15.163:0/3144537032 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1893980377 0 0) 0x7fa83c0b0750 con 0x7fa83c0b2140 2023-10-09T06:30:49.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.447+0000 7fa83a7fc700 1 -- 172.21.15.163:0/3144537032 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa824003570 con 0x7fa83c0b2140 2023-10-09T06:30:49.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.447+0000 7fa83b7fe700 1 --2- 172.21.15.163:0/3144537032 >> v2:172.21.15.163:3300/0 conn(0x7fa83c0afc40 0x7fa83c0b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.447+0000 7fdf4effd700 1 --2- 172.21.15.163:0/510119492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf50158440 0x7fdf50068aa0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdf500ad380 tx=0x7fdf440091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6599945b75516569 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.448+0000 7fa83a7fc700 1 -- 172.21.15.163:0/3144537032 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2401357471 0 0) 0x7fa824003570 con 0x7fa83c0b2140 2023-10-09T06:30:49.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.448+0000 7fa74ffff700 1 --2- 172.21.15.163:0/2572152356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7500c9510 0x7fa750141330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa7440098e0 tx=0x7fa7440091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=545b50b61cb3d3bc server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.448+0000 7fdf4dffb700 1 -- 172.21.15.163:0/510119492 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44011630 con 0x7fdf50158440 2023-10-09T06:30:49.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.448+0000 7fa83b7fe700 1 -- 172.21.15.163:0/3144537032 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa83c0b0010 msgr2=0x7fa83c15df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:49.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.448+0000 7fa83b7fe700 1 --2- 172.21.15.163:0/3144537032 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa83c0b0010 0x7fa83c15df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.448+0000 7fa83b7fe700 1 -- 172.21.15.163:0/3144537032 >> v1:172.21.15.163:6789/0 conn(0x7fa83c0b2140 legacy=0x7fa83c15d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.448+0000 7fa83b7fe700 1 -- 172.21.15.163:0/3144537032 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83c0ad030 con 0x7fa83c0afc40 2023-10-09T06:30:49.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.448+0000 7fa74e7fc700 1 -- 172.21.15.163:0/2572152356 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa744009e80 con 0x7fa7500c9510 2023-10-09T06:30:49.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.448+0000 7fa74e7fc700 1 -- 172.21.15.163:0/2572152356 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa74400b860 con 0x7fa7500c9510 2023-10-09T06:30:49.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.448+0000 7fa74e7fc700 1 -- 172.21.15.163:0/2572152356 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74400bbe0 con 0x7fa7500c9510 2023-10-09T06:30:49.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.448+0000 7fa83b7fe700 1 --2- 172.21.15.163:0/3144537032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa83c0afc40 0x7fa83c0b1c00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa834009e60 tx=0x7fa834009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb64f0dab2a78fc server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.448+0000 7fdf4dffb700 1 -- 172.21.15.163:0/510119492 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf44011790 con 0x7fdf50158440 2023-10-09T06:30:49.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.448+0000 7fdf4dffb700 1 -- 172.21.15.163:0/510119492 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44011b30 con 0x7fdf50158440 2023-10-09T06:30:49.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.448+0000 7f1059ffb700 1 -- 172.21.15.163:0/1490128315 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1060027b00 con 0x7f10640b0010 2023-10-09T06:30:49.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.448+0000 7f1059ffb700 1 --2- 172.21.15.163:0/1490128315 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1048067080 0x7f1048069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.460 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.448+0000 7fa83a7fc700 1 -- 172.21.15.163:0/3144537032 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83400e070 con 0x7fa83c0afc40 2023-10-09T06:30:49.460 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.448+0000 7fa83a7fc700 1 -- 172.21.15.163:0/3144537032 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83400b7a0 con 0x7fa83c0afc40 2023-10-09T06:30:49.460 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.448+0000 7fa83a7fc700 1 -- 172.21.15.163:0/3144537032 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c001470 con 0x7fa83c0afc40 2023-10-09T06:30:49.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.448+0000 7f1059ffb700 1 -- 172.21.15.163:0/1490128315 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1060022070 con 0x7f10640b0010 2023-10-09T06:30:49.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.448+0000 7f106980b700 1 --2- 172.21.15.163:0/1490128315 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1048067080 0x7f1048069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.448+0000 7f1059ffb700 1 --2- 172.21.15.163:0/1490128315 >> [v2:172.21.15.163:6800/90866,v1:172.21.15.163:6801/90866] conn(0x7f104806f210 0x7f1048071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.448+0000 7f1059ffb700 1 -- 172.21.15.163:0/1490128315 --> [v2:172.21.15.163:6800/90866,v1:172.21.15.163:6801/90866] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1048071cd0 con 0x7f104806f210 2023-10-09T06:30:49.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.449+0000 7f1059ffb700 1 -- 172.21.15.163:0/1490128315 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1060053860 con 0x7f10640b0010 2023-10-09T06:30:49.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.449+0000 7fdf55914700 1 -- 172.21.15.163:0/510119492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf50158440 msgr2=0x7fdf50068aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.449+0000 7fdf55914700 1 --2- 172.21.15.163:0/510119492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf50158440 0x7fdf50068aa0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdf500ad380 tx=0x7fdf440091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.449+0000 7f106a00c700 1 --2- 172.21.15.163:0/1490128315 >> [v2:172.21.15.163:6800/90866,v1:172.21.15.163:6801/90866] conn(0x7f104806f210 0x7f1048071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.449+0000 7fdf55914700 1 -- 172.21.15.163:0/510119492 shutdown_connections 2023-10-09T06:30:49.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.449+0000 7fdf55914700 1 --2- 172.21.15.163:0/510119492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf50158440 0x7fdf50068aa0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.449+0000 7fdf55914700 1 --2- 172.21.15.163:0/510119492 >> v2:172.21.15.163:3300/0 conn(0x7fdf500b21b0 0x7fdf500ab8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.449+0000 7fdf55914700 1 -- 172.21.15.163:0/510119492 >> 172.21.15.163:0/510119492 conn(0x7fdf500cd000 msgr2=0x7fdf50068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:49.463 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.449+0000 7fa755cf0700 1 -- 172.21.15.163:0/2572152356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7500c9510 msgr2=0x7fa750141330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.463 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.449+0000 7fa755cf0700 1 --2- 172.21.15.163:0/2572152356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7500c9510 0x7fa750141330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa7440098e0 tx=0x7fa7440091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.463 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.449+0000 7f106980b700 1 --2- 172.21.15.163:0/1490128315 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1048067080 0x7f1048069530 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1054006fd0 tx=0x7f1054008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.449+0000 7fa755cf0700 1 -- 172.21.15.163:0/2572152356 shutdown_connections 2023-10-09T06:30:49.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.449+0000 7fa755cf0700 1 --2- 172.21.15.163:0/2572152356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7500c9510 0x7fa750141330 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.449+0000 7fa755cf0700 1 --2- 172.21.15.163:0/2572152356 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa75006bcd0 0x7fa750140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.449+0000 7fa755cf0700 1 -- 172.21.15.163:0/2572152356 >> 172.21.15.163:0/2572152356 conn(0x7fa7500ce600 msgr2=0x7fa7500cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:49.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.449+0000 7fa755cf0700 1 -- 172.21.15.163:0/2572152356 shutdown_connections 2023-10-09T06:30:49.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.449+0000 7f106a00c700 1 --2- 172.21.15.163:0/1490128315 >> [v2:172.21.15.163:6800/90866,v1:172.21.15.163:6801/90866] conn(0x7f104806f210 0x7f1048071620 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.449+0000 7fa84206a700 1 -- 172.21.15.163:0/3144537032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa83c0afc40 msgr2=0x7fa83c0b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.449+0000 7fa84206a700 1 --2- 172.21.15.163:0/3144537032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa83c0afc40 0x7fa83c0b1c00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa834009e60 tx=0x7fa834009780 comp rx=0 tx=0).stop 2023-10-09T06:30:49.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.449+0000 7fa84206a700 1 -- 172.21.15.163:0/3144537032 shutdown_connections 2023-10-09T06:30:49.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.449+0000 7fa84206a700 1 --2- 172.21.15.163:0/3144537032 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa83c0b0010 0x7fa83c15df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.449+0000 7fa84206a700 1 --2- 172.21.15.163:0/3144537032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa83c0afc40 0x7fa83c0b1c00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.449+0000 7fa84206a700 1 -- 172.21.15.163:0/3144537032 >> 172.21.15.163:0/3144537032 conn(0x7fa83c06a320 msgr2=0x7fa83c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:49.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.449+0000 7fdf55914700 1 -- 172.21.15.163:0/510119492 shutdown_connections 2023-10-09T06:30:49.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.449+0000 7fdf55914700 1 -- 172.21.15.163:0/510119492 wait complete. 2023-10-09T06:30:49.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.450+0000 7fa84206a700 1 -- 172.21.15.163:0/3144537032 shutdown_connections 2023-10-09T06:30:49.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.450+0000 7f1059ffb700 1 -- 172.21.15.163:0/1490128315 <== osd.2 v2:172.21.15.163:6800/90866 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f1048071cd0 con 0x7f104806f210 2023-10-09T06:30:49.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.450+0000 7fe7fa98b700 1 Processor -- start 2023-10-09T06:30:49.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.450+0000 7fe7fa98b700 1 -- start start 2023-10-09T06:30:49.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.450+0000 7fa84206a700 1 -- 172.21.15.163:0/3144537032 wait complete. 2023-10-09T06:30:49.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.451+0000 7fdf55914700 1 Processor -- start 2023-10-09T06:30:49.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.451+0000 7fdf55914700 1 -- start start 2023-10-09T06:30:49.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.449+0000 7fa755cf0700 1 -- 172.21.15.163:0/2572152356 wait complete. 2023-10-09T06:30:49.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.451+0000 7fa755cf0700 1 Processor -- start 2023-10-09T06:30:49.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.451+0000 7fe7fa98b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe7f415dc70 0x7fe7f40b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.451+0000 7fe7fa98b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7f40cb540 0x7fe7f40cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.451+0000 7fe7fa98b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7f40cebb0 con 0x7fe7f4069f90 2023-10-09T06:30:49.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.451+0000 7fe7fa98b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7f40b31f0 con 0x7fe7f415dc70 2023-10-09T06:30:49.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.451+0000 7fe7fa98b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f40b2f10 con 0x7fe7f40cb540 2023-10-09T06:30:49.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.451+0000 7fe7f37fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe7f4069f90 0x7fe7f40cae30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:47790/0 (socket says 172.21.15.163:47790) 2023-10-09T06:30:49.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.451+0000 7fe7f37fe700 1 -- 172.21.15.163:0/840588599 learned_addr learned my addr 172.21.15.163:0/840588599 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:49.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.451+0000 7fe7f8f28700 1 --2- 172.21.15.163:0/840588599 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7f40cb540 0x7fe7f40cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.451+0000 7fe7f3fff700 1 --2- 172.21.15.163:0/840588599 >> v2:172.21.15.181:3300/0 conn(0x7fe7f415dc70 0x7fe7f40b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.451+0000 7fe7f2ffd700 1 -- 172.21.15.163:0/840588599 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1209051996 0 0) 0x7fe7f40cebb0 con 0x7fe7f4069f90 2023-10-09T06:30:49.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.452+0000 7fe7f2ffd700 1 -- 172.21.15.163:0/840588599 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7dc003570 con 0x7fe7f4069f90 2023-10-09T06:30:49.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.451+0000 7fa84206a700 1 Processor -- start 2023-10-09T06:30:49.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.452+0000 7fe7f8f28700 1 -- 172.21.15.163:0/840588599 >> v2:172.21.15.181:3300/0 conn(0x7fe7f415dc70 msgr2=0x7fe7f40b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:49.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.452+0000 7fe7f8f28700 1 --2- 172.21.15.163:0/840588599 >> v2:172.21.15.181:3300/0 conn(0x7fe7f415dc70 0x7fe7f40b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.452+0000 7fe7f8f28700 1 -- 172.21.15.163:0/840588599 >> v1:172.21.15.181:6789/0 conn(0x7fe7f4069f90 legacy=0x7fe7f40cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.452+0000 7fa84206a700 1 -- start start 2023-10-09T06:30:49.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.452+0000 7fe7f8f28700 1 -- 172.21.15.163:0/840588599 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f40ce560 con 0x7fe7f40cb540 2023-10-09T06:30:49.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.452+0000 7fdf55914700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf50068fe0 0x7fdf50163500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.452+0000 7fdf55914700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf500b21b0 0x7fdf50148670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.452+0000 7fdf55914700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf50158440 0x7fdf5015af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.452+0000 7fdf55914700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf50163a40 con 0x7fdf500b21b0 2023-10-09T06:30:49.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.452+0000 7fdf55914700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf50163070 con 0x7fdf50158440 2023-10-09T06:30:49.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.452+0000 7fdf55914700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf501624a0 con 0x7fdf50068fe0 2023-10-09T06:30:49.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.452+0000 7fe7f8f28700 1 --2- 172.21.15.163:0/840588599 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7f40cb540 0x7fe7f40cf040 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe7e40098e0 tx=0x7fe7e40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63f46a4d770e11ad server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.452+0000 7fdf4effd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf50068fe0 0x7fdf50163500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.452+0000 7fdf4effd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf50068fe0 0x7fdf50163500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:44618/0 (socket says 172.21.15.163:44618) 2023-10-09T06:30:49.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.452+0000 7fdf4effd700 1 -- 172.21.15.163:0/2813587497 learned_addr learned my addr 172.21.15.163:0/2813587497 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:49.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.452+0000 7fdf4e7fc700 1 --2- 172.21.15.163:0/2813587497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf500b21b0 0x7fdf50148670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.452+0000 7fe7f2ffd700 1 -- 172.21.15.163:0/840588599 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4011630 con 0x7fe7f40cb540 2023-10-09T06:30:49.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.452+0000 7fe7f2ffd700 1 -- 172.21.15.163:0/840588599 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e4011790 con 0x7fe7f40cb540 2023-10-09T06:30:49.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.452+0000 7fe7f2ffd700 1 -- 172.21.15.163:0/840588599 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4011b30 con 0x7fe7f40cb540 2023-10-09T06:30:49.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.452+0000 7fdf4e7fc700 1 -- 172.21.15.163:0/2813587497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf50068fe0 msgr2=0x7fdf50163500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fdf4e7fc700 1 --2- 172.21.15.163:0/2813587497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf50068fe0 0x7fdf50163500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fdf4e7fc700 1 -- 172.21.15.163:0/2813587497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf50158440 msgr2=0x7fdf5015af00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-09T06:30:49.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fdf4e7fc700 1 --2- 172.21.15.163:0/2813587497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf50158440 0x7fdf5015af00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fdf4e7fc700 1 -- 172.21.15.163:0/2813587497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf44009420 con 0x7fdf500b21b0 2023-10-09T06:30:49.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fdf4effd700 1 --2- 172.21.15.163:0/2813587497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf50068fe0 0x7fdf50163500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:30:49.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.451+0000 7fa755cf0700 1 -- start start 2023-10-09T06:30:49.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fa755cf0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa75006bcd0 0x7fa7500b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fa755cf0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa75006c0a0 0x7fa7500b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fa755cf0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7500c9510 0x7fa75016e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fa755cf0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7500c8910 con 0x7fa7500c9510 2023-10-09T06:30:49.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fa755cf0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7500c7d00 con 0x7fa75006c0a0 2023-10-09T06:30:49.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fa755cf0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7500c7fe0 con 0x7fa75006bcd0 2023-10-09T06:30:49.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fa74f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa75006bcd0 0x7fa7500b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fa74f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa75006bcd0 0x7fa7500b35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:44640/0 (socket says 172.21.15.163:44640) 2023-10-09T06:30:49.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fa74f7fe700 1 -- 172.21.15.163:0/2531632320 learned_addr learned my addr 172.21.15.163:0/2531632320 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:49.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fa74ffff700 1 --2- 172.21.15.163:0/2531632320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7500c9510 0x7fa75016e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.481 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fa84206a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa83c0afc40 0x7fa83c142550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.481 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fa84206a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa83c0b0010 0x7fa83c1444b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.481 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fa84206a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa83c0b2140 0x7fa83c15f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.481 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fa84206a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c143740 con 0x7fa83c0b0010 2023-10-09T06:30:49.481 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fa84206a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c142b30 con 0x7fa83c0b2140 2023-10-09T06:30:49.481 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fa84206a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c142e10 con 0x7fa83c0afc40 2023-10-09T06:30:49.481 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fa83b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa83c0afc40 0x7fa83c142550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.481 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fa83affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa83c0b0010 0x7fa83c1444b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.481 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fa83affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa83c0b0010 0x7fa83c1444b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43750/0 (socket says 172.21.15.163:43750) 2023-10-09T06:30:49.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fa83affd700 1 -- 172.21.15.163:0/4197527724 learned_addr learned my addr 172.21.15.163:0/4197527724 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:49.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fdf4e7fc700 1 --2- 172.21.15.163:0/2813587497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf500b21b0 0x7fdf50148670 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000c580 tx=0x7fdf40007d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fdf36ffd700 1 -- 172.21.15.163:0/2813587497 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf40005c00 con 0x7fdf500b21b0 2023-10-09T06:30:49.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fdf36ffd700 1 -- 172.21.15.163:0/2813587497 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf40005d60 con 0x7fdf500b21b0 2023-10-09T06:30:49.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fdf36ffd700 1 -- 172.21.15.163:0/2813587497 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf400072e0 con 0x7fdf500b21b0 2023-10-09T06:30:49.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fdf55914700 1 -- 172.21.15.163:0/2813587497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf50162740 con 0x7fdf500b21b0 2023-10-09T06:30:49.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fdf55914700 1 -- 172.21.15.163:0/2813587497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf50159400 con 0x7fdf500b21b0 2023-10-09T06:30:49.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fa74ffff700 1 -- 172.21.15.163:0/2531632320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa75006bcd0 msgr2=0x7fa7500b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fa74ffff700 1 --2- 172.21.15.163:0/2531632320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa75006bcd0 0x7fa7500b35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fa74ffff700 1 -- 172.21.15.163:0/2531632320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa75006c0a0 msgr2=0x7fa7500b3af0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-09T06:30:49.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fa74ffff700 1 --2- 172.21.15.163:0/2531632320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa75006c0a0 0x7fa7500b3af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fa74ffff700 1 -- 172.21.15.163:0/2531632320 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa744009420 con 0x7fa7500c9510 2023-10-09T06:30:49.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.454+0000 7fa74ffff700 1 --2- 172.21.15.163:0/2531632320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7500c9510 0x7fa75016e4a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa744016000 tx=0x7fa744017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.454+0000 7fa7377fe700 1 -- 172.21.15.163:0/2531632320 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa744006e50 con 0x7fa7500c9510 2023-10-09T06:30:49.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fe7fa98b700 1 -- 172.21.15.163:0/840588599 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7f40cb540 msgr2=0x7fe7f40cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fe7fa98b700 1 --2- 172.21.15.163:0/840588599 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7f40cb540 0x7fe7f40cf040 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe7e40098e0 tx=0x7fe7e40091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fe7fa98b700 1 -- 172.21.15.163:0/840588599 shutdown_connections 2023-10-09T06:30:49.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fe7fa98b700 1 --2- 172.21.15.163:0/840588599 >> v2:172.21.15.181:3300/0 conn(0x7fe7f415dc70 0x7fe7f40b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fe7fa98b700 1 --2- 172.21.15.163:0/840588599 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7f40cb540 0x7fe7f40cf040 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.453+0000 7fe7fa98b700 1 -- 172.21.15.163:0/840588599 >> 172.21.15.163:0/840588599 conn(0x7fe7f4068e20 msgr2=0x7fe7f40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:49.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.454+0000 7fe7fa98b700 1 -- 172.21.15.163:0/840588599 shutdown_connections 2023-10-09T06:30:49.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.454+0000 7fa83b7fe700 1 -- 172.21.15.163:0/4197527724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa83c0b2140 msgr2=0x7fa83c15f350 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-09T06:30:49.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.454+0000 7fa83b7fe700 1 --2- 172.21.15.163:0/4197527724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa83c0b2140 0x7fa83c15f350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.454+0000 7fa83b7fe700 1 -- 172.21.15.163:0/4197527724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa83c0b0010 msgr2=0x7fa83c1444b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.454+0000 7fa83b7fe700 1 --2- 172.21.15.163:0/4197527724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa83c0b0010 0x7fa83c1444b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.454+0000 7fa83b7fe700 1 -- 172.21.15.163:0/4197527724 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa834009a20 con 0x7fa83c0afc40 2023-10-09T06:30:49.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.454+0000 7fa7377fe700 1 -- 172.21.15.163:0/2531632320 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa744026450 con 0x7fa7500c9510 2023-10-09T06:30:49.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.454+0000 7fa755cf0700 1 -- 172.21.15.163:0/2531632320 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7500c82c0 con 0x7fa7500c9510 2023-10-09T06:30:49.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.454+0000 7fa755cf0700 1 -- 172.21.15.163:0/2531632320 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7500c5860 con 0x7fa7500c9510 2023-10-09T06:30:49.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.454+0000 7fa7377fe700 1 -- 172.21.15.163:0/2531632320 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7440267f0 con 0x7fa7500c9510 2023-10-09T06:30:49.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.455+0000 7fdf36ffd700 1 -- 172.21.15.163:0/2813587497 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdf40007690 con 0x7fdf500b21b0 2023-10-09T06:30:49.486 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.454+0000 7fa83b7fe700 1 --2- 172.21.15.163:0/4197527724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa83c0afc40 0x7fa83c142550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa834015000 tx=0x7fa834018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.486 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.455+0000 7fa755cf0700 1 -- 172.21.15.163:0/2531632320 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa730000fd0 con 0x7fa7500c9510 2023-10-09T06:30:49.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.455+0000 7fa8237fe700 1 -- 172.21.15.163:0/4197527724 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8340128f0 con 0x7fa83c0afc40 2023-10-09T06:30:49.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.455+0000 7fa8237fe700 1 -- 172.21.15.163:0/4197527724 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa834012ab0 con 0x7fa83c0afc40 2023-10-09T06:30:49.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.454+0000 7fe7fa98b700 1 -- 172.21.15.163:0/840588599 wait complete. 2023-10-09T06:30:49.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.455+0000 7fa84206a700 1 -- 172.21.15.163:0/4197527724 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa83c1430f0 con 0x7fa83c0afc40 2023-10-09T06:30:49.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.455+0000 7fe7fa98b700 1 Processor -- start 2023-10-09T06:30:49.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.455+0000 7fa8237fe700 1 -- 172.21.15.163:0/4197527724 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa834012e70 con 0x7fa83c0afc40 2023-10-09T06:30:49.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.455+0000 7fdf36ffd700 1 --2- 172.21.15.163:0/2813587497 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdf3c066f30 0x7fdf3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.455+0000 7fdf36ffd700 1 -- 172.21.15.163:0/2813587497 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdf400837d0 con 0x7fdf500b21b0 2023-10-09T06:30:49.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.456+0000 7fdf4effd700 1 --2- 172.21.15.163:0/2813587497 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdf3c066f30 0x7fdf3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.449+0000 7f31a58e0700 1 Processor -- start 2023-10-09T06:30:49.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.449+0000 7f31a58e0700 1 -- start start 2023-10-09T06:30:49.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.450+0000 7f31a58e0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31a00b1c50 0x7f31a00879a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.450+0000 7f31a58e0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f31a0068aa0 0x7f31a0068f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.450+0000 7f31a58e0700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a0069450 con 0x7f31a00875d0 2023-10-09T06:30:49.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.450+0000 7f31a58e0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a00abfd0 con 0x7f31a0068aa0 2023-10-09T06:30:49.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.450+0000 7f31a58e0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a00acdf0 con 0x7f31a00b1c50 2023-10-09T06:30:49.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.451+0000 7f319f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f31a0068aa0 0x7f31a0068f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.451+0000 7f319effd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f31a00875d0 0x7f31a00ab8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60846/0 (socket says 172.21.15.163:60846) 2023-10-09T06:30:49.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.451+0000 7f319f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f31a0068aa0 0x7f31a0068f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:50304/0 (socket says 172.21.15.163:50304) 2023-10-09T06:30:49.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.451+0000 7f319effd700 1 -- 172.21.15.163:0/1421882178 learned_addr learned my addr 172.21.15.163:0/1421882178 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:49.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.451+0000 7f319e7fc700 1 --2- 172.21.15.163:0/1421882178 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31a00b1c50 0x7f31a00879a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.456+0000 7fa7f302d700 1 Processor -- start 2023-10-09T06:30:49.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.456+0000 7f319e7fc700 1 -- 172.21.15.163:0/1421882178 >> v2:172.21.15.181:3300/0 conn(0x7f31a0068aa0 msgr2=0x7f31a0068f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:49.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.456+0000 7f319e7fc700 1 --2- 172.21.15.163:0/1421882178 >> v2:172.21.15.181:3300/0 conn(0x7f31a0068aa0 0x7f31a0068f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.456+0000 7f319e7fc700 1 -- 172.21.15.163:0/1421882178 >> v1:172.21.15.163:6789/0 conn(0x7f31a00875d0 legacy=0x7f31a00ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.456+0000 7f319e7fc700 1 -- 172.21.15.163:0/1421882178 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a00ac1e0 con 0x7f31a00b1c50 2023-10-09T06:30:49.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.456+0000 7fa7377fe700 1 -- 172.21.15.163:0/2531632320 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa744026c20 con 0x7fa7500c9510 2023-10-09T06:30:49.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.451+0000 7f319dffb700 1 -- 172.21.15.163:0/1421882178 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3698437679 0 0) 0x7f31a0069450 con 0x7f31a00875d0 2023-10-09T06:30:49.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.456+0000 7fdf55914700 1 --2- 172.21.15.163:0/2813587497 >> [v2:172.21.15.181:6824/91143,v1:172.21.15.181:6825/91143] conn(0x7fdf50162930 0x7fdf501563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.456+0000 7fdf55914700 1 -- 172.21.15.163:0/2813587497 --> [v2:172.21.15.181:6824/91143,v1:172.21.15.181:6825/91143] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf5015e650 con 0x7fdf50162930 2023-10-09T06:30:49.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.456+0000 7fa84206a700 1 -- 172.21.15.163:0/4197527724 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa83c1449f0 con 0x7fa83c0afc40 2023-10-09T06:30:49.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.457+0000 7fa7377fe700 1 --2- 172.21.15.163:0/2531632320 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa73c066f30 0x7fa73c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.457+0000 7fa7377fe700 1 -- 172.21.15.163:0/2531632320 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa744086280 con 0x7fa7500c9510 2023-10-09T06:30:49.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.457+0000 7fe7fa98b700 1 -- start start 2023-10-09T06:30:49.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.457+0000 7fa7377fe700 1 --2- 172.21.15.163:0/2531632320 >> [v2:172.21.15.163:6824/90867,v1:172.21.15.163:6825/90867] conn(0x7fa73c06f140 0x7fa73c071550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.457+0000 7fa7377fe700 1 -- 172.21.15.163:0/2531632320 --> [v2:172.21.15.163:6824/90867,v1:172.21.15.163:6825/90867] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa73c071c00 con 0x7fa73c06f140 2023-10-09T06:30:49.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.457+0000 7fa7377fe700 1 -- 172.21.15.163:0/2531632320 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa744086650 con 0x7fa7500c9510 2023-10-09T06:30:49.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.457+0000 7fa84206a700 1 -- 172.21.15.163:0/4197527724 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa83c04c3c0 con 0x7fa83c0afc40 2023-10-09T06:30:49.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.457+0000 7fa8237fe700 1 -- 172.21.15.163:0/4197527724 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa834024070 con 0x7fa83c0afc40 2023-10-09T06:30:49.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.457+0000 7f319e7fc700 1 --2- 172.21.15.163:0/1421882178 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31a00b1c50 0x7f31a00879a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f31980061a0 tx=0x7f319800a320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63490d4114e7c5e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7f319dffb700 1 -- 172.21.15.163:0/1421882178 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319800c0d0 con 0x7f31a00b1c50 2023-10-09T06:30:49.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7f319dffb700 1 -- 172.21.15.163:0/1421882178 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319800c230 con 0x7f31a00b1c50 2023-10-09T06:30:49.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7f319dffb700 1 -- 172.21.15.163:0/1421882178 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319800a6b0 con 0x7f31a00b1c50 2023-10-09T06:30:49.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7fa74f7fe700 1 --2- 172.21.15.163:0/2531632320 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa73c066f30 0x7fa73c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7fa74effd700 1 --2- 172.21.15.163:0/2531632320 >> [v2:172.21.15.163:6824/90867,v1:172.21.15.163:6825/90867] conn(0x7fa73c06f140 0x7fa73c071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7fe7fa98b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7f4069f90 0x7fe7f40b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7fe7fa98b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7f40cb540 0x7fe7f4144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7fe7fa98b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7f415dc70 0x7fe7f40ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7fe7fa98b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f41568b0 con 0x7fe7f415dc70 2023-10-09T06:30:49.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7fe7fa98b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f40b5610 con 0x7fe7f40cb540 2023-10-09T06:30:49.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7fe7fa98b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f4145110 con 0x7fe7f4069f90 2023-10-09T06:30:49.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7fe7f8f28700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7f415dc70 0x7fe7f40ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7fe7f8f28700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7f415dc70 0x7fe7f40ca990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43780/0 (socket says 172.21.15.163:43780) 2023-10-09T06:30:49.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7fe7f8f28700 1 -- 172.21.15.163:0/913241622 learned_addr learned my addr 172.21.15.163:0/913241622 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:49.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7fe7f3fff700 1 --2- 172.21.15.163:0/913241622 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7f4069f90 0x7fe7f40b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7fe7f37fe700 1 --2- 172.21.15.163:0/913241622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7f40cb540 0x7fe7f4144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7fa8237fe700 1 --2- 172.21.15.163:0/4197527724 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa828067000 0x7fa8280694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7fa8237fe700 1 -- 172.21.15.163:0/4197527724 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa834085e50 con 0x7fa83c0afc40 2023-10-09T06:30:49.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7fa8237fe700 1 --2- 172.21.15.163:0/4197527724 >> [v2:172.21.15.163:6812/90865,v1:172.21.15.163:6814/90865] conn(0x7fa82806f210 0x7fa828071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7fa8237fe700 1 -- 172.21.15.163:0/4197527724 --> [v2:172.21.15.163:6812/90865,v1:172.21.15.163:6814/90865] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa828071cd0 con 0x7fa82806f210 2023-10-09T06:30:49.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7fa8237fe700 1 -- 172.21.15.163:0/4197527724 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa834086150 con 0x7fa83c0afc40 2023-10-09T06:30:49.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7fa83bfff700 1 --2- 172.21.15.163:0/4197527724 >> [v2:172.21.15.163:6812/90865,v1:172.21.15.163:6814/90865] conn(0x7fa82806f210 0x7fa828071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7fa83affd700 1 --2- 172.21.15.163:0/4197527724 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa828067000 0x7fa8280694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7fdf4f7fe700 1 --2- 172.21.15.163:0/2813587497 >> [v2:172.21.15.181:6824/91143,v1:172.21.15.181:6825/91143] conn(0x7fdf50162930 0x7fdf501563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.456+0000 7fa7f302d700 1 -- start start 2023-10-09T06:30:49.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.457+0000 7fa7f302d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7ec0cba20 0x7fa7ec0ac970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7fa7f302d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7ec0b2490 con 0x7fa7ec068aa0 2023-10-09T06:30:49.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7fa7f302d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7ec0ac4e0 con 0x7fa7ec156760 2023-10-09T06:30:49.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7fa7f302d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec0aceb0 con 0x7fa7ec0cba20 2023-10-09T06:30:49.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7fa7f0dc9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7ec0cba20 0x7fa7ec0ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7fa7f0dc9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7ec0cba20 0x7fa7ec0ac970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:44656/0 (socket says 172.21.15.163:44656) 2023-10-09T06:30:49.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.458+0000 7fa7f0dc9700 1 -- 172.21.15.163:0/1788457645 learned_addr learned my addr 172.21.15.163:0/1788457645 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:49.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.459+0000 7fe7f3fff700 1 -- 172.21.15.163:0/913241622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7f40cb540 msgr2=0x7fe7f4144bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.459+0000 7fe7f3fff700 1 --2- 172.21.15.163:0/913241622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7f40cb540 0x7fe7f4144bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.459+0000 7fe7f3fff700 1 -- 172.21.15.163:0/913241622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7f415dc70 msgr2=0x7fe7f40ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.459+0000 7fe7f3fff700 1 --2- 172.21.15.163:0/913241622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7f415dc70 0x7fe7f40ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.459+0000 7fe7f3fff700 1 -- 172.21.15.163:0/913241622 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e4009420 con 0x7fe7f4069f90 2023-10-09T06:30:49.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.459+0000 7fe7f8f28700 1 --2- 172.21.15.163:0/913241622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7f415dc70 0x7fe7f40ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:30:49.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.459+0000 7fa83bfff700 1 --2- 172.21.15.163:0/4197527724 >> [v2:172.21.15.163:6812/90865,v1:172.21.15.163:6814/90865] conn(0x7fa82806f210 0x7fa828071620 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.459+0000 7fa7eb7fe700 1 -- 172.21.15.163:0/1788457645 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1653414197 0 0) 0x7fa7e4001490 con 0x7fa7ec068aa0 2023-10-09T06:30:49.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.459+0000 7fa7eb7fe700 1 -- 172.21.15.163:0/1788457645 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7d4003570 con 0x7fa7ec068aa0 2023-10-09T06:30:49.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.459+0000 7fa74effd700 1 --2- 172.21.15.163:0/2531632320 >> [v2:172.21.15.163:6824/90867,v1:172.21.15.163:6825/90867] conn(0x7fa73c06f140 0x7fa73c071550 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.459+0000 7fa7f0dc9700 1 -- 172.21.15.163:0/1788457645 >> v1:172.21.15.181:6789/0 conn(0x7fa7ec156760 legacy=0x7fa7ec0ad630 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-09T06:30:49.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.459+0000 7fa7f0dc9700 1 -- 172.21.15.163:0/1788457645 >> v1:172.21.15.163:6789/0 conn(0x7fa7ec068aa0 legacy=0x7fa7ec156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.459+0000 7fa7f0dc9700 1 -- 172.21.15.163:0/1788457645 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ec156cd0 con 0x7fa7ec0cba20 2023-10-09T06:30:49.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.459+0000 7fa7f0dc9700 1 --2- 172.21.15.163:0/1788457645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7ec0cba20 0x7fa7ec0ac970 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00099e0 tx=0x7fa7e0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8f7a0052e7ae39b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.459+0000 7fa7eb7fe700 1 -- 172.21.15.163:0/1788457645 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e000b820 con 0x7fa7ec0cba20 2023-10-09T06:30:49.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.459+0000 7fa7eb7fe700 1 -- 172.21.15.163:0/1788457645 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e000b980 con 0x7fa7ec0cba20 2023-10-09T06:30:49.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.459+0000 7fa7eb7fe700 1 -- 172.21.15.163:0/1788457645 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e000bd20 con 0x7fa7ec0cba20 2023-10-09T06:30:49.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.459+0000 7fdf4effd700 1 --2- 172.21.15.163:0/2813587497 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdf3c066f30 0x7fdf3c0693e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdf440093c0 tx=0x7fdf44017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.459+0000 7fa83affd700 1 --2- 172.21.15.163:0/4197527724 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa828067000 0x7fa8280694b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa83c154530 tx=0x7fa82c00a490 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.459+0000 7fa8237fe700 1 -- 172.21.15.163:0/4197527724 <== osd.3 v2:172.21.15.163:6812/90865 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fa828071cd0 con 0x7fa82806f210 2023-10-09T06:30:49.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.460+0000 7fe7f3fff700 1 --2- 172.21.15.163:0/913241622 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7f4069f90 0x7fe7f40b5aa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800e900 tx=0x7fe7e80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.460+0000 7fe7dbfff700 1 -- 172.21.15.163:0/913241622 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8013680 con 0x7fe7f4069f90 2023-10-09T06:30:49.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.460+0000 7fa7377fe700 1 -- 172.21.15.163:0/2531632320 <== osd.0 v2:172.21.15.163:6824/90867 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fa73c071c00 con 0x7fa73c06f140 2023-10-09T06:30:49.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.460+0000 7fa7f302d700 1 -- 172.21.15.163:0/1788457645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7ec0cba20 msgr2=0x7fa7ec0ac970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.460+0000 7fa7f302d700 1 --2- 172.21.15.163:0/1788457645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7ec0cba20 0x7fa7ec0ac970 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00099e0 tx=0x7fa7e0009250 comp rx=0 tx=0).stop 2023-10-09T06:30:49.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.460+0000 7fe7dbfff700 1 -- 172.21.15.163:0/913241622 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e80137e0 con 0x7fe7f4069f90 2023-10-09T06:30:49.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.460+0000 7fe7dbfff700 1 -- 172.21.15.163:0/913241622 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8013ba0 con 0x7fe7f4069f90 2023-10-09T06:30:49.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.461+0000 7fe7fa98b700 1 -- 172.21.15.163:0/913241622 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f40b4a00 con 0x7fe7f4069f90 2023-10-09T06:30:49.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.461+0000 7fe7fa98b700 1 -- 172.21.15.163:0/913241622 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f40b4fc0 con 0x7fe7f4069f90 2023-10-09T06:30:49.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.461+0000 7f31a58e0700 1 -- 172.21.15.163:0/1421882178 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31a00b1c50 msgr2=0x7f31a00879a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.461+0000 7f31a58e0700 1 --2- 172.21.15.163:0/1421882178 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31a00b1c50 0x7f31a00879a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f31980061a0 tx=0x7f319800a320 comp rx=0 tx=0).stop 2023-10-09T06:30:49.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.461+0000 7fe7fa98b700 1 -- 172.21.15.163:0/913241622 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe7f404c3c0 con 0x7fe7f4069f90 2023-10-09T06:30:49.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.462+0000 7fa7f302d700 1 -- 172.21.15.163:0/1788457645 shutdown_connections 2023-10-09T06:30:49.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.462+0000 7fa7f302d700 1 --2- 172.21.15.163:0/1788457645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7ec0cba20 0x7fa7ec0ac970 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.462+0000 7fa7f302d700 1 -- 172.21.15.163:0/1788457645 >> 172.21.15.163:0/1788457645 conn(0x7fa7ec0cd000 msgr2=0x7fa7ec068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:49.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.462+0000 7fa74f7fe700 1 --2- 172.21.15.163:0/2531632320 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa73c066f30 0x7fa73c0693e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa740009ca0 tx=0x7fa740009250 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.462+0000 7fa7f302d700 1 -- 172.21.15.163:0/1788457645 shutdown_connections 2023-10-09T06:30:49.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.462+0000 7fa7f302d700 1 -- 172.21.15.163:0/1788457645 wait complete. 2023-10-09T06:30:49.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.462+0000 7fdf4f7fe700 1 --2- 172.21.15.163:0/2813587497 >> [v2:172.21.15.181:6824/91143,v1:172.21.15.181:6825/91143] conn(0x7fdf50162930 0x7fdf501563a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.462+0000 7fe7dbfff700 1 -- 172.21.15.163:0/913241622 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe7e8005a00 con 0x7fe7f4069f90 2023-10-09T06:30:49.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.463+0000 7fe7dbfff700 1 --2- 172.21.15.163:0/913241622 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe7e0067000 0x7fe7e00694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.463+0000 7fe7dbfff700 1 -- 172.21.15.163:0/913241622 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe7e80848f0 con 0x7fe7f4069f90 2023-10-09T06:30:49.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.463+0000 7fe7f37fe700 1 --2- 172.21.15.163:0/913241622 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe7e0067000 0x7fe7e00694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.463+0000 7fe7dbfff700 1 --2- 172.21.15.163:0/913241622 >> [v2:172.21.15.163:6808/90868,v1:172.21.15.163:6809/90868] conn(0x7fe7e006f210 0x7fe7e0071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.463+0000 7fe7dbfff700 1 -- 172.21.15.163:0/913241622 --> [v2:172.21.15.163:6808/90868,v1:172.21.15.163:6809/90868] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe7e0071cd0 con 0x7fe7e006f210 2023-10-09T06:30:49.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.463+0000 7fe7dbfff700 1 -- 172.21.15.163:0/913241622 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7e8038ea0 con 0x7fe7f4069f90 2023-10-09T06:30:49.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.463+0000 7fe7f8f28700 1 --2- 172.21.15.163:0/913241622 >> [v2:172.21.15.163:6808/90868,v1:172.21.15.163:6809/90868] conn(0x7fe7e006f210 0x7fe7e0071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.463+0000 7fe7f37fe700 1 --2- 172.21.15.163:0/913241622 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe7e0067000 0x7fe7e00694b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe7f40cc840 tx=0x7fe7ec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.464+0000 7fa7f302d700 1 Processor -- start 2023-10-09T06:30:49.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.464+0000 7fe7f8f28700 1 --2- 172.21.15.163:0/913241622 >> [v2:172.21.15.163:6808/90868,v1:172.21.15.163:6809/90868] conn(0x7fe7e006f210 0x7fe7e0071620 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.464+0000 7fa7f302d700 1 -- start start 2023-10-09T06:30:49.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.464+0000 7fe7dbfff700 1 -- 172.21.15.163:0/913241622 <== osd.1 v2:172.21.15.163:6808/90868 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fe7e0071cd0 con 0x7fe7e006f210 2023-10-09T06:30:49.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.465+0000 7fdf36ffd700 1 -- 172.21.15.163:0/2813587497 <== osd.4 v2:172.21.15.181:6824/91143 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fdf5015e650 con 0x7fdf50162930 2023-10-09T06:30:49.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.465+0000 7f31a58e0700 1 -- 172.21.15.163:0/1421882178 shutdown_connections 2023-10-09T06:30:49.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.465+0000 7f31a58e0700 1 --2- 172.21.15.163:0/1421882178 >> v2:172.21.15.181:3300/0 conn(0x7f31a0068aa0 0x7f31a0068f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.465+0000 7f31a58e0700 1 --2- 172.21.15.163:0/1421882178 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31a00b1c50 0x7f31a00879a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.465+0000 7f31a58e0700 1 -- 172.21.15.163:0/1421882178 >> 172.21.15.163:0/1421882178 conn(0x7f31a00cd000 msgr2=0x7f31a0068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:49.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.465+0000 7fa7f302d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7ec068aa0 0x7fa7ec0a7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.465+0000 7fa7f302d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7ec0cba20 0x7fa7ec0a93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.465+0000 7fa7f302d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7ec156760 0x7fa7ec0aa850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.465+0000 7fa7f302d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec0a73f0 con 0x7fa7ec0cba20 2023-10-09T06:30:49.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.465+0000 7fa7f302d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec0a67e0 con 0x7fa7ec156760 2023-10-09T06:30:49.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.465+0000 7fa7f302d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec0a6ac0 con 0x7fa7ec068aa0 2023-10-09T06:30:49.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.465+0000 7f31a58e0700 1 -- 172.21.15.163:0/1421882178 shutdown_connections 2023-10-09T06:30:49.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.465+0000 7f31a58e0700 1 -- 172.21.15.163:0/1421882178 wait complete. 2023-10-09T06:30:49.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.467+0000 7f31a58e0700 1 Processor -- start 2023-10-09T06:30:49.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.465+0000 7fa7ebfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7ec0cba20 0x7fa7ec0a93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.465+0000 7fa7ebfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7ec0cba20 0x7fa7ec0a93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43794/0 (socket says 172.21.15.163:43794) 2023-10-09T06:30:49.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.465+0000 7fa7ebfff700 1 -- 172.21.15.163:0/1082161781 learned_addr learned my addr 172.21.15.163:0/1082161781 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:49.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.466+0000 7fa7ebfff700 1 -- 172.21.15.163:0/1082161781 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7ec068aa0 msgr2=0x7fa7ec0a7880 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-09T06:30:49.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.466+0000 7fa7ebfff700 1 --2- 172.21.15.163:0/1082161781 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7ec068aa0 0x7fa7ec0a7880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.466+0000 7fa7ebfff700 1 -- 172.21.15.163:0/1082161781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7ec156760 msgr2=0x7fa7ec0aa850 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-09T06:30:49.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.466+0000 7fa7ebfff700 1 --2- 172.21.15.163:0/1082161781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7ec156760 0x7fa7ec0aa850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.466+0000 7fa7ebfff700 1 -- 172.21.15.163:0/1082161781 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e00094f0 con 0x7fa7ec0cba20 2023-10-09T06:30:49.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.466+0000 7fa7ebfff700 1 --2- 172.21.15.163:0/1082161781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7ec0cba20 0x7fa7ec0a93a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa7e400cb40 tx=0x7fa7e40027f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.467+0000 7fa7e8ff9700 1 -- 172.21.15.163:0/1082161781 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e400a7e0 con 0x7fa7ec0cba20 2023-10-09T06:30:49.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.467+0000 7fa7e8ff9700 1 -- 172.21.15.163:0/1082161781 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e400a940 con 0x7fa7ec0cba20 2023-10-09T06:30:49.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.467+0000 7fa7e8ff9700 1 -- 172.21.15.163:0/1082161781 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e400ad00 con 0x7fa7ec0cba20 2023-10-09T06:30:49.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.468+0000 7f31a58e0700 1 -- start start 2023-10-09T06:30:49.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.469+0000 7f31a58e0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31a0068aa0 0x7f31a0172c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.469+0000 7f31a58e0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31a00875d0 0x7f31a0181810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.469+0000 7f31a58e0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31a00b1c50 0x7f31a0181d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.469+0000 7f31a58e0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a016a830 con 0x7f31a0068aa0 2023-10-09T06:30:49.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.469+0000 7f31a58e0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a0182290 con 0x7f31a00875d0 2023-10-09T06:30:49.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.469+0000 7f31a58e0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a016a200 con 0x7f31a00b1c50 2023-10-09T06:30:49.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.469+0000 7f319effd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31a0068aa0 0x7f31a0172c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.469+0000 7f319effd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31a0068aa0 0x7f31a0172c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43796/0 (socket says 172.21.15.163:43796) 2023-10-09T06:30:49.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.469+0000 7f319effd700 1 -- 172.21.15.163:0/4073745895 learned_addr learned my addr 172.21.15.163:0/4073745895 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:49.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.469+0000 7f319effd700 1 -- 172.21.15.163:0/4073745895 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31a00b1c50 msgr2=0x7f31a0181d50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-09T06:30:49.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.469+0000 7f319effd700 1 --2- 172.21.15.163:0/4073745895 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31a00b1c50 0x7f31a0181d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.470+0000 7f319effd700 1 -- 172.21.15.163:0/4073745895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31a00875d0 msgr2=0x7f31a0181810 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-09T06:30:49.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.470+0000 7f319effd700 1 --2- 172.21.15.163:0/4073745895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31a00875d0 0x7f31a0181810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.470+0000 7f319effd700 1 -- 172.21.15.163:0/4073745895 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319800c8d0 con 0x7f31a0068aa0 2023-10-09T06:30:49.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.470+0000 7f319effd700 1 --2- 172.21.15.163:0/4073745895 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31a0068aa0 0x7f31a0172c00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f319000c5b0 tx=0x7f3190001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.469+0000 7fa7f302d700 1 -- 172.21.15.163:0/1082161781 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7ec0a6da0 con 0x7fa7ec0cba20 2023-10-09T06:30:49.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.469+0000 7fa7f302d700 1 -- 172.21.15.163:0/1082161781 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7ec0a6490 con 0x7fa7ec0cba20 2023-10-09T06:30:49.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.472+0000 7fa7e8ff9700 1 -- 172.21.15.163:0/1082161781 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa7e4012630 con 0x7fa7ec0cba20 2023-10-09T06:30:49.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.473+0000 7fa7e8ff9700 1 --2- 172.21.15.163:0/1082161781 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa7d8067080 0x7fa7d8069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.473+0000 7fa7e8ff9700 1 -- 172.21.15.163:0/1082161781 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa7e4084740 con 0x7fa7ec0cba20 2023-10-09T06:30:49.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.473+0000 7fe33871d700 1 Processor -- start 2023-10-09T06:30:49.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.473+0000 7fe33871d700 1 -- start start 2023-10-09T06:30:49.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.473+0000 7fa7f0dc9700 1 --2- 172.21.15.163:0/1082161781 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa7d8067080 0x7fa7d8069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.474+0000 7fa7f0dc9700 1 --2- 172.21.15.163:0/1082161781 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa7d8067080 0x7fa7d8069530 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00099e0 tx=0x7fa7e0016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.475+0000 7fe33871d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe33006dba0 0x7fe33006df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.475+0000 7fe33871d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe3300c9ed0 0x7fe330159980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.475+0000 7fe33871d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe33015a8b0 con 0x7fe3300699d0 2023-10-09T06:30:49.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.475+0000 7fe33871d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3300c7cb0 con 0x7fe33006dba0 2023-10-09T06:30:49.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.475+0000 7fe33871d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe33015a2f0 con 0x7fe3300c9ed0 2023-10-09T06:30:49.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.475+0000 7fe335cb8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe3300c9ed0 0x7fe330159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.475+0000 7fe336cba700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe3300699d0 0x7fe3300c8140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:47802/0 (socket says 172.21.15.163:47802) 2023-10-09T06:30:49.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.475+0000 7fe335cb8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe3300c9ed0 0x7fe330159980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:50366/0 (socket says 172.21.15.163:50366) 2023-10-09T06:30:49.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.475+0000 7fe336cba700 1 -- 172.21.15.163:0/2558540685 learned_addr learned my addr 172.21.15.163:0/2558540685 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:49.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.475+0000 7fe335cb8700 1 -- 172.21.15.163:0/2558540685 >> v2:172.21.15.163:3300/0 conn(0x7fe33006dba0 msgr2=0x7fe33006df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:49.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.475+0000 7fe335cb8700 1 --2- 172.21.15.163:0/2558540685 >> v2:172.21.15.163:3300/0 conn(0x7fe33006dba0 0x7fe33006df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.475+0000 7fe335cb8700 1 -- 172.21.15.163:0/2558540685 >> v1:172.21.15.181:6789/0 conn(0x7fe3300699d0 legacy=0x7fe3300c8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.475+0000 7fe335cb8700 1 -- 172.21.15.163:0/2558540685 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33015a5d0 con 0x7fe3300c9ed0 2023-10-09T06:30:49.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.475+0000 7fe3354b7700 1 -- 172.21.15.163:0/2558540685 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2819781093 0 0) 0x7fe33015a8b0 con 0x7fe3300699d0 2023-10-09T06:30:49.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.479+0000 7fe335cb8700 1 --2- 172.21.15.163:0/2558540685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3300c9ed0 0x7fe330159980 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe33006d7e0 tx=0x7fe3200091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=160d47e42ba10dd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.479+0000 7f106ba6f700 1 -- 172.21.15.163:0/1490128315 --> [v2:172.21.15.163:6800/90866,v1:172.21.15.163:6801/90866] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f10640ad310 con 0x7f104806f210 2023-10-09T06:30:49.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.479+0000 7f3186ffd700 1 -- 172.21.15.163:0/4073745895 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3190005b30 con 0x7f31a0068aa0 2023-10-09T06:30:49.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.479+0000 7fe3354b7700 1 -- 172.21.15.163:0/2558540685 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32000b6b0 con 0x7fe3300c9ed0 2023-10-09T06:30:49.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.480+0000 7fe3354b7700 1 -- 172.21.15.163:0/2558540685 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe32000b810 con 0x7fe3300c9ed0 2023-10-09T06:30:49.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.480+0000 7fe3354b7700 1 -- 172.21.15.163:0/2558540685 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32000bbb0 con 0x7fe3300c9ed0 2023-10-09T06:30:49.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.479+0000 7f3186ffd700 1 -- 172.21.15.163:0/4073745895 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3190005c90 con 0x7f31a0068aa0 2023-10-09T06:30:49.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.479+0000 7f3186ffd700 1 -- 172.21.15.163:0/4073745895 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3190007320 con 0x7f31a0068aa0 2023-10-09T06:30:49.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.479+0000 7f31a58e0700 1 -- 172.21.15.163:0/4073745895 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a01758b0 con 0x7f31a0068aa0 2023-10-09T06:30:49.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.480+0000 7f31a58e0700 1 -- 172.21.15.163:0/4073745895 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a016a4c0 con 0x7f31a0068aa0 2023-10-09T06:30:49.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.480+0000 7f31a58e0700 1 -- 172.21.15.163:0/4073745895 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f31a004c3c0 con 0x7f31a0068aa0 2023-10-09T06:30:49.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.481+0000 7fe33871d700 1 -- 172.21.15.163:0/2558540685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3300c9ed0 msgr2=0x7fe330159980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.481+0000 7fe33871d700 1 --2- 172.21.15.163:0/2558540685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3300c9ed0 0x7fe330159980 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe33006d7e0 tx=0x7fe3200091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.481+0000 7fe33871d700 1 -- 172.21.15.163:0/2558540685 shutdown_connections 2023-10-09T06:30:49.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.481+0000 7fe33871d700 1 --2- 172.21.15.163:0/2558540685 >> v2:172.21.15.163:3300/0 conn(0x7fe33006dba0 0x7fe33006df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.481+0000 7fe33871d700 1 --2- 172.21.15.163:0/2558540685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3300c9ed0 0x7fe330159980 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.481+0000 7fe33871d700 1 -- 172.21.15.163:0/2558540685 >> 172.21.15.163:0/2558540685 conn(0x7fe330068390 msgr2=0x7fe330068790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:49.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.482+0000 7fe33871d700 1 -- 172.21.15.163:0/2558540685 shutdown_connections 2023-10-09T06:30:49.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.482+0000 7f1059ffb700 1 -- 172.21.15.163:0/1490128315 <== osd.2 v2:172.21.15.163:6800/90866 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f10640ad310 con 0x7f104806f210 2023-10-09T06:30:49.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.482+0000 7fa84206a700 1 -- 172.21.15.163:0/4197527724 --> [v2:172.21.15.163:6812/90865,v1:172.21.15.163:6814/90865] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa83c0ad310 con 0x7fa82806f210 2023-10-09T06:30:49.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.482+0000 7f106ba6f700 1 -- 172.21.15.163:0/1490128315 >> [v2:172.21.15.163:6800/90866,v1:172.21.15.163:6801/90866] conn(0x7f104806f210 msgr2=0x7f1048071620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.482+0000 7f106ba6f700 1 --2- 172.21.15.163:0/1490128315 >> [v2:172.21.15.163:6800/90866,v1:172.21.15.163:6801/90866] conn(0x7f104806f210 0x7f1048071620 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.482+0000 7f106ba6f700 1 -- 172.21.15.163:0/1490128315 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1048067080 msgr2=0x7f1048069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.482+0000 7f106ba6f700 1 --2- 172.21.15.163:0/1490128315 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1048067080 0x7f1048069530 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1054006fd0 tx=0x7f1054008040 comp rx=0 tx=0).stop 2023-10-09T06:30:49.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.482+0000 7f106ba6f700 1 -- 172.21.15.163:0/1490128315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10640b0010 msgr2=0x7f1064152330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.482+0000 7f106ba6f700 1 --2- 172.21.15.163:0/1490128315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10640b0010 0x7f1064152330 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1064064290 tx=0x7f1060016000 comp rx=0 tx=0).stop 2023-10-09T06:30:49.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.483+0000 7f106a00c700 1 -- 172.21.15.163:0/1490128315 reap_dead start 2023-10-09T06:30:49.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.482+0000 7fe33871d700 1 -- 172.21.15.163:0/2558540685 wait complete. 2023-10-09T06:30:49.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.484+0000 7f106ba6f700 1 -- 172.21.15.163:0/1490128315 shutdown_connections 2023-10-09T06:30:49.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.484+0000 7f106ba6f700 1 -- 172.21.15.163:0/1490128315 >> 172.21.15.163:0/1490128315 conn(0x7f106406a320 msgr2=0x7f106415c930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:49.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.484+0000 7fe33871d700 1 Processor -- start 2023-10-09T06:30:49.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.484+0000 7fe33871d700 1 -- start start 2023-10-09T06:30:49.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.484+0000 7f106ba6f700 1 -- 172.21.15.163:0/1490128315 shutdown_connections 2023-10-09T06:30:49.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.484+0000 7f3186ffd700 1 -- 172.21.15.163:0/4073745895 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f319001b3b0 con 0x7f31a0068aa0 2023-10-09T06:30:49.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.484+0000 7f3186ffd700 1 --2- 172.21.15.163:0/4073745895 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f318c067080 0x7f318c069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.484+0000 7f3186ffd700 1 -- 172.21.15.163:0/4073745895 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f31900841e0 con 0x7f31a0068aa0 2023-10-09T06:30:49.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.484+0000 7f319e7fc700 1 --2- 172.21.15.163:0/4073745895 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f318c067080 0x7f318c069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.484+0000 7f3186ffd700 1 --2- 172.21.15.163:0/4073745895 >> [v2:172.21.15.181:6800/91130,v1:172.21.15.181:6801/91130] conn(0x7f318c06f290 0x7f318c0716a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.485+0000 7f3186ffd700 1 -- 172.21.15.163:0/4073745895 --> [v2:172.21.15.181:6800/91130,v1:172.21.15.181:6801/91130] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f318c071d50 con 0x7f318c06f290 2023-10-09T06:30:49.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.485+0000 7f3186ffd700 1 -- 172.21.15.163:0/4073745895 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3190084510 con 0x7f31a0068aa0 2023-10-09T06:30:49.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.485+0000 7f106ba6f700 1 -- 172.21.15.163:0/1490128315 wait complete. 2023-10-09T06:30:49.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.485+0000 7fdf55914700 1 -- 172.21.15.163:0/2813587497 --> [v2:172.21.15.181:6824/91143,v1:172.21.15.181:6825/91143] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdf5006f8b0 con 0x7fdf50162930 2023-10-09T06:30:49.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.485+0000 7f319f7fe700 1 --2- 172.21.15.163:0/4073745895 >> [v2:172.21.15.181:6800/91130,v1:172.21.15.181:6801/91130] conn(0x7f318c06f290 0x7f318c0716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.485+0000 7fe33871d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3300699d0 0x7fe3300c2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.485+0000 7fdf36ffd700 1 -- 172.21.15.163:0/2813587497 <== osd.4 v2:172.21.15.181:6824/91143 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdf5006f8b0 con 0x7fdf50162930 2023-10-09T06:30:49.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.485+0000 7f319e7fc700 1 --2- 172.21.15.163:0/4073745895 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f318c067080 0x7f318c069530 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f31a0172870 tx=0x7f319801a000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.485+0000 7fe33871d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe33006dba0 0x7fe330156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.485+0000 7fe33871d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3300c9ed0 0x7fe330149e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.485+0000 7fe33871d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3300c24d0 con 0x7fe3300c9ed0 2023-10-09T06:30:49.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.485+0000 7fe33871d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3300c18c0 con 0x7fe33006dba0 2023-10-09T06:30:49.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.485+0000 7fe33871d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3300c1ba0 con 0x7fe3300699d0 2023-10-09T06:30:49.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.485+0000 7fe3364b9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3300699d0 0x7fe3300c2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.485+0000 7fe335cb8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe33006dba0 0x7fe330156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.485+0000 7fe3364b9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3300699d0 0x7fe3300c2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:44694/0 (socket says 172.21.15.163:44694) 2023-10-09T06:30:49.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.485+0000 7fe3364b9700 1 -- 172.21.15.163:0/177108980 learned_addr learned my addr 172.21.15.163:0/177108980 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:49.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.485+0000 7f319f7fe700 1 --2- 172.21.15.163:0/4073745895 >> [v2:172.21.15.181:6800/91130,v1:172.21.15.181:6801/91130] conn(0x7f318c06f290 0x7f318c0716a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.485+0000 7fdf55914700 1 -- 172.21.15.163:0/2813587497 >> [v2:172.21.15.181:6824/91143,v1:172.21.15.181:6825/91143] conn(0x7fdf50162930 msgr2=0x7fdf501563a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.485+0000 7fdf55914700 1 --2- 172.21.15.163:0/2813587497 >> [v2:172.21.15.181:6824/91143,v1:172.21.15.181:6825/91143] conn(0x7fdf50162930 0x7fdf501563a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.485+0000 7fe3364b9700 1 -- 172.21.15.163:0/177108980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe33006dba0 msgr2=0x7fe330156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.485+0000 7fe3364b9700 1 --2- 172.21.15.163:0/177108980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe33006dba0 0x7fe330156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.485+0000 7fe3364b9700 1 -- 172.21.15.163:0/177108980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3300c9ed0 msgr2=0x7fe330149e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.485+0000 7fe336cba700 1 --2- 172.21.15.163:0/177108980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3300c9ed0 0x7fe330149e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.485+0000 7fe3364b9700 1 --2- 172.21.15.163:0/177108980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3300c9ed0 0x7fe330149e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.486+0000 7fe3364b9700 1 -- 172.21.15.163:0/177108980 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe320009420 con 0x7fe3300699d0 2023-10-09T06:30:49.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.486+0000 7fdf55914700 1 -- 172.21.15.163:0/2813587497 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdf3c066f30 msgr2=0x7fdf3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.486+0000 7fdf55914700 1 --2- 172.21.15.163:0/2813587497 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdf3c066f30 0x7fdf3c0693e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdf440093c0 tx=0x7fdf44017000 comp rx=0 tx=0).stop 2023-10-09T06:30:49.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.486+0000 7fdf55914700 1 -- 172.21.15.163:0/2813587497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf500b21b0 msgr2=0x7fdf50148670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.486+0000 7fdf55914700 1 --2- 172.21.15.163:0/2813587497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf500b21b0 0x7fdf50148670 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000c580 tx=0x7fdf40007d50 comp rx=0 tx=0).stop 2023-10-09T06:30:49.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.486+0000 7fe336cba700 1 --2- 172.21.15.163:0/177108980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3300c9ed0 0x7fe330149e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:30:49.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.486+0000 7fdf4f7fe700 1 -- 172.21.15.163:0/2813587497 reap_dead start 2023-10-09T06:30:49.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.486+0000 7fdf55914700 1 -- 172.21.15.163:0/2813587497 shutdown_connections 2023-10-09T06:30:49.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.486+0000 7fdf55914700 1 -- 172.21.15.163:0/2813587497 >> 172.21.15.163:0/2813587497 conn(0x7fdf500cd000 msgr2=0x7fdf500b1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:49.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.486+0000 7fdf55914700 1 -- 172.21.15.163:0/2813587497 shutdown_connections 2023-10-09T06:30:49.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.486+0000 7fdf55914700 1 -- 172.21.15.163:0/2813587497 wait complete. 2023-10-09T06:30:49.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.486+0000 7fe3364b9700 1 --2- 172.21.15.163:0/177108980 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3300699d0 0x7fe3300c2960 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe32c011e40 tx=0x7fe32c011af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.486+0000 7f3186ffd700 1 -- 172.21.15.163:0/4073745895 <== osd.5 v2:172.21.15.181:6800/91130 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f318c071d50 con 0x7f318c06f290 2023-10-09T06:30:49.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.487+0000 7fe7fa98b700 1 -- 172.21.15.163:0/913241622 --> [v2:172.21.15.163:6808/90868,v1:172.21.15.163:6809/90868] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe7f40ce840 con 0x7fe7e006f210 2023-10-09T06:30:49.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.487+0000 7fa8237fe700 1 -- 172.21.15.163:0/4197527724 <== osd.3 v2:172.21.15.163:6812/90865 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa83c0ad310 con 0x7fa82806f210 2023-10-09T06:30:49.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.488+0000 7fe7dbfff700 1 -- 172.21.15.163:0/913241622 <== osd.1 v2:172.21.15.163:6808/90868 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe7f40ce840 con 0x7fe7e006f210 2023-10-09T06:30:49.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.488+0000 7fe7fa98b700 1 -- 172.21.15.163:0/913241622 >> [v2:172.21.15.163:6808/90868,v1:172.21.15.163:6809/90868] conn(0x7fe7e006f210 msgr2=0x7fe7e0071620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.488+0000 7fe7fa98b700 1 --2- 172.21.15.163:0/913241622 >> [v2:172.21.15.163:6808/90868,v1:172.21.15.163:6809/90868] conn(0x7fe7e006f210 0x7fe7e0071620 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.488+0000 7fa84206a700 1 -- 172.21.15.163:0/4197527724 >> [v2:172.21.15.163:6812/90865,v1:172.21.15.163:6814/90865] conn(0x7fa82806f210 msgr2=0x7fa828071620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.488+0000 7fa84206a700 1 --2- 172.21.15.163:0/4197527724 >> [v2:172.21.15.163:6812/90865,v1:172.21.15.163:6814/90865] conn(0x7fa82806f210 0x7fa828071620 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.488+0000 7fe3267fc700 1 -- 172.21.15.163:0/177108980 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c014c20 con 0x7fe3300699d0 2023-10-09T06:30:49.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.488+0000 7fa84206a700 1 -- 172.21.15.163:0/4197527724 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa828067000 msgr2=0x7fa8280694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.488+0000 7fa84206a700 1 --2- 172.21.15.163:0/4197527724 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa828067000 0x7fa8280694b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa83c154530 tx=0x7fa82c00a490 comp rx=0 tx=0).stop 2023-10-09T06:30:49.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.488+0000 7fa84206a700 1 -- 172.21.15.163:0/4197527724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa83c0afc40 msgr2=0x7fa83c142550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.488+0000 7fa84206a700 1 --2- 172.21.15.163:0/4197527724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa83c0afc40 0x7fa83c142550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa834015000 tx=0x7fa834018000 comp rx=0 tx=0).stop 2023-10-09T06:30:49.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.488+0000 7fe3267fc700 1 -- 172.21.15.163:0/177108980 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe32c014d80 con 0x7fe3300699d0 2023-10-09T06:30:49.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.488+0000 7fe3267fc700 1 -- 172.21.15.163:0/177108980 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c0110d0 con 0x7fe3300699d0 2023-10-09T06:30:49.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.488+0000 7fe7fa98b700 1 -- 172.21.15.163:0/913241622 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe7e0067000 msgr2=0x7fe7e00694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.488+0000 7fe7fa98b700 1 --2- 172.21.15.163:0/913241622 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe7e0067000 0x7fe7e00694b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe7f40cc840 tx=0x7fe7ec009040 comp rx=0 tx=0).stop 2023-10-09T06:30:49.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.488+0000 7fe7fa98b700 1 -- 172.21.15.163:0/913241622 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7f4069f90 msgr2=0x7fe7f40b5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.488+0000 7fe7fa98b700 1 --2- 172.21.15.163:0/913241622 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7f4069f90 0x7fe7f40b5aa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800e900 tx=0x7fe7e80091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.488+0000 7fa83bfff700 1 -- 172.21.15.163:0/4197527724 reap_dead start 2023-10-09T06:30:49.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.488+0000 7fe7f8f28700 1 -- 172.21.15.163:0/913241622 reap_dead start 2023-10-09T06:30:49.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.489+0000 7fe7fa98b700 1 -- 172.21.15.163:0/913241622 shutdown_connections 2023-10-09T06:30:49.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.489+0000 7fe7fa98b700 1 -- 172.21.15.163:0/913241622 >> 172.21.15.163:0/913241622 conn(0x7fe7f4068e20 msgr2=0x7fe7f40b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:49.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.489+0000 7fe7fa98b700 1 -- 172.21.15.163:0/913241622 shutdown_connections 2023-10-09T06:30:49.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.489+0000 7fe7fa98b700 1 -- 172.21.15.163:0/913241622 wait complete. 2023-10-09T06:30:49.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.489+0000 7fe33871d700 1 -- 172.21.15.163:0/177108980 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3300c1e80 con 0x7fe3300699d0 2023-10-09T06:30:49.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.489+0000 7fe33871d700 1 -- 172.21.15.163:0/177108980 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe330145b60 con 0x7fe3300699d0 2023-10-09T06:30:49.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.490+0000 7fe33871d700 1 -- 172.21.15.163:0/177108980 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe3300c2ea0 con 0x7fe3300699d0 2023-10-09T06:30:49.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.490+0000 7fe3267fc700 1 -- 172.21.15.163:0/177108980 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe32c01a020 con 0x7fe3300699d0 2023-10-09T06:30:49.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.491+0000 7fe3267fc700 1 --2- 172.21.15.163:0/177108980 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe314067000 0x7fe3140694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.491+0000 7fe3267fc700 1 -- 172.21.15.163:0/177108980 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe32c08bc40 con 0x7fe3300699d0 2023-10-09T06:30:49.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.491+0000 7fe335cb8700 1 --2- 172.21.15.163:0/177108980 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe314067000 0x7fe3140694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.491+0000 7fe3267fc700 1 --2- 172.21.15.163:0/177108980 >> [v2:172.21.15.181:6816/91132,v1:172.21.15.181:6817/91132] conn(0x7fe31406f190 0x7fe3140715a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.491+0000 7fe3267fc700 1 -- 172.21.15.163:0/177108980 --> [v2:172.21.15.181:6816/91132,v1:172.21.15.181:6817/91132] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe314071c50 con 0x7fe31406f190 2023-10-09T06:30:49.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.492+0000 7fe336cba700 1 --2- 172.21.15.163:0/177108980 >> [v2:172.21.15.181:6816/91132,v1:172.21.15.181:6817/91132] conn(0x7fe31406f190 0x7fe3140715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.492+0000 7fa7f302d700 1 --2- 172.21.15.163:0/1082161781 >> [v2:172.21.15.181:6808/91131,v1:172.21.15.181:6809/91131] conn(0x7fa7ec0a7dc0 0x7fa7ec0aad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.492+0000 7fa7f302d700 1 -- 172.21.15.163:0/1082161781 --> [v2:172.21.15.181:6808/91131,v1:172.21.15.181:6809/91131] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa7ec04c3c0 con 0x7fa7ec0a7dc0 2023-10-09T06:30:49.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.492+0000 7fa7f15ca700 1 --2- 172.21.15.163:0/1082161781 >> [v2:172.21.15.181:6808/91131,v1:172.21.15.181:6809/91131] conn(0x7fa7ec0a7dc0 0x7fa7ec0aad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.492+0000 7fe336cba700 1 --2- 172.21.15.163:0/177108980 >> [v2:172.21.15.181:6816/91132,v1:172.21.15.181:6817/91132] conn(0x7fe31406f190 0x7fe3140715a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.492+0000 7fe3267fc700 1 -- 172.21.15.163:0/177108980 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe32c058f90 con 0x7fe3300699d0 2023-10-09T06:30:49.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.492+0000 7fa755cf0700 1 -- 172.21.15.163:0/2531632320 --> [v2:172.21.15.163:6824/90867,v1:172.21.15.163:6825/90867] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa730002d30 con 0x7fa73c06f140 2023-10-09T06:30:49.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.493+0000 7fa7377fe700 1 -- 172.21.15.163:0/2531632320 <== osd.0 v2:172.21.15.163:6824/90867 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa730002d30 con 0x7fa73c06f140 2023-10-09T06:30:49.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.493+0000 7fe3267fc700 1 -- 172.21.15.163:0/177108980 <== osd.7 v2:172.21.15.181:6816/91132 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fe314071c50 con 0x7fe31406f190 2023-10-09T06:30:49.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.493+0000 7fa7f15ca700 1 --2- 172.21.15.163:0/1082161781 >> [v2:172.21.15.181:6808/91131,v1:172.21.15.181:6809/91131] conn(0x7fa7ec0a7dc0 0x7fa7ec0aad90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.494+0000 7fa7e8ff9700 1 -- 172.21.15.163:0/1082161781 <== osd.6 v2:172.21.15.181:6808/91131 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fa7ec04c3c0 con 0x7fa7ec0a7dc0 2023-10-09T06:30:49.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.497+0000 7fa84206a700 1 -- 172.21.15.163:0/4197527724 shutdown_connections 2023-10-09T06:30:49.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.497+0000 7fa84206a700 1 -- 172.21.15.163:0/4197527724 >> 172.21.15.163:0/4197527724 conn(0x7fa83c06a320 msgr2=0x7fa83c0af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:49.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.498+0000 7fa84206a700 1 -- 172.21.15.163:0/4197527724 shutdown_connections 2023-10-09T06:30:49.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.498+0000 7fe335cb8700 1 --2- 172.21.15.163:0/177108980 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe314067000 0x7fe3140694b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe3200093f0 tx=0x7fe320017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.493+0000 7fa755cf0700 1 -- 172.21.15.163:0/2531632320 >> [v2:172.21.15.163:6824/90867,v1:172.21.15.163:6825/90867] conn(0x7fa73c06f140 msgr2=0x7fa73c071550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.493+0000 7fa755cf0700 1 --2- 172.21.15.163:0/2531632320 >> [v2:172.21.15.163:6824/90867,v1:172.21.15.163:6825/90867] conn(0x7fa73c06f140 0x7fa73c071550 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.493+0000 7fa755cf0700 1 -- 172.21.15.163:0/2531632320 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa73c066f30 msgr2=0x7fa73c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.493+0000 7fa755cf0700 1 --2- 172.21.15.163:0/2531632320 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa73c066f30 0x7fa73c0693e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa740009ca0 tx=0x7fa740009250 comp rx=0 tx=0).stop 2023-10-09T06:30:49.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.493+0000 7fa755cf0700 1 -- 172.21.15.163:0/2531632320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7500c9510 msgr2=0x7fa75016e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.493+0000 7fa755cf0700 1 --2- 172.21.15.163:0/2531632320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7500c9510 0x7fa75016e4a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa744016000 tx=0x7fa744017000 comp rx=0 tx=0).stop 2023-10-09T06:30:49.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.497+0000 7fa74ffff700 1 -- 172.21.15.163:0/2531632320 reap_dead start 2023-10-09T06:30:49.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.502+0000 7fa84206a700 1 -- 172.21.15.163:0/4197527724 wait complete. 2023-10-09T06:30:49.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.502+0000 7fa755cf0700 1 -- 172.21.15.163:0/2531632320 shutdown_connections 2023-10-09T06:30:49.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.502+0000 7fa755cf0700 1 -- 172.21.15.163:0/2531632320 >> 172.21.15.163:0/2531632320 conn(0x7fa7500ce600 msgr2=0x7fa7500cc150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:49.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.503+0000 7fa755cf0700 1 -- 172.21.15.163:0/2531632320 shutdown_connections 2023-10-09T06:30:49.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.505+0000 7fa755cf0700 1 -- 172.21.15.163:0/2531632320 wait complete. 2023-10-09T06:30:49.558 INFO:teuthology.orchestra.run.smithi163.stdout:55834574857 2023-10-09T06:30:49.559 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-09T06:30:49.559 INFO:teuthology.orchestra.run.smithi163.stdout:55834574855 2023-10-09T06:30:49.560 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-09T06:30:49.560 INFO:teuthology.orchestra.run.smithi163.stdout:55834574857 2023-10-09T06:30:49.561 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-09T06:30:49.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.510+0000 7fe33871d700 1 -- 172.21.15.163:0/177108980 --> [v2:172.21.15.181:6816/91132,v1:172.21.15.181:6817/91132] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe33004c3c0 con 0x7fe31406f190 2023-10-09T06:30:49.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.511+0000 7fe3267fc700 1 -- 172.21.15.163:0/177108980 <== osd.7 v2:172.21.15.181:6816/91132 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe33004c3c0 con 0x7fe31406f190 2023-10-09T06:30:49.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.511+0000 7fe33871d700 1 -- 172.21.15.163:0/177108980 >> [v2:172.21.15.181:6816/91132,v1:172.21.15.181:6817/91132] conn(0x7fe31406f190 msgr2=0x7fe3140715a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.511+0000 7fe33871d700 1 --2- 172.21.15.163:0/177108980 >> [v2:172.21.15.181:6816/91132,v1:172.21.15.181:6817/91132] conn(0x7fe31406f190 0x7fe3140715a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.511+0000 7fe33871d700 1 -- 172.21.15.163:0/177108980 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe314067000 msgr2=0x7fe3140694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.511+0000 7fe33871d700 1 --2- 172.21.15.163:0/177108980 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe314067000 0x7fe3140694b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe3200093f0 tx=0x7fe320017000 comp rx=0 tx=0).stop 2023-10-09T06:30:49.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.511+0000 7fe33871d700 1 -- 172.21.15.163:0/177108980 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3300699d0 msgr2=0x7fe3300c2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.511+0000 7fe33871d700 1 --2- 172.21.15.163:0/177108980 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3300699d0 0x7fe3300c2960 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe32c011e40 tx=0x7fe32c011af0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.511+0000 7fe336cba700 1 -- 172.21.15.163:0/177108980 reap_dead start 2023-10-09T06:30:49.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.512+0000 7fe33871d700 1 -- 172.21.15.163:0/177108980 shutdown_connections 2023-10-09T06:30:49.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.512+0000 7fe33871d700 1 -- 172.21.15.163:0/177108980 >> 172.21.15.163:0/177108980 conn(0x7fe330068390 msgr2=0x7fe3300c8140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:49.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.512+0000 7fe33871d700 1 -- 172.21.15.163:0/177108980 shutdown_connections 2023-10-09T06:30:49.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.512+0000 7fe33871d700 1 -- 172.21.15.163:0/177108980 wait complete. 2023-10-09T06:30:49.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.512+0000 7fa7f302d700 1 -- 172.21.15.163:0/1082161781 --> [v2:172.21.15.181:6808/91131,v1:172.21.15.181:6809/91131] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa7ec168f00 con 0x7fa7ec0a7dc0 2023-10-09T06:30:49.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.513+0000 7fa7e8ff9700 1 -- 172.21.15.163:0/1082161781 <== osd.6 v2:172.21.15.181:6808/91131 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa7ec168f00 con 0x7fa7ec0a7dc0 2023-10-09T06:30:49.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.513+0000 7fa7f302d700 1 -- 172.21.15.163:0/1082161781 >> [v2:172.21.15.181:6808/91131,v1:172.21.15.181:6809/91131] conn(0x7fa7ec0a7dc0 msgr2=0x7fa7ec0aad90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.513+0000 7fa7f302d700 1 --2- 172.21.15.163:0/1082161781 >> [v2:172.21.15.181:6808/91131,v1:172.21.15.181:6809/91131] conn(0x7fa7ec0a7dc0 0x7fa7ec0aad90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.513+0000 7fa7f302d700 1 -- 172.21.15.163:0/1082161781 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa7d8067080 msgr2=0x7fa7d8069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.513+0000 7fa7f302d700 1 --2- 172.21.15.163:0/1082161781 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa7d8067080 0x7fa7d8069530 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00099e0 tx=0x7fa7e0016000 comp rx=0 tx=0).stop 2023-10-09T06:30:49.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.513+0000 7fa7f302d700 1 -- 172.21.15.163:0/1082161781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7ec0cba20 msgr2=0x7fa7ec0a93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.513+0000 7fa7f302d700 1 --2- 172.21.15.163:0/1082161781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7ec0cba20 0x7fa7ec0a93a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa7e400cb40 tx=0x7fa7e40027f0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.513+0000 7fa7f15ca700 1 -- 172.21.15.163:0/1082161781 reap_dead start 2023-10-09T06:30:49.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.513+0000 7fa7f302d700 1 -- 172.21.15.163:0/1082161781 shutdown_connections 2023-10-09T06:30:49.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.513+0000 7fa7f302d700 1 -- 172.21.15.163:0/1082161781 >> 172.21.15.163:0/1082161781 conn(0x7fa7ec0cd000 msgr2=0x7fa7ec153350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:49.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.514+0000 7fa7f302d700 1 -- 172.21.15.163:0/1082161781 shutdown_connections 2023-10-09T06:30:49.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.514+0000 7fa7f302d700 1 -- 172.21.15.163:0/1082161781 wait complete. 2023-10-09T06:30:49.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.514+0000 7f31a58e0700 1 -- 172.21.15.163:0/4073745895 --> [v2:172.21.15.181:6800/91130,v1:172.21.15.181:6801/91130] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f31a00acfd0 con 0x7f318c06f290 2023-10-09T06:30:49.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.515+0000 7f3186ffd700 1 -- 172.21.15.163:0/4073745895 <== osd.5 v2:172.21.15.181:6800/91130 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f31a00acfd0 con 0x7f318c06f290 2023-10-09T06:30:49.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.515+0000 7f31a58e0700 1 -- 172.21.15.163:0/4073745895 >> [v2:172.21.15.181:6800/91130,v1:172.21.15.181:6801/91130] conn(0x7f318c06f290 msgr2=0x7f318c0716a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.515+0000 7f31a58e0700 1 --2- 172.21.15.163:0/4073745895 >> [v2:172.21.15.181:6800/91130,v1:172.21.15.181:6801/91130] conn(0x7f318c06f290 0x7f318c0716a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.515+0000 7f31a58e0700 1 -- 172.21.15.163:0/4073745895 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f318c067080 msgr2=0x7f318c069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.515+0000 7f31a58e0700 1 --2- 172.21.15.163:0/4073745895 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f318c067080 0x7f318c069530 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f31a0172870 tx=0x7f319801a000 comp rx=0 tx=0).stop 2023-10-09T06:30:49.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.515+0000 7f31a58e0700 1 -- 172.21.15.163:0/4073745895 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31a0068aa0 msgr2=0x7f31a0172c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.515+0000 7f31a58e0700 1 --2- 172.21.15.163:0/4073745895 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31a0068aa0 0x7f31a0172c00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f319000c5b0 tx=0x7f3190001070 comp rx=0 tx=0).stop 2023-10-09T06:30:49.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.515+0000 7f319f7fe700 1 -- 172.21.15.163:0/4073745895 reap_dead start 2023-10-09T06:30:49.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.515+0000 7f31a58e0700 1 -- 172.21.15.163:0/4073745895 shutdown_connections 2023-10-09T06:30:49.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.515+0000 7f31a58e0700 1 -- 172.21.15.163:0/4073745895 >> 172.21.15.163:0/4073745895 conn(0x7f31a00cd000 msgr2=0x7f31a0165240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:49.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.517+0000 7f31a58e0700 1 -- 172.21.15.163:0/4073745895 shutdown_connections 2023-10-09T06:30:49.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.517+0000 7f31a58e0700 1 -- 172.21.15.163:0/4073745895 wait complete. 2023-10-09T06:30:49.568 INFO:teuthology.orchestra.run.smithi163.stdout:55834574856 2023-10-09T06:30:49.569 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-09T06:30:49.570 INFO:teuthology.orchestra.run.smithi163.stdout:55834574855 2023-10-09T06:30:49.570 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-09T06:30:49.571 INFO:teuthology.orchestra.run.smithi163.stdout:55834574856 2023-10-09T06:30:49.571 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-09T06:30:49.572 INFO:teuthology.orchestra.run.smithi163.stdout:55834574854 2023-10-09T06:30:49.572 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-09T06:30:49.573 INFO:teuthology.orchestra.run.smithi163.stdout:55834574855 2023-10-09T06:30:49.573 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-09T06:30:49.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.792+0000 7f6edd0ad700 1 Processor -- start 2023-10-09T06:30:49.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.792+0000 7f6edd0ad700 1 -- start start 2023-10-09T06:30:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.794+0000 7f6edd0ad700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ed80b1c50 0x7f6ed80ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.794+0000 7f6edd0ad700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6ed80875d0 0x7f6ed8087a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.794+0000 7f6edd0ad700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ed80acdf0 con 0x7f6ed8068aa0 2023-10-09T06:30:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.794+0000 7f6edd0ad700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ed80ac4c0 con 0x7f6ed80875d0 2023-10-09T06:30:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.794+0000 7f6edd0ad700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed80ac7a0 con 0x7f6ed80b1c50 2023-10-09T06:30:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.794+0000 7f6ed6d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ed80b1c50 0x7f6ed80ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.794+0000 7f6ed6d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ed80b1c50 0x7f6ed80ab8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:44698/0 (socket says 172.21.15.163:44698) 2023-10-09T06:30:49.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.794+0000 7f6ed6d9d700 1 -- 172.21.15.163:0/3698195623 learned_addr learned my addr 172.21.15.163:0/3698195623 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:49.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.795+0000 7f3508b38700 1 Processor -- start 2023-10-09T06:30:49.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.795+0000 7f3508b38700 1 -- start start 2023-10-09T06:30:49.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.795+0000 7f6ed6d9d700 1 -- 172.21.15.163:0/3698195623 >> v2:172.21.15.163:3300/0 conn(0x7f6ed80875d0 msgr2=0x7f6ed8087a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:49.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.795+0000 7f6ed6d9d700 1 --2- 172.21.15.163:0/3698195623 >> v2:172.21.15.163:3300/0 conn(0x7f6ed80875d0 0x7f6ed8087a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.795+0000 7f6ed6d9d700 1 -- 172.21.15.163:0/3698195623 >> v1:172.21.15.181:6789/0 conn(0x7f6ed8068aa0 legacy=0x7f6ed816a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.795+0000 7f6ed6d9d700 1 -- 172.21.15.163:0/3698195623 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed80aca80 con 0x7f6ed80b1c50 2023-10-09T06:30:49.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.795+0000 7f6ed6d9d700 1 --2- 172.21.15.163:0/3698195623 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ed80b1c50 0x7f6ed80ab8c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80099b0 tx=0x7f6ec8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69c3c545c08b4b52 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.796+0000 7f6ed5d9b700 1 -- 172.21.15.163:0/3698195623 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec800b8f0 con 0x7f6ed80b1c50 2023-10-09T06:30:49.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.796+0000 7f3508b38700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f35040691b0 0x7f35040ac970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.796+0000 7f3508b38700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3504156050 0x7f35041564b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.796+0000 7f3508b38700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35040b2490 con 0x7f35040cba20 2023-10-09T06:30:49.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.796+0000 7f3508b38700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35040aceb0 con 0x7f35040691b0 2023-10-09T06:30:49.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.796+0000 7f3508b38700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35040ac4e0 con 0x7f3504156050 2023-10-09T06:30:49.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.796+0000 7f350259c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f35040cba20 0x7f3504068aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60854/0 (socket says 172.21.15.163:60854) 2023-10-09T06:30:49.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.796+0000 7f350259c700 1 -- 172.21.15.163:0/2707567934 learned_addr learned my addr 172.21.15.163:0/2707567934 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:49.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.796+0000 7f3501d9b700 1 --2- 172.21.15.163:0/2707567934 >> v2:172.21.15.181:3300/0 conn(0x7f35040691b0 0x7f35040ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.797+0000 7f3501d9b700 1 -- 172.21.15.163:0/2707567934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3504156050 msgr2=0x7f35041564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:49.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.797+0000 7f3501d9b700 1 --2- 172.21.15.163:0/2707567934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3504156050 0x7f35041564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.797+0000 7f3501d9b700 1 -- 172.21.15.163:0/2707567934 >> v1:172.21.15.163:6789/0 conn(0x7f35040cba20 legacy=0x7f3504068aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.797+0000 7f3501d9b700 1 -- 172.21.15.163:0/2707567934 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3504156cd0 con 0x7f35040691b0 2023-10-09T06:30:49.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.796+0000 7f6ed5d9b700 1 -- 172.21.15.163:0/3698195623 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ec800ba50 con 0x7f6ed80b1c50 2023-10-09T06:30:49.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.796+0000 7f6ed5d9b700 1 -- 172.21.15.163:0/3698195623 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec800bdf0 con 0x7f6ed80b1c50 2023-10-09T06:30:49.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.797+0000 7f6edd0ad700 1 -- 172.21.15.163:0/3698195623 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ed80b1c50 msgr2=0x7f6ed80ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.797+0000 7f6edd0ad700 1 --2- 172.21.15.163:0/3698195623 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ed80b1c50 0x7f6ed80ab8c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80099b0 tx=0x7f6ec8009250 comp rx=0 tx=0).stop 2023-10-09T06:30:49.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.797+0000 7f6edd0ad700 1 -- 172.21.15.163:0/3698195623 shutdown_connections 2023-10-09T06:30:49.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.797+0000 7f6edd0ad700 1 --2- 172.21.15.163:0/3698195623 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ed80b1c50 0x7f6ed80ab8c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.797+0000 7f6edd0ad700 1 --2- 172.21.15.163:0/3698195623 >> v2:172.21.15.163:3300/0 conn(0x7f6ed80875d0 0x7f6ed8087a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.797+0000 7f6edd0ad700 1 -- 172.21.15.163:0/3698195623 >> 172.21.15.163:0/3698195623 conn(0x7f6ed80cd000 msgr2=0x7f6ed8068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:49.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.798+0000 7f7d86f0f700 1 Processor -- start 2023-10-09T06:30:49.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.798+0000 7f3501d9b700 1 --2- 172.21.15.163:0/2707567934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35040691b0 0x7f35040ac970 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f34f40088f0 tx=0x7f34f4008210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48b6860aeb5bb15d server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.798+0000 7f350159a700 1 -- 172.21.15.163:0/2707567934 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f4010070 con 0x7f35040691b0 2023-10-09T06:30:49.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.798+0000 7f350159a700 1 -- 172.21.15.163:0/2707567934 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f400d980 con 0x7f35040691b0 2023-10-09T06:30:49.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.798+0000 7f350159a700 1 -- 172.21.15.163:0/2707567934 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f400dd20 con 0x7f35040691b0 2023-10-09T06:30:49.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.798+0000 7f69459dc700 1 Processor -- start 2023-10-09T06:30:49.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.798+0000 7f6edd0ad700 1 -- 172.21.15.163:0/3698195623 shutdown_connections 2023-10-09T06:30:49.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.798+0000 7f69459dc700 1 -- start start 2023-10-09T06:30:49.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.799+0000 7f69459dc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f69400afc40 0x7f69400b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.799+0000 7f69459dc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f69400b2140 0x7f694015d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.799+0000 7f69459dc700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69400b0750 con 0x7f69400b0010 2023-10-09T06:30:49.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.799+0000 7f69459dc700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69400aca70 con 0x7f69400b2140 2023-10-09T06:30:49.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.799+0000 7f69459dc700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69400acd50 con 0x7f69400afc40 2023-10-09T06:30:49.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.798+0000 7f1ea248c700 1 Processor -- start 2023-10-09T06:30:49.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.798+0000 7f1ea248c700 1 -- start start 2023-10-09T06:30:49.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.799+0000 7f1ea248c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e9c0691b0 0x7f1e9c0b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.799+0000 7f1ea248c700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e9c15f260 con 0x7f1e9c158440 2023-10-09T06:30:49.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.799+0000 7f1ea248c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e9c0b2460 con 0x7f1e9c0b1fe0 2023-10-09T06:30:49.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.799+0000 7f1ea248c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c0aef20 con 0x7f1e9c0691b0 2023-10-09T06:30:49.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.799+0000 7f693effd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f69400afc40 0x7f69400b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.799+0000 7f693e7fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f69400b2140 0x7f694015d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.799+0000 7f693effd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f69400afc40 0x7f69400b1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:50390/0 (socket says 172.21.15.163:50390) 2023-10-09T06:30:49.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.799+0000 7f693e7fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f69400b2140 0x7f694015d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43834/0 (socket says 172.21.15.163:43834) 2023-10-09T06:30:49.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.799+0000 7f693effd700 1 -- 172.21.15.163:0/3592054919 learned_addr learned my addr 172.21.15.163:0/3592054919 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:49.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.800+0000 7f1ea0a29700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f1e9c0b1fe0 0x7f1e9c0ab8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:47828/0 (socket says 172.21.15.163:47828) 2023-10-09T06:30:49.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.800+0000 7f1ea0a29700 1 -- 172.21.15.163:0/694023123 learned_addr learned my addr 172.21.15.163:0/694023123 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:49.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.798+0000 7f6edd0ad700 1 -- 172.21.15.163:0/3698195623 wait complete. 2023-10-09T06:30:49.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.800+0000 7f693e7fc700 1 -- 172.21.15.163:0/3592054919 >> v2:172.21.15.181:3300/0 conn(0x7f69400afc40 msgr2=0x7f69400b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:49.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.800+0000 7f693e7fc700 1 --2- 172.21.15.163:0/3592054919 >> v2:172.21.15.181:3300/0 conn(0x7f69400afc40 0x7f69400b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.800+0000 7f693e7fc700 1 -- 172.21.15.163:0/3592054919 >> v1:172.21.15.163:6789/0 conn(0x7f69400b0010 legacy=0x7f694015dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:49.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.800+0000 7f693e7fc700 1 -- 172.21.15.163:0/3592054919 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69400ad030 con 0x7f69400b2140 2023-10-09T06:30:49.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.800+0000 7f1e9b7fe700 1 --2- 172.21.15.163:0/694023123 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e9c0691b0 0x7f1e9c0b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.800+0000 7f693e7fc700 1 --2- 172.21.15.163:0/3592054919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69400b2140 0x7f694015d850 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6940063d40 tx=0x7f69340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3845faea13cb945 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.800+0000 7f1e9b7fe700 1 -- 172.21.15.163:0/694023123 >> v1:172.21.15.181:6789/0 conn(0x7f1e9c0b1fe0 legacy=0x7f1e9c0ab8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.800+0000 7f1e9b7fe700 1 -- 172.21.15.163:0/694023123 >> v1:172.21.15.163:6789/0 conn(0x7f1e9c158440 legacy=0x7f1e9c068aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-09T06:30:49.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.800+0000 7f1e9affd700 1 -- 172.21.15.163:0/694023123 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1079186933 0 0) 0x7f1e9c0b2460 con 0x7f1e9c0b1fe0 2023-10-09T06:30:49.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.800+0000 7f1e9b7fe700 1 -- 172.21.15.163:0/694023123 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e9c15e380 con 0x7f1e9c0691b0 2023-10-09T06:30:49.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.800+0000 7f3508b38700 1 -- 172.21.15.163:0/2707567934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35040691b0 msgr2=0x7f35040ac970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.800+0000 7f3508b38700 1 --2- 172.21.15.163:0/2707567934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35040691b0 0x7f35040ac970 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f34f40088f0 tx=0x7f34f4008210 comp rx=0 tx=0).stop 2023-10-09T06:30:49.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.798+0000 7f7d86f0f700 1 -- start start 2023-10-09T06:30:49.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.799+0000 7f7d86f0f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7d800b0010 0x7f7d8015d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.799+0000 7f7d86f0f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7d8015dd90 0x7f7d800adb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.799+0000 7f7d86f0f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d800b0750 con 0x7f7d800afc40 2023-10-09T06:30:49.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.799+0000 7f7d86f0f700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d800aca70 con 0x7f7d8015dd90 2023-10-09T06:30:49.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.799+0000 7f7d86f0f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d800acd50 con 0x7f7d800b0010 2023-10-09T06:30:49.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.800+0000 7f6edd0ad700 1 Processor -- start 2023-10-09T06:30:49.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.800+0000 7f693dffb700 1 -- 172.21.15.163:0/3592054919 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6934011630 con 0x7f69400b2140 2023-10-09T06:30:49.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.800+0000 7f693dffb700 1 -- 172.21.15.163:0/3592054919 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6934011790 con 0x7f69400b2140 2023-10-09T06:30:49.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.800+0000 7f693dffb700 1 -- 172.21.15.163:0/3592054919 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6934011b30 con 0x7f69400b2140 2023-10-09T06:30:49.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.800+0000 7f1e9b7fe700 1 --2- 172.21.15.163:0/694023123 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e9c0691b0 0x7f1e9c0b1aa0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1e90009910 tx=0x7f1e900091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdeb87320c470e2a server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.800+0000 7f6edd0ad700 1 -- start start 2023-10-09T06:30:49.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.800+0000 7f7d7ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7d800b0010 0x7f7d8015d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.800+0000 7f7d7ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7d800b0010 0x7f7d8015d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:44712/0 (socket says 172.21.15.163:44712) 2023-10-09T06:30:49.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.800+0000 7f7d7ffff700 1 -- 172.21.15.163:0/1502194700 learned_addr learned my addr 172.21.15.163:0/1502194700 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:49.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.801+0000 7f7d854ac700 1 --2- 172.21.15.163:0/1502194700 >> v2:172.21.15.163:3300/0 conn(0x7f7d8015dd90 0x7f7d800adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.801+0000 7f1e9affd700 1 -- 172.21.15.163:0/694023123 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9000b820 con 0x7f1e9c0691b0 2023-10-09T06:30:49.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.801+0000 7f1e9affd700 1 -- 172.21.15.163:0/694023123 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9000b980 con 0x7f1e9c0691b0 2023-10-09T06:30:49.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.801+0000 7f1e9affd700 1 -- 172.21.15.163:0/694023123 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9000bd20 con 0x7f1e9c0691b0 2023-10-09T06:30:49.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.801+0000 7f7d7ffff700 1 -- 172.21.15.163:0/1502194700 >> v2:172.21.15.163:3300/0 conn(0x7f7d8015dd90 msgr2=0x7f7d800adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:49.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.801+0000 7f7d7ffff700 1 --2- 172.21.15.163:0/1502194700 >> v2:172.21.15.163:3300/0 conn(0x7f7d8015dd90 0x7f7d800adb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.801+0000 7f7d7ffff700 1 -- 172.21.15.163:0/1502194700 >> v1:172.21.15.163:6789/0 conn(0x7f7d800afc40 legacy=0x7f7d800b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:49.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.801+0000 7f7d7ffff700 1 -- 172.21.15.163:0/1502194700 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d800ad030 con 0x7f7d800b0010 2023-10-09T06:30:49.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.801+0000 7f3508b38700 1 -- 172.21.15.163:0/2707567934 shutdown_connections 2023-10-09T06:30:49.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.801+0000 7f3508b38700 1 --2- 172.21.15.163:0/2707567934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3504156050 0x7f35041564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.801+0000 7f3508b38700 1 --2- 172.21.15.163:0/2707567934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35040691b0 0x7f35040ac970 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.801+0000 7f3508b38700 1 -- 172.21.15.163:0/2707567934 >> 172.21.15.163:0/2707567934 conn(0x7f35040cd000 msgr2=0x7f3504068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:49.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.801+0000 7f7d7ffff700 1 --2- 172.21.15.163:0/1502194700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7d800b0010 0x7f7d8015d850 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009f20 tx=0x7f7d78009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c95392b9f6676e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.801+0000 7f635e6eb700 1 Processor -- start 2023-10-09T06:30:49.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.801+0000 7f3508b38700 1 -- 172.21.15.163:0/2707567934 shutdown_connections 2023-10-09T06:30:49.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.801+0000 7f69459dc700 1 -- 172.21.15.163:0/3592054919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69400b2140 msgr2=0x7f694015d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.801+0000 7f69459dc700 1 --2- 172.21.15.163:0/3592054919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69400b2140 0x7f694015d850 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6940063d40 tx=0x7f69340091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.814 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.801+0000 7f635e6eb700 1 -- start start 2023-10-09T06:30:49.814 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.801+0000 7f6edd0ad700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ed8068aa0 0x7f6ed815cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.814 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.801+0000 7f6edd0ad700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ed80875d0 0x7f6ed815e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.814 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.801+0000 7f6edd0ad700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ed80b1c50 0x7f6ed81546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.814 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.801+0000 7f6edd0ad700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed8086990 con 0x7f6ed80875d0 2023-10-09T06:30:49.814 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.801+0000 7f6edd0ad700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed815d540 con 0x7f6ed8068aa0 2023-10-09T06:30:49.815 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.801+0000 7f6edd0ad700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed8157fb0 con 0x7f6ed80b1c50 2023-10-09T06:30:49.815 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.802+0000 7f6ed659c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ed80875d0 0x7f6ed815e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.815 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.802+0000 7f1ea248c700 1 -- 172.21.15.163:0/694023123 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e9c0691b0 msgr2=0x7f1e9c0b1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.815 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.802+0000 7f1ea248c700 1 --2- 172.21.15.163:0/694023123 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e9c0691b0 0x7f1e9c0b1aa0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1e90009910 tx=0x7f1e900091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.815 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.802+0000 7f6ed659c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ed80875d0 0x7f6ed815e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43842/0 (socket says 172.21.15.163:43842) 2023-10-09T06:30:49.815 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.802+0000 7f6ed659c700 1 -- 172.21.15.163:0/4274266991 learned_addr learned my addr 172.21.15.163:0/4274266991 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:49.816 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.802+0000 7f1ea248c700 1 -- 172.21.15.163:0/694023123 shutdown_connections 2023-10-09T06:30:49.816 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.802+0000 7f1ea248c700 1 --2- 172.21.15.163:0/694023123 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e9c0691b0 0x7f1e9c0b1aa0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.816 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.802+0000 7f1ea248c700 1 -- 172.21.15.163:0/694023123 >> 172.21.15.163:0/694023123 conn(0x7f1e9c0cd000 msgr2=0x7f1e9c068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:49.816 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.802+0000 7f6ed759e700 1 --2- 172.21.15.163:0/4274266991 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ed80b1c50 0x7f6ed81546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.816 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.802+0000 7f1ea248c700 1 -- 172.21.15.163:0/694023123 shutdown_connections 2023-10-09T06:30:49.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.802+0000 7f7d7f7fe700 1 -- 172.21.15.163:0/1502194700 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7800e070 con 0x7f7d800b0010 2023-10-09T06:30:49.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.802+0000 7f7d7f7fe700 1 -- 172.21.15.163:0/1502194700 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d7800b990 con 0x7f7d800b0010 2023-10-09T06:30:49.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.802+0000 7f7d7f7fe700 1 -- 172.21.15.163:0/1502194700 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d780092e0 con 0x7f7d800b0010 2023-10-09T06:30:49.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.802+0000 7f6ed659c700 1 -- 172.21.15.163:0/4274266991 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ed80b1c50 msgr2=0x7f6ed81546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.802+0000 7f6ed659c700 1 --2- 172.21.15.163:0/4274266991 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ed80b1c50 0x7f6ed81546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.802+0000 7f6ed659c700 1 -- 172.21.15.163:0/4274266991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ed8068aa0 msgr2=0x7f6ed815cc60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-09T06:30:49.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.802+0000 7f6ed659c700 1 --2- 172.21.15.163:0/4274266991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ed8068aa0 0x7f6ed815cc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.802+0000 7f6ed659c700 1 -- 172.21.15.163:0/4274266991 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec80094f0 con 0x7f6ed80875d0 2023-10-09T06:30:49.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.802+0000 7f69459dc700 1 -- 172.21.15.163:0/3592054919 shutdown_connections 2023-10-09T06:30:49.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.802+0000 7f69459dc700 1 --2- 172.21.15.163:0/3592054919 >> v2:172.21.15.181:3300/0 conn(0x7f69400afc40 0x7f69400b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.802+0000 7f69459dc700 1 --2- 172.21.15.163:0/3592054919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69400b2140 0x7f694015d850 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.819 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.802+0000 7f69459dc700 1 -- 172.21.15.163:0/3592054919 >> 172.21.15.163:0/3592054919 conn(0x7f694006a320 msgr2=0x7f69400cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:49.819 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.802+0000 7f69459dc700 1 -- 172.21.15.163:0/3592054919 shutdown_connections 2023-10-09T06:30:49.819 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.802+0000 7f6ed659c700 1 --2- 172.21.15.163:0/4274266991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ed80875d0 0x7f6ed815e7f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000bc30 tx=0x7f6ed0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.819 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.803+0000 7f635e6eb700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f63580691b0 0x7f63580ac7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.803+0000 7f635e6eb700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63580accf0 con 0x7f63580b1f40 2023-10-09T06:30:49.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.803+0000 7f635e6eb700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f635815a440 con 0x7f6358159fb0 2023-10-09T06:30:49.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.803+0000 7f635e6eb700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f635815a720 con 0x7f63580691b0 2023-10-09T06:30:49.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.803+0000 7f63577fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6358159fb0 0x7f6358068aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:47832/0 (socket says 172.21.15.163:47832) 2023-10-09T06:30:49.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.803+0000 7f63577fe700 1 -- 172.21.15.163:0/3375902368 learned_addr learned my addr 172.21.15.163:0/3375902368 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:49.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.801+0000 7f3508b38700 1 -- 172.21.15.163:0/2707567934 wait complete. 2023-10-09T06:30:49.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.803+0000 7f3508b38700 1 Processor -- start 2023-10-09T06:30:49.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.803+0000 7f6ebeffd700 1 -- 172.21.15.163:0/4274266991 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed00092e0 con 0x7f6ed80875d0 2023-10-09T06:30:49.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.803+0000 7f6edd0ad700 1 -- 172.21.15.163:0/4274266991 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ed8158290 con 0x7f6ed80875d0 2023-10-09T06:30:49.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.803+0000 7f6edd0ad700 1 -- 172.21.15.163:0/4274266991 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ed8157c60 con 0x7f6ed80875d0 2023-10-09T06:30:49.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.803+0000 7f6ebeffd700 1 -- 172.21.15.163:0/4274266991 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed000c6f0 con 0x7f6ed80875d0 2023-10-09T06:30:49.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.803+0000 7f6ebeffd700 1 -- 172.21.15.163:0/4274266991 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed000cad0 con 0x7f6ed80875d0 2023-10-09T06:30:49.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.803+0000 7f7d86f0f700 1 -- 172.21.15.163:0/1502194700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7d800b0010 msgr2=0x7f7d8015d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.803+0000 7f7d86f0f700 1 --2- 172.21.15.163:0/1502194700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7d800b0010 0x7f7d8015d850 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009f20 tx=0x7f7d78009840 comp rx=0 tx=0).stop 2023-10-09T06:30:49.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.802+0000 7f1ea248c700 1 -- 172.21.15.163:0/694023123 wait complete. 2023-10-09T06:30:49.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.803+0000 7f1ea248c700 1 Processor -- start 2023-10-09T06:30:49.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.803+0000 7f69459dc700 1 -- 172.21.15.163:0/3592054919 wait complete. 2023-10-09T06:30:49.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.804+0000 7f1ea248c700 1 -- start start 2023-10-09T06:30:49.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.804+0000 7f6356ffd700 1 -- 172.21.15.163:0/3375902368 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 113658027 0 0) 0x7f635815a440 con 0x7f6358159fb0 2023-10-09T06:30:49.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.804+0000 7f6356ffd700 1 -- 172.21.15.163:0/3375902368 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6340003570 con 0x7f6358159fb0 2023-10-09T06:30:49.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.804+0000 7f69459dc700 1 Processor -- start 2023-10-09T06:30:49.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.804+0000 7f635cc88700 1 --2- 172.21.15.163:0/3375902368 >> v2:172.21.15.163:3300/0 conn(0x7f63580691b0 0x7f63580ac7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.804+0000 7f3508b38700 1 -- start start 2023-10-09T06:30:49.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.804+0000 7f7d86f0f700 1 -- 172.21.15.163:0/1502194700 shutdown_connections 2023-10-09T06:30:49.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.804+0000 7f7d86f0f700 1 --2- 172.21.15.163:0/1502194700 >> v2:172.21.15.163:3300/0 conn(0x7f7d8015dd90 0x7f7d800adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.804+0000 7f7d86f0f700 1 --2- 172.21.15.163:0/1502194700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7d800b0010 0x7f7d8015d850 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.804+0000 7f7d86f0f700 1 -- 172.21.15.163:0/1502194700 >> 172.21.15.163:0/1502194700 conn(0x7f7d8006a320 msgr2=0x7f7d800cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:49.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.804+0000 7f635cc88700 1 -- 172.21.15.163:0/3375902368 >> v1:172.21.15.181:6789/0 conn(0x7f6358159fb0 legacy=0x7f6358068aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.804+0000 7f635cc88700 1 -- 172.21.15.163:0/3375902368 >> v1:172.21.15.163:6789/0 conn(0x7f63580b1f40 legacy=0x7f63580aabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.804+0000 7f635cc88700 1 -- 172.21.15.163:0/3375902368 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f635815aa00 con 0x7f63580691b0 2023-10-09T06:30:49.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.805+0000 7f635cc88700 1 --2- 172.21.15.163:0/3375902368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63580691b0 0x7f63580ac7b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6350009e60 tx=0x7f6350009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a2cf08b103e90c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.805+0000 7f1ea248c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e9c0691b0 0x7f1e9c161e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.805+0000 7f1ea248c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e9c0b1fe0 0x7f1e9c1347f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.805+0000 7f1ea248c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e9c158440 0x7f1e9c15abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.805+0000 7f1ea248c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c133500 con 0x7f1e9c0691b0 2023-10-09T06:30:49.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.805+0000 7f1ea248c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c134360 con 0x7f1e9c158440 2023-10-09T06:30:49.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.805+0000 7f1ea248c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c133750 con 0x7f1e9c0b1fe0 2023-10-09T06:30:49.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.805+0000 7f6356ffd700 1 -- 172.21.15.163:0/3375902368 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635000e070 con 0x7f63580691b0 2023-10-09T06:30:49.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.805+0000 7f6356ffd700 1 -- 172.21.15.163:0/3375902368 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635000bb20 con 0x7f63580691b0 2023-10-09T06:30:49.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.805+0000 7f6356ffd700 1 -- 172.21.15.163:0/3375902368 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6350012430 con 0x7f63580691b0 2023-10-09T06:30:49.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.804+0000 7f69459dc700 1 -- start start 2023-10-09T06:30:49.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.805+0000 7f69459dc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69400afc40 0x7f694013cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.805+0000 7f69459dc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69400b0010 0x7f6940156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.805+0000 7f69459dc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69400b2140 0x7f6940156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.805+0000 7f3508b38700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35040691b0 0x7f35040a8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.805+0000 7f3508b38700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35040cba20 0x7f3504152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.805+0000 7f3508b38700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3504156050 0x7f35040a7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.805+0000 7f3508b38700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35040a6440 con 0x7f35040cba20 2023-10-09T06:30:49.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.805+0000 7f3508b38700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35040a9310 con 0x7f35040691b0 2023-10-09T06:30:49.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.805+0000 7f69459dc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6940151080 con 0x7f69400b2140 2023-10-09T06:30:49.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.805+0000 7f69459dc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6940156c70 con 0x7f69400b0010 2023-10-09T06:30:49.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.805+0000 7f69459dc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6940156f50 con 0x7f69400afc40 2023-10-09T06:30:49.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.805+0000 7f693effd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69400afc40 0x7f694013cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.805+0000 7f3508b38700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35041534f0 con 0x7f3504156050 2023-10-09T06:30:49.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.805+0000 7f6ebeffd700 1 -- 172.21.15.163:0/4274266991 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ed0012070 con 0x7f6ed80875d0 2023-10-09T06:30:49.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.805+0000 7f693effd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69400afc40 0x7f694013cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:44730/0 (socket says 172.21.15.163:44730) 2023-10-09T06:30:49.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.805+0000 7f693effd700 1 -- 172.21.15.163:0/266313166 learned_addr learned my addr 172.21.15.163:0/266313166 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:49.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7f1ea0a29700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e9c158440 0x7f1e9c15abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7f1ea0a29700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e9c158440 0x7f1e9c15abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:50402/0 (socket says 172.21.15.163:50402) 2023-10-09T06:30:49.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7f1ea0a29700 1 -- 172.21.15.163:0/1427583306 learned_addr learned my addr 172.21.15.163:0/1427583306 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:49.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7f1e9bfff700 1 --2- 172.21.15.163:0/1427583306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e9c0691b0 0x7f1e9c161e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7f1e9b7fe700 1 --2- 172.21.15.163:0/1427583306 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e9c0b1fe0 0x7f1e9c1347f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.807+0000 7f1e9b7fe700 1 -- 172.21.15.163:0/1427583306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e9c158440 msgr2=0x7f1e9c15abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.807+0000 7f1e9b7fe700 1 --2- 172.21.15.163:0/1427583306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e9c158440 0x7f1e9c15abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.807+0000 7f1e9b7fe700 1 -- 172.21.15.163:0/1427583306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e9c0691b0 msgr2=0x7f1e9c161e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.807+0000 7f1e9b7fe700 1 --2- 172.21.15.163:0/1427583306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e9c0691b0 0x7f1e9c161e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.807+0000 7f1e9b7fe700 1 -- 172.21.15.163:0/1427583306 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e90009420 con 0x7f1e9c0b1fe0 2023-10-09T06:30:49.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.807+0000 7f1e9b7fe700 1 --2- 172.21.15.163:0/1427583306 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e9c0b1fe0 0x7f1e9c1347f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1e90015000 tx=0x7f1e90016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.807+0000 7f1e83fff700 1 -- 172.21.15.163:0/1427583306 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e90011d20 con 0x7f1e9c0b1fe0 2023-10-09T06:30:49.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.807+0000 7f1e83fff700 1 -- 172.21.15.163:0/1427583306 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e90020070 con 0x7f1e9c0b1fe0 2023-10-09T06:30:49.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.807+0000 7f1e83fff700 1 -- 172.21.15.163:0/1427583306 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9001b6a0 con 0x7f1e9c0b1fe0 2023-10-09T06:30:49.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.807+0000 7f1ea248c700 1 -- 172.21.15.163:0/1427583306 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e9c133a30 con 0x7f1e9c0b1fe0 2023-10-09T06:30:49.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.807+0000 7f1ea248c700 1 -- 172.21.15.163:0/1427583306 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e9c133ff0 con 0x7f1e9c0b1fe0 2023-10-09T06:30:49.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7f635e6eb700 1 -- 172.21.15.163:0/3375902368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63580691b0 msgr2=0x7f63580ac7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7f635e6eb700 1 --2- 172.21.15.163:0/3375902368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63580691b0 0x7f63580ac7b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6350009e60 tx=0x7f6350009780 comp rx=0 tx=0).stop 2023-10-09T06:30:49.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7f635e6eb700 1 -- 172.21.15.163:0/3375902368 shutdown_connections 2023-10-09T06:30:49.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7f635e6eb700 1 --2- 172.21.15.163:0/3375902368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63580691b0 0x7f63580ac7b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7f635e6eb700 1 -- 172.21.15.163:0/3375902368 >> 172.21.15.163:0/3375902368 conn(0x7f63580cd000 msgr2=0x7f6358068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:49.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.807+0000 7f635e6eb700 1 -- 172.21.15.163:0/3375902368 shutdown_connections 2023-10-09T06:30:49.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.805+0000 7f350259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35040691b0 0x7f35040a8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.805+0000 7f350259c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35040691b0 0x7f35040a8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:50426/0 (socket says 172.21.15.163:50426) 2023-10-09T06:30:49.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7f350259c700 1 -- 172.21.15.163:0/1816284136 learned_addr learned my addr 172.21.15.163:0/1816284136 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:49.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7f3501d9b700 1 --2- 172.21.15.163:0/1816284136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35040cba20 0x7f3504152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7f3501d9b700 1 -- 172.21.15.163:0/1816284136 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3504156050 msgr2=0x7f35040a7080 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-09T06:30:49.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7f3501d9b700 1 --2- 172.21.15.163:0/1816284136 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3504156050 0x7f35040a7080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7f3501d9b700 1 -- 172.21.15.163:0/1816284136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35040691b0 msgr2=0x7f35040a8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7f3501d9b700 1 --2- 172.21.15.163:0/1816284136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35040691b0 0x7f35040a8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7f3501d9b700 1 -- 172.21.15.163:0/1816284136 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f40084b0 con 0x7f35040cba20 2023-10-09T06:30:49.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7f3501d9b700 1 --2- 172.21.15.163:0/1816284136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35040cba20 0x7f3504152360 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f34f4018000 tx=0x7f34f401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.807+0000 7f34f27fc700 1 -- 172.21.15.163:0/1816284136 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f4016af0 con 0x7f35040cba20 2023-10-09T06:30:49.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.807+0000 7f34f27fc700 1 -- 172.21.15.163:0/1816284136 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f4016c70 con 0x7f35040cba20 2023-10-09T06:30:49.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.807+0000 7f34f27fc700 1 -- 172.21.15.163:0/1816284136 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f400b0e0 con 0x7f35040cba20 2023-10-09T06:30:49.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7f6ebeffd700 1 --2- 172.21.15.163:0/4274266991 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6ec4067000 0x7f6ec40694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7f6ebeffd700 1 -- 172.21.15.163:0/4274266991 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6ed0084050 con 0x7f6ed80875d0 2023-10-09T06:30:49.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7f6ed6d9d700 1 --2- 172.21.15.163:0/4274266991 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6ec4067000 0x7f6ec40694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.807+0000 7f6edd0ad700 1 -- 172.21.15.163:0/4274266991 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eb80052b0 con 0x7f6ed80875d0 2023-10-09T06:30:49.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.805+0000 7f693f7fe700 1 --2- 172.21.15.163:0/266313166 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69400b2140 0x7f6940156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7f693effd700 1 -- 172.21.15.163:0/266313166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69400b0010 msgr2=0x7f6940156140 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-09T06:30:49.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7f693effd700 1 --2- 172.21.15.163:0/266313166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69400b0010 0x7f6940156140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7f693effd700 1 -- 172.21.15.163:0/266313166 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69400b2140 msgr2=0x7f6940156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7f693effd700 1 --2- 172.21.15.163:0/266313166 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69400b2140 0x7f6940156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.835 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7f693effd700 1 -- 172.21.15.163:0/266313166 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6934009420 con 0x7f69400afc40 2023-10-09T06:30:49.835 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7f7d86f0f700 1 -- 172.21.15.163:0/1502194700 shutdown_connections 2023-10-09T06:30:49.835 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7f7d86f0f700 1 -- 172.21.15.163:0/1502194700 wait complete. 2023-10-09T06:30:49.835 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.807+0000 7f9f26b7f700 1 Processor -- start 2023-10-09T06:30:49.835 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.807+0000 7f9f26b7f700 1 -- start start 2023-10-09T06:30:49.836 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7ff513f32700 1 Processor -- start 2023-10-09T06:30:49.836 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.806+0000 7ff513f32700 1 -- start start 2023-10-09T06:30:49.836 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.807+0000 7ff513f32700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff50c156760 0x7ff50c0ac970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.836 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.807+0000 7ff513f32700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff50c0b2490 con 0x7ff50c0cba20 2023-10-09T06:30:49.836 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.807+0000 7ff513f32700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff50c0ac4e0 con 0x7ff50c0691b0 2023-10-09T06:30:49.836 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.807+0000 7ff513f32700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff50c0aceb0 con 0x7ff50c156760 2023-10-09T06:30:49.836 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.807+0000 7ff511cce700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff50c0cba20 0x7ff50c068aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60898/0 (socket says 172.21.15.163:60898) 2023-10-09T06:30:49.836 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.807+0000 7ff511cce700 1 -- 172.21.15.163:0/3537207068 learned_addr learned my addr 172.21.15.163:0/3537207068 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:49.836 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.808+0000 7f7d86f0f700 1 Processor -- start 2023-10-09T06:30:49.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.808+0000 7f693effd700 1 --2- 172.21.15.163:0/266313166 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69400afc40 0x7f694013cfe0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f693000ea50 tx=0x7f6930009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.808+0000 7ff5124cf700 1 --2- 172.21.15.163:0/3537207068 >> v2:172.21.15.163:3300/0 conn(0x7ff50c156760 0x7ff50c0ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.808+0000 7f1ea248c700 1 -- 172.21.15.163:0/1427583306 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e9c0ab880 con 0x7f1e9c0b1fe0 2023-10-09T06:30:49.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f1e83fff700 1 -- 172.21.15.163:0/1427583306 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1e90022030 con 0x7f1e9c0b1fe0 2023-10-09T06:30:49.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.807+0000 7f635e6eb700 1 -- 172.21.15.163:0/3375902368 wait complete. 2023-10-09T06:30:49.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.808+0000 7f3508b38700 1 -- 172.21.15.163:0/1816284136 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35040a5ee0 con 0x7f35040cba20 2023-10-09T06:30:49.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.808+0000 7f3508b38700 1 -- 172.21.15.163:0/1816284136 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35040a6b60 con 0x7f35040cba20 2023-10-09T06:30:49.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f6ed6d9d700 1 --2- 172.21.15.163:0/4274266991 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6ec4067000 0x7f6ec40694b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80099b0 tx=0x7f6ec8017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.808+0000 7f7d86f0f700 1 -- start start 2023-10-09T06:30:49.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f7d86f0f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7d800afc40 0x7f7d80156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f7d86f0f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d800b0010 0x7f7d800ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f7d86f0f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d8015dd90 0x7f7d800adf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f7d86f0f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d801555e0 con 0x7f7d800b0010 2023-10-09T06:30:49.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f7d86f0f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d80154960 con 0x7f7d8015dd90 2023-10-09T06:30:49.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f7d86f0f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d80154c40 con 0x7f7d800afc40 2023-10-09T06:30:49.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f7d7ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d800b0010 0x7f7d800ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f7d7ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d800b0010 0x7f7d800ad9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43916/0 (socket says 172.21.15.163:43916) 2023-10-09T06:30:49.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f7d7ffff700 1 -- 172.21.15.163:0/3253748411 learned_addr learned my addr 172.21.15.163:0/3253748411 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:49.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f7d84cab700 1 --2- 172.21.15.163:0/3253748411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7d800afc40 0x7f7d80156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f7d854ac700 1 --2- 172.21.15.163:0/3253748411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d8015dd90 0x7f7d800adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7ff5124cf700 1 -- 172.21.15.163:0/3537207068 >> v1:172.21.15.181:6789/0 conn(0x7ff50c0691b0 legacy=0x7ff50c156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7ff5124cf700 1 -- 172.21.15.163:0/3537207068 >> v1:172.21.15.163:6789/0 conn(0x7ff50c0cba20 legacy=0x7ff50c068aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:49.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7ff5124cf700 1 -- 172.21.15.163:0/3537207068 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50c156cd0 con 0x7ff50c156760 2023-10-09T06:30:49.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f6926ffd700 1 -- 172.21.15.163:0/266313166 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693000eee0 con 0x7f69400afc40 2023-10-09T06:30:49.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f3508b38700 1 -- 172.21.15.163:0/1816284136 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35040a6d80 con 0x7f35040cba20 2023-10-09T06:30:49.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.810+0000 7f1e83fff700 1 --2- 172.21.15.163:0/1427583306 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1e88066fb0 0x7f1e88069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.810+0000 7f1e83fff700 1 -- 172.21.15.163:0/1427583306 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1e900864d0 con 0x7f1e9c0b1fe0 2023-10-09T06:30:49.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.810+0000 7ff5124cf700 1 --2- 172.21.15.163:0/3537207068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff50c156760 0x7ff50c0ac970 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff504008870 tx=0x7ff504008210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a23a36a0aed3993 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f7d7ffff700 1 -- 172.21.15.163:0/3253748411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7d800afc40 msgr2=0x7f7d80156140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f7d7ffff700 1 --2- 172.21.15.163:0/3253748411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7d800afc40 0x7f7d80156140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f7d7ffff700 1 -- 172.21.15.163:0/3253748411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d8015dd90 msgr2=0x7f7d800adf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f7d7ffff700 1 --2- 172.21.15.163:0/3253748411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d8015dd90 0x7f7d800adf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f7d7ffff700 1 -- 172.21.15.163:0/3253748411 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d78009ae0 con 0x7f7d800b0010 2023-10-09T06:30:49.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.810+0000 7f7d7ffff700 1 --2- 172.21.15.163:0/3253748411 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d800b0010 0x7f7d800ad9f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7d800ace90 tx=0x7f7d78017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.810+0000 7f1e9bfff700 1 --2- 172.21.15.163:0/1427583306 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1e88066fb0 0x7f1e88069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f635e6eb700 1 Processor -- start 2023-10-09T06:30:49.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.810+0000 7f635e6eb700 1 -- start start 2023-10-09T06:30:49.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.808+0000 7f9f26b7f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f201575f0 0x7f9f200abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.808+0000 7f9f26b7f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9f20068aa0 0x7f9f20068ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.808+0000 7f9f26b7f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f200acfe0 con 0x7f9f201579c0 2023-10-09T06:30:49.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.808+0000 7f9f26b7f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f20069420 con 0x7f9f20068aa0 2023-10-09T06:30:49.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.808+0000 7f9f26b7f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f200ac3d0 con 0x7f9f201575f0 2023-10-09T06:30:49.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f9f2491b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f201575f0 0x7f9f200abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f9f2491b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f201575f0 0x7f9f200abdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:44762/0 (socket says 172.21.15.163:44762) 2023-10-09T06:30:49.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f9f2491b700 1 -- 172.21.15.163:0/3436241861 learned_addr learned my addr 172.21.15.163:0/3436241861 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:49.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f9f2511c700 1 --2- 172.21.15.163:0/3436241861 >> v2:172.21.15.181:3300/0 conn(0x7f9f20068aa0 0x7f9f20068ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f9f2511c700 1 -- 172.21.15.163:0/3436241861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f201575f0 msgr2=0x7f9f200abdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:49.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f9f2511c700 1 --2- 172.21.15.163:0/3436241861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f201575f0 0x7f9f200abdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f9f2511c700 1 -- 172.21.15.163:0/3436241861 >> v1:172.21.15.163:6789/0 conn(0x7f9f201579c0 legacy=0x7f9f200cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f9f2511c700 1 -- 172.21.15.163:0/3436241861 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f200ac6b0 con 0x7f9f20068aa0 2023-10-09T06:30:49.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f9f2511c700 1 --2- 172.21.15.163:0/3436241861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f20068aa0 0x7f9f20068ee0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009e60 tx=0x7f9f18009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac71fc5d2d1947f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.810+0000 7f9f1f7fe700 1 -- 172.21.15.163:0/3436241861 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1800e070 con 0x7f9f20068aa0 2023-10-09T06:30:49.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.810+0000 7f9f1f7fe700 1 -- 172.21.15.163:0/3436241861 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1800bb60 con 0x7f9f20068aa0 2023-10-09T06:30:49.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.810+0000 7f9f1f7fe700 1 -- 172.21.15.163:0/3436241861 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18012430 con 0x7f9f20068aa0 2023-10-09T06:30:49.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f6926ffd700 1 -- 172.21.15.163:0/266313166 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693000c850 con 0x7f69400afc40 2023-10-09T06:30:49.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f6926ffd700 1 -- 172.21.15.163:0/266313166 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693000cbd0 con 0x7f69400afc40 2023-10-09T06:30:49.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.809+0000 7f69459dc700 1 -- 172.21.15.163:0/266313166 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6940157230 con 0x7f69400afc40 2023-10-09T06:30:49.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.810+0000 7f69459dc700 1 -- 172.21.15.163:0/266313166 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f694014ed60 con 0x7f69400afc40 2023-10-09T06:30:49.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.811+0000 7f1e9bfff700 1 --2- 172.21.15.163:0/1427583306 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1e88066fb0 0x7f1e88069460 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c147800 tx=0x7f1e8c009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.811+0000 7f635e6eb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63580691b0 0x7f6358159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.811+0000 7f635e6eb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63580b1f40 0x7f63580c9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.811+0000 7f635e6eb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6358159fb0 0x7f635815f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.811+0000 7f635e6eb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63580a69d0 con 0x7f6358159fb0 2023-10-09T06:30:49.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.811+0000 7f635e6eb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63580a66a0 con 0x7f63580691b0 2023-10-09T06:30:49.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.811+0000 7f635e6eb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63580ca240 con 0x7f63580b1f40 2023-10-09T06:30:49.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.811+0000 7f635cc88700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6358159fb0 0x7f635815f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.811+0000 7f6357fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63580691b0 0x7f6358159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.811+0000 7f635cc88700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6358159fb0 0x7f635815f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43920/0 (socket says 172.21.15.163:43920) 2023-10-09T06:30:49.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.811+0000 7f635cc88700 1 -- 172.21.15.163:0/2442839527 learned_addr learned my addr 172.21.15.163:0/2442839527 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:49.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.811+0000 7f6357fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63580691b0 0x7f6358159710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:50462/0 (socket says 172.21.15.163:50462) 2023-10-09T06:30:49.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.811+0000 7f63577fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63580b1f40 0x7f63580c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.811+0000 7f9f26b7f700 1 -- 172.21.15.163:0/3436241861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f20068aa0 msgr2=0x7f9f20068ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.811+0000 7f9f26b7f700 1 --2- 172.21.15.163:0/3436241861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f20068aa0 0x7f9f20068ee0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009e60 tx=0x7f9f18009780 comp rx=0 tx=0).stop 2023-10-09T06:30:49.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.811+0000 7f9f26b7f700 1 -- 172.21.15.163:0/3436241861 shutdown_connections 2023-10-09T06:30:49.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.811+0000 7f9f26b7f700 1 --2- 172.21.15.163:0/3436241861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f20068aa0 0x7f9f20068ee0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.811+0000 7f9f26b7f700 1 --2- 172.21.15.163:0/3436241861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f201575f0 0x7f9f200abdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.811+0000 7f9f26b7f700 1 -- 172.21.15.163:0/3436241861 >> 172.21.15.163:0/3436241861 conn(0x7f9f200cd000 msgr2=0x7f9f20068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:49.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.811+0000 7f9f26b7f700 1 -- 172.21.15.163:0/3436241861 shutdown_connections 2023-10-09T06:30:49.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.811+0000 7f9f26b7f700 1 -- 172.21.15.163:0/3436241861 wait complete. 2023-10-09T06:30:49.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.811+0000 7f635cc88700 1 -- 172.21.15.163:0/2442839527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63580b1f40 msgr2=0x7f63580c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.811+0000 7f635cc88700 1 --2- 172.21.15.163:0/2442839527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63580b1f40 0x7f63580c9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.811+0000 7f635cc88700 1 -- 172.21.15.163:0/2442839527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63580691b0 msgr2=0x7f6358159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.811+0000 7f635cc88700 1 --2- 172.21.15.163:0/2442839527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63580691b0 0x7f6358159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.811+0000 7f635cc88700 1 -- 172.21.15.163:0/2442839527 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6350009a20 con 0x7f6358159fb0 2023-10-09T06:30:49.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.811+0000 7f6357fff700 1 --2- 172.21.15.163:0/2442839527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63580691b0 0x7f6358159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:30:49.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.811+0000 7f63577fe700 1 --2- 172.21.15.163:0/2442839527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63580b1f40 0x7f63580c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:30:49.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.811+0000 7f635cc88700 1 --2- 172.21.15.163:0/2442839527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6358159fb0 0x7f635815f930 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f635806cf40 tx=0x7f6350018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.812+0000 7f69459dc700 1 -- 172.21.15.163:0/266313166 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f694004c3c0 con 0x7f69400afc40 2023-10-09T06:30:49.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.812+0000 7f9f26b7f700 1 Processor -- start 2023-10-09T06:30:49.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.812+0000 7f9f26b7f700 1 -- start start 2023-10-09T06:30:49.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.813+0000 7f9f26b7f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f20068aa0 0x7f9f200af5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.813+0000 7f9f26b7f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f201575f0 0x7f9f201563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.813+0000 7f9f26b7f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f201579c0 0x7f9f2014ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.813+0000 7f9f26b7f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f200b0d90 con 0x7f9f201579c0 2023-10-09T06:30:49.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.813+0000 7f9f26b7f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2015ab30 con 0x7f9f20068aa0 2023-10-09T06:30:49.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.813+0000 7f9f26b7f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f200afea0 con 0x7f9f201575f0 2023-10-09T06:30:49.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.813+0000 7f9f2511c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f201579c0 0x7f9f2014ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.813+0000 7f9f2511c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f201579c0 0x7f9f2014ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43934/0 (socket says 172.21.15.163:43934) 2023-10-09T06:30:49.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.813+0000 7f9f2511c700 1 -- 172.21.15.163:0/2645116452 learned_addr learned my addr 172.21.15.163:0/2645116452 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:49.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.813+0000 7f9f2491b700 1 --2- 172.21.15.163:0/2645116452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f20068aa0 0x7f9f200af5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.813+0000 7f9f1ffff700 1 --2- 172.21.15.163:0/2645116452 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f201575f0 0x7f9f201563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.813+0000 7f9f2511c700 1 -- 172.21.15.163:0/2645116452 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f201575f0 msgr2=0x7f9f201563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.813+0000 7f9f2511c700 1 --2- 172.21.15.163:0/2645116452 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f201575f0 0x7f9f201563a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.813+0000 7f9f2511c700 1 -- 172.21.15.163:0/2645116452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f20068aa0 msgr2=0x7f9f200af5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.813+0000 7f9f2511c700 1 --2- 172.21.15.163:0/2645116452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f20068aa0 0x7f9f200af5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.813+0000 7f9f2511c700 1 -- 172.21.15.163:0/2645116452 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f18009a20 con 0x7f9f201579c0 2023-10-09T06:30:49.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.814+0000 7f9f2511c700 1 --2- 172.21.15.163:0/2645116452 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f201579c0 0x7f9f2014ccf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9f2006cf40 tx=0x7f9f18019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.814+0000 7f6ebeffd700 1 -- 172.21.15.163:0/4274266991 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ed0051340 con 0x7f6ed80875d0 2023-10-09T06:30:49.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.814+0000 7ff510ccc700 1 -- 172.21.15.163:0/3537207068 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff504010070 con 0x7ff50c156760 2023-10-09T06:30:49.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.814+0000 7f9f1cff9700 1 -- 172.21.15.163:0/2645116452 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18027470 con 0x7f9f201579c0 2023-10-09T06:30:49.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.814+0000 7f9f26b7f700 1 -- 172.21.15.163:0/2645116452 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f200b0180 con 0x7f9f201579c0 2023-10-09T06:30:49.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.814+0000 7f9f26b7f700 1 -- 172.21.15.163:0/2645116452 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f200b0740 con 0x7f9f201579c0 2023-10-09T06:30:49.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.814+0000 7f633ffff700 1 -- 172.21.15.163:0/2442839527 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6350027470 con 0x7f6358159fb0 2023-10-09T06:30:49.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.814+0000 7f635e6eb700 1 -- 172.21.15.163:0/2442839527 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63580ca520 con 0x7f6358159fb0 2023-10-09T06:30:49.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.814+0000 7f635e6eb700 1 -- 172.21.15.163:0/2442839527 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6358159c50 con 0x7f6358159fb0 2023-10-09T06:30:49.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.814+0000 7f633ffff700 1 -- 172.21.15.163:0/2442839527 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63500275f0 con 0x7f6358159fb0 2023-10-09T06:30:49.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.814+0000 7f633ffff700 1 -- 172.21.15.163:0/2442839527 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63500279b0 con 0x7f6358159fb0 2023-10-09T06:30:49.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.814+0000 7ff510ccc700 1 -- 172.21.15.163:0/3537207068 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50400d9a0 con 0x7ff50c156760 2023-10-09T06:30:49.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.814+0000 7ff510ccc700 1 -- 172.21.15.163:0/3537207068 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50400dd40 con 0x7ff50c156760 2023-10-09T06:30:49.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.815+0000 7ff513f32700 1 -- 172.21.15.163:0/3537207068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff50c156760 msgr2=0x7ff50c0ac970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.815+0000 7ff513f32700 1 --2- 172.21.15.163:0/3537207068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff50c156760 0x7ff50c0ac970 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff504008870 tx=0x7ff504008210 comp rx=0 tx=0).stop 2023-10-09T06:30:49.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.815+0000 7f34f27fc700 1 -- 172.21.15.163:0/1816284136 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f34f401d040 con 0x7f35040cba20 2023-10-09T06:30:49.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.816+0000 7f6926ffd700 1 -- 172.21.15.163:0/266313166 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6930013470 con 0x7f69400afc40 2023-10-09T06:30:49.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.815+0000 7f9f26b7f700 1 -- 172.21.15.163:0/2645116452 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f200b0f40 con 0x7f9f201579c0 2023-10-09T06:30:49.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.816+0000 7f9f1cff9700 1 -- 172.21.15.163:0/2645116452 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f180275d0 con 0x7f9f201579c0 2023-10-09T06:30:49.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.816+0000 7f9f1cff9700 1 -- 172.21.15.163:0/2645116452 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18027970 con 0x7f9f201579c0 2023-10-09T06:30:49.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.816+0000 7ff513f32700 1 -- 172.21.15.163:0/3537207068 shutdown_connections 2023-10-09T06:30:49.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.816+0000 7ff513f32700 1 --2- 172.21.15.163:0/3537207068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff50c156760 0x7ff50c0ac970 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.816+0000 7ff513f32700 1 -- 172.21.15.163:0/3537207068 >> 172.21.15.163:0/3537207068 conn(0x7ff50c0cd000 msgr2=0x7ff50c068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:49.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.816+0000 7ff513f32700 1 -- 172.21.15.163:0/3537207068 shutdown_connections 2023-10-09T06:30:49.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.816+0000 7ff513f32700 1 -- 172.21.15.163:0/3537207068 wait complete. 2023-10-09T06:30:49.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.816+0000 7f6926ffd700 1 --2- 172.21.15.163:0/266313166 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f692c067000 0x7f692c0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.816+0000 7f34f27fc700 1 --2- 172.21.15.163:0/1816284136 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f34e4066f30 0x7f34e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.816+0000 7f34f27fc700 1 -- 172.21.15.163:0/1816284136 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f34f40859c0 con 0x7f35040cba20 2023-10-09T06:30:49.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.816+0000 7f350259c700 1 --2- 172.21.15.163:0/1816284136 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f34e4066f30 0x7f34e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.816+0000 7f6926ffd700 1 -- 172.21.15.163:0/266313166 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6930084dd0 con 0x7f69400afc40 2023-10-09T06:30:49.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.816+0000 7f693e7fc700 1 --2- 172.21.15.163:0/266313166 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f692c067000 0x7f692c0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.817+0000 7f350259c700 1 --2- 172.21.15.163:0/1816284136 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f34e4066f30 0x7f34e40693e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f34f8001d20 tx=0x7f34f8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.817+0000 7f693e7fc700 1 --2- 172.21.15.163:0/266313166 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f692c067000 0x7f692c0694b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f694015ca70 tx=0x7f6934016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.817+0000 7ff513f32700 1 Processor -- start 2023-10-09T06:30:49.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.817+0000 7ff513f32700 1 -- start start 2023-10-09T06:30:49.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.818+0000 7f7d7cff9700 1 -- 172.21.15.163:0/3253748411 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d78012890 con 0x7f7d800b0010 2023-10-09T06:30:49.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.818+0000 7f7d7cff9700 1 -- 172.21.15.163:0/3253748411 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d78012a10 con 0x7f7d800b0010 2023-10-09T06:30:49.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.818+0000 7f7d86f0f700 1 -- 172.21.15.163:0/3253748411 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d80154f20 con 0x7f7d800b0010 2023-10-09T06:30:49.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.818+0000 7f7d7cff9700 1 -- 172.21.15.163:0/3253748411 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d78012dd0 con 0x7f7d800b0010 2023-10-09T06:30:49.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.818+0000 7f7d86f0f700 1 -- 172.21.15.163:0/3253748411 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d80154580 con 0x7f7d800b0010 2023-10-09T06:30:49.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.818+0000 7ff513f32700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff50c0691b0 0x7ff50c140f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.818+0000 7ff513f32700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff50c0cba20 0x7ff50c15b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.818+0000 7ff513f32700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff50c156760 0x7ff50c0a6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.818+0000 7ff513f32700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c15b0b0 con 0x7ff50c0cba20 2023-10-09T06:30:49.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.818+0000 7ff513f32700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c15ac20 con 0x7ff50c156760 2023-10-09T06:30:49.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.818+0000 7ff513f32700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c15a050 con 0x7ff50c0691b0 2023-10-09T06:30:49.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.818+0000 7ff511cce700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff50c0691b0 0x7ff50c140f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.818+0000 7ff511cce700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff50c0691b0 0x7ff50c140f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:44796/0 (socket says 172.21.15.163:44796) 2023-10-09T06:30:49.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.818+0000 7ff511cce700 1 -- 172.21.15.163:0/2142010895 learned_addr learned my addr 172.21.15.163:0/2142010895 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:49.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.819+0000 7ff511cce700 1 -- 172.21.15.163:0/2142010895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff50c156760 msgr2=0x7ff50c0a6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.819+0000 7ff511cce700 1 --2- 172.21.15.163:0/2142010895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff50c156760 0x7ff50c0a6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.819+0000 7ff511cce700 1 -- 172.21.15.163:0/2142010895 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff50c0cba20 msgr2=0x7ff50c15b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:49.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.819+0000 7ff511cce700 1 --2- 172.21.15.163:0/2142010895 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff50c0cba20 0x7ff50c15b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:49.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.819+0000 7ff511cce700 1 -- 172.21.15.163:0/2142010895 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5040084b0 con 0x7ff50c0691b0 2023-10-09T06:30:49.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.819+0000 7ff511cce700 1 --2- 172.21.15.163:0/2142010895 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff50c0691b0 0x7ff50c140f30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00c610 tx=0x7ff4fc004a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.819+0000 7f7d86f0f700 1 -- 172.21.15.163:0/3253748411 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d8004c3c0 con 0x7f7d800b0010 2023-10-09T06:30:49.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.819+0000 7ff501ffb700 1 -- 172.21.15.163:0/2142010895 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc005b30 con 0x7ff50c0691b0 2023-10-09T06:30:49.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.819+0000 7ff501ffb700 1 -- 172.21.15.163:0/2142010895 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4fc005cb0 con 0x7ff50c0691b0 2023-10-09T06:30:49.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.819+0000 7ff501ffb700 1 -- 172.21.15.163:0/2142010895 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc007320 con 0x7ff50c0691b0 2023-10-09T06:30:49.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.819+0000 7ff513f32700 1 -- 172.21.15.163:0/2142010895 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff50c15a2f0 con 0x7ff50c0691b0 2023-10-09T06:30:49.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.819+0000 7f6926ffd700 1 -- 172.21.15.163:0/266313166 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f69300521a0 con 0x7f69400afc40 2023-10-09T06:30:49.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.819+0000 7ff513f32700 1 -- 172.21.15.163:0/2142010895 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff50c15a8b0 con 0x7ff50c0691b0 2023-10-09T06:30:49.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.820+0000 7f34f27fc700 1 -- 172.21.15.163:0/1816284136 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f34f4052e50 con 0x7f35040cba20 2023-10-09T06:30:49.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.821+0000 7ff501ffb700 1 -- 172.21.15.163:0/2142010895 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff4fc007690 con 0x7ff50c0691b0 2023-10-09T06:30:49.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.821+0000 7ff513f32700 1 -- 172.21.15.163:0/2142010895 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff50c04c3c0 con 0x7ff50c0691b0 2023-10-09T06:30:49.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.821+0000 7ff501ffb700 1 --2- 172.21.15.163:0/2142010895 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff4f0066f30 0x7ff4f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.821+0000 7ff501ffb700 1 -- 172.21.15.163:0/2142010895 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff4fc083b40 con 0x7ff50c0691b0 2023-10-09T06:30:49.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.821+0000 7ff5114cd700 1 --2- 172.21.15.163:0/2142010895 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff4f0066f30 0x7ff4f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.815+0000 7f1e83fff700 1 -- 172.21.15.163:0/1427583306 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1e900536a0 con 0x7f1e9c0b1fe0 2023-10-09T06:30:49.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.822+0000 7ff5114cd700 1 --2- 172.21.15.163:0/2142010895 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff4f0066f30 0x7ff4f00693e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff50c0ab620 tx=0x7ff508007e60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.822+0000 7f633dffb700 1 -- 172.21.15.163:0/2442839527 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63580a6500 con 0x7f6358159fb0 2023-10-09T06:30:49.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.822+0000 7f7d7cff9700 1 -- 172.21.15.163:0/3253748411 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7d7801d630 con 0x7f7d800b0010 2023-10-09T06:30:49.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.823+0000 7f7d7cff9700 1 --2- 172.21.15.163:0/3253748411 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7d6c067000 0x7f7d6c0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.823+0000 7f7d7cff9700 1 -- 172.21.15.163:0/3253748411 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7d78086400 con 0x7f7d800b0010 2023-10-09T06:30:49.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.823+0000 7f7d84cab700 1 --2- 172.21.15.163:0/3253748411 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7d6c067000 0x7f7d6c0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.823+0000 7f7d84cab700 1 --2- 172.21.15.163:0/3253748411 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7d6c067000 0x7f7d6c0694b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7d80156c50 tx=0x7f7d740090f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.826+0000 7f9f1cff9700 1 -- 172.21.15.163:0/2645116452 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9f18024070 con 0x7f9f201579c0 2023-10-09T06:30:49.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.825+0000 7f633ffff700 1 -- 172.21.15.163:0/2442839527 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6350024070 con 0x7f6358159fb0 2023-10-09T06:30:49.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.826+0000 7f633ffff700 1 --2- 172.21.15.163:0/2442839527 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6344066f00 0x7f63440693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.826+0000 7f633ffff700 1 -- 172.21.15.163:0/2442839527 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6350087380 con 0x7f6358159fb0 2023-10-09T06:30:49.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.826+0000 7f6357fff700 1 --2- 172.21.15.163:0/2442839527 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6344066f00 0x7f63440693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.826+0000 7f9f1cff9700 1 --2- 172.21.15.163:0/2645116452 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9f0c066f30 0x7f9f0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:49.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.827+0000 7f9f1cff9700 1 -- 172.21.15.163:0/2645116452 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9f18087220 con 0x7f9f201579c0 2023-10-09T06:30:49.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.827+0000 7f9f2491b700 1 --2- 172.21.15.163:0/2645116452 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9f0c066f30 0x7f9f0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:49.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.827+0000 7f6357fff700 1 --2- 172.21.15.163:0/2442839527 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6344066f00 0x7f63440693b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6348001d70 tx=0x7f6348009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.827+0000 7f9f2491b700 1 --2- 172.21.15.163:0/2645116452 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9f0c066f30 0x7f9f0c0693e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9f14006d40 tx=0x7f9f14009250 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:49.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.830+0000 7ff501ffb700 1 -- 172.21.15.163:0/2142010895 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff4fc050db0 con 0x7ff50c0691b0 2023-10-09T06:30:49.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.830+0000 7f9f1cff9700 1 -- 172.21.15.163:0/2645116452 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f180543f0 con 0x7f9f201579c0 2023-10-09T06:30:49.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.832+0000 7f7d7cff9700 1 -- 172.21.15.163:0/3253748411 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7d780537d0 con 0x7f7d800b0010 2023-10-09T06:30:49.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:49.839+0000 7f633ffff700 1 -- 172.21.15.163:0/2442839527 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6350054680 con 0x7f6358159fb0 2023-10-09T06:30:50.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.196+0000 7f6edd0ad700 1 -- 172.21.15.163:0/4274266991 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6eb8005830 con 0x7f6ed80875d0 2023-10-09T06:30:50.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.206+0000 7ff513f32700 1 -- 172.21.15.163:0/2142010895 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7ff50c0ad410 con 0x7ff50c0691b0 2023-10-09T06:30:50.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.209+0000 7f1ea248c700 1 -- 172.21.15.163:0/1427583306 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1e9c1344d0 con 0x7f1e9c0b1fe0 2023-10-09T06:30:50.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.210+0000 7f1e83fff700 1 -- 172.21.15.163:0/1427583306 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1e90056cc0 con 0x7f1e9c0b1fe0 2023-10-09T06:30:50.210 INFO:teuthology.orchestra.run.smithi163.stdout:55834574856 2023-10-09T06:30:50.212 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.212+0000 7f69459dc700 1 -- 172.21.15.163:0/266313166 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f69400ad310 con 0x7f69400afc40 2023-10-09T06:30:50.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.212+0000 7f6ebeffd700 1 -- 172.21.15.163:0/4274266991 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6ed0054960 con 0x7f6ed80875d0 2023-10-09T06:30:50.213 INFO:teuthology.orchestra.run.smithi163.stdout:55834574855 2023-10-09T06:30:50.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.212+0000 7ff501ffb700 1 -- 172.21.15.163:0/2142010895 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff4fc017020 con 0x7ff50c0691b0 2023-10-09T06:30:50.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.213+0000 7f1ea248c700 1 -- 172.21.15.163:0/1427583306 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1e88066fb0 msgr2=0x7f1e88069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:50.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.213+0000 7f1ea248c700 1 --2- 172.21.15.163:0/1427583306 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1e88066fb0 0x7f1e88069460 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c147800 tx=0x7f1e8c009060 comp rx=0 tx=0).stop 2023-10-09T06:30:50.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.213+0000 7f1ea248c700 1 -- 172.21.15.163:0/1427583306 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e9c0b1fe0 msgr2=0x7f1e9c1347f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:50.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.213+0000 7f1ea248c700 1 --2- 172.21.15.163:0/1427583306 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e9c0b1fe0 0x7f1e9c1347f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1e90015000 tx=0x7f1e90016000 comp rx=0 tx=0).stop 2023-10-09T06:30:50.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.214+0000 7f3508b38700 1 -- 172.21.15.163:0/1816284136 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f350404c3c0 con 0x7f35040cba20 2023-10-09T06:30:50.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.214+0000 7f34f27fc700 1 -- 172.21.15.163:0/1816284136 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f34f4056470 con 0x7f35040cba20 2023-10-09T06:30:50.216 INFO:teuthology.orchestra.run.smithi163.stdout:55834574856 2023-10-09T06:30:50.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.215+0000 7f6edd0ad700 1 -- 172.21.15.163:0/4274266991 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6ec4067000 msgr2=0x7f6ec40694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:50.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.215+0000 7f6edd0ad700 1 --2- 172.21.15.163:0/4274266991 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6ec4067000 0x7f6ec40694b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80099b0 tx=0x7f6ec8017000 comp rx=0 tx=0).stop 2023-10-09T06:30:50.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.215+0000 7f6edd0ad700 1 -- 172.21.15.163:0/4274266991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ed80875d0 msgr2=0x7f6ed815e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:50.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.215+0000 7f6edd0ad700 1 --2- 172.21.15.163:0/4274266991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ed80875d0 0x7f6ed815e7f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000bc30 tx=0x7f6ed0009840 comp rx=0 tx=0).stop 2023-10-09T06:30:50.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.215+0000 7f6edd0ad700 1 -- 172.21.15.163:0/4274266991 shutdown_connections 2023-10-09T06:30:50.218 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.215+0000 7f6edd0ad700 1 --2- 172.21.15.163:0/4274266991 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6ec4067000 0x7f6ec40694b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.218 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.215+0000 7f6edd0ad700 1 --2- 172.21.15.163:0/4274266991 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ed80b1c50 0x7f6ed81546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.218 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.215+0000 7f6edd0ad700 1 --2- 172.21.15.163:0/4274266991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ed8068aa0 0x7f6ed815cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.218 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.215+0000 7f6edd0ad700 1 --2- 172.21.15.163:0/4274266991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ed80875d0 0x7f6ed815e7f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.218 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.215+0000 7f6edd0ad700 1 -- 172.21.15.163:0/4274266991 >> 172.21.15.163:0/4274266991 conn(0x7f6ed80cd000 msgr2=0x7f6ed8168d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:50.218 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.215+0000 7f6edd0ad700 1 -- 172.21.15.163:0/4274266991 shutdown_connections 2023-10-09T06:30:50.218 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.215+0000 7f6edd0ad700 1 -- 172.21.15.163:0/4274266991 wait complete. 2023-10-09T06:30:50.219 INFO:teuthology.orchestra.run.smithi163.stdout:55834574854 2023-10-09T06:30:50.219 INFO:teuthology.orchestra.run.smithi163.stdout:55834574854 2023-10-09T06:30:50.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.216+0000 7f6926ffd700 1 -- 172.21.15.163:0/266313166 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6930017020 con 0x7f69400afc40 2023-10-09T06:30:50.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.218+0000 7ff513f32700 1 -- 172.21.15.163:0/2142010895 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff4f0066f30 msgr2=0x7ff4f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:50.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.218+0000 7ff513f32700 1 --2- 172.21.15.163:0/2142010895 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff4f0066f30 0x7ff4f00693e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff50c0ab620 tx=0x7ff508007e60 comp rx=0 tx=0).stop 2023-10-09T06:30:50.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.218+0000 7ff513f32700 1 -- 172.21.15.163:0/2142010895 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff50c0691b0 msgr2=0x7ff50c140f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:50.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.218+0000 7ff513f32700 1 --2- 172.21.15.163:0/2142010895 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff50c0691b0 0x7ff50c140f30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00c610 tx=0x7ff4fc004a80 comp rx=0 tx=0).stop 2023-10-09T06:30:50.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.218+0000 7f3508b38700 1 -- 172.21.15.163:0/1816284136 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f34e4066f30 msgr2=0x7f34e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:50.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.218+0000 7f3508b38700 1 --2- 172.21.15.163:0/1816284136 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f34e4066f30 0x7f34e40693e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f34f8001d20 tx=0x7f34f8009040 comp rx=0 tx=0).stop 2023-10-09T06:30:50.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.218+0000 7f3508b38700 1 -- 172.21.15.163:0/1816284136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35040cba20 msgr2=0x7f3504152360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:50.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.218+0000 7f3508b38700 1 --2- 172.21.15.163:0/1816284136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35040cba20 0x7f3504152360 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f34f4018000 tx=0x7f34f401b000 comp rx=0 tx=0).stop 2023-10-09T06:30:50.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.220+0000 7f3508b38700 1 -- 172.21.15.163:0/1816284136 shutdown_connections 2023-10-09T06:30:50.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.220+0000 7f3508b38700 1 --2- 172.21.15.163:0/1816284136 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f34e4066f30 0x7f34e40693e0 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f34f8001d20 tx=0x7f34f8009040 comp rx=0 tx=0).stop 2023-10-09T06:30:50.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.220+0000 7f3508b38700 1 --2- 172.21.15.163:0/1816284136 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3504156050 0x7f35040a7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.220+0000 7f3508b38700 1 --2- 172.21.15.163:0/1816284136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35040691b0 0x7f35040a8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.220+0000 7f3508b38700 1 --2- 172.21.15.163:0/1816284136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35040cba20 0x7f3504152360 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.220+0000 7f3508b38700 1 -- 172.21.15.163:0/1816284136 >> 172.21.15.163:0/1816284136 conn(0x7f35040cd000 msgr2=0x7f3504154500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:50.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.222+0000 7f1ea248c700 1 -- 172.21.15.163:0/1427583306 shutdown_connections 2023-10-09T06:30:50.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.222+0000 7f1ea248c700 1 --2- 172.21.15.163:0/1427583306 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1e88066fb0 0x7f1e88069460 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.222+0000 7f1ea248c700 1 --2- 172.21.15.163:0/1427583306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e9c0691b0 0x7f1e9c161e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.222+0000 7f1ea248c700 1 --2- 172.21.15.163:0/1427583306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e9c158440 0x7f1e9c15abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.222+0000 7f1ea248c700 1 --2- 172.21.15.163:0/1427583306 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e9c0b1fe0 0x7f1e9c1347f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.222+0000 7f1ea248c700 1 -- 172.21.15.163:0/1427583306 >> 172.21.15.163:0/1427583306 conn(0x7f1e9c0cd000 msgr2=0x7f1e9c0abd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:50.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.222+0000 7f7d86f0f700 1 -- 172.21.15.163:0/3253748411 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7d800ad310 con 0x7f7d800b0010 2023-10-09T06:30:50.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.222+0000 7f1ea248c700 1 -- 172.21.15.163:0/1427583306 shutdown_connections 2023-10-09T06:30:50.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.222+0000 7f1ea248c700 1 -- 172.21.15.163:0/1427583306 wait complete. 2023-10-09T06:30:50.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.222+0000 7f7d7cff9700 1 -- 172.21.15.163:0/3253748411 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7d78009420 con 0x7f7d800b0010 2023-10-09T06:30:50.226 INFO:teuthology.orchestra.run.smithi163.stdout:55834574855 2023-10-09T06:30:50.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.224+0000 7ff513f32700 1 -- 172.21.15.163:0/2142010895 shutdown_connections 2023-10-09T06:30:50.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.224+0000 7ff513f32700 1 --2- 172.21.15.163:0/2142010895 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff4f0066f30 0x7ff4f00693e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.224+0000 7ff513f32700 1 --2- 172.21.15.163:0/2142010895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff50c156760 0x7ff50c0a6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.224+0000 7ff513f32700 1 --2- 172.21.15.163:0/2142010895 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff50c0691b0 0x7ff50c140f30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.224+0000 7ff513f32700 1 --2- 172.21.15.163:0/2142010895 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff50c0cba20 0x7ff50c15b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.224+0000 7ff513f32700 1 -- 172.21.15.163:0/2142010895 >> 172.21.15.163:0/2142010895 conn(0x7ff50c0cd000 msgr2=0x7ff50c0afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:50.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.224+0000 7ff513f32700 1 -- 172.21.15.163:0/2142010895 shutdown_connections 2023-10-09T06:30:50.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.224+0000 7ff513f32700 1 -- 172.21.15.163:0/2142010895 wait complete. 2023-10-09T06:30:50.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.225+0000 7f3508b38700 1 -- 172.21.15.163:0/1816284136 shutdown_connections 2023-10-09T06:30:50.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.225+0000 7f9f26b7f700 1 -- 172.21.15.163:0/2645116452 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f9f200ac9f0 con 0x7f9f201579c0 2023-10-09T06:30:50.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.225+0000 7f3508b38700 1 -- 172.21.15.163:0/1816284136 wait complete. 2023-10-09T06:30:50.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.225+0000 7f9f1cff9700 1 -- 172.21.15.163:0/2645116452 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9f1802c090 con 0x7f9f201579c0 2023-10-09T06:30:50.229 INFO:teuthology.orchestra.run.smithi163.stdout:55834574853 2023-10-09T06:30:50.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.226+0000 7f69459dc700 1 -- 172.21.15.163:0/266313166 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f692c067000 msgr2=0x7f692c0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:50.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.226+0000 7f69459dc700 1 --2- 172.21.15.163:0/266313166 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f692c067000 0x7f692c0694b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f694015ca70 tx=0x7f6934016000 comp rx=0 tx=0).stop 2023-10-09T06:30:50.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.226+0000 7f69459dc700 1 -- 172.21.15.163:0/266313166 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69400afc40 msgr2=0x7f694013cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:50.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.226+0000 7f69459dc700 1 --2- 172.21.15.163:0/266313166 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69400afc40 0x7f694013cfe0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f693000ea50 tx=0x7f6930009250 comp rx=0 tx=0).stop 2023-10-09T06:30:50.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.226+0000 7f69459dc700 1 -- 172.21.15.163:0/266313166 shutdown_connections 2023-10-09T06:30:50.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.226+0000 7f69459dc700 1 --2- 172.21.15.163:0/266313166 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f692c067000 0x7f692c0694b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.226+0000 7f69459dc700 1 --2- 172.21.15.163:0/266313166 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69400b2140 0x7f6940156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.226+0000 7f69459dc700 1 --2- 172.21.15.163:0/266313166 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69400afc40 0x7f694013cfe0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.226+0000 7f69459dc700 1 --2- 172.21.15.163:0/266313166 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69400b0010 0x7f6940156140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.226+0000 7f69459dc700 1 -- 172.21.15.163:0/266313166 >> 172.21.15.163:0/266313166 conn(0x7f694006a320 msgr2=0x7f694014f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:50.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.226+0000 7f69459dc700 1 -- 172.21.15.163:0/266313166 shutdown_connections 2023-10-09T06:30:50.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.226+0000 7f69459dc700 1 -- 172.21.15.163:0/266313166 wait complete. 2023-10-09T06:30:50.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.228+0000 7f9f26b7f700 1 -- 172.21.15.163:0/2645116452 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9f0c066f30 msgr2=0x7f9f0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:50.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.228+0000 7f9f26b7f700 1 --2- 172.21.15.163:0/2645116452 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9f0c066f30 0x7f9f0c0693e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9f14006d40 tx=0x7f9f14009250 comp rx=0 tx=0).stop 2023-10-09T06:30:50.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.228+0000 7f9f26b7f700 1 -- 172.21.15.163:0/2645116452 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f201579c0 msgr2=0x7f9f2014ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:50.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.228+0000 7f9f26b7f700 1 --2- 172.21.15.163:0/2645116452 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f201579c0 0x7f9f2014ccf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9f2006cf40 tx=0x7f9f18019000 comp rx=0 tx=0).stop 2023-10-09T06:30:50.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.230+0000 7f633dffb700 1 -- 172.21.15.163:0/2442839527 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f63580a6500 con 0x7f6358159fb0 2023-10-09T06:30:50.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.232+0000 7f633ffff700 1 -- 172.21.15.163:0/2442839527 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f63580a6500 con 0x7f6358159fb0 2023-10-09T06:30:50.234 INFO:teuthology.orchestra.run.smithi163.stdout:55834574854 2023-10-09T06:30:50.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.234+0000 7f9f26b7f700 1 -- 172.21.15.163:0/2645116452 shutdown_connections 2023-10-09T06:30:50.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.234+0000 7f9f26b7f700 1 --2- 172.21.15.163:0/2645116452 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9f0c066f30 0x7f9f0c0693e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.234+0000 7f9f26b7f700 1 --2- 172.21.15.163:0/2645116452 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f201579c0 0x7f9f2014ccf0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.234+0000 7f9f26b7f700 1 --2- 172.21.15.163:0/2645116452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f20068aa0 0x7f9f200af5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.234+0000 7f9f26b7f700 1 --2- 172.21.15.163:0/2645116452 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f201575f0 0x7f9f201563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.234+0000 7f9f26b7f700 1 -- 172.21.15.163:0/2645116452 >> 172.21.15.163:0/2645116452 conn(0x7f9f200cd000 msgr2=0x7f9f200ca7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:50.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.234+0000 7f633dffb700 1 -- 172.21.15.163:0/2442839527 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6344066f00 msgr2=0x7f63440693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:50.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.234+0000 7f633dffb700 1 --2- 172.21.15.163:0/2442839527 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6344066f00 0x7f63440693b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6348001d70 tx=0x7f6348009040 comp rx=0 tx=0).stop 2023-10-09T06:30:50.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.234+0000 7f633dffb700 1 -- 172.21.15.163:0/2442839527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6358159fb0 msgr2=0x7f635815f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:50.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.234+0000 7f633dffb700 1 --2- 172.21.15.163:0/2442839527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6358159fb0 0x7f635815f930 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f635806cf40 tx=0x7f6350018000 comp rx=0 tx=0).stop 2023-10-09T06:30:50.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.235+0000 7f9f26b7f700 1 -- 172.21.15.163:0/2645116452 shutdown_connections 2023-10-09T06:30:50.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.235+0000 7f9f26b7f700 1 -- 172.21.15.163:0/2645116452 wait complete. 2023-10-09T06:30:50.239 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.3 2023-10-09T06:30:50.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.238+0000 7f7d86f0f700 1 -- 172.21.15.163:0/3253748411 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7d6c067000 msgr2=0x7f7d6c0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:50.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.238+0000 7f7d86f0f700 1 --2- 172.21.15.163:0/3253748411 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7d6c067000 0x7f7d6c0694b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7d80156c50 tx=0x7f7d740090f0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.238+0000 7f7d86f0f700 1 -- 172.21.15.163:0/3253748411 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d800b0010 msgr2=0x7f7d800ad9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:50.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.238+0000 7f7d86f0f700 1 --2- 172.21.15.163:0/3253748411 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d800b0010 0x7f7d800ad9f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7d800ace90 tx=0x7f7d78017000 comp rx=0 tx=0).stop 2023-10-09T06:30:50.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.238+0000 7f633dffb700 1 -- 172.21.15.163:0/2442839527 shutdown_connections 2023-10-09T06:30:50.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.238+0000 7f633dffb700 1 --2- 172.21.15.163:0/2442839527 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6344066f00 0x7f63440693b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.238+0000 7f633dffb700 1 --2- 172.21.15.163:0/2442839527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6358159fb0 0x7f635815f930 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.238+0000 7f633dffb700 1 --2- 172.21.15.163:0/2442839527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63580691b0 0x7f6358159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.238+0000 7f633dffb700 1 --2- 172.21.15.163:0/2442839527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63580b1f40 0x7f63580c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.238+0000 7f633dffb700 1 -- 172.21.15.163:0/2442839527 >> 172.21.15.163:0/2442839527 conn(0x7f63580cd000 msgr2=0x7f6358068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:50.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.240+0000 7f633dffb700 1 -- 172.21.15.163:0/2442839527 shutdown_connections 2023-10-09T06:30:50.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.240+0000 7f633dffb700 1 -- 172.21.15.163:0/2442839527 wait complete. 2023-10-09T06:30:50.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.246+0000 7f7d86f0f700 1 -- 172.21.15.163:0/3253748411 shutdown_connections 2023-10-09T06:30:50.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.246+0000 7f7d86f0f700 1 --2- 172.21.15.163:0/3253748411 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7d6c067000 0x7f7d6c0694b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.246+0000 7f7d86f0f700 1 --2- 172.21.15.163:0/3253748411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7d8015dd90 0x7f7d800adf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.246+0000 7f7d86f0f700 1 --2- 172.21.15.163:0/3253748411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7d800afc40 0x7f7d80156140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.246+0000 7f7d86f0f700 1 --2- 172.21.15.163:0/3253748411 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d800b0010 0x7f7d800ad9f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:50.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.246+0000 7f7d86f0f700 1 -- 172.21.15.163:0/3253748411 >> 172.21.15.163:0/3253748411 conn(0x7f7d8006a320 msgr2=0x7f7d800abbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:50.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.246+0000 7f7d86f0f700 1 -- 172.21.15.163:0/3253748411 shutdown_connections 2023-10-09T06:30:50.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:50.246+0000 7f7d86f0f700 1 -- 172.21.15.163:0/3253748411 wait complete. 2023-10-09T06:30:50.249 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2023-10-09T06:30:50.250 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.2 2023-10-09T06:30:50.250 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.1 2023-10-09T06:30:50.253 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.7 2023-10-09T06:30:50.259 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2023-10-09T06:30:50.263 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2023-10-09T06:30:50.264 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.0 2023-10-09T06:30:51.240 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-09T06:30:51.250 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-09T06:30:51.250 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-09T06:30:51.252 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-09T06:30:51.254 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-09T06:30:51.260 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-09T06:30:51.263 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-09T06:30:51.265 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-09T06:30:51.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.423+0000 7f59c4def700 1 Processor -- start 2023-10-09T06:30:51.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.423+0000 7f59c4def700 1 -- start start 2023-10-09T06:30:51.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.425+0000 7f59c4def700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f59c00691b0 0x7f59c00879a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.425+0000 7f59c4def700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59c00acdf0 con 0x7f59c00b1c50 2023-10-09T06:30:51.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.425+0000 7f59c4def700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59c00ac1e0 con 0x7f59c00875d0 2023-10-09T06:30:51.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.425+0000 7f59c4def700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59c00abfd0 con 0x7f59c00691b0 2023-10-09T06:30:51.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.425+0000 7f59be59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f59c00875d0 0x7f59c00ab8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:47850/0 (socket says 172.21.15.163:47850) 2023-10-09T06:30:51.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.425+0000 7f59be59c700 1 -- 172.21.15.163:0/3175131098 learned_addr learned my addr 172.21.15.163:0/3175131098 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:51.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.425+0000 7f59bed9d700 1 --2- 172.21.15.163:0/3175131098 >> v2:172.21.15.181:3300/0 conn(0x7f59c00691b0 0x7f59c00879a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.426+0000 7f59bd59a700 1 -- 172.21.15.163:0/3175131098 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 720315212 0 0) 0x7f59c00acdf0 con 0x7f59c00b1c50 2023-10-09T06:30:51.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.426+0000 7f59bd59a700 1 -- 172.21.15.163:0/3175131098 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59a8003570 con 0x7f59c00b1c50 2023-10-09T06:30:51.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.426+0000 7f59bd59a700 1 -- 172.21.15.163:0/3175131098 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2884066381 0 0) 0x7f59a8003570 con 0x7f59c00b1c50 2023-10-09T06:30:51.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.427+0000 7f59bd59a700 1 -- 172.21.15.163:0/3175131098 >> v2:172.21.15.181:3300/0 conn(0x7f59c00691b0 msgr2=0x7f59c00879a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:51.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.427+0000 7f59bd59a700 1 --2- 172.21.15.163:0/3175131098 >> v2:172.21.15.181:3300/0 conn(0x7f59c00691b0 0x7f59c00879a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.427+0000 7f59bd59a700 1 -- 172.21.15.163:0/3175131098 >> v1:172.21.15.181:6789/0 conn(0x7f59c00875d0 legacy=0x7f59c00ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.427+0000 7f59bd59a700 1 -- 172.21.15.163:0/3175131098 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c00ac4c0 con 0x7f59c00b1c50 2023-10-09T06:30:51.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.427+0000 7f59bd59a700 1 -- 172.21.15.163:0/3175131098 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f59b80017f0 con 0x7f59c00b1c50 2023-10-09T06:30:51.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.427+0000 7f59bd59a700 1 -- 172.21.15.163:0/3175131098 >> v1:172.21.15.163:6789/0 conn(0x7f59c00b1c50 legacy=0x7f59c0068aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.427+0000 7f59bd59a700 1 --2- 172.21.15.163:0/3175131098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59a8003a60 0x7f59a8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.427+0000 7f59bd59a700 1 --2- 172.21.15.163:0/3175131098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59a8004810 0x7f59a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.427+0000 7f59bd59a700 1 --2- 172.21.15.163:0/3175131098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59a8009a50 0x7f59a8005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.427+0000 7f59bd59a700 1 -- 172.21.15.163:0/3175131098 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c00abfd0 con 0x7f59a8009a50 2023-10-09T06:30:51.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.427+0000 7f59bd59a700 1 -- 172.21.15.163:0/3175131098 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a8014990 con 0x7f59a8004810 2023-10-09T06:30:51.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.427+0000 7f59bd59a700 1 -- 172.21.15.163:0/3175131098 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59a8009e40 con 0x7f59a8003a60 2023-10-09T06:30:51.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.427+0000 7f59bed9d700 1 --2- 172.21.15.163:0/3175131098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59a8009a50 0x7f59a8005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.428+0000 7f59bed9d700 1 -- 172.21.15.163:0/3175131098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59a8003a60 msgr2=0x7f59a8003e70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-09T06:30:51.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.428+0000 7f59bed9d700 1 --2- 172.21.15.163:0/3175131098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59a8003a60 0x7f59a8003e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.428+0000 7f59bed9d700 1 -- 172.21.15.163:0/3175131098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59a8004810 msgr2=0x7f59a8004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-09T06:30:51.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.428+0000 7f59bed9d700 1 --2- 172.21.15.163:0/3175131098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59a8004810 0x7f59a8004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.428+0000 7f59bed9d700 1 -- 172.21.15.163:0/3175131098 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c00ac4c0 con 0x7f59a8009a50 2023-10-09T06:30:51.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.428+0000 7f59bed9d700 1 -- 172.21.15.163:0/3175131098 reap_dead start 2023-10-09T06:30:51.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.428+0000 7f59bed9d700 1 --2- 172.21.15.163:0/3175131098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59a8009a50 0x7f59a8005250 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f59c00695b0 tx=0x7f59b400b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1d53ed406d8f31 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:51.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.428+0000 7f59bd59a700 1 -- 172.21.15.163:0/3175131098 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b4006ad0 con 0x7f59a8009a50 2023-10-09T06:30:51.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.428+0000 7f59bd59a700 1 -- 172.21.15.163:0/3175131098 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b4006c30 con 0x7f59a8009a50 2023-10-09T06:30:51.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.428+0000 7f59bd59a700 1 -- 172.21.15.163:0/3175131098 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b4006040 con 0x7f59a8009a50 2023-10-09T06:30:51.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.429+0000 7f59c4def700 1 -- 172.21.15.163:0/3175131098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59a8009a50 msgr2=0x7f59a8005250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.429+0000 7f59c4def700 1 --2- 172.21.15.163:0/3175131098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59a8009a50 0x7f59a8005250 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f59c00695b0 tx=0x7f59b400b420 comp rx=0 tx=0).stop 2023-10-09T06:30:51.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.429+0000 7f59c4def700 1 -- 172.21.15.163:0/3175131098 shutdown_connections 2023-10-09T06:30:51.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.429+0000 7f59c4def700 1 --2- 172.21.15.163:0/3175131098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59a8009a50 0x7f59a8005250 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.429+0000 7f59c4def700 1 -- 172.21.15.163:0/3175131098 >> 172.21.15.163:0/3175131098 conn(0x7f59c00cd000 msgr2=0x7f59c0068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:51.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.430+0000 7f59c4def700 1 -- 172.21.15.163:0/3175131098 shutdown_connections 2023-10-09T06:30:51.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.430+0000 7f59c4def700 1 -- 172.21.15.163:0/3175131098 wait complete. 2023-10-09T06:30:51.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.432+0000 7f59c4def700 1 Processor -- start 2023-10-09T06:30:51.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.432+0000 7f59c4def700 1 -- start start 2023-10-09T06:30:51.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.433+0000 7f59c4def700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59a8009a50 0x7f59c0086a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.433+0000 7f59c4def700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59c0167e20 0x7f59c0168290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.433+0000 7f59c4def700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59c00ae4a0 0x7f59c00ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.433+0000 7f59c4def700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c00aee80 con 0x7f59c00ae4a0 2023-10-09T06:30:51.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.433+0000 7f59c4def700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c0086f60 con 0x7f59a8009a50 2023-10-09T06:30:51.437 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.433+0000 7f59c4def700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c01687d0 con 0x7f59c0167e20 2023-10-09T06:30:51.437 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.433+0000 7f59bed9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59c00ae4a0 0x7f59c00ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.437 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.433+0000 7f59bed9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59c00ae4a0 0x7f59c00ae940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43962/0 (socket says 172.21.15.163:43962) 2023-10-09T06:30:51.437 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.433+0000 7f59bed9d700 1 -- 172.21.15.163:0/3827090069 learned_addr learned my addr 172.21.15.163:0/3827090069 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:51.437 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.433+0000 7f59bdd9b700 1 --2- 172.21.15.163:0/3827090069 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59c0167e20 0x7f59c0168290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.437 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.434+0000 7f59bed9d700 1 -- 172.21.15.163:0/3827090069 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59c0167e20 msgr2=0x7f59c0168290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.437 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.434+0000 7f59bed9d700 1 --2- 172.21.15.163:0/3827090069 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59c0167e20 0x7f59c0168290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.437 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.434+0000 7f59bed9d700 1 -- 172.21.15.163:0/3827090069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59a8009a50 msgr2=0x7f59c0086a20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-09T06:30:51.437 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.434+0000 7f59bed9d700 1 --2- 172.21.15.163:0/3827090069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59a8009a50 0x7f59c0086a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.437 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.434+0000 7f59bed9d700 1 -- 172.21.15.163:0/3827090069 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c0169610 con 0x7f59c00ae4a0 2023-10-09T06:30:51.437 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.434+0000 7f59bed9d700 1 --2- 172.21.15.163:0/3827090069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59c00ae4a0 0x7f59c00ae940 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f59c00b1670 tx=0x7f59b4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:51.438 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.435+0000 7f59ae7fc700 1 -- 172.21.15.163:0/3827090069 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b40072b0 con 0x7f59c00ae4a0 2023-10-09T06:30:51.438 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.435+0000 7f59ae7fc700 1 -- 172.21.15.163:0/3827090069 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b4007450 con 0x7f59c00ae4a0 2023-10-09T06:30:51.438 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.435+0000 7f59ae7fc700 1 -- 172.21.15.163:0/3827090069 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b4007810 con 0x7f59c00ae4a0 2023-10-09T06:30:51.438 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.435+0000 7f59c4def700 1 -- 172.21.15.163:0/3827090069 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59c01607f0 con 0x7f59c00ae4a0 2023-10-09T06:30:51.438 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.435+0000 7f59c4def700 1 -- 172.21.15.163:0/3827090069 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c0168ce0 con 0x7f59c00ae4a0 2023-10-09T06:30:51.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.439+0000 7f59ae7fc700 1 -- 172.21.15.163:0/3827090069 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f59b4023440 con 0x7f59c00ae4a0 2023-10-09T06:30:51.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.439+0000 7f59ae7fc700 1 --2- 172.21.15.163:0/3827090069 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f59a8068720 0x7f59a806abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.439+0000 7f59ae7fc700 1 -- 172.21.15.163:0/3827090069 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f59b4085ce0 con 0x7f59c00ae4a0 2023-10-09T06:30:51.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.441+0000 7f59c4def700 1 -- 172.21.15.163:0/3827090069 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59a00052e0 con 0x7f59c00ae4a0 2023-10-09T06:30:51.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.442+0000 7f59be59c700 1 --2- 172.21.15.163:0/3827090069 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f59a8068720 0x7f59a806abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.449+0000 7f59ae7fc700 1 -- 172.21.15.163:0/3827090069 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59b4052fe0 con 0x7f59c00ae4a0 2023-10-09T06:30:51.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.451+0000 7f59be59c700 1 --2- 172.21.15.163:0/3827090069 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f59a8068720 0x7f59a806abd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f59b0004c30 tx=0x7f59b0009170 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:51.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.480+0000 7fb0c662f700 1 Processor -- start 2023-10-09T06:30:51.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.481+0000 7fb0c662f700 1 -- start start 2023-10-09T06:30:51.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.482+0000 7fb0c662f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0c00691b0 0x7fb0c00b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.482+0000 7fb0c662f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb0c00b1fe0 0x7fb0c00ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.482+0000 7fb0c662f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0c015f260 con 0x7fb0c0158440 2023-10-09T06:30:51.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.482+0000 7fb0c662f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c00aef20 con 0x7fb0c00b1fe0 2023-10-09T06:30:51.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.482+0000 7fb0c662f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c00b2460 con 0x7fb0c00691b0 2023-10-09T06:30:51.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.482+0000 7fb0c4bcc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb0c00b1fe0 0x7fb0c00ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.482+0000 7fb0c4bcc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb0c00b1fe0 0x7fb0c00ab8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:50520/0 (socket says 172.21.15.163:50520) 2023-10-09T06:30:51.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.482+0000 7fb0c4bcc700 1 -- 172.21.15.163:0/625475198 learned_addr learned my addr 172.21.15.163:0/625475198 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:51.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.482+0000 7fb0bf7fe700 1 --2- 172.21.15.163:0/625475198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0c00691b0 0x7fb0c00b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.483+0000 7fb0c4bcc700 1 -- 172.21.15.163:0/625475198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0c00691b0 msgr2=0x7fb0c00b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:51.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.483+0000 7fb0c4bcc700 1 --2- 172.21.15.163:0/625475198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0c00691b0 0x7fb0c00b1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.483+0000 7fb0c4bcc700 1 -- 172.21.15.163:0/625475198 >> v1:172.21.15.181:6789/0 conn(0x7fb0c0158440 legacy=0x7fb0c0068aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:51.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.483+0000 7fb0c4bcc700 1 -- 172.21.15.163:0/625475198 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c015e380 con 0x7fb0c00b1fe0 2023-10-09T06:30:51.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.483+0000 7fb0c4bcc700 1 --2- 172.21.15.163:0/625475198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0c00b1fe0 0x7fb0c00ab8a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009f50 tx=0x7fb0b8009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8ead3007ddcfa1a server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:51.486 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.484+0000 7f93c28d1700 1 Processor -- start 2023-10-09T06:30:51.486 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.484+0000 7f93c28d1700 1 -- start start 2023-10-09T06:30:51.486 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.484+0000 7f4b019db700 1 Processor -- start 2023-10-09T06:30:51.486 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.484+0000 7f4b019db700 1 -- start start 2023-10-09T06:30:51.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.484+0000 7f5b8699e700 1 Processor -- start 2023-10-09T06:30:51.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.485+0000 7f5b8699e700 1 -- start start 2023-10-09T06:30:51.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.485+0000 7fdf28e2d700 1 Processor -- start 2023-10-09T06:30:51.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.485+0000 7f4b019db700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4afc06bcd0 0x7f4afc140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.485+0000 7f4b019db700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4afc06c0a0 0x7f4afc141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.485+0000 7f4b019db700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4afc15c2a0 con 0x7f4afc0c8e00 2023-10-09T06:30:51.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.485+0000 7f4b019db700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4afc15b690 con 0x7f4afc06bcd0 2023-10-09T06:30:51.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.485+0000 7f4b019db700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc15b970 con 0x7f4afc06c0a0 2023-10-09T06:30:51.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.486+0000 7f4afb7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4afc0c8e00 0x7f4afc0c9260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:47894/0 (socket says 172.21.15.163:47894) 2023-10-09T06:30:51.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.486+0000 7f4afb7fe700 1 -- 172.21.15.163:0/1564141342 learned_addr learned my addr 172.21.15.163:0/1564141342 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:51.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.485+0000 7f93c28d1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f93bc0b0010 0x7f93bc15d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.485+0000 7f93c28d1700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93bc0b0750 con 0x7f93bc15dd90 2023-10-09T06:30:51.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.485+0000 7f93c28d1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93bc0aca70 con 0x7f93bc0afc40 2023-10-09T06:30:51.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.485+0000 7f93c28d1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93bc0acd50 con 0x7f93bc0b0010 2023-10-09T06:30:51.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.486+0000 7f93bbfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f93bc0afc40 0x7f93bc0b1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:47880/0 (socket says 172.21.15.163:47880) 2023-10-09T06:30:51.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.486+0000 7f93bbfff700 1 -- 172.21.15.163:0/3712286596 learned_addr learned my addr 172.21.15.163:0/3712286596 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:51.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.486+0000 7f5b8699e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5b800691b0 0x7f5b800ac970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.486+0000 7f5b8699e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b80156050 0x7f5b801564b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.486+0000 7f5b8699e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b800b2490 con 0x7f5b800cba20 2023-10-09T06:30:51.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.486+0000 7f5b8699e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b800aceb0 con 0x7f5b800691b0 2023-10-09T06:30:51.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.486+0000 7f5b8699e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b800ac4e0 con 0x7f5b80156050 2023-10-09T06:30:51.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.485+0000 7fb0beffd700 1 -- 172.21.15.163:0/625475198 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b800e070 con 0x7fb0c00b1fe0 2023-10-09T06:30:51.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.485+0000 7fb0beffd700 1 -- 172.21.15.163:0/625475198 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b800ba30 con 0x7fb0c00b1fe0 2023-10-09T06:30:51.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.485+0000 7fb0beffd700 1 -- 172.21.15.163:0/625475198 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b80092e0 con 0x7fb0c00b1fe0 2023-10-09T06:30:51.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.486+0000 7f93bb7fe700 1 --2- 172.21.15.163:0/3712286596 >> v2:172.21.15.181:3300/0 conn(0x7f93bc0b0010 0x7f93bc15d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.486+0000 7f4afaffd700 1 --2- 172.21.15.163:0/1564141342 >> v2:172.21.15.163:3300/0 conn(0x7f4afc06bcd0 0x7f4afc140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.486+0000 7f4afa7fc700 1 --2- 172.21.15.163:0/1564141342 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4afc06c0a0 0x7f4afc141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.486+0000 7f5b7f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5b800691b0 0x7f5b800ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.486+0000 7f5b7f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5b800691b0 0x7f5b800ac970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:50552/0 (socket says 172.21.15.163:50552) 2023-10-09T06:30:51.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.486+0000 7f5b7f7fe700 1 -- 172.21.15.163:0/4265658866 learned_addr learned my addr 172.21.15.163:0/4265658866 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:51.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.486+0000 7f93bb7fe700 1 -- 172.21.15.163:0/3712286596 >> v1:172.21.15.181:6789/0 conn(0x7f93bc0afc40 legacy=0x7f93bc0b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.486+0000 7f93bb7fe700 1 -- 172.21.15.163:0/3712286596 >> v1:172.21.15.163:6789/0 conn(0x7f93bc15dd90 legacy=0x7f93bc0adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.486+0000 7f93bb7fe700 1 -- 172.21.15.163:0/3712286596 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93bc0ad030 con 0x7f93bc0b0010 2023-10-09T06:30:51.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.485+0000 7fdf28e2d700 1 -- start start 2023-10-09T06:30:51.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7fdf28e2d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf24155780 0x7fdf2406da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7fdf28e2d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdf2406dfd0 0x7fdf240c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7f93bb7fe700 1 --2- 172.21.15.163:0/3712286596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93bc0b0010 0x7f93bc15d850 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f93ac009860 tx=0x7f93ac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6bd17b715ce4888 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:51.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7f4afaffd700 1 -- 172.21.15.163:0/1564141342 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4afc06c0a0 msgr2=0x7f4afc141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:51.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7f4afaffd700 1 --2- 172.21.15.163:0/1564141342 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4afc06c0a0 0x7f4afc141330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7f4afaffd700 1 -- 172.21.15.163:0/1564141342 >> v1:172.21.15.181:6789/0 conn(0x7f4afc0c8e00 legacy=0x7f4afc0c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7f4afaffd700 1 -- 172.21.15.163:0/1564141342 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4afc15bc50 con 0x7f4afc06bcd0 2023-10-09T06:30:51.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7fdf28e2d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf2406e430 con 0x7fdf2406a730 2023-10-09T06:30:51.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7fdf28e2d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf24156e70 con 0x7fdf2406dfd0 2023-10-09T06:30:51.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7fdf28e2d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2406a2a0 con 0x7fdf24155780 2023-10-09T06:30:51.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7f5b7f7fe700 1 -- 172.21.15.163:0/4265658866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b80156050 msgr2=0x7f5b801564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:51.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7f5b7f7fe700 1 --2- 172.21.15.163:0/4265658866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b80156050 0x7f5b801564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7f5b7f7fe700 1 -- 172.21.15.163:0/4265658866 >> v1:172.21.15.181:6789/0 conn(0x7f5b800cba20 legacy=0x7f5b80068aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:51.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7f5b7f7fe700 1 -- 172.21.15.163:0/4265658866 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b80156cd0 con 0x7f5b800691b0 2023-10-09T06:30:51.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7fdf21d9b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fdf2406a730 0x7fdf240c8fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60924/0 (socket says 172.21.15.163:60924) 2023-10-09T06:30:51.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7fdf21d9b700 1 -- 172.21.15.163:0/3202511158 learned_addr learned my addr 172.21.15.163:0/3202511158 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:51.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7fb0c662f700 1 -- 172.21.15.163:0/625475198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0c00b1fe0 msgr2=0x7fb0c00ab8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7fb0c662f700 1 --2- 172.21.15.163:0/625475198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0c00b1fe0 0x7fb0c00ab8a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009f50 tx=0x7fb0b8009840 comp rx=0 tx=0).stop 2023-10-09T06:30:51.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7fb0c662f700 1 -- 172.21.15.163:0/625475198 shutdown_connections 2023-10-09T06:30:51.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7fb0c662f700 1 --2- 172.21.15.163:0/625475198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0c00b1fe0 0x7fb0c00ab8a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7fb0c662f700 1 --2- 172.21.15.163:0/625475198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0c00691b0 0x7fb0c00b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7fb0c662f700 1 -- 172.21.15.163:0/625475198 >> 172.21.15.163:0/625475198 conn(0x7fb0c00cd000 msgr2=0x7fb0c0068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:51.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7f4afaffd700 1 --2- 172.21.15.163:0/1564141342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4afc06bcd0 0x7f4afc140df0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4af00098e0 tx=0x7f4af00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdca1762c6653b7c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:51.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7f5b7f7fe700 1 --2- 172.21.15.163:0/4265658866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b800691b0 0x7f5b800ac970 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5b740089b0 tx=0x7f5b740082d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=249e1228f8fce3b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:51.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7fdf22d9d700 1 --2- 172.21.15.163:0/3202511158 >> v2:172.21.15.163:3300/0 conn(0x7fdf2406dfd0 0x7fdf240c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7fb0c662f700 1 -- 172.21.15.163:0/625475198 shutdown_connections 2023-10-09T06:30:51.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7f5b7effd700 1 -- 172.21.15.163:0/4265658866 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b74010070 con 0x7f5b800691b0 2023-10-09T06:30:51.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7f5b7effd700 1 -- 172.21.15.163:0/4265658866 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7400daf0 con 0x7f5b800691b0 2023-10-09T06:30:51.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7f5b7effd700 1 -- 172.21.15.163:0/4265658866 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7400de90 con 0x7f5b800691b0 2023-10-09T06:30:51.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7fdf22d9d700 1 -- 172.21.15.163:0/3202511158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf24155780 msgr2=0x7fdf2406da90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-09T06:30:51.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7f4af9ffb700 1 -- 172.21.15.163:0/1564141342 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af0009e80 con 0x7f4afc06bcd0 2023-10-09T06:30:51.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.488+0000 7f4af9ffb700 1 -- 172.21.15.163:0/1564141342 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af000b860 con 0x7f4afc06bcd0 2023-10-09T06:30:51.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.488+0000 7f4af9ffb700 1 -- 172.21.15.163:0/1564141342 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af000bbe0 con 0x7f4afc06bcd0 2023-10-09T06:30:51.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7fb0c662f700 1 -- 172.21.15.163:0/625475198 wait complete. 2023-10-09T06:30:51.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.488+0000 7f5b8699e700 1 -- 172.21.15.163:0/4265658866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b800691b0 msgr2=0x7f5b800ac970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.488+0000 7f5b8699e700 1 --2- 172.21.15.163:0/4265658866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b800691b0 0x7f5b800ac970 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5b740089b0 tx=0x7f5b740082d0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.488+0000 7f93baffd700 1 -- 172.21.15.163:0/3712286596 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac011630 con 0x7f93bc0b0010 2023-10-09T06:30:51.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.488+0000 7f93baffd700 1 -- 172.21.15.163:0/3712286596 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93ac011790 con 0x7f93bc0b0010 2023-10-09T06:30:51.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.488+0000 7f93baffd700 1 -- 172.21.15.163:0/3712286596 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac011b50 con 0x7f93bc0b0010 2023-10-09T06:30:51.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.488+0000 7f4b019db700 1 -- 172.21.15.163:0/1564141342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4afc06bcd0 msgr2=0x7f4afc140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.488+0000 7f4b019db700 1 --2- 172.21.15.163:0/1564141342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4afc06bcd0 0x7f4afc140df0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4af00098e0 tx=0x7f4af00091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.487+0000 7fdf22d9d700 1 --2- 172.21.15.163:0/3202511158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf24155780 0x7fdf2406da90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.488+0000 7fdf22d9d700 1 -- 172.21.15.163:0/3202511158 >> v1:172.21.15.163:6789/0 conn(0x7fdf2406a730 legacy=0x7fdf240c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:51.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.488+0000 7fdf22d9d700 1 -- 172.21.15.163:0/3202511158 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf24156b90 con 0x7fdf2406dfd0 2023-10-09T06:30:51.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.489+0000 7fdf22d9d700 1 --2- 172.21.15.163:0/3202511158 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf2406dfd0 0x7fdf240c96e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009d50 tx=0x7fdf1c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df21edd3dc85b0a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:51.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.489+0000 7fb0c662f700 1 Processor -- start 2023-10-09T06:30:51.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.489+0000 7fdf2159a700 1 -- 172.21.15.163:0/3202511158 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c00bb20 con 0x7fdf2406dfd0 2023-10-09T06:30:51.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.489+0000 7fdf2159a700 1 -- 172.21.15.163:0/3202511158 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1c00bca0 con 0x7fdf2406dfd0 2023-10-09T06:30:51.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.489+0000 7fdf2159a700 1 -- 172.21.15.163:0/3202511158 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c0106d0 con 0x7fdf2406dfd0 2023-10-09T06:30:51.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.489+0000 7f93c28d1700 1 -- 172.21.15.163:0/3712286596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93bc0b0010 msgr2=0x7f93bc15d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.489+0000 7f93c28d1700 1 --2- 172.21.15.163:0/3712286596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93bc0b0010 0x7f93bc15d850 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f93ac009860 tx=0x7f93ac0091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.489+0000 7f5b8699e700 1 -- 172.21.15.163:0/4265658866 shutdown_connections 2023-10-09T06:30:51.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.489+0000 7f5b8699e700 1 --2- 172.21.15.163:0/4265658866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b80156050 0x7f5b801564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.489+0000 7f5b8699e700 1 --2- 172.21.15.163:0/4265658866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b800691b0 0x7f5b800ac970 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.489+0000 7f5b8699e700 1 -- 172.21.15.163:0/4265658866 >> 172.21.15.163:0/4265658866 conn(0x7f5b800cd000 msgr2=0x7f5b80068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:51.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.489+0000 7f93c28d1700 1 -- 172.21.15.163:0/3712286596 shutdown_connections 2023-10-09T06:30:51.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.489+0000 7f93c28d1700 1 --2- 172.21.15.163:0/3712286596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93bc0b0010 0x7f93bc15d850 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.489+0000 7f93c28d1700 1 -- 172.21.15.163:0/3712286596 >> 172.21.15.163:0/3712286596 conn(0x7f93bc06a320 msgr2=0x7f93bc0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:51.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.489+0000 7fbe1432e700 1 Processor -- start 2023-10-09T06:30:51.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.489+0000 7fbe1432e700 1 -- start start 2023-10-09T06:30:51.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.489+0000 7f5b8699e700 1 -- 172.21.15.163:0/4265658866 shutdown_connections 2023-10-09T06:30:51.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.489+0000 7f4b019db700 1 -- 172.21.15.163:0/1564141342 shutdown_connections 2023-10-09T06:30:51.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.489+0000 7f4b019db700 1 --2- 172.21.15.163:0/1564141342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4afc06bcd0 0x7f4afc140df0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.489+0000 7f4b019db700 1 --2- 172.21.15.163:0/1564141342 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4afc06c0a0 0x7f4afc141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.489+0000 7f4b019db700 1 -- 172.21.15.163:0/1564141342 >> 172.21.15.163:0/1564141342 conn(0x7f4afc0ce600 msgr2=0x7f4afc0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:51.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.490+0000 7f4b019db700 1 -- 172.21.15.163:0/1564141342 shutdown_connections 2023-10-09T06:30:51.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.490+0000 7fdf28e2d700 1 -- 172.21.15.163:0/3202511158 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf2406dfd0 msgr2=0x7fdf240c96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.490+0000 7fdf28e2d700 1 --2- 172.21.15.163:0/3202511158 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf2406dfd0 0x7fdf240c96e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009d50 tx=0x7fdf1c009780 comp rx=0 tx=0).stop 2023-10-09T06:30:51.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.490+0000 7fdf28e2d700 1 -- 172.21.15.163:0/3202511158 shutdown_connections 2023-10-09T06:30:51.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.490+0000 7fdf28e2d700 1 --2- 172.21.15.163:0/3202511158 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf2406dfd0 0x7fdf240c96e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.490+0000 7fdf28e2d700 1 --2- 172.21.15.163:0/3202511158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf24155780 0x7fdf2406da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.490+0000 7fdf28e2d700 1 -- 172.21.15.163:0/3202511158 >> 172.21.15.163:0/3202511158 conn(0x7fdf24068390 msgr2=0x7fdf24068790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:51.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.489+0000 7fb0c662f700 1 -- start start 2023-10-09T06:30:51.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.490+0000 7fb0c662f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0c00691b0 0x7fb0c0157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.490+0000 7fb0c662f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0c00b1fe0 0x7fb0c0143df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.490+0000 7fb0c662f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0c0158440 0x7fb0c0161930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.490+0000 7fb0c662f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c006f780 con 0x7fb0c00691b0 2023-10-09T06:30:51.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.490+0000 7fb0c662f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c0147380 con 0x7fb0c00b1fe0 2023-10-09T06:30:51.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.490+0000 7fb0c662f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c0147940 con 0x7fb0c0158440 2023-10-09T06:30:51.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.490+0000 7f93c28d1700 1 -- 172.21.15.163:0/3712286596 shutdown_connections 2023-10-09T06:30:51.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.491+0000 7f93c28d1700 1 -- 172.21.15.163:0/3712286596 wait complete. 2023-10-09T06:30:51.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.491+0000 7fb0bffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0c00691b0 0x7fb0c0157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.491+0000 7fb0bffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0c00691b0 0x7fb0c0157bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43994/0 (socket says 172.21.15.163:43994) 2023-10-09T06:30:51.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.491+0000 7fb0bffff700 1 -- 172.21.15.163:0/4195622518 learned_addr learned my addr 172.21.15.163:0/4195622518 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:51.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.489+0000 7f5b8699e700 1 -- 172.21.15.163:0/4265658866 wait complete. 2023-10-09T06:30:51.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.491+0000 7fbe1432e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbe0c0691b0 0x7fbe0c166710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.491+0000 7fbe1432e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbe0c0afe00 0x7fbe0c0ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.491+0000 7fbe1432e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe0c0ad400 con 0x7fbe0c0ac170 2023-10-09T06:30:51.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.491+0000 7fbe1432e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe0c087e70 con 0x7fbe0c0691b0 2023-10-09T06:30:51.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.491+0000 7fbe1432e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe0c0ac8f0 con 0x7fbe0c0afe00 2023-10-09T06:30:51.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.491+0000 7f2ea6896700 1 Processor -- start 2023-10-09T06:30:51.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.491+0000 7f5b8699e700 1 Processor -- start 2023-10-09T06:30:51.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.491+0000 7fbe120ca700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbe0c0ac170 0x7fbe0c068aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:47922/0 (socket says 172.21.15.163:47922) 2023-10-09T06:30:51.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.491+0000 7fbe120ca700 1 -- 172.21.15.163:0/1461175576 learned_addr learned my addr 172.21.15.163:0/1461175576 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:51.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.491+0000 7fb0c4bcc700 1 --2- 172.21.15.163:0/4195622518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0c0158440 0x7fb0c0161930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.492+0000 7fb0bf7fe700 1 --2- 172.21.15.163:0/4195622518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0c00b1fe0 0x7fb0c0143df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.492+0000 7fb0bffff700 1 -- 172.21.15.163:0/4195622518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0c0158440 msgr2=0x7fb0c0161930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.492+0000 7fb0bffff700 1 --2- 172.21.15.163:0/4195622518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0c0158440 0x7fb0c0161930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.492+0000 7fb0bffff700 1 -- 172.21.15.163:0/4195622518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0c00b1fe0 msgr2=0x7fb0c0143df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.492+0000 7fb0bffff700 1 --2- 172.21.15.163:0/4195622518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0c00b1fe0 0x7fb0c0143df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.492+0000 7fb0bffff700 1 -- 172.21.15.163:0/4195622518 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b8009ae0 con 0x7fb0c00691b0 2023-10-09T06:30:51.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.492+0000 7f5b8699e700 1 -- start start 2023-10-09T06:30:51.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.492+0000 7fbe128cb700 1 --2- 172.21.15.163:0/1461175576 >> v2:172.21.15.181:3300/0 conn(0x7fbe0c0afe00 0x7fbe0c0ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.492+0000 7fbe128cb700 1 -- 172.21.15.163:0/1461175576 >> v2:172.21.15.163:3300/0 conn(0x7fbe0c0691b0 msgr2=0x7fbe0c166710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:51.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.492+0000 7fbe128cb700 1 --2- 172.21.15.163:0/1461175576 >> v2:172.21.15.163:3300/0 conn(0x7fbe0c0691b0 0x7fbe0c166710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.492+0000 7fbe128cb700 1 -- 172.21.15.163:0/1461175576 >> v1:172.21.15.181:6789/0 conn(0x7fbe0c0ac170 legacy=0x7fbe0c068aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.492+0000 7fbe128cb700 1 -- 172.21.15.163:0/1461175576 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe0c0acbd0 con 0x7fbe0c0afe00 2023-10-09T06:30:51.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.492+0000 7fbe128cb700 1 --2- 172.21.15.163:0/1461175576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe0c0afe00 0x7fbe0c0ab8e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c16b310 tx=0x7fbdfc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7833f763569f30ce server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:51.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.490+0000 7f4b019db700 1 -- 172.21.15.163:0/1564141342 wait complete. 2023-10-09T06:30:51.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.492+0000 7f4b019db700 1 Processor -- start 2023-10-09T06:30:51.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.491+0000 7f2ea6896700 1 -- start start 2023-10-09T06:30:51.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.493+0000 7f2ea6896700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2ea0156760 0x7f2ea00ac970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.493+0000 7f2ea6896700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea00b2490 con 0x7f2ea00cba20 2023-10-09T06:30:51.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.493+0000 7f2ea6896700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea00ac4e0 con 0x7f2ea00691b0 2023-10-09T06:30:51.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.493+0000 7f2ea6896700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea00aceb0 con 0x7f2ea0156760 2023-10-09T06:30:51.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.493+0000 7f2e9f7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f2ea00691b0 0x7f2ea0156050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:47938/0 (socket says 172.21.15.163:47938) 2023-10-09T06:30:51.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.493+0000 7f2e9f7fe700 1 -- 172.21.15.163:0/2550355421 learned_addr learned my addr 172.21.15.163:0/2550355421 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:51.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.493+0000 7f5b8699e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b800691b0 0x7f5b8016c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.493+0000 7f5b8699e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b800cba20 0x7f5b80145610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.493+0000 7f5b8699e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b80156050 0x7f5b80145b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.493+0000 7f5b8699e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8015a9a0 con 0x7f5b80156050 2023-10-09T06:30:51.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.493+0000 7f5b8699e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8015a6a0 con 0x7f5b800cba20 2023-10-09T06:30:51.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.493+0000 7f5b8699e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8016ce10 con 0x7f5b800691b0 2023-10-09T06:30:51.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.492+0000 7fb0bffff700 1 --2- 172.21.15.163:0/4195622518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0c00691b0 0x7fb0c0157bf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400b5e0 tx=0x7fb0b4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:51.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.492+0000 7fb0a7fff700 1 -- 172.21.15.163:0/4195622518 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b400f510 con 0x7fb0c00691b0 2023-10-09T06:30:51.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.492+0000 7fb0c662f700 1 -- 172.21.15.163:0/4195622518 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c0144480 con 0x7fb0c00691b0 2023-10-09T06:30:51.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.492+0000 7fb0c662f700 1 -- 172.21.15.163:0/4195622518 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c01475d0 con 0x7fb0c00691b0 2023-10-09T06:30:51.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.492+0000 7fb0a7fff700 1 -- 172.21.15.163:0/4195622518 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b400f690 con 0x7fb0c00691b0 2023-10-09T06:30:51.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.492+0000 7fb0a7fff700 1 -- 172.21.15.163:0/4195622518 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b400fa50 con 0x7fb0c00691b0 2023-10-09T06:30:51.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.492+0000 7f4b019db700 1 -- start start 2023-10-09T06:30:51.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.493+0000 7fbe110c8700 1 -- 172.21.15.163:0/1461175576 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc00b740 con 0x7fbe0c0afe00 2023-10-09T06:30:51.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.493+0000 7fbe110c8700 1 -- 172.21.15.163:0/1461175576 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdfc00b8a0 con 0x7fbe0c0afe00 2023-10-09T06:30:51.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.493+0000 7fbe110c8700 1 -- 172.21.15.163:0/1461175576 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc00bc40 con 0x7fbe0c0afe00 2023-10-09T06:30:51.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.493+0000 7f5b84f3b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b80156050 0x7f5b80145b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.493+0000 7f5b84f3b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b80156050 0x7f5b80145b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44038/0 (socket says 172.21.15.163:44038) 2023-10-09T06:30:51.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.493+0000 7f5b84f3b700 1 -- 172.21.15.163:0/3856040502 learned_addr learned my addr 172.21.15.163:0/3856040502 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:51.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.493+0000 7f93c28d1700 1 Processor -- start 2023-10-09T06:30:51.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.493+0000 7f5b84f3b700 1 -- 172.21.15.163:0/3856040502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b800691b0 msgr2=0x7f5b8016c400 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-09T06:30:51.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.493+0000 7f5b84f3b700 1 --2- 172.21.15.163:0/3856040502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b800691b0 0x7f5b8016c400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.493+0000 7f5b84f3b700 1 -- 172.21.15.163:0/3856040502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b800cba20 msgr2=0x7f5b80145610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7f2ea4e33700 1 --2- 172.21.15.163:0/2550355421 >> v2:172.21.15.163:3300/0 conn(0x7f2ea0156760 0x7f2ea00ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7f5b84f3b700 1 --2- 172.21.15.163:0/3856040502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b800cba20 0x7f5b80145610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7f5b84f3b700 1 -- 172.21.15.163:0/3856040502 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b74008570 con 0x7f5b80156050 2023-10-09T06:30:51.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7fdf28e2d700 1 -- 172.21.15.163:0/3202511158 shutdown_connections 2023-10-09T06:30:51.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.493+0000 7f4b019db700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4afc06bcd0 0x7f4afc136db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.493+0000 7f4b019db700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4afc06c0a0 0x7f4afc1309c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.493+0000 7f4b019db700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4afc0c8e00 0x7f4afc133ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.493+0000 7f4b019db700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc134ec0 con 0x7f4afc0c8e00 2023-10-09T06:30:51.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.493+0000 7f4b019db700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc135180 con 0x7f4afc06c0a0 2023-10-09T06:30:51.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.493+0000 7f4b019db700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc135460 con 0x7f4afc06bcd0 2023-10-09T06:30:51.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7f4afaffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4afc06bcd0 0x7f4afc136db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7f4afaffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4afc06bcd0 0x7f4afc136db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:44852/0 (socket says 172.21.15.163:44852) 2023-10-09T06:30:51.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7f4afaffd700 1 -- 172.21.15.163:0/1938007466 learned_addr learned my addr 172.21.15.163:0/1938007466 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:51.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7f4afa7fc700 1 --2- 172.21.15.163:0/1938007466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4afc06c0a0 0x7f4afc1309c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7fdf28e2d700 1 -- 172.21.15.163:0/3202511158 wait complete. 2023-10-09T06:30:51.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7fb0c662f700 1 -- 172.21.15.163:0/4195622518 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0a00052b0 con 0x7fb0c00691b0 2023-10-09T06:30:51.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7f2ea4e33700 1 -- 172.21.15.163:0/2550355421 >> v1:172.21.15.181:6789/0 conn(0x7f2ea00691b0 legacy=0x7f2ea0156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7f2ea4e33700 1 -- 172.21.15.163:0/2550355421 >> v1:172.21.15.163:6789/0 conn(0x7f2ea00cba20 legacy=0x7f2ea0068aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7f2ea4e33700 1 -- 172.21.15.163:0/2550355421 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea0156cd0 con 0x7f2ea0156760 2023-10-09T06:30:51.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7f4afb7fe700 1 --2- 172.21.15.163:0/1938007466 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4afc0c8e00 0x7f4afc133ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7f4afa7fc700 1 -- 172.21.15.163:0/1938007466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4afc06bcd0 msgr2=0x7f4afc136db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7fbe1432e700 1 -- 172.21.15.163:0/1461175576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe0c0afe00 msgr2=0x7fbe0c0ab8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7fbe1432e700 1 --2- 172.21.15.163:0/1461175576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe0c0afe00 0x7fbe0c0ab8e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c16b310 tx=0x7fbdfc0091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7f4afa7fc700 1 --2- 172.21.15.163:0/1938007466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4afc06bcd0 0x7f4afc136db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7f4afa7fc700 1 -- 172.21.15.163:0/1938007466 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4afc0c8e00 msgr2=0x7f4afc133ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7f4afa7fc700 1 --2- 172.21.15.163:0/1938007466 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4afc0c8e00 0x7f4afc133ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7f4afa7fc700 1 -- 172.21.15.163:0/1938007466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af0009420 con 0x7f4afc06c0a0 2023-10-09T06:30:51.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7f5b84f3b700 1 --2- 172.21.15.163:0/3856040502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b80156050 0x7f5b80145b50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000bc30 tx=0x7f5b70009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:51.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7f4afa7fc700 1 --2- 172.21.15.163:0/1938007466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4afc06c0a0 0x7f4afc1309c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4af400cf00 tx=0x7f4af40077f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:51.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7fbe1432e700 1 -- 172.21.15.163:0/1461175576 shutdown_connections 2023-10-09T06:30:51.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7fbe1432e700 1 --2- 172.21.15.163:0/1461175576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe0c0afe00 0x7fbe0c0ab8e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7fbe1432e700 1 --2- 172.21.15.163:0/1461175576 >> v2:172.21.15.163:3300/0 conn(0x7fbe0c0691b0 0x7fbe0c166710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7fbe1432e700 1 -- 172.21.15.163:0/1461175576 >> 172.21.15.163:0/1461175576 conn(0x7fbe0c0cd000 msgr2=0x7fbe0c068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:51.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7fbe1432e700 1 -- 172.21.15.163:0/1461175576 shutdown_connections 2023-10-09T06:30:51.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7f2ea4e33700 1 --2- 172.21.15.163:0/2550355421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ea0156760 0x7f2ea00ac970 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2e9800af70 tx=0x7f2e9800a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb430b47f053318 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:51.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7fbe1432e700 1 -- 172.21.15.163:0/1461175576 wait complete. 2023-10-09T06:30:51.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.494+0000 7f93c28d1700 1 -- start start 2023-10-09T06:30:51.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.495+0000 7f93c28d1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93bc0afc40 0x7f93bc14ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.495+0000 7f93c28d1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93bc0b0010 0x7f93bc161740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.495+0000 7fdf28e2d700 1 Processor -- start 2023-10-09T06:30:51.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.495+0000 7f93c28d1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93bc15dd90 0x7f93bc06e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.495+0000 7f93c28d1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc16f580 con 0x7f93bc15dd90 2023-10-09T06:30:51.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.495+0000 7f93c28d1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc16f860 con 0x7f93bc0b0010 2023-10-09T06:30:51.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.495+0000 7f93c28d1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc16fb40 con 0x7f93bc0afc40 2023-10-09T06:30:51.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.495+0000 7f93bbfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93bc0afc40 0x7f93bc14ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.495+0000 7f93c0e6e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93bc15dd90 0x7f93bc06e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.495+0000 7f93bbfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93bc0afc40 0x7f93bc14ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:44862/0 (socket says 172.21.15.163:44862) 2023-10-09T06:30:51.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.495+0000 7f93bbfff700 1 -- 172.21.15.163:0/775666794 learned_addr learned my addr 172.21.15.163:0/775666794 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:51.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.495+0000 7f93bb7fe700 1 --2- 172.21.15.163:0/775666794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93bc0b0010 0x7f93bc161740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.496+0000 7f4ae2ffd700 1 -- 172.21.15.163:0/1938007466 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af400ad70 con 0x7f4afc06c0a0 2023-10-09T06:30:51.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.496+0000 7f93bbfff700 1 -- 172.21.15.163:0/775666794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93bc0b0010 msgr2=0x7f93bc161740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.496+0000 7f93bbfff700 1 --2- 172.21.15.163:0/775666794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93bc0b0010 0x7f93bc161740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.496+0000 7f93bbfff700 1 -- 172.21.15.163:0/775666794 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93bc15dd90 msgr2=0x7f93bc06e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.496+0000 7f93bbfff700 1 --2- 172.21.15.163:0/775666794 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93bc15dd90 0x7f93bc06e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.496+0000 7f93bbfff700 1 -- 172.21.15.163:0/775666794 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ac009420 con 0x7f93bc0afc40 2023-10-09T06:30:51.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.496+0000 7f4ae2ffd700 1 -- 172.21.15.163:0/1938007466 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af4016430 con 0x7f4afc06c0a0 2023-10-09T06:30:51.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.496+0000 7f4ae2ffd700 1 -- 172.21.15.163:0/1938007466 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af40167f0 con 0x7f4afc06c0a0 2023-10-09T06:30:51.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.496+0000 7fbe1432e700 1 Processor -- start 2023-10-09T06:30:51.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.496+0000 7f93bbfff700 1 --2- 172.21.15.163:0/775666794 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93bc0afc40 0x7f93bc14ea30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f93b000c810 tx=0x7f93b0007ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:51.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.496+0000 7f93a3fff700 1 -- 172.21.15.163:0/775666794 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0005b30 con 0x7f93bc0afc40 2023-10-09T06:30:51.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.496+0000 7f93a3fff700 1 -- 172.21.15.163:0/775666794 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b0005cb0 con 0x7f93bc0afc40 2023-10-09T06:30:51.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.496+0000 7f93c28d1700 1 -- 172.21.15.163:0/775666794 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93bc16fe20 con 0x7f93bc0afc40 2023-10-09T06:30:51.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.496+0000 7f93a3fff700 1 -- 172.21.15.163:0/775666794 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0007320 con 0x7f93bc0afc40 2023-10-09T06:30:51.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.496+0000 7f93c28d1700 1 -- 172.21.15.163:0/775666794 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93bc1703e0 con 0x7f93bc0afc40 2023-10-09T06:30:51.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.496+0000 7f4b019db700 1 -- 172.21.15.163:0/1938007466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4afc135740 con 0x7f4afc06c0a0 2023-10-09T06:30:51.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.497+0000 7fbe1432e700 1 -- start start 2023-10-09T06:30:51.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.495+0000 7fdf28e2d700 1 -- start start 2023-10-09T06:30:51.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.496+0000 7fdf28e2d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf2406dfd0 0x7fdf24149370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.496+0000 7fdf28e2d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf24155780 0x7fdf241450a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.496+0000 7fdf28e2d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf2414cef0 0x7fdf2414d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.496+0000 7fdf28e2d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2414c620 con 0x7fdf2406dfd0 2023-10-09T06:30:51.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.496+0000 7fdf28e2d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf24146ce0 con 0x7fdf24155780 2023-10-09T06:30:51.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.496+0000 7fdf28e2d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2414c340 con 0x7fdf2414cef0 2023-10-09T06:30:51.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.497+0000 7fdf21d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf24155780 0x7fdf241450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.497+0000 7fdf21d9b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf24155780 0x7fdf241450a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:50600/0 (socket says 172.21.15.163:50600) 2023-10-09T06:30:51.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.497+0000 7fdf21d9b700 1 -- 172.21.15.163:0/1804202013 learned_addr learned my addr 172.21.15.163:0/1804202013 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:51.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.497+0000 7fdf2259c700 1 --2- 172.21.15.163:0/1804202013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf2406dfd0 0x7fdf24149370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.497+0000 7f4b019db700 1 -- 172.21.15.163:0/1938007466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4afc130f00 con 0x7f4afc06c0a0 2023-10-09T06:30:51.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.498+0000 7f4b019db700 1 -- 172.21.15.163:0/1938007466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4afc135c40 con 0x7f4afc06c0a0 2023-10-09T06:30:51.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.498+0000 7fbe1432e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe0c0691b0 0x7fbe0c167190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.498+0000 7fbe1432e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe0c0ac170 0x7fbe0c1537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.498+0000 7fbe1432e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe0c0afe00 0x7fbe0c086930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.498+0000 7fbe1432e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0c168380 con 0x7fbe0c0ac170 2023-10-09T06:30:51.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.498+0000 7fbe1432e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0c167770 con 0x7fbe0c0691b0 2023-10-09T06:30:51.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.498+0000 7fbe1432e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0c167a50 con 0x7fbe0c0afe00 2023-10-09T06:30:51.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.498+0000 7fdf22d9d700 1 --2- 172.21.15.163:0/1804202013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf2414cef0 0x7fdf2414d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.498+0000 7fdf2259c700 1 -- 172.21.15.163:0/1804202013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf2414cef0 msgr2=0x7fdf2414d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.498+0000 7fdf2259c700 1 --2- 172.21.15.163:0/1804202013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf2414cef0 0x7fdf2414d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.498+0000 7fdf2259c700 1 -- 172.21.15.163:0/1804202013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf24155780 msgr2=0x7fdf241450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.498+0000 7fdf2259c700 1 --2- 172.21.15.163:0/1804202013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf24155780 0x7fdf241450a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.498+0000 7fdf2259c700 1 -- 172.21.15.163:0/1804202013 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1c00d040 con 0x7fdf2406dfd0 2023-10-09T06:30:51.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.498+0000 7fdf22d9d700 1 --2- 172.21.15.163:0/1804202013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf2414cef0 0x7fdf2414d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:30:51.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.498+0000 7f2e9effd700 1 -- 172.21.15.163:0/2550355421 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9800f070 con 0x7f2ea0156760 2023-10-09T06:30:51.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.498+0000 7f93a3fff700 1 -- 172.21.15.163:0/775666794 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f93b001b3b0 con 0x7f93bc0afc40 2023-10-09T06:30:51.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.498+0000 7f93a3fff700 1 --2- 172.21.15.163:0/775666794 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f93a8066f30 0x7f93a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.499+0000 7f93a3fff700 1 -- 172.21.15.163:0/775666794 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f93b00839e0 con 0x7f93bc0afc40 2023-10-09T06:30:51.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.498+0000 7fbe118c9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe0c0ac170 0x7fbe0c1537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.498+0000 7fbe118c9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe0c0ac170 0x7fbe0c1537c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44074/0 (socket says 172.21.15.163:44074) 2023-10-09T06:30:51.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.498+0000 7fbe118c9700 1 -- 172.21.15.163:0/1161603513 learned_addr learned my addr 172.21.15.163:0/1161603513 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:51.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.498+0000 7fbe120ca700 1 --2- 172.21.15.163:0/1161603513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe0c0691b0 0x7fbe0c167190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.498+0000 7fbe118c9700 1 -- 172.21.15.163:0/1161603513 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe0c0afe00 msgr2=0x7fbe0c086930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-09T06:30:51.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.498+0000 7fbe118c9700 1 --2- 172.21.15.163:0/1161603513 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe0c0afe00 0x7fbe0c086930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.498+0000 7fbe118c9700 1 -- 172.21.15.163:0/1161603513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe0c0691b0 msgr2=0x7fbe0c167190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.498+0000 7fbe118c9700 1 --2- 172.21.15.163:0/1161603513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe0c0691b0 0x7fbe0c167190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.498+0000 7fbe118c9700 1 -- 172.21.15.163:0/1161603513 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdfc009420 con 0x7fbe0c0ac170 2023-10-09T06:30:51.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.499+0000 7fbe118c9700 1 --2- 172.21.15.163:0/1161603513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe0c0ac170 0x7fbe0c1537c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009ea0 tx=0x7fbe08009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:51.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.499+0000 7f93bb7fe700 1 --2- 172.21.15.163:0/775666794 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f93a8066f30 0x7f93a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.499+0000 7fb0a7fff700 1 -- 172.21.15.163:0/4195622518 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb0b4024430 con 0x7fb0c00691b0 2023-10-09T06:30:51.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.499+0000 7fdf2259c700 1 --2- 172.21.15.163:0/1804202013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf2406dfd0 0x7fdf24149370 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800b510 tx=0x7fdf18009060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:51.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.499+0000 7f93bb7fe700 1 --2- 172.21.15.163:0/775666794 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f93a8066f30 0x7f93a80693e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0091e0 tx=0x7f93ac016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:51.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.500+0000 7fb0a7fff700 1 --2- 172.21.15.163:0/4195622518 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb0ac066f30 0x7fb0ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.500+0000 7f4ae2ffd700 1 -- 172.21.15.163:0/1938007466 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4af4003020 con 0x7f4afc06c0a0 2023-10-09T06:30:51.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.500+0000 7fb0a7fff700 1 -- 172.21.15.163:0/4195622518 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb0b4084b20 con 0x7fb0c00691b0 2023-10-09T06:30:51.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.500+0000 7fb0bf7fe700 1 --2- 172.21.15.163:0/4195622518 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb0ac066f30 0x7fb0ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.500+0000 7f4ae2ffd700 1 --2- 172.21.15.163:0/1938007466 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4ae8067000 0x7f4ae80694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.500+0000 7f4ae2ffd700 1 -- 172.21.15.163:0/1938007466 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4af4012070 con 0x7f4afc06c0a0 2023-10-09T06:30:51.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.500+0000 7f4afaffd700 1 --2- 172.21.15.163:0/1938007466 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4ae8067000 0x7f4ae80694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.500+0000 7fb0bf7fe700 1 --2- 172.21.15.163:0/4195622518 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb0ac066f30 0x7fb0ac0693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0006fd0 tx=0x7fb0b0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:51.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.501+0000 7f4afaffd700 1 --2- 172.21.15.163:0/1938007466 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4ae8067000 0x7f4ae80694b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4af00091e0 tx=0x7f4af0017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:51.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.498+0000 7f5b67fff700 1 -- 172.21.15.163:0/3856040502 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b700092e0 con 0x7f5b80156050 2023-10-09T06:30:51.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.498+0000 7f5b8699e700 1 -- 172.21.15.163:0/3856040502 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b8016d0f0 con 0x7f5b80156050 2023-10-09T06:30:51.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.498+0000 7f5b8699e700 1 -- 172.21.15.163:0/3856040502 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b8015f260 con 0x7f5b80156050 2023-10-09T06:30:51.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.500+0000 7f5b8699e700 1 -- 172.21.15.163:0/3856040502 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b8015ab50 con 0x7f5b80156050 2023-10-09T06:30:51.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.502+0000 7f93c28d1700 1 -- 172.21.15.163:0/775666794 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93bc04c3c0 con 0x7f93bc0afc40 2023-10-09T06:30:51.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.503+0000 7f4ae2ffd700 1 -- 172.21.15.163:0/1938007466 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4af4052c30 con 0x7f4afc06c0a0 2023-10-09T06:30:51.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.503+0000 7fdf127fc700 1 -- 172.21.15.163:0/1804202013 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1800f510 con 0x7fdf2406dfd0 2023-10-09T06:30:51.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.504+0000 7fdf28e2d700 1 -- 172.21.15.163:0/1804202013 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf24146690 con 0x7fdf2406dfd0 2023-10-09T06:30:51.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.504+0000 7fdf28e2d700 1 -- 172.21.15.163:0/1804202013 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf241455e0 con 0x7fdf2406dfd0 2023-10-09T06:30:51.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.504+0000 7fdf127fc700 1 -- 172.21.15.163:0/1804202013 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1800f880 con 0x7fdf2406dfd0 2023-10-09T06:30:51.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.504+0000 7fdf127fc700 1 -- 172.21.15.163:0/1804202013 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1800f9f0 con 0x7fdf2406dfd0 2023-10-09T06:30:51.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.505+0000 7fdf28e2d700 1 -- 172.21.15.163:0/1804202013 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf241464f0 con 0x7fdf2406dfd0 2023-10-09T06:30:51.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.504+0000 7fb0a7fff700 1 -- 172.21.15.163:0/4195622518 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0b4051f20 con 0x7fb0c00691b0 2023-10-09T06:30:51.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.503+0000 7f2e9effd700 1 -- 172.21.15.163:0/2550355421 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9800a2e0 con 0x7f2ea0156760 2023-10-09T06:30:51.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.503+0000 7f2e9effd700 1 -- 172.21.15.163:0/2550355421 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9800cd20 con 0x7f2ea0156760 2023-10-09T06:30:51.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.504+0000 7f2ea6896700 1 -- 172.21.15.163:0/2550355421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ea0156760 msgr2=0x7f2ea00ac970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.504+0000 7f2ea6896700 1 --2- 172.21.15.163:0/2550355421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ea0156760 0x7f2ea00ac970 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2e9800af70 tx=0x7f2e9800a890 comp rx=0 tx=0).stop 2023-10-09T06:30:51.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.504+0000 7f2ea6896700 1 -- 172.21.15.163:0/2550355421 shutdown_connections 2023-10-09T06:30:51.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.504+0000 7f2ea6896700 1 --2- 172.21.15.163:0/2550355421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ea0156760 0x7f2ea00ac970 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.504+0000 7f2ea6896700 1 -- 172.21.15.163:0/2550355421 >> 172.21.15.163:0/2550355421 conn(0x7f2ea00cd000 msgr2=0x7f2ea0068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:51.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.505+0000 7f2ea6896700 1 -- 172.21.15.163:0/2550355421 shutdown_connections 2023-10-09T06:30:51.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.505+0000 7f2ea6896700 1 -- 172.21.15.163:0/2550355421 wait complete. 2023-10-09T06:30:51.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.505+0000 7fdf127fc700 1 -- 172.21.15.163:0/1804202013 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdf1800fb50 con 0x7fdf2406dfd0 2023-10-09T06:30:51.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.505+0000 7fdf127fc700 1 --2- 172.21.15.163:0/1804202013 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdf04066f30 0x7fdf040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.506+0000 7fdf21d9b700 1 --2- 172.21.15.163:0/1804202013 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdf04066f30 0x7fdf040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.507+0000 7f2ea6896700 1 Processor -- start 2023-10-09T06:30:51.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.507+0000 7f93a3fff700 1 -- 172.21.15.163:0/775666794 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f93b0017080 con 0x7f93bc0afc40 2023-10-09T06:30:51.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.507+0000 7fdf21d9b700 1 --2- 172.21.15.163:0/1804202013 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdf04066f30 0x7fdf040693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdf24155130 tx=0x7fdf14006120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:51.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.507+0000 7f2ea6896700 1 -- start start 2023-10-09T06:30:51.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.508+0000 7f2ea6896700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ea00691b0 0x7f2ea0149f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.508+0000 7f2ea6896700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ea00cba20 0x7f2ea014d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.508+0000 7f2ea6896700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ea0156760 0x7f2ea0150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.508+0000 7f2ea6896700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea014da80 con 0x7f2ea00cba20 2023-10-09T06:30:51.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.508+0000 7f2ea6896700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea014ffb0 con 0x7f2ea00691b0 2023-10-09T06:30:51.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.508+0000 7f2ea6896700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea00ab0f0 con 0x7f2ea0156760 2023-10-09T06:30:51.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.509+0000 7f2e9ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ea00691b0 0x7f2ea0149f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.509+0000 7f2e9ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ea00691b0 0x7f2ea0149f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:50606/0 (socket says 172.21.15.163:50606) 2023-10-09T06:30:51.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.509+0000 7f2e9ffff700 1 -- 172.21.15.163:0/708108347 learned_addr learned my addr 172.21.15.163:0/708108347 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:51.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.509+0000 7f2ea4e33700 1 --2- 172.21.15.163:0/708108347 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ea0156760 0x7f2ea0150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.509+0000 7f2e9ffff700 1 -- 172.21.15.163:0/708108347 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ea0156760 msgr2=0x7f2ea0150760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.509+0000 7f2e9ffff700 1 --2- 172.21.15.163:0/708108347 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ea0156760 0x7f2ea0150760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.509+0000 7f2e9ffff700 1 -- 172.21.15.163:0/708108347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ea00cba20 msgr2=0x7f2ea014d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.509+0000 7f2e9ffff700 1 --2- 172.21.15.163:0/708108347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ea00cba20 0x7f2ea014d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.509+0000 7f2e9ffff700 1 -- 172.21.15.163:0/708108347 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9800ab30 con 0x7f2ea00691b0 2023-10-09T06:30:51.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.509+0000 7f5b67fff700 1 -- 172.21.15.163:0/3856040502 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7000c9b0 con 0x7f5b80156050 2023-10-09T06:30:51.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.509+0000 7f5b67fff700 1 -- 172.21.15.163:0/3856040502 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7000cb20 con 0x7f5b80156050 2023-10-09T06:30:51.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.509+0000 7f2e9ffff700 1 --2- 172.21.15.163:0/708108347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ea00691b0 0x7f2ea0149f40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2e9400c590 tx=0x7f2e9400c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:51.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.509+0000 7f2e87fff700 1 -- 172.21.15.163:0/708108347 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e94011590 con 0x7f2ea00691b0 2023-10-09T06:30:51.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.509+0000 7f2e87fff700 1 -- 172.21.15.163:0/708108347 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e94011710 con 0x7f2ea00691b0 2023-10-09T06:30:51.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.509+0000 7f2e87fff700 1 -- 172.21.15.163:0/708108347 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e94011ad0 con 0x7f2ea00691b0 2023-10-09T06:30:51.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.510+0000 7fbe027fc700 1 -- 172.21.15.163:0/1161603513 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe08014d00 con 0x7fbe0c0ac170 2023-10-09T06:30:51.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.510+0000 7f2ea6896700 1 -- 172.21.15.163:0/708108347 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea014f960 con 0x7f2ea00691b0 2023-10-09T06:30:51.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.510+0000 7f2ea6896700 1 -- 172.21.15.163:0/708108347 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea014cf20 con 0x7f2ea00691b0 2023-10-09T06:30:51.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.510+0000 7f5b67fff700 1 -- 172.21.15.163:0/3856040502 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5b70012070 con 0x7f5b80156050 2023-10-09T06:30:51.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.511+0000 7f5b67fff700 1 --2- 172.21.15.163:0/3856040502 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5b6c066f30 0x7f5b6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.511+0000 7f5b67fff700 1 -- 172.21.15.163:0/3856040502 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5b70084520 con 0x7f5b80156050 2023-10-09T06:30:51.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.511+0000 7f5b7ffff700 1 --2- 172.21.15.163:0/3856040502 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5b6c066f30 0x7f5b6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.511+0000 7f2ea6896700 1 -- 172.21.15.163:0/708108347 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ea014f7c0 con 0x7f2ea00691b0 2023-10-09T06:30:51.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.511+0000 7fbe027fc700 1 -- 172.21.15.163:0/1161603513 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe08014e60 con 0x7fbe0c0ac170 2023-10-09T06:30:51.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.511+0000 7fbe027fc700 1 -- 172.21.15.163:0/1161603513 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0801d6a0 con 0x7fbe0c0ac170 2023-10-09T06:30:51.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.512+0000 7fbe1432e700 1 -- 172.21.15.163:0/1161603513 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe0c167d30 con 0x7fbe0c0ac170 2023-10-09T06:30:51.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.512+0000 7fbe1432e700 1 -- 172.21.15.163:0/1161603513 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe0c086e70 con 0x7fbe0c0ac170 2023-10-09T06:30:51.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.512+0000 7f5b7ffff700 1 --2- 172.21.15.163:0/3856040502 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5b6c066f30 0x7f5b6c0693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5b78009740 tx=0x7f5b78006c60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:51.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.512+0000 7fbe1432e700 1 -- 172.21.15.163:0/1161603513 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe0c04c3c0 con 0x7fbe0c0ac170 2023-10-09T06:30:51.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.514+0000 7f5b67fff700 1 -- 172.21.15.163:0/3856040502 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5b70051790 con 0x7f5b80156050 2023-10-09T06:30:51.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.516+0000 7fdf127fc700 1 -- 172.21.15.163:0/1804202013 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdf1800e260 con 0x7fdf2406dfd0 2023-10-09T06:30:51.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.518+0000 7fdf127fc700 1 -- 172.21.15.163:0/1804202013 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdf241464f0 con 0x7fdf2406dfd0 2023-10-09T06:30:51.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.519+0000 7f2e87fff700 1 -- 172.21.15.163:0/708108347 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2e9401b3b0 con 0x7f2ea00691b0 2023-10-09T06:30:51.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.519+0000 7f2e87fff700 1 --2- 172.21.15.163:0/708108347 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2e8c066fb0 0x7f2e8c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.519+0000 7f2e87fff700 1 -- 172.21.15.163:0/708108347 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2e94083900 con 0x7f2ea00691b0 2023-10-09T06:30:51.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.520+0000 7f2e9f7fe700 1 --2- 172.21.15.163:0/708108347 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2e8c066fb0 0x7f2e8c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.520+0000 7fbe027fc700 1 -- 172.21.15.163:0/1161603513 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbe0801a070 con 0x7fbe0c0ac170 2023-10-09T06:30:51.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.520+0000 7fbe027fc700 1 --2- 172.21.15.163:0/1161603513 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbdf0066f30 0x7fbdf00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:51.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.520+0000 7fbe027fc700 1 -- 172.21.15.163:0/1161603513 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbe0808c590 con 0x7fbe0c0ac170 2023-10-09T06:30:51.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.521+0000 7f2e9f7fe700 1 --2- 172.21.15.163:0/708108347 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2e8c066fb0 0x7f2e8c069460 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2ea00abae0 tx=0x7f2e90005b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:51.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.523+0000 7fbe120ca700 1 --2- 172.21.15.163:0/1161603513 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbdf0066f30 0x7fbdf00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:51.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.523+0000 7fbe120ca700 1 --2- 172.21.15.163:0/1161603513 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbdf0066f30 0x7fbdf00693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbe040021b0 tx=0x7fbe04009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:51.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.524+0000 7fbe027fc700 1 -- 172.21.15.163:0/1161603513 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbe08059960 con 0x7fbe0c0ac170 2023-10-09T06:30:51.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.524+0000 7f2e87fff700 1 -- 172.21.15.163:0/708108347 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2ea014f7c0 con 0x7f2ea00691b0 2023-10-09T06:30:51.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.839+0000 7f59c4def700 1 -- 172.21.15.163:0/3827090069 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f59a0005590 con 0x7f59c00ae4a0 2023-10-09T06:30:51.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.839+0000 7f59ae7fc700 1 -- 172.21.15.163:0/3827090069 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f59b4028070 con 0x7f59c00ae4a0 2023-10-09T06:30:51.843 INFO:teuthology.orchestra.run.smithi163.stdout:55834574856 2023-10-09T06:30:51.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.843+0000 7f59c4def700 1 -- 172.21.15.163:0/3827090069 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f59a8068720 msgr2=0x7f59a806abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.843+0000 7f59c4def700 1 --2- 172.21.15.163:0/3827090069 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f59a8068720 0x7f59a806abd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f59b0004c30 tx=0x7f59b0009170 comp rx=0 tx=0).stop 2023-10-09T06:30:51.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.843+0000 7f59c4def700 1 -- 172.21.15.163:0/3827090069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59c00ae4a0 msgr2=0x7f59c00ae940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.843+0000 7f59c4def700 1 --2- 172.21.15.163:0/3827090069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59c00ae4a0 0x7f59c00ae940 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f59c00b1670 tx=0x7f59b4014000 comp rx=0 tx=0).stop 2023-10-09T06:30:51.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.843+0000 7f59c4def700 1 -- 172.21.15.163:0/3827090069 shutdown_connections 2023-10-09T06:30:51.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.843+0000 7f59c4def700 1 --2- 172.21.15.163:0/3827090069 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f59a8068720 0x7f59a806abd0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.843+0000 7f59c4def700 1 --2- 172.21.15.163:0/3827090069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59c00ae4a0 0x7f59c00ae940 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.843+0000 7f59c4def700 1 --2- 172.21.15.163:0/3827090069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59a8009a50 0x7f59c0086a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.843+0000 7f59c4def700 1 --2- 172.21.15.163:0/3827090069 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59c0167e20 0x7f59c0168290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.843+0000 7f59c4def700 1 -- 172.21.15.163:0/3827090069 >> 172.21.15.163:0/3827090069 conn(0x7f59c00cd000 msgr2=0x7f59c006aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:51.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.844+0000 7f59c4def700 1 -- 172.21.15.163:0/3827090069 shutdown_connections 2023-10-09T06:30:51.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.844+0000 7f59c4def700 1 -- 172.21.15.163:0/3827090069 wait complete. 2023-10-09T06:30:51.862 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2023-10-09T06:30:51.863 DEBUG:teuthology.parallel:result is None 2023-10-09T06:30:51.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.878+0000 7f4b019db700 1 -- 172.21.15.163:0/1938007466 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f4afc04c3c0 con 0x7f4afc06c0a0 2023-10-09T06:30:51.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.878+0000 7f4ae2ffd700 1 -- 172.21.15.163:0/1938007466 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4af4056250 con 0x7f4afc06c0a0 2023-10-09T06:30:51.879 INFO:teuthology.orchestra.run.smithi163.stdout:55834574855 2023-10-09T06:30:51.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.882+0000 7f4b019db700 1 -- 172.21.15.163:0/1938007466 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4ae8067000 msgr2=0x7f4ae80694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.882+0000 7f4b019db700 1 --2- 172.21.15.163:0/1938007466 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4ae8067000 0x7f4ae80694b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4af00091e0 tx=0x7f4af0017000 comp rx=0 tx=0).stop 2023-10-09T06:30:51.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.882+0000 7f4b019db700 1 -- 172.21.15.163:0/1938007466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4afc06c0a0 msgr2=0x7f4afc1309c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.882+0000 7f4b019db700 1 --2- 172.21.15.163:0/1938007466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4afc06c0a0 0x7f4afc1309c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4af400cf00 tx=0x7f4af40077f0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.882+0000 7f4b019db700 1 -- 172.21.15.163:0/1938007466 shutdown_connections 2023-10-09T06:30:51.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.882+0000 7f4b019db700 1 --2- 172.21.15.163:0/1938007466 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4ae8067000 0x7f4ae80694b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.882+0000 7f4b019db700 1 --2- 172.21.15.163:0/1938007466 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4afc0c8e00 0x7f4afc133ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.882+0000 7f4b019db700 1 --2- 172.21.15.163:0/1938007466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4afc06bcd0 0x7f4afc136db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.882+0000 7f4b019db700 1 --2- 172.21.15.163:0/1938007466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4afc06c0a0 0x7f4afc1309c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.882+0000 7f4b019db700 1 -- 172.21.15.163:0/1938007466 >> 172.21.15.163:0/1938007466 conn(0x7f4afc0ce600 msgr2=0x7f4afc0c41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:51.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.886+0000 7f4b019db700 1 -- 172.21.15.163:0/1938007466 shutdown_connections 2023-10-09T06:30:51.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.886+0000 7f4b019db700 1 -- 172.21.15.163:0/1938007466 wait complete. 2023-10-09T06:30:51.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.895+0000 7fb0c662f700 1 -- 172.21.15.163:0/4195622518 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb0a0005830 con 0x7fb0c00691b0 2023-10-09T06:30:51.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.896+0000 7fb0a7fff700 1 -- 172.21.15.163:0/4195622518 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb0b4055540 con 0x7fb0c00691b0 2023-10-09T06:30:51.897 INFO:teuthology.orchestra.run.smithi163.stdout:55834574857 2023-10-09T06:30:51.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.900+0000 7fb0c662f700 1 -- 172.21.15.163:0/4195622518 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb0ac066f30 msgr2=0x7fb0ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.900+0000 7fb0c662f700 1 --2- 172.21.15.163:0/4195622518 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb0ac066f30 0x7fb0ac0693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0006fd0 tx=0x7fb0b0008040 comp rx=0 tx=0).stop 2023-10-09T06:30:51.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.900+0000 7fb0c662f700 1 -- 172.21.15.163:0/4195622518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0c00691b0 msgr2=0x7fb0c0157bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.900+0000 7fb0c662f700 1 --2- 172.21.15.163:0/4195622518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0c00691b0 0x7fb0c0157bf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400b5e0 tx=0x7fb0b4009120 comp rx=0 tx=0).stop 2023-10-09T06:30:51.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.900+0000 7fbe1432e700 1 -- 172.21.15.163:0/1161603513 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fbe0c1684c0 con 0x7fbe0c0ac170 2023-10-09T06:30:51.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.900+0000 7fb0c662f700 1 -- 172.21.15.163:0/4195622518 shutdown_connections 2023-10-09T06:30:51.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.900+0000 7fb0c662f700 1 --2- 172.21.15.163:0/4195622518 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb0ac066f30 0x7fb0ac0693e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.900+0000 7fb0c662f700 1 --2- 172.21.15.163:0/4195622518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0c00691b0 0x7fb0c0157bf0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.900+0000 7fb0c662f700 1 --2- 172.21.15.163:0/4195622518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0c0158440 0x7fb0c0161930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.900+0000 7fb0c662f700 1 --2- 172.21.15.163:0/4195622518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0c00b1fe0 0x7fb0c0143df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.900+0000 7fb0c662f700 1 -- 172.21.15.163:0/4195622518 >> 172.21.15.163:0/4195622518 conn(0x7fb0c00cd000 msgr2=0x7fb0c0146240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:51.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.900+0000 7f93c28d1700 1 -- 172.21.15.163:0/775666794 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f93bc0ad310 con 0x7f93bc0afc40 2023-10-09T06:30:51.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.901+0000 7fbe027fc700 1 -- 172.21.15.163:0/1161603513 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbe0805cf80 con 0x7fbe0c0ac170 2023-10-09T06:30:51.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.901+0000 7fb0c662f700 1 -- 172.21.15.163:0/4195622518 shutdown_connections 2023-10-09T06:30:51.903 INFO:teuthology.orchestra.run.smithi163.stdout:55834574855 2023-10-09T06:30:51.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.901+0000 7fb0c662f700 1 -- 172.21.15.163:0/4195622518 wait complete. 2023-10-09T06:30:51.905 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2023-10-09T06:30:51.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.901+0000 7f93a3fff700 1 -- 172.21.15.163:0/775666794 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f93b0050c50 con 0x7f93bc0afc40 2023-10-09T06:30:51.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.903+0000 7f5b8699e700 1 -- 172.21.15.163:0/3856040502 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f5b8015ab50 con 0x7f5b80156050 2023-10-09T06:30:51.906 DEBUG:teuthology.parallel:result is None 2023-10-09T06:30:51.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.903+0000 7fbe1432e700 1 -- 172.21.15.163:0/1161603513 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbdf0066f30 msgr2=0x7fbdf00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.903+0000 7fbe1432e700 1 --2- 172.21.15.163:0/1161603513 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbdf0066f30 0x7fbdf00693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbe040021b0 tx=0x7fbe04009040 comp rx=0 tx=0).stop 2023-10-09T06:30:51.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.903+0000 7fbe1432e700 1 -- 172.21.15.163:0/1161603513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe0c0ac170 msgr2=0x7fbe0c1537c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.903+0000 7fbe1432e700 1 --2- 172.21.15.163:0/1161603513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe0c0ac170 0x7fbe0c1537c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009ea0 tx=0x7fbe08009220 comp rx=0 tx=0).stop 2023-10-09T06:30:51.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.903+0000 7f5b67fff700 1 -- 172.21.15.163:0/3856040502 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5b8015ab50 con 0x7f5b80156050 2023-10-09T06:30:51.907 INFO:teuthology.orchestra.run.smithi163.stdout:55834574854 2023-10-09T06:30:51.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.904+0000 7fbe1432e700 1 -- 172.21.15.163:0/1161603513 shutdown_connections 2023-10-09T06:30:51.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.904+0000 7fbe1432e700 1 --2- 172.21.15.163:0/1161603513 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbdf0066f30 0x7fbdf00693e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.904+0000 7fbe1432e700 1 --2- 172.21.15.163:0/1161603513 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe0c0afe00 0x7fbe0c086930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.904+0000 7fbe1432e700 1 --2- 172.21.15.163:0/1161603513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe0c0691b0 0x7fbe0c167190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.904+0000 7fbe1432e700 1 --2- 172.21.15.163:0/1161603513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe0c0ac170 0x7fbe0c1537c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.904+0000 7fbe1432e700 1 -- 172.21.15.163:0/1161603513 >> 172.21.15.163:0/1161603513 conn(0x7fbe0c0cd000 msgr2=0x7fbe0c0af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:51.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.904+0000 7fbe1432e700 1 -- 172.21.15.163:0/1161603513 shutdown_connections 2023-10-09T06:30:51.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.904+0000 7fbe1432e700 1 -- 172.21.15.163:0/1161603513 wait complete. 2023-10-09T06:30:51.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.906+0000 7f5b8699e700 1 -- 172.21.15.163:0/3856040502 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5b6c066f30 msgr2=0x7f5b6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.906+0000 7f5b8699e700 1 --2- 172.21.15.163:0/3856040502 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5b6c066f30 0x7f5b6c0693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5b78009740 tx=0x7f5b78006c60 comp rx=0 tx=0).stop 2023-10-09T06:30:51.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.906+0000 7f5b8699e700 1 -- 172.21.15.163:0/3856040502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b80156050 msgr2=0x7f5b80145b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.906+0000 7f5b8699e700 1 --2- 172.21.15.163:0/3856040502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b80156050 0x7f5b80145b50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000bc30 tx=0x7f5b70009840 comp rx=0 tx=0).stop 2023-10-09T06:30:51.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.908+0000 7f5b8699e700 1 -- 172.21.15.163:0/3856040502 shutdown_connections 2023-10-09T06:30:51.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.908+0000 7f5b8699e700 1 --2- 172.21.15.163:0/3856040502 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5b6c066f30 0x7f5b6c0693e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.908+0000 7f5b8699e700 1 --2- 172.21.15.163:0/3856040502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b80156050 0x7f5b80145b50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.908+0000 7f5b8699e700 1 --2- 172.21.15.163:0/3856040502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b800691b0 0x7f5b8016c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.908+0000 7f5b8699e700 1 --2- 172.21.15.163:0/3856040502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b800cba20 0x7f5b80145610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.908+0000 7f5b8699e700 1 -- 172.21.15.163:0/3856040502 >> 172.21.15.163:0/3856040502 conn(0x7f5b800cd000 msgr2=0x7f5b800afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:51.911 INFO:teuthology.orchestra.run.smithi163.stdout:55834574857 2023-10-09T06:30:51.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.908+0000 7f5b8699e700 1 -- 172.21.15.163:0/3856040502 shutdown_connections 2023-10-09T06:30:51.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.908+0000 7f5b8699e700 1 -- 172.21.15.163:0/3856040502 wait complete. 2023-10-09T06:30:51.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.909+0000 7f2ea6896700 1 -- 172.21.15.163:0/708108347 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2ea00b2490 con 0x7f2ea00691b0 2023-10-09T06:30:51.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.910+0000 7f2e87fff700 1 -- 172.21.15.163:0/708108347 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2ea00b2490 con 0x7f2ea00691b0 2023-10-09T06:30:51.912 INFO:teuthology.orchestra.run.smithi163.stdout:55834574856 2023-10-09T06:30:51.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.911+0000 7f93c28d1700 1 -- 172.21.15.163:0/775666794 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f93a8066f30 msgr2=0x7f93a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.911+0000 7f93c28d1700 1 --2- 172.21.15.163:0/775666794 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f93a8066f30 0x7f93a80693e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0091e0 tx=0x7f93ac016000 comp rx=0 tx=0).stop 2023-10-09T06:30:51.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.911+0000 7f93c28d1700 1 -- 172.21.15.163:0/775666794 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93bc0afc40 msgr2=0x7f93bc14ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.911+0000 7f93c28d1700 1 --2- 172.21.15.163:0/775666794 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93bc0afc40 0x7f93bc14ea30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f93b000c810 tx=0x7f93b0007ef0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.912+0000 7f93c28d1700 1 -- 172.21.15.163:0/775666794 shutdown_connections 2023-10-09T06:30:51.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.912+0000 7f93c28d1700 1 --2- 172.21.15.163:0/775666794 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f93a8066f30 0x7f93a80693e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.912+0000 7f93c28d1700 1 --2- 172.21.15.163:0/775666794 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93bc15dd90 0x7f93bc06e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.912+0000 7f93c28d1700 1 --2- 172.21.15.163:0/775666794 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93bc0afc40 0x7f93bc14ea30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.912+0000 7f93c28d1700 1 --2- 172.21.15.163:0/775666794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93bc0b0010 0x7f93bc161740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.912+0000 7f93c28d1700 1 -- 172.21.15.163:0/775666794 >> 172.21.15.163:0/775666794 conn(0x7f93bc06a320 msgr2=0x7f93bc14f880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:51.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.912+0000 7f93c28d1700 1 -- 172.21.15.163:0/775666794 shutdown_connections 2023-10-09T06:30:51.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.912+0000 7f93c28d1700 1 -- 172.21.15.163:0/775666794 wait complete. 2023-10-09T06:30:51.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.913+0000 7f2ea6896700 1 -- 172.21.15.163:0/708108347 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2e8c066fb0 msgr2=0x7f2e8c069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.913+0000 7f2ea6896700 1 --2- 172.21.15.163:0/708108347 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2e8c066fb0 0x7f2e8c069460 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2ea00abae0 tx=0x7f2e90005b50 comp rx=0 tx=0).stop 2023-10-09T06:30:51.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.913+0000 7f2ea6896700 1 -- 172.21.15.163:0/708108347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ea00691b0 msgr2=0x7f2ea0149f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.913+0000 7f2ea6896700 1 --2- 172.21.15.163:0/708108347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ea00691b0 0x7f2ea0149f40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2e9400c590 tx=0x7f2e9400c8a0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.914+0000 7f2ea6896700 1 -- 172.21.15.163:0/708108347 shutdown_connections 2023-10-09T06:30:51.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.914+0000 7f2ea6896700 1 --2- 172.21.15.163:0/708108347 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2e8c066fb0 0x7f2e8c069460 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.914+0000 7f2ea6896700 1 --2- 172.21.15.163:0/708108347 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ea0156760 0x7f2ea0150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.914+0000 7f2ea6896700 1 --2- 172.21.15.163:0/708108347 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ea00691b0 0x7f2ea0149f40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.914+0000 7f2ea6896700 1 --2- 172.21.15.163:0/708108347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ea00cba20 0x7f2ea014d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.914+0000 7f2ea6896700 1 -- 172.21.15.163:0/708108347 >> 172.21.15.163:0/708108347 conn(0x7f2ea00cd000 msgr2=0x7f2ea014dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:51.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.914+0000 7f2ea6896700 1 -- 172.21.15.163:0/708108347 shutdown_connections 2023-10-09T06:30:51.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.914+0000 7f2ea6896700 1 -- 172.21.15.163:0/708108347 wait complete. 2023-10-09T06:30:51.922 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.1 2023-10-09T06:30:51.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.920+0000 7fdf28e2d700 1 -- 172.21.15.163:0/1804202013 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fdf241464f0 con 0x7fdf2406dfd0 2023-10-09T06:30:51.922 DEBUG:teuthology.parallel:result is None 2023-10-09T06:30:51.923 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.922+0000 7fdf127fc700 1 -- 172.21.15.163:0/1804202013 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdf241464f0 con 0x7fdf2406dfd0 2023-10-09T06:30:51.923 INFO:teuthology.orchestra.run.smithi163.stdout:55834574855 2023-10-09T06:30:51.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.925+0000 7fdf28e2d700 1 -- 172.21.15.163:0/1804202013 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdf04066f30 msgr2=0x7fdf040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.925+0000 7fdf28e2d700 1 --2- 172.21.15.163:0/1804202013 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdf04066f30 0x7fdf040693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdf24155130 tx=0x7fdf14006120 comp rx=0 tx=0).stop 2023-10-09T06:30:51.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.925+0000 7fdf28e2d700 1 -- 172.21.15.163:0/1804202013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf2406dfd0 msgr2=0x7fdf24149370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:51.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.925+0000 7fdf28e2d700 1 --2- 172.21.15.163:0/1804202013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf2406dfd0 0x7fdf24149370 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800b510 tx=0x7fdf18009060 comp rx=0 tx=0).stop 2023-10-09T06:30:51.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.925+0000 7fdf28e2d700 1 -- 172.21.15.163:0/1804202013 shutdown_connections 2023-10-09T06:30:51.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.925+0000 7fdf28e2d700 1 --2- 172.21.15.163:0/1804202013 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdf04066f30 0x7fdf040693e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.925+0000 7fdf28e2d700 1 --2- 172.21.15.163:0/1804202013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf2406dfd0 0x7fdf24149370 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.926+0000 7fdf28e2d700 1 --2- 172.21.15.163:0/1804202013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf2414cef0 0x7fdf2414d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.926+0000 7fdf28e2d700 1 --2- 172.21.15.163:0/1804202013 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf24155780 0x7fdf241450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:51.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.926+0000 7fdf28e2d700 1 -- 172.21.15.163:0/1804202013 >> 172.21.15.163:0/1804202013 conn(0x7fdf24068390 msgr2=0x7fdf240af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:51.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.926+0000 7fdf28e2d700 1 -- 172.21.15.163:0/1804202013 shutdown_connections 2023-10-09T06:30:51.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:51.926+0000 7fdf28e2d700 1 -- 172.21.15.163:0/1804202013 wait complete. 2023-10-09T06:30:51.929 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2023-10-09T06:30:51.930 DEBUG:teuthology.parallel:result is None 2023-10-09T06:30:51.930 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2023-10-09T06:30:51.931 DEBUG:teuthology.parallel:result is None 2023-10-09T06:30:51.932 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.2 2023-10-09T06:30:51.932 DEBUG:teuthology.parallel:result is None 2023-10-09T06:30:51.934 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.0 2023-10-09T06:30:51.934 DEBUG:teuthology.parallel:result is None 2023-10-09T06:30:51.940 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2023-10-09T06:30:51.940 DEBUG:teuthology.parallel:result is None 2023-10-09T06:30:51.941 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-09T06:30:51.941 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:30:52.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.094+0000 7fe5eb1bd700 1 Processor -- start 2023-10-09T06:30:52.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.094+0000 7fe5eb1bd700 1 -- start start 2023-10-09T06:30:52.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.094+0000 7fe5eb1bd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e406bcd0 0x7fe5e4140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:52.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.094+0000 7fe5eb1bd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe5e40c9510 0x7fe5e4141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:52.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.094+0000 7fe5eb1bd700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5e415c2a0 con 0x7fe5e406c0a0 2023-10-09T06:30:52.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.094+0000 7fe5eb1bd700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e415b690 con 0x7fe5e40c9510 2023-10-09T06:30:52.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.094+0000 7fe5eb1bd700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e415b970 con 0x7fe5e406bcd0 2023-10-09T06:30:52.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.095+0000 7fe5e975a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe5e40c9510 0x7fe5e4141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:52.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.095+0000 7fe5e975a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe5e40c9510 0x7fe5e4141330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:50612/0 (socket says 172.21.15.163:50612) 2023-10-09T06:30:52.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.095+0000 7fe5dbfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe5e406c0a0 0x7fe5e40c8e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:47952/0 (socket says 172.21.15.163:47952) 2023-10-09T06:30:52.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.095+0000 7fe5e975a700 1 -- 172.21.15.163:0/4148902411 learned_addr learned my addr 172.21.15.163:0/4148902411 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:52.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.095+0000 7fe5e8f59700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e406bcd0 0x7fe5e4140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:52.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.095+0000 7fe5db7fe700 1 -- 172.21.15.163:0/4148902411 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2983072354 0 0) 0x7fe5e415c2a0 con 0x7fe5e406c0a0 2023-10-09T06:30:52.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.095+0000 7fe5db7fe700 1 -- 172.21.15.163:0/4148902411 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5cc003570 con 0x7fe5e406c0a0 2023-10-09T06:30:52.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.095+0000 7fe5e8f59700 1 -- 172.21.15.163:0/4148902411 >> v2:172.21.15.181:3300/0 conn(0x7fe5e40c9510 msgr2=0x7fe5e4141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:52.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.095+0000 7fe5e8f59700 1 --2- 172.21.15.163:0/4148902411 >> v2:172.21.15.181:3300/0 conn(0x7fe5e40c9510 0x7fe5e4141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:52.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.095+0000 7fe5e8f59700 1 -- 172.21.15.163:0/4148902411 >> v1:172.21.15.181:6789/0 conn(0x7fe5e406c0a0 legacy=0x7fe5e40c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:52.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.095+0000 7fe5e8f59700 1 -- 172.21.15.163:0/4148902411 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e415bc50 con 0x7fe5e406bcd0 2023-10-09T06:30:52.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.095+0000 7fe5e975a700 1 --2- 172.21.15.163:0/4148902411 >> v2:172.21.15.181:3300/0 conn(0x7fe5e40c9510 0x7fe5e4141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:30:52.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.095+0000 7fe5e8f59700 1 --2- 172.21.15.163:0/4148902411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e406bcd0 0x7fe5e4140df0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe5e00098e0 tx=0x7fe5e00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=206ef96229b1e0e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:52.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.096+0000 7fe5db7fe700 1 -- 172.21.15.163:0/4148902411 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0011630 con 0x7fe5e406bcd0 2023-10-09T06:30:52.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.096+0000 7fe5db7fe700 1 -- 172.21.15.163:0/4148902411 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e0011790 con 0x7fe5e406bcd0 2023-10-09T06:30:52.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.096+0000 7fe5db7fe700 1 -- 172.21.15.163:0/4148902411 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0011b30 con 0x7fe5e406bcd0 2023-10-09T06:30:52.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.096+0000 7fe5eb1bd700 1 -- 172.21.15.163:0/4148902411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e406bcd0 msgr2=0x7fe5e4140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:52.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.096+0000 7fe5eb1bd700 1 --2- 172.21.15.163:0/4148902411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e406bcd0 0x7fe5e4140df0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe5e00098e0 tx=0x7fe5e00091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:52.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.096+0000 7fe5eb1bd700 1 -- 172.21.15.163:0/4148902411 shutdown_connections 2023-10-09T06:30:52.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.096+0000 7fe5eb1bd700 1 --2- 172.21.15.163:0/4148902411 >> v2:172.21.15.181:3300/0 conn(0x7fe5e40c9510 0x7fe5e4141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:52.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.096+0000 7fe5eb1bd700 1 --2- 172.21.15.163:0/4148902411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e406bcd0 0x7fe5e4140df0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:52.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.096+0000 7fe5eb1bd700 1 -- 172.21.15.163:0/4148902411 >> 172.21.15.163:0/4148902411 conn(0x7fe5e40ce600 msgr2=0x7fe5e40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:52.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.096+0000 7fe5eb1bd700 1 -- 172.21.15.163:0/4148902411 shutdown_connections 2023-10-09T06:30:52.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.096+0000 7fe5eb1bd700 1 -- 172.21.15.163:0/4148902411 wait complete. 2023-10-09T06:30:52.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.097+0000 7fe5eb1bd700 1 Processor -- start 2023-10-09T06:30:52.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.097+0000 7fe5eb1bd700 1 -- start start 2023-10-09T06:30:52.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.098+0000 7fe5eb1bd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e406bcd0 0x7fe5e41553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:52.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.098+0000 7fe5eb1bd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e406c0a0 0x7fe5e41533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:52.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.098+0000 7fe5eb1bd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e40c9510 0x7fe5e4153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:52.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.098+0000 7fe5eb1bd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e4156650 con 0x7fe5e406c0a0 2023-10-09T06:30:52.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.098+0000 7fe5eb1bd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e4155a40 con 0x7fe5e406bcd0 2023-10-09T06:30:52.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.098+0000 7fe5eb1bd700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e4155d20 con 0x7fe5e40c9510 2023-10-09T06:30:52.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.098+0000 7fe5dbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e406c0a0 0x7fe5e41533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:52.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.098+0000 7fe5dbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e406c0a0 0x7fe5e41533f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44088/0 (socket says 172.21.15.163:44088) 2023-10-09T06:30:52.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.098+0000 7fe5dbfff700 1 -- 172.21.15.163:0/4240340882 learned_addr learned my addr 172.21.15.163:0/4240340882 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:52.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.098+0000 7fe5e8f59700 1 --2- 172.21.15.163:0/4240340882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e406bcd0 0x7fe5e41553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:52.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.098+0000 7fe5e975a700 1 --2- 172.21.15.163:0/4240340882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e40c9510 0x7fe5e4153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:52.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.098+0000 7fe5dbfff700 1 -- 172.21.15.163:0/4240340882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e40c9510 msgr2=0x7fe5e4153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:52.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.098+0000 7fe5dbfff700 1 --2- 172.21.15.163:0/4240340882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e40c9510 0x7fe5e4153930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:52.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.098+0000 7fe5dbfff700 1 -- 172.21.15.163:0/4240340882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e406bcd0 msgr2=0x7fe5e41553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:52.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.099+0000 7fe5dbfff700 1 --2- 172.21.15.163:0/4240340882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e406bcd0 0x7fe5e41553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:52.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.099+0000 7fe5dbfff700 1 -- 172.21.15.163:0/4240340882 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5dc009ae0 con 0x7fe5e406c0a0 2023-10-09T06:30:52.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.099+0000 7fe5e8f59700 1 --2- 172.21.15.163:0/4240340882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e406bcd0 0x7fe5e41553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:30:52.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.099+0000 7fe5e975a700 1 --2- 172.21.15.163:0/4240340882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e40c9510 0x7fe5e4153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:30:52.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.099+0000 7fe5dbfff700 1 --2- 172.21.15.163:0/4240340882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e406c0a0 0x7fe5e41533f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe5d400c7c0 tx=0x7fe5d4007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:52.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.099+0000 7fe5d8ff9700 1 -- 172.21.15.163:0/4240340882 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4007300 con 0x7fe5e406c0a0 2023-10-09T06:30:52.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.099+0000 7fe5d8ff9700 1 -- 172.21.15.163:0/4240340882 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d4007480 con 0x7fe5e406c0a0 2023-10-09T06:30:52.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.099+0000 7fe5d8ff9700 1 -- 172.21.15.163:0/4240340882 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4004830 con 0x7fe5e406c0a0 2023-10-09T06:30:52.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.099+0000 7fe5eb1bd700 1 -- 172.21.15.163:0/4240340882 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e0009420 con 0x7fe5e406c0a0 2023-10-09T06:30:52.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.099+0000 7fe5eb1bd700 1 -- 172.21.15.163:0/4240340882 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e41562e0 con 0x7fe5e406c0a0 2023-10-09T06:30:52.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.100+0000 7fe5eb1bd700 1 -- 172.21.15.163:0/4240340882 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e404c3c0 con 0x7fe5e406c0a0 2023-10-09T06:30:52.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.100+0000 7fe5d8ff9700 1 -- 172.21.15.163:0/4240340882 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe5d4004ba0 con 0x7fe5e406c0a0 2023-10-09T06:30:52.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.100+0000 7fe5d8ff9700 1 --2- 172.21.15.163:0/4240340882 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe5d0066f30 0x7fe5d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:52.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.101+0000 7fe5d8ff9700 1 -- 172.21.15.163:0/4240340882 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe5d4083a10 con 0x7fe5e406c0a0 2023-10-09T06:30:52.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.101+0000 7fe5e8f59700 1 --2- 172.21.15.163:0/4240340882 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe5d0066f30 0x7fe5d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:52.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.101+0000 7fe5e8f59700 1 --2- 172.21.15.163:0/4240340882 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe5d0066f30 0x7fe5d00693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0003080 tx=0x7fe5e0017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:52.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.105+0000 7fe5d8ff9700 1 -- 172.21.15.163:0/4240340882 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5d4050d00 con 0x7fe5e406c0a0 2023-10-09T06:30:52.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.284+0000 7fe5eb1bd700 1 -- 172.21.15.163:0/4240340882 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe5e4156790 con 0x7fe5d0066f30 2023-10-09T06:30:52.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.292+0000 7fe5d8ff9700 1 -- 172.21.15.163:0/4240340882 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264511 (secure 0 0 0) 0x7fe5e4156790 con 0x7fe5d0066f30 2023-10-09T06:30:52.297 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:30:52.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.295+0000 7fe5eb1bd700 1 -- 172.21.15.163:0/4240340882 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe5d0066f30 msgr2=0x7fe5d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:52.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.295+0000 7fe5eb1bd700 1 --2- 172.21.15.163:0/4240340882 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe5d0066f30 0x7fe5d00693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0003080 tx=0x7fe5e0017000 comp rx=0 tx=0).stop 2023-10-09T06:30:52.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.295+0000 7fe5eb1bd700 1 -- 172.21.15.163:0/4240340882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e406c0a0 msgr2=0x7fe5e41533f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:52.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.295+0000 7fe5eb1bd700 1 --2- 172.21.15.163:0/4240340882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e406c0a0 0x7fe5e41533f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe5d400c7c0 tx=0x7fe5d4007e00 comp rx=0 tx=0).stop 2023-10-09T06:30:52.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.295+0000 7fe5eb1bd700 1 -- 172.21.15.163:0/4240340882 shutdown_connections 2023-10-09T06:30:52.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.295+0000 7fe5eb1bd700 1 --2- 172.21.15.163:0/4240340882 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe5d0066f30 0x7fe5d00693e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:52.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.295+0000 7fe5eb1bd700 1 --2- 172.21.15.163:0/4240340882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e40c9510 0x7fe5e4153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:52.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.295+0000 7fe5eb1bd700 1 --2- 172.21.15.163:0/4240340882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e406bcd0 0x7fe5e41553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:52.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.295+0000 7fe5eb1bd700 1 --2- 172.21.15.163:0/4240340882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e406c0a0 0x7fe5e41533f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:52.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.295+0000 7fe5eb1bd700 1 -- 172.21.15.163:0/4240340882 >> 172.21.15.163:0/4240340882 conn(0x7fe5e40ce600 msgr2=0x7fe5e40c8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:52.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.295+0000 7fe5eb1bd700 1 -- 172.21.15.163:0/4240340882 shutdown_connections 2023-10-09T06:30:52.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.295+0000 7fe5eb1bd700 1 -- 172.21.15.163:0/4240340882 wait complete. 2023-10-09T06:30:52.302 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:30:52.308 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-09T06:30:50.296038+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381128,"kb_used_data":2824,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593592,"statfs":{"total":773094113280,"available":770655838208,"internally_reserved":0,"allocated":2891776,"data_stored":1750580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.553953"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.492500+0000","last_change":"2023-10-09T06:30:43.487077+0000","last_active":"2023-10-09T06:30:43.492500+0000","last_peered":"2023-10-09T06:30:43.492500+0000","last_clean":"2023-10-09T06:30:43.492500+0000","last_became_active":"2023-10-09T06:30:43.486817+0000","last_became_peered":"2023-10-09T06:30:43.486817+0000","last_unstale":"2023-10-09T06:30:43.492500+0000","last_undegraded":"2023-10-09T06:30:43.492500+0000","last_fullsized":"2023-10-09T06:30:43.492500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:59:57.434765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.486972+0000","last_change":"2023-10-09T06:30:43.484119+0000","last_active":"2023-10-09T06:30:43.486972+0000","last_peered":"2023-10-09T06:30:43.486972+0000","last_clean":"2023-10-09T06:30:43.486972+0000","last_became_active":"2023-10-09T06:30:43.483865+0000","last_became_peered":"2023-10-09T06:30:43.483865+0000","last_unstale":"2023-10-09T06:30:43.486972+0000","last_undegraded":"2023-10-09T06:30:43.486972+0000","last_fullsized":"2023-10-09T06:30:43.486972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:06:29.986154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.484893+0000","last_change":"2023-10-09T06:30:43.480754+0000","last_active":"2023-10-09T06:30:43.484893+0000","last_peered":"2023-10-09T06:30:43.484893+0000","last_clean":"2023-10-09T06:30:43.484893+0000","last_became_active":"2023-10-09T06:30:43.480474+0000","last_became_peered":"2023-10-09T06:30:43.480474+0000","last_unstale":"2023-10-09T06:30:43.484893+0000","last_undegraded":"2023-10-09T06:30:43.484893+0000","last_fullsized":"2023-10-09T06:30:43.484893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:56:01.192585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493176+0000","last_change":"2023-10-09T06:30:43.489012+0000","last_active":"2023-10-09T06:30:43.493176+0000","last_peered":"2023-10-09T06:30:43.493176+0000","last_clean":"2023-10-09T06:30:43.493176+0000","last_became_active":"2023-10-09T06:30:43.488614+0000","last_became_peered":"2023-10-09T06:30:43.488614+0000","last_unstale":"2023-10-09T06:30:43.493176+0000","last_undegraded":"2023-10-09T06:30:43.493176+0000","last_fullsized":"2023-10-09T06:30:43.493176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:41:00.854052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.486903+0000","last_change":"2023-10-09T06:30:43.483013+0000","last_active":"2023-10-09T06:30:43.486903+0000","last_peered":"2023-10-09T06:30:43.486903+0000","last_clean":"2023-10-09T06:30:43.486903+0000","last_became_active":"2023-10-09T06:30:43.482373+0000","last_became_peered":"2023-10-09T06:30:43.482373+0000","last_unstale":"2023-10-09T06:30:43.486903+0000","last_undegraded":"2023-10-09T06:30:43.486903+0000","last_fullsized":"2023-10-09T06:30:43.486903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:46:57.531379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.492270+0000","last_change":"2023-10-09T06:30:43.488813+0000","last_active":"2023-10-09T06:30:43.492270+0000","last_peered":"2023-10-09T06:30:43.492270+0000","last_clean":"2023-10-09T06:30:43.492270+0000","last_became_active":"2023-10-09T06:30:43.488532+0000","last_became_peered":"2023-10-09T06:30:43.488532+0000","last_unstale":"2023-10-09T06:30:43.492270+0000","last_undegraded":"2023-10-09T06:30:43.492270+0000","last_fullsized":"2023-10-09T06:30:43.492270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:57:57.207775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.491984+0000","last_change":"2023-10-09T06:30:43.486418+0000","last_active":"2023-10-09T06:30:43.491984+0000","last_peered":"2023-10-09T06:30:43.491984+0000","last_clean":"2023-10-09T06:30:43.491984+0000","last_became_active":"2023-10-09T06:30:43.486128+0000","last_became_peered":"2023-10-09T06:30:43.486128+0000","last_unstale":"2023-10-09T06:30:43.491984+0000","last_undegraded":"2023-10-09T06:30:43.491984+0000","last_fullsized":"2023-10-09T06:30:43.491984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:57:57.207775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493719+0000","last_change":"2023-10-09T06:30:43.490024+0000","last_active":"2023-10-09T06:30:43.493719+0000","last_peered":"2023-10-09T06:30:43.493719+0000","last_clean":"2023-10-09T06:30:43.493719+0000","last_became_active":"2023-10-09T06:30:43.489785+0000","last_became_peered":"2023-10-09T06:30:43.489785+0000","last_unstale":"2023-10-09T06:30:43.493719+0000","last_undegraded":"2023-10-09T06:30:43.493719+0000","last_fullsized":"2023-10-09T06:30:43.493719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:02:03.930820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496624+0000","last_change":"2023-10-09T06:30:43.492372+0000","last_active":"2023-10-09T06:30:43.496624+0000","last_peered":"2023-10-09T06:30:43.496624+0000","last_clean":"2023-10-09T06:30:43.496624+0000","last_became_active":"2023-10-09T06:30:43.492120+0000","last_became_peered":"2023-10-09T06:30:43.492120+0000","last_unstale":"2023-10-09T06:30:43.496624+0000","last_undegraded":"2023-10-09T06:30:43.496624+0000","last_fullsized":"2023-10-09T06:30:43.496624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:20:42.701396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493788+0000","last_change":"2023-10-09T06:30:43.489507+0000","last_active":"2023-10-09T06:30:43.493788+0000","last_peered":"2023-10-09T06:30:43.493788+0000","last_clean":"2023-10-09T06:30:43.493788+0000","last_became_active":"2023-10-09T06:30:43.489279+0000","last_became_peered":"2023-10-09T06:30:43.489279+0000","last_unstale":"2023-10-09T06:30:43.493788+0000","last_undegraded":"2023-10-09T06:30:43.493788+0000","last_fullsized":"2023-10-09T06:30:43.493788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:05:59.526736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496502+0000","last_change":"2023-10-09T06:30:43.491554+0000","last_active":"2023-10-09T06:30:43.496502+0000","last_peered":"2023-10-09T06:30:43.496502+0000","last_clean":"2023-10-09T06:30:43.496502+0000","last_became_active":"2023-10-09T06:30:43.491305+0000","last_became_peered":"2023-10-09T06:30:43.491305+0000","last_unstale":"2023-10-09T06:30:43.496502+0000","last_undegraded":"2023-10-09T06:30:43.496502+0000","last_fullsized":"2023-10-09T06:30:43.496502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:15:54.706732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.483269+0000","last_change":"2023-10-09T06:30:43.478613+0000","last_active":"2023-10-09T06:30:43.483269+0000","last_peered":"2023-10-09T06:30:43.483269+0000","last_clean":"2023-10-09T06:30:43.483269+0000","last_became_active":"2023-10-09T06:30:43.477580+0000","last_became_peered":"2023-10-09T06:30:43.477580+0000","last_unstale":"2023-10-09T06:30:43.483269+0000","last_undegraded":"2023-10-09T06:30:43.483269+0000","last_fullsized":"2023-10-09T06:30:43.483269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:15:34.625915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.491028+0000","last_change":"2023-10-09T06:30:43.485665+0000","last_active":"2023-10-09T06:30:43.491028+0000","last_peered":"2023-10-09T06:30:43.491028+0000","last_clean":"2023-10-09T06:30:43.491028+0000","last_became_active":"2023-10-09T06:30:43.485346+0000","last_became_peered":"2023-10-09T06:30:43.485346+0000","last_unstale":"2023-10-09T06:30:43.491028+0000","last_undegraded":"2023-10-09T06:30:43.491028+0000","last_fullsized":"2023-10-09T06:30:43.491028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:56:01.192585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.487663+0000","last_change":"2023-10-09T06:30:43.483174+0000","last_active":"2023-10-09T06:30:43.487663+0000","last_peered":"2023-10-09T06:30:43.487663+0000","last_clean":"2023-10-09T06:30:43.487663+0000","last_became_active":"2023-10-09T06:30:43.482518+0000","last_became_peered":"2023-10-09T06:30:43.482518+0000","last_unstale":"2023-10-09T06:30:43.487663+0000","last_undegraded":"2023-10-09T06:30:43.487663+0000","last_fullsized":"2023-10-09T06:30:43.487663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:56:48.342402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493620+0000","last_change":"2023-10-09T06:30:43.489829+0000","last_active":"2023-10-09T06:30:43.493620+0000","last_peered":"2023-10-09T06:30:43.493620+0000","last_clean":"2023-10-09T06:30:43.493620+0000","last_became_active":"2023-10-09T06:30:43.489549+0000","last_became_peered":"2023-10-09T06:30:43.489549+0000","last_unstale":"2023-10-09T06:30:43.493620+0000","last_undegraded":"2023-10-09T06:30:43.493620+0000","last_fullsized":"2023-10-09T06:30:43.493620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:56:01.192585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496629+0000","last_change":"2023-10-09T06:30:43.491356+0000","last_active":"2023-10-09T06:30:43.496629+0000","last_peered":"2023-10-09T06:30:43.496629+0000","last_clean":"2023-10-09T06:30:43.496629+0000","last_became_active":"2023-10-09T06:30:43.491058+0000","last_became_peered":"2023-10-09T06:30:43.491058+0000","last_unstale":"2023-10-09T06:30:43.496629+0000","last_undegraded":"2023-10-09T06:30:43.496629+0000","last_fullsized":"2023-10-09T06:30:43.496629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:03:07.803820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.492196+0000","last_change":"2023-10-09T06:30:43.489230+0000","last_active":"2023-10-09T06:30:43.492196+0000","last_peered":"2023-10-09T06:30:43.492196+0000","last_clean":"2023-10-09T06:30:43.492196+0000","last_became_active":"2023-10-09T06:30:43.488094+0000","last_became_peered":"2023-10-09T06:30:43.488094+0000","last_unstale":"2023-10-09T06:30:43.492196+0000","last_undegraded":"2023-10-09T06:30:43.492196+0000","last_fullsized":"2023-10-09T06:30:43.492196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:36:01.020241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.488118+0000","last_change":"2023-10-09T06:30:43.486270+0000","last_active":"2023-10-09T06:30:43.488118+0000","last_peered":"2023-10-09T06:30:43.488118+0000","last_clean":"2023-10-09T06:30:43.488118+0000","last_became_active":"2023-10-09T06:30:43.486003+0000","last_became_peered":"2023-10-09T06:30:43.486003+0000","last_unstale":"2023-10-09T06:30:43.488118+0000","last_undegraded":"2023-10-09T06:30:43.488118+0000","last_fullsized":"2023-10-09T06:30:43.488118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:55:20.971921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.485740+0000","last_change":"2023-10-09T06:30:43.482479+0000","last_active":"2023-10-09T06:30:43.485740+0000","last_peered":"2023-10-09T06:30:43.485740+0000","last_clean":"2023-10-09T06:30:43.485740+0000","last_became_active":"2023-10-09T06:30:43.482203+0000","last_became_peered":"2023-10-09T06:30:43.482203+0000","last_unstale":"2023-10-09T06:30:43.485740+0000","last_undegraded":"2023-10-09T06:30:43.485740+0000","last_fullsized":"2023-10-09T06:30:43.485740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:41:39.976781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.761744+0000","last_change":"2023-10-09T06:30:42.761358+0000","last_active":"2023-10-09T06:30:43.761744+0000","last_peered":"2023-10-09T06:30:43.761744+0000","last_clean":"2023-10-09T06:30:43.761744+0000","last_became_active":"2023-10-09T06:30:42.761139+0000","last_became_peered":"2023-10-09T06:30:42.761139+0000","last_unstale":"2023-10-09T06:30:43.761744+0000","last_undegraded":"2023-10-09T06:30:43.761744+0000","last_fullsized":"2023-10-09T06:30:43.761744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:27:28.658955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.490991+0000","last_change":"2023-10-09T06:30:43.485519+0000","last_active":"2023-10-09T06:30:43.490991+0000","last_peered":"2023-10-09T06:30:43.490991+0000","last_clean":"2023-10-09T06:30:43.490991+0000","last_became_active":"2023-10-09T06:30:43.485289+0000","last_became_peered":"2023-10-09T06:30:43.485289+0000","last_unstale":"2023-10-09T06:30:43.490991+0000","last_undegraded":"2023-10-09T06:30:43.490991+0000","last_fullsized":"2023-10-09T06:30:43.490991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:41:39.976781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.764141+0000","last_change":"2023-10-09T06:30:42.758771+0000","last_active":"2023-10-09T06:30:43.764141+0000","last_peered":"2023-10-09T06:30:43.764141+0000","last_clean":"2023-10-09T06:30:43.764141+0000","last_became_active":"2023-10-09T06:30:42.756764+0000","last_became_peered":"2023-10-09T06:30:42.756764+0000","last_unstale":"2023-10-09T06:30:43.764141+0000","last_undegraded":"2023-10-09T06:30:43.764141+0000","last_fullsized":"2023-10-09T06:30:43.764141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:26:03.347834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493888+0000","last_change":"2023-10-09T06:30:43.491484+0000","last_active":"2023-10-09T06:30:43.493888+0000","last_peered":"2023-10-09T06:30:43.493888+0000","last_clean":"2023-10-09T06:30:43.493888+0000","last_became_active":"2023-10-09T06:30:43.491319+0000","last_became_peered":"2023-10-09T06:30:43.491319+0000","last_unstale":"2023-10-09T06:30:43.493888+0000","last_undegraded":"2023-10-09T06:30:43.493888+0000","last_fullsized":"2023-10-09T06:30:43.493888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:12:09.812942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.761120+0000","last_change":"2023-10-09T06:30:42.763648+0000","last_active":"2023-10-09T06:30:43.761120+0000","last_peered":"2023-10-09T06:30:43.761120+0000","last_clean":"2023-10-09T06:30:43.761120+0000","last_became_active":"2023-10-09T06:30:42.763348+0000","last_became_peered":"2023-10-09T06:30:42.763348+0000","last_unstale":"2023-10-09T06:30:43.761120+0000","last_undegraded":"2023-10-09T06:30:43.761120+0000","last_fullsized":"2023-10-09T06:30:43.761120+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:55:19.967211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496719+0000","last_change":"2023-10-09T06:30:43.491778+0000","last_active":"2023-10-09T06:30:43.496719+0000","last_peered":"2023-10-09T06:30:43.496719+0000","last_clean":"2023-10-09T06:30:43.496719+0000","last_became_active":"2023-10-09T06:30:43.491137+0000","last_became_peered":"2023-10-09T06:30:43.491137+0000","last_unstale":"2023-10-09T06:30:43.496719+0000","last_undegraded":"2023-10-09T06:30:43.496719+0000","last_fullsized":"2023-10-09T06:30:43.496719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:46:57.531379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.761164+0000","last_change":"2023-10-09T06:30:42.761346+0000","last_active":"2023-10-09T06:30:43.761164+0000","last_peered":"2023-10-09T06:30:43.761164+0000","last_clean":"2023-10-09T06:30:43.761164+0000","last_became_active":"2023-10-09T06:30:42.761028+0000","last_became_peered":"2023-10-09T06:30:42.761028+0000","last_unstale":"2023-10-09T06:30:43.761164+0000","last_undegraded":"2023-10-09T06:30:43.761164+0000","last_fullsized":"2023-10-09T06:30:43.761164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:20:58.653921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.492949+0000","last_change":"2023-10-09T06:30:43.489975+0000","last_active":"2023-10-09T06:30:43.492949+0000","last_peered":"2023-10-09T06:30:43.492949+0000","last_clean":"2023-10-09T06:30:43.492949+0000","last_became_active":"2023-10-09T06:30:43.489712+0000","last_became_peered":"2023-10-09T06:30:43.489712+0000","last_unstale":"2023-10-09T06:30:43.492949+0000","last_undegraded":"2023-10-09T06:30:43.492949+0000","last_fullsized":"2023-10-09T06:30:43.492949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:41:39.976781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.762311+0000","last_change":"2023-10-09T06:30:42.757689+0000","last_active":"2023-10-09T06:30:43.762311+0000","last_peered":"2023-10-09T06:30:43.762311+0000","last_clean":"2023-10-09T06:30:43.762311+0000","last_became_active":"2023-10-09T06:30:42.757270+0000","last_became_peered":"2023-10-09T06:30:42.757270+0000","last_unstale":"2023-10-09T06:30:43.762311+0000","last_undegraded":"2023-10-09T06:30:43.762311+0000","last_fullsized":"2023-10-09T06:30:43.762311+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:52:48.682306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.490754+0000","last_change":"2023-10-09T06:30:43.486187+0000","last_active":"2023-10-09T06:30:43.490754+0000","last_peered":"2023-10-09T06:30:43.490754+0000","last_clean":"2023-10-09T06:30:43.490754+0000","last_became_active":"2023-10-09T06:30:43.485917+0000","last_became_peered":"2023-10-09T06:30:43.485917+0000","last_unstale":"2023-10-09T06:30:43.490754+0000","last_undegraded":"2023-10-09T06:30:43.490754+0000","last_fullsized":"2023-10-09T06:30:43.490754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:20:59.658630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.763447+0000","last_change":"2023-10-09T06:30:42.758939+0000","last_active":"2023-10-09T06:30:43.763447+0000","last_peered":"2023-10-09T06:30:43.763447+0000","last_clean":"2023-10-09T06:30:43.763447+0000","last_became_active":"2023-10-09T06:30:42.758667+0000","last_became_peered":"2023-10-09T06:30:42.758667+0000","last_unstale":"2023-10-09T06:30:43.763447+0000","last_undegraded":"2023-10-09T06:30:43.763447+0000","last_fullsized":"2023-10-09T06:30:43.763447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:59:42.424421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.488120+0000","last_change":"2023-10-09T06:30:43.484481+0000","last_active":"2023-10-09T06:30:43.488120+0000","last_peered":"2023-10-09T06:30:43.488120+0000","last_clean":"2023-10-09T06:30:43.488120+0000","last_became_active":"2023-10-09T06:30:43.483640+0000","last_became_peered":"2023-10-09T06:30:43.483640+0000","last_unstale":"2023-10-09T06:30:43.488120+0000","last_undegraded":"2023-10-09T06:30:43.488120+0000","last_fullsized":"2023-10-09T06:30:43.488120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:38:10.342140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.474106+0000","last_change":"2023-10-09T06:30:42.758834+0000","last_active":"2023-10-09T06:30:43.474106+0000","last_peered":"2023-10-09T06:30:43.474106+0000","last_clean":"2023-10-09T06:30:43.474106+0000","last_became_active":"2023-10-09T06:30:42.758561+0000","last_became_peered":"2023-10-09T06:30:42.758561+0000","last_unstale":"2023-10-09T06:30:43.474106+0000","last_undegraded":"2023-10-09T06:30:43.474106+0000","last_fullsized":"2023-10-09T06:30:43.474106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:46:54.709506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.490866+0000","last_change":"2023-10-09T06:30:43.483623+0000","last_active":"2023-10-09T06:30:43.490866+0000","last_peered":"2023-10-09T06:30:43.490866+0000","last_clean":"2023-10-09T06:30:43.490866+0000","last_became_active":"2023-10-09T06:30:43.483435+0000","last_became_peered":"2023-10-09T06:30:43.483435+0000","last_unstale":"2023-10-09T06:30:43.490866+0000","last_undegraded":"2023-10-09T06:30:43.490866+0000","last_fullsized":"2023-10-09T06:30:43.490866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:57:57.207775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.485588+0000","last_change":"2023-10-09T06:30:42.761685+0000","last_active":"2023-10-09T06:30:43.485588+0000","last_peered":"2023-10-09T06:30:43.485588+0000","last_clean":"2023-10-09T06:30:43.485588+0000","last_became_active":"2023-10-09T06:30:42.761432+0000","last_became_peered":"2023-10-09T06:30:42.761432+0000","last_unstale":"2023-10-09T06:30:43.485588+0000","last_undegraded":"2023-10-09T06:30:43.485588+0000","last_fullsized":"2023-10-09T06:30:43.485588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:11:08.967445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.486800+0000","last_change":"2023-10-09T06:30:43.483621+0000","last_active":"2023-10-09T06:30:43.486800+0000","last_peered":"2023-10-09T06:30:43.486800+0000","last_clean":"2023-10-09T06:30:43.486800+0000","last_became_active":"2023-10-09T06:30:43.483365+0000","last_became_peered":"2023-10-09T06:30:43.483365+0000","last_unstale":"2023-10-09T06:30:43.486800+0000","last_undegraded":"2023-10-09T06:30:43.486800+0000","last_fullsized":"2023-10-09T06:30:43.486800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:57:59.094337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.470554+0000","last_change":"2023-10-09T06:30:42.757148+0000","last_active":"2023-10-09T06:30:43.470554+0000","last_peered":"2023-10-09T06:30:43.470554+0000","last_clean":"2023-10-09T06:30:43.470554+0000","last_became_active":"2023-10-09T06:30:42.756630+0000","last_became_peered":"2023-10-09T06:30:42.756630+0000","last_unstale":"2023-10-09T06:30:43.470554+0000","last_undegraded":"2023-10-09T06:30:43.470554+0000","last_fullsized":"2023-10-09T06:30:43.470554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:11:08.967445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496741+0000","last_change":"2023-10-09T06:30:43.492800+0000","last_active":"2023-10-09T06:30:43.496741+0000","last_peered":"2023-10-09T06:30:43.496741+0000","last_clean":"2023-10-09T06:30:43.496741+0000","last_became_active":"2023-10-09T06:30:43.492567+0000","last_became_peered":"2023-10-09T06:30:43.492567+0000","last_unstale":"2023-10-09T06:30:43.496741+0000","last_undegraded":"2023-10-09T06:30:43.496741+0000","last_fullsized":"2023-10-09T06:30:43.496741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:05:59.526736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.482776+0000","last_change":"2023-10-09T06:30:42.759966+0000","last_active":"2023-10-09T06:30:43.482776+0000","last_peered":"2023-10-09T06:30:43.482776+0000","last_clean":"2023-10-09T06:30:43.482776+0000","last_became_active":"2023-10-09T06:30:42.759686+0000","last_became_peered":"2023-10-09T06:30:42.759686+0000","last_unstale":"2023-10-09T06:30:43.482776+0000","last_undegraded":"2023-10-09T06:30:43.482776+0000","last_fullsized":"2023-10-09T06:30:43.482776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:46:54.709506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493906+0000","last_change":"2023-10-09T06:30:43.491580+0000","last_active":"2023-10-09T06:30:43.493906+0000","last_peered":"2023-10-09T06:30:43.493906+0000","last_clean":"2023-10-09T06:30:43.493906+0000","last_became_active":"2023-10-09T06:30:43.491323+0000","last_became_peered":"2023-10-09T06:30:43.491323+0000","last_unstale":"2023-10-09T06:30:43.493906+0000","last_undegraded":"2023-10-09T06:30:43.493906+0000","last_fullsized":"2023-10-09T06:30:43.493906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:02:03.930820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.761837+0000","last_change":"2023-10-09T06:30:42.758290+0000","last_active":"2023-10-09T06:30:43.761837+0000","last_peered":"2023-10-09T06:30:43.761837+0000","last_clean":"2023-10-09T06:30:43.761837+0000","last_became_active":"2023-10-09T06:30:42.758008+0000","last_became_peered":"2023-10-09T06:30:42.758008+0000","last_unstale":"2023-10-09T06:30:43.761837+0000","last_undegraded":"2023-10-09T06:30:43.761837+0000","last_fullsized":"2023-10-09T06:30:43.761837+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:20:58.653921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.483249+0000","last_change":"2023-10-09T06:30:43.478610+0000","last_active":"2023-10-09T06:30:43.483249+0000","last_peered":"2023-10-09T06:30:43.483249+0000","last_clean":"2023-10-09T06:30:43.483249+0000","last_became_active":"2023-10-09T06:30:43.477942+0000","last_became_peered":"2023-10-09T06:30:43.477942+0000","last_unstale":"2023-10-09T06:30:43.483249+0000","last_undegraded":"2023-10-09T06:30:43.483249+0000","last_fullsized":"2023-10-09T06:30:43.483249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:29:12.176820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.766388+0000","last_change":"2023-10-09T06:30:42.759743+0000","last_active":"2023-10-09T06:30:43.766388+0000","last_peered":"2023-10-09T06:30:43.766388+0000","last_clean":"2023-10-09T06:30:43.766388+0000","last_became_active":"2023-10-09T06:30:42.757647+0000","last_became_peered":"2023-10-09T06:30:42.757647+0000","last_unstale":"2023-10-09T06:30:43.766388+0000","last_undegraded":"2023-10-09T06:30:43.766388+0000","last_fullsized":"2023-10-09T06:30:43.766388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:46:54.709506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.485759+0000","last_change":"2023-10-09T06:30:43.484192+0000","last_active":"2023-10-09T06:30:43.485759+0000","last_peered":"2023-10-09T06:30:43.485759+0000","last_clean":"2023-10-09T06:30:43.485759+0000","last_became_active":"2023-10-09T06:30:43.483935+0000","last_became_peered":"2023-10-09T06:30:43.483935+0000","last_unstale":"2023-10-09T06:30:43.485759+0000","last_undegraded":"2023-10-09T06:30:43.485759+0000","last_fullsized":"2023-10-09T06:30:43.485759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:36:01.020241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.763949+0000","last_change":"2023-10-09T06:30:42.760604+0000","last_active":"2023-10-09T06:30:43.763949+0000","last_peered":"2023-10-09T06:30:43.763949+0000","last_clean":"2023-10-09T06:30:43.763949+0000","last_became_active":"2023-10-09T06:30:42.760363+0000","last_became_peered":"2023-10-09T06:30:42.760363+0000","last_unstale":"2023-10-09T06:30:43.763949+0000","last_undegraded":"2023-10-09T06:30:43.763949+0000","last_fullsized":"2023-10-09T06:30:43.763949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:26:03.347834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.484683+0000","last_change":"2023-10-09T06:30:43.479684+0000","last_active":"2023-10-09T06:30:43.484683+0000","last_peered":"2023-10-09T06:30:43.484683+0000","last_clean":"2023-10-09T06:30:43.484683+0000","last_became_active":"2023-10-09T06:30:43.479262+0000","last_became_peered":"2023-10-09T06:30:43.479262+0000","last_unstale":"2023-10-09T06:30:43.484683+0000","last_undegraded":"2023-10-09T06:30:43.484683+0000","last_fullsized":"2023-10-09T06:30:43.484683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:09:10.695002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.482211+0000","last_change":"2023-10-09T06:30:42.762087+0000","last_active":"2023-10-09T06:30:43.482211+0000","last_peered":"2023-10-09T06:30:43.482211+0000","last_clean":"2023-10-09T06:30:43.482211+0000","last_became_active":"2023-10-09T06:30:42.761818+0000","last_became_peered":"2023-10-09T06:30:42.761818+0000","last_unstale":"2023-10-09T06:30:43.482211+0000","last_undegraded":"2023-10-09T06:30:43.482211+0000","last_fullsized":"2023-10-09T06:30:43.482211+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:45:19.887880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.486923+0000","last_change":"2023-10-09T06:30:43.483220+0000","last_active":"2023-10-09T06:30:43.486923+0000","last_peered":"2023-10-09T06:30:43.486923+0000","last_clean":"2023-10-09T06:30:43.486923+0000","last_became_active":"2023-10-09T06:30:43.482733+0000","last_became_peered":"2023-10-09T06:30:43.482733+0000","last_unstale":"2023-10-09T06:30:43.486923+0000","last_undegraded":"2023-10-09T06:30:43.486923+0000","last_fullsized":"2023-10-09T06:30:43.486923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:43:44.457796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.470076+0000","last_change":"2023-10-09T06:30:42.759270+0000","last_active":"2023-10-09T06:30:43.470076+0000","last_peered":"2023-10-09T06:30:43.470076+0000","last_clean":"2023-10-09T06:30:43.470076+0000","last_became_active":"2023-10-09T06:30:42.758836+0000","last_became_peered":"2023-10-09T06:30:42.758836+0000","last_unstale":"2023-10-09T06:30:43.470076+0000","last_undegraded":"2023-10-09T06:30:43.470076+0000","last_fullsized":"2023-10-09T06:30:43.470076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:36:40.691450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.484511+0000","last_change":"2023-10-09T06:30:43.480359+0000","last_active":"2023-10-09T06:30:43.484511+0000","last_peered":"2023-10-09T06:30:43.484511+0000","last_clean":"2023-10-09T06:30:43.484511+0000","last_became_active":"2023-10-09T06:30:43.480040+0000","last_became_peered":"2023-10-09T06:30:43.480040+0000","last_unstale":"2023-10-09T06:30:43.484511+0000","last_undegraded":"2023-10-09T06:30:43.484511+0000","last_fullsized":"2023-10-09T06:30:43.484511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:06:29.986154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.764519+0000","last_change":"2023-10-09T06:30:42.756126+0000","last_active":"2023-10-09T06:30:43.764519+0000","last_peered":"2023-10-09T06:30:43.764519+0000","last_clean":"2023-10-09T06:30:43.764519+0000","last_became_active":"2023-10-09T06:30:42.755132+0000","last_became_peered":"2023-10-09T06:30:42.755132+0000","last_unstale":"2023-10-09T06:30:43.764519+0000","last_undegraded":"2023-10-09T06:30:43.764519+0000","last_fullsized":"2023-10-09T06:30:43.764519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:45:19.887880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.492004+0000","last_change":"2023-10-09T06:30:43.487909+0000","last_active":"2023-10-09T06:30:43.492004+0000","last_peered":"2023-10-09T06:30:43.492004+0000","last_clean":"2023-10-09T06:30:43.492004+0000","last_became_active":"2023-10-09T06:30:43.487634+0000","last_became_peered":"2023-10-09T06:30:43.487634+0000","last_unstale":"2023-10-09T06:30:43.492004+0000","last_undegraded":"2023-10-09T06:30:43.492004+0000","last_fullsized":"2023-10-09T06:30:43.492004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:41:00.854052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.760383+0000","last_change":"2023-10-09T06:30:42.756538+0000","last_active":"2023-10-09T06:30:43.760383+0000","last_peered":"2023-10-09T06:30:43.760383+0000","last_clean":"2023-10-09T06:30:43.760383+0000","last_became_active":"2023-10-09T06:30:42.756109+0000","last_became_peered":"2023-10-09T06:30:42.756109+0000","last_unstale":"2023-10-09T06:30:43.760383+0000","last_undegraded":"2023-10-09T06:30:43.760383+0000","last_fullsized":"2023-10-09T06:30:43.760383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:30:02.609828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.485735+0000","last_change":"2023-10-09T06:30:43.483554+0000","last_active":"2023-10-09T06:30:43.485735+0000","last_peered":"2023-10-09T06:30:43.485735+0000","last_clean":"2023-10-09T06:30:43.485735+0000","last_became_active":"2023-10-09T06:30:43.483342+0000","last_became_peered":"2023-10-09T06:30:43.483342+0000","last_unstale":"2023-10-09T06:30:43.485735+0000","last_undegraded":"2023-10-09T06:30:43.485735+0000","last_fullsized":"2023-10-09T06:30:43.485735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:06:04.546054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.478131+0000","last_change":"2023-10-09T06:30:42.754490+0000","last_active":"2023-10-09T06:30:43.478131+0000","last_peered":"2023-10-09T06:30:43.478131+0000","last_clean":"2023-10-09T06:30:43.478131+0000","last_became_active":"2023-10-09T06:30:42.754238+0000","last_became_peered":"2023-10-09T06:30:42.754238+0000","last_unstale":"2023-10-09T06:30:43.478131+0000","last_undegraded":"2023-10-09T06:30:43.478131+0000","last_fullsized":"2023-10-09T06:30:43.478131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:45:19.887880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496827+0000","last_change":"2023-10-09T06:30:43.493269+0000","last_active":"2023-10-09T06:30:43.496827+0000","last_peered":"2023-10-09T06:30:43.496827+0000","last_clean":"2023-10-09T06:30:43.496827+0000","last_became_active":"2023-10-09T06:30:43.491762+0000","last_became_peered":"2023-10-09T06:30:43.491762+0000","last_unstale":"2023-10-09T06:30:43.496827+0000","last_undegraded":"2023-10-09T06:30:43.496827+0000","last_fullsized":"2023-10-09T06:30:43.496827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:10:42.930778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.482114+0000","last_change":"2023-10-09T06:30:42.763709+0000","last_active":"2023-10-09T06:30:43.482114+0000","last_peered":"2023-10-09T06:30:43.482114+0000","last_clean":"2023-10-09T06:30:43.482114+0000","last_became_active":"2023-10-09T06:30:42.763455+0000","last_became_peered":"2023-10-09T06:30:42.763455+0000","last_unstale":"2023-10-09T06:30:43.482114+0000","last_undegraded":"2023-10-09T06:30:43.482114+0000","last_fullsized":"2023-10-09T06:30:43.482114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:59:42.424421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.490783+0000","last_change":"2023-10-09T06:30:43.486278+0000","last_active":"2023-10-09T06:30:43.490783+0000","last_peered":"2023-10-09T06:30:43.490783+0000","last_clean":"2023-10-09T06:30:43.490783+0000","last_became_active":"2023-10-09T06:30:43.485877+0000","last_became_peered":"2023-10-09T06:30:43.485877+0000","last_unstale":"2023-10-09T06:30:43.490783+0000","last_undegraded":"2023-10-09T06:30:43.490783+0000","last_fullsized":"2023-10-09T06:30:43.490783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:55:20.971921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.486699+0000","last_change":"2023-10-09T06:30:43.482444+0000","last_active":"2023-10-09T06:30:43.486699+0000","last_peered":"2023-10-09T06:30:43.486699+0000","last_clean":"2023-10-09T06:30:43.486699+0000","last_became_active":"2023-10-09T06:30:43.482168+0000","last_became_peered":"2023-10-09T06:30:43.482168+0000","last_unstale":"2023-10-09T06:30:43.486699+0000","last_undegraded":"2023-10-09T06:30:43.486699+0000","last_fullsized":"2023-10-09T06:30:43.486699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:56:48.342402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.476414+0000","last_change":"2023-10-09T06:30:42.755534+0000","last_active":"2023-10-09T06:30:43.476414+0000","last_peered":"2023-10-09T06:30:43.476414+0000","last_clean":"2023-10-09T06:30:43.476414+0000","last_became_active":"2023-10-09T06:30:42.755246+0000","last_became_peered":"2023-10-09T06:30:42.755246+0000","last_unstale":"2023-10-09T06:30:43.476414+0000","last_undegraded":"2023-10-09T06:30:43.476414+0000","last_fullsized":"2023-10-09T06:30:43.476414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:52:48.682306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493672+0000","last_change":"2023-10-09T06:30:43.488993+0000","last_active":"2023-10-09T06:30:43.493672+0000","last_peered":"2023-10-09T06:30:43.493672+0000","last_clean":"2023-10-09T06:30:43.493672+0000","last_became_active":"2023-10-09T06:30:43.488701+0000","last_became_peered":"2023-10-09T06:30:43.488701+0000","last_unstale":"2023-10-09T06:30:43.493672+0000","last_undegraded":"2023-10-09T06:30:43.493672+0000","last_fullsized":"2023-10-09T06:30:43.493672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:05:59.526736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.479498+0000","last_change":"2023-10-09T06:30:42.757897+0000","last_active":"2023-10-09T06:30:43.479498+0000","last_peered":"2023-10-09T06:30:43.479498+0000","last_clean":"2023-10-09T06:30:43.479498+0000","last_became_active":"2023-10-09T06:30:42.757432+0000","last_became_peered":"2023-10-09T06:30:42.757432+0000","last_unstale":"2023-10-09T06:30:43.479498+0000","last_undegraded":"2023-10-09T06:30:43.479498+0000","last_fullsized":"2023-10-09T06:30:43.479498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:36:40.691450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":163,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.474256+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:30:43.474256+0000","last_peered":"2023-10-09T06:30:43.474256+0000","last_clean":"2023-10-09T06:30:43.474256+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:30:43.474256+0000","last_undegraded":"2023-10-09T06:30:43.474256+0000","last_fullsized":"2023-10-09T06:30:43.474256+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.484921+0000","last_change":"2023-10-09T06:30:43.479970+0000","last_active":"2023-10-09T06:30:43.484921+0000","last_peered":"2023-10-09T06:30:43.484921+0000","last_clean":"2023-10-09T06:30:43.484921+0000","last_became_active":"2023-10-09T06:30:43.479731+0000","last_became_peered":"2023-10-09T06:30:43.479731+0000","last_unstale":"2023-10-09T06:30:43.484921+0000","last_undegraded":"2023-10-09T06:30:43.484921+0000","last_fullsized":"2023-10-09T06:30:43.484921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:11:59.504301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.475172+0000","last_change":"2023-10-09T06:30:42.763408+0000","last_active":"2023-10-09T06:30:43.475172+0000","last_peered":"2023-10-09T06:30:43.475172+0000","last_clean":"2023-10-09T06:30:43.475172+0000","last_became_active":"2023-10-09T06:30:42.763129+0000","last_became_peered":"2023-10-09T06:30:42.763129+0000","last_unstale":"2023-10-09T06:30:43.475172+0000","last_undegraded":"2023-10-09T06:30:43.475172+0000","last_fullsized":"2023-10-09T06:30:43.475172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:29:11.172110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.475746+0000","last_change":"2023-10-09T06:30:42.766229+0000","last_active":"2023-10-09T06:30:43.475746+0000","last_peered":"2023-10-09T06:30:43.475746+0000","last_clean":"2023-10-09T06:30:43.475746+0000","last_became_active":"2023-10-09T06:30:42.765968+0000","last_became_peered":"2023-10-09T06:30:42.765968+0000","last_unstale":"2023-10-09T06:30:43.475746+0000","last_undegraded":"2023-10-09T06:30:43.475746+0000","last_fullsized":"2023-10-09T06:30:43.475746+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:59:42.424421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.486657+0000","last_change":"2023-10-09T06:30:43.482450+0000","last_active":"2023-10-09T06:30:43.486657+0000","last_peered":"2023-10-09T06:30:43.486657+0000","last_clean":"2023-10-09T06:30:43.486657+0000","last_became_active":"2023-10-09T06:30:43.482187+0000","last_became_peered":"2023-10-09T06:30:43.482187+0000","last_unstale":"2023-10-09T06:30:43.486657+0000","last_undegraded":"2023-10-09T06:30:43.486657+0000","last_fullsized":"2023-10-09T06:30:43.486657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:09:10.695002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-09T06:30:44.472751+0000","last_change":"2023-10-09T06:30:42.759163+0000","last_active":"2023-10-09T06:30:44.472751+0000","last_peered":"2023-10-09T06:30:44.472751+0000","last_clean":"2023-10-09T06:30:44.472751+0000","last_became_active":"2023-10-09T06:30:42.758583+0000","last_became_peered":"2023-10-09T06:30:42.758583+0000","last_unstale":"2023-10-09T06:30:44.472751+0000","last_undegraded":"2023-10-09T06:30:44.472751+0000","last_fullsized":"2023-10-09T06:30:44.472751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:36:40.691450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.487871+0000","last_change":"2023-10-09T06:30:43.480222+0000","last_active":"2023-10-09T06:30:43.487871+0000","last_peered":"2023-10-09T06:30:43.487871+0000","last_clean":"2023-10-09T06:30:43.487871+0000","last_became_active":"2023-10-09T06:30:43.479958+0000","last_became_peered":"2023-10-09T06:30:43.479958+0000","last_unstale":"2023-10-09T06:30:43.487871+0000","last_undegraded":"2023-10-09T06:30:43.487871+0000","last_fullsized":"2023-10-09T06:30:43.487871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:57:59.094337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.761656+0000","last_change":"2023-10-09T06:30:42.757526+0000","last_active":"2023-10-09T06:30:43.761656+0000","last_peered":"2023-10-09T06:30:43.761656+0000","last_clean":"2023-10-09T06:30:43.761656+0000","last_became_active":"2023-10-09T06:30:42.756934+0000","last_became_peered":"2023-10-09T06:30:42.756934+0000","last_unstale":"2023-10-09T06:30:43.761656+0000","last_undegraded":"2023-10-09T06:30:43.761656+0000","last_fullsized":"2023-10-09T06:30:43.761656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:45:19.887880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.487633+0000","last_change":"2023-10-09T06:30:43.482622+0000","last_active":"2023-10-09T06:30:43.487633+0000","last_peered":"2023-10-09T06:30:43.487633+0000","last_clean":"2023-10-09T06:30:43.487633+0000","last_became_active":"2023-10-09T06:30:43.482362+0000","last_became_peered":"2023-10-09T06:30:43.482362+0000","last_unstale":"2023-10-09T06:30:43.487633+0000","last_undegraded":"2023-10-09T06:30:43.487633+0000","last_fullsized":"2023-10-09T06:30:43.487633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:15:54.706732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.474220+0000","last_change":"2023-10-09T06:30:42.764161+0000","last_active":"2023-10-09T06:30:43.474220+0000","last_peered":"2023-10-09T06:30:43.474220+0000","last_clean":"2023-10-09T06:30:43.474220+0000","last_became_active":"2023-10-09T06:30:42.763872+0000","last_became_peered":"2023-10-09T06:30:42.763872+0000","last_unstale":"2023-10-09T06:30:43.474220+0000","last_undegraded":"2023-10-09T06:30:43.474220+0000","last_fullsized":"2023-10-09T06:30:43.474220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:36:40.691450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.491559+0000","last_change":"2023-10-09T06:30:43.487127+0000","last_active":"2023-10-09T06:30:43.491559+0000","last_peered":"2023-10-09T06:30:43.491559+0000","last_clean":"2023-10-09T06:30:43.491559+0000","last_became_active":"2023-10-09T06:30:43.486428+0000","last_became_peered":"2023-10-09T06:30:43.486428+0000","last_unstale":"2023-10-09T06:30:43.491559+0000","last_undegraded":"2023-10-09T06:30:43.491559+0000","last_fullsized":"2023-10-09T06:30:43.491559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:59:43.429131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.478013+0000","last_change":"2023-10-09T06:30:42.758491+0000","last_active":"2023-10-09T06:30:43.478013+0000","last_peered":"2023-10-09T06:30:43.478013+0000","last_clean":"2023-10-09T06:30:43.478013+0000","last_became_active":"2023-10-09T06:30:42.757552+0000","last_became_peered":"2023-10-09T06:30:42.757552+0000","last_unstale":"2023-10-09T06:30:43.478013+0000","last_undegraded":"2023-10-09T06:30:43.478013+0000","last_fullsized":"2023-10-09T06:30:43.478013+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:11:08.967445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.490847+0000","last_change":"2023-10-09T06:30:43.483055+0000","last_active":"2023-10-09T06:30:43.490847+0000","last_peered":"2023-10-09T06:30:43.490847+0000","last_clean":"2023-10-09T06:30:43.490847+0000","last_became_active":"2023-10-09T06:30:43.482794+0000","last_became_peered":"2023-10-09T06:30:43.482794+0000","last_unstale":"2023-10-09T06:30:43.490847+0000","last_undegraded":"2023-10-09T06:30:43.490847+0000","last_fullsized":"2023-10-09T06:30:43.490847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:12:09.812942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.478093+0000","last_change":"2023-10-09T06:30:42.755096+0000","last_active":"2023-10-09T06:30:43.478093+0000","last_peered":"2023-10-09T06:30:43.478093+0000","last_clean":"2023-10-09T06:30:43.478093+0000","last_became_active":"2023-10-09T06:30:42.754834+0000","last_became_peered":"2023-10-09T06:30:42.754834+0000","last_unstale":"2023-10-09T06:30:43.478093+0000","last_undegraded":"2023-10-09T06:30:43.478093+0000","last_fullsized":"2023-10-09T06:30:43.478093+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:52:48.682306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.491004+0000","last_change":"2023-10-09T06:30:43.485563+0000","last_active":"2023-10-09T06:30:43.491004+0000","last_peered":"2023-10-09T06:30:43.491004+0000","last_clean":"2023-10-09T06:30:43.491004+0000","last_became_active":"2023-10-09T06:30:43.485051+0000","last_became_peered":"2023-10-09T06:30:43.485051+0000","last_unstale":"2023-10-09T06:30:43.491004+0000","last_undegraded":"2023-10-09T06:30:43.491004+0000","last_fullsized":"2023-10-09T06:30:43.491004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:36:01.020241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.763572+0000","last_change":"2023-10-09T06:30:42.762583+0000","last_active":"2023-10-09T06:30:43.763572+0000","last_peered":"2023-10-09T06:30:43.763572+0000","last_clean":"2023-10-09T06:30:43.763572+0000","last_became_active":"2023-10-09T06:30:42.762319+0000","last_became_peered":"2023-10-09T06:30:42.762319+0000","last_unstale":"2023-10-09T06:30:43.763572+0000","last_undegraded":"2023-10-09T06:30:43.763572+0000","last_fullsized":"2023-10-09T06:30:43.763572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:15:33.621205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.484629+0000","last_change":"2023-10-09T06:30:43.481138+0000","last_active":"2023-10-09T06:30:43.484629+0000","last_peered":"2023-10-09T06:30:43.484629+0000","last_clean":"2023-10-09T06:30:43.484629+0000","last_became_active":"2023-10-09T06:30:43.480789+0000","last_became_peered":"2023-10-09T06:30:43.480789+0000","last_unstale":"2023-10-09T06:30:43.484629+0000","last_undegraded":"2023-10-09T06:30:43.484629+0000","last_fullsized":"2023-10-09T06:30:43.484629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:12:09.812942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.475749+0000","last_change":"2023-10-09T06:30:42.754630+0000","last_active":"2023-10-09T06:30:43.475749+0000","last_peered":"2023-10-09T06:30:43.475749+0000","last_clean":"2023-10-09T06:30:43.475749+0000","last_became_active":"2023-10-09T06:30:42.754309+0000","last_became_peered":"2023-10-09T06:30:42.754309+0000","last_unstale":"2023-10-09T06:30:43.475749+0000","last_undegraded":"2023-10-09T06:30:43.475749+0000","last_fullsized":"2023-10-09T06:30:43.475749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:46:54.709506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496592+0000","last_change":"2023-10-09T06:30:43.492044+0000","last_active":"2023-10-09T06:30:43.496592+0000","last_peered":"2023-10-09T06:30:43.496592+0000","last_clean":"2023-10-09T06:30:43.496592+0000","last_became_active":"2023-10-09T06:30:43.491715+0000","last_became_peered":"2023-10-09T06:30:43.491715+0000","last_unstale":"2023-10-09T06:30:43.496592+0000","last_undegraded":"2023-10-09T06:30:43.496592+0000","last_fullsized":"2023-10-09T06:30:43.496592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:10:42.930778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.764217+0000","last_change":"2023-10-09T06:30:42.759144+0000","last_active":"2023-10-09T06:30:43.764217+0000","last_peered":"2023-10-09T06:30:43.764217+0000","last_clean":"2023-10-09T06:30:43.764217+0000","last_became_active":"2023-10-09T06:30:42.758729+0000","last_became_peered":"2023-10-09T06:30:42.758729+0000","last_unstale":"2023-10-09T06:30:43.764217+0000","last_undegraded":"2023-10-09T06:30:43.764217+0000","last_fullsized":"2023-10-09T06:30:43.764217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:46:54.709506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496783+0000","last_change":"2023-10-09T06:30:43.493008+0000","last_active":"2023-10-09T06:30:43.496783+0000","last_peered":"2023-10-09T06:30:43.496783+0000","last_clean":"2023-10-09T06:30:43.496783+0000","last_became_active":"2023-10-09T06:30:43.491584+0000","last_became_peered":"2023-10-09T06:30:43.491584+0000","last_unstale":"2023-10-09T06:30:43.496783+0000","last_undegraded":"2023-10-09T06:30:43.496783+0000","last_fullsized":"2023-10-09T06:30:43.496783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:43:44.457796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.762576+0000","last_change":"2023-10-09T06:30:42.756415+0000","last_active":"2023-10-09T06:30:43.762576+0000","last_peered":"2023-10-09T06:30:43.762576+0000","last_clean":"2023-10-09T06:30:43.762576+0000","last_became_active":"2023-10-09T06:30:42.755726+0000","last_became_peered":"2023-10-09T06:30:42.755726+0000","last_unstale":"2023-10-09T06:30:43.762576+0000","last_undegraded":"2023-10-09T06:30:43.762576+0000","last_fullsized":"2023-10-09T06:30:43.762576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:26:03.347834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.485808+0000","last_change":"2023-10-09T06:30:43.482852+0000","last_active":"2023-10-09T06:30:43.485808+0000","last_peered":"2023-10-09T06:30:43.485808+0000","last_clean":"2023-10-09T06:30:43.485808+0000","last_became_active":"2023-10-09T06:30:43.482581+0000","last_became_peered":"2023-10-09T06:30:43.482581+0000","last_unstale":"2023-10-09T06:30:43.485808+0000","last_undegraded":"2023-10-09T06:30:43.485808+0000","last_fullsized":"2023-10-09T06:30:43.485808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:03:07.803820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.490571+0000","last_change":"2023-10-09T06:30:43.485037+0000","last_active":"2023-10-09T06:30:43.490571+0000","last_peered":"2023-10-09T06:30:43.490571+0000","last_clean":"2023-10-09T06:30:43.490571+0000","last_became_active":"2023-10-09T06:30:43.484747+0000","last_became_peered":"2023-10-09T06:30:43.484747+0000","last_unstale":"2023-10-09T06:30:43.490571+0000","last_undegraded":"2023-10-09T06:30:43.490571+0000","last_fullsized":"2023-10-09T06:30:43.490571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:57:59.094337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493668+0000","last_change":"2023-10-09T06:30:43.488771+0000","last_active":"2023-10-09T06:30:43.493668+0000","last_peered":"2023-10-09T06:30:43.493668+0000","last_clean":"2023-10-09T06:30:43.493668+0000","last_became_active":"2023-10-09T06:30:43.488505+0000","last_became_peered":"2023-10-09T06:30:43.488505+0000","last_unstale":"2023-10-09T06:30:43.493668+0000","last_undegraded":"2023-10-09T06:30:43.493668+0000","last_fullsized":"2023-10-09T06:30:43.493668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:20:59.658630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496662+0000","last_change":"2023-10-09T06:30:43.492057+0000","last_active":"2023-10-09T06:30:43.496662+0000","last_peered":"2023-10-09T06:30:43.496662+0000","last_clean":"2023-10-09T06:30:43.496662+0000","last_became_active":"2023-10-09T06:30:43.491469+0000","last_became_peered":"2023-10-09T06:30:43.491469+0000","last_unstale":"2023-10-09T06:30:43.496662+0000","last_undegraded":"2023-10-09T06:30:43.496662+0000","last_fullsized":"2023-10-09T06:30:43.496662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:59:57.434765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.488149+0000","last_change":"2023-10-09T06:30:43.484938+0000","last_active":"2023-10-09T06:30:43.488149+0000","last_peered":"2023-10-09T06:30:43.488149+0000","last_clean":"2023-10-09T06:30:43.488149+0000","last_became_active":"2023-10-09T06:30:43.484188+0000","last_became_peered":"2023-10-09T06:30:43.484188+0000","last_unstale":"2023-10-09T06:30:43.488149+0000","last_undegraded":"2023-10-09T06:30:43.488149+0000","last_fullsized":"2023-10-09T06:30:43.488149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:57:59.094337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.488046+0000","last_change":"2023-10-09T06:30:43.481165+0000","last_active":"2023-10-09T06:30:43.488046+0000","last_peered":"2023-10-09T06:30:43.488046+0000","last_clean":"2023-10-09T06:30:43.488046+0000","last_became_active":"2023-10-09T06:30:43.480896+0000","last_became_peered":"2023-10-09T06:30:43.480896+0000","last_unstale":"2023-10-09T06:30:43.488046+0000","last_undegraded":"2023-10-09T06:30:43.488046+0000","last_fullsized":"2023-10-09T06:30:43.488046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:56:48.342402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.486824+0000","last_change":"2023-10-09T06:30:43.483125+0000","last_active":"2023-10-09T06:30:43.486824+0000","last_peered":"2023-10-09T06:30:43.486824+0000","last_clean":"2023-10-09T06:30:43.486824+0000","last_became_active":"2023-10-09T06:30:43.482886+0000","last_became_peered":"2023-10-09T06:30:43.482886+0000","last_unstale":"2023-10-09T06:30:43.486824+0000","last_undegraded":"2023-10-09T06:30:43.486824+0000","last_fullsized":"2023-10-09T06:30:43.486824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:15:54.706732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.484766+0000","last_change":"2023-10-09T06:30:43.479321+0000","last_active":"2023-10-09T06:30:43.484766+0000","last_peered":"2023-10-09T06:30:43.484766+0000","last_clean":"2023-10-09T06:30:43.484766+0000","last_became_active":"2023-10-09T06:30:43.477877+0000","last_became_peered":"2023-10-09T06:30:43.477877+0000","last_unstale":"2023-10-09T06:30:43.484766+0000","last_undegraded":"2023-10-09T06:30:43.484766+0000","last_fullsized":"2023-10-09T06:30:43.484766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:09:10.695002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.487894+0000","last_change":"2023-10-09T06:30:43.480511+0000","last_active":"2023-10-09T06:30:43.487894+0000","last_peered":"2023-10-09T06:30:43.487894+0000","last_clean":"2023-10-09T06:30:43.487894+0000","last_became_active":"2023-10-09T06:30:43.480250+0000","last_became_peered":"2023-10-09T06:30:43.480250+0000","last_unstale":"2023-10-09T06:30:43.487894+0000","last_undegraded":"2023-10-09T06:30:43.487894+0000","last_fullsized":"2023-10-09T06:30:43.487894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:15:54.706732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.491696+0000","last_change":"2023-10-09T06:30:43.488975+0000","last_active":"2023-10-09T06:30:43.491696+0000","last_peered":"2023-10-09T06:30:43.491696+0000","last_clean":"2023-10-09T06:30:43.491696+0000","last_became_active":"2023-10-09T06:30:43.488639+0000","last_became_peered":"2023-10-09T06:30:43.488639+0000","last_unstale":"2023-10-09T06:30:43.491696+0000","last_undegraded":"2023-10-09T06:30:43.491696+0000","last_fullsized":"2023-10-09T06:30:43.491696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:06:04.546054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.487823+0000","last_change":"2023-10-09T06:30:43.479862+0000","last_active":"2023-10-09T06:30:43.487823+0000","last_peered":"2023-10-09T06:30:43.487823+0000","last_clean":"2023-10-09T06:30:43.487823+0000","last_became_active":"2023-10-09T06:30:43.479597+0000","last_became_peered":"2023-10-09T06:30:43.479597+0000","last_unstale":"2023-10-09T06:30:43.487823+0000","last_undegraded":"2023-10-09T06:30:43.487823+0000","last_fullsized":"2023-10-09T06:30:43.487823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:09:10.695002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496763+0000","last_change":"2023-10-09T06:30:43.492872+0000","last_active":"2023-10-09T06:30:43.496763+0000","last_peered":"2023-10-09T06:30:43.496763+0000","last_clean":"2023-10-09T06:30:43.496763+0000","last_became_active":"2023-10-09T06:30:43.492634+0000","last_became_peered":"2023-10-09T06:30:43.492634+0000","last_unstale":"2023-10-09T06:30:43.496763+0000","last_undegraded":"2023-10-09T06:30:43.496763+0000","last_fullsized":"2023-10-09T06:30:43.496763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:02:03.930820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.488131+0000","last_change":"2023-10-09T06:30:43.484753+0000","last_active":"2023-10-09T06:30:43.488131+0000","last_peered":"2023-10-09T06:30:43.488131+0000","last_clean":"2023-10-09T06:30:43.488131+0000","last_became_active":"2023-10-09T06:30:43.483916+0000","last_became_peered":"2023-10-09T06:30:43.483916+0000","last_unstale":"2023-10-09T06:30:43.488131+0000","last_undegraded":"2023-10-09T06:30:43.488131+0000","last_fullsized":"2023-10-09T06:30:43.488131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:06:29.986154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.484622+0000","last_change":"2023-10-09T06:30:43.481252+0000","last_active":"2023-10-09T06:30:43.484622+0000","last_peered":"2023-10-09T06:30:43.484622+0000","last_clean":"2023-10-09T06:30:43.484622+0000","last_became_active":"2023-10-09T06:30:43.481008+0000","last_became_peered":"2023-10-09T06:30:43.481008+0000","last_unstale":"2023-10-09T06:30:43.484622+0000","last_undegraded":"2023-10-09T06:30:43.484622+0000","last_fullsized":"2023-10-09T06:30:43.484622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:02:03.930820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298072,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073768,"statfs":{"total":96636764160,"available":96331538432,"internally_reserved":0,"allocated":802816,"data_stored":661576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":72190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":212992,"data_stored":70814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298080,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073760,"statfs":{"total":96636764160,"available":96331530240,"internally_reserved":0,"allocated":811008,"data_stored":661632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":70654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":72190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":70780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":208896,"data_stored":70744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:30:52.314 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:30:52.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.461+0000 7fa0bfe24700 1 Processor -- start 2023-10-09T06:30:52.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.462+0000 7fa0bfe24700 1 -- start start 2023-10-09T06:30:52.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.462+0000 7fa0bfe24700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa0b806aa40 0x7fa0b80cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:52.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.462+0000 7fa0bfe24700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0b80c91e0 0x7fa0b8160390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:52.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.462+0000 7fa0bfe24700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0b80cc210 con 0x7fa0b806bac0 2023-10-09T06:30:52.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.462+0000 7fa0bfe24700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0b80cc4f0 con 0x7fa0b806aa40 2023-10-09T06:30:52.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.462+0000 7fa0bfe24700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b80cc7d0 con 0x7fa0b80c91e0 2023-10-09T06:30:52.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.463+0000 7fa0be3c1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0b80c91e0 0x7fa0b8160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:52.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.463+0000 7fa0bdbc0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa0b806aa40 0x7fa0b80cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:52.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.463+0000 7fa0bd3bf700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa0b806bac0 0x7fa0b806ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:47958/0 (socket says 172.21.15.163:47958) 2023-10-09T06:30:52.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.463+0000 7fa0bdbc0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa0b806aa40 0x7fa0b80cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:50634/0 (socket says 172.21.15.163:50634) 2023-10-09T06:30:52.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.463+0000 7fa0be3c1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0b80c91e0 0x7fa0b8160390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:44900/0 (socket says 172.21.15.163:44900) 2023-10-09T06:30:52.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.463+0000 7fa0bd3bf700 1 -- 172.21.15.163:0/2878656583 learned_addr learned my addr 172.21.15.163:0/2878656583 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:52.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.463+0000 7fa0bdbc0700 1 -- 172.21.15.163:0/2878656583 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0b80c91e0 msgr2=0x7fa0b8160390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:52.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.463+0000 7fa0bdbc0700 1 --2- 172.21.15.163:0/2878656583 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0b80c91e0 0x7fa0b8160390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:52.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.463+0000 7fa0bdbc0700 1 -- 172.21.15.163:0/2878656583 >> v1:172.21.15.181:6789/0 conn(0x7fa0b806bac0 legacy=0x7fa0b806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:52.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.463+0000 7fa0bdbc0700 1 -- 172.21.15.163:0/2878656583 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b80ccab0 con 0x7fa0b806aa40 2023-10-09T06:30:52.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.463+0000 7fa0bcbbe700 1 -- 172.21.15.163:0/2878656583 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1731158274 0 0) 0x7fa0b80cc210 con 0x7fa0b806bac0 2023-10-09T06:30:52.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.463+0000 7fa0be3c1700 1 --2- 172.21.15.163:0/2878656583 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0b80c91e0 0x7fa0b8160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:30:52.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.463+0000 7fa0bdbc0700 1 --2- 172.21.15.163:0/2878656583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0b806aa40 0x7fa0b80cbc00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40098d0 tx=0x7fa0b4009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5631e95e74da2fd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:52.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.463+0000 7fa0bcbbe700 1 -- 172.21.15.163:0/2878656583 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4005c10 con 0x7fa0b806aa40 2023-10-09T06:30:52.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.463+0000 7fa0bcbbe700 1 -- 172.21.15.163:0/2878656583 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b4005d70 con 0x7fa0b806aa40 2023-10-09T06:30:52.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.463+0000 7fa0bcbbe700 1 -- 172.21.15.163:0/2878656583 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4006280 con 0x7fa0b806aa40 2023-10-09T06:30:52.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.464+0000 7fa0bfe24700 1 -- 172.21.15.163:0/2878656583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0b806aa40 msgr2=0x7fa0b80cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:52.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.464+0000 7fa0bfe24700 1 --2- 172.21.15.163:0/2878656583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0b806aa40 0x7fa0b80cbc00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40098d0 tx=0x7fa0b4009120 comp rx=0 tx=0).stop 2023-10-09T06:30:52.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.464+0000 7fa0bfe24700 1 -- 172.21.15.163:0/2878656583 shutdown_connections 2023-10-09T06:30:52.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.464+0000 7fa0bfe24700 1 --2- 172.21.15.163:0/2878656583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0b806aa40 0x7fa0b80cbc00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:52.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.464+0000 7fa0bfe24700 1 --2- 172.21.15.163:0/2878656583 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0b80c91e0 0x7fa0b8160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:52.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.464+0000 7fa0bfe24700 1 -- 172.21.15.163:0/2878656583 >> 172.21.15.163:0/2878656583 conn(0x7fa0b80cd670 msgr2=0x7fa0b80cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:52.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.464+0000 7fa0bfe24700 1 -- 172.21.15.163:0/2878656583 shutdown_connections 2023-10-09T06:30:52.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.464+0000 7fa0bfe24700 1 -- 172.21.15.163:0/2878656583 wait complete. 2023-10-09T06:30:52.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.465+0000 7fa0bfe24700 1 Processor -- start 2023-10-09T06:30:52.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.465+0000 7fa0bfe24700 1 -- start start 2023-10-09T06:30:52.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.466+0000 7fa0bfe24700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0b806aa40 0x7fa0b8158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:52.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.466+0000 7fa0bfe24700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0b806bac0 0x7fa0b81681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:52.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.466+0000 7fa0bfe24700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0b80c91e0 0x7fa0b80c38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:52.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.466+0000 7fa0bfe24700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b814f9a0 con 0x7fa0b806aa40 2023-10-09T06:30:52.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.466+0000 7fa0bfe24700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b81686f0 con 0x7fa0b806bac0 2023-10-09T06:30:52.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.466+0000 7fa0bfe24700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b8158d10 con 0x7fa0b80c91e0 2023-10-09T06:30:52.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.466+0000 7fa0bdbc0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0b806aa40 0x7fa0b8158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:52.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.466+0000 7fa0bdbc0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0b806aa40 0x7fa0b8158710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44100/0 (socket says 172.21.15.163:44100) 2023-10-09T06:30:52.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.466+0000 7fa0bdbc0700 1 -- 172.21.15.163:0/2809110270 learned_addr learned my addr 172.21.15.163:0/2809110270 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:52.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.466+0000 7fa0bd3bf700 1 --2- 172.21.15.163:0/2809110270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0b806bac0 0x7fa0b81681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:52.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.466+0000 7fa0be3c1700 1 --2- 172.21.15.163:0/2809110270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0b80c91e0 0x7fa0b80c38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:52.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.466+0000 7fa0bdbc0700 1 -- 172.21.15.163:0/2809110270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0b80c91e0 msgr2=0x7fa0b80c38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:52.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.466+0000 7fa0bdbc0700 1 --2- 172.21.15.163:0/2809110270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0b80c91e0 0x7fa0b80c38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:52.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.466+0000 7fa0bdbc0700 1 -- 172.21.15.163:0/2809110270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0b806bac0 msgr2=0x7fa0b81681b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:52.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.466+0000 7fa0bdbc0700 1 --2- 172.21.15.163:0/2809110270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0b806bac0 0x7fa0b81681b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:52.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.466+0000 7fa0bdbc0700 1 -- 172.21.15.163:0/2809110270 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b0009a20 con 0x7fa0b806aa40 2023-10-09T06:30:52.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.466+0000 7fa0bdbc0700 1 --2- 172.21.15.163:0/2809110270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0b806aa40 0x7fa0b8158710 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009f70 tx=0x7fa0b4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:52.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.466+0000 7fa0adffb700 1 -- 172.21.15.163:0/2809110270 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4024070 con 0x7fa0b806aa40 2023-10-09T06:30:52.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.466+0000 7fa0adffb700 1 -- 172.21.15.163:0/2809110270 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b401da70 con 0x7fa0b806aa40 2023-10-09T06:30:52.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.466+0000 7fa0bfe24700 1 -- 172.21.15.163:0/2809110270 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b40093c0 con 0x7fa0b806aa40 2023-10-09T06:30:52.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.467+0000 7fa0adffb700 1 -- 172.21.15.163:0/2809110270 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b401de30 con 0x7fa0b806aa40 2023-10-09T06:30:52.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.467+0000 7fa0bfe24700 1 -- 172.21.15.163:0/2809110270 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b81592d0 con 0x7fa0b806aa40 2023-10-09T06:30:52.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.468+0000 7fa0bfe24700 1 -- 172.21.15.163:0/2809110270 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b804c3c0 con 0x7fa0b806aa40 2023-10-09T06:30:52.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.468+0000 7fa0adffb700 1 -- 172.21.15.163:0/2809110270 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa0b4011630 con 0x7fa0b806aa40 2023-10-09T06:30:52.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.468+0000 7fa0adffb700 1 --2- 172.21.15.163:0/2809110270 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa09c066f60 0x7fa09c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:52.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.468+0000 7fa0adffb700 1 -- 172.21.15.163:0/2809110270 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa0b40895d0 con 0x7fa0b806aa40 2023-10-09T06:30:52.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.469+0000 7fa0bd3bf700 1 --2- 172.21.15.163:0/2809110270 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa09c066f60 0x7fa09c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:52.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.469+0000 7fa0bd3bf700 1 --2- 172.21.15.163:0/2809110270 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa09c066f60 0x7fa09c069410 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa0b806e660 tx=0x7fa0a8007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:52.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.473+0000 7fa0adffb700 1 -- 172.21.15.163:0/2809110270 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa0b4056900 con 0x7fa0b806aa40 2023-10-09T06:30:52.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.653+0000 7fa0bfe24700 1 -- 172.21.15.163:0/2809110270 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa0b80c7cb0 con 0x7fa09c066f60 2023-10-09T06:30:52.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.660+0000 7fa0adffb700 1 -- 172.21.15.163:0/2809110270 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264518 (secure 0 0 0) 0x7fa0b80c7cb0 con 0x7fa09c066f60 2023-10-09T06:30:52.664 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:30:52.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.663+0000 7fa0bfe24700 1 -- 172.21.15.163:0/2809110270 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa09c066f60 msgr2=0x7fa09c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:52.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.663+0000 7fa0bfe24700 1 --2- 172.21.15.163:0/2809110270 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa09c066f60 0x7fa09c069410 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa0b806e660 tx=0x7fa0a8007d30 comp rx=0 tx=0).stop 2023-10-09T06:30:52.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.663+0000 7fa0bfe24700 1 -- 172.21.15.163:0/2809110270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0b806aa40 msgr2=0x7fa0b8158710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:52.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.663+0000 7fa0bfe24700 1 --2- 172.21.15.163:0/2809110270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0b806aa40 0x7fa0b8158710 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009f70 tx=0x7fa0b4016000 comp rx=0 tx=0).stop 2023-10-09T06:30:52.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.663+0000 7fa0bfe24700 1 -- 172.21.15.163:0/2809110270 shutdown_connections 2023-10-09T06:30:52.667 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.663+0000 7fa0bfe24700 1 --2- 172.21.15.163:0/2809110270 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa09c066f60 0x7fa09c069410 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:52.667 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.663+0000 7fa0bfe24700 1 --2- 172.21.15.163:0/2809110270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0b806aa40 0x7fa0b8158710 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:52.667 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.663+0000 7fa0bfe24700 1 --2- 172.21.15.163:0/2809110270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0b80c91e0 0x7fa0b80c38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:52.667 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.663+0000 7fa0bfe24700 1 --2- 172.21.15.163:0/2809110270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0b806bac0 0x7fa0b81681b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:52.667 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.663+0000 7fa0bfe24700 1 -- 172.21.15.163:0/2809110270 >> 172.21.15.163:0/2809110270 conn(0x7fa0b80cd670 msgr2=0x7fa0b80c2530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:52.667 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.663+0000 7fa0bfe24700 1 -- 172.21.15.163:0/2809110270 shutdown_connections 2023-10-09T06:30:52.667 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.663+0000 7fa0bfe24700 1 -- 172.21.15.163:0/2809110270 wait complete. 2023-10-09T06:30:52.667 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:30:52.676 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-09T06:30:52.296574+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381128,"kb_used_data":2824,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593592,"statfs":{"total":773094113280,"available":770655838208,"internally_reserved":0,"allocated":2891776,"data_stored":1750580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.828687"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.492500+0000","last_change":"2023-10-09T06:30:43.487077+0000","last_active":"2023-10-09T06:30:43.492500+0000","last_peered":"2023-10-09T06:30:43.492500+0000","last_clean":"2023-10-09T06:30:43.492500+0000","last_became_active":"2023-10-09T06:30:43.486817+0000","last_became_peered":"2023-10-09T06:30:43.486817+0000","last_unstale":"2023-10-09T06:30:43.492500+0000","last_undegraded":"2023-10-09T06:30:43.492500+0000","last_fullsized":"2023-10-09T06:30:43.492500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:59:57.434765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.486972+0000","last_change":"2023-10-09T06:30:43.484119+0000","last_active":"2023-10-09T06:30:43.486972+0000","last_peered":"2023-10-09T06:30:43.486972+0000","last_clean":"2023-10-09T06:30:43.486972+0000","last_became_active":"2023-10-09T06:30:43.483865+0000","last_became_peered":"2023-10-09T06:30:43.483865+0000","last_unstale":"2023-10-09T06:30:43.486972+0000","last_undegraded":"2023-10-09T06:30:43.486972+0000","last_fullsized":"2023-10-09T06:30:43.486972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:06:29.986154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.484893+0000","last_change":"2023-10-09T06:30:43.480754+0000","last_active":"2023-10-09T06:30:43.484893+0000","last_peered":"2023-10-09T06:30:43.484893+0000","last_clean":"2023-10-09T06:30:43.484893+0000","last_became_active":"2023-10-09T06:30:43.480474+0000","last_became_peered":"2023-10-09T06:30:43.480474+0000","last_unstale":"2023-10-09T06:30:43.484893+0000","last_undegraded":"2023-10-09T06:30:43.484893+0000","last_fullsized":"2023-10-09T06:30:43.484893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:56:01.192585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493176+0000","last_change":"2023-10-09T06:30:43.489012+0000","last_active":"2023-10-09T06:30:43.493176+0000","last_peered":"2023-10-09T06:30:43.493176+0000","last_clean":"2023-10-09T06:30:43.493176+0000","last_became_active":"2023-10-09T06:30:43.488614+0000","last_became_peered":"2023-10-09T06:30:43.488614+0000","last_unstale":"2023-10-09T06:30:43.493176+0000","last_undegraded":"2023-10-09T06:30:43.493176+0000","last_fullsized":"2023-10-09T06:30:43.493176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:41:00.854052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.486903+0000","last_change":"2023-10-09T06:30:43.483013+0000","last_active":"2023-10-09T06:30:43.486903+0000","last_peered":"2023-10-09T06:30:43.486903+0000","last_clean":"2023-10-09T06:30:43.486903+0000","last_became_active":"2023-10-09T06:30:43.482373+0000","last_became_peered":"2023-10-09T06:30:43.482373+0000","last_unstale":"2023-10-09T06:30:43.486903+0000","last_undegraded":"2023-10-09T06:30:43.486903+0000","last_fullsized":"2023-10-09T06:30:43.486903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:46:57.531379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.492270+0000","last_change":"2023-10-09T06:30:43.488813+0000","last_active":"2023-10-09T06:30:43.492270+0000","last_peered":"2023-10-09T06:30:43.492270+0000","last_clean":"2023-10-09T06:30:43.492270+0000","last_became_active":"2023-10-09T06:30:43.488532+0000","last_became_peered":"2023-10-09T06:30:43.488532+0000","last_unstale":"2023-10-09T06:30:43.492270+0000","last_undegraded":"2023-10-09T06:30:43.492270+0000","last_fullsized":"2023-10-09T06:30:43.492270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:57:57.207775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.491984+0000","last_change":"2023-10-09T06:30:43.486418+0000","last_active":"2023-10-09T06:30:43.491984+0000","last_peered":"2023-10-09T06:30:43.491984+0000","last_clean":"2023-10-09T06:30:43.491984+0000","last_became_active":"2023-10-09T06:30:43.486128+0000","last_became_peered":"2023-10-09T06:30:43.486128+0000","last_unstale":"2023-10-09T06:30:43.491984+0000","last_undegraded":"2023-10-09T06:30:43.491984+0000","last_fullsized":"2023-10-09T06:30:43.491984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:57:57.207775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493719+0000","last_change":"2023-10-09T06:30:43.490024+0000","last_active":"2023-10-09T06:30:43.493719+0000","last_peered":"2023-10-09T06:30:43.493719+0000","last_clean":"2023-10-09T06:30:43.493719+0000","last_became_active":"2023-10-09T06:30:43.489785+0000","last_became_peered":"2023-10-09T06:30:43.489785+0000","last_unstale":"2023-10-09T06:30:43.493719+0000","last_undegraded":"2023-10-09T06:30:43.493719+0000","last_fullsized":"2023-10-09T06:30:43.493719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:02:03.930820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496624+0000","last_change":"2023-10-09T06:30:43.492372+0000","last_active":"2023-10-09T06:30:43.496624+0000","last_peered":"2023-10-09T06:30:43.496624+0000","last_clean":"2023-10-09T06:30:43.496624+0000","last_became_active":"2023-10-09T06:30:43.492120+0000","last_became_peered":"2023-10-09T06:30:43.492120+0000","last_unstale":"2023-10-09T06:30:43.496624+0000","last_undegraded":"2023-10-09T06:30:43.496624+0000","last_fullsized":"2023-10-09T06:30:43.496624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:20:42.701396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493788+0000","last_change":"2023-10-09T06:30:43.489507+0000","last_active":"2023-10-09T06:30:43.493788+0000","last_peered":"2023-10-09T06:30:43.493788+0000","last_clean":"2023-10-09T06:30:43.493788+0000","last_became_active":"2023-10-09T06:30:43.489279+0000","last_became_peered":"2023-10-09T06:30:43.489279+0000","last_unstale":"2023-10-09T06:30:43.493788+0000","last_undegraded":"2023-10-09T06:30:43.493788+0000","last_fullsized":"2023-10-09T06:30:43.493788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:05:59.526736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496502+0000","last_change":"2023-10-09T06:30:43.491554+0000","last_active":"2023-10-09T06:30:43.496502+0000","last_peered":"2023-10-09T06:30:43.496502+0000","last_clean":"2023-10-09T06:30:43.496502+0000","last_became_active":"2023-10-09T06:30:43.491305+0000","last_became_peered":"2023-10-09T06:30:43.491305+0000","last_unstale":"2023-10-09T06:30:43.496502+0000","last_undegraded":"2023-10-09T06:30:43.496502+0000","last_fullsized":"2023-10-09T06:30:43.496502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:15:54.706732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.483269+0000","last_change":"2023-10-09T06:30:43.478613+0000","last_active":"2023-10-09T06:30:43.483269+0000","last_peered":"2023-10-09T06:30:43.483269+0000","last_clean":"2023-10-09T06:30:43.483269+0000","last_became_active":"2023-10-09T06:30:43.477580+0000","last_became_peered":"2023-10-09T06:30:43.477580+0000","last_unstale":"2023-10-09T06:30:43.483269+0000","last_undegraded":"2023-10-09T06:30:43.483269+0000","last_fullsized":"2023-10-09T06:30:43.483269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:15:34.625915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.491028+0000","last_change":"2023-10-09T06:30:43.485665+0000","last_active":"2023-10-09T06:30:43.491028+0000","last_peered":"2023-10-09T06:30:43.491028+0000","last_clean":"2023-10-09T06:30:43.491028+0000","last_became_active":"2023-10-09T06:30:43.485346+0000","last_became_peered":"2023-10-09T06:30:43.485346+0000","last_unstale":"2023-10-09T06:30:43.491028+0000","last_undegraded":"2023-10-09T06:30:43.491028+0000","last_fullsized":"2023-10-09T06:30:43.491028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:56:01.192585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.487663+0000","last_change":"2023-10-09T06:30:43.483174+0000","last_active":"2023-10-09T06:30:43.487663+0000","last_peered":"2023-10-09T06:30:43.487663+0000","last_clean":"2023-10-09T06:30:43.487663+0000","last_became_active":"2023-10-09T06:30:43.482518+0000","last_became_peered":"2023-10-09T06:30:43.482518+0000","last_unstale":"2023-10-09T06:30:43.487663+0000","last_undegraded":"2023-10-09T06:30:43.487663+0000","last_fullsized":"2023-10-09T06:30:43.487663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:56:48.342402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493620+0000","last_change":"2023-10-09T06:30:43.489829+0000","last_active":"2023-10-09T06:30:43.493620+0000","last_peered":"2023-10-09T06:30:43.493620+0000","last_clean":"2023-10-09T06:30:43.493620+0000","last_became_active":"2023-10-09T06:30:43.489549+0000","last_became_peered":"2023-10-09T06:30:43.489549+0000","last_unstale":"2023-10-09T06:30:43.493620+0000","last_undegraded":"2023-10-09T06:30:43.493620+0000","last_fullsized":"2023-10-09T06:30:43.493620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:56:01.192585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496629+0000","last_change":"2023-10-09T06:30:43.491356+0000","last_active":"2023-10-09T06:30:43.496629+0000","last_peered":"2023-10-09T06:30:43.496629+0000","last_clean":"2023-10-09T06:30:43.496629+0000","last_became_active":"2023-10-09T06:30:43.491058+0000","last_became_peered":"2023-10-09T06:30:43.491058+0000","last_unstale":"2023-10-09T06:30:43.496629+0000","last_undegraded":"2023-10-09T06:30:43.496629+0000","last_fullsized":"2023-10-09T06:30:43.496629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:03:07.803820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.492196+0000","last_change":"2023-10-09T06:30:43.489230+0000","last_active":"2023-10-09T06:30:43.492196+0000","last_peered":"2023-10-09T06:30:43.492196+0000","last_clean":"2023-10-09T06:30:43.492196+0000","last_became_active":"2023-10-09T06:30:43.488094+0000","last_became_peered":"2023-10-09T06:30:43.488094+0000","last_unstale":"2023-10-09T06:30:43.492196+0000","last_undegraded":"2023-10-09T06:30:43.492196+0000","last_fullsized":"2023-10-09T06:30:43.492196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:36:01.020241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.488118+0000","last_change":"2023-10-09T06:30:43.486270+0000","last_active":"2023-10-09T06:30:43.488118+0000","last_peered":"2023-10-09T06:30:43.488118+0000","last_clean":"2023-10-09T06:30:43.488118+0000","last_became_active":"2023-10-09T06:30:43.486003+0000","last_became_peered":"2023-10-09T06:30:43.486003+0000","last_unstale":"2023-10-09T06:30:43.488118+0000","last_undegraded":"2023-10-09T06:30:43.488118+0000","last_fullsized":"2023-10-09T06:30:43.488118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:55:20.971921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.485740+0000","last_change":"2023-10-09T06:30:43.482479+0000","last_active":"2023-10-09T06:30:43.485740+0000","last_peered":"2023-10-09T06:30:43.485740+0000","last_clean":"2023-10-09T06:30:43.485740+0000","last_became_active":"2023-10-09T06:30:43.482203+0000","last_became_peered":"2023-10-09T06:30:43.482203+0000","last_unstale":"2023-10-09T06:30:43.485740+0000","last_undegraded":"2023-10-09T06:30:43.485740+0000","last_fullsized":"2023-10-09T06:30:43.485740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:41:39.976781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.761744+0000","last_change":"2023-10-09T06:30:42.761358+0000","last_active":"2023-10-09T06:30:43.761744+0000","last_peered":"2023-10-09T06:30:43.761744+0000","last_clean":"2023-10-09T06:30:43.761744+0000","last_became_active":"2023-10-09T06:30:42.761139+0000","last_became_peered":"2023-10-09T06:30:42.761139+0000","last_unstale":"2023-10-09T06:30:43.761744+0000","last_undegraded":"2023-10-09T06:30:43.761744+0000","last_fullsized":"2023-10-09T06:30:43.761744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:27:28.658955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.490991+0000","last_change":"2023-10-09T06:30:43.485519+0000","last_active":"2023-10-09T06:30:43.490991+0000","last_peered":"2023-10-09T06:30:43.490991+0000","last_clean":"2023-10-09T06:30:43.490991+0000","last_became_active":"2023-10-09T06:30:43.485289+0000","last_became_peered":"2023-10-09T06:30:43.485289+0000","last_unstale":"2023-10-09T06:30:43.490991+0000","last_undegraded":"2023-10-09T06:30:43.490991+0000","last_fullsized":"2023-10-09T06:30:43.490991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:41:39.976781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.764141+0000","last_change":"2023-10-09T06:30:42.758771+0000","last_active":"2023-10-09T06:30:43.764141+0000","last_peered":"2023-10-09T06:30:43.764141+0000","last_clean":"2023-10-09T06:30:43.764141+0000","last_became_active":"2023-10-09T06:30:42.756764+0000","last_became_peered":"2023-10-09T06:30:42.756764+0000","last_unstale":"2023-10-09T06:30:43.764141+0000","last_undegraded":"2023-10-09T06:30:43.764141+0000","last_fullsized":"2023-10-09T06:30:43.764141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:26:03.347834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493888+0000","last_change":"2023-10-09T06:30:43.491484+0000","last_active":"2023-10-09T06:30:43.493888+0000","last_peered":"2023-10-09T06:30:43.493888+0000","last_clean":"2023-10-09T06:30:43.493888+0000","last_became_active":"2023-10-09T06:30:43.491319+0000","last_became_peered":"2023-10-09T06:30:43.491319+0000","last_unstale":"2023-10-09T06:30:43.493888+0000","last_undegraded":"2023-10-09T06:30:43.493888+0000","last_fullsized":"2023-10-09T06:30:43.493888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:12:09.812942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.761120+0000","last_change":"2023-10-09T06:30:42.763648+0000","last_active":"2023-10-09T06:30:43.761120+0000","last_peered":"2023-10-09T06:30:43.761120+0000","last_clean":"2023-10-09T06:30:43.761120+0000","last_became_active":"2023-10-09T06:30:42.763348+0000","last_became_peered":"2023-10-09T06:30:42.763348+0000","last_unstale":"2023-10-09T06:30:43.761120+0000","last_undegraded":"2023-10-09T06:30:43.761120+0000","last_fullsized":"2023-10-09T06:30:43.761120+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:55:19.967211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496719+0000","last_change":"2023-10-09T06:30:43.491778+0000","last_active":"2023-10-09T06:30:43.496719+0000","last_peered":"2023-10-09T06:30:43.496719+0000","last_clean":"2023-10-09T06:30:43.496719+0000","last_became_active":"2023-10-09T06:30:43.491137+0000","last_became_peered":"2023-10-09T06:30:43.491137+0000","last_unstale":"2023-10-09T06:30:43.496719+0000","last_undegraded":"2023-10-09T06:30:43.496719+0000","last_fullsized":"2023-10-09T06:30:43.496719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:46:57.531379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.761164+0000","last_change":"2023-10-09T06:30:42.761346+0000","last_active":"2023-10-09T06:30:43.761164+0000","last_peered":"2023-10-09T06:30:43.761164+0000","last_clean":"2023-10-09T06:30:43.761164+0000","last_became_active":"2023-10-09T06:30:42.761028+0000","last_became_peered":"2023-10-09T06:30:42.761028+0000","last_unstale":"2023-10-09T06:30:43.761164+0000","last_undegraded":"2023-10-09T06:30:43.761164+0000","last_fullsized":"2023-10-09T06:30:43.761164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:20:58.653921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.492949+0000","last_change":"2023-10-09T06:30:43.489975+0000","last_active":"2023-10-09T06:30:43.492949+0000","last_peered":"2023-10-09T06:30:43.492949+0000","last_clean":"2023-10-09T06:30:43.492949+0000","last_became_active":"2023-10-09T06:30:43.489712+0000","last_became_peered":"2023-10-09T06:30:43.489712+0000","last_unstale":"2023-10-09T06:30:43.492949+0000","last_undegraded":"2023-10-09T06:30:43.492949+0000","last_fullsized":"2023-10-09T06:30:43.492949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:41:39.976781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.762311+0000","last_change":"2023-10-09T06:30:42.757689+0000","last_active":"2023-10-09T06:30:43.762311+0000","last_peered":"2023-10-09T06:30:43.762311+0000","last_clean":"2023-10-09T06:30:43.762311+0000","last_became_active":"2023-10-09T06:30:42.757270+0000","last_became_peered":"2023-10-09T06:30:42.757270+0000","last_unstale":"2023-10-09T06:30:43.762311+0000","last_undegraded":"2023-10-09T06:30:43.762311+0000","last_fullsized":"2023-10-09T06:30:43.762311+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:52:48.682306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.490754+0000","last_change":"2023-10-09T06:30:43.486187+0000","last_active":"2023-10-09T06:30:43.490754+0000","last_peered":"2023-10-09T06:30:43.490754+0000","last_clean":"2023-10-09T06:30:43.490754+0000","last_became_active":"2023-10-09T06:30:43.485917+0000","last_became_peered":"2023-10-09T06:30:43.485917+0000","last_unstale":"2023-10-09T06:30:43.490754+0000","last_undegraded":"2023-10-09T06:30:43.490754+0000","last_fullsized":"2023-10-09T06:30:43.490754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:20:59.658630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.763447+0000","last_change":"2023-10-09T06:30:42.758939+0000","last_active":"2023-10-09T06:30:43.763447+0000","last_peered":"2023-10-09T06:30:43.763447+0000","last_clean":"2023-10-09T06:30:43.763447+0000","last_became_active":"2023-10-09T06:30:42.758667+0000","last_became_peered":"2023-10-09T06:30:42.758667+0000","last_unstale":"2023-10-09T06:30:43.763447+0000","last_undegraded":"2023-10-09T06:30:43.763447+0000","last_fullsized":"2023-10-09T06:30:43.763447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:59:42.424421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.488120+0000","last_change":"2023-10-09T06:30:43.484481+0000","last_active":"2023-10-09T06:30:43.488120+0000","last_peered":"2023-10-09T06:30:43.488120+0000","last_clean":"2023-10-09T06:30:43.488120+0000","last_became_active":"2023-10-09T06:30:43.483640+0000","last_became_peered":"2023-10-09T06:30:43.483640+0000","last_unstale":"2023-10-09T06:30:43.488120+0000","last_undegraded":"2023-10-09T06:30:43.488120+0000","last_fullsized":"2023-10-09T06:30:43.488120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:38:10.342140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.474106+0000","last_change":"2023-10-09T06:30:42.758834+0000","last_active":"2023-10-09T06:30:43.474106+0000","last_peered":"2023-10-09T06:30:43.474106+0000","last_clean":"2023-10-09T06:30:43.474106+0000","last_became_active":"2023-10-09T06:30:42.758561+0000","last_became_peered":"2023-10-09T06:30:42.758561+0000","last_unstale":"2023-10-09T06:30:43.474106+0000","last_undegraded":"2023-10-09T06:30:43.474106+0000","last_fullsized":"2023-10-09T06:30:43.474106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:46:54.709506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.490866+0000","last_change":"2023-10-09T06:30:43.483623+0000","last_active":"2023-10-09T06:30:43.490866+0000","last_peered":"2023-10-09T06:30:43.490866+0000","last_clean":"2023-10-09T06:30:43.490866+0000","last_became_active":"2023-10-09T06:30:43.483435+0000","last_became_peered":"2023-10-09T06:30:43.483435+0000","last_unstale":"2023-10-09T06:30:43.490866+0000","last_undegraded":"2023-10-09T06:30:43.490866+0000","last_fullsized":"2023-10-09T06:30:43.490866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:57:57.207775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.485588+0000","last_change":"2023-10-09T06:30:42.761685+0000","last_active":"2023-10-09T06:30:43.485588+0000","last_peered":"2023-10-09T06:30:43.485588+0000","last_clean":"2023-10-09T06:30:43.485588+0000","last_became_active":"2023-10-09T06:30:42.761432+0000","last_became_peered":"2023-10-09T06:30:42.761432+0000","last_unstale":"2023-10-09T06:30:43.485588+0000","last_undegraded":"2023-10-09T06:30:43.485588+0000","last_fullsized":"2023-10-09T06:30:43.485588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:11:08.967445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.486800+0000","last_change":"2023-10-09T06:30:43.483621+0000","last_active":"2023-10-09T06:30:43.486800+0000","last_peered":"2023-10-09T06:30:43.486800+0000","last_clean":"2023-10-09T06:30:43.486800+0000","last_became_active":"2023-10-09T06:30:43.483365+0000","last_became_peered":"2023-10-09T06:30:43.483365+0000","last_unstale":"2023-10-09T06:30:43.486800+0000","last_undegraded":"2023-10-09T06:30:43.486800+0000","last_fullsized":"2023-10-09T06:30:43.486800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:57:59.094337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.470554+0000","last_change":"2023-10-09T06:30:42.757148+0000","last_active":"2023-10-09T06:30:43.470554+0000","last_peered":"2023-10-09T06:30:43.470554+0000","last_clean":"2023-10-09T06:30:43.470554+0000","last_became_active":"2023-10-09T06:30:42.756630+0000","last_became_peered":"2023-10-09T06:30:42.756630+0000","last_unstale":"2023-10-09T06:30:43.470554+0000","last_undegraded":"2023-10-09T06:30:43.470554+0000","last_fullsized":"2023-10-09T06:30:43.470554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:11:08.967445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496741+0000","last_change":"2023-10-09T06:30:43.492800+0000","last_active":"2023-10-09T06:30:43.496741+0000","last_peered":"2023-10-09T06:30:43.496741+0000","last_clean":"2023-10-09T06:30:43.496741+0000","last_became_active":"2023-10-09T06:30:43.492567+0000","last_became_peered":"2023-10-09T06:30:43.492567+0000","last_unstale":"2023-10-09T06:30:43.496741+0000","last_undegraded":"2023-10-09T06:30:43.496741+0000","last_fullsized":"2023-10-09T06:30:43.496741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:05:59.526736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.482776+0000","last_change":"2023-10-09T06:30:42.759966+0000","last_active":"2023-10-09T06:30:43.482776+0000","last_peered":"2023-10-09T06:30:43.482776+0000","last_clean":"2023-10-09T06:30:43.482776+0000","last_became_active":"2023-10-09T06:30:42.759686+0000","last_became_peered":"2023-10-09T06:30:42.759686+0000","last_unstale":"2023-10-09T06:30:43.482776+0000","last_undegraded":"2023-10-09T06:30:43.482776+0000","last_fullsized":"2023-10-09T06:30:43.482776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:46:54.709506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493906+0000","last_change":"2023-10-09T06:30:43.491580+0000","last_active":"2023-10-09T06:30:43.493906+0000","last_peered":"2023-10-09T06:30:43.493906+0000","last_clean":"2023-10-09T06:30:43.493906+0000","last_became_active":"2023-10-09T06:30:43.491323+0000","last_became_peered":"2023-10-09T06:30:43.491323+0000","last_unstale":"2023-10-09T06:30:43.493906+0000","last_undegraded":"2023-10-09T06:30:43.493906+0000","last_fullsized":"2023-10-09T06:30:43.493906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:02:03.930820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.761837+0000","last_change":"2023-10-09T06:30:42.758290+0000","last_active":"2023-10-09T06:30:43.761837+0000","last_peered":"2023-10-09T06:30:43.761837+0000","last_clean":"2023-10-09T06:30:43.761837+0000","last_became_active":"2023-10-09T06:30:42.758008+0000","last_became_peered":"2023-10-09T06:30:42.758008+0000","last_unstale":"2023-10-09T06:30:43.761837+0000","last_undegraded":"2023-10-09T06:30:43.761837+0000","last_fullsized":"2023-10-09T06:30:43.761837+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:20:58.653921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.483249+0000","last_change":"2023-10-09T06:30:43.478610+0000","last_active":"2023-10-09T06:30:43.483249+0000","last_peered":"2023-10-09T06:30:43.483249+0000","last_clean":"2023-10-09T06:30:43.483249+0000","last_became_active":"2023-10-09T06:30:43.477942+0000","last_became_peered":"2023-10-09T06:30:43.477942+0000","last_unstale":"2023-10-09T06:30:43.483249+0000","last_undegraded":"2023-10-09T06:30:43.483249+0000","last_fullsized":"2023-10-09T06:30:43.483249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:29:12.176820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.766388+0000","last_change":"2023-10-09T06:30:42.759743+0000","last_active":"2023-10-09T06:30:43.766388+0000","last_peered":"2023-10-09T06:30:43.766388+0000","last_clean":"2023-10-09T06:30:43.766388+0000","last_became_active":"2023-10-09T06:30:42.757647+0000","last_became_peered":"2023-10-09T06:30:42.757647+0000","last_unstale":"2023-10-09T06:30:43.766388+0000","last_undegraded":"2023-10-09T06:30:43.766388+0000","last_fullsized":"2023-10-09T06:30:43.766388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:46:54.709506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.485759+0000","last_change":"2023-10-09T06:30:43.484192+0000","last_active":"2023-10-09T06:30:43.485759+0000","last_peered":"2023-10-09T06:30:43.485759+0000","last_clean":"2023-10-09T06:30:43.485759+0000","last_became_active":"2023-10-09T06:30:43.483935+0000","last_became_peered":"2023-10-09T06:30:43.483935+0000","last_unstale":"2023-10-09T06:30:43.485759+0000","last_undegraded":"2023-10-09T06:30:43.485759+0000","last_fullsized":"2023-10-09T06:30:43.485759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:36:01.020241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.763949+0000","last_change":"2023-10-09T06:30:42.760604+0000","last_active":"2023-10-09T06:30:43.763949+0000","last_peered":"2023-10-09T06:30:43.763949+0000","last_clean":"2023-10-09T06:30:43.763949+0000","last_became_active":"2023-10-09T06:30:42.760363+0000","last_became_peered":"2023-10-09T06:30:42.760363+0000","last_unstale":"2023-10-09T06:30:43.763949+0000","last_undegraded":"2023-10-09T06:30:43.763949+0000","last_fullsized":"2023-10-09T06:30:43.763949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:26:03.347834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.484683+0000","last_change":"2023-10-09T06:30:43.479684+0000","last_active":"2023-10-09T06:30:43.484683+0000","last_peered":"2023-10-09T06:30:43.484683+0000","last_clean":"2023-10-09T06:30:43.484683+0000","last_became_active":"2023-10-09T06:30:43.479262+0000","last_became_peered":"2023-10-09T06:30:43.479262+0000","last_unstale":"2023-10-09T06:30:43.484683+0000","last_undegraded":"2023-10-09T06:30:43.484683+0000","last_fullsized":"2023-10-09T06:30:43.484683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:09:10.695002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.482211+0000","last_change":"2023-10-09T06:30:42.762087+0000","last_active":"2023-10-09T06:30:43.482211+0000","last_peered":"2023-10-09T06:30:43.482211+0000","last_clean":"2023-10-09T06:30:43.482211+0000","last_became_active":"2023-10-09T06:30:42.761818+0000","last_became_peered":"2023-10-09T06:30:42.761818+0000","last_unstale":"2023-10-09T06:30:43.482211+0000","last_undegraded":"2023-10-09T06:30:43.482211+0000","last_fullsized":"2023-10-09T06:30:43.482211+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:45:19.887880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.486923+0000","last_change":"2023-10-09T06:30:43.483220+0000","last_active":"2023-10-09T06:30:43.486923+0000","last_peered":"2023-10-09T06:30:43.486923+0000","last_clean":"2023-10-09T06:30:43.486923+0000","last_became_active":"2023-10-09T06:30:43.482733+0000","last_became_peered":"2023-10-09T06:30:43.482733+0000","last_unstale":"2023-10-09T06:30:43.486923+0000","last_undegraded":"2023-10-09T06:30:43.486923+0000","last_fullsized":"2023-10-09T06:30:43.486923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:43:44.457796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.470076+0000","last_change":"2023-10-09T06:30:42.759270+0000","last_active":"2023-10-09T06:30:43.470076+0000","last_peered":"2023-10-09T06:30:43.470076+0000","last_clean":"2023-10-09T06:30:43.470076+0000","last_became_active":"2023-10-09T06:30:42.758836+0000","last_became_peered":"2023-10-09T06:30:42.758836+0000","last_unstale":"2023-10-09T06:30:43.470076+0000","last_undegraded":"2023-10-09T06:30:43.470076+0000","last_fullsized":"2023-10-09T06:30:43.470076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:36:40.691450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.484511+0000","last_change":"2023-10-09T06:30:43.480359+0000","last_active":"2023-10-09T06:30:43.484511+0000","last_peered":"2023-10-09T06:30:43.484511+0000","last_clean":"2023-10-09T06:30:43.484511+0000","last_became_active":"2023-10-09T06:30:43.480040+0000","last_became_peered":"2023-10-09T06:30:43.480040+0000","last_unstale":"2023-10-09T06:30:43.484511+0000","last_undegraded":"2023-10-09T06:30:43.484511+0000","last_fullsized":"2023-10-09T06:30:43.484511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:06:29.986154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.764519+0000","last_change":"2023-10-09T06:30:42.756126+0000","last_active":"2023-10-09T06:30:43.764519+0000","last_peered":"2023-10-09T06:30:43.764519+0000","last_clean":"2023-10-09T06:30:43.764519+0000","last_became_active":"2023-10-09T06:30:42.755132+0000","last_became_peered":"2023-10-09T06:30:42.755132+0000","last_unstale":"2023-10-09T06:30:43.764519+0000","last_undegraded":"2023-10-09T06:30:43.764519+0000","last_fullsized":"2023-10-09T06:30:43.764519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:45:19.887880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.492004+0000","last_change":"2023-10-09T06:30:43.487909+0000","last_active":"2023-10-09T06:30:43.492004+0000","last_peered":"2023-10-09T06:30:43.492004+0000","last_clean":"2023-10-09T06:30:43.492004+0000","last_became_active":"2023-10-09T06:30:43.487634+0000","last_became_peered":"2023-10-09T06:30:43.487634+0000","last_unstale":"2023-10-09T06:30:43.492004+0000","last_undegraded":"2023-10-09T06:30:43.492004+0000","last_fullsized":"2023-10-09T06:30:43.492004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:41:00.854052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.760383+0000","last_change":"2023-10-09T06:30:42.756538+0000","last_active":"2023-10-09T06:30:43.760383+0000","last_peered":"2023-10-09T06:30:43.760383+0000","last_clean":"2023-10-09T06:30:43.760383+0000","last_became_active":"2023-10-09T06:30:42.756109+0000","last_became_peered":"2023-10-09T06:30:42.756109+0000","last_unstale":"2023-10-09T06:30:43.760383+0000","last_undegraded":"2023-10-09T06:30:43.760383+0000","last_fullsized":"2023-10-09T06:30:43.760383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:30:02.609828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.485735+0000","last_change":"2023-10-09T06:30:43.483554+0000","last_active":"2023-10-09T06:30:43.485735+0000","last_peered":"2023-10-09T06:30:43.485735+0000","last_clean":"2023-10-09T06:30:43.485735+0000","last_became_active":"2023-10-09T06:30:43.483342+0000","last_became_peered":"2023-10-09T06:30:43.483342+0000","last_unstale":"2023-10-09T06:30:43.485735+0000","last_undegraded":"2023-10-09T06:30:43.485735+0000","last_fullsized":"2023-10-09T06:30:43.485735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:06:04.546054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.478131+0000","last_change":"2023-10-09T06:30:42.754490+0000","last_active":"2023-10-09T06:30:43.478131+0000","last_peered":"2023-10-09T06:30:43.478131+0000","last_clean":"2023-10-09T06:30:43.478131+0000","last_became_active":"2023-10-09T06:30:42.754238+0000","last_became_peered":"2023-10-09T06:30:42.754238+0000","last_unstale":"2023-10-09T06:30:43.478131+0000","last_undegraded":"2023-10-09T06:30:43.478131+0000","last_fullsized":"2023-10-09T06:30:43.478131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:45:19.887880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496827+0000","last_change":"2023-10-09T06:30:43.493269+0000","last_active":"2023-10-09T06:30:43.496827+0000","last_peered":"2023-10-09T06:30:43.496827+0000","last_clean":"2023-10-09T06:30:43.496827+0000","last_became_active":"2023-10-09T06:30:43.491762+0000","last_became_peered":"2023-10-09T06:30:43.491762+0000","last_unstale":"2023-10-09T06:30:43.496827+0000","last_undegraded":"2023-10-09T06:30:43.496827+0000","last_fullsized":"2023-10-09T06:30:43.496827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:10:42.930778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.482114+0000","last_change":"2023-10-09T06:30:42.763709+0000","last_active":"2023-10-09T06:30:43.482114+0000","last_peered":"2023-10-09T06:30:43.482114+0000","last_clean":"2023-10-09T06:30:43.482114+0000","last_became_active":"2023-10-09T06:30:42.763455+0000","last_became_peered":"2023-10-09T06:30:42.763455+0000","last_unstale":"2023-10-09T06:30:43.482114+0000","last_undegraded":"2023-10-09T06:30:43.482114+0000","last_fullsized":"2023-10-09T06:30:43.482114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:59:42.424421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.490783+0000","last_change":"2023-10-09T06:30:43.486278+0000","last_active":"2023-10-09T06:30:43.490783+0000","last_peered":"2023-10-09T06:30:43.490783+0000","last_clean":"2023-10-09T06:30:43.490783+0000","last_became_active":"2023-10-09T06:30:43.485877+0000","last_became_peered":"2023-10-09T06:30:43.485877+0000","last_unstale":"2023-10-09T06:30:43.490783+0000","last_undegraded":"2023-10-09T06:30:43.490783+0000","last_fullsized":"2023-10-09T06:30:43.490783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:55:20.971921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.486699+0000","last_change":"2023-10-09T06:30:43.482444+0000","last_active":"2023-10-09T06:30:43.486699+0000","last_peered":"2023-10-09T06:30:43.486699+0000","last_clean":"2023-10-09T06:30:43.486699+0000","last_became_active":"2023-10-09T06:30:43.482168+0000","last_became_peered":"2023-10-09T06:30:43.482168+0000","last_unstale":"2023-10-09T06:30:43.486699+0000","last_undegraded":"2023-10-09T06:30:43.486699+0000","last_fullsized":"2023-10-09T06:30:43.486699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:56:48.342402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.476414+0000","last_change":"2023-10-09T06:30:42.755534+0000","last_active":"2023-10-09T06:30:43.476414+0000","last_peered":"2023-10-09T06:30:43.476414+0000","last_clean":"2023-10-09T06:30:43.476414+0000","last_became_active":"2023-10-09T06:30:42.755246+0000","last_became_peered":"2023-10-09T06:30:42.755246+0000","last_unstale":"2023-10-09T06:30:43.476414+0000","last_undegraded":"2023-10-09T06:30:43.476414+0000","last_fullsized":"2023-10-09T06:30:43.476414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:52:48.682306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493672+0000","last_change":"2023-10-09T06:30:43.488993+0000","last_active":"2023-10-09T06:30:43.493672+0000","last_peered":"2023-10-09T06:30:43.493672+0000","last_clean":"2023-10-09T06:30:43.493672+0000","last_became_active":"2023-10-09T06:30:43.488701+0000","last_became_peered":"2023-10-09T06:30:43.488701+0000","last_unstale":"2023-10-09T06:30:43.493672+0000","last_undegraded":"2023-10-09T06:30:43.493672+0000","last_fullsized":"2023-10-09T06:30:43.493672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:05:59.526736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.479498+0000","last_change":"2023-10-09T06:30:42.757897+0000","last_active":"2023-10-09T06:30:43.479498+0000","last_peered":"2023-10-09T06:30:43.479498+0000","last_clean":"2023-10-09T06:30:43.479498+0000","last_became_active":"2023-10-09T06:30:42.757432+0000","last_became_peered":"2023-10-09T06:30:42.757432+0000","last_unstale":"2023-10-09T06:30:43.479498+0000","last_undegraded":"2023-10-09T06:30:43.479498+0000","last_fullsized":"2023-10-09T06:30:43.479498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:36:40.691450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":163,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.474256+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:30:43.474256+0000","last_peered":"2023-10-09T06:30:43.474256+0000","last_clean":"2023-10-09T06:30:43.474256+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:30:43.474256+0000","last_undegraded":"2023-10-09T06:30:43.474256+0000","last_fullsized":"2023-10-09T06:30:43.474256+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.484921+0000","last_change":"2023-10-09T06:30:43.479970+0000","last_active":"2023-10-09T06:30:43.484921+0000","last_peered":"2023-10-09T06:30:43.484921+0000","last_clean":"2023-10-09T06:30:43.484921+0000","last_became_active":"2023-10-09T06:30:43.479731+0000","last_became_peered":"2023-10-09T06:30:43.479731+0000","last_unstale":"2023-10-09T06:30:43.484921+0000","last_undegraded":"2023-10-09T06:30:43.484921+0000","last_fullsized":"2023-10-09T06:30:43.484921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:11:59.504301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.475172+0000","last_change":"2023-10-09T06:30:42.763408+0000","last_active":"2023-10-09T06:30:43.475172+0000","last_peered":"2023-10-09T06:30:43.475172+0000","last_clean":"2023-10-09T06:30:43.475172+0000","last_became_active":"2023-10-09T06:30:42.763129+0000","last_became_peered":"2023-10-09T06:30:42.763129+0000","last_unstale":"2023-10-09T06:30:43.475172+0000","last_undegraded":"2023-10-09T06:30:43.475172+0000","last_fullsized":"2023-10-09T06:30:43.475172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:29:11.172110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.475746+0000","last_change":"2023-10-09T06:30:42.766229+0000","last_active":"2023-10-09T06:30:43.475746+0000","last_peered":"2023-10-09T06:30:43.475746+0000","last_clean":"2023-10-09T06:30:43.475746+0000","last_became_active":"2023-10-09T06:30:42.765968+0000","last_became_peered":"2023-10-09T06:30:42.765968+0000","last_unstale":"2023-10-09T06:30:43.475746+0000","last_undegraded":"2023-10-09T06:30:43.475746+0000","last_fullsized":"2023-10-09T06:30:43.475746+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:59:42.424421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.486657+0000","last_change":"2023-10-09T06:30:43.482450+0000","last_active":"2023-10-09T06:30:43.486657+0000","last_peered":"2023-10-09T06:30:43.486657+0000","last_clean":"2023-10-09T06:30:43.486657+0000","last_became_active":"2023-10-09T06:30:43.482187+0000","last_became_peered":"2023-10-09T06:30:43.482187+0000","last_unstale":"2023-10-09T06:30:43.486657+0000","last_undegraded":"2023-10-09T06:30:43.486657+0000","last_fullsized":"2023-10-09T06:30:43.486657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:09:10.695002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-09T06:30:44.472751+0000","last_change":"2023-10-09T06:30:42.759163+0000","last_active":"2023-10-09T06:30:44.472751+0000","last_peered":"2023-10-09T06:30:44.472751+0000","last_clean":"2023-10-09T06:30:44.472751+0000","last_became_active":"2023-10-09T06:30:42.758583+0000","last_became_peered":"2023-10-09T06:30:42.758583+0000","last_unstale":"2023-10-09T06:30:44.472751+0000","last_undegraded":"2023-10-09T06:30:44.472751+0000","last_fullsized":"2023-10-09T06:30:44.472751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:36:40.691450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.487871+0000","last_change":"2023-10-09T06:30:43.480222+0000","last_active":"2023-10-09T06:30:43.487871+0000","last_peered":"2023-10-09T06:30:43.487871+0000","last_clean":"2023-10-09T06:30:43.487871+0000","last_became_active":"2023-10-09T06:30:43.479958+0000","last_became_peered":"2023-10-09T06:30:43.479958+0000","last_unstale":"2023-10-09T06:30:43.487871+0000","last_undegraded":"2023-10-09T06:30:43.487871+0000","last_fullsized":"2023-10-09T06:30:43.487871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:57:59.094337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.761656+0000","last_change":"2023-10-09T06:30:42.757526+0000","last_active":"2023-10-09T06:30:43.761656+0000","last_peered":"2023-10-09T06:30:43.761656+0000","last_clean":"2023-10-09T06:30:43.761656+0000","last_became_active":"2023-10-09T06:30:42.756934+0000","last_became_peered":"2023-10-09T06:30:42.756934+0000","last_unstale":"2023-10-09T06:30:43.761656+0000","last_undegraded":"2023-10-09T06:30:43.761656+0000","last_fullsized":"2023-10-09T06:30:43.761656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:45:19.887880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.487633+0000","last_change":"2023-10-09T06:30:43.482622+0000","last_active":"2023-10-09T06:30:43.487633+0000","last_peered":"2023-10-09T06:30:43.487633+0000","last_clean":"2023-10-09T06:30:43.487633+0000","last_became_active":"2023-10-09T06:30:43.482362+0000","last_became_peered":"2023-10-09T06:30:43.482362+0000","last_unstale":"2023-10-09T06:30:43.487633+0000","last_undegraded":"2023-10-09T06:30:43.487633+0000","last_fullsized":"2023-10-09T06:30:43.487633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:15:54.706732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.474220+0000","last_change":"2023-10-09T06:30:42.764161+0000","last_active":"2023-10-09T06:30:43.474220+0000","last_peered":"2023-10-09T06:30:43.474220+0000","last_clean":"2023-10-09T06:30:43.474220+0000","last_became_active":"2023-10-09T06:30:42.763872+0000","last_became_peered":"2023-10-09T06:30:42.763872+0000","last_unstale":"2023-10-09T06:30:43.474220+0000","last_undegraded":"2023-10-09T06:30:43.474220+0000","last_fullsized":"2023-10-09T06:30:43.474220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:36:40.691450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.491559+0000","last_change":"2023-10-09T06:30:43.487127+0000","last_active":"2023-10-09T06:30:43.491559+0000","last_peered":"2023-10-09T06:30:43.491559+0000","last_clean":"2023-10-09T06:30:43.491559+0000","last_became_active":"2023-10-09T06:30:43.486428+0000","last_became_peered":"2023-10-09T06:30:43.486428+0000","last_unstale":"2023-10-09T06:30:43.491559+0000","last_undegraded":"2023-10-09T06:30:43.491559+0000","last_fullsized":"2023-10-09T06:30:43.491559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:59:43.429131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.478013+0000","last_change":"2023-10-09T06:30:42.758491+0000","last_active":"2023-10-09T06:30:43.478013+0000","last_peered":"2023-10-09T06:30:43.478013+0000","last_clean":"2023-10-09T06:30:43.478013+0000","last_became_active":"2023-10-09T06:30:42.757552+0000","last_became_peered":"2023-10-09T06:30:42.757552+0000","last_unstale":"2023-10-09T06:30:43.478013+0000","last_undegraded":"2023-10-09T06:30:43.478013+0000","last_fullsized":"2023-10-09T06:30:43.478013+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:11:08.967445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.490847+0000","last_change":"2023-10-09T06:30:43.483055+0000","last_active":"2023-10-09T06:30:43.490847+0000","last_peered":"2023-10-09T06:30:43.490847+0000","last_clean":"2023-10-09T06:30:43.490847+0000","last_became_active":"2023-10-09T06:30:43.482794+0000","last_became_peered":"2023-10-09T06:30:43.482794+0000","last_unstale":"2023-10-09T06:30:43.490847+0000","last_undegraded":"2023-10-09T06:30:43.490847+0000","last_fullsized":"2023-10-09T06:30:43.490847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:12:09.812942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.478093+0000","last_change":"2023-10-09T06:30:42.755096+0000","last_active":"2023-10-09T06:30:43.478093+0000","last_peered":"2023-10-09T06:30:43.478093+0000","last_clean":"2023-10-09T06:30:43.478093+0000","last_became_active":"2023-10-09T06:30:42.754834+0000","last_became_peered":"2023-10-09T06:30:42.754834+0000","last_unstale":"2023-10-09T06:30:43.478093+0000","last_undegraded":"2023-10-09T06:30:43.478093+0000","last_fullsized":"2023-10-09T06:30:43.478093+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:52:48.682306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.491004+0000","last_change":"2023-10-09T06:30:43.485563+0000","last_active":"2023-10-09T06:30:43.491004+0000","last_peered":"2023-10-09T06:30:43.491004+0000","last_clean":"2023-10-09T06:30:43.491004+0000","last_became_active":"2023-10-09T06:30:43.485051+0000","last_became_peered":"2023-10-09T06:30:43.485051+0000","last_unstale":"2023-10-09T06:30:43.491004+0000","last_undegraded":"2023-10-09T06:30:43.491004+0000","last_fullsized":"2023-10-09T06:30:43.491004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:36:01.020241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.763572+0000","last_change":"2023-10-09T06:30:42.762583+0000","last_active":"2023-10-09T06:30:43.763572+0000","last_peered":"2023-10-09T06:30:43.763572+0000","last_clean":"2023-10-09T06:30:43.763572+0000","last_became_active":"2023-10-09T06:30:42.762319+0000","last_became_peered":"2023-10-09T06:30:42.762319+0000","last_unstale":"2023-10-09T06:30:43.763572+0000","last_undegraded":"2023-10-09T06:30:43.763572+0000","last_fullsized":"2023-10-09T06:30:43.763572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:15:33.621205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.484629+0000","last_change":"2023-10-09T06:30:43.481138+0000","last_active":"2023-10-09T06:30:43.484629+0000","last_peered":"2023-10-09T06:30:43.484629+0000","last_clean":"2023-10-09T06:30:43.484629+0000","last_became_active":"2023-10-09T06:30:43.480789+0000","last_became_peered":"2023-10-09T06:30:43.480789+0000","last_unstale":"2023-10-09T06:30:43.484629+0000","last_undegraded":"2023-10-09T06:30:43.484629+0000","last_fullsized":"2023-10-09T06:30:43.484629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:12:09.812942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.475749+0000","last_change":"2023-10-09T06:30:42.754630+0000","last_active":"2023-10-09T06:30:43.475749+0000","last_peered":"2023-10-09T06:30:43.475749+0000","last_clean":"2023-10-09T06:30:43.475749+0000","last_became_active":"2023-10-09T06:30:42.754309+0000","last_became_peered":"2023-10-09T06:30:42.754309+0000","last_unstale":"2023-10-09T06:30:43.475749+0000","last_undegraded":"2023-10-09T06:30:43.475749+0000","last_fullsized":"2023-10-09T06:30:43.475749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:46:54.709506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496592+0000","last_change":"2023-10-09T06:30:43.492044+0000","last_active":"2023-10-09T06:30:43.496592+0000","last_peered":"2023-10-09T06:30:43.496592+0000","last_clean":"2023-10-09T06:30:43.496592+0000","last_became_active":"2023-10-09T06:30:43.491715+0000","last_became_peered":"2023-10-09T06:30:43.491715+0000","last_unstale":"2023-10-09T06:30:43.496592+0000","last_undegraded":"2023-10-09T06:30:43.496592+0000","last_fullsized":"2023-10-09T06:30:43.496592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:10:42.930778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.764217+0000","last_change":"2023-10-09T06:30:42.759144+0000","last_active":"2023-10-09T06:30:43.764217+0000","last_peered":"2023-10-09T06:30:43.764217+0000","last_clean":"2023-10-09T06:30:43.764217+0000","last_became_active":"2023-10-09T06:30:42.758729+0000","last_became_peered":"2023-10-09T06:30:42.758729+0000","last_unstale":"2023-10-09T06:30:43.764217+0000","last_undegraded":"2023-10-09T06:30:43.764217+0000","last_fullsized":"2023-10-09T06:30:43.764217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:46:54.709506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496783+0000","last_change":"2023-10-09T06:30:43.493008+0000","last_active":"2023-10-09T06:30:43.496783+0000","last_peered":"2023-10-09T06:30:43.496783+0000","last_clean":"2023-10-09T06:30:43.496783+0000","last_became_active":"2023-10-09T06:30:43.491584+0000","last_became_peered":"2023-10-09T06:30:43.491584+0000","last_unstale":"2023-10-09T06:30:43.496783+0000","last_undegraded":"2023-10-09T06:30:43.496783+0000","last_fullsized":"2023-10-09T06:30:43.496783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:43:44.457796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.762576+0000","last_change":"2023-10-09T06:30:42.756415+0000","last_active":"2023-10-09T06:30:43.762576+0000","last_peered":"2023-10-09T06:30:43.762576+0000","last_clean":"2023-10-09T06:30:43.762576+0000","last_became_active":"2023-10-09T06:30:42.755726+0000","last_became_peered":"2023-10-09T06:30:42.755726+0000","last_unstale":"2023-10-09T06:30:43.762576+0000","last_undegraded":"2023-10-09T06:30:43.762576+0000","last_fullsized":"2023-10-09T06:30:43.762576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:41.732537+0000","last_clean_scrub_stamp":"2023-10-09T06:30:41.732537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:26:03.347834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.485808+0000","last_change":"2023-10-09T06:30:43.482852+0000","last_active":"2023-10-09T06:30:43.485808+0000","last_peered":"2023-10-09T06:30:43.485808+0000","last_clean":"2023-10-09T06:30:43.485808+0000","last_became_active":"2023-10-09T06:30:43.482581+0000","last_became_peered":"2023-10-09T06:30:43.482581+0000","last_unstale":"2023-10-09T06:30:43.485808+0000","last_undegraded":"2023-10-09T06:30:43.485808+0000","last_fullsized":"2023-10-09T06:30:43.485808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:03:07.803820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.490571+0000","last_change":"2023-10-09T06:30:43.485037+0000","last_active":"2023-10-09T06:30:43.490571+0000","last_peered":"2023-10-09T06:30:43.490571+0000","last_clean":"2023-10-09T06:30:43.490571+0000","last_became_active":"2023-10-09T06:30:43.484747+0000","last_became_peered":"2023-10-09T06:30:43.484747+0000","last_unstale":"2023-10-09T06:30:43.490571+0000","last_undegraded":"2023-10-09T06:30:43.490571+0000","last_fullsized":"2023-10-09T06:30:43.490571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:57:59.094337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.493668+0000","last_change":"2023-10-09T06:30:43.488771+0000","last_active":"2023-10-09T06:30:43.493668+0000","last_peered":"2023-10-09T06:30:43.493668+0000","last_clean":"2023-10-09T06:30:43.493668+0000","last_became_active":"2023-10-09T06:30:43.488505+0000","last_became_peered":"2023-10-09T06:30:43.488505+0000","last_unstale":"2023-10-09T06:30:43.493668+0000","last_undegraded":"2023-10-09T06:30:43.493668+0000","last_fullsized":"2023-10-09T06:30:43.493668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:20:59.658630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496662+0000","last_change":"2023-10-09T06:30:43.492057+0000","last_active":"2023-10-09T06:30:43.496662+0000","last_peered":"2023-10-09T06:30:43.496662+0000","last_clean":"2023-10-09T06:30:43.496662+0000","last_became_active":"2023-10-09T06:30:43.491469+0000","last_became_peered":"2023-10-09T06:30:43.491469+0000","last_unstale":"2023-10-09T06:30:43.496662+0000","last_undegraded":"2023-10-09T06:30:43.496662+0000","last_fullsized":"2023-10-09T06:30:43.496662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:59:57.434765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.488149+0000","last_change":"2023-10-09T06:30:43.484938+0000","last_active":"2023-10-09T06:30:43.488149+0000","last_peered":"2023-10-09T06:30:43.488149+0000","last_clean":"2023-10-09T06:30:43.488149+0000","last_became_active":"2023-10-09T06:30:43.484188+0000","last_became_peered":"2023-10-09T06:30:43.484188+0000","last_unstale":"2023-10-09T06:30:43.488149+0000","last_undegraded":"2023-10-09T06:30:43.488149+0000","last_fullsized":"2023-10-09T06:30:43.488149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:57:59.094337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.488046+0000","last_change":"2023-10-09T06:30:43.481165+0000","last_active":"2023-10-09T06:30:43.488046+0000","last_peered":"2023-10-09T06:30:43.488046+0000","last_clean":"2023-10-09T06:30:43.488046+0000","last_became_active":"2023-10-09T06:30:43.480896+0000","last_became_peered":"2023-10-09T06:30:43.480896+0000","last_unstale":"2023-10-09T06:30:43.488046+0000","last_undegraded":"2023-10-09T06:30:43.488046+0000","last_fullsized":"2023-10-09T06:30:43.488046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:56:48.342402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.486824+0000","last_change":"2023-10-09T06:30:43.483125+0000","last_active":"2023-10-09T06:30:43.486824+0000","last_peered":"2023-10-09T06:30:43.486824+0000","last_clean":"2023-10-09T06:30:43.486824+0000","last_became_active":"2023-10-09T06:30:43.482886+0000","last_became_peered":"2023-10-09T06:30:43.482886+0000","last_unstale":"2023-10-09T06:30:43.486824+0000","last_undegraded":"2023-10-09T06:30:43.486824+0000","last_fullsized":"2023-10-09T06:30:43.486824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:15:54.706732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.484766+0000","last_change":"2023-10-09T06:30:43.479321+0000","last_active":"2023-10-09T06:30:43.484766+0000","last_peered":"2023-10-09T06:30:43.484766+0000","last_clean":"2023-10-09T06:30:43.484766+0000","last_became_active":"2023-10-09T06:30:43.477877+0000","last_became_peered":"2023-10-09T06:30:43.477877+0000","last_unstale":"2023-10-09T06:30:43.484766+0000","last_undegraded":"2023-10-09T06:30:43.484766+0000","last_fullsized":"2023-10-09T06:30:43.484766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:09:10.695002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.487894+0000","last_change":"2023-10-09T06:30:43.480511+0000","last_active":"2023-10-09T06:30:43.487894+0000","last_peered":"2023-10-09T06:30:43.487894+0000","last_clean":"2023-10-09T06:30:43.487894+0000","last_became_active":"2023-10-09T06:30:43.480250+0000","last_became_peered":"2023-10-09T06:30:43.480250+0000","last_unstale":"2023-10-09T06:30:43.487894+0000","last_undegraded":"2023-10-09T06:30:43.487894+0000","last_fullsized":"2023-10-09T06:30:43.487894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:15:54.706732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.491696+0000","last_change":"2023-10-09T06:30:43.488975+0000","last_active":"2023-10-09T06:30:43.491696+0000","last_peered":"2023-10-09T06:30:43.491696+0000","last_clean":"2023-10-09T06:30:43.491696+0000","last_became_active":"2023-10-09T06:30:43.488639+0000","last_became_peered":"2023-10-09T06:30:43.488639+0000","last_unstale":"2023-10-09T06:30:43.491696+0000","last_undegraded":"2023-10-09T06:30:43.491696+0000","last_fullsized":"2023-10-09T06:30:43.491696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:06:04.546054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.487823+0000","last_change":"2023-10-09T06:30:43.479862+0000","last_active":"2023-10-09T06:30:43.487823+0000","last_peered":"2023-10-09T06:30:43.487823+0000","last_clean":"2023-10-09T06:30:43.487823+0000","last_became_active":"2023-10-09T06:30:43.479597+0000","last_became_peered":"2023-10-09T06:30:43.479597+0000","last_unstale":"2023-10-09T06:30:43.487823+0000","last_undegraded":"2023-10-09T06:30:43.487823+0000","last_fullsized":"2023-10-09T06:30:43.487823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:09:10.695002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.496763+0000","last_change":"2023-10-09T06:30:43.492872+0000","last_active":"2023-10-09T06:30:43.496763+0000","last_peered":"2023-10-09T06:30:43.496763+0000","last_clean":"2023-10-09T06:30:43.496763+0000","last_became_active":"2023-10-09T06:30:43.492634+0000","last_became_peered":"2023-10-09T06:30:43.492634+0000","last_unstale":"2023-10-09T06:30:43.496763+0000","last_undegraded":"2023-10-09T06:30:43.496763+0000","last_fullsized":"2023-10-09T06:30:43.496763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:02:03.930820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.488131+0000","last_change":"2023-10-09T06:30:43.484753+0000","last_active":"2023-10-09T06:30:43.488131+0000","last_peered":"2023-10-09T06:30:43.488131+0000","last_clean":"2023-10-09T06:30:43.488131+0000","last_became_active":"2023-10-09T06:30:43.483916+0000","last_became_peered":"2023-10-09T06:30:43.483916+0000","last_unstale":"2023-10-09T06:30:43.488131+0000","last_undegraded":"2023-10-09T06:30:43.488131+0000","last_fullsized":"2023-10-09T06:30:43.488131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:06:29.986154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-09T06:30:43.484622+0000","last_change":"2023-10-09T06:30:43.481252+0000","last_active":"2023-10-09T06:30:43.484622+0000","last_peered":"2023-10-09T06:30:43.484622+0000","last_clean":"2023-10-09T06:30:43.484622+0000","last_became_active":"2023-10-09T06:30:43.481008+0000","last_became_peered":"2023-10-09T06:30:43.481008+0000","last_unstale":"2023-10-09T06:30:43.484622+0000","last_undegraded":"2023-10-09T06:30:43.484622+0000","last_fullsized":"2023-10-09T06:30:43.484622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:42.737247+0000","last_clean_scrub_stamp":"2023-10-09T06:30:42.737247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:02:03.930820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298072,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073768,"statfs":{"total":96636764160,"available":96331538432,"internally_reserved":0,"allocated":802816,"data_stored":661576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":72190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":212992,"data_stored":70814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.9459999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.9160000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298080,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073760,"statfs":{"total":96636764160,"available":96331530240,"internally_reserved":0,"allocated":811008,"data_stored":661632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.554}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.5449999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":70654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.890000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.2999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.1879999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":72190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":70780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":208896,"data_stored":70744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:30:52.682 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-09T06:30:52.683 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-09T06:30:52.683 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-09T06:30:52.683 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-09T06:30:52.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.828+0000 7fc2e9833700 1 Processor -- start 2023-10-09T06:30:52.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.828+0000 7fc2e9833700 1 -- start start 2023-10-09T06:30:52.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.829+0000 7fc2e9833700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc2e40cd910 0x7fc2e40ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:52.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.829+0000 7fc2e9833700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2e40b5b00 con 0x7fc2e4159ec0 2023-10-09T06:30:52.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.829+0000 7fc2e9833700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2e40b5da0 con 0x7fc2e40b55a0 2023-10-09T06:30:52.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.829+0000 7fc2e9833700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e40b6080 con 0x7fc2e40cd910 2023-10-09T06:30:52.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.829+0000 7fc2e2ffd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fc2e4159ec0 0x7fc2e40b4e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60944/0 (socket says 172.21.15.163:60944) 2023-10-09T06:30:52.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.829+0000 7fc2e2ffd700 1 -- 172.21.15.163:0/182014209 learned_addr learned my addr 172.21.15.163:0/182014209 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:52.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.829+0000 7fc2e27fc700 1 --2- 172.21.15.163:0/182014209 >> v2:172.21.15.181:3300/0 conn(0x7fc2e40cd910 0x7fc2e40ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:52.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.829+0000 7fc2e1ffb700 1 -- 172.21.15.163:0/182014209 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2015073281 0 0) 0x7fc2e40b5b00 con 0x7fc2e4159ec0 2023-10-09T06:30:52.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.829+0000 7fc2e1ffb700 1 -- 172.21.15.163:0/182014209 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2d0003570 con 0x7fc2e4159ec0 2023-10-09T06:30:52.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.830+0000 7fc2e1ffb700 1 -- 172.21.15.163:0/182014209 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 240348746 0 0) 0x7fc2d0003570 con 0x7fc2e4159ec0 2023-10-09T06:30:52.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.830+0000 7fc2e1ffb700 1 -- 172.21.15.163:0/182014209 >> v2:172.21.15.181:3300/0 conn(0x7fc2e40cd910 msgr2=0x7fc2e40ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:52.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.830+0000 7fc2e1ffb700 1 --2- 172.21.15.163:0/182014209 >> v2:172.21.15.181:3300/0 conn(0x7fc2e40cd910 0x7fc2e40ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:52.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.830+0000 7fc2e1ffb700 1 -- 172.21.15.163:0/182014209 >> v1:172.21.15.181:6789/0 conn(0x7fc2e40b55a0 legacy=0x7fc2e40ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:52.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.830+0000 7fc2e1ffb700 1 -- 172.21.15.163:0/182014209 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e40cc100 con 0x7fc2e4159ec0 2023-10-09T06:30:52.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.830+0000 7fc2e1ffb700 1 -- 172.21.15.163:0/182014209 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fc2d4001970 con 0x7fc2e4159ec0 2023-10-09T06:30:52.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.830+0000 7fc2e27fc700 1 --2- 172.21.15.163:0/182014209 >> v2:172.21.15.181:3300/0 conn(0x7fc2e40cd910 0x7fc2e40ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:30:52.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.830+0000 7fc2e1ffb700 1 -- 172.21.15.163:0/182014209 >> v1:172.21.15.163:6789/0 conn(0x7fc2e4159ec0 legacy=0x7fc2e40b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:52.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.830+0000 7fc2e1ffb700 1 --2- 172.21.15.163:0/182014209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2d0003a90 0x7fc2d0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:52.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.830+0000 7fc2e1ffb700 1 --2- 172.21.15.163:0/182014209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2d0004810 0x7fc2d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:52.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.830+0000 7fc2e1ffb700 1 --2- 172.21.15.163:0/182014209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2d0005210 0x7fc2d000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:52.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.830+0000 7fc2e1ffb700 1 -- 172.21.15.163:0/182014209 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e40b6080 con 0x7fc2d0004810 2023-10-09T06:30:52.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.830+0000 7fc2e1ffb700 1 -- 172.21.15.163:0/182014209 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d000c130 con 0x7fc2d0003a90 2023-10-09T06:30:52.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.830+0000 7fc2e1ffb700 1 -- 172.21.15.163:0/182014209 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d000c2c0 con 0x7fc2d0005210 2023-10-09T06:30:52.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.830+0000 7fc2e27fc700 1 --2- 172.21.15.163:0/182014209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2d0004810 0x7fc2d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:52.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.830+0000 7fc2e2ffd700 1 --2- 172.21.15.163:0/182014209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2d0003a90 0x7fc2d0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:52.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.830+0000 7fc2e37fe700 1 --2- 172.21.15.163:0/182014209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2d0005210 0x7fc2d000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:52.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.830+0000 7fc2e27fc700 1 -- 172.21.15.163:0/182014209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2d0005210 msgr2=0x7fc2d000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:52.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.830+0000 7fc2e27fc700 1 --2- 172.21.15.163:0/182014209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2d0005210 0x7fc2d000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:52.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.830+0000 7fc2e27fc700 1 -- 172.21.15.163:0/182014209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2d0003a90 msgr2=0x7fc2d0003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:52.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.830+0000 7fc2e27fc700 1 --2- 172.21.15.163:0/182014209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2d0003a90 0x7fc2d0003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:52.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.830+0000 7fc2e37fe700 1 -- 172.21.15.163:0/182014209 reap_dead start 2023-10-09T06:30:52.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.830+0000 7fc2e27fc700 1 -- 172.21.15.163:0/182014209 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d0003570 con 0x7fc2d0004810 2023-10-09T06:30:52.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.830+0000 7fc2e27fc700 1 --2- 172.21.15.163:0/182014209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2d0004810 0x7fc2d0004c80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc009df0 tx=0x7fc2cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c756310abc5502 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:52.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.830+0000 7fc2e1ffb700 1 -- 172.21.15.163:0/182014209 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc009740 con 0x7fc2d0004810 2023-10-09T06:30:52.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.830+0000 7fc2e1ffb700 1 -- 172.21.15.163:0/182014209 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2cc00e8f0 con 0x7fc2d0004810 2023-10-09T06:30:52.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.830+0000 7fc2e1ffb700 1 -- 172.21.15.163:0/182014209 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc00ecb0 con 0x7fc2d0004810 2023-10-09T06:30:52.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.831+0000 7fc2e9833700 1 -- 172.21.15.163:0/182014209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2d0004810 msgr2=0x7fc2d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:52.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.831+0000 7fc2e9833700 1 --2- 172.21.15.163:0/182014209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2d0004810 0x7fc2d0004c80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc009df0 tx=0x7fc2cc0091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:52.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.831+0000 7fc2e9833700 1 -- 172.21.15.163:0/182014209 shutdown_connections 2023-10-09T06:30:52.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.831+0000 7fc2e9833700 1 --2- 172.21.15.163:0/182014209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2d0004810 0x7fc2d0004c80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:52.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.831+0000 7fc2e9833700 1 -- 172.21.15.163:0/182014209 >> 172.21.15.163:0/182014209 conn(0x7fc2e40999f0 msgr2=0x7fc2e40b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:52.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.831+0000 7fc2e9833700 1 -- 172.21.15.163:0/182014209 shutdown_connections 2023-10-09T06:30:52.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.831+0000 7fc2e9833700 1 -- 172.21.15.163:0/182014209 wait complete. 2023-10-09T06:30:52.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.832+0000 7fc2e9833700 1 Processor -- start 2023-10-09T06:30:52.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.832+0000 7fc2e9833700 1 -- start start 2023-10-09T06:30:52.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.833+0000 7fc2e9833700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2d0004810 0x7fc2e40d0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:52.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.833+0000 7fc2e9833700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2e409a4b0 0x7fc2e409a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:52.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.833+0000 7fc2e9833700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2e4069360 0x7fc2e40697e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:52.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.833+0000 7fc2e9833700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e409b200 con 0x7fc2e409a4b0 2023-10-09T06:30:52.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.833+0000 7fc2e9833700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e409b4e0 con 0x7fc2e4069360 2023-10-09T06:30:52.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.833+0000 7fc2e9833700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e409b7c0 con 0x7fc2d0004810 2023-10-09T06:30:52.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.833+0000 7fc2e27fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2e409a4b0 0x7fc2e409a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:52.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.833+0000 7fc2e27fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2e409a4b0 0x7fc2e409a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44108/0 (socket says 172.21.15.163:44108) 2023-10-09T06:30:52.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.833+0000 7fc2e27fc700 1 -- 172.21.15.163:0/981495823 learned_addr learned my addr 172.21.15.163:0/981495823 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:52.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.833+0000 7fc2e37fe700 1 --2- 172.21.15.163:0/981495823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2e4069360 0x7fc2e40697e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:52.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.833+0000 7fc2e27fc700 1 -- 172.21.15.163:0/981495823 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2d0004810 msgr2=0x7fc2e40d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:52.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.833+0000 7fc2e2ffd700 1 --2- 172.21.15.163:0/981495823 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2d0004810 0x7fc2e40d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:52.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.833+0000 7fc2e27fc700 1 --2- 172.21.15.163:0/981495823 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2d0004810 0x7fc2e40d0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:52.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.833+0000 7fc2e27fc700 1 -- 172.21.15.163:0/981495823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2e4069360 msgr2=0x7fc2e40697e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:52.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.833+0000 7fc2e27fc700 1 --2- 172.21.15.163:0/981495823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2e4069360 0x7fc2e40697e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:52.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.833+0000 7fc2e27fc700 1 -- 172.21.15.163:0/981495823 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e409baa0 con 0x7fc2e409a4b0 2023-10-09T06:30:52.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.833+0000 7fc2e27fc700 1 --2- 172.21.15.163:0/981495823 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2e409a4b0 0x7fc2e409a920 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc0091e0 tx=0x7fc2cc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:52.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.834+0000 7fc2daffd700 1 -- 172.21.15.163:0/981495823 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc022070 con 0x7fc2e409a4b0 2023-10-09T06:30:52.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.834+0000 7fc2daffd700 1 -- 172.21.15.163:0/981495823 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2cc009740 con 0x7fc2e409a4b0 2023-10-09T06:30:52.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.834+0000 7fc2e9833700 1 -- 172.21.15.163:0/981495823 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e409bd80 con 0x7fc2e409a4b0 2023-10-09T06:30:52.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.834+0000 7fc2daffd700 1 -- 172.21.15.163:0/981495823 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc01c450 con 0x7fc2e409a4b0 2023-10-09T06:30:52.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.834+0000 7fc2e9833700 1 -- 172.21.15.163:0/981495823 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e40d0e90 con 0x7fc2e409a4b0 2023-10-09T06:30:52.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.835+0000 7fc2e9833700 1 -- 172.21.15.163:0/981495823 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2e40b2a80 con 0x7fc2e409a4b0 2023-10-09T06:30:52.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.835+0000 7fc2daffd700 1 -- 172.21.15.163:0/981495823 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc2cc026070 con 0x7fc2e409a4b0 2023-10-09T06:30:52.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.835+0000 7fc2daffd700 1 --2- 172.21.15.163:0/981495823 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc2c4066f30 0x7fc2c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:52.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.836+0000 7fc2daffd700 1 -- 172.21.15.163:0/981495823 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc2cc085e50 con 0x7fc2e409a4b0 2023-10-09T06:30:52.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.836+0000 7fc2e2ffd700 1 --2- 172.21.15.163:0/981495823 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc2c4066f30 0x7fc2c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:52.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.836+0000 7fc2e2ffd700 1 --2- 172.21.15.163:0/981495823 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc2c4066f30 0x7fc2c40693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4001550 tx=0x7fc2d4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:52.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:52.840+0000 7fc2daffd700 1 -- 172.21.15.163:0/981495823 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc2cc0530c0 con 0x7fc2e409a4b0 2023-10-09T06:30:53.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.060+0000 7fc2e9833700 1 -- 172.21.15.163:0/981495823 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc2e409b200 con 0x7fc2e409a4b0 2023-10-09T06:30:53.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.060+0000 7fc2daffd700 1 -- 172.21.15.163:0/981495823 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fc2cc02d040 con 0x7fc2e409a4b0 2023-10-09T06:30:53.062 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:30:53.062 INFO:teuthology.orchestra.run.smithi163.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-09T06:30:53.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.062+0000 7fc2e9833700 1 -- 172.21.15.163:0/981495823 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc2c4066f30 msgr2=0x7fc2c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:53.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.062+0000 7fc2e9833700 1 --2- 172.21.15.163:0/981495823 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc2c4066f30 0x7fc2c40693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4001550 tx=0x7fc2d4009040 comp rx=0 tx=0).stop 2023-10-09T06:30:53.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.062+0000 7fc2e9833700 1 -- 172.21.15.163:0/981495823 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2e409a4b0 msgr2=0x7fc2e409a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:53.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.062+0000 7fc2e9833700 1 --2- 172.21.15.163:0/981495823 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2e409a4b0 0x7fc2e409a920 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc0091e0 tx=0x7fc2cc01a000 comp rx=0 tx=0).stop 2023-10-09T06:30:53.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.062+0000 7fc2e9833700 1 -- 172.21.15.163:0/981495823 shutdown_connections 2023-10-09T06:30:53.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.062+0000 7fc2e9833700 1 --2- 172.21.15.163:0/981495823 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc2c4066f30 0x7fc2c40693e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:53.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.062+0000 7fc2e9833700 1 --2- 172.21.15.163:0/981495823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2e4069360 0x7fc2e40697e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:53.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.062+0000 7fc2e9833700 1 --2- 172.21.15.163:0/981495823 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2d0004810 0x7fc2e40d0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:53.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.062+0000 7fc2e9833700 1 --2- 172.21.15.163:0/981495823 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2e409a4b0 0x7fc2e409a920 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:53.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.062+0000 7fc2e9833700 1 -- 172.21.15.163:0/981495823 >> 172.21.15.163:0/981495823 conn(0x7fc2e40999f0 msgr2=0x7fc2e41592d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:53.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.062+0000 7fc2e9833700 1 -- 172.21.15.163:0/981495823 shutdown_connections 2023-10-09T06:30:53.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.062+0000 7fc2e9833700 1 -- 172.21.15.163:0/981495823 wait complete. 2023-10-09T06:30:53.075 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-09T06:30:53.075 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:30:53.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.189+0000 7f2d17fb2700 1 Processor -- start 2023-10-09T06:30:53.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.189+0000 7f2d17fb2700 1 -- start start 2023-10-09T06:30:53.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.189+0000 7f2d17fb2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2d10159890 0x7f2d1015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:53.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.190+0000 7f2d17fb2700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d1015a160 con 0x7f2d10159c60 2023-10-09T06:30:53.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.190+0000 7f2d17fb2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d100d13a0 con 0x7f2d10069560 2023-10-09T06:30:53.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.190+0000 7f2d17fb2700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d1015e140 con 0x7f2d10159890 2023-10-09T06:30:53.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.190+0000 7f2d15d4e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2d10159890 0x7f2d1015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:53.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.190+0000 7f2d1554d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f2d10159c60 0x7f2d1015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60946/0 (socket says 172.21.15.163:60946) 2023-10-09T06:30:53.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.190+0000 7f2d1554d700 1 -- 172.21.15.163:0/2376028534 learned_addr learned my addr 172.21.15.163:0/2376028534 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:53.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.190+0000 7f2d15d4e700 1 -- 172.21.15.163:0/2376028534 >> v1:172.21.15.181:6789/0 conn(0x7f2d10069560 legacy=0x7f2d100ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:53.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.190+0000 7f2d15d4e700 1 -- 172.21.15.163:0/2376028534 >> v1:172.21.15.163:6789/0 conn(0x7f2d10159c60 legacy=0x7f2d1015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:53.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.190+0000 7f2d15d4e700 1 -- 172.21.15.163:0/2376028534 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d1015a3d0 con 0x7f2d10159890 2023-10-09T06:30:53.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.190+0000 7f2d14d4c700 1 -- 172.21.15.163:0/2376028534 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4286592511 0 0) 0x7f2d1015a160 con 0x7f2d10159c60 2023-10-09T06:30:53.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.190+0000 7f2d15d4e700 1 --2- 172.21.15.163:0/2376028534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d10159890 0x7f2d1015abd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009e60 tx=0x7f2d00009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea918d944999102 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:53.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.190+0000 7f2d14d4c700 1 -- 172.21.15.163:0/2376028534 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0000e070 con 0x7f2d10159890 2023-10-09T06:30:53.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.190+0000 7f2d14d4c700 1 -- 172.21.15.163:0/2376028534 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0000bc30 con 0x7f2d10159890 2023-10-09T06:30:53.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.190+0000 7f2d14d4c700 1 -- 172.21.15.163:0/2376028534 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00012430 con 0x7f2d10159890 2023-10-09T06:30:53.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.191+0000 7f2d17fb2700 1 -- 172.21.15.163:0/2376028534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d10159890 msgr2=0x7f2d1015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:53.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.191+0000 7f2d17fb2700 1 --2- 172.21.15.163:0/2376028534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d10159890 0x7f2d1015abd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009e60 tx=0x7f2d00009780 comp rx=0 tx=0).stop 2023-10-09T06:30:53.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.191+0000 7f2d17fb2700 1 -- 172.21.15.163:0/2376028534 shutdown_connections 2023-10-09T06:30:53.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.191+0000 7f2d17fb2700 1 --2- 172.21.15.163:0/2376028534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d10159890 0x7f2d1015abd0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:53.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.191+0000 7f2d17fb2700 1 -- 172.21.15.163:0/2376028534 >> 172.21.15.163:0/2376028534 conn(0x7f2d10068e20 msgr2=0x7f2d100d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:53.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.191+0000 7f2d17fb2700 1 -- 172.21.15.163:0/2376028534 shutdown_connections 2023-10-09T06:30:53.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.191+0000 7f2d17fb2700 1 -- 172.21.15.163:0/2376028534 wait complete. 2023-10-09T06:30:53.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.192+0000 7f2d17fb2700 1 Processor -- start 2023-10-09T06:30:53.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.192+0000 7f2d17fb2700 1 -- start start 2023-10-09T06:30:53.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.193+0000 7f2d17fb2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d10069560 0x7f2d1014d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:53.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.193+0000 7f2d17fb2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d10159890 0x7f2d1015d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:53.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.193+0000 7f2d17fb2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d10159c60 0x7f2d1015db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:53.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.193+0000 7f2d17fb2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d100b4200 con 0x7f2d10159c60 2023-10-09T06:30:53.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.193+0000 7f2d17fb2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d100b3630 con 0x7f2d10069560 2023-10-09T06:30:53.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.193+0000 7f2d17fb2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d100b38d0 con 0x7f2d10159890 2023-10-09T06:30:53.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.193+0000 7f2d1654f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d10159c60 0x7f2d1015db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:53.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.193+0000 7f2d1654f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d10159c60 0x7f2d1015db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44118/0 (socket says 172.21.15.163:44118) 2023-10-09T06:30:53.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.193+0000 7f2d1654f700 1 -- 172.21.15.163:0/905398951 learned_addr learned my addr 172.21.15.163:0/905398951 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:53.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.193+0000 7f2d15d4e700 1 --2- 172.21.15.163:0/905398951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d10069560 0x7f2d1014d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:53.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.193+0000 7f2d1554d700 1 --2- 172.21.15.163:0/905398951 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d10159890 0x7f2d1015d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:53.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.193+0000 7f2d1654f700 1 -- 172.21.15.163:0/905398951 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d10159890 msgr2=0x7f2d1015d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:53.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.193+0000 7f2d1654f700 1 --2- 172.21.15.163:0/905398951 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d10159890 0x7f2d1015d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:53.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.193+0000 7f2d1654f700 1 -- 172.21.15.163:0/905398951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d10069560 msgr2=0x7f2d1014d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:53.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.193+0000 7f2d1654f700 1 --2- 172.21.15.163:0/905398951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d10069560 0x7f2d1014d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:53.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.193+0000 7f2d1654f700 1 -- 172.21.15.163:0/905398951 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d00009a20 con 0x7f2d10159c60 2023-10-09T06:30:53.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.193+0000 7f2d1654f700 1 --2- 172.21.15.163:0/905398951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d10159c60 0x7f2d1015db60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2d0800c560 tx=0x7f2d08007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:53.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.193+0000 7f2d0dffb700 1 -- 172.21.15.163:0/905398951 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d08011590 con 0x7f2d10159c60 2023-10-09T06:30:53.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.193+0000 7f2d0dffb700 1 -- 172.21.15.163:0/905398951 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d080116f0 con 0x7f2d10159c60 2023-10-09T06:30:53.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.193+0000 7f2d17fb2700 1 -- 172.21.15.163:0/905398951 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d100b3bb0 con 0x7f2d10159c60 2023-10-09T06:30:53.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.193+0000 7f2d0dffb700 1 -- 172.21.15.163:0/905398951 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d08011ab0 con 0x7f2d10159c60 2023-10-09T06:30:53.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.193+0000 7f2d17fb2700 1 -- 172.21.15.163:0/905398951 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d1016e2a0 con 0x7f2d10159c60 2023-10-09T06:30:53.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.194+0000 7f2d17fb2700 1 -- 172.21.15.163:0/905398951 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d10159120 con 0x7f2d10159c60 2023-10-09T06:30:53.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.195+0000 7f2d0dffb700 1 -- 172.21.15.163:0/905398951 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2d0801b3b0 con 0x7f2d10159c60 2023-10-09T06:30:53.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.195+0000 7f2d0dffb700 1 --2- 172.21.15.163:0/905398951 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2cfc066f60 0x7f2cfc069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:53.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.195+0000 7f2d0dffb700 1 -- 172.21.15.163:0/905398951 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2d080838f0 con 0x7f2d10159c60 2023-10-09T06:30:53.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.195+0000 7f2d15d4e700 1 --2- 172.21.15.163:0/905398951 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2cfc066f60 0x7f2cfc069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:53.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.195+0000 7f2d15d4e700 1 --2- 172.21.15.163:0/905398951 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2cfc066f60 0x7f2cfc069410 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009e60 tx=0x7f2d00019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:53.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.199+0000 7f2d0dffb700 1 -- 172.21.15.163:0/905398951 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d08050b60 con 0x7f2d10159c60 2023-10-09T06:30:53.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.382+0000 7f2d17fb2700 1 -- 172.21.15.163:0/905398951 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d1004c3c0 con 0x7f2d10159c60 2023-10-09T06:30:53.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.383+0000 7f2d0dffb700 1 -- 172.21.15.163:0/905398951 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f2d08017070 con 0x7f2d10159c60 2023-10-09T06:30:53.384 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:30:53.384 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":20,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:30:44.466923+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:30:41.369014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-09T06:30:42.103001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:30:53.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.386+0000 7f2d17fb2700 1 -- 172.21.15.163:0/905398951 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2cfc066f60 msgr2=0x7f2cfc069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:53.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.386+0000 7f2d17fb2700 1 --2- 172.21.15.163:0/905398951 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2cfc066f60 0x7f2cfc069410 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009e60 tx=0x7f2d00019000 comp rx=0 tx=0).stop 2023-10-09T06:30:53.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.386+0000 7f2d17fb2700 1 -- 172.21.15.163:0/905398951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d10159c60 msgr2=0x7f2d1015db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:53.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.386+0000 7f2d17fb2700 1 --2- 172.21.15.163:0/905398951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d10159c60 0x7f2d1015db60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2d0800c560 tx=0x7f2d08007ef0 comp rx=0 tx=0).stop 2023-10-09T06:30:53.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.386+0000 7f2d17fb2700 1 -- 172.21.15.163:0/905398951 shutdown_connections 2023-10-09T06:30:53.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.386+0000 7f2d17fb2700 1 --2- 172.21.15.163:0/905398951 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2cfc066f60 0x7f2cfc069410 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:53.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.386+0000 7f2d17fb2700 1 --2- 172.21.15.163:0/905398951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d10159c60 0x7f2d1015db60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:53.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.386+0000 7f2d17fb2700 1 --2- 172.21.15.163:0/905398951 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d10069560 0x7f2d1014d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:53.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.386+0000 7f2d17fb2700 1 --2- 172.21.15.163:0/905398951 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d10159890 0x7f2d1015d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:53.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.386+0000 7f2d17fb2700 1 -- 172.21.15.163:0/905398951 >> 172.21.15.163:0/905398951 conn(0x7f2d10068e20 msgr2=0x7f2d100b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:53.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.386+0000 7f2d17fb2700 1 -- 172.21.15.163:0/905398951 shutdown_connections 2023-10-09T06:30:53.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.386+0000 7f2d17fb2700 1 -- 172.21.15.163:0/905398951 wait complete. 2023-10-09T06:30:53.398 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-09T06:30:41.369014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-09T06:30:42.103001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-09T06:30:53.399 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:30:53.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.516+0000 7fe63634b700 1 Processor -- start 2023-10-09T06:30:53.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.516+0000 7fe63634b700 1 -- start start 2023-10-09T06:30:53.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.517+0000 7fe63634b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe6300af480 0x7fe6300ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:53.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.517+0000 7fe63634b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe630156a60 0x7fe6300694a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:53.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.517+0000 7fe63634b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe630157e30 con 0x7fe630156690 2023-10-09T06:30:53.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.517+0000 7fe63634b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe63006aa00 con 0x7fe6300af480 2023-10-09T06:30:53.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.517+0000 7fe63634b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6300699e0 con 0x7fe630156a60 2023-10-09T06:30:53.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.517+0000 7fe62f7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe6300af480 0x7fe6300ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:53.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.517+0000 7fe62ffff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fe630156690 0x7fe6300c86c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60952/0 (socket says 172.21.15.163:60952) 2023-10-09T06:30:53.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.517+0000 7fe62ffff700 1 -- 172.21.15.163:0/3926029167 learned_addr learned my addr 172.21.15.163:0/3926029167 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:53.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.518+0000 7fe6348e8700 1 --2- 172.21.15.163:0/3926029167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe630156a60 0x7fe6300694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:53.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.518+0000 7fe62f7fe700 1 -- 172.21.15.163:0/3926029167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe630156a60 msgr2=0x7fe6300694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:53.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.518+0000 7fe62f7fe700 1 --2- 172.21.15.163:0/3926029167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe630156a60 0x7fe6300694a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:53.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.518+0000 7fe62f7fe700 1 -- 172.21.15.163:0/3926029167 >> v1:172.21.15.163:6789/0 conn(0x7fe630156690 legacy=0x7fe6300c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:53.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.518+0000 7fe62f7fe700 1 -- 172.21.15.163:0/3926029167 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6301578b0 con 0x7fe6300af480 2023-10-09T06:30:53.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.518+0000 7fe62effd700 1 -- 172.21.15.163:0/3926029167 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3936230033 0 0) 0x7fe630157e30 con 0x7fe630156690 2023-10-09T06:30:53.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.518+0000 7fe62f7fe700 1 --2- 172.21.15.163:0/3926029167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6300af480 0x7fe6300ca9d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe6240098e0 tx=0x7fe6240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612d2e15b5a5fea3 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:53.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.518+0000 7fe62effd700 1 -- 172.21.15.163:0/3926029167 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe624009e80 con 0x7fe6300af480 2023-10-09T06:30:53.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.518+0000 7fe62effd700 1 -- 172.21.15.163:0/3926029167 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62400b860 con 0x7fe6300af480 2023-10-09T06:30:53.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.518+0000 7fe62effd700 1 -- 172.21.15.163:0/3926029167 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62400bbe0 con 0x7fe6300af480 2023-10-09T06:30:53.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.518+0000 7fe63634b700 1 -- 172.21.15.163:0/3926029167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6300af480 msgr2=0x7fe6300ca9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:53.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.518+0000 7fe63634b700 1 --2- 172.21.15.163:0/3926029167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6300af480 0x7fe6300ca9d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe6240098e0 tx=0x7fe6240091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:53.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.518+0000 7fe63634b700 1 -- 172.21.15.163:0/3926029167 shutdown_connections 2023-10-09T06:30:53.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.518+0000 7fe63634b700 1 --2- 172.21.15.163:0/3926029167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe630156a60 0x7fe6300694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:53.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.518+0000 7fe63634b700 1 --2- 172.21.15.163:0/3926029167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6300af480 0x7fe6300ca9d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:53.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.518+0000 7fe63634b700 1 -- 172.21.15.163:0/3926029167 >> 172.21.15.163:0/3926029167 conn(0x7fe6300c9870 msgr2=0x7fe6300c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:53.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.519+0000 7fe63634b700 1 -- 172.21.15.163:0/3926029167 shutdown_connections 2023-10-09T06:30:53.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.519+0000 7fe63634b700 1 -- 172.21.15.163:0/3926029167 wait complete. 2023-10-09T06:30:53.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.520+0000 7fe63634b700 1 Processor -- start 2023-10-09T06:30:53.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.520+0000 7fe63634b700 1 -- start start 2023-10-09T06:30:53.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.520+0000 7fe63634b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6300af480 0x7fe6301595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:53.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.520+0000 7fe63634b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe630156690 0x7fe63014cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:53.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.520+0000 7fe63634b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe630156a60 0x7fe630068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:53.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.520+0000 7fe63634b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63015b7a0 con 0x7fe630156a60 2023-10-09T06:30:53.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.520+0000 7fe63634b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe630159c30 con 0x7fe630156690 2023-10-09T06:30:53.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.520+0000 7fe63634b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe63015ae70 con 0x7fe6300af480 2023-10-09T06:30:53.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.521+0000 7fe6348e8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe630156a60 0x7fe630068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:53.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.521+0000 7fe6348e8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe630156a60 0x7fe630068390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44136/0 (socket says 172.21.15.163:44136) 2023-10-09T06:30:53.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.521+0000 7fe6348e8700 1 -- 172.21.15.163:0/3555613639 learned_addr learned my addr 172.21.15.163:0/3555613639 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:53.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.521+0000 7fe62ffff700 1 --2- 172.21.15.163:0/3555613639 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6300af480 0x7fe6301595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:53.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.521+0000 7fe62f7fe700 1 --2- 172.21.15.163:0/3555613639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe630156690 0x7fe63014cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:53.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.521+0000 7fe6348e8700 1 -- 172.21.15.163:0/3555613639 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6300af480 msgr2=0x7fe6301595a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:53.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.521+0000 7fe6348e8700 1 --2- 172.21.15.163:0/3555613639 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6300af480 0x7fe6301595a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:53.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.521+0000 7fe6348e8700 1 -- 172.21.15.163:0/3555613639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe630156690 msgr2=0x7fe63014cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:53.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.521+0000 7fe6348e8700 1 --2- 172.21.15.163:0/3555613639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe630156690 0x7fe63014cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:53.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.521+0000 7fe6348e8700 1 -- 172.21.15.163:0/3555613639 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe624009420 con 0x7fe630156a60 2023-10-09T06:30:53.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.521+0000 7fe62f7fe700 1 --2- 172.21.15.163:0/3555613639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe630156690 0x7fe63014cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:30:53.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.521+0000 7fe6348e8700 1 --2- 172.21.15.163:0/3555613639 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe630156a60 0x7fe630068390 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe62800ceb0 tx=0x7fe628007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:53.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.521+0000 7fe617fff700 1 -- 172.21.15.163:0/3555613639 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62800ae40 con 0x7fe630156a60 2023-10-09T06:30:53.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.521+0000 7fe617fff700 1 -- 172.21.15.163:0/3555613639 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe628016430 con 0x7fe630156a60 2023-10-09T06:30:53.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.521+0000 7fe617fff700 1 -- 172.21.15.163:0/3555613639 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6280167d0 con 0x7fe630156a60 2023-10-09T06:30:53.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.521+0000 7fe63634b700 1 -- 172.21.15.163:0/3555613639 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe63015b150 con 0x7fe630156a60 2023-10-09T06:30:53.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.521+0000 7fe63634b700 1 -- 172.21.15.163:0/3555613639 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe63014d090 con 0x7fe630156a60 2023-10-09T06:30:53.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.522+0000 7fe63634b700 1 -- 172.21.15.163:0/3555613639 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe63004c3c0 con 0x7fe630156a60 2023-10-09T06:30:53.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.522+0000 7fe617fff700 1 -- 172.21.15.163:0/3555613639 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe628003020 con 0x7fe630156a60 2023-10-09T06:30:53.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.523+0000 7fe617fff700 1 --2- 172.21.15.163:0/3555613639 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe61c066f30 0x7fe61c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:53.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.523+0000 7fe617fff700 1 -- 172.21.15.163:0/3555613639 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe628012070 con 0x7fe630156a60 2023-10-09T06:30:53.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.523+0000 7fe62ffff700 1 --2- 172.21.15.163:0/3555613639 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe61c066f30 0x7fe61c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:53.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.523+0000 7fe62ffff700 1 --2- 172.21.15.163:0/3555613639 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe61c066f30 0x7fe61c0693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe6200011e0 tx=0x7fe620009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:53.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.527+0000 7fe617fff700 1 -- 172.21.15.163:0/3555613639 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe628052750 con 0x7fe630156a60 2023-10-09T06:30:53.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.708+0000 7fe63634b700 1 -- 172.21.15.163:0/3555613639 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe63006f270 con 0x7fe630156a60 2023-10-09T06:30:53.710 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.709+0000 7fe617fff700 1 -- 172.21.15.163:0/3555613639 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe628055d70 con 0x7fe630156a60 2023-10-09T06:30:53.710 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:30:53.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.711+0000 7fe63634b700 1 -- 172.21.15.163:0/3555613639 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe61c066f30 msgr2=0x7fe61c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:53.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.711+0000 7fe63634b700 1 --2- 172.21.15.163:0/3555613639 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe61c066f30 0x7fe61c0693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe6200011e0 tx=0x7fe620009040 comp rx=0 tx=0).stop 2023-10-09T06:30:53.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.711+0000 7fe63634b700 1 -- 172.21.15.163:0/3555613639 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe630156a60 msgr2=0x7fe630068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:53.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.711+0000 7fe63634b700 1 --2- 172.21.15.163:0/3555613639 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe630156a60 0x7fe630068390 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe62800ceb0 tx=0x7fe628007870 comp rx=0 tx=0).stop 2023-10-09T06:30:53.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.711+0000 7fe63634b700 1 -- 172.21.15.163:0/3555613639 shutdown_connections 2023-10-09T06:30:53.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.711+0000 7fe63634b700 1 --2- 172.21.15.163:0/3555613639 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe61c066f30 0x7fe61c0693e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:53.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.711+0000 7fe63634b700 1 --2- 172.21.15.163:0/3555613639 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe630156a60 0x7fe630068390 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:53.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.711+0000 7fe63634b700 1 --2- 172.21.15.163:0/3555613639 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6300af480 0x7fe6301595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:53.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.711+0000 7fe63634b700 1 --2- 172.21.15.163:0/3555613639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe630156690 0x7fe63014cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:53.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.711+0000 7fe63634b700 1 -- 172.21.15.163:0/3555613639 >> 172.21.15.163:0/3555613639 conn(0x7fe6300c9870 msgr2=0x7fe63006b550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:53.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.711+0000 7fe63634b700 1 -- 172.21.15.163:0/3555613639 shutdown_connections 2023-10-09T06:30:53.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.711+0000 7fe63634b700 1 -- 172.21.15.163:0/3555613639 wait complete. 2023-10-09T06:30:53.724 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-09T06:30:53.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.840+0000 7f73750c8700 1 Processor -- start 2023-10-09T06:30:53.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.840+0000 7f73750c8700 1 -- start start 2023-10-09T06:30:53.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.841+0000 7f73750c8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f73700b4e70 0x7f73700ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:53.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.841+0000 7f73750c8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73700b1b40 0x7f737015d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:53.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.841+0000 7f73750c8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73700cfb00 con 0x7f73700b16d0 2023-10-09T06:30:53.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.841+0000 7f73750c8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73700b5c40 con 0x7f73700b4e70 2023-10-09T06:30:53.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.841+0000 7f73750c8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73700b5f20 con 0x7f73700b1b40 2023-10-09T06:30:53.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.841+0000 7f73748c7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73700b1b40 0x7f737015d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:53.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.841+0000 7f736ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f73700b4e70 0x7f73700ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:53.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.841+0000 7f73748c7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73700b1b40 0x7f737015d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:44992/0 (socket says 172.21.15.163:44992) 2023-10-09T06:30:53.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.841+0000 7f73748c7700 1 -- 172.21.15.163:0/3596714911 learned_addr learned my addr 172.21.15.163:0/3596714911 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:53.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.841+0000 7f736ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f73700b4e70 0x7f73700ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:50702/0 (socket says 172.21.15.163:50702) 2023-10-09T06:30:53.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.841+0000 7f736f7fe700 1 --1- 172.21.15.163:0/3596714911 >> v1:172.21.15.181:6789/0 conn(0x7f73700b16d0 0x7f737015d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:47992/0 (socket says 172.21.15.163:47992) 2023-10-09T06:30:53.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.841+0000 7f73748c7700 1 -- 172.21.15.163:0/3596714911 >> v2:172.21.15.181:3300/0 conn(0x7f73700b4e70 msgr2=0x7f73700ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:53.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.841+0000 7f73748c7700 1 --2- 172.21.15.163:0/3596714911 >> v2:172.21.15.181:3300/0 conn(0x7f73700b4e70 0x7f73700ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:53.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.842+0000 7f73748c7700 1 -- 172.21.15.163:0/3596714911 >> v1:172.21.15.181:6789/0 conn(0x7f73700b16d0 legacy=0x7f737015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:53.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.842+0000 7f73748c7700 1 -- 172.21.15.163:0/3596714911 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73700b6200 con 0x7f73700b1b40 2023-10-09T06:30:53.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.842+0000 7f736effd700 1 -- 172.21.15.163:0/3596714911 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4045080559 0 0) 0x7f73700cfb00 con 0x7f73700b16d0 2023-10-09T06:30:53.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.842+0000 7f736ffff700 1 --2- 172.21.15.163:0/3596714911 >> v2:172.21.15.181:3300/0 conn(0x7f73700b4e70 0x7f73700ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:30:53.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.842+0000 7f73748c7700 1 --2- 172.21.15.163:0/3596714911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73700b1b40 0x7f737015d950 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7368009f20 tx=0x7f7368009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fdab3671f0634bf server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:53.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.842+0000 7f736effd700 1 -- 172.21.15.163:0/3596714911 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736800e070 con 0x7f73700b1b40 2023-10-09T06:30:53.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.842+0000 7f736effd700 1 -- 172.21.15.163:0/3596714911 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736800b9d0 con 0x7f73700b1b40 2023-10-09T06:30:53.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.842+0000 7f736effd700 1 -- 172.21.15.163:0/3596714911 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73680092e0 con 0x7f73700b1b40 2023-10-09T06:30:53.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.842+0000 7f73750c8700 1 -- 172.21.15.163:0/3596714911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73700b1b40 msgr2=0x7f737015d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:53.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.842+0000 7f73750c8700 1 --2- 172.21.15.163:0/3596714911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73700b1b40 0x7f737015d950 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7368009f20 tx=0x7f7368009840 comp rx=0 tx=0).stop 2023-10-09T06:30:53.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.842+0000 7f73750c8700 1 -- 172.21.15.163:0/3596714911 shutdown_connections 2023-10-09T06:30:53.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.842+0000 7f73750c8700 1 --2- 172.21.15.163:0/3596714911 >> v2:172.21.15.181:3300/0 conn(0x7f73700b4e70 0x7f73700ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:53.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.842+0000 7f73750c8700 1 --2- 172.21.15.163:0/3596714911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73700b1b40 0x7f737015d950 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:53.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.842+0000 7f73750c8700 1 -- 172.21.15.163:0/3596714911 >> 172.21.15.163:0/3596714911 conn(0x7f73700d10c0 msgr2=0x7f73700affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:53.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.842+0000 7f73750c8700 1 -- 172.21.15.163:0/3596714911 shutdown_connections 2023-10-09T06:30:53.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.843+0000 7f73750c8700 1 -- 172.21.15.163:0/3596714911 wait complete. 2023-10-09T06:30:53.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.844+0000 7f73750c8700 1 Processor -- start 2023-10-09T06:30:53.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.844+0000 7f73750c8700 1 -- start start 2023-10-09T06:30:53.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.844+0000 7f73750c8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73700b16d0 0x7f7370169850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:53.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.844+0000 7f73750c8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73700b1b40 0x7f737014af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:53.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.844+0000 7f73750c8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73700b4e70 0x7f737015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:53.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.844+0000 7f73750c8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73700cc760 con 0x7f73700b16d0 2023-10-09T06:30:53.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.844+0000 7f73750c8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7370169d90 con 0x7f73700b4e70 2023-10-09T06:30:53.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.844+0000 7f73750c8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73700cc1a0 con 0x7f73700b1b40 2023-10-09T06:30:53.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.845+0000 7f736ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73700b16d0 0x7f7370169850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:53.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.845+0000 7f736ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73700b16d0 0x7f7370169850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44138/0 (socket says 172.21.15.163:44138) 2023-10-09T06:30:53.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.845+0000 7f736ffff700 1 -- 172.21.15.163:0/1939219165 learned_addr learned my addr 172.21.15.163:0/1939219165 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:53.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.845+0000 7f73748c7700 1 --2- 172.21.15.163:0/1939219165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73700b4e70 0x7f737015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:53.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.845+0000 7f736f7fe700 1 --2- 172.21.15.163:0/1939219165 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73700b1b40 0x7f737014af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:53.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.845+0000 7f736ffff700 1 -- 172.21.15.163:0/1939219165 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73700b1b40 msgr2=0x7f737014af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:53.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.845+0000 7f736ffff700 1 --2- 172.21.15.163:0/1939219165 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73700b1b40 0x7f737014af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:53.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.845+0000 7f736ffff700 1 -- 172.21.15.163:0/1939219165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73700b4e70 msgr2=0x7f737015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:53.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.845+0000 7f736ffff700 1 --2- 172.21.15.163:0/1939219165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73700b4e70 0x7f737015e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:53.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.845+0000 7f736ffff700 1 -- 172.21.15.163:0/1939219165 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7360009420 con 0x7f73700b16d0 2023-10-09T06:30:53.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.845+0000 7f736ffff700 1 --2- 172.21.15.163:0/1939219165 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73700b16d0 0x7f7370169850 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f736000ec80 tx=0x7f73600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:53.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.845+0000 7f7357fff700 1 -- 172.21.15.163:0/1939219165 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736000c920 con 0x7f73700b16d0 2023-10-09T06:30:53.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.845+0000 7f7357fff700 1 -- 172.21.15.163:0/1939219165 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736000ca80 con 0x7f73700b16d0 2023-10-09T06:30:53.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.845+0000 7f73750c8700 1 -- 172.21.15.163:0/1939219165 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7368009ae0 con 0x7f73700b16d0 2023-10-09T06:30:53.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.845+0000 7f7357fff700 1 -- 172.21.15.163:0/1939219165 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736000ce00 con 0x7f73700b16d0 2023-10-09T06:30:53.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.845+0000 7f73750c8700 1 -- 172.21.15.163:0/1939219165 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73700c74b0 con 0x7f73700b16d0 2023-10-09T06:30:53.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.846+0000 7f73750c8700 1 -- 172.21.15.163:0/1939219165 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73700c7b00 con 0x7f73700b16d0 2023-10-09T06:30:53.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.846+0000 7f7357fff700 1 -- 172.21.15.163:0/1939219165 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7360013680 con 0x7f73700b16d0 2023-10-09T06:30:53.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.847+0000 7f7357fff700 1 --2- 172.21.15.163:0/1939219165 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f735c066f30 0x7f735c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:53.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.847+0000 7f7357fff700 1 -- 172.21.15.163:0/1939219165 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7360084910 con 0x7f73700b16d0 2023-10-09T06:30:53.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.847+0000 7f736f7fe700 1 --2- 172.21.15.163:0/1939219165 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f735c066f30 0x7f735c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:53.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.847+0000 7f736f7fe700 1 --2- 172.21.15.163:0/1939219165 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f735c066f30 0x7f735c0693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f737015fef0 tx=0x7f7364007ef0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:53.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:53.851+0000 7f7357fff700 1 -- 172.21.15.163:0/1939219165 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7360051b80 con 0x7f73700b16d0 2023-10-09T06:30:54.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.032+0000 7f73750c8700 1 -- 172.21.15.163:0/1939219165 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f737004bb80 con 0x7f73700b16d0 2023-10-09T06:30:54.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.032+0000 7f7357fff700 1 -- 172.21.15.163:0/1939219165 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f73600551a0 con 0x7f73700b16d0 2023-10-09T06:30:54.033 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 32 2023-10-09T06:30:54.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.034+0000 7f73750c8700 1 -- 172.21.15.163:0/1939219165 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f735c066f30 msgr2=0x7f735c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:54.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.034+0000 7f73750c8700 1 --2- 172.21.15.163:0/1939219165 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f735c066f30 0x7f735c0693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f737015fef0 tx=0x7f7364007ef0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.034+0000 7f73750c8700 1 -- 172.21.15.163:0/1939219165 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73700b16d0 msgr2=0x7f7370169850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:54.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.034+0000 7f73750c8700 1 --2- 172.21.15.163:0/1939219165 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73700b16d0 0x7f7370169850 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f736000ec80 tx=0x7f73600091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.034+0000 7f73750c8700 1 -- 172.21.15.163:0/1939219165 shutdown_connections 2023-10-09T06:30:54.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.034+0000 7f73750c8700 1 --2- 172.21.15.163:0/1939219165 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f735c066f30 0x7f735c0693e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.034+0000 7f73750c8700 1 --2- 172.21.15.163:0/1939219165 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73700b16d0 0x7f7370169850 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.034+0000 7f73750c8700 1 --2- 172.21.15.163:0/1939219165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73700b4e70 0x7f737015e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.034+0000 7f73750c8700 1 --2- 172.21.15.163:0/1939219165 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73700b1b40 0x7f737014af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.034+0000 7f73750c8700 1 -- 172.21.15.163:0/1939219165 >> 172.21.15.163:0/1939219165 conn(0x7f73700d10c0 msgr2=0x7f73700ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:54.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.035+0000 7f73750c8700 1 -- 172.21.15.163:0/1939219165 shutdown_connections 2023-10-09T06:30:54.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.035+0000 7f73750c8700 1 -- 172.21.15.163:0/1939219165 wait complete. 2023-10-09T06:30:54.047 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-09T06:30:54.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.164+0000 7f4ed440f700 1 Processor -- start 2023-10-09T06:30:54.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.164+0000 7f4ed440f700 1 -- start start 2023-10-09T06:30:54.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.165+0000 7f4ed440f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4ecc159890 0x7f4ecc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:54.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.165+0000 7f4ed440f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ecc159c60 0x7f4ecc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:54.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.165+0000 7f4ed440f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ecc15a160 con 0x7f4ecc15db10 2023-10-09T06:30:54.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.165+0000 7f4ed440f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ecc0d13a0 con 0x7f4ecc159890 2023-10-09T06:30:54.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.165+0000 7f4ed440f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc15e140 con 0x7f4ecc159c60 2023-10-09T06:30:54.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.165+0000 7f4ed29ac700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f4ecc15db10 0x7f4ecc069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60960/0 (socket says 172.21.15.163:60960) 2023-10-09T06:30:54.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.165+0000 7f4ed29ac700 1 -- 172.21.15.163:0/1327258739 learned_addr learned my addr 172.21.15.163:0/1327258739 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:54.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.165+0000 7f4ed19aa700 1 --2- 172.21.15.163:0/1327258739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ecc159c60 0x7f4ecc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:54.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.165+0000 7f4ed21ab700 1 --2- 172.21.15.163:0/1327258739 >> v2:172.21.15.181:3300/0 conn(0x7f4ecc159890 0x7f4ecc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:54.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.165+0000 7f4ed11a9700 1 -- 172.21.15.163:0/1327258739 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2280788627 0 0) 0x7f4ecc15a160 con 0x7f4ecc15db10 2023-10-09T06:30:54.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.165+0000 7f4ed11a9700 1 -- 172.21.15.163:0/1327258739 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4eb8003570 con 0x7f4ecc15db10 2023-10-09T06:30:54.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.165+0000 7f4ed11a9700 1 -- 172.21.15.163:0/1327258739 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 792141077 0 0) 0x7f4eb8003570 con 0x7f4ecc15db10 2023-10-09T06:30:54.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.165+0000 7f4ed21ab700 1 -- 172.21.15.163:0/1327258739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ecc159c60 msgr2=0x7f4ecc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:54.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.165+0000 7f4ed21ab700 1 --2- 172.21.15.163:0/1327258739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ecc159c60 0x7f4ecc15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.165+0000 7f4ed21ab700 1 -- 172.21.15.163:0/1327258739 >> v1:172.21.15.163:6789/0 conn(0x7f4ecc15db10 legacy=0x7f4ecc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:54.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.165+0000 7f4ed21ab700 1 -- 172.21.15.163:0/1327258739 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ecc15a3d0 con 0x7f4ecc159890 2023-10-09T06:30:54.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.165+0000 7f4ed19aa700 1 --2- 172.21.15.163:0/1327258739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ecc159c60 0x7f4ecc15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:30:54.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.165+0000 7f4ed21ab700 1 --2- 172.21.15.163:0/1327258739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ecc159890 0x7f4ecc15abd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009e60 tx=0x7f4ec8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8914c79742264ecd server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:54.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.166+0000 7f4ed11a9700 1 -- 172.21.15.163:0/1327258739 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec800e070 con 0x7f4ecc159890 2023-10-09T06:30:54.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.166+0000 7f4ed11a9700 1 -- 172.21.15.163:0/1327258739 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec800b7a0 con 0x7f4ecc159890 2023-10-09T06:30:54.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.166+0000 7f4ed11a9700 1 -- 172.21.15.163:0/1327258739 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4001470 con 0x7f4ecc159890 2023-10-09T06:30:54.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.166+0000 7f4ed440f700 1 -- 172.21.15.163:0/1327258739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ecc159890 msgr2=0x7f4ecc15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:54.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.166+0000 7f4ed440f700 1 --2- 172.21.15.163:0/1327258739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ecc159890 0x7f4ecc15abd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009e60 tx=0x7f4ec8009780 comp rx=0 tx=0).stop 2023-10-09T06:30:54.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.166+0000 7f4ed440f700 1 -- 172.21.15.163:0/1327258739 shutdown_connections 2023-10-09T06:30:54.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.166+0000 7f4ed440f700 1 --2- 172.21.15.163:0/1327258739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ecc159890 0x7f4ecc15abd0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.166+0000 7f4ed440f700 1 --2- 172.21.15.163:0/1327258739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ecc159c60 0x7f4ecc15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.166+0000 7f4ed440f700 1 -- 172.21.15.163:0/1327258739 >> 172.21.15.163:0/1327258739 conn(0x7f4ecc068e20 msgr2=0x7f4ecc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:54.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.166+0000 7f4ed440f700 1 -- 172.21.15.163:0/1327258739 shutdown_connections 2023-10-09T06:30:54.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.166+0000 7f4ed440f700 1 -- 172.21.15.163:0/1327258739 wait complete. 2023-10-09T06:30:54.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.167+0000 7f4ed440f700 1 Processor -- start 2023-10-09T06:30:54.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.167+0000 7f4ed440f700 1 -- start start 2023-10-09T06:30:54.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.168+0000 7f4ed440f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ecc159890 0x7f4ecc16a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:54.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.168+0000 7f4ed440f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ecc159c60 0x7f4ecc1574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:54.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.168+0000 7f4ed440f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ecc15db10 0x7f4ecc0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:54.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.168+0000 7f4ed440f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc16ad60 con 0x7f4ecc159c60 2023-10-09T06:30:54.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.168+0000 7f4ed440f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc0b5830 con 0x7f4ecc15db10 2023-10-09T06:30:54.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.168+0000 7f4ed440f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc0b5b10 con 0x7f4ecc159890 2023-10-09T06:30:54.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.168+0000 7f4ed19aa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ecc159c60 0x7f4ecc1574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:54.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.168+0000 7f4ed19aa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ecc159c60 0x7f4ecc1574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44146/0 (socket says 172.21.15.163:44146) 2023-10-09T06:30:54.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.168+0000 7f4ed19aa700 1 -- 172.21.15.163:0/3402981145 learned_addr learned my addr 172.21.15.163:0/3402981145 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:54.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.168+0000 7f4ed29ac700 1 --2- 172.21.15.163:0/3402981145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ecc15db10 0x7f4ecc0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:54.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.168+0000 7f4ed21ab700 1 --2- 172.21.15.163:0/3402981145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ecc159890 0x7f4ecc16a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:54.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.168+0000 7f4ed19aa700 1 -- 172.21.15.163:0/3402981145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ecc159890 msgr2=0x7f4ecc16a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:54.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.168+0000 7f4ed19aa700 1 --2- 172.21.15.163:0/3402981145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ecc159890 0x7f4ecc16a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.168+0000 7f4ed19aa700 1 -- 172.21.15.163:0/3402981145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ecc15db10 msgr2=0x7f4ecc0b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:54.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.168+0000 7f4ed19aa700 1 --2- 172.21.15.163:0/3402981145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ecc15db10 0x7f4ecc0b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.168+0000 7f4ed19aa700 1 -- 172.21.15.163:0/3402981145 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ebc009420 con 0x7f4ecc159c60 2023-10-09T06:30:54.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.168+0000 7f4ed29ac700 1 --2- 172.21.15.163:0/3402981145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ecc15db10 0x7f4ecc0b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:30:54.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.168+0000 7f4ed19aa700 1 --2- 172.21.15.163:0/3402981145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ecc159c60 0x7f4ecc1574c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00ec80 tx=0x7f4ebc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:54.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.169+0000 7f4ec27fc700 1 -- 172.21.15.163:0/3402981145 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc00c9d0 con 0x7f4ecc159c60 2023-10-09T06:30:54.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.169+0000 7f4ec27fc700 1 -- 172.21.15.163:0/3402981145 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ebc00cb30 con 0x7f4ecc159c60 2023-10-09T06:30:54.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.169+0000 7f4ed440f700 1 -- 172.21.15.163:0/3402981145 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec8009a20 con 0x7f4ecc159c60 2023-10-09T06:30:54.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.169+0000 7f4ec27fc700 1 -- 172.21.15.163:0/3402981145 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc013470 con 0x7f4ecc159c60 2023-10-09T06:30:54.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.169+0000 7f4ed440f700 1 -- 172.21.15.163:0/3402981145 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ecc0b63b0 con 0x7f4ecc159c60 2023-10-09T06:30:54.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.169+0000 7f4ed440f700 1 -- 172.21.15.163:0/3402981145 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ecc04c3c0 con 0x7f4ecc159c60 2023-10-09T06:30:54.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.170+0000 7f4ec27fc700 1 -- 172.21.15.163:0/3402981145 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4ebc013820 con 0x7f4ecc159c60 2023-10-09T06:30:54.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.170+0000 7f4ec27fc700 1 --2- 172.21.15.163:0/3402981145 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4eb0066f30 0x7f4eb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:54.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.170+0000 7f4ec27fc700 1 -- 172.21.15.163:0/3402981145 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4ebc0847c0 con 0x7f4ecc159c60 2023-10-09T06:30:54.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.170+0000 7f4ed21ab700 1 --2- 172.21.15.163:0/3402981145 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4eb0066f30 0x7f4eb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:54.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.171+0000 7f4ed21ab700 1 --2- 172.21.15.163:0/3402981145 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4eb0066f30 0x7f4eb00693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80099f0 tx=0x7f4ec8018000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:54.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.174+0000 7f4ec27fc700 1 -- 172.21.15.163:0/3402981145 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ebc051a30 con 0x7f4ecc159c60 2023-10-09T06:30:54.355 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.354+0000 7f4ed440f700 1 -- 172.21.15.163:0/3402981145 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4ecc0d15d0 con 0x7f4ecc159c60 2023-10-09T06:30:54.355 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.354+0000 7f4ec27fc700 1 -- 172.21.15.163:0/3402981145 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4ebc055050 con 0x7f4ecc159c60 2023-10-09T06:30:54.355 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 64 2023-10-09T06:30:54.357 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.356+0000 7f4ed440f700 1 -- 172.21.15.163:0/3402981145 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4eb0066f30 msgr2=0x7f4eb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:54.358 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.356+0000 7f4ed440f700 1 --2- 172.21.15.163:0/3402981145 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4eb0066f30 0x7f4eb00693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80099f0 tx=0x7f4ec8018000 comp rx=0 tx=0).stop 2023-10-09T06:30:54.358 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.356+0000 7f4ed440f700 1 -- 172.21.15.163:0/3402981145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ecc159c60 msgr2=0x7f4ecc1574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:54.358 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.356+0000 7f4ed440f700 1 --2- 172.21.15.163:0/3402981145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ecc159c60 0x7f4ecc1574c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00ec80 tx=0x7f4ebc0091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.358 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.356+0000 7f4ed440f700 1 -- 172.21.15.163:0/3402981145 shutdown_connections 2023-10-09T06:30:54.358 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.356+0000 7f4ed440f700 1 --2- 172.21.15.163:0/3402981145 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4eb0066f30 0x7f4eb00693e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.358 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.356+0000 7f4ed440f700 1 --2- 172.21.15.163:0/3402981145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ecc15db10 0x7f4ecc0b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.358 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.356+0000 7f4ed440f700 1 --2- 172.21.15.163:0/3402981145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ecc159890 0x7f4ecc16a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.358 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.356+0000 7f4ed440f700 1 --2- 172.21.15.163:0/3402981145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ecc159c60 0x7f4ecc1574c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.358 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.356+0000 7f4ed440f700 1 -- 172.21.15.163:0/3402981145 >> 172.21.15.163:0/3402981145 conn(0x7f4ecc068e20 msgr2=0x7f4ecc153060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:54.358 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.356+0000 7f4ed440f700 1 -- 172.21.15.163:0/3402981145 shutdown_connections 2023-10-09T06:30:54.358 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.356+0000 7f4ed440f700 1 -- 172.21.15.163:0/3402981145 wait complete. 2023-10-09T06:30:54.369 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:30:54.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.483+0000 7f724f945700 1 Processor -- start 2023-10-09T06:30:54.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.483+0000 7f724f945700 1 -- start start 2023-10-09T06:30:54.486 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.484+0000 7f724f945700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f724806aa40 0x7f72480cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:54.486 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.484+0000 7f724f945700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f724806bac0 0x7f724806ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:54.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.484+0000 7f724f945700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72480cc210 con 0x7f724806b350 2023-10-09T06:30:54.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.484+0000 7f724f945700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72480cc4f0 con 0x7f724806bac0 2023-10-09T06:30:54.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.484+0000 7f724f945700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72480cc7d0 con 0x7f724806aa40 2023-10-09T06:30:54.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.484+0000 7f724cee0700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f724806bac0 0x7f724806ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:54.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.484+0000 7f724dee2700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f724806b350 0x7f72480c81d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60966/0 (socket says 172.21.15.163:60966) 2023-10-09T06:30:54.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.484+0000 7f724dee2700 1 -- 172.21.15.163:0/938567072 learned_addr learned my addr 172.21.15.163:0/938567072 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:54.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.485+0000 7f724d6e1700 1 --2- 172.21.15.163:0/938567072 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f724806aa40 0x7f72480cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:54.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.485+0000 7f724cee0700 1 -- 172.21.15.163:0/938567072 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f724806aa40 msgr2=0x7f72480cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:54.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.485+0000 7f724cee0700 1 --2- 172.21.15.163:0/938567072 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f724806aa40 0x7f72480cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.485+0000 7f724cee0700 1 -- 172.21.15.163:0/938567072 >> v1:172.21.15.163:6789/0 conn(0x7f724806b350 legacy=0x7f72480c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:54.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.485+0000 7f723ffff700 1 -- 172.21.15.163:0/938567072 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2206496218 0 0) 0x7f72480cc210 con 0x7f724806b350 2023-10-09T06:30:54.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.485+0000 7f724cee0700 1 -- 172.21.15.163:0/938567072 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72480ccab0 con 0x7f724806bac0 2023-10-09T06:30:54.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.485+0000 7f724cee0700 1 --2- 172.21.15.163:0/938567072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f724806bac0 0x7f724806ae10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f72380098e0 tx=0x7f72380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f395230cba0f94c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:54.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.485+0000 7f723ffff700 1 -- 172.21.15.163:0/938567072 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7238009e80 con 0x7f724806bac0 2023-10-09T06:30:54.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.485+0000 7f723ffff700 1 -- 172.21.15.163:0/938567072 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723800b860 con 0x7f724806bac0 2023-10-09T06:30:54.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.485+0000 7f723ffff700 1 -- 172.21.15.163:0/938567072 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723800bbe0 con 0x7f724806bac0 2023-10-09T06:30:54.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.485+0000 7f724f945700 1 -- 172.21.15.163:0/938567072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f724806bac0 msgr2=0x7f724806ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:54.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.485+0000 7f724f945700 1 --2- 172.21.15.163:0/938567072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f724806bac0 0x7f724806ae10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f72380098e0 tx=0x7f72380091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.485+0000 7f724f945700 1 -- 172.21.15.163:0/938567072 shutdown_connections 2023-10-09T06:30:54.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.485+0000 7f724f945700 1 --2- 172.21.15.163:0/938567072 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f724806aa40 0x7f72480cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.485+0000 7f724f945700 1 --2- 172.21.15.163:0/938567072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f724806bac0 0x7f724806ae10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.485+0000 7f724f945700 1 -- 172.21.15.163:0/938567072 >> 172.21.15.163:0/938567072 conn(0x7f72480cd670 msgr2=0x7f72480cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:54.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.485+0000 7f724f945700 1 -- 172.21.15.163:0/938567072 shutdown_connections 2023-10-09T06:30:54.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.486+0000 7f724f945700 1 -- 172.21.15.163:0/938567072 wait complete. 2023-10-09T06:30:54.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.487+0000 7f724f945700 1 Processor -- start 2023-10-09T06:30:54.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.487+0000 7f724f945700 1 -- start start 2023-10-09T06:30:54.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.487+0000 7f724f945700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f724806aa40 0x7f724815aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:54.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.487+0000 7f724f945700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f724806b350 0x7f724814d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:54.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.487+0000 7f724f945700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f724806bac0 0x7f72480b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:54.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.487+0000 7f724f945700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7248154d30 con 0x7f724806bac0 2023-10-09T06:30:54.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.487+0000 7f724f945700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7248154440 con 0x7f724806b350 2023-10-09T06:30:54.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.487+0000 7f724f945700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f724815aff0 con 0x7f724806aa40 2023-10-09T06:30:54.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.487+0000 7f724dee2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f724806bac0 0x7f72480b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:54.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.488+0000 7f724dee2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f724806bac0 0x7f72480b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44164/0 (socket says 172.21.15.163:44164) 2023-10-09T06:30:54.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.488+0000 7f724dee2700 1 -- 172.21.15.163:0/2281019215 learned_addr learned my addr 172.21.15.163:0/2281019215 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:54.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.488+0000 7f724d6e1700 1 --2- 172.21.15.163:0/2281019215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f724806aa40 0x7f724815aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:54.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.488+0000 7f724dee2700 1 -- 172.21.15.163:0/2281019215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f724806aa40 msgr2=0x7f724815aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:54.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.488+0000 7f724dee2700 1 --2- 172.21.15.163:0/2281019215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f724806aa40 0x7f724815aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.488+0000 7f724dee2700 1 -- 172.21.15.163:0/2281019215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f724806b350 msgr2=0x7f724814d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:54.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.488+0000 7f724dee2700 1 --2- 172.21.15.163:0/2281019215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f724806b350 0x7f724814d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.488+0000 7f724dee2700 1 -- 172.21.15.163:0/2281019215 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7238009420 con 0x7f724806bac0 2023-10-09T06:30:54.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.488+0000 7f724dee2700 1 --2- 172.21.15.163:0/2281019215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f724806bac0 0x7f72480b0410 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f724000cc90 tx=0x7f7240001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:54.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.488+0000 7f723d7fa700 1 -- 172.21.15.163:0/2281019215 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724000cea0 con 0x7f724806bac0 2023-10-09T06:30:54.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.488+0000 7f723d7fa700 1 -- 172.21.15.163:0/2281019215 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724000ab70 con 0x7f724806bac0 2023-10-09T06:30:54.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.488+0000 7f724f945700 1 -- 172.21.15.163:0/2281019215 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72481546e0 con 0x7f724806bac0 2023-10-09T06:30:54.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.488+0000 7f723d7fa700 1 -- 172.21.15.163:0/2281019215 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72400070f0 con 0x7f724806bac0 2023-10-09T06:30:54.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.488+0000 7f724f945700 1 -- 172.21.15.163:0/2281019215 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72481551c0 con 0x7f724806bac0 2023-10-09T06:30:54.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.489+0000 7f724f945700 1 -- 172.21.15.163:0/2281019215 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7248154ea0 con 0x7f724806bac0 2023-10-09T06:30:54.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.489+0000 7f723d7fa700 1 -- 172.21.15.163:0/2281019215 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f72400074a0 con 0x7f724806bac0 2023-10-09T06:30:54.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.490+0000 7f723d7fa700 1 --2- 172.21.15.163:0/2281019215 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f722c066f30 0x7f722c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:54.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.490+0000 7f723d7fa700 1 -- 172.21.15.163:0/2281019215 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f72400849b0 con 0x7f724806bac0 2023-10-09T06:30:54.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.490+0000 7f724d6e1700 1 --2- 172.21.15.163:0/2281019215 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f722c066f30 0x7f722c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:54.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.490+0000 7f724d6e1700 1 --2- 172.21.15.163:0/2281019215 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f722c066f30 0x7f722c0693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7244009970 tx=0x7f7244008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:54.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.494+0000 7f723d7fa700 1 -- 172.21.15.163:0/2281019215 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7240051ca0 con 0x7f724806bac0 2023-10-09T06:30:54.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.719+0000 7f724f945700 1 -- 172.21.15.163:0/2281019215 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f724804c3c0 con 0x7f724806bac0 2023-10-09T06:30:54.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.720+0000 7f723d7fa700 1 -- 172.21.15.163:0/2281019215 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4603 (secure 0 0 0) 0x7f72400552c0 con 0x7f724806bac0 2023-10-09T06:30:54.721 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:30:54.721 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3440080308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3440080308},{"type":"v1","addr":"172.21.15.181:6837","nonce":3440080308}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1990962358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1990962358},{"type":"v1","addr":"172.21.15.163:6835","nonce":1990962358}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2270076605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2270076605},{"type":"v1","addr":"172.21.15.163:6834","nonce":2270076605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:30:43.133124+0000","modified":"2023-10-09T06:30:46.760523+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/1149296546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1149296546},{"type":"v1","addr":"172.21.15.181:6836","nonce":1149296546}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-09T06:30:54.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.722+0000 7f724f945700 1 -- 172.21.15.163:0/2281019215 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f722c066f30 msgr2=0x7f722c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:54.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.722+0000 7f724f945700 1 --2- 172.21.15.163:0/2281019215 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f722c066f30 0x7f722c0693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7244009970 tx=0x7f7244008040 comp rx=0 tx=0).stop 2023-10-09T06:30:54.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.722+0000 7f724f945700 1 -- 172.21.15.163:0/2281019215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f724806bac0 msgr2=0x7f72480b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:54.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.722+0000 7f724f945700 1 --2- 172.21.15.163:0/2281019215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f724806bac0 0x7f72480b0410 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f724000cc90 tx=0x7f7240001a80 comp rx=0 tx=0).stop 2023-10-09T06:30:54.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.722+0000 7f724f945700 1 -- 172.21.15.163:0/2281019215 shutdown_connections 2023-10-09T06:30:54.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.722+0000 7f724f945700 1 --2- 172.21.15.163:0/2281019215 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f722c066f30 0x7f722c0693e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.722+0000 7f724f945700 1 --2- 172.21.15.163:0/2281019215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f724806bac0 0x7f72480b0410 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.722+0000 7f724f945700 1 --2- 172.21.15.163:0/2281019215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f724806aa40 0x7f724815aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.722+0000 7f724f945700 1 --2- 172.21.15.163:0/2281019215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f724806b350 0x7f724814d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.722+0000 7f724f945700 1 -- 172.21.15.163:0/2281019215 >> 172.21.15.163:0/2281019215 conn(0x7f72480cd670 msgr2=0x7f72480cb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:54.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.722+0000 7f724f945700 1 -- 172.21.15.163:0/2281019215 shutdown_connections 2023-10-09T06:30:54.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.722+0000 7f724f945700 1 -- 172.21.15.163:0/2281019215 wait complete. 2023-10-09T06:30:54.725 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 8 2023-10-09T06:30:54.735 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:30:54.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.848+0000 7f082887b700 1 Processor -- start 2023-10-09T06:30:54.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.848+0000 7f082887b700 1 -- start start 2023-10-09T06:30:54.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.849+0000 7f082887b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0824069560 0x7f08240699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:54.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.849+0000 7f082887b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0824069ee0 con 0x7f0824159c60 2023-10-09T06:30:54.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.849+0000 7f082887b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f082415a160 con 0x7f0824159890 2023-10-09T06:30:54.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.849+0000 7f082887b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08240d13a0 con 0x7f0824069560 2023-10-09T06:30:54.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.849+0000 7f0822d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0824069560 0x7f08240699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:54.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.849+0000 7f0821d9b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f0824159c60 0x7f082415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60970/0 (socket says 172.21.15.163:60970) 2023-10-09T06:30:54.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.849+0000 7f0821d9b700 1 -- 172.21.15.163:0/100537646 learned_addr learned my addr 172.21.15.163:0/100537646 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:54.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.849+0000 7f0822d9d700 1 -- 172.21.15.163:0/100537646 >> v1:172.21.15.181:6789/0 conn(0x7f0824159890 legacy=0x7f082415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:54.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.850+0000 7f0822d9d700 1 -- 172.21.15.163:0/100537646 >> v1:172.21.15.163:6789/0 conn(0x7f0824159c60 legacy=0x7f082415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:54.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.850+0000 7f0822d9d700 1 -- 172.21.15.163:0/100537646 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082415e140 con 0x7f0824069560 2023-10-09T06:30:54.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.850+0000 7f082159a700 1 -- 172.21.15.163:0/100537646 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2178966141 0 0) 0x7f0824069ee0 con 0x7f0824159c60 2023-10-09T06:30:54.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.850+0000 7f0822d9d700 1 --2- 172.21.15.163:0/100537646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0824069560 0x7f08240699a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f081c0098e0 tx=0x7f081c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842a9779ffd58a61 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:54.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.850+0000 7f082159a700 1 -- 172.21.15.163:0/100537646 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081c009e80 con 0x7f0824069560 2023-10-09T06:30:54.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.850+0000 7f082159a700 1 -- 172.21.15.163:0/100537646 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081c00b860 con 0x7f0824069560 2023-10-09T06:30:54.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.850+0000 7f082159a700 1 -- 172.21.15.163:0/100537646 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081c00bbe0 con 0x7f0824069560 2023-10-09T06:30:54.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.850+0000 7f082887b700 1 -- 172.21.15.163:0/100537646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0824069560 msgr2=0x7f08240699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:54.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.850+0000 7f082887b700 1 --2- 172.21.15.163:0/100537646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0824069560 0x7f08240699a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f081c0098e0 tx=0x7f081c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.850+0000 7f082887b700 1 -- 172.21.15.163:0/100537646 shutdown_connections 2023-10-09T06:30:54.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.850+0000 7f082887b700 1 --2- 172.21.15.163:0/100537646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0824069560 0x7f08240699a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.850+0000 7f082887b700 1 -- 172.21.15.163:0/100537646 >> 172.21.15.163:0/100537646 conn(0x7f0824068e20 msgr2=0x7f08240d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:54.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.850+0000 7f082887b700 1 -- 172.21.15.163:0/100537646 shutdown_connections 2023-10-09T06:30:54.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.850+0000 7f082887b700 1 -- 172.21.15.163:0/100537646 wait complete. 2023-10-09T06:30:54.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.851+0000 7f082887b700 1 Processor -- start 2023-10-09T06:30:54.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.851+0000 7f082887b700 1 -- start start 2023-10-09T06:30:54.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.852+0000 7f082887b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0824069560 0x7f08241543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:54.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.852+0000 7f082887b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0824159890 0x7f082415c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:54.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.852+0000 7f082887b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0824159c60 0x7f082415d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:54.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.852+0000 7f082887b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08240ca2f0 con 0x7f0824069560 2023-10-09T06:30:54.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.852+0000 7f082887b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082415c240 con 0x7f0824159c60 2023-10-09T06:30:54.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.852+0000 7f082887b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0824154920 con 0x7f0824159890 2023-10-09T06:30:54.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.852+0000 7f082259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0824069560 0x7f08241543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:54.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.852+0000 7f082259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0824069560 0x7f08241543e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44176/0 (socket says 172.21.15.163:44176) 2023-10-09T06:30:54.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.852+0000 7f082259c700 1 -- 172.21.15.163:0/4004902945 learned_addr learned my addr 172.21.15.163:0/4004902945 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:54.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.852+0000 7f0821d9b700 1 --2- 172.21.15.163:0/4004902945 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0824159890 0x7f082415c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:54.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.852+0000 7f0822d9d700 1 --2- 172.21.15.163:0/4004902945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0824159c60 0x7f082415d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:54.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.852+0000 7f082259c700 1 -- 172.21.15.163:0/4004902945 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0824159890 msgr2=0x7f082415c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:54.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.852+0000 7f082259c700 1 --2- 172.21.15.163:0/4004902945 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0824159890 0x7f082415c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.852+0000 7f082259c700 1 -- 172.21.15.163:0/4004902945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0824159c60 msgr2=0x7f082415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:54.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.852+0000 7f082259c700 1 --2- 172.21.15.163:0/4004902945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0824159c60 0x7f082415d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:54.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.852+0000 7f082259c700 1 -- 172.21.15.163:0/4004902945 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081c009420 con 0x7f0824069560 2023-10-09T06:30:54.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.853+0000 7f082259c700 1 --2- 172.21.15.163:0/4004902945 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0824069560 0x7f08241543e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f081400cb40 tx=0x7f08140017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:54.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.853+0000 7f08127fc700 1 -- 172.21.15.163:0/4004902945 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081400a7a0 con 0x7f0824069560 2023-10-09T06:30:54.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.853+0000 7f08127fc700 1 -- 172.21.15.163:0/4004902945 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081400a920 con 0x7f0824069560 2023-10-09T06:30:54.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.853+0000 7f08127fc700 1 -- 172.21.15.163:0/4004902945 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081400ace0 con 0x7f0824069560 2023-10-09T06:30:54.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.853+0000 7f082887b700 1 -- 172.21.15.163:0/4004902945 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f082415b670 con 0x7f0824069560 2023-10-09T06:30:54.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.853+0000 7f082887b700 1 -- 172.21.15.163:0/4004902945 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f082415bbf0 con 0x7f0824069560 2023-10-09T06:30:54.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.854+0000 7f082887b700 1 -- 172.21.15.163:0/4004902945 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f082404c3c0 con 0x7f0824069560 2023-10-09T06:30:54.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.854+0000 7f08127fc700 1 -- 172.21.15.163:0/4004902945 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0814003030 con 0x7f0824069560 2023-10-09T06:30:54.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.854+0000 7f08127fc700 1 --2- 172.21.15.163:0/4004902945 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0804066f30 0x7f08040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:54.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.854+0000 7f08127fc700 1 -- 172.21.15.163:0/4004902945 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0814084650 con 0x7f0824069560 2023-10-09T06:30:54.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.855+0000 7f0821d9b700 1 --2- 172.21.15.163:0/4004902945 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0804066f30 0x7f08040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:54.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.855+0000 7f0821d9b700 1 --2- 172.21.15.163:0/4004902945 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0804066f30 0x7f08040693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0818000c00 tx=0x7f0818009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:54.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:54.858+0000 7f08127fc700 1 -- 172.21.15.163:0/4004902945 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0814051970 con 0x7f0824069560 2023-10-09T06:30:55.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.042+0000 7f082887b700 1 -- 172.21.15.163:0/4004902945 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f082415bf90 con 0x7f0824069560 2023-10-09T06:30:55.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.043+0000 7f08127fc700 1 -- 172.21.15.163:0/4004902945 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f0814054f90 con 0x7f0824069560 2023-10-09T06:30:55.044 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:30:55.044 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":20,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:30:44.466923+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:30:41.369014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-09T06:30:42.103001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:30:55.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.045+0000 7f082887b700 1 -- 172.21.15.163:0/4004902945 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0804066f30 msgr2=0x7f08040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:55.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.045+0000 7f082887b700 1 --2- 172.21.15.163:0/4004902945 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0804066f30 0x7f08040693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0818000c00 tx=0x7f0818009040 comp rx=0 tx=0).stop 2023-10-09T06:30:55.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.045+0000 7f082887b700 1 -- 172.21.15.163:0/4004902945 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0824069560 msgr2=0x7f08241543e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:55.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.045+0000 7f082887b700 1 --2- 172.21.15.163:0/4004902945 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0824069560 0x7f08241543e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f081400cb40 tx=0x7f08140017b0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.045+0000 7f082887b700 1 -- 172.21.15.163:0/4004902945 shutdown_connections 2023-10-09T06:30:55.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.045+0000 7f082887b700 1 --2- 172.21.15.163:0/4004902945 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0804066f30 0x7f08040693e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.045+0000 7f082887b700 1 --2- 172.21.15.163:0/4004902945 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0824069560 0x7f08241543e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.045+0000 7f082887b700 1 --2- 172.21.15.163:0/4004902945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0824159c60 0x7f082415d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.045+0000 7f082887b700 1 --2- 172.21.15.163:0/4004902945 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0824159890 0x7f082415c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.045+0000 7f082887b700 1 -- 172.21.15.163:0/4004902945 >> 172.21.15.163:0/4004902945 conn(0x7f0824068e20 msgr2=0x7f08240cadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:55.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.046+0000 7f082887b700 1 -- 172.21.15.163:0/4004902945 shutdown_connections 2023-10-09T06:30:55.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.046+0000 7f082887b700 1 -- 172.21.15.163:0/4004902945 wait complete. 2023-10-09T06:30:55.058 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-09T06:30:41.369014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-09T06:30:42.103001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-09T06:30:55.059 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:30:55.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.175+0000 7f9a57b85700 1 Processor -- start 2023-10-09T06:30:55.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.175+0000 7f9a57b85700 1 -- start start 2023-10-09T06:30:55.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.176+0000 7f9a57b85700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9a50159890 0x7f9a5015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:55.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.176+0000 7f9a57b85700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9a50069560 0x7f9a500699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:55.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.176+0000 7f9a57b85700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a50069ee0 con 0x7f9a50159c60 2023-10-09T06:30:55.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.176+0000 7f9a57b85700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a5015a160 con 0x7f9a50069560 2023-10-09T06:30:55.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.176+0000 7f9a57b85700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a500d13a0 con 0x7f9a50159890 2023-10-09T06:30:55.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.176+0000 7f9a56122700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9a50069560 0x7f9a500699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:55.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.176+0000 7f9a56122700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9a50069560 0x7f9a500699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44180/0 (socket says 172.21.15.163:44180) 2023-10-09T06:30:55.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.176+0000 7f9a56122700 1 -- 172.21.15.163:0/1337655867 learned_addr learned my addr 172.21.15.163:0/1337655867 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:55.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.176+0000 7f9a55921700 1 --2- 172.21.15.163:0/1337655867 >> v2:172.21.15.181:3300/0 conn(0x7f9a50159890 0x7f9a5015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:55.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.176+0000 7f9a56122700 1 -- 172.21.15.163:0/1337655867 >> v2:172.21.15.181:3300/0 conn(0x7f9a50159890 msgr2=0x7f9a5015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:55.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.176+0000 7f9a56122700 1 --2- 172.21.15.163:0/1337655867 >> v2:172.21.15.181:3300/0 conn(0x7f9a50159890 0x7f9a5015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.176+0000 7f9a56122700 1 -- 172.21.15.163:0/1337655867 >> v1:172.21.15.181:6789/0 conn(0x7f9a50159c60 legacy=0x7f9a5015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:55.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.176+0000 7f9a56122700 1 -- 172.21.15.163:0/1337655867 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5015e140 con 0x7f9a50069560 2023-10-09T06:30:55.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.176+0000 7f9a55921700 1 --2- 172.21.15.163:0/1337655867 >> v2:172.21.15.181:3300/0 conn(0x7f9a50159890 0x7f9a5015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:30:55.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.176+0000 7f9a56122700 1 --2- 172.21.15.163:0/1337655867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a50069560 0x7f9a500699a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9a500d1090 tx=0x7f9a48009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d70c35996b3fdf server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:55.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.177+0000 7f9a5491f700 1 -- 172.21.15.163:0/1337655867 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4800e070 con 0x7f9a50069560 2023-10-09T06:30:55.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.177+0000 7f9a5491f700 1 -- 172.21.15.163:0/1337655867 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4800ba10 con 0x7f9a50069560 2023-10-09T06:30:55.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.177+0000 7f9a5491f700 1 -- 172.21.15.163:0/1337655867 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a480092e0 con 0x7f9a50069560 2023-10-09T06:30:55.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.177+0000 7f9a57b85700 1 -- 172.21.15.163:0/1337655867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a50069560 msgr2=0x7f9a500699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:55.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.177+0000 7f9a57b85700 1 --2- 172.21.15.163:0/1337655867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a50069560 0x7f9a500699a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9a500d1090 tx=0x7f9a48009840 comp rx=0 tx=0).stop 2023-10-09T06:30:55.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.177+0000 7f9a57b85700 1 -- 172.21.15.163:0/1337655867 shutdown_connections 2023-10-09T06:30:55.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.177+0000 7f9a57b85700 1 --2- 172.21.15.163:0/1337655867 >> v2:172.21.15.181:3300/0 conn(0x7f9a50159890 0x7f9a5015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.177+0000 7f9a57b85700 1 --2- 172.21.15.163:0/1337655867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a50069560 0x7f9a500699a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.177+0000 7f9a57b85700 1 -- 172.21.15.163:0/1337655867 >> 172.21.15.163:0/1337655867 conn(0x7f9a50068e20 msgr2=0x7f9a500d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:55.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.177+0000 7f9a57b85700 1 -- 172.21.15.163:0/1337655867 shutdown_connections 2023-10-09T06:30:55.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.177+0000 7f9a57b85700 1 -- 172.21.15.163:0/1337655867 wait complete. 2023-10-09T06:30:55.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.178+0000 7f9a57b85700 1 Processor -- start 2023-10-09T06:30:55.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.178+0000 7f9a57b85700 1 -- start start 2023-10-09T06:30:55.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.179+0000 7f9a57b85700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a50069560 0x7f9a50154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:55.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.179+0000 7f9a57b85700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a50159890 0x7f9a500c9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:55.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.179+0000 7f9a57b85700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a50159c60 0x7f9a500c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:55.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.179+0000 7f9a57b85700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a500b1b20 con 0x7f9a50159c60 2023-10-09T06:30:55.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.179+0000 7f9a57b85700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a500ca700 con 0x7f9a50159890 2023-10-09T06:30:55.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.179+0000 7f9a57b85700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a500c9e60 con 0x7f9a50069560 2023-10-09T06:30:55.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.179+0000 7f9a56122700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a50159c60 0x7f9a500c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:55.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.179+0000 7f9a56122700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a50159c60 0x7f9a500c97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44192/0 (socket says 172.21.15.163:44192) 2023-10-09T06:30:55.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.179+0000 7f9a56122700 1 -- 172.21.15.163:0/3896734272 learned_addr learned my addr 172.21.15.163:0/3896734272 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:55.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.179+0000 7f9a55120700 1 --2- 172.21.15.163:0/3896734272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a50159890 0x7f9a500c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:55.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.179+0000 7f9a55921700 1 --2- 172.21.15.163:0/3896734272 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a50069560 0x7f9a50154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:55.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.179+0000 7f9a56122700 1 -- 172.21.15.163:0/3896734272 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a50069560 msgr2=0x7f9a50154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:55.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.179+0000 7f9a56122700 1 --2- 172.21.15.163:0/3896734272 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a50069560 0x7f9a50154450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.179+0000 7f9a56122700 1 -- 172.21.15.163:0/3896734272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a50159890 msgr2=0x7f9a500c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:55.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.179+0000 7f9a56122700 1 --2- 172.21.15.163:0/3896734272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a50159890 0x7f9a500c9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.179+0000 7f9a56122700 1 -- 172.21.15.163:0/3896734272 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a48009ae0 con 0x7f9a50159c60 2023-10-09T06:30:55.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.180+0000 7f9a56122700 1 --2- 172.21.15.163:0/3896734272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a50159c60 0x7f9a500c97c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9a48009fd0 tx=0x7f9a48017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:55.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.180+0000 7f9a45ffb700 1 -- 172.21.15.163:0/3896734272 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a480032f0 con 0x7f9a50159c60 2023-10-09T06:30:55.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.180+0000 7f9a45ffb700 1 -- 172.21.15.163:0/3896734272 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a48012d30 con 0x7f9a50159c60 2023-10-09T06:30:55.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.180+0000 7f9a57b85700 1 -- 172.21.15.163:0/3896734272 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a500ca140 con 0x7f9a50159c60 2023-10-09T06:30:55.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.180+0000 7f9a45ffb700 1 -- 172.21.15.163:0/3896734272 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a48012ea0 con 0x7f9a50159c60 2023-10-09T06:30:55.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.180+0000 7f9a57b85700 1 -- 172.21.15.163:0/3896734272 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a500b36a0 con 0x7f9a50159c60 2023-10-09T06:30:55.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.181+0000 7f9a57b85700 1 -- 172.21.15.163:0/3896734272 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a5004c3c0 con 0x7f9a50159c60 2023-10-09T06:30:55.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.181+0000 7f9a45ffb700 1 -- 172.21.15.163:0/3896734272 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9a4801d940 con 0x7f9a50159c60 2023-10-09T06:30:55.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.182+0000 7f9a45ffb700 1 --2- 172.21.15.163:0/3896734272 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9a34066f30 0x7f9a340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:55.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.182+0000 7f9a45ffb700 1 -- 172.21.15.163:0/3896734272 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9a48085e00 con 0x7f9a50159c60 2023-10-09T06:30:55.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.182+0000 7f9a55921700 1 --2- 172.21.15.163:0/3896734272 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9a34066f30 0x7f9a340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:55.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.182+0000 7f9a55921700 1 --2- 172.21.15.163:0/3896734272 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9a34066f30 0x7f9a340693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00adc0 tx=0x7f9a4c006f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:55.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.186+0000 7f9a45ffb700 1 -- 172.21.15.163:0/3896734272 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a48052fd0 con 0x7f9a50159c60 2023-10-09T06:30:55.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.367+0000 7f9a57b85700 1 -- 172.21.15.163:0/3896734272 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9a5006a5b0 con 0x7f9a50159c60 2023-10-09T06:30:55.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.367+0000 7f9a45ffb700 1 -- 172.21.15.163:0/3896734272 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9a480565f0 con 0x7f9a50159c60 2023-10-09T06:30:55.368 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:30:55.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.369+0000 7f9a57b85700 1 -- 172.21.15.163:0/3896734272 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9a34066f30 msgr2=0x7f9a340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:55.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.369+0000 7f9a57b85700 1 --2- 172.21.15.163:0/3896734272 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9a34066f30 0x7f9a340693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00adc0 tx=0x7f9a4c006f90 comp rx=0 tx=0).stop 2023-10-09T06:30:55.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.369+0000 7f9a57b85700 1 -- 172.21.15.163:0/3896734272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a50159c60 msgr2=0x7f9a500c97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:55.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.369+0000 7f9a57b85700 1 --2- 172.21.15.163:0/3896734272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a50159c60 0x7f9a500c97c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9a48009fd0 tx=0x7f9a48017000 comp rx=0 tx=0).stop 2023-10-09T06:30:55.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.369+0000 7f9a57b85700 1 -- 172.21.15.163:0/3896734272 shutdown_connections 2023-10-09T06:30:55.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.369+0000 7f9a57b85700 1 --2- 172.21.15.163:0/3896734272 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9a34066f30 0x7f9a340693e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.369+0000 7f9a57b85700 1 --2- 172.21.15.163:0/3896734272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a50159c60 0x7f9a500c97c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.369+0000 7f9a57b85700 1 --2- 172.21.15.163:0/3896734272 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a50069560 0x7f9a50154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.369+0000 7f9a57b85700 1 --2- 172.21.15.163:0/3896734272 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a50159890 0x7f9a500c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.369+0000 7f9a57b85700 1 -- 172.21.15.163:0/3896734272 >> 172.21.15.163:0/3896734272 conn(0x7f9a50068e20 msgr2=0x7f9a50157250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:55.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.369+0000 7f9a57b85700 1 -- 172.21.15.163:0/3896734272 shutdown_connections 2023-10-09T06:30:55.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.369+0000 7f9a57b85700 1 -- 172.21.15.163:0/3896734272 wait complete. 2023-10-09T06:30:55.382 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-09T06:30:55.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.499+0000 7f9b64f65700 1 Processor -- start 2023-10-09T06:30:55.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.499+0000 7f9b64f65700 1 -- start start 2023-10-09T06:30:55.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.500+0000 7f9b64f65700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9b60159890 0x7f9b6015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:55.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.500+0000 7f9b64f65700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b6015a160 con 0x7f9b60069560 2023-10-09T06:30:55.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.500+0000 7f9b64f65700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b600d13a0 con 0x7f9b60159c60 2023-10-09T06:30:55.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.500+0000 7f9b64f65700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b6015e140 con 0x7f9b60159890 2023-10-09T06:30:55.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.500+0000 7f9b5ed9d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f9b60069560 0x7f9b600ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60974/0 (socket says 172.21.15.163:60974) 2023-10-09T06:30:55.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.500+0000 7f9b5ed9d700 1 -- 172.21.15.163:0/3844456068 learned_addr learned my addr 172.21.15.163:0/3844456068 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:55.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.500+0000 7f9b5e59c700 1 --2- 172.21.15.163:0/3844456068 >> v2:172.21.15.181:3300/0 conn(0x7f9b60159890 0x7f9b6015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:55.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.500+0000 7f9b5d59a700 1 -- 172.21.15.163:0/3844456068 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2770656825 0 0) 0x7f9b6015a160 con 0x7f9b60069560 2023-10-09T06:30:55.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.500+0000 7f9b5d59a700 1 -- 172.21.15.163:0/3844456068 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b48003570 con 0x7f9b60069560 2023-10-09T06:30:55.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.500+0000 7f9b5d59a700 1 -- 172.21.15.163:0/3844456068 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1642758243 0 0) 0x7f9b48003570 con 0x7f9b60069560 2023-10-09T06:30:55.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.500+0000 7f9b5d59a700 1 -- 172.21.15.163:0/3844456068 >> v2:172.21.15.181:3300/0 conn(0x7f9b60159890 msgr2=0x7f9b6015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:55.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.500+0000 7f9b5d59a700 1 --2- 172.21.15.163:0/3844456068 >> v2:172.21.15.181:3300/0 conn(0x7f9b60159890 0x7f9b6015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.500+0000 7f9b5d59a700 1 -- 172.21.15.163:0/3844456068 >> v1:172.21.15.181:6789/0 conn(0x7f9b60159c60 legacy=0x7f9b6015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:55.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.500+0000 7f9b5d59a700 1 -- 172.21.15.163:0/3844456068 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b6015a3d0 con 0x7f9b60069560 2023-10-09T06:30:55.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.500+0000 7f9b5d59a700 1 -- 172.21.15.163:0/3844456068 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f9b50001470 con 0x7f9b60069560 2023-10-09T06:30:55.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.500+0000 7f9b5d59a700 1 -- 172.21.15.163:0/3844456068 >> v1:172.21.15.163:6789/0 conn(0x7f9b60069560 legacy=0x7f9b600ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:55.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.500+0000 7f9b5d59a700 1 --2- 172.21.15.163:0/3844456068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b48003a60 0x7f9b48003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:55.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.500+0000 7f9b5d59a700 1 --2- 172.21.15.163:0/3844456068 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b48004810 0x7f9b48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:55.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.500+0000 7f9b5d59a700 1 --2- 172.21.15.163:0/3844456068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b480051c0 0x7f9b4800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:55.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.501+0000 7f9b5d59a700 1 -- 172.21.15.163:0/3844456068 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6015e140 con 0x7f9b48003a60 2023-10-09T06:30:55.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.501+0000 7f9b5d59a700 1 -- 172.21.15.163:0/3844456068 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4800c0d0 con 0x7f9b480051c0 2023-10-09T06:30:55.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.501+0000 7f9b5d59a700 1 -- 172.21.15.163:0/3844456068 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4800c290 con 0x7f9b48004810 2023-10-09T06:30:55.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.501+0000 7f9b5e59c700 1 --2- 172.21.15.163:0/3844456068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b48003a60 0x7f9b48003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:55.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.501+0000 7f9b5dd9b700 1 --2- 172.21.15.163:0/3844456068 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b48004810 0x7f9b48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:55.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.501+0000 7f9b5ed9d700 1 --2- 172.21.15.163:0/3844456068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b480051c0 0x7f9b4800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:55.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.501+0000 7f9b5e59c700 1 -- 172.21.15.163:0/3844456068 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b48004810 msgr2=0x7f9b48004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:55.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.501+0000 7f9b5e59c700 1 --2- 172.21.15.163:0/3844456068 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b48004810 0x7f9b48004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.501+0000 7f9b5e59c700 1 -- 172.21.15.163:0/3844456068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b480051c0 msgr2=0x7f9b4800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:55.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.501+0000 7f9b5e59c700 1 --2- 172.21.15.163:0/3844456068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b480051c0 0x7f9b4800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.501+0000 7f9b5e59c700 1 -- 172.21.15.163:0/3844456068 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b48003570 con 0x7f9b48003a60 2023-10-09T06:30:55.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.501+0000 7f9b5ed9d700 1 --2- 172.21.15.163:0/3844456068 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b480051c0 0x7f9b4800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:30:55.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.501+0000 7f9b5ed9d700 1 -- 172.21.15.163:0/3844456068 reap_dead start 2023-10-09T06:30:55.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.501+0000 7f9b5e59c700 1 --2- 172.21.15.163:0/3844456068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b48003a60 0x7f9b48003e70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9b58009c00 tx=0x7f9b580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dbd8609946143a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:55.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.501+0000 7f9b5d59a700 1 -- 172.21.15.163:0/3844456068 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5800e710 con 0x7f9b48003a60 2023-10-09T06:30:55.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.501+0000 7f9b5d59a700 1 -- 172.21.15.163:0/3844456068 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b5800e870 con 0x7f9b48003a60 2023-10-09T06:30:55.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.501+0000 7f9b5d59a700 1 -- 172.21.15.163:0/3844456068 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5800ec30 con 0x7f9b48003a60 2023-10-09T06:30:55.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.502+0000 7f9b64f65700 1 -- 172.21.15.163:0/3844456068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b48003a60 msgr2=0x7f9b48003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:55.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.502+0000 7f9b64f65700 1 --2- 172.21.15.163:0/3844456068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b48003a60 0x7f9b48003e70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9b58009c00 tx=0x7f9b580091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.502+0000 7f9b64f65700 1 -- 172.21.15.163:0/3844456068 shutdown_connections 2023-10-09T06:30:55.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.502+0000 7f9b64f65700 1 --2- 172.21.15.163:0/3844456068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b48003a60 0x7f9b48003e70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.502+0000 7f9b64f65700 1 -- 172.21.15.163:0/3844456068 >> 172.21.15.163:0/3844456068 conn(0x7f9b60068e20 msgr2=0x7f9b600d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:55.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.502+0000 7f9b64f65700 1 -- 172.21.15.163:0/3844456068 shutdown_connections 2023-10-09T06:30:55.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.502+0000 7f9b64f65700 1 -- 172.21.15.163:0/3844456068 wait complete. 2023-10-09T06:30:55.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.503+0000 7f9b64f65700 1 Processor -- start 2023-10-09T06:30:55.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.503+0000 7f9b64f65700 1 -- start start 2023-10-09T06:30:55.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.503+0000 7f9b64f65700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b48003a60 0x7f9b60141330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:55.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.503+0000 7f9b64f65700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b600b3590 0x7f9b600b5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:55.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.503+0000 7f9b64f65700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b600b3a00 0x7f9b600b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:55.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.503+0000 7f9b64f65700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b600b62e0 con 0x7f9b600b3a00 2023-10-09T06:30:55.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.503+0000 7f9b64f65700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b600ce070 con 0x7f9b600b3590 2023-10-09T06:30:55.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.503+0000 7f9b64f65700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b600b3f80 con 0x7f9b48003a60 2023-10-09T06:30:55.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.504+0000 7f9b5ed9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b600b3a00 0x7f9b600b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:55.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.504+0000 7f9b5ed9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b600b3a00 0x7f9b600b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44222/0 (socket says 172.21.15.163:44222) 2023-10-09T06:30:55.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.504+0000 7f9b5ed9d700 1 -- 172.21.15.163:0/1366987846 learned_addr learned my addr 172.21.15.163:0/1366987846 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:55.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.504+0000 7f9b5e59c700 1 --2- 172.21.15.163:0/1366987846 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b48003a60 0x7f9b60141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:55.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.504+0000 7f9b5dd9b700 1 --2- 172.21.15.163:0/1366987846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b600b3590 0x7f9b600b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:55.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.504+0000 7f9b5ed9d700 1 -- 172.21.15.163:0/1366987846 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b48003a60 msgr2=0x7f9b60141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:55.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.504+0000 7f9b5ed9d700 1 --2- 172.21.15.163:0/1366987846 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b48003a60 0x7f9b60141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.504+0000 7f9b5ed9d700 1 -- 172.21.15.163:0/1366987846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b600b3590 msgr2=0x7f9b600b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:55.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.504+0000 7f9b5ed9d700 1 --2- 172.21.15.163:0/1366987846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b600b3590 0x7f9b600b5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.504+0000 7f9b5ed9d700 1 -- 172.21.15.163:0/1366987846 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b600b4260 con 0x7f9b600b3a00 2023-10-09T06:30:55.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.504+0000 7f9b5ed9d700 1 --2- 172.21.15.163:0/1366987846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b600b3a00 0x7f9b600b5da0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9b50000fd0 tx=0x7f9b50001470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:55.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.504+0000 7f9b4e7fc700 1 -- 172.21.15.163:0/1366987846 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b50012590 con 0x7f9b600b3a00 2023-10-09T06:30:55.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.504+0000 7f9b4e7fc700 1 -- 172.21.15.163:0/1366987846 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b500126f0 con 0x7f9b600b3a00 2023-10-09T06:30:55.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.504+0000 7f9b4e7fc700 1 -- 172.21.15.163:0/1366987846 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b50012a90 con 0x7f9b600b3a00 2023-10-09T06:30:55.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.504+0000 7f9b64f65700 1 -- 172.21.15.163:0/1366987846 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b600cda20 con 0x7f9b600b3a00 2023-10-09T06:30:55.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.504+0000 7f9b64f65700 1 -- 172.21.15.163:0/1366987846 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b60151f50 con 0x7f9b600b3a00 2023-10-09T06:30:55.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.505+0000 7f9b64f65700 1 -- 172.21.15.163:0/1366987846 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b6004c3c0 con 0x7f9b600b3a00 2023-10-09T06:30:55.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.505+0000 7f9b4e7fc700 1 -- 172.21.15.163:0/1366987846 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9b5001d460 con 0x7f9b600b3a00 2023-10-09T06:30:55.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.506+0000 7f9b4e7fc700 1 --2- 172.21.15.163:0/1366987846 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9b40066f60 0x7f9b40069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:55.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.506+0000 7f9b4e7fc700 1 -- 172.21.15.163:0/1366987846 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9b50084220 con 0x7f9b600b3a00 2023-10-09T06:30:55.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.506+0000 7f9b5e59c700 1 --2- 172.21.15.163:0/1366987846 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9b40066f60 0x7f9b40069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:55.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.506+0000 7f9b5e59c700 1 --2- 172.21.15.163:0/1366987846 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9b40066f60 0x7f9b40069410 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9b580091e0 tx=0x7f9b5801a000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:55.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.510+0000 7f9b4e7fc700 1 -- 172.21.15.163:0/1366987846 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9b50051550 con 0x7f9b600b3a00 2023-10-09T06:30:55.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.690+0000 7f9b64f65700 1 -- 172.21.15.163:0/1366987846 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9b601522f0 con 0x7f9b600b3a00 2023-10-09T06:30:55.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.690+0000 7f9b4e7fc700 1 -- 172.21.15.163:0/1366987846 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9b50054b70 con 0x7f9b600b3a00 2023-10-09T06:30:55.691 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 32 2023-10-09T06:30:55.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.692+0000 7f9b64f65700 1 -- 172.21.15.163:0/1366987846 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9b40066f60 msgr2=0x7f9b40069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:55.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.692+0000 7f9b64f65700 1 --2- 172.21.15.163:0/1366987846 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9b40066f60 0x7f9b40069410 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9b580091e0 tx=0x7f9b5801a000 comp rx=0 tx=0).stop 2023-10-09T06:30:55.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.692+0000 7f9b64f65700 1 -- 172.21.15.163:0/1366987846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b600b3a00 msgr2=0x7f9b600b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:55.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.692+0000 7f9b64f65700 1 --2- 172.21.15.163:0/1366987846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b600b3a00 0x7f9b600b5da0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9b50000fd0 tx=0x7f9b50001470 comp rx=0 tx=0).stop 2023-10-09T06:30:55.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.692+0000 7f9b64f65700 1 -- 172.21.15.163:0/1366987846 shutdown_connections 2023-10-09T06:30:55.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.692+0000 7f9b64f65700 1 --2- 172.21.15.163:0/1366987846 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9b40066f60 0x7f9b40069410 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.692+0000 7f9b64f65700 1 --2- 172.21.15.163:0/1366987846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b600b3a00 0x7f9b600b5da0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.692+0000 7f9b64f65700 1 --2- 172.21.15.163:0/1366987846 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b48003a60 0x7f9b60141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.692+0000 7f9b64f65700 1 --2- 172.21.15.163:0/1366987846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b600b3590 0x7f9b600b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.692+0000 7f9b64f65700 1 -- 172.21.15.163:0/1366987846 >> 172.21.15.163:0/1366987846 conn(0x7f9b60068e20 msgr2=0x7f9b6015d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:55.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.692+0000 7f9b64f65700 1 -- 172.21.15.163:0/1366987846 shutdown_connections 2023-10-09T06:30:55.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.692+0000 7f9b64f65700 1 -- 172.21.15.163:0/1366987846 wait complete. 2023-10-09T06:30:55.706 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-09T06:30:55.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.824+0000 7f5c296a0700 1 Processor -- start 2023-10-09T06:30:55.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.824+0000 7f5c296a0700 1 -- start start 2023-10-09T06:30:55.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.824+0000 7f5c296a0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c24159c60 0x7f5c2415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:55.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.824+0000 7f5c296a0700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5c2415db10 0x7f5c24069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:55.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.824+0000 7f5c296a0700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c2415a160 con 0x7f5c24159890 2023-10-09T06:30:55.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.824+0000 7f5c296a0700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c240d13a0 con 0x7f5c2415db10 2023-10-09T06:30:55.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.824+0000 7f5c296a0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2415e140 con 0x7f5c24159c60 2023-10-09T06:30:55.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.824+0000 7f5c237fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5c2415db10 0x7f5c24069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:55.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.825+0000 7f5c22ffd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f5c24159890 0x7f5c2415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60984/0 (socket says 172.21.15.163:60984) 2023-10-09T06:30:55.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.825+0000 7f5c22ffd700 1 -- 172.21.15.163:0/298914138 learned_addr learned my addr 172.21.15.163:0/298914138 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:55.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.825+0000 7f5c227fc700 1 --2- 172.21.15.163:0/298914138 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c24159c60 0x7f5c2415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:55.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.825+0000 7f5c237fe700 1 -- 172.21.15.163:0/298914138 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c24159c60 msgr2=0x7f5c2415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:55.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.825+0000 7f5c237fe700 1 --2- 172.21.15.163:0/298914138 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c24159c60 0x7f5c2415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.825+0000 7f5c237fe700 1 -- 172.21.15.163:0/298914138 >> v1:172.21.15.163:6789/0 conn(0x7f5c24159890 legacy=0x7f5c2415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:55.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.825+0000 7f5c237fe700 1 -- 172.21.15.163:0/298914138 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c2415a3d0 con 0x7f5c2415db10 2023-10-09T06:30:55.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.825+0000 7f5c21ffb700 1 -- 172.21.15.163:0/298914138 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 656676729 0 0) 0x7f5c2415a160 con 0x7f5c24159890 2023-10-09T06:30:55.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.825+0000 7f5c237fe700 1 --2- 172.21.15.163:0/298914138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c2415db10 0x7f5c24069560 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c0098e0 tx=0x7f5c1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef573ef8d48bb61 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:55.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.825+0000 7f5c21ffb700 1 -- 172.21.15.163:0/298914138 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c009e80 con 0x7f5c2415db10 2023-10-09T06:30:55.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.825+0000 7f5c21ffb700 1 -- 172.21.15.163:0/298914138 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1c00b860 con 0x7f5c2415db10 2023-10-09T06:30:55.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.825+0000 7f5c21ffb700 1 -- 172.21.15.163:0/298914138 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c00bbe0 con 0x7f5c2415db10 2023-10-09T06:30:55.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.825+0000 7f5c296a0700 1 -- 172.21.15.163:0/298914138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c2415db10 msgr2=0x7f5c24069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:55.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.825+0000 7f5c296a0700 1 --2- 172.21.15.163:0/298914138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c2415db10 0x7f5c24069560 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c0098e0 tx=0x7f5c1c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.826+0000 7f5c296a0700 1 -- 172.21.15.163:0/298914138 shutdown_connections 2023-10-09T06:30:55.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.826+0000 7f5c296a0700 1 --2- 172.21.15.163:0/298914138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c2415db10 0x7f5c24069560 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.826+0000 7f5c296a0700 1 --2- 172.21.15.163:0/298914138 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c24159c60 0x7f5c2415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.826+0000 7f5c296a0700 1 -- 172.21.15.163:0/298914138 >> 172.21.15.163:0/298914138 conn(0x7f5c24068e20 msgr2=0x7f5c240d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:55.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.826+0000 7f5c296a0700 1 -- 172.21.15.163:0/298914138 shutdown_connections 2023-10-09T06:30:55.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.826+0000 7f5c296a0700 1 -- 172.21.15.163:0/298914138 wait complete. 2023-10-09T06:30:55.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.827+0000 7f5c296a0700 1 Processor -- start 2023-10-09T06:30:55.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.827+0000 7f5c296a0700 1 -- start start 2023-10-09T06:30:55.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.827+0000 7f5c296a0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c24159890 0x7f5c2414c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:55.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.827+0000 7f5c296a0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c24159c60 0x7f5c24156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:55.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.827+0000 7f5c296a0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c2415db10 0x7f5c241573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:55.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.828+0000 7f5c296a0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c240cfb60 con 0x7f5c24159890 2023-10-09T06:30:55.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.828+0000 7f5c296a0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c240cf230 con 0x7f5c2415db10 2023-10-09T06:30:55.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.828+0000 7f5c296a0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c240cf510 con 0x7f5c24159c60 2023-10-09T06:30:55.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.828+0000 7f5c22ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c24159890 0x7f5c2414c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:55.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.828+0000 7f5c22ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c24159890 0x7f5c2414c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44234/0 (socket says 172.21.15.163:44234) 2023-10-09T06:30:55.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.828+0000 7f5c22ffd700 1 -- 172.21.15.163:0/1662397187 learned_addr learned my addr 172.21.15.163:0/1662397187 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:55.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.828+0000 7f5c227fc700 1 --2- 172.21.15.163:0/1662397187 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c24159c60 0x7f5c24156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:55.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.828+0000 7f5c237fe700 1 --2- 172.21.15.163:0/1662397187 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c2415db10 0x7f5c241573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:55.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.828+0000 7f5c22ffd700 1 -- 172.21.15.163:0/1662397187 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c24159c60 msgr2=0x7f5c24156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:55.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.828+0000 7f5c22ffd700 1 --2- 172.21.15.163:0/1662397187 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c24159c60 0x7f5c24156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.828+0000 7f5c22ffd700 1 -- 172.21.15.163:0/1662397187 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c2415db10 msgr2=0x7f5c241573d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:55.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.828+0000 7f5c22ffd700 1 --2- 172.21.15.163:0/1662397187 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c2415db10 0x7f5c241573d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:55.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.828+0000 7f5c22ffd700 1 -- 172.21.15.163:0/1662397187 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1c009420 con 0x7f5c24159890 2023-10-09T06:30:55.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.828+0000 7f5c237fe700 1 --2- 172.21.15.163:0/1662397187 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c2415db10 0x7f5c241573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:30:55.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.828+0000 7f5c22ffd700 1 --2- 172.21.15.163:0/1662397187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c24159890 0x7f5c2414c690 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5c1400cd80 tx=0x7f5c14001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:55.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.828+0000 7f5c0affd700 1 -- 172.21.15.163:0/1662397187 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1400ab10 con 0x7f5c24159890 2023-10-09T06:30:55.835 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.828+0000 7f5c0affd700 1 -- 172.21.15.163:0/1662397187 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1400ac70 con 0x7f5c24159890 2023-10-09T06:30:55.835 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.828+0000 7f5c0affd700 1 -- 172.21.15.163:0/1662397187 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c140070f0 con 0x7f5c24159890 2023-10-09T06:30:55.835 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.828+0000 7f5c296a0700 1 -- 172.21.15.163:0/1662397187 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c240cf7f0 con 0x7f5c24159890 2023-10-09T06:30:55.835 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.828+0000 7f5c296a0700 1 -- 172.21.15.163:0/1662397187 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c240cee50 con 0x7f5c24159890 2023-10-09T06:30:55.835 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.829+0000 7f5c296a0700 1 -- 172.21.15.163:0/1662397187 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c2404c3c0 con 0x7f5c24159890 2023-10-09T06:30:55.835 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.830+0000 7f5c0affd700 1 -- 172.21.15.163:0/1662397187 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5c140074e0 con 0x7f5c24159890 2023-10-09T06:30:55.835 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.830+0000 7f5c0affd700 1 --2- 172.21.15.163:0/1662397187 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5c10066f30 0x7f5c100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:55.835 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.830+0000 7f5c0affd700 1 -- 172.21.15.163:0/1662397187 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5c14084980 con 0x7f5c24159890 2023-10-09T06:30:55.835 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.830+0000 7f5c227fc700 1 --2- 172.21.15.163:0/1662397187 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5c10066f30 0x7f5c100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:55.835 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.830+0000 7f5c227fc700 1 --2- 172.21.15.163:0/1662397187 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5c10066f30 0x7f5c100693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5c240cc610 tx=0x7f5c18008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:55.836 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:55.834+0000 7f5c0affd700 1 -- 172.21.15.163:0/1662397187 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c14051c70 con 0x7f5c24159890 2023-10-09T06:30:56.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.017+0000 7f5c296a0700 1 -- 172.21.15.163:0/1662397187 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5c240caba0 con 0x7f5c24159890 2023-10-09T06:30:56.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.017+0000 7f5c0affd700 1 -- 172.21.15.163:0/1662397187 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5c14001c20 con 0x7f5c24159890 2023-10-09T06:30:56.102 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 64 2023-10-09T06:30:56.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.019+0000 7f5c296a0700 1 -- 172.21.15.163:0/1662397187 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5c10066f30 msgr2=0x7f5c100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:56.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.019+0000 7f5c296a0700 1 --2- 172.21.15.163:0/1662397187 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5c10066f30 0x7f5c100693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5c240cc610 tx=0x7f5c18008040 comp rx=0 tx=0).stop 2023-10-09T06:30:56.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.019+0000 7f5c296a0700 1 -- 172.21.15.163:0/1662397187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c24159890 msgr2=0x7f5c2414c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:56.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.019+0000 7f5c296a0700 1 --2- 172.21.15.163:0/1662397187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c24159890 0x7f5c2414c690 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5c1400cd80 tx=0x7f5c14001ae0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.020+0000 7f5c296a0700 1 -- 172.21.15.163:0/1662397187 shutdown_connections 2023-10-09T06:30:56.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.020+0000 7f5c296a0700 1 --2- 172.21.15.163:0/1662397187 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5c10066f30 0x7f5c100693e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.020+0000 7f5c296a0700 1 --2- 172.21.15.163:0/1662397187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c24159890 0x7f5c2414c690 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.020+0000 7f5c296a0700 1 --2- 172.21.15.163:0/1662397187 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c2415db10 0x7f5c241573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.020+0000 7f5c296a0700 1 --2- 172.21.15.163:0/1662397187 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c24159c60 0x7f5c24156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.020+0000 7f5c296a0700 1 -- 172.21.15.163:0/1662397187 >> 172.21.15.163:0/1662397187 conn(0x7f5c24068e20 msgr2=0x7f5c2414d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:56.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.020+0000 7f5c296a0700 1 -- 172.21.15.163:0/1662397187 shutdown_connections 2023-10-09T06:30:56.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.020+0000 7f5c296a0700 1 -- 172.21.15.163:0/1662397187 wait complete. 2023-10-09T06:30:56.107 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:30:56.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.225+0000 7fa8e4ff4700 1 Processor -- start 2023-10-09T06:30:56.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.225+0000 7fa8e4ff4700 1 -- start start 2023-10-09T06:30:56.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.226+0000 7fa8e4ff4700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa8e0159c60 0x7fa8e015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:56.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.226+0000 7fa8e4ff4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa8e015db10 0x7fa8e0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:56.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.226+0000 7fa8e4ff4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8e015a160 con 0x7fa8e0159890 2023-10-09T06:30:56.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.226+0000 7fa8e4ff4700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e015df50 con 0x7fa8e0159c60 2023-10-09T06:30:56.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.226+0000 7fa8e4ff4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e00d13a0 con 0x7fa8e015db10 2023-10-09T06:30:56.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.226+0000 7fa8ddd9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa8e0159c60 0x7fa8e015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:56.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.226+0000 7fa8ddd9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa8e0159c60 0x7fa8e015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44242/0 (socket says 172.21.15.163:44242) 2023-10-09T06:30:56.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.226+0000 7fa8ddd9b700 1 -- 172.21.15.163:0/1139279839 learned_addr learned my addr 172.21.15.163:0/1139279839 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:56.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.226+0000 7fa8ded9d700 1 --2- 172.21.15.163:0/1139279839 >> v2:172.21.15.181:3300/0 conn(0x7fa8e015db10 0x7fa8e0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:56.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.227+0000 7fa8ddd9b700 1 -- 172.21.15.163:0/1139279839 >> v2:172.21.15.181:3300/0 conn(0x7fa8e015db10 msgr2=0x7fa8e0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:56.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.227+0000 7fa8ddd9b700 1 --2- 172.21.15.163:0/1139279839 >> v2:172.21.15.181:3300/0 conn(0x7fa8e015db10 0x7fa8e0069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.227+0000 7fa8ddd9b700 1 -- 172.21.15.163:0/1139279839 >> v1:172.21.15.181:6789/0 conn(0x7fa8e0159890 legacy=0x7fa8e015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:56.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.227+0000 7fa8ddd9b700 1 -- 172.21.15.163:0/1139279839 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e015e140 con 0x7fa8e0159c60 2023-10-09T06:30:56.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.227+0000 7fa8ddd9b700 1 --2- 172.21.15.163:0/1139279839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8e0159c60 0x7fa8e015d5d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa8e006a890 tx=0x7fa8d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db44d26a1994c3b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:56.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.227+0000 7fa8dd59a700 1 -- 172.21.15.163:0/1139279839 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4009e70 con 0x7fa8e0159c60 2023-10-09T06:30:56.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.227+0000 7fa8dd59a700 1 -- 172.21.15.163:0/1139279839 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d400b860 con 0x7fa8e0159c60 2023-10-09T06:30:56.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.227+0000 7fa8dd59a700 1 -- 172.21.15.163:0/1139279839 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d400bc00 con 0x7fa8e0159c60 2023-10-09T06:30:56.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.227+0000 7fa8e4ff4700 1 -- 172.21.15.163:0/1139279839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8e0159c60 msgr2=0x7fa8e015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:56.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.227+0000 7fa8e4ff4700 1 --2- 172.21.15.163:0/1139279839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8e0159c60 0x7fa8e015d5d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa8e006a890 tx=0x7fa8d4009250 comp rx=0 tx=0).stop 2023-10-09T06:30:56.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.227+0000 7fa8e4ff4700 1 -- 172.21.15.163:0/1139279839 shutdown_connections 2023-10-09T06:30:56.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.227+0000 7fa8e4ff4700 1 --2- 172.21.15.163:0/1139279839 >> v2:172.21.15.181:3300/0 conn(0x7fa8e015db10 0x7fa8e0069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.227+0000 7fa8e4ff4700 1 --2- 172.21.15.163:0/1139279839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8e0159c60 0x7fa8e015d5d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.227+0000 7fa8e4ff4700 1 -- 172.21.15.163:0/1139279839 >> 172.21.15.163:0/1139279839 conn(0x7fa8e0068e20 msgr2=0x7fa8e00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:56.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.227+0000 7fa8e4ff4700 1 -- 172.21.15.163:0/1139279839 shutdown_connections 2023-10-09T06:30:56.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.227+0000 7fa8e4ff4700 1 -- 172.21.15.163:0/1139279839 wait complete. 2023-10-09T06:30:56.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.228+0000 7fa8e4ff4700 1 Processor -- start 2023-10-09T06:30:56.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.228+0000 7fa8e4ff4700 1 -- start start 2023-10-09T06:30:56.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.229+0000 7fa8e4ff4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8e0159890 0x7fa8e0140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:56.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.229+0000 7fa8e4ff4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8e0159c60 0x7fa8e00b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:56.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.229+0000 7fa8e4ff4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8e015db10 0x7fa8e00b5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:56.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.229+0000 7fa8e4ff4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e0141300 con 0x7fa8e015db10 2023-10-09T06:30:56.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.229+0000 7fa8e4ff4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e00cd6d0 con 0x7fa8e0159890 2023-10-09T06:30:56.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.229+0000 7fa8e4ff4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e00cd9b0 con 0x7fa8e0159c60 2023-10-09T06:30:56.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.229+0000 7fa8ded9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8e015db10 0x7fa8e00b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:56.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.229+0000 7fa8ded9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8e015db10 0x7fa8e00b5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44252/0 (socket says 172.21.15.163:44252) 2023-10-09T06:30:56.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.229+0000 7fa8ded9d700 1 -- 172.21.15.163:0/37479354 learned_addr learned my addr 172.21.15.163:0/37479354 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:56.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.229+0000 7fa8ddd9b700 1 --2- 172.21.15.163:0/37479354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8e0159c60 0x7fa8e00b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:56.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.229+0000 7fa8de59c700 1 --2- 172.21.15.163:0/37479354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8e0159890 0x7fa8e0140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:56.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.230+0000 7fa8ded9d700 1 -- 172.21.15.163:0/37479354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8e0159c60 msgr2=0x7fa8e00b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:56.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.230+0000 7fa8ded9d700 1 --2- 172.21.15.163:0/37479354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8e0159c60 0x7fa8e00b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.230+0000 7fa8ded9d700 1 -- 172.21.15.163:0/37479354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8e0159890 msgr2=0x7fa8e0140dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:56.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.230+0000 7fa8ded9d700 1 --2- 172.21.15.163:0/37479354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8e0159890 0x7fa8e0140dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.230+0000 7fa8ded9d700 1 -- 172.21.15.163:0/37479354 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d40094f0 con 0x7fa8e015db10 2023-10-09T06:30:56.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.230+0000 7fa8ddd9b700 1 --2- 172.21.15.163:0/37479354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8e0159c60 0x7fa8e00b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:30:56.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.230+0000 7fa8ded9d700 1 --2- 172.21.15.163:0/37479354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8e015db10 0x7fa8e00b5250 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa8d800e8d0 tx=0x7fa8d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:56.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.230+0000 7fa8ce7fc700 1 -- 172.21.15.163:0/37479354 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d800ee70 con 0x7fa8e015db10 2023-10-09T06:30:56.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.230+0000 7fa8ce7fc700 1 -- 172.21.15.163:0/37479354 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d800c850 con 0x7fa8e015db10 2023-10-09T06:30:56.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.230+0000 7fa8ce7fc700 1 -- 172.21.15.163:0/37479354 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d800cc50 con 0x7fa8e015db10 2023-10-09T06:30:56.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.230+0000 7fa8e4ff4700 1 -- 172.21.15.163:0/37479354 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8e00cdc90 con 0x7fa8e015db10 2023-10-09T06:30:56.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.230+0000 7fa8e4ff4700 1 -- 172.21.15.163:0/37479354 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e00ce250 con 0x7fa8e015db10 2023-10-09T06:30:56.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.231+0000 7fa8e4ff4700 1 -- 172.21.15.163:0/37479354 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8e00ce440 con 0x7fa8e015db10 2023-10-09T06:30:56.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.231+0000 7fa8ce7fc700 1 -- 172.21.15.163:0/37479354 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa8d8013680 con 0x7fa8e015db10 2023-10-09T06:30:56.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.232+0000 7fa8ce7fc700 1 --2- 172.21.15.163:0/37479354 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa8c0066f30 0x7fa8c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:56.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.232+0000 7fa8de59c700 1 --2- 172.21.15.163:0/37479354 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa8c0066f30 0x7fa8c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:56.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.232+0000 7fa8ce7fc700 1 -- 172.21.15.163:0/37479354 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa8d8084680 con 0x7fa8e015db10 2023-10-09T06:30:56.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.232+0000 7fa8de59c700 1 --2- 172.21.15.163:0/37479354 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa8c0066f30 0x7fa8c00693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0007fd0 tx=0x7fa8d0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:56.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.236+0000 7fa8ce7fc700 1 -- 172.21.15.163:0/37479354 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa8d80518f0 con 0x7fa8e015db10 2023-10-09T06:30:56.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.461+0000 7fa8e4ff4700 1 -- 172.21.15.163:0/37479354 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8e004c3c0 con 0x7fa8e015db10 2023-10-09T06:30:56.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.461+0000 7fa8ce7fc700 1 -- 172.21.15.163:0/37479354 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4603 (secure 0 0 0) 0x7fa8d8054f10 con 0x7fa8e015db10 2023-10-09T06:30:56.463 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:30:56.463 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3440080308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3440080308},{"type":"v1","addr":"172.21.15.181:6837","nonce":3440080308}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1990962358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1990962358},{"type":"v1","addr":"172.21.15.163:6835","nonce":1990962358}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2270076605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2270076605},{"type":"v1","addr":"172.21.15.163:6834","nonce":2270076605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:30:43.133124+0000","modified":"2023-10-09T06:30:46.760523+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/1149296546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1149296546},{"type":"v1","addr":"172.21.15.181:6836","nonce":1149296546}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-09T06:30:56.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.464+0000 7fa8e4ff4700 1 -- 172.21.15.163:0/37479354 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa8c0066f30 msgr2=0x7fa8c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:56.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.464+0000 7fa8e4ff4700 1 --2- 172.21.15.163:0/37479354 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa8c0066f30 0x7fa8c00693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0007fd0 tx=0x7fa8d0009040 comp rx=0 tx=0).stop 2023-10-09T06:30:56.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.464+0000 7fa8e4ff4700 1 -- 172.21.15.163:0/37479354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8e015db10 msgr2=0x7fa8e00b5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:56.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.464+0000 7fa8e4ff4700 1 --2- 172.21.15.163:0/37479354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8e015db10 0x7fa8e00b5250 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa8d800e8d0 tx=0x7fa8d80091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.464+0000 7fa8e4ff4700 1 -- 172.21.15.163:0/37479354 shutdown_connections 2023-10-09T06:30:56.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.464+0000 7fa8e4ff4700 1 --2- 172.21.15.163:0/37479354 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa8c0066f30 0x7fa8c00693e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.464+0000 7fa8e4ff4700 1 --2- 172.21.15.163:0/37479354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8e015db10 0x7fa8e00b5250 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.464+0000 7fa8e4ff4700 1 --2- 172.21.15.163:0/37479354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8e0159890 0x7fa8e0140dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.464+0000 7fa8e4ff4700 1 --2- 172.21.15.163:0/37479354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8e0159c60 0x7fa8e00b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.464+0000 7fa8e4ff4700 1 -- 172.21.15.163:0/37479354 >> 172.21.15.163:0/37479354 conn(0x7fa8e0068e20 msgr2=0x7fa8e0157230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:56.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.464+0000 7fa8e4ff4700 1 -- 172.21.15.163:0/37479354 shutdown_connections 2023-10-09T06:30:56.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.464+0000 7fa8e4ff4700 1 -- 172.21.15.163:0/37479354 wait complete. 2023-10-09T06:30:56.467 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 8 2023-10-09T06:30:56.477 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:30:56.592 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.591+0000 7f45c9689700 1 Processor -- start 2023-10-09T06:30:56.592 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.592+0000 7f45c9689700 1 -- start start 2023-10-09T06:30:56.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.592+0000 7f45c9689700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f45c406bcd0 0x7f45c4140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:56.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.592+0000 7f45c9689700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45c40c9510 0x7f45c4141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:56.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.592+0000 7f45c9689700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45c415c2a0 con 0x7f45c406c0a0 2023-10-09T06:30:56.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.592+0000 7f45c9689700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45c415b690 con 0x7f45c406bcd0 2023-10-09T06:30:56.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.592+0000 7f45c9689700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c415b970 con 0x7f45c40c9510 2023-10-09T06:30:56.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.592+0000 7f45c27fc700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f45c406c0a0 0x7f45c40c8e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60992/0 (socket says 172.21.15.163:60992) 2023-10-09T06:30:56.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.592+0000 7f45c27fc700 1 -- 172.21.15.163:0/4162398101 learned_addr learned my addr 172.21.15.163:0/4162398101 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:56.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.593+0000 7f45c2ffd700 1 --2- 172.21.15.163:0/4162398101 >> v2:172.21.15.181:3300/0 conn(0x7f45c406bcd0 0x7f45c4140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:56.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.593+0000 7f45c37fe700 1 --2- 172.21.15.163:0/4162398101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45c40c9510 0x7f45c4141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:56.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.593+0000 7f45c1ffb700 1 -- 172.21.15.163:0/4162398101 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2376628873 0 0) 0x7f45c415c2a0 con 0x7f45c406c0a0 2023-10-09T06:30:56.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.593+0000 7f45c1ffb700 1 -- 172.21.15.163:0/4162398101 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45ac003570 con 0x7f45c406c0a0 2023-10-09T06:30:56.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.593+0000 7f45c1ffb700 1 -- 172.21.15.163:0/4162398101 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 229230446 0 0) 0x7f45ac003570 con 0x7f45c406c0a0 2023-10-09T06:30:56.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.593+0000 7f45c1ffb700 1 -- 172.21.15.163:0/4162398101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45c40c9510 msgr2=0x7f45c4141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:56.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.593+0000 7f45c1ffb700 1 --2- 172.21.15.163:0/4162398101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45c40c9510 0x7f45c4141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.593+0000 7f45c1ffb700 1 -- 172.21.15.163:0/4162398101 >> v2:172.21.15.181:3300/0 conn(0x7f45c406bcd0 msgr2=0x7f45c4140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:56.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.593+0000 7f45c1ffb700 1 --2- 172.21.15.163:0/4162398101 >> v2:172.21.15.181:3300/0 conn(0x7f45c406bcd0 0x7f45c4140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.593+0000 7f45c1ffb700 1 -- 172.21.15.163:0/4162398101 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c415bc50 con 0x7f45c406c0a0 2023-10-09T06:30:56.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.593+0000 7f45c1ffb700 1 -- 172.21.15.163:0/4162398101 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f45b8001470 con 0x7f45c406c0a0 2023-10-09T06:30:56.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.593+0000 7f45c1ffb700 1 -- 172.21.15.163:0/4162398101 >> v1:172.21.15.163:6789/0 conn(0x7f45c406c0a0 legacy=0x7f45c40c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:56.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.593+0000 7f45c1ffb700 1 --2- 172.21.15.163:0/4162398101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45ac003a60 0x7f45ac003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:56.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.593+0000 7f45c1ffb700 1 --2- 172.21.15.163:0/4162398101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45ac004810 0x7f45ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:56.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.593+0000 7f45c1ffb700 1 --2- 172.21.15.163:0/4162398101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45ac0051c0 0x7f45ac00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:56.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.593+0000 7f45c1ffb700 1 -- 172.21.15.163:0/4162398101 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c415b690 con 0x7f45ac003a60 2023-10-09T06:30:56.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.593+0000 7f45c1ffb700 1 -- 172.21.15.163:0/4162398101 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c415b970 con 0x7f45ac0051c0 2023-10-09T06:30:56.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.593+0000 7f45c1ffb700 1 -- 172.21.15.163:0/4162398101 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac00bfa0 con 0x7f45ac004810 2023-10-09T06:30:56.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.593+0000 7f45c37fe700 1 --2- 172.21.15.163:0/4162398101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45c40c9510 0x7f45c4141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:30:56.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.593+0000 7f45c2ffd700 1 --2- 172.21.15.163:0/4162398101 >> v2:172.21.15.181:3300/0 conn(0x7f45c406bcd0 0x7f45c4140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:30:56.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.593+0000 7f45c2ffd700 1 --2- 172.21.15.163:0/4162398101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45ac003a60 0x7f45ac003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:56.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.593+0000 7f45c27fc700 1 --2- 172.21.15.163:0/4162398101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45ac004810 0x7f45ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:56.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.593+0000 7f45c37fe700 1 --2- 172.21.15.163:0/4162398101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45ac0051c0 0x7f45ac00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:56.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.593+0000 7f45c2ffd700 1 -- 172.21.15.163:0/4162398101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45ac004810 msgr2=0x7f45ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:56.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.593+0000 7f45c2ffd700 1 --2- 172.21.15.163:0/4162398101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45ac004810 0x7f45ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.593+0000 7f45c2ffd700 1 -- 172.21.15.163:0/4162398101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45ac0051c0 msgr2=0x7f45ac00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:56.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.593+0000 7f45c2ffd700 1 --2- 172.21.15.163:0/4162398101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45ac0051c0 0x7f45ac00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.593+0000 7f45c2ffd700 1 -- 172.21.15.163:0/4162398101 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45ac003570 con 0x7f45ac003a60 2023-10-09T06:30:56.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.594+0000 7f45c2ffd700 1 --2- 172.21.15.163:0/4162398101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45ac003a60 0x7f45ac003e70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f45b400e3c0 tx=0x7f45b400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5843aaf824482cb server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:56.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.594+0000 7f45c1ffb700 1 -- 172.21.15.163:0/4162398101 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b400ec70 con 0x7f45ac003a60 2023-10-09T06:30:56.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.594+0000 7f45c1ffb700 1 -- 172.21.15.163:0/4162398101 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b400f040 con 0x7f45ac003a60 2023-10-09T06:30:56.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.594+0000 7f45c1ffb700 1 -- 172.21.15.163:0/4162398101 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b4014480 con 0x7f45ac003a60 2023-10-09T06:30:56.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.594+0000 7f45c9689700 1 -- 172.21.15.163:0/4162398101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45ac003a60 msgr2=0x7f45ac003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:56.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.594+0000 7f45c9689700 1 --2- 172.21.15.163:0/4162398101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45ac003a60 0x7f45ac003e70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f45b400e3c0 tx=0x7f45b400b040 comp rx=0 tx=0).stop 2023-10-09T06:30:56.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.594+0000 7f45c37fe700 1 -- 172.21.15.163:0/4162398101 reap_dead start 2023-10-09T06:30:56.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.594+0000 7f45c9689700 1 -- 172.21.15.163:0/4162398101 shutdown_connections 2023-10-09T06:30:56.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.594+0000 7f45c9689700 1 -- 172.21.15.163:0/4162398101 >> 172.21.15.163:0/4162398101 conn(0x7f45c40ce600 msgr2=0x7f45c40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:56.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.594+0000 7f45c9689700 1 -- 172.21.15.163:0/4162398101 shutdown_connections 2023-10-09T06:30:56.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.594+0000 7f45c9689700 1 -- 172.21.15.163:0/4162398101 wait complete. 2023-10-09T06:30:56.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.595+0000 7f45c9689700 1 Processor -- start 2023-10-09T06:30:56.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.595+0000 7f45c9689700 1 -- start start 2023-10-09T06:30:56.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.596+0000 7f45c9689700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45ac003a60 0x7f45c40c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:56.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.596+0000 7f45c9689700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45c413d810 0x7f45c41583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:56.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.596+0000 7f45c9689700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45c40c5ec0 0x7f45c40c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:56.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.596+0000 7f45c9689700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c40c6870 con 0x7f45c40c5ec0 2023-10-09T06:30:56.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.596+0000 7f45c9689700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c40b1f00 con 0x7f45ac003a60 2023-10-09T06:30:56.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.596+0000 7f45c9689700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c413e090 con 0x7f45c413d810 2023-10-09T06:30:56.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.596+0000 7f45c37fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45c40c5ec0 0x7f45c40c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:56.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.596+0000 7f45c37fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45c40c5ec0 0x7f45c40c6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44266/0 (socket says 172.21.15.163:44266) 2023-10-09T06:30:56.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.596+0000 7f45c37fe700 1 -- 172.21.15.163:0/1929387169 learned_addr learned my addr 172.21.15.163:0/1929387169 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:56.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.596+0000 7f45c2ffd700 1 --2- 172.21.15.163:0/1929387169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45ac003a60 0x7f45c40c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:56.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.596+0000 7f45c27fc700 1 --2- 172.21.15.163:0/1929387169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45c413d810 0x7f45c41583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:56.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.597+0000 7f45c37fe700 1 -- 172.21.15.163:0/1929387169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45c413d810 msgr2=0x7f45c41583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:56.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.597+0000 7f45c37fe700 1 --2- 172.21.15.163:0/1929387169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45c413d810 0x7f45c41583e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.597+0000 7f45c37fe700 1 -- 172.21.15.163:0/1929387169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45ac003a60 msgr2=0x7f45c40c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:56.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.597+0000 7f45c37fe700 1 --2- 172.21.15.163:0/1929387169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45ac003a60 0x7f45c40c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.597+0000 7f45c37fe700 1 -- 172.21.15.163:0/1929387169 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c413e370 con 0x7f45c40c5ec0 2023-10-09T06:30:56.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.597+0000 7f45c27fc700 1 --2- 172.21.15.163:0/1929387169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45c413d810 0x7f45c41583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:30:56.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.597+0000 7f45c2ffd700 1 --2- 172.21.15.163:0/1929387169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45ac003a60 0x7f45c40c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:30:56.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.597+0000 7f45c37fe700 1 --2- 172.21.15.163:0/1929387169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45c40c5ec0 0x7f45c40c6330 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f45c40cf230 tx=0x7f45bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:56.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.597+0000 7f45aaffd700 1 -- 172.21.15.163:0/1929387169 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc00f980 con 0x7f45c40c5ec0 2023-10-09T06:30:56.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.597+0000 7f45aaffd700 1 -- 172.21.15.163:0/1929387169 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45bc00fae0 con 0x7f45c40c5ec0 2023-10-09T06:30:56.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.597+0000 7f45aaffd700 1 -- 172.21.15.163:0/1929387169 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc00fe80 con 0x7f45c40c5ec0 2023-10-09T06:30:56.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.597+0000 7f45c9689700 1 -- 172.21.15.163:0/1929387169 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45c413e650 con 0x7f45c40c5ec0 2023-10-09T06:30:56.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.597+0000 7f45c9689700 1 -- 172.21.15.163:0/1929387169 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c4158920 con 0x7f45c40c5ec0 2023-10-09T06:30:56.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.598+0000 7f45c9689700 1 -- 172.21.15.163:0/1929387169 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45c40b2070 con 0x7f45c40c5ec0 2023-10-09T06:30:56.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.598+0000 7f45aaffd700 1 -- 172.21.15.163:0/1929387169 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f45bc01a030 con 0x7f45c40c5ec0 2023-10-09T06:30:56.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.599+0000 7f45aaffd700 1 --2- 172.21.15.163:0/1929387169 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f45b0066fb0 0x7f45b0069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:56.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.599+0000 7f45aaffd700 1 -- 172.21.15.163:0/1929387169 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f45bc018070 con 0x7f45c40c5ec0 2023-10-09T06:30:56.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.599+0000 7f45c2ffd700 1 --2- 172.21.15.163:0/1929387169 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f45b0066fb0 0x7f45b0069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:56.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.600+0000 7f45c2ffd700 1 --2- 172.21.15.163:0/1929387169 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f45b0066fb0 0x7f45b0069460 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f45b400e200 tx=0x7f45b4019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:56.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.603+0000 7f45aaffd700 1 -- 172.21.15.163:0/1929387169 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f45bc051dd0 con 0x7f45c40c5ec0 2023-10-09T06:30:56.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.787+0000 7f45c9689700 1 -- 172.21.15.163:0/1929387169 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45c404c3c0 con 0x7f45c40c5ec0 2023-10-09T06:30:56.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.788+0000 7f45aaffd700 1 -- 172.21.15.163:0/1929387169 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f45bc022090 con 0x7f45c40c5ec0 2023-10-09T06:30:56.789 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:30:56.789 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":20,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:30:44.466923+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:30:41.369014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-09T06:30:42.103001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:30:56.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.790+0000 7f45c9689700 1 -- 172.21.15.163:0/1929387169 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f45b0066fb0 msgr2=0x7f45b0069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:56.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.790+0000 7f45c9689700 1 --2- 172.21.15.163:0/1929387169 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f45b0066fb0 0x7f45b0069460 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f45b400e200 tx=0x7f45b4019000 comp rx=0 tx=0).stop 2023-10-09T06:30:56.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.790+0000 7f45c9689700 1 -- 172.21.15.163:0/1929387169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45c40c5ec0 msgr2=0x7f45c40c6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:56.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.790+0000 7f45c9689700 1 --2- 172.21.15.163:0/1929387169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45c40c5ec0 0x7f45c40c6330 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f45c40cf230 tx=0x7f45bc0091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.790+0000 7f45c9689700 1 -- 172.21.15.163:0/1929387169 shutdown_connections 2023-10-09T06:30:56.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.790+0000 7f45c9689700 1 --2- 172.21.15.163:0/1929387169 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f45b0066fb0 0x7f45b0069460 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.790+0000 7f45c9689700 1 --2- 172.21.15.163:0/1929387169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45c40c5ec0 0x7f45c40c6330 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.790+0000 7f45c9689700 1 --2- 172.21.15.163:0/1929387169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45ac003a60 0x7f45c40c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.790+0000 7f45c9689700 1 --2- 172.21.15.163:0/1929387169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45c413d810 0x7f45c41583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.790+0000 7f45c9689700 1 -- 172.21.15.163:0/1929387169 >> 172.21.15.163:0/1929387169 conn(0x7f45c40ce600 msgr2=0x7f45c4159420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:56.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.790+0000 7f45c9689700 1 -- 172.21.15.163:0/1929387169 shutdown_connections 2023-10-09T06:30:56.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.790+0000 7f45c9689700 1 -- 172.21.15.163:0/1929387169 wait complete. 2023-10-09T06:30:56.804 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:30:56.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.917+0000 7f0fe73ac700 1 Processor -- start 2023-10-09T06:30:56.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.917+0000 7f0fe73ac700 1 -- start start 2023-10-09T06:30:56.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.918+0000 7f0fe73ac700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0fe0159890 0x7f0fe015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:56.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.918+0000 7f0fe73ac700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe0159c60 0x7f0fe015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:56.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.918+0000 7f0fe73ac700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fe015a160 con 0x7f0fe015db10 2023-10-09T06:30:56.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.918+0000 7f0fe73ac700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe00d13a0 con 0x7f0fe0159890 2023-10-09T06:30:56.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.918+0000 7f0fe73ac700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe015e140 con 0x7f0fe0159c60 2023-10-09T06:30:56.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.918+0000 7f0fe5949700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f0fe015db10 0x7f0fe0069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60996/0 (socket says 172.21.15.163:60996) 2023-10-09T06:30:56.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.918+0000 7f0fe5148700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0fe0159890 0x7f0fe015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:56.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.918+0000 7f0fe5949700 1 -- 172.21.15.163:0/1598334902 learned_addr learned my addr 172.21.15.163:0/1598334902 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:56.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.918+0000 7f0fe4947700 1 --2- 172.21.15.163:0/1598334902 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe0159c60 0x7f0fe015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:56.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.918+0000 7f0fd7fff700 1 -- 172.21.15.163:0/1598334902 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1838633507 0 0) 0x7f0fe015a160 con 0x7f0fe015db10 2023-10-09T06:30:56.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.918+0000 7f0fd7fff700 1 -- 172.21.15.163:0/1598334902 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fcc003570 con 0x7f0fe015db10 2023-10-09T06:30:56.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.918+0000 7f0fe5148700 1 -- 172.21.15.163:0/1598334902 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe0159c60 msgr2=0x7f0fe015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:56.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.918+0000 7f0fe5148700 1 --2- 172.21.15.163:0/1598334902 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe0159c60 0x7f0fe015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.918+0000 7f0fe5148700 1 -- 172.21.15.163:0/1598334902 >> v1:172.21.15.163:6789/0 conn(0x7f0fe015db10 legacy=0x7f0fe0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:56.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.918+0000 7f0fe5148700 1 -- 172.21.15.163:0/1598334902 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe015a3d0 con 0x7f0fe0159890 2023-10-09T06:30:56.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.918+0000 7f0fe5148700 1 --2- 172.21.15.163:0/1598334902 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fe0159890 0x7f0fe015abd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc0098e0 tx=0x7f0fdc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3353d0a1b9fdecc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:56.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.918+0000 7f0fd7fff700 1 -- 172.21.15.163:0/1598334902 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc00b640 con 0x7f0fe0159890 2023-10-09T06:30:56.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.918+0000 7f0fd7fff700 1 -- 172.21.15.163:0/1598334902 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fdc00b7a0 con 0x7f0fe0159890 2023-10-09T06:30:56.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.918+0000 7f0fd7fff700 1 -- 172.21.15.163:0/1598334902 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc00bb40 con 0x7f0fe0159890 2023-10-09T06:30:56.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.919+0000 7f0fe73ac700 1 -- 172.21.15.163:0/1598334902 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fe0159890 msgr2=0x7f0fe015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:56.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.919+0000 7f0fe73ac700 1 --2- 172.21.15.163:0/1598334902 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fe0159890 0x7f0fe015abd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc0098e0 tx=0x7f0fdc0091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.919+0000 7f0fe73ac700 1 -- 172.21.15.163:0/1598334902 shutdown_connections 2023-10-09T06:30:56.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.919+0000 7f0fe73ac700 1 --2- 172.21.15.163:0/1598334902 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fe0159890 0x7f0fe015abd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.919+0000 7f0fe73ac700 1 --2- 172.21.15.163:0/1598334902 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe0159c60 0x7f0fe015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.919+0000 7f0fe73ac700 1 -- 172.21.15.163:0/1598334902 >> 172.21.15.163:0/1598334902 conn(0x7f0fe0068e20 msgr2=0x7f0fe00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:56.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.919+0000 7f0fe73ac700 1 -- 172.21.15.163:0/1598334902 shutdown_connections 2023-10-09T06:30:56.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.919+0000 7f0fe73ac700 1 -- 172.21.15.163:0/1598334902 wait complete. 2023-10-09T06:30:56.925 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.920+0000 7f0fe73ac700 1 Processor -- start 2023-10-09T06:30:56.925 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.920+0000 7f0fe73ac700 1 -- start start 2023-10-09T06:30:56.925 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.921+0000 7f0fe73ac700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe0159890 0x7f0fe0152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:56.925 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.921+0000 7f0fe73ac700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fe0159c60 0x7f0fe00cda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:56.925 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.921+0000 7f0fe73ac700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fe015db10 0x7f0fe00b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:56.925 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.921+0000 7f0fe73ac700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe00ce320 con 0x7f0fe015db10 2023-10-09T06:30:56.925 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.921+0000 7f0fe73ac700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe00ce600 con 0x7f0fe0159c60 2023-10-09T06:30:56.925 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.921+0000 7f0fe73ac700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe00ce8e0 con 0x7f0fe0159890 2023-10-09T06:30:56.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.921+0000 7f0fe5949700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fe015db10 0x7f0fe00b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:56.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.921+0000 7f0fe5949700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fe015db10 0x7f0fe00b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44282/0 (socket says 172.21.15.163:44282) 2023-10-09T06:30:56.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.921+0000 7f0fe5949700 1 -- 172.21.15.163:0/2769434247 learned_addr learned my addr 172.21.15.163:0/2769434247 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:56.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.921+0000 7f0fe4947700 1 --2- 172.21.15.163:0/2769434247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fe0159c60 0x7f0fe00cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:56.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.921+0000 7f0fe5148700 1 --2- 172.21.15.163:0/2769434247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe0159890 0x7f0fe0152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:56.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.921+0000 7f0fe5949700 1 -- 172.21.15.163:0/2769434247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe0159890 msgr2=0x7f0fe0152210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:56.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.921+0000 7f0fe5949700 1 --2- 172.21.15.163:0/2769434247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe0159890 0x7f0fe0152210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.921+0000 7f0fe5949700 1 -- 172.21.15.163:0/2769434247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fe0159c60 msgr2=0x7f0fe00cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:56.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.921+0000 7f0fe5949700 1 --2- 172.21.15.163:0/2769434247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fe0159c60 0x7f0fe00cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:56.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.921+0000 7f0fe5949700 1 -- 172.21.15.163:0/2769434247 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fdc009420 con 0x7f0fe015db10 2023-10-09T06:30:56.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.921+0000 7f0fe5148700 1 --2- 172.21.15.163:0/2769434247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe0159890 0x7f0fe0152210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:30:56.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.922+0000 7f0fe5949700 1 --2- 172.21.15.163:0/2769434247 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fe015db10 0x7f0fe00b3590 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8001460 tx=0x7f0fd8001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:56.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.922+0000 7f0fd57fa700 1 -- 172.21.15.163:0/2769434247 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd800a950 con 0x7f0fe015db10 2023-10-09T06:30:56.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.922+0000 7f0fd57fa700 1 -- 172.21.15.163:0/2769434247 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd800aab0 con 0x7f0fe015db10 2023-10-09T06:30:56.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.922+0000 7f0fe73ac700 1 -- 172.21.15.163:0/2769434247 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe00c8a10 con 0x7f0fe015db10 2023-10-09T06:30:56.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.922+0000 7f0fd57fa700 1 -- 172.21.15.163:0/2769434247 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd800ae30 con 0x7f0fe015db10 2023-10-09T06:30:56.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.922+0000 7f0fe73ac700 1 -- 172.21.15.163:0/2769434247 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe00cdfc0 con 0x7f0fe015db10 2023-10-09T06:30:56.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.923+0000 7f0fe73ac700 1 -- 172.21.15.163:0/2769434247 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fe004c3c0 con 0x7f0fe015db10 2023-10-09T06:30:56.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.923+0000 7f0fd57fa700 1 -- 172.21.15.163:0/2769434247 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0fd8007300 con 0x7f0fe015db10 2023-10-09T06:30:56.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.923+0000 7f0fd57fa700 1 --2- 172.21.15.163:0/2769434247 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0fc4066f30 0x7f0fc40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:56.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.924+0000 7f0fd57fa700 1 -- 172.21.15.163:0/2769434247 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0fd8084900 con 0x7f0fe015db10 2023-10-09T06:30:56.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.924+0000 7f0fe5148700 1 --2- 172.21.15.163:0/2769434247 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0fc4066f30 0x7f0fc40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:56.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.924+0000 7f0fe5148700 1 --2- 172.21.15.163:0/2769434247 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0fc4066f30 0x7f0fc40693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc003080 tx=0x7f0fdc017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:56.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:56.928+0000 7f0fd57fa700 1 -- 172.21.15.163:0/2769434247 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0fd8051bf0 con 0x7f0fe015db10 2023-10-09T06:30:57.152 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.151+0000 7f0fe73ac700 1 -- 172.21.15.163:0/2769434247 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0fe006a5b0 con 0x7f0fe015db10 2023-10-09T06:30:57.153 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.152+0000 7f0fd57fa700 1 -- 172.21.15.163:0/2769434247 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4603 (secure 0 0 0) 0x7f0fd8055210 con 0x7f0fe015db10 2023-10-09T06:30:57.153 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:30:57.153 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3440080308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3440080308},{"type":"v1","addr":"172.21.15.181:6837","nonce":3440080308}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1990962358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1990962358},{"type":"v1","addr":"172.21.15.163:6835","nonce":1990962358}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2270076605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2270076605},{"type":"v1","addr":"172.21.15.163:6834","nonce":2270076605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:30:43.133124+0000","modified":"2023-10-09T06:30:46.760523+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/1149296546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1149296546},{"type":"v1","addr":"172.21.15.181:6836","nonce":1149296546}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-09T06:30:57.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.154+0000 7f0fe73ac700 1 -- 172.21.15.163:0/2769434247 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0fc4066f30 msgr2=0x7f0fc40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:57.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.154+0000 7f0fe73ac700 1 --2- 172.21.15.163:0/2769434247 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0fc4066f30 0x7f0fc40693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc003080 tx=0x7f0fdc017000 comp rx=0 tx=0).stop 2023-10-09T06:30:57.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.154+0000 7f0fe73ac700 1 -- 172.21.15.163:0/2769434247 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fe015db10 msgr2=0x7f0fe00b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:57.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.154+0000 7f0fe73ac700 1 --2- 172.21.15.163:0/2769434247 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fe015db10 0x7f0fe00b3590 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8001460 tx=0x7f0fd8001a40 comp rx=0 tx=0).stop 2023-10-09T06:30:57.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.154+0000 7f0fe73ac700 1 -- 172.21.15.163:0/2769434247 shutdown_connections 2023-10-09T06:30:57.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.154+0000 7f0fe73ac700 1 --2- 172.21.15.163:0/2769434247 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0fc4066f30 0x7f0fc40693e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.154+0000 7f0fe73ac700 1 --2- 172.21.15.163:0/2769434247 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0fe015db10 0x7f0fe00b3590 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.154+0000 7f0fe73ac700 1 --2- 172.21.15.163:0/2769434247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe0159890 0x7f0fe0152210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.154+0000 7f0fe73ac700 1 --2- 172.21.15.163:0/2769434247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fe0159c60 0x7f0fe00cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.154+0000 7f0fe73ac700 1 -- 172.21.15.163:0/2769434247 >> 172.21.15.163:0/2769434247 conn(0x7f0fe0068e20 msgr2=0x7f0fe015bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:57.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.154+0000 7f0fe73ac700 1 -- 172.21.15.163:0/2769434247 shutdown_connections 2023-10-09T06:30:57.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.154+0000 7f0fe73ac700 1 -- 172.21.15.163:0/2769434247 wait complete. 2023-10-09T06:30:57.157 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 8 2023-10-09T06:30:57.570 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-09T06:30:43.133124+0000', 'modified': '2023-10-09T06:30:46.760523+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6836/1149296546', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 1149296546}, {'type': 'v1', 'addr': '172.21.15.181:6836', 'nonce': 1149296546}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-09T06:30:57.570 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-09T06:30:57.570 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-09T06:30:57.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.687+0000 7f8153fff700 1 Processor -- start 2023-10-09T06:30:57.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.687+0000 7f8153fff700 1 -- start start 2023-10-09T06:30:57.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.688+0000 7f8153fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81540c8d10 0x7f815406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:57.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.688+0000 7f8153fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8154156930 0x7f81540adee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:57.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.688+0000 7f8153fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f815406f6f0 con 0x7f815406e570 2023-10-09T06:30:57.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.688+0000 7f8153fff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f815406eae0 con 0x7f8154156930 2023-10-09T06:30:57.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.688+0000 7f8153fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f815406edc0 con 0x7f81540c8d10 2023-10-09T06:30:57.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.688+0000 7f81527fc700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f815406e570 0x7f8154156220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60998/0 (socket says 172.21.15.163:60998) 2023-10-09T06:30:57.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.688+0000 7f81527fc700 1 -- 172.21.15.163:0/3168758062 learned_addr learned my addr 172.21.15.163:0/3168758062 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:57.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.688+0000 7f8152ffd700 1 --2- 172.21.15.163:0/3168758062 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81540c8d10 0x7f815406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:57.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.688+0000 7f81537fe700 1 --2- 172.21.15.163:0/3168758062 >> v2:172.21.15.181:3300/0 conn(0x7f8154156930 0x7f81540adee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:57.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.688+0000 7f8151ffb700 1 -- 172.21.15.163:0/3168758062 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1464528182 0 0) 0x7f815406f6f0 con 0x7f815406e570 2023-10-09T06:30:57.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.688+0000 7f8151ffb700 1 -- 172.21.15.163:0/3168758062 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8140003570 con 0x7f815406e570 2023-10-09T06:30:57.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.689+0000 7f8151ffb700 1 -- 172.21.15.163:0/3168758062 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3581174460 0 0) 0x7f8140003570 con 0x7f815406e570 2023-10-09T06:30:57.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.689+0000 7f81537fe700 1 -- 172.21.15.163:0/3168758062 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81540c8d10 msgr2=0x7f815406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:57.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.689+0000 7f81537fe700 1 --2- 172.21.15.163:0/3168758062 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81540c8d10 0x7f815406c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.689+0000 7f81537fe700 1 -- 172.21.15.163:0/3168758062 >> v1:172.21.15.163:6789/0 conn(0x7f815406e570 legacy=0x7f8154156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:57.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.689+0000 7f81537fe700 1 -- 172.21.15.163:0/3168758062 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f815406f0a0 con 0x7f8154156930 2023-10-09T06:30:57.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.689+0000 7f8152ffd700 1 --2- 172.21.15.163:0/3168758062 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81540c8d10 0x7f815406c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:30:57.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.689+0000 7f81537fe700 1 --2- 172.21.15.163:0/3168758062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8154156930 0x7f81540adee0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f814c0098e0 tx=0x7f814c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47cb6fe6db129c5e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:57.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.689+0000 7f8151ffb700 1 -- 172.21.15.163:0/3168758062 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c0015f0 con 0x7f8154156930 2023-10-09T06:30:57.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.689+0000 7f8151ffb700 1 -- 172.21.15.163:0/3168758062 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814c011420 con 0x7f8154156930 2023-10-09T06:30:57.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.689+0000 7f8151ffb700 1 -- 172.21.15.163:0/3168758062 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c0117c0 con 0x7f8154156930 2023-10-09T06:30:57.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.689+0000 7f8153fff700 1 -- 172.21.15.163:0/3168758062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8154156930 msgr2=0x7f81540adee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:57.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.689+0000 7f8153fff700 1 --2- 172.21.15.163:0/3168758062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8154156930 0x7f81540adee0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f814c0098e0 tx=0x7f814c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.690+0000 7f8153fff700 1 -- 172.21.15.163:0/3168758062 shutdown_connections 2023-10-09T06:30:57.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.690+0000 7f8153fff700 1 --2- 172.21.15.163:0/3168758062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8154156930 0x7f81540adee0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.690+0000 7f8153fff700 1 --2- 172.21.15.163:0/3168758062 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81540c8d10 0x7f815406c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.690+0000 7f8153fff700 1 -- 172.21.15.163:0/3168758062 >> 172.21.15.163:0/3168758062 conn(0x7f815406b450 msgr2=0x7f81540cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:57.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.690+0000 7f8153fff700 1 -- 172.21.15.163:0/3168758062 shutdown_connections 2023-10-09T06:30:57.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.690+0000 7f8153fff700 1 -- 172.21.15.163:0/3168758062 wait complete. 2023-10-09T06:30:57.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.691+0000 7f8153fff700 1 Processor -- start 2023-10-09T06:30:57.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.691+0000 7f8153fff700 1 -- start start 2023-10-09T06:30:57.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.691+0000 7f8153fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f815406e570 0x7f8154154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:57.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.691+0000 7f8153fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81540c8d10 0x7f81540c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:57.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.691+0000 7f8153fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8154156930 0x7f815415b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:57.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.691+0000 7f8153fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815416de70 con 0x7f8154156930 2023-10-09T06:30:57.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.691+0000 7f8153fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815416cf80 con 0x7f81540c8d10 2023-10-09T06:30:57.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.691+0000 7f8153fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f815416d260 con 0x7f815406e570 2023-10-09T06:30:57.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.692+0000 7f81537fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8154156930 0x7f815415b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:57.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.692+0000 7f81537fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8154156930 0x7f815415b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44284/0 (socket says 172.21.15.163:44284) 2023-10-09T06:30:57.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.692+0000 7f81537fe700 1 -- 172.21.15.163:0/705137455 learned_addr learned my addr 172.21.15.163:0/705137455 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:57.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.692+0000 7f8152ffd700 1 --2- 172.21.15.163:0/705137455 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f815406e570 0x7f8154154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:57.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.692+0000 7f81527fc700 1 --2- 172.21.15.163:0/705137455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81540c8d10 0x7f81540c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:57.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.692+0000 7f81537fe700 1 -- 172.21.15.163:0/705137455 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f815406e570 msgr2=0x7f8154154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:57.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.692+0000 7f81537fe700 1 --2- 172.21.15.163:0/705137455 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f815406e570 0x7f8154154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.692+0000 7f81537fe700 1 -- 172.21.15.163:0/705137455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81540c8d10 msgr2=0x7f81540c4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:57.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.692+0000 7f81537fe700 1 --2- 172.21.15.163:0/705137455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81540c8d10 0x7f81540c4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.692+0000 7f81537fe700 1 -- 172.21.15.163:0/705137455 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8144009a20 con 0x7f8154156930 2023-10-09T06:30:57.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.692+0000 7f81537fe700 1 --2- 172.21.15.163:0/705137455 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8154156930 0x7f815415b7e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f814c0030c0 tx=0x7f814c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:57.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.692+0000 7f814affd700 1 -- 172.21.15.163:0/705137455 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c01cd10 con 0x7f8154156930 2023-10-09T06:30:57.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.692+0000 7f814affd700 1 -- 172.21.15.163:0/705137455 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814c01ce70 con 0x7f8154156930 2023-10-09T06:30:57.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.692+0000 7f8153fff700 1 -- 172.21.15.163:0/705137455 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f814c009420 con 0x7f8154156930 2023-10-09T06:30:57.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.692+0000 7f814affd700 1 -- 172.21.15.163:0/705137455 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c01a680 con 0x7f8154156930 2023-10-09T06:30:57.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.692+0000 7f8153fff700 1 -- 172.21.15.163:0/705137455 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f815416d820 con 0x7f8154156930 2023-10-09T06:30:57.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.694+0000 7f814affd700 1 -- 172.21.15.163:0/705137455 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f814c01aa70 con 0x7f8154156930 2023-10-09T06:30:57.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.694+0000 7f814affd700 1 --2- 172.21.15.163:0/705137455 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8134066f30 0x7f81340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:57.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.694+0000 7f814affd700 1 -- 172.21.15.163:0/705137455 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f814c085f40 con 0x7f8154156930 2023-10-09T06:30:57.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.694+0000 7f8152ffd700 1 --2- 172.21.15.163:0/705137455 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8134066f30 0x7f81340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:57.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.695+0000 7f8152ffd700 1 --2- 172.21.15.163:0/705137455 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8134066f30 0x7f81340693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8144009fd0 tx=0x7f814400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:57.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.696+0000 7f816285cb80 1 Processor -- start 2023-10-09T06:30:57.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.696+0000 7f816285cb80 1 -- start start 2023-10-09T06:30:57.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.697+0000 7f816285cb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a62e7f2480 0x55a62e7d17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:57.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.697+0000 7f816285cb80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55a62e7a55f0 0x55a62e7f2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:57.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.697+0000 7f816285cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a62e7a7b80 0x55a62e7a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:57.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.697+0000 7f816285cb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62e6fea70 con 0x55a62e7f2480 2023-10-09T06:30:57.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.697+0000 7f816285cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62e6fc270 con 0x55a62e7a7b80 2023-10-09T06:30:57.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.697+0000 7f816285cb80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a62e6f8060 con 0x55a62e7a55f0 2023-10-09T06:30:57.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.697+0000 7f8152ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a62e7f2480 0x55a62e7d17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:57.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.697+0000 7f8152ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a62e7f2480 0x55a62e7d17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44298/0 (socket says 172.21.15.163:44298) 2023-10-09T06:30:57.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.697+0000 7f8152ffd700 1 -- 172.21.15.163:0/1342934537 learned_addr learned my addr 172.21.15.163:0/1342934537 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:57.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.697+0000 7f81527fc700 1 --2- 172.21.15.163:0/1342934537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55a62e7a55f0 0x55a62e7f2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:57.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.697+0000 7f81537fe700 1 --2- 172.21.15.163:0/1342934537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a62e7a7b80 0x55a62e7a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:57.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.697+0000 7f8152ffd700 1 -- 172.21.15.163:0/1342934537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55a62e7a55f0 msgr2=0x55a62e7f2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:57.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.697+0000 7f8152ffd700 1 --2- 172.21.15.163:0/1342934537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55a62e7a55f0 0x55a62e7f2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.697+0000 7f8152ffd700 1 -- 172.21.15.163:0/1342934537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a62e7a7b80 msgr2=0x55a62e7a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:57.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.697+0000 7f8152ffd700 1 --2- 172.21.15.163:0/1342934537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a62e7a7b80 0x55a62e7a7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.697+0000 7f8152ffd700 1 -- 172.21.15.163:0/1342934537 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a62e592270 con 0x55a62e7f2480 2023-10-09T06:30:57.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.697+0000 7f81537fe700 1 --2- 172.21.15.163:0/1342934537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a62e7a7b80 0x55a62e7a7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:30:57.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.698+0000 7f8152ffd700 1 --2- 172.21.15.163:0/1342934537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a62e7f2480 0x55a62e7d17d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8144015930 tx=0x7f8144016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f48908614eea1da7 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:57.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.698+0000 7f8133fff700 1 -- 172.21.15.163:0/1342934537 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814400e040 con 0x55a62e7f2480 2023-10-09T06:30:57.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.698+0000 7f8133fff700 1 -- 172.21.15.163:0/1342934537 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814401b070 con 0x55a62e7f2480 2023-10-09T06:30:57.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.698+0000 7f8133fff700 1 -- 172.21.15.163:0/1342934537 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81440179e0 con 0x55a62e7f2480 2023-10-09T06:30:57.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.698+0000 7f816285cb80 1 -- 172.21.15.163:0/1342934537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a62e7f2480 msgr2=0x55a62e7d17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:57.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.698+0000 7f816285cb80 1 --2- 172.21.15.163:0/1342934537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a62e7f2480 0x55a62e7d17d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8144015930 tx=0x7f8144016000 comp rx=0 tx=0).stop 2023-10-09T06:30:57.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.698+0000 7f816285cb80 1 -- 172.21.15.163:0/1342934537 shutdown_connections 2023-10-09T06:30:57.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.698+0000 7f816285cb80 1 --2- 172.21.15.163:0/1342934537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a62e7f2480 0x55a62e7d17d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.698+0000 7f816285cb80 1 --2- 172.21.15.163:0/1342934537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a62e7a7b80 0x55a62e7a7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.698+0000 7f816285cb80 1 --2- 172.21.15.163:0/1342934537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55a62e7a55f0 0x55a62e7f2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.698+0000 7f816285cb80 1 -- 172.21.15.163:0/1342934537 >> 172.21.15.163:0/1342934537 conn(0x55a62e7a90c0 msgr2=0x55a62e7a98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:57.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.698+0000 7f816285cb80 1 -- 172.21.15.163:0/1342934537 shutdown_connections 2023-10-09T06:30:57.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.698+0000 7f816285cb80 1 -- 172.21.15.163:0/1342934537 wait complete. 2023-10-09T06:30:57.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.699+0000 7f816285cb80 1 -- start start 2023-10-09T06:30:57.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.699+0000 7f816285cb80 1 Processor -- start 2023-10-09T06:30:57.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.700+0000 7f816285cb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a62e7a55f0 0x55a62e7cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:57.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.700+0000 7f816285cb80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55a62e7a7b80 0x55a62e7f1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:57.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.700+0000 7f816285cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a62e7f2480 0x55a62e7f1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:57.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.700+0000 7f816285cb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62e6a3890 con 0x55a62e7a55f0 2023-10-09T06:30:57.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.700+0000 7f816285cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62e6beea0 con 0x55a62e7f2480 2023-10-09T06:30:57.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.700+0000 7f816285cb80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a62e57ae30 con 0x55a62e7a7b80 2023-10-09T06:30:57.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.700+0000 7f816285cb80 10 client.0 _ll_register_callbacks cb 0x55a62e76b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-09T06:30:57.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.700+0000 7f8152ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a62e7a55f0 0x55a62e7cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:57.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.700+0000 7f8152ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a62e7a55f0 0x55a62e7cbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44300/0 (socket says 172.21.15.163:44300) 2023-10-09T06:30:57.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.700+0000 7f8152ffd700 1 -- 172.21.15.163:0/4258655147 learned_addr learned my addr 172.21.15.163:0/4258655147 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:57.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.700+0000 7f81527fc700 1 --2- 172.21.15.163:0/4258655147 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55a62e7a7b80 0x55a62e7f1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:57.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.700+0000 7f8152ffd700 1 -- 172.21.15.163:0/4258655147 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55a62e7a7b80 msgr2=0x55a62e7f1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:57.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.700+0000 7f8152ffd700 1 --2- 172.21.15.163:0/4258655147 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55a62e7a7b80 0x55a62e7f1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.700+0000 7f81537fe700 1 --2- 172.21.15.163:0/4258655147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a62e7f2480 0x55a62e7f1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:57.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.700+0000 7f8152ffd700 1 -- 172.21.15.163:0/4258655147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a62e7f2480 msgr2=0x55a62e7f1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:57.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.700+0000 7f8152ffd700 1 --2- 172.21.15.163:0/4258655147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a62e7f2480 0x55a62e7f1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.700+0000 7f8152ffd700 1 -- 172.21.15.163:0/4258655147 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81440155e0 con 0x55a62e7a55f0 2023-10-09T06:30:57.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.700+0000 7f8152ffd700 1 --2- 172.21.15.163:0/4258655147 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a62e7a55f0 0x55a62e7cbf30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8144009ef0 tx=0x7f814401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82b58adcaeb2457 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:57.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.700+0000 7f81317fa700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-10-09T06:30:57.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.701+0000 7f81317fa700 1 -- 172.21.15.163:0/4258655147 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814400e040 con 0x55a62e7a55f0 2023-10-09T06:30:57.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.701+0000 7f81317fa700 1 -- 172.21.15.163:0/4258655147 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8144032070 con 0x55a62e7a55f0 2023-10-09T06:30:57.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.701+0000 7f816285cb80 1 -- 172.21.15.163:0/4258655147 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a62e79ccd0 con 0x55a62e7a55f0 2023-10-09T06:30:57.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.701+0000 7f81317fa700 1 -- 172.21.15.163:0/4258655147 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8144022d10 con 0x55a62e7a55f0 2023-10-09T06:30:57.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.701+0000 7f816285cb80 10 client.4650 fetch_fsmap learned FSMap version 8 2023-10-09T06:30:57.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.701+0000 7f816285cb80 1 -- 172.21.15.163:0/4258655147 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55a62e534fe0 con 0x55a62e7a55f0 2023-10-09T06:30:57.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.701+0000 7f81317fa700 1 -- 172.21.15.163:0/4258655147 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a62e79ccd0 con 0x55a62e7a55f0 2023-10-09T06:30:57.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.701+0000 7f81317fa700 1 -- 172.21.15.163:0/4258655147 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f8144036430 con 0x55a62e7a55f0 2023-10-09T06:30:57.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.701+0000 7f816285cb80 10 client.4650 fetch_fsmap finished waiting for FSMap version 8 2023-10-09T06:30:57.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.701+0000 7f816285cb80 10 client.4650 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-09T06:30:57.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.701+0000 7f816285cb80 20 client.4650 populate_metadata read hostname 'smithi163' 2023-10-09T06:30:57.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.701+0000 7f816285cb80 1 --2- 172.21.15.163:0/4258655147 >> [v2:172.21.15.181:6834/1149296546,v1:172.21.15.181:6836/1149296546] conn(0x55a62e7d6e90 0x55a62e7d7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:57.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.701+0000 7f816285cb80 4 client.4650 mds_command: new command op to 4285 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-09T06:30:57.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.701+0000 7f816285cb80 1 -- 172.21.15.163:0/4258655147 --> [v2:172.21.15.181:6834/1149296546,v1:172.21.15.181:6836/1149296546] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a62e779aa0 con 0x55a62e7d6e90 2023-10-09T06:30:57.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.701+0000 7f81527fc700 1 --2- 172.21.15.163:0/4258655147 >> [v2:172.21.15.181:6834/1149296546,v1:172.21.15.181:6836/1149296546] conn(0x55a62e7d6e90 0x55a62e7d7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:57.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.702+0000 7f81317fa700 10 client.4650 ms_handle_connect on v2:172.21.15.181:6834/1149296546 2023-10-09T06:30:57.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.702+0000 7f81527fc700 1 --2- 172.21.15.163:0/4258655147 >> [v2:172.21.15.181:6834/1149296546,v1:172.21.15.181:6836/1149296546] conn(0x55a62e7d6e90 0x55a62e7d7260 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed549c02664e25cd server_cookie=d47955a101e78df9 in_seq=0 out_seq=0 2023-10-09T06:30:57.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.702+0000 7f81317fa700 1 -- 172.21.15.163:0/4258655147 <== mds.0 v2:172.21.15.181:6834/1149296546 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55a62e779aa0 con 0x55a62e7d6e90 2023-10-09T06:30:57.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.702+0000 7f81317fa700 10 client.4650 handle_command_reply: tid=0 2023-10-09T06:30:57.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.702+0000 7f816285cb80 1 client.4650 shutdown 2023-10-09T06:30:57.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.702+0000 7f816285cb80 1 -- 172.21.15.163:0/4258655147 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a62e7a55f0 msgr2=0x55a62e7cbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:57.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.702+0000 7f816285cb80 1 --2- 172.21.15.163:0/4258655147 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a62e7a55f0 0x55a62e7cbf30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8144009ef0 tx=0x7f814401b000 comp rx=0 tx=0).stop 2023-10-09T06:30:57.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.702+0000 7f816285cb80 1 -- 172.21.15.163:0/4258655147 shutdown_connections 2023-10-09T06:30:57.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.702+0000 7f816285cb80 1 --2- 172.21.15.163:0/4258655147 >> [v2:172.21.15.181:6834/1149296546,v1:172.21.15.181:6836/1149296546] conn(0x55a62e7d6e90 0x55a62e7d7260 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.702+0000 7f816285cb80 1 --2- 172.21.15.163:0/4258655147 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a62e7a55f0 0x55a62e7cbf30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.702+0000 7f81317fa700 0 client.4650 ms_handle_reset on v2:172.21.15.181:6834/1149296546 2023-10-09T06:30:57.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.702+0000 7f816285cb80 1 --2- 172.21.15.163:0/4258655147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a62e7f2480 0x55a62e7f1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.702+0000 7f816285cb80 1 --2- 172.21.15.163:0/4258655147 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55a62e7a7b80 0x55a62e7f1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.702+0000 7f816285cb80 1 -- 172.21.15.163:0/4258655147 >> 172.21.15.163:0/4258655147 conn(0x55a62e7a90c0 msgr2=0x55a62e7ab530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:57.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.702+0000 7f816285cb80 1 -- 172.21.15.163:0/4258655147 shutdown_connections 2023-10-09T06:30:57.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.702+0000 7f816285cb80 1 -- 172.21.15.163:0/4258655147 wait complete. 2023-10-09T06:30:57.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.703+0000 7f816285cb80 20 client.4650 trim_cache size 0 max 16384 2023-10-09T06:30:57.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.719+0000 7f816285cb80 1 Processor -- start 2023-10-09T06:30:57.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.719+0000 7f816285cb80 1 -- start start 2023-10-09T06:30:57.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.720+0000 7f816285cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a62e7a55f0 0x55a62e7a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:57.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.720+0000 7f816285cb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a62e7a7b80 0x55a62e7cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:57.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.720+0000 7f816285cb80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55a62e7f2480 0x55a62e7cc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:57.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.720+0000 7f816285cb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62e7aa6c0 con 0x55a62e7a7b80 2023-10-09T06:30:57.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.720+0000 7f816285cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62e7a6cc0 con 0x55a62e7a55f0 2023-10-09T06:30:57.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.720+0000 7f816285cb80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a62e7f3240 con 0x55a62e7f2480 2023-10-09T06:30:57.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.720+0000 7f81527fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a62e7a7b80 0x55a62e7cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:57.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.720+0000 7f81527fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a62e7a7b80 0x55a62e7cbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44304/0 (socket says 172.21.15.163:44304) 2023-10-09T06:30:57.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.720+0000 7f81527fc700 1 -- 172.21.15.163:0/428757194 learned_addr learned my addr 172.21.15.163:0/428757194 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:57.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.720+0000 7f81537fe700 1 --2- 172.21.15.163:0/428757194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55a62e7f2480 0x55a62e7cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:57.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.720+0000 7f8152ffd700 1 --2- 172.21.15.163:0/428757194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a62e7a55f0 0x55a62e7a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:57.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.720+0000 7f81527fc700 1 -- 172.21.15.163:0/428757194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55a62e7f2480 msgr2=0x55a62e7cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:57.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.720+0000 7f81527fc700 1 --2- 172.21.15.163:0/428757194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55a62e7f2480 0x55a62e7cc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.720+0000 7f81527fc700 1 -- 172.21.15.163:0/428757194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a62e7a55f0 msgr2=0x55a62e7a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:57.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.720+0000 7f81527fc700 1 --2- 172.21.15.163:0/428757194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a62e7a55f0 0x55a62e7a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.720+0000 7f81527fc700 1 -- 172.21.15.163:0/428757194 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a62e7d2c90 con 0x55a62e7a7b80 2023-10-09T06:30:57.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.720+0000 7f8152ffd700 1 --2- 172.21.15.163:0/428757194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a62e7a55f0 0x55a62e7a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:30:57.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.720+0000 7f81527fc700 1 --2- 172.21.15.163:0/428757194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a62e7a7b80 0x55a62e7cbf30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8154155e40 tx=0x7f813c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd7d3d6e5cc0c87 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:57.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.721+0000 7f8148ff9700 1 -- 172.21.15.163:0/428757194 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c005dd0 con 0x55a62e7a7b80 2023-10-09T06:30:57.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.721+0000 7f8148ff9700 1 -- 172.21.15.163:0/428757194 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813c006480 con 0x55a62e7a7b80 2023-10-09T06:30:57.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.721+0000 7f8148ff9700 1 -- 172.21.15.163:0/428757194 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c01a630 con 0x55a62e7a7b80 2023-10-09T06:30:57.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.721+0000 7f816285cb80 1 -- 172.21.15.163:0/428757194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a62e7a7b80 msgr2=0x55a62e7cbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:57.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.721+0000 7f816285cb80 1 --2- 172.21.15.163:0/428757194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a62e7a7b80 0x55a62e7cbf30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8154155e40 tx=0x7f813c009040 comp rx=0 tx=0).stop 2023-10-09T06:30:57.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.721+0000 7f816285cb80 1 -- 172.21.15.163:0/428757194 shutdown_connections 2023-10-09T06:30:57.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.721+0000 7f816285cb80 1 --2- 172.21.15.163:0/428757194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55a62e7f2480 0x55a62e7cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.721+0000 7f816285cb80 1 --2- 172.21.15.163:0/428757194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a62e7a55f0 0x55a62e7a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.721+0000 7f816285cb80 1 --2- 172.21.15.163:0/428757194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a62e7a7b80 0x55a62e7cbf30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.721+0000 7f816285cb80 1 -- 172.21.15.163:0/428757194 >> 172.21.15.163:0/428757194 conn(0x55a62e7a90c0 msgr2=0x55a62e7d4260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:57.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.721+0000 7f816285cb80 1 -- 172.21.15.163:0/428757194 shutdown_connections 2023-10-09T06:30:57.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.721+0000 7f816285cb80 1 -- 172.21.15.163:0/428757194 wait complete. 2023-10-09T06:30:57.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.722+0000 7f816285cb80 1 -- start start 2023-10-09T06:30:57.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.722+0000 7f816285cb80 1 Processor -- start 2023-10-09T06:30:57.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.723+0000 7f816285cb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a62e7a55f0 0x55a62e8031f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:57.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.723+0000 7f816285cb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a62e7a7b80 0x55a62e804b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:57.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.723+0000 7f816285cb80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55a62e7f2480 0x55a62e805080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:57.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.723+0000 7f816285cb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62e7a53b0 con 0x55a62e7a7b80 2023-10-09T06:30:57.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.723+0000 7f816285cb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62e801750 con 0x55a62e7a55f0 2023-10-09T06:30:57.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.723+0000 7f816285cb80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a62e8029f0 con 0x55a62e7f2480 2023-10-09T06:30:57.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.723+0000 7f816285cb80 10 client.0 _ll_register_callbacks cb 0x55a62e76b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-09T06:30:57.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.723+0000 7f81527fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a62e7a7b80 0x55a62e804b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:57.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.723+0000 7f81527fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a62e7a7b80 0x55a62e804b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44310/0 (socket says 172.21.15.163:44310) 2023-10-09T06:30:57.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.723+0000 7f81527fc700 1 -- 172.21.15.163:0/1227007900 learned_addr learned my addr 172.21.15.163:0/1227007900 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:57.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.723+0000 7f81537fe700 1 --2- 172.21.15.163:0/1227007900 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55a62e7f2480 0x55a62e805080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:57.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.723+0000 7f8152ffd700 1 --2- 172.21.15.163:0/1227007900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a62e7a55f0 0x55a62e8031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:57.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.723+0000 7f81527fc700 1 -- 172.21.15.163:0/1227007900 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55a62e7f2480 msgr2=0x55a62e805080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:57.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.723+0000 7f81527fc700 1 --2- 172.21.15.163:0/1227007900 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55a62e7f2480 0x55a62e805080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.723+0000 7f81527fc700 1 -- 172.21.15.163:0/1227007900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a62e7a55f0 msgr2=0x55a62e8031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:57.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.723+0000 7f81527fc700 1 --2- 172.21.15.163:0/1227007900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a62e7a55f0 0x55a62e8031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.723+0000 7f81527fc700 1 -- 172.21.15.163:0/1227007900 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f813c0057f0 con 0x55a62e7a7b80 2023-10-09T06:30:57.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.723+0000 7f81527fc700 1 --2- 172.21.15.163:0/1227007900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a62e7a7b80 0x55a62e804b40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f815415b680 tx=0x7f813c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81bc7e48f51ff85 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:57.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.723+0000 7f81317fa700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-10-09T06:30:57.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.723+0000 7f81317fa700 1 -- 172.21.15.163:0/1227007900 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c02a070 con 0x55a62e7a7b80 2023-10-09T06:30:57.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.723+0000 7f81317fa700 1 -- 172.21.15.163:0/1227007900 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f813c0065f0 con 0x55a62e7a7b80 2023-10-09T06:30:57.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.723+0000 7f816285cb80 1 -- 172.21.15.163:0/1227007900 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f814401b070 con 0x55a62e7a7b80 2023-10-09T06:30:57.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.723+0000 7f81317fa700 1 -- 172.21.15.163:0/1227007900 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f813c0067b0 con 0x55a62e7a7b80 2023-10-09T06:30:57.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.723+0000 7f816285cb80 10 client.4656 fetch_fsmap learned FSMap version 8 2023-10-09T06:30:57.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.723+0000 7f816285cb80 1 -- 172.21.15.163:0/1227007900 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55a62e6c6780 con 0x55a62e7a7b80 2023-10-09T06:30:57.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.724+0000 7f81317fa700 1 -- 172.21.15.163:0/1227007900 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814401b070 con 0x55a62e7a7b80 2023-10-09T06:30:57.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.724+0000 7f81317fa700 1 -- 172.21.15.163:0/1227007900 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f813c01ac70 con 0x55a62e7a7b80 2023-10-09T06:30:57.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.724+0000 7f816285cb80 10 client.4656 fetch_fsmap finished waiting for FSMap version 8 2023-10-09T06:30:57.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.724+0000 7f816285cb80 10 client.4656 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-09T06:30:57.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.724+0000 7f816285cb80 20 client.4656 populate_metadata read hostname 'smithi163' 2023-10-09T06:30:57.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.724+0000 7f816285cb80 1 --2- 172.21.15.163:0/1227007900 >> [v2:172.21.15.181:6834/1149296546,v1:172.21.15.181:6836/1149296546] conn(0x55a62e79e220 0x55a62e79e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:57.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.724+0000 7f816285cb80 4 client.4656 mds_command: new command op to 4285 tid=0[{"prefix": "status"}] 2023-10-09T06:30:57.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.724+0000 7f816285cb80 1 -- 172.21.15.163:0/1227007900 --> [v2:172.21.15.181:6834/1149296546,v1:172.21.15.181:6836/1149296546] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a62e69ae00 con 0x55a62e79e220 2023-10-09T06:30:57.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.724+0000 7f8152ffd700 1 --2- 172.21.15.163:0/1227007900 >> [v2:172.21.15.181:6834/1149296546,v1:172.21.15.181:6836/1149296546] conn(0x55a62e79e220 0x55a62e79e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:57.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.724+0000 7f81317fa700 10 client.4656 ms_handle_connect on v2:172.21.15.181:6834/1149296546 2023-10-09T06:30:57.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.724+0000 7f8152ffd700 1 --2- 172.21.15.163:0/1227007900 >> [v2:172.21.15.181:6834/1149296546,v1:172.21.15.181:6836/1149296546] conn(0x55a62e79e220 0x55a62e79e5f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b11c7de4c6cba5d server_cookie=7a9b0653c50e964a in_seq=0 out_seq=0 2023-10-09T06:30:57.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.724+0000 7f81317fa700 1 -- 172.21.15.163:0/1227007900 <== mds.0 v2:172.21.15.181:6834/1149296546 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x55a62e69ae00 con 0x55a62e79e220 2023-10-09T06:30:57.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.724+0000 7f81317fa700 10 client.4656 handle_command_reply: tid=0 2023-10-09T06:30:57.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.724+0000 7f816285cb80 1 client.4656 shutdown 2023-10-09T06:30:57.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.724+0000 7f816285cb80 1 -- 172.21.15.163:0/1227007900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a62e7a7b80 msgr2=0x55a62e804b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:57.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.724+0000 7f816285cb80 1 --2- 172.21.15.163:0/1227007900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a62e7a7b80 0x55a62e804b40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f815415b680 tx=0x7f813c01c000 comp rx=0 tx=0).stop 2023-10-09T06:30:57.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.725+0000 7f816285cb80 1 -- 172.21.15.163:0/1227007900 shutdown_connections 2023-10-09T06:30:57.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.725+0000 7f816285cb80 1 --2- 172.21.15.163:0/1227007900 >> [v2:172.21.15.181:6834/1149296546,v1:172.21.15.181:6836/1149296546] conn(0x55a62e79e220 0x55a62e79e5f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.725+0000 7f816285cb80 1 --2- 172.21.15.163:0/1227007900 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55a62e7f2480 0x55a62e805080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.725+0000 7f81317fa700 0 client.4656 ms_handle_reset on v2:172.21.15.181:6834/1149296546 2023-10-09T06:30:57.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.725+0000 7f816285cb80 1 --2- 172.21.15.163:0/1227007900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55a62e7a55f0 0x55a62e8031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.725+0000 7f816285cb80 1 --2- 172.21.15.163:0/1227007900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a62e7a7b80 0x55a62e804b40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.725+0000 7f816285cb80 1 -- 172.21.15.163:0/1227007900 >> 172.21.15.163:0/1227007900 conn(0x55a62e7a90c0 msgr2=0x55a62e7d7490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:57.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.725+0000 7f816285cb80 1 -- 172.21.15.163:0/1227007900 shutdown_connections 2023-10-09T06:30:57.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.725+0000 7f816285cb80 1 -- 172.21.15.163:0/1227007900 wait complete. 2023-10-09T06:30:57.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.725+0000 7f816285cb80 20 client.4656 trim_cache size 0 max 16384 2023-10-09T06:30:57.742 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:30:57.742 INFO:teuthology.orchestra.run.smithi163.stdout: "cluster_fsid": "2520b8f7-539f-4bae-9f17-1fce66f66ddd", 2023-10-09T06:30:57.742 INFO:teuthology.orchestra.run.smithi163.stdout: "whoami": 0, 2023-10-09T06:30:57.742 INFO:teuthology.orchestra.run.smithi163.stdout: "id": 4285, 2023-10-09T06:30:57.742 INFO:teuthology.orchestra.run.smithi163.stdout: "want_state": "up:active", 2023-10-09T06:30:57.742 INFO:teuthology.orchestra.run.smithi163.stdout: "state": "up:active", 2023-10-09T06:30:57.742 INFO:teuthology.orchestra.run.smithi163.stdout: "fs_name": "cephfs", 2023-10-09T06:30:57.743 INFO:teuthology.orchestra.run.smithi163.stdout: "rank_uptime": 13.973605119, 2023-10-09T06:30:57.743 INFO:teuthology.orchestra.run.smithi163.stdout: "mdsmap_epoch": 8, 2023-10-09T06:30:57.743 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch": 20, 2023-10-09T06:30:57.743 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch_barrier": 20, 2023-10-09T06:30:57.743 INFO:teuthology.orchestra.run.smithi163.stdout: "uptime": 21.001102011 2023-10-09T06:30:57.743 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:30:57.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.725+0000 7f8153fff700 1 -- 172.21.15.163:0/705137455 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8134066f30 msgr2=0x7f81340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:57.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.725+0000 7f8153fff700 1 --2- 172.21.15.163:0/705137455 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8134066f30 0x7f81340693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8144009fd0 tx=0x7f814400b040 comp rx=0 tx=0).stop 2023-10-09T06:30:57.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.725+0000 7f8153fff700 1 -- 172.21.15.163:0/705137455 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8154156930 msgr2=0x7f815415b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:57.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.725+0000 7f8153fff700 1 --2- 172.21.15.163:0/705137455 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8154156930 0x7f815415b7e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f814c0030c0 tx=0x7f814c016000 comp rx=0 tx=0).stop 2023-10-09T06:30:57.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.725+0000 7f8153fff700 1 -- 172.21.15.163:0/705137455 shutdown_connections 2023-10-09T06:30:57.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.725+0000 7f8153fff700 1 --2- 172.21.15.163:0/705137455 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8134066f30 0x7f81340693e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.746 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.725+0000 7f8153fff700 1 --2- 172.21.15.163:0/705137455 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8154156930 0x7f815415b7e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.746 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.725+0000 7f8153fff700 1 --2- 172.21.15.163:0/705137455 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f815406e570 0x7f8154154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.746 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.725+0000 7f8153fff700 1 --2- 172.21.15.163:0/705137455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81540c8d10 0x7f81540c4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:57.746 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.725+0000 7f8153fff700 1 -- 172.21.15.163:0/705137455 >> 172.21.15.163:0/705137455 conn(0x7f815406b450 msgr2=0x7f81540b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:57.746 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.725+0000 7f8153fff700 1 -- 172.21.15.163:0/705137455 shutdown_connections 2023-10-09T06:30:57.746 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:57.725+0000 7f8153fff700 1 -- 172.21.15.163:0/705137455 wait complete. 2023-10-09T06:30:57.747 INFO:teuthology.run_tasks:Running task kclient... 2023-10-09T06:30:57.770 INFO:tasks.kclient:Mounting kernel clients... 2023-10-09T06:30:57.771 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-09T06:30:57.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:30:57.771 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2023-10-09T06:30:57.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:30:57.787 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link delete ceph-brx 2023-10-09T06:30:57.853 INFO:teuthology.orchestra.run.smithi163.stderr:Cannot find device "ceph-brx" 2023-10-09T06:30:57.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-09T06:30:57.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:30:57.855 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2023-10-09T06:30:57.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:30:57.910 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link delete ceph-brx 2023-10-09T06:30:57.975 INFO:teuthology.orchestra.run.smithi163.stderr:Cannot find device "ceph-brx" 2023-10-09T06:30:57.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-09T06:30:57.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:30:57.976 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-09T06:30:57.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:30:57.994 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link delete ceph-brx 2023-10-09T06:30:58.066 INFO:teuthology.orchestra.run.smithi181.stderr:Cannot find device "ceph-brx" 2023-10-09T06:30:58.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-09T06:30:58.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:30:58.069 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-09T06:30:58.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:30:58.129 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link delete ceph-brx 2023-10-09T06:30:58.201 INFO:teuthology.orchestra.run.smithi181.stderr:Cannot find device "ceph-brx" 2023-10-09T06:30:58.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-09T06:30:58.203 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-10-09T06:30:58.203 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-09T06:30:58.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.317+0000 7efd4a5db700 1 Processor -- start 2023-10-09T06:30:58.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.317+0000 7efd4a5db700 1 -- start start 2023-10-09T06:30:58.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.318+0000 7efd4a5db700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd44159890 0x7efd4415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:58.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.318+0000 7efd4a5db700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efd44159c60 0x7efd4415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:58.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.318+0000 7efd4a5db700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd4415a160 con 0x7efd4415db10 2023-10-09T06:30:58.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.318+0000 7efd4a5db700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd440d13a0 con 0x7efd44159c60 2023-10-09T06:30:58.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.318+0000 7efd4a5db700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd4415e140 con 0x7efd44159890 2023-10-09T06:30:58.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.318+0000 7efd43fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd44159890 0x7efd4415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:58.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.318+0000 7efd437fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efd44159c60 0x7efd4415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:58.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.318+0000 7efd48b78700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7efd4415db10 0x7efd44069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:48036/0 (socket says 172.21.15.163:48036) 2023-10-09T06:30:58.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.318+0000 7efd48b78700 1 -- 172.21.15.163:0/1963091864 learned_addr learned my addr 172.21.15.163:0/1963091864 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:30:58.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.319+0000 7efd43fff700 1 -- 172.21.15.163:0/1963091864 >> v2:172.21.15.181:3300/0 conn(0x7efd44159c60 msgr2=0x7efd4415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:30:58.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.319+0000 7efd43fff700 1 --2- 172.21.15.163:0/1963091864 >> v2:172.21.15.181:3300/0 conn(0x7efd44159c60 0x7efd4415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:58.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.319+0000 7efd42ffd700 1 -- 172.21.15.163:0/1963091864 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1454498044 0 0) 0x7efd4415a160 con 0x7efd4415db10 2023-10-09T06:30:58.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.319+0000 7efd43fff700 1 -- 172.21.15.163:0/1963091864 >> v1:172.21.15.181:6789/0 conn(0x7efd4415db10 legacy=0x7efd44069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:58.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.319+0000 7efd43fff700 1 -- 172.21.15.163:0/1963091864 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd4415a3d0 con 0x7efd44159890 2023-10-09T06:30:58.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.319+0000 7efd43fff700 1 --2- 172.21.15.163:0/1963091864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd44159890 0x7efd4415abd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efd340098e0 tx=0x7efd340091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb8c5d79aa6b92e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:58.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.319+0000 7efd42ffd700 1 -- 172.21.15.163:0/1963091864 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3400b640 con 0x7efd44159890 2023-10-09T06:30:58.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.319+0000 7efd42ffd700 1 -- 172.21.15.163:0/1963091864 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd3400b7a0 con 0x7efd44159890 2023-10-09T06:30:58.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.319+0000 7efd42ffd700 1 -- 172.21.15.163:0/1963091864 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3400bb40 con 0x7efd44159890 2023-10-09T06:30:58.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.319+0000 7efd4a5db700 1 -- 172.21.15.163:0/1963091864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd44159890 msgr2=0x7efd4415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:58.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.319+0000 7efd4a5db700 1 --2- 172.21.15.163:0/1963091864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd44159890 0x7efd4415abd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efd340098e0 tx=0x7efd340091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:58.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.319+0000 7efd4a5db700 1 -- 172.21.15.163:0/1963091864 shutdown_connections 2023-10-09T06:30:58.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.319+0000 7efd4a5db700 1 --2- 172.21.15.163:0/1963091864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd44159890 0x7efd4415abd0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:58.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.319+0000 7efd4a5db700 1 --2- 172.21.15.163:0/1963091864 >> v2:172.21.15.181:3300/0 conn(0x7efd44159c60 0x7efd4415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:58.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.319+0000 7efd4a5db700 1 -- 172.21.15.163:0/1963091864 >> 172.21.15.163:0/1963091864 conn(0x7efd44068e20 msgr2=0x7efd440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:58.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.320+0000 7efd4a5db700 1 -- 172.21.15.163:0/1963091864 shutdown_connections 2023-10-09T06:30:58.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.320+0000 7efd4a5db700 1 -- 172.21.15.163:0/1963091864 wait complete. 2023-10-09T06:30:58.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.321+0000 7efd4a5db700 1 Processor -- start 2023-10-09T06:30:58.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.321+0000 7efd4a5db700 1 -- start start 2023-10-09T06:30:58.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.321+0000 7efd4a5db700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd44159890 0x7efd440b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:58.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.321+0000 7efd4a5db700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd44159c60 0x7efd440ce170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:58.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.321+0000 7efd4a5db700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd4415db10 0x7efd440ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:58.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.321+0000 7efd4a5db700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd440cf920 con 0x7efd44159c60 2023-10-09T06:30:58.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.321+0000 7efd4a5db700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd441529d0 con 0x7efd44159890 2023-10-09T06:30:58.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.321+0000 7efd4a5db700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd440ced10 con 0x7efd4415db10 2023-10-09T06:30:58.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.322+0000 7efd437fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd44159c60 0x7efd440ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:58.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.322+0000 7efd437fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd44159c60 0x7efd440ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44324/0 (socket says 172.21.15.163:44324) 2023-10-09T06:30:58.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.322+0000 7efd437fe700 1 -- 172.21.15.163:0/133334079 learned_addr learned my addr 172.21.15.163:0/133334079 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:30:58.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.322+0000 7efd48b78700 1 --2- 172.21.15.163:0/133334079 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd4415db10 0x7efd440ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:58.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.322+0000 7efd43fff700 1 --2- 172.21.15.163:0/133334079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd44159890 0x7efd440b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:58.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.322+0000 7efd437fe700 1 -- 172.21.15.163:0/133334079 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd4415db10 msgr2=0x7efd440ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:58.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.322+0000 7efd437fe700 1 --2- 172.21.15.163:0/133334079 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd4415db10 0x7efd440ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:58.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.322+0000 7efd437fe700 1 -- 172.21.15.163:0/133334079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd44159890 msgr2=0x7efd440b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:58.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.322+0000 7efd437fe700 1 --2- 172.21.15.163:0/133334079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd44159890 0x7efd440b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:58.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.322+0000 7efd437fe700 1 -- 172.21.15.163:0/133334079 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd34009420 con 0x7efd44159c60 2023-10-09T06:30:58.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.322+0000 7efd437fe700 1 --2- 172.21.15.163:0/133334079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd44159c60 0x7efd440ce170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efd3800ec50 tx=0x7efd380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:58.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.322+0000 7efd2bfff700 1 -- 172.21.15.163:0/133334079 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3800c850 con 0x7efd44159c60 2023-10-09T06:30:58.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.322+0000 7efd2bfff700 1 -- 172.21.15.163:0/133334079 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd3800c9b0 con 0x7efd44159c60 2023-10-09T06:30:58.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.322+0000 7efd4a5db700 1 -- 172.21.15.163:0/133334079 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd440ceff0 con 0x7efd44159c60 2023-10-09T06:30:58.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.322+0000 7efd2bfff700 1 -- 172.21.15.163:0/133334079 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3800cd70 con 0x7efd44159c60 2023-10-09T06:30:58.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.322+0000 7efd4a5db700 1 -- 172.21.15.163:0/133334079 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd440cf5b0 con 0x7efd44159c60 2023-10-09T06:30:58.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.323+0000 7efd4a5db700 1 -- 172.21.15.163:0/133334079 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd4404c3c0 con 0x7efd44159c60 2023-10-09T06:30:58.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.324+0000 7efd2bfff700 1 -- 172.21.15.163:0/133334079 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efd38013680 con 0x7efd44159c60 2023-10-09T06:30:58.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.324+0000 7efd2bfff700 1 --2- 172.21.15.163:0/133334079 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7efd30066f30 0x7efd300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:30:58.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.324+0000 7efd2bfff700 1 -- 172.21.15.163:0/133334079 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efd380847b0 con 0x7efd44159c60 2023-10-09T06:30:58.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.324+0000 7efd43fff700 1 --2- 172.21.15.163:0/133334079 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7efd30066f30 0x7efd300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:30:58.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.324+0000 7efd43fff700 1 --2- 172.21.15.163:0/133334079 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7efd30066f30 0x7efd300693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efd340093f0 tx=0x7efd34017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:30:58.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.328+0000 7efd2bfff700 1 -- 172.21.15.163:0/133334079 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd38051a20 con 0x7efd44159c60 2023-10-09T06:30:58.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.557+0000 7efd4a5db700 1 -- 172.21.15.163:0/133334079 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efd440cfa60 con 0x7efd44159c60 2023-10-09T06:30:58.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.558+0000 7efd2bfff700 1 -- 172.21.15.163:0/133334079 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7efd38055040 con 0x7efd44159c60 2023-10-09T06:30:58.559 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-09T06:30:58.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.559+0000 7efd4a5db700 1 -- 172.21.15.163:0/133334079 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7efd30066f30 msgr2=0x7efd300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:58.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.560+0000 7efd4a5db700 1 --2- 172.21.15.163:0/133334079 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7efd30066f30 0x7efd300693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efd340093f0 tx=0x7efd34017000 comp rx=0 tx=0).stop 2023-10-09T06:30:58.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.560+0000 7efd4a5db700 1 -- 172.21.15.163:0/133334079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd44159c60 msgr2=0x7efd440ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:30:58.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.560+0000 7efd4a5db700 1 --2- 172.21.15.163:0/133334079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd44159c60 0x7efd440ce170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efd3800ec50 tx=0x7efd380091a0 comp rx=0 tx=0).stop 2023-10-09T06:30:58.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.560+0000 7efd4a5db700 1 -- 172.21.15.163:0/133334079 shutdown_connections 2023-10-09T06:30:58.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.560+0000 7efd4a5db700 1 --2- 172.21.15.163:0/133334079 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7efd30066f30 0x7efd300693e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:58.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.560+0000 7efd4a5db700 1 --2- 172.21.15.163:0/133334079 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd4415db10 0x7efd440ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:58.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.560+0000 7efd4a5db700 1 --2- 172.21.15.163:0/133334079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd44159890 0x7efd440b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:58.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.560+0000 7efd4a5db700 1 --2- 172.21.15.163:0/133334079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd44159c60 0x7efd440ce170 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:30:58.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.560+0000 7efd4a5db700 1 -- 172.21.15.163:0/133334079 >> 172.21.15.163:0/133334079 conn(0x7efd44068e20 msgr2=0x7efd440b1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:30:58.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.560+0000 7efd4a5db700 1 -- 172.21.15.163:0/133334079 shutdown_connections 2023-10-09T06:30:58.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:30:58.560+0000 7efd4a5db700 1 -- 172.21.15.163:0/133334079 wait complete. 2023-10-09T06:30:58.574 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-09T06:30:58.574 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-09T06:30:58.574 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi163.front.sepia.ceph.com 2023-10-09T06:30:58.574 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-09T06:30:58.574 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-09T06:30:58.574 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-09T06:30:58.574 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-09T06:30:58.574 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-09T06:30:58.574 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:30:58.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:30:58.575 DEBUG:teuthology.orchestra.run.smithi163:> ip addr 2023-10-09T06:30:58.590 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:30:58.590 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:30:58.590 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:30:58.590 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:30:58.590 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2023-10-09T06:30:58.590 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:30:58.590 INFO:teuthology.orchestra.run.smithi163.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:30:58.590 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:06 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:30:58.591 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f0 2023-10-09T06:30:58.591 INFO:teuthology.orchestra.run.smithi163.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:30:58.591 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:07 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:30:58.591 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f1 2023-10-09T06:30:58.591 INFO:teuthology.orchestra.run.smithi163.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:30:58.591 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5a brd ff:ff:ff:ff:ff:ff 2023-10-09T06:30:58.591 INFO:teuthology.orchestra.run.smithi163.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-09T06:30:58.592 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5b brd ff:ff:ff:ff:ff:ff 2023-10-09T06:30:58.592 INFO:teuthology.orchestra.run.smithi163.stdout: inet 172.21.15.163/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-09T06:30:58.592 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft 42465sec preferred_lft 42465sec 2023-10-09T06:30:58.592 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::ec4:7aff:fe88:7d5b/64 scope link 2023-10-09T06:30:58.592 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:30:58.593 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-09T06:30:58.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:30:58.593 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-09T06:30:58.593 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-09T06:30:58.593 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link add name ceph-brx type bridge 2023-10-09T06:30:58.593 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip addr flush dev ceph-brx 2023-10-09T06:30:58.594 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set ceph-brx up 2023-10-09T06:30:58.594 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-09T06:30:58.594 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-09T06:30:58.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:30:58.763 DEBUG:teuthology.orchestra.run.smithi163:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-09T06:30:58.788 INFO:teuthology.orchestra.run.smithi163.stdout:1 2023-10-09T06:30:58.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:30:58.789 DEBUG:teuthology.orchestra.run.smithi163:> route 2023-10-09T06:30:58.861 INFO:teuthology.orchestra.run.smithi163.stdout:Kernel IP routing table 2023-10-09T06:30:58.861 INFO:teuthology.orchestra.run.smithi163.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-09T06:30:58.861 INFO:teuthology.orchestra.run.smithi163.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-09T06:30:58.861 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-09T06:30:58.861 INFO:teuthology.orchestra.run.smithi163.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-09T06:30:58.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:30:58.862 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-09T06:30:58.862 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-09T06:30:58.862 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-09T06:30:58.862 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-09T06:30:58.862 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-09T06:30:58.862 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-09T06:30:59.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:30:59.119 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2023-10-09T06:30:59.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:30:59.134 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list-id 2023-10-09T06:30:59.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:30:59.190 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-09T06:30:59.190 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-09T06:30:59.190 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-09T06:30:59.190 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-09T06:30:59.190 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-09T06:30:59.279 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-09T06:30:59.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:30:59.279 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-09T06:30:59.279 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-09T06:30:59.279 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-09T06:30:59.279 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-09T06:30:59.279 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-09T06:30:59.279 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-09T06:30:59.280 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-09T06:30:59.280 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-09T06:30:59.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:30:59.484 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-09T06:30:59.484 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-09T06:30:59.485 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set brx.0 up 2023-10-09T06:30:59.485 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set dev brx.0 master ceph-brx 2023-10-09T06:30:59.485 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-09T06:30:59.577 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:30:59.577 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-09T06:30:59.591 INFO:teuthology.orchestra.run.smithi163.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:30:59.591 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:30:59.592 DEBUG:teuthology.orchestra.run.smithi163:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-09T06:30:59.646 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-09T06:30:59.646 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-09T06:30:59.646 DEBUG:teuthology.orchestra.run.smithi163:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:30:59.738 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:30:59.738 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:30:59.829 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:30:59.829 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:30:59.830 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:30:59.830 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with new device syntax: 0@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:30:59.830 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:30:59.830 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:30:59.830 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:30:59.830 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:30:59.852 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:30:59.852 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:30:59.852 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:30:59.853 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with new device syntax: 0@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:30:59.853 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:30:59.853 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:30:59.853 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:30:59.853 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:30:59.853 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790" 2023-10-09T06:30:59.853 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790:/ 2023-10-09T06:30:59.853 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=2520b8f7-539f-4bae-9f17-1fce66f66ddd" will pass to kernel 2023-10-09T06:30:59.854 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-09T06:30:59.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:30:59.855 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-09T06:30:59.890 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-10-09T06:30:59.891 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-09T06:31:00.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.053+0000 7f226682c700 1 Processor -- start 2023-10-09T06:31:00.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.053+0000 7f226682c700 1 -- start start 2023-10-09T06:31:00.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.054+0000 7f226682c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2260159c60 0x7f226015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:00.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.054+0000 7f226682c700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f226015a160 con 0x7f226015db10 2023-10-09T06:31:00.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.054+0000 7f226682c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22600d13a0 con 0x7f2260159890 2023-10-09T06:31:00.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.054+0000 7f226682c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226015e140 con 0x7f2260159c60 2023-10-09T06:31:00.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.054+0000 7f2264dc9700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f226015db10 0x7f2260069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:47746/0 (socket says 172.21.15.163:47746) 2023-10-09T06:31:00.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.054+0000 7f2264dc9700 1 -- 172.21.15.163:0/3443573939 learned_addr learned my addr 172.21.15.163:0/3443573939 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:00.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.054+0000 7f225f7fe700 1 --2- 172.21.15.163:0/3443573939 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2260159c60 0x7f226015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:00.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.054+0000 7f225effd700 1 -- 172.21.15.163:0/3443573939 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4113829497 0 0) 0x7f226015a160 con 0x7f226015db10 2023-10-09T06:31:00.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.054+0000 7f225effd700 1 -- 172.21.15.163:0/3443573939 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f224c003570 con 0x7f226015db10 2023-10-09T06:31:00.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.054+0000 7f225effd700 1 -- 172.21.15.163:0/3443573939 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2405619570 0 0) 0x7f22600d13a0 con 0x7f2260159890 2023-10-09T06:31:00.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.054+0000 7f225effd700 1 -- 172.21.15.163:0/3443573939 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f226015a160 con 0x7f2260159890 2023-10-09T06:31:00.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.054+0000 7f225f7fe700 1 -- 172.21.15.163:0/3443573939 >> v1:172.21.15.181:6789/0 conn(0x7f2260159890 legacy=0x7f226015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:00.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.054+0000 7f225f7fe700 1 -- 172.21.15.163:0/3443573939 >> v1:172.21.15.163:6789/0 conn(0x7f226015db10 legacy=0x7f2260069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:00.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.054+0000 7f225f7fe700 1 -- 172.21.15.163:0/3443573939 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226015a3d0 con 0x7f2260159c60 2023-10-09T06:31:00.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.055+0000 7f225f7fe700 1 --2- 172.21.15.163:0/3443573939 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2260159c60 0x7f226015d5d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2250009910 tx=0x7f22500091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f98ae9243778e428 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:00.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.055+0000 7f225effd700 1 -- 172.21.15.163:0/3443573939 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2250011630 con 0x7f2260159c60 2023-10-09T06:31:00.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.055+0000 7f225effd700 1 -- 172.21.15.163:0/3443573939 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2250011790 con 0x7f2260159c60 2023-10-09T06:31:00.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.055+0000 7f225effd700 1 -- 172.21.15.163:0/3443573939 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2250011b30 con 0x7f2260159c60 2023-10-09T06:31:00.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.055+0000 7f226682c700 1 -- 172.21.15.163:0/3443573939 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2260159c60 msgr2=0x7f226015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:00.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.055+0000 7f226682c700 1 --2- 172.21.15.163:0/3443573939 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2260159c60 0x7f226015d5d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2250009910 tx=0x7f22500091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:00.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.055+0000 7f226682c700 1 -- 172.21.15.163:0/3443573939 shutdown_connections 2023-10-09T06:31:00.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.055+0000 7f226682c700 1 --2- 172.21.15.163:0/3443573939 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2260159c60 0x7f226015d5d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:00.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.055+0000 7f226682c700 1 -- 172.21.15.163:0/3443573939 >> 172.21.15.163:0/3443573939 conn(0x7f2260068e20 msgr2=0x7f22600d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:00.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.055+0000 7f226682c700 1 -- 172.21.15.163:0/3443573939 shutdown_connections 2023-10-09T06:31:00.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.055+0000 7f226682c700 1 -- 172.21.15.163:0/3443573939 wait complete. 2023-10-09T06:31:00.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.056+0000 7f226682c700 1 Processor -- start 2023-10-09T06:31:00.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.057+0000 7f226682c700 1 -- start start 2023-10-09T06:31:00.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.057+0000 7f226682c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2260159890 0x7f226013f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:00.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.057+0000 7f226682c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2260159c60 0x7f226016a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:00.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.057+0000 7f226682c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f226015db10 0x7f226016aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:00.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.057+0000 7f226682c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226013f7b0 con 0x7f2260159890 2023-10-09T06:31:00.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.057+0000 7f226682c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226014c260 con 0x7f226015db10 2023-10-09T06:31:00.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.057+0000 7f226682c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226016d440 con 0x7f2260159c60 2023-10-09T06:31:00.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.057+0000 7f225ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2260159890 0x7f226013f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:00.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.057+0000 7f225ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2260159890 0x7f226013f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35660/0 (socket says 172.21.15.163:35660) 2023-10-09T06:31:00.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.057+0000 7f225ffff700 1 -- 172.21.15.163:0/3035894911 learned_addr learned my addr 172.21.15.163:0/3035894911 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:00.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.058+0000 7f225f7fe700 1 --2- 172.21.15.163:0/3035894911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2260159c60 0x7f226016a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:00.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.058+0000 7f2264dc9700 1 --2- 172.21.15.163:0/3035894911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f226015db10 0x7f226016aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:00.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.058+0000 7f225ffff700 1 -- 172.21.15.163:0/3035894911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2260159c60 msgr2=0x7f226016a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:00.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.058+0000 7f225ffff700 1 --2- 172.21.15.163:0/3035894911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2260159c60 0x7f226016a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:00.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.058+0000 7f225ffff700 1 -- 172.21.15.163:0/3035894911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f226015db10 msgr2=0x7f226016aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:00.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.058+0000 7f225ffff700 1 --2- 172.21.15.163:0/3035894911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f226015db10 0x7f226016aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:00.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.058+0000 7f225ffff700 1 -- 172.21.15.163:0/3035894911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2250009420 con 0x7f2260159890 2023-10-09T06:31:00.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.058+0000 7f225ffff700 1 --2- 172.21.15.163:0/3035894911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2260159890 0x7f226013f270 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f224800c790 tx=0x7f2248007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:00.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.058+0000 7f2257fff700 1 -- 172.21.15.163:0/3035894911 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2248010510 con 0x7f2260159890 2023-10-09T06:31:00.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.058+0000 7f2257fff700 1 -- 172.21.15.163:0/3035894911 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2248010670 con 0x7f2260159890 2023-10-09T06:31:00.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.058+0000 7f226682c700 1 -- 172.21.15.163:0/3035894911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f226016d720 con 0x7f2260159890 2023-10-09T06:31:00.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.058+0000 7f2257fff700 1 -- 172.21.15.163:0/3035894911 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2248010a10 con 0x7f2260159890 2023-10-09T06:31:00.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.058+0000 7f226682c700 1 -- 172.21.15.163:0/3035894911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f226016dce0 con 0x7f2260159890 2023-10-09T06:31:00.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.059+0000 7f226682c700 1 -- 172.21.15.163:0/3035894911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f226004c3c0 con 0x7f2260159890 2023-10-09T06:31:00.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.059+0000 7f2257fff700 1 -- 172.21.15.163:0/3035894911 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f224801b400 con 0x7f2260159890 2023-10-09T06:31:00.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.060+0000 7f2257fff700 1 --2- 172.21.15.163:0/3035894911 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2240066f30 0x7f22400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:00.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.060+0000 7f2257fff700 1 -- 172.21.15.163:0/3035894911 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f22480842e0 con 0x7f2260159890 2023-10-09T06:31:00.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.060+0000 7f225f7fe700 1 --2- 172.21.15.163:0/3035894911 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2240066f30 0x7f22400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:00.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.061+0000 7f225f7fe700 1 --2- 172.21.15.163:0/3035894911 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2240066f30 0x7f22400693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f226015a090 tx=0x7f2250017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:00.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.064+0000 7f2257fff700 1 -- 172.21.15.163:0/3035894911 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2248051550 con 0x7f2260159890 2023-10-09T06:31:00.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.292+0000 7f226682c700 1 -- 172.21.15.163:0/3035894911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f226006a5b0 con 0x7f2260159890 2023-10-09T06:31:00.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.292+0000 7f2257fff700 1 -- 172.21.15.163:0/3035894911 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2248054b70 con 0x7f2260159890 2023-10-09T06:31:00.293 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-09T06:31:00.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.294+0000 7f226682c700 1 -- 172.21.15.163:0/3035894911 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2240066f30 msgr2=0x7f22400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:00.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.294+0000 7f226682c700 1 --2- 172.21.15.163:0/3035894911 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2240066f30 0x7f22400693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f226015a090 tx=0x7f2250017000 comp rx=0 tx=0).stop 2023-10-09T06:31:00.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.294+0000 7f226682c700 1 -- 172.21.15.163:0/3035894911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2260159890 msgr2=0x7f226013f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:00.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.294+0000 7f226682c700 1 --2- 172.21.15.163:0/3035894911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2260159890 0x7f226013f270 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f224800c790 tx=0x7f2248007e10 comp rx=0 tx=0).stop 2023-10-09T06:31:00.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.294+0000 7f226682c700 1 -- 172.21.15.163:0/3035894911 shutdown_connections 2023-10-09T06:31:00.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.294+0000 7f226682c700 1 --2- 172.21.15.163:0/3035894911 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2240066f30 0x7f22400693e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:00.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.294+0000 7f226682c700 1 --2- 172.21.15.163:0/3035894911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2260159890 0x7f226013f270 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:00.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.294+0000 7f226682c700 1 --2- 172.21.15.163:0/3035894911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f226015db10 0x7f226016aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:00.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.294+0000 7f226682c700 1 --2- 172.21.15.163:0/3035894911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2260159c60 0x7f226016a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:00.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.294+0000 7f226682c700 1 -- 172.21.15.163:0/3035894911 >> 172.21.15.163:0/3035894911 conn(0x7f2260068e20 msgr2=0x7f226015c110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:00.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.294+0000 7f226682c700 1 -- 172.21.15.163:0/3035894911 shutdown_connections 2023-10-09T06:31:00.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:00.294+0000 7f226682c700 1 -- 172.21.15.163:0/3035894911 wait complete. 2023-10-09T06:31:00.307 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-09T06:31:00.307 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-09T06:31:00.307 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi163.front.sepia.ceph.com 2023-10-09T06:31:00.307 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-09T06:31:00.307 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-09T06:31:00.307 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-09T06:31:00.307 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-09T06:31:00.307 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-09T06:31:00.308 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-09T06:31:00.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:00.308 DEBUG:teuthology.orchestra.run.smithi163:> ip addr 2023-10-09T06:31:00.322 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:31:00.323 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:31:00.323 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:31:00.323 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:00.323 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2023-10-09T06:31:00.323 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:00.323 INFO:teuthology.orchestra.run.smithi163.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:31:00.323 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:06 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:31:00.323 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f0 2023-10-09T06:31:00.324 INFO:teuthology.orchestra.run.smithi163.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:31:00.324 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:07 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:31:00.324 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f1 2023-10-09T06:31:00.324 INFO:teuthology.orchestra.run.smithi163.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:31:00.324 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5a brd ff:ff:ff:ff:ff:ff 2023-10-09T06:31:00.324 INFO:teuthology.orchestra.run.smithi163.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-09T06:31:00.324 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5b brd ff:ff:ff:ff:ff:ff 2023-10-09T06:31:00.324 INFO:teuthology.orchestra.run.smithi163.stdout: inet 172.21.15.163/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-09T06:31:00.324 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft 42463sec preferred_lft 42463sec 2023-10-09T06:31:00.324 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::ec4:7aff:fe88:7d5b/64 scope link 2023-10-09T06:31:00.325 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:00.325 INFO:teuthology.orchestra.run.smithi163.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:31:00.325 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 2e:01:84:b9:f4:8e brd ff:ff:ff:ff:ff:ff 2023-10-09T06:31:00.325 INFO:teuthology.orchestra.run.smithi163.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-09T06:31:00.325 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:00.325 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::d03c:93ff:fe9e:c8cb/64 scope link 2023-10-09T06:31:00.325 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:00.325 INFO:teuthology.orchestra.run.smithi163.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:31:00.325 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 2e:01:84:b9:f4:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-09T06:31:00.325 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::2c01:84ff:feb9:f48e/64 scope link tentative 2023-10-09T06:31:00.326 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:00.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:00.326 DEBUG:teuthology.orchestra.run.smithi163:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-09T06:31:00.386 INFO:teuthology.orchestra.run.smithi163.stdout:1 2023-10-09T06:31:00.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:00.387 DEBUG:teuthology.orchestra.run.smithi163:> route 2023-10-09T06:31:00.443 INFO:teuthology.orchestra.run.smithi163.stdout:Kernel IP routing table 2023-10-09T06:31:00.444 INFO:teuthology.orchestra.run.smithi163.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-09T06:31:00.444 INFO:teuthology.orchestra.run.smithi163.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-09T06:31:00.444 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-09T06:31:00.444 INFO:teuthology.orchestra.run.smithi163.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-09T06:31:00.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:00.444 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-09T06:31:00.444 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-09T06:31:00.445 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-09T06:31:00.445 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-09T06:31:00.445 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-09T06:31:00.445 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-09T06:31:00.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:00.545 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2023-10-09T06:31:00.560 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-09T06:31:00.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:00.561 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list-id 2023-10-09T06:31:00.615 INFO:teuthology.orchestra.run.smithi163.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-09T06:31:00.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:00.616 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-09T06:31:00.617 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-09T06:31:00.617 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-09T06:31:00.617 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-09T06:31:00.617 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-09T06:31:00.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:00.704 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-09T06:31:00.739 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:31:00.739 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:31:00.739 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:31:00.740 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:00.740 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2023-10-09T06:31:00.740 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:00.740 INFO:teuthology.orchestra.run.smithi163.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-09T06:31:00.740 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0a:23:fa:0e:15:45 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-09T06:31:00.740 INFO:teuthology.orchestra.run.smithi163.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-09T06:31:00.740 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:00.740 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::823:faff:fe0e:1545/64 scope link tentative 2023-10-09T06:31:00.740 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:00.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:00.750 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-09T06:31:00.824 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:31:00.824 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:31:00.825 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:31:00.825 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:00.825 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2023-10-09T06:31:00.825 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:00.825 INFO:teuthology.orchestra.run.smithi163.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-09T06:31:00.825 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0a:23:fa:0e:15:45 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-09T06:31:00.825 INFO:teuthology.orchestra.run.smithi163.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-09T06:31:00.825 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:00.825 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::823:faff:fe0e:1545/64 scope link tentative 2023-10-09T06:31:00.825 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:00.831 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-09T06:31:00.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:00.832 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-09T06:31:00.832 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-09T06:31:00.832 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-09T06:31:00.832 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-09T06:31:00.832 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-09T06:31:00.832 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-09T06:31:00.832 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-09T06:31:00.832 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-09T06:31:01.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:01.045 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-09T06:31:01.045 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-09T06:31:01.046 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set brx.1 up 2023-10-09T06:31:01.046 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set dev brx.1 master ceph-brx 2023-10-09T06:31:01.046 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-09T06:31:01.126 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:31:01.126 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-09T06:31:01.140 INFO:teuthology.orchestra.run.smithi163.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-09T06:31:01.141 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:31:01.141 DEBUG:teuthology.orchestra.run.smithi163:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-09T06:31:01.195 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-10-09T06:31:01.195 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-09T06:31:01.195 DEBUG:teuthology.orchestra.run.smithi163:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:31:01.274 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:31:01.274 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:31:01.293 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:31:01.293 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:31:01.293 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:31:01.293 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with new device syntax: 2@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:31:01.293 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:31:01.293 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:31:01.293 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:31:01.293 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:31:01.315 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:31:01.315 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:31:01.315 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:31:01.315 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with new device syntax: 2@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:31:01.315 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:31:01.315 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:31:01.315 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:31:01.315 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:31:01.315 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790" 2023-10-09T06:31:01.316 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790:/ 2023-10-09T06:31:01.316 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,key=2,mds_namespace=cephfs,fsid=2520b8f7-539f-4bae-9f17-1fce66f66ddd" will pass to kernel 2023-10-09T06:31:01.317 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-09T06:31:01.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:01.317 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-09T06:31:01.381 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-10-09T06:31:01.381 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-09T06:31:01.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.537+0000 7f5dc90ce700 1 Processor -- start 2023-10-09T06:31:01.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.537+0000 7f5dc90ce700 1 -- start start 2023-10-09T06:31:01.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.538+0000 7f5dc90ce700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5dc406b520 0x7f5dc40cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:01.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.538+0000 7f5dc90ce700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc40cc210 con 0x7f5dc40c91e0 2023-10-09T06:31:01.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.538+0000 7f5dc90ce700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc40cc4f0 con 0x7f5dc406aa40 2023-10-09T06:31:01.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.538+0000 7f5dc90ce700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40cc7d0 con 0x7f5dc406b520 2023-10-09T06:31:01.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.538+0000 7f5dc259c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f5dc40c91e0 0x7f5dc406ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:47756/0 (socket says 172.21.15.163:47756) 2023-10-09T06:31:01.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.538+0000 7f5dc259c700 1 -- 172.21.15.163:0/839408170 learned_addr learned my addr 172.21.15.163:0/839408170 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:01.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.538+0000 7f5dc1d9b700 1 -- 172.21.15.163:0/839408170 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2867850759 0 0) 0x7f5dc40cc210 con 0x7f5dc40c91e0 2023-10-09T06:31:01.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.538+0000 7f5dc1d9b700 1 -- 172.21.15.163:0/839408170 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dac003570 con 0x7f5dc40c91e0 2023-10-09T06:31:01.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.538+0000 7f5dc359e700 1 --2- 172.21.15.163:0/839408170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5dc406b520 0x7f5dc40cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:01.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.538+0000 7f5dc1d9b700 1 -- 172.21.15.163:0/839408170 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1840454035 0 0) 0x7f5dac003570 con 0x7f5dc40c91e0 2023-10-09T06:31:01.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.538+0000 7f5dc1d9b700 1 -- 172.21.15.163:0/839408170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5dc406b520 msgr2=0x7f5dc40cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:01.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.538+0000 7f5dc1d9b700 1 --2- 172.21.15.163:0/839408170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5dc406b520 0x7f5dc40cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:01.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.538+0000 7f5dc1d9b700 1 -- 172.21.15.163:0/839408170 >> v1:172.21.15.181:6789/0 conn(0x7f5dc406aa40 legacy=0x7f5dc406bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:01.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.538+0000 7f5dc1d9b700 1 -- 172.21.15.163:0/839408170 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc40ccab0 con 0x7f5dc40c91e0 2023-10-09T06:31:01.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.538+0000 7f5dc1d9b700 1 -- 172.21.15.163:0/839408170 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f5db8001470 con 0x7f5dc40c91e0 2023-10-09T06:31:01.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.538+0000 7f5dc1d9b700 1 -- 172.21.15.163:0/839408170 >> v1:172.21.15.163:6789/0 conn(0x7f5dc40c91e0 legacy=0x7f5dc406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:01.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.538+0000 7f5dc1d9b700 1 --2- 172.21.15.163:0/839408170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dac003a90 0x7f5dac003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:01.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.538+0000 7f5dc1d9b700 1 --2- 172.21.15.163:0/839408170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dac004810 0x7f5dac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:01.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.538+0000 7f5dc1d9b700 1 --2- 172.21.15.163:0/839408170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5dc406b520 0x7f5dac0051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:01.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.538+0000 7f5dc1d9b700 1 -- 172.21.15.163:0/839408170 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40cc7d0 con 0x7f5dac003a90 2023-10-09T06:31:01.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.538+0000 7f5dc1d9b700 1 -- 172.21.15.163:0/839408170 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac00bb60 con 0x7f5dac004810 2023-10-09T06:31:01.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.538+0000 7f5dc1d9b700 1 -- 172.21.15.163:0/839408170 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac00bd20 con 0x7f5dc406b520 2023-10-09T06:31:01.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.538+0000 7f5dc2d9d700 1 --2- 172.21.15.163:0/839408170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dac003a90 0x7f5dac003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:01.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.538+0000 7f5dc259c700 1 --2- 172.21.15.163:0/839408170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dac004810 0x7f5dac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:01.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.539+0000 7f5dc359e700 1 --2- 172.21.15.163:0/839408170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5dc406b520 0x7f5dac0051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:01.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.539+0000 7f5dc2d9d700 1 -- 172.21.15.163:0/839408170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5dc406b520 msgr2=0x7f5dac0051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:01.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.539+0000 7f5dc2d9d700 1 --2- 172.21.15.163:0/839408170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5dc406b520 0x7f5dac0051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:01.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.539+0000 7f5dc2d9d700 1 -- 172.21.15.163:0/839408170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dac004810 msgr2=0x7f5dac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:01.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.539+0000 7f5dc2d9d700 1 --2- 172.21.15.163:0/839408170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dac004810 0x7f5dac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:01.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.539+0000 7f5dc2d9d700 1 -- 172.21.15.163:0/839408170 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dac003570 con 0x7f5dac003a90 2023-10-09T06:31:01.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.539+0000 7f5dc2d9d700 1 --2- 172.21.15.163:0/839408170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dac003a90 0x7f5dac003ea0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5db40018d0 tx=0x7f5db4007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40cf7ac867cf1ee3 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:01.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.539+0000 7f5dc1d9b700 1 -- 172.21.15.163:0/839408170 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db4001290 con 0x7f5dac003a90 2023-10-09T06:31:01.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.539+0000 7f5dc1d9b700 1 -- 172.21.15.163:0/839408170 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db4012420 con 0x7f5dac003a90 2023-10-09T06:31:01.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.539+0000 7f5dc1d9b700 1 -- 172.21.15.163:0/839408170 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db40127e0 con 0x7f5dac003a90 2023-10-09T06:31:01.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.539+0000 7f5dc90ce700 1 -- 172.21.15.163:0/839408170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dac003a90 msgr2=0x7f5dac003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:01.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.539+0000 7f5dc90ce700 1 --2- 172.21.15.163:0/839408170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dac003a90 0x7f5dac003ea0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5db40018d0 tx=0x7f5db4007d80 comp rx=0 tx=0).stop 2023-10-09T06:31:01.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.539+0000 7f5dc359e700 1 -- 172.21.15.163:0/839408170 reap_dead start 2023-10-09T06:31:01.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.539+0000 7f5dc90ce700 1 -- 172.21.15.163:0/839408170 shutdown_connections 2023-10-09T06:31:01.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.539+0000 7f5dc90ce700 1 -- 172.21.15.163:0/839408170 >> 172.21.15.163:0/839408170 conn(0x7f5dc40cd670 msgr2=0x7f5dc40cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:01.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.539+0000 7f5dc90ce700 1 -- 172.21.15.163:0/839408170 shutdown_connections 2023-10-09T06:31:01.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.539+0000 7f5dc90ce700 1 -- 172.21.15.163:0/839408170 wait complete. 2023-10-09T06:31:01.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.541+0000 7f5dc90ce700 1 Processor -- start 2023-10-09T06:31:01.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.541+0000 7f5dc90ce700 1 -- start start 2023-10-09T06:31:01.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.541+0000 7f5dc90ce700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dac003a90 0x7f5dc414b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:01.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.541+0000 7f5dc90ce700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dc4167cb0 0x7f5dc4154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:01.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.541+0000 7f5dc90ce700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5dc4153ca0 0x7f5dc40c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:01.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.541+0000 7f5dc90ce700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc4158780 con 0x7f5dac003a90 2023-10-09T06:31:01.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.541+0000 7f5dc90ce700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc4158910 con 0x7f5dc4167cb0 2023-10-09T06:31:01.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.541+0000 7f5dc90ce700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc4154120 con 0x7f5dc4153ca0 2023-10-09T06:31:01.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.541+0000 7f5dc2d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dac003a90 0x7f5dc414b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:01.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.541+0000 7f5dc2d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dac003a90 0x7f5dc414b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35674/0 (socket says 172.21.15.163:35674) 2023-10-09T06:31:01.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.541+0000 7f5dc2d9d700 1 -- 172.21.15.163:0/2603526057 learned_addr learned my addr 172.21.15.163:0/2603526057 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:01.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.542+0000 7f5dc259c700 1 --2- 172.21.15.163:0/2603526057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dc4167cb0 0x7f5dc4154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:01.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.542+0000 7f5dc359e700 1 --2- 172.21.15.163:0/2603526057 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5dc4153ca0 0x7f5dc40c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:01.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.542+0000 7f5dc2d9d700 1 -- 172.21.15.163:0/2603526057 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5dc4153ca0 msgr2=0x7f5dc40c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:01.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.542+0000 7f5dc2d9d700 1 --2- 172.21.15.163:0/2603526057 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5dc4153ca0 0x7f5dc40c6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:01.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.542+0000 7f5dc2d9d700 1 -- 172.21.15.163:0/2603526057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dc4167cb0 msgr2=0x7f5dc4154df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:01.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.542+0000 7f5dc2d9d700 1 --2- 172.21.15.163:0/2603526057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dc4167cb0 0x7f5dc4154df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:01.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.542+0000 7f5dc2d9d700 1 -- 172.21.15.163:0/2603526057 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc41554e0 con 0x7f5dac003a90 2023-10-09T06:31:01.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.542+0000 7f5dc2d9d700 1 --2- 172.21.15.163:0/2603526057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dac003a90 0x7f5dc414b860 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5db4012fd0 tx=0x7f5db4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:01.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.542+0000 7f5daaffd700 1 -- 172.21.15.163:0/2603526057 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db401f620 con 0x7f5dac003a90 2023-10-09T06:31:01.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.542+0000 7f5daaffd700 1 -- 172.21.15.163:0/2603526057 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db401f7a0 con 0x7f5dac003a90 2023-10-09T06:31:01.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.542+0000 7f5daaffd700 1 -- 172.21.15.163:0/2603526057 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db401fb60 con 0x7f5dac003a90 2023-10-09T06:31:01.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.542+0000 7f5dc90ce700 1 -- 172.21.15.163:0/2603526057 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc4156bb0 con 0x7f5dac003a90 2023-10-09T06:31:01.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.542+0000 7f5dc90ce700 1 -- 172.21.15.163:0/2603526057 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc4157170 con 0x7f5dac003a90 2023-10-09T06:31:01.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.543+0000 7f5dc90ce700 1 -- 172.21.15.163:0/2603526057 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc404c3c0 con 0x7f5dac003a90 2023-10-09T06:31:01.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.544+0000 7f5daaffd700 1 -- 172.21.15.163:0/2603526057 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5db401d3a0 con 0x7f5dac003a90 2023-10-09T06:31:01.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.544+0000 7f5daaffd700 1 --2- 172.21.15.163:0/2603526057 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5db0066f30 0x7f5db00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:01.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.544+0000 7f5daaffd700 1 -- 172.21.15.163:0/2603526057 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5db4085bb0 con 0x7f5dac003a90 2023-10-09T06:31:01.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.544+0000 7f5dc259c700 1 --2- 172.21.15.163:0/2603526057 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5db0066f30 0x7f5db00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:01.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.545+0000 7f5dc259c700 1 --2- 172.21.15.163:0/2603526057 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5db0066f30 0x7f5db00693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5db80010e0 tx=0x7f5db8001e50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:01.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.548+0000 7f5daaffd700 1 -- 172.21.15.163:0/2603526057 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5db4052e20 con 0x7f5dac003a90 2023-10-09T06:31:01.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.773+0000 7f5dc90ce700 1 -- 172.21.15.163:0/2603526057 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5dc40c7cb0 con 0x7f5dac003a90 2023-10-09T06:31:01.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.773+0000 7f5daaffd700 1 -- 172.21.15.163:0/2603526057 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5db402d060 con 0x7f5dac003a90 2023-10-09T06:31:01.775 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-09T06:31:01.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.775+0000 7f5dc90ce700 1 -- 172.21.15.163:0/2603526057 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5db0066f30 msgr2=0x7f5db00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:01.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.775+0000 7f5dc90ce700 1 --2- 172.21.15.163:0/2603526057 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5db0066f30 0x7f5db00693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5db80010e0 tx=0x7f5db8001e50 comp rx=0 tx=0).stop 2023-10-09T06:31:01.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.775+0000 7f5dc90ce700 1 -- 172.21.15.163:0/2603526057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dac003a90 msgr2=0x7f5dc414b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:01.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.775+0000 7f5dc90ce700 1 --2- 172.21.15.163:0/2603526057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dac003a90 0x7f5dc414b860 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5db4012fd0 tx=0x7f5db4015000 comp rx=0 tx=0).stop 2023-10-09T06:31:01.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.775+0000 7f5dc90ce700 1 -- 172.21.15.163:0/2603526057 shutdown_connections 2023-10-09T06:31:01.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.775+0000 7f5dc90ce700 1 --2- 172.21.15.163:0/2603526057 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5db0066f30 0x7f5db00693e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:01.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.775+0000 7f5dc90ce700 1 --2- 172.21.15.163:0/2603526057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dac003a90 0x7f5dc414b860 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:01.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.775+0000 7f5dc90ce700 1 --2- 172.21.15.163:0/2603526057 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5dc4153ca0 0x7f5dc40c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:01.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.775+0000 7f5dc90ce700 1 --2- 172.21.15.163:0/2603526057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dc4167cb0 0x7f5dc4154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:01.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.775+0000 7f5dc90ce700 1 -- 172.21.15.163:0/2603526057 >> 172.21.15.163:0/2603526057 conn(0x7f5dc40cd670 msgr2=0x7f5dc406ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:01.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.776+0000 7f5dc90ce700 1 -- 172.21.15.163:0/2603526057 shutdown_connections 2023-10-09T06:31:01.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:01.776+0000 7f5dc90ce700 1 -- 172.21.15.163:0/2603526057 wait complete. 2023-10-09T06:31:01.789 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-09T06:31:01.789 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-09T06:31:01.789 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-09T06:31:01.789 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-09T06:31:01.789 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-09T06:31:01.789 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-09T06:31:01.789 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-09T06:31:01.789 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-09T06:31:01.789 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:31:01.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:01.790 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-09T06:31:01.808 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:31:01.808 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:31:01.808 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:31:01.808 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:01.808 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-09T06:31:01.808 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:01.808 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:31:01.808 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:31:01.809 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2023-10-09T06:31:01.809 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:31:01.809 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:31:01.809 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2023-10-09T06:31:01.809 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:31:01.809 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-09T06:31:01.809 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-09T06:31:01.809 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-09T06:31:01.810 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-09T06:31:01.810 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42438sec preferred_lft 42438sec 2023-10-09T06:31:01.810 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2023-10-09T06:31:01.810 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:01.810 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-09T06:31:01.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:01.811 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-09T06:31:01.811 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-09T06:31:01.811 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link add name ceph-brx type bridge 2023-10-09T06:31:01.811 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip addr flush dev ceph-brx 2023-10-09T06:31:01.811 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set ceph-brx up 2023-10-09T06:31:01.811 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-09T06:31:01.811 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-09T06:31:01.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:01.984 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-09T06:31:02.023 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-09T06:31:02.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:02.025 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-09T06:31:02.099 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-09T06:31:03.900 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-09T06:31:03.900 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-09T06:31:03.900 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-09T06:31:03.900 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-09T06:31:03.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:03.901 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-09T06:31:03.901 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-09T06:31:03.901 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-09T06:31:03.901 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-09T06:31:03.901 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-09T06:31:03.901 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-09T06:31:04.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:04.164 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-09T06:31:04.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:04.179 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list-id 2023-10-09T06:31:04.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:04.238 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-09T06:31:04.239 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-09T06:31:04.239 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-09T06:31:04.239 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-09T06:31:04.239 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-09T06:31:04.332 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-09T06:31:04.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:04.333 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-09T06:31:04.333 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-09T06:31:04.333 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-09T06:31:04.333 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-09T06:31:04.333 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-09T06:31:04.333 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-09T06:31:04.334 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-09T06:31:04.334 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-09T06:31:04.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:04.523 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-09T06:31:04.523 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-09T06:31:04.524 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-09T06:31:04.524 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set dev brx.0 master ceph-brx 2023-10-09T06:31:04.524 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-09T06:31:04.616 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:31:04.616 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-09T06:31:04.631 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:31:04.632 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:31:04.632 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-09T06:31:04.688 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-09T06:31:04.688 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-09T06:31:04.688 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:31:04.785 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:31:04.785 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:31:04.876 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:31:04.876 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:31:04.876 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:31:04.876 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 1@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:31:04.876 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:31:04.876 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:31:04.877 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:31:04.877 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:31:04.898 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:31:04.898 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:31:04.898 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:31:04.898 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 1@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:31:04.898 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:31:04.898 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:31:04.898 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:31:04.899 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:31:04.899 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790" 2023-10-09T06:31:04.899 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790:/ 2023-10-09T06:31:04.899 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=2520b8f7-539f-4bae-9f17-1fce66f66ddd" will pass to kernel 2023-10-09T06:31:04.901 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-09T06:31:04.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:04.901 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-09T06:31:04.926 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-10-09T06:31:04.926 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-09T06:31:05.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.052+0000 7faa9dbc6700 1 Processor -- start 2023-10-09T06:31:05.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.052+0000 7faa9dbc6700 1 -- start start 2023-10-09T06:31:05.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.053+0000 7faa9dbc6700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7faa98159890 0x7faa9815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:05.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.053+0000 7faa9dbc6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faa98159c60 0x7faa9815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:05.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.053+0000 7faa9dbc6700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa9815a160 con 0x7faa9815db10 2023-10-09T06:31:05.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.053+0000 7faa9dbc6700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa9815df50 con 0x7faa98159890 2023-10-09T06:31:05.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.053+0000 7faa9dbc6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa980d13a0 con 0x7faa98159c60 2023-10-09T06:31:05.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.053+0000 7faa977fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7faa98159890 0x7faa9815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:05.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.053+0000 7faa97fff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7faa9815db10 0x7faa98069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:47758/0 (socket says 172.21.15.163:47758) 2023-10-09T06:31:05.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.053+0000 7faa97fff700 1 -- 172.21.15.163:0/19443076 learned_addr learned my addr 172.21.15.163:0/19443076 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:05.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.053+0000 7faa96ffd700 1 --2- 172.21.15.163:0/19443076 >> v2:172.21.15.181:3300/0 conn(0x7faa98159c60 0x7faa9815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:05.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.054+0000 7faa977fe700 1 -- 172.21.15.163:0/19443076 >> v2:172.21.15.181:3300/0 conn(0x7faa98159c60 msgr2=0x7faa9815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:05.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.054+0000 7faa977fe700 1 --2- 172.21.15.163:0/19443076 >> v2:172.21.15.181:3300/0 conn(0x7faa98159c60 0x7faa9815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:05.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.054+0000 7faa977fe700 1 -- 172.21.15.163:0/19443076 >> v1:172.21.15.163:6789/0 conn(0x7faa9815db10 legacy=0x7faa98069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:05.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.054+0000 7faa977fe700 1 -- 172.21.15.163:0/19443076 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa9815e140 con 0x7faa98159890 2023-10-09T06:31:05.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.054+0000 7faa967fc700 1 -- 172.21.15.163:0/19443076 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3673696726 0 0) 0x7faa9815a160 con 0x7faa9815db10 2023-10-09T06:31:05.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.054+0000 7faa977fe700 1 --2- 172.21.15.163:0/19443076 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa98159890 0x7faa9815abd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7faa980b67f0 tx=0x7faa88009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d41ca869fa7ffb0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:05.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.054+0000 7faa967fc700 1 -- 172.21.15.163:0/19443076 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8800e070 con 0x7faa98159890 2023-10-09T06:31:05.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.054+0000 7faa967fc700 1 -- 172.21.15.163:0/19443076 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa8800bb60 con 0x7faa98159890 2023-10-09T06:31:05.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.054+0000 7faa967fc700 1 -- 172.21.15.163:0/19443076 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa88012430 con 0x7faa98159890 2023-10-09T06:31:05.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.054+0000 7faa9dbc6700 1 -- 172.21.15.163:0/19443076 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa98159890 msgr2=0x7faa9815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:05.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.054+0000 7faa9dbc6700 1 --2- 172.21.15.163:0/19443076 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa98159890 0x7faa9815abd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7faa980b67f0 tx=0x7faa88009780 comp rx=0 tx=0).stop 2023-10-09T06:31:05.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.054+0000 7faa9dbc6700 1 -- 172.21.15.163:0/19443076 shutdown_connections 2023-10-09T06:31:05.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.054+0000 7faa9dbc6700 1 --2- 172.21.15.163:0/19443076 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa98159890 0x7faa9815abd0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:05.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.054+0000 7faa9dbc6700 1 --2- 172.21.15.163:0/19443076 >> v2:172.21.15.181:3300/0 conn(0x7faa98159c60 0x7faa9815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:05.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.054+0000 7faa9dbc6700 1 -- 172.21.15.163:0/19443076 >> 172.21.15.163:0/19443076 conn(0x7faa98068e20 msgr2=0x7faa980d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:05.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.054+0000 7faa9dbc6700 1 -- 172.21.15.163:0/19443076 shutdown_connections 2023-10-09T06:31:05.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.054+0000 7faa9dbc6700 1 -- 172.21.15.163:0/19443076 wait complete. 2023-10-09T06:31:05.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.056+0000 7faa9dbc6700 1 Processor -- start 2023-10-09T06:31:05.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.056+0000 7faa9dbc6700 1 -- start start 2023-10-09T06:31:05.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.056+0000 7faa9dbc6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa98159890 0x7faa980c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:05.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.056+0000 7faa9dbc6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa98159c60 0x7faa981576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:05.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.057+0000 7faa9dbc6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa9815db10 0x7faa980c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:05.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.057+0000 7faa9dbc6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa980c91b0 con 0x7faa98159c60 2023-10-09T06:31:05.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.057+0000 7faa9dbc6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9816a480 con 0x7faa98159890 2023-10-09T06:31:05.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.057+0000 7faa9dbc6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa980c8ed0 con 0x7faa9815db10 2023-10-09T06:31:05.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.057+0000 7faa96ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa98159c60 0x7faa981576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:05.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.057+0000 7faa96ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa98159c60 0x7faa981576a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35700/0 (socket says 172.21.15.163:35700) 2023-10-09T06:31:05.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.057+0000 7faa96ffd700 1 -- 172.21.15.163:0/4187243503 learned_addr learned my addr 172.21.15.163:0/4187243503 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:05.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.057+0000 7faa977fe700 1 --2- 172.21.15.163:0/4187243503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa98159890 0x7faa980c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:05.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.057+0000 7faa97fff700 1 --2- 172.21.15.163:0/4187243503 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa9815db10 0x7faa980c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:05.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.057+0000 7faa96ffd700 1 -- 172.21.15.163:0/4187243503 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa9815db10 msgr2=0x7faa980c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:05.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.057+0000 7faa96ffd700 1 --2- 172.21.15.163:0/4187243503 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa9815db10 0x7faa980c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:05.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.057+0000 7faa96ffd700 1 -- 172.21.15.163:0/4187243503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa98159890 msgr2=0x7faa980c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:05.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.057+0000 7faa96ffd700 1 --2- 172.21.15.163:0/4187243503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa98159890 0x7faa980c8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:05.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.057+0000 7faa96ffd700 1 -- 172.21.15.163:0/4187243503 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa88009a20 con 0x7faa98159c60 2023-10-09T06:31:05.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.057+0000 7faa96ffd700 1 --2- 172.21.15.163:0/4187243503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa98159c60 0x7faa981576a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00ecb0 tx=0x7faa8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:05.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.057+0000 7faa7f7fe700 1 -- 172.21.15.163:0/4187243503 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8c00c850 con 0x7faa98159c60 2023-10-09T06:31:05.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.057+0000 7faa7f7fe700 1 -- 172.21.15.163:0/4187243503 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa8c00c9b0 con 0x7faa98159c60 2023-10-09T06:31:05.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.057+0000 7faa7f7fe700 1 -- 172.21.15.163:0/4187243503 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8c00cd50 con 0x7faa98159c60 2023-10-09T06:31:05.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.057+0000 7faa9dbc6700 1 -- 172.21.15.163:0/4187243503 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa98169e30 con 0x7faa98159c60 2023-10-09T06:31:05.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.057+0000 7faa9dbc6700 1 -- 172.21.15.163:0/4187243503 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa980c8bc0 con 0x7faa98159c60 2023-10-09T06:31:05.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.058+0000 7faa9dbc6700 1 -- 172.21.15.163:0/4187243503 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa9816a610 con 0x7faa98159c60 2023-10-09T06:31:05.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.059+0000 7faa7f7fe700 1 -- 172.21.15.163:0/4187243503 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faa8c013680 con 0x7faa98159c60 2023-10-09T06:31:05.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.059+0000 7faa7f7fe700 1 --2- 172.21.15.163:0/4187243503 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faa84066f30 0x7faa840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:05.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.059+0000 7faa7f7fe700 1 -- 172.21.15.163:0/4187243503 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7faa8c084790 con 0x7faa98159c60 2023-10-09T06:31:05.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.059+0000 7faa977fe700 1 --2- 172.21.15.163:0/4187243503 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faa84066f30 0x7faa840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:05.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.060+0000 7faa977fe700 1 --2- 172.21.15.163:0/4187243503 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faa84066f30 0x7faa840693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7faa880097c0 tx=0x7faa88019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:05.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.063+0000 7faa7f7fe700 1 -- 172.21.15.163:0/4187243503 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa8c051a00 con 0x7faa98159c60 2023-10-09T06:31:05.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.289+0000 7faa9dbc6700 1 -- 172.21.15.163:0/4187243503 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7faa9804c3c0 con 0x7faa98159c60 2023-10-09T06:31:05.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.289+0000 7faa7f7fe700 1 -- 172.21.15.163:0/4187243503 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7faa8c055020 con 0x7faa98159c60 2023-10-09T06:31:05.291 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-09T06:31:05.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.291+0000 7faa9dbc6700 1 -- 172.21.15.163:0/4187243503 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faa84066f30 msgr2=0x7faa840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:05.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.291+0000 7faa9dbc6700 1 --2- 172.21.15.163:0/4187243503 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faa84066f30 0x7faa840693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7faa880097c0 tx=0x7faa88019000 comp rx=0 tx=0).stop 2023-10-09T06:31:05.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.291+0000 7faa9dbc6700 1 -- 172.21.15.163:0/4187243503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa98159c60 msgr2=0x7faa981576a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:05.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.291+0000 7faa9dbc6700 1 --2- 172.21.15.163:0/4187243503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa98159c60 0x7faa981576a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00ecb0 tx=0x7faa8c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:05.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.291+0000 7faa9dbc6700 1 -- 172.21.15.163:0/4187243503 shutdown_connections 2023-10-09T06:31:05.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.291+0000 7faa9dbc6700 1 --2- 172.21.15.163:0/4187243503 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faa84066f30 0x7faa840693e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:05.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.291+0000 7faa9dbc6700 1 --2- 172.21.15.163:0/4187243503 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa9815db10 0x7faa980c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:05.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.291+0000 7faa9dbc6700 1 --2- 172.21.15.163:0/4187243503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa98159890 0x7faa980c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:05.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.291+0000 7faa9dbc6700 1 --2- 172.21.15.163:0/4187243503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa98159c60 0x7faa981576a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:05.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.291+0000 7faa9dbc6700 1 -- 172.21.15.163:0/4187243503 >> 172.21.15.163:0/4187243503 conn(0x7faa98068e20 msgr2=0x7faa98153060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:05.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.291+0000 7faa9dbc6700 1 -- 172.21.15.163:0/4187243503 shutdown_connections 2023-10-09T06:31:05.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:05.291+0000 7faa9dbc6700 1 -- 172.21.15.163:0/4187243503 wait complete. 2023-10-09T06:31:05.304 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-09T06:31:05.304 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-09T06:31:05.305 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-09T06:31:05.305 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-09T06:31:05.305 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-09T06:31:05.305 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-09T06:31:05.305 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-09T06:31:05.305 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-09T06:31:05.305 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-09T06:31:05.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:05.305 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-09T06:31:05.321 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:31:05.322 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:31:05.322 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:31:05.322 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:05.322 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-09T06:31:05.322 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:05.322 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:31:05.322 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:31:05.322 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2023-10-09T06:31:05.323 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:31:05.323 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:31:05.323 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2023-10-09T06:31:05.323 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:31:05.323 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-09T06:31:05.323 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-09T06:31:05.323 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-09T06:31:05.323 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-09T06:31:05.323 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42435sec preferred_lft 42435sec 2023-10-09T06:31:05.325 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2023-10-09T06:31:05.325 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:05.325 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:31:05.326 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 72:62:e6:72:7c:2e brd ff:ff:ff:ff:ff:ff 2023-10-09T06:31:05.326 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-09T06:31:05.326 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:05.326 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::e89a:4cff:fefb:7198/64 scope link 2023-10-09T06:31:05.326 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:05.326 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:31:05.326 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 72:62:e6:72:7c:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-09T06:31:05.326 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::7062:e6ff:fe72:7c2e/64 scope link tentative 2023-10-09T06:31:05.326 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:05.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:05.327 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-09T06:31:05.388 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-09T06:31:05.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:05.390 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-09T06:31:05.446 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-09T06:31:05.446 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-09T06:31:05.446 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-09T06:31:05.446 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-09T06:31:05.446 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-09T06:31:05.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:05.447 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-09T06:31:05.447 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-09T06:31:05.447 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-09T06:31:05.447 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-09T06:31:05.447 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-09T06:31:05.447 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-09T06:31:05.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:05.550 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-09T06:31:05.567 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-09T06:31:05.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:05.567 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list-id 2023-10-09T06:31:05.622 INFO:teuthology.orchestra.run.smithi181.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-09T06:31:05.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:05.623 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-09T06:31:05.623 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-09T06:31:05.623 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-09T06:31:05.623 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-09T06:31:05.623 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-09T06:31:05.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:05.713 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-09T06:31:05.757 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:31:05.757 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:31:05.758 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:31:05.758 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:05.758 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-09T06:31:05.758 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:05.759 INFO:teuthology.orchestra.run.smithi181.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-09T06:31:05.759 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 62:fe:99:36:c0:9a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-09T06:31:05.759 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-09T06:31:05.759 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:05.759 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::60fe:99ff:fe36:c09a/64 scope link tentative 2023-10-09T06:31:05.760 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:05.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:05.767 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-09T06:31:05.806 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:31:05.806 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:31:05.806 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:31:05.807 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:05.807 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-09T06:31:05.807 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:05.807 INFO:teuthology.orchestra.run.smithi181.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-09T06:31:05.807 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 62:fe:99:36:c0:9a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-09T06:31:05.807 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-09T06:31:05.807 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:05.807 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::60fe:99ff:fe36:c09a/64 scope link tentative 2023-10-09T06:31:05.807 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:05.824 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-09T06:31:05.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:05.825 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-09T06:31:05.825 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-09T06:31:05.826 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-09T06:31:05.826 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-09T06:31:05.826 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-09T06:31:05.826 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-09T06:31:05.826 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-09T06:31:05.827 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-09T06:31:06.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:06.075 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-09T06:31:06.075 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-09T06:31:06.075 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.1 up 2023-10-09T06:31:06.075 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set dev brx.1 master ceph-brx 2023-10-09T06:31:06.075 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-09T06:31:06.155 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:31:06.155 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-09T06:31:06.170 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-09T06:31:06.170 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:31:06.170 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-09T06:31:06.224 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-10-09T06:31:06.224 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-09T06:31:06.224 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:31:06.304 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:31:06.304 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:31:06.323 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:31:06.323 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:31:06.323 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:31:06.323 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 3@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:31:06.323 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:31:06.323 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:31:06.323 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:31:06.323 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:31:06.345 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:31:06.345 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:31:06.346 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:31:06.346 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 3@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:31:06.346 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:31:06.346 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:31:06.346 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:31:06.346 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:31:06.346 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790" 2023-10-09T06:31:06.346 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790:/ 2023-10-09T06:31:06.346 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,key=3,mds_namespace=cephfs,fsid=2520b8f7-539f-4bae-9f17-1fce66f66ddd" will pass to kernel 2023-10-09T06:31:06.350 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-09T06:31:06.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:06.350 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-09T06:31:06.414 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-09T06:31:06.436 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:31:06.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.565+0000 7f6a2675a700 1 Processor -- start 2023-10-09T06:31:06.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.565+0000 7f6a2675a700 1 -- start start 2023-10-09T06:31:06.569 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.566+0000 7f6a2675a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6a2006c0f0 0x7f6a20159c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:06.569 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.566+0000 7f6a2675a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6a200b3810 0x7f6a200b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:06.569 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.566+0000 7f6a2675a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a20143420 con 0x7f6a20143050 2023-10-09T06:31:06.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.566+0000 7f6a2675a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a2006a1e0 con 0x7f6a200b3810 2023-10-09T06:31:06.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.566+0000 7f6a2675a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a200695d0 con 0x7f6a2006c0f0 2023-10-09T06:31:06.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.566+0000 7f6a25f59700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6a200b3810 0x7f6a200b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:06.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.566+0000 7f6a24f57700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6a2006c0f0 0x7f6a20159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:06.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.566+0000 7f6a25f59700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6a200b3810 0x7f6a200b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35702/0 (socket says 172.21.15.163:35702) 2023-10-09T06:31:06.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.566+0000 7f6a25f59700 1 -- 172.21.15.163:0/3863735441 learned_addr learned my addr 172.21.15.163:0/3863735441 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:06.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.566+0000 7f6a25758700 1 --1- 172.21.15.163:0/3863735441 >> v1:172.21.15.181:6789/0 conn(0x7f6a20143050 0x7f6a2006b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:52290/0 (socket says 172.21.15.163:52290) 2023-10-09T06:31:06.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.567+0000 7f6a25f59700 1 -- 172.21.15.163:0/3863735441 >> v2:172.21.15.181:3300/0 conn(0x7f6a2006c0f0 msgr2=0x7f6a20159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:06.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.567+0000 7f6a25f59700 1 --2- 172.21.15.163:0/3863735441 >> v2:172.21.15.181:3300/0 conn(0x7f6a2006c0f0 0x7f6a20159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:06.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.567+0000 7f6a25f59700 1 -- 172.21.15.163:0/3863735441 >> v1:172.21.15.181:6789/0 conn(0x7f6a20143050 legacy=0x7f6a2006b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:06.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.567+0000 7f6a25f59700 1 -- 172.21.15.163:0/3863735441 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a200698b0 con 0x7f6a200b3810 2023-10-09T06:31:06.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.567+0000 7f6a17fff700 1 -- 172.21.15.163:0/3863735441 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2175039797 0 0) 0x7f6a20143420 con 0x7f6a20143050 2023-10-09T06:31:06.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.567+0000 7f6a24f57700 1 --2- 172.21.15.163:0/3863735441 >> v2:172.21.15.181:3300/0 conn(0x7f6a2006c0f0 0x7f6a20159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:31:06.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.567+0000 7f6a25f59700 1 --2- 172.21.15.163:0/3863735441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a200b3810 0x7f6a200b3c30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6a2006ca20 tx=0x7f6a18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fddbd2345e1f7735 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:06.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.567+0000 7f6a17fff700 1 -- 172.21.15.163:0/3863735441 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a18009e70 con 0x7f6a200b3810 2023-10-09T06:31:06.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.567+0000 7f6a17fff700 1 -- 172.21.15.163:0/3863735441 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1800b860 con 0x7f6a200b3810 2023-10-09T06:31:06.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.567+0000 7f6a17fff700 1 -- 172.21.15.163:0/3863735441 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1800bc20 con 0x7f6a200b3810 2023-10-09T06:31:06.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.567+0000 7f6a2675a700 1 -- 172.21.15.163:0/3863735441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a200b3810 msgr2=0x7f6a200b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:06.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.567+0000 7f6a2675a700 1 --2- 172.21.15.163:0/3863735441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a200b3810 0x7f6a200b3c30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6a2006ca20 tx=0x7f6a18009250 comp rx=0 tx=0).stop 2023-10-09T06:31:06.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.568+0000 7f6a2675a700 1 -- 172.21.15.163:0/3863735441 shutdown_connections 2023-10-09T06:31:06.572 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.568+0000 7f6a2675a700 1 --2- 172.21.15.163:0/3863735441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a200b3810 0x7f6a200b3c30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:06.572 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.568+0000 7f6a2675a700 1 --2- 172.21.15.163:0/3863735441 >> v2:172.21.15.181:3300/0 conn(0x7f6a2006c0f0 0x7f6a20159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:06.572 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.568+0000 7f6a2675a700 1 -- 172.21.15.163:0/3863735441 >> 172.21.15.163:0/3863735441 conn(0x7f6a200ce430 msgr2=0x7f6a200ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:06.572 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.568+0000 7f6a2675a700 1 -- 172.21.15.163:0/3863735441 shutdown_connections 2023-10-09T06:31:06.572 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.568+0000 7f6a2675a700 1 -- 172.21.15.163:0/3863735441 wait complete. 2023-10-09T06:31:06.572 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.569+0000 7f6a2675a700 1 Processor -- start 2023-10-09T06:31:06.572 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.569+0000 7f6a2675a700 1 -- start start 2023-10-09T06:31:06.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.570+0000 7f6a2675a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a2006c0f0 0x7f6a2013b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:06.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.570+0000 7f6a2675a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a200b3810 0x7f6a2013b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:06.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.570+0000 7f6a2675a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a20143050 0x7f6a2013e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:06.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.570+0000 7f6a2675a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2015a2b0 con 0x7f6a2006c0f0 2023-10-09T06:31:06.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.570+0000 7f6a2675a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2013f280 con 0x7f6a200b3810 2023-10-09T06:31:06.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.570+0000 7f6a2675a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2013f560 con 0x7f6a20143050 2023-10-09T06:31:06.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.570+0000 7f6a25758700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a2006c0f0 0x7f6a2013b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:06.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.570+0000 7f6a25758700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a2006c0f0 0x7f6a2013b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35716/0 (socket says 172.21.15.163:35716) 2023-10-09T06:31:06.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.570+0000 7f6a25758700 1 -- 172.21.15.163:0/1569641852 learned_addr learned my addr 172.21.15.163:0/1569641852 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:06.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.570+0000 7f6a24f57700 1 --2- 172.21.15.163:0/1569641852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a200b3810 0x7f6a2013b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:06.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.570+0000 7f6a25f59700 1 --2- 172.21.15.163:0/1569641852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a20143050 0x7f6a2013e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:06.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.570+0000 7f6a25758700 1 -- 172.21.15.163:0/1569641852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a20143050 msgr2=0x7f6a2013e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:06.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.570+0000 7f6a25758700 1 --2- 172.21.15.163:0/1569641852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a20143050 0x7f6a2013e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:06.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.570+0000 7f6a25758700 1 -- 172.21.15.163:0/1569641852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a200b3810 msgr2=0x7f6a2013b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:06.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.570+0000 7f6a25758700 1 --2- 172.21.15.163:0/1569641852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a200b3810 0x7f6a2013b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:06.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.570+0000 7f6a25758700 1 -- 172.21.15.163:0/1569641852 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a10009420 con 0x7f6a2006c0f0 2023-10-09T06:31:06.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.570+0000 7f6a25758700 1 --2- 172.21.15.163:0/1569641852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a2006c0f0 0x7f6a2013b2a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c00cc90 tx=0x7f6a1c001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:06.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.570+0000 7f6a157fa700 1 -- 172.21.15.163:0/1569641852 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c00a8b0 con 0x7f6a2006c0f0 2023-10-09T06:31:06.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.570+0000 7f6a157fa700 1 -- 172.21.15.163:0/1569641852 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1c00aa10 con 0x7f6a2006c0f0 2023-10-09T06:31:06.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.570+0000 7f6a157fa700 1 -- 172.21.15.163:0/1569641852 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c00adb0 con 0x7f6a2006c0f0 2023-10-09T06:31:06.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.570+0000 7f6a2675a700 1 -- 172.21.15.163:0/1569641852 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a180094f0 con 0x7f6a2006c0f0 2023-10-09T06:31:06.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.570+0000 7f6a2675a700 1 -- 172.21.15.163:0/1569641852 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a2013f840 con 0x7f6a2006c0f0 2023-10-09T06:31:06.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.571+0000 7f6a2675a700 1 -- 172.21.15.163:0/1569641852 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a2004c3c0 con 0x7f6a2006c0f0 2023-10-09T06:31:06.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.572+0000 7f6a157fa700 1 -- 172.21.15.163:0/1569641852 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6a1c007300 con 0x7f6a2006c0f0 2023-10-09T06:31:06.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.572+0000 7f6a157fa700 1 --2- 172.21.15.163:0/1569641852 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6a04066f30 0x7f6a040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:06.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.572+0000 7f6a157fa700 1 -- 172.21.15.163:0/1569641852 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6a1c0848a0 con 0x7f6a2006c0f0 2023-10-09T06:31:06.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.572+0000 7f6a24f57700 1 --2- 172.21.15.163:0/1569641852 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6a04066f30 0x7f6a040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:06.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.573+0000 7f6a24f57700 1 --2- 172.21.15.163:0/1569641852 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6a04066f30 0x7f6a040693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6a2015fb30 tx=0x7f6a100091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:06.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.576+0000 7f6a157fa700 1 -- 172.21.15.163:0/1569641852 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6a1c051b90 con 0x7f6a2006c0f0 2023-10-09T06:31:06.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.759+0000 7f6a2675a700 1 -- 172.21.15.163:0/1569641852 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a20069b90 con 0x7f6a2006c0f0 2023-10-09T06:31:06.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.760+0000 7f6a157fa700 1 -- 172.21.15.163:0/1569641852 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f6a1c0551b0 con 0x7f6a2006c0f0 2023-10-09T06:31:06.763 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:06.763 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":20,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:30:44.466923+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:30:41.369014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-09T06:30:42.103001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:31:06.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.762+0000 7f6a2675a700 1 -- 172.21.15.163:0/1569641852 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6a04066f30 msgr2=0x7f6a040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:06.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.762+0000 7f6a2675a700 1 --2- 172.21.15.163:0/1569641852 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6a04066f30 0x7f6a040693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6a2015fb30 tx=0x7f6a100091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:06.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.762+0000 7f6a2675a700 1 -- 172.21.15.163:0/1569641852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a2006c0f0 msgr2=0x7f6a2013b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:06.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.762+0000 7f6a2675a700 1 --2- 172.21.15.163:0/1569641852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a2006c0f0 0x7f6a2013b2a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c00cc90 tx=0x7f6a1c001aa0 comp rx=0 tx=0).stop 2023-10-09T06:31:06.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.762+0000 7f6a2675a700 1 -- 172.21.15.163:0/1569641852 shutdown_connections 2023-10-09T06:31:06.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.762+0000 7f6a2675a700 1 --2- 172.21.15.163:0/1569641852 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6a04066f30 0x7f6a040693e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:06.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.762+0000 7f6a2675a700 1 --2- 172.21.15.163:0/1569641852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a2006c0f0 0x7f6a2013b2a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:06.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.762+0000 7f6a2675a700 1 --2- 172.21.15.163:0/1569641852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a20143050 0x7f6a2013e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:06.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.762+0000 7f6a2675a700 1 --2- 172.21.15.163:0/1569641852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a200b3810 0x7f6a2013b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:06.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.762+0000 7f6a2675a700 1 -- 172.21.15.163:0/1569641852 >> 172.21.15.163:0/1569641852 conn(0x7f6a200ce430 msgr2=0x7f6a200adda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:06.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.762+0000 7f6a2675a700 1 -- 172.21.15.163:0/1569641852 shutdown_connections 2023-10-09T06:31:06.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.762+0000 7f6a2675a700 1 -- 172.21.15.163:0/1569641852 wait complete. 2023-10-09T06:31:06.776 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-09T06:30:41.369014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-09T06:30:42.103001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-09T06:31:06.777 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:31:06.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.892+0000 7fd0b071f700 1 Processor -- start 2023-10-09T06:31:06.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.892+0000 7fd0b071f700 1 -- start start 2023-10-09T06:31:06.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.893+0000 7fd0b071f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd0a8159890 0x7fd0a815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:06.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.893+0000 7fd0b071f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0a8159c60 0x7fd0a815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:06.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.893+0000 7fd0b071f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0a815a160 con 0x7fd0a815db10 2023-10-09T06:31:06.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.893+0000 7fd0b071f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0a80d13a0 con 0x7fd0a8159890 2023-10-09T06:31:06.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.893+0000 7fd0b071f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a815e140 con 0x7fd0a8159c60 2023-10-09T06:31:06.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.894+0000 7fd0ae4bb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd0a8159890 0x7fd0a815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:06.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.894+0000 7fd0adcba700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0a8159c60 0x7fd0a815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:06.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.894+0000 7fd0aecbc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd0a815db10 0x7fd0a8069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:52294/0 (socket says 172.21.15.163:52294) 2023-10-09T06:31:06.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.894+0000 7fd0aecbc700 1 -- 172.21.15.163:0/204974648 learned_addr learned my addr 172.21.15.163:0/204974648 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:06.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.894+0000 7fd0ae4bb700 1 -- 172.21.15.163:0/204974648 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0a8159c60 msgr2=0x7fd0a815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:06.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.894+0000 7fd0ae4bb700 1 --2- 172.21.15.163:0/204974648 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0a8159c60 0x7fd0a815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:06.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.894+0000 7fd0ae4bb700 1 -- 172.21.15.163:0/204974648 >> v1:172.21.15.181:6789/0 conn(0x7fd0a815db10 legacy=0x7fd0a8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:06.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.894+0000 7fd0ae4bb700 1 -- 172.21.15.163:0/204974648 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a815a3d0 con 0x7fd0a8159890 2023-10-09T06:31:06.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.894+0000 7fd0ad4b9700 1 -- 172.21.15.163:0/204974648 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3114493442 0 0) 0x7fd0a815a160 con 0x7fd0a815db10 2023-10-09T06:31:06.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.894+0000 7fd0ae4bb700 1 --2- 172.21.15.163:0/204974648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0a8159890 0x7fd0a815abd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd098009e60 tx=0x7fd098009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8927e3cb8f1b538e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:06.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.894+0000 7fd0ad4b9700 1 -- 172.21.15.163:0/204974648 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09800e070 con 0x7fd0a8159890 2023-10-09T06:31:06.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.894+0000 7fd0ad4b9700 1 -- 172.21.15.163:0/204974648 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd09800bb60 con 0x7fd0a8159890 2023-10-09T06:31:06.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.894+0000 7fd0ad4b9700 1 -- 172.21.15.163:0/204974648 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd098012430 con 0x7fd0a8159890 2023-10-09T06:31:06.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.895+0000 7fd0b071f700 1 -- 172.21.15.163:0/204974648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0a8159890 msgr2=0x7fd0a815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:06.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.895+0000 7fd0b071f700 1 --2- 172.21.15.163:0/204974648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0a8159890 0x7fd0a815abd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd098009e60 tx=0x7fd098009780 comp rx=0 tx=0).stop 2023-10-09T06:31:06.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.895+0000 7fd0b071f700 1 -- 172.21.15.163:0/204974648 shutdown_connections 2023-10-09T06:31:06.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.895+0000 7fd0b071f700 1 --2- 172.21.15.163:0/204974648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0a8159890 0x7fd0a815abd0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:06.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.895+0000 7fd0b071f700 1 --2- 172.21.15.163:0/204974648 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0a8159c60 0x7fd0a815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:06.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.895+0000 7fd0b071f700 1 -- 172.21.15.163:0/204974648 >> 172.21.15.163:0/204974648 conn(0x7fd0a8068e20 msgr2=0x7fd0a80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:06.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.895+0000 7fd0b071f700 1 -- 172.21.15.163:0/204974648 shutdown_connections 2023-10-09T06:31:06.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.895+0000 7fd0b071f700 1 -- 172.21.15.163:0/204974648 wait complete. 2023-10-09T06:31:06.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.896+0000 7fd0b071f700 1 Processor -- start 2023-10-09T06:31:06.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.896+0000 7fd0b071f700 1 -- start start 2023-10-09T06:31:06.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.897+0000 7fd0b071f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0a8159890 0x7fd0a80b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:06.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.897+0000 7fd0b071f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0a8159c60 0x7fd0a80cf320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:06.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.897+0000 7fd0b071f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0a815db10 0x7fd0a816d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:06.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.897+0000 7fd0b071f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a80b46c0 con 0x7fd0a8159890 2023-10-09T06:31:06.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.897+0000 7fd0b071f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a80b4100 con 0x7fd0a8159c60 2023-10-09T06:31:06.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.897+0000 7fd0b071f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a80b43e0 con 0x7fd0a815db10 2023-10-09T06:31:06.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.897+0000 7fd0ae4bb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0a8159890 0x7fd0a80b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:06.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.897+0000 7fd0ae4bb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0a8159890 0x7fd0a80b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35722/0 (socket says 172.21.15.163:35722) 2023-10-09T06:31:06.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.897+0000 7fd0ae4bb700 1 -- 172.21.15.163:0/342709793 learned_addr learned my addr 172.21.15.163:0/342709793 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:06.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.897+0000 7fd0adcba700 1 --2- 172.21.15.163:0/342709793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0a8159c60 0x7fd0a80cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:06.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.897+0000 7fd0aecbc700 1 --2- 172.21.15.163:0/342709793 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0a815db10 0x7fd0a816d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:06.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.897+0000 7fd0adcba700 1 -- 172.21.15.163:0/342709793 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0a815db10 msgr2=0x7fd0a816d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:06.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.897+0000 7fd0adcba700 1 --2- 172.21.15.163:0/342709793 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0a815db10 0x7fd0a816d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:06.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.897+0000 7fd0adcba700 1 -- 172.21.15.163:0/342709793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0a8159890 msgr2=0x7fd0a80b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:06.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.897+0000 7fd0adcba700 1 --2- 172.21.15.163:0/342709793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0a8159890 0x7fd0a80b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:06.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.897+0000 7fd0adcba700 1 -- 172.21.15.163:0/342709793 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd098009a20 con 0x7fd0a8159c60 2023-10-09T06:31:06.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.897+0000 7fd0adcba700 1 --2- 172.21.15.163:0/342709793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0a8159c60 0x7fd0a80cf320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00ea20 tx=0x7fd09c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:06.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.897+0000 7fd0a67fc700 1 -- 172.21.15.163:0/342709793 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c0095d0 con 0x7fd0a8159c60 2023-10-09T06:31:06.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.898+0000 7fd0a67fc700 1 -- 172.21.15.163:0/342709793 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd09c00c7f0 con 0x7fd0a8159c60 2023-10-09T06:31:06.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.898+0000 7fd0b071f700 1 -- 172.21.15.163:0/342709793 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a80c9a90 con 0x7fd0a8159c60 2023-10-09T06:31:06.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.898+0000 7fd0a67fc700 1 -- 172.21.15.163:0/342709793 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c00cbb0 con 0x7fd0a8159c60 2023-10-09T06:31:06.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.898+0000 7fd0b071f700 1 -- 172.21.15.163:0/342709793 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a80cf860 con 0x7fd0a8159c60 2023-10-09T06:31:06.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.898+0000 7fd0b071f700 1 -- 172.21.15.163:0/342709793 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0a80c9f30 con 0x7fd0a8159c60 2023-10-09T06:31:06.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.899+0000 7fd0a67fc700 1 -- 172.21.15.163:0/342709793 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd09c013470 con 0x7fd0a8159c60 2023-10-09T06:31:06.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.899+0000 7fd0a67fc700 1 --2- 172.21.15.163:0/342709793 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd094066f30 0x7fd0940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:06.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.899+0000 7fd0a67fc700 1 -- 172.21.15.163:0/342709793 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd09c0845b0 con 0x7fd0a8159c60 2023-10-09T06:31:06.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.900+0000 7fd0ae4bb700 1 --2- 172.21.15.163:0/342709793 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd094066f30 0x7fd0940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:06.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.900+0000 7fd0ae4bb700 1 --2- 172.21.15.163:0/342709793 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd094066f30 0x7fd0940693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd0a80ca480 tx=0x7fd098019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:06.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:06.905+0000 7fd0a67fc700 1 -- 172.21.15.163:0/342709793 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd09c051820 con 0x7fd0a8159c60 2023-10-09T06:31:07.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.085+0000 7fd0b071f700 1 -- 172.21.15.163:0/342709793 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd0a804c3c0 con 0x7fd0a8159c60 2023-10-09T06:31:07.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.085+0000 7fd0a67fc700 1 -- 172.21.15.163:0/342709793 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd09c054e40 con 0x7fd0a8159c60 2023-10-09T06:31:07.086 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:31:07.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.087+0000 7fd0b071f700 1 -- 172.21.15.163:0/342709793 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd094066f30 msgr2=0x7fd0940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:07.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.087+0000 7fd0b071f700 1 --2- 172.21.15.163:0/342709793 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd094066f30 0x7fd0940693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd0a80ca480 tx=0x7fd098019000 comp rx=0 tx=0).stop 2023-10-09T06:31:07.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.087+0000 7fd0b071f700 1 -- 172.21.15.163:0/342709793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0a8159c60 msgr2=0x7fd0a80cf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:07.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.087+0000 7fd0b071f700 1 --2- 172.21.15.163:0/342709793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0a8159c60 0x7fd0a80cf320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00ea20 tx=0x7fd09c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.088+0000 7fd0b071f700 1 -- 172.21.15.163:0/342709793 shutdown_connections 2023-10-09T06:31:07.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.088+0000 7fd0b071f700 1 --2- 172.21.15.163:0/342709793 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd094066f30 0x7fd0940693e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.088+0000 7fd0b071f700 1 --2- 172.21.15.163:0/342709793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0a8159890 0x7fd0a80b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.088+0000 7fd0b071f700 1 --2- 172.21.15.163:0/342709793 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0a815db10 0x7fd0a816d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.088+0000 7fd0b071f700 1 --2- 172.21.15.163:0/342709793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0a8159c60 0x7fd0a80cf320 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.088+0000 7fd0b071f700 1 -- 172.21.15.163:0/342709793 >> 172.21.15.163:0/342709793 conn(0x7fd0a8068e20 msgr2=0x7fd0a80b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:07.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.088+0000 7fd0b071f700 1 -- 172.21.15.163:0/342709793 shutdown_connections 2023-10-09T06:31:07.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.088+0000 7fd0b071f700 1 -- 172.21.15.163:0/342709793 wait complete. 2023-10-09T06:31:07.101 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-09T06:31:07.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.218+0000 7f0a9359e700 1 Processor -- start 2023-10-09T06:31:07.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.218+0000 7f0a9359e700 1 -- start start 2023-10-09T06:31:07.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.219+0000 7f0a9359e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0a940aef00 0x7f0a9415d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:07.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.219+0000 7f0a9359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a9406e470 0x7f0a9406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:07.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.219+0000 7f0a9359e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a940aea70 con 0x7f0a9415a3b0 2023-10-09T06:31:07.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.219+0000 7f0a9359e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a9406ee30 con 0x7f0a940aef00 2023-10-09T06:31:07.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.219+0000 7f0a9359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a940ae480 con 0x7f0a9406e470 2023-10-09T06:31:07.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.220+0000 7f0a92d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a9406e470 0x7f0a9406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:07.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.220+0000 7f0a91d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0a940aef00 0x7f0a9415d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:07.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.220+0000 7f0a91d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0a940aef00 0x7f0a9415d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:48446/0 (socket says 172.21.15.163:48446) 2023-10-09T06:31:07.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.220+0000 7f0a9259c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f0a9415a3b0 0x7f0a9415a780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:52308/0 (socket says 172.21.15.163:52308) 2023-10-09T06:31:07.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.220+0000 7f0a91d9b700 1 -- 172.21.15.163:0/4144872017 learned_addr learned my addr 172.21.15.163:0/4144872017 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:07.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.220+0000 7f0a91d9b700 1 -- 172.21.15.163:0/4144872017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a9406e470 msgr2=0x7f0a9406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:07.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.220+0000 7f0a91d9b700 1 --2- 172.21.15.163:0/4144872017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a9406e470 0x7f0a9406e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.220+0000 7f0a91d9b700 1 -- 172.21.15.163:0/4144872017 >> v1:172.21.15.181:6789/0 conn(0x7f0a9415a3b0 legacy=0x7f0a9415a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:07.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.220+0000 7f0a91d9b700 1 -- 172.21.15.163:0/4144872017 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9415dd30 con 0x7f0a940aef00 2023-10-09T06:31:07.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.220+0000 7f0a9159a700 1 -- 172.21.15.163:0/4144872017 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3707267151 0 0) 0x7f0a940aea70 con 0x7f0a9415a3b0 2023-10-09T06:31:07.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.220+0000 7f0a91d9b700 1 --2- 172.21.15.163:0/4144872017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a940aef00 0x7f0a9415d3c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0a880099b0 tx=0x7f0a88009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=531365ed70c55af7 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:07.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.220+0000 7f0a9159a700 1 -- 172.21.15.163:0/4144872017 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a88009ea0 con 0x7f0a940aef00 2023-10-09T06:31:07.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.220+0000 7f0a9159a700 1 -- 172.21.15.163:0/4144872017 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8800b860 con 0x7f0a940aef00 2023-10-09T06:31:07.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.220+0000 7f0a9159a700 1 -- 172.21.15.163:0/4144872017 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8800bc00 con 0x7f0a940aef00 2023-10-09T06:31:07.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.221+0000 7f0a9359e700 1 -- 172.21.15.163:0/4144872017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a940aef00 msgr2=0x7f0a9415d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:07.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.221+0000 7f0a9359e700 1 --2- 172.21.15.163:0/4144872017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a940aef00 0x7f0a9415d3c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0a880099b0 tx=0x7f0a88009250 comp rx=0 tx=0).stop 2023-10-09T06:31:07.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.221+0000 7f0a9359e700 1 -- 172.21.15.163:0/4144872017 shutdown_connections 2023-10-09T06:31:07.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.221+0000 7f0a9359e700 1 --2- 172.21.15.163:0/4144872017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a9406e470 0x7f0a9406e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.221+0000 7f0a9359e700 1 --2- 172.21.15.163:0/4144872017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a940aef00 0x7f0a9415d3c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.221+0000 7f0a9359e700 1 -- 172.21.15.163:0/4144872017 >> 172.21.15.163:0/4144872017 conn(0x7f0a940c9f70 msgr2=0x7f0a9406dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:07.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.221+0000 7f0a9359e700 1 -- 172.21.15.163:0/4144872017 shutdown_connections 2023-10-09T06:31:07.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.221+0000 7f0a9359e700 1 -- 172.21.15.163:0/4144872017 wait complete. 2023-10-09T06:31:07.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.222+0000 7f0a9359e700 1 Processor -- start 2023-10-09T06:31:07.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.222+0000 7f0a9359e700 1 -- start start 2023-10-09T06:31:07.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.223+0000 7f0a9359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a9406e470 0x7f0a94169d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:07.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.223+0000 7f0a9359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a940aef00 0x7f0a9414bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:07.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.223+0000 7f0a9359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a9415a3b0 0x7f0a940c2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:07.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.223+0000 7f0a9359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9416acc0 con 0x7f0a9406e470 2023-10-09T06:31:07.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.223+0000 7f0a9359e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9416d570 con 0x7f0a940aef00 2023-10-09T06:31:07.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.223+0000 7f0a9359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9416a420 con 0x7f0a9415a3b0 2023-10-09T06:31:07.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.223+0000 7f0a9259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a9406e470 0x7f0a94169d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:07.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.223+0000 7f0a9259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a9406e470 0x7f0a94169d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35730/0 (socket says 172.21.15.163:35730) 2023-10-09T06:31:07.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.223+0000 7f0a9259c700 1 -- 172.21.15.163:0/3426624285 learned_addr learned my addr 172.21.15.163:0/3426624285 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:07.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.223+0000 7f0a92d9d700 1 --2- 172.21.15.163:0/3426624285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a9415a3b0 0x7f0a940c2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:07.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.223+0000 7f0a91d9b700 1 --2- 172.21.15.163:0/3426624285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a940aef00 0x7f0a9414bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:07.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.223+0000 7f0a9259c700 1 -- 172.21.15.163:0/3426624285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a9415a3b0 msgr2=0x7f0a940c2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:07.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.223+0000 7f0a9259c700 1 --2- 172.21.15.163:0/3426624285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a9415a3b0 0x7f0a940c2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.223+0000 7f0a9259c700 1 -- 172.21.15.163:0/3426624285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a940aef00 msgr2=0x7f0a9414bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:07.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.223+0000 7f0a9259c700 1 --2- 172.21.15.163:0/3426624285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a940aef00 0x7f0a9414bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.223+0000 7f0a9259c700 1 -- 172.21.15.163:0/3426624285 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a880094f0 con 0x7f0a9406e470 2023-10-09T06:31:07.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.223+0000 7f0a91d9b700 1 --2- 172.21.15.163:0/3426624285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a940aef00 0x7f0a9414bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:07.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.224+0000 7f0a9259c700 1 --2- 172.21.15.163:0/3426624285 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a9406e470 0x7f0a94169d40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400cd30 tx=0x7f0a84001af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:07.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.224+0000 7f0a827fc700 1 -- 172.21.15.163:0/3426624285 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84005df0 con 0x7f0a9406e470 2023-10-09T06:31:07.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.224+0000 7f0a827fc700 1 -- 172.21.15.163:0/3426624285 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8400a960 con 0x7f0a9406e470 2023-10-09T06:31:07.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.224+0000 7f0a9359e700 1 -- 172.21.15.163:0/3426624285 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a9416a700 con 0x7f0a9406e470 2023-10-09T06:31:07.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.224+0000 7f0a827fc700 1 -- 172.21.15.163:0/3426624285 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8400ad20 con 0x7f0a9406e470 2023-10-09T06:31:07.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.224+0000 7f0a9359e700 1 -- 172.21.15.163:0/3426624285 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a9414c430 con 0x7f0a9406e470 2023-10-09T06:31:07.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.225+0000 7f0a9359e700 1 -- 172.21.15.163:0/3426624285 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a9416d340 con 0x7f0a9406e470 2023-10-09T06:31:07.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.225+0000 7f0a827fc700 1 -- 172.21.15.163:0/3426624285 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0a84007300 con 0x7f0a9406e470 2023-10-09T06:31:07.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.225+0000 7f0a827fc700 1 --2- 172.21.15.163:0/3426624285 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0a74066f30 0x7f0a740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:07.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.225+0000 7f0a827fc700 1 -- 172.21.15.163:0/3426624285 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0a84084700 con 0x7f0a9406e470 2023-10-09T06:31:07.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.225+0000 7f0a91d9b700 1 --2- 172.21.15.163:0/3426624285 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0a74066f30 0x7f0a740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:07.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.226+0000 7f0a91d9b700 1 --2- 172.21.15.163:0/3426624285 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0a74066f30 0x7f0a740693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0a880099b0 tx=0x7f0a88017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:07.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.229+0000 7f0a827fc700 1 -- 172.21.15.163:0/3426624285 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a9416d340 con 0x7f0a9406e470 2023-10-09T06:31:07.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.413+0000 7f0a9359e700 1 -- 172.21.15.163:0/3426624285 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0a9416d340 con 0x7f0a9406e470 2023-10-09T06:31:07.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.414+0000 7f0a827fc700 1 -- 172.21.15.163:0/3426624285 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0a9416d340 con 0x7f0a9406e470 2023-10-09T06:31:07.415 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 32 2023-10-09T06:31:07.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.416+0000 7f0a9359e700 1 -- 172.21.15.163:0/3426624285 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0a74066f30 msgr2=0x7f0a740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:07.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.416+0000 7f0a9359e700 1 --2- 172.21.15.163:0/3426624285 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0a74066f30 0x7f0a740693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0a880099b0 tx=0x7f0a88017000 comp rx=0 tx=0).stop 2023-10-09T06:31:07.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.416+0000 7f0a9359e700 1 -- 172.21.15.163:0/3426624285 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a9406e470 msgr2=0x7f0a94169d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:07.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.416+0000 7f0a9359e700 1 --2- 172.21.15.163:0/3426624285 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a9406e470 0x7f0a94169d40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400cd30 tx=0x7f0a84001af0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.416+0000 7f0a9359e700 1 -- 172.21.15.163:0/3426624285 shutdown_connections 2023-10-09T06:31:07.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.416+0000 7f0a9359e700 1 --2- 172.21.15.163:0/3426624285 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0a74066f30 0x7f0a740693e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.416+0000 7f0a9359e700 1 --2- 172.21.15.163:0/3426624285 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a9406e470 0x7f0a94169d40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.416+0000 7f0a9359e700 1 --2- 172.21.15.163:0/3426624285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a9415a3b0 0x7f0a940c2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.416+0000 7f0a9359e700 1 --2- 172.21.15.163:0/3426624285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a940aef00 0x7f0a9414bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.416+0000 7f0a9359e700 1 -- 172.21.15.163:0/3426624285 >> 172.21.15.163:0/3426624285 conn(0x7f0a940c9f70 msgr2=0x7f0a9414b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:07.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.416+0000 7f0a9359e700 1 -- 172.21.15.163:0/3426624285 shutdown_connections 2023-10-09T06:31:07.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.416+0000 7f0a9359e700 1 -- 172.21.15.163:0/3426624285 wait complete. 2023-10-09T06:31:07.429 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-09T06:31:07.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.548+0000 7f8dc82c6700 1 Processor -- start 2023-10-09T06:31:07.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.548+0000 7f8dc82c6700 1 -- start start 2023-10-09T06:31:07.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.549+0000 7f8dc82c6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8dc00af250 0x7f8dc0068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:07.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.549+0000 7f8dc82c6700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8dc00688d0 0x7f8dc0157d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:07.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.549+0000 7f8dc82c6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc006b030 con 0x7f8dc0157960 2023-10-09T06:31:07.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.549+0000 7f8dc82c6700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dc00ac760 con 0x7f8dc00688d0 2023-10-09T06:31:07.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.549+0000 7f8dc82c6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc00af830 con 0x7f8dc00af250 2023-10-09T06:31:07.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.549+0000 7f8dc6863700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8dc00688d0 0x7f8dc0157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:07.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.549+0000 7f8dc6863700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8dc00688d0 0x7f8dc0157d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35746/0 (socket says 172.21.15.163:35746) 2023-10-09T06:31:07.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.549+0000 7f8dc6863700 1 -- 172.21.15.163:0/3331415216 learned_addr learned my addr 172.21.15.163:0/3331415216 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:07.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.550+0000 7f8dc5861700 1 --2- 172.21.15.163:0/3331415216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8dc00af250 0x7f8dc0068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:07.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.550+0000 7f8dc6863700 1 -- 172.21.15.163:0/3331415216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8dc00af250 msgr2=0x7f8dc0068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:07.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.550+0000 7f8dc6863700 1 --2- 172.21.15.163:0/3331415216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8dc00af250 0x7f8dc0068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.550+0000 7f8dc6863700 1 -- 172.21.15.163:0/3331415216 >> v1:172.21.15.181:6789/0 conn(0x7f8dc0157960 legacy=0x7f8dc006b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:07.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.550+0000 7f8dc6863700 1 -- 172.21.15.163:0/3331415216 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc006bbe0 con 0x7f8dc00688d0 2023-10-09T06:31:07.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.550+0000 7f8dc6863700 1 --2- 172.21.15.163:0/3331415216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8dc00688d0 0x7f8dc0157d30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8db80099b0 tx=0x7f8db8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94923dc9c819232 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:07.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.550+0000 7f8dc5060700 1 -- 172.21.15.163:0/3331415216 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db8009ea0 con 0x7f8dc00688d0 2023-10-09T06:31:07.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.550+0000 7f8dc5060700 1 -- 172.21.15.163:0/3331415216 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db800b860 con 0x7f8dc00688d0 2023-10-09T06:31:07.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.550+0000 7f8dc5060700 1 -- 172.21.15.163:0/3331415216 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db800bc00 con 0x7f8dc00688d0 2023-10-09T06:31:07.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.551+0000 7f8dc82c6700 1 -- 172.21.15.163:0/3331415216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8dc00688d0 msgr2=0x7f8dc0157d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:07.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.551+0000 7f8dc82c6700 1 --2- 172.21.15.163:0/3331415216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8dc00688d0 0x7f8dc0157d30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8db80099b0 tx=0x7f8db8009250 comp rx=0 tx=0).stop 2023-10-09T06:31:07.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.551+0000 7f8dc82c6700 1 -- 172.21.15.163:0/3331415216 shutdown_connections 2023-10-09T06:31:07.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.551+0000 7f8dc82c6700 1 --2- 172.21.15.163:0/3331415216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8dc00688d0 0x7f8dc0157d30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.551+0000 7f8dc82c6700 1 --2- 172.21.15.163:0/3331415216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8dc00af250 0x7f8dc0068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.551+0000 7f8dc82c6700 1 -- 172.21.15.163:0/3331415216 >> 172.21.15.163:0/3331415216 conn(0x7f8dc00c9870 msgr2=0x7f8dc00c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:07.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.551+0000 7f8dc82c6700 1 -- 172.21.15.163:0/3331415216 shutdown_connections 2023-10-09T06:31:07.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.551+0000 7f8dc82c6700 1 -- 172.21.15.163:0/3331415216 wait complete. 2023-10-09T06:31:07.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.552+0000 7f8dc82c6700 1 Processor -- start 2023-10-09T06:31:07.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.552+0000 7f8dc82c6700 1 -- start start 2023-10-09T06:31:07.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.553+0000 7f8dc82c6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8dc00688d0 0x7f8dc014e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:07.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.553+0000 7f8dc82c6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8dc00af250 0x7f8dc00ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:07.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.553+0000 7f8dc82c6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dc0157960 0x7f8dc015baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:07.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.553+0000 7f8dc82c6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc00ab8c0 con 0x7f8dc00688d0 2023-10-09T06:31:07.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.553+0000 7f8dc82c6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc00abba0 con 0x7f8dc0157960 2023-10-09T06:31:07.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.553+0000 7f8dc82c6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc00abe80 con 0x7f8dc00af250 2023-10-09T06:31:07.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.553+0000 7f8dc6062700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8dc00688d0 0x7f8dc014e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:07.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.553+0000 7f8dc6062700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8dc00688d0 0x7f8dc014e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35748/0 (socket says 172.21.15.163:35748) 2023-10-09T06:31:07.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.553+0000 7f8dc6062700 1 -- 172.21.15.163:0/3449446366 learned_addr learned my addr 172.21.15.163:0/3449446366 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:07.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.553+0000 7f8dc5861700 1 --2- 172.21.15.163:0/3449446366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8dc00af250 0x7f8dc00ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:07.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.553+0000 7f8dc6863700 1 --2- 172.21.15.163:0/3449446366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dc0157960 0x7f8dc015baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:07.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.553+0000 7f8dc6062700 1 -- 172.21.15.163:0/3449446366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8dc00af250 msgr2=0x7f8dc00ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:07.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.553+0000 7f8dc6062700 1 --2- 172.21.15.163:0/3449446366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8dc00af250 0x7f8dc00ac8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.553+0000 7f8dc6062700 1 -- 172.21.15.163:0/3449446366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dc0157960 msgr2=0x7f8dc015baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:07.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.553+0000 7f8dc6062700 1 --2- 172.21.15.163:0/3449446366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dc0157960 0x7f8dc015baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.553+0000 7f8dc6062700 1 -- 172.21.15.163:0/3449446366 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db80094f0 con 0x7f8dc00688d0 2023-10-09T06:31:07.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.553+0000 7f8dc6062700 1 --2- 172.21.15.163:0/3449446366 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8dc00688d0 0x7f8dc014e2b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc00cb40 tx=0x7f8dbc001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:07.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.553+0000 7f8db67fc700 1 -- 172.21.15.163:0/3449446366 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc002060 con 0x7f8dc00688d0 2023-10-09T06:31:07.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.553+0000 7f8db67fc700 1 -- 172.21.15.163:0/3449446366 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dbc00a7e0 con 0x7f8dc00688d0 2023-10-09T06:31:07.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.553+0000 7f8dc82c6700 1 -- 172.21.15.163:0/3449446366 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc00ac160 con 0x7f8dc00688d0 2023-10-09T06:31:07.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.553+0000 7f8db67fc700 1 -- 172.21.15.163:0/3449446366 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc00aba0 con 0x7f8dc00688d0 2023-10-09T06:31:07.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.553+0000 7f8dc82c6700 1 -- 172.21.15.163:0/3449446366 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc00acde0 con 0x7f8dc00688d0 2023-10-09T06:31:07.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.554+0000 7f8dc82c6700 1 -- 172.21.15.163:0/3449446366 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc006f280 con 0x7f8dc00688d0 2023-10-09T06:31:07.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.555+0000 7f8db67fc700 1 -- 172.21.15.163:0/3449446366 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8dbc003030 con 0x7f8dc00688d0 2023-10-09T06:31:07.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.555+0000 7f8db67fc700 1 --2- 172.21.15.163:0/3449446366 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8da4067000 0x7f8da40694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:07.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.555+0000 7f8db67fc700 1 -- 172.21.15.163:0/3449446366 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8dbc0843f0 con 0x7f8dc00688d0 2023-10-09T06:31:07.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.555+0000 7f8dc5861700 1 --2- 172.21.15.163:0/3449446366 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8da4067000 0x7f8da40694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:07.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.556+0000 7f8dc5861700 1 --2- 172.21.15.163:0/3449446366 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8da4067000 0x7f8da40694b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8dc00c3d10 tx=0x7f8db00091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:07.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.559+0000 7f8db67fc700 1 -- 172.21.15.163:0/3449446366 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8dbc0516e0 con 0x7f8dc00688d0 2023-10-09T06:31:07.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.739+0000 7f8dc82c6700 1 -- 172.21.15.163:0/3449446366 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8dc0069d20 con 0x7f8dc00688d0 2023-10-09T06:31:07.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.739+0000 7f8db67fc700 1 -- 172.21.15.163:0/3449446366 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8dbc054d00 con 0x7f8dc00688d0 2023-10-09T06:31:07.740 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 64 2023-10-09T06:31:07.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.741+0000 7f8dc82c6700 1 -- 172.21.15.163:0/3449446366 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8da4067000 msgr2=0x7f8da40694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:07.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.741+0000 7f8dc82c6700 1 --2- 172.21.15.163:0/3449446366 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8da4067000 0x7f8da40694b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8dc00c3d10 tx=0x7f8db00091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.741+0000 7f8dc82c6700 1 -- 172.21.15.163:0/3449446366 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8dc00688d0 msgr2=0x7f8dc014e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:07.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.741+0000 7f8dc82c6700 1 --2- 172.21.15.163:0/3449446366 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8dc00688d0 0x7f8dc014e2b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc00cb40 tx=0x7f8dbc001770 comp rx=0 tx=0).stop 2023-10-09T06:31:07.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.741+0000 7f8dc82c6700 1 -- 172.21.15.163:0/3449446366 shutdown_connections 2023-10-09T06:31:07.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.741+0000 7f8dc82c6700 1 --2- 172.21.15.163:0/3449446366 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8da4067000 0x7f8da40694b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.741+0000 7f8dc82c6700 1 --2- 172.21.15.163:0/3449446366 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8dc00688d0 0x7f8dc014e2b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.741+0000 7f8dc82c6700 1 --2- 172.21.15.163:0/3449446366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8dc0157960 0x7f8dc015baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.741+0000 7f8dc82c6700 1 --2- 172.21.15.163:0/3449446366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8dc00af250 0x7f8dc00ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.741+0000 7f8dc82c6700 1 -- 172.21.15.163:0/3449446366 >> 172.21.15.163:0/3449446366 conn(0x7f8dc00c9870 msgr2=0x7f8dc01590b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:07.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.741+0000 7f8dc82c6700 1 -- 172.21.15.163:0/3449446366 shutdown_connections 2023-10-09T06:31:07.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.741+0000 7f8dc82c6700 1 -- 172.21.15.163:0/3449446366 wait complete. 2023-10-09T06:31:07.754 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:31:07.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.869+0000 7fa1fa8e7700 1 Processor -- start 2023-10-09T06:31:07.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.869+0000 7fa1fa8e7700 1 -- start start 2023-10-09T06:31:07.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.870+0000 7fa1fa8e7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa1f4156590 0x7fa1f40b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:07.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.870+0000 7fa1fa8e7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa1f40b25a0 0x7fa1f41581d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:07.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.870+0000 7fa1fa8e7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f406eaf0 con 0x7fa1f406c2c0 2023-10-09T06:31:07.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.870+0000 7fa1fa8e7700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1f415ab00 con 0x7fa1f4156590 2023-10-09T06:31:07.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.870+0000 7fa1fa8e7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1f415b080 con 0x7fa1f40b25a0 2023-10-09T06:31:07.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.870+0000 7fa1f98e5700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa1f4156590 0x7fa1f40b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:07.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.870+0000 7fa1f98e5700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa1f4156590 0x7fa1f40b04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35752/0 (socket says 172.21.15.163:35752) 2023-10-09T06:31:07.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.870+0000 7fa1f98e5700 1 -- 172.21.15.163:0/3931456115 learned_addr learned my addr 172.21.15.163:0/3931456115 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:07.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.870+0000 7fa1fa0e6700 1 --2- 172.21.15.163:0/3931456115 >> v2:172.21.15.181:3300/0 conn(0x7fa1f40b25a0 0x7fa1f41581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:07.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.870+0000 7fa1f98e5700 1 -- 172.21.15.163:0/3931456115 >> v2:172.21.15.181:3300/0 conn(0x7fa1f40b25a0 msgr2=0x7fa1f41581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:07.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.870+0000 7fa1f98e5700 1 --2- 172.21.15.163:0/3931456115 >> v2:172.21.15.181:3300/0 conn(0x7fa1f40b25a0 0x7fa1f41581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.870+0000 7fa1f98e5700 1 -- 172.21.15.163:0/3931456115 >> v1:172.21.15.181:6789/0 conn(0x7fa1f406c2c0 legacy=0x7fa1f40b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:07.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.870+0000 7fa1f98e5700 1 -- 172.21.15.163:0/3931456115 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f40b09e0 con 0x7fa1f4156590 2023-10-09T06:31:07.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.870+0000 7fa1fa0e6700 1 --2- 172.21.15.163:0/3931456115 >> v2:172.21.15.181:3300/0 conn(0x7fa1f40b25a0 0x7fa1f41581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:07.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.870+0000 7fa1f98e5700 1 --2- 172.21.15.163:0/3931456115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1f4156590 0x7fa1f40b04a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4063630 tx=0x7fa1ec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6d0a40b7c7bed8 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:07.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.870+0000 7fa1f88e3700 1 -- 172.21.15.163:0/3931456115 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec00e070 con 0x7fa1f4156590 2023-10-09T06:31:07.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.870+0000 7fa1f88e3700 1 -- 172.21.15.163:0/3931456115 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ec00ba10 con 0x7fa1f4156590 2023-10-09T06:31:07.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.870+0000 7fa1f88e3700 1 -- 172.21.15.163:0/3931456115 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec0092e0 con 0x7fa1f4156590 2023-10-09T06:31:07.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.870+0000 7fa1fa8e7700 1 -- 172.21.15.163:0/3931456115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1f4156590 msgr2=0x7fa1f40b04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:07.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.870+0000 7fa1fa8e7700 1 --2- 172.21.15.163:0/3931456115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1f4156590 0x7fa1f40b04a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4063630 tx=0x7fa1ec009840 comp rx=0 tx=0).stop 2023-10-09T06:31:07.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.870+0000 7fa1fa8e7700 1 -- 172.21.15.163:0/3931456115 shutdown_connections 2023-10-09T06:31:07.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.870+0000 7fa1fa8e7700 1 --2- 172.21.15.163:0/3931456115 >> v2:172.21.15.181:3300/0 conn(0x7fa1f40b25a0 0x7fa1f41581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.870+0000 7fa1fa8e7700 1 --2- 172.21.15.163:0/3931456115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1f4156590 0x7fa1f40b04a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.870+0000 7fa1fa8e7700 1 -- 172.21.15.163:0/3931456115 >> 172.21.15.163:0/3931456115 conn(0x7fa1f406ddb0 msgr2=0x7fa1f406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:07.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.870+0000 7fa1fa8e7700 1 -- 172.21.15.163:0/3931456115 shutdown_connections 2023-10-09T06:31:07.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.870+0000 7fa1fa8e7700 1 -- 172.21.15.163:0/3931456115 wait complete. 2023-10-09T06:31:07.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.871+0000 7fa1fa8e7700 1 Processor -- start 2023-10-09T06:31:07.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.871+0000 7fa1fa8e7700 1 -- start start 2023-10-09T06:31:07.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.872+0000 7fa1fa8e7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1f406c2c0 0x7fa1f4150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:07.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.872+0000 7fa1fa8e7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1f40b25a0 0x7fa1f41581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:07.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.872+0000 7fa1fa8e7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1f4156590 0x7fa1f4158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:07.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.872+0000 7fa1fa8e7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f4169020 con 0x7fa1f40b25a0 2023-10-09T06:31:07.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.872+0000 7fa1fa8e7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f40c4ec0 con 0x7fa1f4156590 2023-10-09T06:31:07.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.872+0000 7fa1fa8e7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f40c4590 con 0x7fa1f406c2c0 2023-10-09T06:31:07.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.872+0000 7fa1f90e4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1f40b25a0 0x7fa1f41581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:07.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.872+0000 7fa1f90e4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1f40b25a0 0x7fa1f41581d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35768/0 (socket says 172.21.15.163:35768) 2023-10-09T06:31:07.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.872+0000 7fa1f90e4700 1 -- 172.21.15.163:0/913466595 learned_addr learned my addr 172.21.15.163:0/913466595 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:07.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.872+0000 7fa1f98e5700 1 --2- 172.21.15.163:0/913466595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1f406c2c0 0x7fa1f4150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:07.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.872+0000 7fa1fa0e6700 1 --2- 172.21.15.163:0/913466595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1f4156590 0x7fa1f4158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:07.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.872+0000 7fa1f90e4700 1 -- 172.21.15.163:0/913466595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1f406c2c0 msgr2=0x7fa1f4150630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:07.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.872+0000 7fa1f90e4700 1 --2- 172.21.15.163:0/913466595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1f406c2c0 0x7fa1f4150630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.872+0000 7fa1f90e4700 1 -- 172.21.15.163:0/913466595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1f4156590 msgr2=0x7fa1f4158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:07.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.872+0000 7fa1f90e4700 1 --2- 172.21.15.163:0/913466595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1f4156590 0x7fa1f4158710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:07.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.872+0000 7fa1f90e4700 1 -- 172.21.15.163:0/913466595 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ec009ae0 con 0x7fa1f40b25a0 2023-10-09T06:31:07.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.872+0000 7fa1fa0e6700 1 --2- 172.21.15.163:0/913466595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1f4156590 0x7fa1f4158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:07.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.873+0000 7fa1f90e4700 1 --2- 172.21.15.163:0/913466595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1f40b25a0 0x7fa1f41581d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa1e400c600 tx=0x7fa1e4007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:07.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.873+0000 7fa1e9ffb700 1 -- 172.21.15.163:0/913466595 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4011590 con 0x7fa1f40b25a0 2023-10-09T06:31:07.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.873+0000 7fa1e9ffb700 1 -- 172.21.15.163:0/913466595 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e4011710 con 0x7fa1f40b25a0 2023-10-09T06:31:07.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.873+0000 7fa1fa8e7700 1 -- 172.21.15.163:0/913466595 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f40c4870 con 0x7fa1f40b25a0 2023-10-09T06:31:07.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.873+0000 7fa1e9ffb700 1 -- 172.21.15.163:0/913466595 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4011ad0 con 0x7fa1f40b25a0 2023-10-09T06:31:07.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.873+0000 7fa1fa8e7700 1 -- 172.21.15.163:0/913466595 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f40c5350 con 0x7fa1f40b25a0 2023-10-09T06:31:07.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.874+0000 7fa1fa8e7700 1 -- 172.21.15.163:0/913466595 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f40c5050 con 0x7fa1f40b25a0 2023-10-09T06:31:07.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.874+0000 7fa1e9ffb700 1 -- 172.21.15.163:0/913466595 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa1e401c460 con 0x7fa1f40b25a0 2023-10-09T06:31:07.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.874+0000 7fa1e9ffb700 1 --2- 172.21.15.163:0/913466595 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa1d8066f30 0x7fa1d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:07.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.875+0000 7fa1e9ffb700 1 -- 172.21.15.163:0/913466595 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa1e4084220 con 0x7fa1f40b25a0 2023-10-09T06:31:07.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.875+0000 7fa1f98e5700 1 --2- 172.21.15.163:0/913466595 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa1d8066f30 0x7fa1d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:07.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.875+0000 7fa1f98e5700 1 --2- 172.21.15.163:0/913466595 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa1d8066f30 0x7fa1d80693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009880 tx=0x7fa1ec017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:07.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:07.880+0000 7fa1e9ffb700 1 -- 172.21.15.163:0/913466595 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa1e4051490 con 0x7fa1f40b25a0 2023-10-09T06:31:08.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.060+0000 7fa1fa8e7700 1 -- 172.21.15.163:0/913466595 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1f404c3c0 con 0x7fa1f40b25a0 2023-10-09T06:31:08.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.061+0000 7fa1e9ffb700 1 -- 172.21.15.163:0/913466595 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fa1e4054ab0 con 0x7fa1f40b25a0 2023-10-09T06:31:08.062 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:08.062 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":20,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:30:44.466923+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:30:41.369014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-09T06:30:42.103001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:31:08.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.063+0000 7fa1fa8e7700 1 -- 172.21.15.163:0/913466595 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa1d8066f30 msgr2=0x7fa1d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:08.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.063+0000 7fa1fa8e7700 1 --2- 172.21.15.163:0/913466595 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa1d8066f30 0x7fa1d80693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009880 tx=0x7fa1ec017000 comp rx=0 tx=0).stop 2023-10-09T06:31:08.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.063+0000 7fa1fa8e7700 1 -- 172.21.15.163:0/913466595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1f40b25a0 msgr2=0x7fa1f41581d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:08.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.063+0000 7fa1fa8e7700 1 --2- 172.21.15.163:0/913466595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1f40b25a0 0x7fa1f41581d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa1e400c600 tx=0x7fa1e4007f60 comp rx=0 tx=0).stop 2023-10-09T06:31:08.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.063+0000 7fa1fa8e7700 1 -- 172.21.15.163:0/913466595 shutdown_connections 2023-10-09T06:31:08.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.063+0000 7fa1fa8e7700 1 --2- 172.21.15.163:0/913466595 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa1d8066f30 0x7fa1d80693e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.063+0000 7fa1fa8e7700 1 --2- 172.21.15.163:0/913466595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1f4156590 0x7fa1f4158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.063+0000 7fa1fa8e7700 1 --2- 172.21.15.163:0/913466595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1f406c2c0 0x7fa1f4150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.063+0000 7fa1fa8e7700 1 --2- 172.21.15.163:0/913466595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1f40b25a0 0x7fa1f41581d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.063+0000 7fa1fa8e7700 1 -- 172.21.15.163:0/913466595 >> 172.21.15.163:0/913466595 conn(0x7fa1f406ddb0 msgr2=0x7fa1f4158da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:08.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.063+0000 7fa1fa8e7700 1 -- 172.21.15.163:0/913466595 shutdown_connections 2023-10-09T06:31:08.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.063+0000 7fa1fa8e7700 1 -- 172.21.15.163:0/913466595 wait complete. 2023-10-09T06:31:08.076 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-09T06:30:41.369014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-09T06:30:42.103001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-09T06:31:08.077 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:31:08.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.194+0000 7f2bb5827700 1 Processor -- start 2023-10-09T06:31:08.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.194+0000 7f2bb5827700 1 -- start start 2023-10-09T06:31:08.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2bb5827700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2bb00b4e70 0x7f2bb00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:08.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2bb5827700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb00cfb00 con 0x7f2bb00b16d0 2023-10-09T06:31:08.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2bb5827700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb015e060 con 0x7f2bb00b1b40 2023-10-09T06:31:08.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2bb5827700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bb00b5c40 con 0x7f2bb00b4e70 2023-10-09T06:31:08.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2baffff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f2bb00b16d0 0x7f2bb015d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:47760/0 (socket says 172.21.15.163:47760) 2023-10-09T06:31:08.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2baffff700 1 -- 172.21.15.163:0/888214956 learned_addr learned my addr 172.21.15.163:0/888214956 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:08.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2bb4825700 1 --2- 172.21.15.163:0/888214956 >> v2:172.21.15.181:3300/0 conn(0x7f2bb00b4e70 0x7f2bb00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:08.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2baf7fe700 1 -- 172.21.15.163:0/888214956 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1257781057 0 0) 0x7f2bb00cfb00 con 0x7f2bb00b16d0 2023-10-09T06:31:08.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2baf7fe700 1 -- 172.21.15.163:0/888214956 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ba0003570 con 0x7f2bb00b16d0 2023-10-09T06:31:08.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2baf7fe700 1 -- 172.21.15.163:0/888214956 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 111239648 0 0) 0x7f2ba0003570 con 0x7f2bb00b16d0 2023-10-09T06:31:08.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2baf7fe700 1 -- 172.21.15.163:0/888214956 >> v2:172.21.15.181:3300/0 conn(0x7f2bb00b4e70 msgr2=0x7f2bb00ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:08.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2baf7fe700 1 --2- 172.21.15.163:0/888214956 >> v2:172.21.15.181:3300/0 conn(0x7f2bb00b4e70 0x7f2bb00ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2baf7fe700 1 -- 172.21.15.163:0/888214956 >> v1:172.21.15.181:6789/0 conn(0x7f2bb00b1b40 legacy=0x7f2bb015d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:08.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2baf7fe700 1 -- 172.21.15.163:0/888214956 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb00b5f20 con 0x7f2bb00b16d0 2023-10-09T06:31:08.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2baf7fe700 1 -- 172.21.15.163:0/888214956 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f2b9c001470 con 0x7f2bb00b16d0 2023-10-09T06:31:08.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2baf7fe700 1 -- 172.21.15.163:0/888214956 >> v1:172.21.15.163:6789/0 conn(0x7f2bb00b16d0 legacy=0x7f2bb015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:08.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2baf7fe700 1 --2- 172.21.15.163:0/888214956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ba0003a60 0x7f2ba0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:08.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2baf7fe700 1 --2- 172.21.15.163:0/888214956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ba0004810 0x7f2ba0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:08.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2baf7fe700 1 --2- 172.21.15.163:0/888214956 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ba00051c0 0x7f2ba000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:08.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2baf7fe700 1 -- 172.21.15.163:0/888214956 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb00b5c40 con 0x7f2ba0003a60 2023-10-09T06:31:08.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2baf7fe700 1 -- 172.21.15.163:0/888214956 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba000c160 con 0x7f2ba0004810 2023-10-09T06:31:08.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2baf7fe700 1 -- 172.21.15.163:0/888214956 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba000c320 con 0x7f2ba00051c0 2023-10-09T06:31:08.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2bb4825700 1 --2- 172.21.15.163:0/888214956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ba0003a60 0x7f2ba0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:08.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2baffff700 1 --2- 172.21.15.163:0/888214956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ba0004810 0x7f2ba0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:08.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2bb5026700 1 --2- 172.21.15.163:0/888214956 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ba00051c0 0x7f2ba000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:08.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2bb4825700 1 -- 172.21.15.163:0/888214956 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ba00051c0 msgr2=0x7f2ba000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:08.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2bb4825700 1 --2- 172.21.15.163:0/888214956 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ba00051c0 0x7f2ba000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2bb4825700 1 -- 172.21.15.163:0/888214956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ba0004810 msgr2=0x7f2ba0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:08.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2bb4825700 1 --2- 172.21.15.163:0/888214956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ba0004810 0x7f2ba0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2bb4825700 1 -- 172.21.15.163:0/888214956 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba0003570 con 0x7f2ba0003a60 2023-10-09T06:31:08.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2bb5026700 1 -- 172.21.15.163:0/888214956 reap_dead start 2023-10-09T06:31:08.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2bb4825700 1 --2- 172.21.15.163:0/888214956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ba0003a60 0x7f2ba0003e70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009c00 tx=0x7f2ba40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85b9f588d0912f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:08.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2baf7fe700 1 -- 172.21.15.163:0/888214956 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba400e710 con 0x7f2ba0003a60 2023-10-09T06:31:08.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2baf7fe700 1 -- 172.21.15.163:0/888214956 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba400e870 con 0x7f2ba0003a60 2023-10-09T06:31:08.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.195+0000 7f2baf7fe700 1 -- 172.21.15.163:0/888214956 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba400ec30 con 0x7f2ba0003a60 2023-10-09T06:31:08.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.196+0000 7f2bb5827700 1 -- 172.21.15.163:0/888214956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ba0003a60 msgr2=0x7f2ba0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:08.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.196+0000 7f2bb5827700 1 --2- 172.21.15.163:0/888214956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ba0003a60 0x7f2ba0003e70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009c00 tx=0x7f2ba40091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.196+0000 7f2bb5827700 1 -- 172.21.15.163:0/888214956 shutdown_connections 2023-10-09T06:31:08.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.196+0000 7f2bb5827700 1 --2- 172.21.15.163:0/888214956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ba0003a60 0x7f2ba0003e70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.196+0000 7f2bb5827700 1 -- 172.21.15.163:0/888214956 >> 172.21.15.163:0/888214956 conn(0x7f2bb00d10c0 msgr2=0x7f2bb00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:08.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.196+0000 7f2bb5827700 1 -- 172.21.15.163:0/888214956 shutdown_connections 2023-10-09T06:31:08.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.196+0000 7f2bb5827700 1 -- 172.21.15.163:0/888214956 wait complete. 2023-10-09T06:31:08.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.197+0000 7f2bb5827700 1 Processor -- start 2023-10-09T06:31:08.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.197+0000 7f2bb5827700 1 -- start start 2023-10-09T06:31:08.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.199+0000 7f2bb5827700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ba0003a60 0x7f2bb00c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:08.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.199+0000 7f2bb5827700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bb00c7ef0 0x7f2bb016a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:08.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.199+0000 7f2bb5827700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bb00c6600 0x7f2bb00ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:08.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.199+0000 7f2bb5827700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb00c8660 con 0x7f2bb00c7ef0 2023-10-09T06:31:08.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.199+0000 7f2bb5827700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb00c9000 con 0x7f2bb00c6600 2023-10-09T06:31:08.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.199+0000 7f2bb5827700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb00c8380 con 0x7f2ba0003a60 2023-10-09T06:31:08.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.199+0000 7f2baffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bb00c7ef0 0x7f2bb016a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:08.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.199+0000 7f2baffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bb00c7ef0 0x7f2bb016a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35776/0 (socket says 172.21.15.163:35776) 2023-10-09T06:31:08.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.199+0000 7f2baffff700 1 -- 172.21.15.163:0/581446154 learned_addr learned my addr 172.21.15.163:0/581446154 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:08.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.199+0000 7f2bb4825700 1 --2- 172.21.15.163:0/581446154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ba0003a60 0x7f2bb00c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:08.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.199+0000 7f2bb5026700 1 --2- 172.21.15.163:0/581446154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bb00c6600 0x7f2bb00ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:08.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.199+0000 7f2baffff700 1 -- 172.21.15.163:0/581446154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ba0003a60 msgr2=0x7f2bb00c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:08.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.199+0000 7f2baffff700 1 --2- 172.21.15.163:0/581446154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ba0003a60 0x7f2bb00c9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.199+0000 7f2baffff700 1 -- 172.21.15.163:0/581446154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bb00c6600 msgr2=0x7f2bb00ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:08.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.199+0000 7f2baffff700 1 --2- 172.21.15.163:0/581446154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bb00c6600 0x7f2bb00ca510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.199+0000 7f2baffff700 1 -- 172.21.15.163:0/581446154 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb00c89b0 con 0x7f2bb00c7ef0 2023-10-09T06:31:08.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.199+0000 7f2baffff700 1 --2- 172.21.15.163:0/581446154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bb00c7ef0 0x7f2bb016a940 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c0073a0 tx=0x7f2b9c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:08.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.199+0000 7f2bacff9700 1 -- 172.21.15.163:0/581446154 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c007780 con 0x7f2bb00c7ef0 2023-10-09T06:31:08.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.199+0000 7f2bacff9700 1 -- 172.21.15.163:0/581446154 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b9c00f8d0 con 0x7f2bb00c7ef0 2023-10-09T06:31:08.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.199+0000 7f2bb5827700 1 -- 172.21.15.163:0/581446154 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb00c8c90 con 0x7f2bb00c7ef0 2023-10-09T06:31:08.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.199+0000 7f2bacff9700 1 -- 172.21.15.163:0/581446154 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c00fc90 con 0x7f2bb00c7ef0 2023-10-09T06:31:08.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.199+0000 7f2bb5827700 1 -- 172.21.15.163:0/581446154 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb00c6a60 con 0x7f2bb00c7ef0 2023-10-09T06:31:08.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.199+0000 7f2bb5827700 1 -- 172.21.15.163:0/581446154 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb0088990 con 0x7f2bb00c7ef0 2023-10-09T06:31:08.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.200+0000 7f2bacff9700 1 -- 172.21.15.163:0/581446154 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2b9c01e3b0 con 0x7f2bb00c7ef0 2023-10-09T06:31:08.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.200+0000 7f2bacff9700 1 --2- 172.21.15.163:0/581446154 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2b94066f30 0x7f2b940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:08.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.200+0000 7f2bacff9700 1 -- 172.21.15.163:0/581446154 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2b9c084270 con 0x7f2bb00c7ef0 2023-10-09T06:31:08.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.200+0000 7f2bb4825700 1 --2- 172.21.15.163:0/581446154 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2b94066f30 0x7f2b940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:08.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.201+0000 7f2bb4825700 1 --2- 172.21.15.163:0/581446154 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2b94066f30 0x7f2b940693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40091e0 tx=0x7f2ba401a000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:08.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.205+0000 7f2bacff9700 1 -- 172.21.15.163:0/581446154 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2b9c0514e0 con 0x7f2bb00c7ef0 2023-10-09T06:31:08.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.384+0000 7f2bb5827700 1 -- 172.21.15.163:0/581446154 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2bb00c9140 con 0x7f2bb00c7ef0 2023-10-09T06:31:08.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.385+0000 7f2bacff9700 1 -- 172.21.15.163:0/581446154 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2b9c054b00 con 0x7f2bb00c7ef0 2023-10-09T06:31:08.386 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:31:08.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.387+0000 7f2bb5827700 1 -- 172.21.15.163:0/581446154 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2b94066f30 msgr2=0x7f2b940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:08.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.387+0000 7f2bb5827700 1 --2- 172.21.15.163:0/581446154 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2b94066f30 0x7f2b940693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40091e0 tx=0x7f2ba401a000 comp rx=0 tx=0).stop 2023-10-09T06:31:08.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.387+0000 7f2bb5827700 1 -- 172.21.15.163:0/581446154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bb00c7ef0 msgr2=0x7f2bb016a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:08.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.387+0000 7f2bb5827700 1 --2- 172.21.15.163:0/581446154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bb00c7ef0 0x7f2bb016a940 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c0073a0 tx=0x7f2b9c001e50 comp rx=0 tx=0).stop 2023-10-09T06:31:08.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.387+0000 7f2bb5827700 1 -- 172.21.15.163:0/581446154 shutdown_connections 2023-10-09T06:31:08.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.387+0000 7f2bb5827700 1 --2- 172.21.15.163:0/581446154 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2b94066f30 0x7f2b940693e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.387+0000 7f2bb5827700 1 --2- 172.21.15.163:0/581446154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2bb00c6600 0x7f2bb00ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.387+0000 7f2bb5827700 1 --2- 172.21.15.163:0/581446154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ba0003a60 0x7f2bb00c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.387+0000 7f2bb5827700 1 --2- 172.21.15.163:0/581446154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bb00c7ef0 0x7f2bb016a940 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.387+0000 7f2bb5827700 1 -- 172.21.15.163:0/581446154 >> 172.21.15.163:0/581446154 conn(0x7f2bb00d10c0 msgr2=0x7f2bb0068c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:08.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.387+0000 7f2bb5827700 1 -- 172.21.15.163:0/581446154 shutdown_connections 2023-10-09T06:31:08.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.387+0000 7f2bb5827700 1 -- 172.21.15.163:0/581446154 wait complete. 2023-10-09T06:31:08.400 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-09T06:31:08.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.517+0000 7f3e6802d700 1 Processor -- start 2023-10-09T06:31:08.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.517+0000 7f3e6802d700 1 -- start start 2023-10-09T06:31:08.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.518+0000 7f3e6802d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3e60159890 0x7f3e6015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:08.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.518+0000 7f3e6802d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e60069560 0x7f3e600699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:08.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.518+0000 7f3e6802d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e60069ee0 con 0x7f3e60159c60 2023-10-09T06:31:08.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.518+0000 7f3e6802d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e6015a160 con 0x7f3e60159890 2023-10-09T06:31:08.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.518+0000 7f3e6802d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e600d13a0 con 0x7f3e60069560 2023-10-09T06:31:08.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.518+0000 7f3e65dc9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3e60159890 0x7f3e6015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:08.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.518+0000 7f3e65dc9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3e60159890 0x7f3e6015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53036/0 (socket says 172.21.15.163:53036) 2023-10-09T06:31:08.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.518+0000 7f3e65dc9700 1 -- 172.21.15.163:0/2223221523 learned_addr learned my addr 172.21.15.163:0/2223221523 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:08.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.518+0000 7f3e665ca700 1 --2- 172.21.15.163:0/2223221523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e60069560 0x7f3e600699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:08.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.518+0000 7f3e65dc9700 1 -- 172.21.15.163:0/2223221523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e60069560 msgr2=0x7f3e600699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:08.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.518+0000 7f3e65dc9700 1 --2- 172.21.15.163:0/2223221523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e60069560 0x7f3e600699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.518+0000 7f3e65dc9700 1 -- 172.21.15.163:0/2223221523 >> v1:172.21.15.181:6789/0 conn(0x7f3e60159c60 legacy=0x7f3e6015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:08.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.518+0000 7f3e65dc9700 1 -- 172.21.15.163:0/2223221523 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e6015e140 con 0x7f3e60159890 2023-10-09T06:31:08.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.519+0000 7f3e65dc9700 1 --2- 172.21.15.163:0/2223221523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e60159890 0x7f3e6015abd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009f20 tx=0x7f3e50009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20c11d404a05d49 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:08.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.519+0000 7f3e64dc7700 1 -- 172.21.15.163:0/2223221523 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5000e070 con 0x7f3e60159890 2023-10-09T06:31:08.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.519+0000 7f3e64dc7700 1 -- 172.21.15.163:0/2223221523 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e5000b9d0 con 0x7f3e60159890 2023-10-09T06:31:08.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.519+0000 7f3e64dc7700 1 -- 172.21.15.163:0/2223221523 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e500092e0 con 0x7f3e60159890 2023-10-09T06:31:08.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.519+0000 7f3e6802d700 1 -- 172.21.15.163:0/2223221523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e60159890 msgr2=0x7f3e6015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:08.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.519+0000 7f3e6802d700 1 --2- 172.21.15.163:0/2223221523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e60159890 0x7f3e6015abd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009f20 tx=0x7f3e50009840 comp rx=0 tx=0).stop 2023-10-09T06:31:08.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.519+0000 7f3e6802d700 1 -- 172.21.15.163:0/2223221523 shutdown_connections 2023-10-09T06:31:08.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.519+0000 7f3e6802d700 1 --2- 172.21.15.163:0/2223221523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e60069560 0x7f3e600699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.519+0000 7f3e6802d700 1 --2- 172.21.15.163:0/2223221523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e60159890 0x7f3e6015abd0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.519+0000 7f3e6802d700 1 -- 172.21.15.163:0/2223221523 >> 172.21.15.163:0/2223221523 conn(0x7f3e60068e20 msgr2=0x7f3e600d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:08.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.519+0000 7f3e6802d700 1 -- 172.21.15.163:0/2223221523 shutdown_connections 2023-10-09T06:31:08.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.519+0000 7f3e6802d700 1 -- 172.21.15.163:0/2223221523 wait complete. 2023-10-09T06:31:08.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.520+0000 7f3e6802d700 1 Processor -- start 2023-10-09T06:31:08.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.520+0000 7f3e6802d700 1 -- start start 2023-10-09T06:31:08.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.521+0000 7f3e6802d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e60069560 0x7f3e601471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:08.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.521+0000 7f3e6802d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e60159890 0x7f3e600b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:08.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.521+0000 7f3e6802d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e60159c60 0x7f3e600b5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:08.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.521+0000 7f3e6802d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e600c78a0 con 0x7f3e60159890 2023-10-09T06:31:08.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.521+0000 7f3e6802d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e600b41c0 con 0x7f3e60159c60 2023-10-09T06:31:08.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.521+0000 7f3e6802d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e600b44a0 con 0x7f3e60069560 2023-10-09T06:31:08.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.521+0000 7f3e655c8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e60159890 0x7f3e600b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:08.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.521+0000 7f3e655c8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e60159890 0x7f3e600b4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53046/0 (socket says 172.21.15.163:53046) 2023-10-09T06:31:08.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.521+0000 7f3e655c8700 1 -- 172.21.15.163:0/1377168930 learned_addr learned my addr 172.21.15.163:0/1377168930 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:08.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.521+0000 7f3e65dc9700 1 --2- 172.21.15.163:0/1377168930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e60069560 0x7f3e601471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:08.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.522+0000 7f3e665ca700 1 --2- 172.21.15.163:0/1377168930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e60159c60 0x7f3e600b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:08.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.522+0000 7f3e655c8700 1 -- 172.21.15.163:0/1377168930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e60069560 msgr2=0x7f3e601471f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:08.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.522+0000 7f3e655c8700 1 --2- 172.21.15.163:0/1377168930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e60069560 0x7f3e601471f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.522+0000 7f3e655c8700 1 -- 172.21.15.163:0/1377168930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e60159c60 msgr2=0x7f3e600b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:08.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.522+0000 7f3e655c8700 1 --2- 172.21.15.163:0/1377168930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e60159c60 0x7f3e600b5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.522+0000 7f3e655c8700 1 -- 172.21.15.163:0/1377168930 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e50009ae0 con 0x7f3e60159890 2023-10-09T06:31:08.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.522+0000 7f3e655c8700 1 --2- 172.21.15.163:0/1377168930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e60159890 0x7f3e600b4e00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400c590 tx=0x7f3e5400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:08.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.522+0000 7f3e5dffb700 1 -- 172.21.15.163:0/1377168930 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e54011590 con 0x7f3e60159890 2023-10-09T06:31:08.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.522+0000 7f3e5dffb700 1 -- 172.21.15.163:0/1377168930 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e540116f0 con 0x7f3e60159890 2023-10-09T06:31:08.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.522+0000 7f3e5dffb700 1 -- 172.21.15.163:0/1377168930 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e54011ab0 con 0x7f3e60159890 2023-10-09T06:31:08.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.522+0000 7f3e6802d700 1 -- 172.21.15.163:0/1377168930 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e600c7af0 con 0x7f3e60159890 2023-10-09T06:31:08.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.522+0000 7f3e6802d700 1 -- 172.21.15.163:0/1377168930 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e60147730 con 0x7f3e60159890 2023-10-09T06:31:08.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.523+0000 7f3e6802d700 1 -- 172.21.15.163:0/1377168930 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e6004c3c0 con 0x7f3e60159890 2023-10-09T06:31:08.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.523+0000 7f3e5dffb700 1 -- 172.21.15.163:0/1377168930 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3e5401b3b0 con 0x7f3e60159890 2023-10-09T06:31:08.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.524+0000 7f3e5dffb700 1 --2- 172.21.15.163:0/1377168930 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3e4c066f30 0x7f3e4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:08.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.524+0000 7f3e5dffb700 1 -- 172.21.15.163:0/1377168930 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3e54083a90 con 0x7f3e60159890 2023-10-09T06:31:08.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.524+0000 7f3e65dc9700 1 --2- 172.21.15.163:0/1377168930 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3e4c066f30 0x7f3e4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:08.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.524+0000 7f3e65dc9700 1 --2- 172.21.15.163:0/1377168930 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3e4c066f30 0x7f3e4c0693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009880 tx=0x7f3e50018000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:08.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.529+0000 7f3e5dffb700 1 -- 172.21.15.163:0/1377168930 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3e54050d00 con 0x7f3e60159890 2023-10-09T06:31:08.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.711+0000 7f3e6802d700 1 -- 172.21.15.163:0/1377168930 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3e6006a5b0 con 0x7f3e60159890 2023-10-09T06:31:08.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.711+0000 7f3e5dffb700 1 -- 172.21.15.163:0/1377168930 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3e540170a0 con 0x7f3e60159890 2023-10-09T06:31:08.711 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 32 2023-10-09T06:31:08.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.713+0000 7f3e6802d700 1 -- 172.21.15.163:0/1377168930 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3e4c066f30 msgr2=0x7f3e4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:08.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.713+0000 7f3e6802d700 1 --2- 172.21.15.163:0/1377168930 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3e4c066f30 0x7f3e4c0693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009880 tx=0x7f3e50018000 comp rx=0 tx=0).stop 2023-10-09T06:31:08.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.713+0000 7f3e6802d700 1 -- 172.21.15.163:0/1377168930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e60159890 msgr2=0x7f3e600b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:08.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.713+0000 7f3e6802d700 1 --2- 172.21.15.163:0/1377168930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e60159890 0x7f3e600b4e00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400c590 tx=0x7f3e5400c950 comp rx=0 tx=0).stop 2023-10-09T06:31:08.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.713+0000 7f3e6802d700 1 -- 172.21.15.163:0/1377168930 shutdown_connections 2023-10-09T06:31:08.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.713+0000 7f3e6802d700 1 --2- 172.21.15.163:0/1377168930 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3e4c066f30 0x7f3e4c0693e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.713+0000 7f3e6802d700 1 --2- 172.21.15.163:0/1377168930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e60159c60 0x7f3e600b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.713+0000 7f3e6802d700 1 --2- 172.21.15.163:0/1377168930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e60069560 0x7f3e601471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.713+0000 7f3e6802d700 1 --2- 172.21.15.163:0/1377168930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e60159890 0x7f3e600b4e00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.713+0000 7f3e6802d700 1 -- 172.21.15.163:0/1377168930 >> 172.21.15.163:0/1377168930 conn(0x7f3e60068e20 msgr2=0x7f3e600b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:08.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.713+0000 7f3e6802d700 1 -- 172.21.15.163:0/1377168930 shutdown_connections 2023-10-09T06:31:08.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.713+0000 7f3e6802d700 1 -- 172.21.15.163:0/1377168930 wait complete. 2023-10-09T06:31:08.725 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-09T06:31:08.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.844+0000 7fb7bc0d9700 1 Processor -- start 2023-10-09T06:31:08.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.844+0000 7fb7bc0d9700 1 -- start start 2023-10-09T06:31:08.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.845+0000 7fb7bc0d9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb7b4159890 0x7fb7b415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:08.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.845+0000 7fb7bc0d9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7b4159c60 0x7fb7b415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:08.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.845+0000 7fb7bc0d9700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7b415a160 con 0x7fb7b415db10 2023-10-09T06:31:08.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.845+0000 7fb7bc0d9700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7b415df50 con 0x7fb7b4159890 2023-10-09T06:31:08.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.845+0000 7fb7bc0d9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7b40d13a0 con 0x7fb7b4159c60 2023-10-09T06:31:08.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.845+0000 7fb7b9e75700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb7b4159890 0x7fb7b415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:08.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.845+0000 7fb7ba676700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb7b415db10 0x7fb7b4069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53168/0 (socket says 172.21.15.163:53168) 2023-10-09T06:31:08.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.845+0000 7fb7b9e75700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb7b4159890 0x7fb7b415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53054/0 (socket says 172.21.15.163:53054) 2023-10-09T06:31:08.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.845+0000 7fb7ba676700 1 -- 172.21.15.163:0/978623054 learned_addr learned my addr 172.21.15.163:0/978623054 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:08.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.845+0000 7fb7b9674700 1 --2- 172.21.15.163:0/978623054 >> v2:172.21.15.181:3300/0 conn(0x7fb7b4159c60 0x7fb7b415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:08.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.845+0000 7fb7b9e75700 1 -- 172.21.15.163:0/978623054 >> v2:172.21.15.181:3300/0 conn(0x7fb7b4159c60 msgr2=0x7fb7b415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:08.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.845+0000 7fb7b9e75700 1 --2- 172.21.15.163:0/978623054 >> v2:172.21.15.181:3300/0 conn(0x7fb7b4159c60 0x7fb7b415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.845+0000 7fb7b9e75700 1 -- 172.21.15.163:0/978623054 >> v1:172.21.15.163:6789/0 conn(0x7fb7b415db10 legacy=0x7fb7b4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:08.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.845+0000 7fb7b9e75700 1 -- 172.21.15.163:0/978623054 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b415e140 con 0x7fb7b4159890 2023-10-09T06:31:08.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.845+0000 7fb7b8e73700 1 -- 172.21.15.163:0/978623054 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 158687945 0 0) 0x7fb7b415a160 con 0x7fb7b415db10 2023-10-09T06:31:08.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.845+0000 7fb7b9e75700 1 --2- 172.21.15.163:0/978623054 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7b4159890 0x7fb7b415abd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb7b40b67f0 tx=0x7fb7b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7f6fc17f0a151b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:08.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.845+0000 7fb7b8e73700 1 -- 172.21.15.163:0/978623054 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b000b6b0 con 0x7fb7b4159890 2023-10-09T06:31:08.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.845+0000 7fb7b8e73700 1 -- 172.21.15.163:0/978623054 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b000b810 con 0x7fb7b4159890 2023-10-09T06:31:08.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.845+0000 7fb7b8e73700 1 -- 172.21.15.163:0/978623054 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b000bbb0 con 0x7fb7b4159890 2023-10-09T06:31:08.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.846+0000 7fb7bc0d9700 1 -- 172.21.15.163:0/978623054 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7b4159890 msgr2=0x7fb7b415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:08.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.846+0000 7fb7bc0d9700 1 --2- 172.21.15.163:0/978623054 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7b4159890 0x7fb7b415abd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb7b40b67f0 tx=0x7fb7b00091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.846+0000 7fb7bc0d9700 1 -- 172.21.15.163:0/978623054 shutdown_connections 2023-10-09T06:31:08.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.846+0000 7fb7bc0d9700 1 --2- 172.21.15.163:0/978623054 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7b4159890 0x7fb7b415abd0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.846+0000 7fb7bc0d9700 1 --2- 172.21.15.163:0/978623054 >> v2:172.21.15.181:3300/0 conn(0x7fb7b4159c60 0x7fb7b415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.846+0000 7fb7bc0d9700 1 -- 172.21.15.163:0/978623054 >> 172.21.15.163:0/978623054 conn(0x7fb7b4068e20 msgr2=0x7fb7b40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:08.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.846+0000 7fb7bc0d9700 1 -- 172.21.15.163:0/978623054 shutdown_connections 2023-10-09T06:31:08.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.846+0000 7fb7bc0d9700 1 -- 172.21.15.163:0/978623054 wait complete. 2023-10-09T06:31:08.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.847+0000 7fb7bc0d9700 1 Processor -- start 2023-10-09T06:31:08.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.847+0000 7fb7bc0d9700 1 -- start start 2023-10-09T06:31:08.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.848+0000 7fb7bc0d9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7b4159890 0x7fb7b4151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:08.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.848+0000 7fb7bc0d9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7b4159c60 0x7fb7b415c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:08.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.848+0000 7fb7bc0d9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7b415db10 0x7fb7b415cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:08.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.848+0000 7fb7bc0d9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b40cd560 con 0x7fb7b4159890 2023-10-09T06:31:08.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.848+0000 7fb7bc0d9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b40cd840 con 0x7fb7b4159c60 2023-10-09T06:31:08.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.848+0000 7fb7bc0d9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b40cdb20 con 0x7fb7b415db10 2023-10-09T06:31:08.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.848+0000 7fb7b9e75700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7b4159890 0x7fb7b4151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:08.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.848+0000 7fb7b9e75700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7b4159890 0x7fb7b4151f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53060/0 (socket says 172.21.15.163:53060) 2023-10-09T06:31:08.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.848+0000 7fb7b9e75700 1 -- 172.21.15.163:0/578858691 learned_addr learned my addr 172.21.15.163:0/578858691 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:08.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.848+0000 7fb7b9674700 1 --2- 172.21.15.163:0/578858691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7b4159c60 0x7fb7b415c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:08.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.848+0000 7fb7ba676700 1 --2- 172.21.15.163:0/578858691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7b415db10 0x7fb7b415cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:08.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.848+0000 7fb7b9e75700 1 -- 172.21.15.163:0/578858691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7b415db10 msgr2=0x7fb7b415cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:08.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.848+0000 7fb7b9e75700 1 --2- 172.21.15.163:0/578858691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7b415db10 0x7fb7b415cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.848+0000 7fb7b9e75700 1 -- 172.21.15.163:0/578858691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7b4159c60 msgr2=0x7fb7b415c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:08.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.848+0000 7fb7b9e75700 1 --2- 172.21.15.163:0/578858691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7b4159c60 0x7fb7b415c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:08.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.848+0000 7fb7b9e75700 1 -- 172.21.15.163:0/578858691 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b0009420 con 0x7fb7b4159890 2023-10-09T06:31:08.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.848+0000 7fb7ba676700 1 --2- 172.21.15.163:0/578858691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7b415db10 0x7fb7b415cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:08.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.849+0000 7fb7b9e75700 1 --2- 172.21.15.163:0/578858691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7b4159890 0x7fb7b4151f80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4070e20 tx=0x7fb7b0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:08.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.849+0000 7fb7a9ffb700 1 -- 172.21.15.163:0/578858691 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b000bb90 con 0x7fb7b4159890 2023-10-09T06:31:08.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.849+0000 7fb7a9ffb700 1 -- 172.21.15.163:0/578858691 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b0011420 con 0x7fb7b4159890 2023-10-09T06:31:08.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.849+0000 7fb7bc0d9700 1 -- 172.21.15.163:0/578858691 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7b40cde00 con 0x7fb7b4159890 2023-10-09T06:31:08.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.849+0000 7fb7a9ffb700 1 -- 172.21.15.163:0/578858691 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b00117c0 con 0x7fb7b4159890 2023-10-09T06:31:08.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.849+0000 7fb7bc0d9700 1 -- 172.21.15.163:0/578858691 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b40b36d0 con 0x7fb7b4159890 2023-10-09T06:31:08.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.850+0000 7fb7bc0d9700 1 -- 172.21.15.163:0/578858691 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b404c3c0 con 0x7fb7b4159890 2023-10-09T06:31:08.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.850+0000 7fb7a9ffb700 1 -- 172.21.15.163:0/578858691 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb7b0011bf0 con 0x7fb7b4159890 2023-10-09T06:31:08.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.851+0000 7fb7a9ffb700 1 --2- 172.21.15.163:0/578858691 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb798066f30 0x7fb7980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:08.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.851+0000 7fb7a9ffb700 1 -- 172.21.15.163:0/578858691 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb7b0085e80 con 0x7fb7b4159890 2023-10-09T06:31:08.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.851+0000 7fb7b9674700 1 --2- 172.21.15.163:0/578858691 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb798066f30 0x7fb7980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:08.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.852+0000 7fb7b9674700 1 --2- 172.21.15.163:0/578858691 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb798066f30 0x7fb7980693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb7b40c93f0 tx=0x7fb7a4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:08.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:08.855+0000 7fb7a9ffb700 1 -- 172.21.15.163:0/578858691 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb7b00530f0 con 0x7fb7b4159890 2023-10-09T06:31:09.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.037+0000 7fb7bc0d9700 1 -- 172.21.15.163:0/578858691 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb7b406a5b0 con 0x7fb7b4159890 2023-10-09T06:31:09.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.037+0000 7fb7a9ffb700 1 -- 172.21.15.163:0/578858691 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb7b0056710 con 0x7fb7b4159890 2023-10-09T06:31:09.037 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 64 2023-10-09T06:31:09.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.039+0000 7fb7bc0d9700 1 -- 172.21.15.163:0/578858691 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb798066f30 msgr2=0x7fb7980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:09.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.039+0000 7fb7bc0d9700 1 --2- 172.21.15.163:0/578858691 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb798066f30 0x7fb7980693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb7b40c93f0 tx=0x7fb7a4008040 comp rx=0 tx=0).stop 2023-10-09T06:31:09.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.039+0000 7fb7bc0d9700 1 -- 172.21.15.163:0/578858691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7b4159890 msgr2=0x7fb7b4151f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:09.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.039+0000 7fb7bc0d9700 1 --2- 172.21.15.163:0/578858691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7b4159890 0x7fb7b4151f80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4070e20 tx=0x7fb7b0017000 comp rx=0 tx=0).stop 2023-10-09T06:31:09.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.040+0000 7fb7bc0d9700 1 -- 172.21.15.163:0/578858691 shutdown_connections 2023-10-09T06:31:09.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.040+0000 7fb7bc0d9700 1 --2- 172.21.15.163:0/578858691 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb798066f30 0x7fb7980693e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:09.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.040+0000 7fb7bc0d9700 1 --2- 172.21.15.163:0/578858691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7b4159890 0x7fb7b4151f80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:09.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.040+0000 7fb7bc0d9700 1 --2- 172.21.15.163:0/578858691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7b415db10 0x7fb7b415cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:09.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.040+0000 7fb7bc0d9700 1 --2- 172.21.15.163:0/578858691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7b4159c60 0x7fb7b415c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:09.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.040+0000 7fb7bc0d9700 1 -- 172.21.15.163:0/578858691 >> 172.21.15.163:0/578858691 conn(0x7fb7b4068e20 msgr2=0x7fb7b40c5590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:09.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.040+0000 7fb7bc0d9700 1 -- 172.21.15.163:0/578858691 shutdown_connections 2023-10-09T06:31:09.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.040+0000 7fb7bc0d9700 1 -- 172.21.15.163:0/578858691 wait complete. 2023-10-09T06:31:09.051 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:31:09.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.167+0000 7f98001f9700 1 Processor -- start 2023-10-09T06:31:09.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.167+0000 7f98001f9700 1 -- start start 2023-10-09T06:31:09.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.168+0000 7f98001f9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f97f80ccf90 0x7f97f80ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:09.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.168+0000 7f98001f9700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f80cc300 con 0x7f97f80b1420 2023-10-09T06:31:09.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.168+0000 7f98001f9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f80ad370 con 0x7f97f80cbf30 2023-10-09T06:31:09.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.168+0000 7f98001f9700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f8156bf0 con 0x7f97f80ccf90 2023-10-09T06:31:09.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.168+0000 7f97fe796700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f97f80ccf90 0x7f97f80ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:09.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.168+0000 7f97fd794700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f97f80b1420 0x7f97f80c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53180/0 (socket says 172.21.15.163:53180) 2023-10-09T06:31:09.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.168+0000 7f97fe796700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f97f80ccf90 0x7f97f80ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53076/0 (socket says 172.21.15.163:53076) 2023-10-09T06:31:09.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.168+0000 7f97fd794700 1 -- 172.21.15.163:0/1447226852 learned_addr learned my addr 172.21.15.163:0/1447226852 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:09.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.169+0000 7f97fe796700 1 -- 172.21.15.163:0/1447226852 >> v1:172.21.15.181:6789/0 conn(0x7f97f80cbf30 legacy=0x7f97f80b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:09.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.169+0000 7f97fe796700 1 -- 172.21.15.163:0/1447226852 >> v1:172.21.15.163:6789/0 conn(0x7f97f80b1420 legacy=0x7f97f80c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:09.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.169+0000 7f97fcf93700 1 -- 172.21.15.163:0/1447226852 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 497114538 0 0) 0x7f97f80cc300 con 0x7f97f80b1420 2023-10-09T06:31:09.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.169+0000 7f97fe796700 1 -- 172.21.15.163:0/1447226852 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f80ad650 con 0x7f97f80ccf90 2023-10-09T06:31:09.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.169+0000 7f97fe796700 1 --2- 172.21.15.163:0/1447226852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f80ccf90 0x7f97f80ae410 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f97f000bb90 tx=0x7f97f000b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d81ea3be26bbf72 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:09.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.169+0000 7f97fcf93700 1 -- 172.21.15.163:0/1447226852 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f000db60 con 0x7f97f80ccf90 2023-10-09T06:31:09.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.169+0000 7f97fcf93700 1 -- 172.21.15.163:0/1447226852 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f000dcc0 con 0x7f97f80ccf90 2023-10-09T06:31:09.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.169+0000 7f97fcf93700 1 -- 172.21.15.163:0/1447226852 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f00106a0 con 0x7f97f80ccf90 2023-10-09T06:31:09.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.169+0000 7f98001f9700 1 -- 172.21.15.163:0/1447226852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f80ccf90 msgr2=0x7f97f80ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:09.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.169+0000 7f98001f9700 1 --2- 172.21.15.163:0/1447226852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f80ccf90 0x7f97f80ae410 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f97f000bb90 tx=0x7f97f000b430 comp rx=0 tx=0).stop 2023-10-09T06:31:09.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.170+0000 7f98001f9700 1 -- 172.21.15.163:0/1447226852 shutdown_connections 2023-10-09T06:31:09.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.170+0000 7f98001f9700 1 --2- 172.21.15.163:0/1447226852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f80ccf90 0x7f97f80ae410 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:09.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.170+0000 7f98001f9700 1 -- 172.21.15.163:0/1447226852 >> 172.21.15.163:0/1447226852 conn(0x7f97f806a320 msgr2=0x7f97f80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:09.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.170+0000 7f98001f9700 1 -- 172.21.15.163:0/1447226852 shutdown_connections 2023-10-09T06:31:09.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.170+0000 7f98001f9700 1 -- 172.21.15.163:0/1447226852 wait complete. 2023-10-09T06:31:09.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.171+0000 7f98001f9700 1 Processor -- start 2023-10-09T06:31:09.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.171+0000 7f98001f9700 1 -- start start 2023-10-09T06:31:09.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.171+0000 7f98001f9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97f80b1420 0x7f97f8150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:09.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.171+0000 7f98001f9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97f80cbf30 0x7f97f8155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:09.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.171+0000 7f98001f9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f80ccf90 0x7f97f8156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:09.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.171+0000 7f98001f9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f80c1c00 con 0x7f97f80ccf90 2023-10-09T06:31:09.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.171+0000 7f98001f9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f8158c90 con 0x7f97f80cbf30 2023-10-09T06:31:09.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.171+0000 7f98001f9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f80c12d0 con 0x7f97f80b1420 2023-10-09T06:31:09.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.172+0000 7f97fe796700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f80ccf90 0x7f97f8156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:09.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.172+0000 7f97fe796700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f80ccf90 0x7f97f8156240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53092/0 (socket says 172.21.15.163:53092) 2023-10-09T06:31:09.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.172+0000 7f97fe796700 1 -- 172.21.15.163:0/4282760502 learned_addr learned my addr 172.21.15.163:0/4282760502 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:09.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.172+0000 7f97fdf95700 1 --2- 172.21.15.163:0/4282760502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97f80b1420 0x7f97f8150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:09.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.172+0000 7f97fd794700 1 --2- 172.21.15.163:0/4282760502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97f80cbf30 0x7f97f8155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:09.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.172+0000 7f97fe796700 1 -- 172.21.15.163:0/4282760502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97f80b1420 msgr2=0x7f97f8150f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:09.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.172+0000 7f97fe796700 1 --2- 172.21.15.163:0/4282760502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97f80b1420 0x7f97f8150f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:09.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.172+0000 7f97fe796700 1 -- 172.21.15.163:0/4282760502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97f80cbf30 msgr2=0x7f97f8155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:09.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.172+0000 7f97fe796700 1 --2- 172.21.15.163:0/4282760502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97f80cbf30 0x7f97f8155d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:09.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.172+0000 7f97fe796700 1 -- 172.21.15.163:0/4282760502 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f000b6d0 con 0x7f97f80ccf90 2023-10-09T06:31:09.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.172+0000 7f97fdf95700 1 --2- 172.21.15.163:0/4282760502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97f80b1420 0x7f97f8150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:09.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.172+0000 7f97fd794700 1 --2- 172.21.15.163:0/4282760502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97f80cbf30 0x7f97f8155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:09.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.172+0000 7f97fe796700 1 --2- 172.21.15.163:0/4282760502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f80ccf90 0x7f97f8156240 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f97f000b2c0 tx=0x7f97f001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:09.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.172+0000 7f97f5ffb700 1 -- 172.21.15.163:0/4282760502 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f000de30 con 0x7f97f80ccf90 2023-10-09T06:31:09.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.172+0000 7f97f5ffb700 1 -- 172.21.15.163:0/4282760502 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f0018a30 con 0x7f97f80ccf90 2023-10-09T06:31:09.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.172+0000 7f98001f9700 1 -- 172.21.15.163:0/4282760502 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f80c15b0 con 0x7f97f80ccf90 2023-10-09T06:31:09.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.172+0000 7f97f5ffb700 1 -- 172.21.15.163:0/4282760502 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f0018db0 con 0x7f97f80ccf90 2023-10-09T06:31:09.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.172+0000 7f98001f9700 1 -- 172.21.15.163:0/4282760502 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f8165c40 con 0x7f97f80ccf90 2023-10-09T06:31:09.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.173+0000 7f98001f9700 1 -- 172.21.15.163:0/4282760502 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f80c1d70 con 0x7f97f80ccf90 2023-10-09T06:31:09.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.174+0000 7f97f5ffb700 1 -- 172.21.15.163:0/4282760502 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f97f0021030 con 0x7f97f80ccf90 2023-10-09T06:31:09.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.174+0000 7f97f5ffb700 1 --2- 172.21.15.163:0/4282760502 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f97e4066f30 0x7f97e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:09.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.174+0000 7f97f5ffb700 1 -- 172.21.15.163:0/4282760502 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f97f0014070 con 0x7f97f80ccf90 2023-10-09T06:31:09.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.174+0000 7f97fdf95700 1 --2- 172.21.15.163:0/4282760502 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f97e4066f30 0x7f97e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:09.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.175+0000 7f97fdf95700 1 --2- 172.21.15.163:0/4282760502 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f97e4066f30 0x7f97e40693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f97e8007fd0 tx=0x7f97e8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:09.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.178+0000 7f97f5ffb700 1 -- 172.21.15.163:0/4282760502 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97f00557f0 con 0x7f97f80ccf90 2023-10-09T06:31:09.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.361+0000 7f98001f9700 1 -- 172.21.15.163:0/4282760502 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97f804c3c0 con 0x7f97f80ccf90 2023-10-09T06:31:09.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.362+0000 7f97f5ffb700 1 -- 172.21.15.163:0/4282760502 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f97f0058e10 con 0x7f97f80ccf90 2023-10-09T06:31:09.362 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:09.362 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":20,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:30:44.466923+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:30:41.369014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-09T06:30:42.103001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:31:09.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.364+0000 7f98001f9700 1 -- 172.21.15.163:0/4282760502 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f97e4066f30 msgr2=0x7f97e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:09.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.364+0000 7f98001f9700 1 --2- 172.21.15.163:0/4282760502 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f97e4066f30 0x7f97e40693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f97e8007fd0 tx=0x7f97e8009040 comp rx=0 tx=0).stop 2023-10-09T06:31:09.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.364+0000 7f98001f9700 1 -- 172.21.15.163:0/4282760502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f80ccf90 msgr2=0x7f97f8156240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:09.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.364+0000 7f98001f9700 1 --2- 172.21.15.163:0/4282760502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f80ccf90 0x7f97f8156240 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f97f000b2c0 tx=0x7f97f001a000 comp rx=0 tx=0).stop 2023-10-09T06:31:09.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.364+0000 7f98001f9700 1 -- 172.21.15.163:0/4282760502 shutdown_connections 2023-10-09T06:31:09.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.364+0000 7f98001f9700 1 --2- 172.21.15.163:0/4282760502 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f97e4066f30 0x7f97e40693e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:09.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.364+0000 7f98001f9700 1 --2- 172.21.15.163:0/4282760502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f80ccf90 0x7f97f8156240 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:09.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.364+0000 7f98001f9700 1 --2- 172.21.15.163:0/4282760502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97f80b1420 0x7f97f8150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:09.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.364+0000 7f98001f9700 1 --2- 172.21.15.163:0/4282760502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97f80cbf30 0x7f97f8155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:09.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.364+0000 7f98001f9700 1 -- 172.21.15.163:0/4282760502 >> 172.21.15.163:0/4282760502 conn(0x7f97f806a320 msgr2=0x7f97f806ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:09.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.364+0000 7f98001f9700 1 -- 172.21.15.163:0/4282760502 shutdown_connections 2023-10-09T06:31:09.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.364+0000 7f98001f9700 1 -- 172.21.15.163:0/4282760502 wait complete. 2023-10-09T06:31:09.377 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-09T06:30:41.369014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-09T06:30:42.103001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-09T06:31:09.377 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:31:09.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.495+0000 7f657ef24700 1 Processor -- start 2023-10-09T06:31:09.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.495+0000 7f657ef24700 1 -- start start 2023-10-09T06:31:09.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.496+0000 7f657ef24700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6578159a70 0x7f65780c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:09.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.496+0000 7f657ef24700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f657806bfa0 0x7f65780c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:09.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.496+0000 7f657ef24700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65780ca920 con 0x7f6578068390 2023-10-09T06:31:09.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.496+0000 7f657ef24700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f657806af00 con 0x7f657806bfa0 2023-10-09T06:31:09.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.496+0000 7f657ef24700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657806b1e0 con 0x7f6578159a70 2023-10-09T06:31:09.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.496+0000 7f657e723700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f6578068390 0x7f65780caec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53182/0 (socket says 172.21.15.163:53182) 2023-10-09T06:31:09.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.496+0000 7f657e723700 1 -- 172.21.15.163:0/586181183 learned_addr learned my addr 172.21.15.163:0/586181183 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:09.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.496+0000 7f657df22700 1 --2- 172.21.15.163:0/586181183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6578159a70 0x7f65780c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:09.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.496+0000 7f657d721700 1 --2- 172.21.15.163:0/586181183 >> v2:172.21.15.181:3300/0 conn(0x7f657806bfa0 0x7f65780c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:09.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.496+0000 7f657cf20700 1 -- 172.21.15.163:0/586181183 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 667032506 0 0) 0x7f65780ca920 con 0x7f6578068390 2023-10-09T06:31:09.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.496+0000 7f657cf20700 1 -- 172.21.15.163:0/586181183 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6564003570 con 0x7f6578068390 2023-10-09T06:31:09.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.496+0000 7f657d721700 1 -- 172.21.15.163:0/586181183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6578159a70 msgr2=0x7f65780c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:09.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.496+0000 7f657d721700 1 --2- 172.21.15.163:0/586181183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6578159a70 0x7f65780c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:09.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.496+0000 7f657d721700 1 -- 172.21.15.163:0/586181183 >> v1:172.21.15.163:6789/0 conn(0x7f6578068390 legacy=0x7f65780caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:09.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.496+0000 7f657d721700 1 -- 172.21.15.163:0/586181183 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657806b4c0 con 0x7f657806bfa0 2023-10-09T06:31:09.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.496+0000 7f657df22700 1 --2- 172.21.15.163:0/586181183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6578159a70 0x7f65780c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:31:09.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.497+0000 7f657d721700 1 --2- 172.21.15.163:0/586181183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f657806bfa0 0x7f65780c8830 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f65680098d0 tx=0x7f6568009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=659b8e43f8c59c20 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:09.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.497+0000 7f657cf20700 1 -- 172.21.15.163:0/586181183 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568005c10 con 0x7f657806bfa0 2023-10-09T06:31:09.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.497+0000 7f657cf20700 1 -- 172.21.15.163:0/586181183 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6568005d70 con 0x7f657806bfa0 2023-10-09T06:31:09.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.497+0000 7f657cf20700 1 -- 172.21.15.163:0/586181183 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568006280 con 0x7f657806bfa0 2023-10-09T06:31:09.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.497+0000 7f657ef24700 1 -- 172.21.15.163:0/586181183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f657806bfa0 msgr2=0x7f65780c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:09.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.497+0000 7f657ef24700 1 --2- 172.21.15.163:0/586181183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f657806bfa0 0x7f65780c8830 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f65680098d0 tx=0x7f6568009120 comp rx=0 tx=0).stop 2023-10-09T06:31:09.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.497+0000 7f657ef24700 1 -- 172.21.15.163:0/586181183 shutdown_connections 2023-10-09T06:31:09.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.497+0000 7f657ef24700 1 --2- 172.21.15.163:0/586181183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6578159a70 0x7f65780c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:09.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.497+0000 7f657ef24700 1 --2- 172.21.15.163:0/586181183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f657806bfa0 0x7f65780c8830 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:09.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.497+0000 7f657ef24700 1 -- 172.21.15.163:0/586181183 >> 172.21.15.163:0/586181183 conn(0x7f65780cddc0 msgr2=0x7f65780c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:09.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.497+0000 7f657ef24700 1 -- 172.21.15.163:0/586181183 shutdown_connections 2023-10-09T06:31:09.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.497+0000 7f657ef24700 1 -- 172.21.15.163:0/586181183 wait complete. 2023-10-09T06:31:09.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.498+0000 7f657ef24700 1 Processor -- start 2023-10-09T06:31:09.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.499+0000 7f657ef24700 1 -- start start 2023-10-09T06:31:09.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.499+0000 7f657ef24700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6578068390 0x7f65780cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:09.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.499+0000 7f657ef24700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f657806bfa0 0x7f6578151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:09.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.499+0000 7f657ef24700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6578159a70 0x7f6578152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:09.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.499+0000 7f657ef24700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65780b2980 con 0x7f6578068390 2023-10-09T06:31:09.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.499+0000 7f657ef24700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657816e430 con 0x7f6578159a70 2023-10-09T06:31:09.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.499+0000 7f657ef24700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65780b2330 con 0x7f657806bfa0 2023-10-09T06:31:09.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.499+0000 7f657df22700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6578068390 0x7f65780cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:09.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.499+0000 7f657df22700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6578068390 0x7f65780cbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53102/0 (socket says 172.21.15.163:53102) 2023-10-09T06:31:09.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.499+0000 7f657df22700 1 -- 172.21.15.163:0/191829380 learned_addr learned my addr 172.21.15.163:0/191829380 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:09.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.500+0000 7f657df22700 1 -- 172.21.15.163:0/191829380 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f657806bfa0 msgr2=0x7f6578151fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:09.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.500+0000 7f657df22700 1 --2- 172.21.15.163:0/191829380 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f657806bfa0 0x7f6578151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:09.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.500+0000 7f657e723700 1 --2- 172.21.15.163:0/191829380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6578159a70 0x7f6578152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:09.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.500+0000 7f657df22700 1 -- 172.21.15.163:0/191829380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6578159a70 msgr2=0x7f6578152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:09.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.500+0000 7f657df22700 1 --2- 172.21.15.163:0/191829380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6578159a70 0x7f6578152500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:09.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.500+0000 7f657df22700 1 -- 172.21.15.163:0/191829380 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6574009420 con 0x7f6578068390 2023-10-09T06:31:09.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.500+0000 7f657df22700 1 --2- 172.21.15.163:0/191829380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6578068390 0x7f65780cbce0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f657400ec80 tx=0x7f65740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:09.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.500+0000 7f656dffb700 1 -- 172.21.15.163:0/191829380 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657400c9d0 con 0x7f6578068390 2023-10-09T06:31:09.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.500+0000 7f656dffb700 1 -- 172.21.15.163:0/191829380 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657400cb30 con 0x7f6578068390 2023-10-09T06:31:09.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.500+0000 7f656dffb700 1 -- 172.21.15.163:0/191829380 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657400ceb0 con 0x7f6578068390 2023-10-09T06:31:09.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.500+0000 7f657ef24700 1 -- 172.21.15.163:0/191829380 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65680093c0 con 0x7f6578068390 2023-10-09T06:31:09.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.500+0000 7f657ef24700 1 -- 172.21.15.163:0/191829380 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65780cc220 con 0x7f6578068390 2023-10-09T06:31:09.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.501+0000 7f657ef24700 1 -- 172.21.15.163:0/191829380 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65780b2190 con 0x7f6578068390 2023-10-09T06:31:09.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.501+0000 7f656dffb700 1 -- 172.21.15.163:0/191829380 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6574013680 con 0x7f6578068390 2023-10-09T06:31:09.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.502+0000 7f656dffb700 1 --2- 172.21.15.163:0/191829380 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f655c066f30 0x7f655c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:09.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.502+0000 7f656dffb700 1 -- 172.21.15.163:0/191829380 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6574084980 con 0x7f6578068390 2023-10-09T06:31:09.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.502+0000 7f657d721700 1 --2- 172.21.15.163:0/191829380 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f655c066f30 0x7f655c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:09.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.503+0000 7f657d721700 1 --2- 172.21.15.163:0/191829380 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f655c066f30 0x7f655c0693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f65780b10c0 tx=0x7f6568016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:09.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.506+0000 7f656dffb700 1 -- 172.21.15.163:0/191829380 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f65780b2190 con 0x7f6578068390 2023-10-09T06:31:09.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.690+0000 7f657ef24700 1 -- 172.21.15.163:0/191829380 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f65780b2190 con 0x7f6578068390 2023-10-09T06:31:09.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.691+0000 7f656dffb700 1 -- 172.21.15.163:0/191829380 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f65780b2190 con 0x7f6578068390 2023-10-09T06:31:09.691 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:31:09.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.693+0000 7f657ef24700 1 -- 172.21.15.163:0/191829380 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f655c066f30 msgr2=0x7f655c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:09.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.693+0000 7f657ef24700 1 --2- 172.21.15.163:0/191829380 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f655c066f30 0x7f655c0693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f65780b10c0 tx=0x7f6568016000 comp rx=0 tx=0).stop 2023-10-09T06:31:09.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.693+0000 7f657ef24700 1 -- 172.21.15.163:0/191829380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6578068390 msgr2=0x7f65780cbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:09.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.693+0000 7f657ef24700 1 --2- 172.21.15.163:0/191829380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6578068390 0x7f65780cbce0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f657400ec80 tx=0x7f65740091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:09.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.693+0000 7f657ef24700 1 -- 172.21.15.163:0/191829380 shutdown_connections 2023-10-09T06:31:09.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.693+0000 7f657ef24700 1 --2- 172.21.15.163:0/191829380 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f655c066f30 0x7f655c0693e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:09.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.693+0000 7f657ef24700 1 --2- 172.21.15.163:0/191829380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6578068390 0x7f65780cbce0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:09.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.693+0000 7f657ef24700 1 --2- 172.21.15.163:0/191829380 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6578159a70 0x7f6578152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:09.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.693+0000 7f657ef24700 1 --2- 172.21.15.163:0/191829380 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f657806bfa0 0x7f6578151fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:09.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.693+0000 7f657ef24700 1 -- 172.21.15.163:0/191829380 >> 172.21.15.163:0/191829380 conn(0x7f65780cddc0 msgr2=0x7f6578153260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:09.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.693+0000 7f657ef24700 1 -- 172.21.15.163:0/191829380 shutdown_connections 2023-10-09T06:31:09.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.693+0000 7f657ef24700 1 -- 172.21.15.163:0/191829380 wait complete. 2023-10-09T06:31:09.704 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-09T06:31:09.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.822+0000 7f07a9766700 1 Processor -- start 2023-10-09T06:31:09.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.822+0000 7f07a9766700 1 -- start start 2023-10-09T06:31:09.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.823+0000 7f07a9766700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f07a4159890 0x7f07a415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:09.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.823+0000 7f07a9766700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f07a4069560 0x7f07a40699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:09.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.823+0000 7f07a9766700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07a4069ee0 con 0x7f07a4159c60 2023-10-09T06:31:09.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.823+0000 7f07a9766700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a415a160 con 0x7f07a4159890 2023-10-09T06:31:09.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.823+0000 7f07a9766700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a40d13a0 con 0x7f07a4069560 2023-10-09T06:31:09.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.823+0000 7f07a2ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f07a4159890 0x7f07a415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:09.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.823+0000 7f07a27fc700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f07a4159c60 0x7f07a415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53190/0 (socket says 172.21.15.163:53190) 2023-10-09T06:31:09.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.823+0000 7f07a27fc700 1 -- 172.21.15.163:0/587838865 learned_addr learned my addr 172.21.15.163:0/587838865 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:09.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.824+0000 7f07a37fe700 1 --2- 172.21.15.163:0/587838865 >> v2:172.21.15.181:3300/0 conn(0x7f07a4069560 0x7f07a40699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:09.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.824+0000 7f07a2ffd700 1 -- 172.21.15.163:0/587838865 >> v2:172.21.15.181:3300/0 conn(0x7f07a4069560 msgr2=0x7f07a40699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:09.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.824+0000 7f07a2ffd700 1 --2- 172.21.15.163:0/587838865 >> v2:172.21.15.181:3300/0 conn(0x7f07a4069560 0x7f07a40699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:09.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.824+0000 7f07a2ffd700 1 -- 172.21.15.163:0/587838865 >> v1:172.21.15.163:6789/0 conn(0x7f07a4159c60 legacy=0x7f07a415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:09.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.824+0000 7f07a1ffb700 1 -- 172.21.15.163:0/587838865 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3308315081 0 0) 0x7f07a4069ee0 con 0x7f07a4159c60 2023-10-09T06:31:09.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.824+0000 7f07a2ffd700 1 -- 172.21.15.163:0/587838865 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a415e140 con 0x7f07a4159890 2023-10-09T06:31:09.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.824+0000 7f07a2ffd700 1 --2- 172.21.15.163:0/587838865 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a4159890 0x7f07a415abd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f07a406a890 tx=0x7f0794009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52cc73888e6eafc server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:09.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.824+0000 7f07a1ffb700 1 -- 172.21.15.163:0/587838865 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079400e070 con 0x7f07a4159890 2023-10-09T06:31:09.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.824+0000 7f07a1ffb700 1 -- 172.21.15.163:0/587838865 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079400bc30 con 0x7f07a4159890 2023-10-09T06:31:09.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.824+0000 7f07a1ffb700 1 -- 172.21.15.163:0/587838865 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0794012430 con 0x7f07a4159890 2023-10-09T06:31:09.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.824+0000 7f07a9766700 1 -- 172.21.15.163:0/587838865 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a4159890 msgr2=0x7f07a415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:09.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.824+0000 7f07a9766700 1 --2- 172.21.15.163:0/587838865 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a4159890 0x7f07a415abd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f07a406a890 tx=0x7f0794009780 comp rx=0 tx=0).stop 2023-10-09T06:31:09.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.824+0000 7f07a9766700 1 -- 172.21.15.163:0/587838865 shutdown_connections 2023-10-09T06:31:09.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.825+0000 7f07a9766700 1 --2- 172.21.15.163:0/587838865 >> v2:172.21.15.181:3300/0 conn(0x7f07a4069560 0x7f07a40699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:09.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.825+0000 7f07a9766700 1 --2- 172.21.15.163:0/587838865 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a4159890 0x7f07a415abd0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:09.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.825+0000 7f07a9766700 1 -- 172.21.15.163:0/587838865 >> 172.21.15.163:0/587838865 conn(0x7f07a4068e20 msgr2=0x7f07a40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:09.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.825+0000 7f07a9766700 1 -- 172.21.15.163:0/587838865 shutdown_connections 2023-10-09T06:31:09.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.825+0000 7f07a9766700 1 -- 172.21.15.163:0/587838865 wait complete. 2023-10-09T06:31:09.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.826+0000 7f07a9766700 1 Processor -- start 2023-10-09T06:31:09.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.826+0000 7f07a9766700 1 -- start start 2023-10-09T06:31:09.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.827+0000 7f07a9766700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a4069560 0x7f07a415c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:09.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.827+0000 7f07a9766700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a4159890 0x7f07a4149c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:09.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.827+0000 7f07a9766700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07a4159c60 0x7f07a415ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:09.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.827+0000 7f07a9766700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a4153180 con 0x7f07a4069560 2023-10-09T06:31:09.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.827+0000 7f07a9766700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a40cd380 con 0x7f07a4159890 2023-10-09T06:31:09.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.827+0000 7f07a9766700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a414a160 con 0x7f07a4159c60 2023-10-09T06:31:09.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.827+0000 7f07a2ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a4069560 0x7f07a415c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:09.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.827+0000 7f07a2ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a4069560 0x7f07a415c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53122/0 (socket says 172.21.15.163:53122) 2023-10-09T06:31:09.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.827+0000 7f07a2ffd700 1 -- 172.21.15.163:0/34633306 learned_addr learned my addr 172.21.15.163:0/34633306 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:09.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.827+0000 7f07a27fc700 1 --2- 172.21.15.163:0/34633306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a4159890 0x7f07a4149c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:09.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.827+0000 7f07a37fe700 1 --2- 172.21.15.163:0/34633306 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07a4159c60 0x7f07a415ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:09.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.827+0000 7f07a2ffd700 1 -- 172.21.15.163:0/34633306 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07a4159c60 msgr2=0x7f07a415ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:09.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.827+0000 7f07a2ffd700 1 --2- 172.21.15.163:0/34633306 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07a4159c60 0x7f07a415ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:09.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.827+0000 7f07a2ffd700 1 -- 172.21.15.163:0/34633306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a4159890 msgr2=0x7f07a4149c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:09.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.827+0000 7f07a2ffd700 1 --2- 172.21.15.163:0/34633306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a4159890 0x7f07a4149c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:09.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.827+0000 7f07a2ffd700 1 -- 172.21.15.163:0/34633306 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0794009a20 con 0x7f07a4069560 2023-10-09T06:31:09.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.827+0000 7f07a27fc700 1 --2- 172.21.15.163:0/34633306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a4159890 0x7f07a4149c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:09.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.827+0000 7f07a2ffd700 1 --2- 172.21.15.163:0/34633306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a4069560 0x7f07a415c630 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f07940099f0 tx=0x7f0794019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:09.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.827+0000 7f078affd700 1 -- 172.21.15.163:0/34633306 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0794027470 con 0x7f07a4069560 2023-10-09T06:31:09.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.827+0000 7f078affd700 1 -- 172.21.15.163:0/34633306 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0794027610 con 0x7f07a4069560 2023-10-09T06:31:09.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.827+0000 7f07a9766700 1 -- 172.21.15.163:0/34633306 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a415cfb0 con 0x7f07a4069560 2023-10-09T06:31:09.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.827+0000 7f078affd700 1 -- 172.21.15.163:0/34633306 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07940279d0 con 0x7f07a4069560 2023-10-09T06:31:09.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.827+0000 7f07a9766700 1 -- 172.21.15.163:0/34633306 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a415d570 con 0x7f07a4069560 2023-10-09T06:31:09.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.828+0000 7f07a9766700 1 -- 172.21.15.163:0/34633306 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a404c3c0 con 0x7f07a4069560 2023-10-09T06:31:09.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.829+0000 7f078affd700 1 -- 172.21.15.163:0/34633306 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0794024070 con 0x7f07a4069560 2023-10-09T06:31:09.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.829+0000 7f078affd700 1 --2- 172.21.15.163:0/34633306 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0790066f30 0x7f07900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:09.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.829+0000 7f078affd700 1 -- 172.21.15.163:0/34633306 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0794086ca0 con 0x7f07a4069560 2023-10-09T06:31:09.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.829+0000 7f07a27fc700 1 --2- 172.21.15.163:0/34633306 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0790066f30 0x7f07900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:09.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.830+0000 7f07a27fc700 1 --2- 172.21.15.163:0/34633306 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0790066f30 0x7f07900693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f07a414abb0 tx=0x7f0798009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:09.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:09.833+0000 7f078affd700 1 -- 172.21.15.163:0/34633306 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0794053f10 con 0x7f07a4069560 2023-10-09T06:31:10.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.014+0000 7f07a9766700 1 -- 172.21.15.163:0/34633306 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f07a406a5b0 con 0x7f07a4069560 2023-10-09T06:31:10.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.015+0000 7f078affd700 1 -- 172.21.15.163:0/34633306 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0794057530 con 0x7f07a4069560 2023-10-09T06:31:10.015 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 32 2023-10-09T06:31:10.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.017+0000 7f07a9766700 1 -- 172.21.15.163:0/34633306 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0790066f30 msgr2=0x7f07900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:10.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.017+0000 7f07a9766700 1 --2- 172.21.15.163:0/34633306 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0790066f30 0x7f07900693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f07a414abb0 tx=0x7f0798009040 comp rx=0 tx=0).stop 2023-10-09T06:31:10.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.017+0000 7f07a9766700 1 -- 172.21.15.163:0/34633306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a4069560 msgr2=0x7f07a415c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:10.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.017+0000 7f07a9766700 1 --2- 172.21.15.163:0/34633306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a4069560 0x7f07a415c630 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f07940099f0 tx=0x7f0794019000 comp rx=0 tx=0).stop 2023-10-09T06:31:10.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.017+0000 7f07a9766700 1 -- 172.21.15.163:0/34633306 shutdown_connections 2023-10-09T06:31:10.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.017+0000 7f07a9766700 1 --2- 172.21.15.163:0/34633306 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0790066f30 0x7f07900693e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.017+0000 7f07a9766700 1 --2- 172.21.15.163:0/34633306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a4069560 0x7f07a415c630 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.017+0000 7f07a9766700 1 --2- 172.21.15.163:0/34633306 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07a4159c60 0x7f07a415ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.017+0000 7f07a9766700 1 --2- 172.21.15.163:0/34633306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a4159890 0x7f07a4149c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.017+0000 7f07a9766700 1 -- 172.21.15.163:0/34633306 >> 172.21.15.163:0/34633306 conn(0x7f07a4068e20 msgr2=0x7f07a40b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:10.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.017+0000 7f07a9766700 1 -- 172.21.15.163:0/34633306 shutdown_connections 2023-10-09T06:31:10.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.017+0000 7f07a9766700 1 -- 172.21.15.163:0/34633306 wait complete. 2023-10-09T06:31:10.029 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-09T06:31:10.154 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.146+0000 7fa9155cf700 1 Processor -- start 2023-10-09T06:31:10.154 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.146+0000 7fa9155cf700 1 -- start start 2023-10-09T06:31:10.154 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.147+0000 7fa9155cf700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa91006bcd0 0x7fa9100caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:10.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.147+0000 7fa9155cf700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa91006c0a0 0x7fa91015a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:10.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.147+0000 7fa9155cf700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa91015d7e0 con 0x7fa91015df50 2023-10-09T06:31:10.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.147+0000 7fa9155cf700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa91015dac0 con 0x7fa91006bcd0 2023-10-09T06:31:10.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.147+0000 7fa9155cf700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa91015aa30 con 0x7fa91006c0a0 2023-10-09T06:31:10.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.147+0000 7fa90effd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa91006bcd0 0x7fa9100caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:10.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.147+0000 7fa90f7fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fa91015df50 0x7fa9100c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53192/0 (socket says 172.21.15.163:53192) 2023-10-09T06:31:10.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.147+0000 7fa90f7fe700 1 -- 172.21.15.163:0/3257621130 learned_addr learned my addr 172.21.15.163:0/3257621130 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:10.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.147+0000 7fa90e7fc700 1 --2- 172.21.15.163:0/3257621130 >> v2:172.21.15.181:3300/0 conn(0x7fa91006c0a0 0x7fa91015a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:10.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.147+0000 7fa90effd700 1 -- 172.21.15.163:0/3257621130 >> v2:172.21.15.181:3300/0 conn(0x7fa91006c0a0 msgr2=0x7fa91015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:10.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.147+0000 7fa90effd700 1 --2- 172.21.15.163:0/3257621130 >> v2:172.21.15.181:3300/0 conn(0x7fa91006c0a0 0x7fa91015a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.147+0000 7fa90effd700 1 -- 172.21.15.163:0/3257621130 >> v1:172.21.15.163:6789/0 conn(0x7fa91015df50 legacy=0x7fa9100c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:10.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.147+0000 7fa90dffb700 1 -- 172.21.15.163:0/3257621130 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2957576898 0 0) 0x7fa91015d7e0 con 0x7fa91015df50 2023-10-09T06:31:10.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.147+0000 7fa90effd700 1 -- 172.21.15.163:0/3257621130 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91015ad10 con 0x7fa91006bcd0 2023-10-09T06:31:10.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.148+0000 7fa90effd700 1 --2- 172.21.15.163:0/3257621130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa91006bcd0 0x7fa9100caa90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa91006b4e0 tx=0x7fa900009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f4dcb823ba1b185 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:10.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.148+0000 7fa90dffb700 1 -- 172.21.15.163:0/3257621130 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90000e070 con 0x7fa91006bcd0 2023-10-09T06:31:10.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.148+0000 7fa90dffb700 1 -- 172.21.15.163:0/3257621130 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90000bb60 con 0x7fa91006bcd0 2023-10-09T06:31:10.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.148+0000 7fa90dffb700 1 -- 172.21.15.163:0/3257621130 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa900012430 con 0x7fa91006bcd0 2023-10-09T06:31:10.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.148+0000 7fa9155cf700 1 -- 172.21.15.163:0/3257621130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa91006bcd0 msgr2=0x7fa9100caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:10.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.148+0000 7fa9155cf700 1 --2- 172.21.15.163:0/3257621130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa91006bcd0 0x7fa9100caa90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa91006b4e0 tx=0x7fa900009780 comp rx=0 tx=0).stop 2023-10-09T06:31:10.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.148+0000 7fa9155cf700 1 -- 172.21.15.163:0/3257621130 shutdown_connections 2023-10-09T06:31:10.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.148+0000 7fa9155cf700 1 --2- 172.21.15.163:0/3257621130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa91006bcd0 0x7fa9100caa90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.148+0000 7fa9155cf700 1 --2- 172.21.15.163:0/3257621130 >> v2:172.21.15.181:3300/0 conn(0x7fa91006c0a0 0x7fa91015a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.148+0000 7fa9155cf700 1 -- 172.21.15.163:0/3257621130 >> 172.21.15.163:0/3257621130 conn(0x7fa9100ce600 msgr2=0x7fa9100cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:10.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.148+0000 7fa9155cf700 1 -- 172.21.15.163:0/3257621130 shutdown_connections 2023-10-09T06:31:10.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.148+0000 7fa9155cf700 1 -- 172.21.15.163:0/3257621130 wait complete. 2023-10-09T06:31:10.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.150+0000 7fa9155cf700 1 Processor -- start 2023-10-09T06:31:10.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.150+0000 7fa9155cf700 1 -- start start 2023-10-09T06:31:10.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.150+0000 7fa9155cf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa91006bcd0 0x7fa9100c33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:10.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.150+0000 7fa9155cf700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa91006c0a0 0x7fa91016df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:10.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.150+0000 7fa9155cf700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa91015df50 0x7fa9100b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:10.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.150+0000 7fa9155cf700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9100c4770 con 0x7fa91006c0a0 2023-10-09T06:31:10.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.150+0000 7fa9155cf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9100c38e0 con 0x7fa91006bcd0 2023-10-09T06:31:10.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.150+0000 7fa9155cf700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9100c3b60 con 0x7fa91015df50 2023-10-09T06:31:10.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.150+0000 7fa90e7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa91006c0a0 0x7fa91016df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:10.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.150+0000 7fa90e7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa91006c0a0 0x7fa91016df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53132/0 (socket says 172.21.15.163:53132) 2023-10-09T06:31:10.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.150+0000 7fa90e7fc700 1 -- 172.21.15.163:0/3126165731 learned_addr learned my addr 172.21.15.163:0/3126165731 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:10.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.151+0000 7fa90effd700 1 --2- 172.21.15.163:0/3126165731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa91006bcd0 0x7fa9100c33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:10.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.151+0000 7fa90f7fe700 1 --2- 172.21.15.163:0/3126165731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa91015df50 0x7fa9100b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:10.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.151+0000 7fa90e7fc700 1 -- 172.21.15.163:0/3126165731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa91015df50 msgr2=0x7fa9100b1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:10.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.151+0000 7fa90e7fc700 1 --2- 172.21.15.163:0/3126165731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa91015df50 0x7fa9100b1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.151+0000 7fa90e7fc700 1 -- 172.21.15.163:0/3126165731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa91006bcd0 msgr2=0x7fa9100c33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:10.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.151+0000 7fa90e7fc700 1 --2- 172.21.15.163:0/3126165731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa91006bcd0 0x7fa9100c33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.151+0000 7fa90e7fc700 1 -- 172.21.15.163:0/3126165731 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa900009a20 con 0x7fa91006c0a0 2023-10-09T06:31:10.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.151+0000 7fa90f7fe700 1 --2- 172.21.15.163:0/3126165731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa91015df50 0x7fa9100b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:10.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.151+0000 7fa90e7fc700 1 --2- 172.21.15.163:0/3126165731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa91006c0a0 0x7fa91016df20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa90400e980 tx=0x7fa9040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:10.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.151+0000 7fa8f6ffd700 1 -- 172.21.15.163:0/3126165731 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90400ee70 con 0x7fa91006c0a0 2023-10-09T06:31:10.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.151+0000 7fa8f6ffd700 1 -- 172.21.15.163:0/3126165731 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90400c850 con 0x7fa91006c0a0 2023-10-09T06:31:10.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.151+0000 7fa8f6ffd700 1 -- 172.21.15.163:0/3126165731 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90400cc50 con 0x7fa91006c0a0 2023-10-09T06:31:10.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.151+0000 7fa9155cf700 1 -- 172.21.15.163:0/3126165731 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9100c3e40 con 0x7fa91006c0a0 2023-10-09T06:31:10.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.151+0000 7fa9155cf700 1 -- 172.21.15.163:0/3126165731 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9100c4400 con 0x7fa91006c0a0 2023-10-09T06:31:10.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.152+0000 7fa9155cf700 1 -- 172.21.15.163:0/3126165731 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9100c4920 con 0x7fa91006c0a0 2023-10-09T06:31:10.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.152+0000 7fa8f6ffd700 1 -- 172.21.15.163:0/3126165731 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa904013680 con 0x7fa91006c0a0 2023-10-09T06:31:10.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.153+0000 7fa8f6ffd700 1 --2- 172.21.15.163:0/3126165731 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa8fc066f30 0x7fa8fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:10.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.153+0000 7fa8f6ffd700 1 -- 172.21.15.163:0/3126165731 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa904084710 con 0x7fa91006c0a0 2023-10-09T06:31:10.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.153+0000 7fa90effd700 1 --2- 172.21.15.163:0/3126165731 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa8fc066f30 0x7fa8fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:10.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.154+0000 7fa90effd700 1 --2- 172.21.15.163:0/3126165731 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa8fc066f30 0x7fa8fc0693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa91006b4e0 tx=0x7fa900019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:10.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.157+0000 7fa8f6ffd700 1 -- 172.21.15.163:0/3126165731 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa9100c4920 con 0x7fa91006c0a0 2023-10-09T06:31:10.338 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.338+0000 7fa9155cf700 1 -- 172.21.15.163:0/3126165731 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa9100c4920 con 0x7fa91006c0a0 2023-10-09T06:31:10.338 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.338+0000 7fa8f6ffd700 1 -- 172.21.15.163:0/3126165731 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa9100c4920 con 0x7fa91006c0a0 2023-10-09T06:31:10.338 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 64 2023-10-09T06:31:10.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.340+0000 7fa9155cf700 1 -- 172.21.15.163:0/3126165731 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa8fc066f30 msgr2=0x7fa8fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:10.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.340+0000 7fa9155cf700 1 --2- 172.21.15.163:0/3126165731 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa8fc066f30 0x7fa8fc0693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa91006b4e0 tx=0x7fa900019000 comp rx=0 tx=0).stop 2023-10-09T06:31:10.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.340+0000 7fa9155cf700 1 -- 172.21.15.163:0/3126165731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa91006c0a0 msgr2=0x7fa91016df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:10.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.340+0000 7fa9155cf700 1 --2- 172.21.15.163:0/3126165731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa91006c0a0 0x7fa91016df20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa90400e980 tx=0x7fa9040091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.340+0000 7fa9155cf700 1 -- 172.21.15.163:0/3126165731 shutdown_connections 2023-10-09T06:31:10.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.340+0000 7fa9155cf700 1 --2- 172.21.15.163:0/3126165731 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa8fc066f30 0x7fa8fc0693e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.340+0000 7fa9155cf700 1 --2- 172.21.15.163:0/3126165731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa91015df50 0x7fa9100b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.340+0000 7fa9155cf700 1 --2- 172.21.15.163:0/3126165731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa91006bcd0 0x7fa9100c33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.340+0000 7fa9155cf700 1 --2- 172.21.15.163:0/3126165731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa91006c0a0 0x7fa91016df20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.341 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.340+0000 7fa9155cf700 1 -- 172.21.15.163:0/3126165731 >> 172.21.15.163:0/3126165731 conn(0x7fa9100ce600 msgr2=0x7fa91015c360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:10.341 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.340+0000 7fa9155cf700 1 -- 172.21.15.163:0/3126165731 shutdown_connections 2023-10-09T06:31:10.341 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.340+0000 7fa9155cf700 1 -- 172.21.15.163:0/3126165731 wait complete. 2023-10-09T06:31:10.352 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:31:10.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.467+0000 7fe6af442700 1 Processor -- start 2023-10-09T06:31:10.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.468+0000 7fe6af442700 1 -- start start 2023-10-09T06:31:10.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.468+0000 7fe6af442700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe6a80c83c0 0x7fe6a8164690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:10.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.468+0000 7fe6af442700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6a806bdb0 con 0x7fe6a80aca80 2023-10-09T06:31:10.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.468+0000 7fe6af442700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6a80cb370 con 0x7fe6a8069d60 2023-10-09T06:31:10.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.468+0000 7fe6af442700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a8159f60 con 0x7fe6a80c83c0 2023-10-09T06:31:10.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.468+0000 7fe6ad9df700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe6a80c83c0 0x7fe6a8164690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:10.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.469+0000 7fe6ad1de700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fe6a80aca80 0x7fe6a806aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53204/0 (socket says 172.21.15.163:53204) 2023-10-09T06:31:10.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.469+0000 7fe6ad1de700 1 -- 172.21.15.163:0/776921697 learned_addr learned my addr 172.21.15.163:0/776921697 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:10.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.469+0000 7fe6ad9df700 1 -- 172.21.15.163:0/776921697 >> v1:172.21.15.181:6789/0 conn(0x7fe6a8069d60 legacy=0x7fe6a80c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:10.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.469+0000 7fe6ad9df700 1 -- 172.21.15.163:0/776921697 >> v1:172.21.15.163:6789/0 conn(0x7fe6a80aca80 legacy=0x7fe6a806aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:10.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.469+0000 7fe6ad9df700 1 -- 172.21.15.163:0/776921697 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a806b2b0 con 0x7fe6a80c83c0 2023-10-09T06:31:10.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.469+0000 7fe6a7fff700 1 -- 172.21.15.163:0/776921697 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2641275277 0 0) 0x7fe6a806bdb0 con 0x7fe6a80aca80 2023-10-09T06:31:10.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.469+0000 7fe6ad9df700 1 --2- 172.21.15.163:0/776921697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6a80c83c0 0x7fe6a8164690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe6a00098e0 tx=0x7fe6a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95648886a0eeaf29 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:10.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.469+0000 7fe6a7fff700 1 -- 172.21.15.163:0/776921697 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0009e80 con 0x7fe6a80c83c0 2023-10-09T06:31:10.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.469+0000 7fe6a7fff700 1 -- 172.21.15.163:0/776921697 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a000b860 con 0x7fe6a80c83c0 2023-10-09T06:31:10.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.469+0000 7fe6a7fff700 1 -- 172.21.15.163:0/776921697 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a000bbe0 con 0x7fe6a80c83c0 2023-10-09T06:31:10.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.470+0000 7fe6af442700 1 -- 172.21.15.163:0/776921697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6a80c83c0 msgr2=0x7fe6a8164690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:10.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.470+0000 7fe6af442700 1 --2- 172.21.15.163:0/776921697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6a80c83c0 0x7fe6a8164690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe6a00098e0 tx=0x7fe6a00091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.470+0000 7fe6af442700 1 -- 172.21.15.163:0/776921697 shutdown_connections 2023-10-09T06:31:10.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.470+0000 7fe6af442700 1 --2- 172.21.15.163:0/776921697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6a80c83c0 0x7fe6a8164690 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.470+0000 7fe6af442700 1 -- 172.21.15.163:0/776921697 >> 172.21.15.163:0/776921697 conn(0x7fe6a80cd670 msgr2=0x7fe6a80cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:10.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.470+0000 7fe6af442700 1 -- 172.21.15.163:0/776921697 shutdown_connections 2023-10-09T06:31:10.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.470+0000 7fe6af442700 1 -- 172.21.15.163:0/776921697 wait complete. 2023-10-09T06:31:10.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.471+0000 7fe6af442700 1 Processor -- start 2023-10-09T06:31:10.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.471+0000 7fe6af442700 1 -- start start 2023-10-09T06:31:10.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.472+0000 7fe6af442700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6a8069d60 0x7fe6a8157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:10.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.472+0000 7fe6af442700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6a80aca80 0x7fe6a80af060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:10.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.472+0000 7fe6af442700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6a80c83c0 0x7fe6a80b1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:10.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.472+0000 7fe6af442700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a80aebd0 con 0x7fe6a80c83c0 2023-10-09T06:31:10.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.472+0000 7fe6af442700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a80adfc0 con 0x7fe6a80aca80 2023-10-09T06:31:10.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.472+0000 7fe6af442700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a80ae2a0 con 0x7fe6a8069d60 2023-10-09T06:31:10.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.472+0000 7fe6ad9df700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6a80c83c0 0x7fe6a80b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:10.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.472+0000 7fe6ad9df700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6a80c83c0 0x7fe6a80b1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53162/0 (socket says 172.21.15.163:53162) 2023-10-09T06:31:10.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.472+0000 7fe6ad9df700 1 -- 172.21.15.163:0/1392353341 learned_addr learned my addr 172.21.15.163:0/1392353341 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:10.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.472+0000 7fe6ad1de700 1 --2- 172.21.15.163:0/1392353341 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6a8069d60 0x7fe6a8157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:10.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.472+0000 7fe6ac9dd700 1 --2- 172.21.15.163:0/1392353341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6a80aca80 0x7fe6a80af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:10.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.472+0000 7fe6ad9df700 1 -- 172.21.15.163:0/1392353341 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6a8069d60 msgr2=0x7fe6a8157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:10.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.472+0000 7fe6ad9df700 1 --2- 172.21.15.163:0/1392353341 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6a8069d60 0x7fe6a8157cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.472+0000 7fe6ad9df700 1 -- 172.21.15.163:0/1392353341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6a80aca80 msgr2=0x7fe6a80af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:10.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.472+0000 7fe6ad9df700 1 --2- 172.21.15.163:0/1392353341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6a80aca80 0x7fe6a80af060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.472+0000 7fe6ad9df700 1 -- 172.21.15.163:0/1392353341 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a0009420 con 0x7fe6a80c83c0 2023-10-09T06:31:10.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.472+0000 7fe6ad1de700 1 --2- 172.21.15.163:0/1392353341 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6a8069d60 0x7fe6a8157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:10.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.472+0000 7fe6ad9df700 1 --2- 172.21.15.163:0/1392353341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6a80c83c0 0x7fe6a80b1060 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0013000 tx=0x7fe6a0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:10.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.472+0000 7fe6a57fa700 1 -- 172.21.15.163:0/1392353341 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a001f070 con 0x7fe6a80c83c0 2023-10-09T06:31:10.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.473+0000 7fe6a57fa700 1 -- 172.21.15.163:0/1392353341 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a000baa0 con 0x7fe6a80c83c0 2023-10-09T06:31:10.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.473+0000 7fe6af442700 1 -- 172.21.15.163:0/1392353341 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a80ae580 con 0x7fe6a80c83c0 2023-10-09T06:31:10.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.473+0000 7fe6a57fa700 1 -- 172.21.15.163:0/1392353341 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0025670 con 0x7fe6a80c83c0 2023-10-09T06:31:10.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.473+0000 7fe6af442700 1 -- 172.21.15.163:0/1392353341 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a815d650 con 0x7fe6a80c83c0 2023-10-09T06:31:10.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.474+0000 7fe6af442700 1 -- 172.21.15.163:0/1392353341 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6a80aed80 con 0x7fe6a80c83c0 2023-10-09T06:31:10.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.474+0000 7fe6a57fa700 1 -- 172.21.15.163:0/1392353341 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe6a0023070 con 0x7fe6a80c83c0 2023-10-09T06:31:10.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.474+0000 7fe6a57fa700 1 --2- 172.21.15.163:0/1392353341 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe694066f30 0x7fe6940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:10.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.474+0000 7fe6a57fa700 1 -- 172.21.15.163:0/1392353341 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe6a0085c90 con 0x7fe6a80c83c0 2023-10-09T06:31:10.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.475+0000 7fe6ad1de700 1 --2- 172.21.15.163:0/1392353341 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe694066f30 0x7fe6940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:10.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.475+0000 7fe6ad1de700 1 --2- 172.21.15.163:0/1392353341 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe694066f30 0x7fe6940693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe698007fd0 tx=0x7fe698009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:10.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.479+0000 7fe6a57fa700 1 -- 172.21.15.163:0/1392353341 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe6a80aed80 con 0x7fe6a80c83c0 2023-10-09T06:31:10.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.664+0000 7fe6af442700 1 -- 172.21.15.163:0/1392353341 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6a804c3c0 con 0x7fe6a80c83c0 2023-10-09T06:31:10.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.665+0000 7fe6a57fa700 1 -- 172.21.15.163:0/1392353341 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fe6a0052f00 con 0x7fe6a80c83c0 2023-10-09T06:31:10.665 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:10.665 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":20,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:30:44.466923+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:30:41.369014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-09T06:30:42.103001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:31:10.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.668+0000 7fe6af442700 1 -- 172.21.15.163:0/1392353341 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe694066f30 msgr2=0x7fe6940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:10.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.668+0000 7fe6af442700 1 --2- 172.21.15.163:0/1392353341 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe694066f30 0x7fe6940693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe698007fd0 tx=0x7fe698009040 comp rx=0 tx=0).stop 2023-10-09T06:31:10.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.668+0000 7fe6af442700 1 -- 172.21.15.163:0/1392353341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6a80c83c0 msgr2=0x7fe6a80b1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:10.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.668+0000 7fe6af442700 1 --2- 172.21.15.163:0/1392353341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6a80c83c0 0x7fe6a80b1060 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0013000 tx=0x7fe6a0017000 comp rx=0 tx=0).stop 2023-10-09T06:31:10.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.668+0000 7fe6af442700 1 -- 172.21.15.163:0/1392353341 shutdown_connections 2023-10-09T06:31:10.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.668+0000 7fe6af442700 1 --2- 172.21.15.163:0/1392353341 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe694066f30 0x7fe6940693e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.668+0000 7fe6af442700 1 --2- 172.21.15.163:0/1392353341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6a80c83c0 0x7fe6a80b1060 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.668+0000 7fe6af442700 1 --2- 172.21.15.163:0/1392353341 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6a8069d60 0x7fe6a8157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.668+0000 7fe6af442700 1 --2- 172.21.15.163:0/1392353341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6a80aca80 0x7fe6a80af060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.668+0000 7fe6af442700 1 -- 172.21.15.163:0/1392353341 >> 172.21.15.163:0/1392353341 conn(0x7fe6a80cd670 msgr2=0x7fe6a80c27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:10.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.668+0000 7fe6af442700 1 -- 172.21.15.163:0/1392353341 shutdown_connections 2023-10-09T06:31:10.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.668+0000 7fe6af442700 1 -- 172.21.15.163:0/1392353341 wait complete. 2023-10-09T06:31:10.680 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-09T06:30:41.369014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-09T06:30:42.103001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-09T06:31:10.680 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:31:10.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.798+0000 7f27ef154700 1 Processor -- start 2023-10-09T06:31:10.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.798+0000 7f27ef154700 1 -- start start 2023-10-09T06:31:10.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.799+0000 7f27ef154700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f27e806aa40 0x7f27e80cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:10.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.799+0000 7f27ef154700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f27e80c91e0 0x7f27e8160390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:10.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.799+0000 7f27ef154700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27e80cc210 con 0x7f27e806bac0 2023-10-09T06:31:10.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.799+0000 7f27ef154700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e80cc4f0 con 0x7f27e806aa40 2023-10-09T06:31:10.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.799+0000 7f27ef154700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e80cc7d0 con 0x7f27e80c91e0 2023-10-09T06:31:10.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.799+0000 7f27ecef0700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f27e806aa40 0x7f27e80cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:10.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.799+0000 7f27ecef0700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f27e806aa40 0x7f27e80cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53172/0 (socket says 172.21.15.163:53172) 2023-10-09T06:31:10.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.799+0000 7f27ecef0700 1 -- 172.21.15.163:0/1335577523 learned_addr learned my addr 172.21.15.163:0/1335577523 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:10.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.799+0000 7f27e7fff700 1 --1- 172.21.15.163:0/1335577523 >> v1:172.21.15.163:6789/0 conn(0x7f27e806bac0 0x7f27e806ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53212/0 (socket says 172.21.15.163:53212) 2023-10-09T06:31:10.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.800+0000 7f27ed6f1700 1 --2- 172.21.15.163:0/1335577523 >> v2:172.21.15.181:3300/0 conn(0x7f27e80c91e0 0x7f27e8160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:10.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.800+0000 7f27ecef0700 1 -- 172.21.15.163:0/1335577523 >> v2:172.21.15.181:3300/0 conn(0x7f27e80c91e0 msgr2=0x7f27e8160390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:10.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.800+0000 7f27ecef0700 1 --2- 172.21.15.163:0/1335577523 >> v2:172.21.15.181:3300/0 conn(0x7f27e80c91e0 0x7f27e8160390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.800+0000 7f27ecef0700 1 -- 172.21.15.163:0/1335577523 >> v1:172.21.15.163:6789/0 conn(0x7f27e806bac0 legacy=0x7f27e806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:10.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.800+0000 7f27ecef0700 1 -- 172.21.15.163:0/1335577523 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e80ccab0 con 0x7f27e806aa40 2023-10-09T06:31:10.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.800+0000 7f27e77fe700 1 -- 172.21.15.163:0/1335577523 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1999068487 0 0) 0x7f27e80cc210 con 0x7f27e806bac0 2023-10-09T06:31:10.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.800+0000 7f27ed6f1700 1 --2- 172.21.15.163:0/1335577523 >> v2:172.21.15.181:3300/0 conn(0x7f27e80c91e0 0x7f27e8160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:10.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.800+0000 7f27ecef0700 1 --2- 172.21.15.163:0/1335577523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27e806aa40 0x7f27e80cbc00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f27e806e510 tx=0x7f27d8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af8188980463408 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:10.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.800+0000 7f27e77fe700 1 -- 172.21.15.163:0/1335577523 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d800e070 con 0x7f27e806aa40 2023-10-09T06:31:10.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.800+0000 7f27e77fe700 1 -- 172.21.15.163:0/1335577523 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d800bad0 con 0x7f27e806aa40 2023-10-09T06:31:10.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.800+0000 7f27e77fe700 1 -- 172.21.15.163:0/1335577523 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d80092e0 con 0x7f27e806aa40 2023-10-09T06:31:10.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.800+0000 7f27ef154700 1 -- 172.21.15.163:0/1335577523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27e806aa40 msgr2=0x7f27e80cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:10.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.800+0000 7f27ef154700 1 --2- 172.21.15.163:0/1335577523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27e806aa40 0x7f27e80cbc00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f27e806e510 tx=0x7f27d8009840 comp rx=0 tx=0).stop 2023-10-09T06:31:10.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.801+0000 7f27ef154700 1 -- 172.21.15.163:0/1335577523 shutdown_connections 2023-10-09T06:31:10.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.801+0000 7f27ef154700 1 --2- 172.21.15.163:0/1335577523 >> v2:172.21.15.181:3300/0 conn(0x7f27e80c91e0 0x7f27e8160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.801+0000 7f27ef154700 1 --2- 172.21.15.163:0/1335577523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27e806aa40 0x7f27e80cbc00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.801+0000 7f27ef154700 1 -- 172.21.15.163:0/1335577523 >> 172.21.15.163:0/1335577523 conn(0x7f27e80cd670 msgr2=0x7f27e80cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:10.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.801+0000 7f27ef154700 1 -- 172.21.15.163:0/1335577523 shutdown_connections 2023-10-09T06:31:10.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.801+0000 7f27ef154700 1 -- 172.21.15.163:0/1335577523 wait complete. 2023-10-09T06:31:10.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.802+0000 7f27ef154700 1 Processor -- start 2023-10-09T06:31:10.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.802+0000 7f27ef154700 1 -- start start 2023-10-09T06:31:10.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.802+0000 7f27ef154700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27e806aa40 0x7f27e814e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:10.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.802+0000 7f27ef154700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27e806bac0 0x7f27e8068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:10.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.803+0000 7f27ef154700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27e80c91e0 0x7f27e80688d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:10.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.803+0000 7f27ef154700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e814e8f0 con 0x7f27e806aa40 2023-10-09T06:31:10.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.803+0000 7f27ef154700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e814fc00 con 0x7f27e806bac0 2023-10-09T06:31:10.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.803+0000 7f27ef154700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e814eff0 con 0x7f27e80c91e0 2023-10-09T06:31:10.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.803+0000 7f27ecef0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27e806aa40 0x7f27e814e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:10.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.803+0000 7f27ecef0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27e806aa40 0x7f27e814e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53180/0 (socket says 172.21.15.163:53180) 2023-10-09T06:31:10.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.803+0000 7f27ecef0700 1 -- 172.21.15.163:0/105715191 learned_addr learned my addr 172.21.15.163:0/105715191 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:10.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.803+0000 7f27e7fff700 1 --2- 172.21.15.163:0/105715191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27e806bac0 0x7f27e8068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:10.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.803+0000 7f27ed6f1700 1 --2- 172.21.15.163:0/105715191 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27e80c91e0 0x7f27e80688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:10.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.803+0000 7f27ecef0700 1 -- 172.21.15.163:0/105715191 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27e80c91e0 msgr2=0x7f27e80688d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:10.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.803+0000 7f27ecef0700 1 --2- 172.21.15.163:0/105715191 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27e80c91e0 0x7f27e80688d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.803+0000 7f27ecef0700 1 -- 172.21.15.163:0/105715191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27e806bac0 msgr2=0x7f27e8068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:10.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.803+0000 7f27ecef0700 1 --2- 172.21.15.163:0/105715191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27e806bac0 0x7f27e8068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.803+0000 7f27ecef0700 1 -- 172.21.15.163:0/105715191 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d8009ae0 con 0x7f27e806aa40 2023-10-09T06:31:10.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.803+0000 7f27ecef0700 1 --2- 172.21.15.163:0/105715191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27e806aa40 0x7f27e814e3b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f27d8012000 tx=0x7f27d8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:10.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.803+0000 7f27e4ff9700 1 -- 172.21.15.163:0/105715191 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d8008be0 con 0x7f27e806aa40 2023-10-09T06:31:10.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.803+0000 7f27e4ff9700 1 -- 172.21.15.163:0/105715191 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d8008d60 con 0x7f27e806aa40 2023-10-09T06:31:10.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.803+0000 7f27ef154700 1 -- 172.21.15.163:0/105715191 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e814f2d0 con 0x7f27e806aa40 2023-10-09T06:31:10.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.803+0000 7f27e4ff9700 1 -- 172.21.15.163:0/105715191 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d80266b0 con 0x7f27e806aa40 2023-10-09T06:31:10.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.803+0000 7f27ef154700 1 -- 172.21.15.163:0/105715191 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e814f890 con 0x7f27e806aa40 2023-10-09T06:31:10.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.804+0000 7f27ef154700 1 -- 172.21.15.163:0/105715191 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e806f7e0 con 0x7f27e806aa40 2023-10-09T06:31:10.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.804+0000 7f27e4ff9700 1 -- 172.21.15.163:0/105715191 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f27d8003040 con 0x7f27e806aa40 2023-10-09T06:31:10.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.805+0000 7f27e4ff9700 1 --2- 172.21.15.163:0/105715191 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f27d4066f30 0x7f27d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:10.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.805+0000 7f27e4ff9700 1 -- 172.21.15.163:0/105715191 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f27d8086570 con 0x7f27e806aa40 2023-10-09T06:31:10.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.805+0000 7f27e7fff700 1 --2- 172.21.15.163:0/105715191 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f27d4066f30 0x7f27d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:10.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.805+0000 7f27e7fff700 1 --2- 172.21.15.163:0/105715191 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f27d4066f30 0x7f27d40693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f27e816d5e0 tx=0x7f27dc007bf0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:10.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.809+0000 7f27e4ff9700 1 -- 172.21.15.163:0/105715191 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f27d80537e0 con 0x7f27e806aa40 2023-10-09T06:31:10.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.991+0000 7f27ef154700 1 -- 172.21.15.163:0/105715191 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f27e804c3c0 con 0x7f27e806aa40 2023-10-09T06:31:10.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.992+0000 7f27e4ff9700 1 -- 172.21.15.163:0/105715191 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f27d8009940 con 0x7f27e806aa40 2023-10-09T06:31:10.992 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:31:10.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.994+0000 7f27ef154700 1 -- 172.21.15.163:0/105715191 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f27d4066f30 msgr2=0x7f27d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:10.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.994+0000 7f27ef154700 1 --2- 172.21.15.163:0/105715191 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f27d4066f30 0x7f27d40693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f27e816d5e0 tx=0x7f27dc007bf0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.994+0000 7f27ef154700 1 -- 172.21.15.163:0/105715191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27e806aa40 msgr2=0x7f27e814e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:10.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.994+0000 7f27ef154700 1 --2- 172.21.15.163:0/105715191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27e806aa40 0x7f27e814e3b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f27d8012000 tx=0x7f27d8018000 comp rx=0 tx=0).stop 2023-10-09T06:31:10.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.994+0000 7f27ef154700 1 -- 172.21.15.163:0/105715191 shutdown_connections 2023-10-09T06:31:10.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.994+0000 7f27ef154700 1 --2- 172.21.15.163:0/105715191 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f27d4066f30 0x7f27d40693e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.994+0000 7f27ef154700 1 --2- 172.21.15.163:0/105715191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27e806aa40 0x7f27e814e3b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.994+0000 7f27ef154700 1 --2- 172.21.15.163:0/105715191 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27e80c91e0 0x7f27e80688d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.994+0000 7f27ef154700 1 --2- 172.21.15.163:0/105715191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27e806bac0 0x7f27e8068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:10.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.994+0000 7f27ef154700 1 -- 172.21.15.163:0/105715191 >> 172.21.15.163:0/105715191 conn(0x7f27e80cd670 msgr2=0x7f27e8152570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:10.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.994+0000 7f27ef154700 1 -- 172.21.15.163:0/105715191 shutdown_connections 2023-10-09T06:31:10.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:10.994+0000 7f27ef154700 1 -- 172.21.15.163:0/105715191 wait complete. 2023-10-09T06:31:11.006 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-09T06:31:11.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.124+0000 7f287e94a700 1 Processor -- start 2023-10-09T06:31:11.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.124+0000 7f287e94a700 1 -- start start 2023-10-09T06:31:11.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.125+0000 7f287e94a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2878159890 0x7f287815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:11.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.125+0000 7f287e94a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2878159c60 0x7f287815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:11.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.125+0000 7f287e94a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f287815a160 con 0x7f287815db10 2023-10-09T06:31:11.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.125+0000 7f287e94a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28780d13a0 con 0x7f2878159c60 2023-10-09T06:31:11.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.125+0000 7f287e94a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287815e140 con 0x7f2878159890 2023-10-09T06:31:11.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.125+0000 7f28777fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2878159c60 0x7f287815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:11.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.125+0000 7f28777fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2878159c60 0x7f287815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53184/0 (socket says 172.21.15.163:53184) 2023-10-09T06:31:11.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.125+0000 7f28777fe700 1 -- 172.21.15.163:0/1465591261 learned_addr learned my addr 172.21.15.163:0/1465591261 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:11.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.125+0000 7f2877fff700 1 --2- 172.21.15.163:0/1465591261 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2878159890 0x7f287815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:11.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.125+0000 7f28777fe700 1 -- 172.21.15.163:0/1465591261 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2878159890 msgr2=0x7f287815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:11.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.125+0000 7f28777fe700 1 --2- 172.21.15.163:0/1465591261 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2878159890 0x7f287815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:11.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.125+0000 7f28777fe700 1 -- 172.21.15.163:0/1465591261 >> v1:172.21.15.181:6789/0 conn(0x7f287815db10 legacy=0x7f2878069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:11.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.125+0000 7f28777fe700 1 -- 172.21.15.163:0/1465591261 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287815a3d0 con 0x7f2878159c60 2023-10-09T06:31:11.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.125+0000 7f28777fe700 1 --2- 172.21.15.163:0/1465591261 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2878159c60 0x7f287815d5d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f28680099b0 tx=0x7f2868009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6635cb409290733b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:11.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.125+0000 7f2876ffd700 1 -- 172.21.15.163:0/1465591261 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2868009ea0 con 0x7f2878159c60 2023-10-09T06:31:11.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.126+0000 7f2876ffd700 1 -- 172.21.15.163:0/1465591261 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f286800b860 con 0x7f2878159c60 2023-10-09T06:31:11.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.126+0000 7f2876ffd700 1 -- 172.21.15.163:0/1465591261 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286800bc00 con 0x7f2878159c60 2023-10-09T06:31:11.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.126+0000 7f287e94a700 1 -- 172.21.15.163:0/1465591261 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2878159c60 msgr2=0x7f287815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:11.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.126+0000 7f287e94a700 1 --2- 172.21.15.163:0/1465591261 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2878159c60 0x7f287815d5d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f28680099b0 tx=0x7f2868009250 comp rx=0 tx=0).stop 2023-10-09T06:31:11.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.126+0000 7f287e94a700 1 -- 172.21.15.163:0/1465591261 shutdown_connections 2023-10-09T06:31:11.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.126+0000 7f287e94a700 1 --2- 172.21.15.163:0/1465591261 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2878159890 0x7f287815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:11.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.126+0000 7f287e94a700 1 --2- 172.21.15.163:0/1465591261 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2878159c60 0x7f287815d5d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:11.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.126+0000 7f287e94a700 1 -- 172.21.15.163:0/1465591261 >> 172.21.15.163:0/1465591261 conn(0x7f2878068e20 msgr2=0x7f28780d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:11.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.126+0000 7f287e94a700 1 -- 172.21.15.163:0/1465591261 shutdown_connections 2023-10-09T06:31:11.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.126+0000 7f287e94a700 1 -- 172.21.15.163:0/1465591261 wait complete. 2023-10-09T06:31:11.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.127+0000 7f287e94a700 1 Processor -- start 2023-10-09T06:31:11.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.127+0000 7f287e94a700 1 -- start start 2023-10-09T06:31:11.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.128+0000 7f287e94a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2878159890 0x7f2878148160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:11.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.128+0000 7f287e94a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2878159c60 0x7f28780b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:11.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.128+0000 7f287e94a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f287815db10 0x7f28780b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:11.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.128+0000 7f287e94a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2878153770 con 0x7f2878159c60 2023-10-09T06:31:11.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.128+0000 7f287e94a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287816a310 con 0x7f2878159890 2023-10-09T06:31:11.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.128+0000 7f287e94a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28780b62d0 con 0x7f287815db10 2023-10-09T06:31:11.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.128+0000 7f28777fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2878159c60 0x7f28780b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:11.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.128+0000 7f28777fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2878159c60 0x7f28780b56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53198/0 (socket says 172.21.15.163:53198) 2023-10-09T06:31:11.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.128+0000 7f28777fe700 1 -- 172.21.15.163:0/1374497354 learned_addr learned my addr 172.21.15.163:0/1374497354 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:11.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.128+0000 7f287cee7700 1 --2- 172.21.15.163:0/1374497354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f287815db10 0x7f28780b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:11.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.128+0000 7f28777fe700 1 -- 172.21.15.163:0/1374497354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f287815db10 msgr2=0x7f28780b5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:11.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.128+0000 7f28777fe700 1 --2- 172.21.15.163:0/1374497354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f287815db10 0x7f28780b5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:11.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.128+0000 7f28777fe700 1 -- 172.21.15.163:0/1374497354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2878159890 msgr2=0x7f2878148160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:11.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.128+0000 7f28777fe700 1 --2- 172.21.15.163:0/1374497354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2878159890 0x7f2878148160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:11.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.128+0000 7f28777fe700 1 -- 172.21.15.163:0/1374497354 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28680094f0 con 0x7f2878159c60 2023-10-09T06:31:11.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.129+0000 7f28777fe700 1 --2- 172.21.15.163:0/1374497354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2878159c60 0x7f28780b56f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f28780899c0 tx=0x7f2868017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:11.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.129+0000 7f285ffff700 1 -- 172.21.15.163:0/1374497354 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286800bbe0 con 0x7f2878159c60 2023-10-09T06:31:11.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.129+0000 7f285ffff700 1 -- 172.21.15.163:0/1374497354 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2868011420 con 0x7f2878159c60 2023-10-09T06:31:11.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.129+0000 7f287e94a700 1 -- 172.21.15.163:0/1374497354 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2878169cc0 con 0x7f2878159c60 2023-10-09T06:31:11.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.129+0000 7f285ffff700 1 -- 172.21.15.163:0/1374497354 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28680117a0 con 0x7f2878159c60 2023-10-09T06:31:11.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.129+0000 7f287e94a700 1 -- 172.21.15.163:0/1374497354 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28781699b0 con 0x7f2878159c60 2023-10-09T06:31:11.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.130+0000 7f287e94a700 1 -- 172.21.15.163:0/1374497354 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f287804c3c0 con 0x7f2878159c60 2023-10-09T06:31:11.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.130+0000 7f285ffff700 1 -- 172.21.15.163:0/1374497354 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2868011bd0 con 0x7f2878159c60 2023-10-09T06:31:11.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.131+0000 7f285ffff700 1 --2- 172.21.15.163:0/1374497354 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2864066f30 0x7f28640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:11.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.131+0000 7f285ffff700 1 -- 172.21.15.163:0/1374497354 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2868085ec0 con 0x7f2878159c60 2023-10-09T06:31:11.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.131+0000 7f2877fff700 1 --2- 172.21.15.163:0/1374497354 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2864066f30 0x7f28640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:11.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.131+0000 7f2877fff700 1 --2- 172.21.15.163:0/1374497354 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2864066f30 0x7f28640693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f286c009bb0 tx=0x7f286c0091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:11.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.135+0000 7f285ffff700 1 -- 172.21.15.163:0/1374497354 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2868053130 con 0x7f2878159c60 2023-10-09T06:31:11.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.315+0000 7f287e94a700 1 -- 172.21.15.163:0/1374497354 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f287816a480 con 0x7f2878159c60 2023-10-09T06:31:11.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.315+0000 7f285ffff700 1 -- 172.21.15.163:0/1374497354 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2868056750 con 0x7f2878159c60 2023-10-09T06:31:11.315 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 32 2023-10-09T06:31:11.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.318+0000 7f287e94a700 1 -- 172.21.15.163:0/1374497354 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2864066f30 msgr2=0x7f28640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:11.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.318+0000 7f287e94a700 1 --2- 172.21.15.163:0/1374497354 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2864066f30 0x7f28640693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f286c009bb0 tx=0x7f286c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:11.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.318+0000 7f287e94a700 1 -- 172.21.15.163:0/1374497354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2878159c60 msgr2=0x7f28780b56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:11.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.318+0000 7f287e94a700 1 --2- 172.21.15.163:0/1374497354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2878159c60 0x7f28780b56f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f28780899c0 tx=0x7f2868017000 comp rx=0 tx=0).stop 2023-10-09T06:31:11.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.318+0000 7f287e94a700 1 -- 172.21.15.163:0/1374497354 shutdown_connections 2023-10-09T06:31:11.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.318+0000 7f287e94a700 1 --2- 172.21.15.163:0/1374497354 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2864066f30 0x7f28640693e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:11.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.318+0000 7f287e94a700 1 --2- 172.21.15.163:0/1374497354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f287815db10 0x7f28780b5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:11.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.318+0000 7f287e94a700 1 --2- 172.21.15.163:0/1374497354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2878159890 0x7f2878148160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:11.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.318+0000 7f287e94a700 1 --2- 172.21.15.163:0/1374497354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2878159c60 0x7f28780b56f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:11.319 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.318+0000 7f287e94a700 1 -- 172.21.15.163:0/1374497354 >> 172.21.15.163:0/1374497354 conn(0x7f2878068e20 msgr2=0x7f2878154310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:11.319 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.318+0000 7f287e94a700 1 -- 172.21.15.163:0/1374497354 shutdown_connections 2023-10-09T06:31:11.319 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.318+0000 7f287e94a700 1 -- 172.21.15.163:0/1374497354 wait complete. 2023-10-09T06:31:11.329 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-09T06:31:11.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.447+0000 7f1a5359e700 1 Processor -- start 2023-10-09T06:31:11.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.447+0000 7f1a5359e700 1 -- start start 2023-10-09T06:31:11.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.447+0000 7f1a5359e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1a540b4e70 0x7f1a540ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:11.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.448+0000 7f1a5359e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a540cfb00 con 0x7f1a540b1b40 2023-10-09T06:31:11.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.448+0000 7f1a5359e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a5415e060 con 0x7f1a540b16d0 2023-10-09T06:31:11.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.448+0000 7f1a5359e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a540b5c40 con 0x7f1a540b4e70 2023-10-09T06:31:11.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.448+0000 7f1a52d9d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f1a540b1b40 0x7f1a5415d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53228/0 (socket says 172.21.15.163:53228) 2023-10-09T06:31:11.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.448+0000 7f1a52d9d700 1 -- 172.21.15.163:0/3761703749 learned_addr learned my addr 172.21.15.163:0/3761703749 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:11.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.448+0000 7f1a5259c700 1 --2- 172.21.15.163:0/3761703749 >> v2:172.21.15.181:3300/0 conn(0x7f1a540b4e70 0x7f1a540ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:11.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.448+0000 7f1a51d9b700 1 -- 172.21.15.163:0/3761703749 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3711369955 0 0) 0x7f1a540cfb00 con 0x7f1a540b1b40 2023-10-09T06:31:11.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.448+0000 7f1a51d9b700 1 -- 172.21.15.163:0/3761703749 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a40003570 con 0x7f1a540b1b40 2023-10-09T06:31:11.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.448+0000 7f1a5259c700 1 -- 172.21.15.163:0/3761703749 >> v1:172.21.15.181:6789/0 conn(0x7f1a540b16d0 legacy=0x7f1a5415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:11.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.448+0000 7f1a5259c700 1 -- 172.21.15.163:0/3761703749 >> v1:172.21.15.163:6789/0 conn(0x7f1a540b1b40 legacy=0x7f1a5415d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:11.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.448+0000 7f1a5259c700 1 -- 172.21.15.163:0/3761703749 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a540b5f20 con 0x7f1a540b4e70 2023-10-09T06:31:11.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.448+0000 7f1a5259c700 1 --2- 172.21.15.163:0/3761703749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a540b4e70 0x7f1a540ccff0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009e60 tx=0x7f1a44009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d57b80e5e8d0ca1 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:11.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.448+0000 7f1a51d9b700 1 -- 172.21.15.163:0/3761703749 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4400e070 con 0x7f1a540b4e70 2023-10-09T06:31:11.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.449+0000 7f1a51d9b700 1 -- 172.21.15.163:0/3761703749 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4400bb60 con 0x7f1a540b4e70 2023-10-09T06:31:11.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.449+0000 7f1a51d9b700 1 -- 172.21.15.163:0/3761703749 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a44012430 con 0x7f1a540b4e70 2023-10-09T06:31:11.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.449+0000 7f1a5359e700 1 -- 172.21.15.163:0/3761703749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a540b4e70 msgr2=0x7f1a540ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:11.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.449+0000 7f1a5359e700 1 --2- 172.21.15.163:0/3761703749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a540b4e70 0x7f1a540ccff0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009e60 tx=0x7f1a44009780 comp rx=0 tx=0).stop 2023-10-09T06:31:11.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.449+0000 7f1a5359e700 1 -- 172.21.15.163:0/3761703749 shutdown_connections 2023-10-09T06:31:11.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.449+0000 7f1a5359e700 1 --2- 172.21.15.163:0/3761703749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a540b4e70 0x7f1a540ccff0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:11.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.449+0000 7f1a5359e700 1 -- 172.21.15.163:0/3761703749 >> 172.21.15.163:0/3761703749 conn(0x7f1a540d10c0 msgr2=0x7f1a540affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:11.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.449+0000 7f1a5359e700 1 -- 172.21.15.163:0/3761703749 shutdown_connections 2023-10-09T06:31:11.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.449+0000 7f1a5359e700 1 -- 172.21.15.163:0/3761703749 wait complete. 2023-10-09T06:31:11.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.450+0000 7f1a5359e700 1 Processor -- start 2023-10-09T06:31:11.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.450+0000 7f1a5359e700 1 -- start start 2023-10-09T06:31:11.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.451+0000 7f1a5359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a540b16d0 0x7f1a541696c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:11.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.451+0000 7f1a5359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a540b1b40 0x7f1a5415b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:11.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.451+0000 7f1a5359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a540b4e70 0x7f1a540c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:11.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.451+0000 7f1a5359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5415cef0 con 0x7f1a540b1b40 2023-10-09T06:31:11.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.451+0000 7f1a5359e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5415c2e0 con 0x7f1a540b16d0 2023-10-09T06:31:11.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.451+0000 7f1a5359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5415c5c0 con 0x7f1a540b4e70 2023-10-09T06:31:11.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.451+0000 7f1a49d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a540b1b40 0x7f1a5415b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:11.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.451+0000 7f1a49d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a540b1b40 0x7f1a5415b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53208/0 (socket says 172.21.15.163:53208) 2023-10-09T06:31:11.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.451+0000 7f1a49d9b700 1 -- 172.21.15.163:0/1571480144 learned_addr learned my addr 172.21.15.163:0/1571480144 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:11.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.451+0000 7f1a5259c700 1 --2- 172.21.15.163:0/1571480144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a540b16d0 0x7f1a541696c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:11.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.451+0000 7f1a52d9d700 1 --2- 172.21.15.163:0/1571480144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a540b4e70 0x7f1a540c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:11.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.451+0000 7f1a49d9b700 1 -- 172.21.15.163:0/1571480144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a540b4e70 msgr2=0x7f1a540c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:11.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.451+0000 7f1a49d9b700 1 --2- 172.21.15.163:0/1571480144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a540b4e70 0x7f1a540c7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:11.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.451+0000 7f1a49d9b700 1 -- 172.21.15.163:0/1571480144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a540b16d0 msgr2=0x7f1a541696c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:11.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.451+0000 7f1a49d9b700 1 --2- 172.21.15.163:0/1571480144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a540b16d0 0x7f1a541696c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:11.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.451+0000 7f1a49d9b700 1 -- 172.21.15.163:0/1571480144 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a44009a20 con 0x7f1a540b1b40 2023-10-09T06:31:11.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.451+0000 7f1a52d9d700 1 --2- 172.21.15.163:0/1571480144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a540b4e70 0x7f1a540c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:11.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.451+0000 7f1a5259c700 1 --2- 172.21.15.163:0/1571480144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a540b16d0 0x7f1a541696c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:11.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.452+0000 7f1a49d9b700 1 --2- 172.21.15.163:0/1571480144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a540b1b40 0x7f1a5415b890 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00c590 tx=0x7f1a3c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:11.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.452+0000 7f1a4affd700 1 -- 172.21.15.163:0/1571480144 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c011590 con 0x7f1a540b1b40 2023-10-09T06:31:11.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.452+0000 7f1a4affd700 1 -- 172.21.15.163:0/1571480144 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a3c0116f0 con 0x7f1a540b1b40 2023-10-09T06:31:11.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.452+0000 7f1a5359e700 1 -- 172.21.15.163:0/1571480144 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a5415c8a0 con 0x7f1a540b1b40 2023-10-09T06:31:11.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.452+0000 7f1a4affd700 1 -- 172.21.15.163:0/1571480144 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c011ab0 con 0x7f1a540b1b40 2023-10-09T06:31:11.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.452+0000 7f1a5359e700 1 -- 172.21.15.163:0/1571480144 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a5416a220 con 0x7f1a540b1b40 2023-10-09T06:31:11.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.453+0000 7f1a5359e700 1 -- 172.21.15.163:0/1571480144 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a5415d0a0 con 0x7f1a540b1b40 2023-10-09T06:31:11.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.453+0000 7f1a4affd700 1 -- 172.21.15.163:0/1571480144 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1a3c01b3b0 con 0x7f1a540b1b40 2023-10-09T06:31:11.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.454+0000 7f1a4affd700 1 --2- 172.21.15.163:0/1571480144 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a34066f30 0x7f1a340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:11.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.454+0000 7f1a4affd700 1 -- 172.21.15.163:0/1571480144 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1a3c083b00 con 0x7f1a540b1b40 2023-10-09T06:31:11.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.454+0000 7f1a5259c700 1 --2- 172.21.15.163:0/1571480144 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a34066f30 0x7f1a340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:11.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.454+0000 7f1a5259c700 1 --2- 172.21.15.163:0/1571480144 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a34066f30 0x7f1a340693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009e60 tx=0x7f1a44019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:11.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.458+0000 7f1a4affd700 1 -- 172.21.15.163:0/1571480144 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a5415d0a0 con 0x7f1a540b1b40 2023-10-09T06:31:11.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.642+0000 7f1a5359e700 1 -- 172.21.15.163:0/1571480144 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1a5415d030 con 0x7f1a540b1b40 2023-10-09T06:31:11.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.642+0000 7f1a4affd700 1 -- 172.21.15.163:0/1571480144 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1a3c017020 con 0x7f1a540b1b40 2023-10-09T06:31:11.642 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 64 2023-10-09T06:31:11.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.644+0000 7f1a5359e700 1 -- 172.21.15.163:0/1571480144 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a34066f30 msgr2=0x7f1a340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:11.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.644+0000 7f1a5359e700 1 --2- 172.21.15.163:0/1571480144 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a34066f30 0x7f1a340693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009e60 tx=0x7f1a44019000 comp rx=0 tx=0).stop 2023-10-09T06:31:11.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.644+0000 7f1a5359e700 1 -- 172.21.15.163:0/1571480144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a540b1b40 msgr2=0x7f1a5415b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:11.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.644+0000 7f1a5359e700 1 --2- 172.21.15.163:0/1571480144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a540b1b40 0x7f1a5415b890 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00c590 tx=0x7f1a3c00c950 comp rx=0 tx=0).stop 2023-10-09T06:31:11.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.644+0000 7f1a5359e700 1 -- 172.21.15.163:0/1571480144 shutdown_connections 2023-10-09T06:31:11.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.644+0000 7f1a5359e700 1 --2- 172.21.15.163:0/1571480144 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a34066f30 0x7f1a340693e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:11.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.644+0000 7f1a5359e700 1 --2- 172.21.15.163:0/1571480144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a540b4e70 0x7f1a540c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:11.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.644+0000 7f1a5359e700 1 --2- 172.21.15.163:0/1571480144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a540b16d0 0x7f1a541696c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:11.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.644+0000 7f1a5359e700 1 --2- 172.21.15.163:0/1571480144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a540b1b40 0x7f1a5415b890 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:11.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.644+0000 7f1a5359e700 1 -- 172.21.15.163:0/1571480144 >> 172.21.15.163:0/1571480144 conn(0x7f1a540d10c0 msgr2=0x7f1a54159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:11.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.644+0000 7f1a5359e700 1 -- 172.21.15.163:0/1571480144 shutdown_connections 2023-10-09T06:31:11.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.644+0000 7f1a5359e700 1 -- 172.21.15.163:0/1571480144 wait complete. 2023-10-09T06:31:11.807 INFO:tasks.cephfs_test_runner:Starting test: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) That the volume can be created and then cleans up 2023-10-09T06:31:11.808 INFO:tasks.cephfs_test_runner:test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2023-10-09T06:31:11.808 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create' 2023-10-09T06:31:11.923 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.923+0000 7f87015dd700 1 Processor -- start 2023-10-09T06:31:11.923 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.923+0000 7f87015dd700 1 -- start start 2023-10-09T06:31:11.925 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.924+0000 7f87015dd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86fc159890 0x7f86fc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:11.925 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.924+0000 7f87015dd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f86fc159c60 0x7f86fc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:11.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.924+0000 7f87015dd700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86fc15a160 con 0x7f86fc15db10 2023-10-09T06:31:11.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.924+0000 7f87015dd700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86fc0d13a0 con 0x7f86fc159c60 2023-10-09T06:31:11.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.924+0000 7f87015dd700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86fc15e140 con 0x7f86fc159890 2023-10-09T06:31:11.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.924+0000 7f86fa7fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f86fc159c60 0x7f86fc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:11.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.924+0000 7f86fb7fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f86fc15db10 0x7f86fc069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53230/0 (socket says 172.21.15.163:53230) 2023-10-09T06:31:11.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.924+0000 7f86fa7fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f86fc159c60 0x7f86fc15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53224/0 (socket says 172.21.15.163:53224) 2023-10-09T06:31:11.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.924+0000 7f86fb7fe700 1 -- 172.21.15.163:0/1464301242 learned_addr learned my addr 172.21.15.163:0/1464301242 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:11.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.924+0000 7f86faffd700 1 --2- 172.21.15.163:0/1464301242 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86fc159890 0x7f86fc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:11.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.924+0000 7f86fa7fc700 1 -- 172.21.15.163:0/1464301242 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86fc159890 msgr2=0x7f86fc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:11.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.924+0000 7f86fa7fc700 1 --2- 172.21.15.163:0/1464301242 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86fc159890 0x7f86fc15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:11.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.924+0000 7f86fa7fc700 1 -- 172.21.15.163:0/1464301242 >> v1:172.21.15.163:6789/0 conn(0x7f86fc15db10 legacy=0x7f86fc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:11.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.924+0000 7f86fa7fc700 1 -- 172.21.15.163:0/1464301242 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86fc15a3d0 con 0x7f86fc159c60 2023-10-09T06:31:11.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.924+0000 7f86f9ffb700 1 -- 172.21.15.163:0/1464301242 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2992082675 0 0) 0x7f86fc15a160 con 0x7f86fc15db10 2023-10-09T06:31:11.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.925+0000 7f86fa7fc700 1 --2- 172.21.15.163:0/1464301242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86fc159c60 0x7f86fc15d5d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f86e40098e0 tx=0x7f86e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898450f8f5b12b94 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:11.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.925+0000 7f86f9ffb700 1 -- 172.21.15.163:0/1464301242 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4009e80 con 0x7f86fc159c60 2023-10-09T06:31:11.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.925+0000 7f86f9ffb700 1 -- 172.21.15.163:0/1464301242 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e400b860 con 0x7f86fc159c60 2023-10-09T06:31:11.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.925+0000 7f86f9ffb700 1 -- 172.21.15.163:0/1464301242 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e400bbe0 con 0x7f86fc159c60 2023-10-09T06:31:11.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.925+0000 7f87015dd700 1 -- 172.21.15.163:0/1464301242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86fc159c60 msgr2=0x7f86fc15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:11.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.925+0000 7f87015dd700 1 --2- 172.21.15.163:0/1464301242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86fc159c60 0x7f86fc15d5d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f86e40098e0 tx=0x7f86e40091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:11.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.925+0000 7f87015dd700 1 -- 172.21.15.163:0/1464301242 shutdown_connections 2023-10-09T06:31:11.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.925+0000 7f87015dd700 1 --2- 172.21.15.163:0/1464301242 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86fc159890 0x7f86fc15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:11.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.925+0000 7f87015dd700 1 --2- 172.21.15.163:0/1464301242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86fc159c60 0x7f86fc15d5d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:11.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.925+0000 7f87015dd700 1 -- 172.21.15.163:0/1464301242 >> 172.21.15.163:0/1464301242 conn(0x7f86fc068e20 msgr2=0x7f86fc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:11.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.925+0000 7f87015dd700 1 -- 172.21.15.163:0/1464301242 shutdown_connections 2023-10-09T06:31:11.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.925+0000 7f87015dd700 1 -- 172.21.15.163:0/1464301242 wait complete. 2023-10-09T06:31:11.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.926+0000 7f87015dd700 1 Processor -- start 2023-10-09T06:31:11.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.927+0000 7f87015dd700 1 -- start start 2023-10-09T06:31:11.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.927+0000 7f87015dd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86fc159890 0x7f86fc0c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:11.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.927+0000 7f87015dd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86fc159c60 0x7f86fc1471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:11.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.927+0000 7f87015dd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86fc15db10 0x7f86fc0c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:11.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.927+0000 7f87015dd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86fc0ce7a0 con 0x7f86fc159c60 2023-10-09T06:31:11.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.927+0000 7f87015dd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86fc0cea40 con 0x7f86fc15db10 2023-10-09T06:31:11.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.927+0000 7f87015dd700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86fc0ced20 con 0x7f86fc159890 2023-10-09T06:31:11.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.927+0000 7f86fa7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86fc159c60 0x7f86fc1471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:11.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.927+0000 7f86fa7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86fc159c60 0x7f86fc1471f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53228/0 (socket says 172.21.15.163:53228) 2023-10-09T06:31:11.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.927+0000 7f86fa7fc700 1 -- 172.21.15.163:0/2778115299 learned_addr learned my addr 172.21.15.163:0/2778115299 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:11.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.927+0000 7f86faffd700 1 --2- 172.21.15.163:0/2778115299 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86fc159890 0x7f86fc0c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:11.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.928+0000 7f86fb7fe700 1 --2- 172.21.15.163:0/2778115299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86fc15db10 0x7f86fc0c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:11.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.928+0000 7f86fa7fc700 1 -- 172.21.15.163:0/2778115299 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86fc159890 msgr2=0x7f86fc0c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:11.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.928+0000 7f86fa7fc700 1 --2- 172.21.15.163:0/2778115299 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86fc159890 0x7f86fc0c9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:11.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.928+0000 7f86fa7fc700 1 -- 172.21.15.163:0/2778115299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86fc15db10 msgr2=0x7f86fc0c7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:11.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.928+0000 7f86fa7fc700 1 --2- 172.21.15.163:0/2778115299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86fc15db10 0x7f86fc0c7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:11.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.928+0000 7f86fa7fc700 1 -- 172.21.15.163:0/2778115299 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e4009420 con 0x7f86fc159c60 2023-10-09T06:31:11.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.928+0000 7f86faffd700 1 --2- 172.21.15.163:0/2778115299 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86fc159890 0x7f86fc0c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:11.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.928+0000 7f86fb7fe700 1 --2- 172.21.15.163:0/2778115299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86fc15db10 0x7f86fc0c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:11.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.928+0000 7f86fa7fc700 1 --2- 172.21.15.163:0/2778115299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86fc159c60 0x7f86fc1471f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f86e4013000 tx=0x7f86e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:11.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.928+0000 7f86f2ffd700 1 -- 172.21.15.163:0/2778115299 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e400bbe0 con 0x7f86fc159c60 2023-10-09T06:31:11.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.928+0000 7f86f2ffd700 1 -- 172.21.15.163:0/2778115299 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e4011420 con 0x7f86fc159c60 2023-10-09T06:31:11.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.928+0000 7f87015dd700 1 -- 172.21.15.163:0/2778115299 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86fc0cf000 con 0x7f86fc159c60 2023-10-09T06:31:11.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.928+0000 7f86f2ffd700 1 -- 172.21.15.163:0/2778115299 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e40117e0 con 0x7f86fc159c60 2023-10-09T06:31:11.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.928+0000 7f87015dd700 1 -- 172.21.15.163:0/2778115299 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86fc147730 con 0x7f86fc159c60 2023-10-09T06:31:11.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.929+0000 7f87015dd700 1 -- 172.21.15.163:0/2778115299 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86fc0cf240 con 0x7f86fc159c60 2023-10-09T06:31:11.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.929+0000 7f86f2ffd700 1 -- 172.21.15.163:0/2778115299 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f86e4011bd0 con 0x7f86fc159c60 2023-10-09T06:31:11.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.930+0000 7f86f2ffd700 1 --2- 172.21.15.163:0/2778115299 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f86dc066f30 0x7f86dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:11.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.930+0000 7f86f2ffd700 1 -- 172.21.15.163:0/2778115299 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f86e4086120 con 0x7f86fc159c60 2023-10-09T06:31:11.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.930+0000 7f86faffd700 1 --2- 172.21.15.163:0/2778115299 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f86dc066f30 0x7f86dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:11.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.930+0000 7f86faffd700 1 --2- 172.21.15.163:0/2778115299 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f86dc066f30 0x7f86dc0693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00d3e0 tx=0x7f86ec007550 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:11.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:11.934+0000 7f86f2ffd700 1 -- 172.21.15.163:0/2778115299 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f86fc0cf240 con 0x7f86fc159c60 2023-10-09T06:31:12.194 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.194+0000 7f87015dd700 1 -- 172.21.15.163:0/2778115299 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]} v 0) v1 -- 0x7f86fc0cf240 con 0x7f86fc159c60 2023-10-09T06:31:12.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.558+0000 7f86f2ffd700 1 -- 172.21.15.163:0/2778115299 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]}]=0 v49) v1 ==== 138+0+0 (secure 0 0 0) 0x7f86fc0cf240 con 0x7f86fc159c60 2023-10-09T06:31:12.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.560+0000 7f87015dd700 1 -- 172.21.15.163:0/2778115299 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f86dc066f30 msgr2=0x7f86dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:12.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.560+0000 7f87015dd700 1 --2- 172.21.15.163:0/2778115299 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f86dc066f30 0x7f86dc0693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00d3e0 tx=0x7f86ec007550 comp rx=0 tx=0).stop 2023-10-09T06:31:12.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.561+0000 7f87015dd700 1 -- 172.21.15.163:0/2778115299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86fc159c60 msgr2=0x7f86fc1471f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:12.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.561+0000 7f87015dd700 1 --2- 172.21.15.163:0/2778115299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86fc159c60 0x7f86fc1471f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f86e4013000 tx=0x7f86e4017000 comp rx=0 tx=0).stop 2023-10-09T06:31:12.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.561+0000 7f87015dd700 1 -- 172.21.15.163:0/2778115299 shutdown_connections 2023-10-09T06:31:12.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.561+0000 7f87015dd700 1 --2- 172.21.15.163:0/2778115299 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f86dc066f30 0x7f86dc0693e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:12.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.561+0000 7f87015dd700 1 --2- 172.21.15.163:0/2778115299 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86fc15db10 0x7f86fc0c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:12.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.561+0000 7f87015dd700 1 --2- 172.21.15.163:0/2778115299 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86fc159890 0x7f86fc0c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:12.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.561+0000 7f87015dd700 1 --2- 172.21.15.163:0/2778115299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86fc159c60 0x7f86fc1471f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:12.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.561+0000 7f87015dd700 1 -- 172.21.15.163:0/2778115299 >> 172.21.15.163:0/2778115299 conn(0x7f86fc068e20 msgr2=0x7f86fc152fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:12.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.561+0000 7f87015dd700 1 -- 172.21.15.163:0/2778115299 shutdown_connections 2023-10-09T06:31:12.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.561+0000 7f87015dd700 1 -- 172.21.15.163:0/2778115299 wait complete. 2023-10-09T06:31:12.573 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-09T06:31:12.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.690+0000 7f6049091700 1 Processor -- start 2023-10-09T06:31:12.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.690+0000 7f6049091700 1 -- start start 2023-10-09T06:31:12.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.691+0000 7f6049091700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6044159890 0x7f604415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:12.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.691+0000 7f6049091700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6044159c60 0x7f604415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:12.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.691+0000 7f6049091700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f604415db10 0x7f6044069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:12.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.691+0000 7f6049091700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f604415a160 con 0x7f6044159890 2023-10-09T06:31:12.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.691+0000 7f6049091700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60440d13a0 con 0x7f604415db10 2023-10-09T06:31:12.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.691+0000 7f6049091700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f604415e140 con 0x7f6044159c60 2023-10-09T06:31:12.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.691+0000 7f6042d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6044159890 0x7f604415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:12.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.691+0000 7f6042d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6044159890 0x7f604415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53244/0 (socket says 172.21.15.163:53244) 2023-10-09T06:31:12.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.691+0000 7f6042d9d700 1 -- 172.21.15.163:0/3864452816 learned_addr learned my addr 172.21.15.163:0/3864452816 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:12.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.691+0000 7f604259c700 1 --2- 172.21.15.163:0/3864452816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6044159c60 0x7f604415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:12.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.691+0000 7f604359e700 1 --2- 172.21.15.163:0/3864452816 >> v2:172.21.15.181:3300/0 conn(0x7f604415db10 0x7f6044069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:12.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.691+0000 7f6042d9d700 1 -- 172.21.15.163:0/3864452816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6044159c60 msgr2=0x7f604415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:12.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.691+0000 7f6042d9d700 1 --2- 172.21.15.163:0/3864452816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6044159c60 0x7f604415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:12.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.691+0000 7f6042d9d700 1 -- 172.21.15.163:0/3864452816 >> v2:172.21.15.181:3300/0 conn(0x7f604415db10 msgr2=0x7f6044069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:12.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.691+0000 7f6042d9d700 1 --2- 172.21.15.163:0/3864452816 >> v2:172.21.15.181:3300/0 conn(0x7f604415db10 0x7f6044069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:12.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.691+0000 7f6042d9d700 1 -- 172.21.15.163:0/3864452816 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f604415a3d0 con 0x7f6044159890 2023-10-09T06:31:12.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.691+0000 7f604359e700 1 --2- 172.21.15.163:0/3864452816 >> v2:172.21.15.181:3300/0 conn(0x7f604415db10 0x7f6044069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:12.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.691+0000 7f604259c700 1 --2- 172.21.15.163:0/3864452816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6044159c60 0x7f604415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:12.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.691+0000 7f6042d9d700 1 --2- 172.21.15.163:0/3864452816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6044159890 0x7f604415abd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6044089b50 tx=0x7f6034009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b397b5548d1d93 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:12.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.692+0000 7f6041d9b700 1 -- 172.21.15.163:0/3864452816 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603400e070 con 0x7f6044159890 2023-10-09T06:31:12.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.692+0000 7f6041d9b700 1 -- 172.21.15.163:0/3864452816 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603400ba50 con 0x7f6044159890 2023-10-09T06:31:12.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.692+0000 7f6041d9b700 1 -- 172.21.15.163:0/3864452816 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60340092e0 con 0x7f6044159890 2023-10-09T06:31:12.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.692+0000 7f6049091700 1 -- 172.21.15.163:0/3864452816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6044159890 msgr2=0x7f604415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:12.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.692+0000 7f6049091700 1 --2- 172.21.15.163:0/3864452816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6044159890 0x7f604415abd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6044089b50 tx=0x7f6034009840 comp rx=0 tx=0).stop 2023-10-09T06:31:12.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.692+0000 7f6049091700 1 -- 172.21.15.163:0/3864452816 shutdown_connections 2023-10-09T06:31:12.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.692+0000 7f6049091700 1 --2- 172.21.15.163:0/3864452816 >> v2:172.21.15.181:3300/0 conn(0x7f604415db10 0x7f6044069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:12.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.692+0000 7f6049091700 1 --2- 172.21.15.163:0/3864452816 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6044159890 0x7f604415abd0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:12.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.692+0000 7f6049091700 1 --2- 172.21.15.163:0/3864452816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6044159c60 0x7f604415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:12.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.692+0000 7f6049091700 1 -- 172.21.15.163:0/3864452816 >> 172.21.15.163:0/3864452816 conn(0x7f6044068e20 msgr2=0x7f60440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:12.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.692+0000 7f6049091700 1 -- 172.21.15.163:0/3864452816 shutdown_connections 2023-10-09T06:31:12.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.692+0000 7f6049091700 1 -- 172.21.15.163:0/3864452816 wait complete. 2023-10-09T06:31:12.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.693+0000 7f6049091700 1 Processor -- start 2023-10-09T06:31:12.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.693+0000 7f6049091700 1 -- start start 2023-10-09T06:31:12.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.694+0000 7f6049091700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6044159890 0x7f604414c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:12.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.694+0000 7f6049091700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6044159c60 0x7f60440cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:12.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.694+0000 7f6049091700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f604415db10 0x7f6044151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:12.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.694+0000 7f6049091700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604414caf0 con 0x7f604415db10 2023-10-09T06:31:12.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.694+0000 7f6049091700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604414b2c0 con 0x7f6044159c60 2023-10-09T06:31:12.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.694+0000 7f6049091700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f604414c120 con 0x7f6044159890 2023-10-09T06:31:12.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.694+0000 7f604359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f604415db10 0x7f6044151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:12.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.694+0000 7f604359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f604415db10 0x7f6044151370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53246/0 (socket says 172.21.15.163:53246) 2023-10-09T06:31:12.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.694+0000 7f604359e700 1 -- 172.21.15.163:0/255039826 learned_addr learned my addr 172.21.15.163:0/255039826 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:12.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.694+0000 7f6042d9d700 1 --2- 172.21.15.163:0/255039826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6044159890 0x7f604414c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:12.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.694+0000 7f604259c700 1 --2- 172.21.15.163:0/255039826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6044159c60 0x7f60440cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:12.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.695+0000 7f604359e700 1 -- 172.21.15.163:0/255039826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6044159890 msgr2=0x7f604414c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:12.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.695+0000 7f604359e700 1 --2- 172.21.15.163:0/255039826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6044159890 0x7f604414c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:12.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.695+0000 7f604359e700 1 -- 172.21.15.163:0/255039826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6044159c60 msgr2=0x7f60440cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:12.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.695+0000 7f604359e700 1 --2- 172.21.15.163:0/255039826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6044159c60 0x7f60440cdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:12.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.695+0000 7f604359e700 1 -- 172.21.15.163:0/255039826 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6034009ae0 con 0x7f604415db10 2023-10-09T06:31:12.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.695+0000 7f604359e700 1 --2- 172.21.15.163:0/255039826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f604415db10 0x7f6044151370 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f603c00c880 tx=0x7f603c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:12.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.695+0000 7f603affd700 1 -- 172.21.15.163:0/255039826 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c011600 con 0x7f604415db10 2023-10-09T06:31:12.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.695+0000 7f603affd700 1 -- 172.21.15.163:0/255039826 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603c011760 con 0x7f604415db10 2023-10-09T06:31:12.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.695+0000 7f6049091700 1 -- 172.21.15.163:0/255039826 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f604414b510 con 0x7f604415db10 2023-10-09T06:31:12.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.695+0000 7f603affd700 1 -- 172.21.15.163:0/255039826 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c011b20 con 0x7f604415db10 2023-10-09T06:31:12.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.695+0000 7f6049091700 1 -- 172.21.15.163:0/255039826 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f604414bad0 con 0x7f604415db10 2023-10-09T06:31:12.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.696+0000 7f6049091700 1 -- 172.21.15.163:0/255039826 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f604404c3c0 con 0x7f604415db10 2023-10-09T06:31:12.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.696+0000 7f603affd700 1 -- 172.21.15.163:0/255039826 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f603c018350 con 0x7f604415db10 2023-10-09T06:31:12.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.696+0000 7f603affd700 1 --2- 172.21.15.163:0/255039826 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6024066fb0 0x7f6024069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:12.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.697+0000 7f603affd700 1 -- 172.21.15.163:0/255039826 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f603c084470 con 0x7f604415db10 2023-10-09T06:31:12.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.697+0000 7f6042d9d700 1 --2- 172.21.15.163:0/255039826 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6024066fb0 0x7f6024069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:12.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.697+0000 7f6042d9d700 1 --2- 172.21.15.163:0/255039826 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6024066fb0 0x7f6024069460 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6034009880 tx=0x7f6034018000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:12.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.701+0000 7f603affd700 1 -- 172.21.15.163:0/255039826 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f603c0550d0 con 0x7f604415db10 2023-10-09T06:31:12.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.883+0000 7f6049091700 1 -- 172.21.15.163:0/255039826 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f604414c260 con 0x7f604415db10 2023-10-09T06:31:12.884 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:31:12.887 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:31:12.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.888+0000 7f603affd700 1 -- 172.21.15.163:0/255039826 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f603c054c60 con 0x7f604415db10 2023-10-09T06:31:12.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.890+0000 7f6049091700 1 -- 172.21.15.163:0/255039826 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6024066fb0 msgr2=0x7f6024069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:12.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.890+0000 7f6049091700 1 --2- 172.21.15.163:0/255039826 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6024066fb0 0x7f6024069460 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6034009880 tx=0x7f6034018000 comp rx=0 tx=0).stop 2023-10-09T06:31:12.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.890+0000 7f6049091700 1 -- 172.21.15.163:0/255039826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f604415db10 msgr2=0x7f6044151370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:12.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.890+0000 7f6049091700 1 --2- 172.21.15.163:0/255039826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f604415db10 0x7f6044151370 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f603c00c880 tx=0x7f603c008040 comp rx=0 tx=0).stop 2023-10-09T06:31:12.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.890+0000 7f6049091700 1 -- 172.21.15.163:0/255039826 shutdown_connections 2023-10-09T06:31:12.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.890+0000 7f6049091700 1 --2- 172.21.15.163:0/255039826 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6024066fb0 0x7f6024069460 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:12.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.890+0000 7f6049091700 1 --2- 172.21.15.163:0/255039826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f604415db10 0x7f6044151370 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:12.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.890+0000 7f6049091700 1 --2- 172.21.15.163:0/255039826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6044159890 0x7f604414c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:12.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.890+0000 7f6049091700 1 --2- 172.21.15.163:0/255039826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6044159c60 0x7f60440cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:12.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.890+0000 7f6049091700 1 -- 172.21.15.163:0/255039826 >> 172.21.15.163:0/255039826 conn(0x7f6044068e20 msgr2=0x7f604414d660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:12.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.890+0000 7f6049091700 1 -- 172.21.15.163:0/255039826 shutdown_connections 2023-10-09T06:31:12.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:12.890+0000 7f6049091700 1 -- 172.21.15.163:0/255039826 wait complete. 2023-10-09T06:31:12.903 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-09T06:31:12.904 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:31:13.005 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:31:13.045 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-09T06:31:13.045 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-09T06:31:13.045 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:31:13.046 DEBUG:teuthology.orchestra.run.smithi163:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-10-09T06:31:13.092 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi163.front.sepia.ceph.com 2023-10-09T06:31:13.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:13.093 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-09T06:31:13.108 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-09T06:31:13.108 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:31:13.108 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-10-09T06:31:13.145 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-09T06:31:13.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:13.146 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-09T06:31:13.201 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-10-09T06:31:13.201 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:31:13.202 DEBUG:teuthology.orchestra.run.smithi163:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-10-09T06:31:13.237 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi163.front.sepia.ceph.com 2023-10-09T06:31:13.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:13.238 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-09T06:31:13.293 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-10-09T06:31:13.293 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:31:13.293 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-10-09T06:31:13.332 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-09T06:31:13.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:13.332 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-09T06:31:13.387 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-09T06:31:13.388 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:31:13.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.504+0000 7f90aa654700 1 Processor -- start 2023-10-09T06:31:13.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.504+0000 7f90aa654700 1 -- start start 2023-10-09T06:31:13.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.504+0000 7f90aa654700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f90a40af480 0x7f90a40ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:13.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.504+0000 7f90aa654700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90a4157e30 con 0x7f90a4156a60 2023-10-09T06:31:13.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.504+0000 7f90aa654700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90a41578b0 con 0x7f90a4156690 2023-10-09T06:31:13.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.504+0000 7f90aa654700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90a406aa00 con 0x7f90a40af480 2023-10-09T06:31:13.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.505+0000 7f90a37fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f90a40af480 0x7f90a40ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:13.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.505+0000 7f90a8bf1700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f90a4156a60 0x7f90a406ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53244/0 (socket says 172.21.15.163:53244) 2023-10-09T06:31:13.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.505+0000 7f90a8bf1700 1 -- 172.21.15.163:0/1044724252 learned_addr learned my addr 172.21.15.163:0/1044724252 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:13.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.505+0000 7f90a37fe700 1 -- 172.21.15.163:0/1044724252 >> v1:172.21.15.181:6789/0 conn(0x7f90a4156690 legacy=0x7f90a40c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:13.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.505+0000 7f90a37fe700 1 -- 172.21.15.163:0/1044724252 >> v1:172.21.15.163:6789/0 conn(0x7f90a4156a60 legacy=0x7f90a406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:13.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.505+0000 7f90a2ffd700 1 -- 172.21.15.163:0/1044724252 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 623080125 0 0) 0x7f90a41578b0 con 0x7f90a4156690 2023-10-09T06:31:13.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.505+0000 7f90a37fe700 1 -- 172.21.15.163:0/1044724252 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a4157b50 con 0x7f90a40af480 2023-10-09T06:31:13.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.505+0000 7f90a37fe700 1 --2- 172.21.15.163:0/1044724252 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90a40af480 0x7f90a40ca9d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f90980098e0 tx=0x7f90980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d13e25ab4cd5d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:13.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.505+0000 7f90a2ffd700 1 -- 172.21.15.163:0/1044724252 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9098009e80 con 0x7f90a40af480 2023-10-09T06:31:13.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.505+0000 7f90a2ffd700 1 -- 172.21.15.163:0/1044724252 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909800b760 con 0x7f90a40af480 2023-10-09T06:31:13.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.505+0000 7f90a2ffd700 1 -- 172.21.15.163:0/1044724252 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909800bae0 con 0x7f90a40af480 2023-10-09T06:31:13.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.506+0000 7f90aa654700 1 -- 172.21.15.163:0/1044724252 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90a40af480 msgr2=0x7f90a40ca9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:13.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.506+0000 7f90aa654700 1 --2- 172.21.15.163:0/1044724252 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90a40af480 0x7f90a40ca9d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f90980098e0 tx=0x7f90980091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:13.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.506+0000 7f90aa654700 1 -- 172.21.15.163:0/1044724252 shutdown_connections 2023-10-09T06:31:13.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.506+0000 7f90aa654700 1 --2- 172.21.15.163:0/1044724252 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90a40af480 0x7f90a40ca9d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:13.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.506+0000 7f90aa654700 1 -- 172.21.15.163:0/1044724252 >> 172.21.15.163:0/1044724252 conn(0x7f90a40c9870 msgr2=0x7f90a40c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:13.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.506+0000 7f90aa654700 1 -- 172.21.15.163:0/1044724252 shutdown_connections 2023-10-09T06:31:13.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.506+0000 7f90aa654700 1 -- 172.21.15.163:0/1044724252 wait complete. 2023-10-09T06:31:13.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.507+0000 7f90aa654700 1 Processor -- start 2023-10-09T06:31:13.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.507+0000 7f90aa654700 1 -- start start 2023-10-09T06:31:13.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.508+0000 7f90aa654700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90a40af480 0x7f90a41477a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:13.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.508+0000 7f90aa654700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90a4156690 0x7f90a4155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:13.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.508+0000 7f90aa654700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90a4156a60 0x7f90a4155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:13.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.508+0000 7f90aa654700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a40c3fe0 con 0x7f90a40af480 2023-10-09T06:31:13.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.508+0000 7f90aa654700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a40c42c0 con 0x7f90a4156690 2023-10-09T06:31:13.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.508+0000 7f90aa654700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a40c45a0 con 0x7f90a4156a60 2023-10-09T06:31:13.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.508+0000 7f90a3fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90a40af480 0x7f90a41477a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:13.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.508+0000 7f90a3fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90a40af480 0x7f90a41477a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53260/0 (socket says 172.21.15.163:53260) 2023-10-09T06:31:13.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.508+0000 7f90a3fff700 1 -- 172.21.15.163:0/557053990 learned_addr learned my addr 172.21.15.163:0/557053990 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:13.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.508+0000 7f90a8bf1700 1 --2- 172.21.15.163:0/557053990 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90a4156a60 0x7f90a4155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:13.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.508+0000 7f90a37fe700 1 --2- 172.21.15.163:0/557053990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90a4156690 0x7f90a4155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:13.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.508+0000 7f90a3fff700 1 -- 172.21.15.163:0/557053990 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90a4156a60 msgr2=0x7f90a4155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:13.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.508+0000 7f90a3fff700 1 --2- 172.21.15.163:0/557053990 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90a4156a60 0x7f90a4155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:13.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.508+0000 7f90a3fff700 1 -- 172.21.15.163:0/557053990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90a4156690 msgr2=0x7f90a4155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:13.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.508+0000 7f90a3fff700 1 --2- 172.21.15.163:0/557053990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90a4156690 0x7f90a4155650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:13.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.508+0000 7f90a3fff700 1 -- 172.21.15.163:0/557053990 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9098009420 con 0x7f90a40af480 2023-10-09T06:31:13.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.508+0000 7f90a8bf1700 1 --2- 172.21.15.163:0/557053990 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90a4156a60 0x7f90a4155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:13.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.508+0000 7f90a3fff700 1 --2- 172.21.15.163:0/557053990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90a40af480 0x7f90a41477a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f909400e9f0 tx=0x7f9094007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:13.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.509+0000 7f908bfff700 1 -- 172.21.15.163:0/557053990 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909400c9f0 con 0x7f90a40af480 2023-10-09T06:31:13.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.509+0000 7f908bfff700 1 -- 172.21.15.163:0/557053990 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909400cb70 con 0x7f90a40af480 2023-10-09T06:31:13.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.509+0000 7f90aa654700 1 -- 172.21.15.163:0/557053990 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a40c4880 con 0x7f90a40af480 2023-10-09T06:31:13.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.509+0000 7f908bfff700 1 -- 172.21.15.163:0/557053990 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909401a480 con 0x7f90a40af480 2023-10-09T06:31:13.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.509+0000 7f90aa654700 1 -- 172.21.15.163:0/557053990 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a40ca510 con 0x7f90a40af480 2023-10-09T06:31:13.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.509+0000 7f90aa654700 1 -- 172.21.15.163:0/557053990 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a404c3c0 con 0x7f90a40af480 2023-10-09T06:31:13.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.510+0000 7f908bfff700 1 -- 172.21.15.163:0/557053990 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9094011470 con 0x7f90a40af480 2023-10-09T06:31:13.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.510+0000 7f908bfff700 1 --2- 172.21.15.163:0/557053990 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9090066f30 0x7f90900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:13.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.510+0000 7f908bfff700 1 -- 172.21.15.163:0/557053990 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9094015070 con 0x7f90a40af480 2023-10-09T06:31:13.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.510+0000 7f90a37fe700 1 --2- 172.21.15.163:0/557053990 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9090066f30 0x7f90900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:13.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.511+0000 7f90a37fe700 1 --2- 172.21.15.163:0/557053990 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9090066f30 0x7f90900693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f90980030c0 tx=0x7f9098016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:13.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.515+0000 7f908bfff700 1 -- 172.21.15.163:0/557053990 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9094088e60 con 0x7f90a40af480 2023-10-09T06:31:13.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.741+0000 7f90aa654700 1 -- 172.21.15.163:0/557053990 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90a415a010 con 0x7f90a40af480 2023-10-09T06:31:13.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.741+0000 7f908bfff700 1 -- 172.21.15.163:0/557053990 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4603 (secure 0 0 0) 0x7f9094053ef0 con 0x7f90a40af480 2023-10-09T06:31:13.741 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:13.741 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3440080308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3440080308},{"type":"v1","addr":"172.21.15.181:6837","nonce":3440080308}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1990962358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1990962358},{"type":"v1","addr":"172.21.15.163:6835","nonce":1990962358}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2270076605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2270076605},{"type":"v1","addr":"172.21.15.163:6834","nonce":2270076605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:30:43.133124+0000","modified":"2023-10-09T06:30:46.760523+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/1149296546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1149296546},{"type":"v1","addr":"172.21.15.181:6836","nonce":1149296546}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-09T06:31:13.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.743+0000 7f90aa654700 1 -- 172.21.15.163:0/557053990 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9090066f30 msgr2=0x7f90900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:13.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.743+0000 7f90aa654700 1 --2- 172.21.15.163:0/557053990 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9090066f30 0x7f90900693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f90980030c0 tx=0x7f9098016000 comp rx=0 tx=0).stop 2023-10-09T06:31:13.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.743+0000 7f90aa654700 1 -- 172.21.15.163:0/557053990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90a40af480 msgr2=0x7f90a41477a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:13.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.743+0000 7f90aa654700 1 --2- 172.21.15.163:0/557053990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90a40af480 0x7f90a41477a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f909400e9f0 tx=0x7f9094007e80 comp rx=0 tx=0).stop 2023-10-09T06:31:13.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.743+0000 7f90aa654700 1 -- 172.21.15.163:0/557053990 shutdown_connections 2023-10-09T06:31:13.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.743+0000 7f90aa654700 1 --2- 172.21.15.163:0/557053990 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9090066f30 0x7f90900693e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:13.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.743+0000 7f90aa654700 1 --2- 172.21.15.163:0/557053990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90a40af480 0x7f90a41477a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:13.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.743+0000 7f90aa654700 1 --2- 172.21.15.163:0/557053990 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90a4156a60 0x7f90a4155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:13.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.743+0000 7f90aa654700 1 --2- 172.21.15.163:0/557053990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90a4156690 0x7f90a4155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:13.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.743+0000 7f90aa654700 1 -- 172.21.15.163:0/557053990 >> 172.21.15.163:0/557053990 conn(0x7f90a40c9870 msgr2=0x7f90a40aec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:13.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.743+0000 7f90aa654700 1 -- 172.21.15.163:0/557053990 shutdown_connections 2023-10-09T06:31:13.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.744+0000 7f90aa654700 1 -- 172.21.15.163:0/557053990 wait complete. 2023-10-09T06:31:13.744 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 8 2023-10-09T06:31:13.755 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:31:13.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.870+0000 7fc1cb2cb700 1 Processor -- start 2023-10-09T06:31:13.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.870+0000 7fc1cb2cb700 1 -- start start 2023-10-09T06:31:13.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.871+0000 7fc1cb2cb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc1c41597f0 0x7fc1c40b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:13.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.871+0000 7fc1cb2cb700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc1c415d230 0x7fc1c415d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:13.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.871+0000 7fc1cb2cb700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c415db70 con 0x7fc1c4159420 2023-10-09T06:31:13.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.871+0000 7fc1cb2cb700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c40b11d0 con 0x7fc1c415d230 2023-10-09T06:31:13.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.871+0000 7fc1cb2cb700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c40b14b0 con 0x7fc1c41597f0 2023-10-09T06:31:13.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.871+0000 7fc1caaca700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc1c415d230 0x7fc1c415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:13.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.871+0000 7fc1caaca700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc1c415d230 0x7fc1c415d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53270/0 (socket says 172.21.15.163:53270) 2023-10-09T06:31:13.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.871+0000 7fc1caaca700 1 -- 172.21.15.163:0/861155958 learned_addr learned my addr 172.21.15.163:0/861155958 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:13.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.871+0000 7fc1c9ac8700 1 --2- 172.21.15.163:0/861155958 >> v2:172.21.15.181:3300/0 conn(0x7fc1c41597f0 0x7fc1c40b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:13.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.871+0000 7fc1caaca700 1 -- 172.21.15.163:0/861155958 >> v2:172.21.15.181:3300/0 conn(0x7fc1c41597f0 msgr2=0x7fc1c40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:13.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.871+0000 7fc1caaca700 1 --2- 172.21.15.163:0/861155958 >> v2:172.21.15.181:3300/0 conn(0x7fc1c41597f0 0x7fc1c40b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:13.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.871+0000 7fc1caaca700 1 -- 172.21.15.163:0/861155958 >> v1:172.21.15.181:6789/0 conn(0x7fc1c4159420 legacy=0x7fc1c40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:13.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.871+0000 7fc1caaca700 1 -- 172.21.15.163:0/861155958 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c40b1790 con 0x7fc1c415d230 2023-10-09T06:31:13.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.871+0000 7fc1caaca700 1 --2- 172.21.15.163:0/861155958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1c415d230 0x7fc1c415d630 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4089750 tx=0x7fc1b4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f09da2043b32c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:13.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.871+0000 7fc1c92c7700 1 -- 172.21.15.163:0/861155958 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b400e070 con 0x7fc1c415d230 2023-10-09T06:31:13.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.872+0000 7fc1c92c7700 1 -- 172.21.15.163:0/861155958 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b400ba20 con 0x7fc1c415d230 2023-10-09T06:31:13.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.872+0000 7fc1c92c7700 1 -- 172.21.15.163:0/861155958 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b40092e0 con 0x7fc1c415d230 2023-10-09T06:31:13.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.872+0000 7fc1cb2cb700 1 -- 172.21.15.163:0/861155958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1c415d230 msgr2=0x7fc1c415d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:13.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.872+0000 7fc1cb2cb700 1 --2- 172.21.15.163:0/861155958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1c415d230 0x7fc1c415d630 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4089750 tx=0x7fc1b4009840 comp rx=0 tx=0).stop 2023-10-09T06:31:13.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.872+0000 7fc1cb2cb700 1 -- 172.21.15.163:0/861155958 shutdown_connections 2023-10-09T06:31:13.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.872+0000 7fc1cb2cb700 1 --2- 172.21.15.163:0/861155958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1c415d230 0x7fc1c415d630 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:13.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.872+0000 7fc1cb2cb700 1 --2- 172.21.15.163:0/861155958 >> v2:172.21.15.181:3300/0 conn(0x7fc1c41597f0 0x7fc1c40b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:13.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.872+0000 7fc1cb2cb700 1 -- 172.21.15.163:0/861155958 >> 172.21.15.163:0/861155958 conn(0x7fc1c40d0660 msgr2=0x7fc1c40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:13.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.872+0000 7fc1cb2cb700 1 -- 172.21.15.163:0/861155958 shutdown_connections 2023-10-09T06:31:13.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.872+0000 7fc1cb2cb700 1 -- 172.21.15.163:0/861155958 wait complete. 2023-10-09T06:31:13.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.873+0000 7fc1cb2cb700 1 Processor -- start 2023-10-09T06:31:13.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.873+0000 7fc1cb2cb700 1 -- start start 2023-10-09T06:31:13.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.874+0000 7fc1cb2cb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1c4159420 0x7fc1c415c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:13.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.874+0000 7fc1cb2cb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1c41597f0 0x7fc1c415a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:13.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.874+0000 7fc1cb2cb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c415d230 0x7fc1c4153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:13.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.874+0000 7fc1cb2cb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c415b650 con 0x7fc1c4159420 2023-10-09T06:31:13.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.874+0000 7fc1cb2cb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c415b090 con 0x7fc1c415d230 2023-10-09T06:31:13.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.874+0000 7fc1cb2cb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c415b370 con 0x7fc1c41597f0 2023-10-09T06:31:13.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.874+0000 7fc1ca2c9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1c4159420 0x7fc1c415c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:13.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.874+0000 7fc1ca2c9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1c4159420 0x7fc1c415c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53272/0 (socket says 172.21.15.163:53272) 2023-10-09T06:31:13.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.874+0000 7fc1ca2c9700 1 -- 172.21.15.163:0/3422126560 learned_addr learned my addr 172.21.15.163:0/3422126560 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:13.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.874+0000 7fc1caaca700 1 --2- 172.21.15.163:0/3422126560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c415d230 0x7fc1c4153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:13.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.874+0000 7fc1c9ac8700 1 --2- 172.21.15.163:0/3422126560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1c41597f0 0x7fc1c415a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:13.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.874+0000 7fc1ca2c9700 1 -- 172.21.15.163:0/3422126560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1c41597f0 msgr2=0x7fc1c415a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:13.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.874+0000 7fc1ca2c9700 1 --2- 172.21.15.163:0/3422126560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1c41597f0 0x7fc1c415a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:13.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.874+0000 7fc1ca2c9700 1 -- 172.21.15.163:0/3422126560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c415d230 msgr2=0x7fc1c4153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:13.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.874+0000 7fc1ca2c9700 1 --2- 172.21.15.163:0/3422126560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c415d230 0x7fc1c4153e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:13.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.874+0000 7fc1ca2c9700 1 -- 172.21.15.163:0/3422126560 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b4009ae0 con 0x7fc1c4159420 2023-10-09T06:31:13.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.875+0000 7fc1ca2c9700 1 --2- 172.21.15.163:0/3422126560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1c4159420 0x7fc1c415c940 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00c590 tx=0x7fc1bc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:13.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.875+0000 7fc1c27fc700 1 -- 172.21.15.163:0/3422126560 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc011590 con 0x7fc1c4159420 2023-10-09T06:31:13.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.875+0000 7fc1c27fc700 1 -- 172.21.15.163:0/3422126560 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1bc011710 con 0x7fc1c4159420 2023-10-09T06:31:13.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.875+0000 7fc1cb2cb700 1 -- 172.21.15.163:0/3422126560 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c40c55a0 con 0x7fc1c4159420 2023-10-09T06:31:13.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.875+0000 7fc1c27fc700 1 -- 172.21.15.163:0/3422126560 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc011ad0 con 0x7fc1c4159420 2023-10-09T06:31:13.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.875+0000 7fc1cb2cb700 1 -- 172.21.15.163:0/3422126560 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c40c7430 con 0x7fc1c4159420 2023-10-09T06:31:13.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.876+0000 7fc1cb2cb700 1 -- 172.21.15.163:0/3422126560 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c40c5400 con 0x7fc1c4159420 2023-10-09T06:31:13.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.876+0000 7fc1c27fc700 1 -- 172.21.15.163:0/3422126560 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc1bc01b3b0 con 0x7fc1c4159420 2023-10-09T06:31:13.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.877+0000 7fc1c27fc700 1 --2- 172.21.15.163:0/3422126560 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc1b0066f30 0x7fc1b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:13.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.877+0000 7fc1c27fc700 1 -- 172.21.15.163:0/3422126560 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc1bc083900 con 0x7fc1c4159420 2023-10-09T06:31:13.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.877+0000 7fc1c9ac8700 1 --2- 172.21.15.163:0/3422126560 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc1b0066f30 0x7fc1b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:13.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.877+0000 7fc1c9ac8700 1 --2- 172.21.15.163:0/3422126560 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc1b0066f30 0x7fc1b00693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40af880 tx=0x7fc1b80091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:13.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:13.881+0000 7fc1c27fc700 1 -- 172.21.15.163:0/3422126560 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc1c40c5400 con 0x7fc1c4159420 2023-10-09T06:31:14.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.062+0000 7fc1cb2cb700 1 -- 172.21.15.163:0/3422126560 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1c40c5400 con 0x7fc1c4159420 2023-10-09T06:31:14.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.063+0000 7fc1c27fc700 1 -- 172.21.15.163:0/3422126560 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fc1c40c5400 con 0x7fc1c4159420 2023-10-09T06:31:14.063 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:14.063 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":20,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:30:44.466923+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:30:41.369014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-09T06:30:42.103001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:31:14.066 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.065+0000 7fc1cb2cb700 1 -- 172.21.15.163:0/3422126560 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc1b0066f30 msgr2=0x7fc1b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:14.066 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.065+0000 7fc1cb2cb700 1 --2- 172.21.15.163:0/3422126560 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc1b0066f30 0x7fc1b00693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40af880 tx=0x7fc1b80091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.066 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.066+0000 7fc1cb2cb700 1 -- 172.21.15.163:0/3422126560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1c4159420 msgr2=0x7fc1c415c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:14.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.066+0000 7fc1cb2cb700 1 --2- 172.21.15.163:0/3422126560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1c4159420 0x7fc1c415c940 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00c590 tx=0x7fc1bc00c8a0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.066+0000 7fc1cb2cb700 1 -- 172.21.15.163:0/3422126560 shutdown_connections 2023-10-09T06:31:14.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.066+0000 7fc1cb2cb700 1 --2- 172.21.15.163:0/3422126560 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc1b0066f30 0x7fc1b00693e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.066+0000 7fc1cb2cb700 1 --2- 172.21.15.163:0/3422126560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1c4159420 0x7fc1c415c940 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.066+0000 7fc1cb2cb700 1 --2- 172.21.15.163:0/3422126560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1c415d230 0x7fc1c4153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.066+0000 7fc1cb2cb700 1 --2- 172.21.15.163:0/3422126560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1c41597f0 0x7fc1c415a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.066+0000 7fc1cb2cb700 1 -- 172.21.15.163:0/3422126560 >> 172.21.15.163:0/3422126560 conn(0x7fc1c40d0660 msgr2=0x7fc1c40b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:14.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.066+0000 7fc1cb2cb700 1 -- 172.21.15.163:0/3422126560 shutdown_connections 2023-10-09T06:31:14.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.066+0000 7fc1cb2cb700 1 -- 172.21.15.163:0/3422126560 wait complete. 2023-10-09T06:31:14.078 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-09T06:30:41.369014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-09T06:30:42.103001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-09T06:31:14.078 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:31:14.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.195+0000 7ff3b1314700 1 Processor -- start 2023-10-09T06:31:14.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.196+0000 7ff3b1314700 1 -- start start 2023-10-09T06:31:14.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.196+0000 7ff3b1314700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff3ac069560 0x7ff3ac0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:14.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.196+0000 7ff3b1314700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3ac069ee0 con 0x7ff3ac159c60 2023-10-09T06:31:14.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.196+0000 7ff3b1314700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3ac15a160 con 0x7ff3ac159890 2023-10-09T06:31:14.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.196+0000 7ff3b1314700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3ac0d13a0 con 0x7ff3ac069560 2023-10-09T06:31:14.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.197+0000 7ff3ab7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff3ac069560 0x7ff3ac0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:14.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.197+0000 7ff3a27fc700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff3ac159c60 0x7ff3ac15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53258/0 (socket says 172.21.15.163:53258) 2023-10-09T06:31:14.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.197+0000 7ff3a27fc700 1 -- 172.21.15.163:0/4177931037 learned_addr learned my addr 172.21.15.163:0/4177931037 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:14.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.197+0000 7ff3ab7fe700 1 -- 172.21.15.163:0/4177931037 >> v1:172.21.15.181:6789/0 conn(0x7ff3ac159890 legacy=0x7ff3ac15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:14.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.197+0000 7ff3ab7fe700 1 -- 172.21.15.163:0/4177931037 >> v1:172.21.15.163:6789/0 conn(0x7ff3ac159c60 legacy=0x7ff3ac15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:14.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.197+0000 7ff3ab7fe700 1 -- 172.21.15.163:0/4177931037 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ac15e140 con 0x7ff3ac069560 2023-10-09T06:31:14.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.197+0000 7ff3aa7fc700 1 -- 172.21.15.163:0/4177931037 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 153051666 0 0) 0x7ff3ac069ee0 con 0x7ff3ac159c60 2023-10-09T06:31:14.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.197+0000 7ff3ab7fe700 1 --2- 172.21.15.163:0/4177931037 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3ac069560 0x7ff3ac0699a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009e60 tx=0x7ff3a4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0c3e5fbd779670 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:14.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.197+0000 7ff3aa7fc700 1 -- 172.21.15.163:0/4177931037 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a400e070 con 0x7ff3ac069560 2023-10-09T06:31:14.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.197+0000 7ff3aa7fc700 1 -- 172.21.15.163:0/4177931037 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a400bc30 con 0x7ff3ac069560 2023-10-09T06:31:14.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.197+0000 7ff3aa7fc700 1 -- 172.21.15.163:0/4177931037 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4012430 con 0x7ff3ac069560 2023-10-09T06:31:14.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.198+0000 7ff3b1314700 1 -- 172.21.15.163:0/4177931037 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3ac069560 msgr2=0x7ff3ac0699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:14.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.198+0000 7ff3b1314700 1 --2- 172.21.15.163:0/4177931037 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3ac069560 0x7ff3ac0699a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009e60 tx=0x7ff3a4009780 comp rx=0 tx=0).stop 2023-10-09T06:31:14.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.198+0000 7ff3b1314700 1 -- 172.21.15.163:0/4177931037 shutdown_connections 2023-10-09T06:31:14.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.198+0000 7ff3b1314700 1 --2- 172.21.15.163:0/4177931037 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3ac069560 0x7ff3ac0699a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.198+0000 7ff3b1314700 1 -- 172.21.15.163:0/4177931037 >> 172.21.15.163:0/4177931037 conn(0x7ff3ac068e20 msgr2=0x7ff3ac0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:14.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.198+0000 7ff3b1314700 1 -- 172.21.15.163:0/4177931037 shutdown_connections 2023-10-09T06:31:14.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.198+0000 7ff3b1314700 1 -- 172.21.15.163:0/4177931037 wait complete. 2023-10-09T06:31:14.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.199+0000 7ff3b1314700 1 Processor -- start 2023-10-09T06:31:14.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.199+0000 7ff3b1314700 1 -- start start 2023-10-09T06:31:14.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.200+0000 7ff3b1314700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3ac069560 0x7ff3ac15b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:14.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.200+0000 7ff3b1314700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3ac159890 0x7ff3ac0cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:14.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.200+0000 7ff3b1314700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3ac159c60 0x7ff3ac1452f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:14.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.200+0000 7ff3b1314700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac15cb90 con 0x7ff3ac159c60 2023-10-09T06:31:14.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.200+0000 7ff3b1314700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac15bf80 con 0x7ff3ac159890 2023-10-09T06:31:14.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.200+0000 7ff3b1314700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac15c260 con 0x7ff3ac069560 2023-10-09T06:31:14.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.200+0000 7ff3ab7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3ac159c60 0x7ff3ac1452f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:14.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.200+0000 7ff3ab7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3ac159c60 0x7ff3ac1452f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53300/0 (socket says 172.21.15.163:53300) 2023-10-09T06:31:14.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.200+0000 7ff3ab7fe700 1 -- 172.21.15.163:0/544924880 learned_addr learned my addr 172.21.15.163:0/544924880 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:14.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.200+0000 7ff3a27fc700 1 --2- 172.21.15.163:0/544924880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3ac159890 0x7ff3ac0cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:14.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.200+0000 7ff3aaffd700 1 --2- 172.21.15.163:0/544924880 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3ac069560 0x7ff3ac15b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:14.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.200+0000 7ff3ab7fe700 1 -- 172.21.15.163:0/544924880 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3ac069560 msgr2=0x7ff3ac15b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:14.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.200+0000 7ff3ab7fe700 1 --2- 172.21.15.163:0/544924880 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3ac069560 0x7ff3ac15b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.200+0000 7ff3ab7fe700 1 -- 172.21.15.163:0/544924880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3ac159890 msgr2=0x7ff3ac0cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:14.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.200+0000 7ff3ab7fe700 1 --2- 172.21.15.163:0/544924880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3ac159890 0x7ff3ac0cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.200+0000 7ff3ab7fe700 1 -- 172.21.15.163:0/544924880 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a4009a20 con 0x7ff3ac159c60 2023-10-09T06:31:14.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.200+0000 7ff3ab7fe700 1 --2- 172.21.15.163:0/544924880 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3ac159c60 0x7ff3ac1452f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4013000 tx=0x7ff3a4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:14.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.200+0000 7ff3a37fe700 1 -- 172.21.15.163:0/544924880 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a401b420 con 0x7ff3ac159c60 2023-10-09T06:31:14.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.200+0000 7ff3a37fe700 1 -- 172.21.15.163:0/544924880 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a401b580 con 0x7ff3ac159c60 2023-10-09T06:31:14.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.200+0000 7ff3b1314700 1 -- 172.21.15.163:0/544924880 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ac15c540 con 0x7ff3ac159c60 2023-10-09T06:31:14.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.200+0000 7ff3a37fe700 1 -- 172.21.15.163:0/544924880 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a401b900 con 0x7ff3ac159c60 2023-10-09T06:31:14.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.201+0000 7ff3b1314700 1 -- 172.21.15.163:0/544924880 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ac145830 con 0x7ff3ac159c60 2023-10-09T06:31:14.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.202+0000 7ff3b1314700 1 -- 172.21.15.163:0/544924880 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ac15cd00 con 0x7ff3ac159c60 2023-10-09T06:31:14.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.202+0000 7ff3a37fe700 1 -- 172.21.15.163:0/544924880 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff3a401bc70 con 0x7ff3ac159c60 2023-10-09T06:31:14.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.202+0000 7ff3a37fe700 1 --2- 172.21.15.163:0/544924880 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff38c066f30 0x7ff38c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:14.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.202+0000 7ff3a37fe700 1 -- 172.21.15.163:0/544924880 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff3a4086340 con 0x7ff3ac159c60 2023-10-09T06:31:14.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.202+0000 7ff3aaffd700 1 --2- 172.21.15.163:0/544924880 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff38c066f30 0x7ff38c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:14.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.203+0000 7ff3aaffd700 1 --2- 172.21.15.163:0/544924880 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff38c066f30 0x7ff38c0693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff39c003a80 tx=0x7ff39c007ec0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:14.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.206+0000 7ff3a37fe700 1 -- 172.21.15.163:0/544924880 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff3a4053660 con 0x7ff3ac159c60 2023-10-09T06:31:14.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.389+0000 7ff3b1314700 1 -- 172.21.15.163:0/544924880 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff3ac04c3c0 con 0x7ff3ac159c60 2023-10-09T06:31:14.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.390+0000 7ff3a37fe700 1 -- 172.21.15.163:0/544924880 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff3a402a020 con 0x7ff3ac159c60 2023-10-09T06:31:14.390 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:31:14.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.392+0000 7ff3b1314700 1 -- 172.21.15.163:0/544924880 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff38c066f30 msgr2=0x7ff38c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:14.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.392+0000 7ff3b1314700 1 --2- 172.21.15.163:0/544924880 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff38c066f30 0x7ff38c0693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff39c003a80 tx=0x7ff39c007ec0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.392+0000 7ff3b1314700 1 -- 172.21.15.163:0/544924880 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3ac159c60 msgr2=0x7ff3ac1452f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:14.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.392+0000 7ff3b1314700 1 --2- 172.21.15.163:0/544924880 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3ac159c60 0x7ff3ac1452f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4013000 tx=0x7ff3a4019000 comp rx=0 tx=0).stop 2023-10-09T06:31:14.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.392+0000 7ff3b1314700 1 -- 172.21.15.163:0/544924880 shutdown_connections 2023-10-09T06:31:14.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.392+0000 7ff3b1314700 1 --2- 172.21.15.163:0/544924880 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff38c066f30 0x7ff38c0693e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.392+0000 7ff3b1314700 1 --2- 172.21.15.163:0/544924880 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3ac159c60 0x7ff3ac1452f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.392+0000 7ff3b1314700 1 --2- 172.21.15.163:0/544924880 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3ac069560 0x7ff3ac15b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.392+0000 7ff3b1314700 1 --2- 172.21.15.163:0/544924880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3ac159890 0x7ff3ac0cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.392+0000 7ff3b1314700 1 -- 172.21.15.163:0/544924880 >> 172.21.15.163:0/544924880 conn(0x7ff3ac068e20 msgr2=0x7ff3ac1553d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:14.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.392+0000 7ff3b1314700 1 -- 172.21.15.163:0/544924880 shutdown_connections 2023-10-09T06:31:14.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.392+0000 7ff3b1314700 1 -- 172.21.15.163:0/544924880 wait complete. 2023-10-09T06:31:14.404 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-09T06:31:14.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.521+0000 7f4d3f320700 1 Processor -- start 2023-10-09T06:31:14.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.521+0000 7f4d3f320700 1 -- start start 2023-10-09T06:31:14.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.522+0000 7f4d3f320700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4d38159c60 0x7f4d3815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:14.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.522+0000 7f4d3f320700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4d3815db10 0x7f4d38069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:14.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.522+0000 7f4d3f320700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d3815a160 con 0x7f4d38159890 2023-10-09T06:31:14.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.522+0000 7f4d3f320700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d3815df50 con 0x7f4d3815db10 2023-10-09T06:31:14.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.522+0000 7f4d3f320700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d380d13a0 con 0x7f4d38159c60 2023-10-09T06:31:14.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.522+0000 7f4d3d8bd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4d3815db10 0x7f4d38069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:14.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.522+0000 7f4d3d0bc700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f4d38159890 0x7f4d3815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53262/0 (socket says 172.21.15.163:53262) 2023-10-09T06:31:14.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.522+0000 7f4d3d0bc700 1 -- 172.21.15.163:0/451632377 learned_addr learned my addr 172.21.15.163:0/451632377 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:14.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.522+0000 7f4d3c8bb700 1 --2- 172.21.15.163:0/451632377 >> v2:172.21.15.181:3300/0 conn(0x7f4d38159c60 0x7f4d3815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:14.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.522+0000 7f4d3d8bd700 1 -- 172.21.15.163:0/451632377 >> v2:172.21.15.181:3300/0 conn(0x7f4d38159c60 msgr2=0x7f4d3815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:14.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.522+0000 7f4d3d8bd700 1 --2- 172.21.15.163:0/451632377 >> v2:172.21.15.181:3300/0 conn(0x7f4d38159c60 0x7f4d3815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.522+0000 7f4d3d8bd700 1 -- 172.21.15.163:0/451632377 >> v1:172.21.15.163:6789/0 conn(0x7f4d38159890 legacy=0x7f4d3815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:14.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.522+0000 7f4d3d8bd700 1 -- 172.21.15.163:0/451632377 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3815e140 con 0x7f4d3815db10 2023-10-09T06:31:14.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.523+0000 7f4d3d8bd700 1 --2- 172.21.15.163:0/451632377 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d3815db10 0x7f4d38069560 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4d380b67f0 tx=0x7f4d30009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae47825e7864cac6 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:14.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.523+0000 7f4d2ffff700 1 -- 172.21.15.163:0/451632377 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3000e070 con 0x7f4d3815db10 2023-10-09T06:31:14.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.523+0000 7f4d2ffff700 1 -- 172.21.15.163:0/451632377 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d3000bc30 con 0x7f4d3815db10 2023-10-09T06:31:14.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.523+0000 7f4d2ffff700 1 -- 172.21.15.163:0/451632377 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d30012430 con 0x7f4d3815db10 2023-10-09T06:31:14.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.523+0000 7f4d3f320700 1 -- 172.21.15.163:0/451632377 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d3815db10 msgr2=0x7f4d38069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:14.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.523+0000 7f4d3f320700 1 --2- 172.21.15.163:0/451632377 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d3815db10 0x7f4d38069560 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4d380b67f0 tx=0x7f4d30009780 comp rx=0 tx=0).stop 2023-10-09T06:31:14.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.523+0000 7f4d3f320700 1 -- 172.21.15.163:0/451632377 shutdown_connections 2023-10-09T06:31:14.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.523+0000 7f4d3f320700 1 --2- 172.21.15.163:0/451632377 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d3815db10 0x7f4d38069560 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.523+0000 7f4d3f320700 1 --2- 172.21.15.163:0/451632377 >> v2:172.21.15.181:3300/0 conn(0x7f4d38159c60 0x7f4d3815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.523+0000 7f4d3f320700 1 -- 172.21.15.163:0/451632377 >> 172.21.15.163:0/451632377 conn(0x7f4d38068e20 msgr2=0x7f4d380d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:14.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.523+0000 7f4d3f320700 1 -- 172.21.15.163:0/451632377 shutdown_connections 2023-10-09T06:31:14.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.523+0000 7f4d3f320700 1 -- 172.21.15.163:0/451632377 wait complete. 2023-10-09T06:31:14.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.524+0000 7f4d3f320700 1 Processor -- start 2023-10-09T06:31:14.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.525+0000 7f4d3f320700 1 -- start start 2023-10-09T06:31:14.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.525+0000 7f4d3f320700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d38159890 0x7f4d38153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:14.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.525+0000 7f4d3f320700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d38159c60 0x7f4d38150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:14.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.525+0000 7f4d3f320700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d3815db10 0x7f4d380b1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:14.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.525+0000 7f4d3f320700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d380b1280 con 0x7f4d38159890 2023-10-09T06:31:14.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.525+0000 7f4d3f320700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d38153c90 con 0x7f4d38159c60 2023-10-09T06:31:14.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.525+0000 7f4d3f320700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d38151440 con 0x7f4d3815db10 2023-10-09T06:31:14.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.525+0000 7f4d3d0bc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d38159890 0x7f4d38153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:14.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.525+0000 7f4d3d0bc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d38159890 0x7f4d38153750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53306/0 (socket says 172.21.15.163:53306) 2023-10-09T06:31:14.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.525+0000 7f4d3d0bc700 1 -- 172.21.15.163:0/2535620904 learned_addr learned my addr 172.21.15.163:0/2535620904 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:14.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.526+0000 7f4d3d0bc700 1 -- 172.21.15.163:0/2535620904 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d3815db10 msgr2=0x7f4d380b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:14.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.526+0000 7f4d3d0bc700 1 --2- 172.21.15.163:0/2535620904 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d3815db10 0x7f4d380b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.526+0000 7f4d3d0bc700 1 -- 172.21.15.163:0/2535620904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d38159c60 msgr2=0x7f4d38150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:14.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.526+0000 7f4d3d0bc700 1 --2- 172.21.15.163:0/2535620904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d38159c60 0x7f4d38150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.526+0000 7f4d3d0bc700 1 -- 172.21.15.163:0/2535620904 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d30009a20 con 0x7f4d38159890 2023-10-09T06:31:14.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.526+0000 7f4d3d0bc700 1 --2- 172.21.15.163:0/2535620904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d38159890 0x7f4d38153750 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400c7c0 tx=0x7f4d34007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:14.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.526+0000 7f4d2d7fa700 1 -- 172.21.15.163:0/2535620904 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d34005b30 con 0x7f4d38159890 2023-10-09T06:31:14.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.526+0000 7f4d2d7fa700 1 -- 172.21.15.163:0/2535620904 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d34005cb0 con 0x7f4d38159890 2023-10-09T06:31:14.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.526+0000 7f4d3f320700 1 -- 172.21.15.163:0/2535620904 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d380b0670 con 0x7f4d38159890 2023-10-09T06:31:14.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.526+0000 7f4d2d7fa700 1 -- 172.21.15.163:0/2535620904 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d34004a80 con 0x7f4d38159890 2023-10-09T06:31:14.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.526+0000 7f4d3f320700 1 -- 172.21.15.163:0/2535620904 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d380b0c30 con 0x7f4d38159890 2023-10-09T06:31:14.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.527+0000 7f4d3f320700 1 -- 172.21.15.163:0/2535620904 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d380b1560 con 0x7f4d38159890 2023-10-09T06:31:14.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.527+0000 7f4d2d7fa700 1 -- 172.21.15.163:0/2535620904 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4d340070f0 con 0x7f4d38159890 2023-10-09T06:31:14.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.528+0000 7f4d2d7fa700 1 --2- 172.21.15.163:0/2535620904 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4d1c066f30 0x7f4d1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:14.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.528+0000 7f4d2d7fa700 1 -- 172.21.15.163:0/2535620904 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4d340838f0 con 0x7f4d38159890 2023-10-09T06:31:14.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.528+0000 7f4d3c8bb700 1 --2- 172.21.15.163:0/2535620904 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4d1c066f30 0x7f4d1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:14.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.528+0000 7f4d3c8bb700 1 --2- 172.21.15.163:0/2535620904 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4d1c066f30 0x7f4d1c0693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4d380b6340 tx=0x7f4d28008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:14.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.532+0000 7f4d2d7fa700 1 -- 172.21.15.163:0/2535620904 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d34050be0 con 0x7f4d38159890 2023-10-09T06:31:14.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.717+0000 7f4d3f320700 1 -- 172.21.15.163:0/2535620904 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4d3804c3c0 con 0x7f4d38159890 2023-10-09T06:31:14.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.717+0000 7f4d2d7fa700 1 -- 172.21.15.163:0/2535620904 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4d34018020 con 0x7f4d38159890 2023-10-09T06:31:14.717 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 32 2023-10-09T06:31:14.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.719+0000 7f4d3f320700 1 -- 172.21.15.163:0/2535620904 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4d1c066f30 msgr2=0x7f4d1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:14.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.719+0000 7f4d3f320700 1 --2- 172.21.15.163:0/2535620904 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4d1c066f30 0x7f4d1c0693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4d380b6340 tx=0x7f4d28008040 comp rx=0 tx=0).stop 2023-10-09T06:31:14.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.719+0000 7f4d3f320700 1 -- 172.21.15.163:0/2535620904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d38159890 msgr2=0x7f4d38153750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:14.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.719+0000 7f4d3f320700 1 --2- 172.21.15.163:0/2535620904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d38159890 0x7f4d38153750 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400c7c0 tx=0x7f4d34007f70 comp rx=0 tx=0).stop 2023-10-09T06:31:14.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.719+0000 7f4d3f320700 1 -- 172.21.15.163:0/2535620904 shutdown_connections 2023-10-09T06:31:14.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.719+0000 7f4d3f320700 1 --2- 172.21.15.163:0/2535620904 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4d1c066f30 0x7f4d1c0693e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.719+0000 7f4d3f320700 1 --2- 172.21.15.163:0/2535620904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d38159890 0x7f4d38153750 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.719+0000 7f4d3f320700 1 --2- 172.21.15.163:0/2535620904 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d3815db10 0x7f4d380b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.719+0000 7f4d3f320700 1 --2- 172.21.15.163:0/2535620904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d38159c60 0x7f4d38150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.719+0000 7f4d3f320700 1 -- 172.21.15.163:0/2535620904 >> 172.21.15.163:0/2535620904 conn(0x7f4d38068e20 msgr2=0x7f4d38152d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:14.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.719+0000 7f4d3f320700 1 -- 172.21.15.163:0/2535620904 shutdown_connections 2023-10-09T06:31:14.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.719+0000 7f4d3f320700 1 -- 172.21.15.163:0/2535620904 wait complete. 2023-10-09T06:31:14.731 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-09T06:31:14.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.850+0000 7fbd71ee1700 1 Processor -- start 2023-10-09T06:31:14.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.850+0000 7fbd71ee1700 1 -- start start 2023-10-09T06:31:14.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.850+0000 7fbd71ee1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbd6c159890 0x7fbd6c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:14.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.850+0000 7fbd71ee1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd6c159c60 0x7fbd6c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:14.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.850+0000 7fbd71ee1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbd6c15db10 0x7fbd6c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:14.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.850+0000 7fbd71ee1700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd6c15a160 con 0x7fbd6c15db10 2023-10-09T06:31:14.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.850+0000 7fbd71ee1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd6c0d13a0 con 0x7fbd6c159890 2023-10-09T06:31:14.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.850+0000 7fbd71ee1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c15e140 con 0x7fbd6c159c60 2023-10-09T06:31:14.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.851+0000 7fbd6bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbd6c15db10 0x7fbd6c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:14.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.851+0000 7fbd6bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbd6c15db10 0x7fbd6c069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53310/0 (socket says 172.21.15.163:53310) 2023-10-09T06:31:14.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.851+0000 7fbd6bfff700 1 -- 172.21.15.163:0/1477137253 learned_addr learned my addr 172.21.15.163:0/1477137253 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:14.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.851+0000 7fbd6affd700 1 --2- 172.21.15.163:0/1477137253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd6c159c60 0x7fbd6c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:14.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.851+0000 7fbd6b7fe700 1 --2- 172.21.15.163:0/1477137253 >> v2:172.21.15.181:3300/0 conn(0x7fbd6c159890 0x7fbd6c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:14.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.851+0000 7fbd6bfff700 1 -- 172.21.15.163:0/1477137253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd6c159c60 msgr2=0x7fbd6c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:14.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.851+0000 7fbd6bfff700 1 --2- 172.21.15.163:0/1477137253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd6c159c60 0x7fbd6c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.851+0000 7fbd6bfff700 1 -- 172.21.15.163:0/1477137253 >> v2:172.21.15.181:3300/0 conn(0x7fbd6c159890 msgr2=0x7fbd6c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:14.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.851+0000 7fbd6bfff700 1 --2- 172.21.15.163:0/1477137253 >> v2:172.21.15.181:3300/0 conn(0x7fbd6c159890 0x7fbd6c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.851+0000 7fbd6bfff700 1 -- 172.21.15.163:0/1477137253 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6c15a3d0 con 0x7fbd6c15db10 2023-10-09T06:31:14.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.851+0000 7fbd6bfff700 1 --2- 172.21.15.163:0/1477137253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd6c15db10 0x7fbd6c069560 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c0d1090 tx=0x7fbd64009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b075eb015960de6 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:14.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.851+0000 7fbd6a7fc700 1 -- 172.21.15.163:0/1477137253 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6400e070 con 0x7fbd6c15db10 2023-10-09T06:31:14.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.851+0000 7fbd6a7fc700 1 -- 172.21.15.163:0/1477137253 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6400ba50 con 0x7fbd6c15db10 2023-10-09T06:31:14.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.851+0000 7fbd6a7fc700 1 -- 172.21.15.163:0/1477137253 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd640092e0 con 0x7fbd6c15db10 2023-10-09T06:31:14.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.852+0000 7fbd71ee1700 1 -- 172.21.15.163:0/1477137253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd6c15db10 msgr2=0x7fbd6c069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:14.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.852+0000 7fbd71ee1700 1 --2- 172.21.15.163:0/1477137253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd6c15db10 0x7fbd6c069560 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c0d1090 tx=0x7fbd64009840 comp rx=0 tx=0).stop 2023-10-09T06:31:14.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.852+0000 7fbd71ee1700 1 -- 172.21.15.163:0/1477137253 shutdown_connections 2023-10-09T06:31:14.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.852+0000 7fbd71ee1700 1 --2- 172.21.15.163:0/1477137253 >> v2:172.21.15.181:3300/0 conn(0x7fbd6c159890 0x7fbd6c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.852+0000 7fbd71ee1700 1 --2- 172.21.15.163:0/1477137253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd6c15db10 0x7fbd6c069560 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.852+0000 7fbd71ee1700 1 --2- 172.21.15.163:0/1477137253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd6c159c60 0x7fbd6c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.852+0000 7fbd71ee1700 1 -- 172.21.15.163:0/1477137253 >> 172.21.15.163:0/1477137253 conn(0x7fbd6c068e20 msgr2=0x7fbd6c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:14.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.852+0000 7fbd71ee1700 1 -- 172.21.15.163:0/1477137253 shutdown_connections 2023-10-09T06:31:14.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.852+0000 7fbd71ee1700 1 -- 172.21.15.163:0/1477137253 wait complete. 2023-10-09T06:31:14.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.853+0000 7fbd71ee1700 1 Processor -- start 2023-10-09T06:31:14.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.853+0000 7fbd71ee1700 1 -- start start 2023-10-09T06:31:14.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.854+0000 7fbd71ee1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd6c159890 0x7fbd6c14c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:14.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.854+0000 7fbd71ee1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd6c159c60 0x7fbd6c152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:14.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.854+0000 7fbd71ee1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd6c15db10 0x7fbd6c15e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:14.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.854+0000 7fbd71ee1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c0b1340 con 0x7fbd6c15db10 2023-10-09T06:31:14.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.854+0000 7fbd71ee1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c0b1620 con 0x7fbd6c159c60 2023-10-09T06:31:14.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.854+0000 7fbd71ee1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c0b1900 con 0x7fbd6c159890 2023-10-09T06:31:14.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.854+0000 7fbd6bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd6c15db10 0x7fbd6c15e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:14.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.854+0000 7fbd6bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd6c15db10 0x7fbd6c15e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53314/0 (socket says 172.21.15.163:53314) 2023-10-09T06:31:14.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.854+0000 7fbd6bfff700 1 -- 172.21.15.163:0/3533193602 learned_addr learned my addr 172.21.15.163:0/3533193602 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:14.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.854+0000 7fbd6affd700 1 --2- 172.21.15.163:0/3533193602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd6c159c60 0x7fbd6c152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:14.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.854+0000 7fbd6b7fe700 1 --2- 172.21.15.163:0/3533193602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd6c159890 0x7fbd6c14c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:14.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.854+0000 7fbd6bfff700 1 -- 172.21.15.163:0/3533193602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd6c159890 msgr2=0x7fbd6c14c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:14.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.854+0000 7fbd6bfff700 1 --2- 172.21.15.163:0/3533193602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd6c159890 0x7fbd6c14c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.854+0000 7fbd6bfff700 1 -- 172.21.15.163:0/3533193602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd6c159c60 msgr2=0x7fbd6c152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:14.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.854+0000 7fbd6bfff700 1 --2- 172.21.15.163:0/3533193602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd6c159c60 0x7fbd6c152350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:14.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.854+0000 7fbd6bfff700 1 -- 172.21.15.163:0/3533193602 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd64009ae0 con 0x7fbd6c15db10 2023-10-09T06:31:14.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.854+0000 7fbd6affd700 1 --2- 172.21.15.163:0/3533193602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd6c159c60 0x7fbd6c152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:14.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.854+0000 7fbd6bfff700 1 --2- 172.21.15.163:0/3533193602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd6c15db10 0x7fbd6c15e340 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c070e20 tx=0x7fbd64018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:14.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.855+0000 7fbd637fe700 1 -- 172.21.15.163:0/3533193602 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd640032f0 con 0x7fbd6c15db10 2023-10-09T06:31:14.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.855+0000 7fbd637fe700 1 -- 172.21.15.163:0/3533193602 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd64008e80 con 0x7fbd6c15db10 2023-10-09T06:31:14.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.855+0000 7fbd71ee1700 1 -- 172.21.15.163:0/3533193602 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd6c0b1be0 con 0x7fbd6c15db10 2023-10-09T06:31:14.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.855+0000 7fbd637fe700 1 -- 172.21.15.163:0/3533193602 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6401c660 con 0x7fbd6c15db10 2023-10-09T06:31:14.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.855+0000 7fbd71ee1700 1 -- 172.21.15.163:0/3533193602 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd6c0b0fd0 con 0x7fbd6c15db10 2023-10-09T06:31:14.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.856+0000 7fbd71ee1700 1 -- 172.21.15.163:0/3533193602 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd6c04c3c0 con 0x7fbd6c15db10 2023-10-09T06:31:14.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.856+0000 7fbd637fe700 1 -- 172.21.15.163:0/3533193602 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbd64003040 con 0x7fbd6c15db10 2023-10-09T06:31:14.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.856+0000 7fbd637fe700 1 --2- 172.21.15.163:0/3533193602 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbd4c066f30 0x7fbd4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:14.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.857+0000 7fbd637fe700 1 -- 172.21.15.163:0/3533193602 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbd64085e40 con 0x7fbd6c15db10 2023-10-09T06:31:14.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.857+0000 7fbd6b7fe700 1 --2- 172.21.15.163:0/3533193602 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbd4c066f30 0x7fbd4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:14.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.857+0000 7fbd6b7fe700 1 --2- 172.21.15.163:0/3533193602 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbd4c066f30 0x7fbd4c0693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbd54006fd0 tx=0x7fbd54008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:14.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:14.861+0000 7fbd637fe700 1 -- 172.21.15.163:0/3533193602 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd640530b0 con 0x7fbd6c15db10 2023-10-09T06:31:15.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.041+0000 7fbd71ee1700 1 -- 172.21.15.163:0/3533193602 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbd6c153420 con 0x7fbd6c15db10 2023-10-09T06:31:15.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.041+0000 7fbd637fe700 1 -- 172.21.15.163:0/3533193602 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbd64009940 con 0x7fbd6c15db10 2023-10-09T06:31:15.041 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 64 2023-10-09T06:31:15.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.044+0000 7fbd71ee1700 1 -- 172.21.15.163:0/3533193602 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbd4c066f30 msgr2=0x7fbd4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:15.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.044+0000 7fbd71ee1700 1 --2- 172.21.15.163:0/3533193602 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbd4c066f30 0x7fbd4c0693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbd54006fd0 tx=0x7fbd54008040 comp rx=0 tx=0).stop 2023-10-09T06:31:15.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.044+0000 7fbd71ee1700 1 -- 172.21.15.163:0/3533193602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd6c15db10 msgr2=0x7fbd6c15e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:15.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.044+0000 7fbd71ee1700 1 --2- 172.21.15.163:0/3533193602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd6c15db10 0x7fbd6c15e340 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c070e20 tx=0x7fbd64018000 comp rx=0 tx=0).stop 2023-10-09T06:31:15.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.044+0000 7fbd71ee1700 1 -- 172.21.15.163:0/3533193602 shutdown_connections 2023-10-09T06:31:15.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.044+0000 7fbd71ee1700 1 --2- 172.21.15.163:0/3533193602 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbd4c066f30 0x7fbd4c0693e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:15.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.044+0000 7fbd71ee1700 1 --2- 172.21.15.163:0/3533193602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd6c15db10 0x7fbd6c15e340 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:15.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.044+0000 7fbd71ee1700 1 --2- 172.21.15.163:0/3533193602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd6c159890 0x7fbd6c14c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:15.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.044+0000 7fbd71ee1700 1 --2- 172.21.15.163:0/3533193602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd6c159c60 0x7fbd6c152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:15.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.044+0000 7fbd71ee1700 1 -- 172.21.15.163:0/3533193602 >> 172.21.15.163:0/3533193602 conn(0x7fbd6c068e20 msgr2=0x7fbd6c154eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:15.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.044+0000 7fbd71ee1700 1 -- 172.21.15.163:0/3533193602 shutdown_connections 2023-10-09T06:31:15.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.044+0000 7fbd71ee1700 1 -- 172.21.15.163:0/3533193602 wait complete. 2023-10-09T06:31:15.056 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:31:15.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.170+0000 7f364111d700 1 Processor -- start 2023-10-09T06:31:15.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.170+0000 7f364111d700 1 -- start start 2023-10-09T06:31:15.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.171+0000 7f364111d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f363c069560 0x7f363c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:15.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.171+0000 7f364111d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f363c069ee0 con 0x7f363c159890 2023-10-09T06:31:15.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.171+0000 7f364111d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f363c15a160 con 0x7f363c159c60 2023-10-09T06:31:15.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.171+0000 7f364111d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f363c0d13a0 con 0x7f363c069560 2023-10-09T06:31:15.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.171+0000 7f363b59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f363c069560 0x7f363c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:15.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.171+0000 7f363ad9d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f363c159890 0x7f363c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53274/0 (socket says 172.21.15.163:53274) 2023-10-09T06:31:15.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.171+0000 7f363ad9d700 1 -- 172.21.15.163:0/3290329660 learned_addr learned my addr 172.21.15.163:0/3290329660 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:15.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.171+0000 7f363b59e700 1 -- 172.21.15.163:0/3290329660 >> v1:172.21.15.181:6789/0 conn(0x7f363c159c60 legacy=0x7f363c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:15.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.171+0000 7f363b59e700 1 -- 172.21.15.163:0/3290329660 >> v1:172.21.15.163:6789/0 conn(0x7f363c159890 legacy=0x7f363c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:15.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.171+0000 7f363b59e700 1 -- 172.21.15.163:0/3290329660 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c15e140 con 0x7f363c069560 2023-10-09T06:31:15.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.171+0000 7f3639d9b700 1 -- 172.21.15.163:0/3290329660 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2429702871 0 0) 0x7f363c069ee0 con 0x7f363c159890 2023-10-09T06:31:15.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.171+0000 7f363b59e700 1 --2- 172.21.15.163:0/3290329660 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f363c069560 0x7f363c0699a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3634009e60 tx=0x7f3634009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203a9dce4fb2abe7 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:15.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.172+0000 7f3639d9b700 1 -- 172.21.15.163:0/3290329660 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363400e070 con 0x7f363c069560 2023-10-09T06:31:15.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.172+0000 7f3639d9b700 1 -- 172.21.15.163:0/3290329660 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363400bbf0 con 0x7f363c069560 2023-10-09T06:31:15.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.172+0000 7f3639d9b700 1 -- 172.21.15.163:0/3290329660 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634012430 con 0x7f363c069560 2023-10-09T06:31:15.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.172+0000 7f364111d700 1 -- 172.21.15.163:0/3290329660 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f363c069560 msgr2=0x7f363c0699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:15.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.172+0000 7f364111d700 1 --2- 172.21.15.163:0/3290329660 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f363c069560 0x7f363c0699a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3634009e60 tx=0x7f3634009780 comp rx=0 tx=0).stop 2023-10-09T06:31:15.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.172+0000 7f364111d700 1 -- 172.21.15.163:0/3290329660 shutdown_connections 2023-10-09T06:31:15.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.172+0000 7f364111d700 1 --2- 172.21.15.163:0/3290329660 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f363c069560 0x7f363c0699a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:15.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.172+0000 7f364111d700 1 -- 172.21.15.163:0/3290329660 >> 172.21.15.163:0/3290329660 conn(0x7f363c068e20 msgr2=0x7f363c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:15.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.172+0000 7f364111d700 1 -- 172.21.15.163:0/3290329660 shutdown_connections 2023-10-09T06:31:15.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.172+0000 7f364111d700 1 -- 172.21.15.163:0/3290329660 wait complete. 2023-10-09T06:31:15.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.173+0000 7f364111d700 1 Processor -- start 2023-10-09T06:31:15.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.173+0000 7f364111d700 1 -- start start 2023-10-09T06:31:15.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.174+0000 7f364111d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f363c069560 0x7f363c1534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:15.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.174+0000 7f364111d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f363c159890 0x7f363c0b5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:15.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.174+0000 7f364111d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f363c159c60 0x7f363c154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:15.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.174+0000 7f364111d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c0cd0c0 con 0x7f363c159c60 2023-10-09T06:31:15.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.174+0000 7f364111d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c0cd960 con 0x7f363c069560 2023-10-09T06:31:15.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.174+0000 7f364111d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f363c153a30 con 0x7f363c159890 2023-10-09T06:31:15.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.174+0000 7f363b59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f363c159c60 0x7f363c154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:15.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.174+0000 7f363b59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f363c159c60 0x7f363c154060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53322/0 (socket says 172.21.15.163:53322) 2023-10-09T06:31:15.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.174+0000 7f363b59e700 1 -- 172.21.15.163:0/4159501049 learned_addr learned my addr 172.21.15.163:0/4159501049 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:15.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.174+0000 7f363ad9d700 1 --2- 172.21.15.163:0/4159501049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f363c069560 0x7f363c1534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:15.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.174+0000 7f363a59c700 1 --2- 172.21.15.163:0/4159501049 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f363c159890 0x7f363c0b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:15.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.175+0000 7f363b59e700 1 -- 172.21.15.163:0/4159501049 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f363c159890 msgr2=0x7f363c0b5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:15.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.175+0000 7f363b59e700 1 --2- 172.21.15.163:0/4159501049 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f363c159890 0x7f363c0b5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:15.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.175+0000 7f363b59e700 1 -- 172.21.15.163:0/4159501049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f363c069560 msgr2=0x7f363c1534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:15.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.175+0000 7f363b59e700 1 --2- 172.21.15.163:0/4159501049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f363c069560 0x7f363c1534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:15.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.175+0000 7f363b59e700 1 -- 172.21.15.163:0/4159501049 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3634009a20 con 0x7f363c159c60 2023-10-09T06:31:15.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.175+0000 7f363a59c700 1 --2- 172.21.15.163:0/4159501049 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f363c159890 0x7f363c0b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:15.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.175+0000 7f363b59e700 1 --2- 172.21.15.163:0/4159501049 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f363c159c60 0x7f363c154060 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f363400b6c0 tx=0x7f3634018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:15.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.175+0000 7f3622ffd700 1 -- 172.21.15.163:0/4159501049 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363401e710 con 0x7f363c159c60 2023-10-09T06:31:15.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.175+0000 7f3622ffd700 1 -- 172.21.15.163:0/4159501049 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363401e870 con 0x7f363c159c60 2023-10-09T06:31:15.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.175+0000 7f3622ffd700 1 -- 172.21.15.163:0/4159501049 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363401ec30 con 0x7f363c159c60 2023-10-09T06:31:15.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.175+0000 7f364111d700 1 -- 172.21.15.163:0/4159501049 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f363c0b62c0 con 0x7f363c159c60 2023-10-09T06:31:15.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.175+0000 7f364111d700 1 -- 172.21.15.163:0/4159501049 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f363c0cd5f0 con 0x7f363c159c60 2023-10-09T06:31:15.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.176+0000 7f364111d700 1 -- 172.21.15.163:0/4159501049 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f363c0cdad0 con 0x7f363c159c60 2023-10-09T06:31:15.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.176+0000 7f3622ffd700 1 -- 172.21.15.163:0/4159501049 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3634024070 con 0x7f363c159c60 2023-10-09T06:31:15.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.177+0000 7f3622ffd700 1 --2- 172.21.15.163:0/4159501049 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3628066f30 0x7f36280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:15.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.177+0000 7f3622ffd700 1 -- 172.21.15.163:0/4159501049 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3634087050 con 0x7f363c159c60 2023-10-09T06:31:15.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.177+0000 7f363ad9d700 1 --2- 172.21.15.163:0/4159501049 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3628066f30 0x7f36280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:15.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.177+0000 7f363ad9d700 1 --2- 172.21.15.163:0/4159501049 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3628066f30 0x7f36280693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f362c0011e0 tx=0x7f362c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:15.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.181+0000 7f3622ffd700 1 -- 172.21.15.163:0/4159501049 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f36340542c0 con 0x7f363c159c60 2023-10-09T06:31:15.407 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.407+0000 7f364111d700 1 -- 172.21.15.163:0/4159501049 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f363c04c3c0 con 0x7f363c159c60 2023-10-09T06:31:15.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.408+0000 7f3622ffd700 1 -- 172.21.15.163:0/4159501049 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4603 (secure 0 0 0) 0x7f36340578e0 con 0x7f363c159c60 2023-10-09T06:31:15.408 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:15.408 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3440080308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3440080308},{"type":"v1","addr":"172.21.15.181:6837","nonce":3440080308}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1990962358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1990962358},{"type":"v1","addr":"172.21.15.163:6835","nonce":1990962358}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2270076605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2270076605},{"type":"v1","addr":"172.21.15.163:6834","nonce":2270076605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:30:43.133124+0000","modified":"2023-10-09T06:30:46.760523+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/1149296546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1149296546},{"type":"v1","addr":"172.21.15.181:6836","nonce":1149296546}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-09T06:31:15.410 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.410+0000 7f364111d700 1 -- 172.21.15.163:0/4159501049 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3628066f30 msgr2=0x7f36280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:15.410 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.410+0000 7f364111d700 1 --2- 172.21.15.163:0/4159501049 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3628066f30 0x7f36280693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f362c0011e0 tx=0x7f362c009040 comp rx=0 tx=0).stop 2023-10-09T06:31:15.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.410+0000 7f364111d700 1 -- 172.21.15.163:0/4159501049 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f363c159c60 msgr2=0x7f363c154060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:15.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.410+0000 7f364111d700 1 --2- 172.21.15.163:0/4159501049 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f363c159c60 0x7f363c154060 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f363400b6c0 tx=0x7f3634018000 comp rx=0 tx=0).stop 2023-10-09T06:31:15.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.410+0000 7f364111d700 1 -- 172.21.15.163:0/4159501049 shutdown_connections 2023-10-09T06:31:15.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.410+0000 7f364111d700 1 --2- 172.21.15.163:0/4159501049 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3628066f30 0x7f36280693e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:15.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.410+0000 7f364111d700 1 --2- 172.21.15.163:0/4159501049 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f363c159c60 0x7f363c154060 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:15.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.410+0000 7f364111d700 1 --2- 172.21.15.163:0/4159501049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f363c069560 0x7f363c1534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:15.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.410+0000 7f364111d700 1 --2- 172.21.15.163:0/4159501049 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f363c159890 0x7f363c0b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:15.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.410+0000 7f364111d700 1 -- 172.21.15.163:0/4159501049 >> 172.21.15.163:0/4159501049 conn(0x7f363c068e20 msgr2=0x7f363c0d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:15.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.410+0000 7f364111d700 1 -- 172.21.15.163:0/4159501049 shutdown_connections 2023-10-09T06:31:15.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.410+0000 7f364111d700 1 -- 172.21.15.163:0/4159501049 wait complete. 2023-10-09T06:31:15.412 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 8 2023-10-09T06:31:15.423 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:31:15.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.540+0000 7f2825c67700 1 Processor -- start 2023-10-09T06:31:15.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.540+0000 7f2825c67700 1 -- start start 2023-10-09T06:31:15.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.541+0000 7f2825c67700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2820143050 0x7f2820159c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:15.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.541+0000 7f2825c67700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f282006b9e0 0x7f282006be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:15.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.541+0000 7f2825c67700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28200b3810 0x7f28200b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:15.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.541+0000 7f2825c67700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f282006c390 con 0x7f2820143050 2023-10-09T06:31:15.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.541+0000 7f2825c67700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2820143420 con 0x7f282006b9e0 2023-10-09T06:31:15.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.541+0000 7f2825c67700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f282006a1e0 con 0x7f28200b3810 2023-10-09T06:31:15.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.541+0000 7f2824c65700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2820143050 0x7f2820159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:15.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.541+0000 7f2824c65700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2820143050 0x7f2820159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53334/0 (socket says 172.21.15.163:53334) 2023-10-09T06:31:15.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.541+0000 7f2824c65700 1 -- 172.21.15.163:0/380438364 learned_addr learned my addr 172.21.15.163:0/380438364 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:15.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.541+0000 7f2825466700 1 --2- 172.21.15.163:0/380438364 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28200b3810 0x7f28200b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:15.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.541+0000 7f281ffff700 1 --2- 172.21.15.163:0/380438364 >> v2:172.21.15.181:3300/0 conn(0x7f282006b9e0 0x7f282006be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:15.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.541+0000 7f2824c65700 1 -- 172.21.15.163:0/380438364 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28200b3810 msgr2=0x7f28200b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:15.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.541+0000 7f2824c65700 1 --2- 172.21.15.163:0/380438364 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28200b3810 0x7f28200b3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:15.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.541+0000 7f2824c65700 1 -- 172.21.15.163:0/380438364 >> v2:172.21.15.181:3300/0 conn(0x7f282006b9e0 msgr2=0x7f282006be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:15.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.541+0000 7f2824c65700 1 --2- 172.21.15.163:0/380438364 >> v2:172.21.15.181:3300/0 conn(0x7f282006b9e0 0x7f282006be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:15.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.541+0000 7f2824c65700 1 -- 172.21.15.163:0/380438364 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28200695d0 con 0x7f2820143050 2023-10-09T06:31:15.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.542+0000 7f2824c65700 1 --2- 172.21.15.163:0/380438364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2820143050 0x7f2820159c90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f282006ca20 tx=0x7f2810009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b206ea74fffbe56 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:15.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.542+0000 7f281f7fe700 1 -- 172.21.15.163:0/380438364 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281000e070 con 0x7f2820143050 2023-10-09T06:31:15.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.542+0000 7f281f7fe700 1 -- 172.21.15.163:0/380438364 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f281000b970 con 0x7f2820143050 2023-10-09T06:31:15.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.542+0000 7f281f7fe700 1 -- 172.21.15.163:0/380438364 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28100092e0 con 0x7f2820143050 2023-10-09T06:31:15.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.542+0000 7f2825c67700 1 -- 172.21.15.163:0/380438364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2820143050 msgr2=0x7f2820159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:15.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.542+0000 7f2825c67700 1 --2- 172.21.15.163:0/380438364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2820143050 0x7f2820159c90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f282006ca20 tx=0x7f2810009840 comp rx=0 tx=0).stop 2023-10-09T06:31:15.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.542+0000 7f2825c67700 1 -- 172.21.15.163:0/380438364 shutdown_connections 2023-10-09T06:31:15.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.542+0000 7f2825c67700 1 --2- 172.21.15.163:0/380438364 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28200b3810 0x7f28200b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:15.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.542+0000 7f2825c67700 1 --2- 172.21.15.163:0/380438364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2820143050 0x7f2820159c90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:15.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.542+0000 7f2825c67700 1 --2- 172.21.15.163:0/380438364 >> v2:172.21.15.181:3300/0 conn(0x7f282006b9e0 0x7f282006be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:15.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.542+0000 7f2825c67700 1 -- 172.21.15.163:0/380438364 >> 172.21.15.163:0/380438364 conn(0x7f28200ce430 msgr2=0x7f28200ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:15.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.542+0000 7f2825c67700 1 -- 172.21.15.163:0/380438364 shutdown_connections 2023-10-09T06:31:15.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.542+0000 7f2825c67700 1 -- 172.21.15.163:0/380438364 wait complete. 2023-10-09T06:31:15.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.543+0000 7f2825c67700 1 Processor -- start 2023-10-09T06:31:15.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.544+0000 7f2825c67700 1 -- start start 2023-10-09T06:31:15.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.544+0000 7f2825c67700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f282006b9e0 0x7f2820140180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:15.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.544+0000 7f2825c67700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28200b3810 0x7f28200af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:15.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.544+0000 7f2825c67700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2820143050 0x7f28200aff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:15.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.544+0000 7f2825c67700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2820141100 con 0x7f282006b9e0 2023-10-09T06:31:15.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.544+0000 7f2825c67700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2820158320 con 0x7f2820143050 2023-10-09T06:31:15.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.544+0000 7f2825c67700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f282013ab10 con 0x7f28200b3810 2023-10-09T06:31:15.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.544+0000 7f2824c65700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f282006b9e0 0x7f2820140180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:15.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.544+0000 7f2824c65700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f282006b9e0 0x7f2820140180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53350/0 (socket says 172.21.15.163:53350) 2023-10-09T06:31:15.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.544+0000 7f2824c65700 1 -- 172.21.15.163:0/202638030 learned_addr learned my addr 172.21.15.163:0/202638030 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:15.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.545+0000 7f281ffff700 1 --2- 172.21.15.163:0/202638030 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28200b3810 0x7f28200af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:15.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.545+0000 7f2825466700 1 --2- 172.21.15.163:0/202638030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2820143050 0x7f28200aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:15.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.545+0000 7f2824c65700 1 -- 172.21.15.163:0/202638030 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28200b3810 msgr2=0x7f28200af9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:15.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.545+0000 7f2824c65700 1 --2- 172.21.15.163:0/202638030 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28200b3810 0x7f28200af9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:15.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.545+0000 7f2824c65700 1 -- 172.21.15.163:0/202638030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2820143050 msgr2=0x7f28200aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:15.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.545+0000 7f2824c65700 1 --2- 172.21.15.163:0/202638030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2820143050 0x7f28200aff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:15.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.545+0000 7f2824c65700 1 -- 172.21.15.163:0/202638030 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2810009ae0 con 0x7f282006b9e0 2023-10-09T06:31:15.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.545+0000 7f2824c65700 1 --2- 172.21.15.163:0/202638030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f282006b9e0 0x7f2820140180 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f282006edf0 tx=0x7f2810018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:15.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.545+0000 7f281cff9700 1 -- 172.21.15.163:0/202638030 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2810008720 con 0x7f282006b9e0 2023-10-09T06:31:15.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.545+0000 7f281cff9700 1 -- 172.21.15.163:0/202638030 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28100088a0 con 0x7f282006b9e0 2023-10-09T06:31:15.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.545+0000 7f281cff9700 1 -- 172.21.15.163:0/202638030 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2810008c60 con 0x7f282006b9e0 2023-10-09T06:31:15.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.545+0000 7f2825c67700 1 -- 172.21.15.163:0/202638030 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f282013a7e0 con 0x7f282006b9e0 2023-10-09T06:31:15.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.545+0000 7f2825c67700 1 -- 172.21.15.163:0/202638030 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2820140b40 con 0x7f282006b9e0 2023-10-09T06:31:15.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.546+0000 7f2825c67700 1 -- 172.21.15.163:0/202638030 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f282004c3c0 con 0x7f282006b9e0 2023-10-09T06:31:15.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.546+0000 7f281cff9700 1 -- 172.21.15.163:0/202638030 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2810003040 con 0x7f282006b9e0 2023-10-09T06:31:15.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.547+0000 7f281cff9700 1 --2- 172.21.15.163:0/202638030 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f280c066f30 0x7f280c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:15.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.547+0000 7f281cff9700 1 -- 172.21.15.163:0/202638030 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2810085d40 con 0x7f282006b9e0 2023-10-09T06:31:15.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.547+0000 7f281ffff700 1 --2- 172.21.15.163:0/202638030 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f280c066f30 0x7f280c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:15.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.548+0000 7f281ffff700 1 --2- 172.21.15.163:0/202638030 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f280c066f30 0x7f280c0693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2820140130 tx=0x7f28140091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:15.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.551+0000 7f281cff9700 1 -- 172.21.15.163:0/202638030 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2810052fb0 con 0x7f282006b9e0 2023-10-09T06:31:15.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.737+0000 7f2825c67700 1 -- 172.21.15.163:0/202638030 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28200698b0 con 0x7f282006b9e0 2023-10-09T06:31:15.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.738+0000 7f281cff9700 1 -- 172.21.15.163:0/202638030 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f28100565d0 con 0x7f282006b9e0 2023-10-09T06:31:15.737 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:15.738 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":20,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:30:44.466923+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:30:41.369014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-09T06:30:42.103001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:31:15.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.740+0000 7f2825c67700 1 -- 172.21.15.163:0/202638030 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f280c066f30 msgr2=0x7f280c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:15.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.740+0000 7f2825c67700 1 --2- 172.21.15.163:0/202638030 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f280c066f30 0x7f280c0693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2820140130 tx=0x7f28140091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:15.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.740+0000 7f2825c67700 1 -- 172.21.15.163:0/202638030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f282006b9e0 msgr2=0x7f2820140180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:15.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.740+0000 7f2825c67700 1 --2- 172.21.15.163:0/202638030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f282006b9e0 0x7f2820140180 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f282006edf0 tx=0x7f2810018000 comp rx=0 tx=0).stop 2023-10-09T06:31:15.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.740+0000 7f2825c67700 1 -- 172.21.15.163:0/202638030 shutdown_connections 2023-10-09T06:31:15.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.740+0000 7f2825c67700 1 --2- 172.21.15.163:0/202638030 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f280c066f30 0x7f280c0693e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:15.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.740+0000 7f2825c67700 1 --2- 172.21.15.163:0/202638030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f282006b9e0 0x7f2820140180 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:15.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.740+0000 7f2825c67700 1 --2- 172.21.15.163:0/202638030 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2820143050 0x7f28200aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:15.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.740+0000 7f2825c67700 1 --2- 172.21.15.163:0/202638030 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28200b3810 0x7f28200af9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:15.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.740+0000 7f2825c67700 1 -- 172.21.15.163:0/202638030 >> 172.21.15.163:0/202638030 conn(0x7f28200ce430 msgr2=0x7f28200adcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:15.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.740+0000 7f2825c67700 1 -- 172.21.15.163:0/202638030 shutdown_connections 2023-10-09T06:31:15.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.740+0000 7f2825c67700 1 -- 172.21.15.163:0/202638030 wait complete. 2023-10-09T06:31:15.752 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-09T06:31:15.752 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:31:15.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.867+0000 7f698d236700 1 Processor -- start 2023-10-09T06:31:15.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.867+0000 7f698d236700 1 -- start start 2023-10-09T06:31:15.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.868+0000 7f698d236700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f69880af250 0x7f6988068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:15.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.868+0000 7f698d236700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f69880688d0 0x7f6988157d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:15.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.868+0000 7f698d236700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f698806b030 con 0x7f6988157960 2023-10-09T06:31:15.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.868+0000 7f698d236700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69880ac760 con 0x7f69880688d0 2023-10-09T06:31:15.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.868+0000 7f698d236700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69880af830 con 0x7f69880af250 2023-10-09T06:31:15.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.868+0000 7f6986d9d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f6988157960 0x7f698806b4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53290/0 (socket says 172.21.15.163:53290) 2023-10-09T06:31:15.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.868+0000 7f698759e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f69880688d0 0x7f6988157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:15.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.868+0000 7f6986d9d700 1 -- 172.21.15.163:0/3912087592 learned_addr learned my addr 172.21.15.163:0/3912087592 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:15.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.868+0000 7f697e59c700 1 --2- 172.21.15.163:0/3912087592 >> v2:172.21.15.181:3300/0 conn(0x7f69880af250 0x7f6988068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:15.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.869+0000 7f698759e700 1 -- 172.21.15.163:0/3912087592 >> v2:172.21.15.181:3300/0 conn(0x7f69880af250 msgr2=0x7f6988068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:15.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.869+0000 7f698759e700 1 --2- 172.21.15.163:0/3912087592 >> v2:172.21.15.181:3300/0 conn(0x7f69880af250 0x7f6988068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:15.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.869+0000 7f698759e700 1 -- 172.21.15.163:0/3912087592 >> v1:172.21.15.163:6789/0 conn(0x7f6988157960 legacy=0x7f698806b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:15.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.869+0000 7f698659c700 1 -- 172.21.15.163:0/3912087592 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3333850889 0 0) 0x7f698806b030 con 0x7f6988157960 2023-10-09T06:31:15.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.869+0000 7f698759e700 1 -- 172.21.15.163:0/3912087592 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f698806bbe0 con 0x7f69880688d0 2023-10-09T06:31:15.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.869+0000 7f698759e700 1 --2- 172.21.15.163:0/3912087592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69880688d0 0x7f6988157d30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f698806fe40 tx=0x7f6980009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f25d842de9af8a0a server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:15.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.869+0000 7f698659c700 1 -- 172.21.15.163:0/3912087592 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698000e070 con 0x7f69880688d0 2023-10-09T06:31:15.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.869+0000 7f698659c700 1 -- 172.21.15.163:0/3912087592 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698000bc30 con 0x7f69880688d0 2023-10-09T06:31:15.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.869+0000 7f698659c700 1 -- 172.21.15.163:0/3912087592 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6980012430 con 0x7f69880688d0 2023-10-09T06:31:15.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.869+0000 7f698d236700 1 -- 172.21.15.163:0/3912087592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69880688d0 msgr2=0x7f6988157d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:15.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.869+0000 7f698d236700 1 --2- 172.21.15.163:0/3912087592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69880688d0 0x7f6988157d30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f698806fe40 tx=0x7f6980009780 comp rx=0 tx=0).stop 2023-10-09T06:31:15.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.869+0000 7f698d236700 1 -- 172.21.15.163:0/3912087592 shutdown_connections 2023-10-09T06:31:15.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.869+0000 7f698d236700 1 --2- 172.21.15.163:0/3912087592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69880688d0 0x7f6988157d30 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:15.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.869+0000 7f698d236700 1 --2- 172.21.15.163:0/3912087592 >> v2:172.21.15.181:3300/0 conn(0x7f69880af250 0x7f6988068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:15.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.869+0000 7f698d236700 1 -- 172.21.15.163:0/3912087592 >> 172.21.15.163:0/3912087592 conn(0x7f69880c9870 msgr2=0x7f69880c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:15.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.869+0000 7f698d236700 1 -- 172.21.15.163:0/3912087592 shutdown_connections 2023-10-09T06:31:15.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.869+0000 7f698d236700 1 -- 172.21.15.163:0/3912087592 wait complete. 2023-10-09T06:31:15.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.871+0000 7f698d236700 1 Processor -- start 2023-10-09T06:31:15.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.871+0000 7f698d236700 1 -- start start 2023-10-09T06:31:15.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.871+0000 7f698d236700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69880688d0 0x7f69880b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:15.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.871+0000 7f698d236700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69880af250 0x7f698806e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:15.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.871+0000 7f698d236700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6988157960 0x7f698806e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:15.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.871+0000 7f698d236700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69881679d0 con 0x7f69880af250 2023-10-09T06:31:15.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.871+0000 7f698d236700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6988167c00 con 0x7f69880688d0 2023-10-09T06:31:15.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.871+0000 7f698d236700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f698816bc70 con 0x7f6988157960 2023-10-09T06:31:15.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.872+0000 7f697e59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69880af250 0x7f698806e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:15.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.872+0000 7f697e59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69880af250 0x7f698806e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53374/0 (socket says 172.21.15.163:53374) 2023-10-09T06:31:15.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.872+0000 7f697e59c700 1 -- 172.21.15.163:0/2403279962 learned_addr learned my addr 172.21.15.163:0/2403279962 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:15.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.872+0000 7f698759e700 1 --2- 172.21.15.163:0/2403279962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6988157960 0x7f698806e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:15.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.872+0000 7f6986d9d700 1 --2- 172.21.15.163:0/2403279962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69880688d0 0x7f69880b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:15.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.872+0000 7f697e59c700 1 -- 172.21.15.163:0/2403279962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6988157960 msgr2=0x7f698806e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:15.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.872+0000 7f697e59c700 1 --2- 172.21.15.163:0/2403279962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6988157960 0x7f698806e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:15.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.872+0000 7f697e59c700 1 -- 172.21.15.163:0/2403279962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69880688d0 msgr2=0x7f69880b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:15.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.872+0000 7f697e59c700 1 --2- 172.21.15.163:0/2403279962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69880688d0 0x7f69880b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:15.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.872+0000 7f697e59c700 1 -- 172.21.15.163:0/2403279962 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6980009a20 con 0x7f69880af250 2023-10-09T06:31:15.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.872+0000 7f6986d9d700 1 --2- 172.21.15.163:0/2403279962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69880688d0 0x7f69880b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:15.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.872+0000 7f697e59c700 1 --2- 172.21.15.163:0/2403279962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69880af250 0x7f698806e0f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f697000d890 tx=0x7f6970008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:15.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.872+0000 7f697f7fe700 1 -- 172.21.15.163:0/2403279962 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697000b600 con 0x7f69880af250 2023-10-09T06:31:15.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.872+0000 7f697f7fe700 1 -- 172.21.15.163:0/2403279962 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697000b760 con 0x7f69880af250 2023-10-09T06:31:15.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.872+0000 7f697f7fe700 1 -- 172.21.15.163:0/2403279962 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697000bae0 con 0x7f69880af250 2023-10-09T06:31:15.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.872+0000 7f698d236700 1 -- 172.21.15.163:0/2403279962 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6988167ec0 con 0x7f69880af250 2023-10-09T06:31:15.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.872+0000 7f698d236700 1 -- 172.21.15.163:0/2403279962 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69880b29e0 con 0x7f69880af250 2023-10-09T06:31:15.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.873+0000 7f698d236700 1 -- 172.21.15.163:0/2403279962 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6988168510 con 0x7f69880af250 2023-10-09T06:31:15.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.873+0000 7f697f7fe700 1 -- 172.21.15.163:0/2403279962 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f697001e460 con 0x7f69880af250 2023-10-09T06:31:15.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.874+0000 7f697f7fe700 1 --2- 172.21.15.163:0/2403279962 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6968066f30 0x7f69680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:15.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.874+0000 7f697f7fe700 1 -- 172.21.15.163:0/2403279962 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6970084be0 con 0x7f69880af250 2023-10-09T06:31:15.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.874+0000 7f6986d9d700 1 --2- 172.21.15.163:0/2403279962 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6968066f30 0x7f69680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:15.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.874+0000 7f6986d9d700 1 --2- 172.21.15.163:0/2403279962 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6968066f30 0x7f69680693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6978001890 tx=0x7f6978009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:15.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:15.878+0000 7f697f7fe700 1 -- 172.21.15.163:0/2403279962 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6970051e50 con 0x7f69880af250 2023-10-09T06:31:16.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.104+0000 7f698d236700 1 -- 172.21.15.163:0/2403279962 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f698804c3c0 con 0x7f69880af250 2023-10-09T06:31:16.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.104+0000 7f697f7fe700 1 -- 172.21.15.163:0/2403279962 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4603 (secure 0 0 0) 0x7f6970055470 con 0x7f69880af250 2023-10-09T06:31:16.104 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:16.105 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3440080308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3440080308},{"type":"v1","addr":"172.21.15.181:6837","nonce":3440080308}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1990962358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1990962358},{"type":"v1","addr":"172.21.15.163:6835","nonce":1990962358}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2270076605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2270076605},{"type":"v1","addr":"172.21.15.163:6834","nonce":2270076605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:30:43.133124+0000","modified":"2023-10-09T06:30:46.760523+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/1149296546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1149296546},{"type":"v1","addr":"172.21.15.181:6836","nonce":1149296546}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-09T06:31:16.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.106+0000 7f698d236700 1 -- 172.21.15.163:0/2403279962 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6968066f30 msgr2=0x7f69680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:16.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.106+0000 7f698d236700 1 --2- 172.21.15.163:0/2403279962 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6968066f30 0x7f69680693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6978001890 tx=0x7f6978009040 comp rx=0 tx=0).stop 2023-10-09T06:31:16.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.106+0000 7f698d236700 1 -- 172.21.15.163:0/2403279962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69880af250 msgr2=0x7f698806e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:16.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.106+0000 7f698d236700 1 --2- 172.21.15.163:0/2403279962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69880af250 0x7f698806e0f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f697000d890 tx=0x7f6970008040 comp rx=0 tx=0).stop 2023-10-09T06:31:16.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.106+0000 7f698d236700 1 -- 172.21.15.163:0/2403279962 shutdown_connections 2023-10-09T06:31:16.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.106+0000 7f698d236700 1 --2- 172.21.15.163:0/2403279962 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6968066f30 0x7f69680693e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:16.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.106+0000 7f698d236700 1 --2- 172.21.15.163:0/2403279962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6988157960 0x7f698806e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:16.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.106+0000 7f698d236700 1 --2- 172.21.15.163:0/2403279962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69880688d0 0x7f69880b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:16.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.106+0000 7f698d236700 1 --2- 172.21.15.163:0/2403279962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69880af250 0x7f698806e0f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:16.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.106+0000 7f698d236700 1 -- 172.21.15.163:0/2403279962 >> 172.21.15.163:0/2403279962 conn(0x7f69880c9870 msgr2=0x7f6988155640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:16.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.106+0000 7f698d236700 1 -- 172.21.15.163:0/2403279962 shutdown_connections 2023-10-09T06:31:16.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.106+0000 7f698d236700 1 -- 172.21.15.163:0/2403279962 wait complete. 2023-10-09T06:31:16.108 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 8 2023-10-09T06:31:16.118 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:31:16.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.233+0000 7f1ed37c7700 1 Processor -- start 2023-10-09T06:31:16.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.233+0000 7f1ed37c7700 1 -- start start 2023-10-09T06:31:16.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.234+0000 7f1ed37c7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1ecc159c60 0x7f1ecc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:16.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.234+0000 7f1ed37c7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ecc15db10 0x7f1ecc069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:16.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.234+0000 7f1ed37c7700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ecc15a160 con 0x7f1ecc159890 2023-10-09T06:31:16.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.234+0000 7f1ed37c7700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ecc15df50 con 0x7f1ecc159c60 2023-10-09T06:31:16.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.234+0000 7f1ed37c7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc0d13a0 con 0x7f1ecc15db10 2023-10-09T06:31:16.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.234+0000 7f1ed1563700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f1ecc159890 0x7f1ecc15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53294/0 (socket says 172.21.15.163:53294) 2023-10-09T06:31:16.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.234+0000 7f1ed1563700 1 -- 172.21.15.163:0/3905535026 learned_addr learned my addr 172.21.15.163:0/3905535026 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:16.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.234+0000 7f1ed0d62700 1 --2- 172.21.15.163:0/3905535026 >> v2:172.21.15.163:3300/0 conn(0x7f1ecc159c60 0x7f1ecc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:16.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.234+0000 7f1ed1d64700 1 --2- 172.21.15.163:0/3905535026 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ecc15db10 0x7f1ecc069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:16.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.235+0000 7f1ec3fff700 1 -- 172.21.15.163:0/3905535026 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4262115984 0 0) 0x7f1ecc15a160 con 0x7f1ecc159890 2023-10-09T06:31:16.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.235+0000 7f1ec3fff700 1 -- 172.21.15.163:0/3905535026 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1eb8003570 con 0x7f1ecc159890 2023-10-09T06:31:16.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.235+0000 7f1ed0d62700 1 -- 172.21.15.163:0/3905535026 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ecc15db10 msgr2=0x7f1ecc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:16.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.235+0000 7f1ed0d62700 1 --2- 172.21.15.163:0/3905535026 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ecc15db10 0x7f1ecc069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:16.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.235+0000 7f1ed0d62700 1 -- 172.21.15.163:0/3905535026 >> v1:172.21.15.163:6789/0 conn(0x7f1ecc159890 legacy=0x7f1ecc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:16.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.235+0000 7f1ed0d62700 1 -- 172.21.15.163:0/3905535026 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ecc15e140 con 0x7f1ecc159c60 2023-10-09T06:31:16.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.235+0000 7f1ed0d62700 1 --2- 172.21.15.163:0/3905535026 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ecc159c60 0x7f1ecc15d5d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0098e0 tx=0x7f1ebc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf8323a81742c6e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:16.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.235+0000 7f1ec3fff700 1 -- 172.21.15.163:0/3905535026 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc011630 con 0x7f1ecc159c60 2023-10-09T06:31:16.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.235+0000 7f1ec3fff700 1 -- 172.21.15.163:0/3905535026 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ebc011790 con 0x7f1ecc159c60 2023-10-09T06:31:16.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.235+0000 7f1ec3fff700 1 -- 172.21.15.163:0/3905535026 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc011b30 con 0x7f1ecc159c60 2023-10-09T06:31:16.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.235+0000 7f1ed37c7700 1 -- 172.21.15.163:0/3905535026 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ecc159c60 msgr2=0x7f1ecc15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:16.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.235+0000 7f1ed37c7700 1 --2- 172.21.15.163:0/3905535026 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ecc159c60 0x7f1ecc15d5d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0098e0 tx=0x7f1ebc0091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:16.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.236+0000 7f1ed37c7700 1 -- 172.21.15.163:0/3905535026 shutdown_connections 2023-10-09T06:31:16.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.236+0000 7f1ed37c7700 1 --2- 172.21.15.163:0/3905535026 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ecc15db10 0x7f1ecc069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:16.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.236+0000 7f1ed37c7700 1 --2- 172.21.15.163:0/3905535026 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ecc159c60 0x7f1ecc15d5d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:16.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.236+0000 7f1ed37c7700 1 -- 172.21.15.163:0/3905535026 >> 172.21.15.163:0/3905535026 conn(0x7f1ecc068e20 msgr2=0x7f1ecc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:16.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.236+0000 7f1ed37c7700 1 -- 172.21.15.163:0/3905535026 shutdown_connections 2023-10-09T06:31:16.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.236+0000 7f1ed37c7700 1 -- 172.21.15.163:0/3905535026 wait complete. 2023-10-09T06:31:16.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.237+0000 7f1ed37c7700 1 Processor -- start 2023-10-09T06:31:16.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.237+0000 7f1ed37c7700 1 -- start start 2023-10-09T06:31:16.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.238+0000 7f1ed37c7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ecc159890 0x7f1ecc14b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:16.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.238+0000 7f1ed37c7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ecc159c60 0x7f1ecc151580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:16.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.238+0000 7f1ed37c7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ecc15db10 0x7f1ecc151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:16.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.238+0000 7f1ed37c7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc15cab0 con 0x7f1ecc159c60 2023-10-09T06:31:16.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.238+0000 7f1ed37c7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc15c180 con 0x7f1ecc159890 2023-10-09T06:31:16.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.238+0000 7f1ed37c7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc15c460 con 0x7f1ecc15db10 2023-10-09T06:31:16.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.238+0000 7f1ed0d62700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ecc159c60 0x7f1ecc151580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:16.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.238+0000 7f1ed0d62700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ecc159c60 0x7f1ecc151580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53388/0 (socket says 172.21.15.163:53388) 2023-10-09T06:31:16.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.238+0000 7f1ed0d62700 1 -- 172.21.15.163:0/1234872512 learned_addr learned my addr 172.21.15.163:0/1234872512 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:16.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.238+0000 7f1ed1d64700 1 --2- 172.21.15.163:0/1234872512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ecc15db10 0x7f1ecc151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:16.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.238+0000 7f1ed1563700 1 --2- 172.21.15.163:0/1234872512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ecc159890 0x7f1ecc14b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:16.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.238+0000 7f1ed0d62700 1 -- 172.21.15.163:0/1234872512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ecc15db10 msgr2=0x7f1ecc151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:16.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.238+0000 7f1ed0d62700 1 --2- 172.21.15.163:0/1234872512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ecc15db10 0x7f1ecc151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:16.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.238+0000 7f1ed0d62700 1 -- 172.21.15.163:0/1234872512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ecc159890 msgr2=0x7f1ecc14b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:16.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.238+0000 7f1ed0d62700 1 --2- 172.21.15.163:0/1234872512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ecc159890 0x7f1ecc14b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:16.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.238+0000 7f1ed0d62700 1 -- 172.21.15.163:0/1234872512 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc009420 con 0x7f1ecc159c60 2023-10-09T06:31:16.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.238+0000 7f1ed1563700 1 --2- 172.21.15.163:0/1234872512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ecc159890 0x7f1ecc14b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:16.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.238+0000 7f1ed0d62700 1 --2- 172.21.15.163:0/1234872512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ecc159c60 0x7f1ecc151580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc0899c0 tx=0x7f1ebc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:16.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.238+0000 7f1ec17fa700 1 -- 172.21.15.163:0/1234872512 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc011b10 con 0x7f1ecc159c60 2023-10-09T06:31:16.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.238+0000 7f1ec17fa700 1 -- 172.21.15.163:0/1234872512 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ebc0253f0 con 0x7f1ecc159c60 2023-10-09T06:31:16.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.238+0000 7f1ed37c7700 1 -- 172.21.15.163:0/1234872512 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ecc15c740 con 0x7f1ecc159c60 2023-10-09T06:31:16.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.238+0000 7f1ec17fa700 1 -- 172.21.15.163:0/1234872512 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc025790 con 0x7f1ecc159c60 2023-10-09T06:31:16.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.238+0000 7f1ed37c7700 1 -- 172.21.15.163:0/1234872512 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ecc152290 con 0x7f1ecc159c60 2023-10-09T06:31:16.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.239+0000 7f1ed37c7700 1 -- 172.21.15.163:0/1234872512 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ecc04c3c0 con 0x7f1ecc159c60 2023-10-09T06:31:16.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.240+0000 7f1ec17fa700 1 -- 172.21.15.163:0/1234872512 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1ebc025b80 con 0x7f1ecc159c60 2023-10-09T06:31:16.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.240+0000 7f1ec17fa700 1 --2- 172.21.15.163:0/1234872512 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1eb0066f30 0x7f1eb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:16.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.240+0000 7f1ec17fa700 1 -- 172.21.15.163:0/1234872512 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1ebc085d80 con 0x7f1ecc159c60 2023-10-09T06:31:16.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.240+0000 7f1ed1563700 1 --2- 172.21.15.163:0/1234872512 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1eb0066f30 0x7f1eb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:16.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.241+0000 7f1ed1563700 1 --2- 172.21.15.163:0/1234872512 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1eb0066f30 0x7f1eb00693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8005c50 tx=0x7f1ec8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:16.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.244+0000 7f1ec17fa700 1 -- 172.21.15.163:0/1234872512 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ebc052ff0 con 0x7f1ecc159c60 2023-10-09T06:31:16.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.469+0000 7f1ed37c7700 1 -- 172.21.15.163:0/1234872512 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ecc15cbf0 con 0x7f1ecc159c60 2023-10-09T06:31:16.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.470+0000 7f1ec17fa700 1 -- 172.21.15.163:0/1234872512 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4603 (secure 0 0 0) 0x7f1ebc015090 con 0x7f1ecc159c60 2023-10-09T06:31:16.470 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:16.470 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3440080308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3440080308},{"type":"v1","addr":"172.21.15.181:6837","nonce":3440080308}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1990962358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1990962358},{"type":"v1","addr":"172.21.15.163:6835","nonce":1990962358}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2270076605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2270076605},{"type":"v1","addr":"172.21.15.163:6834","nonce":2270076605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:30:43.133124+0000","modified":"2023-10-09T06:30:46.760523+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6836/1149296546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1149296546},{"type":"v1","addr":"172.21.15.181:6836","nonce":1149296546}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-09T06:31:16.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.472+0000 7f1ed37c7700 1 -- 172.21.15.163:0/1234872512 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1eb0066f30 msgr2=0x7f1eb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:16.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.472+0000 7f1ed37c7700 1 --2- 172.21.15.163:0/1234872512 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1eb0066f30 0x7f1eb00693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8005c50 tx=0x7f1ec8009040 comp rx=0 tx=0).stop 2023-10-09T06:31:16.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.472+0000 7f1ed37c7700 1 -- 172.21.15.163:0/1234872512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ecc159c60 msgr2=0x7f1ecc151580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:16.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.472+0000 7f1ed37c7700 1 --2- 172.21.15.163:0/1234872512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ecc159c60 0x7f1ecc151580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc0899c0 tx=0x7f1ebc017000 comp rx=0 tx=0).stop 2023-10-09T06:31:16.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.472+0000 7f1ed37c7700 1 -- 172.21.15.163:0/1234872512 shutdown_connections 2023-10-09T06:31:16.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.472+0000 7f1ed37c7700 1 --2- 172.21.15.163:0/1234872512 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1eb0066f30 0x7f1eb00693e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:16.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.472+0000 7f1ed37c7700 1 --2- 172.21.15.163:0/1234872512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ecc15db10 0x7f1ecc151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:16.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.472+0000 7f1ed37c7700 1 --2- 172.21.15.163:0/1234872512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ecc159890 0x7f1ecc14b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:16.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.472+0000 7f1ed37c7700 1 --2- 172.21.15.163:0/1234872512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ecc159c60 0x7f1ecc151580 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:16.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.472+0000 7f1ed37c7700 1 -- 172.21.15.163:0/1234872512 >> 172.21.15.163:0/1234872512 conn(0x7f1ecc068e20 msgr2=0x7f1ecc157570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:16.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.472+0000 7f1ed37c7700 1 -- 172.21.15.163:0/1234872512 shutdown_connections 2023-10-09T06:31:16.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.472+0000 7f1ed37c7700 1 -- 172.21.15.163:0/1234872512 wait complete. 2023-10-09T06:31:16.474 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 8 2023-10-09T06:31:16.484 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:31:16.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.601+0000 7feee0f93700 1 Processor -- start 2023-10-09T06:31:16.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.601+0000 7feee0f93700 1 -- start start 2023-10-09T06:31:16.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.602+0000 7feee0f93700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feedc06bcd0 0x7feedc140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:16.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.602+0000 7feee0f93700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feedc0c9510 0x7feedc141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:16.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.602+0000 7feee0f93700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feedc15c2a0 con 0x7feedc06c0a0 2023-10-09T06:31:16.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.602+0000 7feee0f93700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feedc15b690 con 0x7feedc06bcd0 2023-10-09T06:31:16.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.602+0000 7feee0f93700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feedc15b970 con 0x7feedc0c9510 2023-10-09T06:31:16.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.602+0000 7feedad9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feedc0c9510 0x7feedc141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:16.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.602+0000 7feeda59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feedc06bcd0 0x7feedc140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:16.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.602+0000 7feed1d9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7feedc06c0a0 0x7feedc0c8e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:42496/0 (socket says 172.21.15.163:42496) 2023-10-09T06:31:16.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.602+0000 7feed1d9b700 1 -- 172.21.15.163:0/2744787984 learned_addr learned my addr 172.21.15.163:0/2744787984 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:16.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.603+0000 7feedad9d700 1 -- 172.21.15.163:0/2744787984 >> v2:172.21.15.181:3300/0 conn(0x7feedc06bcd0 msgr2=0x7feedc140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:16.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.603+0000 7feedad9d700 1 --2- 172.21.15.163:0/2744787984 >> v2:172.21.15.181:3300/0 conn(0x7feedc06bcd0 0x7feedc140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:16.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.603+0000 7feedad9d700 1 -- 172.21.15.163:0/2744787984 >> v1:172.21.15.181:6789/0 conn(0x7feedc06c0a0 legacy=0x7feedc0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:16.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.603+0000 7feedad9d700 1 -- 172.21.15.163:0/2744787984 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feedc15bc50 con 0x7feedc0c9510 2023-10-09T06:31:16.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.603+0000 7feeda59c700 1 --2- 172.21.15.163:0/2744787984 >> v2:172.21.15.181:3300/0 conn(0x7feedc06bcd0 0x7feedc140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:31:16.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.603+0000 7feed9d9b700 1 -- 172.21.15.163:0/2744787984 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3074530252 0 0) 0x7feedc15c2a0 con 0x7feedc06c0a0 2023-10-09T06:31:16.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.603+0000 7feedad9d700 1 --2- 172.21.15.163:0/2744787984 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feedc0c9510 0x7feedc141330 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7feed4009e60 tx=0x7feed4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55f91ca4d3ac157b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:16.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.603+0000 7feed9d9b700 1 -- 172.21.15.163:0/2744787984 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed400e070 con 0x7feedc0c9510 2023-10-09T06:31:16.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.603+0000 7feed9d9b700 1 -- 172.21.15.163:0/2744787984 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed400bb60 con 0x7feedc0c9510 2023-10-09T06:31:16.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.603+0000 7feed9d9b700 1 -- 172.21.15.163:0/2744787984 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed4012430 con 0x7feedc0c9510 2023-10-09T06:31:16.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.604+0000 7feee0f93700 1 -- 172.21.15.163:0/2744787984 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feedc0c9510 msgr2=0x7feedc141330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:16.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.604+0000 7feee0f93700 1 --2- 172.21.15.163:0/2744787984 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feedc0c9510 0x7feedc141330 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7feed4009e60 tx=0x7feed4009780 comp rx=0 tx=0).stop 2023-10-09T06:31:16.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.604+0000 7feee0f93700 1 -- 172.21.15.163:0/2744787984 shutdown_connections 2023-10-09T06:31:16.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.604+0000 7feee0f93700 1 --2- 172.21.15.163:0/2744787984 >> v2:172.21.15.181:3300/0 conn(0x7feedc06bcd0 0x7feedc140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:16.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.604+0000 7feee0f93700 1 --2- 172.21.15.163:0/2744787984 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feedc0c9510 0x7feedc141330 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:16.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.604+0000 7feee0f93700 1 -- 172.21.15.163:0/2744787984 >> 172.21.15.163:0/2744787984 conn(0x7feedc0ce600 msgr2=0x7feedc0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:16.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.604+0000 7feee0f93700 1 -- 172.21.15.163:0/2744787984 shutdown_connections 2023-10-09T06:31:16.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.604+0000 7feee0f93700 1 -- 172.21.15.163:0/2744787984 wait complete. 2023-10-09T06:31:16.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.605+0000 7feee0f93700 1 Processor -- start 2023-10-09T06:31:16.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.605+0000 7feee0f93700 1 -- start start 2023-10-09T06:31:16.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.606+0000 7feee0f93700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feedc06bcd0 0x7feedc13b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:16.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.606+0000 7feee0f93700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feedc06c0a0 0x7feedc0b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:16.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.606+0000 7feee0f93700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feedc0c9510 0x7feedc16b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:16.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.606+0000 7feee0f93700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0b36a0 con 0x7feedc0c9510 2023-10-09T06:31:16.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.606+0000 7feee0f93700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0b3980 con 0x7feedc06bcd0 2023-10-09T06:31:16.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.606+0000 7feee0f93700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0b3c60 con 0x7feedc06c0a0 2023-10-09T06:31:16.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.606+0000 7feedad9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feedc0c9510 0x7feedc16b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:16.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.606+0000 7feedad9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feedc0c9510 0x7feedc16b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53404/0 (socket says 172.21.15.163:53404) 2023-10-09T06:31:16.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.606+0000 7feedad9d700 1 -- 172.21.15.163:0/3689674231 learned_addr learned my addr 172.21.15.163:0/3689674231 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:16.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.606+0000 7feed1d9b700 1 --2- 172.21.15.163:0/3689674231 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feedc06c0a0 0x7feedc0b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:16.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.606+0000 7feeda59c700 1 --2- 172.21.15.163:0/3689674231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feedc06bcd0 0x7feedc13b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:16.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.606+0000 7feedad9d700 1 -- 172.21.15.163:0/3689674231 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feedc06c0a0 msgr2=0x7feedc0b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:16.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.606+0000 7feedad9d700 1 --2- 172.21.15.163:0/3689674231 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feedc06c0a0 0x7feedc0b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:16.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.606+0000 7feedad9d700 1 -- 172.21.15.163:0/3689674231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feedc06bcd0 msgr2=0x7feedc13b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:16.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.606+0000 7feedad9d700 1 --2- 172.21.15.163:0/3689674231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feedc06bcd0 0x7feedc13b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:16.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.606+0000 7feedad9d700 1 -- 172.21.15.163:0/3689674231 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feecc009420 con 0x7feedc0c9510 2023-10-09T06:31:16.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.606+0000 7feedad9d700 1 --2- 172.21.15.163:0/3689674231 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feedc0c9510 0x7feedc16b460 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7feed400b6c0 tx=0x7feed4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:16.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.606+0000 7feed2ffd700 1 -- 172.21.15.163:0/3689674231 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed4027470 con 0x7feedc0c9510 2023-10-09T06:31:16.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.607+0000 7feed2ffd700 1 -- 172.21.15.163:0/3689674231 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed40275d0 con 0x7feedc0c9510 2023-10-09T06:31:16.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.607+0000 7feee0f93700 1 -- 172.21.15.163:0/3689674231 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed4009a20 con 0x7feedc0c9510 2023-10-09T06:31:16.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.607+0000 7feed2ffd700 1 -- 172.21.15.163:0/3689674231 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed4027990 con 0x7feedc0c9510 2023-10-09T06:31:16.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.607+0000 7feee0f93700 1 -- 172.21.15.163:0/3689674231 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feedc159220 con 0x7feedc0c9510 2023-10-09T06:31:16.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.608+0000 7feee0f93700 1 -- 172.21.15.163:0/3689674231 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feedc159440 con 0x7feedc0c9510 2023-10-09T06:31:16.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.608+0000 7feed2ffd700 1 -- 172.21.15.163:0/3689674231 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feed4024070 con 0x7feedc0c9510 2023-10-09T06:31:16.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.608+0000 7feed2ffd700 1 --2- 172.21.15.163:0/3689674231 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7feebc066f30 0x7feebc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:16.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.608+0000 7feed2ffd700 1 -- 172.21.15.163:0/3689674231 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7feed4086ce0 con 0x7feedc0c9510 2023-10-09T06:31:16.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.608+0000 7feeda59c700 1 --2- 172.21.15.163:0/3689674231 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7feebc066f30 0x7feebc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:16.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.609+0000 7feeda59c700 1 --2- 172.21.15.163:0/3689674231 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7feebc066f30 0x7feebc0693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7feecc009f40 tx=0x7feecc0091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:16.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.612+0000 7feed2ffd700 1 -- 172.21.15.163:0/3689674231 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feed4053f50 con 0x7feedc0c9510 2023-10-09T06:31:16.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.799+0000 7feee0f93700 1 -- 172.21.15.163:0/3689674231 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feedc04c3c0 con 0x7feedc0c9510 2023-10-09T06:31:16.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.800+0000 7feed2ffd700 1 -- 172.21.15.163:0/3689674231 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7feed4057570 con 0x7feedc0c9510 2023-10-09T06:31:16.800 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:16.800 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":20,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:30:44.466923+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:30:41.369014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-09T06:30:42.103001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:31:16.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.802+0000 7feee0f93700 1 -- 172.21.15.163:0/3689674231 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7feebc066f30 msgr2=0x7feebc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:16.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.802+0000 7feee0f93700 1 --2- 172.21.15.163:0/3689674231 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7feebc066f30 0x7feebc0693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7feecc009f40 tx=0x7feecc0091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:16.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.802+0000 7feee0f93700 1 -- 172.21.15.163:0/3689674231 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feedc0c9510 msgr2=0x7feedc16b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:16.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.802+0000 7feee0f93700 1 --2- 172.21.15.163:0/3689674231 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feedc0c9510 0x7feedc16b460 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7feed400b6c0 tx=0x7feed4019000 comp rx=0 tx=0).stop 2023-10-09T06:31:16.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.802+0000 7feee0f93700 1 -- 172.21.15.163:0/3689674231 shutdown_connections 2023-10-09T06:31:16.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.802+0000 7feee0f93700 1 --2- 172.21.15.163:0/3689674231 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7feebc066f30 0x7feebc0693e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:16.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.802+0000 7feee0f93700 1 --2- 172.21.15.163:0/3689674231 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feedc0c9510 0x7feedc16b460 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:16.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.802+0000 7feee0f93700 1 --2- 172.21.15.163:0/3689674231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feedc06bcd0 0x7feedc13b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:16.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.802+0000 7feee0f93700 1 --2- 172.21.15.163:0/3689674231 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feedc06c0a0 0x7feedc0b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:16.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.802+0000 7feee0f93700 1 -- 172.21.15.163:0/3689674231 >> 172.21.15.163:0/3689674231 conn(0x7feedc0ce600 msgr2=0x7feedc158330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:16.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.802+0000 7feee0f93700 1 -- 172.21.15.163:0/3689674231 shutdown_connections 2023-10-09T06:31:16.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.802+0000 7feee0f93700 1 -- 172.21.15.163:0/3689674231 wait complete. 2023-10-09T06:31:16.814 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-09T06:31:16.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.930+0000 7f605e8be700 1 Processor -- start 2023-10-09T06:31:16.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.930+0000 7f605e8be700 1 -- start start 2023-10-09T06:31:16.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.931+0000 7f605e8be700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6058159c60 0x7f605815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:16.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.931+0000 7f605e8be700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f605815db10 0x7f6058069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:16.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.931+0000 7f605e8be700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f605815a160 con 0x7f6058159890 2023-10-09T06:31:16.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.931+0000 7f605e8be700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f605815df50 con 0x7f6058159c60 2023-10-09T06:31:16.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.931+0000 7f605e8be700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60580d13a0 con 0x7f605815db10 2023-10-09T06:31:16.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.931+0000 7f605ce5b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f605815db10 0x7f6058069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:16.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.931+0000 7f60577fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6058159c60 0x7f605815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:16.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.931+0000 7f6057fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6058159890 0x7f605815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:42502/0 (socket says 172.21.15.163:42502) 2023-10-09T06:31:16.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.931+0000 7f6057fff700 1 -- 172.21.15.163:0/3787143735 learned_addr learned my addr 172.21.15.163:0/3787143735 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:16.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.932+0000 7f6056ffd700 1 -- 172.21.15.163:0/3787143735 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 30149267 0 0) 0x7f605815a160 con 0x7f6058159890 2023-10-09T06:31:16.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.932+0000 7f6056ffd700 1 -- 172.21.15.163:0/3787143735 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6040003570 con 0x7f6058159890 2023-10-09T06:31:16.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.932+0000 7f605ce5b700 1 -- 172.21.15.163:0/3787143735 >> v2:172.21.15.181:3300/0 conn(0x7f6058159c60 msgr2=0x7f605815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:16.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.932+0000 7f605ce5b700 1 --2- 172.21.15.163:0/3787143735 >> v2:172.21.15.181:3300/0 conn(0x7f6058159c60 0x7f605815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:16.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.932+0000 7f605ce5b700 1 -- 172.21.15.163:0/3787143735 >> v1:172.21.15.181:6789/0 conn(0x7f6058159890 legacy=0x7f605815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:16.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.932+0000 7f605ce5b700 1 -- 172.21.15.163:0/3787143735 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605815e140 con 0x7f605815db10 2023-10-09T06:31:16.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.932+0000 7f605ce5b700 1 --2- 172.21.15.163:0/3787143735 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f605815db10 0x7f6058069560 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6050009e60 tx=0x7f6050009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c84c62f0fcb94a9c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:16.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.932+0000 7f6056ffd700 1 -- 172.21.15.163:0/3787143735 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605000e070 con 0x7f605815db10 2023-10-09T06:31:16.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.932+0000 7f6056ffd700 1 -- 172.21.15.163:0/3787143735 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f605000bab0 con 0x7f605815db10 2023-10-09T06:31:16.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.932+0000 7f6056ffd700 1 -- 172.21.15.163:0/3787143735 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605000be30 con 0x7f605815db10 2023-10-09T06:31:16.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.932+0000 7f605e8be700 1 -- 172.21.15.163:0/3787143735 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f605815db10 msgr2=0x7f6058069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:16.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.932+0000 7f605e8be700 1 --2- 172.21.15.163:0/3787143735 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f605815db10 0x7f6058069560 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6050009e60 tx=0x7f6050009780 comp rx=0 tx=0).stop 2023-10-09T06:31:16.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.932+0000 7f605e8be700 1 -- 172.21.15.163:0/3787143735 shutdown_connections 2023-10-09T06:31:16.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.932+0000 7f605e8be700 1 --2- 172.21.15.163:0/3787143735 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f605815db10 0x7f6058069560 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:16.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.932+0000 7f605e8be700 1 --2- 172.21.15.163:0/3787143735 >> v2:172.21.15.181:3300/0 conn(0x7f6058159c60 0x7f605815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:16.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.932+0000 7f605e8be700 1 -- 172.21.15.163:0/3787143735 >> 172.21.15.163:0/3787143735 conn(0x7f6058068e20 msgr2=0x7f60580d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:16.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.933+0000 7f605e8be700 1 -- 172.21.15.163:0/3787143735 shutdown_connections 2023-10-09T06:31:16.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.933+0000 7f605e8be700 1 -- 172.21.15.163:0/3787143735 wait complete. 2023-10-09T06:31:16.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.934+0000 7f605e8be700 1 Processor -- start 2023-10-09T06:31:16.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.934+0000 7f605e8be700 1 -- start start 2023-10-09T06:31:16.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.934+0000 7f605e8be700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6058159890 0x7f60580b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:16.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.934+0000 7f605e8be700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6058159c60 0x7f60580ce380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:16.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.934+0000 7f605e8be700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f605815db10 0x7f6058154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:16.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.934+0000 7f605e8be700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60580ca200 con 0x7f6058159890 2023-10-09T06:31:16.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.934+0000 7f605e8be700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60580c9160 con 0x7f6058159c60 2023-10-09T06:31:16.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.934+0000 7f605e8be700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60580c9440 con 0x7f605815db10 2023-10-09T06:31:16.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.935+0000 7f6057fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6058159890 0x7f60580b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:16.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.935+0000 7f6057fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6058159890 0x7f60580b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53416/0 (socket says 172.21.15.163:53416) 2023-10-09T06:31:16.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.935+0000 7f6057fff700 1 -- 172.21.15.163:0/35031201 learned_addr learned my addr 172.21.15.163:0/35031201 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:16.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.935+0000 7f60577fe700 1 --2- 172.21.15.163:0/35031201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6058159c60 0x7f60580ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:16.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.935+0000 7f605ce5b700 1 --2- 172.21.15.163:0/35031201 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f605815db10 0x7f6058154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:16.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.935+0000 7f6057fff700 1 -- 172.21.15.163:0/35031201 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f605815db10 msgr2=0x7f6058154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:16.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.935+0000 7f6057fff700 1 --2- 172.21.15.163:0/35031201 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f605815db10 0x7f6058154e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:16.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.935+0000 7f6057fff700 1 -- 172.21.15.163:0/35031201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6058159c60 msgr2=0x7f60580ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:16.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.935+0000 7f6057fff700 1 --2- 172.21.15.163:0/35031201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6058159c60 0x7f60580ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:16.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.935+0000 7f6057fff700 1 -- 172.21.15.163:0/35031201 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6050009a20 con 0x7f6058159890 2023-10-09T06:31:16.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.935+0000 7f6057fff700 1 --2- 172.21.15.163:0/35031201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6058159890 0x7f60580b5da0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f60480011e0 tx=0x7f6048007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:16.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.935+0000 7f603ffff700 1 -- 172.21.15.163:0/35031201 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6048005b30 con 0x7f6058159890 2023-10-09T06:31:16.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.935+0000 7f603ffff700 1 -- 172.21.15.163:0/35031201 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6048005c90 con 0x7f6058159890 2023-10-09T06:31:16.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.935+0000 7f605e8be700 1 -- 172.21.15.163:0/35031201 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60580c9720 con 0x7f6058159890 2023-10-09T06:31:16.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.935+0000 7f603ffff700 1 -- 172.21.15.163:0/35031201 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6048007300 con 0x7f6058159890 2023-10-09T06:31:16.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.935+0000 7f605e8be700 1 -- 172.21.15.163:0/35031201 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60580c9ce0 con 0x7f6058159890 2023-10-09T06:31:16.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.936+0000 7f605e8be700 1 -- 172.21.15.163:0/35031201 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f605804c3c0 con 0x7f6058159890 2023-10-09T06:31:16.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.937+0000 7f603ffff700 1 -- 172.21.15.163:0/35031201 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6048007670 con 0x7f6058159890 2023-10-09T06:31:16.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.937+0000 7f603ffff700 1 --2- 172.21.15.163:0/35031201 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6044066f30 0x7f60440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:16.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.937+0000 7f603ffff700 1 -- 172.21.15.163:0/35031201 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6048083940 con 0x7f6058159890 2023-10-09T06:31:16.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.937+0000 7f60577fe700 1 --2- 172.21.15.163:0/35031201 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6044066f30 0x7f60440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:16.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.938+0000 7f60577fe700 1 --2- 172.21.15.163:0/35031201 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6044066f30 0x7f60440693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f60580cfb40 tx=0x7f604c0091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:16.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:16.941+0000 7f603ffff700 1 -- 172.21.15.163:0/35031201 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6048050bb0 con 0x7f6058159890 2023-10-09T06:31:17.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.168+0000 7f605e8be700 1 -- 172.21.15.163:0/35031201 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f605806a5b0 con 0x7f6058159890 2023-10-09T06:31:17.168 INFO:tasks.ceph.mon.a.smithi163.stderr:2023-10-09T06:31:17.169+0000 7ff251c04700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-09T06:31:17.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.172+0000 7f603ffff700 1 -- 172.21.15.163:0/35031201 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6048017020 con 0x7f6058159890 2023-10-09T06:31:17.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.175+0000 7f605e8be700 1 -- 172.21.15.163:0/35031201 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6044066f30 msgr2=0x7f60440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:17.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.175+0000 7f605e8be700 1 --2- 172.21.15.163:0/35031201 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6044066f30 0x7f60440693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f60580cfb40 tx=0x7f604c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:17.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.175+0000 7f605e8be700 1 -- 172.21.15.163:0/35031201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6058159890 msgr2=0x7f60580b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:17.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.175+0000 7f605e8be700 1 --2- 172.21.15.163:0/35031201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6058159890 0x7f60580b5da0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f60480011e0 tx=0x7f6048007df0 comp rx=0 tx=0).stop 2023-10-09T06:31:17.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.175+0000 7f605e8be700 1 -- 172.21.15.163:0/35031201 shutdown_connections 2023-10-09T06:31:17.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.175+0000 7f605e8be700 1 --2- 172.21.15.163:0/35031201 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6044066f30 0x7f60440693e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:17.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.175+0000 7f605e8be700 1 --2- 172.21.15.163:0/35031201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6058159890 0x7f60580b5da0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:17.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.175+0000 7f605e8be700 1 --2- 172.21.15.163:0/35031201 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f605815db10 0x7f6058154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:17.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.175+0000 7f605e8be700 1 --2- 172.21.15.163:0/35031201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6058159c60 0x7f60580ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:17.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.175+0000 7f605e8be700 1 -- 172.21.15.163:0/35031201 >> 172.21.15.163:0/35031201 conn(0x7f6058068e20 msgr2=0x7f60580b0540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:17.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.175+0000 7f605e8be700 1 -- 172.21.15.163:0/35031201 shutdown_connections 2023-10-09T06:31:17.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.175+0000 7f605e8be700 1 -- 172.21.15.163:0/35031201 wait complete. 2023-10-09T06:31:17.178 INFO:tasks.ceph.mds.b.smithi181.stderr: -1671> 2023-10-09T06:30:47.731+0000 7fef6a0a8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-09T06:31:17.179 INFO:teuthology.orchestra.run.smithi163.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-09T06:31:17.187 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-09T06:31:17.199 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-09T06:31:17.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.305+0000 7f3917a0d700 1 Processor -- start 2023-10-09T06:31:17.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.306+0000 7f3917a0d700 1 -- start start 2023-10-09T06:31:17.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.306+0000 7f3917a0d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3910159890 0x7f391015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:17.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.306+0000 7f3917a0d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3910159c60 0x7f391015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:17.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.306+0000 7f3917a0d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f391015a160 con 0x7f391015db10 2023-10-09T06:31:17.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.306+0000 7f3917a0d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39100d13a0 con 0x7f3910159c60 2023-10-09T06:31:17.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.306+0000 7f3917a0d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f391015e140 con 0x7f3910159890 2023-10-09T06:31:17.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.307+0000 7f3914fa8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3910159c60 0x7f391015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:17.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.307+0000 7f39157a9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3910159890 0x7f391015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:17.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.307+0000 7f39157a9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3910159890 0x7f391015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:47012/0 (socket says 172.21.15.163:47012) 2023-10-09T06:31:17.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.307+0000 7f3915faa700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f391015db10 0x7f3910069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:42508/0 (socket says 172.21.15.163:42508) 2023-10-09T06:31:17.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.307+0000 7f39157a9700 1 -- 172.21.15.163:0/2390272995 learned_addr learned my addr 172.21.15.163:0/2390272995 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:17.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.307+0000 7f3914fa8700 1 -- 172.21.15.163:0/2390272995 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3910159890 msgr2=0x7f391015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:17.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.307+0000 7f3914fa8700 1 --2- 172.21.15.163:0/2390272995 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3910159890 0x7f391015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:17.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.307+0000 7f3914fa8700 1 -- 172.21.15.163:0/2390272995 >> v1:172.21.15.181:6789/0 conn(0x7f391015db10 legacy=0x7f3910069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:17.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.307+0000 7f3914fa8700 1 -- 172.21.15.163:0/2390272995 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f391015a3d0 con 0x7f3910159c60 2023-10-09T06:31:17.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.307+0000 7f3907fff700 1 -- 172.21.15.163:0/2390272995 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706877799 0 0) 0x7f391015a160 con 0x7f391015db10 2023-10-09T06:31:17.310 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.307+0000 7f39157a9700 1 --2- 172.21.15.163:0/2390272995 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3910159890 0x7f391015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:31:17.310 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.307+0000 7f3914fa8700 1 --2- 172.21.15.163:0/2390272995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3910159c60 0x7f391015d5d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f39000098e0 tx=0x7f39000091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=478276adee72a83f server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:17.310 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.307+0000 7f3907fff700 1 -- 172.21.15.163:0/2390272995 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390000b640 con 0x7f3910159c60 2023-10-09T06:31:17.310 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.308+0000 7f3907fff700 1 -- 172.21.15.163:0/2390272995 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390000b7a0 con 0x7f3910159c60 2023-10-09T06:31:17.310 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.308+0000 7f3907fff700 1 -- 172.21.15.163:0/2390272995 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390000bb40 con 0x7f3910159c60 2023-10-09T06:31:17.310 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.308+0000 7f3917a0d700 1 -- 172.21.15.163:0/2390272995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3910159c60 msgr2=0x7f391015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:17.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.308+0000 7f3917a0d700 1 --2- 172.21.15.163:0/2390272995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3910159c60 0x7f391015d5d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f39000098e0 tx=0x7f39000091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:17.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.308+0000 7f3917a0d700 1 -- 172.21.15.163:0/2390272995 shutdown_connections 2023-10-09T06:31:17.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.308+0000 7f3917a0d700 1 --2- 172.21.15.163:0/2390272995 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3910159890 0x7f391015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:17.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.308+0000 7f3917a0d700 1 --2- 172.21.15.163:0/2390272995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3910159c60 0x7f391015d5d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:17.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.308+0000 7f3917a0d700 1 -- 172.21.15.163:0/2390272995 >> 172.21.15.163:0/2390272995 conn(0x7f3910068e20 msgr2=0x7f39100d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:17.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.308+0000 7f3917a0d700 1 -- 172.21.15.163:0/2390272995 shutdown_connections 2023-10-09T06:31:17.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.308+0000 7f3917a0d700 1 -- 172.21.15.163:0/2390272995 wait complete. 2023-10-09T06:31:17.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.309+0000 7f3917a0d700 1 Processor -- start 2023-10-09T06:31:17.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.309+0000 7f3917a0d700 1 -- start start 2023-10-09T06:31:17.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.310+0000 7f3917a0d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3910159890 0x7f39100b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:17.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.310+0000 7f3917a0d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3910159c60 0x7f391016d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:17.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.310+0000 7f3917a0d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f391015db10 0x7f3910150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:17.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.310+0000 7f3917a0d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391016e5e0 con 0x7f3910159c60 2023-10-09T06:31:17.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.310+0000 7f3917a0d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391015b600 con 0x7f3910159890 2023-10-09T06:31:17.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.310+0000 7f3917a0d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39100b2850 con 0x7f391015db10 2023-10-09T06:31:17.319 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.310+0000 7f3914fa8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3910159c60 0x7f391016d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:17.319 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.310+0000 7f3914fa8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3910159c60 0x7f391016d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53430/0 (socket says 172.21.15.163:53430) 2023-10-09T06:31:17.319 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.310+0000 7f3914fa8700 1 -- 172.21.15.163:0/2645323609 learned_addr learned my addr 172.21.15.163:0/2645323609 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:17.319 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.310+0000 7f39157a9700 1 --2- 172.21.15.163:0/2645323609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3910159890 0x7f39100b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:17.319 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.310+0000 7f3915faa700 1 --2- 172.21.15.163:0/2645323609 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f391015db10 0x7f3910150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:17.319 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.311+0000 7f3914fa8700 1 -- 172.21.15.163:0/2645323609 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f391015db10 msgr2=0x7f3910150d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:17.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.311+0000 7f3914fa8700 1 --2- 172.21.15.163:0/2645323609 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f391015db10 0x7f3910150d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:17.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.311+0000 7f3914fa8700 1 -- 172.21.15.163:0/2645323609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3910159890 msgr2=0x7f39100b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:17.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.311+0000 7f3914fa8700 1 --2- 172.21.15.163:0/2645323609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3910159890 0x7f39100b19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:17.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.311+0000 7f3914fa8700 1 -- 172.21.15.163:0/2645323609 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390c0094f0 con 0x7f3910159c60 2023-10-09T06:31:17.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.311+0000 7f39157a9700 1 --2- 172.21.15.163:0/2645323609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3910159890 0x7f39100b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:17.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.311+0000 7f3914fa8700 1 --2- 172.21.15.163:0/2645323609 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3910159c60 0x7f391016d930 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3900013000 tx=0x7f3900017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:17.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.311+0000 7f39057fa700 1 -- 172.21.15.163:0/2645323609 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390000bb20 con 0x7f3910159c60 2023-10-09T06:31:17.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.311+0000 7f39057fa700 1 -- 172.21.15.163:0/2645323609 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3900011420 con 0x7f3910159c60 2023-10-09T06:31:17.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.311+0000 7f39057fa700 1 -- 172.21.15.163:0/2645323609 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39000117c0 con 0x7f3910159c60 2023-10-09T06:31:17.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.311+0000 7f3917a0d700 1 -- 172.21.15.163:0/2645323609 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3900009420 con 0x7f3910159c60 2023-10-09T06:31:17.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.311+0000 7f3917a0d700 1 -- 172.21.15.163:0/2645323609 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39100b2290 con 0x7f3910159c60 2023-10-09T06:31:17.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.312+0000 7f3917a0d700 1 -- 172.21.15.163:0/2645323609 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f391015b790 con 0x7f3910159c60 2023-10-09T06:31:17.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.312+0000 7f39057fa700 1 -- 172.21.15.163:0/2645323609 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3900011b70 con 0x7f3910159c60 2023-10-09T06:31:17.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.313+0000 7f39057fa700 1 --2- 172.21.15.163:0/2645323609 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f38f4066f30 0x7f38f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:17.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.313+0000 7f39057fa700 1 -- 172.21.15.163:0/2645323609 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f3900085bc0 con 0x7f3910159c60 2023-10-09T06:31:17.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.313+0000 7f39157a9700 1 --2- 172.21.15.163:0/2645323609 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f38f4066f30 0x7f38f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:17.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.313+0000 7f39157a9700 1 --2- 172.21.15.163:0/2645323609 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f38f4066f30 0x7f38f40693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f390c0094c0 tx=0x7f390c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:17.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.317+0000 7f39057fa700 1 -- 172.21.15.163:0/2645323609 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3900052de0 con 0x7f3910159c60 2023-10-09T06:31:17.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:17.541+0000 7f3917a0d700 1 -- 172.21.15.163:0/2645323609 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f391004c3c0 con 0x7f3910159c60 2023-10-09T06:31:18.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.172+0000 7f39057fa700 1 -- 172.21.15.163:0/2645323609 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3900056400 con 0x7f3910159c60 2023-10-09T06:31:18.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.174+0000 7f3917a0d700 1 -- 172.21.15.163:0/2645323609 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f38f4066f30 msgr2=0x7f38f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:18.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.174+0000 7f3917a0d700 1 --2- 172.21.15.163:0/2645323609 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f38f4066f30 0x7f38f40693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f390c0094c0 tx=0x7f390c008040 comp rx=0 tx=0).stop 2023-10-09T06:31:18.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.174+0000 7f3917a0d700 1 -- 172.21.15.163:0/2645323609 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3910159c60 msgr2=0x7f391016d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:18.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.174+0000 7f3917a0d700 1 --2- 172.21.15.163:0/2645323609 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3910159c60 0x7f391016d930 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3900013000 tx=0x7f3900017000 comp rx=0 tx=0).stop 2023-10-09T06:31:18.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.174+0000 7f3917a0d700 1 -- 172.21.15.163:0/2645323609 shutdown_connections 2023-10-09T06:31:18.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.174+0000 7f3917a0d700 1 --2- 172.21.15.163:0/2645323609 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f38f4066f30 0x7f38f40693e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:18.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.174+0000 7f3917a0d700 1 --2- 172.21.15.163:0/2645323609 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f391015db10 0x7f3910150d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:18.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.174+0000 7f3917a0d700 1 --2- 172.21.15.163:0/2645323609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3910159890 0x7f39100b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:18.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.174+0000 7f3917a0d700 1 --2- 172.21.15.163:0/2645323609 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3910159c60 0x7f391016d930 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:18.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.174+0000 7f3917a0d700 1 -- 172.21.15.163:0/2645323609 >> 172.21.15.163:0/2645323609 conn(0x7f3910068e20 msgr2=0x7f3910152f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:18.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.174+0000 7f3917a0d700 1 -- 172.21.15.163:0/2645323609 shutdown_connections 2023-10-09T06:31:18.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.174+0000 7f3917a0d700 1 -- 172.21.15.163:0/2645323609 wait complete. 2023-10-09T06:31:18.188 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-09T06:31:18.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.413+0000 7f312e355700 1 Processor -- start 2023-10-09T06:31:18.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.413+0000 7f312e355700 1 -- start start 2023-10-09T06:31:18.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.414+0000 7f312e355700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31281563b0 0x7f3128156780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:18.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.414+0000 7f312e355700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f312815a9b0 con 0x7f3128068890 2023-10-09T06:31:18.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.414+0000 7f312e355700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f312815ac90 con 0x7f312815a440 2023-10-09T06:31:18.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.414+0000 7f312e355700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312815af70 con 0x7f31281563b0 2023-10-09T06:31:18.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.414+0000 7f312c8f2700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f3128068890 0x7f31280c8230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53306/0 (socket says 172.21.15.163:53306) 2023-10-09T06:31:18.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.414+0000 7f312c8f2700 1 -- 172.21.15.163:0/2449034498 learned_addr learned my addr 172.21.15.163:0/2449034498 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:18.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.414+0000 7f3127fff700 1 --2- 172.21.15.163:0/2449034498 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31281563b0 0x7f3128156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:18.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.414+0000 7f3126ffd700 1 -- 172.21.15.163:0/2449034498 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1667715643 0 0) 0x7f312815a9b0 con 0x7f3128068890 2023-10-09T06:31:18.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.414+0000 7f3126ffd700 1 -- 172.21.15.163:0/2449034498 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3110003570 con 0x7f3128068890 2023-10-09T06:31:18.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.415+0000 7f3126ffd700 1 -- 172.21.15.163:0/2449034498 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2979450297 0 0) 0x7f3110003570 con 0x7f3128068890 2023-10-09T06:31:18.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.415+0000 7f3126ffd700 1 -- 172.21.15.163:0/2449034498 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31281563b0 msgr2=0x7f3128156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:18.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.415+0000 7f3126ffd700 1 --2- 172.21.15.163:0/2449034498 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31281563b0 0x7f3128156780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:18.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.415+0000 7f3126ffd700 1 -- 172.21.15.163:0/2449034498 >> v1:172.21.15.181:6789/0 conn(0x7f312815a440 legacy=0x7f31280acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:18.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.415+0000 7f3126ffd700 1 -- 172.21.15.163:0/2449034498 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312815b250 con 0x7f3128068890 2023-10-09T06:31:18.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.415+0000 7f3126ffd700 1 -- 172.21.15.163:0/2449034498 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f31200017f0 con 0x7f3128068890 2023-10-09T06:31:18.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.415+0000 7f3126ffd700 1 -- 172.21.15.163:0/2449034498 >> v1:172.21.15.163:6789/0 conn(0x7f3128068890 legacy=0x7f31280c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:18.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.415+0000 7f3126ffd700 1 --2- 172.21.15.163:0/2449034498 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31281563b0 0x7f3110003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:18.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.415+0000 7f3126ffd700 1 --2- 172.21.15.163:0/2449034498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3110003fd0 0x7f3110004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:18.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.415+0000 7f3126ffd700 1 --2- 172.21.15.163:0/2449034498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3110004d50 0x7f31100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:18.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.415+0000 7f3126ffd700 1 -- 172.21.15.163:0/2449034498 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312815af70 con 0x7f3110003fd0 2023-10-09T06:31:18.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.415+0000 7f3126ffd700 1 -- 172.21.15.163:0/2449034498 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311000bb40 con 0x7f3110004d50 2023-10-09T06:31:18.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.415+0000 7f3126ffd700 1 -- 172.21.15.163:0/2449034498 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f311000bd00 con 0x7f31281563b0 2023-10-09T06:31:18.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.415+0000 7f31277fe700 1 --2- 172.21.15.163:0/2449034498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3110003fd0 0x7f3110004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:18.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.415+0000 7f3127fff700 1 --2- 172.21.15.163:0/2449034498 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31281563b0 0x7f3110003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:18.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.415+0000 7f312c8f2700 1 --2- 172.21.15.163:0/2449034498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3110004d50 0x7f31100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:18.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.415+0000 7f31277fe700 1 -- 172.21.15.163:0/2449034498 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31281563b0 msgr2=0x7f3110003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:18.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.415+0000 7f31277fe700 1 --2- 172.21.15.163:0/2449034498 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31281563b0 0x7f3110003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:18.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.415+0000 7f31277fe700 1 -- 172.21.15.163:0/2449034498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3110004d50 msgr2=0x7f31100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:18.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.415+0000 7f31277fe700 1 --2- 172.21.15.163:0/2449034498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3110004d50 0x7f31100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:18.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.415+0000 7f31277fe700 1 -- 172.21.15.163:0/2449034498 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3110003570 con 0x7f3110003fd0 2023-10-09T06:31:18.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.415+0000 7f31277fe700 1 --2- 172.21.15.163:0/2449034498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3110003fd0 0x7f3110004810 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f311c0019a0 tx=0x7f311c007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b4fa19704cb141 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:18.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.415+0000 7f3126ffd700 1 -- 172.21.15.163:0/2449034498 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c00c5d0 con 0x7f3110003fd0 2023-10-09T06:31:18.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.415+0000 7f3126ffd700 1 -- 172.21.15.163:0/2449034498 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311c00c750 con 0x7f3110003fd0 2023-10-09T06:31:18.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.415+0000 7f3126ffd700 1 -- 172.21.15.163:0/2449034498 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c00cb10 con 0x7f3110003fd0 2023-10-09T06:31:18.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.416+0000 7f312e355700 1 -- 172.21.15.163:0/2449034498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3110003fd0 msgr2=0x7f3110004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:18.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.416+0000 7f312e355700 1 --2- 172.21.15.163:0/2449034498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3110003fd0 0x7f3110004810 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f311c0019a0 tx=0x7f311c007d10 comp rx=0 tx=0).stop 2023-10-09T06:31:18.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.416+0000 7f312c8f2700 1 -- 172.21.15.163:0/2449034498 reap_dead start 2023-10-09T06:31:18.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.416+0000 7f312e355700 1 -- 172.21.15.163:0/2449034498 shutdown_connections 2023-10-09T06:31:18.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.416+0000 7f312e355700 1 -- 172.21.15.163:0/2449034498 >> 172.21.15.163:0/2449034498 conn(0x7f31280c8f90 msgr2=0x7f31280c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:18.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.416+0000 7f312e355700 1 -- 172.21.15.163:0/2449034498 shutdown_connections 2023-10-09T06:31:18.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.416+0000 7f312e355700 1 -- 172.21.15.163:0/2449034498 wait complete. 2023-10-09T06:31:18.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.417+0000 7f312e355700 1 Processor -- start 2023-10-09T06:31:18.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.417+0000 7f312e355700 1 -- start start 2023-10-09T06:31:18.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.418+0000 7f312e355700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3110003fd0 0x7f312814fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:18.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.418+0000 7f312e355700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31280c0e30 0x7f31280af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:18.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.418+0000 7f312e355700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31280c8230 0x7f31280c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:18.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.418+0000 7f312e355700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31280c8c10 con 0x7f31280c0e30 2023-10-09T06:31:18.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.418+0000 7f312e355700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31281502f0 con 0x7f3110003fd0 2023-10-09T06:31:18.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.418+0000 7f312e355700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31280afb20 con 0x7f31280c8230 2023-10-09T06:31:18.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.418+0000 7f31277fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31280c0e30 0x7f31280af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:18.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.418+0000 7f31277fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31280c0e30 0x7f31280af2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53456/0 (socket says 172.21.15.163:53456) 2023-10-09T06:31:18.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.418+0000 7f31277fe700 1 -- 172.21.15.163:0/1419970939 learned_addr learned my addr 172.21.15.163:0/1419970939 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:18.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.418+0000 7f312c8f2700 1 --2- 172.21.15.163:0/1419970939 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31280c8230 0x7f31280c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:18.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.418+0000 7f3127fff700 1 --2- 172.21.15.163:0/1419970939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3110003fd0 0x7f312814fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:18.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.418+0000 7f31277fe700 1 -- 172.21.15.163:0/1419970939 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31280c8230 msgr2=0x7f31280c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:18.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.418+0000 7f31277fe700 1 --2- 172.21.15.163:0/1419970939 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31280c8230 0x7f31280c86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:18.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.418+0000 7f31277fe700 1 -- 172.21.15.163:0/1419970939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3110003fd0 msgr2=0x7f312814fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:18.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.418+0000 7f31277fe700 1 --2- 172.21.15.163:0/1419970939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3110003fd0 0x7f312814fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:18.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.418+0000 7f31277fe700 1 -- 172.21.15.163:0/1419970939 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31280afe00 con 0x7f31280c0e30 2023-10-09T06:31:18.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.418+0000 7f3127fff700 1 --2- 172.21.15.163:0/1419970939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3110003fd0 0x7f312814fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:18.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.419+0000 7f31277fe700 1 --2- 172.21.15.163:0/1419970939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31280c0e30 0x7f31280af2d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f311c00c3c0 tx=0x7f311c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:18.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.419+0000 7f310ffff700 1 -- 172.21.15.163:0/1419970939 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c007af0 con 0x7f31280c0e30 2023-10-09T06:31:18.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.419+0000 7f310ffff700 1 -- 172.21.15.163:0/1419970939 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311c00c5d0 con 0x7f31280c0e30 2023-10-09T06:31:18.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.419+0000 7f310ffff700 1 -- 172.21.15.163:0/1419970939 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c00cb10 con 0x7f31280c0e30 2023-10-09T06:31:18.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.419+0000 7f312e355700 1 -- 172.21.15.163:0/1419970939 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31280b00e0 con 0x7f31280c0e30 2023-10-09T06:31:18.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.419+0000 7f312e355700 1 -- 172.21.15.163:0/1419970939 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31280b06a0 con 0x7f31280c0e30 2023-10-09T06:31:18.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.420+0000 7f312e355700 1 -- 172.21.15.163:0/1419970939 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31280718b0 con 0x7f31280c0e30 2023-10-09T06:31:18.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.420+0000 7f310ffff700 1 -- 172.21.15.163:0/1419970939 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f311c019030 con 0x7f31280c0e30 2023-10-09T06:31:18.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.421+0000 7f310ffff700 1 --2- 172.21.15.163:0/1419970939 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3114066f30 0x7f31140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:18.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.421+0000 7f310ffff700 1 -- 172.21.15.163:0/1419970939 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f311c085b30 con 0x7f31280c0e30 2023-10-09T06:31:18.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.421+0000 7f3127fff700 1 --2- 172.21.15.163:0/1419970939 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3114066f30 0x7f31140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:18.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.421+0000 7f3127fff700 1 --2- 172.21.15.163:0/1419970939 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3114066f30 0x7f31140693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3118006f00 tx=0x7f3118008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:18.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.425+0000 7f310ffff700 1 -- 172.21.15.163:0/1419970939 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f311c052e10 con 0x7f31280c0e30 2023-10-09T06:31:18.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:18.612+0000 7f312e355700 1 -- 172.21.15.163:0/1419970939 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f312806d6a0 con 0x7f31280c0e30 2023-10-09T06:31:19.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.586+0000 7f310ffff700 1 -- 172.21.15.163:0/1419970939 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v22) v1 ==== 181+0+0 (secure 0 0 0) 0x7f311c0220d0 con 0x7f31280c0e30 2023-10-09T06:31:19.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.589+0000 7f312e355700 1 -- 172.21.15.163:0/1419970939 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3114066f30 msgr2=0x7f31140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:19.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.589+0000 7f312e355700 1 --2- 172.21.15.163:0/1419970939 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3114066f30 0x7f31140693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3118006f00 tx=0x7f3118008040 comp rx=0 tx=0).stop 2023-10-09T06:31:19.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.589+0000 7f312e355700 1 -- 172.21.15.163:0/1419970939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31280c0e30 msgr2=0x7f31280af2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:19.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.589+0000 7f312e355700 1 --2- 172.21.15.163:0/1419970939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31280c0e30 0x7f31280af2d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f311c00c3c0 tx=0x7f311c015000 comp rx=0 tx=0).stop 2023-10-09T06:31:19.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.589+0000 7f312e355700 1 -- 172.21.15.163:0/1419970939 shutdown_connections 2023-10-09T06:31:19.591 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.589+0000 7f312e355700 1 --2- 172.21.15.163:0/1419970939 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3114066f30 0x7f31140693e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:19.591 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.589+0000 7f312e355700 1 --2- 172.21.15.163:0/1419970939 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31280c8230 0x7f31280c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:19.591 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.589+0000 7f312e355700 1 --2- 172.21.15.163:0/1419970939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3110003fd0 0x7f312814fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:19.591 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.589+0000 7f312e355700 1 --2- 172.21.15.163:0/1419970939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31280c0e30 0x7f31280af2d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:19.591 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.589+0000 7f312e355700 1 -- 172.21.15.163:0/1419970939 >> 172.21.15.163:0/1419970939 conn(0x7f31280c8f90 msgr2=0x7f31280ae920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:19.592 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.589+0000 7f312e355700 1 -- 172.21.15.163:0/1419970939 shutdown_connections 2023-10-09T06:31:19.592 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.589+0000 7f312e355700 1 -- 172.21.15.163:0/1419970939 wait complete. 2023-10-09T06:31:19.592 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_metadata' removed 2023-10-09T06:31:19.603 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-09T06:31:19.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.720+0000 7f04dfffb700 1 Processor -- start 2023-10-09T06:31:19.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.721+0000 7f04dfffb700 1 -- start start 2023-10-09T06:31:19.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.721+0000 7f04dfffb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f04d81597b0 0x7f04d80af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:19.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.721+0000 7f04dfffb700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04d8069d80 con 0x7f04d8159b80 2023-10-09T06:31:19.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.721+0000 7f04dfffb700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04d80aa680 con 0x7f04d806a9c0 2023-10-09T06:31:19.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.721+0000 7f04dfffb700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04d8068390 con 0x7f04d81597b0 2023-10-09T06:31:19.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.722+0000 7f04dd596700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f04d8159b80 0x7f04d806a2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:38742/0 (socket says 172.21.15.163:38742) 2023-10-09T06:31:19.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.722+0000 7f04dd596700 1 -- 172.21.15.163:0/4132070807 learned_addr learned my addr 172.21.15.163:0/4132070807 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:19.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.722+0000 7f04ddd97700 1 --2- 172.21.15.163:0/4132070807 >> v2:172.21.15.181:3300/0 conn(0x7f04d81597b0 0x7f04d80af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:19.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.722+0000 7f04dcd95700 1 -- 172.21.15.163:0/4132070807 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1733421572 0 0) 0x7f04d8069d80 con 0x7f04d8159b80 2023-10-09T06:31:19.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.722+0000 7f04dcd95700 1 -- 172.21.15.163:0/4132070807 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04c4003570 con 0x7f04d8159b80 2023-10-09T06:31:19.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.722+0000 7f04dcd95700 1 -- 172.21.15.163:0/4132070807 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 905928164 0 0) 0x7f04c4003570 con 0x7f04d8159b80 2023-10-09T06:31:19.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.722+0000 7f04dcd95700 1 -- 172.21.15.163:0/4132070807 >> v2:172.21.15.181:3300/0 conn(0x7f04d81597b0 msgr2=0x7f04d80af1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:19.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.722+0000 7f04dcd95700 1 --2- 172.21.15.163:0/4132070807 >> v2:172.21.15.181:3300/0 conn(0x7f04d81597b0 0x7f04d80af1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:19.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.722+0000 7f04dcd95700 1 -- 172.21.15.163:0/4132070807 >> v1:172.21.15.181:6789/0 conn(0x7f04d806a9c0 legacy=0x7f04d8163650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:19.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.722+0000 7f04dcd95700 1 -- 172.21.15.163:0/4132070807 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d80aa960 con 0x7f04d8159b80 2023-10-09T06:31:19.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.722+0000 7f04dcd95700 1 -- 172.21.15.163:0/4132070807 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f04c8001470 con 0x7f04d8159b80 2023-10-09T06:31:19.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.722+0000 7f04dcd95700 1 -- 172.21.15.163:0/4132070807 >> v1:172.21.15.163:6789/0 conn(0x7f04d8159b80 legacy=0x7f04d806a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:19.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.722+0000 7f04dcd95700 1 --2- 172.21.15.163:0/4132070807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04c4003a60 0x7f04c4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:19.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.722+0000 7f04dcd95700 1 --2- 172.21.15.163:0/4132070807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04c4004810 0x7f04c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:19.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.722+0000 7f04dcd95700 1 --2- 172.21.15.163:0/4132070807 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04c40051c0 0x7f04c400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:19.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.722+0000 7f04dcd95700 1 -- 172.21.15.163:0/4132070807 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d8068390 con 0x7f04c4004810 2023-10-09T06:31:19.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.722+0000 7f04dcd95700 1 -- 172.21.15.163:0/4132070807 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c400c160 con 0x7f04c4003a60 2023-10-09T06:31:19.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.722+0000 7f04dcd95700 1 -- 172.21.15.163:0/4132070807 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c400c320 con 0x7f04c40051c0 2023-10-09T06:31:19.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.722+0000 7f04dd596700 1 --2- 172.21.15.163:0/4132070807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04c4004810 0x7f04c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:19.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.722+0000 7f04ddd97700 1 --2- 172.21.15.163:0/4132070807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04c4003a60 0x7f04c4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:19.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.722+0000 7f04de598700 1 --2- 172.21.15.163:0/4132070807 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04c40051c0 0x7f04c400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:19.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.722+0000 7f04dd596700 1 -- 172.21.15.163:0/4132070807 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04c40051c0 msgr2=0x7f04c400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:19.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.722+0000 7f04dd596700 1 --2- 172.21.15.163:0/4132070807 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04c40051c0 0x7f04c400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:19.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.722+0000 7f04dd596700 1 -- 172.21.15.163:0/4132070807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04c4003a60 msgr2=0x7f04c4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:19.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.722+0000 7f04dd596700 1 --2- 172.21.15.163:0/4132070807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04c4003a60 0x7f04c4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:19.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.722+0000 7f04dd596700 1 -- 172.21.15.163:0/4132070807 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c4003570 con 0x7f04c4004810 2023-10-09T06:31:19.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.722+0000 7f04de598700 1 --2- 172.21.15.163:0/4132070807 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04c40051c0 0x7f04c400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:19.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.723+0000 7f04de598700 1 -- 172.21.15.163:0/4132070807 reap_dead start 2023-10-09T06:31:19.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.723+0000 7f04dd596700 1 --2- 172.21.15.163:0/4132070807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04c4004810 0x7f04c4004c80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f04c800c390 tx=0x7f04d8069f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b33b6dcf386a27f server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:19.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.723+0000 7f04dcd95700 1 -- 172.21.15.163:0/4132070807 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c800cc60 con 0x7f04c4004810 2023-10-09T06:31:19.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.723+0000 7f04dcd95700 1 -- 172.21.15.163:0/4132070807 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c800cdc0 con 0x7f04c4004810 2023-10-09T06:31:19.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.723+0000 7f04dcd95700 1 -- 172.21.15.163:0/4132070807 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c8005b70 con 0x7f04c4004810 2023-10-09T06:31:19.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.723+0000 7f04dfffb700 1 -- 172.21.15.163:0/4132070807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04c4004810 msgr2=0x7f04c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:19.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.723+0000 7f04dfffb700 1 --2- 172.21.15.163:0/4132070807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04c4004810 0x7f04c4004c80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f04c800c390 tx=0x7f04d8069f10 comp rx=0 tx=0).stop 2023-10-09T06:31:19.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.723+0000 7f04dfffb700 1 -- 172.21.15.163:0/4132070807 shutdown_connections 2023-10-09T06:31:19.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.723+0000 7f04dfffb700 1 --2- 172.21.15.163:0/4132070807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04c4004810 0x7f04c4004c80 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:19.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.723+0000 7f04dfffb700 1 -- 172.21.15.163:0/4132070807 >> 172.21.15.163:0/4132070807 conn(0x7f04d80cd880 msgr2=0x7f04d80ae930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:19.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.723+0000 7f04dfffb700 1 -- 172.21.15.163:0/4132070807 shutdown_connections 2023-10-09T06:31:19.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.723+0000 7f04dfffb700 1 -- 172.21.15.163:0/4132070807 wait complete. 2023-10-09T06:31:19.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.724+0000 7f04dfffb700 1 Processor -- start 2023-10-09T06:31:19.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.725+0000 7f04dfffb700 1 -- start start 2023-10-09T06:31:19.731 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.725+0000 7f04dfffb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04c4004810 0x7f04d815c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:19.731 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.725+0000 7f04dfffb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04d814ac90 0x7f04d80c3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:19.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.725+0000 7f04dfffb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04d80c4490 0x7f04d80c1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:19.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.725+0000 7f04dfffb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d815da70 con 0x7f04c4004810 2023-10-09T06:31:19.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.725+0000 7f04dfffb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d815d5e0 con 0x7f04d814ac90 2023-10-09T06:31:19.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.725+0000 7f04dfffb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d80c4910 con 0x7f04d80c4490 2023-10-09T06:31:19.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.725+0000 7f04ddd97700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04c4004810 0x7f04d815c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:19.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.725+0000 7f04ddd97700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04c4004810 0x7f04d815c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43198/0 (socket says 172.21.15.163:43198) 2023-10-09T06:31:19.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.725+0000 7f04ddd97700 1 -- 172.21.15.163:0/2505444917 learned_addr learned my addr 172.21.15.163:0/2505444917 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:19.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.725+0000 7f04dd596700 1 --2- 172.21.15.163:0/2505444917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04d814ac90 0x7f04d80c3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:19.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.725+0000 7f04de598700 1 --2- 172.21.15.163:0/2505444917 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04d80c4490 0x7f04d80c1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:19.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.726+0000 7f04ddd97700 1 -- 172.21.15.163:0/2505444917 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04d80c4490 msgr2=0x7f04d80c1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:19.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.726+0000 7f04ddd97700 1 --2- 172.21.15.163:0/2505444917 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04d80c4490 0x7f04d80c1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:19.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.726+0000 7f04ddd97700 1 -- 172.21.15.163:0/2505444917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04d814ac90 msgr2=0x7f04d80c3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:19.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.726+0000 7f04ddd97700 1 --2- 172.21.15.163:0/2505444917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04d814ac90 0x7f04d80c3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:19.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.726+0000 7f04ddd97700 1 -- 172.21.15.163:0/2505444917 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d815c9d0 con 0x7f04c4004810 2023-10-09T06:31:19.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.726+0000 7f04ddd97700 1 --2- 172.21.15.163:0/2505444917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04c4004810 0x7f04d815c060 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f04d4011fb0 tx=0x7f04d400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:19.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.726+0000 7f04cdffb700 1 -- 172.21.15.163:0/2505444917 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d400e040 con 0x7f04c4004810 2023-10-09T06:31:19.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.726+0000 7f04cdffb700 1 -- 172.21.15.163:0/2505444917 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d400f040 con 0x7f04c4004810 2023-10-09T06:31:19.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.726+0000 7f04dfffb700 1 -- 172.21.15.163:0/2505444917 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d815ccb0 con 0x7f04c4004810 2023-10-09T06:31:19.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.726+0000 7f04cdffb700 1 -- 172.21.15.163:0/2505444917 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d4015430 con 0x7f04c4004810 2023-10-09T06:31:19.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.726+0000 7f04dfffb700 1 -- 172.21.15.163:0/2505444917 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d815d270 con 0x7f04c4004810 2023-10-09T06:31:19.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.727+0000 7f04dfffb700 1 -- 172.21.15.163:0/2505444917 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04d8069d40 con 0x7f04c4004810 2023-10-09T06:31:19.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.727+0000 7f04cdffb700 1 -- 172.21.15.163:0/2505444917 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f04d40157e0 con 0x7f04c4004810 2023-10-09T06:31:19.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.728+0000 7f04cdffb700 1 --2- 172.21.15.163:0/2505444917 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f04bc066f30 0x7f04bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:19.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.728+0000 7f04cdffb700 1 -- 172.21.15.163:0/2505444917 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f04d4084fe0 con 0x7f04c4004810 2023-10-09T06:31:19.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.728+0000 7f04dd596700 1 --2- 172.21.15.163:0/2505444917 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f04bc066f30 0x7f04bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:19.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.729+0000 7f04dd596700 1 --2- 172.21.15.163:0/2505444917 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f04bc066f30 0x7f04bc0693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f04c8007cd0 tx=0x7f04c8015000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:19.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.732+0000 7f04cdffb700 1 -- 172.21.15.163:0/2505444917 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f04d40523b0 con 0x7f04c4004810 2023-10-09T06:31:19.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:19.920+0000 7f04dfffb700 1 -- 172.21.15.163:0/2505444917 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f04d80718b0 con 0x7f04c4004810 2023-10-09T06:31:20.591 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.591+0000 7f04cdffb700 1 -- 172.21.15.163:0/2505444917 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7f04d40559d0 con 0x7f04c4004810 2023-10-09T06:31:20.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.594+0000 7f04dfffb700 1 -- 172.21.15.163:0/2505444917 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f04bc066f30 msgr2=0x7f04bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:20.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.594+0000 7f04dfffb700 1 --2- 172.21.15.163:0/2505444917 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f04bc066f30 0x7f04bc0693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f04c8007cd0 tx=0x7f04c8015000 comp rx=0 tx=0).stop 2023-10-09T06:31:20.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.594+0000 7f04dfffb700 1 -- 172.21.15.163:0/2505444917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04c4004810 msgr2=0x7f04d815c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:20.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.594+0000 7f04dfffb700 1 --2- 172.21.15.163:0/2505444917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04c4004810 0x7f04d815c060 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f04d4011fb0 tx=0x7f04d400b040 comp rx=0 tx=0).stop 2023-10-09T06:31:20.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.595+0000 7f04dfffb700 1 -- 172.21.15.163:0/2505444917 shutdown_connections 2023-10-09T06:31:20.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.595+0000 7f04dfffb700 1 --2- 172.21.15.163:0/2505444917 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f04bc066f30 0x7f04bc0693e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:20.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.595+0000 7f04dfffb700 1 --2- 172.21.15.163:0/2505444917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04c4004810 0x7f04d815c060 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:20.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.595+0000 7f04dfffb700 1 --2- 172.21.15.163:0/2505444917 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04d80c4490 0x7f04d80c1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:20.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.595+0000 7f04dfffb700 1 --2- 172.21.15.163:0/2505444917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04d814ac90 0x7f04d80c3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:20.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.595+0000 7f04dfffb700 1 -- 172.21.15.163:0/2505444917 >> 172.21.15.163:0/2505444917 conn(0x7f04d80cd880 msgr2=0x7f04d80acac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:20.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.595+0000 7f04dfffb700 1 -- 172.21.15.163:0/2505444917 shutdown_connections 2023-10-09T06:31:20.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.595+0000 7f04dfffb700 1 -- 172.21.15.163:0/2505444917 wait complete. 2023-10-09T06:31:20.597 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_data' removed 2023-10-09T06:31:20.610 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-09T06:31:20.610 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-09T06:31:20.611 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-09T06:31:20.611 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:31:20.611 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-09T06:31:20.611 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-09T06:31:20.612 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-09T06:31:20.612 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:31:20.613 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-09T06:31:20.613 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-09T06:31:20.613 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-09T06:31:20.613 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:31:20.613 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-09T06:31:20.613 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-09T06:31:20.614 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-09T06:31:20.614 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:31:20.620 INFO:tasks.ceph.mds.a:Stopped 2023-10-09T06:31:20.620 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-09T06:31:20.621 INFO:tasks.ceph.mds.c:Stopped 2023-10-09T06:31:20.621 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-09T06:31:20.624 INFO:tasks.ceph.mds.b:Stopped 2023-10-09T06:31:20.624 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-09T06:31:20.625 INFO:tasks.ceph.mds.d:Stopped 2023-10-09T06:31:20.625 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-09T06:31:20.662 INFO:tasks.ceph.mds.a:Started 2023-10-09T06:31:20.662 DEBUG:teuthology.parallel:result is None 2023-10-09T06:31:20.663 INFO:tasks.ceph.mds.c:Started 2023-10-09T06:31:20.663 DEBUG:teuthology.parallel:result is None 2023-10-09T06:31:20.667 INFO:tasks.ceph.mds.b:Started 2023-10-09T06:31:20.667 DEBUG:teuthology.parallel:result is None 2023-10-09T06:31:20.668 INFO:tasks.ceph.mds.d:Started 2023-10-09T06:31:20.668 DEBUG:teuthology.parallel:result is None 2023-10-09T06:31:20.668 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-09T06:31:20.737 INFO:tasks.ceph.mds.a.smithi163.stdout:starting mds.a at 2023-10-09T06:31:20.737 INFO:tasks.ceph.mds.c.smithi163.stdout:starting mds.c at 2023-10-09T06:31:20.737 INFO:tasks.ceph.mds.c.smithi163.stderr:2023-10-09T06:31:20.737+0000 7fd74dbe5700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-09T06:31:20.746 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-09T06:31:20.746 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-09T06:31:20.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.790+0000 7f8baf8a7700 1 Processor -- start 2023-10-09T06:31:20.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.790+0000 7f8baf8a7700 1 -- start start 2023-10-09T06:31:20.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.791+0000 7f8baf8a7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8ba806aa40 0x7f8ba80cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:20.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.791+0000 7f8baf8a7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8ba80c91e0 0x7f8ba8160390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:20.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.791+0000 7f8baf8a7700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ba80cc210 con 0x7f8ba806bac0 2023-10-09T06:31:20.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.791+0000 7f8baf8a7700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ba80cc4f0 con 0x7f8ba80c91e0 2023-10-09T06:31:20.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.791+0000 7f8baf8a7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ba80cc7d0 con 0x7f8ba806aa40 2023-10-09T06:31:20.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.791+0000 7f8bade44700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8ba80c91e0 0x7f8ba8160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:20.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.791+0000 7f8bace42700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f8ba806bac0 0x7f8ba806ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:38748/0 (socket says 172.21.15.163:38748) 2023-10-09T06:31:20.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.791+0000 7f8bace42700 1 -- 172.21.15.163:0/105369378 learned_addr learned my addr 172.21.15.163:0/105369378 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:20.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.791+0000 7f8bad643700 1 --2- 172.21.15.163:0/105369378 >> v2:172.21.15.181:3300/0 conn(0x7f8ba806aa40 0x7f8ba80cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:20.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.791+0000 7f8bade44700 1 -- 172.21.15.163:0/105369378 >> v2:172.21.15.181:3300/0 conn(0x7f8ba806aa40 msgr2=0x7f8ba80cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:20.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.791+0000 7f8bade44700 1 --2- 172.21.15.163:0/105369378 >> v2:172.21.15.181:3300/0 conn(0x7f8ba806aa40 0x7f8ba80cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:20.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.791+0000 7f8bade44700 1 -- 172.21.15.163:0/105369378 >> v1:172.21.15.163:6789/0 conn(0x7f8ba806bac0 legacy=0x7f8ba806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:20.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.791+0000 7f8bade44700 1 -- 172.21.15.163:0/105369378 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba80ccab0 con 0x7f8ba80c91e0 2023-10-09T06:31:20.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.791+0000 7f8b9ffff700 1 -- 172.21.15.163:0/105369378 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3248109428 0 0) 0x7f8ba80cc210 con 0x7f8ba806bac0 2023-10-09T06:31:20.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.791+0000 7f8bade44700 1 --2- 172.21.15.163:0/105369378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ba80c91e0 0x7f8ba8160390 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8ba806e570 tx=0x7f8ba0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef03f0b89b55c623 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:20.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.792+0000 7f8b9ffff700 1 -- 172.21.15.163:0/105369378 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba000e070 con 0x7f8ba80c91e0 2023-10-09T06:31:20.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.792+0000 7f8b9ffff700 1 -- 172.21.15.163:0/105369378 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba000bb60 con 0x7f8ba80c91e0 2023-10-09T06:31:20.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.792+0000 7f8b9ffff700 1 -- 172.21.15.163:0/105369378 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0012430 con 0x7f8ba80c91e0 2023-10-09T06:31:20.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.792+0000 7f8baf8a7700 1 -- 172.21.15.163:0/105369378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ba80c91e0 msgr2=0x7f8ba8160390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:20.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.792+0000 7f8baf8a7700 1 --2- 172.21.15.163:0/105369378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ba80c91e0 0x7f8ba8160390 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8ba806e570 tx=0x7f8ba0009780 comp rx=0 tx=0).stop 2023-10-09T06:31:20.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.792+0000 7f8baf8a7700 1 -- 172.21.15.163:0/105369378 shutdown_connections 2023-10-09T06:31:20.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.792+0000 7f8baf8a7700 1 --2- 172.21.15.163:0/105369378 >> v2:172.21.15.181:3300/0 conn(0x7f8ba806aa40 0x7f8ba80cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:20.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.792+0000 7f8baf8a7700 1 --2- 172.21.15.163:0/105369378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ba80c91e0 0x7f8ba8160390 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:20.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.792+0000 7f8baf8a7700 1 -- 172.21.15.163:0/105369378 >> 172.21.15.163:0/105369378 conn(0x7f8ba80cd670 msgr2=0x7f8ba80cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:20.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.792+0000 7f8baf8a7700 1 -- 172.21.15.163:0/105369378 shutdown_connections 2023-10-09T06:31:20.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.792+0000 7f8baf8a7700 1 -- 172.21.15.163:0/105369378 wait complete. 2023-10-09T06:31:20.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.793+0000 7f8baf8a7700 1 Processor -- start 2023-10-09T06:31:20.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.793+0000 7f8baf8a7700 1 -- start start 2023-10-09T06:31:20.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.794+0000 7f8baf8a7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ba806aa40 0x7f8ba80b0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:20.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.794+0000 7f8baf8a7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ba806bac0 0x7f8ba814ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:20.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.794+0000 7f8baf8a7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ba80c91e0 0x7f8ba814f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:20.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.794+0000 7f8baf8a7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba8164fc0 con 0x7f8ba80c91e0 2023-10-09T06:31:20.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.794+0000 7f8baf8a7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba8165260 con 0x7f8ba806aa40 2023-10-09T06:31:20.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.794+0000 7f8baf8a7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba8165540 con 0x7f8ba806bac0 2023-10-09T06:31:20.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.794+0000 7f8bade44700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ba80c91e0 0x7f8ba814f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:20.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.794+0000 7f8bade44700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ba80c91e0 0x7f8ba814f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43258/0 (socket says 172.21.15.163:43258) 2023-10-09T06:31:20.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.794+0000 7f8bade44700 1 -- 172.21.15.163:0/2271124857 learned_addr learned my addr 172.21.15.163:0/2271124857 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:20.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.794+0000 7f8bace42700 1 --2- 172.21.15.163:0/2271124857 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ba806bac0 0x7f8ba814ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:20.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.794+0000 7f8bad643700 1 --2- 172.21.15.163:0/2271124857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ba806aa40 0x7f8ba80b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:20.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.795+0000 7f8bade44700 1 -- 172.21.15.163:0/2271124857 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ba806bac0 msgr2=0x7f8ba814ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:20.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.795+0000 7f8bade44700 1 --2- 172.21.15.163:0/2271124857 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ba806bac0 0x7f8ba814ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:20.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.795+0000 7f8bade44700 1 -- 172.21.15.163:0/2271124857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ba806aa40 msgr2=0x7f8ba80b0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:20.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.795+0000 7f8bade44700 1 --2- 172.21.15.163:0/2271124857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ba806aa40 0x7f8ba80b0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:20.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.795+0000 7f8bade44700 1 -- 172.21.15.163:0/2271124857 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba0009a20 con 0x7f8ba80c91e0 2023-10-09T06:31:20.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.795+0000 7f8bad643700 1 --2- 172.21.15.163:0/2271124857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ba806aa40 0x7f8ba80b0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:20.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.795+0000 7f8bace42700 1 --2- 172.21.15.163:0/2271124857 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ba806bac0 0x7f8ba814ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:20.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.795+0000 7f8bade44700 1 --2- 172.21.15.163:0/2271124857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ba80c91e0 0x7f8ba814f390 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8ba806c6a0 tx=0x7f8ba0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:20.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.795+0000 7f8b9d7fa700 1 -- 172.21.15.163:0/2271124857 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0012e60 con 0x7f8ba80c91e0 2023-10-09T06:31:20.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.795+0000 7f8b9d7fa700 1 -- 172.21.15.163:0/2271124857 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba001d630 con 0x7f8ba80c91e0 2023-10-09T06:31:20.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.795+0000 7f8baf8a7700 1 -- 172.21.15.163:0/2271124857 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ba8165820 con 0x7f8ba80c91e0 2023-10-09T06:31:20.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.795+0000 7f8b9d7fa700 1 -- 172.21.15.163:0/2271124857 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba001d9f0 con 0x7f8ba80c91e0 2023-10-09T06:31:20.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.795+0000 7f8baf8a7700 1 -- 172.21.15.163:0/2271124857 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ba8166080 con 0x7f8ba80c91e0 2023-10-09T06:31:20.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.796+0000 7f8baf8a7700 1 -- 172.21.15.163:0/2271124857 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ba8165d40 con 0x7f8ba80c91e0 2023-10-09T06:31:20.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.796+0000 7f8b9d7fa700 1 -- 172.21.15.163:0/2271124857 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8ba0024070 con 0x7f8ba80c91e0 2023-10-09T06:31:20.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.797+0000 7f8b9d7fa700 1 --2- 172.21.15.163:0/2271124857 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8b8c066f30 0x7f8b8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:20.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.797+0000 7f8b9d7fa700 1 -- 172.21.15.163:0/2271124857 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f8ba0086ad0 con 0x7f8ba80c91e0 2023-10-09T06:31:20.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.797+0000 7f8bad643700 1 --2- 172.21.15.163:0/2271124857 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8b8c066f30 0x7f8b8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:20.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.797+0000 7f8bad643700 1 --2- 172.21.15.163:0/2271124857 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8b8c066f30 0x7f8b8c0693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4009d70 tx=0x7f8ba4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:20.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.801+0000 7f8b9d7fa700 1 -- 172.21.15.163:0/2271124857 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ba8165d40 con 0x7f8ba80c91e0 2023-10-09T06:31:20.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:20.999+0000 7f8baf8a7700 1 -- 172.21.15.163:0/2271124857 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8ba8165d40 con 0x7f8ba80c91e0 2023-10-09T06:31:21.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.603+0000 7f8b9d7fa700 1 -- 172.21.15.163:0/2271124857 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v25) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8ba8165d40 con 0x7f8ba80c91e0 2023-10-09T06:31:21.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.606+0000 7f8baf8a7700 1 -- 172.21.15.163:0/2271124857 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8b8c066f30 msgr2=0x7f8b8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:21.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.606+0000 7f8baf8a7700 1 --2- 172.21.15.163:0/2271124857 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8b8c066f30 0x7f8b8c0693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4009d70 tx=0x7f8ba4008040 comp rx=0 tx=0).stop 2023-10-09T06:31:21.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.606+0000 7f8baf8a7700 1 -- 172.21.15.163:0/2271124857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ba80c91e0 msgr2=0x7f8ba814f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:21.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.606+0000 7f8baf8a7700 1 --2- 172.21.15.163:0/2271124857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ba80c91e0 0x7f8ba814f390 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8ba806c6a0 tx=0x7f8ba0019000 comp rx=0 tx=0).stop 2023-10-09T06:31:21.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.606+0000 7f8baf8a7700 1 -- 172.21.15.163:0/2271124857 shutdown_connections 2023-10-09T06:31:21.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.606+0000 7f8baf8a7700 1 --2- 172.21.15.163:0/2271124857 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8b8c066f30 0x7f8b8c0693e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:21.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.606+0000 7f8baf8a7700 1 --2- 172.21.15.163:0/2271124857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ba80c91e0 0x7f8ba814f390 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:21.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.606+0000 7f8baf8a7700 1 --2- 172.21.15.163:0/2271124857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ba806aa40 0x7f8ba80b0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:21.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.606+0000 7f8baf8a7700 1 --2- 172.21.15.163:0/2271124857 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ba806bac0 0x7f8ba814ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:21.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.606+0000 7f8baf8a7700 1 -- 172.21.15.163:0/2271124857 >> 172.21.15.163:0/2271124857 conn(0x7f8ba80cd670 msgr2=0x7f8ba814ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:21.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.606+0000 7f8baf8a7700 1 -- 172.21.15.163:0/2271124857 shutdown_connections 2023-10-09T06:31:21.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.606+0000 7f8baf8a7700 1 -- 172.21.15.163:0/2271124857 wait complete. 2023-10-09T06:31:21.608 INFO:teuthology.orchestra.run.smithi163.stderr: removed all blocklist entries 2023-10-09T06:31:21.619 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-09T06:31:21.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.733+0000 7f3fc63d0700 1 Processor -- start 2023-10-09T06:31:21.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.733+0000 7f3fc63d0700 1 -- start start 2023-10-09T06:31:21.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.734+0000 7f3fc63d0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3fc0159890 0x7f3fc015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:21.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.734+0000 7f3fc63d0700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3fc0159c60 0x7f3fc015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:21.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.734+0000 7f3fc63d0700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc015a160 con 0x7f3fc015db10 2023-10-09T06:31:21.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.734+0000 7f3fc63d0700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fc015df50 con 0x7f3fc0159c60 2023-10-09T06:31:21.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.734+0000 7f3fc63d0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fc00d13a0 con 0x7f3fc0159890 2023-10-09T06:31:21.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.734+0000 7f3fbf7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3fc0159c60 0x7f3fc015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:21.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.734+0000 7f3fc496d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f3fc015db10 0x7f3fc0069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:38764/0 (socket says 172.21.15.163:38764) 2023-10-09T06:31:21.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.734+0000 7f3fc496d700 1 -- 172.21.15.163:0/3771261988 learned_addr learned my addr 172.21.15.163:0/3771261988 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:21.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.734+0000 7f3fbf7fe700 1 --2- 172.21.15.163:0/3771261988 >> v2:172.21.15.163:3300/0 conn(0x7f3fc0159c60 0x7f3fc015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43266/0 (socket says 172.21.15.163:43266) 2023-10-09T06:31:21.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.734+0000 7f3fbffff700 1 --2- 172.21.15.163:0/3771261988 >> v2:172.21.15.181:3300/0 conn(0x7f3fc0159890 0x7f3fc015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:21.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.734+0000 7f3fbf7fe700 1 -- 172.21.15.163:0/3771261988 >> v2:172.21.15.181:3300/0 conn(0x7f3fc0159890 msgr2=0x7f3fc015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:21.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.734+0000 7f3fbf7fe700 1 --2- 172.21.15.163:0/3771261988 >> v2:172.21.15.181:3300/0 conn(0x7f3fc0159890 0x7f3fc015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:21.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.734+0000 7f3fbf7fe700 1 -- 172.21.15.163:0/3771261988 >> v1:172.21.15.163:6789/0 conn(0x7f3fc015db10 legacy=0x7f3fc0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:21.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.734+0000 7f3fbf7fe700 1 -- 172.21.15.163:0/3771261988 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc015e140 con 0x7f3fc0159c60 2023-10-09T06:31:21.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.734+0000 7f3fbeffd700 1 -- 172.21.15.163:0/3771261988 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1127680462 0 0) 0x7f3fc015a160 con 0x7f3fc015db10 2023-10-09T06:31:21.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.734+0000 7f3fbf7fe700 1 --2- 172.21.15.163:0/3771261988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fc0159c60 0x7f3fc015d5d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00d1090 tx=0x7f3fb40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08941b3444fa141 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:21.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.734+0000 7f3fbeffd700 1 -- 172.21.15.163:0/3771261988 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4011630 con 0x7f3fc0159c60 2023-10-09T06:31:21.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.734+0000 7f3fbeffd700 1 -- 172.21.15.163:0/3771261988 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb4011790 con 0x7f3fc0159c60 2023-10-09T06:31:21.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.734+0000 7f3fbeffd700 1 -- 172.21.15.163:0/3771261988 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4011b30 con 0x7f3fc0159c60 2023-10-09T06:31:21.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.735+0000 7f3fc63d0700 1 -- 172.21.15.163:0/3771261988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fc0159c60 msgr2=0x7f3fc015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:21.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.735+0000 7f3fc63d0700 1 --2- 172.21.15.163:0/3771261988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fc0159c60 0x7f3fc015d5d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00d1090 tx=0x7f3fb40091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:21.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.735+0000 7f3fc63d0700 1 -- 172.21.15.163:0/3771261988 shutdown_connections 2023-10-09T06:31:21.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.735+0000 7f3fc63d0700 1 --2- 172.21.15.163:0/3771261988 >> v2:172.21.15.181:3300/0 conn(0x7f3fc0159890 0x7f3fc015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:21.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.735+0000 7f3fc63d0700 1 --2- 172.21.15.163:0/3771261988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fc0159c60 0x7f3fc015d5d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:21.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.735+0000 7f3fc63d0700 1 -- 172.21.15.163:0/3771261988 >> 172.21.15.163:0/3771261988 conn(0x7f3fc0068e20 msgr2=0x7f3fc00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:21.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.735+0000 7f3fc63d0700 1 -- 172.21.15.163:0/3771261988 shutdown_connections 2023-10-09T06:31:21.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.735+0000 7f3fc63d0700 1 -- 172.21.15.163:0/3771261988 wait complete. 2023-10-09T06:31:21.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.736+0000 7f3fc63d0700 1 Processor -- start 2023-10-09T06:31:21.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.736+0000 7f3fc63d0700 1 -- start start 2023-10-09T06:31:21.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.737+0000 7f3fc63d0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fc0159890 0x7f3fc00b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:21.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.737+0000 7f3fc63d0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fc0159c60 0x7f3fc01523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:21.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.737+0000 7f3fc63d0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fc015db10 0x7f3fc0069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:21.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.737+0000 7f3fc63d0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc00ce800 con 0x7f3fc0159c60 2023-10-09T06:31:21.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.737+0000 7f3fc63d0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc015bd60 con 0x7f3fc015db10 2023-10-09T06:31:21.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.737+0000 7f3fc63d0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc015c040 con 0x7f3fc0159890 2023-10-09T06:31:21.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.737+0000 7f3fbf7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fc0159c60 0x7f3fc01523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:21.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.737+0000 7f3fbf7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fc0159c60 0x7f3fc01523a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43282/0 (socket says 172.21.15.163:43282) 2023-10-09T06:31:21.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.737+0000 7f3fbf7fe700 1 -- 172.21.15.163:0/1030317466 learned_addr learned my addr 172.21.15.163:0/1030317466 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:21.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.737+0000 7f3fbffff700 1 --2- 172.21.15.163:0/1030317466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fc0159890 0x7f3fc00b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:21.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.737+0000 7f3fc496d700 1 --2- 172.21.15.163:0/1030317466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fc015db10 0x7f3fc0069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:21.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.737+0000 7f3fbf7fe700 1 -- 172.21.15.163:0/1030317466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fc0159890 msgr2=0x7f3fc00b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:21.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.737+0000 7f3fbf7fe700 1 --2- 172.21.15.163:0/1030317466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fc0159890 0x7f3fc00b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:21.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.737+0000 7f3fbf7fe700 1 -- 172.21.15.163:0/1030317466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fc015db10 msgr2=0x7f3fc0069530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:21.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.737+0000 7f3fbf7fe700 1 --2- 172.21.15.163:0/1030317466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fc015db10 0x7f3fc0069530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:21.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.737+0000 7f3fbf7fe700 1 -- 172.21.15.163:0/1030317466 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb4009420 con 0x7f3fc0159c60 2023-10-09T06:31:21.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.737+0000 7f3fbf7fe700 1 --2- 172.21.15.163:0/1030317466 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fc0159c60 0x7f3fc01523a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0070e20 tx=0x7f3fb4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:21.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.738+0000 7f3fa7fff700 1 -- 172.21.15.163:0/1030317466 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4011b10 con 0x7f3fc0159c60 2023-10-09T06:31:21.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.738+0000 7f3fa7fff700 1 -- 172.21.15.163:0/1030317466 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb40253f0 con 0x7f3fc0159c60 2023-10-09T06:31:21.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.738+0000 7f3fc63d0700 1 -- 172.21.15.163:0/1030317466 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc015c320 con 0x7f3fc0159c60 2023-10-09T06:31:21.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.738+0000 7f3fa7fff700 1 -- 172.21.15.163:0/1030317466 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4025790 con 0x7f3fc0159c60 2023-10-09T06:31:21.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.738+0000 7f3fc63d0700 1 -- 172.21.15.163:0/1030317466 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc015c8e0 con 0x7f3fc0159c60 2023-10-09T06:31:21.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.739+0000 7f3fc63d0700 1 -- 172.21.15.163:0/1030317466 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc015cad0 con 0x7f3fc0159c60 2023-10-09T06:31:21.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.739+0000 7f3fa7fff700 1 -- 172.21.15.163:0/1030317466 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3fb4025b80 con 0x7f3fc0159c60 2023-10-09T06:31:21.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.739+0000 7f3fa7fff700 1 --2- 172.21.15.163:0/1030317466 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3fac066f30 0x7f3fac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:21.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.739+0000 7f3fa7fff700 1 -- 172.21.15.163:0/1030317466 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f3fb4085ac0 con 0x7f3fc0159c60 2023-10-09T06:31:21.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.739+0000 7f3fbffff700 1 --2- 172.21.15.163:0/1030317466 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3fac066f30 0x7f3fac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:21.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.740+0000 7f3fbffff700 1 --2- 172.21.15.163:0/1030317466 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3fac066f30 0x7f3fac0693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009d40 tx=0x7f3fb0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:21.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:21.744+0000 7f3fa7fff700 1 -- 172.21.15.163:0/1030317466 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3fb4053080 con 0x7f3fc0159c60 2023-10-09T06:31:22.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.000+0000 7f3fc63d0700 1 -- 172.21.15.163:0/1030317466 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3fc004c3c0 con 0x7f3fc0159c60 2023-10-09T06:31:22.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.000+0000 7f3fa7fff700 1 -- 172.21.15.163:0/1030317466 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f3fb4015090 con 0x7f3fc0159c60 2023-10-09T06:31:22.001 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:22.001 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:31:22.001 INFO:teuthology.orchestra.run.smithi163.stdout: "auth_dump": [ 2023-10-09T06:31:22.001 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:22.001 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.a", 2023-10-09T06:31:22.002 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNltXtwDxAAlnrWRsfnZDLEmsJVowh3Mg==", 2023-10-09T06:31:22.002 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:22.002 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:31:22.002 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:31:22.002 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:31:22.002 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:22.002 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:22.002 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:22.002 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:22.002 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.b", 2023-10-09T06:31:22.002 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlTdGhGhAAXuldy0HrFd+44sSu/EV3qA==", 2023-10-09T06:31:22.003 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:22.003 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:31:22.003 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:31:22.003 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:31:22.003 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:22.003 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:22.003 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:22.003 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:22.003 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.c", 2023-10-09T06:31:22.003 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlGis7FhAAYG76TiOFX8QkTwjNX2+Ncw==", 2023-10-09T06:31:22.003 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:22.004 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:31:22.004 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:31:22.004 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:31:22.004 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:22.004 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:22.004 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:22.004 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:22.004 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.d", 2023-10-09T06:31:22.004 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlKhRuIRAAfhG5Gpc7klXJloYXepOhsg==", 2023-10-09T06:31:22.004 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:22.004 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:31:22.004 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:31:22.005 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:31:22.005 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:22.005 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:22.005 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:22.005 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:22.005 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.0", 2023-10-09T06:31:22.005 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDcnSNlrQeFOxAA+s6UitrTn5TK1N7kgySoXw==", 2023-10-09T06:31:22.005 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:22.005 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:31:22.005 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:31:22.005 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:22.006 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:22.006 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:22.006 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:22.006 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.1", 2023-10-09T06:31:22.006 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDfnSNlleccORAAnB+lNUFblivdoY44JfELxA==", 2023-10-09T06:31:22.006 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:22.006 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:31:22.006 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:31:22.006 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:22.006 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:22.006 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:22.006 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:22.007 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.2", 2023-10-09T06:31:22.007 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDinSNlFoB4NxAAT0+trRBij5rqgc+F0eWUfQ==", 2023-10-09T06:31:22.007 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:22.007 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:31:22.007 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:31:22.007 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:22.007 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:22.007 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:22.007 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:22.007 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.3", 2023-10-09T06:31:22.007 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDlnSNlRRQ6NhAABoBtrFjSvQujolq0Pxrd1Q==", 2023-10-09T06:31:22.007 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:22.008 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:31:22.008 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:31:22.008 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:22.008 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:22.008 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:22.008 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:22.008 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.4", 2023-10-09T06:31:22.008 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDrnSNl308uOxAA5OJyPLGQmVSJqha5d+nKKQ==", 2023-10-09T06:31:22.008 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:22.008 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:31:22.008 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:31:22.008 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:22.009 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:22.009 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:22.009 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:22.009 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.5", 2023-10-09T06:31:22.009 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDunSNlHnK+OBAAU7l4dCJ82SbGSzBEm8Axzw==", 2023-10-09T06:31:22.009 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:22.009 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:31:22.009 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:31:22.009 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:22.009 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:22.009 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:22.009 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:22.009 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.6", 2023-10-09T06:31:22.010 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDxnSNlwUSLOBAAN5N+F5ho+Co8pBh4Rz+lXQ==", 2023-10-09T06:31:22.010 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:22.010 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:31:22.010 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:31:22.010 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:22.010 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:22.010 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:22.010 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:22.010 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.7", 2023-10-09T06:31:22.010 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQD0nSNl2gxGORAAoE8OLGEifcEx/km8rfSV9g==", 2023-10-09T06:31:22.010 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:22.011 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:31:22.011 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:31:22.011 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:22.011 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:22.011 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:22.011 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:22.011 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.0", 2023-10-09T06:31:22.011 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlnxY5JRAApr6czcpMdQYYtXQ9yg2q6A==", 2023-10-09T06:31:22.011 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:22.011 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:31:22.011 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:31:22.011 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:31:22.011 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:31:22.012 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:22.012 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:22.012 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:22.012 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.1", 2023-10-09T06:31:22.012 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlk6ybLRAAuShqTDAtymGdxN8KBdZDDA==", 2023-10-09T06:31:22.012 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:22.012 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:31:22.012 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:31:22.012 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:31:22.012 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:31:22.012 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:22.013 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:22.013 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:22.013 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.2", 2023-10-09T06:31:22.013 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlYyOmKhAAQmpr+mGfJH4GUa28dtlJwA==", 2023-10-09T06:31:22.013 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:22.013 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:31:22.013 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:31:22.013 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:31:22.013 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:31:22.013 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:22.013 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:22.013 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:22.013 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.3", 2023-10-09T06:31:22.014 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlD2H1MhAA4zvaXY/Gd0kndzi+QSwp7w==", 2023-10-09T06:31:22.014 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:22.014 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:31:22.014 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:31:22.014 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:31:22.014 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:31:22.014 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:22.014 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:22.014 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:22.014 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.admin", 2023-10-09T06:31:22.014 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDYnSNlsHH4LxAA85xeFEP1CUwoblaePYk5zA==", 2023-10-09T06:31:22.014 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:22.015 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:31:22.015 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:31:22.015 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:31:22.015 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:22.015 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:22.015 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:22.015 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:22.015 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.bootstrap-mds", 2023-10-09T06:31:22.015 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQD6nSNl8xSlKhAAtLQ4IQOddqOO0g+i7DSIyg==", 2023-10-09T06:31:22.015 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:22.015 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile bootstrap-mds" 2023-10-09T06:31:22.015 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:22.016 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:22.016 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:22.016 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.bootstrap-mgr", 2023-10-09T06:31:22.016 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQD6nSNlIyClKhAAPTBl/GgKj+0OQ3BvSUVZDg==", 2023-10-09T06:31:22.016 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:22.016 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-09T06:31:22.016 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:22.016 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:22.016 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:22.016 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.bootstrap-osd", 2023-10-09T06:31:22.016 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQD6nSNlyielKhAA7nRqCgkUvIdJNWRMNm3LLQ==", 2023-10-09T06:31:22.016 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:22.016 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile bootstrap-osd" 2023-10-09T06:31:22.017 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:22.017 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:22.017 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:22.017 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.bootstrap-rbd", 2023-10-09T06:31:22.017 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQD6nSNlFi+lKhAAd74vg+ws94yK0se96gKoNw==", 2023-10-09T06:31:22.017 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:22.017 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-09T06:31:22.017 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:22.017 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:22.017 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:22.017 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-09T06:31:22.017 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQD6nSNlgzalKhAAyFDa5/G/LVKsezUnoLuNiA==", 2023-10-09T06:31:22.018 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:22.018 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-09T06:31:22.018 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:22.018 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:22.018 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:22.018 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.bootstrap-rgw", 2023-10-09T06:31:22.018 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQD6nSNlLD6lKhAAvW7yR5Mt3im+CSCOX9XFJg==", 2023-10-09T06:31:22.018 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:22.018 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-09T06:31:22.018 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:22.018 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:22.018 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:22.018 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mgr.x", 2023-10-09T06:31:22.019 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNl2sloDBAA0FoWN9iovCl6783o3/btiA==", 2023-10-09T06:31:22.019 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:22.019 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:31:22.019 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile mgr", 2023-10-09T06:31:22.019 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:22.019 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:22.019 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:22.019 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:22.019 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mgr.y", 2023-10-09T06:31:22.019 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlcFWVCBAAR2dKPg01ciTiNpSX0JIKFg==", 2023-10-09T06:31:22.019 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:22.019 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:31:22.019 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile mgr", 2023-10-09T06:31:22.020 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:22.020 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:22.020 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:22.020 INFO:teuthology.orchestra.run.smithi163.stdout: ] 2023-10-09T06:31:22.020 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:31:22.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.002+0000 7f3fc63d0700 1 -- 172.21.15.163:0/1030317466 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3fac066f30 msgr2=0x7f3fac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:22.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.002+0000 7f3fc63d0700 1 --2- 172.21.15.163:0/1030317466 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3fac066f30 0x7f3fac0693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009d40 tx=0x7f3fb0008040 comp rx=0 tx=0).stop 2023-10-09T06:31:22.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.002+0000 7f3fc63d0700 1 -- 172.21.15.163:0/1030317466 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fc0159c60 msgr2=0x7f3fc01523a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:22.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.002+0000 7f3fc63d0700 1 --2- 172.21.15.163:0/1030317466 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fc0159c60 0x7f3fc01523a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0070e20 tx=0x7f3fb4017000 comp rx=0 tx=0).stop 2023-10-09T06:31:22.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.002+0000 7f3fc63d0700 1 -- 172.21.15.163:0/1030317466 shutdown_connections 2023-10-09T06:31:22.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.002+0000 7f3fc63d0700 1 --2- 172.21.15.163:0/1030317466 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3fac066f30 0x7f3fac0693e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:22.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.002+0000 7f3fc63d0700 1 --2- 172.21.15.163:0/1030317466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fc015db10 0x7f3fc0069530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:22.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.002+0000 7f3fc63d0700 1 --2- 172.21.15.163:0/1030317466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fc0159890 0x7f3fc00b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:22.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.002+0000 7f3fc63d0700 1 --2- 172.21.15.163:0/1030317466 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fc0159c60 0x7f3fc01523a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:22.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.002+0000 7f3fc63d0700 1 -- 172.21.15.163:0/1030317466 >> 172.21.15.163:0/1030317466 conn(0x7f3fc0068e20 msgr2=0x7f3fc01573a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:22.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.003+0000 7f3fc63d0700 1 -- 172.21.15.163:0/1030317466 shutdown_connections 2023-10-09T06:31:22.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.003+0000 7f3fc63d0700 1 -- 172.21.15.163:0/1030317466 wait complete. 2023-10-09T06:31:22.023 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-09T06:31:22.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.137+0000 7f91dd26a700 1 Processor -- start 2023-10-09T06:31:22.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.137+0000 7f91dd26a700 1 -- start start 2023-10-09T06:31:22.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.138+0000 7f91dd26a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f91d81593b0 0x7f91d8159780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:22.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.138+0000 7f91dd26a700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d80cfe20 con 0x7f91d80ceaa0 2023-10-09T06:31:22.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.138+0000 7f91dd26a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d80cf210 con 0x7f91d80b1e50 2023-10-09T06:31:22.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.138+0000 7f91dd26a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d80cf4f0 con 0x7f91d81593b0 2023-10-09T06:31:22.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.138+0000 7f91d77fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f91d80ceaa0 0x7f91d80b1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:38774/0 (socket says 172.21.15.163:38774) 2023-10-09T06:31:22.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.138+0000 7f91d77fe700 1 -- 172.21.15.163:0/1626934213 learned_addr learned my addr 172.21.15.163:0/1626934213 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:22.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.138+0000 7f91d7fff700 1 --2- 172.21.15.163:0/1626934213 >> v2:172.21.15.181:3300/0 conn(0x7f91d81593b0 0x7f91d8159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:22.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.139+0000 7f91d6ffd700 1 -- 172.21.15.163:0/1626934213 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2127779209 0 0) 0x7f91d80cfe20 con 0x7f91d80ceaa0 2023-10-09T06:31:22.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.139+0000 7f91d6ffd700 1 -- 172.21.15.163:0/1626934213 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91c0003570 con 0x7f91d80ceaa0 2023-10-09T06:31:22.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.139+0000 7f91d7fff700 1 -- 172.21.15.163:0/1626934213 >> v1:172.21.15.181:6789/0 conn(0x7f91d80b1e50 legacy=0x7f91d80b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:22.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.139+0000 7f91d7fff700 1 -- 172.21.15.163:0/1626934213 >> v1:172.21.15.163:6789/0 conn(0x7f91d80ceaa0 legacy=0x7f91d80b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:22.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.139+0000 7f91d7fff700 1 -- 172.21.15.163:0/1626934213 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d80cf7d0 con 0x7f91d81593b0 2023-10-09T06:31:22.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.139+0000 7f91d7fff700 1 --2- 172.21.15.163:0/1626934213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91d81593b0 0x7f91d8159780 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f91c80098e0 tx=0x7f91c80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a79377b866e6bdea server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:22.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.139+0000 7f91d6ffd700 1 -- 172.21.15.163:0/1626934213 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c800b640 con 0x7f91d81593b0 2023-10-09T06:31:22.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.139+0000 7f91d6ffd700 1 -- 172.21.15.163:0/1626934213 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c800b7a0 con 0x7f91d81593b0 2023-10-09T06:31:22.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.139+0000 7f91d6ffd700 1 -- 172.21.15.163:0/1626934213 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c800bb40 con 0x7f91d81593b0 2023-10-09T06:31:22.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.140+0000 7f91dd26a700 1 -- 172.21.15.163:0/1626934213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91d81593b0 msgr2=0x7f91d8159780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:22.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.140+0000 7f91dd26a700 1 --2- 172.21.15.163:0/1626934213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91d81593b0 0x7f91d8159780 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f91c80098e0 tx=0x7f91c80091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:22.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.140+0000 7f91dd26a700 1 -- 172.21.15.163:0/1626934213 shutdown_connections 2023-10-09T06:31:22.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.140+0000 7f91dd26a700 1 --2- 172.21.15.163:0/1626934213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91d81593b0 0x7f91d8159780 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:22.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.140+0000 7f91dd26a700 1 -- 172.21.15.163:0/1626934213 >> 172.21.15.163:0/1626934213 conn(0x7f91d80d1130 msgr2=0x7f91d80b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:22.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.140+0000 7f91dd26a700 1 -- 172.21.15.163:0/1626934213 shutdown_connections 2023-10-09T06:31:22.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.140+0000 7f91dd26a700 1 -- 172.21.15.163:0/1626934213 wait complete. 2023-10-09T06:31:22.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.141+0000 7f91dd26a700 1 Processor -- start 2023-10-09T06:31:22.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.141+0000 7f91dd26a700 1 -- start start 2023-10-09T06:31:22.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.142+0000 7f91dd26a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91d80b1e50 0x7f91d815afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:22.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.142+0000 7f91dd26a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91d80ceaa0 0x7f91d814e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:22.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.142+0000 7f91dd26a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91d81593b0 0x7f91d81515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:22.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.142+0000 7f91dd26a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d815b4e0 con 0x7f91d80ceaa0 2023-10-09T06:31:22.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.142+0000 7f91dd26a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d815ab10 con 0x7f91d81593b0 2023-10-09T06:31:22.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.142+0000 7f91dd26a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d8159f00 con 0x7f91d80b1e50 2023-10-09T06:31:22.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.142+0000 7f91d77fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91d80ceaa0 0x7f91d814e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:22.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.142+0000 7f91d77fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91d80ceaa0 0x7f91d814e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43286/0 (socket says 172.21.15.163:43286) 2023-10-09T06:31:22.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.142+0000 7f91d77fe700 1 -- 172.21.15.163:0/1964411881 learned_addr learned my addr 172.21.15.163:0/1964411881 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:22.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.142+0000 7f91d7fff700 1 --2- 172.21.15.163:0/1964411881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91d80b1e50 0x7f91d815afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:22.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.142+0000 7f91dca69700 1 --2- 172.21.15.163:0/1964411881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91d81593b0 0x7f91d81515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:22.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.142+0000 7f91d77fe700 1 -- 172.21.15.163:0/1964411881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91d80b1e50 msgr2=0x7f91d815afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:22.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.142+0000 7f91d77fe700 1 --2- 172.21.15.163:0/1964411881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91d80b1e50 0x7f91d815afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:22.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.142+0000 7f91d77fe700 1 -- 172.21.15.163:0/1964411881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91d81593b0 msgr2=0x7f91d81515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:22.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.142+0000 7f91d77fe700 1 --2- 172.21.15.163:0/1964411881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91d81593b0 0x7f91d81515c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:22.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.142+0000 7f91d77fe700 1 -- 172.21.15.163:0/1964411881 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c8009420 con 0x7f91d80ceaa0 2023-10-09T06:31:22.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.142+0000 7f91d7fff700 1 --2- 172.21.15.163:0/1964411881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91d80b1e50 0x7f91d815afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:22.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.142+0000 7f91d77fe700 1 --2- 172.21.15.163:0/1964411881 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91d80ceaa0 0x7f91d814e160 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00c7c0 tx=0x7f91cc007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:22.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.142+0000 7f91bffff700 1 -- 172.21.15.163:0/1964411881 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc005b30 con 0x7f91d80ceaa0 2023-10-09T06:31:22.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.142+0000 7f91bffff700 1 -- 172.21.15.163:0/1964411881 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91cc005c90 con 0x7f91d80ceaa0 2023-10-09T06:31:22.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.142+0000 7f91dd26a700 1 -- 172.21.15.163:0/1964411881 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d815a1e0 con 0x7f91d80ceaa0 2023-10-09T06:31:22.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.142+0000 7f91bffff700 1 -- 172.21.15.163:0/1964411881 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc007340 con 0x7f91d80ceaa0 2023-10-09T06:31:22.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.143+0000 7f91dd26a700 1 -- 172.21.15.163:0/1964411881 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d815a7a0 con 0x7f91d80ceaa0 2023-10-09T06:31:22.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.143+0000 7f91dd26a700 1 -- 172.21.15.163:0/1964411881 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d80b3030 con 0x7f91d80ceaa0 2023-10-09T06:31:22.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.144+0000 7f91bffff700 1 -- 172.21.15.163:0/1964411881 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f91cc01b400 con 0x7f91d80ceaa0 2023-10-09T06:31:22.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.144+0000 7f91bffff700 1 --2- 172.21.15.163:0/1964411881 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f91c4066f30 0x7f91c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:22.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.144+0000 7f91bffff700 1 -- 172.21.15.163:0/1964411881 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f91cc083c70 con 0x7f91d80ceaa0 2023-10-09T06:31:22.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.144+0000 7f91d7fff700 1 --2- 172.21.15.163:0/1964411881 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f91c4066f30 0x7f91c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:22.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.145+0000 7f91d7fff700 1 --2- 172.21.15.163:0/1964411881 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f91c4066f30 0x7f91c40693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f91c8003080 tx=0x7f91c8017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:22.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.148+0000 7f91bffff700 1 -- 172.21.15.163:0/1964411881 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f91cc0512b0 con 0x7f91d80ceaa0 2023-10-09T06:31:22.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.405+0000 7f91dd26a700 1 -- 172.21.15.163:0/1964411881 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f91d804c3c0 con 0x7f91d80ceaa0 2023-10-09T06:31:22.409 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.409+0000 7f91bffff700 1 -- 172.21.15.163:0/1964411881 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f91cc0548d0 con 0x7f91d80ceaa0 2023-10-09T06:31:22.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.411+0000 7f91dd26a700 1 -- 172.21.15.163:0/1964411881 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f91c4066f30 msgr2=0x7f91c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:22.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.411+0000 7f91dd26a700 1 --2- 172.21.15.163:0/1964411881 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f91c4066f30 0x7f91c40693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f91c8003080 tx=0x7f91c8017000 comp rx=0 tx=0).stop 2023-10-09T06:31:22.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.411+0000 7f91dd26a700 1 -- 172.21.15.163:0/1964411881 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91d80ceaa0 msgr2=0x7f91d814e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:22.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.411+0000 7f91dd26a700 1 --2- 172.21.15.163:0/1964411881 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91d80ceaa0 0x7f91d814e160 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00c7c0 tx=0x7f91cc007e00 comp rx=0 tx=0).stop 2023-10-09T06:31:22.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.411+0000 7f91dd26a700 1 -- 172.21.15.163:0/1964411881 shutdown_connections 2023-10-09T06:31:22.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.411+0000 7f91dd26a700 1 --2- 172.21.15.163:0/1964411881 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f91c4066f30 0x7f91c40693e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:22.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.411+0000 7f91dd26a700 1 --2- 172.21.15.163:0/1964411881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91d81593b0 0x7f91d81515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:22.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.411+0000 7f91dd26a700 1 --2- 172.21.15.163:0/1964411881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91d80b1e50 0x7f91d815afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:22.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.411+0000 7f91dd26a700 1 --2- 172.21.15.163:0/1964411881 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91d80ceaa0 0x7f91d814e160 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:22.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.411+0000 7f91dd26a700 1 -- 172.21.15.163:0/1964411881 >> 172.21.15.163:0/1964411881 conn(0x7f91d80d1130 msgr2=0x7f91d815bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:22.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.411+0000 7f91dd26a700 1 -- 172.21.15.163:0/1964411881 shutdown_connections 2023-10-09T06:31:22.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.411+0000 7f91dd26a700 1 -- 172.21.15.163:0/1964411881 wait complete. 2023-10-09T06:31:22.424 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-09T06:31:22.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.538+0000 7f509dc20700 1 Processor -- start 2023-10-09T06:31:22.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.538+0000 7f509dc20700 1 -- start start 2023-10-09T06:31:22.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.539+0000 7f509dc20700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5098159c60 0x7f509815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:22.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.539+0000 7f509dc20700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f509815db10 0x7f5098069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:22.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.539+0000 7f509dc20700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f509815a160 con 0x7f5098159890 2023-10-09T06:31:22.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.539+0000 7f509dc20700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50980d13a0 con 0x7f509815db10 2023-10-09T06:31:22.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.539+0000 7f509dc20700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f509815e140 con 0x7f5098159c60 2023-10-09T06:31:22.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.539+0000 7f5097fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f509815db10 0x7f5098069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:22.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.539+0000 7f5096ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5098159c60 0x7f509815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:22.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.539+0000 7f50977fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f5098159890 0x7f509815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:34778/0 (socket says 172.21.15.163:34778) 2023-10-09T06:31:22.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.539+0000 7f50977fe700 1 -- 172.21.15.163:0/1424730025 learned_addr learned my addr 172.21.15.163:0/1424730025 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:22.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.540+0000 7f50967fc700 1 -- 172.21.15.163:0/1424730025 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2688371021 0 0) 0x7f509815a160 con 0x7f5098159890 2023-10-09T06:31:22.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.540+0000 7f50967fc700 1 -- 172.21.15.163:0/1424730025 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5080003570 con 0x7f5098159890 2023-10-09T06:31:22.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.540+0000 7f5096ffd700 1 -- 172.21.15.163:0/1424730025 >> v2:172.21.15.181:3300/0 conn(0x7f509815db10 msgr2=0x7f5098069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:22.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.540+0000 7f5096ffd700 1 --2- 172.21.15.163:0/1424730025 >> v2:172.21.15.181:3300/0 conn(0x7f509815db10 0x7f5098069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:22.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.540+0000 7f5096ffd700 1 -- 172.21.15.163:0/1424730025 >> v1:172.21.15.181:6789/0 conn(0x7f5098159890 legacy=0x7f509815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:22.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.540+0000 7f5096ffd700 1 -- 172.21.15.163:0/1424730025 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509815a3d0 con 0x7f5098159c60 2023-10-09T06:31:22.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.540+0000 7f5097fff700 1 --2- 172.21.15.163:0/1424730025 >> v2:172.21.15.181:3300/0 conn(0x7f509815db10 0x7f5098069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:31:22.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.540+0000 7f5096ffd700 1 --2- 172.21.15.163:0/1424730025 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5098159c60 0x7f509815d5d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f508c0098e0 tx=0x7f508c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b85f5afd9722d016 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:22.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.540+0000 7f50967fc700 1 -- 172.21.15.163:0/1424730025 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c011630 con 0x7f5098159c60 2023-10-09T06:31:22.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.540+0000 7f50967fc700 1 -- 172.21.15.163:0/1424730025 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508c011790 con 0x7f5098159c60 2023-10-09T06:31:22.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.540+0000 7f50967fc700 1 -- 172.21.15.163:0/1424730025 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c011b30 con 0x7f5098159c60 2023-10-09T06:31:22.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.540+0000 7f509dc20700 1 -- 172.21.15.163:0/1424730025 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5098159c60 msgr2=0x7f509815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:22.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.540+0000 7f509dc20700 1 --2- 172.21.15.163:0/1424730025 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5098159c60 0x7f509815d5d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f508c0098e0 tx=0x7f508c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:22.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.540+0000 7f509dc20700 1 -- 172.21.15.163:0/1424730025 shutdown_connections 2023-10-09T06:31:22.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.540+0000 7f509dc20700 1 --2- 172.21.15.163:0/1424730025 >> v2:172.21.15.181:3300/0 conn(0x7f509815db10 0x7f5098069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:22.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.540+0000 7f509dc20700 1 --2- 172.21.15.163:0/1424730025 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5098159c60 0x7f509815d5d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:22.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.540+0000 7f509dc20700 1 -- 172.21.15.163:0/1424730025 >> 172.21.15.163:0/1424730025 conn(0x7f5098068e20 msgr2=0x7f50980d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:22.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.541+0000 7f509dc20700 1 -- 172.21.15.163:0/1424730025 shutdown_connections 2023-10-09T06:31:22.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.541+0000 7f509dc20700 1 -- 172.21.15.163:0/1424730025 wait complete. 2023-10-09T06:31:22.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.542+0000 7f509dc20700 1 Processor -- start 2023-10-09T06:31:22.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.542+0000 7f509dc20700 1 -- start start 2023-10-09T06:31:22.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.542+0000 7f509dc20700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5098159890 0x7f5098169920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:22.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.542+0000 7f509dc20700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5098159c60 0x7f509816c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:22.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.542+0000 7f509dc20700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f509815db10 0x7f509815c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:22.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.542+0000 7f509dc20700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509816cc80 con 0x7f5098159c60 2023-10-09T06:31:22.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.542+0000 7f509dc20700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50980c8770 con 0x7f5098159890 2023-10-09T06:31:22.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.542+0000 7f509dc20700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50980b46d0 con 0x7f509815db10 2023-10-09T06:31:22.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.543+0000 7f5096ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5098159c60 0x7f509816c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:22.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.543+0000 7f5096ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5098159c60 0x7f509816c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43288/0 (socket says 172.21.15.163:43288) 2023-10-09T06:31:22.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.543+0000 7f50977fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5098159890 0x7f5098169920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:22.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.543+0000 7f5096ffd700 1 -- 172.21.15.163:0/578541120 learned_addr learned my addr 172.21.15.163:0/578541120 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:22.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.543+0000 7f5097fff700 1 --2- 172.21.15.163:0/578541120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f509815db10 0x7f509815c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:22.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.543+0000 7f5096ffd700 1 -- 172.21.15.163:0/578541120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f509815db10 msgr2=0x7f509815c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:22.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.543+0000 7f5096ffd700 1 --2- 172.21.15.163:0/578541120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f509815db10 0x7f509815c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:22.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.543+0000 7f5096ffd700 1 -- 172.21.15.163:0/578541120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5098159890 msgr2=0x7f5098169920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:22.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.543+0000 7f5096ffd700 1 --2- 172.21.15.163:0/578541120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5098159890 0x7f5098169920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:22.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.543+0000 7f5096ffd700 1 -- 172.21.15.163:0/578541120 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5090009420 con 0x7f5098159c60 2023-10-09T06:31:22.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.543+0000 7f5097fff700 1 --2- 172.21.15.163:0/578541120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f509815db10 0x7f509815c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:31:22.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.543+0000 7f5096ffd700 1 --2- 172.21.15.163:0/578541120 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5098159c60 0x7f509816c740 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f508c013000 tx=0x7f508c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:22.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.543+0000 7f507f7fe700 1 -- 172.21.15.163:0/578541120 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c011b10 con 0x7f5098159c60 2023-10-09T06:31:22.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.543+0000 7f507f7fe700 1 -- 172.21.15.163:0/578541120 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508c0253f0 con 0x7f5098159c60 2023-10-09T06:31:22.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.543+0000 7f509dc20700 1 -- 172.21.15.163:0/578541120 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f508c009420 con 0x7f5098159c60 2023-10-09T06:31:22.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.543+0000 7f507f7fe700 1 -- 172.21.15.163:0/578541120 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c0257b0 con 0x7f5098159c60 2023-10-09T06:31:22.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.543+0000 7f509dc20700 1 -- 172.21.15.163:0/578541120 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50980b4c90 con 0x7f5098159c60 2023-10-09T06:31:22.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.544+0000 7f509dc20700 1 -- 172.21.15.163:0/578541120 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50980b4eb0 con 0x7f5098159c60 2023-10-09T06:31:22.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.545+0000 7f507f7fe700 1 -- 172.21.15.163:0/578541120 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f508c025ba0 con 0x7f5098159c60 2023-10-09T06:31:22.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.545+0000 7f507f7fe700 1 --2- 172.21.15.163:0/578541120 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5084066f30 0x7f50840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:22.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.545+0000 7f507f7fe700 1 -- 172.21.15.163:0/578541120 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f508c085920 con 0x7f5098159c60 2023-10-09T06:31:22.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.545+0000 7f50977fe700 1 --2- 172.21.15.163:0/578541120 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5084066f30 0x7f50840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:22.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.546+0000 7f50977fe700 1 --2- 172.21.15.163:0/578541120 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5084066f30 0x7f50840693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5088005c50 tx=0x7f5088016040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:22.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.549+0000 7f507f7fe700 1 -- 172.21.15.163:0/578541120 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f508c052fa0 con 0x7f5098159c60 2023-10-09T06:31:22.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.810+0000 7f509dc20700 1 -- 172.21.15.163:0/578541120 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f509804c3c0 con 0x7f5098159c60 2023-10-09T06:31:22.815 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.814+0000 7f507f7fe700 1 -- 172.21.15.163:0/578541120 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f508c0565c0 con 0x7f5098159c60 2023-10-09T06:31:22.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.816+0000 7f509dc20700 1 -- 172.21.15.163:0/578541120 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5084066f30 msgr2=0x7f50840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:22.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.816+0000 7f509dc20700 1 --2- 172.21.15.163:0/578541120 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5084066f30 0x7f50840693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5088005c50 tx=0x7f5088016040 comp rx=0 tx=0).stop 2023-10-09T06:31:22.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.816+0000 7f509dc20700 1 -- 172.21.15.163:0/578541120 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5098159c60 msgr2=0x7f509816c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:22.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.816+0000 7f509dc20700 1 --2- 172.21.15.163:0/578541120 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5098159c60 0x7f509816c740 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f508c013000 tx=0x7f508c017000 comp rx=0 tx=0).stop 2023-10-09T06:31:22.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.816+0000 7f509dc20700 1 -- 172.21.15.163:0/578541120 shutdown_connections 2023-10-09T06:31:22.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.816+0000 7f509dc20700 1 --2- 172.21.15.163:0/578541120 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5084066f30 0x7f50840693e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:22.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.816+0000 7f509dc20700 1 --2- 172.21.15.163:0/578541120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f509815db10 0x7f509815c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:22.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.816+0000 7f509dc20700 1 --2- 172.21.15.163:0/578541120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5098159890 0x7f5098169920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:22.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.816+0000 7f509dc20700 1 --2- 172.21.15.163:0/578541120 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5098159c60 0x7f509816c740 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:22.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.816+0000 7f509dc20700 1 -- 172.21.15.163:0/578541120 >> 172.21.15.163:0/578541120 conn(0x7f5098068e20 msgr2=0x7f5098153160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:22.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.816+0000 7f509dc20700 1 -- 172.21.15.163:0/578541120 shutdown_connections 2023-10-09T06:31:22.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.816+0000 7f509dc20700 1 -- 172.21.15.163:0/578541120 wait complete. 2023-10-09T06:31:22.829 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-09T06:31:22.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.944+0000 7f023b8ae700 1 Processor -- start 2023-10-09T06:31:22.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.944+0000 7f023b8ae700 1 -- start start 2023-10-09T06:31:22.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.945+0000 7f023b8ae700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0234069560 0x7f02340699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:22.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.945+0000 7f023b8ae700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0234069ee0 con 0x7f0234159c60 2023-10-09T06:31:22.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.945+0000 7f023b8ae700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f023415a160 con 0x7f0234159890 2023-10-09T06:31:22.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.945+0000 7f023b8ae700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02340d13a0 con 0x7f0234069560 2023-10-09T06:31:22.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.945+0000 7f0239e4b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0234069560 0x7f02340699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:22.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.945+0000 7f0238e49700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f0234159c60 0x7f023415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:38782/0 (socket says 172.21.15.163:38782) 2023-10-09T06:31:22.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.945+0000 7f0239e4b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0234069560 0x7f02340699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43298/0 (socket says 172.21.15.163:43298) 2023-10-09T06:31:22.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.945+0000 7f0238e49700 1 -- 172.21.15.163:0/2859442775 learned_addr learned my addr 172.21.15.163:0/2859442775 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:22.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.946+0000 7f0239e4b700 1 -- 172.21.15.163:0/2859442775 >> v1:172.21.15.181:6789/0 conn(0x7f0234159890 legacy=0x7f023415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:22.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.946+0000 7f0239e4b700 1 -- 172.21.15.163:0/2859442775 >> v1:172.21.15.163:6789/0 conn(0x7f0234159c60 legacy=0x7f023415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:22.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.946+0000 7f0239e4b700 1 -- 172.21.15.163:0/2859442775 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f023415e140 con 0x7f0234069560 2023-10-09T06:31:22.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.946+0000 7f0239e4b700 1 --2- 172.21.15.163:0/2859442775 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0234069560 0x7f02340699a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f022c0098e0 tx=0x7f022c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c122e140ad5c74be server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:22.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.946+0000 7f022bfff700 1 -- 172.21.15.163:0/2859442775 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022c009e80 con 0x7f0234069560 2023-10-09T06:31:22.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.946+0000 7f022bfff700 1 -- 172.21.15.163:0/2859442775 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f022c00b860 con 0x7f0234069560 2023-10-09T06:31:22.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.946+0000 7f022bfff700 1 -- 172.21.15.163:0/2859442775 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022c00bbe0 con 0x7f0234069560 2023-10-09T06:31:22.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.946+0000 7f023b8ae700 1 -- 172.21.15.163:0/2859442775 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0234069560 msgr2=0x7f02340699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:22.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.946+0000 7f023b8ae700 1 --2- 172.21.15.163:0/2859442775 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0234069560 0x7f02340699a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f022c0098e0 tx=0x7f022c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:22.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.946+0000 7f023b8ae700 1 -- 172.21.15.163:0/2859442775 shutdown_connections 2023-10-09T06:31:22.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.946+0000 7f023b8ae700 1 --2- 172.21.15.163:0/2859442775 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0234069560 0x7f02340699a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:22.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.946+0000 7f023b8ae700 1 -- 172.21.15.163:0/2859442775 >> 172.21.15.163:0/2859442775 conn(0x7f0234068e20 msgr2=0x7f02340d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:22.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.947+0000 7f023b8ae700 1 -- 172.21.15.163:0/2859442775 shutdown_connections 2023-10-09T06:31:22.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.947+0000 7f023b8ae700 1 -- 172.21.15.163:0/2859442775 wait complete. 2023-10-09T06:31:22.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.948+0000 7f023b8ae700 1 Processor -- start 2023-10-09T06:31:22.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.948+0000 7f023b8ae700 1 -- start start 2023-10-09T06:31:22.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.949+0000 7f023b8ae700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0234069560 0x7f0234158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:22.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.949+0000 7f023b8ae700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0234159890 0x7f02340c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:22.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.949+0000 7f023b8ae700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0234159c60 0x7f02340ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:22.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.949+0000 7f023b8ae700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02340c8550 con 0x7f0234159890 2023-10-09T06:31:22.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.949+0000 7f023b8ae700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02341588b0 con 0x7f0234159c60 2023-10-09T06:31:22.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.949+0000 7f023b8ae700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02340c7940 con 0x7f0234069560 2023-10-09T06:31:22.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.949+0000 7f0238e49700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0234159890 0x7f02340c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:22.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.949+0000 7f0238e49700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0234159890 0x7f02340c9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43310/0 (socket says 172.21.15.163:43310) 2023-10-09T06:31:22.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.949+0000 7f0238e49700 1 -- 172.21.15.163:0/27697500 learned_addr learned my addr 172.21.15.163:0/27697500 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:22.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.949+0000 7f023964a700 1 --2- 172.21.15.163:0/27697500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0234069560 0x7f0234158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:22.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.949+0000 7f0239e4b700 1 --2- 172.21.15.163:0/27697500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0234159c60 0x7f02340ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:22.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.949+0000 7f0238e49700 1 -- 172.21.15.163:0/27697500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0234069560 msgr2=0x7f0234158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:22.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.949+0000 7f0238e49700 1 --2- 172.21.15.163:0/27697500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0234069560 0x7f0234158370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:22.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.949+0000 7f0238e49700 1 -- 172.21.15.163:0/27697500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0234159c60 msgr2=0x7f02340ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:22.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.949+0000 7f0238e49700 1 --2- 172.21.15.163:0/27697500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0234159c60 0x7f02340ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:22.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.949+0000 7f0238e49700 1 -- 172.21.15.163:0/27697500 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022c009420 con 0x7f0234159890 2023-10-09T06:31:22.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.949+0000 7f023964a700 1 --2- 172.21.15.163:0/27697500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0234069560 0x7f0234158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:22.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.949+0000 7f0238e49700 1 --2- 172.21.15.163:0/27697500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0234159890 0x7f02340c9e80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f022400c560 tx=0x7f0224001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:22.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.949+0000 7f02297fa700 1 -- 172.21.15.163:0/27697500 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0224005b30 con 0x7f0234159890 2023-10-09T06:31:22.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.949+0000 7f02297fa700 1 -- 172.21.15.163:0/27697500 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0224005cb0 con 0x7f0234159890 2023-10-09T06:31:22.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.949+0000 7f02297fa700 1 -- 172.21.15.163:0/27697500 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0224007320 con 0x7f0234159890 2023-10-09T06:31:22.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.949+0000 7f023b8ae700 1 -- 172.21.15.163:0/27697500 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02340c7c20 con 0x7f0234159890 2023-10-09T06:31:22.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.949+0000 7f023b8ae700 1 -- 172.21.15.163:0/27697500 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02340c81e0 con 0x7f0234159890 2023-10-09T06:31:22.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.950+0000 7f023b8ae700 1 -- 172.21.15.163:0/27697500 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02340c86c0 con 0x7f0234159890 2023-10-09T06:31:22.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.951+0000 7f02297fa700 1 -- 172.21.15.163:0/27697500 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f022401b3b0 con 0x7f0234159890 2023-10-09T06:31:22.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.951+0000 7f02297fa700 1 --2- 172.21.15.163:0/27697500 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0218066f30 0x7f02180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:22.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.951+0000 7f02297fa700 1 -- 172.21.15.163:0/27697500 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f0224083700 con 0x7f0234159890 2023-10-09T06:31:22.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.951+0000 7f023964a700 1 --2- 172.21.15.163:0/27697500 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0218066f30 0x7f02180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:22.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.952+0000 7f023964a700 1 --2- 172.21.15.163:0/27697500 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0218066f30 0x7f02180693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0230007fd0 tx=0x7f0230009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:22.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:22.955+0000 7f02297fa700 1 -- 172.21.15.163:0/27697500 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0224050cc0 con 0x7f0234159890 2023-10-09T06:31:23.212 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.210+0000 7f023b8ae700 1 -- 172.21.15.163:0/27697500 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f023404c3c0 con 0x7f0234159890 2023-10-09T06:31:23.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.213+0000 7f02297fa700 1 -- 172.21.15.163:0/27697500 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f02240170a0 con 0x7f0234159890 2023-10-09T06:31:23.218 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.215+0000 7f023b8ae700 1 -- 172.21.15.163:0/27697500 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0218066f30 msgr2=0x7f02180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:23.218 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.215+0000 7f023b8ae700 1 --2- 172.21.15.163:0/27697500 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0218066f30 0x7f02180693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0230007fd0 tx=0x7f0230009040 comp rx=0 tx=0).stop 2023-10-09T06:31:23.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.215+0000 7f023b8ae700 1 -- 172.21.15.163:0/27697500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0234159890 msgr2=0x7f02340c9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:23.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.215+0000 7f023b8ae700 1 --2- 172.21.15.163:0/27697500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0234159890 0x7f02340c9e80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f022400c560 tx=0x7f0224001020 comp rx=0 tx=0).stop 2023-10-09T06:31:23.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.215+0000 7f023b8ae700 1 -- 172.21.15.163:0/27697500 shutdown_connections 2023-10-09T06:31:23.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.215+0000 7f023b8ae700 1 --2- 172.21.15.163:0/27697500 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0218066f30 0x7f02180693e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:23.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.215+0000 7f023b8ae700 1 --2- 172.21.15.163:0/27697500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0234159c60 0x7f02340ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:23.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.215+0000 7f023b8ae700 1 --2- 172.21.15.163:0/27697500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0234069560 0x7f0234158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:23.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.215+0000 7f023b8ae700 1 --2- 172.21.15.163:0/27697500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0234159890 0x7f02340c9e80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:23.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.215+0000 7f023b8ae700 1 -- 172.21.15.163:0/27697500 >> 172.21.15.163:0/27697500 conn(0x7f0234068e20 msgr2=0x7f02340b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:23.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.215+0000 7f023b8ae700 1 -- 172.21.15.163:0/27697500 shutdown_connections 2023-10-09T06:31:23.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.215+0000 7f023b8ae700 1 -- 172.21.15.163:0/27697500 wait complete. 2023-10-09T06:31:23.229 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-09T06:31:23.351 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.349+0000 7f7e34755700 1 Processor -- start 2023-10-09T06:31:23.351 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.349+0000 7f7e34755700 1 -- start start 2023-10-09T06:31:23.353 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.350+0000 7f7e34755700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7e2c159890 0x7f7e2c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:23.353 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.350+0000 7f7e34755700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7e2c069560 0x7f7e2c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:23.353 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.350+0000 7f7e34755700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e2c069ee0 con 0x7f7e2c159c60 2023-10-09T06:31:23.353 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.350+0000 7f7e34755700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e2c15a160 con 0x7f7e2c159890 2023-10-09T06:31:23.354 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.350+0000 7f7e34755700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e2c0d13a0 con 0x7f7e2c069560 2023-10-09T06:31:23.354 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.350+0000 7f7e324f1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7e2c159890 0x7f7e2c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:23.354 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.350+0000 7f7e324f1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7e2c159890 0x7f7e2c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43314/0 (socket says 172.21.15.163:43314) 2023-10-09T06:31:23.354 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.350+0000 7f7e324f1700 1 -- 172.21.15.163:0/3707174613 learned_addr learned my addr 172.21.15.163:0/3707174613 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:23.354 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.351+0000 7f7e324f1700 1 -- 172.21.15.163:0/3707174613 >> v2:172.21.15.181:3300/0 conn(0x7f7e2c069560 msgr2=0x7f7e2c0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:23.354 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.351+0000 7f7e324f1700 1 --2- 172.21.15.163:0/3707174613 >> v2:172.21.15.181:3300/0 conn(0x7f7e2c069560 0x7f7e2c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:23.354 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.351+0000 7f7e324f1700 1 -- 172.21.15.163:0/3707174613 >> v1:172.21.15.181:6789/0 conn(0x7f7e2c159c60 legacy=0x7f7e2c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:23.354 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.351+0000 7f7e324f1700 1 -- 172.21.15.163:0/3707174613 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2c15e140 con 0x7f7e2c159890 2023-10-09T06:31:23.354 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.351+0000 7f7e324f1700 1 --2- 172.21.15.163:0/3707174613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e2c159890 0x7f7e2c15abd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c06a890 tx=0x7f7e28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a7809f973d8a04 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:23.354 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.351+0000 7f7e314ef700 1 -- 172.21.15.163:0/3707174613 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2800b820 con 0x7f7e2c159890 2023-10-09T06:31:23.354 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.351+0000 7f7e314ef700 1 -- 172.21.15.163:0/3707174613 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2800b980 con 0x7f7e2c159890 2023-10-09T06:31:23.355 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.351+0000 7f7e314ef700 1 -- 172.21.15.163:0/3707174613 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2800bd20 con 0x7f7e2c159890 2023-10-09T06:31:23.355 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.351+0000 7f7e34755700 1 -- 172.21.15.163:0/3707174613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e2c159890 msgr2=0x7f7e2c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:23.355 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.351+0000 7f7e34755700 1 --2- 172.21.15.163:0/3707174613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e2c159890 0x7f7e2c15abd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c06a890 tx=0x7f7e28009250 comp rx=0 tx=0).stop 2023-10-09T06:31:23.355 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.351+0000 7f7e34755700 1 -- 172.21.15.163:0/3707174613 shutdown_connections 2023-10-09T06:31:23.355 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.351+0000 7f7e34755700 1 --2- 172.21.15.163:0/3707174613 >> v2:172.21.15.181:3300/0 conn(0x7f7e2c069560 0x7f7e2c0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:23.355 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.351+0000 7f7e34755700 1 --2- 172.21.15.163:0/3707174613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e2c159890 0x7f7e2c15abd0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:23.355 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.351+0000 7f7e34755700 1 -- 172.21.15.163:0/3707174613 >> 172.21.15.163:0/3707174613 conn(0x7f7e2c068e20 msgr2=0x7f7e2c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:23.355 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.351+0000 7f7e34755700 1 -- 172.21.15.163:0/3707174613 shutdown_connections 2023-10-09T06:31:23.355 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.351+0000 7f7e34755700 1 -- 172.21.15.163:0/3707174613 wait complete. 2023-10-09T06:31:23.358 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.352+0000 7f7e34755700 1 Processor -- start 2023-10-09T06:31:23.359 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.353+0000 7f7e34755700 1 -- start start 2023-10-09T06:31:23.359 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.353+0000 7f7e34755700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e2c069560 0x7f7e2c152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:23.359 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.353+0000 7f7e34755700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e2c159890 0x7f7e2c1584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:23.359 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.353+0000 7f7e34755700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e2c159c60 0x7f7e2c0c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:23.359 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.353+0000 7f7e34755700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c152d60 con 0x7f7e2c069560 2023-10-09T06:31:23.359 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.353+0000 7f7e34755700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c0c92e0 con 0x7f7e2c159890 2023-10-09T06:31:23.359 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.353+0000 7f7e34755700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c0c86d0 con 0x7f7e2c159c60 2023-10-09T06:31:23.359 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.353+0000 7f7e324f1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e2c069560 0x7f7e2c152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:23.359 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.353+0000 7f7e324f1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e2c069560 0x7f7e2c152820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43316/0 (socket says 172.21.15.163:43316) 2023-10-09T06:31:23.360 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.353+0000 7f7e324f1700 1 -- 172.21.15.163:0/3483855530 learned_addr learned my addr 172.21.15.163:0/3483855530 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:23.360 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.354+0000 7f7e324f1700 1 -- 172.21.15.163:0/3483855530 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e2c159c60 msgr2=0x7f7e2c0c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:23.360 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.354+0000 7f7e324f1700 1 --2- 172.21.15.163:0/3483855530 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e2c159c60 0x7f7e2c0c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:23.360 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.354+0000 7f7e324f1700 1 -- 172.21.15.163:0/3483855530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e2c159890 msgr2=0x7f7e2c1584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:23.360 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.354+0000 7f7e324f1700 1 --2- 172.21.15.163:0/3483855530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e2c159890 0x7f7e2c1584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:23.360 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.354+0000 7f7e324f1700 1 -- 172.21.15.163:0/3483855530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e280094f0 con 0x7f7e2c069560 2023-10-09T06:31:23.360 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.354+0000 7f7e324f1700 1 --2- 172.21.15.163:0/3483855530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e2c069560 0x7f7e2c152820 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7e28003080 tx=0x7f7e28016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:23.360 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.354+0000 7f7e227fc700 1 -- 172.21.15.163:0/3483855530 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e28011e20 con 0x7f7e2c069560 2023-10-09T06:31:23.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.354+0000 7f7e227fc700 1 -- 172.21.15.163:0/3483855530 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2801f070 con 0x7f7e2c069560 2023-10-09T06:31:23.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.354+0000 7f7e227fc700 1 -- 172.21.15.163:0/3483855530 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2801a8a0 con 0x7f7e2c069560 2023-10-09T06:31:23.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.354+0000 7f7e34755700 1 -- 172.21.15.163:0/3483855530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e2c0c89b0 con 0x7f7e2c069560 2023-10-09T06:31:23.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.354+0000 7f7e34755700 1 -- 172.21.15.163:0/3483855530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2c0c8f70 con 0x7f7e2c069560 2023-10-09T06:31:23.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.355+0000 7f7e34755700 1 -- 172.21.15.163:0/3483855530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e2c04c3c0 con 0x7f7e2c069560 2023-10-09T06:31:23.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.355+0000 7f7e227fc700 1 -- 172.21.15.163:0/3483855530 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7e2801ac90 con 0x7f7e2c069560 2023-10-09T06:31:23.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.356+0000 7f7e227fc700 1 --2- 172.21.15.163:0/3483855530 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7e10066f30 0x7f7e100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:23.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.356+0000 7f7e227fc700 1 -- 172.21.15.163:0/3483855530 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f7e28021070 con 0x7f7e2c069560 2023-10-09T06:31:23.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.356+0000 7f7e31cf0700 1 --2- 172.21.15.163:0/3483855530 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7e10066f30 0x7f7e100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:23.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.357+0000 7f7e31cf0700 1 --2- 172.21.15.163:0/3483855530 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7e10066f30 0x7f7e100693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c009720 tx=0x7f7e1c006c40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:23.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.360+0000 7f7e227fc700 1 -- 172.21.15.163:0/3483855530 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7e28053ac0 con 0x7f7e2c069560 2023-10-09T06:31:23.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.618+0000 7f7e34755700 1 -- 172.21.15.163:0/3483855530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f7e2c0b63c0 con 0x7f7e2c069560 2023-10-09T06:31:23.624 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.623+0000 7f7e227fc700 1 -- 172.21.15.163:0/3483855530 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f7e2802b0a0 con 0x7f7e2c069560 2023-10-09T06:31:23.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.625+0000 7f7e34755700 1 -- 172.21.15.163:0/3483855530 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7e10066f30 msgr2=0x7f7e100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:23.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.626+0000 7f7e34755700 1 --2- 172.21.15.163:0/3483855530 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7e10066f30 0x7f7e100693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c009720 tx=0x7f7e1c006c40 comp rx=0 tx=0).stop 2023-10-09T06:31:23.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.626+0000 7f7e34755700 1 -- 172.21.15.163:0/3483855530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e2c069560 msgr2=0x7f7e2c152820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:23.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.626+0000 7f7e34755700 1 --2- 172.21.15.163:0/3483855530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e2c069560 0x7f7e2c152820 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7e28003080 tx=0x7f7e28016000 comp rx=0 tx=0).stop 2023-10-09T06:31:23.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.626+0000 7f7e34755700 1 -- 172.21.15.163:0/3483855530 shutdown_connections 2023-10-09T06:31:23.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.626+0000 7f7e34755700 1 --2- 172.21.15.163:0/3483855530 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7e10066f30 0x7f7e100693e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:23.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.626+0000 7f7e34755700 1 --2- 172.21.15.163:0/3483855530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e2c069560 0x7f7e2c152820 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:23.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.626+0000 7f7e34755700 1 --2- 172.21.15.163:0/3483855530 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e2c159c60 0x7f7e2c0c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:23.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.626+0000 7f7e34755700 1 --2- 172.21.15.163:0/3483855530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e2c159890 0x7f7e2c1584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:23.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.626+0000 7f7e34755700 1 -- 172.21.15.163:0/3483855530 >> 172.21.15.163:0/3483855530 conn(0x7f7e2c068e20 msgr2=0x7f7e2c0b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:23.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.626+0000 7f7e34755700 1 -- 172.21.15.163:0/3483855530 shutdown_connections 2023-10-09T06:31:23.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.626+0000 7f7e34755700 1 -- 172.21.15.163:0/3483855530 wait complete. 2023-10-09T06:31:23.638 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-09T06:31:23.754 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.753+0000 7ffb66cfb700 1 Processor -- start 2023-10-09T06:31:23.754 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.753+0000 7ffb66cfb700 1 -- start start 2023-10-09T06:31:23.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.754+0000 7ffb66cfb700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ffb60143050 0x7ffb60159c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:23.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.754+0000 7ffb66cfb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffb600b3f20 0x7ffb6006be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:23.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.754+0000 7ffb66cfb700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb6006c390 con 0x7ffb6006b9e0 2023-10-09T06:31:23.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.754+0000 7ffb66cfb700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb60143420 con 0x7ffb60143050 2023-10-09T06:31:23.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.754+0000 7ffb66cfb700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb6006a1e0 con 0x7ffb600b3f20 2023-10-09T06:31:23.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.754+0000 7ffb65cf9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ffb60143050 0x7ffb60159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:23.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.754+0000 7ffb654f8700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ffb6006b9e0 0x7ffb600b3810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:38798/0 (socket says 172.21.15.163:38798) 2023-10-09T06:31:23.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.754+0000 7ffb654f8700 1 -- 172.21.15.163:0/2575799446 learned_addr learned my addr 172.21.15.163:0/2575799446 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:23.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.754+0000 7ffb65cf9700 1 --2- 172.21.15.163:0/2575799446 >> v2:172.21.15.163:3300/0 conn(0x7ffb60143050 0x7ffb60159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43318/0 (socket says 172.21.15.163:43318) 2023-10-09T06:31:23.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.754+0000 7ffb664fa700 1 --2- 172.21.15.163:0/2575799446 >> v2:172.21.15.181:3300/0 conn(0x7ffb600b3f20 0x7ffb6006be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:23.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.754+0000 7ffb65cf9700 1 -- 172.21.15.163:0/2575799446 >> v2:172.21.15.181:3300/0 conn(0x7ffb600b3f20 msgr2=0x7ffb6006be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:23.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.754+0000 7ffb65cf9700 1 --2- 172.21.15.163:0/2575799446 >> v2:172.21.15.181:3300/0 conn(0x7ffb600b3f20 0x7ffb6006be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:23.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.754+0000 7ffb65cf9700 1 -- 172.21.15.163:0/2575799446 >> v1:172.21.15.163:6789/0 conn(0x7ffb6006b9e0 legacy=0x7ffb600b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:23.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.754+0000 7ffb65cf9700 1 -- 172.21.15.163:0/2575799446 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb600695d0 con 0x7ffb60143050 2023-10-09T06:31:23.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.754+0000 7ffb64cf7700 1 -- 172.21.15.163:0/2575799446 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1145743660 0 0) 0x7ffb6006c390 con 0x7ffb6006b9e0 2023-10-09T06:31:23.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.755+0000 7ffb65cf9700 1 --2- 172.21.15.163:0/2575799446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb60143050 0x7ffb60159c90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ffb600cda70 tx=0x7ffb58009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7207c1c96e69ddf5 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:23.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.755+0000 7ffb64cf7700 1 -- 172.21.15.163:0/2575799446 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5800e070 con 0x7ffb60143050 2023-10-09T06:31:23.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.755+0000 7ffb64cf7700 1 -- 172.21.15.163:0/2575799446 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5800bc30 con 0x7ffb60143050 2023-10-09T06:31:23.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.755+0000 7ffb64cf7700 1 -- 172.21.15.163:0/2575799446 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb58012430 con 0x7ffb60143050 2023-10-09T06:31:23.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.755+0000 7ffb66cfb700 1 -- 172.21.15.163:0/2575799446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb60143050 msgr2=0x7ffb60159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:23.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.755+0000 7ffb66cfb700 1 --2- 172.21.15.163:0/2575799446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb60143050 0x7ffb60159c90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ffb600cda70 tx=0x7ffb58009780 comp rx=0 tx=0).stop 2023-10-09T06:31:23.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.755+0000 7ffb66cfb700 1 -- 172.21.15.163:0/2575799446 shutdown_connections 2023-10-09T06:31:23.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.755+0000 7ffb66cfb700 1 --2- 172.21.15.163:0/2575799446 >> v2:172.21.15.181:3300/0 conn(0x7ffb600b3f20 0x7ffb6006be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:23.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.755+0000 7ffb66cfb700 1 --2- 172.21.15.163:0/2575799446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb60143050 0x7ffb60159c90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:23.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.755+0000 7ffb66cfb700 1 -- 172.21.15.163:0/2575799446 >> 172.21.15.163:0/2575799446 conn(0x7ffb600ce430 msgr2=0x7ffb600ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:23.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.755+0000 7ffb66cfb700 1 -- 172.21.15.163:0/2575799446 shutdown_connections 2023-10-09T06:31:23.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.755+0000 7ffb66cfb700 1 -- 172.21.15.163:0/2575799446 wait complete. 2023-10-09T06:31:23.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.756+0000 7ffb66cfb700 1 Processor -- start 2023-10-09T06:31:23.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.756+0000 7ffb66cfb700 1 -- start start 2023-10-09T06:31:23.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.757+0000 7ffb66cfb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb6006b9e0 0x7ffb60158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:23.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.757+0000 7ffb66cfb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb600b3f20 0x7ffb6015d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:23.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.757+0000 7ffb66cfb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb60143050 0x7ffb600ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:23.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.757+0000 7ffb66cfb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6015e170 con 0x7ffb600b3f20 2023-10-09T06:31:23.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.757+0000 7ffb66cfb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6015e450 con 0x7ffb60143050 2023-10-09T06:31:23.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.757+0000 7ffb66cfb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb601396c0 con 0x7ffb6006b9e0 2023-10-09T06:31:23.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.757+0000 7ffb654f8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb600b3f20 0x7ffb6015d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:23.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.757+0000 7ffb654f8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb600b3f20 0x7ffb6015d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43332/0 (socket says 172.21.15.163:43332) 2023-10-09T06:31:23.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.757+0000 7ffb654f8700 1 -- 172.21.15.163:0/81179449 learned_addr learned my addr 172.21.15.163:0/81179449 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:23.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.757+0000 7ffb654f8700 1 -- 172.21.15.163:0/81179449 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb6006b9e0 msgr2=0x7ffb60158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:23.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.757+0000 7ffb654f8700 1 --2- 172.21.15.163:0/81179449 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb6006b9e0 0x7ffb60158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:23.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.758+0000 7ffb654f8700 1 -- 172.21.15.163:0/81179449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb60143050 msgr2=0x7ffb600ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:23.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.758+0000 7ffb654f8700 1 --2- 172.21.15.163:0/81179449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb60143050 0x7ffb600ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:23.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.758+0000 7ffb654f8700 1 -- 172.21.15.163:0/81179449 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb58009a20 con 0x7ffb600b3f20 2023-10-09T06:31:23.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.758+0000 7ffb654f8700 1 --2- 172.21.15.163:0/81179449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb600b3f20 0x7ffb6015d7d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000c5b0 tx=0x7ffb50001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:23.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.758+0000 7ffb55ffb700 1 -- 172.21.15.163:0/81179449 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb50005b30 con 0x7ffb600b3f20 2023-10-09T06:31:23.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.758+0000 7ffb55ffb700 1 -- 172.21.15.163:0/81179449 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb50005c90 con 0x7ffb600b3f20 2023-10-09T06:31:23.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.758+0000 7ffb66cfb700 1 -- 172.21.15.163:0/81179449 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb601399a0 con 0x7ffb600b3f20 2023-10-09T06:31:23.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.758+0000 7ffb55ffb700 1 -- 172.21.15.163:0/81179449 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb50007300 con 0x7ffb600b3f20 2023-10-09T06:31:23.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.758+0000 7ffb66cfb700 1 -- 172.21.15.163:0/81179449 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb60158860 con 0x7ffb600b3f20 2023-10-09T06:31:23.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.759+0000 7ffb66cfb700 1 -- 172.21.15.163:0/81179449 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb60139ec0 con 0x7ffb600b3f20 2023-10-09T06:31:23.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.759+0000 7ffb55ffb700 1 -- 172.21.15.163:0/81179449 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffb5001b3b0 con 0x7ffb600b3f20 2023-10-09T06:31:23.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.760+0000 7ffb55ffb700 1 --2- 172.21.15.163:0/81179449 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffb44066f30 0x7ffb440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:23.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.760+0000 7ffb55ffb700 1 -- 172.21.15.163:0/81179449 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ffb50083730 con 0x7ffb600b3f20 2023-10-09T06:31:23.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.760+0000 7ffb65cf9700 1 --2- 172.21.15.163:0/81179449 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffb44066f30 0x7ffb440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:23.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.761+0000 7ffb65cf9700 1 --2- 172.21.15.163:0/81179449 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffb44066f30 0x7ffb440693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffb600cda70 tx=0x7ffb58019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:23.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:23.764+0000 7ffb55ffb700 1 -- 172.21.15.163:0/81179449 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffb60139ec0 con 0x7ffb600b3f20 2023-10-09T06:31:24.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.026+0000 7ffb66cfb700 1 -- 172.21.15.163:0/81179449 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7ffb60139ec0 con 0x7ffb600b3f20 2023-10-09T06:31:24.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.031+0000 7ffb55ffb700 1 -- 172.21.15.163:0/81179449 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7ffb60139ec0 con 0x7ffb600b3f20 2023-10-09T06:31:24.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.033+0000 7ffb66cfb700 1 -- 172.21.15.163:0/81179449 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffb44066f30 msgr2=0x7ffb440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:24.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.033+0000 7ffb66cfb700 1 --2- 172.21.15.163:0/81179449 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffb44066f30 0x7ffb440693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffb600cda70 tx=0x7ffb58019000 comp rx=0 tx=0).stop 2023-10-09T06:31:24.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.033+0000 7ffb66cfb700 1 -- 172.21.15.163:0/81179449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb600b3f20 msgr2=0x7ffb6015d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:24.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.033+0000 7ffb66cfb700 1 --2- 172.21.15.163:0/81179449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb600b3f20 0x7ffb6015d7d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000c5b0 tx=0x7ffb50001070 comp rx=0 tx=0).stop 2023-10-09T06:31:24.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.033+0000 7ffb66cfb700 1 -- 172.21.15.163:0/81179449 shutdown_connections 2023-10-09T06:31:24.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.033+0000 7ffb66cfb700 1 --2- 172.21.15.163:0/81179449 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffb44066f30 0x7ffb440693e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.033+0000 7ffb66cfb700 1 --2- 172.21.15.163:0/81179449 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb60143050 0x7ffb600ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.033+0000 7ffb66cfb700 1 --2- 172.21.15.163:0/81179449 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb6006b9e0 0x7ffb60158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.033+0000 7ffb66cfb700 1 --2- 172.21.15.163:0/81179449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb600b3f20 0x7ffb6015d7d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.033+0000 7ffb66cfb700 1 -- 172.21.15.163:0/81179449 >> 172.21.15.163:0/81179449 conn(0x7ffb600ce430 msgr2=0x7ffb60140df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:24.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.033+0000 7ffb66cfb700 1 -- 172.21.15.163:0/81179449 shutdown_connections 2023-10-09T06:31:24.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.033+0000 7ffb66cfb700 1 -- 172.21.15.163:0/81179449 wait complete. 2023-10-09T06:31:24.046 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-09T06:31:24.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.164+0000 7fa16e240700 1 Processor -- start 2023-10-09T06:31:24.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.164+0000 7fa16e240700 1 -- start start 2023-10-09T06:31:24.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.165+0000 7fa16e240700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1681597f0 0x7fa1680b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:24.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.165+0000 7fa16e240700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1680b11d0 con 0x7fa16815d230 2023-10-09T06:31:24.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.165+0000 7fa16e240700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1680b14b0 con 0x7fa168159420 2023-10-09T06:31:24.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.165+0000 7fa16e240700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1680b1790 con 0x7fa1681597f0 2023-10-09T06:31:24.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.165+0000 7fa16da3f700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fa16815d230 0x7fa1680b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:38804/0 (socket says 172.21.15.163:38804) 2023-10-09T06:31:24.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.165+0000 7fa16da3f700 1 -- 172.21.15.163:0/3364904707 learned_addr learned my addr 172.21.15.163:0/3364904707 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:24.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.165+0000 7fa16ca3d700 1 --2- 172.21.15.163:0/3364904707 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1681597f0 0x7fa1680b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:24.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.165+0000 7fa15ffff700 1 -- 172.21.15.163:0/3364904707 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 591161260 0 0) 0x7fa1680b11d0 con 0x7fa16815d230 2023-10-09T06:31:24.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.165+0000 7fa15ffff700 1 -- 172.21.15.163:0/3364904707 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa154003570 con 0x7fa16815d230 2023-10-09T06:31:24.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.165+0000 7fa15ffff700 1 -- 172.21.15.163:0/3364904707 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 10923908 0 0) 0x7fa154003570 con 0x7fa16815d230 2023-10-09T06:31:24.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.165+0000 7fa15ffff700 1 -- 172.21.15.163:0/3364904707 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1681597f0 msgr2=0x7fa1680b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:24.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.165+0000 7fa15ffff700 1 --2- 172.21.15.163:0/3364904707 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1681597f0 0x7fa1680b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.165+0000 7fa15ffff700 1 -- 172.21.15.163:0/3364904707 >> v1:172.21.15.181:6789/0 conn(0x7fa168159420 legacy=0x7fa1680b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:24.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.165+0000 7fa15ffff700 1 -- 172.21.15.163:0/3364904707 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1680b1a70 con 0x7fa16815d230 2023-10-09T06:31:24.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.165+0000 7fa15ffff700 1 -- 172.21.15.163:0/3364904707 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fa1600017f0 con 0x7fa16815d230 2023-10-09T06:31:24.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.166+0000 7fa15ffff700 1 -- 172.21.15.163:0/3364904707 >> v1:172.21.15.163:6789/0 conn(0x7fa16815d230 legacy=0x7fa1680b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:24.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.166+0000 7fa15ffff700 1 --2- 172.21.15.163:0/3364904707 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1681597f0 0x7fa154003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:24.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.166+0000 7fa15ffff700 1 --2- 172.21.15.163:0/3364904707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa154004010 0x7fa154004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:24.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.166+0000 7fa15ffff700 1 --2- 172.21.15.163:0/3364904707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa154004d50 0x7fa1540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:24.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.166+0000 7fa15ffff700 1 -- 172.21.15.163:0/3364904707 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1680b1790 con 0x7fa154004d50 2023-10-09T06:31:24.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.166+0000 7fa15ffff700 1 -- 172.21.15.163:0/3364904707 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15400bbf0 con 0x7fa154004010 2023-10-09T06:31:24.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.166+0000 7fa15ffff700 1 -- 172.21.15.163:0/3364904707 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15400bdb0 con 0x7fa1681597f0 2023-10-09T06:31:24.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.166+0000 7fa16da3f700 1 --2- 172.21.15.163:0/3364904707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa154004d50 0x7fa1540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:24.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.166+0000 7fa16d23e700 1 --2- 172.21.15.163:0/3364904707 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1681597f0 0x7fa154003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:24.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.166+0000 7fa16ca3d700 1 --2- 172.21.15.163:0/3364904707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa154004010 0x7fa154004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:24.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.166+0000 7fa16da3f700 1 -- 172.21.15.163:0/3364904707 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1681597f0 msgr2=0x7fa154003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:24.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.166+0000 7fa16da3f700 1 --2- 172.21.15.163:0/3364904707 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1681597f0 0x7fa154003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.166+0000 7fa16da3f700 1 -- 172.21.15.163:0/3364904707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa154004010 msgr2=0x7fa154004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:24.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.166+0000 7fa16da3f700 1 --2- 172.21.15.163:0/3364904707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa154004010 0x7fa154004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.166+0000 7fa16da3f700 1 -- 172.21.15.163:0/3364904707 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa154003570 con 0x7fa154004d50 2023-10-09T06:31:24.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.166+0000 7fa16da3f700 1 --2- 172.21.15.163:0/3364904707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa154004d50 0x7fa1540051c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa16000a900 tx=0x7fa1600021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a880f0db30886a56 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:24.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.166+0000 7fa15ffff700 1 -- 172.21.15.163:0/3364904707 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1600070d0 con 0x7fa154004d50 2023-10-09T06:31:24.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.166+0000 7fa15ffff700 1 -- 172.21.15.163:0/3364904707 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa160007230 con 0x7fa154004d50 2023-10-09T06:31:24.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.166+0000 7fa15ffff700 1 -- 172.21.15.163:0/3364904707 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1600075f0 con 0x7fa154004d50 2023-10-09T06:31:24.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.167+0000 7fa16e240700 1 -- 172.21.15.163:0/3364904707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa154004d50 msgr2=0x7fa1540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:24.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.167+0000 7fa16e240700 1 --2- 172.21.15.163:0/3364904707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa154004d50 0x7fa1540051c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa16000a900 tx=0x7fa1600021d0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.167+0000 7fa16da3f700 1 -- 172.21.15.163:0/3364904707 reap_dead start 2023-10-09T06:31:24.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.167+0000 7fa16e240700 1 -- 172.21.15.163:0/3364904707 shutdown_connections 2023-10-09T06:31:24.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.167+0000 7fa16e240700 1 -- 172.21.15.163:0/3364904707 >> 172.21.15.163:0/3364904707 conn(0x7fa1680d0660 msgr2=0x7fa1680d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:24.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.167+0000 7fa16e240700 1 -- 172.21.15.163:0/3364904707 shutdown_connections 2023-10-09T06:31:24.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.167+0000 7fa16e240700 1 -- 172.21.15.163:0/3364904707 wait complete. 2023-10-09T06:31:24.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.168+0000 7fa16e240700 1 Processor -- start 2023-10-09T06:31:24.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.168+0000 7fa16e240700 1 -- start start 2023-10-09T06:31:24.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.169+0000 7fa16e240700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa168143bd0 0x7fa1680ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:24.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.169+0000 7fa16e240700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa16815b0d0 0x7fa1680cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:24.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.169+0000 7fa16e240700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa168144140 0x7fa1681445c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:24.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.169+0000 7fa16e240700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa168144b00 con 0x7fa168143bd0 2023-10-09T06:31:24.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.169+0000 7fa16e240700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1680cab40 con 0x7fa168144140 2023-10-09T06:31:24.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.169+0000 7fa16e240700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1680cae20 con 0x7fa16815b0d0 2023-10-09T06:31:24.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.169+0000 7fa16d23e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa168143bd0 0x7fa1680ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:24.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.169+0000 7fa16d23e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa168143bd0 0x7fa1680ca480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43358/0 (socket says 172.21.15.163:43358) 2023-10-09T06:31:24.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.169+0000 7fa16d23e700 1 -- 172.21.15.163:0/955315385 learned_addr learned my addr 172.21.15.163:0/955315385 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:24.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.169+0000 7fa16ca3d700 1 --2- 172.21.15.163:0/955315385 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa16815b0d0 0x7fa1680cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:24.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.169+0000 7fa16d23e700 1 -- 172.21.15.163:0/955315385 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa16815b0d0 msgr2=0x7fa1680cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:24.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.169+0000 7fa16d23e700 1 --2- 172.21.15.163:0/955315385 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa16815b0d0 0x7fa1680cbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.169+0000 7fa16d23e700 1 -- 172.21.15.163:0/955315385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa168144140 msgr2=0x7fa1681445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:24.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.169+0000 7fa16d23e700 1 --2- 172.21.15.163:0/955315385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa168144140 0x7fa1681445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.169+0000 7fa16d23e700 1 -- 172.21.15.163:0/955315385 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1680cb100 con 0x7fa168143bd0 2023-10-09T06:31:24.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.169+0000 7fa16d23e700 1 --2- 172.21.15.163:0/955315385 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa168143bd0 0x7fa1680ca480 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa164002920 tx=0x7fa164007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:24.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.169+0000 7fa15d7fa700 1 -- 172.21.15.163:0/955315385 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa164003410 con 0x7fa168143bd0 2023-10-09T06:31:24.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.169+0000 7fa15d7fa700 1 -- 172.21.15.163:0/955315385 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16400f8d0 con 0x7fa168143bd0 2023-10-09T06:31:24.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.169+0000 7fa16e240700 1 -- 172.21.15.163:0/955315385 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1680cb3e0 con 0x7fa168143bd0 2023-10-09T06:31:24.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.169+0000 7fa15d7fa700 1 -- 172.21.15.163:0/955315385 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16400fc70 con 0x7fa168143bd0 2023-10-09T06:31:24.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.170+0000 7fa16e240700 1 -- 172.21.15.163:0/955315385 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa16815b520 con 0x7fa168143bd0 2023-10-09T06:31:24.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.170+0000 7fa16e240700 1 -- 172.21.15.163:0/955315385 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1680cb900 con 0x7fa168143bd0 2023-10-09T06:31:24.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.171+0000 7fa15d7fa700 1 -- 172.21.15.163:0/955315385 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa164013600 con 0x7fa168143bd0 2023-10-09T06:31:24.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.171+0000 7fa15d7fa700 1 --2- 172.21.15.163:0/955315385 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa14c066f30 0x7fa14c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:24.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.171+0000 7fa15d7fa700 1 -- 172.21.15.163:0/955315385 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fa1640840c0 con 0x7fa168143bd0 2023-10-09T06:31:24.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.172+0000 7fa16ca3d700 1 --2- 172.21.15.163:0/955315385 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa14c066f30 0x7fa14c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:24.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.172+0000 7fa16ca3d700 1 --2- 172.21.15.163:0/955315385 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa14c066f30 0x7fa14c0693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa158006f00 tx=0x7fa158008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:24.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.175+0000 7fa15d7fa700 1 -- 172.21.15.163:0/955315385 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa1680cb900 con 0x7fa168143bd0 2023-10-09T06:31:24.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.434+0000 7fa16e240700 1 -- 172.21.15.163:0/955315385 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fa1680cb900 con 0x7fa168143bd0 2023-10-09T06:31:24.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.440+0000 7fa15d7fa700 1 -- 172.21.15.163:0/955315385 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa1640515e0 con 0x7fa168143bd0 2023-10-09T06:31:24.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.442+0000 7fa16e240700 1 -- 172.21.15.163:0/955315385 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa14c066f30 msgr2=0x7fa14c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:24.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.442+0000 7fa16e240700 1 --2- 172.21.15.163:0/955315385 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa14c066f30 0x7fa14c0693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa158006f00 tx=0x7fa158008040 comp rx=0 tx=0).stop 2023-10-09T06:31:24.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.442+0000 7fa16e240700 1 -- 172.21.15.163:0/955315385 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa168143bd0 msgr2=0x7fa1680ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:24.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.442+0000 7fa16e240700 1 --2- 172.21.15.163:0/955315385 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa168143bd0 0x7fa1680ca480 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa164002920 tx=0x7fa164007d30 comp rx=0 tx=0).stop 2023-10-09T06:31:24.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.442+0000 7fa16e240700 1 -- 172.21.15.163:0/955315385 shutdown_connections 2023-10-09T06:31:24.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.442+0000 7fa16e240700 1 --2- 172.21.15.163:0/955315385 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa14c066f30 0x7fa14c0693e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.442+0000 7fa16e240700 1 --2- 172.21.15.163:0/955315385 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa168143bd0 0x7fa1680ca480 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.442+0000 7fa16e240700 1 --2- 172.21.15.163:0/955315385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa168144140 0x7fa1681445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.442+0000 7fa16e240700 1 --2- 172.21.15.163:0/955315385 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa16815b0d0 0x7fa1680cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.442+0000 7fa16e240700 1 -- 172.21.15.163:0/955315385 >> 172.21.15.163:0/955315385 conn(0x7fa1680d0660 msgr2=0x7fa168068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:24.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.442+0000 7fa16e240700 1 -- 172.21.15.163:0/955315385 shutdown_connections 2023-10-09T06:31:24.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.442+0000 7fa16e240700 1 -- 172.21.15.163:0/955315385 wait complete. 2023-10-09T06:31:24.457 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:31:24.572 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.571+0000 7f97accdd700 1 Processor -- start 2023-10-09T06:31:24.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.571+0000 7f97accdd700 1 -- start start 2023-10-09T06:31:24.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.572+0000 7f97accdd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97a80c91e0 0x7f97a80cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:24.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.572+0000 7f97accdd700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97a80cc210 con 0x7f97a806aa40 2023-10-09T06:31:24.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.572+0000 7f97accdd700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97a80cc4f0 con 0x7f97a806ae10 2023-10-09T06:31:24.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.572+0000 7f97accdd700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a80cc7d0 con 0x7f97a80c91e0 2023-10-09T06:31:24.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.572+0000 7f97a659c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f97a806aa40 0x7f97a806bac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:38814/0 (socket says 172.21.15.163:38814) 2023-10-09T06:31:24.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.572+0000 7f97a659c700 1 -- 172.21.15.163:0/3235869170 learned_addr learned my addr 172.21.15.163:0/3235869170 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:24.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.573+0000 7f979dd9b700 1 --2- 172.21.15.163:0/3235869170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97a80c91e0 0x7f97a80cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:24.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.573+0000 7f97a5d9b700 1 -- 172.21.15.163:0/3235869170 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1962780148 0 0) 0x7f97a80cc210 con 0x7f97a806aa40 2023-10-09T06:31:24.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.573+0000 7f97a5d9b700 1 -- 172.21.15.163:0/3235869170 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9794005780 con 0x7f97a806aa40 2023-10-09T06:31:24.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.573+0000 7f97a5d9b700 1 -- 172.21.15.163:0/3235869170 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3426338202 0 0) 0x7f9794005780 con 0x7f97a806aa40 2023-10-09T06:31:24.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.573+0000 7f97a5d9b700 1 -- 172.21.15.163:0/3235869170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97a80c91e0 msgr2=0x7f97a80cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:24.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.573+0000 7f97a5d9b700 1 --2- 172.21.15.163:0/3235869170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97a80c91e0 0x7f97a80cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.573+0000 7f97a5d9b700 1 -- 172.21.15.163:0/3235869170 >> v1:172.21.15.181:6789/0 conn(0x7f97a806ae10 legacy=0x7f97a806b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:24.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.573+0000 7f97a5d9b700 1 -- 172.21.15.163:0/3235869170 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a80ccab0 con 0x7f97a806aa40 2023-10-09T06:31:24.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.573+0000 7f97a5d9b700 1 -- 172.21.15.163:0/3235869170 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f97980015f0 con 0x7f97a806aa40 2023-10-09T06:31:24.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.573+0000 7f97a5d9b700 1 -- 172.21.15.163:0/3235869170 >> v1:172.21.15.163:6789/0 conn(0x7f97a806aa40 legacy=0x7f97a806bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:24.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.573+0000 7f97a5d9b700 1 --2- 172.21.15.163:0/3235869170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97a80c91e0 0x7f9794005ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:24.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.573+0000 7f97a5d9b700 1 --2- 172.21.15.163:0/3235869170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9794006220 0x7f9794006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:24.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.573+0000 7f97a5d9b700 1 --2- 172.21.15.163:0/3235869170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9794006f60 0x7f97940073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:24.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.573+0000 7f97a5d9b700 1 -- 172.21.15.163:0/3235869170 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a80cc7d0 con 0x7f9794006f60 2023-10-09T06:31:24.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.573+0000 7f97a5d9b700 1 -- 172.21.15.163:0/3235869170 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979400de00 con 0x7f9794006220 2023-10-09T06:31:24.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.573+0000 7f97a5d9b700 1 -- 172.21.15.163:0/3235869170 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979400dfc0 con 0x7f97a80c91e0 2023-10-09T06:31:24.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.573+0000 7f97a6d9d700 1 --2- 172.21.15.163:0/3235869170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9794006f60 0x7f97940073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:24.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.573+0000 7f97a659c700 1 --2- 172.21.15.163:0/3235869170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97a80c91e0 0x7f9794005ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:24.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.573+0000 7f979dd9b700 1 --2- 172.21.15.163:0/3235869170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9794006220 0x7f9794006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:24.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.573+0000 7f97a6d9d700 1 -- 172.21.15.163:0/3235869170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97a80c91e0 msgr2=0x7f9794005ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:24.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.573+0000 7f97a6d9d700 1 --2- 172.21.15.163:0/3235869170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97a80c91e0 0x7f9794005ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.573+0000 7f97a6d9d700 1 -- 172.21.15.163:0/3235869170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9794006220 msgr2=0x7f9794006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:24.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.573+0000 7f97a6d9d700 1 --2- 172.21.15.163:0/3235869170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9794006220 0x7f9794006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.573+0000 7f97a6d9d700 1 -- 172.21.15.163:0/3235869170 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9794005780 con 0x7f9794006f60 2023-10-09T06:31:24.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.574+0000 7f97a6d9d700 1 --2- 172.21.15.163:0/3235869170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9794006f60 0x7f97940073d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f97a0001f90 tx=0x7f97a0001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78fdf35decf33786 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:24.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.574+0000 7f97a5d9b700 1 -- 172.21.15.163:0/3235869170 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a0001950 con 0x7f9794006f60 2023-10-09T06:31:24.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.574+0000 7f97a5d9b700 1 -- 172.21.15.163:0/3235869170 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a000c8b0 con 0x7f9794006f60 2023-10-09T06:31:24.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.574+0000 7f97a5d9b700 1 -- 172.21.15.163:0/3235869170 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a000cc70 con 0x7f9794006f60 2023-10-09T06:31:24.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.574+0000 7f97accdd700 1 -- 172.21.15.163:0/3235869170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9794006f60 msgr2=0x7f97940073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:24.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.574+0000 7f97accdd700 1 --2- 172.21.15.163:0/3235869170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9794006f60 0x7f97940073d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f97a0001f90 tx=0x7f97a0001640 comp rx=0 tx=0).stop 2023-10-09T06:31:24.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.574+0000 7f97a6d9d700 1 -- 172.21.15.163:0/3235869170 reap_dead start 2023-10-09T06:31:24.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.574+0000 7f97accdd700 1 -- 172.21.15.163:0/3235869170 shutdown_connections 2023-10-09T06:31:24.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.574+0000 7f97accdd700 1 -- 172.21.15.163:0/3235869170 >> 172.21.15.163:0/3235869170 conn(0x7f97a80cd670 msgr2=0x7f97a80cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:24.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.574+0000 7f97accdd700 1 -- 172.21.15.163:0/3235869170 shutdown_connections 2023-10-09T06:31:24.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.574+0000 7f97accdd700 1 -- 172.21.15.163:0/3235869170 wait complete. 2023-10-09T06:31:24.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.575+0000 7f97accdd700 1 Processor -- start 2023-10-09T06:31:24.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.576+0000 7f97accdd700 1 -- start start 2023-10-09T06:31:24.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.576+0000 7f97accdd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9794006f60 0x7f97a814e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:24.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.576+0000 7f97accdd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97a814f1a0 0x7f97a8159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:24.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.576+0000 7f97accdd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97a814f610 0x7f97a815a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:24.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.576+0000 7f97accdd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a8155410 con 0x7f97a814f1a0 2023-10-09T06:31:24.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.576+0000 7f97accdd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a80b0410 con 0x7f97a814f610 2023-10-09T06:31:24.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.576+0000 7f97accdd700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a8154800 con 0x7f9794006f60 2023-10-09T06:31:24.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.576+0000 7f979dd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97a814f1a0 0x7f97a8159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:24.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.576+0000 7f979dd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97a814f1a0 0x7f97a8159e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43366/0 (socket says 172.21.15.163:43366) 2023-10-09T06:31:24.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.576+0000 7f979dd9b700 1 -- 172.21.15.163:0/2722180566 learned_addr learned my addr 172.21.15.163:0/2722180566 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:24.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.576+0000 7f97a6d9d700 1 --2- 172.21.15.163:0/2722180566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97a814f610 0x7f97a815a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:24.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.576+0000 7f97a659c700 1 --2- 172.21.15.163:0/2722180566 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9794006f60 0x7f97a814e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:24.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.577+0000 7f979dd9b700 1 -- 172.21.15.163:0/2722180566 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9794006f60 msgr2=0x7f97a814e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:24.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.577+0000 7f979dd9b700 1 --2- 172.21.15.163:0/2722180566 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9794006f60 0x7f97a814e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.577+0000 7f979dd9b700 1 -- 172.21.15.163:0/2722180566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97a814f610 msgr2=0x7f97a815a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:24.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.577+0000 7f979dd9b700 1 --2- 172.21.15.163:0/2722180566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97a814f610 0x7f97a815a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.577+0000 7f979dd9b700 1 -- 172.21.15.163:0/2722180566 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a8154ae0 con 0x7f97a814f1a0 2023-10-09T06:31:24.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.577+0000 7f97a6d9d700 1 --2- 172.21.15.163:0/2722180566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97a814f610 0x7f97a815a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:24.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.577+0000 7f97a659c700 1 --2- 172.21.15.163:0/2722180566 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9794006f60 0x7f97a814e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:24.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.577+0000 7f979dd9b700 1 --2- 172.21.15.163:0/2722180566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97a814f1a0 0x7f97a8159e10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9790011b40 tx=0x7f97900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:24.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.577+0000 7f979effd700 1 -- 172.21.15.163:0/2722180566 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979000fa10 con 0x7f97a814f1a0 2023-10-09T06:31:24.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.577+0000 7f979effd700 1 -- 172.21.15.163:0/2722180566 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979000fb70 con 0x7f97a814f1a0 2023-10-09T06:31:24.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.577+0000 7f979effd700 1 -- 172.21.15.163:0/2722180566 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979001d480 con 0x7f97a814f1a0 2023-10-09T06:31:24.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.577+0000 7f97accdd700 1 -- 172.21.15.163:0/2722180566 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a8154dc0 con 0x7f97a814f1a0 2023-10-09T06:31:24.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.577+0000 7f97accdd700 1 -- 172.21.15.163:0/2722180566 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a814ec30 con 0x7f97a814f1a0 2023-10-09T06:31:24.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.578+0000 7f97accdd700 1 -- 172.21.15.163:0/2722180566 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a804c3c0 con 0x7f97a814f1a0 2023-10-09T06:31:24.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.578+0000 7f979effd700 1 -- 172.21.15.163:0/2722180566 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f979001d870 con 0x7f97a814f1a0 2023-10-09T06:31:24.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.579+0000 7f979effd700 1 --2- 172.21.15.163:0/2722180566 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9788066f30 0x7f97880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:24.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.579+0000 7f979effd700 1 -- 172.21.15.163:0/2722180566 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f9790018070 con 0x7f97a814f1a0 2023-10-09T06:31:24.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.579+0000 7f97a659c700 1 --2- 172.21.15.163:0/2722180566 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9788066f30 0x7f97880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:24.588 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.579+0000 7f97a659c700 1 --2- 172.21.15.163:0/2722180566 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9788066f30 0x7f97880693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9798001720 tx=0x7f9798009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:24.588 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.583+0000 7f979effd700 1 -- 172.21.15.163:0/2722180566 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9790052590 con 0x7f97a814f1a0 2023-10-09T06:31:24.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.769+0000 7f97accdd700 1 -- 172.21.15.163:0/2722180566 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97a8156b10 con 0x7f97a814f1a0 2023-10-09T06:31:24.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.770+0000 7f979effd700 1 -- 172.21.15.163:0/2722180566 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+12182 (secure 0 0 0) 0x7f9790022020 con 0x7f97a814f1a0 2023-10-09T06:31:24.771 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:24.771 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":27,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:31:21.613749+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:31:24.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.772+0000 7f97accdd700 1 -- 172.21.15.163:0/2722180566 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9788066f30 msgr2=0x7f97880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:24.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.772+0000 7f97accdd700 1 --2- 172.21.15.163:0/2722180566 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9788066f30 0x7f97880693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9798001720 tx=0x7f9798009040 comp rx=0 tx=0).stop 2023-10-09T06:31:24.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.772+0000 7f97accdd700 1 -- 172.21.15.163:0/2722180566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97a814f1a0 msgr2=0x7f97a8159e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:24.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.772+0000 7f97accdd700 1 --2- 172.21.15.163:0/2722180566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97a814f1a0 0x7f97a8159e10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9790011b40 tx=0x7f97900091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.772+0000 7f97accdd700 1 -- 172.21.15.163:0/2722180566 shutdown_connections 2023-10-09T06:31:24.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.772+0000 7f97accdd700 1 --2- 172.21.15.163:0/2722180566 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9788066f30 0x7f97880693e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.772+0000 7f97accdd700 1 --2- 172.21.15.163:0/2722180566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97a814f610 0x7f97a815a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.772+0000 7f97accdd700 1 --2- 172.21.15.163:0/2722180566 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9794006f60 0x7f97a814e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.772+0000 7f97accdd700 1 --2- 172.21.15.163:0/2722180566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97a814f1a0 0x7f97a8159e10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.772+0000 7f97accdd700 1 -- 172.21.15.163:0/2722180566 >> 172.21.15.163:0/2722180566 conn(0x7f97a80cd670 msgr2=0x7f97a81512a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:24.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.772+0000 7f97accdd700 1 -- 172.21.15.163:0/2722180566 shutdown_connections 2023-10-09T06:31:24.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.772+0000 7f97accdd700 1 -- 172.21.15.163:0/2722180566 wait complete. 2023-10-09T06:31:24.785 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-09T06:31:24.786 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:31:24.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.903+0000 7f801df4f700 1 Processor -- start 2023-10-09T06:31:24.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.903+0000 7f801df4f700 1 -- start start 2023-10-09T06:31:24.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.904+0000 7f801df4f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8018141500 0x7f80180c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:24.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.904+0000 7f801df4f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f801806c0a0 0x7f80180c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:24.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.904+0000 7f801df4f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f801815c2a0 con 0x7f801806bcd0 2023-10-09T06:31:24.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.904+0000 7f801df4f700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80180c9880 con 0x7f8018141500 2023-10-09T06:31:24.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.904+0000 7f801df4f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f801815b690 con 0x7f801806c0a0 2023-10-09T06:31:24.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.904+0000 7f8016ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8018141500 0x7f80180c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:24.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.904+0000 7f80177fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f801806bcd0 0x7f8018140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:38816/0 (socket says 172.21.15.163:38816) 2023-10-09T06:31:24.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.904+0000 7f80177fe700 1 -- 172.21.15.163:0/3951493056 learned_addr learned my addr 172.21.15.163:0/3951493056 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:24.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.905+0000 7f8016ffd700 1 -- 172.21.15.163:0/3951493056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f801806c0a0 msgr2=0x7f80180c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:24.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.905+0000 7f8016ffd700 1 --2- 172.21.15.163:0/3951493056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f801806c0a0 0x7f80180c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.905+0000 7f8016ffd700 1 -- 172.21.15.163:0/3951493056 >> v1:172.21.15.163:6789/0 conn(0x7f801806bcd0 legacy=0x7f8018140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:24.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.905+0000 7f8016ffd700 1 -- 172.21.15.163:0/3951493056 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801815b970 con 0x7f8018141500 2023-10-09T06:31:24.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.905+0000 7f80167fc700 1 -- 172.21.15.163:0/3951493056 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1060447045 0 0) 0x7f801815c2a0 con 0x7f801806bcd0 2023-10-09T06:31:24.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.905+0000 7f8016ffd700 1 --2- 172.21.15.163:0/3951493056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8018141500 0x7f80180c8e00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f800c0098e0 tx=0x7f800c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646175916356de1f server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:24.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.905+0000 7f80167fc700 1 -- 172.21.15.163:0/3951493056 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c009e80 con 0x7f8018141500 2023-10-09T06:31:24.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.905+0000 7f80167fc700 1 -- 172.21.15.163:0/3951493056 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800c00b860 con 0x7f8018141500 2023-10-09T06:31:24.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.905+0000 7f80167fc700 1 -- 172.21.15.163:0/3951493056 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c00bbe0 con 0x7f8018141500 2023-10-09T06:31:24.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.905+0000 7f801df4f700 1 -- 172.21.15.163:0/3951493056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8018141500 msgr2=0x7f80180c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:24.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.905+0000 7f801df4f700 1 --2- 172.21.15.163:0/3951493056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8018141500 0x7f80180c8e00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f800c0098e0 tx=0x7f800c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.905+0000 7f801df4f700 1 -- 172.21.15.163:0/3951493056 shutdown_connections 2023-10-09T06:31:24.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.905+0000 7f801df4f700 1 --2- 172.21.15.163:0/3951493056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f801806c0a0 0x7f80180c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.905+0000 7f801df4f700 1 --2- 172.21.15.163:0/3951493056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8018141500 0x7f80180c8e00 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.906+0000 7f801df4f700 1 -- 172.21.15.163:0/3951493056 >> 172.21.15.163:0/3951493056 conn(0x7f80180ce600 msgr2=0x7f80180cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:24.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.906+0000 7f801df4f700 1 -- 172.21.15.163:0/3951493056 shutdown_connections 2023-10-09T06:31:24.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.906+0000 7f801df4f700 1 -- 172.21.15.163:0/3951493056 wait complete. 2023-10-09T06:31:24.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.907+0000 7f801df4f700 1 Processor -- start 2023-10-09T06:31:24.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.907+0000 7f801df4f700 1 -- start start 2023-10-09T06:31:24.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.908+0000 7f801df4f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f801806bcd0 0x7f80180a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:24.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.908+0000 7f801df4f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f801806c0a0 0x7f8018168860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:24.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.908+0000 7f801df4f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8018141500 0x7f801813edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:24.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.908+0000 7f801df4f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8018168da0 con 0x7f801806c0a0 2023-10-09T06:31:24.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.908+0000 7f801df4f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8018169f60 con 0x7f801806bcd0 2023-10-09T06:31:24.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.908+0000 7f801df4f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f801813f300 con 0x7f8018141500 2023-10-09T06:31:24.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.908+0000 7f8016ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f801806c0a0 0x7f8018168860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:24.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.908+0000 7f8016ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f801806c0a0 0x7f8018168860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43382/0 (socket says 172.21.15.163:43382) 2023-10-09T06:31:24.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.908+0000 7f8016ffd700 1 -- 172.21.15.163:0/2998395372 learned_addr learned my addr 172.21.15.163:0/2998395372 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:24.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.908+0000 7f8017fff700 1 --2- 172.21.15.163:0/2998395372 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8018141500 0x7f801813edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:24.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.908+0000 7f80177fe700 1 --2- 172.21.15.163:0/2998395372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f801806bcd0 0x7f80180a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:24.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.908+0000 7f8016ffd700 1 -- 172.21.15.163:0/2998395372 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8018141500 msgr2=0x7f801813edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:24.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.908+0000 7f8016ffd700 1 --2- 172.21.15.163:0/2998395372 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8018141500 0x7f801813edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.908+0000 7f8016ffd700 1 -- 172.21.15.163:0/2998395372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f801806bcd0 msgr2=0x7f80180a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:24.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.908+0000 7f8016ffd700 1 --2- 172.21.15.163:0/2998395372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f801806bcd0 0x7f80180a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:24.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.908+0000 7f8016ffd700 1 -- 172.21.15.163:0/2998395372 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800c009420 con 0x7f801806c0a0 2023-10-09T06:31:24.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.908+0000 7f8016ffd700 1 --2- 172.21.15.163:0/2998395372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f801806c0a0 0x7f8018168860 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8018069220 tx=0x7f800c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:24.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.908+0000 7f7fff7fe700 1 -- 172.21.15.163:0/2998395372 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c01a430 con 0x7f801806c0a0 2023-10-09T06:31:24.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.908+0000 7f7fff7fe700 1 -- 172.21.15.163:0/2998395372 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800c01a5b0 con 0x7f801806c0a0 2023-10-09T06:31:24.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.908+0000 7f7fff7fe700 1 -- 172.21.15.163:0/2998395372 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c01a970 con 0x7f801806c0a0 2023-10-09T06:31:24.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.908+0000 7f801df4f700 1 -- 172.21.15.163:0/2998395372 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8018169350 con 0x7f801806c0a0 2023-10-09T06:31:24.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.908+0000 7f801df4f700 1 -- 172.21.15.163:0/2998395372 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8018169910 con 0x7f801806c0a0 2023-10-09T06:31:24.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.909+0000 7f801df4f700 1 -- 172.21.15.163:0/2998395372 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f801816a0d0 con 0x7f801806c0a0 2023-10-09T06:31:24.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.910+0000 7f7fff7fe700 1 -- 172.21.15.163:0/2998395372 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f800c011420 con 0x7f801806c0a0 2023-10-09T06:31:24.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.910+0000 7f7fff7fe700 1 --2- 172.21.15.163:0/2998395372 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8004066f30 0x7f80040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:24.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.910+0000 7f7fff7fe700 1 -- 172.21.15.163:0/2998395372 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f800c086050 con 0x7f801806c0a0 2023-10-09T06:31:24.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.910+0000 7f80177fe700 1 --2- 172.21.15.163:0/2998395372 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8004066f30 0x7f80040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:24.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.911+0000 7f80177fe700 1 --2- 172.21.15.163:0/2998395372 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8004066f30 0x7f80040693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f80180cd0b0 tx=0x7f8008009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:24.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:24.914+0000 7f7fff7fe700 1 -- 172.21.15.163:0/2998395372 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f800c053610 con 0x7f801806c0a0 2023-10-09T06:31:25.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.096+0000 7f801df4f700 1 -- 172.21.15.163:0/2998395372 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f801804c3c0 con 0x7f801806c0a0 2023-10-09T06:31:25.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.096+0000 7f7fff7fe700 1 -- 172.21.15.163:0/2998395372 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f800c056c30 con 0x7f801806c0a0 2023-10-09T06:31:25.097 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:31:25.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.098+0000 7f801df4f700 1 -- 172.21.15.163:0/2998395372 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8004066f30 msgr2=0x7f80040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:25.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.098+0000 7f801df4f700 1 --2- 172.21.15.163:0/2998395372 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8004066f30 0x7f80040693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f80180cd0b0 tx=0x7f8008009040 comp rx=0 tx=0).stop 2023-10-09T06:31:25.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.098+0000 7f801df4f700 1 -- 172.21.15.163:0/2998395372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f801806c0a0 msgr2=0x7f8018168860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:25.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.098+0000 7f801df4f700 1 --2- 172.21.15.163:0/2998395372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f801806c0a0 0x7f8018168860 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8018069220 tx=0x7f800c017000 comp rx=0 tx=0).stop 2023-10-09T06:31:25.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.098+0000 7f801df4f700 1 -- 172.21.15.163:0/2998395372 shutdown_connections 2023-10-09T06:31:25.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.098+0000 7f801df4f700 1 --2- 172.21.15.163:0/2998395372 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8004066f30 0x7f80040693e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:25.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.098+0000 7f801df4f700 1 --2- 172.21.15.163:0/2998395372 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8018141500 0x7f801813edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:25.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.098+0000 7f801df4f700 1 --2- 172.21.15.163:0/2998395372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f801806bcd0 0x7f80180a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:25.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.098+0000 7f801df4f700 1 --2- 172.21.15.163:0/2998395372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f801806c0a0 0x7f8018168860 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:25.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.098+0000 7f801df4f700 1 -- 172.21.15.163:0/2998395372 >> 172.21.15.163:0/2998395372 conn(0x7f80180ce600 msgr2=0x7f801813fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:25.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.098+0000 7f801df4f700 1 -- 172.21.15.163:0/2998395372 shutdown_connections 2023-10-09T06:31:25.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.098+0000 7f801df4f700 1 -- 172.21.15.163:0/2998395372 wait complete. 2023-10-09T06:31:25.111 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-09T06:31:25.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.224+0000 7ff6b740f700 1 Processor -- start 2023-10-09T06:31:25.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.225+0000 7ff6b740f700 1 -- start start 2023-10-09T06:31:25.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.225+0000 7ff6b740f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff6b0141500 0x7ff6b00c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:25.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.225+0000 7ff6b740f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff6b006c0a0 0x7ff6b00c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:25.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.226+0000 7ff6b740f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6b015c2a0 con 0x7ff6b006bcd0 2023-10-09T06:31:25.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.226+0000 7ff6b740f700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6b00c9880 con 0x7ff6b006c0a0 2023-10-09T06:31:25.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.226+0000 7ff6b740f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6b015b690 con 0x7ff6b0141500 2023-10-09T06:31:25.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.226+0000 7ff6b59ac700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff6b006c0a0 0x7ff6b00c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:25.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.226+0000 7ff6b59ac700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff6b006c0a0 0x7ff6b00c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43394/0 (socket says 172.21.15.163:43394) 2023-10-09T06:31:25.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.226+0000 7ff6b59ac700 1 -- 172.21.15.163:0/1972892357 learned_addr learned my addr 172.21.15.163:0/1972892357 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:25.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.226+0000 7ff6b49aa700 1 --2- 172.21.15.163:0/1972892357 >> v2:172.21.15.181:3300/0 conn(0x7ff6b0141500 0x7ff6b00c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:25.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.226+0000 7ff6b59ac700 1 -- 172.21.15.163:0/1972892357 >> v2:172.21.15.181:3300/0 conn(0x7ff6b0141500 msgr2=0x7ff6b00c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:25.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.226+0000 7ff6b59ac700 1 --2- 172.21.15.163:0/1972892357 >> v2:172.21.15.181:3300/0 conn(0x7ff6b0141500 0x7ff6b00c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:25.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.226+0000 7ff6b59ac700 1 -- 172.21.15.163:0/1972892357 >> v1:172.21.15.181:6789/0 conn(0x7ff6b006bcd0 legacy=0x7ff6b0140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:25.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.226+0000 7ff6b59ac700 1 -- 172.21.15.163:0/1972892357 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b015b970 con 0x7ff6b006c0a0 2023-10-09T06:31:25.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.227+0000 7ff6b59ac700 1 --2- 172.21.15.163:0/1972892357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6b006c0a0 0x7ff6b00c9340 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff6b006c6e0 tx=0x7ff6a8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f696cc5673f8bb server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:25.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.227+0000 7ff6a7fff700 1 -- 172.21.15.163:0/1972892357 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a800e070 con 0x7ff6b006c0a0 2023-10-09T06:31:25.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.227+0000 7ff6a7fff700 1 -- 172.21.15.163:0/1972892357 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a800ba20 con 0x7ff6b006c0a0 2023-10-09T06:31:25.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.227+0000 7ff6a7fff700 1 -- 172.21.15.163:0/1972892357 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a80092e0 con 0x7ff6b006c0a0 2023-10-09T06:31:25.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.227+0000 7ff6b740f700 1 -- 172.21.15.163:0/1972892357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6b006c0a0 msgr2=0x7ff6b00c9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:25.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.227+0000 7ff6b740f700 1 --2- 172.21.15.163:0/1972892357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6b006c0a0 0x7ff6b00c9340 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff6b006c6e0 tx=0x7ff6a8009840 comp rx=0 tx=0).stop 2023-10-09T06:31:25.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.227+0000 7ff6b740f700 1 -- 172.21.15.163:0/1972892357 shutdown_connections 2023-10-09T06:31:25.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.227+0000 7ff6b740f700 1 --2- 172.21.15.163:0/1972892357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6b006c0a0 0x7ff6b00c9340 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:25.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.227+0000 7ff6b740f700 1 --2- 172.21.15.163:0/1972892357 >> v2:172.21.15.181:3300/0 conn(0x7ff6b0141500 0x7ff6b00c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:25.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.227+0000 7ff6b740f700 1 -- 172.21.15.163:0/1972892357 >> 172.21.15.163:0/1972892357 conn(0x7ff6b00ce600 msgr2=0x7ff6b00cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:25.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.227+0000 7ff6b740f700 1 -- 172.21.15.163:0/1972892357 shutdown_connections 2023-10-09T06:31:25.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.227+0000 7ff6b740f700 1 -- 172.21.15.163:0/1972892357 wait complete. 2023-10-09T06:31:25.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.228+0000 7ff6b740f700 1 Processor -- start 2023-10-09T06:31:25.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.228+0000 7ff6b740f700 1 -- start start 2023-10-09T06:31:25.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.229+0000 7ff6b740f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6b006bcd0 0x7ff6b00c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:25.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.229+0000 7ff6b740f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6b006c0a0 0x7ff6b0157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:25.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.229+0000 7ff6b740f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6b0141500 0x7ff6b01575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:25.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.229+0000 7ff6b740f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b00c8630 con 0x7ff6b0141500 2023-10-09T06:31:25.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.229+0000 7ff6b740f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b00b2220 con 0x7ff6b006c0a0 2023-10-09T06:31:25.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.229+0000 7ff6b740f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b013d470 con 0x7ff6b006bcd0 2023-10-09T06:31:25.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.229+0000 7ff6b59ac700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6b0141500 0x7ff6b01575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:25.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.229+0000 7ff6b59ac700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6b0141500 0x7ff6b01575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43410/0 (socket says 172.21.15.163:43410) 2023-10-09T06:31:25.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.229+0000 7ff6b59ac700 1 -- 172.21.15.163:0/45317180 learned_addr learned my addr 172.21.15.163:0/45317180 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:25.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.230+0000 7ff6b49aa700 1 --2- 172.21.15.163:0/45317180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6b006c0a0 0x7ff6b0157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:25.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.230+0000 7ff6b51ab700 1 --2- 172.21.15.163:0/45317180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6b006bcd0 0x7ff6b00c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:25.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.230+0000 7ff6b59ac700 1 -- 172.21.15.163:0/45317180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6b006bcd0 msgr2=0x7ff6b00c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:25.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.230+0000 7ff6b59ac700 1 --2- 172.21.15.163:0/45317180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6b006bcd0 0x7ff6b00c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:25.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.230+0000 7ff6b59ac700 1 -- 172.21.15.163:0/45317180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6b006c0a0 msgr2=0x7ff6b0157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:25.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.230+0000 7ff6b59ac700 1 --2- 172.21.15.163:0/45317180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6b006c0a0 0x7ff6b0157060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:25.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.230+0000 7ff6b59ac700 1 -- 172.21.15.163:0/45317180 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a8009ae0 con 0x7ff6b0141500 2023-10-09T06:31:25.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.230+0000 7ff6b59ac700 1 --2- 172.21.15.163:0/45317180 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6b0141500 0x7ff6b01575a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff6a80072f0 tx=0x7ff6a8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:25.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.230+0000 7ff6a57fa700 1 -- 172.21.15.163:0/45317180 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a8008c60 con 0x7ff6b0141500 2023-10-09T06:31:25.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.230+0000 7ff6a57fa700 1 -- 172.21.15.163:0/45317180 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a8008dc0 con 0x7ff6b0141500 2023-10-09T06:31:25.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.230+0000 7ff6a57fa700 1 -- 172.21.15.163:0/45317180 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a801d670 con 0x7ff6b0141500 2023-10-09T06:31:25.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.230+0000 7ff6b740f700 1 -- 172.21.15.163:0/45317180 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b013d750 con 0x7ff6b0141500 2023-10-09T06:31:25.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.230+0000 7ff6b740f700 1 -- 172.21.15.163:0/45317180 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b013dd10 con 0x7ff6b0141500 2023-10-09T06:31:25.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.231+0000 7ff6b740f700 1 -- 172.21.15.163:0/45317180 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b013df00 con 0x7ff6b0141500 2023-10-09T06:31:25.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.231+0000 7ff6a57fa700 1 -- 172.21.15.163:0/45317180 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff6a8003040 con 0x7ff6b0141500 2023-10-09T06:31:25.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.232+0000 7ff6a57fa700 1 --2- 172.21.15.163:0/45317180 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff694066f60 0x7ff694069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:25.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.232+0000 7ff6a57fa700 1 -- 172.21.15.163:0/45317180 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff6a8085e30 con 0x7ff6b0141500 2023-10-09T06:31:25.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.232+0000 7ff6b51ab700 1 --2- 172.21.15.163:0/45317180 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff694066f60 0x7ff694069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:25.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.232+0000 7ff6b51ab700 1 --2- 172.21.15.163:0/45317180 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff694066f60 0x7ff694069410 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac003860 tx=0x7ff6ac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:25.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.236+0000 7ff6a57fa700 1 -- 172.21.15.163:0/45317180 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6a80533f0 con 0x7ff6b0141500 2023-10-09T06:31:25.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.419+0000 7ff6b740f700 1 -- 172.21.15.163:0/45317180 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff6b004c3c0 con 0x7ff6b0141500 2023-10-09T06:31:25.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.419+0000 7ff6a57fa700 1 -- 172.21.15.163:0/45317180 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v27) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff6a8022060 con 0x7ff6b0141500 2023-10-09T06:31:25.420 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:25.421 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:31:25.421 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:25.421 INFO:teuthology.orchestra.run.smithi163.stdout: "poolnum": 1, 2023-10-09T06:31:25.421 INFO:teuthology.orchestra.run.smithi163.stdout: "poolname": ".mgr" 2023-10-09T06:31:25.421 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:25.421 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:31:25.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.421+0000 7ff6b740f700 1 -- 172.21.15.163:0/45317180 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff694066f60 msgr2=0x7ff694069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:25.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.421+0000 7ff6b740f700 1 --2- 172.21.15.163:0/45317180 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff694066f60 0x7ff694069410 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac003860 tx=0x7ff6ac009040 comp rx=0 tx=0).stop 2023-10-09T06:31:25.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.421+0000 7ff6b740f700 1 -- 172.21.15.163:0/45317180 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6b0141500 msgr2=0x7ff6b01575a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:25.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.421+0000 7ff6b740f700 1 --2- 172.21.15.163:0/45317180 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6b0141500 0x7ff6b01575a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff6a80072f0 tx=0x7ff6a8018000 comp rx=0 tx=0).stop 2023-10-09T06:31:25.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.421+0000 7ff6b740f700 1 -- 172.21.15.163:0/45317180 shutdown_connections 2023-10-09T06:31:25.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.421+0000 7ff6b740f700 1 --2- 172.21.15.163:0/45317180 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff694066f60 0x7ff694069410 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:25.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.421+0000 7ff6b740f700 1 --2- 172.21.15.163:0/45317180 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6b0141500 0x7ff6b01575a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:25.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.421+0000 7ff6b740f700 1 --2- 172.21.15.163:0/45317180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6b006bcd0 0x7ff6b00c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:25.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.421+0000 7ff6b740f700 1 --2- 172.21.15.163:0/45317180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6b006c0a0 0x7ff6b0157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:25.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.421+0000 7ff6b740f700 1 -- 172.21.15.163:0/45317180 >> 172.21.15.163:0/45317180 conn(0x7ff6b00ce600 msgr2=0x7ff6b006cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:25.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.422+0000 7ff6b740f700 1 -- 172.21.15.163:0/45317180 shutdown_connections 2023-10-09T06:31:25.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.422+0000 7ff6b740f700 1 -- 172.21.15.163:0/45317180 wait complete. 2023-10-09T06:31:25.434 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-09T06:31:25.435 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-09T06:31:25.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.558+0000 7f67f0b84700 1 Processor -- start 2023-10-09T06:31:25.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.558+0000 7f67f0b84700 1 -- start start 2023-10-09T06:31:25.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.558+0000 7f67f0b84700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f67ec155360 0x7f67ec0b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:25.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.558+0000 7f67f0b84700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f67ec0b40f0 0x7f67ec0ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:25.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.558+0000 7f67f0b84700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67ec0ceb60 0x7f67ec06d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:25.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.558+0000 7f67f0b84700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67ec0b4590 con 0x7f67ec155360 2023-10-09T06:31:25.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.558+0000 7f67f0b84700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67ec15c320 con 0x7f67ec0b40f0 2023-10-09T06:31:25.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.558+0000 7f67f0b84700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec0cefc0 con 0x7f67ec0ceb60 2023-10-09T06:31:25.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.559+0000 7f67ea59c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f67ec155360 0x7f67ec0b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:25.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.559+0000 7f67ea59c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f67ec155360 0x7f67ec0b3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43424/0 (socket says 172.21.15.163:43424) 2023-10-09T06:31:25.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.559+0000 7f67ea59c700 1 -- 172.21.15.163:0/291443092 learned_addr learned my addr 172.21.15.163:0/291443092 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:25.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.559+0000 7f67e1d9b700 1 --2- 172.21.15.163:0/291443092 >> v2:172.21.15.181:3300/0 conn(0x7f67ec0b40f0 0x7f67ec0ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:25.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.559+0000 7f67ead9d700 1 --2- 172.21.15.163:0/291443092 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67ec0ceb60 0x7f67ec06d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:25.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.559+0000 7f67ea59c700 1 -- 172.21.15.163:0/291443092 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67ec0ceb60 msgr2=0x7f67ec06d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:25.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.559+0000 7f67ea59c700 1 --2- 172.21.15.163:0/291443092 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67ec0ceb60 0x7f67ec06d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:25.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.559+0000 7f67ea59c700 1 -- 172.21.15.163:0/291443092 >> v2:172.21.15.181:3300/0 conn(0x7f67ec0b40f0 msgr2=0x7f67ec0ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:25.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.559+0000 7f67ea59c700 1 --2- 172.21.15.163:0/291443092 >> v2:172.21.15.181:3300/0 conn(0x7f67ec0b40f0 0x7f67ec0ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:25.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.559+0000 7f67ea59c700 1 -- 172.21.15.163:0/291443092 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ec15b710 con 0x7f67ec155360 2023-10-09T06:31:25.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.559+0000 7f67ea59c700 1 --2- 172.21.15.163:0/291443092 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67ec155360 0x7f67ec0b3bb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f67ec070f90 tx=0x7f67dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa577a7300f4fa29 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:25.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.559+0000 7f67e9d9b700 1 -- 172.21.15.163:0/291443092 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc00b640 con 0x7f67ec155360 2023-10-09T06:31:25.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.559+0000 7f67e9d9b700 1 -- 172.21.15.163:0/291443092 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67dc00b7a0 con 0x7f67ec155360 2023-10-09T06:31:25.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.559+0000 7f67e9d9b700 1 -- 172.21.15.163:0/291443092 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc00bb60 con 0x7f67ec155360 2023-10-09T06:31:25.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.560+0000 7f67f0b84700 1 -- 172.21.15.163:0/291443092 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67ec155360 msgr2=0x7f67ec0b3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:25.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.560+0000 7f67f0b84700 1 --2- 172.21.15.163:0/291443092 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67ec155360 0x7f67ec0b3bb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f67ec070f90 tx=0x7f67dc009250 comp rx=0 tx=0).stop 2023-10-09T06:31:25.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.560+0000 7f67f0b84700 1 -- 172.21.15.163:0/291443092 shutdown_connections 2023-10-09T06:31:25.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.560+0000 7f67f0b84700 1 --2- 172.21.15.163:0/291443092 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67ec0ceb60 0x7f67ec06d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:25.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.560+0000 7f67f0b84700 1 --2- 172.21.15.163:0/291443092 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67ec155360 0x7f67ec0b3bb0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:25.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.560+0000 7f67f0b84700 1 --2- 172.21.15.163:0/291443092 >> v2:172.21.15.181:3300/0 conn(0x7f67ec0b40f0 0x7f67ec0ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:25.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.560+0000 7f67f0b84700 1 -- 172.21.15.163:0/291443092 >> 172.21.15.163:0/291443092 conn(0x7f67ec06baf0 msgr2=0x7f67ec06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:25.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.560+0000 7f67f0b84700 1 -- 172.21.15.163:0/291443092 shutdown_connections 2023-10-09T06:31:25.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.560+0000 7f67f0b84700 1 -- 172.21.15.163:0/291443092 wait complete. 2023-10-09T06:31:25.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.561+0000 7f67f0b84700 1 Processor -- start 2023-10-09T06:31:25.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.561+0000 7f67f0b84700 1 -- start start 2023-10-09T06:31:25.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.562+0000 7f67f0b84700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67ec0b40f0 0x7f67ec159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:25.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.562+0000 7f67f0b84700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67ec0ceb60 0x7f67ec156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:25.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.562+0000 7f67f0b84700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67ec155360 0x7f67ec14a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:25.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.562+0000 7f67f0b84700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec156890 con 0x7f67ec155360 2023-10-09T06:31:25.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.562+0000 7f67f0b84700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec151be0 con 0x7f67ec0ceb60 2023-10-09T06:31:25.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.562+0000 7f67f0b84700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec156b10 con 0x7f67ec0b40f0 2023-10-09T06:31:25.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.562+0000 7f67ead9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67ec155360 0x7f67ec14a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:25.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.562+0000 7f67ead9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67ec155360 0x7f67ec14a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43428/0 (socket says 172.21.15.163:43428) 2023-10-09T06:31:25.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.562+0000 7f67ead9d700 1 -- 172.21.15.163:0/3455961594 learned_addr learned my addr 172.21.15.163:0/3455961594 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:25.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.562+0000 7f67ea59c700 1 --2- 172.21.15.163:0/3455961594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67ec0b40f0 0x7f67ec159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:25.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.562+0000 7f67e1d9b700 1 --2- 172.21.15.163:0/3455961594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67ec0ceb60 0x7f67ec156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:25.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.562+0000 7f67ead9d700 1 -- 172.21.15.163:0/3455961594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67ec0b40f0 msgr2=0x7f67ec159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:25.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.562+0000 7f67ead9d700 1 --2- 172.21.15.163:0/3455961594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67ec0b40f0 0x7f67ec159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:25.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.562+0000 7f67ead9d700 1 -- 172.21.15.163:0/3455961594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67ec0ceb60 msgr2=0x7f67ec156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:25.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.562+0000 7f67ead9d700 1 --2- 172.21.15.163:0/3455961594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67ec0ceb60 0x7f67ec156350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:25.569 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.562+0000 7f67ead9d700 1 -- 172.21.15.163:0/3455961594 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67dc0094f0 con 0x7f67ec155360 2023-10-09T06:31:25.569 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.562+0000 7f67ead9d700 1 --2- 172.21.15.163:0/3455961594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67ec155360 0x7f67ec14a380 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f67e40124f0 tx=0x7f67e400f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:25.569 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.562+0000 7f67e2ffd700 1 -- 172.21.15.163:0/3455961594 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e400d0d0 con 0x7f67ec155360 2023-10-09T06:31:25.569 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.562+0000 7f67e2ffd700 1 -- 172.21.15.163:0/3455961594 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e4013040 con 0x7f67ec155360 2023-10-09T06:31:25.569 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.563+0000 7f67f0b84700 1 -- 172.21.15.163:0/3455961594 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67ec156df0 con 0x7f67ec155360 2023-10-09T06:31:25.569 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.563+0000 7f67e2ffd700 1 -- 172.21.15.163:0/3455961594 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e400d4a0 con 0x7f67ec155360 2023-10-09T06:31:25.569 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.563+0000 7f67f0b84700 1 -- 172.21.15.163:0/3455961594 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67ec1573b0 con 0x7f67ec155360 2023-10-09T06:31:25.569 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.563+0000 7f67f0b84700 1 -- 172.21.15.163:0/3455961594 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67ec151de0 con 0x7f67ec155360 2023-10-09T06:31:25.569 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.564+0000 7f67e2ffd700 1 -- 172.21.15.163:0/3455961594 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f67e400b960 con 0x7f67ec155360 2023-10-09T06:31:25.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.564+0000 7f67e2ffd700 1 --2- 172.21.15.163:0/3455961594 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f67cc066f30 0x7f67cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:25.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.564+0000 7f67e2ffd700 1 -- 172.21.15.163:0/3455961594 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f67e4088a90 con 0x7f67ec155360 2023-10-09T06:31:25.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.565+0000 7f67ea59c700 1 --2- 172.21.15.163:0/3455961594 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f67cc066f30 0x7f67cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:25.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.565+0000 7f67ea59c700 1 --2- 172.21.15.163:0/3455961594 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f67cc066f30 0x7f67cc0693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f67ec070f90 tx=0x7f67dc017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:25.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.569+0000 7f67e2ffd700 1 -- 172.21.15.163:0/3455961594 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f67ec151de0 con 0x7f67ec155360 2023-10-09T06:31:25.754 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:25.752+0000 7f67f0b84700 1 -- 172.21.15.163:0/3455961594 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f67ec151da0 con 0x7f67ec155360 2023-10-09T06:31:26.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.625+0000 7f67e2ffd700 1 -- 172.21.15.163:0/3455961594 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v28) v1 ==== 138+0+0 (secure 0 0 0) 0x7f67e4056100 con 0x7f67ec155360 2023-10-09T06:31:26.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.630+0000 7f67f0b84700 1 -- 172.21.15.163:0/3455961594 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f67cc066f30 msgr2=0x7f67cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:26.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.630+0000 7f67f0b84700 1 --2- 172.21.15.163:0/3455961594 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f67cc066f30 0x7f67cc0693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f67ec070f90 tx=0x7f67dc017000 comp rx=0 tx=0).stop 2023-10-09T06:31:26.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.630+0000 7f67f0b84700 1 -- 172.21.15.163:0/3455961594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67ec155360 msgr2=0x7f67ec14a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:26.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.630+0000 7f67f0b84700 1 --2- 172.21.15.163:0/3455961594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67ec155360 0x7f67ec14a380 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f67e40124f0 tx=0x7f67e400f040 comp rx=0 tx=0).stop 2023-10-09T06:31:26.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.630+0000 7f67f0b84700 1 -- 172.21.15.163:0/3455961594 shutdown_connections 2023-10-09T06:31:26.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.630+0000 7f67f0b84700 1 --2- 172.21.15.163:0/3455961594 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f67cc066f30 0x7f67cc0693e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:26.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.630+0000 7f67f0b84700 1 --2- 172.21.15.163:0/3455961594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67ec155360 0x7f67ec14a380 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:26.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.630+0000 7f67f0b84700 1 --2- 172.21.15.163:0/3455961594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67ec0b40f0 0x7f67ec159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:26.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.630+0000 7f67f0b84700 1 --2- 172.21.15.163:0/3455961594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67ec0ceb60 0x7f67ec156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:26.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.630+0000 7f67f0b84700 1 -- 172.21.15.163:0/3455961594 >> 172.21.15.163:0/3455961594 conn(0x7f67ec06baf0 msgr2=0x7f67ec0b0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:26.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.630+0000 7f67f0b84700 1 -- 172.21.15.163:0/3455961594 shutdown_connections 2023-10-09T06:31:26.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.630+0000 7f67f0b84700 1 -- 172.21.15.163:0/3455961594 wait complete. 2023-10-09T06:31:26.632 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_metadata' created 2023-10-09T06:31:26.645 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-09T06:31:26.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.768+0000 7feeb3283700 1 Processor -- start 2023-10-09T06:31:26.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.768+0000 7feeb3283700 1 -- start start 2023-10-09T06:31:26.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.769+0000 7feeb3283700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7feeac0cf480 0x7feeac15d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:26.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.769+0000 7feeb3283700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feeac15d760 0x7feeac0b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:26.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.769+0000 7feeb3283700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeac0b10f0 con 0x7feeac1595d0 2023-10-09T06:31:26.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.769+0000 7feeb3283700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeac15db80 con 0x7feeac0cf480 2023-10-09T06:31:26.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.769+0000 7feeb3283700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeac15e000 con 0x7feeac15d760 2023-10-09T06:31:26.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.769+0000 7feeb1a80700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7feeac0cf480 0x7feeac15d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:26.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.769+0000 7feeb1a80700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7feeac0cf480 0x7feeac15d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43442/0 (socket says 172.21.15.163:43442) 2023-10-09T06:31:26.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.769+0000 7feeb2281700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7feeac1595d0 0x7feeac0ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:38818/0 (socket says 172.21.15.163:38818) 2023-10-09T06:31:26.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.769+0000 7feeb1a80700 1 -- 172.21.15.163:0/3574029726 learned_addr learned my addr 172.21.15.163:0/3574029726 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:26.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.769+0000 7feeb1a80700 1 -- 172.21.15.163:0/3574029726 >> v2:172.21.15.181:3300/0 conn(0x7feeac15d760 msgr2=0x7feeac0b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:26.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.769+0000 7feeb2a82700 1 --2- 172.21.15.163:0/3574029726 >> v2:172.21.15.181:3300/0 conn(0x7feeac15d760 0x7feeac0b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:26.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.769+0000 7feeb127f700 1 -- 172.21.15.163:0/3574029726 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2320742605 0 0) 0x7feeac0b10f0 con 0x7feeac1595d0 2023-10-09T06:31:26.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.769+0000 7feeb1a80700 1 --2- 172.21.15.163:0/3574029726 >> v2:172.21.15.181:3300/0 conn(0x7feeac15d760 0x7feeac0b1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:26.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.769+0000 7feeb1a80700 1 -- 172.21.15.163:0/3574029726 >> v1:172.21.15.163:6789/0 conn(0x7feeac1595d0 legacy=0x7feeac0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:26.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.769+0000 7feeb1a80700 1 -- 172.21.15.163:0/3574029726 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeac0cf990 con 0x7feeac0cf480 2023-10-09T06:31:26.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.770+0000 7feeb1a80700 1 --2- 172.21.15.163:0/3574029726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feeac0cf480 0x7feeac15d220 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7feeac0b3110 tx=0x7fee9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bfdba320da018e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:26.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.770+0000 7feeb127f700 1 -- 172.21.15.163:0/3574029726 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c00b860 con 0x7feeac0cf480 2023-10-09T06:31:26.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.770+0000 7feeb127f700 1 -- 172.21.15.163:0/3574029726 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9c00b9c0 con 0x7feeac0cf480 2023-10-09T06:31:26.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.770+0000 7feeb127f700 1 -- 172.21.15.163:0/3574029726 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c00bd60 con 0x7feeac0cf480 2023-10-09T06:31:26.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.770+0000 7feeb3283700 1 -- 172.21.15.163:0/3574029726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feeac0cf480 msgr2=0x7feeac15d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:26.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.770+0000 7feeb3283700 1 --2- 172.21.15.163:0/3574029726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feeac0cf480 0x7feeac15d220 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7feeac0b3110 tx=0x7fee9c009250 comp rx=0 tx=0).stop 2023-10-09T06:31:26.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.770+0000 7feeb3283700 1 -- 172.21.15.163:0/3574029726 shutdown_connections 2023-10-09T06:31:26.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.770+0000 7feeb3283700 1 --2- 172.21.15.163:0/3574029726 >> v2:172.21.15.181:3300/0 conn(0x7feeac15d760 0x7feeac0b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:26.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.770+0000 7feeb3283700 1 --2- 172.21.15.163:0/3574029726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feeac0cf480 0x7feeac15d220 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:26.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.770+0000 7feeb3283700 1 -- 172.21.15.163:0/3574029726 >> 172.21.15.163:0/3574029726 conn(0x7feeac0d05f0 msgr2=0x7feeac0d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:26.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.770+0000 7feeb3283700 1 -- 172.21.15.163:0/3574029726 shutdown_connections 2023-10-09T06:31:26.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.770+0000 7feeb3283700 1 -- 172.21.15.163:0/3574029726 wait complete. 2023-10-09T06:31:26.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.771+0000 7feeb3283700 1 Processor -- start 2023-10-09T06:31:26.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.772+0000 7feeb3283700 1 -- start start 2023-10-09T06:31:26.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.772+0000 7feeb3283700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feeac0cf480 0x7feeac16a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:26.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.772+0000 7feeb3283700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeac1595d0 0x7feeac0c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:26.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.772+0000 7feeb3283700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeac15d760 0x7feeac068d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:26.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.772+0000 7feeb3283700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac16aad0 con 0x7feeac0cf480 2023-10-09T06:31:26.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.772+0000 7feeb3283700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac157190 con 0x7feeac15d760 2023-10-09T06:31:26.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.772+0000 7feeb3283700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeac156bd0 con 0x7feeac1595d0 2023-10-09T06:31:26.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.772+0000 7feeb2281700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feeac0cf480 0x7feeac16a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:26.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.772+0000 7feeb2281700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feeac0cf480 0x7feeac16a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43458/0 (socket says 172.21.15.163:43458) 2023-10-09T06:31:26.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.772+0000 7feeb2281700 1 -- 172.21.15.163:0/2967909069 learned_addr learned my addr 172.21.15.163:0/2967909069 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:26.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.772+0000 7feeb1a80700 1 --2- 172.21.15.163:0/2967909069 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeac1595d0 0x7feeac0c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:26.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.772+0000 7feeb2a82700 1 --2- 172.21.15.163:0/2967909069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeac15d760 0x7feeac068d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:26.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.773+0000 7feeb2281700 1 -- 172.21.15.163:0/2967909069 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeac1595d0 msgr2=0x7feeac0c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:26.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.773+0000 7feeb2281700 1 --2- 172.21.15.163:0/2967909069 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeac1595d0 0x7feeac0c5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:26.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.773+0000 7feeb2281700 1 -- 172.21.15.163:0/2967909069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeac15d760 msgr2=0x7feeac068d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:26.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.773+0000 7feeb2281700 1 --2- 172.21.15.163:0/2967909069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeac15d760 0x7feeac068d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:26.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.773+0000 7feeb2281700 1 -- 172.21.15.163:0/2967909069 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9c0094f0 con 0x7feeac0cf480 2023-10-09T06:31:26.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.773+0000 7feeb2281700 1 --2- 172.21.15.163:0/2967909069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feeac0cf480 0x7feeac16a590 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feea800c810 tx=0x7feea8007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:26.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.773+0000 7feea27fc700 1 -- 172.21.15.163:0/2967909069 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea8007300 con 0x7feeac0cf480 2023-10-09T06:31:26.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.773+0000 7feea27fc700 1 -- 172.21.15.163:0/2967909069 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea8007460 con 0x7feeac0cf480 2023-10-09T06:31:26.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.773+0000 7feeb3283700 1 -- 172.21.15.163:0/2967909069 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeac152ab0 con 0x7feeac0cf480 2023-10-09T06:31:26.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.773+0000 7feea27fc700 1 -- 172.21.15.163:0/2967909069 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea80057f0 con 0x7feeac0cf480 2023-10-09T06:31:26.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.773+0000 7feeb3283700 1 -- 172.21.15.163:0/2967909069 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeac0cca90 con 0x7feeac0cf480 2023-10-09T06:31:26.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.774+0000 7feeb3283700 1 -- 172.21.15.163:0/2967909069 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeac04c3c0 con 0x7feeac0cf480 2023-10-09T06:31:26.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.774+0000 7feea27fc700 1 -- 172.21.15.163:0/2967909069 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feea8005bc0 con 0x7feeac0cf480 2023-10-09T06:31:26.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.775+0000 7feea27fc700 1 --2- 172.21.15.163:0/2967909069 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fee90066f30 0x7fee900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:26.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.775+0000 7feea27fc700 1 -- 172.21.15.163:0/2967909069 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5223+0+0 (secure 0 0 0) 0x7feea8083ed0 con 0x7feeac0cf480 2023-10-09T06:31:26.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.775+0000 7feeb1a80700 1 --2- 172.21.15.163:0/2967909069 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fee90066f30 0x7fee900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:26.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.775+0000 7feeb1a80700 1 --2- 172.21.15.163:0/2967909069 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fee90066f30 0x7fee900693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7feeac0b3110 tx=0x7fee9c017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:26.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.779+0000 7feea27fc700 1 -- 172.21.15.163:0/2967909069 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feea80513a0 con 0x7feeac0cf480 2023-10-09T06:31:26.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:26.962+0000 7feeb3283700 1 -- 172.21.15.163:0/2967909069 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7feeac0b3860 con 0x7feeac0cf480 2023-10-09T06:31:27.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.628+0000 7feea27fc700 1 -- 172.21.15.163:0/2967909069 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v29) v1 ==== 170+0+0 (secure 0 0 0) 0x7feea80549c0 con 0x7feeac0cf480 2023-10-09T06:31:27.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.632+0000 7feeb3283700 1 -- 172.21.15.163:0/2967909069 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fee90066f30 msgr2=0x7fee900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:27.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.632+0000 7feeb3283700 1 --2- 172.21.15.163:0/2967909069 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fee90066f30 0x7fee900693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7feeac0b3110 tx=0x7fee9c017000 comp rx=0 tx=0).stop 2023-10-09T06:31:27.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.632+0000 7feeb3283700 1 -- 172.21.15.163:0/2967909069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feeac0cf480 msgr2=0x7feeac16a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:27.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.632+0000 7feeb3283700 1 --2- 172.21.15.163:0/2967909069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feeac0cf480 0x7feeac16a590 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feea800c810 tx=0x7feea8007d20 comp rx=0 tx=0).stop 2023-10-09T06:31:27.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.632+0000 7feeb3283700 1 -- 172.21.15.163:0/2967909069 shutdown_connections 2023-10-09T06:31:27.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.632+0000 7feeb3283700 1 --2- 172.21.15.163:0/2967909069 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fee90066f30 0x7fee900693e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:27.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.632+0000 7feeb3283700 1 --2- 172.21.15.163:0/2967909069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feeac0cf480 0x7feeac16a590 secure :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feea800c810 tx=0x7feea8007d20 comp rx=0 tx=0).stop 2023-10-09T06:31:27.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.632+0000 7feeb3283700 1 --2- 172.21.15.163:0/2967909069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeac15d760 0x7feeac068d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:27.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.632+0000 7feeb3283700 1 --2- 172.21.15.163:0/2967909069 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeac1595d0 0x7feeac0c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:27.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.632+0000 7feeb3283700 1 -- 172.21.15.163:0/2967909069 >> 172.21.15.163:0/2967909069 conn(0x7feeac0d05f0 msgr2=0x7feeac0af190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:27.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.637+0000 7feeb3283700 1 -- 172.21.15.163:0/2967909069 shutdown_connections 2023-10-09T06:31:27.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.637+0000 7feeb3283700 1 -- 172.21.15.163:0/2967909069 wait complete. 2023-10-09T06:31:27.644 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_data' created 2023-10-09T06:31:27.656 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-09T06:31:27.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.773+0000 7f2412482700 1 Processor -- start 2023-10-09T06:31:27.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.773+0000 7f2412482700 1 -- start start 2023-10-09T06:31:27.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.774+0000 7f2412482700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f240c159890 0x7f240c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:27.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.774+0000 7f2412482700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f240c069560 0x7f240c0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:27.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.774+0000 7f2412482700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f240c15a160 con 0x7f240c159c60 2023-10-09T06:31:27.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.774+0000 7f2412482700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f240c069f20 con 0x7f240c069560 2023-10-09T06:31:27.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.774+0000 7f2412482700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f240c0d13a0 con 0x7f240c159890 2023-10-09T06:31:27.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.774+0000 7f2410a1f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f240c069560 0x7f240c0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:27.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.774+0000 7f2410a1f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f240c069560 0x7f240c0699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43470/0 (socket says 172.21.15.163:43470) 2023-10-09T06:31:27.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.774+0000 7f2410a1f700 1 -- 172.21.15.163:0/1964828546 learned_addr learned my addr 172.21.15.163:0/1964828546 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:27.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.774+0000 7f240bfff700 1 --2- 172.21.15.163:0/1964828546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f240c159890 0x7f240c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:27.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.775+0000 7f2410a1f700 1 -- 172.21.15.163:0/1964828546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f240c159890 msgr2=0x7f240c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:27.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.775+0000 7f2410a1f700 1 --2- 172.21.15.163:0/1964828546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f240c159890 0x7f240c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:27.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.775+0000 7f2410a1f700 1 -- 172.21.15.163:0/1964828546 >> v1:172.21.15.181:6789/0 conn(0x7f240c159c60 legacy=0x7f240c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:27.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.775+0000 7f2410a1f700 1 -- 172.21.15.163:0/1964828546 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f240c15e140 con 0x7f240c069560 2023-10-09T06:31:27.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.775+0000 7f2410a1f700 1 --2- 172.21.15.163:0/1964828546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f240c069560 0x7f240c0699e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2404009f20 tx=0x7f2404009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f516dc8d8cd108 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:27.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.775+0000 7f240b7fe700 1 -- 172.21.15.163:0/1964828546 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240400e070 con 0x7f240c069560 2023-10-09T06:31:27.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.775+0000 7f240b7fe700 1 -- 172.21.15.163:0/1964828546 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f240400b9d0 con 0x7f240c069560 2023-10-09T06:31:27.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.775+0000 7f240b7fe700 1 -- 172.21.15.163:0/1964828546 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24040092e0 con 0x7f240c069560 2023-10-09T06:31:27.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.775+0000 7f2412482700 1 -- 172.21.15.163:0/1964828546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f240c069560 msgr2=0x7f240c0699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:27.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.775+0000 7f2412482700 1 --2- 172.21.15.163:0/1964828546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f240c069560 0x7f240c0699e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2404009f20 tx=0x7f2404009840 comp rx=0 tx=0).stop 2023-10-09T06:31:27.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.775+0000 7f2412482700 1 -- 172.21.15.163:0/1964828546 shutdown_connections 2023-10-09T06:31:27.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.775+0000 7f2412482700 1 --2- 172.21.15.163:0/1964828546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f240c069560 0x7f240c0699e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:27.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.775+0000 7f2412482700 1 --2- 172.21.15.163:0/1964828546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f240c159890 0x7f240c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:27.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.775+0000 7f2412482700 1 -- 172.21.15.163:0/1964828546 >> 172.21.15.163:0/1964828546 conn(0x7f240c068e20 msgr2=0x7f240c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:27.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.776+0000 7f2412482700 1 -- 172.21.15.163:0/1964828546 shutdown_connections 2023-10-09T06:31:27.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.776+0000 7f2412482700 1 -- 172.21.15.163:0/1964828546 wait complete. 2023-10-09T06:31:27.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.777+0000 7f2412482700 1 Processor -- start 2023-10-09T06:31:27.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.777+0000 7f2412482700 1 -- start start 2023-10-09T06:31:27.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.777+0000 7f2412482700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f240c069560 0x7f240c0cf780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:27.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.777+0000 7f2412482700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f240c159890 0x7f240c0cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:27.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.777+0000 7f2412482700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f240c159c60 0x7f240c14f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:27.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.777+0000 7f2412482700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240c0ce2d0 con 0x7f240c159c60 2023-10-09T06:31:27.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.777+0000 7f2412482700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240c1513c0 con 0x7f240c159890 2023-10-09T06:31:27.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.777+0000 7f2412482700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f240c0ce050 con 0x7f240c069560 2023-10-09T06:31:27.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.777+0000 7f2410a1f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f240c159c60 0x7f240c14f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:27.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.777+0000 7f2410a1f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f240c159c60 0x7f240c14f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:43476/0 (socket says 172.21.15.163:43476) 2023-10-09T06:31:27.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.777+0000 7f2410a1f700 1 -- 172.21.15.163:0/2677203700 learned_addr learned my addr 172.21.15.163:0/2677203700 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:27.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.777+0000 7f240bfff700 1 --2- 172.21.15.163:0/2677203700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f240c069560 0x7f240c0cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:27.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.777+0000 7f2403fff700 1 --2- 172.21.15.163:0/2677203700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f240c159890 0x7f240c0cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:27.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.778+0000 7f2410a1f700 1 -- 172.21.15.163:0/2677203700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f240c069560 msgr2=0x7f240c0cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:27.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.778+0000 7f2410a1f700 1 --2- 172.21.15.163:0/2677203700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f240c069560 0x7f240c0cf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:27.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.778+0000 7f2410a1f700 1 -- 172.21.15.163:0/2677203700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f240c159890 msgr2=0x7f240c0cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:27.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.778+0000 7f2410a1f700 1 --2- 172.21.15.163:0/2677203700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f240c159890 0x7f240c0cdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:27.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.778+0000 7f2410a1f700 1 -- 172.21.15.163:0/2677203700 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2404009ae0 con 0x7f240c159c60 2023-10-09T06:31:27.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.778+0000 7f2410a1f700 1 --2- 172.21.15.163:0/2677203700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f240c159c60 0x7f240c14f8d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2404009430 tx=0x7f2404018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:27.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.778+0000 7f2408ff9700 1 -- 172.21.15.163:0/2677203700 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2404008b00 con 0x7f240c159c60 2023-10-09T06:31:27.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.778+0000 7f2408ff9700 1 -- 172.21.15.163:0/2677203700 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2404008c60 con 0x7f240c159c60 2023-10-09T06:31:27.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.778+0000 7f2408ff9700 1 -- 172.21.15.163:0/2677203700 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2404025460 con 0x7f240c159c60 2023-10-09T06:31:27.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.778+0000 7f2412482700 1 -- 172.21.15.163:0/2677203700 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f240c0c6700 con 0x7f240c159c60 2023-10-09T06:31:27.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.778+0000 7f2412482700 1 -- 172.21.15.163:0/2677203700 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f240c0c6420 con 0x7f240c159c60 2023-10-09T06:31:27.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.779+0000 7f2412482700 1 -- 172.21.15.163:0/2677203700 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f240c151570 con 0x7f240c159c60 2023-10-09T06:31:27.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.779+0000 7f2408ff9700 1 -- 172.21.15.163:0/2677203700 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2404003040 con 0x7f240c159c60 2023-10-09T06:31:27.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.780+0000 7f2408ff9700 1 --2- 172.21.15.163:0/2677203700 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f23ec066f30 0x7f23ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:27.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.780+0000 7f2408ff9700 1 -- 172.21.15.163:0/2677203700 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f2404085990 con 0x7f240c159c60 2023-10-09T06:31:27.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.780+0000 7f240bfff700 1 --2- 172.21.15.163:0/2677203700 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f23ec066f30 0x7f23ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:27.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.780+0000 7f240bfff700 1 --2- 172.21.15.163:0/2677203700 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f23ec066f30 0x7f23ec0693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f240c16cdf0 tx=0x7f23fc0091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:27.785 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:27.784+0000 7f2408ff9700 1 -- 172.21.15.163:0/2677203700 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f240c151570 con 0x7f240c159c60 2023-10-09T06:31:28.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.009+0000 7f2412482700 1 -- 172.21.15.163:0/2677203700 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f240c151570 con 0x7f240c159c60 2023-10-09T06:31:28.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.640+0000 7f2408ff9700 1 -- 172.21.15.163:0/2677203700 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f240c151570 con 0x7f240c159c60 2023-10-09T06:31:28.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.642+0000 7f2412482700 1 -- 172.21.15.163:0/2677203700 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f23ec066f30 msgr2=0x7f23ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:28.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.642+0000 7f2412482700 1 --2- 172.21.15.163:0/2677203700 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f23ec066f30 0x7f23ec0693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f240c16cdf0 tx=0x7f23fc0091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:28.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.642+0000 7f2412482700 1 -- 172.21.15.163:0/2677203700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f240c159c60 msgr2=0x7f240c14f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:28.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.642+0000 7f2412482700 1 --2- 172.21.15.163:0/2677203700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f240c159c60 0x7f240c14f8d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2404009430 tx=0x7f2404018000 comp rx=0 tx=0).stop 2023-10-09T06:31:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.642+0000 7f2412482700 1 -- 172.21.15.163:0/2677203700 shutdown_connections 2023-10-09T06:31:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.642+0000 7f2412482700 1 --2- 172.21.15.163:0/2677203700 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f23ec066f30 0x7f23ec0693e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.642+0000 7f2412482700 1 --2- 172.21.15.163:0/2677203700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f240c159c60 0x7f240c14f8d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.642+0000 7f2412482700 1 --2- 172.21.15.163:0/2677203700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f240c069560 0x7f240c0cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.642+0000 7f2412482700 1 --2- 172.21.15.163:0/2677203700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f240c159890 0x7f240c0cdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.642+0000 7f2412482700 1 -- 172.21.15.163:0/2677203700 >> 172.21.15.163:0/2677203700 conn(0x7f240c068e20 msgr2=0x7f240c158400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.642+0000 7f2412482700 1 -- 172.21.15.163:0/2677203700 shutdown_connections 2023-10-09T06:31:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.642+0000 7f2412482700 1 -- 172.21.15.163:0/2677203700 wait complete. 2023-10-09T06:31:28.647 INFO:teuthology.orchestra.run.smithi163.stderr:new fs with metadata pool 4 and data pool 5 2023-10-09T06:31:28.659 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:31:28.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.775+0000 7f2426aa1700 1 Processor -- start 2023-10-09T06:31:28.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.775+0000 7f2426aa1700 1 -- start start 2023-10-09T06:31:28.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.776+0000 7f2426aa1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2420159890 0x7f242015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:28.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.776+0000 7f2426aa1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2420069560 0x7f24200699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:28.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.776+0000 7f2426aa1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2420069ee0 con 0x7f2420159c60 2023-10-09T06:31:28.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.776+0000 7f2426aa1700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f242015a160 con 0x7f2420159890 2023-10-09T06:31:28.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.776+0000 7f2426aa1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24200d13a0 con 0x7f2420069560 2023-10-09T06:31:28.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.776+0000 7f242483d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2420159890 0x7f242015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:28.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.776+0000 7f242483d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2420159890 0x7f242015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33364/0 (socket says 172.21.15.163:33364) 2023-10-09T06:31:28.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.776+0000 7f242483d700 1 -- 172.21.15.163:0/2928401340 learned_addr learned my addr 172.21.15.163:0/2928401340 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:28.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.776+0000 7f242503e700 1 --2- 172.21.15.163:0/2928401340 >> v2:172.21.15.181:3300/0 conn(0x7f2420069560 0x7f24200699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:28.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.776+0000 7f242483d700 1 -- 172.21.15.163:0/2928401340 >> v2:172.21.15.181:3300/0 conn(0x7f2420069560 msgr2=0x7f24200699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:28.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.776+0000 7f242483d700 1 --2- 172.21.15.163:0/2928401340 >> v2:172.21.15.181:3300/0 conn(0x7f2420069560 0x7f24200699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:28.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.776+0000 7f242483d700 1 -- 172.21.15.163:0/2928401340 >> v1:172.21.15.181:6789/0 conn(0x7f2420159c60 legacy=0x7f242015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:28.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.776+0000 7f242483d700 1 -- 172.21.15.163:0/2928401340 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242015e140 con 0x7f2420159890 2023-10-09T06:31:28.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.777+0000 7f242483d700 1 --2- 172.21.15.163:0/2928401340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2420159890 0x7f242015abd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f242006a890 tx=0x7f241c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0e90878e2bd13c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:28.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.777+0000 7f24177fe700 1 -- 172.21.15.163:0/2928401340 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c00b820 con 0x7f2420159890 2023-10-09T06:31:28.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.777+0000 7f24177fe700 1 -- 172.21.15.163:0/2928401340 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241c00b980 con 0x7f2420159890 2023-10-09T06:31:28.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.777+0000 7f24177fe700 1 -- 172.21.15.163:0/2928401340 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c00bd20 con 0x7f2420159890 2023-10-09T06:31:28.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.777+0000 7f2426aa1700 1 -- 172.21.15.163:0/2928401340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2420159890 msgr2=0x7f242015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:28.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.777+0000 7f2426aa1700 1 --2- 172.21.15.163:0/2928401340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2420159890 0x7f242015abd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f242006a890 tx=0x7f241c009250 comp rx=0 tx=0).stop 2023-10-09T06:31:28.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.777+0000 7f2426aa1700 1 -- 172.21.15.163:0/2928401340 shutdown_connections 2023-10-09T06:31:28.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.777+0000 7f2426aa1700 1 --2- 172.21.15.163:0/2928401340 >> v2:172.21.15.181:3300/0 conn(0x7f2420069560 0x7f24200699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:28.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.777+0000 7f2426aa1700 1 --2- 172.21.15.163:0/2928401340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2420159890 0x7f242015abd0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:28.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.777+0000 7f2426aa1700 1 -- 172.21.15.163:0/2928401340 >> 172.21.15.163:0/2928401340 conn(0x7f2420068e20 msgr2=0x7f24200d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:28.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.777+0000 7f2426aa1700 1 -- 172.21.15.163:0/2928401340 shutdown_connections 2023-10-09T06:31:28.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.777+0000 7f2426aa1700 1 -- 172.21.15.163:0/2928401340 wait complete. 2023-10-09T06:31:28.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.778+0000 7f2426aa1700 1 Processor -- start 2023-10-09T06:31:28.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.779+0000 7f2426aa1700 1 -- start start 2023-10-09T06:31:28.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.779+0000 7f2426aa1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2420069560 0x7f24200c9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:28.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.779+0000 7f2426aa1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2420159890 0x7f2420149440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:28.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.779+0000 7f2426aa1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2420159c60 0x7f24200b1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:28.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.779+0000 7f2426aa1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24200cef50 con 0x7f2420159890 2023-10-09T06:31:28.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.779+0000 7f2426aa1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24200cf230 con 0x7f2420159c60 2023-10-09T06:31:28.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.779+0000 7f2426aa1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24200cf510 con 0x7f2420069560 2023-10-09T06:31:28.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.779+0000 7f2417fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2420159890 0x7f2420149440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:28.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.779+0000 7f2417fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2420159890 0x7f2420149440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33372/0 (socket says 172.21.15.163:33372) 2023-10-09T06:31:28.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.779+0000 7f2417fff700 1 -- 172.21.15.163:0/3052902869 learned_addr learned my addr 172.21.15.163:0/3052902869 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:28.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.780+0000 7f242483d700 1 --2- 172.21.15.163:0/3052902869 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2420069560 0x7f24200c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:28.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.780+0000 7f242503e700 1 --2- 172.21.15.163:0/3052902869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2420159c60 0x7f24200b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:28.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.780+0000 7f2417fff700 1 -- 172.21.15.163:0/3052902869 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2420069560 msgr2=0x7f24200c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:28.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.780+0000 7f2417fff700 1 --2- 172.21.15.163:0/3052902869 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2420069560 0x7f24200c9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:28.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.780+0000 7f2417fff700 1 -- 172.21.15.163:0/3052902869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2420159c60 msgr2=0x7f24200b1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:28.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.780+0000 7f2417fff700 1 --2- 172.21.15.163:0/3052902869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2420159c60 0x7f24200b1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:28.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.780+0000 7f2417fff700 1 -- 172.21.15.163:0/3052902869 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241c0094f0 con 0x7f2420159890 2023-10-09T06:31:28.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.780+0000 7f2417fff700 1 --2- 172.21.15.163:0/3052902869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2420159890 0x7f2420149440 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f241000b5e0 tx=0x7f2410009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:28.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.780+0000 7f2414ff9700 1 -- 172.21.15.163:0/3052902869 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241000f510 con 0x7f2420159890 2023-10-09T06:31:28.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.780+0000 7f2414ff9700 1 -- 172.21.15.163:0/3052902869 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241000f670 con 0x7f2420159890 2023-10-09T06:31:28.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.780+0000 7f2426aa1700 1 -- 172.21.15.163:0/3052902869 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24200cf7f0 con 0x7f2420159890 2023-10-09T06:31:28.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.780+0000 7f2414ff9700 1 -- 172.21.15.163:0/3052902869 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241000fa30 con 0x7f2420159890 2023-10-09T06:31:28.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.780+0000 7f2426aa1700 1 -- 172.21.15.163:0/3052902869 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24200c97c0 con 0x7f2420159890 2023-10-09T06:31:28.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.781+0000 7f2426aa1700 1 -- 172.21.15.163:0/3052902869 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24200cfd10 con 0x7f2420159890 2023-10-09T06:31:28.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.781+0000 7f2414ff9700 1 -- 172.21.15.163:0/3052902869 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f241001a400 con 0x7f2420159890 2023-10-09T06:31:28.785 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.782+0000 7f2414ff9700 1 --2- 172.21.15.163:0/3052902869 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f240c066f30 0x7f240c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:28.785 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.782+0000 7f2414ff9700 1 -- 172.21.15.163:0/3052902869 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2410084060 con 0x7f2420159890 2023-10-09T06:31:28.785 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.782+0000 7f242483d700 1 --2- 172.21.15.163:0/3052902869 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f240c066f30 0x7f240c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:28.785 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.782+0000 7f242483d700 1 --2- 172.21.15.163:0/3052902869 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f240c066f30 0x7f240c0693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f242006a890 tx=0x7f241c016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:28.787 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.786+0000 7f2414ff9700 1 -- 172.21.15.163:0/3052902869 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f24200cfd10 con 0x7f2420159890 2023-10-09T06:31:28.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.968+0000 7f2426aa1700 1 -- 172.21.15.163:0/3052902869 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24200cfd10 con 0x7f2420159890 2023-10-09T06:31:28.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.969+0000 7f2414ff9700 1 -- 172.21.15.163:0/3052902869 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+15411 (secure 0 0 0) 0x7f24200cfd10 con 0x7f2420159890 2023-10-09T06:31:28.970 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:28.970 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":30,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:31:28.010876+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:31:25.770415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-09T06:31:26.980879+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:31:28.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.971+0000 7f2426aa1700 1 -- 172.21.15.163:0/3052902869 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f240c066f30 msgr2=0x7f240c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:28.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.972+0000 7f2426aa1700 1 --2- 172.21.15.163:0/3052902869 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f240c066f30 0x7f240c0693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f242006a890 tx=0x7f241c016000 comp rx=0 tx=0).stop 2023-10-09T06:31:28.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.972+0000 7f2426aa1700 1 -- 172.21.15.163:0/3052902869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2420159890 msgr2=0x7f2420149440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:28.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.972+0000 7f2426aa1700 1 --2- 172.21.15.163:0/3052902869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2420159890 0x7f2420149440 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f241000b5e0 tx=0x7f2410009120 comp rx=0 tx=0).stop 2023-10-09T06:31:28.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.972+0000 7f2426aa1700 1 -- 172.21.15.163:0/3052902869 shutdown_connections 2023-10-09T06:31:28.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.972+0000 7f2426aa1700 1 --2- 172.21.15.163:0/3052902869 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f240c066f30 0x7f240c0693e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:28.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.972+0000 7f2426aa1700 1 --2- 172.21.15.163:0/3052902869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2420159c60 0x7f24200b1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:28.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.972+0000 7f2426aa1700 1 --2- 172.21.15.163:0/3052902869 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2420069560 0x7f24200c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:28.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.972+0000 7f2426aa1700 1 --2- 172.21.15.163:0/3052902869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2420159890 0x7f2420149440 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:28.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.972+0000 7f2426aa1700 1 -- 172.21.15.163:0/3052902869 >> 172.21.15.163:0/3052902869 conn(0x7f2420068e20 msgr2=0x7f2420153300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:28.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.972+0000 7f2426aa1700 1 -- 172.21.15.163:0/3052902869 shutdown_connections 2023-10-09T06:31:28.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:28.972+0000 7f2426aa1700 1 -- 172.21.15.163:0/3052902869 wait complete. 2023-10-09T06:31:28.985 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:31:29.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.098+0000 7f6dbffff700 1 Processor -- start 2023-10-09T06:31:29.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.098+0000 7f6dbffff700 1 -- start start 2023-10-09T06:31:29.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.099+0000 7f6dbffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6dc0159420 0x7f6dc00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:29.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.099+0000 7f6dbffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6dc01597f0 0x7f6dc00b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:29.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.099+0000 7f6dbffff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dc00b11d0 con 0x7f6dc015d230 2023-10-09T06:31:29.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.099+0000 7f6dbffff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dc00b14b0 con 0x7f6dc0159420 2023-10-09T06:31:29.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.099+0000 7f6dbffff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dc00b1790 con 0x7f6dc01597f0 2023-10-09T06:31:29.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.099+0000 7f6dbeffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6dc0159420 0x7f6dc00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:29.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.099+0000 7f6dbeffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6dc0159420 0x7f6dc00b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33376/0 (socket says 172.21.15.163:33376) 2023-10-09T06:31:29.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.099+0000 7f6dbeffd700 1 -- 172.21.15.163:0/2572582220 learned_addr learned my addr 172.21.15.163:0/2572582220 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:29.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.099+0000 7f6dbe7fc700 1 --2- 172.21.15.163:0/2572582220 >> v2:172.21.15.181:3300/0 conn(0x7f6dc01597f0 0x7f6dc00b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:29.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.099+0000 7f6dbeffd700 1 -- 172.21.15.163:0/2572582220 >> v2:172.21.15.181:3300/0 conn(0x7f6dc01597f0 msgr2=0x7f6dc00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:29.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.099+0000 7f6dbeffd700 1 --2- 172.21.15.163:0/2572582220 >> v2:172.21.15.181:3300/0 conn(0x7f6dc01597f0 0x7f6dc00b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:29.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.099+0000 7f6dbeffd700 1 -- 172.21.15.163:0/2572582220 >> v1:172.21.15.181:6789/0 conn(0x7f6dc015d230 legacy=0x7f6dc015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:29.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.099+0000 7f6dbeffd700 1 -- 172.21.15.163:0/2572582220 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc00b1a70 con 0x7f6dc0159420 2023-10-09T06:31:29.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.099+0000 7f6dbe7fc700 1 --2- 172.21.15.163:0/2572582220 >> v2:172.21.15.181:3300/0 conn(0x7f6dc01597f0 0x7f6dc00b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:29.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.100+0000 7f6dbeffd700 1 --2- 172.21.15.163:0/2572582220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dc0159420 0x7f6dc00b5f40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0089750 tx=0x7f6db0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78fb769ca66464ba server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:29.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.100+0000 7f6dbdffb700 1 -- 172.21.15.163:0/2572582220 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db000e070 con 0x7f6dc0159420 2023-10-09T06:31:29.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.100+0000 7f6dbdffb700 1 -- 172.21.15.163:0/2572582220 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db000ba10 con 0x7f6dc0159420 2023-10-09T06:31:29.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.100+0000 7f6dbdffb700 1 -- 172.21.15.163:0/2572582220 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db00092e0 con 0x7f6dc0159420 2023-10-09T06:31:29.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.100+0000 7f6dbffff700 1 -- 172.21.15.163:0/2572582220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dc0159420 msgr2=0x7f6dc00b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:29.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.100+0000 7f6dbffff700 1 --2- 172.21.15.163:0/2572582220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dc0159420 0x7f6dc00b5f40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0089750 tx=0x7f6db0009840 comp rx=0 tx=0).stop 2023-10-09T06:31:29.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.100+0000 7f6dbffff700 1 -- 172.21.15.163:0/2572582220 shutdown_connections 2023-10-09T06:31:29.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.100+0000 7f6dbffff700 1 --2- 172.21.15.163:0/2572582220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dc0159420 0x7f6dc00b5f40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:29.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.100+0000 7f6dbffff700 1 --2- 172.21.15.163:0/2572582220 >> v2:172.21.15.181:3300/0 conn(0x7f6dc01597f0 0x7f6dc00b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:29.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.100+0000 7f6dbffff700 1 -- 172.21.15.163:0/2572582220 >> 172.21.15.163:0/2572582220 conn(0x7f6dc00d0660 msgr2=0x7f6dc00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:29.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.100+0000 7f6dbffff700 1 -- 172.21.15.163:0/2572582220 shutdown_connections 2023-10-09T06:31:29.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.100+0000 7f6dbffff700 1 -- 172.21.15.163:0/2572582220 wait complete. 2023-10-09T06:31:29.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.101+0000 7f6dbffff700 1 Processor -- start 2023-10-09T06:31:29.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.102+0000 7f6dbffff700 1 -- start start 2023-10-09T06:31:29.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.102+0000 7f6dbffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dc0159420 0x7f6dc00c7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:29.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.102+0000 7f6dbffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6dc01597f0 0x7f6dc0156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:29.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.102+0000 7f6dbffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dc015d230 0x7f6dc015a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:29.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.102+0000 7f6dbffff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc0169ae0 con 0x7f6dc0159420 2023-10-09T06:31:29.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.102+0000 7f6dbffff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc0169dc0 con 0x7f6dc015d230 2023-10-09T06:31:29.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.102+0000 7f6dbffff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc0156790 con 0x7f6dc01597f0 2023-10-09T06:31:29.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.102+0000 7f6dbeffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dc0159420 0x7f6dc00c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:29.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.102+0000 7f6dbeffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dc0159420 0x7f6dc00c7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33378/0 (socket says 172.21.15.163:33378) 2023-10-09T06:31:29.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.102+0000 7f6dbeffd700 1 -- 172.21.15.163:0/1514197620 learned_addr learned my addr 172.21.15.163:0/1514197620 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:29.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.103+0000 7f6dbe7fc700 1 --2- 172.21.15.163:0/1514197620 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6dc01597f0 0x7f6dc0156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:29.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.103+0000 7f6dbf7fe700 1 --2- 172.21.15.163:0/1514197620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dc015d230 0x7f6dc015a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:29.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.103+0000 7f6dbeffd700 1 -- 172.21.15.163:0/1514197620 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6dc01597f0 msgr2=0x7f6dc0156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:29.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.103+0000 7f6dbeffd700 1 --2- 172.21.15.163:0/1514197620 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6dc01597f0 0x7f6dc0156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:29.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.103+0000 7f6dbeffd700 1 -- 172.21.15.163:0/1514197620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dc015d230 msgr2=0x7f6dc015a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:29.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.103+0000 7f6dbeffd700 1 --2- 172.21.15.163:0/1514197620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dc015d230 0x7f6dc015a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:29.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.103+0000 7f6dbeffd700 1 -- 172.21.15.163:0/1514197620 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db0009ae0 con 0x7f6dc0159420 2023-10-09T06:31:29.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.103+0000 7f6dbeffd700 1 --2- 172.21.15.163:0/1514197620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dc0159420 0x7f6dc00c7670 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6db000beb0 tx=0x7f6db0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:29.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.103+0000 7f6da6ffd700 1 -- 172.21.15.163:0/1514197620 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db00129a0 con 0x7f6dc0159420 2023-10-09T06:31:29.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.103+0000 7f6da6ffd700 1 -- 172.21.15.163:0/1514197620 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db0012b00 con 0x7f6dc0159420 2023-10-09T06:31:29.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.103+0000 7f6dbffff700 1 -- 172.21.15.163:0/1514197620 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc015abc0 con 0x7f6dc0159420 2023-10-09T06:31:29.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.103+0000 7f6da6ffd700 1 -- 172.21.15.163:0/1514197620 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db0012ea0 con 0x7f6dc0159420 2023-10-09T06:31:29.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.103+0000 7f6dbffff700 1 -- 172.21.15.163:0/1514197620 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc015b150 con 0x7f6dc0159420 2023-10-09T06:31:29.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.104+0000 7f6dbffff700 1 -- 172.21.15.163:0/1514197620 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dc004c3c0 con 0x7f6dc0159420 2023-10-09T06:31:29.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.104+0000 7f6da6ffd700 1 -- 172.21.15.163:0/1514197620 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6db001d630 con 0x7f6dc0159420 2023-10-09T06:31:29.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.105+0000 7f6da6ffd700 1 --2- 172.21.15.163:0/1514197620 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6dac066f30 0x7f6dac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:29.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.105+0000 7f6da6ffd700 1 -- 172.21.15.163:0/1514197620 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6db0085be0 con 0x7f6dc0159420 2023-10-09T06:31:29.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.105+0000 7f6dbe7fc700 1 --2- 172.21.15.163:0/1514197620 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6dac066f30 0x7f6dac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:29.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.105+0000 7f6dbe7fc700 1 --2- 172.21.15.163:0/1514197620 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6dac066f30 0x7f6dac0693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6db4006f00 tx=0x7f6db4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:29.111 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.109+0000 7f6da6ffd700 1 -- 172.21.15.163:0/1514197620 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6db0052e50 con 0x7f6dc0159420 2023-10-09T06:31:29.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.292+0000 7f6dbffff700 1 -- 172.21.15.163:0/1514197620 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6dc00b1d50 con 0x7f6dc0159420 2023-10-09T06:31:29.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.293+0000 7f6da6ffd700 1 -- 172.21.15.163:0/1514197620 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+15411 (secure 0 0 0) 0x7f6db0056470 con 0x7f6dc0159420 2023-10-09T06:31:29.294 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:29.295 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":30,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:31:28.010876+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:31:25.770415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-09T06:31:26.980879+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:31:29.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.295+0000 7f6dbffff700 1 -- 172.21.15.163:0/1514197620 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6dac066f30 msgr2=0x7f6dac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:29.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.295+0000 7f6dbffff700 1 --2- 172.21.15.163:0/1514197620 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6dac066f30 0x7f6dac0693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6db4006f00 tx=0x7f6db4008040 comp rx=0 tx=0).stop 2023-10-09T06:31:29.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.295+0000 7f6dbffff700 1 -- 172.21.15.163:0/1514197620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dc0159420 msgr2=0x7f6dc00c7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:29.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.295+0000 7f6dbffff700 1 --2- 172.21.15.163:0/1514197620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dc0159420 0x7f6dc00c7670 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6db000beb0 tx=0x7f6db0017000 comp rx=0 tx=0).stop 2023-10-09T06:31:29.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.295+0000 7f6dbffff700 1 -- 172.21.15.163:0/1514197620 shutdown_connections 2023-10-09T06:31:29.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.295+0000 7f6dbffff700 1 --2- 172.21.15.163:0/1514197620 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6dac066f30 0x7f6dac0693e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:29.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.295+0000 7f6dbffff700 1 --2- 172.21.15.163:0/1514197620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dc0159420 0x7f6dc00c7670 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:29.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.295+0000 7f6dbffff700 1 --2- 172.21.15.163:0/1514197620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dc015d230 0x7f6dc015a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:29.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.295+0000 7f6dbffff700 1 --2- 172.21.15.163:0/1514197620 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6dc01597f0 0x7f6dc0156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:29.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.295+0000 7f6dbffff700 1 -- 172.21.15.163:0/1514197620 >> 172.21.15.163:0/1514197620 conn(0x7f6dc00d0660 msgr2=0x7f6dc0069ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:29.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.295+0000 7f6dbffff700 1 -- 172.21.15.163:0/1514197620 shutdown_connections 2023-10-09T06:31:29.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.296+0000 7f6dbffff700 1 -- 172.21.15.163:0/1514197620 wait complete. 2023-10-09T06:31:29.309 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-09T06:31:29.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.425+0000 7f453633d700 1 Processor -- start 2023-10-09T06:31:29.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.425+0000 7f453633d700 1 -- start start 2023-10-09T06:31:29.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.426+0000 7f453633d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f453006c0a0 0x7f45300caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:29.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.426+0000 7f453633d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f453015df50 0x7f453015e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:29.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.426+0000 7f453633d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f453015d7e0 con 0x7f453006bcd0 2023-10-09T06:31:29.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.426+0000 7f453633d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f453015dac0 con 0x7f453015df50 2023-10-09T06:31:29.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.426+0000 7f453633d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f453015aa30 con 0x7f453006c0a0 2023-10-09T06:31:29.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.426+0000 7f45348da700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f453015df50 0x7f453015e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:29.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.426+0000 7f45348da700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f453015df50 0x7f453015e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33384/0 (socket says 172.21.15.163:33384) 2023-10-09T06:31:29.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.426+0000 7f45348da700 1 -- 172.21.15.163:0/3863171301 learned_addr learned my addr 172.21.15.163:0/3863171301 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:29.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.427+0000 7f452f7fe700 1 --2- 172.21.15.163:0/3863171301 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f453006c0a0 0x7f45300caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:29.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.427+0000 7f45348da700 1 -- 172.21.15.163:0/3863171301 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f453006c0a0 msgr2=0x7f45300caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:29.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.427+0000 7f45348da700 1 --2- 172.21.15.163:0/3863171301 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f453006c0a0 0x7f45300caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:29.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.427+0000 7f45348da700 1 -- 172.21.15.163:0/3863171301 >> v1:172.21.15.181:6789/0 conn(0x7f453006bcd0 legacy=0x7f453015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:29.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.427+0000 7f45348da700 1 -- 172.21.15.163:0/3863171301 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453015ad10 con 0x7f453015df50 2023-10-09T06:31:29.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.427+0000 7f45348da700 1 --2- 172.21.15.163:0/3863171301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f453015df50 0x7f453015e3c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4528009f20 tx=0x7f4528009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9350d8f6f49c945 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:29.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.427+0000 7f452effd700 1 -- 172.21.15.163:0/3863171301 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452800e070 con 0x7f453015df50 2023-10-09T06:31:29.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.427+0000 7f452effd700 1 -- 172.21.15.163:0/3863171301 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452800ba80 con 0x7f453015df50 2023-10-09T06:31:29.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.427+0000 7f452effd700 1 -- 172.21.15.163:0/3863171301 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45280092e0 con 0x7f453015df50 2023-10-09T06:31:29.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.428+0000 7f453633d700 1 -- 172.21.15.163:0/3863171301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f453015df50 msgr2=0x7f453015e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:29.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.428+0000 7f453633d700 1 --2- 172.21.15.163:0/3863171301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f453015df50 0x7f453015e3c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4528009f20 tx=0x7f4528009840 comp rx=0 tx=0).stop 2023-10-09T06:31:29.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.428+0000 7f453633d700 1 -- 172.21.15.163:0/3863171301 shutdown_connections 2023-10-09T06:31:29.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.428+0000 7f453633d700 1 --2- 172.21.15.163:0/3863171301 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f453015df50 0x7f453015e3c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:29.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.428+0000 7f453633d700 1 --2- 172.21.15.163:0/3863171301 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f453006c0a0 0x7f45300caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:29.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.428+0000 7f453633d700 1 -- 172.21.15.163:0/3863171301 >> 172.21.15.163:0/3863171301 conn(0x7f45300ce600 msgr2=0x7f45300cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:29.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.428+0000 7f453633d700 1 -- 172.21.15.163:0/3863171301 shutdown_connections 2023-10-09T06:31:29.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.428+0000 7f453633d700 1 -- 172.21.15.163:0/3863171301 wait complete. 2023-10-09T06:31:29.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.429+0000 7f453633d700 1 Processor -- start 2023-10-09T06:31:29.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.429+0000 7f453633d700 1 -- start start 2023-10-09T06:31:29.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.430+0000 7f453633d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f453006bcd0 0x7f45300b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:29.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.430+0000 7f453633d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f453006c0a0 0x7f45301565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:29.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.430+0000 7f453633d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f453015df50 0x7f45300b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:29.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.430+0000 7f453633d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45300b2970 con 0x7f453006bcd0 2023-10-09T06:31:29.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.430+0000 7f453633d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4530156b30 con 0x7f453015df50 2023-10-09T06:31:29.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.430+0000 7f453633d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45300b1d60 con 0x7f453006c0a0 2023-10-09T06:31:29.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.430+0000 7f452ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f453006bcd0 0x7f45300b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:29.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.430+0000 7f452ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f453006bcd0 0x7f45300b2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33388/0 (socket says 172.21.15.163:33388) 2023-10-09T06:31:29.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.430+0000 7f452ffff700 1 -- 172.21.15.163:0/3311136568 learned_addr learned my addr 172.21.15.163:0/3311136568 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:29.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.430+0000 7f452f7fe700 1 --2- 172.21.15.163:0/3311136568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f453006c0a0 0x7f45301565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:29.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.430+0000 7f45348da700 1 --2- 172.21.15.163:0/3311136568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f453015df50 0x7f45300b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:29.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.430+0000 7f452ffff700 1 -- 172.21.15.163:0/3311136568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f453006c0a0 msgr2=0x7f45301565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:29.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.430+0000 7f452ffff700 1 --2- 172.21.15.163:0/3311136568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f453006c0a0 0x7f45301565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:29.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.430+0000 7f452ffff700 1 -- 172.21.15.163:0/3311136568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f453015df50 msgr2=0x7f45300b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:29.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.430+0000 7f452ffff700 1 --2- 172.21.15.163:0/3311136568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f453015df50 0x7f45300b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:29.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.430+0000 7f452ffff700 1 -- 172.21.15.163:0/3311136568 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4528009ae0 con 0x7f453006bcd0 2023-10-09T06:31:29.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.430+0000 7f45348da700 1 --2- 172.21.15.163:0/3311136568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f453015df50 0x7f45300b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:29.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.430+0000 7f452ffff700 1 --2- 172.21.15.163:0/3311136568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f453006bcd0 0x7f45300b2e00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f452400c510 tx=0x7f452400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:29.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.430+0000 7f4517fff700 1 -- 172.21.15.163:0/3311136568 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4524011590 con 0x7f453006bcd0 2023-10-09T06:31:29.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.430+0000 7f4517fff700 1 -- 172.21.15.163:0/3311136568 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4524011710 con 0x7f453006bcd0 2023-10-09T06:31:29.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.430+0000 7f453633d700 1 -- 172.21.15.163:0/3311136568 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45300b2040 con 0x7f453006bcd0 2023-10-09T06:31:29.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.430+0000 7f4517fff700 1 -- 172.21.15.163:0/3311136568 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4524011ad0 con 0x7f453006bcd0 2023-10-09T06:31:29.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.431+0000 7f453633d700 1 -- 172.21.15.163:0/3311136568 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45300b2600 con 0x7f453006bcd0 2023-10-09T06:31:29.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.431+0000 7f453633d700 1 -- 172.21.15.163:0/3311136568 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f453004c3c0 con 0x7f453006bcd0 2023-10-09T06:31:29.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.432+0000 7f4517fff700 1 -- 172.21.15.163:0/3311136568 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f452401b3b0 con 0x7f453006bcd0 2023-10-09T06:31:29.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.432+0000 7f4517fff700 1 --2- 172.21.15.163:0/3311136568 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f451c066f30 0x7f451c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:29.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.432+0000 7f4517fff700 1 -- 172.21.15.163:0/3311136568 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4524083980 con 0x7f453006bcd0 2023-10-09T06:31:29.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.432+0000 7f452f7fe700 1 --2- 172.21.15.163:0/3311136568 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f451c066f30 0x7f451c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:29.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.433+0000 7f452f7fe700 1 --2- 172.21.15.163:0/3311136568 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f451c066f30 0x7f451c0693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f45300c5020 tx=0x7f4520008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:29.437 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.436+0000 7f4517fff700 1 -- 172.21.15.163:0/3311136568 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4524050bf0 con 0x7f453006bcd0 2023-10-09T06:31:29.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:29.663+0000 7f453633d700 1 -- 172.21.15.163:0/3311136568 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f453015aff0 con 0x7f453006bcd0 2023-10-09T06:31:30.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.660+0000 7f4517fff700 1 -- 172.21.15.163:0/3311136568 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4524017070 con 0x7f453006bcd0 2023-10-09T06:31:30.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.663+0000 7f453633d700 1 -- 172.21.15.163:0/3311136568 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f451c066f30 msgr2=0x7f451c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:30.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.663+0000 7f453633d700 1 --2- 172.21.15.163:0/3311136568 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f451c066f30 0x7f451c0693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f45300c5020 tx=0x7f4520008040 comp rx=0 tx=0).stop 2023-10-09T06:31:30.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.663+0000 7f453633d700 1 -- 172.21.15.163:0/3311136568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f453006bcd0 msgr2=0x7f45300b2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:30.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.663+0000 7f453633d700 1 --2- 172.21.15.163:0/3311136568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f453006bcd0 0x7f45300b2e00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f452400c510 tx=0x7f452400c820 comp rx=0 tx=0).stop 2023-10-09T06:31:30.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.663+0000 7f453633d700 1 -- 172.21.15.163:0/3311136568 shutdown_connections 2023-10-09T06:31:30.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.663+0000 7f453633d700 1 --2- 172.21.15.163:0/3311136568 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f451c066f30 0x7f451c0693e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:30.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.663+0000 7f453633d700 1 --2- 172.21.15.163:0/3311136568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f453006bcd0 0x7f45300b2e00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:30.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.663+0000 7f453633d700 1 --2- 172.21.15.163:0/3311136568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f453015df50 0x7f45300b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:30.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.663+0000 7f453633d700 1 --2- 172.21.15.163:0/3311136568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f453006c0a0 0x7f45301565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:30.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.663+0000 7f453633d700 1 -- 172.21.15.163:0/3311136568 >> 172.21.15.163:0/3311136568 conn(0x7f45300ce600 msgr2=0x7f4530069480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:30.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.663+0000 7f453633d700 1 -- 172.21.15.163:0/3311136568 shutdown_connections 2023-10-09T06:31:30.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.663+0000 7f453633d700 1 -- 172.21.15.163:0/3311136568 wait complete. 2023-10-09T06:31:30.676 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-09T06:31:30.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.793+0000 7f2dcd8fa700 1 Processor -- start 2023-10-09T06:31:30.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.793+0000 7f2dcd8fa700 1 -- start start 2023-10-09T06:31:30.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.794+0000 7f2dcd8fa700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2dc80b16d0 0x7f2dc80b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:30.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.794+0000 7f2dcd8fa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dc80b33b0 0x7f2dc80b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:30.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.794+0000 7f2dcd8fa700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dc80b2080 con 0x7f2dc80b4e70 2023-10-09T06:31:30.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.794+0000 7f2dcd8fa700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dc80cfb00 con 0x7f2dc80b16d0 2023-10-09T06:31:30.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.794+0000 7f2dcd8fa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc80b5c40 con 0x7f2dc80b33b0 2023-10-09T06:31:30.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.794+0000 7f2dc7fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2dc80b16d0 0x7f2dc80b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:30.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.794+0000 7f2dc7fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2dc80b16d0 0x7f2dc80b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33400/0 (socket says 172.21.15.163:33400) 2023-10-09T06:31:30.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.794+0000 7f2dc7fff700 1 -- 172.21.15.163:0/4261689716 learned_addr learned my addr 172.21.15.163:0/4261689716 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:30.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.794+0000 7f2dc7fff700 1 -- 172.21.15.163:0/4261689716 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dc80b33b0 msgr2=0x7f2dc80b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:30.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.794+0000 7f2dc7fff700 1 --2- 172.21.15.163:0/4261689716 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dc80b33b0 0x7f2dc80b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:30.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.794+0000 7f2dc7fff700 1 -- 172.21.15.163:0/4261689716 >> v1:172.21.15.181:6789/0 conn(0x7f2dc80b4e70 legacy=0x7f2dc80ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:30.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.794+0000 7f2dc7fff700 1 -- 172.21.15.163:0/4261689716 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc80b5f20 con 0x7f2dc80b16d0 2023-10-09T06:31:30.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.794+0000 7f2dc7fff700 1 --2- 172.21.15.163:0/4261689716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2dc80b16d0 0x7f2dc80b1b40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2db00099b0 tx=0x7f2db0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=108d32578993a66c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:30.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.794+0000 7f2dc77fe700 1 -- 172.21.15.163:0/4261689716 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db000b820 con 0x7f2dc80b16d0 2023-10-09T06:31:30.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.794+0000 7f2dc77fe700 1 -- 172.21.15.163:0/4261689716 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db000b980 con 0x7f2dc80b16d0 2023-10-09T06:31:30.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.794+0000 7f2dc77fe700 1 -- 172.21.15.163:0/4261689716 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db000bd40 con 0x7f2dc80b16d0 2023-10-09T06:31:30.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.795+0000 7f2dcd8fa700 1 -- 172.21.15.163:0/4261689716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2dc80b16d0 msgr2=0x7f2dc80b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:30.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.795+0000 7f2dcd8fa700 1 --2- 172.21.15.163:0/4261689716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2dc80b16d0 0x7f2dc80b1b40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2db00099b0 tx=0x7f2db0009250 comp rx=0 tx=0).stop 2023-10-09T06:31:30.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.795+0000 7f2dcd8fa700 1 -- 172.21.15.163:0/4261689716 shutdown_connections 2023-10-09T06:31:30.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.795+0000 7f2dcd8fa700 1 --2- 172.21.15.163:0/4261689716 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dc80b33b0 0x7f2dc80b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:30.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.795+0000 7f2dcd8fa700 1 --2- 172.21.15.163:0/4261689716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2dc80b16d0 0x7f2dc80b1b40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:30.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.795+0000 7f2dcd8fa700 1 -- 172.21.15.163:0/4261689716 >> 172.21.15.163:0/4261689716 conn(0x7f2dc80d10c0 msgr2=0x7f2dc80affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:30.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.795+0000 7f2dcd8fa700 1 -- 172.21.15.163:0/4261689716 shutdown_connections 2023-10-09T06:31:30.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.795+0000 7f2dcd8fa700 1 -- 172.21.15.163:0/4261689716 wait complete. 2023-10-09T06:31:30.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.796+0000 7f2dcd8fa700 1 Processor -- start 2023-10-09T06:31:30.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.796+0000 7f2dcd8fa700 1 -- start start 2023-10-09T06:31:30.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.797+0000 7f2dcd8fa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dc80b16d0 0x7f2dc816d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:30.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.797+0000 7f2dcd8fa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc80b33b0 0x7f2dc816dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:30.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.797+0000 7f2dcd8fa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2dc80b4e70 0x7f2dc816a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:30.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.797+0000 7f2dcd8fa700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc816bc30 con 0x7f2dc80b4e70 2023-10-09T06:31:30.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.797+0000 7f2dcd8fa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc80c90e0 con 0x7f2dc80b33b0 2023-10-09T06:31:30.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.797+0000 7f2dcd8fa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc816b300 con 0x7f2dc80b16d0 2023-10-09T06:31:30.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.797+0000 7f2dcd0f9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2dc80b4e70 0x7f2dc816a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:30.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.797+0000 7f2dcd0f9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2dc80b4e70 0x7f2dc816a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33404/0 (socket says 172.21.15.163:33404) 2023-10-09T06:31:30.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.797+0000 7f2dcd0f9700 1 -- 172.21.15.163:0/220912316 learned_addr learned my addr 172.21.15.163:0/220912316 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:30.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.797+0000 7f2dcc8f8700 1 --2- 172.21.15.163:0/220912316 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dc80b16d0 0x7f2dc816d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:30.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.797+0000 7f2dc7fff700 1 --2- 172.21.15.163:0/220912316 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc80b33b0 0x7f2dc816dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:30.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.797+0000 7f2dcd0f9700 1 -- 172.21.15.163:0/220912316 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dc80b16d0 msgr2=0x7f2dc816d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:30.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.797+0000 7f2dcd0f9700 1 --2- 172.21.15.163:0/220912316 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dc80b16d0 0x7f2dc816d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:30.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.797+0000 7f2dcd0f9700 1 -- 172.21.15.163:0/220912316 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc80b33b0 msgr2=0x7f2dc816dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:30.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.797+0000 7f2dcd0f9700 1 --2- 172.21.15.163:0/220912316 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc80b33b0 0x7f2dc816dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:30.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.797+0000 7f2dcd0f9700 1 -- 172.21.15.163:0/220912316 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db00094f0 con 0x7f2dc80b4e70 2023-10-09T06:31:30.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.797+0000 7f2dcd0f9700 1 --2- 172.21.15.163:0/220912316 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2dc80b4e70 0x7f2dc816a090 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2db800b810 tx=0x7f2db80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:30.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.798+0000 7f2dc4ff9700 1 -- 172.21.15.163:0/220912316 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db800f510 con 0x7f2dc80b4e70 2023-10-09T06:31:30.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.798+0000 7f2dc4ff9700 1 -- 172.21.15.163:0/220912316 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db800f670 con 0x7f2dc80b4e70 2023-10-09T06:31:30.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.798+0000 7f2dc4ff9700 1 -- 172.21.15.163:0/220912316 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db800fa30 con 0x7f2dc80b4e70 2023-10-09T06:31:30.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.798+0000 7f2dcd8fa700 1 -- 172.21.15.163:0/220912316 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dc816b5e0 con 0x7f2dc80b4e70 2023-10-09T06:31:30.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.798+0000 7f2dcd8fa700 1 -- 172.21.15.163:0/220912316 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc8147a00 con 0x7f2dc80b4e70 2023-10-09T06:31:30.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.799+0000 7f2dcd8fa700 1 -- 172.21.15.163:0/220912316 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dc8147c50 con 0x7f2dc80b4e70 2023-10-09T06:31:30.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.799+0000 7f2dc4ff9700 1 -- 172.21.15.163:0/220912316 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2db801a400 con 0x7f2dc80b4e70 2023-10-09T06:31:30.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.799+0000 7f2dc4ff9700 1 --2- 172.21.15.163:0/220912316 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2da8066f30 0x7f2da80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:30.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.799+0000 7f2dc4ff9700 1 -- 172.21.15.163:0/220912316 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2db8083fd0 con 0x7f2dc80b4e70 2023-10-09T06:31:30.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.800+0000 7f2dcc8f8700 1 --2- 172.21.15.163:0/220912316 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2da8066f30 0x7f2da80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:30.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.800+0000 7f2dcc8f8700 1 --2- 172.21.15.163:0/220912316 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2da8066f30 0x7f2da80693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2dc00075c0 tx=0x7f2dc0009780 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:30.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:30.803+0000 7f2dc4ff9700 1 -- 172.21.15.163:0/220912316 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2db80512c0 con 0x7f2dc80b4e70 2023-10-09T06:31:31.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.028+0000 7f2dcd8fa700 1 -- 172.21.15.163:0/220912316 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2dc80cf780 con 0x7f2dc80b4e70 2023-10-09T06:31:31.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.664+0000 7f2dc4ff9700 1 -- 172.21.15.163:0/220912316 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2db80548e0 con 0x7f2dc80b4e70 2023-10-09T06:31:31.667 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.666+0000 7f2dcd8fa700 1 -- 172.21.15.163:0/220912316 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2da8066f30 msgr2=0x7f2da80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:31.667 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.666+0000 7f2dcd8fa700 1 --2- 172.21.15.163:0/220912316 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2da8066f30 0x7f2da80693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2dc00075c0 tx=0x7f2dc0009780 comp rx=0 tx=0).stop 2023-10-09T06:31:31.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.666+0000 7f2dcd8fa700 1 -- 172.21.15.163:0/220912316 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2dc80b4e70 msgr2=0x7f2dc816a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:31.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.666+0000 7f2dcd8fa700 1 --2- 172.21.15.163:0/220912316 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2dc80b4e70 0x7f2dc816a090 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2db800b810 tx=0x7f2db80091d0 comp rx=0 tx=0).stop 2023-10-09T06:31:31.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.666+0000 7f2dcd8fa700 1 -- 172.21.15.163:0/220912316 shutdown_connections 2023-10-09T06:31:31.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.666+0000 7f2dcd8fa700 1 --2- 172.21.15.163:0/220912316 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2da8066f30 0x7f2da80693e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:31.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.666+0000 7f2dcd8fa700 1 --2- 172.21.15.163:0/220912316 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2dc80b4e70 0x7f2dc816a090 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:31.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.666+0000 7f2dcd8fa700 1 --2- 172.21.15.163:0/220912316 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dc80b16d0 0x7f2dc816d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:31.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.666+0000 7f2dcd8fa700 1 --2- 172.21.15.163:0/220912316 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc80b33b0 0x7f2dc816dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:31.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.666+0000 7f2dcd8fa700 1 -- 172.21.15.163:0/220912316 >> 172.21.15.163:0/220912316 conn(0x7f2dc80d10c0 msgr2=0x7f2dc8159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:31.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.666+0000 7f2dcd8fa700 1 -- 172.21.15.163:0/220912316 shutdown_connections 2023-10-09T06:31:31.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.666+0000 7f2dcd8fa700 1 -- 172.21.15.163:0/220912316 wait complete. 2023-10-09T06:31:31.680 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:31:31.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.794+0000 7fc99d01e700 1 Processor -- start 2023-10-09T06:31:31.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.794+0000 7fc99d01e700 1 -- start start 2023-10-09T06:31:31.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.795+0000 7fc99d01e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc9980aca80 0x7fc99806aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:31.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.795+0000 7fc99d01e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc99806bdb0 con 0x7fc9980c83c0 2023-10-09T06:31:31.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.795+0000 7fc99d01e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9980cb370 con 0x7fc998069d60 2023-10-09T06:31:31.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.795+0000 7fc99d01e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc998159f60 con 0x7fc9980aca80 2023-10-09T06:31:31.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.795+0000 7fc996d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc9980aca80 0x7fc99806aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:31.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.795+0000 7fc99759e700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fc9980c83c0 0x7fc9980cb800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33860/0 (socket says 172.21.15.163:33860) 2023-10-09T06:31:31.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.795+0000 7fc996d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc9980aca80 0x7fc99806aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33414/0 (socket says 172.21.15.163:33414) 2023-10-09T06:31:31.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.795+0000 7fc99759e700 1 -- 172.21.15.163:0/192302490 learned_addr learned my addr 172.21.15.163:0/192302490 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:31.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.796+0000 7fc996d9d700 1 -- 172.21.15.163:0/192302490 >> v1:172.21.15.181:6789/0 conn(0x7fc998069d60 legacy=0x7fc9980c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:31.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.796+0000 7fc996d9d700 1 -- 172.21.15.163:0/192302490 >> v1:172.21.15.163:6789/0 conn(0x7fc9980c83c0 legacy=0x7fc9980cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:31.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.796+0000 7fc99659c700 1 -- 172.21.15.163:0/192302490 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778805303 0 0) 0x7fc99806bdb0 con 0x7fc9980c83c0 2023-10-09T06:31:31.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.796+0000 7fc996d9d700 1 -- 172.21.15.163:0/192302490 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99806b510 con 0x7fc9980aca80 2023-10-09T06:31:31.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.796+0000 7fc996d9d700 1 --2- 172.21.15.163:0/192302490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9980aca80 0x7fc99806aba0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc9880098d0 tx=0x7fc988009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc2d67a621624ec server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:31.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.796+0000 7fc99659c700 1 -- 172.21.15.163:0/192302490 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc988009e70 con 0x7fc9980aca80 2023-10-09T06:31:31.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.796+0000 7fc99659c700 1 -- 172.21.15.163:0/192302490 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc988005c10 con 0x7fc9980aca80 2023-10-09T06:31:31.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.796+0000 7fc99659c700 1 -- 172.21.15.163:0/192302490 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98800b650 con 0x7fc9980aca80 2023-10-09T06:31:31.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.796+0000 7fc99d01e700 1 -- 172.21.15.163:0/192302490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9980aca80 msgr2=0x7fc99806aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:31.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.796+0000 7fc99d01e700 1 --2- 172.21.15.163:0/192302490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9980aca80 0x7fc99806aba0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc9880098d0 tx=0x7fc988009120 comp rx=0 tx=0).stop 2023-10-09T06:31:31.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.796+0000 7fc99d01e700 1 -- 172.21.15.163:0/192302490 shutdown_connections 2023-10-09T06:31:31.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.796+0000 7fc99d01e700 1 --2- 172.21.15.163:0/192302490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9980aca80 0x7fc99806aba0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:31.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.796+0000 7fc99d01e700 1 -- 172.21.15.163:0/192302490 >> 172.21.15.163:0/192302490 conn(0x7fc9980cd670 msgr2=0x7fc9980cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:31.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.796+0000 7fc99d01e700 1 -- 172.21.15.163:0/192302490 shutdown_connections 2023-10-09T06:31:31.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.796+0000 7fc99d01e700 1 -- 172.21.15.163:0/192302490 wait complete. 2023-10-09T06:31:31.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.798+0000 7fc99d01e700 1 Processor -- start 2023-10-09T06:31:31.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.798+0000 7fc99d01e700 1 -- start start 2023-10-09T06:31:31.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.798+0000 7fc99d01e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc998069d60 0x7fc99816d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:31.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.798+0000 7fc99d01e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9980aca80 0x7fc99816da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:31.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.798+0000 7fc99d01e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9980c83c0 0x7fc99815be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:31.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.798+0000 7fc99d01e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9980ae780 con 0x7fc9980c83c0 2023-10-09T06:31:31.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.798+0000 7fc99d01e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99816b050 con 0x7fc998069d60 2023-10-09T06:31:31.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.798+0000 7fc99d01e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99815cc80 con 0x7fc9980aca80 2023-10-09T06:31:31.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.799+0000 7fc99759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9980c83c0 0x7fc99815be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:31.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.799+0000 7fc99759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9980c83c0 0x7fc99815be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33426/0 (socket says 172.21.15.163:33426) 2023-10-09T06:31:31.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.799+0000 7fc99759e700 1 -- 172.21.15.163:0/599677995 learned_addr learned my addr 172.21.15.163:0/599677995 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:31.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.799+0000 7fc996d9d700 1 --2- 172.21.15.163:0/599677995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc998069d60 0x7fc99816d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:31.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.799+0000 7fc98e59c700 1 --2- 172.21.15.163:0/599677995 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9980aca80 0x7fc99816da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:31.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.799+0000 7fc99759e700 1 -- 172.21.15.163:0/599677995 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9980aca80 msgr2=0x7fc99816da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:31.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.799+0000 7fc99759e700 1 --2- 172.21.15.163:0/599677995 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9980aca80 0x7fc99816da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:31.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.799+0000 7fc99759e700 1 -- 172.21.15.163:0/599677995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc998069d60 msgr2=0x7fc99816d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:31.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.799+0000 7fc99759e700 1 --2- 172.21.15.163:0/599677995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc998069d60 0x7fc99816d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:31.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.799+0000 7fc99759e700 1 -- 172.21.15.163:0/599677995 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9880093c0 con 0x7fc9980c83c0 2023-10-09T06:31:31.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.799+0000 7fc996d9d700 1 --2- 172.21.15.163:0/599677995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc998069d60 0x7fc99816d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:31.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.799+0000 7fc99759e700 1 --2- 172.21.15.163:0/599677995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9980c83c0 0x7fc99815be90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc99000cb90 tx=0x7fc990001940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:31.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.799+0000 7fc98f7fe700 1 -- 172.21.15.163:0/599677995 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99000a910 con 0x7fc9980c83c0 2023-10-09T06:31:31.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.799+0000 7fc98f7fe700 1 -- 172.21.15.163:0/599677995 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99000aa90 con 0x7fc9980c83c0 2023-10-09T06:31:31.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.799+0000 7fc99d01e700 1 -- 172.21.15.163:0/599677995 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc99815cf60 con 0x7fc9980c83c0 2023-10-09T06:31:31.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.799+0000 7fc98f7fe700 1 -- 172.21.15.163:0/599677995 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99000ae50 con 0x7fc9980c83c0 2023-10-09T06:31:31.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.799+0000 7fc99d01e700 1 -- 172.21.15.163:0/599677995 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9980ae410 con 0x7fc9980c83c0 2023-10-09T06:31:31.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.800+0000 7fc99d01e700 1 -- 172.21.15.163:0/599677995 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc99804c3c0 con 0x7fc9980c83c0 2023-10-09T06:31:31.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.800+0000 7fc98f7fe700 1 -- 172.21.15.163:0/599677995 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc990007300 con 0x7fc9980c83c0 2023-10-09T06:31:31.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.801+0000 7fc98f7fe700 1 --2- 172.21.15.163:0/599677995 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc978066f30 0x7fc9780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:31.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.801+0000 7fc98f7fe700 1 -- 172.21.15.163:0/599677995 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc990084900 con 0x7fc9980c83c0 2023-10-09T06:31:31.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.801+0000 7fc996d9d700 1 --2- 172.21.15.163:0/599677995 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc978066f30 0x7fc9780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:31.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.801+0000 7fc996d9d700 1 --2- 172.21.15.163:0/599677995 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc978066f30 0x7fc9780693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc9980c2ab0 tx=0x7fc988016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:31.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:31.805+0000 7fc98f7fe700 1 -- 172.21.15.163:0/599677995 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc990051bf0 con 0x7fc9980c83c0 2023-10-09T06:31:32.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.026+0000 7fc99d01e700 1 -- 172.21.15.163:0/599677995 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9980ae8c0 con 0x7fc9980c83c0 2023-10-09T06:31:32.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.026+0000 7fc98f7fe700 1 -- 172.21.15.163:0/599677995 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fc990055210 con 0x7fc9980c83c0 2023-10-09T06:31:32.027 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:32.027 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4622,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2368375038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2368375038},{"type":"v1","addr":"172.21.15.181:6836","nonce":2368375038}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4929,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/4112854733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4112854733},{"type":"v1","addr":"172.21.15.163:6834","nonce":4112854733}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4932,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2241937708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2241937708},{"type":"v1","addr":"172.21.15.163:6835","nonce":2241937708}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:31:28.011264+0000","modified":"2023-10-09T06:31:31.660915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4944},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4944":{"gid":4944,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2708262268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2708262268},{"type":"v1","addr":"172.21.15.181:6837","nonce":2708262268}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-09T06:31:32.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.028+0000 7fc99d01e700 1 -- 172.21.15.163:0/599677995 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc978066f30 msgr2=0x7fc9780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:32.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.028+0000 7fc99d01e700 1 --2- 172.21.15.163:0/599677995 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc978066f30 0x7fc9780693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc9980c2ab0 tx=0x7fc988016000 comp rx=0 tx=0).stop 2023-10-09T06:31:32.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.028+0000 7fc99d01e700 1 -- 172.21.15.163:0/599677995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9980c83c0 msgr2=0x7fc99815be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:32.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.028+0000 7fc99d01e700 1 --2- 172.21.15.163:0/599677995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9980c83c0 0x7fc99815be90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc99000cb90 tx=0x7fc990001940 comp rx=0 tx=0).stop 2023-10-09T06:31:32.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.028+0000 7fc99d01e700 1 -- 172.21.15.163:0/599677995 shutdown_connections 2023-10-09T06:31:32.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.028+0000 7fc99d01e700 1 --2- 172.21.15.163:0/599677995 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc978066f30 0x7fc9780693e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.028+0000 7fc99d01e700 1 --2- 172.21.15.163:0/599677995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9980c83c0 0x7fc99815be90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.028+0000 7fc99d01e700 1 --2- 172.21.15.163:0/599677995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc998069d60 0x7fc99816d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.028+0000 7fc99d01e700 1 --2- 172.21.15.163:0/599677995 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9980aca80 0x7fc99816da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.028+0000 7fc99d01e700 1 -- 172.21.15.163:0/599677995 >> 172.21.15.163:0/599677995 conn(0x7fc9980cd670 msgr2=0x7fc9981586f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:32.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.028+0000 7fc99d01e700 1 -- 172.21.15.163:0/599677995 shutdown_connections 2023-10-09T06:31:32.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.028+0000 7fc99d01e700 1 -- 172.21.15.163:0/599677995 wait complete. 2023-10-09T06:31:32.030 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 17 2023-10-09T06:31:32.042 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:31:32.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.156+0000 7f929b025700 1 Processor -- start 2023-10-09T06:31:32.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.156+0000 7f929b025700 1 -- start start 2023-10-09T06:31:32.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.157+0000 7f929b025700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9294159420 0x7f92940b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:32.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.157+0000 7f929b025700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92940b2980 con 0x7f92941597f0 2023-10-09T06:31:32.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.157+0000 7f929b025700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92940b11d0 con 0x7f929415d230 2023-10-09T06:31:32.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.157+0000 7f929b025700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92940b14b0 con 0x7f9294159420 2023-10-09T06:31:32.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.157+0000 7f9299822700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f92941597f0 0x7f92940b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33868/0 (socket says 172.21.15.163:33868) 2023-10-09T06:31:32.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.157+0000 7f9299822700 1 -- 172.21.15.163:0/1075867167 learned_addr learned my addr 172.21.15.163:0/1075867167 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:32.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.157+0000 7f929a023700 1 --2- 172.21.15.163:0/1075867167 >> v2:172.21.15.181:3300/0 conn(0x7f9294159420 0x7f92940b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:32.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.157+0000 7f9299021700 1 -- 172.21.15.163:0/1075867167 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1040743149 0 0) 0x7f92940b2980 con 0x7f92941597f0 2023-10-09T06:31:32.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.157+0000 7f9299021700 1 -- 172.21.15.163:0/1075867167 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9280003570 con 0x7f92941597f0 2023-10-09T06:31:32.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.158+0000 7f9299021700 1 -- 172.21.15.163:0/1075867167 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2427611396 0 0) 0x7f9280003570 con 0x7f92941597f0 2023-10-09T06:31:32.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.158+0000 7f9299021700 1 -- 172.21.15.163:0/1075867167 >> v2:172.21.15.181:3300/0 conn(0x7f9294159420 msgr2=0x7f92940b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:32.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.158+0000 7f9299021700 1 --2- 172.21.15.163:0/1075867167 >> v2:172.21.15.181:3300/0 conn(0x7f9294159420 0x7f92940b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.158+0000 7f9299021700 1 -- 172.21.15.163:0/1075867167 >> v1:172.21.15.181:6789/0 conn(0x7f929415d230 legacy=0x7f929415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:32.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.158+0000 7f9299021700 1 -- 172.21.15.163:0/1075867167 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92940b1790 con 0x7f92941597f0 2023-10-09T06:31:32.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.158+0000 7f9299021700 1 -- 172.21.15.163:0/1075867167 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f9284001470 con 0x7f92941597f0 2023-10-09T06:31:32.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.158+0000 7f9299021700 1 -- 172.21.15.163:0/1075867167 >> v1:172.21.15.163:6789/0 conn(0x7f92941597f0 legacy=0x7f92940b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:32.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.158+0000 7f9299021700 1 --2- 172.21.15.163:0/1075867167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9280003a60 0x7f9280003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:32.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.158+0000 7f9299021700 1 --2- 172.21.15.163:0/1075867167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9280004810 0x7f9280004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:32.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.158+0000 7f9299021700 1 --2- 172.21.15.163:0/1075867167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92800051c0 0x7f928000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:32.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.158+0000 7f9299021700 1 -- 172.21.15.163:0/1075867167 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92940b14b0 con 0x7f9280004810 2023-10-09T06:31:32.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.158+0000 7f9299021700 1 -- 172.21.15.163:0/1075867167 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928000c0d0 con 0x7f92800051c0 2023-10-09T06:31:32.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.158+0000 7f9299021700 1 -- 172.21.15.163:0/1075867167 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f928000c290 con 0x7f9280003a60 2023-10-09T06:31:32.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.158+0000 7f9299822700 1 --2- 172.21.15.163:0/1075867167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9280004810 0x7f9280004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:32.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.158+0000 7f929a023700 1 --2- 172.21.15.163:0/1075867167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9280003a60 0x7f9280003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:32.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.158+0000 7f929a824700 1 --2- 172.21.15.163:0/1075867167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92800051c0 0x7f928000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:32.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.158+0000 7f9299822700 1 -- 172.21.15.163:0/1075867167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9280003a60 msgr2=0x7f9280003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:32.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.158+0000 7f9299822700 1 --2- 172.21.15.163:0/1075867167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9280003a60 0x7f9280003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.158+0000 7f9299822700 1 -- 172.21.15.163:0/1075867167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92800051c0 msgr2=0x7f928000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:32.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.158+0000 7f9299822700 1 --2- 172.21.15.163:0/1075867167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92800051c0 0x7f928000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.158+0000 7f929a824700 1 -- 172.21.15.163:0/1075867167 reap_dead start 2023-10-09T06:31:32.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.158+0000 7f9299822700 1 -- 172.21.15.163:0/1075867167 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9280003570 con 0x7f9280004810 2023-10-09T06:31:32.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.158+0000 7f9299822700 1 --2- 172.21.15.163:0/1075867167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9280004810 0x7f9280004c80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f928400c390 tx=0x7f9284001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ccccbf89f95cd5c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:32.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.158+0000 7f9299021700 1 -- 172.21.15.163:0/1075867167 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928400cc60 con 0x7f9280004810 2023-10-09T06:31:32.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.158+0000 7f9299021700 1 -- 172.21.15.163:0/1075867167 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f928400cdc0 con 0x7f9280004810 2023-10-09T06:31:32.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.158+0000 7f9299021700 1 -- 172.21.15.163:0/1075867167 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9284005b50 con 0x7f9280004810 2023-10-09T06:31:32.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.159+0000 7f929b025700 1 -- 172.21.15.163:0/1075867167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9280004810 msgr2=0x7f9280004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:32.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.159+0000 7f929b025700 1 --2- 172.21.15.163:0/1075867167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9280004810 0x7f9280004c80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f928400c390 tx=0x7f9284001e50 comp rx=0 tx=0).stop 2023-10-09T06:31:32.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.159+0000 7f929b025700 1 -- 172.21.15.163:0/1075867167 shutdown_connections 2023-10-09T06:31:32.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.159+0000 7f929b025700 1 --2- 172.21.15.163:0/1075867167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9280004810 0x7f9280004c80 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.159+0000 7f929b025700 1 -- 172.21.15.163:0/1075867167 >> 172.21.15.163:0/1075867167 conn(0x7f92940d0660 msgr2=0x7f92940d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:32.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.159+0000 7f929b025700 1 -- 172.21.15.163:0/1075867167 shutdown_connections 2023-10-09T06:31:32.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.159+0000 7f929b025700 1 -- 172.21.15.163:0/1075867167 wait complete. 2023-10-09T06:31:32.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.160+0000 7f929b025700 1 Processor -- start 2023-10-09T06:31:32.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.160+0000 7f929b025700 1 -- start start 2023-10-09T06:31:32.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.161+0000 7f929b025700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9280004810 0x7f9294155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:32.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.161+0000 7f929b025700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9294155a20 0x7f9294155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:32.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.161+0000 7f929b025700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92940c5840 0x7f92940c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:32.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.161+0000 7f929b025700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92940c7fd0 con 0x7f9294155a20 2023-10-09T06:31:32.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.161+0000 7f929b025700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92940c6220 con 0x7f9280004810 2023-10-09T06:31:32.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.161+0000 7f929b025700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f929415c4d0 con 0x7f92940c5840 2023-10-09T06:31:32.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.161+0000 7f9299822700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9294155a20 0x7f9294155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:32.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.161+0000 7f9299822700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9294155a20 0x7f9294155e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33434/0 (socket says 172.21.15.163:33434) 2023-10-09T06:31:32.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.161+0000 7f9299822700 1 -- 172.21.15.163:0/3874145115 learned_addr learned my addr 172.21.15.163:0/3874145115 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:32.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.161+0000 7f929a023700 1 --2- 172.21.15.163:0/3874145115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9280004810 0x7f9294155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:32.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.161+0000 7f929a824700 1 --2- 172.21.15.163:0/3874145115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92940c5840 0x7f92940c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:32.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.161+0000 7f9299822700 1 -- 172.21.15.163:0/3874145115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92940c5840 msgr2=0x7f92940c5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:32.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.161+0000 7f9299822700 1 --2- 172.21.15.163:0/3874145115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92940c5840 0x7f92940c5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.161+0000 7f9299822700 1 -- 172.21.15.163:0/3874145115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9280004810 msgr2=0x7f9294155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:32.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.161+0000 7f9299822700 1 --2- 172.21.15.163:0/3874145115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9280004810 0x7f9294155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.161+0000 7f9299822700 1 -- 172.21.15.163:0/3874145115 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f929415c770 con 0x7f9294155a20 2023-10-09T06:31:32.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.161+0000 7f9299822700 1 --2- 172.21.15.163:0/3874145115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9294155a20 0x7f9294155e90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f928400c980 tx=0x7f9284015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:32.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.162+0000 7f928a7fc700 1 -- 172.21.15.163:0/3874145115 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9284001430 con 0x7f9294155a20 2023-10-09T06:31:32.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.162+0000 7f928a7fc700 1 -- 172.21.15.163:0/3874145115 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92840070d0 con 0x7f9294155a20 2023-10-09T06:31:32.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.162+0000 7f928a7fc700 1 -- 172.21.15.163:0/3874145115 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9284007490 con 0x7f9294155a20 2023-10-09T06:31:32.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.162+0000 7f929b025700 1 -- 172.21.15.163:0/3874145115 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f929415ca50 con 0x7f9294155a20 2023-10-09T06:31:32.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.162+0000 7f929b025700 1 -- 172.21.15.163:0/3874145115 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9294155780 con 0x7f9294155a20 2023-10-09T06:31:32.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.163+0000 7f929b025700 1 -- 172.21.15.163:0/3874145115 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9294159ca0 con 0x7f9294155a20 2023-10-09T06:31:32.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.163+0000 7f928a7fc700 1 -- 172.21.15.163:0/3874145115 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9284012400 con 0x7f9294155a20 2023-10-09T06:31:32.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.163+0000 7f928a7fc700 1 --2- 172.21.15.163:0/3874145115 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9278066f30 0x7f92780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:32.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.164+0000 7f928a7fc700 1 -- 172.21.15.163:0/3874145115 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9284084fc0 con 0x7f9294155a20 2023-10-09T06:31:32.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.164+0000 7f929a023700 1 --2- 172.21.15.163:0/3874145115 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9278066f30 0x7f92780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:32.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.164+0000 7f929a023700 1 --2- 172.21.15.163:0/3874145115 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9278066f30 0x7f92780693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f92940d0130 tx=0x7f92900091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:32.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.168+0000 7f928a7fc700 1 -- 172.21.15.163:0/3874145115 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f92840b65a0 con 0x7f9294155a20 2023-10-09T06:31:32.354 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.353+0000 7f929b025700 1 -- 172.21.15.163:0/3874145115 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f929404c3c0 con 0x7f9294155a20 2023-10-09T06:31:32.355 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.354+0000 7f928a7fc700 1 -- 172.21.15.163:0/3874145115 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f9284017090 con 0x7f9294155a20 2023-10-09T06:31:32.355 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:32.355 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":31,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:31:29.474739+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:31:25.770415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-09T06:31:26.980879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:31:32.357 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.356+0000 7f929b025700 1 -- 172.21.15.163:0/3874145115 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9278066f30 msgr2=0x7f92780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:32.357 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.356+0000 7f929b025700 1 --2- 172.21.15.163:0/3874145115 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9278066f30 0x7f92780693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f92940d0130 tx=0x7f92900091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.357 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.356+0000 7f929b025700 1 -- 172.21.15.163:0/3874145115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9294155a20 msgr2=0x7f9294155e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:32.358 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.356+0000 7f929b025700 1 --2- 172.21.15.163:0/3874145115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9294155a20 0x7f9294155e90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f928400c980 tx=0x7f9284015000 comp rx=0 tx=0).stop 2023-10-09T06:31:32.358 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.356+0000 7f929b025700 1 -- 172.21.15.163:0/3874145115 shutdown_connections 2023-10-09T06:31:32.358 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.356+0000 7f929b025700 1 --2- 172.21.15.163:0/3874145115 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9278066f30 0x7f92780693e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.358 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.356+0000 7f929b025700 1 --2- 172.21.15.163:0/3874145115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92940c5840 0x7f92940c5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.358 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.356+0000 7f929b025700 1 --2- 172.21.15.163:0/3874145115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9280004810 0x7f9294155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.358 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.356+0000 7f929b025700 1 --2- 172.21.15.163:0/3874145115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9294155a20 0x7f9294155e90 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.358 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.356+0000 7f929b025700 1 -- 172.21.15.163:0/3874145115 >> 172.21.15.163:0/3874145115 conn(0x7f92940d0660 msgr2=0x7f92940b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:32.358 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.356+0000 7f929b025700 1 -- 172.21.15.163:0/3874145115 shutdown_connections 2023-10-09T06:31:32.358 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.356+0000 7f929b025700 1 -- 172.21.15.163:0/3874145115 wait complete. 2023-10-09T06:31:32.370 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-09T06:31:32.370 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:31:32.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.483+0000 7f1f61c91700 1 Processor -- start 2023-10-09T06:31:32.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.483+0000 7f1f61c91700 1 -- start start 2023-10-09T06:31:32.486 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.484+0000 7f1f61c91700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1f5c159890 0x7f1f5c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:32.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.484+0000 7f1f61c91700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f5c159c60 0x7f1f5c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:32.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.484+0000 7f1f61c91700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f5c15a160 con 0x7f1f5c15db10 2023-10-09T06:31:32.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.484+0000 7f1f61c91700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f5c0d13a0 con 0x7f1f5c159890 2023-10-09T06:31:32.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.484+0000 7f1f61c91700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c15e140 con 0x7f1f5c159c60 2023-10-09T06:31:32.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.484+0000 7f1f5b7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1f5c159890 0x7f1f5c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:32.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.484+0000 7f1f5bfff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f1f5c15db10 0x7f1f5c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33880/0 (socket says 172.21.15.163:33880) 2023-10-09T06:31:32.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.484+0000 7f1f5bfff700 1 -- 172.21.15.163:0/3165494716 learned_addr learned my addr 172.21.15.163:0/3165494716 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:32.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.484+0000 7f1f52ffd700 1 --2- 172.21.15.163:0/3165494716 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f5c159c60 0x7f1f5c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:32.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.485+0000 7f1f5b7fe700 1 -- 172.21.15.163:0/3165494716 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f5c159c60 msgr2=0x7f1f5c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:32.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.485+0000 7f1f5b7fe700 1 --2- 172.21.15.163:0/3165494716 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f5c159c60 0x7f1f5c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.485+0000 7f1f5b7fe700 1 -- 172.21.15.163:0/3165494716 >> v1:172.21.15.163:6789/0 conn(0x7f1f5c15db10 legacy=0x7f1f5c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:32.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.485+0000 7f1f5b7fe700 1 -- 172.21.15.163:0/3165494716 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c15a3d0 con 0x7f1f5c159890 2023-10-09T06:31:32.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.485+0000 7f1f5affd700 1 -- 172.21.15.163:0/3165494716 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 70631350 0 0) 0x7f1f5c15a160 con 0x7f1f5c15db10 2023-10-09T06:31:32.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.485+0000 7f1f5b7fe700 1 --2- 172.21.15.163:0/3165494716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f5c159890 0x7f1f5c15abd0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c0098e0 tx=0x7f1f4c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175b5dc781668bc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:32.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.485+0000 7f1f5affd700 1 -- 172.21.15.163:0/3165494716 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c009e80 con 0x7f1f5c159890 2023-10-09T06:31:32.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.485+0000 7f1f5affd700 1 -- 172.21.15.163:0/3165494716 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4c00b860 con 0x7f1f5c159890 2023-10-09T06:31:32.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.485+0000 7f1f5affd700 1 -- 172.21.15.163:0/3165494716 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c00bbe0 con 0x7f1f5c159890 2023-10-09T06:31:32.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.485+0000 7f1f61c91700 1 -- 172.21.15.163:0/3165494716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f5c159890 msgr2=0x7f1f5c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:32.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.485+0000 7f1f61c91700 1 --2- 172.21.15.163:0/3165494716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f5c159890 0x7f1f5c15abd0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c0098e0 tx=0x7f1f4c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.485+0000 7f1f61c91700 1 -- 172.21.15.163:0/3165494716 shutdown_connections 2023-10-09T06:31:32.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.485+0000 7f1f61c91700 1 --2- 172.21.15.163:0/3165494716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f5c159890 0x7f1f5c15abd0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.485+0000 7f1f61c91700 1 --2- 172.21.15.163:0/3165494716 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f5c159c60 0x7f1f5c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.486+0000 7f1f61c91700 1 -- 172.21.15.163:0/3165494716 >> 172.21.15.163:0/3165494716 conn(0x7f1f5c068e20 msgr2=0x7f1f5c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:32.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.486+0000 7f1f61c91700 1 -- 172.21.15.163:0/3165494716 shutdown_connections 2023-10-09T06:31:32.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.486+0000 7f1f61c91700 1 -- 172.21.15.163:0/3165494716 wait complete. 2023-10-09T06:31:32.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.487+0000 7f1f61c91700 1 Processor -- start 2023-10-09T06:31:32.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.487+0000 7f1f61c91700 1 -- start start 2023-10-09T06:31:32.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.487+0000 7f1f61c91700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f5c159890 0x7f1f5c0cf120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:32.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.487+0000 7f1f61c91700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f5c159c60 0x7f1f5c0c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:32.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.487+0000 7f1f61c91700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f5c15db10 0x7f1f5c0cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:32.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.488+0000 7f1f61c91700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c0ce140 con 0x7f1f5c159c60 2023-10-09T06:31:32.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.488+0000 7f1f61c91700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c0b36c0 con 0x7f1f5c159890 2023-10-09T06:31:32.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.488+0000 7f1f61c91700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c0cd600 con 0x7f1f5c15db10 2023-10-09T06:31:32.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.488+0000 7f1f52ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f5c159c60 0x7f1f5c0c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:32.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.488+0000 7f1f52ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f5c159c60 0x7f1f5c0c7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33456/0 (socket says 172.21.15.163:33456) 2023-10-09T06:31:32.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.488+0000 7f1f52ffd700 1 -- 172.21.15.163:0/978770154 learned_addr learned my addr 172.21.15.163:0/978770154 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:32.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.488+0000 7f1f5b7fe700 1 --2- 172.21.15.163:0/978770154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f5c159890 0x7f1f5c0cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:32.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.488+0000 7f1f5bfff700 1 --2- 172.21.15.163:0/978770154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f5c15db10 0x7f1f5c0cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:32.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.488+0000 7f1f52ffd700 1 -- 172.21.15.163:0/978770154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f5c15db10 msgr2=0x7f1f5c0cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:32.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.488+0000 7f1f52ffd700 1 --2- 172.21.15.163:0/978770154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f5c15db10 0x7f1f5c0cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.488+0000 7f1f52ffd700 1 -- 172.21.15.163:0/978770154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f5c159890 msgr2=0x7f1f5c0cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:32.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.488+0000 7f1f52ffd700 1 --2- 172.21.15.163:0/978770154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f5c159890 0x7f1f5c0cf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.488+0000 7f1f52ffd700 1 -- 172.21.15.163:0/978770154 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c009420 con 0x7f1f5c159c60 2023-10-09T06:31:32.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.488+0000 7f1f52ffd700 1 --2- 172.21.15.163:0/978770154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f5c159c60 0x7f1f5c0c7df0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400d890 tx=0x7f1f44008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:32.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.488+0000 7f1f53fff700 1 -- 172.21.15.163:0/978770154 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f440125f0 con 0x7f1f5c159c60 2023-10-09T06:31:32.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.488+0000 7f1f53fff700 1 -- 172.21.15.163:0/978770154 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f44012750 con 0x7f1f5c159c60 2023-10-09T06:31:32.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.488+0000 7f1f61c91700 1 -- 172.21.15.163:0/978770154 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5c0cd8a0 con 0x7f1f5c159c60 2023-10-09T06:31:32.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.488+0000 7f1f53fff700 1 -- 172.21.15.163:0/978770154 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f44012af0 con 0x7f1f5c159c60 2023-10-09T06:31:32.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.488+0000 7f1f61c91700 1 -- 172.21.15.163:0/978770154 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f5c0cde60 con 0x7f1f5c159c60 2023-10-09T06:31:32.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.489+0000 7f1f61c91700 1 -- 172.21.15.163:0/978770154 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f5c04c3c0 con 0x7f1f5c159c60 2023-10-09T06:31:32.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.490+0000 7f1f53fff700 1 -- 172.21.15.163:0/978770154 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1f4401e460 con 0x7f1f5c159c60 2023-10-09T06:31:32.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.490+0000 7f1f53fff700 1 --2- 172.21.15.163:0/978770154 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1f3c066f30 0x7f1f3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:32.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.490+0000 7f1f53fff700 1 -- 172.21.15.163:0/978770154 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1f44084a80 con 0x7f1f5c159c60 2023-10-09T06:31:32.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.490+0000 7f1f5b7fe700 1 --2- 172.21.15.163:0/978770154 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1f3c066f30 0x7f1f3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:32.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.491+0000 7f1f5b7fe700 1 --2- 172.21.15.163:0/978770154 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1f3c066f30 0x7f1f3c0693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c0091e0 tx=0x7f1f4c017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:32.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.494+0000 7f1f53fff700 1 -- 172.21.15.163:0/978770154 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f44055780 con 0x7f1f5c159c60 2023-10-09T06:31:32.611 INFO:tasks.ceph.mds.d.smithi181.stderr:2023-10-09T06:31:32.611+0000 7fd505e65700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-09T06:31:32.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.675+0000 7f1f61c91700 1 -- 172.21.15.163:0/978770154 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1f5c0b3800 con 0x7f1f3c066f30 2023-10-09T06:31:32.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.684+0000 7f1f53fff700 1 -- 172.21.15.163:0/978770154 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276890 (secure 0 0 0) 0x7f1f5c0b3800 con 0x7f1f3c066f30 2023-10-09T06:31:32.688 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:32.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.687+0000 7f1f61c91700 1 -- 172.21.15.163:0/978770154 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1f3c066f30 msgr2=0x7f1f3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:32.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.687+0000 7f1f61c91700 1 --2- 172.21.15.163:0/978770154 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1f3c066f30 0x7f1f3c0693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c0091e0 tx=0x7f1f4c017000 comp rx=0 tx=0).stop 2023-10-09T06:31:32.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.687+0000 7f1f61c91700 1 -- 172.21.15.163:0/978770154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f5c159c60 msgr2=0x7f1f5c0c7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:32.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.687+0000 7f1f61c91700 1 --2- 172.21.15.163:0/978770154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f5c159c60 0x7f1f5c0c7df0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400d890 tx=0x7f1f44008040 comp rx=0 tx=0).stop 2023-10-09T06:31:32.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.687+0000 7f1f61c91700 1 -- 172.21.15.163:0/978770154 shutdown_connections 2023-10-09T06:31:32.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.687+0000 7f1f61c91700 1 --2- 172.21.15.163:0/978770154 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1f3c066f30 0x7f1f3c0693e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.687+0000 7f1f61c91700 1 --2- 172.21.15.163:0/978770154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f5c15db10 0x7f1f5c0cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.687+0000 7f1f61c91700 1 --2- 172.21.15.163:0/978770154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f5c159890 0x7f1f5c0cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.687+0000 7f1f61c91700 1 --2- 172.21.15.163:0/978770154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f5c159c60 0x7f1f5c0c7df0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.687+0000 7f1f61c91700 1 -- 172.21.15.163:0/978770154 >> 172.21.15.163:0/978770154 conn(0x7f1f5c068e20 msgr2=0x7f1f5c14d680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:32.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.688+0000 7f1f61c91700 1 -- 172.21.15.163:0/978770154 shutdown_connections 2023-10-09T06:31:32.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.688+0000 7f1f61c91700 1 -- 172.21.15.163:0/978770154 wait complete. 2023-10-09T06:31:32.691 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:31:32.701 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2023-10-09T06:31:32.305559+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382236,"kb_used_data":3932,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592484,"statfs":{"total":773094113280,"available":770654703616,"internally_reserved":0,"allocated":4026368,"data_stored":2248772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.682389"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.497592+0000","last_change":"2023-10-09T06:31:28.495980+0000","last_active":"2023-10-09T06:31:28.497592+0000","last_peered":"2023-10-09T06:31:28.497592+0000","last_clean":"2023-10-09T06:31:28.497592+0000","last_became_active":"2023-10-09T06:31:28.495707+0000","last_became_peered":"2023-10-09T06:31:28.495707+0000","last_unstale":"2023-10-09T06:31:28.497592+0000","last_undegraded":"2023-10-09T06:31:28.497592+0000","last_fullsized":"2023-10-09T06:31:28.497592+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:06:14.186814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.484754+0000","last_change":"2023-10-09T06:31:28.483133+0000","last_active":"2023-10-09T06:31:28.484754+0000","last_peered":"2023-10-09T06:31:28.484754+0000","last_clean":"2023-10-09T06:31:28.484754+0000","last_became_active":"2023-10-09T06:31:28.482856+0000","last_became_peered":"2023-10-09T06:31:28.482856+0000","last_unstale":"2023-10-09T06:31:28.484754+0000","last_undegraded":"2023-10-09T06:31:28.484754+0000","last_fullsized":"2023-10-09T06:31:28.484754+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:37:09.825142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502573+0000","last_change":"2023-10-09T06:31:28.498279+0000","last_active":"2023-10-09T06:31:28.502573+0000","last_peered":"2023-10-09T06:31:28.502573+0000","last_clean":"2023-10-09T06:31:28.502573+0000","last_became_active":"2023-10-09T06:31:28.497687+0000","last_became_peered":"2023-10-09T06:31:28.497687+0000","last_unstale":"2023-10-09T06:31:28.502573+0000","last_undegraded":"2023-10-09T06:31:28.502573+0000","last_fullsized":"2023-10-09T06:31:28.502573+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:57:49.644159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502794+0000","last_change":"2023-10-09T06:31:28.493773+0000","last_active":"2023-10-09T06:31:28.502794+0000","last_peered":"2023-10-09T06:31:28.502794+0000","last_clean":"2023-10-09T06:31:28.502794+0000","last_became_active":"2023-10-09T06:31:28.493299+0000","last_became_peered":"2023-10-09T06:31:28.493299+0000","last_unstale":"2023-10-09T06:31:28.502794+0000","last_undegraded":"2023-10-09T06:31:28.502794+0000","last_fullsized":"2023-10-09T06:31:28.502794+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:04:57.646593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502910+0000","last_change":"2023-10-09T06:31:28.495674+0000","last_active":"2023-10-09T06:31:28.502910+0000","last_peered":"2023-10-09T06:31:28.502910+0000","last_clean":"2023-10-09T06:31:28.502910+0000","last_became_active":"2023-10-09T06:31:28.495347+0000","last_became_peered":"2023-10-09T06:31:28.495347+0000","last_unstale":"2023-10-09T06:31:28.502910+0000","last_undegraded":"2023-10-09T06:31:28.502910+0000","last_fullsized":"2023-10-09T06:31:28.502910+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:55:28.765990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.500246+0000","last_change":"2023-10-09T06:31:28.493198+0000","last_active":"2023-10-09T06:31:28.500246+0000","last_peered":"2023-10-09T06:31:28.500246+0000","last_clean":"2023-10-09T06:31:28.500246+0000","last_became_active":"2023-10-09T06:31:28.492946+0000","last_became_peered":"2023-10-09T06:31:28.492946+0000","last_unstale":"2023-10-09T06:31:28.500246+0000","last_undegraded":"2023-10-09T06:31:28.500246+0000","last_fullsized":"2023-10-09T06:31:28.500246+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:04:20.370118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.497319+0000","last_change":"2023-10-09T06:31:28.490883+0000","last_active":"2023-10-09T06:31:28.497319+0000","last_peered":"2023-10-09T06:31:28.497319+0000","last_clean":"2023-10-09T06:31:28.497319+0000","last_became_active":"2023-10-09T06:31:28.490521+0000","last_became_peered":"2023-10-09T06:31:28.490521+0000","last_unstale":"2023-10-09T06:31:28.497319+0000","last_undegraded":"2023-10-09T06:31:28.497319+0000","last_fullsized":"2023-10-09T06:31:28.497319+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:43:43.515688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.496962+0000","last_change":"2023-10-09T06:31:28.488826+0000","last_active":"2023-10-09T06:31:28.496962+0000","last_peered":"2023-10-09T06:31:28.496962+0000","last_clean":"2023-10-09T06:31:28.496962+0000","last_became_active":"2023-10-09T06:31:28.488556+0000","last_became_peered":"2023-10-09T06:31:28.488556+0000","last_unstale":"2023-10-09T06:31:28.496962+0000","last_undegraded":"2023-10-09T06:31:28.496962+0000","last_fullsized":"2023-10-09T06:31:28.496962+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:27:40.174948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.494220+0000","last_change":"2023-10-09T06:31:28.491340+0000","last_active":"2023-10-09T06:31:28.494220+0000","last_peered":"2023-10-09T06:31:28.494220+0000","last_clean":"2023-10-09T06:31:28.494220+0000","last_became_active":"2023-10-09T06:31:28.491087+0000","last_became_peered":"2023-10-09T06:31:28.491087+0000","last_unstale":"2023-10-09T06:31:28.494220+0000","last_undegraded":"2023-10-09T06:31:28.494220+0000","last_fullsized":"2023-10-09T06:31:28.494220+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:04:57.646593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502930+0000","last_change":"2023-10-09T06:31:28.495914+0000","last_active":"2023-10-09T06:31:28.502930+0000","last_peered":"2023-10-09T06:31:28.502930+0000","last_clean":"2023-10-09T06:31:28.502930+0000","last_became_active":"2023-10-09T06:31:28.495661+0000","last_became_peered":"2023-10-09T06:31:28.495661+0000","last_unstale":"2023-10-09T06:31:28.502930+0000","last_undegraded":"2023-10-09T06:31:28.502930+0000","last_fullsized":"2023-10-09T06:31:28.502930+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:27:18.901979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.497281+0000","last_change":"2023-10-09T06:31:28.490597+0000","last_active":"2023-10-09T06:31:28.497281+0000","last_peered":"2023-10-09T06:31:28.497281+0000","last_clean":"2023-10-09T06:31:28.497281+0000","last_became_active":"2023-10-09T06:31:28.490338+0000","last_became_peered":"2023-10-09T06:31:28.490338+0000","last_unstale":"2023-10-09T06:31:28.497281+0000","last_undegraded":"2023-10-09T06:31:28.497281+0000","last_fullsized":"2023-10-09T06:31:28.497281+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:14:19.995870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.498699+0000","last_change":"2023-10-09T06:31:28.496333+0000","last_active":"2023-10-09T06:31:28.498699+0000","last_peered":"2023-10-09T06:31:28.498699+0000","last_clean":"2023-10-09T06:31:28.498699+0000","last_became_active":"2023-10-09T06:31:28.495820+0000","last_became_peered":"2023-10-09T06:31:28.495820+0000","last_unstale":"2023-10-09T06:31:28.498699+0000","last_undegraded":"2023-10-09T06:31:28.498699+0000","last_fullsized":"2023-10-09T06:31:28.498699+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:04:20.370118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.493013+0000","last_change":"2023-10-09T06:31:28.489488+0000","last_active":"2023-10-09T06:31:28.493013+0000","last_peered":"2023-10-09T06:31:28.493013+0000","last_clean":"2023-10-09T06:31:28.493013+0000","last_became_active":"2023-10-09T06:31:28.488817+0000","last_became_peered":"2023-10-09T06:31:28.488817+0000","last_unstale":"2023-10-09T06:31:28.493013+0000","last_undegraded":"2023-10-09T06:31:28.493013+0000","last_fullsized":"2023-10-09T06:31:28.493013+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:28:37.013947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.500141+0000","last_change":"2023-10-09T06:31:28.489437+0000","last_active":"2023-10-09T06:31:28.500141+0000","last_peered":"2023-10-09T06:31:28.500141+0000","last_clean":"2023-10-09T06:31:28.500141+0000","last_became_active":"2023-10-09T06:31:28.489185+0000","last_became_peered":"2023-10-09T06:31:28.489185+0000","last_unstale":"2023-10-09T06:31:28.500141+0000","last_undegraded":"2023-10-09T06:31:28.500141+0000","last_fullsized":"2023-10-09T06:31:28.500141+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:27:40.174948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502593+0000","last_change":"2023-10-09T06:31:28.498393+0000","last_active":"2023-10-09T06:31:28.502593+0000","last_peered":"2023-10-09T06:31:28.502593+0000","last_clean":"2023-10-09T06:31:28.502593+0000","last_became_active":"2023-10-09T06:31:28.497388+0000","last_became_peered":"2023-10-09T06:31:28.497388+0000","last_unstale":"2023-10-09T06:31:28.502593+0000","last_undegraded":"2023-10-09T06:31:28.502593+0000","last_fullsized":"2023-10-09T06:31:28.502593+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:43:43.515688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.493239+0000","last_change":"2023-10-09T06:31:28.485519+0000","last_active":"2023-10-09T06:31:28.493239+0000","last_peered":"2023-10-09T06:31:28.493239+0000","last_clean":"2023-10-09T06:31:28.493239+0000","last_became_active":"2023-10-09T06:31:28.485243+0000","last_became_peered":"2023-10-09T06:31:28.485243+0000","last_unstale":"2023-10-09T06:31:28.493239+0000","last_undegraded":"2023-10-09T06:31:28.493239+0000","last_fullsized":"2023-10-09T06:31:28.493239+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:04:27.115048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.497586+0000","last_change":"2023-10-09T06:31:28.491917+0000","last_active":"2023-10-09T06:31:28.497586+0000","last_peered":"2023-10-09T06:31:28.497586+0000","last_clean":"2023-10-09T06:31:28.497586+0000","last_became_active":"2023-10-09T06:31:28.491649+0000","last_became_peered":"2023-10-09T06:31:28.491649+0000","last_unstale":"2023-10-09T06:31:28.497586+0000","last_undegraded":"2023-10-09T06:31:28.497586+0000","last_fullsized":"2023-10-09T06:31:28.497586+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:27:18.901979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.500207+0000","last_change":"2023-10-09T06:31:28.491824+0000","last_active":"2023-10-09T06:31:28.500207+0000","last_peered":"2023-10-09T06:31:28.500207+0000","last_clean":"2023-10-09T06:31:28.500207+0000","last_became_active":"2023-10-09T06:31:28.491632+0000","last_became_peered":"2023-10-09T06:31:28.491632+0000","last_unstale":"2023-10-09T06:31:28.500207+0000","last_undegraded":"2023-10-09T06:31:28.500207+0000","last_fullsized":"2023-10-09T06:31:28.500207+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:24:49.566571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.500206+0000","last_change":"2023-10-09T06:31:28.491813+0000","last_active":"2023-10-09T06:31:28.500206+0000","last_peered":"2023-10-09T06:31:28.500206+0000","last_clean":"2023-10-09T06:31:28.500206+0000","last_became_active":"2023-10-09T06:31:28.491503+0000","last_became_peered":"2023-10-09T06:31:28.491503+0000","last_unstale":"2023-10-09T06:31:28.500206+0000","last_undegraded":"2023-10-09T06:31:28.500206+0000","last_fullsized":"2023-10-09T06:31:28.500206+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:43:43.515688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502852+0000","last_change":"2023-10-09T06:31:28.495453+0000","last_active":"2023-10-09T06:31:28.502852+0000","last_peered":"2023-10-09T06:31:28.502852+0000","last_clean":"2023-10-09T06:31:28.502852+0000","last_became_active":"2023-10-09T06:31:28.495221+0000","last_became_peered":"2023-10-09T06:31:28.495221+0000","last_unstale":"2023-10-09T06:31:28.502852+0000","last_undegraded":"2023-10-09T06:31:28.502852+0000","last_fullsized":"2023-10-09T06:31:28.502852+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:16:48.908526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.490965+0000","last_change":"2023-10-09T06:31:28.483955+0000","last_active":"2023-10-09T06:31:28.490965+0000","last_peered":"2023-10-09T06:31:28.490965+0000","last_clean":"2023-10-09T06:31:28.490965+0000","last_became_active":"2023-10-09T06:31:28.483688+0000","last_became_peered":"2023-10-09T06:31:28.483688+0000","last_unstale":"2023-10-09T06:31:28.490965+0000","last_undegraded":"2023-10-09T06:31:28.490965+0000","last_fullsized":"2023-10-09T06:31:28.490965+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:06:14.186814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.491942+0000","last_change":"2023-10-09T06:31:28.486094+0000","last_active":"2023-10-09T06:31:28.491942+0000","last_peered":"2023-10-09T06:31:28.491942+0000","last_clean":"2023-10-09T06:31:28.491942+0000","last_became_active":"2023-10-09T06:31:28.485819+0000","last_became_peered":"2023-10-09T06:31:28.485819+0000","last_unstale":"2023-10-09T06:31:28.491942+0000","last_undegraded":"2023-10-09T06:31:28.491942+0000","last_fullsized":"2023-10-09T06:31:28.491942+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:46:46.193647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.486329+0000","last_change":"2023-10-09T06:31:28.484836+0000","last_active":"2023-10-09T06:31:28.486329+0000","last_peered":"2023-10-09T06:31:28.486329+0000","last_clean":"2023-10-09T06:31:28.486329+0000","last_became_active":"2023-10-09T06:31:28.484552+0000","last_became_peered":"2023-10-09T06:31:28.484552+0000","last_unstale":"2023-10-09T06:31:28.486329+0000","last_undegraded":"2023-10-09T06:31:28.486329+0000","last_fullsized":"2023-10-09T06:31:28.486329+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:03:19.212337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.497139+0000","last_change":"2023-10-09T06:31:28.492708+0000","last_active":"2023-10-09T06:31:28.497139+0000","last_peered":"2023-10-09T06:31:28.497139+0000","last_clean":"2023-10-09T06:31:28.497139+0000","last_became_active":"2023-10-09T06:31:28.492363+0000","last_became_peered":"2023-10-09T06:31:28.492363+0000","last_unstale":"2023-10-09T06:31:28.497139+0000","last_undegraded":"2023-10-09T06:31:28.497139+0000","last_fullsized":"2023-10-09T06:31:28.497139+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:27:40.174948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502870+0000","last_change":"2023-10-09T06:31:28.494485+0000","last_active":"2023-10-09T06:31:28.502870+0000","last_peered":"2023-10-09T06:31:28.502870+0000","last_clean":"2023-10-09T06:31:28.502870+0000","last_became_active":"2023-10-09T06:31:28.494246+0000","last_became_peered":"2023-10-09T06:31:28.494246+0000","last_unstale":"2023-10-09T06:31:28.502870+0000","last_undegraded":"2023-10-09T06:31:28.502870+0000","last_fullsized":"2023-10-09T06:31:28.502870+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:04:19.137467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.494546+0000","last_change":"2023-10-09T06:31:28.485577+0000","last_active":"2023-10-09T06:31:28.494546+0000","last_peered":"2023-10-09T06:31:28.494546+0000","last_clean":"2023-10-09T06:31:28.494546+0000","last_became_active":"2023-10-09T06:31:28.485337+0000","last_became_peered":"2023-10-09T06:31:28.485337+0000","last_unstale":"2023-10-09T06:31:28.494546+0000","last_undegraded":"2023-10-09T06:31:28.494546+0000","last_fullsized":"2023-10-09T06:31:28.494546+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:07:19.040388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502897+0000","last_change":"2023-10-09T06:31:28.495012+0000","last_active":"2023-10-09T06:31:28.502897+0000","last_peered":"2023-10-09T06:31:28.502897+0000","last_clean":"2023-10-09T06:31:28.502897+0000","last_became_active":"2023-10-09T06:31:28.494745+0000","last_became_peered":"2023-10-09T06:31:28.494745+0000","last_unstale":"2023-10-09T06:31:28.502897+0000","last_undegraded":"2023-10-09T06:31:28.502897+0000","last_fullsized":"2023-10-09T06:31:28.502897+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:11:09.342940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.498413+0000","last_change":"2023-10-09T06:31:28.492881+0000","last_active":"2023-10-09T06:31:28.498413+0000","last_peered":"2023-10-09T06:31:28.498413+0000","last_clean":"2023-10-09T06:31:28.498413+0000","last_became_active":"2023-10-09T06:31:28.492572+0000","last_became_peered":"2023-10-09T06:31:28.492572+0000","last_unstale":"2023-10-09T06:31:28.498413+0000","last_undegraded":"2023-10-09T06:31:28.498413+0000","last_fullsized":"2023-10-09T06:31:28.498413+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:19:42.372022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.493295+0000","last_change":"2023-10-09T06:31:28.488456+0000","last_active":"2023-10-09T06:31:28.493295+0000","last_peered":"2023-10-09T06:31:28.493295+0000","last_clean":"2023-10-09T06:31:28.493295+0000","last_became_active":"2023-10-09T06:31:28.485910+0000","last_became_peered":"2023-10-09T06:31:28.485910+0000","last_unstale":"2023-10-09T06:31:28.493295+0000","last_undegraded":"2023-10-09T06:31:28.493295+0000","last_fullsized":"2023-10-09T06:31:28.493295+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:21:46.563134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.497017+0000","last_change":"2023-10-09T06:31:28.490889+0000","last_active":"2023-10-09T06:31:28.497017+0000","last_peered":"2023-10-09T06:31:28.497017+0000","last_clean":"2023-10-09T06:31:28.497017+0000","last_became_active":"2023-10-09T06:31:28.490616+0000","last_became_peered":"2023-10-09T06:31:28.490616+0000","last_unstale":"2023-10-09T06:31:28.497017+0000","last_undegraded":"2023-10-09T06:31:28.497017+0000","last_fullsized":"2023-10-09T06:31:28.497017+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:04:19.137467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.494098+0000","last_change":"2023-10-09T06:31:28.488564+0000","last_active":"2023-10-09T06:31:28.494098+0000","last_peered":"2023-10-09T06:31:28.494098+0000","last_clean":"2023-10-09T06:31:28.494098+0000","last_became_active":"2023-10-09T06:31:28.488290+0000","last_became_peered":"2023-10-09T06:31:28.488290+0000","last_unstale":"2023-10-09T06:31:28.494098+0000","last_undegraded":"2023-10-09T06:31:28.494098+0000","last_fullsized":"2023-10-09T06:31:28.494098+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:17:14.862943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.498364+0000","last_change":"2023-10-09T06:31:28.494811+0000","last_active":"2023-10-09T06:31:28.498364+0000","last_peered":"2023-10-09T06:31:28.498364+0000","last_clean":"2023-10-09T06:31:28.498364+0000","last_became_active":"2023-10-09T06:31:28.494310+0000","last_became_peered":"2023-10-09T06:31:28.494310+0000","last_unstale":"2023-10-09T06:31:28.498364+0000","last_undegraded":"2023-10-09T06:31:28.498364+0000","last_fullsized":"2023-10-09T06:31:28.498364+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:21:46.563134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.489218+0000","last_change":"2023-10-09T06:31:28.485466+0000","last_active":"2023-10-09T06:31:28.489218+0000","last_peered":"2023-10-09T06:31:28.489218+0000","last_clean":"2023-10-09T06:31:28.489218+0000","last_became_active":"2023-10-09T06:31:28.485170+0000","last_became_peered":"2023-10-09T06:31:28.485170+0000","last_unstale":"2023-10-09T06:31:28.489218+0000","last_undegraded":"2023-10-09T06:31:28.489218+0000","last_fullsized":"2023-10-09T06:31:28.489218+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:32:49.816742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'4","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.649786+0000","last_change":"2023-10-09T06:31:27.639225+0000","last_active":"2023-10-09T06:31:28.649786+0000","last_peered":"2023-10-09T06:31:28.649786+0000","last_clean":"2023-10-09T06:31:28.649786+0000","last_became_active":"2023-10-09T06:31:27.638934+0000","last_became_peered":"2023-10-09T06:31:27.638934+0000","last_unstale":"2023-10-09T06:31:28.649786+0000","last_undegraded":"2023-10-09T06:31:28.649786+0000","last_fullsized":"2023-10-09T06:31:28.649786+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:28:44.546879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.496951+0000","last_change":"2023-10-09T06:31:28.490147+0000","last_active":"2023-10-09T06:31:28.496951+0000","last_peered":"2023-10-09T06:31:28.496951+0000","last_clean":"2023-10-09T06:31:28.496951+0000","last_became_active":"2023-10-09T06:31:28.489622+0000","last_became_peered":"2023-10-09T06:31:28.489622+0000","last_unstale":"2023-10-09T06:31:28.496951+0000","last_undegraded":"2023-10-09T06:31:28.496951+0000","last_fullsized":"2023-10-09T06:31:28.496951+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:11:09.342940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.647123+0000","last_change":"2023-10-09T06:31:27.646459+0000","last_active":"2023-10-09T06:31:28.647123+0000","last_peered":"2023-10-09T06:31:28.647123+0000","last_clean":"2023-10-09T06:31:28.647123+0000","last_became_active":"2023-10-09T06:31:27.646114+0000","last_became_peered":"2023-10-09T06:31:27.646114+0000","last_unstale":"2023-10-09T06:31:28.647123+0000","last_undegraded":"2023-10-09T06:31:28.647123+0000","last_fullsized":"2023-10-09T06:31:28.647123+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:04:16.068904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.491714+0000","last_change":"2023-10-09T06:31:28.484818+0000","last_active":"2023-10-09T06:31:28.491714+0000","last_peered":"2023-10-09T06:31:28.491714+0000","last_clean":"2023-10-09T06:31:28.491714+0000","last_became_active":"2023-10-09T06:31:28.484459+0000","last_became_peered":"2023-10-09T06:31:28.484459+0000","last_unstale":"2023-10-09T06:31:28.491714+0000","last_undegraded":"2023-10-09T06:31:28.491714+0000","last_fullsized":"2023-10-09T06:31:28.491714+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:24:53.175922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.652351+0000","last_change":"2023-10-09T06:31:27.641559+0000","last_active":"2023-10-09T06:31:28.652351+0000","last_peered":"2023-10-09T06:31:28.652351+0000","last_clean":"2023-10-09T06:31:28.652351+0000","last_became_active":"2023-10-09T06:31:27.641319+0000","last_became_peered":"2023-10-09T06:31:27.641319+0000","last_unstale":"2023-10-09T06:31:28.652351+0000","last_undegraded":"2023-10-09T06:31:28.652351+0000","last_fullsized":"2023-10-09T06:31:28.652351+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:11:08.334515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502860+0000","last_change":"2023-10-09T06:31:28.494483+0000","last_active":"2023-10-09T06:31:28.502860+0000","last_peered":"2023-10-09T06:31:28.502860+0000","last_clean":"2023-10-09T06:31:28.502860+0000","last_became_active":"2023-10-09T06:31:28.493408+0000","last_became_peered":"2023-10-09T06:31:28.493408+0000","last_unstale":"2023-10-09T06:31:28.502860+0000","last_undegraded":"2023-10-09T06:31:28.502860+0000","last_fullsized":"2023-10-09T06:31:28.502860+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:46:46.193647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.654501+0000","last_change":"2023-10-09T06:31:27.639485+0000","last_active":"2023-10-09T06:31:28.654501+0000","last_peered":"2023-10-09T06:31:28.654501+0000","last_clean":"2023-10-09T06:31:28.654501+0000","last_became_active":"2023-10-09T06:31:27.639233+0000","last_became_peered":"2023-10-09T06:31:27.639233+0000","last_unstale":"2023-10-09T06:31:28.654501+0000","last_undegraded":"2023-10-09T06:31:28.654501+0000","last_fullsized":"2023-10-09T06:31:28.654501+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:15:59.697671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.498298+0000","last_change":"2023-10-09T06:31:28.492731+0000","last_active":"2023-10-09T06:31:28.498298+0000","last_peered":"2023-10-09T06:31:28.498298+0000","last_clean":"2023-10-09T06:31:28.498298+0000","last_became_active":"2023-10-09T06:31:28.492472+0000","last_became_peered":"2023-10-09T06:31:28.492472+0000","last_unstale":"2023-10-09T06:31:28.498298+0000","last_undegraded":"2023-10-09T06:31:28.498298+0000","last_fullsized":"2023-10-09T06:31:28.498298+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:04:57.646593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.652145+0000","last_change":"2023-10-09T06:31:27.640688+0000","last_active":"2023-10-09T06:31:28.652145+0000","last_peered":"2023-10-09T06:31:28.652145+0000","last_clean":"2023-10-09T06:31:28.652145+0000","last_became_active":"2023-10-09T06:31:27.640484+0000","last_became_peered":"2023-10-09T06:31:27.640484+0000","last_unstale":"2023-10-09T06:31:28.652145+0000","last_undegraded":"2023-10-09T06:31:28.652145+0000","last_fullsized":"2023-10-09T06:31:28.652145+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:45:32.742331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.498463+0000","last_change":"2023-10-09T06:31:28.493835+0000","last_active":"2023-10-09T06:31:28.498463+0000","last_peered":"2023-10-09T06:31:28.498463+0000","last_clean":"2023-10-09T06:31:28.498463+0000","last_became_active":"2023-10-09T06:31:28.493561+0000","last_became_peered":"2023-10-09T06:31:28.493561+0000","last_unstale":"2023-10-09T06:31:28.498463+0000","last_undegraded":"2023-10-09T06:31:28.498463+0000","last_fullsized":"2023-10-09T06:31:28.498463+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:07:19.040388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.651751+0000","last_change":"2023-10-09T06:31:27.635744+0000","last_active":"2023-10-09T06:31:28.651751+0000","last_peered":"2023-10-09T06:31:28.651751+0000","last_clean":"2023-10-09T06:31:28.651751+0000","last_became_active":"2023-10-09T06:31:27.635463+0000","last_became_peered":"2023-10-09T06:31:27.635463+0000","last_unstale":"2023-10-09T06:31:28.651751+0000","last_undegraded":"2023-10-09T06:31:28.651751+0000","last_fullsized":"2023-10-09T06:31:28.651751+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:55:27.757566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.487549+0000","last_change":"2023-10-09T06:31:28.481016+0000","last_active":"2023-10-09T06:31:28.487549+0000","last_peered":"2023-10-09T06:31:28.487549+0000","last_clean":"2023-10-09T06:31:28.487549+0000","last_became_active":"2023-10-09T06:31:28.480735+0000","last_became_peered":"2023-10-09T06:31:28.480735+0000","last_unstale":"2023-10-09T06:31:28.487549+0000","last_undegraded":"2023-10-09T06:31:28.487549+0000","last_fullsized":"2023-10-09T06:31:28.487549+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:32:49.816742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.479458+0000","last_change":"2023-10-09T06:31:27.639056+0000","last_active":"2023-10-09T06:31:28.479458+0000","last_peered":"2023-10-09T06:31:28.479458+0000","last_clean":"2023-10-09T06:31:28.479458+0000","last_became_active":"2023-10-09T06:31:27.638826+0000","last_became_peered":"2023-10-09T06:31:27.638826+0000","last_unstale":"2023-10-09T06:31:28.479458+0000","last_undegraded":"2023-10-09T06:31:28.479458+0000","last_fullsized":"2023-10-09T06:31:28.479458+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:21:40.094599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502821+0000","last_change":"2023-10-09T06:31:28.494189+0000","last_active":"2023-10-09T06:31:28.502821+0000","last_peered":"2023-10-09T06:31:28.502821+0000","last_clean":"2023-10-09T06:31:28.502821+0000","last_became_active":"2023-10-09T06:31:28.493914+0000","last_became_peered":"2023-10-09T06:31:28.493914+0000","last_unstale":"2023-10-09T06:31:28.502821+0000","last_undegraded":"2023-10-09T06:31:28.502821+0000","last_fullsized":"2023-10-09T06:31:28.502821+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:24:53.175922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.478957+0000","last_change":"2023-10-09T06:31:27.633322+0000","last_active":"2023-10-09T06:31:28.478957+0000","last_peered":"2023-10-09T06:31:28.478957+0000","last_clean":"2023-10-09T06:31:28.478957+0000","last_became_active":"2023-10-09T06:31:27.633016+0000","last_became_peered":"2023-10-09T06:31:27.633016+0000","last_unstale":"2023-10-09T06:31:28.478957+0000","last_undegraded":"2023-10-09T06:31:28.478957+0000","last_fullsized":"2023-10-09T06:31:28.478957+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:46:49.848435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.505257+0000","last_change":"2023-10-09T06:31:28.500793+0000","last_active":"2023-10-09T06:31:28.505257+0000","last_peered":"2023-10-09T06:31:28.505257+0000","last_clean":"2023-10-09T06:31:28.505257+0000","last_became_active":"2023-10-09T06:31:28.500515+0000","last_became_peered":"2023-10-09T06:31:28.500515+0000","last_unstale":"2023-10-09T06:31:28.505257+0000","last_undegraded":"2023-10-09T06:31:28.505257+0000","last_fullsized":"2023-10-09T06:31:28.505257+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:01:45.048047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.480201+0000","last_change":"2023-10-09T06:31:27.643456+0000","last_active":"2023-10-09T06:31:28.480201+0000","last_peered":"2023-10-09T06:31:28.480201+0000","last_clean":"2023-10-09T06:31:28.480201+0000","last_became_active":"2023-10-09T06:31:27.643271+0000","last_became_peered":"2023-10-09T06:31:27.643271+0000","last_unstale":"2023-10-09T06:31:28.480201+0000","last_undegraded":"2023-10-09T06:31:28.480201+0000","last_fullsized":"2023-10-09T06:31:28.480201+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:43:54.672002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.500111+0000","last_change":"2023-10-09T06:31:28.491099+0000","last_active":"2023-10-09T06:31:28.500111+0000","last_peered":"2023-10-09T06:31:28.500111+0000","last_clean":"2023-10-09T06:31:28.500111+0000","last_became_active":"2023-10-09T06:31:28.490841+0000","last_became_peered":"2023-10-09T06:31:28.490841+0000","last_unstale":"2023-10-09T06:31:28.500111+0000","last_undegraded":"2023-10-09T06:31:28.500111+0000","last_fullsized":"2023-10-09T06:31:28.500111+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:57:49.644159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.476831+0000","last_change":"2023-10-09T06:31:27.640486+0000","last_active":"2023-10-09T06:31:28.476831+0000","last_peered":"2023-10-09T06:31:28.476831+0000","last_clean":"2023-10-09T06:31:28.476831+0000","last_became_active":"2023-10-09T06:31:27.640218+0000","last_became_peered":"2023-10-09T06:31:27.640218+0000","last_unstale":"2023-10-09T06:31:28.476831+0000","last_undegraded":"2023-10-09T06:31:28.476831+0000","last_fullsized":"2023-10-09T06:31:28.476831+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:10:57.417447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.491689+0000","last_change":"2023-10-09T06:31:28.484559+0000","last_active":"2023-10-09T06:31:28.491689+0000","last_peered":"2023-10-09T06:31:28.491689+0000","last_clean":"2023-10-09T06:31:28.491689+0000","last_became_active":"2023-10-09T06:31:28.484319+0000","last_became_peered":"2023-10-09T06:31:28.484319+0000","last_unstale":"2023-10-09T06:31:28.491689+0000","last_undegraded":"2023-10-09T06:31:28.491689+0000","last_fullsized":"2023-10-09T06:31:28.491689+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:38:14.098330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"30'3","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.649566+0000","last_change":"2023-10-09T06:31:27.646080+0000","last_active":"2023-10-09T06:31:28.649566+0000","last_peered":"2023-10-09T06:31:28.649566+0000","last_clean":"2023-10-09T06:31:28.649566+0000","last_became_active":"2023-10-09T06:31:27.645829+0000","last_became_peered":"2023-10-09T06:31:27.645829+0000","last_unstale":"2023-10-09T06:31:28.649566+0000","last_undegraded":"2023-10-09T06:31:28.649566+0000","last_fullsized":"2023-10-09T06:31:28.649566+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:06:13.178390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.499831+0000","last_change":"2023-10-09T06:31:28.497844+0000","last_active":"2023-10-09T06:31:28.499831+0000","last_peered":"2023-10-09T06:31:28.499831+0000","last_clean":"2023-10-09T06:31:28.499831+0000","last_became_active":"2023-10-09T06:31:28.497604+0000","last_became_peered":"2023-10-09T06:31:28.497604+0000","last_unstale":"2023-10-09T06:31:28.499831+0000","last_undegraded":"2023-10-09T06:31:28.499831+0000","last_fullsized":"2023-10-09T06:31:28.499831+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:24:49.566571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"30'6","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.654595+0000","last_change":"2023-10-09T06:31:27.645570+0000","last_active":"2023-10-09T06:31:28.654595+0000","last_peered":"2023-10-09T06:31:28.654595+0000","last_clean":"2023-10-09T06:31:28.654595+0000","last_became_active":"2023-10-09T06:31:27.645359+0000","last_became_peered":"2023-10-09T06:31:27.645359+0000","last_unstale":"2023-10-09T06:31:28.654595+0000","last_undegraded":"2023-10-09T06:31:28.654595+0000","last_fullsized":"2023-10-09T06:31:28.654595+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:06:13.178390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502816+0000","last_change":"2023-10-09T06:31:28.494111+0000","last_active":"2023-10-09T06:31:28.502816+0000","last_peered":"2023-10-09T06:31:28.502816+0000","last_clean":"2023-10-09T06:31:28.502816+0000","last_became_active":"2023-10-09T06:31:28.493872+0000","last_became_peered":"2023-10-09T06:31:28.493872+0000","last_unstale":"2023-10-09T06:31:28.502816+0000","last_undegraded":"2023-10-09T06:31:28.502816+0000","last_fullsized":"2023-10-09T06:31:28.502816+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:18:22.705769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.649372+0000","last_change":"2023-10-09T06:31:27.634344+0000","last_active":"2023-10-09T06:31:28.649372+0000","last_peered":"2023-10-09T06:31:28.649372+0000","last_clean":"2023-10-09T06:31:28.649372+0000","last_became_active":"2023-10-09T06:31:27.634108+0000","last_became_peered":"2023-10-09T06:31:27.634108+0000","last_unstale":"2023-10-09T06:31:28.649372+0000","last_undegraded":"2023-10-09T06:31:28.649372+0000","last_fullsized":"2023-10-09T06:31:28.649372+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:36:12.446877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.495021+0000","last_change":"2023-10-09T06:31:28.488130+0000","last_active":"2023-10-09T06:31:28.495021+0000","last_peered":"2023-10-09T06:31:28.495021+0000","last_clean":"2023-10-09T06:31:28.495021+0000","last_became_active":"2023-10-09T06:31:28.487693+0000","last_became_peered":"2023-10-09T06:31:28.487693+0000","last_unstale":"2023-10-09T06:31:28.495021+0000","last_undegraded":"2023-10-09T06:31:28.495021+0000","last_fullsized":"2023-10-09T06:31:28.495021+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:01:45.048047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.479599+0000","last_change":"2023-10-09T06:31:27.643457+0000","last_active":"2023-10-09T06:31:28.479599+0000","last_peered":"2023-10-09T06:31:28.479599+0000","last_clean":"2023-10-09T06:31:28.479599+0000","last_became_active":"2023-10-09T06:31:27.643162+0000","last_became_peered":"2023-10-09T06:31:27.643162+0000","last_unstale":"2023-10-09T06:31:28.479599+0000","last_undegraded":"2023-10-09T06:31:28.479599+0000","last_fullsized":"2023-10-09T06:31:28.479599+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:10:57.417447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.497505+0000","last_change":"2023-10-09T06:31:28.492070+0000","last_active":"2023-10-09T06:31:28.497505+0000","last_peered":"2023-10-09T06:31:28.497505+0000","last_clean":"2023-10-09T06:31:28.497505+0000","last_became_active":"2023-10-09T06:31:28.491773+0000","last_became_peered":"2023-10-09T06:31:28.491773+0000","last_unstale":"2023-10-09T06:31:28.497505+0000","last_undegraded":"2023-10-09T06:31:28.497505+0000","last_fullsized":"2023-10-09T06:31:28.497505+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:19:42.372022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.479556+0000","last_change":"2023-10-09T06:31:27.646007+0000","last_active":"2023-10-09T06:31:28.479556+0000","last_peered":"2023-10-09T06:31:28.479556+0000","last_clean":"2023-10-09T06:31:28.479556+0000","last_became_active":"2023-10-09T06:31:27.645794+0000","last_became_peered":"2023-10-09T06:31:27.645794+0000","last_unstale":"2023-10-09T06:31:28.479556+0000","last_undegraded":"2023-10-09T06:31:28.479556+0000","last_fullsized":"2023-10-09T06:31:28.479556+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:17:13.854519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.494118+0000","last_change":"2023-10-09T06:31:28.489243+0000","last_active":"2023-10-09T06:31:28.494118+0000","last_peered":"2023-10-09T06:31:28.494118+0000","last_clean":"2023-10-09T06:31:28.494118+0000","last_became_active":"2023-10-09T06:31:28.488994+0000","last_became_peered":"2023-10-09T06:31:28.488994+0000","last_unstale":"2023-10-09T06:31:28.494118+0000","last_undegraded":"2023-10-09T06:31:28.494118+0000","last_fullsized":"2023-10-09T06:31:28.494118+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:43:55.680426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"30'8","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.652981+0000","last_change":"2023-10-09T06:31:27.642240+0000","last_active":"2023-10-09T06:31:28.652981+0000","last_peered":"2023-10-09T06:31:28.652981+0000","last_clean":"2023-10-09T06:31:28.652981+0000","last_became_active":"2023-10-09T06:31:27.642018+0000","last_became_peered":"2023-10-09T06:31:27.642018+0000","last_unstale":"2023-10-09T06:31:28.652981+0000","last_undegraded":"2023-10-09T06:31:28.652981+0000","last_fullsized":"2023-10-09T06:31:28.652981+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:03:18.203913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.498003+0000","last_change":"2023-10-09T06:31:28.494181+0000","last_active":"2023-10-09T06:31:28.498003+0000","last_peered":"2023-10-09T06:31:28.498003+0000","last_clean":"2023-10-09T06:31:28.498003+0000","last_became_active":"2023-10-09T06:31:28.493896+0000","last_became_peered":"2023-10-09T06:31:28.493896+0000","last_unstale":"2023-10-09T06:31:28.498003+0000","last_undegraded":"2023-10-09T06:31:28.498003+0000","last_fullsized":"2023-10-09T06:31:28.498003+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:18:22.705769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.652048+0000","last_change":"2023-10-09T06:31:27.644823+0000","last_active":"2023-10-09T06:31:28.652048+0000","last_peered":"2023-10-09T06:31:28.652048+0000","last_clean":"2023-10-09T06:31:28.652048+0000","last_became_active":"2023-10-09T06:31:27.644559+0000","last_became_peered":"2023-10-09T06:31:27.644559+0000","last_unstale":"2023-10-09T06:31:28.652048+0000","last_undegraded":"2023-10-09T06:31:28.652048+0000","last_fullsized":"2023-10-09T06:31:28.652048+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:40:56.010130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.498719+0000","last_change":"2023-10-09T06:31:28.496483+0000","last_active":"2023-10-09T06:31:28.498719+0000","last_peered":"2023-10-09T06:31:28.498719+0000","last_clean":"2023-10-09T06:31:28.498719+0000","last_became_active":"2023-10-09T06:31:28.495371+0000","last_became_peered":"2023-10-09T06:31:28.495371+0000","last_unstale":"2023-10-09T06:31:28.498719+0000","last_undegraded":"2023-10-09T06:31:28.498719+0000","last_fullsized":"2023-10-09T06:31:28.498719+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:19:05.828847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.649209+0000","last_change":"2023-10-09T06:31:27.633242+0000","last_active":"2023-10-09T06:31:28.649209+0000","last_peered":"2023-10-09T06:31:28.649209+0000","last_clean":"2023-10-09T06:31:28.649209+0000","last_became_active":"2023-10-09T06:31:27.632966+0000","last_became_peered":"2023-10-09T06:31:27.632966+0000","last_unstale":"2023-10-09T06:31:28.649209+0000","last_undegraded":"2023-10-09T06:31:28.649209+0000","last_fullsized":"2023-10-09T06:31:28.649209+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:21:31.383047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502843+0000","last_change":"2023-10-09T06:31:28.494416+0000","last_active":"2023-10-09T06:31:28.502843+0000","last_peered":"2023-10-09T06:31:28.502843+0000","last_clean":"2023-10-09T06:31:28.502843+0000","last_became_active":"2023-10-09T06:31:28.493364+0000","last_became_peered":"2023-10-09T06:31:28.493364+0000","last_unstale":"2023-10-09T06:31:28.502843+0000","last_undegraded":"2023-10-09T06:31:28.502843+0000","last_fullsized":"2023-10-09T06:31:28.502843+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:21:46.563134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.649094+0000","last_change":"2023-10-09T06:31:27.639679+0000","last_active":"2023-10-09T06:31:28.649094+0000","last_peered":"2023-10-09T06:31:28.649094+0000","last_clean":"2023-10-09T06:31:28.649094+0000","last_became_active":"2023-10-09T06:31:27.639407+0000","last_became_peered":"2023-10-09T06:31:27.639407+0000","last_unstale":"2023-10-09T06:31:28.649094+0000","last_undegraded":"2023-10-09T06:31:28.649094+0000","last_fullsized":"2023-10-09T06:31:28.649094+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:28:44.546879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502789+0000","last_change":"2023-10-09T06:31:28.494046+0000","last_active":"2023-10-09T06:31:28.502789+0000","last_peered":"2023-10-09T06:31:28.502789+0000","last_clean":"2023-10-09T06:31:28.502789+0000","last_became_active":"2023-10-09T06:31:28.493797+0000","last_became_peered":"2023-10-09T06:31:28.493797+0000","last_unstale":"2023-10-09T06:31:28.502789+0000","last_undegraded":"2023-10-09T06:31:28.502789+0000","last_fullsized":"2023-10-09T06:31:28.502789+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:45:33.750755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.479033+0000","last_change":"2023-10-09T06:31:27.637164+0000","last_active":"2023-10-09T06:31:28.479033+0000","last_peered":"2023-10-09T06:31:28.479033+0000","last_clean":"2023-10-09T06:31:28.479033+0000","last_became_active":"2023-10-09T06:31:27.636892+0000","last_became_peered":"2023-10-09T06:31:27.636892+0000","last_unstale":"2023-10-09T06:31:28.479033+0000","last_undegraded":"2023-10-09T06:31:28.479033+0000","last_fullsized":"2023-10-09T06:31:28.479033+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:16:47.900101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502706+0000","last_change":"2023-10-09T06:31:28.492700+0000","last_active":"2023-10-09T06:31:28.502706+0000","last_peered":"2023-10-09T06:31:28.502706+0000","last_clean":"2023-10-09T06:31:28.502706+0000","last_became_active":"2023-10-09T06:31:28.492433+0000","last_became_peered":"2023-10-09T06:31:28.492433+0000","last_unstale":"2023-10-09T06:31:28.502706+0000","last_undegraded":"2023-10-09T06:31:28.502706+0000","last_fullsized":"2023-10-09T06:31:28.502706+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:41:44.126125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.652503+0000","last_change":"2023-10-09T06:31:27.641050+0000","last_active":"2023-10-09T06:31:28.652503+0000","last_peered":"2023-10-09T06:31:28.652503+0000","last_clean":"2023-10-09T06:31:28.652503+0000","last_became_active":"2023-10-09T06:31:27.640671+0000","last_became_peered":"2023-10-09T06:31:27.640671+0000","last_unstale":"2023-10-09T06:31:28.652503+0000","last_undegraded":"2023-10-09T06:31:28.652503+0000","last_fullsized":"2023-10-09T06:31:28.652503+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:40:56.010130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.500020+0000","last_change":"2023-10-09T06:31:28.490807+0000","last_active":"2023-10-09T06:31:28.500020+0000","last_peered":"2023-10-09T06:31:28.500020+0000","last_clean":"2023-10-09T06:31:28.500020+0000","last_became_active":"2023-10-09T06:31:28.490633+0000","last_became_peered":"2023-10-09T06:31:28.490633+0000","last_unstale":"2023-10-09T06:31:28.500020+0000","last_undegraded":"2023-10-09T06:31:28.500020+0000","last_fullsized":"2023-10-09T06:31:28.500020+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:21:32.391472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.479466+0000","last_change":"2023-10-09T06:31:27.639623+0000","last_active":"2023-10-09T06:31:28.479466+0000","last_peered":"2023-10-09T06:31:28.479466+0000","last_clean":"2023-10-09T06:31:28.479466+0000","last_became_active":"2023-10-09T06:31:27.639373+0000","last_became_peered":"2023-10-09T06:31:27.639373+0000","last_unstale":"2023-10-09T06:31:28.479466+0000","last_undegraded":"2023-10-09T06:31:28.479466+0000","last_fullsized":"2023-10-09T06:31:28.479466+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:10:57.417447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.493032+0000","last_change":"2023-10-09T06:31:28.489315+0000","last_active":"2023-10-09T06:31:28.493032+0000","last_peered":"2023-10-09T06:31:28.493032+0000","last_clean":"2023-10-09T06:31:28.493032+0000","last_became_active":"2023-10-09T06:31:28.489138+0000","last_became_peered":"2023-10-09T06:31:28.489138+0000","last_unstale":"2023-10-09T06:31:28.493032+0000","last_undegraded":"2023-10-09T06:31:28.493032+0000","last_fullsized":"2023-10-09T06:31:28.493032+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:46:50.856860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.478602+0000","last_change":"2023-10-09T06:31:27.640233+0000","last_active":"2023-10-09T06:31:28.478602+0000","last_peered":"2023-10-09T06:31:28.478602+0000","last_clean":"2023-10-09T06:31:28.478602+0000","last_became_active":"2023-10-09T06:31:27.639981+0000","last_became_peered":"2023-10-09T06:31:27.639981+0000","last_unstale":"2023-10-09T06:31:28.478602+0000","last_undegraded":"2023-10-09T06:31:28.478602+0000","last_fullsized":"2023-10-09T06:31:28.478602+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:10:57.417447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.498485+0000","last_change":"2023-10-09T06:31:28.496255+0000","last_active":"2023-10-09T06:31:28.498485+0000","last_peered":"2023-10-09T06:31:28.498485+0000","last_clean":"2023-10-09T06:31:28.498485+0000","last_became_active":"2023-10-09T06:31:28.496007+0000","last_became_peered":"2023-10-09T06:31:28.496007+0000","last_unstale":"2023-10-09T06:31:28.498485+0000","last_undegraded":"2023-10-09T06:31:28.498485+0000","last_fullsized":"2023-10-09T06:31:28.498485+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:19:05.828847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.488080+0000","last_change":"2023-10-09T06:31:27.646200+0000","last_active":"2023-10-09T06:31:28.488080+0000","last_peered":"2023-10-09T06:31:28.488080+0000","last_clean":"2023-10-09T06:31:28.488080+0000","last_became_active":"2023-10-09T06:31:27.645948+0000","last_became_peered":"2023-10-09T06:31:27.645948+0000","last_unstale":"2023-10-09T06:31:28.488080+0000","last_undegraded":"2023-10-09T06:31:28.488080+0000","last_fullsized":"2023-10-09T06:31:28.488080+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:11:08.334515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502794+0000","last_change":"2023-10-09T06:31:28.494049+0000","last_active":"2023-10-09T06:31:28.502794+0000","last_peered":"2023-10-09T06:31:28.502794+0000","last_clean":"2023-10-09T06:31:28.502794+0000","last_became_active":"2023-10-09T06:31:28.493781+0000","last_became_peered":"2023-10-09T06:31:28.493781+0000","last_unstale":"2023-10-09T06:31:28.502794+0000","last_undegraded":"2023-10-09T06:31:28.502794+0000","last_fullsized":"2023-10-09T06:31:28.502794+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:36:13.455301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.651965+0000","last_change":"2023-10-09T06:31:27.644008+0000","last_active":"2023-10-09T06:31:28.651965+0000","last_peered":"2023-10-09T06:31:28.651965+0000","last_clean":"2023-10-09T06:31:28.651965+0000","last_became_active":"2023-10-09T06:31:27.643749+0000","last_became_peered":"2023-10-09T06:31:27.643749+0000","last_unstale":"2023-10-09T06:31:28.651965+0000","last_undegraded":"2023-10-09T06:31:28.651965+0000","last_fullsized":"2023-10-09T06:31:28.651965+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:07:55.664183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.499833+0000","last_change":"2023-10-09T06:31:28.498014+0000","last_active":"2023-10-09T06:31:28.499833+0000","last_peered":"2023-10-09T06:31:28.499833+0000","last_clean":"2023-10-09T06:31:28.499833+0000","last_became_active":"2023-10-09T06:31:28.497727+0000","last_became_peered":"2023-10-09T06:31:28.497727+0000","last_unstale":"2023-10-09T06:31:28.499833+0000","last_undegraded":"2023-10-09T06:31:28.499833+0000","last_fullsized":"2023-10-09T06:31:28.499833+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:04:27.115048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.476928+0000","last_change":"2023-10-09T06:31:27.640065+0000","last_active":"2023-10-09T06:31:28.476928+0000","last_peered":"2023-10-09T06:31:28.476928+0000","last_clean":"2023-10-09T06:31:28.476928+0000","last_became_active":"2023-10-09T06:31:27.639605+0000","last_became_peered":"2023-10-09T06:31:27.639605+0000","last_unstale":"2023-10-09T06:31:28.476928+0000","last_undegraded":"2023-10-09T06:31:28.476928+0000","last_fullsized":"2023-10-09T06:31:28.476928+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:07:55.664183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.489078+0000","last_change":"2023-10-09T06:31:28.486228+0000","last_active":"2023-10-09T06:31:28.489078+0000","last_peered":"2023-10-09T06:31:28.489078+0000","last_clean":"2023-10-09T06:31:28.489078+0000","last_became_active":"2023-10-09T06:31:28.485969+0000","last_became_peered":"2023-10-09T06:31:28.485969+0000","last_unstale":"2023-10-09T06:31:28.489078+0000","last_undegraded":"2023-10-09T06:31:28.489078+0000","last_fullsized":"2023-10-09T06:31:28.489078+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:06:14.186814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.483139+0000","last_change":"2023-10-09T06:31:27.644209+0000","last_active":"2023-10-09T06:31:28.483139+0000","last_peered":"2023-10-09T06:31:28.483139+0000","last_clean":"2023-10-09T06:31:28.483139+0000","last_became_active":"2023-10-09T06:31:27.643833+0000","last_became_peered":"2023-10-09T06:31:27.643833+0000","last_unstale":"2023-10-09T06:31:28.483139+0000","last_undegraded":"2023-10-09T06:31:28.483139+0000","last_fullsized":"2023-10-09T06:31:28.483139+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:28:44.546879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.494022+0000","last_change":"2023-10-09T06:31:28.485910+0000","last_active":"2023-10-09T06:31:28.494022+0000","last_peered":"2023-10-09T06:31:28.494022+0000","last_clean":"2023-10-09T06:31:28.494022+0000","last_became_active":"2023-10-09T06:31:28.485664+0000","last_became_peered":"2023-10-09T06:31:28.485664+0000","last_unstale":"2023-10-09T06:31:28.494022+0000","last_undegraded":"2023-10-09T06:31:28.494022+0000","last_fullsized":"2023-10-09T06:31:28.494022+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:21:41.103023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.476672+0000","last_change":"2023-10-09T06:31:27.640901+0000","last_active":"2023-10-09T06:31:28.476672+0000","last_peered":"2023-10-09T06:31:28.476672+0000","last_clean":"2023-10-09T06:31:28.476672+0000","last_became_active":"2023-10-09T06:31:27.640325+0000","last_became_peered":"2023-10-09T06:31:27.640325+0000","last_unstale":"2023-10-09T06:31:28.476672+0000","last_undegraded":"2023-10-09T06:31:28.476672+0000","last_fullsized":"2023-10-09T06:31:28.476672+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:28:44.546879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.484793+0000","last_change":"2023-10-09T06:31:28.481613+0000","last_active":"2023-10-09T06:31:28.484793+0000","last_peered":"2023-10-09T06:31:28.484793+0000","last_clean":"2023-10-09T06:31:28.484793+0000","last_became_active":"2023-10-09T06:31:28.481367+0000","last_became_peered":"2023-10-09T06:31:28.481367+0000","last_unstale":"2023-10-09T06:31:28.484793+0000","last_undegraded":"2023-10-09T06:31:28.484793+0000","last_fullsized":"2023-10-09T06:31:28.484793+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:11:09.342940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.479697+0000","last_change":"2023-10-09T06:31:27.644101+0000","last_active":"2023-10-09T06:31:28.479697+0000","last_peered":"2023-10-09T06:31:28.479697+0000","last_clean":"2023-10-09T06:31:28.479697+0000","last_became_active":"2023-10-09T06:31:27.643822+0000","last_became_peered":"2023-10-09T06:31:27.643822+0000","last_unstale":"2023-10-09T06:31:28.479697+0000","last_undegraded":"2023-10-09T06:31:28.479697+0000","last_fullsized":"2023-10-09T06:31:28.479697+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:04:16.068904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.500218+0000","last_change":"2023-10-09T06:31:28.492050+0000","last_active":"2023-10-09T06:31:28.500218+0000","last_peered":"2023-10-09T06:31:28.500218+0000","last_clean":"2023-10-09T06:31:28.500218+0000","last_became_active":"2023-10-09T06:31:28.491526+0000","last_became_peered":"2023-10-09T06:31:28.491526+0000","last_unstale":"2023-10-09T06:31:28.500218+0000","last_undegraded":"2023-10-09T06:31:28.500218+0000","last_fullsized":"2023-10-09T06:31:28.500218+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:04:27.115048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.478850+0000","last_change":"2023-10-09T06:31:27.639038+0000","last_active":"2023-10-09T06:31:28.478850+0000","last_peered":"2023-10-09T06:31:28.478850+0000","last_clean":"2023-10-09T06:31:28.478850+0000","last_became_active":"2023-10-09T06:31:27.638750+0000","last_became_peered":"2023-10-09T06:31:27.638750+0000","last_unstale":"2023-10-09T06:31:28.478850+0000","last_undegraded":"2023-10-09T06:31:28.478850+0000","last_fullsized":"2023-10-09T06:31:28.478850+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:28:36.005523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.487583+0000","last_change":"2023-10-09T06:31:28.483029+0000","last_active":"2023-10-09T06:31:28.487583+0000","last_peered":"2023-10-09T06:31:28.487583+0000","last_clean":"2023-10-09T06:31:28.487583+0000","last_became_active":"2023-10-09T06:31:28.482755+0000","last_became_peered":"2023-10-09T06:31:28.482755+0000","last_unstale":"2023-10-09T06:31:28.487583+0000","last_undegraded":"2023-10-09T06:31:28.487583+0000","last_fullsized":"2023-10-09T06:31:28.487583+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:03:19.212337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.479689+0000","last_change":"2023-10-09T06:31:27.646583+0000","last_active":"2023-10-09T06:31:28.479689+0000","last_peered":"2023-10-09T06:31:28.479689+0000","last_clean":"2023-10-09T06:31:28.479689+0000","last_became_active":"2023-10-09T06:31:27.646339+0000","last_became_peered":"2023-10-09T06:31:27.646339+0000","last_unstale":"2023-10-09T06:31:28.479689+0000","last_undegraded":"2023-10-09T06:31:28.479689+0000","last_fullsized":"2023-10-09T06:31:28.479689+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:32:48.808318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.497045+0000","last_change":"2023-10-09T06:31:28.490160+0000","last_active":"2023-10-09T06:31:28.497045+0000","last_peered":"2023-10-09T06:31:28.497045+0000","last_clean":"2023-10-09T06:31:28.497045+0000","last_became_active":"2023-10-09T06:31:28.489900+0000","last_became_peered":"2023-10-09T06:31:28.489900+0000","last_unstale":"2023-10-09T06:31:28.497045+0000","last_undegraded":"2023-10-09T06:31:28.497045+0000","last_fullsized":"2023-10-09T06:31:28.497045+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:46:46.193647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.479220+0000","last_change":"2023-10-09T06:31:27.642746+0000","last_active":"2023-10-09T06:31:28.479220+0000","last_peered":"2023-10-09T06:31:28.479220+0000","last_clean":"2023-10-09T06:31:28.479220+0000","last_became_active":"2023-10-09T06:31:27.642477+0000","last_became_peered":"2023-10-09T06:31:27.642477+0000","last_unstale":"2023-10-09T06:31:28.479220+0000","last_undegraded":"2023-10-09T06:31:28.479220+0000","last_fullsized":"2023-10-09T06:31:28.479220+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:45:32.742331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":171,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.482158+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:31:28.482158+0000","last_peered":"2023-10-09T06:31:28.482158+0000","last_clean":"2023-10-09T06:31:28.482158+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:31:28.482158+0000","last_undegraded":"2023-10-09T06:31:28.482158+0000","last_fullsized":"2023-10-09T06:31:28.482158+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298212,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073628,"statfs":{"total":96636764160,"available":96331395072,"internally_reserved":0,"allocated":946176,"data_stored":724172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297636,"kb_used_data":348,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074204,"statfs":{"total":96636764160,"available":96331984896,"internally_reserved":0,"allocated":356352,"data_stored":134302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.64600000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.497}]},{"osd":2,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.59099999999999997}]},{"osd":3,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":4.7510000000000003,"5min":4.7510000000000003,"15min":4.7510000000000003},"last":0.628}]},{"osd":4,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.56100000000000005}]},{"osd":5,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.66200000000000003}]},{"osd":7,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.68300000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":135272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298208,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073632,"statfs":{"total":96636764160,"available":96331399168,"internally_reserved":0,"allocated":942080,"data_stored":724082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.438,"5min":0.438,"15min":0.438},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.30199999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.374}]},{"osd":3,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.35499999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":5.484,"5min":5.484,"15min":5.484},"last":0.67700000000000005}]},{"osd":5,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":5.5449999999999999,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.59799999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.65800000000000003}]},{"osd":7,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":5.5800000000000001,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.58999999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297620,"kb_used_data":332,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074220,"statfs":{"total":96636764160,"available":96332001280,"internally_reserved":0,"allocated":339968,"data_stored":127496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.222}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":135272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.22},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.36099999999999999}]},{"osd":1,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.28299999999999997}]},{"osd":2,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.313}]},{"osd":4,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.439}]},{"osd":5,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.47299999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.57199999999999995}]},{"osd":7,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.495},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.45500000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297636,"kb_used_data":348,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074204,"statfs":{"total":96636764160,"available":96331984896,"internally_reserved":0,"allocated":356352,"data_stored":133818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.439}]},{"osd":1,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.46300000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.59499999999999997}]},{"osd":3,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.55400000000000005}]},{"osd":5,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.77000000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.75600000000000001}]},{"osd":7,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.82399999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297636,"kb_used_data":348,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074204,"statfs":{"total":96636764160,"available":96331984896,"internally_reserved":0,"allocated":356352,"data_stored":134358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:31:32.707 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:31:32.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.856+0000 7fec0470f700 1 Processor -- start 2023-10-09T06:31:32.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.856+0000 7fec0470f700 1 -- start start 2023-10-09T06:31:32.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.857+0000 7fec0470f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7febfc157960 0x7febfc0af250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:32.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.857+0000 7fec0470f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7febfc06b4d0 0x7febfc068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:32.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.857+0000 7fec0470f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febfc06b970 0x7febfc0688d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:32.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.857+0000 7fec0470f700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febfc0ac760 con 0x7febfc06b4d0 2023-10-09T06:31:32.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.857+0000 7fec0470f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febfc0af830 con 0x7febfc157960 2023-10-09T06:31:32.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.857+0000 7fec0470f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febfc068e10 con 0x7febfc06b970 2023-10-09T06:31:32.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.857+0000 7fec01caa700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7febfc06b4d0 0x7febfc068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:32.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.857+0000 7fec01caa700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7febfc06b4d0 0x7febfc068390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33468/0 (socket says 172.21.15.163:33468) 2023-10-09T06:31:32.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.857+0000 7fec01caa700 1 -- 172.21.15.163:0/1499976982 learned_addr learned my addr 172.21.15.163:0/1499976982 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:32.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.857+0000 7fec024ab700 1 --2- 172.21.15.163:0/1499976982 >> v2:172.21.15.181:3300/0 conn(0x7febfc157960 0x7febfc0af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:32.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.857+0000 7fec02cac700 1 --2- 172.21.15.163:0/1499976982 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febfc06b970 0x7febfc0688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:32.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.857+0000 7fec01caa700 1 -- 172.21.15.163:0/1499976982 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febfc06b970 msgr2=0x7febfc0688d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:32.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.857+0000 7fec01caa700 1 --2- 172.21.15.163:0/1499976982 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febfc06b970 0x7febfc0688d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.857+0000 7fec01caa700 1 -- 172.21.15.163:0/1499976982 >> v2:172.21.15.181:3300/0 conn(0x7febfc157960 msgr2=0x7febfc0af250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:32.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.857+0000 7fec01caa700 1 --2- 172.21.15.163:0/1499976982 >> v2:172.21.15.181:3300/0 conn(0x7febfc157960 0x7febfc0af250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.857+0000 7fec01caa700 1 -- 172.21.15.163:0/1499976982 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febfc06b030 con 0x7febfc06b4d0 2023-10-09T06:31:32.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.857+0000 7fec024ab700 1 --2- 172.21.15.163:0/1499976982 >> v2:172.21.15.181:3300/0 conn(0x7febfc157960 0x7febfc0af250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:32.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.857+0000 7fec02cac700 1 --2- 172.21.15.163:0/1499976982 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febfc06b970 0x7febfc0688d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:32.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.857+0000 7fec01caa700 1 --2- 172.21.15.163:0/1499976982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febfc06b4d0 0x7febfc068390 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7febfc0ccdd0 tx=0x7febec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ee600ce00ebdc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:32.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.857+0000 7fec014a9700 1 -- 172.21.15.163:0/1499976982 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec00b860 con 0x7febfc06b4d0 2023-10-09T06:31:32.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.857+0000 7fec014a9700 1 -- 172.21.15.163:0/1499976982 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febec00b9c0 con 0x7febfc06b4d0 2023-10-09T06:31:32.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.857+0000 7fec014a9700 1 -- 172.21.15.163:0/1499976982 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec00bd60 con 0x7febfc06b4d0 2023-10-09T06:31:32.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.858+0000 7fec0470f700 1 -- 172.21.15.163:0/1499976982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febfc06b4d0 msgr2=0x7febfc068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:32.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.858+0000 7fec0470f700 1 --2- 172.21.15.163:0/1499976982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febfc06b4d0 0x7febfc068390 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7febfc0ccdd0 tx=0x7febec009250 comp rx=0 tx=0).stop 2023-10-09T06:31:32.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.858+0000 7fec0470f700 1 -- 172.21.15.163:0/1499976982 shutdown_connections 2023-10-09T06:31:32.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.858+0000 7fec0470f700 1 --2- 172.21.15.163:0/1499976982 >> v2:172.21.15.181:3300/0 conn(0x7febfc157960 0x7febfc0af250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.858+0000 7fec0470f700 1 --2- 172.21.15.163:0/1499976982 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febfc06b970 0x7febfc0688d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.858+0000 7fec0470f700 1 --2- 172.21.15.163:0/1499976982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febfc06b4d0 0x7febfc068390 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.858+0000 7fec0470f700 1 -- 172.21.15.163:0/1499976982 >> 172.21.15.163:0/1499976982 conn(0x7febfc0c9870 msgr2=0x7febfc0c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:32.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.858+0000 7fec0470f700 1 -- 172.21.15.163:0/1499976982 shutdown_connections 2023-10-09T06:31:32.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.858+0000 7fec0470f700 1 -- 172.21.15.163:0/1499976982 wait complete. 2023-10-09T06:31:32.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.859+0000 7fec0470f700 1 Processor -- start 2023-10-09T06:31:32.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.859+0000 7fec0470f700 1 -- start start 2023-10-09T06:31:32.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.860+0000 7fec0470f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febfc06b4d0 0x7febfc16c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:32.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.860+0000 7fec0470f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febfc06b970 0x7febfc1527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:32.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.860+0000 7fec0470f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febfc157960 0x7febfc168850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:32.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.860+0000 7fec0470f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febfc169820 con 0x7febfc06b4d0 2023-10-09T06:31:32.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.860+0000 7fec0470f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febfc16ccd0 con 0x7febfc157960 2023-10-09T06:31:32.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.860+0000 7fec0470f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febfc16bfb0 con 0x7febfc06b970 2023-10-09T06:31:32.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.860+0000 7fec024ab700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febfc06b4d0 0x7febfc16c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:32.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.860+0000 7fec024ab700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febfc06b4d0 0x7febfc16c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33470/0 (socket says 172.21.15.163:33470) 2023-10-09T06:31:32.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.860+0000 7fec024ab700 1 -- 172.21.15.163:0/2193747891 learned_addr learned my addr 172.21.15.163:0/2193747891 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:32.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.860+0000 7fec01caa700 1 --2- 172.21.15.163:0/2193747891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febfc06b970 0x7febfc1527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:32.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.860+0000 7fec02cac700 1 --2- 172.21.15.163:0/2193747891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febfc157960 0x7febfc168850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:32.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.860+0000 7fec024ab700 1 -- 172.21.15.163:0/2193747891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febfc06b970 msgr2=0x7febfc1527c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:32.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.860+0000 7fec024ab700 1 --2- 172.21.15.163:0/2193747891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febfc06b970 0x7febfc1527c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.860+0000 7fec024ab700 1 -- 172.21.15.163:0/2193747891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febfc157960 msgr2=0x7febfc168850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:32.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.860+0000 7fec024ab700 1 --2- 172.21.15.163:0/2193747891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febfc157960 0x7febfc168850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:32.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.860+0000 7fec024ab700 1 -- 172.21.15.163:0/2193747891 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febec0094f0 con 0x7febfc06b4d0 2023-10-09T06:31:32.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.860+0000 7fec024ab700 1 --2- 172.21.15.163:0/2193747891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febfc06b4d0 0x7febfc16c790 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7febf800c850 tx=0x7febf8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:32.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.860+0000 7febf27fc700 1 -- 172.21.15.163:0/2193747891 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf8011600 con 0x7febfc06b4d0 2023-10-09T06:31:32.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.860+0000 7febf27fc700 1 -- 172.21.15.163:0/2193747891 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf8011760 con 0x7febfc06b4d0 2023-10-09T06:31:32.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.860+0000 7fec0470f700 1 -- 172.21.15.163:0/2193747891 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febfc169260 con 0x7febfc06b4d0 2023-10-09T06:31:32.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.861+0000 7febf27fc700 1 -- 172.21.15.163:0/2193747891 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf8011b00 con 0x7febfc06b4d0 2023-10-09T06:31:32.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.861+0000 7fec0470f700 1 -- 172.21.15.163:0/2193747891 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febfc16c270 con 0x7febfc06b4d0 2023-10-09T06:31:32.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.862+0000 7fec0470f700 1 -- 172.21.15.163:0/2193747891 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febfc04c3c0 con 0x7febfc06b4d0 2023-10-09T06:31:32.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.862+0000 7febf27fc700 1 -- 172.21.15.163:0/2193747891 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7febf80073b0 con 0x7febfc06b4d0 2023-10-09T06:31:32.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.862+0000 7febf27fc700 1 --2- 172.21.15.163:0/2193747891 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7febe0066f30 0x7febe00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:32.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.862+0000 7febf27fc700 1 -- 172.21.15.163:0/2193747891 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7febf8084530 con 0x7febfc06b4d0 2023-10-09T06:31:32.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.862+0000 7fec01caa700 1 --2- 172.21.15.163:0/2193747891 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7febe0066f30 0x7febe00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:32.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.863+0000 7fec01caa700 1 --2- 172.21.15.163:0/2193747891 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7febe0066f30 0x7febe00693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7febfc0ccdd0 tx=0x7febec017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:32.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:32.866+0000 7febf27fc700 1 -- 172.21.15.163:0/2193747891 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7febf8055230 con 0x7febfc06b4d0 2023-10-09T06:31:33.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.045+0000 7fec0470f700 1 -- 172.21.15.163:0/2193747891 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7febfc0c8580 con 0x7febe0066f30 2023-10-09T06:31:33.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.053+0000 7febf27fc700 1 -- 172.21.15.163:0/2193747891 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276890 (secure 0 0 0) 0x7febfc0c8580 con 0x7febe0066f30 2023-10-09T06:31:33.057 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:33.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.056+0000 7fec0470f700 1 -- 172.21.15.163:0/2193747891 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7febe0066f30 msgr2=0x7febe00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:33.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.056+0000 7fec0470f700 1 --2- 172.21.15.163:0/2193747891 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7febe0066f30 0x7febe00693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7febfc0ccdd0 tx=0x7febec017000 comp rx=0 tx=0).stop 2023-10-09T06:31:33.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.056+0000 7fec0470f700 1 -- 172.21.15.163:0/2193747891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febfc06b4d0 msgr2=0x7febfc16c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:33.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.056+0000 7fec0470f700 1 --2- 172.21.15.163:0/2193747891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febfc06b4d0 0x7febfc16c790 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7febf800c850 tx=0x7febf8008040 comp rx=0 tx=0).stop 2023-10-09T06:31:33.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.056+0000 7fec0470f700 1 -- 172.21.15.163:0/2193747891 shutdown_connections 2023-10-09T06:31:33.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.056+0000 7fec0470f700 1 --2- 172.21.15.163:0/2193747891 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7febe0066f30 0x7febe00693e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:33.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.056+0000 7fec0470f700 1 --2- 172.21.15.163:0/2193747891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febfc06b4d0 0x7febfc16c790 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:33.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.056+0000 7fec0470f700 1 --2- 172.21.15.163:0/2193747891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7febfc157960 0x7febfc168850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:33.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.056+0000 7fec0470f700 1 --2- 172.21.15.163:0/2193747891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febfc06b970 0x7febfc1527c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:33.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.056+0000 7fec0470f700 1 -- 172.21.15.163:0/2193747891 >> 172.21.15.163:0/2193747891 conn(0x7febfc0c9870 msgr2=0x7febfc068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:33.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.056+0000 7fec0470f700 1 -- 172.21.15.163:0/2193747891 shutdown_connections 2023-10-09T06:31:33.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.056+0000 7fec0470f700 1 -- 172.21.15.163:0/2193747891 wait complete. 2023-10-09T06:31:33.060 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:31:33.069 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2023-10-09T06:31:32.305559+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382236,"kb_used_data":3932,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592484,"statfs":{"total":773094113280,"available":770654703616,"internally_reserved":0,"allocated":4026368,"data_stored":2248772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.682389"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.497592+0000","last_change":"2023-10-09T06:31:28.495980+0000","last_active":"2023-10-09T06:31:28.497592+0000","last_peered":"2023-10-09T06:31:28.497592+0000","last_clean":"2023-10-09T06:31:28.497592+0000","last_became_active":"2023-10-09T06:31:28.495707+0000","last_became_peered":"2023-10-09T06:31:28.495707+0000","last_unstale":"2023-10-09T06:31:28.497592+0000","last_undegraded":"2023-10-09T06:31:28.497592+0000","last_fullsized":"2023-10-09T06:31:28.497592+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:06:14.186814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.484754+0000","last_change":"2023-10-09T06:31:28.483133+0000","last_active":"2023-10-09T06:31:28.484754+0000","last_peered":"2023-10-09T06:31:28.484754+0000","last_clean":"2023-10-09T06:31:28.484754+0000","last_became_active":"2023-10-09T06:31:28.482856+0000","last_became_peered":"2023-10-09T06:31:28.482856+0000","last_unstale":"2023-10-09T06:31:28.484754+0000","last_undegraded":"2023-10-09T06:31:28.484754+0000","last_fullsized":"2023-10-09T06:31:28.484754+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:37:09.825142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502573+0000","last_change":"2023-10-09T06:31:28.498279+0000","last_active":"2023-10-09T06:31:28.502573+0000","last_peered":"2023-10-09T06:31:28.502573+0000","last_clean":"2023-10-09T06:31:28.502573+0000","last_became_active":"2023-10-09T06:31:28.497687+0000","last_became_peered":"2023-10-09T06:31:28.497687+0000","last_unstale":"2023-10-09T06:31:28.502573+0000","last_undegraded":"2023-10-09T06:31:28.502573+0000","last_fullsized":"2023-10-09T06:31:28.502573+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:57:49.644159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502794+0000","last_change":"2023-10-09T06:31:28.493773+0000","last_active":"2023-10-09T06:31:28.502794+0000","last_peered":"2023-10-09T06:31:28.502794+0000","last_clean":"2023-10-09T06:31:28.502794+0000","last_became_active":"2023-10-09T06:31:28.493299+0000","last_became_peered":"2023-10-09T06:31:28.493299+0000","last_unstale":"2023-10-09T06:31:28.502794+0000","last_undegraded":"2023-10-09T06:31:28.502794+0000","last_fullsized":"2023-10-09T06:31:28.502794+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:04:57.646593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502910+0000","last_change":"2023-10-09T06:31:28.495674+0000","last_active":"2023-10-09T06:31:28.502910+0000","last_peered":"2023-10-09T06:31:28.502910+0000","last_clean":"2023-10-09T06:31:28.502910+0000","last_became_active":"2023-10-09T06:31:28.495347+0000","last_became_peered":"2023-10-09T06:31:28.495347+0000","last_unstale":"2023-10-09T06:31:28.502910+0000","last_undegraded":"2023-10-09T06:31:28.502910+0000","last_fullsized":"2023-10-09T06:31:28.502910+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:55:28.765990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.500246+0000","last_change":"2023-10-09T06:31:28.493198+0000","last_active":"2023-10-09T06:31:28.500246+0000","last_peered":"2023-10-09T06:31:28.500246+0000","last_clean":"2023-10-09T06:31:28.500246+0000","last_became_active":"2023-10-09T06:31:28.492946+0000","last_became_peered":"2023-10-09T06:31:28.492946+0000","last_unstale":"2023-10-09T06:31:28.500246+0000","last_undegraded":"2023-10-09T06:31:28.500246+0000","last_fullsized":"2023-10-09T06:31:28.500246+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:04:20.370118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.497319+0000","last_change":"2023-10-09T06:31:28.490883+0000","last_active":"2023-10-09T06:31:28.497319+0000","last_peered":"2023-10-09T06:31:28.497319+0000","last_clean":"2023-10-09T06:31:28.497319+0000","last_became_active":"2023-10-09T06:31:28.490521+0000","last_became_peered":"2023-10-09T06:31:28.490521+0000","last_unstale":"2023-10-09T06:31:28.497319+0000","last_undegraded":"2023-10-09T06:31:28.497319+0000","last_fullsized":"2023-10-09T06:31:28.497319+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:43:43.515688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.496962+0000","last_change":"2023-10-09T06:31:28.488826+0000","last_active":"2023-10-09T06:31:28.496962+0000","last_peered":"2023-10-09T06:31:28.496962+0000","last_clean":"2023-10-09T06:31:28.496962+0000","last_became_active":"2023-10-09T06:31:28.488556+0000","last_became_peered":"2023-10-09T06:31:28.488556+0000","last_unstale":"2023-10-09T06:31:28.496962+0000","last_undegraded":"2023-10-09T06:31:28.496962+0000","last_fullsized":"2023-10-09T06:31:28.496962+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:27:40.174948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.494220+0000","last_change":"2023-10-09T06:31:28.491340+0000","last_active":"2023-10-09T06:31:28.494220+0000","last_peered":"2023-10-09T06:31:28.494220+0000","last_clean":"2023-10-09T06:31:28.494220+0000","last_became_active":"2023-10-09T06:31:28.491087+0000","last_became_peered":"2023-10-09T06:31:28.491087+0000","last_unstale":"2023-10-09T06:31:28.494220+0000","last_undegraded":"2023-10-09T06:31:28.494220+0000","last_fullsized":"2023-10-09T06:31:28.494220+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:04:57.646593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502930+0000","last_change":"2023-10-09T06:31:28.495914+0000","last_active":"2023-10-09T06:31:28.502930+0000","last_peered":"2023-10-09T06:31:28.502930+0000","last_clean":"2023-10-09T06:31:28.502930+0000","last_became_active":"2023-10-09T06:31:28.495661+0000","last_became_peered":"2023-10-09T06:31:28.495661+0000","last_unstale":"2023-10-09T06:31:28.502930+0000","last_undegraded":"2023-10-09T06:31:28.502930+0000","last_fullsized":"2023-10-09T06:31:28.502930+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:27:18.901979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.497281+0000","last_change":"2023-10-09T06:31:28.490597+0000","last_active":"2023-10-09T06:31:28.497281+0000","last_peered":"2023-10-09T06:31:28.497281+0000","last_clean":"2023-10-09T06:31:28.497281+0000","last_became_active":"2023-10-09T06:31:28.490338+0000","last_became_peered":"2023-10-09T06:31:28.490338+0000","last_unstale":"2023-10-09T06:31:28.497281+0000","last_undegraded":"2023-10-09T06:31:28.497281+0000","last_fullsized":"2023-10-09T06:31:28.497281+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:14:19.995870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.498699+0000","last_change":"2023-10-09T06:31:28.496333+0000","last_active":"2023-10-09T06:31:28.498699+0000","last_peered":"2023-10-09T06:31:28.498699+0000","last_clean":"2023-10-09T06:31:28.498699+0000","last_became_active":"2023-10-09T06:31:28.495820+0000","last_became_peered":"2023-10-09T06:31:28.495820+0000","last_unstale":"2023-10-09T06:31:28.498699+0000","last_undegraded":"2023-10-09T06:31:28.498699+0000","last_fullsized":"2023-10-09T06:31:28.498699+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:04:20.370118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.493013+0000","last_change":"2023-10-09T06:31:28.489488+0000","last_active":"2023-10-09T06:31:28.493013+0000","last_peered":"2023-10-09T06:31:28.493013+0000","last_clean":"2023-10-09T06:31:28.493013+0000","last_became_active":"2023-10-09T06:31:28.488817+0000","last_became_peered":"2023-10-09T06:31:28.488817+0000","last_unstale":"2023-10-09T06:31:28.493013+0000","last_undegraded":"2023-10-09T06:31:28.493013+0000","last_fullsized":"2023-10-09T06:31:28.493013+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:28:37.013947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.500141+0000","last_change":"2023-10-09T06:31:28.489437+0000","last_active":"2023-10-09T06:31:28.500141+0000","last_peered":"2023-10-09T06:31:28.500141+0000","last_clean":"2023-10-09T06:31:28.500141+0000","last_became_active":"2023-10-09T06:31:28.489185+0000","last_became_peered":"2023-10-09T06:31:28.489185+0000","last_unstale":"2023-10-09T06:31:28.500141+0000","last_undegraded":"2023-10-09T06:31:28.500141+0000","last_fullsized":"2023-10-09T06:31:28.500141+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:27:40.174948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502593+0000","last_change":"2023-10-09T06:31:28.498393+0000","last_active":"2023-10-09T06:31:28.502593+0000","last_peered":"2023-10-09T06:31:28.502593+0000","last_clean":"2023-10-09T06:31:28.502593+0000","last_became_active":"2023-10-09T06:31:28.497388+0000","last_became_peered":"2023-10-09T06:31:28.497388+0000","last_unstale":"2023-10-09T06:31:28.502593+0000","last_undegraded":"2023-10-09T06:31:28.502593+0000","last_fullsized":"2023-10-09T06:31:28.502593+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:43:43.515688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.493239+0000","last_change":"2023-10-09T06:31:28.485519+0000","last_active":"2023-10-09T06:31:28.493239+0000","last_peered":"2023-10-09T06:31:28.493239+0000","last_clean":"2023-10-09T06:31:28.493239+0000","last_became_active":"2023-10-09T06:31:28.485243+0000","last_became_peered":"2023-10-09T06:31:28.485243+0000","last_unstale":"2023-10-09T06:31:28.493239+0000","last_undegraded":"2023-10-09T06:31:28.493239+0000","last_fullsized":"2023-10-09T06:31:28.493239+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:04:27.115048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.497586+0000","last_change":"2023-10-09T06:31:28.491917+0000","last_active":"2023-10-09T06:31:28.497586+0000","last_peered":"2023-10-09T06:31:28.497586+0000","last_clean":"2023-10-09T06:31:28.497586+0000","last_became_active":"2023-10-09T06:31:28.491649+0000","last_became_peered":"2023-10-09T06:31:28.491649+0000","last_unstale":"2023-10-09T06:31:28.497586+0000","last_undegraded":"2023-10-09T06:31:28.497586+0000","last_fullsized":"2023-10-09T06:31:28.497586+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:27:18.901979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.500207+0000","last_change":"2023-10-09T06:31:28.491824+0000","last_active":"2023-10-09T06:31:28.500207+0000","last_peered":"2023-10-09T06:31:28.500207+0000","last_clean":"2023-10-09T06:31:28.500207+0000","last_became_active":"2023-10-09T06:31:28.491632+0000","last_became_peered":"2023-10-09T06:31:28.491632+0000","last_unstale":"2023-10-09T06:31:28.500207+0000","last_undegraded":"2023-10-09T06:31:28.500207+0000","last_fullsized":"2023-10-09T06:31:28.500207+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:24:49.566571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.500206+0000","last_change":"2023-10-09T06:31:28.491813+0000","last_active":"2023-10-09T06:31:28.500206+0000","last_peered":"2023-10-09T06:31:28.500206+0000","last_clean":"2023-10-09T06:31:28.500206+0000","last_became_active":"2023-10-09T06:31:28.491503+0000","last_became_peered":"2023-10-09T06:31:28.491503+0000","last_unstale":"2023-10-09T06:31:28.500206+0000","last_undegraded":"2023-10-09T06:31:28.500206+0000","last_fullsized":"2023-10-09T06:31:28.500206+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:43:43.515688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502852+0000","last_change":"2023-10-09T06:31:28.495453+0000","last_active":"2023-10-09T06:31:28.502852+0000","last_peered":"2023-10-09T06:31:28.502852+0000","last_clean":"2023-10-09T06:31:28.502852+0000","last_became_active":"2023-10-09T06:31:28.495221+0000","last_became_peered":"2023-10-09T06:31:28.495221+0000","last_unstale":"2023-10-09T06:31:28.502852+0000","last_undegraded":"2023-10-09T06:31:28.502852+0000","last_fullsized":"2023-10-09T06:31:28.502852+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:16:48.908526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.490965+0000","last_change":"2023-10-09T06:31:28.483955+0000","last_active":"2023-10-09T06:31:28.490965+0000","last_peered":"2023-10-09T06:31:28.490965+0000","last_clean":"2023-10-09T06:31:28.490965+0000","last_became_active":"2023-10-09T06:31:28.483688+0000","last_became_peered":"2023-10-09T06:31:28.483688+0000","last_unstale":"2023-10-09T06:31:28.490965+0000","last_undegraded":"2023-10-09T06:31:28.490965+0000","last_fullsized":"2023-10-09T06:31:28.490965+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:06:14.186814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.491942+0000","last_change":"2023-10-09T06:31:28.486094+0000","last_active":"2023-10-09T06:31:28.491942+0000","last_peered":"2023-10-09T06:31:28.491942+0000","last_clean":"2023-10-09T06:31:28.491942+0000","last_became_active":"2023-10-09T06:31:28.485819+0000","last_became_peered":"2023-10-09T06:31:28.485819+0000","last_unstale":"2023-10-09T06:31:28.491942+0000","last_undegraded":"2023-10-09T06:31:28.491942+0000","last_fullsized":"2023-10-09T06:31:28.491942+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:46:46.193647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.486329+0000","last_change":"2023-10-09T06:31:28.484836+0000","last_active":"2023-10-09T06:31:28.486329+0000","last_peered":"2023-10-09T06:31:28.486329+0000","last_clean":"2023-10-09T06:31:28.486329+0000","last_became_active":"2023-10-09T06:31:28.484552+0000","last_became_peered":"2023-10-09T06:31:28.484552+0000","last_unstale":"2023-10-09T06:31:28.486329+0000","last_undegraded":"2023-10-09T06:31:28.486329+0000","last_fullsized":"2023-10-09T06:31:28.486329+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:03:19.212337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.497139+0000","last_change":"2023-10-09T06:31:28.492708+0000","last_active":"2023-10-09T06:31:28.497139+0000","last_peered":"2023-10-09T06:31:28.497139+0000","last_clean":"2023-10-09T06:31:28.497139+0000","last_became_active":"2023-10-09T06:31:28.492363+0000","last_became_peered":"2023-10-09T06:31:28.492363+0000","last_unstale":"2023-10-09T06:31:28.497139+0000","last_undegraded":"2023-10-09T06:31:28.497139+0000","last_fullsized":"2023-10-09T06:31:28.497139+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:27:40.174948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502870+0000","last_change":"2023-10-09T06:31:28.494485+0000","last_active":"2023-10-09T06:31:28.502870+0000","last_peered":"2023-10-09T06:31:28.502870+0000","last_clean":"2023-10-09T06:31:28.502870+0000","last_became_active":"2023-10-09T06:31:28.494246+0000","last_became_peered":"2023-10-09T06:31:28.494246+0000","last_unstale":"2023-10-09T06:31:28.502870+0000","last_undegraded":"2023-10-09T06:31:28.502870+0000","last_fullsized":"2023-10-09T06:31:28.502870+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:04:19.137467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.494546+0000","last_change":"2023-10-09T06:31:28.485577+0000","last_active":"2023-10-09T06:31:28.494546+0000","last_peered":"2023-10-09T06:31:28.494546+0000","last_clean":"2023-10-09T06:31:28.494546+0000","last_became_active":"2023-10-09T06:31:28.485337+0000","last_became_peered":"2023-10-09T06:31:28.485337+0000","last_unstale":"2023-10-09T06:31:28.494546+0000","last_undegraded":"2023-10-09T06:31:28.494546+0000","last_fullsized":"2023-10-09T06:31:28.494546+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:07:19.040388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502897+0000","last_change":"2023-10-09T06:31:28.495012+0000","last_active":"2023-10-09T06:31:28.502897+0000","last_peered":"2023-10-09T06:31:28.502897+0000","last_clean":"2023-10-09T06:31:28.502897+0000","last_became_active":"2023-10-09T06:31:28.494745+0000","last_became_peered":"2023-10-09T06:31:28.494745+0000","last_unstale":"2023-10-09T06:31:28.502897+0000","last_undegraded":"2023-10-09T06:31:28.502897+0000","last_fullsized":"2023-10-09T06:31:28.502897+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:11:09.342940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.498413+0000","last_change":"2023-10-09T06:31:28.492881+0000","last_active":"2023-10-09T06:31:28.498413+0000","last_peered":"2023-10-09T06:31:28.498413+0000","last_clean":"2023-10-09T06:31:28.498413+0000","last_became_active":"2023-10-09T06:31:28.492572+0000","last_became_peered":"2023-10-09T06:31:28.492572+0000","last_unstale":"2023-10-09T06:31:28.498413+0000","last_undegraded":"2023-10-09T06:31:28.498413+0000","last_fullsized":"2023-10-09T06:31:28.498413+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:19:42.372022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.493295+0000","last_change":"2023-10-09T06:31:28.488456+0000","last_active":"2023-10-09T06:31:28.493295+0000","last_peered":"2023-10-09T06:31:28.493295+0000","last_clean":"2023-10-09T06:31:28.493295+0000","last_became_active":"2023-10-09T06:31:28.485910+0000","last_became_peered":"2023-10-09T06:31:28.485910+0000","last_unstale":"2023-10-09T06:31:28.493295+0000","last_undegraded":"2023-10-09T06:31:28.493295+0000","last_fullsized":"2023-10-09T06:31:28.493295+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:21:46.563134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.497017+0000","last_change":"2023-10-09T06:31:28.490889+0000","last_active":"2023-10-09T06:31:28.497017+0000","last_peered":"2023-10-09T06:31:28.497017+0000","last_clean":"2023-10-09T06:31:28.497017+0000","last_became_active":"2023-10-09T06:31:28.490616+0000","last_became_peered":"2023-10-09T06:31:28.490616+0000","last_unstale":"2023-10-09T06:31:28.497017+0000","last_undegraded":"2023-10-09T06:31:28.497017+0000","last_fullsized":"2023-10-09T06:31:28.497017+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:04:19.137467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.494098+0000","last_change":"2023-10-09T06:31:28.488564+0000","last_active":"2023-10-09T06:31:28.494098+0000","last_peered":"2023-10-09T06:31:28.494098+0000","last_clean":"2023-10-09T06:31:28.494098+0000","last_became_active":"2023-10-09T06:31:28.488290+0000","last_became_peered":"2023-10-09T06:31:28.488290+0000","last_unstale":"2023-10-09T06:31:28.494098+0000","last_undegraded":"2023-10-09T06:31:28.494098+0000","last_fullsized":"2023-10-09T06:31:28.494098+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:17:14.862943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.498364+0000","last_change":"2023-10-09T06:31:28.494811+0000","last_active":"2023-10-09T06:31:28.498364+0000","last_peered":"2023-10-09T06:31:28.498364+0000","last_clean":"2023-10-09T06:31:28.498364+0000","last_became_active":"2023-10-09T06:31:28.494310+0000","last_became_peered":"2023-10-09T06:31:28.494310+0000","last_unstale":"2023-10-09T06:31:28.498364+0000","last_undegraded":"2023-10-09T06:31:28.498364+0000","last_fullsized":"2023-10-09T06:31:28.498364+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:21:46.563134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.489218+0000","last_change":"2023-10-09T06:31:28.485466+0000","last_active":"2023-10-09T06:31:28.489218+0000","last_peered":"2023-10-09T06:31:28.489218+0000","last_clean":"2023-10-09T06:31:28.489218+0000","last_became_active":"2023-10-09T06:31:28.485170+0000","last_became_peered":"2023-10-09T06:31:28.485170+0000","last_unstale":"2023-10-09T06:31:28.489218+0000","last_undegraded":"2023-10-09T06:31:28.489218+0000","last_fullsized":"2023-10-09T06:31:28.489218+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:32:49.816742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'4","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.649786+0000","last_change":"2023-10-09T06:31:27.639225+0000","last_active":"2023-10-09T06:31:28.649786+0000","last_peered":"2023-10-09T06:31:28.649786+0000","last_clean":"2023-10-09T06:31:28.649786+0000","last_became_active":"2023-10-09T06:31:27.638934+0000","last_became_peered":"2023-10-09T06:31:27.638934+0000","last_unstale":"2023-10-09T06:31:28.649786+0000","last_undegraded":"2023-10-09T06:31:28.649786+0000","last_fullsized":"2023-10-09T06:31:28.649786+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:28:44.546879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.496951+0000","last_change":"2023-10-09T06:31:28.490147+0000","last_active":"2023-10-09T06:31:28.496951+0000","last_peered":"2023-10-09T06:31:28.496951+0000","last_clean":"2023-10-09T06:31:28.496951+0000","last_became_active":"2023-10-09T06:31:28.489622+0000","last_became_peered":"2023-10-09T06:31:28.489622+0000","last_unstale":"2023-10-09T06:31:28.496951+0000","last_undegraded":"2023-10-09T06:31:28.496951+0000","last_fullsized":"2023-10-09T06:31:28.496951+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:11:09.342940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.647123+0000","last_change":"2023-10-09T06:31:27.646459+0000","last_active":"2023-10-09T06:31:28.647123+0000","last_peered":"2023-10-09T06:31:28.647123+0000","last_clean":"2023-10-09T06:31:28.647123+0000","last_became_active":"2023-10-09T06:31:27.646114+0000","last_became_peered":"2023-10-09T06:31:27.646114+0000","last_unstale":"2023-10-09T06:31:28.647123+0000","last_undegraded":"2023-10-09T06:31:28.647123+0000","last_fullsized":"2023-10-09T06:31:28.647123+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:04:16.068904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.491714+0000","last_change":"2023-10-09T06:31:28.484818+0000","last_active":"2023-10-09T06:31:28.491714+0000","last_peered":"2023-10-09T06:31:28.491714+0000","last_clean":"2023-10-09T06:31:28.491714+0000","last_became_active":"2023-10-09T06:31:28.484459+0000","last_became_peered":"2023-10-09T06:31:28.484459+0000","last_unstale":"2023-10-09T06:31:28.491714+0000","last_undegraded":"2023-10-09T06:31:28.491714+0000","last_fullsized":"2023-10-09T06:31:28.491714+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:24:53.175922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.652351+0000","last_change":"2023-10-09T06:31:27.641559+0000","last_active":"2023-10-09T06:31:28.652351+0000","last_peered":"2023-10-09T06:31:28.652351+0000","last_clean":"2023-10-09T06:31:28.652351+0000","last_became_active":"2023-10-09T06:31:27.641319+0000","last_became_peered":"2023-10-09T06:31:27.641319+0000","last_unstale":"2023-10-09T06:31:28.652351+0000","last_undegraded":"2023-10-09T06:31:28.652351+0000","last_fullsized":"2023-10-09T06:31:28.652351+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:11:08.334515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502860+0000","last_change":"2023-10-09T06:31:28.494483+0000","last_active":"2023-10-09T06:31:28.502860+0000","last_peered":"2023-10-09T06:31:28.502860+0000","last_clean":"2023-10-09T06:31:28.502860+0000","last_became_active":"2023-10-09T06:31:28.493408+0000","last_became_peered":"2023-10-09T06:31:28.493408+0000","last_unstale":"2023-10-09T06:31:28.502860+0000","last_undegraded":"2023-10-09T06:31:28.502860+0000","last_fullsized":"2023-10-09T06:31:28.502860+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:46:46.193647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.654501+0000","last_change":"2023-10-09T06:31:27.639485+0000","last_active":"2023-10-09T06:31:28.654501+0000","last_peered":"2023-10-09T06:31:28.654501+0000","last_clean":"2023-10-09T06:31:28.654501+0000","last_became_active":"2023-10-09T06:31:27.639233+0000","last_became_peered":"2023-10-09T06:31:27.639233+0000","last_unstale":"2023-10-09T06:31:28.654501+0000","last_undegraded":"2023-10-09T06:31:28.654501+0000","last_fullsized":"2023-10-09T06:31:28.654501+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:15:59.697671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.498298+0000","last_change":"2023-10-09T06:31:28.492731+0000","last_active":"2023-10-09T06:31:28.498298+0000","last_peered":"2023-10-09T06:31:28.498298+0000","last_clean":"2023-10-09T06:31:28.498298+0000","last_became_active":"2023-10-09T06:31:28.492472+0000","last_became_peered":"2023-10-09T06:31:28.492472+0000","last_unstale":"2023-10-09T06:31:28.498298+0000","last_undegraded":"2023-10-09T06:31:28.498298+0000","last_fullsized":"2023-10-09T06:31:28.498298+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:04:57.646593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.652145+0000","last_change":"2023-10-09T06:31:27.640688+0000","last_active":"2023-10-09T06:31:28.652145+0000","last_peered":"2023-10-09T06:31:28.652145+0000","last_clean":"2023-10-09T06:31:28.652145+0000","last_became_active":"2023-10-09T06:31:27.640484+0000","last_became_peered":"2023-10-09T06:31:27.640484+0000","last_unstale":"2023-10-09T06:31:28.652145+0000","last_undegraded":"2023-10-09T06:31:28.652145+0000","last_fullsized":"2023-10-09T06:31:28.652145+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:45:32.742331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.498463+0000","last_change":"2023-10-09T06:31:28.493835+0000","last_active":"2023-10-09T06:31:28.498463+0000","last_peered":"2023-10-09T06:31:28.498463+0000","last_clean":"2023-10-09T06:31:28.498463+0000","last_became_active":"2023-10-09T06:31:28.493561+0000","last_became_peered":"2023-10-09T06:31:28.493561+0000","last_unstale":"2023-10-09T06:31:28.498463+0000","last_undegraded":"2023-10-09T06:31:28.498463+0000","last_fullsized":"2023-10-09T06:31:28.498463+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:07:19.040388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.651751+0000","last_change":"2023-10-09T06:31:27.635744+0000","last_active":"2023-10-09T06:31:28.651751+0000","last_peered":"2023-10-09T06:31:28.651751+0000","last_clean":"2023-10-09T06:31:28.651751+0000","last_became_active":"2023-10-09T06:31:27.635463+0000","last_became_peered":"2023-10-09T06:31:27.635463+0000","last_unstale":"2023-10-09T06:31:28.651751+0000","last_undegraded":"2023-10-09T06:31:28.651751+0000","last_fullsized":"2023-10-09T06:31:28.651751+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:55:27.757566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.487549+0000","last_change":"2023-10-09T06:31:28.481016+0000","last_active":"2023-10-09T06:31:28.487549+0000","last_peered":"2023-10-09T06:31:28.487549+0000","last_clean":"2023-10-09T06:31:28.487549+0000","last_became_active":"2023-10-09T06:31:28.480735+0000","last_became_peered":"2023-10-09T06:31:28.480735+0000","last_unstale":"2023-10-09T06:31:28.487549+0000","last_undegraded":"2023-10-09T06:31:28.487549+0000","last_fullsized":"2023-10-09T06:31:28.487549+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:32:49.816742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.479458+0000","last_change":"2023-10-09T06:31:27.639056+0000","last_active":"2023-10-09T06:31:28.479458+0000","last_peered":"2023-10-09T06:31:28.479458+0000","last_clean":"2023-10-09T06:31:28.479458+0000","last_became_active":"2023-10-09T06:31:27.638826+0000","last_became_peered":"2023-10-09T06:31:27.638826+0000","last_unstale":"2023-10-09T06:31:28.479458+0000","last_undegraded":"2023-10-09T06:31:28.479458+0000","last_fullsized":"2023-10-09T06:31:28.479458+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:21:40.094599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502821+0000","last_change":"2023-10-09T06:31:28.494189+0000","last_active":"2023-10-09T06:31:28.502821+0000","last_peered":"2023-10-09T06:31:28.502821+0000","last_clean":"2023-10-09T06:31:28.502821+0000","last_became_active":"2023-10-09T06:31:28.493914+0000","last_became_peered":"2023-10-09T06:31:28.493914+0000","last_unstale":"2023-10-09T06:31:28.502821+0000","last_undegraded":"2023-10-09T06:31:28.502821+0000","last_fullsized":"2023-10-09T06:31:28.502821+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:24:53.175922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.478957+0000","last_change":"2023-10-09T06:31:27.633322+0000","last_active":"2023-10-09T06:31:28.478957+0000","last_peered":"2023-10-09T06:31:28.478957+0000","last_clean":"2023-10-09T06:31:28.478957+0000","last_became_active":"2023-10-09T06:31:27.633016+0000","last_became_peered":"2023-10-09T06:31:27.633016+0000","last_unstale":"2023-10-09T06:31:28.478957+0000","last_undegraded":"2023-10-09T06:31:28.478957+0000","last_fullsized":"2023-10-09T06:31:28.478957+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:46:49.848435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.505257+0000","last_change":"2023-10-09T06:31:28.500793+0000","last_active":"2023-10-09T06:31:28.505257+0000","last_peered":"2023-10-09T06:31:28.505257+0000","last_clean":"2023-10-09T06:31:28.505257+0000","last_became_active":"2023-10-09T06:31:28.500515+0000","last_became_peered":"2023-10-09T06:31:28.500515+0000","last_unstale":"2023-10-09T06:31:28.505257+0000","last_undegraded":"2023-10-09T06:31:28.505257+0000","last_fullsized":"2023-10-09T06:31:28.505257+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:01:45.048047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.480201+0000","last_change":"2023-10-09T06:31:27.643456+0000","last_active":"2023-10-09T06:31:28.480201+0000","last_peered":"2023-10-09T06:31:28.480201+0000","last_clean":"2023-10-09T06:31:28.480201+0000","last_became_active":"2023-10-09T06:31:27.643271+0000","last_became_peered":"2023-10-09T06:31:27.643271+0000","last_unstale":"2023-10-09T06:31:28.480201+0000","last_undegraded":"2023-10-09T06:31:28.480201+0000","last_fullsized":"2023-10-09T06:31:28.480201+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:43:54.672002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.500111+0000","last_change":"2023-10-09T06:31:28.491099+0000","last_active":"2023-10-09T06:31:28.500111+0000","last_peered":"2023-10-09T06:31:28.500111+0000","last_clean":"2023-10-09T06:31:28.500111+0000","last_became_active":"2023-10-09T06:31:28.490841+0000","last_became_peered":"2023-10-09T06:31:28.490841+0000","last_unstale":"2023-10-09T06:31:28.500111+0000","last_undegraded":"2023-10-09T06:31:28.500111+0000","last_fullsized":"2023-10-09T06:31:28.500111+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:57:49.644159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.476831+0000","last_change":"2023-10-09T06:31:27.640486+0000","last_active":"2023-10-09T06:31:28.476831+0000","last_peered":"2023-10-09T06:31:28.476831+0000","last_clean":"2023-10-09T06:31:28.476831+0000","last_became_active":"2023-10-09T06:31:27.640218+0000","last_became_peered":"2023-10-09T06:31:27.640218+0000","last_unstale":"2023-10-09T06:31:28.476831+0000","last_undegraded":"2023-10-09T06:31:28.476831+0000","last_fullsized":"2023-10-09T06:31:28.476831+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:10:57.417447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.491689+0000","last_change":"2023-10-09T06:31:28.484559+0000","last_active":"2023-10-09T06:31:28.491689+0000","last_peered":"2023-10-09T06:31:28.491689+0000","last_clean":"2023-10-09T06:31:28.491689+0000","last_became_active":"2023-10-09T06:31:28.484319+0000","last_became_peered":"2023-10-09T06:31:28.484319+0000","last_unstale":"2023-10-09T06:31:28.491689+0000","last_undegraded":"2023-10-09T06:31:28.491689+0000","last_fullsized":"2023-10-09T06:31:28.491689+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:38:14.098330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"30'3","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.649566+0000","last_change":"2023-10-09T06:31:27.646080+0000","last_active":"2023-10-09T06:31:28.649566+0000","last_peered":"2023-10-09T06:31:28.649566+0000","last_clean":"2023-10-09T06:31:28.649566+0000","last_became_active":"2023-10-09T06:31:27.645829+0000","last_became_peered":"2023-10-09T06:31:27.645829+0000","last_unstale":"2023-10-09T06:31:28.649566+0000","last_undegraded":"2023-10-09T06:31:28.649566+0000","last_fullsized":"2023-10-09T06:31:28.649566+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:06:13.178390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.499831+0000","last_change":"2023-10-09T06:31:28.497844+0000","last_active":"2023-10-09T06:31:28.499831+0000","last_peered":"2023-10-09T06:31:28.499831+0000","last_clean":"2023-10-09T06:31:28.499831+0000","last_became_active":"2023-10-09T06:31:28.497604+0000","last_became_peered":"2023-10-09T06:31:28.497604+0000","last_unstale":"2023-10-09T06:31:28.499831+0000","last_undegraded":"2023-10-09T06:31:28.499831+0000","last_fullsized":"2023-10-09T06:31:28.499831+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:24:49.566571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"30'6","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.654595+0000","last_change":"2023-10-09T06:31:27.645570+0000","last_active":"2023-10-09T06:31:28.654595+0000","last_peered":"2023-10-09T06:31:28.654595+0000","last_clean":"2023-10-09T06:31:28.654595+0000","last_became_active":"2023-10-09T06:31:27.645359+0000","last_became_peered":"2023-10-09T06:31:27.645359+0000","last_unstale":"2023-10-09T06:31:28.654595+0000","last_undegraded":"2023-10-09T06:31:28.654595+0000","last_fullsized":"2023-10-09T06:31:28.654595+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:06:13.178390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502816+0000","last_change":"2023-10-09T06:31:28.494111+0000","last_active":"2023-10-09T06:31:28.502816+0000","last_peered":"2023-10-09T06:31:28.502816+0000","last_clean":"2023-10-09T06:31:28.502816+0000","last_became_active":"2023-10-09T06:31:28.493872+0000","last_became_peered":"2023-10-09T06:31:28.493872+0000","last_unstale":"2023-10-09T06:31:28.502816+0000","last_undegraded":"2023-10-09T06:31:28.502816+0000","last_fullsized":"2023-10-09T06:31:28.502816+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:18:22.705769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.649372+0000","last_change":"2023-10-09T06:31:27.634344+0000","last_active":"2023-10-09T06:31:28.649372+0000","last_peered":"2023-10-09T06:31:28.649372+0000","last_clean":"2023-10-09T06:31:28.649372+0000","last_became_active":"2023-10-09T06:31:27.634108+0000","last_became_peered":"2023-10-09T06:31:27.634108+0000","last_unstale":"2023-10-09T06:31:28.649372+0000","last_undegraded":"2023-10-09T06:31:28.649372+0000","last_fullsized":"2023-10-09T06:31:28.649372+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:36:12.446877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.495021+0000","last_change":"2023-10-09T06:31:28.488130+0000","last_active":"2023-10-09T06:31:28.495021+0000","last_peered":"2023-10-09T06:31:28.495021+0000","last_clean":"2023-10-09T06:31:28.495021+0000","last_became_active":"2023-10-09T06:31:28.487693+0000","last_became_peered":"2023-10-09T06:31:28.487693+0000","last_unstale":"2023-10-09T06:31:28.495021+0000","last_undegraded":"2023-10-09T06:31:28.495021+0000","last_fullsized":"2023-10-09T06:31:28.495021+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:01:45.048047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.479599+0000","last_change":"2023-10-09T06:31:27.643457+0000","last_active":"2023-10-09T06:31:28.479599+0000","last_peered":"2023-10-09T06:31:28.479599+0000","last_clean":"2023-10-09T06:31:28.479599+0000","last_became_active":"2023-10-09T06:31:27.643162+0000","last_became_peered":"2023-10-09T06:31:27.643162+0000","last_unstale":"2023-10-09T06:31:28.479599+0000","last_undegraded":"2023-10-09T06:31:28.479599+0000","last_fullsized":"2023-10-09T06:31:28.479599+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:10:57.417447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.497505+0000","last_change":"2023-10-09T06:31:28.492070+0000","last_active":"2023-10-09T06:31:28.497505+0000","last_peered":"2023-10-09T06:31:28.497505+0000","last_clean":"2023-10-09T06:31:28.497505+0000","last_became_active":"2023-10-09T06:31:28.491773+0000","last_became_peered":"2023-10-09T06:31:28.491773+0000","last_unstale":"2023-10-09T06:31:28.497505+0000","last_undegraded":"2023-10-09T06:31:28.497505+0000","last_fullsized":"2023-10-09T06:31:28.497505+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:19:42.372022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.479556+0000","last_change":"2023-10-09T06:31:27.646007+0000","last_active":"2023-10-09T06:31:28.479556+0000","last_peered":"2023-10-09T06:31:28.479556+0000","last_clean":"2023-10-09T06:31:28.479556+0000","last_became_active":"2023-10-09T06:31:27.645794+0000","last_became_peered":"2023-10-09T06:31:27.645794+0000","last_unstale":"2023-10-09T06:31:28.479556+0000","last_undegraded":"2023-10-09T06:31:28.479556+0000","last_fullsized":"2023-10-09T06:31:28.479556+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:17:13.854519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.494118+0000","last_change":"2023-10-09T06:31:28.489243+0000","last_active":"2023-10-09T06:31:28.494118+0000","last_peered":"2023-10-09T06:31:28.494118+0000","last_clean":"2023-10-09T06:31:28.494118+0000","last_became_active":"2023-10-09T06:31:28.488994+0000","last_became_peered":"2023-10-09T06:31:28.488994+0000","last_unstale":"2023-10-09T06:31:28.494118+0000","last_undegraded":"2023-10-09T06:31:28.494118+0000","last_fullsized":"2023-10-09T06:31:28.494118+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:43:55.680426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"30'8","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.652981+0000","last_change":"2023-10-09T06:31:27.642240+0000","last_active":"2023-10-09T06:31:28.652981+0000","last_peered":"2023-10-09T06:31:28.652981+0000","last_clean":"2023-10-09T06:31:28.652981+0000","last_became_active":"2023-10-09T06:31:27.642018+0000","last_became_peered":"2023-10-09T06:31:27.642018+0000","last_unstale":"2023-10-09T06:31:28.652981+0000","last_undegraded":"2023-10-09T06:31:28.652981+0000","last_fullsized":"2023-10-09T06:31:28.652981+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:03:18.203913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.498003+0000","last_change":"2023-10-09T06:31:28.494181+0000","last_active":"2023-10-09T06:31:28.498003+0000","last_peered":"2023-10-09T06:31:28.498003+0000","last_clean":"2023-10-09T06:31:28.498003+0000","last_became_active":"2023-10-09T06:31:28.493896+0000","last_became_peered":"2023-10-09T06:31:28.493896+0000","last_unstale":"2023-10-09T06:31:28.498003+0000","last_undegraded":"2023-10-09T06:31:28.498003+0000","last_fullsized":"2023-10-09T06:31:28.498003+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:18:22.705769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.652048+0000","last_change":"2023-10-09T06:31:27.644823+0000","last_active":"2023-10-09T06:31:28.652048+0000","last_peered":"2023-10-09T06:31:28.652048+0000","last_clean":"2023-10-09T06:31:28.652048+0000","last_became_active":"2023-10-09T06:31:27.644559+0000","last_became_peered":"2023-10-09T06:31:27.644559+0000","last_unstale":"2023-10-09T06:31:28.652048+0000","last_undegraded":"2023-10-09T06:31:28.652048+0000","last_fullsized":"2023-10-09T06:31:28.652048+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:40:56.010130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.498719+0000","last_change":"2023-10-09T06:31:28.496483+0000","last_active":"2023-10-09T06:31:28.498719+0000","last_peered":"2023-10-09T06:31:28.498719+0000","last_clean":"2023-10-09T06:31:28.498719+0000","last_became_active":"2023-10-09T06:31:28.495371+0000","last_became_peered":"2023-10-09T06:31:28.495371+0000","last_unstale":"2023-10-09T06:31:28.498719+0000","last_undegraded":"2023-10-09T06:31:28.498719+0000","last_fullsized":"2023-10-09T06:31:28.498719+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:19:05.828847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.649209+0000","last_change":"2023-10-09T06:31:27.633242+0000","last_active":"2023-10-09T06:31:28.649209+0000","last_peered":"2023-10-09T06:31:28.649209+0000","last_clean":"2023-10-09T06:31:28.649209+0000","last_became_active":"2023-10-09T06:31:27.632966+0000","last_became_peered":"2023-10-09T06:31:27.632966+0000","last_unstale":"2023-10-09T06:31:28.649209+0000","last_undegraded":"2023-10-09T06:31:28.649209+0000","last_fullsized":"2023-10-09T06:31:28.649209+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:21:31.383047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502843+0000","last_change":"2023-10-09T06:31:28.494416+0000","last_active":"2023-10-09T06:31:28.502843+0000","last_peered":"2023-10-09T06:31:28.502843+0000","last_clean":"2023-10-09T06:31:28.502843+0000","last_became_active":"2023-10-09T06:31:28.493364+0000","last_became_peered":"2023-10-09T06:31:28.493364+0000","last_unstale":"2023-10-09T06:31:28.502843+0000","last_undegraded":"2023-10-09T06:31:28.502843+0000","last_fullsized":"2023-10-09T06:31:28.502843+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:21:46.563134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"30'1","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.649094+0000","last_change":"2023-10-09T06:31:27.639679+0000","last_active":"2023-10-09T06:31:28.649094+0000","last_peered":"2023-10-09T06:31:28.649094+0000","last_clean":"2023-10-09T06:31:28.649094+0000","last_became_active":"2023-10-09T06:31:27.639407+0000","last_became_peered":"2023-10-09T06:31:27.639407+0000","last_unstale":"2023-10-09T06:31:28.649094+0000","last_undegraded":"2023-10-09T06:31:28.649094+0000","last_fullsized":"2023-10-09T06:31:28.649094+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:28:44.546879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502789+0000","last_change":"2023-10-09T06:31:28.494046+0000","last_active":"2023-10-09T06:31:28.502789+0000","last_peered":"2023-10-09T06:31:28.502789+0000","last_clean":"2023-10-09T06:31:28.502789+0000","last_became_active":"2023-10-09T06:31:28.493797+0000","last_became_peered":"2023-10-09T06:31:28.493797+0000","last_unstale":"2023-10-09T06:31:28.502789+0000","last_undegraded":"2023-10-09T06:31:28.502789+0000","last_fullsized":"2023-10-09T06:31:28.502789+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:45:33.750755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.479033+0000","last_change":"2023-10-09T06:31:27.637164+0000","last_active":"2023-10-09T06:31:28.479033+0000","last_peered":"2023-10-09T06:31:28.479033+0000","last_clean":"2023-10-09T06:31:28.479033+0000","last_became_active":"2023-10-09T06:31:27.636892+0000","last_became_peered":"2023-10-09T06:31:27.636892+0000","last_unstale":"2023-10-09T06:31:28.479033+0000","last_undegraded":"2023-10-09T06:31:28.479033+0000","last_fullsized":"2023-10-09T06:31:28.479033+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:16:47.900101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502706+0000","last_change":"2023-10-09T06:31:28.492700+0000","last_active":"2023-10-09T06:31:28.502706+0000","last_peered":"2023-10-09T06:31:28.502706+0000","last_clean":"2023-10-09T06:31:28.502706+0000","last_became_active":"2023-10-09T06:31:28.492433+0000","last_became_peered":"2023-10-09T06:31:28.492433+0000","last_unstale":"2023-10-09T06:31:28.502706+0000","last_undegraded":"2023-10-09T06:31:28.502706+0000","last_fullsized":"2023-10-09T06:31:28.502706+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:41:44.126125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.652503+0000","last_change":"2023-10-09T06:31:27.641050+0000","last_active":"2023-10-09T06:31:28.652503+0000","last_peered":"2023-10-09T06:31:28.652503+0000","last_clean":"2023-10-09T06:31:28.652503+0000","last_became_active":"2023-10-09T06:31:27.640671+0000","last_became_peered":"2023-10-09T06:31:27.640671+0000","last_unstale":"2023-10-09T06:31:28.652503+0000","last_undegraded":"2023-10-09T06:31:28.652503+0000","last_fullsized":"2023-10-09T06:31:28.652503+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:40:56.010130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.500020+0000","last_change":"2023-10-09T06:31:28.490807+0000","last_active":"2023-10-09T06:31:28.500020+0000","last_peered":"2023-10-09T06:31:28.500020+0000","last_clean":"2023-10-09T06:31:28.500020+0000","last_became_active":"2023-10-09T06:31:28.490633+0000","last_became_peered":"2023-10-09T06:31:28.490633+0000","last_unstale":"2023-10-09T06:31:28.500020+0000","last_undegraded":"2023-10-09T06:31:28.500020+0000","last_fullsized":"2023-10-09T06:31:28.500020+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:21:32.391472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.479466+0000","last_change":"2023-10-09T06:31:27.639623+0000","last_active":"2023-10-09T06:31:28.479466+0000","last_peered":"2023-10-09T06:31:28.479466+0000","last_clean":"2023-10-09T06:31:28.479466+0000","last_became_active":"2023-10-09T06:31:27.639373+0000","last_became_peered":"2023-10-09T06:31:27.639373+0000","last_unstale":"2023-10-09T06:31:28.479466+0000","last_undegraded":"2023-10-09T06:31:28.479466+0000","last_fullsized":"2023-10-09T06:31:28.479466+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:10:57.417447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.493032+0000","last_change":"2023-10-09T06:31:28.489315+0000","last_active":"2023-10-09T06:31:28.493032+0000","last_peered":"2023-10-09T06:31:28.493032+0000","last_clean":"2023-10-09T06:31:28.493032+0000","last_became_active":"2023-10-09T06:31:28.489138+0000","last_became_peered":"2023-10-09T06:31:28.489138+0000","last_unstale":"2023-10-09T06:31:28.493032+0000","last_undegraded":"2023-10-09T06:31:28.493032+0000","last_fullsized":"2023-10-09T06:31:28.493032+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:46:50.856860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.478602+0000","last_change":"2023-10-09T06:31:27.640233+0000","last_active":"2023-10-09T06:31:28.478602+0000","last_peered":"2023-10-09T06:31:28.478602+0000","last_clean":"2023-10-09T06:31:28.478602+0000","last_became_active":"2023-10-09T06:31:27.639981+0000","last_became_peered":"2023-10-09T06:31:27.639981+0000","last_unstale":"2023-10-09T06:31:28.478602+0000","last_undegraded":"2023-10-09T06:31:28.478602+0000","last_fullsized":"2023-10-09T06:31:28.478602+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:10:57.417447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.498485+0000","last_change":"2023-10-09T06:31:28.496255+0000","last_active":"2023-10-09T06:31:28.498485+0000","last_peered":"2023-10-09T06:31:28.498485+0000","last_clean":"2023-10-09T06:31:28.498485+0000","last_became_active":"2023-10-09T06:31:28.496007+0000","last_became_peered":"2023-10-09T06:31:28.496007+0000","last_unstale":"2023-10-09T06:31:28.498485+0000","last_undegraded":"2023-10-09T06:31:28.498485+0000","last_fullsized":"2023-10-09T06:31:28.498485+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:19:05.828847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.488080+0000","last_change":"2023-10-09T06:31:27.646200+0000","last_active":"2023-10-09T06:31:28.488080+0000","last_peered":"2023-10-09T06:31:28.488080+0000","last_clean":"2023-10-09T06:31:28.488080+0000","last_became_active":"2023-10-09T06:31:27.645948+0000","last_became_peered":"2023-10-09T06:31:27.645948+0000","last_unstale":"2023-10-09T06:31:28.488080+0000","last_undegraded":"2023-10-09T06:31:28.488080+0000","last_fullsized":"2023-10-09T06:31:28.488080+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:11:08.334515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.502794+0000","last_change":"2023-10-09T06:31:28.494049+0000","last_active":"2023-10-09T06:31:28.502794+0000","last_peered":"2023-10-09T06:31:28.502794+0000","last_clean":"2023-10-09T06:31:28.502794+0000","last_became_active":"2023-10-09T06:31:28.493781+0000","last_became_peered":"2023-10-09T06:31:28.493781+0000","last_unstale":"2023-10-09T06:31:28.502794+0000","last_undegraded":"2023-10-09T06:31:28.502794+0000","last_fullsized":"2023-10-09T06:31:28.502794+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:36:13.455301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"30'2","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.651965+0000","last_change":"2023-10-09T06:31:27.644008+0000","last_active":"2023-10-09T06:31:28.651965+0000","last_peered":"2023-10-09T06:31:28.651965+0000","last_clean":"2023-10-09T06:31:28.651965+0000","last_became_active":"2023-10-09T06:31:27.643749+0000","last_became_peered":"2023-10-09T06:31:27.643749+0000","last_unstale":"2023-10-09T06:31:28.651965+0000","last_undegraded":"2023-10-09T06:31:28.651965+0000","last_fullsized":"2023-10-09T06:31:28.651965+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:07:55.664183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.499833+0000","last_change":"2023-10-09T06:31:28.498014+0000","last_active":"2023-10-09T06:31:28.499833+0000","last_peered":"2023-10-09T06:31:28.499833+0000","last_clean":"2023-10-09T06:31:28.499833+0000","last_became_active":"2023-10-09T06:31:28.497727+0000","last_became_peered":"2023-10-09T06:31:28.497727+0000","last_unstale":"2023-10-09T06:31:28.499833+0000","last_undegraded":"2023-10-09T06:31:28.499833+0000","last_fullsized":"2023-10-09T06:31:28.499833+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:04:27.115048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.476928+0000","last_change":"2023-10-09T06:31:27.640065+0000","last_active":"2023-10-09T06:31:28.476928+0000","last_peered":"2023-10-09T06:31:28.476928+0000","last_clean":"2023-10-09T06:31:28.476928+0000","last_became_active":"2023-10-09T06:31:27.639605+0000","last_became_peered":"2023-10-09T06:31:27.639605+0000","last_unstale":"2023-10-09T06:31:28.476928+0000","last_undegraded":"2023-10-09T06:31:28.476928+0000","last_fullsized":"2023-10-09T06:31:28.476928+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:07:55.664183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.489078+0000","last_change":"2023-10-09T06:31:28.486228+0000","last_active":"2023-10-09T06:31:28.489078+0000","last_peered":"2023-10-09T06:31:28.489078+0000","last_clean":"2023-10-09T06:31:28.489078+0000","last_became_active":"2023-10-09T06:31:28.485969+0000","last_became_peered":"2023-10-09T06:31:28.485969+0000","last_unstale":"2023-10-09T06:31:28.489078+0000","last_undegraded":"2023-10-09T06:31:28.489078+0000","last_fullsized":"2023-10-09T06:31:28.489078+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:06:14.186814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.483139+0000","last_change":"2023-10-09T06:31:27.644209+0000","last_active":"2023-10-09T06:31:28.483139+0000","last_peered":"2023-10-09T06:31:28.483139+0000","last_clean":"2023-10-09T06:31:28.483139+0000","last_became_active":"2023-10-09T06:31:27.643833+0000","last_became_peered":"2023-10-09T06:31:27.643833+0000","last_unstale":"2023-10-09T06:31:28.483139+0000","last_undegraded":"2023-10-09T06:31:28.483139+0000","last_fullsized":"2023-10-09T06:31:28.483139+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:28:44.546879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.494022+0000","last_change":"2023-10-09T06:31:28.485910+0000","last_active":"2023-10-09T06:31:28.494022+0000","last_peered":"2023-10-09T06:31:28.494022+0000","last_clean":"2023-10-09T06:31:28.494022+0000","last_became_active":"2023-10-09T06:31:28.485664+0000","last_became_peered":"2023-10-09T06:31:28.485664+0000","last_unstale":"2023-10-09T06:31:28.494022+0000","last_undegraded":"2023-10-09T06:31:28.494022+0000","last_fullsized":"2023-10-09T06:31:28.494022+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:21:41.103023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.476672+0000","last_change":"2023-10-09T06:31:27.640901+0000","last_active":"2023-10-09T06:31:28.476672+0000","last_peered":"2023-10-09T06:31:28.476672+0000","last_clean":"2023-10-09T06:31:28.476672+0000","last_became_active":"2023-10-09T06:31:27.640325+0000","last_became_peered":"2023-10-09T06:31:27.640325+0000","last_unstale":"2023-10-09T06:31:28.476672+0000","last_undegraded":"2023-10-09T06:31:28.476672+0000","last_fullsized":"2023-10-09T06:31:28.476672+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:28:44.546879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.484793+0000","last_change":"2023-10-09T06:31:28.481613+0000","last_active":"2023-10-09T06:31:28.484793+0000","last_peered":"2023-10-09T06:31:28.484793+0000","last_clean":"2023-10-09T06:31:28.484793+0000","last_became_active":"2023-10-09T06:31:28.481367+0000","last_became_peered":"2023-10-09T06:31:28.481367+0000","last_unstale":"2023-10-09T06:31:28.484793+0000","last_undegraded":"2023-10-09T06:31:28.484793+0000","last_fullsized":"2023-10-09T06:31:28.484793+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:11:09.342940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.479697+0000","last_change":"2023-10-09T06:31:27.644101+0000","last_active":"2023-10-09T06:31:28.479697+0000","last_peered":"2023-10-09T06:31:28.479697+0000","last_clean":"2023-10-09T06:31:28.479697+0000","last_became_active":"2023-10-09T06:31:27.643822+0000","last_became_peered":"2023-10-09T06:31:27.643822+0000","last_unstale":"2023-10-09T06:31:28.479697+0000","last_undegraded":"2023-10-09T06:31:28.479697+0000","last_fullsized":"2023-10-09T06:31:28.479697+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:04:16.068904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.500218+0000","last_change":"2023-10-09T06:31:28.492050+0000","last_active":"2023-10-09T06:31:28.500218+0000","last_peered":"2023-10-09T06:31:28.500218+0000","last_clean":"2023-10-09T06:31:28.500218+0000","last_became_active":"2023-10-09T06:31:28.491526+0000","last_became_peered":"2023-10-09T06:31:28.491526+0000","last_unstale":"2023-10-09T06:31:28.500218+0000","last_undegraded":"2023-10-09T06:31:28.500218+0000","last_fullsized":"2023-10-09T06:31:28.500218+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:04:27.115048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.478850+0000","last_change":"2023-10-09T06:31:27.639038+0000","last_active":"2023-10-09T06:31:28.478850+0000","last_peered":"2023-10-09T06:31:28.478850+0000","last_clean":"2023-10-09T06:31:28.478850+0000","last_became_active":"2023-10-09T06:31:27.638750+0000","last_became_peered":"2023-10-09T06:31:27.638750+0000","last_unstale":"2023-10-09T06:31:28.478850+0000","last_undegraded":"2023-10-09T06:31:28.478850+0000","last_fullsized":"2023-10-09T06:31:28.478850+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:28:36.005523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.487583+0000","last_change":"2023-10-09T06:31:28.483029+0000","last_active":"2023-10-09T06:31:28.487583+0000","last_peered":"2023-10-09T06:31:28.487583+0000","last_clean":"2023-10-09T06:31:28.487583+0000","last_became_active":"2023-10-09T06:31:28.482755+0000","last_became_peered":"2023-10-09T06:31:28.482755+0000","last_unstale":"2023-10-09T06:31:28.487583+0000","last_undegraded":"2023-10-09T06:31:28.487583+0000","last_fullsized":"2023-10-09T06:31:28.487583+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:03:19.212337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.479689+0000","last_change":"2023-10-09T06:31:27.646583+0000","last_active":"2023-10-09T06:31:28.479689+0000","last_peered":"2023-10-09T06:31:28.479689+0000","last_clean":"2023-10-09T06:31:28.479689+0000","last_became_active":"2023-10-09T06:31:27.646339+0000","last_became_peered":"2023-10-09T06:31:27.646339+0000","last_unstale":"2023-10-09T06:31:28.479689+0000","last_undegraded":"2023-10-09T06:31:28.479689+0000","last_fullsized":"2023-10-09T06:31:28.479689+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:32:48.808318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.497045+0000","last_change":"2023-10-09T06:31:28.490160+0000","last_active":"2023-10-09T06:31:28.497045+0000","last_peered":"2023-10-09T06:31:28.497045+0000","last_clean":"2023-10-09T06:31:28.497045+0000","last_became_active":"2023-10-09T06:31:28.489900+0000","last_became_peered":"2023-10-09T06:31:28.489900+0000","last_unstale":"2023-10-09T06:31:28.497045+0000","last_undegraded":"2023-10-09T06:31:28.497045+0000","last_fullsized":"2023-10-09T06:31:28.497045+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:27.622304+0000","last_clean_scrub_stamp":"2023-10-09T06:31:27.622304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:46:46.193647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.479220+0000","last_change":"2023-10-09T06:31:27.642746+0000","last_active":"2023-10-09T06:31:28.479220+0000","last_peered":"2023-10-09T06:31:28.479220+0000","last_clean":"2023-10-09T06:31:28.479220+0000","last_became_active":"2023-10-09T06:31:27.642477+0000","last_became_peered":"2023-10-09T06:31:27.642477+0000","last_unstale":"2023-10-09T06:31:28.479220+0000","last_undegraded":"2023-10-09T06:31:28.479220+0000","last_fullsized":"2023-10-09T06:31:28.479220+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:26.613880+0000","last_clean_scrub_stamp":"2023-10-09T06:31:26.613880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:45:32.742331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":171,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-09T06:31:28.482158+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:31:28.482158+0000","last_peered":"2023-10-09T06:31:28.482158+0000","last_clean":"2023-10-09T06:31:28.482158+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:31:28.482158+0000","last_undegraded":"2023-10-09T06:31:28.482158+0000","last_fullsized":"2023-10-09T06:31:28.482158+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298212,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073628,"statfs":{"total":96636764160,"available":96331395072,"internally_reserved":0,"allocated":946176,"data_stored":724172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297636,"kb_used_data":348,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074204,"statfs":{"total":96636764160,"available":96331984896,"internally_reserved":0,"allocated":356352,"data_stored":134302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.64600000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.497}]},{"osd":2,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.59099999999999997}]},{"osd":3,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":4.7510000000000003,"5min":4.7510000000000003,"15min":4.7510000000000003},"last":0.628}]},{"osd":4,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.56100000000000005}]},{"osd":5,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.66200000000000003}]},{"osd":7,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.68300000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":135272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298208,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073632,"statfs":{"total":96636764160,"available":96331399168,"internally_reserved":0,"allocated":942080,"data_stored":724082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.438,"5min":0.438,"15min":0.438},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.30199999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.374}]},{"osd":3,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.35499999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":5.484,"5min":5.484,"15min":5.484},"last":0.67700000000000005}]},{"osd":5,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":5.5449999999999999,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.59799999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.65800000000000003}]},{"osd":7,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":5.5800000000000001,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.58999999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297620,"kb_used_data":332,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074220,"statfs":{"total":96636764160,"available":96332001280,"internally_reserved":0,"allocated":339968,"data_stored":127496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.222}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":135272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.22},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.36099999999999999}]},{"osd":1,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.28299999999999997}]},{"osd":2,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.313}]},{"osd":4,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.439}]},{"osd":5,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.47299999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.57199999999999995}]},{"osd":7,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.495},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.45500000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297636,"kb_used_data":348,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074204,"statfs":{"total":96636764160,"available":96331984896,"internally_reserved":0,"allocated":356352,"data_stored":133818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.439}]},{"osd":1,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.46300000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.59499999999999997}]},{"osd":3,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.55400000000000005}]},{"osd":5,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.77000000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.75600000000000001}]},{"osd":7,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.82399999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297636,"kb_used_data":348,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074204,"statfs":{"total":96636764160,"available":96331984896,"internally_reserved":0,"allocated":356352,"data_stored":134358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:31:33.076 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-09T06:31:33.077 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-09T06:31:33.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.230+0000 7fde4f773700 1 Processor -- start 2023-10-09T06:31:33.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.230+0000 7fde4f773700 1 -- start start 2023-10-09T06:31:33.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.231+0000 7fde4f773700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fde48069c70 0x7fde480cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:33.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.231+0000 7fde4f773700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde480ce380 con 0x7fde48159950 2023-10-09T06:31:33.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.231+0000 7fde4f773700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde480cec20 con 0x7fde480af320 2023-10-09T06:31:33.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.231+0000 7fde4f773700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde480afa90 con 0x7fde48069c70 2023-10-09T06:31:33.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.231+0000 7fde4d50f700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fde48159950 0x7fde48069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33890/0 (socket says 172.21.15.163:33890) 2023-10-09T06:31:33.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.231+0000 7fde4d50f700 1 -- 172.21.15.163:0/281010572 learned_addr learned my addr 172.21.15.163:0/281010572 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:33.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.231+0000 7fde4cd0e700 1 --2- 172.21.15.163:0/281010572 >> v2:172.21.15.181:3300/0 conn(0x7fde48069c70 0x7fde480cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:33.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.231+0000 7fde3ffff700 1 -- 172.21.15.163:0/281010572 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2902220304 0 0) 0x7fde480ce380 con 0x7fde48159950 2023-10-09T06:31:33.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.231+0000 7fde3ffff700 1 -- 172.21.15.163:0/281010572 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde34003570 con 0x7fde48159950 2023-10-09T06:31:33.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.231+0000 7fde3ffff700 1 -- 172.21.15.163:0/281010572 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1321117861 0 0) 0x7fde34003570 con 0x7fde48159950 2023-10-09T06:31:33.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.231+0000 7fde3ffff700 1 -- 172.21.15.163:0/281010572 >> v2:172.21.15.181:3300/0 conn(0x7fde48069c70 msgr2=0x7fde480cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:33.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.231+0000 7fde3ffff700 1 --2- 172.21.15.163:0/281010572 >> v2:172.21.15.181:3300/0 conn(0x7fde48069c70 0x7fde480cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:33.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.231+0000 7fde3ffff700 1 -- 172.21.15.163:0/281010572 >> v1:172.21.15.181:6789/0 conn(0x7fde480af320 legacy=0x7fde480cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:33.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.231+0000 7fde3ffff700 1 -- 172.21.15.163:0/281010572 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde480ce5d0 con 0x7fde48159950 2023-10-09T06:31:33.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.231+0000 7fde3ffff700 1 -- 172.21.15.163:0/281010572 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fde440017f0 con 0x7fde48159950 2023-10-09T06:31:33.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.231+0000 7fde3ffff700 1 -- 172.21.15.163:0/281010572 >> v1:172.21.15.163:6789/0 conn(0x7fde48159950 legacy=0x7fde48069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:33.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.231+0000 7fde3ffff700 1 --2- 172.21.15.163:0/281010572 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde34003a60 0x7fde34003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:33.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.231+0000 7fde3ffff700 1 --2- 172.21.15.163:0/281010572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde34004810 0x7fde34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:33.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.231+0000 7fde3ffff700 1 --2- 172.21.15.163:0/281010572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde340051c0 0x7fde3400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:33.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.231+0000 7fde3ffff700 1 -- 172.21.15.163:0/281010572 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde480afa90 con 0x7fde34004810 2023-10-09T06:31:33.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.231+0000 7fde3ffff700 1 -- 172.21.15.163:0/281010572 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3400c160 con 0x7fde340051c0 2023-10-09T06:31:33.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.231+0000 7fde3ffff700 1 -- 172.21.15.163:0/281010572 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3400c320 con 0x7fde34003a60 2023-10-09T06:31:33.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.231+0000 7fde4cd0e700 1 --2- 172.21.15.163:0/281010572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde34004810 0x7fde34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:33.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.232+0000 7fde4d50f700 1 --2- 172.21.15.163:0/281010572 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde34003a60 0x7fde34003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:33.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.232+0000 7fde4dd10700 1 --2- 172.21.15.163:0/281010572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde340051c0 0x7fde3400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:33.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.232+0000 7fde4cd0e700 1 -- 172.21.15.163:0/281010572 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde34003a60 msgr2=0x7fde34003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:33.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.232+0000 7fde4cd0e700 1 --2- 172.21.15.163:0/281010572 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde34003a60 0x7fde34003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:33.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.232+0000 7fde4cd0e700 1 -- 172.21.15.163:0/281010572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde340051c0 msgr2=0x7fde3400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:33.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.232+0000 7fde4cd0e700 1 --2- 172.21.15.163:0/281010572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde340051c0 0x7fde3400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:33.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.232+0000 7fde4dd10700 1 -- 172.21.15.163:0/281010572 reap_dead start 2023-10-09T06:31:33.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.232+0000 7fde4cd0e700 1 -- 172.21.15.163:0/281010572 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde34003570 con 0x7fde34004810 2023-10-09T06:31:33.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.232+0000 7fde4cd0e700 1 --2- 172.21.15.163:0/281010572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde34004810 0x7fde34004c80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fde38009c00 tx=0x7fde380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cee86630bb93999 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:33.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.232+0000 7fde3ffff700 1 -- 172.21.15.163:0/281010572 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3800e710 con 0x7fde34004810 2023-10-09T06:31:33.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.232+0000 7fde3ffff700 1 -- 172.21.15.163:0/281010572 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3800e870 con 0x7fde34004810 2023-10-09T06:31:33.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.232+0000 7fde3ffff700 1 -- 172.21.15.163:0/281010572 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3800ec30 con 0x7fde34004810 2023-10-09T06:31:33.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.232+0000 7fde4f773700 1 -- 172.21.15.163:0/281010572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde34004810 msgr2=0x7fde34004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:33.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.232+0000 7fde4f773700 1 --2- 172.21.15.163:0/281010572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde34004810 0x7fde34004c80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fde38009c00 tx=0x7fde380091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:33.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.233+0000 7fde4f773700 1 -- 172.21.15.163:0/281010572 shutdown_connections 2023-10-09T06:31:33.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.233+0000 7fde4f773700 1 --2- 172.21.15.163:0/281010572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde34004810 0x7fde34004c80 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:33.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.233+0000 7fde4f773700 1 -- 172.21.15.163:0/281010572 >> 172.21.15.163:0/281010572 conn(0x7fde48068e00 msgr2=0x7fde480d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:33.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.233+0000 7fde4f773700 1 -- 172.21.15.163:0/281010572 shutdown_connections 2023-10-09T06:31:33.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.233+0000 7fde4f773700 1 -- 172.21.15.163:0/281010572 wait complete. 2023-10-09T06:31:33.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.234+0000 7fde4f773700 1 Processor -- start 2023-10-09T06:31:33.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.234+0000 7fde4f773700 1 -- start start 2023-10-09T06:31:33.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.235+0000 7fde4f773700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde34004810 0x7fde480ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:33.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.235+0000 7fde4f773700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde4815a620 0x7fde4815aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:33.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.235+0000 7fde4f773700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde480c7860 0x7fde480c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:33.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.235+0000 7fde4f773700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde480c8240 con 0x7fde480c7860 2023-10-09T06:31:33.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.235+0000 7fde4f773700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde480c9bf0 con 0x7fde4815a620 2023-10-09T06:31:33.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.235+0000 7fde4f773700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde4815afd0 con 0x7fde34004810 2023-10-09T06:31:33.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.235+0000 7fde4dd10700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde480c7860 0x7fde480c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:33.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.235+0000 7fde4dd10700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde480c7860 0x7fde480c7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33484/0 (socket says 172.21.15.163:33484) 2023-10-09T06:31:33.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.235+0000 7fde4dd10700 1 -- 172.21.15.163:0/564012220 learned_addr learned my addr 172.21.15.163:0/564012220 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:33.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.235+0000 7fde4d50f700 1 --2- 172.21.15.163:0/564012220 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde34004810 0x7fde480ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:33.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.235+0000 7fde4cd0e700 1 --2- 172.21.15.163:0/564012220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde4815a620 0x7fde4815aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:33.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.235+0000 7fde4dd10700 1 -- 172.21.15.163:0/564012220 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde34004810 msgr2=0x7fde480ca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:33.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.235+0000 7fde4dd10700 1 --2- 172.21.15.163:0/564012220 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde34004810 0x7fde480ca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:33.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.235+0000 7fde4dd10700 1 -- 172.21.15.163:0/564012220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde4815a620 msgr2=0x7fde4815aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:33.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.235+0000 7fde4dd10700 1 --2- 172.21.15.163:0/564012220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde4815a620 0x7fde4815aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:33.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.235+0000 7fde4dd10700 1 -- 172.21.15.163:0/564012220 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4815be10 con 0x7fde480c7860 2023-10-09T06:31:33.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.235+0000 7fde4dd10700 1 --2- 172.21.15.163:0/564012220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde480c7860 0x7fde480c7d00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fde480cad80 tx=0x7fde40009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:33.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.235+0000 7fde3d7fa700 1 -- 172.21.15.163:0/564012220 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4000ee40 con 0x7fde480c7860 2023-10-09T06:31:33.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.235+0000 7fde3d7fa700 1 -- 172.21.15.163:0/564012220 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde40006820 con 0x7fde480c7860 2023-10-09T06:31:33.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.235+0000 7fde4f773700 1 -- 172.21.15.163:0/564012220 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde4815b200 con 0x7fde480c7860 2023-10-09T06:31:33.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.235+0000 7fde3d7fa700 1 -- 172.21.15.163:0/564012220 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40006be0 con 0x7fde480c7860 2023-10-09T06:31:33.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.235+0000 7fde4f773700 1 -- 172.21.15.163:0/564012220 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4815b7c0 con 0x7fde480c7860 2023-10-09T06:31:33.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.236+0000 7fde4f773700 1 -- 172.21.15.163:0/564012220 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4815bfa0 con 0x7fde480c7860 2023-10-09T06:31:33.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.237+0000 7fde3d7fa700 1 -- 172.21.15.163:0/564012220 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fde40007030 con 0x7fde480c7860 2023-10-09T06:31:33.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.237+0000 7fde3d7fa700 1 --2- 172.21.15.163:0/564012220 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fde2c066f30 0x7fde2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:33.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.237+0000 7fde3d7fa700 1 -- 172.21.15.163:0/564012220 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fde40083640 con 0x7fde480c7860 2023-10-09T06:31:33.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.237+0000 7fde4d50f700 1 --2- 172.21.15.163:0/564012220 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fde2c066f30 0x7fde2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:33.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.238+0000 7fde4d50f700 1 --2- 172.21.15.163:0/564012220 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fde2c066f30 0x7fde2c0693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fde44001350 tx=0x7fde480ceb90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:33.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.241+0000 7fde3d7fa700 1 -- 172.21.15.163:0/564012220 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde4004d5a0 con 0x7fde480c7860 2023-10-09T06:31:33.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.504+0000 7fde4f773700 1 -- 172.21.15.163:0/564012220 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fde480c66d0 con 0x7fde480c7860 2023-10-09T06:31:33.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.507+0000 7fde3d7fa700 1 -- 172.21.15.163:0/564012220 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7fde400543f0 con 0x7fde480c7860 2023-10-09T06:31:33.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.509+0000 7fde4f773700 1 -- 172.21.15.163:0/564012220 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fde2c066f30 msgr2=0x7fde2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:33.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.509+0000 7fde4f773700 1 --2- 172.21.15.163:0/564012220 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fde2c066f30 0x7fde2c0693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fde44001350 tx=0x7fde480ceb90 comp rx=0 tx=0).stop 2023-10-09T06:31:33.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.509+0000 7fde4f773700 1 -- 172.21.15.163:0/564012220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde480c7860 msgr2=0x7fde480c7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:33.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.509+0000 7fde4f773700 1 --2- 172.21.15.163:0/564012220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde480c7860 0x7fde480c7d00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fde480cad80 tx=0x7fde40009040 comp rx=0 tx=0).stop 2023-10-09T06:31:33.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.509+0000 7fde4f773700 1 -- 172.21.15.163:0/564012220 shutdown_connections 2023-10-09T06:31:33.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.509+0000 7fde4f773700 1 --2- 172.21.15.163:0/564012220 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fde2c066f30 0x7fde2c0693e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:33.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.509+0000 7fde4f773700 1 --2- 172.21.15.163:0/564012220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde480c7860 0x7fde480c7d00 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:33.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.509+0000 7fde4f773700 1 --2- 172.21.15.163:0/564012220 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde34004810 0x7fde480ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:33.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.509+0000 7fde4f773700 1 --2- 172.21.15.163:0/564012220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde4815a620 0x7fde4815aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:33.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.509+0000 7fde4f773700 1 -- 172.21.15.163:0/564012220 >> 172.21.15.163:0/564012220 conn(0x7fde48068e00 msgr2=0x7fde480cf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:33.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.509+0000 7fde4f773700 1 -- 172.21.15.163:0/564012220 shutdown_connections 2023-10-09T06:31:33.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.509+0000 7fde4f773700 1 -- 172.21.15.163:0/564012220 wait complete. 2023-10-09T06:31:33.512 INFO:teuthology.orchestra.run.smithi163.stderr:updated caps for client.0 2023-10-09T06:31:33.522 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:31:33.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.636+0000 7f97bd070700 1 Processor -- start 2023-10-09T06:31:33.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.636+0000 7f97bd070700 1 -- start start 2023-10-09T06:31:33.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.637+0000 7f97bd070700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f97b815d230 0x7f97b815d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:33.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.637+0000 7f97bd070700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b80b2980 con 0x7f97b81597f0 2023-10-09T06:31:33.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.637+0000 7f97bd070700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b815db70 con 0x7f97b8159420 2023-10-09T06:31:33.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.637+0000 7f97bd070700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b80b11d0 con 0x7f97b815d230 2023-10-09T06:31:33.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.637+0000 7f97bc86f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f97b815d230 0x7f97b815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:33.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.637+0000 7f97b77fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f97b81597f0 0x7f97b80b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33900/0 (socket says 172.21.15.163:33900) 2023-10-09T06:31:33.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.637+0000 7f97b77fe700 1 -- 172.21.15.163:0/922536130 learned_addr learned my addr 172.21.15.163:0/922536130 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:33.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.637+0000 7f97bc86f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f97b815d230 0x7f97b815d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33486/0 (socket says 172.21.15.163:33486) 2023-10-09T06:31:33.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.637+0000 7f97bc86f700 1 -- 172.21.15.163:0/922536130 >> v1:172.21.15.181:6789/0 conn(0x7f97b8159420 legacy=0x7f97b80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:33.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.637+0000 7f97bc86f700 1 -- 172.21.15.163:0/922536130 >> v1:172.21.15.163:6789/0 conn(0x7f97b81597f0 legacy=0x7f97b80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:33.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.637+0000 7f97bc86f700 1 -- 172.21.15.163:0/922536130 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b80b14b0 con 0x7f97b815d230 2023-10-09T06:31:33.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.637+0000 7f97b6ffd700 1 -- 172.21.15.163:0/922536130 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 559570318 0 0) 0x7f97b80b2980 con 0x7f97b81597f0 2023-10-09T06:31:33.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.637+0000 7f97bc86f700 1 --2- 172.21.15.163:0/922536130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97b815d230 0x7f97b815d630 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f97b0009e60 tx=0x7f97b0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3746c6853476b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:33.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.638+0000 7f97b6ffd700 1 -- 172.21.15.163:0/922536130 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b000e070 con 0x7f97b815d230 2023-10-09T06:31:33.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.638+0000 7f97b6ffd700 1 -- 172.21.15.163:0/922536130 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b000bc30 con 0x7f97b815d230 2023-10-09T06:31:33.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.638+0000 7f97b6ffd700 1 -- 172.21.15.163:0/922536130 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b0012430 con 0x7f97b815d230 2023-10-09T06:31:33.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.638+0000 7f97bd070700 1 -- 172.21.15.163:0/922536130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97b815d230 msgr2=0x7f97b815d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:33.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.638+0000 7f97bd070700 1 --2- 172.21.15.163:0/922536130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97b815d230 0x7f97b815d630 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f97b0009e60 tx=0x7f97b0009780 comp rx=0 tx=0).stop 2023-10-09T06:31:33.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.638+0000 7f97bd070700 1 -- 172.21.15.163:0/922536130 shutdown_connections 2023-10-09T06:31:33.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.638+0000 7f97bd070700 1 --2- 172.21.15.163:0/922536130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97b815d230 0x7f97b815d630 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:33.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.638+0000 7f97bd070700 1 -- 172.21.15.163:0/922536130 >> 172.21.15.163:0/922536130 conn(0x7f97b80d0660 msgr2=0x7f97b80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:33.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.638+0000 7f97bd070700 1 -- 172.21.15.163:0/922536130 shutdown_connections 2023-10-09T06:31:33.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.638+0000 7f97bd070700 1 -- 172.21.15.163:0/922536130 wait complete. 2023-10-09T06:31:33.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.639+0000 7f97bd070700 1 Processor -- start 2023-10-09T06:31:33.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.639+0000 7f97bd070700 1 -- start start 2023-10-09T06:31:33.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.640+0000 7f97bd070700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97b8159420 0x7f97b80c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:33.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.640+0000 7f97bd070700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97b81597f0 0x7f97b815c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:33.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.640+0000 7f97bd070700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97b815d230 0x7f97b8155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:33.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.640+0000 7f97bd070700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b815b600 con 0x7f97b8159420 2023-10-09T06:31:33.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.640+0000 7f97bd070700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b815a9f0 con 0x7f97b815d230 2023-10-09T06:31:33.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.640+0000 7f97bd070700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b815acd0 con 0x7f97b81597f0 2023-10-09T06:31:33.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.640+0000 7f97b7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97b8159420 0x7f97b80c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:33.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.640+0000 7f97b7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97b8159420 0x7f97b80c5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33488/0 (socket says 172.21.15.163:33488) 2023-10-09T06:31:33.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.640+0000 7f97b7fff700 1 -- 172.21.15.163:0/3150443817 learned_addr learned my addr 172.21.15.163:0/3150443817 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:33.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.640+0000 7f97bc86f700 1 --2- 172.21.15.163:0/3150443817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97b815d230 0x7f97b8155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:33.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.640+0000 7f97b77fe700 1 --2- 172.21.15.163:0/3150443817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97b81597f0 0x7f97b815c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:33.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.640+0000 7f97b7fff700 1 -- 172.21.15.163:0/3150443817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97b81597f0 msgr2=0x7f97b815c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:33.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.641+0000 7f97b7fff700 1 --2- 172.21.15.163:0/3150443817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97b81597f0 0x7f97b815c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:33.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.641+0000 7f97b7fff700 1 -- 172.21.15.163:0/3150443817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97b815d230 msgr2=0x7f97b8155ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:33.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.641+0000 7f97b7fff700 1 --2- 172.21.15.163:0/3150443817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97b815d230 0x7f97b8155ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:33.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.641+0000 7f97b7fff700 1 -- 172.21.15.163:0/3150443817 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b0009a20 con 0x7f97b8159420 2023-10-09T06:31:33.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.641+0000 7f97bc86f700 1 --2- 172.21.15.163:0/3150443817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97b815d230 0x7f97b8155ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:33.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.641+0000 7f97b7fff700 1 --2- 172.21.15.163:0/3150443817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97b8159420 0x7f97b80c5400 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f97a800c590 tx=0x7f97a800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:33.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.641+0000 7f979ffff700 1 -- 172.21.15.163:0/3150443817 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a8011590 con 0x7f97b8159420 2023-10-09T06:31:33.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.641+0000 7f979ffff700 1 -- 172.21.15.163:0/3150443817 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a80116f0 con 0x7f97b8159420 2023-10-09T06:31:33.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.641+0000 7f979ffff700 1 -- 172.21.15.163:0/3150443817 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a8011ab0 con 0x7f97b8159420 2023-10-09T06:31:33.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.641+0000 7f97bd070700 1 -- 172.21.15.163:0/3150443817 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b815afb0 con 0x7f97b8159420 2023-10-09T06:31:33.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.641+0000 7f97bd070700 1 -- 172.21.15.163:0/3150443817 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b815ba90 con 0x7f97b8159420 2023-10-09T06:31:33.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.642+0000 7f97bd070700 1 -- 172.21.15.163:0/3150443817 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b804c3c0 con 0x7f97b8159420 2023-10-09T06:31:33.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.642+0000 7f979ffff700 1 -- 172.21.15.163:0/3150443817 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f97a801b3b0 con 0x7f97b8159420 2023-10-09T06:31:33.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.643+0000 7f979ffff700 1 --2- 172.21.15.163:0/3150443817 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f97a4066f30 0x7f97a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:33.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.643+0000 7f979ffff700 1 -- 172.21.15.163:0/3150443817 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f97a8083a90 con 0x7f97b8159420 2023-10-09T06:31:33.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.643+0000 7f97b77fe700 1 --2- 172.21.15.163:0/3150443817 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f97a4066f30 0x7f97a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:33.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.643+0000 7f97b77fe700 1 --2- 172.21.15.163:0/3150443817 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f97a4066f30 0x7f97a40693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f97ac007fd0 tx=0x7f97ac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:33.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.647+0000 7f979ffff700 1 -- 172.21.15.163:0/3150443817 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97a8050d00 con 0x7f97b8159420 2023-10-09T06:31:33.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.872+0000 7f97bd070700 1 -- 172.21.15.163:0/3150443817 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97b80b1790 con 0x7f97b8159420 2023-10-09T06:31:33.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.873+0000 7f979ffff700 1 -- 172.21.15.163:0/3150443817 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f97a8017020 con 0x7f97b8159420 2023-10-09T06:31:33.874 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:33.874 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4622,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2368375038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2368375038},{"type":"v1","addr":"172.21.15.181:6836","nonce":2368375038}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4929,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/4112854733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4112854733},{"type":"v1","addr":"172.21.15.163:6834","nonce":4112854733}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4932,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2241937708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2241937708},{"type":"v1","addr":"172.21.15.163:6835","nonce":2241937708}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:31:28.011264+0000","modified":"2023-10-09T06:31:31.660915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4944},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4944":{"gid":4944,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2708262268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2708262268},{"type":"v1","addr":"172.21.15.181:6837","nonce":2708262268}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-09T06:31:33.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.875+0000 7f97bd070700 1 -- 172.21.15.163:0/3150443817 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f97a4066f30 msgr2=0x7f97a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:33.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.875+0000 7f97bd070700 1 --2- 172.21.15.163:0/3150443817 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f97a4066f30 0x7f97a40693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f97ac007fd0 tx=0x7f97ac009040 comp rx=0 tx=0).stop 2023-10-09T06:31:33.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.875+0000 7f97bd070700 1 -- 172.21.15.163:0/3150443817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97b8159420 msgr2=0x7f97b80c5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:33.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.875+0000 7f97bd070700 1 --2- 172.21.15.163:0/3150443817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97b8159420 0x7f97b80c5400 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f97a800c590 tx=0x7f97a800c950 comp rx=0 tx=0).stop 2023-10-09T06:31:33.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.875+0000 7f97bd070700 1 -- 172.21.15.163:0/3150443817 shutdown_connections 2023-10-09T06:31:33.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.875+0000 7f97bd070700 1 --2- 172.21.15.163:0/3150443817 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f97a4066f30 0x7f97a40693e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:33.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.875+0000 7f97bd070700 1 --2- 172.21.15.163:0/3150443817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97b8159420 0x7f97b80c5400 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:33.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.875+0000 7f97bd070700 1 --2- 172.21.15.163:0/3150443817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97b815d230 0x7f97b8155ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:33.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.875+0000 7f97bd070700 1 --2- 172.21.15.163:0/3150443817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97b81597f0 0x7f97b815c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:33.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.875+0000 7f97bd070700 1 -- 172.21.15.163:0/3150443817 >> 172.21.15.163:0/3150443817 conn(0x7f97b80d0660 msgr2=0x7f97b80d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:33.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.875+0000 7f97bd070700 1 -- 172.21.15.163:0/3150443817 shutdown_connections 2023-10-09T06:31:33.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:33.875+0000 7f97bd070700 1 -- 172.21.15.163:0/3150443817 wait complete. 2023-10-09T06:31:33.878 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 17 2023-10-09T06:31:34.290 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-09T06:31:28.011264+0000', 'modified': '2023-10-09T06:31:31.660915+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4944}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4944': {'gid': 4944, 'name': 'd', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6837/2708262268', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6835', 'nonce': 2708262268}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 2708262268}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-09T06:31:34.291 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-09T06:31:34.291 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-09T06:31:34.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.407+0000 7f6f9edcb700 1 Processor -- start 2023-10-09T06:31:34.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.407+0000 7f6f9edcb700 1 -- start start 2023-10-09T06:31:34.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.408+0000 7f6f9edcb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f9806bcd0 0x7f6f980caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:34.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.408+0000 7f6f9edcb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6f9806c0a0 0x7f6f9815a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:34.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.408+0000 7f6f9edcb700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6f9815df50 0x7f6f9815e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:34.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.408+0000 7f6f9edcb700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f9815d7e0 con 0x7f6f9815df50 2023-10-09T06:31:34.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.408+0000 7f6f9edcb700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f9815dac0 con 0x7f6f9806c0a0 2023-10-09T06:31:34.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.408+0000 7f6f9edcb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9815aa30 con 0x7f6f9806bcd0 2023-10-09T06:31:34.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.408+0000 7f6f9d368700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6f9815df50 0x7f6f9815e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:34.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.408+0000 7f6f9d368700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6f9815df50 0x7f6f9815e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33496/0 (socket says 172.21.15.163:33496) 2023-10-09T06:31:34.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.408+0000 7f6f9d368700 1 -- 172.21.15.163:0/2385010951 learned_addr learned my addr 172.21.15.163:0/2385010951 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:34.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.408+0000 7f6f9cb67700 1 --2- 172.21.15.163:0/2385010951 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f9806bcd0 0x7f6f980caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:34.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.408+0000 7f6f8ffff700 1 --2- 172.21.15.163:0/2385010951 >> v2:172.21.15.181:3300/0 conn(0x7f6f9806c0a0 0x7f6f9815a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:34.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.409+0000 7f6f9d368700 1 -- 172.21.15.163:0/2385010951 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f9806bcd0 msgr2=0x7f6f980caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:34.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.409+0000 7f6f9d368700 1 --2- 172.21.15.163:0/2385010951 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f9806bcd0 0x7f6f980caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.409+0000 7f6f9d368700 1 -- 172.21.15.163:0/2385010951 >> v2:172.21.15.181:3300/0 conn(0x7f6f9806c0a0 msgr2=0x7f6f9815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:34.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.409+0000 7f6f9d368700 1 --2- 172.21.15.163:0/2385010951 >> v2:172.21.15.181:3300/0 conn(0x7f6f9806c0a0 0x7f6f9815a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.409+0000 7f6f9d368700 1 -- 172.21.15.163:0/2385010951 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9815ad10 con 0x7f6f9815df50 2023-10-09T06:31:34.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.409+0000 7f6f9d368700 1 --2- 172.21.15.163:0/2385010951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f9815df50 0x7f6f9815e3b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6f980c5660 tx=0x7f6f90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4402a2d102154fc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:34.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.409+0000 7f6f8f7fe700 1 -- 172.21.15.163:0/2385010951 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9000b640 con 0x7f6f9815df50 2023-10-09T06:31:34.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.409+0000 7f6f8f7fe700 1 -- 172.21.15.163:0/2385010951 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9000b7a0 con 0x7f6f9815df50 2023-10-09T06:31:34.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.409+0000 7f6f8f7fe700 1 -- 172.21.15.163:0/2385010951 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9000bb40 con 0x7f6f9815df50 2023-10-09T06:31:34.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.409+0000 7f6f9edcb700 1 -- 172.21.15.163:0/2385010951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f9815df50 msgr2=0x7f6f9815e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:34.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.409+0000 7f6f9edcb700 1 --2- 172.21.15.163:0/2385010951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f9815df50 0x7f6f9815e3b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6f980c5660 tx=0x7f6f90009250 comp rx=0 tx=0).stop 2023-10-09T06:31:34.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.410+0000 7f6f9edcb700 1 -- 172.21.15.163:0/2385010951 shutdown_connections 2023-10-09T06:31:34.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.410+0000 7f6f9edcb700 1 --2- 172.21.15.163:0/2385010951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f9815df50 0x7f6f9815e3b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.410+0000 7f6f9edcb700 1 --2- 172.21.15.163:0/2385010951 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f9806bcd0 0x7f6f980caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.410+0000 7f6f9edcb700 1 --2- 172.21.15.163:0/2385010951 >> v2:172.21.15.181:3300/0 conn(0x7f6f9806c0a0 0x7f6f9815a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.410+0000 7f6f9edcb700 1 -- 172.21.15.163:0/2385010951 >> 172.21.15.163:0/2385010951 conn(0x7f6f980ce600 msgr2=0x7f6f980cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:34.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.410+0000 7f6f9edcb700 1 -- 172.21.15.163:0/2385010951 shutdown_connections 2023-10-09T06:31:34.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.410+0000 7f6f9edcb700 1 -- 172.21.15.163:0/2385010951 wait complete. 2023-10-09T06:31:34.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.411+0000 7f6f9edcb700 1 Processor -- start 2023-10-09T06:31:34.416 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.411+0000 7f6f9edcb700 1 -- start start 2023-10-09T06:31:34.416 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.411+0000 7f6f9edcb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f9806bcd0 0x7f6f98154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:34.416 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.412+0000 7f6f9edcb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f9806c0a0 0x7f6f980b2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:34.416 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.412+0000 7f6f9edcb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f9815df50 0x7f6f980b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:34.416 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.412+0000 7f6f9edcb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f980b3720 con 0x7f6f9806c0a0 2023-10-09T06:31:34.416 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.412+0000 7f6f9edcb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f980b2b10 con 0x7f6f9815df50 2023-10-09T06:31:34.416 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.412+0000 7f6f9edcb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f980b2df0 con 0x7f6f9806bcd0 2023-10-09T06:31:34.416 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.412+0000 7f6f8ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f9806c0a0 0x7f6f980b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:34.416 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.412+0000 7f6f8ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f9806c0a0 0x7f6f980b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33502/0 (socket says 172.21.15.163:33502) 2023-10-09T06:31:34.416 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.412+0000 7f6f8ffff700 1 -- 172.21.15.163:0/1578309810 learned_addr learned my addr 172.21.15.163:0/1578309810 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:34.416 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.412+0000 7f6f9d368700 1 --2- 172.21.15.163:0/1578309810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f9815df50 0x7f6f980b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:34.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.412+0000 7f6f9cb67700 1 --2- 172.21.15.163:0/1578309810 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f9806bcd0 0x7f6f98154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:34.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.412+0000 7f6f8ffff700 1 -- 172.21.15.163:0/1578309810 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f9806bcd0 msgr2=0x7f6f98154250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:34.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.412+0000 7f6f8ffff700 1 --2- 172.21.15.163:0/1578309810 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f9806bcd0 0x7f6f98154250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.412+0000 7f6f8ffff700 1 -- 172.21.15.163:0/1578309810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f9815df50 msgr2=0x7f6f980b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:34.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.412+0000 7f6f8ffff700 1 --2- 172.21.15.163:0/1578309810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f9815df50 0x7f6f980b3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.412+0000 7f6f8ffff700 1 -- 172.21.15.163:0/1578309810 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f900094f0 con 0x7f6f9806c0a0 2023-10-09T06:31:34.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.412+0000 7f6f9cb67700 1 --2- 172.21.15.163:0/1578309810 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f9806bcd0 0x7f6f98154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:34.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.412+0000 7f6f9d368700 1 --2- 172.21.15.163:0/1578309810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f9815df50 0x7f6f980b3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:34.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.412+0000 7f6f8ffff700 1 --2- 172.21.15.163:0/1578309810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f9806c0a0 0x7f6f980b2270 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800e930 tx=0x7f6f880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:34.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.412+0000 7f6f8cff9700 1 -- 172.21.15.163:0/1578309810 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8800eed0 con 0x7f6f9806c0a0 2023-10-09T06:31:34.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.412+0000 7f6f8cff9700 1 -- 172.21.15.163:0/1578309810 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8800c850 con 0x7f6f9806c0a0 2023-10-09T06:31:34.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.412+0000 7f6f9edcb700 1 -- 172.21.15.163:0/1578309810 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f980b30d0 con 0x7f6f9806c0a0 2023-10-09T06:31:34.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.412+0000 7f6f8cff9700 1 -- 172.21.15.163:0/1578309810 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8800cbf0 con 0x7f6f9806c0a0 2023-10-09T06:31:34.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.412+0000 7f6f9edcb700 1 -- 172.21.15.163:0/1578309810 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f980b27b0 con 0x7f6f9806c0a0 2023-10-09T06:31:34.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.414+0000 7f6f8cff9700 1 -- 172.21.15.163:0/1578309810 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6f88013470 con 0x7f6f9806c0a0 2023-10-09T06:31:34.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.414+0000 7f6f8cff9700 1 --2- 172.21.15.163:0/1578309810 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6f84066f30 0x7f6f840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:34.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.414+0000 7f6f8cff9700 1 -- 172.21.15.163:0/1578309810 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6f88084720 con 0x7f6f9806c0a0 2023-10-09T06:31:34.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.414+0000 7f6f9cb67700 1 --2- 172.21.15.163:0/1578309810 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6f84066f30 0x7f6f840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:34.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.415+0000 7f6f9cb67700 1 --2- 172.21.15.163:0/1578309810 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6f84066f30 0x7f6f840693e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6f94007620 tx=0x7f6f9400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:34.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.416+0000 7f6fa80c3b80 1 Processor -- start 2023-10-09T06:31:34.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.416+0000 7f6fa80c3b80 1 -- start start 2023-10-09T06:31:34.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.417+0000 7f6fa80c3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98b30ba50 0x55d98b30be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:34.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.417+0000 7f6fa80c3b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d98b355d00 0x55d98b334a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:34.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.417+0000 7f6fa80c3b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d98b334f60 0x55d98b307cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:34.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.417+0000 7f6fa80c3b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98b262a70 con 0x55d98b334f60 2023-10-09T06:31:34.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.417+0000 7f6fa80c3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98b260270 con 0x55d98b30ba50 2023-10-09T06:31:34.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.417+0000 7f6fa80c3b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d98b25c060 con 0x55d98b355d00 2023-10-09T06:31:34.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.417+0000 7f6f9d368700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d98b334f60 0x55d98b307cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:34.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.417+0000 7f6f9d368700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d98b334f60 0x55d98b307cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33512/0 (socket says 172.21.15.163:33512) 2023-10-09T06:31:34.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.417+0000 7f6f9d368700 1 -- 172.21.15.163:0/953889714 learned_addr learned my addr 172.21.15.163:0/953889714 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:34.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.417+0000 7f6f9cb67700 1 --2- 172.21.15.163:0/953889714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98b30ba50 0x55d98b30be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:34.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.417+0000 7f6f8ffff700 1 --2- 172.21.15.163:0/953889714 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d98b355d00 0x55d98b334a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:34.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.417+0000 7f6f9d368700 1 -- 172.21.15.163:0/953889714 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d98b355d00 msgr2=0x55d98b334a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:34.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.417+0000 7f6f9d368700 1 --2- 172.21.15.163:0/953889714 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d98b355d00 0x55d98b334a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.417+0000 7f6f9d368700 1 -- 172.21.15.163:0/953889714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98b30ba50 msgr2=0x55d98b30be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:34.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.417+0000 7f6f9d368700 1 --2- 172.21.15.163:0/953889714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98b30ba50 0x55d98b30be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.417+0000 7f6f9d368700 1 -- 172.21.15.163:0/953889714 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d98b0f6270 con 0x55d98b334f60 2023-10-09T06:31:34.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.417+0000 7f6f9d368700 1 --2- 172.21.15.163:0/953889714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d98b334f60 0x55d98b307cf0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6f90006fd0 tx=0x7f6f90017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e313de7a4e9a7853 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:34.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.417+0000 7f6f7dffb700 1 -- 172.21.15.163:0/953889714 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f90020070 con 0x55d98b334f60 2023-10-09T06:31:34.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.417+0000 7f6f7dffb700 1 -- 172.21.15.163:0/953889714 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9000beb0 con 0x55d98b334f60 2023-10-09T06:31:34.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.417+0000 7f6f7dffb700 1 -- 172.21.15.163:0/953889714 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f90006d50 con 0x55d98b334f60 2023-10-09T06:31:34.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.418+0000 7f6fa80c3b80 1 -- 172.21.15.163:0/953889714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d98b334f60 msgr2=0x55d98b307cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:34.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.418+0000 7f6fa80c3b80 1 --2- 172.21.15.163:0/953889714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d98b334f60 0x55d98b307cf0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6f90006fd0 tx=0x7f6f90017000 comp rx=0 tx=0).stop 2023-10-09T06:31:34.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.418+0000 7f6fa80c3b80 1 -- 172.21.15.163:0/953889714 shutdown_connections 2023-10-09T06:31:34.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.418+0000 7f6fa80c3b80 1 --2- 172.21.15.163:0/953889714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d98b334f60 0x55d98b307cf0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.418+0000 7f6fa80c3b80 1 --2- 172.21.15.163:0/953889714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98b30ba50 0x55d98b30be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.418+0000 7f6fa80c3b80 1 --2- 172.21.15.163:0/953889714 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d98b355d00 0x55d98b334a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.418+0000 7f6fa80c3b80 1 -- 172.21.15.163:0/953889714 >> 172.21.15.163:0/953889714 conn(0x55d98b30d370 msgr2=0x55d98b30d740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:34.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.418+0000 7f6fa80c3b80 1 -- 172.21.15.163:0/953889714 shutdown_connections 2023-10-09T06:31:34.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.418+0000 7f6fa80c3b80 1 -- 172.21.15.163:0/953889714 wait complete. 2023-10-09T06:31:34.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.419+0000 7f6fa80c3b80 1 -- start start 2023-10-09T06:31:34.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.419+0000 7f6fa80c3b80 1 Processor -- start 2023-10-09T06:31:34.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.419+0000 7f6fa80c3b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d98b30ba50 0x55d98b362a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:34.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.419+0000 7f6fa80c3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98b334f60 0x55d98b306c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:34.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.419+0000 7f6fa80c3b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d98b355d00 0x55d98b307160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:34.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.419+0000 7f6fa80c3b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98b207890 con 0x55d98b355d00 2023-10-09T06:31:34.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.419+0000 7f6fa80c3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98b222ea0 con 0x55d98b334f60 2023-10-09T06:31:34.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.419+0000 7f6fa80c3b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d98b0dee30 con 0x55d98b30ba50 2023-10-09T06:31:34.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.420+0000 7f6fa80c3b80 10 client.0 _ll_register_callbacks cb 0x55d98b2cf110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-09T06:31:34.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.420+0000 7f6f9d368700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d98b355d00 0x55d98b307160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:34.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.420+0000 7f6f9d368700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d98b355d00 0x55d98b307160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33514/0 (socket says 172.21.15.163:33514) 2023-10-09T06:31:34.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.420+0000 7f6f9d368700 1 -- 172.21.15.163:0/2377315059 learned_addr learned my addr 172.21.15.163:0/2377315059 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:34.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.420+0000 7f6f9cb67700 1 --2- 172.21.15.163:0/2377315059 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d98b30ba50 0x55d98b362a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:34.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.420+0000 7f6f8ffff700 1 --2- 172.21.15.163:0/2377315059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98b334f60 0x55d98b306c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:34.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.420+0000 7f6f9d368700 1 -- 172.21.15.163:0/2377315059 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d98b30ba50 msgr2=0x55d98b362a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:34.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.420+0000 7f6f9d368700 1 --2- 172.21.15.163:0/2377315059 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d98b30ba50 0x55d98b362a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.420+0000 7f6f9d368700 1 -- 172.21.15.163:0/2377315059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98b334f60 msgr2=0x55d98b306c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:34.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.420+0000 7f6f9d368700 1 --2- 172.21.15.163:0/2377315059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98b334f60 0x55d98b306c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.420+0000 7f6f9d368700 1 -- 172.21.15.163:0/2377315059 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9815ad10 con 0x55d98b355d00 2023-10-09T06:31:34.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.420+0000 7f6f9d368700 1 --2- 172.21.15.163:0/2377315059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d98b355d00 0x55d98b307160 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6f9001c000 tx=0x7f6f90020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb3e85045e5dd15 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:34.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.420+0000 7f6f76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-10-09T06:31:34.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.420+0000 7f6f76ffd700 1 -- 172.21.15.163:0/2377315059 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9001a840 con 0x55d98b355d00 2023-10-09T06:31:34.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.420+0000 7f6f76ffd700 1 -- 172.21.15.163:0/2377315059 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6f9001a9a0 con 0x55d98b355d00 2023-10-09T06:31:34.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.420+0000 7f6fa80c3b80 1 -- 172.21.15.163:0/2377315059 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d98b300b70 con 0x55d98b355d00 2023-10-09T06:31:34.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.420+0000 7f6f76ffd700 1 -- 172.21.15.163:0/2377315059 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f9001ad70 con 0x55d98b355d00 2023-10-09T06:31:34.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.420+0000 7f6fa80c3b80 10 client.5127 fetch_fsmap learned FSMap version 17 2023-10-09T06:31:34.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.420+0000 7f6fa80c3b80 1 -- 172.21.15.163:0/2377315059 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55d98b098fe0 con 0x55d98b355d00 2023-10-09T06:31:34.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.420+0000 7f6f76ffd700 1 -- 172.21.15.163:0/2377315059 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d98b300b70 con 0x55d98b355d00 2023-10-09T06:31:34.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.421+0000 7f6f76ffd700 1 -- 172.21.15.163:0/2377315059 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f6f9002f440 con 0x55d98b355d00 2023-10-09T06:31:34.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.421+0000 7f6fa80c3b80 10 client.5127 fetch_fsmap finished waiting for FSMap version 17 2023-10-09T06:31:34.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.421+0000 7f6fa80c3b80 10 client.5127 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-09T06:31:34.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.421+0000 7f6fa80c3b80 20 client.5127 populate_metadata read hostname 'smithi163' 2023-10-09T06:31:34.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.421+0000 7f6fa80c3b80 1 --2- 172.21.15.163:0/2377315059 >> [v2:172.21.15.181:6835/2708262268,v1:172.21.15.181:6837/2708262268] conn(0x55d98b34b9d0 0x55d98b34bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:34.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.421+0000 7f6fa80c3b80 4 client.5127 mds_command: new command op to 4944 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-09T06:31:34.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.421+0000 7f6fa80c3b80 1 -- 172.21.15.163:0/2377315059 --> [v2:172.21.15.181:6835/2708262268,v1:172.21.15.181:6837/2708262268] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d98b2ddaa0 con 0x55d98b34b9d0 2023-10-09T06:31:34.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.421+0000 7f6f9cb67700 1 --2- 172.21.15.163:0/2377315059 >> [v2:172.21.15.181:6835/2708262268,v1:172.21.15.181:6837/2708262268] conn(0x55d98b34b9d0 0x55d98b34bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:34.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.421+0000 7f6f76ffd700 10 client.5127 ms_handle_connect on v2:172.21.15.181:6835/2708262268 2023-10-09T06:31:34.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.421+0000 7f6f9cb67700 1 --2- 172.21.15.163:0/2377315059 >> [v2:172.21.15.181:6835/2708262268,v1:172.21.15.181:6837/2708262268] conn(0x55d98b34b9d0 0x55d98b34bda0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bbebbeb1f4d3948 server_cookie=74f322ad39088edc in_seq=0 out_seq=0 2023-10-09T06:31:34.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.422+0000 7f6f76ffd700 1 -- 172.21.15.163:0/2377315059 <== mds.0 v2:172.21.15.181:6835/2708262268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55d98b2ddaa0 con 0x55d98b34b9d0 2023-10-09T06:31:34.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.422+0000 7f6f76ffd700 10 client.5127 handle_command_reply: tid=0 2023-10-09T06:31:34.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.422+0000 7f6fa80c3b80 1 client.5127 shutdown 2023-10-09T06:31:34.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.422+0000 7f6fa80c3b80 1 -- 172.21.15.163:0/2377315059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d98b355d00 msgr2=0x55d98b307160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:34.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.422+0000 7f6fa80c3b80 1 --2- 172.21.15.163:0/2377315059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d98b355d00 0x55d98b307160 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6f9001c000 tx=0x7f6f90020000 comp rx=0 tx=0).stop 2023-10-09T06:31:34.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.422+0000 7f6fa80c3b80 1 -- 172.21.15.163:0/2377315059 shutdown_connections 2023-10-09T06:31:34.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.422+0000 7f6fa80c3b80 1 --2- 172.21.15.163:0/2377315059 >> [v2:172.21.15.181:6835/2708262268,v1:172.21.15.181:6837/2708262268] conn(0x55d98b34b9d0 0x55d98b34bda0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.422+0000 7f6fa80c3b80 1 --2- 172.21.15.163:0/2377315059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d98b355d00 0x55d98b307160 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.422+0000 7f6fa80c3b80 1 --2- 172.21.15.163:0/2377315059 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d98b30ba50 0x55d98b362a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.422+0000 7f6f76ffd700 0 client.5127 ms_handle_reset on v2:172.21.15.181:6835/2708262268 2023-10-09T06:31:34.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.422+0000 7f6fa80c3b80 1 --2- 172.21.15.163:0/2377315059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98b334f60 0x55d98b306c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.422+0000 7f6fa80c3b80 1 -- 172.21.15.163:0/2377315059 >> 172.21.15.163:0/2377315059 conn(0x55d98b30d370 msgr2=0x55d98b351550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:34.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.422+0000 7f6fa80c3b80 1 -- 172.21.15.163:0/2377315059 shutdown_connections 2023-10-09T06:31:34.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.422+0000 7f6fa80c3b80 1 -- 172.21.15.163:0/2377315059 wait complete. 2023-10-09T06:31:34.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.422+0000 7f6fa80c3b80 20 client.5127 trim_cache size 0 max 16384 2023-10-09T06:31:34.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.439+0000 7f6fa80c3b80 1 Processor -- start 2023-10-09T06:31:34.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.439+0000 7f6fa80c3b80 1 -- start start 2023-10-09T06:31:34.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.440+0000 7f6fa80c3b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d98b30ba50 0x55d98b362940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:34.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.440+0000 7f6fa80c3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98b334f60 0x55d98b361370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:34.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.440+0000 7f6fa80c3b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d98b355d00 0x55d98b3618b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:34.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.440+0000 7f6fa80c3b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98b3079b0 con 0x55d98b30ba50 2023-10-09T06:31:34.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.440+0000 7f6fa80c3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98b362e80 con 0x55d98b334f60 2023-10-09T06:31:34.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.440+0000 7f6fa80c3b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d98b34f890 con 0x55d98b355d00 2023-10-09T06:31:34.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.440+0000 7f6f9cb67700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d98b30ba50 0x55d98b362940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:34.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.440+0000 7f6f9cb67700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d98b30ba50 0x55d98b362940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33526/0 (socket says 172.21.15.163:33526) 2023-10-09T06:31:34.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.440+0000 7f6f9cb67700 1 -- 172.21.15.163:0/3347381991 learned_addr learned my addr 172.21.15.163:0/3347381991 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:34.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.440+0000 7f6f9cb67700 1 -- 172.21.15.163:0/3347381991 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d98b355d00 msgr2=0x55d98b3618b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:34.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.440+0000 7f6f9cb67700 1 --2- 172.21.15.163:0/3347381991 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d98b355d00 0x55d98b3618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.440+0000 7f6f8ffff700 1 --2- 172.21.15.163:0/3347381991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98b334f60 0x55d98b361370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:34.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.440+0000 7f6f9cb67700 1 -- 172.21.15.163:0/3347381991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98b334f60 msgr2=0x55d98b361370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:34.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.440+0000 7f6f9cb67700 1 --2- 172.21.15.163:0/3347381991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98b334f60 0x55d98b361370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.440+0000 7f6f9cb67700 1 -- 172.21.15.163:0/3347381991 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d98b22a780 con 0x55d98b30ba50 2023-10-09T06:31:34.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.440+0000 7f6f9cb67700 1 --2- 172.21.15.163:0/3347381991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d98b30ba50 0x55d98b362940 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6f94018a70 tx=0x7f6f9402c250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb2b362b176a67b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:34.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.440+0000 7f6f7e7fc700 1 -- 172.21.15.163:0/3347381991 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9402b070 con 0x55d98b30ba50 2023-10-09T06:31:34.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.440+0000 7f6f7e7fc700 1 -- 172.21.15.163:0/3347381991 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9400f050 con 0x55d98b30ba50 2023-10-09T06:31:34.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.440+0000 7f6f7e7fc700 1 -- 172.21.15.163:0/3347381991 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9402f660 con 0x55d98b30ba50 2023-10-09T06:31:34.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.441+0000 7f6fa80c3b80 1 -- 172.21.15.163:0/3347381991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d98b30ba50 msgr2=0x55d98b362940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:34.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.441+0000 7f6fa80c3b80 1 --2- 172.21.15.163:0/3347381991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d98b30ba50 0x55d98b362940 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6f94018a70 tx=0x7f6f9402c250 comp rx=0 tx=0).stop 2023-10-09T06:31:34.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.441+0000 7f6fa80c3b80 1 -- 172.21.15.163:0/3347381991 shutdown_connections 2023-10-09T06:31:34.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.441+0000 7f6fa80c3b80 1 --2- 172.21.15.163:0/3347381991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d98b30ba50 0x55d98b362940 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.441+0000 7f6fa80c3b80 1 --2- 172.21.15.163:0/3347381991 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d98b355d00 0x55d98b3618b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.441+0000 7f6fa80c3b80 1 --2- 172.21.15.163:0/3347381991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98b334f60 0x55d98b361370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.441+0000 7f6fa80c3b80 1 -- 172.21.15.163:0/3347381991 >> 172.21.15.163:0/3347381991 conn(0x55d98b30d370 msgr2=0x55d98b351550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:34.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.441+0000 7f6fa80c3b80 1 -- 172.21.15.163:0/3347381991 shutdown_connections 2023-10-09T06:31:34.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.441+0000 7f6fa80c3b80 1 -- 172.21.15.163:0/3347381991 wait complete. 2023-10-09T06:31:34.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.442+0000 7f6fa80c3b80 1 -- start start 2023-10-09T06:31:34.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.442+0000 7f6fa80c3b80 1 Processor -- start 2023-10-09T06:31:34.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.442+0000 7f6fa80c3b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98b30ba50 0x55d98b361370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:34.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.442+0000 7f6fa80c3b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d98b334f60 0x55d98b3618b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:34.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.443+0000 7f6fa80c3b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d98b355d00 0x55d98b356170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:34.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.443+0000 7f6fa80c3b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98b34cdd0 con 0x55d98b355d00 2023-10-09T06:31:34.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.443+0000 7f6fa80c3b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98b3b8d90 con 0x55d98b30ba50 2023-10-09T06:31:34.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.443+0000 7f6fa80c3b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d98b3b8a70 con 0x55d98b334f60 2023-10-09T06:31:34.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.443+0000 7f6fa80c3b80 10 client.0 _ll_register_callbacks cb 0x55d98b2cf110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-09T06:31:34.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.443+0000 7f6f9d368700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d98b355d00 0x55d98b356170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:34.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.443+0000 7f6f9d368700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d98b355d00 0x55d98b356170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33540/0 (socket says 172.21.15.163:33540) 2023-10-09T06:31:34.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.443+0000 7f6f9d368700 1 -- 172.21.15.163:0/1387658402 learned_addr learned my addr 172.21.15.163:0/1387658402 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:34.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.443+0000 7f6f9cb67700 1 --2- 172.21.15.163:0/1387658402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98b30ba50 0x55d98b361370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:34.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.443+0000 7f6f8ffff700 1 --2- 172.21.15.163:0/1387658402 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d98b334f60 0x55d98b3618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:34.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.443+0000 7f6f9d368700 1 -- 172.21.15.163:0/1387658402 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d98b334f60 msgr2=0x55d98b3618b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:34.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.443+0000 7f6f9d368700 1 --2- 172.21.15.163:0/1387658402 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d98b334f60 0x55d98b3618b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.443+0000 7f6f9d368700 1 -- 172.21.15.163:0/1387658402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98b30ba50 msgr2=0x55d98b361370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:34.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.443+0000 7f6f9d368700 1 --2- 172.21.15.163:0/1387658402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98b30ba50 0x55d98b361370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.443+0000 7f6f9d368700 1 -- 172.21.15.163:0/1387658402 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f94018820 con 0x55d98b355d00 2023-10-09T06:31:34.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.443+0000 7f6f9d368700 1 --2- 172.21.15.163:0/1387658402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d98b355d00 0x55d98b356170 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6f90009220 tx=0x7f6f9002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82321032c0c4d730 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:34.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.443+0000 7f6f76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-10-09T06:31:34.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.443+0000 7f6f76ffd700 1 -- 172.21.15.163:0/1387658402 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f90020040 con 0x55d98b355d00 2023-10-09T06:31:34.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.443+0000 7f6f76ffd700 1 -- 172.21.15.163:0/1387658402 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6f9002f900 con 0x55d98b355d00 2023-10-09T06:31:34.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.443+0000 7f6fa80c3b80 1 -- 172.21.15.163:0/1387658402 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f9000beb0 con 0x55d98b355d00 2023-10-09T06:31:34.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.443+0000 7f6f76ffd700 1 -- 172.21.15.163:0/1387658402 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f90000dd0 con 0x55d98b355d00 2023-10-09T06:31:34.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.443+0000 7f6fa80c3b80 10 client.5133 fetch_fsmap learned FSMap version 17 2023-10-09T06:31:34.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.443+0000 7f6fa80c3b80 1 -- 172.21.15.163:0/1387658402 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55d98b211470 con 0x55d98b355d00 2023-10-09T06:31:34.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.443+0000 7f6f76ffd700 1 -- 172.21.15.163:0/1387658402 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9000beb0 con 0x55d98b355d00 2023-10-09T06:31:34.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.444+0000 7f6f76ffd700 1 -- 172.21.15.163:0/1387658402 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f6f9002fb40 con 0x55d98b355d00 2023-10-09T06:31:34.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.444+0000 7f6fa80c3b80 10 client.5133 fetch_fsmap finished waiting for FSMap version 17 2023-10-09T06:31:34.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.444+0000 7f6fa80c3b80 10 client.5133 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-09T06:31:34.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.444+0000 7f6fa80c3b80 20 client.5133 populate_metadata read hostname 'smithi163' 2023-10-09T06:31:34.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.444+0000 7f6fa80c3b80 1 --2- 172.21.15.163:0/1387658402 >> [v2:172.21.15.181:6835/2708262268,v1:172.21.15.181:6837/2708262268] conn(0x55d98b3566b0 0x55d98b356a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:34.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.444+0000 7f6fa80c3b80 4 client.5133 mds_command: new command op to 4944 tid=0[{"prefix": "status"}] 2023-10-09T06:31:34.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.444+0000 7f6fa80c3b80 1 -- 172.21.15.163:0/1387658402 --> [v2:172.21.15.181:6835/2708262268,v1:172.21.15.181:6837/2708262268] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d98b1fee00 con 0x55d98b3566b0 2023-10-09T06:31:34.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.444+0000 7f6f9cb67700 1 --2- 172.21.15.163:0/1387658402 >> [v2:172.21.15.181:6835/2708262268,v1:172.21.15.181:6837/2708262268] conn(0x55d98b3566b0 0x55d98b356a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:34.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.444+0000 7f6f76ffd700 10 client.5133 ms_handle_connect on v2:172.21.15.181:6835/2708262268 2023-10-09T06:31:34.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.444+0000 7f6f9cb67700 1 --2- 172.21.15.163:0/1387658402 >> [v2:172.21.15.181:6835/2708262268,v1:172.21.15.181:6837/2708262268] conn(0x55d98b3566b0 0x55d98b356a80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28e92d4b2597333 server_cookie=d3420484d968c6a2 in_seq=0 out_seq=0 2023-10-09T06:31:34.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.444+0000 7f6f76ffd700 1 -- 172.21.15.163:0/1387658402 <== mds.0 v2:172.21.15.181:6835/2708262268 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55d98b1fee00 con 0x55d98b3566b0 2023-10-09T06:31:34.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.444+0000 7f6f76ffd700 10 client.5133 handle_command_reply: tid=0 2023-10-09T06:31:34.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.444+0000 7f6fa80c3b80 1 client.5133 shutdown 2023-10-09T06:31:34.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.444+0000 7f6fa80c3b80 1 -- 172.21.15.163:0/1387658402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d98b355d00 msgr2=0x55d98b356170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:34.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.444+0000 7f6fa80c3b80 1 --2- 172.21.15.163:0/1387658402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d98b355d00 0x55d98b356170 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6f90009220 tx=0x7f6f9002c000 comp rx=0 tx=0).stop 2023-10-09T06:31:34.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.444+0000 7f6fa80c3b80 1 -- 172.21.15.163:0/1387658402 shutdown_connections 2023-10-09T06:31:34.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.444+0000 7f6fa80c3b80 1 --2- 172.21.15.163:0/1387658402 >> [v2:172.21.15.181:6835/2708262268,v1:172.21.15.181:6837/2708262268] conn(0x55d98b3566b0 0x55d98b356a80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.445+0000 7f6fa80c3b80 1 --2- 172.21.15.163:0/1387658402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d98b355d00 0x55d98b356170 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.445+0000 7f6f76ffd700 0 client.5133 ms_handle_reset on v2:172.21.15.181:6835/2708262268 2023-10-09T06:31:34.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.445+0000 7f6fa80c3b80 1 --2- 172.21.15.163:0/1387658402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d98b30ba50 0x55d98b361370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.445+0000 7f6fa80c3b80 1 --2- 172.21.15.163:0/1387658402 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d98b334f60 0x55d98b3618b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.445+0000 7f6fa80c3b80 1 -- 172.21.15.163:0/1387658402 >> 172.21.15.163:0/1387658402 conn(0x55d98b30d370 msgr2=0x55d98b308bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:34.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.445+0000 7f6fa80c3b80 1 -- 172.21.15.163:0/1387658402 shutdown_connections 2023-10-09T06:31:34.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.445+0000 7f6fa80c3b80 1 -- 172.21.15.163:0/1387658402 wait complete. 2023-10-09T06:31:34.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.445+0000 7f6fa80c3b80 20 client.5133 trim_cache size 0 max 16384 2023-10-09T06:31:34.453 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:31:34.453 INFO:teuthology.orchestra.run.smithi163.stdout: "cluster_fsid": "2520b8f7-539f-4bae-9f17-1fce66f66ddd", 2023-10-09T06:31:34.453 INFO:teuthology.orchestra.run.smithi163.stdout: "whoami": 0, 2023-10-09T06:31:34.453 INFO:teuthology.orchestra.run.smithi163.stdout: "id": 4944, 2023-10-09T06:31:34.453 INFO:teuthology.orchestra.run.smithi163.stdout: "want_state": "up:active", 2023-10-09T06:31:34.453 INFO:teuthology.orchestra.run.smithi163.stdout: "state": "up:active", 2023-10-09T06:31:34.453 INFO:teuthology.orchestra.run.smithi163.stdout: "fs_name": "cephfs", 2023-10-09T06:31:34.453 INFO:teuthology.orchestra.run.smithi163.stdout: "rank_uptime": 5.8053870209999996, 2023-10-09T06:31:34.453 INFO:teuthology.orchestra.run.smithi163.stdout: "mdsmap_epoch": 17, 2023-10-09T06:31:34.454 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch": 31, 2023-10-09T06:31:34.454 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch_barrier": 31, 2023-10-09T06:31:34.454 INFO:teuthology.orchestra.run.smithi163.stdout: "uptime": 13.697683083999999 2023-10-09T06:31:34.454 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:31:34.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.445+0000 7f6f9edcb700 1 -- 172.21.15.163:0/1578309810 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6f84066f30 msgr2=0x7f6f840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:34.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.445+0000 7f6f9edcb700 1 --2- 172.21.15.163:0/1578309810 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6f84066f30 0x7f6f840693e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6f94007620 tx=0x7f6f9400b040 comp rx=0 tx=0).stop 2023-10-09T06:31:34.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.445+0000 7f6f9edcb700 1 -- 172.21.15.163:0/1578309810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f9806c0a0 msgr2=0x7f6f980b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:34.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.445+0000 7f6f9edcb700 1 --2- 172.21.15.163:0/1578309810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f9806c0a0 0x7f6f980b2270 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800e930 tx=0x7f6f880091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.445+0000 7f6f9edcb700 1 -- 172.21.15.163:0/1578309810 shutdown_connections 2023-10-09T06:31:34.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.445+0000 7f6f9edcb700 1 --2- 172.21.15.163:0/1578309810 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6f84066f30 0x7f6f840693e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.445+0000 7f6f9edcb700 1 --2- 172.21.15.163:0/1578309810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f9815df50 0x7f6f980b3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.445+0000 7f6f9edcb700 1 --2- 172.21.15.163:0/1578309810 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f9806bcd0 0x7f6f98154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.445+0000 7f6f9edcb700 1 --2- 172.21.15.163:0/1578309810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f9806c0a0 0x7f6f980b2270 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.445+0000 7f6f9edcb700 1 -- 172.21.15.163:0/1578309810 >> 172.21.15.163:0/1578309810 conn(0x7f6f980ce600 msgr2=0x7f6f9814f420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:34.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.445+0000 7f6f9edcb700 1 -- 172.21.15.163:0/1578309810 shutdown_connections 2023-10-09T06:31:34.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.445+0000 7f6f9edcb700 1 -- 172.21.15.163:0/1578309810 wait complete. 2023-10-09T06:31:34.458 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-09T06:31:34.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.572+0000 7fd8532a1700 1 Processor -- start 2023-10-09T06:31:34.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.572+0000 7fd8532a1700 1 -- start start 2023-10-09T06:31:34.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.573+0000 7fd8532a1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd84c159c60 0x7fd84c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:34.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.573+0000 7fd8532a1700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd84c15a160 con 0x7fd84c15db10 2023-10-09T06:31:34.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.573+0000 7fd8532a1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd84c0d13a0 con 0x7fd84c159890 2023-10-09T06:31:34.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.573+0000 7fd8532a1700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd84c15df50 con 0x7fd84c159c60 2023-10-09T06:31:34.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.573+0000 7fd85083c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd84c159c60 0x7fd84c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:34.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.573+0000 7fd85183e700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fd84c15db10 0x7fd84c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33904/0 (socket says 172.21.15.163:33904) 2023-10-09T06:31:34.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.573+0000 7fd85083c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd84c159c60 0x7fd84c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33556/0 (socket says 172.21.15.163:33556) 2023-10-09T06:31:34.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.573+0000 7fd85183e700 1 -- 172.21.15.163:0/3058299608 learned_addr learned my addr 172.21.15.163:0/3058299608 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:34.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.574+0000 7fd85083c700 1 -- 172.21.15.163:0/3058299608 >> v1:172.21.15.181:6789/0 conn(0x7fd84c159890 legacy=0x7fd84c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:34.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.574+0000 7fd85083c700 1 -- 172.21.15.163:0/3058299608 >> v1:172.21.15.163:6789/0 conn(0x7fd84c15db10 legacy=0x7fd84c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:34.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.574+0000 7fd85083c700 1 -- 172.21.15.163:0/3058299608 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84c15e140 con 0x7fd84c159c60 2023-10-09T06:31:34.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.574+0000 7fd84bfff700 1 -- 172.21.15.163:0/3058299608 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1098954184 0 0) 0x7fd84c15a160 con 0x7fd84c15db10 2023-10-09T06:31:34.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.574+0000 7fd85083c700 1 --2- 172.21.15.163:0/3058299608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd84c159c60 0x7fd84c15d5d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd8400098e0 tx=0x7fd8400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e04fde51b95caa server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:34.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.574+0000 7fd84bfff700 1 -- 172.21.15.163:0/3058299608 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd840009e80 con 0x7fd84c159c60 2023-10-09T06:31:34.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.574+0000 7fd84bfff700 1 -- 172.21.15.163:0/3058299608 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84000b860 con 0x7fd84c159c60 2023-10-09T06:31:34.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.574+0000 7fd84bfff700 1 -- 172.21.15.163:0/3058299608 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84000bbe0 con 0x7fd84c159c60 2023-10-09T06:31:34.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.574+0000 7fd8532a1700 1 -- 172.21.15.163:0/3058299608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd84c159c60 msgr2=0x7fd84c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:34.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.574+0000 7fd8532a1700 1 --2- 172.21.15.163:0/3058299608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd84c159c60 0x7fd84c15d5d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd8400098e0 tx=0x7fd8400091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.574+0000 7fd8532a1700 1 -- 172.21.15.163:0/3058299608 shutdown_connections 2023-10-09T06:31:34.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.574+0000 7fd8532a1700 1 --2- 172.21.15.163:0/3058299608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd84c159c60 0x7fd84c15d5d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.574+0000 7fd8532a1700 1 -- 172.21.15.163:0/3058299608 >> 172.21.15.163:0/3058299608 conn(0x7fd84c068e20 msgr2=0x7fd84c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:34.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.575+0000 7fd8532a1700 1 -- 172.21.15.163:0/3058299608 shutdown_connections 2023-10-09T06:31:34.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.575+0000 7fd8532a1700 1 -- 172.21.15.163:0/3058299608 wait complete. 2023-10-09T06:31:34.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.576+0000 7fd8532a1700 1 Processor -- start 2023-10-09T06:31:34.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.576+0000 7fd8532a1700 1 -- start start 2023-10-09T06:31:34.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.576+0000 7fd8532a1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd84c159890 0x7fd84c16ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:34.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.576+0000 7fd8532a1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd84c159c60 0x7fd84c16b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:34.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.576+0000 7fd8532a1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd84c15db10 0x7fd84c169ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:34.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.576+0000 7fd8532a1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c16aea0 con 0x7fd84c15db10 2023-10-09T06:31:34.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.576+0000 7fd8532a1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c16a8e0 con 0x7fd84c159c60 2023-10-09T06:31:34.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.576+0000 7fd8532a1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c16abc0 con 0x7fd84c159890 2023-10-09T06:31:34.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.577+0000 7fd85183e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd84c15db10 0x7fd84c169ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:34.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.577+0000 7fd85183e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd84c15db10 0x7fd84c169ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33564/0 (socket says 172.21.15.163:33564) 2023-10-09T06:31:34.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.577+0000 7fd85183e700 1 -- 172.21.15.163:0/4165416508 learned_addr learned my addr 172.21.15.163:0/4165416508 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:34.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.577+0000 7fd85083c700 1 --2- 172.21.15.163:0/4165416508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd84c159c60 0x7fd84c16b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:34.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.577+0000 7fd85103d700 1 --2- 172.21.15.163:0/4165416508 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd84c159890 0x7fd84c16ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:34.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.577+0000 7fd85183e700 1 -- 172.21.15.163:0/4165416508 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd84c159890 msgr2=0x7fd84c16ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:34.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.577+0000 7fd85183e700 1 --2- 172.21.15.163:0/4165416508 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd84c159890 0x7fd84c16ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.577+0000 7fd85183e700 1 -- 172.21.15.163:0/4165416508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd84c159c60 msgr2=0x7fd84c16b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:34.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.577+0000 7fd85183e700 1 --2- 172.21.15.163:0/4165416508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd84c159c60 0x7fd84c16b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.577+0000 7fd85183e700 1 -- 172.21.15.163:0/4165416508 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd840009420 con 0x7fd84c15db10 2023-10-09T06:31:34.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.577+0000 7fd85083c700 1 --2- 172.21.15.163:0/4165416508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd84c159c60 0x7fd84c16b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:34.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.577+0000 7fd85183e700 1 --2- 172.21.15.163:0/4165416508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd84c15db10 0x7fd84c169ed0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd84400c530 tx=0x7fd844007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:34.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.577+0000 7fd8497fa700 1 -- 172.21.15.163:0/4165416508 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd844005b30 con 0x7fd84c15db10 2023-10-09T06:31:34.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.577+0000 7fd8497fa700 1 -- 172.21.15.163:0/4165416508 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd844005c90 con 0x7fd84c15db10 2023-10-09T06:31:34.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.577+0000 7fd8532a1700 1 -- 172.21.15.163:0/4165416508 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd84c0b36c0 con 0x7fd84c15db10 2023-10-09T06:31:34.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.577+0000 7fd8497fa700 1 -- 172.21.15.163:0/4165416508 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd844007320 con 0x7fd84c15db10 2023-10-09T06:31:34.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.577+0000 7fd8532a1700 1 -- 172.21.15.163:0/4165416508 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd84c0b5330 con 0x7fd84c15db10 2023-10-09T06:31:34.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.578+0000 7fd8532a1700 1 -- 172.21.15.163:0/4165416508 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd84c0b54f0 con 0x7fd84c15db10 2023-10-09T06:31:34.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.579+0000 7fd8497fa700 1 -- 172.21.15.163:0/4165416508 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd84401b3b0 con 0x7fd84c15db10 2023-10-09T06:31:34.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.579+0000 7fd8497fa700 1 --2- 172.21.15.163:0/4165416508 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd838066f30 0x7fd8380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:34.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.579+0000 7fd8497fa700 1 -- 172.21.15.163:0/4165416508 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd844083a10 con 0x7fd84c15db10 2023-10-09T06:31:34.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.579+0000 7fd85103d700 1 --2- 172.21.15.163:0/4165416508 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd838066f30 0x7fd8380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:34.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.580+0000 7fd85103d700 1 --2- 172.21.15.163:0/4165416508 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd838066f30 0x7fd8380693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd83c007fd0 tx=0x7fd83c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:34.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.583+0000 7fd8497fa700 1 -- 172.21.15.163:0/4165416508 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd844050c80 con 0x7fd84c15db10 2023-10-09T06:31:34.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.812+0000 7fd8532a1700 1 -- 172.21.15.163:0/4165416508 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd84c04c3c0 con 0x7fd84c15db10 2023-10-09T06:31:34.814 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.813+0000 7fd8497fa700 1 -- 172.21.15.163:0/4165416508 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd844017070 con 0x7fd84c15db10 2023-10-09T06:31:34.814 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-09T06:31:34.816 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.815+0000 7fd8532a1700 1 -- 172.21.15.163:0/4165416508 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd838066f30 msgr2=0x7fd8380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:34.816 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.815+0000 7fd8532a1700 1 --2- 172.21.15.163:0/4165416508 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd838066f30 0x7fd8380693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd83c007fd0 tx=0x7fd83c009040 comp rx=0 tx=0).stop 2023-10-09T06:31:34.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.815+0000 7fd8532a1700 1 -- 172.21.15.163:0/4165416508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd84c15db10 msgr2=0x7fd84c169ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:34.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.815+0000 7fd8532a1700 1 --2- 172.21.15.163:0/4165416508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd84c15db10 0x7fd84c169ed0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd84400c530 tx=0x7fd844007f90 comp rx=0 tx=0).stop 2023-10-09T06:31:34.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.815+0000 7fd8532a1700 1 -- 172.21.15.163:0/4165416508 shutdown_connections 2023-10-09T06:31:34.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.815+0000 7fd8532a1700 1 --2- 172.21.15.163:0/4165416508 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd838066f30 0x7fd8380693e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.815+0000 7fd8532a1700 1 --2- 172.21.15.163:0/4165416508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd84c15db10 0x7fd84c169ed0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.815+0000 7fd8532a1700 1 --2- 172.21.15.163:0/4165416508 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd84c159890 0x7fd84c16ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.815+0000 7fd8532a1700 1 --2- 172.21.15.163:0/4165416508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd84c159c60 0x7fd84c16b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:34.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.815+0000 7fd8532a1700 1 -- 172.21.15.163:0/4165416508 >> 172.21.15.163:0/4165416508 conn(0x7fd84c068e20 msgr2=0x7fd84c0c70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:34.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.815+0000 7fd8532a1700 1 -- 172.21.15.163:0/4165416508 shutdown_connections 2023-10-09T06:31:34.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:34.815+0000 7fd8532a1700 1 -- 172.21.15.163:0/4165416508 wait complete. 2023-10-09T06:31:34.827 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-09T06:31:34.827 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-09T06:31:34.828 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi163.front.sepia.ceph.com 2023-10-09T06:31:34.828 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-09T06:31:34.828 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-09T06:31:34.828 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-09T06:31:34.828 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-09T06:31:34.828 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-09T06:31:34.828 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:31:34.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:34.828 DEBUG:teuthology.orchestra.run.smithi163:> ip addr 2023-10-09T06:31:34.843 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:31:34.844 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:31:34.844 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:31:34.844 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:34.844 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2023-10-09T06:31:34.844 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:34.844 INFO:teuthology.orchestra.run.smithi163.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:31:34.844 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:06 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:31:34.844 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f0 2023-10-09T06:31:34.844 INFO:teuthology.orchestra.run.smithi163.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:31:34.845 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:07 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:31:34.845 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f1 2023-10-09T06:31:34.845 INFO:teuthology.orchestra.run.smithi163.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:31:34.845 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5a brd ff:ff:ff:ff:ff:ff 2023-10-09T06:31:34.845 INFO:teuthology.orchestra.run.smithi163.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-09T06:31:34.845 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5b brd ff:ff:ff:ff:ff:ff 2023-10-09T06:31:34.845 INFO:teuthology.orchestra.run.smithi163.stdout: inet 172.21.15.163/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-09T06:31:34.845 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft 42428sec preferred_lft 42428sec 2023-10-09T06:31:34.845 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::ec4:7aff:fe88:7d5b/64 scope link 2023-10-09T06:31:34.845 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:34.846 INFO:teuthology.orchestra.run.smithi163.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-09T06:31:34.846 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 2e:01:84:b9:f4:8e brd ff:ff:ff:ff:ff:ff 2023-10-09T06:31:34.846 INFO:teuthology.orchestra.run.smithi163.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-09T06:31:34.846 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:34.846 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::d03c:93ff:fe9e:c8cb/64 scope link 2023-10-09T06:31:34.846 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:34.846 INFO:teuthology.orchestra.run.smithi163.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:31:34.846 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 2e:01:84:b9:f4:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-09T06:31:34.846 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::2c01:84ff:feb9:f48e/64 scope link 2023-10-09T06:31:34.847 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:34.847 INFO:teuthology.orchestra.run.smithi163.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:31:34.847 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether c6:88:3e:d7:c1:56 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-09T06:31:34.847 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::c488:3eff:fed7:c156/64 scope link 2023-10-09T06:31:34.847 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:34.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:34.848 DEBUG:teuthology.orchestra.run.smithi163:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-09T06:31:34.906 INFO:teuthology.orchestra.run.smithi163.stdout:1 2023-10-09T06:31:34.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:34.908 DEBUG:teuthology.orchestra.run.smithi163:> route 2023-10-09T06:31:34.965 INFO:teuthology.orchestra.run.smithi163.stdout:Kernel IP routing table 2023-10-09T06:31:34.965 INFO:teuthology.orchestra.run.smithi163.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-09T06:31:34.965 INFO:teuthology.orchestra.run.smithi163.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-09T06:31:34.966 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-09T06:31:34.966 INFO:teuthology.orchestra.run.smithi163.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-09T06:31:34.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:34.966 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-09T06:31:34.966 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-09T06:31:34.966 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-09T06:31:34.966 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-09T06:31:34.966 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-09T06:31:34.967 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-09T06:31:35.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:35.066 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2023-10-09T06:31:35.080 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-09T06:31:35.081 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-09T06:31:35.081 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:31:35.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:35.081 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set brx.0 up 2023-10-09T06:31:35.145 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:31:35.146 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-09T06:31:35.200 INFO:teuthology.orchestra.run.smithi163.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:31:35.201 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:31:35.201 DEBUG:teuthology.orchestra.run.smithi163:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-09T06:31:35.255 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-09T06:31:35.255 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-09T06:31:35.255 DEBUG:teuthology.orchestra.run.smithi163:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:31:35.334 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:31:35.334 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:31:35.353 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:31:35.353 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:31:35.353 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:31:35.353 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with new device syntax: 0@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:31:35.353 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:31:35.353 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:31:35.353 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:31:35.353 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:31:35.375 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:31:35.376 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:31:35.376 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:31:35.376 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with new device syntax: 0@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:31:35.376 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:31:35.376 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:31:35.376 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:31:35.376 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:31:35.376 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790" 2023-10-09T06:31:35.376 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790:/ 2023-10-09T06:31:35.376 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=2520b8f7-539f-4bae-9f17-1fce66f66ddd" will pass to kernel 2023-10-09T06:31:35.377 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-09T06:31:35.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:35.378 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-09T06:31:35.443 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-09T06:31:35.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.597+0000 7f55063fc700 1 Processor -- start 2023-10-09T06:31:35.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.597+0000 7f55063fc700 1 -- start start 2023-10-09T06:31:35.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.598+0000 7f55063fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5500159c60 0x7f550015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:35.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.598+0000 7f55063fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f550015db10 0x7f5500069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:35.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.598+0000 7f55063fc700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f550015a160 con 0x7f5500159890 2023-10-09T06:31:35.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.598+0000 7f55063fc700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f550015df50 con 0x7f550015db10 2023-10-09T06:31:35.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.598+0000 7f55063fc700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55000d13a0 con 0x7f5500159c60 2023-10-09T06:31:35.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.598+0000 7f5504999700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f550015db10 0x7f5500069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:35.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.598+0000 7f54fffff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f5500159890 0x7f550015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33920/0 (socket says 172.21.15.163:33920) 2023-10-09T06:31:35.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.598+0000 7f54fffff700 1 -- 172.21.15.163:0/2070944639 learned_addr learned my addr 172.21.15.163:0/2070944639 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:35.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.598+0000 7f5504999700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f550015db10 0x7f5500069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33576/0 (socket says 172.21.15.163:33576) 2023-10-09T06:31:35.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.599+0000 7f5504999700 1 -- 172.21.15.163:0/2070944639 >> v2:172.21.15.181:3300/0 conn(0x7f5500159c60 msgr2=0x7f550015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:35.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.599+0000 7f5504999700 1 --2- 172.21.15.163:0/2070944639 >> v2:172.21.15.181:3300/0 conn(0x7f5500159c60 0x7f550015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:35.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.599+0000 7f5504999700 1 -- 172.21.15.163:0/2070944639 >> v1:172.21.15.163:6789/0 conn(0x7f5500159890 legacy=0x7f550015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:35.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.599+0000 7f5504999700 1 -- 172.21.15.163:0/2070944639 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550015e140 con 0x7f550015db10 2023-10-09T06:31:35.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.599+0000 7f5504999700 1 --2- 172.21.15.163:0/2070944639 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f550015db10 0x7f5500069560 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f55000b67f0 tx=0x7f54f8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4cbd8b432d41a62 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:35.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.599+0000 7f54feffd700 1 -- 172.21.15.163:0/2070944639 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f800e070 con 0x7f550015db10 2023-10-09T06:31:35.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.599+0000 7f54feffd700 1 -- 172.21.15.163:0/2070944639 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f800bb30 con 0x7f550015db10 2023-10-09T06:31:35.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.599+0000 7f54feffd700 1 -- 172.21.15.163:0/2070944639 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f8012430 con 0x7f550015db10 2023-10-09T06:31:35.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.599+0000 7f55063fc700 1 -- 172.21.15.163:0/2070944639 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f550015db10 msgr2=0x7f5500069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:35.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.599+0000 7f55063fc700 1 --2- 172.21.15.163:0/2070944639 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f550015db10 0x7f5500069560 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f55000b67f0 tx=0x7f54f8009780 comp rx=0 tx=0).stop 2023-10-09T06:31:35.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.599+0000 7f55063fc700 1 -- 172.21.15.163:0/2070944639 shutdown_connections 2023-10-09T06:31:35.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.600+0000 7f55063fc700 1 --2- 172.21.15.163:0/2070944639 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f550015db10 0x7f5500069560 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:35.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.600+0000 7f55063fc700 1 --2- 172.21.15.163:0/2070944639 >> v2:172.21.15.181:3300/0 conn(0x7f5500159c60 0x7f550015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:35.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.600+0000 7f55063fc700 1 -- 172.21.15.163:0/2070944639 >> 172.21.15.163:0/2070944639 conn(0x7f5500068e20 msgr2=0x7f55000d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:35.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.600+0000 7f55063fc700 1 -- 172.21.15.163:0/2070944639 shutdown_connections 2023-10-09T06:31:35.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.600+0000 7f55063fc700 1 -- 172.21.15.163:0/2070944639 wait complete. 2023-10-09T06:31:35.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.601+0000 7f55063fc700 1 Processor -- start 2023-10-09T06:31:35.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.601+0000 7f55063fc700 1 -- start start 2023-10-09T06:31:35.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.601+0000 7f55063fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5500159890 0x7f55000b5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:35.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.601+0000 7f55063fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5500159c60 0x7f550016d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:35.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.602+0000 7f55063fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f550015db10 0x7f55000cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:35.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.602+0000 7f55063fc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5500153800 con 0x7f5500159890 2023-10-09T06:31:35.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.602+0000 7f55063fc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55000b6110 con 0x7f5500159c60 2023-10-09T06:31:35.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.602+0000 7f55063fc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f550016da40 con 0x7f550015db10 2023-10-09T06:31:35.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.602+0000 7f54fffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5500159890 0x7f55000b5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:35.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.602+0000 7f54fffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5500159890 0x7f55000b5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33578/0 (socket says 172.21.15.163:33578) 2023-10-09T06:31:35.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.602+0000 7f54fffff700 1 -- 172.21.15.163:0/1906993896 learned_addr learned my addr 172.21.15.163:0/1906993896 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:35.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.602+0000 7f54ff7fe700 1 --2- 172.21.15.163:0/1906993896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5500159c60 0x7f550016d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:35.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.602+0000 7f54fffff700 1 -- 172.21.15.163:0/1906993896 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f550015db10 msgr2=0x7f55000cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:35.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.602+0000 7f5504999700 1 --2- 172.21.15.163:0/1906993896 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f550015db10 0x7f55000cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:35.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.602+0000 7f54fffff700 1 --2- 172.21.15.163:0/1906993896 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f550015db10 0x7f55000cd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:35.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.602+0000 7f54fffff700 1 -- 172.21.15.163:0/1906993896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5500159c60 msgr2=0x7f550016d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:35.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.602+0000 7f54fffff700 1 --2- 172.21.15.163:0/1906993896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5500159c60 0x7f550016d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:35.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.602+0000 7f54fffff700 1 -- 172.21.15.163:0/1906993896 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f8009a20 con 0x7f5500159890 2023-10-09T06:31:35.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.602+0000 7f54ff7fe700 1 --2- 172.21.15.163:0/1906993896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5500159c60 0x7f550016d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:35.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.602+0000 7f54fffff700 1 --2- 172.21.15.163:0/1906993896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5500159890 0x7f55000b5bd0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f54f000c810 tx=0x7f54f0007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:35.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.602+0000 7f54e7fff700 1 -- 172.21.15.163:0/1906993896 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f0005b30 con 0x7f5500159890 2023-10-09T06:31:35.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.602+0000 7f54e7fff700 1 -- 172.21.15.163:0/1906993896 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f0005c90 con 0x7f5500159890 2023-10-09T06:31:35.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.602+0000 7f55063fc700 1 -- 172.21.15.163:0/1906993896 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f550016dd20 con 0x7f5500159890 2023-10-09T06:31:35.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.602+0000 7f54e7fff700 1 -- 172.21.15.163:0/1906993896 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f0007340 con 0x7f5500159890 2023-10-09T06:31:35.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.602+0000 7f55063fc700 1 -- 172.21.15.163:0/1906993896 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55000b09c0 con 0x7f5500159890 2023-10-09T06:31:35.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.603+0000 7f55063fc700 1 -- 172.21.15.163:0/1906993896 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55000b0540 con 0x7f5500159890 2023-10-09T06:31:35.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.604+0000 7f54e7fff700 1 -- 172.21.15.163:0/1906993896 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f54f001b400 con 0x7f5500159890 2023-10-09T06:31:35.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.604+0000 7f54e7fff700 1 --2- 172.21.15.163:0/1906993896 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f54ec066f30 0x7f54ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:35.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.604+0000 7f54e7fff700 1 -- 172.21.15.163:0/1906993896 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f54f0084150 con 0x7f5500159890 2023-10-09T06:31:35.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.604+0000 7f54ff7fe700 1 --2- 172.21.15.163:0/1906993896 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f54ec066f30 0x7f54ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:35.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.605+0000 7f54ff7fe700 1 --2- 172.21.15.163:0/1906993896 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f54ec066f30 0x7f54ec0693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f54f4006fd0 tx=0x7f54f4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:35.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.608+0000 7f54e7fff700 1 -- 172.21.15.163:0/1906993896 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55000b0540 con 0x7f5500159890 2023-10-09T06:31:35.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.837+0000 7f55063fc700 1 -- 172.21.15.163:0/1906993896 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f55000b0540 con 0x7f5500159890 2023-10-09T06:31:35.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.837+0000 7f54e7fff700 1 -- 172.21.15.163:0/1906993896 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f55000b0540 con 0x7f5500159890 2023-10-09T06:31:35.838 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-09T06:31:35.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.840+0000 7f55063fc700 1 -- 172.21.15.163:0/1906993896 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f54ec066f30 msgr2=0x7f54ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:35.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.840+0000 7f55063fc700 1 --2- 172.21.15.163:0/1906993896 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f54ec066f30 0x7f54ec0693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f54f4006fd0 tx=0x7f54f4008040 comp rx=0 tx=0).stop 2023-10-09T06:31:35.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.840+0000 7f55063fc700 1 -- 172.21.15.163:0/1906993896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5500159890 msgr2=0x7f55000b5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:35.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.840+0000 7f55063fc700 1 --2- 172.21.15.163:0/1906993896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5500159890 0x7f55000b5bd0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f54f000c810 tx=0x7f54f0007f30 comp rx=0 tx=0).stop 2023-10-09T06:31:35.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.840+0000 7f55063fc700 1 -- 172.21.15.163:0/1906993896 shutdown_connections 2023-10-09T06:31:35.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.840+0000 7f55063fc700 1 --2- 172.21.15.163:0/1906993896 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f54ec066f30 0x7f54ec0693e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:35.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.840+0000 7f55063fc700 1 --2- 172.21.15.163:0/1906993896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5500159890 0x7f55000b5bd0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:35.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.840+0000 7f55063fc700 1 --2- 172.21.15.163:0/1906993896 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f550015db10 0x7f55000cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:35.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.840+0000 7f55063fc700 1 --2- 172.21.15.163:0/1906993896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5500159c60 0x7f550016d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:35.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.840+0000 7f55063fc700 1 -- 172.21.15.163:0/1906993896 >> 172.21.15.163:0/1906993896 conn(0x7f5500068e20 msgr2=0x7f5500152b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:35.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.840+0000 7f55063fc700 1 -- 172.21.15.163:0/1906993896 shutdown_connections 2023-10-09T06:31:35.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:35.840+0000 7f55063fc700 1 -- 172.21.15.163:0/1906993896 wait complete. 2023-10-09T06:31:35.852 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-09T06:31:35.852 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-09T06:31:35.853 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-09T06:31:35.853 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-09T06:31:35.853 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-09T06:31:35.853 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-09T06:31:35.853 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-09T06:31:35.853 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-09T06:31:35.853 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:31:35.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:35.853 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-09T06:31:35.872 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:31:35.872 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:31:35.872 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:31:35.872 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:35.872 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-09T06:31:35.872 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:35.872 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:31:35.872 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:31:35.873 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2023-10-09T06:31:35.873 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:31:35.873 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:31:35.873 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2023-10-09T06:31:35.873 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:31:35.873 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-09T06:31:35.873 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-09T06:31:35.873 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-09T06:31:35.873 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-09T06:31:35.873 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42404sec preferred_lft 42404sec 2023-10-09T06:31:35.873 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2023-10-09T06:31:35.874 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:35.874 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-09T06:31:35.874 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 72:62:e6:72:7c:2e brd ff:ff:ff:ff:ff:ff 2023-10-09T06:31:35.874 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-09T06:31:35.874 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:35.874 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::e89a:4cff:fefb:7198/64 scope link 2023-10-09T06:31:35.874 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:35.874 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:31:35.875 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 72:62:e6:72:7c:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-09T06:31:35.875 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::7062:e6ff:fe72:7c2e/64 scope link 2023-10-09T06:31:35.875 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:35.875 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:31:35.875 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether fa:4e:a3:9d:5a:5f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-09T06:31:35.876 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::f84e:a3ff:fe9d:5a5f/64 scope link 2023-10-09T06:31:35.876 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:31:35.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:35.876 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-09T06:31:35.939 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-09T06:31:35.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:35.940 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-09T06:31:36.002 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-09T06:31:36.003 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-09T06:31:36.003 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-09T06:31:36.003 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-09T06:31:36.004 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-09T06:31:36.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:36.004 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-09T06:31:36.004 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-09T06:31:36.004 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-09T06:31:36.004 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-09T06:31:36.004 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-09T06:31:36.005 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-09T06:31:36.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:36.107 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-09T06:31:36.121 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-09T06:31:36.122 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-09T06:31:36.122 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:31:36.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:36.122 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-09T06:31:36.187 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:31:36.187 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-09T06:31:36.241 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:31:36.241 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:31:36.242 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-09T06:31:36.295 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-09T06:31:36.295 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-09T06:31:36.296 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:31:36.375 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:31:36.375 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:31:36.393 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:31:36.393 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:31:36.394 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:31:36.394 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 1@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:31:36.394 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:31:36.394 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:31:36.394 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:31:36.394 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:31:36.415 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:31:36.416 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:31:36.416 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:31:36.416 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 1@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:31:36.416 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:31:36.416 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:31:36.416 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:31:36.416 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:31:36.416 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790" 2023-10-09T06:31:36.416 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790:/ 2023-10-09T06:31:36.416 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=2520b8f7-539f-4bae-9f17-1fce66f66ddd" will pass to kernel 2023-10-09T06:31:36.418 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-09T06:31:36.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:36.418 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-09T06:31:36.482 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-09T06:31:36.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.605+0000 7f24c73de700 1 Processor -- start 2023-10-09T06:31:36.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.605+0000 7f24c73de700 1 -- start start 2023-10-09T06:31:36.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.606+0000 7f24c73de700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f24c0153f50 0x7f24c00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:36.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.606+0000 7f24c73de700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24c0154320 0x7f24c006d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:36.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.606+0000 7f24c73de700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24c00b4570 con 0x7f24c015a190 2023-10-09T06:31:36.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.606+0000 7f24c73de700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24c015a730 con 0x7f24c0153f50 2023-10-09T06:31:36.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.606+0000 7f24c73de700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c015aa10 con 0x7f24c0154320 2023-10-09T06:31:36.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.607+0000 7f24c517a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f24c0153f50 0x7f24c00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:36.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.607+0000 7f24c4979700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24c0154320 0x7f24c006d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:36.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.607+0000 7f24c4979700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24c0154320 0x7f24c006d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:56368/0 (socket says 172.21.15.163:56368) 2023-10-09T06:31:36.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.607+0000 7f24c597b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f24c015a190 0x7f24c015d790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:49200/0 (socket says 172.21.15.163:49200) 2023-10-09T06:31:36.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.607+0000 7f24c517a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f24c0153f50 0x7f24c00b2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:41224/0 (socket says 172.21.15.163:41224) 2023-10-09T06:31:36.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.607+0000 7f24c4979700 1 -- 172.21.15.163:0/802489698 learned_addr learned my addr 172.21.15.163:0/802489698 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:36.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.607+0000 7f24c4979700 1 -- 172.21.15.163:0/802489698 >> v2:172.21.15.181:3300/0 conn(0x7f24c0153f50 msgr2=0x7f24c00b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:36.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.607+0000 7f24c4979700 1 --2- 172.21.15.163:0/802489698 >> v2:172.21.15.181:3300/0 conn(0x7f24c0153f50 0x7f24c00b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:36.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.607+0000 7f24c4979700 1 -- 172.21.15.163:0/802489698 >> v1:172.21.15.181:6789/0 conn(0x7f24c015a190 legacy=0x7f24c015d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:36.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.607+0000 7f24b7fff700 1 -- 172.21.15.163:0/802489698 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2408309736 0 0) 0x7f24c00b4570 con 0x7f24c015a190 2023-10-09T06:31:36.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.607+0000 7f24c4979700 1 -- 172.21.15.163:0/802489698 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c015acf0 con 0x7f24c0154320 2023-10-09T06:31:36.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.607+0000 7f24c517a700 1 --2- 172.21.15.163:0/802489698 >> v2:172.21.15.181:3300/0 conn(0x7f24c0153f50 0x7f24c00b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:31:36.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.607+0000 7f24c4979700 1 --2- 172.21.15.163:0/802489698 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24c0154320 0x7f24c006d130 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f24b00099b0 tx=0x7f24b0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2133a52906a7b52 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:36.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.607+0000 7f24b7fff700 1 -- 172.21.15.163:0/802489698 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b0009ea0 con 0x7f24c0154320 2023-10-09T06:31:36.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.607+0000 7f24b7fff700 1 -- 172.21.15.163:0/802489698 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b000b860 con 0x7f24c0154320 2023-10-09T06:31:36.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.607+0000 7f24b7fff700 1 -- 172.21.15.163:0/802489698 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b000bc00 con 0x7f24c0154320 2023-10-09T06:31:36.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.608+0000 7f24c73de700 1 -- 172.21.15.163:0/802489698 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24c0154320 msgr2=0x7f24c006d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:36.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.608+0000 7f24c73de700 1 --2- 172.21.15.163:0/802489698 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24c0154320 0x7f24c006d130 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f24b00099b0 tx=0x7f24b0009250 comp rx=0 tx=0).stop 2023-10-09T06:31:36.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.608+0000 7f24c73de700 1 -- 172.21.15.163:0/802489698 shutdown_connections 2023-10-09T06:31:36.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.608+0000 7f24c73de700 1 --2- 172.21.15.163:0/802489698 >> v2:172.21.15.181:3300/0 conn(0x7f24c0153f50 0x7f24c00b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:36.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.608+0000 7f24c73de700 1 --2- 172.21.15.163:0/802489698 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24c0154320 0x7f24c006d130 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:36.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.608+0000 7f24c73de700 1 -- 172.21.15.163:0/802489698 >> 172.21.15.163:0/802489698 conn(0x7f24c006baf0 msgr2=0x7f24c006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:36.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.608+0000 7f24c73de700 1 -- 172.21.15.163:0/802489698 shutdown_connections 2023-10-09T06:31:36.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.608+0000 7f24c73de700 1 -- 172.21.15.163:0/802489698 wait complete. 2023-10-09T06:31:36.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.609+0000 7f24c73de700 1 Processor -- start 2023-10-09T06:31:36.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.609+0000 7f24c73de700 1 -- start start 2023-10-09T06:31:36.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.610+0000 7f24c73de700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24c0153f50 0x7f24c0150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:36.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.610+0000 7f24c73de700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24c0154320 0x7f24c00c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:36.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.610+0000 7f24c73de700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24c015a190 0x7f24c00c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:36.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.610+0000 7f24c73de700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c015bc20 con 0x7f24c015a190 2023-10-09T06:31:36.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.610+0000 7f24c73de700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c015b360 con 0x7f24c0154320 2023-10-09T06:31:36.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.610+0000 7f24c73de700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c015c0b0 con 0x7f24c0153f50 2023-10-09T06:31:36.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.610+0000 7f24c597b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24c015a190 0x7f24c00c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:36.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.610+0000 7f24c597b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24c015a190 0x7f24c00c99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33588/0 (socket says 172.21.15.163:33588) 2023-10-09T06:31:36.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.610+0000 7f24c597b700 1 -- 172.21.15.163:0/1081958899 learned_addr learned my addr 172.21.15.163:0/1081958899 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:36.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.610+0000 7f24c4979700 1 --2- 172.21.15.163:0/1081958899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24c0154320 0x7f24c00c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:36.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.610+0000 7f24c517a700 1 --2- 172.21.15.163:0/1081958899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24c0153f50 0x7f24c0150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:36.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.610+0000 7f24c597b700 1 -- 172.21.15.163:0/1081958899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24c0153f50 msgr2=0x7f24c0150370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:36.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.610+0000 7f24c597b700 1 --2- 172.21.15.163:0/1081958899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24c0153f50 0x7f24c0150370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:36.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.610+0000 7f24c597b700 1 -- 172.21.15.163:0/1081958899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24c0154320 msgr2=0x7f24c00c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:36.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.610+0000 7f24c597b700 1 --2- 172.21.15.163:0/1081958899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24c0154320 0x7f24c00c9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:36.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.610+0000 7f24c597b700 1 -- 172.21.15.163:0/1081958899 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24bc009420 con 0x7f24c015a190 2023-10-09T06:31:36.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.611+0000 7f24c4979700 1 --2- 172.21.15.163:0/1081958899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24c0154320 0x7f24c00c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:31:36.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.611+0000 7f24c597b700 1 --2- 172.21.15.163:0/1081958899 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24c015a190 0x7f24c00c99a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f24b800ccf0 tx=0x7f24b8001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:36.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.611+0000 7f24b57fa700 1 -- 172.21.15.163:0/1081958899 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b8005df0 con 0x7f24c015a190 2023-10-09T06:31:36.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.611+0000 7f24b57fa700 1 -- 172.21.15.163:0/1081958899 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b800a5f0 con 0x7f24c015a190 2023-10-09T06:31:36.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.611+0000 7f24c73de700 1 -- 172.21.15.163:0/1081958899 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b00094f0 con 0x7f24c015a190 2023-10-09T06:31:36.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.611+0000 7f24b57fa700 1 -- 172.21.15.163:0/1081958899 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b800a9b0 con 0x7f24c015a190 2023-10-09T06:31:36.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.611+0000 7f24c73de700 1 -- 172.21.15.163:0/1081958899 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c015b8b0 con 0x7f24c015a190 2023-10-09T06:31:36.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.612+0000 7f24c73de700 1 -- 172.21.15.163:0/1081958899 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24c00b4010 con 0x7f24c015a190 2023-10-09T06:31:36.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.612+0000 7f24b57fa700 1 -- 172.21.15.163:0/1081958899 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f24b80070f0 con 0x7f24c015a190 2023-10-09T06:31:36.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.612+0000 7f24b57fa700 1 --2- 172.21.15.163:0/1081958899 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f24a4066f30 0x7f24a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:36.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.613+0000 7f24b57fa700 1 -- 172.21.15.163:0/1081958899 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f24b8084600 con 0x7f24c015a190 2023-10-09T06:31:36.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.613+0000 7f24c517a700 1 --2- 172.21.15.163:0/1081958899 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f24a4066f30 0x7f24a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:36.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.613+0000 7f24c517a700 1 --2- 172.21.15.163:0/1081958899 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f24a4066f30 0x7f24a40693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f24bc009f40 tx=0x7f24bc0091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:36.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.617+0000 7f24b57fa700 1 -- 172.21.15.163:0/1081958899 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f24b8051920 con 0x7f24c015a190 2023-10-09T06:31:36.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:36.845+0000 7f24c73de700 1 -- 172.21.15.163:0/1081958899 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f24c00718b0 con 0x7f24c015a190 2023-10-09T06:31:37.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.682+0000 7f24b57fa700 1 -- 172.21.15.163:0/1081958899 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 136+0+0 (secure 0 0 0) 0x7f24b8054f40 con 0x7f24c015a190 2023-10-09T06:31:37.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.685+0000 7f24c73de700 1 -- 172.21.15.163:0/1081958899 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f24a4066f30 msgr2=0x7f24a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:37.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.685+0000 7f24c73de700 1 --2- 172.21.15.163:0/1081958899 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f24a4066f30 0x7f24a40693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f24bc009f40 tx=0x7f24bc0091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:37.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.685+0000 7f24c73de700 1 -- 172.21.15.163:0/1081958899 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24c015a190 msgr2=0x7f24c00c99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:37.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.685+0000 7f24c73de700 1 --2- 172.21.15.163:0/1081958899 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24c015a190 0x7f24c00c99a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f24b800ccf0 tx=0x7f24b8001a70 comp rx=0 tx=0).stop 2023-10-09T06:31:37.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.685+0000 7f24c73de700 1 -- 172.21.15.163:0/1081958899 shutdown_connections 2023-10-09T06:31:37.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.685+0000 7f24c73de700 1 --2- 172.21.15.163:0/1081958899 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f24a4066f30 0x7f24a40693e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:37.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.685+0000 7f24c73de700 1 --2- 172.21.15.163:0/1081958899 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24c015a190 0x7f24c00c99a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:37.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.685+0000 7f24c73de700 1 --2- 172.21.15.163:0/1081958899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24c0153f50 0x7f24c0150370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:37.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.685+0000 7f24c73de700 1 --2- 172.21.15.163:0/1081958899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24c0154320 0x7f24c00c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:37.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.685+0000 7f24c73de700 1 -- 172.21.15.163:0/1081958899 >> 172.21.15.163:0/1081958899 conn(0x7f24c006baf0 msgr2=0x7f24c00adc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:37.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.685+0000 7f24c73de700 1 -- 172.21.15.163:0/1081958899 shutdown_connections 2023-10-09T06:31:37.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.685+0000 7f24c73de700 1 -- 172.21.15.163:0/1081958899 wait complete. 2023-10-09T06:31:37.699 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-09T06:31:37.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.817+0000 7ff831e4a700 1 Processor -- start 2023-10-09T06:31:37.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.817+0000 7ff831e4a700 1 -- start start 2023-10-09T06:31:37.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.818+0000 7ff831e4a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff82c156310 0x7ff82c0c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:37.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.818+0000 7ff831e4a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff82c1566e0 0x7ff82c0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:37.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.818+0000 7ff831e4a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff82c0ca860 con 0x7ff82c0ca300 2023-10-09T06:31:37.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.818+0000 7ff831e4a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff82c0cab40 con 0x7ff82c1566e0 2023-10-09T06:31:37.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.818+0000 7ff831e4a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c0cae20 con 0x7ff82c156310 2023-10-09T06:31:37.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.818+0000 7ff82bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff82c1566e0 0x7ff82c0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:37.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.818+0000 7ff82bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff82c1566e0 0x7ff82c0ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33602/0 (socket says 172.21.15.163:33602) 2023-10-09T06:31:37.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.818+0000 7ff82bfff700 1 -- 172.21.15.163:0/3355666729 learned_addr learned my addr 172.21.15.163:0/3355666729 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:37.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.818+0000 7ff830e48700 1 --2- 172.21.15.163:0/3355666729 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff82c156310 0x7ff82c0c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:37.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.818+0000 7ff82bfff700 1 -- 172.21.15.163:0/3355666729 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff82c156310 msgr2=0x7ff82c0c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:37.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.818+0000 7ff82bfff700 1 --2- 172.21.15.163:0/3355666729 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff82c156310 0x7ff82c0c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:37.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.818+0000 7ff82bfff700 1 -- 172.21.15.163:0/3355666729 >> v1:172.21.15.181:6789/0 conn(0x7ff82c0ca300 legacy=0x7ff82c0c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:37.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.818+0000 7ff82bfff700 1 -- 172.21.15.163:0/3355666729 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82c0cb100 con 0x7ff82c1566e0 2023-10-09T06:31:37.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.818+0000 7ff82bfff700 1 --2- 172.21.15.163:0/3355666729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff82c1566e0 0x7ff82c0ae000 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00d1a0 tx=0x7ff81c00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce349819050c263b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:37.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.819+0000 7ff82b7fe700 1 -- 172.21.15.163:0/3355666729 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c010070 con 0x7ff82c1566e0 2023-10-09T06:31:37.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.819+0000 7ff82b7fe700 1 -- 172.21.15.163:0/3355666729 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81c00de80 con 0x7ff82c1566e0 2023-10-09T06:31:37.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.819+0000 7ff82b7fe700 1 -- 172.21.15.163:0/3355666729 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c00b320 con 0x7ff82c1566e0 2023-10-09T06:31:37.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.819+0000 7ff831e4a700 1 -- 172.21.15.163:0/3355666729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff82c1566e0 msgr2=0x7ff82c0ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:37.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.819+0000 7ff831e4a700 1 --2- 172.21.15.163:0/3355666729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff82c1566e0 0x7ff82c0ae000 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00d1a0 tx=0x7ff81c00ba50 comp rx=0 tx=0).stop 2023-10-09T06:31:37.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.819+0000 7ff831e4a700 1 -- 172.21.15.163:0/3355666729 shutdown_connections 2023-10-09T06:31:37.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.819+0000 7ff831e4a700 1 --2- 172.21.15.163:0/3355666729 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff82c156310 0x7ff82c0c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:37.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.819+0000 7ff831e4a700 1 --2- 172.21.15.163:0/3355666729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff82c1566e0 0x7ff82c0ae000 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:37.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.819+0000 7ff831e4a700 1 -- 172.21.15.163:0/3355666729 >> 172.21.15.163:0/3355666729 conn(0x7ff82c0cc1f0 msgr2=0x7ff82c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:37.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.819+0000 7ff831e4a700 1 -- 172.21.15.163:0/3355666729 shutdown_connections 2023-10-09T06:31:37.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.819+0000 7ff831e4a700 1 -- 172.21.15.163:0/3355666729 wait complete. 2023-10-09T06:31:37.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.820+0000 7ff831e4a700 1 Processor -- start 2023-10-09T06:31:37.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.820+0000 7ff831e4a700 1 -- start start 2023-10-09T06:31:37.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.821+0000 7ff831e4a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff82c0ca300 0x7ff82c0af940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:37.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.821+0000 7ff831e4a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff82c156310 0x7ff82c13b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:37.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.821+0000 7ff831e4a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff82c1566e0 0x7ff82c0c1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:37.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.821+0000 7ff831e4a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c0afe80 con 0x7ff82c1566e0 2023-10-09T06:31:37.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.821+0000 7ff831e4a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c157e70 con 0x7ff82c0ca300 2023-10-09T06:31:37.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.821+0000 7ff831e4a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c1663f0 con 0x7ff82c156310 2023-10-09T06:31:37.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.821+0000 7ff831649700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff82c1566e0 0x7ff82c0c1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:37.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.821+0000 7ff831649700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff82c1566e0 0x7ff82c0c1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33606/0 (socket says 172.21.15.163:33606) 2023-10-09T06:31:37.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.821+0000 7ff831649700 1 -- 172.21.15.163:0/616238908 learned_addr learned my addr 172.21.15.163:0/616238908 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:37.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.821+0000 7ff830e48700 1 --2- 172.21.15.163:0/616238908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff82c0ca300 0x7ff82c0af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:37.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.821+0000 7ff82bfff700 1 --2- 172.21.15.163:0/616238908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff82c156310 0x7ff82c13b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:37.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.822+0000 7ff831649700 1 -- 172.21.15.163:0/616238908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff82c156310 msgr2=0x7ff82c13b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:37.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.822+0000 7ff831649700 1 --2- 172.21.15.163:0/616238908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff82c156310 0x7ff82c13b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:37.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.822+0000 7ff831649700 1 -- 172.21.15.163:0/616238908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff82c0ca300 msgr2=0x7ff82c0af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:37.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.822+0000 7ff831649700 1 --2- 172.21.15.163:0/616238908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff82c0ca300 0x7ff82c0af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:37.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.822+0000 7ff831649700 1 -- 172.21.15.163:0/616238908 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c00bcf0 con 0x7ff82c1566e0 2023-10-09T06:31:37.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.822+0000 7ff831649700 1 --2- 172.21.15.163:0/616238908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff82c1566e0 0x7ff82c0c1880 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff81400b4e0 tx=0x7ff814006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:37.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.822+0000 7ff828ff9700 1 -- 172.21.15.163:0/616238908 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81400be90 con 0x7ff82c1566e0 2023-10-09T06:31:37.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.822+0000 7ff828ff9700 1 -- 172.21.15.163:0/616238908 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81400f510 con 0x7ff82c1566e0 2023-10-09T06:31:37.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.822+0000 7ff831e4a700 1 -- 172.21.15.163:0/616238908 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff82c166730 con 0x7ff82c1566e0 2023-10-09T06:31:37.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.822+0000 7ff828ff9700 1 -- 172.21.15.163:0/616238908 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81400f8d0 con 0x7ff82c1566e0 2023-10-09T06:31:37.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.822+0000 7ff831e4a700 1 -- 172.21.15.163:0/616238908 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82c13bd90 con 0x7ff82c1566e0 2023-10-09T06:31:37.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.823+0000 7ff831e4a700 1 -- 172.21.15.163:0/616238908 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff82c04c3c0 con 0x7ff82c1566e0 2023-10-09T06:31:37.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.823+0000 7ff828ff9700 1 -- 172.21.15.163:0/616238908 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff81400fd00 con 0x7ff82c1566e0 2023-10-09T06:31:37.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.824+0000 7ff828ff9700 1 --2- 172.21.15.163:0/616238908 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff80c066f30 0x7ff80c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:37.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.824+0000 7ff828ff9700 1 -- 172.21.15.163:0/616238908 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff814083e80 con 0x7ff82c1566e0 2023-10-09T06:31:37.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.824+0000 7ff830e48700 1 --2- 172.21.15.163:0/616238908 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff80c066f30 0x7ff80c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:37.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.825+0000 7ff830e48700 1 --2- 172.21.15.163:0/616238908 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff80c066f30 0x7ff80c0693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff82c1427d0 tx=0x7ff824008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:37.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:37.828+0000 7ff828ff9700 1 -- 172.21.15.163:0/616238908 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff814051170 con 0x7ff82c1566e0 2023-10-09T06:31:38.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.037+0000 7ff831e4a700 1 -- 172.21.15.163:0/616238908 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff82c0cd0b0 con 0x7ff80c066f30 2023-10-09T06:31:38.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.039+0000 7ff828ff9700 1 -- 172.21.15.163:0/616238908 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7ff82c0cd0b0 con 0x7ff80c066f30 2023-10-09T06:31:38.040 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:31:38.040 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:38.040 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "cephfs" 2023-10-09T06:31:38.040 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:38.040 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:31:38.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.041+0000 7ff831e4a700 1 -- 172.21.15.163:0/616238908 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff80c066f30 msgr2=0x7ff80c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:38.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.041+0000 7ff831e4a700 1 --2- 172.21.15.163:0/616238908 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff80c066f30 0x7ff80c0693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff82c1427d0 tx=0x7ff824008040 comp rx=0 tx=0).stop 2023-10-09T06:31:38.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.041+0000 7ff831e4a700 1 -- 172.21.15.163:0/616238908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff82c1566e0 msgr2=0x7ff82c0c1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:38.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.041+0000 7ff831e4a700 1 --2- 172.21.15.163:0/616238908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff82c1566e0 0x7ff82c0c1880 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff81400b4e0 tx=0x7ff814006f90 comp rx=0 tx=0).stop 2023-10-09T06:31:38.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.041+0000 7ff831e4a700 1 -- 172.21.15.163:0/616238908 shutdown_connections 2023-10-09T06:31:38.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.041+0000 7ff831e4a700 1 --2- 172.21.15.163:0/616238908 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff80c066f30 0x7ff80c0693e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:38.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.041+0000 7ff831e4a700 1 --2- 172.21.15.163:0/616238908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff82c1566e0 0x7ff82c0c1880 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:38.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.041+0000 7ff831e4a700 1 --2- 172.21.15.163:0/616238908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff82c0ca300 0x7ff82c0af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:38.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.041+0000 7ff831e4a700 1 --2- 172.21.15.163:0/616238908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff82c156310 0x7ff82c13b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:38.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.041+0000 7ff831e4a700 1 -- 172.21.15.163:0/616238908 >> 172.21.15.163:0/616238908 conn(0x7ff82c0cc1f0 msgr2=0x7ff82c0c0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:38.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.041+0000 7ff831e4a700 1 -- 172.21.15.163:0/616238908 shutdown_connections 2023-10-09T06:31:38.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.041+0000 7ff831e4a700 1 -- 172.21.15.163:0/616238908 wait complete. 2023-10-09T06:31:38.055 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-09T06:31:38.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.172+0000 7fd1dd782700 1 Processor -- start 2023-10-09T06:31:38.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.172+0000 7fd1dd782700 1 -- start start 2023-10-09T06:31:38.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.173+0000 7fd1dd782700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd1d8159890 0x7fd1d815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:38.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.173+0000 7fd1dd782700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd1d8159c60 0x7fd1d815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:38.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.173+0000 7fd1dd782700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1d815db10 0x7fd1d8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:38.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.173+0000 7fd1dd782700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1d815df50 con 0x7fd1d8159890 2023-10-09T06:31:38.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.173+0000 7fd1dd782700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1d815a160 con 0x7fd1d8159c60 2023-10-09T06:31:38.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.173+0000 7fd1dd782700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d80d13a0 con 0x7fd1d815db10 2023-10-09T06:31:38.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.173+0000 7fd1d6ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd1d8159890 0x7fd1d815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:38.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.173+0000 7fd1d6ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd1d8159890 0x7fd1d815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33610/0 (socket says 172.21.15.163:33610) 2023-10-09T06:31:38.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.173+0000 7fd1d6ffd700 1 -- 172.21.15.163:0/3957556268 learned_addr learned my addr 172.21.15.163:0/3957556268 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:38.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.174+0000 7fd1d6ffd700 1 -- 172.21.15.163:0/3957556268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1d815db10 msgr2=0x7fd1d8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:38.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.174+0000 7fd1d77fe700 1 --2- 172.21.15.163:0/3957556268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1d815db10 0x7fd1d8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:38.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.174+0000 7fd1d6ffd700 1 --2- 172.21.15.163:0/3957556268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1d815db10 0x7fd1d8069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:38.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.174+0000 7fd1d6ffd700 1 -- 172.21.15.163:0/3957556268 >> v2:172.21.15.181:3300/0 conn(0x7fd1d8159c60 msgr2=0x7fd1d815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:38.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.174+0000 7fd1d6ffd700 1 --2- 172.21.15.163:0/3957556268 >> v2:172.21.15.181:3300/0 conn(0x7fd1d8159c60 0x7fd1d815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:38.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.174+0000 7fd1d6ffd700 1 -- 172.21.15.163:0/3957556268 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d815e140 con 0x7fd1d8159890 2023-10-09T06:31:38.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.174+0000 7fd1d6ffd700 1 --2- 172.21.15.163:0/3957556268 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1d8159890 0x7fd1d815abd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd1d80b67f0 tx=0x7fd1c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8772158107fd90c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:38.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.174+0000 7fd1d5ffb700 1 -- 172.21.15.163:0/3957556268 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c800b860 con 0x7fd1d8159890 2023-10-09T06:31:38.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.174+0000 7fd1d5ffb700 1 -- 172.21.15.163:0/3957556268 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c800b9c0 con 0x7fd1d8159890 2023-10-09T06:31:38.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.174+0000 7fd1d5ffb700 1 -- 172.21.15.163:0/3957556268 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c800bd60 con 0x7fd1d8159890 2023-10-09T06:31:38.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.174+0000 7fd1dd782700 1 -- 172.21.15.163:0/3957556268 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1d8159890 msgr2=0x7fd1d815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:38.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.174+0000 7fd1dd782700 1 --2- 172.21.15.163:0/3957556268 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1d8159890 0x7fd1d815abd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd1d80b67f0 tx=0x7fd1c8009250 comp rx=0 tx=0).stop 2023-10-09T06:31:38.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.174+0000 7fd1dd782700 1 -- 172.21.15.163:0/3957556268 shutdown_connections 2023-10-09T06:31:38.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.174+0000 7fd1dd782700 1 --2- 172.21.15.163:0/3957556268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1d815db10 0x7fd1d8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:38.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.174+0000 7fd1dd782700 1 --2- 172.21.15.163:0/3957556268 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1d8159890 0x7fd1d815abd0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:38.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.174+0000 7fd1dd782700 1 --2- 172.21.15.163:0/3957556268 >> v2:172.21.15.181:3300/0 conn(0x7fd1d8159c60 0x7fd1d815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:38.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.174+0000 7fd1dd782700 1 -- 172.21.15.163:0/3957556268 >> 172.21.15.163:0/3957556268 conn(0x7fd1d8068e20 msgr2=0x7fd1d80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:38.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.175+0000 7fd1dd782700 1 -- 172.21.15.163:0/3957556268 shutdown_connections 2023-10-09T06:31:38.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.175+0000 7fd1dd782700 1 -- 172.21.15.163:0/3957556268 wait complete. 2023-10-09T06:31:38.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.176+0000 7fd1dd782700 1 Processor -- start 2023-10-09T06:31:38.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.176+0000 7fd1dd782700 1 -- start start 2023-10-09T06:31:38.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.176+0000 7fd1dd782700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1d8159890 0x7fd1d81478a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:38.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.176+0000 7fd1dd782700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1d8159c60 0x7fd1d80b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:38.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.176+0000 7fd1dd782700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1d815db10 0x7fd1d80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:38.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.177+0000 7fd1dd782700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d80c9a40 con 0x7fd1d8159890 2023-10-09T06:31:38.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.177+0000 7fd1dd782700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d80b3c10 con 0x7fd1d8159c60 2023-10-09T06:31:38.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.177+0000 7fd1dd782700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d8147de0 con 0x7fd1d815db10 2023-10-09T06:31:38.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.177+0000 7fd1d6ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1d8159890 0x7fd1d81478a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:38.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.177+0000 7fd1d6ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1d8159890 0x7fd1d81478a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33616/0 (socket says 172.21.15.163:33616) 2023-10-09T06:31:38.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.177+0000 7fd1d6ffd700 1 -- 172.21.15.163:0/1026286795 learned_addr learned my addr 172.21.15.163:0/1026286795 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:38.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.177+0000 7fd1d77fe700 1 --2- 172.21.15.163:0/1026286795 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1d815db10 0x7fd1d80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:38.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.177+0000 7fd1d67fc700 1 --2- 172.21.15.163:0/1026286795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1d8159c60 0x7fd1d80b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:38.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.177+0000 7fd1d6ffd700 1 -- 172.21.15.163:0/1026286795 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1d815db10 msgr2=0x7fd1d80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:38.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.177+0000 7fd1d6ffd700 1 --2- 172.21.15.163:0/1026286795 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1d815db10 0x7fd1d80b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:38.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.177+0000 7fd1d6ffd700 1 -- 172.21.15.163:0/1026286795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1d8159c60 msgr2=0x7fd1d80b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:38.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.177+0000 7fd1d6ffd700 1 --2- 172.21.15.163:0/1026286795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1d8159c60 0x7fd1d80b5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:38.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.177+0000 7fd1d6ffd700 1 -- 172.21.15.163:0/1026286795 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c80094f0 con 0x7fd1d8159890 2023-10-09T06:31:38.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.177+0000 7fd1d67fc700 1 --2- 172.21.15.163:0/1026286795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1d8159c60 0x7fd1d80b5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:38.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.177+0000 7fd1d77fe700 1 --2- 172.21.15.163:0/1026286795 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1d815db10 0x7fd1d80b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:38.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.177+0000 7fd1d6ffd700 1 --2- 172.21.15.163:0/1026286795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1d8159890 0x7fd1d81478a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009290 tx=0x7fd1c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:38.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.177+0000 7fd1beffd700 1 -- 172.21.15.163:0/1026286795 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8026480 con 0x7fd1d8159890 2023-10-09T06:31:38.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.177+0000 7fd1beffd700 1 -- 172.21.15.163:0/1026286795 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c801f070 con 0x7fd1d8159890 2023-10-09T06:31:38.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.177+0000 7fd1dd782700 1 -- 172.21.15.163:0/1026286795 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d80b3630 con 0x7fd1d8159890 2023-10-09T06:31:38.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.177+0000 7fd1beffd700 1 -- 172.21.15.163:0/1026286795 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8026830 con 0x7fd1d8159890 2023-10-09T06:31:38.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.177+0000 7fd1dd782700 1 -- 172.21.15.163:0/1026286795 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d80b38a0 con 0x7fd1d8159890 2023-10-09T06:31:38.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.178+0000 7fd1dd782700 1 -- 172.21.15.163:0/1026286795 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1d804c3c0 con 0x7fd1d8159890 2023-10-09T06:31:38.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.179+0000 7fd1beffd700 1 -- 172.21.15.163:0/1026286795 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd1c8026ba0 con 0x7fd1d8159890 2023-10-09T06:31:38.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.179+0000 7fd1beffd700 1 --2- 172.21.15.163:0/1026286795 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd1c4066f30 0x7fd1c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:38.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.179+0000 7fd1beffd700 1 -- 172.21.15.163:0/1026286795 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd1c8021070 con 0x7fd1d8159890 2023-10-09T06:31:38.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.179+0000 7fd1d67fc700 1 --2- 172.21.15.163:0/1026286795 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd1c4066f30 0x7fd1c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:38.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.180+0000 7fd1d67fc700 1 --2- 172.21.15.163:0/1026286795 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd1c4066f30 0x7fd1c40693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd1d80b2540 tx=0x7fd1cc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:38.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.183+0000 7fd1beffd700 1 -- 172.21.15.163:0/1026286795 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd1c8053640 con 0x7fd1d8159890 2023-10-09T06:31:38.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.364+0000 7fd1dd782700 1 -- 172.21.15.163:0/1026286795 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd1d806a5b0 con 0x7fd1d8159890 2023-10-09T06:31:38.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.364+0000 7fd1beffd700 1 -- 172.21.15.163:0/1026286795 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd1c8056c60 con 0x7fd1d8159890 2023-10-09T06:31:38.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.367+0000 7fd1dd782700 1 -- 172.21.15.163:0/1026286795 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd1c4066f30 msgr2=0x7fd1c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:38.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.367+0000 7fd1dd782700 1 --2- 172.21.15.163:0/1026286795 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd1c4066f30 0x7fd1c40693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd1d80b2540 tx=0x7fd1cc008040 comp rx=0 tx=0).stop 2023-10-09T06:31:38.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.367+0000 7fd1dd782700 1 -- 172.21.15.163:0/1026286795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1d8159890 msgr2=0x7fd1d81478a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:38.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.367+0000 7fd1dd782700 1 --2- 172.21.15.163:0/1026286795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1d8159890 0x7fd1d81478a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009290 tx=0x7fd1c8017000 comp rx=0 tx=0).stop 2023-10-09T06:31:38.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.367+0000 7fd1dd782700 1 -- 172.21.15.163:0/1026286795 shutdown_connections 2023-10-09T06:31:38.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.367+0000 7fd1dd782700 1 --2- 172.21.15.163:0/1026286795 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd1c4066f30 0x7fd1c40693e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:38.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.367+0000 7fd1dd782700 1 --2- 172.21.15.163:0/1026286795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1d8159890 0x7fd1d81478a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:38.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.367+0000 7fd1dd782700 1 --2- 172.21.15.163:0/1026286795 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1d815db10 0x7fd1d80b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:38.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.367+0000 7fd1dd782700 1 --2- 172.21.15.163:0/1026286795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1d8159c60 0x7fd1d80b5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:38.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.367+0000 7fd1dd782700 1 -- 172.21.15.163:0/1026286795 >> 172.21.15.163:0/1026286795 conn(0x7fd1d8068e20 msgr2=0x7fd1d80b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:38.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.367+0000 7fd1dd782700 1 -- 172.21.15.163:0/1026286795 shutdown_connections 2023-10-09T06:31:38.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.367+0000 7fd1dd782700 1 -- 172.21.15.163:0/1026286795 wait complete. 2023-10-09T06:31:38.379 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create volume_0000000000308465 2023-10-09T06:31:38.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.496+0000 7f4ece2cb700 1 Processor -- start 2023-10-09T06:31:38.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.496+0000 7f4ece2cb700 1 -- start start 2023-10-09T06:31:38.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.497+0000 7f4ece2cb700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4ec8159c60 0x7f4ec815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:38.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.497+0000 7f4ece2cb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ec815db10 0x7f4ec8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:38.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.497+0000 7f4ece2cb700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec815a160 con 0x7f4ec8159890 2023-10-09T06:31:38.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.497+0000 7f4ece2cb700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec815df50 con 0x7f4ec8159c60 2023-10-09T06:31:38.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.497+0000 7f4ece2cb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec80d13a0 con 0x7f4ec815db10 2023-10-09T06:31:38.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.497+0000 7f4ec77fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4ec8159c60 0x7f4ec815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:38.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.497+0000 7f4ec77fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4ec8159c60 0x7f4ec815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40586/0 (socket says 172.21.15.163:40586) 2023-10-09T06:31:38.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.497+0000 7f4ec77fe700 1 -- 172.21.15.163:0/3708295235 learned_addr learned my addr 172.21.15.163:0/3708295235 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:38.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.497+0000 7f4ecc868700 1 --2- 172.21.15.163:0/3708295235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ec815db10 0x7f4ec8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:38.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.497+0000 7f4ec77fe700 1 -- 172.21.15.163:0/3708295235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ec815db10 msgr2=0x7f4ec8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:38.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.497+0000 7f4ec77fe700 1 --2- 172.21.15.163:0/3708295235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ec815db10 0x7f4ec8069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:38.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.497+0000 7f4ec77fe700 1 -- 172.21.15.163:0/3708295235 >> v1:172.21.15.181:6789/0 conn(0x7f4ec8159890 legacy=0x7f4ec815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:38.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.497+0000 7f4ec77fe700 1 -- 172.21.15.163:0/3708295235 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec815e140 con 0x7f4ec8159c60 2023-10-09T06:31:38.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.497+0000 7f4ecc868700 1 --2- 172.21.15.163:0/3708295235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ec815db10 0x7f4ec8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:38.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.498+0000 7f4ec77fe700 1 --2- 172.21.15.163:0/3708295235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec8159c60 0x7f4ec815d5d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc0099b0 tx=0x7f4ebc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f646b1c76157fd server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:38.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.498+0000 7f4ec6ffd700 1 -- 172.21.15.163:0/3708295235 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc00b820 con 0x7f4ec8159c60 2023-10-09T06:31:38.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.498+0000 7f4ec6ffd700 1 -- 172.21.15.163:0/3708295235 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ebc00b980 con 0x7f4ec8159c60 2023-10-09T06:31:38.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.498+0000 7f4ec6ffd700 1 -- 172.21.15.163:0/3708295235 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc00bd20 con 0x7f4ec8159c60 2023-10-09T06:31:38.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.498+0000 7f4ece2cb700 1 -- 172.21.15.163:0/3708295235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec8159c60 msgr2=0x7f4ec815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:38.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.498+0000 7f4ece2cb700 1 --2- 172.21.15.163:0/3708295235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec8159c60 0x7f4ec815d5d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc0099b0 tx=0x7f4ebc009250 comp rx=0 tx=0).stop 2023-10-09T06:31:38.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.498+0000 7f4ece2cb700 1 -- 172.21.15.163:0/3708295235 shutdown_connections 2023-10-09T06:31:38.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.498+0000 7f4ece2cb700 1 --2- 172.21.15.163:0/3708295235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ec815db10 0x7f4ec8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:38.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.498+0000 7f4ece2cb700 1 --2- 172.21.15.163:0/3708295235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec8159c60 0x7f4ec815d5d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:38.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.498+0000 7f4ece2cb700 1 -- 172.21.15.163:0/3708295235 >> 172.21.15.163:0/3708295235 conn(0x7f4ec8068e20 msgr2=0x7f4ec80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:38.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.498+0000 7f4ece2cb700 1 -- 172.21.15.163:0/3708295235 shutdown_connections 2023-10-09T06:31:38.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.498+0000 7f4ece2cb700 1 -- 172.21.15.163:0/3708295235 wait complete. 2023-10-09T06:31:38.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.499+0000 7f4ece2cb700 1 Processor -- start 2023-10-09T06:31:38.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.500+0000 7f4ece2cb700 1 -- start start 2023-10-09T06:31:38.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.500+0000 7f4ece2cb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec8159890 0x7f4ec816d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:38.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.500+0000 7f4ece2cb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ec8159c60 0x7f4ec816d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:38.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.500+0000 7f4ece2cb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ec815db10 0x7f4ec80b5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:38.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.500+0000 7f4ece2cb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec81529d0 con 0x7f4ec8159890 2023-10-09T06:31:38.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.500+0000 7f4ece2cb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec80b0540 con 0x7f4ec8159c60 2023-10-09T06:31:38.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.500+0000 7f4ece2cb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec80b6030 con 0x7f4ec815db10 2023-10-09T06:31:38.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.500+0000 7f4ec7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec8159890 0x7f4ec816d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:38.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.500+0000 7f4ec7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec8159890 0x7f4ec816d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40594/0 (socket says 172.21.15.163:40594) 2023-10-09T06:31:38.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.500+0000 7f4ec7fff700 1 -- 172.21.15.163:0/1078899283 learned_addr learned my addr 172.21.15.163:0/1078899283 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:38.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.500+0000 7f4ec77fe700 1 --2- 172.21.15.163:0/1078899283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ec8159c60 0x7f4ec816d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:38.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.501+0000 7f4ecc868700 1 --2- 172.21.15.163:0/1078899283 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ec815db10 0x7f4ec80b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:38.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.501+0000 7f4ec7fff700 1 -- 172.21.15.163:0/1078899283 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ec815db10 msgr2=0x7f4ec80b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:38.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.501+0000 7f4ec7fff700 1 --2- 172.21.15.163:0/1078899283 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ec815db10 0x7f4ec80b5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:38.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.501+0000 7f4ec7fff700 1 -- 172.21.15.163:0/1078899283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ec8159c60 msgr2=0x7f4ec816d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:38.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.501+0000 7f4ec7fff700 1 --2- 172.21.15.163:0/1078899283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ec8159c60 0x7f4ec816d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:38.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.501+0000 7f4ec7fff700 1 -- 172.21.15.163:0/1078899283 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ebc0094f0 con 0x7f4ec8159890 2023-10-09T06:31:38.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.501+0000 7f4ec7fff700 1 --2- 172.21.15.163:0/1078899283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec8159890 0x7f4ec816d220 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800c5d0 tx=0x7f4eb8007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:38.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.501+0000 7f4eaffff700 1 -- 172.21.15.163:0/1078899283 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8011590 con 0x7f4ec8159890 2023-10-09T06:31:38.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.501+0000 7f4eaffff700 1 -- 172.21.15.163:0/1078899283 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb8011710 con 0x7f4ec8159890 2023-10-09T06:31:38.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.501+0000 7f4ece2cb700 1 -- 172.21.15.163:0/1078899283 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec80b6310 con 0x7f4ec8159890 2023-10-09T06:31:38.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.501+0000 7f4eaffff700 1 -- 172.21.15.163:0/1078899283 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8011ad0 con 0x7f4ec8159890 2023-10-09T06:31:38.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.501+0000 7f4ece2cb700 1 -- 172.21.15.163:0/1078899283 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec80b0ac0 con 0x7f4ec8159890 2023-10-09T06:31:38.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.502+0000 7f4ece2cb700 1 -- 172.21.15.163:0/1078899283 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec8064ed0 con 0x7f4ec8159890 2023-10-09T06:31:38.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.502+0000 7f4eaffff700 1 -- 172.21.15.163:0/1078899283 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4eb801c460 con 0x7f4ec8159890 2023-10-09T06:31:38.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.503+0000 7f4eaffff700 1 --2- 172.21.15.163:0/1078899283 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4eb4066f30 0x7f4eb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:38.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.503+0000 7f4eaffff700 1 -- 172.21.15.163:0/1078899283 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4eb80841b0 con 0x7f4ec8159890 2023-10-09T06:31:38.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.503+0000 7f4ec77fe700 1 --2- 172.21.15.163:0/1078899283 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4eb4066f30 0x7f4eb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:38.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.504+0000 7f4ec77fe700 1 --2- 172.21.15.163:0/1078899283 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4eb4066f30 0x7f4eb40693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc0099b0 tx=0x7f4ebc016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:38.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.507+0000 7f4eaffff700 1 -- 172.21.15.163:0/1078899283 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4eb8051420 con 0x7f4ec8159890 2023-10-09T06:31:38.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:38.718+0000 7f4ece2cb700 1 -- 172.21.15.163:0/1078899283 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "volume_0000000000308465", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ec804c3c0 con 0x7f4eb4066f30 2023-10-09T06:31:41.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.717+0000 7f4eaffff700 1 -- 172.21.15.163:0/1078899283 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) v1 ==== 60+0+0 (secure 0 0 0) 0x7f4ec804c3c0 con 0x7f4eb4066f30 2023-10-09T06:31:41.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.722+0000 7f4ece2cb700 1 -- 172.21.15.163:0/1078899283 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4eb4066f30 msgr2=0x7f4eb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:41.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.722+0000 7f4ece2cb700 1 --2- 172.21.15.163:0/1078899283 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4eb4066f30 0x7f4eb40693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc0099b0 tx=0x7f4ebc016000 comp rx=0 tx=0).stop 2023-10-09T06:31:41.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.722+0000 7f4ece2cb700 1 -- 172.21.15.163:0/1078899283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec8159890 msgr2=0x7f4ec816d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:41.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.722+0000 7f4ece2cb700 1 --2- 172.21.15.163:0/1078899283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec8159890 0x7f4ec816d220 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800c5d0 tx=0x7f4eb8007f60 comp rx=0 tx=0).stop 2023-10-09T06:31:41.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.722+0000 7f4ece2cb700 1 -- 172.21.15.163:0/1078899283 shutdown_connections 2023-10-09T06:31:41.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.722+0000 7f4ece2cb700 1 --2- 172.21.15.163:0/1078899283 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4eb4066f30 0x7f4eb40693e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:41.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.722+0000 7f4ece2cb700 1 --2- 172.21.15.163:0/1078899283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec8159890 0x7f4ec816d220 secure :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800c5d0 tx=0x7f4eb8007f60 comp rx=0 tx=0).stop 2023-10-09T06:31:41.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.722+0000 7f4ece2cb700 1 --2- 172.21.15.163:0/1078899283 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ec815db10 0x7f4ec80b5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:41.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.722+0000 7f4ece2cb700 1 --2- 172.21.15.163:0/1078899283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ec8159c60 0x7f4ec816d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:41.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.722+0000 7f4ece2cb700 1 -- 172.21.15.163:0/1078899283 >> 172.21.15.163:0/1078899283 conn(0x7f4ec8068e20 msgr2=0x7f4ec8155100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:41.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.723+0000 7f4ece2cb700 1 -- 172.21.15.163:0/1078899283 shutdown_connections 2023-10-09T06:31:41.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.723+0000 7f4ece2cb700 1 -- 172.21.15.163:0/1078899283 wait complete. 2023-10-09T06:31:41.726 INFO:teuthology.orchestra.run.smithi163.stderr:Volume created successfully (no MDS daemons created) 2023-10-09T06:31:41.740 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-09T06:31:41.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.860+0000 7f30ee0a1700 1 Processor -- start 2023-10-09T06:31:41.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.860+0000 7f30ee0a1700 1 -- start start 2023-10-09T06:31:41.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.861+0000 7f30ee0a1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f30e8159890 0x7f30e815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:41.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.861+0000 7f30ee0a1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f30e8159c60 0x7f30e815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:41.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.861+0000 7f30ee0a1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30e815db10 0x7f30e8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:41.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.861+0000 7f30ee0a1700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e815df50 con 0x7f30e8159c60 2023-10-09T06:31:41.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.861+0000 7f30ee0a1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e815a160 con 0x7f30e8159890 2023-10-09T06:31:41.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.861+0000 7f30ee0a1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e80d13a0 con 0x7f30e815db10 2023-10-09T06:31:41.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.861+0000 7f30e6ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f30e8159c60 0x7f30e815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:41.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.861+0000 7f30e6ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f30e8159c60 0x7f30e815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40608/0 (socket says 172.21.15.163:40608) 2023-10-09T06:31:41.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.861+0000 7f30e6ffd700 1 -- 172.21.15.163:0/2601143790 learned_addr learned my addr 172.21.15.163:0/2601143790 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:41.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.861+0000 7f30e7fff700 1 --2- 172.21.15.163:0/2601143790 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30e815db10 0x7f30e8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:41.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.861+0000 7f30e77fe700 1 --2- 172.21.15.163:0/2601143790 >> v2:172.21.15.181:3300/0 conn(0x7f30e8159890 0x7f30e815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:41.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.862+0000 7f30e6ffd700 1 -- 172.21.15.163:0/2601143790 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30e815db10 msgr2=0x7f30e8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:41.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.862+0000 7f30e6ffd700 1 --2- 172.21.15.163:0/2601143790 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30e815db10 0x7f30e8069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:41.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.862+0000 7f30e6ffd700 1 -- 172.21.15.163:0/2601143790 >> v2:172.21.15.181:3300/0 conn(0x7f30e8159890 msgr2=0x7f30e815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:41.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.862+0000 7f30e6ffd700 1 --2- 172.21.15.163:0/2601143790 >> v2:172.21.15.181:3300/0 conn(0x7f30e8159890 0x7f30e815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:41.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.862+0000 7f30e6ffd700 1 -- 172.21.15.163:0/2601143790 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e815e140 con 0x7f30e8159c60 2023-10-09T06:31:41.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.862+0000 7f30e6ffd700 1 --2- 172.21.15.163:0/2601143790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30e8159c60 0x7f30e815d5d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f30e80d1090 tx=0x7f30dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b6d439cd767ec8 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:41.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.862+0000 7f30e67fc700 1 -- 172.21.15.163:0/2601143790 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc00b640 con 0x7f30e8159c60 2023-10-09T06:31:41.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.862+0000 7f30e67fc700 1 -- 172.21.15.163:0/2601143790 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30dc00b7a0 con 0x7f30e8159c60 2023-10-09T06:31:41.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.862+0000 7f30e67fc700 1 -- 172.21.15.163:0/2601143790 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc00bb60 con 0x7f30e8159c60 2023-10-09T06:31:41.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.862+0000 7f30ee0a1700 1 -- 172.21.15.163:0/2601143790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30e8159c60 msgr2=0x7f30e815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:41.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.862+0000 7f30ee0a1700 1 --2- 172.21.15.163:0/2601143790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30e8159c60 0x7f30e815d5d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f30e80d1090 tx=0x7f30dc009250 comp rx=0 tx=0).stop 2023-10-09T06:31:41.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.862+0000 7f30ee0a1700 1 -- 172.21.15.163:0/2601143790 shutdown_connections 2023-10-09T06:31:41.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.862+0000 7f30ee0a1700 1 --2- 172.21.15.163:0/2601143790 >> v2:172.21.15.181:3300/0 conn(0x7f30e8159890 0x7f30e815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:41.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.862+0000 7f30ee0a1700 1 --2- 172.21.15.163:0/2601143790 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30e815db10 0x7f30e8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:41.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.862+0000 7f30ee0a1700 1 --2- 172.21.15.163:0/2601143790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30e8159c60 0x7f30e815d5d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:41.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.862+0000 7f30ee0a1700 1 -- 172.21.15.163:0/2601143790 >> 172.21.15.163:0/2601143790 conn(0x7f30e8068e20 msgr2=0x7f30e80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:41.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.862+0000 7f30ee0a1700 1 -- 172.21.15.163:0/2601143790 shutdown_connections 2023-10-09T06:31:41.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.862+0000 7f30ee0a1700 1 -- 172.21.15.163:0/2601143790 wait complete. 2023-10-09T06:31:41.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.864+0000 7f30ee0a1700 1 Processor -- start 2023-10-09T06:31:41.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.864+0000 7f30ee0a1700 1 -- start start 2023-10-09T06:31:41.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.864+0000 7f30ee0a1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30e8159890 0x7f30e80b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:41.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.864+0000 7f30ee0a1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30e8159c60 0x7f30e80b2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:41.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.864+0000 7f30ee0a1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30e815db10 0x7f30e80b5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:41.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.864+0000 7f30ee0a1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e80b05d0 con 0x7f30e8159c60 2023-10-09T06:31:41.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.864+0000 7f30ee0a1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e80b46a0 con 0x7f30e8159890 2023-10-09T06:31:41.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.864+0000 7f30ee0a1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e80b4980 con 0x7f30e815db10 2023-10-09T06:31:41.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.864+0000 7f30e6ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30e8159c60 0x7f30e80b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:41.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.865+0000 7f30e6ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30e8159c60 0x7f30e80b2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40622/0 (socket says 172.21.15.163:40622) 2023-10-09T06:31:41.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.865+0000 7f30e6ffd700 1 -- 172.21.15.163:0/3068485050 learned_addr learned my addr 172.21.15.163:0/3068485050 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:41.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.865+0000 7f30e77fe700 1 --2- 172.21.15.163:0/3068485050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30e8159890 0x7f30e80b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:41.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.865+0000 7f30e7fff700 1 --2- 172.21.15.163:0/3068485050 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30e815db10 0x7f30e80b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:41.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.865+0000 7f30e6ffd700 1 -- 172.21.15.163:0/3068485050 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30e815db10 msgr2=0x7f30e80b5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:41.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.865+0000 7f30e6ffd700 1 --2- 172.21.15.163:0/3068485050 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30e815db10 0x7f30e80b5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:41.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.865+0000 7f30e6ffd700 1 -- 172.21.15.163:0/3068485050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30e8159890 msgr2=0x7f30e80b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:41.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.865+0000 7f30e6ffd700 1 --2- 172.21.15.163:0/3068485050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30e8159890 0x7f30e80b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:41.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.865+0000 7f30e6ffd700 1 -- 172.21.15.163:0/3068485050 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30dc0094f0 con 0x7f30e8159c60 2023-10-09T06:31:41.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.865+0000 7f30e6ffd700 1 --2- 172.21.15.163:0/3068485050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30e8159c60 0x7f30e80b2430 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f30e8070e20 tx=0x7f30dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:41.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.865+0000 7f30cf7fe700 1 -- 172.21.15.163:0/3068485050 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc00b910 con 0x7f30e8159c60 2023-10-09T06:31:41.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.865+0000 7f30cf7fe700 1 -- 172.21.15.163:0/3068485050 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30dc00ba70 con 0x7f30e8159c60 2023-10-09T06:31:41.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.865+0000 7f30ee0a1700 1 -- 172.21.15.163:0/3068485050 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30e80b07e0 con 0x7f30e8159c60 2023-10-09T06:31:41.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.865+0000 7f30cf7fe700 1 -- 172.21.15.163:0/3068485050 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc011630 con 0x7f30e8159c60 2023-10-09T06:31:41.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.865+0000 7f30ee0a1700 1 -- 172.21.15.163:0/3068485050 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e80c8560 con 0x7f30e8159c60 2023-10-09T06:31:41.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.866+0000 7f30ee0a1700 1 -- 172.21.15.163:0/3068485050 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e80b0d00 con 0x7f30e8159c60 2023-10-09T06:31:41.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.867+0000 7f30cf7fe700 1 -- 172.21.15.163:0/3068485050 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f30dc0119a0 con 0x7f30e8159c60 2023-10-09T06:31:41.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.867+0000 7f30cf7fe700 1 --2- 172.21.15.163:0/3068485050 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f30d4066f30 0x7f30d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:41.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.867+0000 7f30cf7fe700 1 -- 172.21.15.163:0/3068485050 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f30dc085ff0 con 0x7f30e8159c60 2023-10-09T06:31:41.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.867+0000 7f30e77fe700 1 --2- 172.21.15.163:0/3068485050 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f30d4066f30 0x7f30d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:41.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.868+0000 7f30e77fe700 1 --2- 172.21.15.163:0/3068485050 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f30d4066f30 0x7f30d40693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f30d8009d10 tx=0x7f30d80091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:41.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:41.871+0000 7f30cf7fe700 1 -- 172.21.15.163:0/3068485050 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f30e80b0d00 con 0x7f30e8159c60 2023-10-09T06:31:42.079 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.078+0000 7f30ee0a1700 1 -- 172.21.15.163:0/3068485050 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f30e80b0d00 con 0x7f30d4066f30 2023-10-09T06:31:42.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.081+0000 7f30cf7fe700 1 -- 172.21.15.163:0/3068485050 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+96 (secure 0 0 0) 0x7f30e80b0d00 con 0x7f30d4066f30 2023-10-09T06:31:42.081 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:31:42.081 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:42.081 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "cephfs" 2023-10-09T06:31:42.081 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:42.081 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:42.082 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "volume_0000000000308465" 2023-10-09T06:31:42.082 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:42.082 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:31:42.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.082+0000 7f30ee0a1700 1 -- 172.21.15.163:0/3068485050 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f30d4066f30 msgr2=0x7f30d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:42.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.082+0000 7f30ee0a1700 1 --2- 172.21.15.163:0/3068485050 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f30d4066f30 0x7f30d40693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f30d8009d10 tx=0x7f30d80091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.082+0000 7f30ee0a1700 1 -- 172.21.15.163:0/3068485050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30e8159c60 msgr2=0x7f30e80b2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:42.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.082+0000 7f30ee0a1700 1 --2- 172.21.15.163:0/3068485050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30e8159c60 0x7f30e80b2430 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f30e8070e20 tx=0x7f30dc017000 comp rx=0 tx=0).stop 2023-10-09T06:31:42.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.082+0000 7f30ee0a1700 1 -- 172.21.15.163:0/3068485050 shutdown_connections 2023-10-09T06:31:42.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.082+0000 7f30ee0a1700 1 --2- 172.21.15.163:0/3068485050 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f30d4066f30 0x7f30d40693e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.082+0000 7f30ee0a1700 1 --2- 172.21.15.163:0/3068485050 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30e815db10 0x7f30e80b5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.082+0000 7f30ee0a1700 1 --2- 172.21.15.163:0/3068485050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30e8159890 0x7f30e80b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.082+0000 7f30ee0a1700 1 --2- 172.21.15.163:0/3068485050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30e8159c60 0x7f30e80b2430 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.082+0000 7f30ee0a1700 1 -- 172.21.15.163:0/3068485050 >> 172.21.15.163:0/3068485050 conn(0x7f30e8068e20 msgr2=0x7f30e80cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:42.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.082+0000 7f30ee0a1700 1 -- 172.21.15.163:0/3068485050 shutdown_connections 2023-10-09T06:31:42.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.082+0000 7f30ee0a1700 1 -- 172.21.15.163:0/3068485050 wait complete. 2023-10-09T06:31:42.095 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool ls detail --format=json 2023-10-09T06:31:42.211 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.210+0000 7f519d886700 1 Processor -- start 2023-10-09T06:31:42.211 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.210+0000 7f519d886700 1 -- start start 2023-10-09T06:31:42.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.211+0000 7f519d886700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5198140720 0x7f5198068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:42.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.211+0000 7f519d886700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f51980688d0 0x7f51980c8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:42.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.211+0000 7f519d886700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5198068d30 con 0x7f5198140350 2023-10-09T06:31:42.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.211+0000 7f519d886700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51980c7de0 con 0x7f5198140720 2023-10-09T06:31:42.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.211+0000 7f519d886700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5198140c10 con 0x7f51980688d0 2023-10-09T06:31:42.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.211+0000 7f518e7fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5198140720 0x7f5198068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:42.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.211+0000 7f518e7fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5198140720 0x7f5198068390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40638/0 (socket says 172.21.15.163:40638) 2023-10-09T06:31:42.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.211+0000 7f518e7fc700 1 -- 172.21.15.163:0/3543482940 learned_addr learned my addr 172.21.15.163:0/3543482940 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:42.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.211+0000 7f51977fe700 1 --2- 172.21.15.163:0/3543482940 >> v2:172.21.15.181:3300/0 conn(0x7f51980688d0 0x7f51980c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:42.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.212+0000 7f518e7fc700 1 -- 172.21.15.163:0/3543482940 >> v2:172.21.15.181:3300/0 conn(0x7f51980688d0 msgr2=0x7f51980c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:42.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.212+0000 7f518e7fc700 1 --2- 172.21.15.163:0/3543482940 >> v2:172.21.15.181:3300/0 conn(0x7f51980688d0 0x7f51980c8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.212+0000 7f518e7fc700 1 -- 172.21.15.163:0/3543482940 >> v1:172.21.15.181:6789/0 conn(0x7f5198140350 legacy=0x7f5198069c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:42.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.212+0000 7f518e7fc700 1 -- 172.21.15.163:0/3543482940 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5198141490 con 0x7f5198140720 2023-10-09T06:31:42.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.212+0000 7f51977fe700 1 --2- 172.21.15.163:0/3543482940 >> v2:172.21.15.181:3300/0 conn(0x7f51980688d0 0x7f51980c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:42.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.212+0000 7f518e7fc700 1 --2- 172.21.15.163:0/3543482940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5198140720 0x7f5198068390 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f51980c7c40 tx=0x7f5180009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3482fef636568feb server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:42.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.212+0000 7f51967fc700 1 -- 172.21.15.163:0/3543482940 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518000b860 con 0x7f5198140720 2023-10-09T06:31:42.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.212+0000 7f51967fc700 1 -- 172.21.15.163:0/3543482940 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518000b9c0 con 0x7f5198140720 2023-10-09T06:31:42.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.212+0000 7f51967fc700 1 -- 172.21.15.163:0/3543482940 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518000bd60 con 0x7f5198140720 2023-10-09T06:31:42.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.212+0000 7f519d886700 1 -- 172.21.15.163:0/3543482940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5198140720 msgr2=0x7f5198068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:42.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.212+0000 7f519d886700 1 --2- 172.21.15.163:0/3543482940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5198140720 0x7f5198068390 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f51980c7c40 tx=0x7f5180009250 comp rx=0 tx=0).stop 2023-10-09T06:31:42.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.212+0000 7f519d886700 1 -- 172.21.15.163:0/3543482940 shutdown_connections 2023-10-09T06:31:42.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.212+0000 7f519d886700 1 --2- 172.21.15.163:0/3543482940 >> v2:172.21.15.181:3300/0 conn(0x7f51980688d0 0x7f51980c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.212+0000 7f519d886700 1 --2- 172.21.15.163:0/3543482940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5198140720 0x7f5198068390 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.212+0000 7f519d886700 1 -- 172.21.15.163:0/3543482940 >> 172.21.15.163:0/3543482940 conn(0x7f51980cd670 msgr2=0x7f51980cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:42.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.212+0000 7f519d886700 1 -- 172.21.15.163:0/3543482940 shutdown_connections 2023-10-09T06:31:42.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.212+0000 7f519d886700 1 -- 172.21.15.163:0/3543482940 wait complete. 2023-10-09T06:31:42.218 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.214+0000 7f519d886700 1 Processor -- start 2023-10-09T06:31:42.218 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.214+0000 7f519d886700 1 -- start start 2023-10-09T06:31:42.218 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.214+0000 7f519d886700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51980688d0 0x7f51980aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:42.218 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.214+0000 7f519d886700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5198140350 0x7f51981598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:42.218 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.214+0000 7f519d886700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5198140720 0x7f5198159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:42.218 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.214+0000 7f519d886700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519813cc30 con 0x7f51980688d0 2023-10-09T06:31:42.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.214+0000 7f519d886700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51980c4930 con 0x7f5198140350 2023-10-09T06:31:42.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.214+0000 7f519d886700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51980aff40 con 0x7f5198140720 2023-10-09T06:31:42.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.215+0000 7f5196ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51980688d0 0x7f51980aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:42.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.215+0000 7f5196ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51980688d0 0x7f51980aef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40640/0 (socket says 172.21.15.163:40640) 2023-10-09T06:31:42.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.215+0000 7f5196ffd700 1 -- 172.21.15.163:0/2655740602 learned_addr learned my addr 172.21.15.163:0/2655740602 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:42.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.215+0000 7f518e7fc700 1 --2- 172.21.15.163:0/2655740602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5198140350 0x7f51981598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:42.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.215+0000 7f51977fe700 1 --2- 172.21.15.163:0/2655740602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5198140720 0x7f5198159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:42.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.215+0000 7f5196ffd700 1 -- 172.21.15.163:0/2655740602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5198140720 msgr2=0x7f5198159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:42.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.215+0000 7f5196ffd700 1 --2- 172.21.15.163:0/2655740602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5198140720 0x7f5198159e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.215+0000 7f5196ffd700 1 -- 172.21.15.163:0/2655740602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5198140350 msgr2=0x7f51981598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:42.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.215+0000 7f5196ffd700 1 --2- 172.21.15.163:0/2655740602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5198140350 0x7f51981598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.215+0000 7f5196ffd700 1 -- 172.21.15.163:0/2655740602 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51800094f0 con 0x7f51980688d0 2023-10-09T06:31:42.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.215+0000 7f5196ffd700 1 --2- 172.21.15.163:0/2655740602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51980688d0 0x7f51980aef40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f518800c7f0 tx=0x7f5188007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:42.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.215+0000 7f518f7fe700 1 -- 172.21.15.163:0/2655740602 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5188011590 con 0x7f51980688d0 2023-10-09T06:31:42.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.215+0000 7f518f7fe700 1 -- 172.21.15.163:0/2655740602 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5188011710 con 0x7f51980688d0 2023-10-09T06:31:42.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.215+0000 7f519d886700 1 -- 172.21.15.163:0/2655740602 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51980b0920 con 0x7f51980688d0 2023-10-09T06:31:42.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.215+0000 7f518f7fe700 1 -- 172.21.15.163:0/2655740602 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5188011ad0 con 0x7f51980688d0 2023-10-09T06:31:42.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.215+0000 7f519d886700 1 -- 172.21.15.163:0/2655740602 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51980afc60 con 0x7f51980688d0 2023-10-09T06:31:42.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.216+0000 7f519d886700 1 -- 172.21.15.163:0/2655740602 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f519804c3c0 con 0x7f51980688d0 2023-10-09T06:31:42.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.217+0000 7f518f7fe700 1 -- 172.21.15.163:0/2655740602 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f518801b3b0 con 0x7f51980688d0 2023-10-09T06:31:42.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.217+0000 7f518f7fe700 1 --2- 172.21.15.163:0/2655740602 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5178066f30 0x7f51780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:42.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.217+0000 7f518f7fe700 1 -- 172.21.15.163:0/2655740602 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f5188083a30 con 0x7f51980688d0 2023-10-09T06:31:42.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.217+0000 7f518e7fc700 1 --2- 172.21.15.163:0/2655740602 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5178066f30 0x7f51780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:42.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.218+0000 7f518e7fc700 1 --2- 172.21.15.163:0/2655740602 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5178066f30 0x7f51780693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f51980c7c40 tx=0x7f5180017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:42.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.221+0000 7f518f7fe700 1 -- 172.21.15.163:0/2655740602 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f51880509d0 con 0x7f51980688d0 2023-10-09T06:31:42.402 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.402+0000 7f519d886700 1 -- 172.21.15.163:0/2655740602 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool ls", "detail": "detail", "format": "json"} v 0) v1 -- 0x7f5198140ed0 con 0x7f51980688d0 2023-10-09T06:31:42.403 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.402+0000 7f518f7fe700 1 -- 172.21.15.163:0/2655740602 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool ls", "detail": "detail", "format": "json"}]=0 v34) v1 ==== 97+0+8050 (secure 0 0 0) 0x7f5188017070 con 0x7f51980688d0 2023-10-09T06:31:42.403 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:42.405 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.404+0000 7f519d886700 1 -- 172.21.15.163:0/2655740602 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5178066f30 msgr2=0x7f51780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:42.405 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.404+0000 7f519d886700 1 --2- 172.21.15.163:0/2655740602 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5178066f30 0x7f51780693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f51980c7c40 tx=0x7f5180017000 comp rx=0 tx=0).stop 2023-10-09T06:31:42.405 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.404+0000 7f519d886700 1 -- 172.21.15.163:0/2655740602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51980688d0 msgr2=0x7f51980aef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:42.405 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.404+0000 7f519d886700 1 --2- 172.21.15.163:0/2655740602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51980688d0 0x7f51980aef40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f518800c7f0 tx=0x7f5188007f70 comp rx=0 tx=0).stop 2023-10-09T06:31:42.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.404+0000 7f519d886700 1 -- 172.21.15.163:0/2655740602 shutdown_connections 2023-10-09T06:31:42.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.404+0000 7f519d886700 1 --2- 172.21.15.163:0/2655740602 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5178066f30 0x7f51780693e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.404+0000 7f519d886700 1 --2- 172.21.15.163:0/2655740602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51980688d0 0x7f51980aef40 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.404+0000 7f519d886700 1 --2- 172.21.15.163:0/2655740602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5198140720 0x7f5198159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.404+0000 7f519d886700 1 --2- 172.21.15.163:0/2655740602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5198140350 0x7f51981598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.404+0000 7f519d886700 1 -- 172.21.15.163:0/2655740602 >> 172.21.15.163:0/2655740602 conn(0x7f51980cd670 msgr2=0x7f519813d400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:42.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.404+0000 7f519d886700 1 -- 172.21.15.163:0/2655740602 shutdown_connections 2023-10-09T06:31:42.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.404+0000 7f519d886700 1 -- 172.21.15.163:0/2655740602 wait complete. 2023-10-09T06:31:42.417 INFO:teuthology.orchestra.run.smithi163.stdout:[{"pool_id":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool_id":4,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:31:25.770415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool_id":5,"pool_name":"cephfs_data","create_time":"2023-10-09T06:31:26.980879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool_id":6,"pool_name":"cephfs.volume_0000000000308465.meta","create_time":"2023-10-09T06:31:38.739080+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000308465"}}},{"pool_id":7,"pool_name":"cephfs.volume_0000000000308465.data","create_time":"2023-10-09T06:31:39.721199+0000","flags":163841,"flags_names":"hashpspool,creating,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000308465"}}}] 2023-10-09T06:31:42.417 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs get volume_0000000000308465 --format=json 2023-10-09T06:31:42.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.532+0000 7f1a8a279700 1 Processor -- start 2023-10-09T06:31:42.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.532+0000 7f1a8a279700 1 -- start start 2023-10-09T06:31:42.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.533+0000 7f1a8a279700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a84159c60 0x7f1a8415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:42.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.533+0000 7f1a8a279700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1a8415db10 0x7f1a84069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:42.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.533+0000 7f1a8a279700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a8415a160 con 0x7f1a84159890 2023-10-09T06:31:42.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.533+0000 7f1a8a279700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a840d13a0 con 0x7f1a8415db10 2023-10-09T06:31:42.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.533+0000 7f1a8a279700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8415e140 con 0x7f1a84159c60 2023-10-09T06:31:42.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.533+0000 7f1a88816700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1a8415db10 0x7f1a84069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:42.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.533+0000 7f1a88816700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1a8415db10 0x7f1a84069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40656/0 (socket says 172.21.15.163:40656) 2023-10-09T06:31:42.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.533+0000 7f1a88816700 1 -- 172.21.15.163:0/446648599 learned_addr learned my addr 172.21.15.163:0/446648599 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:42.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.533+0000 7f1a837fe700 1 --2- 172.21.15.163:0/446648599 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a84159c60 0x7f1a8415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:42.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.533+0000 7f1a88816700 1 -- 172.21.15.163:0/446648599 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a84159c60 msgr2=0x7f1a8415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:42.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.533+0000 7f1a88816700 1 --2- 172.21.15.163:0/446648599 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a84159c60 0x7f1a8415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.533+0000 7f1a88816700 1 -- 172.21.15.163:0/446648599 >> v1:172.21.15.181:6789/0 conn(0x7f1a84159890 legacy=0x7f1a8415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:42.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.533+0000 7f1a88816700 1 -- 172.21.15.163:0/446648599 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a8415a3d0 con 0x7f1a8415db10 2023-10-09T06:31:42.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.533+0000 7f1a837fe700 1 --2- 172.21.15.163:0/446648599 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a84159c60 0x7f1a8415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:42.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.533+0000 7f1a88816700 1 --2- 172.21.15.163:0/446648599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a8415db10 0x7f1a84069560 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c009f20 tx=0x7f1a7c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9fbd728e227289 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:42.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.534+0000 7f1a82ffd700 1 -- 172.21.15.163:0/446648599 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c00e070 con 0x7f1a8415db10 2023-10-09T06:31:42.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.534+0000 7f1a82ffd700 1 -- 172.21.15.163:0/446648599 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a7c00ba80 con 0x7f1a8415db10 2023-10-09T06:31:42.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.534+0000 7f1a82ffd700 1 -- 172.21.15.163:0/446648599 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c0092e0 con 0x7f1a8415db10 2023-10-09T06:31:42.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.534+0000 7f1a8a279700 1 -- 172.21.15.163:0/446648599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a8415db10 msgr2=0x7f1a84069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:42.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.534+0000 7f1a8a279700 1 --2- 172.21.15.163:0/446648599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a8415db10 0x7f1a84069560 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c009f20 tx=0x7f1a7c009840 comp rx=0 tx=0).stop 2023-10-09T06:31:42.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.534+0000 7f1a8a279700 1 -- 172.21.15.163:0/446648599 shutdown_connections 2023-10-09T06:31:42.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.534+0000 7f1a8a279700 1 --2- 172.21.15.163:0/446648599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a8415db10 0x7f1a84069560 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.534+0000 7f1a8a279700 1 --2- 172.21.15.163:0/446648599 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a84159c60 0x7f1a8415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.534+0000 7f1a8a279700 1 -- 172.21.15.163:0/446648599 >> 172.21.15.163:0/446648599 conn(0x7f1a84068e20 msgr2=0x7f1a840d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:42.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.534+0000 7f1a8a279700 1 -- 172.21.15.163:0/446648599 shutdown_connections 2023-10-09T06:31:42.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.534+0000 7f1a8a279700 1 -- 172.21.15.163:0/446648599 wait complete. 2023-10-09T06:31:42.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.535+0000 7f1a8a279700 1 Processor -- start 2023-10-09T06:31:42.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.535+0000 7f1a8a279700 1 -- start start 2023-10-09T06:31:42.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.536+0000 7f1a8a279700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a84159890 0x7f1a840ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:42.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.536+0000 7f1a8a279700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a84159c60 0x7f1a84147280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:42.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.536+0000 7f1a8a279700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a8415db10 0x7f1a84153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:42.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.536+0000 7f1a8a279700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a840b4750 con 0x7f1a84159c60 2023-10-09T06:31:42.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.536+0000 7f1a8a279700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8416d660 con 0x7f1a8415db10 2023-10-09T06:31:42.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.536+0000 7f1a8a279700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8416d940 con 0x7f1a84159890 2023-10-09T06:31:42.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.536+0000 7f1a837fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a84159c60 0x7f1a84147280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:42.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.536+0000 7f1a837fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a84159c60 0x7f1a84147280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40672/0 (socket says 172.21.15.163:40672) 2023-10-09T06:31:42.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.536+0000 7f1a837fe700 1 -- 172.21.15.163:0/523334575 learned_addr learned my addr 172.21.15.163:0/523334575 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:42.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.536+0000 7f1a88816700 1 --2- 172.21.15.163:0/523334575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a8415db10 0x7f1a84153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:42.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.536+0000 7f1a83fff700 1 --2- 172.21.15.163:0/523334575 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a84159890 0x7f1a840ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:42.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.537+0000 7f1a837fe700 1 -- 172.21.15.163:0/523334575 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a84159890 msgr2=0x7f1a840ca310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:42.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.537+0000 7f1a837fe700 1 --2- 172.21.15.163:0/523334575 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a84159890 0x7f1a840ca310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.537+0000 7f1a837fe700 1 -- 172.21.15.163:0/523334575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a8415db10 msgr2=0x7f1a84153850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:42.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.537+0000 7f1a837fe700 1 --2- 172.21.15.163:0/523334575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a8415db10 0x7f1a84153850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.537+0000 7f1a837fe700 1 -- 172.21.15.163:0/523334575 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a7c009ae0 con 0x7f1a84159c60 2023-10-09T06:31:42.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.537+0000 7f1a837fe700 1 --2- 172.21.15.163:0/523334575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a84159c60 0x7f1a84147280 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1a7800e8a0 tx=0x7f1a78008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:42.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.537+0000 7f1a6bfff700 1 -- 172.21.15.163:0/523334575 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a78013410 con 0x7f1a84159c60 2023-10-09T06:31:42.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.537+0000 7f1a6bfff700 1 -- 172.21.15.163:0/523334575 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a78013570 con 0x7f1a84159c60 2023-10-09T06:31:42.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.537+0000 7f1a8a279700 1 -- 172.21.15.163:0/523334575 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a8416dc20 con 0x7f1a84159c60 2023-10-09T06:31:42.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.537+0000 7f1a6bfff700 1 -- 172.21.15.163:0/523334575 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a78013930 con 0x7f1a84159c60 2023-10-09T06:31:42.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.537+0000 7f1a8a279700 1 -- 172.21.15.163:0/523334575 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a841477c0 con 0x7f1a84159c60 2023-10-09T06:31:42.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.538+0000 7f1a8a279700 1 -- 172.21.15.163:0/523334575 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a8404c3c0 con 0x7f1a84159c60 2023-10-09T06:31:42.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.538+0000 7f1a6bfff700 1 -- 172.21.15.163:0/523334575 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1a78013ca0 con 0x7f1a84159c60 2023-10-09T06:31:42.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.539+0000 7f1a6bfff700 1 --2- 172.21.15.163:0/523334575 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a70066f30 0x7f1a700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:42.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.539+0000 7f1a6bfff700 1 -- 172.21.15.163:0/523334575 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f1a78084550 con 0x7f1a84159c60 2023-10-09T06:31:42.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.539+0000 7f1a83fff700 1 --2- 172.21.15.163:0/523334575 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a70066f30 0x7f1a700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:42.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.539+0000 7f1a83fff700 1 --2- 172.21.15.163:0/523334575 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a70066f30 0x7f1a700693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1a74003a80 tx=0x7f1a74007ec0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:42.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.543+0000 7f1a6bfff700 1 -- 172.21.15.163:0/523334575 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a78054ed0 con 0x7f1a84159c60 2023-10-09T06:31:42.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.776+0000 7f1a8a279700 1 -- 172.21.15.163:0/523334575 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs get", "fs_name": "volume_0000000000308465", "format": "json"} v 0) v1 -- 0x7f1a8415b0c0 con 0x7f1a84159c60 2023-10-09T06:31:42.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.777+0000 7f1a6bfff700 1 -- 172.21.15.163:0/523334575 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs get", "fs_name": "volume_0000000000308465", "format": "json"}]=0 v21) v1 ==== 110+0+1847 (secure 0 0 0) 0x7f1a78054a60 con 0x7f1a84159c60 2023-10-09T06:31:42.778 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:42.778 INFO:teuthology.orchestra.run.smithi163.stdout:{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:31:40.710079+0000","modified":"2023-10-09T06:31:42.712009+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4932},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4932":{"gid":4932,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.163:6835/2241937708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2241937708},{"type":"v1","addr":"172.21.15.163:6835","nonce":2241937708}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000308465","balancer":"","standby_count_wanted":1},"id":3} 2023-10-09T06:31:42.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.779+0000 7f1a8a279700 1 -- 172.21.15.163:0/523334575 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a70066f30 msgr2=0x7f1a700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:42.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.779+0000 7f1a8a279700 1 --2- 172.21.15.163:0/523334575 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a70066f30 0x7f1a700693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1a74003a80 tx=0x7f1a74007ec0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.779+0000 7f1a8a279700 1 -- 172.21.15.163:0/523334575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a84159c60 msgr2=0x7f1a84147280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:42.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.779+0000 7f1a8a279700 1 --2- 172.21.15.163:0/523334575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a84159c60 0x7f1a84147280 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1a7800e8a0 tx=0x7f1a78008040 comp rx=0 tx=0).stop 2023-10-09T06:31:42.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.779+0000 7f1a8a279700 1 -- 172.21.15.163:0/523334575 shutdown_connections 2023-10-09T06:31:42.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.779+0000 7f1a8a279700 1 --2- 172.21.15.163:0/523334575 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a70066f30 0x7f1a700693e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.779+0000 7f1a8a279700 1 --2- 172.21.15.163:0/523334575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a8415db10 0x7f1a84153850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.779+0000 7f1a8a279700 1 --2- 172.21.15.163:0/523334575 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a84159890 0x7f1a840ca310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.779+0000 7f1a8a279700 1 --2- 172.21.15.163:0/523334575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a84159c60 0x7f1a84147280 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.779+0000 7f1a8a279700 1 -- 172.21.15.163:0/523334575 >> 172.21.15.163:0/523334575 conn(0x7f1a84068e20 msgr2=0x7f1a8415b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:42.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.779+0000 7f1a8a279700 1 -- 172.21.15.163:0/523334575 shutdown_connections 2023-10-09T06:31:42.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.779+0000 7f1a8a279700 1 -- 172.21.15.163:0/523334575 wait complete. 2023-10-09T06:31:42.792 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000000308465 --yes-i-really-mean-it 2023-10-09T06:31:42.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.910+0000 7fdf2de27700 1 Processor -- start 2023-10-09T06:31:42.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.910+0000 7fdf2de27700 1 -- start start 2023-10-09T06:31:42.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.910+0000 7fdf2de27700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf2806c0a0 0x7fdf280caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:42.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.910+0000 7fdf2de27700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf2815d7e0 con 0x7fdf2815df50 2023-10-09T06:31:42.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.910+0000 7fdf2de27700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf2815dac0 con 0x7fdf2806bcd0 2023-10-09T06:31:42.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.910+0000 7fdf2de27700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2815aa30 con 0x7fdf2806c0a0 2023-10-09T06:31:42.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.911+0000 7fdf27fff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fdf2815df50 0x7fdf280c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:49762/0 (socket says 172.21.15.163:49762) 2023-10-09T06:31:42.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.911+0000 7fdf27fff700 1 -- 172.21.15.163:0/1952595325 learned_addr learned my addr 172.21.15.163:0/1952595325 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:42.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.911+0000 7fdf267fc700 1 -- 172.21.15.163:0/1952595325 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 211775606 0 0) 0x7fdf2815d7e0 con 0x7fdf2815df50 2023-10-09T06:31:42.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.911+0000 7fdf267fc700 1 -- 172.21.15.163:0/1952595325 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf10003570 con 0x7fdf2815df50 2023-10-09T06:31:42.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.911+0000 7fdf26ffd700 1 --2- 172.21.15.163:0/1952595325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf2806c0a0 0x7fdf280caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:42.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.911+0000 7fdf267fc700 1 -- 172.21.15.163:0/1952595325 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3794857675 0 0) 0x7fdf10003570 con 0x7fdf2815df50 2023-10-09T06:31:42.919 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.911+0000 7fdf267fc700 1 -- 172.21.15.163:0/1952595325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf2806c0a0 msgr2=0x7fdf280caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:42.919 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.911+0000 7fdf267fc700 1 --2- 172.21.15.163:0/1952595325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf2806c0a0 0x7fdf280caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.919 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.911+0000 7fdf267fc700 1 -- 172.21.15.163:0/1952595325 >> v1:172.21.15.181:6789/0 conn(0x7fdf2806bcd0 legacy=0x7fdf2815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:42.919 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.911+0000 7fdf267fc700 1 -- 172.21.15.163:0/1952595325 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2815ad10 con 0x7fdf2815df50 2023-10-09T06:31:42.919 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.911+0000 7fdf267fc700 1 -- 172.21.15.163:0/1952595325 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fdf20001470 con 0x7fdf2815df50 2023-10-09T06:31:42.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.911+0000 7fdf267fc700 1 -- 172.21.15.163:0/1952595325 >> v1:172.21.15.163:6789/0 conn(0x7fdf2815df50 legacy=0x7fdf280c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:42.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.911+0000 7fdf267fc700 1 --2- 172.21.15.163:0/1952595325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf2806c0a0 0x7fdf10003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:42.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.911+0000 7fdf267fc700 1 --2- 172.21.15.163:0/1952595325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf10004010 0x7fdf10004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:42.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.911+0000 7fdf267fc700 1 --2- 172.21.15.163:0/1952595325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf10004d50 0x7fdf100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:42.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.911+0000 7fdf267fc700 1 -- 172.21.15.163:0/1952595325 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2815aa30 con 0x7fdf10004010 2023-10-09T06:31:42.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.911+0000 7fdf267fc700 1 -- 172.21.15.163:0/1952595325 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1000bc70 con 0x7fdf10004d50 2023-10-09T06:31:42.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.911+0000 7fdf267fc700 1 -- 172.21.15.163:0/1952595325 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1000be30 con 0x7fdf2806c0a0 2023-10-09T06:31:42.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.911+0000 7fdf26ffd700 1 --2- 172.21.15.163:0/1952595325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf10004010 0x7fdf10004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:42.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.911+0000 7fdf277fe700 1 --2- 172.21.15.163:0/1952595325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf2806c0a0 0x7fdf10003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:42.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.911+0000 7fdf27fff700 1 --2- 172.21.15.163:0/1952595325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf10004d50 0x7fdf100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:42.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.911+0000 7fdf26ffd700 1 -- 172.21.15.163:0/1952595325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf2806c0a0 msgr2=0x7fdf10003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:42.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.911+0000 7fdf26ffd700 1 --2- 172.21.15.163:0/1952595325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf2806c0a0 0x7fdf10003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.911+0000 7fdf26ffd700 1 -- 172.21.15.163:0/1952595325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf10004d50 msgr2=0x7fdf100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:42.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.911+0000 7fdf26ffd700 1 --2- 172.21.15.163:0/1952595325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf10004d50 0x7fdf100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.911+0000 7fdf26ffd700 1 -- 172.21.15.163:0/1952595325 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf10003570 con 0x7fdf10004010 2023-10-09T06:31:42.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.911+0000 7fdf27fff700 1 --2- 172.21.15.163:0/1952595325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf10004d50 0x7fdf100051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:42.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.912+0000 7fdf26ffd700 1 --2- 172.21.15.163:0/1952595325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf10004010 0x7fdf10004810 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c006f00 tx=0x7fdf1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe60ba7e87a1aac server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:42.923 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.912+0000 7fdf267fc700 1 -- 172.21.15.163:0/1952595325 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c00e710 con 0x7fdf10004010 2023-10-09T06:31:42.923 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.912+0000 7fdf267fc700 1 -- 172.21.15.163:0/1952595325 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1c00e870 con 0x7fdf10004010 2023-10-09T06:31:42.923 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.912+0000 7fdf267fc700 1 -- 172.21.15.163:0/1952595325 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c00ec30 con 0x7fdf10004010 2023-10-09T06:31:42.923 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.912+0000 7fdf2de27700 1 -- 172.21.15.163:0/1952595325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf10004010 msgr2=0x7fdf10004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:42.923 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.912+0000 7fdf2de27700 1 --2- 172.21.15.163:0/1952595325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf10004010 0x7fdf10004810 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c006f00 tx=0x7fdf1c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.923 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.912+0000 7fdf27fff700 1 -- 172.21.15.163:0/1952595325 reap_dead start 2023-10-09T06:31:42.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.912+0000 7fdf2de27700 1 -- 172.21.15.163:0/1952595325 shutdown_connections 2023-10-09T06:31:42.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.912+0000 7fdf2de27700 1 -- 172.21.15.163:0/1952595325 >> 172.21.15.163:0/1952595325 conn(0x7fdf280ce600 msgr2=0x7fdf280cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:42.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.912+0000 7fdf2de27700 1 -- 172.21.15.163:0/1952595325 shutdown_connections 2023-10-09T06:31:42.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.912+0000 7fdf2de27700 1 -- 172.21.15.163:0/1952595325 wait complete. 2023-10-09T06:31:42.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.914+0000 7fdf2de27700 1 Processor -- start 2023-10-09T06:31:42.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.914+0000 7fdf2de27700 1 -- start start 2023-10-09T06:31:42.925 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.914+0000 7fdf2de27700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf10004010 0x7fdf280b3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:42.925 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.914+0000 7fdf2de27700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf280c6960 0x7fdf281567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:42.925 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.914+0000 7fdf2de27700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf28069c70 0x7fdf2806a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:42.925 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.914+0000 7fdf2de27700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf280c56b0 con 0x7fdf28069c70 2023-10-09T06:31:42.925 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.914+0000 7fdf2de27700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2806a610 con 0x7fdf10004010 2023-10-09T06:31:42.925 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.914+0000 7fdf2de27700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf280c64d0 con 0x7fdf280c6960 2023-10-09T06:31:42.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.914+0000 7fdf27fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf28069c70 0x7fdf2806a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:42.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.914+0000 7fdf27fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf28069c70 0x7fdf2806a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40688/0 (socket says 172.21.15.163:40688) 2023-10-09T06:31:42.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.914+0000 7fdf27fff700 1 -- 172.21.15.163:0/2473750787 learned_addr learned my addr 172.21.15.163:0/2473750787 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:42.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.914+0000 7fdf277fe700 1 --2- 172.21.15.163:0/2473750787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf10004010 0x7fdf280b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:42.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.914+0000 7fdf26ffd700 1 --2- 172.21.15.163:0/2473750787 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf280c6960 0x7fdf281567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:42.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.915+0000 7fdf27fff700 1 -- 172.21.15.163:0/2473750787 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf280c6960 msgr2=0x7fdf281567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:42.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.915+0000 7fdf27fff700 1 --2- 172.21.15.163:0/2473750787 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf280c6960 0x7fdf281567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.915+0000 7fdf27fff700 1 -- 172.21.15.163:0/2473750787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf10004010 msgr2=0x7fdf280b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:42.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.915+0000 7fdf27fff700 1 --2- 172.21.15.163:0/2473750787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf10004010 0x7fdf280b3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:42.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.915+0000 7fdf27fff700 1 -- 172.21.15.163:0/2473750787 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf280c58c0 con 0x7fdf28069c70 2023-10-09T06:31:42.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.915+0000 7fdf27fff700 1 --2- 172.21.15.163:0/2473750787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf28069c70 0x7fdf2806a0d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdf20000fd0 tx=0x7fdf20009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:42.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.915+0000 7fdf0f7fe700 1 -- 172.21.15.163:0/2473750787 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf20007e10 con 0x7fdf28069c70 2023-10-09T06:31:42.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.915+0000 7fdf0f7fe700 1 -- 172.21.15.163:0/2473750787 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf20007360 con 0x7fdf28069c70 2023-10-09T06:31:42.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.915+0000 7fdf2de27700 1 -- 172.21.15.163:0/2473750787 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf280c5ba0 con 0x7fdf28069c70 2023-10-09T06:31:42.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.915+0000 7fdf0f7fe700 1 -- 172.21.15.163:0/2473750787 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf20007700 con 0x7fdf28069c70 2023-10-09T06:31:42.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.915+0000 7fdf2de27700 1 -- 172.21.15.163:0/2473750787 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf280c6160 con 0x7fdf28069c70 2023-10-09T06:31:42.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.916+0000 7fdf2de27700 1 -- 172.21.15.163:0/2473750787 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf280c6660 con 0x7fdf28069c70 2023-10-09T06:31:42.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.916+0000 7fdf0f7fe700 1 -- 172.21.15.163:0/2473750787 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdf20004a40 con 0x7fdf28069c70 2023-10-09T06:31:42.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.917+0000 7fdf0f7fe700 1 --2- 172.21.15.163:0/2473750787 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdf14066f30 0x7fdf140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:42.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.917+0000 7fdf0f7fe700 1 -- 172.21.15.163:0/2473750787 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fdf20084600 con 0x7fdf28069c70 2023-10-09T06:31:42.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.917+0000 7fdf277fe700 1 --2- 172.21.15.163:0/2473750787 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdf14066f30 0x7fdf140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:42.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.917+0000 7fdf277fe700 1 --2- 172.21.15.163:0/2473750787 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdf14066f30 0x7fdf140693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdf18002f20 tx=0x7fdf18001640 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:42.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:42.921+0000 7fdf0f7fe700 1 -- 172.21.15.163:0/2473750787 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdf20051570 con 0x7fdf28069c70 2023-10-09T06:31:42.946 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:31:42.948 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:31:43.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:43.138+0000 7fdf2de27700 1 -- 172.21.15.163:0/2473750787 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "volume_0000000000308465", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf2804c3c0 con 0x7fdf14066f30 2023-10-09T06:31:43.143 INFO:tasks.ceph.mon.a.smithi163.stderr:2023-10-09T06:31:43.142+0000 7ff251c04700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-09T06:31:43.147 INFO:tasks.ceph.mds.c.smithi163.stderr: -1133> 2023-10-09T06:31:20.737+0000 7fd74dbe5700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-09T06:31:43.169 INFO:tasks.ceph.mds.c.smithi163.stdout:starting mds.c at 2023-10-09T06:31:46.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.170+0000 7fdf0f7fe700 1 -- 172.21.15.163:0/2473750787 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+110 (secure 0 0 0) 0x7fdf2804c3c0 con 0x7fdf14066f30 2023-10-09T06:31:46.171 INFO:teuthology.orchestra.run.smithi163.stdout:metadata pool: cephfs.volume_0000000000308465.meta data pool: ['cephfs.volume_0000000000308465.data'] removed 2023-10-09T06:31:46.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.172+0000 7fdf2de27700 1 -- 172.21.15.163:0/2473750787 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdf14066f30 msgr2=0x7fdf140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:46.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.172+0000 7fdf2de27700 1 --2- 172.21.15.163:0/2473750787 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdf14066f30 0x7fdf140693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdf18002f20 tx=0x7fdf18001640 comp rx=0 tx=0).stop 2023-10-09T06:31:46.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.173+0000 7fdf2de27700 1 -- 172.21.15.163:0/2473750787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf28069c70 msgr2=0x7fdf2806a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:46.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.173+0000 7fdf2de27700 1 --2- 172.21.15.163:0/2473750787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf28069c70 0x7fdf2806a0d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdf20000fd0 tx=0x7fdf20009040 comp rx=0 tx=0).stop 2023-10-09T06:31:46.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.173+0000 7fdf2de27700 1 -- 172.21.15.163:0/2473750787 shutdown_connections 2023-10-09T06:31:46.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.173+0000 7fdf2de27700 1 --2- 172.21.15.163:0/2473750787 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdf14066f30 0x7fdf140693e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:46.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.173+0000 7fdf2de27700 1 --2- 172.21.15.163:0/2473750787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf28069c70 0x7fdf2806a0d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:46.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.173+0000 7fdf2de27700 1 --2- 172.21.15.163:0/2473750787 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf10004010 0x7fdf280b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:46.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.173+0000 7fdf2de27700 1 --2- 172.21.15.163:0/2473750787 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf280c6960 0x7fdf281567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:46.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.173+0000 7fdf2de27700 1 -- 172.21.15.163:0/2473750787 >> 172.21.15.163:0/2473750787 conn(0x7fdf280ce600 msgr2=0x7fdf2815b590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:46.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.173+0000 7fdf2de27700 1 -- 172.21.15.163:0/2473750787 shutdown_connections 2023-10-09T06:31:46.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.173+0000 7fdf2de27700 1 -- 172.21.15.163:0/2473750787 wait complete. 2023-10-09T06:31:46.188 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-09T06:31:46.189 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:31:46.230 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:31:46.274 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-09T06:31:46.274 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-09T06:31:46.274 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:31:46.275 DEBUG:teuthology.orchestra.run.smithi163:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-09T06:31:46.317 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi163.front.sepia.ceph.com 2023-10-09T06:31:46.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:46.317 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-09T06:31:46.373 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-09T06:31:46.373 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:31:46.374 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-09T06:31:46.415 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-09T06:31:46.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:31:46.416 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-09T06:31:46.472 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:31:46.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.587+0000 7ffb0f60d700 1 Processor -- start 2023-10-09T06:31:46.588 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.587+0000 7ffb0f60d700 1 -- start start 2023-10-09T06:31:46.588 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.588+0000 7ffb0f60d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffb08159840 0x7ffb080cb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:46.588 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.588+0000 7ffb0f60d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb080cba70 0x7ffb080694a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:46.588 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.588+0000 7ffb0f60d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb080cfb30 con 0x7ffb08159c10 2023-10-09T06:31:46.588 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.588+0000 7ffb0f60d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb080cbf10 con 0x7ffb08159840 2023-10-09T06:31:46.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.588+0000 7ffb0f60d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb080cf4e0 con 0x7ffb080cba70 2023-10-09T06:31:46.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.588+0000 7ffb0cba8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb080cba70 0x7ffb080694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:46.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.588+0000 7ffb0d3a9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffb08159840 0x7ffb080cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:46.591 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.588+0000 7ffb0cba8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb080cba70 0x7ffb080694a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:56282/0 (socket says 172.21.15.163:56282) 2023-10-09T06:31:46.591 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.588+0000 7ffb0cba8700 1 -- 172.21.15.163:0/1542373581 learned_addr learned my addr 172.21.15.163:0/1542373581 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:46.591 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.588+0000 7ffb0d3a9700 1 -- 172.21.15.163:0/1542373581 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb080cba70 msgr2=0x7ffb080694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:46.591 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.588+0000 7ffb0d3a9700 1 --2- 172.21.15.163:0/1542373581 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb080cba70 0x7ffb080694a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:46.591 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.588+0000 7ffb0d3a9700 1 -- 172.21.15.163:0/1542373581 >> v1:172.21.15.181:6789/0 conn(0x7ffb08159c10 legacy=0x7ffb080ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:46.591 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.588+0000 7ffb0d3a9700 1 -- 172.21.15.163:0/1542373581 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb080cf7c0 con 0x7ffb08159840 2023-10-09T06:31:46.591 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.588+0000 7ffafffff700 1 -- 172.21.15.163:0/1542373581 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2144358997 0 0) 0x7ffb080cfb30 con 0x7ffb08159c10 2023-10-09T06:31:46.591 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.588+0000 7ffb0cba8700 1 --2- 172.21.15.163:0/1542373581 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb080cba70 0x7ffb080694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:31:46.592 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.589+0000 7ffb0d3a9700 1 --2- 172.21.15.163:0/1542373581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb08159840 0x7ffb080cb530 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffb040098e0 tx=0x7ffb040091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71ba33b6a84b1327 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:46.592 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.589+0000 7ffafffff700 1 -- 172.21.15.163:0/1542373581 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0400b640 con 0x7ffb08159840 2023-10-09T06:31:46.592 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.589+0000 7ffafffff700 1 -- 172.21.15.163:0/1542373581 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0400b7a0 con 0x7ffb08159840 2023-10-09T06:31:46.592 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.589+0000 7ffafffff700 1 -- 172.21.15.163:0/1542373581 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0400bb40 con 0x7ffb08159840 2023-10-09T06:31:46.592 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.589+0000 7ffb0f60d700 1 -- 172.21.15.163:0/1542373581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb08159840 msgr2=0x7ffb080cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:46.593 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.589+0000 7ffb0f60d700 1 --2- 172.21.15.163:0/1542373581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb08159840 0x7ffb080cb530 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffb040098e0 tx=0x7ffb040091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:46.593 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.589+0000 7ffb0f60d700 1 -- 172.21.15.163:0/1542373581 shutdown_connections 2023-10-09T06:31:46.593 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.589+0000 7ffb0f60d700 1 --2- 172.21.15.163:0/1542373581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb08159840 0x7ffb080cb530 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:46.593 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.589+0000 7ffb0f60d700 1 --2- 172.21.15.163:0/1542373581 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb080cba70 0x7ffb080694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:46.593 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.589+0000 7ffb0f60d700 1 -- 172.21.15.163:0/1542373581 >> 172.21.15.163:0/1542373581 conn(0x7ffb08068e10 msgr2=0x7ffb080d0760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:46.593 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.589+0000 7ffb0f60d700 1 -- 172.21.15.163:0/1542373581 shutdown_connections 2023-10-09T06:31:46.593 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.589+0000 7ffb0f60d700 1 -- 172.21.15.163:0/1542373581 wait complete. 2023-10-09T06:31:46.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.590+0000 7ffb0f60d700 1 Processor -- start 2023-10-09T06:31:46.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.590+0000 7ffb0f60d700 1 -- start start 2023-10-09T06:31:46.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.591+0000 7ffb0f60d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb080cba70 0x7ffb0814f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:46.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.591+0000 7ffb0f60d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb08159840 0x7ffb080c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:46.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.591+0000 7ffb0f60d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb08159c10 0x7ffb08157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:46.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.591+0000 7ffb0f60d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb080ca520 con 0x7ffb08159c10 2023-10-09T06:31:46.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.591+0000 7ffb0f60d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0814f820 con 0x7ffb08159840 2023-10-09T06:31:46.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.591+0000 7ffb0f60d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb080c9910 con 0x7ffb080cba70 2023-10-09T06:31:46.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.591+0000 7ffb0dbaa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb08159c10 0x7ffb08157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:46.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.591+0000 7ffb0dbaa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb08159c10 0x7ffb08157b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40728/0 (socket says 172.21.15.163:40728) 2023-10-09T06:31:46.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.591+0000 7ffb0dbaa700 1 -- 172.21.15.163:0/751181657 learned_addr learned my addr 172.21.15.163:0/751181657 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:46.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.592+0000 7ffb0cba8700 1 --2- 172.21.15.163:0/751181657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb08159840 0x7ffb080c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:46.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.592+0000 7ffb0d3a9700 1 --2- 172.21.15.163:0/751181657 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb080cba70 0x7ffb0814f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:46.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.592+0000 7ffb0dbaa700 1 -- 172.21.15.163:0/751181657 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb080cba70 msgr2=0x7ffb0814f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:46.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.592+0000 7ffb0dbaa700 1 --2- 172.21.15.163:0/751181657 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb080cba70 0x7ffb0814f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:46.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.592+0000 7ffb0dbaa700 1 -- 172.21.15.163:0/751181657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb08159840 msgr2=0x7ffb080c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:46.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.592+0000 7ffb0dbaa700 1 --2- 172.21.15.163:0/751181657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb08159840 0x7ffb080c8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:46.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.592+0000 7ffb0dbaa700 1 -- 172.21.15.163:0/751181657 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf80094f0 con 0x7ffb08159c10 2023-10-09T06:31:46.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.592+0000 7ffb0cba8700 1 --2- 172.21.15.163:0/751181657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb08159840 0x7ffb080c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:46.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.592+0000 7ffb0dbaa700 1 --2- 172.21.15.163:0/751181657 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb08159c10 0x7ffb08157b20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000c890 tx=0x7ffb00001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:46.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.592+0000 7ffafd7fa700 1 -- 172.21.15.163:0/751181657 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb00017070 con 0x7ffb08159c10 2023-10-09T06:31:46.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.592+0000 7ffafd7fa700 1 -- 172.21.15.163:0/751181657 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb00006ba0 con 0x7ffb08159c10 2023-10-09T06:31:46.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.592+0000 7ffb0f60d700 1 -- 172.21.15.163:0/751181657 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb04009420 con 0x7ffb08159c10 2023-10-09T06:31:46.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.592+0000 7ffafd7fa700 1 -- 172.21.15.163:0/751181657 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb00012630 con 0x7ffb08159c10 2023-10-09T06:31:46.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.592+0000 7ffb0f60d700 1 -- 172.21.15.163:0/751181657 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb080c9ed0 con 0x7ffb08159c10 2023-10-09T06:31:46.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.593+0000 7ffb0f60d700 1 -- 172.21.15.163:0/751181657 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb08169ae0 con 0x7ffb08159c10 2023-10-09T06:31:46.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.593+0000 7ffafd7fa700 1 -- 172.21.15.163:0/751181657 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffb00012a60 con 0x7ffb08159c10 2023-10-09T06:31:46.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.594+0000 7ffafd7fa700 1 --2- 172.21.15.163:0/751181657 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffaec066f30 0x7ffaec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:46.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.594+0000 7ffafd7fa700 1 -- 172.21.15.163:0/751181657 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5791+0+0 (secure 0 0 0) 0x7ffb000842d0 con 0x7ffb08159c10 2023-10-09T06:31:46.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.594+0000 7ffb0d3a9700 1 --2- 172.21.15.163:0/751181657 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffaec066f30 0x7ffaec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:46.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.594+0000 7ffb0d3a9700 1 --2- 172.21.15.163:0/751181657 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffaec066f30 0x7ffaec0693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffb040093f0 tx=0x7ffb04017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:46.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.598+0000 7ffafd7fa700 1 -- 172.21.15.163:0/751181657 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffb08169ae0 con 0x7ffb08159c10 2023-10-09T06:31:46.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.827+0000 7ffb0f60d700 1 -- 172.21.15.163:0/751181657 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb08169ae0 con 0x7ffb08159c10 2023-10-09T06:31:46.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.827+0000 7ffafd7fa700 1 -- 172.21.15.163:0/751181657 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4609 (secure 0 0 0) 0x7ffb08169ae0 con 0x7ffb08159c10 2023-10-09T06:31:46.828 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:46.828 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4622,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2368375038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2368375038},{"type":"v1","addr":"172.21.15.181:6836","nonce":2368375038}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4929,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/4112854733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4112854733},{"type":"v1","addr":"172.21.15.163:6834","nonce":4112854733}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5202,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2200503829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2200503829},{"type":"v1","addr":"172.21.15.163:6835","nonce":2200503829}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:31:28.011264+0000","modified":"2023-10-09T06:31:31.660915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4944},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4944":{"gid":4944,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2708262268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2708262268},{"type":"v1","addr":"172.21.15.181:6837","nonce":2708262268}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-09T06:31:46.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.829+0000 7ffb0f60d700 1 -- 172.21.15.163:0/751181657 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffaec066f30 msgr2=0x7ffaec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:46.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.829+0000 7ffb0f60d700 1 --2- 172.21.15.163:0/751181657 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffaec066f30 0x7ffaec0693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffb040093f0 tx=0x7ffb04017000 comp rx=0 tx=0).stop 2023-10-09T06:31:46.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.829+0000 7ffb0f60d700 1 -- 172.21.15.163:0/751181657 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb08159c10 msgr2=0x7ffb08157b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:46.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.829+0000 7ffb0f60d700 1 --2- 172.21.15.163:0/751181657 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb08159c10 0x7ffb08157b20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000c890 tx=0x7ffb00001670 comp rx=0 tx=0).stop 2023-10-09T06:31:46.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.829+0000 7ffb0f60d700 1 -- 172.21.15.163:0/751181657 shutdown_connections 2023-10-09T06:31:46.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.829+0000 7ffb0f60d700 1 --2- 172.21.15.163:0/751181657 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffaec066f30 0x7ffaec0693e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:46.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.829+0000 7ffb0f60d700 1 --2- 172.21.15.163:0/751181657 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb08159c10 0x7ffb08157b20 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:46.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.829+0000 7ffb0f60d700 1 --2- 172.21.15.163:0/751181657 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb080cba70 0x7ffb0814f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:46.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.829+0000 7ffb0f60d700 1 --2- 172.21.15.163:0/751181657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb08159840 0x7ffb080c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:46.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.829+0000 7ffb0f60d700 1 -- 172.21.15.163:0/751181657 >> 172.21.15.163:0/751181657 conn(0x7ffb08068e10 msgr2=0x7ffb080cd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:46.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.829+0000 7ffb0f60d700 1 -- 172.21.15.163:0/751181657 shutdown_connections 2023-10-09T06:31:46.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.829+0000 7ffb0f60d700 1 -- 172.21.15.163:0/751181657 wait complete. 2023-10-09T06:31:46.831 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 23 2023-10-09T06:31:46.842 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:31:46.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.955+0000 7f89d2ccc700 1 Processor -- start 2023-10-09T06:31:46.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.956+0000 7f89d2ccc700 1 -- start start 2023-10-09T06:31:46.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.956+0000 7f89d2ccc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f89cc1574d0 0x7f89cc0cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:46.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.956+0000 7f89d2ccc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f89cc06c0f0 0x7f89cc0b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:46.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.956+0000 7f89d2ccc700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89cc0b16a0 con 0x7f89cc0cc3c0 2023-10-09T06:31:46.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.956+0000 7f89d2ccc700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89cc0b1c60 con 0x7f89cc1574d0 2023-10-09T06:31:46.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.956+0000 7f89d2ccc700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89cc1578a0 con 0x7f89cc06c0f0 2023-10-09T06:31:46.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.956+0000 7f89d1cca700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f89cc1574d0 0x7f89cc0cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:46.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.957+0000 7f89d1cca700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f89cc1574d0 0x7f89cc0cd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40744/0 (socket says 172.21.15.163:40744) 2023-10-09T06:31:46.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.957+0000 7f89d1cca700 1 -- 172.21.15.163:0/1863055714 learned_addr learned my addr 172.21.15.163:0/1863055714 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:46.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.957+0000 7f89d24cb700 1 --2- 172.21.15.163:0/1863055714 >> v2:172.21.15.181:3300/0 conn(0x7f89cc06c0f0 0x7f89cc0b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:46.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.957+0000 7f89d1cca700 1 -- 172.21.15.163:0/1863055714 >> v2:172.21.15.181:3300/0 conn(0x7f89cc06c0f0 msgr2=0x7f89cc0b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:46.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.957+0000 7f89d1cca700 1 --2- 172.21.15.163:0/1863055714 >> v2:172.21.15.181:3300/0 conn(0x7f89cc06c0f0 0x7f89cc0b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:46.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.957+0000 7f89d1cca700 1 -- 172.21.15.163:0/1863055714 >> v1:172.21.15.181:6789/0 conn(0x7f89cc0cc3c0 legacy=0x7f89cc06b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:46.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.957+0000 7f89d1cca700 1 -- 172.21.15.163:0/1863055714 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89cc0cdb70 con 0x7f89cc1574d0 2023-10-09T06:31:46.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.957+0000 7f89d1cca700 1 --2- 172.21.15.163:0/1863055714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89cc1574d0 0x7f89cc0cd270 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f89cc071300 tx=0x7f89c8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474151fe05d7fd1c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:46.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.957+0000 7f89d0cc8700 1 -- 172.21.15.163:0/1863055714 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c8005c10 con 0x7f89cc1574d0 2023-10-09T06:31:46.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.957+0000 7f89d0cc8700 1 -- 172.21.15.163:0/1863055714 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c8005d70 con 0x7f89cc1574d0 2023-10-09T06:31:46.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.957+0000 7f89d0cc8700 1 -- 172.21.15.163:0/1863055714 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c800b840 con 0x7f89cc1574d0 2023-10-09T06:31:46.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.958+0000 7f89d2ccc700 1 -- 172.21.15.163:0/1863055714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89cc1574d0 msgr2=0x7f89cc0cd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:46.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.958+0000 7f89d2ccc700 1 --2- 172.21.15.163:0/1863055714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89cc1574d0 0x7f89cc0cd270 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f89cc071300 tx=0x7f89c8009220 comp rx=0 tx=0).stop 2023-10-09T06:31:46.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.958+0000 7f89d2ccc700 1 -- 172.21.15.163:0/1863055714 shutdown_connections 2023-10-09T06:31:46.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.958+0000 7f89d2ccc700 1 --2- 172.21.15.163:0/1863055714 >> v2:172.21.15.181:3300/0 conn(0x7f89cc06c0f0 0x7f89cc0b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:46.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.958+0000 7f89d2ccc700 1 --2- 172.21.15.163:0/1863055714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89cc1574d0 0x7f89cc0cd270 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:46.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.958+0000 7f89d2ccc700 1 -- 172.21.15.163:0/1863055714 >> 172.21.15.163:0/1863055714 conn(0x7f89cc0ce430 msgr2=0x7f89cc0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:46.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.958+0000 7f89d2ccc700 1 -- 172.21.15.163:0/1863055714 shutdown_connections 2023-10-09T06:31:46.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.958+0000 7f89d2ccc700 1 -- 172.21.15.163:0/1863055714 wait complete. 2023-10-09T06:31:46.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.959+0000 7f89d2ccc700 1 Processor -- start 2023-10-09T06:31:46.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.959+0000 7f89d2ccc700 1 -- start start 2023-10-09T06:31:46.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.959+0000 7f89d2ccc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89cc06c0f0 0x7f89cc152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:46.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.959+0000 7f89d2ccc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89cc0cc3c0 0x7f89cc15a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:46.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.959+0000 7f89d2ccc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89cc1574d0 0x7f89cc150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:46.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.959+0000 7f89d2ccc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc15b1c0 con 0x7f89cc0cc3c0 2023-10-09T06:31:46.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.959+0000 7f89d2ccc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc15b4a0 con 0x7f89cc06c0f0 2023-10-09T06:31:46.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.959+0000 7f89d2ccc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc15b780 con 0x7f89cc1574d0 2023-10-09T06:31:46.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.960+0000 7f89d14c9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89cc0cc3c0 0x7f89cc15a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:46.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.960+0000 7f89d14c9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89cc0cc3c0 0x7f89cc15a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40750/0 (socket says 172.21.15.163:40750) 2023-10-09T06:31:46.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.960+0000 7f89d14c9700 1 -- 172.21.15.163:0/1144731269 learned_addr learned my addr 172.21.15.163:0/1144731269 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:46.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.960+0000 7f89d1cca700 1 --2- 172.21.15.163:0/1144731269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89cc06c0f0 0x7f89cc152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:46.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.960+0000 7f89d24cb700 1 --2- 172.21.15.163:0/1144731269 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89cc1574d0 0x7f89cc150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:46.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.960+0000 7f89d14c9700 1 -- 172.21.15.163:0/1144731269 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89cc1574d0 msgr2=0x7f89cc150ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:46.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.960+0000 7f89d14c9700 1 --2- 172.21.15.163:0/1144731269 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89cc1574d0 0x7f89cc150ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:46.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.960+0000 7f89d14c9700 1 -- 172.21.15.163:0/1144731269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89cc06c0f0 msgr2=0x7f89cc152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:46.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.960+0000 7f89d14c9700 1 --2- 172.21.15.163:0/1144731269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89cc06c0f0 0x7f89cc152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:46.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.960+0000 7f89d14c9700 1 -- 172.21.15.163:0/1144731269 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c8009480 con 0x7f89cc0cc3c0 2023-10-09T06:31:46.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.960+0000 7f89d14c9700 1 --2- 172.21.15.163:0/1144731269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89cc0cc3c0 0x7f89cc15a800 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f89bc00c590 tx=0x7f89bc007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:46.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.960+0000 7f89c1ffb700 1 -- 172.21.15.163:0/1144731269 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc011590 con 0x7f89cc0cc3c0 2023-10-09T06:31:46.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.960+0000 7f89c1ffb700 1 -- 172.21.15.163:0/1144731269 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89bc0116f0 con 0x7f89cc0cc3c0 2023-10-09T06:31:46.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.960+0000 7f89d2ccc700 1 -- 172.21.15.163:0/1144731269 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89cc15ba60 con 0x7f89cc0cc3c0 2023-10-09T06:31:46.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.960+0000 7f89c1ffb700 1 -- 172.21.15.163:0/1144731269 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc011ab0 con 0x7f89cc0cc3c0 2023-10-09T06:31:46.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.960+0000 7f89d2ccc700 1 -- 172.21.15.163:0/1144731269 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89cc0af160 con 0x7f89cc0cc3c0 2023-10-09T06:31:46.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.961+0000 7f89d2ccc700 1 -- 172.21.15.163:0/1144731269 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89cc04c3c0 con 0x7f89cc0cc3c0 2023-10-09T06:31:46.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.962+0000 7f89c1ffb700 1 -- 172.21.15.163:0/1144731269 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f89bc01c460 con 0x7f89cc0cc3c0 2023-10-09T06:31:46.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.962+0000 7f89c1ffb700 1 --2- 172.21.15.163:0/1144731269 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f89b0066f30 0x7f89b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:46.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.962+0000 7f89c1ffb700 1 -- 172.21.15.163:0/1144731269 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f89bc0841d0 con 0x7f89cc0cc3c0 2023-10-09T06:31:46.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.962+0000 7f89d1cca700 1 --2- 172.21.15.163:0/1144731269 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f89b0066f30 0x7f89b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:46.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.963+0000 7f89d1cca700 1 --2- 172.21.15.163:0/1144731269 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f89b0066f30 0x7f89b00693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f89c8003080 tx=0x7f89c8015000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:46.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:46.966+0000 7f89c1ffb700 1 -- 172.21.15.163:0/1144731269 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89bc0513f0 con 0x7f89cc0cc3c0 2023-10-09T06:31:47.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.148+0000 7f89d2ccc700 1 -- 172.21.15.163:0/1144731269 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89cc0cde50 con 0x7f89cc0cc3c0 2023-10-09T06:31:47.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.149+0000 7f89c1ffb700 1 -- 172.21.15.163:0/1144731269 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+15529 (secure 0 0 0) 0x7f89bc054a10 con 0x7f89cc0cc3c0 2023-10-09T06:31:47.149 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:47.150 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":39,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:31:46.160317+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:31:25.770415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-09T06:31:26.980879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6835/2241937708":"2023-10-10T06:31:43.143850+0000","172.21.15.163:6833/2241937708":"2023-10-10T06:31:43.143850+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:31:47.152 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.151+0000 7f89d2ccc700 1 -- 172.21.15.163:0/1144731269 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f89b0066f30 msgr2=0x7f89b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:47.152 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.151+0000 7f89d2ccc700 1 --2- 172.21.15.163:0/1144731269 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f89b0066f30 0x7f89b00693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f89c8003080 tx=0x7f89c8015000 comp rx=0 tx=0).stop 2023-10-09T06:31:47.152 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.151+0000 7f89d2ccc700 1 -- 172.21.15.163:0/1144731269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89cc0cc3c0 msgr2=0x7f89cc15a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:47.152 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.151+0000 7f89d2ccc700 1 --2- 172.21.15.163:0/1144731269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89cc0cc3c0 0x7f89cc15a800 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f89bc00c590 tx=0x7f89bc007f30 comp rx=0 tx=0).stop 2023-10-09T06:31:47.152 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.151+0000 7f89d2ccc700 1 -- 172.21.15.163:0/1144731269 shutdown_connections 2023-10-09T06:31:47.152 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.151+0000 7f89d2ccc700 1 --2- 172.21.15.163:0/1144731269 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f89b0066f30 0x7f89b00693e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.153 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.151+0000 7f89d2ccc700 1 --2- 172.21.15.163:0/1144731269 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89cc1574d0 0x7f89cc150ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.153 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.151+0000 7f89d2ccc700 1 --2- 172.21.15.163:0/1144731269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89cc06c0f0 0x7f89cc152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.153 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.151+0000 7f89d2ccc700 1 --2- 172.21.15.163:0/1144731269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89cc0cc3c0 0x7f89cc15a800 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.153 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.151+0000 7f89d2ccc700 1 -- 172.21.15.163:0/1144731269 >> 172.21.15.163:0/1144731269 conn(0x7f89cc0ce430 msgr2=0x7f89cc153ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:47.153 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.151+0000 7f89d2ccc700 1 -- 172.21.15.163:0/1144731269 shutdown_connections 2023-10-09T06:31:47.153 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.151+0000 7f89d2ccc700 1 -- 172.21.15.163:0/1144731269 wait complete. 2023-10-09T06:31:47.164 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-09T06:31:25.770415+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-10-09T06:31:26.980879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-09T06:31:47.165 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:31:47.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.282+0000 7fcd4c542700 1 Processor -- start 2023-10-09T06:31:47.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.282+0000 7fcd4c542700 1 -- start start 2023-10-09T06:31:47.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.283+0000 7fcd4c542700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd44069d60 0x7fcd440c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:47.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.283+0000 7fcd4c542700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd4406bdb0 con 0x7fcd440c81f0 2023-10-09T06:31:47.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.283+0000 7fcd4c542700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd440cb370 con 0x7fcd440aca80 2023-10-09T06:31:47.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.283+0000 7fcd4c542700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd440c8670 con 0x7fcd44069d60 2023-10-09T06:31:47.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.283+0000 7fcd4aadf700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fcd440c81f0 0x7fcd440cb800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:49768/0 (socket says 172.21.15.163:49768) 2023-10-09T06:31:47.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.283+0000 7fcd4aadf700 1 -- 172.21.15.163:0/1878605550 learned_addr learned my addr 172.21.15.163:0/1878605550 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:47.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.283+0000 7fcd49add700 1 --2- 172.21.15.163:0/1878605550 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd44069d60 0x7fcd440c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:47.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.283+0000 7fcd492dc700 1 -- 172.21.15.163:0/1878605550 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1587465680 0 0) 0x7fcd4406bdb0 con 0x7fcd440c81f0 2023-10-09T06:31:47.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.283+0000 7fcd492dc700 1 -- 172.21.15.163:0/1878605550 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd2c003570 con 0x7fcd440c81f0 2023-10-09T06:31:47.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.283+0000 7fcd492dc700 1 -- 172.21.15.163:0/1878605550 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3672947854 0 0) 0x7fcd2c003570 con 0x7fcd440c81f0 2023-10-09T06:31:47.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.283+0000 7fcd492dc700 1 -- 172.21.15.163:0/1878605550 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd44069d60 msgr2=0x7fcd440c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:47.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.283+0000 7fcd492dc700 1 --2- 172.21.15.163:0/1878605550 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd44069d60 0x7fcd440c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.283+0000 7fcd492dc700 1 -- 172.21.15.163:0/1878605550 >> v1:172.21.15.181:6789/0 conn(0x7fcd440aca80 legacy=0x7fcd4406aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:47.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.283+0000 7fcd492dc700 1 -- 172.21.15.163:0/1878605550 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd44159f60 con 0x7fcd440c81f0 2023-10-09T06:31:47.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.283+0000 7fcd492dc700 1 -- 172.21.15.163:0/1878605550 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fcd34001470 con 0x7fcd440c81f0 2023-10-09T06:31:47.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.283+0000 7fcd492dc700 1 -- 172.21.15.163:0/1878605550 >> v1:172.21.15.163:6789/0 conn(0x7fcd440c81f0 legacy=0x7fcd440cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:47.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.283+0000 7fcd492dc700 1 --2- 172.21.15.163:0/1878605550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd2c003a90 0x7fcd2c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:47.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.283+0000 7fcd492dc700 1 --2- 172.21.15.163:0/1878605550 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd44069d60 0x7fcd2c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:47.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.283+0000 7fcd492dc700 1 --2- 172.21.15.163:0/1878605550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd2c004d50 0x7fcd2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:47.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.284+0000 7fcd492dc700 1 -- 172.21.15.163:0/1878605550 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd440c8670 con 0x7fcd2c003a90 2023-10-09T06:31:47.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.284+0000 7fcd492dc700 1 -- 172.21.15.163:0/1878605550 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c00bb70 con 0x7fcd2c004d50 2023-10-09T06:31:47.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.284+0000 7fcd492dc700 1 -- 172.21.15.163:0/1878605550 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c00bd30 con 0x7fcd44069d60 2023-10-09T06:31:47.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.284+0000 7fcd4a2de700 1 --2- 172.21.15.163:0/1878605550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd2c003a90 0x7fcd2c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:47.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.284+0000 7fcd49add700 1 --2- 172.21.15.163:0/1878605550 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd44069d60 0x7fcd2c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:47.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.284+0000 7fcd4aadf700 1 --2- 172.21.15.163:0/1878605550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd2c004d50 0x7fcd2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:47.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.284+0000 7fcd4a2de700 1 -- 172.21.15.163:0/1878605550 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd44069d60 msgr2=0x7fcd2c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:47.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.284+0000 7fcd4a2de700 1 --2- 172.21.15.163:0/1878605550 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd44069d60 0x7fcd2c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.284+0000 7fcd4a2de700 1 -- 172.21.15.163:0/1878605550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd2c004d50 msgr2=0x7fcd2c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:47.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.284+0000 7fcd4a2de700 1 --2- 172.21.15.163:0/1878605550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd2c004d50 0x7fcd2c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.284+0000 7fcd4a2de700 1 -- 172.21.15.163:0/1878605550 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd2c003570 con 0x7fcd2c003a90 2023-10-09T06:31:47.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.284+0000 7fcd4aadf700 1 --2- 172.21.15.163:0/1878605550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd2c004d50 0x7fcd2c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:47.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.284+0000 7fcd4a2de700 1 --2- 172.21.15.163:0/1878605550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd2c003a90 0x7fcd2c003ea0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c00c250 tx=0x7fcd3c001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705ec5b8fa948dfc server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:47.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.284+0000 7fcd492dc700 1 -- 172.21.15.163:0/1878605550 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c001db0 con 0x7fcd2c003a90 2023-10-09T06:31:47.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.284+0000 7fcd492dc700 1 -- 172.21.15.163:0/1878605550 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3c00cb80 con 0x7fcd2c003a90 2023-10-09T06:31:47.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.284+0000 7fcd492dc700 1 -- 172.21.15.163:0/1878605550 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c012420 con 0x7fcd2c003a90 2023-10-09T06:31:47.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.285+0000 7fcd4c542700 1 -- 172.21.15.163:0/1878605550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd2c003a90 msgr2=0x7fcd2c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:47.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.285+0000 7fcd4c542700 1 --2- 172.21.15.163:0/1878605550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd2c003a90 0x7fcd2c003ea0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c00c250 tx=0x7fcd3c001aa0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.285+0000 7fcd4aadf700 1 -- 172.21.15.163:0/1878605550 reap_dead start 2023-10-09T06:31:47.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.285+0000 7fcd4c542700 1 -- 172.21.15.163:0/1878605550 shutdown_connections 2023-10-09T06:31:47.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.285+0000 7fcd4c542700 1 -- 172.21.15.163:0/1878605550 >> 172.21.15.163:0/1878605550 conn(0x7fcd440cd670 msgr2=0x7fcd440cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:47.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.285+0000 7fcd4c542700 1 -- 172.21.15.163:0/1878605550 shutdown_connections 2023-10-09T06:31:47.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.285+0000 7fcd4c542700 1 -- 172.21.15.163:0/1878605550 wait complete. 2023-10-09T06:31:47.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.286+0000 7fcd4c542700 1 Processor -- start 2023-10-09T06:31:47.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.286+0000 7fcd4c542700 1 -- start start 2023-10-09T06:31:47.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.286+0000 7fcd4c542700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd2c003a90 0x7fcd4416d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:47.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.286+0000 7fcd4c542700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd440b1b00 0x7fcd44157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:47.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.286+0000 7fcd4c542700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd4416a8b0 0x7fcd440b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:47.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.286+0000 7fcd4c542700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4416a420 con 0x7fcd440b1b00 2023-10-09T06:31:47.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.286+0000 7fcd4c542700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4416a0f0 con 0x7fcd4416a8b0 2023-10-09T06:31:47.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.286+0000 7fcd4c542700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4416dee0 con 0x7fcd2c003a90 2023-10-09T06:31:47.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.287+0000 7fcd49add700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd440b1b00 0x7fcd44157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:47.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.287+0000 7fcd49add700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd440b1b00 0x7fcd44157990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40770/0 (socket says 172.21.15.163:40770) 2023-10-09T06:31:47.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.287+0000 7fcd49add700 1 -- 172.21.15.163:0/2902314852 learned_addr learned my addr 172.21.15.163:0/2902314852 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:47.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.287+0000 7fcd4a2de700 1 --2- 172.21.15.163:0/2902314852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd2c003a90 0x7fcd4416d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:47.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.287+0000 7fcd4aadf700 1 --2- 172.21.15.163:0/2902314852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd4416a8b0 0x7fcd440b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:47.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.287+0000 7fcd49add700 1 -- 172.21.15.163:0/2902314852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd2c003a90 msgr2=0x7fcd4416d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:47.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.287+0000 7fcd49add700 1 --2- 172.21.15.163:0/2902314852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd2c003a90 0x7fcd4416d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.287+0000 7fcd49add700 1 -- 172.21.15.163:0/2902314852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd4416a8b0 msgr2=0x7fcd440b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:47.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.287+0000 7fcd49add700 1 --2- 172.21.15.163:0/2902314852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd4416a8b0 0x7fcd440b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.287+0000 7fcd49add700 1 -- 172.21.15.163:0/2902314852 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd4416e1c0 con 0x7fcd440b1b00 2023-10-09T06:31:47.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.287+0000 7fcd49add700 1 --2- 172.21.15.163:0/2902314852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd440b1b00 0x7fcd44157990 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fcd38011b40 tx=0x7fcd380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:47.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.287+0000 7fcd427fc700 1 -- 172.21.15.163:0/2902314852 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3800f940 con 0x7fcd440b1b00 2023-10-09T06:31:47.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.287+0000 7fcd427fc700 1 -- 172.21.15.163:0/2902314852 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3800faa0 con 0x7fcd440b1b00 2023-10-09T06:31:47.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.287+0000 7fcd4c542700 1 -- 172.21.15.163:0/2902314852 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd4416e4a0 con 0x7fcd440b1b00 2023-10-09T06:31:47.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.287+0000 7fcd427fc700 1 -- 172.21.15.163:0/2902314852 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3800fe20 con 0x7fcd440b1b00 2023-10-09T06:31:47.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.287+0000 7fcd4c542700 1 -- 172.21.15.163:0/2902314852 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd44157ed0 con 0x7fcd440b1b00 2023-10-09T06:31:47.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.288+0000 7fcd4c542700 1 -- 172.21.15.163:0/2902314852 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd4404c3c0 con 0x7fcd440b1b00 2023-10-09T06:31:47.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.289+0000 7fcd427fc700 1 -- 172.21.15.163:0/2902314852 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcd38014650 con 0x7fcd440b1b00 2023-10-09T06:31:47.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.289+0000 7fcd427fc700 1 --2- 172.21.15.163:0/2902314852 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcd30066f30 0x7fcd300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:47.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.289+0000 7fcd427fc700 1 -- 172.21.15.163:0/2902314852 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fcd38018030 con 0x7fcd440b1b00 2023-10-09T06:31:47.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.289+0000 7fcd4a2de700 1 --2- 172.21.15.163:0/2902314852 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcd30066f30 0x7fcd300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:47.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.290+0000 7fcd4a2de700 1 --2- 172.21.15.163:0/2902314852 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcd30066f30 0x7fcd300693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c00c250 tx=0x7fcd3c016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:47.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.293+0000 7fcd427fc700 1 -- 172.21.15.163:0/2902314852 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd380520c0 con 0x7fcd440b1b00 2023-10-09T06:31:47.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.474+0000 7fcd4c542700 1 -- 172.21.15.163:0/2902314852 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcd44159b20 con 0x7fcd440b1b00 2023-10-09T06:31:47.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.475+0000 7fcd427fc700 1 -- 172.21.15.163:0/2902314852 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcd380220a0 con 0x7fcd440b1b00 2023-10-09T06:31:47.476 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:31:47.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.477+0000 7fcd4c542700 1 -- 172.21.15.163:0/2902314852 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcd30066f30 msgr2=0x7fcd300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:47.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.477+0000 7fcd4c542700 1 --2- 172.21.15.163:0/2902314852 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcd30066f30 0x7fcd300693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c00c250 tx=0x7fcd3c016000 comp rx=0 tx=0).stop 2023-10-09T06:31:47.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.477+0000 7fcd4c542700 1 -- 172.21.15.163:0/2902314852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd440b1b00 msgr2=0x7fcd44157990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:47.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.477+0000 7fcd4c542700 1 --2- 172.21.15.163:0/2902314852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd440b1b00 0x7fcd44157990 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fcd38011b40 tx=0x7fcd380091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.477+0000 7fcd4c542700 1 -- 172.21.15.163:0/2902314852 shutdown_connections 2023-10-09T06:31:47.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.477+0000 7fcd4c542700 1 --2- 172.21.15.163:0/2902314852 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcd30066f30 0x7fcd300693e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.477+0000 7fcd4c542700 1 --2- 172.21.15.163:0/2902314852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd4416a8b0 0x7fcd440b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.477+0000 7fcd4c542700 1 --2- 172.21.15.163:0/2902314852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd2c003a90 0x7fcd4416d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.477+0000 7fcd4c542700 1 --2- 172.21.15.163:0/2902314852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd440b1b00 0x7fcd44157990 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.477+0000 7fcd4c542700 1 -- 172.21.15.163:0/2902314852 >> 172.21.15.163:0/2902314852 conn(0x7fcd440cd670 msgr2=0x7fcd44068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:47.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.477+0000 7fcd4c542700 1 -- 172.21.15.163:0/2902314852 shutdown_connections 2023-10-09T06:31:47.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.477+0000 7fcd4c542700 1 -- 172.21.15.163:0/2902314852 wait complete. 2023-10-09T06:31:47.490 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-09T06:31:47.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.606+0000 7f0340cf9700 1 Processor -- start 2023-10-09T06:31:47.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.606+0000 7f0340cf9700 1 -- start start 2023-10-09T06:31:47.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.607+0000 7f0340cf9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f033c159890 0x7f033c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:47.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.607+0000 7f0340cf9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f033c069560 0x7f033c0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:47.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.607+0000 7f0340cf9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f033c15a160 con 0x7f033c159c60 2023-10-09T06:31:47.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.607+0000 7f0340cf9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f033c069f20 con 0x7f033c069560 2023-10-09T06:31:47.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.607+0000 7f0340cf9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033c0d13a0 con 0x7f033c159890 2023-10-09T06:31:47.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.607+0000 7f033ad9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f033c069560 0x7f033c0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:47.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.607+0000 7f033a59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f033c159890 0x7f033c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:47.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.607+0000 7f033a59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f033c159890 0x7f033c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:56330/0 (socket says 172.21.15.163:56330) 2023-10-09T06:31:47.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.607+0000 7f0331d9b700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f033c159c60 0x7f033c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:46848/0 (socket says 172.21.15.163:46848) 2023-10-09T06:31:47.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.607+0000 7f033ad9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f033c069560 0x7f033c0699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:55632/0 (socket says 172.21.15.163:55632) 2023-10-09T06:31:47.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.607+0000 7f0331d9b700 1 -- 172.21.15.163:0/4190093255 learned_addr learned my addr 172.21.15.163:0/4190093255 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:47.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.608+0000 7f033a59c700 1 -- 172.21.15.163:0/4190093255 >> v2:172.21.15.181:3300/0 conn(0x7f033c069560 msgr2=0x7f033c0699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:47.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.608+0000 7f033a59c700 1 --2- 172.21.15.163:0/4190093255 >> v2:172.21.15.181:3300/0 conn(0x7f033c069560 0x7f033c0699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.608+0000 7f033a59c700 1 -- 172.21.15.163:0/4190093255 >> v1:172.21.15.181:6789/0 conn(0x7f033c159c60 legacy=0x7f033c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:47.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.608+0000 7f033a59c700 1 -- 172.21.15.163:0/4190093255 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033c15e140 con 0x7f033c159890 2023-10-09T06:31:47.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.608+0000 7f033ad9d700 1 --2- 172.21.15.163:0/4190093255 >> v2:172.21.15.181:3300/0 conn(0x7f033c069560 0x7f033c0699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:31:47.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.608+0000 7f033a59c700 1 --2- 172.21.15.163:0/4190093255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f033c159890 0x7f033c15abd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f032c0098e0 tx=0x7f032c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e3bcc6dd6e5c276 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:47.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.608+0000 7f0339d9b700 1 -- 172.21.15.163:0/4190093255 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c00b640 con 0x7f033c159890 2023-10-09T06:31:47.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.608+0000 7f0339d9b700 1 -- 172.21.15.163:0/4190093255 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032c00b7a0 con 0x7f033c159890 2023-10-09T06:31:47.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.608+0000 7f0339d9b700 1 -- 172.21.15.163:0/4190093255 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c00bb40 con 0x7f033c159890 2023-10-09T06:31:47.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.608+0000 7f0340cf9700 1 -- 172.21.15.163:0/4190093255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f033c159890 msgr2=0x7f033c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:47.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.608+0000 7f0340cf9700 1 --2- 172.21.15.163:0/4190093255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f033c159890 0x7f033c15abd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f032c0098e0 tx=0x7f032c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.608+0000 7f0340cf9700 1 -- 172.21.15.163:0/4190093255 shutdown_connections 2023-10-09T06:31:47.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.608+0000 7f0340cf9700 1 --2- 172.21.15.163:0/4190093255 >> v2:172.21.15.181:3300/0 conn(0x7f033c069560 0x7f033c0699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.608+0000 7f0340cf9700 1 --2- 172.21.15.163:0/4190093255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f033c159890 0x7f033c15abd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.608+0000 7f0340cf9700 1 -- 172.21.15.163:0/4190093255 >> 172.21.15.163:0/4190093255 conn(0x7f033c068e20 msgr2=0x7f033c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:47.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.609+0000 7f0340cf9700 1 -- 172.21.15.163:0/4190093255 shutdown_connections 2023-10-09T06:31:47.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.609+0000 7f0340cf9700 1 -- 172.21.15.163:0/4190093255 wait complete. 2023-10-09T06:31:47.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.610+0000 7f0340cf9700 1 Processor -- start 2023-10-09T06:31:47.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.610+0000 7f0340cf9700 1 -- start start 2023-10-09T06:31:47.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.610+0000 7f0340cf9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f033c069560 0x7f033c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:47.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.610+0000 7f0340cf9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f033c159890 0x7f033c1577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:47.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.610+0000 7f0340cf9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f033c159c60 0x7f033c0b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:47.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.610+0000 7f0340cf9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c0b4e90 con 0x7f033c069560 2023-10-09T06:31:47.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.610+0000 7f0340cf9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c0b00d0 con 0x7f033c159c60 2023-10-09T06:31:47.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.610+0000 7f0340cf9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033c0b4280 con 0x7f033c159890 2023-10-09T06:31:47.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.611+0000 7f033a59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f033c069560 0x7f033c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:47.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.611+0000 7f033a59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f033c069560 0x7f033c0afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40774/0 (socket says 172.21.15.163:40774) 2023-10-09T06:31:47.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.611+0000 7f033a59c700 1 -- 172.21.15.163:0/1893705553 learned_addr learned my addr 172.21.15.163:0/1893705553 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:47.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.611+0000 7f033ad9d700 1 --2- 172.21.15.163:0/1893705553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f033c159c60 0x7f033c0b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:47.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.611+0000 7f0331d9b700 1 --2- 172.21.15.163:0/1893705553 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f033c159890 0x7f033c1577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:47.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.611+0000 7f033a59c700 1 -- 172.21.15.163:0/1893705553 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f033c159890 msgr2=0x7f033c1577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:47.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.611+0000 7f033a59c700 1 --2- 172.21.15.163:0/1893705553 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f033c159890 0x7f033c1577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.611+0000 7f033a59c700 1 -- 172.21.15.163:0/1893705553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f033c159c60 msgr2=0x7f033c0b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:47.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.611+0000 7f033a59c700 1 --2- 172.21.15.163:0/1893705553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f033c159c60 0x7f033c0b53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.611+0000 7f033a59c700 1 -- 172.21.15.163:0/1893705553 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0334009a20 con 0x7f033c069560 2023-10-09T06:31:47.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.611+0000 7f033ad9d700 1 --2- 172.21.15.163:0/1893705553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f033c159c60 0x7f033c0b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:31:47.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.611+0000 7f033a59c700 1 --2- 172.21.15.163:0/1893705553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f033c069560 0x7f033c0afb90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f033c070e20 tx=0x7f032c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:47.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.611+0000 7f0332ffd700 1 -- 172.21.15.163:0/1893705553 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c01bcf0 con 0x7f033c069560 2023-10-09T06:31:47.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.611+0000 7f0332ffd700 1 -- 172.21.15.163:0/1893705553 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032c01be70 con 0x7f033c069560 2023-10-09T06:31:47.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.611+0000 7f0340cf9700 1 -- 172.21.15.163:0/1893705553 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f032c009420 con 0x7f033c069560 2023-10-09T06:31:47.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.611+0000 7f0332ffd700 1 -- 172.21.15.163:0/1893705553 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c011650 con 0x7f033c069560 2023-10-09T06:31:47.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.611+0000 7f0340cf9700 1 -- 172.21.15.163:0/1893705553 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f033c0b4840 con 0x7f033c069560 2023-10-09T06:31:47.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.612+0000 7f0340cf9700 1 -- 172.21.15.163:0/1893705553 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f033c04c3c0 con 0x7f033c069560 2023-10-09T06:31:47.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.613+0000 7f0332ffd700 1 -- 172.21.15.163:0/1893705553 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f032c011a00 con 0x7f033c069560 2023-10-09T06:31:47.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.613+0000 7f0332ffd700 1 --2- 172.21.15.163:0/1893705553 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f031c066f30 0x7f031c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:47.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.613+0000 7f0332ffd700 1 -- 172.21.15.163:0/1893705553 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f032c085ae0 con 0x7f033c069560 2023-10-09T06:31:47.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.613+0000 7f0331d9b700 1 --2- 172.21.15.163:0/1893705553 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f031c066f30 0x7f031c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:47.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.614+0000 7f0331d9b700 1 --2- 172.21.15.163:0/1893705553 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f031c066f30 0x7f031c0693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f033c15a850 tx=0x7f0324007eb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:47.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.617+0000 7f0332ffd700 1 -- 172.21.15.163:0/1893705553 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f032c052dc0 con 0x7f033c069560 2023-10-09T06:31:47.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.798+0000 7f0340cf9700 1 -- 172.21.15.163:0/1893705553 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f033c06a5b0 con 0x7f033c069560 2023-10-09T06:31:47.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.799+0000 7f0332ffd700 1 -- 172.21.15.163:0/1893705553 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) v1 ==== 104+0+11 (secure 0 0 0) 0x7f032c015020 con 0x7f033c069560 2023-10-09T06:31:47.800 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 32 2023-10-09T06:31:47.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.801+0000 7f0340cf9700 1 -- 172.21.15.163:0/1893705553 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f031c066f30 msgr2=0x7f031c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:47.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.801+0000 7f0340cf9700 1 --2- 172.21.15.163:0/1893705553 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f031c066f30 0x7f031c0693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f033c15a850 tx=0x7f0324007eb0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.801+0000 7f0340cf9700 1 -- 172.21.15.163:0/1893705553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f033c069560 msgr2=0x7f033c0afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:47.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.801+0000 7f0340cf9700 1 --2- 172.21.15.163:0/1893705553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f033c069560 0x7f033c0afb90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f033c070e20 tx=0x7f032c017000 comp rx=0 tx=0).stop 2023-10-09T06:31:47.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.801+0000 7f0340cf9700 1 -- 172.21.15.163:0/1893705553 shutdown_connections 2023-10-09T06:31:47.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.801+0000 7f0340cf9700 1 --2- 172.21.15.163:0/1893705553 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f031c066f30 0x7f031c0693e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.801+0000 7f0340cf9700 1 --2- 172.21.15.163:0/1893705553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f033c069560 0x7f033c0afb90 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.801+0000 7f0340cf9700 1 --2- 172.21.15.163:0/1893705553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f033c159c60 0x7f033c0b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.801+0000 7f0340cf9700 1 --2- 172.21.15.163:0/1893705553 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f033c159890 0x7f033c1577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.801+0000 7f0340cf9700 1 -- 172.21.15.163:0/1893705553 >> 172.21.15.163:0/1893705553 conn(0x7f033c068e20 msgr2=0x7f033c1586b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:47.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.801+0000 7f0340cf9700 1 -- 172.21.15.163:0/1893705553 shutdown_connections 2023-10-09T06:31:47.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.801+0000 7f0340cf9700 1 -- 172.21.15.163:0/1893705553 wait complete. 2023-10-09T06:31:47.814 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-09T06:31:47.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.929+0000 7f281e8c1700 1 Processor -- start 2023-10-09T06:31:47.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.930+0000 7f281e8c1700 1 -- start start 2023-10-09T06:31:47.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.930+0000 7f281e8c1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f28180b16d0 0x7f28180b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:47.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.930+0000 7f281e8c1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28180b33b0 0x7f28180b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:47.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.930+0000 7f281e8c1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28180b2080 con 0x7f28180b4e70 2023-10-09T06:31:47.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.930+0000 7f281e8c1700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28180cfb00 con 0x7f28180b16d0 2023-10-09T06:31:47.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.930+0000 7f281e8c1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28180b5c40 con 0x7f28180b33b0 2023-10-09T06:31:47.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.931+0000 7f281d0be700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f28180b16d0 0x7f28180b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:47.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.931+0000 7f281d0be700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f28180b16d0 0x7f28180b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40778/0 (socket says 172.21.15.163:40778) 2023-10-09T06:31:47.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.931+0000 7f281d0be700 1 -- 172.21.15.163:0/3192678723 learned_addr learned my addr 172.21.15.163:0/3192678723 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:47.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.931+0000 7f281e0c0700 1 --2- 172.21.15.163:0/3192678723 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28180b33b0 0x7f28180b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:47.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.931+0000 7f281d0be700 1 -- 172.21.15.163:0/3192678723 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28180b33b0 msgr2=0x7f28180b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:47.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.931+0000 7f281d0be700 1 --2- 172.21.15.163:0/3192678723 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28180b33b0 0x7f28180b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.931+0000 7f281d0be700 1 -- 172.21.15.163:0/3192678723 >> v1:172.21.15.181:6789/0 conn(0x7f28180b4e70 legacy=0x7f28180ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:47.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.931+0000 7f281d0be700 1 -- 172.21.15.163:0/3192678723 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28180b5f20 con 0x7f28180b16d0 2023-10-09T06:31:47.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.931+0000 7f281d0be700 1 --2- 172.21.15.163:0/3192678723 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28180b16d0 0x7f28180b1b40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f28080099b0 tx=0x7f2808009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ffe659b0d896aed server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:47.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.931+0000 7f281c8bd700 1 -- 172.21.15.163:0/3192678723 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2808009ea0 con 0x7f28180b16d0 2023-10-09T06:31:47.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.931+0000 7f281c8bd700 1 -- 172.21.15.163:0/3192678723 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280800b860 con 0x7f28180b16d0 2023-10-09T06:31:47.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.931+0000 7f281c8bd700 1 -- 172.21.15.163:0/3192678723 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280800bc20 con 0x7f28180b16d0 2023-10-09T06:31:47.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.932+0000 7f281e8c1700 1 -- 172.21.15.163:0/3192678723 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28180b16d0 msgr2=0x7f28180b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:47.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.932+0000 7f281e8c1700 1 --2- 172.21.15.163:0/3192678723 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28180b16d0 0x7f28180b1b40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f28080099b0 tx=0x7f2808009250 comp rx=0 tx=0).stop 2023-10-09T06:31:47.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.932+0000 7f281e8c1700 1 -- 172.21.15.163:0/3192678723 shutdown_connections 2023-10-09T06:31:47.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.932+0000 7f281e8c1700 1 --2- 172.21.15.163:0/3192678723 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28180b33b0 0x7f28180b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.932+0000 7f281e8c1700 1 --2- 172.21.15.163:0/3192678723 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28180b16d0 0x7f28180b1b40 secure :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f28080099b0 tx=0x7f2808009250 comp rx=0 tx=0).stop 2023-10-09T06:31:47.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.932+0000 7f281e8c1700 1 -- 172.21.15.163:0/3192678723 >> 172.21.15.163:0/3192678723 conn(0x7f28180d10c0 msgr2=0x7f28180affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:47.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.932+0000 7f281e8c1700 1 -- 172.21.15.163:0/3192678723 shutdown_connections 2023-10-09T06:31:47.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.932+0000 7f281e8c1700 1 -- 172.21.15.163:0/3192678723 wait complete. 2023-10-09T06:31:47.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.933+0000 7f281e8c1700 1 Processor -- start 2023-10-09T06:31:47.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.933+0000 7f281e8c1700 1 -- start start 2023-10-09T06:31:47.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.934+0000 7f281e8c1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28180b33b0 0x7f28181477b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:47.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.934+0000 7f281e8c1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28180b4e70 0x7f28180a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:47.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.934+0000 7f281e8c1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28180a7440 0x7f28180c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:47.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.934+0000 7f281e8c1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281816b000 con 0x7f28180a7440 2023-10-09T06:31:47.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.934+0000 7f281e8c1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28180a7890 con 0x7f28180b33b0 2023-10-09T06:31:47.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.934+0000 7f281e8c1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2818147cf0 con 0x7f28180b4e70 2023-10-09T06:31:47.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.934+0000 7f281e0c0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28180a7440 0x7f28180c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:47.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.934+0000 7f281e0c0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28180a7440 0x7f28180c77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40782/0 (socket says 172.21.15.163:40782) 2023-10-09T06:31:47.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.934+0000 7f281e0c0700 1 -- 172.21.15.163:0/242235442 learned_addr learned my addr 172.21.15.163:0/242235442 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:47.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.934+0000 7f281d0be700 1 --2- 172.21.15.163:0/242235442 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28180b4e70 0x7f28180a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:47.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.934+0000 7f281e0c0700 1 -- 172.21.15.163:0/242235442 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28180b4e70 msgr2=0x7f28180a6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:47.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.934+0000 7f281e0c0700 1 --2- 172.21.15.163:0/242235442 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28180b4e70 0x7f28180a6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.934+0000 7f281e0c0700 1 -- 172.21.15.163:0/242235442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28180b33b0 msgr2=0x7f28181477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:47.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.934+0000 7f281e0c0700 1 --2- 172.21.15.163:0/242235442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28180b33b0 0x7f28181477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:47.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.934+0000 7f281e0c0700 1 -- 172.21.15.163:0/242235442 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28080094f0 con 0x7f28180a7440 2023-10-09T06:31:47.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.934+0000 7f281e0c0700 1 --2- 172.21.15.163:0/242235442 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28180a7440 0x7f28180c77b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2810009e90 tx=0x7f281000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:47.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.935+0000 7f280dffb700 1 -- 172.21.15.163:0/242235442 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281000cc20 con 0x7f28180a7440 2023-10-09T06:31:47.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.935+0000 7f280dffb700 1 -- 172.21.15.163:0/242235442 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f281000f040 con 0x7f28180a7440 2023-10-09T06:31:47.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.935+0000 7f281e8c1700 1 -- 172.21.15.163:0/242235442 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28180c8540 con 0x7f28180a7440 2023-10-09T06:31:47.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.935+0000 7f280dffb700 1 -- 172.21.15.163:0/242235442 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2810014480 con 0x7f28180a7440 2023-10-09T06:31:47.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.935+0000 7f281e8c1700 1 -- 172.21.15.163:0/242235442 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28180c8b00 con 0x7f28180a7440 2023-10-09T06:31:47.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.936+0000 7f281e8c1700 1 -- 172.21.15.163:0/242235442 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2818088990 con 0x7f28180a7440 2023-10-09T06:31:47.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.936+0000 7f280dffb700 1 -- 172.21.15.163:0/242235442 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f28100147f0 con 0x7f28180a7440 2023-10-09T06:31:47.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.937+0000 7f280dffb700 1 --2- 172.21.15.163:0/242235442 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f27fc066f30 0x7f27fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:47.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.937+0000 7f280dffb700 1 -- 172.21.15.163:0/242235442 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f2810085290 con 0x7f28180a7440 2023-10-09T06:31:47.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.937+0000 7f281d8bf700 1 --2- 172.21.15.163:0/242235442 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f27fc066f30 0x7f27fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:47.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.937+0000 7f281d8bf700 1 --2- 172.21.15.163:0/242235442 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f27fc066f30 0x7f27fc0693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f28180c9350 tx=0x7f2814007b40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:47.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:47.941+0000 7f280dffb700 1 -- 172.21.15.163:0/242235442 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2810055f40 con 0x7f28180a7440 2023-10-09T06:31:48.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.124+0000 7f281e8c1700 1 -- 172.21.15.163:0/242235442 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f28180b14b0 con 0x7f28180a7440 2023-10-09T06:31:48.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.125+0000 7f280dffb700 1 -- 172.21.15.163:0/242235442 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2810055ad0 con 0x7f28180a7440 2023-10-09T06:31:48.126 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 64 2023-10-09T06:31:48.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.127+0000 7f281e8c1700 1 -- 172.21.15.163:0/242235442 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f27fc066f30 msgr2=0x7f27fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:48.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.127+0000 7f281e8c1700 1 --2- 172.21.15.163:0/242235442 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f27fc066f30 0x7f27fc0693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f28180c9350 tx=0x7f2814007b40 comp rx=0 tx=0).stop 2023-10-09T06:31:48.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.127+0000 7f281e8c1700 1 -- 172.21.15.163:0/242235442 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28180a7440 msgr2=0x7f28180c77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:48.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.127+0000 7f281e8c1700 1 --2- 172.21.15.163:0/242235442 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28180a7440 0x7f28180c77b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2810009e90 tx=0x7f281000b040 comp rx=0 tx=0).stop 2023-10-09T06:31:48.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.127+0000 7f281e8c1700 1 -- 172.21.15.163:0/242235442 shutdown_connections 2023-10-09T06:31:48.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.127+0000 7f281e8c1700 1 --2- 172.21.15.163:0/242235442 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f27fc066f30 0x7f27fc0693e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:48.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.127+0000 7f281e8c1700 1 --2- 172.21.15.163:0/242235442 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28180a7440 0x7f28180c77b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:48.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.127+0000 7f281e8c1700 1 --2- 172.21.15.163:0/242235442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28180b33b0 0x7f28181477b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:48.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.127+0000 7f281e8c1700 1 --2- 172.21.15.163:0/242235442 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28180b4e70 0x7f28180a6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:48.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.127+0000 7f281e8c1700 1 -- 172.21.15.163:0/242235442 >> 172.21.15.163:0/242235442 conn(0x7f28180d10c0 msgr2=0x7f28180b0840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:48.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.127+0000 7f281e8c1700 1 -- 172.21.15.163:0/242235442 shutdown_connections 2023-10-09T06:31:48.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.127+0000 7f281e8c1700 1 -- 172.21.15.163:0/242235442 wait complete. 2023-10-09T06:31:48.140 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:31:48.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.255+0000 7fb6c6959700 1 Processor -- start 2023-10-09T06:31:48.256 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.255+0000 7fb6c6959700 1 -- start start 2023-10-09T06:31:48.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.255+0000 7fb6c6959700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6c0159c60 0x7fb6c015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:48.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.255+0000 7fb6c6959700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6c015a160 con 0x7fb6c0159890 2023-10-09T06:31:48.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.255+0000 7fb6c6959700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6c00d13a0 con 0x7fb6c015db10 2023-10-09T06:31:48.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.255+0000 7fb6c6959700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c015e140 con 0x7fb6c0159c60 2023-10-09T06:31:48.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.256+0000 7fb6bffff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb6c0159890 0x7fb6c015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:49782/0 (socket says 172.21.15.163:49782) 2023-10-09T06:31:48.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.256+0000 7fb6bffff700 1 -- 172.21.15.163:0/3866218285 learned_addr learned my addr 172.21.15.163:0/3866218285 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:48.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.256+0000 7fb6bf7fe700 1 --2- 172.21.15.163:0/3866218285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6c0159c60 0x7fb6c015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:48.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.256+0000 7fb6beffd700 1 -- 172.21.15.163:0/3866218285 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4030940182 0 0) 0x7fb6c015a160 con 0x7fb6c0159890 2023-10-09T06:31:48.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.256+0000 7fb6beffd700 1 -- 172.21.15.163:0/3866218285 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6a8003570 con 0x7fb6c0159890 2023-10-09T06:31:48.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.256+0000 7fb6beffd700 1 -- 172.21.15.163:0/3866218285 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4144783817 0 0) 0x7fb6c00d13a0 con 0x7fb6c015db10 2023-10-09T06:31:48.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.256+0000 7fb6beffd700 1 -- 172.21.15.163:0/3866218285 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6c015a160 con 0x7fb6c015db10 2023-10-09T06:31:48.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.256+0000 7fb6beffd700 1 -- 172.21.15.163:0/3866218285 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3074875363 0 0) 0x7fb6a8003570 con 0x7fb6c0159890 2023-10-09T06:31:48.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.256+0000 7fb6bf7fe700 1 -- 172.21.15.163:0/3866218285 >> v1:172.21.15.181:6789/0 conn(0x7fb6c015db10 legacy=0x7fb6c0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:48.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.256+0000 7fb6bf7fe700 1 -- 172.21.15.163:0/3866218285 >> v1:172.21.15.163:6789/0 conn(0x7fb6c0159890 legacy=0x7fb6c015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:48.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.256+0000 7fb6bf7fe700 1 -- 172.21.15.163:0/3866218285 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c015a3d0 con 0x7fb6c0159c60 2023-10-09T06:31:48.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.256+0000 7fb6bf7fe700 1 --2- 172.21.15.163:0/3866218285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6c0159c60 0x7fb6c015d5d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4009910 tx=0x7fb6b40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d95e2bf205b97cc server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:48.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.256+0000 7fb6beffd700 1 -- 172.21.15.163:0/3866218285 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0001470 con 0x7fb6c0159c60 2023-10-09T06:31:48.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.256+0000 7fb6beffd700 1 -- 172.21.15.163:0/3866218285 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b4011420 con 0x7fb6c0159c60 2023-10-09T06:31:48.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.256+0000 7fb6beffd700 1 -- 172.21.15.163:0/3866218285 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b40117c0 con 0x7fb6c0159c60 2023-10-09T06:31:48.261 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.257+0000 7fb6c6959700 1 -- 172.21.15.163:0/3866218285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6c0159c60 msgr2=0x7fb6c015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:48.261 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.257+0000 7fb6c6959700 1 --2- 172.21.15.163:0/3866218285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6c0159c60 0x7fb6c015d5d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4009910 tx=0x7fb6b40091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:48.261 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.257+0000 7fb6c6959700 1 -- 172.21.15.163:0/3866218285 shutdown_connections 2023-10-09T06:31:48.261 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.257+0000 7fb6c6959700 1 --2- 172.21.15.163:0/3866218285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6c0159c60 0x7fb6c015d5d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:48.261 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.257+0000 7fb6c6959700 1 -- 172.21.15.163:0/3866218285 >> 172.21.15.163:0/3866218285 conn(0x7fb6c0068e20 msgr2=0x7fb6c00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:48.261 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.257+0000 7fb6c6959700 1 -- 172.21.15.163:0/3866218285 shutdown_connections 2023-10-09T06:31:48.261 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.257+0000 7fb6c6959700 1 -- 172.21.15.163:0/3866218285 wait complete. 2023-10-09T06:31:48.261 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.258+0000 7fb6c6959700 1 Processor -- start 2023-10-09T06:31:48.261 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.258+0000 7fb6c6959700 1 -- start start 2023-10-09T06:31:48.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.259+0000 7fb6c6959700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6c0159890 0x7fb6c00b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:48.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.259+0000 7fb6c6959700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6c0159c60 0x7fb6c0147380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:48.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.259+0000 7fb6c6959700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6c015db10 0x7fb6c00cf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:48.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.259+0000 7fb6c6959700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c00cf2a0 con 0x7fb6c0159c60 2023-10-09T06:31:48.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.259+0000 7fb6c6959700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c00b6460 con 0x7fb6c0159890 2023-10-09T06:31:48.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.259+0000 7fb6c6959700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c01478c0 con 0x7fb6c015db10 2023-10-09T06:31:48.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.259+0000 7fb6bf7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6c0159c60 0x7fb6c0147380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:48.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.259+0000 7fb6bf7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6c0159c60 0x7fb6c0147380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40790/0 (socket says 172.21.15.163:40790) 2023-10-09T06:31:48.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.259+0000 7fb6bf7fe700 1 -- 172.21.15.163:0/816305497 learned_addr learned my addr 172.21.15.163:0/816305497 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:48.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.259+0000 7fb6bffff700 1 --2- 172.21.15.163:0/816305497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6c0159890 0x7fb6c00b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:48.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.259+0000 7fb6bf7fe700 1 -- 172.21.15.163:0/816305497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6c015db10 msgr2=0x7fb6c00cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:48.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.259+0000 7fb6c4ef6700 1 --2- 172.21.15.163:0/816305497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6c015db10 0x7fb6c00cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:48.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.259+0000 7fb6bf7fe700 1 --2- 172.21.15.163:0/816305497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6c015db10 0x7fb6c00cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:48.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.259+0000 7fb6bf7fe700 1 -- 172.21.15.163:0/816305497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6c0159890 msgr2=0x7fb6c00b5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:48.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.259+0000 7fb6bf7fe700 1 --2- 172.21.15.163:0/816305497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6c0159890 0x7fb6c00b5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:48.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.259+0000 7fb6bf7fe700 1 -- 172.21.15.163:0/816305497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b4009420 con 0x7fb6c0159c60 2023-10-09T06:31:48.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.259+0000 7fb6bffff700 1 --2- 172.21.15.163:0/816305497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6c0159890 0x7fb6c00b5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:48.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.259+0000 7fb6c4ef6700 1 --2- 172.21.15.163:0/816305497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6c015db10 0x7fb6c00cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:48.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.260+0000 7fb6bf7fe700 1 --2- 172.21.15.163:0/816305497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6c0159c60 0x7fb6c0147380 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb6b40030c0 tx=0x7fb6b4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:48.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.260+0000 7fb6a7fff700 1 -- 172.21.15.163:0/816305497 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b401cc50 con 0x7fb6c0159c60 2023-10-09T06:31:48.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.260+0000 7fb6a7fff700 1 -- 172.21.15.163:0/816305497 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b401cdb0 con 0x7fb6c0159c60 2023-10-09T06:31:48.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.260+0000 7fb6c6959700 1 -- 172.21.15.163:0/816305497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6c00ce5e0 con 0x7fb6c0159c60 2023-10-09T06:31:48.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.260+0000 7fb6a7fff700 1 -- 172.21.15.163:0/816305497 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b401a6a0 con 0x7fb6c0159c60 2023-10-09T06:31:48.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.260+0000 7fb6c6959700 1 -- 172.21.15.163:0/816305497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c00cef30 con 0x7fb6c0159c60 2023-10-09T06:31:48.267 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.261+0000 7fb6c6959700 1 -- 172.21.15.163:0/816305497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6c00ceab0 con 0x7fb6c0159c60 2023-10-09T06:31:48.267 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.261+0000 7fb6a7fff700 1 -- 172.21.15.163:0/816305497 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb6b401aad0 con 0x7fb6c0159c60 2023-10-09T06:31:48.267 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.262+0000 7fb6a7fff700 1 --2- 172.21.15.163:0/816305497 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb6ac066f30 0x7fb6ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:48.267 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.262+0000 7fb6a7fff700 1 -- 172.21.15.163:0/816305497 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fb6b4086530 con 0x7fb6c0159c60 2023-10-09T06:31:48.267 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.262+0000 7fb6bffff700 1 --2- 172.21.15.163:0/816305497 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb6ac066f30 0x7fb6ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:48.267 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.262+0000 7fb6bffff700 1 --2- 172.21.15.163:0/816305497 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb6ac066f30 0x7fb6ac0693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb6b00015a0 tx=0x7fb6b0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:48.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.266+0000 7fb6a7fff700 1 -- 172.21.15.163:0/816305497 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6c00ceab0 con 0x7fb6c0159c60 2023-10-09T06:31:48.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.493+0000 7fb6c6959700 1 -- 172.21.15.163:0/816305497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6c00ceab0 con 0x7fb6c0159c60 2023-10-09T06:31:48.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.494+0000 7fb6a7fff700 1 -- 172.21.15.163:0/816305497 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fb6c00ceab0 con 0x7fb6c0159c60 2023-10-09T06:31:48.495 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:48.495 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4622,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2368375038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2368375038},{"type":"v1","addr":"172.21.15.181:6836","nonce":2368375038}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4929,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/4112854733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4112854733},{"type":"v1","addr":"172.21.15.163:6834","nonce":4112854733}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5202,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2200503829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2200503829},{"type":"v1","addr":"172.21.15.163:6835","nonce":2200503829}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:31:28.011264+0000","modified":"2023-10-09T06:31:31.660915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4944},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4944":{"gid":4944,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2708262268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2708262268},{"type":"v1","addr":"172.21.15.181:6837","nonce":2708262268}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-09T06:31:48.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.496+0000 7fb6c6959700 1 -- 172.21.15.163:0/816305497 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb6ac066f30 msgr2=0x7fb6ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:48.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.496+0000 7fb6c6959700 1 --2- 172.21.15.163:0/816305497 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb6ac066f30 0x7fb6ac0693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb6b00015a0 tx=0x7fb6b0009040 comp rx=0 tx=0).stop 2023-10-09T06:31:48.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.496+0000 7fb6c6959700 1 -- 172.21.15.163:0/816305497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6c0159c60 msgr2=0x7fb6c0147380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:48.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.496+0000 7fb6c6959700 1 --2- 172.21.15.163:0/816305497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6c0159c60 0x7fb6c0147380 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb6b40030c0 tx=0x7fb6b4016000 comp rx=0 tx=0).stop 2023-10-09T06:31:48.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.496+0000 7fb6c6959700 1 -- 172.21.15.163:0/816305497 shutdown_connections 2023-10-09T06:31:48.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.496+0000 7fb6c6959700 1 --2- 172.21.15.163:0/816305497 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb6ac066f30 0x7fb6ac0693e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:48.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.496+0000 7fb6c6959700 1 --2- 172.21.15.163:0/816305497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb6c015db10 0x7fb6c00cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:48.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.496+0000 7fb6c6959700 1 --2- 172.21.15.163:0/816305497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb6c0159890 0x7fb6c00b5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:48.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.496+0000 7fb6c6959700 1 --2- 172.21.15.163:0/816305497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6c0159c60 0x7fb6c0147380 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:48.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.496+0000 7fb6c6959700 1 -- 172.21.15.163:0/816305497 >> 172.21.15.163:0/816305497 conn(0x7fb6c0068e20 msgr2=0x7fb6c01586b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:48.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.496+0000 7fb6c6959700 1 -- 172.21.15.163:0/816305497 shutdown_connections 2023-10-09T06:31:48.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.496+0000 7fb6c6959700 1 -- 172.21.15.163:0/816305497 wait complete. 2023-10-09T06:31:48.498 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 23 2023-10-09T06:31:48.508 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:31:48.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.622+0000 7fbac9ed8700 1 Processor -- start 2023-10-09T06:31:48.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.622+0000 7fbac9ed8700 1 -- start start 2023-10-09T06:31:48.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.623+0000 7fbac9ed8700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbac4159890 0x7fbac415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:48.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.623+0000 7fbac9ed8700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac415a160 con 0x7fbac4069560 2023-10-09T06:31:48.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.623+0000 7fbac9ed8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac40d13a0 con 0x7fbac4159c60 2023-10-09T06:31:48.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.623+0000 7fbac9ed8700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbac415e140 con 0x7fbac4159890 2023-10-09T06:31:48.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.623+0000 7fbac37fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbac4159890 0x7fbac415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:48.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.623+0000 7fbac3fff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fbac4069560 0x7fbac40ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:44670/0 (socket says 172.21.15.163:44670) 2023-10-09T06:31:48.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.623+0000 7fbac3fff700 1 -- 172.21.15.163:0/1059725264 learned_addr learned my addr 172.21.15.163:0/1059725264 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:48.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.623+0000 7fbac37fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbac4159890 0x7fbac415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39632/0 (socket says 172.21.15.163:39632) 2023-10-09T06:31:48.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.623+0000 7fbac37fe700 1 -- 172.21.15.163:0/1059725264 >> v1:172.21.15.181:6789/0 conn(0x7fbac4159c60 legacy=0x7fbac415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:48.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.623+0000 7fbac37fe700 1 -- 172.21.15.163:0/1059725264 >> v1:172.21.15.163:6789/0 conn(0x7fbac4069560 legacy=0x7fbac40ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:48.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.623+0000 7fbac37fe700 1 -- 172.21.15.163:0/1059725264 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac415a3d0 con 0x7fbac4159890 2023-10-09T06:31:48.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.623+0000 7fbac27fc700 1 -- 172.21.15.163:0/1059725264 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224231244 0 0) 0x7fbac415a160 con 0x7fbac4069560 2023-10-09T06:31:48.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.623+0000 7fbac37fe700 1 --2- 172.21.15.163:0/1059725264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbac4159890 0x7fbac415abd0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbab40098e0 tx=0x7fbab40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1d637f7e6acaf6 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:48.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.623+0000 7fbac27fc700 1 -- 172.21.15.163:0/1059725264 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab4009e80 con 0x7fbac4159890 2023-10-09T06:31:48.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.623+0000 7fbac27fc700 1 -- 172.21.15.163:0/1059725264 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab400b860 con 0x7fbac4159890 2023-10-09T06:31:48.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.623+0000 7fbac27fc700 1 -- 172.21.15.163:0/1059725264 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab400bbe0 con 0x7fbac4159890 2023-10-09T06:31:48.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.624+0000 7fbac9ed8700 1 -- 172.21.15.163:0/1059725264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbac4159890 msgr2=0x7fbac415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:48.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.624+0000 7fbac9ed8700 1 --2- 172.21.15.163:0/1059725264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbac4159890 0x7fbac415abd0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbab40098e0 tx=0x7fbab40091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:48.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.624+0000 7fbac9ed8700 1 -- 172.21.15.163:0/1059725264 shutdown_connections 2023-10-09T06:31:48.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.624+0000 7fbac9ed8700 1 --2- 172.21.15.163:0/1059725264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbac4159890 0x7fbac415abd0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:48.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.624+0000 7fbac9ed8700 1 -- 172.21.15.163:0/1059725264 >> 172.21.15.163:0/1059725264 conn(0x7fbac4068e20 msgr2=0x7fbac40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:48.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.624+0000 7fbac9ed8700 1 -- 172.21.15.163:0/1059725264 shutdown_connections 2023-10-09T06:31:48.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.624+0000 7fbac9ed8700 1 -- 172.21.15.163:0/1059725264 wait complete. 2023-10-09T06:31:48.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.625+0000 7fbac9ed8700 1 Processor -- start 2023-10-09T06:31:48.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.625+0000 7fbac9ed8700 1 -- start start 2023-10-09T06:31:48.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.626+0000 7fbac9ed8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbac4069560 0x7fbac4154b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:48.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.626+0000 7fbac9ed8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbac4159890 0x7fbac415d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:48.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.626+0000 7fbac9ed8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac4159c60 0x7fbac415c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:48.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.626+0000 7fbac9ed8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac414cba0 con 0x7fbac4069560 2023-10-09T06:31:48.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.626+0000 7fbac9ed8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac416b440 con 0x7fbac4159c60 2023-10-09T06:31:48.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.626+0000 7fbac9ed8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbac415ca30 con 0x7fbac4159890 2023-10-09T06:31:48.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.626+0000 7fbac37fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbac4069560 0x7fbac4154b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:48.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.626+0000 7fbac37fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbac4069560 0x7fbac4154b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39642/0 (socket says 172.21.15.163:39642) 2023-10-09T06:31:48.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.626+0000 7fbac37fe700 1 -- 172.21.15.163:0/68131075 learned_addr learned my addr 172.21.15.163:0/68131075 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:48.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.626+0000 7fbac37fe700 1 -- 172.21.15.163:0/68131075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbac4159890 msgr2=0x7fbac415d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:48.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.626+0000 7fbac37fe700 1 --2- 172.21.15.163:0/68131075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbac4159890 0x7fbac415d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:48.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.626+0000 7fbac3fff700 1 --2- 172.21.15.163:0/68131075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac4159c60 0x7fbac415c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:48.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.626+0000 7fbac37fe700 1 -- 172.21.15.163:0/68131075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac4159c60 msgr2=0x7fbac415c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:48.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.626+0000 7fbac37fe700 1 --2- 172.21.15.163:0/68131075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac4159c60 0x7fbac415c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:48.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.626+0000 7fbac37fe700 1 -- 172.21.15.163:0/68131075 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab4009420 con 0x7fbac4069560 2023-10-09T06:31:48.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.626+0000 7fbac37fe700 1 --2- 172.21.15.163:0/68131075 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbac4069560 0x7fbac4154b80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbac4070e20 tx=0x7fbab4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:48.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.627+0000 7fbaab7fe700 1 -- 172.21.15.163:0/68131075 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab401f070 con 0x7fbac4069560 2023-10-09T06:31:48.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.627+0000 7fbaab7fe700 1 -- 172.21.15.163:0/68131075 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab4011420 con 0x7fbac4069560 2023-10-09T06:31:48.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.627+0000 7fbac9ed8700 1 -- 172.21.15.163:0/68131075 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbac416b160 con 0x7fbac4069560 2023-10-09T06:31:48.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.627+0000 7fbaab7fe700 1 -- 172.21.15.163:0/68131075 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab400bbe0 con 0x7fbac4069560 2023-10-09T06:31:48.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.627+0000 7fbac9ed8700 1 -- 172.21.15.163:0/68131075 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac415cfb0 con 0x7fbac4069560 2023-10-09T06:31:48.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.628+0000 7fbac9ed8700 1 -- 172.21.15.163:0/68131075 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbac404c3c0 con 0x7fbac4069560 2023-10-09T06:31:48.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.628+0000 7fbaab7fe700 1 -- 172.21.15.163:0/68131075 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbab4011a50 con 0x7fbac4069560 2023-10-09T06:31:48.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.628+0000 7fbaab7fe700 1 --2- 172.21.15.163:0/68131075 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbab0066f30 0x7fbab00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:48.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.629+0000 7fbaab7fe700 1 -- 172.21.15.163:0/68131075 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fbab4085bb0 con 0x7fbac4069560 2023-10-09T06:31:48.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.629+0000 7fbac2ffd700 1 --2- 172.21.15.163:0/68131075 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbab0066f30 0x7fbab00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:48.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.629+0000 7fbac2ffd700 1 --2- 172.21.15.163:0/68131075 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbab0066f30 0x7fbab00693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbac40cf270 tx=0x7fbab8007b40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:48.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.633+0000 7fbaab7fe700 1 -- 172.21.15.163:0/68131075 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbab4052dd0 con 0x7fbac4069560 2023-10-09T06:31:48.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.820+0000 7fbac9ed8700 1 -- 172.21.15.163:0/68131075 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbac4158ef0 con 0x7fbac4069560 2023-10-09T06:31:48.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.821+0000 7fbaab7fe700 1 -- 172.21.15.163:0/68131075 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+15529 (secure 0 0 0) 0x7fbab4015090 con 0x7fbac4069560 2023-10-09T06:31:48.821 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:48.822 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":39,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:31:46.160317+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:31:25.770415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-09T06:31:26.980879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6835/2241937708":"2023-10-10T06:31:43.143850+0000","172.21.15.163:6833/2241937708":"2023-10-10T06:31:43.143850+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:31:48.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.823+0000 7fbac9ed8700 1 -- 172.21.15.163:0/68131075 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbab0066f30 msgr2=0x7fbab00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:48.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.823+0000 7fbac9ed8700 1 --2- 172.21.15.163:0/68131075 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbab0066f30 0x7fbab00693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbac40cf270 tx=0x7fbab8007b40 comp rx=0 tx=0).stop 2023-10-09T06:31:48.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.823+0000 7fbac9ed8700 1 -- 172.21.15.163:0/68131075 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbac4069560 msgr2=0x7fbac4154b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:48.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.823+0000 7fbac9ed8700 1 --2- 172.21.15.163:0/68131075 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbac4069560 0x7fbac4154b80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbac4070e20 tx=0x7fbab4017000 comp rx=0 tx=0).stop 2023-10-09T06:31:48.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.823+0000 7fbac9ed8700 1 -- 172.21.15.163:0/68131075 shutdown_connections 2023-10-09T06:31:48.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.823+0000 7fbac9ed8700 1 --2- 172.21.15.163:0/68131075 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbab0066f30 0x7fbab00693e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:48.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.823+0000 7fbac9ed8700 1 --2- 172.21.15.163:0/68131075 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbac4069560 0x7fbac4154b80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:48.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.823+0000 7fbac9ed8700 1 --2- 172.21.15.163:0/68131075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac4159c60 0x7fbac415c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:48.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.823+0000 7fbac9ed8700 1 --2- 172.21.15.163:0/68131075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbac4159890 0x7fbac415d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:48.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.823+0000 7fbac9ed8700 1 -- 172.21.15.163:0/68131075 >> 172.21.15.163:0/68131075 conn(0x7fbac4068e20 msgr2=0x7fbac4152da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:48.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.823+0000 7fbac9ed8700 1 -- 172.21.15.163:0/68131075 shutdown_connections 2023-10-09T06:31:48.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.823+0000 7fbac9ed8700 1 -- 172.21.15.163:0/68131075 wait complete. 2023-10-09T06:31:48.836 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-09T06:31:48.837 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:31:48.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.949+0000 7f252bc25700 1 Processor -- start 2023-10-09T06:31:48.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.949+0000 7f252bc25700 1 -- start start 2023-10-09T06:31:48.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.950+0000 7f252bc25700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25241599e0 0x7f2524069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:48.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.950+0000 7f252bc25700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25240c96a0 con 0x7f252406a2b0 2023-10-09T06:31:48.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.950+0000 7f252bc25700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25240cb9e0 con 0x7f2524159db0 2023-10-09T06:31:48.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.950+0000 7f252bc25700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25240cab60 con 0x7f25241599e0 2023-10-09T06:31:48.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.950+0000 7f252a1c2700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f252406a2b0 0x7f252406a710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:44676/0 (socket says 172.21.15.163:44676) 2023-10-09T06:31:48.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.950+0000 7f252a1c2700 1 -- 172.21.15.163:0/3512741351 learned_addr learned my addr 172.21.15.163:0/3512741351 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:48.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.951+0000 7f25299c1700 1 --2- 172.21.15.163:0/3512741351 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25241599e0 0x7f2524069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:48.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.951+0000 7f25289bf700 1 -- 172.21.15.163:0/3512741351 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3093759390 0 0) 0x7f25240c96a0 con 0x7f252406a2b0 2023-10-09T06:31:48.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.951+0000 7f25289bf700 1 -- 172.21.15.163:0/3512741351 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2510003570 con 0x7f252406a2b0 2023-10-09T06:31:48.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.951+0000 7f25289bf700 1 -- 172.21.15.163:0/3512741351 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1452208794 0 0) 0x7f2510003570 con 0x7f252406a2b0 2023-10-09T06:31:48.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.951+0000 7f25289bf700 1 -- 172.21.15.163:0/3512741351 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25241599e0 msgr2=0x7f2524069ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:48.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.951+0000 7f25289bf700 1 --2- 172.21.15.163:0/3512741351 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25241599e0 0x7f2524069ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:48.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.951+0000 7f25289bf700 1 -- 172.21.15.163:0/3512741351 >> v1:172.21.15.181:6789/0 conn(0x7f2524159db0 legacy=0x7f2524069080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:48.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.951+0000 7f25289bf700 1 -- 172.21.15.163:0/3512741351 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25240cb420 con 0x7f252406a2b0 2023-10-09T06:31:48.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.951+0000 7f25289bf700 1 -- 172.21.15.163:0/3512741351 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f251c0017f0 con 0x7f252406a2b0 2023-10-09T06:31:48.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.951+0000 7f25289bf700 1 -- 172.21.15.163:0/3512741351 >> v1:172.21.15.163:6789/0 conn(0x7f252406a2b0 legacy=0x7f252406a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:48.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.951+0000 7f25289bf700 1 --2- 172.21.15.163:0/3512741351 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25241599e0 0x7f2510003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:48.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.951+0000 7f25289bf700 1 --2- 172.21.15.163:0/3512741351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2510004010 0x7f2510004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:48.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.951+0000 7f25289bf700 1 --2- 172.21.15.163:0/3512741351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2510004d50 0x7f25100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:48.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.951+0000 7f25289bf700 1 -- 172.21.15.163:0/3512741351 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25240cab60 con 0x7f2510004d50 2023-10-09T06:31:48.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.951+0000 7f25289bf700 1 -- 172.21.15.163:0/3512741351 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251000bbf0 con 0x7f2510004010 2023-10-09T06:31:48.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.951+0000 7f25289bf700 1 -- 172.21.15.163:0/3512741351 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251000bdb0 con 0x7f25241599e0 2023-10-09T06:31:48.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.951+0000 7f252a1c2700 1 --2- 172.21.15.163:0/3512741351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2510004d50 0x7f25100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:48.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.951+0000 7f25299c1700 1 --2- 172.21.15.163:0/3512741351 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25241599e0 0x7f2510003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:48.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.951+0000 7f25291c0700 1 --2- 172.21.15.163:0/3512741351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2510004010 0x7f2510004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:48.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.951+0000 7f252a1c2700 1 -- 172.21.15.163:0/3512741351 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25241599e0 msgr2=0x7f2510003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:48.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.951+0000 7f252a1c2700 1 --2- 172.21.15.163:0/3512741351 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25241599e0 0x7f2510003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:48.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.951+0000 7f252a1c2700 1 -- 172.21.15.163:0/3512741351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2510004010 msgr2=0x7f2510004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:48.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.951+0000 7f252a1c2700 1 --2- 172.21.15.163:0/3512741351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2510004010 0x7f2510004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:48.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.951+0000 7f252a1c2700 1 -- 172.21.15.163:0/3512741351 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2510003570 con 0x7f2510004d50 2023-10-09T06:31:48.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.952+0000 7f252a1c2700 1 --2- 172.21.15.163:0/3512741351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2510004d50 0x7f25100051c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f251c00a900 tx=0x7f251c0021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5156c359e90ce00c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:48.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.952+0000 7f25289bf700 1 -- 172.21.15.163:0/3512741351 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c0070d0 con 0x7f2510004d50 2023-10-09T06:31:48.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.952+0000 7f25289bf700 1 -- 172.21.15.163:0/3512741351 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f251c007230 con 0x7f2510004d50 2023-10-09T06:31:48.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.952+0000 7f25289bf700 1 -- 172.21.15.163:0/3512741351 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c0075f0 con 0x7f2510004d50 2023-10-09T06:31:48.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.952+0000 7f252bc25700 1 -- 172.21.15.163:0/3512741351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2510004d50 msgr2=0x7f25100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:48.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.952+0000 7f252bc25700 1 --2- 172.21.15.163:0/3512741351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2510004d50 0x7f25100051c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f251c00a900 tx=0x7f251c0021d0 comp rx=0 tx=0).stop 2023-10-09T06:31:48.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.952+0000 7f252a1c2700 1 -- 172.21.15.163:0/3512741351 reap_dead start 2023-10-09T06:31:48.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.952+0000 7f252bc25700 1 -- 172.21.15.163:0/3512741351 shutdown_connections 2023-10-09T06:31:48.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.952+0000 7f252bc25700 1 -- 172.21.15.163:0/3512741351 >> 172.21.15.163:0/3512741351 conn(0x7f25240c9860 msgr2=0x7f25240c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:48.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.952+0000 7f252bc25700 1 -- 172.21.15.163:0/3512741351 shutdown_connections 2023-10-09T06:31:48.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.952+0000 7f252bc25700 1 -- 172.21.15.163:0/3512741351 wait complete. 2023-10-09T06:31:48.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.953+0000 7f252bc25700 1 Processor -- start 2023-10-09T06:31:48.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.953+0000 7f252bc25700 1 -- start start 2023-10-09T06:31:48.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.954+0000 7f252bc25700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2524155560 0x7f25240c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:48.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.954+0000 7f252bc25700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25240c3ea0 0x7f25240b1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:48.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.954+0000 7f252bc25700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25240c6370 0x7f25240c67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:48.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.954+0000 7f252bc25700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25240c5710 con 0x7f25240c3ea0 2023-10-09T06:31:48.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.954+0000 7f252bc25700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25240c6d30 con 0x7f2524155560 2023-10-09T06:31:48.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.954+0000 7f252bc25700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f252415cf00 con 0x7f25240c6370 2023-10-09T06:31:48.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.954+0000 7f25291c0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25240c3ea0 0x7f25240b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:48.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.954+0000 7f25291c0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25240c3ea0 0x7f25240b1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39656/0 (socket says 172.21.15.163:39656) 2023-10-09T06:31:48.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.954+0000 7f25291c0700 1 -- 172.21.15.163:0/93104892 learned_addr learned my addr 172.21.15.163:0/93104892 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:48.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.954+0000 7f25299c1700 1 --2- 172.21.15.163:0/93104892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2524155560 0x7f25240c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:48.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.954+0000 7f252a1c2700 1 --2- 172.21.15.163:0/93104892 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25240c6370 0x7f25240c67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:48.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.954+0000 7f25291c0700 1 -- 172.21.15.163:0/93104892 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25240c6370 msgr2=0x7f25240c67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:48.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.954+0000 7f25291c0700 1 --2- 172.21.15.163:0/93104892 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25240c6370 0x7f25240c67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:48.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.954+0000 7f25291c0700 1 -- 172.21.15.163:0/93104892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2524155560 msgr2=0x7f25240c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:48.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.954+0000 7f25291c0700 1 --2- 172.21.15.163:0/93104892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2524155560 0x7f25240c4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:48.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.954+0000 7f25291c0700 1 -- 172.21.15.163:0/93104892 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25240b1690 con 0x7f25240c3ea0 2023-10-09T06:31:48.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.954+0000 7f25299c1700 1 --2- 172.21.15.163:0/93104892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2524155560 0x7f25240c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:48.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.955+0000 7f25291c0700 1 --2- 172.21.15.163:0/93104892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25240c3ea0 0x7f25240b1150 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2514002950 tx=0x7f2514007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:48.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.955+0000 7f2519ffb700 1 -- 172.21.15.163:0/93104892 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251400f8d0 con 0x7f25240c3ea0 2023-10-09T06:31:48.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.955+0000 7f2519ffb700 1 -- 172.21.15.163:0/93104892 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f251400fa30 con 0x7f25240c3ea0 2023-10-09T06:31:48.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.955+0000 7f252bc25700 1 -- 172.21.15.163:0/93104892 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25240c5430 con 0x7f25240c3ea0 2023-10-09T06:31:48.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.955+0000 7f2519ffb700 1 -- 172.21.15.163:0/93104892 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251400fdf0 con 0x7f25240c3ea0 2023-10-09T06:31:48.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.955+0000 7f252bc25700 1 -- 172.21.15.163:0/93104892 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f252415a620 con 0x7f25240c3ea0 2023-10-09T06:31:48.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.956+0000 7f252bc25700 1 -- 172.21.15.163:0/93104892 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f252415a1a0 con 0x7f25240c3ea0 2023-10-09T06:31:48.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.956+0000 7f2519ffb700 1 -- 172.21.15.163:0/93104892 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2514013600 con 0x7f25240c3ea0 2023-10-09T06:31:48.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.957+0000 7f2519ffb700 1 --2- 172.21.15.163:0/93104892 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2508066f30 0x7f25080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:48.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.957+0000 7f2519ffb700 1 -- 172.21.15.163:0/93104892 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f2514084a90 con 0x7f25240c3ea0 2023-10-09T06:31:48.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.957+0000 7f25299c1700 1 --2- 172.21.15.163:0/93104892 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2508066f30 0x7f25080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:48.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.957+0000 7f25299c1700 1 --2- 172.21.15.163:0/93104892 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2508066f30 0x7f25080693e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f25240c27b0 tx=0x7f2520008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:48.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:48.961+0000 7f2519ffb700 1 -- 172.21.15.163:0/93104892 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f252415a1a0 con 0x7f25240c3ea0 2023-10-09T06:31:49.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.186+0000 7f252bc25700 1 -- 172.21.15.163:0/93104892 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f252415a1a0 con 0x7f25240c3ea0 2023-10-09T06:31:49.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.187+0000 7f2519ffb700 1 -- 172.21.15.163:0/93104892 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f252415a1a0 con 0x7f25240c3ea0 2023-10-09T06:31:49.188 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:49.188 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4622,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2368375038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2368375038},{"type":"v1","addr":"172.21.15.181:6836","nonce":2368375038}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4929,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/4112854733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4112854733},{"type":"v1","addr":"172.21.15.163:6834","nonce":4112854733}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5202,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2200503829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2200503829},{"type":"v1","addr":"172.21.15.163:6835","nonce":2200503829}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:31:28.011264+0000","modified":"2023-10-09T06:31:31.660915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4944},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4944":{"gid":4944,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2708262268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2708262268},{"type":"v1","addr":"172.21.15.181:6837","nonce":2708262268}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-09T06:31:49.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.189+0000 7f252bc25700 1 -- 172.21.15.163:0/93104892 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2508066f30 msgr2=0x7f25080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:49.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.189+0000 7f252bc25700 1 --2- 172.21.15.163:0/93104892 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2508066f30 0x7f25080693e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f25240c27b0 tx=0x7f2520008040 comp rx=0 tx=0).stop 2023-10-09T06:31:49.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.189+0000 7f252bc25700 1 -- 172.21.15.163:0/93104892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25240c3ea0 msgr2=0x7f25240b1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:49.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.189+0000 7f252bc25700 1 --2- 172.21.15.163:0/93104892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25240c3ea0 0x7f25240b1150 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2514002950 tx=0x7f2514007d30 comp rx=0 tx=0).stop 2023-10-09T06:31:49.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.189+0000 7f252bc25700 1 -- 172.21.15.163:0/93104892 shutdown_connections 2023-10-09T06:31:49.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.189+0000 7f252bc25700 1 --2- 172.21.15.163:0/93104892 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2508066f30 0x7f25080693e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:49.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.189+0000 7f252bc25700 1 --2- 172.21.15.163:0/93104892 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25240c6370 0x7f25240c67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:49.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.189+0000 7f252bc25700 1 --2- 172.21.15.163:0/93104892 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2524155560 0x7f25240c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:49.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.189+0000 7f252bc25700 1 --2- 172.21.15.163:0/93104892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25240c3ea0 0x7f25240b1150 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:49.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.189+0000 7f252bc25700 1 -- 172.21.15.163:0/93104892 >> 172.21.15.163:0/93104892 conn(0x7f25240c9860 msgr2=0x7f25240c29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:49.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.189+0000 7f252bc25700 1 -- 172.21.15.163:0/93104892 shutdown_connections 2023-10-09T06:31:49.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.189+0000 7f252bc25700 1 -- 172.21.15.163:0/93104892 wait complete. 2023-10-09T06:31:49.191 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 23 2023-10-09T06:31:49.202 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:31:49.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.316+0000 7fec0bde2700 1 Processor -- start 2023-10-09T06:31:49.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.316+0000 7fec0bde2700 1 -- start start 2023-10-09T06:31:49.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.317+0000 7fec0bde2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fec04141500 0x7fec040c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:49.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.317+0000 7fec0bde2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec0406c0a0 0x7fec040c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:49.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.317+0000 7fec0bde2700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec0415c2a0 con 0x7fec0406bcd0 2023-10-09T06:31:49.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.317+0000 7fec0bde2700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec040c9880 con 0x7fec04141500 2023-10-09T06:31:49.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.317+0000 7fec0bde2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec0415b690 con 0x7fec0406c0a0 2023-10-09T06:31:49.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.317+0000 7fec0937d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fec04141500 0x7fec040c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:49.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.317+0000 7fec09b7e700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fec0406bcd0 0x7fec04140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:44692/0 (socket says 172.21.15.163:44692) 2023-10-09T06:31:49.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.317+0000 7fec09b7e700 1 -- 172.21.15.163:0/2016048998 learned_addr learned my addr 172.21.15.163:0/2016048998 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:49.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.317+0000 7fec0a37f700 1 --2- 172.21.15.163:0/2016048998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec0406c0a0 0x7fec040c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:49.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.317+0000 7fec0937d700 1 -- 172.21.15.163:0/2016048998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec0406c0a0 msgr2=0x7fec040c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:49.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.317+0000 7fec0937d700 1 --2- 172.21.15.163:0/2016048998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec0406c0a0 0x7fec040c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:49.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.317+0000 7fec0937d700 1 -- 172.21.15.163:0/2016048998 >> v1:172.21.15.163:6789/0 conn(0x7fec0406bcd0 legacy=0x7fec04140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:49.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.317+0000 7fec0937d700 1 -- 172.21.15.163:0/2016048998 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec0415b970 con 0x7fec04141500 2023-10-09T06:31:49.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.318+0000 7fec0937d700 1 --2- 172.21.15.163:0/2016048998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec04141500 0x7fec040c8e00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7febf80098e0 tx=0x7febf80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bae29a97fac2251 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:49.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.318+0000 7fec08b7c700 1 -- 172.21.15.163:0/2016048998 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf8009e80 con 0x7fec04141500 2023-10-09T06:31:49.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.318+0000 7fec08b7c700 1 -- 172.21.15.163:0/2016048998 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf800b860 con 0x7fec04141500 2023-10-09T06:31:49.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.318+0000 7fec08b7c700 1 -- 172.21.15.163:0/2016048998 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf800bbe0 con 0x7fec04141500 2023-10-09T06:31:49.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.318+0000 7fec0bde2700 1 -- 172.21.15.163:0/2016048998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec04141500 msgr2=0x7fec040c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:49.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.318+0000 7fec0bde2700 1 --2- 172.21.15.163:0/2016048998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec04141500 0x7fec040c8e00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7febf80098e0 tx=0x7febf80091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:49.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.318+0000 7fec0bde2700 1 -- 172.21.15.163:0/2016048998 shutdown_connections 2023-10-09T06:31:49.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.318+0000 7fec0bde2700 1 --2- 172.21.15.163:0/2016048998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec0406c0a0 0x7fec040c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:49.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.318+0000 7fec0bde2700 1 --2- 172.21.15.163:0/2016048998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec04141500 0x7fec040c8e00 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:49.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.318+0000 7fec0bde2700 1 -- 172.21.15.163:0/2016048998 >> 172.21.15.163:0/2016048998 conn(0x7fec040ce600 msgr2=0x7fec040cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:49.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.318+0000 7fec0bde2700 1 -- 172.21.15.163:0/2016048998 shutdown_connections 2023-10-09T06:31:49.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.318+0000 7fec0bde2700 1 -- 172.21.15.163:0/2016048998 wait complete. 2023-10-09T06:31:49.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.319+0000 7fec0bde2700 1 Processor -- start 2023-10-09T06:31:49.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.319+0000 7fec0bde2700 1 -- start start 2023-10-09T06:31:49.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.320+0000 7fec0bde2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec0406bcd0 0x7fec0413f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:49.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.320+0000 7fec0bde2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec0406c0a0 0x7fec040c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:49.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.320+0000 7fec0bde2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec04141500 0x7fec040b2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:49.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.320+0000 7fec0bde2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0416e1f0 con 0x7fec0406c0a0 2023-10-09T06:31:49.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.320+0000 7fec0bde2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec040b41b0 con 0x7fec04141500 2023-10-09T06:31:49.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.320+0000 7fec0bde2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec0416d5e0 con 0x7fec0406bcd0 2023-10-09T06:31:49.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.320+0000 7fec0937d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec0406c0a0 0x7fec040c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:49.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.320+0000 7fec0937d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec0406c0a0 0x7fec040c8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39668/0 (socket says 172.21.15.163:39668) 2023-10-09T06:31:49.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.320+0000 7fec0937d700 1 -- 172.21.15.163:0/3018345489 learned_addr learned my addr 172.21.15.163:0/3018345489 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:49.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.321+0000 7fec09b7e700 1 --2- 172.21.15.163:0/3018345489 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec0406bcd0 0x7fec0413f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:49.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.321+0000 7fec0937d700 1 -- 172.21.15.163:0/3018345489 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec0406bcd0 msgr2=0x7fec0413f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:49.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.321+0000 7fec0937d700 1 --2- 172.21.15.163:0/3018345489 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec0406bcd0 0x7fec0413f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:49.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.321+0000 7fec0937d700 1 -- 172.21.15.163:0/3018345489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec04141500 msgr2=0x7fec040b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:49.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.321+0000 7fec0937d700 1 --2- 172.21.15.163:0/3018345489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec04141500 0x7fec040b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:49.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.321+0000 7fec0937d700 1 -- 172.21.15.163:0/3018345489 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf8009420 con 0x7fec0406c0a0 2023-10-09T06:31:49.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.321+0000 7fec0937d700 1 --2- 172.21.15.163:0/3018345489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec0406c0a0 0x7fec040c8500 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7febf800b3c0 tx=0x7febf8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:49.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.321+0000 7fec01ffb700 1 -- 172.21.15.163:0/3018345489 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf800bba0 con 0x7fec0406c0a0 2023-10-09T06:31:49.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.321+0000 7fec01ffb700 1 -- 172.21.15.163:0/3018345489 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf8025420 con 0x7fec0406c0a0 2023-10-09T06:31:49.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.321+0000 7fec0bde2700 1 -- 172.21.15.163:0/3018345489 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec0416d8c0 con 0x7fec0406c0a0 2023-10-09T06:31:49.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.321+0000 7fec01ffb700 1 -- 172.21.15.163:0/3018345489 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf80257e0 con 0x7fec0406c0a0 2023-10-09T06:31:49.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.321+0000 7fec0bde2700 1 -- 172.21.15.163:0/3018345489 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec0416de80 con 0x7fec0406c0a0 2023-10-09T06:31:49.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.322+0000 7fec0bde2700 1 -- 172.21.15.163:0/3018345489 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec0416e3a0 con 0x7fec0406c0a0 2023-10-09T06:31:49.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.322+0000 7fec01ffb700 1 -- 172.21.15.163:0/3018345489 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7febf8023070 con 0x7fec0406c0a0 2023-10-09T06:31:49.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.323+0000 7fec01ffb700 1 --2- 172.21.15.163:0/3018345489 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7febf0066f30 0x7febf00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:49.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.323+0000 7fec01ffb700 1 -- 172.21.15.163:0/3018345489 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5791+0+0 (secure 0 0 0) 0x7febf8085cd0 con 0x7fec0406c0a0 2023-10-09T06:31:49.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.323+0000 7fec09b7e700 1 --2- 172.21.15.163:0/3018345489 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7febf0066f30 0x7febf00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:49.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.324+0000 7fec09b7e700 1 --2- 172.21.15.163:0/3018345489 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7febf0066f30 0x7febf00693e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7febfc001780 tx=0x7febfc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:49.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.327+0000 7fec01ffb700 1 -- 172.21.15.163:0/3018345489 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec0416e3a0 con 0x7fec0406c0a0 2023-10-09T06:31:49.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.552+0000 7fec0bde2700 1 -- 172.21.15.163:0/3018345489 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec0416e3a0 con 0x7fec0406c0a0 2023-10-09T06:31:49.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.553+0000 7fec01ffb700 1 -- 172.21.15.163:0/3018345489 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fec0416e3a0 con 0x7fec0406c0a0 2023-10-09T06:31:49.554 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:49.554 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4622,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2368375038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2368375038},{"type":"v1","addr":"172.21.15.181:6836","nonce":2368375038}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4929,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/4112854733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4112854733},{"type":"v1","addr":"172.21.15.163:6834","nonce":4112854733}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5202,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2200503829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2200503829},{"type":"v1","addr":"172.21.15.163:6835","nonce":2200503829}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:31:28.011264+0000","modified":"2023-10-09T06:31:31.660915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4944},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4944":{"gid":4944,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2708262268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2708262268},{"type":"v1","addr":"172.21.15.181:6837","nonce":2708262268}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-09T06:31:49.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.555+0000 7fec0bde2700 1 -- 172.21.15.163:0/3018345489 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7febf0066f30 msgr2=0x7febf00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:49.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.555+0000 7fec0bde2700 1 --2- 172.21.15.163:0/3018345489 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7febf0066f30 0x7febf00693e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7febfc001780 tx=0x7febfc009040 comp rx=0 tx=0).stop 2023-10-09T06:31:49.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.555+0000 7fec0bde2700 1 -- 172.21.15.163:0/3018345489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec0406c0a0 msgr2=0x7fec040c8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:49.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.555+0000 7fec0bde2700 1 --2- 172.21.15.163:0/3018345489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec0406c0a0 0x7fec040c8500 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7febf800b3c0 tx=0x7febf8017000 comp rx=0 tx=0).stop 2023-10-09T06:31:49.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.555+0000 7fec0bde2700 1 -- 172.21.15.163:0/3018345489 shutdown_connections 2023-10-09T06:31:49.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.555+0000 7fec0bde2700 1 --2- 172.21.15.163:0/3018345489 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7febf0066f30 0x7febf00693e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:49.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.555+0000 7fec0bde2700 1 --2- 172.21.15.163:0/3018345489 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec04141500 0x7fec040b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:49.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.555+0000 7fec0bde2700 1 --2- 172.21.15.163:0/3018345489 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec0406bcd0 0x7fec0413f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:49.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.555+0000 7fec0bde2700 1 --2- 172.21.15.163:0/3018345489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec0406c0a0 0x7fec040c8500 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:49.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.555+0000 7fec0bde2700 1 -- 172.21.15.163:0/3018345489 >> 172.21.15.163:0/3018345489 conn(0x7fec040ce600 msgr2=0x7fec040cd330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:49.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.555+0000 7fec0bde2700 1 -- 172.21.15.163:0/3018345489 shutdown_connections 2023-10-09T06:31:49.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.555+0000 7fec0bde2700 1 -- 172.21.15.163:0/3018345489 wait complete. 2023-10-09T06:31:49.558 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 23 2023-10-09T06:31:49.568 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:31:49.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.681+0000 7f948bef2700 1 Processor -- start 2023-10-09T06:31:49.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.681+0000 7f948bef2700 1 -- start start 2023-10-09T06:31:49.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.682+0000 7f948bef2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9484159c60 0x7f948415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:49.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.682+0000 7f948bef2700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f948415a160 con 0x7f948415db10 2023-10-09T06:31:49.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.682+0000 7f948bef2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94840d13a0 con 0x7f9484159890 2023-10-09T06:31:49.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.682+0000 7f948bef2700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f948415df50 con 0x7f9484159c60 2023-10-09T06:31:49.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.682+0000 7f948948d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9484159c60 0x7f948415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:49.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.682+0000 7f948a48f700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f948415db10 0x7f9484069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:44708/0 (socket says 172.21.15.163:44708) 2023-10-09T06:31:49.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.682+0000 7f948a48f700 1 -- 172.21.15.163:0/1744919122 learned_addr learned my addr 172.21.15.163:0/1744919122 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:49.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.683+0000 7f948948d700 1 -- 172.21.15.163:0/1744919122 >> v1:172.21.15.181:6789/0 conn(0x7f9484159890 legacy=0x7f948415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:49.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.683+0000 7f948948d700 1 -- 172.21.15.163:0/1744919122 >> v1:172.21.15.163:6789/0 conn(0x7f948415db10 legacy=0x7f9484069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:49.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.683+0000 7f9488c8c700 1 -- 172.21.15.163:0/1744919122 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3387105157 0 0) 0x7f948415a160 con 0x7f948415db10 2023-10-09T06:31:49.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.683+0000 7f948948d700 1 -- 172.21.15.163:0/1744919122 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948415e140 con 0x7f9484159c60 2023-10-09T06:31:49.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.683+0000 7f948948d700 1 --2- 172.21.15.163:0/1744919122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9484159c60 0x7f948415d5d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f94740098e0 tx=0x7f94740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef51c43a052c221 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:49.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.683+0000 7f9488c8c700 1 -- 172.21.15.163:0/1744919122 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9474009e80 con 0x7f9484159c60 2023-10-09T06:31:49.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.683+0000 7f9488c8c700 1 -- 172.21.15.163:0/1744919122 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947400b820 con 0x7f9484159c60 2023-10-09T06:31:49.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.683+0000 7f9488c8c700 1 -- 172.21.15.163:0/1744919122 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947400bba0 con 0x7f9484159c60 2023-10-09T06:31:49.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.683+0000 7f948bef2700 1 -- 172.21.15.163:0/1744919122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9484159c60 msgr2=0x7f948415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:49.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.683+0000 7f948bef2700 1 --2- 172.21.15.163:0/1744919122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9484159c60 0x7f948415d5d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f94740098e0 tx=0x7f94740091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:49.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.684+0000 7f948bef2700 1 -- 172.21.15.163:0/1744919122 shutdown_connections 2023-10-09T06:31:49.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.684+0000 7f948bef2700 1 --2- 172.21.15.163:0/1744919122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9484159c60 0x7f948415d5d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:49.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.684+0000 7f948bef2700 1 -- 172.21.15.163:0/1744919122 >> 172.21.15.163:0/1744919122 conn(0x7f9484068e20 msgr2=0x7f94840d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:49.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.684+0000 7f948bef2700 1 -- 172.21.15.163:0/1744919122 shutdown_connections 2023-10-09T06:31:49.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.684+0000 7f948bef2700 1 -- 172.21.15.163:0/1744919122 wait complete. 2023-10-09T06:31:49.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.685+0000 7f948bef2700 1 Processor -- start 2023-10-09T06:31:49.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.685+0000 7f948bef2700 1 -- start start 2023-10-09T06:31:49.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.685+0000 7f948bef2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9484159890 0x7f94841474b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:49.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.685+0000 7f948bef2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9484159c60 0x7f94840cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:49.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.685+0000 7f948bef2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f948415db10 0x7f94840b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:49.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.685+0000 7f948bef2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948416db40 con 0x7f9484159890 2023-10-09T06:31:49.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.685+0000 7f948bef2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948416cf70 con 0x7f9484159c60 2023-10-09T06:31:49.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.685+0000 7f948bef2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948416d210 con 0x7f948415db10 2023-10-09T06:31:49.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.685+0000 7f9489c8e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9484159890 0x7f94841474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:49.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.686+0000 7f9489c8e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9484159890 0x7f94841474b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39672/0 (socket says 172.21.15.163:39672) 2023-10-09T06:31:49.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.686+0000 7f9489c8e700 1 -- 172.21.15.163:0/2252720361 learned_addr learned my addr 172.21.15.163:0/2252720361 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:49.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.686+0000 7f948948d700 1 --2- 172.21.15.163:0/2252720361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9484159c60 0x7f94840cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:49.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.686+0000 7f948a48f700 1 --2- 172.21.15.163:0/2252720361 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f948415db10 0x7f94840b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:49.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.686+0000 7f9489c8e700 1 -- 172.21.15.163:0/2252720361 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f948415db10 msgr2=0x7f94840b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:49.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.686+0000 7f9489c8e700 1 --2- 172.21.15.163:0/2252720361 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f948415db10 0x7f94840b1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:49.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.686+0000 7f9489c8e700 1 -- 172.21.15.163:0/2252720361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9484159c60 msgr2=0x7f94840cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:49.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.686+0000 7f9489c8e700 1 --2- 172.21.15.163:0/2252720361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9484159c60 0x7f94840cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:49.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.686+0000 7f9489c8e700 1 -- 172.21.15.163:0/2252720361 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9474009420 con 0x7f9484159890 2023-10-09T06:31:49.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.686+0000 7f948a48f700 1 --2- 172.21.15.163:0/2252720361 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f948415db10 0x7f94840b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:49.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.686+0000 7f9489c8e700 1 --2- 172.21.15.163:0/2252720361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9484159890 0x7f94841474b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f948000b5e0 tx=0x7f9480009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:49.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.686+0000 7f9479ffb700 1 -- 172.21.15.163:0/2252720361 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948000f510 con 0x7f9484159890 2023-10-09T06:31:49.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.686+0000 7f9479ffb700 1 -- 172.21.15.163:0/2252720361 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948000f670 con 0x7f9484159890 2023-10-09T06:31:49.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.686+0000 7f948bef2700 1 -- 172.21.15.163:0/2252720361 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f948416d4f0 con 0x7f9484159890 2023-10-09T06:31:49.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.686+0000 7f9479ffb700 1 -- 172.21.15.163:0/2252720361 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948000fa30 con 0x7f9484159890 2023-10-09T06:31:49.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.686+0000 7f948bef2700 1 -- 172.21.15.163:0/2252720361 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94840cf130 con 0x7f9484159890 2023-10-09T06:31:49.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.687+0000 7f948bef2700 1 -- 172.21.15.163:0/2252720361 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f948404c3c0 con 0x7f9484159890 2023-10-09T06:31:49.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.687+0000 7f9479ffb700 1 -- 172.21.15.163:0/2252720361 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f948001a400 con 0x7f9484159890 2023-10-09T06:31:49.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.688+0000 7f9479ffb700 1 --2- 172.21.15.163:0/2252720361 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9468066f30 0x7f94680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:49.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.688+0000 7f9479ffb700 1 -- 172.21.15.163:0/2252720361 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f94800840b0 con 0x7f9484159890 2023-10-09T06:31:49.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.688+0000 7f948948d700 1 --2- 172.21.15.163:0/2252720361 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9468066f30 0x7f94680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:49.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.689+0000 7f948948d700 1 --2- 172.21.15.163:0/2252720361 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9468066f30 0x7f94680693e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f94740030c0 tx=0x7f9474016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:49.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.692+0000 7f9479ffb700 1 -- 172.21.15.163:0/2252720361 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9480051350 con 0x7f9484159890 2023-10-09T06:31:49.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.874+0000 7f948bef2700 1 -- 172.21.15.163:0/2252720361 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f948406a5b0 con 0x7f9484159890 2023-10-09T06:31:49.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.874+0000 7f9479ffb700 1 -- 172.21.15.163:0/2252720361 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+15529 (secure 0 0 0) 0x7f9480054970 con 0x7f9484159890 2023-10-09T06:31:49.875 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:49.876 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":39,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:31:46.160317+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:31:25.770415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-09T06:31:26.980879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6835/2241937708":"2023-10-10T06:31:43.143850+0000","172.21.15.163:6833/2241937708":"2023-10-10T06:31:43.143850+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:31:49.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.877+0000 7f948bef2700 1 -- 172.21.15.163:0/2252720361 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9468066f30 msgr2=0x7f94680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:49.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.877+0000 7f948bef2700 1 --2- 172.21.15.163:0/2252720361 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9468066f30 0x7f94680693e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f94740030c0 tx=0x7f9474016000 comp rx=0 tx=0).stop 2023-10-09T06:31:49.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.877+0000 7f948bef2700 1 -- 172.21.15.163:0/2252720361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9484159890 msgr2=0x7f94841474b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:49.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.877+0000 7f948bef2700 1 --2- 172.21.15.163:0/2252720361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9484159890 0x7f94841474b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f948000b5e0 tx=0x7f9480009120 comp rx=0 tx=0).stop 2023-10-09T06:31:49.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.877+0000 7f948bef2700 1 -- 172.21.15.163:0/2252720361 shutdown_connections 2023-10-09T06:31:49.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.877+0000 7f948bef2700 1 --2- 172.21.15.163:0/2252720361 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9468066f30 0x7f94680693e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:49.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.877+0000 7f948bef2700 1 --2- 172.21.15.163:0/2252720361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9484159890 0x7f94841474b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:49.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.877+0000 7f948bef2700 1 --2- 172.21.15.163:0/2252720361 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f948415db10 0x7f94840b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:49.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.877+0000 7f948bef2700 1 --2- 172.21.15.163:0/2252720361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9484159c60 0x7f94840cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:49.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.877+0000 7f948bef2700 1 -- 172.21.15.163:0/2252720361 >> 172.21.15.163:0/2252720361 conn(0x7f9484068e20 msgr2=0x7f94841574d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:49.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.877+0000 7f948bef2700 1 -- 172.21.15.163:0/2252720361 shutdown_connections 2023-10-09T06:31:49.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:49.877+0000 7f948bef2700 1 -- 172.21.15.163:0/2252720361 wait complete. 2023-10-09T06:31:49.892 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-09T06:31:50.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.006+0000 7fe6f6ed3700 1 Processor -- start 2023-10-09T06:31:50.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.006+0000 7fe6f6ed3700 1 -- start start 2023-10-09T06:31:50.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.007+0000 7fe6f6ed3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe6f006c0a0 0x7fe6f00caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:50.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.007+0000 7fe6f6ed3700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe6f015df50 0x7fe6f015e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:50.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.007+0000 7fe6f6ed3700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f015d7e0 con 0x7fe6f006bcd0 2023-10-09T06:31:50.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.007+0000 7fe6f6ed3700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f015dac0 con 0x7fe6f015df50 2023-10-09T06:31:50.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.007+0000 7fe6f6ed3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f015aa30 con 0x7fe6f006c0a0 2023-10-09T06:31:50.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.007+0000 7fe6f5470700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe6f015df50 0x7fe6f015e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:50.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.007+0000 7fe6f4c6f700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fe6f006bcd0 0x7fe6f015a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:44716/0 (socket says 172.21.15.163:44716) 2023-10-09T06:31:50.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.007+0000 7fe6f4c6f700 1 -- 172.21.15.163:0/1464169782 learned_addr learned my addr 172.21.15.163:0/1464169782 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:50.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.008+0000 7fe6effff700 1 --2- 172.21.15.163:0/1464169782 >> v2:172.21.15.181:3300/0 conn(0x7fe6f006c0a0 0x7fe6f00caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:50.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.008+0000 7fe6f5470700 1 -- 172.21.15.163:0/1464169782 >> v2:172.21.15.181:3300/0 conn(0x7fe6f006c0a0 msgr2=0x7fe6f00caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:50.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.008+0000 7fe6f5470700 1 --2- 172.21.15.163:0/1464169782 >> v2:172.21.15.181:3300/0 conn(0x7fe6f006c0a0 0x7fe6f00caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:50.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.008+0000 7fe6f5470700 1 -- 172.21.15.163:0/1464169782 >> v1:172.21.15.163:6789/0 conn(0x7fe6f006bcd0 legacy=0x7fe6f015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:50.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.008+0000 7fe6f5470700 1 -- 172.21.15.163:0/1464169782 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f015ad10 con 0x7fe6f015df50 2023-10-09T06:31:50.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.008+0000 7fe6ef7fe700 1 -- 172.21.15.163:0/1464169782 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 837326539 0 0) 0x7fe6f015d7e0 con 0x7fe6f006bcd0 2023-10-09T06:31:50.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.008+0000 7fe6effff700 1 --2- 172.21.15.163:0/1464169782 >> v2:172.21.15.181:3300/0 conn(0x7fe6f006c0a0 0x7fe6f00caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:50.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.008+0000 7fe6f5470700 1 --2- 172.21.15.163:0/1464169782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6f015df50 0x7fe6f015e3b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe6f006b4e0 tx=0x7fe6e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91d3a75f760132e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:50.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.008+0000 7fe6ef7fe700 1 -- 172.21.15.163:0/1464169782 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e800b6b0 con 0x7fe6f015df50 2023-10-09T06:31:50.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.008+0000 7fe6ef7fe700 1 -- 172.21.15.163:0/1464169782 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e800b810 con 0x7fe6f015df50 2023-10-09T06:31:50.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.008+0000 7fe6ef7fe700 1 -- 172.21.15.163:0/1464169782 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e800bbb0 con 0x7fe6f015df50 2023-10-09T06:31:50.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.008+0000 7fe6f6ed3700 1 -- 172.21.15.163:0/1464169782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6f015df50 msgr2=0x7fe6f015e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:50.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.008+0000 7fe6f6ed3700 1 --2- 172.21.15.163:0/1464169782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6f015df50 0x7fe6f015e3b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe6f006b4e0 tx=0x7fe6e80091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:50.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.009+0000 7fe6f6ed3700 1 -- 172.21.15.163:0/1464169782 shutdown_connections 2023-10-09T06:31:50.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.009+0000 7fe6f6ed3700 1 --2- 172.21.15.163:0/1464169782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6f015df50 0x7fe6f015e3b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:50.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.009+0000 7fe6f6ed3700 1 --2- 172.21.15.163:0/1464169782 >> v2:172.21.15.181:3300/0 conn(0x7fe6f006c0a0 0x7fe6f00caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:50.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.009+0000 7fe6f6ed3700 1 -- 172.21.15.163:0/1464169782 >> 172.21.15.163:0/1464169782 conn(0x7fe6f00ce600 msgr2=0x7fe6f00cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:50.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.009+0000 7fe6f6ed3700 1 -- 172.21.15.163:0/1464169782 shutdown_connections 2023-10-09T06:31:50.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.009+0000 7fe6f6ed3700 1 -- 172.21.15.163:0/1464169782 wait complete. 2023-10-09T06:31:50.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.010+0000 7fe6f6ed3700 1 Processor -- start 2023-10-09T06:31:50.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.010+0000 7fe6f6ed3700 1 -- start start 2023-10-09T06:31:50.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.011+0000 7fe6f6ed3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6f006bcd0 0x7fe6f00b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:50.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.011+0000 7fe6f6ed3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6f006c0a0 0x7fe6f00c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:50.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.011+0000 7fe6f6ed3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6f015df50 0x7fe6f00b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:50.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.011+0000 7fe6f6ed3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f00c7c60 con 0x7fe6f006bcd0 2023-10-09T06:31:50.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.011+0000 7fe6f6ed3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f00b2930 con 0x7fe6f015df50 2023-10-09T06:31:50.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.011+0000 7fe6f6ed3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f00c7050 con 0x7fe6f006c0a0 2023-10-09T06:31:50.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.011+0000 7fe6f4c6f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6f006bcd0 0x7fe6f00b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:50.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.011+0000 7fe6f4c6f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6f006bcd0 0x7fe6f00b23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39686/0 (socket says 172.21.15.163:39686) 2023-10-09T06:31:50.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.011+0000 7fe6f4c6f700 1 -- 172.21.15.163:0/735963333 learned_addr learned my addr 172.21.15.163:0/735963333 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:50.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.011+0000 7fe6f5470700 1 --2- 172.21.15.163:0/735963333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6f015df50 0x7fe6f00b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:50.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.011+0000 7fe6effff700 1 --2- 172.21.15.163:0/735963333 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6f006c0a0 0x7fe6f00c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:50.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.011+0000 7fe6f4c6f700 1 -- 172.21.15.163:0/735963333 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6f006c0a0 msgr2=0x7fe6f00c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:50.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.011+0000 7fe6f4c6f700 1 --2- 172.21.15.163:0/735963333 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6f006c0a0 0x7fe6f00c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:50.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.011+0000 7fe6f4c6f700 1 -- 172.21.15.163:0/735963333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6f015df50 msgr2=0x7fe6f00b35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:50.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.011+0000 7fe6f4c6f700 1 --2- 172.21.15.163:0/735963333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6f015df50 0x7fe6f00b35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:50.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.011+0000 7fe6f4c6f700 1 -- 172.21.15.163:0/735963333 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e8009420 con 0x7fe6f006bcd0 2023-10-09T06:31:50.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.011+0000 7fe6f4c6f700 1 --2- 172.21.15.163:0/735963333 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6f006bcd0 0x7fe6f00b23f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000cc70 tx=0x7fe6e0001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:50.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.011+0000 7fe6ecff9700 1 -- 172.21.15.163:0/735963333 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e000a890 con 0x7fe6f006bcd0 2023-10-09T06:31:50.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.011+0000 7fe6ecff9700 1 -- 172.21.15.163:0/735963333 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e000a9f0 con 0x7fe6f006bcd0 2023-10-09T06:31:50.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.011+0000 7fe6f6ed3700 1 -- 172.21.15.163:0/735963333 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f00c7330 con 0x7fe6f006bcd0 2023-10-09T06:31:50.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.011+0000 7fe6ecff9700 1 -- 172.21.15.163:0/735963333 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e000ad70 con 0x7fe6f006bcd0 2023-10-09T06:31:50.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.011+0000 7fe6f6ed3700 1 -- 172.21.15.163:0/735963333 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f00c78f0 con 0x7fe6f006bcd0 2023-10-09T06:31:50.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.012+0000 7fe6f6ed3700 1 -- 172.21.15.163:0/735963333 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f004c3c0 con 0x7fe6f006bcd0 2023-10-09T06:31:50.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.013+0000 7fe6ecff9700 1 -- 172.21.15.163:0/735963333 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe6e0007300 con 0x7fe6f006bcd0 2023-10-09T06:31:50.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.013+0000 7fe6ecff9700 1 --2- 172.21.15.163:0/735963333 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe6dc066f30 0x7fe6dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:50.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.013+0000 7fe6ecff9700 1 -- 172.21.15.163:0/735963333 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fe6e0084890 con 0x7fe6f006bcd0 2023-10-09T06:31:50.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.013+0000 7fe6effff700 1 --2- 172.21.15.163:0/735963333 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe6dc066f30 0x7fe6dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:50.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.014+0000 7fe6effff700 1 --2- 172.21.15.163:0/735963333 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe6dc066f30 0x7fe6dc0693e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0070c30 tx=0x7fe6e4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:50.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.017+0000 7fe6ecff9700 1 -- 172.21.15.163:0/735963333 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe6e0051b30 con 0x7fe6f006bcd0 2023-10-09T06:31:50.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.241+0000 7fe6f6ed3700 1 -- 172.21.15.163:0/735963333 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe6f015aff0 con 0x7fe6f006bcd0 2023-10-09T06:31:50.243 INFO:tasks.ceph.mon.a.smithi163.stderr:2023-10-09T06:31:50.242+0000 7ff251c04700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-09T06:31:50.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.245+0000 7fe6ecff9700 1 -- 172.21.15.163:0/735963333 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe6e0055150 con 0x7fe6f006bcd0 2023-10-09T06:31:50.248 INFO:tasks.ceph.mds.d.smithi181.stderr: -1123> 2023-10-09T06:31:32.611+0000 7fd505e65700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-09T06:31:50.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.247+0000 7fe6f6ed3700 1 -- 172.21.15.163:0/735963333 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe6dc066f30 msgr2=0x7fe6dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:50.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.247+0000 7fe6f6ed3700 1 --2- 172.21.15.163:0/735963333 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe6dc066f30 0x7fe6dc0693e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0070c30 tx=0x7fe6e4008040 comp rx=0 tx=0).stop 2023-10-09T06:31:50.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.247+0000 7fe6f6ed3700 1 -- 172.21.15.163:0/735963333 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6f006bcd0 msgr2=0x7fe6f00b23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:50.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.247+0000 7fe6f6ed3700 1 --2- 172.21.15.163:0/735963333 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6f006bcd0 0x7fe6f00b23f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000cc70 tx=0x7fe6e0001a30 comp rx=0 tx=0).stop 2023-10-09T06:31:50.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.247+0000 7fe6f6ed3700 1 -- 172.21.15.163:0/735963333 shutdown_connections 2023-10-09T06:31:50.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.247+0000 7fe6f6ed3700 1 --2- 172.21.15.163:0/735963333 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe6dc066f30 0x7fe6dc0693e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:50.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.247+0000 7fe6f6ed3700 1 --2- 172.21.15.163:0/735963333 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6f006bcd0 0x7fe6f00b23f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:50.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.247+0000 7fe6f6ed3700 1 --2- 172.21.15.163:0/735963333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6f015df50 0x7fe6f00b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:50.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.247+0000 7fe6f6ed3700 1 --2- 172.21.15.163:0/735963333 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6f006c0a0 0x7fe6f00c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:50.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.247+0000 7fe6f6ed3700 1 -- 172.21.15.163:0/735963333 >> 172.21.15.163:0/735963333 conn(0x7fe6f00ce600 msgr2=0x7fe6f00c8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:50.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.248+0000 7fe6f6ed3700 1 -- 172.21.15.163:0/735963333 shutdown_connections 2023-10-09T06:31:50.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.248+0000 7fe6f6ed3700 1 -- 172.21.15.163:0/735963333 wait complete. 2023-10-09T06:31:50.250 INFO:teuthology.orchestra.run.smithi163.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-09T06:31:50.260 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-09T06:31:50.272 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-09T06:31:50.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.376+0000 7f0d41689700 1 Processor -- start 2023-10-09T06:31:50.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.376+0000 7f0d41689700 1 -- start start 2023-10-09T06:31:50.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.377+0000 7f0d41689700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0d3c0b4e70 0x7f0d3c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:50.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.377+0000 7f0d41689700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0d3c0b1b40 0x7f0d3c15d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:50.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.377+0000 7f0d41689700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d3c0cfb00 con 0x7f0d3c0b16d0 2023-10-09T06:31:50.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.377+0000 7f0d41689700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d3c0b5c40 con 0x7f0d3c0b4e70 2023-10-09T06:31:50.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.377+0000 7f0d41689700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d3c0b5f20 con 0x7f0d3c0b1b40 2023-10-09T06:31:50.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.377+0000 7f0d3bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0d3c0b4e70 0x7f0d3c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:50.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.377+0000 7f0d3bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0d3c0b4e70 0x7f0d3c0ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39696/0 (socket says 172.21.15.163:39696) 2023-10-09T06:31:50.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.377+0000 7f0d3bfff700 1 -- 172.21.15.163:0/754550086 learned_addr learned my addr 172.21.15.163:0/754550086 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:50.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.377+0000 7f0d40e88700 1 --2- 172.21.15.163:0/754550086 >> v2:172.21.15.181:3300/0 conn(0x7f0d3c0b1b40 0x7f0d3c15d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:50.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.377+0000 7f0d3bfff700 1 -- 172.21.15.163:0/754550086 >> v2:172.21.15.181:3300/0 conn(0x7f0d3c0b1b40 msgr2=0x7f0d3c15d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:50.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.377+0000 7f0d3bfff700 1 --2- 172.21.15.163:0/754550086 >> v2:172.21.15.181:3300/0 conn(0x7f0d3c0b1b40 0x7f0d3c15d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:50.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.377+0000 7f0d3bfff700 1 -- 172.21.15.163:0/754550086 >> v1:172.21.15.181:6789/0 conn(0x7f0d3c0b16d0 legacy=0x7f0d3c15d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:50.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.377+0000 7f0d3bfff700 1 -- 172.21.15.163:0/754550086 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3c0b6200 con 0x7f0d3c0b4e70 2023-10-09T06:31:50.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.377+0000 7f0d3bfff700 1 --2- 172.21.15.163:0/754550086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d3c0b4e70 0x7f0d3c0ccff0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c071380 tx=0x7f0d2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c3a439bf9987d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:50.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.378+0000 7f0d3affd700 1 -- 172.21.15.163:0/754550086 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c00b860 con 0x7f0d3c0b4e70 2023-10-09T06:31:50.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.378+0000 7f0d3affd700 1 -- 172.21.15.163:0/754550086 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2c00b9c0 con 0x7f0d3c0b4e70 2023-10-09T06:31:50.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.378+0000 7f0d3affd700 1 -- 172.21.15.163:0/754550086 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c00bd60 con 0x7f0d3c0b4e70 2023-10-09T06:31:50.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.378+0000 7f0d41689700 1 -- 172.21.15.163:0/754550086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d3c0b4e70 msgr2=0x7f0d3c0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:50.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.378+0000 7f0d41689700 1 --2- 172.21.15.163:0/754550086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d3c0b4e70 0x7f0d3c0ccff0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c071380 tx=0x7f0d2c009250 comp rx=0 tx=0).stop 2023-10-09T06:31:50.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.378+0000 7f0d41689700 1 -- 172.21.15.163:0/754550086 shutdown_connections 2023-10-09T06:31:50.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.378+0000 7f0d41689700 1 --2- 172.21.15.163:0/754550086 >> v2:172.21.15.181:3300/0 conn(0x7f0d3c0b1b40 0x7f0d3c15d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:50.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.378+0000 7f0d41689700 1 --2- 172.21.15.163:0/754550086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d3c0b4e70 0x7f0d3c0ccff0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:50.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.378+0000 7f0d41689700 1 -- 172.21.15.163:0/754550086 >> 172.21.15.163:0/754550086 conn(0x7f0d3c0d10c0 msgr2=0x7f0d3c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:50.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.378+0000 7f0d41689700 1 -- 172.21.15.163:0/754550086 shutdown_connections 2023-10-09T06:31:50.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.378+0000 7f0d41689700 1 -- 172.21.15.163:0/754550086 wait complete. 2023-10-09T06:31:50.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.379+0000 7f0d41689700 1 Processor -- start 2023-10-09T06:31:50.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.379+0000 7f0d41689700 1 -- start start 2023-10-09T06:31:50.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.380+0000 7f0d41689700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d3c0b16d0 0x7f0d3c15e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:50.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.380+0000 7f0d41689700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d3c0b1b40 0x7f0d3c15a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:50.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.380+0000 7f0d41689700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d3c0b4e70 0x7f0d3c0cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:50.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.380+0000 7f0d41689700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c15a2c0 con 0x7f0d3c0b1b40 2023-10-09T06:31:50.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.380+0000 7f0d41689700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c158640 con 0x7f0d3c0b16d0 2023-10-09T06:31:50.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.380+0000 7f0d41689700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3c159ce0 con 0x7f0d3c0b4e70 2023-10-09T06:31:50.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.380+0000 7f0d3b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d3c0b1b40 0x7f0d3c15a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:50.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.380+0000 7f0d3b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d3c0b1b40 0x7f0d3c15a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39712/0 (socket says 172.21.15.163:39712) 2023-10-09T06:31:50.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.380+0000 7f0d3b7fe700 1 -- 172.21.15.163:0/1240180542 learned_addr learned my addr 172.21.15.163:0/1240180542 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:50.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.380+0000 7f0d3bfff700 1 --2- 172.21.15.163:0/1240180542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d3c0b16d0 0x7f0d3c15e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:50.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.380+0000 7f0d40e88700 1 --2- 172.21.15.163:0/1240180542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d3c0b4e70 0x7f0d3c0cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:50.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.380+0000 7f0d3b7fe700 1 -- 172.21.15.163:0/1240180542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d3c0b4e70 msgr2=0x7f0d3c0cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:50.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.380+0000 7f0d3b7fe700 1 --2- 172.21.15.163:0/1240180542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d3c0b4e70 0x7f0d3c0cb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:50.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.380+0000 7f0d3b7fe700 1 -- 172.21.15.163:0/1240180542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d3c0b16d0 msgr2=0x7f0d3c15e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:50.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.380+0000 7f0d3b7fe700 1 --2- 172.21.15.163:0/1240180542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d3c0b16d0 0x7f0d3c15e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:50.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.380+0000 7f0d3b7fe700 1 -- 172.21.15.163:0/1240180542 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d2c0094f0 con 0x7f0d3c0b1b40 2023-10-09T06:31:50.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.381+0000 7f0d3bfff700 1 --2- 172.21.15.163:0/1240180542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d3c0b16d0 0x7f0d3c15e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:50.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.381+0000 7f0d40e88700 1 --2- 172.21.15.163:0/1240180542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d3c0b4e70 0x7f0d3c0cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:50.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.381+0000 7f0d3b7fe700 1 --2- 172.21.15.163:0/1240180542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d3c0b1b40 0x7f0d3c15a750 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000c5c0 tx=0x7f0d3000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:50.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.381+0000 7f0d23fff700 1 -- 172.21.15.163:0/1240180542 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d30011590 con 0x7f0d3c0b1b40 2023-10-09T06:31:50.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.381+0000 7f0d23fff700 1 -- 172.21.15.163:0/1240180542 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d30011710 con 0x7f0d3c0b1b40 2023-10-09T06:31:50.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.381+0000 7f0d41689700 1 -- 172.21.15.163:0/1240180542 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d3c1608b0 con 0x7f0d3c0b1b40 2023-10-09T06:31:50.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.381+0000 7f0d23fff700 1 -- 172.21.15.163:0/1240180542 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d30011ad0 con 0x7f0d3c0b1b40 2023-10-09T06:31:50.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.381+0000 7f0d41689700 1 -- 172.21.15.163:0/1240180542 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d3c160b90 con 0x7f0d3c0b1b40 2023-10-09T06:31:50.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.382+0000 7f0d41689700 1 -- 172.21.15.163:0/1240180542 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d3c15a470 con 0x7f0d3c0b1b40 2023-10-09T06:31:50.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.382+0000 7f0d23fff700 1 -- 172.21.15.163:0/1240180542 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0d3001b3b0 con 0x7f0d3c0b1b40 2023-10-09T06:31:50.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.382+0000 7f0d23fff700 1 --2- 172.21.15.163:0/1240180542 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0d28066f30 0x7f0d280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:50.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.383+0000 7f0d23fff700 1 -- 172.21.15.163:0/1240180542 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5877+0+0 (secure 0 0 0) 0x7f0d30083b70 con 0x7f0d3c0b1b40 2023-10-09T06:31:50.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.383+0000 7f0d3bfff700 1 --2- 172.21.15.163:0/1240180542 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0d28066f30 0x7f0d280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:50.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.383+0000 7f0d3bfff700 1 --2- 172.21.15.163:0/1240180542 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0d28066f30 0x7f0d280693e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c071380 tx=0x7f0d2c017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:50.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.387+0000 7f0d23fff700 1 -- 172.21.15.163:0/1240180542 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d3c15a470 con 0x7f0d3c0b1b40 2023-10-09T06:31:50.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:50.614+0000 7f0d41689700 1 -- 172.21.15.163:0/1240180542 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0d3c15a400 con 0x7f0d3c0b1b40 2023-10-09T06:31:51.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.248+0000 7f0d23fff700 1 -- 172.21.15.163:0/1240180542 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0d30017020 con 0x7f0d3c0b1b40 2023-10-09T06:31:51.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.250+0000 7f0d41689700 1 -- 172.21.15.163:0/1240180542 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0d28066f30 msgr2=0x7f0d280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:51.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.250+0000 7f0d41689700 1 --2- 172.21.15.163:0/1240180542 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0d28066f30 0x7f0d280693e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c071380 tx=0x7f0d2c017000 comp rx=0 tx=0).stop 2023-10-09T06:31:51.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.251+0000 7f0d41689700 1 -- 172.21.15.163:0/1240180542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d3c0b1b40 msgr2=0x7f0d3c15a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:51.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.251+0000 7f0d41689700 1 --2- 172.21.15.163:0/1240180542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d3c0b1b40 0x7f0d3c15a750 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000c5c0 tx=0x7f0d3000c980 comp rx=0 tx=0).stop 2023-10-09T06:31:51.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.251+0000 7f0d41689700 1 -- 172.21.15.163:0/1240180542 shutdown_connections 2023-10-09T06:31:51.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.251+0000 7f0d41689700 1 --2- 172.21.15.163:0/1240180542 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0d28066f30 0x7f0d280693e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:51.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.251+0000 7f0d41689700 1 --2- 172.21.15.163:0/1240180542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d3c0b4e70 0x7f0d3c0cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:51.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.251+0000 7f0d41689700 1 --2- 172.21.15.163:0/1240180542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d3c0b16d0 0x7f0d3c15e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:51.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.251+0000 7f0d41689700 1 --2- 172.21.15.163:0/1240180542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d3c0b1b40 0x7f0d3c15a750 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:51.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.251+0000 7f0d41689700 1 -- 172.21.15.163:0/1240180542 >> 172.21.15.163:0/1240180542 conn(0x7f0d3c0d10c0 msgr2=0x7f0d3c0b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:51.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.251+0000 7f0d41689700 1 -- 172.21.15.163:0/1240180542 shutdown_connections 2023-10-09T06:31:51.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.251+0000 7f0d41689700 1 -- 172.21.15.163:0/1240180542 wait complete. 2023-10-09T06:31:51.264 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-09T06:31:51.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.382+0000 7f3bb5ae0700 1 Processor -- start 2023-10-09T06:31:51.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.382+0000 7f3bb5ae0700 1 -- start start 2023-10-09T06:31:51.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.382+0000 7f3bb5ae0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3bb00cd910 0x7f3bb00ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:51.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.382+0000 7f3bb5ae0700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bb00b5b00 con 0x7f3bb00b55a0 2023-10-09T06:31:51.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.382+0000 7f3bb5ae0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bb00b5da0 con 0x7f3bb0159ec0 2023-10-09T06:31:51.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.382+0000 7f3bb5ae0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bb00b6080 con 0x7f3bb00cd910 2023-10-09T06:31:51.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.383+0000 7f3baffff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f3bb00b55a0 0x7f3bb00ce880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:44722/0 (socket says 172.21.15.163:44722) 2023-10-09T06:31:51.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.383+0000 7f3baffff700 1 -- 172.21.15.163:0/628544693 learned_addr learned my addr 172.21.15.163:0/628544693 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:51.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.383+0000 7f3ba6ffd700 1 --2- 172.21.15.163:0/628544693 >> v2:172.21.15.181:3300/0 conn(0x7f3bb00cd910 0x7f3bb00ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:51.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.383+0000 7f3baeffd700 1 -- 172.21.15.163:0/628544693 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3241040701 0 0) 0x7f3bb00b5b00 con 0x7f3bb00b55a0 2023-10-09T06:31:51.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.383+0000 7f3baeffd700 1 -- 172.21.15.163:0/628544693 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b90003570 con 0x7f3bb00b55a0 2023-10-09T06:31:51.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.383+0000 7f3baeffd700 1 -- 172.21.15.163:0/628544693 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2961562796 0 0) 0x7f3b90003570 con 0x7f3bb00b55a0 2023-10-09T06:31:51.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.383+0000 7f3baeffd700 1 -- 172.21.15.163:0/628544693 >> v2:172.21.15.181:3300/0 conn(0x7f3bb00cd910 msgr2=0x7f3bb00ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:51.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.383+0000 7f3baeffd700 1 --2- 172.21.15.163:0/628544693 >> v2:172.21.15.181:3300/0 conn(0x7f3bb00cd910 0x7f3bb00ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:51.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.383+0000 7f3baeffd700 1 -- 172.21.15.163:0/628544693 >> v1:172.21.15.181:6789/0 conn(0x7f3bb0159ec0 legacy=0x7f3bb00b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:51.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.383+0000 7f3baeffd700 1 -- 172.21.15.163:0/628544693 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb00cc100 con 0x7f3bb00b55a0 2023-10-09T06:31:51.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.383+0000 7f3baeffd700 1 -- 172.21.15.163:0/628544693 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f3ba80017f0 con 0x7f3bb00b55a0 2023-10-09T06:31:51.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.383+0000 7f3baeffd700 1 -- 172.21.15.163:0/628544693 >> v1:172.21.15.163:6789/0 conn(0x7f3bb00b55a0 legacy=0x7f3bb00ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:51.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.383+0000 7f3baeffd700 1 --2- 172.21.15.163:0/628544693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b90003a90 0x7f3b90003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:51.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.383+0000 7f3baeffd700 1 --2- 172.21.15.163:0/628544693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b90004810 0x7f3b90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:51.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.383+0000 7f3baeffd700 1 --2- 172.21.15.163:0/628544693 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b90005210 0x7f3b9000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:51.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.383+0000 7f3baeffd700 1 -- 172.21.15.163:0/628544693 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb00b6080 con 0x7f3b90003a90 2023-10-09T06:31:51.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.383+0000 7f3baeffd700 1 -- 172.21.15.163:0/628544693 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9000c130 con 0x7f3b90004810 2023-10-09T06:31:51.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.383+0000 7f3baeffd700 1 -- 172.21.15.163:0/628544693 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9000c2c0 con 0x7f3b90005210 2023-10-09T06:31:51.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.383+0000 7f3baf7fe700 1 --2- 172.21.15.163:0/628544693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b90003a90 0x7f3b90003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:51.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.383+0000 7f3ba6ffd700 1 --2- 172.21.15.163:0/628544693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b90004810 0x7f3b90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:51.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.383+0000 7f3baffff700 1 --2- 172.21.15.163:0/628544693 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b90005210 0x7f3b9000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:51.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.384+0000 7f3baf7fe700 1 -- 172.21.15.163:0/628544693 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b90005210 msgr2=0x7f3b9000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:51.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.384+0000 7f3baf7fe700 1 --2- 172.21.15.163:0/628544693 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b90005210 0x7f3b9000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:51.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.384+0000 7f3baf7fe700 1 -- 172.21.15.163:0/628544693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b90004810 msgr2=0x7f3b90004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:51.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.384+0000 7f3baf7fe700 1 --2- 172.21.15.163:0/628544693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b90004810 0x7f3b90004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:51.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.384+0000 7f3baf7fe700 1 -- 172.21.15.163:0/628544693 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b90003570 con 0x7f3b90003a90 2023-10-09T06:31:51.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.384+0000 7f3baffff700 1 --2- 172.21.15.163:0/628544693 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b90005210 0x7f3b9000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:51.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.384+0000 7f3baffff700 1 -- 172.21.15.163:0/628544693 reap_dead start 2023-10-09T06:31:51.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.384+0000 7f3baf7fe700 1 --2- 172.21.15.163:0/628544693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b90003a90 0x7f3b90003ea0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0001900 tx=0x7f3ba0007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e878ba40fa5a726 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:51.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.384+0000 7f3baeffd700 1 -- 172.21.15.163:0/628544693 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0001290 con 0x7f3b90003a90 2023-10-09T06:31:51.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.384+0000 7f3baeffd700 1 -- 172.21.15.163:0/628544693 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba0012420 con 0x7f3b90003a90 2023-10-09T06:31:51.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.384+0000 7f3baeffd700 1 -- 172.21.15.163:0/628544693 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba00127e0 con 0x7f3b90003a90 2023-10-09T06:31:51.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.384+0000 7f3bb5ae0700 1 -- 172.21.15.163:0/628544693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b90003a90 msgr2=0x7f3b90003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:51.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.384+0000 7f3bb5ae0700 1 --2- 172.21.15.163:0/628544693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b90003a90 0x7f3b90003ea0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0001900 tx=0x7f3ba0007d80 comp rx=0 tx=0).stop 2023-10-09T06:31:51.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.384+0000 7f3bb5ae0700 1 -- 172.21.15.163:0/628544693 shutdown_connections 2023-10-09T06:31:51.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.384+0000 7f3bb5ae0700 1 --2- 172.21.15.163:0/628544693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b90003a90 0x7f3b90003ea0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:51.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.384+0000 7f3bb5ae0700 1 -- 172.21.15.163:0/628544693 >> 172.21.15.163:0/628544693 conn(0x7f3bb00999f0 msgr2=0x7f3bb00b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:51.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.385+0000 7f3bb5ae0700 1 -- 172.21.15.163:0/628544693 shutdown_connections 2023-10-09T06:31:51.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.385+0000 7f3bb5ae0700 1 -- 172.21.15.163:0/628544693 wait complete. 2023-10-09T06:31:51.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.386+0000 7f3bb5ae0700 1 Processor -- start 2023-10-09T06:31:51.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.386+0000 7f3bb5ae0700 1 -- start start 2023-10-09T06:31:51.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.386+0000 7f3bb5ae0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b90003a90 0x7f3bb016a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:51.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.386+0000 7f3bb5ae0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bb00c5eb0 0x7f3bb00c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:51.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.386+0000 7f3bb5ae0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bb016ac10 0x7f3bb016b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:51.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.386+0000 7f3bb5ae0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb00c6860 con 0x7f3bb016ac10 2023-10-09T06:31:51.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.386+0000 7f3bb5ae0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb016c580 con 0x7f3b90003a90 2023-10-09T06:31:51.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.386+0000 7f3bb5ae0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb016b970 con 0x7f3bb00c5eb0 2023-10-09T06:31:51.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.387+0000 7f3baffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bb016ac10 0x7f3bb016b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:51.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.387+0000 7f3baffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bb016ac10 0x7f3bb016b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39722/0 (socket says 172.21.15.163:39722) 2023-10-09T06:31:51.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.387+0000 7f3baffff700 1 -- 172.21.15.163:0/1481267082 learned_addr learned my addr 172.21.15.163:0/1481267082 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:51.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.387+0000 7f3baf7fe700 1 --2- 172.21.15.163:0/1481267082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b90003a90 0x7f3bb016a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:51.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.387+0000 7f3ba6ffd700 1 --2- 172.21.15.163:0/1481267082 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bb00c5eb0 0x7f3bb00c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:51.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.387+0000 7f3baffff700 1 -- 172.21.15.163:0/1481267082 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bb00c5eb0 msgr2=0x7f3bb00c6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:51.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.387+0000 7f3baffff700 1 --2- 172.21.15.163:0/1481267082 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bb00c5eb0 0x7f3bb00c6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:51.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.387+0000 7f3baffff700 1 -- 172.21.15.163:0/1481267082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b90003a90 msgr2=0x7f3bb016a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:51.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.387+0000 7f3baffff700 1 --2- 172.21.15.163:0/1481267082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b90003a90 0x7f3bb016a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:51.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.387+0000 7f3baffff700 1 -- 172.21.15.163:0/1481267082 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb016bc50 con 0x7f3bb016ac10 2023-10-09T06:31:51.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.387+0000 7f3baffff700 1 --2- 172.21.15.163:0/1481267082 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bb016ac10 0x7f3bb016b090 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3bb008ad70 tx=0x7f3ba80017f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:51.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.387+0000 7f3ba7fff700 1 -- 172.21.15.163:0/1481267082 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba800e900 con 0x7f3bb016ac10 2023-10-09T06:31:51.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.387+0000 7f3ba7fff700 1 -- 172.21.15.163:0/1481267082 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba800ea60 con 0x7f3bb016ac10 2023-10-09T06:31:51.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.387+0000 7f3ba7fff700 1 -- 172.21.15.163:0/1481267082 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba800ede0 con 0x7f3bb016ac10 2023-10-09T06:31:51.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.387+0000 7f3bb5ae0700 1 -- 172.21.15.163:0/1481267082 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb016bf30 con 0x7f3bb016ac10 2023-10-09T06:31:51.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.387+0000 7f3bb5ae0700 1 -- 172.21.15.163:0/1481267082 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb016a7f0 con 0x7f3bb016ac10 2023-10-09T06:31:51.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.388+0000 7f3bb5ae0700 1 -- 172.21.15.163:0/1481267082 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bb0063c90 con 0x7f3bb016ac10 2023-10-09T06:31:51.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.389+0000 7f3ba7fff700 1 -- 172.21.15.163:0/1481267082 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3ba8013630 con 0x7f3bb016ac10 2023-10-09T06:31:51.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.389+0000 7f3ba7fff700 1 --2- 172.21.15.163:0/1481267082 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3b94066f30 0x7f3b940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:51.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.389+0000 7f3ba7fff700 1 -- 172.21.15.163:0/1481267082 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5877+0+0 (secure 0 0 0) 0x7f3ba8084840 con 0x7f3bb016ac10 2023-10-09T06:31:51.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.389+0000 7f3baf7fe700 1 --2- 172.21.15.163:0/1481267082 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3b94066f30 0x7f3b940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:51.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.390+0000 7f3baf7fe700 1 --2- 172.21.15.163:0/1481267082 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3b94066f30 0x7f3b940693e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0001060 tx=0x7f3ba0015000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:51.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.394+0000 7f3ba7fff700 1 -- 172.21.15.163:0/1481267082 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ba8051a00 con 0x7f3bb016ac10 2023-10-09T06:31:51.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:51.577+0000 7f3bb5ae0700 1 -- 172.21.15.163:0/1481267082 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3bb00718b0 con 0x7f3bb016ac10 2023-10-09T06:31:52.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.495+0000 7f3ba7fff700 1 -- 172.21.15.163:0/1481267082 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v41) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3ba8055020 con 0x7f3bb016ac10 2023-10-09T06:31:52.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.497+0000 7f3bb5ae0700 1 -- 172.21.15.163:0/1481267082 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3b94066f30 msgr2=0x7f3b940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:52.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.497+0000 7f3bb5ae0700 1 --2- 172.21.15.163:0/1481267082 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3b94066f30 0x7f3b940693e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0001060 tx=0x7f3ba0015000 comp rx=0 tx=0).stop 2023-10-09T06:31:52.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.497+0000 7f3bb5ae0700 1 -- 172.21.15.163:0/1481267082 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bb016ac10 msgr2=0x7f3bb016b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:52.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.497+0000 7f3bb5ae0700 1 --2- 172.21.15.163:0/1481267082 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bb016ac10 0x7f3bb016b090 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3bb008ad70 tx=0x7f3ba80017f0 comp rx=0 tx=0).stop 2023-10-09T06:31:52.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.498+0000 7f3bb5ae0700 1 -- 172.21.15.163:0/1481267082 shutdown_connections 2023-10-09T06:31:52.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.498+0000 7f3bb5ae0700 1 --2- 172.21.15.163:0/1481267082 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3b94066f30 0x7f3b940693e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:52.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.498+0000 7f3bb5ae0700 1 --2- 172.21.15.163:0/1481267082 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bb016ac10 0x7f3bb016b090 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:52.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.498+0000 7f3bb5ae0700 1 --2- 172.21.15.163:0/1481267082 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b90003a90 0x7f3bb016a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:52.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.498+0000 7f3bb5ae0700 1 --2- 172.21.15.163:0/1481267082 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bb00c5eb0 0x7f3bb00c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:52.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.498+0000 7f3bb5ae0700 1 -- 172.21.15.163:0/1481267082 >> 172.21.15.163:0/1481267082 conn(0x7f3bb00999f0 msgr2=0x7f3bb0159240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:52.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.498+0000 7f3bb5ae0700 1 -- 172.21.15.163:0/1481267082 shutdown_connections 2023-10-09T06:31:52.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.498+0000 7f3bb5ae0700 1 -- 172.21.15.163:0/1481267082 wait complete. 2023-10-09T06:31:52.499 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_metadata' removed 2023-10-09T06:31:52.511 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-09T06:31:52.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.633+0000 7ffa82279700 1 Processor -- start 2023-10-09T06:31:52.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.633+0000 7ffa82279700 1 -- start start 2023-10-09T06:31:52.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.634+0000 7ffa82279700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa7c0ac0e0 0x7ffa7c0afa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:52.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.634+0000 7ffa82279700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ffa7c06f0b0 0x7ffa7c15d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:52.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.634+0000 7ffa82279700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa7c06e510 con 0x7ffa7c1597d0 2023-10-09T06:31:52.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.634+0000 7ffa82279700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa7c069180 con 0x7ffa7c06f0b0 2023-10-09T06:31:52.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.634+0000 7ffa82279700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c06dec0 con 0x7ffa7c0ac0e0 2023-10-09T06:31:52.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.634+0000 7ffa81277700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ffa7c1597d0 0x7ffa7c06e9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:44724/0 (socket says 172.21.15.163:44724) 2023-10-09T06:31:52.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.634+0000 7ffa81277700 1 -- 172.21.15.163:0/2090920566 learned_addr learned my addr 172.21.15.163:0/2090920566 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:52.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.634+0000 7ffa81a78700 1 --2- 172.21.15.163:0/2090920566 >> v2:172.21.15.163:3300/0 conn(0x7ffa7c06f0b0 0x7ffa7c15d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:52.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.634+0000 7ffa73fff700 1 -- 172.21.15.163:0/2090920566 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2885897259 0 0) 0x7ffa7c06e510 con 0x7ffa7c1597d0 2023-10-09T06:31:52.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.635+0000 7ffa73fff700 1 -- 172.21.15.163:0/2090920566 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa68003570 con 0x7ffa7c1597d0 2023-10-09T06:31:52.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.635+0000 7ffa80a76700 1 --2- 172.21.15.163:0/2090920566 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa7c0ac0e0 0x7ffa7c0afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:52.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.635+0000 7ffa81a78700 1 -- 172.21.15.163:0/2090920566 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa7c0ac0e0 msgr2=0x7ffa7c0afa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:52.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.635+0000 7ffa81a78700 1 --2- 172.21.15.163:0/2090920566 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa7c0ac0e0 0x7ffa7c0afa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:52.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.635+0000 7ffa81a78700 1 -- 172.21.15.163:0/2090920566 >> v1:172.21.15.163:6789/0 conn(0x7ffa7c1597d0 legacy=0x7ffa7c06e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:52.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.635+0000 7ffa81a78700 1 -- 172.21.15.163:0/2090920566 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa7c06e1a0 con 0x7ffa7c06f0b0 2023-10-09T06:31:52.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.635+0000 7ffa81a78700 1 --2- 172.21.15.163:0/2090920566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa7c06f0b0 0x7ffa7c15d440 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ffa740098e0 tx=0x7ffa740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a355982d839ffa62 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:52.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.635+0000 7ffa73fff700 1 -- 172.21.15.163:0/2090920566 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7400b640 con 0x7ffa7c06f0b0 2023-10-09T06:31:52.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.635+0000 7ffa73fff700 1 -- 172.21.15.163:0/2090920566 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa7400b7a0 con 0x7ffa7c06f0b0 2023-10-09T06:31:52.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.635+0000 7ffa73fff700 1 -- 172.21.15.163:0/2090920566 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7400bb40 con 0x7ffa7c06f0b0 2023-10-09T06:31:52.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.635+0000 7ffa82279700 1 -- 172.21.15.163:0/2090920566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa7c06f0b0 msgr2=0x7ffa7c15d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:52.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.635+0000 7ffa82279700 1 --2- 172.21.15.163:0/2090920566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa7c06f0b0 0x7ffa7c15d440 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ffa740098e0 tx=0x7ffa740091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:52.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.635+0000 7ffa82279700 1 -- 172.21.15.163:0/2090920566 shutdown_connections 2023-10-09T06:31:52.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.635+0000 7ffa82279700 1 --2- 172.21.15.163:0/2090920566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa7c06f0b0 0x7ffa7c15d440 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:52.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.635+0000 7ffa82279700 1 --2- 172.21.15.163:0/2090920566 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa7c0ac0e0 0x7ffa7c0afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:52.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.635+0000 7ffa82279700 1 -- 172.21.15.163:0/2090920566 >> 172.21.15.163:0/2090920566 conn(0x7ffa7c06adf0 msgr2=0x7ffa7c0ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:52.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.636+0000 7ffa82279700 1 -- 172.21.15.163:0/2090920566 shutdown_connections 2023-10-09T06:31:52.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.636+0000 7ffa82279700 1 -- 172.21.15.163:0/2090920566 wait complete. 2023-10-09T06:31:52.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.637+0000 7ffa82279700 1 Processor -- start 2023-10-09T06:31:52.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.637+0000 7ffa82279700 1 -- start start 2023-10-09T06:31:52.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.637+0000 7ffa82279700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa7c06f0b0 0x7ffa7c141f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:52.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.638+0000 7ffa82279700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa7c1597d0 0x7ffa7c157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:52.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.638+0000 7ffa82279700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa7c157890 0x7ffa7c155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:52.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.638+0000 7ffa82279700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c15c340 con 0x7ffa7c06f0b0 2023-10-09T06:31:52.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.638+0000 7ffa82279700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c157cc0 con 0x7ffa7c1597d0 2023-10-09T06:31:52.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.638+0000 7ffa82279700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7c156670 con 0x7ffa7c157890 2023-10-09T06:31:52.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.638+0000 7ffa81277700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa7c06f0b0 0x7ffa7c141f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:52.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.638+0000 7ffa81277700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa7c06f0b0 0x7ffa7c141f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39748/0 (socket says 172.21.15.163:39748) 2023-10-09T06:31:52.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.638+0000 7ffa81277700 1 -- 172.21.15.163:0/860277699 learned_addr learned my addr 172.21.15.163:0/860277699 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:52.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.638+0000 7ffa81a78700 1 --2- 172.21.15.163:0/860277699 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa7c157890 0x7ffa7c155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:52.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.638+0000 7ffa80a76700 1 --2- 172.21.15.163:0/860277699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa7c1597d0 0x7ffa7c157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:52.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.638+0000 7ffa81277700 1 -- 172.21.15.163:0/860277699 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa7c157890 msgr2=0x7ffa7c155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:52.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.638+0000 7ffa81277700 1 --2- 172.21.15.163:0/860277699 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa7c157890 0x7ffa7c155940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:52.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.638+0000 7ffa81277700 1 -- 172.21.15.163:0/860277699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa7c1597d0 msgr2=0x7ffa7c157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:52.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.638+0000 7ffa81277700 1 --2- 172.21.15.163:0/860277699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa7c1597d0 0x7ffa7c157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:52.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.638+0000 7ffa81277700 1 -- 172.21.15.163:0/860277699 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa74009420 con 0x7ffa7c06f0b0 2023-10-09T06:31:52.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.638+0000 7ffa80a76700 1 --2- 172.21.15.163:0/860277699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa7c1597d0 0x7ffa7c157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:52.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.638+0000 7ffa81277700 1 --2- 172.21.15.163:0/860277699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa7c06f0b0 0x7ffa7c141f60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800cc70 tx=0x7ffa78001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:52.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.638+0000 7ffa717fa700 1 -- 172.21.15.163:0/860277699 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7800a900 con 0x7ffa7c06f0b0 2023-10-09T06:31:52.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.638+0000 7ffa717fa700 1 -- 172.21.15.163:0/860277699 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa7800aa80 con 0x7ffa7c06f0b0 2023-10-09T06:31:52.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.638+0000 7ffa717fa700 1 -- 172.21.15.163:0/860277699 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7800ae40 con 0x7ffa7c06f0b0 2023-10-09T06:31:52.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.638+0000 7ffa82279700 1 -- 172.21.15.163:0/860277699 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa7c156950 con 0x7ffa7c06f0b0 2023-10-09T06:31:52.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.638+0000 7ffa82279700 1 -- 172.21.15.163:0/860277699 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa7c15bfd0 con 0x7ffa7c06f0b0 2023-10-09T06:31:52.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.639+0000 7ffa82279700 1 -- 172.21.15.163:0/860277699 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa7c04c3c0 con 0x7ffa7c06f0b0 2023-10-09T06:31:52.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.639+0000 7ffa717fa700 1 -- 172.21.15.163:0/860277699 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffa78007300 con 0x7ffa7c06f0b0 2023-10-09T06:31:52.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.640+0000 7ffa717fa700 1 --2- 172.21.15.163:0/860277699 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffa60066f30 0x7ffa600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:52.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.640+0000 7ffa717fa700 1 -- 172.21.15.163:0/860277699 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5439+0+0 (secure 0 0 0) 0x7ffa78084820 con 0x7ffa7c06f0b0 2023-10-09T06:31:52.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.640+0000 7ffa80a76700 1 --2- 172.21.15.163:0/860277699 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffa60066f30 0x7ffa600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:52.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.640+0000 7ffa80a76700 1 --2- 172.21.15.163:0/860277699 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffa60066f30 0x7ffa600693e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c16d0b0 tx=0x7ffa6c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:52.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.644+0000 7ffa717fa700 1 -- 172.21.15.163:0/860277699 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa78051c20 con 0x7ffa7c06f0b0 2023-10-09T06:31:52.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:52.829+0000 7ffa82279700 1 -- 172.21.15.163:0/860277699 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ffa7c0cd4a0 con 0x7ffa7c06f0b0 2023-10-09T06:31:53.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.494+0000 7ffa717fa700 1 -- 172.21.15.163:0/860277699 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v42) v1 ==== 169+0+0 (secure 0 0 0) 0x7ffa78055240 con 0x7ffa7c06f0b0 2023-10-09T06:31:53.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.497+0000 7ffa82279700 1 -- 172.21.15.163:0/860277699 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffa60066f30 msgr2=0x7ffa600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:53.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.497+0000 7ffa82279700 1 --2- 172.21.15.163:0/860277699 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffa60066f30 0x7ffa600693e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c16d0b0 tx=0x7ffa6c008040 comp rx=0 tx=0).stop 2023-10-09T06:31:53.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.497+0000 7ffa82279700 1 -- 172.21.15.163:0/860277699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa7c06f0b0 msgr2=0x7ffa7c141f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:53.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.497+0000 7ffa82279700 1 --2- 172.21.15.163:0/860277699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa7c06f0b0 0x7ffa7c141f60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800cc70 tx=0x7ffa78001a70 comp rx=0 tx=0).stop 2023-10-09T06:31:53.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.497+0000 7ffa82279700 1 -- 172.21.15.163:0/860277699 shutdown_connections 2023-10-09T06:31:53.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.497+0000 7ffa82279700 1 --2- 172.21.15.163:0/860277699 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffa60066f30 0x7ffa600693e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:53.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.497+0000 7ffa82279700 1 --2- 172.21.15.163:0/860277699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa7c06f0b0 0x7ffa7c141f60 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:53.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.497+0000 7ffa82279700 1 --2- 172.21.15.163:0/860277699 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa7c157890 0x7ffa7c155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:53.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.497+0000 7ffa82279700 1 --2- 172.21.15.163:0/860277699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa7c1597d0 0x7ffa7c157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:53.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.497+0000 7ffa82279700 1 -- 172.21.15.163:0/860277699 >> 172.21.15.163:0/860277699 conn(0x7ffa7c06adf0 msgr2=0x7ffa7c0afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:53.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.497+0000 7ffa82279700 1 -- 172.21.15.163:0/860277699 shutdown_connections 2023-10-09T06:31:53.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.498+0000 7ffa82279700 1 -- 172.21.15.163:0/860277699 wait complete. 2023-10-09T06:31:53.499 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_data' removed 2023-10-09T06:31:53.511 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-09T06:31:53.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.627+0000 7f954df81700 1 Processor -- start 2023-10-09T06:31:53.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.627+0000 7f954df81700 1 -- start start 2023-10-09T06:31:53.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.628+0000 7f954df81700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95480b4e70 0x7f95480ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:53.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.628+0000 7f954df81700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f95480b16d0 0x7f95480b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:53.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.628+0000 7f954df81700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95480b2080 con 0x7f954815d240 2023-10-09T06:31:53.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.628+0000 7f954df81700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95480cfb00 con 0x7f95480b16d0 2023-10-09T06:31:53.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.628+0000 7f954df81700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95480b5c40 con 0x7f95480b4e70 2023-10-09T06:31:53.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.628+0000 7f9547fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f95480b16d0 0x7f95480b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:53.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.628+0000 7f954d780700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f954815d240 0x7f954815d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:44726/0 (socket says 172.21.15.163:44726) 2023-10-09T06:31:53.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.628+0000 7f954d780700 1 -- 172.21.15.163:0/2912062494 learned_addr learned my addr 172.21.15.163:0/2912062494 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:53.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.628+0000 7f9547fff700 1 -- 172.21.15.163:0/2912062494 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95480b4e70 msgr2=0x7f95480ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:53.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.628+0000 7f954cf7f700 1 --2- 172.21.15.163:0/2912062494 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95480b4e70 0x7f95480ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:53.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.628+0000 7f95477fe700 1 -- 172.21.15.163:0/2912062494 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3404809252 0 0) 0x7f95480b2080 con 0x7f954815d240 2023-10-09T06:31:53.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.628+0000 7f9547fff700 1 --2- 172.21.15.163:0/2912062494 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95480b4e70 0x7f95480ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:53.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.628+0000 7f9547fff700 1 -- 172.21.15.163:0/2912062494 >> v1:172.21.15.163:6789/0 conn(0x7f954815d240 legacy=0x7f954815d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:53.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.628+0000 7f9547fff700 1 -- 172.21.15.163:0/2912062494 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95480b5f20 con 0x7f95480b16d0 2023-10-09T06:31:53.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.628+0000 7f954cf7f700 1 --2- 172.21.15.163:0/2912062494 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95480b4e70 0x7f95480ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:53.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.628+0000 7f9547fff700 1 --2- 172.21.15.163:0/2912062494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95480b16d0 0x7f95480b1b40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f95340098e0 tx=0x7f95340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38f95fe41450a62 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:53.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.629+0000 7f95477fe700 1 -- 172.21.15.163:0/2912062494 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9534009e80 con 0x7f95480b16d0 2023-10-09T06:31:53.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.629+0000 7f95477fe700 1 -- 172.21.15.163:0/2912062494 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953400b860 con 0x7f95480b16d0 2023-10-09T06:31:53.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.629+0000 7f95477fe700 1 -- 172.21.15.163:0/2912062494 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953400bbe0 con 0x7f95480b16d0 2023-10-09T06:31:53.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.629+0000 7f954df81700 1 -- 172.21.15.163:0/2912062494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95480b16d0 msgr2=0x7f95480b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:53.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.629+0000 7f954df81700 1 --2- 172.21.15.163:0/2912062494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95480b16d0 0x7f95480b1b40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f95340098e0 tx=0x7f95340091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:53.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.629+0000 7f954df81700 1 -- 172.21.15.163:0/2912062494 shutdown_connections 2023-10-09T06:31:53.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.629+0000 7f954df81700 1 --2- 172.21.15.163:0/2912062494 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95480b4e70 0x7f95480ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:53.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.629+0000 7f954df81700 1 --2- 172.21.15.163:0/2912062494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95480b16d0 0x7f95480b1b40 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:53.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.629+0000 7f954df81700 1 -- 172.21.15.163:0/2912062494 >> 172.21.15.163:0/2912062494 conn(0x7f95480d10c0 msgr2=0x7f95480affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:53.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.629+0000 7f954df81700 1 -- 172.21.15.163:0/2912062494 shutdown_connections 2023-10-09T06:31:53.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.629+0000 7f954df81700 1 -- 172.21.15.163:0/2912062494 wait complete. 2023-10-09T06:31:53.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.630+0000 7f954df81700 1 Processor -- start 2023-10-09T06:31:53.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.630+0000 7f954df81700 1 -- start start 2023-10-09T06:31:53.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.631+0000 7f954df81700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95480b16d0 0x7f954816daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:53.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.631+0000 7f954df81700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95480b4e70 0x7f954815b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:53.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.631+0000 7f954df81700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f954815d240 0x7f9548169b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:53.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.631+0000 7f954df81700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954816aad0 con 0x7f954815d240 2023-10-09T06:31:53.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.631+0000 7f954df81700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954816e030 con 0x7f95480b16d0 2023-10-09T06:31:53.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.631+0000 7f954df81700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954816a510 con 0x7f95480b4e70 2023-10-09T06:31:53.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.631+0000 7f954d780700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f954815d240 0x7f9548169b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:53.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.631+0000 7f954d780700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f954815d240 0x7f9548169b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39764/0 (socket says 172.21.15.163:39764) 2023-10-09T06:31:53.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.631+0000 7f954d780700 1 -- 172.21.15.163:0/1116114631 learned_addr learned my addr 172.21.15.163:0/1116114631 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:53.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.631+0000 7f954d780700 1 -- 172.21.15.163:0/1116114631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95480b4e70 msgr2=0x7f954815b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:53.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.631+0000 7f9547fff700 1 --2- 172.21.15.163:0/1116114631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95480b4e70 0x7f954815b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:53.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.631+0000 7f954cf7f700 1 --2- 172.21.15.163:0/1116114631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95480b16d0 0x7f954816daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:53.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.631+0000 7f954d780700 1 --2- 172.21.15.163:0/1116114631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95480b4e70 0x7f954815b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:53.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.631+0000 7f954d780700 1 -- 172.21.15.163:0/1116114631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95480b16d0 msgr2=0x7f954816daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:53.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.631+0000 7f954d780700 1 --2- 172.21.15.163:0/1116114631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95480b16d0 0x7f954816daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:53.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.631+0000 7f954d780700 1 -- 172.21.15.163:0/1116114631 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9534009420 con 0x7f954815d240 2023-10-09T06:31:53.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.631+0000 7f9547fff700 1 --2- 172.21.15.163:0/1116114631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95480b4e70 0x7f954815b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:53.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.631+0000 7f954cf7f700 1 --2- 172.21.15.163:0/1116114631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95480b16d0 0x7f954816daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:53.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.632+0000 7f954d780700 1 --2- 172.21.15.163:0/1116114631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f954815d240 0x7f9548169b20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f954000cbf0 tx=0x7f95400019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:53.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.632+0000 7f9544ff9700 1 -- 172.21.15.163:0/1116114631 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954000a9c0 con 0x7f954815d240 2023-10-09T06:31:53.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.632+0000 7f9544ff9700 1 -- 172.21.15.163:0/1116114631 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954000ab20 con 0x7f954815d240 2023-10-09T06:31:53.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.632+0000 7f954df81700 1 -- 172.21.15.163:0/1116114631 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f954816a7f0 con 0x7f954815d240 2023-10-09T06:31:53.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.632+0000 7f9544ff9700 1 -- 172.21.15.163:0/1116114631 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95400070f0 con 0x7f954815d240 2023-10-09T06:31:53.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.632+0000 7f954df81700 1 -- 172.21.15.163:0/1116114631 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f954816d5d0 con 0x7f954815d240 2023-10-09T06:31:53.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.633+0000 7f954df81700 1 -- 172.21.15.163:0/1116114631 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f954816d7f0 con 0x7f954815d240 2023-10-09T06:31:53.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.633+0000 7f9544ff9700 1 -- 172.21.15.163:0/1116114631 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9540007520 con 0x7f954815d240 2023-10-09T06:31:53.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.634+0000 7f9544ff9700 1 --2- 172.21.15.163:0/1116114631 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f952c066f30 0x7f952c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:53.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.634+0000 7f9544ff9700 1 -- 172.21.15.163:0/1116114631 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f95400846e0 con 0x7f954815d240 2023-10-09T06:31:53.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.634+0000 7f954cf7f700 1 --2- 172.21.15.163:0/1116114631 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f952c066f30 0x7f952c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:53.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.634+0000 7f954cf7f700 1 --2- 172.21.15.163:0/1116114631 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f952c066f30 0x7f952c0693e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f953c008b20 tx=0x7f953c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:53.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.638+0000 7f9544ff9700 1 -- 172.21.15.163:0/1116114631 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9540051c70 con 0x7f954815d240 2023-10-09T06:31:53.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.820+0000 7f954df81700 1 -- 172.21.15.163:0/1116114631 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f954804bb80 con 0x7f954815d240 2023-10-09T06:31:53.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.824+0000 7f9544ff9700 1 -- 172.21.15.163:0/1116114631 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9540055290 con 0x7f954815d240 2023-10-09T06:31:53.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.826+0000 7f954df81700 1 -- 172.21.15.163:0/1116114631 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f952c066f30 msgr2=0x7f952c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:53.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.826+0000 7f954df81700 1 --2- 172.21.15.163:0/1116114631 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f952c066f30 0x7f952c0693e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f953c008b20 tx=0x7f953c008040 comp rx=0 tx=0).stop 2023-10-09T06:31:53.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.826+0000 7f954df81700 1 -- 172.21.15.163:0/1116114631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f954815d240 msgr2=0x7f9548169b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:53.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.826+0000 7f954df81700 1 --2- 172.21.15.163:0/1116114631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f954815d240 0x7f9548169b20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f954000cbf0 tx=0x7f95400019b0 comp rx=0 tx=0).stop 2023-10-09T06:31:53.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.826+0000 7f954df81700 1 -- 172.21.15.163:0/1116114631 shutdown_connections 2023-10-09T06:31:53.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.826+0000 7f954df81700 1 --2- 172.21.15.163:0/1116114631 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f952c066f30 0x7f952c0693e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:53.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.826+0000 7f954df81700 1 --2- 172.21.15.163:0/1116114631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f954815d240 0x7f9548169b20 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:53.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.826+0000 7f954df81700 1 --2- 172.21.15.163:0/1116114631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95480b16d0 0x7f954816daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:53.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.826+0000 7f954df81700 1 --2- 172.21.15.163:0/1116114631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95480b4e70 0x7f954815b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:53.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.826+0000 7f954df81700 1 -- 172.21.15.163:0/1116114631 >> 172.21.15.163:0/1116114631 conn(0x7f95480d10c0 msgr2=0x7f95480ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:53.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.826+0000 7f954df81700 1 -- 172.21.15.163:0/1116114631 shutdown_connections 2023-10-09T06:31:53.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.826+0000 7f954df81700 1 -- 172.21.15.163:0/1116114631 wait complete. 2023-10-09T06:31:53.838 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create' 2023-10-09T06:31:53.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.952+0000 7faa37fd9700 1 Processor -- start 2023-10-09T06:31:53.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.952+0000 7faa37fd9700 1 -- start start 2023-10-09T06:31:53.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.953+0000 7faa37fd9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faa3006bcd0 0x7faa300caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:53.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.953+0000 7faa37fd9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa3015df50 0x7faa3015e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:53.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.953+0000 7faa37fd9700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa3015d7e0 con 0x7faa3006c0a0 2023-10-09T06:31:53.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.953+0000 7faa37fd9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa3015dac0 con 0x7faa3006bcd0 2023-10-09T06:31:53.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.953+0000 7faa37fd9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa3015aa30 con 0x7faa3015df50 2023-10-09T06:31:53.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.953+0000 7faa35574700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7faa3006c0a0 0x7faa3015a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:44730/0 (socket says 172.21.15.163:44730) 2023-10-09T06:31:53.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.953+0000 7faa35574700 1 -- 172.21.15.163:0/421883712 learned_addr learned my addr 172.21.15.163:0/421883712 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:53.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.953+0000 7faa35d75700 1 --2- 172.21.15.163:0/421883712 >> v2:172.21.15.181:3300/0 conn(0x7faa3006bcd0 0x7faa300caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:53.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.953+0000 7faa36576700 1 --2- 172.21.15.163:0/421883712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa3015df50 0x7faa3015e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:53.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.953+0000 7faa34d73700 1 -- 172.21.15.163:0/421883712 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2238281621 0 0) 0x7faa3015d7e0 con 0x7faa3006c0a0 2023-10-09T06:31:53.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.953+0000 7faa34d73700 1 -- 172.21.15.163:0/421883712 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa1c003570 con 0x7faa3006c0a0 2023-10-09T06:31:53.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.953+0000 7faa34d73700 1 -- 172.21.15.163:0/421883712 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 186286808 0 0) 0x7faa1c003570 con 0x7faa3006c0a0 2023-10-09T06:31:53.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.953+0000 7faa34d73700 1 -- 172.21.15.163:0/421883712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa3015df50 msgr2=0x7faa3015e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:53.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.953+0000 7faa34d73700 1 --2- 172.21.15.163:0/421883712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa3015df50 0x7faa3015e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:53.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.953+0000 7faa34d73700 1 -- 172.21.15.163:0/421883712 >> v2:172.21.15.181:3300/0 conn(0x7faa3006bcd0 msgr2=0x7faa300caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:53.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.953+0000 7faa34d73700 1 --2- 172.21.15.163:0/421883712 >> v2:172.21.15.181:3300/0 conn(0x7faa3006bcd0 0x7faa300caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:53.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.953+0000 7faa34d73700 1 -- 172.21.15.163:0/421883712 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa3015ad10 con 0x7faa3006c0a0 2023-10-09T06:31:53.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.953+0000 7faa34d73700 1 -- 172.21.15.163:0/421883712 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7faa20001470 con 0x7faa3006c0a0 2023-10-09T06:31:53.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.953+0000 7faa34d73700 1 -- 172.21.15.163:0/421883712 >> v1:172.21.15.163:6789/0 conn(0x7faa3006c0a0 legacy=0x7faa3015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:53.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.953+0000 7faa34d73700 1 --2- 172.21.15.163:0/421883712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa1c003a60 0x7faa1c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:53.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.953+0000 7faa34d73700 1 --2- 172.21.15.163:0/421883712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa1c004810 0x7faa1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:53.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.953+0000 7faa34d73700 1 --2- 172.21.15.163:0/421883712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa3015df50 0x7faa1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:53.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.954+0000 7faa34d73700 1 -- 172.21.15.163:0/421883712 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3015dac0 con 0x7faa1c004810 2023-10-09T06:31:53.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.954+0000 7faa34d73700 1 -- 172.21.15.163:0/421883712 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3015aa30 con 0x7faa1c003a60 2023-10-09T06:31:53.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.954+0000 7faa34d73700 1 -- 172.21.15.163:0/421883712 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c00bae0 con 0x7faa3015df50 2023-10-09T06:31:53.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.954+0000 7faa35574700 1 --2- 172.21.15.163:0/421883712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa1c004810 0x7faa1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:53.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.954+0000 7faa35d75700 1 --2- 172.21.15.163:0/421883712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa1c003a60 0x7faa1c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:53.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.954+0000 7faa36576700 1 --2- 172.21.15.163:0/421883712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa3015df50 0x7faa1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:53.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.954+0000 7faa35574700 1 -- 172.21.15.163:0/421883712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa3015df50 msgr2=0x7faa1c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:53.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.954+0000 7faa35574700 1 --2- 172.21.15.163:0/421883712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa3015df50 0x7faa1c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:53.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.954+0000 7faa35574700 1 -- 172.21.15.163:0/421883712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa1c003a60 msgr2=0x7faa1c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:53.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.954+0000 7faa35574700 1 --2- 172.21.15.163:0/421883712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa1c003a60 0x7faa1c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:53.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.954+0000 7faa35574700 1 -- 172.21.15.163:0/421883712 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c003570 con 0x7faa1c004810 2023-10-09T06:31:53.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.954+0000 7faa35574700 1 --2- 172.21.15.163:0/421883712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa1c004810 0x7faa1c004c80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7faa2000c3c0 tx=0x7faa20001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c05414e49f277c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:53.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.954+0000 7faa34d73700 1 -- 172.21.15.163:0/421883712 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2000cc90 con 0x7faa1c004810 2023-10-09T06:31:53.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.954+0000 7faa34d73700 1 -- 172.21.15.163:0/421883712 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa2000ce10 con 0x7faa1c004810 2023-10-09T06:31:53.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.954+0000 7faa34d73700 1 -- 172.21.15.163:0/421883712 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa20005b70 con 0x7faa1c004810 2023-10-09T06:31:53.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.955+0000 7faa37fd9700 1 -- 172.21.15.163:0/421883712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa1c004810 msgr2=0x7faa1c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:53.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.955+0000 7faa37fd9700 1 --2- 172.21.15.163:0/421883712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa1c004810 0x7faa1c004c80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7faa2000c3c0 tx=0x7faa20001e90 comp rx=0 tx=0).stop 2023-10-09T06:31:53.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.955+0000 7faa36576700 1 -- 172.21.15.163:0/421883712 reap_dead start 2023-10-09T06:31:53.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.955+0000 7faa37fd9700 1 -- 172.21.15.163:0/421883712 shutdown_connections 2023-10-09T06:31:53.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.955+0000 7faa37fd9700 1 -- 172.21.15.163:0/421883712 >> 172.21.15.163:0/421883712 conn(0x7faa300ce600 msgr2=0x7faa300cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:53.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.955+0000 7faa37fd9700 1 -- 172.21.15.163:0/421883712 shutdown_connections 2023-10-09T06:31:53.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.955+0000 7faa37fd9700 1 -- 172.21.15.163:0/421883712 wait complete. 2023-10-09T06:31:53.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.956+0000 7faa37fd9700 1 Processor -- start 2023-10-09T06:31:53.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.956+0000 7faa37fd9700 1 -- start start 2023-10-09T06:31:53.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.957+0000 7faa37fd9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa1c004810 0x7faa300b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:53.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.957+0000 7faa37fd9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa301460f0 0x7faa3006a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:53.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.957+0000 7faa37fd9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa30146560 0x7faa3006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:53.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.957+0000 7faa37fd9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3016e270 con 0x7faa30146560 2023-10-09T06:31:53.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.957+0000 7faa37fd9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa300c7760 con 0x7faa301460f0 2023-10-09T06:31:53.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.957+0000 7faa37fd9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa300c7a40 con 0x7faa1c004810 2023-10-09T06:31:53.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.957+0000 7faa36576700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa30146560 0x7faa3006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:53.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.957+0000 7faa36576700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa30146560 0x7faa3006ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39786/0 (socket says 172.21.15.163:39786) 2023-10-09T06:31:53.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.957+0000 7faa36576700 1 -- 172.21.15.163:0/3389896110 learned_addr learned my addr 172.21.15.163:0/3389896110 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:53.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.957+0000 7faa35574700 1 --2- 172.21.15.163:0/3389896110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa301460f0 0x7faa3006a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:53.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.957+0000 7faa35d75700 1 --2- 172.21.15.163:0/3389896110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa1c004810 0x7faa300b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:53.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.957+0000 7faa36576700 1 -- 172.21.15.163:0/3389896110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa1c004810 msgr2=0x7faa300b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:53.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.957+0000 7faa36576700 1 --2- 172.21.15.163:0/3389896110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa1c004810 0x7faa300b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:53.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.957+0000 7faa36576700 1 -- 172.21.15.163:0/3389896110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa301460f0 msgr2=0x7faa3006a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:53.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.957+0000 7faa36576700 1 --2- 172.21.15.163:0/3389896110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa301460f0 0x7faa3006a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:53.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.957+0000 7faa36576700 1 -- 172.21.15.163:0/3389896110 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa300c7d20 con 0x7faa30146560 2023-10-09T06:31:53.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.957+0000 7faa36576700 1 --2- 172.21.15.163:0/3389896110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa30146560 0x7faa3006ab40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7faa3006b640 tx=0x7faa2800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:53.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.957+0000 7faa25ffb700 1 -- 172.21.15.163:0/3389896110 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2800e070 con 0x7faa30146560 2023-10-09T06:31:53.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.957+0000 7faa25ffb700 1 -- 172.21.15.163:0/3389896110 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa28011b80 con 0x7faa30146560 2023-10-09T06:31:53.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.957+0000 7faa37fd9700 1 -- 172.21.15.163:0/3389896110 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa300c8000 con 0x7faa30146560 2023-10-09T06:31:53.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.957+0000 7faa25ffb700 1 -- 172.21.15.163:0/3389896110 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa28016470 con 0x7faa30146560 2023-10-09T06:31:53.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.958+0000 7faa37fd9700 1 -- 172.21.15.163:0/3389896110 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa300c56b0 con 0x7faa30146560 2023-10-09T06:31:53.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.958+0000 7faa37fd9700 1 -- 172.21.15.163:0/3389896110 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa300c8520 con 0x7faa30146560 2023-10-09T06:31:53.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.959+0000 7faa25ffb700 1 -- 172.21.15.163:0/3389896110 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faa28016820 con 0x7faa30146560 2023-10-09T06:31:53.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.959+0000 7faa25ffb700 1 --2- 172.21.15.163:0/3389896110 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faa14066f30 0x7faa140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:53.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.959+0000 7faa25ffb700 1 -- 172.21.15.163:0/3389896110 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5025+0+0 (secure 0 0 0) 0x7faa28084c60 con 0x7faa30146560 2023-10-09T06:31:53.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.960+0000 7faa35d75700 1 --2- 172.21.15.163:0/3389896110 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faa14066f30 0x7faa140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:53.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.960+0000 7faa35d75700 1 --2- 172.21.15.163:0/3389896110 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faa14066f30 0x7faa140693e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faa2c006f00 tx=0x7faa2c0091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:53.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:53.964+0000 7faa25ffb700 1 -- 172.21.15.163:0/3389896110 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa300c8520 con 0x7faa30146560 2023-10-09T06:31:54.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.221+0000 7faa37fd9700 1 -- 172.21.15.163:0/3389896110 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]} v 0) v1 -- 0x7faa300c8520 con 0x7faa30146560 2023-10-09T06:31:54.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.495+0000 7faa25ffb700 1 -- 172.21.15.163:0/3389896110 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]}]=0 v90) v1 ==== 135+0+0 (secure 0 0 0) 0x7faa300c8520 con 0x7faa30146560 2023-10-09T06:31:54.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.498+0000 7faa37fd9700 1 -- 172.21.15.163:0/3389896110 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faa14066f30 msgr2=0x7faa140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:54.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.498+0000 7faa37fd9700 1 --2- 172.21.15.163:0/3389896110 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faa14066f30 0x7faa140693e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faa2c006f00 tx=0x7faa2c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:54.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.498+0000 7faa37fd9700 1 -- 172.21.15.163:0/3389896110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa30146560 msgr2=0x7faa3006ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:54.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.498+0000 7faa37fd9700 1 --2- 172.21.15.163:0/3389896110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa30146560 0x7faa3006ab40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7faa3006b640 tx=0x7faa2800b040 comp rx=0 tx=0).stop 2023-10-09T06:31:54.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.498+0000 7faa37fd9700 1 -- 172.21.15.163:0/3389896110 shutdown_connections 2023-10-09T06:31:54.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.498+0000 7faa37fd9700 1 --2- 172.21.15.163:0/3389896110 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faa14066f30 0x7faa140693e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:54.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.498+0000 7faa37fd9700 1 --2- 172.21.15.163:0/3389896110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa30146560 0x7faa3006ab40 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:54.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.498+0000 7faa37fd9700 1 --2- 172.21.15.163:0/3389896110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa1c004810 0x7faa300b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:54.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.498+0000 7faa37fd9700 1 --2- 172.21.15.163:0/3389896110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa301460f0 0x7faa3006a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:54.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.498+0000 7faa37fd9700 1 -- 172.21.15.163:0/3389896110 >> 172.21.15.163:0/3389896110 conn(0x7faa300ce600 msgr2=0x7faa3015c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:54.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.498+0000 7faa37fd9700 1 -- 172.21.15.163:0/3389896110 shutdown_connections 2023-10-09T06:31:54.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.498+0000 7faa37fd9700 1 -- 172.21.15.163:0/3389896110 wait complete. 2023-10-09T06:31:54.510 INFO:tasks.cephfs_test_runner:That the volume can be created and then cleans up ... ok 2023-10-09T06:31:54.510 INFO:tasks.cephfs_test_runner:Starting test: test_volume_info (tasks.cephfs.test_volumes.TestVolumes) Tests the 'fs volume info' command 2023-10-09T06:31:54.510 INFO:tasks.cephfs_test_runner:test_volume_info (tasks.cephfs.test_volumes.TestVolumes) 2023-10-09T06:31:54.511 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info' 2023-10-09T06:31:54.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.627+0000 7f616b59e700 1 Processor -- start 2023-10-09T06:31:54.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.627+0000 7f616b59e700 1 -- start start 2023-10-09T06:31:54.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.628+0000 7f616b59e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f616c159420 0x7f616c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:54.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.628+0000 7f616b59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f616c1597f0 0x7f616c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:54.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.628+0000 7f616b59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f616c0b11d0 con 0x7f616c15d230 2023-10-09T06:31:54.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.628+0000 7f616b59e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f616c0b14b0 con 0x7f616c159420 2023-10-09T06:31:54.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.628+0000 7f616b59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f616c0b1790 con 0x7f616c1597f0 2023-10-09T06:31:54.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.628+0000 7f616ad9d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f616c15d230 0x7f616c0b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:44744/0 (socket says 172.21.15.163:44744) 2023-10-09T06:31:54.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.628+0000 7f616ad9d700 1 -- 172.21.15.163:0/1725712437 learned_addr learned my addr 172.21.15.163:0/1725712437 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:54.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.628+0000 7f616a59c700 1 --2- 172.21.15.163:0/1725712437 >> v2:172.21.15.181:3300/0 conn(0x7f616c159420 0x7f616c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:54.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.628+0000 7f6169d9b700 1 --2- 172.21.15.163:0/1725712437 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f616c1597f0 0x7f616c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:54.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.628+0000 7f616959a700 1 -- 172.21.15.163:0/1725712437 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3361753680 0 0) 0x7f616c0b11d0 con 0x7f616c15d230 2023-10-09T06:31:54.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.629+0000 7f616959a700 1 -- 172.21.15.163:0/1725712437 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6154003570 con 0x7f616c15d230 2023-10-09T06:31:54.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.629+0000 7f616959a700 1 -- 172.21.15.163:0/1725712437 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1287990457 0 0) 0x7f6154003570 con 0x7f616c15d230 2023-10-09T06:31:54.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.629+0000 7f616959a700 1 -- 172.21.15.163:0/1725712437 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f616c1597f0 msgr2=0x7f616c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:54.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.629+0000 7f616959a700 1 --2- 172.21.15.163:0/1725712437 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f616c1597f0 0x7f616c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:54.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.629+0000 7f616959a700 1 -- 172.21.15.163:0/1725712437 >> v2:172.21.15.181:3300/0 conn(0x7f616c159420 msgr2=0x7f616c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:54.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.629+0000 7f616959a700 1 --2- 172.21.15.163:0/1725712437 >> v2:172.21.15.181:3300/0 conn(0x7f616c159420 0x7f616c0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:54.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.629+0000 7f616959a700 1 -- 172.21.15.163:0/1725712437 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616c0b1a70 con 0x7f616c15d230 2023-10-09T06:31:54.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.629+0000 7f616959a700 1 -- 172.21.15.163:0/1725712437 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f6164001470 con 0x7f616c15d230 2023-10-09T06:31:54.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.629+0000 7f616959a700 1 -- 172.21.15.163:0/1725712437 >> v1:172.21.15.163:6789/0 conn(0x7f616c15d230 legacy=0x7f616c0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:54.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.629+0000 7f616959a700 1 --2- 172.21.15.163:0/1725712437 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f616c1597f0 0x7f6154003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:54.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.629+0000 7f616959a700 1 --2- 172.21.15.163:0/1725712437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6154003fe0 0x7f6154004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:54.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.629+0000 7f616959a700 1 --2- 172.21.15.163:0/1725712437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6154004d50 0x7f61540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:54.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.629+0000 7f616959a700 1 -- 172.21.15.163:0/1725712437 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616c0b14b0 con 0x7f6154003fe0 2023-10-09T06:31:54.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.629+0000 7f616959a700 1 -- 172.21.15.163:0/1725712437 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616c0b1790 con 0x7f6154004d50 2023-10-09T06:31:54.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.629+0000 7f616959a700 1 -- 172.21.15.163:0/1725712437 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f615400ba90 con 0x7f616c1597f0 2023-10-09T06:31:54.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.629+0000 7f6169d9b700 1 --2- 172.21.15.163:0/1725712437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6154003fe0 0x7f6154004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:54.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.629+0000 7f616ad9d700 1 --2- 172.21.15.163:0/1725712437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6154004d50 0x7f61540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:54.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.629+0000 7f616a59c700 1 --2- 172.21.15.163:0/1725712437 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f616c1597f0 0x7f6154003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:54.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.629+0000 7f6169d9b700 1 -- 172.21.15.163:0/1725712437 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f616c1597f0 msgr2=0x7f6154003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:54.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.629+0000 7f6169d9b700 1 --2- 172.21.15.163:0/1725712437 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f616c1597f0 0x7f6154003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:54.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.629+0000 7f6169d9b700 1 -- 172.21.15.163:0/1725712437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6154004d50 msgr2=0x7f61540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:54.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.629+0000 7f6169d9b700 1 --2- 172.21.15.163:0/1725712437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6154004d50 0x7f61540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:54.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.629+0000 7f6169d9b700 1 -- 172.21.15.163:0/1725712437 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6154003570 con 0x7f6154003fe0 2023-10-09T06:31:54.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.629+0000 7f616ad9d700 1 --2- 172.21.15.163:0/1725712437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6154004d50 0x7f61540051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:54.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.629+0000 7f6169d9b700 1 --2- 172.21.15.163:0/1725712437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6154003fe0 0x7f6154004810 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6160009c00 tx=0x7f61600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff0e98ea32929c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:54.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.629+0000 7f616959a700 1 -- 172.21.15.163:0/1725712437 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616000e6b0 con 0x7f6154003fe0 2023-10-09T06:31:54.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.629+0000 7f616959a700 1 -- 172.21.15.163:0/1725712437 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616000e810 con 0x7f6154003fe0 2023-10-09T06:31:54.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.629+0000 7f616959a700 1 -- 172.21.15.163:0/1725712437 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616000ebd0 con 0x7f6154003fe0 2023-10-09T06:31:54.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.630+0000 7f616b59e700 1 -- 172.21.15.163:0/1725712437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6154003fe0 msgr2=0x7f6154004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:54.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.630+0000 7f616b59e700 1 --2- 172.21.15.163:0/1725712437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6154003fe0 0x7f6154004810 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6160009c00 tx=0x7f61600091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:54.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.630+0000 7f616ad9d700 1 -- 172.21.15.163:0/1725712437 reap_dead start 2023-10-09T06:31:54.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.630+0000 7f616b59e700 1 -- 172.21.15.163:0/1725712437 shutdown_connections 2023-10-09T06:31:54.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.630+0000 7f616b59e700 1 -- 172.21.15.163:0/1725712437 >> 172.21.15.163:0/1725712437 conn(0x7f616c0d0660 msgr2=0x7f616c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:54.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.630+0000 7f616b59e700 1 -- 172.21.15.163:0/1725712437 shutdown_connections 2023-10-09T06:31:54.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.630+0000 7f616b59e700 1 -- 172.21.15.163:0/1725712437 wait complete. 2023-10-09T06:31:54.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.631+0000 7f616b59e700 1 Processor -- start 2023-10-09T06:31:54.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.631+0000 7f616b59e700 1 -- start start 2023-10-09T06:31:54.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.632+0000 7f616b59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6154003fe0 0x7f616c150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:54.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.632+0000 7f616b59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f616c152a80 0x7f616c150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:54.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.632+0000 7f616b59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f616c0b54d0 0x7f616c0b5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:54.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.632+0000 7f616b59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616c151cc0 con 0x7f6154003fe0 2023-10-09T06:31:54.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.632+0000 7f616b59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616c1696e0 con 0x7f616c0b54d0 2023-10-09T06:31:54.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.632+0000 7f616b59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f616c1507d0 con 0x7f616c152a80 2023-10-09T06:31:54.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.632+0000 7f616a59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6154003fe0 0x7f616c150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:54.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.632+0000 7f616a59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6154003fe0 0x7f616c150290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39804/0 (socket says 172.21.15.163:39804) 2023-10-09T06:31:54.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.632+0000 7f616a59c700 1 -- 172.21.15.163:0/3144238905 learned_addr learned my addr 172.21.15.163:0/3144238905 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:54.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.632+0000 7f616a59c700 1 -- 172.21.15.163:0/3144238905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f616c152a80 msgr2=0x7f616c150ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:54.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.632+0000 7f616a59c700 1 --2- 172.21.15.163:0/3144238905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f616c152a80 0x7f616c150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:54.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.632+0000 7f616a59c700 1 -- 172.21.15.163:0/3144238905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f616c0b54d0 msgr2=0x7f616c0b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:54.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.632+0000 7f616a59c700 1 --2- 172.21.15.163:0/3144238905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f616c0b54d0 0x7f616c0b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:54.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.632+0000 7f616a59c700 1 -- 172.21.15.163:0/3144238905 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616c0c5510 con 0x7f6154003fe0 2023-10-09T06:31:54.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.632+0000 7f616a59c700 1 --2- 172.21.15.163:0/3144238905 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6154003fe0 0x7f616c150290 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f615c0115b0 tx=0x7f615c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:54.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.632+0000 7f615a7fc700 1 -- 172.21.15.163:0/3144238905 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c00e030 con 0x7f6154003fe0 2023-10-09T06:31:54.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.632+0000 7f615a7fc700 1 -- 172.21.15.163:0/3144238905 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615c016470 con 0x7f6154003fe0 2023-10-09T06:31:54.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.632+0000 7f615a7fc700 1 -- 172.21.15.163:0/3144238905 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c016810 con 0x7f6154003fe0 2023-10-09T06:31:54.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.632+0000 7f616b59e700 1 -- 172.21.15.163:0/3144238905 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f616c0b5e80 con 0x7f6154003fe0 2023-10-09T06:31:54.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.633+0000 7f616b59e700 1 -- 172.21.15.163:0/3144238905 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f616c151700 con 0x7f6154003fe0 2023-10-09T06:31:54.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.634+0000 7f616b59e700 1 -- 172.21.15.163:0/3144238905 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f616c169a70 con 0x7f6154003fe0 2023-10-09T06:31:54.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.634+0000 7f615a7fc700 1 -- 172.21.15.163:0/3144238905 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f615c016bc0 con 0x7f6154003fe0 2023-10-09T06:31:54.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.634+0000 7f615a7fc700 1 --2- 172.21.15.163:0/3144238905 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f614c066f30 0x7f614c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:54.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.634+0000 7f615a7fc700 1 -- 172.21.15.163:0/3144238905 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f615c00f070 con 0x7f6154003fe0 2023-10-09T06:31:54.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.634+0000 7f6169d9b700 1 --2- 172.21.15.163:0/3144238905 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f614c066f30 0x7f614c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:54.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.635+0000 7f6169d9b700 1 --2- 172.21.15.163:0/3144238905 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f614c066f30 0x7f614c0693e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f616c158430 tx=0x7f616001a000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:54.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.638+0000 7f615a7fc700 1 -- 172.21.15.163:0/3144238905 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f615c052810 con 0x7f6154003fe0 2023-10-09T06:31:54.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:54.904+0000 7f616b59e700 1 -- 172.21.15.163:0/3144238905 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info"]} v 0) v1 -- 0x7f616c0b1d50 con 0x7f6154003fe0 2023-10-09T06:31:55.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.497+0000 7f615a7fc700 1 -- 172.21.15.163:0/3144238905 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info"]}]=0 v91) v1 ==== 136+0+0 (secure 0 0 0) 0x7f615c055e30 con 0x7f6154003fe0 2023-10-09T06:31:55.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.499+0000 7f616b59e700 1 -- 172.21.15.163:0/3144238905 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f614c066f30 msgr2=0x7f614c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:55.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.499+0000 7f616b59e700 1 --2- 172.21.15.163:0/3144238905 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f614c066f30 0x7f614c0693e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f616c158430 tx=0x7f616001a000 comp rx=0 tx=0).stop 2023-10-09T06:31:55.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.499+0000 7f616b59e700 1 -- 172.21.15.163:0/3144238905 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6154003fe0 msgr2=0x7f616c150290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:55.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.499+0000 7f616b59e700 1 --2- 172.21.15.163:0/3144238905 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6154003fe0 0x7f616c150290 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f615c0115b0 tx=0x7f615c00b040 comp rx=0 tx=0).stop 2023-10-09T06:31:55.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.499+0000 7f616b59e700 1 -- 172.21.15.163:0/3144238905 shutdown_connections 2023-10-09T06:31:55.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.499+0000 7f616b59e700 1 --2- 172.21.15.163:0/3144238905 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f614c066f30 0x7f614c0693e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:55.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.499+0000 7f616b59e700 1 --2- 172.21.15.163:0/3144238905 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6154003fe0 0x7f616c150290 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:55.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.499+0000 7f616b59e700 1 --2- 172.21.15.163:0/3144238905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f616c0b54d0 0x7f616c0b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:55.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.499+0000 7f616b59e700 1 --2- 172.21.15.163:0/3144238905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f616c152a80 0x7f616c150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:55.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.499+0000 7f616b59e700 1 -- 172.21.15.163:0/3144238905 >> 172.21.15.163:0/3144238905 conn(0x7f616c0d0660 msgr2=0x7f616c0b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:55.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.499+0000 7f616b59e700 1 -- 172.21.15.163:0/3144238905 shutdown_connections 2023-10-09T06:31:55.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.499+0000 7f616b59e700 1 -- 172.21.15.163:0/3144238905 wait complete. 2023-10-09T06:31:55.513 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-09T06:31:55.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.633+0000 7f76032e9700 1 Processor -- start 2023-10-09T06:31:55.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.633+0000 7f76032e9700 1 -- start start 2023-10-09T06:31:55.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.634+0000 7f76032e9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f75fc06c0a0 0x7f75fc0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:55.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.634+0000 7f76032e9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75fc15df50 0x7f75fc15e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:55.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.634+0000 7f76032e9700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75fc15d7e0 con 0x7f75fc06bcd0 2023-10-09T06:31:55.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.634+0000 7f76032e9700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75fc15dac0 con 0x7f75fc06c0a0 2023-10-09T06:31:55.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.634+0000 7f76032e9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc15aa30 con 0x7f75fc15df50 2023-10-09T06:31:55.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.634+0000 7f7600884700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f75fc06c0a0 0x7f75fc0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:55.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.634+0000 7f7601085700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f75fc06bcd0 0x7f75fc15a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:44750/0 (socket says 172.21.15.163:44750) 2023-10-09T06:31:55.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.634+0000 7f7601085700 1 -- 172.21.15.163:0/1377577109 learned_addr learned my addr 172.21.15.163:0/1377577109 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:55.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.635+0000 7f7601886700 1 --2- 172.21.15.163:0/1377577109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75fc15df50 0x7f75fc15e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:55.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.635+0000 7f7600884700 1 -- 172.21.15.163:0/1377577109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75fc15df50 msgr2=0x7f75fc15e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:55.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.635+0000 7f7600884700 1 --2- 172.21.15.163:0/1377577109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75fc15df50 0x7f75fc15e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:55.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.635+0000 7f7600884700 1 -- 172.21.15.163:0/1377577109 >> v1:172.21.15.163:6789/0 conn(0x7f75fc06bcd0 legacy=0x7f75fc15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:55.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.635+0000 7f75f3fff700 1 -- 172.21.15.163:0/1377577109 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1127230018 0 0) 0x7f75fc15d7e0 con 0x7f75fc06bcd0 2023-10-09T06:31:55.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.635+0000 7f7600884700 1 -- 172.21.15.163:0/1377577109 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75fc15ad10 con 0x7f75fc06c0a0 2023-10-09T06:31:55.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.635+0000 7f7601886700 1 --2- 172.21.15.163:0/1377577109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75fc15df50 0x7f75fc15e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:55.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.635+0000 7f7600884700 1 --2- 172.21.15.163:0/1377577109 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75fc06c0a0 0x7f75fc0caa90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f75ec0098e0 tx=0x7f75ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bdbe053715ae70c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:55.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.635+0000 7f75f3fff700 1 -- 172.21.15.163:0/1377577109 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec009e80 con 0x7f75fc06c0a0 2023-10-09T06:31:55.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.635+0000 7f75f3fff700 1 -- 172.21.15.163:0/1377577109 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75ec00b860 con 0x7f75fc06c0a0 2023-10-09T06:31:55.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.635+0000 7f75f3fff700 1 -- 172.21.15.163:0/1377577109 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec00bbe0 con 0x7f75fc06c0a0 2023-10-09T06:31:55.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.635+0000 7f76032e9700 1 -- 172.21.15.163:0/1377577109 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75fc06c0a0 msgr2=0x7f75fc0caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:55.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.635+0000 7f76032e9700 1 --2- 172.21.15.163:0/1377577109 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75fc06c0a0 0x7f75fc0caa90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f75ec0098e0 tx=0x7f75ec0091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:55.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.635+0000 7f76032e9700 1 -- 172.21.15.163:0/1377577109 shutdown_connections 2023-10-09T06:31:55.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.635+0000 7f76032e9700 1 --2- 172.21.15.163:0/1377577109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75fc15df50 0x7f75fc15e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:55.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.635+0000 7f76032e9700 1 --2- 172.21.15.163:0/1377577109 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75fc06c0a0 0x7f75fc0caa90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:55.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.636+0000 7f76032e9700 1 -- 172.21.15.163:0/1377577109 >> 172.21.15.163:0/1377577109 conn(0x7f75fc0ce600 msgr2=0x7f75fc0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:55.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.636+0000 7f76032e9700 1 -- 172.21.15.163:0/1377577109 shutdown_connections 2023-10-09T06:31:55.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.636+0000 7f76032e9700 1 -- 172.21.15.163:0/1377577109 wait complete. 2023-10-09T06:31:55.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.637+0000 7f76032e9700 1 Processor -- start 2023-10-09T06:31:55.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.637+0000 7f76032e9700 1 -- start start 2023-10-09T06:31:55.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.638+0000 7f76032e9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75fc06bcd0 0x7f75fc0b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:55.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.638+0000 7f76032e9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75fc06c0a0 0x7f75fc0b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:55.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.638+0000 7f76032e9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75fc15df50 0x7f75fc0b32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:55.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.638+0000 7f76032e9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc156eb0 con 0x7f75fc15df50 2023-10-09T06:31:55.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.638+0000 7f76032e9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc0c56b0 con 0x7f75fc06c0a0 2023-10-09T06:31:55.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.638+0000 7f76032e9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc0b37f0 con 0x7f75fc06bcd0 2023-10-09T06:31:55.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.638+0000 7f7601886700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75fc15df50 0x7f75fc0b32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:55.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.638+0000 7f7601886700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75fc15df50 0x7f75fc0b32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39824/0 (socket says 172.21.15.163:39824) 2023-10-09T06:31:55.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.638+0000 7f7601886700 1 -- 172.21.15.163:0/3829471791 learned_addr learned my addr 172.21.15.163:0/3829471791 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:55.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.638+0000 7f7600884700 1 --2- 172.21.15.163:0/3829471791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75fc06c0a0 0x7f75fc0b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:55.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.638+0000 7f7601085700 1 --2- 172.21.15.163:0/3829471791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75fc06bcd0 0x7f75fc0b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:55.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.638+0000 7f7601886700 1 -- 172.21.15.163:0/3829471791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75fc06bcd0 msgr2=0x7f75fc0b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:55.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.638+0000 7f7601886700 1 --2- 172.21.15.163:0/3829471791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75fc06bcd0 0x7f75fc0b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:55.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.638+0000 7f7601886700 1 -- 172.21.15.163:0/3829471791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75fc06c0a0 msgr2=0x7f75fc0b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:55.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.638+0000 7f7601886700 1 --2- 172.21.15.163:0/3829471791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75fc06c0a0 0x7f75fc0b00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:55.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.638+0000 7f7601886700 1 -- 172.21.15.163:0/3829471791 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ec009420 con 0x7f75fc15df50 2023-10-09T06:31:55.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.638+0000 7f7601886700 1 --2- 172.21.15.163:0/3829471791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75fc15df50 0x7f75fc0b32b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f75f400c880 tx=0x7f75f4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:55.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.638+0000 7f75f17fa700 1 -- 172.21.15.163:0/3829471791 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f4011600 con 0x7f75fc15df50 2023-10-09T06:31:55.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.638+0000 7f75f17fa700 1 -- 172.21.15.163:0/3829471791 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f4011760 con 0x7f75fc15df50 2023-10-09T06:31:55.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.638+0000 7f76032e9700 1 -- 172.21.15.163:0/3829471791 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75fc0b2370 con 0x7f75fc15df50 2023-10-09T06:31:55.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.638+0000 7f75f17fa700 1 -- 172.21.15.163:0/3829471791 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f4011b20 con 0x7f75fc15df50 2023-10-09T06:31:55.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.638+0000 7f76032e9700 1 -- 172.21.15.163:0/3829471791 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75fc0b2930 con 0x7f75fc15df50 2023-10-09T06:31:55.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.639+0000 7f76032e9700 1 -- 172.21.15.163:0/3829471791 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75fc0c5840 con 0x7f75fc15df50 2023-10-09T06:31:55.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.640+0000 7f75f17fa700 1 -- 172.21.15.163:0/3829471791 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f75f401a400 con 0x7f75fc15df50 2023-10-09T06:31:55.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.640+0000 7f75f17fa700 1 --2- 172.21.15.163:0/3829471791 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f75e0066f60 0x7f75e0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:55.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.640+0000 7f75f17fa700 1 -- 172.21.15.163:0/3829471791 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f75f4084490 con 0x7f75fc15df50 2023-10-09T06:31:55.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.640+0000 7f7601085700 1 --2- 172.21.15.163:0/3829471791 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f75e0066f60 0x7f75e0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:55.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.641+0000 7f7601085700 1 --2- 172.21.15.163:0/3829471791 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f75e0066f60 0x7f75e0069410 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f75f8007240 tx=0x7f75f8001970 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:55.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.644+0000 7f75f17fa700 1 -- 172.21.15.163:0/3829471791 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f75f40554e0 con 0x7f75fc15df50 2023-10-09T06:31:55.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.826+0000 7f76032e9700 1 -- 172.21.15.163:0/3829471791 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f75fc04c430 con 0x7f75fc15df50 2023-10-09T06:31:55.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.831+0000 7f75f17fa700 1 -- 172.21.15.163:0/3829471791 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f75fc04c430 con 0x7f75fc15df50 2023-10-09T06:31:55.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.833+0000 7f76032e9700 1 -- 172.21.15.163:0/3829471791 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f75e0066f60 msgr2=0x7f75e0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:55.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.833+0000 7f76032e9700 1 --2- 172.21.15.163:0/3829471791 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f75e0066f60 0x7f75e0069410 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f75f8007240 tx=0x7f75f8001970 comp rx=0 tx=0).stop 2023-10-09T06:31:55.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.833+0000 7f76032e9700 1 -- 172.21.15.163:0/3829471791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75fc15df50 msgr2=0x7f75fc0b32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:55.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.833+0000 7f76032e9700 1 --2- 172.21.15.163:0/3829471791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75fc15df50 0x7f75fc0b32b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f75f400c880 tx=0x7f75f4008040 comp rx=0 tx=0).stop 2023-10-09T06:31:55.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.833+0000 7f76032e9700 1 -- 172.21.15.163:0/3829471791 shutdown_connections 2023-10-09T06:31:55.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.833+0000 7f76032e9700 1 --2- 172.21.15.163:0/3829471791 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f75e0066f60 0x7f75e0069410 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:55.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.833+0000 7f76032e9700 1 --2- 172.21.15.163:0/3829471791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75fc15df50 0x7f75fc0b32b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:55.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.833+0000 7f76032e9700 1 --2- 172.21.15.163:0/3829471791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75fc06bcd0 0x7f75fc0b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:55.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.833+0000 7f76032e9700 1 --2- 172.21.15.163:0/3829471791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75fc06c0a0 0x7f75fc0b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:55.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.833+0000 7f76032e9700 1 -- 172.21.15.163:0/3829471791 >> 172.21.15.163:0/3829471791 conn(0x7f75fc0ce600 msgr2=0x7f75fc06a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:55.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.833+0000 7f76032e9700 1 -- 172.21.15.163:0/3829471791 shutdown_connections 2023-10-09T06:31:55.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:55.833+0000 7f76032e9700 1 -- 172.21.15.163:0/3829471791 wait complete. 2023-10-09T06:31:55.844 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-09T06:31:55.845 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:31:55.881 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:31:55.926 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-09T06:31:55.926 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-09T06:31:55.926 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:31:56.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.041+0000 7fd700bf6700 1 Processor -- start 2023-10-09T06:31:56.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.041+0000 7fd700bf6700 1 -- start start 2023-10-09T06:31:56.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.042+0000 7fd700bf6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6fc06bcd0 0x7fd6fc140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:56.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.042+0000 7fd700bf6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd6fc0c9510 0x7fd6fc141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:56.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.042+0000 7fd700bf6700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6fc15c2a0 con 0x7fd6fc06c0a0 2023-10-09T06:31:56.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.042+0000 7fd700bf6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6fc15b690 con 0x7fd6fc0c9510 2023-10-09T06:31:56.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.042+0000 7fd700bf6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc15b970 con 0x7fd6fc06bcd0 2023-10-09T06:31:56.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.042+0000 7fd6f3fff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fd6fc06c0a0 0x7fd6fc0c8e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:44756/0 (socket says 172.21.15.163:44756) 2023-10-09T06:31:56.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.042+0000 7fd6f3fff700 1 -- 172.21.15.163:0/4012023253 learned_addr learned my addr 172.21.15.163:0/4012023253 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:56.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.043+0000 7fd6fad9d700 1 --2- 172.21.15.163:0/4012023253 >> v2:172.21.15.181:3300/0 conn(0x7fd6fc0c9510 0x7fd6fc141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:56.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.043+0000 7fd6fa59c700 1 --2- 172.21.15.163:0/4012023253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6fc06bcd0 0x7fd6fc140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:56.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.043+0000 7fd6f9d9b700 1 -- 172.21.15.163:0/4012023253 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1470297895 0 0) 0x7fd6fc15c2a0 con 0x7fd6fc06c0a0 2023-10-09T06:31:56.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.043+0000 7fd6f9d9b700 1 -- 172.21.15.163:0/4012023253 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6e8003570 con 0x7fd6fc06c0a0 2023-10-09T06:31:56.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.043+0000 7fd6fa59c700 1 -- 172.21.15.163:0/4012023253 >> v2:172.21.15.181:3300/0 conn(0x7fd6fc0c9510 msgr2=0x7fd6fc141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:56.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.043+0000 7fd6fa59c700 1 --2- 172.21.15.163:0/4012023253 >> v2:172.21.15.181:3300/0 conn(0x7fd6fc0c9510 0x7fd6fc141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:56.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.043+0000 7fd6fa59c700 1 -- 172.21.15.163:0/4012023253 >> v1:172.21.15.163:6789/0 conn(0x7fd6fc06c0a0 legacy=0x7fd6fc0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:56.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.043+0000 7fd6fa59c700 1 -- 172.21.15.163:0/4012023253 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6fc15bc50 con 0x7fd6fc06bcd0 2023-10-09T06:31:56.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.043+0000 7fd6fad9d700 1 --2- 172.21.15.163:0/4012023253 >> v2:172.21.15.181:3300/0 conn(0x7fd6fc0c9510 0x7fd6fc141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:31:56.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.043+0000 7fd6fa59c700 1 --2- 172.21.15.163:0/4012023253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6fc06bcd0 0x7fd6fc140df0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0098e0 tx=0x7fd6ec0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cfbd672df8987af server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:56.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.043+0000 7fd6f9d9b700 1 -- 172.21.15.163:0/4012023253 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec011630 con 0x7fd6fc06bcd0 2023-10-09T06:31:56.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.043+0000 7fd6f9d9b700 1 -- 172.21.15.163:0/4012023253 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6ec011790 con 0x7fd6fc06bcd0 2023-10-09T06:31:56.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.043+0000 7fd6f9d9b700 1 -- 172.21.15.163:0/4012023253 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec011b10 con 0x7fd6fc06bcd0 2023-10-09T06:31:56.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.044+0000 7fd700bf6700 1 -- 172.21.15.163:0/4012023253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6fc06bcd0 msgr2=0x7fd6fc140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:56.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.044+0000 7fd700bf6700 1 --2- 172.21.15.163:0/4012023253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6fc06bcd0 0x7fd6fc140df0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0098e0 tx=0x7fd6ec0091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:56.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.044+0000 7fd700bf6700 1 -- 172.21.15.163:0/4012023253 shutdown_connections 2023-10-09T06:31:56.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.044+0000 7fd700bf6700 1 --2- 172.21.15.163:0/4012023253 >> v2:172.21.15.181:3300/0 conn(0x7fd6fc0c9510 0x7fd6fc141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:56.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.044+0000 7fd700bf6700 1 --2- 172.21.15.163:0/4012023253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6fc06bcd0 0x7fd6fc140df0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:56.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.044+0000 7fd700bf6700 1 -- 172.21.15.163:0/4012023253 >> 172.21.15.163:0/4012023253 conn(0x7fd6fc0ce600 msgr2=0x7fd6fc0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:56.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.044+0000 7fd700bf6700 1 -- 172.21.15.163:0/4012023253 shutdown_connections 2023-10-09T06:31:56.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.044+0000 7fd700bf6700 1 -- 172.21.15.163:0/4012023253 wait complete. 2023-10-09T06:31:56.049 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.045+0000 7fd700bf6700 1 Processor -- start 2023-10-09T06:31:56.049 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.045+0000 7fd700bf6700 1 -- start start 2023-10-09T06:31:56.049 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.046+0000 7fd700bf6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6fc06bcd0 0x7fd6fc0b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:56.049 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.046+0000 7fd700bf6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6fc06c0a0 0x7fd6fc0b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:56.049 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.046+0000 7fd700bf6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6fc0c9510 0x7fd6fc069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:56.049 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.046+0000 7fd700bf6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc13c790 con 0x7fd6fc0c9510 2023-10-09T06:31:56.049 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.046+0000 7fd700bf6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc0b3630 con 0x7fd6fc06c0a0 2023-10-09T06:31:56.049 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.046+0000 7fd700bf6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc0b38f0 con 0x7fd6fc06bcd0 2023-10-09T06:31:56.049 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.046+0000 7fd6fad9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6fc0c9510 0x7fd6fc069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:56.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.046+0000 7fd6fad9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6fc0c9510 0x7fd6fc069fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39838/0 (socket says 172.21.15.163:39838) 2023-10-09T06:31:56.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.046+0000 7fd6fad9d700 1 -- 172.21.15.163:0/3585796144 learned_addr learned my addr 172.21.15.163:0/3585796144 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:56.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.046+0000 7fd6f3fff700 1 --2- 172.21.15.163:0/3585796144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6fc06c0a0 0x7fd6fc0b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:56.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.046+0000 7fd6fa59c700 1 --2- 172.21.15.163:0/3585796144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6fc06bcd0 0x7fd6fc0b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:56.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.046+0000 7fd6fad9d700 1 -- 172.21.15.163:0/3585796144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6fc06bcd0 msgr2=0x7fd6fc0b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:56.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.046+0000 7fd6fad9d700 1 --2- 172.21.15.163:0/3585796144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6fc06bcd0 0x7fd6fc0b2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:56.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.046+0000 7fd6fad9d700 1 -- 172.21.15.163:0/3585796144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6fc06c0a0 msgr2=0x7fd6fc0b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:56.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.046+0000 7fd6fad9d700 1 --2- 172.21.15.163:0/3585796144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6fc06c0a0 0x7fd6fc0b30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:56.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.046+0000 7fd6fad9d700 1 -- 172.21.15.163:0/3585796144 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f4009a20 con 0x7fd6fc0c9510 2023-10-09T06:31:56.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.046+0000 7fd6fad9d700 1 --2- 172.21.15.163:0/3585796144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6fc0c9510 0x7fd6fc069fc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009fc0 tx=0x7fd6f400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:56.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.046+0000 7fd6f27fc700 1 -- 172.21.15.163:0/3585796144 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f400cbd0 con 0x7fd6fc0c9510 2023-10-09T06:31:56.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.046+0000 7fd6f27fc700 1 -- 172.21.15.163:0/3585796144 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f400f040 con 0x7fd6fc0c9510 2023-10-09T06:31:56.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.046+0000 7fd700bf6700 1 -- 172.21.15.163:0/3585796144 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ec009420 con 0x7fd6fc0c9510 2023-10-09T06:31:56.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.046+0000 7fd6f27fc700 1 -- 172.21.15.163:0/3585796144 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4014480 con 0x7fd6fc0c9510 2023-10-09T06:31:56.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.046+0000 7fd700bf6700 1 -- 172.21.15.163:0/3585796144 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6fc0c56b0 con 0x7fd6fc0c9510 2023-10-09T06:31:56.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.047+0000 7fd700bf6700 1 -- 172.21.15.163:0/3585796144 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6fc04c3c0 con 0x7fd6fc0c9510 2023-10-09T06:31:56.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.048+0000 7fd6f27fc700 1 -- 172.21.15.163:0/3585796144 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd6f4003020 con 0x7fd6fc0c9510 2023-10-09T06:31:56.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.048+0000 7fd6f27fc700 1 --2- 172.21.15.163:0/3585796144 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd6dc066f30 0x7fd6dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:56.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.048+0000 7fd6f27fc700 1 -- 172.21.15.163:0/3585796144 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fd6f4084bb0 con 0x7fd6fc0c9510 2023-10-09T06:31:56.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.048+0000 7fd6fa59c700 1 --2- 172.21.15.163:0/3585796144 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd6dc066f30 0x7fd6dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:56.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.049+0000 7fd6fa59c700 1 --2- 172.21.15.163:0/3585796144 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd6dc066f30 0x7fd6dc0693e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0091e0 tx=0x7fd6ec017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:56.052 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.052+0000 7fd6f27fc700 1 -- 172.21.15.163:0/3585796144 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd6f40520c0 con 0x7fd6fc0c9510 2023-10-09T06:31:56.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.274+0000 7fd700bf6700 1 -- 172.21.15.163:0/3585796144 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6fc06c9d0 con 0x7fd6fc0c9510 2023-10-09T06:31:56.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.274+0000 7fd6f27fc700 1 -- 172.21.15.163:0/3585796144 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+3541 (secure 0 0 0) 0x7fd6f4018df0 con 0x7fd6fc0c9510 2023-10-09T06:31:56.274 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:56.274 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":25,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4622,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2368375038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2368375038},{"type":"v1","addr":"172.21.15.181:6836","nonce":2368375038}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1871607757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1871607757},{"type":"v1","addr":"172.21.15.181:6837","nonce":1871607757}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":4929,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/4112854733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4112854733},{"type":"v1","addr":"172.21.15.163:6834","nonce":4112854733}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5202,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2200503829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2200503829},{"type":"v1","addr":"172.21.15.163:6835","nonce":2200503829}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[]} 2023-10-09T06:31:56.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.276+0000 7fd700bf6700 1 -- 172.21.15.163:0/3585796144 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd6dc066f30 msgr2=0x7fd6dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:56.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.276+0000 7fd700bf6700 1 --2- 172.21.15.163:0/3585796144 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd6dc066f30 0x7fd6dc0693e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0091e0 tx=0x7fd6ec017000 comp rx=0 tx=0).stop 2023-10-09T06:31:56.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.276+0000 7fd700bf6700 1 -- 172.21.15.163:0/3585796144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6fc0c9510 msgr2=0x7fd6fc069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:56.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.276+0000 7fd700bf6700 1 --2- 172.21.15.163:0/3585796144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6fc0c9510 0x7fd6fc069fc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009fc0 tx=0x7fd6f400b040 comp rx=0 tx=0).stop 2023-10-09T06:31:56.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.276+0000 7fd700bf6700 1 -- 172.21.15.163:0/3585796144 shutdown_connections 2023-10-09T06:31:56.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.276+0000 7fd700bf6700 1 --2- 172.21.15.163:0/3585796144 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd6dc066f30 0x7fd6dc0693e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:56.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.276+0000 7fd700bf6700 1 --2- 172.21.15.163:0/3585796144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6fc0c9510 0x7fd6fc069fc0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:56.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.276+0000 7fd700bf6700 1 --2- 172.21.15.163:0/3585796144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6fc06bcd0 0x7fd6fc0b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:56.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.276+0000 7fd700bf6700 1 --2- 172.21.15.163:0/3585796144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6fc06c0a0 0x7fd6fc0b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:56.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.276+0000 7fd700bf6700 1 -- 172.21.15.163:0/3585796144 >> 172.21.15.163:0/3585796144 conn(0x7fd6fc0ce600 msgr2=0x7fd6fc0c8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:56.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.276+0000 7fd700bf6700 1 -- 172.21.15.163:0/3585796144 shutdown_connections 2023-10-09T06:31:56.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.276+0000 7fd700bf6700 1 -- 172.21.15.163:0/3585796144 wait complete. 2023-10-09T06:31:56.278 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 25 2023-10-09T06:31:56.291 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-09T06:31:56.291 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-09T06:31:56.291 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-09T06:31:56.291 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:31:56.291 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-09T06:31:56.292 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-09T06:31:56.292 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-09T06:31:56.292 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:31:56.292 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-09T06:31:56.292 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-09T06:31:56.292 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-09T06:31:56.292 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:31:56.293 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-09T06:31:56.293 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-09T06:31:56.293 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-09T06:31:56.293 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:31:56.302 INFO:tasks.ceph.mds.a:Stopped 2023-10-09T06:31:56.302 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-09T06:31:56.303 INFO:tasks.ceph.mds.c:Stopped 2023-10-09T06:31:56.303 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-09T06:31:56.304 INFO:tasks.ceph.mds.d:Stopped 2023-10-09T06:31:56.304 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-09T06:31:56.305 INFO:tasks.ceph.mds.b:Stopped 2023-10-09T06:31:56.305 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-09T06:31:56.345 INFO:tasks.ceph.mds.a:Started 2023-10-09T06:31:56.345 DEBUG:teuthology.parallel:result is None 2023-10-09T06:31:56.346 INFO:tasks.ceph.mds.c:Started 2023-10-09T06:31:56.346 DEBUG:teuthology.parallel:result is None 2023-10-09T06:31:56.347 INFO:tasks.ceph.mds.d:Started 2023-10-09T06:31:56.347 DEBUG:teuthology.parallel:result is None 2023-10-09T06:31:56.348 INFO:tasks.ceph.mds.b:Started 2023-10-09T06:31:56.348 DEBUG:teuthology.parallel:result is None 2023-10-09T06:31:56.348 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-09T06:31:56.418 INFO:tasks.ceph.mds.a.smithi163.stdout:starting mds.a at 2023-10-09T06:31:56.419 INFO:tasks.ceph.mds.c.smithi163.stdout:starting mds.c at 2023-10-09T06:31:56.427 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-09T06:31:56.427 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-09T06:31:56.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.470+0000 7f8c22eee700 1 Processor -- start 2023-10-09T06:31:56.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.470+0000 7f8c22eee700 1 -- start start 2023-10-09T06:31:56.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.471+0000 7f8c22eee700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c1c159420 0x7f8c1c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:56.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.471+0000 7f8c22eee700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8c1c1597f0 0x7f8c1c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:56.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.471+0000 7f8c22eee700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8c1c15d230 0x7f8c1c15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:56.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.471+0000 7f8c22eee700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c1c15dbd0 con 0x7f8c1c1597f0 2023-10-09T06:31:56.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.471+0000 7f8c22eee700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c1c0b11d0 con 0x7f8c1c15d230 2023-10-09T06:31:56.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.471+0000 7f8c22eee700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c0b14b0 con 0x7f8c1c159420 2023-10-09T06:31:56.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.471+0000 7f8c216eb700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8c1c1597f0 0x7f8c1c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:56.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.471+0000 7f8c216eb700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8c1c1597f0 0x7f8c1c0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39898/0 (socket says 172.21.15.163:39898) 2023-10-09T06:31:56.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.471+0000 7f8c216eb700 1 -- 172.21.15.163:0/1600023052 learned_addr learned my addr 172.21.15.163:0/1600023052 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:56.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.471+0000 7f8c226ed700 1 --2- 172.21.15.163:0/1600023052 >> v2:172.21.15.181:3300/0 conn(0x7f8c1c15d230 0x7f8c1c15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:56.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.471+0000 7f8c21eec700 1 --2- 172.21.15.163:0/1600023052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c1c159420 0x7f8c1c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:56.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.471+0000 7f8c216eb700 1 -- 172.21.15.163:0/1600023052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c1c159420 msgr2=0x7f8c1c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:56.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.472+0000 7f8c216eb700 1 --2- 172.21.15.163:0/1600023052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c1c159420 0x7f8c1c0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:56.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.472+0000 7f8c216eb700 1 -- 172.21.15.163:0/1600023052 >> v2:172.21.15.181:3300/0 conn(0x7f8c1c15d230 msgr2=0x7f8c1c15d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:56.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.472+0000 7f8c216eb700 1 --2- 172.21.15.163:0/1600023052 >> v2:172.21.15.181:3300/0 conn(0x7f8c1c15d230 0x7f8c1c15d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:56.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.472+0000 7f8c216eb700 1 -- 172.21.15.163:0/1600023052 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1c0b1790 con 0x7f8c1c1597f0 2023-10-09T06:31:56.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.472+0000 7f8c216eb700 1 --2- 172.21.15.163:0/1600023052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c1c1597f0 0x7f8c1c0b2270 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c0d1720 tx=0x7f8c0c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6e4bc8f2ad36b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:56.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.472+0000 7f8c20eea700 1 -- 172.21.15.163:0/1600023052 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c00b640 con 0x7f8c1c1597f0 2023-10-09T06:31:56.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.472+0000 7f8c20eea700 1 -- 172.21.15.163:0/1600023052 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0c00b7a0 con 0x7f8c1c1597f0 2023-10-09T06:31:56.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.472+0000 7f8c20eea700 1 -- 172.21.15.163:0/1600023052 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c00bb60 con 0x7f8c1c1597f0 2023-10-09T06:31:56.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.472+0000 7f8c22eee700 1 -- 172.21.15.163:0/1600023052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c1c1597f0 msgr2=0x7f8c1c0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:56.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.472+0000 7f8c22eee700 1 --2- 172.21.15.163:0/1600023052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c1c1597f0 0x7f8c1c0b2270 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c0d1720 tx=0x7f8c0c009250 comp rx=0 tx=0).stop 2023-10-09T06:31:56.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.472+0000 7f8c22eee700 1 -- 172.21.15.163:0/1600023052 shutdown_connections 2023-10-09T06:31:56.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.472+0000 7f8c22eee700 1 --2- 172.21.15.163:0/1600023052 >> v2:172.21.15.181:3300/0 conn(0x7f8c1c15d230 0x7f8c1c15d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:56.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.472+0000 7f8c22eee700 1 --2- 172.21.15.163:0/1600023052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c1c159420 0x7f8c1c0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:56.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.472+0000 7f8c22eee700 1 --2- 172.21.15.163:0/1600023052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c1c1597f0 0x7f8c1c0b2270 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:56.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.472+0000 7f8c22eee700 1 -- 172.21.15.163:0/1600023052 >> 172.21.15.163:0/1600023052 conn(0x7f8c1c0d0660 msgr2=0x7f8c1c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:56.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.472+0000 7f8c22eee700 1 -- 172.21.15.163:0/1600023052 shutdown_connections 2023-10-09T06:31:56.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.472+0000 7f8c22eee700 1 -- 172.21.15.163:0/1600023052 wait complete. 2023-10-09T06:31:56.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.473+0000 7f8c22eee700 1 Processor -- start 2023-10-09T06:31:56.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.474+0000 7f8c22eee700 1 -- start start 2023-10-09T06:31:56.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.474+0000 7f8c22eee700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c1c159420 0x7f8c1c0cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:56.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.474+0000 7f8c22eee700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c1c1597f0 0x7f8c1c155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:56.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.474+0000 7f8c22eee700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c1c15d230 0x7f8c1c156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:56.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.474+0000 7f8c22eee700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c0c63d0 con 0x7f8c1c159420 2023-10-09T06:31:56.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.474+0000 7f8c22eee700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c0cb950 con 0x7f8c1c1597f0 2023-10-09T06:31:56.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.474+0000 7f8c22eee700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c0cb0a0 con 0x7f8c1c15d230 2023-10-09T06:31:56.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.474+0000 7f8c21eec700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c1c159420 0x7f8c1c0cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:56.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.474+0000 7f8c21eec700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c1c159420 0x7f8c1c0cbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39906/0 (socket says 172.21.15.163:39906) 2023-10-09T06:31:56.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.474+0000 7f8c21eec700 1 -- 172.21.15.163:0/461128647 learned_addr learned my addr 172.21.15.163:0/461128647 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:56.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.474+0000 7f8c226ed700 1 --2- 172.21.15.163:0/461128647 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c1c15d230 0x7f8c1c156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:56.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.474+0000 7f8c216eb700 1 --2- 172.21.15.163:0/461128647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c1c1597f0 0x7f8c1c155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:56.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.475+0000 7f8c21eec700 1 -- 172.21.15.163:0/461128647 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c1c15d230 msgr2=0x7f8c1c156070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:56.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.475+0000 7f8c21eec700 1 --2- 172.21.15.163:0/461128647 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c1c15d230 0x7f8c1c156070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:56.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.475+0000 7f8c21eec700 1 -- 172.21.15.163:0/461128647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c1c1597f0 msgr2=0x7f8c1c155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:56.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.475+0000 7f8c21eec700 1 --2- 172.21.15.163:0/461128647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c1c1597f0 0x7f8c1c155b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:56.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.475+0000 7f8c21eec700 1 -- 172.21.15.163:0/461128647 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c0c0094f0 con 0x7f8c1c159420 2023-10-09T06:31:56.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.475+0000 7f8c226ed700 1 --2- 172.21.15.163:0/461128647 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c1c15d230 0x7f8c1c156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:56.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.475+0000 7f8c216eb700 1 --2- 172.21.15.163:0/461128647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c1c1597f0 0x7f8c1c155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:56.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.475+0000 7f8c21eec700 1 --2- 172.21.15.163:0/461128647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c1c159420 0x7f8c1c0cbde0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800ec80 tx=0x7f8c180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:56.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.475+0000 7f8c11ffb700 1 -- 172.21.15.163:0/461128647 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1800c9d0 con 0x7f8c1c159420 2023-10-09T06:31:56.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.475+0000 7f8c11ffb700 1 -- 172.21.15.163:0/461128647 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c1800cb30 con 0x7f8c1c159420 2023-10-09T06:31:56.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.475+0000 7f8c22eee700 1 -- 172.21.15.163:0/461128647 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1c0c60f0 con 0x7f8c1c159420 2023-10-09T06:31:56.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.475+0000 7f8c11ffb700 1 -- 172.21.15.163:0/461128647 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c18013470 con 0x7f8c1c159420 2023-10-09T06:31:56.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.475+0000 7f8c22eee700 1 -- 172.21.15.163:0/461128647 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c1c0cb5e0 con 0x7f8c1c159420 2023-10-09T06:31:56.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.476+0000 7f8c22eee700 1 -- 172.21.15.163:0/461128647 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c1c0cbc30 con 0x7f8c1c159420 2023-10-09T06:31:56.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.476+0000 7f8c11ffb700 1 -- 172.21.15.163:0/461128647 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8c18013820 con 0x7f8c1c159420 2023-10-09T06:31:56.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.477+0000 7f8c11ffb700 1 --2- 172.21.15.163:0/461128647 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8c00066f30 0x7f8c000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:56.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.477+0000 7f8c11ffb700 1 -- 172.21.15.163:0/461128647 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f8c18084680 con 0x7f8c1c159420 2023-10-09T06:31:56.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.477+0000 7f8c216eb700 1 --2- 172.21.15.163:0/461128647 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8c00066f30 0x7f8c000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:56.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.478+0000 7f8c216eb700 1 --2- 172.21.15.163:0/461128647 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8c00066f30 0x7f8c000693e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c0d1720 tx=0x7f8c0c017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:56.481 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.481+0000 7f8c11ffb700 1 -- 172.21.15.163:0/461128647 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c18051b90 con 0x7f8c1c159420 2023-10-09T06:31:56.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:56.682+0000 7f8c22eee700 1 -- 172.21.15.163:0/461128647 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8c1c04c3c0 con 0x7f8c1c159420 2023-10-09T06:31:57.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.513+0000 7f8c11ffb700 1 -- 172.21.15.163:0/461128647 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v47) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8c180551b0 con 0x7f8c1c159420 2023-10-09T06:31:57.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.516+0000 7f8c22eee700 1 -- 172.21.15.163:0/461128647 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8c00066f30 msgr2=0x7f8c000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:57.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.516+0000 7f8c22eee700 1 --2- 172.21.15.163:0/461128647 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8c00066f30 0x7f8c000693e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c0d1720 tx=0x7f8c0c017000 comp rx=0 tx=0).stop 2023-10-09T06:31:57.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.516+0000 7f8c22eee700 1 -- 172.21.15.163:0/461128647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c1c159420 msgr2=0x7f8c1c0cbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:57.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.516+0000 7f8c22eee700 1 --2- 172.21.15.163:0/461128647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c1c159420 0x7f8c1c0cbde0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800ec80 tx=0x7f8c180091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:57.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.516+0000 7f8c22eee700 1 -- 172.21.15.163:0/461128647 shutdown_connections 2023-10-09T06:31:57.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.516+0000 7f8c22eee700 1 --2- 172.21.15.163:0/461128647 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8c00066f30 0x7f8c000693e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:57.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.516+0000 7f8c22eee700 1 --2- 172.21.15.163:0/461128647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c1c159420 0x7f8c1c0cbde0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:57.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.516+0000 7f8c22eee700 1 --2- 172.21.15.163:0/461128647 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c1c15d230 0x7f8c1c156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:57.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.516+0000 7f8c22eee700 1 --2- 172.21.15.163:0/461128647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c1c1597f0 0x7f8c1c155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:57.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.516+0000 7f8c22eee700 1 -- 172.21.15.163:0/461128647 >> 172.21.15.163:0/461128647 conn(0x7f8c1c0d0660 msgr2=0x7f8c1c0b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:57.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.516+0000 7f8c22eee700 1 -- 172.21.15.163:0/461128647 shutdown_connections 2023-10-09T06:31:57.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.516+0000 7f8c22eee700 1 -- 172.21.15.163:0/461128647 wait complete. 2023-10-09T06:31:57.517 INFO:teuthology.orchestra.run.smithi163.stderr: removed all blocklist entries 2023-10-09T06:31:57.530 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-09T06:31:57.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.649+0000 7f82ffd9d700 1 Processor -- start 2023-10-09T06:31:57.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.650+0000 7f82ffd9d700 1 -- start start 2023-10-09T06:31:57.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.650+0000 7f82ffd9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82f806bcd0 0x7f82f8140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:57.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.650+0000 7f82ffd9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f82f80c9510 0x7f82f8141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:57.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.650+0000 7f82ffd9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f815c2a0 con 0x7f82f806c0a0 2023-10-09T06:31:57.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.650+0000 7f82ffd9d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f815b690 con 0x7f82f80c9510 2023-10-09T06:31:57.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.650+0000 7f82ffd9d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f815b970 con 0x7f82f806bcd0 2023-10-09T06:31:57.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.650+0000 7f82fd338700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f82f806c0a0 0x7f82f80c8e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:44784/0 (socket says 172.21.15.163:44784) 2023-10-09T06:31:57.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.650+0000 7f82fd338700 1 -- 172.21.15.163:0/1065974011 learned_addr learned my addr 172.21.15.163:0/1065974011 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:57.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.651+0000 7f82fcb37700 1 -- 172.21.15.163:0/1065974011 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2640279242 0 0) 0x7f82f815c2a0 con 0x7f82f806c0a0 2023-10-09T06:31:57.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.651+0000 7f82fcb37700 1 -- 172.21.15.163:0/1065974011 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82e4003570 con 0x7f82f806c0a0 2023-10-09T06:31:57.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.651+0000 7f82fe33a700 1 --2- 172.21.15.163:0/1065974011 >> v2:172.21.15.181:3300/0 conn(0x7f82f80c9510 0x7f82f8141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:57.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.651+0000 7f82fdb39700 1 --2- 172.21.15.163:0/1065974011 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82f806bcd0 0x7f82f8140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:57.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.651+0000 7f82fcb37700 1 -- 172.21.15.163:0/1065974011 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1772123025 0 0) 0x7f82e4003570 con 0x7f82f806c0a0 2023-10-09T06:31:57.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.651+0000 7f82fcb37700 1 -- 172.21.15.163:0/1065974011 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82f806bcd0 msgr2=0x7f82f8140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:57.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.651+0000 7f82fcb37700 1 --2- 172.21.15.163:0/1065974011 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82f806bcd0 0x7f82f8140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:57.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.651+0000 7f82fcb37700 1 -- 172.21.15.163:0/1065974011 >> v2:172.21.15.181:3300/0 conn(0x7f82f80c9510 msgr2=0x7f82f8141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:57.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.651+0000 7f82fcb37700 1 --2- 172.21.15.163:0/1065974011 >> v2:172.21.15.181:3300/0 conn(0x7f82f80c9510 0x7f82f8141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:57.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.651+0000 7f82fcb37700 1 -- 172.21.15.163:0/1065974011 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f815bc50 con 0x7f82f806c0a0 2023-10-09T06:31:57.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.651+0000 7f82fcb37700 1 -- 172.21.15.163:0/1065974011 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f82e8001470 con 0x7f82f806c0a0 2023-10-09T06:31:57.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.651+0000 7f82fcb37700 1 -- 172.21.15.163:0/1065974011 >> v1:172.21.15.163:6789/0 conn(0x7f82f806c0a0 legacy=0x7f82f80c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:57.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.651+0000 7f82fcb37700 1 --2- 172.21.15.163:0/1065974011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82e4003a60 0x7f82e4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:57.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.651+0000 7f82fcb37700 1 --2- 172.21.15.163:0/1065974011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82e4004810 0x7f82e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:57.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.651+0000 7f82fcb37700 1 --2- 172.21.15.163:0/1065974011 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82f806bcd0 0x7f82e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:57.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.651+0000 7f82fcb37700 1 -- 172.21.15.163:0/1065974011 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f815b690 con 0x7f82e4004810 2023-10-09T06:31:57.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.651+0000 7f82fcb37700 1 -- 172.21.15.163:0/1065974011 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f815b970 con 0x7f82e4003a60 2023-10-09T06:31:57.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.651+0000 7f82fcb37700 1 -- 172.21.15.163:0/1065974011 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e400ba60 con 0x7f82f806bcd0 2023-10-09T06:31:57.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.651+0000 7f82fd338700 1 --2- 172.21.15.163:0/1065974011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82e4004810 0x7f82e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:57.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.651+0000 7f82fdb39700 1 --2- 172.21.15.163:0/1065974011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82e4003a60 0x7f82e4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:57.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.651+0000 7f82fe33a700 1 --2- 172.21.15.163:0/1065974011 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82f806bcd0 0x7f82e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:57.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.651+0000 7f82fd338700 1 -- 172.21.15.163:0/1065974011 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82f806bcd0 msgr2=0x7f82e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:57.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.651+0000 7f82fd338700 1 --2- 172.21.15.163:0/1065974011 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82f806bcd0 0x7f82e40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:57.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.651+0000 7f82fd338700 1 -- 172.21.15.163:0/1065974011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82e4003a60 msgr2=0x7f82e4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:57.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.651+0000 7f82fd338700 1 --2- 172.21.15.163:0/1065974011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82e4003a60 0x7f82e4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:57.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.651+0000 7f82fd338700 1 -- 172.21.15.163:0/1065974011 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e4003570 con 0x7f82e4004810 2023-10-09T06:31:57.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.651+0000 7f82fdb39700 1 --2- 172.21.15.163:0/1065974011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82e4003a60 0x7f82e4003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:57.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.652+0000 7f82fd338700 1 --2- 172.21.15.163:0/1065974011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82e4004810 0x7f82e4004c80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f82e800c3f0 tx=0x7f82f806e470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7b2fa138fd853b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:57.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.652+0000 7f82fcb37700 1 -- 172.21.15.163:0/1065974011 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e800cc80 con 0x7f82e4004810 2023-10-09T06:31:57.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.652+0000 7f82fcb37700 1 -- 172.21.15.163:0/1065974011 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e800cde0 con 0x7f82e4004810 2023-10-09T06:31:57.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.652+0000 7f82fcb37700 1 -- 172.21.15.163:0/1065974011 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8005b70 con 0x7f82e4004810 2023-10-09T06:31:57.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.652+0000 7f82ffd9d700 1 -- 172.21.15.163:0/1065974011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82e4004810 msgr2=0x7f82e4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:57.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.652+0000 7f82ffd9d700 1 --2- 172.21.15.163:0/1065974011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82e4004810 0x7f82e4004c80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f82e800c3f0 tx=0x7f82f806e470 comp rx=0 tx=0).stop 2023-10-09T06:31:57.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.652+0000 7f82fe33a700 1 -- 172.21.15.163:0/1065974011 reap_dead start 2023-10-09T06:31:57.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.652+0000 7f82ffd9d700 1 -- 172.21.15.163:0/1065974011 shutdown_connections 2023-10-09T06:31:57.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.652+0000 7f82ffd9d700 1 -- 172.21.15.163:0/1065974011 >> 172.21.15.163:0/1065974011 conn(0x7f82f80ce600 msgr2=0x7f82f80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:57.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.652+0000 7f82ffd9d700 1 -- 172.21.15.163:0/1065974011 shutdown_connections 2023-10-09T06:31:57.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.652+0000 7f82ffd9d700 1 -- 172.21.15.163:0/1065974011 wait complete. 2023-10-09T06:31:57.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.653+0000 7f82ffd9d700 1 Processor -- start 2023-10-09T06:31:57.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.653+0000 7f82ffd9d700 1 -- start start 2023-10-09T06:31:57.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.654+0000 7f82ffd9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82e4004810 0x7f82f80a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:57.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.654+0000 7f82ffd9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82f813efd0 0x7f82f80c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:57.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.654+0000 7f82ffd9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82f80b2dc0 0x7f82f813e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:57.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.654+0000 7f82ffd9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f80a5b10 con 0x7f82f80b2dc0 2023-10-09T06:31:57.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.654+0000 7f82ffd9d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f80c74d0 con 0x7f82f813efd0 2023-10-09T06:31:57.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.654+0000 7f82ffd9d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f80c77a0 con 0x7f82e4004810 2023-10-09T06:31:57.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.654+0000 7f82fe33a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82f80b2dc0 0x7f82f813e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:57.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.654+0000 7f82fe33a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82f80b2dc0 0x7f82f813e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39922/0 (socket says 172.21.15.163:39922) 2023-10-09T06:31:57.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.654+0000 7f82fe33a700 1 -- 172.21.15.163:0/2855730416 learned_addr learned my addr 172.21.15.163:0/2855730416 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:57.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.654+0000 7f82fd338700 1 --2- 172.21.15.163:0/2855730416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82f813efd0 0x7f82f80c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:57.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.654+0000 7f82fdb39700 1 --2- 172.21.15.163:0/2855730416 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82e4004810 0x7f82f80a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:57.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.655+0000 7f82fe33a700 1 -- 172.21.15.163:0/2855730416 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82e4004810 msgr2=0x7f82f80a5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:57.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.655+0000 7f82fe33a700 1 --2- 172.21.15.163:0/2855730416 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82e4004810 0x7f82f80a5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:57.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.655+0000 7f82fe33a700 1 -- 172.21.15.163:0/2855730416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82f813efd0 msgr2=0x7f82f80c6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:57.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.655+0000 7f82fe33a700 1 --2- 172.21.15.163:0/2855730416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82f813efd0 0x7f82f80c6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:57.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.655+0000 7f82fe33a700 1 -- 172.21.15.163:0/2855730416 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f80c7a80 con 0x7f82f80b2dc0 2023-10-09T06:31:57.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.655+0000 7f82fe33a700 1 --2- 172.21.15.163:0/2855730416 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82f80b2dc0 0x7f82f813e410 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f82f80c98e0 tx=0x7f82f000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:57.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.655+0000 7f82edffb700 1 -- 172.21.15.163:0/2855730416 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f000e070 con 0x7f82f80b2dc0 2023-10-09T06:31:57.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.655+0000 7f82edffb700 1 -- 172.21.15.163:0/2855730416 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f0011c30 con 0x7f82f80b2dc0 2023-10-09T06:31:57.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.655+0000 7f82edffb700 1 -- 172.21.15.163:0/2855730416 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f0016470 con 0x7f82f80b2dc0 2023-10-09T06:31:57.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.655+0000 7f82ffd9d700 1 -- 172.21.15.163:0/2855730416 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f80a54c0 con 0x7f82f80b2dc0 2023-10-09T06:31:57.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.655+0000 7f82ffd9d700 1 -- 172.21.15.163:0/2855730416 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f80b3210 con 0x7f82f80b2dc0 2023-10-09T06:31:57.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.656+0000 7f82ffd9d700 1 -- 172.21.15.163:0/2855730416 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f804c3c0 con 0x7f82f80b2dc0 2023-10-09T06:31:57.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.656+0000 7f82edffb700 1 -- 172.21.15.163:0/2855730416 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f82f0016860 con 0x7f82f80b2dc0 2023-10-09T06:31:57.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.657+0000 7f82edffb700 1 --2- 172.21.15.163:0/2855730416 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f82dc066f30 0x7f82dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:57.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.657+0000 7f82edffb700 1 -- 172.21.15.163:0/2855730416 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f82f0084ce0 con 0x7f82f80b2dc0 2023-10-09T06:31:57.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.657+0000 7f82fdb39700 1 --2- 172.21.15.163:0/2855730416 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f82dc066f30 0x7f82dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:57.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.658+0000 7f82fdb39700 1 --2- 172.21.15.163:0/2855730416 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f82dc066f30 0x7f82dc0693e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f82f40093f0 tx=0x7f82f4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:57.662 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.661+0000 7f82edffb700 1 -- 172.21.15.163:0/2855730416 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82f0055d30 con 0x7f82f80b2dc0 2023-10-09T06:31:57.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.914+0000 7f82ffd9d700 1 -- 172.21.15.163:0/2855730416 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f82f806c9d0 con 0x7f82f80b2dc0 2023-10-09T06:31:57.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.914+0000 7f82edffb700 1 -- 172.21.15.163:0/2855730416 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f82f00558c0 con 0x7f82f80b2dc0 2023-10-09T06:31:57.914 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:57.914 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:31:57.914 INFO:teuthology.orchestra.run.smithi163.stdout: "auth_dump": [ 2023-10-09T06:31:57.915 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:57.915 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.a", 2023-10-09T06:31:57.915 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNltXtwDxAAlnrWRsfnZDLEmsJVowh3Mg==", 2023-10-09T06:31:57.915 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:57.915 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:31:57.915 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:31:57.915 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:31:57.915 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:57.915 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:57.915 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:57.915 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:57.915 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.b", 2023-10-09T06:31:57.916 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlTdGhGhAAXuldy0HrFd+44sSu/EV3qA==", 2023-10-09T06:31:57.916 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:57.916 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:31:57.916 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:31:57.916 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:31:57.916 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:57.916 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:57.916 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:57.916 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:57.916 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.c", 2023-10-09T06:31:57.916 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlGis7FhAAYG76TiOFX8QkTwjNX2+Ncw==", 2023-10-09T06:31:57.916 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:57.917 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:31:57.917 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:31:57.917 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:31:57.917 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:57.917 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:57.917 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:57.917 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:57.917 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.d", 2023-10-09T06:31:57.917 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlKhRuIRAAfhG5Gpc7klXJloYXepOhsg==", 2023-10-09T06:31:57.917 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:57.917 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:31:57.917 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:31:57.917 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:31:57.918 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:57.918 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:57.918 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:57.918 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:57.918 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.0", 2023-10-09T06:31:57.918 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDcnSNlrQeFOxAA+s6UitrTn5TK1N7kgySoXw==", 2023-10-09T06:31:57.918 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:57.918 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:31:57.918 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:31:57.918 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:57.918 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:57.918 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:57.919 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:57.919 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.1", 2023-10-09T06:31:57.919 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDfnSNlleccORAAnB+lNUFblivdoY44JfELxA==", 2023-10-09T06:31:57.919 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:57.919 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:31:57.919 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:31:57.919 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:57.919 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:57.919 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:57.919 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:57.919 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.2", 2023-10-09T06:31:57.919 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDinSNlFoB4NxAAT0+trRBij5rqgc+F0eWUfQ==", 2023-10-09T06:31:57.919 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:57.920 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:31:57.920 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:31:57.920 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:57.920 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:57.920 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:57.920 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:57.920 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.3", 2023-10-09T06:31:57.920 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDlnSNlRRQ6NhAABoBtrFjSvQujolq0Pxrd1Q==", 2023-10-09T06:31:57.920 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:57.920 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:31:57.920 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:31:57.920 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:57.921 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:57.921 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:57.921 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:57.921 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.4", 2023-10-09T06:31:57.921 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDrnSNl308uOxAA5OJyPLGQmVSJqha5d+nKKQ==", 2023-10-09T06:31:57.921 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:57.921 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:31:57.921 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:31:57.921 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:57.921 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:57.921 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:57.921 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:57.921 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.5", 2023-10-09T06:31:57.922 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDunSNlHnK+OBAAU7l4dCJ82SbGSzBEm8Axzw==", 2023-10-09T06:31:57.922 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:57.922 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:31:57.922 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:31:57.922 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:57.922 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:57.922 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:57.922 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:57.922 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.6", 2023-10-09T06:31:57.922 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDxnSNlwUSLOBAAN5N+F5ho+Co8pBh4Rz+lXQ==", 2023-10-09T06:31:57.922 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:57.922 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:31:57.923 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:31:57.923 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:57.923 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:57.923 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:57.923 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:57.923 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.7", 2023-10-09T06:31:57.923 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQD0nSNl2gxGORAAoE8OLGEifcEx/km8rfSV9g==", 2023-10-09T06:31:57.923 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:57.923 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:31:57.923 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:31:57.923 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:57.923 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:57.923 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:57.924 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:57.924 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.0", 2023-10-09T06:31:57.924 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlnxY5JRAApr6czcpMdQYYtXQ9yg2q6A==", 2023-10-09T06:31:57.924 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:57.924 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:31:57.924 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow r", 2023-10-09T06:31:57.924 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-09T06:31:57.924 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:57.924 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:57.924 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:57.924 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.1", 2023-10-09T06:31:57.924 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlk6ybLRAAuShqTDAtymGdxN8KBdZDDA==", 2023-10-09T06:31:57.925 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:57.925 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:31:57.925 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:31:57.925 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:31:57.925 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:31:57.925 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:57.925 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:57.925 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:57.925 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.2", 2023-10-09T06:31:57.925 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlYyOmKhAAQmpr+mGfJH4GUa28dtlJwA==", 2023-10-09T06:31:57.925 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:57.925 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:31:57.925 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:31:57.926 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:31:57.926 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:31:57.926 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:57.926 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:57.926 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:57.926 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.3", 2023-10-09T06:31:57.926 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlD2H1MhAA4zvaXY/Gd0kndzi+QSwp7w==", 2023-10-09T06:31:57.926 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:57.926 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:31:57.926 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:31:57.926 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:31:57.926 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:31:57.927 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:57.927 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:57.927 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:57.927 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.admin", 2023-10-09T06:31:57.927 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDYnSNlsHH4LxAA85xeFEP1CUwoblaePYk5zA==", 2023-10-09T06:31:57.927 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:57.927 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:31:57.927 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:31:57.927 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:31:57.927 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:57.927 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:57.927 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:57.927 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:57.928 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mgr.x", 2023-10-09T06:31:57.928 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNl2sloDBAA0FoWN9iovCl6783o3/btiA==", 2023-10-09T06:31:57.928 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:57.928 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:31:57.928 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile mgr", 2023-10-09T06:31:57.928 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:57.928 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:57.928 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:31:57.928 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:57.928 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mgr.y", 2023-10-09T06:31:57.928 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlcFWVCBAAR2dKPg01ciTiNpSX0JIKFg==", 2023-10-09T06:31:57.928 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:31:57.929 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:31:57.929 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile mgr", 2023-10-09T06:31:57.929 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:31:57.929 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:57.929 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:57.929 INFO:teuthology.orchestra.run.smithi163.stdout: ] 2023-10-09T06:31:57.929 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:31:57.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.916+0000 7f82ffd9d700 1 -- 172.21.15.163:0/2855730416 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f82dc066f30 msgr2=0x7f82dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:57.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.916+0000 7f82ffd9d700 1 --2- 172.21.15.163:0/2855730416 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f82dc066f30 0x7f82dc0693e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f82f40093f0 tx=0x7f82f4008040 comp rx=0 tx=0).stop 2023-10-09T06:31:57.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.916+0000 7f82ffd9d700 1 -- 172.21.15.163:0/2855730416 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82f80b2dc0 msgr2=0x7f82f813e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:57.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.916+0000 7f82ffd9d700 1 --2- 172.21.15.163:0/2855730416 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82f80b2dc0 0x7f82f813e410 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f82f80c98e0 tx=0x7f82f000b040 comp rx=0 tx=0).stop 2023-10-09T06:31:57.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.916+0000 7f82ffd9d700 1 -- 172.21.15.163:0/2855730416 shutdown_connections 2023-10-09T06:31:57.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.916+0000 7f82ffd9d700 1 --2- 172.21.15.163:0/2855730416 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f82dc066f30 0x7f82dc0693e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:57.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.916+0000 7f82ffd9d700 1 --2- 172.21.15.163:0/2855730416 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82f80b2dc0 0x7f82f813e410 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:57.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.916+0000 7f82ffd9d700 1 --2- 172.21.15.163:0/2855730416 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82e4004810 0x7f82f80a5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:57.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.916+0000 7f82ffd9d700 1 --2- 172.21.15.163:0/2855730416 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82f813efd0 0x7f82f80c6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:57.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.916+0000 7f82ffd9d700 1 -- 172.21.15.163:0/2855730416 >> 172.21.15.163:0/2855730416 conn(0x7f82f80ce600 msgr2=0x7f82f80c3810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:57.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.916+0000 7f82ffd9d700 1 -- 172.21.15.163:0/2855730416 shutdown_connections 2023-10-09T06:31:57.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:57.916+0000 7f82ffd9d700 1 -- 172.21.15.163:0/2855730416 wait complete. 2023-10-09T06:31:57.932 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:31:58.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.047+0000 7f1b35ff1700 1 Processor -- start 2023-10-09T06:31:58.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.047+0000 7f1b35ff1700 1 -- start start 2023-10-09T06:31:58.049 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.048+0000 7f1b35ff1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1b3006bcd0 0x7f1b300caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:58.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.048+0000 7f1b35ff1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b3006c0a0 0x7f1b3015a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:58.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.048+0000 7f1b35ff1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b3015d7e0 con 0x7f1b3015df50 2023-10-09T06:31:58.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.048+0000 7f1b35ff1700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b3015dac0 con 0x7f1b3006bcd0 2023-10-09T06:31:58.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.048+0000 7f1b35ff1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3015aa30 con 0x7f1b3006c0a0 2023-10-09T06:31:58.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.048+0000 7f1b2f7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1b3006bcd0 0x7f1b300caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:58.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.048+0000 7f1b2f7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1b3006bcd0 0x7f1b300caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39938/0 (socket says 172.21.15.163:39938) 2023-10-09T06:31:58.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.048+0000 7f1b2f7fe700 1 -- 172.21.15.163:0/1315370191 learned_addr learned my addr 172.21.15.163:0/1315370191 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:58.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.049+0000 7f1b2effd700 1 --2- 172.21.15.163:0/1315370191 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b3006c0a0 0x7f1b3015a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:58.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.049+0000 7f1b2f7fe700 1 -- 172.21.15.163:0/1315370191 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b3006c0a0 msgr2=0x7f1b3015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:58.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.049+0000 7f1b2f7fe700 1 --2- 172.21.15.163:0/1315370191 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b3006c0a0 0x7f1b3015a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:58.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.049+0000 7f1b2f7fe700 1 -- 172.21.15.163:0/1315370191 >> v1:172.21.15.181:6789/0 conn(0x7f1b3015df50 legacy=0x7f1b300c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:58.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.049+0000 7f1b2f7fe700 1 -- 172.21.15.163:0/1315370191 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b3015ad10 con 0x7f1b3006bcd0 2023-10-09T06:31:58.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.049+0000 7f1b2effd700 1 --2- 172.21.15.163:0/1315370191 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b3006c0a0 0x7f1b3015a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:58.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.049+0000 7f1b2f7fe700 1 --2- 172.21.15.163:0/1315370191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b3006bcd0 0x7f1b300caa90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1b20009f20 tx=0x7f1b20009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea37339973603941 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:58.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.049+0000 7f1b2e7fc700 1 -- 172.21.15.163:0/1315370191 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2000e070 con 0x7f1b3006bcd0 2023-10-09T06:31:58.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.049+0000 7f1b2e7fc700 1 -- 172.21.15.163:0/1315370191 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b2000ba40 con 0x7f1b3006bcd0 2023-10-09T06:31:58.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.049+0000 7f1b2e7fc700 1 -- 172.21.15.163:0/1315370191 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b200092e0 con 0x7f1b3006bcd0 2023-10-09T06:31:58.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.049+0000 7f1b35ff1700 1 -- 172.21.15.163:0/1315370191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b3006bcd0 msgr2=0x7f1b300caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:58.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.049+0000 7f1b35ff1700 1 --2- 172.21.15.163:0/1315370191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b3006bcd0 0x7f1b300caa90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1b20009f20 tx=0x7f1b20009840 comp rx=0 tx=0).stop 2023-10-09T06:31:58.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.050+0000 7f1b35ff1700 1 -- 172.21.15.163:0/1315370191 shutdown_connections 2023-10-09T06:31:58.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.050+0000 7f1b35ff1700 1 --2- 172.21.15.163:0/1315370191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b3006bcd0 0x7f1b300caa90 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:58.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.050+0000 7f1b35ff1700 1 --2- 172.21.15.163:0/1315370191 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b3006c0a0 0x7f1b3015a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:58.052 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.050+0000 7f1b35ff1700 1 -- 172.21.15.163:0/1315370191 >> 172.21.15.163:0/1315370191 conn(0x7f1b300ce600 msgr2=0x7f1b300cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:58.052 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.050+0000 7f1b35ff1700 1 -- 172.21.15.163:0/1315370191 shutdown_connections 2023-10-09T06:31:58.052 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.050+0000 7f1b35ff1700 1 -- 172.21.15.163:0/1315370191 wait complete. 2023-10-09T06:31:58.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.051+0000 7f1b35ff1700 1 Processor -- start 2023-10-09T06:31:58.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.051+0000 7f1b35ff1700 1 -- start start 2023-10-09T06:31:58.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.052+0000 7f1b35ff1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b3006bcd0 0x7f1b30145310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:58.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.052+0000 7f1b35ff1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b3006c0a0 0x7f1b3016a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:58.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.052+0000 7f1b35ff1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b3015df50 0x7f1b30069e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:58.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.052+0000 7f1b35ff1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b30145850 con 0x7f1b3006c0a0 2023-10-09T06:31:58.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.052+0000 7f1b35ff1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b300b13a0 con 0x7f1b3015df50 2023-10-09T06:31:58.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.052+0000 7f1b35ff1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b300b1960 con 0x7f1b3006bcd0 2023-10-09T06:31:58.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.052+0000 7f1b2effd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b3006c0a0 0x7f1b3016a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:58.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.052+0000 7f1b2effd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b3006c0a0 0x7f1b3016a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39944/0 (socket says 172.21.15.163:39944) 2023-10-09T06:31:58.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.052+0000 7f1b2effd700 1 -- 172.21.15.163:0/902467497 learned_addr learned my addr 172.21.15.163:0/902467497 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:58.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.052+0000 7f1b2f7fe700 1 --2- 172.21.15.163:0/902467497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b3006bcd0 0x7f1b30145310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:58.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.052+0000 7f1b2ffff700 1 --2- 172.21.15.163:0/902467497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b3015df50 0x7f1b30069e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:58.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.052+0000 7f1b2effd700 1 -- 172.21.15.163:0/902467497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b3006bcd0 msgr2=0x7f1b30145310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:58.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.052+0000 7f1b2effd700 1 --2- 172.21.15.163:0/902467497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b3006bcd0 0x7f1b30145310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:58.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.052+0000 7f1b2effd700 1 -- 172.21.15.163:0/902467497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b3015df50 msgr2=0x7f1b30069e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:58.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.052+0000 7f1b2effd700 1 --2- 172.21.15.163:0/902467497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b3015df50 0x7f1b30069e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:58.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.052+0000 7f1b2effd700 1 -- 172.21.15.163:0/902467497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b20009ae0 con 0x7f1b3006c0a0 2023-10-09T06:31:58.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.052+0000 7f1b2effd700 1 --2- 172.21.15.163:0/902467497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b3006c0a0 0x7f1b3016a060 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1b2400c860 tx=0x7f1b24008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:58.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.052+0000 7f1b177fe700 1 -- 172.21.15.163:0/902467497 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b24011600 con 0x7f1b3006c0a0 2023-10-09T06:31:58.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.052+0000 7f1b177fe700 1 -- 172.21.15.163:0/902467497 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b24011760 con 0x7f1b3006c0a0 2023-10-09T06:31:58.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.052+0000 7f1b35ff1700 1 -- 172.21.15.163:0/902467497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b3016ab70 con 0x7f1b3006c0a0 2023-10-09T06:31:58.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.052+0000 7f1b177fe700 1 -- 172.21.15.163:0/902467497 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b24011b20 con 0x7f1b3006c0a0 2023-10-09T06:31:58.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.052+0000 7f1b35ff1700 1 -- 172.21.15.163:0/902467497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b300b15f0 con 0x7f1b3006c0a0 2023-10-09T06:31:58.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.053+0000 7f1b35ff1700 1 -- 172.21.15.163:0/902467497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b3004c3c0 con 0x7f1b3006c0a0 2023-10-09T06:31:58.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.054+0000 7f1b177fe700 1 -- 172.21.15.163:0/902467497 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1b240193b0 con 0x7f1b3006c0a0 2023-10-09T06:31:58.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.054+0000 7f1b177fe700 1 --2- 172.21.15.163:0/902467497 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1b1c066f30 0x7f1b1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:58.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.054+0000 7f1b177fe700 1 -- 172.21.15.163:0/902467497 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f1b240840c0 con 0x7f1b3006c0a0 2023-10-09T06:31:58.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.054+0000 7f1b2f7fe700 1 --2- 172.21.15.163:0/902467497 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1b1c066f30 0x7f1b1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:58.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.055+0000 7f1b2f7fe700 1 --2- 172.21.15.163:0/902467497 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1b1c066f30 0x7f1b1c0693e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1b20009ab0 tx=0x7f1b20017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:58.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.058+0000 7f1b177fe700 1 -- 172.21.15.163:0/902467497 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b24051680 con 0x7f1b3006c0a0 2023-10-09T06:31:58.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.243+0000 7f1b35ff1700 1 -- 172.21.15.163:0/902467497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b3006c9d0 con 0x7f1b3006c0a0 2023-10-09T06:31:58.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.244+0000 7f1b177fe700 1 -- 172.21.15.163:0/902467497 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+12182 (secure 0 0 0) 0x7f1b24054ca0 con 0x7f1b3006c0a0 2023-10-09T06:31:58.244 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:58.244 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":47,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:31:57.506545+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:31:58.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.246+0000 7f1b35ff1700 1 -- 172.21.15.163:0/902467497 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1b1c066f30 msgr2=0x7f1b1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:58.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.246+0000 7f1b35ff1700 1 --2- 172.21.15.163:0/902467497 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1b1c066f30 0x7f1b1c0693e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1b20009ab0 tx=0x7f1b20017000 comp rx=0 tx=0).stop 2023-10-09T06:31:58.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.246+0000 7f1b35ff1700 1 -- 172.21.15.163:0/902467497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b3006c0a0 msgr2=0x7f1b3016a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:58.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.246+0000 7f1b35ff1700 1 --2- 172.21.15.163:0/902467497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b3006c0a0 0x7f1b3016a060 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1b2400c860 tx=0x7f1b24008040 comp rx=0 tx=0).stop 2023-10-09T06:31:58.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.246+0000 7f1b35ff1700 1 -- 172.21.15.163:0/902467497 shutdown_connections 2023-10-09T06:31:58.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.246+0000 7f1b35ff1700 1 --2- 172.21.15.163:0/902467497 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1b1c066f30 0x7f1b1c0693e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:58.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.246+0000 7f1b35ff1700 1 --2- 172.21.15.163:0/902467497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b3015df50 0x7f1b30069e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:58.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.246+0000 7f1b35ff1700 1 --2- 172.21.15.163:0/902467497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b3006bcd0 0x7f1b30145310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:58.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.246+0000 7f1b35ff1700 1 --2- 172.21.15.163:0/902467497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b3006c0a0 0x7f1b3016a060 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:58.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.246+0000 7f1b35ff1700 1 -- 172.21.15.163:0/902467497 >> 172.21.15.163:0/902467497 conn(0x7f1b300ce600 msgr2=0x7f1b30157640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:58.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.246+0000 7f1b35ff1700 1 -- 172.21.15.163:0/902467497 shutdown_connections 2023-10-09T06:31:58.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.246+0000 7f1b35ff1700 1 -- 172.21.15.163:0/902467497 wait complete. 2023-10-09T06:31:58.258 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-09T06:31:58.258 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:31:58.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.375+0000 7f24362f4700 1 Processor -- start 2023-10-09T06:31:58.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.375+0000 7f24362f4700 1 -- start start 2023-10-09T06:31:58.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.376+0000 7f24362f4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2430159890 0x7f243015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:58.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.376+0000 7f24362f4700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2430069560 0x7f24300699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:58.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.376+0000 7f24362f4700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2430069ee0 con 0x7f2430159c60 2023-10-09T06:31:58.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.376+0000 7f24362f4700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f243015a160 con 0x7f2430069560 2023-10-09T06:31:58.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.376+0000 7f24362f4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24300d13a0 con 0x7f2430159890 2023-10-09T06:31:58.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.376+0000 7f2434891700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2430069560 0x7f24300699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:58.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.376+0000 7f242f7fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f2430159c60 0x7f243015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:44800/0 (socket says 172.21.15.163:44800) 2023-10-09T06:31:58.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.376+0000 7f2434891700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2430069560 0x7f24300699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39960/0 (socket says 172.21.15.163:39960) 2023-10-09T06:31:58.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.376+0000 7f242f7fe700 1 -- 172.21.15.163:0/834476432 learned_addr learned my addr 172.21.15.163:0/834476432 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:58.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.376+0000 7f242ffff700 1 --2- 172.21.15.163:0/834476432 >> v2:172.21.15.181:3300/0 conn(0x7f2430159890 0x7f243015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:58.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.376+0000 7f2434891700 1 -- 172.21.15.163:0/834476432 >> v2:172.21.15.181:3300/0 conn(0x7f2430159890 msgr2=0x7f243015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:58.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.376+0000 7f2434891700 1 --2- 172.21.15.163:0/834476432 >> v2:172.21.15.181:3300/0 conn(0x7f2430159890 0x7f243015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:58.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.376+0000 7f2434891700 1 -- 172.21.15.163:0/834476432 >> v1:172.21.15.163:6789/0 conn(0x7f2430159c60 legacy=0x7f243015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:58.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.376+0000 7f242effd700 1 -- 172.21.15.163:0/834476432 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1128832379 0 0) 0x7f2430069ee0 con 0x7f2430159c60 2023-10-09T06:31:58.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.376+0000 7f2434891700 1 -- 172.21.15.163:0/834476432 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243015e140 con 0x7f2430069560 2023-10-09T06:31:58.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.377+0000 7f2434891700 1 --2- 172.21.15.163:0/834476432 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2430069560 0x7f24300699a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f24300d1090 tx=0x7f2428009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e9f06475a5a2f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:58.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.377+0000 7f242effd700 1 -- 172.21.15.163:0/834476432 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242800e070 con 0x7f2430069560 2023-10-09T06:31:58.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.377+0000 7f242effd700 1 -- 172.21.15.163:0/834476432 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f242800bb60 con 0x7f2430069560 2023-10-09T06:31:58.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.377+0000 7f242effd700 1 -- 172.21.15.163:0/834476432 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2428012430 con 0x7f2430069560 2023-10-09T06:31:58.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.377+0000 7f24362f4700 1 -- 172.21.15.163:0/834476432 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2430069560 msgr2=0x7f24300699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:58.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.377+0000 7f24362f4700 1 --2- 172.21.15.163:0/834476432 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2430069560 0x7f24300699a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f24300d1090 tx=0x7f2428009780 comp rx=0 tx=0).stop 2023-10-09T06:31:58.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.377+0000 7f24362f4700 1 -- 172.21.15.163:0/834476432 shutdown_connections 2023-10-09T06:31:58.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.377+0000 7f24362f4700 1 --2- 172.21.15.163:0/834476432 >> v2:172.21.15.181:3300/0 conn(0x7f2430159890 0x7f243015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:58.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.377+0000 7f24362f4700 1 --2- 172.21.15.163:0/834476432 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2430069560 0x7f24300699a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:58.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.377+0000 7f24362f4700 1 -- 172.21.15.163:0/834476432 >> 172.21.15.163:0/834476432 conn(0x7f2430068e20 msgr2=0x7f24300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:58.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.377+0000 7f24362f4700 1 -- 172.21.15.163:0/834476432 shutdown_connections 2023-10-09T06:31:58.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.377+0000 7f24362f4700 1 -- 172.21.15.163:0/834476432 wait complete. 2023-10-09T06:31:58.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.378+0000 7f24362f4700 1 Processor -- start 2023-10-09T06:31:58.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.379+0000 7f24362f4700 1 -- start start 2023-10-09T06:31:58.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.379+0000 7f24362f4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2430069560 0x7f24301472f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:58.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.379+0000 7f24362f4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2430159890 0x7f24300c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:58.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.379+0000 7f24362f4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2430159c60 0x7f24300cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:58.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.379+0000 7f24362f4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24300cd3f0 con 0x7f2430159c60 2023-10-09T06:31:58.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.379+0000 7f24362f4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24300cf090 con 0x7f2430069560 2023-10-09T06:31:58.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.379+0000 7f24362f4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24300ce7c0 con 0x7f2430159890 2023-10-09T06:31:58.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.379+0000 7f2434891700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2430159c60 0x7f24300cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:58.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.379+0000 7f2434891700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2430159c60 0x7f24300cadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39972/0 (socket says 172.21.15.163:39972) 2023-10-09T06:31:58.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.379+0000 7f2434891700 1 -- 172.21.15.163:0/1475422682 learned_addr learned my addr 172.21.15.163:0/1475422682 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:58.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.379+0000 7f242ffff700 1 --2- 172.21.15.163:0/1475422682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2430069560 0x7f24301472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:58.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.379+0000 7f242f7fe700 1 --2- 172.21.15.163:0/1475422682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2430159890 0x7f24300c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:58.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.380+0000 7f2434891700 1 -- 172.21.15.163:0/1475422682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2430159890 msgr2=0x7f24300c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:58.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.380+0000 7f2434891700 1 --2- 172.21.15.163:0/1475422682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2430159890 0x7f24300c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:58.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.380+0000 7f2434891700 1 -- 172.21.15.163:0/1475422682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2430069560 msgr2=0x7f24301472f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:58.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.380+0000 7f2434891700 1 --2- 172.21.15.163:0/1475422682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2430069560 0x7f24301472f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:58.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.380+0000 7f2434891700 1 -- 172.21.15.163:0/1475422682 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2428009a20 con 0x7f2430159c60 2023-10-09T06:31:58.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.380+0000 7f2434891700 1 --2- 172.21.15.163:0/1475422682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2430159c60 0x7f24300cadf0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2430070e20 tx=0x7f2428019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:58.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.380+0000 7f2417fff700 1 -- 172.21.15.163:0/1475422682 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2428027470 con 0x7f2430159c60 2023-10-09T06:31:58.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.380+0000 7f2417fff700 1 -- 172.21.15.163:0/1475422682 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24280275d0 con 0x7f2430159c60 2023-10-09T06:31:58.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.380+0000 7f2417fff700 1 -- 172.21.15.163:0/1475422682 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2428027990 con 0x7f2430159c60 2023-10-09T06:31:58.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.380+0000 7f24362f4700 1 -- 172.21.15.163:0/1475422682 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24301527b0 con 0x7f2430159c60 2023-10-09T06:31:58.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.380+0000 7f24362f4700 1 -- 172.21.15.163:0/1475422682 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24300ceac0 con 0x7f2430159c60 2023-10-09T06:31:58.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.381+0000 7f24362f4700 1 -- 172.21.15.163:0/1475422682 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f243004c3c0 con 0x7f2430159c60 2023-10-09T06:31:58.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.381+0000 7f2417fff700 1 -- 172.21.15.163:0/1475422682 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2428024070 con 0x7f2430159c60 2023-10-09T06:31:58.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.382+0000 7f2417fff700 1 --2- 172.21.15.163:0/1475422682 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f241c066f30 0x7f241c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:58.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.382+0000 7f2417fff700 1 -- 172.21.15.163:0/1475422682 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f2428086bc0 con 0x7f2430159c60 2023-10-09T06:31:58.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.382+0000 7f242ffff700 1 --2- 172.21.15.163:0/1475422682 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f241c066f30 0x7f241c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:58.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.382+0000 7f242ffff700 1 --2- 172.21.15.163:0/1475422682 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f241c066f30 0x7f241c0693e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2420009db0 tx=0x7f24200091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:58.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.386+0000 7f2417fff700 1 -- 172.21.15.163:0/1475422682 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2428054180 con 0x7f2430159c60 2023-10-09T06:31:58.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.570+0000 7f24362f4700 1 -- 172.21.15.163:0/1475422682 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f24300cf1d0 con 0x7f2430159c60 2023-10-09T06:31:58.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.571+0000 7f2417fff700 1 -- 172.21.15.163:0/1475422682 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f242802c020 con 0x7f2430159c60 2023-10-09T06:31:58.571 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:31:58.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.572+0000 7f24362f4700 1 -- 172.21.15.163:0/1475422682 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f241c066f30 msgr2=0x7f241c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:58.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.572+0000 7f24362f4700 1 --2- 172.21.15.163:0/1475422682 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f241c066f30 0x7f241c0693e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2420009db0 tx=0x7f24200091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:58.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.572+0000 7f24362f4700 1 -- 172.21.15.163:0/1475422682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2430159c60 msgr2=0x7f24300cadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:58.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.572+0000 7f24362f4700 1 --2- 172.21.15.163:0/1475422682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2430159c60 0x7f24300cadf0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2430070e20 tx=0x7f2428019000 comp rx=0 tx=0).stop 2023-10-09T06:31:58.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.572+0000 7f24362f4700 1 -- 172.21.15.163:0/1475422682 shutdown_connections 2023-10-09T06:31:58.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.572+0000 7f24362f4700 1 --2- 172.21.15.163:0/1475422682 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f241c066f30 0x7f241c0693e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:58.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.572+0000 7f24362f4700 1 --2- 172.21.15.163:0/1475422682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2430159c60 0x7f24300cadf0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:58.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.572+0000 7f24362f4700 1 --2- 172.21.15.163:0/1475422682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2430069560 0x7f24301472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:58.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.573+0000 7f24362f4700 1 --2- 172.21.15.163:0/1475422682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2430159890 0x7f24300c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:58.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.573+0000 7f24362f4700 1 -- 172.21.15.163:0/1475422682 >> 172.21.15.163:0/1475422682 conn(0x7f2430068e20 msgr2=0x7f2430152d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:58.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.573+0000 7f24362f4700 1 -- 172.21.15.163:0/1475422682 shutdown_connections 2023-10-09T06:31:58.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.573+0000 7f24362f4700 1 -- 172.21.15.163:0/1475422682 wait complete. 2023-10-09T06:31:58.584 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-09T06:31:58.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.699+0000 7ff56e595700 1 Processor -- start 2023-10-09T06:31:58.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.700+0000 7ff56e595700 1 -- start start 2023-10-09T06:31:58.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.700+0000 7ff56e595700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff56806bcd0 0x7ff568140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:58.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.700+0000 7ff56e595700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff5680c9510 0x7ff568141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:58.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.700+0000 7ff56e595700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff56815c2a0 con 0x7ff56806c0a0 2023-10-09T06:31:58.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.700+0000 7ff56e595700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff56815b690 con 0x7ff5680c9510 2023-10-09T06:31:58.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.700+0000 7ff56e595700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff56815b970 con 0x7ff56806bcd0 2023-10-09T06:31:58.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.700+0000 7ff56cb32700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff5680c9510 0x7ff568141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:58.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.701+0000 7ff5677fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff56806c0a0 0x7ff5680c8e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:50796/0 (socket says 172.21.15.163:50796) 2023-10-09T06:31:58.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.701+0000 7ff5677fe700 1 -- 172.21.15.163:0/557877374 learned_addr learned my addr 172.21.15.163:0/557877374 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:58.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.701+0000 7ff567fff700 1 --2- 172.21.15.163:0/557877374 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff56806bcd0 0x7ff568140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:58.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.701+0000 7ff56cb32700 1 -- 172.21.15.163:0/557877374 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff56806bcd0 msgr2=0x7ff568140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:31:58.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.701+0000 7ff56cb32700 1 --2- 172.21.15.163:0/557877374 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff56806bcd0 0x7ff568140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:58.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.701+0000 7ff56cb32700 1 -- 172.21.15.163:0/557877374 >> v1:172.21.15.163:6789/0 conn(0x7ff56806c0a0 legacy=0x7ff5680c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:58.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.701+0000 7ff56cb32700 1 -- 172.21.15.163:0/557877374 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff56815bc50 con 0x7ff5680c9510 2023-10-09T06:31:58.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.701+0000 7ff566ffd700 1 -- 172.21.15.163:0/557877374 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1519653342 0 0) 0x7ff56815c2a0 con 0x7ff56806c0a0 2023-10-09T06:31:58.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.701+0000 7ff567fff700 1 --2- 172.21.15.163:0/557877374 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff56806bcd0 0x7ff568140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:58.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.701+0000 7ff56cb32700 1 --2- 172.21.15.163:0/557877374 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5680c9510 0x7ff568141330 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff560009e60 tx=0x7ff560009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62efd719d467e799 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:58.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.701+0000 7ff566ffd700 1 -- 172.21.15.163:0/557877374 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56000e070 con 0x7ff5680c9510 2023-10-09T06:31:58.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.701+0000 7ff566ffd700 1 -- 172.21.15.163:0/557877374 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff56000bc30 con 0x7ff5680c9510 2023-10-09T06:31:58.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.701+0000 7ff566ffd700 1 -- 172.21.15.163:0/557877374 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff560012430 con 0x7ff5680c9510 2023-10-09T06:31:58.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.702+0000 7ff56e595700 1 -- 172.21.15.163:0/557877374 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5680c9510 msgr2=0x7ff568141330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:58.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.702+0000 7ff56e595700 1 --2- 172.21.15.163:0/557877374 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5680c9510 0x7ff568141330 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff560009e60 tx=0x7ff560009780 comp rx=0 tx=0).stop 2023-10-09T06:31:58.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.702+0000 7ff56e595700 1 -- 172.21.15.163:0/557877374 shutdown_connections 2023-10-09T06:31:58.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.702+0000 7ff56e595700 1 --2- 172.21.15.163:0/557877374 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5680c9510 0x7ff568141330 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:58.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.702+0000 7ff56e595700 1 --2- 172.21.15.163:0/557877374 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff56806bcd0 0x7ff568140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:58.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.702+0000 7ff56e595700 1 -- 172.21.15.163:0/557877374 >> 172.21.15.163:0/557877374 conn(0x7ff5680ce600 msgr2=0x7ff5680cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:58.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.702+0000 7ff56e595700 1 -- 172.21.15.163:0/557877374 shutdown_connections 2023-10-09T06:31:58.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.702+0000 7ff56e595700 1 -- 172.21.15.163:0/557877374 wait complete. 2023-10-09T06:31:58.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.703+0000 7ff56e595700 1 Processor -- start 2023-10-09T06:31:58.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.703+0000 7ff56e595700 1 -- start start 2023-10-09T06:31:58.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.704+0000 7ff56e595700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff56806bcd0 0x7ff56816b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:58.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.704+0000 7ff56e595700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff56806c0a0 0x7ff568158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:58.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.704+0000 7ff56e595700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5680c9510 0x7ff5680b3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:58.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.704+0000 7ff56e595700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56816bec0 con 0x7ff56806bcd0 2023-10-09T06:31:58.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.704+0000 7ff56e595700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5680c5850 con 0x7ff56806c0a0 2023-10-09T06:31:58.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.704+0000 7ff56e595700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5680c5af0 con 0x7ff5680c9510 2023-10-09T06:31:58.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.704+0000 7ff567fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff56806bcd0 0x7ff56816b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:58.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.704+0000 7ff567fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff56806bcd0 0x7ff56816b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58642/0 (socket says 172.21.15.163:58642) 2023-10-09T06:31:58.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.704+0000 7ff567fff700 1 -- 172.21.15.163:0/747570295 learned_addr learned my addr 172.21.15.163:0/747570295 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:58.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.704+0000 7ff567fff700 1 -- 172.21.15.163:0/747570295 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5680c9510 msgr2=0x7ff5680b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:58.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.704+0000 7ff567fff700 1 --2- 172.21.15.163:0/747570295 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5680c9510 0x7ff5680b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:58.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.704+0000 7ff567fff700 1 -- 172.21.15.163:0/747570295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff56806c0a0 msgr2=0x7ff568158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:58.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.704+0000 7ff567fff700 1 --2- 172.21.15.163:0/747570295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff56806c0a0 0x7ff568158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:58.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.704+0000 7ff567fff700 1 -- 172.21.15.163:0/747570295 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff560009a20 con 0x7ff56806bcd0 2023-10-09T06:31:58.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.704+0000 7ff567fff700 1 --2- 172.21.15.163:0/747570295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff56806bcd0 0x7ff56816b980 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff55800c850 tx=0x7ff558008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:58.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.704+0000 7ff54ffff700 1 -- 172.21.15.163:0/747570295 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558011600 con 0x7ff56806bcd0 2023-10-09T06:31:58.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.704+0000 7ff54ffff700 1 -- 172.21.15.163:0/747570295 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff558011760 con 0x7ff56806bcd0 2023-10-09T06:31:58.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.704+0000 7ff56e595700 1 -- 172.21.15.163:0/747570295 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5680c5dd0 con 0x7ff56806bcd0 2023-10-09T06:31:58.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.704+0000 7ff54ffff700 1 -- 172.21.15.163:0/747570295 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558011b00 con 0x7ff56806bcd0 2023-10-09T06:31:58.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.704+0000 7ff56e595700 1 -- 172.21.15.163:0/747570295 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5680c6390 con 0x7ff56806bcd0 2023-10-09T06:31:58.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.705+0000 7ff56e595700 1 -- 172.21.15.163:0/747570295 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff56804c3c0 con 0x7ff56806bcd0 2023-10-09T06:31:58.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.706+0000 7ff54ffff700 1 -- 172.21.15.163:0/747570295 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff5580073b0 con 0x7ff56806bcd0 2023-10-09T06:31:58.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.706+0000 7ff54ffff700 1 --2- 172.21.15.163:0/747570295 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff554066f30 0x7ff5540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:58.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.706+0000 7ff54ffff700 1 -- 172.21.15.163:0/747570295 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff558084160 con 0x7ff56806bcd0 2023-10-09T06:31:58.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.706+0000 7ff5677fe700 1 --2- 172.21.15.163:0/747570295 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff554066f30 0x7ff5540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:58.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.707+0000 7ff5677fe700 1 --2- 172.21.15.163:0/747570295 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff554066f30 0x7ff5540693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0011e0 tx=0x7ff55c0079d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:58.710 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.710+0000 7ff54ffff700 1 -- 172.21.15.163:0/747570295 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff5580551b0 con 0x7ff56806bcd0 2023-10-09T06:31:58.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.896+0000 7ff56e595700 1 -- 172.21.15.163:0/747570295 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff56806c9d0 con 0x7ff56806bcd0 2023-10-09T06:31:58.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.897+0000 7ff54ffff700 1 -- 172.21.15.163:0/747570295 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v47) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff558054d40 con 0x7ff56806bcd0 2023-10-09T06:31:58.897 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:31:58.897 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:31:58.897 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:31:58.897 INFO:teuthology.orchestra.run.smithi163.stdout: "poolnum": 1, 2023-10-09T06:31:58.897 INFO:teuthology.orchestra.run.smithi163.stdout: "poolname": ".mgr" 2023-10-09T06:31:58.897 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:31:58.898 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:31:58.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.899+0000 7ff56e595700 1 -- 172.21.15.163:0/747570295 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff554066f30 msgr2=0x7ff5540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:58.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.899+0000 7ff56e595700 1 --2- 172.21.15.163:0/747570295 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff554066f30 0x7ff5540693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0011e0 tx=0x7ff55c0079d0 comp rx=0 tx=0).stop 2023-10-09T06:31:58.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.899+0000 7ff56e595700 1 -- 172.21.15.163:0/747570295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff56806bcd0 msgr2=0x7ff56816b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:58.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.899+0000 7ff56e595700 1 --2- 172.21.15.163:0/747570295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff56806bcd0 0x7ff56816b980 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff55800c850 tx=0x7ff558008040 comp rx=0 tx=0).stop 2023-10-09T06:31:58.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.899+0000 7ff56e595700 1 -- 172.21.15.163:0/747570295 shutdown_connections 2023-10-09T06:31:58.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.899+0000 7ff56e595700 1 --2- 172.21.15.163:0/747570295 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff554066f30 0x7ff5540693e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:58.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.899+0000 7ff56e595700 1 --2- 172.21.15.163:0/747570295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff56806bcd0 0x7ff56816b980 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:58.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.899+0000 7ff56e595700 1 --2- 172.21.15.163:0/747570295 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5680c9510 0x7ff5680b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:58.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.899+0000 7ff56e595700 1 --2- 172.21.15.163:0/747570295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff56806c0a0 0x7ff568158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:58.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.899+0000 7ff56e595700 1 -- 172.21.15.163:0/747570295 >> 172.21.15.163:0/747570295 conn(0x7ff5680ce600 msgr2=0x7ff5680cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:58.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.899+0000 7ff56e595700 1 -- 172.21.15.163:0/747570295 shutdown_connections 2023-10-09T06:31:58.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:58.899+0000 7ff56e595700 1 -- 172.21.15.163:0/747570295 wait complete. 2023-10-09T06:31:58.911 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-09T06:31:58.911 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-09T06:31:59.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.029+0000 7f56e2c38700 1 Processor -- start 2023-10-09T06:31:59.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.029+0000 7f56e2c38700 1 -- start start 2023-10-09T06:31:59.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.030+0000 7f56e2c38700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f56dc0b55a0 0x7f56dc164400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:59.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.030+0000 7f56e2c38700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56dc0cfa60 con 0x7f56dc0cd910 2023-10-09T06:31:59.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.030+0000 7f56e2c38700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56dc0b5b00 con 0x7f56dc159ec0 2023-10-09T06:31:59.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.030+0000 7f56e2c38700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56dc0b5da0 con 0x7f56dc0b55a0 2023-10-09T06:31:59.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.030+0000 7f56dbfff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f56dc0cd910 0x7f56dc0ce340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:50804/0 (socket says 172.21.15.163:50804) 2023-10-09T06:31:59.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.030+0000 7f56dbfff700 1 -- 172.21.15.163:0/275263701 learned_addr learned my addr 172.21.15.163:0/275263701 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:59.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.030+0000 7f56e11d5700 1 --2- 172.21.15.163:0/275263701 >> v2:172.21.15.181:3300/0 conn(0x7f56dc0b55a0 0x7f56dc164400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:59.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.030+0000 7f56db7fe700 1 -- 172.21.15.163:0/275263701 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4078806319 0 0) 0x7f56dc0cfa60 con 0x7f56dc0cd910 2023-10-09T06:31:59.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.030+0000 7f56db7fe700 1 -- 172.21.15.163:0/275263701 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56cc003570 con 0x7f56dc0cd910 2023-10-09T06:31:59.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.030+0000 7f56e11d5700 1 -- 172.21.15.163:0/275263701 >> v1:172.21.15.181:6789/0 conn(0x7f56dc159ec0 legacy=0x7f56dc0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:59.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.030+0000 7f56e11d5700 1 -- 172.21.15.163:0/275263701 >> v1:172.21.15.163:6789/0 conn(0x7f56dc0cd910 legacy=0x7f56dc0ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:59.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.030+0000 7f56e11d5700 1 -- 172.21.15.163:0/275263701 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56dc0b6080 con 0x7f56dc0b55a0 2023-10-09T06:31:59.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.031+0000 7f56e11d5700 1 --2- 172.21.15.163:0/275263701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56dc0b55a0 0x7f56dc164400 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f56d4009e90 tx=0x7f56d4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f03503749024a11 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:59.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.031+0000 7f56db7fe700 1 -- 172.21.15.163:0/275263701 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d400e070 con 0x7f56dc0b55a0 2023-10-09T06:31:59.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.031+0000 7f56db7fe700 1 -- 172.21.15.163:0/275263701 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d400bc40 con 0x7f56dc0b55a0 2023-10-09T06:31:59.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.031+0000 7f56db7fe700 1 -- 172.21.15.163:0/275263701 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d4012430 con 0x7f56dc0b55a0 2023-10-09T06:31:59.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.031+0000 7f56e2c38700 1 -- 172.21.15.163:0/275263701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56dc0b55a0 msgr2=0x7f56dc164400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:59.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.031+0000 7f56e2c38700 1 --2- 172.21.15.163:0/275263701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56dc0b55a0 0x7f56dc164400 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f56d4009e90 tx=0x7f56d4009780 comp rx=0 tx=0).stop 2023-10-09T06:31:59.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.031+0000 7f56e2c38700 1 -- 172.21.15.163:0/275263701 shutdown_connections 2023-10-09T06:31:59.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.031+0000 7f56e2c38700 1 --2- 172.21.15.163:0/275263701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56dc0b55a0 0x7f56dc164400 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:59.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.031+0000 7f56e2c38700 1 -- 172.21.15.163:0/275263701 >> 172.21.15.163:0/275263701 conn(0x7f56dc0999f0 msgr2=0x7f56dc0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:59.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.031+0000 7f56e2c38700 1 -- 172.21.15.163:0/275263701 shutdown_connections 2023-10-09T06:31:59.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.031+0000 7f56e2c38700 1 -- 172.21.15.163:0/275263701 wait complete. 2023-10-09T06:31:59.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.033+0000 7f56e2c38700 1 Processor -- start 2023-10-09T06:31:59.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.033+0000 7f56e2c38700 1 -- start start 2023-10-09T06:31:59.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.033+0000 7f56e2c38700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56dc0b55a0 0x7f56dc09ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:59.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.033+0000 7f56e2c38700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56dc159ec0 0x7f56dc156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:59.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.033+0000 7f56e2c38700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56dc09c590 0x7f56dc09ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:59.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.033+0000 7f56e2c38700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc157660 con 0x7f56dc09c590 2023-10-09T06:31:59.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.033+0000 7f56e2c38700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc157120 con 0x7f56dc159ec0 2023-10-09T06:31:59.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.033+0000 7f56e2c38700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc068880 con 0x7f56dc0b55a0 2023-10-09T06:31:59.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.034+0000 7f56e11d5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56dc09c590 0x7f56dc09ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:59.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.034+0000 7f56e11d5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56dc09c590 0x7f56dc09ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58650/0 (socket says 172.21.15.163:58650) 2023-10-09T06:31:59.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.034+0000 7f56e11d5700 1 -- 172.21.15.163:0/2606931320 learned_addr learned my addr 172.21.15.163:0/2606931320 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:59.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.034+0000 7f56e09d4700 1 --2- 172.21.15.163:0/2606931320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56dc0b55a0 0x7f56dc09ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:59.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.034+0000 7f56dbfff700 1 --2- 172.21.15.163:0/2606931320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56dc159ec0 0x7f56dc156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:59.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.034+0000 7f56e11d5700 1 -- 172.21.15.163:0/2606931320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56dc0b55a0 msgr2=0x7f56dc09ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:59.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.034+0000 7f56e11d5700 1 --2- 172.21.15.163:0/2606931320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56dc0b55a0 0x7f56dc09ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:59.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.034+0000 7f56e11d5700 1 -- 172.21.15.163:0/2606931320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56dc159ec0 msgr2=0x7f56dc156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:59.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.034+0000 7f56e11d5700 1 --2- 172.21.15.163:0/2606931320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56dc159ec0 0x7f56dc156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:59.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.034+0000 7f56e11d5700 1 -- 172.21.15.163:0/2606931320 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d4009a20 con 0x7f56dc09c590 2023-10-09T06:31:59.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.034+0000 7f56dbfff700 1 --2- 172.21.15.163:0/2606931320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56dc159ec0 0x7f56dc156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:59.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.034+0000 7f56e11d5700 1 --2- 172.21.15.163:0/2606931320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56dc09c590 0x7f56dc09ca30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f56d400b6c0 tx=0x7f56d4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:59.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.034+0000 7f56d8ff9700 1 -- 172.21.15.163:0/2606931320 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d4012e60 con 0x7f56dc09c590 2023-10-09T06:31:59.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.034+0000 7f56d8ff9700 1 -- 172.21.15.163:0/2606931320 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d401b630 con 0x7f56dc09c590 2023-10-09T06:31:59.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.034+0000 7f56d8ff9700 1 -- 172.21.15.163:0/2606931320 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d401b9f0 con 0x7f56dc09c590 2023-10-09T06:31:59.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.034+0000 7f56e2c38700 1 -- 172.21.15.163:0/2606931320 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56dc09b500 con 0x7f56dc09c590 2023-10-09T06:31:59.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.034+0000 7f56e2c38700 1 -- 172.21.15.163:0/2606931320 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56dc068510 con 0x7f56dc09c590 2023-10-09T06:31:59.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.035+0000 7f56e2c38700 1 -- 172.21.15.163:0/2606931320 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56dc09ba00 con 0x7f56dc09c590 2023-10-09T06:31:59.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.035+0000 7f56d8ff9700 1 -- 172.21.15.163:0/2606931320 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f56d401c400 con 0x7f56dc09c590 2023-10-09T06:31:59.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.036+0000 7f56d8ff9700 1 --2- 172.21.15.163:0/2606931320 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f56c0066f30 0x7f56c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:59.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.036+0000 7f56d8ff9700 1 -- 172.21.15.163:0/2606931320 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f56d40860f0 con 0x7f56dc09c590 2023-10-09T06:31:59.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.036+0000 7f56e09d4700 1 --2- 172.21.15.163:0/2606931320 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f56c0066f30 0x7f56c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:59.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.037+0000 7f56e09d4700 1 --2- 172.21.15.163:0/2606931320 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f56c0066f30 0x7f56c00693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f56dc069fd0 tx=0x7f56d0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:59.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.040+0000 7f56d8ff9700 1 -- 172.21.15.163:0/2606931320 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f56d4057140 con 0x7f56dc09c590 2023-10-09T06:31:59.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.222+0000 7f56e2c38700 1 -- 172.21.15.163:0/2606931320 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f56dc063c90 con 0x7f56dc09c590 2023-10-09T06:31:59.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.531+0000 7f56d8ff9700 1 -- 172.21.15.163:0/2606931320 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v48) v1 ==== 138+0+0 (secure 0 0 0) 0x7f56d4056cd0 con 0x7f56dc09c590 2023-10-09T06:31:59.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.534+0000 7f56e2c38700 1 -- 172.21.15.163:0/2606931320 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f56c0066f30 msgr2=0x7f56c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:59.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.534+0000 7f56e2c38700 1 --2- 172.21.15.163:0/2606931320 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f56c0066f30 0x7f56c00693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f56dc069fd0 tx=0x7f56d0009040 comp rx=0 tx=0).stop 2023-10-09T06:31:59.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.534+0000 7f56e2c38700 1 -- 172.21.15.163:0/2606931320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56dc09c590 msgr2=0x7f56dc09ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:59.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.534+0000 7f56e2c38700 1 --2- 172.21.15.163:0/2606931320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56dc09c590 0x7f56dc09ca30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f56d400b6c0 tx=0x7f56d4019000 comp rx=0 tx=0).stop 2023-10-09T06:31:59.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.535+0000 7f56e2c38700 1 -- 172.21.15.163:0/2606931320 shutdown_connections 2023-10-09T06:31:59.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.535+0000 7f56e2c38700 1 --2- 172.21.15.163:0/2606931320 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f56c0066f30 0x7f56c00693e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:59.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.535+0000 7f56e2c38700 1 --2- 172.21.15.163:0/2606931320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56dc09c590 0x7f56dc09ca30 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:59.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.535+0000 7f56e2c38700 1 --2- 172.21.15.163:0/2606931320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56dc0b55a0 0x7f56dc09ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:59.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.535+0000 7f56e2c38700 1 --2- 172.21.15.163:0/2606931320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56dc159ec0 0x7f56dc156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:59.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.535+0000 7f56e2c38700 1 -- 172.21.15.163:0/2606931320 >> 172.21.15.163:0/2606931320 conn(0x7f56dc0999f0 msgr2=0x7f56dc1590d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:59.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.535+0000 7f56e2c38700 1 -- 172.21.15.163:0/2606931320 shutdown_connections 2023-10-09T06:31:59.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.535+0000 7f56e2c38700 1 -- 172.21.15.163:0/2606931320 wait complete. 2023-10-09T06:31:59.544 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_metadata' created 2023-10-09T06:31:59.548 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-09T06:31:59.671 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.671+0000 7fe314686700 1 Processor -- start 2023-10-09T06:31:59.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.671+0000 7fe314686700 1 -- start start 2023-10-09T06:31:59.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.672+0000 7fe314686700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe30c155ce0 0x7fe30c0cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:59.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.672+0000 7fe314686700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe30c0cc1f0 con 0x7fe30c069700 2023-10-09T06:31:59.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.672+0000 7fe314686700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe30c068830 con 0x7fe30c1560b0 2023-10-09T06:31:59.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.672+0000 7fe314686700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe30c0cbc30 con 0x7fe30c155ce0 2023-10-09T06:31:59.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.673+0000 7fe312422700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe30c155ce0 0x7fe30c0cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:59.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.673+0000 7fe311c21700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fe30c069700 0x7fe30c068cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:50820/0 (socket says 172.21.15.163:50820) 2023-10-09T06:31:59.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.673+0000 7fe312422700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe30c155ce0 0x7fe30c0cb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58654/0 (socket says 172.21.15.163:58654) 2023-10-09T06:31:59.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.673+0000 7fe311c21700 1 -- 172.21.15.163:0/2972110993 learned_addr learned my addr 172.21.15.163:0/2972110993 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:31:59.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.673+0000 7fe312422700 1 -- 172.21.15.163:0/2972110993 >> v1:172.21.15.181:6789/0 conn(0x7fe30c1560b0 legacy=0x7fe30c06a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:59.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.673+0000 7fe312422700 1 -- 172.21.15.163:0/2972110993 >> v1:172.21.15.163:6789/0 conn(0x7fe30c069700 legacy=0x7fe30c068cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:59.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.673+0000 7fe311420700 1 -- 172.21.15.163:0/2972110993 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2179010074 0 0) 0x7fe30c0cc1f0 con 0x7fe30c069700 2023-10-09T06:31:59.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.673+0000 7fe312422700 1 -- 172.21.15.163:0/2972110993 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30c0cbf10 con 0x7fe30c155ce0 2023-10-09T06:31:59.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.673+0000 7fe312422700 1 --2- 172.21.15.163:0/2972110993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe30c155ce0 0x7fe30c0cb5c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe308009910 tx=0x7fe3080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0e47d5c8e5b48e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:59.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.673+0000 7fe311420700 1 -- 172.21.15.163:0/2972110993 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30800b820 con 0x7fe30c155ce0 2023-10-09T06:31:59.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.673+0000 7fe311420700 1 -- 172.21.15.163:0/2972110993 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30800b980 con 0x7fe30c155ce0 2023-10-09T06:31:59.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.673+0000 7fe311420700 1 -- 172.21.15.163:0/2972110993 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30800bd20 con 0x7fe30c155ce0 2023-10-09T06:31:59.677 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.674+0000 7fe314686700 1 -- 172.21.15.163:0/2972110993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe30c155ce0 msgr2=0x7fe30c0cb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:59.677 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.674+0000 7fe314686700 1 --2- 172.21.15.163:0/2972110993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe30c155ce0 0x7fe30c0cb5c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe308009910 tx=0x7fe3080091a0 comp rx=0 tx=0).stop 2023-10-09T06:31:59.677 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.674+0000 7fe314686700 1 -- 172.21.15.163:0/2972110993 shutdown_connections 2023-10-09T06:31:59.677 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.674+0000 7fe314686700 1 --2- 172.21.15.163:0/2972110993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe30c155ce0 0x7fe30c0cb5c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:59.677 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.674+0000 7fe314686700 1 -- 172.21.15.163:0/2972110993 >> 172.21.15.163:0/2972110993 conn(0x7fe30c0c9870 msgr2=0x7fe30c0c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:31:59.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.674+0000 7fe314686700 1 -- 172.21.15.163:0/2972110993 shutdown_connections 2023-10-09T06:31:59.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.674+0000 7fe314686700 1 -- 172.21.15.163:0/2972110993 wait complete. 2023-10-09T06:31:59.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.675+0000 7fe314686700 1 Processor -- start 2023-10-09T06:31:59.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.675+0000 7fe314686700 1 -- start start 2023-10-09T06:31:59.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.676+0000 7fe314686700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe30c069700 0x7fe30c152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:59.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.676+0000 7fe314686700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe30c155ce0 0x7fe30c16aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:59.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.676+0000 7fe314686700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe30c1560b0 0x7fe30c0b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:59.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.676+0000 7fe314686700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c0c5600 con 0x7fe30c1560b0 2023-10-09T06:31:59.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.676+0000 7fe314686700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c157090 con 0x7fe30c069700 2023-10-09T06:31:59.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.676+0000 7fe314686700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c153230 con 0x7fe30c155ce0 2023-10-09T06:31:59.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.676+0000 7fe312c23700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe30c1560b0 0x7fe30c0b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:59.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.676+0000 7fe312c23700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe30c1560b0 0x7fe30c0b1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58664/0 (socket says 172.21.15.163:58664) 2023-10-09T06:31:59.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.676+0000 7fe312c23700 1 -- 172.21.15.163:0/329652916 learned_addr learned my addr 172.21.15.163:0/329652916 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:31:59.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.676+0000 7fe312422700 1 --2- 172.21.15.163:0/329652916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe30c069700 0x7fe30c152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:59.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.676+0000 7fe311c21700 1 --2- 172.21.15.163:0/329652916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe30c155ce0 0x7fe30c16aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:59.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.676+0000 7fe312c23700 1 -- 172.21.15.163:0/329652916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe30c155ce0 msgr2=0x7fe30c16aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:59.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.676+0000 7fe312c23700 1 --2- 172.21.15.163:0/329652916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe30c155ce0 0x7fe30c16aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:59.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.676+0000 7fe312c23700 1 -- 172.21.15.163:0/329652916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe30c069700 msgr2=0x7fe30c152cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:31:59.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.676+0000 7fe312c23700 1 --2- 172.21.15.163:0/329652916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe30c069700 0x7fe30c152cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:31:59.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.676+0000 7fe312c23700 1 -- 172.21.15.163:0/329652916 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe308009420 con 0x7fe30c1560b0 2023-10-09T06:31:59.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.676+0000 7fe311c21700 1 --2- 172.21.15.163:0/329652916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe30c155ce0 0x7fe30c16aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:31:59.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.676+0000 7fe312422700 1 --2- 172.21.15.163:0/329652916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe30c069700 0x7fe30c152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:31:59.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.676+0000 7fe312c23700 1 --2- 172.21.15.163:0/329652916 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe30c1560b0 0x7fe30c0b1f60 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe30400c870 tx=0x7fe304001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:59.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.676+0000 7fe3027fc700 1 -- 172.21.15.163:0/329652916 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe304012630 con 0x7fe30c1560b0 2023-10-09T06:31:59.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.676+0000 7fe3027fc700 1 -- 172.21.15.163:0/329652916 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3040127b0 con 0x7fe30c1560b0 2023-10-09T06:31:59.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.676+0000 7fe314686700 1 -- 172.21.15.163:0/329652916 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe30c157330 con 0x7fe30c1560b0 2023-10-09T06:31:59.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.676+0000 7fe3027fc700 1 -- 172.21.15.163:0/329652916 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe304012b70 con 0x7fe30c1560b0 2023-10-09T06:31:59.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.676+0000 7fe314686700 1 -- 172.21.15.163:0/329652916 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe30c0c50c0 con 0x7fe30c1560b0 2023-10-09T06:31:59.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.677+0000 7fe314686700 1 -- 172.21.15.163:0/329652916 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe30c0718b0 con 0x7fe30c1560b0 2023-10-09T06:31:59.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.678+0000 7fe3027fc700 1 -- 172.21.15.163:0/329652916 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe304003030 con 0x7fe30c1560b0 2023-10-09T06:31:59.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.678+0000 7fe3027fc700 1 --2- 172.21.15.163:0/329652916 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe2f0066f30 0x7fe2f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:31:59.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.678+0000 7fe3027fc700 1 -- 172.21.15.163:0/329652916 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fe3040842d0 con 0x7fe30c1560b0 2023-10-09T06:31:59.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.679+0000 7fe312422700 1 --2- 172.21.15.163:0/329652916 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe2f0066f30 0x7fe2f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:31:59.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.679+0000 7fe312422700 1 --2- 172.21.15.163:0/329652916 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe2f0066f30 0x7fe2f00693e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe3080093f0 tx=0x7fe308017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:31:59.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.683+0000 7fe3027fc700 1 -- 172.21.15.163:0/329652916 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe304051720 con 0x7fe30c1560b0 2023-10-09T06:31:59.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:31:59.865+0000 7fe314686700 1 -- 172.21.15.163:0/329652916 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe30c0c53e0 con 0x7fe30c1560b0 2023-10-09T06:32:00.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.535+0000 7fe3027fc700 1 -- 172.21.15.163:0/329652916 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v49) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe304054d40 con 0x7fe30c1560b0 2023-10-09T06:32:00.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.547+0000 7fe314686700 1 -- 172.21.15.163:0/329652916 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe2f0066f30 msgr2=0x7fe2f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:00.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.547+0000 7fe314686700 1 --2- 172.21.15.163:0/329652916 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe2f0066f30 0x7fe2f00693e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe3080093f0 tx=0x7fe308017000 comp rx=0 tx=0).stop 2023-10-09T06:32:00.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.547+0000 7fe314686700 1 -- 172.21.15.163:0/329652916 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe30c1560b0 msgr2=0x7fe30c0b1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:00.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.547+0000 7fe314686700 1 --2- 172.21.15.163:0/329652916 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe30c1560b0 0x7fe30c0b1f60 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe30400c870 tx=0x7fe304001670 comp rx=0 tx=0).stop 2023-10-09T06:32:00.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.550+0000 7fe314686700 1 -- 172.21.15.163:0/329652916 shutdown_connections 2023-10-09T06:32:00.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.550+0000 7fe314686700 1 --2- 172.21.15.163:0/329652916 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe2f0066f30 0x7fe2f00693e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:00.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.550+0000 7fe314686700 1 --2- 172.21.15.163:0/329652916 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe30c1560b0 0x7fe30c0b1f60 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:00.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.550+0000 7fe314686700 1 --2- 172.21.15.163:0/329652916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe30c069700 0x7fe30c152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:00.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.550+0000 7fe314686700 1 --2- 172.21.15.163:0/329652916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe30c155ce0 0x7fe30c16aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:00.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.550+0000 7fe314686700 1 -- 172.21.15.163:0/329652916 >> 172.21.15.163:0/329652916 conn(0x7fe30c0c9870 msgr2=0x7fe30c06a220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:00.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.551+0000 7fe314686700 1 -- 172.21.15.163:0/329652916 shutdown_connections 2023-10-09T06:32:00.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.551+0000 7fe314686700 1 -- 172.21.15.163:0/329652916 wait complete. 2023-10-09T06:32:00.552 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_data' created 2023-10-09T06:32:00.564 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-09T06:32:00.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.684+0000 7fcdfc3e3700 1 Processor -- start 2023-10-09T06:32:00.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.684+0000 7fcdfc3e3700 1 -- start start 2023-10-09T06:32:00.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.685+0000 7fcdfc3e3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcdf4159b10 0x7fcdf40cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:00.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.685+0000 7fcdfc3e3700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcdf415d730 0x7fcdf415dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:00.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.685+0000 7fcdfc3e3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf40c74b0 con 0x7fcdf4159ee0 2023-10-09T06:32:00.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.685+0000 7fcdfc3e3700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf40c8700 con 0x7fcdf415d730 2023-10-09T06:32:00.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.685+0000 7fcdfc3e3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf40c9070 con 0x7fcdf4159b10 2023-10-09T06:32:00.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.685+0000 7fcdf997e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcdf415d730 0x7fcdf415dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:00.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.685+0000 7fcdf997e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcdf415d730 0x7fcdf415dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58680/0 (socket says 172.21.15.163:58680) 2023-10-09T06:32:00.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.685+0000 7fcdf997e700 1 -- 172.21.15.163:0/1999337413 learned_addr learned my addr 172.21.15.163:0/1999337413 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:00.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.685+0000 7fcdfa17f700 1 --2- 172.21.15.163:0/1999337413 >> v2:172.21.15.181:3300/0 conn(0x7fcdf4159b10 0x7fcdf40cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:00.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.685+0000 7fcdf997e700 1 -- 172.21.15.163:0/1999337413 >> v2:172.21.15.181:3300/0 conn(0x7fcdf4159b10 msgr2=0x7fcdf40cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:00.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.685+0000 7fcdf997e700 1 --2- 172.21.15.163:0/1999337413 >> v2:172.21.15.181:3300/0 conn(0x7fcdf4159b10 0x7fcdf40cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:00.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.685+0000 7fcdf997e700 1 -- 172.21.15.163:0/1999337413 >> v1:172.21.15.181:6789/0 conn(0x7fcdf4159ee0 legacy=0x7fcdf415a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:00.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.685+0000 7fcdf997e700 1 -- 172.21.15.163:0/1999337413 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf415e0e0 con 0x7fcdf415d730 2023-10-09T06:32:00.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.685+0000 7fcdf997e700 1 --2- 172.21.15.163:0/1999337413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdf415d730 0x7fcdf415dba0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4063e90 tx=0x7fcde4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534923ad806774ae server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:00.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.686+0000 7fcdf917d700 1 -- 172.21.15.163:0/1999337413 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde400b8f0 con 0x7fcdf415d730 2023-10-09T06:32:00.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.686+0000 7fcdf917d700 1 -- 172.21.15.163:0/1999337413 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde400ba50 con 0x7fcdf415d730 2023-10-09T06:32:00.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.686+0000 7fcdf917d700 1 -- 172.21.15.163:0/1999337413 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde400bdf0 con 0x7fcdf415d730 2023-10-09T06:32:00.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.686+0000 7fcdfc3e3700 1 -- 172.21.15.163:0/1999337413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdf415d730 msgr2=0x7fcdf415dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:00.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.686+0000 7fcdfc3e3700 1 --2- 172.21.15.163:0/1999337413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdf415d730 0x7fcdf415dba0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4063e90 tx=0x7fcde4009250 comp rx=0 tx=0).stop 2023-10-09T06:32:00.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.686+0000 7fcdfc3e3700 1 -- 172.21.15.163:0/1999337413 shutdown_connections 2023-10-09T06:32:00.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.686+0000 7fcdfc3e3700 1 --2- 172.21.15.163:0/1999337413 >> v2:172.21.15.181:3300/0 conn(0x7fcdf4159b10 0x7fcdf40cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:00.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.686+0000 7fcdfc3e3700 1 --2- 172.21.15.163:0/1999337413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdf415d730 0x7fcdf415dba0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:00.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.686+0000 7fcdfc3e3700 1 -- 172.21.15.163:0/1999337413 >> 172.21.15.163:0/1999337413 conn(0x7fcdf40c9870 msgr2=0x7fcdf40c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:00.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.686+0000 7fcdfc3e3700 1 -- 172.21.15.163:0/1999337413 shutdown_connections 2023-10-09T06:32:00.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.686+0000 7fcdfc3e3700 1 -- 172.21.15.163:0/1999337413 wait complete. 2023-10-09T06:32:00.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.687+0000 7fcdfc3e3700 1 Processor -- start 2023-10-09T06:32:00.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.687+0000 7fcdfc3e3700 1 -- start start 2023-10-09T06:32:00.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.688+0000 7fcdfc3e3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdf4159b10 0x7fcdf41458b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:00.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.688+0000 7fcdfc3e3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcdf4159ee0 0x7fcdf415b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:00.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.688+0000 7fcdfc3e3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdf415d730 0x7fcdf406a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:00.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.688+0000 7fcdfc3e3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf415a300 con 0x7fcdf4159b10 2023-10-09T06:32:00.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.688+0000 7fcdfc3e3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf415ae40 con 0x7fcdf415d730 2023-10-09T06:32:00.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.688+0000 7fcdfc3e3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf4145df0 con 0x7fcdf4159ee0 2023-10-09T06:32:00.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.688+0000 7fcdfa17f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdf4159b10 0x7fcdf41458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:00.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.688+0000 7fcdfa17f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdf4159b10 0x7fcdf41458b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58694/0 (socket says 172.21.15.163:58694) 2023-10-09T06:32:00.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.688+0000 7fcdfa17f700 1 -- 172.21.15.163:0/1749749645 learned_addr learned my addr 172.21.15.163:0/1749749645 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:00.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.688+0000 7fcdf997e700 1 --2- 172.21.15.163:0/1749749645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcdf4159ee0 0x7fcdf415b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:00.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.688+0000 7fcdfa980700 1 --2- 172.21.15.163:0/1749749645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdf415d730 0x7fcdf406a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:00.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.688+0000 7fcdfa17f700 1 -- 172.21.15.163:0/1749749645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcdf4159ee0 msgr2=0x7fcdf415b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:00.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.688+0000 7fcdfa17f700 1 --2- 172.21.15.163:0/1749749645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcdf4159ee0 0x7fcdf415b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:00.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.688+0000 7fcdfa17f700 1 -- 172.21.15.163:0/1749749645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdf415d730 msgr2=0x7fcdf406a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:00.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.688+0000 7fcdfa17f700 1 --2- 172.21.15.163:0/1749749645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdf415d730 0x7fcdf406a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:00.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.688+0000 7fcdfa17f700 1 -- 172.21.15.163:0/1749749645 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde40094f0 con 0x7fcdf4159b10 2023-10-09T06:32:00.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.688+0000 7fcdfa17f700 1 --2- 172.21.15.163:0/1749749645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdf4159b10 0x7fcdf41458b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00e190 tx=0x7fcdec009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:00.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.689+0000 7fcdea7fc700 1 -- 172.21.15.163:0/1749749645 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec00ee30 con 0x7fcdf4159b10 2023-10-09T06:32:00.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.689+0000 7fcdea7fc700 1 -- 172.21.15.163:0/1749749645 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdec014420 con 0x7fcdf4159b10 2023-10-09T06:32:00.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.689+0000 7fcdfc3e3700 1 -- 172.21.15.163:0/1749749645 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf415a510 con 0x7fcdf4159b10 2023-10-09T06:32:00.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.689+0000 7fcdea7fc700 1 -- 172.21.15.163:0/1749749645 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec0147e0 con 0x7fcdf4159b10 2023-10-09T06:32:00.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.689+0000 7fcdfc3e3700 1 -- 172.21.15.163:0/1749749645 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf415aad0 con 0x7fcdf4159b10 2023-10-09T06:32:00.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.690+0000 7fcdfc3e3700 1 -- 172.21.15.163:0/1749749645 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf404c3c0 con 0x7fcdf4159b10 2023-10-09T06:32:00.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.690+0000 7fcdea7fc700 1 -- 172.21.15.163:0/1749749645 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcdec014b90 con 0x7fcdf4159b10 2023-10-09T06:32:00.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.690+0000 7fcdea7fc700 1 --2- 172.21.15.163:0/1749749645 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcdd8066f30 0x7fcdd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:00.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.690+0000 7fcdea7fc700 1 -- 172.21.15.163:0/1749749645 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fcdec084ce0 con 0x7fcdf4159b10 2023-10-09T06:32:00.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.691+0000 7fcdf997e700 1 --2- 172.21.15.163:0/1749749645 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcdd8066f30 0x7fcdd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:00.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.691+0000 7fcdf997e700 1 --2- 172.21.15.163:0/1749749645 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcdd8066f30 0x7fcdd80693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4063e90 tx=0x7fcde4017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:00.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.695+0000 7fcdea7fc700 1 -- 172.21.15.163:0/1749749645 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcdec051fb0 con 0x7fcdf4159b10 2023-10-09T06:32:00.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:00.923+0000 7fcdfc3e3700 1 -- 172.21.15.163:0/1749749645 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fcdf41563d0 con 0x7fcdf4159b10 2023-10-09T06:32:01.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.550+0000 7fcdea7fc700 1 -- 172.21.15.163:0/1749749645 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v27) v1 ==== 172+0+0 (secure 0 0 0) 0x7fcdec009a60 con 0x7fcdf4159b10 2023-10-09T06:32:01.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.556+0000 7fcdfc3e3700 1 -- 172.21.15.163:0/1749749645 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcdd8066f30 msgr2=0x7fcdd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:01.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.556+0000 7fcdfc3e3700 1 --2- 172.21.15.163:0/1749749645 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcdd8066f30 0x7fcdd80693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4063e90 tx=0x7fcde4017000 comp rx=0 tx=0).stop 2023-10-09T06:32:01.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.556+0000 7fcdfc3e3700 1 -- 172.21.15.163:0/1749749645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdf4159b10 msgr2=0x7fcdf41458b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:01.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.556+0000 7fcdfc3e3700 1 --2- 172.21.15.163:0/1749749645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdf4159b10 0x7fcdf41458b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00e190 tx=0x7fcdec009920 comp rx=0 tx=0).stop 2023-10-09T06:32:01.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.557+0000 7fcdfc3e3700 1 -- 172.21.15.163:0/1749749645 shutdown_connections 2023-10-09T06:32:01.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.557+0000 7fcdfc3e3700 1 --2- 172.21.15.163:0/1749749645 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcdd8066f30 0x7fcdd80693e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:01.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.557+0000 7fcdfc3e3700 1 --2- 172.21.15.163:0/1749749645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdf4159b10 0x7fcdf41458b0 secure :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00e190 tx=0x7fcdec009920 comp rx=0 tx=0).stop 2023-10-09T06:32:01.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.557+0000 7fcdfc3e3700 1 --2- 172.21.15.163:0/1749749645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdf415d730 0x7fcdf406a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:01.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.557+0000 7fcdfc3e3700 1 --2- 172.21.15.163:0/1749749645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcdf4159ee0 0x7fcdf415b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:01.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.557+0000 7fcdfc3e3700 1 -- 172.21.15.163:0/1749749645 >> 172.21.15.163:0/1749749645 conn(0x7fcdf40c9870 msgr2=0x7fcdf4156800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:01.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.558+0000 7fcdfc3e3700 1 -- 172.21.15.163:0/1749749645 shutdown_connections 2023-10-09T06:32:01.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.558+0000 7fcdfc3e3700 1 -- 172.21.15.163:0/1749749645 wait complete. 2023-10-09T06:32:01.560 INFO:teuthology.orchestra.run.smithi163.stderr:new fs with metadata pool 8 and data pool 9 2023-10-09T06:32:01.573 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:32:01.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.688+0000 7ff3295e4700 1 Processor -- start 2023-10-09T06:32:01.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.688+0000 7ff3295e4700 1 -- start start 2023-10-09T06:32:01.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.689+0000 7ff3295e4700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff324159c60 0x7ff32415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:01.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.689+0000 7ff3295e4700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff32415a160 con 0x7ff32415db10 2023-10-09T06:32:01.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.689+0000 7ff3295e4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3240d13a0 con 0x7ff324159890 2023-10-09T06:32:01.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.689+0000 7ff3295e4700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff32415df50 con 0x7ff324159c60 2023-10-09T06:32:01.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.689+0000 7ff31a7fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff324159c60 0x7ff32415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:01.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.689+0000 7ff3237fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff32415db10 0x7ff324069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:50828/0 (socket says 172.21.15.163:50828) 2023-10-09T06:32:01.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.689+0000 7ff3237fe700 1 -- 172.21.15.163:0/3612505900 learned_addr learned my addr 172.21.15.163:0/3612505900 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:01.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.689+0000 7ff31a7fc700 1 --2- 172.21.15.163:0/3612505900 >> v2:172.21.15.163:3300/0 conn(0x7ff324159c60 0x7ff32415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58708/0 (socket says 172.21.15.163:58708) 2023-10-09T06:32:01.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.690+0000 7ff31a7fc700 1 -- 172.21.15.163:0/3612505900 >> v1:172.21.15.181:6789/0 conn(0x7ff324159890 legacy=0x7ff32415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:01.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.690+0000 7ff31a7fc700 1 -- 172.21.15.163:0/3612505900 >> v1:172.21.15.163:6789/0 conn(0x7ff32415db10 legacy=0x7ff324069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:01.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.690+0000 7ff31a7fc700 1 -- 172.21.15.163:0/3612505900 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32415e140 con 0x7ff324159c60 2023-10-09T06:32:01.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.690+0000 7ff3227fc700 1 -- 172.21.15.163:0/3612505900 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 24728845 0 0) 0x7ff32415a160 con 0x7ff32415db10 2023-10-09T06:32:01.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.690+0000 7ff31a7fc700 1 --2- 172.21.15.163:0/3612505900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff324159c60 0x7ff32415d5d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff30c0098e0 tx=0x7ff30c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec0e0df56d5295e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:01.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.690+0000 7ff3227fc700 1 -- 172.21.15.163:0/3612505900 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c009e80 con 0x7ff324159c60 2023-10-09T06:32:01.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.690+0000 7ff3227fc700 1 -- 172.21.15.163:0/3612505900 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30c00b860 con 0x7ff324159c60 2023-10-09T06:32:01.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.690+0000 7ff3227fc700 1 -- 172.21.15.163:0/3612505900 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c00bbe0 con 0x7ff324159c60 2023-10-09T06:32:01.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.690+0000 7ff3295e4700 1 -- 172.21.15.163:0/3612505900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff324159c60 msgr2=0x7ff32415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:01.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.690+0000 7ff3295e4700 1 --2- 172.21.15.163:0/3612505900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff324159c60 0x7ff32415d5d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff30c0098e0 tx=0x7ff30c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:01.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.691+0000 7ff3295e4700 1 -- 172.21.15.163:0/3612505900 shutdown_connections 2023-10-09T06:32:01.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.691+0000 7ff3295e4700 1 --2- 172.21.15.163:0/3612505900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff324159c60 0x7ff32415d5d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:01.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.691+0000 7ff3295e4700 1 -- 172.21.15.163:0/3612505900 >> 172.21.15.163:0/3612505900 conn(0x7ff324068e20 msgr2=0x7ff3240d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:01.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.691+0000 7ff3295e4700 1 -- 172.21.15.163:0/3612505900 shutdown_connections 2023-10-09T06:32:01.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.691+0000 7ff3295e4700 1 -- 172.21.15.163:0/3612505900 wait complete. 2023-10-09T06:32:01.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.692+0000 7ff3295e4700 1 Processor -- start 2023-10-09T06:32:01.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.692+0000 7ff3295e4700 1 -- start start 2023-10-09T06:32:01.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.692+0000 7ff3295e4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff324159890 0x7ff32416ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:01.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.692+0000 7ff3295e4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff324159c60 0x7ff32416b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:01.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.692+0000 7ff3295e4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff32415db10 0x7ff324169ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:01.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.692+0000 7ff3295e4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32416aea0 con 0x7ff324159c60 2023-10-09T06:32:01.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.692+0000 7ff3295e4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32416a8e0 con 0x7ff324159890 2023-10-09T06:32:01.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.692+0000 7ff3295e4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32416abc0 con 0x7ff32415db10 2023-10-09T06:32:01.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.693+0000 7ff31a7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff324159c60 0x7ff32416b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:01.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.693+0000 7ff31a7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff324159c60 0x7ff32416b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58710/0 (socket says 172.21.15.163:58710) 2023-10-09T06:32:01.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.693+0000 7ff31a7fc700 1 -- 172.21.15.163:0/2602151526 learned_addr learned my addr 172.21.15.163:0/2602151526 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:01.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.693+0000 7ff322ffd700 1 --2- 172.21.15.163:0/2602151526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff324159890 0x7ff32416ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:01.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.693+0000 7ff3237fe700 1 --2- 172.21.15.163:0/2602151526 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff32415db10 0x7ff324169ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:01.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.693+0000 7ff31a7fc700 1 -- 172.21.15.163:0/2602151526 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff32415db10 msgr2=0x7ff324169ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:01.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.693+0000 7ff31a7fc700 1 --2- 172.21.15.163:0/2602151526 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff32415db10 0x7ff324169ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:01.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.693+0000 7ff31a7fc700 1 -- 172.21.15.163:0/2602151526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff324159890 msgr2=0x7ff32416ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:01.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.693+0000 7ff31a7fc700 1 --2- 172.21.15.163:0/2602151526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff324159890 0x7ff32416ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:01.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.693+0000 7ff31a7fc700 1 -- 172.21.15.163:0/2602151526 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30c009420 con 0x7ff324159c60 2023-10-09T06:32:01.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.693+0000 7ff3237fe700 1 --2- 172.21.15.163:0/2602151526 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff32415db10 0x7ff324169ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:01.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.693+0000 7ff31a7fc700 1 --2- 172.21.15.163:0/2602151526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff324159c60 0x7ff32416b080 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff324070e20 tx=0x7ff30c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:01.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.693+0000 7ff31b7fe700 1 -- 172.21.15.163:0/2602151526 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c01f070 con 0x7ff324159c60 2023-10-09T06:32:01.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.693+0000 7ff3295e4700 1 -- 172.21.15.163:0/2602151526 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3240b36c0 con 0x7ff324159c60 2023-10-09T06:32:01.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.693+0000 7ff31b7fe700 1 -- 172.21.15.163:0/2602151526 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30c000c90 con 0x7ff324159c60 2023-10-09T06:32:01.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.693+0000 7ff31b7fe700 1 -- 172.21.15.163:0/2602151526 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c025420 con 0x7ff324159c60 2023-10-09T06:32:01.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.693+0000 7ff3295e4700 1 -- 172.21.15.163:0/2602151526 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3240b5330 con 0x7ff324159c60 2023-10-09T06:32:01.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.694+0000 7ff3295e4700 1 -- 172.21.15.163:0/2602151526 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3240b5520 con 0x7ff324159c60 2023-10-09T06:32:01.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.695+0000 7ff31b7fe700 1 -- 172.21.15.163:0/2602151526 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff30c023070 con 0x7ff324159c60 2023-10-09T06:32:01.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.695+0000 7ff31b7fe700 1 --2- 172.21.15.163:0/2602151526 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff304066f30 0x7ff3040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:01.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.695+0000 7ff31b7fe700 1 -- 172.21.15.163:0/2602151526 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff30c085ae0 con 0x7ff324159c60 2023-10-09T06:32:01.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.695+0000 7ff322ffd700 1 --2- 172.21.15.163:0/2602151526 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff304066f30 0x7ff3040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:01.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.695+0000 7ff322ffd700 1 --2- 172.21.15.163:0/2602151526 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff304066f30 0x7ff3040693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff314003a80 tx=0x7ff314007ec0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:01.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.699+0000 7ff31b7fe700 1 -- 172.21.15.163:0/2602151526 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff3240b5520 con 0x7ff324159c60 2023-10-09T06:32:01.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.880+0000 7ff3295e4700 1 -- 172.21.15.163:0/2602151526 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3240b5520 con 0x7ff324159c60 2023-10-09T06:32:01.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.881+0000 7ff31b7fe700 1 -- 172.21.15.163:0/2602151526 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+15411 (secure 0 0 0) 0x7ff3240b5520 con 0x7ff324159c60 2023-10-09T06:32:01.882 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:01.882 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":50,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:32:00.923776+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:31:59.239255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-09T06:31:59.882874+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:32:01.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.883+0000 7ff3295e4700 1 -- 172.21.15.163:0/2602151526 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff304066f30 msgr2=0x7ff3040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:01.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.883+0000 7ff3295e4700 1 --2- 172.21.15.163:0/2602151526 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff304066f30 0x7ff3040693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff314003a80 tx=0x7ff314007ec0 comp rx=0 tx=0).stop 2023-10-09T06:32:01.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.883+0000 7ff3295e4700 1 -- 172.21.15.163:0/2602151526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff324159c60 msgr2=0x7ff32416b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:01.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.883+0000 7ff3295e4700 1 --2- 172.21.15.163:0/2602151526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff324159c60 0x7ff32416b080 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff324070e20 tx=0x7ff30c017000 comp rx=0 tx=0).stop 2023-10-09T06:32:01.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.883+0000 7ff3295e4700 1 -- 172.21.15.163:0/2602151526 shutdown_connections 2023-10-09T06:32:01.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.883+0000 7ff3295e4700 1 --2- 172.21.15.163:0/2602151526 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff304066f30 0x7ff3040693e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:01.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.883+0000 7ff3295e4700 1 --2- 172.21.15.163:0/2602151526 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff32415db10 0x7ff324169ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:01.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.883+0000 7ff3295e4700 1 --2- 172.21.15.163:0/2602151526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff324159890 0x7ff32416ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:01.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.883+0000 7ff3295e4700 1 --2- 172.21.15.163:0/2602151526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff324159c60 0x7ff32416b080 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:01.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.883+0000 7ff3295e4700 1 -- 172.21.15.163:0/2602151526 >> 172.21.15.163:0/2602151526 conn(0x7ff324068e20 msgr2=0x7ff3240c70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:01.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.883+0000 7ff3295e4700 1 -- 172.21.15.163:0/2602151526 shutdown_connections 2023-10-09T06:32:01.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:01.883+0000 7ff3295e4700 1 -- 172.21.15.163:0/2602151526 wait complete. 2023-10-09T06:32:01.896 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:32:02.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.010+0000 7fc7841ad700 1 Processor -- start 2023-10-09T06:32:02.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.010+0000 7fc7841ad700 1 -- start start 2023-10-09T06:32:02.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.011+0000 7fc7841ad700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc77c159890 0x7fc77c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:02.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.011+0000 7fc7841ad700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc77c069560 0x7fc77c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:02.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.011+0000 7fc7841ad700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc77c069ee0 con 0x7fc77c159c60 2023-10-09T06:32:02.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.011+0000 7fc7841ad700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc77c15a160 con 0x7fc77c159890 2023-10-09T06:32:02.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.011+0000 7fc7841ad700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc77c0d13a0 con 0x7fc77c069560 2023-10-09T06:32:02.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.011+0000 7fc781f49700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc77c159890 0x7fc77c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:02.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.011+0000 7fc781748700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fc77c159c60 0x7fc77c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:50840/0 (socket says 172.21.15.163:50840) 2023-10-09T06:32:02.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.011+0000 7fc781748700 1 -- 172.21.15.163:0/2401127721 learned_addr learned my addr 172.21.15.163:0/2401127721 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:02.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.011+0000 7fc781f49700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc77c159890 0x7fc77c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58718/0 (socket says 172.21.15.163:58718) 2023-10-09T06:32:02.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.011+0000 7fc78274a700 1 --2- 172.21.15.163:0/2401127721 >> v2:172.21.15.181:3300/0 conn(0x7fc77c069560 0x7fc77c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:02.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.012+0000 7fc781f49700 1 -- 172.21.15.163:0/2401127721 >> v2:172.21.15.181:3300/0 conn(0x7fc77c069560 msgr2=0x7fc77c0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:02.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.012+0000 7fc781f49700 1 --2- 172.21.15.163:0/2401127721 >> v2:172.21.15.181:3300/0 conn(0x7fc77c069560 0x7fc77c0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:02.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.012+0000 7fc781f49700 1 -- 172.21.15.163:0/2401127721 >> v1:172.21.15.163:6789/0 conn(0x7fc77c159c60 legacy=0x7fc77c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:02.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.012+0000 7fc781f49700 1 -- 172.21.15.163:0/2401127721 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77c15e140 con 0x7fc77c159890 2023-10-09T06:32:02.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.012+0000 7fc780f47700 1 -- 172.21.15.163:0/2401127721 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 432926032 0 0) 0x7fc77c069ee0 con 0x7fc77c159c60 2023-10-09T06:32:02.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.012+0000 7fc781f49700 1 --2- 172.21.15.163:0/2401127721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc77c159890 0x7fc77c15abd0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc77c06a890 tx=0x7fc778009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37e331996a42ae3 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:02.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.012+0000 7fc780f47700 1 -- 172.21.15.163:0/2401127721 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77800e070 con 0x7fc77c159890 2023-10-09T06:32:02.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.012+0000 7fc780f47700 1 -- 172.21.15.163:0/2401127721 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77800bb60 con 0x7fc77c159890 2023-10-09T06:32:02.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.012+0000 7fc780f47700 1 -- 172.21.15.163:0/2401127721 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc778012430 con 0x7fc77c159890 2023-10-09T06:32:02.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.012+0000 7fc7841ad700 1 -- 172.21.15.163:0/2401127721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc77c159890 msgr2=0x7fc77c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:02.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.012+0000 7fc7841ad700 1 --2- 172.21.15.163:0/2401127721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc77c159890 0x7fc77c15abd0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc77c06a890 tx=0x7fc778009780 comp rx=0 tx=0).stop 2023-10-09T06:32:02.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.012+0000 7fc7841ad700 1 -- 172.21.15.163:0/2401127721 shutdown_connections 2023-10-09T06:32:02.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.012+0000 7fc7841ad700 1 --2- 172.21.15.163:0/2401127721 >> v2:172.21.15.181:3300/0 conn(0x7fc77c069560 0x7fc77c0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:02.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.012+0000 7fc7841ad700 1 --2- 172.21.15.163:0/2401127721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc77c159890 0x7fc77c15abd0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:02.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.012+0000 7fc7841ad700 1 -- 172.21.15.163:0/2401127721 >> 172.21.15.163:0/2401127721 conn(0x7fc77c068e20 msgr2=0x7fc77c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:02.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.013+0000 7fc7841ad700 1 -- 172.21.15.163:0/2401127721 shutdown_connections 2023-10-09T06:32:02.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.013+0000 7fc7841ad700 1 -- 172.21.15.163:0/2401127721 wait complete. 2023-10-09T06:32:02.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.014+0000 7fc7841ad700 1 Processor -- start 2023-10-09T06:32:02.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.014+0000 7fc7841ad700 1 -- start start 2023-10-09T06:32:02.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.014+0000 7fc7841ad700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc77c069560 0x7fc77c14c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:02.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.014+0000 7fc7841ad700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc77c159890 0x7fc77c0cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:02.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.014+0000 7fc7841ad700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc77c159c60 0x7fc77c155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:02.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.014+0000 7fc7841ad700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c0cef60 con 0x7fc77c069560 2023-10-09T06:32:02.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.014+0000 7fc7841ad700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c1522b0 con 0x7fc77c159c60 2023-10-09T06:32:02.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.014+0000 7fc7841ad700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c0ce910 con 0x7fc77c159890 2023-10-09T06:32:02.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.014+0000 7fc781f49700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc77c069560 0x7fc77c14c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:02.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.014+0000 7fc781f49700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc77c069560 0x7fc77c14c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58730/0 (socket says 172.21.15.163:58730) 2023-10-09T06:32:02.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.014+0000 7fc781f49700 1 -- 172.21.15.163:0/3605960784 learned_addr learned my addr 172.21.15.163:0/3605960784 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:02.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.015+0000 7fc781748700 1 --2- 172.21.15.163:0/3605960784 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc77c159890 0x7fc77c0cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:02.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.015+0000 7fc78274a700 1 --2- 172.21.15.163:0/3605960784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc77c159c60 0x7fc77c155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:02.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.015+0000 7fc781f49700 1 -- 172.21.15.163:0/3605960784 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc77c159890 msgr2=0x7fc77c0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:02.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.015+0000 7fc781f49700 1 --2- 172.21.15.163:0/3605960784 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc77c159890 0x7fc77c0cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:02.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.015+0000 7fc781f49700 1 -- 172.21.15.163:0/3605960784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc77c159c60 msgr2=0x7fc77c155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:02.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.015+0000 7fc781f49700 1 --2- 172.21.15.163:0/3605960784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc77c159c60 0x7fc77c155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:02.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.015+0000 7fc781f49700 1 -- 172.21.15.163:0/3605960784 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc778009a20 con 0x7fc77c069560 2023-10-09T06:32:02.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.015+0000 7fc781f49700 1 --2- 172.21.15.163:0/3605960784 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc77c069560 0x7fc77c14c9e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc778009fd0 tx=0x7fc778019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:02.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.015+0000 7fc771ffb700 1 -- 172.21.15.163:0/3605960784 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc778012db0 con 0x7fc77c069560 2023-10-09T06:32:02.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.015+0000 7fc771ffb700 1 -- 172.21.15.163:0/3605960784 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc778027470 con 0x7fc77c069560 2023-10-09T06:32:02.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.015+0000 7fc7841ad700 1 -- 172.21.15.163:0/3605960784 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc77c0cebf0 con 0x7fc77c069560 2023-10-09T06:32:02.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.015+0000 7fc771ffb700 1 -- 172.21.15.163:0/3605960784 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc778027830 con 0x7fc77c069560 2023-10-09T06:32:02.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.015+0000 7fc7841ad700 1 -- 172.21.15.163:0/3605960784 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77c0cf930 con 0x7fc77c069560 2023-10-09T06:32:02.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.016+0000 7fc7841ad700 1 -- 172.21.15.163:0/3605960784 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc77c0cf240 con 0x7fc77c069560 2023-10-09T06:32:02.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.017+0000 7fc771ffb700 1 -- 172.21.15.163:0/3605960784 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc778024070 con 0x7fc77c069560 2023-10-09T06:32:02.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.017+0000 7fc771ffb700 1 --2- 172.21.15.163:0/3605960784 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc760066f30 0x7fc7600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:02.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.017+0000 7fc771ffb700 1 -- 172.21.15.163:0/3605960784 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc778086fa0 con 0x7fc77c069560 2023-10-09T06:32:02.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.017+0000 7fc781748700 1 --2- 172.21.15.163:0/3605960784 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc760066f30 0x7fc7600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:02.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.018+0000 7fc781748700 1 --2- 172.21.15.163:0/3605960784 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc760066f30 0x7fc7600693e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc77c16ce70 tx=0x7fc76c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:02.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.021+0000 7fc771ffb700 1 -- 172.21.15.163:0/3605960784 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7780b85a0 con 0x7fc77c069560 2023-10-09T06:32:02.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.205+0000 7fc7841ad700 1 -- 172.21.15.163:0/3605960784 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc77c0ced80 con 0x7fc77c069560 2023-10-09T06:32:02.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.206+0000 7fc771ffb700 1 -- 172.21.15.163:0/3605960784 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+15411 (secure 0 0 0) 0x7fc77805be60 con 0x7fc77c069560 2023-10-09T06:32:02.206 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:02.207 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":50,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:32:00.923776+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:31:59.239255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-09T06:31:59.882874+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:32:02.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.208+0000 7fc7841ad700 1 -- 172.21.15.163:0/3605960784 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc760066f30 msgr2=0x7fc7600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:02.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.208+0000 7fc7841ad700 1 --2- 172.21.15.163:0/3605960784 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc760066f30 0x7fc7600693e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc77c16ce70 tx=0x7fc76c009040 comp rx=0 tx=0).stop 2023-10-09T06:32:02.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.208+0000 7fc7841ad700 1 -- 172.21.15.163:0/3605960784 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc77c069560 msgr2=0x7fc77c14c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:02.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.208+0000 7fc7841ad700 1 --2- 172.21.15.163:0/3605960784 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc77c069560 0x7fc77c14c9e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc778009fd0 tx=0x7fc778019000 comp rx=0 tx=0).stop 2023-10-09T06:32:02.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.208+0000 7fc7841ad700 1 -- 172.21.15.163:0/3605960784 shutdown_connections 2023-10-09T06:32:02.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.208+0000 7fc7841ad700 1 --2- 172.21.15.163:0/3605960784 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc760066f30 0x7fc7600693e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:02.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.208+0000 7fc7841ad700 1 --2- 172.21.15.163:0/3605960784 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc77c069560 0x7fc77c14c9e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:02.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.208+0000 7fc7841ad700 1 --2- 172.21.15.163:0/3605960784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc77c159c60 0x7fc77c155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:02.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.208+0000 7fc7841ad700 1 --2- 172.21.15.163:0/3605960784 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc77c159890 0x7fc77c0cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:02.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.208+0000 7fc7841ad700 1 -- 172.21.15.163:0/3605960784 >> 172.21.15.163:0/3605960784 conn(0x7fc77c068e20 msgr2=0x7fc77c0b22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:02.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.208+0000 7fc7841ad700 1 -- 172.21.15.163:0/3605960784 shutdown_connections 2023-10-09T06:32:02.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.208+0000 7fc7841ad700 1 -- 172.21.15.163:0/3605960784 wait complete. 2023-10-09T06:32:02.221 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-09T06:32:02.338 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.337+0000 7fdd33548700 1 Processor -- start 2023-10-09T06:32:02.338 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.337+0000 7fdd33548700 1 -- start start 2023-10-09T06:32:02.342 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.338+0000 7fdd33548700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd2c159890 0x7fdd2c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:02.342 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.338+0000 7fdd33548700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd2c15a160 con 0x7fdd2c069560 2023-10-09T06:32:02.342 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.338+0000 7fdd33548700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd2c0d13a0 con 0x7fdd2c159c60 2023-10-09T06:32:02.342 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.338+0000 7fdd33548700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c15e140 con 0x7fdd2c159890 2023-10-09T06:32:02.342 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.338+0000 7fdd31ae5700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fdd2c069560 0x7fdd2c0ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:50848/0 (socket says 172.21.15.163:50848) 2023-10-09T06:32:02.342 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.338+0000 7fdd31ae5700 1 -- 172.21.15.163:0/660796110 learned_addr learned my addr 172.21.15.163:0/660796110 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:02.342 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.339+0000 7fdd312e4700 1 --2- 172.21.15.163:0/660796110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd2c159890 0x7fdd2c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:02.343 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.339+0000 7fdd2bfff700 1 -- 172.21.15.163:0/660796110 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1229598087 0 0) 0x7fdd2c15a160 con 0x7fdd2c069560 2023-10-09T06:32:02.343 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.339+0000 7fdd2bfff700 1 -- 172.21.15.163:0/660796110 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd14003570 con 0x7fdd2c069560 2023-10-09T06:32:02.343 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.339+0000 7fdd2bfff700 1 -- 172.21.15.163:0/660796110 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3199196868 0 0) 0x7fdd2c0d13a0 con 0x7fdd2c159c60 2023-10-09T06:32:02.343 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.339+0000 7fdd2bfff700 1 -- 172.21.15.163:0/660796110 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd2c15a160 con 0x7fdd2c159c60 2023-10-09T06:32:02.343 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.339+0000 7fdd2bfff700 1 -- 172.21.15.163:0/660796110 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3134831487 0 0) 0x7fdd14003570 con 0x7fdd2c069560 2023-10-09T06:32:02.343 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.339+0000 7fdd2bfff700 1 -- 172.21.15.163:0/660796110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd2c159890 msgr2=0x7fdd2c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:02.343 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.339+0000 7fdd2bfff700 1 --2- 172.21.15.163:0/660796110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd2c159890 0x7fdd2c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:02.343 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.339+0000 7fdd2bfff700 1 -- 172.21.15.163:0/660796110 >> v1:172.21.15.181:6789/0 conn(0x7fdd2c159c60 legacy=0x7fdd2c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:02.343 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.339+0000 7fdd2bfff700 1 -- 172.21.15.163:0/660796110 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd2c15a3d0 con 0x7fdd2c069560 2023-10-09T06:32:02.343 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.339+0000 7fdd2bfff700 1 -- 172.21.15.163:0/660796110 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fdd24001470 con 0x7fdd2c069560 2023-10-09T06:32:02.343 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.339+0000 7fdd2bfff700 1 -- 172.21.15.163:0/660796110 >> v1:172.21.15.163:6789/0 conn(0x7fdd2c069560 legacy=0x7fdd2c0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:02.343 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.339+0000 7fdd2bfff700 1 --2- 172.21.15.163:0/660796110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd14003c20 0x7fdd14004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:02.344 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.339+0000 7fdd2bfff700 1 --2- 172.21.15.163:0/660796110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd14004810 0x7fdd14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:02.344 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.339+0000 7fdd2bfff700 1 --2- 172.21.15.163:0/660796110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd140051c0 0x7fdd1400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:02.344 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.339+0000 7fdd2bfff700 1 -- 172.21.15.163:0/660796110 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c15e140 con 0x7fdd14003c20 2023-10-09T06:32:02.344 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.339+0000 7fdd2bfff700 1 -- 172.21.15.163:0/660796110 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1400c020 con 0x7fdd14004810 2023-10-09T06:32:02.344 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.339+0000 7fdd2bfff700 1 -- 172.21.15.163:0/660796110 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1400c1e0 con 0x7fdd140051c0 2023-10-09T06:32:02.344 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.339+0000 7fdd312e4700 1 --2- 172.21.15.163:0/660796110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd2c159890 0x7fdd2c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:32:02.344 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.339+0000 7fdd312e4700 1 --2- 172.21.15.163:0/660796110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd14003c20 0x7fdd14004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:02.344 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.339+0000 7fdd30ae3700 1 --2- 172.21.15.163:0/660796110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd14004810 0x7fdd14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:02.344 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.339+0000 7fdd31ae5700 1 --2- 172.21.15.163:0/660796110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd140051c0 0x7fdd1400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:02.344 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.339+0000 7fdd312e4700 1 -- 172.21.15.163:0/660796110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd140051c0 msgr2=0x7fdd1400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:02.344 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.339+0000 7fdd312e4700 1 --2- 172.21.15.163:0/660796110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd140051c0 0x7fdd1400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:02.344 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.340+0000 7fdd312e4700 1 -- 172.21.15.163:0/660796110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd14004810 msgr2=0x7fdd14004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:02.344 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.340+0000 7fdd312e4700 1 --2- 172.21.15.163:0/660796110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd14004810 0x7fdd14004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:02.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.340+0000 7fdd312e4700 1 -- 172.21.15.163:0/660796110 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd14003570 con 0x7fdd14003c20 2023-10-09T06:32:02.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.340+0000 7fdd312e4700 1 --2- 172.21.15.163:0/660796110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd14003c20 0x7fdd14004010 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c065120 tx=0x7fdd1c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96fe9445eb7914a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:02.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.340+0000 7fdd2bfff700 1 -- 172.21.15.163:0/660796110 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c00eaa0 con 0x7fdd14003c20 2023-10-09T06:32:02.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.340+0000 7fdd2bfff700 1 -- 172.21.15.163:0/660796110 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1c00f040 con 0x7fdd14003c20 2023-10-09T06:32:02.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.340+0000 7fdd2bfff700 1 -- 172.21.15.163:0/660796110 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c00ee30 con 0x7fdd14003c20 2023-10-09T06:32:02.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.340+0000 7fdd33548700 1 -- 172.21.15.163:0/660796110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd14003c20 msgr2=0x7fdd14004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:02.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.340+0000 7fdd33548700 1 --2- 172.21.15.163:0/660796110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd14003c20 0x7fdd14004010 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c065120 tx=0x7fdd1c00b040 comp rx=0 tx=0).stop 2023-10-09T06:32:02.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.340+0000 7fdd31ae5700 1 -- 172.21.15.163:0/660796110 reap_dead start 2023-10-09T06:32:02.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.340+0000 7fdd33548700 1 -- 172.21.15.163:0/660796110 shutdown_connections 2023-10-09T06:32:02.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.340+0000 7fdd33548700 1 -- 172.21.15.163:0/660796110 >> 172.21.15.163:0/660796110 conn(0x7fdd2c068e20 msgr2=0x7fdd2c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:02.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.340+0000 7fdd33548700 1 -- 172.21.15.163:0/660796110 shutdown_connections 2023-10-09T06:32:02.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.340+0000 7fdd33548700 1 -- 172.21.15.163:0/660796110 wait complete. 2023-10-09T06:32:02.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.341+0000 7fdd33548700 1 Processor -- start 2023-10-09T06:32:02.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.341+0000 7fdd33548700 1 -- start start 2023-10-09T06:32:02.348 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.342+0000 7fdd33548700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd14003c20 0x7fdd2c0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:02.348 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.342+0000 7fdd33548700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd2c0b3ad0 0x7fdd2c0b5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:02.348 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.342+0000 7fdd33548700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd2c0b5a80 0x7fdd2c0b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:02.348 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.342+0000 7fdd33548700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c0b3f40 con 0x7fdd2c0b3ad0 2023-10-09T06:32:02.349 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.342+0000 7fdd33548700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c0ce080 con 0x7fdd2c0b5a80 2023-10-09T06:32:02.349 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.342+0000 7fdd33548700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c0b6460 con 0x7fdd14003c20 2023-10-09T06:32:02.349 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.342+0000 7fdd30ae3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd2c0b3ad0 0x7fdd2c0b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:02.349 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.342+0000 7fdd30ae3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd2c0b3ad0 0x7fdd2c0b5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58750/0 (socket says 172.21.15.163:58750) 2023-10-09T06:32:02.349 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.342+0000 7fdd30ae3700 1 -- 172.21.15.163:0/1481195741 learned_addr learned my addr 172.21.15.163:0/1481195741 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:02.349 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.342+0000 7fdd312e4700 1 --2- 172.21.15.163:0/1481195741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd14003c20 0x7fdd2c0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:02.349 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.343+0000 7fdd31ae5700 1 --2- 172.21.15.163:0/1481195741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd2c0b5a80 0x7fdd2c0b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:02.349 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.343+0000 7fdd30ae3700 1 -- 172.21.15.163:0/1481195741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd14003c20 msgr2=0x7fdd2c0b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:02.349 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.343+0000 7fdd30ae3700 1 --2- 172.21.15.163:0/1481195741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd14003c20 0x7fdd2c0b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:02.349 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.343+0000 7fdd30ae3700 1 -- 172.21.15.163:0/1481195741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd2c0b5a80 msgr2=0x7fdd2c0b5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:02.349 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.343+0000 7fdd30ae3700 1 --2- 172.21.15.163:0/1481195741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd2c0b5a80 0x7fdd2c0b5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:02.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.343+0000 7fdd30ae3700 1 -- 172.21.15.163:0/1481195741 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd2c0b2410 con 0x7fdd2c0b3ad0 2023-10-09T06:32:02.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.343+0000 7fdd30ae3700 1 --2- 172.21.15.163:0/1481195741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd2c0b3ad0 0x7fdd2c0b5540 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdd20002c00 tx=0x7fdd20007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:02.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.343+0000 7fdd297fa700 1 -- 172.21.15.163:0/1481195741 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2000f8d0 con 0x7fdd2c0b3ad0 2023-10-09T06:32:02.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.343+0000 7fdd297fa700 1 -- 172.21.15.163:0/1481195741 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd2000fa50 con 0x7fdd2c0b3ad0 2023-10-09T06:32:02.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.343+0000 7fdd33548700 1 -- 172.21.15.163:0/1481195741 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd2c0cd750 con 0x7fdd2c0b3ad0 2023-10-09T06:32:02.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.343+0000 7fdd297fa700 1 -- 172.21.15.163:0/1481195741 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2000fe10 con 0x7fdd2c0b3ad0 2023-10-09T06:32:02.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.343+0000 7fdd33548700 1 -- 172.21.15.163:0/1481195741 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd2c0cdd10 con 0x7fdd2c0b3ad0 2023-10-09T06:32:02.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.344+0000 7fdd33548700 1 -- 172.21.15.163:0/1481195741 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd2c0ce1f0 con 0x7fdd2c0b3ad0 2023-10-09T06:32:02.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.344+0000 7fdd297fa700 1 -- 172.21.15.163:0/1481195741 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdd20006a30 con 0x7fdd2c0b3ad0 2023-10-09T06:32:02.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.345+0000 7fdd297fa700 1 --2- 172.21.15.163:0/1481195741 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdd18066f30 0x7fdd180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:02.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.345+0000 7fdd297fa700 1 -- 172.21.15.163:0/1481195741 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdd20084850 con 0x7fdd2c0b3ad0 2023-10-09T06:32:02.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.345+0000 7fdd312e4700 1 --2- 172.21.15.163:0/1481195741 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdd18066f30 0x7fdd180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:02.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.345+0000 7fdd312e4700 1 --2- 172.21.15.163:0/1481195741 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdd18066f30 0x7fdd180693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0099f0 tx=0x7fdd1c016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:02.351 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.349+0000 7fdd297fa700 1 -- 172.21.15.163:0/1481195741 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd20051ac0 con 0x7fdd2c0b3ad0 2023-10-09T06:32:02.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:02.576+0000 7fdd33548700 1 -- 172.21.15.163:0/1481195741 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fdd2c04c3c0 con 0x7fdd2c0b3ad0 2023-10-09T06:32:03.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.562+0000 7fdd297fa700 1 -- 172.21.15.163:0/1481195741 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v30) v1 ==== 118+0+0 (secure 0 0 0) 0x7fdd200550e0 con 0x7fdd2c0b3ad0 2023-10-09T06:32:03.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.565+0000 7fdd33548700 1 -- 172.21.15.163:0/1481195741 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdd18066f30 msgr2=0x7fdd180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:03.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.565+0000 7fdd33548700 1 --2- 172.21.15.163:0/1481195741 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdd18066f30 0x7fdd180693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0099f0 tx=0x7fdd1c016000 comp rx=0 tx=0).stop 2023-10-09T06:32:03.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.565+0000 7fdd33548700 1 -- 172.21.15.163:0/1481195741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd2c0b3ad0 msgr2=0x7fdd2c0b5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:03.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.565+0000 7fdd33548700 1 --2- 172.21.15.163:0/1481195741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd2c0b3ad0 0x7fdd2c0b5540 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdd20002c00 tx=0x7fdd20007d30 comp rx=0 tx=0).stop 2023-10-09T06:32:03.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.565+0000 7fdd33548700 1 -- 172.21.15.163:0/1481195741 shutdown_connections 2023-10-09T06:32:03.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.565+0000 7fdd33548700 1 --2- 172.21.15.163:0/1481195741 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdd18066f30 0x7fdd180693e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:03.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.565+0000 7fdd33548700 1 --2- 172.21.15.163:0/1481195741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd2c0b5a80 0x7fdd2c0b5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:03.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.565+0000 7fdd33548700 1 --2- 172.21.15.163:0/1481195741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd14003c20 0x7fdd2c0b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:03.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.565+0000 7fdd33548700 1 --2- 172.21.15.163:0/1481195741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd2c0b3ad0 0x7fdd2c0b5540 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:03.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.565+0000 7fdd33548700 1 -- 172.21.15.163:0/1481195741 >> 172.21.15.163:0/1481195741 conn(0x7fdd2c068e20 msgr2=0x7fdd2c0d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:03.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.565+0000 7fdd33548700 1 -- 172.21.15.163:0/1481195741 shutdown_connections 2023-10-09T06:32:03.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.565+0000 7fdd33548700 1 -- 172.21.15.163:0/1481195741 wait complete. 2023-10-09T06:32:03.579 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-09T06:32:03.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.698+0000 7f94b2821700 1 Processor -- start 2023-10-09T06:32:03.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.698+0000 7f94b2821700 1 -- start start 2023-10-09T06:32:03.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.699+0000 7f94b2821700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f94ac159890 0x7f94ac15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:03.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.699+0000 7f94b2821700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94ac159c60 0x7f94ac15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:03.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.699+0000 7f94b2821700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94ac15a160 con 0x7f94ac15db10 2023-10-09T06:32:03.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.699+0000 7f94b2821700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94ac0d13a0 con 0x7f94ac159890 2023-10-09T06:32:03.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.699+0000 7f94b2821700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac15e140 con 0x7f94ac159c60 2023-10-09T06:32:03.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.699+0000 7f94abfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f94ac159890 0x7f94ac15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:03.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.699+0000 7f94ab7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94ac159c60 0x7f94ac15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:03.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.699+0000 7f94b0dbe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f94ac15db10 0x7f94ac069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:54374/0 (socket says 172.21.15.163:54374) 2023-10-09T06:32:03.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.699+0000 7f94ab7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94ac159c60 0x7f94ac15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:51716/0 (socket says 172.21.15.163:51716) 2023-10-09T06:32:03.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.699+0000 7f94abfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f94ac159890 0x7f94ac15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:49446/0 (socket says 172.21.15.163:49446) 2023-10-09T06:32:03.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.699+0000 7f94b0dbe700 1 -- 172.21.15.163:0/545938243 learned_addr learned my addr 172.21.15.163:0/545938243 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:03.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.700+0000 7f94aaffd700 1 -- 172.21.15.163:0/545938243 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1161407599 0 0) 0x7f94ac15a160 con 0x7f94ac15db10 2023-10-09T06:32:03.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.700+0000 7f94ab7fe700 1 -- 172.21.15.163:0/545938243 >> v2:172.21.15.181:3300/0 conn(0x7f94ac159890 msgr2=0x7f94ac15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:03.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.700+0000 7f94ab7fe700 1 --2- 172.21.15.163:0/545938243 >> v2:172.21.15.181:3300/0 conn(0x7f94ac159890 0x7f94ac15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:03.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.700+0000 7f94ab7fe700 1 -- 172.21.15.163:0/545938243 >> v1:172.21.15.181:6789/0 conn(0x7f94ac15db10 legacy=0x7f94ac069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:03.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.700+0000 7f94ab7fe700 1 -- 172.21.15.163:0/545938243 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ac15a3d0 con 0x7f94ac159c60 2023-10-09T06:32:03.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.700+0000 7f94abfff700 1 --2- 172.21.15.163:0/545938243 >> v2:172.21.15.181:3300/0 conn(0x7f94ac159890 0x7f94ac15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:32:03.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.700+0000 7f94ab7fe700 1 --2- 172.21.15.163:0/545938243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94ac159c60 0x7f94ac15d5d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f94a00098e0 tx=0x7f94a00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d45b6e282be1ad5 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:03.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.700+0000 7f94aaffd700 1 -- 172.21.15.163:0/545938243 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a000b640 con 0x7f94ac159c60 2023-10-09T06:32:03.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.700+0000 7f94aaffd700 1 -- 172.21.15.163:0/545938243 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a000b7a0 con 0x7f94ac159c60 2023-10-09T06:32:03.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.700+0000 7f94aaffd700 1 -- 172.21.15.163:0/545938243 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a000bb40 con 0x7f94ac159c60 2023-10-09T06:32:03.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.700+0000 7f94b2821700 1 -- 172.21.15.163:0/545938243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94ac159c60 msgr2=0x7f94ac15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:03.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.700+0000 7f94b2821700 1 --2- 172.21.15.163:0/545938243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94ac159c60 0x7f94ac15d5d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f94a00098e0 tx=0x7f94a00091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:03.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.700+0000 7f94b2821700 1 -- 172.21.15.163:0/545938243 shutdown_connections 2023-10-09T06:32:03.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.700+0000 7f94b2821700 1 --2- 172.21.15.163:0/545938243 >> v2:172.21.15.181:3300/0 conn(0x7f94ac159890 0x7f94ac15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:03.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.700+0000 7f94b2821700 1 --2- 172.21.15.163:0/545938243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94ac159c60 0x7f94ac15d5d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:03.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.700+0000 7f94b2821700 1 -- 172.21.15.163:0/545938243 >> 172.21.15.163:0/545938243 conn(0x7f94ac068e20 msgr2=0x7f94ac0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:03.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.701+0000 7f94b2821700 1 -- 172.21.15.163:0/545938243 shutdown_connections 2023-10-09T06:32:03.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.701+0000 7f94b2821700 1 -- 172.21.15.163:0/545938243 wait complete. 2023-10-09T06:32:03.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.702+0000 7f94b2821700 1 Processor -- start 2023-10-09T06:32:03.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.702+0000 7f94b2821700 1 -- start start 2023-10-09T06:32:03.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.702+0000 7f94b2821700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94ac159890 0x7f94ac152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:03.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.702+0000 7f94b2821700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94ac159c60 0x7f94ac0b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:03.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.702+0000 7f94b2821700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94ac15db10 0x7f94ac0b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:03.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.702+0000 7f94b2821700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac0cec20 con 0x7f94ac15db10 2023-10-09T06:32:03.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.702+0000 7f94b2821700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac0ce010 con 0x7f94ac159c60 2023-10-09T06:32:03.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.702+0000 7f94b2821700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac0ce2f0 con 0x7f94ac159890 2023-10-09T06:32:03.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.703+0000 7f94b0dbe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94ac15db10 0x7f94ac0b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:03.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.703+0000 7f94b0dbe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94ac15db10 0x7f94ac0b5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58762/0 (socket says 172.21.15.163:58762) 2023-10-09T06:32:03.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.703+0000 7f94b0dbe700 1 -- 172.21.15.163:0/1050967119 learned_addr learned my addr 172.21.15.163:0/1050967119 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:03.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.703+0000 7f94abfff700 1 --2- 172.21.15.163:0/1050967119 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94ac159890 0x7f94ac152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:03.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.703+0000 7f94ab7fe700 1 --2- 172.21.15.163:0/1050967119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94ac159c60 0x7f94ac0b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:03.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.703+0000 7f94b0dbe700 1 -- 172.21.15.163:0/1050967119 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94ac159890 msgr2=0x7f94ac152800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:03.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.703+0000 7f94b0dbe700 1 --2- 172.21.15.163:0/1050967119 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94ac159890 0x7f94ac152800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:03.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.703+0000 7f94b0dbe700 1 -- 172.21.15.163:0/1050967119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94ac159c60 msgr2=0x7f94ac0b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:03.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.703+0000 7f94b0dbe700 1 --2- 172.21.15.163:0/1050967119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94ac159c60 0x7f94ac0b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:03.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.703+0000 7f94b0dbe700 1 -- 172.21.15.163:0/1050967119 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949c009420 con 0x7f94ac15db10 2023-10-09T06:32:03.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.703+0000 7f94abfff700 1 --2- 172.21.15.163:0/1050967119 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94ac159890 0x7f94ac152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:03.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.703+0000 7f94b0dbe700 1 --2- 172.21.15.163:0/1050967119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94ac15db10 0x7f94ac0b5a20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f94a400cbf0 tx=0x7f94a40019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:03.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.703+0000 7f9493fff700 1 -- 172.21.15.163:0/1050967119 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a400a9c0 con 0x7f94ac15db10 2023-10-09T06:32:03.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.703+0000 7f9493fff700 1 -- 172.21.15.163:0/1050967119 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a400ab20 con 0x7f94ac15db10 2023-10-09T06:32:03.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.703+0000 7f94b2821700 1 -- 172.21.15.163:0/1050967119 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a0009420 con 0x7f94ac15db10 2023-10-09T06:32:03.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.703+0000 7f9493fff700 1 -- 172.21.15.163:0/1050967119 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a40070f0 con 0x7f94ac15db10 2023-10-09T06:32:03.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.703+0000 7f94b2821700 1 -- 172.21.15.163:0/1050967119 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ac0ce8b0 con 0x7f94ac15db10 2023-10-09T06:32:03.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.704+0000 7f94b2821700 1 -- 172.21.15.163:0/1050967119 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94ac0890e0 con 0x7f94ac15db10 2023-10-09T06:32:03.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.705+0000 7f9493fff700 1 -- 172.21.15.163:0/1050967119 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f94a4007520 con 0x7f94ac15db10 2023-10-09T06:32:03.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.705+0000 7f9493fff700 1 --2- 172.21.15.163:0/1050967119 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9498066f30 0x7f94980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:03.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.705+0000 7f9493fff700 1 -- 172.21.15.163:0/1050967119 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f94a4084980 con 0x7f94ac15db10 2023-10-09T06:32:03.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.705+0000 7f94abfff700 1 --2- 172.21.15.163:0/1050967119 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9498066f30 0x7f94980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:03.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.706+0000 7f94abfff700 1 --2- 172.21.15.163:0/1050967119 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9498066f30 0x7f94980693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f949c0093f0 tx=0x7f949c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:03.710 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.709+0000 7f9493fff700 1 -- 172.21.15.163:0/1050967119 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f94a4051c70 con 0x7f94ac15db10 2023-10-09T06:32:03.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:03.932+0000 7f94b2821700 1 -- 172.21.15.163:0/1050967119 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f94ac04c3c0 con 0x7f94ac15db10 2023-10-09T06:32:04.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.570+0000 7f9493fff700 1 -- 172.21.15.163:0/1050967119 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v31) v1 ==== 122+0+0 (secure 0 0 0) 0x7f94a4055290 con 0x7f94ac15db10 2023-10-09T06:32:04.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.572+0000 7f94b2821700 1 -- 172.21.15.163:0/1050967119 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9498066f30 msgr2=0x7f94980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:04.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.572+0000 7f94b2821700 1 --2- 172.21.15.163:0/1050967119 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9498066f30 0x7f94980693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f949c0093f0 tx=0x7f949c008040 comp rx=0 tx=0).stop 2023-10-09T06:32:04.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.572+0000 7f94b2821700 1 -- 172.21.15.163:0/1050967119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94ac15db10 msgr2=0x7f94ac0b5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:04.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.572+0000 7f94b2821700 1 --2- 172.21.15.163:0/1050967119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94ac15db10 0x7f94ac0b5a20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f94a400cbf0 tx=0x7f94a40019b0 comp rx=0 tx=0).stop 2023-10-09T06:32:04.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.572+0000 7f94b2821700 1 -- 172.21.15.163:0/1050967119 shutdown_connections 2023-10-09T06:32:04.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.572+0000 7f94b2821700 1 --2- 172.21.15.163:0/1050967119 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9498066f30 0x7f94980693e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:04.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.572+0000 7f94b2821700 1 --2- 172.21.15.163:0/1050967119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94ac15db10 0x7f94ac0b5a20 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:04.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.572+0000 7f94b2821700 1 --2- 172.21.15.163:0/1050967119 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94ac159890 0x7f94ac152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:04.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.572+0000 7f94b2821700 1 --2- 172.21.15.163:0/1050967119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94ac159c60 0x7f94ac0b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:04.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.572+0000 7f94b2821700 1 -- 172.21.15.163:0/1050967119 >> 172.21.15.163:0/1050967119 conn(0x7f94ac068e20 msgr2=0x7f94ac152f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:04.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.572+0000 7f94b2821700 1 -- 172.21.15.163:0/1050967119 shutdown_connections 2023-10-09T06:32:04.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.572+0000 7f94b2821700 1 -- 172.21.15.163:0/1050967119 wait complete. 2023-10-09T06:32:04.585 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:32:04.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.698+0000 7f6d17474700 1 Processor -- start 2023-10-09T06:32:04.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.698+0000 7f6d17474700 1 -- start start 2023-10-09T06:32:04.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.699+0000 7f6d17474700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6d1006bcd0 0x7f6d100caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:04.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.699+0000 7f6d17474700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d1006c0a0 0x7f6d1015a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:04.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.699+0000 7f6d17474700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6d1015df50 0x7f6d1015e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:04.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.699+0000 7f6d17474700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d1015d7e0 con 0x7f6d1015df50 2023-10-09T06:32:04.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.699+0000 7f6d17474700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d1015dac0 con 0x7f6d1006bcd0 2023-10-09T06:32:04.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.699+0000 7f6d17474700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1015aa30 con 0x7f6d1006c0a0 2023-10-09T06:32:04.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.699+0000 7f6d15a11700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6d1015df50 0x7f6d1015e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:04.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.699+0000 7f6d15a11700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6d1015df50 0x7f6d1015e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58770/0 (socket says 172.21.15.163:58770) 2023-10-09T06:32:04.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.699+0000 7f6d15a11700 1 -- 172.21.15.163:0/4008317938 learned_addr learned my addr 172.21.15.163:0/4008317938 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:04.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.699+0000 7f6d15210700 1 --2- 172.21.15.163:0/4008317938 >> v2:172.21.15.181:3300/0 conn(0x7f6d1006bcd0 0x7f6d100caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:04.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.699+0000 7f6d14a0f700 1 --2- 172.21.15.163:0/4008317938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d1006c0a0 0x7f6d1015a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:04.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.700+0000 7f6d15a11700 1 -- 172.21.15.163:0/4008317938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d1006c0a0 msgr2=0x7f6d1015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:04.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.700+0000 7f6d15a11700 1 --2- 172.21.15.163:0/4008317938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d1006c0a0 0x7f6d1015a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:04.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.700+0000 7f6d15a11700 1 -- 172.21.15.163:0/4008317938 >> v2:172.21.15.181:3300/0 conn(0x7f6d1006bcd0 msgr2=0x7f6d100caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:04.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.700+0000 7f6d15a11700 1 --2- 172.21.15.163:0/4008317938 >> v2:172.21.15.181:3300/0 conn(0x7f6d1006bcd0 0x7f6d100caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:04.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.700+0000 7f6d15a11700 1 -- 172.21.15.163:0/4008317938 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1015ad10 con 0x7f6d1015df50 2023-10-09T06:32:04.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.700+0000 7f6d15a11700 1 --2- 172.21.15.163:0/4008317938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d1015df50 0x7f6d1015e3c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6d100691a0 tx=0x7f6d08009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8fe84827764844b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:04.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.700+0000 7f6d07fff700 1 -- 172.21.15.163:0/4008317938 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0800b640 con 0x7f6d1015df50 2023-10-09T06:32:04.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.700+0000 7f6d07fff700 1 -- 172.21.15.163:0/4008317938 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0800b7a0 con 0x7f6d1015df50 2023-10-09T06:32:04.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.700+0000 7f6d07fff700 1 -- 172.21.15.163:0/4008317938 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0800bb40 con 0x7f6d1015df50 2023-10-09T06:32:04.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.700+0000 7f6d17474700 1 -- 172.21.15.163:0/4008317938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d1015df50 msgr2=0x7f6d1015e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:04.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.700+0000 7f6d17474700 1 --2- 172.21.15.163:0/4008317938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d1015df50 0x7f6d1015e3c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6d100691a0 tx=0x7f6d08009250 comp rx=0 tx=0).stop 2023-10-09T06:32:04.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.701+0000 7f6d17474700 1 -- 172.21.15.163:0/4008317938 shutdown_connections 2023-10-09T06:32:04.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.701+0000 7f6d17474700 1 --2- 172.21.15.163:0/4008317938 >> v2:172.21.15.181:3300/0 conn(0x7f6d1006bcd0 0x7f6d100caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:04.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.701+0000 7f6d17474700 1 --2- 172.21.15.163:0/4008317938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d1015df50 0x7f6d1015e3c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:04.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.701+0000 7f6d17474700 1 --2- 172.21.15.163:0/4008317938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d1006c0a0 0x7f6d1015a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:04.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.701+0000 7f6d17474700 1 -- 172.21.15.163:0/4008317938 >> 172.21.15.163:0/4008317938 conn(0x7f6d100ce600 msgr2=0x7f6d100cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:04.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.701+0000 7f6d17474700 1 -- 172.21.15.163:0/4008317938 shutdown_connections 2023-10-09T06:32:04.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.701+0000 7f6d17474700 1 -- 172.21.15.163:0/4008317938 wait complete. 2023-10-09T06:32:04.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.702+0000 7f6d17474700 1 Processor -- start 2023-10-09T06:32:04.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.702+0000 7f6d17474700 1 -- start start 2023-10-09T06:32:04.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.703+0000 7f6d17474700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d1006bcd0 0x7f6d100b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:04.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.703+0000 7f6d17474700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d1006c0a0 0x7f6d1014e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:04.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.703+0000 7f6d17474700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d1015df50 0x7f6d10154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:04.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.703+0000 7f6d17474700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d100b3010 con 0x7f6d1006bcd0 2023-10-09T06:32:04.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.703+0000 7f6d17474700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1016a9d0 con 0x7f6d1015df50 2023-10-09T06:32:04.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.703+0000 7f6d17474700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1016acb0 con 0x7f6d1006c0a0 2023-10-09T06:32:04.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.703+0000 7f6d15210700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d1006bcd0 0x7f6d100b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:04.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.703+0000 7f6d15210700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d1006bcd0 0x7f6d100b2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58774/0 (socket says 172.21.15.163:58774) 2023-10-09T06:32:04.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.703+0000 7f6d15210700 1 -- 172.21.15.163:0/1531980409 learned_addr learned my addr 172.21.15.163:0/1531980409 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:04.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.703+0000 7f6d14a0f700 1 --2- 172.21.15.163:0/1531980409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d1006c0a0 0x7f6d1014e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:04.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.703+0000 7f6d15a11700 1 --2- 172.21.15.163:0/1531980409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d1015df50 0x7f6d10154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:04.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.703+0000 7f6d15210700 1 -- 172.21.15.163:0/1531980409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d1006c0a0 msgr2=0x7f6d1014e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:04.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.703+0000 7f6d15210700 1 --2- 172.21.15.163:0/1531980409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d1006c0a0 0x7f6d1014e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:04.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.703+0000 7f6d15210700 1 -- 172.21.15.163:0/1531980409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d1015df50 msgr2=0x7f6d10154d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:04.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.703+0000 7f6d15210700 1 --2- 172.21.15.163:0/1531980409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d1015df50 0x7f6d10154d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:04.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.703+0000 7f6d15210700 1 -- 172.21.15.163:0/1531980409 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d080094f0 con 0x7f6d1006bcd0 2023-10-09T06:32:04.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.703+0000 7f6d15a11700 1 --2- 172.21.15.163:0/1531980409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d1015df50 0x7f6d10154d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:32:04.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.703+0000 7f6d15210700 1 --2- 172.21.15.163:0/1531980409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d1006bcd0 0x7f6d100b2ad0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009e50 tx=0x7f6d0c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:04.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.703+0000 7f6d057fa700 1 -- 172.21.15.163:0/1531980409 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c00cc20 con 0x7f6d1006bcd0 2023-10-09T06:32:04.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.703+0000 7f6d057fa700 1 -- 172.21.15.163:0/1531980409 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0c00f040 con 0x7f6d1006bcd0 2023-10-09T06:32:04.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.703+0000 7f6d17474700 1 -- 172.21.15.163:0/1531980409 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1016af90 con 0x7f6d1006bcd0 2023-10-09T06:32:04.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.703+0000 7f6d057fa700 1 -- 172.21.15.163:0/1531980409 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c014480 con 0x7f6d1006bcd0 2023-10-09T06:32:04.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.703+0000 7f6d17474700 1 -- 172.21.15.163:0/1531980409 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d1014eca0 con 0x7f6d1006bcd0 2023-10-09T06:32:04.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.704+0000 7f6d17474700 1 -- 172.21.15.163:0/1531980409 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d1004c3c0 con 0x7f6d1006bcd0 2023-10-09T06:32:04.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.704+0000 7f6d057fa700 1 -- 172.21.15.163:0/1531980409 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6d0c0147f0 con 0x7f6d1006bcd0 2023-10-09T06:32:04.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.705+0000 7f6d057fa700 1 --2- 172.21.15.163:0/1531980409 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6cf4066f30 0x7f6cf40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:04.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.705+0000 7f6d057fa700 1 -- 172.21.15.163:0/1531980409 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6d0c0851f0 con 0x7f6d1006bcd0 2023-10-09T06:32:04.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.705+0000 7f6d14a0f700 1 --2- 172.21.15.163:0/1531980409 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6cf4066f30 0x7f6cf40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:04.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.706+0000 7f6d14a0f700 1 --2- 172.21.15.163:0/1531980409 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6cf4066f30 0x7f6cf40693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6d100b28c0 tx=0x7f6d00008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:04.710 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.709+0000 7f6d057fa700 1 -- 172.21.15.163:0/1531980409 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d0c052460 con 0x7f6d1006bcd0 2023-10-09T06:32:04.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.934+0000 7f6d17474700 1 -- 172.21.15.163:0/1531980409 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d1015aff0 con 0x7f6d1006bcd0 2023-10-09T06:32:04.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.934+0000 7f6d057fa700 1 -- 172.21.15.163:0/1531980409 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f6d0c055a80 con 0x7f6d1006bcd0 2023-10-09T06:32:04.936 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:04.936 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/819128421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":819128421},{"type":"v1","addr":"172.21.15.181:6836","nonce":819128421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3339496202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3339496202},{"type":"v1","addr":"172.21.15.163:6834","nonce":3339496202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5352,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1726174482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1726174482},{"type":"v1","addr":"172.21.15.163:6835","nonce":1726174482}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:32:00.924137+0000","modified":"2023-10-09T06:32:04.563062+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5370},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5370":{"gid":5370,"name":"b","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2945196635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2945196635},{"type":"v1","addr":"172.21.15.181:6837","nonce":2945196635}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-09T06:32:04.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.936+0000 7f6d17474700 1 -- 172.21.15.163:0/1531980409 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6cf4066f30 msgr2=0x7f6cf40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:04.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.936+0000 7f6d17474700 1 --2- 172.21.15.163:0/1531980409 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6cf4066f30 0x7f6cf40693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6d100b28c0 tx=0x7f6d00008040 comp rx=0 tx=0).stop 2023-10-09T06:32:04.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.936+0000 7f6d17474700 1 -- 172.21.15.163:0/1531980409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d1006bcd0 msgr2=0x7f6d100b2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:04.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.936+0000 7f6d17474700 1 --2- 172.21.15.163:0/1531980409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d1006bcd0 0x7f6d100b2ad0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009e50 tx=0x7f6d0c00b040 comp rx=0 tx=0).stop 2023-10-09T06:32:04.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.937+0000 7f6d17474700 1 -- 172.21.15.163:0/1531980409 shutdown_connections 2023-10-09T06:32:04.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.937+0000 7f6d17474700 1 --2- 172.21.15.163:0/1531980409 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6cf4066f30 0x7f6cf40693e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:04.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.937+0000 7f6d17474700 1 --2- 172.21.15.163:0/1531980409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d1006bcd0 0x7f6d100b2ad0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:04.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.937+0000 7f6d17474700 1 --2- 172.21.15.163:0/1531980409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d1015df50 0x7f6d10154d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:04.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.937+0000 7f6d17474700 1 --2- 172.21.15.163:0/1531980409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d1006c0a0 0x7f6d1014e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:04.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.937+0000 7f6d17474700 1 -- 172.21.15.163:0/1531980409 >> 172.21.15.163:0/1531980409 conn(0x7f6d100ce600 msgr2=0x7f6d100c8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:04.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.937+0000 7f6d17474700 1 -- 172.21.15.163:0/1531980409 shutdown_connections 2023-10-09T06:32:04.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:04.937+0000 7f6d17474700 1 -- 172.21.15.163:0/1531980409 wait complete. 2023-10-09T06:32:04.939 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 31 2023-10-09T06:32:04.949 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:32:05.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.065+0000 7fe1f718f700 1 Processor -- start 2023-10-09T06:32:05.066 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.065+0000 7fe1f718f700 1 -- start start 2023-10-09T06:32:05.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.065+0000 7fe1f718f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1f0140720 0x7fe1f0068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:05.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.065+0000 7fe1f718f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe1f00688d0 0x7fe1f00c8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:05.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.065+0000 7fe1f718f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1f00c7de0 con 0x7fe1f0140350 2023-10-09T06:32:05.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.065+0000 7fe1f718f700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1f0068d50 con 0x7fe1f00688d0 2023-10-09T06:32:05.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.065+0000 7fe1f718f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f0140c10 con 0x7fe1f0140720 2023-10-09T06:32:05.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.065+0000 7fe1f572c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe1f00688d0 0x7fe1f00c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:05.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.066+0000 7fe1f572c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe1f00688d0 0x7fe1f00c8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58778/0 (socket says 172.21.15.163:58778) 2023-10-09T06:32:05.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.066+0000 7fe1f572c700 1 -- 172.21.15.163:0/924190346 learned_addr learned my addr 172.21.15.163:0/924190346 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:05.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.066+0000 7fe1effff700 1 --2- 172.21.15.163:0/924190346 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1f0140720 0x7fe1f0068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:05.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.066+0000 7fe1f572c700 1 -- 172.21.15.163:0/924190346 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1f0140720 msgr2=0x7fe1f0068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:05.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.066+0000 7fe1f572c700 1 --2- 172.21.15.163:0/924190346 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1f0140720 0x7fe1f0068390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:05.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.066+0000 7fe1f572c700 1 -- 172.21.15.163:0/924190346 >> v1:172.21.15.181:6789/0 conn(0x7fe1f0140350 legacy=0x7fe1f0069c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:05.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.066+0000 7fe1f572c700 1 -- 172.21.15.163:0/924190346 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f0141490 con 0x7fe1f00688d0 2023-10-09T06:32:05.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.066+0000 7fe1f572c700 1 --2- 172.21.15.163:0/924190346 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1f00688d0 0x7fe1f00c8270 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8009f20 tx=0x7fe1e8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff5cf560629a221 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:05.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.066+0000 7fe1ef7fe700 1 -- 172.21.15.163:0/924190346 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e800e070 con 0x7fe1f00688d0 2023-10-09T06:32:05.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.066+0000 7fe1ef7fe700 1 -- 172.21.15.163:0/924190346 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e800ba40 con 0x7fe1f00688d0 2023-10-09T06:32:05.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.066+0000 7fe1ef7fe700 1 -- 172.21.15.163:0/924190346 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e80092e0 con 0x7fe1f00688d0 2023-10-09T06:32:05.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.067+0000 7fe1f718f700 1 -- 172.21.15.163:0/924190346 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1f00688d0 msgr2=0x7fe1f00c8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:05.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.067+0000 7fe1f718f700 1 --2- 172.21.15.163:0/924190346 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1f00688d0 0x7fe1f00c8270 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8009f20 tx=0x7fe1e8009840 comp rx=0 tx=0).stop 2023-10-09T06:32:05.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.067+0000 7fe1f718f700 1 -- 172.21.15.163:0/924190346 shutdown_connections 2023-10-09T06:32:05.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.067+0000 7fe1f718f700 1 --2- 172.21.15.163:0/924190346 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1f00688d0 0x7fe1f00c8270 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:05.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.067+0000 7fe1f718f700 1 --2- 172.21.15.163:0/924190346 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1f0140720 0x7fe1f0068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:05.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.067+0000 7fe1f718f700 1 -- 172.21.15.163:0/924190346 >> 172.21.15.163:0/924190346 conn(0x7fe1f00cd670 msgr2=0x7fe1f00cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:05.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.067+0000 7fe1f718f700 1 -- 172.21.15.163:0/924190346 shutdown_connections 2023-10-09T06:32:05.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.067+0000 7fe1f718f700 1 -- 172.21.15.163:0/924190346 wait complete. 2023-10-09T06:32:05.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.068+0000 7fe1f718f700 1 Processor -- start 2023-10-09T06:32:05.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.068+0000 7fe1f718f700 1 -- start start 2023-10-09T06:32:05.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.069+0000 7fe1f718f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1f00688d0 0x7fe1f00c4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:05.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.069+0000 7fe1f718f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1f0140350 0x7fe1f00ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:05.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.069+0000 7fe1f718f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1f0140720 0x7fe1f015a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:05.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.069+0000 7fe1f718f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f00c5d10 con 0x7fe1f00688d0 2023-10-09T06:32:05.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.069+0000 7fe1f718f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f00c5090 con 0x7fe1f0140350 2023-10-09T06:32:05.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.069+0000 7fe1f718f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f00c5370 con 0x7fe1f0140720 2023-10-09T06:32:05.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.069+0000 7fe1f4f2b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1f00688d0 0x7fe1f00c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:05.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.069+0000 7fe1f4f2b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1f00688d0 0x7fe1f00c4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58780/0 (socket says 172.21.15.163:58780) 2023-10-09T06:32:05.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.069+0000 7fe1f4f2b700 1 -- 172.21.15.163:0/3442021187 learned_addr learned my addr 172.21.15.163:0/3442021187 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:05.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.069+0000 7fe1effff700 1 --2- 172.21.15.163:0/3442021187 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1f0140350 0x7fe1f00ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:05.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.069+0000 7fe1f572c700 1 --2- 172.21.15.163:0/3442021187 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1f0140720 0x7fe1f015a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:05.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.069+0000 7fe1f4f2b700 1 -- 172.21.15.163:0/3442021187 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1f0140720 msgr2=0x7fe1f015a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:05.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.069+0000 7fe1f4f2b700 1 --2- 172.21.15.163:0/3442021187 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1f0140720 0x7fe1f015a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:05.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.069+0000 7fe1f4f2b700 1 -- 172.21.15.163:0/3442021187 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1f0140350 msgr2=0x7fe1f00ca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:05.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.069+0000 7fe1f4f2b700 1 --2- 172.21.15.163:0/3442021187 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1f0140350 0x7fe1f00ca4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:05.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.069+0000 7fe1f4f2b700 1 -- 172.21.15.163:0/3442021187 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e8009ae0 con 0x7fe1f00688d0 2023-10-09T06:32:05.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.069+0000 7fe1f572c700 1 --2- 172.21.15.163:0/3442021187 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1f0140720 0x7fe1f015a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:05.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.069+0000 7fe1f4f2b700 1 --2- 172.21.15.163:0/3442021187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1f00688d0 0x7fe1f00c4720 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400c5b0 tx=0x7fe1e40020e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:05.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.069+0000 7fe1ecff9700 1 -- 172.21.15.163:0/3442021187 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4011590 con 0x7fe1f00688d0 2023-10-09T06:32:05.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.069+0000 7fe1ecff9700 1 -- 172.21.15.163:0/3442021187 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e40116f0 con 0x7fe1f00688d0 2023-10-09T06:32:05.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.070+0000 7fe1f718f700 1 -- 172.21.15.163:0/3442021187 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f00c5650 con 0x7fe1f00688d0 2023-10-09T06:32:05.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.070+0000 7fe1ecff9700 1 -- 172.21.15.163:0/3442021187 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4011a90 con 0x7fe1f00688d0 2023-10-09T06:32:05.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.070+0000 7fe1f718f700 1 -- 172.21.15.163:0/3442021187 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f00c4c60 con 0x7fe1f00688d0 2023-10-09T06:32:05.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.071+0000 7fe1f718f700 1 -- 172.21.15.163:0/3442021187 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1f0158ac0 con 0x7fe1f00688d0 2023-10-09T06:32:05.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.071+0000 7fe1ecff9700 1 -- 172.21.15.163:0/3442021187 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe1e4012420 con 0x7fe1f00688d0 2023-10-09T06:32:05.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.071+0000 7fe1ecff9700 1 --2- 172.21.15.163:0/3442021187 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe1d4066f30 0x7fe1d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:05.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.071+0000 7fe1ecff9700 1 -- 172.21.15.163:0/3442021187 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe1e4083ff0 con 0x7fe1f00688d0 2023-10-09T06:32:05.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.072+0000 7fe1effff700 1 --2- 172.21.15.163:0/3442021187 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe1d4066f30 0x7fe1d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:05.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.072+0000 7fe1effff700 1 --2- 172.21.15.163:0/3442021187 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe1d4066f30 0x7fe1d40693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe1f013cdc0 tx=0x7fe1dc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:05.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.076+0000 7fe1ecff9700 1 -- 172.21.15.163:0/3442021187 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe1e4051260 con 0x7fe1f00688d0 2023-10-09T06:32:05.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.258+0000 7fe1f718f700 1 -- 172.21.15.163:0/3442021187 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1f00c4e50 con 0x7fe1f00688d0 2023-10-09T06:32:05.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.259+0000 7fe1ecff9700 1 -- 172.21.15.163:0/3442021187 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fe1f00c4e50 con 0x7fe1f00688d0 2023-10-09T06:32:05.260 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:05.260 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":51,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:32:02.534579+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:31:59.239255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-09T06:31:59.882874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:32:05.262 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.261+0000 7fe1f718f700 1 -- 172.21.15.163:0/3442021187 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe1d4066f30 msgr2=0x7fe1d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:05.262 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.261+0000 7fe1f718f700 1 --2- 172.21.15.163:0/3442021187 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe1d4066f30 0x7fe1d40693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe1f013cdc0 tx=0x7fe1dc008040 comp rx=0 tx=0).stop 2023-10-09T06:32:05.262 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.261+0000 7fe1f718f700 1 -- 172.21.15.163:0/3442021187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1f00688d0 msgr2=0x7fe1f00c4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:05.262 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.261+0000 7fe1f718f700 1 --2- 172.21.15.163:0/3442021187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1f00688d0 0x7fe1f00c4720 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400c5b0 tx=0x7fe1e40020e0 comp rx=0 tx=0).stop 2023-10-09T06:32:05.262 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.261+0000 7fe1f718f700 1 -- 172.21.15.163:0/3442021187 shutdown_connections 2023-10-09T06:32:05.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.261+0000 7fe1f718f700 1 --2- 172.21.15.163:0/3442021187 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe1d4066f30 0x7fe1d40693e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:05.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.261+0000 7fe1f718f700 1 --2- 172.21.15.163:0/3442021187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1f00688d0 0x7fe1f00c4720 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:05.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.261+0000 7fe1f718f700 1 --2- 172.21.15.163:0/3442021187 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1f0140720 0x7fe1f015a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:05.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.261+0000 7fe1f718f700 1 --2- 172.21.15.163:0/3442021187 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1f0140350 0x7fe1f00ca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:05.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.261+0000 7fe1f718f700 1 -- 172.21.15.163:0/3442021187 >> 172.21.15.163:0/3442021187 conn(0x7fe1f00cd670 msgr2=0x7fe1f013cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:05.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.261+0000 7fe1f718f700 1 -- 172.21.15.163:0/3442021187 shutdown_connections 2023-10-09T06:32:05.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.261+0000 7fe1f718f700 1 -- 172.21.15.163:0/3442021187 wait complete. 2023-10-09T06:32:05.274 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-09T06:32:05.274 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:32:05.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.388+0000 7f3eb48ad700 1 Processor -- start 2023-10-09T06:32:05.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.388+0000 7f3eb48ad700 1 -- start start 2023-10-09T06:32:05.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.389+0000 7f3eb48ad700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3eb015d230 0x7f3eb015d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:05.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.389+0000 7f3eb48ad700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eb00b2980 con 0x7f3eb01597f0 2023-10-09T06:32:05.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.389+0000 7f3eb48ad700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eb015db70 con 0x7f3eb0159420 2023-10-09T06:32:05.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.389+0000 7f3eb48ad700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eb00b11d0 con 0x7f3eb015d230 2023-10-09T06:32:05.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.389+0000 7f3eaeffd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f3eb01597f0 0x7f3eb00b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:50852/0 (socket says 172.21.15.163:50852) 2023-10-09T06:32:05.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.389+0000 7f3eaeffd700 1 -- 172.21.15.163:0/3028752222 learned_addr learned my addr 172.21.15.163:0/3028752222 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:05.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.389+0000 7f3eaffff700 1 --2- 172.21.15.163:0/3028752222 >> v2:172.21.15.181:3300/0 conn(0x7f3eb015d230 0x7f3eb015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:05.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.389+0000 7f3eae7fc700 1 -- 172.21.15.163:0/3028752222 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2676726243 0 0) 0x7f3eb00b2980 con 0x7f3eb01597f0 2023-10-09T06:32:05.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.389+0000 7f3eae7fc700 1 -- 172.21.15.163:0/3028752222 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e98003570 con 0x7f3eb01597f0 2023-10-09T06:32:05.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.389+0000 7f3eae7fc700 1 -- 172.21.15.163:0/3028752222 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 380291611 0 0) 0x7f3eb015db70 con 0x7f3eb0159420 2023-10-09T06:32:05.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.389+0000 7f3eae7fc700 1 -- 172.21.15.163:0/3028752222 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3eb00b2980 con 0x7f3eb0159420 2023-10-09T06:32:05.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.390+0000 7f3eaffff700 1 -- 172.21.15.163:0/3028752222 >> v1:172.21.15.181:6789/0 conn(0x7f3eb0159420 legacy=0x7f3eb00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:05.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.390+0000 7f3eaffff700 1 -- 172.21.15.163:0/3028752222 >> v1:172.21.15.163:6789/0 conn(0x7f3eb01597f0 legacy=0x7f3eb00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:05.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.390+0000 7f3eae7fc700 1 -- 172.21.15.163:0/3028752222 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2183137076 0 0) 0x7f3e98003570 con 0x7f3eb01597f0 2023-10-09T06:32:05.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.390+0000 7f3eaffff700 1 -- 172.21.15.163:0/3028752222 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb00b14b0 con 0x7f3eb015d230 2023-10-09T06:32:05.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.390+0000 7f3eaffff700 1 --2- 172.21.15.163:0/3028752222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3eb015d230 0x7f3eb015d630 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80098e0 tx=0x7f3ea80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5764c836d5976524 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:05.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.390+0000 7f3eae7fc700 1 -- 172.21.15.163:0/3028752222 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4001470 con 0x7f3eb015d230 2023-10-09T06:32:05.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.390+0000 7f3eae7fc700 1 -- 172.21.15.163:0/3028752222 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea8011420 con 0x7f3eb015d230 2023-10-09T06:32:05.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.390+0000 7f3eae7fc700 1 -- 172.21.15.163:0/3028752222 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea80117a0 con 0x7f3eb015d230 2023-10-09T06:32:05.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.390+0000 7f3eb48ad700 1 -- 172.21.15.163:0/3028752222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3eb015d230 msgr2=0x7f3eb015d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:05.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.390+0000 7f3eb48ad700 1 --2- 172.21.15.163:0/3028752222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3eb015d230 0x7f3eb015d630 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80098e0 tx=0x7f3ea80091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:05.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.390+0000 7f3eb48ad700 1 -- 172.21.15.163:0/3028752222 shutdown_connections 2023-10-09T06:32:05.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.390+0000 7f3eb48ad700 1 --2- 172.21.15.163:0/3028752222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3eb015d230 0x7f3eb015d630 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:05.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.390+0000 7f3eb48ad700 1 -- 172.21.15.163:0/3028752222 >> 172.21.15.163:0/3028752222 conn(0x7f3eb00d0660 msgr2=0x7f3eb00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:05.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.391+0000 7f3eb48ad700 1 -- 172.21.15.163:0/3028752222 shutdown_connections 2023-10-09T06:32:05.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.391+0000 7f3eb48ad700 1 -- 172.21.15.163:0/3028752222 wait complete. 2023-10-09T06:32:05.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.392+0000 7f3eb48ad700 1 Processor -- start 2023-10-09T06:32:05.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.392+0000 7f3eb48ad700 1 -- start start 2023-10-09T06:32:05.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.392+0000 7f3eb48ad700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3eb0159420 0x7f3eb015af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:05.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.392+0000 7f3eb48ad700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3eb01597f0 0x7f3eb01548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:05.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.392+0000 7f3eb48ad700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3eb015d230 0x7f3eb00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:05.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.392+0000 7f3eb48ad700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb0154e00 con 0x7f3eb0159420 2023-10-09T06:32:05.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.392+0000 7f3eb48ad700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb015c1a0 con 0x7f3eb01597f0 2023-10-09T06:32:05.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.392+0000 7f3eb48ad700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb00c5da0 con 0x7f3eb015d230 2023-10-09T06:32:05.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.393+0000 7f3eaf7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3eb0159420 0x7f3eb015af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:05.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.393+0000 7f3eaf7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3eb0159420 0x7f3eb015af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58782/0 (socket says 172.21.15.163:58782) 2023-10-09T06:32:05.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.393+0000 7f3eaf7fe700 1 -- 172.21.15.163:0/3142944655 learned_addr learned my addr 172.21.15.163:0/3142944655 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:05.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.393+0000 7f3eaeffd700 1 --2- 172.21.15.163:0/3142944655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3eb01597f0 0x7f3eb01548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:05.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.393+0000 7f3eaffff700 1 --2- 172.21.15.163:0/3142944655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3eb015d230 0x7f3eb00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:05.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.393+0000 7f3eaf7fe700 1 -- 172.21.15.163:0/3142944655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3eb015d230 msgr2=0x7f3eb00b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:05.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.393+0000 7f3eaf7fe700 1 --2- 172.21.15.163:0/3142944655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3eb015d230 0x7f3eb00b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:05.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.393+0000 7f3eaf7fe700 1 -- 172.21.15.163:0/3142944655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3eb01597f0 msgr2=0x7f3eb01548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:05.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.393+0000 7f3eaf7fe700 1 --2- 172.21.15.163:0/3142944655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3eb01597f0 0x7f3eb01548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:05.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.393+0000 7f3eaf7fe700 1 -- 172.21.15.163:0/3142944655 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea8009420 con 0x7f3eb0159420 2023-10-09T06:32:05.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.393+0000 7f3eaeffd700 1 --2- 172.21.15.163:0/3142944655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3eb01597f0 0x7f3eb01548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:32:05.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.393+0000 7f3eaffff700 1 --2- 172.21.15.163:0/3142944655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3eb015d230 0x7f3eb00b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:32:05.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.393+0000 7f3eaf7fe700 1 --2- 172.21.15.163:0/3142944655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3eb0159420 0x7f3eb015af70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000cb10 tx=0x7f3ea0002820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:05.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.393+0000 7f3e977fe700 1 -- 172.21.15.163:0/3142944655 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea000a760 con 0x7f3eb0159420 2023-10-09T06:32:05.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.393+0000 7f3e977fe700 1 -- 172.21.15.163:0/3142944655 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea000a8c0 con 0x7f3eb0159420 2023-10-09T06:32:05.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.393+0000 7f3eb48ad700 1 -- 172.21.15.163:0/3142944655 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb00c6080 con 0x7f3eb0159420 2023-10-09T06:32:05.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.393+0000 7f3e977fe700 1 -- 172.21.15.163:0/3142944655 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea000ac60 con 0x7f3eb0159420 2023-10-09T06:32:05.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.393+0000 7f3eb48ad700 1 -- 172.21.15.163:0/3142944655 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb015c720 con 0x7f3eb0159420 2023-10-09T06:32:05.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.394+0000 7f3eb48ad700 1 -- 172.21.15.163:0/3142944655 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eb00b3400 con 0x7f3eb0159420 2023-10-09T06:32:05.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.394+0000 7f3e977fe700 1 -- 172.21.15.163:0/3142944655 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3ea0012630 con 0x7f3eb0159420 2023-10-09T06:32:05.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.395+0000 7f3e977fe700 1 --2- 172.21.15.163:0/3142944655 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3e9c066f30 0x7f3e9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:05.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.395+0000 7f3e977fe700 1 -- 172.21.15.163:0/3142944655 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3ea0084940 con 0x7f3eb0159420 2023-10-09T06:32:05.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.395+0000 7f3eaeffd700 1 --2- 172.21.15.163:0/3142944655 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3e9c066f30 0x7f3e9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:05.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.395+0000 7f3eaeffd700 1 --2- 172.21.15.163:0/3142944655 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3e9c066f30 0x7f3e9c0693e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40011e0 tx=0x7f3ea4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:05.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.399+0000 7f3e977fe700 1 -- 172.21.15.163:0/3142944655 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ea0051bb0 con 0x7f3eb0159420 2023-10-09T06:32:05.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.577+0000 7f3eb48ad700 1 -- 172.21.15.163:0/3142944655 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3eb004c3c0 con 0x7f3e9c066f30 2023-10-09T06:32:05.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.586+0000 7f3e977fe700 1 -- 172.21.15.163:0/3142944655 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278616 (secure 0 0 0) 0x7f3eb004c3c0 con 0x7f3e9c066f30 2023-10-09T06:32:05.589 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:05.592 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.589+0000 7f3eb48ad700 1 -- 172.21.15.163:0/3142944655 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3e9c066f30 msgr2=0x7f3e9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:05.592 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.589+0000 7f3eb48ad700 1 --2- 172.21.15.163:0/3142944655 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3e9c066f30 0x7f3e9c0693e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40011e0 tx=0x7f3ea4009040 comp rx=0 tx=0).stop 2023-10-09T06:32:05.592 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.589+0000 7f3eb48ad700 1 -- 172.21.15.163:0/3142944655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3eb0159420 msgr2=0x7f3eb015af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:05.592 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.589+0000 7f3eb48ad700 1 --2- 172.21.15.163:0/3142944655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3eb0159420 0x7f3eb015af70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000cb10 tx=0x7f3ea0002820 comp rx=0 tx=0).stop 2023-10-09T06:32:05.592 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.589+0000 7f3eb48ad700 1 -- 172.21.15.163:0/3142944655 shutdown_connections 2023-10-09T06:32:05.592 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.589+0000 7f3eb48ad700 1 --2- 172.21.15.163:0/3142944655 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3e9c066f30 0x7f3e9c0693e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:05.592 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.589+0000 7f3eb48ad700 1 --2- 172.21.15.163:0/3142944655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3eb0159420 0x7f3eb015af70 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:05.592 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.589+0000 7f3eb48ad700 1 --2- 172.21.15.163:0/3142944655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3eb015d230 0x7f3eb00b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:05.592 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.589+0000 7f3eb48ad700 1 --2- 172.21.15.163:0/3142944655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3eb01597f0 0x7f3eb01548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:05.593 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.589+0000 7f3eb48ad700 1 -- 172.21.15.163:0/3142944655 >> 172.21.15.163:0/3142944655 conn(0x7f3eb00d0660 msgr2=0x7f3eb00695e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:05.593 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.589+0000 7f3eb48ad700 1 -- 172.21.15.163:0/3142944655 shutdown_connections 2023-10-09T06:32:05.593 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.589+0000 7f3eb48ad700 1 -- 172.21.15.163:0/3142944655 wait complete. 2023-10-09T06:32:05.593 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:32:05.601 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2023-10-09T06:32:04.312673+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2383932,"kb_used_data":5628,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752590788,"statfs":{"total":773094113280,"available":770652966912,"internally_reserved":0,"allocated":5763072,"data_stored":3091859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.784371"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.534276+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:27:36.084920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.533419+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:12:02.253891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.534148+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:13:58.204471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.533401+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:50:59.843352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.530394+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:18:26.280667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.534148+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:38:45.645525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.533726+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:50:42.169512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.530561+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:13:51.635379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.536758+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:50:42.169512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.533027+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:33:49.674544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.536716+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:53:13.331862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.530127+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:05:30.361612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.534168+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:03:45.617105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.530103+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:37:38.387598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.532859+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:20:24.757239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.536684+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:50:59.843352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.532906+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:06:50.002887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.534173+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:34:11.248280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.532900+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:54:00.414345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.532597+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:55:32.071358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.533751+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:07:20.517549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.536298+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:55:32.071358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.532856+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:18:11.772295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.535335+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:28:24.618183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.532539+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:28:24.618183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.530109+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:37:54.586944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.532775+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:27:36.084920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.535221+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:18:26.280667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.530762+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:28:24.618183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.535235+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:13:51.635379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.532926+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:24:43.123753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.530388+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:25:30.966439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":187,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-09T06:31:57.512233+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:31:57.512233+0000","last_peered":"2023-10-09T06:31:57.512233+0000","last_clean":"2023-10-09T06:31:57.512233+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:31:57.512233+0000","last_undegraded":"2023-10-09T06:31:57.512233+0000","last_fullsized":"2023-10-09T06:31:57.512233+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298428,"kb_used_data":1140,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073412,"statfs":{"total":96636764160,"available":96331173888,"internally_reserved":0,"allocated":1167360,"data_stored":829984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.504}]},{"osd":1,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.55600000000000005}]},{"osd":2,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.438}]},{"osd":3,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.625}]},{"osd":4,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.65100000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.70499999999999996}]},{"osd":6,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.68100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297848,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073992,"statfs":{"total":96636764160,"available":96331767808,"internally_reserved":0,"allocated":573440,"data_stored":239616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.63500000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.57099999999999995}]},{"osd":2,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.58699999999999997}]},{"osd":3,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":4.7510000000000003,"5min":4.7510000000000003,"15min":4.7510000000000003},"last":0.72899999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.70299999999999996}]},{"osd":5,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.64500000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.60899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297848,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073992,"statfs":{"total":96636764160,"available":96331767808,"internally_reserved":0,"allocated":573440,"data_stored":239616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.31,"15min":0.31},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.33800000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.35899999999999999}]},{"osd":3,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":5.8529999999999998,"5min":5.8529999999999998,"15min":5.8529999999999998},"last":0.372}]},{"osd":4,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.498},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.47499999999999998}]},{"osd":5,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":5.9459999999999997,"5min":5.9459999999999997,"15min":5.9459999999999997},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.54200000000000004}]},{"osd":6,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":5.9160000000000004,"5min":5.9160000000000004,"15min":5.9160000000000004},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.433}]},{"osd":7,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":5.9729999999999999,"5min":5.9729999999999999,"15min":5.9729999999999999},"last":0.53200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298428,"kb_used_data":1140,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073412,"statfs":{"total":96636764160,"available":96331173888,"internally_reserved":0,"allocated":1167360,"data_stored":829984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.438,"5min":0.438,"15min":0.438},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.41299999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.436},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.45300000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.373},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.51300000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":5.484,"5min":5.484,"15min":5.484},"last":0.52600000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":5.5449999999999999,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.65100000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.61599999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":5.5800000000000001,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.63600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297836,"kb_used_data":548,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074004,"statfs":{"total":96636764160,"available":96331780096,"internally_reserved":0,"allocated":561152,"data_stored":233811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.318},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":7.2699999999999996,"5min":7.2699999999999996,"15min":7.2699999999999996},"last":0.432}]},{"osd":1,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":7.327,"5min":7.327,"15min":7.327},"last":0.38300000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.54000000000000004}]},{"osd":4,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":7.2389999999999999,"5min":7.2389999999999999,"15min":7.2389999999999999},"last":0.46700000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":10.890000000000001,"5min":10.890000000000001,"15min":10.890000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.58399999999999996}]},{"osd":6,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":7.2999999999999998,"5min":7.2999999999999998,"15min":7.2999999999999998},"last":0.499},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.60599999999999998}]},{"osd":7,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":7.1879999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.63400000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297848,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073992,"statfs":{"total":96636764160,"available":96331767808,"internally_reserved":0,"allocated":573440,"data_stored":239616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.38700000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.33400000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.29099999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.438}]},{"osd":5,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.53600000000000003}]},{"osd":6,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.503}]},{"osd":7,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.41099999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297848,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073992,"statfs":{"total":96636764160,"available":96331767808,"internally_reserved":0,"allocated":573440,"data_stored":239616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.56499999999999995}]},{"osd":1,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.749},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.76000000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.64500000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.59499999999999997}]},{"osd":5,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.371},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.54500000000000004}]},{"osd":6,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.41599999999999998}]},{"osd":7,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.441}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297848,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073992,"statfs":{"total":96636764160,"available":96331767808,"internally_reserved":0,"allocated":573440,"data_stored":239616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.38700000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.40999999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.54900000000000004}]},{"osd":3,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.439},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.45500000000000002}]},{"osd":4,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.56100000000000005}]},{"osd":6,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.57099999999999995}]},{"osd":7,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.61399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:32:05.608 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:32:05.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.755+0000 7feee6e12700 1 Processor -- start 2023-10-09T06:32:05.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.755+0000 7feee6e12700 1 -- start start 2023-10-09T06:32:05.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.756+0000 7feee6e12700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7feee006bcd0 0x7feee0140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:05.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.756+0000 7feee6e12700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feee00c9510 0x7feee0141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:05.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.756+0000 7feee6e12700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feee015c2a0 con 0x7feee006c0a0 2023-10-09T06:32:05.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.756+0000 7feee6e12700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feee015b690 con 0x7feee006bcd0 2023-10-09T06:32:05.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.756+0000 7feee6e12700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee015b970 con 0x7feee00c9510 2023-10-09T06:32:05.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.756+0000 7feee4bae700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7feee006bcd0 0x7feee0140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:05.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.756+0000 7feedffff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7feee006c0a0 0x7feee00c8e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:50868/0 (socket says 172.21.15.163:50868) 2023-10-09T06:32:05.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.756+0000 7feedffff700 1 -- 172.21.15.163:0/1084171946 learned_addr learned my addr 172.21.15.163:0/1084171946 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:05.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.756+0000 7feee4bae700 1 -- 172.21.15.163:0/1084171946 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feee00c9510 msgr2=0x7feee0141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:05.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.756+0000 7feee53af700 1 --2- 172.21.15.163:0/1084171946 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feee00c9510 0x7feee0141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:05.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.756+0000 7feedf7fe700 1 -- 172.21.15.163:0/1084171946 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3537224096 0 0) 0x7feee015c2a0 con 0x7feee006c0a0 2023-10-09T06:32:05.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.756+0000 7feee4bae700 1 --2- 172.21.15.163:0/1084171946 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feee00c9510 0x7feee0141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:05.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.756+0000 7feee4bae700 1 -- 172.21.15.163:0/1084171946 >> v1:172.21.15.163:6789/0 conn(0x7feee006c0a0 legacy=0x7feee00c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:05.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.756+0000 7feee4bae700 1 -- 172.21.15.163:0/1084171946 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee015bc50 con 0x7feee006bcd0 2023-10-09T06:32:05.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.756+0000 7feee53af700 1 --2- 172.21.15.163:0/1084171946 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feee00c9510 0x7feee0141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:32:05.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.756+0000 7feee4bae700 1 --2- 172.21.15.163:0/1084171946 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feee006bcd0 0x7feee0140df0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7feed40098e0 tx=0x7feed40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83107d9ad30cb024 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:05.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.756+0000 7feedf7fe700 1 -- 172.21.15.163:0/1084171946 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed4009e80 con 0x7feee006bcd0 2023-10-09T06:32:05.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.756+0000 7feedf7fe700 1 -- 172.21.15.163:0/1084171946 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed400b860 con 0x7feee006bcd0 2023-10-09T06:32:05.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.756+0000 7feedf7fe700 1 -- 172.21.15.163:0/1084171946 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed400bbe0 con 0x7feee006bcd0 2023-10-09T06:32:05.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.757+0000 7feee6e12700 1 -- 172.21.15.163:0/1084171946 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feee006bcd0 msgr2=0x7feee0140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:05.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.757+0000 7feee6e12700 1 --2- 172.21.15.163:0/1084171946 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feee006bcd0 0x7feee0140df0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7feed40098e0 tx=0x7feed40091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:05.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.757+0000 7feee6e12700 1 -- 172.21.15.163:0/1084171946 shutdown_connections 2023-10-09T06:32:05.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.757+0000 7feee6e12700 1 --2- 172.21.15.163:0/1084171946 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feee00c9510 0x7feee0141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:05.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.757+0000 7feee6e12700 1 --2- 172.21.15.163:0/1084171946 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feee006bcd0 0x7feee0140df0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:05.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.757+0000 7feee6e12700 1 -- 172.21.15.163:0/1084171946 >> 172.21.15.163:0/1084171946 conn(0x7feee00ce600 msgr2=0x7feee00cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:05.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.757+0000 7feee6e12700 1 -- 172.21.15.163:0/1084171946 shutdown_connections 2023-10-09T06:32:05.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.757+0000 7feee6e12700 1 -- 172.21.15.163:0/1084171946 wait complete. 2023-10-09T06:32:05.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.758+0000 7feee6e12700 1 Processor -- start 2023-10-09T06:32:05.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.758+0000 7feee6e12700 1 -- start start 2023-10-09T06:32:05.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.759+0000 7feee6e12700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feee006bcd0 0x7feee01571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:05.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.759+0000 7feee6e12700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feee006c0a0 0x7feee013cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:05.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.759+0000 7feee6e12700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feee00c9510 0x7feee013ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:05.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.759+0000 7feee6e12700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee0168bb0 con 0x7feee006bcd0 2023-10-09T06:32:05.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.759+0000 7feee6e12700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee01404c0 con 0x7feee00c9510 2023-10-09T06:32:05.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.759+0000 7feee6e12700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee0167ce0 con 0x7feee006c0a0 2023-10-09T06:32:05.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.759+0000 7feee4bae700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feee006bcd0 0x7feee01571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:05.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.759+0000 7feee4bae700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feee006bcd0 0x7feee01571d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58796/0 (socket says 172.21.15.163:58796) 2023-10-09T06:32:05.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.759+0000 7feee4bae700 1 -- 172.21.15.163:0/2658727210 learned_addr learned my addr 172.21.15.163:0/2658727210 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:05.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.759+0000 7feee4bae700 1 -- 172.21.15.163:0/2658727210 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feee006c0a0 msgr2=0x7feee013cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:05.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.759+0000 7feee4bae700 1 --2- 172.21.15.163:0/2658727210 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feee006c0a0 0x7feee013cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:05.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.759+0000 7feee4bae700 1 -- 172.21.15.163:0/2658727210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feee00c9510 msgr2=0x7feee013ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:05.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.759+0000 7feee4bae700 1 --2- 172.21.15.163:0/2658727210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feee00c9510 0x7feee013ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:05.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.759+0000 7feee4bae700 1 -- 172.21.15.163:0/2658727210 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed4009420 con 0x7feee006bcd0 2023-10-09T06:32:05.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.759+0000 7feee4bae700 1 --2- 172.21.15.163:0/2658727210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feee006bcd0 0x7feee01571d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7feed400b3c0 tx=0x7feed4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:05.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.760+0000 7feedcff9700 1 -- 172.21.15.163:0/2658727210 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed400baa0 con 0x7feee006bcd0 2023-10-09T06:32:05.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.760+0000 7feedcff9700 1 -- 172.21.15.163:0/2658727210 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed4025420 con 0x7feee006bcd0 2023-10-09T06:32:05.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.760+0000 7feedcff9700 1 -- 172.21.15.163:0/2658727210 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed40257e0 con 0x7feee006bcd0 2023-10-09T06:32:05.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.760+0000 7feee6e12700 1 -- 172.21.15.163:0/2658727210 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee0167fa0 con 0x7feee006bcd0 2023-10-09T06:32:05.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.760+0000 7feee6e12700 1 -- 172.21.15.163:0/2658727210 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee0168560 con 0x7feee006bcd0 2023-10-09T06:32:05.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.761+0000 7feee6e12700 1 -- 172.21.15.163:0/2658727210 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feee004c3c0 con 0x7feee006bcd0 2023-10-09T06:32:05.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.761+0000 7feedcff9700 1 -- 172.21.15.163:0/2658727210 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feed4023070 con 0x7feee006bcd0 2023-10-09T06:32:05.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.761+0000 7feedcff9700 1 --2- 172.21.15.163:0/2658727210 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7feec4066f30 0x7feec40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:05.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.761+0000 7feedcff9700 1 -- 172.21.15.163:0/2658727210 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7feed4085c60 con 0x7feee006bcd0 2023-10-09T06:32:05.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.762+0000 7feedffff700 1 --2- 172.21.15.163:0/2658727210 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7feec4066f30 0x7feec40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:05.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.762+0000 7feedffff700 1 --2- 172.21.15.163:0/2658727210 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7feec4066f30 0x7feec40693e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7feecc0011e0 tx=0x7feecc0079d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:05.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.765+0000 7feedcff9700 1 -- 172.21.15.163:0/2658727210 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feed4052ed0 con 0x7feee006bcd0 2023-10-09T06:32:05.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.944+0000 7feee6e12700 1 -- 172.21.15.163:0/2658727210 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feee006c9d0 con 0x7feec4066f30 2023-10-09T06:32:05.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.952+0000 7feedcff9700 1 -- 172.21.15.163:0/2658727210 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278616 (secure 0 0 0) 0x7feee006c9d0 con 0x7feec4066f30 2023-10-09T06:32:05.956 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:05.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.955+0000 7feee6e12700 1 -- 172.21.15.163:0/2658727210 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7feec4066f30 msgr2=0x7feec40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:05.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.955+0000 7feee6e12700 1 --2- 172.21.15.163:0/2658727210 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7feec4066f30 0x7feec40693e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7feecc0011e0 tx=0x7feecc0079d0 comp rx=0 tx=0).stop 2023-10-09T06:32:05.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.955+0000 7feee6e12700 1 -- 172.21.15.163:0/2658727210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feee006bcd0 msgr2=0x7feee01571d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:05.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.955+0000 7feee6e12700 1 --2- 172.21.15.163:0/2658727210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feee006bcd0 0x7feee01571d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7feed400b3c0 tx=0x7feed4017000 comp rx=0 tx=0).stop 2023-10-09T06:32:05.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.955+0000 7feee6e12700 1 -- 172.21.15.163:0/2658727210 shutdown_connections 2023-10-09T06:32:05.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.955+0000 7feee6e12700 1 --2- 172.21.15.163:0/2658727210 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7feec4066f30 0x7feec40693e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:05.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.955+0000 7feee6e12700 1 --2- 172.21.15.163:0/2658727210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feee006bcd0 0x7feee01571d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:05.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.955+0000 7feee6e12700 1 --2- 172.21.15.163:0/2658727210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feee00c9510 0x7feee013ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:05.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.955+0000 7feee6e12700 1 --2- 172.21.15.163:0/2658727210 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feee006c0a0 0x7feee013cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:05.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.955+0000 7feee6e12700 1 -- 172.21.15.163:0/2658727210 >> 172.21.15.163:0/2658727210 conn(0x7feee00ce600 msgr2=0x7feee00cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:05.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.955+0000 7feee6e12700 1 -- 172.21.15.163:0/2658727210 shutdown_connections 2023-10-09T06:32:05.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:05.955+0000 7feee6e12700 1 -- 172.21.15.163:0/2658727210 wait complete. 2023-10-09T06:32:05.959 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:32:05.968 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2023-10-09T06:32:04.312673+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2383932,"kb_used_data":5628,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752590788,"statfs":{"total":773094113280,"available":770652966912,"internally_reserved":0,"allocated":5763072,"data_stored":3091859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.784371"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.534276+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:27:36.084920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.533419+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:12:02.253891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.534148+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:13:58.204471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.533401+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:50:59.843352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.530394+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:18:26.280667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.534148+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:38:45.645525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.533726+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:50:42.169512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.530561+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:13:51.635379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.536758+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:50:42.169512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.533027+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:33:49.674544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.536716+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:53:13.331862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.530127+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:05:30.361612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.534168+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:03:45.617105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.530103+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:37:38.387598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.532859+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:20:24.757239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.536684+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:50:59.843352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.532906+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:06:50.002887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.534173+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:34:11.248280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.532900+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:54:00.414345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.532597+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:55:32.071358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.533751+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:07:20.517549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.536298+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:55:32.071358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.532856+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:18:11.772295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.535335+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:28:24.618183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.532539+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:28:24.618183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.530109+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:37:54.586944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.532775+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:27:36.084920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.535221+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:18:26.280667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.530762+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:28:24.618183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.535235+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:13:51.635379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.532926+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:24:43.123753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.530388+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:25:30.966439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":187,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-09T06:31:57.512233+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:31:57.512233+0000","last_peered":"2023-10-09T06:31:57.512233+0000","last_clean":"2023-10-09T06:31:57.512233+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:31:57.512233+0000","last_undegraded":"2023-10-09T06:31:57.512233+0000","last_fullsized":"2023-10-09T06:31:57.512233+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298428,"kb_used_data":1140,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073412,"statfs":{"total":96636764160,"available":96331173888,"internally_reserved":0,"allocated":1167360,"data_stored":829984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.504}]},{"osd":1,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.55600000000000005}]},{"osd":2,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.438}]},{"osd":3,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.625}]},{"osd":4,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.65100000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.70499999999999996}]},{"osd":6,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.68100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297848,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073992,"statfs":{"total":96636764160,"available":96331767808,"internally_reserved":0,"allocated":573440,"data_stored":239616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.63500000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.57099999999999995}]},{"osd":2,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.58699999999999997}]},{"osd":3,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":4.7510000000000003,"5min":4.7510000000000003,"15min":4.7510000000000003},"last":0.72899999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.70299999999999996}]},{"osd":5,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.64500000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.60899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297848,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073992,"statfs":{"total":96636764160,"available":96331767808,"internally_reserved":0,"allocated":573440,"data_stored":239616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.31,"15min":0.31},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.33800000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.35899999999999999}]},{"osd":3,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":5.8529999999999998,"5min":5.8529999999999998,"15min":5.8529999999999998},"last":0.372}]},{"osd":4,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.498},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.47499999999999998}]},{"osd":5,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":5.9459999999999997,"5min":5.9459999999999997,"15min":5.9459999999999997},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.54200000000000004}]},{"osd":6,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":5.9160000000000004,"5min":5.9160000000000004,"15min":5.9160000000000004},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.433}]},{"osd":7,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":5.9729999999999999,"5min":5.9729999999999999,"15min":5.9729999999999999},"last":0.53200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298428,"kb_used_data":1140,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073412,"statfs":{"total":96636764160,"available":96331173888,"internally_reserved":0,"allocated":1167360,"data_stored":829984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.438,"5min":0.438,"15min":0.438},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.41299999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.436},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.45300000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.373},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.51300000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":5.484,"5min":5.484,"15min":5.484},"last":0.52600000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":5.5449999999999999,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.65100000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.61599999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":5.5800000000000001,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.63600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297836,"kb_used_data":548,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074004,"statfs":{"total":96636764160,"available":96331780096,"internally_reserved":0,"allocated":561152,"data_stored":233811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.318},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":7.2699999999999996,"5min":7.2699999999999996,"15min":7.2699999999999996},"last":0.432}]},{"osd":1,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":7.327,"5min":7.327,"15min":7.327},"last":0.38300000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.54000000000000004}]},{"osd":4,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":7.2389999999999999,"5min":7.2389999999999999,"15min":7.2389999999999999},"last":0.46700000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":10.890000000000001,"5min":10.890000000000001,"15min":10.890000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.58399999999999996}]},{"osd":6,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":7.2999999999999998,"5min":7.2999999999999998,"15min":7.2999999999999998},"last":0.499},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.60599999999999998}]},{"osd":7,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":7.1879999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.63400000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297848,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073992,"statfs":{"total":96636764160,"available":96331767808,"internally_reserved":0,"allocated":573440,"data_stored":239616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.38700000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.33400000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.29099999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.438}]},{"osd":5,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.53600000000000003}]},{"osd":6,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.503}]},{"osd":7,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.41099999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297848,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073992,"statfs":{"total":96636764160,"available":96331767808,"internally_reserved":0,"allocated":573440,"data_stored":239616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.56499999999999995}]},{"osd":1,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.749},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.76000000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.64500000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.59499999999999997}]},{"osd":5,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.371},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.54500000000000004}]},{"osd":6,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.41599999999999998}]},{"osd":7,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.441}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297848,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073992,"statfs":{"total":96636764160,"available":96331767808,"internally_reserved":0,"allocated":573440,"data_stored":239616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.38700000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.40999999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.54900000000000004}]},{"osd":3,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.439},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.45500000000000002}]},{"osd":4,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.56100000000000005}]},{"osd":6,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.57099999999999995}]},{"osd":7,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.61399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:32:05.974 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:32:06.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.122+0000 7f3a21b28700 1 Processor -- start 2023-10-09T06:32:06.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.122+0000 7f3a21b28700 1 -- start start 2023-10-09T06:32:06.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.123+0000 7f3a21b28700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a1c159890 0x7f3a1c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:06.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.123+0000 7f3a21b28700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3a1c159c60 0x7f3a1c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:06.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.123+0000 7f3a21b28700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a1c15a160 con 0x7f3a1c15db10 2023-10-09T06:32:06.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.123+0000 7f3a21b28700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a1c0d13a0 con 0x7f3a1c159c60 2023-10-09T06:32:06.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.123+0000 7f3a21b28700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c15e140 con 0x7f3a1c159890 2023-10-09T06:32:06.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.123+0000 7f3a1affd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3a1c159c60 0x7f3a1c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:06.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.123+0000 7f3a1bfff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f3a1c15db10 0x7f3a1c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:50880/0 (socket says 172.21.15.163:50880) 2023-10-09T06:32:06.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.123+0000 7f3a1affd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3a1c159c60 0x7f3a1c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58808/0 (socket says 172.21.15.163:58808) 2023-10-09T06:32:06.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.123+0000 7f3a1bfff700 1 -- 172.21.15.163:0/852492230 learned_addr learned my addr 172.21.15.163:0/852492230 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:06.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.123+0000 7f3a1affd700 1 -- 172.21.15.163:0/852492230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a1c159890 msgr2=0x7f3a1c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:06.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.123+0000 7f3a1affd700 1 --2- 172.21.15.163:0/852492230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a1c159890 0x7f3a1c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:06.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.123+0000 7f3a1affd700 1 -- 172.21.15.163:0/852492230 >> v1:172.21.15.163:6789/0 conn(0x7f3a1c15db10 legacy=0x7f3a1c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:06.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.124+0000 7f3a1affd700 1 -- 172.21.15.163:0/852492230 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1c15a3d0 con 0x7f3a1c159c60 2023-10-09T06:32:06.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.123+0000 7f3a1a7fc700 1 -- 172.21.15.163:0/852492230 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1299470006 0 0) 0x7f3a1c15a160 con 0x7f3a1c15db10 2023-10-09T06:32:06.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.124+0000 7f3a1affd700 1 --2- 172.21.15.163:0/852492230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a1c159c60 0x7f3a1c15d5d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3a100098e0 tx=0x7f3a100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da96fe4c73ca1852 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:06.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.124+0000 7f3a1a7fc700 1 -- 172.21.15.163:0/852492230 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a10009e80 con 0x7f3a1c159c60 2023-10-09T06:32:06.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.124+0000 7f3a1a7fc700 1 -- 172.21.15.163:0/852492230 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1000b860 con 0x7f3a1c159c60 2023-10-09T06:32:06.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.124+0000 7f3a1a7fc700 1 -- 172.21.15.163:0/852492230 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1000bbe0 con 0x7f3a1c159c60 2023-10-09T06:32:06.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.124+0000 7f3a21b28700 1 -- 172.21.15.163:0/852492230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a1c159c60 msgr2=0x7f3a1c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:06.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.124+0000 7f3a21b28700 1 --2- 172.21.15.163:0/852492230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a1c159c60 0x7f3a1c15d5d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3a100098e0 tx=0x7f3a100091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:06.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.124+0000 7f3a21b28700 1 -- 172.21.15.163:0/852492230 shutdown_connections 2023-10-09T06:32:06.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.124+0000 7f3a21b28700 1 --2- 172.21.15.163:0/852492230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a1c159890 0x7f3a1c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:06.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.124+0000 7f3a21b28700 1 --2- 172.21.15.163:0/852492230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a1c159c60 0x7f3a1c15d5d0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:06.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.124+0000 7f3a21b28700 1 -- 172.21.15.163:0/852492230 >> 172.21.15.163:0/852492230 conn(0x7f3a1c068e20 msgr2=0x7f3a1c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:06.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.124+0000 7f3a21b28700 1 -- 172.21.15.163:0/852492230 shutdown_connections 2023-10-09T06:32:06.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.124+0000 7f3a21b28700 1 -- 172.21.15.163:0/852492230 wait complete. 2023-10-09T06:32:06.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.126+0000 7f3a21b28700 1 Processor -- start 2023-10-09T06:32:06.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.126+0000 7f3a21b28700 1 -- start start 2023-10-09T06:32:06.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.126+0000 7f3a21b28700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a1c159890 0x7f3a1c1571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:06.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.126+0000 7f3a21b28700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a1c159c60 0x7f3a1c14af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:06.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.126+0000 7f3a21b28700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a1c15db10 0x7f3a1c14b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:06.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.126+0000 7f3a21b28700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c1521e0 con 0x7f3a1c15db10 2023-10-09T06:32:06.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.126+0000 7f3a21b28700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c1524c0 con 0x7f3a1c159890 2023-10-09T06:32:06.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.126+0000 7f3a21b28700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c1527a0 con 0x7f3a1c159c60 2023-10-09T06:32:06.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.126+0000 7f3a1bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a1c15db10 0x7f3a1c14b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:06.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.126+0000 7f3a1bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a1c15db10 0x7f3a1c14b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58814/0 (socket says 172.21.15.163:58814) 2023-10-09T06:32:06.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.126+0000 7f3a1bfff700 1 -- 172.21.15.163:0/3305917641 learned_addr learned my addr 172.21.15.163:0/3305917641 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:06.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.127+0000 7f3a1affd700 1 --2- 172.21.15.163:0/3305917641 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a1c159c60 0x7f3a1c14af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:06.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.127+0000 7f3a1b7fe700 1 --2- 172.21.15.163:0/3305917641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a1c159890 0x7f3a1c1571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:06.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.127+0000 7f3a1bfff700 1 -- 172.21.15.163:0/3305917641 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a1c159c60 msgr2=0x7f3a1c14af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:06.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.127+0000 7f3a1bfff700 1 --2- 172.21.15.163:0/3305917641 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a1c159c60 0x7f3a1c14af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:06.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.127+0000 7f3a1bfff700 1 -- 172.21.15.163:0/3305917641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a1c159890 msgr2=0x7f3a1c1571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:06.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.127+0000 7f3a1bfff700 1 --2- 172.21.15.163:0/3305917641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a1c159890 0x7f3a1c1571c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:06.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.127+0000 7f3a1bfff700 1 -- 172.21.15.163:0/3305917641 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a10009420 con 0x7f3a1c15db10 2023-10-09T06:32:06.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.127+0000 7f3a1bfff700 1 --2- 172.21.15.163:0/3305917641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a1c15db10 0x7f3a1c14b490 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3a1400cbf0 tx=0x7f3a140019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:06.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.127+0000 7f3a037fe700 1 -- 172.21.15.163:0/3305917641 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1400a9c0 con 0x7f3a1c15db10 2023-10-09T06:32:06.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.127+0000 7f3a037fe700 1 -- 172.21.15.163:0/3305917641 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1400ab20 con 0x7f3a1c15db10 2023-10-09T06:32:06.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.127+0000 7f3a21b28700 1 -- 172.21.15.163:0/3305917641 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a1c152a80 con 0x7f3a1c15db10 2023-10-09T06:32:06.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.127+0000 7f3a037fe700 1 -- 172.21.15.163:0/3305917641 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a140070f0 con 0x7f3a1c15db10 2023-10-09T06:32:06.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.127+0000 7f3a21b28700 1 -- 172.21.15.163:0/3305917641 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1c157700 con 0x7f3a1c15db10 2023-10-09T06:32:06.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.128+0000 7f3a21b28700 1 -- 172.21.15.163:0/3305917641 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a1c152fa0 con 0x7f3a1c15db10 2023-10-09T06:32:06.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.128+0000 7f3a037fe700 1 -- 172.21.15.163:0/3305917641 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3a14007520 con 0x7f3a1c15db10 2023-10-09T06:32:06.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.129+0000 7f3a037fe700 1 --2- 172.21.15.163:0/3305917641 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3a08066f30 0x7f3a080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:06.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.129+0000 7f3a037fe700 1 -- 172.21.15.163:0/3305917641 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3a14084910 con 0x7f3a1c15db10 2023-10-09T06:32:06.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.129+0000 7f3a1b7fe700 1 --2- 172.21.15.163:0/3305917641 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3a08066f30 0x7f3a080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:06.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.129+0000 7f3a1b7fe700 1 --2- 172.21.15.163:0/3305917641 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3a08066f30 0x7f3a080693e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c006fd0 tx=0x7f3a0c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:06.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.133+0000 7f3a037fe700 1 -- 172.21.15.163:0/3305917641 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3a1c152fa0 con 0x7f3a1c15db10 2023-10-09T06:32:06.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.310+0000 7f3a21b28700 1 -- 172.21.15.163:0/3305917641 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3a1c152fa0 con 0x7f3a08066f30 2023-10-09T06:32:06.319 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.318+0000 7f3a037fe700 1 -- 172.21.15.163:0/3305917641 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278616 (secure 0 0 0) 0x7f3a1c152fa0 con 0x7f3a08066f30 2023-10-09T06:32:06.325 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:06.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.321+0000 7f3a21b28700 1 -- 172.21.15.163:0/3305917641 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3a08066f30 msgr2=0x7f3a080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:06.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.321+0000 7f3a21b28700 1 --2- 172.21.15.163:0/3305917641 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3a08066f30 0x7f3a080693e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c006fd0 tx=0x7f3a0c008040 comp rx=0 tx=0).stop 2023-10-09T06:32:06.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.321+0000 7f3a21b28700 1 -- 172.21.15.163:0/3305917641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a1c15db10 msgr2=0x7f3a1c14b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:06.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.321+0000 7f3a21b28700 1 --2- 172.21.15.163:0/3305917641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a1c15db10 0x7f3a1c14b490 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3a1400cbf0 tx=0x7f3a140019b0 comp rx=0 tx=0).stop 2023-10-09T06:32:06.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.321+0000 7f3a21b28700 1 -- 172.21.15.163:0/3305917641 shutdown_connections 2023-10-09T06:32:06.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.321+0000 7f3a21b28700 1 --2- 172.21.15.163:0/3305917641 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3a08066f30 0x7f3a080693e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:06.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.321+0000 7f3a21b28700 1 --2- 172.21.15.163:0/3305917641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a1c15db10 0x7f3a1c14b490 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:06.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.321+0000 7f3a21b28700 1 --2- 172.21.15.163:0/3305917641 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a1c159890 0x7f3a1c1571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:06.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.321+0000 7f3a21b28700 1 --2- 172.21.15.163:0/3305917641 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a1c159c60 0x7f3a1c14af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:06.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.321+0000 7f3a21b28700 1 -- 172.21.15.163:0/3305917641 >> 172.21.15.163:0/3305917641 conn(0x7f3a1c068e20 msgr2=0x7f3a1c158880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:06.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.321+0000 7f3a21b28700 1 -- 172.21.15.163:0/3305917641 shutdown_connections 2023-10-09T06:32:06.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.321+0000 7f3a21b28700 1 -- 172.21.15.163:0/3305917641 wait complete. 2023-10-09T06:32:06.329 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:32:06.334 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2023-10-09T06:32:04.312673+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2383932,"kb_used_data":5628,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752590788,"statfs":{"total":773094113280,"available":770652966912,"internally_reserved":0,"allocated":5763072,"data_stored":3091859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.784371"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.534276+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:27:36.084920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.533419+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:12:02.253891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.534148+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:13:58.204471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.533401+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:50:59.843352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.530394+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:18:26.280667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.534148+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:38:45.645525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.533726+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:50:42.169512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.530561+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:13:51.635379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.536758+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:50:42.169512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.533027+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:33:49.674544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.536716+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:53:13.331862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.530127+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:05:30.361612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.534168+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:03:45.617105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.530103+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:37:38.387598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.532859+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:20:24.757239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.536684+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:50:59.843352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.532906+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:06:50.002887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.534173+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:34:11.248280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.532900+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:54:00.414345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.532597+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:55:32.071358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.533751+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:07:20.517549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.536298+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:55:32.071358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.532856+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:18:11.772295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.535335+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:28:24.618183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.532539+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:28:24.618183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.530109+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:37:54.586944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.532775+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:27:36.084920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.535221+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:18:26.280667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.530762+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:28:24.618183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.535235+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:13:51.635379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.532926+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:24:43.123753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:00.527624+0000","last_change":"2023-10-09T06:32:00.527624+0000","last_active":"2023-10-09T06:32:00.527624+0000","last_peered":"2023-10-09T06:32:00.527624+0000","last_clean":"2023-10-09T06:32:00.527624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:00.527624+0000","last_undegraded":"2023-10-09T06:32:00.527624+0000","last_fullsized":"2023-10-09T06:32:00.527624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:31:59.518607+0000","last_change":"2023-10-09T06:31:59.530388+0000","last_active":"2023-10-09T06:31:59.518607+0000","last_peered":"2023-10-09T06:31:59.518607+0000","last_clean":"2023-10-09T06:31:59.518607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:31:59.518607+0000","last_undegraded":"2023-10-09T06:31:59.518607+0000","last_fullsized":"2023-10-09T06:31:59.518607+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:25:30.966439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":187,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-09T06:31:57.512233+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:31:57.512233+0000","last_peered":"2023-10-09T06:31:57.512233+0000","last_clean":"2023-10-09T06:31:57.512233+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:31:57.512233+0000","last_undegraded":"2023-10-09T06:31:57.512233+0000","last_fullsized":"2023-10-09T06:31:57.512233+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298428,"kb_used_data":1140,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073412,"statfs":{"total":96636764160,"available":96331173888,"internally_reserved":0,"allocated":1167360,"data_stored":829984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.504}]},{"osd":1,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.55600000000000005}]},{"osd":2,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.438}]},{"osd":3,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.625}]},{"osd":4,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.65100000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.70499999999999996}]},{"osd":6,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.68100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297848,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073992,"statfs":{"total":96636764160,"available":96331767808,"internally_reserved":0,"allocated":573440,"data_stored":239616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.63500000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.57099999999999995}]},{"osd":2,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.58699999999999997}]},{"osd":3,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":4.7510000000000003,"5min":4.7510000000000003,"15min":4.7510000000000003},"last":0.72899999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.70299999999999996}]},{"osd":5,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.64500000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.60899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297848,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073992,"statfs":{"total":96636764160,"available":96331767808,"internally_reserved":0,"allocated":573440,"data_stored":239616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.31,"15min":0.31},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.33800000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.35899999999999999}]},{"osd":3,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":5.8529999999999998,"5min":5.8529999999999998,"15min":5.8529999999999998},"last":0.372}]},{"osd":4,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.498},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.47499999999999998}]},{"osd":5,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":5.9459999999999997,"5min":5.9459999999999997,"15min":5.9459999999999997},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.54200000000000004}]},{"osd":6,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":5.9160000000000004,"5min":5.9160000000000004,"15min":5.9160000000000004},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.433}]},{"osd":7,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":5.9729999999999999,"5min":5.9729999999999999,"15min":5.9729999999999999},"last":0.53200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298428,"kb_used_data":1140,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073412,"statfs":{"total":96636764160,"available":96331173888,"internally_reserved":0,"allocated":1167360,"data_stored":829984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.438,"5min":0.438,"15min":0.438},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.41299999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.436},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.45300000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.373},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.51300000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":5.484,"5min":5.484,"15min":5.484},"last":0.52600000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":5.5449999999999999,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.65100000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.61599999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":5.5800000000000001,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.63600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297836,"kb_used_data":548,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074004,"statfs":{"total":96636764160,"available":96331780096,"internally_reserved":0,"allocated":561152,"data_stored":233811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.318},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":7.2699999999999996,"5min":7.2699999999999996,"15min":7.2699999999999996},"last":0.432}]},{"osd":1,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":7.327,"5min":7.327,"15min":7.327},"last":0.38300000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.54000000000000004}]},{"osd":4,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":7.2389999999999999,"5min":7.2389999999999999,"15min":7.2389999999999999},"last":0.46700000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":10.890000000000001,"5min":10.890000000000001,"15min":10.890000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.58399999999999996}]},{"osd":6,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":7.2999999999999998,"5min":7.2999999999999998,"15min":7.2999999999999998},"last":0.499},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.60599999999999998}]},{"osd":7,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":7.1879999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.63400000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297848,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073992,"statfs":{"total":96636764160,"available":96331767808,"internally_reserved":0,"allocated":573440,"data_stored":239616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.38700000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.33400000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.29099999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.438}]},{"osd":5,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.53600000000000003}]},{"osd":6,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.503}]},{"osd":7,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.41099999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297848,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073992,"statfs":{"total":96636764160,"available":96331767808,"internally_reserved":0,"allocated":573440,"data_stored":239616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.56499999999999995}]},{"osd":1,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.749},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.76000000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.64500000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.59499999999999997}]},{"osd":5,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.371},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.54500000000000004}]},{"osd":6,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.41599999999999998}]},{"osd":7,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.441}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297848,"kb_used_data":560,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073992,"statfs":{"total":96636764160,"available":96331767808,"internally_reserved":0,"allocated":573440,"data_stored":239616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.38700000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.40999999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.54900000000000004}]},{"osd":3,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.439},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.45500000000000002}]},{"osd":4,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.56100000000000005}]},{"osd":6,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.57099999999999995}]},{"osd":7,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.61399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:32:06.340 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3f is not active+clean 2023-10-09T06:32:06.340 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.340 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3e is not active+clean 2023-10-09T06:32:06.340 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.340 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3d is not active+clean 2023-10-09T06:32:06.340 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.340 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3c is not active+clean 2023-10-09T06:32:06.340 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.341 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3b is not active+clean 2023-10-09T06:32:06.341 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.341 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3a is not active+clean 2023-10-09T06:32:06.341 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.341 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.39 is not active+clean 2023-10-09T06:32:06.341 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.341 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.38 is not active+clean 2023-10-09T06:32:06.341 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.342 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.37 is not active+clean 2023-10-09T06:32:06.342 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.342 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.36 is not active+clean 2023-10-09T06:32:06.342 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.342 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.35 is not active+clean 2023-10-09T06:32:06.342 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.342 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.34 is not active+clean 2023-10-09T06:32:06.342 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.342 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.33 is not active+clean 2023-10-09T06:32:06.343 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.343 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.32 is not active+clean 2023-10-09T06:32:06.343 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.343 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.31 is not active+clean 2023-10-09T06:32:06.343 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.343 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.30 is not active+clean 2023-10-09T06:32:06.343 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.343 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2f is not active+clean 2023-10-09T06:32:06.343 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.344 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2e is not active+clean 2023-10-09T06:32:06.344 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.344 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2d is not active+clean 2023-10-09T06:32:06.344 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.344 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2c is not active+clean 2023-10-09T06:32:06.344 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.344 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2b is not active+clean 2023-10-09T06:32:06.344 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.344 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2a is not active+clean 2023-10-09T06:32:06.344 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.345 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.29 is not active+clean 2023-10-09T06:32:06.345 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.345 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.28 is not active+clean 2023-10-09T06:32:06.345 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.345 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.27 is not active+clean 2023-10-09T06:32:06.345 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.345 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.26 is not active+clean 2023-10-09T06:32:06.345 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.345 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.25 is not active+clean 2023-10-09T06:32:06.346 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.346 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.24 is not active+clean 2023-10-09T06:32:06.346 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.346 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.23 is not active+clean 2023-10-09T06:32:06.346 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.346 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.22 is not active+clean 2023-10-09T06:32:06.346 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.346 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.21 is not active+clean 2023-10-09T06:32:06.346 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.346 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.20 is not active+clean 2023-10-09T06:32:06.347 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.347 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1e is not active+clean 2023-10-09T06:32:06.347 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.347 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1f is not active+clean 2023-10-09T06:32:06.347 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.534276+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T07:27:36.084920+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-09T06:32:06.347 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1f is not active+clean 2023-10-09T06:32:06.347 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.348 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1e is not active+clean 2023-10-09T06:32:06.348 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.533419+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T16:12:02.253891+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-09T06:32:06.348 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1c is not active+clean 2023-10-09T06:32:06.348 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.348 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1d is not active+clean 2023-10-09T06:32:06.348 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.534148+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T15:13:58.204471+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-09T06:32:06.348 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1d is not active+clean 2023-10-09T06:32:06.348 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1c is not active+clean 2023-10-09T06:32:06.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.533401+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T12:50:59.843352+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-09T06:32:06.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1a is not active+clean 2023-10-09T06:32:06.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1b is not active+clean 2023-10-09T06:32:06.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.530394+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T15:18:26.280667+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-09T06:32:06.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1b is not active+clean 2023-10-09T06:32:06.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1a is not active+clean 2023-10-09T06:32:06.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.534148+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T17:38:45.645525+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-09T06:32:06.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.18 is not active+clean 2023-10-09T06:32:06.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.19 is not active+clean 2023-10-09T06:32:06.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.533726+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T16:50:42.169512+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-09T06:32:06.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.19 is not active+clean 2023-10-09T06:32:06.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.18 is not active+clean 2023-10-09T06:32:06.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.530561+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T07:13:51.635379+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-09T06:32:06.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.16 is not active+clean 2023-10-09T06:32:06.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.17 is not active+clean 2023-10-09T06:32:06.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.536758+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T16:50:42.169512+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-09T06:32:06.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.17 is not active+clean 2023-10-09T06:32:06.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2023-10-09T06:32:06.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.533027+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T14:33:49.674544+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-09T06:32:06.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.14 is not active+clean 2023-10-09T06:32:06.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.15 is not active+clean 2023-10-09T06:32:06.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.536716+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T14:53:13.331862+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-09T06:32:06.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.15 is not active+clean 2023-10-09T06:32:06.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.14 is not active+clean 2023-10-09T06:32:06.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.530127+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T14:05:30.361612+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-09T06:32:06.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.12 is not active+clean 2023-10-09T06:32:06.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2023-10-09T06:32:06.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.534168+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T10:03:45.617105+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-09T06:32:06.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.13 is not active+clean 2023-10-09T06:32:06.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.12 is not active+clean 2023-10-09T06:32:06.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.530103+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T13:37:38.387598+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-09T06:32:06.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.10 is not active+clean 2023-10-09T06:32:06.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.11 is not active+clean 2023-10-09T06:32:06.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.532859+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T11:20:24.757239+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-09T06:32:06.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.11 is not active+clean 2023-10-09T06:32:06.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.355 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.10 is not active+clean 2023-10-09T06:32:06.355 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.536684+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T12:50:59.843352+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-09T06:32:06.355 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.e is not active+clean 2023-10-09T06:32:06.355 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.355 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.f is not active+clean 2023-10-09T06:32:06.355 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.532906+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T17:06:50.002887+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-09T06:32:06.355 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.f is not active+clean 2023-10-09T06:32:06.356 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.356 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.e is not active+clean 2023-10-09T06:32:06.356 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.534173+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T17:34:11.248280+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-09T06:32:06.356 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.c is not active+clean 2023-10-09T06:32:06.356 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.356 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.d is not active+clean 2023-10-09T06:32:06.356 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.532900+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T09:54:00.414345+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-09T06:32:06.357 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.d is not active+clean 2023-10-09T06:32:06.357 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.357 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.c is not active+clean 2023-10-09T06:32:06.357 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.532597+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T11:55:32.071358+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-09T06:32:06.357 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.a is not active+clean 2023-10-09T06:32:06.357 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.357 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.b is not active+clean 2023-10-09T06:32:06.357 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.533751+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T10:07:20.517549+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-09T06:32:06.358 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.b is not active+clean 2023-10-09T06:32:06.358 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.358 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.a is not active+clean 2023-10-09T06:32:06.358 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.536298+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T11:55:32.071358+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-09T06:32:06.358 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.9 is not active+clean 2023-10-09T06:32:06.359 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.359 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.8 is not active+clean 2023-10-09T06:32:06.359 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.532856+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T10:18:11.772295+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-09T06:32:06.359 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.6 is not active+clean 2023-10-09T06:32:06.359 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.359 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.7 is not active+clean 2023-10-09T06:32:06.360 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.535335+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T18:28:24.618183+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-09T06:32:06.360 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.7 is not active+clean 2023-10-09T06:32:06.360 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.360 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.6 is not active+clean 2023-10-09T06:32:06.360 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.532539+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T18:28:24.618183+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-09T06:32:06.361 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.4 is not active+clean 2023-10-09T06:32:06.361 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.361 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.5 is not active+clean 2023-10-09T06:32:06.361 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.530109+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T15:37:54.586944+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-09T06:32:06.361 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.5 is not active+clean 2023-10-09T06:32:06.361 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.362 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.4 is not active+clean 2023-10-09T06:32:06.362 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.532775+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T07:27:36.084920+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-09T06:32:06.362 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2 is not active+clean 2023-10-09T06:32:06.362 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.362 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3 is not active+clean 2023-10-09T06:32:06.362 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.535221+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T15:18:26.280667+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-09T06:32:06.362 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3 is not active+clean 2023-10-09T06:32:06.363 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.363 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2 is not active+clean 2023-10-09T06:32:06.363 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.530762+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T18:28:24.618183+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-09T06:32:06.363 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.0 is not active+clean 2023-10-09T06:32:06.363 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.363 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1 is not active+clean 2023-10-09T06:32:06.363 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.535235+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T07:13:51.635379+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-09T06:32:06.363 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1 is not active+clean 2023-10-09T06:32:06.363 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.364 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2023-10-09T06:32:06.364 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.532926+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T16:24:43.123753+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-09T06:32:06.364 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.8 is not active+clean 2023-10-09T06:32:06.364 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:00.527624+0000', 'last_change': '2023-10-09T06:32:00.527624+0000', 'last_active': '2023-10-09T06:32:00.527624+0000', 'last_peered': '2023-10-09T06:32:00.527624+0000', 'last_clean': '2023-10-09T06:32:00.527624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:00.527624+0000', 'last_undegraded': '2023-10-09T06:32:00.527624+0000', 'last_fullsized': '2023-10-09T06:32:00.527624+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:00.527624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:06.364 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.9 is not active+clean 2023-10-09T06:32:06.364 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:31:59.518607+0000', 'last_change': '2023-10-09T06:31:59.530388+0000', 'last_active': '2023-10-09T06:31:59.518607+0000', 'last_peered': '2023-10-09T06:31:59.518607+0000', 'last_clean': '2023-10-09T06:31:59.518607+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:31:59.518607+0000', 'last_undegraded': '2023-10-09T06:31:59.518607+0000', 'last_fullsized': '2023-10-09T06:31:59.518607+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'last_clean_scrub_stamp': '2023-10-09T06:31:59.518607+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T18:25:30.966439+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-09T06:32:06.365 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-09T06:32:06.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.488+0000 7f751ec78700 1 Processor -- start 2023-10-09T06:32:06.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.489+0000 7f751ec78700 1 -- start start 2023-10-09T06:32:06.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.489+0000 7f751ec78700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f75181563e0 0x7f75181567b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:06.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.489+0000 7f751ec78700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75180c9350 0x7f751806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:06.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.489+0000 7f751ec78700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7518157780 con 0x7f75180c97c0 2023-10-09T06:32:06.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.489+0000 7f751ec78700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7518156e50 con 0x7f75181563e0 2023-10-09T06:32:06.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.489+0000 7f751ec78700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7518157130 con 0x7f75180c9350 2023-10-09T06:32:06.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.490+0000 7f751e477700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f75180c97c0 0x7f751806d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:50896/0 (socket says 172.21.15.163:50896) 2023-10-09T06:32:06.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.490+0000 7f751e477700 1 -- 172.21.15.163:0/3031185901 learned_addr learned my addr 172.21.15.163:0/3031185901 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:06.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.490+0000 7f751dc76700 1 --2- 172.21.15.163:0/3031185901 >> v2:172.21.15.181:3300/0 conn(0x7f75181563e0 0x7f75181567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:06.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.490+0000 7f751d475700 1 --2- 172.21.15.163:0/3031185901 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75180c9350 0x7f751806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:06.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.490+0000 7f751cc74700 1 -- 172.21.15.163:0/3031185901 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2739709531 0 0) 0x7f7518157780 con 0x7f75180c97c0 2023-10-09T06:32:06.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.490+0000 7f751cc74700 1 -- 172.21.15.163:0/3031185901 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7504003570 con 0x7f75180c97c0 2023-10-09T06:32:06.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.490+0000 7f751cc74700 1 -- 172.21.15.163:0/3031185901 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2984875711 0 0) 0x7f7504003570 con 0x7f75180c97c0 2023-10-09T06:32:06.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.490+0000 7f751cc74700 1 -- 172.21.15.163:0/3031185901 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75180c9350 msgr2=0x7f751806e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:06.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.490+0000 7f751cc74700 1 --2- 172.21.15.163:0/3031185901 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75180c9350 0x7f751806e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:06.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.490+0000 7f751cc74700 1 -- 172.21.15.163:0/3031185901 >> v2:172.21.15.181:3300/0 conn(0x7f75181563e0 msgr2=0x7f75181567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:06.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.490+0000 7f751cc74700 1 --2- 172.21.15.163:0/3031185901 >> v2:172.21.15.181:3300/0 conn(0x7f75181563e0 0x7f75181567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:06.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.490+0000 7f751cc74700 1 -- 172.21.15.163:0/3031185901 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7518157410 con 0x7f75180c97c0 2023-10-09T06:32:06.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.490+0000 7f751cc74700 1 -- 172.21.15.163:0/3031185901 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f7510001470 con 0x7f75180c97c0 2023-10-09T06:32:06.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.490+0000 7f751dc76700 1 --2- 172.21.15.163:0/3031185901 >> v2:172.21.15.181:3300/0 conn(0x7f75181563e0 0x7f75181567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:32:06.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.490+0000 7f751cc74700 1 -- 172.21.15.163:0/3031185901 >> v1:172.21.15.163:6789/0 conn(0x7f75180c97c0 legacy=0x7f751806d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:06.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.490+0000 7f751cc74700 1 --2- 172.21.15.163:0/3031185901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7504003a60 0x7f7504003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:06.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.490+0000 7f751cc74700 1 --2- 172.21.15.163:0/3031185901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7504004810 0x7f7504004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:06.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.490+0000 7f751cc74700 1 --2- 172.21.15.163:0/3031185901 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75180c9350 0x7f75040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:06.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.490+0000 7f751cc74700 1 -- 172.21.15.163:0/3031185901 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7518156e50 con 0x7f7504003a60 2023-10-09T06:32:06.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.490+0000 7f751cc74700 1 -- 172.21.15.163:0/3031185901 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7518157130 con 0x7f7504004810 2023-10-09T06:32:06.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.490+0000 7f751cc74700 1 -- 172.21.15.163:0/3031185901 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f750400bb40 con 0x7f75180c9350 2023-10-09T06:32:06.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.490+0000 7f751dc76700 1 --2- 172.21.15.163:0/3031185901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7504003a60 0x7f7504003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:06.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.490+0000 7f751d475700 1 --2- 172.21.15.163:0/3031185901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7504004810 0x7f7504004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:06.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.490+0000 7f751e477700 1 --2- 172.21.15.163:0/3031185901 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75180c9350 0x7f75040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:06.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.491+0000 7f751dc76700 1 -- 172.21.15.163:0/3031185901 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75180c9350 msgr2=0x7f75040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:06.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.491+0000 7f751dc76700 1 --2- 172.21.15.163:0/3031185901 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75180c9350 0x7f75040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:06.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.491+0000 7f751dc76700 1 -- 172.21.15.163:0/3031185901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7504004810 msgr2=0x7f7504004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:06.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.491+0000 7f751dc76700 1 --2- 172.21.15.163:0/3031185901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7504004810 0x7f7504004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:06.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.491+0000 7f751dc76700 1 -- 172.21.15.163:0/3031185901 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7504003570 con 0x7f7504003a60 2023-10-09T06:32:06.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.491+0000 7f751dc76700 1 --2- 172.21.15.163:0/3031185901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7504003a60 0x7f7504003e70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f751400e3c0 tx=0x7f751400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5daf110b2a553685 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:06.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.491+0000 7f751cc74700 1 -- 172.21.15.163:0/3031185901 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751400ec70 con 0x7f7504003a60 2023-10-09T06:32:06.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.491+0000 7f751cc74700 1 -- 172.21.15.163:0/3031185901 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f751400f040 con 0x7f7504003a60 2023-10-09T06:32:06.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.491+0000 7f751cc74700 1 -- 172.21.15.163:0/3031185901 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7514014480 con 0x7f7504003a60 2023-10-09T06:32:06.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.491+0000 7f751ec78700 1 -- 172.21.15.163:0/3031185901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7504003a60 msgr2=0x7f7504003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:06.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.491+0000 7f751ec78700 1 --2- 172.21.15.163:0/3031185901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7504003a60 0x7f7504003e70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f751400e3c0 tx=0x7f751400b040 comp rx=0 tx=0).stop 2023-10-09T06:32:06.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.491+0000 7f751e477700 1 -- 172.21.15.163:0/3031185901 reap_dead start 2023-10-09T06:32:06.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.491+0000 7f751ec78700 1 -- 172.21.15.163:0/3031185901 shutdown_connections 2023-10-09T06:32:06.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.491+0000 7f751ec78700 1 -- 172.21.15.163:0/3031185901 >> 172.21.15.163:0/3031185901 conn(0x7f75180ccca0 msgr2=0x7f75180c8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:06.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.492+0000 7f751ec78700 1 -- 172.21.15.163:0/3031185901 shutdown_connections 2023-10-09T06:32:06.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.492+0000 7f751ec78700 1 -- 172.21.15.163:0/3031185901 wait complete. 2023-10-09T06:32:06.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.493+0000 7f751ec78700 1 Processor -- start 2023-10-09T06:32:06.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.493+0000 7f751ec78700 1 -- start start 2023-10-09T06:32:06.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.493+0000 7f751ec78700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7504003a60 0x7f75180ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:06.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.493+0000 7f751ec78700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75180a9c60 0x7f751814a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:06.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.493+0000 7f751ec78700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75180aa580 0x7f751806d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:06.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.493+0000 7f751ec78700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751814ac90 con 0x7f7504003a60 2023-10-09T06:32:06.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.493+0000 7f751ec78700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75180ca260 con 0x7f75180aa580 2023-10-09T06:32:06.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.493+0000 7f751ec78700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75180aae60 con 0x7f75180a9c60 2023-10-09T06:32:06.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.493+0000 7f751dc76700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7504003a60 0x7f75180ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:06.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.493+0000 7f751dc76700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7504003a60 0x7f75180ca6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58832/0 (socket says 172.21.15.163:58832) 2023-10-09T06:32:06.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.493+0000 7f751dc76700 1 -- 172.21.15.163:0/3365060731 learned_addr learned my addr 172.21.15.163:0/3365060731 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:06.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.494+0000 7f751d475700 1 --2- 172.21.15.163:0/3365060731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75180a9c60 0x7f751814a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:06.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.494+0000 7f751e477700 1 --2- 172.21.15.163:0/3365060731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75180aa580 0x7f751806d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:06.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.494+0000 7f751dc76700 1 -- 172.21.15.163:0/3365060731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75180a9c60 msgr2=0x7f751814a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:06.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.494+0000 7f751dc76700 1 --2- 172.21.15.163:0/3365060731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75180a9c60 0x7f751814a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:06.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.494+0000 7f751dc76700 1 -- 172.21.15.163:0/3365060731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75180aa580 msgr2=0x7f751806d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:06.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.494+0000 7f751dc76700 1 --2- 172.21.15.163:0/3365060731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75180aa580 0x7f751806d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:06.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.494+0000 7f751dc76700 1 -- 172.21.15.163:0/3365060731 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75180ab140 con 0x7f7504003a60 2023-10-09T06:32:06.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.494+0000 7f751d475700 1 --2- 172.21.15.163:0/3365060731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75180a9c60 0x7f751814a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:06.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.494+0000 7f751e477700 1 --2- 172.21.15.163:0/3365060731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75180aa580 0x7f751806d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:06.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.494+0000 7f751dc76700 1 --2- 172.21.15.163:0/3365060731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7504003a60 0x7f75180ca6f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f751400e9b0 tx=0x7f751401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:06.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.494+0000 7f750dffb700 1 -- 172.21.15.163:0/3365060731 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751401f650 con 0x7f7504003a60 2023-10-09T06:32:06.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.494+0000 7f750dffb700 1 -- 172.21.15.163:0/3365060731 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f751401f7f0 con 0x7f7504003a60 2023-10-09T06:32:06.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.494+0000 7f751ec78700 1 -- 172.21.15.163:0/3365060731 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75180c9ef0 con 0x7f7504003a60 2023-10-09T06:32:06.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.494+0000 7f750dffb700 1 -- 172.21.15.163:0/3365060731 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751401fbb0 con 0x7f7504003a60 2023-10-09T06:32:06.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.494+0000 7f751ec78700 1 -- 172.21.15.163:0/3365060731 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75180aabe0 con 0x7f7504003a60 2023-10-09T06:32:06.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.495+0000 7f751ec78700 1 -- 172.21.15.163:0/3365060731 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75180cac30 con 0x7f7504003a60 2023-10-09T06:32:06.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.495+0000 7f750dffb700 1 -- 172.21.15.163:0/3365060731 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7514003040 con 0x7f7504003a60 2023-10-09T06:32:06.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.496+0000 7f750dffb700 1 --2- 172.21.15.163:0/3365060731 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f74fc066f30 0x7f74fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:06.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.496+0000 7f750dffb700 1 -- 172.21.15.163:0/3365060731 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7514086940 con 0x7f7504003a60 2023-10-09T06:32:06.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.496+0000 7f751d475700 1 --2- 172.21.15.163:0/3365060731 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f74fc066f30 0x7f74fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:06.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.496+0000 7f751d475700 1 --2- 172.21.15.163:0/3365060731 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f74fc066f30 0x7f74fc0693e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f75180b14a0 tx=0x7f7508008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:06.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.500+0000 7f750dffb700 1 -- 172.21.15.163:0/3365060731 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7514053bb0 con 0x7f7504003a60 2023-10-09T06:32:06.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.759+0000 7f751ec78700 1 -- 172.21.15.163:0/3365060731 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f751804c3c0 con 0x7f7504003a60 2023-10-09T06:32:06.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.760+0000 7f750dffb700 1 -- 172.21.15.163:0/3365060731 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1434 (secure 0 0 0) 0x7f75140571d0 con 0x7f7504003a60 2023-10-09T06:32:06.761 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:06.761 INFO:teuthology.orchestra.run.smithi163.stdout:{"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":108,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":51,"num_osds":8,"num_up_osds":8,"osd_up_since":1696833029,"num_in_osds":8,"osd_in_since":1696833021,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592674,"bytes_used":2441519104,"bytes_avail":770652594176,"bytes_total":773094113280,"write_bytes_sec":2215,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":31,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"b","status":"up:active","gid":5370}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2023-10-09T06:31:50.309802+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-09T06:32:06.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.762+0000 7f751ec78700 1 -- 172.21.15.163:0/3365060731 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f74fc066f30 msgr2=0x7f74fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:06.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.762+0000 7f751ec78700 1 --2- 172.21.15.163:0/3365060731 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f74fc066f30 0x7f74fc0693e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f75180b14a0 tx=0x7f7508008040 comp rx=0 tx=0).stop 2023-10-09T06:32:06.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.762+0000 7f751ec78700 1 -- 172.21.15.163:0/3365060731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7504003a60 msgr2=0x7f75180ca6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:06.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.762+0000 7f751ec78700 1 --2- 172.21.15.163:0/3365060731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7504003a60 0x7f75180ca6f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f751400e9b0 tx=0x7f751401a000 comp rx=0 tx=0).stop 2023-10-09T06:32:06.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.762+0000 7f751ec78700 1 -- 172.21.15.163:0/3365060731 shutdown_connections 2023-10-09T06:32:06.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.762+0000 7f751ec78700 1 --2- 172.21.15.163:0/3365060731 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f74fc066f30 0x7f74fc0693e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:06.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.762+0000 7f751ec78700 1 --2- 172.21.15.163:0/3365060731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7504003a60 0x7f75180ca6f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:06.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.762+0000 7f751ec78700 1 --2- 172.21.15.163:0/3365060731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75180aa580 0x7f751806d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:06.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.762+0000 7f751ec78700 1 --2- 172.21.15.163:0/3365060731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75180a9c60 0x7f751814a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:06.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.762+0000 7f751ec78700 1 -- 172.21.15.163:0/3365060731 >> 172.21.15.163:0/3365060731 conn(0x7f75180ccca0 msgr2=0x7f7518153fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:06.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.762+0000 7f751ec78700 1 -- 172.21.15.163:0/3365060731 shutdown_connections 2023-10-09T06:32:06.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.762+0000 7f751ec78700 1 -- 172.21.15.163:0/3365060731 wait complete. 2023-10-09T06:32:06.774 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-09T06:32:06.775 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:32:06.837 INFO:tasks.ceph.mds.b.smithi181.stderr:2023-10-09T06:32:06.836+0000 7fc11c7c5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-09T06:32:06.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.888+0000 7f09eee6a700 1 Processor -- start 2023-10-09T06:32:06.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.888+0000 7f09eee6a700 1 -- start start 2023-10-09T06:32:06.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.889+0000 7f09eee6a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f09e80cc3c0 0x7f09e806b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:06.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.889+0000 7f09eee6a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f09e806bf20 0x7f09e80b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:06.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.889+0000 7f09eee6a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09e80b16a0 con 0x7f09e81574d0 2023-10-09T06:32:06.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.889+0000 7f09eee6a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09e80cd980 con 0x7f09e80cc3c0 2023-10-09T06:32:06.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.889+0000 7f09eee6a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09e80b1c60 con 0x7f09e806bf20 2023-10-09T06:32:06.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.889+0000 7f09ed667700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f09e80cc3c0 0x7f09e806b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:06.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.889+0000 7f09ed667700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f09e80cc3c0 0x7f09e806b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58840/0 (socket says 172.21.15.163:58840) 2023-10-09T06:32:06.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.889+0000 7f09ed667700 1 -- 172.21.15.163:0/1309852713 learned_addr learned my addr 172.21.15.163:0/1309852713 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:06.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.889+0000 7f09ee669700 1 --2- 172.21.15.163:0/1309852713 >> v2:172.21.15.181:3300/0 conn(0x7f09e806bf20 0x7f09e80b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:06.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.889+0000 7f09ed667700 1 -- 172.21.15.163:0/1309852713 >> v2:172.21.15.181:3300/0 conn(0x7f09e806bf20 msgr2=0x7f09e80b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:06.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.889+0000 7f09ed667700 1 --2- 172.21.15.163:0/1309852713 >> v2:172.21.15.181:3300/0 conn(0x7f09e806bf20 0x7f09e80b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:06.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.889+0000 7f09ed667700 1 -- 172.21.15.163:0/1309852713 >> v1:172.21.15.181:6789/0 conn(0x7f09e81574d0 legacy=0x7f09e80cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:06.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.889+0000 7f09ed667700 1 -- 172.21.15.163:0/1309852713 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e806c360 con 0x7f09e80cc3c0 2023-10-09T06:32:06.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.890+0000 7f09ed667700 1 --2- 172.21.15.163:0/1309852713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09e80cc3c0 0x7f09e806b9e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f09e8071300 tx=0x7f09d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4af4827cc9acea0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:06.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.890+0000 7f09ece66700 1 -- 172.21.15.163:0/1309852713 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d800b820 con 0x7f09e80cc3c0 2023-10-09T06:32:06.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.890+0000 7f09ece66700 1 -- 172.21.15.163:0/1309852713 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d800b980 con 0x7f09e80cc3c0 2023-10-09T06:32:06.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.890+0000 7f09ece66700 1 -- 172.21.15.163:0/1309852713 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d800bd20 con 0x7f09e80cc3c0 2023-10-09T06:32:06.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.890+0000 7f09eee6a700 1 -- 172.21.15.163:0/1309852713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09e80cc3c0 msgr2=0x7f09e806b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:06.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.890+0000 7f09eee6a700 1 --2- 172.21.15.163:0/1309852713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09e80cc3c0 0x7f09e806b9e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f09e8071300 tx=0x7f09d8009250 comp rx=0 tx=0).stop 2023-10-09T06:32:06.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.890+0000 7f09eee6a700 1 -- 172.21.15.163:0/1309852713 shutdown_connections 2023-10-09T06:32:06.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.890+0000 7f09eee6a700 1 --2- 172.21.15.163:0/1309852713 >> v2:172.21.15.181:3300/0 conn(0x7f09e806bf20 0x7f09e80b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:06.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.890+0000 7f09eee6a700 1 --2- 172.21.15.163:0/1309852713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09e80cc3c0 0x7f09e806b9e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:06.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.890+0000 7f09eee6a700 1 -- 172.21.15.163:0/1309852713 >> 172.21.15.163:0/1309852713 conn(0x7f09e80ce430 msgr2=0x7f09e80ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:06.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.890+0000 7f09eee6a700 1 -- 172.21.15.163:0/1309852713 shutdown_connections 2023-10-09T06:32:06.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.890+0000 7f09eee6a700 1 -- 172.21.15.163:0/1309852713 wait complete. 2023-10-09T06:32:06.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.891+0000 7f09eee6a700 1 Processor -- start 2023-10-09T06:32:06.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.892+0000 7f09eee6a700 1 -- start start 2023-10-09T06:32:06.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.892+0000 7f09eee6a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e806bf20 0x7f09e80afc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:06.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.892+0000 7f09eee6a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09e80cc3c0 0x7f09e816b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:06.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.892+0000 7f09eee6a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09e81574d0 0x7f09e8151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:06.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.892+0000 7f09eee6a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e80ae200 con 0x7f09e80cc3c0 2023-10-09T06:32:06.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.892+0000 7f09eee6a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e815c3b0 con 0x7f09e81574d0 2023-10-09T06:32:06.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.892+0000 7f09eee6a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e815c690 con 0x7f09e806bf20 2023-10-09T06:32:06.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.892+0000 7f09ed667700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09e80cc3c0 0x7f09e816b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:06.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.892+0000 7f09ed667700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09e80cc3c0 0x7f09e816b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58856/0 (socket says 172.21.15.163:58856) 2023-10-09T06:32:06.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.892+0000 7f09ed667700 1 -- 172.21.15.163:0/1504895240 learned_addr learned my addr 172.21.15.163:0/1504895240 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:06.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.892+0000 7f09ee669700 1 --2- 172.21.15.163:0/1504895240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09e81574d0 0x7f09e8151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:06.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.893+0000 7f09ede68700 1 --2- 172.21.15.163:0/1504895240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e806bf20 0x7f09e80afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:06.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.893+0000 7f09ed667700 1 -- 172.21.15.163:0/1504895240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e806bf20 msgr2=0x7f09e80afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:06.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.893+0000 7f09ed667700 1 --2- 172.21.15.163:0/1504895240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e806bf20 0x7f09e80afc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:06.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.893+0000 7f09ed667700 1 -- 172.21.15.163:0/1504895240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09e81574d0 msgr2=0x7f09e8151160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:06.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.893+0000 7f09ed667700 1 --2- 172.21.15.163:0/1504895240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09e81574d0 0x7f09e8151160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:06.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.893+0000 7f09ed667700 1 -- 172.21.15.163:0/1504895240 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d80094f0 con 0x7f09e80cc3c0 2023-10-09T06:32:06.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.893+0000 7f09ee669700 1 --2- 172.21.15.163:0/1504895240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09e81574d0 0x7f09e8151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:06.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.893+0000 7f09ed667700 1 --2- 172.21.15.163:0/1504895240 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09e80cc3c0 0x7f09e816b2f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f09d80030c0 tx=0x7f09d8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:06.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.893+0000 7f09ddffb700 1 -- 172.21.15.163:0/1504895240 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d8011e60 con 0x7f09e80cc3c0 2023-10-09T06:32:06.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.893+0000 7f09ddffb700 1 -- 172.21.15.163:0/1504895240 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d8020070 con 0x7f09e80cc3c0 2023-10-09T06:32:06.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.893+0000 7f09eee6a700 1 -- 172.21.15.163:0/1504895240 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e80ade90 con 0x7f09e80cc3c0 2023-10-09T06:32:06.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.893+0000 7f09ddffb700 1 -- 172.21.15.163:0/1504895240 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d801b6a0 con 0x7f09e80cc3c0 2023-10-09T06:32:06.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.893+0000 7f09eee6a700 1 -- 172.21.15.163:0/1504895240 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e80b01b0 con 0x7f09e80cc3c0 2023-10-09T06:32:06.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.894+0000 7f09eee6a700 1 -- 172.21.15.163:0/1504895240 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09e804c3c0 con 0x7f09e80cc3c0 2023-10-09T06:32:06.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.894+0000 7f09ddffb700 1 -- 172.21.15.163:0/1504895240 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f09d8022030 con 0x7f09e80cc3c0 2023-10-09T06:32:06.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.895+0000 7f09ddffb700 1 --2- 172.21.15.163:0/1504895240 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f09cc066f30 0x7f09cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:06.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.895+0000 7f09ddffb700 1 -- 172.21.15.163:0/1504895240 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f09d80864e0 con 0x7f09e80cc3c0 2023-10-09T06:32:06.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.895+0000 7f09ede68700 1 --2- 172.21.15.163:0/1504895240 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f09cc066f30 0x7f09cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:06.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.896+0000 7f09ede68700 1 --2- 172.21.15.163:0/1504895240 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f09cc066f30 0x7f09cc0693e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f09e4009930 tx=0x7f09e4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:06.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:06.899+0000 7f09ddffb700 1 -- 172.21.15.163:0/1504895240 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f09d8053750 con 0x7f09e80cc3c0 2023-10-09T06:32:07.078 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:07.077+0000 7f09eee6a700 1 -- 172.21.15.163:0/1504895240 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f09e804c3c0 con 0x7f09cc066f30 2023-10-09T06:32:07.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:07.085+0000 7f09ddffb700 1 -- 172.21.15.163:0/1504895240 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289462 (secure 0 0 0) 0x7f09e804c3c0 con 0x7f09cc066f30 2023-10-09T06:32:07.089 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:07.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:07.089+0000 7f09eee6a700 1 -- 172.21.15.163:0/1504895240 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f09cc066f30 msgr2=0x7f09cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:07.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:07.089+0000 7f09eee6a700 1 --2- 172.21.15.163:0/1504895240 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f09cc066f30 0x7f09cc0693e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f09e4009930 tx=0x7f09e4008040 comp rx=0 tx=0).stop 2023-10-09T06:32:07.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:07.089+0000 7f09eee6a700 1 -- 172.21.15.163:0/1504895240 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09e80cc3c0 msgr2=0x7f09e816b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:07.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:07.089+0000 7f09eee6a700 1 --2- 172.21.15.163:0/1504895240 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09e80cc3c0 0x7f09e816b2f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f09d80030c0 tx=0x7f09d8016000 comp rx=0 tx=0).stop 2023-10-09T06:32:07.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:07.089+0000 7f09eee6a700 1 -- 172.21.15.163:0/1504895240 shutdown_connections 2023-10-09T06:32:07.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:07.089+0000 7f09eee6a700 1 --2- 172.21.15.163:0/1504895240 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f09cc066f30 0x7f09cc0693e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:07.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:07.089+0000 7f09eee6a700 1 --2- 172.21.15.163:0/1504895240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09e81574d0 0x7f09e8151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:07.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:07.089+0000 7f09eee6a700 1 --2- 172.21.15.163:0/1504895240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e806bf20 0x7f09e80afc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:07.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:07.089+0000 7f09eee6a700 1 --2- 172.21.15.163:0/1504895240 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09e80cc3c0 0x7f09e816b2f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:07.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:07.089+0000 7f09eee6a700 1 -- 172.21.15.163:0/1504895240 >> 172.21.15.163:0/1504895240 conn(0x7f09e80ce430 msgr2=0x7f09e815d310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:07.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:07.089+0000 7f09eee6a700 1 -- 172.21.15.163:0/1504895240 shutdown_connections 2023-10-09T06:32:07.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:07.089+0000 7f09eee6a700 1 -- 172.21.15.163:0/1504895240 wait complete. 2023-10-09T06:32:07.093 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:32:07.101 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2023-10-09T06:32:06.319167+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2384296,"kb_used_data":5992,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752590424,"statfs":{"total":773094113280,"available":770652594176,"internally_reserved":0,"allocated":6135808,"data_stored":3256724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.007431"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.547405+0000","last_change":"2023-10-09T06:32:01.546315+0000","last_active":"2023-10-09T06:32:01.547405+0000","last_peered":"2023-10-09T06:32:01.547405+0000","last_clean":"2023-10-09T06:32:01.547405+0000","last_became_active":"2023-10-09T06:32:01.546088+0000","last_became_peered":"2023-10-09T06:32:01.546088+0000","last_unstale":"2023-10-09T06:32:01.547405+0000","last_undegraded":"2023-10-09T06:32:01.547405+0000","last_fullsized":"2023-10-09T06:32:01.547405+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:28:42.282791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557348+0000","last_change":"2023-10-09T06:32:01.549902+0000","last_active":"2023-10-09T06:32:01.557348+0000","last_peered":"2023-10-09T06:32:01.557348+0000","last_clean":"2023-10-09T06:32:01.557348+0000","last_became_active":"2023-10-09T06:32:01.549676+0000","last_became_peered":"2023-10-09T06:32:01.549676+0000","last_unstale":"2023-10-09T06:32:01.557348+0000","last_undegraded":"2023-10-09T06:32:01.557348+0000","last_fullsized":"2023-10-09T06:32:01.557348+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:39:07.097204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.563367+0000","last_change":"2023-10-09T06:32:01.557929+0000","last_active":"2023-10-09T06:32:01.563367+0000","last_peered":"2023-10-09T06:32:01.563367+0000","last_clean":"2023-10-09T06:32:01.563367+0000","last_became_active":"2023-10-09T06:32:01.557615+0000","last_became_peered":"2023-10-09T06:32:01.557615+0000","last_unstale":"2023-10-09T06:32:01.563367+0000","last_undegraded":"2023-10-09T06:32:01.563367+0000","last_fullsized":"2023-10-09T06:32:01.563367+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:23:35.817847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.560323+0000","last_change":"2023-10-09T06:32:01.553722+0000","last_active":"2023-10-09T06:32:01.560323+0000","last_peered":"2023-10-09T06:32:01.560323+0000","last_clean":"2023-10-09T06:32:01.560323+0000","last_became_active":"2023-10-09T06:32:01.553256+0000","last_became_peered":"2023-10-09T06:32:01.553256+0000","last_unstale":"2023-10-09T06:32:01.560323+0000","last_undegraded":"2023-10-09T06:32:01.560323+0000","last_fullsized":"2023-10-09T06:32:01.560323+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:05:58.355776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557481+0000","last_change":"2023-10-09T06:32:01.551457+0000","last_active":"2023-10-09T06:32:01.557481+0000","last_peered":"2023-10-09T06:32:01.557481+0000","last_clean":"2023-10-09T06:32:01.557481+0000","last_became_active":"2023-10-09T06:32:01.551187+0000","last_became_peered":"2023-10-09T06:32:01.551187+0000","last_unstale":"2023-10-09T06:32:01.557481+0000","last_undegraded":"2023-10-09T06:32:01.557481+0000","last_fullsized":"2023-10-09T06:32:01.557481+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:04:48.598681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.554978+0000","last_change":"2023-10-09T06:32:01.547943+0000","last_active":"2023-10-09T06:32:01.554978+0000","last_peered":"2023-10-09T06:32:01.554978+0000","last_clean":"2023-10-09T06:32:01.554978+0000","last_became_active":"2023-10-09T06:32:01.547681+0000","last_became_peered":"2023-10-09T06:32:01.547681+0000","last_unstale":"2023-10-09T06:32:01.554978+0000","last_undegraded":"2023-10-09T06:32:01.554978+0000","last_fullsized":"2023-10-09T06:32:01.554978+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:33:50.683561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.550217+0000","last_change":"2023-10-09T06:32:01.548020+0000","last_active":"2023-10-09T06:32:01.550217+0000","last_peered":"2023-10-09T06:32:01.550217+0000","last_clean":"2023-10-09T06:32:01.550217+0000","last_became_active":"2023-10-09T06:32:01.547616+0000","last_became_peered":"2023-10-09T06:32:01.547616+0000","last_unstale":"2023-10-09T06:32:01.550217+0000","last_undegraded":"2023-10-09T06:32:01.550217+0000","last_fullsized":"2023-10-09T06:32:01.550217+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:56:41.671239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.551257+0000","last_change":"2023-10-09T06:32:01.543275+0000","last_active":"2023-10-09T06:32:01.551257+0000","last_peered":"2023-10-09T06:32:01.551257+0000","last_clean":"2023-10-09T06:32:01.551257+0000","last_became_active":"2023-10-09T06:32:01.542967+0000","last_became_peered":"2023-10-09T06:32:01.542967+0000","last_unstale":"2023-10-09T06:32:01.551257+0000","last_undegraded":"2023-10-09T06:32:01.551257+0000","last_fullsized":"2023-10-09T06:32:01.551257+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:31:03.003654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.551341+0000","last_change":"2023-10-09T06:32:01.548201+0000","last_active":"2023-10-09T06:32:01.551341+0000","last_peered":"2023-10-09T06:32:01.551341+0000","last_clean":"2023-10-09T06:32:01.551341+0000","last_became_active":"2023-10-09T06:32:01.547931+0000","last_became_peered":"2023-10-09T06:32:01.547931+0000","last_unstale":"2023-10-09T06:32:01.551341+0000","last_undegraded":"2023-10-09T06:32:01.551341+0000","last_fullsized":"2023-10-09T06:32:01.551341+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:08:04.692947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.552210+0000","last_change":"2023-10-09T06:32:01.549247+0000","last_active":"2023-10-09T06:32:01.552210+0000","last_peered":"2023-10-09T06:32:01.552210+0000","last_clean":"2023-10-09T06:32:01.552210+0000","last_became_active":"2023-10-09T06:32:01.548950+0000","last_became_peered":"2023-10-09T06:32:01.548950+0000","last_unstale":"2023-10-09T06:32:01.552210+0000","last_undegraded":"2023-10-09T06:32:01.552210+0000","last_fullsized":"2023-10-09T06:32:01.552210+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:54:11.409702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.558101+0000","last_change":"2023-10-09T06:32:01.551494+0000","last_active":"2023-10-09T06:32:01.558101+0000","last_peered":"2023-10-09T06:32:01.558101+0000","last_clean":"2023-10-09T06:32:01.558101+0000","last_became_active":"2023-10-09T06:32:01.551260+0000","last_became_peered":"2023-10-09T06:32:01.551260+0000","last_unstale":"2023-10-09T06:32:01.558101+0000","last_undegraded":"2023-10-09T06:32:01.558101+0000","last_fullsized":"2023-10-09T06:32:01.558101+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:01:36.140759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.563667+0000","last_change":"2023-10-09T06:32:01.559281+0000","last_active":"2023-10-09T06:32:01.563667+0000","last_peered":"2023-10-09T06:32:01.563667+0000","last_clean":"2023-10-09T06:32:01.563667+0000","last_became_active":"2023-10-09T06:32:01.558951+0000","last_became_peered":"2023-10-09T06:32:01.558951+0000","last_unstale":"2023-10-09T06:32:01.563667+0000","last_undegraded":"2023-10-09T06:32:01.563667+0000","last_fullsized":"2023-10-09T06:32:01.563667+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:22:02.577699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.554790+0000","last_change":"2023-10-09T06:32:01.549386+0000","last_active":"2023-10-09T06:32:01.554790+0000","last_peered":"2023-10-09T06:32:01.554790+0000","last_clean":"2023-10-09T06:32:01.554790+0000","last_became_active":"2023-10-09T06:32:01.549019+0000","last_became_peered":"2023-10-09T06:32:01.549019+0000","last_unstale":"2023-10-09T06:32:01.554790+0000","last_undegraded":"2023-10-09T06:32:01.554790+0000","last_fullsized":"2023-10-09T06:32:01.554790+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:40:48.249891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.549090+0000","last_change":"2023-10-09T06:32:01.545232+0000","last_active":"2023-10-09T06:32:01.549090+0000","last_peered":"2023-10-09T06:32:01.549090+0000","last_clean":"2023-10-09T06:32:01.549090+0000","last_became_active":"2023-10-09T06:32:01.544899+0000","last_became_peered":"2023-10-09T06:32:01.544899+0000","last_unstale":"2023-10-09T06:32:01.549090+0000","last_undegraded":"2023-10-09T06:32:01.549090+0000","last_fullsized":"2023-10-09T06:32:01.549090+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:07:33.202557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.560332+0000","last_change":"2023-10-09T06:32:01.551970+0000","last_active":"2023-10-09T06:32:01.560332+0000","last_peered":"2023-10-09T06:32:01.560332+0000","last_clean":"2023-10-09T06:32:01.560332+0000","last_became_active":"2023-10-09T06:32:01.551713+0000","last_became_peered":"2023-10-09T06:32:01.551713+0000","last_unstale":"2023-10-09T06:32:01.560332+0000","last_undegraded":"2023-10-09T06:32:01.560332+0000","last_fullsized":"2023-10-09T06:32:01.560332+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:01:54.333566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557876+0000","last_change":"2023-10-09T06:32:01.550967+0000","last_active":"2023-10-09T06:32:01.557876+0000","last_peered":"2023-10-09T06:32:01.557876+0000","last_clean":"2023-10-09T06:32:01.557876+0000","last_became_active":"2023-10-09T06:32:01.550632+0000","last_became_peered":"2023-10-09T06:32:01.550632+0000","last_unstale":"2023-10-09T06:32:01.557876+0000","last_undegraded":"2023-10-09T06:32:01.557876+0000","last_fullsized":"2023-10-09T06:32:01.557876+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:33:31.855017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.549746+0000","last_change":"2023-10-09T06:32:01.546644+0000","last_active":"2023-10-09T06:32:01.549746+0000","last_peered":"2023-10-09T06:32:01.549746+0000","last_clean":"2023-10-09T06:32:01.549746+0000","last_became_active":"2023-10-09T06:32:01.546386+0000","last_became_peered":"2023-10-09T06:32:01.546386+0000","last_unstale":"2023-10-09T06:32:01.549746+0000","last_undegraded":"2023-10-09T06:32:01.549746+0000","last_fullsized":"2023-10-09T06:32:01.549746+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:36:55.014483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557666+0000","last_change":"2023-10-09T06:32:01.548982+0000","last_active":"2023-10-09T06:32:01.557666+0000","last_peered":"2023-10-09T06:32:01.557666+0000","last_clean":"2023-10-09T06:32:01.557666+0000","last_became_active":"2023-10-09T06:32:01.548720+0000","last_became_peered":"2023-10-09T06:32:01.548720+0000","last_unstale":"2023-10-09T06:32:01.557666+0000","last_undegraded":"2023-10-09T06:32:01.557666+0000","last_fullsized":"2023-10-09T06:32:01.557666+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:45:10.684091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.547399+0000","last_change":"2023-10-09T06:32:01.546674+0000","last_active":"2023-10-09T06:32:01.547399+0000","last_peered":"2023-10-09T06:32:01.547399+0000","last_clean":"2023-10-09T06:32:01.547399+0000","last_became_active":"2023-10-09T06:32:01.546425+0000","last_became_peered":"2023-10-09T06:32:01.546425+0000","last_unstale":"2023-10-09T06:32:01.547399+0000","last_undegraded":"2023-10-09T06:32:01.547399+0000","last_fullsized":"2023-10-09T06:32:01.547399+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:04:49.102247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.549378+0000","last_change":"2023-10-09T06:32:01.544774+0000","last_active":"2023-10-09T06:32:01.549378+0000","last_peered":"2023-10-09T06:32:01.549378+0000","last_clean":"2023-10-09T06:32:01.549378+0000","last_became_active":"2023-10-09T06:32:01.544523+0000","last_became_peered":"2023-10-09T06:32:01.544523+0000","last_unstale":"2023-10-09T06:32:01.549378+0000","last_undegraded":"2023-10-09T06:32:01.549378+0000","last_fullsized":"2023-10-09T06:32:01.549378+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:53:31.570299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.558952+0000","last_change":"2023-10-09T06:32:01.549300+0000","last_active":"2023-10-09T06:32:01.558952+0000","last_peered":"2023-10-09T06:32:01.558952+0000","last_clean":"2023-10-09T06:32:01.558952+0000","last_became_active":"2023-10-09T06:32:01.549065+0000","last_became_peered":"2023-10-09T06:32:01.549065+0000","last_unstale":"2023-10-09T06:32:01.558952+0000","last_undegraded":"2023-10-09T06:32:01.558952+0000","last_fullsized":"2023-10-09T06:32:01.558952+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:33:35.922723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557409+0000","last_change":"2023-10-09T06:32:01.549115+0000","last_active":"2023-10-09T06:32:01.557409+0000","last_peered":"2023-10-09T06:32:01.557409+0000","last_clean":"2023-10-09T06:32:01.557409+0000","last_became_active":"2023-10-09T06:32:01.548747+0000","last_became_peered":"2023-10-09T06:32:01.548747+0000","last_unstale":"2023-10-09T06:32:01.557409+0000","last_undegraded":"2023-10-09T06:32:01.557409+0000","last_fullsized":"2023-10-09T06:32:01.557409+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:05:36.594528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.559914+0000","last_change":"2023-10-09T06:32:01.553968+0000","last_active":"2023-10-09T06:32:01.559914+0000","last_peered":"2023-10-09T06:32:01.559914+0000","last_clean":"2023-10-09T06:32:01.559914+0000","last_became_active":"2023-10-09T06:32:01.553732+0000","last_became_peered":"2023-10-09T06:32:01.553732+0000","last_unstale":"2023-10-09T06:32:01.559914+0000","last_undegraded":"2023-10-09T06:32:01.559914+0000","last_fullsized":"2023-10-09T06:32:01.559914+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:05:58.355776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.552237+0000","last_change":"2023-10-09T06:32:01.546952+0000","last_active":"2023-10-09T06:32:01.552237+0000","last_peered":"2023-10-09T06:32:01.552237+0000","last_clean":"2023-10-09T06:32:01.552237+0000","last_became_active":"2023-10-09T06:32:01.546698+0000","last_became_peered":"2023-10-09T06:32:01.546698+0000","last_unstale":"2023-10-09T06:32:01.552237+0000","last_undegraded":"2023-10-09T06:32:01.552237+0000","last_fullsized":"2023-10-09T06:32:01.552237+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:05:31.370629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557101+0000","last_change":"2023-10-09T06:32:01.549766+0000","last_active":"2023-10-09T06:32:01.557101+0000","last_peered":"2023-10-09T06:32:01.557101+0000","last_clean":"2023-10-09T06:32:01.557101+0000","last_became_active":"2023-10-09T06:32:01.549522+0000","last_became_peered":"2023-10-09T06:32:01.549522+0000","last_unstale":"2023-10-09T06:32:01.557101+0000","last_undegraded":"2023-10-09T06:32:01.557101+0000","last_fullsized":"2023-10-09T06:32:01.557101+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:58:24.857028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.554021+0000","last_change":"2023-10-09T06:32:01.549901+0000","last_active":"2023-10-09T06:32:01.554021+0000","last_peered":"2023-10-09T06:32:01.554021+0000","last_clean":"2023-10-09T06:32:01.554021+0000","last_became_active":"2023-10-09T06:32:01.549644+0000","last_became_peered":"2023-10-09T06:32:01.549644+0000","last_unstale":"2023-10-09T06:32:01.554021+0000","last_undegraded":"2023-10-09T06:32:01.554021+0000","last_fullsized":"2023-10-09T06:32:01.554021+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:58:30.024948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557869+0000","last_change":"2023-10-09T06:32:01.553855+0000","last_active":"2023-10-09T06:32:01.557869+0000","last_peered":"2023-10-09T06:32:01.557869+0000","last_clean":"2023-10-09T06:32:01.557869+0000","last_became_active":"2023-10-09T06:32:01.553375+0000","last_became_peered":"2023-10-09T06:32:01.553375+0000","last_unstale":"2023-10-09T06:32:01.557869+0000","last_undegraded":"2023-10-09T06:32:01.557869+0000","last_fullsized":"2023-10-09T06:32:01.557869+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:12:03.262908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.558086+0000","last_change":"2023-10-09T06:32:01.548558+0000","last_active":"2023-10-09T06:32:01.558086+0000","last_peered":"2023-10-09T06:32:01.558086+0000","last_clean":"2023-10-09T06:32:01.558086+0000","last_became_active":"2023-10-09T06:32:01.548280+0000","last_became_peered":"2023-10-09T06:32:01.548280+0000","last_unstale":"2023-10-09T06:32:01.558086+0000","last_undegraded":"2023-10-09T06:32:01.558086+0000","last_fullsized":"2023-10-09T06:32:01.558086+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:11:26.654414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.552958+0000","last_change":"2023-10-09T06:32:01.547913+0000","last_active":"2023-10-09T06:32:01.552958+0000","last_peered":"2023-10-09T06:32:01.552958+0000","last_clean":"2023-10-09T06:32:01.552958+0000","last_became_active":"2023-10-09T06:32:01.547674+0000","last_became_peered":"2023-10-09T06:32:01.547674+0000","last_unstale":"2023-10-09T06:32:01.552958+0000","last_undegraded":"2023-10-09T06:32:01.552958+0000","last_fullsized":"2023-10-09T06:32:01.552958+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:23:55.906233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.558030+0000","last_change":"2023-10-09T06:32:01.550076+0000","last_active":"2023-10-09T06:32:01.558030+0000","last_peered":"2023-10-09T06:32:01.558030+0000","last_clean":"2023-10-09T06:32:01.558030+0000","last_became_active":"2023-10-09T06:32:01.549826+0000","last_became_peered":"2023-10-09T06:32:01.549826+0000","last_unstale":"2023-10-09T06:32:01.558030+0000","last_undegraded":"2023-10-09T06:32:01.558030+0000","last_fullsized":"2023-10-09T06:32:01.558030+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:33:31.855017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557984+0000","last_change":"2023-10-09T06:32:01.552934+0000","last_active":"2023-10-09T06:32:01.557984+0000","last_peered":"2023-10-09T06:32:01.557984+0000","last_clean":"2023-10-09T06:32:01.557984+0000","last_became_active":"2023-10-09T06:32:01.552675+0000","last_became_peered":"2023-10-09T06:32:01.552675+0000","last_unstale":"2023-10-09T06:32:01.557984+0000","last_undegraded":"2023-10-09T06:32:01.557984+0000","last_fullsized":"2023-10-09T06:32:01.557984+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:25:25.364284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557897+0000","last_change":"2023-10-09T06:32:01.553939+0000","last_active":"2023-10-09T06:32:01.557897+0000","last_peered":"2023-10-09T06:32:01.557897+0000","last_clean":"2023-10-09T06:32:01.557897+0000","last_became_active":"2023-10-09T06:32:01.553385+0000","last_became_peered":"2023-10-09T06:32:01.553385+0000","last_unstale":"2023-10-09T06:32:01.557897+0000","last_undegraded":"2023-10-09T06:32:01.557897+0000","last_fullsized":"2023-10-09T06:32:01.557897+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:58:30.024948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.560995+0000","last_change":"2023-10-09T06:32:01.557999+0000","last_active":"2023-10-09T06:32:01.560995+0000","last_peered":"2023-10-09T06:32:01.560995+0000","last_clean":"2023-10-09T06:32:01.560995+0000","last_became_active":"2023-10-09T06:32:01.557709+0000","last_became_peered":"2023-10-09T06:32:01.557709+0000","last_unstale":"2023-10-09T06:32:01.560995+0000","last_undegraded":"2023-10-09T06:32:01.560995+0000","last_fullsized":"2023-10-09T06:32:01.560995+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:03:46.626123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"50'4","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.564351+0000","last_change":"2023-10-09T06:32:00.545378+0000","last_active":"2023-10-09T06:32:01.564351+0000","last_peered":"2023-10-09T06:32:01.564351+0000","last_clean":"2023-10-09T06:32:01.564351+0000","last_became_active":"2023-10-09T06:32:00.545102+0000","last_became_peered":"2023-10-09T06:32:00.545102+0000","last_unstale":"2023-10-09T06:32:01.564351+0000","last_undegraded":"2023-10-09T06:32:01.564351+0000","last_fullsized":"2023-10-09T06:32:01.564351+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:54:00.414345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.559887+0000","last_change":"2023-10-09T06:32:01.555281+0000","last_active":"2023-10-09T06:32:01.559887+0000","last_peered":"2023-10-09T06:32:01.559887+0000","last_clean":"2023-10-09T06:32:01.559887+0000","last_became_active":"2023-10-09T06:32:01.554604+0000","last_became_peered":"2023-10-09T06:32:01.554604+0000","last_unstale":"2023-10-09T06:32:01.559887+0000","last_undegraded":"2023-10-09T06:32:01.559887+0000","last_fullsized":"2023-10-09T06:32:01.559887+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:00:06.478019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"50'1","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.565941+0000","last_change":"2023-10-09T06:32:00.550462+0000","last_active":"2023-10-09T06:32:01.565941+0000","last_peered":"2023-10-09T06:32:01.565941+0000","last_clean":"2023-10-09T06:32:01.565941+0000","last_became_active":"2023-10-09T06:32:00.549842+0000","last_became_peered":"2023-10-09T06:32:00.549842+0000","last_unstale":"2023-10-09T06:32:01.565941+0000","last_undegraded":"2023-10-09T06:32:01.565941+0000","last_fullsized":"2023-10-09T06:32:01.565941+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:05:57.346759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.558884+0000","last_change":"2023-10-09T06:32:01.554546+0000","last_active":"2023-10-09T06:32:01.558884+0000","last_peered":"2023-10-09T06:32:01.558884+0000","last_clean":"2023-10-09T06:32:01.558884+0000","last_became_active":"2023-10-09T06:32:01.554347+0000","last_became_peered":"2023-10-09T06:32:01.554347+0000","last_unstale":"2023-10-09T06:32:01.558884+0000","last_undegraded":"2023-10-09T06:32:01.558884+0000","last_fullsized":"2023-10-09T06:32:01.558884+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:00:06.478019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"50'2","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.565716+0000","last_change":"2023-10-09T06:32:00.546653+0000","last_active":"2023-10-09T06:32:01.565716+0000","last_peered":"2023-10-09T06:32:01.565716+0000","last_clean":"2023-10-09T06:32:01.565716+0000","last_became_active":"2023-10-09T06:32:00.546397+0000","last_became_peered":"2023-10-09T06:32:00.546397+0000","last_unstale":"2023-10-09T06:32:01.565716+0000","last_undegraded":"2023-10-09T06:32:01.565716+0000","last_fullsized":"2023-10-09T06:32:01.565716+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:53:13.331862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557906+0000","last_change":"2023-10-09T06:32:01.554059+0000","last_active":"2023-10-09T06:32:01.557906+0000","last_peered":"2023-10-09T06:32:01.557906+0000","last_clean":"2023-10-09T06:32:01.557906+0000","last_became_active":"2023-10-09T06:32:01.553777+0000","last_became_peered":"2023-10-09T06:32:01.553777+0000","last_unstale":"2023-10-09T06:32:01.557906+0000","last_undegraded":"2023-10-09T06:32:01.557906+0000","last_fullsized":"2023-10-09T06:32:01.557906+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:15:23.971809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"50'2","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.569382+0000","last_change":"2023-10-09T06:32:00.549247+0000","last_active":"2023-10-09T06:32:01.569382+0000","last_peered":"2023-10-09T06:32:01.569382+0000","last_clean":"2023-10-09T06:32:01.569382+0000","last_became_active":"2023-10-09T06:32:00.548979+0000","last_became_peered":"2023-10-09T06:32:00.548979+0000","last_unstale":"2023-10-09T06:32:01.569382+0000","last_undegraded":"2023-10-09T06:32:01.569382+0000","last_fullsized":"2023-10-09T06:32:01.569382+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:54:10.400685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557390+0000","last_change":"2023-10-09T06:32:01.548960+0000","last_active":"2023-10-09T06:32:01.557390+0000","last_peered":"2023-10-09T06:32:01.557390+0000","last_clean":"2023-10-09T06:32:01.557390+0000","last_became_active":"2023-10-09T06:32:01.548470+0000","last_became_peered":"2023-10-09T06:32:01.548470+0000","last_unstale":"2023-10-09T06:32:01.557390+0000","last_undegraded":"2023-10-09T06:32:01.557390+0000","last_fullsized":"2023-10-09T06:32:01.557390+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:42:18.468732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"50'1","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.561735+0000","last_change":"2023-10-09T06:32:00.550842+0000","last_active":"2023-10-09T06:32:01.561735+0000","last_peered":"2023-10-09T06:32:01.561735+0000","last_clean":"2023-10-09T06:32:01.561735+0000","last_became_active":"2023-10-09T06:32:00.550169+0000","last_became_peered":"2023-10-09T06:32:00.550169+0000","last_unstale":"2023-10-09T06:32:01.561735+0000","last_undegraded":"2023-10-09T06:32:01.561735+0000","last_fullsized":"2023-10-09T06:32:01.561735+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:55:32.071358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.549611+0000","last_change":"2023-10-09T06:32:01.545867+0000","last_active":"2023-10-09T06:32:01.549611+0000","last_peered":"2023-10-09T06:32:01.549611+0000","last_clean":"2023-10-09T06:32:01.549611+0000","last_became_active":"2023-10-09T06:32:01.545598+0000","last_became_peered":"2023-10-09T06:32:01.545598+0000","last_unstale":"2023-10-09T06:32:01.549611+0000","last_undegraded":"2023-10-09T06:32:01.549611+0000","last_fullsized":"2023-10-09T06:32:01.549611+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:59:16.905155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"50'1","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.562680+0000","last_change":"2023-10-09T06:32:00.543541+0000","last_active":"2023-10-09T06:32:01.562680+0000","last_peered":"2023-10-09T06:32:01.562680+0000","last_clean":"2023-10-09T06:32:01.562680+0000","last_became_active":"2023-10-09T06:32:00.543306+0000","last_became_peered":"2023-10-09T06:32:00.543306+0000","last_unstale":"2023-10-09T06:32:01.562680+0000","last_undegraded":"2023-10-09T06:32:01.562680+0000","last_fullsized":"2023-10-09T06:32:01.562680+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:20:24.757239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.561463+0000","last_change":"2023-10-09T06:32:01.557153+0000","last_active":"2023-10-09T06:32:01.561463+0000","last_peered":"2023-10-09T06:32:01.561463+0000","last_clean":"2023-10-09T06:32:01.561463+0000","last_became_active":"2023-10-09T06:32:01.556893+0000","last_became_peered":"2023-10-09T06:32:01.556893+0000","last_unstale":"2023-10-09T06:32:01.561463+0000","last_undegraded":"2023-10-09T06:32:01.561463+0000","last_fullsized":"2023-10-09T06:32:01.561463+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:18:27.289684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.545979+0000","last_change":"2023-10-09T06:32:00.546348+0000","last_active":"2023-10-09T06:32:01.545979+0000","last_peered":"2023-10-09T06:32:01.545979+0000","last_clean":"2023-10-09T06:32:01.545979+0000","last_became_active":"2023-10-09T06:32:00.546066+0000","last_became_peered":"2023-10-09T06:32:00.546066+0000","last_unstale":"2023-10-09T06:32:01.545979+0000","last_undegraded":"2023-10-09T06:32:01.545979+0000","last_fullsized":"2023-10-09T06:32:01.545979+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:01:53.324549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.549630+0000","last_change":"2023-10-09T06:32:01.543028+0000","last_active":"2023-10-09T06:32:01.549630+0000","last_peered":"2023-10-09T06:32:01.549630+0000","last_clean":"2023-10-09T06:32:01.549630+0000","last_became_active":"2023-10-09T06:32:01.542729+0000","last_became_peered":"2023-10-09T06:32:01.542729+0000","last_unstale":"2023-10-09T06:32:01.549630+0000","last_undegraded":"2023-10-09T06:32:01.549630+0000","last_fullsized":"2023-10-09T06:32:01.549630+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:00:16.216853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.540538+0000","last_change":"2023-10-09T06:32:00.549861+0000","last_active":"2023-10-09T06:32:01.540538+0000","last_peered":"2023-10-09T06:32:01.540538+0000","last_clean":"2023-10-09T06:32:01.540538+0000","last_became_active":"2023-10-09T06:32:00.549591+0000","last_became_peered":"2023-10-09T06:32:00.549591+0000","last_unstale":"2023-10-09T06:32:01.540538+0000","last_undegraded":"2023-10-09T06:32:01.540538+0000","last_fullsized":"2023-10-09T06:32:01.540538+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:38:45.645525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.561623+0000","last_change":"2023-10-09T06:32:01.553913+0000","last_active":"2023-10-09T06:32:01.561623+0000","last_peered":"2023-10-09T06:32:01.561623+0000","last_clean":"2023-10-09T06:32:01.561623+0000","last_became_active":"2023-10-09T06:32:01.553569+0000","last_became_peered":"2023-10-09T06:32:01.553569+0000","last_unstale":"2023-10-09T06:32:01.561623+0000","last_undegraded":"2023-10-09T06:32:01.561623+0000","last_fullsized":"2023-10-09T06:32:01.561623+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:50:43.178529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.544752+0000","last_change":"2023-10-09T06:32:00.555261+0000","last_active":"2023-10-09T06:32:01.544752+0000","last_peered":"2023-10-09T06:32:01.544752+0000","last_clean":"2023-10-09T06:32:01.544752+0000","last_became_active":"2023-10-09T06:32:00.555013+0000","last_became_peered":"2023-10-09T06:32:00.555013+0000","last_unstale":"2023-10-09T06:32:01.544752+0000","last_undegraded":"2023-10-09T06:32:01.544752+0000","last_fullsized":"2023-10-09T06:32:01.544752+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:54:10.400685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.560665+0000","last_change":"2023-10-09T06:32:01.554821+0000","last_active":"2023-10-09T06:32:01.560665+0000","last_peered":"2023-10-09T06:32:01.560665+0000","last_clean":"2023-10-09T06:32:01.560665+0000","last_became_active":"2023-10-09T06:32:01.554527+0000","last_became_peered":"2023-10-09T06:32:01.554527+0000","last_unstale":"2023-10-09T06:32:01.560665+0000","last_undegraded":"2023-10-09T06:32:01.560665+0000","last_fullsized":"2023-10-09T06:32:01.560665+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:07:21.526566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.536545+0000","last_change":"2023-10-09T06:32:00.549017+0000","last_active":"2023-10-09T06:32:01.536545+0000","last_peered":"2023-10-09T06:32:01.536545+0000","last_clean":"2023-10-09T06:32:01.536545+0000","last_became_active":"2023-10-09T06:32:00.548751+0000","last_became_peered":"2023-10-09T06:32:00.548751+0000","last_unstale":"2023-10-09T06:32:01.536545+0000","last_undegraded":"2023-10-09T06:32:01.536545+0000","last_fullsized":"2023-10-09T06:32:01.536545+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:15:22.962792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.552305+0000","last_change":"2023-10-09T06:32:01.549865+0000","last_active":"2023-10-09T06:32:01.552305+0000","last_peered":"2023-10-09T06:32:01.552305+0000","last_clean":"2023-10-09T06:32:01.552305+0000","last_became_active":"2023-10-09T06:32:01.549634+0000","last_became_peered":"2023-10-09T06:32:01.549634+0000","last_unstale":"2023-10-09T06:32:01.552305+0000","last_undegraded":"2023-10-09T06:32:01.552305+0000","last_fullsized":"2023-10-09T06:32:01.552305+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:08:04.692947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"50'3","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.566006+0000","last_change":"2023-10-09T06:32:00.555635+0000","last_active":"2023-10-09T06:32:01.566006+0000","last_peered":"2023-10-09T06:32:01.566006+0000","last_clean":"2023-10-09T06:32:01.566006+0000","last_became_active":"2023-10-09T06:32:00.555298+0000","last_became_peered":"2023-10-09T06:32:00.555298+0000","last_unstale":"2023-10-09T06:32:01.566006+0000","last_undegraded":"2023-10-09T06:32:01.566006+0000","last_fullsized":"2023-10-09T06:32:01.566006+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:01:53.324549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.558868+0000","last_change":"2023-10-09T06:32:01.553480+0000","last_active":"2023-10-09T06:32:01.558868+0000","last_peered":"2023-10-09T06:32:01.558868+0000","last_clean":"2023-10-09T06:32:01.558868+0000","last_became_active":"2023-10-09T06:32:01.552758+0000","last_became_peered":"2023-10-09T06:32:01.552758+0000","last_unstale":"2023-10-09T06:32:01.558868+0000","last_undegraded":"2023-10-09T06:32:01.558868+0000","last_fullsized":"2023-10-09T06:32:01.558868+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:15:00.958265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"50'6","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.566725+0000","last_change":"2023-10-09T06:32:00.545191+0000","last_active":"2023-10-09T06:32:01.566725+0000","last_peered":"2023-10-09T06:32:01.566725+0000","last_clean":"2023-10-09T06:32:01.566725+0000","last_became_active":"2023-10-09T06:32:00.544858+0000","last_became_peered":"2023-10-09T06:32:00.544858+0000","last_unstale":"2023-10-09T06:32:01.566725+0000","last_undegraded":"2023-10-09T06:32:01.566725+0000","last_fullsized":"2023-10-09T06:32:01.566725+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:19:47.545066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.559010+0000","last_change":"2023-10-09T06:32:01.553317+0000","last_active":"2023-10-09T06:32:01.559010+0000","last_peered":"2023-10-09T06:32:01.559010+0000","last_clean":"2023-10-09T06:32:01.559010+0000","last_became_active":"2023-10-09T06:32:01.553037+0000","last_became_peered":"2023-10-09T06:32:01.553037+0000","last_unstale":"2023-10-09T06:32:01.559010+0000","last_undegraded":"2023-10-09T06:32:01.559010+0000","last_fullsized":"2023-10-09T06:32:01.559010+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:33:31.855017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"50'2","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.565830+0000","last_change":"2023-10-09T06:32:00.550589+0000","last_active":"2023-10-09T06:32:01.565830+0000","last_peered":"2023-10-09T06:32:01.565830+0000","last_clean":"2023-10-09T06:32:01.565830+0000","last_became_active":"2023-10-09T06:32:00.549957+0000","last_became_peered":"2023-10-09T06:32:00.549957+0000","last_unstale":"2023-10-09T06:32:01.565830+0000","last_undegraded":"2023-10-09T06:32:01.565830+0000","last_fullsized":"2023-10-09T06:32:01.565830+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:22:01.568682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.565148+0000","last_change":"2023-10-09T06:32:01.560174+0000","last_active":"2023-10-09T06:32:01.565148+0000","last_peered":"2023-10-09T06:32:01.565148+0000","last_clean":"2023-10-09T06:32:01.565148+0000","last_became_active":"2023-10-09T06:32:01.559130+0000","last_became_peered":"2023-10-09T06:32:01.559130+0000","last_unstale":"2023-10-09T06:32:01.565148+0000","last_undegraded":"2023-10-09T06:32:01.565148+0000","last_fullsized":"2023-10-09T06:32:01.565148+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:15:23.971809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.539904+0000","last_change":"2023-10-09T06:32:00.548780+0000","last_active":"2023-10-09T06:32:01.539904+0000","last_peered":"2023-10-09T06:32:01.539904+0000","last_clean":"2023-10-09T06:32:01.539904+0000","last_became_active":"2023-10-09T06:32:00.548527+0000","last_became_peered":"2023-10-09T06:32:00.548527+0000","last_unstale":"2023-10-09T06:32:01.539904+0000","last_undegraded":"2023-10-09T06:32:01.539904+0000","last_fullsized":"2023-10-09T06:32:01.539904+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:20:07.450666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.551125+0000","last_change":"2023-10-09T06:32:01.549607+0000","last_active":"2023-10-09T06:32:01.551125+0000","last_peered":"2023-10-09T06:32:01.551125+0000","last_clean":"2023-10-09T06:32:01.551125+0000","last_became_active":"2023-10-09T06:32:01.549319+0000","last_became_peered":"2023-10-09T06:32:01.549319+0000","last_unstale":"2023-10-09T06:32:01.551125+0000","last_undegraded":"2023-10-09T06:32:01.551125+0000","last_fullsized":"2023-10-09T06:32:01.551125+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:54:11.409702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.546061+0000","last_change":"2023-10-09T06:32:00.549229+0000","last_active":"2023-10-09T06:32:01.546061+0000","last_peered":"2023-10-09T06:32:01.546061+0000","last_clean":"2023-10-09T06:32:01.546061+0000","last_became_active":"2023-10-09T06:32:00.548942+0000","last_became_peered":"2023-10-09T06:32:00.548942+0000","last_unstale":"2023-10-09T06:32:01.546061+0000","last_undegraded":"2023-10-09T06:32:01.546061+0000","last_fullsized":"2023-10-09T06:32:01.546061+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:08:03.683930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.559890+0000","last_change":"2023-10-09T06:32:01.553501+0000","last_active":"2023-10-09T06:32:01.559890+0000","last_peered":"2023-10-09T06:32:01.559890+0000","last_clean":"2023-10-09T06:32:01.559890+0000","last_became_active":"2023-10-09T06:32:01.552951+0000","last_became_peered":"2023-10-09T06:32:01.552951+0000","last_unstale":"2023-10-09T06:32:01.559890+0000","last_undegraded":"2023-10-09T06:32:01.559890+0000","last_fullsized":"2023-10-09T06:32:01.559890+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:03:11.998248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"50'8","reported_seq":18,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.566201+0000","last_change":"2023-10-09T06:32:00.556211+0000","last_active":"2023-10-09T06:32:01.566201+0000","last_peered":"2023-10-09T06:32:01.566201+0000","last_clean":"2023-10-09T06:32:01.566201+0000","last_became_active":"2023-10-09T06:32:00.556012+0000","last_became_peered":"2023-10-09T06:32:00.556012+0000","last_unstale":"2023-10-09T06:32:01.566201+0000","last_undegraded":"2023-10-09T06:32:01.566201+0000","last_fullsized":"2023-10-09T06:32:01.566201+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:56:40.662222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.558277+0000","last_change":"2023-10-09T06:32:01.554064+0000","last_active":"2023-10-09T06:32:01.558277+0000","last_peered":"2023-10-09T06:32:01.558277+0000","last_clean":"2023-10-09T06:32:01.558277+0000","last_became_active":"2023-10-09T06:32:01.553854+0000","last_became_peered":"2023-10-09T06:32:01.553854+0000","last_unstale":"2023-10-09T06:32:01.558277+0000","last_undegraded":"2023-10-09T06:32:01.558277+0000","last_fullsized":"2023-10-09T06:32:01.558277+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:03:46.626123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"50'1","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.565659+0000","last_change":"2023-10-09T06:32:00.549311+0000","last_active":"2023-10-09T06:32:01.565659+0000","last_peered":"2023-10-09T06:32:01.565659+0000","last_clean":"2023-10-09T06:32:01.565659+0000","last_became_active":"2023-10-09T06:32:00.549080+0000","last_became_peered":"2023-10-09T06:32:00.549080+0000","last_unstale":"2023-10-09T06:32:01.565659+0000","last_undegraded":"2023-10-09T06:32:01.565659+0000","last_fullsized":"2023-10-09T06:32:01.565659+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:33:30.846000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.559243+0000","last_change":"2023-10-09T06:32:01.553252+0000","last_active":"2023-10-09T06:32:01.559243+0000","last_peered":"2023-10-09T06:32:01.559243+0000","last_clean":"2023-10-09T06:32:01.559243+0000","last_became_active":"2023-10-09T06:32:01.552892+0000","last_became_peered":"2023-10-09T06:32:01.552892+0000","last_unstale":"2023-10-09T06:32:01.559243+0000","last_undegraded":"2023-10-09T06:32:01.559243+0000","last_fullsized":"2023-10-09T06:32:01.559243+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:15:23.971809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"50'2","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.566069+0000","last_change":"2023-10-09T06:32:00.547761+0000","last_active":"2023-10-09T06:32:01.566069+0000","last_peered":"2023-10-09T06:32:01.566069+0000","last_clean":"2023-10-09T06:32:01.566069+0000","last_became_active":"2023-10-09T06:32:00.547505+0000","last_became_peered":"2023-10-09T06:32:00.547505+0000","last_unstale":"2023-10-09T06:32:01.566069+0000","last_undegraded":"2023-10-09T06:32:01.566069+0000","last_fullsized":"2023-10-09T06:32:01.566069+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:50:59.843352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557563+0000","last_change":"2023-10-09T06:32:01.551087+0000","last_active":"2023-10-09T06:32:01.557563+0000","last_peered":"2023-10-09T06:32:01.557563+0000","last_clean":"2023-10-09T06:32:01.557563+0000","last_became_active":"2023-10-09T06:32:01.550836+0000","last_became_peered":"2023-10-09T06:32:01.550836+0000","last_unstale":"2023-10-09T06:32:01.557563+0000","last_undegraded":"2023-10-09T06:32:01.557563+0000","last_fullsized":"2023-10-09T06:32:01.557563+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:58:42.426644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"50'1","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.566771+0000","last_change":"2023-10-09T06:32:00.549151+0000","last_active":"2023-10-09T06:32:01.566771+0000","last_peered":"2023-10-09T06:32:01.566771+0000","last_clean":"2023-10-09T06:32:01.566771+0000","last_became_active":"2023-10-09T06:32:00.548893+0000","last_became_peered":"2023-10-09T06:32:00.548893+0000","last_unstale":"2023-10-09T06:32:01.566771+0000","last_undegraded":"2023-10-09T06:32:01.566771+0000","last_fullsized":"2023-10-09T06:32:01.566771+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:07:32.193540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.558887+0000","last_change":"2023-10-09T06:32:01.553538+0000","last_active":"2023-10-09T06:32:01.558887+0000","last_peered":"2023-10-09T06:32:01.558887+0000","last_clean":"2023-10-09T06:32:01.558887+0000","last_became_active":"2023-10-09T06:32:01.552796+0000","last_became_peered":"2023-10-09T06:32:01.552796+0000","last_unstale":"2023-10-09T06:32:01.558887+0000","last_undegraded":"2023-10-09T06:32:01.558887+0000","last_fullsized":"2023-10-09T06:32:01.558887+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:01:34.922109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.541180+0000","last_change":"2023-10-09T06:32:00.550205+0000","last_active":"2023-10-09T06:32:01.541180+0000","last_peered":"2023-10-09T06:32:01.541180+0000","last_clean":"2023-10-09T06:32:01.541180+0000","last_became_active":"2023-10-09T06:32:00.549939+0000","last_became_peered":"2023-10-09T06:32:00.549939+0000","last_unstale":"2023-10-09T06:32:01.541180+0000","last_undegraded":"2023-10-09T06:32:01.541180+0000","last_fullsized":"2023-10-09T06:32:01.541180+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:18:11.772295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.558989+0000","last_change":"2023-10-09T06:32:01.553159+0000","last_active":"2023-10-09T06:32:01.558989+0000","last_peered":"2023-10-09T06:32:01.558989+0000","last_clean":"2023-10-09T06:32:01.558989+0000","last_became_active":"2023-10-09T06:32:01.552830+0000","last_became_peered":"2023-10-09T06:32:01.552830+0000","last_unstale":"2023-10-09T06:32:01.558989+0000","last_undegraded":"2023-10-09T06:32:01.558989+0000","last_fullsized":"2023-10-09T06:32:01.558989+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:33:50.683561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"50'2","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.565580+0000","last_change":"2023-10-09T06:32:00.550579+0000","last_active":"2023-10-09T06:32:01.565580+0000","last_peered":"2023-10-09T06:32:01.565580+0000","last_clean":"2023-10-09T06:32:01.565580+0000","last_became_active":"2023-10-09T06:32:00.549929+0000","last_became_peered":"2023-10-09T06:32:00.549929+0000","last_unstale":"2023-10-09T06:32:01.565580+0000","last_undegraded":"2023-10-09T06:32:01.565580+0000","last_fullsized":"2023-10-09T06:32:01.565580+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:03:10.989230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557785+0000","last_change":"2023-10-09T06:32:01.552855+0000","last_active":"2023-10-09T06:32:01.557785+0000","last_peered":"2023-10-09T06:32:01.557785+0000","last_clean":"2023-10-09T06:32:01.557785+0000","last_became_active":"2023-10-09T06:32:01.550520+0000","last_became_peered":"2023-10-09T06:32:01.550520+0000","last_unstale":"2023-10-09T06:32:01.557785+0000","last_undegraded":"2023-10-09T06:32:01.557785+0000","last_fullsized":"2023-10-09T06:32:01.557785+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:07:33.202557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.544170+0000","last_change":"2023-10-09T06:32:00.549467+0000","last_active":"2023-10-09T06:32:01.544170+0000","last_peered":"2023-10-09T06:32:01.544170+0000","last_clean":"2023-10-09T06:32:01.544170+0000","last_became_active":"2023-10-09T06:32:00.549177+0000","last_became_peered":"2023-10-09T06:32:00.549177+0000","last_unstale":"2023-10-09T06:32:01.544170+0000","last_undegraded":"2023-10-09T06:32:01.544170+0000","last_fullsized":"2023-10-09T06:32:01.544170+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:13:58.204471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.558523+0000","last_change":"2023-10-09T06:32:01.557336+0000","last_active":"2023-10-09T06:32:01.558523+0000","last_peered":"2023-10-09T06:32:01.558523+0000","last_clean":"2023-10-09T06:32:01.558523+0000","last_became_active":"2023-10-09T06:32:01.557103+0000","last_became_peered":"2023-10-09T06:32:01.557103+0000","last_unstale":"2023-10-09T06:32:01.558523+0000","last_undegraded":"2023-10-09T06:32:01.558523+0000","last_fullsized":"2023-10-09T06:32:01.558523+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:22:02.577699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.545523+0000","last_change":"2023-10-09T06:32:00.552189+0000","last_active":"2023-10-09T06:32:01.545523+0000","last_peered":"2023-10-09T06:32:01.545523+0000","last_clean":"2023-10-09T06:32:01.545523+0000","last_became_active":"2023-10-09T06:32:00.551888+0000","last_became_peered":"2023-10-09T06:32:00.551888+0000","last_unstale":"2023-10-09T06:32:01.545523+0000","last_undegraded":"2023-10-09T06:32:01.545523+0000","last_fullsized":"2023-10-09T06:32:01.545523+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:53:13.331862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.563713+0000","last_change":"2023-10-09T06:32:01.559460+0000","last_active":"2023-10-09T06:32:01.563713+0000","last_peered":"2023-10-09T06:32:01.563713+0000","last_clean":"2023-10-09T06:32:01.563713+0000","last_became_active":"2023-10-09T06:32:01.559193+0000","last_became_peered":"2023-10-09T06:32:01.559193+0000","last_unstale":"2023-10-09T06:32:01.563713+0000","last_undegraded":"2023-10-09T06:32:01.563713+0000","last_fullsized":"2023-10-09T06:32:01.563713+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:03:11.998248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"50'2","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.563021+0000","last_change":"2023-10-09T06:32:00.552045+0000","last_active":"2023-10-09T06:32:01.563021+0000","last_peered":"2023-10-09T06:32:01.563021+0000","last_clean":"2023-10-09T06:32:01.563021+0000","last_became_active":"2023-10-09T06:32:00.551778+0000","last_became_peered":"2023-10-09T06:32:00.551778+0000","last_unstale":"2023-10-09T06:32:01.563021+0000","last_undegraded":"2023-10-09T06:32:01.563021+0000","last_fullsized":"2023-10-09T06:32:01.563021+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:34:11.248280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.563834+0000","last_change":"2023-10-09T06:32:01.559146+0000","last_active":"2023-10-09T06:32:01.563834+0000","last_peered":"2023-10-09T06:32:01.563834+0000","last_clean":"2023-10-09T06:32:01.563834+0000","last_became_active":"2023-10-09T06:32:01.558847+0000","last_became_peered":"2023-10-09T06:32:01.558847+0000","last_unstale":"2023-10-09T06:32:01.563834+0000","last_undegraded":"2023-10-09T06:32:01.563834+0000","last_fullsized":"2023-10-09T06:32:01.563834+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:03:46.626123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.540913+0000","last_change":"2023-10-09T06:32:00.542936+0000","last_active":"2023-10-09T06:32:01.540913+0000","last_peered":"2023-10-09T06:32:01.540913+0000","last_clean":"2023-10-09T06:32:01.540913+0000","last_became_active":"2023-10-09T06:32:00.542687+0000","last_became_peered":"2023-10-09T06:32:00.542687+0000","last_unstale":"2023-10-09T06:32:01.540913+0000","last_undegraded":"2023-10-09T06:32:01.540913+0000","last_fullsized":"2023-10-09T06:32:01.540913+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:19:47.545066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.552287+0000","last_change":"2023-10-09T06:32:01.549830+0000","last_active":"2023-10-09T06:32:01.552287+0000","last_peered":"2023-10-09T06:32:01.552287+0000","last_clean":"2023-10-09T06:32:01.552287+0000","last_became_active":"2023-10-09T06:32:01.549593+0000","last_became_peered":"2023-10-09T06:32:01.549593+0000","last_unstale":"2023-10-09T06:32:01.552287+0000","last_undegraded":"2023-10-09T06:32:01.552287+0000","last_fullsized":"2023-10-09T06:32:01.552287+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:07:33.202557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.539561+0000","last_change":"2023-10-09T06:32:00.544144+0000","last_active":"2023-10-09T06:32:01.539561+0000","last_peered":"2023-10-09T06:32:01.539561+0000","last_clean":"2023-10-09T06:32:01.539561+0000","last_became_active":"2023-10-09T06:32:00.543899+0000","last_became_peered":"2023-10-09T06:32:00.543899+0000","last_unstale":"2023-10-09T06:32:01.539561+0000","last_undegraded":"2023-10-09T06:32:01.539561+0000","last_fullsized":"2023-10-09T06:32:01.539561+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:20:24.504501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557888+0000","last_change":"2023-10-09T06:32:01.554041+0000","last_active":"2023-10-09T06:32:01.557888+0000","last_peered":"2023-10-09T06:32:01.557888+0000","last_clean":"2023-10-09T06:32:01.557888+0000","last_became_active":"2023-10-09T06:32:01.553605+0000","last_became_peered":"2023-10-09T06:32:01.553605+0000","last_unstale":"2023-10-09T06:32:01.557888+0000","last_undegraded":"2023-10-09T06:32:01.557888+0000","last_fullsized":"2023-10-09T06:32:01.557888+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:07:21.526566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.547215+0000","last_change":"2023-10-09T06:32:00.546419+0000","last_active":"2023-10-09T06:32:01.547215+0000","last_peered":"2023-10-09T06:32:01.547215+0000","last_clean":"2023-10-09T06:32:01.547215+0000","last_became_active":"2023-10-09T06:32:00.545725+0000","last_became_peered":"2023-10-09T06:32:00.545725+0000","last_unstale":"2023-10-09T06:32:01.547215+0000","last_undegraded":"2023-10-09T06:32:01.547215+0000","last_fullsized":"2023-10-09T06:32:01.547215+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:56:40.662222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557427+0000","last_change":"2023-10-09T06:32:01.552019+0000","last_active":"2023-10-09T06:32:01.557427+0000","last_peered":"2023-10-09T06:32:01.557427+0000","last_clean":"2023-10-09T06:32:01.557427+0000","last_became_active":"2023-10-09T06:32:01.551738+0000","last_became_peered":"2023-10-09T06:32:01.551738+0000","last_unstale":"2023-10-09T06:32:01.557427+0000","last_undegraded":"2023-10-09T06:32:01.557427+0000","last_fullsized":"2023-10-09T06:32:01.557427+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:59:04.361941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.539629+0000","last_change":"2023-10-09T06:32:00.547984+0000","last_active":"2023-10-09T06:32:01.539629+0000","last_peered":"2023-10-09T06:32:01.539629+0000","last_clean":"2023-10-09T06:32:01.539629+0000","last_became_active":"2023-10-09T06:32:00.547757+0000","last_became_peered":"2023-10-09T06:32:00.547757+0000","last_unstale":"2023-10-09T06:32:01.539629+0000","last_undegraded":"2023-10-09T06:32:01.539629+0000","last_fullsized":"2023-10-09T06:32:01.539629+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:19:47.545066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.558973+0000","last_change":"2023-10-09T06:32:01.549769+0000","last_active":"2023-10-09T06:32:01.558973+0000","last_peered":"2023-10-09T06:32:01.558973+0000","last_clean":"2023-10-09T06:32:01.558973+0000","last_became_active":"2023-10-09T06:32:01.549499+0000","last_became_peered":"2023-10-09T06:32:01.549499+0000","last_unstale":"2023-10-09T06:32:01.558973+0000","last_undegraded":"2023-10-09T06:32:01.558973+0000","last_fullsized":"2023-10-09T06:32:01.558973+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:58:30.024948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.540704+0000","last_change":"2023-10-09T06:32:00.552838+0000","last_active":"2023-10-09T06:32:01.540704+0000","last_peered":"2023-10-09T06:32:01.540704+0000","last_clean":"2023-10-09T06:32:01.540704+0000","last_became_active":"2023-10-09T06:32:00.552539+0000","last_became_peered":"2023-10-09T06:32:00.552539+0000","last_unstale":"2023-10-09T06:32:01.540704+0000","last_undegraded":"2023-10-09T06:32:01.540704+0000","last_fullsized":"2023-10-09T06:32:01.540704+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:34:11.248280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.558072+0000","last_change":"2023-10-09T06:32:01.550552+0000","last_active":"2023-10-09T06:32:01.558072+0000","last_peered":"2023-10-09T06:32:01.558072+0000","last_clean":"2023-10-09T06:32:01.558072+0000","last_became_active":"2023-10-09T06:32:01.550269+0000","last_became_peered":"2023-10-09T06:32:01.550269+0000","last_unstale":"2023-10-09T06:32:01.558072+0000","last_undegraded":"2023-10-09T06:32:01.558072+0000","last_fullsized":"2023-10-09T06:32:01.558072+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:01:54.333566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.544287+0000","last_change":"2023-10-09T06:32:00.546841+0000","last_active":"2023-10-09T06:32:01.544287+0000","last_peered":"2023-10-09T06:32:01.544287+0000","last_clean":"2023-10-09T06:32:01.544287+0000","last_became_active":"2023-10-09T06:32:00.546540+0000","last_became_peered":"2023-10-09T06:32:00.546540+0000","last_unstale":"2023-10-09T06:32:01.544287+0000","last_undegraded":"2023-10-09T06:32:01.544287+0000","last_fullsized":"2023-10-09T06:32:01.544287+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:38:45.645525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.549652+0000","last_change":"2023-10-09T06:32:01.544400+0000","last_active":"2023-10-09T06:32:01.549652+0000","last_peered":"2023-10-09T06:32:01.549652+0000","last_clean":"2023-10-09T06:32:01.549652+0000","last_became_active":"2023-10-09T06:32:01.542947+0000","last_became_peered":"2023-10-09T06:32:01.542947+0000","last_unstale":"2023-10-09T06:32:01.549652+0000","last_undegraded":"2023-10-09T06:32:01.549652+0000","last_fullsized":"2023-10-09T06:32:01.549652+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:34:46.591947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.537115+0000","last_change":"2023-10-09T06:32:00.551080+0000","last_active":"2023-10-09T06:32:01.537115+0000","last_peered":"2023-10-09T06:32:01.537115+0000","last_clean":"2023-10-09T06:32:01.537115+0000","last_became_active":"2023-10-09T06:32:00.550608+0000","last_became_peered":"2023-10-09T06:32:00.550608+0000","last_unstale":"2023-10-09T06:32:01.537115+0000","last_undegraded":"2023-10-09T06:32:01.537115+0000","last_fullsized":"2023-10-09T06:32:01.537115+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:58:29.015931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.551290+0000","last_change":"2023-10-09T06:32:01.545252+0000","last_active":"2023-10-09T06:32:01.551290+0000","last_peered":"2023-10-09T06:32:01.551290+0000","last_clean":"2023-10-09T06:32:01.551290+0000","last_became_active":"2023-10-09T06:32:01.545003+0000","last_became_peered":"2023-10-09T06:32:01.545003+0000","last_unstale":"2023-10-09T06:32:01.551290+0000","last_undegraded":"2023-10-09T06:32:01.551290+0000","last_fullsized":"2023-10-09T06:32:01.551290+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:24:43.541920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.540056+0000","last_change":"2023-10-09T06:32:00.548403+0000","last_active":"2023-10-09T06:32:01.540056+0000","last_peered":"2023-10-09T06:32:01.540056+0000","last_clean":"2023-10-09T06:32:01.540056+0000","last_became_active":"2023-10-09T06:32:00.548144+0000","last_became_peered":"2023-10-09T06:32:00.548144+0000","last_unstale":"2023-10-09T06:32:01.540056+0000","last_undegraded":"2023-10-09T06:32:01.540056+0000","last_fullsized":"2023-10-09T06:32:01.540056+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:30:34.904806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":190,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.542789+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:32:01.542789+0000","last_peered":"2023-10-09T06:32:01.542789+0000","last_clean":"2023-10-09T06:32:01.542789+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:32:01.542789+0000","last_undegraded":"2023-10-09T06:32:01.542789+0000","last_fullsized":"2023-10-09T06:32:01.542789+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298472,"kb_used_data":1184,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073368,"statfs":{"total":96636764160,"available":96331128832,"internally_reserved":0,"allocated":1212416,"data_stored":849450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.438},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.32100000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.45900000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.504},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.61699999999999999}]},{"osd":3,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.65800000000000003}]},{"osd":4,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.63200000000000001}]},{"osd":5,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.59299999999999997}]},{"osd":6,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.67100000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297884,"kb_used_data":596,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073956,"statfs":{"total":96636764160,"available":96331730944,"internally_reserved":0,"allocated":610304,"data_stored":258922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.61299999999999999}]},{"osd":1,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.54900000000000004}]},{"osd":2,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.66700000000000004}]},{"osd":3,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.503},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":4.7510000000000003,"5min":4.7510000000000003,"15min":4.7510000000000003},"last":0.59299999999999997}]},{"osd":4,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.65200000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.73699999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.67900000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297888,"kb_used_data":600,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073952,"statfs":{"total":96636764160,"available":96331726848,"internally_reserved":0,"allocated":614400,"data_stored":259012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.31,"15min":0.31},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.30199999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.32700000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":5.8529999999999998,"5min":5.8529999999999998,"15min":5.8529999999999998},"last":0.25900000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.627},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.61699999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":5.9459999999999997,"5min":5.9459999999999997,"15min":5.9459999999999997},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.57999999999999996}]},{"osd":6,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":5.9160000000000004,"5min":5.9160000000000004,"15min":5.9160000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.48799999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":5.9729999999999999,"5min":5.9729999999999999,"15min":5.9729999999999999},"last":0.50700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298480,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073360,"statfs":{"total":96636764160,"available":96331120640,"internally_reserved":0,"allocated":1220608,"data_stored":851380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.438,"5min":0.438,"15min":0.438},"last":0.222},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.26200000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.307}]},{"osd":3,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.318}]},{"osd":4,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":5.484,"5min":5.484,"15min":5.484},"last":0.36099999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":5.5449999999999999,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.441}]},{"osd":6,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.41799999999999998}]},{"osd":7,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.434},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":5.5800000000000001,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.501}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297892,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073948,"statfs":{"total":96636764160,"available":96331722752,"internally_reserved":0,"allocated":618496,"data_stored":258978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":7.2699999999999996,"5min":7.2699999999999996,"15min":7.2699999999999996},"last":0.46999999999999997}]},{"osd":1,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":7.327,"5min":7.327,"15min":7.327},"last":0.35799999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.34399999999999997}]},{"osd":4,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":7.2389999999999999,"5min":7.2389999999999999,"15min":7.2389999999999999},"last":0.443}]},{"osd":5,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":10.890000000000001,"5min":10.890000000000001,"15min":10.890000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.379}]},{"osd":6,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":7.2999999999999998,"5min":7.2999999999999998,"15min":7.2999999999999998},"last":0.499},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.42899999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":7.1879999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.49099999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297892,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073948,"statfs":{"total":96636764160,"available":96331722752,"internally_reserved":0,"allocated":618496,"data_stored":258978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.35599999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.41899999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.376}]},{"osd":4,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.46999999999999997}]},{"osd":5,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.56599999999999995}]},{"osd":6,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.51700000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.52600000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297896,"kb_used_data":608,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073944,"statfs":{"total":96636764160,"available":96331718656,"internally_reserved":0,"allocated":622592,"data_stored":260458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.56699999999999995}]},{"osd":1,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.61499999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.71699999999999997}]},{"osd":3,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.749}]},{"osd":5,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.76700000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.78100000000000003}]},{"osd":7,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.80000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297892,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073948,"statfs":{"total":96636764160,"available":96331722752,"internally_reserved":0,"allocated":618496,"data_stored":259546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.497}]},{"osd":1,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.47599999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.51800000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.441},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.57599999999999996}]},{"osd":4,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.55500000000000005}]},{"osd":6,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.54300000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.63400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:32:10.108 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:32:10.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.236+0000 7fe28db0b700 1 Processor -- start 2023-10-09T06:32:10.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.236+0000 7fe28db0b700 1 -- start start 2023-10-09T06:32:10.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.237+0000 7fe28db0b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe288159c60 0x7fe28815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:10.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.237+0000 7fe28db0b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe28815db10 0x7fe288069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:10.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.237+0000 7fe28db0b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe28815a160 con 0x7fe288159890 2023-10-09T06:32:10.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.237+0000 7fe28db0b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe28815df50 con 0x7fe288159c60 2023-10-09T06:32:10.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.237+0000 7fe28db0b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2880d13a0 con 0x7fe28815db10 2023-10-09T06:32:10.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.237+0000 7fe286ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe288159c60 0x7fe28815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:10.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.237+0000 7fe286ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe288159c60 0x7fe28815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:32814/0 (socket says 172.21.15.163:32814) 2023-10-09T06:32:10.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.237+0000 7fe286ffd700 1 -- 172.21.15.163:0/77167144 learned_addr learned my addr 172.21.15.163:0/77167144 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:10.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.237+0000 7fe2877fe700 1 --1- 172.21.15.163:0/77167144 >> v1:172.21.15.163:6789/0 conn(0x7fe288159890 0x7fe28815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:56698/0 (socket says 172.21.15.163:56698) 2023-10-09T06:32:10.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.237+0000 7fe287fff700 1 --2- 172.21.15.163:0/77167144 >> v2:172.21.15.181:3300/0 conn(0x7fe28815db10 0x7fe288069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:10.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.237+0000 7fe286ffd700 1 -- 172.21.15.163:0/77167144 >> v2:172.21.15.181:3300/0 conn(0x7fe28815db10 msgr2=0x7fe288069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:10.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.237+0000 7fe286ffd700 1 --2- 172.21.15.163:0/77167144 >> v2:172.21.15.181:3300/0 conn(0x7fe28815db10 0x7fe288069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:10.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.237+0000 7fe286ffd700 1 -- 172.21.15.163:0/77167144 >> v1:172.21.15.163:6789/0 conn(0x7fe288159890 legacy=0x7fe28815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:10.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.237+0000 7fe286ffd700 1 -- 172.21.15.163:0/77167144 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28815e140 con 0x7fe288159c60 2023-10-09T06:32:10.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.237+0000 7fe2867fc700 1 -- 172.21.15.163:0/77167144 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1354727551 0 0) 0x7fe28815a160 con 0x7fe288159890 2023-10-09T06:32:10.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.238+0000 7fe286ffd700 1 --2- 172.21.15.163:0/77167144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe288159c60 0x7fe28815d5d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe28806a890 tx=0x7fe278009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2a44b9316533cd server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:10.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.238+0000 7fe2867fc700 1 -- 172.21.15.163:0/77167144 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27800b860 con 0x7fe288159c60 2023-10-09T06:32:10.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.238+0000 7fe2867fc700 1 -- 172.21.15.163:0/77167144 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27800b9c0 con 0x7fe288159c60 2023-10-09T06:32:10.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.238+0000 7fe2867fc700 1 -- 172.21.15.163:0/77167144 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27800bd60 con 0x7fe288159c60 2023-10-09T06:32:10.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.238+0000 7fe28db0b700 1 -- 172.21.15.163:0/77167144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe288159c60 msgr2=0x7fe28815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:10.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.238+0000 7fe28db0b700 1 --2- 172.21.15.163:0/77167144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe288159c60 0x7fe28815d5d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe28806a890 tx=0x7fe278009250 comp rx=0 tx=0).stop 2023-10-09T06:32:10.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.238+0000 7fe28db0b700 1 -- 172.21.15.163:0/77167144 shutdown_connections 2023-10-09T06:32:10.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.238+0000 7fe28db0b700 1 --2- 172.21.15.163:0/77167144 >> v2:172.21.15.181:3300/0 conn(0x7fe28815db10 0x7fe288069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:10.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.238+0000 7fe28db0b700 1 --2- 172.21.15.163:0/77167144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe288159c60 0x7fe28815d5d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:10.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.238+0000 7fe28db0b700 1 -- 172.21.15.163:0/77167144 >> 172.21.15.163:0/77167144 conn(0x7fe288068e20 msgr2=0x7fe2880d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:10.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.238+0000 7fe28db0b700 1 -- 172.21.15.163:0/77167144 shutdown_connections 2023-10-09T06:32:10.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.238+0000 7fe28db0b700 1 -- 172.21.15.163:0/77167144 wait complete. 2023-10-09T06:32:10.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.240+0000 7fe28db0b700 1 Processor -- start 2023-10-09T06:32:10.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.240+0000 7fe28db0b700 1 -- start start 2023-10-09T06:32:10.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.240+0000 7fe28db0b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe288159890 0x7fe2880ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:10.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.240+0000 7fe28db0b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe288159c60 0x7fe2880b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:10.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.240+0000 7fe28db0b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe28815db10 0x7fe2880b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:10.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.240+0000 7fe28db0b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28814cdd0 con 0x7fe288159c60 2023-10-09T06:32:10.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.240+0000 7fe28db0b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2880cfe70 con 0x7fe28815db10 2023-10-09T06:32:10.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.240+0000 7fe28db0b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2880b2520 con 0x7fe288159890 2023-10-09T06:32:10.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.240+0000 7fe286ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe288159c60 0x7fe2880b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:10.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.240+0000 7fe286ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe288159c60 0x7fe2880b0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:32824/0 (socket says 172.21.15.163:32824) 2023-10-09T06:32:10.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.240+0000 7fe286ffd700 1 -- 172.21.15.163:0/4186272856 learned_addr learned my addr 172.21.15.163:0/4186272856 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:10.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.241+0000 7fe287fff700 1 --2- 172.21.15.163:0/4186272856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe28815db10 0x7fe2880b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:10.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.241+0000 7fe2877fe700 1 --2- 172.21.15.163:0/4186272856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe288159890 0x7fe2880ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:10.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.241+0000 7fe286ffd700 1 -- 172.21.15.163:0/4186272856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe288159890 msgr2=0x7fe2880ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:10.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.241+0000 7fe286ffd700 1 --2- 172.21.15.163:0/4186272856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe288159890 0x7fe2880ce0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:10.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.241+0000 7fe286ffd700 1 -- 172.21.15.163:0/4186272856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe28815db10 msgr2=0x7fe2880b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:10.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.241+0000 7fe286ffd700 1 --2- 172.21.15.163:0/4186272856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe28815db10 0x7fe2880b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:10.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.241+0000 7fe286ffd700 1 -- 172.21.15.163:0/4186272856 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2780094f0 con 0x7fe288159c60 2023-10-09T06:32:10.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.241+0000 7fe286ffd700 1 --2- 172.21.15.163:0/4186272856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe288159c60 0x7fe2880b0d20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe278009290 tx=0x7fe278017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:10.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.241+0000 7fe27f7fe700 1 -- 172.21.15.163:0/4186272856 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe278026450 con 0x7fe288159c60 2023-10-09T06:32:10.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.241+0000 7fe27f7fe700 1 -- 172.21.15.163:0/4186272856 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2780265b0 con 0x7fe288159c60 2023-10-09T06:32:10.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.241+0000 7fe28db0b700 1 -- 172.21.15.163:0/4186272856 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe28816d560 con 0x7fe288159c60 2023-10-09T06:32:10.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.241+0000 7fe27f7fe700 1 -- 172.21.15.163:0/4186272856 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe278026950 con 0x7fe288159c60 2023-10-09T06:32:10.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.241+0000 7fe28db0b700 1 -- 172.21.15.163:0/4186272856 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe28816da90 con 0x7fe288159c60 2023-10-09T06:32:10.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.242+0000 7fe28db0b700 1 -- 172.21.15.163:0/4186272856 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe28804c3c0 con 0x7fe288159c60 2023-10-09T06:32:10.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.242+0000 7fe27f7fe700 1 -- 172.21.15.163:0/4186272856 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe2780303f0 con 0x7fe288159c60 2023-10-09T06:32:10.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.243+0000 7fe27f7fe700 1 --2- 172.21.15.163:0/4186272856 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe268066f30 0x7fe2680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:10.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.243+0000 7fe27f7fe700 1 -- 172.21.15.163:0/4186272856 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe2780863b0 con 0x7fe288159c60 2023-10-09T06:32:10.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.243+0000 7fe2877fe700 1 --2- 172.21.15.163:0/4186272856 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe268066f30 0x7fe2680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:10.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.243+0000 7fe2877fe700 1 --2- 172.21.15.163:0/4186272856 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe268066f30 0x7fe2680693e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe2880c7420 tx=0x7fe270007bf0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:10.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.247+0000 7fe27f7fe700 1 -- 172.21.15.163:0/4186272856 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2780536a0 con 0x7fe288159c60 2023-10-09T06:32:10.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.426+0000 7fe28db0b700 1 -- 172.21.15.163:0/4186272856 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe288157490 con 0x7fe268066f30 2023-10-09T06:32:10.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.434+0000 7fe27f7fe700 1 -- 172.21.15.163:0/4186272856 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289462 (secure 0 0 0) 0x7fe288157490 con 0x7fe268066f30 2023-10-09T06:32:10.438 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:10.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.437+0000 7fe28db0b700 1 -- 172.21.15.163:0/4186272856 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe268066f30 msgr2=0x7fe2680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:10.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.437+0000 7fe28db0b700 1 --2- 172.21.15.163:0/4186272856 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe268066f30 0x7fe2680693e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe2880c7420 tx=0x7fe270007bf0 comp rx=0 tx=0).stop 2023-10-09T06:32:10.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.438+0000 7fe28db0b700 1 -- 172.21.15.163:0/4186272856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe288159c60 msgr2=0x7fe2880b0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:10.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.438+0000 7fe28db0b700 1 --2- 172.21.15.163:0/4186272856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe288159c60 0x7fe2880b0d20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe278009290 tx=0x7fe278017000 comp rx=0 tx=0).stop 2023-10-09T06:32:10.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.438+0000 7fe28db0b700 1 -- 172.21.15.163:0/4186272856 shutdown_connections 2023-10-09T06:32:10.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.438+0000 7fe28db0b700 1 --2- 172.21.15.163:0/4186272856 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe268066f30 0x7fe2680693e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:10.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.438+0000 7fe28db0b700 1 --2- 172.21.15.163:0/4186272856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe28815db10 0x7fe2880b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:10.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.438+0000 7fe28db0b700 1 --2- 172.21.15.163:0/4186272856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe288159890 0x7fe2880ce0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:10.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.438+0000 7fe28db0b700 1 --2- 172.21.15.163:0/4186272856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe288159c60 0x7fe2880b0d20 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:10.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.438+0000 7fe28db0b700 1 -- 172.21.15.163:0/4186272856 >> 172.21.15.163:0/4186272856 conn(0x7fe288068e20 msgr2=0x7fe2881545d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:10.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.438+0000 7fe28db0b700 1 -- 172.21.15.163:0/4186272856 shutdown_connections 2023-10-09T06:32:10.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.438+0000 7fe28db0b700 1 -- 172.21.15.163:0/4186272856 wait complete. 2023-10-09T06:32:10.442 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:32:10.450 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2023-10-09T06:32:10.320208+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2384296,"kb_used_data":5992,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752590424,"statfs":{"total":773094113280,"available":770652594176,"internally_reserved":0,"allocated":6135808,"data_stored":3256724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.784967"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.547405+0000","last_change":"2023-10-09T06:32:01.546315+0000","last_active":"2023-10-09T06:32:01.547405+0000","last_peered":"2023-10-09T06:32:01.547405+0000","last_clean":"2023-10-09T06:32:01.547405+0000","last_became_active":"2023-10-09T06:32:01.546088+0000","last_became_peered":"2023-10-09T06:32:01.546088+0000","last_unstale":"2023-10-09T06:32:01.547405+0000","last_undegraded":"2023-10-09T06:32:01.547405+0000","last_fullsized":"2023-10-09T06:32:01.547405+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:28:42.282791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557348+0000","last_change":"2023-10-09T06:32:01.549902+0000","last_active":"2023-10-09T06:32:01.557348+0000","last_peered":"2023-10-09T06:32:01.557348+0000","last_clean":"2023-10-09T06:32:01.557348+0000","last_became_active":"2023-10-09T06:32:01.549676+0000","last_became_peered":"2023-10-09T06:32:01.549676+0000","last_unstale":"2023-10-09T06:32:01.557348+0000","last_undegraded":"2023-10-09T06:32:01.557348+0000","last_fullsized":"2023-10-09T06:32:01.557348+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:39:07.097204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.563367+0000","last_change":"2023-10-09T06:32:01.557929+0000","last_active":"2023-10-09T06:32:01.563367+0000","last_peered":"2023-10-09T06:32:01.563367+0000","last_clean":"2023-10-09T06:32:01.563367+0000","last_became_active":"2023-10-09T06:32:01.557615+0000","last_became_peered":"2023-10-09T06:32:01.557615+0000","last_unstale":"2023-10-09T06:32:01.563367+0000","last_undegraded":"2023-10-09T06:32:01.563367+0000","last_fullsized":"2023-10-09T06:32:01.563367+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:23:35.817847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.560323+0000","last_change":"2023-10-09T06:32:01.553722+0000","last_active":"2023-10-09T06:32:01.560323+0000","last_peered":"2023-10-09T06:32:01.560323+0000","last_clean":"2023-10-09T06:32:01.560323+0000","last_became_active":"2023-10-09T06:32:01.553256+0000","last_became_peered":"2023-10-09T06:32:01.553256+0000","last_unstale":"2023-10-09T06:32:01.560323+0000","last_undegraded":"2023-10-09T06:32:01.560323+0000","last_fullsized":"2023-10-09T06:32:01.560323+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:05:58.355776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557481+0000","last_change":"2023-10-09T06:32:01.551457+0000","last_active":"2023-10-09T06:32:01.557481+0000","last_peered":"2023-10-09T06:32:01.557481+0000","last_clean":"2023-10-09T06:32:01.557481+0000","last_became_active":"2023-10-09T06:32:01.551187+0000","last_became_peered":"2023-10-09T06:32:01.551187+0000","last_unstale":"2023-10-09T06:32:01.557481+0000","last_undegraded":"2023-10-09T06:32:01.557481+0000","last_fullsized":"2023-10-09T06:32:01.557481+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:04:48.598681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.554978+0000","last_change":"2023-10-09T06:32:01.547943+0000","last_active":"2023-10-09T06:32:01.554978+0000","last_peered":"2023-10-09T06:32:01.554978+0000","last_clean":"2023-10-09T06:32:01.554978+0000","last_became_active":"2023-10-09T06:32:01.547681+0000","last_became_peered":"2023-10-09T06:32:01.547681+0000","last_unstale":"2023-10-09T06:32:01.554978+0000","last_undegraded":"2023-10-09T06:32:01.554978+0000","last_fullsized":"2023-10-09T06:32:01.554978+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:33:50.683561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.550217+0000","last_change":"2023-10-09T06:32:01.548020+0000","last_active":"2023-10-09T06:32:01.550217+0000","last_peered":"2023-10-09T06:32:01.550217+0000","last_clean":"2023-10-09T06:32:01.550217+0000","last_became_active":"2023-10-09T06:32:01.547616+0000","last_became_peered":"2023-10-09T06:32:01.547616+0000","last_unstale":"2023-10-09T06:32:01.550217+0000","last_undegraded":"2023-10-09T06:32:01.550217+0000","last_fullsized":"2023-10-09T06:32:01.550217+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:56:41.671239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.551257+0000","last_change":"2023-10-09T06:32:01.543275+0000","last_active":"2023-10-09T06:32:01.551257+0000","last_peered":"2023-10-09T06:32:01.551257+0000","last_clean":"2023-10-09T06:32:01.551257+0000","last_became_active":"2023-10-09T06:32:01.542967+0000","last_became_peered":"2023-10-09T06:32:01.542967+0000","last_unstale":"2023-10-09T06:32:01.551257+0000","last_undegraded":"2023-10-09T06:32:01.551257+0000","last_fullsized":"2023-10-09T06:32:01.551257+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:31:03.003654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.551341+0000","last_change":"2023-10-09T06:32:01.548201+0000","last_active":"2023-10-09T06:32:01.551341+0000","last_peered":"2023-10-09T06:32:01.551341+0000","last_clean":"2023-10-09T06:32:01.551341+0000","last_became_active":"2023-10-09T06:32:01.547931+0000","last_became_peered":"2023-10-09T06:32:01.547931+0000","last_unstale":"2023-10-09T06:32:01.551341+0000","last_undegraded":"2023-10-09T06:32:01.551341+0000","last_fullsized":"2023-10-09T06:32:01.551341+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:08:04.692947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.552210+0000","last_change":"2023-10-09T06:32:01.549247+0000","last_active":"2023-10-09T06:32:01.552210+0000","last_peered":"2023-10-09T06:32:01.552210+0000","last_clean":"2023-10-09T06:32:01.552210+0000","last_became_active":"2023-10-09T06:32:01.548950+0000","last_became_peered":"2023-10-09T06:32:01.548950+0000","last_unstale":"2023-10-09T06:32:01.552210+0000","last_undegraded":"2023-10-09T06:32:01.552210+0000","last_fullsized":"2023-10-09T06:32:01.552210+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:54:11.409702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.558101+0000","last_change":"2023-10-09T06:32:01.551494+0000","last_active":"2023-10-09T06:32:01.558101+0000","last_peered":"2023-10-09T06:32:01.558101+0000","last_clean":"2023-10-09T06:32:01.558101+0000","last_became_active":"2023-10-09T06:32:01.551260+0000","last_became_peered":"2023-10-09T06:32:01.551260+0000","last_unstale":"2023-10-09T06:32:01.558101+0000","last_undegraded":"2023-10-09T06:32:01.558101+0000","last_fullsized":"2023-10-09T06:32:01.558101+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:01:36.140759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.563667+0000","last_change":"2023-10-09T06:32:01.559281+0000","last_active":"2023-10-09T06:32:01.563667+0000","last_peered":"2023-10-09T06:32:01.563667+0000","last_clean":"2023-10-09T06:32:01.563667+0000","last_became_active":"2023-10-09T06:32:01.558951+0000","last_became_peered":"2023-10-09T06:32:01.558951+0000","last_unstale":"2023-10-09T06:32:01.563667+0000","last_undegraded":"2023-10-09T06:32:01.563667+0000","last_fullsized":"2023-10-09T06:32:01.563667+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:22:02.577699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.554790+0000","last_change":"2023-10-09T06:32:01.549386+0000","last_active":"2023-10-09T06:32:01.554790+0000","last_peered":"2023-10-09T06:32:01.554790+0000","last_clean":"2023-10-09T06:32:01.554790+0000","last_became_active":"2023-10-09T06:32:01.549019+0000","last_became_peered":"2023-10-09T06:32:01.549019+0000","last_unstale":"2023-10-09T06:32:01.554790+0000","last_undegraded":"2023-10-09T06:32:01.554790+0000","last_fullsized":"2023-10-09T06:32:01.554790+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:40:48.249891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.549090+0000","last_change":"2023-10-09T06:32:01.545232+0000","last_active":"2023-10-09T06:32:01.549090+0000","last_peered":"2023-10-09T06:32:01.549090+0000","last_clean":"2023-10-09T06:32:01.549090+0000","last_became_active":"2023-10-09T06:32:01.544899+0000","last_became_peered":"2023-10-09T06:32:01.544899+0000","last_unstale":"2023-10-09T06:32:01.549090+0000","last_undegraded":"2023-10-09T06:32:01.549090+0000","last_fullsized":"2023-10-09T06:32:01.549090+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:07:33.202557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.560332+0000","last_change":"2023-10-09T06:32:01.551970+0000","last_active":"2023-10-09T06:32:01.560332+0000","last_peered":"2023-10-09T06:32:01.560332+0000","last_clean":"2023-10-09T06:32:01.560332+0000","last_became_active":"2023-10-09T06:32:01.551713+0000","last_became_peered":"2023-10-09T06:32:01.551713+0000","last_unstale":"2023-10-09T06:32:01.560332+0000","last_undegraded":"2023-10-09T06:32:01.560332+0000","last_fullsized":"2023-10-09T06:32:01.560332+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:01:54.333566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557876+0000","last_change":"2023-10-09T06:32:01.550967+0000","last_active":"2023-10-09T06:32:01.557876+0000","last_peered":"2023-10-09T06:32:01.557876+0000","last_clean":"2023-10-09T06:32:01.557876+0000","last_became_active":"2023-10-09T06:32:01.550632+0000","last_became_peered":"2023-10-09T06:32:01.550632+0000","last_unstale":"2023-10-09T06:32:01.557876+0000","last_undegraded":"2023-10-09T06:32:01.557876+0000","last_fullsized":"2023-10-09T06:32:01.557876+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:33:31.855017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.549746+0000","last_change":"2023-10-09T06:32:01.546644+0000","last_active":"2023-10-09T06:32:01.549746+0000","last_peered":"2023-10-09T06:32:01.549746+0000","last_clean":"2023-10-09T06:32:01.549746+0000","last_became_active":"2023-10-09T06:32:01.546386+0000","last_became_peered":"2023-10-09T06:32:01.546386+0000","last_unstale":"2023-10-09T06:32:01.549746+0000","last_undegraded":"2023-10-09T06:32:01.549746+0000","last_fullsized":"2023-10-09T06:32:01.549746+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:36:55.014483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557666+0000","last_change":"2023-10-09T06:32:01.548982+0000","last_active":"2023-10-09T06:32:01.557666+0000","last_peered":"2023-10-09T06:32:01.557666+0000","last_clean":"2023-10-09T06:32:01.557666+0000","last_became_active":"2023-10-09T06:32:01.548720+0000","last_became_peered":"2023-10-09T06:32:01.548720+0000","last_unstale":"2023-10-09T06:32:01.557666+0000","last_undegraded":"2023-10-09T06:32:01.557666+0000","last_fullsized":"2023-10-09T06:32:01.557666+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:45:10.684091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.547399+0000","last_change":"2023-10-09T06:32:01.546674+0000","last_active":"2023-10-09T06:32:01.547399+0000","last_peered":"2023-10-09T06:32:01.547399+0000","last_clean":"2023-10-09T06:32:01.547399+0000","last_became_active":"2023-10-09T06:32:01.546425+0000","last_became_peered":"2023-10-09T06:32:01.546425+0000","last_unstale":"2023-10-09T06:32:01.547399+0000","last_undegraded":"2023-10-09T06:32:01.547399+0000","last_fullsized":"2023-10-09T06:32:01.547399+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:04:49.102247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.549378+0000","last_change":"2023-10-09T06:32:01.544774+0000","last_active":"2023-10-09T06:32:01.549378+0000","last_peered":"2023-10-09T06:32:01.549378+0000","last_clean":"2023-10-09T06:32:01.549378+0000","last_became_active":"2023-10-09T06:32:01.544523+0000","last_became_peered":"2023-10-09T06:32:01.544523+0000","last_unstale":"2023-10-09T06:32:01.549378+0000","last_undegraded":"2023-10-09T06:32:01.549378+0000","last_fullsized":"2023-10-09T06:32:01.549378+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:53:31.570299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.558952+0000","last_change":"2023-10-09T06:32:01.549300+0000","last_active":"2023-10-09T06:32:01.558952+0000","last_peered":"2023-10-09T06:32:01.558952+0000","last_clean":"2023-10-09T06:32:01.558952+0000","last_became_active":"2023-10-09T06:32:01.549065+0000","last_became_peered":"2023-10-09T06:32:01.549065+0000","last_unstale":"2023-10-09T06:32:01.558952+0000","last_undegraded":"2023-10-09T06:32:01.558952+0000","last_fullsized":"2023-10-09T06:32:01.558952+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:33:35.922723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557409+0000","last_change":"2023-10-09T06:32:01.549115+0000","last_active":"2023-10-09T06:32:01.557409+0000","last_peered":"2023-10-09T06:32:01.557409+0000","last_clean":"2023-10-09T06:32:01.557409+0000","last_became_active":"2023-10-09T06:32:01.548747+0000","last_became_peered":"2023-10-09T06:32:01.548747+0000","last_unstale":"2023-10-09T06:32:01.557409+0000","last_undegraded":"2023-10-09T06:32:01.557409+0000","last_fullsized":"2023-10-09T06:32:01.557409+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:05:36.594528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.559914+0000","last_change":"2023-10-09T06:32:01.553968+0000","last_active":"2023-10-09T06:32:01.559914+0000","last_peered":"2023-10-09T06:32:01.559914+0000","last_clean":"2023-10-09T06:32:01.559914+0000","last_became_active":"2023-10-09T06:32:01.553732+0000","last_became_peered":"2023-10-09T06:32:01.553732+0000","last_unstale":"2023-10-09T06:32:01.559914+0000","last_undegraded":"2023-10-09T06:32:01.559914+0000","last_fullsized":"2023-10-09T06:32:01.559914+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:05:58.355776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.552237+0000","last_change":"2023-10-09T06:32:01.546952+0000","last_active":"2023-10-09T06:32:01.552237+0000","last_peered":"2023-10-09T06:32:01.552237+0000","last_clean":"2023-10-09T06:32:01.552237+0000","last_became_active":"2023-10-09T06:32:01.546698+0000","last_became_peered":"2023-10-09T06:32:01.546698+0000","last_unstale":"2023-10-09T06:32:01.552237+0000","last_undegraded":"2023-10-09T06:32:01.552237+0000","last_fullsized":"2023-10-09T06:32:01.552237+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:05:31.370629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557101+0000","last_change":"2023-10-09T06:32:01.549766+0000","last_active":"2023-10-09T06:32:01.557101+0000","last_peered":"2023-10-09T06:32:01.557101+0000","last_clean":"2023-10-09T06:32:01.557101+0000","last_became_active":"2023-10-09T06:32:01.549522+0000","last_became_peered":"2023-10-09T06:32:01.549522+0000","last_unstale":"2023-10-09T06:32:01.557101+0000","last_undegraded":"2023-10-09T06:32:01.557101+0000","last_fullsized":"2023-10-09T06:32:01.557101+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:58:24.857028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.554021+0000","last_change":"2023-10-09T06:32:01.549901+0000","last_active":"2023-10-09T06:32:01.554021+0000","last_peered":"2023-10-09T06:32:01.554021+0000","last_clean":"2023-10-09T06:32:01.554021+0000","last_became_active":"2023-10-09T06:32:01.549644+0000","last_became_peered":"2023-10-09T06:32:01.549644+0000","last_unstale":"2023-10-09T06:32:01.554021+0000","last_undegraded":"2023-10-09T06:32:01.554021+0000","last_fullsized":"2023-10-09T06:32:01.554021+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:58:30.024948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557869+0000","last_change":"2023-10-09T06:32:01.553855+0000","last_active":"2023-10-09T06:32:01.557869+0000","last_peered":"2023-10-09T06:32:01.557869+0000","last_clean":"2023-10-09T06:32:01.557869+0000","last_became_active":"2023-10-09T06:32:01.553375+0000","last_became_peered":"2023-10-09T06:32:01.553375+0000","last_unstale":"2023-10-09T06:32:01.557869+0000","last_undegraded":"2023-10-09T06:32:01.557869+0000","last_fullsized":"2023-10-09T06:32:01.557869+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:12:03.262908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.558086+0000","last_change":"2023-10-09T06:32:01.548558+0000","last_active":"2023-10-09T06:32:01.558086+0000","last_peered":"2023-10-09T06:32:01.558086+0000","last_clean":"2023-10-09T06:32:01.558086+0000","last_became_active":"2023-10-09T06:32:01.548280+0000","last_became_peered":"2023-10-09T06:32:01.548280+0000","last_unstale":"2023-10-09T06:32:01.558086+0000","last_undegraded":"2023-10-09T06:32:01.558086+0000","last_fullsized":"2023-10-09T06:32:01.558086+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:11:26.654414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.552958+0000","last_change":"2023-10-09T06:32:01.547913+0000","last_active":"2023-10-09T06:32:01.552958+0000","last_peered":"2023-10-09T06:32:01.552958+0000","last_clean":"2023-10-09T06:32:01.552958+0000","last_became_active":"2023-10-09T06:32:01.547674+0000","last_became_peered":"2023-10-09T06:32:01.547674+0000","last_unstale":"2023-10-09T06:32:01.552958+0000","last_undegraded":"2023-10-09T06:32:01.552958+0000","last_fullsized":"2023-10-09T06:32:01.552958+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:23:55.906233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.558030+0000","last_change":"2023-10-09T06:32:01.550076+0000","last_active":"2023-10-09T06:32:01.558030+0000","last_peered":"2023-10-09T06:32:01.558030+0000","last_clean":"2023-10-09T06:32:01.558030+0000","last_became_active":"2023-10-09T06:32:01.549826+0000","last_became_peered":"2023-10-09T06:32:01.549826+0000","last_unstale":"2023-10-09T06:32:01.558030+0000","last_undegraded":"2023-10-09T06:32:01.558030+0000","last_fullsized":"2023-10-09T06:32:01.558030+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:33:31.855017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557984+0000","last_change":"2023-10-09T06:32:01.552934+0000","last_active":"2023-10-09T06:32:01.557984+0000","last_peered":"2023-10-09T06:32:01.557984+0000","last_clean":"2023-10-09T06:32:01.557984+0000","last_became_active":"2023-10-09T06:32:01.552675+0000","last_became_peered":"2023-10-09T06:32:01.552675+0000","last_unstale":"2023-10-09T06:32:01.557984+0000","last_undegraded":"2023-10-09T06:32:01.557984+0000","last_fullsized":"2023-10-09T06:32:01.557984+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:25:25.364284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557897+0000","last_change":"2023-10-09T06:32:01.553939+0000","last_active":"2023-10-09T06:32:01.557897+0000","last_peered":"2023-10-09T06:32:01.557897+0000","last_clean":"2023-10-09T06:32:01.557897+0000","last_became_active":"2023-10-09T06:32:01.553385+0000","last_became_peered":"2023-10-09T06:32:01.553385+0000","last_unstale":"2023-10-09T06:32:01.557897+0000","last_undegraded":"2023-10-09T06:32:01.557897+0000","last_fullsized":"2023-10-09T06:32:01.557897+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:58:30.024948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.560995+0000","last_change":"2023-10-09T06:32:01.557999+0000","last_active":"2023-10-09T06:32:01.560995+0000","last_peered":"2023-10-09T06:32:01.560995+0000","last_clean":"2023-10-09T06:32:01.560995+0000","last_became_active":"2023-10-09T06:32:01.557709+0000","last_became_peered":"2023-10-09T06:32:01.557709+0000","last_unstale":"2023-10-09T06:32:01.560995+0000","last_undegraded":"2023-10-09T06:32:01.560995+0000","last_fullsized":"2023-10-09T06:32:01.560995+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:03:46.626123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"50'4","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.564351+0000","last_change":"2023-10-09T06:32:00.545378+0000","last_active":"2023-10-09T06:32:01.564351+0000","last_peered":"2023-10-09T06:32:01.564351+0000","last_clean":"2023-10-09T06:32:01.564351+0000","last_became_active":"2023-10-09T06:32:00.545102+0000","last_became_peered":"2023-10-09T06:32:00.545102+0000","last_unstale":"2023-10-09T06:32:01.564351+0000","last_undegraded":"2023-10-09T06:32:01.564351+0000","last_fullsized":"2023-10-09T06:32:01.564351+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:54:00.414345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.559887+0000","last_change":"2023-10-09T06:32:01.555281+0000","last_active":"2023-10-09T06:32:01.559887+0000","last_peered":"2023-10-09T06:32:01.559887+0000","last_clean":"2023-10-09T06:32:01.559887+0000","last_became_active":"2023-10-09T06:32:01.554604+0000","last_became_peered":"2023-10-09T06:32:01.554604+0000","last_unstale":"2023-10-09T06:32:01.559887+0000","last_undegraded":"2023-10-09T06:32:01.559887+0000","last_fullsized":"2023-10-09T06:32:01.559887+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:00:06.478019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"50'1","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.565941+0000","last_change":"2023-10-09T06:32:00.550462+0000","last_active":"2023-10-09T06:32:01.565941+0000","last_peered":"2023-10-09T06:32:01.565941+0000","last_clean":"2023-10-09T06:32:01.565941+0000","last_became_active":"2023-10-09T06:32:00.549842+0000","last_became_peered":"2023-10-09T06:32:00.549842+0000","last_unstale":"2023-10-09T06:32:01.565941+0000","last_undegraded":"2023-10-09T06:32:01.565941+0000","last_fullsized":"2023-10-09T06:32:01.565941+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:05:57.346759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.558884+0000","last_change":"2023-10-09T06:32:01.554546+0000","last_active":"2023-10-09T06:32:01.558884+0000","last_peered":"2023-10-09T06:32:01.558884+0000","last_clean":"2023-10-09T06:32:01.558884+0000","last_became_active":"2023-10-09T06:32:01.554347+0000","last_became_peered":"2023-10-09T06:32:01.554347+0000","last_unstale":"2023-10-09T06:32:01.558884+0000","last_undegraded":"2023-10-09T06:32:01.558884+0000","last_fullsized":"2023-10-09T06:32:01.558884+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:00:06.478019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"50'2","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.565716+0000","last_change":"2023-10-09T06:32:00.546653+0000","last_active":"2023-10-09T06:32:01.565716+0000","last_peered":"2023-10-09T06:32:01.565716+0000","last_clean":"2023-10-09T06:32:01.565716+0000","last_became_active":"2023-10-09T06:32:00.546397+0000","last_became_peered":"2023-10-09T06:32:00.546397+0000","last_unstale":"2023-10-09T06:32:01.565716+0000","last_undegraded":"2023-10-09T06:32:01.565716+0000","last_fullsized":"2023-10-09T06:32:01.565716+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:53:13.331862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557906+0000","last_change":"2023-10-09T06:32:01.554059+0000","last_active":"2023-10-09T06:32:01.557906+0000","last_peered":"2023-10-09T06:32:01.557906+0000","last_clean":"2023-10-09T06:32:01.557906+0000","last_became_active":"2023-10-09T06:32:01.553777+0000","last_became_peered":"2023-10-09T06:32:01.553777+0000","last_unstale":"2023-10-09T06:32:01.557906+0000","last_undegraded":"2023-10-09T06:32:01.557906+0000","last_fullsized":"2023-10-09T06:32:01.557906+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:15:23.971809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"50'2","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.569382+0000","last_change":"2023-10-09T06:32:00.549247+0000","last_active":"2023-10-09T06:32:01.569382+0000","last_peered":"2023-10-09T06:32:01.569382+0000","last_clean":"2023-10-09T06:32:01.569382+0000","last_became_active":"2023-10-09T06:32:00.548979+0000","last_became_peered":"2023-10-09T06:32:00.548979+0000","last_unstale":"2023-10-09T06:32:01.569382+0000","last_undegraded":"2023-10-09T06:32:01.569382+0000","last_fullsized":"2023-10-09T06:32:01.569382+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:54:10.400685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557390+0000","last_change":"2023-10-09T06:32:01.548960+0000","last_active":"2023-10-09T06:32:01.557390+0000","last_peered":"2023-10-09T06:32:01.557390+0000","last_clean":"2023-10-09T06:32:01.557390+0000","last_became_active":"2023-10-09T06:32:01.548470+0000","last_became_peered":"2023-10-09T06:32:01.548470+0000","last_unstale":"2023-10-09T06:32:01.557390+0000","last_undegraded":"2023-10-09T06:32:01.557390+0000","last_fullsized":"2023-10-09T06:32:01.557390+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:42:18.468732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"50'1","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.561735+0000","last_change":"2023-10-09T06:32:00.550842+0000","last_active":"2023-10-09T06:32:01.561735+0000","last_peered":"2023-10-09T06:32:01.561735+0000","last_clean":"2023-10-09T06:32:01.561735+0000","last_became_active":"2023-10-09T06:32:00.550169+0000","last_became_peered":"2023-10-09T06:32:00.550169+0000","last_unstale":"2023-10-09T06:32:01.561735+0000","last_undegraded":"2023-10-09T06:32:01.561735+0000","last_fullsized":"2023-10-09T06:32:01.561735+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:55:32.071358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.549611+0000","last_change":"2023-10-09T06:32:01.545867+0000","last_active":"2023-10-09T06:32:01.549611+0000","last_peered":"2023-10-09T06:32:01.549611+0000","last_clean":"2023-10-09T06:32:01.549611+0000","last_became_active":"2023-10-09T06:32:01.545598+0000","last_became_peered":"2023-10-09T06:32:01.545598+0000","last_unstale":"2023-10-09T06:32:01.549611+0000","last_undegraded":"2023-10-09T06:32:01.549611+0000","last_fullsized":"2023-10-09T06:32:01.549611+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:59:16.905155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"50'1","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.562680+0000","last_change":"2023-10-09T06:32:00.543541+0000","last_active":"2023-10-09T06:32:01.562680+0000","last_peered":"2023-10-09T06:32:01.562680+0000","last_clean":"2023-10-09T06:32:01.562680+0000","last_became_active":"2023-10-09T06:32:00.543306+0000","last_became_peered":"2023-10-09T06:32:00.543306+0000","last_unstale":"2023-10-09T06:32:01.562680+0000","last_undegraded":"2023-10-09T06:32:01.562680+0000","last_fullsized":"2023-10-09T06:32:01.562680+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:20:24.757239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.561463+0000","last_change":"2023-10-09T06:32:01.557153+0000","last_active":"2023-10-09T06:32:01.561463+0000","last_peered":"2023-10-09T06:32:01.561463+0000","last_clean":"2023-10-09T06:32:01.561463+0000","last_became_active":"2023-10-09T06:32:01.556893+0000","last_became_peered":"2023-10-09T06:32:01.556893+0000","last_unstale":"2023-10-09T06:32:01.561463+0000","last_undegraded":"2023-10-09T06:32:01.561463+0000","last_fullsized":"2023-10-09T06:32:01.561463+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:18:27.289684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.545979+0000","last_change":"2023-10-09T06:32:00.546348+0000","last_active":"2023-10-09T06:32:01.545979+0000","last_peered":"2023-10-09T06:32:01.545979+0000","last_clean":"2023-10-09T06:32:01.545979+0000","last_became_active":"2023-10-09T06:32:00.546066+0000","last_became_peered":"2023-10-09T06:32:00.546066+0000","last_unstale":"2023-10-09T06:32:01.545979+0000","last_undegraded":"2023-10-09T06:32:01.545979+0000","last_fullsized":"2023-10-09T06:32:01.545979+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:01:53.324549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.549630+0000","last_change":"2023-10-09T06:32:01.543028+0000","last_active":"2023-10-09T06:32:01.549630+0000","last_peered":"2023-10-09T06:32:01.549630+0000","last_clean":"2023-10-09T06:32:01.549630+0000","last_became_active":"2023-10-09T06:32:01.542729+0000","last_became_peered":"2023-10-09T06:32:01.542729+0000","last_unstale":"2023-10-09T06:32:01.549630+0000","last_undegraded":"2023-10-09T06:32:01.549630+0000","last_fullsized":"2023-10-09T06:32:01.549630+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:00:16.216853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.540538+0000","last_change":"2023-10-09T06:32:00.549861+0000","last_active":"2023-10-09T06:32:01.540538+0000","last_peered":"2023-10-09T06:32:01.540538+0000","last_clean":"2023-10-09T06:32:01.540538+0000","last_became_active":"2023-10-09T06:32:00.549591+0000","last_became_peered":"2023-10-09T06:32:00.549591+0000","last_unstale":"2023-10-09T06:32:01.540538+0000","last_undegraded":"2023-10-09T06:32:01.540538+0000","last_fullsized":"2023-10-09T06:32:01.540538+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:38:45.645525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.561623+0000","last_change":"2023-10-09T06:32:01.553913+0000","last_active":"2023-10-09T06:32:01.561623+0000","last_peered":"2023-10-09T06:32:01.561623+0000","last_clean":"2023-10-09T06:32:01.561623+0000","last_became_active":"2023-10-09T06:32:01.553569+0000","last_became_peered":"2023-10-09T06:32:01.553569+0000","last_unstale":"2023-10-09T06:32:01.561623+0000","last_undegraded":"2023-10-09T06:32:01.561623+0000","last_fullsized":"2023-10-09T06:32:01.561623+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:50:43.178529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.544752+0000","last_change":"2023-10-09T06:32:00.555261+0000","last_active":"2023-10-09T06:32:01.544752+0000","last_peered":"2023-10-09T06:32:01.544752+0000","last_clean":"2023-10-09T06:32:01.544752+0000","last_became_active":"2023-10-09T06:32:00.555013+0000","last_became_peered":"2023-10-09T06:32:00.555013+0000","last_unstale":"2023-10-09T06:32:01.544752+0000","last_undegraded":"2023-10-09T06:32:01.544752+0000","last_fullsized":"2023-10-09T06:32:01.544752+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:54:10.400685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.560665+0000","last_change":"2023-10-09T06:32:01.554821+0000","last_active":"2023-10-09T06:32:01.560665+0000","last_peered":"2023-10-09T06:32:01.560665+0000","last_clean":"2023-10-09T06:32:01.560665+0000","last_became_active":"2023-10-09T06:32:01.554527+0000","last_became_peered":"2023-10-09T06:32:01.554527+0000","last_unstale":"2023-10-09T06:32:01.560665+0000","last_undegraded":"2023-10-09T06:32:01.560665+0000","last_fullsized":"2023-10-09T06:32:01.560665+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:07:21.526566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.536545+0000","last_change":"2023-10-09T06:32:00.549017+0000","last_active":"2023-10-09T06:32:01.536545+0000","last_peered":"2023-10-09T06:32:01.536545+0000","last_clean":"2023-10-09T06:32:01.536545+0000","last_became_active":"2023-10-09T06:32:00.548751+0000","last_became_peered":"2023-10-09T06:32:00.548751+0000","last_unstale":"2023-10-09T06:32:01.536545+0000","last_undegraded":"2023-10-09T06:32:01.536545+0000","last_fullsized":"2023-10-09T06:32:01.536545+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:15:22.962792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.552305+0000","last_change":"2023-10-09T06:32:01.549865+0000","last_active":"2023-10-09T06:32:01.552305+0000","last_peered":"2023-10-09T06:32:01.552305+0000","last_clean":"2023-10-09T06:32:01.552305+0000","last_became_active":"2023-10-09T06:32:01.549634+0000","last_became_peered":"2023-10-09T06:32:01.549634+0000","last_unstale":"2023-10-09T06:32:01.552305+0000","last_undegraded":"2023-10-09T06:32:01.552305+0000","last_fullsized":"2023-10-09T06:32:01.552305+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:08:04.692947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"50'3","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.566006+0000","last_change":"2023-10-09T06:32:00.555635+0000","last_active":"2023-10-09T06:32:01.566006+0000","last_peered":"2023-10-09T06:32:01.566006+0000","last_clean":"2023-10-09T06:32:01.566006+0000","last_became_active":"2023-10-09T06:32:00.555298+0000","last_became_peered":"2023-10-09T06:32:00.555298+0000","last_unstale":"2023-10-09T06:32:01.566006+0000","last_undegraded":"2023-10-09T06:32:01.566006+0000","last_fullsized":"2023-10-09T06:32:01.566006+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:01:53.324549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.558868+0000","last_change":"2023-10-09T06:32:01.553480+0000","last_active":"2023-10-09T06:32:01.558868+0000","last_peered":"2023-10-09T06:32:01.558868+0000","last_clean":"2023-10-09T06:32:01.558868+0000","last_became_active":"2023-10-09T06:32:01.552758+0000","last_became_peered":"2023-10-09T06:32:01.552758+0000","last_unstale":"2023-10-09T06:32:01.558868+0000","last_undegraded":"2023-10-09T06:32:01.558868+0000","last_fullsized":"2023-10-09T06:32:01.558868+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:15:00.958265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"50'6","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.566725+0000","last_change":"2023-10-09T06:32:00.545191+0000","last_active":"2023-10-09T06:32:01.566725+0000","last_peered":"2023-10-09T06:32:01.566725+0000","last_clean":"2023-10-09T06:32:01.566725+0000","last_became_active":"2023-10-09T06:32:00.544858+0000","last_became_peered":"2023-10-09T06:32:00.544858+0000","last_unstale":"2023-10-09T06:32:01.566725+0000","last_undegraded":"2023-10-09T06:32:01.566725+0000","last_fullsized":"2023-10-09T06:32:01.566725+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:19:47.545066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.559010+0000","last_change":"2023-10-09T06:32:01.553317+0000","last_active":"2023-10-09T06:32:01.559010+0000","last_peered":"2023-10-09T06:32:01.559010+0000","last_clean":"2023-10-09T06:32:01.559010+0000","last_became_active":"2023-10-09T06:32:01.553037+0000","last_became_peered":"2023-10-09T06:32:01.553037+0000","last_unstale":"2023-10-09T06:32:01.559010+0000","last_undegraded":"2023-10-09T06:32:01.559010+0000","last_fullsized":"2023-10-09T06:32:01.559010+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:33:31.855017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"50'2","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.565830+0000","last_change":"2023-10-09T06:32:00.550589+0000","last_active":"2023-10-09T06:32:01.565830+0000","last_peered":"2023-10-09T06:32:01.565830+0000","last_clean":"2023-10-09T06:32:01.565830+0000","last_became_active":"2023-10-09T06:32:00.549957+0000","last_became_peered":"2023-10-09T06:32:00.549957+0000","last_unstale":"2023-10-09T06:32:01.565830+0000","last_undegraded":"2023-10-09T06:32:01.565830+0000","last_fullsized":"2023-10-09T06:32:01.565830+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:22:01.568682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.565148+0000","last_change":"2023-10-09T06:32:01.560174+0000","last_active":"2023-10-09T06:32:01.565148+0000","last_peered":"2023-10-09T06:32:01.565148+0000","last_clean":"2023-10-09T06:32:01.565148+0000","last_became_active":"2023-10-09T06:32:01.559130+0000","last_became_peered":"2023-10-09T06:32:01.559130+0000","last_unstale":"2023-10-09T06:32:01.565148+0000","last_undegraded":"2023-10-09T06:32:01.565148+0000","last_fullsized":"2023-10-09T06:32:01.565148+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:15:23.971809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.539904+0000","last_change":"2023-10-09T06:32:00.548780+0000","last_active":"2023-10-09T06:32:01.539904+0000","last_peered":"2023-10-09T06:32:01.539904+0000","last_clean":"2023-10-09T06:32:01.539904+0000","last_became_active":"2023-10-09T06:32:00.548527+0000","last_became_peered":"2023-10-09T06:32:00.548527+0000","last_unstale":"2023-10-09T06:32:01.539904+0000","last_undegraded":"2023-10-09T06:32:01.539904+0000","last_fullsized":"2023-10-09T06:32:01.539904+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:20:07.450666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.551125+0000","last_change":"2023-10-09T06:32:01.549607+0000","last_active":"2023-10-09T06:32:01.551125+0000","last_peered":"2023-10-09T06:32:01.551125+0000","last_clean":"2023-10-09T06:32:01.551125+0000","last_became_active":"2023-10-09T06:32:01.549319+0000","last_became_peered":"2023-10-09T06:32:01.549319+0000","last_unstale":"2023-10-09T06:32:01.551125+0000","last_undegraded":"2023-10-09T06:32:01.551125+0000","last_fullsized":"2023-10-09T06:32:01.551125+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:54:11.409702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.546061+0000","last_change":"2023-10-09T06:32:00.549229+0000","last_active":"2023-10-09T06:32:01.546061+0000","last_peered":"2023-10-09T06:32:01.546061+0000","last_clean":"2023-10-09T06:32:01.546061+0000","last_became_active":"2023-10-09T06:32:00.548942+0000","last_became_peered":"2023-10-09T06:32:00.548942+0000","last_unstale":"2023-10-09T06:32:01.546061+0000","last_undegraded":"2023-10-09T06:32:01.546061+0000","last_fullsized":"2023-10-09T06:32:01.546061+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:08:03.683930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.559890+0000","last_change":"2023-10-09T06:32:01.553501+0000","last_active":"2023-10-09T06:32:01.559890+0000","last_peered":"2023-10-09T06:32:01.559890+0000","last_clean":"2023-10-09T06:32:01.559890+0000","last_became_active":"2023-10-09T06:32:01.552951+0000","last_became_peered":"2023-10-09T06:32:01.552951+0000","last_unstale":"2023-10-09T06:32:01.559890+0000","last_undegraded":"2023-10-09T06:32:01.559890+0000","last_fullsized":"2023-10-09T06:32:01.559890+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:03:11.998248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"50'8","reported_seq":18,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.566201+0000","last_change":"2023-10-09T06:32:00.556211+0000","last_active":"2023-10-09T06:32:01.566201+0000","last_peered":"2023-10-09T06:32:01.566201+0000","last_clean":"2023-10-09T06:32:01.566201+0000","last_became_active":"2023-10-09T06:32:00.556012+0000","last_became_peered":"2023-10-09T06:32:00.556012+0000","last_unstale":"2023-10-09T06:32:01.566201+0000","last_undegraded":"2023-10-09T06:32:01.566201+0000","last_fullsized":"2023-10-09T06:32:01.566201+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:56:40.662222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.558277+0000","last_change":"2023-10-09T06:32:01.554064+0000","last_active":"2023-10-09T06:32:01.558277+0000","last_peered":"2023-10-09T06:32:01.558277+0000","last_clean":"2023-10-09T06:32:01.558277+0000","last_became_active":"2023-10-09T06:32:01.553854+0000","last_became_peered":"2023-10-09T06:32:01.553854+0000","last_unstale":"2023-10-09T06:32:01.558277+0000","last_undegraded":"2023-10-09T06:32:01.558277+0000","last_fullsized":"2023-10-09T06:32:01.558277+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:03:46.626123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"50'1","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.565659+0000","last_change":"2023-10-09T06:32:00.549311+0000","last_active":"2023-10-09T06:32:01.565659+0000","last_peered":"2023-10-09T06:32:01.565659+0000","last_clean":"2023-10-09T06:32:01.565659+0000","last_became_active":"2023-10-09T06:32:00.549080+0000","last_became_peered":"2023-10-09T06:32:00.549080+0000","last_unstale":"2023-10-09T06:32:01.565659+0000","last_undegraded":"2023-10-09T06:32:01.565659+0000","last_fullsized":"2023-10-09T06:32:01.565659+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:33:30.846000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.559243+0000","last_change":"2023-10-09T06:32:01.553252+0000","last_active":"2023-10-09T06:32:01.559243+0000","last_peered":"2023-10-09T06:32:01.559243+0000","last_clean":"2023-10-09T06:32:01.559243+0000","last_became_active":"2023-10-09T06:32:01.552892+0000","last_became_peered":"2023-10-09T06:32:01.552892+0000","last_unstale":"2023-10-09T06:32:01.559243+0000","last_undegraded":"2023-10-09T06:32:01.559243+0000","last_fullsized":"2023-10-09T06:32:01.559243+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:15:23.971809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"50'2","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.566069+0000","last_change":"2023-10-09T06:32:00.547761+0000","last_active":"2023-10-09T06:32:01.566069+0000","last_peered":"2023-10-09T06:32:01.566069+0000","last_clean":"2023-10-09T06:32:01.566069+0000","last_became_active":"2023-10-09T06:32:00.547505+0000","last_became_peered":"2023-10-09T06:32:00.547505+0000","last_unstale":"2023-10-09T06:32:01.566069+0000","last_undegraded":"2023-10-09T06:32:01.566069+0000","last_fullsized":"2023-10-09T06:32:01.566069+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:50:59.843352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557563+0000","last_change":"2023-10-09T06:32:01.551087+0000","last_active":"2023-10-09T06:32:01.557563+0000","last_peered":"2023-10-09T06:32:01.557563+0000","last_clean":"2023-10-09T06:32:01.557563+0000","last_became_active":"2023-10-09T06:32:01.550836+0000","last_became_peered":"2023-10-09T06:32:01.550836+0000","last_unstale":"2023-10-09T06:32:01.557563+0000","last_undegraded":"2023-10-09T06:32:01.557563+0000","last_fullsized":"2023-10-09T06:32:01.557563+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:58:42.426644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"50'1","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.566771+0000","last_change":"2023-10-09T06:32:00.549151+0000","last_active":"2023-10-09T06:32:01.566771+0000","last_peered":"2023-10-09T06:32:01.566771+0000","last_clean":"2023-10-09T06:32:01.566771+0000","last_became_active":"2023-10-09T06:32:00.548893+0000","last_became_peered":"2023-10-09T06:32:00.548893+0000","last_unstale":"2023-10-09T06:32:01.566771+0000","last_undegraded":"2023-10-09T06:32:01.566771+0000","last_fullsized":"2023-10-09T06:32:01.566771+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:07:32.193540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.558887+0000","last_change":"2023-10-09T06:32:01.553538+0000","last_active":"2023-10-09T06:32:01.558887+0000","last_peered":"2023-10-09T06:32:01.558887+0000","last_clean":"2023-10-09T06:32:01.558887+0000","last_became_active":"2023-10-09T06:32:01.552796+0000","last_became_peered":"2023-10-09T06:32:01.552796+0000","last_unstale":"2023-10-09T06:32:01.558887+0000","last_undegraded":"2023-10-09T06:32:01.558887+0000","last_fullsized":"2023-10-09T06:32:01.558887+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:01:34.922109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.541180+0000","last_change":"2023-10-09T06:32:00.550205+0000","last_active":"2023-10-09T06:32:01.541180+0000","last_peered":"2023-10-09T06:32:01.541180+0000","last_clean":"2023-10-09T06:32:01.541180+0000","last_became_active":"2023-10-09T06:32:00.549939+0000","last_became_peered":"2023-10-09T06:32:00.549939+0000","last_unstale":"2023-10-09T06:32:01.541180+0000","last_undegraded":"2023-10-09T06:32:01.541180+0000","last_fullsized":"2023-10-09T06:32:01.541180+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:18:11.772295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.558989+0000","last_change":"2023-10-09T06:32:01.553159+0000","last_active":"2023-10-09T06:32:01.558989+0000","last_peered":"2023-10-09T06:32:01.558989+0000","last_clean":"2023-10-09T06:32:01.558989+0000","last_became_active":"2023-10-09T06:32:01.552830+0000","last_became_peered":"2023-10-09T06:32:01.552830+0000","last_unstale":"2023-10-09T06:32:01.558989+0000","last_undegraded":"2023-10-09T06:32:01.558989+0000","last_fullsized":"2023-10-09T06:32:01.558989+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:33:50.683561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"50'2","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.565580+0000","last_change":"2023-10-09T06:32:00.550579+0000","last_active":"2023-10-09T06:32:01.565580+0000","last_peered":"2023-10-09T06:32:01.565580+0000","last_clean":"2023-10-09T06:32:01.565580+0000","last_became_active":"2023-10-09T06:32:00.549929+0000","last_became_peered":"2023-10-09T06:32:00.549929+0000","last_unstale":"2023-10-09T06:32:01.565580+0000","last_undegraded":"2023-10-09T06:32:01.565580+0000","last_fullsized":"2023-10-09T06:32:01.565580+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:03:10.989230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557785+0000","last_change":"2023-10-09T06:32:01.552855+0000","last_active":"2023-10-09T06:32:01.557785+0000","last_peered":"2023-10-09T06:32:01.557785+0000","last_clean":"2023-10-09T06:32:01.557785+0000","last_became_active":"2023-10-09T06:32:01.550520+0000","last_became_peered":"2023-10-09T06:32:01.550520+0000","last_unstale":"2023-10-09T06:32:01.557785+0000","last_undegraded":"2023-10-09T06:32:01.557785+0000","last_fullsized":"2023-10-09T06:32:01.557785+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:07:33.202557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.544170+0000","last_change":"2023-10-09T06:32:00.549467+0000","last_active":"2023-10-09T06:32:01.544170+0000","last_peered":"2023-10-09T06:32:01.544170+0000","last_clean":"2023-10-09T06:32:01.544170+0000","last_became_active":"2023-10-09T06:32:00.549177+0000","last_became_peered":"2023-10-09T06:32:00.549177+0000","last_unstale":"2023-10-09T06:32:01.544170+0000","last_undegraded":"2023-10-09T06:32:01.544170+0000","last_fullsized":"2023-10-09T06:32:01.544170+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:13:58.204471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.558523+0000","last_change":"2023-10-09T06:32:01.557336+0000","last_active":"2023-10-09T06:32:01.558523+0000","last_peered":"2023-10-09T06:32:01.558523+0000","last_clean":"2023-10-09T06:32:01.558523+0000","last_became_active":"2023-10-09T06:32:01.557103+0000","last_became_peered":"2023-10-09T06:32:01.557103+0000","last_unstale":"2023-10-09T06:32:01.558523+0000","last_undegraded":"2023-10-09T06:32:01.558523+0000","last_fullsized":"2023-10-09T06:32:01.558523+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:22:02.577699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.545523+0000","last_change":"2023-10-09T06:32:00.552189+0000","last_active":"2023-10-09T06:32:01.545523+0000","last_peered":"2023-10-09T06:32:01.545523+0000","last_clean":"2023-10-09T06:32:01.545523+0000","last_became_active":"2023-10-09T06:32:00.551888+0000","last_became_peered":"2023-10-09T06:32:00.551888+0000","last_unstale":"2023-10-09T06:32:01.545523+0000","last_undegraded":"2023-10-09T06:32:01.545523+0000","last_fullsized":"2023-10-09T06:32:01.545523+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:53:13.331862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.563713+0000","last_change":"2023-10-09T06:32:01.559460+0000","last_active":"2023-10-09T06:32:01.563713+0000","last_peered":"2023-10-09T06:32:01.563713+0000","last_clean":"2023-10-09T06:32:01.563713+0000","last_became_active":"2023-10-09T06:32:01.559193+0000","last_became_peered":"2023-10-09T06:32:01.559193+0000","last_unstale":"2023-10-09T06:32:01.563713+0000","last_undegraded":"2023-10-09T06:32:01.563713+0000","last_fullsized":"2023-10-09T06:32:01.563713+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:03:11.998248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"50'2","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.563021+0000","last_change":"2023-10-09T06:32:00.552045+0000","last_active":"2023-10-09T06:32:01.563021+0000","last_peered":"2023-10-09T06:32:01.563021+0000","last_clean":"2023-10-09T06:32:01.563021+0000","last_became_active":"2023-10-09T06:32:00.551778+0000","last_became_peered":"2023-10-09T06:32:00.551778+0000","last_unstale":"2023-10-09T06:32:01.563021+0000","last_undegraded":"2023-10-09T06:32:01.563021+0000","last_fullsized":"2023-10-09T06:32:01.563021+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:34:11.248280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.563834+0000","last_change":"2023-10-09T06:32:01.559146+0000","last_active":"2023-10-09T06:32:01.563834+0000","last_peered":"2023-10-09T06:32:01.563834+0000","last_clean":"2023-10-09T06:32:01.563834+0000","last_became_active":"2023-10-09T06:32:01.558847+0000","last_became_peered":"2023-10-09T06:32:01.558847+0000","last_unstale":"2023-10-09T06:32:01.563834+0000","last_undegraded":"2023-10-09T06:32:01.563834+0000","last_fullsized":"2023-10-09T06:32:01.563834+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:03:46.626123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.540913+0000","last_change":"2023-10-09T06:32:00.542936+0000","last_active":"2023-10-09T06:32:01.540913+0000","last_peered":"2023-10-09T06:32:01.540913+0000","last_clean":"2023-10-09T06:32:01.540913+0000","last_became_active":"2023-10-09T06:32:00.542687+0000","last_became_peered":"2023-10-09T06:32:00.542687+0000","last_unstale":"2023-10-09T06:32:01.540913+0000","last_undegraded":"2023-10-09T06:32:01.540913+0000","last_fullsized":"2023-10-09T06:32:01.540913+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:19:47.545066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.552287+0000","last_change":"2023-10-09T06:32:01.549830+0000","last_active":"2023-10-09T06:32:01.552287+0000","last_peered":"2023-10-09T06:32:01.552287+0000","last_clean":"2023-10-09T06:32:01.552287+0000","last_became_active":"2023-10-09T06:32:01.549593+0000","last_became_peered":"2023-10-09T06:32:01.549593+0000","last_unstale":"2023-10-09T06:32:01.552287+0000","last_undegraded":"2023-10-09T06:32:01.552287+0000","last_fullsized":"2023-10-09T06:32:01.552287+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:07:33.202557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.539561+0000","last_change":"2023-10-09T06:32:00.544144+0000","last_active":"2023-10-09T06:32:01.539561+0000","last_peered":"2023-10-09T06:32:01.539561+0000","last_clean":"2023-10-09T06:32:01.539561+0000","last_became_active":"2023-10-09T06:32:00.543899+0000","last_became_peered":"2023-10-09T06:32:00.543899+0000","last_unstale":"2023-10-09T06:32:01.539561+0000","last_undegraded":"2023-10-09T06:32:01.539561+0000","last_fullsized":"2023-10-09T06:32:01.539561+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:20:24.504501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557888+0000","last_change":"2023-10-09T06:32:01.554041+0000","last_active":"2023-10-09T06:32:01.557888+0000","last_peered":"2023-10-09T06:32:01.557888+0000","last_clean":"2023-10-09T06:32:01.557888+0000","last_became_active":"2023-10-09T06:32:01.553605+0000","last_became_peered":"2023-10-09T06:32:01.553605+0000","last_unstale":"2023-10-09T06:32:01.557888+0000","last_undegraded":"2023-10-09T06:32:01.557888+0000","last_fullsized":"2023-10-09T06:32:01.557888+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:07:21.526566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.547215+0000","last_change":"2023-10-09T06:32:00.546419+0000","last_active":"2023-10-09T06:32:01.547215+0000","last_peered":"2023-10-09T06:32:01.547215+0000","last_clean":"2023-10-09T06:32:01.547215+0000","last_became_active":"2023-10-09T06:32:00.545725+0000","last_became_peered":"2023-10-09T06:32:00.545725+0000","last_unstale":"2023-10-09T06:32:01.547215+0000","last_undegraded":"2023-10-09T06:32:01.547215+0000","last_fullsized":"2023-10-09T06:32:01.547215+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:56:40.662222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.557427+0000","last_change":"2023-10-09T06:32:01.552019+0000","last_active":"2023-10-09T06:32:01.557427+0000","last_peered":"2023-10-09T06:32:01.557427+0000","last_clean":"2023-10-09T06:32:01.557427+0000","last_became_active":"2023-10-09T06:32:01.551738+0000","last_became_peered":"2023-10-09T06:32:01.551738+0000","last_unstale":"2023-10-09T06:32:01.557427+0000","last_undegraded":"2023-10-09T06:32:01.557427+0000","last_fullsized":"2023-10-09T06:32:01.557427+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:59:04.361941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.539629+0000","last_change":"2023-10-09T06:32:00.547984+0000","last_active":"2023-10-09T06:32:01.539629+0000","last_peered":"2023-10-09T06:32:01.539629+0000","last_clean":"2023-10-09T06:32:01.539629+0000","last_became_active":"2023-10-09T06:32:00.547757+0000","last_became_peered":"2023-10-09T06:32:00.547757+0000","last_unstale":"2023-10-09T06:32:01.539629+0000","last_undegraded":"2023-10-09T06:32:01.539629+0000","last_fullsized":"2023-10-09T06:32:01.539629+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:19:47.545066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.558973+0000","last_change":"2023-10-09T06:32:01.549769+0000","last_active":"2023-10-09T06:32:01.558973+0000","last_peered":"2023-10-09T06:32:01.558973+0000","last_clean":"2023-10-09T06:32:01.558973+0000","last_became_active":"2023-10-09T06:32:01.549499+0000","last_became_peered":"2023-10-09T06:32:01.549499+0000","last_unstale":"2023-10-09T06:32:01.558973+0000","last_undegraded":"2023-10-09T06:32:01.558973+0000","last_fullsized":"2023-10-09T06:32:01.558973+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:58:30.024948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.540704+0000","last_change":"2023-10-09T06:32:00.552838+0000","last_active":"2023-10-09T06:32:01.540704+0000","last_peered":"2023-10-09T06:32:01.540704+0000","last_clean":"2023-10-09T06:32:01.540704+0000","last_became_active":"2023-10-09T06:32:00.552539+0000","last_became_peered":"2023-10-09T06:32:00.552539+0000","last_unstale":"2023-10-09T06:32:01.540704+0000","last_undegraded":"2023-10-09T06:32:01.540704+0000","last_fullsized":"2023-10-09T06:32:01.540704+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:34:11.248280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.558072+0000","last_change":"2023-10-09T06:32:01.550552+0000","last_active":"2023-10-09T06:32:01.558072+0000","last_peered":"2023-10-09T06:32:01.558072+0000","last_clean":"2023-10-09T06:32:01.558072+0000","last_became_active":"2023-10-09T06:32:01.550269+0000","last_became_peered":"2023-10-09T06:32:01.550269+0000","last_unstale":"2023-10-09T06:32:01.558072+0000","last_undegraded":"2023-10-09T06:32:01.558072+0000","last_fullsized":"2023-10-09T06:32:01.558072+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:01:54.333566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.544287+0000","last_change":"2023-10-09T06:32:00.546841+0000","last_active":"2023-10-09T06:32:01.544287+0000","last_peered":"2023-10-09T06:32:01.544287+0000","last_clean":"2023-10-09T06:32:01.544287+0000","last_became_active":"2023-10-09T06:32:00.546540+0000","last_became_peered":"2023-10-09T06:32:00.546540+0000","last_unstale":"2023-10-09T06:32:01.544287+0000","last_undegraded":"2023-10-09T06:32:01.544287+0000","last_fullsized":"2023-10-09T06:32:01.544287+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:38:45.645525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.549652+0000","last_change":"2023-10-09T06:32:01.544400+0000","last_active":"2023-10-09T06:32:01.549652+0000","last_peered":"2023-10-09T06:32:01.549652+0000","last_clean":"2023-10-09T06:32:01.549652+0000","last_became_active":"2023-10-09T06:32:01.542947+0000","last_became_peered":"2023-10-09T06:32:01.542947+0000","last_unstale":"2023-10-09T06:32:01.549652+0000","last_undegraded":"2023-10-09T06:32:01.549652+0000","last_fullsized":"2023-10-09T06:32:01.549652+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:34:46.591947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.537115+0000","last_change":"2023-10-09T06:32:00.551080+0000","last_active":"2023-10-09T06:32:01.537115+0000","last_peered":"2023-10-09T06:32:01.537115+0000","last_clean":"2023-10-09T06:32:01.537115+0000","last_became_active":"2023-10-09T06:32:00.550608+0000","last_became_peered":"2023-10-09T06:32:00.550608+0000","last_unstale":"2023-10-09T06:32:01.537115+0000","last_undegraded":"2023-10-09T06:32:01.537115+0000","last_fullsized":"2023-10-09T06:32:01.537115+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:58:29.015931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.551290+0000","last_change":"2023-10-09T06:32:01.545252+0000","last_active":"2023-10-09T06:32:01.551290+0000","last_peered":"2023-10-09T06:32:01.551290+0000","last_clean":"2023-10-09T06:32:01.551290+0000","last_became_active":"2023-10-09T06:32:01.545003+0000","last_became_peered":"2023-10-09T06:32:01.545003+0000","last_unstale":"2023-10-09T06:32:01.551290+0000","last_undegraded":"2023-10-09T06:32:01.551290+0000","last_fullsized":"2023-10-09T06:32:01.551290+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:00.527624+0000","last_clean_scrub_stamp":"2023-10-09T06:32:00.527624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:24:43.541920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.540056+0000","last_change":"2023-10-09T06:32:00.548403+0000","last_active":"2023-10-09T06:32:01.540056+0000","last_peered":"2023-10-09T06:32:01.540056+0000","last_clean":"2023-10-09T06:32:01.540056+0000","last_became_active":"2023-10-09T06:32:00.548144+0000","last_became_peered":"2023-10-09T06:32:00.548144+0000","last_unstale":"2023-10-09T06:32:01.540056+0000","last_undegraded":"2023-10-09T06:32:01.540056+0000","last_fullsized":"2023-10-09T06:32:01.540056+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:31:59.518607+0000","last_clean_scrub_stamp":"2023-10-09T06:31:59.518607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:30:34.904806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":190,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-09T06:32:01.542789+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:32:01.542789+0000","last_peered":"2023-10-09T06:32:01.542789+0000","last_clean":"2023-10-09T06:32:01.542789+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:32:01.542789+0000","last_undegraded":"2023-10-09T06:32:01.542789+0000","last_fullsized":"2023-10-09T06:32:01.542789+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298472,"kb_used_data":1184,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073368,"statfs":{"total":96636764160,"available":96331128832,"internally_reserved":0,"allocated":1212416,"data_stored":849450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.438},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.32100000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.45900000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.504},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.61699999999999999}]},{"osd":3,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.65800000000000003}]},{"osd":4,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.63200000000000001}]},{"osd":5,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.59299999999999997}]},{"osd":6,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.67100000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297884,"kb_used_data":596,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073956,"statfs":{"total":96636764160,"available":96331730944,"internally_reserved":0,"allocated":610304,"data_stored":258922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.61299999999999999}]},{"osd":1,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.54900000000000004}]},{"osd":2,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.66700000000000004}]},{"osd":3,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.503},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":4.7510000000000003,"5min":4.7510000000000003,"15min":4.7510000000000003},"last":0.59299999999999997}]},{"osd":4,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.65200000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.73699999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.67900000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297888,"kb_used_data":600,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073952,"statfs":{"total":96636764160,"available":96331726848,"internally_reserved":0,"allocated":614400,"data_stored":259012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.31,"15min":0.31},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.30199999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.32700000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":5.8529999999999998,"5min":5.8529999999999998,"15min":5.8529999999999998},"last":0.25900000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.627},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.61699999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":5.9459999999999997,"5min":5.9459999999999997,"15min":5.9459999999999997},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.57999999999999996}]},{"osd":6,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":5.9160000000000004,"5min":5.9160000000000004,"15min":5.9160000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.48799999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":5.9729999999999999,"5min":5.9729999999999999,"15min":5.9729999999999999},"last":0.50700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298480,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073360,"statfs":{"total":96636764160,"available":96331120640,"internally_reserved":0,"allocated":1220608,"data_stored":851380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.438,"5min":0.438,"15min":0.438},"last":0.222},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.26200000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.307}]},{"osd":3,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.318}]},{"osd":4,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":5.484,"5min":5.484,"15min":5.484},"last":0.36099999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":5.5449999999999999,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.441}]},{"osd":6,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.41799999999999998}]},{"osd":7,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.434},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":5.5800000000000001,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.501}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297892,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073948,"statfs":{"total":96636764160,"available":96331722752,"internally_reserved":0,"allocated":618496,"data_stored":258978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":7.2699999999999996,"5min":7.2699999999999996,"15min":7.2699999999999996},"last":0.46999999999999997}]},{"osd":1,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":7.327,"5min":7.327,"15min":7.327},"last":0.35799999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.34399999999999997}]},{"osd":4,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":7.2389999999999999,"5min":7.2389999999999999,"15min":7.2389999999999999},"last":0.443}]},{"osd":5,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":10.890000000000001,"5min":10.890000000000001,"15min":10.890000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.379}]},{"osd":6,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":7.2999999999999998,"5min":7.2999999999999998,"15min":7.2999999999999998},"last":0.499},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.42899999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":7.1879999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.49099999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297892,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073948,"statfs":{"total":96636764160,"available":96331722752,"internally_reserved":0,"allocated":618496,"data_stored":258978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.35599999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.41899999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.376}]},{"osd":4,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.46999999999999997}]},{"osd":5,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.56599999999999995}]},{"osd":6,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.51700000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.52600000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297896,"kb_used_data":608,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073944,"statfs":{"total":96636764160,"available":96331718656,"internally_reserved":0,"allocated":622592,"data_stored":260458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.56699999999999995}]},{"osd":1,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.61499999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.71699999999999997}]},{"osd":3,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.749}]},{"osd":5,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.76700000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.78100000000000003}]},{"osd":7,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.80000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297892,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073948,"statfs":{"total":96636764160,"available":96331722752,"internally_reserved":0,"allocated":618496,"data_stored":259546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.497}]},{"osd":1,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.47599999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.51800000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.441},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.57599999999999996}]},{"osd":4,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.55500000000000005}]},{"osd":6,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.54300000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.63400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:32:10.457 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-09T06:32:10.457 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-09T06:32:10.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.612+0000 7f1a49166700 1 Processor -- start 2023-10-09T06:32:10.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.612+0000 7f1a49166700 1 -- start start 2023-10-09T06:32:10.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.613+0000 7f1a49166700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1a441573f0 0x7f1a4406c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:10.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.613+0000 7f1a49166700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a441577c0 0x7f1a4406cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:10.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.613+0000 7f1a49166700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a4415ad70 con 0x7f1a44068390 2023-10-09T06:32:10.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.613+0000 7f1a49166700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a440c8610 con 0x7f1a441573f0 2023-10-09T06:32:10.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.613+0000 7f1a49166700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4415a160 con 0x7f1a441577c0 2023-10-09T06:32:10.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.614+0000 7f1a43fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1a441573f0 0x7f1a4406c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:10.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.614+0000 7f1a437fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a441577c0 0x7f1a4406cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:10.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.614+0000 7f1a43fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1a441573f0 0x7f1a4406c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:49538/0 (socket says 172.21.15.163:49538) 2023-10-09T06:32:10.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.614+0000 7f1a437fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a441577c0 0x7f1a4406cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:56698/0 (socket says 172.21.15.163:56698) 2023-10-09T06:32:10.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.614+0000 7f1a48965700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f1a44068390 0x7f1a4415d650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:51628/0 (socket says 172.21.15.163:51628) 2023-10-09T06:32:10.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.614+0000 7f1a437fe700 1 -- 172.21.15.163:0/844299372 learned_addr learned my addr 172.21.15.163:0/844299372 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:10.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.614+0000 7f1a437fe700 1 -- 172.21.15.163:0/844299372 >> v2:172.21.15.181:3300/0 conn(0x7f1a441573f0 msgr2=0x7f1a4406c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:10.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.614+0000 7f1a437fe700 1 --2- 172.21.15.163:0/844299372 >> v2:172.21.15.181:3300/0 conn(0x7f1a441573f0 0x7f1a4406c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:10.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.614+0000 7f1a437fe700 1 -- 172.21.15.163:0/844299372 >> v1:172.21.15.181:6789/0 conn(0x7f1a44068390 legacy=0x7f1a4415d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:10.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.614+0000 7f1a437fe700 1 -- 172.21.15.163:0/844299372 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a4415a440 con 0x7f1a441577c0 2023-10-09T06:32:10.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.614+0000 7f1a43fff700 1 --2- 172.21.15.163:0/844299372 >> v2:172.21.15.181:3300/0 conn(0x7f1a441573f0 0x7f1a4406c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:32:10.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.614+0000 7f1a42ffd700 1 -- 172.21.15.163:0/844299372 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1292478916 0 0) 0x7f1a4415ad70 con 0x7f1a44068390 2023-10-09T06:32:10.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.614+0000 7f1a437fe700 1 --2- 172.21.15.163:0/844299372 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a441577c0 0x7f1a4406cbf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1a380099b0 tx=0x7f1a38009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17b616fb8865b12c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:10.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.614+0000 7f1a42ffd700 1 -- 172.21.15.163:0/844299372 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a38009ea0 con 0x7f1a441577c0 2023-10-09T06:32:10.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.614+0000 7f1a42ffd700 1 -- 172.21.15.163:0/844299372 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a3800b860 con 0x7f1a441577c0 2023-10-09T06:32:10.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.614+0000 7f1a42ffd700 1 -- 172.21.15.163:0/844299372 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3800bc00 con 0x7f1a441577c0 2023-10-09T06:32:10.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.615+0000 7f1a49166700 1 -- 172.21.15.163:0/844299372 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a441577c0 msgr2=0x7f1a4406cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:10.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.615+0000 7f1a49166700 1 --2- 172.21.15.163:0/844299372 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a441577c0 0x7f1a4406cbf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1a380099b0 tx=0x7f1a38009250 comp rx=0 tx=0).stop 2023-10-09T06:32:10.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.615+0000 7f1a49166700 1 -- 172.21.15.163:0/844299372 shutdown_connections 2023-10-09T06:32:10.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.615+0000 7f1a49166700 1 --2- 172.21.15.163:0/844299372 >> v2:172.21.15.181:3300/0 conn(0x7f1a441573f0 0x7f1a4406c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:10.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.615+0000 7f1a49166700 1 --2- 172.21.15.163:0/844299372 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a441577c0 0x7f1a4406cbf0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:10.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.615+0000 7f1a49166700 1 -- 172.21.15.163:0/844299372 >> 172.21.15.163:0/844299372 conn(0x7f1a440cddc0 msgr2=0x7f1a440c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:10.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.615+0000 7f1a49166700 1 -- 172.21.15.163:0/844299372 shutdown_connections 2023-10-09T06:32:10.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.615+0000 7f1a49166700 1 -- 172.21.15.163:0/844299372 wait complete. 2023-10-09T06:32:10.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.616+0000 7f1a49166700 1 Processor -- start 2023-10-09T06:32:10.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.616+0000 7f1a49166700 1 -- start start 2023-10-09T06:32:10.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.617+0000 7f1a49166700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a44068390 0x7f1a440b0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:10.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.617+0000 7f1a49166700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a441573f0 0x7f1a44151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:10.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.617+0000 7f1a49166700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a441577c0 0x7f1a440ad3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:10.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.617+0000 7f1a49166700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a440b05d0 con 0x7f1a441577c0 2023-10-09T06:32:10.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.617+0000 7f1a49166700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4416afd0 con 0x7f1a44068390 2023-10-09T06:32:10.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.617+0000 7f1a49166700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4416b2b0 con 0x7f1a441573f0 2023-10-09T06:32:10.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.617+0000 7f1a48965700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a441577c0 0x7f1a440ad3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:10.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.617+0000 7f1a48965700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a441577c0 0x7f1a440ad3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:32830/0 (socket says 172.21.15.163:32830) 2023-10-09T06:32:10.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.617+0000 7f1a48965700 1 -- 172.21.15.163:0/1069261264 learned_addr learned my addr 172.21.15.163:0/1069261264 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:10.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.617+0000 7f1a43fff700 1 --2- 172.21.15.163:0/1069261264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a44068390 0x7f1a440b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:10.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.617+0000 7f1a437fe700 1 --2- 172.21.15.163:0/1069261264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a441573f0 0x7f1a44151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:10.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.617+0000 7f1a48965700 1 -- 172.21.15.163:0/1069261264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a441573f0 msgr2=0x7f1a44151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:10.624 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.617+0000 7f1a48965700 1 --2- 172.21.15.163:0/1069261264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a441573f0 0x7f1a44151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:10.624 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.617+0000 7f1a48965700 1 -- 172.21.15.163:0/1069261264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a44068390 msgr2=0x7f1a440b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:10.624 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.617+0000 7f1a48965700 1 --2- 172.21.15.163:0/1069261264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a44068390 0x7f1a440b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:10.624 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.617+0000 7f1a48965700 1 -- 172.21.15.163:0/1069261264 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a34009420 con 0x7f1a441577c0 2023-10-09T06:32:10.624 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.617+0000 7f1a437fe700 1 --2- 172.21.15.163:0/1069261264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a441573f0 0x7f1a44151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:10.624 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.617+0000 7f1a43fff700 1 --2- 172.21.15.163:0/1069261264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a44068390 0x7f1a440b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:10.624 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.618+0000 7f1a48965700 1 --2- 172.21.15.163:0/1069261264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a441577c0 0x7f1a440ad3f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00cd00 tx=0x7f1a3c001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:10.624 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.618+0000 7f1a2bfff700 1 -- 172.21.15.163:0/1069261264 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c005df0 con 0x7f1a441577c0 2023-10-09T06:32:10.624 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.618+0000 7f1a2bfff700 1 -- 172.21.15.163:0/1069261264 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a3c00a960 con 0x7f1a441577c0 2023-10-09T06:32:10.624 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.618+0000 7f1a49166700 1 -- 172.21.15.163:0/1069261264 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a380094f0 con 0x7f1a441577c0 2023-10-09T06:32:10.624 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.618+0000 7f1a2bfff700 1 -- 172.21.15.163:0/1069261264 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c00ad20 con 0x7f1a441577c0 2023-10-09T06:32:10.624 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.618+0000 7f1a49166700 1 -- 172.21.15.163:0/1069261264 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a4416b870 con 0x7f1a441577c0 2023-10-09T06:32:10.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.619+0000 7f1a49166700 1 -- 172.21.15.163:0/1069261264 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a4404c3c0 con 0x7f1a441577c0 2023-10-09T06:32:10.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.619+0000 7f1a2bfff700 1 -- 172.21.15.163:0/1069261264 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1a3c007300 con 0x7f1a441577c0 2023-10-09T06:32:10.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.620+0000 7f1a2bfff700 1 --2- 172.21.15.163:0/1069261264 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a30066f30 0x7f1a300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:10.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.620+0000 7f1a2bfff700 1 -- 172.21.15.163:0/1069261264 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1a3c0846e0 con 0x7f1a441577c0 2023-10-09T06:32:10.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.620+0000 7f1a43fff700 1 --2- 172.21.15.163:0/1069261264 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a30066f30 0x7f1a300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:10.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.620+0000 7f1a43fff700 1 --2- 172.21.15.163:0/1069261264 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a30066f30 0x7f1a300693e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1a340093f0 tx=0x7f1a34008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:10.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.624+0000 7f1a2bfff700 1 -- 172.21.15.163:0/1069261264 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a3c0519d0 con 0x7f1a441577c0 2023-10-09T06:32:10.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.879+0000 7f1a49166700 1 -- 172.21.15.163:0/1069261264 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f1a4415a750 con 0x7f1a441577c0 2023-10-09T06:32:10.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.882+0000 7f1a2bfff700 1 -- 172.21.15.163:0/1069261264 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7f1a3c054ff0 con 0x7f1a441577c0 2023-10-09T06:32:10.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.885+0000 7f1a49166700 1 -- 172.21.15.163:0/1069261264 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a30066f30 msgr2=0x7f1a300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:10.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.885+0000 7f1a49166700 1 --2- 172.21.15.163:0/1069261264 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a30066f30 0x7f1a300693e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1a340093f0 tx=0x7f1a34008040 comp rx=0 tx=0).stop 2023-10-09T06:32:10.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.885+0000 7f1a49166700 1 -- 172.21.15.163:0/1069261264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a441577c0 msgr2=0x7f1a440ad3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:10.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.885+0000 7f1a49166700 1 --2- 172.21.15.163:0/1069261264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a441577c0 0x7f1a440ad3f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00cd00 tx=0x7f1a3c001ac0 comp rx=0 tx=0).stop 2023-10-09T06:32:10.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.885+0000 7f1a49166700 1 -- 172.21.15.163:0/1069261264 shutdown_connections 2023-10-09T06:32:10.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.885+0000 7f1a49166700 1 --2- 172.21.15.163:0/1069261264 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a30066f30 0x7f1a300693e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:10.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.885+0000 7f1a49166700 1 --2- 172.21.15.163:0/1069261264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a441577c0 0x7f1a440ad3f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:10.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.885+0000 7f1a49166700 1 --2- 172.21.15.163:0/1069261264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a44068390 0x7f1a440b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:10.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.885+0000 7f1a49166700 1 --2- 172.21.15.163:0/1069261264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a441573f0 0x7f1a44151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:10.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.885+0000 7f1a49166700 1 -- 172.21.15.163:0/1069261264 >> 172.21.15.163:0/1069261264 conn(0x7f1a440cddc0 msgr2=0x7f1a44157c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:10.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.885+0000 7f1a49166700 1 -- 172.21.15.163:0/1069261264 shutdown_connections 2023-10-09T06:32:10.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:10.885+0000 7f1a49166700 1 -- 172.21.15.163:0/1069261264 wait complete. 2023-10-09T06:32:10.887 INFO:teuthology.orchestra.run.smithi163.stderr:updated caps for client.0 2023-10-09T06:32:10.898 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:32:11.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.012+0000 7fe5fc233700 1 Processor -- start 2023-10-09T06:32:11.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.012+0000 7fe5fc233700 1 -- start start 2023-10-09T06:32:11.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.013+0000 7fe5fc233700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5f4141500 0x7fe5f40c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.013+0000 7fe5fc233700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe5f406c0a0 0x7fe5f40c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.013+0000 7fe5fc233700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f415c2a0 con 0x7fe5f406bcd0 2023-10-09T06:32:11.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.013+0000 7fe5fc233700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f40c9880 con 0x7fe5f406c0a0 2023-10-09T06:32:11.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.013+0000 7fe5fc233700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f415b690 con 0x7fe5f4141500 2023-10-09T06:32:11.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.013+0000 7fe5f9fcf700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fe5f406bcd0 0x7fe5f4140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:56710/0 (socket says 172.21.15.163:56710) 2023-10-09T06:32:11.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.013+0000 7fe5f9fcf700 1 -- 172.21.15.163:0/4094384845 learned_addr learned my addr 172.21.15.163:0/4094384845 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:11.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.014+0000 7fe5f8fcd700 1 -- 172.21.15.163:0/4094384845 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2120666046 0 0) 0x7fe5f415c2a0 con 0x7fe5f406bcd0 2023-10-09T06:32:11.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.014+0000 7fe5f8fcd700 1 -- 172.21.15.163:0/4094384845 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5e0003570 con 0x7fe5f406bcd0 2023-10-09T06:32:11.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.014+0000 7fe5fa7d0700 1 --2- 172.21.15.163:0/4094384845 >> v2:172.21.15.181:3300/0 conn(0x7fe5f406c0a0 0x7fe5f40c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.014+0000 7fe5f97ce700 1 --2- 172.21.15.163:0/4094384845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5f4141500 0x7fe5f40c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.014+0000 7fe5f8fcd700 1 -- 172.21.15.163:0/4094384845 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 432357750 0 0) 0x7fe5e0003570 con 0x7fe5f406bcd0 2023-10-09T06:32:11.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.014+0000 7fe5f8fcd700 1 -- 172.21.15.163:0/4094384845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5f4141500 msgr2=0x7fe5f40c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:11.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.014+0000 7fe5f8fcd700 1 --2- 172.21.15.163:0/4094384845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5f4141500 0x7fe5f40c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.014+0000 7fe5f8fcd700 1 -- 172.21.15.163:0/4094384845 >> v2:172.21.15.181:3300/0 conn(0x7fe5f406c0a0 msgr2=0x7fe5f40c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:11.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.014+0000 7fe5f8fcd700 1 --2- 172.21.15.163:0/4094384845 >> v2:172.21.15.181:3300/0 conn(0x7fe5f406c0a0 0x7fe5f40c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.014+0000 7fe5f8fcd700 1 -- 172.21.15.163:0/4094384845 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f415b970 con 0x7fe5f406bcd0 2023-10-09T06:32:11.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.014+0000 7fe5f8fcd700 1 -- 172.21.15.163:0/4094384845 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fe5f00017f0 con 0x7fe5f406bcd0 2023-10-09T06:32:11.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.014+0000 7fe5f97ce700 1 --2- 172.21.15.163:0/4094384845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5f4141500 0x7fe5f40c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:11.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.014+0000 7fe5f8fcd700 1 -- 172.21.15.163:0/4094384845 >> v1:172.21.15.163:6789/0 conn(0x7fe5f406bcd0 legacy=0x7fe5f4140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:11.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.014+0000 7fe5f8fcd700 1 --2- 172.21.15.163:0/4094384845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e0003a60 0x7fe5e0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.014+0000 7fe5f8fcd700 1 --2- 172.21.15.163:0/4094384845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5f4141500 0x7fe5e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.014+0000 7fe5f8fcd700 1 --2- 172.21.15.163:0/4094384845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e0004d50 0x7fe5e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.014+0000 7fe5f8fcd700 1 -- 172.21.15.163:0/4094384845 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f40c9880 con 0x7fe5e0004d50 2023-10-09T06:32:11.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.014+0000 7fe5f8fcd700 1 -- 172.21.15.163:0/4094384845 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f415b690 con 0x7fe5e0003a60 2023-10-09T06:32:11.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.014+0000 7fe5f8fcd700 1 -- 172.21.15.163:0/4094384845 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e000bab0 con 0x7fe5f4141500 2023-10-09T06:32:11.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.014+0000 7fe5fa7d0700 1 --2- 172.21.15.163:0/4094384845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e0004d50 0x7fe5e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.014+0000 7fe5f9fcf700 1 --2- 172.21.15.163:0/4094384845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e0003a60 0x7fe5e0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.014+0000 7fe5f97ce700 1 --2- 172.21.15.163:0/4094384845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5f4141500 0x7fe5e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.014+0000 7fe5fa7d0700 1 -- 172.21.15.163:0/4094384845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5f4141500 msgr2=0x7fe5e0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:11.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.014+0000 7fe5fa7d0700 1 --2- 172.21.15.163:0/4094384845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5f4141500 0x7fe5e0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.014+0000 7fe5fa7d0700 1 -- 172.21.15.163:0/4094384845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e0003a60 msgr2=0x7fe5e0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:11.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.014+0000 7fe5fa7d0700 1 --2- 172.21.15.163:0/4094384845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e0003a60 0x7fe5e0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.014+0000 7fe5fa7d0700 1 -- 172.21.15.163:0/4094384845 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e0003570 con 0x7fe5e0004d50 2023-10-09T06:32:11.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.015+0000 7fe5fa7d0700 1 --2- 172.21.15.163:0/4094384845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e0004d50 0x7fe5e00051c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec009c00 tx=0x7fe5ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6eea9ca81e27cde server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:11.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.015+0000 7fe5f8fcd700 1 -- 172.21.15.163:0/4094384845 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec00e710 con 0x7fe5e0004d50 2023-10-09T06:32:11.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.015+0000 7fe5f8fcd700 1 -- 172.21.15.163:0/4094384845 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5ec00e870 con 0x7fe5e0004d50 2023-10-09T06:32:11.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.015+0000 7fe5f8fcd700 1 -- 172.21.15.163:0/4094384845 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec00ec30 con 0x7fe5e0004d50 2023-10-09T06:32:11.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.015+0000 7fe5fc233700 1 -- 172.21.15.163:0/4094384845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e0004d50 msgr2=0x7fe5e00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:11.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.015+0000 7fe5fc233700 1 --2- 172.21.15.163:0/4094384845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e0004d50 0x7fe5e00051c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec009c00 tx=0x7fe5ec0091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.015+0000 7fe5fa7d0700 1 -- 172.21.15.163:0/4094384845 reap_dead start 2023-10-09T06:32:11.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.015+0000 7fe5fc233700 1 -- 172.21.15.163:0/4094384845 shutdown_connections 2023-10-09T06:32:11.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.015+0000 7fe5fc233700 1 -- 172.21.15.163:0/4094384845 >> 172.21.15.163:0/4094384845 conn(0x7fe5f40ce600 msgr2=0x7fe5f40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:11.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.015+0000 7fe5fc233700 1 -- 172.21.15.163:0/4094384845 shutdown_connections 2023-10-09T06:32:11.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.015+0000 7fe5fc233700 1 -- 172.21.15.163:0/4094384845 wait complete. 2023-10-09T06:32:11.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.016+0000 7fe5fc233700 1 Processor -- start 2023-10-09T06:32:11.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.017+0000 7fe5fc233700 1 -- start start 2023-10-09T06:32:11.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.017+0000 7fe5fc233700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e0004d50 0x7fe5f413b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.017+0000 7fe5fc233700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5f40c6ef0 0x7fe5f40c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.017+0000 7fe5fc233700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5f4154ce0 0x7fe5f40c4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.017+0000 7fe5fc233700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f40c5a20 con 0x7fe5f4154ce0 2023-10-09T06:32:11.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.017+0000 7fe5fc233700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f413b7b0 con 0x7fe5f40c6ef0 2023-10-09T06:32:11.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.017+0000 7fe5fc233700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f40c7460 con 0x7fe5e0004d50 2023-10-09T06:32:11.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.017+0000 7fe5fa7d0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5f4154ce0 0x7fe5f40c4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.017+0000 7fe5fa7d0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5f4154ce0 0x7fe5f40c4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:32844/0 (socket says 172.21.15.163:32844) 2023-10-09T06:32:11.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.017+0000 7fe5fa7d0700 1 -- 172.21.15.163:0/4191552225 learned_addr learned my addr 172.21.15.163:0/4191552225 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:11.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.017+0000 7fe5f9fcf700 1 --2- 172.21.15.163:0/4191552225 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e0004d50 0x7fe5f413b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.017+0000 7fe5f97ce700 1 --2- 172.21.15.163:0/4191552225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5f40c6ef0 0x7fe5f40c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.018+0000 7fe5fa7d0700 1 -- 172.21.15.163:0/4191552225 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e0004d50 msgr2=0x7fe5f413b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:11.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.018+0000 7fe5fa7d0700 1 --2- 172.21.15.163:0/4191552225 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e0004d50 0x7fe5f413b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.018+0000 7fe5fa7d0700 1 -- 172.21.15.163:0/4191552225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5f40c6ef0 msgr2=0x7fe5f40c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:11.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.018+0000 7fe5fa7d0700 1 --2- 172.21.15.163:0/4191552225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5f40c6ef0 0x7fe5f40c5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.018+0000 7fe5fa7d0700 1 -- 172.21.15.163:0/4191552225 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f40c7740 con 0x7fe5f4154ce0 2023-10-09T06:32:11.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.018+0000 7fe5fa7d0700 1 --2- 172.21.15.163:0/4191552225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5f4154ce0 0x7fe5f40c4180 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec016000 tx=0x7fe5ec01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:11.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.018+0000 7fe5e9ffb700 1 -- 172.21.15.163:0/4191552225 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec022070 con 0x7fe5f4154ce0 2023-10-09T06:32:11.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.018+0000 7fe5e9ffb700 1 -- 172.21.15.163:0/4191552225 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5ec009110 con 0x7fe5f4154ce0 2023-10-09T06:32:11.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.018+0000 7fe5fc233700 1 -- 172.21.15.163:0/4191552225 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f40c5a20 con 0x7fe5f4154ce0 2023-10-09T06:32:11.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.018+0000 7fe5e9ffb700 1 -- 172.21.15.163:0/4191552225 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec00e710 con 0x7fe5f4154ce0 2023-10-09T06:32:11.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.018+0000 7fe5fc233700 1 -- 172.21.15.163:0/4191552225 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f40c56b0 con 0x7fe5f4154ce0 2023-10-09T06:32:11.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.019+0000 7fe5fc233700 1 -- 172.21.15.163:0/4191552225 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f40c5bb0 con 0x7fe5f4154ce0 2023-10-09T06:32:11.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.020+0000 7fe5e9ffb700 1 -- 172.21.15.163:0/4191552225 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe5ec01c450 con 0x7fe5f4154ce0 2023-10-09T06:32:11.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.020+0000 7fe5e9ffb700 1 --2- 172.21.15.163:0/4191552225 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe5d8066f30 0x7fe5d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.020+0000 7fe5e9ffb700 1 -- 172.21.15.163:0/4191552225 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe5ec085bb0 con 0x7fe5f4154ce0 2023-10-09T06:32:11.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.020+0000 7fe5f9fcf700 1 --2- 172.21.15.163:0/4191552225 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe5d8066f30 0x7fe5d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.021+0000 7fe5f9fcf700 1 --2- 172.21.15.163:0/4191552225 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe5d8066f30 0x7fe5d80693e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000a100 tx=0x7fe5f406e470 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:11.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.024+0000 7fe5e9ffb700 1 -- 172.21.15.163:0/4191552225 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5ec052e20 con 0x7fe5f4154ce0 2023-10-09T06:32:11.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.247+0000 7fe5fc233700 1 -- 172.21.15.163:0/4191552225 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5f404c3c0 con 0x7fe5f4154ce0 2023-10-09T06:32:11.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.248+0000 7fe5e9ffb700 1 -- 172.21.15.163:0/4191552225 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4606 (secure 0 0 0) 0x7fe5ec017070 con 0x7fe5f4154ce0 2023-10-09T06:32:11.248 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:11.249 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/819128421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":819128421},{"type":"v1","addr":"172.21.15.181:6836","nonce":819128421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3339496202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3339496202},{"type":"v1","addr":"172.21.15.163:6834","nonce":3339496202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5352,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1726174482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1726174482},{"type":"v1","addr":"172.21.15.163:6835","nonce":1726174482}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:32:00.924137+0000","modified":"2023-10-09T06:32:04.563062+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5370},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5370":{"gid":5370,"name":"b","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2945196635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2945196635},{"type":"v1","addr":"172.21.15.181:6837","nonce":2945196635}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-09T06:32:11.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.250+0000 7fe5fc233700 1 -- 172.21.15.163:0/4191552225 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe5d8066f30 msgr2=0x7fe5d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:11.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.250+0000 7fe5fc233700 1 --2- 172.21.15.163:0/4191552225 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe5d8066f30 0x7fe5d80693e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000a100 tx=0x7fe5f406e470 comp rx=0 tx=0).stop 2023-10-09T06:32:11.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.250+0000 7fe5fc233700 1 -- 172.21.15.163:0/4191552225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5f4154ce0 msgr2=0x7fe5f40c4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:11.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.250+0000 7fe5fc233700 1 --2- 172.21.15.163:0/4191552225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5f4154ce0 0x7fe5f40c4180 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec016000 tx=0x7fe5ec01a000 comp rx=0 tx=0).stop 2023-10-09T06:32:11.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.250+0000 7fe5fc233700 1 -- 172.21.15.163:0/4191552225 shutdown_connections 2023-10-09T06:32:11.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.250+0000 7fe5fc233700 1 --2- 172.21.15.163:0/4191552225 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe5d8066f30 0x7fe5d80693e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.250+0000 7fe5fc233700 1 --2- 172.21.15.163:0/4191552225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5f4154ce0 0x7fe5f40c4180 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.250+0000 7fe5fc233700 1 --2- 172.21.15.163:0/4191552225 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e0004d50 0x7fe5f413b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.250+0000 7fe5fc233700 1 --2- 172.21.15.163:0/4191552225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5f40c6ef0 0x7fe5f40c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.250+0000 7fe5fc233700 1 -- 172.21.15.163:0/4191552225 >> 172.21.15.163:0/4191552225 conn(0x7fe5f40ce600 msgr2=0x7fe5f40c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:11.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.250+0000 7fe5fc233700 1 -- 172.21.15.163:0/4191552225 shutdown_connections 2023-10-09T06:32:11.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.250+0000 7fe5fc233700 1 -- 172.21.15.163:0/4191552225 wait complete. 2023-10-09T06:32:11.252 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 31 2023-10-09T06:32:11.665 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 31, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-09T06:32:00.924137+0000', 'modified': '2023-10-09T06:32:04.563062+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5370}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5370': {'gid': 5370, 'name': 'b', 'rank': 0, 'incarnation': 27, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6837/2945196635', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6835', 'nonce': 2945196635}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 2945196635}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-09T06:32:11.665 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-09T06:32:11.666 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-09T06:32:11.786 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.785+0000 7f8d2bfff700 1 Processor -- start 2023-10-09T06:32:11.786 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.785+0000 7f8d2bfff700 1 -- start start 2023-10-09T06:32:11.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.786+0000 7f8d2bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8d2c159420 0x7f8d2c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.786+0000 7f8d2bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8d2c15d230 0x7f8d2c15d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.786+0000 7f8d2bfff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d2c0b2980 con 0x7f8d2c1597f0 2023-10-09T06:32:11.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.786+0000 7f8d2bfff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d2c15db70 con 0x7f8d2c15d230 2023-10-09T06:32:11.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.786+0000 7f8d2bfff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d2c0b11d0 con 0x7f8d2c159420 2023-10-09T06:32:11.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.786+0000 7f8d2b7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8d2c15d230 0x7f8d2c15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.786+0000 7f8d2b7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8d2c15d230 0x7f8d2c15d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:32856/0 (socket says 172.21.15.163:32856) 2023-10-09T06:32:11.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.786+0000 7f8d2b7fe700 1 -- 172.21.15.163:0/2424541847 learned_addr learned my addr 172.21.15.163:0/2424541847 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:11.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.786+0000 7f8d2affd700 1 --2- 172.21.15.163:0/2424541847 >> v2:172.21.15.181:3300/0 conn(0x7f8d2c159420 0x7f8d2c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.786+0000 7f8d2b7fe700 1 -- 172.21.15.163:0/2424541847 >> v2:172.21.15.181:3300/0 conn(0x7f8d2c159420 msgr2=0x7f8d2c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:11.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.786+0000 7f8d2b7fe700 1 --2- 172.21.15.163:0/2424541847 >> v2:172.21.15.181:3300/0 conn(0x7f8d2c159420 0x7f8d2c0b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.786+0000 7f8d2b7fe700 1 -- 172.21.15.163:0/2424541847 >> v1:172.21.15.181:6789/0 conn(0x7f8d2c1597f0 legacy=0x7f8d2c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:11.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.786+0000 7f8d2b7fe700 1 -- 172.21.15.163:0/2424541847 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c0b14b0 con 0x7f8d2c15d230 2023-10-09T06:32:11.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.787+0000 7f8d2b7fe700 1 --2- 172.21.15.163:0/2424541847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d2c15d230 0x7f8d2c15d630 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c0b2f20 tx=0x7f8d24009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53cba9297072cdf server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:11.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.787+0000 7f8d2a7fc700 1 -- 172.21.15.163:0/2424541847 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2400e070 con 0x7f8d2c15d230 2023-10-09T06:32:11.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.787+0000 7f8d2a7fc700 1 -- 172.21.15.163:0/2424541847 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d2400ba10 con 0x7f8d2c15d230 2023-10-09T06:32:11.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.787+0000 7f8d2a7fc700 1 -- 172.21.15.163:0/2424541847 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d240092e0 con 0x7f8d2c15d230 2023-10-09T06:32:11.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.787+0000 7f8d2bfff700 1 -- 172.21.15.163:0/2424541847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d2c15d230 msgr2=0x7f8d2c15d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:11.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.787+0000 7f8d2bfff700 1 --2- 172.21.15.163:0/2424541847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d2c15d230 0x7f8d2c15d630 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c0b2f20 tx=0x7f8d24009840 comp rx=0 tx=0).stop 2023-10-09T06:32:11.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.787+0000 7f8d2bfff700 1 -- 172.21.15.163:0/2424541847 shutdown_connections 2023-10-09T06:32:11.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.787+0000 7f8d2bfff700 1 --2- 172.21.15.163:0/2424541847 >> v2:172.21.15.181:3300/0 conn(0x7f8d2c159420 0x7f8d2c0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.787+0000 7f8d2bfff700 1 --2- 172.21.15.163:0/2424541847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d2c15d230 0x7f8d2c15d630 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.787+0000 7f8d2bfff700 1 -- 172.21.15.163:0/2424541847 >> 172.21.15.163:0/2424541847 conn(0x7f8d2c0d0660 msgr2=0x7f8d2c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:11.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.787+0000 7f8d2bfff700 1 -- 172.21.15.163:0/2424541847 shutdown_connections 2023-10-09T06:32:11.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.787+0000 7f8d2bfff700 1 -- 172.21.15.163:0/2424541847 wait complete. 2023-10-09T06:32:11.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.788+0000 7f8d2bfff700 1 Processor -- start 2023-10-09T06:32:11.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.789+0000 7f8d2bfff700 1 -- start start 2023-10-09T06:32:11.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.789+0000 7f8d2bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d2c159420 0x7f8d2c0c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.789+0000 7f8d2bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d2c1597f0 0x7f8d2c168b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.789+0000 7f8d2bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d2c15d230 0x7f8d2c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.789+0000 7f8d2bfff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c16b3b0 con 0x7f8d2c15d230 2023-10-09T06:32:11.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.789+0000 7f8d2bfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c0c5f70 con 0x7f8d2c1597f0 2023-10-09T06:32:11.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.789+0000 7f8d2bfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c16a4c0 con 0x7f8d2c159420 2023-10-09T06:32:11.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.789+0000 7f8d2b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d2c15d230 0x7f8d2c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.789+0000 7f8d2b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d2c15d230 0x7f8d2c0b4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:32862/0 (socket says 172.21.15.163:32862) 2023-10-09T06:32:11.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.789+0000 7f8d2b7fe700 1 -- 172.21.15.163:0/1979384502 learned_addr learned my addr 172.21.15.163:0/1979384502 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:11.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.790+0000 7f8d2affd700 1 --2- 172.21.15.163:0/1979384502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d2c159420 0x7f8d2c0c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.790+0000 7f8d2b7fe700 1 -- 172.21.15.163:0/1979384502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d2c159420 msgr2=0x7f8d2c0c5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:11.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.790+0000 7f8d227fc700 1 --2- 172.21.15.163:0/1979384502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d2c1597f0 0x7f8d2c168b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.790+0000 7f8d2b7fe700 1 --2- 172.21.15.163:0/1979384502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d2c159420 0x7f8d2c0c5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.790+0000 7f8d2b7fe700 1 -- 172.21.15.163:0/1979384502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d2c1597f0 msgr2=0x7f8d2c168b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:11.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.790+0000 7f8d2b7fe700 1 --2- 172.21.15.163:0/1979384502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d2c1597f0 0x7f8d2c168b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.790+0000 7f8d2b7fe700 1 -- 172.21.15.163:0/1979384502 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d24009ae0 con 0x7f8d2c15d230 2023-10-09T06:32:11.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.790+0000 7f8d227fc700 1 --2- 172.21.15.163:0/1979384502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d2c1597f0 0x7f8d2c168b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:32:11.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.790+0000 7f8d2affd700 1 --2- 172.21.15.163:0/1979384502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d2c159420 0x7f8d2c0c5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:32:11.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.790+0000 7f8d2b7fe700 1 --2- 172.21.15.163:0/1979384502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d2c15d230 0x7f8d2c0b4e20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c089340 tx=0x7f8d24017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:11.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.790+0000 7f8d237fe700 1 -- 172.21.15.163:0/1979384502 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d24012e60 con 0x7f8d2c15d230 2023-10-09T06:32:11.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.790+0000 7f8d237fe700 1 -- 172.21.15.163:0/1979384502 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d240092e0 con 0x7f8d2c15d230 2023-10-09T06:32:11.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.790+0000 7f8d2bfff700 1 -- 172.21.15.163:0/1979384502 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d2c16a7a0 con 0x7f8d2c15d230 2023-10-09T06:32:11.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.790+0000 7f8d237fe700 1 -- 172.21.15.163:0/1979384502 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2401d650 con 0x7f8d2c15d230 2023-10-09T06:32:11.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.790+0000 7f8d2bfff700 1 -- 172.21.15.163:0/1979384502 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d2c16ad60 con 0x7f8d2c15d230 2023-10-09T06:32:11.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.791+0000 7f8d237fe700 1 -- 172.21.15.163:0/1979384502 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8d2401da00 con 0x7f8d2c15d230 2023-10-09T06:32:11.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.792+0000 7f8d237fe700 1 --2- 172.21.15.163:0/1979384502 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8d0c066f30 0x7f8d0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.792+0000 7f8d237fe700 1 -- 172.21.15.163:0/1979384502 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8d24085ee0 con 0x7f8d2c15d230 2023-10-09T06:32:11.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.792+0000 7f8d2affd700 1 --2- 172.21.15.163:0/1979384502 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8d0c066f30 0x7f8d0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.792+0000 7f8d2affd700 1 --2- 172.21.15.163:0/1979384502 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8d0c066f30 0x7f8d0c0693e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c157430 tx=0x7f8d1c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:11.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.794+0000 7f8d3a7a4b80 1 Processor -- start 2023-10-09T06:32:11.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.794+0000 7f8d3a7a4b80 1 -- start start 2023-10-09T06:32:11.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.794+0000 7f8d3a7a4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f61df6480 0x556f61dd57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.794+0000 7f8d3a7a4b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556f61da95f0 0x556f61df6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.794+0000 7f8d3a7a4b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556f61dabb80 0x556f61dabff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.794+0000 7f8d3a7a4b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f61d02a70 con 0x556f61da95f0 2023-10-09T06:32:11.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.794+0000 7f8d3a7a4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f61d00270 con 0x556f61df6480 2023-10-09T06:32:11.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.795+0000 7f8d3a7a4b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f61cfc060 con 0x556f61dabb80 2023-10-09T06:32:11.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.795+0000 7f8d227fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556f61da95f0 0x556f61df6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.795+0000 7f8d227fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556f61da95f0 0x556f61df6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:32864/0 (socket says 172.21.15.163:32864) 2023-10-09T06:32:11.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.795+0000 7f8d227fc700 1 -- 172.21.15.163:0/3238210145 learned_addr learned my addr 172.21.15.163:0/3238210145 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:11.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.795+0000 7f8d2affd700 1 --2- 172.21.15.163:0/3238210145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f61df6480 0x556f61dd57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.795+0000 7f8d2b7fe700 1 --2- 172.21.15.163:0/3238210145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556f61dabb80 0x556f61dabff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.795+0000 7f8d227fc700 1 -- 172.21.15.163:0/3238210145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556f61dabb80 msgr2=0x556f61dabff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:11.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.795+0000 7f8d227fc700 1 --2- 172.21.15.163:0/3238210145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556f61dabb80 0x556f61dabff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.795+0000 7f8d227fc700 1 -- 172.21.15.163:0/3238210145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f61df6480 msgr2=0x556f61dd57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:11.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.795+0000 7f8d227fc700 1 --2- 172.21.15.163:0/3238210145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f61df6480 0x556f61dd57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.795+0000 7f8d227fc700 1 -- 172.21.15.163:0/3238210145 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f61b96270 con 0x556f61da95f0 2023-10-09T06:32:11.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.795+0000 7f8d227fc700 1 --2- 172.21.15.163:0/3238210145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556f61da95f0 0x556f61df6850 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8d140028a0 tx=0x7f8d14009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1665fa3ca8ba6a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:11.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.795+0000 7f8d0bfff700 1 -- 172.21.15.163:0/3238210145 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1400d850 con 0x556f61da95f0 2023-10-09T06:32:11.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.795+0000 7f8d0bfff700 1 -- 172.21.15.163:0/3238210145 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1400d9b0 con 0x556f61da95f0 2023-10-09T06:32:11.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.795+0000 7f8d0bfff700 1 -- 172.21.15.163:0/3238210145 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1400dd70 con 0x556f61da95f0 2023-10-09T06:32:11.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.796+0000 7f8d3a7a4b80 1 -- 172.21.15.163:0/3238210145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556f61da95f0 msgr2=0x556f61df6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:11.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.796+0000 7f8d3a7a4b80 1 --2- 172.21.15.163:0/3238210145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556f61da95f0 0x556f61df6850 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8d140028a0 tx=0x7f8d14009040 comp rx=0 tx=0).stop 2023-10-09T06:32:11.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.796+0000 7f8d3a7a4b80 1 -- 172.21.15.163:0/3238210145 shutdown_connections 2023-10-09T06:32:11.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.796+0000 7f8d3a7a4b80 1 --2- 172.21.15.163:0/3238210145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556f61dabb80 0x556f61dabff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.796+0000 7f8d3a7a4b80 1 --2- 172.21.15.163:0/3238210145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f61df6480 0x556f61dd57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.796+0000 7f8d3a7a4b80 1 --2- 172.21.15.163:0/3238210145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556f61da95f0 0x556f61df6850 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.796+0000 7f8d3a7a4b80 1 -- 172.21.15.163:0/3238210145 >> 172.21.15.163:0/3238210145 conn(0x556f61dad0c0 msgr2=0x556f61dad8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:11.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.796+0000 7f8d3a7a4b80 1 -- 172.21.15.163:0/3238210145 shutdown_connections 2023-10-09T06:32:11.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.796+0000 7f8d3a7a4b80 1 -- 172.21.15.163:0/3238210145 wait complete. 2023-10-09T06:32:11.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.797+0000 7f8d3a7a4b80 1 -- start start 2023-10-09T06:32:11.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.797+0000 7f8d3a7a4b80 1 Processor -- start 2023-10-09T06:32:11.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.797+0000 7f8d3a7a4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f61da95f0 0x556f61e01fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.797+0000 7f8d3a7a4b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556f61dabb80 0x556f61df2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.797+0000 7f8d3a7a4b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556f61df6480 0x556f61df39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.798+0000 7f8d3a7a4b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f61ca7890 con 0x556f61df6480 2023-10-09T06:32:11.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.798+0000 7f8d3a7a4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f61cc2ea0 con 0x556f61da95f0 2023-10-09T06:32:11.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.798+0000 7f8d3a7a4b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f61b7ee30 con 0x556f61dabb80 2023-10-09T06:32:11.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.798+0000 7f8d3a7a4b80 10 client.0 _ll_register_callbacks cb 0x556f61d6f2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-09T06:32:11.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.798+0000 7f8d2b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556f61df6480 0x556f61df39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.798+0000 7f8d2b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556f61df6480 0x556f61df39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:32880/0 (socket says 172.21.15.163:32880) 2023-10-09T06:32:11.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.798+0000 7f8d2b7fe700 1 -- 172.21.15.163:0/793443154 learned_addr learned my addr 172.21.15.163:0/793443154 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:11.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.798+0000 7f8d227fc700 1 --2- 172.21.15.163:0/793443154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556f61dabb80 0x556f61df2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.798+0000 7f8d2affd700 1 --2- 172.21.15.163:0/793443154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f61da95f0 0x556f61e01fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.798+0000 7f8d2b7fe700 1 -- 172.21.15.163:0/793443154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556f61dabb80 msgr2=0x556f61df2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:11.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.798+0000 7f8d2b7fe700 1 --2- 172.21.15.163:0/793443154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556f61dabb80 0x556f61df2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.798+0000 7f8d2b7fe700 1 -- 172.21.15.163:0/793443154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f61da95f0 msgr2=0x556f61e01fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:11.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.798+0000 7f8d2b7fe700 1 --2- 172.21.15.163:0/793443154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f61da95f0 0x556f61e01fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.798+0000 7f8d2b7fe700 1 -- 172.21.15.163:0/793443154 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d14002420 con 0x556f61df6480 2023-10-09T06:32:11.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.798+0000 7f8d2b7fe700 1 --2- 172.21.15.163:0/793443154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556f61df6480 0x556f61df39b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8d24009ab0 tx=0x7f8d2405c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843d9208bb88e325 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:11.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.798+0000 7f8d097fa700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-10-09T06:32:11.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.798+0000 7f8d097fa700 1 -- 172.21.15.163:0/793443154 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2400e040 con 0x556f61df6480 2023-10-09T06:32:11.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.798+0000 7f8d097fa700 1 -- 172.21.15.163:0/793443154 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8d24056e10 con 0x556f61df6480 2023-10-09T06:32:11.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.798+0000 7f8d3a7a4b80 1 -- 172.21.15.163:0/793443154 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f61da0cd0 con 0x556f61df6480 2023-10-09T06:32:11.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.798+0000 7f8d097fa700 1 -- 172.21.15.163:0/793443154 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d2405a060 con 0x556f61df6480 2023-10-09T06:32:11.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.798+0000 7f8d3a7a4b80 10 client.5532 fetch_fsmap learned FSMap version 31 2023-10-09T06:32:11.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.798+0000 7f8d3a7a4b80 1 -- 172.21.15.163:0/793443154 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x556f61b38fe0 con 0x556f61df6480 2023-10-09T06:32:11.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.798+0000 7f8d097fa700 1 -- 172.21.15.163:0/793443154 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556f61da0cd0 con 0x556f61df6480 2023-10-09T06:32:11.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.799+0000 7f8d097fa700 1 -- 172.21.15.163:0/793443154 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 31) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f8d2405d920 con 0x556f61df6480 2023-10-09T06:32:11.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.799+0000 7f8d3a7a4b80 10 client.5532 fetch_fsmap finished waiting for FSMap version 31 2023-10-09T06:32:11.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.799+0000 7f8d3a7a4b80 10 client.5532 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-09T06:32:11.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.799+0000 7f8d3a7a4b80 20 client.5532 populate_metadata read hostname 'smithi163' 2023-10-09T06:32:11.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.799+0000 7f8d3a7a4b80 1 --2- 172.21.15.163:0/793443154 >> [v2:172.21.15.181:6835/2945196635,v1:172.21.15.181:6837/2945196635] conn(0x556f61e05b60 0x556f61e05f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.799+0000 7f8d3a7a4b80 4 client.5532 mds_command: new command op to 5370 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-09T06:32:11.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.799+0000 7f8d3a7a4b80 1 -- 172.21.15.163:0/793443154 --> [v2:172.21.15.181:6835/2945196635,v1:172.21.15.181:6837/2945196635] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f61d7daa0 con 0x556f61e05b60 2023-10-09T06:32:11.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.799+0000 7f8d2affd700 1 --2- 172.21.15.163:0/793443154 >> [v2:172.21.15.181:6835/2945196635,v1:172.21.15.181:6837/2945196635] conn(0x556f61e05b60 0x556f61e05f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.799+0000 7f8d097fa700 10 client.5532 ms_handle_connect on v2:172.21.15.181:6835/2945196635 2023-10-09T06:32:11.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.799+0000 7f8d2affd700 1 --2- 172.21.15.163:0/793443154 >> [v2:172.21.15.181:6835/2945196635,v1:172.21.15.181:6837/2945196635] conn(0x556f61e05b60 0x556f61e05f30 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=835a22d497e32134 server_cookie=74473be6904003a5 in_seq=0 out_seq=0 2023-10-09T06:32:11.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.800+0000 7f8d097fa700 1 -- 172.21.15.163:0/793443154 <== mds.0 v2:172.21.15.181:6835/2945196635 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x556f61d7daa0 con 0x556f61e05b60 2023-10-09T06:32:11.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.800+0000 7f8d097fa700 10 client.5532 handle_command_reply: tid=0 2023-10-09T06:32:11.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.800+0000 7f8d3a7a4b80 1 client.5532 shutdown 2023-10-09T06:32:11.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.800+0000 7f8d3a7a4b80 1 -- 172.21.15.163:0/793443154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556f61df6480 msgr2=0x556f61df39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:11.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.800+0000 7f8d3a7a4b80 1 --2- 172.21.15.163:0/793443154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556f61df6480 0x556f61df39b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8d24009ab0 tx=0x7f8d2405c040 comp rx=0 tx=0).stop 2023-10-09T06:32:11.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.800+0000 7f8d3a7a4b80 1 -- 172.21.15.163:0/793443154 shutdown_connections 2023-10-09T06:32:11.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.800+0000 7f8d3a7a4b80 1 --2- 172.21.15.163:0/793443154 >> [v2:172.21.15.181:6835/2945196635,v1:172.21.15.181:6837/2945196635] conn(0x556f61e05b60 0x556f61e05f30 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.800+0000 7f8d3a7a4b80 1 --2- 172.21.15.163:0/793443154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556f61df6480 0x556f61df39b0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.800+0000 7f8d3a7a4b80 1 --2- 172.21.15.163:0/793443154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f61da95f0 0x556f61e01fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.800+0000 7f8d3a7a4b80 1 --2- 172.21.15.163:0/793443154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556f61dabb80 0x556f61df2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.800+0000 7f8d097fa700 0 client.5532 ms_handle_reset on v2:172.21.15.181:6835/2945196635 2023-10-09T06:32:11.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.800+0000 7f8d3a7a4b80 1 -- 172.21.15.163:0/793443154 >> 172.21.15.163:0/793443154 conn(0x556f61dad0c0 msgr2=0x556f61dad8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:11.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.800+0000 7f8d3a7a4b80 1 -- 172.21.15.163:0/793443154 shutdown_connections 2023-10-09T06:32:11.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.800+0000 7f8d3a7a4b80 1 -- 172.21.15.163:0/793443154 wait complete. 2023-10-09T06:32:11.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.800+0000 7f8d3a7a4b80 20 client.5532 trim_cache size 0 max 16384 2023-10-09T06:32:11.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.817+0000 7f8d3a7a4b80 1 Processor -- start 2023-10-09T06:32:11.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.817+0000 7f8d3a7a4b80 1 -- start start 2023-10-09T06:32:11.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.817+0000 7f8d3a7a4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f61da95f0 0x556f61e57dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.817+0000 7f8d3a7a4b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556f61dabb80 0x556f61df4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.817+0000 7f8d3a7a4b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556f61df6480 0x556f61da8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.817+0000 7f8d3a7a4b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f61dab3c0 con 0x556f61df6480 2023-10-09T06:32:11.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.817+0000 7f8d3a7a4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f61e14aa0 con 0x556f61da95f0 2023-10-09T06:32:11.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.818+0000 7f8d3a7a4b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f61daa2d0 con 0x556f61dabb80 2023-10-09T06:32:11.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.818+0000 7f8d2b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556f61df6480 0x556f61da8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.818+0000 7f8d2b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556f61df6480 0x556f61da8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:32888/0 (socket says 172.21.15.163:32888) 2023-10-09T06:32:11.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.818+0000 7f8d2b7fe700 1 -- 172.21.15.163:0/951633450 learned_addr learned my addr 172.21.15.163:0/951633450 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:11.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.818+0000 7f8d227fc700 1 --2- 172.21.15.163:0/951633450 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556f61dabb80 0x556f61df4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.818+0000 7f8d2affd700 1 --2- 172.21.15.163:0/951633450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f61da95f0 0x556f61e57dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.818+0000 7f8d2b7fe700 1 -- 172.21.15.163:0/951633450 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556f61dabb80 msgr2=0x556f61df4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:11.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.818+0000 7f8d2b7fe700 1 --2- 172.21.15.163:0/951633450 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556f61dabb80 0x556f61df4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.818+0000 7f8d2b7fe700 1 -- 172.21.15.163:0/951633450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f61da95f0 msgr2=0x556f61e57dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:11.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.818+0000 7f8d2b7fe700 1 --2- 172.21.15.163:0/951633450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f61da95f0 0x556f61e57dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.818+0000 7f8d2b7fe700 1 -- 172.21.15.163:0/951633450 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f61cca780 con 0x556f61df6480 2023-10-09T06:32:11.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.818+0000 7f8d227fc700 1 --2- 172.21.15.163:0/951633450 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556f61dabb80 0x556f61df4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:32:11.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.818+0000 7f8d2affd700 1 --2- 172.21.15.163:0/951633450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f61da95f0 0x556f61e57dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:32:11.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.818+0000 7f8d2b7fe700 1 --2- 172.21.15.163:0/951633450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556f61df6480 0x556f61da8a40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8d24015040 tx=0x7f8d2406b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b4de3a68018cf9 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:11.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.818+0000 7f8d20ff9700 1 -- 172.21.15.163:0/951633450 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2400e040 con 0x556f61df6480 2023-10-09T06:32:11.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.818+0000 7f8d20ff9700 1 -- 172.21.15.163:0/951633450 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d24009450 con 0x556f61df6480 2023-10-09T06:32:11.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.818+0000 7f8d20ff9700 1 -- 172.21.15.163:0/951633450 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d24056be0 con 0x556f61df6480 2023-10-09T06:32:11.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.819+0000 7f8d3a7a4b80 1 -- 172.21.15.163:0/951633450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556f61df6480 msgr2=0x556f61da8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:11.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.819+0000 7f8d3a7a4b80 1 --2- 172.21.15.163:0/951633450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556f61df6480 0x556f61da8a40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8d24015040 tx=0x7f8d2406b000 comp rx=0 tx=0).stop 2023-10-09T06:32:11.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.819+0000 7f8d3a7a4b80 1 -- 172.21.15.163:0/951633450 shutdown_connections 2023-10-09T06:32:11.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.819+0000 7f8d3a7a4b80 1 --2- 172.21.15.163:0/951633450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556f61df6480 0x556f61da8a40 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.819+0000 7f8d3a7a4b80 1 --2- 172.21.15.163:0/951633450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f61da95f0 0x556f61e57dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.819+0000 7f8d3a7a4b80 1 --2- 172.21.15.163:0/951633450 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556f61dabb80 0x556f61df4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.819+0000 7f8d3a7a4b80 1 -- 172.21.15.163:0/951633450 >> 172.21.15.163:0/951633450 conn(0x556f61dad0c0 msgr2=0x556f61da1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:11.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.819+0000 7f8d3a7a4b80 1 -- 172.21.15.163:0/951633450 shutdown_connections 2023-10-09T06:32:11.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.819+0000 7f8d3a7a4b80 1 -- 172.21.15.163:0/951633450 wait complete. 2023-10-09T06:32:11.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.819+0000 7f8d3a7a4b80 1 -- start start 2023-10-09T06:32:11.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.819+0000 7f8d3a7a4b80 1 Processor -- start 2023-10-09T06:32:11.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.820+0000 7f8d3a7a4b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556f61da95f0 0x556f61da8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.820+0000 7f8d3a7a4b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f61dabb80 0x556f61da8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.820+0000 7f8d3a7a4b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556f61df6480 0x556f61e609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.820+0000 7f8d3a7a4b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f61dd4bc0 con 0x556f61df6480 2023-10-09T06:32:11.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.820+0000 7f8d3a7a4b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f61dcaa90 con 0x556f61dabb80 2023-10-09T06:32:11.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.820+0000 7f8d3a7a4b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f61e04eb0 con 0x556f61da95f0 2023-10-09T06:32:11.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.820+0000 7f8d3a7a4b80 10 client.0 _ll_register_callbacks cb 0x556f61d6f2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-09T06:32:11.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.820+0000 7f8d2b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556f61df6480 0x556f61e609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.820+0000 7f8d2b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556f61df6480 0x556f61e609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:32892/0 (socket says 172.21.15.163:32892) 2023-10-09T06:32:11.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.820+0000 7f8d2b7fe700 1 -- 172.21.15.163:0/2860459122 learned_addr learned my addr 172.21.15.163:0/2860459122 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:11.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.820+0000 7f8d227fc700 1 --2- 172.21.15.163:0/2860459122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f61dabb80 0x556f61da8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.820+0000 7f8d2affd700 1 --2- 172.21.15.163:0/2860459122 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556f61da95f0 0x556f61da8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.821+0000 7f8d2b7fe700 1 -- 172.21.15.163:0/2860459122 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556f61da95f0 msgr2=0x556f61da8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:11.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.821+0000 7f8d2b7fe700 1 --2- 172.21.15.163:0/2860459122 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556f61da95f0 0x556f61da8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.821+0000 7f8d2b7fe700 1 -- 172.21.15.163:0/2860459122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f61dabb80 msgr2=0x556f61da8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:11.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.821+0000 7f8d2b7fe700 1 --2- 172.21.15.163:0/2860459122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f61dabb80 0x556f61da8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.821+0000 7f8d2b7fe700 1 -- 172.21.15.163:0/2860459122 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f61b38fe0 con 0x556f61df6480 2023-10-09T06:32:11.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.821+0000 7f8d2b7fe700 1 --2- 172.21.15.163:0/2860459122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556f61df6480 0x556f61e609b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8d24009ab0 tx=0x7f8d24084000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12c673f6fae19c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:11.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.821+0000 7f8d097fa700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-10-09T06:32:11.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.821+0000 7f8d097fa700 1 -- 172.21.15.163:0/2860459122 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2400e040 con 0x556f61df6480 2023-10-09T06:32:11.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.821+0000 7f8d097fa700 1 -- 172.21.15.163:0/2860459122 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8d24056e10 con 0x556f61df6480 2023-10-09T06:32:11.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.821+0000 7f8d3a7a4b80 1 -- 172.21.15.163:0/2860459122 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d1400d9b0 con 0x556f61df6480 2023-10-09T06:32:11.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.821+0000 7f8d097fa700 1 -- 172.21.15.163:0/2860459122 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d24056be0 con 0x556f61df6480 2023-10-09T06:32:11.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.821+0000 7f8d3a7a4b80 10 client.5538 fetch_fsmap learned FSMap version 31 2023-10-09T06:32:11.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.821+0000 7f8d3a7a4b80 1 -- 172.21.15.163:0/2860459122 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x556f61cb1470 con 0x556f61df6480 2023-10-09T06:32:11.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.821+0000 7f8d097fa700 1 -- 172.21.15.163:0/2860459122 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1400d9b0 con 0x556f61df6480 2023-10-09T06:32:11.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.821+0000 7f8d097fa700 1 -- 172.21.15.163:0/2860459122 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 31) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f8d2405d920 con 0x556f61df6480 2023-10-09T06:32:11.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.821+0000 7f8d3a7a4b80 10 client.5538 fetch_fsmap finished waiting for FSMap version 31 2023-10-09T06:32:11.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.821+0000 7f8d3a7a4b80 10 client.5538 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-09T06:32:11.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.821+0000 7f8d3a7a4b80 20 client.5538 populate_metadata read hostname 'smithi163' 2023-10-09T06:32:11.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.821+0000 7f8d3a7a4b80 1 --2- 172.21.15.163:0/2860459122 >> [v2:172.21.15.181:6835/2945196635,v1:172.21.15.181:6837/2945196635] conn(0x556f61df5560 0x556f61e60ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.821+0000 7f8d3a7a4b80 4 client.5538 mds_command: new command op to 5370 tid=0[{"prefix": "status"}] 2023-10-09T06:32:11.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.821+0000 7f8d3a7a4b80 1 -- 172.21.15.163:0/2860459122 --> [v2:172.21.15.181:6835/2945196635,v1:172.21.15.181:6837/2945196635] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556f61c9ee00 con 0x556f61df5560 2023-10-09T06:32:11.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.821+0000 7f8d2affd700 1 --2- 172.21.15.163:0/2860459122 >> [v2:172.21.15.181:6835/2945196635,v1:172.21.15.181:6837/2945196635] conn(0x556f61df5560 0x556f61e60ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.822+0000 7f8d097fa700 10 client.5538 ms_handle_connect on v2:172.21.15.181:6835/2945196635 2023-10-09T06:32:11.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.822+0000 7f8d2affd700 1 --2- 172.21.15.163:0/2860459122 >> [v2:172.21.15.181:6835/2945196635,v1:172.21.15.181:6837/2945196635] conn(0x556f61df5560 0x556f61e60ef0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=530bc33a864eba31 server_cookie=780d9c492aea8d23 in_seq=0 out_seq=0 2023-10-09T06:32:11.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.822+0000 7f8d097fa700 1 -- 172.21.15.163:0/2860459122 <== mds.0 v2:172.21.15.181:6835/2945196635 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x556f61c9ee00 con 0x556f61df5560 2023-10-09T06:32:11.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.822+0000 7f8d097fa700 10 client.5538 handle_command_reply: tid=0 2023-10-09T06:32:11.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.822+0000 7f8d3a7a4b80 1 client.5538 shutdown 2023-10-09T06:32:11.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.822+0000 7f8d3a7a4b80 1 -- 172.21.15.163:0/2860459122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556f61df6480 msgr2=0x556f61e609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:11.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.822+0000 7f8d3a7a4b80 1 --2- 172.21.15.163:0/2860459122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556f61df6480 0x556f61e609b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8d24009ab0 tx=0x7f8d24084000 comp rx=0 tx=0).stop 2023-10-09T06:32:11.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.822+0000 7f8d3a7a4b80 1 -- 172.21.15.163:0/2860459122 shutdown_connections 2023-10-09T06:32:11.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.822+0000 7f8d3a7a4b80 1 --2- 172.21.15.163:0/2860459122 >> [v2:172.21.15.181:6835/2945196635,v1:172.21.15.181:6837/2945196635] conn(0x556f61df5560 0x556f61e60ef0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.822+0000 7f8d3a7a4b80 1 --2- 172.21.15.163:0/2860459122 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556f61df6480 0x556f61e609b0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.822+0000 7f8d097fa700 0 client.5538 ms_handle_reset on v2:172.21.15.181:6835/2945196635 2023-10-09T06:32:11.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.822+0000 7f8d3a7a4b80 1 --2- 172.21.15.163:0/2860459122 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556f61da95f0 0x556f61da8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.822+0000 7f8d3a7a4b80 1 --2- 172.21.15.163:0/2860459122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556f61dabb80 0x556f61da8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.822+0000 7f8d3a7a4b80 1 -- 172.21.15.163:0/2860459122 >> 172.21.15.163:0/2860459122 conn(0x556f61dad0c0 msgr2=0x556f61e01c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:11.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.822+0000 7f8d3a7a4b80 1 -- 172.21.15.163:0/2860459122 shutdown_connections 2023-10-09T06:32:11.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.822+0000 7f8d3a7a4b80 1 -- 172.21.15.163:0/2860459122 wait complete. 2023-10-09T06:32:11.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.822+0000 7f8d3a7a4b80 20 client.5538 trim_cache size 0 max 16384 2023-10-09T06:32:11.831 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:32:11.831 INFO:teuthology.orchestra.run.smithi163.stdout: "cluster_fsid": "2520b8f7-539f-4bae-9f17-1fce66f66ddd", 2023-10-09T06:32:11.831 INFO:teuthology.orchestra.run.smithi163.stdout: "whoami": 0, 2023-10-09T06:32:11.831 INFO:teuthology.orchestra.run.smithi163.stdout: "id": 5370, 2023-10-09T06:32:11.831 INFO:teuthology.orchestra.run.smithi163.stdout: "want_state": "up:active", 2023-10-09T06:32:11.831 INFO:teuthology.orchestra.run.smithi163.stdout: "state": "up:active", 2023-10-09T06:32:11.831 INFO:teuthology.orchestra.run.smithi163.stdout: "fs_name": "cephfs", 2023-10-09T06:32:11.832 INFO:teuthology.orchestra.run.smithi163.stdout: "rank_uptime": 10.272520758000001, 2023-10-09T06:32:11.832 INFO:teuthology.orchestra.run.smithi163.stdout: "mdsmap_epoch": 31, 2023-10-09T06:32:11.832 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch": 51, 2023-10-09T06:32:11.832 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch_barrier": 51, 2023-10-09T06:32:11.832 INFO:teuthology.orchestra.run.smithi163.stdout: "uptime": 15.393265798 2023-10-09T06:32:11.832 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:32:11.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.823+0000 7f8d2bfff700 1 -- 172.21.15.163:0/1979384502 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8d0c066f30 msgr2=0x7f8d0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:11.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.823+0000 7f8d2bfff700 1 --2- 172.21.15.163:0/1979384502 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8d0c066f30 0x7f8d0c0693e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c157430 tx=0x7f8d1c008040 comp rx=0 tx=0).stop 2023-10-09T06:32:11.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.823+0000 7f8d2bfff700 1 -- 172.21.15.163:0/1979384502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d2c15d230 msgr2=0x7f8d2c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:11.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.823+0000 7f8d2bfff700 1 --2- 172.21.15.163:0/1979384502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d2c15d230 0x7f8d2c0b4e20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c089340 tx=0x7f8d24017000 comp rx=0 tx=0).stop 2023-10-09T06:32:11.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.823+0000 7f8d2bfff700 1 -- 172.21.15.163:0/1979384502 shutdown_connections 2023-10-09T06:32:11.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.823+0000 7f8d2bfff700 1 --2- 172.21.15.163:0/1979384502 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8d0c066f30 0x7f8d0c0693e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.823+0000 7f8d2bfff700 1 --2- 172.21.15.163:0/1979384502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d2c15d230 0x7f8d2c0b4e20 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.823+0000 7f8d2bfff700 1 --2- 172.21.15.163:0/1979384502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d2c159420 0x7f8d2c0c5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.823+0000 7f8d2bfff700 1 --2- 172.21.15.163:0/1979384502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d2c1597f0 0x7f8d2c168b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.823+0000 7f8d2bfff700 1 -- 172.21.15.163:0/1979384502 >> 172.21.15.163:0/1979384502 conn(0x7f8d2c0d0660 msgr2=0x7f8d2c156ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:11.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.823+0000 7f8d2bfff700 1 -- 172.21.15.163:0/1979384502 shutdown_connections 2023-10-09T06:32:11.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.823+0000 7f8d2bfff700 1 -- 172.21.15.163:0/1979384502 wait complete. 2023-10-09T06:32:11.835 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-09T06:32:11.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.949+0000 7f010359e700 1 Processor -- start 2023-10-09T06:32:11.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.949+0000 7f010359e700 1 -- start start 2023-10-09T06:32:11.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.950+0000 7f010359e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0104159420 0x7f01040b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.950+0000 7f010359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f010415d230 0x7f010415d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.950+0000 7f010359e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01040b2980 con 0x7f01041597f0 2023-10-09T06:32:11.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.950+0000 7f010359e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f010415dbb0 con 0x7f0104159420 2023-10-09T06:32:11.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.950+0000 7f010359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01040b11d0 con 0x7f010415d230 2023-10-09T06:32:11.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.950+0000 7f0101d9b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f01041597f0 0x7f01040b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:56712/0 (socket says 172.21.15.163:56712) 2023-10-09T06:32:11.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.950+0000 7f0101d9b700 1 -- 172.21.15.163:0/355312285 learned_addr learned my addr 172.21.15.163:0/355312285 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:11.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.950+0000 7f010259c700 1 --2- 172.21.15.163:0/355312285 >> v2:172.21.15.181:3300/0 conn(0x7f0104159420 0x7f01040b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.950+0000 7f0102d9d700 1 --2- 172.21.15.163:0/355312285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f010415d230 0x7f010415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.950+0000 7f010159a700 1 -- 172.21.15.163:0/355312285 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 809144641 0 0) 0x7f01040b2980 con 0x7f01041597f0 2023-10-09T06:32:11.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.950+0000 7f010159a700 1 -- 172.21.15.163:0/355312285 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00f0003570 con 0x7f01041597f0 2023-10-09T06:32:11.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.950+0000 7f010259c700 1 -- 172.21.15.163:0/355312285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f010415d230 msgr2=0x7f010415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:11.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.950+0000 7f010259c700 1 --2- 172.21.15.163:0/355312285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f010415d230 0x7f010415d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.951+0000 7f010259c700 1 -- 172.21.15.163:0/355312285 >> v1:172.21.15.163:6789/0 conn(0x7f01041597f0 legacy=0x7f01040b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:11.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.951+0000 7f010259c700 1 -- 172.21.15.163:0/355312285 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01040b14b0 con 0x7f0104159420 2023-10-09T06:32:11.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.951+0000 7f0102d9d700 1 --2- 172.21.15.163:0/355312285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f010415d230 0x7f010415d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:32:11.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.951+0000 7f010259c700 1 --2- 172.21.15.163:0/355312285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0104159420 0x7f01040b5f40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0098e0 tx=0x7f00fc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d46e2b07c7e0f2b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:11.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.951+0000 7f010159a700 1 -- 172.21.15.163:0/355312285 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc011630 con 0x7f0104159420 2023-10-09T06:32:11.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.951+0000 7f010159a700 1 -- 172.21.15.163:0/355312285 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00fc011790 con 0x7f0104159420 2023-10-09T06:32:11.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.951+0000 7f010159a700 1 -- 172.21.15.163:0/355312285 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc011b30 con 0x7f0104159420 2023-10-09T06:32:11.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.951+0000 7f010359e700 1 -- 172.21.15.163:0/355312285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0104159420 msgr2=0x7f01040b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:11.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.951+0000 7f010359e700 1 --2- 172.21.15.163:0/355312285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0104159420 0x7f01040b5f40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0098e0 tx=0x7f00fc0091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.951+0000 7f010359e700 1 -- 172.21.15.163:0/355312285 shutdown_connections 2023-10-09T06:32:11.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.951+0000 7f010359e700 1 --2- 172.21.15.163:0/355312285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0104159420 0x7f01040b5f40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.951+0000 7f010359e700 1 --2- 172.21.15.163:0/355312285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f010415d230 0x7f010415d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.951+0000 7f010359e700 1 -- 172.21.15.163:0/355312285 >> 172.21.15.163:0/355312285 conn(0x7f01040d0660 msgr2=0x7f01040d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:11.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.951+0000 7f010359e700 1 -- 172.21.15.163:0/355312285 shutdown_connections 2023-10-09T06:32:11.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.951+0000 7f010359e700 1 -- 172.21.15.163:0/355312285 wait complete. 2023-10-09T06:32:11.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.952+0000 7f010359e700 1 Processor -- start 2023-10-09T06:32:11.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.953+0000 7f010359e700 1 -- start start 2023-10-09T06:32:11.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.953+0000 7f010359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0104159420 0x7f010406e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.953+0000 7f010359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01041597f0 0x7f0104170c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.953+0000 7f010359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f010415d230 0x7f01040c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.953+0000 7f010359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010406b6b0 con 0x7f010415d230 2023-10-09T06:32:11.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.953+0000 7f010359e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010406f190 con 0x7f0104159420 2023-10-09T06:32:11.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.953+0000 7f010359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f010406b900 con 0x7f01041597f0 2023-10-09T06:32:11.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.953+0000 7f0102d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f010415d230 0x7f01040c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.953+0000 7f0102d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f010415d230 0x7f01040c58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:32904/0 (socket says 172.21.15.163:32904) 2023-10-09T06:32:11.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.953+0000 7f0102d9d700 1 -- 172.21.15.163:0/1737855150 learned_addr learned my addr 172.21.15.163:0/1737855150 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:11.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.954+0000 7f010259c700 1 --2- 172.21.15.163:0/1737855150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0104159420 0x7f010406e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.954+0000 7f0101d9b700 1 --2- 172.21.15.163:0/1737855150 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01041597f0 0x7f0104170c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.954+0000 7f0102d9d700 1 -- 172.21.15.163:0/1737855150 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01041597f0 msgr2=0x7f0104170c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:11.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.954+0000 7f0102d9d700 1 --2- 172.21.15.163:0/1737855150 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01041597f0 0x7f0104170c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.954+0000 7f0102d9d700 1 -- 172.21.15.163:0/1737855150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0104159420 msgr2=0x7f010406e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:11.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.954+0000 7f0102d9d700 1 --2- 172.21.15.163:0/1737855150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0104159420 0x7f010406e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:11.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.954+0000 7f0102d9d700 1 -- 172.21.15.163:0/1737855150 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f4009a20 con 0x7f010415d230 2023-10-09T06:32:11.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.954+0000 7f0101d9b700 1 --2- 172.21.15.163:0/1737855150 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01041597f0 0x7f0104170c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:11.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.954+0000 7f0102d9d700 1 --2- 172.21.15.163:0/1737855150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f010415d230 0x7f01040c58d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009f20 tx=0x7f00f400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:11.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.954+0000 7f00fa7fc700 1 -- 172.21.15.163:0/1737855150 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f400cbd0 con 0x7f010415d230 2023-10-09T06:32:11.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.954+0000 7f00fa7fc700 1 -- 172.21.15.163:0/1737855150 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f400f040 con 0x7f010415d230 2023-10-09T06:32:11.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.954+0000 7f010359e700 1 -- 172.21.15.163:0/1737855150 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00fc009420 con 0x7f010415d230 2023-10-09T06:32:11.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.954+0000 7f00fa7fc700 1 -- 172.21.15.163:0/1737855150 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4014480 con 0x7f010415d230 2023-10-09T06:32:11.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.954+0000 7f010359e700 1 -- 172.21.15.163:0/1737855150 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010406bec0 con 0x7f010415d230 2023-10-09T06:32:11.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.955+0000 7f010359e700 1 -- 172.21.15.163:0/1737855150 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f010404c3c0 con 0x7f010415d230 2023-10-09T06:32:11.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.955+0000 7f00fa7fc700 1 -- 172.21.15.163:0/1737855150 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f00f4003020 con 0x7f010415d230 2023-10-09T06:32:11.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.956+0000 7f00fa7fc700 1 --2- 172.21.15.163:0/1737855150 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f00e4066f30 0x7f00e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:11.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.956+0000 7f00fa7fc700 1 -- 172.21.15.163:0/1737855150 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f00f4084e20 con 0x7f010415d230 2023-10-09T06:32:11.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.956+0000 7f010259c700 1 --2- 172.21.15.163:0/1737855150 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f00e4066f30 0x7f00e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:11.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.956+0000 7f010259c700 1 --2- 172.21.15.163:0/1737855150 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f00e4066f30 0x7f00e40693e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0091e0 tx=0x7f00fc017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:11.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:11.960+0000 7f00fa7fc700 1 -- 172.21.15.163:0/1737855150 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f00f4052090 con 0x7f010415d230 2023-10-09T06:32:12.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.184+0000 7f010359e700 1 -- 172.21.15.163:0/1737855150 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f01040b1790 con 0x7f010415d230 2023-10-09T06:32:12.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.185+0000 7f00fa7fc700 1 -- 172.21.15.163:0/1737855150 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v31) v1 ==== 53+0+73 (secure 0 0 0) 0x7f00f4018060 con 0x7f010415d230 2023-10-09T06:32:12.186 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-09T06:32:12.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.187+0000 7f010359e700 1 -- 172.21.15.163:0/1737855150 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f00e4066f30 msgr2=0x7f00e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:12.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.187+0000 7f010359e700 1 --2- 172.21.15.163:0/1737855150 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f00e4066f30 0x7f00e40693e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0091e0 tx=0x7f00fc017000 comp rx=0 tx=0).stop 2023-10-09T06:32:12.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.187+0000 7f010359e700 1 -- 172.21.15.163:0/1737855150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f010415d230 msgr2=0x7f01040c58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:12.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.187+0000 7f010359e700 1 --2- 172.21.15.163:0/1737855150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f010415d230 0x7f01040c58d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009f20 tx=0x7f00f400b040 comp rx=0 tx=0).stop 2023-10-09T06:32:12.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.187+0000 7f010359e700 1 -- 172.21.15.163:0/1737855150 shutdown_connections 2023-10-09T06:32:12.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.187+0000 7f010359e700 1 --2- 172.21.15.163:0/1737855150 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f00e4066f30 0x7f00e40693e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:12.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.187+0000 7f010359e700 1 --2- 172.21.15.163:0/1737855150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f010415d230 0x7f01040c58d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:12.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.187+0000 7f010359e700 1 --2- 172.21.15.163:0/1737855150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0104159420 0x7f010406e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:12.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.187+0000 7f010359e700 1 --2- 172.21.15.163:0/1737855150 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01041597f0 0x7f0104170c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:12.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.187+0000 7f010359e700 1 -- 172.21.15.163:0/1737855150 >> 172.21.15.163:0/1737855150 conn(0x7f01040d0660 msgr2=0x7f01040ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:12.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.187+0000 7f010359e700 1 -- 172.21.15.163:0/1737855150 shutdown_connections 2023-10-09T06:32:12.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.187+0000 7f010359e700 1 -- 172.21.15.163:0/1737855150 wait complete. 2023-10-09T06:32:12.199 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-09T06:32:12.200 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-09T06:32:12.200 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi163.front.sepia.ceph.com 2023-10-09T06:32:12.200 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-09T06:32:12.200 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-09T06:32:12.200 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-09T06:32:12.200 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-09T06:32:12.200 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-09T06:32:12.200 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:32:12.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:12.201 DEBUG:teuthology.orchestra.run.smithi163:> ip addr 2023-10-09T06:32:12.215 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:32:12.216 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:32:12.216 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:32:12.216 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:32:12.216 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2023-10-09T06:32:12.216 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:32:12.216 INFO:teuthology.orchestra.run.smithi163.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:32:12.216 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:06 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:32:12.216 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f0 2023-10-09T06:32:12.216 INFO:teuthology.orchestra.run.smithi163.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:32:12.217 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:07 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:32:12.217 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f1 2023-10-09T06:32:12.218 INFO:teuthology.orchestra.run.smithi163.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:32:12.218 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5a brd ff:ff:ff:ff:ff:ff 2023-10-09T06:32:12.218 INFO:teuthology.orchestra.run.smithi163.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-09T06:32:12.218 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5b brd ff:ff:ff:ff:ff:ff 2023-10-09T06:32:12.218 INFO:teuthology.orchestra.run.smithi163.stdout: inet 172.21.15.163/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-09T06:32:12.218 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft 42391sec preferred_lft 42391sec 2023-10-09T06:32:12.218 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::ec4:7aff:fe88:7d5b/64 scope link 2023-10-09T06:32:12.218 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:32:12.219 INFO:teuthology.orchestra.run.smithi163.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-09T06:32:12.219 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 2e:01:84:b9:f4:8e brd ff:ff:ff:ff:ff:ff 2023-10-09T06:32:12.219 INFO:teuthology.orchestra.run.smithi163.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-09T06:32:12.219 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:32:12.219 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::d03c:93ff:fe9e:c8cb/64 scope link 2023-10-09T06:32:12.219 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:32:12.219 INFO:teuthology.orchestra.run.smithi163.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:32:12.219 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 2e:01:84:b9:f4:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-09T06:32:12.220 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::2c01:84ff:feb9:f48e/64 scope link 2023-10-09T06:32:12.220 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:32:12.220 INFO:teuthology.orchestra.run.smithi163.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:32:12.220 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether c6:88:3e:d7:c1:56 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-09T06:32:12.220 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::c488:3eff:fed7:c156/64 scope link 2023-10-09T06:32:12.220 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:32:12.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:12.221 DEBUG:teuthology.orchestra.run.smithi163:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-09T06:32:12.279 INFO:teuthology.orchestra.run.smithi163.stdout:1 2023-10-09T06:32:12.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:12.281 DEBUG:teuthology.orchestra.run.smithi163:> route 2023-10-09T06:32:12.336 INFO:teuthology.orchestra.run.smithi163.stdout:Kernel IP routing table 2023-10-09T06:32:12.337 INFO:teuthology.orchestra.run.smithi163.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-09T06:32:12.337 INFO:teuthology.orchestra.run.smithi163.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-09T06:32:12.337 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-09T06:32:12.337 INFO:teuthology.orchestra.run.smithi163.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-09T06:32:12.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:12.337 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-09T06:32:12.337 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-09T06:32:12.338 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-09T06:32:12.338 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-09T06:32:12.338 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-09T06:32:12.338 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-09T06:32:12.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:12.438 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2023-10-09T06:32:12.454 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-09T06:32:12.454 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-09T06:32:12.454 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:32:12.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:12.454 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set brx.0 up 2023-10-09T06:32:12.518 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:32:12.518 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-09T06:32:12.572 INFO:teuthology.orchestra.run.smithi163.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:32:12.573 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:32:12.573 DEBUG:teuthology.orchestra.run.smithi163:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-09T06:32:12.627 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-09T06:32:12.627 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-09T06:32:12.627 DEBUG:teuthology.orchestra.run.smithi163:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:32:12.707 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:32:12.707 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:32:12.726 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:32:12.726 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:32:12.726 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:32:12.726 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with new device syntax: 0@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:32:12.726 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:32:12.726 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:32:12.727 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:32:12.727 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:32:12.749 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:32:12.749 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:32:12.750 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:32:12.750 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with new device syntax: 0@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:32:12.750 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:32:12.750 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:32:12.750 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:32:12.750 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:32:12.750 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790" 2023-10-09T06:32:12.750 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790:/ 2023-10-09T06:32:12.750 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=2520b8f7-539f-4bae-9f17-1fce66f66ddd" will pass to kernel 2023-10-09T06:32:12.752 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-09T06:32:12.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:12.752 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-09T06:32:12.817 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-09T06:32:12.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.972+0000 7f57f7525700 1 Processor -- start 2023-10-09T06:32:12.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.972+0000 7f57f7525700 1 -- start start 2023-10-09T06:32:12.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.973+0000 7f57f7525700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f57f0159420 0x7f57f00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:12.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.973+0000 7f57f7525700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f00b2980 con 0x7f57f015d230 2023-10-09T06:32:12.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.973+0000 7f57f7525700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f00b11d0 con 0x7f57f01597f0 2023-10-09T06:32:12.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.973+0000 7f57f7525700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57f00b14b0 con 0x7f57f0159420 2023-10-09T06:32:12.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.973+0000 7f57f6523700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f57f0159420 0x7f57f00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:12.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.973+0000 7f57f6d24700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f57f015d230 0x7f57f015d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:56718/0 (socket says 172.21.15.163:56718) 2023-10-09T06:32:12.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.973+0000 7f57f6523700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f57f0159420 0x7f57f00b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:32910/0 (socket says 172.21.15.163:32910) 2023-10-09T06:32:12.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.973+0000 7f57f6d24700 1 -- 172.21.15.163:0/2588737682 learned_addr learned my addr 172.21.15.163:0/2588737682 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:12.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.973+0000 7f57f6523700 1 -- 172.21.15.163:0/2588737682 >> v1:172.21.15.181:6789/0 conn(0x7f57f01597f0 legacy=0x7f57f00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:12.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.973+0000 7f57f6523700 1 -- 172.21.15.163:0/2588737682 >> v1:172.21.15.163:6789/0 conn(0x7f57f015d230 legacy=0x7f57f015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:12.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.973+0000 7f57f6523700 1 -- 172.21.15.163:0/2588737682 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f00b1790 con 0x7f57f0159420 2023-10-09T06:32:12.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.973+0000 7f57f5521700 1 -- 172.21.15.163:0/2588737682 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1792994841 0 0) 0x7f57f00b2980 con 0x7f57f015d230 2023-10-09T06:32:12.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.974+0000 7f57f6523700 1 --2- 172.21.15.163:0/2588737682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57f0159420 0x7f57f00b5f40 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009e60 tx=0x7f57ec009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1100a6a64326e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:12.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.974+0000 7f57f5521700 1 -- 172.21.15.163:0/2588737682 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec00e070 con 0x7f57f0159420 2023-10-09T06:32:12.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.974+0000 7f57f5521700 1 -- 172.21.15.163:0/2588737682 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ec00bbf0 con 0x7f57f0159420 2023-10-09T06:32:12.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.974+0000 7f57f5521700 1 -- 172.21.15.163:0/2588737682 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec012430 con 0x7f57f0159420 2023-10-09T06:32:12.977 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.974+0000 7f57f7525700 1 -- 172.21.15.163:0/2588737682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57f0159420 msgr2=0x7f57f00b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:12.977 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.974+0000 7f57f7525700 1 --2- 172.21.15.163:0/2588737682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57f0159420 0x7f57f00b5f40 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009e60 tx=0x7f57ec009780 comp rx=0 tx=0).stop 2023-10-09T06:32:12.977 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.974+0000 7f57f7525700 1 -- 172.21.15.163:0/2588737682 shutdown_connections 2023-10-09T06:32:12.977 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.974+0000 7f57f7525700 1 --2- 172.21.15.163:0/2588737682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57f0159420 0x7f57f00b5f40 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:12.977 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.974+0000 7f57f7525700 1 -- 172.21.15.163:0/2588737682 >> 172.21.15.163:0/2588737682 conn(0x7f57f00d0660 msgr2=0x7f57f00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:12.977 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.974+0000 7f57f7525700 1 -- 172.21.15.163:0/2588737682 shutdown_connections 2023-10-09T06:32:12.977 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.974+0000 7f57f7525700 1 -- 172.21.15.163:0/2588737682 wait complete. 2023-10-09T06:32:12.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.975+0000 7f57f7525700 1 Processor -- start 2023-10-09T06:32:12.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.976+0000 7f57f7525700 1 -- start start 2023-10-09T06:32:12.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.976+0000 7f57f7525700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57f0159420 0x7f57f00c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:12.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.976+0000 7f57f7525700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57f01597f0 0x7f57f0155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:12.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.976+0000 7f57f7525700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57f015d230 0x7f57f01581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:12.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.976+0000 7f57f7525700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f00c5bd0 con 0x7f57f01597f0 2023-10-09T06:32:12.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.976+0000 7f57f7525700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f016a520 con 0x7f57f015d230 2023-10-09T06:32:12.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.976+0000 7f57f7525700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f00ca430 con 0x7f57f0159420 2023-10-09T06:32:12.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.976+0000 7f57f5d22700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57f01597f0 0x7f57f0155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:12.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.976+0000 7f57f5d22700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57f01597f0 0x7f57f0155350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:32918/0 (socket says 172.21.15.163:32918) 2023-10-09T06:32:12.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.976+0000 7f57f5d22700 1 -- 172.21.15.163:0/2092454218 learned_addr learned my addr 172.21.15.163:0/2092454218 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:12.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.977+0000 7f57f6d24700 1 --2- 172.21.15.163:0/2092454218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57f015d230 0x7f57f01581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:12.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.977+0000 7f57f5d22700 1 -- 172.21.15.163:0/2092454218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57f0159420 msgr2=0x7f57f00c60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:12.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.977+0000 7f57f5d22700 1 --2- 172.21.15.163:0/2092454218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57f0159420 0x7f57f00c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:12.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.977+0000 7f57f5d22700 1 -- 172.21.15.163:0/2092454218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57f015d230 msgr2=0x7f57f01581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:12.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.977+0000 7f57f5d22700 1 --2- 172.21.15.163:0/2092454218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57f015d230 0x7f57f01581b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:12.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.977+0000 7f57f5d22700 1 -- 172.21.15.163:0/2092454218 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec009a20 con 0x7f57f01597f0 2023-10-09T06:32:12.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.977+0000 7f57f5d22700 1 --2- 172.21.15.163:0/2092454218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57f01597f0 0x7f57f0155350 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f57e000b7e0 tx=0x7f57e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:12.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.977+0000 7f57e67fc700 1 -- 172.21.15.163:0/2092454218 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e000f510 con 0x7f57f01597f0 2023-10-09T06:32:12.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.977+0000 7f57e67fc700 1 -- 172.21.15.163:0/2092454218 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e000f670 con 0x7f57f01597f0 2023-10-09T06:32:12.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.977+0000 7f57e67fc700 1 -- 172.21.15.163:0/2092454218 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e000fa10 con 0x7f57f01597f0 2023-10-09T06:32:12.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.977+0000 7f57f7525700 1 -- 172.21.15.163:0/2092454218 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f00ca710 con 0x7f57f01597f0 2023-10-09T06:32:12.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.977+0000 7f57f7525700 1 -- 172.21.15.163:0/2092454218 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f00c6600 con 0x7f57f01597f0 2023-10-09T06:32:12.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.978+0000 7f57f7525700 1 -- 172.21.15.163:0/2092454218 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f00cac30 con 0x7f57f01597f0 2023-10-09T06:32:12.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.978+0000 7f57e67fc700 1 -- 172.21.15.163:0/2092454218 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f57e001a400 con 0x7f57f01597f0 2023-10-09T06:32:12.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.979+0000 7f57e67fc700 1 --2- 172.21.15.163:0/2092454218 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f57d4066f30 0x7f57d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:12.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.979+0000 7f57e67fc700 1 -- 172.21.15.163:0/2092454218 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f57e0084080 con 0x7f57f01597f0 2023-10-09T06:32:12.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.979+0000 7f57f6523700 1 --2- 172.21.15.163:0/2092454218 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f57d4066f30 0x7f57d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:12.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.979+0000 7f57f6523700 1 --2- 172.21.15.163:0/2092454218 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f57d4066f30 0x7f57d40693e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009e60 tx=0x7f57ec018000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:12.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:12.983+0000 7f57e67fc700 1 -- 172.21.15.163:0/2092454218 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f57f00cac30 con 0x7f57f01597f0 2023-10-09T06:32:13.002 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:32:13.005 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:32:13.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.212+0000 7f57f7525700 1 -- 172.21.15.163:0/2092454218 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f57f00cac30 con 0x7f57f01597f0 2023-10-09T06:32:13.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.213+0000 7f57e67fc700 1 -- 172.21.15.163:0/2092454218 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v31) v1 ==== 53+0+73 (secure 0 0 0) 0x7f57f00cac30 con 0x7f57f01597f0 2023-10-09T06:32:13.214 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-09T06:32:13.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.215+0000 7f57f7525700 1 -- 172.21.15.163:0/2092454218 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f57d4066f30 msgr2=0x7f57d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:13.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.215+0000 7f57f7525700 1 --2- 172.21.15.163:0/2092454218 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f57d4066f30 0x7f57d40693e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009e60 tx=0x7f57ec018000 comp rx=0 tx=0).stop 2023-10-09T06:32:13.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.215+0000 7f57f7525700 1 -- 172.21.15.163:0/2092454218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57f01597f0 msgr2=0x7f57f0155350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:13.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.215+0000 7f57f7525700 1 --2- 172.21.15.163:0/2092454218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57f01597f0 0x7f57f0155350 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f57e000b7e0 tx=0x7f57e00091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:13.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.215+0000 7f57f7525700 1 -- 172.21.15.163:0/2092454218 shutdown_connections 2023-10-09T06:32:13.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.215+0000 7f57f7525700 1 --2- 172.21.15.163:0/2092454218 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f57d4066f30 0x7f57d40693e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:13.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.215+0000 7f57f7525700 1 --2- 172.21.15.163:0/2092454218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57f015d230 0x7f57f01581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:13.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.215+0000 7f57f7525700 1 --2- 172.21.15.163:0/2092454218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57f0159420 0x7f57f00c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:13.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.215+0000 7f57f7525700 1 --2- 172.21.15.163:0/2092454218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57f01597f0 0x7f57f0155350 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:13.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.215+0000 7f57f7525700 1 -- 172.21.15.163:0/2092454218 >> 172.21.15.163:0/2092454218 conn(0x7f57f00d0660 msgr2=0x7f57f0156190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:13.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.215+0000 7f57f7525700 1 -- 172.21.15.163:0/2092454218 shutdown_connections 2023-10-09T06:32:13.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.215+0000 7f57f7525700 1 -- 172.21.15.163:0/2092454218 wait complete. 2023-10-09T06:32:13.227 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-09T06:32:13.227 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-09T06:32:13.227 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-09T06:32:13.227 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-09T06:32:13.228 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-09T06:32:13.228 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-09T06:32:13.228 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-09T06:32:13.228 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-09T06:32:13.228 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:32:13.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:13.228 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-09T06:32:13.247 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:32:13.247 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:32:13.247 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:32:13.247 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:32:13.248 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-09T06:32:13.248 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:32:13.248 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:32:13.248 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:32:13.248 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2023-10-09T06:32:13.248 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:32:13.249 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:32:13.249 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2023-10-09T06:32:13.249 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:32:13.249 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-09T06:32:13.249 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-09T06:32:13.249 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-09T06:32:13.250 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-09T06:32:13.250 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42367sec preferred_lft 42367sec 2023-10-09T06:32:13.250 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2023-10-09T06:32:13.250 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:32:13.250 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-09T06:32:13.251 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 72:62:e6:72:7c:2e brd ff:ff:ff:ff:ff:ff 2023-10-09T06:32:13.251 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-09T06:32:13.251 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:32:13.251 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::e89a:4cff:fefb:7198/64 scope link 2023-10-09T06:32:13.251 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:32:13.251 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:32:13.251 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 72:62:e6:72:7c:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-09T06:32:13.251 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::7062:e6ff:fe72:7c2e/64 scope link 2023-10-09T06:32:13.252 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:32:13.252 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:32:13.252 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether fa:4e:a3:9d:5a:5f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-09T06:32:13.252 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::f84e:a3ff:fe9d:5a5f/64 scope link 2023-10-09T06:32:13.252 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:32:13.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:13.252 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-09T06:32:13.315 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-09T06:32:13.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:13.317 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-09T06:32:13.376 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-09T06:32:13.376 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-09T06:32:13.377 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-09T06:32:13.377 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-09T06:32:13.377 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-09T06:32:13.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:13.377 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-09T06:32:13.377 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-09T06:32:13.377 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-09T06:32:13.377 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-09T06:32:13.378 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-09T06:32:13.378 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-09T06:32:13.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:13.485 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-09T06:32:13.501 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-09T06:32:13.501 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-09T06:32:13.501 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:32:13.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:13.501 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-09T06:32:13.567 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:32:13.567 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-09T06:32:13.623 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:32:13.624 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:32:13.624 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-09T06:32:13.678 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-09T06:32:13.678 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-09T06:32:13.678 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:32:13.758 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:32:13.759 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:32:13.777 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:32:13.777 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:32:13.777 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:32:13.777 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 1@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:32:13.777 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:32:13.777 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:32:13.777 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:32:13.778 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:32:13.799 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:32:13.799 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:32:13.799 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:32:13.799 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 1@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:32:13.799 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:32:13.800 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:32:13.800 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:32:13.800 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:32:13.800 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790" 2023-10-09T06:32:13.800 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790:/ 2023-10-09T06:32:13.800 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=2520b8f7-539f-4bae-9f17-1fce66f66ddd" will pass to kernel 2023-10-09T06:32:13.801 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-09T06:32:13.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:13.802 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-09T06:32:13.866 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-09T06:32:13.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.987+0000 7f4177631700 1 Processor -- start 2023-10-09T06:32:13.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.987+0000 7f4177631700 1 -- start start 2023-10-09T06:32:13.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.988+0000 7f4177631700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41700cd910 0x7f41700ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:13.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.988+0000 7f4177631700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41700b5b00 con 0x7f41700b55a0 2023-10-09T06:32:13.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.988+0000 7f4177631700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41700b5da0 con 0x7f4170159ec0 2023-10-09T06:32:13.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.988+0000 7f4177631700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41700b6080 con 0x7f41700cd910 2023-10-09T06:32:13.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.988+0000 7f4175bce700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f41700b55a0 0x7f41700ce880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:56722/0 (socket says 172.21.15.163:56722) 2023-10-09T06:32:13.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.988+0000 7f4175bce700 1 -- 172.21.15.163:0/4170675188 learned_addr learned my addr 172.21.15.163:0/4170675188 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:13.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.989+0000 7f4174bcc700 1 --2- 172.21.15.163:0/4170675188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41700cd910 0x7f41700ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:13.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.989+0000 7f4167fff700 1 -- 172.21.15.163:0/4170675188 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1395004518 0 0) 0x7f41700b5b00 con 0x7f41700b55a0 2023-10-09T06:32:13.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.989+0000 7f4167fff700 1 -- 172.21.15.163:0/4170675188 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f415c003570 con 0x7f41700b55a0 2023-10-09T06:32:13.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.989+0000 7f4167fff700 1 -- 172.21.15.163:0/4170675188 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3492593450 0 0) 0x7f41700b5da0 con 0x7f4170159ec0 2023-10-09T06:32:13.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.989+0000 7f4167fff700 1 -- 172.21.15.163:0/4170675188 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41700b5b00 con 0x7f4170159ec0 2023-10-09T06:32:13.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.989+0000 7f4167fff700 1 -- 172.21.15.163:0/4170675188 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3323313204 0 0) 0x7f415c003570 con 0x7f41700b55a0 2023-10-09T06:32:13.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.989+0000 7f4167fff700 1 -- 172.21.15.163:0/4170675188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41700cd910 msgr2=0x7f41700ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:13.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.989+0000 7f4167fff700 1 --2- 172.21.15.163:0/4170675188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41700cd910 0x7f41700ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:13.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.989+0000 7f4167fff700 1 -- 172.21.15.163:0/4170675188 >> v1:172.21.15.181:6789/0 conn(0x7f4170159ec0 legacy=0x7f41700b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:13.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.989+0000 7f4167fff700 1 -- 172.21.15.163:0/4170675188 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41700cc100 con 0x7f41700b55a0 2023-10-09T06:32:13.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.989+0000 7f4167fff700 1 -- 172.21.15.163:0/4170675188 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f41680017f0 con 0x7f41700b55a0 2023-10-09T06:32:13.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.989+0000 7f4167fff700 1 -- 172.21.15.163:0/4170675188 >> v1:172.21.15.163:6789/0 conn(0x7f41700b55a0 legacy=0x7f41700ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:13.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.989+0000 7f4167fff700 1 --2- 172.21.15.163:0/4170675188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f415c003c20 0x7f415c003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:13.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.989+0000 7f4167fff700 1 --2- 172.21.15.163:0/4170675188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f415c004810 0x7f415c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:13.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.989+0000 7f4167fff700 1 --2- 172.21.15.163:0/4170675188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f415c0051c0 0x7f415c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:13.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.989+0000 7f4167fff700 1 -- 172.21.15.163:0/4170675188 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41700b6080 con 0x7f415c0051c0 2023-10-09T06:32:13.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.989+0000 7f4167fff700 1 -- 172.21.15.163:0/4170675188 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415c00bfa0 con 0x7f415c004810 2023-10-09T06:32:13.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.989+0000 7f4167fff700 1 -- 172.21.15.163:0/4170675188 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f415c00c160 con 0x7f415c003c20 2023-10-09T06:32:13.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.989+0000 7f4175bce700 1 --2- 172.21.15.163:0/4170675188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f415c0051c0 0x7f415c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:13.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.989+0000 7f4174bcc700 1 --2- 172.21.15.163:0/4170675188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f415c004810 0x7f415c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:13.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.989+0000 7f41753cd700 1 --2- 172.21.15.163:0/4170675188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f415c003c20 0x7f415c003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:13.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.989+0000 7f4175bce700 1 -- 172.21.15.163:0/4170675188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f415c003c20 msgr2=0x7f415c003ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:13.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.989+0000 7f4175bce700 1 --2- 172.21.15.163:0/4170675188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f415c003c20 0x7f415c003ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:13.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.989+0000 7f4175bce700 1 -- 172.21.15.163:0/4170675188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f415c004810 msgr2=0x7f415c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:13.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.989+0000 7f4175bce700 1 --2- 172.21.15.163:0/4170675188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f415c004810 0x7f415c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:13.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.989+0000 7f4175bce700 1 -- 172.21.15.163:0/4170675188 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415c003570 con 0x7f415c0051c0 2023-10-09T06:32:13.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.989+0000 7f4175bce700 1 --2- 172.21.15.163:0/4170675188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f415c0051c0 0x7f415c00ba60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f416800a8d0 tx=0x7f4168002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72020127fb2d8ac0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:13.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.990+0000 7f4167fff700 1 -- 172.21.15.163:0/4170675188 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41680070d0 con 0x7f415c0051c0 2023-10-09T06:32:13.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.990+0000 7f4167fff700 1 -- 172.21.15.163:0/4170675188 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4168007230 con 0x7f415c0051c0 2023-10-09T06:32:13.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.990+0000 7f4167fff700 1 -- 172.21.15.163:0/4170675188 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41680075f0 con 0x7f415c0051c0 2023-10-09T06:32:13.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.990+0000 7f4177631700 1 -- 172.21.15.163:0/4170675188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f415c0051c0 msgr2=0x7f415c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:13.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.990+0000 7f4177631700 1 --2- 172.21.15.163:0/4170675188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f415c0051c0 0x7f415c00ba60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f416800a8d0 tx=0x7f4168002190 comp rx=0 tx=0).stop 2023-10-09T06:32:13.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.990+0000 7f4175bce700 1 -- 172.21.15.163:0/4170675188 reap_dead start 2023-10-09T06:32:13.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.990+0000 7f4177631700 1 -- 172.21.15.163:0/4170675188 shutdown_connections 2023-10-09T06:32:13.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.990+0000 7f4177631700 1 -- 172.21.15.163:0/4170675188 >> 172.21.15.163:0/4170675188 conn(0x7f41700999f0 msgr2=0x7f41700b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:13.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.990+0000 7f4177631700 1 -- 172.21.15.163:0/4170675188 shutdown_connections 2023-10-09T06:32:13.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.990+0000 7f4177631700 1 -- 172.21.15.163:0/4170675188 wait complete. 2023-10-09T06:32:13.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.992+0000 7f4177631700 1 Processor -- start 2023-10-09T06:32:13.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.992+0000 7f4177631700 1 -- start start 2023-10-09T06:32:13.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.992+0000 7f4177631700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4170148170 0x7f417009dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:13.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.992+0000 7f4177631700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4170148540 0x7f4170148a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:13.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.992+0000 7f4177631700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4170148f70 0x7f417009cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:13.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.992+0000 7f4177631700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417009caa0 con 0x7f4170148170 2023-10-09T06:32:13.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.992+0000 7f4177631700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417009c1b0 con 0x7f4170148540 2023-10-09T06:32:13.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.992+0000 7f4177631700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4170158de0 con 0x7f4170148f70 2023-10-09T06:32:13.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.993+0000 7f41753cd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4170148170 0x7f417009dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:13.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.993+0000 7f41753cd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4170148170 0x7f417009dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:32930/0 (socket says 172.21.15.163:32930) 2023-10-09T06:32:13.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.993+0000 7f41753cd700 1 -- 172.21.15.163:0/2003075705 learned_addr learned my addr 172.21.15.163:0/2003075705 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:13.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.993+0000 7f4175bce700 1 --2- 172.21.15.163:0/2003075705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4170148f70 0x7f417009cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:13.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.993+0000 7f4174bcc700 1 --2- 172.21.15.163:0/2003075705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4170148540 0x7f4170148a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:13.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.993+0000 7f41753cd700 1 -- 172.21.15.163:0/2003075705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4170148f70 msgr2=0x7f417009cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:13.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.993+0000 7f41753cd700 1 --2- 172.21.15.163:0/2003075705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4170148f70 0x7f417009cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:13.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.993+0000 7f41753cd700 1 -- 172.21.15.163:0/2003075705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4170148540 msgr2=0x7f4170148a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:13.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.993+0000 7f41753cd700 1 --2- 172.21.15.163:0/2003075705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4170148540 0x7f4170148a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:13.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.993+0000 7f41753cd700 1 -- 172.21.15.163:0/2003075705 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f417009c450 con 0x7f4170148170 2023-10-09T06:32:13.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.993+0000 7f41753cd700 1 --2- 172.21.15.163:0/2003075705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4170148170 0x7f417009dfa0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f416c00f930 tx=0x7f416c007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:14.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.993+0000 7f41657fa700 1 -- 172.21.15.163:0/2003075705 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c002020 con 0x7f4170148170 2023-10-09T06:32:14.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.993+0000 7f41657fa700 1 -- 172.21.15.163:0/2003075705 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416c002180 con 0x7f4170148170 2023-10-09T06:32:14.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.993+0000 7f41657fa700 1 -- 172.21.15.163:0/2003075705 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c00d8d0 con 0x7f4170148170 2023-10-09T06:32:14.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.993+0000 7f4177631700 1 -- 172.21.15.163:0/2003075705 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f417009c730 con 0x7f4170148170 2023-10-09T06:32:14.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.993+0000 7f4177631700 1 -- 172.21.15.163:0/2003075705 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41700ad280 con 0x7f4170148170 2023-10-09T06:32:14.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.994+0000 7f4177631700 1 -- 172.21.15.163:0/2003075705 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f417009cd80 con 0x7f4170148170 2023-10-09T06:32:14.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.995+0000 7f41657fa700 1 -- 172.21.15.163:0/2003075705 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f416c00dcc0 con 0x7f4170148170 2023-10-09T06:32:14.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.995+0000 7f41657fa700 1 --2- 172.21.15.163:0/2003075705 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4154066f30 0x7f41540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:14.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.995+0000 7f41657fa700 1 -- 172.21.15.163:0/2003075705 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f416c084430 con 0x7f4170148170 2023-10-09T06:32:14.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.995+0000 7f4174bcc700 1 --2- 172.21.15.163:0/2003075705 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4154066f30 0x7f41540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:14.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.996+0000 7f4174bcc700 1 --2- 172.21.15.163:0/2003075705 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4154066f30 0x7f41540693e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f41700b2cd0 tx=0x7f41600091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:14.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:13.999+0000 7f41657fa700 1 -- 172.21.15.163:0/2003075705 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f416c0516a0 con 0x7f4170148170 2023-10-09T06:32:14.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.223+0000 7f4177631700 1 -- 172.21.15.163:0/2003075705 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4170063c90 con 0x7f4170148170 2023-10-09T06:32:14.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.597+0000 7f41657fa700 1 -- 172.21.15.163:0/2003075705 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v32) v1 ==== 136+0+0 (secure 0 0 0) 0x7f416c054cc0 con 0x7f4170148170 2023-10-09T06:32:14.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.599+0000 7f4177631700 1 -- 172.21.15.163:0/2003075705 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4154066f30 msgr2=0x7f41540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:14.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.599+0000 7f4177631700 1 --2- 172.21.15.163:0/2003075705 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4154066f30 0x7f41540693e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f41700b2cd0 tx=0x7f41600091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:14.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.599+0000 7f4177631700 1 -- 172.21.15.163:0/2003075705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4170148170 msgr2=0x7f417009dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:14.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.599+0000 7f4177631700 1 --2- 172.21.15.163:0/2003075705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4170148170 0x7f417009dfa0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f416c00f930 tx=0x7f416c007d80 comp rx=0 tx=0).stop 2023-10-09T06:32:14.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.599+0000 7f4177631700 1 -- 172.21.15.163:0/2003075705 shutdown_connections 2023-10-09T06:32:14.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.599+0000 7f4177631700 1 --2- 172.21.15.163:0/2003075705 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4154066f30 0x7f41540693e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:14.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.599+0000 7f4177631700 1 --2- 172.21.15.163:0/2003075705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4170148170 0x7f417009dfa0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:14.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.599+0000 7f4177631700 1 --2- 172.21.15.163:0/2003075705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4170148f70 0x7f417009cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:14.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.599+0000 7f4177631700 1 --2- 172.21.15.163:0/2003075705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4170148540 0x7f4170148a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:14.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.599+0000 7f4177631700 1 -- 172.21.15.163:0/2003075705 >> 172.21.15.163:0/2003075705 conn(0x7f41700999f0 msgr2=0x7f41700b4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:14.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.599+0000 7f4177631700 1 -- 172.21.15.163:0/2003075705 shutdown_connections 2023-10-09T06:32:14.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.599+0000 7f4177631700 1 -- 172.21.15.163:0/2003075705 wait complete. 2023-10-09T06:32:14.613 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-09T06:32:14.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.728+0000 7fd229e25700 1 Processor -- start 2023-10-09T06:32:14.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.728+0000 7fd229e25700 1 -- start start 2023-10-09T06:32:14.731 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.729+0000 7fd229e25700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd224143050 0x7fd224159c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:14.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.729+0000 7fd229e25700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd22406b9e0 0x7fd22406be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:14.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.729+0000 7fd229e25700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22406c390 con 0x7fd2240b3810 2023-10-09T06:32:14.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.729+0000 7fd229e25700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd224143420 con 0x7fd224143050 2023-10-09T06:32:14.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.729+0000 7fd229e25700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd22406a1e0 con 0x7fd22406b9e0 2023-10-09T06:32:14.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.729+0000 7fd228e23700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd224143050 0x7fd224159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:14.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.729+0000 7fd229624700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fd2240b3810 0x7fd2240b3c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:56724/0 (socket says 172.21.15.163:56724) 2023-10-09T06:32:14.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.729+0000 7fd228e23700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd224143050 0x7fd224159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:32932/0 (socket says 172.21.15.163:32932) 2023-10-09T06:32:14.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.729+0000 7fd229624700 1 -- 172.21.15.163:0/83989975 learned_addr learned my addr 172.21.15.163:0/83989975 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:14.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.729+0000 7fd223fff700 1 --2- 172.21.15.163:0/83989975 >> v2:172.21.15.181:3300/0 conn(0x7fd22406b9e0 0x7fd22406be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:14.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.729+0000 7fd228e23700 1 -- 172.21.15.163:0/83989975 >> v2:172.21.15.181:3300/0 conn(0x7fd22406b9e0 msgr2=0x7fd22406be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:14.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.729+0000 7fd228e23700 1 --2- 172.21.15.163:0/83989975 >> v2:172.21.15.181:3300/0 conn(0x7fd22406b9e0 0x7fd22406be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:14.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.729+0000 7fd228e23700 1 -- 172.21.15.163:0/83989975 >> v1:172.21.15.163:6789/0 conn(0x7fd2240b3810 legacy=0x7fd2240b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:14.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.729+0000 7fd228e23700 1 -- 172.21.15.163:0/83989975 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2240695d0 con 0x7fd224143050 2023-10-09T06:32:14.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.729+0000 7fd2237fe700 1 -- 172.21.15.163:0/83989975 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2805242649 0 0) 0x7fd22406c390 con 0x7fd2240b3810 2023-10-09T06:32:14.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.729+0000 7fd228e23700 1 --2- 172.21.15.163:0/83989975 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd224143050 0x7fd224159c90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd22406ca20 tx=0x7fd214009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8236e551bb394f2b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:14.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.729+0000 7fd2237fe700 1 -- 172.21.15.163:0/83989975 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21400e070 con 0x7fd224143050 2023-10-09T06:32:14.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.730+0000 7fd2237fe700 1 -- 172.21.15.163:0/83989975 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd21400bc30 con 0x7fd224143050 2023-10-09T06:32:14.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.730+0000 7fd2237fe700 1 -- 172.21.15.163:0/83989975 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd214012430 con 0x7fd224143050 2023-10-09T06:32:14.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.730+0000 7fd229e25700 1 -- 172.21.15.163:0/83989975 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd224143050 msgr2=0x7fd224159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:14.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.730+0000 7fd229e25700 1 --2- 172.21.15.163:0/83989975 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd224143050 0x7fd224159c90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd22406ca20 tx=0x7fd214009780 comp rx=0 tx=0).stop 2023-10-09T06:32:14.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.730+0000 7fd229e25700 1 -- 172.21.15.163:0/83989975 shutdown_connections 2023-10-09T06:32:14.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.730+0000 7fd229e25700 1 --2- 172.21.15.163:0/83989975 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd224143050 0x7fd224159c90 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:14.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.730+0000 7fd229e25700 1 --2- 172.21.15.163:0/83989975 >> v2:172.21.15.181:3300/0 conn(0x7fd22406b9e0 0x7fd22406be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:14.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.730+0000 7fd229e25700 1 -- 172.21.15.163:0/83989975 >> 172.21.15.163:0/83989975 conn(0x7fd2240ce430 msgr2=0x7fd2240ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:14.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.730+0000 7fd229e25700 1 -- 172.21.15.163:0/83989975 shutdown_connections 2023-10-09T06:32:14.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.730+0000 7fd229e25700 1 -- 172.21.15.163:0/83989975 wait complete. 2023-10-09T06:32:14.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.731+0000 7fd229e25700 1 Processor -- start 2023-10-09T06:32:14.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.731+0000 7fd229e25700 1 -- start start 2023-10-09T06:32:14.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.732+0000 7fd229e25700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd22406b9e0 0x7fd22415ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:14.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.732+0000 7fd229e25700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2240b3810 0x7fd22413d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:14.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.732+0000 7fd229e25700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd224143050 0x7fd22415d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:14.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.732+0000 7fd229e25700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22415e8c0 con 0x7fd224143050 2023-10-09T06:32:14.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.732+0000 7fd229e25700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22415dcb0 con 0x7fd22406b9e0 2023-10-09T06:32:14.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.732+0000 7fd229e25700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd22415df90 con 0x7fd2240b3810 2023-10-09T06:32:14.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.732+0000 7fd229624700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd224143050 0x7fd22415d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:14.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.732+0000 7fd229624700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd224143050 0x7fd22415d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:32946/0 (socket says 172.21.15.163:32946) 2023-10-09T06:32:14.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.732+0000 7fd229624700 1 -- 172.21.15.163:0/1338730118 learned_addr learned my addr 172.21.15.163:0/1338730118 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:14.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.732+0000 7fd223fff700 1 --2- 172.21.15.163:0/1338730118 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2240b3810 0x7fd22413d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:14.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.732+0000 7fd228e23700 1 --2- 172.21.15.163:0/1338730118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd22406b9e0 0x7fd22415ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:14.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.733+0000 7fd229624700 1 -- 172.21.15.163:0/1338730118 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2240b3810 msgr2=0x7fd22413d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:14.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.733+0000 7fd229624700 1 --2- 172.21.15.163:0/1338730118 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2240b3810 0x7fd22413d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:14.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.733+0000 7fd229624700 1 -- 172.21.15.163:0/1338730118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd22406b9e0 msgr2=0x7fd22415ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:14.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.733+0000 7fd229624700 1 --2- 172.21.15.163:0/1338730118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd22406b9e0 0x7fd22415ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:14.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.733+0000 7fd229624700 1 -- 172.21.15.163:0/1338730118 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd214009a20 con 0x7fd224143050 2023-10-09T06:32:14.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.733+0000 7fd229624700 1 --2- 172.21.15.163:0/1338730118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd224143050 0x7fd22415d5b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd21c0011e0 tx=0x7fd21c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:14.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.733+0000 7fd220ff9700 1 -- 172.21.15.163:0/1338730118 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21c005b30 con 0x7fd224143050 2023-10-09T06:32:14.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.733+0000 7fd220ff9700 1 -- 172.21.15.163:0/1338730118 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd21c005c90 con 0x7fd224143050 2023-10-09T06:32:14.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.733+0000 7fd220ff9700 1 -- 172.21.15.163:0/1338730118 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21c007320 con 0x7fd224143050 2023-10-09T06:32:14.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.733+0000 7fd229e25700 1 -- 172.21.15.163:0/1338730118 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd22415e270 con 0x7fd224143050 2023-10-09T06:32:14.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.733+0000 7fd229e25700 1 -- 172.21.15.163:0/1338730118 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd224139d80 con 0x7fd224143050 2023-10-09T06:32:14.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.734+0000 7fd229e25700 1 -- 172.21.15.163:0/1338730118 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd22415ea30 con 0x7fd224143050 2023-10-09T06:32:14.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.734+0000 7fd220ff9700 1 -- 172.21.15.163:0/1338730118 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd21c007690 con 0x7fd224143050 2023-10-09T06:32:14.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.735+0000 7fd220ff9700 1 --2- 172.21.15.163:0/1338730118 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd210066f30 0x7fd2100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:14.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.735+0000 7fd220ff9700 1 -- 172.21.15.163:0/1338730118 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd21c083960 con 0x7fd224143050 2023-10-09T06:32:14.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.735+0000 7fd228e23700 1 --2- 172.21.15.163:0/1338730118 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd210066f30 0x7fd2100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:14.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.735+0000 7fd228e23700 1 --2- 172.21.15.163:0/1338730118 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd210066f30 0x7fd2100693e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd2140099a0 tx=0x7fd214019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:14.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.739+0000 7fd220ff9700 1 -- 172.21.15.163:0/1338730118 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd21c050bd0 con 0x7fd224143050 2023-10-09T06:32:14.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.949+0000 7fd229e25700 1 -- 172.21.15.163:0/1338730118 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd22404c3c0 con 0x7fd210066f30 2023-10-09T06:32:14.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.951+0000 7fd220ff9700 1 -- 172.21.15.163:0/1338730118 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fd22404c3c0 con 0x7fd210066f30 2023-10-09T06:32:14.951 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:32:14.951 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:32:14.952 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "cephfs" 2023-10-09T06:32:14.952 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:32:14.952 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:32:14.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.953+0000 7fd229e25700 1 -- 172.21.15.163:0/1338730118 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd210066f30 msgr2=0x7fd2100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:14.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.953+0000 7fd229e25700 1 --2- 172.21.15.163:0/1338730118 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd210066f30 0x7fd2100693e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd2140099a0 tx=0x7fd214019000 comp rx=0 tx=0).stop 2023-10-09T06:32:14.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.953+0000 7fd229e25700 1 -- 172.21.15.163:0/1338730118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd224143050 msgr2=0x7fd22415d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:14.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.953+0000 7fd229e25700 1 --2- 172.21.15.163:0/1338730118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd224143050 0x7fd22415d5b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd21c0011e0 tx=0x7fd21c007d30 comp rx=0 tx=0).stop 2023-10-09T06:32:14.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.953+0000 7fd229e25700 1 -- 172.21.15.163:0/1338730118 shutdown_connections 2023-10-09T06:32:14.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.953+0000 7fd229e25700 1 --2- 172.21.15.163:0/1338730118 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd210066f30 0x7fd2100693e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:14.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.953+0000 7fd229e25700 1 --2- 172.21.15.163:0/1338730118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd224143050 0x7fd22415d5b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:14.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.953+0000 7fd229e25700 1 --2- 172.21.15.163:0/1338730118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd22406b9e0 0x7fd22415ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:14.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.953+0000 7fd229e25700 1 --2- 172.21.15.163:0/1338730118 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2240b3810 0x7fd22413d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:14.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.953+0000 7fd229e25700 1 -- 172.21.15.163:0/1338730118 >> 172.21.15.163:0/1338730118 conn(0x7fd2240ce430 msgr2=0x7fd22415b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:14.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.953+0000 7fd229e25700 1 -- 172.21.15.163:0/1338730118 shutdown_connections 2023-10-09T06:32:14.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:14.953+0000 7fd229e25700 1 -- 172.21.15.163:0/1338730118 wait complete. 2023-10-09T06:32:14.966 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-09T06:32:15.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.082+0000 7f64dd0dd700 1 Processor -- start 2023-10-09T06:32:15.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.082+0000 7f64dd0dd700 1 -- start start 2023-10-09T06:32:15.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.083+0000 7f64dd0dd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f64d8159890 0x7f64d815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:15.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.083+0000 7f64dd0dd700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d815a160 con 0x7f64d8069560 2023-10-09T06:32:15.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.083+0000 7f64dd0dd700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d80d13a0 con 0x7f64d8159c60 2023-10-09T06:32:15.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.083+0000 7f64dd0dd700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d815e140 con 0x7f64d8159890 2023-10-09T06:32:15.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.083+0000 7f64d759e700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f64d8069560 0x7f64d80ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:56726/0 (socket says 172.21.15.163:56726) 2023-10-09T06:32:15.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.083+0000 7f64d759e700 1 -- 172.21.15.163:0/3571076696 learned_addr learned my addr 172.21.15.163:0/3571076696 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:15.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.083+0000 7f64d6d9d700 1 --2- 172.21.15.163:0/3571076696 >> v2:172.21.15.181:3300/0 conn(0x7f64d8159890 0x7f64d815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:15.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.083+0000 7f64d659c700 1 -- 172.21.15.163:0/3571076696 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1746849492 0 0) 0x7f64d815a160 con 0x7f64d8069560 2023-10-09T06:32:15.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.083+0000 7f64d659c700 1 -- 172.21.15.163:0/3571076696 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64c4003570 con 0x7f64d8069560 2023-10-09T06:32:15.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.084+0000 7f64d659c700 1 -- 172.21.15.163:0/3571076696 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4103774230 0 0) 0x7f64c4003570 con 0x7f64d8069560 2023-10-09T06:32:15.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.084+0000 7f64d659c700 1 -- 172.21.15.163:0/3571076696 >> v2:172.21.15.181:3300/0 conn(0x7f64d8159890 msgr2=0x7f64d815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:15.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.084+0000 7f64d659c700 1 --2- 172.21.15.163:0/3571076696 >> v2:172.21.15.181:3300/0 conn(0x7f64d8159890 0x7f64d815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:15.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.084+0000 7f64d659c700 1 -- 172.21.15.163:0/3571076696 >> v1:172.21.15.181:6789/0 conn(0x7f64d8159c60 legacy=0x7f64d815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:15.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.084+0000 7f64d659c700 1 -- 172.21.15.163:0/3571076696 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d815a3d0 con 0x7f64d8069560 2023-10-09T06:32:15.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.084+0000 7f64d659c700 1 -- 172.21.15.163:0/3571076696 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f64d00017f0 con 0x7f64d8069560 2023-10-09T06:32:15.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.084+0000 7f64d659c700 1 -- 172.21.15.163:0/3571076696 >> v1:172.21.15.163:6789/0 conn(0x7f64d8069560 legacy=0x7f64d80ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:15.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.084+0000 7f64d659c700 1 --2- 172.21.15.163:0/3571076696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64c4003a60 0x7f64c4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:15.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.084+0000 7f64d659c700 1 --2- 172.21.15.163:0/3571076696 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64c4004810 0x7f64c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:15.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.084+0000 7f64d659c700 1 --2- 172.21.15.163:0/3571076696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64c40051c0 0x7f64c400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:15.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.084+0000 7f64d659c700 1 -- 172.21.15.163:0/3571076696 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d815e140 con 0x7f64c4003a60 2023-10-09T06:32:15.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.084+0000 7f64d659c700 1 -- 172.21.15.163:0/3571076696 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c400c0d0 con 0x7f64c40051c0 2023-10-09T06:32:15.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.084+0000 7f64d659c700 1 -- 172.21.15.163:0/3571076696 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c400c290 con 0x7f64c4004810 2023-10-09T06:32:15.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.084+0000 7f64d6d9d700 1 --2- 172.21.15.163:0/3571076696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64c4003a60 0x7f64c4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:15.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.084+0000 7f64d759e700 1 --2- 172.21.15.163:0/3571076696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64c40051c0 0x7f64c400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:15.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.084+0000 7f64ce59c700 1 --2- 172.21.15.163:0/3571076696 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64c4004810 0x7f64c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:15.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.084+0000 7f64d6d9d700 1 -- 172.21.15.163:0/3571076696 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64c4004810 msgr2=0x7f64c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:15.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.084+0000 7f64d6d9d700 1 --2- 172.21.15.163:0/3571076696 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64c4004810 0x7f64c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:15.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.084+0000 7f64d6d9d700 1 -- 172.21.15.163:0/3571076696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64c40051c0 msgr2=0x7f64c400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:15.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.084+0000 7f64d6d9d700 1 --2- 172.21.15.163:0/3571076696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64c40051c0 0x7f64c400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:15.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.084+0000 7f64d6d9d700 1 -- 172.21.15.163:0/3571076696 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c4003570 con 0x7f64c4003a60 2023-10-09T06:32:15.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.084+0000 7f64d759e700 1 -- 172.21.15.163:0/3571076696 reap_dead start 2023-10-09T06:32:15.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.084+0000 7f64d6d9d700 1 --2- 172.21.15.163:0/3571076696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64c4003a60 0x7f64c4003e70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009c00 tx=0x7f64c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e8524d8b483cbc server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:15.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.084+0000 7f64d659c700 1 -- 172.21.15.163:0/3571076696 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c800e710 con 0x7f64c4003a60 2023-10-09T06:32:15.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.084+0000 7f64d659c700 1 -- 172.21.15.163:0/3571076696 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c800e870 con 0x7f64c4003a60 2023-10-09T06:32:15.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.084+0000 7f64d659c700 1 -- 172.21.15.163:0/3571076696 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c800ec30 con 0x7f64c4003a60 2023-10-09T06:32:15.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.085+0000 7f64dd0dd700 1 -- 172.21.15.163:0/3571076696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64c4003a60 msgr2=0x7f64c4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:15.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.085+0000 7f64dd0dd700 1 --2- 172.21.15.163:0/3571076696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64c4003a60 0x7f64c4003e70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009c00 tx=0x7f64c80091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:15.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.085+0000 7f64dd0dd700 1 -- 172.21.15.163:0/3571076696 shutdown_connections 2023-10-09T06:32:15.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.085+0000 7f64dd0dd700 1 --2- 172.21.15.163:0/3571076696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64c4003a60 0x7f64c4003e70 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:15.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.085+0000 7f64dd0dd700 1 -- 172.21.15.163:0/3571076696 >> 172.21.15.163:0/3571076696 conn(0x7f64d8068e20 msgr2=0x7f64d80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:15.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.085+0000 7f64dd0dd700 1 -- 172.21.15.163:0/3571076696 shutdown_connections 2023-10-09T06:32:15.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.085+0000 7f64dd0dd700 1 -- 172.21.15.163:0/3571076696 wait complete. 2023-10-09T06:32:15.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.086+0000 7f64dd0dd700 1 Processor -- start 2023-10-09T06:32:15.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.086+0000 7f64dd0dd700 1 -- start start 2023-10-09T06:32:15.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.087+0000 7f64dd0dd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64c4003a60 0x7f64d80ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:15.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.087+0000 7f64dd0dd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64d80cd1a0 0x7f64d80b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:15.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.087+0000 7f64dd0dd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64d80b1c30 0x7f64d80b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:15.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.087+0000 7f64dd0dd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d80b2610 con 0x7f64d80b1c30 2023-10-09T06:32:15.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.087+0000 7f64dd0dd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d80b4d20 con 0x7f64d80cd1a0 2023-10-09T06:32:15.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.087+0000 7f64dd0dd700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d80b4110 con 0x7f64c4003a60 2023-10-09T06:32:15.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.087+0000 7f64d759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64d80b1c30 0x7f64d80b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:15.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.087+0000 7f64d759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64d80b1c30 0x7f64d80b20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:32964/0 (socket says 172.21.15.163:32964) 2023-10-09T06:32:15.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.087+0000 7f64d759e700 1 -- 172.21.15.163:0/2805693461 learned_addr learned my addr 172.21.15.163:0/2805693461 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:15.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.087+0000 7f64d6d9d700 1 --2- 172.21.15.163:0/2805693461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64c4003a60 0x7f64d80ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:15.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.087+0000 7f64ce59c700 1 --2- 172.21.15.163:0/2805693461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64d80cd1a0 0x7f64d80b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:15.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.087+0000 7f64d759e700 1 -- 172.21.15.163:0/2805693461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64c4003a60 msgr2=0x7f64d80ca4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:15.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.087+0000 7f64d759e700 1 --2- 172.21.15.163:0/2805693461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64c4003a60 0x7f64d80ca4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:15.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.087+0000 7f64d759e700 1 -- 172.21.15.163:0/2805693461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64d80cd1a0 msgr2=0x7f64d80b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:15.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.087+0000 7f64d759e700 1 --2- 172.21.15.163:0/2805693461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64d80cd1a0 0x7f64d80b37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:15.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.087+0000 7f64d759e700 1 -- 172.21.15.163:0/2805693461 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d80b43f0 con 0x7f64d80b1c30 2023-10-09T06:32:15.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.087+0000 7f64d759e700 1 --2- 172.21.15.163:0/2805693461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64d80b1c30 0x7f64d80b20d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f64d80d03b0 tx=0x7f64d00021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:15.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.088+0000 7f64cf7fe700 1 -- 172.21.15.163:0/2805693461 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d000ad60 con 0x7f64d80b1c30 2023-10-09T06:32:15.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.088+0000 7f64cf7fe700 1 -- 172.21.15.163:0/2805693461 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d00133f0 con 0x7f64d80b1c30 2023-10-09T06:32:15.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.088+0000 7f64cf7fe700 1 -- 172.21.15.163:0/2805693461 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d0013790 con 0x7f64d80b1c30 2023-10-09T06:32:15.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.088+0000 7f64dd0dd700 1 -- 172.21.15.163:0/2805693461 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d80b46d0 con 0x7f64d80b1c30 2023-10-09T06:32:15.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.088+0000 7f64dd0dd700 1 -- 172.21.15.163:0/2805693461 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d80caa20 con 0x7f64d80b1c30 2023-10-09T06:32:15.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.089+0000 7f64dd0dd700 1 -- 172.21.15.163:0/2805693461 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d804c3c0 con 0x7f64d80b1c30 2023-10-09T06:32:15.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.089+0000 7f64cf7fe700 1 -- 172.21.15.163:0/2805693461 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f64d0013b80 con 0x7f64d80b1c30 2023-10-09T06:32:15.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.089+0000 7f64cf7fe700 1 --2- 172.21.15.163:0/2805693461 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f64b8066f30 0x7f64b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:15.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.089+0000 7f64cf7fe700 1 -- 172.21.15.163:0/2805693461 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f64d0084cd0 con 0x7f64d80b1c30 2023-10-09T06:32:15.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.090+0000 7f64d6d9d700 1 --2- 172.21.15.163:0/2805693461 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f64b8066f30 0x7f64b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:15.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.090+0000 7f64d6d9d700 1 --2- 172.21.15.163:0/2805693461 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f64b8066f30 0x7f64b80693e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f64c80091e0 tx=0x7f64c801a000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:15.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.094+0000 7f64cf7fe700 1 -- 172.21.15.163:0/2805693461 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f64d0051f40 con 0x7f64d80b1c30 2023-10-09T06:32:15.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.274+0000 7f64dd0dd700 1 -- 172.21.15.163:0/2805693461 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f64d806a5b0 con 0x7f64d80b1c30 2023-10-09T06:32:15.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.275+0000 7f64cf7fe700 1 -- 172.21.15.163:0/2805693461 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f64d0055560 con 0x7f64d80b1c30 2023-10-09T06:32:15.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.277+0000 7f64dd0dd700 1 -- 172.21.15.163:0/2805693461 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f64b8066f30 msgr2=0x7f64b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:15.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.277+0000 7f64dd0dd700 1 --2- 172.21.15.163:0/2805693461 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f64b8066f30 0x7f64b80693e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f64c80091e0 tx=0x7f64c801a000 comp rx=0 tx=0).stop 2023-10-09T06:32:15.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.277+0000 7f64dd0dd700 1 -- 172.21.15.163:0/2805693461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64d80b1c30 msgr2=0x7f64d80b20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:15.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.277+0000 7f64dd0dd700 1 --2- 172.21.15.163:0/2805693461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64d80b1c30 0x7f64d80b20d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f64d80d03b0 tx=0x7f64d00021d0 comp rx=0 tx=0).stop 2023-10-09T06:32:15.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.277+0000 7f64dd0dd700 1 -- 172.21.15.163:0/2805693461 shutdown_connections 2023-10-09T06:32:15.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.277+0000 7f64dd0dd700 1 --2- 172.21.15.163:0/2805693461 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f64b8066f30 0x7f64b80693e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:15.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.277+0000 7f64dd0dd700 1 --2- 172.21.15.163:0/2805693461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64d80b1c30 0x7f64d80b20d0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:15.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.277+0000 7f64dd0dd700 1 --2- 172.21.15.163:0/2805693461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64c4003a60 0x7f64d80ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:15.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.277+0000 7f64dd0dd700 1 --2- 172.21.15.163:0/2805693461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64d80cd1a0 0x7f64d80b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:15.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.277+0000 7f64dd0dd700 1 -- 172.21.15.163:0/2805693461 >> 172.21.15.163:0/2805693461 conn(0x7f64d8068e20 msgr2=0x7f64d815d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:15.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.277+0000 7f64dd0dd700 1 -- 172.21.15.163:0/2805693461 shutdown_connections 2023-10-09T06:32:15.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.277+0000 7f64dd0dd700 1 -- 172.21.15.163:0/2805693461 wait complete. 2023-10-09T06:32:15.290 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs group_0000000000034793 2023-10-09T06:32:15.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.411+0000 7fb5d86de700 1 Processor -- start 2023-10-09T06:32:15.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.411+0000 7fb5d86de700 1 -- start start 2023-10-09T06:32:15.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.412+0000 7fb5d86de700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb5d00cb710 0x7fb5d0159d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:15.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.412+0000 7fb5d86de700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5d00cd1c0 0x7fb5d00cd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:15.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.412+0000 7fb5d86de700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d00689e0 con 0x7fb5d0159930 2023-10-09T06:32:15.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.412+0000 7fb5d86de700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5d00cdb80 con 0x7fb5d00cb710 2023-10-09T06:32:15.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.412+0000 7fb5d86de700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d00cbcc0 con 0x7fb5d00cd1c0 2023-10-09T06:32:15.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.412+0000 7fb5d5c79700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb5d00cb710 0x7fb5d0159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:15.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.412+0000 7fb5d647a700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb5d0159930 0x7fb5d00b3ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:56740/0 (socket says 172.21.15.163:56740) 2023-10-09T06:32:15.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.412+0000 7fb5d5c79700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb5d00cb710 0x7fb5d0159d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:32976/0 (socket says 172.21.15.163:32976) 2023-10-09T06:32:15.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.412+0000 7fb5d647a700 1 -- 172.21.15.163:0/449765187 learned_addr learned my addr 172.21.15.163:0/449765187 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:15.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.412+0000 7fb5d6c7b700 1 --2- 172.21.15.163:0/449765187 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5d00cd1c0 0x7fb5d00cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:15.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.413+0000 7fb5d5c79700 1 -- 172.21.15.163:0/449765187 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5d00cd1c0 msgr2=0x7fb5d00cd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:15.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.413+0000 7fb5d5c79700 1 --2- 172.21.15.163:0/449765187 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5d00cd1c0 0x7fb5d00cd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:15.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.413+0000 7fb5d5c79700 1 -- 172.21.15.163:0/449765187 >> v1:172.21.15.163:6789/0 conn(0x7fb5d0159930 legacy=0x7fb5d00b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:15.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.413+0000 7fb5d5c79700 1 -- 172.21.15.163:0/449765187 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d00cbfa0 con 0x7fb5d00cb710 2023-10-09T06:32:15.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.413+0000 7fb5d5478700 1 -- 172.21.15.163:0/449765187 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4187375869 0 0) 0x7fb5d00689e0 con 0x7fb5d0159930 2023-10-09T06:32:15.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.413+0000 7fb5d6c7b700 1 --2- 172.21.15.163:0/449765187 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5d00cd1c0 0x7fb5d00cd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:15.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.413+0000 7fb5d5c79700 1 --2- 172.21.15.163:0/449765187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5d00cb710 0x7fb5d0159d00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb5c00098e0 tx=0x7fb5c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c9d9385483d0a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:15.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.413+0000 7fb5d5478700 1 -- 172.21.15.163:0/449765187 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0009e80 con 0x7fb5d00cb710 2023-10-09T06:32:15.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.413+0000 7fb5d5478700 1 -- 172.21.15.163:0/449765187 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c000b8f0 con 0x7fb5d00cb710 2023-10-09T06:32:15.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.413+0000 7fb5d5478700 1 -- 172.21.15.163:0/449765187 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c000bc70 con 0x7fb5d00cb710 2023-10-09T06:32:15.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.413+0000 7fb5d86de700 1 -- 172.21.15.163:0/449765187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5d00cb710 msgr2=0x7fb5d0159d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:15.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.413+0000 7fb5d86de700 1 --2- 172.21.15.163:0/449765187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5d00cb710 0x7fb5d0159d00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb5c00098e0 tx=0x7fb5c00091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:15.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.413+0000 7fb5d86de700 1 -- 172.21.15.163:0/449765187 shutdown_connections 2023-10-09T06:32:15.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.413+0000 7fb5d86de700 1 --2- 172.21.15.163:0/449765187 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5d00cd1c0 0x7fb5d00cd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:15.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.413+0000 7fb5d86de700 1 --2- 172.21.15.163:0/449765187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5d00cb710 0x7fb5d0159d00 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:15.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.413+0000 7fb5d86de700 1 -- 172.21.15.163:0/449765187 >> 172.21.15.163:0/449765187 conn(0x7fb5d00999d0 msgr2=0x7fb5d00b3690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:15.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.413+0000 7fb5d86de700 1 -- 172.21.15.163:0/449765187 shutdown_connections 2023-10-09T06:32:15.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.413+0000 7fb5d86de700 1 -- 172.21.15.163:0/449765187 wait complete. 2023-10-09T06:32:15.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.415+0000 7fb5d86de700 1 Processor -- start 2023-10-09T06:32:15.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.415+0000 7fb5d86de700 1 -- start start 2023-10-09T06:32:15.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.415+0000 7fb5d86de700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5d00cb710 0x7fb5d009a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:15.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.415+0000 7fb5d86de700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5d00cd1c0 0x7fb5d0169ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:15.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.415+0000 7fb5d86de700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5d0159930 0x7fb5d016a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:15.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.415+0000 7fb5d86de700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d009a880 con 0x7fb5d00cd1c0 2023-10-09T06:32:15.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.415+0000 7fb5d86de700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d00ca890 con 0x7fb5d0159930 2023-10-09T06:32:15.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.415+0000 7fb5d86de700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d015dea0 con 0x7fb5d00cb710 2023-10-09T06:32:15.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.415+0000 7fb5d5c79700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5d00cd1c0 0x7fb5d0169ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:15.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.415+0000 7fb5d5c79700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5d00cd1c0 0x7fb5d0169ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:32990/0 (socket says 172.21.15.163:32990) 2023-10-09T06:32:15.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.415+0000 7fb5d5c79700 1 -- 172.21.15.163:0/600520292 learned_addr learned my addr 172.21.15.163:0/600520292 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:15.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.416+0000 7fb5d5c79700 1 -- 172.21.15.163:0/600520292 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5d00cb710 msgr2=0x7fb5d009a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:15.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.416+0000 7fb5d5c79700 1 --2- 172.21.15.163:0/600520292 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5d00cb710 0x7fb5d009a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:15.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.416+0000 7fb5d5c79700 1 -- 172.21.15.163:0/600520292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5d0159930 msgr2=0x7fb5d016a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:15.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.416+0000 7fb5d5c79700 1 --2- 172.21.15.163:0/600520292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5d0159930 0x7fb5d016a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:15.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.416+0000 7fb5d5c79700 1 -- 172.21.15.163:0/600520292 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c0009420 con 0x7fb5d00cd1c0 2023-10-09T06:32:15.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.416+0000 7fb5d5c79700 1 --2- 172.21.15.163:0/600520292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5d00cd1c0 0x7fb5d0169ae0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00891d0 tx=0x7fb5c0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:15.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.416+0000 7fb5c67fc700 1 -- 172.21.15.163:0/600520292 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c000bc70 con 0x7fb5d00cd1c0 2023-10-09T06:32:15.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.416+0000 7fb5c67fc700 1 -- 172.21.15.163:0/600520292 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c0011420 con 0x7fb5d00cd1c0 2023-10-09T06:32:15.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.416+0000 7fb5d86de700 1 -- 172.21.15.163:0/600520292 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d015e180 con 0x7fb5d00cd1c0 2023-10-09T06:32:15.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.416+0000 7fb5c67fc700 1 -- 172.21.15.163:0/600520292 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c00117c0 con 0x7fb5d00cd1c0 2023-10-09T06:32:15.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.416+0000 7fb5d86de700 1 -- 172.21.15.163:0/600520292 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d015e510 con 0x7fb5d00cd1c0 2023-10-09T06:32:15.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.417+0000 7fb5d86de700 1 -- 172.21.15.163:0/600520292 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d015ea80 con 0x7fb5d00cd1c0 2023-10-09T06:32:15.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.417+0000 7fb5c67fc700 1 -- 172.21.15.163:0/600520292 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb5c0011b30 con 0x7fb5d00cd1c0 2023-10-09T06:32:15.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.418+0000 7fb5c67fc700 1 --2- 172.21.15.163:0/600520292 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb5b4066f30 0x7fb5b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:15.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.418+0000 7fb5c67fc700 1 -- 172.21.15.163:0/600520292 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb5c0086030 con 0x7fb5d00cd1c0 2023-10-09T06:32:15.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.418+0000 7fb5d647a700 1 --2- 172.21.15.163:0/600520292 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb5b4066f30 0x7fb5b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:15.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.418+0000 7fb5d647a700 1 --2- 172.21.15.163:0/600520292 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb5b4066f30 0x7fb5b40693e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb5d015ece0 tx=0x7fb5cc007a40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:15.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.422+0000 7fb5c67fc700 1 -- 172.21.15.163:0/600520292 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb5c00532a0 con 0x7fb5d00cd1c0 2023-10-09T06:32:15.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.633+0000 7fb5d86de700 1 -- 172.21.15.163:0/600520292 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "group_0000000000034793", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5d004c3c0 con 0x7fb5b4066f30 2023-10-09T06:32:15.639 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:32:15.638+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:32:15.639 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:32:15.638+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:32:15.639 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:32:15.638+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:32:15.639 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:32:15.638+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:32:15.639 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:32:15.638+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:32:15.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.653+0000 7fb5c67fc700 1 -- 172.21.15.163:0/600520292 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb5d004c3c0 con 0x7fb5b4066f30 2023-10-09T06:32:15.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.655+0000 7fb5d86de700 1 -- 172.21.15.163:0/600520292 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb5b4066f30 msgr2=0x7fb5b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:15.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.655+0000 7fb5d86de700 1 --2- 172.21.15.163:0/600520292 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb5b4066f30 0x7fb5b40693e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb5d015ece0 tx=0x7fb5cc007a40 comp rx=0 tx=0).stop 2023-10-09T06:32:15.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.656+0000 7fb5d86de700 1 -- 172.21.15.163:0/600520292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5d00cd1c0 msgr2=0x7fb5d0169ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:15.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.656+0000 7fb5d86de700 1 --2- 172.21.15.163:0/600520292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5d00cd1c0 0x7fb5d0169ae0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00891d0 tx=0x7fb5c0017000 comp rx=0 tx=0).stop 2023-10-09T06:32:15.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.656+0000 7fb5d86de700 1 -- 172.21.15.163:0/600520292 shutdown_connections 2023-10-09T06:32:15.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.656+0000 7fb5d86de700 1 --2- 172.21.15.163:0/600520292 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb5b4066f30 0x7fb5b40693e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:15.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.656+0000 7fb5d86de700 1 --2- 172.21.15.163:0/600520292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5d0159930 0x7fb5d016a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:15.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.656+0000 7fb5d86de700 1 --2- 172.21.15.163:0/600520292 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5d00cb710 0x7fb5d009a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:15.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.656+0000 7fb5d86de700 1 --2- 172.21.15.163:0/600520292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5d00cd1c0 0x7fb5d0169ae0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:15.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.656+0000 7fb5d86de700 1 -- 172.21.15.163:0/600520292 >> 172.21.15.163:0/600520292 conn(0x7fb5d00999d0 msgr2=0x7fb5d0158a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:15.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.656+0000 7fb5d86de700 1 -- 172.21.15.163:0/600520292 shutdown_connections 2023-10-09T06:32:15.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.656+0000 7fb5d86de700 1 -- 172.21.15.163:0/600520292 wait complete. 2023-10-09T06:32:15.668 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume info cephfs 2023-10-09T06:32:15.786 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.785+0000 7f94052bc700 1 Processor -- start 2023-10-09T06:32:15.786 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.785+0000 7f94052bc700 1 -- start start 2023-10-09T06:32:15.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.786+0000 7f94052bc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94000b16d0 0x7f94000b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:15.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.786+0000 7f94052bc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f94000b33b0 0x7f94000b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:15.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.786+0000 7f94052bc700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94000b2080 con 0x7f94000b4e70 2023-10-09T06:32:15.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.786+0000 7f94052bc700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94000cfb00 con 0x7f94000b33b0 2023-10-09T06:32:15.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.786+0000 7f94052bc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94000b5c40 con 0x7f94000b16d0 2023-10-09T06:32:15.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.786+0000 7f9404abb700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f94000b33b0 0x7f94000b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:15.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.786+0000 7f93fffff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f94000b4e70 0x7f94000ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:56742/0 (socket says 172.21.15.163:56742) 2023-10-09T06:32:15.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.786+0000 7f93fffff700 1 -- 172.21.15.163:0/2550546287 learned_addr learned my addr 172.21.15.163:0/2550546287 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:15.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.786+0000 7f9404abb700 1 -- 172.21.15.163:0/2550546287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94000b16d0 msgr2=0x7f94000b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:15.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.786+0000 7f9404abb700 1 --2- 172.21.15.163:0/2550546287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94000b16d0 0x7f94000b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:15.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.786+0000 7f9404abb700 1 -- 172.21.15.163:0/2550546287 >> v1:172.21.15.163:6789/0 conn(0x7f94000b4e70 legacy=0x7f94000ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:15.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.786+0000 7f93feffd700 1 -- 172.21.15.163:0/2550546287 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3099496746 0 0) 0x7f94000b2080 con 0x7f94000b4e70 2023-10-09T06:32:15.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.786+0000 7f9404abb700 1 -- 172.21.15.163:0/2550546287 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94000b5f20 con 0x7f94000b33b0 2023-10-09T06:32:15.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.786+0000 7f9404abb700 1 --2- 172.21.15.163:0/2550546287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94000b33b0 0x7f94000b3830 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f93f8009e60 tx=0x7f93f8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474cbb4460033d63 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:15.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.787+0000 7f93feffd700 1 -- 172.21.15.163:0/2550546287 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f800e070 con 0x7f94000b33b0 2023-10-09T06:32:15.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.787+0000 7f93feffd700 1 -- 172.21.15.163:0/2550546287 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f800bc30 con 0x7f94000b33b0 2023-10-09T06:32:15.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.787+0000 7f93feffd700 1 -- 172.21.15.163:0/2550546287 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f8012430 con 0x7f94000b33b0 2023-10-09T06:32:15.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.787+0000 7f94052bc700 1 -- 172.21.15.163:0/2550546287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94000b33b0 msgr2=0x7f94000b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:15.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.787+0000 7f94052bc700 1 --2- 172.21.15.163:0/2550546287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94000b33b0 0x7f94000b3830 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f93f8009e60 tx=0x7f93f8009780 comp rx=0 tx=0).stop 2023-10-09T06:32:15.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.787+0000 7f94052bc700 1 -- 172.21.15.163:0/2550546287 shutdown_connections 2023-10-09T06:32:15.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.787+0000 7f94052bc700 1 --2- 172.21.15.163:0/2550546287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94000b33b0 0x7f94000b3830 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:15.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.787+0000 7f94052bc700 1 --2- 172.21.15.163:0/2550546287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94000b16d0 0x7f94000b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:15.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.787+0000 7f94052bc700 1 -- 172.21.15.163:0/2550546287 >> 172.21.15.163:0/2550546287 conn(0x7f94000d10c0 msgr2=0x7f94000affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:15.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.787+0000 7f94052bc700 1 -- 172.21.15.163:0/2550546287 shutdown_connections 2023-10-09T06:32:15.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.787+0000 7f94052bc700 1 -- 172.21.15.163:0/2550546287 wait complete. 2023-10-09T06:32:15.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.788+0000 7f94052bc700 1 Processor -- start 2023-10-09T06:32:15.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.789+0000 7f94052bc700 1 -- start start 2023-10-09T06:32:15.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.789+0000 7f94052bc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94000b16d0 0x7f9400144780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:15.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.789+0000 7f94052bc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94000b33b0 0x7f9400157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:15.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.789+0000 7f94052bc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94000b4e70 0x7f9400157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:15.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.789+0000 7f94052bc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9400145260 con 0x7f94000b4e70 2023-10-09T06:32:15.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.789+0000 7f94052bc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94001481a0 con 0x7f94000b33b0 2023-10-09T06:32:15.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.789+0000 7f94052bc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94001454b0 con 0x7f94000b16d0 2023-10-09T06:32:15.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.789+0000 7f9404abb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94000b4e70 0x7f9400157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:15.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.789+0000 7f9404abb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94000b4e70 0x7f9400157f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33008/0 (socket says 172.21.15.163:33008) 2023-10-09T06:32:15.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.789+0000 7f9404abb700 1 -- 172.21.15.163:0/3495890474 learned_addr learned my addr 172.21.15.163:0/3495890474 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:15.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.789+0000 7f93fffff700 1 --2- 172.21.15.163:0/3495890474 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94000b16d0 0x7f9400144780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:15.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.789+0000 7f93ff7fe700 1 --2- 172.21.15.163:0/3495890474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94000b33b0 0x7f9400157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:15.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.790+0000 7f9404abb700 1 -- 172.21.15.163:0/3495890474 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94000b16d0 msgr2=0x7f9400144780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:15.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.790+0000 7f9404abb700 1 --2- 172.21.15.163:0/3495890474 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94000b16d0 0x7f9400144780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:15.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.790+0000 7f9404abb700 1 -- 172.21.15.163:0/3495890474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94000b33b0 msgr2=0x7f9400157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:15.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.790+0000 7f9404abb700 1 --2- 172.21.15.163:0/3495890474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94000b33b0 0x7f9400157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:15.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.790+0000 7f9404abb700 1 -- 172.21.15.163:0/3495890474 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f8009a20 con 0x7f94000b4e70 2023-10-09T06:32:15.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.790+0000 7f93ff7fe700 1 --2- 172.21.15.163:0/3495890474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94000b33b0 0x7f9400157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:15.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.790+0000 7f9404abb700 1 --2- 172.21.15.163:0/3495890474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94000b4e70 0x7f9400157f90 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f93f800b6c0 tx=0x7f93f8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:15.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.790+0000 7f93e7fff700 1 -- 172.21.15.163:0/3495890474 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f80263f0 con 0x7f94000b4e70 2023-10-09T06:32:15.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.790+0000 7f93e7fff700 1 -- 172.21.15.163:0/3495890474 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f8026550 con 0x7f94000b4e70 2023-10-09T06:32:15.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.790+0000 7f94052bc700 1 -- 172.21.15.163:0/3495890474 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9400145790 con 0x7f94000b4e70 2023-10-09T06:32:15.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.790+0000 7f93e7fff700 1 -- 172.21.15.163:0/3495890474 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f80268f0 con 0x7f94000b4e70 2023-10-09T06:32:15.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.790+0000 7f94052bc700 1 -- 172.21.15.163:0/3495890474 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9400145d50 con 0x7f94000b4e70 2023-10-09T06:32:15.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.791+0000 7f94052bc700 1 -- 172.21.15.163:0/3495890474 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9400088990 con 0x7f94000b4e70 2023-10-09T06:32:15.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.791+0000 7f93e7fff700 1 -- 172.21.15.163:0/3495890474 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f93f8026c60 con 0x7f94000b4e70 2023-10-09T06:32:15.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.792+0000 7f93e7fff700 1 --2- 172.21.15.163:0/3495890474 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f93ec066f30 0x7f93ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:15.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.792+0000 7f93e7fff700 1 -- 172.21.15.163:0/3495890474 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f93f8086710 con 0x7f94000b4e70 2023-10-09T06:32:15.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.792+0000 7f93fffff700 1 --2- 172.21.15.163:0/3495890474 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f93ec066f30 0x7f93ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:15.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.792+0000 7f93fffff700 1 --2- 172.21.15.163:0/3495890474 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f93ec066f30 0x7f93ec0693e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f94001587a0 tx=0x7f93f0007f50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:15.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:15.796+0000 7f93e7fff700 1 -- 172.21.15.163:0/3495890474 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f93f80538e0 con 0x7f94000b4e70 2023-10-09T06:32:16.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.005+0000 7f94052bc700 1 -- 172.21.15.163:0/3495890474 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume info", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) v1 -- 0x7f940004bb80 con 0x7f93ec066f30 2023-10-09T06:32:16.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.008+0000 7f93e7fff700 1 -- 172.21.15.163:0/3495890474 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+531 (secure 0 0 0) 0x7f940004bb80 con 0x7f93ec066f30 2023-10-09T06:32:16.009 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:32:16.009 INFO:teuthology.orchestra.run.smithi163.stdout: "mon_addrs": [ 2023-10-09T06:32:16.009 INFO:teuthology.orchestra.run.smithi163.stdout: "172.21.15.163:6789", 2023-10-09T06:32:16.009 INFO:teuthology.orchestra.run.smithi163.stdout: "172.21.15.181:6789", 2023-10-09T06:32:16.010 INFO:teuthology.orchestra.run.smithi163.stdout: "172.21.15.181:6790" 2023-10-09T06:32:16.010 INFO:teuthology.orchestra.run.smithi163.stdout: ], 2023-10-09T06:32:16.010 INFO:teuthology.orchestra.run.smithi163.stdout: "pending_subvolume_deletions": 0, 2023-10-09T06:32:16.010 INFO:teuthology.orchestra.run.smithi163.stdout: "pools": { 2023-10-09T06:32:16.010 INFO:teuthology.orchestra.run.smithi163.stdout: "data": [ 2023-10-09T06:32:16.011 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:32:16.011 INFO:teuthology.orchestra.run.smithi163.stdout: "avail": 365997129728, 2023-10-09T06:32:16.011 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "cephfs_data", 2023-10-09T06:32:16.011 INFO:teuthology.orchestra.run.smithi163.stdout: "used": 0 2023-10-09T06:32:16.012 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:32:16.012 INFO:teuthology.orchestra.run.smithi163.stdout: ], 2023-10-09T06:32:16.012 INFO:teuthology.orchestra.run.smithi163.stdout: "metadata": [ 2023-10-09T06:32:16.012 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:32:16.012 INFO:teuthology.orchestra.run.smithi163.stdout: "avail": 365997129728, 2023-10-09T06:32:16.013 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "cephfs_metadata", 2023-10-09T06:32:16.013 INFO:teuthology.orchestra.run.smithi163.stdout: "used": 65536 2023-10-09T06:32:16.013 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:32:16.013 INFO:teuthology.orchestra.run.smithi163.stdout: ] 2023-10-09T06:32:16.013 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:32:16.014 INFO:teuthology.orchestra.run.smithi163.stdout: "used_size": 0 2023-10-09T06:32:16.014 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:32:16.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.010+0000 7f94052bc700 1 -- 172.21.15.163:0/3495890474 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f93ec066f30 msgr2=0x7f93ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:16.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.010+0000 7f94052bc700 1 --2- 172.21.15.163:0/3495890474 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f93ec066f30 0x7f93ec0693e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f94001587a0 tx=0x7f93f0007f50 comp rx=0 tx=0).stop 2023-10-09T06:32:16.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.010+0000 7f94052bc700 1 -- 172.21.15.163:0/3495890474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94000b4e70 msgr2=0x7f9400157f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:16.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.010+0000 7f94052bc700 1 --2- 172.21.15.163:0/3495890474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94000b4e70 0x7f9400157f90 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f93f800b6c0 tx=0x7f93f8019000 comp rx=0 tx=0).stop 2023-10-09T06:32:16.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.010+0000 7f94052bc700 1 -- 172.21.15.163:0/3495890474 shutdown_connections 2023-10-09T06:32:16.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.010+0000 7f94052bc700 1 --2- 172.21.15.163:0/3495890474 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f93ec066f30 0x7f93ec0693e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:16.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.010+0000 7f94052bc700 1 --2- 172.21.15.163:0/3495890474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94000b4e70 0x7f9400157f90 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:16.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.010+0000 7f94052bc700 1 --2- 172.21.15.163:0/3495890474 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94000b16d0 0x7f9400144780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:16.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.010+0000 7f94052bc700 1 --2- 172.21.15.163:0/3495890474 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94000b33b0 0x7f9400157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:16.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.010+0000 7f94052bc700 1 -- 172.21.15.163:0/3495890474 >> 172.21.15.163:0/3495890474 conn(0x7f94000d10c0 msgr2=0x7f9400159160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:16.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.010+0000 7f94052bc700 1 -- 172.21.15.163:0/3495890474 shutdown_connections 2023-10-09T06:32:16.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.010+0000 7f94052bc700 1 -- 172.21.15.163:0/3495890474 wait complete. 2023-10-09T06:32:16.024 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-09T06:32:16.024 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:32:16.061 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:32:16.105 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-09T06:32:16.105 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-09T06:32:16.106 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:16.106 DEBUG:teuthology.orchestra.run.smithi163:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-09T06:32:16.147 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi163.front.sepia.ceph.com 2023-10-09T06:32:16.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:16.147 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-09T06:32:16.203 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-09T06:32:16.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:16.203 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-09T06:32:16.246 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-09T06:32:16.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:16.247 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-09T06:32:16.304 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:32:16.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.417+0000 7f57e3fff700 1 Processor -- start 2023-10-09T06:32:16.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.417+0000 7f57e3fff700 1 -- start start 2023-10-09T06:32:16.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.418+0000 7f57e3fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f57e406c0f0 0x7f57e41642e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:16.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.418+0000 7f57e3fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e40b40f0 con 0x7f57e4142ea0 2023-10-09T06:32:16.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.418+0000 7f57e3fff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e4069640 con 0x7f57e40ad870 2023-10-09T06:32:16.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.418+0000 7f57e3fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e4069920 con 0x7f57e406c0f0 2023-10-09T06:32:16.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.418+0000 7f57e37fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f57e406c0f0 0x7f57e41642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:16.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.418+0000 7f57e2ffd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f57e4142ea0 0x7f57e40b3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:56748/0 (socket says 172.21.15.163:56748) 2023-10-09T06:32:16.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.418+0000 7f57e37fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f57e406c0f0 0x7f57e41642e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33022/0 (socket says 172.21.15.163:33022) 2023-10-09T06:32:16.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.418+0000 7f57e2ffd700 1 -- 172.21.15.163:0/143581699 learned_addr learned my addr 172.21.15.163:0/143581699 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:16.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.419+0000 7f57e37fe700 1 -- 172.21.15.163:0/143581699 >> v1:172.21.15.181:6789/0 conn(0x7f57e40ad870 legacy=0x7f57e406b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:16.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.419+0000 7f57e37fe700 1 -- 172.21.15.163:0/143581699 >> v1:172.21.15.163:6789/0 conn(0x7f57e4142ea0 legacy=0x7f57e40b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:16.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.419+0000 7f57e37fe700 1 -- 172.21.15.163:0/143581699 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e4069c00 con 0x7f57e406c0f0 2023-10-09T06:32:16.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.419+0000 7f57e37fe700 1 --2- 172.21.15.163:0/143581699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57e406c0f0 0x7f57e41642e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f57dc009e60 tx=0x7f57dc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5fc08f9138c20e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:16.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.419+0000 7f57e27fc700 1 -- 172.21.15.163:0/143581699 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc00e070 con 0x7f57e406c0f0 2023-10-09T06:32:16.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.419+0000 7f57e27fc700 1 -- 172.21.15.163:0/143581699 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57dc00bc30 con 0x7f57e406c0f0 2023-10-09T06:32:16.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.419+0000 7f57e27fc700 1 -- 172.21.15.163:0/143581699 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc012430 con 0x7f57e406c0f0 2023-10-09T06:32:16.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.419+0000 7f57e3fff700 1 -- 172.21.15.163:0/143581699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57e406c0f0 msgr2=0x7f57e41642e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:16.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.419+0000 7f57e3fff700 1 --2- 172.21.15.163:0/143581699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57e406c0f0 0x7f57e41642e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f57dc009e60 tx=0x7f57dc009780 comp rx=0 tx=0).stop 2023-10-09T06:32:16.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.419+0000 7f57e3fff700 1 -- 172.21.15.163:0/143581699 shutdown_connections 2023-10-09T06:32:16.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.419+0000 7f57e3fff700 1 --2- 172.21.15.163:0/143581699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57e406c0f0 0x7f57e41642e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:16.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.419+0000 7f57e3fff700 1 -- 172.21.15.163:0/143581699 >> 172.21.15.163:0/143581699 conn(0x7f57e40ce430 msgr2=0x7f57e40ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:16.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.419+0000 7f57e3fff700 1 -- 172.21.15.163:0/143581699 shutdown_connections 2023-10-09T06:32:16.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.419+0000 7f57e3fff700 1 -- 172.21.15.163:0/143581699 wait complete. 2023-10-09T06:32:16.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.421+0000 7f57e3fff700 1 Processor -- start 2023-10-09T06:32:16.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.421+0000 7f57e3fff700 1 -- start start 2023-10-09T06:32:16.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.421+0000 7f57e3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57e406c0f0 0x7f57e40ace80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:16.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.421+0000 7f57e3fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57e4142ea0 0x7f57e40c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:16.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.421+0000 7f57e3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57e40c7fd0 0x7f57e40c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:16.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.421+0000 7f57e3fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e40c89b0 con 0x7f57e4142ea0 2023-10-09T06:32:16.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.421+0000 7f57e3fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e415bda0 con 0x7f57e406c0f0 2023-10-09T06:32:16.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.421+0000 7f57e3fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e416a010 con 0x7f57e40c7fd0 2023-10-09T06:32:16.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.421+0000 7f57da7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57e4142ea0 0x7f57e40c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:16.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.421+0000 7f57da7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57e4142ea0 0x7f57e40c75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33036/0 (socket says 172.21.15.163:33036) 2023-10-09T06:32:16.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.421+0000 7f57da7fc700 1 -- 172.21.15.163:0/860224722 learned_addr learned my addr 172.21.15.163:0/860224722 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:16.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.422+0000 7f57e2ffd700 1 --2- 172.21.15.163:0/860224722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57e406c0f0 0x7f57e40ace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:16.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.422+0000 7f57e37fe700 1 --2- 172.21.15.163:0/860224722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57e40c7fd0 0x7f57e40c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:16.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.422+0000 7f57da7fc700 1 -- 172.21.15.163:0/860224722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57e40c7fd0 msgr2=0x7f57e40c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:16.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.422+0000 7f57da7fc700 1 --2- 172.21.15.163:0/860224722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57e40c7fd0 0x7f57e40c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:16.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.422+0000 7f57da7fc700 1 -- 172.21.15.163:0/860224722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57e406c0f0 msgr2=0x7f57e40ace80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:16.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.422+0000 7f57da7fc700 1 --2- 172.21.15.163:0/860224722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57e406c0f0 0x7f57e40ace80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:16.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.422+0000 7f57da7fc700 1 -- 172.21.15.163:0/860224722 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57dc009a20 con 0x7f57e4142ea0 2023-10-09T06:32:16.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.422+0000 7f57e37fe700 1 --2- 172.21.15.163:0/860224722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57e40c7fd0 0x7f57e40c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:16.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.422+0000 7f57da7fc700 1 --2- 172.21.15.163:0/860224722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57e4142ea0 0x7f57e40c75e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00c590 tx=0x7f57cc007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:16.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.422+0000 7f57db7fe700 1 -- 172.21.15.163:0/860224722 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc011590 con 0x7f57e4142ea0 2023-10-09T06:32:16.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.422+0000 7f57db7fe700 1 -- 172.21.15.163:0/860224722 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57cc011710 con 0x7f57e4142ea0 2023-10-09T06:32:16.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.422+0000 7f57e3fff700 1 -- 172.21.15.163:0/860224722 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e416a2f0 con 0x7f57e4142ea0 2023-10-09T06:32:16.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.422+0000 7f57db7fe700 1 -- 172.21.15.163:0/860224722 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc011ad0 con 0x7f57e4142ea0 2023-10-09T06:32:16.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.422+0000 7f57e3fff700 1 -- 172.21.15.163:0/860224722 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e416a8b0 con 0x7f57e4142ea0 2023-10-09T06:32:16.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.423+0000 7f57e3fff700 1 -- 172.21.15.163:0/860224722 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e415d730 con 0x7f57e4142ea0 2023-10-09T06:32:16.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.423+0000 7f57db7fe700 1 -- 172.21.15.163:0/860224722 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f57cc01c440 con 0x7f57e4142ea0 2023-10-09T06:32:16.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.424+0000 7f57db7fe700 1 --2- 172.21.15.163:0/860224722 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f57c4066f30 0x7f57c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:16.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.424+0000 7f57db7fe700 1 -- 172.21.15.163:0/860224722 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f57cc083960 con 0x7f57e4142ea0 2023-10-09T06:32:16.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.424+0000 7f57e2ffd700 1 --2- 172.21.15.163:0/860224722 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f57c4066f30 0x7f57c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:16.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.425+0000 7f57e2ffd700 1 --2- 172.21.15.163:0/860224722 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f57c4066f30 0x7f57c40693e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f57e406b410 tx=0x7f57d4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:16.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.428+0000 7f57db7fe700 1 -- 172.21.15.163:0/860224722 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f57e415d730 con 0x7f57e4142ea0 2023-10-09T06:32:16.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.660+0000 7f57e3fff700 1 -- 172.21.15.163:0/860224722 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57e415d730 con 0x7f57e4142ea0 2023-10-09T06:32:16.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.660+0000 7f57db7fe700 1 -- 172.21.15.163:0/860224722 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f57e415d730 con 0x7f57e4142ea0 2023-10-09T06:32:16.661 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:16.661 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/819128421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":819128421},{"type":"v1","addr":"172.21.15.181:6836","nonce":819128421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3339496202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3339496202},{"type":"v1","addr":"172.21.15.163:6834","nonce":3339496202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5352,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1726174482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1726174482},{"type":"v1","addr":"172.21.15.163:6835","nonce":1726174482}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:32:00.924137+0000","modified":"2023-10-09T06:32:04.563062+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5370},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5370":{"gid":5370,"name":"b","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2945196635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2945196635},{"type":"v1","addr":"172.21.15.181:6837","nonce":2945196635}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-09T06:32:16.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.662+0000 7f57e3fff700 1 -- 172.21.15.163:0/860224722 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f57c4066f30 msgr2=0x7f57c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:16.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.662+0000 7f57e3fff700 1 --2- 172.21.15.163:0/860224722 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f57c4066f30 0x7f57c40693e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f57e406b410 tx=0x7f57d4009040 comp rx=0 tx=0).stop 2023-10-09T06:32:16.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.662+0000 7f57e3fff700 1 -- 172.21.15.163:0/860224722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57e4142ea0 msgr2=0x7f57e40c75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:16.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.662+0000 7f57e3fff700 1 --2- 172.21.15.163:0/860224722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57e4142ea0 0x7f57e40c75e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00c590 tx=0x7f57cc007f10 comp rx=0 tx=0).stop 2023-10-09T06:32:16.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.662+0000 7f57e3fff700 1 -- 172.21.15.163:0/860224722 shutdown_connections 2023-10-09T06:32:16.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.662+0000 7f57e3fff700 1 --2- 172.21.15.163:0/860224722 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f57c4066f30 0x7f57c40693e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:16.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.662+0000 7f57e3fff700 1 --2- 172.21.15.163:0/860224722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57e40c7fd0 0x7f57e40c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:16.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.662+0000 7f57e3fff700 1 --2- 172.21.15.163:0/860224722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57e406c0f0 0x7f57e40ace80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:16.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.662+0000 7f57e3fff700 1 --2- 172.21.15.163:0/860224722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57e4142ea0 0x7f57e40c75e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:16.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.662+0000 7f57e3fff700 1 -- 172.21.15.163:0/860224722 >> 172.21.15.163:0/860224722 conn(0x7f57e40ce430 msgr2=0x7f57e4141fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:16.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.662+0000 7f57e3fff700 1 -- 172.21.15.163:0/860224722 shutdown_connections 2023-10-09T06:32:16.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.662+0000 7f57e3fff700 1 -- 172.21.15.163:0/860224722 wait complete. 2023-10-09T06:32:16.665 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 32 2023-10-09T06:32:16.675 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:32:16.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.788+0000 7f5097293700 1 Processor -- start 2023-10-09T06:32:16.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.789+0000 7f5097293700 1 -- start start 2023-10-09T06:32:16.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.789+0000 7f5097293700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50900ccf90 0x7f50900ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:16.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.790+0000 7f5097293700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50900cc300 con 0x7f50900b1420 2023-10-09T06:32:16.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.790+0000 7f5097293700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50900ad370 con 0x7f50900cbf30 2023-10-09T06:32:16.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.790+0000 7f5097293700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5090156bf0 con 0x7f50900ccf90 2023-10-09T06:32:16.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.790+0000 7f509482e700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f50900b1420 0x7f50900c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:56764/0 (socket says 172.21.15.163:56764) 2023-10-09T06:32:16.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.790+0000 7f509482e700 1 -- 172.21.15.163:0/51999983 learned_addr learned my addr 172.21.15.163:0/51999983 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:16.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.790+0000 7f5095830700 1 --2- 172.21.15.163:0/51999983 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50900ccf90 0x7f50900ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:16.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.790+0000 7f5087fff700 1 -- 172.21.15.163:0/51999983 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3599915150 0 0) 0x7f50900cc300 con 0x7f50900b1420 2023-10-09T06:32:16.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.790+0000 7f5087fff700 1 -- 172.21.15.163:0/51999983 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f507c003570 con 0x7f50900b1420 2023-10-09T06:32:16.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.790+0000 7f5087fff700 1 -- 172.21.15.163:0/51999983 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 372685586 0 0) 0x7f50900ad370 con 0x7f50900cbf30 2023-10-09T06:32:16.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.790+0000 7f5087fff700 1 -- 172.21.15.163:0/51999983 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50900cc300 con 0x7f50900cbf30 2023-10-09T06:32:16.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.790+0000 7f5087fff700 1 -- 172.21.15.163:0/51999983 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3298378512 0 0) 0x7f507c003570 con 0x7f50900b1420 2023-10-09T06:32:16.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.790+0000 7f5095830700 1 -- 172.21.15.163:0/51999983 >> v1:172.21.15.181:6789/0 conn(0x7f50900cbf30 legacy=0x7f50900b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:16.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.790+0000 7f5095830700 1 -- 172.21.15.163:0/51999983 >> v1:172.21.15.163:6789/0 conn(0x7f50900b1420 legacy=0x7f50900c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:16.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.790+0000 7f5095830700 1 -- 172.21.15.163:0/51999983 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50900ad650 con 0x7f50900ccf90 2023-10-09T06:32:16.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.790+0000 7f5095830700 1 --2- 172.21.15.163:0/51999983 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50900ccf90 0x7f50900ae410 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5088009e90 tx=0x7f5088009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2450eb1167ae4e76 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:16.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.791+0000 7f5087fff700 1 -- 172.21.15.163:0/51999983 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508800e070 con 0x7f50900ccf90 2023-10-09T06:32:16.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.791+0000 7f5087fff700 1 -- 172.21.15.163:0/51999983 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508800b870 con 0x7f50900ccf90 2023-10-09T06:32:16.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.791+0000 7f5087fff700 1 -- 172.21.15.163:0/51999983 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5080001470 con 0x7f50900ccf90 2023-10-09T06:32:16.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.791+0000 7f5097293700 1 -- 172.21.15.163:0/51999983 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50900ccf90 msgr2=0x7f50900ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:16.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.791+0000 7f5097293700 1 --2- 172.21.15.163:0/51999983 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50900ccf90 0x7f50900ae410 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5088009e90 tx=0x7f5088009780 comp rx=0 tx=0).stop 2023-10-09T06:32:16.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.791+0000 7f5097293700 1 -- 172.21.15.163:0/51999983 shutdown_connections 2023-10-09T06:32:16.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.791+0000 7f5097293700 1 --2- 172.21.15.163:0/51999983 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50900ccf90 0x7f50900ae410 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:16.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.791+0000 7f5097293700 1 -- 172.21.15.163:0/51999983 >> 172.21.15.163:0/51999983 conn(0x7f509006a320 msgr2=0x7f50900cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:16.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.791+0000 7f5097293700 1 -- 172.21.15.163:0/51999983 shutdown_connections 2023-10-09T06:32:16.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.791+0000 7f5097293700 1 -- 172.21.15.163:0/51999983 wait complete. 2023-10-09T06:32:16.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.792+0000 7f5097293700 1 Processor -- start 2023-10-09T06:32:16.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.792+0000 7f5097293700 1 -- start start 2023-10-09T06:32:16.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.793+0000 7f5097293700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50900b1420 0x7f509016a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:16.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.793+0000 7f5097293700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50900cbf30 0x7f509014ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:16.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.793+0000 7f5097293700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50900ccf90 0x7f50900a7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:16.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.793+0000 7f5097293700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509016a900 con 0x7f50900ccf90 2023-10-09T06:32:16.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.793+0000 7f5097293700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50900c1640 con 0x7f50900b1420 2023-10-09T06:32:16.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.793+0000 7f5097293700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50900c1920 con 0x7f50900cbf30 2023-10-09T06:32:16.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.793+0000 7f5095830700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50900ccf90 0x7f50900a7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:16.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.793+0000 7f5095830700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50900ccf90 0x7f50900a7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33046/0 (socket says 172.21.15.163:33046) 2023-10-09T06:32:16.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.793+0000 7f5095830700 1 -- 172.21.15.163:0/757627999 learned_addr learned my addr 172.21.15.163:0/757627999 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:16.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.793+0000 7f509482e700 1 --2- 172.21.15.163:0/757627999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50900cbf30 0x7f509014ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:16.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.794+0000 7f509502f700 1 --2- 172.21.15.163:0/757627999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50900b1420 0x7f509016a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:16.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.794+0000 7f5095830700 1 -- 172.21.15.163:0/757627999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50900cbf30 msgr2=0x7f509014ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:16.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.794+0000 7f5095830700 1 --2- 172.21.15.163:0/757627999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50900cbf30 0x7f509014ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:16.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.794+0000 7f5095830700 1 -- 172.21.15.163:0/757627999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50900b1420 msgr2=0x7f509016a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:16.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.794+0000 7f5095830700 1 --2- 172.21.15.163:0/757627999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50900b1420 0x7f509016a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:16.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.794+0000 7f5095830700 1 -- 172.21.15.163:0/757627999 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5088009a20 con 0x7f50900ccf90 2023-10-09T06:32:16.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.794+0000 7f509482e700 1 --2- 172.21.15.163:0/757627999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50900cbf30 0x7f509014ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:32:16.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.794+0000 7f5095830700 1 --2- 172.21.15.163:0/757627999 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50900ccf90 0x7f50900a7500 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5088009de0 tx=0x7f5088017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:16.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.794+0000 7f50857fa700 1 -- 172.21.15.163:0/757627999 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5088012cb0 con 0x7f50900ccf90 2023-10-09T06:32:16.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.794+0000 7f50857fa700 1 -- 172.21.15.163:0/757627999 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5088012e10 con 0x7f50900ccf90 2023-10-09T06:32:16.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.794+0000 7f5097293700 1 -- 172.21.15.163:0/757627999 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50900c1c00 con 0x7f50900ccf90 2023-10-09T06:32:16.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.794+0000 7f50857fa700 1 -- 172.21.15.163:0/757627999 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508801e680 con 0x7f50900ccf90 2023-10-09T06:32:16.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.794+0000 7f5097293700 1 -- 172.21.15.163:0/757627999 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50900c21c0 con 0x7f50900ccf90 2023-10-09T06:32:16.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.795+0000 7f5097293700 1 -- 172.21.15.163:0/757627999 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50900c23b0 con 0x7f50900ccf90 2023-10-09T06:32:16.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.795+0000 7f50857fa700 1 -- 172.21.15.163:0/757627999 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f508801eab0 con 0x7f50900ccf90 2023-10-09T06:32:16.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.796+0000 7f50857fa700 1 --2- 172.21.15.163:0/757627999 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5074066f30 0x7f50740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:16.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.796+0000 7f50857fa700 1 -- 172.21.15.163:0/757627999 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5088086460 con 0x7f50900ccf90 2023-10-09T06:32:16.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.796+0000 7f509502f700 1 --2- 172.21.15.163:0/757627999 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5074066f30 0x7f50740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:16.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.796+0000 7f509502f700 1 --2- 172.21.15.163:0/757627999 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5074066f30 0x7f50740693e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f508c0027d0 tx=0x7f508c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:16.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.800+0000 7f50857fa700 1 -- 172.21.15.163:0/757627999 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f50880536d0 con 0x7f50900ccf90 2023-10-09T06:32:16.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.983+0000 7f5097293700 1 -- 172.21.15.163:0/757627999 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f509004c3c0 con 0x7f50900ccf90 2023-10-09T06:32:16.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.984+0000 7f50857fa700 1 -- 172.21.15.163:0/757627999 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f5088056cf0 con 0x7f50900ccf90 2023-10-09T06:32:16.985 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:16.985 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":51,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:32:02.534579+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:31:59.239255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-09T06:31:59.882874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:32:16.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.986+0000 7f5097293700 1 -- 172.21.15.163:0/757627999 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5074066f30 msgr2=0x7f50740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:16.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.986+0000 7f5097293700 1 --2- 172.21.15.163:0/757627999 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5074066f30 0x7f50740693e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f508c0027d0 tx=0x7f508c009040 comp rx=0 tx=0).stop 2023-10-09T06:32:16.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.986+0000 7f5097293700 1 -- 172.21.15.163:0/757627999 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50900ccf90 msgr2=0x7f50900a7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:16.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.986+0000 7f5097293700 1 --2- 172.21.15.163:0/757627999 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50900ccf90 0x7f50900a7500 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5088009de0 tx=0x7f5088017000 comp rx=0 tx=0).stop 2023-10-09T06:32:16.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.986+0000 7f5097293700 1 -- 172.21.15.163:0/757627999 shutdown_connections 2023-10-09T06:32:16.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.986+0000 7f5097293700 1 --2- 172.21.15.163:0/757627999 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5074066f30 0x7f50740693e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:16.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.986+0000 7f5097293700 1 --2- 172.21.15.163:0/757627999 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50900ccf90 0x7f50900a7500 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:16.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.986+0000 7f5097293700 1 --2- 172.21.15.163:0/757627999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50900b1420 0x7f509016a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:16.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.986+0000 7f5097293700 1 --2- 172.21.15.163:0/757627999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50900cbf30 0x7f509014ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:16.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.986+0000 7f5097293700 1 -- 172.21.15.163:0/757627999 >> 172.21.15.163:0/757627999 conn(0x7f509006a320 msgr2=0x7f50900cc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:16.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.986+0000 7f5097293700 1 -- 172.21.15.163:0/757627999 shutdown_connections 2023-10-09T06:32:16.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:16.986+0000 7f5097293700 1 -- 172.21.15.163:0/757627999 wait complete. 2023-10-09T06:32:16.999 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-09T06:31:59.239255+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-09T06:31:59.882874+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-09T06:32:16.999 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:32:17.116 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.115+0000 7fb7148fa700 1 Processor -- start 2023-10-09T06:32:17.116 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.115+0000 7fb7148fa700 1 -- start start 2023-10-09T06:32:17.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.116+0000 7fb7148fa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb710159c60 0x7fb71015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:17.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.116+0000 7fb7148fa700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb71015a160 con 0x7fb71015db10 2023-10-09T06:32:17.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.116+0000 7fb7148fa700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7100d13a0 con 0x7fb710159890 2023-10-09T06:32:17.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.116+0000 7fb7148fa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb71015e140 con 0x7fb710159c60 2023-10-09T06:32:17.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.116+0000 7fb70ed9d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb71015db10 0x7fb710069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:56774/0 (socket says 172.21.15.163:56774) 2023-10-09T06:32:17.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.116+0000 7fb70ed9d700 1 -- 172.21.15.163:0/570793461 learned_addr learned my addr 172.21.15.163:0/570793461 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:17.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.116+0000 7fb705d9b700 1 --2- 172.21.15.163:0/570793461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb710159c60 0x7fb71015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:17.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.116+0000 7fb70dd9b700 1 -- 172.21.15.163:0/570793461 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3376383585 0 0) 0x7fb71015a160 con 0x7fb71015db10 2023-10-09T06:32:17.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.116+0000 7fb70dd9b700 1 -- 172.21.15.163:0/570793461 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6fc003570 con 0x7fb71015db10 2023-10-09T06:32:17.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.117+0000 7fb70dd9b700 1 -- 172.21.15.163:0/570793461 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1197231834 0 0) 0x7fb7100d13a0 con 0x7fb710159890 2023-10-09T06:32:17.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.117+0000 7fb70dd9b700 1 -- 172.21.15.163:0/570793461 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb71015a160 con 0x7fb710159890 2023-10-09T06:32:17.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.117+0000 7fb70dd9b700 1 -- 172.21.15.163:0/570793461 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3359380269 0 0) 0x7fb6fc003570 con 0x7fb71015db10 2023-10-09T06:32:17.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.117+0000 7fb705d9b700 1 -- 172.21.15.163:0/570793461 >> v1:172.21.15.181:6789/0 conn(0x7fb710159890 legacy=0x7fb71015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:17.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.117+0000 7fb705d9b700 1 -- 172.21.15.163:0/570793461 >> v1:172.21.15.163:6789/0 conn(0x7fb71015db10 legacy=0x7fb710069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:17.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.117+0000 7fb705d9b700 1 -- 172.21.15.163:0/570793461 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb71015a3d0 con 0x7fb710159c60 2023-10-09T06:32:17.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.117+0000 7fb705d9b700 1 --2- 172.21.15.163:0/570793461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb710159c60 0x7fb71015d5d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8009910 tx=0x7fb6f80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e50414ce49d72a98 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:17.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.117+0000 7fb70dd9b700 1 -- 172.21.15.163:0/570793461 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7080017f0 con 0x7fb710159c60 2023-10-09T06:32:17.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.117+0000 7fb70dd9b700 1 -- 172.21.15.163:0/570793461 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f8011420 con 0x7fb710159c60 2023-10-09T06:32:17.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.117+0000 7fb70dd9b700 1 -- 172.21.15.163:0/570793461 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f80117c0 con 0x7fb710159c60 2023-10-09T06:32:17.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.117+0000 7fb7148fa700 1 -- 172.21.15.163:0/570793461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb710159c60 msgr2=0x7fb71015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:17.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.118+0000 7fb7148fa700 1 --2- 172.21.15.163:0/570793461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb710159c60 0x7fb71015d5d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8009910 tx=0x7fb6f80091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:17.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.118+0000 7fb7148fa700 1 -- 172.21.15.163:0/570793461 shutdown_connections 2023-10-09T06:32:17.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.118+0000 7fb7148fa700 1 --2- 172.21.15.163:0/570793461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb710159c60 0x7fb71015d5d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:17.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.118+0000 7fb7148fa700 1 -- 172.21.15.163:0/570793461 >> 172.21.15.163:0/570793461 conn(0x7fb710068e20 msgr2=0x7fb7100d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:17.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.118+0000 7fb7148fa700 1 -- 172.21.15.163:0/570793461 shutdown_connections 2023-10-09T06:32:17.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.118+0000 7fb7148fa700 1 -- 172.21.15.163:0/570793461 wait complete. 2023-10-09T06:32:17.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.119+0000 7fb7148fa700 1 Processor -- start 2023-10-09T06:32:17.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.119+0000 7fb7148fa700 1 -- start start 2023-10-09T06:32:17.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.120+0000 7fb7148fa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb710159890 0x7fb7101512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:17.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.120+0000 7fb7148fa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb710159c60 0x7fb710151f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:17.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.120+0000 7fb7148fa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb71015db10 0x7fb71014d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:17.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.120+0000 7fb7148fa700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7100c7590 con 0x7fb710159c60 2023-10-09T06:32:17.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.120+0000 7fb7148fa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7100c6950 con 0x7fb71015db10 2023-10-09T06:32:17.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.120+0000 7fb7148fa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7100c6bf0 con 0x7fb710159890 2023-10-09T06:32:17.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.120+0000 7fb705d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb710159c60 0x7fb710151f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:17.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.120+0000 7fb705d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb710159c60 0x7fb710151f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33048/0 (socket says 172.21.15.163:33048) 2023-10-09T06:32:17.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.120+0000 7fb705d9b700 1 -- 172.21.15.163:0/3547339393 learned_addr learned my addr 172.21.15.163:0/3547339393 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:17.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.120+0000 7fb70e59c700 1 --2- 172.21.15.163:0/3547339393 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb710159890 0x7fb7101512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:17.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.120+0000 7fb70ed9d700 1 --2- 172.21.15.163:0/3547339393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb71015db10 0x7fb71014d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:17.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.120+0000 7fb705d9b700 1 -- 172.21.15.163:0/3547339393 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb710159890 msgr2=0x7fb7101512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:17.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.120+0000 7fb705d9b700 1 --2- 172.21.15.163:0/3547339393 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb710159890 0x7fb7101512b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:17.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.120+0000 7fb705d9b700 1 -- 172.21.15.163:0/3547339393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb71015db10 msgr2=0x7fb71014d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:17.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.120+0000 7fb705d9b700 1 --2- 172.21.15.163:0/3547339393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb71015db10 0x7fb71014d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:17.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.120+0000 7fb705d9b700 1 -- 172.21.15.163:0/3547339393 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f8009420 con 0x7fb710159c60 2023-10-09T06:32:17.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.120+0000 7fb705d9b700 1 --2- 172.21.15.163:0/3547339393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb710159c60 0x7fb710151f10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb6f80091e0 tx=0x7fb6f8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:17.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.120+0000 7fb706ffd700 1 -- 172.21.15.163:0/3547339393 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f801cb30 con 0x7fb710159c60 2023-10-09T06:32:17.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.120+0000 7fb706ffd700 1 -- 172.21.15.163:0/3547339393 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f801cc90 con 0x7fb710159c60 2023-10-09T06:32:17.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.120+0000 7fb7148fa700 1 -- 172.21.15.163:0/3547339393 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7100c6ed0 con 0x7fb710159c60 2023-10-09T06:32:17.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.120+0000 7fb706ffd700 1 -- 172.21.15.163:0/3547339393 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f801a6a0 con 0x7fb710159c60 2023-10-09T06:32:17.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.120+0000 7fb7148fa700 1 -- 172.21.15.163:0/3547339393 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7101505e0 con 0x7fb710159c60 2023-10-09T06:32:17.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.121+0000 7fb7148fa700 1 -- 172.21.15.163:0/3547339393 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb71004c3c0 con 0x7fb710159c60 2023-10-09T06:32:17.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.122+0000 7fb706ffd700 1 -- 172.21.15.163:0/3547339393 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb6f801aa90 con 0x7fb710159c60 2023-10-09T06:32:17.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.122+0000 7fb706ffd700 1 --2- 172.21.15.163:0/3547339393 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb6f0066f60 0x7fb6f0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:17.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.122+0000 7fb706ffd700 1 -- 172.21.15.163:0/3547339393 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb6f8086220 con 0x7fb710159c60 2023-10-09T06:32:17.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.122+0000 7fb70e59c700 1 --2- 172.21.15.163:0/3547339393 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb6f0066f60 0x7fb6f0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:17.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.123+0000 7fb70e59c700 1 --2- 172.21.15.163:0/3547339393 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb6f0066f60 0x7fb6f0069410 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb700007fd0 tx=0x7fb700007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:17.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.126+0000 7fb706ffd700 1 -- 172.21.15.163:0/3547339393 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6f8053490 con 0x7fb710159c60 2023-10-09T06:32:17.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.307+0000 7fb7148fa700 1 -- 172.21.15.163:0/3547339393 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb71006a5b0 con 0x7fb710159c60 2023-10-09T06:32:17.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.308+0000 7fb706ffd700 1 -- 172.21.15.163:0/3547339393 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb6f8056ab0 con 0x7fb710159c60 2023-10-09T06:32:17.309 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:32:17.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.310+0000 7fb7148fa700 1 -- 172.21.15.163:0/3547339393 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb6f0066f60 msgr2=0x7fb6f0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:17.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.310+0000 7fb7148fa700 1 --2- 172.21.15.163:0/3547339393 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb6f0066f60 0x7fb6f0069410 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb700007fd0 tx=0x7fb700007d50 comp rx=0 tx=0).stop 2023-10-09T06:32:17.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.310+0000 7fb7148fa700 1 -- 172.21.15.163:0/3547339393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb710159c60 msgr2=0x7fb710151f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:17.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.310+0000 7fb7148fa700 1 --2- 172.21.15.163:0/3547339393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb710159c60 0x7fb710151f10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb6f80091e0 tx=0x7fb6f8016000 comp rx=0 tx=0).stop 2023-10-09T06:32:17.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.310+0000 7fb7148fa700 1 -- 172.21.15.163:0/3547339393 shutdown_connections 2023-10-09T06:32:17.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.310+0000 7fb7148fa700 1 --2- 172.21.15.163:0/3547339393 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb6f0066f60 0x7fb6f0069410 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:17.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.310+0000 7fb7148fa700 1 --2- 172.21.15.163:0/3547339393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb71015db10 0x7fb71014d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:17.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.310+0000 7fb7148fa700 1 --2- 172.21.15.163:0/3547339393 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb710159890 0x7fb7101512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:17.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.310+0000 7fb7148fa700 1 --2- 172.21.15.163:0/3547339393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb710159c60 0x7fb710151f10 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:17.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.310+0000 7fb7148fa700 1 -- 172.21.15.163:0/3547339393 >> 172.21.15.163:0/3547339393 conn(0x7fb710068e20 msgr2=0x7fb710154100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:17.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.310+0000 7fb7148fa700 1 -- 172.21.15.163:0/3547339393 shutdown_connections 2023-10-09T06:32:17.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.310+0000 7fb7148fa700 1 -- 172.21.15.163:0/3547339393 wait complete. 2023-10-09T06:32:17.323 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-09T06:32:17.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.439+0000 7f696d4d9700 1 Processor -- start 2023-10-09T06:32:17.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.439+0000 7f696d4d9700 1 -- start start 2023-10-09T06:32:17.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.440+0000 7f696d4d9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6968157460 0x7f69680b0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:17.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.440+0000 7f696d4d9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69680c91c0 0x7f69680cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:17.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.440+0000 7f696d4d9700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f696815c110 con 0x7f69680caa60 2023-10-09T06:32:17.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.440+0000 7f696d4d9700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f696806c5a0 con 0x7f6968157460 2023-10-09T06:32:17.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.440+0000 7f696d4d9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69680b0b10 con 0x7f69680c91c0 2023-10-09T06:32:17.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.440+0000 7f6966ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6968157460 0x7f69680b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:17.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.440+0000 7f69667fc700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f69680caa60 0x7f69680c8ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:56776/0 (socket says 172.21.15.163:56776) 2023-10-09T06:32:17.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.440+0000 7f69667fc700 1 -- 172.21.15.163:0/3796635100 learned_addr learned my addr 172.21.15.163:0/3796635100 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:17.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.440+0000 7f69677fe700 1 --2- 172.21.15.163:0/3796635100 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69680c91c0 0x7f69680cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:17.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.440+0000 7f6966ffd700 1 -- 172.21.15.163:0/3796635100 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69680c91c0 msgr2=0x7f69680cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:17.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.440+0000 7f6966ffd700 1 --2- 172.21.15.163:0/3796635100 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69680c91c0 0x7f69680cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:17.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.440+0000 7f6966ffd700 1 -- 172.21.15.163:0/3796635100 >> v1:172.21.15.163:6789/0 conn(0x7f69680caa60 legacy=0x7f69680c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:17.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.440+0000 7f6966ffd700 1 -- 172.21.15.163:0/3796635100 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f696815bc80 con 0x7f6968157460 2023-10-09T06:32:17.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.440+0000 7f6965ffb700 1 -- 172.21.15.163:0/3796635100 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4131408321 0 0) 0x7f6950001110 con 0x7f69680caa60 2023-10-09T06:32:17.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.440+0000 7f6966ffd700 1 --2- 172.21.15.163:0/3796635100 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6968157460 0x7f69680b0210 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6960009e60 tx=0x7f6960009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a8484a57a36936 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:17.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.440+0000 7f6965ffb700 1 -- 172.21.15.163:0/3796635100 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696000e070 con 0x7f6968157460 2023-10-09T06:32:17.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.440+0000 7f6965ffb700 1 -- 172.21.15.163:0/3796635100 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f696000bc30 con 0x7f6968157460 2023-10-09T06:32:17.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.440+0000 7f6965ffb700 1 -- 172.21.15.163:0/3796635100 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6960012430 con 0x7f6968157460 2023-10-09T06:32:17.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.441+0000 7f696d4d9700 1 -- 172.21.15.163:0/3796635100 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6968157460 msgr2=0x7f69680b0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:17.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.441+0000 7f696d4d9700 1 --2- 172.21.15.163:0/3796635100 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6968157460 0x7f69680b0210 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6960009e60 tx=0x7f6960009780 comp rx=0 tx=0).stop 2023-10-09T06:32:17.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.441+0000 7f696d4d9700 1 -- 172.21.15.163:0/3796635100 shutdown_connections 2023-10-09T06:32:17.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.441+0000 7f696d4d9700 1 --2- 172.21.15.163:0/3796635100 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69680c91c0 0x7f69680cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:17.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.441+0000 7f696d4d9700 1 --2- 172.21.15.163:0/3796635100 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6968157460 0x7f69680b0210 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:17.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.441+0000 7f696d4d9700 1 -- 172.21.15.163:0/3796635100 >> 172.21.15.163:0/3796635100 conn(0x7f69680ce470 msgr2=0x7f69680ce870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:17.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.441+0000 7f696d4d9700 1 -- 172.21.15.163:0/3796635100 shutdown_connections 2023-10-09T06:32:17.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.441+0000 7f696d4d9700 1 -- 172.21.15.163:0/3796635100 wait complete. 2023-10-09T06:32:17.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.442+0000 7f696d4d9700 1 Processor -- start 2023-10-09T06:32:17.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.442+0000 7f696d4d9700 1 -- start start 2023-10-09T06:32:17.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.443+0000 7f696d4d9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69680c91c0 0x7f6968144e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:17.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.443+0000 7f696d4d9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69680caa60 0x7f6968151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:17.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.443+0000 7f696d4d9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6968157460 0x7f6968151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:17.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.443+0000 7f696d4d9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69680c8080 con 0x7f6968157460 2023-10-09T06:32:17.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.443+0000 7f696d4d9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69680c7470 con 0x7f69680caa60 2023-10-09T06:32:17.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.443+0000 7f696d4d9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69680c7750 con 0x7f69680c91c0 2023-10-09T06:32:17.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.443+0000 7f69677fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6968157460 0x7f6968151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:17.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.443+0000 7f69677fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6968157460 0x7f6968151f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33054/0 (socket says 172.21.15.163:33054) 2023-10-09T06:32:17.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.443+0000 7f69677fe700 1 -- 172.21.15.163:0/1462036194 learned_addr learned my addr 172.21.15.163:0/1462036194 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:17.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.443+0000 7f6966ffd700 1 --2- 172.21.15.163:0/1462036194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69680c91c0 0x7f6968144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:17.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.443+0000 7f69667fc700 1 --2- 172.21.15.163:0/1462036194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69680caa60 0x7f6968151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:17.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.443+0000 7f69677fe700 1 -- 172.21.15.163:0/1462036194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69680c91c0 msgr2=0x7f6968144e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:17.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.443+0000 7f69677fe700 1 --2- 172.21.15.163:0/1462036194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69680c91c0 0x7f6968144e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:17.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.443+0000 7f69677fe700 1 -- 172.21.15.163:0/1462036194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69680caa60 msgr2=0x7f6968151a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:17.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.443+0000 7f69677fe700 1 --2- 172.21.15.163:0/1462036194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69680caa60 0x7f6968151a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:17.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.443+0000 7f69677fe700 1 -- 172.21.15.163:0/1462036194 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6960009a20 con 0x7f6968157460 2023-10-09T06:32:17.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.443+0000 7f6966ffd700 1 --2- 172.21.15.163:0/1462036194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69680c91c0 0x7f6968144e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:17.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.444+0000 7f69677fe700 1 --2- 172.21.15.163:0/1462036194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6968157460 0x7f6968151f60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f695800d860 tx=0x7f6958008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:17.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.444+0000 7f695effd700 1 -- 172.21.15.163:0/1462036194 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69580125f0 con 0x7f6968157460 2023-10-09T06:32:17.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.444+0000 7f695effd700 1 -- 172.21.15.163:0/1462036194 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6958012770 con 0x7f6968157460 2023-10-09T06:32:17.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.444+0000 7f696d4d9700 1 -- 172.21.15.163:0/1462036194 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69680c7a30 con 0x7f6968157460 2023-10-09T06:32:17.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.444+0000 7f695effd700 1 -- 172.21.15.163:0/1462036194 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6958012b30 con 0x7f6968157460 2023-10-09T06:32:17.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.444+0000 7f696d4d9700 1 -- 172.21.15.163:0/1462036194 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69681453b0 con 0x7f6968157460 2023-10-09T06:32:17.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.445+0000 7f696d4d9700 1 -- 172.21.15.163:0/1462036194 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69680c81f0 con 0x7f6968157460 2023-10-09T06:32:17.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.445+0000 7f695effd700 1 -- 172.21.15.163:0/1462036194 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f695801e460 con 0x7f6968157460 2023-10-09T06:32:17.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.445+0000 7f695effd700 1 --2- 172.21.15.163:0/1462036194 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6948066f30 0x7f69480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:17.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.446+0000 7f695effd700 1 -- 172.21.15.163:0/1462036194 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6958084c40 con 0x7f6968157460 2023-10-09T06:32:17.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.446+0000 7f6966ffd700 1 --2- 172.21.15.163:0/1462036194 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6948066f30 0x7f69480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:17.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.446+0000 7f6966ffd700 1 --2- 172.21.15.163:0/1462036194 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6948066f30 0x7f69480693e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6960009e60 tx=0x7f6960019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:17.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.450+0000 7f695effd700 1 -- 172.21.15.163:0/1462036194 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6958051e10 con 0x7f6968157460 2023-10-09T06:32:17.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.607+0000 7f695effd700 1 -- 172.21.15.163:0/1462036194 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f69580521e0 con 0x7f6968157460 2023-10-09T06:32:17.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.636+0000 7f696d4d9700 1 -- 172.21.15.163:0/1462036194 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f696815b350 con 0x7f6968157460 2023-10-09T06:32:17.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.637+0000 7f695effd700 1 -- 172.21.15.163:0/1462036194 <== mon.0 v2:172.21.15.163:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) v1 ==== 104+0+11 (secure 0 0 0) 0x7f69580558a0 con 0x7f6968157460 2023-10-09T06:32:17.638 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 32 2023-10-09T06:32:17.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.639+0000 7f696d4d9700 1 -- 172.21.15.163:0/1462036194 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6948066f30 msgr2=0x7f69480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:17.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.639+0000 7f696d4d9700 1 --2- 172.21.15.163:0/1462036194 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6948066f30 0x7f69480693e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6960009e60 tx=0x7f6960019000 comp rx=0 tx=0).stop 2023-10-09T06:32:17.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.639+0000 7f696d4d9700 1 -- 172.21.15.163:0/1462036194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6968157460 msgr2=0x7f6968151f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:17.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.639+0000 7f696d4d9700 1 --2- 172.21.15.163:0/1462036194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6968157460 0x7f6968151f60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f695800d860 tx=0x7f6958008040 comp rx=0 tx=0).stop 2023-10-09T06:32:17.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.639+0000 7f696d4d9700 1 -- 172.21.15.163:0/1462036194 shutdown_connections 2023-10-09T06:32:17.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.639+0000 7f696d4d9700 1 --2- 172.21.15.163:0/1462036194 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6948066f30 0x7f69480693e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:17.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.639+0000 7f696d4d9700 1 --2- 172.21.15.163:0/1462036194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6968157460 0x7f6968151f60 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:17.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.639+0000 7f696d4d9700 1 --2- 172.21.15.163:0/1462036194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69680c91c0 0x7f6968144e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:17.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.639+0000 7f696d4d9700 1 --2- 172.21.15.163:0/1462036194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69680caa60 0x7f6968151a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:17.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.639+0000 7f696d4d9700 1 -- 172.21.15.163:0/1462036194 >> 172.21.15.163:0/1462036194 conn(0x7f69680ce470 msgr2=0x7f6968151030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:17.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.639+0000 7f696d4d9700 1 -- 172.21.15.163:0/1462036194 shutdown_connections 2023-10-09T06:32:17.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.639+0000 7f696d4d9700 1 -- 172.21.15.163:0/1462036194 wait complete. 2023-10-09T06:32:17.652 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-09T06:32:17.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.768+0000 7f02c8293700 1 Processor -- start 2023-10-09T06:32:17.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.768+0000 7f02c8293700 1 -- start start 2023-10-09T06:32:17.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.769+0000 7f02c8293700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02c00cd1e0 0x7f02c00c9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:17.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.769+0000 7f02c8293700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02c015c2a0 con 0x7f02c006bcd0 2023-10-09T06:32:17.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.769+0000 7f02c8293700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02c015b690 con 0x7f02c0141500 2023-10-09T06:32:17.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.769+0000 7f02c8293700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c015b970 con 0x7f02c00cd1e0 2023-10-09T06:32:17.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.769+0000 7f02c602f700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f02c006bcd0 0x7f02c0140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:56784/0 (socket says 172.21.15.163:56784) 2023-10-09T06:32:17.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.769+0000 7f02c602f700 1 -- 172.21.15.163:0/1571705728 learned_addr learned my addr 172.21.15.163:0/1571705728 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:17.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.770+0000 7f02c6830700 1 --2- 172.21.15.163:0/1571705728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02c00cd1e0 0x7f02c00c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:17.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.770+0000 7f02c502d700 1 -- 172.21.15.163:0/1571705728 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 384149176 0 0) 0x7f02c015c2a0 con 0x7f02c006bcd0 2023-10-09T06:32:17.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.770+0000 7f02c502d700 1 -- 172.21.15.163:0/1571705728 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02ac003570 con 0x7f02c006bcd0 2023-10-09T06:32:17.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.770+0000 7f02c502d700 1 -- 172.21.15.163:0/1571705728 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2449242936 0 0) 0x7f02c015b690 con 0x7f02c0141500 2023-10-09T06:32:17.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.770+0000 7f02c502d700 1 -- 172.21.15.163:0/1571705728 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02c015c2a0 con 0x7f02c0141500 2023-10-09T06:32:17.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.770+0000 7f02c502d700 1 -- 172.21.15.163:0/1571705728 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2088734270 0 0) 0x7f02ac003570 con 0x7f02c006bcd0 2023-10-09T06:32:17.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.770+0000 7f02c502d700 1 -- 172.21.15.163:0/1571705728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02c00cd1e0 msgr2=0x7f02c00c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:17.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.770+0000 7f02c502d700 1 --2- 172.21.15.163:0/1571705728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02c00cd1e0 0x7f02c00c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:17.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.770+0000 7f02c502d700 1 -- 172.21.15.163:0/1571705728 >> v1:172.21.15.181:6789/0 conn(0x7f02c0141500 legacy=0x7f02c00c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:17.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.770+0000 7f02c502d700 1 -- 172.21.15.163:0/1571705728 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c015bc50 con 0x7f02c006bcd0 2023-10-09T06:32:17.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.770+0000 7f02c502d700 1 -- 172.21.15.163:0/1571705728 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f02bc001470 con 0x7f02c006bcd0 2023-10-09T06:32:17.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.770+0000 7f02c502d700 1 -- 172.21.15.163:0/1571705728 >> v1:172.21.15.163:6789/0 conn(0x7f02c006bcd0 legacy=0x7f02c0140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:17.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.770+0000 7f02c502d700 1 --2- 172.21.15.163:0/1571705728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02ac003c20 0x7f02ac004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:17.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.770+0000 7f02c502d700 1 --2- 172.21.15.163:0/1571705728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02c00cd1e0 0x7f02ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:17.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.770+0000 7f02c502d700 1 --2- 172.21.15.163:0/1571705728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02ac004d50 0x7f02ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:17.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.770+0000 7f02c502d700 1 -- 172.21.15.163:0/1571705728 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c015b970 con 0x7f02ac004d50 2023-10-09T06:32:17.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.770+0000 7f02c502d700 1 -- 172.21.15.163:0/1571705728 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac00bb40 con 0x7f02ac003c20 2023-10-09T06:32:17.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.770+0000 7f02c502d700 1 -- 172.21.15.163:0/1571705728 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac00bd00 con 0x7f02c00cd1e0 2023-10-09T06:32:17.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.770+0000 7f02c6830700 1 --2- 172.21.15.163:0/1571705728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02ac004d50 0x7f02ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:17.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.770+0000 7f02c602f700 1 --2- 172.21.15.163:0/1571705728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02ac003c20 0x7f02ac004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:17.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.770+0000 7f02c582e700 1 --2- 172.21.15.163:0/1571705728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02c00cd1e0 0x7f02ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:17.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.770+0000 7f02c6830700 1 -- 172.21.15.163:0/1571705728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02c00cd1e0 msgr2=0x7f02ac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:17.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.770+0000 7f02c6830700 1 --2- 172.21.15.163:0/1571705728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02c00cd1e0 0x7f02ac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:17.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.770+0000 7f02c6830700 1 -- 172.21.15.163:0/1571705728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02ac003c20 msgr2=0x7f02ac004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:17.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.770+0000 7f02c6830700 1 --2- 172.21.15.163:0/1571705728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02ac003c20 0x7f02ac004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:17.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.770+0000 7f02c6830700 1 -- 172.21.15.163:0/1571705728 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ac003570 con 0x7f02ac004d50 2023-10-09T06:32:17.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.771+0000 7f02c6830700 1 --2- 172.21.15.163:0/1571705728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02ac004d50 0x7f02ac0051c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f02b800e210 tx=0x7f02b800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ad2fde31902bfe server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:17.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.771+0000 7f02c502d700 1 -- 172.21.15.163:0/1571705728 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b800eac0 con 0x7f02ac004d50 2023-10-09T06:32:17.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.771+0000 7f02c502d700 1 -- 172.21.15.163:0/1571705728 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b800f040 con 0x7f02ac004d50 2023-10-09T06:32:17.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.771+0000 7f02c502d700 1 -- 172.21.15.163:0/1571705728 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b800ee70 con 0x7f02ac004d50 2023-10-09T06:32:17.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.771+0000 7f02c8293700 1 -- 172.21.15.163:0/1571705728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02ac004d50 msgr2=0x7f02ac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:17.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.771+0000 7f02c8293700 1 --2- 172.21.15.163:0/1571705728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02ac004d50 0x7f02ac0051c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f02b800e210 tx=0x7f02b800b040 comp rx=0 tx=0).stop 2023-10-09T06:32:17.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.771+0000 7f02c6830700 1 -- 172.21.15.163:0/1571705728 reap_dead start 2023-10-09T06:32:17.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.771+0000 7f02c8293700 1 -- 172.21.15.163:0/1571705728 shutdown_connections 2023-10-09T06:32:17.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.771+0000 7f02c8293700 1 -- 172.21.15.163:0/1571705728 >> 172.21.15.163:0/1571705728 conn(0x7f02c00ce600 msgr2=0x7f02c00cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:17.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.771+0000 7f02c8293700 1 -- 172.21.15.163:0/1571705728 shutdown_connections 2023-10-09T06:32:17.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.771+0000 7f02c8293700 1 -- 172.21.15.163:0/1571705728 wait complete. 2023-10-09T06:32:17.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.772+0000 7f02c8293700 1 Processor -- start 2023-10-09T06:32:17.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.772+0000 7f02c8293700 1 -- start start 2023-10-09T06:32:17.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.773+0000 7f02c8293700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02ac004d50 0x7f02c013ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:17.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.773+0000 7f02c8293700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02c00b3520 0x7f02c00b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:17.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.773+0000 7f02c8293700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c015a130 0x7f02c015a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:17.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.773+0000 7f02c8293700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c006b270 con 0x7f02c00b3520 2023-10-09T06:32:17.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.773+0000 7f02c8293700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c00b3ed0 con 0x7f02c015a130 2023-10-09T06:32:17.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.773+0000 7f02c8293700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c015aae0 con 0x7f02ac004d50 2023-10-09T06:32:17.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.773+0000 7f02c582e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02c00b3520 0x7f02c00b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:17.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.773+0000 7f02c582e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02c00b3520 0x7f02c00b3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33084/0 (socket says 172.21.15.163:33084) 2023-10-09T06:32:17.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.773+0000 7f02c582e700 1 -- 172.21.15.163:0/431153216 learned_addr learned my addr 172.21.15.163:0/431153216 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:17.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.773+0000 7f02c602f700 1 --2- 172.21.15.163:0/431153216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02ac004d50 0x7f02c013ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:17.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.773+0000 7f02c6830700 1 --2- 172.21.15.163:0/431153216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c015a130 0x7f02c015a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:17.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.773+0000 7f02c582e700 1 -- 172.21.15.163:0/431153216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02ac004d50 msgr2=0x7f02c013ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:17.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.773+0000 7f02c582e700 1 --2- 172.21.15.163:0/431153216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02ac004d50 0x7f02c013ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:17.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.773+0000 7f02c582e700 1 -- 172.21.15.163:0/431153216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c015a130 msgr2=0x7f02c015a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:17.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.773+0000 7f02c582e700 1 --2- 172.21.15.163:0/431153216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c015a130 0x7f02c015a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:17.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.773+0000 7f02c582e700 1 -- 172.21.15.163:0/431153216 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c016acb0 con 0x7f02c00b3520 2023-10-09T06:32:17.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.773+0000 7f02c602f700 1 --2- 172.21.15.163:0/431153216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02ac004d50 0x7f02c013ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:17.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.773+0000 7f02c6830700 1 --2- 172.21.15.163:0/431153216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c015a130 0x7f02c015a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:17.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.774+0000 7f02c582e700 1 --2- 172.21.15.163:0/431153216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02c00b3520 0x7f02c00b3990 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f02b0002bf0 tx=0x7f02b0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:17.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.774+0000 7f02b67fc700 1 -- 172.21.15.163:0/431153216 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b000fab0 con 0x7f02c00b3520 2023-10-09T06:32:17.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.774+0000 7f02b67fc700 1 -- 172.21.15.163:0/431153216 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b000fc10 con 0x7f02c00b3520 2023-10-09T06:32:17.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.774+0000 7f02c8293700 1 -- 172.21.15.163:0/431153216 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c016b140 con 0x7f02c00b3520 2023-10-09T06:32:17.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.774+0000 7f02b67fc700 1 -- 172.21.15.163:0/431153216 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b00133f0 con 0x7f02c00b3520 2023-10-09T06:32:17.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.774+0000 7f02c8293700 1 -- 172.21.15.163:0/431153216 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c016a660 con 0x7f02c00b3520 2023-10-09T06:32:17.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.775+0000 7f02c8293700 1 -- 172.21.15.163:0/431153216 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02c016ae40 con 0x7f02c00b3520 2023-10-09T06:32:17.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.775+0000 7f02b67fc700 1 -- 172.21.15.163:0/431153216 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f02b0013760 con 0x7f02c00b3520 2023-10-09T06:32:17.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.775+0000 7f02b67fc700 1 --2- 172.21.15.163:0/431153216 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f02a4066fc0 0x7f02a4069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:17.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.775+0000 7f02b67fc700 1 -- 172.21.15.163:0/431153216 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f02b00847a0 con 0x7f02c00b3520 2023-10-09T06:32:17.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.776+0000 7f02c602f700 1 --2- 172.21.15.163:0/431153216 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f02a4066fc0 0x7f02a4069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:17.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.776+0000 7f02c602f700 1 --2- 172.21.15.163:0/431153216 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f02a4066fc0 0x7f02a4069470 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0015a0 tx=0x7f02bc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:17.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.780+0000 7f02b67fc700 1 -- 172.21.15.163:0/431153216 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02b0051a20 con 0x7f02c00b3520 2023-10-09T06:32:17.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.961+0000 7f02c8293700 1 -- 172.21.15.163:0/431153216 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f02c004c3c0 con 0x7f02c00b3520 2023-10-09T06:32:17.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.961+0000 7f02b67fc700 1 -- 172.21.15.163:0/431153216 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) v1 ==== 100+0+11 (secure 0 0 0) 0x7f02b0017020 con 0x7f02c00b3520 2023-10-09T06:32:17.962 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 64 2023-10-09T06:32:17.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.963+0000 7f02c8293700 1 -- 172.21.15.163:0/431153216 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f02a4066fc0 msgr2=0x7f02a4069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:17.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.963+0000 7f02c8293700 1 --2- 172.21.15.163:0/431153216 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f02a4066fc0 0x7f02a4069470 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0015a0 tx=0x7f02bc009040 comp rx=0 tx=0).stop 2023-10-09T06:32:17.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.963+0000 7f02c8293700 1 -- 172.21.15.163:0/431153216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02c00b3520 msgr2=0x7f02c00b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:17.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.963+0000 7f02c8293700 1 --2- 172.21.15.163:0/431153216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02c00b3520 0x7f02c00b3990 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f02b0002bf0 tx=0x7f02b0007d30 comp rx=0 tx=0).stop 2023-10-09T06:32:17.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.963+0000 7f02c8293700 1 -- 172.21.15.163:0/431153216 shutdown_connections 2023-10-09T06:32:17.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.963+0000 7f02c8293700 1 --2- 172.21.15.163:0/431153216 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f02a4066fc0 0x7f02a4069470 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:17.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.963+0000 7f02c8293700 1 --2- 172.21.15.163:0/431153216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c015a130 0x7f02c015a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:17.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.963+0000 7f02c8293700 1 --2- 172.21.15.163:0/431153216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02ac004d50 0x7f02c013ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:17.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.963+0000 7f02c8293700 1 --2- 172.21.15.163:0/431153216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02c00b3520 0x7f02c00b3990 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:17.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.963+0000 7f02c8293700 1 -- 172.21.15.163:0/431153216 >> 172.21.15.163:0/431153216 conn(0x7f02c00ce600 msgr2=0x7f02c00c4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:17.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.963+0000 7f02c8293700 1 -- 172.21.15.163:0/431153216 shutdown_connections 2023-10-09T06:32:17.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:17.963+0000 7f02c8293700 1 -- 172.21.15.163:0/431153216 wait complete. 2023-10-09T06:32:17.976 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:32:18.090 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.089+0000 7f56012a2700 1 Processor -- start 2023-10-09T06:32:18.090 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.089+0000 7f56012a2700 1 -- start start 2023-10-09T06:32:18.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.090+0000 7f56012a2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55fc0b1850 0x7f55fc0b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:18.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.090+0000 7f56012a2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f55fc1579f0 0x7f55fc0abf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:18.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.090+0000 7f56012a2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55fc0cb300 con 0x7f55fc0ae680 2023-10-09T06:32:18.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.090+0000 7f56012a2700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55fc06fc30 con 0x7f55fc1579f0 2023-10-09T06:32:18.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.090+0000 7f56012a2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc0cc260 con 0x7f55fc0b1850 2023-10-09T06:32:18.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.090+0000 7f55fb7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f55fc1579f0 0x7f55fc0abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:18.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.090+0000 7f55fb7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f55fc1579f0 0x7f55fc0abf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33100/0 (socket says 172.21.15.163:33100) 2023-10-09T06:32:18.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.090+0000 7f55fb7fe700 1 -- 172.21.15.163:0/1413250226 learned_addr learned my addr 172.21.15.163:0/1413250226 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:18.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.091+0000 7f55faffd700 1 --2- 172.21.15.163:0/1413250226 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55fc0b1850 0x7f55fc0b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:18.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.091+0000 7f55fb7fe700 1 -- 172.21.15.163:0/1413250226 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55fc0b1850 msgr2=0x7f55fc0b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:18.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.091+0000 7f55fb7fe700 1 --2- 172.21.15.163:0/1413250226 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55fc0b1850 0x7f55fc0b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:18.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.091+0000 7f55fb7fe700 1 -- 172.21.15.163:0/1413250226 >> v1:172.21.15.181:6789/0 conn(0x7f55fc0ae680 legacy=0x7f55fc1572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:18.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.091+0000 7f55fb7fe700 1 -- 172.21.15.163:0/1413250226 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55fc0cb5e0 con 0x7f55fc1579f0 2023-10-09T06:32:18.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.091+0000 7f55faffd700 1 --2- 172.21.15.163:0/1413250226 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55fc0b1850 0x7f55fc0b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:32:18.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.091+0000 7f55fb7fe700 1 --2- 172.21.15.163:0/1413250226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55fc1579f0 0x7f55fc0abf60 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f55f4009f20 tx=0x7f55f4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d071b86c725d284 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:18.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.091+0000 7f55fa7fc700 1 -- 172.21.15.163:0/1413250226 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f400e070 con 0x7f55fc1579f0 2023-10-09T06:32:18.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.091+0000 7f55fa7fc700 1 -- 172.21.15.163:0/1413250226 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f400ba40 con 0x7f55fc1579f0 2023-10-09T06:32:18.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.091+0000 7f55fa7fc700 1 -- 172.21.15.163:0/1413250226 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f40092e0 con 0x7f55fc1579f0 2023-10-09T06:32:18.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.091+0000 7f56012a2700 1 -- 172.21.15.163:0/1413250226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55fc1579f0 msgr2=0x7f55fc0abf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:18.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.091+0000 7f56012a2700 1 --2- 172.21.15.163:0/1413250226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55fc1579f0 0x7f55fc0abf60 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f55f4009f20 tx=0x7f55f4009840 comp rx=0 tx=0).stop 2023-10-09T06:32:18.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.091+0000 7f56012a2700 1 -- 172.21.15.163:0/1413250226 shutdown_connections 2023-10-09T06:32:18.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.091+0000 7f56012a2700 1 --2- 172.21.15.163:0/1413250226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55fc1579f0 0x7f55fc0abf60 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:18.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.091+0000 7f56012a2700 1 --2- 172.21.15.163:0/1413250226 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55fc0b1850 0x7f55fc0b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:18.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.091+0000 7f56012a2700 1 -- 172.21.15.163:0/1413250226 >> 172.21.15.163:0/1413250226 conn(0x7f55fc06a320 msgr2=0x7f55fc0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:18.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.091+0000 7f56012a2700 1 -- 172.21.15.163:0/1413250226 shutdown_connections 2023-10-09T06:32:18.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.092+0000 7f56012a2700 1 -- 172.21.15.163:0/1413250226 wait complete. 2023-10-09T06:32:18.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.093+0000 7f56012a2700 1 Processor -- start 2023-10-09T06:32:18.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.093+0000 7f56012a2700 1 -- start start 2023-10-09T06:32:18.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.093+0000 7f56012a2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55fc0ae680 0x7f55fc1513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:18.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.093+0000 7f56012a2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55fc0b1850 0x7f55fc0aad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:18.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.093+0000 7f56012a2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55fc1579f0 0x7f55fc0a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:18.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.093+0000 7f56012a2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc15c3e0 con 0x7f55fc0ae680 2023-10-09T06:32:18.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.093+0000 7f56012a2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc15c6c0 con 0x7f55fc1579f0 2023-10-09T06:32:18.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.093+0000 7f56012a2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc15c9a0 con 0x7f55fc0b1850 2023-10-09T06:32:18.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.094+0000 7f55faffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55fc0ae680 0x7f55fc1513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:18.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.094+0000 7f55faffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55fc0ae680 0x7f55fc1513d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33102/0 (socket says 172.21.15.163:33102) 2023-10-09T06:32:18.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.094+0000 7f55faffd700 1 -- 172.21.15.163:0/53372909 learned_addr learned my addr 172.21.15.163:0/53372909 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:18.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.094+0000 7f55fb7fe700 1 --2- 172.21.15.163:0/53372909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55fc1579f0 0x7f55fc0a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:18.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.094+0000 7f55f27fc700 1 --2- 172.21.15.163:0/53372909 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55fc0b1850 0x7f55fc0aad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:18.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.094+0000 7f55faffd700 1 -- 172.21.15.163:0/53372909 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55fc0b1850 msgr2=0x7f55fc0aad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:18.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.094+0000 7f55faffd700 1 --2- 172.21.15.163:0/53372909 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55fc0b1850 0x7f55fc0aad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:18.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.094+0000 7f55faffd700 1 -- 172.21.15.163:0/53372909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55fc1579f0 msgr2=0x7f55fc0a6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:18.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.094+0000 7f55faffd700 1 --2- 172.21.15.163:0/53372909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55fc1579f0 0x7f55fc0a6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:18.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.094+0000 7f55faffd700 1 -- 172.21.15.163:0/53372909 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f4009ae0 con 0x7f55fc0ae680 2023-10-09T06:32:18.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.094+0000 7f55f27fc700 1 --2- 172.21.15.163:0/53372909 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55fc0b1850 0x7f55fc0aad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:18.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.094+0000 7f55faffd700 1 --2- 172.21.15.163:0/53372909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55fc0ae680 0x7f55fc1513d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00c830 tx=0x7f55ec008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:18.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.094+0000 7f55f37fe700 1 -- 172.21.15.163:0/53372909 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec011600 con 0x7f55fc0ae680 2023-10-09T06:32:18.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.094+0000 7f55f37fe700 1 -- 172.21.15.163:0/53372909 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ec011760 con 0x7f55fc0ae680 2023-10-09T06:32:18.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.094+0000 7f55f37fe700 1 -- 172.21.15.163:0/53372909 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec011b20 con 0x7f55fc0ae680 2023-10-09T06:32:18.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.094+0000 7f56012a2700 1 -- 172.21.15.163:0/53372909 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55fc15cc80 con 0x7f55fc0ae680 2023-10-09T06:32:18.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.094+0000 7f56012a2700 1 -- 172.21.15.163:0/53372909 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55fc151910 con 0x7f55fc0ae680 2023-10-09T06:32:18.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.095+0000 7f56012a2700 1 -- 172.21.15.163:0/53372909 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55fc04c3c0 con 0x7f55fc0ae680 2023-10-09T06:32:18.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.095+0000 7f55f37fe700 1 -- 172.21.15.163:0/53372909 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f55ec0193b0 con 0x7f55fc0ae680 2023-10-09T06:32:18.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.096+0000 7f55f37fe700 1 --2- 172.21.15.163:0/53372909 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f55dc066fc0 0x7f55dc069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:18.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.096+0000 7f55f37fe700 1 -- 172.21.15.163:0/53372909 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f55ec084600 con 0x7f55fc0ae680 2023-10-09T06:32:18.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.096+0000 7f55f27fc700 1 --2- 172.21.15.163:0/53372909 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f55dc066fc0 0x7f55dc069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:18.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.096+0000 7f55f27fc700 1 --2- 172.21.15.163:0/53372909 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f55dc066fc0 0x7f55dc069470 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f55e4007fd0 tx=0x7f55e4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:18.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.100+0000 7f55f37fe700 1 -- 172.21.15.163:0/53372909 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55ec051770 con 0x7f55fc0ae680 2023-10-09T06:32:18.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.325+0000 7f56012a2700 1 -- 172.21.15.163:0/53372909 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55fc0cb8c0 con 0x7f55fc0ae680 2023-10-09T06:32:18.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.326+0000 7f55f37fe700 1 -- 172.21.15.163:0/53372909 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f55ec054d90 con 0x7f55fc0ae680 2023-10-09T06:32:18.327 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:18.327 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/819128421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":819128421},{"type":"v1","addr":"172.21.15.181:6836","nonce":819128421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3339496202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3339496202},{"type":"v1","addr":"172.21.15.163:6834","nonce":3339496202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5352,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1726174482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1726174482},{"type":"v1","addr":"172.21.15.163:6835","nonce":1726174482}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:32:00.924137+0000","modified":"2023-10-09T06:32:04.563062+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5370},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5370":{"gid":5370,"name":"b","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2945196635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2945196635},{"type":"v1","addr":"172.21.15.181:6837","nonce":2945196635}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-09T06:32:18.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.328+0000 7f56012a2700 1 -- 172.21.15.163:0/53372909 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f55dc066fc0 msgr2=0x7f55dc069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:18.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.328+0000 7f56012a2700 1 --2- 172.21.15.163:0/53372909 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f55dc066fc0 0x7f55dc069470 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f55e4007fd0 tx=0x7f55e4009040 comp rx=0 tx=0).stop 2023-10-09T06:32:18.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.328+0000 7f56012a2700 1 -- 172.21.15.163:0/53372909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55fc0ae680 msgr2=0x7f55fc1513d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:18.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.328+0000 7f56012a2700 1 --2- 172.21.15.163:0/53372909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55fc0ae680 0x7f55fc1513d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00c830 tx=0x7f55ec008040 comp rx=0 tx=0).stop 2023-10-09T06:32:18.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.328+0000 7f56012a2700 1 -- 172.21.15.163:0/53372909 shutdown_connections 2023-10-09T06:32:18.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.328+0000 7f56012a2700 1 --2- 172.21.15.163:0/53372909 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f55dc066fc0 0x7f55dc069470 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:18.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.328+0000 7f56012a2700 1 --2- 172.21.15.163:0/53372909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55fc0ae680 0x7f55fc1513d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:18.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.328+0000 7f56012a2700 1 --2- 172.21.15.163:0/53372909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55fc1579f0 0x7f55fc0a6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:18.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.328+0000 7f56012a2700 1 --2- 172.21.15.163:0/53372909 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55fc0b1850 0x7f55fc0aad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:18.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.328+0000 7f56012a2700 1 -- 172.21.15.163:0/53372909 >> 172.21.15.163:0/53372909 conn(0x7f55fc06a320 msgr2=0x7f55fc153220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:18.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.328+0000 7f56012a2700 1 -- 172.21.15.163:0/53372909 shutdown_connections 2023-10-09T06:32:18.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.328+0000 7f56012a2700 1 -- 172.21.15.163:0/53372909 wait complete. 2023-10-09T06:32:18.331 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 32 2023-10-09T06:32:18.341 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:32:18.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.454+0000 7f96eb59e700 1 Processor -- start 2023-10-09T06:32:18.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.454+0000 7f96eb59e700 1 -- start start 2023-10-09T06:32:18.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.455+0000 7f96eb59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96ec06a0a0 0x7f96ec15b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:18.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.455+0000 7f96eb59e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f96ec06dd20 0x7f96ec15b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:18.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.455+0000 7f96eb59e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96ec06ae70 con 0x7f96ec06d950 2023-10-09T06:32:18.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.455+0000 7f96eb59e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96ec0c8a60 con 0x7f96ec06dd20 2023-10-09T06:32:18.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.455+0000 7f96eb59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec06a540 con 0x7f96ec06a0a0 2023-10-09T06:32:18.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.456+0000 7f96ea59c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f96ec06d950 0x7f96ec0c8350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:51734/0 (socket says 172.21.15.163:51734) 2023-10-09T06:32:18.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.456+0000 7f96e1d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96ec06a0a0 0x7f96ec15b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:18.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.456+0000 7f96ea59c700 1 -- 172.21.15.163:0/4288122162 learned_addr learned my addr 172.21.15.163:0/4288122162 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:18.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.456+0000 7f96ead9d700 1 --2- 172.21.15.163:0/4288122162 >> v2:172.21.15.181:3300/0 conn(0x7f96ec06dd20 0x7f96ec15b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:18.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.456+0000 7f96e9d9b700 1 -- 172.21.15.163:0/4288122162 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2859326818 0 0) 0x7f96ec06ae70 con 0x7f96ec06d950 2023-10-09T06:32:18.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.456+0000 7f96e9d9b700 1 -- 172.21.15.163:0/4288122162 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96d8003570 con 0x7f96ec06d950 2023-10-09T06:32:18.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.456+0000 7f96ead9d700 1 -- 172.21.15.163:0/4288122162 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96ec06a0a0 msgr2=0x7f96ec15b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:18.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.456+0000 7f96ead9d700 1 --2- 172.21.15.163:0/4288122162 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96ec06a0a0 0x7f96ec15b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:18.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.456+0000 7f96ead9d700 1 -- 172.21.15.163:0/4288122162 >> v1:172.21.15.181:6789/0 conn(0x7f96ec06d950 legacy=0x7f96ec0c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:18.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.456+0000 7f96ead9d700 1 -- 172.21.15.163:0/4288122162 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ec06a820 con 0x7f96ec06dd20 2023-10-09T06:32:18.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.456+0000 7f96ead9d700 1 --2- 172.21.15.163:0/4288122162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96ec06dd20 0x7f96ec15b950 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f96e4009e60 tx=0x7f96e4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ce8933c1e944d5c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:18.460 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.456+0000 7f96e9d9b700 1 -- 172.21.15.163:0/4288122162 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e400e070 con 0x7f96ec06dd20 2023-10-09T06:32:18.460 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.456+0000 7f96e9d9b700 1 -- 172.21.15.163:0/4288122162 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e400bab0 con 0x7f96ec06dd20 2023-10-09T06:32:18.460 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.456+0000 7f96e9d9b700 1 -- 172.21.15.163:0/4288122162 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e400be30 con 0x7f96ec06dd20 2023-10-09T06:32:18.460 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.457+0000 7f96eb59e700 1 -- 172.21.15.163:0/4288122162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96ec06dd20 msgr2=0x7f96ec15b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:18.460 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.457+0000 7f96eb59e700 1 --2- 172.21.15.163:0/4288122162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96ec06dd20 0x7f96ec15b950 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f96e4009e60 tx=0x7f96e4009780 comp rx=0 tx=0).stop 2023-10-09T06:32:18.460 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.457+0000 7f96eb59e700 1 -- 172.21.15.163:0/4288122162 shutdown_connections 2023-10-09T06:32:18.460 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.457+0000 7f96eb59e700 1 --2- 172.21.15.163:0/4288122162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96ec06dd20 0x7f96ec15b950 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:18.460 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.457+0000 7f96eb59e700 1 --2- 172.21.15.163:0/4288122162 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96ec06a0a0 0x7f96ec15b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:18.460 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.457+0000 7f96eb59e700 1 -- 172.21.15.163:0/4288122162 >> 172.21.15.163:0/4288122162 conn(0x7f96ec0cc1f0 msgr2=0x7f96ec0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:18.460 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.457+0000 7f96eb59e700 1 -- 172.21.15.163:0/4288122162 shutdown_connections 2023-10-09T06:32:18.460 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.457+0000 7f96eb59e700 1 -- 172.21.15.163:0/4288122162 wait complete. 2023-10-09T06:32:18.463 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.458+0000 7f96eb59e700 1 Processor -- start 2023-10-09T06:32:18.463 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.458+0000 7f96eb59e700 1 -- start start 2023-10-09T06:32:18.463 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.459+0000 7f96eb59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96ec06a0a0 0x7f96ec13dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:18.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.459+0000 7f96eb59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96ec06d950 0x7f96ec0c3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:18.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.459+0000 7f96eb59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96ec06dd20 0x7f96ec15ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:18.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.459+0000 7f96eb59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec13e1e0 con 0x7f96ec06a0a0 2023-10-09T06:32:18.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.459+0000 7f96eb59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec0cac90 con 0x7f96ec06d950 2023-10-09T06:32:18.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.459+0000 7f96eb59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec0c3bc0 con 0x7f96ec06dd20 2023-10-09T06:32:18.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.459+0000 7f96ea59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96ec06a0a0 0x7f96ec13dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:18.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.459+0000 7f96ea59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96ec06a0a0 0x7f96ec13dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:33106/0 (socket says 172.21.15.163:33106) 2023-10-09T06:32:18.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.459+0000 7f96ea59c700 1 -- 172.21.15.163:0/1160575786 learned_addr learned my addr 172.21.15.163:0/1160575786 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:18.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.459+0000 7f96e1d9b700 1 --2- 172.21.15.163:0/1160575786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96ec06d950 0x7f96ec0c3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:18.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.459+0000 7f96ead9d700 1 --2- 172.21.15.163:0/1160575786 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96ec06dd20 0x7f96ec15ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:18.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.459+0000 7f96ea59c700 1 -- 172.21.15.163:0/1160575786 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96ec06dd20 msgr2=0x7f96ec15ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:18.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.459+0000 7f96ea59c700 1 --2- 172.21.15.163:0/1160575786 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96ec06dd20 0x7f96ec15ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:18.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.459+0000 7f96ea59c700 1 -- 172.21.15.163:0/1160575786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96ec06d950 msgr2=0x7f96ec0c3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:18.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.459+0000 7f96ea59c700 1 --2- 172.21.15.163:0/1160575786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96ec06d950 0x7f96ec0c3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:18.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.459+0000 7f96ea59c700 1 -- 172.21.15.163:0/1160575786 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e4009a20 con 0x7f96ec06a0a0 2023-10-09T06:32:18.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.459+0000 7f96e1d9b700 1 --2- 172.21.15.163:0/1160575786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96ec06d950 0x7f96ec0c3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:18.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.459+0000 7f96ea59c700 1 --2- 172.21.15.163:0/1160575786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96ec06a0a0 0x7f96ec13dca0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00c790 tx=0x7f96dc007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:18.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.460+0000 7f96e2ffd700 1 -- 172.21.15.163:0/1160575786 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96dc005b30 con 0x7f96ec06a0a0 2023-10-09T06:32:18.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.460+0000 7f96e2ffd700 1 -- 172.21.15.163:0/1160575786 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96dc005c90 con 0x7f96ec06a0a0 2023-10-09T06:32:18.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.460+0000 7f96eb59e700 1 -- 172.21.15.163:0/1160575786 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96ec0c9dc0 con 0x7f96ec06a0a0 2023-10-09T06:32:18.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.460+0000 7f96e2ffd700 1 -- 172.21.15.163:0/1160575786 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96dc007300 con 0x7f96ec06a0a0 2023-10-09T06:32:18.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.460+0000 7f96eb59e700 1 -- 172.21.15.163:0/1160575786 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96ec0ca360 con 0x7f96ec06a0a0 2023-10-09T06:32:18.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.460+0000 7f96eb59e700 1 -- 172.21.15.163:0/1160575786 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96ec04c3c0 con 0x7f96ec06a0a0 2023-10-09T06:32:18.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.461+0000 7f96e2ffd700 1 -- 172.21.15.163:0/1160575786 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f96dc007670 con 0x7f96ec06a0a0 2023-10-09T06:32:18.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.461+0000 7f96e2ffd700 1 --2- 172.21.15.163:0/1160575786 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f96cc066fc0 0x7f96cc069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:18.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.461+0000 7f96e2ffd700 1 -- 172.21.15.163:0/1160575786 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f96dc083ad0 con 0x7f96ec06a0a0 2023-10-09T06:32:18.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.462+0000 7f96e1d9b700 1 --2- 172.21.15.163:0/1160575786 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f96cc066fc0 0x7f96cc069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:18.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.462+0000 7f96e1d9b700 1 --2- 172.21.15.163:0/1160575786 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f96cc066fc0 0x7f96cc069470 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f96ec16ca90 tx=0x7f96d4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:18.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.466+0000 7f96e2ffd700 1 -- 172.21.15.163:0/1160575786 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f96dc050d50 con 0x7f96ec06a0a0 2023-10-09T06:32:18.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.650+0000 7f96eb59e700 1 -- 172.21.15.163:0/1160575786 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96ec0c12d0 con 0x7f96ec06a0a0 2023-10-09T06:32:18.651 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.651+0000 7f96e2ffd700 1 -- 172.21.15.163:0/1160575786 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f96dc017020 con 0x7f96ec06a0a0 2023-10-09T06:32:18.651 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:18.652 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":51,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:32:02.534579+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:31:59.239255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-09T06:31:59.882874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:32:18.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.653+0000 7f96eb59e700 1 -- 172.21.15.163:0/1160575786 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f96cc066fc0 msgr2=0x7f96cc069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:18.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.653+0000 7f96eb59e700 1 --2- 172.21.15.163:0/1160575786 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f96cc066fc0 0x7f96cc069470 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f96ec16ca90 tx=0x7f96d4008040 comp rx=0 tx=0).stop 2023-10-09T06:32:18.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.653+0000 7f96eb59e700 1 -- 172.21.15.163:0/1160575786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96ec06a0a0 msgr2=0x7f96ec13dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:18.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.653+0000 7f96eb59e700 1 --2- 172.21.15.163:0/1160575786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96ec06a0a0 0x7f96ec13dca0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00c790 tx=0x7f96dc007df0 comp rx=0 tx=0).stop 2023-10-09T06:32:18.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.653+0000 7f96eb59e700 1 -- 172.21.15.163:0/1160575786 shutdown_connections 2023-10-09T06:32:18.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.653+0000 7f96eb59e700 1 --2- 172.21.15.163:0/1160575786 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f96cc066fc0 0x7f96cc069470 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:18.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.653+0000 7f96eb59e700 1 --2- 172.21.15.163:0/1160575786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96ec06a0a0 0x7f96ec13dca0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:18.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.653+0000 7f96eb59e700 1 --2- 172.21.15.163:0/1160575786 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96ec06dd20 0x7f96ec15ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:18.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.653+0000 7f96eb59e700 1 --2- 172.21.15.163:0/1160575786 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96ec06d950 0x7f96ec0c3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:18.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.653+0000 7f96eb59e700 1 -- 172.21.15.163:0/1160575786 >> 172.21.15.163:0/1160575786 conn(0x7f96ec0cc1f0 msgr2=0x7f96ec0c6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:18.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.653+0000 7f96eb59e700 1 -- 172.21.15.163:0/1160575786 shutdown_connections 2023-10-09T06:32:18.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.653+0000 7f96eb59e700 1 -- 172.21.15.163:0/1160575786 wait complete. 2023-10-09T06:32:18.666 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-09T06:32:18.666 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:32:18.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.780+0000 7fd2c9ece700 1 Processor -- start 2023-10-09T06:32:18.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.780+0000 7fd2c9ece700 1 -- start start 2023-10-09T06:32:18.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.781+0000 7fd2c9ece700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd2c406bcd0 0x7fd2c40caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:18.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.781+0000 7fd2c9ece700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c415d7e0 con 0x7fd2c406c0a0 2023-10-09T06:32:18.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.781+0000 7fd2c9ece700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c415dac0 con 0x7fd2c415df50 2023-10-09T06:32:18.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.781+0000 7fd2c9ece700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c415aa30 con 0x7fd2c406bcd0 2023-10-09T06:32:18.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.781+0000 7fd2c2ffd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fd2c406c0a0 0x7fd2c415a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53912/0 (socket says 172.21.15.163:53912) 2023-10-09T06:32:18.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.781+0000 7fd2c2ffd700 1 -- 172.21.15.163:0/3795821471 learned_addr learned my addr 172.21.15.163:0/3795821471 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:18.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.782+0000 7fd2c37fe700 1 --2- 172.21.15.163:0/3795821471 >> v2:172.21.15.181:3300/0 conn(0x7fd2c406bcd0 0x7fd2c40caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:18.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.782+0000 7fd2c27fc700 1 -- 172.21.15.163:0/3795821471 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2222556227 0 0) 0x7fd2c415d7e0 con 0x7fd2c406c0a0 2023-10-09T06:32:18.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.782+0000 7fd2c27fc700 1 -- 172.21.15.163:0/3795821471 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2ac003570 con 0x7fd2c406c0a0 2023-10-09T06:32:18.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.782+0000 7fd2c37fe700 1 -- 172.21.15.163:0/3795821471 >> v1:172.21.15.181:6789/0 conn(0x7fd2c415df50 legacy=0x7fd2c40c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:18.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.782+0000 7fd2c37fe700 1 -- 172.21.15.163:0/3795821471 >> v1:172.21.15.163:6789/0 conn(0x7fd2c406c0a0 legacy=0x7fd2c415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:18.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.782+0000 7fd2c37fe700 1 -- 172.21.15.163:0/3795821471 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c415ad10 con 0x7fd2c406bcd0 2023-10-09T06:32:18.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.782+0000 7fd2c37fe700 1 --2- 172.21.15.163:0/3795821471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2c406bcd0 0x7fd2c40caa90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40098e0 tx=0x7fd2b40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce44c4cad09827c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:18.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.782+0000 7fd2c27fc700 1 -- 172.21.15.163:0/3795821471 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b400b640 con 0x7fd2c406bcd0 2023-10-09T06:32:18.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.782+0000 7fd2c27fc700 1 -- 172.21.15.163:0/3795821471 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b400b7a0 con 0x7fd2c406bcd0 2023-10-09T06:32:18.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.782+0000 7fd2c27fc700 1 -- 172.21.15.163:0/3795821471 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b400bb40 con 0x7fd2c406bcd0 2023-10-09T06:32:18.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.783+0000 7fd2c9ece700 1 -- 172.21.15.163:0/3795821471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2c406bcd0 msgr2=0x7fd2c40caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:18.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.783+0000 7fd2c9ece700 1 --2- 172.21.15.163:0/3795821471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2c406bcd0 0x7fd2c40caa90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40098e0 tx=0x7fd2b40091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:18.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.783+0000 7fd2c9ece700 1 -- 172.21.15.163:0/3795821471 shutdown_connections 2023-10-09T06:32:18.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.783+0000 7fd2c9ece700 1 --2- 172.21.15.163:0/3795821471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2c406bcd0 0x7fd2c40caa90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:18.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.783+0000 7fd2c9ece700 1 -- 172.21.15.163:0/3795821471 >> 172.21.15.163:0/3795821471 conn(0x7fd2c40ce600 msgr2=0x7fd2c40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:18.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.783+0000 7fd2c9ece700 1 -- 172.21.15.163:0/3795821471 shutdown_connections 2023-10-09T06:32:18.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.783+0000 7fd2c9ece700 1 -- 172.21.15.163:0/3795821471 wait complete. 2023-10-09T06:32:18.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.784+0000 7fd2c9ece700 1 Processor -- start 2023-10-09T06:32:18.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.784+0000 7fd2c9ece700 1 -- start start 2023-10-09T06:32:18.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.785+0000 7fd2c9ece700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2c406bcd0 0x7fd2c4146110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:18.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.785+0000 7fd2c9ece700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2c406c0a0 0x7fd2c415bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:18.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.785+0000 7fd2c9ece700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2c415df50 0x7fd2c415c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:18.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.785+0000 7fd2c9ece700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c416af60 con 0x7fd2c406c0a0 2023-10-09T06:32:18.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.785+0000 7fd2c9ece700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c416b7c0 con 0x7fd2c415df50 2023-10-09T06:32:18.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.785+0000 7fd2c9ece700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c416e140 con 0x7fd2c406bcd0 2023-10-09T06:32:18.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.785+0000 7fd2c2ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2c406c0a0 0x7fd2c415bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:18.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.785+0000 7fd2c2ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2c406c0a0 0x7fd2c415bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58384/0 (socket says 172.21.15.163:58384) 2023-10-09T06:32:18.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.785+0000 7fd2c2ffd700 1 -- 172.21.15.163:0/3317727808 learned_addr learned my addr 172.21.15.163:0/3317727808 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:18.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.785+0000 7fd2c37fe700 1 --2- 172.21.15.163:0/3317727808 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2c406bcd0 0x7fd2c4146110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:18.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.785+0000 7fd2c3fff700 1 --2- 172.21.15.163:0/3317727808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2c415df50 0x7fd2c415c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:18.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.785+0000 7fd2c2ffd700 1 -- 172.21.15.163:0/3317727808 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2c406bcd0 msgr2=0x7fd2c4146110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:18.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.785+0000 7fd2c2ffd700 1 --2- 172.21.15.163:0/3317727808 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2c406bcd0 0x7fd2c4146110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:18.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.785+0000 7fd2c2ffd700 1 -- 172.21.15.163:0/3317727808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2c415df50 msgr2=0x7fd2c415c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:18.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.785+0000 7fd2c2ffd700 1 --2- 172.21.15.163:0/3317727808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2c415df50 0x7fd2c415c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:18.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.785+0000 7fd2c2ffd700 1 -- 172.21.15.163:0/3317727808 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b4009420 con 0x7fd2c406c0a0 2023-10-09T06:32:18.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.785+0000 7fd2c37fe700 1 --2- 172.21.15.163:0/3317727808 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2c406bcd0 0x7fd2c4146110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:32:18.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.785+0000 7fd2c3fff700 1 --2- 172.21.15.163:0/3317727808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2c415df50 0x7fd2c415c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:32:18.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.785+0000 7fd2c2ffd700 1 --2- 172.21.15.163:0/3317727808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2c406c0a0 0x7fd2c415bca0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800c590 tx=0x7fd2b8007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:18.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.785+0000 7fd2ab7fe700 1 -- 172.21.15.163:0/3317727808 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8005b30 con 0x7fd2c406c0a0 2023-10-09T06:32:18.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.785+0000 7fd2ab7fe700 1 -- 172.21.15.163:0/3317727808 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b8005cb0 con 0x7fd2c406c0a0 2023-10-09T06:32:18.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.785+0000 7fd2c9ece700 1 -- 172.21.15.163:0/3317727808 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c416b170 con 0x7fd2c406c0a0 2023-10-09T06:32:18.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.785+0000 7fd2ab7fe700 1 -- 172.21.15.163:0/3317727808 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8007320 con 0x7fd2c406c0a0 2023-10-09T06:32:18.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.786+0000 7fd2c9ece700 1 -- 172.21.15.163:0/3317727808 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c4146650 con 0x7fd2c406c0a0 2023-10-09T06:32:18.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.786+0000 7fd2c9ece700 1 -- 172.21.15.163:0/3317727808 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c416b970 con 0x7fd2c406c0a0 2023-10-09T06:32:18.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.787+0000 7fd2ab7fe700 1 -- 172.21.15.163:0/3317727808 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fd2b801b3b0 con 0x7fd2c406c0a0 2023-10-09T06:32:18.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.787+0000 7fd2ab7fe700 1 --2- 172.21.15.163:0/3317727808 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd2b0066fc0 0x7fd2b0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:18.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.787+0000 7fd2ab7fe700 1 -- 172.21.15.163:0/3317727808 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd2b8083c70 con 0x7fd2c406c0a0 2023-10-09T06:32:18.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.788+0000 7fd2c37fe700 1 --2- 172.21.15.163:0/3317727808 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd2b0066fc0 0x7fd2b0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:18.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.788+0000 7fd2c37fe700 1 --2- 172.21.15.163:0/3317727808 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd2b0066fc0 0x7fd2b0069470 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40ccd70 tx=0x7fd2b4017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:18.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:18.791+0000 7fd2ab7fe700 1 -- 172.21.15.163:0/3317727808 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd2c416b970 con 0x7fd2c406c0a0 2023-10-09T06:32:19.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.017+0000 7fd2c9ece700 1 -- 172.21.15.163:0/3317727808 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2c4146810 con 0x7fd2c406c0a0 2023-10-09T06:32:19.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.017+0000 7fd2ab7fe700 1 -- 172.21.15.163:0/3317727808 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4606 (secure 0 0 0) 0x7fd2b8017070 con 0x7fd2c406c0a0 2023-10-09T06:32:19.018 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:19.018 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/819128421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":819128421},{"type":"v1","addr":"172.21.15.181:6836","nonce":819128421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3339496202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3339496202},{"type":"v1","addr":"172.21.15.163:6834","nonce":3339496202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5352,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1726174482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1726174482},{"type":"v1","addr":"172.21.15.163:6835","nonce":1726174482}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:32:00.924137+0000","modified":"2023-10-09T06:32:04.563062+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5370},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5370":{"gid":5370,"name":"b","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2945196635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2945196635},{"type":"v1","addr":"172.21.15.181:6837","nonce":2945196635}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-09T06:32:19.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.019+0000 7fd2c9ece700 1 -- 172.21.15.163:0/3317727808 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd2b0066fc0 msgr2=0x7fd2b0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:19.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.019+0000 7fd2c9ece700 1 --2- 172.21.15.163:0/3317727808 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd2b0066fc0 0x7fd2b0069470 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40ccd70 tx=0x7fd2b4017000 comp rx=0 tx=0).stop 2023-10-09T06:32:19.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.019+0000 7fd2c9ece700 1 -- 172.21.15.163:0/3317727808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2c406c0a0 msgr2=0x7fd2c415bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:19.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.019+0000 7fd2c9ece700 1 --2- 172.21.15.163:0/3317727808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2c406c0a0 0x7fd2c415bca0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800c590 tx=0x7fd2b8007e00 comp rx=0 tx=0).stop 2023-10-09T06:32:19.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.019+0000 7fd2c9ece700 1 -- 172.21.15.163:0/3317727808 shutdown_connections 2023-10-09T06:32:19.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.019+0000 7fd2c9ece700 1 --2- 172.21.15.163:0/3317727808 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd2b0066fc0 0x7fd2b0069470 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.019+0000 7fd2c9ece700 1 --2- 172.21.15.163:0/3317727808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd2c415df50 0x7fd2c415c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.019+0000 7fd2c9ece700 1 --2- 172.21.15.163:0/3317727808 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd2c406bcd0 0x7fd2c4146110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.019+0000 7fd2c9ece700 1 --2- 172.21.15.163:0/3317727808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2c406c0a0 0x7fd2c415bca0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.019+0000 7fd2c9ece700 1 -- 172.21.15.163:0/3317727808 >> 172.21.15.163:0/3317727808 conn(0x7fd2c40ce600 msgr2=0x7fd2c4069f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:19.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.019+0000 7fd2c9ece700 1 -- 172.21.15.163:0/3317727808 shutdown_connections 2023-10-09T06:32:19.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.019+0000 7fd2c9ece700 1 -- 172.21.15.163:0/3317727808 wait complete. 2023-10-09T06:32:19.022 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 32 2023-10-09T06:32:19.032 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:32:19.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.146+0000 7fa4e3883700 1 Processor -- start 2023-10-09T06:32:19.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.146+0000 7fa4e3883700 1 -- start start 2023-10-09T06:32:19.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.147+0000 7fa4e3883700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa4dc069560 0x7fa4dc0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:19.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.147+0000 7fa4e3883700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4dc069ee0 con 0x7fa4dc159890 2023-10-09T06:32:19.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.147+0000 7fa4e3883700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4dc15a160 con 0x7fa4dc159c60 2023-10-09T06:32:19.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.147+0000 7fa4e3883700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4dc0d13a0 con 0x7fa4dc069560 2023-10-09T06:32:19.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.147+0000 7fa4e161f700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fa4dc159890 0x7fa4dc15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53916/0 (socket says 172.21.15.163:53916) 2023-10-09T06:32:19.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.147+0000 7fa4e1e20700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa4dc069560 0x7fa4dc0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:19.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.147+0000 7fa4e161f700 1 -- 172.21.15.163:0/3684033817 learned_addr learned my addr 172.21.15.163:0/3684033817 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:19.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.148+0000 7fa4e1e20700 1 -- 172.21.15.163:0/3684033817 >> v1:172.21.15.181:6789/0 conn(0x7fa4dc159c60 legacy=0x7fa4dc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:19.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.148+0000 7fa4e1e20700 1 -- 172.21.15.163:0/3684033817 >> v1:172.21.15.163:6789/0 conn(0x7fa4dc159890 legacy=0x7fa4dc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:19.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.148+0000 7fa4d3fff700 1 -- 172.21.15.163:0/3684033817 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1325215342 0 0) 0x7fa4dc069ee0 con 0x7fa4dc159890 2023-10-09T06:32:19.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.148+0000 7fa4e1e20700 1 -- 172.21.15.163:0/3684033817 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4dc15e140 con 0x7fa4dc069560 2023-10-09T06:32:19.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.148+0000 7fa4e1e20700 1 --2- 172.21.15.163:0/3684033817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4dc069560 0x7fa4dc0699a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40098e0 tx=0x7fa4d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d7395c0a92659c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:19.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.148+0000 7fa4d3fff700 1 -- 172.21.15.163:0/3684033817 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4009e80 con 0x7fa4dc069560 2023-10-09T06:32:19.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.148+0000 7fa4d3fff700 1 -- 172.21.15.163:0/3684033817 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d400b820 con 0x7fa4dc069560 2023-10-09T06:32:19.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.148+0000 7fa4d3fff700 1 -- 172.21.15.163:0/3684033817 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d400bba0 con 0x7fa4dc069560 2023-10-09T06:32:19.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.148+0000 7fa4e3883700 1 -- 172.21.15.163:0/3684033817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4dc069560 msgr2=0x7fa4dc0699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:19.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.148+0000 7fa4e3883700 1 --2- 172.21.15.163:0/3684033817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4dc069560 0x7fa4dc0699a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40098e0 tx=0x7fa4d40091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.148+0000 7fa4e3883700 1 -- 172.21.15.163:0/3684033817 shutdown_connections 2023-10-09T06:32:19.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.148+0000 7fa4e3883700 1 --2- 172.21.15.163:0/3684033817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4dc069560 0x7fa4dc0699a0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.148+0000 7fa4e3883700 1 -- 172.21.15.163:0/3684033817 >> 172.21.15.163:0/3684033817 conn(0x7fa4dc068e20 msgr2=0x7fa4dc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:19.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.148+0000 7fa4e3883700 1 -- 172.21.15.163:0/3684033817 shutdown_connections 2023-10-09T06:32:19.152 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.148+0000 7fa4e3883700 1 -- 172.21.15.163:0/3684033817 wait complete. 2023-10-09T06:32:19.154 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.149+0000 7fa4e3883700 1 Processor -- start 2023-10-09T06:32:19.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.150+0000 7fa4e3883700 1 -- start start 2023-10-09T06:32:19.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.150+0000 7fa4e3883700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4dc069560 0x7fa4dc1534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:19.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.150+0000 7fa4e3883700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4dc159890 0x7fa4dc0b5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:19.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.150+0000 7fa4e3883700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4dc159c60 0x7fa4dc154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:19.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.150+0000 7fa4e3883700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4dc0cd0c0 con 0x7fa4dc069560 2023-10-09T06:32:19.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.150+0000 7fa4e3883700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4dc0cd960 con 0x7fa4dc159890 2023-10-09T06:32:19.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.150+0000 7fa4e3883700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4dc153a30 con 0x7fa4dc159c60 2023-10-09T06:32:19.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.150+0000 7fa4e161f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4dc069560 0x7fa4dc1534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:19.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.150+0000 7fa4e161f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4dc069560 0x7fa4dc1534f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58394/0 (socket says 172.21.15.163:58394) 2023-10-09T06:32:19.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.150+0000 7fa4e161f700 1 -- 172.21.15.163:0/1536803424 learned_addr learned my addr 172.21.15.163:0/1536803424 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:19.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.151+0000 7fa4e161f700 1 -- 172.21.15.163:0/1536803424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4dc159c60 msgr2=0x7fa4dc154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:19.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.151+0000 7fa4e161f700 1 --2- 172.21.15.163:0/1536803424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4dc159c60 0x7fa4dc154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.151+0000 7fa4e161f700 1 -- 172.21.15.163:0/1536803424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4dc159890 msgr2=0x7fa4dc0b5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:19.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.151+0000 7fa4e0e1e700 1 --2- 172.21.15.163:0/1536803424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4dc159890 0x7fa4dc0b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:19.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.151+0000 7fa4e161f700 1 --2- 172.21.15.163:0/1536803424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4dc159890 0x7fa4dc0b5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.151+0000 7fa4e161f700 1 -- 172.21.15.163:0/1536803424 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d4009420 con 0x7fa4dc069560 2023-10-09T06:32:19.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.151+0000 7fa4e161f700 1 --2- 172.21.15.163:0/1536803424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4dc069560 0x7fa4dc1534f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa4d800cdb0 tx=0x7fa4d8001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:19.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.151+0000 7fa4d17fa700 1 -- 172.21.15.163:0/1536803424 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d800aad0 con 0x7fa4dc069560 2023-10-09T06:32:19.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.151+0000 7fa4d17fa700 1 -- 172.21.15.163:0/1536803424 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d800ac30 con 0x7fa4dc069560 2023-10-09T06:32:19.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.151+0000 7fa4e3883700 1 -- 172.21.15.163:0/1536803424 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4dc0b62c0 con 0x7fa4dc069560 2023-10-09T06:32:19.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.151+0000 7fa4d17fa700 1 -- 172.21.15.163:0/1536803424 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d80070f0 con 0x7fa4dc069560 2023-10-09T06:32:19.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.151+0000 7fa4e3883700 1 -- 172.21.15.163:0/1536803424 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4dc0cd5f0 con 0x7fa4dc069560 2023-10-09T06:32:19.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.152+0000 7fa4e3883700 1 -- 172.21.15.163:0/1536803424 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4dc04c3c0 con 0x7fa4dc069560 2023-10-09T06:32:19.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.152+0000 7fa4d17fa700 1 -- 172.21.15.163:0/1536803424 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fa4d80074a0 con 0x7fa4dc069560 2023-10-09T06:32:19.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.153+0000 7fa4d17fa700 1 --2- 172.21.15.163:0/1536803424 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa4c0066fc0 0x7fa4c0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:19.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.153+0000 7fa4d17fa700 1 -- 172.21.15.163:0/1536803424 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa4d8084a60 con 0x7fa4dc069560 2023-10-09T06:32:19.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.153+0000 7fa4e0e1e700 1 --2- 172.21.15.163:0/1536803424 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa4c0066fc0 0x7fa4c0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:19.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.153+0000 7fa4e0e1e700 1 --2- 172.21.15.163:0/1536803424 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa4c0066fc0 0x7fa4c0069470 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc0cbc00 tx=0x7fa4cc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:19.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.157+0000 7fa4d17fa700 1 -- 172.21.15.163:0/1536803424 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa4d8051d60 con 0x7fa4dc069560 2023-10-09T06:32:19.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.383+0000 7fa4e3883700 1 -- 172.21.15.163:0/1536803424 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4dc06a5b0 con 0x7fa4dc069560 2023-10-09T06:32:19.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.383+0000 7fa4d17fa700 1 -- 172.21.15.163:0/1536803424 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4606 (secure 0 0 0) 0x7fa4d8055380 con 0x7fa4dc069560 2023-10-09T06:32:19.384 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:19.385 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/819128421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":819128421},{"type":"v1","addr":"172.21.15.181:6836","nonce":819128421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3339496202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3339496202},{"type":"v1","addr":"172.21.15.163:6834","nonce":3339496202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5352,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1726174482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1726174482},{"type":"v1","addr":"172.21.15.163:6835","nonce":1726174482}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:32:00.924137+0000","modified":"2023-10-09T06:32:04.563062+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5370},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5370":{"gid":5370,"name":"b","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2945196635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2945196635},{"type":"v1","addr":"172.21.15.181:6837","nonce":2945196635}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-09T06:32:19.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.386+0000 7fa4e3883700 1 -- 172.21.15.163:0/1536803424 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa4c0066fc0 msgr2=0x7fa4c0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:19.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.386+0000 7fa4e3883700 1 --2- 172.21.15.163:0/1536803424 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa4c0066fc0 0x7fa4c0069470 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc0cbc00 tx=0x7fa4cc008040 comp rx=0 tx=0).stop 2023-10-09T06:32:19.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.386+0000 7fa4e3883700 1 -- 172.21.15.163:0/1536803424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4dc069560 msgr2=0x7fa4dc1534f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:19.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.386+0000 7fa4e3883700 1 --2- 172.21.15.163:0/1536803424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4dc069560 0x7fa4dc1534f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa4d800cdb0 tx=0x7fa4d8001ae0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.386+0000 7fa4e3883700 1 -- 172.21.15.163:0/1536803424 shutdown_connections 2023-10-09T06:32:19.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.386+0000 7fa4e3883700 1 --2- 172.21.15.163:0/1536803424 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa4c0066fc0 0x7fa4c0069470 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.386+0000 7fa4e3883700 1 --2- 172.21.15.163:0/1536803424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4dc069560 0x7fa4dc1534f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.386+0000 7fa4e3883700 1 --2- 172.21.15.163:0/1536803424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4dc159c60 0x7fa4dc154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.386+0000 7fa4e3883700 1 --2- 172.21.15.163:0/1536803424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4dc159890 0x7fa4dc0b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.386+0000 7fa4e3883700 1 -- 172.21.15.163:0/1536803424 >> 172.21.15.163:0/1536803424 conn(0x7fa4dc068e20 msgr2=0x7fa4dc0d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:19.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.386+0000 7fa4e3883700 1 -- 172.21.15.163:0/1536803424 shutdown_connections 2023-10-09T06:32:19.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.386+0000 7fa4e3883700 1 -- 172.21.15.163:0/1536803424 wait complete. 2023-10-09T06:32:19.389 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 32 2023-10-09T06:32:19.399 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:32:19.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.513+0000 7f670490d700 1 Processor -- start 2023-10-09T06:32:19.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.513+0000 7f670490d700 1 -- start start 2023-10-09T06:32:19.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.514+0000 7f670490d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f67001574d0 0x7f67000cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:19.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.514+0000 7f670490d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f67000cc3c0 0x7f670006b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:19.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.514+0000 7f670490d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f670006bf20 0x7f670006cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:19.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.514+0000 7f670490d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67000b16a0 con 0x7f67001574d0 2023-10-09T06:32:19.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.514+0000 7f670490d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67000b1c60 con 0x7f67000cc3c0 2023-10-09T06:32:19.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.514+0000 7f670490d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f670006c360 con 0x7f670006bf20 2023-10-09T06:32:19.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.514+0000 7f66ff7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f67001574d0 0x7f67000cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:19.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.514+0000 7f66ff7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f67001574d0 0x7f67000cd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58406/0 (socket says 172.21.15.163:58406) 2023-10-09T06:32:19.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.514+0000 7f66ff7fe700 1 -- 172.21.15.163:0/2725302106 learned_addr learned my addr 172.21.15.163:0/2725302106 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:19.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.514+0000 7f66feffd700 1 --2- 172.21.15.163:0/2725302106 >> v2:172.21.15.181:3300/0 conn(0x7f67000cc3c0 0x7f670006b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:19.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.514+0000 7f66fffff700 1 --2- 172.21.15.163:0/2725302106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f670006bf20 0x7f670006cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:19.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.514+0000 7f66ff7fe700 1 -- 172.21.15.163:0/2725302106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f670006bf20 msgr2=0x7f670006cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:19.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.514+0000 7f66ff7fe700 1 --2- 172.21.15.163:0/2725302106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f670006bf20 0x7f670006cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.514+0000 7f66ff7fe700 1 -- 172.21.15.163:0/2725302106 >> v2:172.21.15.181:3300/0 conn(0x7f67000cc3c0 msgr2=0x7f670006b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:19.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.514+0000 7f66ff7fe700 1 --2- 172.21.15.163:0/2725302106 >> v2:172.21.15.181:3300/0 conn(0x7f67000cc3c0 0x7f670006b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.514+0000 7f66ff7fe700 1 -- 172.21.15.163:0/2725302106 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67001578a0 con 0x7f67001574d0 2023-10-09T06:32:19.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.515+0000 7f66ff7fe700 1 --2- 172.21.15.163:0/2725302106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67001574d0 0x7f67000cd270 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f670006eec0 tx=0x7f66f0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc9c63948c41ddf server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:19.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.515+0000 7f66fe7fc700 1 -- 172.21.15.163:0/2725302106 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f000e070 con 0x7f67001574d0 2023-10-09T06:32:19.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.515+0000 7f66fe7fc700 1 -- 172.21.15.163:0/2725302106 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f000b970 con 0x7f67001574d0 2023-10-09T06:32:19.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.515+0000 7f66fe7fc700 1 -- 172.21.15.163:0/2725302106 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f00092e0 con 0x7f67001574d0 2023-10-09T06:32:19.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.515+0000 7f670490d700 1 -- 172.21.15.163:0/2725302106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67001574d0 msgr2=0x7f67000cd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:19.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.515+0000 7f670490d700 1 --2- 172.21.15.163:0/2725302106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67001574d0 0x7f67000cd270 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f670006eec0 tx=0x7f66f0009840 comp rx=0 tx=0).stop 2023-10-09T06:32:19.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.515+0000 7f670490d700 1 -- 172.21.15.163:0/2725302106 shutdown_connections 2023-10-09T06:32:19.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.515+0000 7f670490d700 1 --2- 172.21.15.163:0/2725302106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f670006bf20 0x7f670006cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.515+0000 7f670490d700 1 --2- 172.21.15.163:0/2725302106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67001574d0 0x7f67000cd270 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.515+0000 7f670490d700 1 --2- 172.21.15.163:0/2725302106 >> v2:172.21.15.181:3300/0 conn(0x7f67000cc3c0 0x7f670006b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.515+0000 7f670490d700 1 -- 172.21.15.163:0/2725302106 >> 172.21.15.163:0/2725302106 conn(0x7f67000ce430 msgr2=0x7f67000ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:19.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.515+0000 7f670490d700 1 -- 172.21.15.163:0/2725302106 shutdown_connections 2023-10-09T06:32:19.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.515+0000 7f670490d700 1 -- 172.21.15.163:0/2725302106 wait complete. 2023-10-09T06:32:19.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.517+0000 7f670490d700 1 Processor -- start 2023-10-09T06:32:19.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.517+0000 7f670490d700 1 -- start start 2023-10-09T06:32:19.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.517+0000 7f670490d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f670006bf20 0x7f67001677d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:19.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.517+0000 7f670490d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67000cc3c0 0x7f670016a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:19.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.517+0000 7f670490d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67001574d0 0x7f67001588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:19.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.517+0000 7f670490d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670015ac80 con 0x7f670006bf20 2023-10-09T06:32:19.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.517+0000 7f670490d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6700167d10 con 0x7f67000cc3c0 2023-10-09T06:32:19.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.517+0000 7f670490d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f670015a070 con 0x7f67001574d0 2023-10-09T06:32:19.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.517+0000 7f66ff7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f670006bf20 0x7f67001677d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:19.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.517+0000 7f66ff7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f670006bf20 0x7f67001677d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58418/0 (socket says 172.21.15.163:58418) 2023-10-09T06:32:19.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.517+0000 7f66ff7fe700 1 -- 172.21.15.163:0/3422434694 learned_addr learned my addr 172.21.15.163:0/3422434694 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:19.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.518+0000 7f66ff7fe700 1 -- 172.21.15.163:0/3422434694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67001574d0 msgr2=0x7f67001588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:19.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.518+0000 7f66ff7fe700 1 --2- 172.21.15.163:0/3422434694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67001574d0 0x7f67001588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.518+0000 7f66ff7fe700 1 -- 172.21.15.163:0/3422434694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67000cc3c0 msgr2=0x7f670016a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:19.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.518+0000 7f66ff7fe700 1 --2- 172.21.15.163:0/3422434694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67000cc3c0 0x7f670016a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.518+0000 7f66ff7fe700 1 -- 172.21.15.163:0/3422434694 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f0009ae0 con 0x7f670006bf20 2023-10-09T06:32:19.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.518+0000 7f66ff7fe700 1 --2- 172.21.15.163:0/3422434694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f670006bf20 0x7f67001677d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f670006edf0 tx=0x7f66f0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:19.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.518+0000 7f66e77fe700 1 -- 172.21.15.163:0/3422434694 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f00088c0 con 0x7f670006bf20 2023-10-09T06:32:19.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.518+0000 7f66e77fe700 1 -- 172.21.15.163:0/3422434694 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f0008a20 con 0x7f670006bf20 2023-10-09T06:32:19.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.518+0000 7f66e77fe700 1 -- 172.21.15.163:0/3422434694 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f0008de0 con 0x7f670006bf20 2023-10-09T06:32:19.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.518+0000 7f670490d700 1 -- 172.21.15.163:0/3422434694 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f670015a350 con 0x7f670006bf20 2023-10-09T06:32:19.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.518+0000 7f670490d700 1 -- 172.21.15.163:0/3422434694 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f670015a910 con 0x7f670006bf20 2023-10-09T06:32:19.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.519+0000 7f670490d700 1 -- 172.21.15.163:0/3422434694 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f670004c3c0 con 0x7f670006bf20 2023-10-09T06:32:19.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.519+0000 7f66e77fe700 1 -- 172.21.15.163:0/3422434694 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f66f0003040 con 0x7f670006bf20 2023-10-09T06:32:19.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.520+0000 7f66e77fe700 1 --2- 172.21.15.163:0/3422434694 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f66ec066fc0 0x7f66ec069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:19.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.520+0000 7f66e77fe700 1 -- 172.21.15.163:0/3422434694 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f66f0086060 con 0x7f670006bf20 2023-10-09T06:32:19.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.520+0000 7f66feffd700 1 --2- 172.21.15.163:0/3422434694 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f66ec066fc0 0x7f66ec069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:19.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.520+0000 7f66feffd700 1 --2- 172.21.15.163:0/3422434694 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f66ec066fc0 0x7f66ec069470 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f67001567b0 tx=0x7f66f40091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:19.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.524+0000 7f66e77fe700 1 -- 172.21.15.163:0/3422434694 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f66f00531d0 con 0x7f670006bf20 2023-10-09T06:32:19.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.710+0000 7f670490d700 1 -- 172.21.15.163:0/3422434694 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67000cdb70 con 0x7f670006bf20 2023-10-09T06:32:19.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.711+0000 7f66e77fe700 1 -- 172.21.15.163:0/3422434694 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f66f00567f0 con 0x7f670006bf20 2023-10-09T06:32:19.712 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:19.712 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":51,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:32:02.534579+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:31:59.239255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-09T06:31:59.882874+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:32:19.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.713+0000 7f670490d700 1 -- 172.21.15.163:0/3422434694 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f66ec066fc0 msgr2=0x7f66ec069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:19.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.713+0000 7f670490d700 1 --2- 172.21.15.163:0/3422434694 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f66ec066fc0 0x7f66ec069470 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f67001567b0 tx=0x7f66f40091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.713+0000 7f670490d700 1 -- 172.21.15.163:0/3422434694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f670006bf20 msgr2=0x7f67001677d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:19.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.713+0000 7f670490d700 1 --2- 172.21.15.163:0/3422434694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f670006bf20 0x7f67001677d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f670006edf0 tx=0x7f66f0018000 comp rx=0 tx=0).stop 2023-10-09T06:32:19.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.713+0000 7f670490d700 1 -- 172.21.15.163:0/3422434694 shutdown_connections 2023-10-09T06:32:19.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.713+0000 7f670490d700 1 --2- 172.21.15.163:0/3422434694 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f66ec066fc0 0x7f66ec069470 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.713+0000 7f670490d700 1 --2- 172.21.15.163:0/3422434694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f670006bf20 0x7f67001677d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.713+0000 7f670490d700 1 --2- 172.21.15.163:0/3422434694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67001574d0 0x7f67001588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.713+0000 7f670490d700 1 --2- 172.21.15.163:0/3422434694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67000cc3c0 0x7f670016a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.713+0000 7f670490d700 1 -- 172.21.15.163:0/3422434694 >> 172.21.15.163:0/3422434694 conn(0x7f67000ce430 msgr2=0x7f6700153d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:19.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.713+0000 7f670490d700 1 -- 172.21.15.163:0/3422434694 shutdown_connections 2023-10-09T06:32:19.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.713+0000 7f670490d700 1 -- 172.21.15.163:0/3422434694 wait complete. 2023-10-09T06:32:19.727 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-09T06:32:19.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.842+0000 7f47cfc3c700 1 Processor -- start 2023-10-09T06:32:19.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.842+0000 7f47cfc3c700 1 -- start start 2023-10-09T06:32:19.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.843+0000 7f47cfc3c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f47c8159890 0x7f47c815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:19.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.843+0000 7f47cfc3c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f47c8159c60 0x7f47c815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:19.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.843+0000 7f47cfc3c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c815a160 con 0x7f47c815db10 2023-10-09T06:32:19.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.843+0000 7f47cfc3c700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c815df50 con 0x7f47c8159c60 2023-10-09T06:32:19.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.843+0000 7f47cfc3c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c80d13a0 con 0x7f47c8159890 2023-10-09T06:32:19.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.843+0000 7f47cd1d7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f47c8159c60 0x7f47c815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:19.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.843+0000 7f47cd1d7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f47c8159c60 0x7f47c815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58430/0 (socket says 172.21.15.163:58430) 2023-10-09T06:32:19.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.843+0000 7f47cd1d7700 1 -- 172.21.15.163:0/3770898699 learned_addr learned my addr 172.21.15.163:0/3770898699 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:19.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.843+0000 7f47cd9d8700 1 --2- 172.21.15.163:0/3770898699 >> v2:172.21.15.181:3300/0 conn(0x7f47c8159890 0x7f47c815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:19.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.843+0000 7f47cd1d7700 1 -- 172.21.15.163:0/3770898699 >> v2:172.21.15.181:3300/0 conn(0x7f47c8159890 msgr2=0x7f47c815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:19.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.843+0000 7f47cd1d7700 1 --2- 172.21.15.163:0/3770898699 >> v2:172.21.15.181:3300/0 conn(0x7f47c8159890 0x7f47c815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.843+0000 7f47cd1d7700 1 -- 172.21.15.163:0/3770898699 >> v1:172.21.15.181:6789/0 conn(0x7f47c815db10 legacy=0x7f47c8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:19.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.843+0000 7f47cd1d7700 1 -- 172.21.15.163:0/3770898699 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c815e140 con 0x7f47c8159c60 2023-10-09T06:32:19.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.843+0000 7f47cd1d7700 1 --2- 172.21.15.163:0/3770898699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47c8159c60 0x7f47c815d5d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f47c80d1090 tx=0x7f47b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ebaadf38901c45 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:19.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.843+0000 7f47cc9d6700 1 -- 172.21.15.163:0/3770898699 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b800b820 con 0x7f47c8159c60 2023-10-09T06:32:19.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.843+0000 7f47cc9d6700 1 -- 172.21.15.163:0/3770898699 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b800b980 con 0x7f47c8159c60 2023-10-09T06:32:19.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.843+0000 7f47cc9d6700 1 -- 172.21.15.163:0/3770898699 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b800bd20 con 0x7f47c8159c60 2023-10-09T06:32:19.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.844+0000 7f47cfc3c700 1 -- 172.21.15.163:0/3770898699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47c8159c60 msgr2=0x7f47c815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:19.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.844+0000 7f47cfc3c700 1 --2- 172.21.15.163:0/3770898699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47c8159c60 0x7f47c815d5d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f47c80d1090 tx=0x7f47b8009250 comp rx=0 tx=0).stop 2023-10-09T06:32:19.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.844+0000 7f47cfc3c700 1 -- 172.21.15.163:0/3770898699 shutdown_connections 2023-10-09T06:32:19.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.844+0000 7f47cfc3c700 1 --2- 172.21.15.163:0/3770898699 >> v2:172.21.15.181:3300/0 conn(0x7f47c8159890 0x7f47c815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.844+0000 7f47cfc3c700 1 --2- 172.21.15.163:0/3770898699 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47c8159c60 0x7f47c815d5d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.844+0000 7f47cfc3c700 1 -- 172.21.15.163:0/3770898699 >> 172.21.15.163:0/3770898699 conn(0x7f47c8068e20 msgr2=0x7f47c80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:19.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.844+0000 7f47cfc3c700 1 -- 172.21.15.163:0/3770898699 shutdown_connections 2023-10-09T06:32:19.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.844+0000 7f47cfc3c700 1 -- 172.21.15.163:0/3770898699 wait complete. 2023-10-09T06:32:19.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.845+0000 7f47cfc3c700 1 Processor -- start 2023-10-09T06:32:19.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.845+0000 7f47cfc3c700 1 -- start start 2023-10-09T06:32:19.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.846+0000 7f47cfc3c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47c8159890 0x7f47c8140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:19.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.846+0000 7f47cfc3c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47c8159c60 0x7f47c80ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:19.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.846+0000 7f47cfc3c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47c815db10 0x7f47c8154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:19.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.846+0000 7f47cfc3c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c80cf4c0 con 0x7f47c8159c60 2023-10-09T06:32:19.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.846+0000 7f47cfc3c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c80cf730 con 0x7f47c815db10 2023-10-09T06:32:19.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.846+0000 7f47cfc3c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c80cfa10 con 0x7f47c8159890 2023-10-09T06:32:19.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.846+0000 7f47cd1d7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47c8159c60 0x7f47c80ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:19.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.846+0000 7f47cd1d7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47c8159c60 0x7f47c80ca0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58440/0 (socket says 172.21.15.163:58440) 2023-10-09T06:32:19.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.846+0000 7f47cd1d7700 1 -- 172.21.15.163:0/2434149886 learned_addr learned my addr 172.21.15.163:0/2434149886 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:19.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.846+0000 7f47ce1d9700 1 --2- 172.21.15.163:0/2434149886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47c815db10 0x7f47c8154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:19.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.846+0000 7f47cd9d8700 1 --2- 172.21.15.163:0/2434149886 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47c8159890 0x7f47c8140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:19.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.846+0000 7f47cd1d7700 1 -- 172.21.15.163:0/2434149886 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47c8159890 msgr2=0x7f47c8140dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:19.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.846+0000 7f47cd1d7700 1 --2- 172.21.15.163:0/2434149886 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47c8159890 0x7f47c8140dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.846+0000 7f47cd1d7700 1 -- 172.21.15.163:0/2434149886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47c815db10 msgr2=0x7f47c8154ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:19.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.846+0000 7f47cd1d7700 1 --2- 172.21.15.163:0/2434149886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47c815db10 0x7f47c8154ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:19.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.846+0000 7f47cd1d7700 1 -- 172.21.15.163:0/2434149886 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b80094f0 con 0x7f47c8159c60 2023-10-09T06:32:19.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.846+0000 7f47ce1d9700 1 --2- 172.21.15.163:0/2434149886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47c815db10 0x7f47c8154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:19.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.846+0000 7f47cd1d7700 1 --2- 172.21.15.163:0/2434149886 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47c8159c60 0x7f47c80ca0c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f47b80030c0 tx=0x7f47b8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:19.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.847+0000 7f47bdffb700 1 -- 172.21.15.163:0/2434149886 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b8011e60 con 0x7f47c8159c60 2023-10-09T06:32:19.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.847+0000 7f47bdffb700 1 -- 172.21.15.163:0/2434149886 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b8020070 con 0x7f47c8159c60 2023-10-09T06:32:19.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.847+0000 7f47cfc3c700 1 -- 172.21.15.163:0/2434149886 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c80cfcf0 con 0x7f47c8159c60 2023-10-09T06:32:19.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.847+0000 7f47bdffb700 1 -- 172.21.15.163:0/2434149886 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b801b6a0 con 0x7f47c8159c60 2023-10-09T06:32:19.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.847+0000 7f47cfc3c700 1 -- 172.21.15.163:0/2434149886 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c80c7a60 con 0x7f47c8159c60 2023-10-09T06:32:19.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.848+0000 7f47cfc3c700 1 -- 172.21.15.163:0/2434149886 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c80c81a0 con 0x7f47c8159c60 2023-10-09T06:32:19.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.848+0000 7f47bdffb700 1 -- 172.21.15.163:0/2434149886 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f47b8022030 con 0x7f47c8159c60 2023-10-09T06:32:19.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.848+0000 7f47bdffb700 1 --2- 172.21.15.163:0/2434149886 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f47ac066ff0 0x7f47ac0694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:19.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.849+0000 7f47bdffb700 1 -- 172.21.15.163:0/2434149886 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f47b80860d0 con 0x7f47c8159c60 2023-10-09T06:32:19.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.849+0000 7f47cd9d8700 1 --2- 172.21.15.163:0/2434149886 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f47ac066ff0 0x7f47ac0694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:19.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.849+0000 7f47cd9d8700 1 --2- 172.21.15.163:0/2434149886 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f47ac066ff0 0x7f47ac0694a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f47c400ab30 tx=0x7f47c4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:19.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:19.853+0000 7f47bdffb700 1 -- 172.21.15.163:0/2434149886 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f47c80c81a0 con 0x7f47c8159c60 2023-10-09T06:32:20.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.077+0000 7f47cfc3c700 1 -- 172.21.15.163:0/2434149886 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f47c80c81a0 con 0x7f47c8159c60 2023-10-09T06:32:20.078 INFO:tasks.ceph.mon.a.smithi163.stderr:2023-10-09T06:32:20.077+0000 7ff251c04700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-09T06:32:20.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.081+0000 7f47bdffb700 1 -- 172.21.15.163:0/2434149886 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) v1 ==== 161+0+0 (secure 0 0 0) 0x7f47c80c81a0 con 0x7f47c8159c60 2023-10-09T06:32:20.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.084+0000 7f47cfc3c700 1 -- 172.21.15.163:0/2434149886 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f47ac066ff0 msgr2=0x7f47ac0694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:20.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.084+0000 7f47cfc3c700 1 --2- 172.21.15.163:0/2434149886 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f47ac066ff0 0x7f47ac0694a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f47c400ab30 tx=0x7f47c4008040 comp rx=0 tx=0).stop 2023-10-09T06:32:20.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.084+0000 7f47cfc3c700 1 -- 172.21.15.163:0/2434149886 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47c8159c60 msgr2=0x7f47c80ca0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:20.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.084+0000 7f47cfc3c700 1 --2- 172.21.15.163:0/2434149886 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47c8159c60 0x7f47c80ca0c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f47b80030c0 tx=0x7f47b8016000 comp rx=0 tx=0).stop 2023-10-09T06:32:20.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.084+0000 7f47cfc3c700 1 -- 172.21.15.163:0/2434149886 shutdown_connections 2023-10-09T06:32:20.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.084+0000 7f47cfc3c700 1 --2- 172.21.15.163:0/2434149886 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f47ac066ff0 0x7f47ac0694a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:20.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.084+0000 7f47cfc3c700 1 --2- 172.21.15.163:0/2434149886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47c815db10 0x7f47c8154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:20.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.084+0000 7f47cfc3c700 1 --2- 172.21.15.163:0/2434149886 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47c8159890 0x7f47c8140dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:20.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.084+0000 7f47cfc3c700 1 --2- 172.21.15.163:0/2434149886 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47c8159c60 0x7f47c80ca0c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:20.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.084+0000 7f47cfc3c700 1 -- 172.21.15.163:0/2434149886 >> 172.21.15.163:0/2434149886 conn(0x7f47c8068e20 msgr2=0x7f47c8158b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:20.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.084+0000 7f47cfc3c700 1 -- 172.21.15.163:0/2434149886 shutdown_connections 2023-10-09T06:32:20.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.084+0000 7f47cfc3c700 1 -- 172.21.15.163:0/2434149886 wait complete. 2023-10-09T06:32:20.086 INFO:tasks.ceph.mds.b.smithi181.stderr: -1837> 2023-10-09T06:32:06.836+0000 7fc11c7c5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-09T06:32:20.086 INFO:teuthology.orchestra.run.smithi163.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-09T06:32:20.097 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-09T06:32:20.108 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-09T06:32:20.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.212+0000 7fc9edbe4700 1 Processor -- start 2023-10-09T06:32:20.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.212+0000 7fc9edbe4700 1 -- start start 2023-10-09T06:32:20.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.213+0000 7fc9edbe4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc9e8159890 0x7fc9e815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:20.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.213+0000 7fc9edbe4700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc9e8159c60 0x7fc9e815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:20.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.213+0000 7fc9edbe4700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e815a160 con 0x7fc9e815db10 2023-10-09T06:32:20.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.213+0000 7fc9edbe4700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e815df50 con 0x7fc9e8159c60 2023-10-09T06:32:20.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.213+0000 7fc9edbe4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e80d13a0 con 0x7fc9e8159890 2023-10-09T06:32:20.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.213+0000 7fc9e6ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc9e8159c60 0x7fc9e815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:20.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.213+0000 7fc9e7fff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fc9e815db10 0x7fc9e8069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53928/0 (socket says 172.21.15.163:53928) 2023-10-09T06:32:20.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.213+0000 7fc9e7fff700 1 -- 172.21.15.163:0/1339948542 learned_addr learned my addr 172.21.15.163:0/1339948542 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:20.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.214+0000 7fc9e77fe700 1 --2- 172.21.15.163:0/1339948542 >> v2:172.21.15.181:3300/0 conn(0x7fc9e8159890 0x7fc9e815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:20.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.214+0000 7fc9e6ffd700 1 -- 172.21.15.163:0/1339948542 >> v2:172.21.15.181:3300/0 conn(0x7fc9e8159890 msgr2=0x7fc9e815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:20.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.214+0000 7fc9e6ffd700 1 --2- 172.21.15.163:0/1339948542 >> v2:172.21.15.181:3300/0 conn(0x7fc9e8159890 0x7fc9e815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:20.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.214+0000 7fc9e6ffd700 1 -- 172.21.15.163:0/1339948542 >> v1:172.21.15.163:6789/0 conn(0x7fc9e815db10 legacy=0x7fc9e8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:20.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.214+0000 7fc9e6ffd700 1 -- 172.21.15.163:0/1339948542 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e815e140 con 0x7fc9e8159c60 2023-10-09T06:32:20.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.214+0000 7fc9e67fc700 1 -- 172.21.15.163:0/1339948542 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 452677214 0 0) 0x7fc9e815a160 con 0x7fc9e815db10 2023-10-09T06:32:20.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.214+0000 7fc9e77fe700 1 --2- 172.21.15.163:0/1339948542 >> v2:172.21.15.181:3300/0 conn(0x7fc9e8159890 0x7fc9e815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:32:20.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.214+0000 7fc9e6ffd700 1 --2- 172.21.15.163:0/1339948542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9e8159c60 0x7fc9e815d5d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc9e80d1090 tx=0x7fc9dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e6e007c760e3721 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:20.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.214+0000 7fc9e67fc700 1 -- 172.21.15.163:0/1339948542 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc00b6b0 con 0x7fc9e8159c60 2023-10-09T06:32:20.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.214+0000 7fc9e67fc700 1 -- 172.21.15.163:0/1339948542 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9dc00b810 con 0x7fc9e8159c60 2023-10-09T06:32:20.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.214+0000 7fc9e67fc700 1 -- 172.21.15.163:0/1339948542 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc00bbb0 con 0x7fc9e8159c60 2023-10-09T06:32:20.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.214+0000 7fc9edbe4700 1 -- 172.21.15.163:0/1339948542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9e8159c60 msgr2=0x7fc9e815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:20.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.214+0000 7fc9edbe4700 1 --2- 172.21.15.163:0/1339948542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9e8159c60 0x7fc9e815d5d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc9e80d1090 tx=0x7fc9dc0091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:20.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.214+0000 7fc9edbe4700 1 -- 172.21.15.163:0/1339948542 shutdown_connections 2023-10-09T06:32:20.218 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.214+0000 7fc9edbe4700 1 --2- 172.21.15.163:0/1339948542 >> v2:172.21.15.181:3300/0 conn(0x7fc9e8159890 0x7fc9e815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:20.218 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.214+0000 7fc9edbe4700 1 --2- 172.21.15.163:0/1339948542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9e8159c60 0x7fc9e815d5d0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:20.218 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.214+0000 7fc9edbe4700 1 -- 172.21.15.163:0/1339948542 >> 172.21.15.163:0/1339948542 conn(0x7fc9e8068e20 msgr2=0x7fc9e80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:20.218 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.214+0000 7fc9edbe4700 1 -- 172.21.15.163:0/1339948542 shutdown_connections 2023-10-09T06:32:20.218 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.214+0000 7fc9edbe4700 1 -- 172.21.15.163:0/1339948542 wait complete. 2023-10-09T06:32:20.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.216+0000 7fc9edbe4700 1 Processor -- start 2023-10-09T06:32:20.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.216+0000 7fc9edbe4700 1 -- start start 2023-10-09T06:32:20.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.216+0000 7fc9edbe4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9e8159890 0x7fc9e8143150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:20.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.216+0000 7fc9edbe4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9e8159c60 0x7fc9e80c9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:20.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.216+0000 7fc9edbe4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9e815db10 0x7fc9e80c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:20.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.216+0000 7fc9edbe4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e8154860 con 0x7fc9e8159890 2023-10-09T06:32:20.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.216+0000 7fc9edbe4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e81537c0 con 0x7fc9e815db10 2023-10-09T06:32:20.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.216+0000 7fc9edbe4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e8153aa0 con 0x7fc9e8159c60 2023-10-09T06:32:20.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.217+0000 7fc9e77fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9e8159890 0x7fc9e8143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:20.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.217+0000 7fc9e77fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9e8159890 0x7fc9e8143150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58470/0 (socket says 172.21.15.163:58470) 2023-10-09T06:32:20.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.217+0000 7fc9e77fe700 1 -- 172.21.15.163:0/2628950561 learned_addr learned my addr 172.21.15.163:0/2628950561 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:20.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.217+0000 7fc9e6ffd700 1 --2- 172.21.15.163:0/2628950561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9e8159c60 0x7fc9e80c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:20.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.217+0000 7fc9e7fff700 1 --2- 172.21.15.163:0/2628950561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9e815db10 0x7fc9e80c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:20.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.217+0000 7fc9e77fe700 1 -- 172.21.15.163:0/2628950561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9e8159c60 msgr2=0x7fc9e80c9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:20.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.217+0000 7fc9e77fe700 1 --2- 172.21.15.163:0/2628950561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9e8159c60 0x7fc9e80c9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:20.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.217+0000 7fc9e77fe700 1 -- 172.21.15.163:0/2628950561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9e815db10 msgr2=0x7fc9e80c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:20.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.217+0000 7fc9e77fe700 1 --2- 172.21.15.163:0/2628950561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9e815db10 0x7fc9e80c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:20.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.217+0000 7fc9e77fe700 1 -- 172.21.15.163:0/2628950561 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9dc009420 con 0x7fc9e8159890 2023-10-09T06:32:20.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.217+0000 7fc9e6ffd700 1 --2- 172.21.15.163:0/2628950561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9e8159c60 0x7fc9e80c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:20.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.217+0000 7fc9e7fff700 1 --2- 172.21.15.163:0/2628950561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9e815db10 0x7fc9e80c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:20.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.217+0000 7fc9e77fe700 1 --2- 172.21.15.163:0/2628950561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9e8159890 0x7fc9e8143150 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc9d800a400 tx=0x7fc9d8007910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:20.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.217+0000 7fc9cf7fe700 1 -- 172.21.15.163:0/2628950561 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d800d650 con 0x7fc9e8159890 2023-10-09T06:32:20.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.217+0000 7fc9cf7fe700 1 -- 172.21.15.163:0/2628950561 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d800d7b0 con 0x7fc9e8159890 2023-10-09T06:32:20.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.217+0000 7fc9edbe4700 1 -- 172.21.15.163:0/2628950561 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e8153d80 con 0x7fc9e8159890 2023-10-09T06:32:20.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.217+0000 7fc9cf7fe700 1 -- 172.21.15.163:0/2628950561 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d800db50 con 0x7fc9e8159890 2023-10-09T06:32:20.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.217+0000 7fc9edbe4700 1 -- 172.21.15.163:0/2628950561 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e8154340 con 0x7fc9e8159890 2023-10-09T06:32:20.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.218+0000 7fc9edbe4700 1 -- 172.21.15.163:0/2628950561 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e804c3c0 con 0x7fc9e8159890 2023-10-09T06:32:20.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.218+0000 7fc9cf7fe700 1 -- 172.21.15.163:0/2628950561 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fc9d8015440 con 0x7fc9e8159890 2023-10-09T06:32:20.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.219+0000 7fc9cf7fe700 1 --2- 172.21.15.163:0/2628950561 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc9d4066fc0 0x7fc9d4069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:20.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.219+0000 7fc9cf7fe700 1 -- 172.21.15.163:0/2628950561 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fc9d8084e50 con 0x7fc9e8159890 2023-10-09T06:32:20.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.219+0000 7fc9e6ffd700 1 --2- 172.21.15.163:0/2628950561 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc9d4066fc0 0x7fc9d4069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:20.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.220+0000 7fc9e6ffd700 1 --2- 172.21.15.163:0/2628950561 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc9d4066fc0 0x7fc9d4069470 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc0093f0 tx=0x7fc9dc017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:20.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.223+0000 7fc9cf7fe700 1 -- 172.21.15.163:0/2628950561 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc9d804e5a0 con 0x7fc9e8159890 2023-10-09T06:32:20.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:20.450+0000 7fc9edbe4700 1 -- 172.21.15.163:0/2628950561 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc9e806a5b0 con 0x7fc9e8159890 2023-10-09T06:32:21.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.084+0000 7fc9cf7fe700 1 -- 172.21.15.163:0/2628950561 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc9d8052130 con 0x7fc9e8159890 2023-10-09T06:32:21.090 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.087+0000 7fc9edbe4700 1 -- 172.21.15.163:0/2628950561 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc9d4066fc0 msgr2=0x7fc9d4069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:21.090 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.087+0000 7fc9edbe4700 1 --2- 172.21.15.163:0/2628950561 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc9d4066fc0 0x7fc9d4069470 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc0093f0 tx=0x7fc9dc017000 comp rx=0 tx=0).stop 2023-10-09T06:32:21.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.087+0000 7fc9edbe4700 1 -- 172.21.15.163:0/2628950561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9e8159890 msgr2=0x7fc9e8143150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:21.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.087+0000 7fc9edbe4700 1 --2- 172.21.15.163:0/2628950561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9e8159890 0x7fc9e8143150 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc9d800a400 tx=0x7fc9d8007910 comp rx=0 tx=0).stop 2023-10-09T06:32:21.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.087+0000 7fc9edbe4700 1 -- 172.21.15.163:0/2628950561 shutdown_connections 2023-10-09T06:32:21.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.087+0000 7fc9edbe4700 1 --2- 172.21.15.163:0/2628950561 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc9d4066fc0 0x7fc9d4069470 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:21.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.087+0000 7fc9edbe4700 1 --2- 172.21.15.163:0/2628950561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9e8159890 0x7fc9e8143150 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:21.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.087+0000 7fc9edbe4700 1 --2- 172.21.15.163:0/2628950561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9e815db10 0x7fc9e80c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:21.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.087+0000 7fc9edbe4700 1 --2- 172.21.15.163:0/2628950561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9e8159c60 0x7fc9e80c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:21.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.087+0000 7fc9edbe4700 1 -- 172.21.15.163:0/2628950561 >> 172.21.15.163:0/2628950561 conn(0x7fc9e8068e20 msgr2=0x7fc9e8157230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:21.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.087+0000 7fc9edbe4700 1 -- 172.21.15.163:0/2628950561 shutdown_connections 2023-10-09T06:32:21.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.087+0000 7fc9edbe4700 1 -- 172.21.15.163:0/2628950561 wait complete. 2023-10-09T06:32:21.100 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-09T06:32:21.218 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.217+0000 7f55882af700 1 Processor -- start 2023-10-09T06:32:21.218 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.217+0000 7f55882af700 1 -- start start 2023-10-09T06:32:21.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.218+0000 7f55882af700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55800cd910 0x7f55800ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:21.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.218+0000 7f55882af700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f55800b55a0 0x7f55800ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:21.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.218+0000 7f55882af700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55800b5b00 con 0x7f5580159ec0 2023-10-09T06:32:21.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.218+0000 7f55882af700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55800b5da0 con 0x7f55800b55a0 2023-10-09T06:32:21.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.218+0000 7f55882af700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55800b6080 con 0x7f55800cd910 2023-10-09T06:32:21.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.218+0000 7f558604b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f5580159ec0 0x7f55800b4e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53940/0 (socket says 172.21.15.163:53940) 2023-10-09T06:32:21.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.218+0000 7f558604b700 1 -- 172.21.15.163:0/3178341389 learned_addr learned my addr 172.21.15.163:0/3178341389 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:21.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.218+0000 7f558684c700 1 --2- 172.21.15.163:0/3178341389 >> v2:172.21.15.163:3300/0 conn(0x7f55800b55a0 0x7f55800ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:21.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.218+0000 7f558584a700 1 --2- 172.21.15.163:0/3178341389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55800cd910 0x7f55800ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:21.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.218+0000 7f5585049700 1 -- 172.21.15.163:0/3178341389 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2995175195 0 0) 0x7f55800b5b00 con 0x7f5580159ec0 2023-10-09T06:32:21.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.218+0000 7f5585049700 1 -- 172.21.15.163:0/3178341389 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f556c003570 con 0x7f5580159ec0 2023-10-09T06:32:21.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.219+0000 7f558684c700 1 -- 172.21.15.163:0/3178341389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55800cd910 msgr2=0x7f55800ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:21.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.219+0000 7f558684c700 1 --2- 172.21.15.163:0/3178341389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55800cd910 0x7f55800ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:21.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.219+0000 7f558684c700 1 -- 172.21.15.163:0/3178341389 >> v1:172.21.15.163:6789/0 conn(0x7f5580159ec0 legacy=0x7f55800b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:21.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.219+0000 7f558684c700 1 -- 172.21.15.163:0/3178341389 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55800cc100 con 0x7f55800b55a0 2023-10-09T06:32:21.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.219+0000 7f558684c700 1 --2- 172.21.15.163:0/3178341389 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55800b55a0 0x7f55800ce880 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f55780098e0 tx=0x7f55780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c408070045ea5f91 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:21.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.219+0000 7f5585049700 1 -- 172.21.15.163:0/3178341389 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5578011630 con 0x7f55800b55a0 2023-10-09T06:32:21.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.219+0000 7f5585049700 1 -- 172.21.15.163:0/3178341389 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5578011790 con 0x7f55800b55a0 2023-10-09T06:32:21.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.219+0000 7f5585049700 1 -- 172.21.15.163:0/3178341389 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5578011b10 con 0x7f55800b55a0 2023-10-09T06:32:21.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.220+0000 7f55882af700 1 -- 172.21.15.163:0/3178341389 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55800b55a0 msgr2=0x7f55800ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:21.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.220+0000 7f55882af700 1 --2- 172.21.15.163:0/3178341389 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55800b55a0 0x7f55800ce880 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f55780098e0 tx=0x7f55780091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:21.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.220+0000 7f55882af700 1 -- 172.21.15.163:0/3178341389 shutdown_connections 2023-10-09T06:32:21.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.220+0000 7f55882af700 1 --2- 172.21.15.163:0/3178341389 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55800b55a0 0x7f55800ce880 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:21.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.220+0000 7f55882af700 1 --2- 172.21.15.163:0/3178341389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55800cd910 0x7f55800ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:21.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.220+0000 7f55882af700 1 -- 172.21.15.163:0/3178341389 >> 172.21.15.163:0/3178341389 conn(0x7f55800999f0 msgr2=0x7f55800b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:21.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.220+0000 7f55882af700 1 -- 172.21.15.163:0/3178341389 shutdown_connections 2023-10-09T06:32:21.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.220+0000 7f55882af700 1 -- 172.21.15.163:0/3178341389 wait complete. 2023-10-09T06:32:21.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.221+0000 7f55882af700 1 Processor -- start 2023-10-09T06:32:21.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.221+0000 7f55882af700 1 -- start start 2023-10-09T06:32:21.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.221+0000 7f55882af700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55800b55a0 0x7f5580156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:21.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.221+0000 7f55882af700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5580159ec0 0x7f55800d09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:21.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.221+0000 7f55882af700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5580153a20 0x7f55800d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:21.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.221+0000 7f55882af700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5580155ca0 con 0x7f5580153a20 2023-10-09T06:32:21.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.221+0000 7f55882af700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5580156f60 con 0x7f55800b55a0 2023-10-09T06:32:21.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.221+0000 7f55882af700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f558016e150 con 0x7f5580159ec0 2023-10-09T06:32:21.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.222+0000 7f558684c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5580153a20 0x7f55800d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:21.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.222+0000 7f558684c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5580153a20 0x7f55800d0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58486/0 (socket says 172.21.15.163:58486) 2023-10-09T06:32:21.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.222+0000 7f558684c700 1 -- 172.21.15.163:0/2703108861 learned_addr learned my addr 172.21.15.163:0/2703108861 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:21.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.222+0000 7f558584a700 1 --2- 172.21.15.163:0/2703108861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5580159ec0 0x7f55800d09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:21.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.222+0000 7f558604b700 1 --2- 172.21.15.163:0/2703108861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55800b55a0 0x7f5580156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:21.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.222+0000 7f558684c700 1 -- 172.21.15.163:0/2703108861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5580159ec0 msgr2=0x7f55800d09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:21.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.222+0000 7f558684c700 1 --2- 172.21.15.163:0/2703108861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5580159ec0 0x7f55800d09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:21.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.222+0000 7f558684c700 1 -- 172.21.15.163:0/2703108861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55800b55a0 msgr2=0x7f5580156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:21.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.222+0000 7f558684c700 1 --2- 172.21.15.163:0/2703108861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55800b55a0 0x7f5580156880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:21.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.222+0000 7f558684c700 1 -- 172.21.15.163:0/2703108861 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5578009420 con 0x7f5580153a20 2023-10-09T06:32:21.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.222+0000 7f558684c700 1 --2- 172.21.15.163:0/2703108861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5580153a20 0x7f55800d0f00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5580089390 tx=0x7f5578017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:21.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.222+0000 7f55767fc700 1 -- 172.21.15.163:0/2703108861 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5578011b10 con 0x7f5580153a20 2023-10-09T06:32:21.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.222+0000 7f55767fc700 1 -- 172.21.15.163:0/2703108861 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5578027450 con 0x7f5580153a20 2023-10-09T06:32:21.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.222+0000 7f55882af700 1 -- 172.21.15.163:0/2703108861 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f558016d390 con 0x7f5580153a20 2023-10-09T06:32:21.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.222+0000 7f55767fc700 1 -- 172.21.15.163:0/2703108861 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5578027810 con 0x7f5580153a20 2023-10-09T06:32:21.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.222+0000 7f55882af700 1 -- 172.21.15.163:0/2703108861 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f558016d950 con 0x7f5580153a20 2023-10-09T06:32:21.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.223+0000 7f55882af700 1 -- 172.21.15.163:0/2703108861 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5580063c90 con 0x7f5580153a20 2023-10-09T06:32:21.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.224+0000 7f55767fc700 1 -- 172.21.15.163:0/2703108861 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f557801a430 con 0x7f5580153a20 2023-10-09T06:32:21.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.224+0000 7f55767fc700 1 --2- 172.21.15.163:0/2703108861 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5564066fc0 0x7f5564069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:21.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.224+0000 7f55767fc700 1 -- 172.21.15.163:0/2703108861 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f5578086510 con 0x7f5580153a20 2023-10-09T06:32:21.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.224+0000 7f558604b700 1 --2- 172.21.15.163:0/2703108861 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5564066fc0 0x7f5564069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:21.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.225+0000 7f558604b700 1 --2- 172.21.15.163:0/2703108861 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5564066fc0 0x7f5564069470 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f557c000f80 tx=0x7f557c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:21.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.228+0000 7f55767fc700 1 -- 172.21.15.163:0/2703108861 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55780537f0 con 0x7f5580153a20 2023-10-09T06:32:21.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:21.416+0000 7f55882af700 1 -- 172.21.15.163:0/2703108861 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f55800718b0 con 0x7f5580153a20 2023-10-09T06:32:22.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.091+0000 7f55767fc700 1 -- 172.21.15.163:0/2703108861 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v53) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5578056e10 con 0x7f5580153a20 2023-10-09T06:32:22.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.095+0000 7f55882af700 1 -- 172.21.15.163:0/2703108861 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5564066fc0 msgr2=0x7f5564069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:22.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.095+0000 7f55882af700 1 --2- 172.21.15.163:0/2703108861 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5564066fc0 0x7f5564069470 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f557c000f80 tx=0x7f557c009040 comp rx=0 tx=0).stop 2023-10-09T06:32:22.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.095+0000 7f55882af700 1 -- 172.21.15.163:0/2703108861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5580153a20 msgr2=0x7f55800d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:22.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.095+0000 7f55882af700 1 --2- 172.21.15.163:0/2703108861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5580153a20 0x7f55800d0f00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5580089390 tx=0x7f5578017000 comp rx=0 tx=0).stop 2023-10-09T06:32:22.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.095+0000 7f55882af700 1 -- 172.21.15.163:0/2703108861 shutdown_connections 2023-10-09T06:32:22.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.095+0000 7f55882af700 1 --2- 172.21.15.163:0/2703108861 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5564066fc0 0x7f5564069470 secure :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f557c000f80 tx=0x7f557c009040 comp rx=0 tx=0).stop 2023-10-09T06:32:22.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.095+0000 7f55882af700 1 --2- 172.21.15.163:0/2703108861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5580153a20 0x7f55800d0f00 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:22.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.095+0000 7f55882af700 1 --2- 172.21.15.163:0/2703108861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55800b55a0 0x7f5580156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:22.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.095+0000 7f55882af700 1 --2- 172.21.15.163:0/2703108861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5580159ec0 0x7f55800d09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:22.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.095+0000 7f55882af700 1 -- 172.21.15.163:0/2703108861 >> 172.21.15.163:0/2703108861 conn(0x7f55800999f0 msgr2=0x7f55800c98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:22.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.095+0000 7f55882af700 1 -- 172.21.15.163:0/2703108861 shutdown_connections 2023-10-09T06:32:22.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.095+0000 7f55882af700 1 -- 172.21.15.163:0/2703108861 wait complete. 2023-10-09T06:32:22.097 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_metadata' removed 2023-10-09T06:32:22.110 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-09T06:32:22.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.230+0000 7fa6083d2700 1 Processor -- start 2023-10-09T06:32:22.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.230+0000 7fa6083d2700 1 -- start start 2023-10-09T06:32:22.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.231+0000 7fa6083d2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa600159ec0 0x7fa6000b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:22.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.231+0000 7fa6083d2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6000b53d0 0x7fa6000cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:22.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.231+0000 7fa6083d2700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6000b5b00 con 0x7fa6000cce40 2023-10-09T06:32:22.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.231+0000 7fa6083d2700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6000b5850 con 0x7fa600159ec0 2023-10-09T06:32:22.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.231+0000 7fa6083d2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6000b5da0 con 0x7fa6000b53d0 2023-10-09T06:32:22.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.231+0000 7fa60616e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa600159ec0 0x7fa6000b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:22.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.231+0000 7fa60696f700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fa6000cce40 0x7fa6000cd2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53950/0 (socket says 172.21.15.163:53950) 2023-10-09T06:32:22.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.231+0000 7fa60696f700 1 -- 172.21.15.163:0/3218420265 learned_addr learned my addr 172.21.15.163:0/3218420265 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:22.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.232+0000 7fa60596d700 1 --2- 172.21.15.163:0/3218420265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6000b53d0 0x7fa6000cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:22.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.232+0000 7fa60616e700 1 -- 172.21.15.163:0/3218420265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6000b53d0 msgr2=0x7fa6000cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:22.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.232+0000 7fa60616e700 1 --2- 172.21.15.163:0/3218420265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6000b53d0 0x7fa6000cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:22.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.232+0000 7fa60616e700 1 -- 172.21.15.163:0/3218420265 >> v1:172.21.15.163:6789/0 conn(0x7fa6000cce40 legacy=0x7fa6000cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:22.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.232+0000 7fa60516c700 1 -- 172.21.15.163:0/3218420265 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1880973906 0 0) 0x7fa6000b5b00 con 0x7fa6000cce40 2023-10-09T06:32:22.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.232+0000 7fa60616e700 1 -- 172.21.15.163:0/3218420265 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6000b6080 con 0x7fa600159ec0 2023-10-09T06:32:22.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.232+0000 7fa60616e700 1 --2- 172.21.15.163:0/3218420265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa600159ec0 0x7fa6000b4e90 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc009eb0 tx=0x7fa5fc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77221d36e50d197 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:22.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.232+0000 7fa60516c700 1 -- 172.21.15.163:0/3218420265 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc00e070 con 0x7fa600159ec0 2023-10-09T06:32:22.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.232+0000 7fa60516c700 1 -- 172.21.15.163:0/3218420265 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5fc00bc30 con 0x7fa600159ec0 2023-10-09T06:32:22.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.232+0000 7fa60516c700 1 -- 172.21.15.163:0/3218420265 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc012430 con 0x7fa600159ec0 2023-10-09T06:32:22.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.232+0000 7fa6083d2700 1 -- 172.21.15.163:0/3218420265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa600159ec0 msgr2=0x7fa6000b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:22.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.232+0000 7fa6083d2700 1 --2- 172.21.15.163:0/3218420265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa600159ec0 0x7fa6000b4e90 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc009eb0 tx=0x7fa5fc009780 comp rx=0 tx=0).stop 2023-10-09T06:32:22.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.233+0000 7fa6083d2700 1 -- 172.21.15.163:0/3218420265 shutdown_connections 2023-10-09T06:32:22.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.233+0000 7fa6083d2700 1 --2- 172.21.15.163:0/3218420265 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa600159ec0 0x7fa6000b4e90 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:22.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.233+0000 7fa6083d2700 1 --2- 172.21.15.163:0/3218420265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6000b53d0 0x7fa6000cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:22.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.233+0000 7fa6083d2700 1 -- 172.21.15.163:0/3218420265 >> 172.21.15.163:0/3218420265 conn(0x7fa6000999f0 msgr2=0x7fa6000b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:22.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.233+0000 7fa6083d2700 1 -- 172.21.15.163:0/3218420265 shutdown_connections 2023-10-09T06:32:22.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.233+0000 7fa6083d2700 1 -- 172.21.15.163:0/3218420265 wait complete. 2023-10-09T06:32:22.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.234+0000 7fa6083d2700 1 Processor -- start 2023-10-09T06:32:22.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.234+0000 7fa6083d2700 1 -- start start 2023-10-09T06:32:22.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.234+0000 7fa6083d2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6000b53d0 0x7fa60009a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:22.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.234+0000 7fa6083d2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6000cce40 0x7fa60009ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:22.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.234+0000 7fa6083d2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa600159ec0 0x7fa600156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:22.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.234+0000 7fa6083d2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa600068da0 con 0x7fa6000b53d0 2023-10-09T06:32:22.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.234+0000 7fa6083d2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60006b1a0 con 0x7fa6000cce40 2023-10-09T06:32:22.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.234+0000 7fa6083d2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa600068ff0 con 0x7fa600159ec0 2023-10-09T06:32:22.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.234+0000 7fa60616e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6000b53d0 0x7fa60009a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:22.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.234+0000 7fa60616e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6000b53d0 0x7fa60009a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58492/0 (socket says 172.21.15.163:58492) 2023-10-09T06:32:22.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.234+0000 7fa60616e700 1 -- 172.21.15.163:0/4257795821 learned_addr learned my addr 172.21.15.163:0/4257795821 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:22.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.235+0000 7fa60696f700 1 --2- 172.21.15.163:0/4257795821 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa600159ec0 0x7fa600156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:22.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.235+0000 7fa60596d700 1 --2- 172.21.15.163:0/4257795821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6000cce40 0x7fa60009ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:22.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.235+0000 7fa60616e700 1 -- 172.21.15.163:0/4257795821 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa600159ec0 msgr2=0x7fa600156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:22.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.235+0000 7fa60616e700 1 --2- 172.21.15.163:0/4257795821 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa600159ec0 0x7fa600156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:22.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.235+0000 7fa60616e700 1 -- 172.21.15.163:0/4257795821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6000cce40 msgr2=0x7fa60009ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:22.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.235+0000 7fa60616e700 1 --2- 172.21.15.163:0/4257795821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6000cce40 0x7fa60009ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:22.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.235+0000 7fa60616e700 1 -- 172.21.15.163:0/4257795821 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5fc009a20 con 0x7fa6000b53d0 2023-10-09T06:32:22.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.235+0000 7fa60696f700 1 --2- 172.21.15.163:0/4257795821 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa600159ec0 0x7fa600156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:32:22.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.235+0000 7fa60596d700 1 --2- 172.21.15.163:0/4257795821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6000cce40 0x7fa60009ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:32:22.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.235+0000 7fa60616e700 1 --2- 172.21.15.163:0/4257795821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6000b53d0 0x7fa60009a910 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc0097c0 tx=0x7fa5fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:22.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.235+0000 7fa5f67fc700 1 -- 172.21.15.163:0/4257795821 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc012ea0 con 0x7fa6000b53d0 2023-10-09T06:32:22.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.235+0000 7fa5f67fc700 1 -- 172.21.15.163:0/4257795821 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5fc01d630 con 0x7fa6000b53d0 2023-10-09T06:32:22.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.235+0000 7fa6083d2700 1 -- 172.21.15.163:0/4257795821 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6000692d0 con 0x7fa6000b53d0 2023-10-09T06:32:22.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.235+0000 7fa5f67fc700 1 -- 172.21.15.163:0/4257795821 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc01d9f0 con 0x7fa6000b53d0 2023-10-09T06:32:22.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.235+0000 7fa6083d2700 1 -- 172.21.15.163:0/4257795821 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa60009b5c0 con 0x7fa6000b53d0 2023-10-09T06:32:22.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.236+0000 7fa6083d2700 1 -- 172.21.15.163:0/4257795821 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa60009b7b0 con 0x7fa6000b53d0 2023-10-09T06:32:22.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.236+0000 7fa5f67fc700 1 -- 172.21.15.163:0/4257795821 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fa5fc024070 con 0x7fa6000b53d0 2023-10-09T06:32:22.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.237+0000 7fa5f67fc700 1 --2- 172.21.15.163:0/4257795821 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa5e4066ff0 0x7fa5e40694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:22.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.237+0000 7fa5f67fc700 1 -- 172.21.15.163:0/4257795821 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5353+0+0 (secure 0 0 0) 0x7fa5fc086cf0 con 0x7fa6000b53d0 2023-10-09T06:32:22.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.237+0000 7fa60596d700 1 --2- 172.21.15.163:0/4257795821 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa5e4066ff0 0x7fa5e40694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:22.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.237+0000 7fa60596d700 1 --2- 172.21.15.163:0/4257795821 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa5e4066ff0 0x7fa5e40694a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0006fd0 tx=0x7fa5f0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:22.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.241+0000 7fa5f67fc700 1 -- 172.21.15.163:0/4257795821 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa60009b7b0 con 0x7fa6000b53d0 2023-10-09T06:32:22.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:22.422+0000 7fa6083d2700 1 -- 172.21.15.163:0/4257795821 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa60009b7b0 con 0x7fa6000b53d0 2023-10-09T06:32:23.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.098+0000 7fa5f67fc700 1 -- 172.21.15.163:0/4257795821 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v54) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa60009b7b0 con 0x7fa6000b53d0 2023-10-09T06:32:23.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.102+0000 7fa6083d2700 1 -- 172.21.15.163:0/4257795821 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa5e4066ff0 msgr2=0x7fa5e40694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:23.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.102+0000 7fa6083d2700 1 --2- 172.21.15.163:0/4257795821 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa5e4066ff0 0x7fa5e40694a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0006fd0 tx=0x7fa5f0008040 comp rx=0 tx=0).stop 2023-10-09T06:32:23.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.102+0000 7fa6083d2700 1 -- 172.21.15.163:0/4257795821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6000b53d0 msgr2=0x7fa60009a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:23.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.102+0000 7fa6083d2700 1 --2- 172.21.15.163:0/4257795821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6000b53d0 0x7fa60009a910 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc0097c0 tx=0x7fa5fc017000 comp rx=0 tx=0).stop 2023-10-09T06:32:23.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.102+0000 7fa6083d2700 1 -- 172.21.15.163:0/4257795821 shutdown_connections 2023-10-09T06:32:23.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.102+0000 7fa6083d2700 1 --2- 172.21.15.163:0/4257795821 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa5e4066ff0 0x7fa5e40694a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:23.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.102+0000 7fa6083d2700 1 --2- 172.21.15.163:0/4257795821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6000b53d0 0x7fa60009a910 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:23.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.102+0000 7fa6083d2700 1 --2- 172.21.15.163:0/4257795821 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa600159ec0 0x7fa600156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:23.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.102+0000 7fa6083d2700 1 --2- 172.21.15.163:0/4257795821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6000cce40 0x7fa60009ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:23.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.102+0000 7fa6083d2700 1 -- 172.21.15.163:0/4257795821 >> 172.21.15.163:0/4257795821 conn(0x7fa6000999f0 msgr2=0x7fa6000c98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:23.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.102+0000 7fa6083d2700 1 -- 172.21.15.163:0/4257795821 shutdown_connections 2023-10-09T06:32:23.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.102+0000 7fa6083d2700 1 -- 172.21.15.163:0/4257795821 wait complete. 2023-10-09T06:32:23.105 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_data' removed 2023-10-09T06:32:23.117 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-09T06:32:23.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.234+0000 7fadd2b4e700 1 Processor -- start 2023-10-09T06:32:23.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.234+0000 7fadd2b4e700 1 -- start start 2023-10-09T06:32:23.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.235+0000 7fadd2b4e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fadcc0b4e70 0x7fadcc0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:23.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.235+0000 7fadd2b4e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadcc0cfb00 con 0x7fadcc0b16d0 2023-10-09T06:32:23.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.235+0000 7fadd2b4e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadcc15e060 con 0x7fadcc0b1b40 2023-10-09T06:32:23.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.235+0000 7fadd2b4e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadcc0b5c40 con 0x7fadcc0b4e70 2023-10-09T06:32:23.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.235+0000 7fadd134b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fadcc0b16d0 0x7fadcc15d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53962/0 (socket says 172.21.15.163:53962) 2023-10-09T06:32:23.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.235+0000 7fadd134b700 1 -- 172.21.15.163:0/1202877691 learned_addr learned my addr 172.21.15.163:0/1202877691 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:23.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.235+0000 7fadd1b4c700 1 --2- 172.21.15.163:0/1202877691 >> v2:172.21.15.181:3300/0 conn(0x7fadcc0b4e70 0x7fadcc0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:23.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.235+0000 7fadd0b4a700 1 -- 172.21.15.163:0/1202877691 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2177973736 0 0) 0x7fadcc0cfb00 con 0x7fadcc0b16d0 2023-10-09T06:32:23.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.235+0000 7fadd0b4a700 1 -- 172.21.15.163:0/1202877691 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadb8003570 con 0x7fadcc0b16d0 2023-10-09T06:32:23.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.236+0000 7fadd0b4a700 1 -- 172.21.15.163:0/1202877691 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2009576613 0 0) 0x7fadb8003570 con 0x7fadcc0b16d0 2023-10-09T06:32:23.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.236+0000 7fadd0b4a700 1 -- 172.21.15.163:0/1202877691 >> v2:172.21.15.181:3300/0 conn(0x7fadcc0b4e70 msgr2=0x7fadcc0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:23.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.236+0000 7fadd0b4a700 1 --2- 172.21.15.163:0/1202877691 >> v2:172.21.15.181:3300/0 conn(0x7fadcc0b4e70 0x7fadcc0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:23.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.236+0000 7fadd0b4a700 1 -- 172.21.15.163:0/1202877691 >> v1:172.21.15.181:6789/0 conn(0x7fadcc0b1b40 legacy=0x7fadcc15d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:23.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.236+0000 7fadd0b4a700 1 -- 172.21.15.163:0/1202877691 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadcc0b5f20 con 0x7fadcc0b16d0 2023-10-09T06:32:23.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.236+0000 7fadd0b4a700 1 -- 172.21.15.163:0/1202877691 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fadbc001470 con 0x7fadcc0b16d0 2023-10-09T06:32:23.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.236+0000 7fadd0b4a700 1 -- 172.21.15.163:0/1202877691 >> v1:172.21.15.163:6789/0 conn(0x7fadcc0b16d0 legacy=0x7fadcc15d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:23.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.236+0000 7fadd0b4a700 1 --2- 172.21.15.163:0/1202877691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadb8003a60 0x7fadb8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:23.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.236+0000 7fadd0b4a700 1 --2- 172.21.15.163:0/1202877691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadb8004810 0x7fadb8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:23.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.236+0000 7fadd0b4a700 1 --2- 172.21.15.163:0/1202877691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadb80051c0 0x7fadb800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:23.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.236+0000 7fadd0b4a700 1 -- 172.21.15.163:0/1202877691 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc0b5c40 con 0x7fadb8004810 2023-10-09T06:32:23.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.236+0000 7fadd0b4a700 1 -- 172.21.15.163:0/1202877691 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb800c0e0 con 0x7fadb80051c0 2023-10-09T06:32:23.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.236+0000 7fadd0b4a700 1 -- 172.21.15.163:0/1202877691 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb800c2a0 con 0x7fadb8003a60 2023-10-09T06:32:23.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.236+0000 7fadd134b700 1 --2- 172.21.15.163:0/1202877691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadb8004810 0x7fadb8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:23.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.236+0000 7fadd234d700 1 --2- 172.21.15.163:0/1202877691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadb80051c0 0x7fadb800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:23.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.236+0000 7fadd1b4c700 1 --2- 172.21.15.163:0/1202877691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadb8003a60 0x7fadb8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:23.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.236+0000 7fadd134b700 1 -- 172.21.15.163:0/1202877691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadb8003a60 msgr2=0x7fadb8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:23.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.236+0000 7fadd134b700 1 --2- 172.21.15.163:0/1202877691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadb8003a60 0x7fadb8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:23.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.236+0000 7fadd134b700 1 -- 172.21.15.163:0/1202877691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadb80051c0 msgr2=0x7fadb800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:23.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.236+0000 7fadd134b700 1 --2- 172.21.15.163:0/1202877691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadb80051c0 0x7fadb800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:23.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.236+0000 7fadd234d700 1 -- 172.21.15.163:0/1202877691 reap_dead start 2023-10-09T06:32:23.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.236+0000 7fadd134b700 1 -- 172.21.15.163:0/1202877691 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb8003570 con 0x7fadb8004810 2023-10-09T06:32:23.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.236+0000 7fadd134b700 1 --2- 172.21.15.163:0/1202877691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadb8004810 0x7fadb8004c80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00c3c0 tx=0x7fadbc001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474582432bc14012 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:23.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.236+0000 7fadd0b4a700 1 -- 172.21.15.163:0/1202877691 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc00cc90 con 0x7fadb8004810 2023-10-09T06:32:23.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.236+0000 7fadd0b4a700 1 -- 172.21.15.163:0/1202877691 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadbc00cdf0 con 0x7fadb8004810 2023-10-09T06:32:23.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.236+0000 7fadd0b4a700 1 -- 172.21.15.163:0/1202877691 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc005b70 con 0x7fadb8004810 2023-10-09T06:32:23.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.237+0000 7fadd2b4e700 1 -- 172.21.15.163:0/1202877691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadb8004810 msgr2=0x7fadb8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:23.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.237+0000 7fadd2b4e700 1 --2- 172.21.15.163:0/1202877691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadb8004810 0x7fadb8004c80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00c3c0 tx=0x7fadbc001e70 comp rx=0 tx=0).stop 2023-10-09T06:32:23.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.237+0000 7fadd2b4e700 1 -- 172.21.15.163:0/1202877691 shutdown_connections 2023-10-09T06:32:23.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.237+0000 7fadd2b4e700 1 --2- 172.21.15.163:0/1202877691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadb8004810 0x7fadb8004c80 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:23.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.237+0000 7fadd2b4e700 1 -- 172.21.15.163:0/1202877691 >> 172.21.15.163:0/1202877691 conn(0x7fadcc0d10c0 msgr2=0x7fadcc0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:23.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.237+0000 7fadd2b4e700 1 -- 172.21.15.163:0/1202877691 shutdown_connections 2023-10-09T06:32:23.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.237+0000 7fadd2b4e700 1 -- 172.21.15.163:0/1202877691 wait complete. 2023-10-09T06:32:23.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.238+0000 7fadd2b4e700 1 Processor -- start 2023-10-09T06:32:23.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.238+0000 7fadd2b4e700 1 -- start start 2023-10-09T06:32:23.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.239+0000 7fadd2b4e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadb8004810 0x7fadcc0c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:23.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.239+0000 7fadd2b4e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadcc0c7a30 0x7fadcc169b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:23.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.239+0000 7fadd2b4e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadcc0c7ea0 0x7fadcc16a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:23.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.239+0000 7fadd2b4e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc16b800 con 0x7fadcc0c7a30 2023-10-09T06:32:23.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.239+0000 7fadd2b4e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc16aed0 con 0x7fadb8004810 2023-10-09T06:32:23.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.239+0000 7fadd2b4e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc16b1b0 con 0x7fadcc0c7ea0 2023-10-09T06:32:23.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.239+0000 7fadd134b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadcc0c7a30 0x7fadcc169b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:23.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.239+0000 7fadd134b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadcc0c7a30 0x7fadcc169b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58518/0 (socket says 172.21.15.163:58518) 2023-10-09T06:32:23.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.239+0000 7fadd134b700 1 -- 172.21.15.163:0/1579776352 learned_addr learned my addr 172.21.15.163:0/1579776352 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:23.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.239+0000 7fadd1b4c700 1 --2- 172.21.15.163:0/1579776352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadb8004810 0x7fadcc0c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:23.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.239+0000 7fadd234d700 1 --2- 172.21.15.163:0/1579776352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadcc0c7ea0 0x7fadcc16a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:23.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.239+0000 7fadd134b700 1 -- 172.21.15.163:0/1579776352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadcc0c7ea0 msgr2=0x7fadcc16a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:23.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.239+0000 7fadd134b700 1 --2- 172.21.15.163:0/1579776352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadcc0c7ea0 0x7fadcc16a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:23.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.239+0000 7fadd134b700 1 -- 172.21.15.163:0/1579776352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadb8004810 msgr2=0x7fadcc0c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:23.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.239+0000 7fadd134b700 1 --2- 172.21.15.163:0/1579776352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadb8004810 0x7fadcc0c74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:23.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.239+0000 7fadd134b700 1 -- 172.21.15.163:0/1579776352 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadcc16b490 con 0x7fadcc0c7a30 2023-10-09T06:32:23.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.239+0000 7fadd134b700 1 --2- 172.21.15.163:0/1579776352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadcc0c7a30 0x7fadcc169b00 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fadb800c0b0 tx=0x7fadbc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:23.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.240+0000 7fadc1ffb700 1 -- 172.21.15.163:0/1579776352 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc022070 con 0x7fadcc0c7a30 2023-10-09T06:32:23.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.240+0000 7fadc1ffb700 1 -- 172.21.15.163:0/1579776352 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadbc001470 con 0x7fadcc0c7a30 2023-10-09T06:32:23.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.240+0000 7fadc1ffb700 1 -- 172.21.15.163:0/1579776352 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc007320 con 0x7fadcc0c7a30 2023-10-09T06:32:23.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.240+0000 7fadd2b4e700 1 -- 172.21.15.163:0/1579776352 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadcc16a580 con 0x7fadcc0c7a30 2023-10-09T06:32:23.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.240+0000 7fadd2b4e700 1 -- 172.21.15.163:0/1579776352 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadcc16aa20 con 0x7fadcc0c7a30 2023-10-09T06:32:23.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.241+0000 7fadd2b4e700 1 -- 172.21.15.163:0/1579776352 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadcc16b970 con 0x7fadcc0c7a30 2023-10-09T06:32:23.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.241+0000 7fadc1ffb700 1 -- 172.21.15.163:0/1579776352 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fadbc007690 con 0x7fadcc0c7a30 2023-10-09T06:32:23.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.241+0000 7fadc1ffb700 1 --2- 172.21.15.163:0/1579776352 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fadb0066fc0 0x7fadb0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:23.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.241+0000 7fadc1ffb700 1 -- 172.21.15.163:0/1579776352 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fadbc0853a0 con 0x7fadcc0c7a30 2023-10-09T06:32:23.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.242+0000 7fadd1b4c700 1 --2- 172.21.15.163:0/1579776352 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fadb0066fc0 0x7fadb0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:23.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.242+0000 7fadd1b4c700 1 --2- 172.21.15.163:0/1579776352 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fadb0066fc0 0x7fadb0069470 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fadc8006f00 tx=0x7fadc8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:23.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.245+0000 7fadc1ffb700 1 -- 172.21.15.163:0/1579776352 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fadbc052920 con 0x7fadcc0c7a30 2023-10-09T06:32:23.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.442+0000 7fadd2b4e700 1 -- 172.21.15.163:0/1579776352 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fadcc0ce1c0 con 0x7fadcc0c7a30 2023-10-09T06:32:23.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.445+0000 7fadc1ffb700 1 -- 172.21.15.163:0/1579776352 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fadbc055f40 con 0x7fadcc0c7a30 2023-10-09T06:32:23.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.448+0000 7fadd2b4e700 1 -- 172.21.15.163:0/1579776352 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fadb0066fc0 msgr2=0x7fadb0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:23.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.448+0000 7fadd2b4e700 1 --2- 172.21.15.163:0/1579776352 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fadb0066fc0 0x7fadb0069470 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fadc8006f00 tx=0x7fadc8008040 comp rx=0 tx=0).stop 2023-10-09T06:32:23.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.448+0000 7fadd2b4e700 1 -- 172.21.15.163:0/1579776352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadcc0c7a30 msgr2=0x7fadcc169b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:23.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.448+0000 7fadd2b4e700 1 --2- 172.21.15.163:0/1579776352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadcc0c7a30 0x7fadcc169b00 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fadb800c0b0 tx=0x7fadbc015000 comp rx=0 tx=0).stop 2023-10-09T06:32:23.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.448+0000 7fadd2b4e700 1 -- 172.21.15.163:0/1579776352 shutdown_connections 2023-10-09T06:32:23.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.448+0000 7fadd2b4e700 1 --2- 172.21.15.163:0/1579776352 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fadb0066fc0 0x7fadb0069470 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:23.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.448+0000 7fadd2b4e700 1 --2- 172.21.15.163:0/1579776352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadcc0c7ea0 0x7fadcc16a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:23.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.448+0000 7fadd2b4e700 1 --2- 172.21.15.163:0/1579776352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadb8004810 0x7fadcc0c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:23.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.448+0000 7fadd2b4e700 1 --2- 172.21.15.163:0/1579776352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadcc0c7a30 0x7fadcc169b00 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:23.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.448+0000 7fadd2b4e700 1 -- 172.21.15.163:0/1579776352 >> 172.21.15.163:0/1579776352 conn(0x7fadcc0d10c0 msgr2=0x7fadcc158e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:23.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.448+0000 7fadd2b4e700 1 -- 172.21.15.163:0/1579776352 shutdown_connections 2023-10-09T06:32:23.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.448+0000 7fadd2b4e700 1 -- 172.21.15.163:0/1579776352 wait complete. 2023-10-09T06:32:23.461 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info' 2023-10-09T06:32:23.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.574+0000 7f7f79911700 1 Processor -- start 2023-10-09T06:32:23.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.574+0000 7f7f79911700 1 -- start start 2023-10-09T06:32:23.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.575+0000 7f7f79911700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f74159890 0x7f7f7415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:23.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.575+0000 7f7f79911700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7f74069560 0x7f7f740699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:23.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.575+0000 7f7f79911700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f7415a160 con 0x7f7f74159c60 2023-10-09T06:32:23.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.575+0000 7f7f79911700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f74069f20 con 0x7f7f74069560 2023-10-09T06:32:23.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.575+0000 7f7f79911700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f740d13a0 con 0x7f7f74159890 2023-10-09T06:32:23.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.575+0000 7f7f737fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7f74069560 0x7f7f740699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:23.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.575+0000 7f7f6a7fc700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f7f74159c60 0x7f7f7415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53978/0 (socket says 172.21.15.163:53978) 2023-10-09T06:32:23.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.575+0000 7f7f6a7fc700 1 -- 172.21.15.163:0/2226716065 learned_addr learned my addr 172.21.15.163:0/2226716065 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:23.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.575+0000 7f7f72ffd700 1 --2- 172.21.15.163:0/2226716065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f74159890 0x7f7f7415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:23.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.575+0000 7f7f737fe700 1 -- 172.21.15.163:0/2226716065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f74159890 msgr2=0x7f7f7415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:23.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.575+0000 7f7f737fe700 1 --2- 172.21.15.163:0/2226716065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f74159890 0x7f7f7415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:23.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.575+0000 7f7f737fe700 1 -- 172.21.15.163:0/2226716065 >> v1:172.21.15.163:6789/0 conn(0x7f7f74159c60 legacy=0x7f7f7415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:23.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.575+0000 7f7f727fc700 1 -- 172.21.15.163:0/2226716065 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1072298083 0 0) 0x7f7f7415a160 con 0x7f7f74159c60 2023-10-09T06:32:23.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.575+0000 7f7f737fe700 1 -- 172.21.15.163:0/2226716065 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f7415e140 con 0x7f7f74069560 2023-10-09T06:32:23.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.575+0000 7f7f72ffd700 1 --2- 172.21.15.163:0/2226716065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f74159890 0x7f7f7415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:23.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.575+0000 7f7f737fe700 1 --2- 172.21.15.163:0/2226716065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f74069560 0x7f7f740699e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009e60 tx=0x7f7f6c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9d684e11e42773 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:23.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.575+0000 7f7f727fc700 1 -- 172.21.15.163:0/2226716065 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c00e070 con 0x7f7f74069560 2023-10-09T06:32:23.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.575+0000 7f7f727fc700 1 -- 172.21.15.163:0/2226716065 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6c00bc30 con 0x7f7f74069560 2023-10-09T06:32:23.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.575+0000 7f7f727fc700 1 -- 172.21.15.163:0/2226716065 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c012430 con 0x7f7f74069560 2023-10-09T06:32:23.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.576+0000 7f7f79911700 1 -- 172.21.15.163:0/2226716065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f74069560 msgr2=0x7f7f740699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:23.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.576+0000 7f7f79911700 1 --2- 172.21.15.163:0/2226716065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f74069560 0x7f7f740699e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009e60 tx=0x7f7f6c009780 comp rx=0 tx=0).stop 2023-10-09T06:32:23.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.576+0000 7f7f79911700 1 -- 172.21.15.163:0/2226716065 shutdown_connections 2023-10-09T06:32:23.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.576+0000 7f7f79911700 1 --2- 172.21.15.163:0/2226716065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f74069560 0x7f7f740699e0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:23.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.576+0000 7f7f79911700 1 --2- 172.21.15.163:0/2226716065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f74159890 0x7f7f7415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:23.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.576+0000 7f7f79911700 1 -- 172.21.15.163:0/2226716065 >> 172.21.15.163:0/2226716065 conn(0x7f7f74068e20 msgr2=0x7f7f740d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:23.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.576+0000 7f7f79911700 1 -- 172.21.15.163:0/2226716065 shutdown_connections 2023-10-09T06:32:23.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.576+0000 7f7f79911700 1 -- 172.21.15.163:0/2226716065 wait complete. 2023-10-09T06:32:23.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.577+0000 7f7f79911700 1 Processor -- start 2023-10-09T06:32:23.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.577+0000 7f7f79911700 1 -- start start 2023-10-09T06:32:23.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.578+0000 7f7f79911700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f74069560 0x7f7f74140fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:23.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.578+0000 7f7f79911700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f74159890 0x7f7f740b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:23.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.578+0000 7f7f79911700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f74159c60 0x7f7f740b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:23.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.578+0000 7f7f79911700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f74155400 con 0x7f7f74159890 2023-10-09T06:32:23.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.578+0000 7f7f79911700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7415c1d0 con 0x7f7f74069560 2023-10-09T06:32:23.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.578+0000 7f7f79911700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f740b43e0 con 0x7f7f74159c60 2023-10-09T06:32:23.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.578+0000 7f7f6a7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f74159890 0x7f7f740b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:23.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.578+0000 7f7f6a7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f74159890 0x7f7f740b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58534/0 (socket says 172.21.15.163:58534) 2023-10-09T06:32:23.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.578+0000 7f7f6a7fc700 1 -- 172.21.15.163:0/3000253106 learned_addr learned my addr 172.21.15.163:0/3000253106 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:23.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.578+0000 7f7f737fe700 1 --2- 172.21.15.163:0/3000253106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f74159c60 0x7f7f740b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:23.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.578+0000 7f7f72ffd700 1 --2- 172.21.15.163:0/3000253106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f74069560 0x7f7f74140fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:23.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.578+0000 7f7f6a7fc700 1 -- 172.21.15.163:0/3000253106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f74159c60 msgr2=0x7f7f740b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:23.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.578+0000 7f7f6a7fc700 1 --2- 172.21.15.163:0/3000253106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f74159c60 0x7f7f740b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:23.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.578+0000 7f7f6a7fc700 1 -- 172.21.15.163:0/3000253106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f74069560 msgr2=0x7f7f74140fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:23.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.578+0000 7f7f6a7fc700 1 --2- 172.21.15.163:0/3000253106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f74069560 0x7f7f74140fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:23.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.578+0000 7f7f6a7fc700 1 -- 172.21.15.163:0/3000253106 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c009a20 con 0x7f7f74159890 2023-10-09T06:32:23.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.578+0000 7f7f6a7fc700 1 --2- 172.21.15.163:0/3000253106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f74159890 0x7f7f740b3590 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00c840 tx=0x7f7f5c007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:23.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.579+0000 7f7f6b7fe700 1 -- 172.21.15.163:0/3000253106 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c005c00 con 0x7f7f74159890 2023-10-09T06:32:23.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.579+0000 7f7f6b7fe700 1 -- 172.21.15.163:0/3000253106 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5c005d60 con 0x7f7f74159890 2023-10-09T06:32:23.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.579+0000 7f7f6b7fe700 1 -- 172.21.15.163:0/3000253106 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c0072e0 con 0x7f7f74159890 2023-10-09T06:32:23.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.579+0000 7f7f79911700 1 -- 172.21.15.163:0/3000253106 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f740b4100 con 0x7f7f74159890 2023-10-09T06:32:23.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.579+0000 7f7f79911700 1 -- 172.21.15.163:0/3000253106 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f7415be60 con 0x7f7f74159890 2023-10-09T06:32:23.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.580+0000 7f7f79911700 1 -- 172.21.15.163:0/3000253106 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f7415b9e0 con 0x7f7f74159890 2023-10-09T06:32:23.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.580+0000 7f7f6b7fe700 1 -- 172.21.15.163:0/3000253106 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f7f5c01b3b0 con 0x7f7f74159890 2023-10-09T06:32:23.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.580+0000 7f7f6b7fe700 1 --2- 172.21.15.163:0/3000253106 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7f54066ff0 0x7f7f540694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:23.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.581+0000 7f7f6b7fe700 1 -- 172.21.15.163:0/3000253106 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f7f5c0836d0 con 0x7f7f74159890 2023-10-09T06:32:23.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.581+0000 7f7f72ffd700 1 --2- 172.21.15.163:0/3000253106 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7f54066ff0 0x7f7f540694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:23.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.581+0000 7f7f72ffd700 1 --2- 172.21.15.163:0/3000253106 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7f54066ff0 0x7f7f540694a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7f64006f00 tx=0x7f7f64008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:23.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.585+0000 7f7f6b7fe700 1 -- 172.21.15.163:0/3000253106 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f5c04b5a0 con 0x7f7f74159890 2023-10-09T06:32:23.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:23.844+0000 7f7f79911700 1 -- 172.21.15.163:0/3000253106 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info"]} v 0) v1 -- 0x7f7f7415c340 con 0x7f7f74159890 2023-10-09T06:32:24.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.097+0000 7f7f6b7fe700 1 -- 172.21.15.163:0/3000253106 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info"]}]=0 v118) v1 ==== 133+0+0 (secure 0 0 0) 0x7f7f7415b9e0 con 0x7f7f74159890 2023-10-09T06:32:24.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.099+0000 7f7f79911700 1 -- 172.21.15.163:0/3000253106 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7f54066ff0 msgr2=0x7f7f540694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:24.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.099+0000 7f7f79911700 1 --2- 172.21.15.163:0/3000253106 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7f54066ff0 0x7f7f540694a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7f64006f00 tx=0x7f7f64008040 comp rx=0 tx=0).stop 2023-10-09T06:32:24.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.099+0000 7f7f79911700 1 -- 172.21.15.163:0/3000253106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f74159890 msgr2=0x7f7f740b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:24.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.099+0000 7f7f79911700 1 --2- 172.21.15.163:0/3000253106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f74159890 0x7f7f740b3590 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00c840 tx=0x7f7f5c007db0 comp rx=0 tx=0).stop 2023-10-09T06:32:24.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.099+0000 7f7f79911700 1 -- 172.21.15.163:0/3000253106 shutdown_connections 2023-10-09T06:32:24.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.099+0000 7f7f79911700 1 --2- 172.21.15.163:0/3000253106 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7f54066ff0 0x7f7f540694a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:24.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.099+0000 7f7f79911700 1 --2- 172.21.15.163:0/3000253106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f74159c60 0x7f7f740b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:24.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.099+0000 7f7f79911700 1 --2- 172.21.15.163:0/3000253106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f74069560 0x7f7f74140fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:24.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.099+0000 7f7f79911700 1 --2- 172.21.15.163:0/3000253106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f74159890 0x7f7f740b3590 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:24.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.099+0000 7f7f79911700 1 -- 172.21.15.163:0/3000253106 >> 172.21.15.163:0/3000253106 conn(0x7f7f74068e20 msgr2=0x7f7f74156e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:24.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.099+0000 7f7f79911700 1 -- 172.21.15.163:0/3000253106 shutdown_connections 2023-10-09T06:32:24.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.099+0000 7f7f79911700 1 -- 172.21.15.163:0/3000253106 wait complete. 2023-10-09T06:32:24.113 INFO:tasks.cephfs_test_runner:Tests the 'fs volume info' command ... ok 2023-10-09T06:32:24.113 INFO:tasks.cephfs_test_runner:Starting test: test_volume_info_pending_subvol_deletions (tasks.cephfs.test_volumes.TestVolumes) Tests the pending_subvolume_deletions in 'fs volume info' command 2023-10-09T06:32:24.113 INFO:tasks.cephfs_test_runner:test_volume_info_pending_subvol_deletions (tasks.cephfs.test_volumes.TestVolumes) 2023-10-09T06:32:24.113 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions' 2023-10-09T06:32:24.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.228+0000 7f2e1d2a0700 1 Processor -- start 2023-10-09T06:32:24.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.228+0000 7f2e1d2a0700 1 -- start start 2023-10-09T06:32:24.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.229+0000 7f2e1d2a0700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2e18069560 0x7f2e180699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:24.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.229+0000 7f2e1d2a0700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e18069ee0 con 0x7f2e18159890 2023-10-09T06:32:24.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.229+0000 7f2e1d2a0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e1815a160 con 0x7f2e18159c60 2023-10-09T06:32:24.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.229+0000 7f2e1d2a0700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e180d13a0 con 0x7f2e18069560 2023-10-09T06:32:24.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.229+0000 7f2e177fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2e18069560 0x7f2e180699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:24.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.229+0000 7f2e16ffd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f2e18159890 0x7f2e1815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53984/0 (socket says 172.21.15.163:53984) 2023-10-09T06:32:24.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.229+0000 7f2e16ffd700 1 -- 172.21.15.163:0/782323556 learned_addr learned my addr 172.21.15.163:0/782323556 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:24.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.229+0000 7f2e177fe700 1 -- 172.21.15.163:0/782323556 >> v1:172.21.15.181:6789/0 conn(0x7f2e18159c60 legacy=0x7f2e1815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:24.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.229+0000 7f2e177fe700 1 -- 172.21.15.163:0/782323556 >> v1:172.21.15.163:6789/0 conn(0x7f2e18159890 legacy=0x7f2e1815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:24.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.229+0000 7f2e177fe700 1 -- 172.21.15.163:0/782323556 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e1815e140 con 0x7f2e18069560 2023-10-09T06:32:24.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.229+0000 7f2e177fe700 1 --2- 172.21.15.163:0/782323556 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e18069560 0x7f2e180699a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2e10009e60 tx=0x7f2e10009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d548ec1df92779 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:24.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.230+0000 7f2e167fc700 1 -- 172.21.15.163:0/782323556 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1000e070 con 0x7f2e18069560 2023-10-09T06:32:24.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.230+0000 7f2e167fc700 1 -- 172.21.15.163:0/782323556 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1000bc30 con 0x7f2e18069560 2023-10-09T06:32:24.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.230+0000 7f2e167fc700 1 -- 172.21.15.163:0/782323556 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e10012430 con 0x7f2e18069560 2023-10-09T06:32:24.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.230+0000 7f2e1d2a0700 1 -- 172.21.15.163:0/782323556 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e18069560 msgr2=0x7f2e180699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:24.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.230+0000 7f2e1d2a0700 1 --2- 172.21.15.163:0/782323556 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e18069560 0x7f2e180699a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2e10009e60 tx=0x7f2e10009780 comp rx=0 tx=0).stop 2023-10-09T06:32:24.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.230+0000 7f2e1d2a0700 1 -- 172.21.15.163:0/782323556 shutdown_connections 2023-10-09T06:32:24.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.230+0000 7f2e1d2a0700 1 --2- 172.21.15.163:0/782323556 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e18069560 0x7f2e180699a0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:24.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.230+0000 7f2e1d2a0700 1 -- 172.21.15.163:0/782323556 >> 172.21.15.163:0/782323556 conn(0x7f2e18068e20 msgr2=0x7f2e180d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:24.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.230+0000 7f2e1d2a0700 1 -- 172.21.15.163:0/782323556 shutdown_connections 2023-10-09T06:32:24.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.230+0000 7f2e1d2a0700 1 -- 172.21.15.163:0/782323556 wait complete. 2023-10-09T06:32:24.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.231+0000 7f2e1d2a0700 1 Processor -- start 2023-10-09T06:32:24.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.231+0000 7f2e1d2a0700 1 -- start start 2023-10-09T06:32:24.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.232+0000 7f2e1d2a0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e18069560 0x7f2e180cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:24.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.232+0000 7f2e1d2a0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e18159890 0x7f2e180b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:24.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.232+0000 7f2e1d2a0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e18159c60 0x7f2e180b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:24.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.232+0000 7f2e1d2a0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e180cea60 con 0x7f2e18159c60 2023-10-09T06:32:24.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.232+0000 7f2e1d2a0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e180ce5d0 con 0x7f2e18159890 2023-10-09T06:32:24.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.232+0000 7f2e1d2a0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e180cd9c0 con 0x7f2e18069560 2023-10-09T06:32:24.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.232+0000 7f2e177fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e18159c60 0x7f2e180b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:24.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.232+0000 7f2e177fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e18159c60 0x7f2e180b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58554/0 (socket says 172.21.15.163:58554) 2023-10-09T06:32:24.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.232+0000 7f2e177fe700 1 -- 172.21.15.163:0/3847638348 learned_addr learned my addr 172.21.15.163:0/3847638348 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:24.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.233+0000 7f2e177fe700 1 -- 172.21.15.163:0/3847638348 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e18069560 msgr2=0x7f2e180cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:24.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.233+0000 7f2e177fe700 1 --2- 172.21.15.163:0/3847638348 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e18069560 0x7f2e180cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:24.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.233+0000 7f2e177fe700 1 -- 172.21.15.163:0/3847638348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e18159890 msgr2=0x7f2e180b4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:24.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.233+0000 7f2e177fe700 1 --2- 172.21.15.163:0/3847638348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e18159890 0x7f2e180b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:24.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.233+0000 7f2e177fe700 1 -- 172.21.15.163:0/3847638348 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e10009a20 con 0x7f2e18159c60 2023-10-09T06:32:24.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.233+0000 7f2e177fe700 1 --2- 172.21.15.163:0/3847638348 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e18159c60 0x7f2e180b51b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2e100097c0 tx=0x7f2e10019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:24.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.233+0000 7f2e0f7fe700 1 -- 172.21.15.163:0/3847638348 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1001e630 con 0x7f2e18159c60 2023-10-09T06:32:24.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.233+0000 7f2e0f7fe700 1 -- 172.21.15.163:0/3847638348 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1001e7b0 con 0x7f2e18159c60 2023-10-09T06:32:24.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.233+0000 7f2e1d2a0700 1 -- 172.21.15.163:0/3847638348 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e180cdca0 con 0x7f2e18159c60 2023-10-09T06:32:24.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.233+0000 7f2e0f7fe700 1 -- 172.21.15.163:0/3847638348 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1001eb70 con 0x7f2e18159c60 2023-10-09T06:32:24.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.233+0000 7f2e1d2a0700 1 -- 172.21.15.163:0/3847638348 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e180ce260 con 0x7f2e18159c60 2023-10-09T06:32:24.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.234+0000 7f2e1d2a0700 1 -- 172.21.15.163:0/3847638348 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e180ce8b0 con 0x7f2e18159c60 2023-10-09T06:32:24.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.234+0000 7f2e0f7fe700 1 -- 172.21.15.163:0/3847638348 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f2e10024070 con 0x7f2e18159c60 2023-10-09T06:32:24.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.235+0000 7f2e0f7fe700 1 --2- 172.21.15.163:0/3847638348 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2df8067010 0x7f2df80694c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:24.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.235+0000 7f2e0f7fe700 1 -- 172.21.15.163:0/3847638348 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2e10086d90 con 0x7f2e18159c60 2023-10-09T06:32:24.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.235+0000 7f2e16ffd700 1 --2- 172.21.15.163:0/3847638348 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2df8067010 0x7f2df80694c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:24.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.235+0000 7f2e16ffd700 1 --2- 172.21.15.163:0/3847638348 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2df8067010 0x7f2df80694c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2e080016a0 tx=0x7f2e08007b30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:24.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.239+0000 7f2e0f7fe700 1 -- 172.21.15.163:0/3847638348 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e10054210 con 0x7f2e18159c60 2023-10-09T06:32:24.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:24.501+0000 7f2e1d2a0700 1 -- 172.21.15.163:0/3847638348 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions"]} v 0) v1 -- 0x7f2e1804c3c0 con 0x7f2e18159c60 2023-10-09T06:32:25.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.100+0000 7f2e0f7fe700 1 -- 172.21.15.163:0/3847638348 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions"]}]=0 v119) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2e1002c0e0 con 0x7f2e18159c60 2023-10-09T06:32:25.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.103+0000 7f2e1d2a0700 1 -- 172.21.15.163:0/3847638348 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2df8067010 msgr2=0x7f2df80694c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:25.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.103+0000 7f2e1d2a0700 1 --2- 172.21.15.163:0/3847638348 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2df8067010 0x7f2df80694c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2e080016a0 tx=0x7f2e08007b30 comp rx=0 tx=0).stop 2023-10-09T06:32:25.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.103+0000 7f2e1d2a0700 1 -- 172.21.15.163:0/3847638348 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e18159c60 msgr2=0x7f2e180b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:25.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.103+0000 7f2e1d2a0700 1 --2- 172.21.15.163:0/3847638348 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e18159c60 0x7f2e180b51b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2e100097c0 tx=0x7f2e10019000 comp rx=0 tx=0).stop 2023-10-09T06:32:25.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.103+0000 7f2e1d2a0700 1 -- 172.21.15.163:0/3847638348 shutdown_connections 2023-10-09T06:32:25.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.103+0000 7f2e1d2a0700 1 --2- 172.21.15.163:0/3847638348 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2df8067010 0x7f2df80694c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:25.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.103+0000 7f2e1d2a0700 1 --2- 172.21.15.163:0/3847638348 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e18159c60 0x7f2e180b51b0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:25.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.103+0000 7f2e1d2a0700 1 --2- 172.21.15.163:0/3847638348 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e18069560 0x7f2e180cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:25.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.103+0000 7f2e1d2a0700 1 --2- 172.21.15.163:0/3847638348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e18159890 0x7f2e180b4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:25.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.103+0000 7f2e1d2a0700 1 -- 172.21.15.163:0/3847638348 >> 172.21.15.163:0/3847638348 conn(0x7f2e18068e20 msgr2=0x7f2e18153180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:25.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.103+0000 7f2e1d2a0700 1 -- 172.21.15.163:0/3847638348 shutdown_connections 2023-10-09T06:32:25.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.103+0000 7f2e1d2a0700 1 -- 172.21.15.163:0/3847638348 wait complete. 2023-10-09T06:32:25.117 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-09T06:32:25.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.236+0000 7f82910e8700 1 Processor -- start 2023-10-09T06:32:25.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.236+0000 7f82910e8700 1 -- start start 2023-10-09T06:32:25.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.237+0000 7f82910e8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f828c0b4e70 0x7f828c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:25.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.237+0000 7f82910e8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f828c0b16d0 0x7f828c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:25.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.237+0000 7f82910e8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f828c0b2080 con 0x7f828c15d240 2023-10-09T06:32:25.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.237+0000 7f82910e8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f828c0cfb00 con 0x7f828c0b4e70 2023-10-09T06:32:25.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.237+0000 7f82910e8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f828c0b5c40 con 0x7f828c0b16d0 2023-10-09T06:32:25.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.237+0000 7f828bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f828c0b4e70 0x7f828c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:25.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.237+0000 7f828b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f828c0b16d0 0x7f828c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:25.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.237+0000 7f828bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f828c0b4e70 0x7f828c0ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:42738/0 (socket says 172.21.15.163:42738) 2023-10-09T06:32:25.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.237+0000 7f828bfff700 1 -- 172.21.15.163:0/3781710630 learned_addr learned my addr 172.21.15.163:0/3781710630 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:25.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.237+0000 7f82908e7700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f828c15d240 0x7f828c15d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:37376/0 (socket says 172.21.15.163:37376) 2023-10-09T06:32:25.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.238+0000 7f828bfff700 1 -- 172.21.15.163:0/3781710630 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f828c0b16d0 msgr2=0x7f828c0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:25.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.238+0000 7f828bfff700 1 --2- 172.21.15.163:0/3781710630 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f828c0b16d0 0x7f828c0b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:25.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.238+0000 7f828bfff700 1 -- 172.21.15.163:0/3781710630 >> v1:172.21.15.181:6789/0 conn(0x7f828c15d240 legacy=0x7f828c15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:25.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.238+0000 7f828bfff700 1 -- 172.21.15.163:0/3781710630 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828c0b5f20 con 0x7f828c0b4e70 2023-10-09T06:32:25.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.238+0000 7f828affd700 1 -- 172.21.15.163:0/3781710630 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3570382896 0 0) 0x7f828c0b2080 con 0x7f828c15d240 2023-10-09T06:32:25.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.238+0000 7f828b7fe700 1 --2- 172.21.15.163:0/3781710630 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f828c0b16d0 0x7f828c0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:32:25.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.238+0000 7f828bfff700 1 --2- 172.21.15.163:0/3781710630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f828c0b4e70 0x7f828c0ccff0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f827c0099b0 tx=0x7f827c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fe65ee31b27413b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:25.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.238+0000 7f828affd700 1 -- 172.21.15.163:0/3781710630 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c009ea0 con 0x7f828c0b4e70 2023-10-09T06:32:25.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.238+0000 7f828affd700 1 -- 172.21.15.163:0/3781710630 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827c00b860 con 0x7f828c0b4e70 2023-10-09T06:32:25.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.238+0000 7f828affd700 1 -- 172.21.15.163:0/3781710630 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c00bc00 con 0x7f828c0b4e70 2023-10-09T06:32:25.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.238+0000 7f82910e8700 1 -- 172.21.15.163:0/3781710630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f828c0b4e70 msgr2=0x7f828c0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:25.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.238+0000 7f82910e8700 1 --2- 172.21.15.163:0/3781710630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f828c0b4e70 0x7f828c0ccff0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f827c0099b0 tx=0x7f827c009250 comp rx=0 tx=0).stop 2023-10-09T06:32:25.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.239+0000 7f82910e8700 1 -- 172.21.15.163:0/3781710630 shutdown_connections 2023-10-09T06:32:25.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.239+0000 7f82910e8700 1 --2- 172.21.15.163:0/3781710630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f828c0b4e70 0x7f828c0ccff0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:25.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.239+0000 7f82910e8700 1 --2- 172.21.15.163:0/3781710630 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f828c0b16d0 0x7f828c0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:25.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.239+0000 7f82910e8700 1 -- 172.21.15.163:0/3781710630 >> 172.21.15.163:0/3781710630 conn(0x7f828c0d10c0 msgr2=0x7f828c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:25.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.239+0000 7f82910e8700 1 -- 172.21.15.163:0/3781710630 shutdown_connections 2023-10-09T06:32:25.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.239+0000 7f82910e8700 1 -- 172.21.15.163:0/3781710630 wait complete. 2023-10-09T06:32:25.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.240+0000 7f82910e8700 1 Processor -- start 2023-10-09T06:32:25.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.240+0000 7f82910e8700 1 -- start start 2023-10-09T06:32:25.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.240+0000 7f82910e8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f828c0b16d0 0x7f828c0a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:25.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.240+0000 7f82910e8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f828c0b4e70 0x7f828c0ca980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:25.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.240+0000 7f82910e8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f828c15d240 0x7f828c1698e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:25.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.240+0000 7f82910e8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c0c9690 con 0x7f828c15d240 2023-10-09T06:32:25.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.240+0000 7f82910e8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c0af8f0 con 0x7f828c0b16d0 2023-10-09T06:32:25.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.241+0000 7f82910e8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f828c0c98e0 con 0x7f828c0b4e70 2023-10-09T06:32:25.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.241+0000 7f82908e7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f828c15d240 0x7f828c1698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:25.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.241+0000 7f82908e7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f828c15d240 0x7f828c1698e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58564/0 (socket says 172.21.15.163:58564) 2023-10-09T06:32:25.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.241+0000 7f82908e7700 1 -- 172.21.15.163:0/1541621704 learned_addr learned my addr 172.21.15.163:0/1541621704 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:25.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.241+0000 7f828bfff700 1 --2- 172.21.15.163:0/1541621704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f828c0b16d0 0x7f828c0a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:25.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.241+0000 7f828b7fe700 1 --2- 172.21.15.163:0/1541621704 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f828c0b4e70 0x7f828c0ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:25.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.241+0000 7f82908e7700 1 -- 172.21.15.163:0/1541621704 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f828c0b4e70 msgr2=0x7f828c0ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:25.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.241+0000 7f82908e7700 1 --2- 172.21.15.163:0/1541621704 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f828c0b4e70 0x7f828c0ca980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:25.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.241+0000 7f82908e7700 1 -- 172.21.15.163:0/1541621704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f828c0b16d0 msgr2=0x7f828c0a6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:25.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.241+0000 7f82908e7700 1 --2- 172.21.15.163:0/1541621704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f828c0b16d0 0x7f828c0a6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:25.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.241+0000 7f82908e7700 1 -- 172.21.15.163:0/1541621704 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8280009420 con 0x7f828c15d240 2023-10-09T06:32:25.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.241+0000 7f828bfff700 1 --2- 172.21.15.163:0/1541621704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f828c0b16d0 0x7f828c0a6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:32:25.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.241+0000 7f82908e7700 1 --2- 172.21.15.163:0/1541621704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f828c15d240 0x7f828c1698e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f828400ccc0 tx=0x7f8284001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:25.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.241+0000 7f8273fff700 1 -- 172.21.15.163:0/1541621704 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8284005df0 con 0x7f828c15d240 2023-10-09T06:32:25.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.241+0000 7f8273fff700 1 -- 172.21.15.163:0/1541621704 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828400a600 con 0x7f828c15d240 2023-10-09T06:32:25.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.241+0000 7f82910e8700 1 -- 172.21.15.163:0/1541621704 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f827c0094f0 con 0x7f828c15d240 2023-10-09T06:32:25.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.241+0000 7f8273fff700 1 -- 172.21.15.163:0/1541621704 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828400a9c0 con 0x7f828c15d240 2023-10-09T06:32:25.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.241+0000 7f82910e8700 1 -- 172.21.15.163:0/1541621704 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828c0c9ea0 con 0x7f828c15d240 2023-10-09T06:32:25.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.242+0000 7f82910e8700 1 -- 172.21.15.163:0/1541621704 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f828c088990 con 0x7f828c15d240 2023-10-09T06:32:25.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.243+0000 7f8273fff700 1 -- 172.21.15.163:0/1541621704 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f82840070f0 con 0x7f828c15d240 2023-10-09T06:32:25.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.243+0000 7f8273fff700 1 --2- 172.21.15.163:0/1541621704 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8278066fc0 0x7f8278069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:25.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.243+0000 7f8273fff700 1 -- 172.21.15.163:0/1541621704 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f8284084280 con 0x7f828c15d240 2023-10-09T06:32:25.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.243+0000 7f828bfff700 1 --2- 172.21.15.163:0/1541621704 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8278066fc0 0x7f8278069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:25.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.244+0000 7f828bfff700 1 --2- 172.21.15.163:0/1541621704 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8278066fc0 0x7f8278069470 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f827c0099b0 tx=0x7f827c017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:25.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.247+0000 7f8273fff700 1 -- 172.21.15.163:0/1541621704 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8284051930 con 0x7f828c15d240 2023-10-09T06:32:25.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.431+0000 7f82910e8700 1 -- 172.21.15.163:0/1541621704 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f828c04bb80 con 0x7f828c15d240 2023-10-09T06:32:25.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.435+0000 7f8273fff700 1 -- 172.21.15.163:0/1541621704 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8284054f50 con 0x7f828c15d240 2023-10-09T06:32:25.438 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.437+0000 7f82910e8700 1 -- 172.21.15.163:0/1541621704 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8278066fc0 msgr2=0x7f8278069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:25.438 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.437+0000 7f82910e8700 1 --2- 172.21.15.163:0/1541621704 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8278066fc0 0x7f8278069470 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f827c0099b0 tx=0x7f827c017000 comp rx=0 tx=0).stop 2023-10-09T06:32:25.438 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.437+0000 7f82910e8700 1 -- 172.21.15.163:0/1541621704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f828c15d240 msgr2=0x7f828c1698e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:25.438 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.437+0000 7f82910e8700 1 --2- 172.21.15.163:0/1541621704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f828c15d240 0x7f828c1698e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f828400ccc0 tx=0x7f8284001a70 comp rx=0 tx=0).stop 2023-10-09T06:32:25.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.437+0000 7f82910e8700 1 -- 172.21.15.163:0/1541621704 shutdown_connections 2023-10-09T06:32:25.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.437+0000 7f82910e8700 1 --2- 172.21.15.163:0/1541621704 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8278066fc0 0x7f8278069470 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:25.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.437+0000 7f82910e8700 1 --2- 172.21.15.163:0/1541621704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f828c15d240 0x7f828c1698e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:25.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.437+0000 7f82910e8700 1 --2- 172.21.15.163:0/1541621704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f828c0b16d0 0x7f828c0a6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:25.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.437+0000 7f82910e8700 1 --2- 172.21.15.163:0/1541621704 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f828c0b4e70 0x7f828c0ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:25.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.437+0000 7f82910e8700 1 -- 172.21.15.163:0/1541621704 >> 172.21.15.163:0/1541621704 conn(0x7f828c0d10c0 msgr2=0x7f828c159050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:25.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.437+0000 7f82910e8700 1 -- 172.21.15.163:0/1541621704 shutdown_connections 2023-10-09T06:32:25.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.437+0000 7f82910e8700 1 -- 172.21.15.163:0/1541621704 wait complete. 2023-10-09T06:32:25.450 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-09T06:32:25.450 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:32:25.487 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:32:25.539 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-09T06:32:25.539 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-09T06:32:25.539 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:32:25.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.656+0000 7f54270f2700 1 Processor -- start 2023-10-09T06:32:25.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.656+0000 7f54270f2700 1 -- start start 2023-10-09T06:32:25.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.657+0000 7f54270f2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54200b5ea0 0x7f5420068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:25.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.657+0000 7f54270f2700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f542015c480 con 0x7f5420068bb0 2023-10-09T06:32:25.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.657+0000 7f54270f2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f542015b870 con 0x7f5420159c10 2023-10-09T06:32:25.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.657+0000 7f54270f2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54200b2a40 con 0x7f54200b5ea0 2023-10-09T06:32:25.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.657+0000 7f542568f700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f5420068bb0 0x7f542015c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53990/0 (socket says 172.21.15.163:53990) 2023-10-09T06:32:25.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.657+0000 7f542568f700 1 -- 172.21.15.163:0/3283634418 learned_addr learned my addr 172.21.15.163:0/3283634418 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:25.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.657+0000 7f541ffff700 1 --2- 172.21.15.163:0/3283634418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54200b5ea0 0x7f5420068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:25.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.657+0000 7f541f7fe700 1 -- 172.21.15.163:0/3283634418 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048861606 0 0) 0x7f542015c480 con 0x7f5420068bb0 2023-10-09T06:32:25.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.657+0000 7f541f7fe700 1 -- 172.21.15.163:0/3283634418 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5410003570 con 0x7f5420068bb0 2023-10-09T06:32:25.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.657+0000 7f541f7fe700 1 -- 172.21.15.163:0/3283634418 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 707638430 0 0) 0x7f5410003570 con 0x7f5420068bb0 2023-10-09T06:32:25.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.657+0000 7f541f7fe700 1 -- 172.21.15.163:0/3283634418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54200b5ea0 msgr2=0x7f5420068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:25.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.657+0000 7f541f7fe700 1 --2- 172.21.15.163:0/3283634418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54200b5ea0 0x7f5420068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:25.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.657+0000 7f541f7fe700 1 -- 172.21.15.163:0/3283634418 >> v1:172.21.15.181:6789/0 conn(0x7f5420159c10 legacy=0x7f542015b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:25.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.657+0000 7f541f7fe700 1 -- 172.21.15.163:0/3283634418 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542015bb50 con 0x7f5420068bb0 2023-10-09T06:32:25.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.657+0000 7f541f7fe700 1 -- 172.21.15.163:0/3283634418 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f54180017f0 con 0x7f5420068bb0 2023-10-09T06:32:25.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.657+0000 7f541f7fe700 1 -- 172.21.15.163:0/3283634418 >> v1:172.21.15.163:6789/0 conn(0x7f5420068bb0 legacy=0x7f542015c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:25.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.657+0000 7f541f7fe700 1 --2- 172.21.15.163:0/3283634418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5410003a90 0x7f5410003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:25.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.657+0000 7f541f7fe700 1 --2- 172.21.15.163:0/3283634418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54200b5ea0 0x7f5410004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:25.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.657+0000 7f541f7fe700 1 --2- 172.21.15.163:0/3283634418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5410004d50 0x7f54100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:25.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.657+0000 7f541f7fe700 1 -- 172.21.15.163:0/3283634418 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541000bb40 con 0x7f5410004d50 2023-10-09T06:32:25.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.657+0000 7f541f7fe700 1 -- 172.21.15.163:0/3283634418 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541000bcb0 con 0x7f5410003a90 2023-10-09T06:32:25.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.657+0000 7f541f7fe700 1 -- 172.21.15.163:0/3283634418 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f541000be70 con 0x7f54200b5ea0 2023-10-09T06:32:25.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.658+0000 7f542568f700 1 --2- 172.21.15.163:0/3283634418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5410004d50 0x7f54100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:25.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.658+0000 7f541ffff700 1 --2- 172.21.15.163:0/3283634418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54200b5ea0 0x7f5410004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:25.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.658+0000 7f5424e8e700 1 --2- 172.21.15.163:0/3283634418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5410003a90 0x7f5410003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:25.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.658+0000 7f542568f700 1 -- 172.21.15.163:0/3283634418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54200b5ea0 msgr2=0x7f5410004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:25.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.658+0000 7f542568f700 1 --2- 172.21.15.163:0/3283634418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54200b5ea0 0x7f5410004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:25.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.658+0000 7f542568f700 1 -- 172.21.15.163:0/3283634418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5410003a90 msgr2=0x7f5410003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:25.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.658+0000 7f542568f700 1 --2- 172.21.15.163:0/3283634418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5410003a90 0x7f5410003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:25.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.658+0000 7f542568f700 1 -- 172.21.15.163:0/3283634418 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5410003570 con 0x7f5410004d50 2023-10-09T06:32:25.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.658+0000 7f542568f700 1 --2- 172.21.15.163:0/3283634418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5410004d50 0x7f54100051c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f541800a930 tx=0x7f54180021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4078b866697f4d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:25.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.658+0000 7f541f7fe700 1 -- 172.21.15.163:0/3283634418 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54180070d0 con 0x7f5410004d50 2023-10-09T06:32:25.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.658+0000 7f541f7fe700 1 -- 172.21.15.163:0/3283634418 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5418007230 con 0x7f5410004d50 2023-10-09T06:32:25.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.658+0000 7f541f7fe700 1 -- 172.21.15.163:0/3283634418 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54180075d0 con 0x7f5410004d50 2023-10-09T06:32:25.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.659+0000 7f54270f2700 1 -- 172.21.15.163:0/3283634418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5410004d50 msgr2=0x7f54100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:25.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.659+0000 7f54270f2700 1 --2- 172.21.15.163:0/3283634418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5410004d50 0x7f54100051c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f541800a930 tx=0x7f54180021d0 comp rx=0 tx=0).stop 2023-10-09T06:32:25.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.659+0000 7f542568f700 1 -- 172.21.15.163:0/3283634418 reap_dead start 2023-10-09T06:32:25.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.659+0000 7f54270f2700 1 -- 172.21.15.163:0/3283634418 shutdown_connections 2023-10-09T06:32:25.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.659+0000 7f54270f2700 1 -- 172.21.15.163:0/3283634418 >> 172.21.15.163:0/3283634418 conn(0x7f5420099a20 msgr2=0x7f54200af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:25.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.659+0000 7f54270f2700 1 -- 172.21.15.163:0/3283634418 shutdown_connections 2023-10-09T06:32:25.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.659+0000 7f54270f2700 1 -- 172.21.15.163:0/3283634418 wait complete. 2023-10-09T06:32:25.667 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.660+0000 7f54270f2700 1 Processor -- start 2023-10-09T06:32:25.667 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.660+0000 7f54270f2700 1 -- start start 2023-10-09T06:32:25.667 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.661+0000 7f54270f2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5410004d50 0x7f5420147170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:25.667 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.661+0000 7f54270f2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54200afd40 0x7f54200b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:25.667 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.661+0000 7f54270f2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f542015dd30 0x7f542015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:25.667 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.661+0000 7f54270f2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54200b06f0 con 0x7f5410004d50 2023-10-09T06:32:25.667 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.661+0000 7f54270f2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542016a950 con 0x7f54200afd40 2023-10-09T06:32:25.667 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.661+0000 7f54270f2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542015ea80 con 0x7f542015dd30 2023-10-09T06:32:25.667 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.661+0000 7f5424e8e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5410004d50 0x7f5420147170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:25.667 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.661+0000 7f5424e8e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5410004d50 0x7f5420147170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58576/0 (socket says 172.21.15.163:58576) 2023-10-09T06:32:25.667 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.661+0000 7f5424e8e700 1 -- 172.21.15.163:0/1777122596 learned_addr learned my addr 172.21.15.163:0/1777122596 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:25.667 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.661+0000 7f542568f700 1 --2- 172.21.15.163:0/1777122596 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f542015dd30 0x7f542015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:25.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.661+0000 7f541ffff700 1 --2- 172.21.15.163:0/1777122596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54200afd40 0x7f54200b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:25.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.661+0000 7f5424e8e700 1 -- 172.21.15.163:0/1777122596 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f542015dd30 msgr2=0x7f542015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:25.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.661+0000 7f5424e8e700 1 --2- 172.21.15.163:0/1777122596 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f542015dd30 0x7f542015e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:25.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.661+0000 7f5424e8e700 1 -- 172.21.15.163:0/1777122596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54200afd40 msgr2=0x7f54200b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:25.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.661+0000 7f5424e8e700 1 --2- 172.21.15.163:0/1777122596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54200afd40 0x7f54200b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:25.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.661+0000 7f5424e8e700 1 -- 172.21.15.163:0/1777122596 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542015ed60 con 0x7f5410004d50 2023-10-09T06:32:25.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.661+0000 7f5424e8e700 1 --2- 172.21.15.163:0/1777122596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5410004d50 0x7f5420147170 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5414002920 tx=0x7f5414007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:25.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.661+0000 7f541cff9700 1 -- 172.21.15.163:0/1777122596 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541400f670 con 0x7f5410004d50 2023-10-09T06:32:25.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.661+0000 7f541cff9700 1 -- 172.21.15.163:0/1777122596 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541400f7d0 con 0x7f5410004d50 2023-10-09T06:32:25.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.661+0000 7f541cff9700 1 -- 172.21.15.163:0/1777122596 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541400fb90 con 0x7f5410004d50 2023-10-09T06:32:25.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.661+0000 7f54270f2700 1 -- 172.21.15.163:0/1777122596 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542015f040 con 0x7f5410004d50 2023-10-09T06:32:25.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.661+0000 7f54270f2700 1 -- 172.21.15.163:0/1777122596 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54201476b0 con 0x7f5410004d50 2023-10-09T06:32:25.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.662+0000 7f54270f2700 1 -- 172.21.15.163:0/1777122596 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f542016aac0 con 0x7f5410004d50 2023-10-09T06:32:25.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.663+0000 7f541cff9700 1 -- 172.21.15.163:0/1777122596 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f54140133f0 con 0x7f5410004d50 2023-10-09T06:32:25.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.663+0000 7f541cff9700 1 --2- 172.21.15.163:0/1777122596 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5404066ff0 0x7f54040694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:25.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.663+0000 7f541cff9700 1 -- 172.21.15.163:0/1777122596 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f5414083fc0 con 0x7f5410004d50 2023-10-09T06:32:25.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.663+0000 7f541ffff700 1 --2- 172.21.15.163:0/1777122596 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5404066ff0 0x7f54040694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:25.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.664+0000 7f541ffff700 1 --2- 172.21.15.163:0/1777122596 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5404066ff0 0x7f54040694a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f542008a050 tx=0x7f540c0091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:25.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.667+0000 7f541cff9700 1 -- 172.21.15.163:0/1777122596 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5414051540 con 0x7f5410004d50 2023-10-09T06:32:25.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.891+0000 7f54270f2700 1 -- 172.21.15.163:0/1777122596 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f542004c3c0 con 0x7f5410004d50 2023-10-09T06:32:25.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.891+0000 7f541cff9700 1 -- 172.21.15.163:0/1777122596 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+3538 (secure 0 0 0) 0x7f5414054b60 con 0x7f5410004d50 2023-10-09T06:32:25.893 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:25.893 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":34,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/819128421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":819128421},{"type":"v1","addr":"172.21.15.181:6836","nonce":819128421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5349,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3339496202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3339496202},{"type":"v1","addr":"172.21.15.163:6834","nonce":3339496202}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5350,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1693653433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1693653433},{"type":"v1","addr":"172.21.15.181:6837","nonce":1693653433}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5352,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1726174482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1726174482},{"type":"v1","addr":"172.21.15.163:6835","nonce":1726174482}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[]} 2023-10-09T06:32:25.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.893+0000 7f54270f2700 1 -- 172.21.15.163:0/1777122596 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5404066ff0 msgr2=0x7f54040694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:25.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.893+0000 7f54270f2700 1 --2- 172.21.15.163:0/1777122596 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5404066ff0 0x7f54040694a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f542008a050 tx=0x7f540c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:25.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.893+0000 7f54270f2700 1 -- 172.21.15.163:0/1777122596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5410004d50 msgr2=0x7f5420147170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:25.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.893+0000 7f54270f2700 1 --2- 172.21.15.163:0/1777122596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5410004d50 0x7f5420147170 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5414002920 tx=0x7f5414007d30 comp rx=0 tx=0).stop 2023-10-09T06:32:25.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.893+0000 7f54270f2700 1 -- 172.21.15.163:0/1777122596 shutdown_connections 2023-10-09T06:32:25.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.893+0000 7f54270f2700 1 --2- 172.21.15.163:0/1777122596 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5404066ff0 0x7f54040694a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:25.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.893+0000 7f54270f2700 1 --2- 172.21.15.163:0/1777122596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5410004d50 0x7f5420147170 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:25.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.893+0000 7f54270f2700 1 --2- 172.21.15.163:0/1777122596 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f542015dd30 0x7f542015e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:25.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.893+0000 7f54270f2700 1 --2- 172.21.15.163:0/1777122596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54200afd40 0x7f54200b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:25.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.893+0000 7f54270f2700 1 -- 172.21.15.163:0/1777122596 >> 172.21.15.163:0/1777122596 conn(0x7f5420099a20 msgr2=0x7f542009ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:25.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.894+0000 7f54270f2700 1 -- 172.21.15.163:0/1777122596 shutdown_connections 2023-10-09T06:32:25.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:25.894+0000 7f54270f2700 1 -- 172.21.15.163:0/1777122596 wait complete. 2023-10-09T06:32:25.896 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 34 2023-10-09T06:32:25.906 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-09T06:32:25.907 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-09T06:32:25.907 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-09T06:32:25.907 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:32:25.907 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-09T06:32:25.907 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-09T06:32:25.907 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-09T06:32:25.908 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:32:25.908 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-09T06:32:25.908 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-09T06:32:25.908 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-09T06:32:25.908 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:32:25.908 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-09T06:32:25.908 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-09T06:32:25.909 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-09T06:32:25.909 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:32:25.916 INFO:tasks.ceph.mds.a:Stopped 2023-10-09T06:32:25.916 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-09T06:32:25.917 INFO:tasks.ceph.mds.c:Stopped 2023-10-09T06:32:25.917 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-09T06:32:25.920 INFO:tasks.ceph.mds.b:Stopped 2023-10-09T06:32:25.920 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-09T06:32:25.920 INFO:tasks.ceph.mds.d:Stopped 2023-10-09T06:32:25.921 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-09T06:32:25.959 INFO:tasks.ceph.mds.a:Started 2023-10-09T06:32:25.960 DEBUG:teuthology.parallel:result is None 2023-10-09T06:32:25.960 INFO:tasks.ceph.mds.c:Started 2023-10-09T06:32:25.960 DEBUG:teuthology.parallel:result is None 2023-10-09T06:32:25.962 INFO:tasks.ceph.mds.b:Started 2023-10-09T06:32:25.963 DEBUG:teuthology.parallel:result is None 2023-10-09T06:32:25.963 INFO:tasks.ceph.mds.d:Started 2023-10-09T06:32:25.963 DEBUG:teuthology.parallel:result is None 2023-10-09T06:32:25.964 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-09T06:32:26.033 INFO:tasks.ceph.mds.a.smithi163.stdout:starting mds.a at 2023-10-09T06:32:26.033 INFO:tasks.ceph.mds.c.smithi163.stdout:starting mds.c at 2023-10-09T06:32:26.042 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-09T06:32:26.042 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-09T06:32:26.043 INFO:tasks.ceph.mds.d.smithi181.stderr:2023-10-09T06:32:26.042+0000 7f32f5f15700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-09T06:32:26.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.087+0000 7f3a1b2a5700 1 Processor -- start 2023-10-09T06:32:26.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.087+0000 7f3a1b2a5700 1 -- start start 2023-10-09T06:32:26.090 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.088+0000 7f3a1b2a5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3a1406bcd0 0x7f3a14140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:26.090 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.088+0000 7f3a1b2a5700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3a1406c0a0 0x7f3a14141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:26.090 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.088+0000 7f3a1b2a5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a1415c2a0 con 0x7f3a140c8e00 2023-10-09T06:32:26.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.088+0000 7f3a1b2a5700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a140c9900 con 0x7f3a1406c0a0 2023-10-09T06:32:26.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.088+0000 7f3a1b2a5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a1415b690 con 0x7f3a1406bcd0 2023-10-09T06:32:26.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.088+0000 7f3a18840700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3a1406c0a0 0x7f3a14141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:26.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.088+0000 7f3a18840700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3a1406c0a0 0x7f3a14141330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58610/0 (socket says 172.21.15.163:58610) 2023-10-09T06:32:26.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.088+0000 7f3a18840700 1 -- 172.21.15.163:0/635334990 learned_addr learned my addr 172.21.15.163:0/635334990 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:26.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.088+0000 7f3a18840700 1 -- 172.21.15.163:0/635334990 >> v2:172.21.15.181:3300/0 conn(0x7f3a1406bcd0 msgr2=0x7f3a14140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:26.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.088+0000 7f3a18840700 1 --2- 172.21.15.163:0/635334990 >> v2:172.21.15.181:3300/0 conn(0x7f3a1406bcd0 0x7f3a14140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:26.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.088+0000 7f3a18840700 1 -- 172.21.15.163:0/635334990 >> v1:172.21.15.181:6789/0 conn(0x7f3a140c8e00 legacy=0x7f3a140c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:26.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.088+0000 7f3a18840700 1 -- 172.21.15.163:0/635334990 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1415b970 con 0x7f3a1406c0a0 2023-10-09T06:32:26.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.088+0000 7f3a18840700 1 --2- 172.21.15.163:0/635334990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a1406c0a0 0x7f3a14141330 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3a140cafd0 tx=0x7f3a04009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a888bcda302cdf14 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:26.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.088+0000 7f3a0bfff700 1 -- 172.21.15.163:0/635334990 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0400b820 con 0x7f3a1406c0a0 2023-10-09T06:32:26.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.088+0000 7f3a0bfff700 1 -- 172.21.15.163:0/635334990 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0400b980 con 0x7f3a1406c0a0 2023-10-09T06:32:26.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.089+0000 7f3a0bfff700 1 -- 172.21.15.163:0/635334990 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0400bd20 con 0x7f3a1406c0a0 2023-10-09T06:32:26.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.089+0000 7f3a1b2a5700 1 -- 172.21.15.163:0/635334990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a1406c0a0 msgr2=0x7f3a14141330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:26.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.089+0000 7f3a1b2a5700 1 --2- 172.21.15.163:0/635334990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a1406c0a0 0x7f3a14141330 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3a140cafd0 tx=0x7f3a04009250 comp rx=0 tx=0).stop 2023-10-09T06:32:26.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.089+0000 7f3a1b2a5700 1 -- 172.21.15.163:0/635334990 shutdown_connections 2023-10-09T06:32:26.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.089+0000 7f3a1b2a5700 1 --2- 172.21.15.163:0/635334990 >> v2:172.21.15.181:3300/0 conn(0x7f3a1406bcd0 0x7f3a14140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:26.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.089+0000 7f3a1b2a5700 1 --2- 172.21.15.163:0/635334990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a1406c0a0 0x7f3a14141330 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:26.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.089+0000 7f3a1b2a5700 1 -- 172.21.15.163:0/635334990 >> 172.21.15.163:0/635334990 conn(0x7f3a140ce600 msgr2=0x7f3a140cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:26.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.089+0000 7f3a1b2a5700 1 -- 172.21.15.163:0/635334990 shutdown_connections 2023-10-09T06:32:26.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.089+0000 7f3a1b2a5700 1 -- 172.21.15.163:0/635334990 wait complete. 2023-10-09T06:32:26.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.090+0000 7f3a1b2a5700 1 Processor -- start 2023-10-09T06:32:26.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.090+0000 7f3a1b2a5700 1 -- start start 2023-10-09T06:32:26.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.091+0000 7f3a1b2a5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a1406bcd0 0x7f3a140c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:26.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.091+0000 7f3a1b2a5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a1406c0a0 0x7f3a1413f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:26.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.091+0000 7f3a1b2a5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a140c8e00 0x7f3a1413fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:26.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.091+0000 7f3a1b2a5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a140c8480 con 0x7f3a140c8e00 2023-10-09T06:32:26.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.091+0000 7f3a1b2a5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a14155b50 con 0x7f3a1406bcd0 2023-10-09T06:32:26.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.091+0000 7f3a1b2a5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a14155da0 con 0x7f3a1406c0a0 2023-10-09T06:32:26.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.091+0000 7f3a19842700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a140c8e00 0x7f3a1413fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:26.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.091+0000 7f3a19842700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a140c8e00 0x7f3a1413fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58626/0 (socket says 172.21.15.163:58626) 2023-10-09T06:32:26.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.091+0000 7f3a19842700 1 -- 172.21.15.163:0/2136417318 learned_addr learned my addr 172.21.15.163:0/2136417318 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:26.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.091+0000 7f3a19041700 1 --2- 172.21.15.163:0/2136417318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a1406bcd0 0x7f3a140c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:26.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.091+0000 7f3a18840700 1 --2- 172.21.15.163:0/2136417318 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a1406c0a0 0x7f3a1413f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:26.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.091+0000 7f3a19842700 1 -- 172.21.15.163:0/2136417318 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a1406c0a0 msgr2=0x7f3a1413f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:26.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.091+0000 7f3a19842700 1 --2- 172.21.15.163:0/2136417318 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a1406c0a0 0x7f3a1413f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:26.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.091+0000 7f3a19842700 1 -- 172.21.15.163:0/2136417318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a1406bcd0 msgr2=0x7f3a140c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:26.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.091+0000 7f3a19842700 1 --2- 172.21.15.163:0/2136417318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a1406bcd0 0x7f3a140c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:26.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.092+0000 7f3a19842700 1 -- 172.21.15.163:0/2136417318 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a040094f0 con 0x7f3a140c8e00 2023-10-09T06:32:26.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.092+0000 7f3a19842700 1 --2- 172.21.15.163:0/2136417318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a140c8e00 0x7f3a1413fea0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00bc30 tx=0x7f3a0c009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:26.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.092+0000 7f3a097fa700 1 -- 172.21.15.163:0/2136417318 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c00c840 con 0x7f3a140c8e00 2023-10-09T06:32:26.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.092+0000 7f3a097fa700 1 -- 172.21.15.163:0/2136417318 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0c00c9e0 con 0x7f3a140c8e00 2023-10-09T06:32:26.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.092+0000 7f3a1b2a5700 1 -- 172.21.15.163:0/2136417318 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a14156080 con 0x7f3a140c8e00 2023-10-09T06:32:26.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.092+0000 7f3a097fa700 1 -- 172.21.15.163:0/2136417318 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c00cda0 con 0x7f3a140c8e00 2023-10-09T06:32:26.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.092+0000 7f3a1b2a5700 1 -- 172.21.15.163:0/2136417318 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a14156640 con 0x7f3a140c8e00 2023-10-09T06:32:26.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.093+0000 7f3a1b2a5700 1 -- 172.21.15.163:0/2136417318 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a1404c3c0 con 0x7f3a140c8e00 2023-10-09T06:32:26.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.093+0000 7f3a097fa700 1 -- 172.21.15.163:0/2136417318 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f3a0c014620 con 0x7f3a140c8e00 2023-10-09T06:32:26.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.093+0000 7f3a097fa700 1 --2- 172.21.15.163:0/2136417318 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f39f8066fc0 0x7f39f8069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:26.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.094+0000 7f3a097fa700 1 -- 172.21.15.163:0/2136417318 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f3a0c0841a0 con 0x7f3a140c8e00 2023-10-09T06:32:26.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.094+0000 7f3a19041700 1 --2- 172.21.15.163:0/2136417318 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f39f8066fc0 0x7f39f8069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:26.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.094+0000 7f3a19041700 1 --2- 172.21.15.163:0/2136417318 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f39f8066fc0 0x7f39f8069470 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009bf0 tx=0x7f3a100091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:26.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.098+0000 7f3a097fa700 1 -- 172.21.15.163:0/2136417318 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3a0c051720 con 0x7f3a140c8e00 2023-10-09T06:32:26.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:26.296+0000 7f3a1b2a5700 1 -- 172.21.15.163:0/2136417318 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3a1406c9d0 con 0x7f3a140c8e00 2023-10-09T06:32:27.116 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.115+0000 7f3a097fa700 1 -- 172.21.15.163:0/2136417318 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v59) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3a0c054d40 con 0x7f3a140c8e00 2023-10-09T06:32:27.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.117+0000 7f3a1b2a5700 1 -- 172.21.15.163:0/2136417318 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f39f8066fc0 msgr2=0x7f39f8069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:27.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.117+0000 7f3a1b2a5700 1 --2- 172.21.15.163:0/2136417318 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f39f8066fc0 0x7f39f8069470 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009bf0 tx=0x7f3a100091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:27.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.117+0000 7f3a1b2a5700 1 -- 172.21.15.163:0/2136417318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a140c8e00 msgr2=0x7f3a1413fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:27.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.117+0000 7f3a1b2a5700 1 --2- 172.21.15.163:0/2136417318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a140c8e00 0x7f3a1413fea0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00bc30 tx=0x7f3a0c009820 comp rx=0 tx=0).stop 2023-10-09T06:32:27.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.118+0000 7f3a1b2a5700 1 -- 172.21.15.163:0/2136417318 shutdown_connections 2023-10-09T06:32:27.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.118+0000 7f3a1b2a5700 1 --2- 172.21.15.163:0/2136417318 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f39f8066fc0 0x7f39f8069470 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:27.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.118+0000 7f3a1b2a5700 1 --2- 172.21.15.163:0/2136417318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a140c8e00 0x7f3a1413fea0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:27.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.118+0000 7f3a1b2a5700 1 --2- 172.21.15.163:0/2136417318 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a1406bcd0 0x7f3a140c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:27.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.118+0000 7f3a1b2a5700 1 --2- 172.21.15.163:0/2136417318 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a1406c0a0 0x7f3a1413f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:27.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.118+0000 7f3a1b2a5700 1 -- 172.21.15.163:0/2136417318 >> 172.21.15.163:0/2136417318 conn(0x7f3a140ce600 msgr2=0x7f3a140c3710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:27.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.118+0000 7f3a1b2a5700 1 -- 172.21.15.163:0/2136417318 shutdown_connections 2023-10-09T06:32:27.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.118+0000 7f3a1b2a5700 1 -- 172.21.15.163:0/2136417318 wait complete. 2023-10-09T06:32:27.120 INFO:teuthology.orchestra.run.smithi163.stderr: removed all blocklist entries 2023-10-09T06:32:27.131 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-09T06:32:27.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.245+0000 7f98eeb7f700 1 Processor -- start 2023-10-09T06:32:27.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.245+0000 7f98eeb7f700 1 -- start start 2023-10-09T06:32:27.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.246+0000 7f98eeb7f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98e8159420 0x7f98e80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:27.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.246+0000 7f98eeb7f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e80b2980 con 0x7f98e815d230 2023-10-09T06:32:27.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.246+0000 7f98eeb7f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e80b11d0 con 0x7f98e81597f0 2023-10-09T06:32:27.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.246+0000 7f98eeb7f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e80b14b0 con 0x7f98e8159420 2023-10-09T06:32:27.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.246+0000 7f98ee37e700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f98e815d230 0x7f98e80b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:54020/0 (socket says 172.21.15.163:54020) 2023-10-09T06:32:27.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.246+0000 7f98ee37e700 1 -- 172.21.15.163:0/2574900561 learned_addr learned my addr 172.21.15.163:0/2574900561 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:27.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.246+0000 7f98edb7d700 1 --2- 172.21.15.163:0/2574900561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98e8159420 0x7f98e80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:27.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.246+0000 7f98ecb7b700 1 -- 172.21.15.163:0/2574900561 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4071247103 0 0) 0x7f98e80b2980 con 0x7f98e815d230 2023-10-09T06:32:27.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.246+0000 7f98ecb7b700 1 -- 172.21.15.163:0/2574900561 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98d4003570 con 0x7f98e815d230 2023-10-09T06:32:27.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98ecb7b700 1 -- 172.21.15.163:0/2574900561 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 324562306 0 0) 0x7f98e80b11d0 con 0x7f98e81597f0 2023-10-09T06:32:27.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98ecb7b700 1 -- 172.21.15.163:0/2574900561 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98e80b2980 con 0x7f98e81597f0 2023-10-09T06:32:27.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98ecb7b700 1 -- 172.21.15.163:0/2574900561 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2847686904 0 0) 0x7f98d4003570 con 0x7f98e815d230 2023-10-09T06:32:27.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98ecb7b700 1 -- 172.21.15.163:0/2574900561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98e8159420 msgr2=0x7f98e80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:27.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98ecb7b700 1 --2- 172.21.15.163:0/2574900561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98e8159420 0x7f98e80b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:27.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98ecb7b700 1 -- 172.21.15.163:0/2574900561 >> v1:172.21.15.181:6789/0 conn(0x7f98e81597f0 legacy=0x7f98e80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:27.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98ecb7b700 1 -- 172.21.15.163:0/2574900561 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e80b1790 con 0x7f98e815d230 2023-10-09T06:32:27.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98ecb7b700 1 -- 172.21.15.163:0/2574900561 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f98e00017f0 con 0x7f98e815d230 2023-10-09T06:32:27.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98ecb7b700 1 -- 172.21.15.163:0/2574900561 >> v1:172.21.15.163:6789/0 conn(0x7f98e815d230 legacy=0x7f98e80b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:27.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98ecb7b700 1 --2- 172.21.15.163:0/2574900561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98d4003c20 0x7f98d4004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:27.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98ecb7b700 1 --2- 172.21.15.163:0/2574900561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98d4004810 0x7f98d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:27.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98ecb7b700 1 --2- 172.21.15.163:0/2574900561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98d40051c0 0x7f98d400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:27.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98ecb7b700 1 -- 172.21.15.163:0/2574900561 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e80b14b0 con 0x7f98d40051c0 2023-10-09T06:32:27.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98ecb7b700 1 -- 172.21.15.163:0/2574900561 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d400c020 con 0x7f98d4004810 2023-10-09T06:32:27.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98ecb7b700 1 -- 172.21.15.163:0/2574900561 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d400c1e0 con 0x7f98d4003c20 2023-10-09T06:32:27.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98edb7d700 1 --2- 172.21.15.163:0/2574900561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98e8159420 0x7f98e80b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:32:27.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98ee37e700 1 --2- 172.21.15.163:0/2574900561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98d40051c0 0x7f98d400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:27.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98ed37c700 1 --2- 172.21.15.163:0/2574900561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98d4004810 0x7f98d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:27.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98edb7d700 1 --2- 172.21.15.163:0/2574900561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98d4003c20 0x7f98d4004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:27.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98ee37e700 1 -- 172.21.15.163:0/2574900561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98d4003c20 msgr2=0x7f98d4004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:27.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98ee37e700 1 --2- 172.21.15.163:0/2574900561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98d4003c20 0x7f98d4004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:27.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98ee37e700 1 -- 172.21.15.163:0/2574900561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98d4004810 msgr2=0x7f98d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:27.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98ee37e700 1 --2- 172.21.15.163:0/2574900561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98d4004810 0x7f98d4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:27.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98ee37e700 1 -- 172.21.15.163:0/2574900561 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d4003570 con 0x7f98d40051c0 2023-10-09T06:32:27.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98edb7d700 1 --2- 172.21.15.163:0/2574900561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98d4003c20 0x7f98d4004010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:27.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98ed37c700 1 --2- 172.21.15.163:0/2574900561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98d4004810 0x7f98d4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:27.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98ee37e700 1 --2- 172.21.15.163:0/2574900561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98d40051c0 0x7f98d400ba60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f98e000a8f0 tx=0x7f98e0002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1173e5cfda1a31ae server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:27.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98ecb7b700 1 -- 172.21.15.163:0/2574900561 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e00070d0 con 0x7f98d40051c0 2023-10-09T06:32:27.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98ecb7b700 1 -- 172.21.15.163:0/2574900561 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e0007250 con 0x7f98d40051c0 2023-10-09T06:32:27.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.247+0000 7f98ecb7b700 1 -- 172.21.15.163:0/2574900561 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e0007610 con 0x7f98d40051c0 2023-10-09T06:32:27.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.248+0000 7f98eeb7f700 1 -- 172.21.15.163:0/2574900561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98d40051c0 msgr2=0x7f98d400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:27.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.248+0000 7f98eeb7f700 1 --2- 172.21.15.163:0/2574900561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98d40051c0 0x7f98d400ba60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f98e000a8f0 tx=0x7f98e0002190 comp rx=0 tx=0).stop 2023-10-09T06:32:27.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.248+0000 7f98ee37e700 1 -- 172.21.15.163:0/2574900561 reap_dead start 2023-10-09T06:32:27.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.248+0000 7f98eeb7f700 1 -- 172.21.15.163:0/2574900561 shutdown_connections 2023-10-09T06:32:27.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.248+0000 7f98eeb7f700 1 -- 172.21.15.163:0/2574900561 >> 172.21.15.163:0/2574900561 conn(0x7f98e80d0660 msgr2=0x7f98e80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:27.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.248+0000 7f98eeb7f700 1 -- 172.21.15.163:0/2574900561 shutdown_connections 2023-10-09T06:32:27.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.248+0000 7f98eeb7f700 1 -- 172.21.15.163:0/2574900561 wait complete. 2023-10-09T06:32:27.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.249+0000 7f98eeb7f700 1 Processor -- start 2023-10-09T06:32:27.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.249+0000 7f98eeb7f700 1 -- start start 2023-10-09T06:32:27.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.250+0000 7f98eeb7f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98e814d280 0x7f98e814f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:27.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.250+0000 7f98eeb7f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98e814f680 0x7f98e8151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:27.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.250+0000 7f98eeb7f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98e8151550 0x7f98e8151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:27.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.250+0000 7f98eeb7f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e815a380 con 0x7f98e814d280 2023-10-09T06:32:27.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.250+0000 7f98eeb7f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e81519d0 con 0x7f98e814f680 2023-10-09T06:32:27.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.250+0000 7f98eeb7f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e815a620 con 0x7f98e8151550 2023-10-09T06:32:27.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.250+0000 7f98edb7d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98e814d280 0x7f98e814f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:27.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.250+0000 7f98edb7d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98e814d280 0x7f98e814f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58644/0 (socket says 172.21.15.163:58644) 2023-10-09T06:32:27.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.250+0000 7f98edb7d700 1 -- 172.21.15.163:0/2130637764 learned_addr learned my addr 172.21.15.163:0/2130637764 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:27.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.250+0000 7f98ed37c700 1 --2- 172.21.15.163:0/2130637764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98e814f680 0x7f98e8151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:27.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.250+0000 7f98ee37e700 1 --2- 172.21.15.163:0/2130637764 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98e8151550 0x7f98e8151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:27.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.250+0000 7f98edb7d700 1 -- 172.21.15.163:0/2130637764 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98e8151550 msgr2=0x7f98e8151cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:27.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.250+0000 7f98edb7d700 1 --2- 172.21.15.163:0/2130637764 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98e8151550 0x7f98e8151cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:27.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.250+0000 7f98edb7d700 1 -- 172.21.15.163:0/2130637764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98e814f680 msgr2=0x7f98e8151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:27.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.250+0000 7f98edb7d700 1 --2- 172.21.15.163:0/2130637764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98e814f680 0x7f98e8151010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:27.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.250+0000 7f98edb7d700 1 -- 172.21.15.163:0/2130637764 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e8150430 con 0x7f98e814d280 2023-10-09T06:32:27.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.250+0000 7f98ee37e700 1 --2- 172.21.15.163:0/2130637764 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98e8151550 0x7f98e8151cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:27.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.250+0000 7f98edb7d700 1 --2- 172.21.15.163:0/2130637764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98e814d280 0x7f98e814f140 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f98e4011910 tx=0x7f98e4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:27.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.250+0000 7f98ddffb700 1 -- 172.21.15.163:0/2130637764 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e400f700 con 0x7f98e814d280 2023-10-09T06:32:27.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.251+0000 7f98ddffb700 1 -- 172.21.15.163:0/2130637764 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e400f860 con 0x7f98e814d280 2023-10-09T06:32:27.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.251+0000 7f98eeb7f700 1 -- 172.21.15.163:0/2130637764 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e814fb90 con 0x7f98e814d280 2023-10-09T06:32:27.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.251+0000 7f98ddffb700 1 -- 172.21.15.163:0/2130637764 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e400fc20 con 0x7f98e814d280 2023-10-09T06:32:27.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.251+0000 7f98eeb7f700 1 -- 172.21.15.163:0/2130637764 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e8150150 con 0x7f98e814d280 2023-10-09T06:32:27.261 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.252+0000 7f98eeb7f700 1 -- 172.21.15.163:0/2130637764 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e804c3c0 con 0x7f98e814d280 2023-10-09T06:32:27.261 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.252+0000 7f98ddffb700 1 -- 172.21.15.163:0/2130637764 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f98e400fd80 con 0x7f98e814d280 2023-10-09T06:32:27.261 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.252+0000 7f98ddffb700 1 --2- 172.21.15.163:0/2130637764 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f98cc067040 0x7f98cc0694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:27.261 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.253+0000 7f98ddffb700 1 -- 172.21.15.163:0/2130637764 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f98e4084b40 con 0x7f98e814d280 2023-10-09T06:32:27.261 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.253+0000 7f98ed37c700 1 --2- 172.21.15.163:0/2130637764 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f98cc067040 0x7f98cc0694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:27.261 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.253+0000 7f98ed37c700 1 --2- 172.21.15.163:0/2130637764 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f98cc067040 0x7f98cc0694f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f98e8069b20 tx=0x7f98d800c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:27.261 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.257+0000 7f98ddffb700 1 -- 172.21.15.163:0/2130637764 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f98e4052010 con 0x7f98e814d280 2023-10-09T06:32:27.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.511+0000 7f98eeb7f700 1 -- 172.21.15.163:0/2130637764 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f98e80b1a70 con 0x7f98e814d280 2023-10-09T06:32:27.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.511+0000 7f98ddffb700 1 -- 172.21.15.163:0/2130637764 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f98e4055630 con 0x7f98e814d280 2023-10-09T06:32:27.512 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:27.512 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:32:27.512 INFO:teuthology.orchestra.run.smithi163.stdout: "auth_dump": [ 2023-10-09T06:32:27.512 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:32:27.512 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.a", 2023-10-09T06:32:27.513 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNltXtwDxAAlnrWRsfnZDLEmsJVowh3Mg==", 2023-10-09T06:32:27.513 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:32:27.513 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:32:27.513 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:32:27.513 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:32:27.513 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:32:27.513 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:32:27.513 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:32:27.513 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:32:27.513 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.b", 2023-10-09T06:32:27.513 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlTdGhGhAAXuldy0HrFd+44sSu/EV3qA==", 2023-10-09T06:32:27.516 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:32:27.516 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:32:27.516 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:32:27.516 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:32:27.516 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:32:27.516 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:32:27.517 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:32:27.517 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:32:27.517 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.c", 2023-10-09T06:32:27.517 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlGis7FhAAYG76TiOFX8QkTwjNX2+Ncw==", 2023-10-09T06:32:27.517 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:32:27.517 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:32:27.517 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:32:27.517 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:32:27.517 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:32:27.517 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:32:27.517 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:32:27.517 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:32:27.518 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.d", 2023-10-09T06:32:27.518 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlKhRuIRAAfhG5Gpc7klXJloYXepOhsg==", 2023-10-09T06:32:27.518 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:32:27.518 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:32:27.518 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:32:27.518 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:32:27.518 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:32:27.518 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:32:27.518 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:32:27.518 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:32:27.518 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.0", 2023-10-09T06:32:27.518 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDcnSNlrQeFOxAA+s6UitrTn5TK1N7kgySoXw==", 2023-10-09T06:32:27.519 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:32:27.519 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:32:27.519 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:32:27.519 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:32:27.519 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:32:27.519 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:32:27.519 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:32:27.519 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.1", 2023-10-09T06:32:27.519 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDfnSNlleccORAAnB+lNUFblivdoY44JfELxA==", 2023-10-09T06:32:27.519 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:32:27.519 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:32:27.519 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:32:27.519 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:32:27.520 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:32:27.520 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:32:27.520 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:32:27.520 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.2", 2023-10-09T06:32:27.520 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDinSNlFoB4NxAAT0+trRBij5rqgc+F0eWUfQ==", 2023-10-09T06:32:27.520 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:32:27.520 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:32:27.520 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:32:27.520 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:32:27.520 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:32:27.520 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:32:27.520 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:32:27.521 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.3", 2023-10-09T06:32:27.521 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDlnSNlRRQ6NhAABoBtrFjSvQujolq0Pxrd1Q==", 2023-10-09T06:32:27.521 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:32:27.521 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:32:27.521 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:32:27.521 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:32:27.521 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:32:27.521 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:32:27.521 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:32:27.521 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.4", 2023-10-09T06:32:27.521 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDrnSNl308uOxAA5OJyPLGQmVSJqha5d+nKKQ==", 2023-10-09T06:32:27.521 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:32:27.521 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:32:27.522 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:32:27.522 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:32:27.522 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:32:27.522 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:32:27.522 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:32:27.522 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.5", 2023-10-09T06:32:27.522 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDunSNlHnK+OBAAU7l4dCJ82SbGSzBEm8Axzw==", 2023-10-09T06:32:27.522 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:32:27.522 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:32:27.522 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:32:27.522 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:32:27.522 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:32:27.523 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:32:27.523 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:32:27.523 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.6", 2023-10-09T06:32:27.523 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDxnSNlwUSLOBAAN5N+F5ho+Co8pBh4Rz+lXQ==", 2023-10-09T06:32:27.523 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:32:27.523 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:32:27.523 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:32:27.523 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:32:27.523 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:32:27.523 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:32:27.523 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:32:27.524 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.7", 2023-10-09T06:32:27.524 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQD0nSNl2gxGORAAoE8OLGEifcEx/km8rfSV9g==", 2023-10-09T06:32:27.524 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:32:27.524 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:32:27.524 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:32:27.524 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:32:27.524 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:32:27.524 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:32:27.524 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:32:27.525 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.0", 2023-10-09T06:32:27.525 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlnxY5JRAApr6czcpMdQYYtXQ9yg2q6A==", 2023-10-09T06:32:27.525 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:32:27.525 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:32:27.525 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow r", 2023-10-09T06:32:27.525 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-09T06:32:27.525 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:32:27.525 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:32:27.525 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:32:27.525 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.1", 2023-10-09T06:32:27.525 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlk6ybLRAAuShqTDAtymGdxN8KBdZDDA==", 2023-10-09T06:32:27.525 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:32:27.525 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:32:27.526 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:32:27.526 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:32:27.526 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:32:27.526 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:32:27.526 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:32:27.526 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:32:27.526 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.2", 2023-10-09T06:32:27.527 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlYyOmKhAAQmpr+mGfJH4GUa28dtlJwA==", 2023-10-09T06:32:27.527 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:32:27.527 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:32:27.527 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:32:27.527 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:32:27.527 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:32:27.527 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:32:27.527 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:32:27.527 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:32:27.527 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.3", 2023-10-09T06:32:27.527 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlD2H1MhAA4zvaXY/Gd0kndzi+QSwp7w==", 2023-10-09T06:32:27.527 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:32:27.528 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:32:27.528 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:32:27.528 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:32:27.528 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:32:27.528 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:32:27.528 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:32:27.528 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:32:27.528 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.admin", 2023-10-09T06:32:27.528 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDYnSNlsHH4LxAA85xeFEP1CUwoblaePYk5zA==", 2023-10-09T06:32:27.529 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:32:27.529 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:32:27.529 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:32:27.529 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:32:27.529 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:32:27.529 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:32:27.529 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:32:27.529 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:32:27.529 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mgr.x", 2023-10-09T06:32:27.529 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNl2sloDBAA0FoWN9iovCl6783o3/btiA==", 2023-10-09T06:32:27.529 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:32:27.530 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:32:27.530 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile mgr", 2023-10-09T06:32:27.530 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:32:27.530 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:32:27.530 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:32:27.530 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:32:27.530 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mgr.y", 2023-10-09T06:32:27.530 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlcFWVCBAAR2dKPg01ciTiNpSX0JIKFg==", 2023-10-09T06:32:27.531 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:32:27.531 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:32:27.531 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile mgr", 2023-10-09T06:32:27.531 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:32:27.531 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:32:27.531 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:32:27.531 INFO:teuthology.orchestra.run.smithi163.stdout: ] 2023-10-09T06:32:27.531 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:32:27.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.513+0000 7f98eeb7f700 1 -- 172.21.15.163:0/2130637764 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f98cc067040 msgr2=0x7f98cc0694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:27.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.513+0000 7f98eeb7f700 1 --2- 172.21.15.163:0/2130637764 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f98cc067040 0x7f98cc0694f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f98e8069b20 tx=0x7f98d800c040 comp rx=0 tx=0).stop 2023-10-09T06:32:27.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.513+0000 7f98eeb7f700 1 -- 172.21.15.163:0/2130637764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98e814d280 msgr2=0x7f98e814f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:27.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.513+0000 7f98eeb7f700 1 --2- 172.21.15.163:0/2130637764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98e814d280 0x7f98e814f140 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f98e4011910 tx=0x7f98e4008040 comp rx=0 tx=0).stop 2023-10-09T06:32:27.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.513+0000 7f98eeb7f700 1 -- 172.21.15.163:0/2130637764 shutdown_connections 2023-10-09T06:32:27.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.513+0000 7f98eeb7f700 1 --2- 172.21.15.163:0/2130637764 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f98cc067040 0x7f98cc0694f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:27.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.513+0000 7f98eeb7f700 1 --2- 172.21.15.163:0/2130637764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98e814d280 0x7f98e814f140 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:27.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.513+0000 7f98eeb7f700 1 --2- 172.21.15.163:0/2130637764 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98e8151550 0x7f98e8151cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:27.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.513+0000 7f98eeb7f700 1 --2- 172.21.15.163:0/2130637764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98e814f680 0x7f98e8151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:27.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.513+0000 7f98eeb7f700 1 -- 172.21.15.163:0/2130637764 >> 172.21.15.163:0/2130637764 conn(0x7f98e80d0660 msgr2=0x7f98e80cc710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:27.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.513+0000 7f98eeb7f700 1 -- 172.21.15.163:0/2130637764 shutdown_connections 2023-10-09T06:32:27.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.513+0000 7f98eeb7f700 1 -- 172.21.15.163:0/2130637764 wait complete. 2023-10-09T06:32:27.535 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:32:27.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.649+0000 7f72c73e1700 1 Processor -- start 2023-10-09T06:32:27.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.650+0000 7f72c73e1700 1 -- start start 2023-10-09T06:32:27.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.650+0000 7f72c73e1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f72c006bcd0 0x7f72c00caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:27.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.650+0000 7f72c73e1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f72c015df50 0x7f72c015e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:27.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.650+0000 7f72c73e1700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72c015d7e0 con 0x7f72c006c0a0 2023-10-09T06:32:27.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.651+0000 7f72c73e1700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72c015dac0 con 0x7f72c015df50 2023-10-09T06:32:27.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.651+0000 7f72c73e1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72c015aa30 con 0x7f72c006bcd0 2023-10-09T06:32:27.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.651+0000 7f72c597e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f72c015df50 0x7f72c015e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:27.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.651+0000 7f72c497c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f72c006c0a0 0x7f72c015a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:54034/0 (socket says 172.21.15.163:54034) 2023-10-09T06:32:27.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.651+0000 7f72c497c700 1 -- 172.21.15.163:0/1375667101 learned_addr learned my addr 172.21.15.163:0/1375667101 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:27.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.651+0000 7f72c517d700 1 --2- 172.21.15.163:0/1375667101 >> v2:172.21.15.181:3300/0 conn(0x7f72c006bcd0 0x7f72c00caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:27.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.651+0000 7f72c597e700 1 -- 172.21.15.163:0/1375667101 >> v2:172.21.15.181:3300/0 conn(0x7f72c006bcd0 msgr2=0x7f72c00caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:27.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.651+0000 7f72c597e700 1 --2- 172.21.15.163:0/1375667101 >> v2:172.21.15.181:3300/0 conn(0x7f72c006bcd0 0x7f72c00caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:27.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.651+0000 7f72c597e700 1 -- 172.21.15.163:0/1375667101 >> v1:172.21.15.163:6789/0 conn(0x7f72c006c0a0 legacy=0x7f72c015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:27.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.651+0000 7f72b7fff700 1 -- 172.21.15.163:0/1375667101 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3704584877 0 0) 0x7f72c015d7e0 con 0x7f72c006c0a0 2023-10-09T06:32:27.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.651+0000 7f72c597e700 1 -- 172.21.15.163:0/1375667101 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c015ad10 con 0x7f72c015df50 2023-10-09T06:32:27.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.651+0000 7f72c517d700 1 --2- 172.21.15.163:0/1375667101 >> v2:172.21.15.181:3300/0 conn(0x7f72c006bcd0 0x7f72c00caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:27.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.651+0000 7f72c597e700 1 --2- 172.21.15.163:0/1375667101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72c015df50 0x7f72c015e3b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f72c00691a0 tx=0x7f72b8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f3d65f3f304548 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:27.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.651+0000 7f72b7fff700 1 -- 172.21.15.163:0/1375667101 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b800e070 con 0x7f72c015df50 2023-10-09T06:32:27.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.651+0000 7f72b7fff700 1 -- 172.21.15.163:0/1375667101 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b800bc30 con 0x7f72c015df50 2023-10-09T06:32:27.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.651+0000 7f72b7fff700 1 -- 172.21.15.163:0/1375667101 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b8012430 con 0x7f72c015df50 2023-10-09T06:32:27.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.652+0000 7f72c73e1700 1 -- 172.21.15.163:0/1375667101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72c015df50 msgr2=0x7f72c015e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:27.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.652+0000 7f72c73e1700 1 --2- 172.21.15.163:0/1375667101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72c015df50 0x7f72c015e3b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f72c00691a0 tx=0x7f72b8009780 comp rx=0 tx=0).stop 2023-10-09T06:32:27.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.652+0000 7f72c73e1700 1 -- 172.21.15.163:0/1375667101 shutdown_connections 2023-10-09T06:32:27.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.652+0000 7f72c73e1700 1 --2- 172.21.15.163:0/1375667101 >> v2:172.21.15.181:3300/0 conn(0x7f72c006bcd0 0x7f72c00caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:27.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.652+0000 7f72c73e1700 1 --2- 172.21.15.163:0/1375667101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72c015df50 0x7f72c015e3b0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:27.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.652+0000 7f72c73e1700 1 -- 172.21.15.163:0/1375667101 >> 172.21.15.163:0/1375667101 conn(0x7f72c00ce600 msgr2=0x7f72c00cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:27.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.652+0000 7f72c73e1700 1 -- 172.21.15.163:0/1375667101 shutdown_connections 2023-10-09T06:32:27.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.652+0000 7f72c73e1700 1 -- 172.21.15.163:0/1375667101 wait complete. 2023-10-09T06:32:27.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.653+0000 7f72c73e1700 1 Processor -- start 2023-10-09T06:32:27.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.653+0000 7f72c73e1700 1 -- start start 2023-10-09T06:32:27.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.654+0000 7f72c73e1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72c006bcd0 0x7f72c01547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:27.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.654+0000 7f72c73e1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72c006c0a0 0x7f72c015c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:27.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.654+0000 7f72c73e1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72c015df50 0x7f72c0169cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:27.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.654+0000 7f72c73e1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c016acd0 con 0x7f72c006c0a0 2023-10-09T06:32:27.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.654+0000 7f72c73e1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c016a200 con 0x7f72c015df50 2023-10-09T06:32:27.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.654+0000 7f72c73e1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c00c58e0 con 0x7f72c006bcd0 2023-10-09T06:32:27.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.654+0000 7f72c497c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72c006c0a0 0x7f72c015c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:27.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.654+0000 7f72c497c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72c006c0a0 0x7f72c015c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58660/0 (socket says 172.21.15.163:58660) 2023-10-09T06:32:27.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.654+0000 7f72c497c700 1 -- 172.21.15.163:0/1659046594 learned_addr learned my addr 172.21.15.163:0/1659046594 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:27.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.654+0000 7f72c517d700 1 --2- 172.21.15.163:0/1659046594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72c006bcd0 0x7f72c01547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:27.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.654+0000 7f72c597e700 1 --2- 172.21.15.163:0/1659046594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72c015df50 0x7f72c0169cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:27.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.654+0000 7f72c497c700 1 -- 172.21.15.163:0/1659046594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72c006bcd0 msgr2=0x7f72c01547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:27.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.654+0000 7f72c497c700 1 --2- 172.21.15.163:0/1659046594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72c006bcd0 0x7f72c01547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:27.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.654+0000 7f72c497c700 1 -- 172.21.15.163:0/1659046594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72c015df50 msgr2=0x7f72c0169cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:27.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.654+0000 7f72c497c700 1 --2- 172.21.15.163:0/1659046594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72c015df50 0x7f72c0169cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:27.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.654+0000 7f72c497c700 1 -- 172.21.15.163:0/1659046594 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b8009a20 con 0x7f72c006c0a0 2023-10-09T06:32:27.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.654+0000 7f72c597e700 1 --2- 172.21.15.163:0/1659046594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72c015df50 0x7f72c0169cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:27.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.654+0000 7f72c497c700 1 --2- 172.21.15.163:0/1659046594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72c006c0a0 0x7f72c015c5c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f72b000c5e0 tx=0x7f72b0001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:27.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.655+0000 7f72b57fa700 1 -- 172.21.15.163:0/1659046594 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b0005b30 con 0x7f72c006c0a0 2023-10-09T06:32:27.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.655+0000 7f72b57fa700 1 -- 172.21.15.163:0/1659046594 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b0005c90 con 0x7f72c006c0a0 2023-10-09T06:32:27.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.655+0000 7f72c73e1700 1 -- 172.21.15.163:0/1659046594 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c016a430 con 0x7f72c006c0a0 2023-10-09T06:32:27.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.655+0000 7f72b57fa700 1 -- 172.21.15.163:0/1659046594 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b00072e0 con 0x7f72c006c0a0 2023-10-09T06:32:27.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.655+0000 7f72c73e1700 1 -- 172.21.15.163:0/1659046594 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c016a9f0 con 0x7f72c006c0a0 2023-10-09T06:32:27.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.656+0000 7f72c73e1700 1 -- 172.21.15.163:0/1659046594 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c00c5690 con 0x7f72c006c0a0 2023-10-09T06:32:27.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.656+0000 7f72b57fa700 1 -- 172.21.15.163:0/1659046594 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f72b001b3b0 con 0x7f72c006c0a0 2023-10-09T06:32:27.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.656+0000 7f72b57fa700 1 --2- 172.21.15.163:0/1659046594 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f72a4066fc0 0x7f72a4069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:27.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.656+0000 7f72b57fa700 1 -- 172.21.15.163:0/1659046594 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f72b0083630 con 0x7f72c006c0a0 2023-10-09T06:32:27.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.656+0000 7f72c517d700 1 --2- 172.21.15.163:0/1659046594 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f72a4066fc0 0x7f72a4069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:27.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.657+0000 7f72c517d700 1 --2- 172.21.15.163:0/1659046594 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f72a4066fc0 0x7f72a4069470 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f72bc006f00 tx=0x7f72bc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:27.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.660+0000 7f72b57fa700 1 -- 172.21.15.163:0/1659046594 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72b004b5a0 con 0x7f72c006c0a0 2023-10-09T06:32:27.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.846+0000 7f72c73e1700 1 -- 172.21.15.163:0/1659046594 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72c004c3c0 con 0x7f72c006c0a0 2023-10-09T06:32:27.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.847+0000 7f72b57fa700 1 -- 172.21.15.163:0/1659046594 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+12182 (secure 0 0 0) 0x7f72b0050cb0 con 0x7f72c006c0a0 2023-10-09T06:32:27.850 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:27.850 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":59,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:32:27.112190+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:32:27.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.849+0000 7f72c73e1700 1 -- 172.21.15.163:0/1659046594 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f72a4066fc0 msgr2=0x7f72a4069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:27.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.849+0000 7f72c73e1700 1 --2- 172.21.15.163:0/1659046594 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f72a4066fc0 0x7f72a4069470 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f72bc006f00 tx=0x7f72bc008040 comp rx=0 tx=0).stop 2023-10-09T06:32:27.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.849+0000 7f72c73e1700 1 -- 172.21.15.163:0/1659046594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72c006c0a0 msgr2=0x7f72c015c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:27.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.849+0000 7f72c73e1700 1 --2- 172.21.15.163:0/1659046594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72c006c0a0 0x7f72c015c5c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f72b000c5e0 tx=0x7f72b0001070 comp rx=0 tx=0).stop 2023-10-09T06:32:27.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.849+0000 7f72c73e1700 1 -- 172.21.15.163:0/1659046594 shutdown_connections 2023-10-09T06:32:27.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.849+0000 7f72c73e1700 1 --2- 172.21.15.163:0/1659046594 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f72a4066fc0 0x7f72a4069470 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:27.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.849+0000 7f72c73e1700 1 --2- 172.21.15.163:0/1659046594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72c015df50 0x7f72c0169cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:27.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.849+0000 7f72c73e1700 1 --2- 172.21.15.163:0/1659046594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72c006bcd0 0x7f72c01547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:27.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.849+0000 7f72c73e1700 1 --2- 172.21.15.163:0/1659046594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72c006c0a0 0x7f72c015c5c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:27.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.849+0000 7f72c73e1700 1 -- 172.21.15.163:0/1659046594 >> 172.21.15.163:0/1659046594 conn(0x7f72c00ce600 msgr2=0x7f72c006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:27.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.849+0000 7f72c73e1700 1 -- 172.21.15.163:0/1659046594 shutdown_connections 2023-10-09T06:32:27.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.849+0000 7f72c73e1700 1 -- 172.21.15.163:0/1659046594 wait complete. 2023-10-09T06:32:27.862 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-09T06:32:27.862 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:32:27.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.979+0000 7f5532dda700 1 Processor -- start 2023-10-09T06:32:27.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.979+0000 7f5532dda700 1 -- start start 2023-10-09T06:32:27.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.980+0000 7f5532dda700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f552c06c0a0 0x7f552c0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:27.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.980+0000 7f5532dda700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f552c15df50 0x7f552c15e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:27.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.980+0000 7f5532dda700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f552c15d7e0 con 0x7f552c06bcd0 2023-10-09T06:32:27.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.980+0000 7f5532dda700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f552c15dac0 con 0x7f552c06c0a0 2023-10-09T06:32:27.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.980+0000 7f5532dda700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f552c15aa30 con 0x7f552c15df50 2023-10-09T06:32:27.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.980+0000 7f5523fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f552c06c0a0 0x7f552c0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:27.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.980+0000 7f5530b76700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f552c06bcd0 0x7f552c15a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:54044/0 (socket says 172.21.15.163:54044) 2023-10-09T06:32:27.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.980+0000 7f5523fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f552c06c0a0 0x7f552c0caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58666/0 (socket says 172.21.15.163:58666) 2023-10-09T06:32:27.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.980+0000 7f5530b76700 1 -- 172.21.15.163:0/1888333141 learned_addr learned my addr 172.21.15.163:0/1888333141 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:27.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.981+0000 7f5531377700 1 --2- 172.21.15.163:0/1888333141 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f552c15df50 0x7f552c15e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:27.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.981+0000 7f5523fff700 1 -- 172.21.15.163:0/1888333141 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f552c15df50 msgr2=0x7f552c15e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:27.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.981+0000 7f5523fff700 1 --2- 172.21.15.163:0/1888333141 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f552c15df50 0x7f552c15e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:27.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.981+0000 7f5523fff700 1 -- 172.21.15.163:0/1888333141 >> v1:172.21.15.163:6789/0 conn(0x7f552c06bcd0 legacy=0x7f552c15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:27.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.981+0000 7f5523fff700 1 -- 172.21.15.163:0/1888333141 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f552c15ad10 con 0x7f552c06c0a0 2023-10-09T06:32:27.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.981+0000 7f5531377700 1 --2- 172.21.15.163:0/1888333141 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f552c15df50 0x7f552c15e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:32:27.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.981+0000 7f55237fe700 1 -- 172.21.15.163:0/1888333141 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1063345810 0 0) 0x7f552c15d7e0 con 0x7f552c06bcd0 2023-10-09T06:32:27.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.981+0000 7f5523fff700 1 --2- 172.21.15.163:0/1888333141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f552c06c0a0 0x7f552c0caa90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f551c0098e0 tx=0x7f551c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b010a00f66b8f4c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:27.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.981+0000 7f55237fe700 1 -- 172.21.15.163:0/1888333141 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c009e80 con 0x7f552c06c0a0 2023-10-09T06:32:27.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.981+0000 7f55237fe700 1 -- 172.21.15.163:0/1888333141 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551c00b860 con 0x7f552c06c0a0 2023-10-09T06:32:27.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.981+0000 7f55237fe700 1 -- 172.21.15.163:0/1888333141 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c00bbe0 con 0x7f552c06c0a0 2023-10-09T06:32:27.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.982+0000 7f5532dda700 1 -- 172.21.15.163:0/1888333141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f552c06c0a0 msgr2=0x7f552c0caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:27.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.982+0000 7f5532dda700 1 --2- 172.21.15.163:0/1888333141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f552c06c0a0 0x7f552c0caa90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f551c0098e0 tx=0x7f551c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:27.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.982+0000 7f5532dda700 1 -- 172.21.15.163:0/1888333141 shutdown_connections 2023-10-09T06:32:27.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.982+0000 7f5532dda700 1 --2- 172.21.15.163:0/1888333141 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f552c15df50 0x7f552c15e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:27.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.982+0000 7f5532dda700 1 --2- 172.21.15.163:0/1888333141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f552c06c0a0 0x7f552c0caa90 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:27.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.982+0000 7f5532dda700 1 -- 172.21.15.163:0/1888333141 >> 172.21.15.163:0/1888333141 conn(0x7f552c0ce600 msgr2=0x7f552c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:27.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.982+0000 7f5532dda700 1 -- 172.21.15.163:0/1888333141 shutdown_connections 2023-10-09T06:32:27.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.982+0000 7f5532dda700 1 -- 172.21.15.163:0/1888333141 wait complete. 2023-10-09T06:32:27.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.983+0000 7f5532dda700 1 Processor -- start 2023-10-09T06:32:27.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.983+0000 7f5532dda700 1 -- start start 2023-10-09T06:32:27.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.984+0000 7f5532dda700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f552c06bcd0 0x7f552c06d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:27.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.984+0000 7f5532dda700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f552c06c0a0 0x7f552c0b2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:27.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.984+0000 7f5532dda700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f552c15df50 0x7f552c157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:27.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.984+0000 7f5532dda700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552c0c5bc0 con 0x7f552c15df50 2023-10-09T06:32:27.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.984+0000 7f5532dda700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552c159b80 con 0x7f552c06c0a0 2023-10-09T06:32:27.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.984+0000 7f5532dda700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f552c1588a0 con 0x7f552c06bcd0 2023-10-09T06:32:27.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.984+0000 7f5531377700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f552c15df50 0x7f552c157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:27.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.984+0000 7f5530b76700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f552c06bcd0 0x7f552c06d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:27.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.984+0000 7f5531377700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f552c15df50 0x7f552c157de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58668/0 (socket says 172.21.15.163:58668) 2023-10-09T06:32:27.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.984+0000 7f5531377700 1 -- 172.21.15.163:0/2421560233 learned_addr learned my addr 172.21.15.163:0/2421560233 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:27.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.984+0000 7f5523fff700 1 --2- 172.21.15.163:0/2421560233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f552c06c0a0 0x7f552c0b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:27.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.984+0000 7f5531377700 1 -- 172.21.15.163:0/2421560233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f552c06bcd0 msgr2=0x7f552c06d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:27.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.984+0000 7f5531377700 1 --2- 172.21.15.163:0/2421560233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f552c06bcd0 0x7f552c06d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:27.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.984+0000 7f5531377700 1 -- 172.21.15.163:0/2421560233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f552c06c0a0 msgr2=0x7f552c0b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:27.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.984+0000 7f5531377700 1 --2- 172.21.15.163:0/2421560233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f552c06c0a0 0x7f552c0b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:27.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.984+0000 7f5531377700 1 -- 172.21.15.163:0/2421560233 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551c009420 con 0x7f552c15df50 2023-10-09T06:32:27.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.984+0000 7f5531377700 1 --2- 172.21.15.163:0/2421560233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f552c15df50 0x7f552c157de0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f552400c880 tx=0x7f5524008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:27.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.985+0000 7f5520ff9700 1 -- 172.21.15.163:0/2421560233 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5524011600 con 0x7f552c15df50 2023-10-09T06:32:27.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.985+0000 7f5520ff9700 1 -- 172.21.15.163:0/2421560233 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5524011760 con 0x7f552c15df50 2023-10-09T06:32:27.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.985+0000 7f5532dda700 1 -- 172.21.15.163:0/2421560233 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f552c158b80 con 0x7f552c15df50 2023-10-09T06:32:27.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.985+0000 7f5520ff9700 1 -- 172.21.15.163:0/2421560233 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5524011b20 con 0x7f552c15df50 2023-10-09T06:32:27.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.985+0000 7f5532dda700 1 -- 172.21.15.163:0/2421560233 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f552c06db70 con 0x7f552c15df50 2023-10-09T06:32:27.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.986+0000 7f5532dda700 1 -- 172.21.15.163:0/2421560233 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f552c158dc0 con 0x7f552c15df50 2023-10-09T06:32:27.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.986+0000 7f5520ff9700 1 -- 172.21.15.163:0/2421560233 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f55240193b0 con 0x7f552c15df50 2023-10-09T06:32:27.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.986+0000 7f5520ff9700 1 --2- 172.21.15.163:0/2421560233 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5518066fc0 0x7f5518069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:27.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.986+0000 7f5520ff9700 1 -- 172.21.15.163:0/2421560233 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f5524083ef0 con 0x7f552c15df50 2023-10-09T06:32:27.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.986+0000 7f5530b76700 1 --2- 172.21.15.163:0/2421560233 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5518066fc0 0x7f5518069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:27.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.987+0000 7f5530b76700 1 --2- 172.21.15.163:0/2421560233 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5518066fc0 0x7f5518069470 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5528007240 tx=0x7f5528001970 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:27.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:27.990+0000 7f5520ff9700 1 -- 172.21.15.163:0/2421560233 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f552c158dc0 con 0x7f552c15df50 2023-10-09T06:32:28.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.172+0000 7f5532dda700 1 -- 172.21.15.163:0/2421560233 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f552c158dc0 con 0x7f552c15df50 2023-10-09T06:32:28.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.173+0000 7f5520ff9700 1 -- 172.21.15.163:0/2421560233 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5524015020 con 0x7f552c15df50 2023-10-09T06:32:28.174 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:32:28.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.175+0000 7f5532dda700 1 -- 172.21.15.163:0/2421560233 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5518066fc0 msgr2=0x7f5518069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:28.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.175+0000 7f5532dda700 1 --2- 172.21.15.163:0/2421560233 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5518066fc0 0x7f5518069470 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5528007240 tx=0x7f5528001970 comp rx=0 tx=0).stop 2023-10-09T06:32:28.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.175+0000 7f5532dda700 1 -- 172.21.15.163:0/2421560233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f552c15df50 msgr2=0x7f552c157de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:28.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.175+0000 7f5532dda700 1 --2- 172.21.15.163:0/2421560233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f552c15df50 0x7f552c157de0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f552400c880 tx=0x7f5524008040 comp rx=0 tx=0).stop 2023-10-09T06:32:28.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.175+0000 7f5532dda700 1 -- 172.21.15.163:0/2421560233 shutdown_connections 2023-10-09T06:32:28.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.175+0000 7f5532dda700 1 --2- 172.21.15.163:0/2421560233 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5518066fc0 0x7f5518069470 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:28.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.175+0000 7f5532dda700 1 --2- 172.21.15.163:0/2421560233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f552c15df50 0x7f552c157de0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:28.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.175+0000 7f5532dda700 1 --2- 172.21.15.163:0/2421560233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f552c06bcd0 0x7f552c06d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:28.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.175+0000 7f5532dda700 1 --2- 172.21.15.163:0/2421560233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f552c06c0a0 0x7f552c0b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:28.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.175+0000 7f5532dda700 1 -- 172.21.15.163:0/2421560233 >> 172.21.15.163:0/2421560233 conn(0x7f552c0ce600 msgr2=0x7f552c06a730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:28.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.175+0000 7f5532dda700 1 -- 172.21.15.163:0/2421560233 shutdown_connections 2023-10-09T06:32:28.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.175+0000 7f5532dda700 1 -- 172.21.15.163:0/2421560233 wait complete. 2023-10-09T06:32:28.190 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-09T06:32:28.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.303+0000 7f65eb6fc700 1 Processor -- start 2023-10-09T06:32:28.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.303+0000 7f65eb6fc700 1 -- start start 2023-10-09T06:32:28.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.304+0000 7f65eb6fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f65e4141580 0x7f65e40cd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:28.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.304+0000 7f65eb6fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f65e40caf80 0x7f65e40b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:28.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.304+0000 7f65eb6fc700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e40ccfb0 con 0x7f65e41411b0 2023-10-09T06:32:28.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.304+0000 7f65eb6fc700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e40b3e50 con 0x7f65e40caf80 2023-10-09T06:32:28.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.304+0000 7f65eb6fc700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e40b40d0 con 0x7f65e4141580 2023-10-09T06:32:28.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.304+0000 7f65e9c99700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f65e40caf80 0x7f65e40b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:28.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.304+0000 7f65e9c99700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f65e40caf80 0x7f65e40b1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58682/0 (socket says 172.21.15.163:58682) 2023-10-09T06:32:28.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.304+0000 7f65e9c99700 1 -- 172.21.15.163:0/4155614853 learned_addr learned my addr 172.21.15.163:0/4155614853 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:28.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.305+0000 7f65e8c97700 1 --2- 172.21.15.163:0/4155614853 >> v2:172.21.15.181:3300/0 conn(0x7f65e4141580 0x7f65e40cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:28.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.305+0000 7f65e9c99700 1 -- 172.21.15.163:0/4155614853 >> v2:172.21.15.181:3300/0 conn(0x7f65e4141580 msgr2=0x7f65e40cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:28.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.305+0000 7f65e9c99700 1 --2- 172.21.15.163:0/4155614853 >> v2:172.21.15.181:3300/0 conn(0x7f65e4141580 0x7f65e40cd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:28.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.305+0000 7f65e9c99700 1 -- 172.21.15.163:0/4155614853 >> v1:172.21.15.181:6789/0 conn(0x7f65e41411b0 legacy=0x7f65e415bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:28.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.305+0000 7f65e9c99700 1 -- 172.21.15.163:0/4155614853 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e40b43b0 con 0x7f65e40caf80 2023-10-09T06:32:28.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.305+0000 7f65e9c99700 1 --2- 172.21.15.163:0/4155614853 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65e40caf80 0x7f65e40b1cc0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f65e40ce300 tx=0x7f65dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d9e9f095b66541 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:28.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.305+0000 7f65e3fff700 1 -- 172.21.15.163:0/4155614853 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc00b860 con 0x7f65e40caf80 2023-10-09T06:32:28.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.305+0000 7f65e3fff700 1 -- 172.21.15.163:0/4155614853 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65dc00b9c0 con 0x7f65e40caf80 2023-10-09T06:32:28.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.305+0000 7f65e3fff700 1 -- 172.21.15.163:0/4155614853 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc00bd60 con 0x7f65e40caf80 2023-10-09T06:32:28.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.305+0000 7f65eb6fc700 1 -- 172.21.15.163:0/4155614853 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65e40caf80 msgr2=0x7f65e40b1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:28.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.305+0000 7f65eb6fc700 1 --2- 172.21.15.163:0/4155614853 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65e40caf80 0x7f65e40b1cc0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f65e40ce300 tx=0x7f65dc009250 comp rx=0 tx=0).stop 2023-10-09T06:32:28.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.306+0000 7f65eb6fc700 1 -- 172.21.15.163:0/4155614853 shutdown_connections 2023-10-09T06:32:28.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.306+0000 7f65eb6fc700 1 --2- 172.21.15.163:0/4155614853 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65e40caf80 0x7f65e40b1cc0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:28.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.306+0000 7f65eb6fc700 1 --2- 172.21.15.163:0/4155614853 >> v2:172.21.15.181:3300/0 conn(0x7f65e4141580 0x7f65e40cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:28.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.306+0000 7f65eb6fc700 1 -- 172.21.15.163:0/4155614853 >> 172.21.15.163:0/4155614853 conn(0x7f65e406bb50 msgr2=0x7f65e406bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:28.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.306+0000 7f65eb6fc700 1 -- 172.21.15.163:0/4155614853 shutdown_connections 2023-10-09T06:32:28.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.306+0000 7f65eb6fc700 1 -- 172.21.15.163:0/4155614853 wait complete. 2023-10-09T06:32:28.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.307+0000 7f65eb6fc700 1 Processor -- start 2023-10-09T06:32:28.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.307+0000 7f65eb6fc700 1 -- start start 2023-10-09T06:32:28.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.307+0000 7f65eb6fc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65e40caf80 0x7f65e415c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:28.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.308+0000 7f65eb6fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e41411b0 0x7f65e40c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:28.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.308+0000 7f65eb6fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65e4141580 0x7f65e40c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:28.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.308+0000 7f65eb6fc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e413e9d0 con 0x7f65e4141580 2023-10-09T06:32:28.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.308+0000 7f65eb6fc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e40c9bc0 con 0x7f65e40caf80 2023-10-09T06:32:28.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.308+0000 7f65eb6fc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e415cb70 con 0x7f65e41411b0 2023-10-09T06:32:28.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.308+0000 7f65e9c99700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65e4141580 0x7f65e40c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:28.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.308+0000 7f65e9c99700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65e4141580 0x7f65e40c89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58684/0 (socket says 172.21.15.163:58684) 2023-10-09T06:32:28.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.308+0000 7f65e9c99700 1 -- 172.21.15.163:0/3452879043 learned_addr learned my addr 172.21.15.163:0/3452879043 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:28.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.308+0000 7f65e8c97700 1 --2- 172.21.15.163:0/3452879043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e41411b0 0x7f65e40c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:28.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.308+0000 7f65e9498700 1 --2- 172.21.15.163:0/3452879043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65e40caf80 0x7f65e415c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:28.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.308+0000 7f65e9c99700 1 -- 172.21.15.163:0/3452879043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e41411b0 msgr2=0x7f65e40c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:28.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.308+0000 7f65e9c99700 1 --2- 172.21.15.163:0/3452879043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e41411b0 0x7f65e40c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:28.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.308+0000 7f65e9c99700 1 -- 172.21.15.163:0/3452879043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65e40caf80 msgr2=0x7f65e415c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:28.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.308+0000 7f65e9c99700 1 --2- 172.21.15.163:0/3452879043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65e40caf80 0x7f65e415c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:28.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.308+0000 7f65e9c99700 1 -- 172.21.15.163:0/3452879043 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65dc0094f0 con 0x7f65e4141580 2023-10-09T06:32:28.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.308+0000 7f65e9c99700 1 --2- 172.21.15.163:0/3452879043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65e4141580 0x7f65e40c89a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f65dc006fd0 tx=0x7f65dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:28.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.308+0000 7f65e17fa700 1 -- 172.21.15.163:0/3452879043 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc011e40 con 0x7f65e4141580 2023-10-09T06:32:28.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.308+0000 7f65e17fa700 1 -- 172.21.15.163:0/3452879043 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65dc01a630 con 0x7f65e4141580 2023-10-09T06:32:28.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.308+0000 7f65eb6fc700 1 -- 172.21.15.163:0/3452879043 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e40c8fb0 con 0x7f65e4141580 2023-10-09T06:32:28.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.308+0000 7f65e17fa700 1 -- 172.21.15.163:0/3452879043 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc01a9f0 con 0x7f65e4141580 2023-10-09T06:32:28.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.308+0000 7f65eb6fc700 1 -- 172.21.15.163:0/3452879043 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e40c9570 con 0x7f65e4141580 2023-10-09T06:32:28.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.309+0000 7f65eb6fc700 1 -- 172.21.15.163:0/3452879043 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e40c9d50 con 0x7f65e4141580 2023-10-09T06:32:28.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.310+0000 7f65e17fa700 1 -- 172.21.15.163:0/3452879043 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f65dc02d440 con 0x7f65e4141580 2023-10-09T06:32:28.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.310+0000 7f65e17fa700 1 --2- 172.21.15.163:0/3452879043 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f65d0066fc0 0x7f65d0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:28.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.310+0000 7f65e17fa700 1 -- 172.21.15.163:0/3452879043 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f65dc085e20 con 0x7f65e4141580 2023-10-09T06:32:28.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.310+0000 7f65e9498700 1 --2- 172.21.15.163:0/3452879043 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f65d0066fc0 0x7f65d0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:28.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.311+0000 7f65e9498700 1 --2- 172.21.15.163:0/3452879043 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f65d0066fc0 0x7f65d0069470 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f65d4007fd0 tx=0x7f65d40016f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:28.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.314+0000 7f65e17fa700 1 -- 172.21.15.163:0/3452879043 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f65dc0533f0 con 0x7f65e4141580 2023-10-09T06:32:28.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.501+0000 7f65eb6fc700 1 -- 172.21.15.163:0/3452879043 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f65e404c3c0 con 0x7f65e4141580 2023-10-09T06:32:28.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.501+0000 7f65e17fa700 1 -- 172.21.15.163:0/3452879043 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v59) v1 ==== 84+0+65 (secure 0 0 0) 0x7f65dc015090 con 0x7f65e4141580 2023-10-09T06:32:28.502 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:28.502 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:32:28.503 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:32:28.503 INFO:teuthology.orchestra.run.smithi163.stdout: "poolnum": 1, 2023-10-09T06:32:28.503 INFO:teuthology.orchestra.run.smithi163.stdout: "poolname": ".mgr" 2023-10-09T06:32:28.503 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:32:28.503 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:32:28.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.503+0000 7f65eb6fc700 1 -- 172.21.15.163:0/3452879043 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f65d0066fc0 msgr2=0x7f65d0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:28.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.503+0000 7f65eb6fc700 1 --2- 172.21.15.163:0/3452879043 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f65d0066fc0 0x7f65d0069470 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f65d4007fd0 tx=0x7f65d40016f0 comp rx=0 tx=0).stop 2023-10-09T06:32:28.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.503+0000 7f65eb6fc700 1 -- 172.21.15.163:0/3452879043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65e4141580 msgr2=0x7f65e40c89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:28.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.503+0000 7f65eb6fc700 1 --2- 172.21.15.163:0/3452879043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65e4141580 0x7f65e40c89a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f65dc006fd0 tx=0x7f65dc017000 comp rx=0 tx=0).stop 2023-10-09T06:32:28.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.504+0000 7f65eb6fc700 1 -- 172.21.15.163:0/3452879043 shutdown_connections 2023-10-09T06:32:28.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.504+0000 7f65eb6fc700 1 --2- 172.21.15.163:0/3452879043 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f65d0066fc0 0x7f65d0069470 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:28.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.504+0000 7f65eb6fc700 1 --2- 172.21.15.163:0/3452879043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65e4141580 0x7f65e40c89a0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:28.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.504+0000 7f65eb6fc700 1 --2- 172.21.15.163:0/3452879043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65e40caf80 0x7f65e415c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:28.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.504+0000 7f65eb6fc700 1 --2- 172.21.15.163:0/3452879043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e41411b0 0x7f65e40c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:28.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.504+0000 7f65eb6fc700 1 -- 172.21.15.163:0/3452879043 >> 172.21.15.163:0/3452879043 conn(0x7f65e406bb50 msgr2=0x7f65e40aeff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:28.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.504+0000 7f65eb6fc700 1 -- 172.21.15.163:0/3452879043 shutdown_connections 2023-10-09T06:32:28.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.504+0000 7f65eb6fc700 1 -- 172.21.15.163:0/3452879043 wait complete. 2023-10-09T06:32:28.518 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-09T06:32:28.518 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-09T06:32:28.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.635+0000 7f9281649700 1 Processor -- start 2023-10-09T06:32:28.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.635+0000 7f9281649700 1 -- start start 2023-10-09T06:32:28.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.636+0000 7f9281649700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f927c0b0790 0x7f927c0cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:28.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.636+0000 7f9281649700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f927c0cfa00 con 0x7f927c1591e0 2023-10-09T06:32:28.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.636+0000 7f9281649700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f927c0cb580 con 0x7f927c0b03c0 2023-10-09T06:32:28.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.636+0000 7f9281649700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f927c0cf160 con 0x7f927c0b0790 2023-10-09T06:32:28.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.636+0000 7f927b7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f927c0b0790 0x7f927c0cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:28.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.636+0000 7f9280e48700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f927c1591e0 0x7f927c159600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:44760/0 (socket says 172.21.15.163:44760) 2023-10-09T06:32:28.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.636+0000 7f9280e48700 1 -- 172.21.15.163:0/4247236575 learned_addr learned my addr 172.21.15.163:0/4247236575 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:28.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.636+0000 7f927b7fe700 1 -- 172.21.15.163:0/4247236575 >> v1:172.21.15.181:6789/0 conn(0x7f927c0b03c0 legacy=0x7f927c0b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:28.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.636+0000 7f927b7fe700 1 -- 172.21.15.163:0/4247236575 >> v1:172.21.15.163:6789/0 conn(0x7f927c1591e0 legacy=0x7f927c159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:28.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.636+0000 7f927b7fe700 1 -- 172.21.15.163:0/4247236575 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927c0cf440 con 0x7f927c0b0790 2023-10-09T06:32:28.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.636+0000 7f927affd700 1 -- 172.21.15.163:0/4247236575 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3599650640 0 0) 0x7f927c0cfa00 con 0x7f927c1591e0 2023-10-09T06:32:28.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.637+0000 7f927b7fe700 1 --2- 172.21.15.163:0/4247236575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f927c0b0790 0x7f927c0cea70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f9270009910 tx=0x7f92700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1525fb748ca7a485 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:28.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.637+0000 7f927affd700 1 -- 172.21.15.163:0/4247236575 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927000b7e0 con 0x7f927c0b0790 2023-10-09T06:32:28.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.637+0000 7f927affd700 1 -- 172.21.15.163:0/4247236575 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927000b940 con 0x7f927c0b0790 2023-10-09T06:32:28.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.637+0000 7f927affd700 1 -- 172.21.15.163:0/4247236575 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927000bce0 con 0x7f927c0b0790 2023-10-09T06:32:28.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.637+0000 7f9281649700 1 -- 172.21.15.163:0/4247236575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f927c0b0790 msgr2=0x7f927c0cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:28.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.637+0000 7f9281649700 1 --2- 172.21.15.163:0/4247236575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f927c0b0790 0x7f927c0cea70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f9270009910 tx=0x7f92700091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:28.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.637+0000 7f9281649700 1 -- 172.21.15.163:0/4247236575 shutdown_connections 2023-10-09T06:32:28.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.637+0000 7f9281649700 1 --2- 172.21.15.163:0/4247236575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f927c0b0790 0x7f927c0cea70 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:28.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.637+0000 7f9281649700 1 -- 172.21.15.163:0/4247236575 >> 172.21.15.163:0/4247236575 conn(0x7f927c063230 msgr2=0x7f927c063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:28.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.637+0000 7f9281649700 1 -- 172.21.15.163:0/4247236575 shutdown_connections 2023-10-09T06:32:28.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.637+0000 7f9281649700 1 -- 172.21.15.163:0/4247236575 wait complete. 2023-10-09T06:32:28.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.638+0000 7f9281649700 1 Processor -- start 2023-10-09T06:32:28.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.639+0000 7f9281649700 1 -- start start 2023-10-09T06:32:28.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.639+0000 7f9281649700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f927c0b03c0 0x7f927c0c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:28.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.639+0000 7f9281649700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f927c0b0790 0x7f927c0c4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:28.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.639+0000 7f9281649700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f927c1591e0 0x7f927c0c6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:28.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.639+0000 7f9281649700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927c0c4e90 con 0x7f927c0b0790 2023-10-09T06:32:28.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.639+0000 7f9281649700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927c0c6660 con 0x7f927c0b03c0 2023-10-09T06:32:28.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.639+0000 7f9281649700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f927c0c5080 con 0x7f927c1591e0 2023-10-09T06:32:28.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.639+0000 7f927b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f927c0b0790 0x7f927c0c4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:28.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.639+0000 7f927b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f927c0b0790 0x7f927c0c4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58820/0 (socket says 172.21.15.163:58820) 2023-10-09T06:32:28.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.639+0000 7f927b7fe700 1 -- 172.21.15.163:0/863647392 learned_addr learned my addr 172.21.15.163:0/863647392 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:28.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.639+0000 7f927bfff700 1 --2- 172.21.15.163:0/863647392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f927c0b03c0 0x7f927c0c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:28.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.640+0000 7f927b7fe700 1 -- 172.21.15.163:0/863647392 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f927c1591e0 msgr2=0x7f927c0c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:28.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.640+0000 7f927b7fe700 1 --2- 172.21.15.163:0/863647392 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f927c1591e0 0x7f927c0c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:28.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.640+0000 7f927b7fe700 1 -- 172.21.15.163:0/863647392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f927c0b03c0 msgr2=0x7f927c0c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.640+0000 7f927b7fe700 1 --2- 172.21.15.163:0/863647392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f927c0b03c0 0x7f927c0c9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.640+0000 7f927b7fe700 1 -- 172.21.15.163:0/863647392 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9270009420 con 0x7f927c0b0790 2023-10-09T06:32:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.640+0000 7f927bfff700 1 --2- 172.21.15.163:0/863647392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f927c0b03c0 0x7f927c0c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:32:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.640+0000 7f927b7fe700 1 --2- 172.21.15.163:0/863647392 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f927c0b0790 0x7f927c0c4950 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f927c0cfe40 tx=0x7f9270016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.640+0000 7f9263fff700 1 -- 172.21.15.163:0/863647392 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9270006af0 con 0x7f927c0b0790 2023-10-09T06:32:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.640+0000 7f9263fff700 1 -- 172.21.15.163:0/863647392 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9270020070 con 0x7f927c0b0790 2023-10-09T06:32:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.640+0000 7f9281649700 1 -- 172.21.15.163:0/863647392 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f927c0c5360 con 0x7f927c0b0790 2023-10-09T06:32:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.640+0000 7f9263fff700 1 -- 172.21.15.163:0/863647392 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927001b450 con 0x7f927c0b0790 2023-10-09T06:32:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.640+0000 7f9281649700 1 -- 172.21.15.163:0/863647392 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f927c0c5920 con 0x7f927c0b0790 2023-10-09T06:32:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.641+0000 7f9281649700 1 -- 172.21.15.163:0/863647392 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f927c04c3c0 con 0x7f927c0b0790 2023-10-09T06:32:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.641+0000 7f9263fff700 1 -- 172.21.15.163:0/863647392 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f9270022030 con 0x7f927c0b0790 2023-10-09T06:32:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.642+0000 7f9263fff700 1 --2- 172.21.15.163:0/863647392 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9268066fc0 0x7f9268069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:28.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.642+0000 7f9263fff700 1 -- 172.21.15.163:0/863647392 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f9270085ff0 con 0x7f927c0b0790 2023-10-09T06:32:28.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.642+0000 7f927bfff700 1 --2- 172.21.15.163:0/863647392 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9268066fc0 0x7f9268069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:28.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.642+0000 7f927bfff700 1 --2- 172.21.15.163:0/863647392 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9268066fc0 0x7f9268069470 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f926c002880 tx=0x7f926c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:28.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.646+0000 7f9263fff700 1 -- 172.21.15.163:0/863647392 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f92700535c0 con 0x7f927c0b0790 2023-10-09T06:32:28.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:28.829+0000 7f9281649700 1 -- 172.21.15.163:0/863647392 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f927c04c3c0 con 0x7f927c0b0790 2023-10-09T06:32:29.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.487+0000 7f9263fff700 1 -- 172.21.15.163:0/863647392 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v60) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9270056be0 con 0x7f927c0b0790 2023-10-09T06:32:29.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.489+0000 7f9281649700 1 -- 172.21.15.163:0/863647392 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9268066fc0 msgr2=0x7f9268069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:29.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.489+0000 7f9281649700 1 --2- 172.21.15.163:0/863647392 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9268066fc0 0x7f9268069470 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f926c002880 tx=0x7f926c009040 comp rx=0 tx=0).stop 2023-10-09T06:32:29.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.489+0000 7f9281649700 1 -- 172.21.15.163:0/863647392 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f927c0b0790 msgr2=0x7f927c0c4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:29.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.489+0000 7f9281649700 1 --2- 172.21.15.163:0/863647392 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f927c0b0790 0x7f927c0c4950 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f927c0cfe40 tx=0x7f9270016000 comp rx=0 tx=0).stop 2023-10-09T06:32:29.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.489+0000 7f9281649700 1 -- 172.21.15.163:0/863647392 shutdown_connections 2023-10-09T06:32:29.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.489+0000 7f9281649700 1 --2- 172.21.15.163:0/863647392 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9268066fc0 0x7f9268069470 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:29.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.489+0000 7f9281649700 1 --2- 172.21.15.163:0/863647392 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f927c1591e0 0x7f927c0c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:29.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.489+0000 7f9281649700 1 --2- 172.21.15.163:0/863647392 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f927c0b03c0 0x7f927c0c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:29.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.489+0000 7f9281649700 1 --2- 172.21.15.163:0/863647392 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f927c0b0790 0x7f927c0c4950 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:29.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.489+0000 7f9281649700 1 -- 172.21.15.163:0/863647392 >> 172.21.15.163:0/863647392 conn(0x7f927c063230 msgr2=0x7f927c0ce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:29.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.489+0000 7f9281649700 1 -- 172.21.15.163:0/863647392 shutdown_connections 2023-10-09T06:32:29.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.489+0000 7f9281649700 1 -- 172.21.15.163:0/863647392 wait complete. 2023-10-09T06:32:29.494 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_metadata' created 2023-10-09T06:32:29.505 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-09T06:32:29.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.625+0000 7f7dcf59e700 1 Processor -- start 2023-10-09T06:32:29.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.625+0000 7f7dcf59e700 1 -- start start 2023-10-09T06:32:29.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.626+0000 7f7dcf59e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7dd01595d0 0x7f7dd00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:29.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.626+0000 7f7dcf59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7dd00cf480 0x7f7dd015d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:29.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.626+0000 7f7dcf59e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dd00b10f0 con 0x7f7dd015d760 2023-10-09T06:32:29.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.626+0000 7f7dcf59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dd00cd530 con 0x7f7dd00cf480 2023-10-09T06:32:29.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.626+0000 7f7dcf59e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dd015db50 con 0x7f7dd01595d0 2023-10-09T06:32:29.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.627+0000 7f7dcdd9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7dd00cf480 0x7f7dd015d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:29.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.627+0000 7f7dcdd9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7dd00cf480 0x7f7dd015d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58830/0 (socket says 172.21.15.163:58830) 2023-10-09T06:32:29.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.627+0000 7f7dcdd9b700 1 -- 172.21.15.163:0/3742125149 learned_addr learned my addr 172.21.15.163:0/3742125149 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:29.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.627+0000 7f7dce59c700 1 --2- 172.21.15.163:0/3742125149 >> v2:172.21.15.181:3300/0 conn(0x7f7dd01595d0 0x7f7dd00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:29.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.627+0000 7f7dcdd9b700 1 -- 172.21.15.163:0/3742125149 >> v2:172.21.15.181:3300/0 conn(0x7f7dd01595d0 msgr2=0x7f7dd00ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:29.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.627+0000 7f7dcdd9b700 1 --2- 172.21.15.163:0/3742125149 >> v2:172.21.15.181:3300/0 conn(0x7f7dd01595d0 0x7f7dd00ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:29.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.627+0000 7f7dcdd9b700 1 -- 172.21.15.163:0/3742125149 >> v1:172.21.15.181:6789/0 conn(0x7f7dd015d760 legacy=0x7f7dd01599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:29.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.627+0000 7f7dcdd9b700 1 -- 172.21.15.163:0/3742125149 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd015e000 con 0x7f7dd00cf480 2023-10-09T06:32:29.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.627+0000 7f7dcdd9b700 1 --2- 172.21.15.163:0/3742125149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7dd00cf480 0x7f7dd015d220 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7dd006a4d0 tx=0x7f7dc4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d550b92bfbc06a server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:29.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.627+0000 7f7dcd59a700 1 -- 172.21.15.163:0/3742125149 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc400b860 con 0x7f7dd00cf480 2023-10-09T06:32:29.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.627+0000 7f7dcd59a700 1 -- 172.21.15.163:0/3742125149 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc400b9c0 con 0x7f7dd00cf480 2023-10-09T06:32:29.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.627+0000 7f7dcd59a700 1 -- 172.21.15.163:0/3742125149 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc400bd60 con 0x7f7dd00cf480 2023-10-09T06:32:29.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.628+0000 7f7dcf59e700 1 -- 172.21.15.163:0/3742125149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7dd00cf480 msgr2=0x7f7dd015d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:29.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.628+0000 7f7dcf59e700 1 --2- 172.21.15.163:0/3742125149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7dd00cf480 0x7f7dd015d220 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7dd006a4d0 tx=0x7f7dc4009250 comp rx=0 tx=0).stop 2023-10-09T06:32:29.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.628+0000 7f7dcf59e700 1 -- 172.21.15.163:0/3742125149 shutdown_connections 2023-10-09T06:32:29.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.628+0000 7f7dcf59e700 1 --2- 172.21.15.163:0/3742125149 >> v2:172.21.15.181:3300/0 conn(0x7f7dd01595d0 0x7f7dd00ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:29.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.628+0000 7f7dcf59e700 1 --2- 172.21.15.163:0/3742125149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7dd00cf480 0x7f7dd015d220 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:29.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.628+0000 7f7dcf59e700 1 -- 172.21.15.163:0/3742125149 >> 172.21.15.163:0/3742125149 conn(0x7f7dd00d05f0 msgr2=0x7f7dd00d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:29.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.628+0000 7f7dcf59e700 1 -- 172.21.15.163:0/3742125149 shutdown_connections 2023-10-09T06:32:29.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.628+0000 7f7dcf59e700 1 -- 172.21.15.163:0/3742125149 wait complete. 2023-10-09T06:32:29.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.629+0000 7f7dcf59e700 1 Processor -- start 2023-10-09T06:32:29.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.629+0000 7f7dcf59e700 1 -- start start 2023-10-09T06:32:29.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.630+0000 7f7dcf59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dd00cf480 0x7f7dd016a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:29.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.630+0000 7f7dcf59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7dd01595d0 0x7f7dd01523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:29.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.630+0000 7f7dcf59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7dd015d760 0x7f7dd0156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:29.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.630+0000 7f7dcf59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd016abe0 con 0x7f7dd01595d0 2023-10-09T06:32:29.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.630+0000 7f7dcf59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd00c7400 con 0x7f7dd00cf480 2023-10-09T06:32:29.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.630+0000 7f7dcf59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd0154db0 con 0x7f7dd015d760 2023-10-09T06:32:29.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.630+0000 7f7dcdd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7dd01595d0 0x7f7dd01523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:29.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.630+0000 7f7dcdd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7dd01595d0 0x7f7dd01523d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58846/0 (socket says 172.21.15.163:58846) 2023-10-09T06:32:29.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.630+0000 7f7dcdd9b700 1 -- 172.21.15.163:0/3896816462 learned_addr learned my addr 172.21.15.163:0/3896816462 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:29.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.630+0000 7f7dce59c700 1 --2- 172.21.15.163:0/3896816462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dd00cf480 0x7f7dd016a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:29.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.630+0000 7f7dced9d700 1 --2- 172.21.15.163:0/3896816462 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7dd015d760 0x7f7dd0156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:29.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.630+0000 7f7dcdd9b700 1 -- 172.21.15.163:0/3896816462 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7dd015d760 msgr2=0x7f7dd0156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:29.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.630+0000 7f7dcdd9b700 1 --2- 172.21.15.163:0/3896816462 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7dd015d760 0x7f7dd0156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:29.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.630+0000 7f7dcdd9b700 1 -- 172.21.15.163:0/3896816462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dd00cf480 msgr2=0x7f7dd016a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:29.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.630+0000 7f7dcdd9b700 1 --2- 172.21.15.163:0/3896816462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dd00cf480 0x7f7dd016a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:29.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.630+0000 7f7dcdd9b700 1 -- 172.21.15.163:0/3896816462 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc40094f0 con 0x7f7dd01595d0 2023-10-09T06:32:29.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.630+0000 7f7dced9d700 1 --2- 172.21.15.163:0/3896816462 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7dd015d760 0x7f7dd0156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:29.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.630+0000 7f7dce59c700 1 --2- 172.21.15.163:0/3896816462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dd00cf480 0x7f7dd016a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:29.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.630+0000 7f7dcdd9b700 1 --2- 172.21.15.163:0/3896816462 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7dd01595d0 0x7f7dd01523d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00895f0 tx=0x7f7dc4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:29.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.630+0000 7f7dbe7fc700 1 -- 172.21.15.163:0/3896816462 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4011e60 con 0x7f7dd01595d0 2023-10-09T06:32:29.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.630+0000 7f7dbe7fc700 1 -- 172.21.15.163:0/3896816462 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc401a630 con 0x7f7dd01595d0 2023-10-09T06:32:29.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.630+0000 7f7dcf59e700 1 -- 172.21.15.163:0/3896816462 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd0158b20 con 0x7f7dd01595d0 2023-10-09T06:32:29.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.630+0000 7f7dbe7fc700 1 -- 172.21.15.163:0/3896816462 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc401a9f0 con 0x7f7dd01595d0 2023-10-09T06:32:29.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.630+0000 7f7dcf59e700 1 -- 172.21.15.163:0/3896816462 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd00c60e0 con 0x7f7dd01595d0 2023-10-09T06:32:29.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.631+0000 7f7dcf59e700 1 -- 172.21.15.163:0/3896816462 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dd004c3c0 con 0x7f7dd01595d0 2023-10-09T06:32:29.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.632+0000 7f7dbe7fc700 1 -- 172.21.15.163:0/3896816462 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f7dc402d440 con 0x7f7dd01595d0 2023-10-09T06:32:29.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.632+0000 7f7dbe7fc700 1 --2- 172.21.15.163:0/3896816462 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7db0066ff0 0x7f7db00694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:29.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.632+0000 7f7dbe7fc700 1 -- 172.21.15.163:0/3896816462 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f7dc4085ed0 con 0x7f7dd01595d0 2023-10-09T06:32:29.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.632+0000 7f7dce59c700 1 --2- 172.21.15.163:0/3896816462 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7db0066ff0 0x7f7db00694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:29.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.633+0000 7f7dce59c700 1 --2- 172.21.15.163:0/3896816462 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7db0066ff0 0x7f7db00694a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0007620 tx=0x7f7dc0009830 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:29.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.637+0000 7f7dbe7fc700 1 -- 172.21.15.163:0/3896816462 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7dc4053330 con 0x7f7dd01595d0 2023-10-09T06:32:29.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:29.819+0000 7f7dcf59e700 1 -- 172.21.15.163:0/3896816462 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7dd00b3860 con 0x7f7dd01595d0 2023-10-09T06:32:30.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.496+0000 7f7dbe7fc700 1 -- 172.21.15.163:0/3896816462 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v61) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7dc4056950 con 0x7f7dd01595d0 2023-10-09T06:32:30.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.501+0000 7f7dcf59e700 1 -- 172.21.15.163:0/3896816462 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7db0066ff0 msgr2=0x7f7db00694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:30.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.501+0000 7f7dcf59e700 1 --2- 172.21.15.163:0/3896816462 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7db0066ff0 0x7f7db00694a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0007620 tx=0x7f7dc0009830 comp rx=0 tx=0).stop 2023-10-09T06:32:30.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.501+0000 7f7dcf59e700 1 -- 172.21.15.163:0/3896816462 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7dd01595d0 msgr2=0x7f7dd01523d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:30.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.501+0000 7f7dcf59e700 1 --2- 172.21.15.163:0/3896816462 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7dd01595d0 0x7f7dd01523d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00895f0 tx=0x7f7dc4017000 comp rx=0 tx=0).stop 2023-10-09T06:32:30.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.501+0000 7f7dcf59e700 1 -- 172.21.15.163:0/3896816462 shutdown_connections 2023-10-09T06:32:30.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.501+0000 7f7dcf59e700 1 --2- 172.21.15.163:0/3896816462 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7db0066ff0 0x7f7db00694a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:30.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.501+0000 7f7dcf59e700 1 --2- 172.21.15.163:0/3896816462 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7dd015d760 0x7f7dd0156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:30.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.501+0000 7f7dcf59e700 1 --2- 172.21.15.163:0/3896816462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7dd00cf480 0x7f7dd016a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:30.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.501+0000 7f7dcf59e700 1 --2- 172.21.15.163:0/3896816462 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7dd01595d0 0x7f7dd01523d0 secure :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00895f0 tx=0x7f7dc4017000 comp rx=0 tx=0).stop 2023-10-09T06:32:30.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.501+0000 7f7dcf59e700 1 -- 172.21.15.163:0/3896816462 >> 172.21.15.163:0/3896816462 conn(0x7f7dd00d05f0 msgr2=0x7f7dd0152cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:30.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.504+0000 7f7dcf59e700 1 -- 172.21.15.163:0/3896816462 shutdown_connections 2023-10-09T06:32:30.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.506+0000 7f7dcf59e700 1 -- 172.21.15.163:0/3896816462 wait complete. 2023-10-09T06:32:30.509 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_data' created 2023-10-09T06:32:30.524 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-09T06:32:30.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.643+0000 7f055bfff700 1 Processor -- start 2023-10-09T06:32:30.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.643+0000 7f055bfff700 1 -- start start 2023-10-09T06:32:30.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.644+0000 7f055bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f055c159a60 0x7f055c0c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:30.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.644+0000 7f055bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f055c0ccc00 0x7f055c069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:30.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.644+0000 7f055bfff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f055c0c9cb0 con 0x7f055c06a020 2023-10-09T06:32:30.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.644+0000 7f055bfff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f055c06dfc0 con 0x7f055c159a60 2023-10-09T06:32:30.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.644+0000 7f055bfff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f055c0c9f30 con 0x7f055c0ccc00 2023-10-09T06:32:30.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.644+0000 7f055affd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f055c159a60 0x7f055c0c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:30.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.644+0000 7f055b7fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f055c06a020 0x7f055c0cb230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:44768/0 (socket says 172.21.15.163:44768) 2023-10-09T06:32:30.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.644+0000 7f055b7fe700 1 -- 172.21.15.163:0/3025160536 learned_addr learned my addr 172.21.15.163:0/3025160536 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:30.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.644+0000 7f055a7fc700 1 --2- 172.21.15.163:0/3025160536 >> v2:172.21.15.181:3300/0 conn(0x7f055c0ccc00 0x7f055c069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:30.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.644+0000 7f055affd700 1 -- 172.21.15.163:0/3025160536 >> v2:172.21.15.181:3300/0 conn(0x7f055c0ccc00 msgr2=0x7f055c069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:30.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.644+0000 7f055affd700 1 --2- 172.21.15.163:0/3025160536 >> v2:172.21.15.181:3300/0 conn(0x7f055c0ccc00 0x7f055c069ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:30.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.644+0000 7f055affd700 1 -- 172.21.15.163:0/3025160536 >> v1:172.21.15.163:6789/0 conn(0x7f055c06a020 legacy=0x7f055c0cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:30.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.644+0000 7f055affd700 1 -- 172.21.15.163:0/3025160536 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f055c0ca470 con 0x7f055c159a60 2023-10-09T06:32:30.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.644+0000 7f0559ffb700 1 -- 172.21.15.163:0/3025160536 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2355627598 0 0) 0x7f055c0c9cb0 con 0x7f055c06a020 2023-10-09T06:32:30.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.645+0000 7f055a7fc700 1 --2- 172.21.15.163:0/3025160536 >> v2:172.21.15.181:3300/0 conn(0x7f055c0ccc00 0x7f055c069ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:32:30.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.645+0000 7f055affd700 1 --2- 172.21.15.163:0/3025160536 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f055c159a60 0x7f055c0c9770 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f055c0cf6a0 tx=0x7f054c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6228999e2659ee9a server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:30.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.645+0000 7f0559ffb700 1 -- 172.21.15.163:0/3025160536 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c00b740 con 0x7f055c159a60 2023-10-09T06:32:30.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.645+0000 7f0559ffb700 1 -- 172.21.15.163:0/3025160536 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054c00b8a0 con 0x7f055c159a60 2023-10-09T06:32:30.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.645+0000 7f0559ffb700 1 -- 172.21.15.163:0/3025160536 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c00bc40 con 0x7f055c159a60 2023-10-09T06:32:30.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.645+0000 7f055bfff700 1 -- 172.21.15.163:0/3025160536 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f055c159a60 msgr2=0x7f055c0c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:30.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.645+0000 7f055bfff700 1 --2- 172.21.15.163:0/3025160536 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f055c159a60 0x7f055c0c9770 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f055c0cf6a0 tx=0x7f054c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:30.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.645+0000 7f055bfff700 1 -- 172.21.15.163:0/3025160536 shutdown_connections 2023-10-09T06:32:30.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.645+0000 7f055bfff700 1 --2- 172.21.15.163:0/3025160536 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f055c159a60 0x7f055c0c9770 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:30.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.645+0000 7f055bfff700 1 --2- 172.21.15.163:0/3025160536 >> v2:172.21.15.181:3300/0 conn(0x7f055c0ccc00 0x7f055c069ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:30.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.645+0000 7f055bfff700 1 -- 172.21.15.163:0/3025160536 >> 172.21.15.163:0/3025160536 conn(0x7f055c06baa0 msgr2=0x7f055c06bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:30.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.645+0000 7f055bfff700 1 -- 172.21.15.163:0/3025160536 shutdown_connections 2023-10-09T06:32:30.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.645+0000 7f055bfff700 1 -- 172.21.15.163:0/3025160536 wait complete. 2023-10-09T06:32:30.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.647+0000 7f055bfff700 1 Processor -- start 2023-10-09T06:32:30.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.647+0000 7f055bfff700 1 -- start start 2023-10-09T06:32:30.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.647+0000 7f055bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f055c06a020 0x7f055c0c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:30.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.647+0000 7f055bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f055c0ccc00 0x7f055c169c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:30.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.647+0000 7f055bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f055c159a60 0x7f055c16a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:30.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.647+0000 7f055bfff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c1553c0 con 0x7f055c0ccc00 2023-10-09T06:32:30.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.647+0000 7f055bfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c0c7530 con 0x7f055c159a60 2023-10-09T06:32:30.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.647+0000 7f055bfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f055c0a9970 con 0x7f055c06a020 2023-10-09T06:32:30.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.647+0000 7f055a7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f055c0ccc00 0x7f055c169c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:30.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.647+0000 7f055a7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f055c0ccc00 0x7f055c169c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58866/0 (socket says 172.21.15.163:58866) 2023-10-09T06:32:30.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.648+0000 7f055a7fc700 1 -- 172.21.15.163:0/492124761 learned_addr learned my addr 172.21.15.163:0/492124761 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:30.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.648+0000 7f055b7fe700 1 --2- 172.21.15.163:0/492124761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f055c159a60 0x7f055c16a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:30.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.648+0000 7f055affd700 1 --2- 172.21.15.163:0/492124761 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f055c06a020 0x7f055c0c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:30.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.648+0000 7f055a7fc700 1 -- 172.21.15.163:0/492124761 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f055c06a020 msgr2=0x7f055c0c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:30.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.648+0000 7f055a7fc700 1 --2- 172.21.15.163:0/492124761 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f055c06a020 0x7f055c0c79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:30.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.648+0000 7f055a7fc700 1 -- 172.21.15.163:0/492124761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f055c159a60 msgr2=0x7f055c16a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:30.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.648+0000 7f055a7fc700 1 --2- 172.21.15.163:0/492124761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f055c159a60 0x7f055c16a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:30.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.648+0000 7f055a7fc700 1 -- 172.21.15.163:0/492124761 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054c009420 con 0x7f055c0ccc00 2023-10-09T06:32:30.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.648+0000 7f055a7fc700 1 --2- 172.21.15.163:0/492124761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f055c0ccc00 0x7f055c169c90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f055000c860 tx=0x7f0550008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:30.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.648+0000 7f0542ffd700 1 -- 172.21.15.163:0/492124761 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0550011600 con 0x7f055c0ccc00 2023-10-09T06:32:30.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.648+0000 7f0542ffd700 1 -- 172.21.15.163:0/492124761 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0550011760 con 0x7f055c0ccc00 2023-10-09T06:32:30.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.648+0000 7f055bfff700 1 -- 172.21.15.163:0/492124761 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f055c0a9c10 con 0x7f055c0ccc00 2023-10-09T06:32:30.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.648+0000 7f0542ffd700 1 -- 172.21.15.163:0/492124761 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0550011b20 con 0x7f055c0ccc00 2023-10-09T06:32:30.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.648+0000 7f055bfff700 1 -- 172.21.15.163:0/492124761 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f055c16a790 con 0x7f055c0ccc00 2023-10-09T06:32:30.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.649+0000 7f055bfff700 1 -- 172.21.15.163:0/492124761 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f055c0c76c0 con 0x7f055c0ccc00 2023-10-09T06:32:30.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.650+0000 7f0542ffd700 1 -- 172.21.15.163:0/492124761 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f05500193b0 con 0x7f055c0ccc00 2023-10-09T06:32:30.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.650+0000 7f0542ffd700 1 --2- 172.21.15.163:0/492124761 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0548066fc0 0x7f0548069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:30.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.650+0000 7f0542ffd700 1 -- 172.21.15.163:0/492124761 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f0550084530 con 0x7f055c0ccc00 2023-10-09T06:32:30.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.650+0000 7f055affd700 1 --2- 172.21.15.163:0/492124761 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0548066fc0 0x7f0548069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:30.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.651+0000 7f055affd700 1 --2- 172.21.15.163:0/492124761 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0548066fc0 0x7f0548069470 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f054c0093f0 tx=0x7f054c017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:30.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.654+0000 7f0542ffd700 1 -- 172.21.15.163:0/492124761 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0550051700 con 0x7f055c0ccc00 2023-10-09T06:32:30.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:30.881+0000 7f055bfff700 1 -- 172.21.15.163:0/492124761 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f055c0ca750 con 0x7f055c0ccc00 2023-10-09T06:32:31.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.500+0000 7f0542ffd700 1 -- 172.21.15.163:0/492124761 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v36) v1 ==== 174+0+0 (secure 0 0 0) 0x7f0550054d20 con 0x7f055c0ccc00 2023-10-09T06:32:31.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.509+0000 7f055bfff700 1 -- 172.21.15.163:0/492124761 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0548066fc0 msgr2=0x7f0548069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:31.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.509+0000 7f055bfff700 1 --2- 172.21.15.163:0/492124761 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0548066fc0 0x7f0548069470 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f054c0093f0 tx=0x7f054c017000 comp rx=0 tx=0).stop 2023-10-09T06:32:31.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.509+0000 7f055bfff700 1 -- 172.21.15.163:0/492124761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f055c0ccc00 msgr2=0x7f055c169c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:31.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.509+0000 7f055bfff700 1 --2- 172.21.15.163:0/492124761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f055c0ccc00 0x7f055c169c90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f055000c860 tx=0x7f0550008040 comp rx=0 tx=0).stop 2023-10-09T06:32:31.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.510+0000 7f055bfff700 1 -- 172.21.15.163:0/492124761 shutdown_connections 2023-10-09T06:32:31.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.510+0000 7f055bfff700 1 --2- 172.21.15.163:0/492124761 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0548066fc0 0x7f0548069470 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:31.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.510+0000 7f055bfff700 1 --2- 172.21.15.163:0/492124761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f055c159a60 0x7f055c16a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:31.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.510+0000 7f055bfff700 1 --2- 172.21.15.163:0/492124761 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f055c06a020 0x7f055c0c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:31.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.510+0000 7f055bfff700 1 --2- 172.21.15.163:0/492124761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f055c0ccc00 0x7f055c169c90 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:31.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.510+0000 7f055bfff700 1 -- 172.21.15.163:0/492124761 >> 172.21.15.163:0/492124761 conn(0x7f055c06baa0 msgr2=0x7f055c15a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:31.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.511+0000 7f055bfff700 1 -- 172.21.15.163:0/492124761 shutdown_connections 2023-10-09T06:32:31.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.511+0000 7f055bfff700 1 -- 172.21.15.163:0/492124761 wait complete. 2023-10-09T06:32:31.513 INFO:teuthology.orchestra.run.smithi163.stderr:new fs with metadata pool 10 and data pool 11 2023-10-09T06:32:31.535 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:32:31.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.652+0000 7fd81bfff700 1 Processor -- start 2023-10-09T06:32:31.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.652+0000 7fd81bfff700 1 -- start start 2023-10-09T06:32:31.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.652+0000 7fd81bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd81c159420 0x7fd81c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:31.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.653+0000 7fd81bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd81c1597f0 0x7fd81c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:31.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.653+0000 7fd81bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd81c15d230 0x7fd81c15d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:31.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.653+0000 7fd81bfff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd81c15dbb0 con 0x7fd81c1597f0 2023-10-09T06:32:31.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.653+0000 7fd81bfff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd81c0b11d0 con 0x7fd81c159420 2023-10-09T06:32:31.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.653+0000 7fd81bfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c0b14b0 con 0x7fd81c15d230 2023-10-09T06:32:31.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.653+0000 7fd81a7fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd81c1597f0 0x7fd81c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:31.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.653+0000 7fd81a7fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd81c1597f0 0x7fd81c0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58880/0 (socket says 172.21.15.163:58880) 2023-10-09T06:32:31.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.653+0000 7fd81a7fc700 1 -- 172.21.15.163:0/2935563596 learned_addr learned my addr 172.21.15.163:0/2935563596 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:31.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.653+0000 7fd81b7fe700 1 --2- 172.21.15.163:0/2935563596 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd81c15d230 0x7fd81c15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:31.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.653+0000 7fd81affd700 1 --2- 172.21.15.163:0/2935563596 >> v2:172.21.15.181:3300/0 conn(0x7fd81c159420 0x7fd81c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:31.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.653+0000 7fd81a7fc700 1 -- 172.21.15.163:0/2935563596 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd81c15d230 msgr2=0x7fd81c15d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:31.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.653+0000 7fd81a7fc700 1 --2- 172.21.15.163:0/2935563596 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd81c15d230 0x7fd81c15d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:31.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.653+0000 7fd81a7fc700 1 -- 172.21.15.163:0/2935563596 >> v2:172.21.15.181:3300/0 conn(0x7fd81c159420 msgr2=0x7fd81c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:31.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.653+0000 7fd81a7fc700 1 --2- 172.21.15.163:0/2935563596 >> v2:172.21.15.181:3300/0 conn(0x7fd81c159420 0x7fd81c0b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:31.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.653+0000 7fd81a7fc700 1 -- 172.21.15.163:0/2935563596 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81c0b1790 con 0x7fd81c1597f0 2023-10-09T06:32:31.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.653+0000 7fd81b7fe700 1 --2- 172.21.15.163:0/2935563596 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd81c15d230 0x7fd81c15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:32:31.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.653+0000 7fd81a7fc700 1 --2- 172.21.15.163:0/2935563596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd81c1597f0 0x7fd81c0b2270 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd81c0b2f20 tx=0x7fd810009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2777ee7847b30278 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:31.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.653+0000 7fd819ffb700 1 -- 172.21.15.163:0/2935563596 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81000b860 con 0x7fd81c1597f0 2023-10-09T06:32:31.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.653+0000 7fd819ffb700 1 -- 172.21.15.163:0/2935563596 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd81000b9c0 con 0x7fd81c1597f0 2023-10-09T06:32:31.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.653+0000 7fd819ffb700 1 -- 172.21.15.163:0/2935563596 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81000bd60 con 0x7fd81c1597f0 2023-10-09T06:32:31.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.654+0000 7fd81bfff700 1 -- 172.21.15.163:0/2935563596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd81c1597f0 msgr2=0x7fd81c0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:31.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.654+0000 7fd81bfff700 1 --2- 172.21.15.163:0/2935563596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd81c1597f0 0x7fd81c0b2270 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd81c0b2f20 tx=0x7fd810009250 comp rx=0 tx=0).stop 2023-10-09T06:32:31.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.654+0000 7fd81bfff700 1 -- 172.21.15.163:0/2935563596 shutdown_connections 2023-10-09T06:32:31.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.654+0000 7fd81bfff700 1 --2- 172.21.15.163:0/2935563596 >> v2:172.21.15.181:3300/0 conn(0x7fd81c159420 0x7fd81c0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:31.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.654+0000 7fd81bfff700 1 --2- 172.21.15.163:0/2935563596 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd81c15d230 0x7fd81c15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:31.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.654+0000 7fd81bfff700 1 --2- 172.21.15.163:0/2935563596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd81c1597f0 0x7fd81c0b2270 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:31.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.654+0000 7fd81bfff700 1 -- 172.21.15.163:0/2935563596 >> 172.21.15.163:0/2935563596 conn(0x7fd81c0d0660 msgr2=0x7fd81c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:31.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.654+0000 7fd81bfff700 1 -- 172.21.15.163:0/2935563596 shutdown_connections 2023-10-09T06:32:31.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.654+0000 7fd81bfff700 1 -- 172.21.15.163:0/2935563596 wait complete. 2023-10-09T06:32:31.662 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.655+0000 7fd81bfff700 1 Processor -- start 2023-10-09T06:32:31.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.655+0000 7fd81bfff700 1 -- start start 2023-10-09T06:32:31.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.656+0000 7fd81bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd81c159420 0x7fd81c159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:31.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.656+0000 7fd81bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd81c1597f0 0x7fd81c153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:31.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.656+0000 7fd81bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd81c15d230 0x7fd81c0cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:31.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.656+0000 7fd81bfff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c15c290 con 0x7fd81c1597f0 2023-10-09T06:32:31.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.656+0000 7fd81bfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c15c720 con 0x7fd81c15d230 2023-10-09T06:32:31.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.656+0000 7fd81bfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c15b680 con 0x7fd81c159420 2023-10-09T06:32:31.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.656+0000 7fd81a7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd81c1597f0 0x7fd81c153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:31.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.656+0000 7fd81a7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd81c1597f0 0x7fd81c153a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58892/0 (socket says 172.21.15.163:58892) 2023-10-09T06:32:31.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.656+0000 7fd81a7fc700 1 -- 172.21.15.163:0/3152359558 learned_addr learned my addr 172.21.15.163:0/3152359558 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:31.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.656+0000 7fd81affd700 1 --2- 172.21.15.163:0/3152359558 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd81c159420 0x7fd81c159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:31.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.656+0000 7fd81b7fe700 1 --2- 172.21.15.163:0/3152359558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd81c15d230 0x7fd81c0cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:31.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.656+0000 7fd81a7fc700 1 -- 172.21.15.163:0/3152359558 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd81c159420 msgr2=0x7fd81c159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:31.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.656+0000 7fd81a7fc700 1 --2- 172.21.15.163:0/3152359558 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd81c159420 0x7fd81c159e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:31.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.656+0000 7fd81a7fc700 1 -- 172.21.15.163:0/3152359558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd81c15d230 msgr2=0x7fd81c0cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:31.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.656+0000 7fd81a7fc700 1 --2- 172.21.15.163:0/3152359558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd81c15d230 0x7fd81c0cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:31.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.656+0000 7fd81a7fc700 1 -- 172.21.15.163:0/3152359558 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8100094f0 con 0x7fd81c1597f0 2023-10-09T06:32:31.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.656+0000 7fd81a7fc700 1 --2- 172.21.15.163:0/3152359558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd81c1597f0 0x7fd81c153a10 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd81c089340 tx=0x7fd810017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:31.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.656+0000 7fd802ffd700 1 -- 172.21.15.163:0/3152359558 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd810011dc0 con 0x7fd81c1597f0 2023-10-09T06:32:31.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.657+0000 7fd802ffd700 1 -- 172.21.15.163:0/3152359558 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd810024440 con 0x7fd81c1597f0 2023-10-09T06:32:31.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.657+0000 7fd81bfff700 1 -- 172.21.15.163:0/3152359558 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd81c15b960 con 0x7fd81c1597f0 2023-10-09T06:32:31.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.657+0000 7fd802ffd700 1 -- 172.21.15.163:0/3152359558 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd810024800 con 0x7fd81c1597f0 2023-10-09T06:32:31.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.657+0000 7fd81bfff700 1 -- 172.21.15.163:0/3152359558 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd81c15bf20 con 0x7fd81c1597f0 2023-10-09T06:32:31.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.657+0000 7fd81bfff700 1 -- 172.21.15.163:0/3152359558 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd81c04c3c0 con 0x7fd81c1597f0 2023-10-09T06:32:31.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.658+0000 7fd802ffd700 1 -- 172.21.15.163:0/3152359558 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fd81001a420 con 0x7fd81c1597f0 2023-10-09T06:32:31.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.658+0000 7fd802ffd700 1 --2- 172.21.15.163:0/3152359558 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd808066fc0 0x7fd808069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:31.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.658+0000 7fd802ffd700 1 -- 172.21.15.163:0/3152359558 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd810086370 con 0x7fd81c1597f0 2023-10-09T06:32:31.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.658+0000 7fd81affd700 1 --2- 172.21.15.163:0/3152359558 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd808066fc0 0x7fd808069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:31.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.659+0000 7fd81affd700 1 --2- 172.21.15.163:0/3152359558 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd808066fc0 0x7fd808069470 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd80c00acb0 tx=0x7fd80c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:31.667 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.663+0000 7fd802ffd700 1 -- 172.21.15.163:0/3152359558 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd8100535f0 con 0x7fd81c1597f0 2023-10-09T06:32:31.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.845+0000 7fd81bfff700 1 -- 172.21.15.163:0/3152359558 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd81c15c3d0 con 0x7fd81c1597f0 2023-10-09T06:32:31.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.845+0000 7fd802ffd700 1 -- 172.21.15.163:0/3152359558 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+15414 (secure 0 0 0) 0x7fd810056c10 con 0x7fd81c1597f0 2023-10-09T06:32:31.846 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:31.847 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":62,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:32:30.882604+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:32:28.847438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-09T06:32:29.837643+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:32:31.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.847+0000 7fd81bfff700 1 -- 172.21.15.163:0/3152359558 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd808066fc0 msgr2=0x7fd808069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:31.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.847+0000 7fd81bfff700 1 --2- 172.21.15.163:0/3152359558 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd808066fc0 0x7fd808069470 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd80c00acb0 tx=0x7fd80c008040 comp rx=0 tx=0).stop 2023-10-09T06:32:31.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.847+0000 7fd81bfff700 1 -- 172.21.15.163:0/3152359558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd81c1597f0 msgr2=0x7fd81c153a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:31.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.847+0000 7fd81bfff700 1 --2- 172.21.15.163:0/3152359558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd81c1597f0 0x7fd81c153a10 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd81c089340 tx=0x7fd810017000 comp rx=0 tx=0).stop 2023-10-09T06:32:31.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.847+0000 7fd81bfff700 1 -- 172.21.15.163:0/3152359558 shutdown_connections 2023-10-09T06:32:31.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.847+0000 7fd81bfff700 1 --2- 172.21.15.163:0/3152359558 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd808066fc0 0x7fd808069470 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:31.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.847+0000 7fd81bfff700 1 --2- 172.21.15.163:0/3152359558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd81c15d230 0x7fd81c0cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:31.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.847+0000 7fd81bfff700 1 --2- 172.21.15.163:0/3152359558 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd81c159420 0x7fd81c159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:31.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.847+0000 7fd81bfff700 1 --2- 172.21.15.163:0/3152359558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd81c1597f0 0x7fd81c153a10 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:31.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.847+0000 7fd81bfff700 1 -- 172.21.15.163:0/3152359558 >> 172.21.15.163:0/3152359558 conn(0x7fd81c0d0660 msgr2=0x7fd81c157210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:31.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.848+0000 7fd81bfff700 1 -- 172.21.15.163:0/3152359558 shutdown_connections 2023-10-09T06:32:31.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.848+0000 7fd81bfff700 1 -- 172.21.15.163:0/3152359558 wait complete. 2023-10-09T06:32:31.861 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:32:31.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.974+0000 7fe62a88b700 1 Processor -- start 2023-10-09T06:32:31.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.974+0000 7fe62a88b700 1 -- start start 2023-10-09T06:32:31.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.975+0000 7fe62a88b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe624141500 0x7fe6240c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:31.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.975+0000 7fe62a88b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe62406c0a0 0x7fe6240c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:31.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.975+0000 7fe62a88b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe62415c2a0 con 0x7fe62406bcd0 2023-10-09T06:32:31.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.975+0000 7fe62a88b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6240c9880 con 0x7fe624141500 2023-10-09T06:32:31.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.975+0000 7fe62a88b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe62415b690 con 0x7fe62406c0a0 2023-10-09T06:32:31.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.975+0000 7fe6237fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe624141500 0x7fe6240c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:31.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.975+0000 7fe623fff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fe62406bcd0 0x7fe624140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:44782/0 (socket says 172.21.15.163:44782) 2023-10-09T06:32:31.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.975+0000 7fe623fff700 1 -- 172.21.15.163:0/126448094 learned_addr learned my addr 172.21.15.163:0/126448094 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:31.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.975+0000 7fe628e28700 1 --2- 172.21.15.163:0/126448094 >> v2:172.21.15.181:3300/0 conn(0x7fe62406c0a0 0x7fe6240c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:31.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.976+0000 7fe6237fe700 1 -- 172.21.15.163:0/126448094 >> v2:172.21.15.181:3300/0 conn(0x7fe62406c0a0 msgr2=0x7fe6240c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:31.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.976+0000 7fe6237fe700 1 --2- 172.21.15.163:0/126448094 >> v2:172.21.15.181:3300/0 conn(0x7fe62406c0a0 0x7fe6240c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:31.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.976+0000 7fe6237fe700 1 -- 172.21.15.163:0/126448094 >> v1:172.21.15.163:6789/0 conn(0x7fe62406bcd0 legacy=0x7fe624140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:31.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.976+0000 7fe6237fe700 1 -- 172.21.15.163:0/126448094 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe62415b970 con 0x7fe624141500 2023-10-09T06:32:31.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.976+0000 7fe622ffd700 1 -- 172.21.15.163:0/126448094 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3878040634 0 0) 0x7fe62415c2a0 con 0x7fe62406bcd0 2023-10-09T06:32:31.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.976+0000 7fe6237fe700 1 --2- 172.21.15.163:0/126448094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe624141500 0x7fe6240c8e00 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe6240cf210 tx=0x7fe6180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ae6342d536c3f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:31.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.976+0000 7fe622ffd700 1 -- 172.21.15.163:0/126448094 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61800b6b0 con 0x7fe624141500 2023-10-09T06:32:31.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.976+0000 7fe622ffd700 1 -- 172.21.15.163:0/126448094 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe61800b810 con 0x7fe624141500 2023-10-09T06:32:31.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.976+0000 7fe622ffd700 1 -- 172.21.15.163:0/126448094 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61800bbb0 con 0x7fe624141500 2023-10-09T06:32:31.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.976+0000 7fe62a88b700 1 -- 172.21.15.163:0/126448094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe624141500 msgr2=0x7fe6240c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:31.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.976+0000 7fe62a88b700 1 --2- 172.21.15.163:0/126448094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe624141500 0x7fe6240c8e00 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe6240cf210 tx=0x7fe6180091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:31.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.976+0000 7fe62a88b700 1 -- 172.21.15.163:0/126448094 shutdown_connections 2023-10-09T06:32:31.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.976+0000 7fe62a88b700 1 --2- 172.21.15.163:0/126448094 >> v2:172.21.15.181:3300/0 conn(0x7fe62406c0a0 0x7fe6240c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:31.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.976+0000 7fe62a88b700 1 --2- 172.21.15.163:0/126448094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe624141500 0x7fe6240c8e00 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:31.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.976+0000 7fe62a88b700 1 -- 172.21.15.163:0/126448094 >> 172.21.15.163:0/126448094 conn(0x7fe6240ce600 msgr2=0x7fe6240cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:31.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.976+0000 7fe62a88b700 1 -- 172.21.15.163:0/126448094 shutdown_connections 2023-10-09T06:32:31.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.976+0000 7fe62a88b700 1 -- 172.21.15.163:0/126448094 wait complete. 2023-10-09T06:32:31.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.978+0000 7fe62a88b700 1 Processor -- start 2023-10-09T06:32:31.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.978+0000 7fe62a88b700 1 -- start start 2023-10-09T06:32:31.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.978+0000 7fe62a88b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe62406bcd0 0x7fe624157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:31.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.978+0000 7fe62a88b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe62406c0a0 0x7fe6240c6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:31.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.978+0000 7fe62a88b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe624141500 0x7fe6240c4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:31.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.978+0000 7fe62a88b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe624167d50 con 0x7fe624141500 2023-10-09T06:32:31.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.978+0000 7fe62a88b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe624157940 con 0x7fe62406c0a0 2023-10-09T06:32:31.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.978+0000 7fe62a88b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6240c7590 con 0x7fe62406bcd0 2023-10-09T06:32:31.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.979+0000 7fe628e28700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe624141500 0x7fe6240c4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:31.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.979+0000 7fe628e28700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe624141500 0x7fe6240c4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58908/0 (socket says 172.21.15.163:58908) 2023-10-09T06:32:31.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.979+0000 7fe628e28700 1 -- 172.21.15.163:0/15826848 learned_addr learned my addr 172.21.15.163:0/15826848 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:31.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.979+0000 7fe623fff700 1 --2- 172.21.15.163:0/15826848 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe62406bcd0 0x7fe624157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:31.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.979+0000 7fe6237fe700 1 --2- 172.21.15.163:0/15826848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe62406c0a0 0x7fe6240c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:31.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.979+0000 7fe628e28700 1 -- 172.21.15.163:0/15826848 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe62406bcd0 msgr2=0x7fe624157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:31.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.979+0000 7fe628e28700 1 --2- 172.21.15.163:0/15826848 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe62406bcd0 0x7fe624157400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:31.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.979+0000 7fe628e28700 1 -- 172.21.15.163:0/15826848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe62406c0a0 msgr2=0x7fe6240c6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:31.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.979+0000 7fe628e28700 1 --2- 172.21.15.163:0/15826848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe62406c0a0 0x7fe6240c6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:31.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.979+0000 7fe628e28700 1 -- 172.21.15.163:0/15826848 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe618009420 con 0x7fe624141500 2023-10-09T06:32:31.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.979+0000 7fe628e28700 1 --2- 172.21.15.163:0/15826848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe624141500 0x7fe6240c4950 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009fa0 tx=0x7fe61c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:31.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.979+0000 7fe60bfff700 1 -- 172.21.15.163:0/15826848 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c00ebc0 con 0x7fe624141500 2023-10-09T06:32:31.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.979+0000 7fe60bfff700 1 -- 172.21.15.163:0/15826848 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe61c00ed20 con 0x7fe624141500 2023-10-09T06:32:31.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.979+0000 7fe62a88b700 1 -- 172.21.15.163:0/15826848 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6240c6a10 con 0x7fe624141500 2023-10-09T06:32:31.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.979+0000 7fe60bfff700 1 -- 172.21.15.163:0/15826848 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c014670 con 0x7fe624141500 2023-10-09T06:32:31.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.979+0000 7fe62a88b700 1 -- 172.21.15.163:0/15826848 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6240c6fd0 con 0x7fe624141500 2023-10-09T06:32:31.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.980+0000 7fe62a88b700 1 -- 172.21.15.163:0/15826848 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe62404c3c0 con 0x7fe624141500 2023-10-09T06:32:31.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.981+0000 7fe60bfff700 1 -- 172.21.15.163:0/15826848 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fe61c0149e0 con 0x7fe624141500 2023-10-09T06:32:31.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.981+0000 7fe60bfff700 1 --2- 172.21.15.163:0/15826848 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe610066fc0 0x7fe610069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:31.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.981+0000 7fe60bfff700 1 -- 172.21.15.163:0/15826848 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe61c084bf0 con 0x7fe624141500 2023-10-09T06:32:31.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.981+0000 7fe623fff700 1 --2- 172.21.15.163:0/15826848 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe610066fc0 0x7fe610069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:31.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.982+0000 7fe623fff700 1 --2- 172.21.15.163:0/15826848 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe610066fc0 0x7fe610069470 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe614007fd0 tx=0x7fe614007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:31.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:31.985+0000 7fe60bfff700 1 -- 172.21.15.163:0/15826848 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe61c051e70 con 0x7fe624141500 2023-10-09T06:32:32.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.172+0000 7fe62a88b700 1 -- 172.21.15.163:0/15826848 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe62406c9d0 con 0x7fe624141500 2023-10-09T06:32:32.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.173+0000 7fe60bfff700 1 -- 172.21.15.163:0/15826848 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+15414 (secure 0 0 0) 0x7fe61c055490 con 0x7fe624141500 2023-10-09T06:32:32.174 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:32.174 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":62,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:32:30.882604+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:32:28.847438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-09T06:32:29.837643+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:32:32.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.175+0000 7fe62a88b700 1 -- 172.21.15.163:0/15826848 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe610066fc0 msgr2=0x7fe610069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:32.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.175+0000 7fe62a88b700 1 --2- 172.21.15.163:0/15826848 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe610066fc0 0x7fe610069470 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe614007fd0 tx=0x7fe614007d30 comp rx=0 tx=0).stop 2023-10-09T06:32:32.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.175+0000 7fe62a88b700 1 -- 172.21.15.163:0/15826848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe624141500 msgr2=0x7fe6240c4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:32.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.175+0000 7fe62a88b700 1 --2- 172.21.15.163:0/15826848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe624141500 0x7fe6240c4950 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009fa0 tx=0x7fe61c009900 comp rx=0 tx=0).stop 2023-10-09T06:32:32.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.175+0000 7fe62a88b700 1 -- 172.21.15.163:0/15826848 shutdown_connections 2023-10-09T06:32:32.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.175+0000 7fe62a88b700 1 --2- 172.21.15.163:0/15826848 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe610066fc0 0x7fe610069470 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:32.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.175+0000 7fe62a88b700 1 --2- 172.21.15.163:0/15826848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe624141500 0x7fe6240c4950 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:32.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.175+0000 7fe62a88b700 1 --2- 172.21.15.163:0/15826848 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe62406bcd0 0x7fe624157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:32.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.175+0000 7fe62a88b700 1 --2- 172.21.15.163:0/15826848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe62406c0a0 0x7fe6240c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:32.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.175+0000 7fe62a88b700 1 -- 172.21.15.163:0/15826848 >> 172.21.15.163:0/15826848 conn(0x7fe6240ce600 msgr2=0x7fe624158080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:32.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.175+0000 7fe62a88b700 1 -- 172.21.15.163:0/15826848 shutdown_connections 2023-10-09T06:32:32.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.175+0000 7fe62a88b700 1 -- 172.21.15.163:0/15826848 wait complete. 2023-10-09T06:32:32.188 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-09T06:32:32.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.304+0000 7fa5bccd6700 1 Processor -- start 2023-10-09T06:32:32.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.304+0000 7fa5bccd6700 1 -- start start 2023-10-09T06:32:32.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.305+0000 7fa5bccd6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa5b8159c60 0x7fa5b815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:32.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.305+0000 7fa5bccd6700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b815a160 con 0x7fa5b8159890 2023-10-09T06:32:32.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.305+0000 7fa5bccd6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b80d13a0 con 0x7fa5b815db10 2023-10-09T06:32:32.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.305+0000 7fa5bccd6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5b815df50 con 0x7fa5b8159c60 2023-10-09T06:32:32.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.305+0000 7fa5b659c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fa5b8159890 0x7fa5b815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:44790/0 (socket says 172.21.15.163:44790) 2023-10-09T06:32:32.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.305+0000 7fa5b659c700 1 -- 172.21.15.163:0/946893914 learned_addr learned my addr 172.21.15.163:0/946893914 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:32.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.306+0000 7fa5b5d9b700 1 -- 172.21.15.163:0/946893914 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2326032790 0 0) 0x7fa5b815a160 con 0x7fa5b8159890 2023-10-09T06:32:32.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.306+0000 7fa5b5d9b700 1 -- 172.21.15.163:0/946893914 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5a4003570 con 0x7fa5b8159890 2023-10-09T06:32:32.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.306+0000 7fa5add9b700 1 --2- 172.21.15.163:0/946893914 >> v2:172.21.15.181:3300/0 conn(0x7fa5b8159c60 0x7fa5b815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:32.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.306+0000 7fa5b5d9b700 1 -- 172.21.15.163:0/946893914 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 823120860 0 0) 0x7fa5a4003570 con 0x7fa5b8159890 2023-10-09T06:32:32.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.306+0000 7fa5b5d9b700 1 -- 172.21.15.163:0/946893914 >> v2:172.21.15.181:3300/0 conn(0x7fa5b8159c60 msgr2=0x7fa5b815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:32.310 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.306+0000 7fa5b5d9b700 1 --2- 172.21.15.163:0/946893914 >> v2:172.21.15.181:3300/0 conn(0x7fa5b8159c60 0x7fa5b815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:32.310 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.306+0000 7fa5b5d9b700 1 -- 172.21.15.163:0/946893914 >> v1:172.21.15.181:6789/0 conn(0x7fa5b815db10 legacy=0x7fa5b8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:32.310 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.306+0000 7fa5b5d9b700 1 -- 172.21.15.163:0/946893914 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b815e140 con 0x7fa5b8159890 2023-10-09T06:32:32.310 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.306+0000 7fa5b5d9b700 1 -- 172.21.15.163:0/946893914 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fa5a8001470 con 0x7fa5b8159890 2023-10-09T06:32:32.310 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.306+0000 7fa5b5d9b700 1 -- 172.21.15.163:0/946893914 >> v1:172.21.15.163:6789/0 conn(0x7fa5b8159890 legacy=0x7fa5b815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:32.310 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.306+0000 7fa5b5d9b700 1 --2- 172.21.15.163:0/946893914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5a4003a60 0x7fa5a4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:32.310 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.306+0000 7fa5b5d9b700 1 --2- 172.21.15.163:0/946893914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5a4004810 0x7fa5a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:32.310 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.306+0000 7fa5b5d9b700 1 --2- 172.21.15.163:0/946893914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5a40051c0 0x7fa5a400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:32.310 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.306+0000 7fa5b5d9b700 1 -- 172.21.15.163:0/946893914 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a400c0a0 con 0x7fa5a40051c0 2023-10-09T06:32:32.310 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.306+0000 7fa5b5d9b700 1 -- 172.21.15.163:0/946893914 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a400c210 con 0x7fa5a4004810 2023-10-09T06:32:32.310 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.306+0000 7fa5b5d9b700 1 -- 172.21.15.163:0/946893914 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a400c3d0 con 0x7fa5a4003a60 2023-10-09T06:32:32.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.306+0000 7fa5b6d9d700 1 --2- 172.21.15.163:0/946893914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5a40051c0 0x7fa5a400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:32.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.306+0000 7fa5add9b700 1 --2- 172.21.15.163:0/946893914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5a4004810 0x7fa5a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:32.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.306+0000 7fa5b659c700 1 --2- 172.21.15.163:0/946893914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5a4003a60 0x7fa5a4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:32.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.306+0000 7fa5b6d9d700 1 -- 172.21.15.163:0/946893914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5a4003a60 msgr2=0x7fa5a4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:32.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.306+0000 7fa5b6d9d700 1 --2- 172.21.15.163:0/946893914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5a4003a60 0x7fa5a4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:32.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.306+0000 7fa5b6d9d700 1 -- 172.21.15.163:0/946893914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5a4004810 msgr2=0x7fa5a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:32.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.306+0000 7fa5b6d9d700 1 --2- 172.21.15.163:0/946893914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5a4004810 0x7fa5a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:32.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.306+0000 7fa5b6d9d700 1 -- 172.21.15.163:0/946893914 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a4003570 con 0x7fa5a40051c0 2023-10-09T06:32:32.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.306+0000 7fa5b6d9d700 1 -- 172.21.15.163:0/946893914 reap_dead start 2023-10-09T06:32:32.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.306+0000 7fa5b6d9d700 1 --2- 172.21.15.163:0/946893914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5a40051c0 0x7fa5a400ba60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0001fc0 tx=0x7fa5b0001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6130977ff593b54a server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:32.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.307+0000 7fa5b5d9b700 1 -- 172.21.15.163:0/946893914 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0001950 con 0x7fa5a40051c0 2023-10-09T06:32:32.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.307+0000 7fa5b5d9b700 1 -- 172.21.15.163:0/946893914 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b000c8b0 con 0x7fa5a40051c0 2023-10-09T06:32:32.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.307+0000 7fa5b5d9b700 1 -- 172.21.15.163:0/946893914 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b000cc50 con 0x7fa5a40051c0 2023-10-09T06:32:32.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.307+0000 7fa5bccd6700 1 -- 172.21.15.163:0/946893914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5a40051c0 msgr2=0x7fa5a400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:32.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.307+0000 7fa5bccd6700 1 --2- 172.21.15.163:0/946893914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5a40051c0 0x7fa5a400ba60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0001fc0 tx=0x7fa5b0001640 comp rx=0 tx=0).stop 2023-10-09T06:32:32.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.307+0000 7fa5bccd6700 1 -- 172.21.15.163:0/946893914 shutdown_connections 2023-10-09T06:32:32.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.307+0000 7fa5bccd6700 1 --2- 172.21.15.163:0/946893914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5a40051c0 0x7fa5a400ba60 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:32.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.307+0000 7fa5bccd6700 1 -- 172.21.15.163:0/946893914 >> 172.21.15.163:0/946893914 conn(0x7fa5b8068e20 msgr2=0x7fa5b80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:32.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.307+0000 7fa5bccd6700 1 -- 172.21.15.163:0/946893914 shutdown_connections 2023-10-09T06:32:32.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.307+0000 7fa5bccd6700 1 -- 172.21.15.163:0/946893914 wait complete. 2023-10-09T06:32:32.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.308+0000 7fa5bccd6700 1 Processor -- start 2023-10-09T06:32:32.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.308+0000 7fa5bccd6700 1 -- start start 2023-10-09T06:32:32.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.309+0000 7fa5bccd6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5a40051c0 0x7fa5b80c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:32.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.309+0000 7fa5bccd6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b815c3b0 0x7fa5b8151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:32.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.309+0000 7fa5bccd6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5b81520f0 0x7fa5b80c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:32.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.309+0000 7fa5bccd6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b80cdaa0 con 0x7fa5a40051c0 2023-10-09T06:32:32.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.309+0000 7fa5bccd6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b815d4a0 con 0x7fa5b815c3b0 2023-10-09T06:32:32.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.309+0000 7fa5bccd6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b815c920 con 0x7fa5b81520f0 2023-10-09T06:32:32.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.309+0000 7fa5b659c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5a40051c0 0x7fa5b80c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:32.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.309+0000 7fa5b659c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5a40051c0 0x7fa5b80c68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58924/0 (socket says 172.21.15.163:58924) 2023-10-09T06:32:32.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.309+0000 7fa5b659c700 1 -- 172.21.15.163:0/2631601253 learned_addr learned my addr 172.21.15.163:0/2631601253 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:32.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.309+0000 7fa5add9b700 1 --2- 172.21.15.163:0/2631601253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b815c3b0 0x7fa5b8151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:32.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.309+0000 7fa5b6d9d700 1 --2- 172.21.15.163:0/2631601253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5b81520f0 0x7fa5b80c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:32.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.310+0000 7fa5b659c700 1 -- 172.21.15.163:0/2631601253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5b81520f0 msgr2=0x7fa5b80c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:32.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.310+0000 7fa5b659c700 1 --2- 172.21.15.163:0/2631601253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5b81520f0 0x7fa5b80c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:32.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.310+0000 7fa5b659c700 1 -- 172.21.15.163:0/2631601253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b815c3b0 msgr2=0x7fa5b8151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:32.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.310+0000 7fa5b659c700 1 --2- 172.21.15.163:0/2631601253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b815c3b0 0x7fa5b8151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:32.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.310+0000 7fa5b659c700 1 -- 172.21.15.163:0/2631601253 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b815cc00 con 0x7fa5a40051c0 2023-10-09T06:32:32.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.310+0000 7fa5b659c700 1 --2- 172.21.15.163:0/2631601253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5a40051c0 0x7fa5b80c68d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80073c0 tx=0x7fa5a8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:32.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.310+0000 7fa5aeffd700 1 -- 172.21.15.163:0/2631601253 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a800f960 con 0x7fa5a40051c0 2023-10-09T06:32:32.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.310+0000 7fa5aeffd700 1 -- 172.21.15.163:0/2631601253 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a800fae0 con 0x7fa5a40051c0 2023-10-09T06:32:32.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.310+0000 7fa5bccd6700 1 -- 172.21.15.163:0/2631601253 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b815cee0 con 0x7fa5a40051c0 2023-10-09T06:32:32.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.310+0000 7fa5aeffd700 1 -- 172.21.15.163:0/2631601253 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a800fea0 con 0x7fa5a40051c0 2023-10-09T06:32:32.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.310+0000 7fa5bccd6700 1 -- 172.21.15.163:0/2631601253 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b80c6e10 con 0x7fa5a40051c0 2023-10-09T06:32:32.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.311+0000 7fa5bccd6700 1 -- 172.21.15.163:0/2631601253 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5b80cdc10 con 0x7fa5a40051c0 2023-10-09T06:32:32.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.311+0000 7fa5aeffd700 1 -- 172.21.15.163:0/2631601253 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fa5a80135c0 con 0x7fa5a40051c0 2023-10-09T06:32:32.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.312+0000 7fa5aeffd700 1 --2- 172.21.15.163:0/2631601253 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa598066fc0 0x7fa598069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:32.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.312+0000 7fa5aeffd700 1 -- 172.21.15.163:0/2631601253 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa5a80845b0 con 0x7fa5a40051c0 2023-10-09T06:32:32.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.312+0000 7fa5add9b700 1 --2- 172.21.15.163:0/2631601253 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa598066fc0 0x7fa598069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:32.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.312+0000 7fa5add9b700 1 --2- 172.21.15.163:0/2631601253 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa598066fc0 0x7fa598069470 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa5b80b6630 tx=0x7fa5a0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:32.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.316+0000 7fa5aeffd700 1 -- 172.21.15.163:0/2631601253 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5a8051730 con 0x7fa5a40051c0 2023-10-09T06:32:32.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:32.542+0000 7fa5bccd6700 1 -- 172.21.15.163:0/2631601253 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa5b804c3c0 con 0x7fa5a40051c0 2023-10-09T06:32:33.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.526+0000 7fa5aeffd700 1 -- 172.21.15.163:0/2631601253 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v39) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa5a8054d50 con 0x7fa5a40051c0 2023-10-09T06:32:33.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.529+0000 7fa5bccd6700 1 -- 172.21.15.163:0/2631601253 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa598066fc0 msgr2=0x7fa598069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:33.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.529+0000 7fa5bccd6700 1 --2- 172.21.15.163:0/2631601253 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa598066fc0 0x7fa598069470 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa5b80b6630 tx=0x7fa5a0008040 comp rx=0 tx=0).stop 2023-10-09T06:32:33.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.529+0000 7fa5bccd6700 1 -- 172.21.15.163:0/2631601253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5a40051c0 msgr2=0x7fa5b80c68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:33.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.529+0000 7fa5bccd6700 1 --2- 172.21.15.163:0/2631601253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5a40051c0 0x7fa5b80c68d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80073c0 tx=0x7fa5a8001e50 comp rx=0 tx=0).stop 2023-10-09T06:32:33.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.529+0000 7fa5bccd6700 1 -- 172.21.15.163:0/2631601253 shutdown_connections 2023-10-09T06:32:33.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.529+0000 7fa5bccd6700 1 --2- 172.21.15.163:0/2631601253 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa598066fc0 0x7fa598069470 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:33.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.529+0000 7fa5bccd6700 1 --2- 172.21.15.163:0/2631601253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5a40051c0 0x7fa5b80c68d0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:33.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.529+0000 7fa5bccd6700 1 --2- 172.21.15.163:0/2631601253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5b81520f0 0x7fa5b80c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:33.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.529+0000 7fa5bccd6700 1 --2- 172.21.15.163:0/2631601253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b815c3b0 0x7fa5b8151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:33.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.529+0000 7fa5bccd6700 1 -- 172.21.15.163:0/2631601253 >> 172.21.15.163:0/2631601253 conn(0x7fa5b8068e20 msgr2=0x7fa5b8153520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:33.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.529+0000 7fa5bccd6700 1 -- 172.21.15.163:0/2631601253 shutdown_connections 2023-10-09T06:32:33.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.529+0000 7fa5bccd6700 1 -- 172.21.15.163:0/2631601253 wait complete. 2023-10-09T06:32:33.545 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-09T06:32:33.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.668+0000 7f9d0ac3e700 1 Processor -- start 2023-10-09T06:32:33.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.668+0000 7f9d0ac3e700 1 -- start start 2023-10-09T06:32:33.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.669+0000 7f9d0ac3e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d04156490 0x7f9d040c7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:33.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.669+0000 7f9d0ac3e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9d040ccee0 0x7f9d040c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:33.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.669+0000 7f9d0ac3e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d040c7a70 con 0x7f9d04156860 2023-10-09T06:32:33.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.669+0000 7f9d0ac3e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d0415a450 con 0x7f9d040ccee0 2023-10-09T06:32:33.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.669+0000 7f9d0ac3e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0415a730 con 0x7f9d04156490 2023-10-09T06:32:33.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.669+0000 7f9cfbfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9d040ccee0 0x7f9d040c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:33.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.669+0000 7f9cfbfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9d040ccee0 0x7f9d040c81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58936/0 (socket says 172.21.15.163:58936) 2023-10-09T06:32:33.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.669+0000 7f9cfbfff700 1 -- 172.21.15.163:0/2498308153 learned_addr learned my addr 172.21.15.163:0/2498308153 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:33.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.670+0000 7f9cfbfff700 1 -- 172.21.15.163:0/2498308153 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d04156490 msgr2=0x7f9d040c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:33.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.670+0000 7f9cfbfff700 1 --2- 172.21.15.163:0/2498308153 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d04156490 0x7f9d040c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:33.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.670+0000 7f9cfbfff700 1 -- 172.21.15.163:0/2498308153 >> v1:172.21.15.181:6789/0 conn(0x7f9d04156860 legacy=0x7f9d0406eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:33.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.670+0000 7f9cfbfff700 1 -- 172.21.15.163:0/2498308153 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0415aa10 con 0x7f9d040ccee0 2023-10-09T06:32:33.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.670+0000 7f9cfbfff700 1 --2- 172.21.15.163:0/2498308153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d040ccee0 0x7f9d040c81a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9d040c7c00 tx=0x7f9cf4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6b8d23545264ce server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:33.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.670+0000 7f9cfb7fe700 1 -- 172.21.15.163:0/2498308153 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf400b820 con 0x7f9d040ccee0 2023-10-09T06:32:33.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.670+0000 7f9cfb7fe700 1 -- 172.21.15.163:0/2498308153 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf400b980 con 0x7f9d040ccee0 2023-10-09T06:32:33.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.670+0000 7f9cfb7fe700 1 -- 172.21.15.163:0/2498308153 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf400bd20 con 0x7f9d040ccee0 2023-10-09T06:32:33.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.670+0000 7f9d0ac3e700 1 -- 172.21.15.163:0/2498308153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d040ccee0 msgr2=0x7f9d040c81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:33.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.671+0000 7f9d0ac3e700 1 --2- 172.21.15.163:0/2498308153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d040ccee0 0x7f9d040c81a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9d040c7c00 tx=0x7f9cf4009250 comp rx=0 tx=0).stop 2023-10-09T06:32:33.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.671+0000 7f9d0ac3e700 1 -- 172.21.15.163:0/2498308153 shutdown_connections 2023-10-09T06:32:33.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.671+0000 7f9d0ac3e700 1 --2- 172.21.15.163:0/2498308153 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d04156490 0x7f9d040c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:33.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.671+0000 7f9d0ac3e700 1 --2- 172.21.15.163:0/2498308153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d040ccee0 0x7f9d040c81a0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:33.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.671+0000 7f9d0ac3e700 1 -- 172.21.15.163:0/2498308153 >> 172.21.15.163:0/2498308153 conn(0x7f9d04068e10 msgr2=0x7f9d040cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:33.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.671+0000 7f9d0ac3e700 1 -- 172.21.15.163:0/2498308153 shutdown_connections 2023-10-09T06:32:33.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.671+0000 7f9d0ac3e700 1 -- 172.21.15.163:0/2498308153 wait complete. 2023-10-09T06:32:33.677 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.672+0000 7f9d0ac3e700 1 Processor -- start 2023-10-09T06:32:33.677 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.672+0000 7f9d0ac3e700 1 -- start start 2023-10-09T06:32:33.677 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.673+0000 7f9d0ac3e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d040ccee0 0x7f9d040b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:33.677 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.673+0000 7f9d0ac3e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d04156490 0x7f9d040af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:33.677 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.673+0000 7f9d0ac3e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d04156860 0x7f9d040af810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:33.677 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.673+0000 7f9d0ac3e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0414e550 con 0x7f9d04156490 2023-10-09T06:32:33.678 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.673+0000 7f9d0ac3e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d04159870 con 0x7f9d04156860 2023-10-09T06:32:33.678 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.673+0000 7f9d0ac3e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0414d940 con 0x7f9d040ccee0 2023-10-09T06:32:33.678 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.673+0000 7f9cfbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d04156490 0x7f9d040af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:33.678 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.673+0000 7f9cfbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d04156490 0x7f9d040af2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58942/0 (socket says 172.21.15.163:58942) 2023-10-09T06:32:33.678 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.673+0000 7f9cfbfff700 1 -- 172.21.15.163:0/2864722038 learned_addr learned my addr 172.21.15.163:0/2864722038 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:33.678 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.673+0000 7f9d089da700 1 --2- 172.21.15.163:0/2864722038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d040ccee0 0x7f9d040b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:33.678 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.673+0000 7f9d091db700 1 --2- 172.21.15.163:0/2864722038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d04156860 0x7f9d040af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:33.678 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.673+0000 7f9cfbfff700 1 -- 172.21.15.163:0/2864722038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d040ccee0 msgr2=0x7f9d040b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:33.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.673+0000 7f9cfbfff700 1 --2- 172.21.15.163:0/2864722038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d040ccee0 0x7f9d040b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:33.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.673+0000 7f9cfbfff700 1 -- 172.21.15.163:0/2864722038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d04156860 msgr2=0x7f9d040af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:33.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.673+0000 7f9cfbfff700 1 --2- 172.21.15.163:0/2864722038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d04156860 0x7f9d040af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:33.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.673+0000 7f9cfbfff700 1 -- 172.21.15.163:0/2864722038 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf40094f0 con 0x7f9d04156490 2023-10-09T06:32:33.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.673+0000 7f9d089da700 1 --2- 172.21.15.163:0/2864722038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d040ccee0 0x7f9d040b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:33.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.673+0000 7f9d091db700 1 --2- 172.21.15.163:0/2864722038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d04156860 0x7f9d040af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:33.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.673+0000 7f9cfbfff700 1 --2- 172.21.15.163:0/2864722038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d04156490 0x7f9d040af2d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4009290 tx=0x7f9cf4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:33.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.673+0000 7f9cf8ff9700 1 -- 172.21.15.163:0/2864722038 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4011e20 con 0x7f9d04156490 2023-10-09T06:32:33.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.673+0000 7f9cf8ff9700 1 -- 172.21.15.163:0/2864722038 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf401f070 con 0x7f9d04156490 2023-10-09T06:32:33.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.673+0000 7f9d0ac3e700 1 -- 172.21.15.163:0/2864722038 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d0414dc20 con 0x7f9d04156490 2023-10-09T06:32:33.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.673+0000 7f9cf8ff9700 1 -- 172.21.15.163:0/2864722038 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf401a890 con 0x7f9d04156490 2023-10-09T06:32:33.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.673+0000 7f9d0ac3e700 1 -- 172.21.15.163:0/2864722038 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0414e1e0 con 0x7f9d04156490 2023-10-09T06:32:33.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.674+0000 7f9d0ac3e700 1 -- 172.21.15.163:0/2864722038 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d0404c3c0 con 0x7f9d04156490 2023-10-09T06:32:33.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.675+0000 7f9cf8ff9700 1 -- 172.21.15.163:0/2864722038 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f9cf401ac40 con 0x7f9d04156490 2023-10-09T06:32:33.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.675+0000 7f9cf8ff9700 1 --2- 172.21.15.163:0/2864722038 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9cf0066fc0 0x7f9cf0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:33.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.675+0000 7f9cf8ff9700 1 -- 172.21.15.163:0/2864722038 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9cf4021070 con 0x7f9d04156490 2023-10-09T06:32:33.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.675+0000 7f9d089da700 1 --2- 172.21.15.163:0/2864722038 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9cf0066fc0 0x7f9cf0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:33.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.676+0000 7f9d089da700 1 --2- 172.21.15.163:0/2864722038 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9cf0066fc0 0x7f9cf0069470 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9d00001370 tx=0x7f9d000096a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:33.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.680+0000 7f9cf8ff9700 1 -- 172.21.15.163:0/2864722038 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9cf40536e0 con 0x7f9d04156490 2023-10-09T06:32:33.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:33.904+0000 7f9d0ac3e700 1 -- 172.21.15.163:0/2864722038 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9d0415acf0 con 0x7f9d04156490 2023-10-09T06:32:34.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.531+0000 7f9cf8ff9700 1 -- 172.21.15.163:0/2864722038 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9cf4056d00 con 0x7f9d04156490 2023-10-09T06:32:34.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.534+0000 7f9d0ac3e700 1 -- 172.21.15.163:0/2864722038 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9cf0066fc0 msgr2=0x7f9cf0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:34.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.534+0000 7f9d0ac3e700 1 --2- 172.21.15.163:0/2864722038 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9cf0066fc0 0x7f9cf0069470 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9d00001370 tx=0x7f9d000096a0 comp rx=0 tx=0).stop 2023-10-09T06:32:34.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.534+0000 7f9d0ac3e700 1 -- 172.21.15.163:0/2864722038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d04156490 msgr2=0x7f9d040af2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:34.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.534+0000 7f9d0ac3e700 1 --2- 172.21.15.163:0/2864722038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d04156490 0x7f9d040af2d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4009290 tx=0x7f9cf4017000 comp rx=0 tx=0).stop 2023-10-09T06:32:34.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.534+0000 7f9d0ac3e700 1 -- 172.21.15.163:0/2864722038 shutdown_connections 2023-10-09T06:32:34.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.534+0000 7f9d0ac3e700 1 --2- 172.21.15.163:0/2864722038 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9cf0066fc0 0x7f9cf0069470 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:34.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.534+0000 7f9d0ac3e700 1 --2- 172.21.15.163:0/2864722038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d04156860 0x7f9d040af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:34.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.534+0000 7f9d0ac3e700 1 --2- 172.21.15.163:0/2864722038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d040ccee0 0x7f9d040b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:34.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.534+0000 7f9d0ac3e700 1 --2- 172.21.15.163:0/2864722038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d04156490 0x7f9d040af2d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:34.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.534+0000 7f9d0ac3e700 1 -- 172.21.15.163:0/2864722038 >> 172.21.15.163:0/2864722038 conn(0x7f9d04068e10 msgr2=0x7f9d04069770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:34.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.534+0000 7f9d0ac3e700 1 -- 172.21.15.163:0/2864722038 shutdown_connections 2023-10-09T06:32:34.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.534+0000 7f9d0ac3e700 1 -- 172.21.15.163:0/2864722038 wait complete. 2023-10-09T06:32:34.548 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:32:34.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.664+0000 7f37d5aee700 1 Processor -- start 2023-10-09T06:32:34.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.664+0000 7f37d5aee700 1 -- start start 2023-10-09T06:32:34.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.665+0000 7f37d5aee700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37d0159890 0x7f37d015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:34.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.665+0000 7f37d5aee700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f37d0159c60 0x7f37d015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:34.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.665+0000 7f37d5aee700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37d015a160 con 0x7f37d015db10 2023-10-09T06:32:34.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.665+0000 7f37d5aee700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37d00d13a0 con 0x7f37d0159c60 2023-10-09T06:32:34.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.665+0000 7f37d5aee700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d015e140 con 0x7f37d0159890 2023-10-09T06:32:34.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.665+0000 7f37ceffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f37d0159c60 0x7f37d015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:34.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.665+0000 7f37cffff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f37d015db10 0x7f37d0069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:44798/0 (socket says 172.21.15.163:44798) 2023-10-09T06:32:34.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.665+0000 7f37cffff700 1 -- 172.21.15.163:0/1261597944 learned_addr learned my addr 172.21.15.163:0/1261597944 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:34.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.665+0000 7f37cf7fe700 1 --2- 172.21.15.163:0/1261597944 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37d0159890 0x7f37d015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:34.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.666+0000 7f37ceffd700 1 -- 172.21.15.163:0/1261597944 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37d0159890 msgr2=0x7f37d015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:34.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.666+0000 7f37ceffd700 1 --2- 172.21.15.163:0/1261597944 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37d0159890 0x7f37d015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:34.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.666+0000 7f37ceffd700 1 -- 172.21.15.163:0/1261597944 >> v1:172.21.15.163:6789/0 conn(0x7f37d015db10 legacy=0x7f37d0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:34.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.666+0000 7f37ceffd700 1 -- 172.21.15.163:0/1261597944 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d015a3d0 con 0x7f37d0159c60 2023-10-09T06:32:34.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.666+0000 7f37ce7fc700 1 -- 172.21.15.163:0/1261597944 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3247398505 0 0) 0x7f37d015a160 con 0x7f37d015db10 2023-10-09T06:32:34.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.666+0000 7f37ceffd700 1 --2- 172.21.15.163:0/1261597944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37d0159c60 0x7f37d015d5d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f37c40098e0 tx=0x7f37c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b2e29531a9ff5 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:34.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.666+0000 7f37ce7fc700 1 -- 172.21.15.163:0/1261597944 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c4009e80 con 0x7f37d0159c60 2023-10-09T06:32:34.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.666+0000 7f37ce7fc700 1 -- 172.21.15.163:0/1261597944 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c400b860 con 0x7f37d0159c60 2023-10-09T06:32:34.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.666+0000 7f37ce7fc700 1 -- 172.21.15.163:0/1261597944 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c400bbe0 con 0x7f37d0159c60 2023-10-09T06:32:34.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.666+0000 7f37d5aee700 1 -- 172.21.15.163:0/1261597944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37d0159c60 msgr2=0x7f37d015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:34.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.666+0000 7f37d5aee700 1 --2- 172.21.15.163:0/1261597944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37d0159c60 0x7f37d015d5d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f37c40098e0 tx=0x7f37c40091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:34.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.666+0000 7f37d5aee700 1 -- 172.21.15.163:0/1261597944 shutdown_connections 2023-10-09T06:32:34.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.666+0000 7f37d5aee700 1 --2- 172.21.15.163:0/1261597944 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37d0159890 0x7f37d015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:34.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.666+0000 7f37d5aee700 1 --2- 172.21.15.163:0/1261597944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37d0159c60 0x7f37d015d5d0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:34.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.666+0000 7f37d5aee700 1 -- 172.21.15.163:0/1261597944 >> 172.21.15.163:0/1261597944 conn(0x7f37d0068e20 msgr2=0x7f37d00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:34.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.666+0000 7f37d5aee700 1 -- 172.21.15.163:0/1261597944 shutdown_connections 2023-10-09T06:32:34.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.667+0000 7f37d5aee700 1 -- 172.21.15.163:0/1261597944 wait complete. 2023-10-09T06:32:34.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.668+0000 7f37d5aee700 1 Processor -- start 2023-10-09T06:32:34.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.668+0000 7f37d5aee700 1 -- start start 2023-10-09T06:32:34.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.668+0000 7f37d5aee700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37d0159890 0x7f37d00c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:34.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.668+0000 7f37d5aee700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37d0159c60 0x7f37d01471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:34.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.668+0000 7f37d5aee700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37d015db10 0x7f37d00c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:34.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.668+0000 7f37d5aee700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d00ce7a0 con 0x7f37d0159c60 2023-10-09T06:32:34.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.668+0000 7f37d5aee700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d00cea40 con 0x7f37d015db10 2023-10-09T06:32:34.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.668+0000 7f37d5aee700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d00ced20 con 0x7f37d0159890 2023-10-09T06:32:34.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.669+0000 7f37ceffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37d0159c60 0x7f37d01471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:34.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.669+0000 7f37ceffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37d0159c60 0x7f37d01471f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58958/0 (socket says 172.21.15.163:58958) 2023-10-09T06:32:34.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.669+0000 7f37ceffd700 1 -- 172.21.15.163:0/3104189434 learned_addr learned my addr 172.21.15.163:0/3104189434 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:34.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.669+0000 7f37cf7fe700 1 --2- 172.21.15.163:0/3104189434 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37d0159890 0x7f37d00c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:34.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.669+0000 7f37cffff700 1 --2- 172.21.15.163:0/3104189434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37d015db10 0x7f37d00c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:34.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.669+0000 7f37ceffd700 1 -- 172.21.15.163:0/3104189434 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37d0159890 msgr2=0x7f37d00c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:34.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.669+0000 7f37ceffd700 1 --2- 172.21.15.163:0/3104189434 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37d0159890 0x7f37d00c9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:34.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.669+0000 7f37ceffd700 1 -- 172.21.15.163:0/3104189434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37d015db10 msgr2=0x7f37d00c7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:34.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.669+0000 7f37ceffd700 1 --2- 172.21.15.163:0/3104189434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37d015db10 0x7f37d00c7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:34.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.669+0000 7f37ceffd700 1 -- 172.21.15.163:0/3104189434 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c4009420 con 0x7f37d0159c60 2023-10-09T06:32:34.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.669+0000 7f37cffff700 1 --2- 172.21.15.163:0/3104189434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37d015db10 0x7f37d00c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:34.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.669+0000 7f37ceffd700 1 --2- 172.21.15.163:0/3104189434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37d0159c60 0x7f37d01471f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f37c4013000 tx=0x7f37c4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:34.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.669+0000 7f37b77fe700 1 -- 172.21.15.163:0/3104189434 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c400bbe0 con 0x7f37d0159c60 2023-10-09T06:32:34.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.669+0000 7f37b77fe700 1 -- 172.21.15.163:0/3104189434 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c4011420 con 0x7f37d0159c60 2023-10-09T06:32:34.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.669+0000 7f37d5aee700 1 -- 172.21.15.163:0/3104189434 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d00cf000 con 0x7f37d0159c60 2023-10-09T06:32:34.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.669+0000 7f37b77fe700 1 -- 172.21.15.163:0/3104189434 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c40117e0 con 0x7f37d0159c60 2023-10-09T06:32:34.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.669+0000 7f37d5aee700 1 -- 172.21.15.163:0/3104189434 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d0147730 con 0x7f37d0159c60 2023-10-09T06:32:34.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.670+0000 7f37d5aee700 1 -- 172.21.15.163:0/3104189434 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d00cf240 con 0x7f37d0159c60 2023-10-09T06:32:34.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.671+0000 7f37b77fe700 1 -- 172.21.15.163:0/3104189434 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f37c4011bd0 con 0x7f37d0159c60 2023-10-09T06:32:34.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.671+0000 7f37b77fe700 1 --2- 172.21.15.163:0/3104189434 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f37bc066fc0 0x7f37bc069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:34.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.671+0000 7f37b77fe700 1 -- 172.21.15.163:0/3104189434 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f37c40860a0 con 0x7f37d0159c60 2023-10-09T06:32:34.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.671+0000 7f37cf7fe700 1 --2- 172.21.15.163:0/3104189434 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f37bc066fc0 0x7f37bc069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:34.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.672+0000 7f37cf7fe700 1 --2- 172.21.15.163:0/3104189434 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f37bc066fc0 0x7f37bc069470 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f37c000aa80 tx=0x7f37c0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:34.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.675+0000 7f37b77fe700 1 -- 172.21.15.163:0/3104189434 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37d00cf240 con 0x7f37d0159c60 2023-10-09T06:32:34.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.901+0000 7f37d5aee700 1 -- 172.21.15.163:0/3104189434 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37d00cf240 con 0x7f37d0159c60 2023-10-09T06:32:34.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.901+0000 7f37b77fe700 1 -- 172.21.15.163:0/3104189434 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4608 (secure 0 0 0) 0x7f37d00cf240 con 0x7f37d0159c60 2023-10-09T06:32:34.902 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:34.902 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5407,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3673547262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3673547262},{"type":"v1","addr":"172.21.15.181:6837","nonce":3673547262}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5410,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1705122825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1705122825},{"type":"v1","addr":"172.21.15.181:6836","nonce":1705122825}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/596332711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":596332711},{"type":"v1","addr":"172.21.15.163:6834","nonce":596332711}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:32:30.882925+0000","modified":"2023-10-09T06:32:34.526904+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/3415100865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3415100865},{"type":"v1","addr":"172.21.15.163:6835","nonce":3415100865}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-09T06:32:34.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.903+0000 7f37d5aee700 1 -- 172.21.15.163:0/3104189434 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f37bc066fc0 msgr2=0x7f37bc069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:34.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.903+0000 7f37d5aee700 1 --2- 172.21.15.163:0/3104189434 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f37bc066fc0 0x7f37bc069470 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f37c000aa80 tx=0x7f37c0008040 comp rx=0 tx=0).stop 2023-10-09T06:32:34.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.903+0000 7f37d5aee700 1 -- 172.21.15.163:0/3104189434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37d0159c60 msgr2=0x7f37d01471f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:34.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.903+0000 7f37d5aee700 1 --2- 172.21.15.163:0/3104189434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37d0159c60 0x7f37d01471f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f37c4013000 tx=0x7f37c4017000 comp rx=0 tx=0).stop 2023-10-09T06:32:34.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.903+0000 7f37d5aee700 1 -- 172.21.15.163:0/3104189434 shutdown_connections 2023-10-09T06:32:34.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.903+0000 7f37d5aee700 1 --2- 172.21.15.163:0/3104189434 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f37bc066fc0 0x7f37bc069470 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:34.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.903+0000 7f37d5aee700 1 --2- 172.21.15.163:0/3104189434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37d015db10 0x7f37d00c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:34.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.903+0000 7f37d5aee700 1 --2- 172.21.15.163:0/3104189434 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37d0159890 0x7f37d00c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:34.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.903+0000 7f37d5aee700 1 --2- 172.21.15.163:0/3104189434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37d0159c60 0x7f37d01471f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:34.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.903+0000 7f37d5aee700 1 -- 172.21.15.163:0/3104189434 >> 172.21.15.163:0/3104189434 conn(0x7f37d0068e20 msgr2=0x7f37d0152fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:34.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.903+0000 7f37d5aee700 1 -- 172.21.15.163:0/3104189434 shutdown_connections 2023-10-09T06:32:34.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:34.903+0000 7f37d5aee700 1 -- 172.21.15.163:0/3104189434 wait complete. 2023-10-09T06:32:34.906 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 40 2023-10-09T06:32:34.916 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:32:35.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.030+0000 7f8c56410700 1 Processor -- start 2023-10-09T06:32:35.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.030+0000 7f8c56410700 1 -- start start 2023-10-09T06:32:35.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.031+0000 7f8c56410700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8c5006bcd0 0x7f8c50140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:35.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.031+0000 7f8c56410700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8c500c9510 0x7f8c50141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:35.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.031+0000 7f8c56410700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c5015c2a0 con 0x7f8c5006c0a0 2023-10-09T06:32:35.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.031+0000 7f8c56410700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c500c9900 con 0x7f8c5006bcd0 2023-10-09T06:32:35.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.031+0000 7f8c56410700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c5015b690 con 0x7f8c500c9510 2023-10-09T06:32:35.034 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.031+0000 7f8c4ffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8c5006bcd0 0x7f8c50140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:35.034 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.031+0000 7f8c4f7fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f8c5006c0a0 0x7f8c500c8e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:44802/0 (socket says 172.21.15.163:44802) 2023-10-09T06:32:35.034 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.031+0000 7f8c4f7fe700 1 -- 172.21.15.163:0/2670666259 learned_addr learned my addr 172.21.15.163:0/2670666259 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:35.034 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.031+0000 7f8c549ad700 1 --2- 172.21.15.163:0/2670666259 >> v2:172.21.15.181:3300/0 conn(0x7f8c500c9510 0x7f8c50141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:35.034 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.031+0000 7f8c4ffff700 1 -- 172.21.15.163:0/2670666259 >> v2:172.21.15.181:3300/0 conn(0x7f8c500c9510 msgr2=0x7f8c50141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:35.034 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.031+0000 7f8c4ffff700 1 --2- 172.21.15.163:0/2670666259 >> v2:172.21.15.181:3300/0 conn(0x7f8c500c9510 0x7f8c50141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.034 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.031+0000 7f8c4ffff700 1 -- 172.21.15.163:0/2670666259 >> v1:172.21.15.163:6789/0 conn(0x7f8c5006c0a0 legacy=0x7f8c500c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:35.034 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.031+0000 7f8c4effd700 1 -- 172.21.15.163:0/2670666259 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3395145142 0 0) 0x7f8c5015c2a0 con 0x7f8c5006c0a0 2023-10-09T06:32:35.034 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.031+0000 7f8c4ffff700 1 -- 172.21.15.163:0/2670666259 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c5015b970 con 0x7f8c5006bcd0 2023-10-09T06:32:35.034 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.032+0000 7f8c4ffff700 1 --2- 172.21.15.163:0/2670666259 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c5006bcd0 0x7f8c50140df0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8c500ce190 tx=0x7f8c400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1077bcc80863164 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:35.034 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.032+0000 7f8c4effd700 1 -- 172.21.15.163:0/2670666259 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4000b6b0 con 0x7f8c5006bcd0 2023-10-09T06:32:35.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.032+0000 7f8c4effd700 1 -- 172.21.15.163:0/2670666259 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4000b810 con 0x7f8c5006bcd0 2023-10-09T06:32:35.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.032+0000 7f8c4effd700 1 -- 172.21.15.163:0/2670666259 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4000bbb0 con 0x7f8c5006bcd0 2023-10-09T06:32:35.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.032+0000 7f8c56410700 1 -- 172.21.15.163:0/2670666259 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c5006bcd0 msgr2=0x7f8c50140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:35.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.032+0000 7f8c56410700 1 --2- 172.21.15.163:0/2670666259 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c5006bcd0 0x7f8c50140df0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8c500ce190 tx=0x7f8c400091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.032+0000 7f8c56410700 1 -- 172.21.15.163:0/2670666259 shutdown_connections 2023-10-09T06:32:35.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.032+0000 7f8c56410700 1 --2- 172.21.15.163:0/2670666259 >> v2:172.21.15.181:3300/0 conn(0x7f8c500c9510 0x7f8c50141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.032+0000 7f8c56410700 1 --2- 172.21.15.163:0/2670666259 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c5006bcd0 0x7f8c50140df0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.032+0000 7f8c56410700 1 -- 172.21.15.163:0/2670666259 >> 172.21.15.163:0/2670666259 conn(0x7f8c500ce600 msgr2=0x7f8c500cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:35.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.032+0000 7f8c56410700 1 -- 172.21.15.163:0/2670666259 shutdown_connections 2023-10-09T06:32:35.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.032+0000 7f8c56410700 1 -- 172.21.15.163:0/2670666259 wait complete. 2023-10-09T06:32:35.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.033+0000 7f8c56410700 1 Processor -- start 2023-10-09T06:32:35.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.034+0000 7f8c56410700 1 -- start start 2023-10-09T06:32:35.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.034+0000 7f8c56410700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c5006bcd0 0x7f8c50157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:35.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.034+0000 7f8c56410700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c5006c0a0 0x7f8c5006a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:35.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.034+0000 7f8c56410700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c500c9510 0x7f8c500b2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:35.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.034+0000 7f8c56410700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c500b3810 con 0x7f8c5006c0a0 2023-10-09T06:32:35.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.034+0000 7f8c56410700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c50157870 con 0x7f8c5006bcd0 2023-10-09T06:32:35.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.034+0000 7f8c56410700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5016d1e0 con 0x7f8c500c9510 2023-10-09T06:32:35.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.034+0000 7f8c4f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c5006c0a0 0x7f8c5006a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:35.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.034+0000 7f8c4f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c5006c0a0 0x7f8c5006a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58972/0 (socket says 172.21.15.163:58972) 2023-10-09T06:32:35.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.034+0000 7f8c4f7fe700 1 -- 172.21.15.163:0/3282696222 learned_addr learned my addr 172.21.15.163:0/3282696222 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:35.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.034+0000 7f8c549ad700 1 --2- 172.21.15.163:0/3282696222 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c500c9510 0x7f8c500b2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:35.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.034+0000 7f8c4ffff700 1 --2- 172.21.15.163:0/3282696222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c5006bcd0 0x7f8c50157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:35.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.035+0000 7f8c4f7fe700 1 -- 172.21.15.163:0/3282696222 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c500c9510 msgr2=0x7f8c500b2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:35.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.035+0000 7f8c4f7fe700 1 --2- 172.21.15.163:0/3282696222 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c500c9510 0x7f8c500b2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.035+0000 7f8c4f7fe700 1 -- 172.21.15.163:0/3282696222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c5006bcd0 msgr2=0x7f8c50157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:35.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.035+0000 7f8c4f7fe700 1 --2- 172.21.15.163:0/3282696222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c5006bcd0 0x7f8c50157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.035+0000 7f8c4f7fe700 1 -- 172.21.15.163:0/3282696222 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c40009420 con 0x7f8c5006c0a0 2023-10-09T06:32:35.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.035+0000 7f8c549ad700 1 --2- 172.21.15.163:0/3282696222 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c500c9510 0x7f8c500b2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:35.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.035+0000 7f8c4ffff700 1 --2- 172.21.15.163:0/3282696222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c5006bcd0 0x7f8c50157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:35.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.035+0000 7f8c4f7fe700 1 --2- 172.21.15.163:0/3282696222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c5006c0a0 0x7f8c5006a520 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8c4400c7e0 tx=0x7f8c44007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:35.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.035+0000 7f8c37fff700 1 -- 172.21.15.163:0/3282696222 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c44005b30 con 0x7f8c5006c0a0 2023-10-09T06:32:35.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.035+0000 7f8c37fff700 1 -- 172.21.15.163:0/3282696222 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c44005c90 con 0x7f8c5006c0a0 2023-10-09T06:32:35.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.035+0000 7f8c37fff700 1 -- 172.21.15.163:0/3282696222 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c44007300 con 0x7f8c5006c0a0 2023-10-09T06:32:35.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.035+0000 7f8c56410700 1 -- 172.21.15.163:0/3282696222 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c5016d480 con 0x7f8c5006c0a0 2023-10-09T06:32:35.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.035+0000 7f8c56410700 1 -- 172.21.15.163:0/3282696222 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c5016da40 con 0x7f8c5006c0a0 2023-10-09T06:32:35.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.036+0000 7f8c56410700 1 -- 172.21.15.163:0/3282696222 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c5004c3c0 con 0x7f8c5006c0a0 2023-10-09T06:32:35.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.036+0000 7f8c37fff700 1 -- 172.21.15.163:0/3282696222 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f8c4401b400 con 0x7f8c5006c0a0 2023-10-09T06:32:35.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.037+0000 7f8c37fff700 1 --2- 172.21.15.163:0/3282696222 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8c3c066fc0 0x7f8c3c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:35.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.037+0000 7f8c37fff700 1 -- 172.21.15.163:0/3282696222 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8c44084030 con 0x7f8c5006c0a0 2023-10-09T06:32:35.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.037+0000 7f8c4ffff700 1 --2- 172.21.15.163:0/3282696222 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8c3c066fc0 0x7f8c3c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:35.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.037+0000 7f8c4ffff700 1 --2- 172.21.15.163:0/3282696222 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8c3c066fc0 0x7f8c3c069470 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8c400093f0 tx=0x7f8c40017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:35.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.041+0000 7f8c37fff700 1 -- 172.21.15.163:0/3282696222 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c44051330 con 0x7f8c5006c0a0 2023-10-09T06:32:35.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.229+0000 7f8c56410700 1 -- 172.21.15.163:0/3282696222 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c5006c9d0 con 0x7f8c5006c0a0 2023-10-09T06:32:35.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.229+0000 7f8c37fff700 1 -- 172.21.15.163:0/3282696222 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+15401 (secure 0 0 0) 0x7f8c44054950 con 0x7f8c5006c0a0 2023-10-09T06:32:35.230 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:35.230 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":63,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:32:32.496598+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:32:28.847438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-09T06:32:29.837643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:32:35.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.232+0000 7f8c56410700 1 -- 172.21.15.163:0/3282696222 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8c3c066fc0 msgr2=0x7f8c3c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:35.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.232+0000 7f8c56410700 1 --2- 172.21.15.163:0/3282696222 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8c3c066fc0 0x7f8c3c069470 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8c400093f0 tx=0x7f8c40017000 comp rx=0 tx=0).stop 2023-10-09T06:32:35.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.232+0000 7f8c56410700 1 -- 172.21.15.163:0/3282696222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c5006c0a0 msgr2=0x7f8c5006a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:35.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.232+0000 7f8c56410700 1 --2- 172.21.15.163:0/3282696222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c5006c0a0 0x7f8c5006a520 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8c4400c7e0 tx=0x7f8c44007df0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.232+0000 7f8c56410700 1 -- 172.21.15.163:0/3282696222 shutdown_connections 2023-10-09T06:32:35.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.232+0000 7f8c56410700 1 --2- 172.21.15.163:0/3282696222 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8c3c066fc0 0x7f8c3c069470 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.232+0000 7f8c56410700 1 --2- 172.21.15.163:0/3282696222 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c500c9510 0x7f8c500b2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.232+0000 7f8c56410700 1 --2- 172.21.15.163:0/3282696222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c5006bcd0 0x7f8c50157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.232+0000 7f8c56410700 1 --2- 172.21.15.163:0/3282696222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c5006c0a0 0x7f8c5006a520 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.232+0000 7f8c56410700 1 -- 172.21.15.163:0/3282696222 >> 172.21.15.163:0/3282696222 conn(0x7f8c500ce600 msgr2=0x7f8c50140bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:35.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.232+0000 7f8c56410700 1 -- 172.21.15.163:0/3282696222 shutdown_connections 2023-10-09T06:32:35.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.232+0000 7f8c56410700 1 -- 172.21.15.163:0/3282696222 wait complete. 2023-10-09T06:32:35.245 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-09T06:32:35.245 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:32:35.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.360+0000 7f0b75941700 1 Processor -- start 2023-10-09T06:32:35.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.361+0000 7f0b75941700 1 -- start start 2023-10-09T06:32:35.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.361+0000 7f0b75941700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b70159890 0x7f0b7015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:35.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.361+0000 7f0b75941700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0b70159c60 0x7f0b7015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:35.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.361+0000 7f0b75941700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b7015a160 con 0x7f0b7015db10 2023-10-09T06:32:35.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.361+0000 7f0b75941700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b700d13a0 con 0x7f0b70159c60 2023-10-09T06:32:35.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.361+0000 7f0b75941700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7015e140 con 0x7f0b70159890 2023-10-09T06:32:35.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.362+0000 7f0b6f7fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f0b7015db10 0x7f0b70069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:44804/0 (socket says 172.21.15.163:44804) 2023-10-09T06:32:35.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.362+0000 7f0b6f7fe700 1 -- 172.21.15.163:0/2875050533 learned_addr learned my addr 172.21.15.163:0/2875050533 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:35.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.362+0000 7f0b67fff700 1 --2- 172.21.15.163:0/2875050533 >> v2:172.21.15.181:3300/0 conn(0x7f0b70159c60 0x7f0b7015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:35.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.362+0000 7f0b6effd700 1 --2- 172.21.15.163:0/2875050533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b70159890 0x7f0b7015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:35.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.362+0000 7f0b6e7fc700 1 -- 172.21.15.163:0/2875050533 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3743290620 0 0) 0x7f0b7015a160 con 0x7f0b7015db10 2023-10-09T06:32:35.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.362+0000 7f0b6e7fc700 1 -- 172.21.15.163:0/2875050533 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b5c003570 con 0x7f0b7015db10 2023-10-09T06:32:35.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.362+0000 7f0b6e7fc700 1 -- 172.21.15.163:0/2875050533 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 733343409 0 0) 0x7f0b5c003570 con 0x7f0b7015db10 2023-10-09T06:32:35.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.362+0000 7f0b6e7fc700 1 -- 172.21.15.163:0/2875050533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b70159890 msgr2=0x7f0b7015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:35.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.362+0000 7f0b6e7fc700 1 --2- 172.21.15.163:0/2875050533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b70159890 0x7f0b7015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.362+0000 7f0b6e7fc700 1 -- 172.21.15.163:0/2875050533 >> v2:172.21.15.181:3300/0 conn(0x7f0b70159c60 msgr2=0x7f0b7015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:35.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.362+0000 7f0b6e7fc700 1 --2- 172.21.15.163:0/2875050533 >> v2:172.21.15.181:3300/0 conn(0x7f0b70159c60 0x7f0b7015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.362+0000 7f0b6e7fc700 1 -- 172.21.15.163:0/2875050533 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b7015a3d0 con 0x7f0b7015db10 2023-10-09T06:32:35.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.362+0000 7f0b6e7fc700 1 -- 172.21.15.163:0/2875050533 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f0b680017f0 con 0x7f0b7015db10 2023-10-09T06:32:35.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.362+0000 7f0b6e7fc700 1 -- 172.21.15.163:0/2875050533 >> v1:172.21.15.163:6789/0 conn(0x7f0b7015db10 legacy=0x7f0b70069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:35.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.362+0000 7f0b6e7fc700 1 --2- 172.21.15.163:0/2875050533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b70159890 0x7f0b5c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:35.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.362+0000 7f0b6e7fc700 1 --2- 172.21.15.163:0/2875050533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b5c003fe0 0x7f0b5c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:35.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.362+0000 7f0b6e7fc700 1 --2- 172.21.15.163:0/2875050533 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b5c004d50 0x7f0b5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:35.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.362+0000 7f0b6e7fc700 1 -- 172.21.15.163:0/2875050533 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b700d13a0 con 0x7f0b5c004d50 2023-10-09T06:32:35.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.362+0000 7f0b6e7fc700 1 -- 172.21.15.163:0/2875050533 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7015e140 con 0x7f0b5c003fe0 2023-10-09T06:32:35.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.362+0000 7f0b6e7fc700 1 -- 172.21.15.163:0/2875050533 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c00ba90 con 0x7f0b70159890 2023-10-09T06:32:35.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.362+0000 7f0b6f7fe700 1 --2- 172.21.15.163:0/2875050533 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b5c004d50 0x7f0b5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:35.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.362+0000 7f0b67fff700 1 --2- 172.21.15.163:0/2875050533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b5c003fe0 0x7f0b5c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:35.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.362+0000 7f0b6effd700 1 --2- 172.21.15.163:0/2875050533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b70159890 0x7f0b5c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:35.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.363+0000 7f0b6f7fe700 1 -- 172.21.15.163:0/2875050533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b70159890 msgr2=0x7f0b5c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:35.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.363+0000 7f0b6f7fe700 1 --2- 172.21.15.163:0/2875050533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b70159890 0x7f0b5c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.363+0000 7f0b6f7fe700 1 -- 172.21.15.163:0/2875050533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b5c003fe0 msgr2=0x7f0b5c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:35.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.363+0000 7f0b6f7fe700 1 --2- 172.21.15.163:0/2875050533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b5c003fe0 0x7f0b5c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.363+0000 7f0b6f7fe700 1 -- 172.21.15.163:0/2875050533 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5c003570 con 0x7f0b5c004d50 2023-10-09T06:32:35.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.363+0000 7f0b6f7fe700 1 --2- 172.21.15.163:0/2875050533 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b5c004d50 0x7f0b5c0051c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800a970 tx=0x7f0b68002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ea3a693f9a3898 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:35.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.363+0000 7f0b6e7fc700 1 -- 172.21.15.163:0/2875050533 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68012440 con 0x7f0b5c004d50 2023-10-09T06:32:35.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.363+0000 7f0b6e7fc700 1 -- 172.21.15.163:0/2875050533 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b680125c0 con 0x7f0b5c004d50 2023-10-09T06:32:35.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.363+0000 7f0b6e7fc700 1 -- 172.21.15.163:0/2875050533 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68012980 con 0x7f0b5c004d50 2023-10-09T06:32:35.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.363+0000 7f0b75941700 1 -- 172.21.15.163:0/2875050533 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b5c004d50 msgr2=0x7f0b5c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:35.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.363+0000 7f0b75941700 1 --2- 172.21.15.163:0/2875050533 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b5c004d50 0x7f0b5c0051c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800a970 tx=0x7f0b68002210 comp rx=0 tx=0).stop 2023-10-09T06:32:35.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.363+0000 7f0b6f7fe700 1 -- 172.21.15.163:0/2875050533 reap_dead start 2023-10-09T06:32:35.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.363+0000 7f0b75941700 1 -- 172.21.15.163:0/2875050533 shutdown_connections 2023-10-09T06:32:35.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.363+0000 7f0b75941700 1 -- 172.21.15.163:0/2875050533 >> 172.21.15.163:0/2875050533 conn(0x7f0b70068e20 msgr2=0x7f0b700d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:35.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.363+0000 7f0b75941700 1 -- 172.21.15.163:0/2875050533 shutdown_connections 2023-10-09T06:32:35.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.363+0000 7f0b75941700 1 -- 172.21.15.163:0/2875050533 wait complete. 2023-10-09T06:32:35.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.365+0000 7f0b75941700 1 Processor -- start 2023-10-09T06:32:35.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.365+0000 7f0b75941700 1 -- start start 2023-10-09T06:32:35.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.365+0000 7f0b75941700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b5c004d50 0x7f0b700b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:35.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.365+0000 7f0b75941700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b70143730 0x7f0b70157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:35.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.365+0000 7f0b75941700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b70157640 0x7f0b7016d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:35.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.365+0000 7f0b75941700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b700b5b60 con 0x7f0b5c004d50 2023-10-09T06:32:35.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.365+0000 7f0b75941700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b700b4f50 con 0x7f0b70143730 2023-10-09T06:32:35.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.365+0000 7f0b75941700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b700b5230 con 0x7f0b70157640 2023-10-09T06:32:35.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.365+0000 7f0b6effd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b5c004d50 0x7f0b700b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:35.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.365+0000 7f0b6effd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b5c004d50 0x7f0b700b5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58978/0 (socket says 172.21.15.163:58978) 2023-10-09T06:32:35.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.365+0000 7f0b6effd700 1 -- 172.21.15.163:0/3776164866 learned_addr learned my addr 172.21.15.163:0/3776164866 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:35.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.365+0000 7f0b67fff700 1 --2- 172.21.15.163:0/3776164866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b70143730 0x7f0b70157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:35.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.365+0000 7f0b6f7fe700 1 --2- 172.21.15.163:0/3776164866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b70157640 0x7f0b7016d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:35.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.366+0000 7f0b6effd700 1 -- 172.21.15.163:0/3776164866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b70157640 msgr2=0x7f0b7016d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:35.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.366+0000 7f0b6effd700 1 --2- 172.21.15.163:0/3776164866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b70157640 0x7f0b7016d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.366+0000 7f0b6effd700 1 -- 172.21.15.163:0/3776164866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b70143730 msgr2=0x7f0b70157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:35.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.366+0000 7f0b6effd700 1 --2- 172.21.15.163:0/3776164866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b70143730 0x7f0b70157100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.366+0000 7f0b6effd700 1 -- 172.21.15.163:0/3776164866 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b700b5510 con 0x7f0b5c004d50 2023-10-09T06:32:35.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.366+0000 7f0b6f7fe700 1 --2- 172.21.15.163:0/3776164866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b70157640 0x7f0b7016d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:35.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.366+0000 7f0b6effd700 1 --2- 172.21.15.163:0/3776164866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b5c004d50 0x7f0b700b5ff0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0b60006ac0 tx=0x7f0b600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:35.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.366+0000 7f0b66ffd700 1 -- 172.21.15.163:0/3776164866 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6000f940 con 0x7f0b5c004d50 2023-10-09T06:32:35.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.366+0000 7f0b66ffd700 1 -- 172.21.15.163:0/3776164866 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6000faa0 con 0x7f0b5c004d50 2023-10-09T06:32:35.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.366+0000 7f0b75941700 1 -- 172.21.15.163:0/3776164866 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b700b57f0 con 0x7f0b5c004d50 2023-10-09T06:32:35.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.366+0000 7f0b66ffd700 1 -- 172.21.15.163:0/3776164866 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6000fe20 con 0x7f0b5c004d50 2023-10-09T06:32:35.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.366+0000 7f0b75941700 1 -- 172.21.15.163:0/3776164866 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b70143ba0 con 0x7f0b5c004d50 2023-10-09T06:32:35.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.367+0000 7f0b75941700 1 -- 172.21.15.163:0/3776164866 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b7004c3c0 con 0x7f0b5c004d50 2023-10-09T06:32:35.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.367+0000 7f0b66ffd700 1 -- 172.21.15.163:0/3776164866 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f0b60014650 con 0x7f0b5c004d50 2023-10-09T06:32:35.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.367+0000 7f0b66ffd700 1 --2- 172.21.15.163:0/3776164866 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0b50066fc0 0x7f0b50069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:35.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.368+0000 7f0b66ffd700 1 -- 172.21.15.163:0/3776164866 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0b60018030 con 0x7f0b5c004d50 2023-10-09T06:32:35.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.368+0000 7f0b67fff700 1 --2- 172.21.15.163:0/3776164866 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0b50066fc0 0x7f0b50069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:35.374 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.368+0000 7f0b67fff700 1 --2- 172.21.15.163:0/3776164866 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0b50066fc0 0x7f0b50069470 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0b70158980 tx=0x7f0b580091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:35.374 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.373+0000 7f0b66ffd700 1 -- 172.21.15.163:0/3776164866 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0b60052210 con 0x7f0b5c004d50 2023-10-09T06:32:35.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.551+0000 7f0b75941700 1 -- 172.21.15.163:0/3776164866 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0b700b4d30 con 0x7f0b50066fc0 2023-10-09T06:32:35.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.560+0000 7f0b66ffd700 1 -- 172.21.15.163:0/3776164866 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278969 (secure 0 0 0) 0x7f0b58019070 con 0x7f0b50066fc0 2023-10-09T06:32:35.564 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:35.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.563+0000 7f0b75941700 1 -- 172.21.15.163:0/3776164866 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0b50066fc0 msgr2=0x7f0b50069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:35.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.563+0000 7f0b75941700 1 --2- 172.21.15.163:0/3776164866 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0b50066fc0 0x7f0b50069470 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0b70158980 tx=0x7f0b580091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.563+0000 7f0b75941700 1 -- 172.21.15.163:0/3776164866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b5c004d50 msgr2=0x7f0b700b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:35.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.563+0000 7f0b75941700 1 --2- 172.21.15.163:0/3776164866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b5c004d50 0x7f0b700b5ff0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0b60006ac0 tx=0x7f0b600091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.563+0000 7f0b75941700 1 -- 172.21.15.163:0/3776164866 shutdown_connections 2023-10-09T06:32:35.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.563+0000 7f0b75941700 1 --2- 172.21.15.163:0/3776164866 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0b50066fc0 0x7f0b50069470 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.563+0000 7f0b75941700 1 --2- 172.21.15.163:0/3776164866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b5c004d50 0x7f0b700b5ff0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.563+0000 7f0b75941700 1 --2- 172.21.15.163:0/3776164866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b70157640 0x7f0b7016d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.563+0000 7f0b75941700 1 --2- 172.21.15.163:0/3776164866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b70143730 0x7f0b70157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.563+0000 7f0b75941700 1 -- 172.21.15.163:0/3776164866 >> 172.21.15.163:0/3776164866 conn(0x7f0b70068e20 msgr2=0x7f0b7015d150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:35.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.563+0000 7f0b75941700 1 -- 172.21.15.163:0/3776164866 shutdown_connections 2023-10-09T06:32:35.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.563+0000 7f0b75941700 1 -- 172.21.15.163:0/3776164866 wait complete. 2023-10-09T06:32:35.567 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:32:35.576 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2023-10-09T06:32:34.325907+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2385160,"kb_used_data":6856,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752589560,"statfs":{"total":773094113280,"available":770651709440,"internally_reserved":0,"allocated":7020544,"data_stored":3644568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.832818"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496433+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:07:08.905652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.494244+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:46:10.202147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.493138+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:59:45.862276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496979+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:04:51.162203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.489029+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:53:36.664262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496973+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:33:04.258326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.497217+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:31:31.960774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496041+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:59:10.821319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.493220+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:39:36.054325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496928+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:35:15.549067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.489029+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:00:07.341785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496682+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:06:05.551649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496858+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:37:23.971604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496225+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:04:58.307911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.493686+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:09:26.237888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.494203+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:06:05.551649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.495351+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:59:11.383765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.492841+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:54:00.527419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.494412+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:24:04.774968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496480+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:10:01.011469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.495246+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:42:47.425853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.493146+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:38:53.216443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.494316+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:36:57.414702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.494868+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:34:30.240150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.492545+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:04:38.863681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496441+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:50:13.029959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.494316+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:04:38.863681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.495968+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:56:27.145572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496659+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:59:45.862276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.492783+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:24:24.863354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.494336+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:34:04.879844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.494948+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:55:18.693079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":197,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-09T06:32:27.324049+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:32:27.324049+0000","last_peered":"2023-10-09T06:32:27.324049+0000","last_clean":"2023-10-09T06:32:27.324049+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:32:27.324049+0000","last_undegraded":"2023-10-09T06:32:27.324049+0000","last_fullsized":"2023-10-09T06:32:27.324049+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298580,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073260,"statfs":{"total":96636764160,"available":96331018240,"internally_reserved":0,"allocated":1323008,"data_stored":898347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.60599999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.55900000000000005}]},{"osd":2,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52600000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.57199999999999995}]},{"osd":4,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.628}]},{"osd":5,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.375},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.50700000000000001}]},{"osd":6,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54600000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298000,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073840,"statfs":{"total":96636764160,"available":96331612160,"internally_reserved":0,"allocated":729088,"data_stored":307979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.52900000000000003}]},{"osd":1,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.56000000000000005}]},{"osd":2,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.502}]},{"osd":3,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":4.7510000000000003,"5min":4.7510000000000003,"15min":4.7510000000000003},"last":0.60699999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.59199999999999997}]},{"osd":5,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.54300000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.44},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.57099999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298000,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073840,"statfs":{"total":96636764160,"available":96331612160,"internally_reserved":0,"allocated":729088,"data_stored":307979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.31,"15min":0.31},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.315},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.371}]},{"osd":2,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.39500000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":5.8529999999999998,"5min":5.8529999999999998,"15min":5.8529999999999998},"last":0.40400000000000003}]},{"osd":4,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.51900000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":5.9459999999999997,"5min":5.9459999999999997,"15min":5.9459999999999997},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.54100000000000004}]},{"osd":6,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":5.9160000000000004,"5min":5.9160000000000004,"15min":5.9160000000000004},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.56699999999999995}]},{"osd":7,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":5.9729999999999999,"5min":5.9729999999999999,"15min":5.9729999999999999},"last":0.55600000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298580,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073260,"statfs":{"total":96636764160,"available":96331018240,"internally_reserved":0,"allocated":1323008,"data_stored":898347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.438,"5min":0.438,"15min":0.438},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.38900000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.45400000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.438}]},{"osd":4,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":5.484,"5min":5.484,"15min":5.484},"last":0.77000000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":5.5449999999999999,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.71499999999999997}]},{"osd":6,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.78700000000000003}]},{"osd":7,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":5.5800000000000001,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.68600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298000,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073840,"statfs":{"total":96636764160,"available":96331612160,"internally_reserved":0,"allocated":729088,"data_stored":307979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.224},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":7.2699999999999996,"5min":7.2699999999999996,"15min":7.2699999999999996},"last":0.27400000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":7.327,"5min":7.327,"15min":7.327},"last":0.318}]},{"osd":3,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.26000000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":7.2389999999999999,"5min":7.2389999999999999,"15min":7.2389999999999999},"last":0.50900000000000001}]},{"osd":5,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":10.890000000000001,"5min":10.890000000000001,"15min":10.890000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52000000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":7.2999999999999998,"5min":7.2999999999999998,"15min":7.2999999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.47199999999999998}]},{"osd":7,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":7.1879999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.60499999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298000,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073840,"statfs":{"total":96636764160,"available":96331612160,"internally_reserved":0,"allocated":729088,"data_stored":307979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.307},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.26700000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.40500000000000003}]},{"osd":2,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.28399999999999997}]},{"osd":4,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.44500000000000001}]},{"osd":5,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.45800000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.499}]},{"osd":7,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.42299999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298000,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073840,"statfs":{"total":96636764160,"available":96331612160,"internally_reserved":0,"allocated":729088,"data_stored":307979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.50700000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.48099999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.57899999999999996}]},{"osd":3,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.52500000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.55600000000000005}]},{"osd":6,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.65400000000000003}]},{"osd":7,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.626},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.67400000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298000,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073840,"statfs":{"total":96636764160,"available":96331612160,"internally_reserved":0,"allocated":729088,"data_stored":307979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.56399999999999995}]},{"osd":1,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.58299999999999996}]},{"osd":2,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59599999999999997}]},{"osd":3,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.53800000000000003}]},{"osd":4,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.625},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.64000000000000001}]},{"osd":6,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.66600000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.69399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:32:35.584 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:32:35.730 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.730+0000 7f0ebb373700 1 Processor -- start 2023-10-09T06:32:35.731 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.730+0000 7f0ebb373700 1 -- start start 2023-10-09T06:32:35.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.732+0000 7f0ebb373700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0eb4159890 0x7f0eb415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:35.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.732+0000 7f0ebb373700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb415a160 con 0x7f0eb4069560 2023-10-09T06:32:35.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.732+0000 7f0ebb373700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb40d13a0 con 0x7f0eb4159c60 2023-10-09T06:32:35.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.732+0000 7f0ebb373700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb415e140 con 0x7f0eb4159890 2023-10-09T06:32:35.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.732+0000 7f0eb910f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0eb4159890 0x7f0eb415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:35.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.732+0000 7f0eb910f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0eb4159890 0x7f0eb415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58992/0 (socket says 172.21.15.163:58992) 2023-10-09T06:32:35.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.732+0000 7f0eb9910700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f0eb4069560 0x7f0eb40ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:44820/0 (socket says 172.21.15.163:44820) 2023-10-09T06:32:35.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.732+0000 7f0eb910f700 1 -- 172.21.15.163:0/4143177437 learned_addr learned my addr 172.21.15.163:0/4143177437 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:35.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.732+0000 7f0eb910f700 1 -- 172.21.15.163:0/4143177437 >> v1:172.21.15.181:6789/0 conn(0x7f0eb4159c60 legacy=0x7f0eb415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:35.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.732+0000 7f0eb910f700 1 -- 172.21.15.163:0/4143177437 >> v1:172.21.15.163:6789/0 conn(0x7f0eb4069560 legacy=0x7f0eb40ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:35.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.732+0000 7f0eb910f700 1 -- 172.21.15.163:0/4143177437 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb415a3d0 con 0x7f0eb4159890 2023-10-09T06:32:35.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.732+0000 7f0eb3fff700 1 -- 172.21.15.163:0/4143177437 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 720832627 0 0) 0x7f0eb415a160 con 0x7f0eb4069560 2023-10-09T06:32:35.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.732+0000 7f0eb910f700 1 --2- 172.21.15.163:0/4143177437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0eb4159890 0x7f0eb415abd0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0eac0099b0 tx=0x7f0eac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3888b22ecab24be0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:35.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.732+0000 7f0eb3fff700 1 -- 172.21.15.163:0/4143177437 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac00b820 con 0x7f0eb4159890 2023-10-09T06:32:35.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.732+0000 7f0eb3fff700 1 -- 172.21.15.163:0/4143177437 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eac00b980 con 0x7f0eb4159890 2023-10-09T06:32:35.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.732+0000 7f0eb3fff700 1 -- 172.21.15.163:0/4143177437 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac00bd20 con 0x7f0eb4159890 2023-10-09T06:32:35.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.732+0000 7f0ebb373700 1 -- 172.21.15.163:0/4143177437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0eb4159890 msgr2=0x7f0eb415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:35.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.732+0000 7f0ebb373700 1 --2- 172.21.15.163:0/4143177437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0eb4159890 0x7f0eb415abd0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0eac0099b0 tx=0x7f0eac009250 comp rx=0 tx=0).stop 2023-10-09T06:32:35.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.732+0000 7f0ebb373700 1 -- 172.21.15.163:0/4143177437 shutdown_connections 2023-10-09T06:32:35.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.732+0000 7f0ebb373700 1 --2- 172.21.15.163:0/4143177437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0eb4159890 0x7f0eb415abd0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.732+0000 7f0ebb373700 1 -- 172.21.15.163:0/4143177437 >> 172.21.15.163:0/4143177437 conn(0x7f0eb4068e20 msgr2=0x7f0eb40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:35.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.732+0000 7f0ebb373700 1 -- 172.21.15.163:0/4143177437 shutdown_connections 2023-10-09T06:32:35.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.732+0000 7f0ebb373700 1 -- 172.21.15.163:0/4143177437 wait complete. 2023-10-09T06:32:35.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.733+0000 7f0ebb373700 1 Processor -- start 2023-10-09T06:32:35.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.733+0000 7f0ebb373700 1 -- start start 2023-10-09T06:32:35.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.734+0000 7f0ebb373700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0eb4069560 0x7f0eb416f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:35.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.734+0000 7f0ebb373700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0eb4159890 0x7f0eb4171780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:35.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.734+0000 7f0ebb373700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0eb4159c60 0x7f0eb4171cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:35.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.734+0000 7f0ebb373700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb415ba50 con 0x7f0eb4069560 2023-10-09T06:32:35.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.734+0000 7f0ebb373700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb415c5d0 con 0x7f0eb4159c60 2023-10-09T06:32:35.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.734+0000 7f0ebb373700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb416f890 con 0x7f0eb4159890 2023-10-09T06:32:35.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.734+0000 7f0eb910f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0eb4069560 0x7f0eb416f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:35.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.734+0000 7f0eb910f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0eb4069560 0x7f0eb416f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59008/0 (socket says 172.21.15.163:59008) 2023-10-09T06:32:35.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.734+0000 7f0eb910f700 1 -- 172.21.15.163:0/2519332671 learned_addr learned my addr 172.21.15.163:0/2519332671 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:35.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.734+0000 7f0eb9910700 1 --2- 172.21.15.163:0/2519332671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0eb4159c60 0x7f0eb4171cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:35.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.734+0000 7f0eb890e700 1 --2- 172.21.15.163:0/2519332671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0eb4159890 0x7f0eb4171780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:35.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.734+0000 7f0eb910f700 1 -- 172.21.15.163:0/2519332671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0eb4159890 msgr2=0x7f0eb4171780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:35.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.734+0000 7f0eb910f700 1 --2- 172.21.15.163:0/2519332671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0eb4159890 0x7f0eb4171780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.734+0000 7f0eb910f700 1 -- 172.21.15.163:0/2519332671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0eb4159c60 msgr2=0x7f0eb4171cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:35.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.734+0000 7f0eb910f700 1 --2- 172.21.15.163:0/2519332671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0eb4159c60 0x7f0eb4171cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.734+0000 7f0eb910f700 1 -- 172.21.15.163:0/2519332671 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eac0094f0 con 0x7f0eb4069560 2023-10-09T06:32:35.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.735+0000 7f0eb910f700 1 --2- 172.21.15.163:0/2519332671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0eb4069560 0x7f0eb416f350 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4070e20 tx=0x7f0eac016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:35.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.735+0000 7f0eb17fa700 1 -- 172.21.15.163:0/2519332671 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac00bc40 con 0x7f0eb4069560 2023-10-09T06:32:35.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.735+0000 7f0eb17fa700 1 -- 172.21.15.163:0/2519332671 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eac020070 con 0x7f0eb4069560 2023-10-09T06:32:35.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.735+0000 7f0eb17fa700 1 -- 172.21.15.163:0/2519332671 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac01c6a0 con 0x7f0eb4069560 2023-10-09T06:32:35.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.735+0000 7f0ebb373700 1 -- 172.21.15.163:0/2519332671 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb415bca0 con 0x7f0eb4069560 2023-10-09T06:32:35.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.735+0000 7f0ebb373700 1 -- 172.21.15.163:0/2519332671 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb415c260 con 0x7f0eb4069560 2023-10-09T06:32:35.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.736+0000 7f0ebb373700 1 -- 172.21.15.163:0/2519332671 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb404c3c0 con 0x7f0eb4069560 2023-10-09T06:32:35.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.736+0000 7f0eb17fa700 1 -- 172.21.15.163:0/2519332671 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f0eac022030 con 0x7f0eb4069560 2023-10-09T06:32:35.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.736+0000 7f0eb17fa700 1 --2- 172.21.15.163:0/2519332671 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0ea0066fc0 0x7f0ea0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:35.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.736+0000 7f0eb17fa700 1 -- 172.21.15.163:0/2519332671 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0eac086500 con 0x7f0eb4069560 2023-10-09T06:32:35.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.737+0000 7f0eb890e700 1 --2- 172.21.15.163:0/2519332671 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0ea0066fc0 0x7f0ea0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:35.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.737+0000 7f0eb890e700 1 --2- 172.21.15.163:0/2519332671 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0ea0066fc0 0x7f0ea0069470 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0eb406a010 tx=0x7f0ea8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:35.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.741+0000 7f0eb17fa700 1 -- 172.21.15.163:0/2519332671 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0eac053780 con 0x7f0eb4069560 2023-10-09T06:32:35.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.920+0000 7f0ebb373700 1 -- 172.21.15.163:0/2519332671 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0eb415c710 con 0x7f0ea0066fc0 2023-10-09T06:32:35.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.928+0000 7f0eb17fa700 1 -- 172.21.15.163:0/2519332671 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278969 (secure 0 0 0) 0x7f0ea8006820 con 0x7f0ea0066fc0 2023-10-09T06:32:35.932 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:35.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.931+0000 7f0ebb373700 1 -- 172.21.15.163:0/2519332671 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0ea0066fc0 msgr2=0x7f0ea0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:35.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.931+0000 7f0ebb373700 1 --2- 172.21.15.163:0/2519332671 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0ea0066fc0 0x7f0ea0069470 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0eb406a010 tx=0x7f0ea8009040 comp rx=0 tx=0).stop 2023-10-09T06:32:35.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.931+0000 7f0ebb373700 1 -- 172.21.15.163:0/2519332671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0eb4069560 msgr2=0x7f0eb416f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:35.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.931+0000 7f0ebb373700 1 --2- 172.21.15.163:0/2519332671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0eb4069560 0x7f0eb416f350 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4070e20 tx=0x7f0eac016000 comp rx=0 tx=0).stop 2023-10-09T06:32:35.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.931+0000 7f0ebb373700 1 -- 172.21.15.163:0/2519332671 shutdown_connections 2023-10-09T06:32:35.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.931+0000 7f0ebb373700 1 --2- 172.21.15.163:0/2519332671 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0ea0066fc0 0x7f0ea0069470 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.931+0000 7f0ebb373700 1 --2- 172.21.15.163:0/2519332671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0eb4069560 0x7f0eb416f350 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.931+0000 7f0ebb373700 1 --2- 172.21.15.163:0/2519332671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0eb4159c60 0x7f0eb4171cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.931+0000 7f0ebb373700 1 --2- 172.21.15.163:0/2519332671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0eb4159890 0x7f0eb4171780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:35.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.931+0000 7f0ebb373700 1 -- 172.21.15.163:0/2519332671 >> 172.21.15.163:0/2519332671 conn(0x7f0eb4068e20 msgr2=0x7f0eb40b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:35.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.931+0000 7f0ebb373700 1 -- 172.21.15.163:0/2519332671 shutdown_connections 2023-10-09T06:32:35.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:35.931+0000 7f0ebb373700 1 -- 172.21.15.163:0/2519332671 wait complete. 2023-10-09T06:32:35.935 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:32:35.943 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2023-10-09T06:32:34.325907+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2385160,"kb_used_data":6856,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752589560,"statfs":{"total":773094113280,"available":770651709440,"internally_reserved":0,"allocated":7020544,"data_stored":3644568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.832818"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496433+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:07:08.905652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.494244+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:46:10.202147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.493138+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:59:45.862276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496979+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:04:51.162203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.489029+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:53:36.664262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496973+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:33:04.258326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.497217+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:31:31.960774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496041+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:59:10.821319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.493220+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:39:36.054325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496928+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:35:15.549067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.489029+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:00:07.341785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496682+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:06:05.551649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496858+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:37:23.971604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496225+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:04:58.307911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.493686+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:09:26.237888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.494203+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:06:05.551649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.495351+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:59:11.383765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.492841+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:54:00.527419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.494412+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:24:04.774968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496480+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:10:01.011469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.495246+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:42:47.425853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.493146+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:38:53.216443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.494316+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:36:57.414702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.494868+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:34:30.240150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.492545+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:04:38.863681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496441+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:50:13.029959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.494316+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:04:38.863681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.495968+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:56:27.145572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496659+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:59:45.862276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.492783+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:24:24.863354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.494336+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:34:04.879844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.494948+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:55:18.693079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":197,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-09T06:32:27.324049+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:32:27.324049+0000","last_peered":"2023-10-09T06:32:27.324049+0000","last_clean":"2023-10-09T06:32:27.324049+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:32:27.324049+0000","last_undegraded":"2023-10-09T06:32:27.324049+0000","last_fullsized":"2023-10-09T06:32:27.324049+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298580,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073260,"statfs":{"total":96636764160,"available":96331018240,"internally_reserved":0,"allocated":1323008,"data_stored":898347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.60599999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.55900000000000005}]},{"osd":2,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52600000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.57199999999999995}]},{"osd":4,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.628}]},{"osd":5,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.375},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.50700000000000001}]},{"osd":6,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54600000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298000,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073840,"statfs":{"total":96636764160,"available":96331612160,"internally_reserved":0,"allocated":729088,"data_stored":307979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.52900000000000003}]},{"osd":1,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.56000000000000005}]},{"osd":2,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.502}]},{"osd":3,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":4.7510000000000003,"5min":4.7510000000000003,"15min":4.7510000000000003},"last":0.60699999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.59199999999999997}]},{"osd":5,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.54300000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.44},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.57099999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298000,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073840,"statfs":{"total":96636764160,"available":96331612160,"internally_reserved":0,"allocated":729088,"data_stored":307979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.31,"15min":0.31},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.315},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.371}]},{"osd":2,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.39500000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":5.8529999999999998,"5min":5.8529999999999998,"15min":5.8529999999999998},"last":0.40400000000000003}]},{"osd":4,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.51900000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":5.9459999999999997,"5min":5.9459999999999997,"15min":5.9459999999999997},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.54100000000000004}]},{"osd":6,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":5.9160000000000004,"5min":5.9160000000000004,"15min":5.9160000000000004},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.56699999999999995}]},{"osd":7,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":5.9729999999999999,"5min":5.9729999999999999,"15min":5.9729999999999999},"last":0.55600000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298580,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073260,"statfs":{"total":96636764160,"available":96331018240,"internally_reserved":0,"allocated":1323008,"data_stored":898347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.438,"5min":0.438,"15min":0.438},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.38900000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.45400000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.438}]},{"osd":4,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":5.484,"5min":5.484,"15min":5.484},"last":0.77000000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":5.5449999999999999,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.71499999999999997}]},{"osd":6,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.78700000000000003}]},{"osd":7,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":5.5800000000000001,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.68600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298000,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073840,"statfs":{"total":96636764160,"available":96331612160,"internally_reserved":0,"allocated":729088,"data_stored":307979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.224},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":7.2699999999999996,"5min":7.2699999999999996,"15min":7.2699999999999996},"last":0.27400000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":7.327,"5min":7.327,"15min":7.327},"last":0.318}]},{"osd":3,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.26000000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":7.2389999999999999,"5min":7.2389999999999999,"15min":7.2389999999999999},"last":0.50900000000000001}]},{"osd":5,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":10.890000000000001,"5min":10.890000000000001,"15min":10.890000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52000000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":7.2999999999999998,"5min":7.2999999999999998,"15min":7.2999999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.47199999999999998}]},{"osd":7,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":7.1879999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.60499999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298000,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073840,"statfs":{"total":96636764160,"available":96331612160,"internally_reserved":0,"allocated":729088,"data_stored":307979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.307},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.26700000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.40500000000000003}]},{"osd":2,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.28399999999999997}]},{"osd":4,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.44500000000000001}]},{"osd":5,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.45800000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.499}]},{"osd":7,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.42299999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298000,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073840,"statfs":{"total":96636764160,"available":96331612160,"internally_reserved":0,"allocated":729088,"data_stored":307979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.50700000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.48099999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.57899999999999996}]},{"osd":3,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.52500000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.55600000000000005}]},{"osd":6,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.65400000000000003}]},{"osd":7,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.626},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.67400000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298000,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073840,"statfs":{"total":96636764160,"available":96331612160,"internally_reserved":0,"allocated":729088,"data_stored":307979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.56399999999999995}]},{"osd":1,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.58299999999999996}]},{"osd":2,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59599999999999997}]},{"osd":3,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.53800000000000003}]},{"osd":4,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.625},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.64000000000000001}]},{"osd":6,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.66600000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.69399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:32:35.993 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:32:36.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.108+0000 7fb76961e700 1 Processor -- start 2023-10-09T06:32:36.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.108+0000 7fb76961e700 1 -- start start 2023-10-09T06:32:36.111 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.109+0000 7fb76961e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb764159c60 0x7fb76415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:36.111 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.109+0000 7fb76961e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb76415a160 con 0x7fb764159890 2023-10-09T06:32:36.112 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.109+0000 7fb76961e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7640d13a0 con 0x7fb76415db10 2023-10-09T06:32:36.112 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.109+0000 7fb76961e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb76415df50 con 0x7fb764159c60 2023-10-09T06:32:36.112 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.109+0000 7fb7627fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb764159c60 0x7fb76415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:36.112 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.109+0000 7fb762ffd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb764159890 0x7fb76415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:44830/0 (socket says 172.21.15.163:44830) 2023-10-09T06:32:36.112 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.109+0000 7fb762ffd700 1 -- 172.21.15.163:0/1155762144 learned_addr learned my addr 172.21.15.163:0/1155762144 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:36.112 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.110+0000 7fb7627fc700 1 -- 172.21.15.163:0/1155762144 >> v1:172.21.15.181:6789/0 conn(0x7fb76415db10 legacy=0x7fb764069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:36.112 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.110+0000 7fb7627fc700 1 -- 172.21.15.163:0/1155762144 >> v1:172.21.15.163:6789/0 conn(0x7fb764159890 legacy=0x7fb76415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:36.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.110+0000 7fb7627fc700 1 -- 172.21.15.163:0/1155762144 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb76415e140 con 0x7fb764159c60 2023-10-09T06:32:36.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.110+0000 7fb7627fc700 1 --2- 172.21.15.163:0/1155762144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb764159c60 0x7fb76415d5d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb7580098e0 tx=0x7fb7580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f6195ea8220c5d server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:36.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.110+0000 7fb761ffb700 1 -- 172.21.15.163:0/1155762144 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb758009e80 con 0x7fb764159c60 2023-10-09T06:32:36.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.110+0000 7fb761ffb700 1 -- 172.21.15.163:0/1155762144 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb75800b760 con 0x7fb764159c60 2023-10-09T06:32:36.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.110+0000 7fb761ffb700 1 -- 172.21.15.163:0/1155762144 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75800bb00 con 0x7fb764159c60 2023-10-09T06:32:36.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.110+0000 7fb76961e700 1 -- 172.21.15.163:0/1155762144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb764159c60 msgr2=0x7fb76415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:36.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.110+0000 7fb76961e700 1 --2- 172.21.15.163:0/1155762144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb764159c60 0x7fb76415d5d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb7580098e0 tx=0x7fb7580091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:36.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.110+0000 7fb76961e700 1 -- 172.21.15.163:0/1155762144 shutdown_connections 2023-10-09T06:32:36.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.110+0000 7fb76961e700 1 --2- 172.21.15.163:0/1155762144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb764159c60 0x7fb76415d5d0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:36.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.110+0000 7fb76961e700 1 -- 172.21.15.163:0/1155762144 >> 172.21.15.163:0/1155762144 conn(0x7fb764068e20 msgr2=0x7fb7640d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:36.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.110+0000 7fb76961e700 1 -- 172.21.15.163:0/1155762144 shutdown_connections 2023-10-09T06:32:36.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.110+0000 7fb76961e700 1 -- 172.21.15.163:0/1155762144 wait complete. 2023-10-09T06:32:36.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.112+0000 7fb76961e700 1 Processor -- start 2023-10-09T06:32:36.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.112+0000 7fb76961e700 1 -- start start 2023-10-09T06:32:36.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.112+0000 7fb76961e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb764159890 0x7fb764154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:36.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.112+0000 7fb76961e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb764159c60 0x7fb7640ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:36.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.112+0000 7fb76961e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb76415db10 0x7fb7640b5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:36.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.112+0000 7fb76961e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7640c9960 con 0x7fb764159890 2023-10-09T06:32:36.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.112+0000 7fb76961e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7640c9c40 con 0x7fb764159c60 2023-10-09T06:32:36.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.112+0000 7fb76961e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb76415b570 con 0x7fb76415db10 2023-10-09T06:32:36.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.112+0000 7fb762ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb764159890 0x7fb764154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:36.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.113+0000 7fb762ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb764159890 0x7fb764154220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59032/0 (socket says 172.21.15.163:59032) 2023-10-09T06:32:36.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.113+0000 7fb762ffd700 1 -- 172.21.15.163:0/4050763726 learned_addr learned my addr 172.21.15.163:0/4050763726 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:36.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.113+0000 7fb7627fc700 1 --2- 172.21.15.163:0/4050763726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb764159c60 0x7fb7640ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:36.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.113+0000 7fb7637fe700 1 --2- 172.21.15.163:0/4050763726 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb76415db10 0x7fb7640b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:36.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.113+0000 7fb762ffd700 1 -- 172.21.15.163:0/4050763726 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb76415db10 msgr2=0x7fb7640b5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:36.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.113+0000 7fb762ffd700 1 --2- 172.21.15.163:0/4050763726 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb76415db10 0x7fb7640b5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:36.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.113+0000 7fb762ffd700 1 -- 172.21.15.163:0/4050763726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb764159c60 msgr2=0x7fb7640ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:36.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.113+0000 7fb762ffd700 1 --2- 172.21.15.163:0/4050763726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb764159c60 0x7fb7640ca600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:36.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.113+0000 7fb762ffd700 1 -- 172.21.15.163:0/4050763726 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb758009420 con 0x7fb764159890 2023-10-09T06:32:36.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.113+0000 7fb762ffd700 1 --2- 172.21.15.163:0/4050763726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb764159890 0x7fb764154220 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb75400c810 tx=0x7fb754007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:36.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.113+0000 7fb74affd700 1 -- 172.21.15.163:0/4050763726 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb754005b30 con 0x7fb764159890 2023-10-09T06:32:36.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.113+0000 7fb74affd700 1 -- 172.21.15.163:0/4050763726 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb754005c90 con 0x7fb764159890 2023-10-09T06:32:36.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.113+0000 7fb74affd700 1 -- 172.21.15.163:0/4050763726 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb754007340 con 0x7fb764159890 2023-10-09T06:32:36.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.113+0000 7fb76961e700 1 -- 172.21.15.163:0/4050763726 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb76415b850 con 0x7fb764159890 2023-10-09T06:32:36.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.113+0000 7fb76961e700 1 -- 172.21.15.163:0/4050763726 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7640cab40 con 0x7fb764159890 2023-10-09T06:32:36.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.114+0000 7fb76961e700 1 -- 172.21.15.163:0/4050763726 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb76415bd70 con 0x7fb764159890 2023-10-09T06:32:36.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.115+0000 7fb74affd700 1 -- 172.21.15.163:0/4050763726 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fb75401b400 con 0x7fb764159890 2023-10-09T06:32:36.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.115+0000 7fb74affd700 1 --2- 172.21.15.163:0/4050763726 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb750066fc0 0x7fb750069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:36.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.115+0000 7fb74affd700 1 -- 172.21.15.163:0/4050763726 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb754084170 con 0x7fb764159890 2023-10-09T06:32:36.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.115+0000 7fb7627fc700 1 --2- 172.21.15.163:0/4050763726 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb750066fc0 0x7fb750069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:36.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.116+0000 7fb7627fc700 1 --2- 172.21.15.163:0/4050763726 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb750066fc0 0x7fb750069470 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb7580091e0 tx=0x7fb758016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:36.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.120+0000 7fb74affd700 1 -- 172.21.15.163:0/4050763726 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb76415bd70 con 0x7fb764159890 2023-10-09T06:32:36.298 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.298+0000 7fb76961e700 1 -- 172.21.15.163:0/4050763726 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb76415bd70 con 0x7fb750066fc0 2023-10-09T06:32:36.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.307+0000 7fb74affd700 1 -- 172.21.15.163:0/4050763726 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278969 (secure 0 0 0) 0x7fb76415bd70 con 0x7fb750066fc0 2023-10-09T06:32:36.310 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:36.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.310+0000 7fb76961e700 1 -- 172.21.15.163:0/4050763726 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb750066fc0 msgr2=0x7fb750069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:36.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.310+0000 7fb76961e700 1 --2- 172.21.15.163:0/4050763726 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb750066fc0 0x7fb750069470 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb7580091e0 tx=0x7fb758016000 comp rx=0 tx=0).stop 2023-10-09T06:32:36.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.310+0000 7fb76961e700 1 -- 172.21.15.163:0/4050763726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb764159890 msgr2=0x7fb764154220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:36.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.310+0000 7fb76961e700 1 --2- 172.21.15.163:0/4050763726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb764159890 0x7fb764154220 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb75400c810 tx=0x7fb754007f30 comp rx=0 tx=0).stop 2023-10-09T06:32:36.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.310+0000 7fb76961e700 1 -- 172.21.15.163:0/4050763726 shutdown_connections 2023-10-09T06:32:36.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.310+0000 7fb76961e700 1 --2- 172.21.15.163:0/4050763726 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb750066fc0 0x7fb750069470 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:36.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.310+0000 7fb76961e700 1 --2- 172.21.15.163:0/4050763726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb764159890 0x7fb764154220 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:36.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.310+0000 7fb76961e700 1 --2- 172.21.15.163:0/4050763726 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb76415db10 0x7fb7640b5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:36.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.310+0000 7fb76961e700 1 --2- 172.21.15.163:0/4050763726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb764159c60 0x7fb7640ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:36.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.310+0000 7fb76961e700 1 -- 172.21.15.163:0/4050763726 >> 172.21.15.163:0/4050763726 conn(0x7fb764068e20 msgr2=0x7fb7640cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:36.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.310+0000 7fb76961e700 1 -- 172.21.15.163:0/4050763726 shutdown_connections 2023-10-09T06:32:36.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.310+0000 7fb76961e700 1 -- 172.21.15.163:0/4050763726 wait complete. 2023-10-09T06:32:36.314 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:32:36.322 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2023-10-09T06:32:34.325907+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2385160,"kb_used_data":6856,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752589560,"statfs":{"total":773094113280,"available":770651709440,"internally_reserved":0,"allocated":7020544,"data_stored":3644568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.832818"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496433+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:07:08.905652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.494244+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:46:10.202147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.493138+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:59:45.862276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496979+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:04:51.162203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.489029+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:53:36.664262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496973+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:33:04.258326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.497217+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:31:31.960774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496041+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:59:10.821319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.493220+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:39:36.054325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496928+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:35:15.549067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.489029+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:00:07.341785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496682+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:06:05.551649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496858+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:37:23.971604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496225+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:04:58.307911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.493686+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:09:26.237888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.494203+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:06:05.551649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.495351+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:59:11.383765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.492841+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:54:00.527419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.494412+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:24:04.774968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496480+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:10:01.011469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.495246+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:42:47.425853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.493146+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:38:53.216443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.494316+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:36:57.414702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.494868+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:34:30.240150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.492545+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:04:38.863681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496441+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:50:13.029959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.494316+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:04:38.863681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.495968+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:56:27.145572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.496659+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:59:45.862276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.492783+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:24:24.863354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.494336+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:34:04.879844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:32:30.487000+0000","last_change":"2023-10-09T06:32:30.487000+0000","last_active":"2023-10-09T06:32:30.487000+0000","last_peered":"2023-10-09T06:32:30.487000+0000","last_clean":"2023-10-09T06:32:30.487000+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:30.487000+0000","last_undegraded":"2023-10-09T06:32:30.487000+0000","last_fullsized":"2023-10-09T06:32:30.487000+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:32:29.484745+0000","last_change":"2023-10-09T06:32:29.494948+0000","last_active":"2023-10-09T06:32:29.484745+0000","last_peered":"2023-10-09T06:32:29.484745+0000","last_clean":"2023-10-09T06:32:29.484745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:32:29.484745+0000","last_undegraded":"2023-10-09T06:32:29.484745+0000","last_fullsized":"2023-10-09T06:32:29.484745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:55:18.693079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":197,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-10-09T06:32:27.324049+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:32:27.324049+0000","last_peered":"2023-10-09T06:32:27.324049+0000","last_clean":"2023-10-09T06:32:27.324049+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:32:27.324049+0000","last_undegraded":"2023-10-09T06:32:27.324049+0000","last_fullsized":"2023-10-09T06:32:27.324049+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298580,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073260,"statfs":{"total":96636764160,"available":96331018240,"internally_reserved":0,"allocated":1323008,"data_stored":898347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.60599999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.55900000000000005}]},{"osd":2,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52600000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.57199999999999995}]},{"osd":4,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.628}]},{"osd":5,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.375},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.50700000000000001}]},{"osd":6,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54600000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298000,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073840,"statfs":{"total":96636764160,"available":96331612160,"internally_reserved":0,"allocated":729088,"data_stored":307979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.52900000000000003}]},{"osd":1,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.56000000000000005}]},{"osd":2,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.502}]},{"osd":3,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":4.7510000000000003,"5min":4.7510000000000003,"15min":4.7510000000000003},"last":0.60699999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.59199999999999997}]},{"osd":5,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.54300000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.44},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.57099999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298000,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073840,"statfs":{"total":96636764160,"available":96331612160,"internally_reserved":0,"allocated":729088,"data_stored":307979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.31,"15min":0.31},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.315},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.371}]},{"osd":2,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.39500000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":5.8529999999999998,"5min":5.8529999999999998,"15min":5.8529999999999998},"last":0.40400000000000003}]},{"osd":4,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.51900000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":5.9459999999999997,"5min":5.9459999999999997,"15min":5.9459999999999997},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.54100000000000004}]},{"osd":6,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":5.9160000000000004,"5min":5.9160000000000004,"15min":5.9160000000000004},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.56699999999999995}]},{"osd":7,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":5.9729999999999999,"5min":5.9729999999999999,"15min":5.9729999999999999},"last":0.55600000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298580,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073260,"statfs":{"total":96636764160,"available":96331018240,"internally_reserved":0,"allocated":1323008,"data_stored":898347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.438,"5min":0.438,"15min":0.438},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.38900000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.45400000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.438}]},{"osd":4,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":5.484,"5min":5.484,"15min":5.484},"last":0.77000000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":5.5449999999999999,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.71499999999999997}]},{"osd":6,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.78700000000000003}]},{"osd":7,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":5.5800000000000001,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.68600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298000,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073840,"statfs":{"total":96636764160,"available":96331612160,"internally_reserved":0,"allocated":729088,"data_stored":307979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.224},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":7.2699999999999996,"5min":7.2699999999999996,"15min":7.2699999999999996},"last":0.27400000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":7.327,"5min":7.327,"15min":7.327},"last":0.318}]},{"osd":3,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.26000000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":7.2389999999999999,"5min":7.2389999999999999,"15min":7.2389999999999999},"last":0.50900000000000001}]},{"osd":5,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":10.890000000000001,"5min":10.890000000000001,"15min":10.890000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52000000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":7.2999999999999998,"5min":7.2999999999999998,"15min":7.2999999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.47199999999999998}]},{"osd":7,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":7.1879999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.60499999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298000,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073840,"statfs":{"total":96636764160,"available":96331612160,"internally_reserved":0,"allocated":729088,"data_stored":307979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.307},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.26700000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.40500000000000003}]},{"osd":2,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.28399999999999997}]},{"osd":4,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.44500000000000001}]},{"osd":5,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.45800000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.499}]},{"osd":7,"last update":"Mon Oct 9 06:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.42299999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298000,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073840,"statfs":{"total":96636764160,"available":96331612160,"internally_reserved":0,"allocated":729088,"data_stored":307979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.50700000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.48099999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.57899999999999996}]},{"osd":3,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.52500000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.55600000000000005}]},{"osd":6,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.65400000000000003}]},{"osd":7,"last update":"Mon Oct 9 06:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.626},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.67400000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298000,"kb_used_data":712,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073840,"statfs":{"total":96636764160,"available":96331612160,"internally_reserved":0,"allocated":729088,"data_stored":307979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.56399999999999995}]},{"osd":1,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.58299999999999996}]},{"osd":2,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59599999999999997}]},{"osd":3,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.53800000000000003}]},{"osd":4,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.625},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.64000000000000001}]},{"osd":6,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.66600000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.69399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:32:36.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3f is not active+clean 2023-10-09T06:32:36.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3e is not active+clean 2023-10-09T06:32:36.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3d is not active+clean 2023-10-09T06:32:36.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3c is not active+clean 2023-10-09T06:32:36.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3b is not active+clean 2023-10-09T06:32:36.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3a is not active+clean 2023-10-09T06:32:36.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.39 is not active+clean 2023-10-09T06:32:36.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.38 is not active+clean 2023-10-09T06:32:36.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.37 is not active+clean 2023-10-09T06:32:36.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.36 is not active+clean 2023-10-09T06:32:36.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.35 is not active+clean 2023-10-09T06:32:36.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.34 is not active+clean 2023-10-09T06:32:36.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.332 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.33 is not active+clean 2023-10-09T06:32:36.332 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.332 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.32 is not active+clean 2023-10-09T06:32:36.332 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.332 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.31 is not active+clean 2023-10-09T06:32:36.332 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.332 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.30 is not active+clean 2023-10-09T06:32:36.332 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.332 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2f is not active+clean 2023-10-09T06:32:36.332 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.333 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2e is not active+clean 2023-10-09T06:32:36.333 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.333 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2d is not active+clean 2023-10-09T06:32:36.333 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.333 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2c is not active+clean 2023-10-09T06:32:36.333 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.333 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2b is not active+clean 2023-10-09T06:32:36.333 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.333 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2a is not active+clean 2023-10-09T06:32:36.334 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.334 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.29 is not active+clean 2023-10-09T06:32:36.334 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.334 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.28 is not active+clean 2023-10-09T06:32:36.334 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.334 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.27 is not active+clean 2023-10-09T06:32:36.334 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.335 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.26 is not active+clean 2023-10-09T06:32:36.335 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.335 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.25 is not active+clean 2023-10-09T06:32:36.335 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.335 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.24 is not active+clean 2023-10-09T06:32:36.335 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.336 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.23 is not active+clean 2023-10-09T06:32:36.336 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.336 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.22 is not active+clean 2023-10-09T06:32:36.336 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.337 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.21 is not active+clean 2023-10-09T06:32:36.337 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.337 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.20 is not active+clean 2023-10-09T06:32:36.337 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.337 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2023-10-09T06:32:36.337 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.337 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1f is not active+clean 2023-10-09T06:32:36.338 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.496433+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T14:07:08.905652+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-09T06:32:36.338 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1f is not active+clean 2023-10-09T06:32:36.338 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.338 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1e is not active+clean 2023-10-09T06:32:36.338 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.494244+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T14:46:10.202147+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-09T06:32:36.339 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1c is not active+clean 2023-10-09T06:32:36.339 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.339 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1d is not active+clean 2023-10-09T06:32:36.339 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.493138+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T09:59:45.862276+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-09T06:32:36.340 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1d is not active+clean 2023-10-09T06:32:36.340 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.340 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1c is not active+clean 2023-10-09T06:32:36.340 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.496979+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T14:04:51.162203+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-09T06:32:36.341 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1a is not active+clean 2023-10-09T06:32:36.341 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.341 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1b is not active+clean 2023-10-09T06:32:36.341 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.489029+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T07:53:36.664262+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-09T06:32:36.342 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1b is not active+clean 2023-10-09T06:32:36.342 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.342 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1a is not active+clean 2023-10-09T06:32:36.342 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.496973+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T11:33:04.258326+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-09T06:32:36.342 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.18 is not active+clean 2023-10-09T06:32:36.343 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.343 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.19 is not active+clean 2023-10-09T06:32:36.343 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.497217+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T13:31:31.960774+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-09T06:32:36.343 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.19 is not active+clean 2023-10-09T06:32:36.343 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.343 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.18 is not active+clean 2023-10-09T06:32:36.343 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.496041+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T08:59:10.821319+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-09T06:32:36.343 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.16 is not active+clean 2023-10-09T06:32:36.344 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.344 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.17 is not active+clean 2023-10-09T06:32:36.344 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.493220+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T08:39:36.054325+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-09T06:32:36.344 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2023-10-09T06:32:36.344 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.344 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.16 is not active+clean 2023-10-09T06:32:36.344 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.496928+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T09:35:15.549067+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-09T06:32:36.344 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.14 is not active+clean 2023-10-09T06:32:36.344 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.345 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.15 is not active+clean 2023-10-09T06:32:36.345 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.489029+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T08:00:07.341785+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-09T06:32:36.345 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.15 is not active+clean 2023-10-09T06:32:36.345 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.346 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.14 is not active+clean 2023-10-09T06:32:36.346 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.496682+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T15:06:05.551649+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-09T06:32:36.346 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.12 is not active+clean 2023-10-09T06:32:36.346 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.346 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.13 is not active+clean 2023-10-09T06:32:36.346 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.496858+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T16:37:23.971604+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-09T06:32:36.346 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.13 is not active+clean 2023-10-09T06:32:36.346 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.347 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.12 is not active+clean 2023-10-09T06:32:36.347 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.496225+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T07:04:58.307911+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-09T06:32:36.347 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.10 is not active+clean 2023-10-09T06:32:36.347 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.347 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.11 is not active+clean 2023-10-09T06:32:36.347 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.493686+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T08:09:26.237888+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-09T06:32:36.347 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.11 is not active+clean 2023-10-09T06:32:36.347 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.348 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.10 is not active+clean 2023-10-09T06:32:36.348 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.494203+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T15:06:05.551649+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-09T06:32:36.348 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.e is not active+clean 2023-10-09T06:32:36.348 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.348 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.f is not active+clean 2023-10-09T06:32:36.348 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.495351+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T16:59:11.383765+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-09T06:32:36.348 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.f is not active+clean 2023-10-09T06:32:36.348 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.348 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.e is not active+clean 2023-10-09T06:32:36.348 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.492841+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T07:54:00.527419+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-09T06:32:36.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.c is not active+clean 2023-10-09T06:32:36.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.d is not active+clean 2023-10-09T06:32:36.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.494412+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T13:24:04.774968+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-09T06:32:36.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.d is not active+clean 2023-10-09T06:32:36.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.c is not active+clean 2023-10-09T06:32:36.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.496480+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T13:10:01.011469+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-09T06:32:36.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.b is not active+clean 2023-10-09T06:32:36.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.a is not active+clean 2023-10-09T06:32:36.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.495246+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T11:42:47.425853+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-09T06:32:36.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.8 is not active+clean 2023-10-09T06:32:36.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.9 is not active+clean 2023-10-09T06:32:36.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.493146+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T16:38:53.216443+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-09T06:32:36.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.9 is not active+clean 2023-10-09T06:32:36.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.8 is not active+clean 2023-10-09T06:32:36.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.494316+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T07:36:57.414702+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-09T06:32:36.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.6 is not active+clean 2023-10-09T06:32:36.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.7 is not active+clean 2023-10-09T06:32:36.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.494868+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T15:34:30.240150+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-09T06:32:36.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.7 is not active+clean 2023-10-09T06:32:36.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.6 is not active+clean 2023-10-09T06:32:36.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.492545+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T12:04:38.863681+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-09T06:32:36.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2023-10-09T06:32:36.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.5 is not active+clean 2023-10-09T06:32:36.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.496441+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T14:50:13.029959+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-09T06:32:36.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.5 is not active+clean 2023-10-09T06:32:36.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.4 is not active+clean 2023-10-09T06:32:36.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.494316+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T12:04:38.863681+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-09T06:32:36.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2 is not active+clean 2023-10-09T06:32:36.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3 is not active+clean 2023-10-09T06:32:36.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.495968+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T14:56:27.145572+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-09T06:32:36.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3 is not active+clean 2023-10-09T06:32:36.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2 is not active+clean 2023-10-09T06:32:36.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.496659+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T09:59:45.862276+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-09T06:32:36.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.0 is not active+clean 2023-10-09T06:32:36.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1 is not active+clean 2023-10-09T06:32:36.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.492783+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T18:24:24.863354+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-09T06:32:36.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1 is not active+clean 2023-10-09T06:32:36.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.355 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.0 is not active+clean 2023-10-09T06:32:36.355 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.494336+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T11:34:04.879844+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-09T06:32:36.355 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.a is not active+clean 2023-10-09T06:32:36.355 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:32:30.487000+0000', 'last_change': '2023-10-09T06:32:30.487000+0000', 'last_active': '2023-10-09T06:32:30.487000+0000', 'last_peered': '2023-10-09T06:32:30.487000+0000', 'last_clean': '2023-10-09T06:32:30.487000+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:30.487000+0000', 'last_undegraded': '2023-10-09T06:32:30.487000+0000', 'last_fullsized': '2023-10-09T06:32:30.487000+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:30.487000+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:32:36.355 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.b is not active+clean 2023-10-09T06:32:36.355 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:32:29.484745+0000', 'last_change': '2023-10-09T06:32:29.494948+0000', 'last_active': '2023-10-09T06:32:29.484745+0000', 'last_peered': '2023-10-09T06:32:29.484745+0000', 'last_clean': '2023-10-09T06:32:29.484745+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:32:29.484745+0000', 'last_undegraded': '2023-10-09T06:32:29.484745+0000', 'last_fullsized': '2023-10-09T06:32:29.484745+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'last_clean_scrub_stamp': '2023-10-09T06:32:29.484745+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T07:55:18.693079+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-09T06:32:36.356 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-09T06:32:36.440 INFO:tasks.ceph.mds.c.smithi163.stderr:2023-10-09T06:32:36.440+0000 7f48e9007700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-09T06:32:36.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.477+0000 7fb276dc7700 1 Processor -- start 2023-10-09T06:32:36.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.477+0000 7fb276dc7700 1 -- start start 2023-10-09T06:32:36.481 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.478+0000 7fb276dc7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2700ce900 0x7fb2700cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:36.481 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.478+0000 7fb276dc7700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2700cfa60 con 0x7fb27015c9a0 2023-10-09T06:32:36.481 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.478+0000 7fb276dc7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2700cee50 con 0x7fb27015cd70 2023-10-09T06:32:36.481 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.478+0000 7fb276dc7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2700cf130 con 0x7fb2700ce900 2023-10-09T06:32:36.481 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.478+0000 7fb275dc5700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb27015c9a0 0x7fb2700cb750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:44834/0 (socket says 172.21.15.163:44834) 2023-10-09T06:32:36.481 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.478+0000 7fb275dc5700 1 -- 172.21.15.163:0/981807005 learned_addr learned my addr 172.21.15.163:0/981807005 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:36.481 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.478+0000 7fb2755c4700 1 --2- 172.21.15.163:0/981807005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2700ce900 0x7fb2700cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:36.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.478+0000 7fb274dc3700 1 -- 172.21.15.163:0/981807005 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1968591894 0 0) 0x7fb2700cfa60 con 0x7fb27015c9a0 2023-10-09T06:32:36.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.478+0000 7fb274dc3700 1 -- 172.21.15.163:0/981807005 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb25c003570 con 0x7fb27015c9a0 2023-10-09T06:32:36.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.478+0000 7fb274dc3700 1 -- 172.21.15.163:0/981807005 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1267560640 0 0) 0x7fb25c003570 con 0x7fb27015c9a0 2023-10-09T06:32:36.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.478+0000 7fb274dc3700 1 -- 172.21.15.163:0/981807005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2700ce900 msgr2=0x7fb2700cfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:36.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.478+0000 7fb274dc3700 1 --2- 172.21.15.163:0/981807005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2700ce900 0x7fb2700cfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:36.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.478+0000 7fb274dc3700 1 -- 172.21.15.163:0/981807005 >> v1:172.21.15.181:6789/0 conn(0x7fb27015cd70 legacy=0x7fb27015d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:36.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.478+0000 7fb274dc3700 1 -- 172.21.15.163:0/981807005 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2700cf410 con 0x7fb27015c9a0 2023-10-09T06:32:36.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.478+0000 7fb274dc3700 1 -- 172.21.15.163:0/981807005 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fb2680017f0 con 0x7fb27015c9a0 2023-10-09T06:32:36.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.479+0000 7fb274dc3700 1 -- 172.21.15.163:0/981807005 >> v1:172.21.15.163:6789/0 conn(0x7fb27015c9a0 legacy=0x7fb2700cb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:36.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.479+0000 7fb274dc3700 1 --2- 172.21.15.163:0/981807005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2700ce900 0x7fb25c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:36.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.479+0000 7fb274dc3700 1 --2- 172.21.15.163:0/981807005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb25c004010 0x7fb25c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:36.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.479+0000 7fb274dc3700 1 --2- 172.21.15.163:0/981807005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb25c004d50 0x7fb25c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:36.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.479+0000 7fb274dc3700 1 -- 172.21.15.163:0/981807005 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2700cf130 con 0x7fb25c004d50 2023-10-09T06:32:36.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.479+0000 7fb274dc3700 1 -- 172.21.15.163:0/981807005 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c00bbf0 con 0x7fb25c004010 2023-10-09T06:32:36.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.479+0000 7fb274dc3700 1 -- 172.21.15.163:0/981807005 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c00bdb0 con 0x7fb2700ce900 2023-10-09T06:32:36.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.479+0000 7fb2765c6700 1 --2- 172.21.15.163:0/981807005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb25c004d50 0x7fb25c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:36.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.479+0000 7fb275dc5700 1 --2- 172.21.15.163:0/981807005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2700ce900 0x7fb25c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:36.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.479+0000 7fb2755c4700 1 --2- 172.21.15.163:0/981807005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb25c004010 0x7fb25c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:36.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.479+0000 7fb2765c6700 1 -- 172.21.15.163:0/981807005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2700ce900 msgr2=0x7fb25c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:36.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.479+0000 7fb2765c6700 1 --2- 172.21.15.163:0/981807005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2700ce900 0x7fb25c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:36.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.479+0000 7fb2765c6700 1 -- 172.21.15.163:0/981807005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb25c004010 msgr2=0x7fb25c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:36.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.479+0000 7fb2765c6700 1 --2- 172.21.15.163:0/981807005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb25c004010 0x7fb25c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:36.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.479+0000 7fb2765c6700 1 -- 172.21.15.163:0/981807005 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25c003570 con 0x7fb25c004d50 2023-10-09T06:32:36.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.479+0000 7fb2765c6700 1 --2- 172.21.15.163:0/981807005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb25c004d50 0x7fb25c0051c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb26c001880 tx=0x7fb26c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d95ce1a85c55e3a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:36.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.479+0000 7fb274dc3700 1 -- 172.21.15.163:0/981807005 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c012420 con 0x7fb25c004d50 2023-10-09T06:32:36.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.479+0000 7fb274dc3700 1 -- 172.21.15.163:0/981807005 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26c012580 con 0x7fb25c004d50 2023-10-09T06:32:36.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.479+0000 7fb274dc3700 1 -- 172.21.15.163:0/981807005 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c012920 con 0x7fb25c004d50 2023-10-09T06:32:36.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.480+0000 7fb276dc7700 1 -- 172.21.15.163:0/981807005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb25c004d50 msgr2=0x7fb25c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:36.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.480+0000 7fb276dc7700 1 --2- 172.21.15.163:0/981807005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb25c004d50 0x7fb25c0051c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb26c001880 tx=0x7fb26c007d30 comp rx=0 tx=0).stop 2023-10-09T06:32:36.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.480+0000 7fb2765c6700 1 -- 172.21.15.163:0/981807005 reap_dead start 2023-10-09T06:32:36.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.480+0000 7fb276dc7700 1 -- 172.21.15.163:0/981807005 shutdown_connections 2023-10-09T06:32:36.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.480+0000 7fb276dc7700 1 -- 172.21.15.163:0/981807005 >> 172.21.15.163:0/981807005 conn(0x7fb2700d05c0 msgr2=0x7fb2700d09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:36.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.480+0000 7fb276dc7700 1 -- 172.21.15.163:0/981807005 shutdown_connections 2023-10-09T06:32:36.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.480+0000 7fb276dc7700 1 -- 172.21.15.163:0/981807005 wait complete. 2023-10-09T06:32:36.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.481+0000 7fb276dc7700 1 Processor -- start 2023-10-09T06:32:36.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.481+0000 7fb276dc7700 1 -- start start 2023-10-09T06:32:36.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.482+0000 7fb276dc7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb270149f50 0x7fb270155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:36.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.482+0000 7fb276dc7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb27014a320 0x7fb2700c5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:36.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.482+0000 7fb276dc7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2700c58a0 0x7fb2700b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:36.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.482+0000 7fb276dc7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2700c5d10 con 0x7fb27014a320 2023-10-09T06:32:36.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.482+0000 7fb276dc7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27015c040 con 0x7fb270149f50 2023-10-09T06:32:36.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.482+0000 7fb276dc7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb27015bd60 con 0x7fb2700c58a0 2023-10-09T06:32:36.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.482+0000 7fb2755c4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb27014a320 0x7fb2700c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:36.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.482+0000 7fb2755c4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb27014a320 0x7fb2700c5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59044/0 (socket says 172.21.15.163:59044) 2023-10-09T06:32:36.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.482+0000 7fb2755c4700 1 -- 172.21.15.163:0/1502968939 learned_addr learned my addr 172.21.15.163:0/1502968939 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:36.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.482+0000 7fb275dc5700 1 --2- 172.21.15.163:0/1502968939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb270149f50 0x7fb270155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:36.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.482+0000 7fb2765c6700 1 --2- 172.21.15.163:0/1502968939 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2700c58a0 0x7fb2700b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:36.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.482+0000 7fb2755c4700 1 -- 172.21.15.163:0/1502968939 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2700c58a0 msgr2=0x7fb2700b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:36.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.482+0000 7fb2755c4700 1 --2- 172.21.15.163:0/1502968939 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2700c58a0 0x7fb2700b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:36.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.482+0000 7fb2755c4700 1 -- 172.21.15.163:0/1502968939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb270149f50 msgr2=0x7fb270155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:36.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.482+0000 7fb2755c4700 1 --2- 172.21.15.163:0/1502968939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb270149f50 0x7fb270155650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:36.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.482+0000 7fb2755c4700 1 -- 172.21.15.163:0/1502968939 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb270158b80 con 0x7fb27014a320 2023-10-09T06:32:36.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.482+0000 7fb2765c6700 1 --2- 172.21.15.163:0/1502968939 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2700c58a0 0x7fb2700b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:36.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.482+0000 7fb275dc5700 1 --2- 172.21.15.163:0/1502968939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb270149f50 0x7fb270155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:36.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.482+0000 7fb2755c4700 1 --2- 172.21.15.163:0/1502968939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb27014a320 0x7fb2700c5360 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb260006ac0 tx=0x7fb2600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:36.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.482+0000 7fb265ffb700 1 -- 172.21.15.163:0/1502968939 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26000fa10 con 0x7fb27014a320 2023-10-09T06:32:36.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.483+0000 7fb265ffb700 1 -- 172.21.15.163:0/1502968939 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26000fb70 con 0x7fb27014a320 2023-10-09T06:32:36.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.483+0000 7fb276dc7700 1 -- 172.21.15.163:0/1502968939 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb270158e60 con 0x7fb27014a320 2023-10-09T06:32:36.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.483+0000 7fb265ffb700 1 -- 172.21.15.163:0/1502968939 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb260014440 con 0x7fb27014a320 2023-10-09T06:32:36.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.483+0000 7fb276dc7700 1 -- 172.21.15.163:0/1502968939 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb270155490 con 0x7fb27014a320 2023-10-09T06:32:36.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.484+0000 7fb276dc7700 1 -- 172.21.15.163:0/1502968939 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2701591d0 con 0x7fb27014a320 2023-10-09T06:32:36.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.484+0000 7fb265ffb700 1 -- 172.21.15.163:0/1502968939 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fb260014830 con 0x7fb27014a320 2023-10-09T06:32:36.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.484+0000 7fb265ffb700 1 --2- 172.21.15.163:0/1502968939 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb254066fc0 0x7fb254069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:36.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.484+0000 7fb265ffb700 1 -- 172.21.15.163:0/1502968939 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb260018030 con 0x7fb27014a320 2023-10-09T06:32:36.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.485+0000 7fb275dc5700 1 --2- 172.21.15.163:0/1502968939 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb254066fc0 0x7fb254069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:36.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.485+0000 7fb275dc5700 1 --2- 172.21.15.163:0/1502968939 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb254066fc0 0x7fb254069470 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb268001190 tx=0x7fb268006040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:36.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.488+0000 7fb265ffb700 1 -- 172.21.15.163:0/1502968939 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2600520d0 con 0x7fb27014a320 2023-10-09T06:32:36.747 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.747+0000 7fb276dc7700 1 -- 172.21.15.163:0/1502968939 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb270155490 con 0x7fb27014a320 2023-10-09T06:32:36.747 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.747+0000 7fb265ffb700 1 -- 172.21.15.163:0/1502968939 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1434 (secure 0 0 0) 0x7fb270155490 con 0x7fb27014a320 2023-10-09T06:32:36.747 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:36.747 INFO:teuthology.orchestra.run.smithi163.stdout:{"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":138,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":63,"num_osds":8,"num_up_osds":8,"osd_up_since":1696833029,"num_in_osds":8,"osd_in_since":1696833021,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592674,"bytes_used":2442764288,"bytes_avail":770651348992,"bytes_total":773094113280,"write_bytes_sec":2218,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":40,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"c","status":"up:active","gid":5748}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2023-10-09T06:32:20.323152+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-09T06:32:36.749 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.749+0000 7fb276dc7700 1 -- 172.21.15.163:0/1502968939 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb254066fc0 msgr2=0x7fb254069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:36.750 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.749+0000 7fb276dc7700 1 --2- 172.21.15.163:0/1502968939 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb254066fc0 0x7fb254069470 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb268001190 tx=0x7fb268006040 comp rx=0 tx=0).stop 2023-10-09T06:32:36.750 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.749+0000 7fb276dc7700 1 -- 172.21.15.163:0/1502968939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb27014a320 msgr2=0x7fb2700c5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:36.750 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.749+0000 7fb276dc7700 1 --2- 172.21.15.163:0/1502968939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb27014a320 0x7fb2700c5360 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb260006ac0 tx=0x7fb2600091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:36.750 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.749+0000 7fb276dc7700 1 -- 172.21.15.163:0/1502968939 shutdown_connections 2023-10-09T06:32:36.750 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.749+0000 7fb276dc7700 1 --2- 172.21.15.163:0/1502968939 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb254066fc0 0x7fb254069470 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:36.751 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.749+0000 7fb276dc7700 1 --2- 172.21.15.163:0/1502968939 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2700c58a0 0x7fb2700b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:36.751 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.749+0000 7fb276dc7700 1 --2- 172.21.15.163:0/1502968939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb270149f50 0x7fb270155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:36.751 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.749+0000 7fb276dc7700 1 --2- 172.21.15.163:0/1502968939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb27014a320 0x7fb2700c5360 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:36.751 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.749+0000 7fb276dc7700 1 -- 172.21.15.163:0/1502968939 >> 172.21.15.163:0/1502968939 conn(0x7fb2700d05c0 msgr2=0x7fb270156290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:36.751 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.750+0000 7fb276dc7700 1 -- 172.21.15.163:0/1502968939 shutdown_connections 2023-10-09T06:32:36.752 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.750+0000 7fb276dc7700 1 -- 172.21.15.163:0/1502968939 wait complete. 2023-10-09T06:32:36.762 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-09T06:32:36.762 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:32:36.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.877+0000 7fbb7759e700 1 Processor -- start 2023-10-09T06:32:36.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.877+0000 7fbb7759e700 1 -- start start 2023-10-09T06:32:36.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.878+0000 7fbb7759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb78150550 0x7fbb78157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:36.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.878+0000 7fbb7759e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbb780c7410 0x7fbb78069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:36.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.878+0000 7fbb7759e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb7806e2a0 con 0x7fbb78150180 2023-10-09T06:32:36.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.878+0000 7fbb7759e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb7806b2e0 con 0x7fbb780c7410 2023-10-09T06:32:36.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.878+0000 7fbb7759e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb780c78e0 con 0x7fbb78150550 2023-10-09T06:32:36.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.878+0000 7fbb7659c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fbb78150180 0x7fbb780c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:44836/0 (socket says 172.21.15.163:44836) 2023-10-09T06:32:36.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.878+0000 7fbb7659c700 1 -- 172.21.15.163:0/597022583 learned_addr learned my addr 172.21.15.163:0/597022583 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:36.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.878+0000 7fbb76d9d700 1 --2- 172.21.15.163:0/597022583 >> v2:172.21.15.181:3300/0 conn(0x7fbb780c7410 0x7fbb78069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:36.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.878+0000 7fbb6dd9b700 1 --2- 172.21.15.163:0/597022583 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb78150550 0x7fbb78157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:36.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.878+0000 7fbb75d9b700 1 -- 172.21.15.163:0/597022583 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 386978171 0 0) 0x7fbb7806e2a0 con 0x7fbb78150180 2023-10-09T06:32:36.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.878+0000 7fbb75d9b700 1 -- 172.21.15.163:0/597022583 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb64003570 con 0x7fbb78150180 2023-10-09T06:32:36.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.878+0000 7fbb75d9b700 1 -- 172.21.15.163:0/597022583 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2427957505 0 0) 0x7fbb64003570 con 0x7fbb78150180 2023-10-09T06:32:36.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.878+0000 7fbb75d9b700 1 -- 172.21.15.163:0/597022583 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb78150550 msgr2=0x7fbb78157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:36.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.878+0000 7fbb75d9b700 1 --2- 172.21.15.163:0/597022583 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb78150550 0x7fbb78157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:36.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.878+0000 7fbb75d9b700 1 -- 172.21.15.163:0/597022583 >> v2:172.21.15.181:3300/0 conn(0x7fbb780c7410 msgr2=0x7fbb78069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:36.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.878+0000 7fbb75d9b700 1 --2- 172.21.15.163:0/597022583 >> v2:172.21.15.181:3300/0 conn(0x7fbb780c7410 0x7fbb78069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:36.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.878+0000 7fbb75d9b700 1 -- 172.21.15.163:0/597022583 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb780c7bc0 con 0x7fbb78150180 2023-10-09T06:32:36.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.878+0000 7fbb75d9b700 1 -- 172.21.15.163:0/597022583 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fbb68001470 con 0x7fbb78150180 2023-10-09T06:32:36.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.879+0000 7fbb75d9b700 1 -- 172.21.15.163:0/597022583 >> v1:172.21.15.163:6789/0 conn(0x7fbb78150180 legacy=0x7fbb780c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:36.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.879+0000 7fbb75d9b700 1 --2- 172.21.15.163:0/597022583 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb64003a60 0x7fbb64003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:36.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.879+0000 7fbb75d9b700 1 --2- 172.21.15.163:0/597022583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb64004810 0x7fbb64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:36.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.879+0000 7fbb75d9b700 1 --2- 172.21.15.163:0/597022583 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb78150550 0x7fbb640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:36.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.879+0000 7fbb75d9b700 1 -- 172.21.15.163:0/597022583 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7806b2e0 con 0x7fbb64003a60 2023-10-09T06:32:36.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.879+0000 7fbb75d9b700 1 -- 172.21.15.163:0/597022583 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb780c78e0 con 0x7fbb64004810 2023-10-09T06:32:36.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.879+0000 7fbb75d9b700 1 -- 172.21.15.163:0/597022583 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6400ba60 con 0x7fbb78150550 2023-10-09T06:32:36.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.879+0000 7fbb7659c700 1 --2- 172.21.15.163:0/597022583 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb64003a60 0x7fbb64003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:36.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.879+0000 7fbb6dd9b700 1 --2- 172.21.15.163:0/597022583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb64004810 0x7fbb64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:36.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.879+0000 7fbb76d9d700 1 --2- 172.21.15.163:0/597022583 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb78150550 0x7fbb640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:36.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.879+0000 7fbb7659c700 1 -- 172.21.15.163:0/597022583 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb78150550 msgr2=0x7fbb640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:36.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.879+0000 7fbb7659c700 1 --2- 172.21.15.163:0/597022583 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb78150550 0x7fbb640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:36.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.879+0000 7fbb7659c700 1 -- 172.21.15.163:0/597022583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb64004810 msgr2=0x7fbb64004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:36.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.879+0000 7fbb7659c700 1 --2- 172.21.15.163:0/597022583 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb64004810 0x7fbb64004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:36.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.879+0000 7fbb7659c700 1 -- 172.21.15.163:0/597022583 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb64003570 con 0x7fbb64003a60 2023-10-09T06:32:36.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.879+0000 7fbb7659c700 1 --2- 172.21.15.163:0/597022583 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb64003a60 0x7fbb64003e70 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800c3f0 tx=0x7fbb68001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d9b0b9c45d9d4c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:36.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.879+0000 7fbb75d9b700 1 -- 172.21.15.163:0/597022583 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6800cc80 con 0x7fbb64003a60 2023-10-09T06:32:36.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.879+0000 7fbb75d9b700 1 -- 172.21.15.163:0/597022583 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6800cde0 con 0x7fbb64003a60 2023-10-09T06:32:36.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.879+0000 7fbb75d9b700 1 -- 172.21.15.163:0/597022583 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb68005b50 con 0x7fbb64003a60 2023-10-09T06:32:36.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.880+0000 7fbb7759e700 1 -- 172.21.15.163:0/597022583 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb64003a60 msgr2=0x7fbb64003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:36.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.880+0000 7fbb7759e700 1 --2- 172.21.15.163:0/597022583 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb64003a60 0x7fbb64003e70 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800c3f0 tx=0x7fbb68001e50 comp rx=0 tx=0).stop 2023-10-09T06:32:36.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.880+0000 7fbb76d9d700 1 -- 172.21.15.163:0/597022583 reap_dead start 2023-10-09T06:32:36.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.880+0000 7fbb7759e700 1 -- 172.21.15.163:0/597022583 shutdown_connections 2023-10-09T06:32:36.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.880+0000 7fbb7759e700 1 -- 172.21.15.163:0/597022583 >> 172.21.15.163:0/597022583 conn(0x7fbb780cc1f0 msgr2=0x7fbb780cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:36.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.880+0000 7fbb7759e700 1 -- 172.21.15.163:0/597022583 shutdown_connections 2023-10-09T06:32:36.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.880+0000 7fbb7759e700 1 -- 172.21.15.163:0/597022583 wait complete. 2023-10-09T06:32:36.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.881+0000 7fbb7759e700 1 Processor -- start 2023-10-09T06:32:36.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.881+0000 7fbb7759e700 1 -- start start 2023-10-09T06:32:36.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.882+0000 7fbb7759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb64003a60 0x7fbb7812f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:36.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.882+0000 7fbb7759e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb78144850 0x7fbb7814e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:36.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.882+0000 7fbb7759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb7814ee20 0x7fbb780c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:36.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.882+0000 7fbb7759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb780ca890 con 0x7fbb7814ee20 2023-10-09T06:32:36.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.882+0000 7fbb7759e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7814f2a0 con 0x7fbb78144850 2023-10-09T06:32:36.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.882+0000 7fbb7759e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb780c4fd0 con 0x7fbb64003a60 2023-10-09T06:32:36.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.882+0000 7fbb76d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb7814ee20 0x7fbb780c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:36.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.882+0000 7fbb76d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb7814ee20 0x7fbb780c32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59064/0 (socket says 172.21.15.163:59064) 2023-10-09T06:32:36.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.882+0000 7fbb76d9d700 1 -- 172.21.15.163:0/1447866394 learned_addr learned my addr 172.21.15.163:0/1447866394 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:36.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.882+0000 7fbb6dd9b700 1 --2- 172.21.15.163:0/1447866394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb78144850 0x7fbb7814e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:36.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.882+0000 7fbb7659c700 1 --2- 172.21.15.163:0/1447866394 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb64003a60 0x7fbb7812f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:36.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.882+0000 7fbb76d9d700 1 -- 172.21.15.163:0/1447866394 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb64003a60 msgr2=0x7fbb7812f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:36.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.882+0000 7fbb76d9d700 1 --2- 172.21.15.163:0/1447866394 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb64003a60 0x7fbb7812f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:36.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.882+0000 7fbb76d9d700 1 -- 172.21.15.163:0/1447866394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb78144850 msgr2=0x7fbb7814e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:36.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.882+0000 7fbb76d9d700 1 --2- 172.21.15.163:0/1447866394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb78144850 0x7fbb7814e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:36.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.882+0000 7fbb76d9d700 1 -- 172.21.15.163:0/1447866394 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb780c3ef0 con 0x7fbb7814ee20 2023-10-09T06:32:36.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.882+0000 7fbb76d9d700 1 --2- 172.21.15.163:0/1447866394 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb7814ee20 0x7fbb780c32a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fbb780c67e0 tx=0x7fbb7000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:36.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.882+0000 7fbb6effd700 1 -- 172.21.15.163:0/1447866394 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7000e040 con 0x7fbb7814ee20 2023-10-09T06:32:36.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.882+0000 7fbb6effd700 1 -- 172.21.15.163:0/1447866394 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7000f040 con 0x7fbb7814ee20 2023-10-09T06:32:36.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.882+0000 7fbb7759e700 1 -- 172.21.15.163:0/1447866394 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb780c41d0 con 0x7fbb7814ee20 2023-10-09T06:32:36.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.883+0000 7fbb6effd700 1 -- 172.21.15.163:0/1447866394 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7000fd40 con 0x7fbb7814ee20 2023-10-09T06:32:36.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.883+0000 7fbb7759e700 1 -- 172.21.15.163:0/1447866394 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb780c4c60 con 0x7fbb7814ee20 2023-10-09T06:32:36.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.883+0000 7fbb7759e700 1 -- 172.21.15.163:0/1447866394 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb7804c3c0 con 0x7fbb7814ee20 2023-10-09T06:32:36.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.884+0000 7fbb6effd700 1 -- 172.21.15.163:0/1447866394 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fbb70015640 con 0x7fbb7814ee20 2023-10-09T06:32:36.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.884+0000 7fbb6effd700 1 --2- 172.21.15.163:0/1447866394 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb58066ff0 0x7fbb580694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:36.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.884+0000 7fbb6effd700 1 -- 172.21.15.163:0/1447866394 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbb70084c60 con 0x7fbb7814ee20 2023-10-09T06:32:36.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.885+0000 7fbb7659c700 1 --2- 172.21.15.163:0/1447866394 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb58066ff0 0x7fbb580694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:36.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.885+0000 7fbb7659c700 1 --2- 172.21.15.163:0/1447866394 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb58066ff0 0x7fbb580694a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800c310 tx=0x7fbb68015000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:36.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:36.889+0000 7fbb6effd700 1 -- 172.21.15.163:0/1447866394 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb70051dd0 con 0x7fbb7814ee20 2023-10-09T06:32:37.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:37.071+0000 7fbb7759e700 1 -- 172.21.15.163:0/1447866394 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbb780c5110 con 0x7fbb58066ff0 2023-10-09T06:32:37.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:37.080+0000 7fbb6effd700 1 -- 172.21.15.163:0/1447866394 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289752 (secure 0 0 0) 0x7fbb780c5110 con 0x7fbb58066ff0 2023-10-09T06:32:37.083 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:37.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:37.083+0000 7fbb7759e700 1 -- 172.21.15.163:0/1447866394 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb58066ff0 msgr2=0x7fbb580694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:37.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:37.083+0000 7fbb7759e700 1 --2- 172.21.15.163:0/1447866394 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb58066ff0 0x7fbb580694a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800c310 tx=0x7fbb68015000 comp rx=0 tx=0).stop 2023-10-09T06:32:37.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:37.083+0000 7fbb7759e700 1 -- 172.21.15.163:0/1447866394 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb7814ee20 msgr2=0x7fbb780c32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:37.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:37.083+0000 7fbb7759e700 1 --2- 172.21.15.163:0/1447866394 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb7814ee20 0x7fbb780c32a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fbb780c67e0 tx=0x7fbb7000b040 comp rx=0 tx=0).stop 2023-10-09T06:32:37.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:37.084+0000 7fbb7759e700 1 -- 172.21.15.163:0/1447866394 shutdown_connections 2023-10-09T06:32:37.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:37.084+0000 7fbb7759e700 1 --2- 172.21.15.163:0/1447866394 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb58066ff0 0x7fbb580694a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:37.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:37.084+0000 7fbb7759e700 1 --2- 172.21.15.163:0/1447866394 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb7814ee20 0x7fbb780c32a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:37.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:37.084+0000 7fbb7759e700 1 --2- 172.21.15.163:0/1447866394 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb64003a60 0x7fbb7812f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:37.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:37.084+0000 7fbb7759e700 1 --2- 172.21.15.163:0/1447866394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb78144850 0x7fbb7814e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:37.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:37.084+0000 7fbb7759e700 1 -- 172.21.15.163:0/1447866394 >> 172.21.15.163:0/1447866394 conn(0x7fbb780cc1f0 msgr2=0x7fbb780ac3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:37.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:37.084+0000 7fbb7759e700 1 -- 172.21.15.163:0/1447866394 shutdown_connections 2023-10-09T06:32:37.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:37.084+0000 7fbb7759e700 1 -- 172.21.15.163:0/1447866394 wait complete. 2023-10-09T06:32:37.086 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:32:37.096 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2023-10-09T06:32:36.326313+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2385512,"kb_used_data":7208,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752589208,"statfs":{"total":773094113280,"available":770651348992,"internally_reserved":0,"allocated":7380992,"data_stored":3803692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001231"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.527160+0000","last_change":"2023-10-09T06:32:31.524637+0000","last_active":"2023-10-09T06:32:31.527160+0000","last_peered":"2023-10-09T06:32:31.527160+0000","last_clean":"2023-10-09T06:32:31.527160+0000","last_became_active":"2023-10-09T06:32:31.524358+0000","last_became_peered":"2023-10-09T06:32:31.524358+0000","last_unstale":"2023-10-09T06:32:31.527160+0000","last_undegraded":"2023-10-09T06:32:31.527160+0000","last_fullsized":"2023-10-09T06:32:31.527160+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:53:37.666517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.512071+0000","last_change":"2023-10-09T06:32:31.509377+0000","last_active":"2023-10-09T06:32:31.512071+0000","last_peered":"2023-10-09T06:32:31.512071+0000","last_clean":"2023-10-09T06:32:31.512071+0000","last_became_active":"2023-10-09T06:32:31.509105+0000","last_became_peered":"2023-10-09T06:32:31.509105+0000","last_unstale":"2023-10-09T06:32:31.512071+0000","last_undegraded":"2023-10-09T06:32:31.512071+0000","last_fullsized":"2023-10-09T06:32:31.512071+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:26:40.475869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.521134+0000","last_change":"2023-10-09T06:32:31.516319+0000","last_active":"2023-10-09T06:32:31.521134+0000","last_peered":"2023-10-09T06:32:31.521134+0000","last_clean":"2023-10-09T06:32:31.521134+0000","last_became_active":"2023-10-09T06:32:31.516098+0000","last_became_peered":"2023-10-09T06:32:31.516098+0000","last_unstale":"2023-10-09T06:32:31.521134+0000","last_undegraded":"2023-10-09T06:32:31.521134+0000","last_fullsized":"2023-10-09T06:32:31.521134+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:01:10.352276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.516721+0000","last_change":"2023-10-09T06:32:31.508795+0000","last_active":"2023-10-09T06:32:31.516721+0000","last_peered":"2023-10-09T06:32:31.516721+0000","last_clean":"2023-10-09T06:32:31.516721+0000","last_became_active":"2023-10-09T06:32:31.508552+0000","last_became_peered":"2023-10-09T06:32:31.508552+0000","last_unstale":"2023-10-09T06:32:31.516721+0000","last_undegraded":"2023-10-09T06:32:31.516721+0000","last_fullsized":"2023-10-09T06:32:31.516721+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:07:25.930259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.521746+0000","last_change":"2023-10-09T06:32:31.516254+0000","last_active":"2023-10-09T06:32:31.521746+0000","last_peered":"2023-10-09T06:32:31.521746+0000","last_clean":"2023-10-09T06:32:31.521746+0000","last_became_active":"2023-10-09T06:32:31.516028+0000","last_became_peered":"2023-10-09T06:32:31.516028+0000","last_unstale":"2023-10-09T06:32:31.521746+0000","last_undegraded":"2023-10-09T06:32:31.521746+0000","last_fullsized":"2023-10-09T06:32:31.521746+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:04:59.310166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.513536+0000","last_change":"2023-10-09T06:32:31.510222+0000","last_active":"2023-10-09T06:32:31.513536+0000","last_peered":"2023-10-09T06:32:31.513536+0000","last_clean":"2023-10-09T06:32:31.513536+0000","last_became_active":"2023-10-09T06:32:31.509976+0000","last_became_peered":"2023-10-09T06:32:31.509976+0000","last_unstale":"2023-10-09T06:32:31.513536+0000","last_undegraded":"2023-10-09T06:32:31.513536+0000","last_fullsized":"2023-10-09T06:32:31.513536+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:46:11.204402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.507551+0000","last_change":"2023-10-09T06:32:31.506356+0000","last_active":"2023-10-09T06:32:31.507551+0000","last_peered":"2023-10-09T06:32:31.507551+0000","last_clean":"2023-10-09T06:32:31.507551+0000","last_became_active":"2023-10-09T06:32:31.506091+0000","last_became_peered":"2023-10-09T06:32:31.506091+0000","last_unstale":"2023-10-09T06:32:31.507551+0000","last_undegraded":"2023-10-09T06:32:31.507551+0000","last_fullsized":"2023-10-09T06:32:31.507551+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:18:48.881835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.510892+0000","last_change":"2023-10-09T06:32:31.507320+0000","last_active":"2023-10-09T06:32:31.510892+0000","last_peered":"2023-10-09T06:32:31.510892+0000","last_clean":"2023-10-09T06:32:31.510892+0000","last_became_active":"2023-10-09T06:32:31.507026+0000","last_became_peered":"2023-10-09T06:32:31.507026+0000","last_unstale":"2023-10-09T06:32:31.510892+0000","last_undegraded":"2023-10-09T06:32:31.510892+0000","last_fullsized":"2023-10-09T06:32:31.510892+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:35:57.211551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.525232+0000","last_change":"2023-10-09T06:32:31.517060+0000","last_active":"2023-10-09T06:32:31.525232+0000","last_peered":"2023-10-09T06:32:31.525232+0000","last_clean":"2023-10-09T06:32:31.525232+0000","last_became_active":"2023-10-09T06:32:31.515127+0000","last_became_peered":"2023-10-09T06:32:31.515127+0000","last_unstale":"2023-10-09T06:32:31.525232+0000","last_undegraded":"2023-10-09T06:32:31.525232+0000","last_fullsized":"2023-10-09T06:32:31.525232+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:59:12.386020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.529375+0000","last_change":"2023-10-09T06:32:31.519838+0000","last_active":"2023-10-09T06:32:31.529375+0000","last_peered":"2023-10-09T06:32:31.529375+0000","last_clean":"2023-10-09T06:32:31.529375+0000","last_became_active":"2023-10-09T06:32:31.519512+0000","last_became_peered":"2023-10-09T06:32:31.519512+0000","last_unstale":"2023-10-09T06:32:31.529375+0000","last_undegraded":"2023-10-09T06:32:31.529375+0000","last_fullsized":"2023-10-09T06:32:31.529375+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:09:27.240143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.526723+0000","last_change":"2023-10-09T06:32:31.521757+0000","last_active":"2023-10-09T06:32:31.526723+0000","last_peered":"2023-10-09T06:32:31.526723+0000","last_clean":"2023-10-09T06:32:31.526723+0000","last_became_active":"2023-10-09T06:32:31.521357+0000","last_became_peered":"2023-10-09T06:32:31.521357+0000","last_unstale":"2023-10-09T06:32:31.526723+0000","last_undegraded":"2023-10-09T06:32:31.526723+0000","last_fullsized":"2023-10-09T06:32:31.526723+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:49:05.440086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.528195+0000","last_change":"2023-10-09T06:32:31.517951+0000","last_active":"2023-10-09T06:32:31.528195+0000","last_peered":"2023-10-09T06:32:31.528195+0000","last_clean":"2023-10-09T06:32:31.528195+0000","last_became_active":"2023-10-09T06:32:31.517424+0000","last_became_peered":"2023-10-09T06:32:31.517424+0000","last_unstale":"2023-10-09T06:32:31.528195+0000","last_undegraded":"2023-10-09T06:32:31.528195+0000","last_fullsized":"2023-10-09T06:32:31.528195+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:09:27.240143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.516805+0000","last_change":"2023-10-09T06:32:31.513229+0000","last_active":"2023-10-09T06:32:31.516805+0000","last_peered":"2023-10-09T06:32:31.516805+0000","last_clean":"2023-10-09T06:32:31.516805+0000","last_became_active":"2023-10-09T06:32:31.512717+0000","last_became_peered":"2023-10-09T06:32:31.512717+0000","last_unstale":"2023-10-09T06:32:31.516805+0000","last_undegraded":"2023-10-09T06:32:31.516805+0000","last_fullsized":"2023-10-09T06:32:31.516805+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:21:48.208957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.525179+0000","last_change":"2023-10-09T06:32:31.518045+0000","last_active":"2023-10-09T06:32:31.525179+0000","last_peered":"2023-10-09T06:32:31.525179+0000","last_clean":"2023-10-09T06:32:31.525179+0000","last_became_active":"2023-10-09T06:32:31.517815+0000","last_became_peered":"2023-10-09T06:32:31.517815+0000","last_unstale":"2023-10-09T06:32:31.525179+0000","last_undegraded":"2023-10-09T06:32:31.525179+0000","last_fullsized":"2023-10-09T06:32:31.525179+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:35:16.551323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.520320+0000","last_change":"2023-10-09T06:32:31.519359+0000","last_active":"2023-10-09T06:32:31.520320+0000","last_peered":"2023-10-09T06:32:31.520320+0000","last_clean":"2023-10-09T06:32:31.520320+0000","last_became_active":"2023-10-09T06:32:31.519098+0000","last_became_peered":"2023-10-09T06:32:31.519098+0000","last_unstale":"2023-10-09T06:32:31.520320+0000","last_undegraded":"2023-10-09T06:32:31.520320+0000","last_fullsized":"2023-10-09T06:32:31.520320+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:33:33.661840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.528856+0000","last_change":"2023-10-09T06:32:31.513317+0000","last_active":"2023-10-09T06:32:31.528856+0000","last_peered":"2023-10-09T06:32:31.528856+0000","last_clean":"2023-10-09T06:32:31.528856+0000","last_became_active":"2023-10-09T06:32:31.511334+0000","last_became_peered":"2023-10-09T06:32:31.511334+0000","last_unstale":"2023-10-09T06:32:31.528856+0000","last_undegraded":"2023-10-09T06:32:31.528856+0000","last_fullsized":"2023-10-09T06:32:31.528856+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:38:54.218698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.525254+0000","last_change":"2023-10-09T06:32:31.517184+0000","last_active":"2023-10-09T06:32:31.525254+0000","last_peered":"2023-10-09T06:32:31.525254+0000","last_clean":"2023-10-09T06:32:31.525254+0000","last_became_active":"2023-10-09T06:32:31.514902+0000","last_became_peered":"2023-10-09T06:32:31.514902+0000","last_unstale":"2023-10-09T06:32:31.525254+0000","last_undegraded":"2023-10-09T06:32:31.525254+0000","last_fullsized":"2023-10-09T06:32:31.525254+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:42:48.428108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.522573+0000","last_change":"2023-10-09T06:32:31.514575+0000","last_active":"2023-10-09T06:32:31.522573+0000","last_peered":"2023-10-09T06:32:31.522573+0000","last_clean":"2023-10-09T06:32:31.522573+0000","last_became_active":"2023-10-09T06:32:31.514322+0000","last_became_peered":"2023-10-09T06:32:31.514322+0000","last_unstale":"2023-10-09T06:32:31.522573+0000","last_undegraded":"2023-10-09T06:32:31.522573+0000","last_fullsized":"2023-10-09T06:32:31.522573+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:04:39.865936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.510995+0000","last_change":"2023-10-09T06:32:31.505630+0000","last_active":"2023-10-09T06:32:31.510995+0000","last_peered":"2023-10-09T06:32:31.510995+0000","last_clean":"2023-10-09T06:32:31.510995+0000","last_became_active":"2023-10-09T06:32:31.505350+0000","last_became_peered":"2023-10-09T06:32:31.505350+0000","last_unstale":"2023-10-09T06:32:31.510995+0000","last_undegraded":"2023-10-09T06:32:31.510995+0000","last_fullsized":"2023-10-09T06:32:31.510995+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:38:54.218698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.519896+0000","last_change":"2023-10-09T06:32:31.515962+0000","last_active":"2023-10-09T06:32:31.519896+0000","last_peered":"2023-10-09T06:32:31.519896+0000","last_clean":"2023-10-09T06:32:31.519896+0000","last_became_active":"2023-10-09T06:32:31.515659+0000","last_became_peered":"2023-10-09T06:32:31.515659+0000","last_unstale":"2023-10-09T06:32:31.519896+0000","last_undegraded":"2023-10-09T06:32:31.519896+0000","last_fullsized":"2023-10-09T06:32:31.519896+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:07:09.728724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.521854+0000","last_change":"2023-10-09T06:32:31.511806+0000","last_active":"2023-10-09T06:32:31.521854+0000","last_peered":"2023-10-09T06:32:31.521854+0000","last_clean":"2023-10-09T06:32:31.521854+0000","last_became_active":"2023-10-09T06:32:31.511427+0000","last_became_peered":"2023-10-09T06:32:31.511427+0000","last_unstale":"2023-10-09T06:32:31.521854+0000","last_undegraded":"2023-10-09T06:32:31.521854+0000","last_fullsized":"2023-10-09T06:32:31.521854+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:04:39.865936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.529158+0000","last_change":"2023-10-09T06:32:31.518668+0000","last_active":"2023-10-09T06:32:31.529158+0000","last_peered":"2023-10-09T06:32:31.529158+0000","last_clean":"2023-10-09T06:32:31.529158+0000","last_became_active":"2023-10-09T06:32:31.518505+0000","last_became_peered":"2023-10-09T06:32:31.518505+0000","last_unstale":"2023-10-09T06:32:31.529158+0000","last_undegraded":"2023-10-09T06:32:31.529158+0000","last_fullsized":"2023-10-09T06:32:31.529158+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:07:25.930259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.523812+0000","last_change":"2023-10-09T06:32:31.519661+0000","last_active":"2023-10-09T06:32:31.523812+0000","last_peered":"2023-10-09T06:32:31.523812+0000","last_clean":"2023-10-09T06:32:31.523812+0000","last_became_active":"2023-10-09T06:32:31.519351+0000","last_became_peered":"2023-10-09T06:32:31.519351+0000","last_unstale":"2023-10-09T06:32:31.523812+0000","last_undegraded":"2023-10-09T06:32:31.523812+0000","last_fullsized":"2023-10-09T06:32:31.523812+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:00:08.344040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.526936+0000","last_change":"2023-10-09T06:32:31.523432+0000","last_active":"2023-10-09T06:32:31.526936+0000","last_peered":"2023-10-09T06:32:31.526936+0000","last_clean":"2023-10-09T06:32:31.526936+0000","last_became_active":"2023-10-09T06:32:31.523182+0000","last_became_peered":"2023-10-09T06:32:31.523182+0000","last_unstale":"2023-10-09T06:32:31.526936+0000","last_undegraded":"2023-10-09T06:32:31.526936+0000","last_fullsized":"2023-10-09T06:32:31.526936+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:39:56.949135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.528881+0000","last_change":"2023-10-09T06:32:31.513310+0000","last_active":"2023-10-09T06:32:31.528881+0000","last_peered":"2023-10-09T06:32:31.528881+0000","last_clean":"2023-10-09T06:32:31.528881+0000","last_became_active":"2023-10-09T06:32:31.513019+0000","last_became_peered":"2023-10-09T06:32:31.513019+0000","last_unstale":"2023-10-09T06:32:31.528881+0000","last_undegraded":"2023-10-09T06:32:31.528881+0000","last_fullsized":"2023-10-09T06:32:31.528881+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:39:47.607716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.506768+0000","last_change":"2023-10-09T06:32:31.505179+0000","last_active":"2023-10-09T06:32:31.506768+0000","last_peered":"2023-10-09T06:32:31.506768+0000","last_clean":"2023-10-09T06:32:31.506768+0000","last_became_active":"2023-10-09T06:32:31.504885+0000","last_became_peered":"2023-10-09T06:32:31.504885+0000","last_unstale":"2023-10-09T06:32:31.506768+0000","last_undegraded":"2023-10-09T06:32:31.506768+0000","last_fullsized":"2023-10-09T06:32:31.506768+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:51:14.139573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.525859+0000","last_change":"2023-10-09T06:32:31.524416+0000","last_active":"2023-10-09T06:32:31.525859+0000","last_peered":"2023-10-09T06:32:31.525859+0000","last_clean":"2023-10-09T06:32:31.525859+0000","last_became_active":"2023-10-09T06:32:31.524023+0000","last_became_peered":"2023-10-09T06:32:31.524023+0000","last_unstale":"2023-10-09T06:32:31.525859+0000","last_undegraded":"2023-10-09T06:32:31.525859+0000","last_fullsized":"2023-10-09T06:32:31.525859+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:56:33.370165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.529382+0000","last_change":"2023-10-09T06:32:31.519822+0000","last_active":"2023-10-09T06:32:31.529382+0000","last_peered":"2023-10-09T06:32:31.529382+0000","last_clean":"2023-10-09T06:32:31.529382+0000","last_became_active":"2023-10-09T06:32:31.519588+0000","last_became_peered":"2023-10-09T06:32:31.519588+0000","last_unstale":"2023-10-09T06:32:31.529382+0000","last_undegraded":"2023-10-09T06:32:31.529382+0000","last_fullsized":"2023-10-09T06:32:31.529382+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:07:09.907907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.522387+0000","last_change":"2023-10-09T06:32:31.520850+0000","last_active":"2023-10-09T06:32:31.522387+0000","last_peered":"2023-10-09T06:32:31.522387+0000","last_clean":"2023-10-09T06:32:31.522387+0000","last_became_active":"2023-10-09T06:32:31.520588+0000","last_became_peered":"2023-10-09T06:32:31.520588+0000","last_unstale":"2023-10-09T06:32:31.522387+0000","last_undegraded":"2023-10-09T06:32:31.522387+0000","last_fullsized":"2023-10-09T06:32:31.522387+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:17:44.451765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.510590+0000","last_change":"2023-10-09T06:32:31.509245+0000","last_active":"2023-10-09T06:32:31.510590+0000","last_peered":"2023-10-09T06:32:31.510590+0000","last_clean":"2023-10-09T06:32:31.510590+0000","last_became_active":"2023-10-09T06:32:31.508438+0000","last_became_peered":"2023-10-09T06:32:31.508438+0000","last_unstale":"2023-10-09T06:32:31.510590+0000","last_undegraded":"2023-10-09T06:32:31.510590+0000","last_fullsized":"2023-10-09T06:32:31.510590+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:49:05.440086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.520738+0000","last_change":"2023-10-09T06:32:31.515818+0000","last_active":"2023-10-09T06:32:31.520738+0000","last_peered":"2023-10-09T06:32:31.520738+0000","last_clean":"2023-10-09T06:32:31.520738+0000","last_became_active":"2023-10-09T06:32:31.515526+0000","last_became_peered":"2023-10-09T06:32:31.515526+0000","last_unstale":"2023-10-09T06:32:31.520738+0000","last_undegraded":"2023-10-09T06:32:31.520738+0000","last_fullsized":"2023-10-09T06:32:31.520738+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:59:49.326137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.521700+0000","last_change":"2023-10-09T06:32:31.519313+0000","last_active":"2023-10-09T06:32:31.521700+0000","last_peered":"2023-10-09T06:32:31.521700+0000","last_clean":"2023-10-09T06:32:31.521700+0000","last_became_active":"2023-10-09T06:32:31.518783+0000","last_became_peered":"2023-10-09T06:32:31.518783+0000","last_unstale":"2023-10-09T06:32:31.521700+0000","last_undegraded":"2023-10-09T06:32:31.521700+0000","last_fullsized":"2023-10-09T06:32:31.521700+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:23:21.996048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.526829+0000","last_change":"2023-10-09T06:32:31.522663+0000","last_active":"2023-10-09T06:32:31.526829+0000","last_peered":"2023-10-09T06:32:31.526829+0000","last_clean":"2023-10-09T06:32:31.526829+0000","last_became_active":"2023-10-09T06:32:31.522387+0000","last_became_peered":"2023-10-09T06:32:31.522387+0000","last_unstale":"2023-10-09T06:32:31.526829+0000","last_undegraded":"2023-10-09T06:32:31.526829+0000","last_fullsized":"2023-10-09T06:32:31.526829+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:54:40.864124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"62'4","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.528535+0000","last_change":"2023-10-09T06:32:30.516709+0000","last_active":"2023-10-09T06:32:31.528535+0000","last_peered":"2023-10-09T06:32:31.528535+0000","last_clean":"2023-10-09T06:32:31.528535+0000","last_became_active":"2023-10-09T06:32:30.516362+0000","last_became_peered":"2023-10-09T06:32:30.516362+0000","last_unstale":"2023-10-09T06:32:31.528535+0000","last_undegraded":"2023-10-09T06:32:31.528535+0000","last_fullsized":"2023-10-09T06:32:31.528535+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:59:22.255688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.521741+0000","last_change":"2023-10-09T06:32:31.515374+0000","last_active":"2023-10-09T06:32:31.521741+0000","last_peered":"2023-10-09T06:32:31.521741+0000","last_clean":"2023-10-09T06:32:31.521741+0000","last_became_active":"2023-10-09T06:32:31.514641+0000","last_became_peered":"2023-10-09T06:32:31.514641+0000","last_unstale":"2023-10-09T06:32:31.521741+0000","last_undegraded":"2023-10-09T06:32:31.521741+0000","last_fullsized":"2023-10-09T06:32:31.521741+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:55:19.695334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"62'1","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.518469+0000","last_change":"2023-10-09T06:32:30.511051+0000","last_active":"2023-10-09T06:32:31.518469+0000","last_peered":"2023-10-09T06:32:31.518469+0000","last_clean":"2023-10-09T06:32:31.518469+0000","last_became_active":"2023-10-09T06:32:30.510868+0000","last_became_peered":"2023-10-09T06:32:30.510868+0000","last_unstale":"2023-10-09T06:32:31.518469+0000","last_undegraded":"2023-10-09T06:32:31.518469+0000","last_fullsized":"2023-10-09T06:32:31.518469+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:07:24.928003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.525279+0000","last_change":"2023-10-09T06:32:31.518608+0000","last_active":"2023-10-09T06:32:31.525279+0000","last_peered":"2023-10-09T06:32:31.525279+0000","last_clean":"2023-10-09T06:32:31.525279+0000","last_became_active":"2023-10-09T06:32:31.516173+0000","last_became_peered":"2023-10-09T06:32:31.516173+0000","last_unstale":"2023-10-09T06:32:31.525279+0000","last_undegraded":"2023-10-09T06:32:31.525279+0000","last_fullsized":"2023-10-09T06:32:31.525279+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:36:58.416957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"62'2","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.529117+0000","last_change":"2023-10-09T06:32:30.512860+0000","last_active":"2023-10-09T06:32:31.529117+0000","last_peered":"2023-10-09T06:32:31.529117+0000","last_clean":"2023-10-09T06:32:31.529117+0000","last_became_active":"2023-10-09T06:32:30.512126+0000","last_became_peered":"2023-10-09T06:32:30.512126+0000","last_unstale":"2023-10-09T06:32:31.529117+0000","last_undegraded":"2023-10-09T06:32:31.529117+0000","last_fullsized":"2023-10-09T06:32:31.529117+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:56:07.601703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.526702+0000","last_change":"2023-10-09T06:32:31.521601+0000","last_active":"2023-10-09T06:32:31.526702+0000","last_peered":"2023-10-09T06:32:31.526702+0000","last_clean":"2023-10-09T06:32:31.526702+0000","last_became_active":"2023-10-09T06:32:31.520977+0000","last_became_peered":"2023-10-09T06:32:31.520977+0000","last_unstale":"2023-10-09T06:32:31.526702+0000","last_undegraded":"2023-10-09T06:32:31.526702+0000","last_fullsized":"2023-10-09T06:32:31.526702+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:51:14.139573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"62'2","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.528397+0000","last_change":"2023-10-09T06:32:30.511961+0000","last_active":"2023-10-09T06:32:31.528397+0000","last_peered":"2023-10-09T06:32:31.528397+0000","last_clean":"2023-10-09T06:32:31.528397+0000","last_became_active":"2023-10-09T06:32:30.511636+0000","last_became_peered":"2023-10-09T06:32:30.511636+0000","last_unstale":"2023-10-09T06:32:31.528397+0000","last_undegraded":"2023-10-09T06:32:31.528397+0000","last_fullsized":"2023-10-09T06:32:31.528397+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:59:33.319061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.521642+0000","last_change":"2023-10-09T06:32:31.517502+0000","last_active":"2023-10-09T06:32:31.521642+0000","last_peered":"2023-10-09T06:32:31.521642+0000","last_clean":"2023-10-09T06:32:31.521642+0000","last_became_active":"2023-10-09T06:32:31.517236+0000","last_became_peered":"2023-10-09T06:32:31.517236+0000","last_unstale":"2023-10-09T06:32:31.521642+0000","last_undegraded":"2023-10-09T06:32:31.521642+0000","last_fullsized":"2023-10-09T06:32:31.521642+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:30:47.845662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"62'1","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.529291+0000","last_change":"2023-10-09T06:32:30.509577+0000","last_active":"2023-10-09T06:32:31.529291+0000","last_peered":"2023-10-09T06:32:31.529291+0000","last_clean":"2023-10-09T06:32:31.529291+0000","last_became_active":"2023-10-09T06:32:30.509228+0000","last_became_peered":"2023-10-09T06:32:30.509228+0000","last_unstale":"2023-10-09T06:32:31.529291+0000","last_undegraded":"2023-10-09T06:32:31.529291+0000","last_fullsized":"2023-10-09T06:32:31.529291+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:32:16.720741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.521834+0000","last_change":"2023-10-09T06:32:31.511162+0000","last_active":"2023-10-09T06:32:31.521834+0000","last_peered":"2023-10-09T06:32:31.521834+0000","last_clean":"2023-10-09T06:32:31.521834+0000","last_became_active":"2023-10-09T06:32:31.510823+0000","last_became_peered":"2023-10-09T06:32:31.510823+0000","last_unstale":"2023-10-09T06:32:31.521834+0000","last_undegraded":"2023-10-09T06:32:31.521834+0000","last_fullsized":"2023-10-09T06:32:31.521834+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:59:23.257943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"62'1","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.520277+0000","last_change":"2023-10-09T06:32:30.511257+0000","last_active":"2023-10-09T06:32:31.520277+0000","last_peered":"2023-10-09T06:32:31.520277+0000","last_clean":"2023-10-09T06:32:31.520277+0000","last_became_active":"2023-10-09T06:32:30.510917+0000","last_became_peered":"2023-10-09T06:32:30.510917+0000","last_unstale":"2023-10-09T06:32:31.520277+0000","last_undegraded":"2023-10-09T06:32:31.520277+0000","last_fullsized":"2023-10-09T06:32:31.520277+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:54:00.527419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.524745+0000","last_change":"2023-10-09T06:32:31.516829+0000","last_active":"2023-10-09T06:32:31.524745+0000","last_peered":"2023-10-09T06:32:31.524745+0000","last_clean":"2023-10-09T06:32:31.524745+0000","last_became_active":"2023-10-09T06:32:31.516624+0000","last_became_peered":"2023-10-09T06:32:31.516624+0000","last_unstale":"2023-10-09T06:32:31.524745+0000","last_undegraded":"2023-10-09T06:32:31.524745+0000","last_fullsized":"2023-10-09T06:32:31.524745+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:15:30.917640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.501662+0000","last_change":"2023-10-09T06:32:30.508800+0000","last_active":"2023-10-09T06:32:31.501662+0000","last_peered":"2023-10-09T06:32:31.501662+0000","last_clean":"2023-10-09T06:32:31.501662+0000","last_became_active":"2023-10-09T06:32:30.508556+0000","last_became_peered":"2023-10-09T06:32:30.508556+0000","last_unstale":"2023-10-09T06:32:31.501662+0000","last_undegraded":"2023-10-09T06:32:31.501662+0000","last_fullsized":"2023-10-09T06:32:31.501662+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:08:35.442207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.520302+0000","last_change":"2023-10-09T06:32:31.517400+0000","last_active":"2023-10-09T06:32:31.520302+0000","last_peered":"2023-10-09T06:32:31.520302+0000","last_clean":"2023-10-09T06:32:31.520302+0000","last_became_active":"2023-10-09T06:32:31.517188+0000","last_became_peered":"2023-10-09T06:32:31.517188+0000","last_unstale":"2023-10-09T06:32:31.520302+0000","last_undegraded":"2023-10-09T06:32:31.520302+0000","last_fullsized":"2023-10-09T06:32:31.520302+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:49:04.586488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.505510+0000","last_change":"2023-10-09T06:32:30.508443+0000","last_active":"2023-10-09T06:32:31.505510+0000","last_peered":"2023-10-09T06:32:31.505510+0000","last_clean":"2023-10-09T06:32:31.505510+0000","last_became_active":"2023-10-09T06:32:30.508073+0000","last_became_peered":"2023-10-09T06:32:30.508073+0000","last_unstale":"2023-10-09T06:32:31.505510+0000","last_undegraded":"2023-10-09T06:32:31.505510+0000","last_fullsized":"2023-10-09T06:32:31.505510+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:34:19.640682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.516952+0000","last_change":"2023-10-09T06:32:31.513286+0000","last_active":"2023-10-09T06:32:31.516952+0000","last_peered":"2023-10-09T06:32:31.516952+0000","last_clean":"2023-10-09T06:32:31.516952+0000","last_became_active":"2023-10-09T06:32:31.513019+0000","last_became_peered":"2023-10-09T06:32:31.513019+0000","last_unstale":"2023-10-09T06:32:31.516952+0000","last_undegraded":"2023-10-09T06:32:31.516952+0000","last_fullsized":"2023-10-09T06:32:31.516952+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:21:07.366679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.506731+0000","last_change":"2023-10-09T06:32:30.510669+0000","last_active":"2023-10-09T06:32:31.506731+0000","last_peered":"2023-10-09T06:32:31.506731+0000","last_clean":"2023-10-09T06:32:31.506731+0000","last_became_active":"2023-10-09T06:32:30.510356+0000","last_became_peered":"2023-10-09T06:32:30.510356+0000","last_unstale":"2023-10-09T06:32:31.506731+0000","last_undegraded":"2023-10-09T06:32:31.506731+0000","last_fullsized":"2023-10-09T06:32:31.506731+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:37:23.971604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.526854+0000","last_change":"2023-10-09T06:32:31.522871+0000","last_active":"2023-10-09T06:32:31.526854+0000","last_peered":"2023-10-09T06:32:31.526854+0000","last_clean":"2023-10-09T06:32:31.526854+0000","last_became_active":"2023-10-09T06:32:31.522597+0000","last_became_peered":"2023-10-09T06:32:31.522597+0000","last_unstale":"2023-10-09T06:32:31.526854+0000","last_undegraded":"2023-10-09T06:32:31.526854+0000","last_fullsized":"2023-10-09T06:32:31.526854+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:22:19.984152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.501373+0000","last_change":"2023-10-09T06:32:30.512173+0000","last_active":"2023-10-09T06:32:31.501373+0000","last_peered":"2023-10-09T06:32:31.501373+0000","last_clean":"2023-10-09T06:32:31.501373+0000","last_became_active":"2023-10-09T06:32:30.511876+0000","last_became_peered":"2023-10-09T06:32:30.511876+0000","last_unstale":"2023-10-09T06:32:31.501373+0000","last_undegraded":"2023-10-09T06:32:31.501373+0000","last_fullsized":"2023-10-09T06:32:31.501373+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:22:34.707397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.512098+0000","last_change":"2023-10-09T06:32:31.509905+0000","last_active":"2023-10-09T06:32:31.512098+0000","last_peered":"2023-10-09T06:32:31.512098+0000","last_clean":"2023-10-09T06:32:31.512098+0000","last_became_active":"2023-10-09T06:32:31.509045+0000","last_became_peered":"2023-10-09T06:32:31.509045+0000","last_unstale":"2023-10-09T06:32:31.512098+0000","last_undegraded":"2023-10-09T06:32:31.512098+0000","last_fullsized":"2023-10-09T06:32:31.512098+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:37:57.966936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"62'3","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.523685+0000","last_change":"2023-10-09T06:32:30.512291+0000","last_active":"2023-10-09T06:32:31.523685+0000","last_peered":"2023-10-09T06:32:31.523685+0000","last_clean":"2023-10-09T06:32:31.523685+0000","last_became_active":"2023-10-09T06:32:30.512051+0000","last_became_peered":"2023-10-09T06:32:30.512051+0000","last_unstale":"2023-10-09T06:32:31.523685+0000","last_undegraded":"2023-10-09T06:32:31.523685+0000","last_fullsized":"2023-10-09T06:32:31.523685+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:22:18.981897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.525244+0000","last_change":"2023-10-09T06:32:31.518519+0000","last_active":"2023-10-09T06:32:31.525244+0000","last_peered":"2023-10-09T06:32:31.525244+0000","last_clean":"2023-10-09T06:32:31.525244+0000","last_became_active":"2023-10-09T06:32:31.518259+0000","last_became_peered":"2023-10-09T06:32:31.518259+0000","last_unstale":"2023-10-09T06:32:31.525244+0000","last_undegraded":"2023-10-09T06:32:31.525244+0000","last_fullsized":"2023-10-09T06:32:31.525244+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:25:13.501296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"62'6","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.533817+0000","last_change":"2023-10-09T06:32:30.510937+0000","last_active":"2023-10-09T06:32:31.533817+0000","last_peered":"2023-10-09T06:32:31.533817+0000","last_clean":"2023-10-09T06:32:31.533817+0000","last_became_active":"2023-10-09T06:32:30.510651+0000","last_became_peered":"2023-10-09T06:32:30.510651+0000","last_unstale":"2023-10-09T06:32:31.533817+0000","last_undegraded":"2023-10-09T06:32:31.533817+0000","last_fullsized":"2023-10-09T06:32:31.533817+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:56:07.601703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.527185+0000","last_change":"2023-10-09T06:32:31.525788+0000","last_active":"2023-10-09T06:32:31.527185+0000","last_peered":"2023-10-09T06:32:31.527185+0000","last_clean":"2023-10-09T06:32:31.527185+0000","last_became_active":"2023-10-09T06:32:31.525526+0000","last_became_peered":"2023-10-09T06:32:31.525526+0000","last_unstale":"2023-10-09T06:32:31.527185+0000","last_undegraded":"2023-10-09T06:32:31.527185+0000","last_fullsized":"2023-10-09T06:32:31.527185+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:23:42.065325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"62'2","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.523162+0000","last_change":"2023-10-09T06:32:30.513665+0000","last_active":"2023-10-09T06:32:31.523162+0000","last_peered":"2023-10-09T06:32:31.523162+0000","last_clean":"2023-10-09T06:32:31.523162+0000","last_became_active":"2023-10-09T06:32:30.513385+0000","last_became_peered":"2023-10-09T06:32:30.513385+0000","last_unstale":"2023-10-09T06:32:31.523162+0000","last_undegraded":"2023-10-09T06:32:31.523162+0000","last_fullsized":"2023-10-09T06:32:31.523162+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:05:29.788926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.512174+0000","last_change":"2023-10-09T06:32:31.509930+0000","last_active":"2023-10-09T06:32:31.512174+0000","last_peered":"2023-10-09T06:32:31.512174+0000","last_clean":"2023-10-09T06:32:31.512174+0000","last_became_active":"2023-10-09T06:32:31.509413+0000","last_became_peered":"2023-10-09T06:32:31.509413+0000","last_unstale":"2023-10-09T06:32:31.512174+0000","last_undegraded":"2023-10-09T06:32:31.512174+0000","last_fullsized":"2023-10-09T06:32:31.512174+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:48:07.062498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.502176+0000","last_change":"2023-10-09T06:32:30.514316+0000","last_active":"2023-10-09T06:32:31.502176+0000","last_peered":"2023-10-09T06:32:31.502176+0000","last_clean":"2023-10-09T06:32:31.502176+0000","last_became_active":"2023-10-09T06:32:30.513987+0000","last_became_peered":"2023-10-09T06:32:30.513987+0000","last_unstale":"2023-10-09T06:32:31.502176+0000","last_undegraded":"2023-10-09T06:32:31.502176+0000","last_fullsized":"2023-10-09T06:32:31.502176+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:39:36.054325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.516768+0000","last_change":"2023-10-09T06:32:31.512769+0000","last_active":"2023-10-09T06:32:31.516768+0000","last_peered":"2023-10-09T06:32:31.516768+0000","last_clean":"2023-10-09T06:32:31.516768+0000","last_became_active":"2023-10-09T06:32:31.512524+0000","last_became_peered":"2023-10-09T06:32:31.512524+0000","last_unstale":"2023-10-09T06:32:31.516768+0000","last_undegraded":"2023-10-09T06:32:31.516768+0000","last_fullsized":"2023-10-09T06:32:31.516768+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:57:17.839161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.500386+0000","last_change":"2023-10-09T06:32:30.514034+0000","last_active":"2023-10-09T06:32:31.500386+0000","last_peered":"2023-10-09T06:32:31.500386+0000","last_clean":"2023-10-09T06:32:31.500386+0000","last_became_active":"2023-10-09T06:32:30.513765+0000","last_became_peered":"2023-10-09T06:32:30.513765+0000","last_unstale":"2023-10-09T06:32:31.500386+0000","last_undegraded":"2023-10-09T06:32:31.500386+0000","last_fullsized":"2023-10-09T06:32:31.500386+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:07:08.905652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.524799+0000","last_change":"2023-10-09T06:32:31.516916+0000","last_active":"2023-10-09T06:32:31.524799+0000","last_peered":"2023-10-09T06:32:31.524799+0000","last_clean":"2023-10-09T06:32:31.524799+0000","last_became_active":"2023-10-09T06:32:31.515173+0000","last_became_peered":"2023-10-09T06:32:31.515173+0000","last_unstale":"2023-10-09T06:32:31.524799+0000","last_undegraded":"2023-10-09T06:32:31.524799+0000","last_fullsized":"2023-10-09T06:32:31.524799+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:24:05.777223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"62'8","reported_seq":18,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.529659+0000","last_change":"2023-10-09T06:32:30.515315+0000","last_active":"2023-10-09T06:32:31.529659+0000","last_peered":"2023-10-09T06:32:31.529659+0000","last_clean":"2023-10-09T06:32:31.529659+0000","last_became_active":"2023-10-09T06:32:30.513454+0000","last_became_peered":"2023-10-09T06:32:30.513454+0000","last_unstale":"2023-10-09T06:32:31.529659+0000","last_undegraded":"2023-10-09T06:32:31.529659+0000","last_fullsized":"2023-10-09T06:32:31.529659+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:08:35.442207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.528934+0000","last_change":"2023-10-09T06:32:31.513564+0000","last_active":"2023-10-09T06:32:31.528934+0000","last_peered":"2023-10-09T06:32:31.528934+0000","last_clean":"2023-10-09T06:32:31.528934+0000","last_became_active":"2023-10-09T06:32:31.511889+0000","last_became_peered":"2023-10-09T06:32:31.511889+0000","last_unstale":"2023-10-09T06:32:31.528934+0000","last_undegraded":"2023-10-09T06:32:31.528934+0000","last_fullsized":"2023-10-09T06:32:31.528934+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:10:02.013724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"62'1","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.526651+0000","last_change":"2023-10-09T06:32:30.511763+0000","last_active":"2023-10-09T06:32:31.526651+0000","last_peered":"2023-10-09T06:32:31.526651+0000","last_clean":"2023-10-09T06:32:31.526651+0000","last_became_active":"2023-10-09T06:32:30.510441+0000","last_became_peered":"2023-10-09T06:32:30.510441+0000","last_unstale":"2023-10-09T06:32:31.526651+0000","last_undegraded":"2023-10-09T06:32:31.526651+0000","last_fullsized":"2023-10-09T06:32:31.526651+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:24:24.863354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.527139+0000","last_change":"2023-10-09T06:32:31.524535+0000","last_active":"2023-10-09T06:32:31.527139+0000","last_peered":"2023-10-09T06:32:31.527139+0000","last_clean":"2023-10-09T06:32:31.527139+0000","last_became_active":"2023-10-09T06:32:31.524298+0000","last_became_peered":"2023-10-09T06:32:31.524298+0000","last_unstale":"2023-10-09T06:32:31.527139+0000","last_undegraded":"2023-10-09T06:32:31.527139+0000","last_fullsized":"2023-10-09T06:32:31.527139+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:21:07.366679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"62'2","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.529096+0000","last_change":"2023-10-09T06:32:30.516087+0000","last_active":"2023-10-09T06:32:31.529096+0000","last_peered":"2023-10-09T06:32:31.529096+0000","last_clean":"2023-10-09T06:32:31.529096+0000","last_became_active":"2023-10-09T06:32:30.511897+0000","last_became_peered":"2023-10-09T06:32:30.511897+0000","last_unstale":"2023-10-09T06:32:31.529096+0000","last_undegraded":"2023-10-09T06:32:31.529096+0000","last_fullsized":"2023-10-09T06:32:31.529096+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:31:31.960774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.528175+0000","last_change":"2023-10-09T06:32:31.517808+0000","last_active":"2023-10-09T06:32:31.528175+0000","last_peered":"2023-10-09T06:32:31.528175+0000","last_clean":"2023-10-09T06:32:31.528175+0000","last_became_active":"2023-10-09T06:32:31.517108+0000","last_became_peered":"2023-10-09T06:32:31.517108+0000","last_unstale":"2023-10-09T06:32:31.528175+0000","last_undegraded":"2023-10-09T06:32:31.528175+0000","last_fullsized":"2023-10-09T06:32:31.528175+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:36:58.416957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"62'1","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.529393+0000","last_change":"2023-10-09T06:32:30.509241+0000","last_active":"2023-10-09T06:32:31.529393+0000","last_peered":"2023-10-09T06:32:31.529393+0000","last_clean":"2023-10-09T06:32:31.529393+0000","last_became_active":"2023-10-09T06:32:30.508842+0000","last_became_peered":"2023-10-09T06:32:30.508842+0000","last_unstale":"2023-10-09T06:32:31.529393+0000","last_undegraded":"2023-10-09T06:32:31.529393+0000","last_fullsized":"2023-10-09T06:32:31.529393+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:25:12.499041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.524721+0000","last_change":"2023-10-09T06:32:31.516415+0000","last_active":"2023-10-09T06:32:31.524721+0000","last_peered":"2023-10-09T06:32:31.524721+0000","last_clean":"2023-10-09T06:32:31.524721+0000","last_became_active":"2023-10-09T06:32:31.516218+0000","last_became_peered":"2023-10-09T06:32:31.516218+0000","last_unstale":"2023-10-09T06:32:31.524721+0000","last_undegraded":"2023-10-09T06:32:31.524721+0000","last_fullsized":"2023-10-09T06:32:31.524721+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:34:05.882099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.509206+0000","last_change":"2023-10-09T06:32:30.514856+0000","last_active":"2023-10-09T06:32:31.509206+0000","last_peered":"2023-10-09T06:32:31.509206+0000","last_clean":"2023-10-09T06:32:31.509206+0000","last_became_active":"2023-10-09T06:32:30.514570+0000","last_became_peered":"2023-10-09T06:32:30.514570+0000","last_unstale":"2023-10-09T06:32:31.509206+0000","last_undegraded":"2023-10-09T06:32:31.509206+0000","last_fullsized":"2023-10-09T06:32:31.509206+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:39:46.605461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.525515+0000","last_change":"2023-10-09T06:32:31.523624+0000","last_active":"2023-10-09T06:32:31.525515+0000","last_peered":"2023-10-09T06:32:31.525515+0000","last_clean":"2023-10-09T06:32:31.525515+0000","last_became_active":"2023-10-09T06:32:31.523370+0000","last_became_peered":"2023-10-09T06:32:31.523370+0000","last_unstale":"2023-10-09T06:32:31.525515+0000","last_undegraded":"2023-10-09T06:32:31.525515+0000","last_fullsized":"2023-10-09T06:32:31.525515+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:52:41.647855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.502266+0000","last_change":"2023-10-09T06:32:30.511452+0000","last_active":"2023-10-09T06:32:31.502266+0000","last_peered":"2023-10-09T06:32:31.502266+0000","last_clean":"2023-10-09T06:32:31.502266+0000","last_became_active":"2023-10-09T06:32:30.511160+0000","last_became_peered":"2023-10-09T06:32:30.511160+0000","last_unstale":"2023-10-09T06:32:31.502266+0000","last_undegraded":"2023-10-09T06:32:31.502266+0000","last_fullsized":"2023-10-09T06:32:31.502266+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:00:45.173974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.525454+0000","last_change":"2023-10-09T06:32:31.523321+0000","last_active":"2023-10-09T06:32:31.525454+0000","last_peered":"2023-10-09T06:32:31.525454+0000","last_clean":"2023-10-09T06:32:31.525454+0000","last_became_active":"2023-10-09T06:32:31.523070+0000","last_became_peered":"2023-10-09T06:32:31.523070+0000","last_unstale":"2023-10-09T06:32:31.525454+0000","last_undegraded":"2023-10-09T06:32:31.525454+0000","last_fullsized":"2023-10-09T06:32:31.525454+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:55:30.195305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.500491+0000","last_change":"2023-10-09T06:32:30.514925+0000","last_active":"2023-10-09T06:32:31.500491+0000","last_peered":"2023-10-09T06:32:31.500491+0000","last_clean":"2023-10-09T06:32:31.500491+0000","last_became_active":"2023-10-09T06:32:30.514621+0000","last_became_peered":"2023-10-09T06:32:30.514621+0000","last_unstale":"2023-10-09T06:32:31.500491+0000","last_undegraded":"2023-10-09T06:32:31.500491+0000","last_fullsized":"2023-10-09T06:32:31.500491+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:10:01.011469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.525536+0000","last_change":"2023-10-09T06:32:31.523106+0000","last_active":"2023-10-09T06:32:31.525536+0000","last_peered":"2023-10-09T06:32:31.525536+0000","last_clean":"2023-10-09T06:32:31.525536+0000","last_became_active":"2023-10-09T06:32:31.522837+0000","last_became_peered":"2023-10-09T06:32:31.522837+0000","last_unstale":"2023-10-09T06:32:31.525536+0000","last_undegraded":"2023-10-09T06:32:31.525536+0000","last_fullsized":"2023-10-09T06:32:31.525536+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:06:23.253040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.505980+0000","last_change":"2023-10-09T06:32:30.513984+0000","last_active":"2023-10-09T06:32:31.505980+0000","last_peered":"2023-10-09T06:32:31.505980+0000","last_clean":"2023-10-09T06:32:31.505980+0000","last_became_active":"2023-10-09T06:32:30.513743+0000","last_became_peered":"2023-10-09T06:32:30.513743+0000","last_unstale":"2023-10-09T06:32:31.505980+0000","last_undegraded":"2023-10-09T06:32:31.505980+0000","last_fullsized":"2023-10-09T06:32:31.505980+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:09:26.237888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.520336+0000","last_change":"2023-10-09T06:32:31.519388+0000","last_active":"2023-10-09T06:32:31.520336+0000","last_peered":"2023-10-09T06:32:31.520336+0000","last_clean":"2023-10-09T06:32:31.520336+0000","last_became_active":"2023-10-09T06:32:31.519126+0000","last_became_peered":"2023-10-09T06:32:31.519126+0000","last_unstale":"2023-10-09T06:32:31.520336+0000","last_undegraded":"2023-10-09T06:32:31.520336+0000","last_fullsized":"2023-10-09T06:32:31.520336+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:18:48.881835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"62'2","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.526529+0000","last_change":"2023-10-09T06:32:30.510835+0000","last_active":"2023-10-09T06:32:31.526529+0000","last_peered":"2023-10-09T06:32:31.526529+0000","last_clean":"2023-10-09T06:32:31.526529+0000","last_became_active":"2023-10-09T06:32:30.510376+0000","last_became_peered":"2023-10-09T06:32:30.510376+0000","last_unstale":"2023-10-09T06:32:31.526529+0000","last_undegraded":"2023-10-09T06:32:31.526529+0000","last_fullsized":"2023-10-09T06:32:31.526529+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:52:29.200026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.508275+0000","last_change":"2023-10-09T06:32:31.506492+0000","last_active":"2023-10-09T06:32:31.508275+0000","last_peered":"2023-10-09T06:32:31.508275+0000","last_clean":"2023-10-09T06:32:31.508275+0000","last_became_active":"2023-10-09T06:32:31.506014+0000","last_became_peered":"2023-10-09T06:32:31.506014+0000","last_unstale":"2023-10-09T06:32:31.508275+0000","last_undegraded":"2023-10-09T06:32:31.508275+0000","last_fullsized":"2023-10-09T06:32:31.508275+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:07:09.728724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.501106+0000","last_change":"2023-10-09T06:32:30.513921+0000","last_active":"2023-10-09T06:32:31.501106+0000","last_peered":"2023-10-09T06:32:31.501106+0000","last_clean":"2023-10-09T06:32:31.501106+0000","last_became_active":"2023-10-09T06:32:30.513649+0000","last_became_peered":"2023-10-09T06:32:30.513649+0000","last_unstale":"2023-10-09T06:32:31.501106+0000","last_undegraded":"2023-10-09T06:32:31.501106+0000","last_fullsized":"2023-10-09T06:32:31.501106+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:07:24.928003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.526850+0000","last_change":"2023-10-09T06:32:31.522803+0000","last_active":"2023-10-09T06:32:31.526850+0000","last_peered":"2023-10-09T06:32:31.526850+0000","last_clean":"2023-10-09T06:32:31.526850+0000","last_became_active":"2023-10-09T06:32:31.522416+0000","last_became_peered":"2023-10-09T06:32:31.522416+0000","last_unstale":"2023-10-09T06:32:31.526850+0000","last_undegraded":"2023-10-09T06:32:31.526850+0000","last_fullsized":"2023-10-09T06:32:31.526850+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:07:09.728724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.507650+0000","last_change":"2023-10-09T06:32:30.506664+0000","last_active":"2023-10-09T06:32:31.507650+0000","last_peered":"2023-10-09T06:32:31.507650+0000","last_clean":"2023-10-09T06:32:31.507650+0000","last_became_active":"2023-10-09T06:32:30.506367+0000","last_became_peered":"2023-10-09T06:32:30.506367+0000","last_unstale":"2023-10-09T06:32:31.507650+0000","last_undegraded":"2023-10-09T06:32:31.507650+0000","last_fullsized":"2023-10-09T06:32:31.507650+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:25:13.089891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.523698+0000","last_change":"2023-10-09T06:32:31.520372+0000","last_active":"2023-10-09T06:32:31.523698+0000","last_peered":"2023-10-09T06:32:31.523698+0000","last_clean":"2023-10-09T06:32:31.523698+0000","last_became_active":"2023-10-09T06:32:31.520059+0000","last_became_peered":"2023-10-09T06:32:31.520059+0000","last_unstale":"2023-10-09T06:32:31.523698+0000","last_undegraded":"2023-10-09T06:32:31.523698+0000","last_fullsized":"2023-10-09T06:32:31.523698+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:17:52.054195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.506496+0000","last_change":"2023-10-09T06:32:30.509435+0000","last_active":"2023-10-09T06:32:31.506496+0000","last_peered":"2023-10-09T06:32:31.506496+0000","last_clean":"2023-10-09T06:32:31.506496+0000","last_became_active":"2023-10-09T06:32:30.509159+0000","last_became_peered":"2023-10-09T06:32:30.509159+0000","last_unstale":"2023-10-09T06:32:31.506496+0000","last_undegraded":"2023-10-09T06:32:31.506496+0000","last_fullsized":"2023-10-09T06:32:31.506496+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:38:53.216443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.525158+0000","last_change":"2023-10-09T06:32:31.517724+0000","last_active":"2023-10-09T06:32:31.525158+0000","last_peered":"2023-10-09T06:32:31.525158+0000","last_clean":"2023-10-09T06:32:31.525158+0000","last_became_active":"2023-10-09T06:32:31.514920+0000","last_became_peered":"2023-10-09T06:32:31.514920+0000","last_unstale":"2023-10-09T06:32:31.525158+0000","last_undegraded":"2023-10-09T06:32:31.525158+0000","last_fullsized":"2023-10-09T06:32:31.525158+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:02:04.881485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.507541+0000","last_change":"2023-10-09T06:32:30.516768+0000","last_active":"2023-10-09T06:32:31.507541+0000","last_peered":"2023-10-09T06:32:31.507541+0000","last_clean":"2023-10-09T06:32:31.507541+0000","last_became_active":"2023-10-09T06:32:30.516451+0000","last_became_peered":"2023-10-09T06:32:30.516451+0000","last_unstale":"2023-10-09T06:32:31.507541+0000","last_undegraded":"2023-10-09T06:32:31.507541+0000","last_fullsized":"2023-10-09T06:32:31.507541+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:07:19.969025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.525400+0000","last_change":"2023-10-09T06:32:31.520802+0000","last_active":"2023-10-09T06:32:31.525400+0000","last_peered":"2023-10-09T06:32:31.525400+0000","last_clean":"2023-10-09T06:32:31.525400+0000","last_became_active":"2023-10-09T06:32:31.520546+0000","last_became_peered":"2023-10-09T06:32:31.520546+0000","last_unstale":"2023-10-09T06:32:31.525400+0000","last_undegraded":"2023-10-09T06:32:31.525400+0000","last_fullsized":"2023-10-09T06:32:31.525400+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:09:27.240143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.501320+0000","last_change":"2023-10-09T06:32:30.509591+0000","last_active":"2023-10-09T06:32:31.501320+0000","last_peered":"2023-10-09T06:32:31.501320+0000","last_clean":"2023-10-09T06:32:31.501320+0000","last_became_active":"2023-10-09T06:32:30.509314+0000","last_became_peered":"2023-10-09T06:32:30.509314+0000","last_unstale":"2023-10-09T06:32:31.501320+0000","last_undegraded":"2023-10-09T06:32:31.501320+0000","last_fullsized":"2023-10-09T06:32:31.501320+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:52:55.778655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.523596+0000","last_change":"2023-10-09T06:32:31.511717+0000","last_active":"2023-10-09T06:32:31.523596+0000","last_peered":"2023-10-09T06:32:31.523596+0000","last_clean":"2023-10-09T06:32:31.523596+0000","last_became_active":"2023-10-09T06:32:31.511374+0000","last_became_peered":"2023-10-09T06:32:31.511374+0000","last_unstale":"2023-10-09T06:32:31.523596+0000","last_undegraded":"2023-10-09T06:32:31.523596+0000","last_fullsized":"2023-10-09T06:32:31.523596+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:38:54.218698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.505884+0000","last_change":"2023-10-09T06:32:30.515328+0000","last_active":"2023-10-09T06:32:31.505884+0000","last_peered":"2023-10-09T06:32:31.505884+0000","last_clean":"2023-10-09T06:32:31.505884+0000","last_became_active":"2023-10-09T06:32:30.513481+0000","last_became_peered":"2023-10-09T06:32:30.513481+0000","last_unstale":"2023-10-09T06:32:31.505884+0000","last_undegraded":"2023-10-09T06:32:31.505884+0000","last_fullsized":"2023-10-09T06:32:31.505884+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:05:29.788926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.523909+0000","last_change":"2023-10-09T06:32:31.520908+0000","last_active":"2023-10-09T06:32:31.523909+0000","last_peered":"2023-10-09T06:32:31.523909+0000","last_clean":"2023-10-09T06:32:31.523909+0000","last_became_active":"2023-10-09T06:32:31.520623+0000","last_became_peered":"2023-10-09T06:32:31.520623+0000","last_unstale":"2023-10-09T06:32:31.523909+0000","last_undegraded":"2023-10-09T06:32:31.523909+0000","last_fullsized":"2023-10-09T06:32:31.523909+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:04:39.865936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.501217+0000","last_change":"2023-10-09T06:32:30.509898+0000","last_active":"2023-10-09T06:32:31.501217+0000","last_peered":"2023-10-09T06:32:31.501217+0000","last_clean":"2023-10-09T06:32:31.501217+0000","last_became_active":"2023-10-09T06:32:30.509639+0000","last_became_peered":"2023-10-09T06:32:30.509639+0000","last_unstale":"2023-10-09T06:32:31.501217+0000","last_undegraded":"2023-10-09T06:32:31.501217+0000","last_fullsized":"2023-10-09T06:32:31.501217+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:24:42.834801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.525465+0000","last_change":"2023-10-09T06:32:31.523514+0000","last_active":"2023-10-09T06:32:31.525465+0000","last_peered":"2023-10-09T06:32:31.525465+0000","last_clean":"2023-10-09T06:32:31.525465+0000","last_became_active":"2023-10-09T06:32:31.523349+0000","last_became_peered":"2023-10-09T06:32:31.523349+0000","last_unstale":"2023-10-09T06:32:31.525465+0000","last_undegraded":"2023-10-09T06:32:31.525465+0000","last_fullsized":"2023-10-09T06:32:31.525465+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:11:52.630319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"62'2","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.525433+0000","last_change":"2023-10-09T06:32:30.512857+0000","last_active":"2023-10-09T06:32:31.525433+0000","last_peered":"2023-10-09T06:32:31.525433+0000","last_clean":"2023-10-09T06:32:31.525433+0000","last_became_active":"2023-10-09T06:32:30.512646+0000","last_became_peered":"2023-10-09T06:32:30.512646+0000","last_unstale":"2023-10-09T06:32:31.525433+0000","last_undegraded":"2023-10-09T06:32:31.525433+0000","last_fullsized":"2023-10-09T06:32:31.525433+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:59:45.594900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":200,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.508856+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:32:31.508856+0000","last_peered":"2023-10-09T06:32:31.508856+0000","last_clean":"2023-10-09T06:32:31.508856+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:32:31.508856+0000","last_undegraded":"2023-10-09T06:32:31.508856+0000","last_fullsized":"2023-10-09T06:32:31.508856+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298624,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073216,"statfs":{"total":96636764160,"available":96330973184,"internally_reserved":0,"allocated":1368064,"data_stored":918249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.501,"15min":0.501},"min":{"1min":0.36099999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.64600000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.53600000000000003}]},{"osd":1,"last update":"Mon Oct 9 06:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.65500000000000003,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52700000000000002,"15min":0.52500000000000002},"min":{"1min":0.39000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.69199999999999995,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.61399999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.50900000000000001,"15min":0.505},"min":{"1min":0.41099999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.65600000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.58099999999999996}]},{"osd":3,"last update":"Mon Oct 9 06:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.55100000000000005,"15min":0.55800000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.68700000000000006,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.50900000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.52900000000000003,"15min":0.53900000000000003},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.626,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.622,"15min":0.628},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.70199999999999996,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.60199999999999998}]},{"osd":5,"last update":"Mon Oct 9 06:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.58299999999999996,"15min":0.59499999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.72999999999999998,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.626,"15min":0.63400000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71899999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.67900000000000005}]},{"osd":6,"last update":"Mon Oct 9 06:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.60199999999999998,"15min":0.61499999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71399999999999997,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.63100000000000001,"15min":0.64400000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.77800000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.63100000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298044,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073796,"statfs":{"total":96636764160,"available":96331567104,"internally_reserved":0,"allocated":774144,"data_stored":327385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.496,"15min":0.501},"min":{"1min":0.33200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.61599999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.51300000000000001,"15min":0.51100000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.66300000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.47699999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.51300000000000001,"15min":0.51900000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.59899999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.61399999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.58399999999999996}]},{"osd":2,"last update":"Mon Oct 9 06:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51300000000000001,"15min":0.51600000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.65100000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69399999999999995,"5min":0.752,"15min":0.752},"last":0.52500000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.54300000000000004,"15min":0.55100000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78400000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.73899999999999999,"15min":0.76700000000000002},"min":{"1min":0.42199999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.74399999999999999,"5min":4.7510000000000003,"15min":4.7510000000000003},"last":0.61799999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.48199999999999998,"15min":0.48499999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70599999999999996,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.40100000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.73599999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53200000000000003,"15min":0.53500000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.67900000000000005,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.60199999999999998,"15min":0.60799999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.73699999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.68300000000000005}]},{"osd":7,"last update":"Mon Oct 9 06:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.58899999999999997,"15min":0.59899999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.68899999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.46400000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.71499999999999997,"5min":0.745,"15min":0.745},"last":0.72099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298056,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073784,"statfs":{"total":96636764160,"available":96331554816,"internally_reserved":0,"allocated":786432,"data_stored":329417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.307,"15min":0.309},"min":{"1min":0.215,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.40100000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.36099999999999999,"15min":0.36399999999999999},"min":{"1min":0.253,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.40100000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.32800000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.35199999999999998,"15min":0.35399999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.379},{"interface":"front","average":{"1min":0.376,"5min":0.38500000000000001,"15min":0.38700000000000001},"min":{"1min":0.29299999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.47699999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.39100000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.26100000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.59799999999999998,"15min":0.64000000000000001},"min":{"1min":0.23799999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.53000000000000003,"5min":5.8529999999999998,"15min":5.8529999999999998},"last":0.34799999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56899999999999995,"15min":0.57199999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79400000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.46500000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.72099999999999997,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.59399999999999997}]},{"osd":5,"last update":"Mon Oct 9 06:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.79100000000000004,"15min":0.83499999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.70899999999999996,"5min":5.9459999999999997,"15min":5.9459999999999997},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.60499999999999998,"15min":0.61299999999999999},"min":{"1min":0.45000000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71199999999999997,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.56499999999999995}]},{"osd":6,"last update":"Mon Oct 9 06:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.76300000000000001,"15min":0.80400000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.69199999999999995,"5min":5.9160000000000004,"15min":5.9160000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.75600000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.68300000000000005}]},{"osd":7,"last update":"Mon Oct 9 06:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.58599999999999997,"15min":0.59499999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.72799999999999998,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.80300000000000005,"15min":0.84999999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.74199999999999999,"5min":5.9729999999999999,"15min":5.9729999999999999},"last":0.64300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298616,"kb_used_data":1328,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073224,"statfs":{"total":96636764160,"available":96330981376,"internally_reserved":0,"allocated":1359872,"data_stored":917661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.35999999999999999,"15min":0.35399999999999998},"min":{"1min":0.24299999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.28699999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.36699999999999999,"15min":0.36799999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54500000000000004,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.39900000000000002,"15min":0.39500000000000002},"min":{"1min":0.307,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.36899999999999999}]},{"osd":3,"last update":"Mon Oct 9 06:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.40699999999999997,"15min":0.41399999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.52900000000000003,"5min":1.554,"15min":1.554},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.41399999999999998,"15min":0.41599999999999998},"min":{"1min":0.28399999999999997,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.60899999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.34899999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56499999999999995,"15min":0.55600000000000005},"min":{"1min":0.39300000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.77600000000000002,"15min":0.81299999999999994},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81799999999999995,"5min":5.484,"15min":5.484},"last":0.55600000000000005}]},{"osd":5,"last update":"Mon Oct 9 06:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.78500000000000003,"15min":0.81899999999999995},"min":{"1min":0.40600000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84399999999999997,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.98699999999999999,"5min":1.113,"15min":1.113},"last":0.47799999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.46999999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0269999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.61899999999999999,"15min":0.61199999999999999},"min":{"1min":0.41799999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.54500000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.434,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.81299999999999994,"15min":0.85099999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.873,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.48999999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298048,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073792,"statfs":{"total":96636764160,"available":96331563008,"internally_reserved":0,"allocated":778240,"data_stored":328829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":7.2699999999999996,"5min":7.2699999999999996,"15min":7.2699999999999996},"last":0.34399999999999997}]},{"osd":1,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":7.327,"5min":7.327,"15min":7.327},"last":0.25800000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.314}]},{"osd":4,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":7.2389999999999999,"5min":7.2389999999999999,"15min":7.2389999999999999},"last":0.47399999999999998}]},{"osd":5,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":10.890000000000001,"5min":10.890000000000001,"15min":10.890000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52100000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":7.2999999999999998,"5min":7.2999999999999998,"15min":7.2999999999999998},"last":0.627},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.54700000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":7.1879999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.48299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298044,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073796,"statfs":{"total":96636764160,"available":96331567104,"internally_reserved":0,"allocated":774144,"data_stored":327405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.36099999999999999,"15min":0.32900000000000001},"min":{"1min":0.245,"5min":0.22,"15min":0.22},"max":{"1min":5.8700000000000001,"5min":5.8700000000000001,"15min":5.8700000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.39700000000000002,"15min":0.37},"min":{"1min":0.249,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":5.6769999999999996,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.32700000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.377,"15min":0.35699999999999998},"min":{"1min":0.255,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":4.0469999999999997,"5min":4.0469999999999997,"15min":4.0469999999999997},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.41199999999999998,"15min":0.39100000000000001},"min":{"1min":0.29799999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":4.1509999999999998,"5min":4.1509999999999998,"15min":4.1509999999999998},"last":0.38300000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.39500000000000002,"15min":0.38100000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":3.1139999999999999,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.40200000000000002,"15min":0.39000000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":3.3180000000000001,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.307}]},{"osd":4,"last update":"Mon Oct 9 06:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.58999999999999997,"15min":0.57399999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.0899999999999999,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":0.432},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.58099999999999996,"15min":0.56499999999999995},"min":{"1min":0.39300000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":3.0710000000000002,"5min":3.0710000000000002,"15min":3.0710000000000002},"last":0.46300000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.61899999999999999,"15min":0.58899999999999997},"min":{"1min":0.42499999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":6.2300000000000004,"5min":6.2300000000000004,"15min":6.2300000000000004},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":3.0219999999999998,"5min":3.0219999999999998,"15min":3.0219999999999998},"last":0.50700000000000001}]},{"osd":6,"last update":"Mon Oct 9 06:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":2.6850000000000001,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.88,"5min":0.68200000000000005,"15min":0.64900000000000002},"min":{"1min":0.443,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":6.5970000000000004,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.61099999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.63500000000000001,"15min":0.61099999999999999},"min":{"1min":0.47799999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":4.2629999999999999,"5min":4.2629999999999999,"15min":4.2629999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.872,"5min":0.64800000000000002,"15min":0.61099999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":7.3650000000000002,"5min":7.3650000000000002,"15min":7.3650000000000002},"last":0.57999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298040,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073800,"statfs":{"total":96636764160,"available":96331571200,"internally_reserved":0,"allocated":770048,"data_stored":327363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73499999999999999,"5min":1.083,"15min":1.083},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69999999999999996,"5min":1.054,"15min":1.054},"last":0.47299999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48899999999999999,"15min":0.48799999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.499}]},{"osd":2,"last update":"Mon Oct 9 06:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.54100000000000004,"15min":0.54700000000000004},"min":{"1min":0.33700000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70699999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.60499999999999998,"15min":0.60999999999999999},"min":{"1min":0.443,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.78100000000000003,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.63900000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55600000000000005,"15min":0.56200000000000006},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.746,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.58899999999999997,"15min":0.59499999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.749,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.61399999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.55700000000000005,"15min":0.57199999999999995},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.74299999999999999,"5min":0.998,"15min":0.998},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.61099999999999999,"15min":0.621},"min":{"1min":0.39200000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76700000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.72999999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.32500000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.78600000000000003,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60799999999999998,"15min":0.61399999999999999},"min":{"1min":0.33500000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79400000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.745}]},{"osd":7,"last update":"Mon Oct 9 06:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.57799999999999996,"15min":0.58499999999999996},"min":{"1min":0.34599999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77100000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.65500000000000003,"15min":0.66700000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80000000000000004,"5min":1.175,"15min":1.175},"last":0.71499999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298040,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073800,"statfs":{"total":96636764160,"available":96331571200,"internally_reserved":0,"allocated":770048,"data_stored":327383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.56399999999999995}]},{"osd":1,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.58299999999999996}]},{"osd":2,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59599999999999997}]},{"osd":3,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.53800000000000003}]},{"osd":4,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.625},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.64000000000000001}]},{"osd":6,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.66600000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.69399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:32:40.102 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:32:40.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.228+0000 7f851f474700 1 Processor -- start 2023-10-09T06:32:40.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.228+0000 7f851f474700 1 -- start start 2023-10-09T06:32:40.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.229+0000 7f851f474700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8518159890 0x7f851815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:40.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.229+0000 7f851f474700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8518069560 0x7f85180699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:40.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.229+0000 7f851f474700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8518069ee0 con 0x7f8518159c60 2023-10-09T06:32:40.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.229+0000 7f851f474700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f851815a160 con 0x7f8518159890 2023-10-09T06:32:40.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.229+0000 7f851f474700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85180d13a0 con 0x7f8518069560 2023-10-09T06:32:40.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.229+0000 7f851d210700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8518159890 0x7f851815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:40.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.229+0000 7f851ca0f700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f8518159c60 0x7f851815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:42770/0 (socket says 172.21.15.163:42770) 2023-10-09T06:32:40.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.229+0000 7f851ca0f700 1 -- 172.21.15.163:0/2221117434 learned_addr learned my addr 172.21.15.163:0/2221117434 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:40.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.230+0000 7f851d210700 1 -- 172.21.15.163:0/2221117434 >> v2:172.21.15.181:3300/0 conn(0x7f8518069560 msgr2=0x7f85180699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:40.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.230+0000 7f851d210700 1 --2- 172.21.15.163:0/2221117434 >> v2:172.21.15.181:3300/0 conn(0x7f8518069560 0x7f85180699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:40.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.230+0000 7f850ffff700 1 -- 172.21.15.163:0/2221117434 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3891657551 0 0) 0x7f8518069ee0 con 0x7f8518159c60 2023-10-09T06:32:40.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.230+0000 7f851d210700 1 -- 172.21.15.163:0/2221117434 >> v1:172.21.15.163:6789/0 conn(0x7f8518159c60 legacy=0x7f851815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:40.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.230+0000 7f851d210700 1 -- 172.21.15.163:0/2221117434 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851815e140 con 0x7f8518159890 2023-10-09T06:32:40.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.230+0000 7f851d210700 1 --2- 172.21.15.163:0/2221117434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8518159890 0x7f851815abd0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f851806a890 tx=0x7f8514009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93448587a2255245 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:40.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.230+0000 7f850ffff700 1 -- 172.21.15.163:0/2221117434 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851400e070 con 0x7f8518159890 2023-10-09T06:32:40.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.230+0000 7f850ffff700 1 -- 172.21.15.163:0/2221117434 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851400bc30 con 0x7f8518159890 2023-10-09T06:32:40.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.230+0000 7f850ffff700 1 -- 172.21.15.163:0/2221117434 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8514012430 con 0x7f8518159890 2023-10-09T06:32:40.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.230+0000 7f851f474700 1 -- 172.21.15.163:0/2221117434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8518159890 msgr2=0x7f851815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:40.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.230+0000 7f851f474700 1 --2- 172.21.15.163:0/2221117434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8518159890 0x7f851815abd0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f851806a890 tx=0x7f8514009780 comp rx=0 tx=0).stop 2023-10-09T06:32:40.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.230+0000 7f851f474700 1 -- 172.21.15.163:0/2221117434 shutdown_connections 2023-10-09T06:32:40.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.230+0000 7f851f474700 1 --2- 172.21.15.163:0/2221117434 >> v2:172.21.15.181:3300/0 conn(0x7f8518069560 0x7f85180699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:40.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.230+0000 7f851f474700 1 --2- 172.21.15.163:0/2221117434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8518159890 0x7f851815abd0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:40.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.230+0000 7f851f474700 1 -- 172.21.15.163:0/2221117434 >> 172.21.15.163:0/2221117434 conn(0x7f8518068e20 msgr2=0x7f85180d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:40.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.231+0000 7f851f474700 1 -- 172.21.15.163:0/2221117434 shutdown_connections 2023-10-09T06:32:40.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.231+0000 7f851f474700 1 -- 172.21.15.163:0/2221117434 wait complete. 2023-10-09T06:32:40.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.232+0000 7f851f474700 1 Processor -- start 2023-10-09T06:32:40.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.232+0000 7f851f474700 1 -- start start 2023-10-09T06:32:40.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.232+0000 7f851f474700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8518069560 0x7f85180b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:40.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.232+0000 7f851f474700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8518159890 0x7f85181576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:40.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.232+0000 7f851f474700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8518159c60 0x7f8518157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:40.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.232+0000 7f851f474700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85180b6440 con 0x7f8518159c60 2023-10-09T06:32:40.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.232+0000 7f851f474700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851815c1e0 con 0x7f8518159890 2023-10-09T06:32:40.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.232+0000 7f851f474700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851815c7a0 con 0x7f8518069560 2023-10-09T06:32:40.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.233+0000 7f851da11700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8518159c60 0x7f8518157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:40.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.233+0000 7f851da11700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8518159c60 0x7f8518157c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:34386/0 (socket says 172.21.15.163:34386) 2023-10-09T06:32:40.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.233+0000 7f851da11700 1 -- 172.21.15.163:0/3950005847 learned_addr learned my addr 172.21.15.163:0/3950005847 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:40.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.233+0000 7f851ca0f700 1 --2- 172.21.15.163:0/3950005847 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8518159890 0x7f85181576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:40.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.233+0000 7f851da11700 1 -- 172.21.15.163:0/3950005847 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8518069560 msgr2=0x7f85180b5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:40.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.233+0000 7f851da11700 1 --2- 172.21.15.163:0/3950005847 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8518069560 0x7f85180b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:40.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.233+0000 7f851da11700 1 -- 172.21.15.163:0/3950005847 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8518159890 msgr2=0x7f85181576c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:40.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.233+0000 7f851da11700 1 --2- 172.21.15.163:0/3950005847 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8518159890 0x7f85181576c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:40.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.233+0000 7f851da11700 1 -- 172.21.15.163:0/3950005847 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8514009a20 con 0x7f8518159c60 2023-10-09T06:32:40.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.233+0000 7f851ca0f700 1 --2- 172.21.15.163:0/3950005847 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8518159890 0x7f85181576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:32:40.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.233+0000 7f851da11700 1 --2- 172.21.15.163:0/3950005847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8518159c60 0x7f8518157c00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f851000b840 tx=0x7f85100091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:40.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.233+0000 7f850d7fa700 1 -- 172.21.15.163:0/3950005847 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851000f510 con 0x7f8518159c60 2023-10-09T06:32:40.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.233+0000 7f850d7fa700 1 -- 172.21.15.163:0/3950005847 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851000f670 con 0x7f8518159c60 2023-10-09T06:32:40.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.233+0000 7f851f474700 1 -- 172.21.15.163:0/3950005847 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85180ce170 con 0x7f8518159c60 2023-10-09T06:32:40.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.233+0000 7f850d7fa700 1 -- 172.21.15.163:0/3950005847 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851000fa30 con 0x7f8518159c60 2023-10-09T06:32:40.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.233+0000 7f851f474700 1 -- 172.21.15.163:0/3950005847 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f851815c430 con 0x7f8518159c60 2023-10-09T06:32:40.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.234+0000 7f851f474700 1 -- 172.21.15.163:0/3950005847 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85180ce670 con 0x7f8518159c60 2023-10-09T06:32:40.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.235+0000 7f850d7fa700 1 -- 172.21.15.163:0/3950005847 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f851001a400 con 0x7f8518159c60 2023-10-09T06:32:40.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.235+0000 7f850d7fa700 1 --2- 172.21.15.163:0/3950005847 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f84fc066fc0 0x7f84fc069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:40.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.235+0000 7f850d7fa700 1 -- 172.21.15.163:0/3950005847 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8510084130 con 0x7f8518159c60 2023-10-09T06:32:40.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.235+0000 7f851d210700 1 --2- 172.21.15.163:0/3950005847 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f84fc066fc0 0x7f84fc069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:40.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.236+0000 7f851d210700 1 --2- 172.21.15.163:0/3950005847 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f84fc066fc0 0x7f84fc069470 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f851806a890 tx=0x7f8514019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:40.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.239+0000 7f850d7fa700 1 -- 172.21.15.163:0/3950005847 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8510051430 con 0x7f8518159c60 2023-10-09T06:32:40.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.419+0000 7f851f474700 1 -- 172.21.15.163:0/3950005847 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f851815c8e0 con 0x7f84fc066fc0 2023-10-09T06:32:40.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.428+0000 7f850d7fa700 1 -- 172.21.15.163:0/3950005847 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289752 (secure 0 0 0) 0x7f851815c8e0 con 0x7f84fc066fc0 2023-10-09T06:32:40.432 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:40.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.431+0000 7f851f474700 1 -- 172.21.15.163:0/3950005847 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f84fc066fc0 msgr2=0x7f84fc069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:40.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.431+0000 7f851f474700 1 --2- 172.21.15.163:0/3950005847 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f84fc066fc0 0x7f84fc069470 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f851806a890 tx=0x7f8514019000 comp rx=0 tx=0).stop 2023-10-09T06:32:40.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.431+0000 7f851f474700 1 -- 172.21.15.163:0/3950005847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8518159c60 msgr2=0x7f8518157c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:40.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.431+0000 7f851f474700 1 --2- 172.21.15.163:0/3950005847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8518159c60 0x7f8518157c00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f851000b840 tx=0x7f85100091d0 comp rx=0 tx=0).stop 2023-10-09T06:32:40.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.431+0000 7f851f474700 1 -- 172.21.15.163:0/3950005847 shutdown_connections 2023-10-09T06:32:40.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.431+0000 7f851f474700 1 --2- 172.21.15.163:0/3950005847 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f84fc066fc0 0x7f84fc069470 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:40.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.431+0000 7f851f474700 1 --2- 172.21.15.163:0/3950005847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8518159c60 0x7f8518157c00 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:40.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.431+0000 7f851f474700 1 --2- 172.21.15.163:0/3950005847 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8518069560 0x7f85180b5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:40.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.431+0000 7f851f474700 1 --2- 172.21.15.163:0/3950005847 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8518159890 0x7f85181576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:40.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.431+0000 7f851f474700 1 -- 172.21.15.163:0/3950005847 >> 172.21.15.163:0/3950005847 conn(0x7f8518068e20 msgr2=0x7f8518149e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:40.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.431+0000 7f851f474700 1 -- 172.21.15.163:0/3950005847 shutdown_connections 2023-10-09T06:32:40.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.431+0000 7f851f474700 1 -- 172.21.15.163:0/3950005847 wait complete. 2023-10-09T06:32:40.436 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:32:40.443 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2023-10-09T06:32:40.327388+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2385512,"kb_used_data":7208,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752589208,"statfs":{"total":773094113280,"available":770651348992,"internally_reserved":0,"allocated":7380992,"data_stored":3803692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.829691"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.527160+0000","last_change":"2023-10-09T06:32:31.524637+0000","last_active":"2023-10-09T06:32:31.527160+0000","last_peered":"2023-10-09T06:32:31.527160+0000","last_clean":"2023-10-09T06:32:31.527160+0000","last_became_active":"2023-10-09T06:32:31.524358+0000","last_became_peered":"2023-10-09T06:32:31.524358+0000","last_unstale":"2023-10-09T06:32:31.527160+0000","last_undegraded":"2023-10-09T06:32:31.527160+0000","last_fullsized":"2023-10-09T06:32:31.527160+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:53:37.666517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.512071+0000","last_change":"2023-10-09T06:32:31.509377+0000","last_active":"2023-10-09T06:32:31.512071+0000","last_peered":"2023-10-09T06:32:31.512071+0000","last_clean":"2023-10-09T06:32:31.512071+0000","last_became_active":"2023-10-09T06:32:31.509105+0000","last_became_peered":"2023-10-09T06:32:31.509105+0000","last_unstale":"2023-10-09T06:32:31.512071+0000","last_undegraded":"2023-10-09T06:32:31.512071+0000","last_fullsized":"2023-10-09T06:32:31.512071+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:26:40.475869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.521134+0000","last_change":"2023-10-09T06:32:31.516319+0000","last_active":"2023-10-09T06:32:31.521134+0000","last_peered":"2023-10-09T06:32:31.521134+0000","last_clean":"2023-10-09T06:32:31.521134+0000","last_became_active":"2023-10-09T06:32:31.516098+0000","last_became_peered":"2023-10-09T06:32:31.516098+0000","last_unstale":"2023-10-09T06:32:31.521134+0000","last_undegraded":"2023-10-09T06:32:31.521134+0000","last_fullsized":"2023-10-09T06:32:31.521134+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:01:10.352276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.516721+0000","last_change":"2023-10-09T06:32:31.508795+0000","last_active":"2023-10-09T06:32:31.516721+0000","last_peered":"2023-10-09T06:32:31.516721+0000","last_clean":"2023-10-09T06:32:31.516721+0000","last_became_active":"2023-10-09T06:32:31.508552+0000","last_became_peered":"2023-10-09T06:32:31.508552+0000","last_unstale":"2023-10-09T06:32:31.516721+0000","last_undegraded":"2023-10-09T06:32:31.516721+0000","last_fullsized":"2023-10-09T06:32:31.516721+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:07:25.930259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.521746+0000","last_change":"2023-10-09T06:32:31.516254+0000","last_active":"2023-10-09T06:32:31.521746+0000","last_peered":"2023-10-09T06:32:31.521746+0000","last_clean":"2023-10-09T06:32:31.521746+0000","last_became_active":"2023-10-09T06:32:31.516028+0000","last_became_peered":"2023-10-09T06:32:31.516028+0000","last_unstale":"2023-10-09T06:32:31.521746+0000","last_undegraded":"2023-10-09T06:32:31.521746+0000","last_fullsized":"2023-10-09T06:32:31.521746+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:04:59.310166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.513536+0000","last_change":"2023-10-09T06:32:31.510222+0000","last_active":"2023-10-09T06:32:31.513536+0000","last_peered":"2023-10-09T06:32:31.513536+0000","last_clean":"2023-10-09T06:32:31.513536+0000","last_became_active":"2023-10-09T06:32:31.509976+0000","last_became_peered":"2023-10-09T06:32:31.509976+0000","last_unstale":"2023-10-09T06:32:31.513536+0000","last_undegraded":"2023-10-09T06:32:31.513536+0000","last_fullsized":"2023-10-09T06:32:31.513536+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:46:11.204402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.507551+0000","last_change":"2023-10-09T06:32:31.506356+0000","last_active":"2023-10-09T06:32:31.507551+0000","last_peered":"2023-10-09T06:32:31.507551+0000","last_clean":"2023-10-09T06:32:31.507551+0000","last_became_active":"2023-10-09T06:32:31.506091+0000","last_became_peered":"2023-10-09T06:32:31.506091+0000","last_unstale":"2023-10-09T06:32:31.507551+0000","last_undegraded":"2023-10-09T06:32:31.507551+0000","last_fullsized":"2023-10-09T06:32:31.507551+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:18:48.881835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.510892+0000","last_change":"2023-10-09T06:32:31.507320+0000","last_active":"2023-10-09T06:32:31.510892+0000","last_peered":"2023-10-09T06:32:31.510892+0000","last_clean":"2023-10-09T06:32:31.510892+0000","last_became_active":"2023-10-09T06:32:31.507026+0000","last_became_peered":"2023-10-09T06:32:31.507026+0000","last_unstale":"2023-10-09T06:32:31.510892+0000","last_undegraded":"2023-10-09T06:32:31.510892+0000","last_fullsized":"2023-10-09T06:32:31.510892+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:35:57.211551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.525232+0000","last_change":"2023-10-09T06:32:31.517060+0000","last_active":"2023-10-09T06:32:31.525232+0000","last_peered":"2023-10-09T06:32:31.525232+0000","last_clean":"2023-10-09T06:32:31.525232+0000","last_became_active":"2023-10-09T06:32:31.515127+0000","last_became_peered":"2023-10-09T06:32:31.515127+0000","last_unstale":"2023-10-09T06:32:31.525232+0000","last_undegraded":"2023-10-09T06:32:31.525232+0000","last_fullsized":"2023-10-09T06:32:31.525232+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:59:12.386020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.529375+0000","last_change":"2023-10-09T06:32:31.519838+0000","last_active":"2023-10-09T06:32:31.529375+0000","last_peered":"2023-10-09T06:32:31.529375+0000","last_clean":"2023-10-09T06:32:31.529375+0000","last_became_active":"2023-10-09T06:32:31.519512+0000","last_became_peered":"2023-10-09T06:32:31.519512+0000","last_unstale":"2023-10-09T06:32:31.529375+0000","last_undegraded":"2023-10-09T06:32:31.529375+0000","last_fullsized":"2023-10-09T06:32:31.529375+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:09:27.240143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.526723+0000","last_change":"2023-10-09T06:32:31.521757+0000","last_active":"2023-10-09T06:32:31.526723+0000","last_peered":"2023-10-09T06:32:31.526723+0000","last_clean":"2023-10-09T06:32:31.526723+0000","last_became_active":"2023-10-09T06:32:31.521357+0000","last_became_peered":"2023-10-09T06:32:31.521357+0000","last_unstale":"2023-10-09T06:32:31.526723+0000","last_undegraded":"2023-10-09T06:32:31.526723+0000","last_fullsized":"2023-10-09T06:32:31.526723+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:49:05.440086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.528195+0000","last_change":"2023-10-09T06:32:31.517951+0000","last_active":"2023-10-09T06:32:31.528195+0000","last_peered":"2023-10-09T06:32:31.528195+0000","last_clean":"2023-10-09T06:32:31.528195+0000","last_became_active":"2023-10-09T06:32:31.517424+0000","last_became_peered":"2023-10-09T06:32:31.517424+0000","last_unstale":"2023-10-09T06:32:31.528195+0000","last_undegraded":"2023-10-09T06:32:31.528195+0000","last_fullsized":"2023-10-09T06:32:31.528195+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:09:27.240143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.516805+0000","last_change":"2023-10-09T06:32:31.513229+0000","last_active":"2023-10-09T06:32:31.516805+0000","last_peered":"2023-10-09T06:32:31.516805+0000","last_clean":"2023-10-09T06:32:31.516805+0000","last_became_active":"2023-10-09T06:32:31.512717+0000","last_became_peered":"2023-10-09T06:32:31.512717+0000","last_unstale":"2023-10-09T06:32:31.516805+0000","last_undegraded":"2023-10-09T06:32:31.516805+0000","last_fullsized":"2023-10-09T06:32:31.516805+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:21:48.208957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.525179+0000","last_change":"2023-10-09T06:32:31.518045+0000","last_active":"2023-10-09T06:32:31.525179+0000","last_peered":"2023-10-09T06:32:31.525179+0000","last_clean":"2023-10-09T06:32:31.525179+0000","last_became_active":"2023-10-09T06:32:31.517815+0000","last_became_peered":"2023-10-09T06:32:31.517815+0000","last_unstale":"2023-10-09T06:32:31.525179+0000","last_undegraded":"2023-10-09T06:32:31.525179+0000","last_fullsized":"2023-10-09T06:32:31.525179+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:35:16.551323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.520320+0000","last_change":"2023-10-09T06:32:31.519359+0000","last_active":"2023-10-09T06:32:31.520320+0000","last_peered":"2023-10-09T06:32:31.520320+0000","last_clean":"2023-10-09T06:32:31.520320+0000","last_became_active":"2023-10-09T06:32:31.519098+0000","last_became_peered":"2023-10-09T06:32:31.519098+0000","last_unstale":"2023-10-09T06:32:31.520320+0000","last_undegraded":"2023-10-09T06:32:31.520320+0000","last_fullsized":"2023-10-09T06:32:31.520320+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:33:33.661840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.528856+0000","last_change":"2023-10-09T06:32:31.513317+0000","last_active":"2023-10-09T06:32:31.528856+0000","last_peered":"2023-10-09T06:32:31.528856+0000","last_clean":"2023-10-09T06:32:31.528856+0000","last_became_active":"2023-10-09T06:32:31.511334+0000","last_became_peered":"2023-10-09T06:32:31.511334+0000","last_unstale":"2023-10-09T06:32:31.528856+0000","last_undegraded":"2023-10-09T06:32:31.528856+0000","last_fullsized":"2023-10-09T06:32:31.528856+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:38:54.218698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.525254+0000","last_change":"2023-10-09T06:32:31.517184+0000","last_active":"2023-10-09T06:32:31.525254+0000","last_peered":"2023-10-09T06:32:31.525254+0000","last_clean":"2023-10-09T06:32:31.525254+0000","last_became_active":"2023-10-09T06:32:31.514902+0000","last_became_peered":"2023-10-09T06:32:31.514902+0000","last_unstale":"2023-10-09T06:32:31.525254+0000","last_undegraded":"2023-10-09T06:32:31.525254+0000","last_fullsized":"2023-10-09T06:32:31.525254+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:42:48.428108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.522573+0000","last_change":"2023-10-09T06:32:31.514575+0000","last_active":"2023-10-09T06:32:31.522573+0000","last_peered":"2023-10-09T06:32:31.522573+0000","last_clean":"2023-10-09T06:32:31.522573+0000","last_became_active":"2023-10-09T06:32:31.514322+0000","last_became_peered":"2023-10-09T06:32:31.514322+0000","last_unstale":"2023-10-09T06:32:31.522573+0000","last_undegraded":"2023-10-09T06:32:31.522573+0000","last_fullsized":"2023-10-09T06:32:31.522573+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:04:39.865936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.510995+0000","last_change":"2023-10-09T06:32:31.505630+0000","last_active":"2023-10-09T06:32:31.510995+0000","last_peered":"2023-10-09T06:32:31.510995+0000","last_clean":"2023-10-09T06:32:31.510995+0000","last_became_active":"2023-10-09T06:32:31.505350+0000","last_became_peered":"2023-10-09T06:32:31.505350+0000","last_unstale":"2023-10-09T06:32:31.510995+0000","last_undegraded":"2023-10-09T06:32:31.510995+0000","last_fullsized":"2023-10-09T06:32:31.510995+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:38:54.218698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.519896+0000","last_change":"2023-10-09T06:32:31.515962+0000","last_active":"2023-10-09T06:32:31.519896+0000","last_peered":"2023-10-09T06:32:31.519896+0000","last_clean":"2023-10-09T06:32:31.519896+0000","last_became_active":"2023-10-09T06:32:31.515659+0000","last_became_peered":"2023-10-09T06:32:31.515659+0000","last_unstale":"2023-10-09T06:32:31.519896+0000","last_undegraded":"2023-10-09T06:32:31.519896+0000","last_fullsized":"2023-10-09T06:32:31.519896+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:07:09.728724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.521854+0000","last_change":"2023-10-09T06:32:31.511806+0000","last_active":"2023-10-09T06:32:31.521854+0000","last_peered":"2023-10-09T06:32:31.521854+0000","last_clean":"2023-10-09T06:32:31.521854+0000","last_became_active":"2023-10-09T06:32:31.511427+0000","last_became_peered":"2023-10-09T06:32:31.511427+0000","last_unstale":"2023-10-09T06:32:31.521854+0000","last_undegraded":"2023-10-09T06:32:31.521854+0000","last_fullsized":"2023-10-09T06:32:31.521854+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:04:39.865936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.529158+0000","last_change":"2023-10-09T06:32:31.518668+0000","last_active":"2023-10-09T06:32:31.529158+0000","last_peered":"2023-10-09T06:32:31.529158+0000","last_clean":"2023-10-09T06:32:31.529158+0000","last_became_active":"2023-10-09T06:32:31.518505+0000","last_became_peered":"2023-10-09T06:32:31.518505+0000","last_unstale":"2023-10-09T06:32:31.529158+0000","last_undegraded":"2023-10-09T06:32:31.529158+0000","last_fullsized":"2023-10-09T06:32:31.529158+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:07:25.930259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.523812+0000","last_change":"2023-10-09T06:32:31.519661+0000","last_active":"2023-10-09T06:32:31.523812+0000","last_peered":"2023-10-09T06:32:31.523812+0000","last_clean":"2023-10-09T06:32:31.523812+0000","last_became_active":"2023-10-09T06:32:31.519351+0000","last_became_peered":"2023-10-09T06:32:31.519351+0000","last_unstale":"2023-10-09T06:32:31.523812+0000","last_undegraded":"2023-10-09T06:32:31.523812+0000","last_fullsized":"2023-10-09T06:32:31.523812+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:00:08.344040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.526936+0000","last_change":"2023-10-09T06:32:31.523432+0000","last_active":"2023-10-09T06:32:31.526936+0000","last_peered":"2023-10-09T06:32:31.526936+0000","last_clean":"2023-10-09T06:32:31.526936+0000","last_became_active":"2023-10-09T06:32:31.523182+0000","last_became_peered":"2023-10-09T06:32:31.523182+0000","last_unstale":"2023-10-09T06:32:31.526936+0000","last_undegraded":"2023-10-09T06:32:31.526936+0000","last_fullsized":"2023-10-09T06:32:31.526936+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:39:56.949135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.528881+0000","last_change":"2023-10-09T06:32:31.513310+0000","last_active":"2023-10-09T06:32:31.528881+0000","last_peered":"2023-10-09T06:32:31.528881+0000","last_clean":"2023-10-09T06:32:31.528881+0000","last_became_active":"2023-10-09T06:32:31.513019+0000","last_became_peered":"2023-10-09T06:32:31.513019+0000","last_unstale":"2023-10-09T06:32:31.528881+0000","last_undegraded":"2023-10-09T06:32:31.528881+0000","last_fullsized":"2023-10-09T06:32:31.528881+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:39:47.607716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.506768+0000","last_change":"2023-10-09T06:32:31.505179+0000","last_active":"2023-10-09T06:32:31.506768+0000","last_peered":"2023-10-09T06:32:31.506768+0000","last_clean":"2023-10-09T06:32:31.506768+0000","last_became_active":"2023-10-09T06:32:31.504885+0000","last_became_peered":"2023-10-09T06:32:31.504885+0000","last_unstale":"2023-10-09T06:32:31.506768+0000","last_undegraded":"2023-10-09T06:32:31.506768+0000","last_fullsized":"2023-10-09T06:32:31.506768+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:51:14.139573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.525859+0000","last_change":"2023-10-09T06:32:31.524416+0000","last_active":"2023-10-09T06:32:31.525859+0000","last_peered":"2023-10-09T06:32:31.525859+0000","last_clean":"2023-10-09T06:32:31.525859+0000","last_became_active":"2023-10-09T06:32:31.524023+0000","last_became_peered":"2023-10-09T06:32:31.524023+0000","last_unstale":"2023-10-09T06:32:31.525859+0000","last_undegraded":"2023-10-09T06:32:31.525859+0000","last_fullsized":"2023-10-09T06:32:31.525859+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:56:33.370165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.529382+0000","last_change":"2023-10-09T06:32:31.519822+0000","last_active":"2023-10-09T06:32:31.529382+0000","last_peered":"2023-10-09T06:32:31.529382+0000","last_clean":"2023-10-09T06:32:31.529382+0000","last_became_active":"2023-10-09T06:32:31.519588+0000","last_became_peered":"2023-10-09T06:32:31.519588+0000","last_unstale":"2023-10-09T06:32:31.529382+0000","last_undegraded":"2023-10-09T06:32:31.529382+0000","last_fullsized":"2023-10-09T06:32:31.529382+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:07:09.907907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.522387+0000","last_change":"2023-10-09T06:32:31.520850+0000","last_active":"2023-10-09T06:32:31.522387+0000","last_peered":"2023-10-09T06:32:31.522387+0000","last_clean":"2023-10-09T06:32:31.522387+0000","last_became_active":"2023-10-09T06:32:31.520588+0000","last_became_peered":"2023-10-09T06:32:31.520588+0000","last_unstale":"2023-10-09T06:32:31.522387+0000","last_undegraded":"2023-10-09T06:32:31.522387+0000","last_fullsized":"2023-10-09T06:32:31.522387+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:17:44.451765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.510590+0000","last_change":"2023-10-09T06:32:31.509245+0000","last_active":"2023-10-09T06:32:31.510590+0000","last_peered":"2023-10-09T06:32:31.510590+0000","last_clean":"2023-10-09T06:32:31.510590+0000","last_became_active":"2023-10-09T06:32:31.508438+0000","last_became_peered":"2023-10-09T06:32:31.508438+0000","last_unstale":"2023-10-09T06:32:31.510590+0000","last_undegraded":"2023-10-09T06:32:31.510590+0000","last_fullsized":"2023-10-09T06:32:31.510590+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:49:05.440086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.520738+0000","last_change":"2023-10-09T06:32:31.515818+0000","last_active":"2023-10-09T06:32:31.520738+0000","last_peered":"2023-10-09T06:32:31.520738+0000","last_clean":"2023-10-09T06:32:31.520738+0000","last_became_active":"2023-10-09T06:32:31.515526+0000","last_became_peered":"2023-10-09T06:32:31.515526+0000","last_unstale":"2023-10-09T06:32:31.520738+0000","last_undegraded":"2023-10-09T06:32:31.520738+0000","last_fullsized":"2023-10-09T06:32:31.520738+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:59:49.326137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.521700+0000","last_change":"2023-10-09T06:32:31.519313+0000","last_active":"2023-10-09T06:32:31.521700+0000","last_peered":"2023-10-09T06:32:31.521700+0000","last_clean":"2023-10-09T06:32:31.521700+0000","last_became_active":"2023-10-09T06:32:31.518783+0000","last_became_peered":"2023-10-09T06:32:31.518783+0000","last_unstale":"2023-10-09T06:32:31.521700+0000","last_undegraded":"2023-10-09T06:32:31.521700+0000","last_fullsized":"2023-10-09T06:32:31.521700+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:23:21.996048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.526829+0000","last_change":"2023-10-09T06:32:31.522663+0000","last_active":"2023-10-09T06:32:31.526829+0000","last_peered":"2023-10-09T06:32:31.526829+0000","last_clean":"2023-10-09T06:32:31.526829+0000","last_became_active":"2023-10-09T06:32:31.522387+0000","last_became_peered":"2023-10-09T06:32:31.522387+0000","last_unstale":"2023-10-09T06:32:31.526829+0000","last_undegraded":"2023-10-09T06:32:31.526829+0000","last_fullsized":"2023-10-09T06:32:31.526829+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:54:40.864124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"62'4","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.528535+0000","last_change":"2023-10-09T06:32:30.516709+0000","last_active":"2023-10-09T06:32:31.528535+0000","last_peered":"2023-10-09T06:32:31.528535+0000","last_clean":"2023-10-09T06:32:31.528535+0000","last_became_active":"2023-10-09T06:32:30.516362+0000","last_became_peered":"2023-10-09T06:32:30.516362+0000","last_unstale":"2023-10-09T06:32:31.528535+0000","last_undegraded":"2023-10-09T06:32:31.528535+0000","last_fullsized":"2023-10-09T06:32:31.528535+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:59:22.255688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.521741+0000","last_change":"2023-10-09T06:32:31.515374+0000","last_active":"2023-10-09T06:32:31.521741+0000","last_peered":"2023-10-09T06:32:31.521741+0000","last_clean":"2023-10-09T06:32:31.521741+0000","last_became_active":"2023-10-09T06:32:31.514641+0000","last_became_peered":"2023-10-09T06:32:31.514641+0000","last_unstale":"2023-10-09T06:32:31.521741+0000","last_undegraded":"2023-10-09T06:32:31.521741+0000","last_fullsized":"2023-10-09T06:32:31.521741+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:55:19.695334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"62'1","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.518469+0000","last_change":"2023-10-09T06:32:30.511051+0000","last_active":"2023-10-09T06:32:31.518469+0000","last_peered":"2023-10-09T06:32:31.518469+0000","last_clean":"2023-10-09T06:32:31.518469+0000","last_became_active":"2023-10-09T06:32:30.510868+0000","last_became_peered":"2023-10-09T06:32:30.510868+0000","last_unstale":"2023-10-09T06:32:31.518469+0000","last_undegraded":"2023-10-09T06:32:31.518469+0000","last_fullsized":"2023-10-09T06:32:31.518469+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:07:24.928003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.525279+0000","last_change":"2023-10-09T06:32:31.518608+0000","last_active":"2023-10-09T06:32:31.525279+0000","last_peered":"2023-10-09T06:32:31.525279+0000","last_clean":"2023-10-09T06:32:31.525279+0000","last_became_active":"2023-10-09T06:32:31.516173+0000","last_became_peered":"2023-10-09T06:32:31.516173+0000","last_unstale":"2023-10-09T06:32:31.525279+0000","last_undegraded":"2023-10-09T06:32:31.525279+0000","last_fullsized":"2023-10-09T06:32:31.525279+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:36:58.416957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"62'2","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.529117+0000","last_change":"2023-10-09T06:32:30.512860+0000","last_active":"2023-10-09T06:32:31.529117+0000","last_peered":"2023-10-09T06:32:31.529117+0000","last_clean":"2023-10-09T06:32:31.529117+0000","last_became_active":"2023-10-09T06:32:30.512126+0000","last_became_peered":"2023-10-09T06:32:30.512126+0000","last_unstale":"2023-10-09T06:32:31.529117+0000","last_undegraded":"2023-10-09T06:32:31.529117+0000","last_fullsized":"2023-10-09T06:32:31.529117+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:56:07.601703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.526702+0000","last_change":"2023-10-09T06:32:31.521601+0000","last_active":"2023-10-09T06:32:31.526702+0000","last_peered":"2023-10-09T06:32:31.526702+0000","last_clean":"2023-10-09T06:32:31.526702+0000","last_became_active":"2023-10-09T06:32:31.520977+0000","last_became_peered":"2023-10-09T06:32:31.520977+0000","last_unstale":"2023-10-09T06:32:31.526702+0000","last_undegraded":"2023-10-09T06:32:31.526702+0000","last_fullsized":"2023-10-09T06:32:31.526702+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:51:14.139573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"62'2","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.528397+0000","last_change":"2023-10-09T06:32:30.511961+0000","last_active":"2023-10-09T06:32:31.528397+0000","last_peered":"2023-10-09T06:32:31.528397+0000","last_clean":"2023-10-09T06:32:31.528397+0000","last_became_active":"2023-10-09T06:32:30.511636+0000","last_became_peered":"2023-10-09T06:32:30.511636+0000","last_unstale":"2023-10-09T06:32:31.528397+0000","last_undegraded":"2023-10-09T06:32:31.528397+0000","last_fullsized":"2023-10-09T06:32:31.528397+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:59:33.319061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.521642+0000","last_change":"2023-10-09T06:32:31.517502+0000","last_active":"2023-10-09T06:32:31.521642+0000","last_peered":"2023-10-09T06:32:31.521642+0000","last_clean":"2023-10-09T06:32:31.521642+0000","last_became_active":"2023-10-09T06:32:31.517236+0000","last_became_peered":"2023-10-09T06:32:31.517236+0000","last_unstale":"2023-10-09T06:32:31.521642+0000","last_undegraded":"2023-10-09T06:32:31.521642+0000","last_fullsized":"2023-10-09T06:32:31.521642+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:30:47.845662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"62'1","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.529291+0000","last_change":"2023-10-09T06:32:30.509577+0000","last_active":"2023-10-09T06:32:31.529291+0000","last_peered":"2023-10-09T06:32:31.529291+0000","last_clean":"2023-10-09T06:32:31.529291+0000","last_became_active":"2023-10-09T06:32:30.509228+0000","last_became_peered":"2023-10-09T06:32:30.509228+0000","last_unstale":"2023-10-09T06:32:31.529291+0000","last_undegraded":"2023-10-09T06:32:31.529291+0000","last_fullsized":"2023-10-09T06:32:31.529291+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:32:16.720741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.521834+0000","last_change":"2023-10-09T06:32:31.511162+0000","last_active":"2023-10-09T06:32:31.521834+0000","last_peered":"2023-10-09T06:32:31.521834+0000","last_clean":"2023-10-09T06:32:31.521834+0000","last_became_active":"2023-10-09T06:32:31.510823+0000","last_became_peered":"2023-10-09T06:32:31.510823+0000","last_unstale":"2023-10-09T06:32:31.521834+0000","last_undegraded":"2023-10-09T06:32:31.521834+0000","last_fullsized":"2023-10-09T06:32:31.521834+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:59:23.257943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"62'1","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.520277+0000","last_change":"2023-10-09T06:32:30.511257+0000","last_active":"2023-10-09T06:32:31.520277+0000","last_peered":"2023-10-09T06:32:31.520277+0000","last_clean":"2023-10-09T06:32:31.520277+0000","last_became_active":"2023-10-09T06:32:30.510917+0000","last_became_peered":"2023-10-09T06:32:30.510917+0000","last_unstale":"2023-10-09T06:32:31.520277+0000","last_undegraded":"2023-10-09T06:32:31.520277+0000","last_fullsized":"2023-10-09T06:32:31.520277+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:54:00.527419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.524745+0000","last_change":"2023-10-09T06:32:31.516829+0000","last_active":"2023-10-09T06:32:31.524745+0000","last_peered":"2023-10-09T06:32:31.524745+0000","last_clean":"2023-10-09T06:32:31.524745+0000","last_became_active":"2023-10-09T06:32:31.516624+0000","last_became_peered":"2023-10-09T06:32:31.516624+0000","last_unstale":"2023-10-09T06:32:31.524745+0000","last_undegraded":"2023-10-09T06:32:31.524745+0000","last_fullsized":"2023-10-09T06:32:31.524745+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:15:30.917640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.501662+0000","last_change":"2023-10-09T06:32:30.508800+0000","last_active":"2023-10-09T06:32:31.501662+0000","last_peered":"2023-10-09T06:32:31.501662+0000","last_clean":"2023-10-09T06:32:31.501662+0000","last_became_active":"2023-10-09T06:32:30.508556+0000","last_became_peered":"2023-10-09T06:32:30.508556+0000","last_unstale":"2023-10-09T06:32:31.501662+0000","last_undegraded":"2023-10-09T06:32:31.501662+0000","last_fullsized":"2023-10-09T06:32:31.501662+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:08:35.442207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.520302+0000","last_change":"2023-10-09T06:32:31.517400+0000","last_active":"2023-10-09T06:32:31.520302+0000","last_peered":"2023-10-09T06:32:31.520302+0000","last_clean":"2023-10-09T06:32:31.520302+0000","last_became_active":"2023-10-09T06:32:31.517188+0000","last_became_peered":"2023-10-09T06:32:31.517188+0000","last_unstale":"2023-10-09T06:32:31.520302+0000","last_undegraded":"2023-10-09T06:32:31.520302+0000","last_fullsized":"2023-10-09T06:32:31.520302+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:49:04.586488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.505510+0000","last_change":"2023-10-09T06:32:30.508443+0000","last_active":"2023-10-09T06:32:31.505510+0000","last_peered":"2023-10-09T06:32:31.505510+0000","last_clean":"2023-10-09T06:32:31.505510+0000","last_became_active":"2023-10-09T06:32:30.508073+0000","last_became_peered":"2023-10-09T06:32:30.508073+0000","last_unstale":"2023-10-09T06:32:31.505510+0000","last_undegraded":"2023-10-09T06:32:31.505510+0000","last_fullsized":"2023-10-09T06:32:31.505510+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:34:19.640682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.516952+0000","last_change":"2023-10-09T06:32:31.513286+0000","last_active":"2023-10-09T06:32:31.516952+0000","last_peered":"2023-10-09T06:32:31.516952+0000","last_clean":"2023-10-09T06:32:31.516952+0000","last_became_active":"2023-10-09T06:32:31.513019+0000","last_became_peered":"2023-10-09T06:32:31.513019+0000","last_unstale":"2023-10-09T06:32:31.516952+0000","last_undegraded":"2023-10-09T06:32:31.516952+0000","last_fullsized":"2023-10-09T06:32:31.516952+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:21:07.366679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.506731+0000","last_change":"2023-10-09T06:32:30.510669+0000","last_active":"2023-10-09T06:32:31.506731+0000","last_peered":"2023-10-09T06:32:31.506731+0000","last_clean":"2023-10-09T06:32:31.506731+0000","last_became_active":"2023-10-09T06:32:30.510356+0000","last_became_peered":"2023-10-09T06:32:30.510356+0000","last_unstale":"2023-10-09T06:32:31.506731+0000","last_undegraded":"2023-10-09T06:32:31.506731+0000","last_fullsized":"2023-10-09T06:32:31.506731+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:37:23.971604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.526854+0000","last_change":"2023-10-09T06:32:31.522871+0000","last_active":"2023-10-09T06:32:31.526854+0000","last_peered":"2023-10-09T06:32:31.526854+0000","last_clean":"2023-10-09T06:32:31.526854+0000","last_became_active":"2023-10-09T06:32:31.522597+0000","last_became_peered":"2023-10-09T06:32:31.522597+0000","last_unstale":"2023-10-09T06:32:31.526854+0000","last_undegraded":"2023-10-09T06:32:31.526854+0000","last_fullsized":"2023-10-09T06:32:31.526854+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:22:19.984152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.501373+0000","last_change":"2023-10-09T06:32:30.512173+0000","last_active":"2023-10-09T06:32:31.501373+0000","last_peered":"2023-10-09T06:32:31.501373+0000","last_clean":"2023-10-09T06:32:31.501373+0000","last_became_active":"2023-10-09T06:32:30.511876+0000","last_became_peered":"2023-10-09T06:32:30.511876+0000","last_unstale":"2023-10-09T06:32:31.501373+0000","last_undegraded":"2023-10-09T06:32:31.501373+0000","last_fullsized":"2023-10-09T06:32:31.501373+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:22:34.707397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.512098+0000","last_change":"2023-10-09T06:32:31.509905+0000","last_active":"2023-10-09T06:32:31.512098+0000","last_peered":"2023-10-09T06:32:31.512098+0000","last_clean":"2023-10-09T06:32:31.512098+0000","last_became_active":"2023-10-09T06:32:31.509045+0000","last_became_peered":"2023-10-09T06:32:31.509045+0000","last_unstale":"2023-10-09T06:32:31.512098+0000","last_undegraded":"2023-10-09T06:32:31.512098+0000","last_fullsized":"2023-10-09T06:32:31.512098+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:37:57.966936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"62'3","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.523685+0000","last_change":"2023-10-09T06:32:30.512291+0000","last_active":"2023-10-09T06:32:31.523685+0000","last_peered":"2023-10-09T06:32:31.523685+0000","last_clean":"2023-10-09T06:32:31.523685+0000","last_became_active":"2023-10-09T06:32:30.512051+0000","last_became_peered":"2023-10-09T06:32:30.512051+0000","last_unstale":"2023-10-09T06:32:31.523685+0000","last_undegraded":"2023-10-09T06:32:31.523685+0000","last_fullsized":"2023-10-09T06:32:31.523685+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:22:18.981897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.525244+0000","last_change":"2023-10-09T06:32:31.518519+0000","last_active":"2023-10-09T06:32:31.525244+0000","last_peered":"2023-10-09T06:32:31.525244+0000","last_clean":"2023-10-09T06:32:31.525244+0000","last_became_active":"2023-10-09T06:32:31.518259+0000","last_became_peered":"2023-10-09T06:32:31.518259+0000","last_unstale":"2023-10-09T06:32:31.525244+0000","last_undegraded":"2023-10-09T06:32:31.525244+0000","last_fullsized":"2023-10-09T06:32:31.525244+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:25:13.501296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"62'6","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.533817+0000","last_change":"2023-10-09T06:32:30.510937+0000","last_active":"2023-10-09T06:32:31.533817+0000","last_peered":"2023-10-09T06:32:31.533817+0000","last_clean":"2023-10-09T06:32:31.533817+0000","last_became_active":"2023-10-09T06:32:30.510651+0000","last_became_peered":"2023-10-09T06:32:30.510651+0000","last_unstale":"2023-10-09T06:32:31.533817+0000","last_undegraded":"2023-10-09T06:32:31.533817+0000","last_fullsized":"2023-10-09T06:32:31.533817+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:56:07.601703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.527185+0000","last_change":"2023-10-09T06:32:31.525788+0000","last_active":"2023-10-09T06:32:31.527185+0000","last_peered":"2023-10-09T06:32:31.527185+0000","last_clean":"2023-10-09T06:32:31.527185+0000","last_became_active":"2023-10-09T06:32:31.525526+0000","last_became_peered":"2023-10-09T06:32:31.525526+0000","last_unstale":"2023-10-09T06:32:31.527185+0000","last_undegraded":"2023-10-09T06:32:31.527185+0000","last_fullsized":"2023-10-09T06:32:31.527185+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:23:42.065325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"62'2","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.523162+0000","last_change":"2023-10-09T06:32:30.513665+0000","last_active":"2023-10-09T06:32:31.523162+0000","last_peered":"2023-10-09T06:32:31.523162+0000","last_clean":"2023-10-09T06:32:31.523162+0000","last_became_active":"2023-10-09T06:32:30.513385+0000","last_became_peered":"2023-10-09T06:32:30.513385+0000","last_unstale":"2023-10-09T06:32:31.523162+0000","last_undegraded":"2023-10-09T06:32:31.523162+0000","last_fullsized":"2023-10-09T06:32:31.523162+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:05:29.788926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.512174+0000","last_change":"2023-10-09T06:32:31.509930+0000","last_active":"2023-10-09T06:32:31.512174+0000","last_peered":"2023-10-09T06:32:31.512174+0000","last_clean":"2023-10-09T06:32:31.512174+0000","last_became_active":"2023-10-09T06:32:31.509413+0000","last_became_peered":"2023-10-09T06:32:31.509413+0000","last_unstale":"2023-10-09T06:32:31.512174+0000","last_undegraded":"2023-10-09T06:32:31.512174+0000","last_fullsized":"2023-10-09T06:32:31.512174+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:48:07.062498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.502176+0000","last_change":"2023-10-09T06:32:30.514316+0000","last_active":"2023-10-09T06:32:31.502176+0000","last_peered":"2023-10-09T06:32:31.502176+0000","last_clean":"2023-10-09T06:32:31.502176+0000","last_became_active":"2023-10-09T06:32:30.513987+0000","last_became_peered":"2023-10-09T06:32:30.513987+0000","last_unstale":"2023-10-09T06:32:31.502176+0000","last_undegraded":"2023-10-09T06:32:31.502176+0000","last_fullsized":"2023-10-09T06:32:31.502176+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:39:36.054325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.516768+0000","last_change":"2023-10-09T06:32:31.512769+0000","last_active":"2023-10-09T06:32:31.516768+0000","last_peered":"2023-10-09T06:32:31.516768+0000","last_clean":"2023-10-09T06:32:31.516768+0000","last_became_active":"2023-10-09T06:32:31.512524+0000","last_became_peered":"2023-10-09T06:32:31.512524+0000","last_unstale":"2023-10-09T06:32:31.516768+0000","last_undegraded":"2023-10-09T06:32:31.516768+0000","last_fullsized":"2023-10-09T06:32:31.516768+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:57:17.839161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.500386+0000","last_change":"2023-10-09T06:32:30.514034+0000","last_active":"2023-10-09T06:32:31.500386+0000","last_peered":"2023-10-09T06:32:31.500386+0000","last_clean":"2023-10-09T06:32:31.500386+0000","last_became_active":"2023-10-09T06:32:30.513765+0000","last_became_peered":"2023-10-09T06:32:30.513765+0000","last_unstale":"2023-10-09T06:32:31.500386+0000","last_undegraded":"2023-10-09T06:32:31.500386+0000","last_fullsized":"2023-10-09T06:32:31.500386+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:07:08.905652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.524799+0000","last_change":"2023-10-09T06:32:31.516916+0000","last_active":"2023-10-09T06:32:31.524799+0000","last_peered":"2023-10-09T06:32:31.524799+0000","last_clean":"2023-10-09T06:32:31.524799+0000","last_became_active":"2023-10-09T06:32:31.515173+0000","last_became_peered":"2023-10-09T06:32:31.515173+0000","last_unstale":"2023-10-09T06:32:31.524799+0000","last_undegraded":"2023-10-09T06:32:31.524799+0000","last_fullsized":"2023-10-09T06:32:31.524799+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:24:05.777223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"62'8","reported_seq":18,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.529659+0000","last_change":"2023-10-09T06:32:30.515315+0000","last_active":"2023-10-09T06:32:31.529659+0000","last_peered":"2023-10-09T06:32:31.529659+0000","last_clean":"2023-10-09T06:32:31.529659+0000","last_became_active":"2023-10-09T06:32:30.513454+0000","last_became_peered":"2023-10-09T06:32:30.513454+0000","last_unstale":"2023-10-09T06:32:31.529659+0000","last_undegraded":"2023-10-09T06:32:31.529659+0000","last_fullsized":"2023-10-09T06:32:31.529659+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:08:35.442207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.528934+0000","last_change":"2023-10-09T06:32:31.513564+0000","last_active":"2023-10-09T06:32:31.528934+0000","last_peered":"2023-10-09T06:32:31.528934+0000","last_clean":"2023-10-09T06:32:31.528934+0000","last_became_active":"2023-10-09T06:32:31.511889+0000","last_became_peered":"2023-10-09T06:32:31.511889+0000","last_unstale":"2023-10-09T06:32:31.528934+0000","last_undegraded":"2023-10-09T06:32:31.528934+0000","last_fullsized":"2023-10-09T06:32:31.528934+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:10:02.013724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"62'1","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.526651+0000","last_change":"2023-10-09T06:32:30.511763+0000","last_active":"2023-10-09T06:32:31.526651+0000","last_peered":"2023-10-09T06:32:31.526651+0000","last_clean":"2023-10-09T06:32:31.526651+0000","last_became_active":"2023-10-09T06:32:30.510441+0000","last_became_peered":"2023-10-09T06:32:30.510441+0000","last_unstale":"2023-10-09T06:32:31.526651+0000","last_undegraded":"2023-10-09T06:32:31.526651+0000","last_fullsized":"2023-10-09T06:32:31.526651+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:24:24.863354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.527139+0000","last_change":"2023-10-09T06:32:31.524535+0000","last_active":"2023-10-09T06:32:31.527139+0000","last_peered":"2023-10-09T06:32:31.527139+0000","last_clean":"2023-10-09T06:32:31.527139+0000","last_became_active":"2023-10-09T06:32:31.524298+0000","last_became_peered":"2023-10-09T06:32:31.524298+0000","last_unstale":"2023-10-09T06:32:31.527139+0000","last_undegraded":"2023-10-09T06:32:31.527139+0000","last_fullsized":"2023-10-09T06:32:31.527139+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:21:07.366679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"62'2","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.529096+0000","last_change":"2023-10-09T06:32:30.516087+0000","last_active":"2023-10-09T06:32:31.529096+0000","last_peered":"2023-10-09T06:32:31.529096+0000","last_clean":"2023-10-09T06:32:31.529096+0000","last_became_active":"2023-10-09T06:32:30.511897+0000","last_became_peered":"2023-10-09T06:32:30.511897+0000","last_unstale":"2023-10-09T06:32:31.529096+0000","last_undegraded":"2023-10-09T06:32:31.529096+0000","last_fullsized":"2023-10-09T06:32:31.529096+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:31:31.960774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.528175+0000","last_change":"2023-10-09T06:32:31.517808+0000","last_active":"2023-10-09T06:32:31.528175+0000","last_peered":"2023-10-09T06:32:31.528175+0000","last_clean":"2023-10-09T06:32:31.528175+0000","last_became_active":"2023-10-09T06:32:31.517108+0000","last_became_peered":"2023-10-09T06:32:31.517108+0000","last_unstale":"2023-10-09T06:32:31.528175+0000","last_undegraded":"2023-10-09T06:32:31.528175+0000","last_fullsized":"2023-10-09T06:32:31.528175+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:36:58.416957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"62'1","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.529393+0000","last_change":"2023-10-09T06:32:30.509241+0000","last_active":"2023-10-09T06:32:31.529393+0000","last_peered":"2023-10-09T06:32:31.529393+0000","last_clean":"2023-10-09T06:32:31.529393+0000","last_became_active":"2023-10-09T06:32:30.508842+0000","last_became_peered":"2023-10-09T06:32:30.508842+0000","last_unstale":"2023-10-09T06:32:31.529393+0000","last_undegraded":"2023-10-09T06:32:31.529393+0000","last_fullsized":"2023-10-09T06:32:31.529393+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:25:12.499041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.524721+0000","last_change":"2023-10-09T06:32:31.516415+0000","last_active":"2023-10-09T06:32:31.524721+0000","last_peered":"2023-10-09T06:32:31.524721+0000","last_clean":"2023-10-09T06:32:31.524721+0000","last_became_active":"2023-10-09T06:32:31.516218+0000","last_became_peered":"2023-10-09T06:32:31.516218+0000","last_unstale":"2023-10-09T06:32:31.524721+0000","last_undegraded":"2023-10-09T06:32:31.524721+0000","last_fullsized":"2023-10-09T06:32:31.524721+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:34:05.882099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.509206+0000","last_change":"2023-10-09T06:32:30.514856+0000","last_active":"2023-10-09T06:32:31.509206+0000","last_peered":"2023-10-09T06:32:31.509206+0000","last_clean":"2023-10-09T06:32:31.509206+0000","last_became_active":"2023-10-09T06:32:30.514570+0000","last_became_peered":"2023-10-09T06:32:30.514570+0000","last_unstale":"2023-10-09T06:32:31.509206+0000","last_undegraded":"2023-10-09T06:32:31.509206+0000","last_fullsized":"2023-10-09T06:32:31.509206+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:39:46.605461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.525515+0000","last_change":"2023-10-09T06:32:31.523624+0000","last_active":"2023-10-09T06:32:31.525515+0000","last_peered":"2023-10-09T06:32:31.525515+0000","last_clean":"2023-10-09T06:32:31.525515+0000","last_became_active":"2023-10-09T06:32:31.523370+0000","last_became_peered":"2023-10-09T06:32:31.523370+0000","last_unstale":"2023-10-09T06:32:31.525515+0000","last_undegraded":"2023-10-09T06:32:31.525515+0000","last_fullsized":"2023-10-09T06:32:31.525515+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:52:41.647855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.502266+0000","last_change":"2023-10-09T06:32:30.511452+0000","last_active":"2023-10-09T06:32:31.502266+0000","last_peered":"2023-10-09T06:32:31.502266+0000","last_clean":"2023-10-09T06:32:31.502266+0000","last_became_active":"2023-10-09T06:32:30.511160+0000","last_became_peered":"2023-10-09T06:32:30.511160+0000","last_unstale":"2023-10-09T06:32:31.502266+0000","last_undegraded":"2023-10-09T06:32:31.502266+0000","last_fullsized":"2023-10-09T06:32:31.502266+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:00:45.173974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.525454+0000","last_change":"2023-10-09T06:32:31.523321+0000","last_active":"2023-10-09T06:32:31.525454+0000","last_peered":"2023-10-09T06:32:31.525454+0000","last_clean":"2023-10-09T06:32:31.525454+0000","last_became_active":"2023-10-09T06:32:31.523070+0000","last_became_peered":"2023-10-09T06:32:31.523070+0000","last_unstale":"2023-10-09T06:32:31.525454+0000","last_undegraded":"2023-10-09T06:32:31.525454+0000","last_fullsized":"2023-10-09T06:32:31.525454+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:55:30.195305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.500491+0000","last_change":"2023-10-09T06:32:30.514925+0000","last_active":"2023-10-09T06:32:31.500491+0000","last_peered":"2023-10-09T06:32:31.500491+0000","last_clean":"2023-10-09T06:32:31.500491+0000","last_became_active":"2023-10-09T06:32:30.514621+0000","last_became_peered":"2023-10-09T06:32:30.514621+0000","last_unstale":"2023-10-09T06:32:31.500491+0000","last_undegraded":"2023-10-09T06:32:31.500491+0000","last_fullsized":"2023-10-09T06:32:31.500491+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:10:01.011469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.525536+0000","last_change":"2023-10-09T06:32:31.523106+0000","last_active":"2023-10-09T06:32:31.525536+0000","last_peered":"2023-10-09T06:32:31.525536+0000","last_clean":"2023-10-09T06:32:31.525536+0000","last_became_active":"2023-10-09T06:32:31.522837+0000","last_became_peered":"2023-10-09T06:32:31.522837+0000","last_unstale":"2023-10-09T06:32:31.525536+0000","last_undegraded":"2023-10-09T06:32:31.525536+0000","last_fullsized":"2023-10-09T06:32:31.525536+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:06:23.253040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.505980+0000","last_change":"2023-10-09T06:32:30.513984+0000","last_active":"2023-10-09T06:32:31.505980+0000","last_peered":"2023-10-09T06:32:31.505980+0000","last_clean":"2023-10-09T06:32:31.505980+0000","last_became_active":"2023-10-09T06:32:30.513743+0000","last_became_peered":"2023-10-09T06:32:30.513743+0000","last_unstale":"2023-10-09T06:32:31.505980+0000","last_undegraded":"2023-10-09T06:32:31.505980+0000","last_fullsized":"2023-10-09T06:32:31.505980+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:09:26.237888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.520336+0000","last_change":"2023-10-09T06:32:31.519388+0000","last_active":"2023-10-09T06:32:31.520336+0000","last_peered":"2023-10-09T06:32:31.520336+0000","last_clean":"2023-10-09T06:32:31.520336+0000","last_became_active":"2023-10-09T06:32:31.519126+0000","last_became_peered":"2023-10-09T06:32:31.519126+0000","last_unstale":"2023-10-09T06:32:31.520336+0000","last_undegraded":"2023-10-09T06:32:31.520336+0000","last_fullsized":"2023-10-09T06:32:31.520336+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:18:48.881835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"62'2","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.526529+0000","last_change":"2023-10-09T06:32:30.510835+0000","last_active":"2023-10-09T06:32:31.526529+0000","last_peered":"2023-10-09T06:32:31.526529+0000","last_clean":"2023-10-09T06:32:31.526529+0000","last_became_active":"2023-10-09T06:32:30.510376+0000","last_became_peered":"2023-10-09T06:32:30.510376+0000","last_unstale":"2023-10-09T06:32:31.526529+0000","last_undegraded":"2023-10-09T06:32:31.526529+0000","last_fullsized":"2023-10-09T06:32:31.526529+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:52:29.200026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.508275+0000","last_change":"2023-10-09T06:32:31.506492+0000","last_active":"2023-10-09T06:32:31.508275+0000","last_peered":"2023-10-09T06:32:31.508275+0000","last_clean":"2023-10-09T06:32:31.508275+0000","last_became_active":"2023-10-09T06:32:31.506014+0000","last_became_peered":"2023-10-09T06:32:31.506014+0000","last_unstale":"2023-10-09T06:32:31.508275+0000","last_undegraded":"2023-10-09T06:32:31.508275+0000","last_fullsized":"2023-10-09T06:32:31.508275+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:07:09.728724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.501106+0000","last_change":"2023-10-09T06:32:30.513921+0000","last_active":"2023-10-09T06:32:31.501106+0000","last_peered":"2023-10-09T06:32:31.501106+0000","last_clean":"2023-10-09T06:32:31.501106+0000","last_became_active":"2023-10-09T06:32:30.513649+0000","last_became_peered":"2023-10-09T06:32:30.513649+0000","last_unstale":"2023-10-09T06:32:31.501106+0000","last_undegraded":"2023-10-09T06:32:31.501106+0000","last_fullsized":"2023-10-09T06:32:31.501106+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:07:24.928003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.526850+0000","last_change":"2023-10-09T06:32:31.522803+0000","last_active":"2023-10-09T06:32:31.526850+0000","last_peered":"2023-10-09T06:32:31.526850+0000","last_clean":"2023-10-09T06:32:31.526850+0000","last_became_active":"2023-10-09T06:32:31.522416+0000","last_became_peered":"2023-10-09T06:32:31.522416+0000","last_unstale":"2023-10-09T06:32:31.526850+0000","last_undegraded":"2023-10-09T06:32:31.526850+0000","last_fullsized":"2023-10-09T06:32:31.526850+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:07:09.728724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.507650+0000","last_change":"2023-10-09T06:32:30.506664+0000","last_active":"2023-10-09T06:32:31.507650+0000","last_peered":"2023-10-09T06:32:31.507650+0000","last_clean":"2023-10-09T06:32:31.507650+0000","last_became_active":"2023-10-09T06:32:30.506367+0000","last_became_peered":"2023-10-09T06:32:30.506367+0000","last_unstale":"2023-10-09T06:32:31.507650+0000","last_undegraded":"2023-10-09T06:32:31.507650+0000","last_fullsized":"2023-10-09T06:32:31.507650+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:25:13.089891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.523698+0000","last_change":"2023-10-09T06:32:31.520372+0000","last_active":"2023-10-09T06:32:31.523698+0000","last_peered":"2023-10-09T06:32:31.523698+0000","last_clean":"2023-10-09T06:32:31.523698+0000","last_became_active":"2023-10-09T06:32:31.520059+0000","last_became_peered":"2023-10-09T06:32:31.520059+0000","last_unstale":"2023-10-09T06:32:31.523698+0000","last_undegraded":"2023-10-09T06:32:31.523698+0000","last_fullsized":"2023-10-09T06:32:31.523698+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:17:52.054195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.506496+0000","last_change":"2023-10-09T06:32:30.509435+0000","last_active":"2023-10-09T06:32:31.506496+0000","last_peered":"2023-10-09T06:32:31.506496+0000","last_clean":"2023-10-09T06:32:31.506496+0000","last_became_active":"2023-10-09T06:32:30.509159+0000","last_became_peered":"2023-10-09T06:32:30.509159+0000","last_unstale":"2023-10-09T06:32:31.506496+0000","last_undegraded":"2023-10-09T06:32:31.506496+0000","last_fullsized":"2023-10-09T06:32:31.506496+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:38:53.216443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.525158+0000","last_change":"2023-10-09T06:32:31.517724+0000","last_active":"2023-10-09T06:32:31.525158+0000","last_peered":"2023-10-09T06:32:31.525158+0000","last_clean":"2023-10-09T06:32:31.525158+0000","last_became_active":"2023-10-09T06:32:31.514920+0000","last_became_peered":"2023-10-09T06:32:31.514920+0000","last_unstale":"2023-10-09T06:32:31.525158+0000","last_undegraded":"2023-10-09T06:32:31.525158+0000","last_fullsized":"2023-10-09T06:32:31.525158+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:02:04.881485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.507541+0000","last_change":"2023-10-09T06:32:30.516768+0000","last_active":"2023-10-09T06:32:31.507541+0000","last_peered":"2023-10-09T06:32:31.507541+0000","last_clean":"2023-10-09T06:32:31.507541+0000","last_became_active":"2023-10-09T06:32:30.516451+0000","last_became_peered":"2023-10-09T06:32:30.516451+0000","last_unstale":"2023-10-09T06:32:31.507541+0000","last_undegraded":"2023-10-09T06:32:31.507541+0000","last_fullsized":"2023-10-09T06:32:31.507541+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:07:19.969025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.525400+0000","last_change":"2023-10-09T06:32:31.520802+0000","last_active":"2023-10-09T06:32:31.525400+0000","last_peered":"2023-10-09T06:32:31.525400+0000","last_clean":"2023-10-09T06:32:31.525400+0000","last_became_active":"2023-10-09T06:32:31.520546+0000","last_became_peered":"2023-10-09T06:32:31.520546+0000","last_unstale":"2023-10-09T06:32:31.525400+0000","last_undegraded":"2023-10-09T06:32:31.525400+0000","last_fullsized":"2023-10-09T06:32:31.525400+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:09:27.240143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.501320+0000","last_change":"2023-10-09T06:32:30.509591+0000","last_active":"2023-10-09T06:32:31.501320+0000","last_peered":"2023-10-09T06:32:31.501320+0000","last_clean":"2023-10-09T06:32:31.501320+0000","last_became_active":"2023-10-09T06:32:30.509314+0000","last_became_peered":"2023-10-09T06:32:30.509314+0000","last_unstale":"2023-10-09T06:32:31.501320+0000","last_undegraded":"2023-10-09T06:32:31.501320+0000","last_fullsized":"2023-10-09T06:32:31.501320+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:52:55.778655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.523596+0000","last_change":"2023-10-09T06:32:31.511717+0000","last_active":"2023-10-09T06:32:31.523596+0000","last_peered":"2023-10-09T06:32:31.523596+0000","last_clean":"2023-10-09T06:32:31.523596+0000","last_became_active":"2023-10-09T06:32:31.511374+0000","last_became_peered":"2023-10-09T06:32:31.511374+0000","last_unstale":"2023-10-09T06:32:31.523596+0000","last_undegraded":"2023-10-09T06:32:31.523596+0000","last_fullsized":"2023-10-09T06:32:31.523596+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:38:54.218698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.505884+0000","last_change":"2023-10-09T06:32:30.515328+0000","last_active":"2023-10-09T06:32:31.505884+0000","last_peered":"2023-10-09T06:32:31.505884+0000","last_clean":"2023-10-09T06:32:31.505884+0000","last_became_active":"2023-10-09T06:32:30.513481+0000","last_became_peered":"2023-10-09T06:32:30.513481+0000","last_unstale":"2023-10-09T06:32:31.505884+0000","last_undegraded":"2023-10-09T06:32:31.505884+0000","last_fullsized":"2023-10-09T06:32:31.505884+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:05:29.788926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.523909+0000","last_change":"2023-10-09T06:32:31.520908+0000","last_active":"2023-10-09T06:32:31.523909+0000","last_peered":"2023-10-09T06:32:31.523909+0000","last_clean":"2023-10-09T06:32:31.523909+0000","last_became_active":"2023-10-09T06:32:31.520623+0000","last_became_peered":"2023-10-09T06:32:31.520623+0000","last_unstale":"2023-10-09T06:32:31.523909+0000","last_undegraded":"2023-10-09T06:32:31.523909+0000","last_fullsized":"2023-10-09T06:32:31.523909+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:04:39.865936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.501217+0000","last_change":"2023-10-09T06:32:30.509898+0000","last_active":"2023-10-09T06:32:31.501217+0000","last_peered":"2023-10-09T06:32:31.501217+0000","last_clean":"2023-10-09T06:32:31.501217+0000","last_became_active":"2023-10-09T06:32:30.509639+0000","last_became_peered":"2023-10-09T06:32:30.509639+0000","last_unstale":"2023-10-09T06:32:31.501217+0000","last_undegraded":"2023-10-09T06:32:31.501217+0000","last_fullsized":"2023-10-09T06:32:31.501217+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:24:42.834801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.525465+0000","last_change":"2023-10-09T06:32:31.523514+0000","last_active":"2023-10-09T06:32:31.525465+0000","last_peered":"2023-10-09T06:32:31.525465+0000","last_clean":"2023-10-09T06:32:31.525465+0000","last_became_active":"2023-10-09T06:32:31.523349+0000","last_became_peered":"2023-10-09T06:32:31.523349+0000","last_unstale":"2023-10-09T06:32:31.525465+0000","last_undegraded":"2023-10-09T06:32:31.525465+0000","last_fullsized":"2023-10-09T06:32:31.525465+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:30.487000+0000","last_clean_scrub_stamp":"2023-10-09T06:32:30.487000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:11:52.630319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"62'2","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.525433+0000","last_change":"2023-10-09T06:32:30.512857+0000","last_active":"2023-10-09T06:32:31.525433+0000","last_peered":"2023-10-09T06:32:31.525433+0000","last_clean":"2023-10-09T06:32:31.525433+0000","last_became_active":"2023-10-09T06:32:30.512646+0000","last_became_peered":"2023-10-09T06:32:30.512646+0000","last_unstale":"2023-10-09T06:32:31.525433+0000","last_undegraded":"2023-10-09T06:32:31.525433+0000","last_fullsized":"2023-10-09T06:32:31.525433+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:32:29.484745+0000","last_clean_scrub_stamp":"2023-10-09T06:32:29.484745+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:59:45.594900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":200,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-09T06:32:31.508856+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:32:31.508856+0000","last_peered":"2023-10-09T06:32:31.508856+0000","last_clean":"2023-10-09T06:32:31.508856+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:32:31.508856+0000","last_undegraded":"2023-10-09T06:32:31.508856+0000","last_fullsized":"2023-10-09T06:32:31.508856+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298624,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073216,"statfs":{"total":96636764160,"available":96330973184,"internally_reserved":0,"allocated":1368064,"data_stored":918249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.501,"15min":0.501},"min":{"1min":0.36099999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.64600000000000002,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.53600000000000003}]},{"osd":1,"last update":"Mon Oct 9 06:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.65500000000000003,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52700000000000002,"15min":0.52500000000000002},"min":{"1min":0.39000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.69199999999999995,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.61399999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.50900000000000001,"15min":0.505},"min":{"1min":0.41099999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.65600000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.58099999999999996}]},{"osd":3,"last update":"Mon Oct 9 06:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.55100000000000005,"15min":0.55800000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60499999999999998,"15min":0.61099999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.68700000000000006,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.50900000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.52900000000000003,"15min":0.53900000000000003},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.626,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.622,"15min":0.628},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.70199999999999996,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.60199999999999998}]},{"osd":5,"last update":"Mon Oct 9 06:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.58299999999999996,"15min":0.59499999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.72999999999999998,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.626,"15min":0.63400000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71899999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.67900000000000005}]},{"osd":6,"last update":"Mon Oct 9 06:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.60199999999999998,"15min":0.61499999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71399999999999997,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.63100000000000001,"15min":0.64400000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.77800000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.63100000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298044,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073796,"statfs":{"total":96636764160,"available":96331567104,"internally_reserved":0,"allocated":774144,"data_stored":327385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.496,"15min":0.501},"min":{"1min":0.33200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.61599999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.51300000000000001,"15min":0.51100000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.66300000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.47699999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.51300000000000001,"15min":0.51900000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.59899999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.61399999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.58399999999999996}]},{"osd":2,"last update":"Mon Oct 9 06:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51300000000000001,"15min":0.51600000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.65100000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69399999999999995,"5min":0.752,"15min":0.752},"last":0.52500000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.54300000000000004,"15min":0.55100000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78400000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.73899999999999999,"15min":0.76700000000000002},"min":{"1min":0.42199999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.74399999999999999,"5min":4.7510000000000003,"15min":4.7510000000000003},"last":0.61799999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.48199999999999998,"15min":0.48499999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70599999999999996,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.40100000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.73599999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53200000000000003,"15min":0.53500000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.67900000000000005,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.60199999999999998,"15min":0.60799999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.73699999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.68300000000000005}]},{"osd":7,"last update":"Mon Oct 9 06:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.58899999999999997,"15min":0.59899999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.68899999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58899999999999997,"15min":0.58999999999999997},"min":{"1min":0.46400000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.71499999999999997,"5min":0.745,"15min":0.745},"last":0.72099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298056,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073784,"statfs":{"total":96636764160,"available":96331554816,"internally_reserved":0,"allocated":786432,"data_stored":329417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.307,"15min":0.309},"min":{"1min":0.215,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.40100000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.36099999999999999,"15min":0.36399999999999999},"min":{"1min":0.253,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.40100000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.32800000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.35199999999999998,"15min":0.35399999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.379},{"interface":"front","average":{"1min":0.376,"5min":0.38500000000000001,"15min":0.38700000000000001},"min":{"1min":0.29299999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.47699999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.39100000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.26100000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.59799999999999998,"15min":0.64000000000000001},"min":{"1min":0.23799999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.53000000000000003,"5min":5.8529999999999998,"15min":5.8529999999999998},"last":0.34799999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56899999999999995,"15min":0.57199999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79400000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.46500000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.72099999999999997,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.59399999999999997}]},{"osd":5,"last update":"Mon Oct 9 06:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.79100000000000004,"15min":0.83499999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.70899999999999996,"5min":5.9459999999999997,"15min":5.9459999999999997},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.60499999999999998,"15min":0.61299999999999999},"min":{"1min":0.45000000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.71199999999999997,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.56499999999999995}]},{"osd":6,"last update":"Mon Oct 9 06:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.76300000000000001,"15min":0.80400000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.69199999999999995,"5min":5.9160000000000004,"15min":5.9160000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.75600000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.68300000000000005}]},{"osd":7,"last update":"Mon Oct 9 06:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.58599999999999997,"15min":0.59499999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.72799999999999998,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.80300000000000005,"15min":0.84999999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.74199999999999999,"5min":5.9729999999999999,"15min":5.9729999999999999},"last":0.64300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298616,"kb_used_data":1328,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073224,"statfs":{"total":96636764160,"available":96330981376,"internally_reserved":0,"allocated":1359872,"data_stored":917661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.35999999999999999,"15min":0.35399999999999998},"min":{"1min":0.24299999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.28699999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.36699999999999999,"15min":0.36799999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54500000000000004,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.39900000000000002,"15min":0.39500000000000002},"min":{"1min":0.307,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.36899999999999999}]},{"osd":3,"last update":"Mon Oct 9 06:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.40699999999999997,"15min":0.41399999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.52900000000000003,"5min":1.554,"15min":1.554},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.41399999999999998,"15min":0.41599999999999998},"min":{"1min":0.28399999999999997,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.60899999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.34899999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56499999999999995,"15min":0.55600000000000005},"min":{"1min":0.39300000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.77600000000000002,"15min":0.81299999999999994},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81799999999999995,"5min":5.484,"15min":5.484},"last":0.55600000000000005}]},{"osd":5,"last update":"Mon Oct 9 06:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.78500000000000003,"15min":0.81899999999999995},"min":{"1min":0.40600000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84399999999999997,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.98699999999999999,"5min":1.113,"15min":1.113},"last":0.47799999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.46999999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0269999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.61899999999999999,"15min":0.61199999999999999},"min":{"1min":0.41799999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.54500000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.434,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.81299999999999994,"15min":0.85099999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.873,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.48999999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298048,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073792,"statfs":{"total":96636764160,"available":96331563008,"internally_reserved":0,"allocated":778240,"data_stored":328829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":7.2699999999999996,"5min":7.2699999999999996,"15min":7.2699999999999996},"last":0.34399999999999997}]},{"osd":1,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":7.327,"5min":7.327,"15min":7.327},"last":0.25800000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.314}]},{"osd":4,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":7.2389999999999999,"5min":7.2389999999999999,"15min":7.2389999999999999},"last":0.47399999999999998}]},{"osd":5,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":10.890000000000001,"5min":10.890000000000001,"15min":10.890000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52100000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":7.2999999999999998,"5min":7.2999999999999998,"15min":7.2999999999999998},"last":0.627},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.54700000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":7.1879999999999997,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.48299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298044,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073796,"statfs":{"total":96636764160,"available":96331567104,"internally_reserved":0,"allocated":774144,"data_stored":327405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.36099999999999999,"15min":0.32900000000000001},"min":{"1min":0.245,"5min":0.22,"15min":0.22},"max":{"1min":5.8700000000000001,"5min":5.8700000000000001,"15min":5.8700000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.39700000000000002,"15min":0.37},"min":{"1min":0.249,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":5.6769999999999996,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.32700000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.377,"15min":0.35699999999999998},"min":{"1min":0.255,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":4.0469999999999997,"5min":4.0469999999999997,"15min":4.0469999999999997},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.41199999999999998,"15min":0.39100000000000001},"min":{"1min":0.29799999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":4.1509999999999998,"5min":4.1509999999999998,"15min":4.1509999999999998},"last":0.38300000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.39500000000000002,"15min":0.38100000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":3.1139999999999999,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.40200000000000002,"15min":0.39000000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":3.3180000000000001,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.307}]},{"osd":4,"last update":"Mon Oct 9 06:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.58999999999999997,"15min":0.57399999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.0899999999999999,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":0.432},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.58099999999999996,"15min":0.56499999999999995},"min":{"1min":0.39300000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":3.0710000000000002,"5min":3.0710000000000002,"15min":3.0710000000000002},"last":0.46300000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.61899999999999999,"15min":0.58899999999999997},"min":{"1min":0.42499999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":6.2300000000000004,"5min":6.2300000000000004,"15min":6.2300000000000004},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":3.0219999999999998,"5min":3.0219999999999998,"15min":3.0219999999999998},"last":0.50700000000000001}]},{"osd":6,"last update":"Mon Oct 9 06:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":2.6850000000000001,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.88,"5min":0.68200000000000005,"15min":0.64900000000000002},"min":{"1min":0.443,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":6.5970000000000004,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.61099999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.63500000000000001,"15min":0.61099999999999999},"min":{"1min":0.47799999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":4.2629999999999999,"5min":4.2629999999999999,"15min":4.2629999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.872,"5min":0.64800000000000002,"15min":0.61099999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":7.3650000000000002,"5min":7.3650000000000002,"15min":7.3650000000000002},"last":0.57999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298040,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073800,"statfs":{"total":96636764160,"available":96331571200,"internally_reserved":0,"allocated":770048,"data_stored":327363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73499999999999999,"5min":1.083,"15min":1.083},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69999999999999996,"5min":1.054,"15min":1.054},"last":0.47299999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48899999999999999,"15min":0.48799999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.499}]},{"osd":2,"last update":"Mon Oct 9 06:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.54100000000000004,"15min":0.54700000000000004},"min":{"1min":0.33700000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70699999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.60499999999999998,"15min":0.60999999999999999},"min":{"1min":0.443,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.78100000000000003,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.63900000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55600000000000005,"15min":0.56200000000000006},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.746,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.58899999999999997,"15min":0.59499999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.749,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.61399999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.55700000000000005,"15min":0.57199999999999995},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.74299999999999999,"5min":0.998,"15min":0.998},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.61099999999999999,"15min":0.621},"min":{"1min":0.39200000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76700000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.72999999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.32500000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.78600000000000003,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60799999999999998,"15min":0.61399999999999999},"min":{"1min":0.33500000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79400000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.745}]},{"osd":7,"last update":"Mon Oct 9 06:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.57799999999999996,"15min":0.58499999999999996},"min":{"1min":0.34599999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77100000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.65500000000000003,"15min":0.66700000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80000000000000004,"5min":1.175,"15min":1.175},"last":0.71499999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298040,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073800,"statfs":{"total":96636764160,"available":96331571200,"internally_reserved":0,"allocated":770048,"data_stored":327383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.56399999999999995}]},{"osd":1,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.58299999999999996}]},{"osd":2,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59599999999999997}]},{"osd":3,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.53800000000000003}]},{"osd":4,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.625},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.64000000000000001}]},{"osd":6,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.66600000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.69399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:32:40.450 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-09T06:32:40.450 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-09T06:32:40.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.606+0000 7fbbbfcf3700 1 Processor -- start 2023-10-09T06:32:40.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.606+0000 7fbbbfcf3700 1 -- start start 2023-10-09T06:32:40.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.607+0000 7fbbbfcf3700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbbb806bcd0 0x7fbbb80c9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:40.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.607+0000 7fbbbfcf3700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb815b790 con 0x7fbbb81414c0 2023-10-09T06:32:40.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.607+0000 7fbbbfcf3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb815ba70 con 0x7fbbb8158d40 2023-10-09T06:32:40.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.607+0000 7fbbbfcf3700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb815bd50 con 0x7fbbb806bcd0 2023-10-09T06:32:40.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.607+0000 7fbbbda8f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbbb806bcd0 0x7fbbb80c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:40.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.607+0000 7fbbbe290700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fbbb81414c0 0x7fbbb80cc290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:42778/0 (socket says 172.21.15.163:42778) 2023-10-09T06:32:40.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.607+0000 7fbbbe290700 1 -- 172.21.15.163:0/3571617247 learned_addr learned my addr 172.21.15.163:0/3571617247 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:40.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.607+0000 7fbbbda8f700 1 -- 172.21.15.163:0/3571617247 >> v1:172.21.15.181:6789/0 conn(0x7fbbb8158d40 legacy=0x7fbbb8140db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:40.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.607+0000 7fbbbda8f700 1 -- 172.21.15.163:0/3571617247 >> v1:172.21.15.163:6789/0 conn(0x7fbbb81414c0 legacy=0x7fbbb80cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:40.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.607+0000 7fbbbda8f700 1 -- 172.21.15.163:0/3571617247 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb815c030 con 0x7fbbb806bcd0 2023-10-09T06:32:40.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.607+0000 7fbbbca8d700 1 -- 172.21.15.163:0/3571617247 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3315754167 0 0) 0x7fbbb815b790 con 0x7fbbb81414c0 2023-10-09T06:32:40.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.607+0000 7fbbbda8f700 1 --2- 172.21.15.163:0/3571617247 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbb806bcd0 0x7fbbb80c9230 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fbbb40098e0 tx=0x7fbbb40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f005cf7a5af2fbc server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:40.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.607+0000 7fbbbca8d700 1 -- 172.21.15.163:0/3571617247 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4009e80 con 0x7fbbb806bcd0 2023-10-09T06:32:40.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.607+0000 7fbbbca8d700 1 -- 172.21.15.163:0/3571617247 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb400b820 con 0x7fbbb806bcd0 2023-10-09T06:32:40.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.607+0000 7fbbbca8d700 1 -- 172.21.15.163:0/3571617247 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb400bba0 con 0x7fbbb806bcd0 2023-10-09T06:32:40.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.608+0000 7fbbbfcf3700 1 -- 172.21.15.163:0/3571617247 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbb806bcd0 msgr2=0x7fbbb80c9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:40.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.608+0000 7fbbbfcf3700 1 --2- 172.21.15.163:0/3571617247 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbb806bcd0 0x7fbbb80c9230 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fbbb40098e0 tx=0x7fbbb40091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:40.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.608+0000 7fbbbfcf3700 1 -- 172.21.15.163:0/3571617247 shutdown_connections 2023-10-09T06:32:40.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.608+0000 7fbbbfcf3700 1 --2- 172.21.15.163:0/3571617247 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbb806bcd0 0x7fbbb80c9230 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:40.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.608+0000 7fbbbfcf3700 1 -- 172.21.15.163:0/3571617247 >> 172.21.15.163:0/3571617247 conn(0x7fbbb80ce5c0 msgr2=0x7fbbb80ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:40.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.608+0000 7fbbbfcf3700 1 -- 172.21.15.163:0/3571617247 shutdown_connections 2023-10-09T06:32:40.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.608+0000 7fbbbfcf3700 1 -- 172.21.15.163:0/3571617247 wait complete. 2023-10-09T06:32:40.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.609+0000 7fbbbfcf3700 1 Processor -- start 2023-10-09T06:32:40.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.609+0000 7fbbbfcf3700 1 -- start start 2023-10-09T06:32:40.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.610+0000 7fbbbfcf3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbb806bcd0 0x7fbbb80c5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:40.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.610+0000 7fbbbfcf3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbb81414c0 0x7fbbb80c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:40.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.610+0000 7fbbbfcf3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbb8158d40 0x7fbbb813c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:40.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.610+0000 7fbbbfcf3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb80c5d50 con 0x7fbbb8158d40 2023-10-09T06:32:40.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.610+0000 7fbbbfcf3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb80c6030 con 0x7fbbb806bcd0 2023-10-09T06:32:40.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.610+0000 7fbbbfcf3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb80c6310 con 0x7fbbb81414c0 2023-10-09T06:32:40.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.610+0000 7fbbbe290700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbb8158d40 0x7fbbb813c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:40.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.610+0000 7fbbbe290700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbb8158d40 0x7fbbb813c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:34416/0 (socket says 172.21.15.163:34416) 2023-10-09T06:32:40.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.610+0000 7fbbbe290700 1 -- 172.21.15.163:0/2979709417 learned_addr learned my addr 172.21.15.163:0/2979709417 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:40.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.610+0000 7fbbbda8f700 1 --2- 172.21.15.163:0/2979709417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbb806bcd0 0x7fbbb80c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:40.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.610+0000 7fbbbd28e700 1 --2- 172.21.15.163:0/2979709417 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbb81414c0 0x7fbbb80c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:40.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.610+0000 7fbbbe290700 1 -- 172.21.15.163:0/2979709417 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbb81414c0 msgr2=0x7fbbb80c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:40.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.610+0000 7fbbbe290700 1 --2- 172.21.15.163:0/2979709417 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbb81414c0 0x7fbbb80c7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:40.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.610+0000 7fbbbe290700 1 -- 172.21.15.163:0/2979709417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbb806bcd0 msgr2=0x7fbbb80c5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:40.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.610+0000 7fbbbe290700 1 --2- 172.21.15.163:0/2979709417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbb806bcd0 0x7fbbb80c5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:40.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.610+0000 7fbbbe290700 1 -- 172.21.15.163:0/2979709417 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb4009420 con 0x7fbbb8158d40 2023-10-09T06:32:40.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.610+0000 7fbbbd28e700 1 --2- 172.21.15.163:0/2979709417 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbb81414c0 0x7fbbb80c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:32:40.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.610+0000 7fbbbe290700 1 --2- 172.21.15.163:0/2979709417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbb8158d40 0x7fbbb813c0b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000ccc0 tx=0x7fbbb0001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:40.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.610+0000 7fbbadffb700 1 -- 172.21.15.163:0/2979709417 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb000ab10 con 0x7fbbb8158d40 2023-10-09T06:32:40.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.610+0000 7fbbadffb700 1 -- 172.21.15.163:0/2979709417 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb000ac70 con 0x7fbbb8158d40 2023-10-09T06:32:40.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.610+0000 7fbbbfcf3700 1 -- 172.21.15.163:0/2979709417 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb80c65f0 con 0x7fbbb8158d40 2023-10-09T06:32:40.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.610+0000 7fbbadffb700 1 -- 172.21.15.163:0/2979709417 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb00070f0 con 0x7fbbb8158d40 2023-10-09T06:32:40.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.611+0000 7fbbbfcf3700 1 -- 172.21.15.163:0/2979709417 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb80c81e0 con 0x7fbbb8158d40 2023-10-09T06:32:40.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.612+0000 7fbbbfcf3700 1 -- 172.21.15.163:0/2979709417 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb813da80 con 0x7fbbb8158d40 2023-10-09T06:32:40.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.612+0000 7fbbadffb700 1 -- 172.21.15.163:0/2979709417 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fbbb00074e0 con 0x7fbbb8158d40 2023-10-09T06:32:40.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.612+0000 7fbbadffb700 1 --2- 172.21.15.163:0/2979709417 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb9c067040 0x7fbb9c0694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:40.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.612+0000 7fbbadffb700 1 -- 172.21.15.163:0/2979709417 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbbb0084a20 con 0x7fbbb8158d40 2023-10-09T06:32:40.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.613+0000 7fbbbda8f700 1 --2- 172.21.15.163:0/2979709417 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb9c067040 0x7fbb9c0694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:40.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.613+0000 7fbbbda8f700 1 --2- 172.21.15.163:0/2979709417 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb9c067040 0x7fbb9c0694f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbbb40030c0 tx=0x7fbbb4016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:40.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.616+0000 7fbbadffb700 1 -- 172.21.15.163:0/2979709417 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbbb813da80 con 0x7fbbb8158d40 2023-10-09T06:32:40.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.877+0000 7fbbbfcf3700 1 -- 172.21.15.163:0/2979709417 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fbbb813da80 con 0x7fbbb8158d40 2023-10-09T06:32:40.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.880+0000 7fbbadffb700 1 -- 172.21.15.163:0/2979709417 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7fbbb813da80 con 0x7fbbb8158d40 2023-10-09T06:32:40.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.883+0000 7fbbbfcf3700 1 -- 172.21.15.163:0/2979709417 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb9c067040 msgr2=0x7fbb9c0694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:40.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.883+0000 7fbbbfcf3700 1 --2- 172.21.15.163:0/2979709417 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb9c067040 0x7fbb9c0694f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbbb40030c0 tx=0x7fbbb4016000 comp rx=0 tx=0).stop 2023-10-09T06:32:40.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.883+0000 7fbbbfcf3700 1 -- 172.21.15.163:0/2979709417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbb8158d40 msgr2=0x7fbbb813c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:40.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.883+0000 7fbbbfcf3700 1 --2- 172.21.15.163:0/2979709417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbb8158d40 0x7fbbb813c0b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000ccc0 tx=0x7fbbb0001a20 comp rx=0 tx=0).stop 2023-10-09T06:32:40.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.883+0000 7fbbbfcf3700 1 -- 172.21.15.163:0/2979709417 shutdown_connections 2023-10-09T06:32:40.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.883+0000 7fbbbfcf3700 1 --2- 172.21.15.163:0/2979709417 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb9c067040 0x7fbb9c0694f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:40.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.883+0000 7fbbbfcf3700 1 --2- 172.21.15.163:0/2979709417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbb8158d40 0x7fbbb813c0b0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:40.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.883+0000 7fbbbfcf3700 1 --2- 172.21.15.163:0/2979709417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbb806bcd0 0x7fbbb80c5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:40.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.883+0000 7fbbbfcf3700 1 --2- 172.21.15.163:0/2979709417 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbb81414c0 0x7fbbb80c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:40.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.883+0000 7fbbbfcf3700 1 -- 172.21.15.163:0/2979709417 >> 172.21.15.163:0/2979709417 conn(0x7fbbb80ce5c0 msgr2=0x7fbbb80cc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:40.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.883+0000 7fbbbfcf3700 1 -- 172.21.15.163:0/2979709417 shutdown_connections 2023-10-09T06:32:40.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:40.883+0000 7fbbbfcf3700 1 -- 172.21.15.163:0/2979709417 wait complete. 2023-10-09T06:32:40.884 INFO:teuthology.orchestra.run.smithi163.stderr:updated caps for client.0 2023-10-09T06:32:40.894 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:32:41.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.008+0000 7ff675019700 1 Processor -- start 2023-10-09T06:32:41.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.008+0000 7ff675019700 1 -- start start 2023-10-09T06:32:41.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.009+0000 7ff675019700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff670159c60 0x7ff67015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.009+0000 7ff675019700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff67015db10 0x7ff670069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.009+0000 7ff675019700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff67015a160 con 0x7ff670159890 2023-10-09T06:32:41.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.009+0000 7ff675019700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6700d13a0 con 0x7ff67015db10 2023-10-09T06:32:41.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.009+0000 7ff675019700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67015e140 con 0x7ff670159c60 2023-10-09T06:32:41.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.009+0000 7ff66f59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff67015db10 0x7ff670069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.009+0000 7ff66f59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff67015db10 0x7ff670069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:34420/0 (socket says 172.21.15.163:34420) 2023-10-09T06:32:41.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.009+0000 7ff66f59e700 1 -- 172.21.15.163:0/3393689470 learned_addr learned my addr 172.21.15.163:0/3393689470 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:41.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.009+0000 7ff66659c700 1 --2- 172.21.15.163:0/3393689470 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff670159c60 0x7ff67015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.009+0000 7ff66f59e700 1 -- 172.21.15.163:0/3393689470 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff670159c60 msgr2=0x7ff67015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:41.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.009+0000 7ff66f59e700 1 --2- 172.21.15.163:0/3393689470 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff670159c60 0x7ff67015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.009+0000 7ff66f59e700 1 -- 172.21.15.163:0/3393689470 >> v1:172.21.15.181:6789/0 conn(0x7ff670159890 legacy=0x7ff67015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:41.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.010+0000 7ff66f59e700 1 -- 172.21.15.163:0/3393689470 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67015a3d0 con 0x7ff67015db10 2023-10-09T06:32:41.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.010+0000 7ff66f59e700 1 --2- 172.21.15.163:0/3393689470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff67015db10 0x7ff670069560 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff668009f20 tx=0x7ff668009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68bbda56670cab5 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:41.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.010+0000 7ff66e59c700 1 -- 172.21.15.163:0/3393689470 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66800e070 con 0x7ff67015db10 2023-10-09T06:32:41.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.010+0000 7ff66e59c700 1 -- 172.21.15.163:0/3393689470 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66800ba80 con 0x7ff67015db10 2023-10-09T06:32:41.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.010+0000 7ff66e59c700 1 -- 172.21.15.163:0/3393689470 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6680092e0 con 0x7ff67015db10 2023-10-09T06:32:41.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.010+0000 7ff675019700 1 -- 172.21.15.163:0/3393689470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff67015db10 msgr2=0x7ff670069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:41.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.010+0000 7ff675019700 1 --2- 172.21.15.163:0/3393689470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff67015db10 0x7ff670069560 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff668009f20 tx=0x7ff668009840 comp rx=0 tx=0).stop 2023-10-09T06:32:41.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.010+0000 7ff675019700 1 -- 172.21.15.163:0/3393689470 shutdown_connections 2023-10-09T06:32:41.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.010+0000 7ff675019700 1 --2- 172.21.15.163:0/3393689470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff67015db10 0x7ff670069560 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.010+0000 7ff675019700 1 --2- 172.21.15.163:0/3393689470 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff670159c60 0x7ff67015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.010+0000 7ff675019700 1 -- 172.21.15.163:0/3393689470 >> 172.21.15.163:0/3393689470 conn(0x7ff670068e20 msgr2=0x7ff6700d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:41.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.010+0000 7ff675019700 1 -- 172.21.15.163:0/3393689470 shutdown_connections 2023-10-09T06:32:41.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.010+0000 7ff675019700 1 -- 172.21.15.163:0/3393689470 wait complete. 2023-10-09T06:32:41.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.012+0000 7ff675019700 1 Processor -- start 2023-10-09T06:32:41.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.012+0000 7ff675019700 1 -- start start 2023-10-09T06:32:41.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.012+0000 7ff675019700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff670159890 0x7ff6700b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.012+0000 7ff675019700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff670159c60 0x7ff67016d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.012+0000 7ff675019700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff67015db10 0x7ff67016d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.012+0000 7ff675019700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6700cff40 con 0x7ff670159890 2023-10-09T06:32:41.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.012+0000 7ff675019700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6700b1750 con 0x7ff670159c60 2023-10-09T06:32:41.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.012+0000 7ff675019700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6700cfc60 con 0x7ff67015db10 2023-10-09T06:32:41.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.012+0000 7ff66ed9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff670159890 0x7ff6700b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.013+0000 7ff66ed9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff670159890 0x7ff6700b1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:34422/0 (socket says 172.21.15.163:34422) 2023-10-09T06:32:41.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.013+0000 7ff66ed9d700 1 -- 172.21.15.163:0/1832207988 learned_addr learned my addr 172.21.15.163:0/1832207988 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:41.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.013+0000 7ff66659c700 1 --2- 172.21.15.163:0/1832207988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff670159c60 0x7ff67016d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.013+0000 7ff66f59e700 1 --2- 172.21.15.163:0/1832207988 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff67015db10 0x7ff67016d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.013+0000 7ff66ed9d700 1 -- 172.21.15.163:0/1832207988 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff67015db10 msgr2=0x7ff67016d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:41.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.013+0000 7ff66ed9d700 1 --2- 172.21.15.163:0/1832207988 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff67015db10 0x7ff67016d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.013+0000 7ff66ed9d700 1 -- 172.21.15.163:0/1832207988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff670159c60 msgr2=0x7ff67016d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:41.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.013+0000 7ff66ed9d700 1 --2- 172.21.15.163:0/1832207988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff670159c60 0x7ff67016d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.013+0000 7ff66ed9d700 1 -- 172.21.15.163:0/1832207988 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff668009ae0 con 0x7ff670159890 2023-10-09T06:32:41.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.013+0000 7ff66ed9d700 1 --2- 172.21.15.163:0/1832207988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff670159890 0x7ff6700b1be0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff66000c590 tx=0x7ff66000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:41.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.013+0000 7ff6677fe700 1 -- 172.21.15.163:0/1832207988 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff660011590 con 0x7ff670159890 2023-10-09T06:32:41.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.013+0000 7ff6677fe700 1 -- 172.21.15.163:0/1832207988 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff660011710 con 0x7ff670159890 2023-10-09T06:32:41.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.013+0000 7ff675019700 1 -- 172.21.15.163:0/1832207988 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6700b1100 con 0x7ff670159890 2023-10-09T06:32:41.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.013+0000 7ff6677fe700 1 -- 172.21.15.163:0/1832207988 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff660011ad0 con 0x7ff670159890 2023-10-09T06:32:41.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.013+0000 7ff675019700 1 -- 172.21.15.163:0/1832207988 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6700cf8f0 con 0x7ff670159890 2023-10-09T06:32:41.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.014+0000 7ff675019700 1 -- 172.21.15.163:0/1832207988 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6700b15a0 con 0x7ff670159890 2023-10-09T06:32:41.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.014+0000 7ff6677fe700 1 -- 172.21.15.163:0/1832207988 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7ff66001b3b0 con 0x7ff670159890 2023-10-09T06:32:41.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.015+0000 7ff6677fe700 1 --2- 172.21.15.163:0/1832207988 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff650066fc0 0x7ff650069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.015+0000 7ff6677fe700 1 -- 172.21.15.163:0/1832207988 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff660083a90 con 0x7ff670159890 2023-10-09T06:32:41.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.015+0000 7ff66659c700 1 --2- 172.21.15.163:0/1832207988 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff650066fc0 0x7ff650069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.015+0000 7ff66659c700 1 --2- 172.21.15.163:0/1832207988 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff650066fc0 0x7ff650069470 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff6701592b0 tx=0x7ff658008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:41.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.019+0000 7ff6677fe700 1 -- 172.21.15.163:0/1832207988 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff660050d10 con 0x7ff670159890 2023-10-09T06:32:41.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.247+0000 7ff675019700 1 -- 172.21.15.163:0/1832207988 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff67004c3c0 con 0x7ff670159890 2023-10-09T06:32:41.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.248+0000 7ff6677fe700 1 -- 172.21.15.163:0/1832207988 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4608 (secure 0 0 0) 0x7ff660017070 con 0x7ff670159890 2023-10-09T06:32:41.248 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:32:41.248 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":40,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5407,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3673547262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3673547262},{"type":"v1","addr":"172.21.15.181:6837","nonce":3673547262}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5410,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1705122825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1705122825},{"type":"v1","addr":"172.21.15.181:6836","nonce":1705122825}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/596332711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":596332711},{"type":"v1","addr":"172.21.15.163:6834","nonce":596332711}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:32:30.882925+0000","modified":"2023-10-09T06:32:34.526904+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/3415100865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3415100865},{"type":"v1","addr":"172.21.15.163:6835","nonce":3415100865}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-09T06:32:41.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.250+0000 7ff675019700 1 -- 172.21.15.163:0/1832207988 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff650066fc0 msgr2=0x7ff650069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:41.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.250+0000 7ff675019700 1 --2- 172.21.15.163:0/1832207988 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff650066fc0 0x7ff650069470 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff6701592b0 tx=0x7ff658008040 comp rx=0 tx=0).stop 2023-10-09T06:32:41.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.250+0000 7ff675019700 1 -- 172.21.15.163:0/1832207988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff670159890 msgr2=0x7ff6700b1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:41.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.250+0000 7ff675019700 1 --2- 172.21.15.163:0/1832207988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff670159890 0x7ff6700b1be0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff66000c590 tx=0x7ff66000c8a0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.250+0000 7ff675019700 1 -- 172.21.15.163:0/1832207988 shutdown_connections 2023-10-09T06:32:41.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.250+0000 7ff675019700 1 --2- 172.21.15.163:0/1832207988 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff650066fc0 0x7ff650069470 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.250+0000 7ff675019700 1 --2- 172.21.15.163:0/1832207988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff670159890 0x7ff6700b1be0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.250+0000 7ff675019700 1 --2- 172.21.15.163:0/1832207988 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff67015db10 0x7ff67016d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.250+0000 7ff675019700 1 --2- 172.21.15.163:0/1832207988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff670159c60 0x7ff67016d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.250+0000 7ff675019700 1 -- 172.21.15.163:0/1832207988 >> 172.21.15.163:0/1832207988 conn(0x7ff670068e20 msgr2=0x7ff670153070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:41.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.250+0000 7ff675019700 1 -- 172.21.15.163:0/1832207988 shutdown_connections 2023-10-09T06:32:41.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.250+0000 7ff675019700 1 -- 172.21.15.163:0/1832207988 wait complete. 2023-10-09T06:32:41.251 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 40 2023-10-09T06:32:41.664 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 40, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-09T06:32:30.882925+0000', 'modified': '2023-10-09T06:32:34.526904+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5748}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5748': {'gid': 5748, 'name': 'c', 'rank': 0, 'incarnation': 36, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.163:6835/3415100865', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6833', 'nonce': 3415100865}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 3415100865}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-09T06:32:41.664 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-09T06:32:41.664 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-09T06:32:41.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.781+0000 7fc300695700 1 Processor -- start 2023-10-09T06:32:41.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.781+0000 7fc300695700 1 -- start start 2023-10-09T06:32:41.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.782+0000 7fc300695700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2f806bcd0 0x7fc2f80caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.782+0000 7fc300695700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc2f815df50 0x7fc2f815e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.782+0000 7fc300695700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2f815d7e0 con 0x7fc2f806c0a0 2023-10-09T06:32:41.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.782+0000 7fc300695700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2f815dac0 con 0x7fc2f815df50 2023-10-09T06:32:41.785 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.782+0000 7fc300695700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f815aa30 con 0x7fc2f806bcd0 2023-10-09T06:32:41.785 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.783+0000 7fc2fe431700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2f806bcd0 0x7fc2f80caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.785 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.783+0000 7fc2fec32700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc2f815df50 0x7fc2f815e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.785 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.783+0000 7fc2fdc30700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc2f806c0a0 0x7fc2f815a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:57418/0 (socket says 172.21.15.163:57418) 2023-10-09T06:32:41.785 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.783+0000 7fc2fdc30700 1 -- 172.21.15.163:0/1211283952 learned_addr learned my addr 172.21.15.163:0/1211283952 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:41.785 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.783+0000 7fc2fec32700 1 -- 172.21.15.163:0/1211283952 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2f806bcd0 msgr2=0x7fc2f80caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:41.785 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.783+0000 7fc2fec32700 1 --2- 172.21.15.163:0/1211283952 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2f806bcd0 0x7fc2f80caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.786 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.783+0000 7fc2fec32700 1 -- 172.21.15.163:0/1211283952 >> v1:172.21.15.181:6789/0 conn(0x7fc2f806c0a0 legacy=0x7fc2f815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:41.786 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.783+0000 7fc2fec32700 1 -- 172.21.15.163:0/1211283952 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f815ad10 con 0x7fc2f815df50 2023-10-09T06:32:41.786 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.783+0000 7fc2fe431700 1 --2- 172.21.15.163:0/1211283952 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2f806bcd0 0x7fc2f80caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:32:41.786 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.783+0000 7fc2fec32700 1 --2- 172.21.15.163:0/1211283952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2f815df50 0x7fc2f815e3b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0009e60 tx=0x7fc2f0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcbd0c908c17062e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:41.786 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.783+0000 7fc2fd42f700 1 -- 172.21.15.163:0/1211283952 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f000e070 con 0x7fc2f815df50 2023-10-09T06:32:41.786 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.784+0000 7fc2fd42f700 1 -- 172.21.15.163:0/1211283952 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f000bb60 con 0x7fc2f815df50 2023-10-09T06:32:41.786 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.784+0000 7fc2fd42f700 1 -- 172.21.15.163:0/1211283952 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0012430 con 0x7fc2f815df50 2023-10-09T06:32:41.786 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.784+0000 7fc300695700 1 -- 172.21.15.163:0/1211283952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2f815df50 msgr2=0x7fc2f815e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:41.786 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.784+0000 7fc300695700 1 --2- 172.21.15.163:0/1211283952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2f815df50 0x7fc2f815e3b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0009e60 tx=0x7fc2f0009780 comp rx=0 tx=0).stop 2023-10-09T06:32:41.786 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.784+0000 7fc300695700 1 -- 172.21.15.163:0/1211283952 shutdown_connections 2023-10-09T06:32:41.786 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.784+0000 7fc300695700 1 --2- 172.21.15.163:0/1211283952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2f815df50 0x7fc2f815e3b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.787 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.784+0000 7fc300695700 1 --2- 172.21.15.163:0/1211283952 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2f806bcd0 0x7fc2f80caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.787 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.784+0000 7fc300695700 1 -- 172.21.15.163:0/1211283952 >> 172.21.15.163:0/1211283952 conn(0x7fc2f80ce600 msgr2=0x7fc2f80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:41.787 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.784+0000 7fc300695700 1 -- 172.21.15.163:0/1211283952 shutdown_connections 2023-10-09T06:32:41.787 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.784+0000 7fc300695700 1 -- 172.21.15.163:0/1211283952 wait complete. 2023-10-09T06:32:41.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.785+0000 7fc300695700 1 Processor -- start 2023-10-09T06:32:41.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.785+0000 7fc300695700 1 -- start start 2023-10-09T06:32:41.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.786+0000 7fc300695700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2f806bcd0 0x7fc2f816ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.786+0000 7fc300695700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2f806c0a0 0x7fc2f814e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.786+0000 7fc300695700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2f815df50 0x7fc2f80b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.786+0000 7fc300695700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f816b0a0 con 0x7fc2f815df50 2023-10-09T06:32:41.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.786+0000 7fc300695700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f80b29c0 con 0x7fc2f806bcd0 2023-10-09T06:32:41.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.786+0000 7fc300695700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f814ea10 con 0x7fc2f806c0a0 2023-10-09T06:32:41.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.786+0000 7fc2fec32700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2f815df50 0x7fc2f80b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.786+0000 7fc2fec32700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2f815df50 0x7fc2f80b2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:34438/0 (socket says 172.21.15.163:34438) 2023-10-09T06:32:41.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.786+0000 7fc2fec32700 1 -- 172.21.15.163:0/2642334487 learned_addr learned my addr 172.21.15.163:0/2642334487 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:41.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.786+0000 7fc2fe431700 1 --2- 172.21.15.163:0/2642334487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2f806bcd0 0x7fc2f816ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.786+0000 7fc2fdc30700 1 --2- 172.21.15.163:0/2642334487 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2f806c0a0 0x7fc2f814e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.786+0000 7fc2fec32700 1 -- 172.21.15.163:0/2642334487 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2f806c0a0 msgr2=0x7fc2f814e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:41.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.786+0000 7fc2fec32700 1 --2- 172.21.15.163:0/2642334487 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2f806c0a0 0x7fc2f814e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.786+0000 7fc2fec32700 1 -- 172.21.15.163:0/2642334487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2f806bcd0 msgr2=0x7fc2f816ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:41.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.786+0000 7fc2fec32700 1 --2- 172.21.15.163:0/2642334487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2f806bcd0 0x7fc2f816ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.786+0000 7fc2fec32700 1 -- 172.21.15.163:0/2642334487 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f4009420 con 0x7fc2f815df50 2023-10-09T06:32:41.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.786+0000 7fc2fdc30700 1 --2- 172.21.15.163:0/2642334487 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2f806c0a0 0x7fc2f814e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:41.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.787+0000 7fc2fec32700 1 --2- 172.21.15.163:0/2642334487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2f815df50 0x7fc2f80b2e50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8069220 tx=0x7fc2f0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:41.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.787+0000 7fc2ee7fc700 1 -- 172.21.15.163:0/2642334487 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0007000 con 0x7fc2f815df50 2023-10-09T06:32:41.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.787+0000 7fc2ee7fc700 1 -- 172.21.15.163:0/2642334487 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f0025440 con 0x7fc2f815df50 2023-10-09T06:32:41.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.787+0000 7fc300695700 1 -- 172.21.15.163:0/2642334487 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f0009a20 con 0x7fc2f815df50 2023-10-09T06:32:41.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.787+0000 7fc2ee7fc700 1 -- 172.21.15.163:0/2642334487 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f00257e0 con 0x7fc2f815df50 2023-10-09T06:32:41.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.787+0000 7fc300695700 1 -- 172.21.15.163:0/2642334487 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f81547f0 con 0x7fc2f815df50 2023-10-09T06:32:41.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.788+0000 7fc2ee7fc700 1 -- 172.21.15.163:0/2642334487 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fc2f0003050 con 0x7fc2f815df50 2023-10-09T06:32:41.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.788+0000 7fc2ee7fc700 1 --2- 172.21.15.163:0/2642334487 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc2dc066ff0 0x7fc2dc0694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.789+0000 7fc2ee7fc700 1 -- 172.21.15.163:0/2642334487 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc2f00862e0 con 0x7fc2f815df50 2023-10-09T06:32:41.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.789+0000 7fc2fe431700 1 --2- 172.21.15.163:0/2642334487 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc2dc066ff0 0x7fc2dc0694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.789+0000 7fc2fe431700 1 --2- 172.21.15.163:0/2642334487 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc2dc066ff0 0x7fc2dc0694a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80cf210 tx=0x7fc2f40091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:41.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.790+0000 7fc30998db80 1 Processor -- start 2023-10-09T06:32:41.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.790+0000 7fc30998db80 1 -- start start 2023-10-09T06:32:41.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.791+0000 7fc30998db80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5617f2cb7a50 0x5617f2cb7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.791+0000 7fc30998db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2d01d00 0x5617f2ce0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.791+0000 7fc30998db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617f2ce0f60 0x5617f2cb3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.791+0000 7fc30998db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f2c0ea70 con 0x5617f2ce0f60 2023-10-09T06:32:41.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.791+0000 7fc30998db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f2c0c270 con 0x5617f2d01d00 2023-10-09T06:32:41.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.791+0000 7fc30998db80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617f2c08060 con 0x5617f2cb7a50 2023-10-09T06:32:41.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.791+0000 7fc2fec32700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617f2ce0f60 0x5617f2cb3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.791+0000 7fc2fec32700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617f2ce0f60 0x5617f2cb3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:34448/0 (socket says 172.21.15.163:34448) 2023-10-09T06:32:41.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.791+0000 7fc2fec32700 1 -- 172.21.15.163:0/1614729619 learned_addr learned my addr 172.21.15.163:0/1614729619 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:41.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.791+0000 7fc2fe431700 1 --2- 172.21.15.163:0/1614729619 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5617f2cb7a50 0x5617f2cb7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.791+0000 7fc2fdc30700 1 --2- 172.21.15.163:0/1614729619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2d01d00 0x5617f2ce0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.792+0000 7fc2fec32700 1 -- 172.21.15.163:0/1614729619 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5617f2cb7a50 msgr2=0x5617f2cb7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:41.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.792+0000 7fc2fec32700 1 --2- 172.21.15.163:0/1614729619 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5617f2cb7a50 0x5617f2cb7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.792+0000 7fc2fec32700 1 -- 172.21.15.163:0/1614729619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2d01d00 msgr2=0x5617f2ce0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:41.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.792+0000 7fc2fec32700 1 --2- 172.21.15.163:0/1614729619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2d01d00 0x5617f2ce0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.792+0000 7fc2fec32700 1 -- 172.21.15.163:0/1614729619 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617f2aa2270 con 0x5617f2ce0f60 2023-10-09T06:32:41.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.792+0000 7fc2fe431700 1 --2- 172.21.15.163:0/1614729619 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5617f2cb7a50 0x5617f2cb7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:41.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.792+0000 7fc2fdc30700 1 --2- 172.21.15.163:0/1614729619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2d01d00 0x5617f2ce0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:41.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.792+0000 7fc2fec32700 1 --2- 172.21.15.163:0/1614729619 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617f2ce0f60 0x5617f2cb3cf0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00570a0 tx=0x7fc2f0056ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f301f47ebeb1751d server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:41.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.792+0000 7fc2db7fe700 1 -- 172.21.15.163:0/1614729619 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f000e040 con 0x5617f2ce0f60 2023-10-09T06:32:41.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.792+0000 7fc2db7fe700 1 -- 172.21.15.163:0/1614729619 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f00860e0 con 0x5617f2ce0f60 2023-10-09T06:32:41.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.792+0000 7fc2db7fe700 1 -- 172.21.15.163:0/1614729619 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f005a360 con 0x5617f2ce0f60 2023-10-09T06:32:41.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.792+0000 7fc30998db80 1 -- 172.21.15.163:0/1614729619 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617f2ce0f60 msgr2=0x5617f2cb3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:41.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.792+0000 7fc30998db80 1 --2- 172.21.15.163:0/1614729619 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617f2ce0f60 0x5617f2cb3cf0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00570a0 tx=0x7fc2f0056ee0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.792+0000 7fc30998db80 1 -- 172.21.15.163:0/1614729619 shutdown_connections 2023-10-09T06:32:41.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.792+0000 7fc30998db80 1 --2- 172.21.15.163:0/1614729619 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617f2ce0f60 0x5617f2cb3cf0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.792+0000 7fc30998db80 1 --2- 172.21.15.163:0/1614729619 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5617f2cb7a50 0x5617f2cb7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.792+0000 7fc30998db80 1 --2- 172.21.15.163:0/1614729619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2d01d00 0x5617f2ce0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.792+0000 7fc30998db80 1 -- 172.21.15.163:0/1614729619 >> 172.21.15.163:0/1614729619 conn(0x5617f2cb9370 msgr2=0x5617f2cb9740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:41.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.792+0000 7fc30998db80 1 -- 172.21.15.163:0/1614729619 shutdown_connections 2023-10-09T06:32:41.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.793+0000 7fc30998db80 1 -- 172.21.15.163:0/1614729619 wait complete. 2023-10-09T06:32:41.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.793+0000 7fc30998db80 1 -- start start 2023-10-09T06:32:41.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.793+0000 7fc30998db80 1 Processor -- start 2023-10-09T06:32:41.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.794+0000 7fc30998db80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5617f2cb7a50 0x5617f2d01730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.794+0000 7fc30998db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2ce0f60 0x5617f2cf9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.794+0000 7fc30998db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617f2d01d00 0x5617f2cf8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.794+0000 7fc30998db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f2bb3890 con 0x5617f2d01d00 2023-10-09T06:32:41.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.794+0000 7fc30998db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f2bceea0 con 0x5617f2ce0f60 2023-10-09T06:32:41.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.794+0000 7fc30998db80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617f2a8ae30 con 0x5617f2cb7a50 2023-10-09T06:32:41.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.794+0000 7fc30998db80 10 client.0 _ll_register_callbacks cb 0x5617f2c7b110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-09T06:32:41.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.794+0000 7fc2fec32700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617f2d01d00 0x5617f2cf8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.794+0000 7fc2fec32700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617f2d01d00 0x5617f2cf8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:34458/0 (socket says 172.21.15.163:34458) 2023-10-09T06:32:41.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.794+0000 7fc2fec32700 1 -- 172.21.15.163:0/4197390317 learned_addr learned my addr 172.21.15.163:0/4197390317 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:41.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.794+0000 7fc2fdc30700 1 --2- 172.21.15.163:0/4197390317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2ce0f60 0x5617f2cf9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.794+0000 7fc2fe431700 1 --2- 172.21.15.163:0/4197390317 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5617f2cb7a50 0x5617f2d01730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.794+0000 7fc2fec32700 1 -- 172.21.15.163:0/4197390317 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5617f2cb7a50 msgr2=0x5617f2d01730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:41.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.794+0000 7fc2fec32700 1 --2- 172.21.15.163:0/4197390317 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5617f2cb7a50 0x5617f2d01730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.794+0000 7fc2fec32700 1 -- 172.21.15.163:0/4197390317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2ce0f60 msgr2=0x5617f2cf9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:41.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.794+0000 7fc2fec32700 1 --2- 172.21.15.163:0/4197390317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2ce0f60 0x5617f2cf9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.794+0000 7fc2fec32700 1 -- 172.21.15.163:0/4197390317 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f81547f0 con 0x5617f2d01d00 2023-10-09T06:32:41.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.794+0000 7fc2fe431700 1 --2- 172.21.15.163:0/4197390317 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5617f2cb7a50 0x5617f2d01730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:41.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.795+0000 7fc2fec32700 1 --2- 172.21.15.163:0/4197390317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617f2d01d00 0x5617f2cf8680 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0053610 tx=0x7fc2f006f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72104d705058b144 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:41.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.795+0000 7fc2d8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-10-09T06:32:41.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.795+0000 7fc2d8ff9700 1 -- 172.21.15.163:0/4197390317 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f000e040 con 0x5617f2d01d00 2023-10-09T06:32:41.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.795+0000 7fc2d8ff9700 1 -- 172.21.15.163:0/4197390317 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc2f007ee40 con 0x5617f2d01d00 2023-10-09T06:32:41.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.795+0000 7fc30998db80 1 -- 172.21.15.163:0/4197390317 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617f2cacb70 con 0x5617f2d01d00 2023-10-09T06:32:41.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.795+0000 7fc2d8ff9700 1 -- 172.21.15.163:0/4197390317 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2f005a360 con 0x5617f2d01d00 2023-10-09T06:32:41.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.795+0000 7fc30998db80 10 client.5937 fetch_fsmap learned FSMap version 40 2023-10-09T06:32:41.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.795+0000 7fc30998db80 1 -- 172.21.15.163:0/4197390317 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x5617f2a44fe0 con 0x5617f2d01d00 2023-10-09T06:32:41.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.795+0000 7fc2d8ff9700 1 -- 172.21.15.163:0/4197390317 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5617f2cacb70 con 0x5617f2d01d00 2023-10-09T06:32:41.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.795+0000 7fc2d8ff9700 1 -- 172.21.15.163:0/4197390317 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 40) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc2f00842e0 con 0x5617f2d01d00 2023-10-09T06:32:41.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.795+0000 7fc30998db80 10 client.5937 fetch_fsmap finished waiting for FSMap version 40 2023-10-09T06:32:41.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.795+0000 7fc30998db80 10 client.5937 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-09T06:32:41.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.795+0000 7fc30998db80 20 client.5937 populate_metadata read hostname 'smithi163' 2023-10-09T06:32:41.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.795+0000 7fc30998db80 1 --2- 172.21.15.163:0/4197390317 >> [v2:172.21.15.163:6833/3415100865,v1:172.21.15.163:6835/3415100865] conn(0x5617f2cb2c20 0x5617f2cb2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.795+0000 7fc30998db80 4 client.5937 mds_command: new command op to 5748 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-09T06:32:41.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.795+0000 7fc30998db80 1 -- 172.21.15.163:0/4197390317 --> [v2:172.21.15.163:6833/3415100865,v1:172.21.15.163:6835/3415100865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5617f2c89aa0 con 0x5617f2cb2c20 2023-10-09T06:32:41.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.795+0000 7fc2fe431700 1 --2- 172.21.15.163:0/4197390317 >> [v2:172.21.15.163:6833/3415100865,v1:172.21.15.163:6835/3415100865] conn(0x5617f2cb2c20 0x5617f2cb2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.796+0000 7fc2d8ff9700 10 client.5937 ms_handle_connect on v2:172.21.15.163:6833/3415100865 2023-10-09T06:32:41.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.796+0000 7fc2fe431700 1 --2- 172.21.15.163:0/4197390317 >> [v2:172.21.15.163:6833/3415100865,v1:172.21.15.163:6835/3415100865] conn(0x5617f2cb2c20 0x5617f2cb2ff0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a540350b9fb6ee85 server_cookie=65ceae85310e321e in_seq=0 out_seq=0 2023-10-09T06:32:41.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.796+0000 7fc2d8ff9700 1 -- 172.21.15.163:0/4197390317 <== mds.0 v2:172.21.15.163:6833/3415100865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5617f2c89aa0 con 0x5617f2cb2c20 2023-10-09T06:32:41.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.796+0000 7fc2d8ff9700 10 client.5937 handle_command_reply: tid=0 2023-10-09T06:32:41.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.796+0000 7fc30998db80 1 client.5937 shutdown 2023-10-09T06:32:41.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.796+0000 7fc30998db80 1 -- 172.21.15.163:0/4197390317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617f2d01d00 msgr2=0x5617f2cf8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:41.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.796+0000 7fc30998db80 1 --2- 172.21.15.163:0/4197390317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617f2d01d00 0x5617f2cf8680 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0053610 tx=0x7fc2f006f000 comp rx=0 tx=0).stop 2023-10-09T06:32:41.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.796+0000 7fc30998db80 1 -- 172.21.15.163:0/4197390317 shutdown_connections 2023-10-09T06:32:41.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.796+0000 7fc30998db80 1 --2- 172.21.15.163:0/4197390317 >> [v2:172.21.15.163:6833/3415100865,v1:172.21.15.163:6835/3415100865] conn(0x5617f2cb2c20 0x5617f2cb2ff0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.796+0000 7fc30998db80 1 --2- 172.21.15.163:0/4197390317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617f2d01d00 0x5617f2cf8680 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.796+0000 7fc2d8ff9700 0 client.5937 ms_handle_reset on v2:172.21.15.163:6833/3415100865 2023-10-09T06:32:41.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.796+0000 7fc30998db80 1 --2- 172.21.15.163:0/4197390317 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5617f2cb7a50 0x5617f2d01730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.796+0000 7fc30998db80 1 --2- 172.21.15.163:0/4197390317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2ce0f60 0x5617f2cf9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.796+0000 7fc30998db80 1 -- 172.21.15.163:0/4197390317 >> 172.21.15.163:0/4197390317 conn(0x5617f2cb9370 msgr2=0x5617f2cfd520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:41.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.796+0000 7fc30998db80 1 -- 172.21.15.163:0/4197390317 shutdown_connections 2023-10-09T06:32:41.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.796+0000 7fc30998db80 1 -- 172.21.15.163:0/4197390317 wait complete. 2023-10-09T06:32:41.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.797+0000 7fc30998db80 20 client.5937 trim_cache size 0 max 16384 2023-10-09T06:32:41.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.813+0000 7fc30998db80 1 Processor -- start 2023-10-09T06:32:41.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.813+0000 7fc30998db80 1 -- start start 2023-10-09T06:32:41.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.814+0000 7fc30998db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617f2cb7a50 0x5617f2d174c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.814+0000 7fc30998db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2ce0f60 0x5617f2cb9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.814+0000 7fc30998db80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5617f2d01d00 0x5617f2d00790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.814+0000 7fc30998db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f2ce7df0 con 0x5617f2cb7a50 2023-10-09T06:32:41.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.814+0000 7fc30998db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f2ce0ad0 con 0x5617f2ce0f60 2023-10-09T06:32:41.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.814+0000 7fc30998db80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617f2ce7b30 con 0x5617f2d01d00 2023-10-09T06:32:41.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.814+0000 7fc2fe431700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617f2cb7a50 0x5617f2d174c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.814+0000 7fc2fe431700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617f2cb7a50 0x5617f2d174c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:34472/0 (socket says 172.21.15.163:34472) 2023-10-09T06:32:41.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.814+0000 7fc2fe431700 1 -- 172.21.15.163:0/1788158887 learned_addr learned my addr 172.21.15.163:0/1788158887 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:41.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.814+0000 7fc2fe431700 1 -- 172.21.15.163:0/1788158887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5617f2d01d00 msgr2=0x5617f2d00790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:41.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.814+0000 7fc2fe431700 1 --2- 172.21.15.163:0/1788158887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5617f2d01d00 0x5617f2d00790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.814+0000 7fc2fe431700 1 -- 172.21.15.163:0/1788158887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2ce0f60 msgr2=0x5617f2cb9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:41.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.814+0000 7fc2fdc30700 1 --2- 172.21.15.163:0/1788158887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2ce0f60 0x5617f2cb9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.814+0000 7fc2fe431700 1 --2- 172.21.15.163:0/1788158887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2ce0f60 0x5617f2cb9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.814+0000 7fc2fe431700 1 -- 172.21.15.163:0/1788158887 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617f2bd6780 con 0x5617f2cb7a50 2023-10-09T06:32:41.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.815+0000 7fc2fdc30700 1 --2- 172.21.15.163:0/1788158887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2ce0f60 0x5617f2cb9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:32:41.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.815+0000 7fc2fe431700 1 --2- 172.21.15.163:0/1788158887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617f2cb7a50 0x5617f2d174c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4011b60 tx=0x7fc2f402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe48c2703cbea51 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:41.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.815+0000 7fc2dbfff700 1 -- 172.21.15.163:0/1788158887 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4014db0 con 0x5617f2cb7a50 2023-10-09T06:32:41.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.815+0000 7fc2dbfff700 1 -- 172.21.15.163:0/1788158887 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f402e480 con 0x5617f2cb7a50 2023-10-09T06:32:41.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.815+0000 7fc2dbfff700 1 -- 172.21.15.163:0/1788158887 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f402e820 con 0x5617f2cb7a50 2023-10-09T06:32:41.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.815+0000 7fc30998db80 1 -- 172.21.15.163:0/1788158887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617f2cb7a50 msgr2=0x5617f2d174c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:41.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.815+0000 7fc30998db80 1 --2- 172.21.15.163:0/1788158887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617f2cb7a50 0x5617f2d174c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4011b60 tx=0x7fc2f402a250 comp rx=0 tx=0).stop 2023-10-09T06:32:41.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.815+0000 7fc30998db80 1 -- 172.21.15.163:0/1788158887 shutdown_connections 2023-10-09T06:32:41.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.815+0000 7fc30998db80 1 --2- 172.21.15.163:0/1788158887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617f2cb7a50 0x5617f2d174c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.815+0000 7fc30998db80 1 --2- 172.21.15.163:0/1788158887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5617f2d01d00 0x5617f2d00790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.815+0000 7fc30998db80 1 --2- 172.21.15.163:0/1788158887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2ce0f60 0x5617f2cb9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.815+0000 7fc30998db80 1 -- 172.21.15.163:0/1788158887 >> 172.21.15.163:0/1788158887 conn(0x5617f2cb9370 msgr2=0x5617f2cb04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:41.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.815+0000 7fc30998db80 1 -- 172.21.15.163:0/1788158887 shutdown_connections 2023-10-09T06:32:41.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.815+0000 7fc30998db80 1 -- 172.21.15.163:0/1788158887 wait complete. 2023-10-09T06:32:41.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.816+0000 7fc30998db80 1 -- start start 2023-10-09T06:32:41.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.816+0000 7fc30998db80 1 Processor -- start 2023-10-09T06:32:41.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.817+0000 7fc30998db80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5617f2cb7a50 0x5617f2d00410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.817+0000 7fc30998db80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2ce0f60 0x5617f2d0d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.817+0000 7fc30998db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617f2d01d00 0x5617f2d10680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.817+0000 7fc30998db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f2cf9f90 con 0x5617f2d01d00 2023-10-09T06:32:41.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.817+0000 7fc30998db80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f2d64b20 con 0x5617f2ce0f60 2023-10-09T06:32:41.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.817+0000 7fc30998db80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617f2d0c610 con 0x5617f2cb7a50 2023-10-09T06:32:41.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.817+0000 7fc30998db80 10 client.0 _ll_register_callbacks cb 0x5617f2c7b110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-09T06:32:41.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.817+0000 7fc2fec32700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617f2d01d00 0x5617f2d10680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.817+0000 7fc2fec32700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617f2d01d00 0x5617f2d10680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:34486/0 (socket says 172.21.15.163:34486) 2023-10-09T06:32:41.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.817+0000 7fc2fec32700 1 -- 172.21.15.163:0/3050650134 learned_addr learned my addr 172.21.15.163:0/3050650134 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:41.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.817+0000 7fc2fdc30700 1 --2- 172.21.15.163:0/3050650134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2ce0f60 0x5617f2d0d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.817+0000 7fc2fe431700 1 --2- 172.21.15.163:0/3050650134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5617f2cb7a50 0x5617f2d00410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.817+0000 7fc2fec32700 1 -- 172.21.15.163:0/3050650134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5617f2cb7a50 msgr2=0x5617f2d00410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:41.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.817+0000 7fc2fec32700 1 --2- 172.21.15.163:0/3050650134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5617f2cb7a50 0x5617f2d00410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.817+0000 7fc2fec32700 1 -- 172.21.15.163:0/3050650134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2ce0f60 msgr2=0x5617f2d0d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:41.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.817+0000 7fc2fec32700 1 --2- 172.21.15.163:0/3050650134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2ce0f60 0x5617f2d0d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.817+0000 7fc2fec32700 1 -- 172.21.15.163:0/3050650134 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f401a040 con 0x5617f2d01d00 2023-10-09T06:32:41.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.817+0000 7fc2fec32700 1 --2- 172.21.15.163:0/3050650134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617f2d01d00 0x5617f2d10680 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0056a90 tx=0x7fc2f0080040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5c1c7b15374332 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:41.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.817+0000 7fc2d8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-10-09T06:32:41.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.817+0000 7fc2d8ff9700 1 -- 172.21.15.163:0/3050650134 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f000e040 con 0x5617f2d01d00 2023-10-09T06:32:41.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.818+0000 7fc2d8ff9700 1 -- 172.21.15.163:0/3050650134 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc2f007ee40 con 0x5617f2d01d00 2023-10-09T06:32:41.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.818+0000 7fc30998db80 1 -- 172.21.15.163:0/3050650134 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2f00860e0 con 0x5617f2d01d00 2023-10-09T06:32:41.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.818+0000 7fc2d8ff9700 1 -- 172.21.15.163:0/3050650134 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2f005a360 con 0x5617f2d01d00 2023-10-09T06:32:41.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.818+0000 7fc30998db80 10 client.5943 fetch_fsmap learned FSMap version 40 2023-10-09T06:32:41.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.818+0000 7fc30998db80 1 -- 172.21.15.163:0/3050650134 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x5617f2bbd470 con 0x5617f2d01d00 2023-10-09T06:32:41.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.818+0000 7fc2d8ff9700 1 -- 172.21.15.163:0/3050650134 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f00860e0 con 0x5617f2d01d00 2023-10-09T06:32:41.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.818+0000 7fc2d8ff9700 1 -- 172.21.15.163:0/3050650134 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 40) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc2f006d3d0 con 0x5617f2d01d00 2023-10-09T06:32:41.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.818+0000 7fc30998db80 10 client.5943 fetch_fsmap finished waiting for FSMap version 40 2023-10-09T06:32:41.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.818+0000 7fc30998db80 10 client.5943 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-09T06:32:41.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.818+0000 7fc30998db80 20 client.5943 populate_metadata read hostname 'smithi163' 2023-10-09T06:32:41.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.818+0000 7fc30998db80 1 --2- 172.21.15.163:0/3050650134 >> [v2:172.21.15.163:6833/3415100865,v1:172.21.15.163:6835/3415100865] conn(0x5617f2d63b80 0x5617f2d63f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.818+0000 7fc30998db80 4 client.5943 mds_command: new command op to 5748 tid=0[{"prefix": "status"}] 2023-10-09T06:32:41.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.818+0000 7fc30998db80 1 -- 172.21.15.163:0/3050650134 --> [v2:172.21.15.163:6833/3415100865,v1:172.21.15.163:6835/3415100865] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5617f2baae00 con 0x5617f2d63b80 2023-10-09T06:32:41.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.818+0000 7fc2fe431700 1 --2- 172.21.15.163:0/3050650134 >> [v2:172.21.15.163:6833/3415100865,v1:172.21.15.163:6835/3415100865] conn(0x5617f2d63b80 0x5617f2d63f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.818+0000 7fc2d8ff9700 10 client.5943 ms_handle_connect on v2:172.21.15.163:6833/3415100865 2023-10-09T06:32:41.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.818+0000 7fc2fe431700 1 --2- 172.21.15.163:0/3050650134 >> [v2:172.21.15.163:6833/3415100865,v1:172.21.15.163:6835/3415100865] conn(0x5617f2d63b80 0x5617f2d63f50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3318bf97ef32e1a6 server_cookie=493e94b73231f3a0 in_seq=0 out_seq=0 2023-10-09T06:32:41.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.819+0000 7fc2d8ff9700 1 -- 172.21.15.163:0/3050650134 <== mds.0 v2:172.21.15.163:6833/3415100865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x5617f2baae00 con 0x5617f2d63b80 2023-10-09T06:32:41.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.819+0000 7fc2d8ff9700 10 client.5943 handle_command_reply: tid=0 2023-10-09T06:32:41.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.819+0000 7fc30998db80 1 client.5943 shutdown 2023-10-09T06:32:41.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.819+0000 7fc30998db80 1 -- 172.21.15.163:0/3050650134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617f2d01d00 msgr2=0x5617f2d10680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:41.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.819+0000 7fc30998db80 1 --2- 172.21.15.163:0/3050650134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617f2d01d00 0x5617f2d10680 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0056a90 tx=0x7fc2f0080040 comp rx=0 tx=0).stop 2023-10-09T06:32:41.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.819+0000 7fc30998db80 1 -- 172.21.15.163:0/3050650134 shutdown_connections 2023-10-09T06:32:41.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.819+0000 7fc30998db80 1 --2- 172.21.15.163:0/3050650134 >> [v2:172.21.15.163:6833/3415100865,v1:172.21.15.163:6835/3415100865] conn(0x5617f2d63b80 0x5617f2d63f50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.819+0000 7fc30998db80 1 --2- 172.21.15.163:0/3050650134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5617f2d01d00 0x5617f2d10680 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.819+0000 7fc30998db80 1 --2- 172.21.15.163:0/3050650134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5617f2cb7a50 0x5617f2d00410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.819+0000 7fc2d8ff9700 0 client.5943 ms_handle_reset on v2:172.21.15.163:6833/3415100865 2023-10-09T06:32:41.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.819+0000 7fc30998db80 1 --2- 172.21.15.163:0/3050650134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5617f2ce0f60 0x5617f2d0d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.819+0000 7fc30998db80 1 -- 172.21.15.163:0/3050650134 >> 172.21.15.163:0/3050650134 conn(0x5617f2cb9370 msgr2=0x5617f2cb9910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:41.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.819+0000 7fc30998db80 1 -- 172.21.15.163:0/3050650134 shutdown_connections 2023-10-09T06:32:41.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.819+0000 7fc30998db80 1 -- 172.21.15.163:0/3050650134 wait complete. 2023-10-09T06:32:41.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.819+0000 7fc30998db80 20 client.5943 trim_cache size 0 max 16384 2023-10-09T06:32:41.829 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:32:41.829 INFO:teuthology.orchestra.run.smithi163.stdout: "cluster_fsid": "2520b8f7-539f-4bae-9f17-1fce66f66ddd", 2023-10-09T06:32:41.829 INFO:teuthology.orchestra.run.smithi163.stdout: "whoami": 0, 2023-10-09T06:32:41.830 INFO:teuthology.orchestra.run.smithi163.stdout: "id": 5748, 2023-10-09T06:32:41.830 INFO:teuthology.orchestra.run.smithi163.stdout: "want_state": "up:active", 2023-10-09T06:32:41.830 INFO:teuthology.orchestra.run.smithi163.stdout: "state": "up:active", 2023-10-09T06:32:41.830 INFO:teuthology.orchestra.run.smithi163.stdout: "fs_name": "cephfs", 2023-10-09T06:32:41.830 INFO:teuthology.orchestra.run.smithi163.stdout: "rank_uptime": 10.322192545, 2023-10-09T06:32:41.830 INFO:teuthology.orchestra.run.smithi163.stdout: "mdsmap_epoch": 40, 2023-10-09T06:32:41.830 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch": 63, 2023-10-09T06:32:41.830 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch_barrier": 63, 2023-10-09T06:32:41.830 INFO:teuthology.orchestra.run.smithi163.stdout: "uptime": 15.783926549 2023-10-09T06:32:41.830 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:32:41.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.819+0000 7fc300695700 1 -- 172.21.15.163:0/2642334487 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc2dc066ff0 msgr2=0x7fc2dc0694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:41.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.819+0000 7fc300695700 1 --2- 172.21.15.163:0/2642334487 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc2dc066ff0 0x7fc2dc0694a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80cf210 tx=0x7fc2f40091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.819+0000 7fc300695700 1 -- 172.21.15.163:0/2642334487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2f815df50 msgr2=0x7fc2f80b2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:41.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.819+0000 7fc300695700 1 --2- 172.21.15.163:0/2642334487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2f815df50 0x7fc2f80b2e50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8069220 tx=0x7fc2f0019000 comp rx=0 tx=0).stop 2023-10-09T06:32:41.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.819+0000 7fc300695700 1 -- 172.21.15.163:0/2642334487 shutdown_connections 2023-10-09T06:32:41.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.819+0000 7fc300695700 1 --2- 172.21.15.163:0/2642334487 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc2dc066ff0 0x7fc2dc0694a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.819+0000 7fc300695700 1 --2- 172.21.15.163:0/2642334487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2f815df50 0x7fc2f80b2e50 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.819+0000 7fc300695700 1 --2- 172.21.15.163:0/2642334487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2f806bcd0 0x7fc2f816ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.819+0000 7fc300695700 1 --2- 172.21.15.163:0/2642334487 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2f806c0a0 0x7fc2f814e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.819+0000 7fc300695700 1 -- 172.21.15.163:0/2642334487 >> 172.21.15.163:0/2642334487 conn(0x7fc2f80ce600 msgr2=0x7fc2f814f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:41.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.819+0000 7fc300695700 1 -- 172.21.15.163:0/2642334487 shutdown_connections 2023-10-09T06:32:41.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.819+0000 7fc300695700 1 -- 172.21.15.163:0/2642334487 wait complete. 2023-10-09T06:32:41.833 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-09T06:32:41.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.947+0000 7fb5aa4cc700 1 Processor -- start 2023-10-09T06:32:41.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.947+0000 7fb5aa4cc700 1 -- start start 2023-10-09T06:32:41.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.948+0000 7fb5aa4cc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb5a4159890 0x7fb5a415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.948+0000 7fb5aa4cc700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5a415a160 con 0x7fb5a4069560 2023-10-09T06:32:41.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.948+0000 7fb5aa4cc700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5a40d13a0 con 0x7fb5a4159c60 2023-10-09T06:32:41.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.948+0000 7fb5aa4cc700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5a415e140 con 0x7fb5a4159890 2023-10-09T06:32:41.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.948+0000 7fb5a8a69700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb5a4069560 0x7fb5a40ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:42788/0 (socket says 172.21.15.163:42788) 2023-10-09T06:32:41.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.948+0000 7fb5a8a69700 1 -- 172.21.15.163:0/2004054585 learned_addr learned my addr 172.21.15.163:0/2004054585 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:41.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.948+0000 7fb5a3fff700 1 --2- 172.21.15.163:0/2004054585 >> v2:172.21.15.181:3300/0 conn(0x7fb5a4159890 0x7fb5a415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.949+0000 7fb5a2ffd700 1 -- 172.21.15.163:0/2004054585 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4282201844 0 0) 0x7fb5a415a160 con 0x7fb5a4069560 2023-10-09T06:32:41.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.949+0000 7fb5a2ffd700 1 -- 172.21.15.163:0/2004054585 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb58c003570 con 0x7fb5a4069560 2023-10-09T06:32:41.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.949+0000 7fb5a3fff700 1 -- 172.21.15.163:0/2004054585 >> v1:172.21.15.181:6789/0 conn(0x7fb5a4159c60 legacy=0x7fb5a415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:41.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.949+0000 7fb5a3fff700 1 -- 172.21.15.163:0/2004054585 >> v1:172.21.15.163:6789/0 conn(0x7fb5a4069560 legacy=0x7fb5a40ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:41.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.949+0000 7fb5a3fff700 1 -- 172.21.15.163:0/2004054585 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a415a3d0 con 0x7fb5a4159890 2023-10-09T06:32:41.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.949+0000 7fb5a3fff700 1 --2- 172.21.15.163:0/2004054585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5a4159890 0x7fb5a415abd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb5940098e0 tx=0x7fb5940091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afa5017535740c2d server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:41.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.949+0000 7fb5a2ffd700 1 -- 172.21.15.163:0/2004054585 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59400b640 con 0x7fb5a4159890 2023-10-09T06:32:41.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.949+0000 7fb5a2ffd700 1 -- 172.21.15.163:0/2004054585 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb59400b7a0 con 0x7fb5a4159890 2023-10-09T06:32:41.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.949+0000 7fb5a2ffd700 1 -- 172.21.15.163:0/2004054585 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59400bb40 con 0x7fb5a4159890 2023-10-09T06:32:41.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.950+0000 7fb5aa4cc700 1 -- 172.21.15.163:0/2004054585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5a4159890 msgr2=0x7fb5a415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:41.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.950+0000 7fb5aa4cc700 1 --2- 172.21.15.163:0/2004054585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5a4159890 0x7fb5a415abd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb5940098e0 tx=0x7fb5940091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.950+0000 7fb5aa4cc700 1 -- 172.21.15.163:0/2004054585 shutdown_connections 2023-10-09T06:32:41.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.950+0000 7fb5aa4cc700 1 --2- 172.21.15.163:0/2004054585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5a4159890 0x7fb5a415abd0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.950+0000 7fb5aa4cc700 1 -- 172.21.15.163:0/2004054585 >> 172.21.15.163:0/2004054585 conn(0x7fb5a4068e20 msgr2=0x7fb5a40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:41.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.950+0000 7fb5aa4cc700 1 -- 172.21.15.163:0/2004054585 shutdown_connections 2023-10-09T06:32:41.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.950+0000 7fb5aa4cc700 1 -- 172.21.15.163:0/2004054585 wait complete. 2023-10-09T06:32:41.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.951+0000 7fb5aa4cc700 1 Processor -- start 2023-10-09T06:32:41.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.951+0000 7fb5aa4cc700 1 -- start start 2023-10-09T06:32:41.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.952+0000 7fb5aa4cc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5a4069560 0x7fb5a4140b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.952+0000 7fb5aa4cc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5a4159890 0x7fb5a40ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.952+0000 7fb5aa4cc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5a4159c60 0x7fb5a40b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.952+0000 7fb5aa4cc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a40b39f0 con 0x7fb5a4159c60 2023-10-09T06:32:41.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.952+0000 7fb5aa4cc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a40b3c40 con 0x7fb5a4159890 2023-10-09T06:32:41.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.952+0000 7fb5aa4cc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a40b3f20 con 0x7fb5a4069560 2023-10-09T06:32:41.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.952+0000 7fb5a8a69700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5a4159c60 0x7fb5a40b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.952+0000 7fb5a8a69700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5a4159c60 0x7fb5a40b1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:34496/0 (socket says 172.21.15.163:34496) 2023-10-09T06:32:41.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.952+0000 7fb5a8a69700 1 -- 172.21.15.163:0/28350329 learned_addr learned my addr 172.21.15.163:0/28350329 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:41.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.952+0000 7fb5a37fe700 1 --2- 172.21.15.163:0/28350329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5a4159890 0x7fb5a40ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.952+0000 7fb5a3fff700 1 --2- 172.21.15.163:0/28350329 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5a4069560 0x7fb5a4140b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.952+0000 7fb5a8a69700 1 -- 172.21.15.163:0/28350329 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5a4069560 msgr2=0x7fb5a4140b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:41.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.952+0000 7fb5a8a69700 1 --2- 172.21.15.163:0/28350329 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5a4069560 0x7fb5a4140b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.952+0000 7fb5a8a69700 1 -- 172.21.15.163:0/28350329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5a4159890 msgr2=0x7fb5a40ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:41.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.952+0000 7fb5a8a69700 1 --2- 172.21.15.163:0/28350329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5a4159890 0x7fb5a40ca140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:41.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.952+0000 7fb5a8a69700 1 -- 172.21.15.163:0/28350329 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb594009420 con 0x7fb5a4159c60 2023-10-09T06:32:41.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.952+0000 7fb5a8a69700 1 --2- 172.21.15.163:0/28350329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5a4159c60 0x7fb5a40b1090 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb59c001460 tx=0x7fb59c001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:41.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.952+0000 7fb58bfff700 1 -- 172.21.15.163:0/28350329 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59c00a890 con 0x7fb5a4159c60 2023-10-09T06:32:41.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.952+0000 7fb58bfff700 1 -- 172.21.15.163:0/28350329 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb59c00a9f0 con 0x7fb5a4159c60 2023-10-09T06:32:41.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.952+0000 7fb58bfff700 1 -- 172.21.15.163:0/28350329 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59c00ad70 con 0x7fb5a4159c60 2023-10-09T06:32:41.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.952+0000 7fb5aa4cc700 1 -- 172.21.15.163:0/28350329 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5a40b4200 con 0x7fb5a4159c60 2023-10-09T06:32:41.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.953+0000 7fb5aa4cc700 1 -- 172.21.15.163:0/28350329 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a41410b0 con 0x7fb5a4159c60 2023-10-09T06:32:41.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.953+0000 7fb5aa4cc700 1 -- 172.21.15.163:0/28350329 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5a40b4720 con 0x7fb5a4159c60 2023-10-09T06:32:41.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.954+0000 7fb58bfff700 1 -- 172.21.15.163:0/28350329 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fb59c007300 con 0x7fb5a4159c60 2023-10-09T06:32:41.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.954+0000 7fb58bfff700 1 --2- 172.21.15.163:0/28350329 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb590066fc0 0x7fb590069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:41.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.954+0000 7fb58bfff700 1 -- 172.21.15.163:0/28350329 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb59c0848f0 con 0x7fb5a4159c60 2023-10-09T06:32:41.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.954+0000 7fb5a3fff700 1 --2- 172.21.15.163:0/28350329 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb590066fc0 0x7fb590069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:41.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.955+0000 7fb5a3fff700 1 --2- 172.21.15.163:0/28350329 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb590066fc0 0x7fb590069470 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb5940093f0 tx=0x7fb594017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:41.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:41.959+0000 7fb58bfff700 1 -- 172.21.15.163:0/28350329 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb5a40b4720 con 0x7fb5a4159c60 2023-10-09T06:32:42.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.186+0000 7fb5aa4cc700 1 -- 172.21.15.163:0/28350329 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb5a40b4720 con 0x7fb5a4159c60 2023-10-09T06:32:42.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.187+0000 7fb58bfff700 1 -- 172.21.15.163:0/28350329 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v40) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb5a40b4720 con 0x7fb5a4159c60 2023-10-09T06:32:42.190 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-09T06:32:42.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.189+0000 7fb5aa4cc700 1 -- 172.21.15.163:0/28350329 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb590066fc0 msgr2=0x7fb590069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:42.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.189+0000 7fb5aa4cc700 1 --2- 172.21.15.163:0/28350329 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb590066fc0 0x7fb590069470 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb5940093f0 tx=0x7fb594017000 comp rx=0 tx=0).stop 2023-10-09T06:32:42.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.189+0000 7fb5aa4cc700 1 -- 172.21.15.163:0/28350329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5a4159c60 msgr2=0x7fb5a40b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:42.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.189+0000 7fb5aa4cc700 1 --2- 172.21.15.163:0/28350329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5a4159c60 0x7fb5a40b1090 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb59c001460 tx=0x7fb59c001ae0 comp rx=0 tx=0).stop 2023-10-09T06:32:42.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.189+0000 7fb5aa4cc700 1 -- 172.21.15.163:0/28350329 shutdown_connections 2023-10-09T06:32:42.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.189+0000 7fb5aa4cc700 1 --2- 172.21.15.163:0/28350329 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb590066fc0 0x7fb590069470 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:42.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.189+0000 7fb5aa4cc700 1 --2- 172.21.15.163:0/28350329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5a4159c60 0x7fb5a40b1090 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:42.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.189+0000 7fb5aa4cc700 1 --2- 172.21.15.163:0/28350329 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5a4069560 0x7fb5a4140b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:42.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.189+0000 7fb5aa4cc700 1 --2- 172.21.15.163:0/28350329 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5a4159890 0x7fb5a40ca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:42.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.189+0000 7fb5aa4cc700 1 -- 172.21.15.163:0/28350329 >> 172.21.15.163:0/28350329 conn(0x7fb5a4068e20 msgr2=0x7fb5a40b1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:42.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.189+0000 7fb5aa4cc700 1 -- 172.21.15.163:0/28350329 shutdown_connections 2023-10-09T06:32:42.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.189+0000 7fb5aa4cc700 1 -- 172.21.15.163:0/28350329 wait complete. 2023-10-09T06:32:42.201 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-09T06:32:42.201 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-09T06:32:42.201 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi163.front.sepia.ceph.com 2023-10-09T06:32:42.201 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-09T06:32:42.201 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-09T06:32:42.201 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-09T06:32:42.201 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-09T06:32:42.202 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-09T06:32:42.202 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:32:42.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:42.202 DEBUG:teuthology.orchestra.run.smithi163:> ip addr 2023-10-09T06:32:42.217 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:32:42.217 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:32:42.217 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:32:42.217 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:32:42.217 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2023-10-09T06:32:42.217 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:32:42.218 INFO:teuthology.orchestra.run.smithi163.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:32:42.218 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:06 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:32:42.218 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f0 2023-10-09T06:32:42.218 INFO:teuthology.orchestra.run.smithi163.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:32:42.218 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:07 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:32:42.218 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f1 2023-10-09T06:32:42.218 INFO:teuthology.orchestra.run.smithi163.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:32:42.218 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5a brd ff:ff:ff:ff:ff:ff 2023-10-09T06:32:42.218 INFO:teuthology.orchestra.run.smithi163.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-09T06:32:42.218 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5b brd ff:ff:ff:ff:ff:ff 2023-10-09T06:32:42.218 INFO:teuthology.orchestra.run.smithi163.stdout: inet 172.21.15.163/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-09T06:32:42.219 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft 42361sec preferred_lft 42361sec 2023-10-09T06:32:42.219 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::ec4:7aff:fe88:7d5b/64 scope link 2023-10-09T06:32:42.219 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:32:42.219 INFO:teuthology.orchestra.run.smithi163.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-09T06:32:42.219 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 2e:01:84:b9:f4:8e brd ff:ff:ff:ff:ff:ff 2023-10-09T06:32:42.219 INFO:teuthology.orchestra.run.smithi163.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-09T06:32:42.219 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:32:42.219 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::d03c:93ff:fe9e:c8cb/64 scope link 2023-10-09T06:32:42.219 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:32:42.219 INFO:teuthology.orchestra.run.smithi163.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:32:42.219 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 2e:01:84:b9:f4:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-09T06:32:42.219 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::2c01:84ff:feb9:f48e/64 scope link 2023-10-09T06:32:42.220 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:32:42.220 INFO:teuthology.orchestra.run.smithi163.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:32:42.220 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether c6:88:3e:d7:c1:56 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-09T06:32:42.220 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::c488:3eff:fed7:c156/64 scope link 2023-10-09T06:32:42.220 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:32:42.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:42.220 DEBUG:teuthology.orchestra.run.smithi163:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-09T06:32:42.281 INFO:teuthology.orchestra.run.smithi163.stdout:1 2023-10-09T06:32:42.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:42.282 DEBUG:teuthology.orchestra.run.smithi163:> route 2023-10-09T06:32:42.339 INFO:teuthology.orchestra.run.smithi163.stdout:Kernel IP routing table 2023-10-09T06:32:42.339 INFO:teuthology.orchestra.run.smithi163.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-09T06:32:42.339 INFO:teuthology.orchestra.run.smithi163.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-09T06:32:42.339 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-09T06:32:42.339 INFO:teuthology.orchestra.run.smithi163.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-09T06:32:42.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:42.340 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-09T06:32:42.340 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-09T06:32:42.340 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-09T06:32:42.340 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-09T06:32:42.340 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-09T06:32:42.341 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-09T06:32:42.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:42.440 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2023-10-09T06:32:42.454 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-09T06:32:42.455 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-09T06:32:42.455 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:32:42.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:42.455 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set brx.0 up 2023-10-09T06:32:42.519 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:32:42.519 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-09T06:32:42.574 INFO:teuthology.orchestra.run.smithi163.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:32:42.575 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:32:42.575 DEBUG:teuthology.orchestra.run.smithi163:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-09T06:32:42.629 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-09T06:32:42.629 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-09T06:32:42.629 DEBUG:teuthology.orchestra.run.smithi163:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:32:42.710 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:32:42.710 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:32:42.729 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:32:42.730 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:32:42.730 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:32:42.730 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with new device syntax: 0@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:32:42.730 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:32:42.730 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:32:42.730 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:32:42.730 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:32:42.751 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:32:42.751 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:32:42.751 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:32:42.751 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with new device syntax: 0@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:32:42.751 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:32:42.751 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:32:42.752 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:32:42.752 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:32:42.752 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790" 2023-10-09T06:32:42.752 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790:/ 2023-10-09T06:32:42.752 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=2520b8f7-539f-4bae-9f17-1fce66f66ddd" will pass to kernel 2023-10-09T06:32:42.753 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-09T06:32:42.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:42.753 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-09T06:32:42.818 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-09T06:32:42.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.975+0000 7f6a4c9c1700 1 Processor -- start 2023-10-09T06:32:42.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.975+0000 7f6a4c9c1700 1 -- start start 2023-10-09T06:32:42.977 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.976+0000 7f6a4c9c1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6a48159890 0x7f6a4815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:42.978 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.976+0000 7f6a4c9c1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6a48069560 0x7f6a480699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:42.978 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.976+0000 7f6a4c9c1700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a48069ee0 con 0x7f6a48159c60 2023-10-09T06:32:42.978 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.976+0000 7f6a4c9c1700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a4815a160 con 0x7f6a48159890 2023-10-09T06:32:42.978 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.976+0000 7f6a4c9c1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a480d13a0 con 0x7f6a48069560 2023-10-09T06:32:42.978 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.976+0000 7f6a4659c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6a48159890 0x7f6a4815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:42.978 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.976+0000 7f6a45d9b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f6a48159c60 0x7f6a4815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:42800/0 (socket says 172.21.15.163:42800) 2023-10-09T06:32:42.978 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.976+0000 7f6a45d9b700 1 -- 172.21.15.163:0/3814372255 learned_addr learned my addr 172.21.15.163:0/3814372255 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:42.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.976+0000 7f6a46d9d700 1 --2- 172.21.15.163:0/3814372255 >> v2:172.21.15.181:3300/0 conn(0x7f6a48069560 0x7f6a480699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:42.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.976+0000 7f6a4659c700 1 -- 172.21.15.163:0/3814372255 >> v2:172.21.15.181:3300/0 conn(0x7f6a48069560 msgr2=0x7f6a480699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:42.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.976+0000 7f6a4659c700 1 --2- 172.21.15.163:0/3814372255 >> v2:172.21.15.181:3300/0 conn(0x7f6a48069560 0x7f6a480699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:42.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.976+0000 7f6a4659c700 1 -- 172.21.15.163:0/3814372255 >> v1:172.21.15.163:6789/0 conn(0x7f6a48159c60 legacy=0x7f6a4815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:42.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.976+0000 7f6a4659c700 1 -- 172.21.15.163:0/3814372255 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a4815e140 con 0x7f6a48159890 2023-10-09T06:32:42.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.976+0000 7f6a4559a700 1 -- 172.21.15.163:0/3814372255 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3034259587 0 0) 0x7f6a48069ee0 con 0x7f6a48159c60 2023-10-09T06:32:42.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.976+0000 7f6a4659c700 1 --2- 172.21.15.163:0/3814372255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a48159890 0x7f6a4815abd0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6a4806a890 tx=0x7f6a40009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83d404c399ed7da server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:42.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.976+0000 7f6a4559a700 1 -- 172.21.15.163:0/3814372255 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4000e070 con 0x7f6a48159890 2023-10-09T06:32:42.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.976+0000 7f6a4559a700 1 -- 172.21.15.163:0/3814372255 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4000bb60 con 0x7f6a48159890 2023-10-09T06:32:42.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.976+0000 7f6a4559a700 1 -- 172.21.15.163:0/3814372255 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a40012430 con 0x7f6a48159890 2023-10-09T06:32:42.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.977+0000 7f6a4c9c1700 1 -- 172.21.15.163:0/3814372255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a48159890 msgr2=0x7f6a4815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:42.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.977+0000 7f6a4c9c1700 1 --2- 172.21.15.163:0/3814372255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a48159890 0x7f6a4815abd0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6a4806a890 tx=0x7f6a40009780 comp rx=0 tx=0).stop 2023-10-09T06:32:42.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.977+0000 7f6a4c9c1700 1 -- 172.21.15.163:0/3814372255 shutdown_connections 2023-10-09T06:32:42.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.977+0000 7f6a4c9c1700 1 --2- 172.21.15.163:0/3814372255 >> v2:172.21.15.181:3300/0 conn(0x7f6a48069560 0x7f6a480699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:42.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.977+0000 7f6a4c9c1700 1 --2- 172.21.15.163:0/3814372255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a48159890 0x7f6a4815abd0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:42.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.977+0000 7f6a4c9c1700 1 -- 172.21.15.163:0/3814372255 >> 172.21.15.163:0/3814372255 conn(0x7f6a48068e20 msgr2=0x7f6a480d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:42.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.977+0000 7f6a4c9c1700 1 -- 172.21.15.163:0/3814372255 shutdown_connections 2023-10-09T06:32:42.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.977+0000 7f6a4c9c1700 1 -- 172.21.15.163:0/3814372255 wait complete. 2023-10-09T06:32:42.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.978+0000 7f6a4c9c1700 1 Processor -- start 2023-10-09T06:32:42.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.978+0000 7f6a4c9c1700 1 -- start start 2023-10-09T06:32:42.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.979+0000 7f6a4c9c1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a48069560 0x7f6a4814c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:42.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.979+0000 7f6a4c9c1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a48159890 0x7f6a480cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:42.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.979+0000 7f6a4c9c1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a48159c60 0x7f6a48155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:42.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.979+0000 7f6a4c9c1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a480cef60 con 0x7f6a48069560 2023-10-09T06:32:42.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.979+0000 7f6a4c9c1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a481522b0 con 0x7f6a48159c60 2023-10-09T06:32:42.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.979+0000 7f6a4c9c1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a480ce910 con 0x7f6a48159890 2023-10-09T06:32:42.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.979+0000 7f6a4659c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a48069560 0x7f6a4814c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:42.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.979+0000 7f6a4659c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a48069560 0x7f6a4814c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:34504/0 (socket says 172.21.15.163:34504) 2023-10-09T06:32:42.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.979+0000 7f6a4659c700 1 -- 172.21.15.163:0/2490382173 learned_addr learned my addr 172.21.15.163:0/2490382173 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:42.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.979+0000 7f6a46d9d700 1 --2- 172.21.15.163:0/2490382173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a48159c60 0x7f6a48155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:42.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.979+0000 7f6a4659c700 1 -- 172.21.15.163:0/2490382173 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a48159890 msgr2=0x7f6a480cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:42.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.979+0000 7f6a4659c700 1 --2- 172.21.15.163:0/2490382173 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a48159890 0x7f6a480cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:42.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.979+0000 7f6a4659c700 1 -- 172.21.15.163:0/2490382173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a48159c60 msgr2=0x7f6a48155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:42.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.979+0000 7f6a4659c700 1 --2- 172.21.15.163:0/2490382173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a48159c60 0x7f6a48155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:42.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.979+0000 7f6a4659c700 1 -- 172.21.15.163:0/2490382173 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a40009a20 con 0x7f6a48069560 2023-10-09T06:32:42.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.979+0000 7f6a46d9d700 1 --2- 172.21.15.163:0/2490382173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a48159c60 0x7f6a48155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:32:42.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.979+0000 7f6a4659c700 1 --2- 172.21.15.163:0/2490382173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a48069560 0x7f6a4814c9e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009fd0 tx=0x7f6a40019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:42.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.979+0000 7f6a3e7fc700 1 -- 172.21.15.163:0/2490382173 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a40012db0 con 0x7f6a48069560 2023-10-09T06:32:42.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.979+0000 7f6a3e7fc700 1 -- 172.21.15.163:0/2490382173 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a40027470 con 0x7f6a48069560 2023-10-09T06:32:42.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.979+0000 7f6a4c9c1700 1 -- 172.21.15.163:0/2490382173 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a480cebf0 con 0x7f6a48069560 2023-10-09T06:32:42.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.980+0000 7f6a3e7fc700 1 -- 172.21.15.163:0/2490382173 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a40027830 con 0x7f6a48069560 2023-10-09T06:32:42.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.980+0000 7f6a4c9c1700 1 -- 172.21.15.163:0/2490382173 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a480cf930 con 0x7f6a48069560 2023-10-09T06:32:42.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.980+0000 7f6a4c9c1700 1 -- 172.21.15.163:0/2490382173 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a480cf240 con 0x7f6a48069560 2023-10-09T06:32:42.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.981+0000 7f6a3e7fc700 1 -- 172.21.15.163:0/2490382173 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f6a40024070 con 0x7f6a48069560 2023-10-09T06:32:42.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.981+0000 7f6a3e7fc700 1 --2- 172.21.15.163:0/2490382173 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6a28066fc0 0x7f6a28069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:42.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.981+0000 7f6a3e7fc700 1 -- 172.21.15.163:0/2490382173 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6a40086c30 con 0x7f6a48069560 2023-10-09T06:32:42.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.981+0000 7f6a45d9b700 1 --2- 172.21.15.163:0/2490382173 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6a28066fc0 0x7f6a28069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:42.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.982+0000 7f6a45d9b700 1 --2- 172.21.15.163:0/2490382173 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6a28066fc0 0x7f6a28069470 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6a4816ce70 tx=0x7f6a30007bd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:42.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:42.985+0000 7f6a3e7fc700 1 -- 172.21.15.163:0/2490382173 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6a40053eb0 con 0x7f6a48069560 2023-10-09T06:32:43.098 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:32:43.100 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:32:43.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.214+0000 7f6a4c9c1700 1 -- 172.21.15.163:0/2490382173 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6a4804c3c0 con 0x7f6a48069560 2023-10-09T06:32:43.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.214+0000 7f6a3e7fc700 1 -- 172.21.15.163:0/2490382173 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v40) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6a400574d0 con 0x7f6a48069560 2023-10-09T06:32:43.214 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-09T06:32:43.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.216+0000 7f6a4c9c1700 1 -- 172.21.15.163:0/2490382173 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6a28066fc0 msgr2=0x7f6a28069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:43.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.216+0000 7f6a4c9c1700 1 --2- 172.21.15.163:0/2490382173 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6a28066fc0 0x7f6a28069470 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6a4816ce70 tx=0x7f6a30007bd0 comp rx=0 tx=0).stop 2023-10-09T06:32:43.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.216+0000 7f6a4c9c1700 1 -- 172.21.15.163:0/2490382173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a48069560 msgr2=0x7f6a4814c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:43.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.216+0000 7f6a4c9c1700 1 --2- 172.21.15.163:0/2490382173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a48069560 0x7f6a4814c9e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009fd0 tx=0x7f6a40019000 comp rx=0 tx=0).stop 2023-10-09T06:32:43.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.216+0000 7f6a4c9c1700 1 -- 172.21.15.163:0/2490382173 shutdown_connections 2023-10-09T06:32:43.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.216+0000 7f6a4c9c1700 1 --2- 172.21.15.163:0/2490382173 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6a28066fc0 0x7f6a28069470 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:43.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.216+0000 7f6a4c9c1700 1 --2- 172.21.15.163:0/2490382173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a48069560 0x7f6a4814c9e0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:43.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.216+0000 7f6a4c9c1700 1 --2- 172.21.15.163:0/2490382173 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a48159c60 0x7f6a48155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:43.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.216+0000 7f6a4c9c1700 1 --2- 172.21.15.163:0/2490382173 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a48159890 0x7f6a480cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:43.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.216+0000 7f6a4c9c1700 1 -- 172.21.15.163:0/2490382173 >> 172.21.15.163:0/2490382173 conn(0x7f6a48068e20 msgr2=0x7f6a480b22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:43.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.216+0000 7f6a4c9c1700 1 -- 172.21.15.163:0/2490382173 shutdown_connections 2023-10-09T06:32:43.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.216+0000 7f6a4c9c1700 1 -- 172.21.15.163:0/2490382173 wait complete. 2023-10-09T06:32:43.228 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-09T06:32:43.228 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-09T06:32:43.228 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-09T06:32:43.228 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-09T06:32:43.228 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-09T06:32:43.228 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-09T06:32:43.228 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-09T06:32:43.228 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-09T06:32:43.229 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:32:43.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:43.229 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-09T06:32:43.246 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:32:43.247 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:32:43.247 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:32:43.247 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:32:43.247 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-09T06:32:43.247 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:32:43.247 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:32:43.247 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:32:43.247 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2023-10-09T06:32:43.247 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:32:43.247 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:32:43.247 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2023-10-09T06:32:43.248 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:32:43.248 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-09T06:32:43.248 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-09T06:32:43.248 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-09T06:32:43.248 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-09T06:32:43.248 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42337sec preferred_lft 42337sec 2023-10-09T06:32:43.248 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2023-10-09T06:32:43.248 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:32:43.248 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-09T06:32:43.248 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 72:62:e6:72:7c:2e brd ff:ff:ff:ff:ff:ff 2023-10-09T06:32:43.248 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-09T06:32:43.248 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:32:43.249 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::e89a:4cff:fefb:7198/64 scope link 2023-10-09T06:32:43.249 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:32:43.249 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:32:43.249 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 72:62:e6:72:7c:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-09T06:32:43.249 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::7062:e6ff:fe72:7c2e/64 scope link 2023-10-09T06:32:43.249 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:32:43.249 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:32:43.249 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether fa:4e:a3:9d:5a:5f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-09T06:32:43.249 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::f84e:a3ff:fe9d:5a5f/64 scope link 2023-10-09T06:32:43.249 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:32:43.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:43.250 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-09T06:32:43.312 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-09T06:32:43.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:43.314 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-09T06:32:43.371 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-09T06:32:43.371 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-09T06:32:43.372 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-09T06:32:43.372 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-09T06:32:43.372 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-09T06:32:43.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:43.372 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-09T06:32:43.372 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-09T06:32:43.372 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-09T06:32:43.372 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-09T06:32:43.373 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-09T06:32:43.373 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-09T06:32:43.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:43.476 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-09T06:32:43.492 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-09T06:32:43.492 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-09T06:32:43.492 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:32:43.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:43.493 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-09T06:32:43.558 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:32:43.558 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-09T06:32:43.613 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:32:43.613 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:32:43.613 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-09T06:32:43.668 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-09T06:32:43.668 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-09T06:32:43.668 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:32:43.748 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:32:43.748 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:32:43.767 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:32:43.767 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:32:43.768 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:32:43.768 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 1@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:32:43.768 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:32:43.768 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:32:43.768 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:32:43.768 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:32:43.789 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:32:43.789 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:32:43.789 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:32:43.789 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 1@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:32:43.789 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:32:43.789 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:32:43.790 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:32:43.790 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:32:43.790 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790" 2023-10-09T06:32:43.790 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790:/ 2023-10-09T06:32:43.790 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=2520b8f7-539f-4bae-9f17-1fce66f66ddd" will pass to kernel 2023-10-09T06:32:43.791 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-09T06:32:43.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:32:43.791 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-09T06:32:43.855 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-09T06:32:43.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.975+0000 7f2b2f7ea700 1 Processor -- start 2023-10-09T06:32:43.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.975+0000 7f2b2f7ea700 1 -- start start 2023-10-09T06:32:43.978 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.976+0000 7f2b2f7ea700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b28159ec0 0x7f2b280b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:43.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.976+0000 7f2b2f7ea700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b280b5b00 con 0x7f2b280cf350 2023-10-09T06:32:43.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.976+0000 7f2b2f7ea700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b280b5da0 con 0x7f2b280b53d0 2023-10-09T06:32:43.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.976+0000 7f2b2f7ea700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b280b6080 con 0x7f2b28159ec0 2023-10-09T06:32:43.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.976+0000 7f2b2dd87700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f2b280cf350 0x7f2b280cd010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:42812/0 (socket says 172.21.15.163:42812) 2023-10-09T06:32:43.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.976+0000 7f2b2dd87700 1 -- 172.21.15.163:0/1768232821 learned_addr learned my addr 172.21.15.163:0/1768232821 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:43.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.977+0000 7f2b1ffff700 1 -- 172.21.15.163:0/1768232821 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3504251461 0 0) 0x7f2b280b5b00 con 0x7f2b280cf350 2023-10-09T06:32:43.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.977+0000 7f2b2d586700 1 --2- 172.21.15.163:0/1768232821 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b28159ec0 0x7f2b280b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:43.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.977+0000 7f2b1ffff700 1 -- 172.21.15.163:0/1768232821 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b14003570 con 0x7f2b280cf350 2023-10-09T06:32:43.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.977+0000 7f2b1ffff700 1 -- 172.21.15.163:0/1768232821 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1170320075 0 0) 0x7f2b14003570 con 0x7f2b280cf350 2023-10-09T06:32:43.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.977+0000 7f2b1ffff700 1 -- 172.21.15.163:0/1768232821 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b28159ec0 msgr2=0x7f2b280b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:43.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.977+0000 7f2b1ffff700 1 --2- 172.21.15.163:0/1768232821 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b28159ec0 0x7f2b280b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:43.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.977+0000 7f2b1ffff700 1 -- 172.21.15.163:0/1768232821 >> v1:172.21.15.181:6789/0 conn(0x7f2b280b53d0 legacy=0x7f2b280cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:43.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.977+0000 7f2b1ffff700 1 -- 172.21.15.163:0/1768232821 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b280cc100 con 0x7f2b280cf350 2023-10-09T06:32:43.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.977+0000 7f2b1ffff700 1 -- 172.21.15.163:0/1768232821 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f2b200017f0 con 0x7f2b280cf350 2023-10-09T06:32:43.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.977+0000 7f2b1ffff700 1 -- 172.21.15.163:0/1768232821 >> v1:172.21.15.163:6789/0 conn(0x7f2b280cf350 legacy=0x7f2b280cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:43.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.977+0000 7f2b1ffff700 1 --2- 172.21.15.163:0/1768232821 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b28159ec0 0x7f2b14003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:43.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.977+0000 7f2b1ffff700 1 --2- 172.21.15.163:0/1768232821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b14004010 0x7f2b14004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:43.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.977+0000 7f2b1ffff700 1 --2- 172.21.15.163:0/1768232821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b14004d50 0x7f2b140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:43.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.977+0000 7f2b1ffff700 1 -- 172.21.15.163:0/1768232821 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b280b6080 con 0x7f2b14004d50 2023-10-09T06:32:43.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.977+0000 7f2b1ffff700 1 -- 172.21.15.163:0/1768232821 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1400bbf0 con 0x7f2b14004010 2023-10-09T06:32:43.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.977+0000 7f2b1ffff700 1 -- 172.21.15.163:0/1768232821 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1400bdb0 con 0x7f2b28159ec0 2023-10-09T06:32:43.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.977+0000 7f2b2dd87700 1 --2- 172.21.15.163:0/1768232821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b14004d50 0x7f2b140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:43.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.977+0000 7f2b2d586700 1 --2- 172.21.15.163:0/1768232821 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b28159ec0 0x7f2b14003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:43.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.977+0000 7f2b2cd85700 1 --2- 172.21.15.163:0/1768232821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b14004010 0x7f2b14004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:43.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.977+0000 7f2b2dd87700 1 -- 172.21.15.163:0/1768232821 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b28159ec0 msgr2=0x7f2b14003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:43.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.977+0000 7f2b2dd87700 1 --2- 172.21.15.163:0/1768232821 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b28159ec0 0x7f2b14003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:43.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.977+0000 7f2b2dd87700 1 -- 172.21.15.163:0/1768232821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b14004010 msgr2=0x7f2b14004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:43.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.977+0000 7f2b2dd87700 1 --2- 172.21.15.163:0/1768232821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b14004010 0x7f2b14004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:43.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.977+0000 7f2b2dd87700 1 -- 172.21.15.163:0/1768232821 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b14003570 con 0x7f2b14004d50 2023-10-09T06:32:43.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.978+0000 7f2b2dd87700 1 --2- 172.21.15.163:0/1768232821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b14004d50 0x7f2b140051c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2b2000a900 tx=0x7f2b200021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5e705812cc3d62 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:43.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.978+0000 7f2b1ffff700 1 -- 172.21.15.163:0/1768232821 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b200070d0 con 0x7f2b14004d50 2023-10-09T06:32:43.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.978+0000 7f2b1ffff700 1 -- 172.21.15.163:0/1768232821 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b20007230 con 0x7f2b14004d50 2023-10-09T06:32:43.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.978+0000 7f2b1ffff700 1 -- 172.21.15.163:0/1768232821 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b200075f0 con 0x7f2b14004d50 2023-10-09T06:32:43.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.978+0000 7f2b2f7ea700 1 -- 172.21.15.163:0/1768232821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b14004d50 msgr2=0x7f2b140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:43.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.978+0000 7f2b2f7ea700 1 --2- 172.21.15.163:0/1768232821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b14004d50 0x7f2b140051c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2b2000a900 tx=0x7f2b200021d0 comp rx=0 tx=0).stop 2023-10-09T06:32:43.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.978+0000 7f2b2dd87700 1 -- 172.21.15.163:0/1768232821 reap_dead start 2023-10-09T06:32:43.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.978+0000 7f2b2f7ea700 1 -- 172.21.15.163:0/1768232821 shutdown_connections 2023-10-09T06:32:43.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.978+0000 7f2b2f7ea700 1 -- 172.21.15.163:0/1768232821 >> 172.21.15.163:0/1768232821 conn(0x7f2b280999f0 msgr2=0x7f2b280b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:43.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.978+0000 7f2b2f7ea700 1 -- 172.21.15.163:0/1768232821 shutdown_connections 2023-10-09T06:32:43.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.978+0000 7f2b2f7ea700 1 -- 172.21.15.163:0/1768232821 wait complete. 2023-10-09T06:32:43.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.979+0000 7f2b2f7ea700 1 Processor -- start 2023-10-09T06:32:43.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.979+0000 7f2b2f7ea700 1 -- start start 2023-10-09T06:32:43.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.980+0000 7f2b2f7ea700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b281572f0 0x7f2b2809a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:43.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.980+0000 7f2b2f7ea700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b2809cd90 0x7f2b280d0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:43.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.980+0000 7f2b2f7ea700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b280d0c40 0x7f2b28069360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:43.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.980+0000 7f2b2f7ea700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b280d10e0 con 0x7f2b281572f0 2023-10-09T06:32:43.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.980+0000 7f2b2f7ea700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b28169980 con 0x7f2b280d0c40 2023-10-09T06:32:43.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.980+0000 7f2b2f7ea700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2809acf0 con 0x7f2b2809cd90 2023-10-09T06:32:43.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.980+0000 7f2b2d586700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b281572f0 0x7f2b2809a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:43.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.980+0000 7f2b2d586700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b281572f0 0x7f2b2809a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:34524/0 (socket says 172.21.15.163:34524) 2023-10-09T06:32:43.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.980+0000 7f2b2d586700 1 -- 172.21.15.163:0/1823348928 learned_addr learned my addr 172.21.15.163:0/1823348928 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:43.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.981+0000 7f2b2cd85700 1 --2- 172.21.15.163:0/1823348928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b2809cd90 0x7f2b280d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:43.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.981+0000 7f2b2dd87700 1 --2- 172.21.15.163:0/1823348928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b280d0c40 0x7f2b28069360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:43.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.981+0000 7f2b2d586700 1 -- 172.21.15.163:0/1823348928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b2809cd90 msgr2=0x7f2b280d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:43.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.981+0000 7f2b2d586700 1 --2- 172.21.15.163:0/1823348928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b2809cd90 0x7f2b280d0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:43.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.981+0000 7f2b2d586700 1 -- 172.21.15.163:0/1823348928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b280d0c40 msgr2=0x7f2b28069360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:43.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.981+0000 7f2b2d586700 1 --2- 172.21.15.163:0/1823348928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b280d0c40 0x7f2b28069360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:43.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.981+0000 7f2b2d586700 1 -- 172.21.15.163:0/1823348928 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2809afd0 con 0x7f2b281572f0 2023-10-09T06:32:43.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.981+0000 7f2b2dd87700 1 --2- 172.21.15.163:0/1823348928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b280d0c40 0x7f2b28069360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:32:43.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.981+0000 7f2b2cd85700 1 --2- 172.21.15.163:0/1823348928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b2809cd90 0x7f2b280d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:43.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.981+0000 7f2b2d586700 1 --2- 172.21.15.163:0/1823348928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b281572f0 0x7f2b2809a460 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2b24011b10 tx=0x7f2b24009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:43.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.981+0000 7f2b1d7fa700 1 -- 172.21.15.163:0/1823348928 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b24005c10 con 0x7f2b281572f0 2023-10-09T06:32:43.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.981+0000 7f2b1d7fa700 1 -- 172.21.15.163:0/1823348928 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b24005d70 con 0x7f2b281572f0 2023-10-09T06:32:43.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.981+0000 7f2b1d7fa700 1 -- 172.21.15.163:0/1823348928 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2400f9b0 con 0x7f2b281572f0 2023-10-09T06:32:43.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.981+0000 7f2b2f7ea700 1 -- 172.21.15.163:0/1823348928 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b2809b2b0 con 0x7f2b281572f0 2023-10-09T06:32:43.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.981+0000 7f2b2f7ea700 1 -- 172.21.15.163:0/1823348928 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b2809a9a0 con 0x7f2b281572f0 2023-10-09T06:32:43.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.982+0000 7f2b2f7ea700 1 -- 172.21.15.163:0/1823348928 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b2809b750 con 0x7f2b281572f0 2023-10-09T06:32:43.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.982+0000 7f2b1d7fa700 1 -- 172.21.15.163:0/1823348928 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f2b2400fd60 con 0x7f2b281572f0 2023-10-09T06:32:43.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.983+0000 7f2b1d7fa700 1 --2- 172.21.15.163:0/1823348928 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2b0c066fc0 0x7f2b0c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:43.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.983+0000 7f2b1d7fa700 1 -- 172.21.15.163:0/1823348928 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2b24084ed0 con 0x7f2b281572f0 2023-10-09T06:32:43.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.983+0000 7f2b2cd85700 1 --2- 172.21.15.163:0/1823348928 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2b0c066fc0 0x7f2b0c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:43.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.983+0000 7f2b2cd85700 1 --2- 172.21.15.163:0/1823348928 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2b0c066fc0 0x7f2b0c069470 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2b2809bea0 tx=0x7f2b18009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:43.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:43.987+0000 7f2b1d7fa700 1 -- 172.21.15.163:0/1823348928 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2b24052150 con 0x7f2b281572f0 2023-10-09T06:32:44.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.214+0000 7f2b2f7ea700 1 -- 172.21.15.163:0/1823348928 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2b28063c90 con 0x7f2b281572f0 2023-10-09T06:32:44.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.556+0000 7f2b1d7fa700 1 -- 172.21.15.163:0/1823348928 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v41) v1 ==== 136+0+0 (secure 0 0 0) 0x7f2b24055770 con 0x7f2b281572f0 2023-10-09T06:32:44.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.559+0000 7f2b2f7ea700 1 -- 172.21.15.163:0/1823348928 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2b0c066fc0 msgr2=0x7f2b0c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:44.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.559+0000 7f2b2f7ea700 1 --2- 172.21.15.163:0/1823348928 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2b0c066fc0 0x7f2b0c069470 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2b2809bea0 tx=0x7f2b18009040 comp rx=0 tx=0).stop 2023-10-09T06:32:44.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.559+0000 7f2b2f7ea700 1 -- 172.21.15.163:0/1823348928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b281572f0 msgr2=0x7f2b2809a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:44.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.559+0000 7f2b2f7ea700 1 --2- 172.21.15.163:0/1823348928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b281572f0 0x7f2b2809a460 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2b24011b10 tx=0x7f2b24009120 comp rx=0 tx=0).stop 2023-10-09T06:32:44.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.559+0000 7f2b2f7ea700 1 -- 172.21.15.163:0/1823348928 shutdown_connections 2023-10-09T06:32:44.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.559+0000 7f2b2f7ea700 1 --2- 172.21.15.163:0/1823348928 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2b0c066fc0 0x7f2b0c069470 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:44.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.559+0000 7f2b2f7ea700 1 --2- 172.21.15.163:0/1823348928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b281572f0 0x7f2b2809a460 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:44.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.559+0000 7f2b2f7ea700 1 --2- 172.21.15.163:0/1823348928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b280d0c40 0x7f2b28069360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:44.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.559+0000 7f2b2f7ea700 1 --2- 172.21.15.163:0/1823348928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b2809cd90 0x7f2b280d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:44.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.559+0000 7f2b2f7ea700 1 -- 172.21.15.163:0/1823348928 >> 172.21.15.163:0/1823348928 conn(0x7f2b280999f0 msgr2=0x7f2b280c9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:44.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.559+0000 7f2b2f7ea700 1 -- 172.21.15.163:0/1823348928 shutdown_connections 2023-10-09T06:32:44.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.559+0000 7f2b2f7ea700 1 -- 172.21.15.163:0/1823348928 wait complete. 2023-10-09T06:32:44.571 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-09T06:32:44.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.687+0000 7f1cb50d2700 1 Processor -- start 2023-10-09T06:32:44.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.688+0000 7f1cb50d2700 1 -- start start 2023-10-09T06:32:44.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.688+0000 7f1cb50d2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1cb00ccf90 0x7f1cb00ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:44.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.688+0000 7f1cb50d2700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cb00cc300 con 0x7f1cb00b1420 2023-10-09T06:32:44.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.688+0000 7f1cb50d2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cb00ad370 con 0x7f1cb00cbf30 2023-10-09T06:32:44.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.688+0000 7f1cb50d2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cb0156bf0 con 0x7f1cb00ccf90 2023-10-09T06:32:44.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.689+0000 7f1cae59c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f1cb00b1420 0x7f1cb00c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:42822/0 (socket says 172.21.15.163:42822) 2023-10-09T06:32:44.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.689+0000 7f1cae59c700 1 -- 172.21.15.163:0/1156586542 learned_addr learned my addr 172.21.15.163:0/1156586542 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:44.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.689+0000 7f1caf59e700 1 --2- 172.21.15.163:0/1156586542 >> v2:172.21.15.181:3300/0 conn(0x7f1cb00ccf90 0x7f1cb00ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:44.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.689+0000 7f1cadd9b700 1 -- 172.21.15.163:0/1156586542 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1946849057 0 0) 0x7f1cb00cc300 con 0x7f1cb00b1420 2023-10-09T06:32:44.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.689+0000 7f1cadd9b700 1 -- 172.21.15.163:0/1156586542 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c98003570 con 0x7f1cb00b1420 2023-10-09T06:32:44.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.689+0000 7f1cadd9b700 1 -- 172.21.15.163:0/1156586542 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2670182892 0 0) 0x7f1cb00ad370 con 0x7f1cb00cbf30 2023-10-09T06:32:44.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.689+0000 7f1cadd9b700 1 -- 172.21.15.163:0/1156586542 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cb00cc300 con 0x7f1cb00cbf30 2023-10-09T06:32:44.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.689+0000 7f1caf59e700 1 -- 172.21.15.163:0/1156586542 >> v1:172.21.15.181:6789/0 conn(0x7f1cb00cbf30 legacy=0x7f1cb00b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:44.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.689+0000 7f1caf59e700 1 -- 172.21.15.163:0/1156586542 >> v1:172.21.15.163:6789/0 conn(0x7f1cb00b1420 legacy=0x7f1cb00c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:44.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.689+0000 7f1caf59e700 1 -- 172.21.15.163:0/1156586542 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb00ad650 con 0x7f1cb00ccf90 2023-10-09T06:32:44.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.689+0000 7f1caf59e700 1 --2- 172.21.15.163:0/1156586542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cb00ccf90 0x7f1cb00ae410 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8009e60 tx=0x7f1ca8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bb9db871f1236f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:44.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.689+0000 7f1cadd9b700 1 -- 172.21.15.163:0/1156586542 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca800e070 con 0x7f1cb00ccf90 2023-10-09T06:32:44.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.689+0000 7f1cadd9b700 1 -- 172.21.15.163:0/1156586542 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca800bab0 con 0x7f1cb00ccf90 2023-10-09T06:32:44.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.689+0000 7f1cadd9b700 1 -- 172.21.15.163:0/1156586542 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca800be70 con 0x7f1cb00ccf90 2023-10-09T06:32:44.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.690+0000 7f1cb50d2700 1 -- 172.21.15.163:0/1156586542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cb00ccf90 msgr2=0x7f1cb00ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:44.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.690+0000 7f1cb50d2700 1 --2- 172.21.15.163:0/1156586542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cb00ccf90 0x7f1cb00ae410 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8009e60 tx=0x7f1ca8009780 comp rx=0 tx=0).stop 2023-10-09T06:32:44.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.690+0000 7f1cb50d2700 1 -- 172.21.15.163:0/1156586542 shutdown_connections 2023-10-09T06:32:44.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.690+0000 7f1cb50d2700 1 --2- 172.21.15.163:0/1156586542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cb00ccf90 0x7f1cb00ae410 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:44.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.690+0000 7f1cb50d2700 1 -- 172.21.15.163:0/1156586542 >> 172.21.15.163:0/1156586542 conn(0x7f1cb006a320 msgr2=0x7f1cb00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:44.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.690+0000 7f1cb50d2700 1 -- 172.21.15.163:0/1156586542 shutdown_connections 2023-10-09T06:32:44.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.690+0000 7f1cb50d2700 1 -- 172.21.15.163:0/1156586542 wait complete. 2023-10-09T06:32:44.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.691+0000 7f1cb50d2700 1 Processor -- start 2023-10-09T06:32:44.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.691+0000 7f1cb50d2700 1 -- start start 2023-10-09T06:32:44.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.692+0000 7f1cb50d2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1cb00b1420 0x7f1cb016bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:44.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.692+0000 7f1cb50d2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cb00cbf30 0x7f1cb00ac140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:44.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.692+0000 7f1cb50d2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cb00ccf90 0x7f1cb0153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:44.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.692+0000 7f1cb50d2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb016c160 con 0x7f1cb00cbf30 2023-10-09T06:32:44.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.692+0000 7f1cb50d2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb00acda0 con 0x7f1cb00ccf90 2023-10-09T06:32:44.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.692+0000 7f1cb50d2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb00cb4c0 con 0x7f1cb00b1420 2023-10-09T06:32:44.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.692+0000 7f1cae59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cb00cbf30 0x7f1cb00ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:44.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.692+0000 7f1cae59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cb00cbf30 0x7f1cb00ac140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:34530/0 (socket says 172.21.15.163:34530) 2023-10-09T06:32:44.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.692+0000 7f1cae59c700 1 -- 172.21.15.163:0/436237019 learned_addr learned my addr 172.21.15.163:0/436237019 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:44.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.692+0000 7f1caf59e700 1 --2- 172.21.15.163:0/436237019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cb00ccf90 0x7f1cb0153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:44.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.692+0000 7f1cae59c700 1 -- 172.21.15.163:0/436237019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1cb00b1420 msgr2=0x7f1cb016bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:44.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.692+0000 7f1cae59c700 1 --2- 172.21.15.163:0/436237019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1cb00b1420 0x7f1cb016bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:44.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.692+0000 7f1cae59c700 1 -- 172.21.15.163:0/436237019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cb00ccf90 msgr2=0x7f1cb0153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:44.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.692+0000 7f1cae59c700 1 --2- 172.21.15.163:0/436237019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cb00ccf90 0x7f1cb0153c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:44.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.692+0000 7f1cae59c700 1 -- 172.21.15.163:0/436237019 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca8009a20 con 0x7f1cb00cbf30 2023-10-09T06:32:44.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.692+0000 7f1caf59e700 1 --2- 172.21.15.163:0/436237019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cb00ccf90 0x7f1cb0153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:32:44.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.692+0000 7f1cae59c700 1 --2- 172.21.15.163:0/436237019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cb00cbf30 0x7f1cb00ac140 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40058d0 tx=0x7f1ca4007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:44.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.693+0000 7f1c96ffd700 1 -- 172.21.15.163:0/436237019 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4005b30 con 0x7f1cb00cbf30 2023-10-09T06:32:44.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.693+0000 7f1c96ffd700 1 -- 172.21.15.163:0/436237019 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca4005c90 con 0x7f1cb00cbf30 2023-10-09T06:32:44.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.693+0000 7f1cb50d2700 1 -- 172.21.15.163:0/436237019 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb0167f60 con 0x7f1cb00cbf30 2023-10-09T06:32:44.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.693+0000 7f1c96ffd700 1 -- 172.21.15.163:0/436237019 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca40072e0 con 0x7f1cb00cbf30 2023-10-09T06:32:44.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.693+0000 7f1cb50d2700 1 -- 172.21.15.163:0/436237019 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb01685a0 con 0x7f1cb00cbf30 2023-10-09T06:32:44.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.694+0000 7f1cb50d2700 1 -- 172.21.15.163:0/436237019 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb0168730 con 0x7f1cb00cbf30 2023-10-09T06:32:44.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.694+0000 7f1c96ffd700 1 -- 172.21.15.163:0/436237019 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f1ca40076d0 con 0x7f1cb00cbf30 2023-10-09T06:32:44.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.694+0000 7f1c96ffd700 1 --2- 172.21.15.163:0/436237019 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1c9c066ff0 0x7f1c9c0694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:44.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.694+0000 7f1c96ffd700 1 -- 172.21.15.163:0/436237019 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1ca40839f0 con 0x7f1cb00cbf30 2023-10-09T06:32:44.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.695+0000 7f1caed9d700 1 --2- 172.21.15.163:0/436237019 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1c9c066ff0 0x7f1c9c0694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:44.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.695+0000 7f1caed9d700 1 --2- 172.21.15.163:0/436237019 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1c9c066ff0 0x7f1c9c0694a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00b0070 tx=0x7f1ca00079d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:44.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.698+0000 7f1c96ffd700 1 -- 172.21.15.163:0/436237019 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ca404b5a0 con 0x7f1cb00cbf30 2023-10-09T06:32:44.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.907+0000 7f1cb50d2700 1 -- 172.21.15.163:0/436237019 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f1cb004c3c0 con 0x7f1c9c066ff0 2023-10-09T06:32:44.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.908+0000 7f1c96ffd700 1 -- 172.21.15.163:0/436237019 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f1cb004c3c0 con 0x7f1c9c066ff0 2023-10-09T06:32:44.908 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:32:44.908 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:32:44.908 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "cephfs" 2023-10-09T06:32:44.908 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:32:44.908 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:32:44.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.910+0000 7f1cb50d2700 1 -- 172.21.15.163:0/436237019 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1c9c066ff0 msgr2=0x7f1c9c0694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:44.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.910+0000 7f1cb50d2700 1 --2- 172.21.15.163:0/436237019 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1c9c066ff0 0x7f1c9c0694a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00b0070 tx=0x7f1ca00079d0 comp rx=0 tx=0).stop 2023-10-09T06:32:44.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.910+0000 7f1cb50d2700 1 -- 172.21.15.163:0/436237019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cb00cbf30 msgr2=0x7f1cb00ac140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:44.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.910+0000 7f1cb50d2700 1 --2- 172.21.15.163:0/436237019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cb00cbf30 0x7f1cb00ac140 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40058d0 tx=0x7f1ca4007e10 comp rx=0 tx=0).stop 2023-10-09T06:32:44.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.910+0000 7f1cb50d2700 1 -- 172.21.15.163:0/436237019 shutdown_connections 2023-10-09T06:32:44.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.910+0000 7f1cb50d2700 1 --2- 172.21.15.163:0/436237019 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1c9c066ff0 0x7f1c9c0694a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:44.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.910+0000 7f1cb50d2700 1 --2- 172.21.15.163:0/436237019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1cb00ccf90 0x7f1cb0153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:44.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.910+0000 7f1cb50d2700 1 --2- 172.21.15.163:0/436237019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1cb00b1420 0x7f1cb016bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:44.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.910+0000 7f1cb50d2700 1 --2- 172.21.15.163:0/436237019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1cb00cbf30 0x7f1cb00ac140 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:44.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.910+0000 7f1cb50d2700 1 -- 172.21.15.163:0/436237019 >> 172.21.15.163:0/436237019 conn(0x7f1cb006a320 msgr2=0x7f1cb006eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:44.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.910+0000 7f1cb50d2700 1 -- 172.21.15.163:0/436237019 shutdown_connections 2023-10-09T06:32:44.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:44.910+0000 7f1cb50d2700 1 -- 172.21.15.163:0/436237019 wait complete. 2023-10-09T06:32:44.922 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-09T06:32:45.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.040+0000 7f100ffff700 1 Processor -- start 2023-10-09T06:32:45.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.040+0000 7f100ffff700 1 -- start start 2023-10-09T06:32:45.049 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.041+0000 7f100ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f10100b16d0 0x7f10100b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:45.049 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.041+0000 7f100ffff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10100b2080 con 0x7f10100b4e70 2023-10-09T06:32:45.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.041+0000 7f100ffff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10100cfb00 con 0x7f10100b33b0 2023-10-09T06:32:45.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.041+0000 7f100ffff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10100b5c40 con 0x7f10100b16d0 2023-10-09T06:32:45.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.041+0000 7f100effd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f10100b4e70 0x7f10100ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:42836/0 (socket says 172.21.15.163:42836) 2023-10-09T06:32:45.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.041+0000 7f100effd700 1 -- 172.21.15.163:0/1813162598 learned_addr learned my addr 172.21.15.163:0/1813162598 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:45.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.041+0000 7f100e7fc700 1 --2- 172.21.15.163:0/1813162598 >> v2:172.21.15.181:3300/0 conn(0x7f10100b16d0 0x7f10100b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:45.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.041+0000 7f100dffb700 1 -- 172.21.15.163:0/1813162598 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 639921068 0 0) 0x7f10100b2080 con 0x7f10100b4e70 2023-10-09T06:32:45.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.041+0000 7f100dffb700 1 -- 172.21.15.163:0/1813162598 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ffc003570 con 0x7f10100b4e70 2023-10-09T06:32:45.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.041+0000 7f100dffb700 1 -- 172.21.15.163:0/1813162598 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3596792463 0 0) 0x7f10100cfb00 con 0x7f10100b33b0 2023-10-09T06:32:45.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.041+0000 7f100dffb700 1 -- 172.21.15.163:0/1813162598 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10100b2080 con 0x7f10100b33b0 2023-10-09T06:32:45.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.042+0000 7f100dffb700 1 -- 172.21.15.163:0/1813162598 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 997232123 0 0) 0x7f0ffc003570 con 0x7f10100b4e70 2023-10-09T06:32:45.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.042+0000 7f100dffb700 1 -- 172.21.15.163:0/1813162598 >> v2:172.21.15.181:3300/0 conn(0x7f10100b16d0 msgr2=0x7f10100b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:45.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.042+0000 7f100dffb700 1 --2- 172.21.15.163:0/1813162598 >> v2:172.21.15.181:3300/0 conn(0x7f10100b16d0 0x7f10100b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:45.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.042+0000 7f100dffb700 1 -- 172.21.15.163:0/1813162598 >> v1:172.21.15.181:6789/0 conn(0x7f10100b33b0 legacy=0x7f10100b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:45.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.042+0000 7f100dffb700 1 -- 172.21.15.163:0/1813162598 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10100b5f20 con 0x7f10100b4e70 2023-10-09T06:32:45.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.042+0000 7f100dffb700 1 -- 172.21.15.163:0/1813162598 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f10080017f0 con 0x7f10100b4e70 2023-10-09T06:32:45.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.042+0000 7f100e7fc700 1 --2- 172.21.15.163:0/1813162598 >> v2:172.21.15.181:3300/0 conn(0x7f10100b16d0 0x7f10100b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:32:45.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.042+0000 7f100dffb700 1 -- 172.21.15.163:0/1813162598 >> v1:172.21.15.163:6789/0 conn(0x7f10100b4e70 legacy=0x7f10100ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:45.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.042+0000 7f100dffb700 1 --2- 172.21.15.163:0/1813162598 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ffc003bf0 0x7f0ffc003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:45.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.042+0000 7f100dffb700 1 --2- 172.21.15.163:0/1813162598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ffc004810 0x7f0ffc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:45.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.042+0000 7f100dffb700 1 --2- 172.21.15.163:0/1813162598 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ffc005230 0x7f0ffc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:45.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.042+0000 7f100dffb700 1 -- 172.21.15.163:0/1813162598 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10100b5c40 con 0x7f0ffc005230 2023-10-09T06:32:45.052 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.042+0000 7f100dffb700 1 -- 172.21.15.163:0/1813162598 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ffc00c200 con 0x7f0ffc004810 2023-10-09T06:32:45.052 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.042+0000 7f100dffb700 1 -- 172.21.15.163:0/1813162598 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ffc00c3c0 con 0x7f0ffc003bf0 2023-10-09T06:32:45.052 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.042+0000 7f100f7fe700 1 --2- 172.21.15.163:0/1813162598 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ffc005230 0x7f0ffc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:45.052 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.042+0000 7f100effd700 1 --2- 172.21.15.163:0/1813162598 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ffc003bf0 0x7f0ffc003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:45.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.042+0000 7f100e7fc700 1 --2- 172.21.15.163:0/1813162598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ffc004810 0x7f0ffc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:45.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.042+0000 7f100f7fe700 1 -- 172.21.15.163:0/1813162598 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ffc003bf0 msgr2=0x7f0ffc003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:45.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.042+0000 7f100f7fe700 1 --2- 172.21.15.163:0/1813162598 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ffc003bf0 0x7f0ffc003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:45.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.042+0000 7f100f7fe700 1 -- 172.21.15.163:0/1813162598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ffc004810 msgr2=0x7f0ffc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:45.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.042+0000 7f100f7fe700 1 --2- 172.21.15.163:0/1813162598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ffc004810 0x7f0ffc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:45.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.042+0000 7f100f7fe700 1 -- 172.21.15.163:0/1813162598 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ffc003570 con 0x7f0ffc005230 2023-10-09T06:32:45.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.042+0000 7f100f7fe700 1 -- 172.21.15.163:0/1813162598 reap_dead start 2023-10-09T06:32:45.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.042+0000 7f100f7fe700 1 --2- 172.21.15.163:0/1813162598 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ffc005230 0x7f0ffc00ba60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f1000001a10 tx=0x7f1000007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15bc5ea3ace19f5b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:45.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.042+0000 7f100dffb700 1 -- 172.21.15.163:0/1813162598 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100000c5d0 con 0x7f0ffc005230 2023-10-09T06:32:45.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.042+0000 7f100dffb700 1 -- 172.21.15.163:0/1813162598 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100000c730 con 0x7f0ffc005230 2023-10-09T06:32:45.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.042+0000 7f100dffb700 1 -- 172.21.15.163:0/1813162598 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100000cad0 con 0x7f0ffc005230 2023-10-09T06:32:45.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.043+0000 7f100ffff700 1 -- 172.21.15.163:0/1813162598 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ffc005230 msgr2=0x7f0ffc00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:45.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.043+0000 7f100ffff700 1 --2- 172.21.15.163:0/1813162598 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ffc005230 0x7f0ffc00ba60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f1000001a10 tx=0x7f1000007d30 comp rx=0 tx=0).stop 2023-10-09T06:32:45.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.043+0000 7f100ffff700 1 -- 172.21.15.163:0/1813162598 shutdown_connections 2023-10-09T06:32:45.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.043+0000 7f100ffff700 1 --2- 172.21.15.163:0/1813162598 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ffc005230 0x7f0ffc00ba60 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:45.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.043+0000 7f100ffff700 1 -- 172.21.15.163:0/1813162598 >> 172.21.15.163:0/1813162598 conn(0x7f10100d10c0 msgr2=0x7f10100affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:45.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.043+0000 7f100ffff700 1 -- 172.21.15.163:0/1813162598 shutdown_connections 2023-10-09T06:32:45.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.043+0000 7f100ffff700 1 -- 172.21.15.163:0/1813162598 wait complete. 2023-10-09T06:32:45.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.044+0000 7f100ffff700 1 Processor -- start 2023-10-09T06:32:45.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.044+0000 7f100ffff700 1 -- start start 2023-10-09T06:32:45.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.045+0000 7f100ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ffc005230 0x7f1010157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:45.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.045+0000 7f100ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10100c9c30 0x7f10100ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:45.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.045+0000 7f100ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10100cc020 0x7f10100cc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:45.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.045+0000 7f100ffff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10100cc9d0 con 0x7f10100cc020 2023-10-09T06:32:45.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.045+0000 7f100ffff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10100cb300 con 0x7f0ffc005230 2023-10-09T06:32:45.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.045+0000 7f100ffff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10100ca6f0 con 0x7f10100c9c30 2023-10-09T06:32:45.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.045+0000 7f100f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10100cc020 0x7f10100cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:45.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.045+0000 7f100f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10100cc020 0x7f10100cc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:34540/0 (socket says 172.21.15.163:34540) 2023-10-09T06:32:45.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.045+0000 7f100f7fe700 1 -- 172.21.15.163:0/3264479185 learned_addr learned my addr 172.21.15.163:0/3264479185 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:45.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.045+0000 7f100effd700 1 --2- 172.21.15.163:0/3264479185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ffc005230 0x7f1010157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:45.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.045+0000 7f100e7fc700 1 --2- 172.21.15.163:0/3264479185 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10100c9c30 0x7f10100ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:45.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.045+0000 7f100f7fe700 1 -- 172.21.15.163:0/3264479185 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10100c9c30 msgr2=0x7f10100ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:45.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.045+0000 7f100f7fe700 1 --2- 172.21.15.163:0/3264479185 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10100c9c30 0x7f10100ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:45.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.045+0000 7f100f7fe700 1 -- 172.21.15.163:0/3264479185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ffc005230 msgr2=0x7f1010157920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:45.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.045+0000 7f100f7fe700 1 --2- 172.21.15.163:0/3264479185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ffc005230 0x7f1010157920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:45.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.045+0000 7f100f7fe700 1 -- 172.21.15.163:0/3264479185 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10100ca9d0 con 0x7f10100cc020 2023-10-09T06:32:45.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.045+0000 7f100effd700 1 --2- 172.21.15.163:0/3264479185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ffc005230 0x7f1010157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:45.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.045+0000 7f100e7fc700 1 --2- 172.21.15.163:0/3264479185 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10100c9c30 0x7f10100ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:32:45.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.045+0000 7f100f7fe700 1 --2- 172.21.15.163:0/3264479185 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10100cc020 0x7f10100cc490 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f101006b650 tx=0x7f1000014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:45.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.046+0000 7f1006ffd700 1 -- 172.21.15.163:0/3264479185 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1000000fe0 con 0x7f10100cc020 2023-10-09T06:32:45.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.046+0000 7f1006ffd700 1 -- 172.21.15.163:0/3264479185 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100000ce80 con 0x7f10100cc020 2023-10-09T06:32:45.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.046+0000 7f1006ffd700 1 -- 172.21.15.163:0/3264479185 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1000007480 con 0x7f10100cc020 2023-10-09T06:32:45.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.046+0000 7f100ffff700 1 -- 172.21.15.163:0/3264479185 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10100cacb0 con 0x7f10100cc020 2023-10-09T06:32:45.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.046+0000 7f100ffff700 1 -- 172.21.15.163:0/3264479185 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10100a7130 con 0x7f10100cc020 2023-10-09T06:32:45.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.047+0000 7f100ffff700 1 -- 172.21.15.163:0/3264479185 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10100cb150 con 0x7f10100cc020 2023-10-09T06:32:45.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.047+0000 7f1006ffd700 1 -- 172.21.15.163:0/3264479185 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f1000013430 con 0x7f10100cc020 2023-10-09T06:32:45.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.047+0000 7f1006ffd700 1 --2- 172.21.15.163:0/3264479185 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0ff0066fc0 0x7f0ff0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:45.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.047+0000 7f1006ffd700 1 -- 172.21.15.163:0/3264479185 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1000084f10 con 0x7f10100cc020 2023-10-09T06:32:45.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.048+0000 7f100effd700 1 --2- 172.21.15.163:0/3264479185 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0ff0066fc0 0x7f0ff0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:45.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.048+0000 7f100effd700 1 --2- 172.21.15.163:0/3264479185 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0ff0066fc0 0x7f0ff0069470 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1008001190 tx=0x7f1008006040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:45.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.051+0000 7f1006ffd700 1 -- 172.21.15.163:0/3264479185 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1000052190 con 0x7f10100cc020 2023-10-09T06:32:45.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.233+0000 7f100ffff700 1 -- 172.21.15.163:0/3264479185 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f10100cb150 con 0x7f10100cc020 2023-10-09T06:32:45.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.233+0000 7f1006ffd700 1 -- 172.21.15.163:0/3264479185 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f10000557b0 con 0x7f10100cc020 2023-10-09T06:32:45.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.235+0000 7f100ffff700 1 -- 172.21.15.163:0/3264479185 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0ff0066fc0 msgr2=0x7f0ff0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:45.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.235+0000 7f100ffff700 1 --2- 172.21.15.163:0/3264479185 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0ff0066fc0 0x7f0ff0069470 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1008001190 tx=0x7f1008006040 comp rx=0 tx=0).stop 2023-10-09T06:32:45.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.235+0000 7f100ffff700 1 -- 172.21.15.163:0/3264479185 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10100cc020 msgr2=0x7f10100cc490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:45.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.235+0000 7f100ffff700 1 --2- 172.21.15.163:0/3264479185 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10100cc020 0x7f10100cc490 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f101006b650 tx=0x7f1000014000 comp rx=0 tx=0).stop 2023-10-09T06:32:45.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.235+0000 7f100ffff700 1 -- 172.21.15.163:0/3264479185 shutdown_connections 2023-10-09T06:32:45.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.235+0000 7f100ffff700 1 --2- 172.21.15.163:0/3264479185 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0ff0066fc0 0x7f0ff0069470 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:45.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.235+0000 7f100ffff700 1 --2- 172.21.15.163:0/3264479185 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10100cc020 0x7f10100cc490 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:45.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.235+0000 7f100ffff700 1 --2- 172.21.15.163:0/3264479185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ffc005230 0x7f1010157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:45.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.235+0000 7f100ffff700 1 --2- 172.21.15.163:0/3264479185 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10100c9c30 0x7f10100ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:45.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.235+0000 7f100ffff700 1 -- 172.21.15.163:0/3264479185 >> 172.21.15.163:0/3264479185 conn(0x7f10100d10c0 msgr2=0x7f1010069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:45.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.235+0000 7f100ffff700 1 -- 172.21.15.163:0/3264479185 shutdown_connections 2023-10-09T06:32:45.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.235+0000 7f100ffff700 1 -- 172.21.15.163:0/3264479185 wait complete. 2023-10-09T06:32:45.247 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000333763 --mode=777 2023-10-09T06:32:45.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.366+0000 7f72d90c1700 1 Processor -- start 2023-10-09T06:32:45.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.366+0000 7f72d90c1700 1 -- start start 2023-10-09T06:32:45.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.367+0000 7f72d90c1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f72d4154320 0x7f72d406d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:45.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.367+0000 7f72d90c1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f72d415a190 0x7f72d406d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:45.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.367+0000 7f72d90c1700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d40b4570 con 0x7f72d4153f50 2023-10-09T06:32:45.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.367+0000 7f72d90c1700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72d40b2710 con 0x7f72d415a190 2023-10-09T06:32:45.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.367+0000 7f72d90c1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72d415a730 con 0x7f72d4154320 2023-10-09T06:32:45.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.367+0000 7f72d2d9d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f72d4153f50 0x7f72d40b2000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:42848/0 (socket says 172.21.15.163:42848) 2023-10-09T06:32:45.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.367+0000 7f72d2d9d700 1 -- 172.21.15.163:0/72988363 learned_addr learned my addr 172.21.15.163:0/72988363 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:45.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.367+0000 7f72d359e700 1 --2- 172.21.15.163:0/72988363 >> v2:172.21.15.163:3300/0 conn(0x7f72d415a190 0x7f72d406d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:45.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.367+0000 7f72d259c700 1 -- 172.21.15.163:0/72988363 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2211040180 0 0) 0x7f72d40b4570 con 0x7f72d4153f50 2023-10-09T06:32:45.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.367+0000 7f72ca59c700 1 --2- 172.21.15.163:0/72988363 >> v2:172.21.15.181:3300/0 conn(0x7f72d4154320 0x7f72d406d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:45.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.368+0000 7f72d259c700 1 -- 172.21.15.163:0/72988363 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72c0003570 con 0x7f72d4153f50 2023-10-09T06:32:45.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.368+0000 7f72d359e700 1 -- 172.21.15.163:0/72988363 >> v2:172.21.15.181:3300/0 conn(0x7f72d4154320 msgr2=0x7f72d406d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:45.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.368+0000 7f72d359e700 1 --2- 172.21.15.163:0/72988363 >> v2:172.21.15.181:3300/0 conn(0x7f72d4154320 0x7f72d406d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:45.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.368+0000 7f72d359e700 1 -- 172.21.15.163:0/72988363 >> v1:172.21.15.163:6789/0 conn(0x7f72d4153f50 legacy=0x7f72d40b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:45.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.368+0000 7f72d359e700 1 -- 172.21.15.163:0/72988363 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d415aa10 con 0x7f72d415a190 2023-10-09T06:32:45.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.368+0000 7f72d359e700 1 --2- 172.21.15.163:0/72988363 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72d415a190 0x7f72d406d670 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f72d40b36a0 tx=0x7f72cc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba35d2135e28531 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:45.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.368+0000 7f72d259c700 1 -- 172.21.15.163:0/72988363 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc00e070 con 0x7f72d415a190 2023-10-09T06:32:45.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.368+0000 7f72d259c700 1 -- 172.21.15.163:0/72988363 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72cc00bb60 con 0x7f72d415a190 2023-10-09T06:32:45.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.368+0000 7f72d259c700 1 -- 172.21.15.163:0/72988363 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc012430 con 0x7f72d415a190 2023-10-09T06:32:45.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.368+0000 7f72d90c1700 1 -- 172.21.15.163:0/72988363 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72d415a190 msgr2=0x7f72d406d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:45.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.368+0000 7f72d90c1700 1 --2- 172.21.15.163:0/72988363 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72d415a190 0x7f72d406d670 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f72d40b36a0 tx=0x7f72cc009780 comp rx=0 tx=0).stop 2023-10-09T06:32:45.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.368+0000 7f72d90c1700 1 -- 172.21.15.163:0/72988363 shutdown_connections 2023-10-09T06:32:45.374 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.368+0000 7f72d90c1700 1 --2- 172.21.15.163:0/72988363 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72d415a190 0x7f72d406d670 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:45.374 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.368+0000 7f72d90c1700 1 --2- 172.21.15.163:0/72988363 >> v2:172.21.15.181:3300/0 conn(0x7f72d4154320 0x7f72d406d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:45.374 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.368+0000 7f72d90c1700 1 -- 172.21.15.163:0/72988363 >> 172.21.15.163:0/72988363 conn(0x7f72d406baf0 msgr2=0x7f72d406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:45.374 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.368+0000 7f72d90c1700 1 -- 172.21.15.163:0/72988363 shutdown_connections 2023-10-09T06:32:45.374 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.368+0000 7f72d90c1700 1 -- 172.21.15.163:0/72988363 wait complete. 2023-10-09T06:32:45.374 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.370+0000 7f72d90c1700 1 Processor -- start 2023-10-09T06:32:45.374 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.370+0000 7f72d90c1700 1 -- start start 2023-10-09T06:32:45.374 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.370+0000 7f72d90c1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72d4153f50 0x7f72d40aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:45.374 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.370+0000 7f72d90c1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d4154320 0x7f72d414b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:45.374 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.370+0000 7f72d90c1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72d415a190 0x7f72d415b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:45.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.370+0000 7f72d90c1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d40af3e0 con 0x7f72d415a190 2023-10-09T06:32:45.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.370+0000 7f72d90c1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d414f080 con 0x7f72d4154320 2023-10-09T06:32:45.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.370+0000 7f72d90c1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d415dd30 con 0x7f72d4153f50 2023-10-09T06:32:45.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.370+0000 7f72d359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72d415a190 0x7f72d415b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:45.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.370+0000 7f72d359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72d415a190 0x7f72d415b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:34570/0 (socket says 172.21.15.163:34570) 2023-10-09T06:32:45.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.370+0000 7f72d359e700 1 -- 172.21.15.163:0/64193450 learned_addr learned my addr 172.21.15.163:0/64193450 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:45.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.371+0000 7f72ca59c700 1 --2- 172.21.15.163:0/64193450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d4154320 0x7f72d414b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:45.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.371+0000 7f72d2d9d700 1 --2- 172.21.15.163:0/64193450 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72d4153f50 0x7f72d40aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:45.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.371+0000 7f72d359e700 1 -- 172.21.15.163:0/64193450 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72d4153f50 msgr2=0x7f72d40aeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:45.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.371+0000 7f72d359e700 1 --2- 172.21.15.163:0/64193450 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72d4153f50 0x7f72d40aeea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:45.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.371+0000 7f72d359e700 1 -- 172.21.15.163:0/64193450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d4154320 msgr2=0x7f72d414b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:45.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.371+0000 7f72d359e700 1 --2- 172.21.15.163:0/64193450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d4154320 0x7f72d414b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:45.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.371+0000 7f72d359e700 1 -- 172.21.15.163:0/64193450 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc009a20 con 0x7f72d415a190 2023-10-09T06:32:45.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.371+0000 7f72d359e700 1 --2- 172.21.15.163:0/64193450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72d415a190 0x7f72d415b410 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f72cc009fd0 tx=0x7f72cc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:45.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.371+0000 7f72cb7fe700 1 -- 172.21.15.163:0/64193450 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc027470 con 0x7f72d415a190 2023-10-09T06:32:45.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.371+0000 7f72cb7fe700 1 -- 172.21.15.163:0/64193450 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72cc0275f0 con 0x7f72d415a190 2023-10-09T06:32:45.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.371+0000 7f72d90c1700 1 -- 172.21.15.163:0/64193450 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d414e1b0 con 0x7f72d415a190 2023-10-09T06:32:45.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.371+0000 7f72cb7fe700 1 -- 172.21.15.163:0/64193450 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc0279b0 con 0x7f72d415a190 2023-10-09T06:32:45.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.371+0000 7f72d90c1700 1 -- 172.21.15.163:0/64193450 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d415d6e0 con 0x7f72d415a190 2023-10-09T06:32:45.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.372+0000 7f72d90c1700 1 -- 172.21.15.163:0/64193450 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d40718b0 con 0x7f72d415a190 2023-10-09T06:32:45.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.372+0000 7f72cb7fe700 1 -- 172.21.15.163:0/64193450 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f72cc024070 con 0x7f72d415a190 2023-10-09T06:32:45.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.373+0000 7f72cb7fe700 1 --2- 172.21.15.163:0/64193450 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f72b4066fc0 0x7f72b4069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:45.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.373+0000 7f72cb7fe700 1 -- 172.21.15.163:0/64193450 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f72cc086f30 con 0x7f72d415a190 2023-10-09T06:32:45.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.373+0000 7f72d2d9d700 1 --2- 172.21.15.163:0/64193450 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f72b4066fc0 0x7f72b4069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:45.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.374+0000 7f72d2d9d700 1 --2- 172.21.15.163:0/64193450 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f72b4066fc0 0x7f72b4069470 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f72c4007fd0 tx=0x7f72c4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:45.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.377+0000 7f72cb7fe700 1 -- 172.21.15.163:0/64193450 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72cc0b85a0 con 0x7f72d415a190 2023-10-09T06:32:45.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.589+0000 7f72d90c1700 1 -- 172.21.15.163:0/64193450 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000333763", "mode": "777", "target": ["mon-mgr", ""]}) v1 -- 0x7f72d415acf0 con 0x7f72b4066fc0 2023-10-09T06:32:45.592 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:32:45.592+0000 7ff9594e4700 -1 client.4961 remove_session_caps still has dirty|flushing caps on 0x10000000001.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2023-10-09T06:32:15.648693+0000 mtime=2023-10-09T06:32:15.648693+0000 ctime=2023-10-09T06:32:15.654595+0000 change_attr=2 caps=- flushing_caps=Ax COMPLETE parents=0x10000000000.head["group_0000000000034793"] has_dir_layout 0x55f9b12f4000) 2023-10-09T06:32:45.596 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:32:45.596+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:32:45.596 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:32:45.596+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:32:45.596 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:32:45.596+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:32:45.596 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:32:45.596+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:32:45.596 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:32:45.596+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:32:45.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.624+0000 7f72cb7fe700 1 -- 172.21.15.163:0/64193450 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f72d415acf0 con 0x7f72b4066fc0 2023-10-09T06:32:45.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.626+0000 7f72d90c1700 1 -- 172.21.15.163:0/64193450 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f72b4066fc0 msgr2=0x7f72b4069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:45.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.626+0000 7f72d90c1700 1 --2- 172.21.15.163:0/64193450 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f72b4066fc0 0x7f72b4069470 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f72c4007fd0 tx=0x7f72c4009040 comp rx=0 tx=0).stop 2023-10-09T06:32:45.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.626+0000 7f72d90c1700 1 -- 172.21.15.163:0/64193450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72d415a190 msgr2=0x7f72d415b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:45.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.626+0000 7f72d90c1700 1 --2- 172.21.15.163:0/64193450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72d415a190 0x7f72d415b410 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f72cc009fd0 tx=0x7f72cc019000 comp rx=0 tx=0).stop 2023-10-09T06:32:45.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.627+0000 7f72d90c1700 1 -- 172.21.15.163:0/64193450 shutdown_connections 2023-10-09T06:32:45.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.627+0000 7f72d90c1700 1 --2- 172.21.15.163:0/64193450 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f72b4066fc0 0x7f72b4069470 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:45.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.627+0000 7f72d90c1700 1 --2- 172.21.15.163:0/64193450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72d415a190 0x7f72d415b410 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:45.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.627+0000 7f72d90c1700 1 --2- 172.21.15.163:0/64193450 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72d4153f50 0x7f72d40aeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:45.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.627+0000 7f72d90c1700 1 --2- 172.21.15.163:0/64193450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72d4154320 0x7f72d414b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:45.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.627+0000 7f72d90c1700 1 -- 172.21.15.163:0/64193450 >> 172.21.15.163:0/64193450 conn(0x7f72d406baf0 msgr2=0x7f72d40adac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:45.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.627+0000 7f72d90c1700 1 -- 172.21.15.163:0/64193450 shutdown_connections 2023-10-09T06:32:45.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.627+0000 7f72d90c1700 1 -- 172.21.15.163:0/64193450 wait complete. 2023-10-09T06:32:45.639 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000333763 2023-10-09T06:32:45.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.757+0000 7ff59ee3c700 1 Processor -- start 2023-10-09T06:32:45.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.757+0000 7ff59ee3c700 1 -- start start 2023-10-09T06:32:45.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.758+0000 7ff59ee3c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff59806eb80 0x7ff598159980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:45.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.758+0000 7ff59ee3c700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5980cbf40 con 0x7ff5980b23f0 2023-10-09T06:32:45.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.758+0000 7ff59ee3c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff59815a6b0 con 0x7ff5980b2020 2023-10-09T06:32:45.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.758+0000 7ff59ee3c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5980cb610 con 0x7ff59806eb80 2023-10-09T06:32:45.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.758+0000 7ff59e63b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff5980b23f0 0x7ff598159ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:42856/0 (socket says 172.21.15.163:42856) 2023-10-09T06:32:45.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.758+0000 7ff59e63b700 1 -- 172.21.15.163:0/2728809719 learned_addr learned my addr 172.21.15.163:0/2728809719 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:32:45.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.759+0000 7ff59d639700 1 --2- 172.21.15.163:0/2728809719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff59806eb80 0x7ff598159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:45.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.759+0000 7ff59ce38700 1 -- 172.21.15.163:0/2728809719 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1516367213 0 0) 0x7ff5980cbf40 con 0x7ff5980b23f0 2023-10-09T06:32:45.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.759+0000 7ff59ce38700 1 -- 172.21.15.163:0/2728809719 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff584004680 con 0x7ff5980b23f0 2023-10-09T06:32:45.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.759+0000 7ff59ce38700 1 -- 172.21.15.163:0/2728809719 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1712914404 0 0) 0x7ff584004680 con 0x7ff5980b23f0 2023-10-09T06:32:45.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.759+0000 7ff59ce38700 1 -- 172.21.15.163:0/2728809719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff59806eb80 msgr2=0x7ff598159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:45.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.759+0000 7ff59ce38700 1 --2- 172.21.15.163:0/2728809719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff59806eb80 0x7ff598159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:45.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.759+0000 7ff59ce38700 1 -- 172.21.15.163:0/2728809719 >> v1:172.21.15.181:6789/0 conn(0x7ff5980b2020 legacy=0x7ff59806e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:45.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.759+0000 7ff59ce38700 1 -- 172.21.15.163:0/2728809719 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5980cb8f0 con 0x7ff5980b23f0 2023-10-09T06:32:45.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.759+0000 7ff59ce38700 1 -- 172.21.15.163:0/2728809719 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7ff590001470 con 0x7ff5980b23f0 2023-10-09T06:32:45.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.759+0000 7ff59ce38700 1 -- 172.21.15.163:0/2728809719 >> v1:172.21.15.163:6789/0 conn(0x7ff5980b23f0 legacy=0x7ff598159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:45.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.759+0000 7ff59ce38700 1 --2- 172.21.15.163:0/2728809719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff584004ba0 0x7ff584004fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:45.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.759+0000 7ff59ce38700 1 --2- 172.21.15.163:0/2728809719 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff584005920 0x7ff584005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:45.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.759+0000 7ff59ce38700 1 --2- 172.21.15.163:0/2728809719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff59806eb80 0x7ff5840062f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:45.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.759+0000 7ff59ce38700 1 -- 172.21.15.163:0/2728809719 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5980cb610 con 0x7ff584005920 2023-10-09T06:32:45.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.759+0000 7ff59ce38700 1 -- 172.21.15.163:0/2728809719 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58400cc70 con 0x7ff584004ba0 2023-10-09T06:32:45.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.759+0000 7ff59ce38700 1 -- 172.21.15.163:0/2728809719 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5840035f0 con 0x7ff59806eb80 2023-10-09T06:32:45.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.759+0000 7ff59d639700 1 --2- 172.21.15.163:0/2728809719 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff584005920 0x7ff584005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:45.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.759+0000 7ff59de3a700 1 --2- 172.21.15.163:0/2728809719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff584004ba0 0x7ff584004fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:45.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.759+0000 7ff59e63b700 1 --2- 172.21.15.163:0/2728809719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff59806eb80 0x7ff5840062f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:45.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.759+0000 7ff59d639700 1 -- 172.21.15.163:0/2728809719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff59806eb80 msgr2=0x7ff5840062f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:45.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.759+0000 7ff59d639700 1 --2- 172.21.15.163:0/2728809719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff59806eb80 0x7ff5840062f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:45.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.759+0000 7ff59d639700 1 -- 172.21.15.163:0/2728809719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff584004ba0 msgr2=0x7ff584004fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:32:45.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.759+0000 7ff59d639700 1 --2- 172.21.15.163:0/2728809719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff584004ba0 0x7ff584004fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:45.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.759+0000 7ff59d639700 1 -- 172.21.15.163:0/2728809719 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff584004680 con 0x7ff584005920 2023-10-09T06:32:45.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.760+0000 7ff59d639700 1 --2- 172.21.15.163:0/2728809719 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff584005920 0x7ff584005d90 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff588009c00 tx=0x7ff5880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d81e85441722ee9 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:45.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.760+0000 7ff59ce38700 1 -- 172.21.15.163:0/2728809719 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58800e710 con 0x7ff584005920 2023-10-09T06:32:45.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.760+0000 7ff59ce38700 1 -- 172.21.15.163:0/2728809719 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58800e870 con 0x7ff584005920 2023-10-09T06:32:45.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.760+0000 7ff59ce38700 1 -- 172.21.15.163:0/2728809719 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58800ec30 con 0x7ff584005920 2023-10-09T06:32:45.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.760+0000 7ff59ee3c700 1 -- 172.21.15.163:0/2728809719 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff584005920 msgr2=0x7ff584005d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:45.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.760+0000 7ff59ee3c700 1 --2- 172.21.15.163:0/2728809719 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff584005920 0x7ff584005d90 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff588009c00 tx=0x7ff5880091a0 comp rx=0 tx=0).stop 2023-10-09T06:32:45.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.760+0000 7ff59e63b700 1 -- 172.21.15.163:0/2728809719 reap_dead start 2023-10-09T06:32:45.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.760+0000 7ff59ee3c700 1 -- 172.21.15.163:0/2728809719 shutdown_connections 2023-10-09T06:32:45.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.760+0000 7ff59ee3c700 1 -- 172.21.15.163:0/2728809719 >> 172.21.15.163:0/2728809719 conn(0x7ff5980c9f70 msgr2=0x7ff59806dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:45.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.760+0000 7ff59ee3c700 1 -- 172.21.15.163:0/2728809719 shutdown_connections 2023-10-09T06:32:45.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.760+0000 7ff59ee3c700 1 -- 172.21.15.163:0/2728809719 wait complete. 2023-10-09T06:32:45.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.761+0000 7ff59ee3c700 1 Processor -- start 2023-10-09T06:32:45.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.761+0000 7ff59ee3c700 1 -- start start 2023-10-09T06:32:45.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.762+0000 7ff59ee3c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff584005920 0x7ff5981570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:45.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.762+0000 7ff59ee3c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff59816a800 0x7ff5980c6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:45.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.762+0000 7ff59ee3c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff59816ac70 0x7ff5980c66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:45.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.762+0000 7ff59ee3c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff598153ce0 con 0x7ff59816a800 2023-10-09T06:32:45.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.762+0000 7ff59ee3c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5981575f0 con 0x7ff584005920 2023-10-09T06:32:45.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.762+0000 7ff59ee3c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5980c6d00 con 0x7ff59816ac70 2023-10-09T06:32:45.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.762+0000 7ff59d639700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff59816a800 0x7ff5980c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:45.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.762+0000 7ff59d639700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff59816a800 0x7ff5980c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:34584/0 (socket says 172.21.15.163:34584) 2023-10-09T06:32:45.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.762+0000 7ff59d639700 1 -- 172.21.15.163:0/763341486 learned_addr learned my addr 172.21.15.163:0/763341486 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:32:45.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.762+0000 7ff59de3a700 1 --2- 172.21.15.163:0/763341486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff584005920 0x7ff5981570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:45.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.762+0000 7ff59e63b700 1 --2- 172.21.15.163:0/763341486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff59816ac70 0x7ff5980c66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:45.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.763+0000 7ff59d639700 1 -- 172.21.15.163:0/763341486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff59816ac70 msgr2=0x7ff5980c66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:45.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.763+0000 7ff59d639700 1 --2- 172.21.15.163:0/763341486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff59816ac70 0x7ff5980c66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:45.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.763+0000 7ff59d639700 1 -- 172.21.15.163:0/763341486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff584005920 msgr2=0x7ff5981570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:45.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.763+0000 7ff59d639700 1 --2- 172.21.15.163:0/763341486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff584005920 0x7ff5981570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:45.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.763+0000 7ff59d639700 1 -- 172.21.15.163:0/763341486 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5981533b0 con 0x7ff59816a800 2023-10-09T06:32:45.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.763+0000 7ff59d639700 1 --2- 172.21.15.163:0/763341486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff59816a800 0x7ff5980c6160 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff58800e4b0 tx=0x7ff58801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:45.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.763+0000 7ff58dffb700 1 -- 172.21.15.163:0/763341486 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff588022070 con 0x7ff59816a800 2023-10-09T06:32:45.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.763+0000 7ff58dffb700 1 -- 172.21.15.163:0/763341486 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58800e9e0 con 0x7ff59816a800 2023-10-09T06:32:45.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.763+0000 7ff59ee3c700 1 -- 172.21.15.163:0/763341486 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff598153690 con 0x7ff59816a800 2023-10-09T06:32:45.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.763+0000 7ff58dffb700 1 -- 172.21.15.163:0/763341486 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff588027470 con 0x7ff59816a800 2023-10-09T06:32:45.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.763+0000 7ff59ee3c700 1 -- 172.21.15.163:0/763341486 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5980c4270 con 0x7ff59816a800 2023-10-09T06:32:45.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.764+0000 7ff59ee3c700 1 -- 172.21.15.163:0/763341486 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff59806cfa0 con 0x7ff59816a800 2023-10-09T06:32:45.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.764+0000 7ff58dffb700 1 -- 172.21.15.163:0/763341486 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7ff58801c450 con 0x7ff59816a800 2023-10-09T06:32:45.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.765+0000 7ff58dffb700 1 --2- 172.21.15.163:0/763341486 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff57c067040 0x7ff57c0694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:32:45.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.765+0000 7ff58dffb700 1 -- 172.21.15.163:0/763341486 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff588086170 con 0x7ff59816a800 2023-10-09T06:32:45.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.765+0000 7ff59de3a700 1 --2- 172.21.15.163:0/763341486 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff57c067040 0x7ff57c0694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:32:45.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.765+0000 7ff59de3a700 1 --2- 172.21.15.163:0/763341486 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff57c067040 0x7ff57c0694f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff59806c7f0 tx=0x7ff594001640 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:32:45.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.769+0000 7ff58dffb700 1 -- 172.21.15.163:0/763341486 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff5880533a0 con 0x7ff59816a800 2023-10-09T06:32:45.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.984+0000 7ff59ee3c700 1 -- 172.21.15.163:0/763341486 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000333763", "target": ["mon-mgr", ""]}) v1 -- 0x7ff598158e30 con 0x7ff57c067040 2023-10-09T06:32:45.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.990+0000 7ff58dffb700 1 -- 172.21.15.163:0/763341486 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7ff598158e30 con 0x7ff57c067040 2023-10-09T06:32:45.991 INFO:teuthology.orchestra.run.smithi163.stdout:/volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd 2023-10-09T06:32:45.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.992+0000 7ff59ee3c700 1 -- 172.21.15.163:0/763341486 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff57c067040 msgr2=0x7ff57c0694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:45.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.992+0000 7ff59ee3c700 1 --2- 172.21.15.163:0/763341486 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff57c067040 0x7ff57c0694f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff59806c7f0 tx=0x7ff594001640 comp rx=0 tx=0).stop 2023-10-09T06:32:45.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.992+0000 7ff59ee3c700 1 -- 172.21.15.163:0/763341486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff59816a800 msgr2=0x7ff5980c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:32:45.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.992+0000 7ff59ee3c700 1 --2- 172.21.15.163:0/763341486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff59816a800 0x7ff5980c6160 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff58800e4b0 tx=0x7ff58801a000 comp rx=0 tx=0).stop 2023-10-09T06:32:45.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.992+0000 7ff59ee3c700 1 -- 172.21.15.163:0/763341486 shutdown_connections 2023-10-09T06:32:45.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.992+0000 7ff59ee3c700 1 --2- 172.21.15.163:0/763341486 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff57c067040 0x7ff57c0694f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:45.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.992+0000 7ff59ee3c700 1 --2- 172.21.15.163:0/763341486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff59816ac70 0x7ff5980c66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:45.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.992+0000 7ff59ee3c700 1 --2- 172.21.15.163:0/763341486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff584005920 0x7ff5981570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:45.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.992+0000 7ff59ee3c700 1 --2- 172.21.15.163:0/763341486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff59816a800 0x7ff5980c6160 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:32:45.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.992+0000 7ff59ee3c700 1 -- 172.21.15.163:0/763341486 >> 172.21.15.163:0/763341486 conn(0x7ff5980c9f70 msgr2=0x7ff5980c2330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:32:45.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.992+0000 7ff59ee3c700 1 -- 172.21.15.163:0/763341486 shutdown_connections 2023-10-09T06:32:45.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:32:45.992+0000 7ff59ee3c700 1 -- 172.21.15.163:0/763341486 wait complete. 2023-10-09T06:32:46.005 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000333763 with 3000 files each 0.1MB size under directory volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd 2023-10-09T06:32:46.005 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:46.005 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.0 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:46.028 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:46.028 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:46.028 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.6841e-05 s, 0.0 kB/s 2023-10-09T06:32:46.028 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:46.028 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:46.084 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:46.084 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:46.084 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9848e-05 s, 0.0 kB/s 2023-10-09T06:32:46.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:46.085 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:46.140 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:46.140 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:46.140 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7274e-05 s, 0.0 kB/s 2023-10-09T06:32:46.141 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:46.141 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.3 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:46.196 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:46.196 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:46.197 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1275e-05 s, 0.0 kB/s 2023-10-09T06:32:46.197 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:46.197 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.4 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:46.252 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:46.252 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:46.253 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8095e-05 s, 0.0 kB/s 2023-10-09T06:32:46.253 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:46.253 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.5 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:46.308 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:46.309 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:46.309 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3325e-05 s, 0.0 kB/s 2023-10-09T06:32:46.309 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:46.309 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.6 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:46.364 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:46.364 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:46.364 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1562e-05 s, 0.0 kB/s 2023-10-09T06:32:46.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:46.365 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.7 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:46.420 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:46.420 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:46.420 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1137e-05 s, 0.0 kB/s 2023-10-09T06:32:46.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:46.421 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.8 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:46.476 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:46.476 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:46.476 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8272e-05 s, 0.0 kB/s 2023-10-09T06:32:46.476 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:46.476 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.9 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:46.532 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:46.533 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:46.533 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.95e-05 s, 0.0 kB/s 2023-10-09T06:32:46.533 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:46.533 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.10 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:46.589 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:46.589 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:46.590 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3934e-05 s, 0.0 kB/s 2023-10-09T06:32:46.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:46.590 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.11 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:46.646 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:46.646 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:46.646 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3485e-05 s, 0.0 kB/s 2023-10-09T06:32:46.646 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:46.647 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.12 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:46.702 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:46.702 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:46.702 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7778e-05 s, 0.0 kB/s 2023-10-09T06:32:46.702 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:46.703 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.13 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:46.758 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:46.758 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:46.758 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6957e-05 s, 0.0 kB/s 2023-10-09T06:32:46.758 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:46.758 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.14 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:46.814 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:46.814 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:46.814 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8512e-05 s, 0.0 kB/s 2023-10-09T06:32:46.814 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:46.814 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.15 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:46.870 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:46.870 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:46.870 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8432e-05 s, 0.0 kB/s 2023-10-09T06:32:46.870 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:46.871 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.16 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:46.925 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:46.926 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:46.926 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2579e-05 s, 0.0 kB/s 2023-10-09T06:32:46.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:46.926 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.17 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:46.981 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:46.981 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:46.981 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3699e-05 s, 0.0 kB/s 2023-10-09T06:32:46.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:46.982 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.18 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:47.037 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:47.037 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:47.037 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7923e-05 s, 0.0 kB/s 2023-10-09T06:32:47.037 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:47.037 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.19 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:47.093 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:47.093 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:47.093 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8775e-05 s, 0.0 kB/s 2023-10-09T06:32:47.093 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:47.093 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.20 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:47.148 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:47.148 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:47.148 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2066e-05 s, 0.0 kB/s 2023-10-09T06:32:47.149 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:47.149 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.21 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:47.203 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:47.204 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:47.204 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.977e-05 s, 0.0 kB/s 2023-10-09T06:32:47.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:47.204 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.22 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:47.259 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:47.259 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:47.259 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7871e-05 s, 0.0 kB/s 2023-10-09T06:32:47.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:47.259 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.23 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:47.313 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:47.313 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:47.314 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.5625e-05 s, 0.0 kB/s 2023-10-09T06:32:47.314 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:47.314 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.24 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:47.368 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:47.368 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:47.369 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9262e-05 s, 0.0 kB/s 2023-10-09T06:32:47.369 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:47.369 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.25 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:47.424 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:47.424 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:47.424 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9797e-05 s, 0.0 kB/s 2023-10-09T06:32:47.424 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:47.424 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.26 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:47.478 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:47.478 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:47.478 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9831e-05 s, 0.0 kB/s 2023-10-09T06:32:47.479 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:47.479 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.27 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:47.534 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:47.535 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:47.535 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8873e-05 s, 0.0 kB/s 2023-10-09T06:32:47.535 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:47.535 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.28 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:47.590 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:47.590 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:47.590 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7761e-05 s, 0.0 kB/s 2023-10-09T06:32:47.591 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:47.591 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.29 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:47.646 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:47.646 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:47.647 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9716e-05 s, 0.0 kB/s 2023-10-09T06:32:47.647 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:47.647 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.30 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:47.702 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:47.702 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:47.702 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.132e-05 s, 0.0 kB/s 2023-10-09T06:32:47.702 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:47.702 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.31 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:47.756 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:47.757 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:47.757 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1379e-05 s, 0.0 kB/s 2023-10-09T06:32:47.757 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:47.757 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.32 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:47.811 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:47.812 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:47.812 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9462e-05 s, 0.0 kB/s 2023-10-09T06:32:47.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:47.812 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.33 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:47.867 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:47.867 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:47.867 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.992e-05 s, 0.0 kB/s 2023-10-09T06:32:47.867 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:47.868 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.34 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:47.922 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:47.923 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:47.923 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7816e-05 s, 0.0 kB/s 2023-10-09T06:32:47.923 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:47.923 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.35 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:47.977 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:47.978 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:47.978 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8201e-05 s, 0.0 kB/s 2023-10-09T06:32:47.978 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:47.978 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.36 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:48.032 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:48.033 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:48.033 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9985e-05 s, 0.0 kB/s 2023-10-09T06:32:48.033 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:48.033 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.37 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:48.088 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:48.088 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:48.088 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6674e-05 s, 0.0 kB/s 2023-10-09T06:32:48.088 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:48.088 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.38 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:48.144 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:48.144 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:48.144 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8944e-05 s, 0.0 kB/s 2023-10-09T06:32:48.144 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:48.144 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.39 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:48.199 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:48.199 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:48.199 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9661e-05 s, 0.0 kB/s 2023-10-09T06:32:48.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:48.199 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.40 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:48.254 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:48.254 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:48.254 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0258e-05 s, 0.0 kB/s 2023-10-09T06:32:48.254 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:48.254 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.41 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:48.309 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:48.309 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:48.309 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7773e-05 s, 0.0 kB/s 2023-10-09T06:32:48.310 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:48.310 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.42 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:48.365 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:48.365 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:48.365 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8381e-05 s, 0.0 kB/s 2023-10-09T06:32:48.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:48.365 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.43 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:48.421 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:48.421 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:48.421 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8939e-05 s, 0.0 kB/s 2023-10-09T06:32:48.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:48.421 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.44 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:48.476 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:48.477 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:48.477 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6484e-05 s, 0.0 kB/s 2023-10-09T06:32:48.477 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:48.477 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.45 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:48.532 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:48.532 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:48.532 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0505e-05 s, 0.0 kB/s 2023-10-09T06:32:48.532 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:48.533 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.46 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:48.587 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:48.588 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:48.588 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.057e-05 s, 0.0 kB/s 2023-10-09T06:32:48.588 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:48.588 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.47 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:48.643 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:48.643 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:48.643 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.5536e-05 s, 0.0 kB/s 2023-10-09T06:32:48.643 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:48.644 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.48 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:48.699 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:48.699 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:48.699 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8128e-05 s, 0.0 kB/s 2023-10-09T06:32:48.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:48.699 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.49 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:48.754 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:48.755 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:48.755 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8348e-05 s, 0.0 kB/s 2023-10-09T06:32:48.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:48.755 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.50 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:48.810 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:48.810 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:48.810 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7312e-05 s, 0.0 kB/s 2023-10-09T06:32:48.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:48.810 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.51 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:48.865 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:48.865 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:48.865 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4019e-05 s, 0.0 kB/s 2023-10-09T06:32:48.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:48.865 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.52 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:48.920 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:48.920 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:48.920 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0739e-05 s, 0.0 kB/s 2023-10-09T06:32:48.921 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:48.921 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.53 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:48.976 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:48.976 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:48.976 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.752e-05 s, 0.0 kB/s 2023-10-09T06:32:48.977 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:48.977 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.54 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:49.032 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:49.032 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:49.032 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6705e-05 s, 0.0 kB/s 2023-10-09T06:32:49.032 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:49.032 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.55 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:49.087 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:49.087 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:49.088 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.022e-05 s, 0.0 kB/s 2023-10-09T06:32:49.088 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:49.089 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.56 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:49.144 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:49.144 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:49.144 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9733e-05 s, 0.0 kB/s 2023-10-09T06:32:49.145 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:49.145 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.57 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:49.200 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:49.200 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:49.200 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8114e-05 s, 0.0 kB/s 2023-10-09T06:32:49.201 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:49.201 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.58 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:49.255 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:49.256 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:49.256 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9931e-05 s, 0.0 kB/s 2023-10-09T06:32:49.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:49.257 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.59 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:49.311 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:49.311 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:49.311 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7601e-05 s, 0.0 kB/s 2023-10-09T06:32:49.311 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:49.311 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.60 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:49.367 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:49.367 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:49.367 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0526e-05 s, 0.0 kB/s 2023-10-09T06:32:49.367 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:49.367 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.61 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:49.422 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:49.422 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:49.422 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9546e-05 s, 0.0 kB/s 2023-10-09T06:32:49.422 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:49.422 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.62 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:49.476 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:49.477 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:49.477 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8295e-05 s, 0.0 kB/s 2023-10-09T06:32:49.477 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:49.477 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.63 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:49.531 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:49.532 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:49.532 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.7564e-05 s, 0.0 kB/s 2023-10-09T06:32:49.532 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:49.532 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.64 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:49.586 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:49.587 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:49.587 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9555e-05 s, 0.0 kB/s 2023-10-09T06:32:49.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:49.587 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.65 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:49.642 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:49.642 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:49.642 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9178e-05 s, 0.0 kB/s 2023-10-09T06:32:49.642 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:49.642 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.66 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:49.698 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:49.698 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:49.698 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7298e-05 s, 0.0 kB/s 2023-10-09T06:32:49.698 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:49.698 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.67 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:49.753 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:49.754 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:49.754 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7379e-05 s, 0.0 kB/s 2023-10-09T06:32:49.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:49.755 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.68 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:49.811 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:49.811 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:49.811 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9792e-05 s, 0.0 kB/s 2023-10-09T06:32:49.811 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:49.812 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.69 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:49.866 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:49.867 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:49.867 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7707e-05 s, 0.0 kB/s 2023-10-09T06:32:49.867 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:49.867 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.70 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:49.921 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:49.921 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:49.922 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7333e-05 s, 0.0 kB/s 2023-10-09T06:32:49.922 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:49.922 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.71 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:49.976 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:49.976 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:49.977 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8791e-05 s, 0.0 kB/s 2023-10-09T06:32:49.977 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:49.977 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.72 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:50.031 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:50.032 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:50.032 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7778e-05 s, 0.0 kB/s 2023-10-09T06:32:50.032 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:50.032 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.73 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:50.087 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:50.087 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:50.087 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8326e-05 s, 0.0 kB/s 2023-10-09T06:32:50.087 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:50.087 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.74 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:50.141 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:50.142 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:50.142 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.029e-05 s, 0.0 kB/s 2023-10-09T06:32:50.142 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:50.142 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.75 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:50.196 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:50.196 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:50.197 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8926e-05 s, 0.0 kB/s 2023-10-09T06:32:50.197 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:50.197 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.76 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:50.252 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:50.252 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:50.252 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.83e-05 s, 0.0 kB/s 2023-10-09T06:32:50.252 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:50.252 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.77 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:50.307 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:50.307 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:50.308 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9589e-05 s, 0.0 kB/s 2023-10-09T06:32:50.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:50.308 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.78 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:50.363 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:50.363 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:50.363 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1024e-05 s, 0.0 kB/s 2023-10-09T06:32:50.363 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:50.363 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.79 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:50.418 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:50.418 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:50.418 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6295e-05 s, 0.0 kB/s 2023-10-09T06:32:50.418 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:50.418 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.80 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:50.473 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:50.473 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:50.473 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9283e-05 s, 0.0 kB/s 2023-10-09T06:32:50.473 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:50.473 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.81 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:50.528 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:50.528 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:50.528 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8259e-05 s, 0.0 kB/s 2023-10-09T06:32:50.528 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:50.528 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.82 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:50.583 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:50.583 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:50.583 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8436e-05 s, 0.0 kB/s 2023-10-09T06:32:50.583 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:50.584 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.83 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:50.639 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:50.639 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:50.639 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0355e-05 s, 0.0 kB/s 2023-10-09T06:32:50.639 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:50.639 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.84 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:50.695 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:50.695 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:50.695 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8466e-05 s, 0.0 kB/s 2023-10-09T06:32:50.695 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:50.695 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.85 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:50.750 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:50.750 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:50.750 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6406e-05 s, 0.0 kB/s 2023-10-09T06:32:50.751 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:50.751 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.86 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:50.806 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:50.806 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:50.806 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9307e-05 s, 0.0 kB/s 2023-10-09T06:32:50.807 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:50.807 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.87 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:50.861 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:50.861 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:50.861 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9578e-05 s, 0.0 kB/s 2023-10-09T06:32:50.862 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:50.862 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.88 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:50.918 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:50.918 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:50.918 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9167e-05 s, 0.0 kB/s 2023-10-09T06:32:50.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:50.918 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.89 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:50.974 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:50.974 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:50.974 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8763e-05 s, 0.0 kB/s 2023-10-09T06:32:50.974 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:50.974 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.90 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:51.028 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:51.028 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:51.028 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.718e-05 s, 0.0 kB/s 2023-10-09T06:32:51.029 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:51.029 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.91 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:51.085 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:51.085 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:51.085 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1601e-05 s, 0.0 kB/s 2023-10-09T06:32:51.086 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:51.086 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.92 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:51.141 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:51.143 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:51.143 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8734e-05 s, 0.0 kB/s 2023-10-09T06:32:51.143 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:51.143 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.93 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:51.197 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:51.197 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:51.197 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0826e-05 s, 0.0 kB/s 2023-10-09T06:32:51.197 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:51.198 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.94 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:51.252 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:51.252 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:51.252 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0178e-05 s, 0.0 kB/s 2023-10-09T06:32:51.253 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:51.253 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.95 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:51.309 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:51.309 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:51.309 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8186e-05 s, 0.0 kB/s 2023-10-09T06:32:51.309 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:51.309 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.96 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:51.367 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:51.912 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:51.912 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.905e-05 s, 0.0 kB/s 2023-10-09T06:32:51.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:51.913 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.97 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:51.929 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:51.930 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:51.930 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1176e-05 s, 0.0 kB/s 2023-10-09T06:32:51.930 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:51.930 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.98 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:51.987 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:51.987 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:51.987 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.029e-05 s, 0.0 kB/s 2023-10-09T06:32:51.987 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:51.987 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.99 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:52.043 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:52.043 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:52.043 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3108e-05 s, 0.0 kB/s 2023-10-09T06:32:52.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:52.044 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.100 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:52.098 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:52.098 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:52.098 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2406e-05 s, 0.0 kB/s 2023-10-09T06:32:52.099 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:52.099 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.101 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:52.153 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:52.153 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:52.154 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1711e-05 s, 0.0 kB/s 2023-10-09T06:32:52.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:52.154 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.102 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:52.209 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:52.209 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:52.209 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7804e-05 s, 0.0 kB/s 2023-10-09T06:32:52.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:52.209 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.103 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:52.265 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:52.265 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:52.265 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7662e-05 s, 0.0 kB/s 2023-10-09T06:32:52.265 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:52.265 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.104 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:52.320 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:52.321 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:52.321 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8828e-05 s, 0.0 kB/s 2023-10-09T06:32:52.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:52.321 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.105 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:52.375 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:52.375 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:52.375 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1548e-05 s, 0.0 kB/s 2023-10-09T06:32:52.376 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:52.376 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.106 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:52.430 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:52.431 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:52.431 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0961e-05 s, 0.0 kB/s 2023-10-09T06:32:52.431 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:52.431 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.107 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:52.486 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:52.486 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:52.486 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.6644e-05 s, 0.0 kB/s 2023-10-09T06:32:52.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:52.487 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.108 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:52.542 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:52.542 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:52.542 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2583e-05 s, 0.0 kB/s 2023-10-09T06:32:52.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:52.542 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.109 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:52.598 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:52.598 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:52.598 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9721e-05 s, 0.0 kB/s 2023-10-09T06:32:52.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:52.599 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.110 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:52.654 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:52.655 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:52.655 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0224e-05 s, 0.0 kB/s 2023-10-09T06:32:52.655 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:52.655 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.111 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:52.710 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:52.710 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:52.710 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8874e-05 s, 0.0 kB/s 2023-10-09T06:32:52.710 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:52.710 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.112 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:52.764 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:52.764 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:52.764 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9974e-05 s, 0.0 kB/s 2023-10-09T06:32:52.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:52.765 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.113 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:52.820 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:52.820 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:52.820 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.059e-05 s, 0.0 kB/s 2023-10-09T06:32:52.820 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:52.820 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.114 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:52.876 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:52.876 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:52.876 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0116e-05 s, 0.0 kB/s 2023-10-09T06:32:52.876 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:52.876 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.115 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:52.931 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:52.932 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:52.932 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9355e-05 s, 0.0 kB/s 2023-10-09T06:32:52.932 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:52.932 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.116 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:52.987 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:52.987 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:52.987 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9436e-05 s, 0.0 kB/s 2023-10-09T06:32:52.987 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:52.987 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.117 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:53.042 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:53.042 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:53.043 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9022e-05 s, 0.0 kB/s 2023-10-09T06:32:53.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:53.043 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.118 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:53.097 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:53.097 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:53.097 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0362e-05 s, 0.0 kB/s 2023-10-09T06:32:53.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:53.098 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.119 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:53.152 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:53.153 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:53.153 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9224e-05 s, 0.0 kB/s 2023-10-09T06:32:53.153 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:53.153 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.120 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:53.207 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:53.208 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:53.208 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.998e-05 s, 0.0 kB/s 2023-10-09T06:32:53.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:53.209 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.121 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:53.264 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:53.264 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:53.264 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.901e-05 s, 0.0 kB/s 2023-10-09T06:32:53.264 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:53.264 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.122 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:53.320 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:53.320 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:53.320 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7245e-05 s, 0.0 kB/s 2023-10-09T06:32:53.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:53.321 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.123 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:53.376 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:53.376 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:53.376 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.747e-05 s, 0.0 kB/s 2023-10-09T06:32:53.376 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:53.376 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.124 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:53.431 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:53.431 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:53.431 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9237e-05 s, 0.0 kB/s 2023-10-09T06:32:53.431 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:53.431 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.125 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:53.486 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:53.486 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:53.486 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8241e-05 s, 0.0 kB/s 2023-10-09T06:32:53.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:53.487 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.126 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:53.543 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:53.543 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:53.543 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8908e-05 s, 0.0 kB/s 2023-10-09T06:32:53.543 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:53.543 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.127 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:53.598 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:53.598 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:53.598 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3679e-05 s, 0.0 kB/s 2023-10-09T06:32:53.599 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:53.599 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.128 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:53.654 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:53.654 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:53.655 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.042e-05 s, 0.0 kB/s 2023-10-09T06:32:53.655 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:53.655 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.129 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:53.709 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:53.709 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:53.709 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9035e-05 s, 0.0 kB/s 2023-10-09T06:32:53.710 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:53.710 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.130 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:53.766 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:53.766 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:53.766 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0272e-05 s, 0.0 kB/s 2023-10-09T06:32:53.766 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:53.767 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.131 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:53.821 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:53.821 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:53.822 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9016e-05 s, 0.0 kB/s 2023-10-09T06:32:53.822 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:53.822 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.132 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:53.877 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:53.877 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:53.877 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8511e-05 s, 0.0 kB/s 2023-10-09T06:32:53.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:53.878 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.133 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:53.933 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:53.933 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:53.933 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0215e-05 s, 0.0 kB/s 2023-10-09T06:32:53.933 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:53.933 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.134 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:53.988 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:53.988 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:53.988 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.772e-05 s, 0.0 kB/s 2023-10-09T06:32:53.988 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:53.988 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.135 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:54.043 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:54.043 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:54.043 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0679e-05 s, 0.0 kB/s 2023-10-09T06:32:54.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:54.043 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.136 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:54.098 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:54.098 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:54.098 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8858e-05 s, 0.0 kB/s 2023-10-09T06:32:54.099 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:54.099 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.137 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:54.154 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:54.154 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:54.154 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6973e-05 s, 0.0 kB/s 2023-10-09T06:32:54.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:54.154 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.138 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:54.209 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:54.210 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:54.210 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8675e-05 s, 0.0 kB/s 2023-10-09T06:32:54.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:54.210 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.139 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:54.264 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:54.265 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:54.265 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8742e-05 s, 0.0 kB/s 2023-10-09T06:32:54.265 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:54.265 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.140 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:54.321 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:54.321 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:54.321 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8066e-05 s, 0.0 kB/s 2023-10-09T06:32:54.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:54.321 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.141 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:54.376 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:54.376 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:54.376 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7761e-05 s, 0.0 kB/s 2023-10-09T06:32:54.376 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:54.377 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.142 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:54.432 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:54.432 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:54.432 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.5238e-05 s, 0.0 kB/s 2023-10-09T06:32:54.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:54.433 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.143 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:54.488 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:54.488 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:54.488 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6927e-05 s, 0.0 kB/s 2023-10-09T06:32:54.488 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:54.488 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.144 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:54.544 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:54.544 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:54.544 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2406e-05 s, 0.0 kB/s 2023-10-09T06:32:54.545 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:54.545 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.145 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:54.600 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:54.600 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:54.600 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8241e-05 s, 0.0 kB/s 2023-10-09T06:32:54.601 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:54.601 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.146 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:54.656 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:54.656 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:54.656 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.995e-05 s, 0.0 kB/s 2023-10-09T06:32:54.656 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:54.656 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.147 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:54.711 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:54.712 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:54.712 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8521e-05 s, 0.0 kB/s 2023-10-09T06:32:54.712 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:54.712 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.148 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:54.766 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:54.767 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:54.767 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9202e-05 s, 0.0 kB/s 2023-10-09T06:32:54.767 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:54.767 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.149 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:54.822 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:54.822 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:54.822 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8149e-05 s, 0.0 kB/s 2023-10-09T06:32:54.822 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:54.822 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.150 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:54.878 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:54.878 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:54.878 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9781e-05 s, 0.0 kB/s 2023-10-09T06:32:54.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:54.878 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.151 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:54.933 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:54.933 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:54.934 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6657e-05 s, 0.0 kB/s 2023-10-09T06:32:54.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:54.934 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.152 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:54.989 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:54.990 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:54.990 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8382e-05 s, 0.0 kB/s 2023-10-09T06:32:54.990 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:54.990 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.153 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:55.045 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:55.045 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:55.045 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0118e-05 s, 0.0 kB/s 2023-10-09T06:32:55.045 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:55.045 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.154 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:55.101 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:55.101 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:55.101 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9524e-05 s, 0.0 kB/s 2023-10-09T06:32:55.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:55.102 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.155 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:55.157 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:55.157 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:55.157 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7095e-05 s, 0.0 kB/s 2023-10-09T06:32:55.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:55.158 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.156 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:55.213 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:55.213 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:55.213 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7864e-05 s, 0.0 kB/s 2023-10-09T06:32:55.213 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:55.214 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.157 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:55.269 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:55.269 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:55.269 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9126e-05 s, 0.0 kB/s 2023-10-09T06:32:55.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:55.269 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.158 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:55.324 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:55.324 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:55.324 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7505e-05 s, 0.0 kB/s 2023-10-09T06:32:55.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:55.324 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.159 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:55.379 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:55.379 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:55.379 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6345e-05 s, 0.0 kB/s 2023-10-09T06:32:55.379 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:55.379 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.160 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:55.435 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:55.436 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:55.436 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0386e-05 s, 0.0 kB/s 2023-10-09T06:32:55.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:55.436 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.161 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:55.490 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:55.490 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:55.490 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9339e-05 s, 0.0 kB/s 2023-10-09T06:32:55.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:55.491 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.162 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:55.546 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:55.546 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:55.546 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8965e-05 s, 0.0 kB/s 2023-10-09T06:32:55.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:55.547 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.163 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:55.602 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:55.602 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:55.602 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8694e-05 s, 0.0 kB/s 2023-10-09T06:32:55.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:55.603 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.164 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:55.658 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:55.658 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:55.658 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9617e-05 s, 0.0 kB/s 2023-10-09T06:32:55.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:55.659 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.165 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:55.714 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:55.714 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:55.714 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6868e-05 s, 0.0 kB/s 2023-10-09T06:32:55.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:55.714 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.166 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:55.769 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:55.769 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:55.769 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1413e-05 s, 0.0 kB/s 2023-10-09T06:32:55.770 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:55.770 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.167 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:55.825 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:55.825 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:55.825 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8615e-05 s, 0.0 kB/s 2023-10-09T06:32:55.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:55.826 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.168 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:55.880 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:55.881 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:55.881 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9869e-05 s, 0.0 kB/s 2023-10-09T06:32:55.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:55.881 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.169 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:55.936 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:55.936 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:55.936 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.4926e-05 s, 0.0 kB/s 2023-10-09T06:32:55.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:55.936 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.170 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:55.992 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:55.992 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:55.992 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9932e-05 s, 0.0 kB/s 2023-10-09T06:32:55.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:55.993 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.171 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:56.048 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:56.049 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:56.049 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0332e-05 s, 0.0 kB/s 2023-10-09T06:32:56.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:56.049 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.172 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:56.105 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:56.105 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:56.105 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9933e-05 s, 0.0 kB/s 2023-10-09T06:32:56.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:56.106 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.173 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:56.161 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:56.162 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:56.162 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7979e-05 s, 0.0 kB/s 2023-10-09T06:32:56.162 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:56.162 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.174 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:56.217 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:56.217 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:56.217 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1157e-05 s, 0.0 kB/s 2023-10-09T06:32:56.218 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:56.218 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.175 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:56.273 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:56.273 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:56.273 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6833e-05 s, 0.0 kB/s 2023-10-09T06:32:56.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:56.273 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.176 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:56.329 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:56.329 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:56.329 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0152e-05 s, 0.0 kB/s 2023-10-09T06:32:56.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:56.330 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.177 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:56.385 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:56.385 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:56.385 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1787e-05 s, 0.0 kB/s 2023-10-09T06:32:56.385 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:56.386 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.178 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:56.441 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:56.441 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:56.441 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.966e-05 s, 0.0 kB/s 2023-10-09T06:32:56.442 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:56.442 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.179 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:56.496 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:56.496 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:56.497 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0711e-05 s, 0.0 kB/s 2023-10-09T06:32:56.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:56.497 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.180 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:56.552 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:56.552 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:56.552 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7833e-05 s, 0.0 kB/s 2023-10-09T06:32:56.552 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:56.553 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.181 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:56.608 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:56.608 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:56.608 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.89e-05 s, 0.0 kB/s 2023-10-09T06:32:56.608 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:56.609 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.182 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:56.664 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:56.664 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:56.664 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7486e-05 s, 0.0 kB/s 2023-10-09T06:32:56.664 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:56.665 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.183 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:56.721 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:56.721 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:56.721 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4748e-05 s, 0.0 kB/s 2023-10-09T06:32:56.722 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:56.722 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.184 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:56.782 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:56.782 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:56.782 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1091e-05 s, 0.0 kB/s 2023-10-09T06:32:56.782 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:56.783 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.185 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:56.845 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:56.845 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:56.845 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9069e-05 s, 0.0 kB/s 2023-10-09T06:32:56.845 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:56.846 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.186 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:56.905 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:56.905 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:56.905 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8713e-05 s, 0.0 kB/s 2023-10-09T06:32:56.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:56.905 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.187 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:56.962 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:56.962 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:56.962 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8283e-05 s, 0.0 kB/s 2023-10-09T06:32:56.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:56.963 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.188 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:57.026 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:57.026 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:57.027 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0039e-05 s, 0.0 kB/s 2023-10-09T06:32:57.027 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:57.027 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.189 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:57.091 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:57.091 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:57.092 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7995e-05 s, 0.0 kB/s 2023-10-09T06:32:57.092 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:57.092 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.190 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:57.148 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:57.148 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:57.149 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9573e-05 s, 0.0 kB/s 2023-10-09T06:32:57.149 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:57.149 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.191 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:57.209 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:57.209 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:57.209 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.898e-05 s, 0.0 kB/s 2023-10-09T06:32:57.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:57.210 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.192 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:57.267 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:57.267 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:57.268 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8888e-05 s, 0.0 kB/s 2023-10-09T06:32:57.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:57.268 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.193 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:57.326 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:57.326 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:57.326 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9731e-05 s, 0.0 kB/s 2023-10-09T06:32:57.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:57.327 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.194 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:57.383 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:57.383 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:57.383 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.869e-05 s, 0.0 kB/s 2023-10-09T06:32:57.384 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:57.384 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.195 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:57.444 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:57.444 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:57.444 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0023e-05 s, 0.0 kB/s 2023-10-09T06:32:57.445 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:57.445 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.196 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:57.502 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:57.502 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:57.502 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.5658e-05 s, 0.0 kB/s 2023-10-09T06:32:57.502 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:57.502 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.197 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:57.561 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:57.561 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:57.561 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8136e-05 s, 0.0 kB/s 2023-10-09T06:32:57.561 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:57.562 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.198 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:57.617 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:57.617 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:57.617 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.064e-05 s, 0.0 kB/s 2023-10-09T06:32:57.617 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:57.617 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.199 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:57.673 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:57.674 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:57.674 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9886e-05 s, 0.0 kB/s 2023-10-09T06:32:57.674 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:57.674 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.200 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:57.729 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:57.729 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:57.729 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.042e-05 s, 0.0 kB/s 2023-10-09T06:32:57.729 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:57.729 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.201 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:57.785 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:57.785 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:57.785 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9828e-05 s, 0.0 kB/s 2023-10-09T06:32:57.786 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:57.786 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.202 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:57.841 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:57.841 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:57.841 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.844e-05 s, 0.0 kB/s 2023-10-09T06:32:57.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:57.842 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.203 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:57.897 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:57.897 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:57.898 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1038e-05 s, 0.0 kB/s 2023-10-09T06:32:57.898 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:57.898 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.204 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:57.952 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:57.952 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:57.952 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6213e-05 s, 0.0 kB/s 2023-10-09T06:32:57.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:57.953 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.205 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:58.009 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:58.009 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:58.009 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0018e-05 s, 0.0 kB/s 2023-10-09T06:32:58.009 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:58.009 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.206 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:58.065 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:58.065 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:58.065 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9938e-05 s, 0.0 kB/s 2023-10-09T06:32:58.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:58.065 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.207 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:58.119 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:58.119 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:58.120 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2179e-05 s, 0.0 kB/s 2023-10-09T06:32:58.120 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:58.120 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.208 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:58.175 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:58.175 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:58.175 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8197e-05 s, 0.0 kB/s 2023-10-09T06:32:58.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:58.175 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.209 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:58.231 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:58.231 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:58.231 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.012e-05 s, 0.0 kB/s 2023-10-09T06:32:58.231 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:58.231 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.210 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:58.285 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:58.286 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:58.286 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8867e-05 s, 0.0 kB/s 2023-10-09T06:32:58.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:58.286 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.211 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:58.341 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:58.341 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:58.341 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9255e-05 s, 0.0 kB/s 2023-10-09T06:32:58.341 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:58.342 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.212 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:58.397 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:58.397 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:58.397 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0352e-05 s, 0.0 kB/s 2023-10-09T06:32:58.397 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:58.397 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.213 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:58.453 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:58.453 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:58.453 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2045e-05 s, 0.0 kB/s 2023-10-09T06:32:58.453 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:58.453 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.214 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:58.508 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:58.508 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:58.508 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.976e-05 s, 0.0 kB/s 2023-10-09T06:32:58.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:58.508 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.215 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:58.564 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:58.564 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:58.564 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0108e-05 s, 0.0 kB/s 2023-10-09T06:32:58.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:58.565 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.216 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:58.619 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:58.620 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:58.620 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.5363e-05 s, 0.0 kB/s 2023-10-09T06:32:58.621 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:58.621 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.217 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:58.676 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:58.676 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:58.677 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7188e-05 s, 0.0 kB/s 2023-10-09T06:32:58.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:58.677 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.218 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:58.731 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:58.732 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:58.732 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2786e-05 s, 0.0 kB/s 2023-10-09T06:32:58.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:58.732 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.219 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:58.787 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:58.787 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:58.787 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9834e-05 s, 0.0 kB/s 2023-10-09T06:32:58.788 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:58.788 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.220 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:58.843 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:58.843 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:58.843 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9562e-05 s, 0.0 kB/s 2023-10-09T06:32:58.843 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:58.843 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.221 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:58.899 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:58.899 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:58.899 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.737e-05 s, 0.0 kB/s 2023-10-09T06:32:58.899 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:58.900 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.222 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:58.954 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:58.955 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:58.955 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0196e-05 s, 0.0 kB/s 2023-10-09T06:32:58.955 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:58.955 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.223 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:59.010 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:59.010 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:59.010 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0371e-05 s, 0.0 kB/s 2023-10-09T06:32:59.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:59.011 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.224 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:59.065 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:59.066 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:59.066 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9304e-05 s, 0.0 kB/s 2023-10-09T06:32:59.066 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:59.066 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.225 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:59.121 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:59.121 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:59.121 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7938e-05 s, 0.0 kB/s 2023-10-09T06:32:59.121 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:59.121 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.226 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:59.177 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:59.177 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:59.177 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9816e-05 s, 0.0 kB/s 2023-10-09T06:32:59.177 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:59.178 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.227 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:59.233 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:59.233 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:59.233 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8469e-05 s, 0.0 kB/s 2023-10-09T06:32:59.234 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:59.234 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.228 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:59.289 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:59.289 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:59.289 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9952e-05 s, 0.0 kB/s 2023-10-09T06:32:59.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:59.289 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.229 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:59.343 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:59.343 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:59.344 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6389e-05 s, 0.0 kB/s 2023-10-09T06:32:59.344 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:59.344 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.230 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:59.399 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:59.399 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:59.399 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7559e-05 s, 0.0 kB/s 2023-10-09T06:32:59.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:59.400 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.231 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:59.455 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:59.455 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:59.455 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9663e-05 s, 0.0 kB/s 2023-10-09T06:32:59.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:59.455 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.232 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:59.510 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:59.510 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:59.510 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7211e-05 s, 0.0 kB/s 2023-10-09T06:32:59.510 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:59.510 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.233 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:59.565 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:59.565 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:59.565 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2698e-05 s, 0.0 kB/s 2023-10-09T06:32:59.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:59.565 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.234 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:59.621 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:59.621 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:59.621 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2325e-05 s, 0.0 kB/s 2023-10-09T06:32:59.621 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:59.622 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.235 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:59.675 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:59.675 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:59.675 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8528e-05 s, 0.0 kB/s 2023-10-09T06:32:59.676 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:59.676 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.236 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:59.731 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:59.731 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:59.732 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9369e-05 s, 0.0 kB/s 2023-10-09T06:32:59.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:59.732 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.237 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:59.786 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:59.787 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:59.787 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9689e-05 s, 0.0 kB/s 2023-10-09T06:32:59.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:59.787 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.238 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:59.842 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:59.842 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:59.842 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3645e-05 s, 0.0 kB/s 2023-10-09T06:32:59.842 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:59.842 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.239 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:59.896 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:59.897 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:59.897 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9383e-05 s, 0.0 kB/s 2023-10-09T06:32:59.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:59.897 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.240 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:32:59.952 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:32:59.952 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:32:59.952 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1921e-05 s, 0.0 kB/s 2023-10-09T06:32:59.952 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:32:59.952 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.241 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:00.007 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:00.007 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:00.008 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9769e-05 s, 0.0 kB/s 2023-10-09T06:33:00.008 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:00.008 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.242 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:00.063 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:00.063 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:00.064 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8679e-05 s, 0.0 kB/s 2023-10-09T06:33:00.064 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:00.064 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.243 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:00.119 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:00.119 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:00.119 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0555e-05 s, 0.0 kB/s 2023-10-09T06:33:00.119 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:00.119 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.244 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:00.175 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:00.175 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:00.176 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9218e-05 s, 0.0 kB/s 2023-10-09T06:33:00.176 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:00.176 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.245 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:00.230 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:00.230 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:00.230 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7086e-05 s, 0.0 kB/s 2023-10-09T06:33:00.231 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:00.231 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.246 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:00.286 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:00.287 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:00.287 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7622e-05 s, 0.0 kB/s 2023-10-09T06:33:00.287 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:00.287 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.247 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:00.341 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:00.342 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:00.342 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8511e-05 s, 0.0 kB/s 2023-10-09T06:33:00.342 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:00.342 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.248 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:00.396 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:00.397 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:00.397 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9198e-05 s, 0.0 kB/s 2023-10-09T06:33:00.397 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:00.397 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.249 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:00.452 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:00.452 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:00.452 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1102e-05 s, 0.0 kB/s 2023-10-09T06:33:00.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:00.452 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.250 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:00.507 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:00.508 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:00.508 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0137e-05 s, 0.0 kB/s 2023-10-09T06:33:00.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:00.508 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.251 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:00.563 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:00.563 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:00.563 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8773e-05 s, 0.0 kB/s 2023-10-09T06:33:00.563 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:00.563 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.252 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:00.619 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:00.619 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:00.619 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8718e-05 s, 0.0 kB/s 2023-10-09T06:33:00.619 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:00.620 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.253 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:00.674 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:00.675 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:00.675 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8875e-05 s, 0.0 kB/s 2023-10-09T06:33:00.675 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:00.675 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.254 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:00.730 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:00.730 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:00.730 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9906e-05 s, 0.0 kB/s 2023-10-09T06:33:00.730 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:00.730 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.255 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:00.786 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:00.787 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:00.787 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8361e-05 s, 0.0 kB/s 2023-10-09T06:33:00.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:00.787 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.256 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:00.842 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:00.842 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:00.842 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2949e-05 s, 0.0 kB/s 2023-10-09T06:33:00.843 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:00.843 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.257 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:00.897 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:00.898 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:00.898 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7925e-05 s, 0.0 kB/s 2023-10-09T06:33:00.898 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:00.898 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.258 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:00.953 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:00.953 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:00.953 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7325e-05 s, 0.0 kB/s 2023-10-09T06:33:00.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:00.954 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.259 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:01.009 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:01.009 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:01.009 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9526e-05 s, 0.0 kB/s 2023-10-09T06:33:01.009 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:01.010 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.260 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:01.066 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:01.067 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:01.067 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8842e-05 s, 0.0 kB/s 2023-10-09T06:33:01.067 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:01.067 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.261 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:01.122 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:01.122 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:01.123 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0233e-05 s, 0.0 kB/s 2023-10-09T06:33:01.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:01.123 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.262 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:01.177 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:01.178 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:01.178 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0272e-05 s, 0.0 kB/s 2023-10-09T06:33:01.178 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:01.178 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.263 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:01.233 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:01.233 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:01.233 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8802e-05 s, 0.0 kB/s 2023-10-09T06:33:01.233 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:01.233 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.264 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:01.288 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:01.289 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:01.289 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0134e-05 s, 0.0 kB/s 2023-10-09T06:33:01.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:01.289 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.265 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:01.344 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:01.344 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:01.344 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8659e-05 s, 0.0 kB/s 2023-10-09T06:33:01.344 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:01.344 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.266 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:01.399 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:01.400 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:01.400 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.815e-05 s, 0.0 kB/s 2023-10-09T06:33:01.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:01.400 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.267 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:01.454 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:01.455 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:01.455 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9406e-05 s, 0.0 kB/s 2023-10-09T06:33:01.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:01.455 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.268 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:01.510 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:01.510 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:01.510 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.7893e-05 s, 0.0 kB/s 2023-10-09T06:33:01.510 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:01.510 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.269 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:01.565 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:01.565 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:01.565 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.221e-05 s, 0.0 kB/s 2023-10-09T06:33:01.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:01.565 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.270 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:01.620 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:01.620 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:01.620 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.097e-05 s, 0.0 kB/s 2023-10-09T06:33:01.620 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:01.621 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.271 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:01.676 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:01.676 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:01.676 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0388e-05 s, 0.0 kB/s 2023-10-09T06:33:01.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:01.677 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.272 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:01.732 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:01.732 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:01.732 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1158e-05 s, 0.0 kB/s 2023-10-09T06:33:01.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:01.732 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.273 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:01.788 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:01.788 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:01.788 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.6588e-05 s, 0.0 kB/s 2023-10-09T06:33:01.788 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:01.788 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.274 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:01.843 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:01.844 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:01.844 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2532e-05 s, 0.0 kB/s 2023-10-09T06:33:01.844 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:01.844 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.275 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:01.898 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:01.899 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:01.899 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0828e-05 s, 0.0 kB/s 2023-10-09T06:33:01.899 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:01.899 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.276 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:01.953 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:01.954 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:01.954 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8534e-05 s, 0.0 kB/s 2023-10-09T06:33:01.954 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:01.954 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.277 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:02.008 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:02.008 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:02.008 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1594e-05 s, 0.0 kB/s 2023-10-09T06:33:02.009 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:02.009 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.278 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:02.065 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:02.065 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:02.065 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0238e-05 s, 0.0 kB/s 2023-10-09T06:33:02.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:02.065 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.279 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:02.126 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:02.126 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:02.127 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.83e-05 s, 0.0 kB/s 2023-10-09T06:33:02.127 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:02.127 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.280 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:02.191 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:02.191 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:02.191 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.919e-05 s, 0.0 kB/s 2023-10-09T06:33:02.192 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:02.192 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.281 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:02.256 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:02.256 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:02.256 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7878e-05 s, 0.0 kB/s 2023-10-09T06:33:02.257 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:02.258 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.282 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:02.314 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:02.314 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:02.315 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4546e-05 s, 0.0 kB/s 2023-10-09T06:33:02.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:02.315 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.283 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:02.375 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:02.376 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:02.376 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9363e-05 s, 0.0 kB/s 2023-10-09T06:33:02.376 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:02.377 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.284 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:02.434 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:02.435 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:02.435 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7854e-05 s, 0.0 kB/s 2023-10-09T06:33:02.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:02.436 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.285 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:02.493 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:02.494 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:02.494 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.806e-05 s, 0.0 kB/s 2023-10-09T06:33:02.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:02.494 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.286 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:02.551 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:02.551 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:02.552 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0969e-05 s, 0.0 kB/s 2023-10-09T06:33:02.552 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:02.552 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.287 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:02.608 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:02.609 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:02.609 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7675e-05 s, 0.0 kB/s 2023-10-09T06:33:02.609 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:02.609 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.288 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:02.665 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:02.666 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:02.666 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0709e-05 s, 0.0 kB/s 2023-10-09T06:33:02.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:02.666 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.289 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:02.723 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:02.723 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:02.723 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9837e-05 s, 0.0 kB/s 2023-10-09T06:33:02.723 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:02.724 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.290 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:02.782 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:02.782 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:02.783 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0736e-05 s, 0.0 kB/s 2023-10-09T06:33:02.783 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:02.783 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.291 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:02.839 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:02.839 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:02.839 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8874e-05 s, 0.0 kB/s 2023-10-09T06:33:02.840 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:02.840 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.292 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:02.895 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:02.895 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:02.895 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8038e-05 s, 0.0 kB/s 2023-10-09T06:33:02.895 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:02.895 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.293 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:02.951 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:02.951 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:02.951 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1536e-05 s, 0.0 kB/s 2023-10-09T06:33:02.952 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:02.952 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.294 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:03.007 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:03.007 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:03.007 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.759e-05 s, 0.0 kB/s 2023-10-09T06:33:03.007 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:03.007 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.295 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:03.063 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:03.063 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:03.063 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.6498e-05 s, 0.0 kB/s 2023-10-09T06:33:03.063 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:03.064 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.296 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:03.118 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:03.119 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:03.119 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9356e-05 s, 0.0 kB/s 2023-10-09T06:33:03.119 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:03.119 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.297 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:03.173 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:03.173 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:03.174 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7568e-05 s, 0.0 kB/s 2023-10-09T06:33:03.174 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:03.174 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.298 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:03.229 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:03.229 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:03.229 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9381e-05 s, 0.0 kB/s 2023-10-09T06:33:03.229 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:03.230 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.299 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:03.285 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:03.285 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:03.285 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.309e-05 s, 0.0 kB/s 2023-10-09T06:33:03.285 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:03.285 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.300 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:03.340 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:03.340 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:03.340 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6951e-05 s, 0.0 kB/s 2023-10-09T06:33:03.340 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:03.341 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.301 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:03.396 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:03.396 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:03.396 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8398e-05 s, 0.0 kB/s 2023-10-09T06:33:03.396 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:03.396 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.302 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:03.452 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:03.452 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:03.452 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7976e-05 s, 0.0 kB/s 2023-10-09T06:33:03.453 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:03.453 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.303 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:03.508 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:03.508 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:03.508 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1388e-05 s, 0.0 kB/s 2023-10-09T06:33:03.509 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:03.509 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.304 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:03.565 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:03.565 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:03.565 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0252e-05 s, 0.0 kB/s 2023-10-09T06:33:03.566 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:03.566 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.305 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:03.621 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:03.621 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:03.621 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9105e-05 s, 0.0 kB/s 2023-10-09T06:33:03.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:03.622 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.306 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:03.677 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:03.677 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:03.678 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1134e-05 s, 0.0 kB/s 2023-10-09T06:33:03.678 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:03.678 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.307 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:03.733 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:03.733 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:03.733 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0301e-05 s, 0.0 kB/s 2023-10-09T06:33:03.733 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:03.733 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.308 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:03.788 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:03.788 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:03.788 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8495e-05 s, 0.0 kB/s 2023-10-09T06:33:03.789 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:03.789 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.309 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:03.844 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:03.845 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:03.845 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9389e-05 s, 0.0 kB/s 2023-10-09T06:33:03.845 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:03.845 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.310 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:03.900 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:03.900 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:03.900 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8778e-05 s, 0.0 kB/s 2023-10-09T06:33:03.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:03.901 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.311 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:03.956 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:03.956 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:03.956 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9735e-05 s, 0.0 kB/s 2023-10-09T06:33:03.956 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:03.957 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.312 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:04.012 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:04.012 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:04.012 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8456e-05 s, 0.0 kB/s 2023-10-09T06:33:04.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:04.012 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.313 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:04.067 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:04.068 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:04.068 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8289e-05 s, 0.0 kB/s 2023-10-09T06:33:04.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:04.068 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.314 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:04.123 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:04.123 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:04.123 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.093e-05 s, 0.0 kB/s 2023-10-09T06:33:04.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:04.124 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.315 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:04.178 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:04.178 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:04.178 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0276e-05 s, 0.0 kB/s 2023-10-09T06:33:04.178 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:04.178 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.316 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:04.232 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:04.233 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:04.233 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1111e-05 s, 0.0 kB/s 2023-10-09T06:33:04.233 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:04.233 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.317 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:04.288 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:04.288 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:04.288 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4598e-05 s, 0.0 kB/s 2023-10-09T06:33:04.288 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:04.288 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.318 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:04.343 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:04.344 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:04.344 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9209e-05 s, 0.0 kB/s 2023-10-09T06:33:04.344 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:04.344 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.319 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:04.399 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:04.399 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:04.399 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7868e-05 s, 0.0 kB/s 2023-10-09T06:33:04.399 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:04.399 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.320 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:04.455 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:04.455 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:04.455 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7519e-05 s, 0.0 kB/s 2023-10-09T06:33:04.456 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:04.456 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.321 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:04.510 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:04.511 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:04.511 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0245e-05 s, 0.0 kB/s 2023-10-09T06:33:04.511 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:04.511 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.322 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:04.565 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:04.566 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:04.566 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3777e-05 s, 0.0 kB/s 2023-10-09T06:33:04.566 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:04.566 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.323 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:04.620 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:04.621 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:04.621 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.887e-05 s, 0.0 kB/s 2023-10-09T06:33:04.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:04.622 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.324 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:04.677 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:04.677 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:04.677 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0994e-05 s, 0.0 kB/s 2023-10-09T06:33:04.678 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:04.678 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.325 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:04.734 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:04.734 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:04.734 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.143e-05 s, 0.0 kB/s 2023-10-09T06:33:04.734 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:04.734 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.326 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:04.789 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:04.789 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:04.789 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9457e-05 s, 0.0 kB/s 2023-10-09T06:33:04.790 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:04.790 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.327 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:04.845 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:04.845 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:04.846 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9488e-05 s, 0.0 kB/s 2023-10-09T06:33:04.846 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:04.846 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.328 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:04.901 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:04.901 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:04.901 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0529e-05 s, 0.0 kB/s 2023-10-09T06:33:04.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:04.901 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.329 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:04.956 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:04.956 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:04.956 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8989e-05 s, 0.0 kB/s 2023-10-09T06:33:04.956 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:04.956 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.330 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:05.012 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:05.012 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:05.012 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1577e-05 s, 0.0 kB/s 2023-10-09T06:33:05.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:05.012 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.331 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:05.067 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:05.067 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:05.067 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7595e-05 s, 0.0 kB/s 2023-10-09T06:33:05.067 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:05.067 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.332 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:05.123 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:05.123 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:05.124 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8425e-05 s, 0.0 kB/s 2023-10-09T06:33:05.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:05.124 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.333 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:05.178 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:05.179 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:05.179 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9199e-05 s, 0.0 kB/s 2023-10-09T06:33:05.179 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:05.179 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.334 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:05.233 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:05.234 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:05.234 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4239e-05 s, 0.0 kB/s 2023-10-09T06:33:05.234 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:05.234 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.335 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:05.288 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:05.289 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:05.289 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0548e-05 s, 0.0 kB/s 2023-10-09T06:33:05.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:05.289 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.336 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:05.345 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:05.345 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:05.345 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8677e-05 s, 0.0 kB/s 2023-10-09T06:33:05.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:05.346 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.337 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:05.401 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:05.401 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:05.401 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9739e-05 s, 0.0 kB/s 2023-10-09T06:33:05.401 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:05.402 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.338 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:05.457 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:05.457 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:05.457 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8515e-05 s, 0.0 kB/s 2023-10-09T06:33:05.457 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:05.458 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.339 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:05.513 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:05.513 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:05.513 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8393e-05 s, 0.0 kB/s 2023-10-09T06:33:05.514 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:05.514 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.340 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:05.569 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:05.569 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:05.569 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0982e-05 s, 0.0 kB/s 2023-10-09T06:33:05.570 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:05.570 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.341 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:05.625 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:05.625 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:05.625 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0891e-05 s, 0.0 kB/s 2023-10-09T06:33:05.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:05.626 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.342 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:05.681 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:05.681 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:05.681 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8636e-05 s, 0.0 kB/s 2023-10-09T06:33:05.682 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:05.683 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.343 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:05.737 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:05.737 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:05.737 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7971e-05 s, 0.0 kB/s 2023-10-09T06:33:05.737 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:05.738 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.344 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:05.793 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:05.793 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:05.793 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7324e-05 s, 0.0 kB/s 2023-10-09T06:33:05.794 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:05.794 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.345 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:05.850 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:05.850 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:05.850 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7933e-05 s, 0.0 kB/s 2023-10-09T06:33:05.850 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:05.850 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.346 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:05.905 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:05.905 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:05.905 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9038e-05 s, 0.0 kB/s 2023-10-09T06:33:05.906 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:05.906 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.347 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:05.960 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:05.961 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:05.961 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8094e-05 s, 0.0 kB/s 2023-10-09T06:33:05.961 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:05.961 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.348 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:06.016 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:06.016 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:06.016 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0635e-05 s, 0.0 kB/s 2023-10-09T06:33:06.016 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:06.016 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.349 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:06.071 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:06.071 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:06.072 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1637e-05 s, 0.0 kB/s 2023-10-09T06:33:06.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:06.072 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.350 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:06.127 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:06.127 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:06.127 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.241e-05 s, 0.0 kB/s 2023-10-09T06:33:06.127 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:06.128 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.351 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:06.182 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:06.182 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:06.182 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9586e-05 s, 0.0 kB/s 2023-10-09T06:33:06.183 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:06.183 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.352 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:06.237 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:06.238 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:06.238 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.998e-05 s, 0.0 kB/s 2023-10-09T06:33:06.238 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:06.238 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.353 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:06.292 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:06.293 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:06.293 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1213e-05 s, 0.0 kB/s 2023-10-09T06:33:06.293 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:06.293 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.354 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:06.348 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:06.348 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:06.348 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0158e-05 s, 0.0 kB/s 2023-10-09T06:33:06.348 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:06.348 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.355 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:06.403 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:06.403 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:06.403 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2924e-05 s, 0.0 kB/s 2023-10-09T06:33:06.403 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:06.404 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.356 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:06.459 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:06.459 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:06.459 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8076e-05 s, 0.0 kB/s 2023-10-09T06:33:06.459 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:06.459 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.357 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:06.515 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:06.515 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:06.515 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7451e-05 s, 0.0 kB/s 2023-10-09T06:33:06.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:06.515 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.358 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:06.569 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:06.570 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:06.570 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1962e-05 s, 0.0 kB/s 2023-10-09T06:33:06.570 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:06.570 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.359 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:06.625 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:06.625 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:06.626 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0935e-05 s, 0.0 kB/s 2023-10-09T06:33:06.626 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:06.626 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.360 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:06.680 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:06.681 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:06.681 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7705e-05 s, 0.0 kB/s 2023-10-09T06:33:06.681 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:06.681 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.361 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:06.735 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:06.736 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:06.736 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9967e-05 s, 0.0 kB/s 2023-10-09T06:33:06.736 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:06.736 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.362 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:06.792 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:06.792 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:06.793 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2571e-05 s, 0.0 kB/s 2023-10-09T06:33:06.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:06.793 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.363 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:06.848 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:06.849 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:06.849 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7961e-05 s, 0.0 kB/s 2023-10-09T06:33:06.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:06.849 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.364 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:06.904 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:06.904 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:06.905 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1498e-05 s, 0.0 kB/s 2023-10-09T06:33:06.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:06.905 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.365 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:06.960 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:06.960 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:06.960 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8646e-05 s, 0.0 kB/s 2023-10-09T06:33:06.961 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:06.961 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.366 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:07.016 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:07.016 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:07.016 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7674e-05 s, 0.0 kB/s 2023-10-09T06:33:07.016 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:07.016 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.367 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:07.072 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:07.072 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:07.072 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9082e-05 s, 0.0 kB/s 2023-10-09T06:33:07.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:07.072 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.368 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:07.127 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:07.127 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:07.127 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.6056e-05 s, 0.0 kB/s 2023-10-09T06:33:07.127 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:07.127 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.369 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:07.185 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:07.185 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:07.186 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7504e-05 s, 0.0 kB/s 2023-10-09T06:33:07.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:07.186 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.370 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:07.241 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:07.241 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:07.242 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8699e-05 s, 0.0 kB/s 2023-10-09T06:33:07.242 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:07.242 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.371 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:07.297 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:07.297 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:07.297 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0194e-05 s, 0.0 kB/s 2023-10-09T06:33:07.298 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:07.298 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.372 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:07.353 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:07.354 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:07.354 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8542e-05 s, 0.0 kB/s 2023-10-09T06:33:07.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:07.354 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.373 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:07.409 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:07.409 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:07.409 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.023e-05 s, 0.0 kB/s 2023-10-09T06:33:07.409 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:07.409 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.374 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:07.465 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:07.465 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:07.465 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0479e-05 s, 0.0 kB/s 2023-10-09T06:33:07.465 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:07.465 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.375 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:07.520 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:07.521 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:07.521 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1584e-05 s, 0.0 kB/s 2023-10-09T06:33:07.521 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:07.521 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.376 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:07.576 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:07.576 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:07.576 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1797e-05 s, 0.0 kB/s 2023-10-09T06:33:07.576 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:07.576 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.377 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:07.633 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:07.633 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:07.633 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0306e-05 s, 0.0 kB/s 2023-10-09T06:33:07.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:07.633 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.378 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:07.689 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:07.690 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:07.690 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1309e-05 s, 0.0 kB/s 2023-10-09T06:33:07.690 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:07.690 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.379 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:07.745 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:07.745 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:07.745 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8937e-05 s, 0.0 kB/s 2023-10-09T06:33:07.745 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:07.746 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.380 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:07.801 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:07.801 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:07.801 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 5.167e-05 s, 0.0 kB/s 2023-10-09T06:33:07.801 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:07.801 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.381 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:07.856 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:07.856 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:07.856 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9372e-05 s, 0.0 kB/s 2023-10-09T06:33:07.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:07.857 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.382 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:07.913 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:07.913 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:07.913 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1174e-05 s, 0.0 kB/s 2023-10-09T06:33:07.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:07.913 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.383 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:07.968 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:07.968 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:07.968 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0657e-05 s, 0.0 kB/s 2023-10-09T06:33:07.968 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:07.968 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.384 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:08.024 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:08.024 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:08.024 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0551e-05 s, 0.0 kB/s 2023-10-09T06:33:08.024 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:08.025 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.385 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:08.080 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:08.080 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:08.080 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7958e-05 s, 0.0 kB/s 2023-10-09T06:33:08.081 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:08.081 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.386 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:08.135 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:08.135 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:08.136 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0941e-05 s, 0.0 kB/s 2023-10-09T06:33:08.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:08.136 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.387 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:08.191 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:08.191 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:08.191 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7759e-05 s, 0.0 kB/s 2023-10-09T06:33:08.192 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:08.192 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.388 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:08.247 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:08.247 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:08.247 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.994e-05 s, 0.0 kB/s 2023-10-09T06:33:08.247 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:08.248 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.389 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:08.303 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:08.303 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:08.303 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0555e-05 s, 0.0 kB/s 2023-10-09T06:33:08.303 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:08.304 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.390 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:08.359 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:08.359 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:08.359 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8809e-05 s, 0.0 kB/s 2023-10-09T06:33:08.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:08.359 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.391 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:08.414 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:08.414 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:08.414 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0945e-05 s, 0.0 kB/s 2023-10-09T06:33:08.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:08.415 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.392 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:08.471 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:08.471 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:08.471 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0747e-05 s, 0.0 kB/s 2023-10-09T06:33:08.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:08.472 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.393 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:08.527 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:08.528 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:08.528 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1297e-05 s, 0.0 kB/s 2023-10-09T06:33:08.528 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:08.528 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.394 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:08.583 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:08.583 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:08.583 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.194e-05 s, 0.0 kB/s 2023-10-09T06:33:08.584 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:08.584 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.395 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:08.639 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:08.640 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:08.640 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.722e-05 s, 0.0 kB/s 2023-10-09T06:33:08.640 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:08.640 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.396 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:08.695 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:08.695 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:08.696 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7583e-05 s, 0.0 kB/s 2023-10-09T06:33:08.696 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:08.696 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.397 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:08.751 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:08.751 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:08.751 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.996e-05 s, 0.0 kB/s 2023-10-09T06:33:08.752 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:08.752 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.398 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:08.807 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:08.807 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:08.807 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0174e-05 s, 0.0 kB/s 2023-10-09T06:33:08.807 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:08.807 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.399 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:08.863 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:08.863 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:08.863 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7705e-05 s, 0.0 kB/s 2023-10-09T06:33:08.863 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:08.863 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.400 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:08.919 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:08.919 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:08.919 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7732e-05 s, 0.0 kB/s 2023-10-09T06:33:08.919 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:08.919 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.401 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:08.974 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:08.974 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:08.974 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0474e-05 s, 0.0 kB/s 2023-10-09T06:33:08.974 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:08.974 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.402 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:09.028 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:09.029 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:09.029 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0054e-05 s, 0.0 kB/s 2023-10-09T06:33:09.029 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:09.029 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.403 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:09.084 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:09.084 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:09.084 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9624e-05 s, 0.0 kB/s 2023-10-09T06:33:09.084 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:09.085 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.404 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:09.140 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:09.140 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:09.140 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1065e-05 s, 0.0 kB/s 2023-10-09T06:33:09.141 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:09.141 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.405 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:09.196 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:09.196 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:09.196 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2622e-05 s, 0.0 kB/s 2023-10-09T06:33:09.197 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:09.197 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.406 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:09.252 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:09.252 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:09.252 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3064e-05 s, 0.0 kB/s 2023-10-09T06:33:09.253 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:09.253 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.407 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:09.308 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:09.308 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:09.308 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0733e-05 s, 0.0 kB/s 2023-10-09T06:33:09.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:09.309 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.408 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:09.364 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:09.364 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:09.364 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2983e-05 s, 0.0 kB/s 2023-10-09T06:33:09.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:09.364 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.409 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:09.419 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:09.419 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:09.419 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.179e-05 s, 0.0 kB/s 2023-10-09T06:33:09.419 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:09.419 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.410 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:09.476 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:09.476 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:09.476 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.122e-05 s, 0.0 kB/s 2023-10-09T06:33:09.476 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:09.476 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.411 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:09.531 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:09.531 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:09.531 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9737e-05 s, 0.0 kB/s 2023-10-09T06:33:09.531 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:09.531 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.412 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:09.587 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:09.587 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:09.587 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9956e-05 s, 0.0 kB/s 2023-10-09T06:33:09.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:09.587 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.413 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:09.643 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:09.643 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:09.644 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9714e-05 s, 0.0 kB/s 2023-10-09T06:33:09.644 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:09.644 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.414 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:09.699 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:09.699 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:09.699 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1132e-05 s, 0.0 kB/s 2023-10-09T06:33:09.700 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:09.700 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.415 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:09.755 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:09.755 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:09.755 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0615e-05 s, 0.0 kB/s 2023-10-09T06:33:09.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:09.756 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.416 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:09.811 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:09.811 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:09.811 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9554e-05 s, 0.0 kB/s 2023-10-09T06:33:09.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:09.812 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.417 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:09.867 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:09.867 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:09.867 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.189e-05 s, 0.0 kB/s 2023-10-09T06:33:09.867 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:09.867 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.418 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:09.923 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:09.924 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:09.924 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9866e-05 s, 0.0 kB/s 2023-10-09T06:33:09.924 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:09.924 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.419 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:09.979 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:09.979 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:09.979 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9477e-05 s, 0.0 kB/s 2023-10-09T06:33:09.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:09.979 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.420 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:10.035 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:10.035 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:10.036 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3401e-05 s, 0.0 kB/s 2023-10-09T06:33:10.036 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:10.036 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.421 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:10.091 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:10.091 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:10.091 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8758e-05 s, 0.0 kB/s 2023-10-09T06:33:10.092 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:10.092 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.422 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:10.147 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:10.147 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:10.147 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7748e-05 s, 0.0 kB/s 2023-10-09T06:33:10.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:10.148 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.423 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:10.203 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:10.203 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:10.203 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0233e-05 s, 0.0 kB/s 2023-10-09T06:33:10.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:10.204 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.424 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:10.259 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:10.259 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:10.259 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2455e-05 s, 0.0 kB/s 2023-10-09T06:33:10.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:10.260 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.425 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:10.315 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:10.315 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:10.315 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8908e-05 s, 0.0 kB/s 2023-10-09T06:33:10.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:10.315 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.426 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:10.370 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:10.370 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:10.370 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9256e-05 s, 0.0 kB/s 2023-10-09T06:33:10.370 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:10.371 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.427 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:10.426 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:10.426 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:10.426 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0587e-05 s, 0.0 kB/s 2023-10-09T06:33:10.426 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:10.427 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.428 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:10.481 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:10.481 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:10.481 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9027e-05 s, 0.0 kB/s 2023-10-09T06:33:10.482 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:10.482 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.429 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:10.537 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:10.538 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:10.538 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0169e-05 s, 0.0 kB/s 2023-10-09T06:33:10.538 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:10.538 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.430 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:10.593 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:10.594 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:10.594 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1221e-05 s, 0.0 kB/s 2023-10-09T06:33:10.594 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:10.594 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.431 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:10.650 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:10.650 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:10.650 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1398e-05 s, 0.0 kB/s 2023-10-09T06:33:10.650 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:10.650 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.432 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:10.705 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:10.705 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:10.705 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9058e-05 s, 0.0 kB/s 2023-10-09T06:33:10.705 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:10.706 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.433 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:10.761 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:10.761 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:10.761 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0247e-05 s, 0.0 kB/s 2023-10-09T06:33:10.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:10.762 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.434 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:10.817 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:10.817 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:10.817 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0578e-05 s, 0.0 kB/s 2023-10-09T06:33:10.817 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:10.818 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.435 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:10.873 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:10.874 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:10.874 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2696e-05 s, 0.0 kB/s 2023-10-09T06:33:10.874 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:10.874 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.436 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:10.930 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:10.930 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:10.930 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0891e-05 s, 0.0 kB/s 2023-10-09T06:33:10.930 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:10.930 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.437 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:10.986 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:10.986 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:10.986 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2135e-05 s, 0.0 kB/s 2023-10-09T06:33:10.987 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:10.987 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.438 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:11.042 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:11.043 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:11.043 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0381e-05 s, 0.0 kB/s 2023-10-09T06:33:11.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:11.043 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.439 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:11.098 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:11.099 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:11.099 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2297e-05 s, 0.0 kB/s 2023-10-09T06:33:11.099 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:11.099 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.440 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:11.154 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:11.154 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:11.154 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0443e-05 s, 0.0 kB/s 2023-10-09T06:33:11.155 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:11.155 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.441 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:11.210 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:11.210 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:11.210 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.036e-05 s, 0.0 kB/s 2023-10-09T06:33:11.211 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:11.211 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.442 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:11.266 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:11.266 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:11.266 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7478e-05 s, 0.0 kB/s 2023-10-09T06:33:11.267 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:11.267 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.443 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:11.322 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:11.323 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:11.323 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0548e-05 s, 0.0 kB/s 2023-10-09T06:33:11.323 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:11.323 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.444 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:11.378 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:11.378 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:11.378 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8259e-05 s, 0.0 kB/s 2023-10-09T06:33:11.378 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:11.379 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.445 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:11.434 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:11.434 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:11.434 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8988e-05 s, 0.0 kB/s 2023-10-09T06:33:11.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:11.434 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.446 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:11.490 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:11.491 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:11.491 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9027e-05 s, 0.0 kB/s 2023-10-09T06:33:11.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:11.491 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.447 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:11.546 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:11.546 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:11.546 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.753e-05 s, 0.0 kB/s 2023-10-09T06:33:11.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:11.547 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.448 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:11.602 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:11.602 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:11.602 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.88e-05 s, 0.0 kB/s 2023-10-09T06:33:11.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:11.603 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.449 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:11.658 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:11.658 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:11.658 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1139e-05 s, 0.0 kB/s 2023-10-09T06:33:11.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:11.659 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.450 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:11.714 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:11.714 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:11.714 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8183e-05 s, 0.0 kB/s 2023-10-09T06:33:11.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:11.715 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.451 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:11.769 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:11.769 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:11.770 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8842e-05 s, 0.0 kB/s 2023-10-09T06:33:11.770 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:11.771 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.452 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:11.825 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:11.826 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:11.826 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3832e-05 s, 0.0 kB/s 2023-10-09T06:33:11.826 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:11.826 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.453 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:11.881 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:11.881 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:11.881 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2415e-05 s, 0.0 kB/s 2023-10-09T06:33:11.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:11.882 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.454 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:11.937 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:11.937 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:11.938 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9052e-05 s, 0.0 kB/s 2023-10-09T06:33:11.938 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:11.938 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.455 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:11.992 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:11.992 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:11.993 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.025e-05 s, 0.0 kB/s 2023-10-09T06:33:11.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:11.993 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.456 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:12.048 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:12.048 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:12.048 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1661e-05 s, 0.0 kB/s 2023-10-09T06:33:12.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:12.049 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.457 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:12.103 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:12.104 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:12.104 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8705e-05 s, 0.0 kB/s 2023-10-09T06:33:12.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:12.104 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.458 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:12.159 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:12.159 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:12.159 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9268e-05 s, 0.0 kB/s 2023-10-09T06:33:12.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:12.159 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.459 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:12.213 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:12.214 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:12.214 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1031e-05 s, 0.0 kB/s 2023-10-09T06:33:12.214 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:12.214 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.460 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:12.269 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:12.269 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:12.269 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.896e-05 s, 0.0 kB/s 2023-10-09T06:33:12.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:12.270 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.461 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:12.325 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:12.325 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:12.325 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.06e-05 s, 0.0 kB/s 2023-10-09T06:33:12.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:12.325 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.462 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:12.381 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:12.381 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:12.381 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9328e-05 s, 0.0 kB/s 2023-10-09T06:33:12.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:12.382 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.463 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:12.436 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:12.437 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:12.437 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9989e-05 s, 0.0 kB/s 2023-10-09T06:33:12.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:12.437 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.464 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:12.492 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:12.492 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:12.492 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0711e-05 s, 0.0 kB/s 2023-10-09T06:33:12.493 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:12.493 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.465 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:12.548 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:12.548 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:12.548 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0236e-05 s, 0.0 kB/s 2023-10-09T06:33:12.549 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:12.549 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.466 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:12.604 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:12.604 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:12.604 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3022e-05 s, 0.0 kB/s 2023-10-09T06:33:12.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:12.605 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.467 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:12.660 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:12.661 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:12.661 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1808e-05 s, 0.0 kB/s 2023-10-09T06:33:12.661 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:12.661 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.468 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:12.716 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:12.716 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:12.716 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.5518e-05 s, 0.0 kB/s 2023-10-09T06:33:12.716 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:12.716 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.469 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:12.771 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:12.772 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:12.772 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9389e-05 s, 0.0 kB/s 2023-10-09T06:33:12.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:12.773 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.470 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:12.827 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:12.827 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:12.827 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9411e-05 s, 0.0 kB/s 2023-10-09T06:33:12.827 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:12.827 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.471 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:12.882 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:12.883 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:12.883 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1578e-05 s, 0.0 kB/s 2023-10-09T06:33:12.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:12.883 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.472 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:12.938 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:12.938 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:12.939 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1238e-05 s, 0.0 kB/s 2023-10-09T06:33:12.939 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:12.939 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.473 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:12.994 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:12.995 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:12.995 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7887e-05 s, 0.0 kB/s 2023-10-09T06:33:12.995 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:12.996 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.474 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:13.050 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:13.051 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:13.051 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.841e-05 s, 0.0 kB/s 2023-10-09T06:33:13.051 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:13.052 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.475 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:13.106 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:13.106 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:13.106 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8923e-05 s, 0.0 kB/s 2023-10-09T06:33:13.106 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:13.106 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.476 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:13.162 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:13.162 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:13.162 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.417e-05 s, 0.0 kB/s 2023-10-09T06:33:13.162 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:13.162 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.477 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:13.164 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:33:13.205 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:33:13.218 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:13.218 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:13.218 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.943e-05 s, 0.0 kB/s 2023-10-09T06:33:13.218 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:13.218 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.478 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:13.305 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:13.305 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:13.305 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9968e-05 s, 0.0 kB/s 2023-10-09T06:33:13.305 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:13.306 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.479 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:13.360 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:13.360 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:13.360 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8232e-05 s, 0.0 kB/s 2023-10-09T06:33:13.361 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:13.361 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.480 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:13.416 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:13.417 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:13.417 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.007e-05 s, 0.0 kB/s 2023-10-09T06:33:13.417 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:13.417 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.481 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:13.472 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:13.473 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:13.473 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8096e-05 s, 0.0 kB/s 2023-10-09T06:33:13.473 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:13.474 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.482 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:13.529 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:13.529 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:13.529 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8576e-05 s, 0.0 kB/s 2023-10-09T06:33:13.529 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:13.529 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.483 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:13.585 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:13.585 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:13.585 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0545e-05 s, 0.0 kB/s 2023-10-09T06:33:13.585 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:13.585 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.484 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:13.640 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:13.640 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:13.640 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1069e-05 s, 0.0 kB/s 2023-10-09T06:33:13.640 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:13.641 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.485 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:13.696 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:13.696 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:13.697 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8147e-05 s, 0.0 kB/s 2023-10-09T06:33:13.697 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:13.697 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.486 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:13.751 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:13.751 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:13.752 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1636e-05 s, 0.0 kB/s 2023-10-09T06:33:13.752 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:13.752 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.487 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:13.807 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:13.807 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:13.807 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7115e-05 s, 0.0 kB/s 2023-10-09T06:33:13.807 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:13.807 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.488 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:13.862 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:13.862 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:13.863 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0315e-05 s, 0.0 kB/s 2023-10-09T06:33:13.863 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:13.864 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.489 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:13.918 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:13.919 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:13.919 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.188e-05 s, 0.0 kB/s 2023-10-09T06:33:13.919 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:13.919 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.490 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:13.974 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:13.974 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:13.975 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8638e-05 s, 0.0 kB/s 2023-10-09T06:33:13.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:13.975 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.491 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:14.030 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:14.030 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:14.030 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8394e-05 s, 0.0 kB/s 2023-10-09T06:33:14.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:14.030 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.492 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:14.085 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:14.085 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:14.085 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7807e-05 s, 0.0 kB/s 2023-10-09T06:33:14.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:14.085 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.493 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:14.140 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:14.140 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:14.140 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8571e-05 s, 0.0 kB/s 2023-10-09T06:33:14.140 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:14.140 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.494 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:14.196 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:14.196 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:14.196 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9396e-05 s, 0.0 kB/s 2023-10-09T06:33:14.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:14.196 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.495 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:14.251 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:14.252 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:14.252 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0016e-05 s, 0.0 kB/s 2023-10-09T06:33:14.252 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:14.252 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.496 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:14.307 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:14.307 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:14.307 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1211e-05 s, 0.0 kB/s 2023-10-09T06:33:14.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:14.308 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.497 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:14.363 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:14.363 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:14.363 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.056e-05 s, 0.0 kB/s 2023-10-09T06:33:14.363 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:14.364 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.498 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:14.419 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:14.419 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:14.419 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8404e-05 s, 0.0 kB/s 2023-10-09T06:33:14.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:14.420 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.499 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:14.475 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:14.475 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:14.475 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9648e-05 s, 0.0 kB/s 2023-10-09T06:33:14.476 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:14.476 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.500 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:14.532 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:14.532 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:14.532 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8413e-05 s, 0.0 kB/s 2023-10-09T06:33:14.532 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:14.532 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.501 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:14.587 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:14.587 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:14.587 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9079e-05 s, 0.0 kB/s 2023-10-09T06:33:14.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:14.587 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.502 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:14.642 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:14.642 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:14.642 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7659e-05 s, 0.0 kB/s 2023-10-09T06:33:14.642 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:14.642 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.503 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:14.698 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:14.698 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:14.698 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9551e-05 s, 0.0 kB/s 2023-10-09T06:33:14.698 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:14.699 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.504 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:14.753 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:14.754 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:14.754 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8459e-05 s, 0.0 kB/s 2023-10-09T06:33:14.754 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:14.754 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.505 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:14.808 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:14.809 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:14.809 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2925e-05 s, 0.0 kB/s 2023-10-09T06:33:14.809 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:14.810 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.506 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:14.863 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:14.864 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:14.864 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0614e-05 s, 0.0 kB/s 2023-10-09T06:33:14.864 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:14.864 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.507 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:14.919 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:14.919 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:14.919 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8385e-05 s, 0.0 kB/s 2023-10-09T06:33:14.920 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:14.920 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.508 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:14.975 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:14.975 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:14.975 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1423e-05 s, 0.0 kB/s 2023-10-09T06:33:14.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:14.976 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.509 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:15.030 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:15.031 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:15.031 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7462e-05 s, 0.0 kB/s 2023-10-09T06:33:15.031 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:15.031 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.510 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:15.085 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:15.086 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:15.086 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.777e-05 s, 0.0 kB/s 2023-10-09T06:33:15.086 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:15.086 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.511 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:15.141 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:15.141 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:15.141 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0571e-05 s, 0.0 kB/s 2023-10-09T06:33:15.141 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:15.141 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.512 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:15.197 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:15.197 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:15.197 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0778e-05 s, 0.0 kB/s 2023-10-09T06:33:15.198 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:15.198 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.513 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:15.253 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:15.253 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:15.253 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7742e-05 s, 0.0 kB/s 2023-10-09T06:33:15.253 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:15.253 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.514 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:15.308 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:15.308 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:15.308 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0946e-05 s, 0.0 kB/s 2023-10-09T06:33:15.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:15.308 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.515 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:15.363 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:15.364 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:15.364 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9418e-05 s, 0.0 kB/s 2023-10-09T06:33:15.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:15.364 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.516 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:15.419 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:15.419 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:15.419 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9269e-05 s, 0.0 kB/s 2023-10-09T06:33:15.419 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:15.419 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.517 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:15.475 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:15.475 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:15.475 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8521e-05 s, 0.0 kB/s 2023-10-09T06:33:15.475 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:15.476 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.518 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:15.531 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:15.531 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:15.531 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.942e-05 s, 0.0 kB/s 2023-10-09T06:33:15.531 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:15.532 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.519 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:15.587 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:15.587 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:15.587 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9063e-05 s, 0.0 kB/s 2023-10-09T06:33:15.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:15.587 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.520 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:15.643 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:15.643 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:15.644 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0058e-05 s, 0.0 kB/s 2023-10-09T06:33:15.644 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:15.644 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.521 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:15.698 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:15.699 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:15.699 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1047e-05 s, 0.0 kB/s 2023-10-09T06:33:15.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:15.699 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.522 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:15.754 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:15.754 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:15.754 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.6544e-05 s, 0.0 kB/s 2023-10-09T06:33:15.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:15.755 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.523 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:15.810 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:15.810 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:15.810 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8351e-05 s, 0.0 kB/s 2023-10-09T06:33:15.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:15.810 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.524 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:15.865 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:15.865 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:15.865 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9729e-05 s, 0.0 kB/s 2023-10-09T06:33:15.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:15.865 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.525 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:15.921 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:15.921 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:15.921 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.895e-05 s, 0.0 kB/s 2023-10-09T06:33:15.921 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:15.922 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.526 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:15.976 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:15.976 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:15.976 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0831e-05 s, 0.0 kB/s 2023-10-09T06:33:15.976 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:15.976 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.527 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:16.031 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:16.032 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:16.032 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7712e-05 s, 0.0 kB/s 2023-10-09T06:33:16.032 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:16.032 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.528 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:16.087 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:16.087 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:16.087 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7077e-05 s, 0.0 kB/s 2023-10-09T06:33:16.088 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:16.088 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.529 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:16.142 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:16.143 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:16.143 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0625e-05 s, 0.0 kB/s 2023-10-09T06:33:16.143 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:16.143 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.530 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:16.198 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:16.198 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:16.198 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.991e-05 s, 0.0 kB/s 2023-10-09T06:33:16.198 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:16.198 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.531 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:16.254 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:16.254 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:16.254 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9577e-05 s, 0.0 kB/s 2023-10-09T06:33:16.254 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:16.255 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.532 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:16.310 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:16.310 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:16.310 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8767e-05 s, 0.0 kB/s 2023-10-09T06:33:16.310 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:16.310 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.533 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:16.365 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:16.365 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:16.365 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0808e-05 s, 0.0 kB/s 2023-10-09T06:33:16.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:16.365 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.534 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:16.420 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:16.420 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:16.420 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7984e-05 s, 0.0 kB/s 2023-10-09T06:33:16.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:16.420 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.535 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:16.475 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:16.476 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:16.476 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.004e-05 s, 0.0 kB/s 2023-10-09T06:33:16.476 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:16.476 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.536 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:16.530 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:16.531 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:16.531 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0711e-05 s, 0.0 kB/s 2023-10-09T06:33:16.531 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:16.531 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.537 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:16.586 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:16.586 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:16.586 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8286e-05 s, 0.0 kB/s 2023-10-09T06:33:16.586 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:16.586 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.538 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:16.641 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:16.641 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:16.641 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1563e-05 s, 0.0 kB/s 2023-10-09T06:33:16.642 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:16.642 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.539 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:16.697 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:16.697 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:16.697 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9459e-05 s, 0.0 kB/s 2023-10-09T06:33:16.697 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:16.698 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.540 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:16.753 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:16.753 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:16.753 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7892e-05 s, 0.0 kB/s 2023-10-09T06:33:16.753 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:16.754 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.541 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:16.809 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:16.809 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:16.809 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7809e-05 s, 0.0 kB/s 2023-10-09T06:33:16.809 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:16.809 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.542 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:16.864 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:16.864 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:16.864 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9566e-05 s, 0.0 kB/s 2023-10-09T06:33:16.864 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:16.864 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.543 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:16.918 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:16.919 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:16.919 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8769e-05 s, 0.0 kB/s 2023-10-09T06:33:16.919 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:16.919 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.544 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:16.974 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:16.974 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:16.974 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9678e-05 s, 0.0 kB/s 2023-10-09T06:33:16.974 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:16.974 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.545 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:17.029 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:17.029 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:17.029 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.839e-05 s, 0.0 kB/s 2023-10-09T06:33:17.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:17.030 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.546 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:17.084 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:17.085 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:17.085 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6516e-05 s, 0.0 kB/s 2023-10-09T06:33:17.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:17.085 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.547 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:17.139 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:17.140 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:17.140 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0741e-05 s, 0.0 kB/s 2023-10-09T06:33:17.140 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:17.140 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.548 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:17.197 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:17.197 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:17.197 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0999e-05 s, 0.0 kB/s 2023-10-09T06:33:17.197 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:17.197 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.549 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:17.253 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:17.253 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:17.253 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1606e-05 s, 0.0 kB/s 2023-10-09T06:33:17.253 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:17.254 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.550 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:17.309 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:17.309 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:17.309 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9354e-05 s, 0.0 kB/s 2023-10-09T06:33:17.310 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:17.310 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.551 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:17.364 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:17.365 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:17.365 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.118e-05 s, 0.0 kB/s 2023-10-09T06:33:17.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:17.365 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.552 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:17.420 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:17.420 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:17.420 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7155e-05 s, 0.0 kB/s 2023-10-09T06:33:17.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:17.420 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.553 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:17.475 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:17.476 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:17.476 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.908e-05 s, 0.0 kB/s 2023-10-09T06:33:17.476 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:17.476 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.554 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:17.531 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:17.531 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:17.531 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8252e-05 s, 0.0 kB/s 2023-10-09T06:33:17.531 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:17.532 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.555 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:17.586 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:17.587 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:17.587 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1303e-05 s, 0.0 kB/s 2023-10-09T06:33:17.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:17.587 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.556 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:17.642 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:17.643 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:17.643 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.292e-05 s, 0.0 kB/s 2023-10-09T06:33:17.643 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:17.643 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.557 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:17.698 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:17.698 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:17.698 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9972e-05 s, 0.0 kB/s 2023-10-09T06:33:17.698 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:17.699 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.558 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:17.756 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:17.756 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:17.757 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8515e-05 s, 0.0 kB/s 2023-10-09T06:33:17.757 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:17.757 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.559 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:17.821 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:17.821 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:17.821 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.888e-05 s, 0.0 kB/s 2023-10-09T06:33:17.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:17.822 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.560 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:17.887 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:17.887 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:17.887 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8083e-05 s, 0.0 kB/s 2023-10-09T06:33:17.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:17.888 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.561 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:17.946 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:17.947 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:17.947 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8625e-05 s, 0.0 kB/s 2023-10-09T06:33:17.947 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:17.947 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.562 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:18.003 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:18.004 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:18.004 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7069e-05 s, 0.0 kB/s 2023-10-09T06:33:18.004 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:18.004 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.563 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:18.060 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:18.061 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:18.061 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0739e-05 s, 0.0 kB/s 2023-10-09T06:33:18.061 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:18.061 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.564 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:18.118 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:18.118 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:18.118 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0721e-05 s, 0.0 kB/s 2023-10-09T06:33:18.119 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:18.119 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.565 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:18.174 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:18.174 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:18.174 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1309e-05 s, 0.0 kB/s 2023-10-09T06:33:18.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:18.175 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.566 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:18.231 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:18.231 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:18.231 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8248e-05 s, 0.0 kB/s 2023-10-09T06:33:18.231 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:18.231 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.567 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:18.293 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:18.293 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:18.294 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9198e-05 s, 0.0 kB/s 2023-10-09T06:33:18.294 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:18.294 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.568 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:18.351 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:18.352 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:18.352 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.036e-05 s, 0.0 kB/s 2023-10-09T06:33:18.352 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:18.352 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.569 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:18.410 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:18.411 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:18.411 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7528e-05 s, 0.0 kB/s 2023-10-09T06:33:18.411 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:18.411 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.570 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:18.470 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:18.470 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:18.471 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0929e-05 s, 0.0 kB/s 2023-10-09T06:33:18.472 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:18.472 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.571 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:18.534 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:18.534 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:18.534 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.176e-05 s, 0.0 kB/s 2023-10-09T06:33:18.534 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:18.535 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.572 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:18.593 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:18.593 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:18.593 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8026e-05 s, 0.0 kB/s 2023-10-09T06:33:18.594 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:18.594 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.573 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:18.652 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:18.652 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:18.653 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.994e-05 s, 0.0 kB/s 2023-10-09T06:33:18.653 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:18.653 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.574 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:18.711 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:18.711 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:18.711 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7792e-05 s, 0.0 kB/s 2023-10-09T06:33:18.712 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:18.712 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.575 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:18.771 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:18.772 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:18.772 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7906e-05 s, 0.0 kB/s 2023-10-09T06:33:18.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:18.773 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.576 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:18.826 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:18.826 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:18.826 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.815e-05 s, 0.0 kB/s 2023-10-09T06:33:18.827 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:18.827 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.577 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:18.882 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:18.882 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:18.882 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9618e-05 s, 0.0 kB/s 2023-10-09T06:33:18.882 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:18.882 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.578 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:18.939 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:18.939 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:18.939 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.98e-05 s, 0.0 kB/s 2023-10-09T06:33:18.939 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:18.939 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.579 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:18.995 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:18.995 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:18.995 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7252e-05 s, 0.0 kB/s 2023-10-09T06:33:18.996 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:18.996 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.580 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:19.051 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:19.051 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:19.051 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8199e-05 s, 0.0 kB/s 2023-10-09T06:33:19.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:19.052 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.581 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:19.107 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:19.107 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:19.107 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6792e-05 s, 0.0 kB/s 2023-10-09T06:33:19.107 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:19.108 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.582 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:19.162 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:19.162 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:19.163 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8003e-05 s, 0.0 kB/s 2023-10-09T06:33:19.163 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:19.164 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.583 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:19.218 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:19.218 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:19.219 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.6038e-05 s, 0.0 kB/s 2023-10-09T06:33:19.219 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:19.220 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.584 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:19.275 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:19.275 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:19.275 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8995e-05 s, 0.0 kB/s 2023-10-09T06:33:19.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:19.275 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.585 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:19.330 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:19.331 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:19.331 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9163e-05 s, 0.0 kB/s 2023-10-09T06:33:19.332 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:19.332 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.586 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:19.387 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:19.387 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:19.387 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9397e-05 s, 0.0 kB/s 2023-10-09T06:33:19.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:19.387 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.587 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:19.442 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:19.443 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:19.443 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9323e-05 s, 0.0 kB/s 2023-10-09T06:33:19.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:19.443 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.588 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:19.498 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:19.498 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:19.498 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8471e-05 s, 0.0 kB/s 2023-10-09T06:33:19.498 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:19.498 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.589 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:19.555 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:19.555 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:19.555 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8466e-05 s, 0.0 kB/s 2023-10-09T06:33:19.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:19.555 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.590 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:19.610 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:19.610 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:19.610 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9522e-05 s, 0.0 kB/s 2023-10-09T06:33:19.610 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:19.610 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.591 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:19.665 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:19.665 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:19.665 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0817e-05 s, 0.0 kB/s 2023-10-09T06:33:19.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:19.666 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.592 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:19.721 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:19.721 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:19.721 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8909e-05 s, 0.0 kB/s 2023-10-09T06:33:19.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:19.721 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.593 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:19.777 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:19.777 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:19.777 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6676e-05 s, 0.0 kB/s 2023-10-09T06:33:19.778 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:19.778 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.594 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:19.833 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:19.833 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:19.833 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9007e-05 s, 0.0 kB/s 2023-10-09T06:33:19.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:19.834 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.595 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:19.888 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:19.889 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:19.889 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8444e-05 s, 0.0 kB/s 2023-10-09T06:33:19.889 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:19.889 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.596 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:19.944 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:19.944 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:19.944 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7572e-05 s, 0.0 kB/s 2023-10-09T06:33:19.944 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:19.944 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.597 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:19.999 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:19.999 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:20.000 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.5921e-05 s, 0.0 kB/s 2023-10-09T06:33:20.000 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:20.000 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.598 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:20.055 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:20.055 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:20.055 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.077e-05 s, 0.0 kB/s 2023-10-09T06:33:20.055 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:20.055 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.599 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:20.110 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:20.110 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:20.111 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1888e-05 s, 0.0 kB/s 2023-10-09T06:33:20.111 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:20.111 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.600 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:20.165 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:20.165 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:20.166 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7623e-05 s, 0.0 kB/s 2023-10-09T06:33:20.166 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:20.166 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.601 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:20.221 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:20.221 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:20.221 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9953e-05 s, 0.0 kB/s 2023-10-09T06:33:20.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:20.221 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.602 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:20.277 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:20.277 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:20.277 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.36e-05 s, 0.0 kB/s 2023-10-09T06:33:20.277 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:20.278 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.603 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:20.333 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:20.333 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:20.333 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9237e-05 s, 0.0 kB/s 2023-10-09T06:33:20.333 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:20.333 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.604 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:20.389 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:20.389 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:20.389 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1254e-05 s, 0.0 kB/s 2023-10-09T06:33:20.389 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:20.389 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.605 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:20.444 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:20.445 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:20.445 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2026e-05 s, 0.0 kB/s 2023-10-09T06:33:20.445 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:20.445 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.606 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:20.500 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:20.500 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:20.500 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8517e-05 s, 0.0 kB/s 2023-10-09T06:33:20.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:20.501 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.607 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:20.555 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:20.556 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:20.556 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9399e-05 s, 0.0 kB/s 2023-10-09T06:33:20.556 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:20.556 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.608 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:20.611 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:20.611 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:20.611 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.923e-05 s, 0.0 kB/s 2023-10-09T06:33:20.611 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:20.611 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.609 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:20.667 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:20.667 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:20.667 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7981e-05 s, 0.0 kB/s 2023-10-09T06:33:20.668 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:20.668 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.610 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:20.724 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:20.724 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:20.724 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1734e-05 s, 0.0 kB/s 2023-10-09T06:33:20.725 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:20.725 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.611 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:20.780 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:20.780 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:20.780 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.151e-05 s, 0.0 kB/s 2023-10-09T06:33:20.780 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:20.780 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.612 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:20.836 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:20.836 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:20.836 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0988e-05 s, 0.0 kB/s 2023-10-09T06:33:20.836 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:20.837 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.613 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:20.892 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:20.892 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:20.892 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9929e-05 s, 0.0 kB/s 2023-10-09T06:33:20.892 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:20.892 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.614 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:20.948 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:20.948 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:20.948 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.4798e-05 s, 0.0 kB/s 2023-10-09T06:33:20.949 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:20.949 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.615 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:21.004 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:21.004 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:21.004 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.012e-05 s, 0.0 kB/s 2023-10-09T06:33:21.004 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:21.005 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.616 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:21.060 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:21.060 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:21.060 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.606e-05 s, 0.0 kB/s 2023-10-09T06:33:21.061 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:21.061 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.617 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:21.117 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:21.117 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:21.117 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0464e-05 s, 0.0 kB/s 2023-10-09T06:33:21.117 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:21.117 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.618 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:21.172 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:21.172 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:21.172 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.755e-05 s, 0.0 kB/s 2023-10-09T06:33:21.173 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:21.173 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.619 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:21.228 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:21.228 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:21.228 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9236e-05 s, 0.0 kB/s 2023-10-09T06:33:21.229 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:21.229 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.620 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:21.284 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:21.285 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:21.285 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9616e-05 s, 0.0 kB/s 2023-10-09T06:33:21.285 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:21.285 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.621 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:21.340 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:21.340 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:21.340 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8388e-05 s, 0.0 kB/s 2023-10-09T06:33:21.340 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:21.341 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.622 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:21.396 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:21.396 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:21.396 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0385e-05 s, 0.0 kB/s 2023-10-09T06:33:21.397 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:21.397 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.623 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:21.451 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:21.451 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:21.452 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1569e-05 s, 0.0 kB/s 2023-10-09T06:33:21.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:21.452 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.624 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:21.507 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:21.507 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:21.507 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0378e-05 s, 0.0 kB/s 2023-10-09T06:33:21.507 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:21.507 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.625 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:21.562 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:21.562 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:21.562 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.219e-05 s, 0.0 kB/s 2023-10-09T06:33:21.563 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:21.564 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.626 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:21.619 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:21.619 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:21.619 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.964e-05 s, 0.0 kB/s 2023-10-09T06:33:21.619 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:21.619 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.627 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:21.675 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:21.676 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:21.676 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1391e-05 s, 0.0 kB/s 2023-10-09T06:33:21.676 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:21.676 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.628 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:21.733 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:21.734 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:21.734 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7521e-05 s, 0.0 kB/s 2023-10-09T06:33:21.734 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:21.734 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.629 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:21.789 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:21.789 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:21.789 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2054e-05 s, 0.0 kB/s 2023-10-09T06:33:21.790 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:21.790 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.630 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:21.845 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:21.845 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:21.845 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8187e-05 s, 0.0 kB/s 2023-10-09T06:33:21.845 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:21.845 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.631 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:21.900 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:21.900 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:21.900 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8036e-05 s, 0.0 kB/s 2023-10-09T06:33:21.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:21.901 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.632 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:21.957 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:21.957 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:21.957 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8181e-05 s, 0.0 kB/s 2023-10-09T06:33:21.957 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:21.957 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.633 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:22.012 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:22.012 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:22.012 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7331e-05 s, 0.0 kB/s 2023-10-09T06:33:22.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:22.012 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.634 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:22.068 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:22.068 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:22.068 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8733e-05 s, 0.0 kB/s 2023-10-09T06:33:22.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:22.068 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.635 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:22.123 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:22.123 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:22.123 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8818e-05 s, 0.0 kB/s 2023-10-09T06:33:22.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:22.124 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.636 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:22.179 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:22.179 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:22.179 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8124e-05 s, 0.0 kB/s 2023-10-09T06:33:22.179 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:22.179 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.637 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:22.234 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:22.235 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:22.235 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6435e-05 s, 0.0 kB/s 2023-10-09T06:33:22.235 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:22.235 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.638 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:22.289 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:22.290 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:22.290 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9547e-05 s, 0.0 kB/s 2023-10-09T06:33:22.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:22.290 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.639 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:22.346 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:22.347 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:22.347 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9076e-05 s, 0.0 kB/s 2023-10-09T06:33:22.348 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:22.348 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.640 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:22.402 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:22.402 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:22.402 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7208e-05 s, 0.0 kB/s 2023-10-09T06:33:22.402 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:22.402 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.641 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:22.457 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:22.457 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:22.457 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9614e-05 s, 0.0 kB/s 2023-10-09T06:33:22.457 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:22.457 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.642 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:22.511 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:22.511 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:22.512 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9388e-05 s, 0.0 kB/s 2023-10-09T06:33:22.512 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:22.512 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.643 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:22.567 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:22.567 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:22.567 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9912e-05 s, 0.0 kB/s 2023-10-09T06:33:22.567 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:22.567 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.644 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:22.622 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:22.622 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:22.622 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8835e-05 s, 0.0 kB/s 2023-10-09T06:33:22.623 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:22.623 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.645 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:22.678 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:22.678 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:22.679 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9971e-05 s, 0.0 kB/s 2023-10-09T06:33:22.679 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:22.679 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.646 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:22.734 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:22.734 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:22.734 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.975e-05 s, 0.0 kB/s 2023-10-09T06:33:22.734 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:22.734 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.647 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:22.790 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:22.790 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:22.790 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7572e-05 s, 0.0 kB/s 2023-10-09T06:33:22.790 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:22.790 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.648 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:22.845 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:22.845 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:22.845 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8475e-05 s, 0.0 kB/s 2023-10-09T06:33:22.846 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:22.846 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.649 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:22.900 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:22.900 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:22.900 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8721e-05 s, 0.0 kB/s 2023-10-09T06:33:22.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:22.901 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.650 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:22.956 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:22.957 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:22.957 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9622e-05 s, 0.0 kB/s 2023-10-09T06:33:22.957 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:22.957 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.651 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:23.011 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:23.012 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:23.012 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8799e-05 s, 0.0 kB/s 2023-10-09T06:33:23.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:23.012 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.652 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:23.066 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:23.067 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:23.067 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6273e-05 s, 0.0 kB/s 2023-10-09T06:33:23.067 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:23.067 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.653 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:23.121 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:23.169 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:23.169 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0812e-05 s, 0.0 kB/s 2023-10-09T06:33:23.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:23.170 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.654 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:23.185 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:23.185 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:23.186 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8729e-05 s, 0.0 kB/s 2023-10-09T06:33:23.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:23.186 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.655 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:23.241 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:23.241 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:23.241 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8501e-05 s, 0.0 kB/s 2023-10-09T06:33:23.241 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:23.241 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.656 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:23.296 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:23.296 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:23.296 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0943e-05 s, 0.0 kB/s 2023-10-09T06:33:23.296 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:23.297 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.657 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:23.352 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:23.352 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:23.352 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7127e-05 s, 0.0 kB/s 2023-10-09T06:33:23.352 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:23.353 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.658 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:23.408 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:23.408 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:23.408 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8465e-05 s, 0.0 kB/s 2023-10-09T06:33:23.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:23.408 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.659 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:23.463 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:23.464 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:23.464 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9031e-05 s, 0.0 kB/s 2023-10-09T06:33:23.464 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:23.464 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.660 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:23.519 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:23.519 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:23.520 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8108e-05 s, 0.0 kB/s 2023-10-09T06:33:23.520 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:23.520 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.661 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:23.575 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:23.575 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:23.575 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0142e-05 s, 0.0 kB/s 2023-10-09T06:33:23.575 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:23.575 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.662 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:23.630 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:23.630 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:23.630 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7869e-05 s, 0.0 kB/s 2023-10-09T06:33:23.630 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:23.630 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.663 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:23.686 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:23.686 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:23.687 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1283e-05 s, 0.0 kB/s 2023-10-09T06:33:23.687 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:23.687 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.664 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:23.742 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:23.742 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:23.742 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9148e-05 s, 0.0 kB/s 2023-10-09T06:33:23.742 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:23.742 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.665 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:23.799 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:23.800 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:23.800 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4139e-05 s, 0.0 kB/s 2023-10-09T06:33:23.800 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:23.800 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.666 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:23.855 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:23.855 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:23.855 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9505e-05 s, 0.0 kB/s 2023-10-09T06:33:23.855 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:23.856 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.667 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:23.910 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:23.911 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:23.911 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.006e-05 s, 0.0 kB/s 2023-10-09T06:33:23.911 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:23.911 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.668 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:23.965 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:23.966 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:23.966 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0165e-05 s, 0.0 kB/s 2023-10-09T06:33:23.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:23.966 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.669 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:24.021 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:24.021 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:24.022 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.786e-05 s, 0.0 kB/s 2023-10-09T06:33:24.022 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:24.022 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.670 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:24.077 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:24.077 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:24.077 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.5229e-05 s, 0.0 kB/s 2023-10-09T06:33:24.077 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:24.077 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.671 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:24.132 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:24.132 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:24.132 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0569e-05 s, 0.0 kB/s 2023-10-09T06:33:24.132 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:24.132 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.672 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:24.186 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:24.187 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:24.187 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9351e-05 s, 0.0 kB/s 2023-10-09T06:33:24.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:24.188 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.673 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:24.242 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:24.242 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:24.242 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9072e-05 s, 0.0 kB/s 2023-10-09T06:33:24.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:24.243 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.674 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:24.297 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:24.297 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:24.298 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.055e-05 s, 0.0 kB/s 2023-10-09T06:33:24.298 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:24.298 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.675 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:24.353 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:24.353 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:24.354 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8584e-05 s, 0.0 kB/s 2023-10-09T06:33:24.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:24.354 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.676 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:24.408 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:24.409 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:24.409 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3191e-05 s, 0.0 kB/s 2023-10-09T06:33:24.409 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:24.409 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.677 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:24.463 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:24.464 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:24.464 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8036e-05 s, 0.0 kB/s 2023-10-09T06:33:24.464 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:24.464 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.678 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:24.518 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:24.519 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:24.519 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6679e-05 s, 0.0 kB/s 2023-10-09T06:33:24.519 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:24.519 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.679 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:24.574 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:24.574 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:24.574 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7334e-05 s, 0.0 kB/s 2023-10-09T06:33:24.574 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:24.575 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.680 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:24.630 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:24.630 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:24.630 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0507e-05 s, 0.0 kB/s 2023-10-09T06:33:24.630 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:24.631 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.681 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:24.686 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:24.687 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:24.687 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7897e-05 s, 0.0 kB/s 2023-10-09T06:33:24.687 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:24.687 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.682 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:24.742 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:24.742 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:24.742 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.057e-05 s, 0.0 kB/s 2023-10-09T06:33:24.742 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:24.743 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.683 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:24.798 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:24.798 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:24.798 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.197e-05 s, 0.0 kB/s 2023-10-09T06:33:24.799 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:24.799 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.684 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:24.854 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:24.854 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:24.854 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9015e-05 s, 0.0 kB/s 2023-10-09T06:33:24.854 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:24.855 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.685 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:24.910 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:24.910 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:24.910 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7686e-05 s, 0.0 kB/s 2023-10-09T06:33:24.910 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:24.911 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.686 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:24.966 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:24.966 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:24.966 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8125e-05 s, 0.0 kB/s 2023-10-09T06:33:24.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:24.966 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.687 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:25.020 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:25.021 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:25.021 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8416e-05 s, 0.0 kB/s 2023-10-09T06:33:25.021 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:25.021 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.688 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:25.076 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:25.076 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:25.076 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0133e-05 s, 0.0 kB/s 2023-10-09T06:33:25.076 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:25.076 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.689 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:25.131 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:25.131 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:25.132 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9833e-05 s, 0.0 kB/s 2023-10-09T06:33:25.132 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:25.132 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.690 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:25.187 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:25.188 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:25.188 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0539e-05 s, 0.0 kB/s 2023-10-09T06:33:25.188 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:25.188 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.691 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:25.243 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:25.243 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:25.243 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7821e-05 s, 0.0 kB/s 2023-10-09T06:33:25.244 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:25.244 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.692 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:25.298 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:25.299 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:25.299 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9167e-05 s, 0.0 kB/s 2023-10-09T06:33:25.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:25.299 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.693 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:25.354 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:25.354 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:25.354 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8282e-05 s, 0.0 kB/s 2023-10-09T06:33:25.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:25.354 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.694 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:25.409 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:25.410 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:25.410 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1137e-05 s, 0.0 kB/s 2023-10-09T06:33:25.410 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:25.410 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.695 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:25.465 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:25.465 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:25.465 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.224e-05 s, 0.0 kB/s 2023-10-09T06:33:25.465 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:25.465 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.696 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:25.520 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:25.520 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:25.520 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7802e-05 s, 0.0 kB/s 2023-10-09T06:33:25.521 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:25.521 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.697 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:25.576 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:25.577 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:25.577 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9818e-05 s, 0.0 kB/s 2023-10-09T06:33:25.577 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:25.577 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.698 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:25.632 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:25.632 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:25.632 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0294e-05 s, 0.0 kB/s 2023-10-09T06:33:25.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:25.633 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.699 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:25.688 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:25.688 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:25.689 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3037e-05 s, 0.0 kB/s 2023-10-09T06:33:25.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:25.689 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.700 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:25.744 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:25.744 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:25.744 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9847e-05 s, 0.0 kB/s 2023-10-09T06:33:25.744 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:25.745 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.701 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:25.800 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:25.800 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:25.800 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7474e-05 s, 0.0 kB/s 2023-10-09T06:33:25.801 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:25.801 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.702 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:25.855 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:25.856 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:25.856 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7291e-05 s, 0.0 kB/s 2023-10-09T06:33:25.856 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:25.856 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.703 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:25.911 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:25.911 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:25.912 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0643e-05 s, 0.0 kB/s 2023-10-09T06:33:25.912 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:25.912 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.704 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:25.967 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:25.967 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:25.967 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8955e-05 s, 0.0 kB/s 2023-10-09T06:33:25.967 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:25.967 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.705 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:26.021 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:26.022 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:26.022 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9987e-05 s, 0.0 kB/s 2023-10-09T06:33:26.022 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:26.022 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.706 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:26.077 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:26.078 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:26.078 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0157e-05 s, 0.0 kB/s 2023-10-09T06:33:26.078 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:26.078 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.707 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:26.133 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:26.133 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:26.133 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.014e-05 s, 0.0 kB/s 2023-10-09T06:33:26.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:26.133 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.708 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:26.190 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:26.190 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:26.190 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1977e-05 s, 0.0 kB/s 2023-10-09T06:33:26.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:26.190 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.709 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:26.246 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:26.246 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:26.246 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7679e-05 s, 0.0 kB/s 2023-10-09T06:33:26.246 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:26.246 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.710 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:26.302 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:26.302 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:26.303 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.6132e-05 s, 0.0 kB/s 2023-10-09T06:33:26.303 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:26.303 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.711 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:26.358 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:26.358 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:26.358 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3036e-05 s, 0.0 kB/s 2023-10-09T06:33:26.358 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:26.358 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.712 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:26.412 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:26.413 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:26.413 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9092e-05 s, 0.0 kB/s 2023-10-09T06:33:26.413 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:26.413 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.713 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:26.467 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:26.468 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:26.468 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8279e-05 s, 0.0 kB/s 2023-10-09T06:33:26.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:26.468 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.714 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:26.522 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:26.522 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:26.522 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9548e-05 s, 0.0 kB/s 2023-10-09T06:33:26.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:26.523 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.715 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:26.578 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:26.578 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:26.578 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7536e-05 s, 0.0 kB/s 2023-10-09T06:33:26.578 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:26.578 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.716 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:26.633 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:26.633 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:26.633 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.8416e-05 s, 0.0 kB/s 2023-10-09T06:33:26.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:26.633 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.717 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:26.689 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:26.689 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:26.690 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8151e-05 s, 0.0 kB/s 2023-10-09T06:33:26.690 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:26.690 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.718 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:26.745 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:26.745 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:26.745 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0146e-05 s, 0.0 kB/s 2023-10-09T06:33:26.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:26.746 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.719 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:26.802 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:26.803 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:26.803 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.925e-05 s, 0.0 kB/s 2023-10-09T06:33:26.803 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:26.804 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.720 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:26.858 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:26.858 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:26.859 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8168e-05 s, 0.0 kB/s 2023-10-09T06:33:26.859 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:26.859 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.721 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:26.914 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:26.914 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:26.914 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.833e-05 s, 0.0 kB/s 2023-10-09T06:33:26.914 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:26.915 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.722 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:26.969 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:26.970 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:26.970 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7675e-05 s, 0.0 kB/s 2023-10-09T06:33:26.970 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:26.970 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.723 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:27.024 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:27.025 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:27.025 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.801e-05 s, 0.0 kB/s 2023-10-09T06:33:27.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:27.025 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.724 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:27.080 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:27.080 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:27.080 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1224e-05 s, 0.0 kB/s 2023-10-09T06:33:27.081 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:27.081 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.725 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:27.136 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:27.136 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:27.136 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7133e-05 s, 0.0 kB/s 2023-10-09T06:33:27.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:27.137 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.726 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:27.192 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:27.193 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:27.193 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.039e-05 s, 0.0 kB/s 2023-10-09T06:33:27.193 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:27.193 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.727 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:27.248 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:27.248 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:27.248 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0555e-05 s, 0.0 kB/s 2023-10-09T06:33:27.248 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:27.248 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.728 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:27.304 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:27.304 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:27.304 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0403e-05 s, 0.0 kB/s 2023-10-09T06:33:27.304 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:27.305 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.729 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:27.359 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:27.360 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:27.360 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3368e-05 s, 0.0 kB/s 2023-10-09T06:33:27.360 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:27.360 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.730 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:27.415 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:27.415 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:27.416 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3384e-05 s, 0.0 kB/s 2023-10-09T06:33:27.416 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:27.416 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.731 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:27.471 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:27.471 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:27.471 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.1746e-05 s, 0.0 kB/s 2023-10-09T06:33:27.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:27.472 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.732 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:27.527 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:27.527 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:27.527 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8075e-05 s, 0.0 kB/s 2023-10-09T06:33:27.527 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:27.527 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.733 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:27.583 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:27.583 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:27.583 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.015e-05 s, 0.0 kB/s 2023-10-09T06:33:27.583 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:27.584 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.734 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:27.638 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:27.638 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:27.638 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7688e-05 s, 0.0 kB/s 2023-10-09T06:33:27.639 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:27.639 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.735 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:27.695 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:27.695 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:27.695 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2208e-05 s, 0.0 kB/s 2023-10-09T06:33:27.696 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:27.696 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.736 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:27.751 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:27.751 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:27.751 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9075e-05 s, 0.0 kB/s 2023-10-09T06:33:27.751 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:27.751 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.737 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:27.806 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:27.807 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:27.807 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9299e-05 s, 0.0 kB/s 2023-10-09T06:33:27.807 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:27.807 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.738 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:27.862 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:27.862 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:27.862 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8661e-05 s, 0.0 kB/s 2023-10-09T06:33:27.862 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:27.862 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.739 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:27.917 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:27.917 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:27.917 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7727e-05 s, 0.0 kB/s 2023-10-09T06:33:27.917 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:27.918 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.740 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:27.973 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:27.973 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:27.973 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0816e-05 s, 0.0 kB/s 2023-10-09T06:33:27.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:27.973 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.741 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:28.029 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:28.029 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:28.029 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3513e-05 s, 0.0 kB/s 2023-10-09T06:33:28.029 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:28.029 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.742 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:28.086 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:28.086 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:28.086 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2304e-05 s, 0.0 kB/s 2023-10-09T06:33:28.087 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:28.087 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.743 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:28.142 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:28.142 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:28.142 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0169e-05 s, 0.0 kB/s 2023-10-09T06:33:28.142 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:28.143 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.744 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:28.198 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:28.198 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:28.199 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.089e-05 s, 0.0 kB/s 2023-10-09T06:33:28.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:28.199 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.745 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:28.254 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:28.254 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:28.254 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9114e-05 s, 0.0 kB/s 2023-10-09T06:33:28.254 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:28.255 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.746 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:28.309 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:28.310 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:28.310 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8419e-05 s, 0.0 kB/s 2023-10-09T06:33:28.310 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:28.310 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.747 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:28.364 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:28.365 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:28.365 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9634e-05 s, 0.0 kB/s 2023-10-09T06:33:28.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:28.366 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.748 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:28.421 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:28.421 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:28.421 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0691e-05 s, 0.0 kB/s 2023-10-09T06:33:28.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:28.421 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.749 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:28.477 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:28.477 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:28.477 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1018e-05 s, 0.0 kB/s 2023-10-09T06:33:28.477 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:28.477 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.750 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:28.534 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:28.534 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:28.534 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8446e-05 s, 0.0 kB/s 2023-10-09T06:33:28.534 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:28.535 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.751 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:28.588 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:28.589 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:28.589 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9445e-05 s, 0.0 kB/s 2023-10-09T06:33:28.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:28.589 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.752 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:28.643 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:28.643 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.722e-05 s, 0.0 kB/s 2023-10-09T06:33:28.644 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:28.645 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.753 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:28.700 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:28.700 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:28.700 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1894e-05 s, 0.0 kB/s 2023-10-09T06:33:28.701 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:28.701 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.754 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:28.756 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:28.756 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:28.756 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7486e-05 s, 0.0 kB/s 2023-10-09T06:33:28.757 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:28.757 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.755 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:28.812 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:28.812 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:28.812 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.989e-05 s, 0.0 kB/s 2023-10-09T06:33:28.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:28.812 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.756 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:28.868 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:28.868 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:28.868 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.037e-05 s, 0.0 kB/s 2023-10-09T06:33:28.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:28.868 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.757 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:28.923 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:28.923 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:28.923 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8072e-05 s, 0.0 kB/s 2023-10-09T06:33:28.924 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:28.924 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.758 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:28.980 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:28.980 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:28.980 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0273e-05 s, 0.0 kB/s 2023-10-09T06:33:28.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:28.980 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.759 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:29.035 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:29.035 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:29.035 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0784e-05 s, 0.0 kB/s 2023-10-09T06:33:29.035 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:29.035 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.760 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:29.091 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:29.091 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:29.091 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1148e-05 s, 0.0 kB/s 2023-10-09T06:33:29.091 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:29.091 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.761 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:29.146 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:29.146 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:29.146 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1128e-05 s, 0.0 kB/s 2023-10-09T06:33:29.146 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:29.147 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.762 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:29.202 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:29.203 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:29.203 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2176e-05 s, 0.0 kB/s 2023-10-09T06:33:29.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:29.203 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.763 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:29.257 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:29.258 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:29.258 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1849e-05 s, 0.0 kB/s 2023-10-09T06:33:29.258 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:29.258 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.764 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:29.314 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:29.314 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:29.314 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8795e-05 s, 0.0 kB/s 2023-10-09T06:33:29.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:29.315 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.765 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:29.370 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:29.370 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:29.370 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9778e-05 s, 0.0 kB/s 2023-10-09T06:33:29.370 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:29.371 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.766 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:29.426 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:29.426 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:29.426 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8436e-05 s, 0.0 kB/s 2023-10-09T06:33:29.426 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:29.427 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.767 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:29.482 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:29.482 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:29.482 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0284e-05 s, 0.0 kB/s 2023-10-09T06:33:29.482 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:29.482 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.768 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:29.538 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:29.538 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:29.538 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1354e-05 s, 0.0 kB/s 2023-10-09T06:33:29.539 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:29.539 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.769 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:29.594 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:29.594 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:29.594 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0862e-05 s, 0.0 kB/s 2023-10-09T06:33:29.594 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:29.595 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.770 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:29.649 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:29.649 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:29.649 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7926e-05 s, 0.0 kB/s 2023-10-09T06:33:29.649 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:29.649 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.771 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:29.704 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:29.704 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:29.704 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0958e-05 s, 0.0 kB/s 2023-10-09T06:33:29.705 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:29.705 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.772 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:29.760 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:29.760 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:29.760 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.935e-05 s, 0.0 kB/s 2023-10-09T06:33:29.760 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:29.761 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.773 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:29.816 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:29.816 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:29.816 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0827e-05 s, 0.0 kB/s 2023-10-09T06:33:29.816 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:29.817 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.774 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:29.872 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:29.872 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:29.872 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0106e-05 s, 0.0 kB/s 2023-10-09T06:33:29.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:29.872 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.775 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:29.930 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:29.930 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:29.930 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.811e-05 s, 0.0 kB/s 2023-10-09T06:33:29.930 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:29.930 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.776 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:29.982 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:29.982 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:29.982 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9404e-05 s, 0.0 kB/s 2023-10-09T06:33:29.983 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:29.983 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.777 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:30.037 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:30.037 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:30.037 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9165e-05 s, 0.0 kB/s 2023-10-09T06:33:30.038 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:30.038 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.778 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:30.094 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:30.094 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:30.094 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9093e-05 s, 0.0 kB/s 2023-10-09T06:33:30.094 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:30.095 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.779 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:30.150 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:30.150 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:30.150 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0269e-05 s, 0.0 kB/s 2023-10-09T06:33:30.150 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:30.151 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.780 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:30.206 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:30.206 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:30.206 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9175e-05 s, 0.0 kB/s 2023-10-09T06:33:30.206 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:30.206 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.781 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:30.262 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:30.262 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:30.262 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7586e-05 s, 0.0 kB/s 2023-10-09T06:33:30.262 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:30.262 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.782 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:30.317 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:30.317 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:30.317 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9141e-05 s, 0.0 kB/s 2023-10-09T06:33:30.317 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:30.318 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.783 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:30.373 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:30.373 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:30.373 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7306e-05 s, 0.0 kB/s 2023-10-09T06:33:30.373 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:30.373 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.784 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:30.428 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:30.428 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:30.428 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0286e-05 s, 0.0 kB/s 2023-10-09T06:33:30.428 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:30.428 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.785 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:30.483 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:30.483 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:30.483 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4468e-05 s, 0.0 kB/s 2023-10-09T06:33:30.483 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:30.483 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.786 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:30.538 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:30.538 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:30.538 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.6563e-05 s, 0.0 kB/s 2023-10-09T06:33:30.538 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:30.539 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.787 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:30.594 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:30.594 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:30.594 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0477e-05 s, 0.0 kB/s 2023-10-09T06:33:30.594 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:30.595 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.788 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:30.650 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:30.650 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:30.650 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0787e-05 s, 0.0 kB/s 2023-10-09T06:33:30.650 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:30.650 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.789 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:30.705 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:30.705 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:30.706 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.19e-05 s, 0.0 kB/s 2023-10-09T06:33:30.706 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:30.706 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.790 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:30.761 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:30.761 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:30.761 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2106e-05 s, 0.0 kB/s 2023-10-09T06:33:30.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:30.762 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.791 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:30.817 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:30.817 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:30.817 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9886e-05 s, 0.0 kB/s 2023-10-09T06:33:30.817 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:30.818 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.792 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:30.873 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:30.873 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:30.873 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0543e-05 s, 0.0 kB/s 2023-10-09T06:33:30.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:30.873 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.793 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:30.929 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:30.929 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:30.929 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9286e-05 s, 0.0 kB/s 2023-10-09T06:33:30.929 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:30.930 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.794 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:30.984 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:30.985 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:30.985 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6887e-05 s, 0.0 kB/s 2023-10-09T06:33:30.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:30.985 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.795 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:31.040 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:31.040 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:31.041 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.9443e-05 s, 0.0 kB/s 2023-10-09T06:33:31.041 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:31.041 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.796 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:31.096 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:31.097 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:31.097 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8436e-05 s, 0.0 kB/s 2023-10-09T06:33:31.097 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:31.097 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.797 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:31.152 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:31.152 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:31.152 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7678e-05 s, 0.0 kB/s 2023-10-09T06:33:31.152 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:31.152 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.798 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:31.207 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:31.207 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:31.207 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1036e-05 s, 0.0 kB/s 2023-10-09T06:33:31.208 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:31.208 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.799 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:31.263 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:31.263 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:31.263 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7373e-05 s, 0.0 kB/s 2023-10-09T06:33:31.263 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:31.263 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.800 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:31.317 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:31.318 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:31.318 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0593e-05 s, 0.0 kB/s 2023-10-09T06:33:31.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:31.318 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.801 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:31.372 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:31.373 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:31.373 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0012e-05 s, 0.0 kB/s 2023-10-09T06:33:31.373 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:31.373 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.802 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:31.428 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:31.428 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:31.428 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1311e-05 s, 0.0 kB/s 2023-10-09T06:33:31.429 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:31.429 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.803 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:31.484 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:31.484 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:31.484 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7779e-05 s, 0.0 kB/s 2023-10-09T06:33:31.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:31.485 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.804 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:31.540 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:31.541 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:31.541 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8822e-05 s, 0.0 kB/s 2023-10-09T06:33:31.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:31.541 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.805 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:31.596 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:31.596 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:31.596 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9377e-05 s, 0.0 kB/s 2023-10-09T06:33:31.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:31.596 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.806 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:31.651 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:31.652 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:31.652 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7876e-05 s, 0.0 kB/s 2023-10-09T06:33:31.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:31.652 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.807 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:31.707 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:31.707 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:31.707 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1129e-05 s, 0.0 kB/s 2023-10-09T06:33:31.708 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:31.708 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.808 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:31.763 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:31.763 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:31.763 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0378e-05 s, 0.0 kB/s 2023-10-09T06:33:31.763 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:31.763 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.809 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:31.819 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:31.819 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:31.819 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0896e-05 s, 0.0 kB/s 2023-10-09T06:33:31.819 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:31.819 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.810 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:31.875 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:31.875 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:31.875 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8008e-05 s, 0.0 kB/s 2023-10-09T06:33:31.876 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:31.876 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.811 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:31.930 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:31.930 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:31.930 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.5823e-05 s, 0.0 kB/s 2023-10-09T06:33:31.931 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:31.931 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.812 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:31.987 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:31.987 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:31.987 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.5661e-05 s, 0.0 kB/s 2023-10-09T06:33:31.987 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:31.987 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.813 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:32.043 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:32.043 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:32.043 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.0249e-05 s, 0.0 kB/s 2023-10-09T06:33:32.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:32.043 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.814 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:32.097 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:32.097 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:32.098 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6946e-05 s, 0.0 kB/s 2023-10-09T06:33:32.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:32.098 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.815 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:32.153 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:32.153 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:32.153 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8684e-05 s, 0.0 kB/s 2023-10-09T06:33:32.153 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:32.153 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.816 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:32.209 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:32.209 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:32.210 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7589e-05 s, 0.0 kB/s 2023-10-09T06:33:32.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:32.210 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.817 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:32.264 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:32.265 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:32.265 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9391e-05 s, 0.0 kB/s 2023-10-09T06:33:32.265 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:32.265 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.818 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:32.319 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:32.319 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:32.320 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.146e-05 s, 0.0 kB/s 2023-10-09T06:33:32.320 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:32.320 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.819 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:32.375 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:32.375 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:32.375 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9492e-05 s, 0.0 kB/s 2023-10-09T06:33:32.375 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:32.375 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.820 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:32.430 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:32.430 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:32.431 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.916e-05 s, 0.0 kB/s 2023-10-09T06:33:32.431 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:32.431 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.821 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:32.486 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:32.486 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:32.486 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1042e-05 s, 0.0 kB/s 2023-10-09T06:33:32.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:32.487 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.822 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:32.542 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:32.542 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:32.542 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1348e-05 s, 0.0 kB/s 2023-10-09T06:33:32.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:32.542 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.823 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:32.598 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:32.598 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:32.598 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3377e-05 s, 0.0 kB/s 2023-10-09T06:33:32.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:32.598 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.824 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:32.653 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:32.653 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:32.653 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0717e-05 s, 0.0 kB/s 2023-10-09T06:33:32.653 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:32.653 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.825 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:32.709 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:32.709 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:32.709 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8173e-05 s, 0.0 kB/s 2023-10-09T06:33:32.710 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:32.710 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.826 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:32.765 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:32.765 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:32.765 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9345e-05 s, 0.0 kB/s 2023-10-09T06:33:32.765 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:32.765 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.827 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:32.821 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:32.821 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:32.821 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0565e-05 s, 0.0 kB/s 2023-10-09T06:33:32.822 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:32.822 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.828 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:32.877 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:32.877 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:32.877 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8639e-05 s, 0.0 kB/s 2023-10-09T06:33:32.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:32.877 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.829 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:32.933 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:32.933 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:32.933 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1166e-05 s, 0.0 kB/s 2023-10-09T06:33:32.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:32.934 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.830 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:32.989 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:32.989 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:32.989 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.822e-05 s, 0.0 kB/s 2023-10-09T06:33:32.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:32.990 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.831 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:33.045 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:33.045 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:33.045 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7844e-05 s, 0.0 kB/s 2023-10-09T06:33:33.045 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:33.046 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.832 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:33.101 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:33.101 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:33.101 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0219e-05 s, 0.0 kB/s 2023-10-09T06:33:33.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:33.101 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.833 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:33.157 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:33.157 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:33.157 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7456e-05 s, 0.0 kB/s 2023-10-09T06:33:33.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:33.157 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.834 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:33.214 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:33.214 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:33.214 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9707e-05 s, 0.0 kB/s 2023-10-09T06:33:33.214 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:33.215 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.835 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:33.270 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:33.270 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:33.270 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8041e-05 s, 0.0 kB/s 2023-10-09T06:33:33.270 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:33.270 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.836 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:33.325 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:33.325 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:33.325 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9973e-05 s, 0.0 kB/s 2023-10-09T06:33:33.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:33.325 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.837 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:33.379 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:33.379 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:33.379 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9978e-05 s, 0.0 kB/s 2023-10-09T06:33:33.380 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:33.380 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.838 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:33.435 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:33.435 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:33.436 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1595e-05 s, 0.0 kB/s 2023-10-09T06:33:33.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:33.436 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.839 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:33.491 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:33.491 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:33.491 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9933e-05 s, 0.0 kB/s 2023-10-09T06:33:33.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:33.491 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.840 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:33.546 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:33.546 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:33.546 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7789e-05 s, 0.0 kB/s 2023-10-09T06:33:33.546 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:33.546 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.841 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:33.602 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:33.602 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:33.602 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.894e-05 s, 0.0 kB/s 2023-10-09T06:33:33.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:33.603 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.842 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:33.658 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:33.658 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:33.658 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1035e-05 s, 0.0 kB/s 2023-10-09T06:33:33.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:33.658 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.843 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:33.714 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:33.716 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:33.717 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8274e-05 s, 0.0 kB/s 2023-10-09T06:33:33.717 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:33.717 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.844 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:33.770 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:33.770 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:33.770 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9773e-05 s, 0.0 kB/s 2023-10-09T06:33:33.770 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:33.771 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.845 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:33.826 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:33.827 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:33.827 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9349e-05 s, 0.0 kB/s 2023-10-09T06:33:33.827 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:33.828 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.846 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:33.882 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:33.882 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:33.883 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0041e-05 s, 0.0 kB/s 2023-10-09T06:33:33.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:33.883 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.847 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:33.938 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:33.938 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:33.938 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1831e-05 s, 0.0 kB/s 2023-10-09T06:33:33.939 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:33.939 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.848 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:33.993 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:33.994 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:33.994 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.7526e-05 s, 0.0 kB/s 2023-10-09T06:33:33.994 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:33.994 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.849 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:34.049 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:34.049 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:34.049 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0189e-05 s, 0.0 kB/s 2023-10-09T06:33:34.050 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:34.050 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.850 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:34.105 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:34.105 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:34.105 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8284e-05 s, 0.0 kB/s 2023-10-09T06:33:34.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:34.106 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.851 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:34.160 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:34.161 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:34.161 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9737e-05 s, 0.0 kB/s 2023-10-09T06:33:34.161 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:34.161 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.852 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:34.216 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:34.216 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:34.217 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8248e-05 s, 0.0 kB/s 2023-10-09T06:33:34.217 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:34.217 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.853 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:34.271 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:34.271 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:34.271 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.885e-05 s, 0.0 kB/s 2023-10-09T06:33:34.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:34.272 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.854 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:34.327 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:34.327 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:34.328 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7141e-05 s, 0.0 kB/s 2023-10-09T06:33:34.328 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:34.328 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.855 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:34.383 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:34.383 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:34.383 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3328e-05 s, 0.0 kB/s 2023-10-09T06:33:34.383 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:34.383 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.856 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:34.439 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:34.439 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:34.439 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.531e-05 s, 0.0 kB/s 2023-10-09T06:33:34.439 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:34.439 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.857 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:34.495 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:34.496 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:34.496 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.5964e-05 s, 0.0 kB/s 2023-10-09T06:33:34.496 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:34.496 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.858 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:34.552 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:34.552 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:34.553 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6438e-05 s, 0.0 kB/s 2023-10-09T06:33:34.553 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:34.554 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.859 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:34.608 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:34.608 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:34.609 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8863e-05 s, 0.0 kB/s 2023-10-09T06:33:34.609 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:34.609 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.860 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:34.664 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:34.665 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:34.665 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0687e-05 s, 0.0 kB/s 2023-10-09T06:33:34.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:34.665 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.861 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:34.720 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:34.720 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:34.720 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2255e-05 s, 0.0 kB/s 2023-10-09T06:33:34.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:34.721 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.862 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:34.776 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:34.776 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:34.776 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7836e-05 s, 0.0 kB/s 2023-10-09T06:33:34.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:34.776 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.863 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:34.832 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:34.832 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:34.832 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1008e-05 s, 0.0 kB/s 2023-10-09T06:33:34.832 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:34.832 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.864 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:34.888 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:34.889 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:34.889 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9214e-05 s, 0.0 kB/s 2023-10-09T06:33:34.889 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:34.889 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.865 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:34.944 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:34.944 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:34.944 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0362e-05 s, 0.0 kB/s 2023-10-09T06:33:34.944 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:34.944 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.866 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:35.000 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:35.000 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:35.001 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1989e-05 s, 0.0 kB/s 2023-10-09T06:33:35.001 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:35.001 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.867 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:35.056 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:35.056 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:35.056 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0194e-05 s, 0.0 kB/s 2023-10-09T06:33:35.056 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:35.056 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.868 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:35.112 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:35.112 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:35.112 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8439e-05 s, 0.0 kB/s 2023-10-09T06:33:35.112 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:35.113 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.869 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:35.167 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:35.168 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:35.168 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.979e-05 s, 0.0 kB/s 2023-10-09T06:33:35.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:35.168 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.870 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:35.223 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:35.223 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:35.223 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7562e-05 s, 0.0 kB/s 2023-10-09T06:33:35.224 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:35.224 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.871 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:35.278 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:35.279 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:35.279 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6935e-05 s, 0.0 kB/s 2023-10-09T06:33:35.279 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:35.279 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.872 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:35.334 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:35.334 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:35.334 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7874e-05 s, 0.0 kB/s 2023-10-09T06:33:35.334 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:35.335 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.873 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:35.390 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:35.390 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:35.390 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.037e-05 s, 0.0 kB/s 2023-10-09T06:33:35.390 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:35.390 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.874 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:35.447 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:35.447 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:35.447 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9049e-05 s, 0.0 kB/s 2023-10-09T06:33:35.447 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:35.447 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.875 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:35.502 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:35.502 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:35.502 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3691e-05 s, 0.0 kB/s 2023-10-09T06:33:35.503 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:35.503 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.876 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:35.558 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:35.558 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:35.558 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3235e-05 s, 0.0 kB/s 2023-10-09T06:33:35.559 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:35.559 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.877 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:35.614 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:35.614 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:35.614 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7612e-05 s, 0.0 kB/s 2023-10-09T06:33:35.614 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:35.615 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.878 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:35.670 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:35.670 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:35.670 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2368e-05 s, 0.0 kB/s 2023-10-09T06:33:35.670 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:35.670 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.879 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:35.726 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:35.726 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:35.726 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9186e-05 s, 0.0 kB/s 2023-10-09T06:33:35.727 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:35.727 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.880 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:35.782 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:35.783 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:35.783 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2413e-05 s, 0.0 kB/s 2023-10-09T06:33:35.783 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:35.783 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.881 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:35.838 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:35.838 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:35.838 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9805e-05 s, 0.0 kB/s 2023-10-09T06:33:35.839 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:35.839 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.882 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:35.894 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:35.895 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:35.895 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.4094e-05 s, 0.0 kB/s 2023-10-09T06:33:35.895 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:35.895 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.883 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:35.950 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:35.951 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:35.951 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.889e-05 s, 0.0 kB/s 2023-10-09T06:33:35.951 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:35.951 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.884 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:36.006 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:36.006 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:36.006 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0614e-05 s, 0.0 kB/s 2023-10-09T06:33:36.006 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:36.007 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.885 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:36.061 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:36.061 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:36.061 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0027e-05 s, 0.0 kB/s 2023-10-09T06:33:36.062 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:36.062 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.886 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:36.118 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:36.118 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:36.118 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7445e-05 s, 0.0 kB/s 2023-10-09T06:33:36.119 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:36.119 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.887 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:36.174 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:36.174 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:36.174 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.944e-05 s, 0.0 kB/s 2023-10-09T06:33:36.174 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:36.175 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.888 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:36.230 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:36.230 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:36.231 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9097e-05 s, 0.0 kB/s 2023-10-09T06:33:36.231 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:36.231 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.889 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:36.286 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:36.286 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:36.286 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0596e-05 s, 0.0 kB/s 2023-10-09T06:33:36.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:36.287 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.890 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:36.342 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:36.342 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:36.342 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.0518e-05 s, 0.0 kB/s 2023-10-09T06:33:36.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:36.343 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.891 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:36.398 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:36.398 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:36.398 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1087e-05 s, 0.0 kB/s 2023-10-09T06:33:36.398 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:36.398 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.892 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:36.453 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:36.454 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:36.454 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0067e-05 s, 0.0 kB/s 2023-10-09T06:33:36.454 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:36.454 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.893 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:36.509 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:36.509 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:36.509 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8119e-05 s, 0.0 kB/s 2023-10-09T06:33:36.509 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:36.510 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.894 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:36.565 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:36.565 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:36.565 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.77e-05 s, 0.0 kB/s 2023-10-09T06:33:36.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:36.565 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.895 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:36.620 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:36.620 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:36.620 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7439e-05 s, 0.0 kB/s 2023-10-09T06:33:36.620 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:36.621 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.896 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:36.676 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:36.676 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:36.676 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1147e-05 s, 0.0 kB/s 2023-10-09T06:33:36.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:36.677 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.897 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:36.731 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:36.732 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:36.732 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7981e-05 s, 0.0 kB/s 2023-10-09T06:33:36.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:36.732 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.898 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:36.787 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:36.787 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:36.787 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9152e-05 s, 0.0 kB/s 2023-10-09T06:33:36.788 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:36.788 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.899 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:36.843 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:36.844 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:36.844 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1701e-05 s, 0.0 kB/s 2023-10-09T06:33:36.844 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:36.844 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.900 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:36.899 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:36.899 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:36.899 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1887e-05 s, 0.0 kB/s 2023-10-09T06:33:36.899 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:36.900 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.901 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:36.955 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:36.955 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:36.955 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.5993e-05 s, 0.0 kB/s 2023-10-09T06:33:36.955 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:36.956 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.902 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:37.013 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:37.013 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:37.013 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1203e-05 s, 0.0 kB/s 2023-10-09T06:33:37.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:37.013 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.903 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:37.068 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:37.068 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:37.068 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9259e-05 s, 0.0 kB/s 2023-10-09T06:33:37.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:37.069 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.904 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:37.124 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:37.124 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:37.124 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4904e-05 s, 0.0 kB/s 2023-10-09T06:33:37.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:37.124 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.905 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:37.179 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:37.179 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:37.179 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8838e-05 s, 0.0 kB/s 2023-10-09T06:33:37.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:37.180 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.906 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:37.235 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:37.235 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:37.235 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7252e-05 s, 0.0 kB/s 2023-10-09T06:33:37.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:37.236 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.907 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:37.291 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:37.291 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:37.291 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9859e-05 s, 0.0 kB/s 2023-10-09T06:33:37.291 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:37.291 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.908 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:37.347 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:37.347 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:37.347 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6909e-05 s, 0.0 kB/s 2023-10-09T06:33:37.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:37.348 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.909 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:37.402 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:37.403 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:37.403 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9386e-05 s, 0.0 kB/s 2023-10-09T06:33:37.403 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:37.403 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.910 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:37.458 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:37.458 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:37.458 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9652e-05 s, 0.0 kB/s 2023-10-09T06:33:37.459 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:37.459 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.911 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:37.514 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:37.514 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:37.514 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9065e-05 s, 0.0 kB/s 2023-10-09T06:33:37.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:37.515 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.912 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:37.570 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:37.571 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:37.571 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.051e-05 s, 0.0 kB/s 2023-10-09T06:33:37.571 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:37.571 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.913 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:37.626 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:37.626 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:37.626 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9477e-05 s, 0.0 kB/s 2023-10-09T06:33:37.626 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:37.627 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.914 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:37.682 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:37.683 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:37.683 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9665e-05 s, 0.0 kB/s 2023-10-09T06:33:37.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:37.683 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.915 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:37.738 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:37.738 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:37.738 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8673e-05 s, 0.0 kB/s 2023-10-09T06:33:37.739 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:37.739 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.916 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:37.794 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:37.794 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:37.794 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.6756e-05 s, 0.0 kB/s 2023-10-09T06:33:37.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:37.795 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.917 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:37.850 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:37.850 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:37.850 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7358e-05 s, 0.0 kB/s 2023-10-09T06:33:37.850 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:37.850 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.918 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:37.905 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:37.905 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:37.905 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7569e-05 s, 0.0 kB/s 2023-10-09T06:33:37.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:37.905 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.919 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:37.960 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:37.961 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:37.961 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9523e-05 s, 0.0 kB/s 2023-10-09T06:33:37.961 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:37.961 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.920 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:38.017 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:38.017 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:38.017 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6769e-05 s, 0.0 kB/s 2023-10-09T06:33:38.017 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:38.018 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.921 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:38.073 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:38.073 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:38.073 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7766e-05 s, 0.0 kB/s 2023-10-09T06:33:38.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:38.074 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.922 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:38.129 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:38.129 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:38.129 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3466e-05 s, 0.0 kB/s 2023-10-09T06:33:38.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:38.129 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.923 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:38.183 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:38.184 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:38.184 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9067e-05 s, 0.0 kB/s 2023-10-09T06:33:38.184 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:38.184 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.924 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:38.239 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:38.239 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:38.239 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8201e-05 s, 0.0 kB/s 2023-10-09T06:33:38.239 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:38.240 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.925 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:38.295 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:38.295 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:38.295 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.5928e-05 s, 0.0 kB/s 2023-10-09T06:33:38.295 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:38.295 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.926 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:38.350 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:38.350 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:38.350 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.97e-05 s, 0.0 kB/s 2023-10-09T06:33:38.350 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:38.351 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.927 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:38.406 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:38.406 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:38.406 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0544e-05 s, 0.0 kB/s 2023-10-09T06:33:38.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:38.407 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.928 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:38.462 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:38.463 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:38.463 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1237e-05 s, 0.0 kB/s 2023-10-09T06:33:38.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:38.463 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.929 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:38.520 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:38.520 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:38.520 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1765e-05 s, 0.0 kB/s 2023-10-09T06:33:38.520 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:38.520 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.930 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:38.578 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:38.578 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:38.578 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9903e-05 s, 0.0 kB/s 2023-10-09T06:33:38.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:38.579 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.931 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:38.641 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:38.642 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:38.642 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6872e-05 s, 0.0 kB/s 2023-10-09T06:33:38.642 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:38.642 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.932 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:38.701 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:38.701 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:38.701 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0437e-05 s, 0.0 kB/s 2023-10-09T06:33:38.701 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:38.702 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.933 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:38.766 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:38.766 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:38.766 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0033e-05 s, 0.0 kB/s 2023-10-09T06:33:38.767 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:38.767 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.934 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:38.832 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:38.832 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:38.833 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7577e-05 s, 0.0 kB/s 2023-10-09T06:33:38.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:38.833 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.935 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:38.885 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:38.886 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:38.886 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0131e-05 s, 0.0 kB/s 2023-10-09T06:33:38.887 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:38.887 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.936 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:38.945 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:38.946 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:38.946 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0187e-05 s, 0.0 kB/s 2023-10-09T06:33:38.946 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:38.946 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.937 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:39.003 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:39.004 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:39.004 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.843e-05 s, 0.0 kB/s 2023-10-09T06:33:39.004 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:39.004 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.938 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:39.064 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:39.064 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:39.064 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9483e-05 s, 0.0 kB/s 2023-10-09T06:33:39.064 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:39.065 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.939 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:39.124 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:39.124 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:39.124 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7731e-05 s, 0.0 kB/s 2023-10-09T06:33:39.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:39.125 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.940 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:39.185 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:39.185 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:39.185 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.776e-05 s, 0.0 kB/s 2023-10-09T06:33:39.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:39.185 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.941 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:39.242 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:39.243 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:39.243 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9442e-05 s, 0.0 kB/s 2023-10-09T06:33:39.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:39.243 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.942 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:39.301 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:39.301 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:39.301 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7977e-05 s, 0.0 kB/s 2023-10-09T06:33:39.302 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:39.302 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.943 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:39.359 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:39.359 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:39.359 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9288e-05 s, 0.0 kB/s 2023-10-09T06:33:39.360 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:39.360 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.944 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:39.415 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:39.415 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:39.415 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.968e-05 s, 0.0 kB/s 2023-10-09T06:33:39.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:39.415 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.945 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:39.471 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:39.472 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:39.472 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7617e-05 s, 0.0 kB/s 2023-10-09T06:33:39.472 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:39.473 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.946 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:39.526 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:39.527 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:39.527 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.063e-05 s, 0.0 kB/s 2023-10-09T06:33:39.528 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:39.528 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.947 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:39.583 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:39.583 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:39.583 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9101e-05 s, 0.0 kB/s 2023-10-09T06:33:39.583 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:39.583 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.948 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:39.639 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:39.639 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:39.639 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7127e-05 s, 0.0 kB/s 2023-10-09T06:33:39.639 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:39.640 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.949 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:39.695 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:39.695 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:39.695 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1472e-05 s, 0.0 kB/s 2023-10-09T06:33:39.696 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:39.696 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.950 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:39.751 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:39.751 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:39.751 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.064e-05 s, 0.0 kB/s 2023-10-09T06:33:39.752 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:39.752 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.951 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:39.807 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:39.807 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:39.807 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.807e-05 s, 0.0 kB/s 2023-10-09T06:33:39.807 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:39.807 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.952 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:39.862 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:39.862 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:39.862 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8661e-05 s, 0.0 kB/s 2023-10-09T06:33:39.862 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:39.863 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.953 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:39.918 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:39.918 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:39.918 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8197e-05 s, 0.0 kB/s 2023-10-09T06:33:39.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:39.918 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.954 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:39.972 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:39.973 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:39.973 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9594e-05 s, 0.0 kB/s 2023-10-09T06:33:39.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:39.973 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.955 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:40.028 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:40.028 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:40.028 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8237e-05 s, 0.0 kB/s 2023-10-09T06:33:40.028 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:40.028 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.956 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:40.083 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:40.084 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:40.084 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9535e-05 s, 0.0 kB/s 2023-10-09T06:33:40.084 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:40.084 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.957 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:40.139 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:40.139 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:40.139 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9078e-05 s, 0.0 kB/s 2023-10-09T06:33:40.139 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:40.139 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.958 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:40.195 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:40.195 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:40.195 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.7108e-05 s, 0.0 kB/s 2023-10-09T06:33:40.195 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:40.195 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.959 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:40.251 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:40.251 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:40.251 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9215e-05 s, 0.0 kB/s 2023-10-09T06:33:40.251 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:40.251 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.960 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:40.307 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:40.307 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:40.307 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8092e-05 s, 0.0 kB/s 2023-10-09T06:33:40.307 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:40.307 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.961 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:40.363 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:40.363 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:40.363 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8207e-05 s, 0.0 kB/s 2023-10-09T06:33:40.363 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:40.364 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.962 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:40.419 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:40.419 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:40.419 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1287e-05 s, 0.0 kB/s 2023-10-09T06:33:40.419 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:40.420 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.963 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:40.475 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:40.475 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:40.475 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9484e-05 s, 0.0 kB/s 2023-10-09T06:33:40.475 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:40.475 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.964 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:40.531 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:40.531 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:40.531 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0673e-05 s, 0.0 kB/s 2023-10-09T06:33:40.532 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:40.532 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.965 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:40.587 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:40.587 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:40.587 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9758e-05 s, 0.0 kB/s 2023-10-09T06:33:40.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:40.587 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.966 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:40.643 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:40.643 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:40.643 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.245e-05 s, 0.0 kB/s 2023-10-09T06:33:40.644 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:40.644 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.967 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:40.699 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:40.699 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:40.699 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.7866e-05 s, 0.0 kB/s 2023-10-09T06:33:40.700 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:40.700 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.968 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:40.755 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:40.755 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:40.755 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8952e-05 s, 0.0 kB/s 2023-10-09T06:33:40.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:40.755 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.969 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:40.811 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:40.811 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:40.811 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7774e-05 s, 0.0 kB/s 2023-10-09T06:33:40.811 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:40.811 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.970 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:40.866 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:40.866 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:40.867 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7555e-05 s, 0.0 kB/s 2023-10-09T06:33:40.867 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:40.867 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.971 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:40.921 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:40.922 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:40.922 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2009e-05 s, 0.0 kB/s 2023-10-09T06:33:40.922 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:40.922 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.972 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:40.976 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:40.976 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:40.976 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1508e-05 s, 0.0 kB/s 2023-10-09T06:33:40.977 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:40.977 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.973 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:41.031 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:41.032 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:41.032 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8147e-05 s, 0.0 kB/s 2023-10-09T06:33:41.032 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:41.032 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.974 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:41.087 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:41.087 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:41.087 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1029e-05 s, 0.0 kB/s 2023-10-09T06:33:41.088 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:41.088 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.975 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:41.143 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:41.143 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:41.143 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.27e-05 s, 0.0 kB/s 2023-10-09T06:33:41.143 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:41.143 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.976 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:41.198 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:41.199 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:41.199 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9188e-05 s, 0.0 kB/s 2023-10-09T06:33:41.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:41.199 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.977 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:41.255 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:41.255 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:41.255 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0839e-05 s, 0.0 kB/s 2023-10-09T06:33:41.255 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:41.255 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.978 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:41.311 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:41.311 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:41.311 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1861e-05 s, 0.0 kB/s 2023-10-09T06:33:41.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:41.312 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.979 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:41.367 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:41.368 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:41.368 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0271e-05 s, 0.0 kB/s 2023-10-09T06:33:41.368 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:41.368 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.980 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:41.423 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:41.423 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:41.423 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.838e-05 s, 0.0 kB/s 2023-10-09T06:33:41.423 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:41.424 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.981 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:41.479 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:41.479 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:41.479 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0369e-05 s, 0.0 kB/s 2023-10-09T06:33:41.479 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:41.479 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.982 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:41.535 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:41.535 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:41.535 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9229e-05 s, 0.0 kB/s 2023-10-09T06:33:41.536 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:41.536 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.983 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:41.591 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:41.591 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:41.591 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9935e-05 s, 0.0 kB/s 2023-10-09T06:33:41.591 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:41.592 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.984 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:41.646 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:41.646 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:41.647 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0054e-05 s, 0.0 kB/s 2023-10-09T06:33:41.647 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:41.648 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.985 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:41.703 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:41.703 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:41.703 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7656e-05 s, 0.0 kB/s 2023-10-09T06:33:41.704 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:41.704 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.986 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:41.759 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:41.759 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:41.759 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9578e-05 s, 0.0 kB/s 2023-10-09T06:33:41.759 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:41.759 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.987 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:41.815 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:41.815 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:41.815 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1366e-05 s, 0.0 kB/s 2023-10-09T06:33:41.815 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:41.816 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.988 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:41.870 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:41.871 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:41.871 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2643e-05 s, 0.0 kB/s 2023-10-09T06:33:41.871 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:41.871 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.989 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:41.926 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:41.926 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:41.926 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.991e-05 s, 0.0 kB/s 2023-10-09T06:33:41.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:41.927 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.990 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:41.982 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:41.982 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:41.983 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0248e-05 s, 0.0 kB/s 2023-10-09T06:33:41.983 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:41.983 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.991 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:42.038 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:42.038 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:42.038 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9718e-05 s, 0.0 kB/s 2023-10-09T06:33:42.038 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:42.038 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.992 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:42.095 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:42.095 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:42.095 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1644e-05 s, 0.0 kB/s 2023-10-09T06:33:42.096 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:42.096 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.993 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:42.151 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:42.151 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:42.151 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8241e-05 s, 0.0 kB/s 2023-10-09T06:33:42.152 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:42.152 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.994 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:42.207 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:42.207 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:42.207 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1709e-05 s, 0.0 kB/s 2023-10-09T06:33:42.207 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:42.207 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.995 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:42.264 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:42.264 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:42.264 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0451e-05 s, 0.0 kB/s 2023-10-09T06:33:42.265 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:42.265 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.996 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:42.320 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:42.320 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:42.320 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0539e-05 s, 0.0 kB/s 2023-10-09T06:33:42.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:42.321 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.997 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:42.375 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:42.376 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:42.376 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.888e-05 s, 0.0 kB/s 2023-10-09T06:33:42.376 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:42.376 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.998 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:42.430 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:42.431 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:42.431 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0998e-05 s, 0.0 kB/s 2023-10-09T06:33:42.431 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:42.431 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.999 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:42.486 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:42.486 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:42.486 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8746e-05 s, 0.0 kB/s 2023-10-09T06:33:42.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:42.487 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1000 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:42.542 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:42.542 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:42.542 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9001e-05 s, 0.0 kB/s 2023-10-09T06:33:42.543 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:42.543 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1001 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:42.597 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:42.597 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:42.597 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0032e-05 s, 0.0 kB/s 2023-10-09T06:33:42.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:42.598 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1002 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:42.654 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:42.655 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:42.655 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9175e-05 s, 0.0 kB/s 2023-10-09T06:33:42.655 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:42.655 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1003 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:42.710 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:42.710 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:42.711 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8666e-05 s, 0.0 kB/s 2023-10-09T06:33:42.711 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:42.711 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1004 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:42.767 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:42.767 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:42.767 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0395e-05 s, 0.0 kB/s 2023-10-09T06:33:42.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:42.768 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1005 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:42.823 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:42.824 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:42.824 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.937e-05 s, 0.0 kB/s 2023-10-09T06:33:42.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:42.825 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1006 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:42.879 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:42.879 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:42.880 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9587e-05 s, 0.0 kB/s 2023-10-09T06:33:42.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:42.880 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1007 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:42.935 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:42.935 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:42.935 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0137e-05 s, 0.0 kB/s 2023-10-09T06:33:42.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:42.936 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1008 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:42.991 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:42.991 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:42.991 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7277e-05 s, 0.0 kB/s 2023-10-09T06:33:42.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:42.991 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1009 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:43.047 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:43.047 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:43.047 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0327e-05 s, 0.0 kB/s 2023-10-09T06:33:43.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:43.048 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1010 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:43.102 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:43.103 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:43.103 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8243e-05 s, 0.0 kB/s 2023-10-09T06:33:43.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:43.104 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1011 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:43.159 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:43.159 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:43.159 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9729e-05 s, 0.0 kB/s 2023-10-09T06:33:43.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:43.159 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1012 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:43.215 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:43.215 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:43.215 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.918e-05 s, 0.0 kB/s 2023-10-09T06:33:43.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:43.215 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1013 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:43.270 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:33:43.271 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:43.272 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:43.272 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0701e-05 s, 0.0 kB/s 2023-10-09T06:33:43.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:43.272 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1014 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:43.313 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:33:43.328 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:43.328 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:43.328 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1388e-05 s, 0.0 kB/s 2023-10-09T06:33:43.329 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:43.329 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1015 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:43.421 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:43.422 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:43.422 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2377e-05 s, 0.0 kB/s 2023-10-09T06:33:43.422 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:43.422 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1016 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:43.478 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:43.479 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:43.479 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0388e-05 s, 0.0 kB/s 2023-10-09T06:33:43.479 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:43.479 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1017 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:43.535 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:43.535 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:43.535 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3748e-05 s, 0.0 kB/s 2023-10-09T06:33:43.536 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:43.536 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1018 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:43.591 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:43.591 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:43.591 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8918e-05 s, 0.0 kB/s 2023-10-09T06:33:43.591 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:43.591 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1019 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:43.647 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:43.647 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:43.647 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0664e-05 s, 0.0 kB/s 2023-10-09T06:33:43.647 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:43.648 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1020 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:43.704 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:43.704 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:43.704 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0502e-05 s, 0.0 kB/s 2023-10-09T06:33:43.704 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:43.704 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1021 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:43.760 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:43.761 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:43.761 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1832e-05 s, 0.0 kB/s 2023-10-09T06:33:43.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:43.761 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1022 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:43.818 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:43.819 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:43.819 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9833e-05 s, 0.0 kB/s 2023-10-09T06:33:43.819 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:43.819 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1023 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:43.875 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:43.876 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:43.876 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.158e-05 s, 0.0 kB/s 2023-10-09T06:33:43.876 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:43.876 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1024 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:43.930 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:43.931 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:43.931 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.39e-05 s, 0.0 kB/s 2023-10-09T06:33:43.931 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:43.931 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1025 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:43.986 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:43.986 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:43.986 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0951e-05 s, 0.0 kB/s 2023-10-09T06:33:43.987 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:43.987 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1026 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:44.042 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:44.042 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:44.042 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2082e-05 s, 0.0 kB/s 2023-10-09T06:33:44.042 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:44.043 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1027 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:44.097 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:44.098 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:44.098 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8242e-05 s, 0.0 kB/s 2023-10-09T06:33:44.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:44.098 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1028 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:44.153 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:44.154 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:44.154 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0194e-05 s, 0.0 kB/s 2023-10-09T06:33:44.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:44.154 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1029 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:44.209 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:44.209 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:44.209 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0135e-05 s, 0.0 kB/s 2023-10-09T06:33:44.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:44.210 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1030 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:44.265 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:44.265 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:44.265 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2006e-05 s, 0.0 kB/s 2023-10-09T06:33:44.265 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:44.266 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1031 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:44.321 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:44.322 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:44.322 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9252e-05 s, 0.0 kB/s 2023-10-09T06:33:44.322 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:44.322 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1032 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:44.378 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:44.378 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:44.378 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7125e-05 s, 0.0 kB/s 2023-10-09T06:33:44.378 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:44.378 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1033 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:44.433 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:44.433 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:44.433 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0226e-05 s, 0.0 kB/s 2023-10-09T06:33:44.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:44.434 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1034 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:44.490 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:44.490 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:44.491 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9575e-05 s, 0.0 kB/s 2023-10-09T06:33:44.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:44.491 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1035 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:44.546 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:44.547 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:44.547 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9188e-05 s, 0.0 kB/s 2023-10-09T06:33:44.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:44.547 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1036 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:44.602 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:44.602 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:44.602 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8825e-05 s, 0.0 kB/s 2023-10-09T06:33:44.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:44.602 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1037 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:44.657 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:44.658 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:44.658 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8567e-05 s, 0.0 kB/s 2023-10-09T06:33:44.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:44.658 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1038 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:44.713 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:44.714 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:44.714 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0816e-05 s, 0.0 kB/s 2023-10-09T06:33:44.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:44.715 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1039 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:44.769 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:44.769 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:44.769 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1446e-05 s, 0.0 kB/s 2023-10-09T06:33:44.770 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:44.770 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1040 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:44.826 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:44.826 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:44.826 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8984e-05 s, 0.0 kB/s 2023-10-09T06:33:44.826 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:44.826 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1041 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:44.881 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:44.881 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:44.881 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9158e-05 s, 0.0 kB/s 2023-10-09T06:33:44.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:44.881 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1042 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:44.937 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:44.937 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:44.937 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8575e-05 s, 0.0 kB/s 2023-10-09T06:33:44.938 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:44.938 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1043 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:44.992 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:44.993 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:44.993 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.7929e-05 s, 0.0 kB/s 2023-10-09T06:33:44.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:44.993 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1044 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:45.048 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:45.048 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:45.049 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7172e-05 s, 0.0 kB/s 2023-10-09T06:33:45.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:45.049 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1045 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:45.104 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:45.104 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:45.105 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9581e-05 s, 0.0 kB/s 2023-10-09T06:33:45.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:45.105 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1046 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:45.160 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:45.160 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:45.161 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0849e-05 s, 0.0 kB/s 2023-10-09T06:33:45.161 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:45.161 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1047 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:45.216 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:45.216 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:45.216 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8479e-05 s, 0.0 kB/s 2023-10-09T06:33:45.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:45.216 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1048 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:45.271 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:45.271 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:45.271 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.963e-05 s, 0.0 kB/s 2023-10-09T06:33:45.271 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:45.272 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1049 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:45.327 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:45.327 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:45.327 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7882e-05 s, 0.0 kB/s 2023-10-09T06:33:45.328 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:45.328 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1050 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:45.382 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:45.383 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:45.383 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9626e-05 s, 0.0 kB/s 2023-10-09T06:33:45.383 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:45.383 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1051 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:45.438 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:45.438 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:45.438 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.064e-05 s, 0.0 kB/s 2023-10-09T06:33:45.438 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:45.438 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1052 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:45.493 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:45.493 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:45.493 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7695e-05 s, 0.0 kB/s 2023-10-09T06:33:45.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:45.494 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1053 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:45.549 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:45.549 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:45.550 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8223e-05 s, 0.0 kB/s 2023-10-09T06:33:45.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:45.550 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1054 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:45.605 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:45.605 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:45.605 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8426e-05 s, 0.0 kB/s 2023-10-09T06:33:45.606 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:45.606 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1055 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:45.661 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:45.661 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:45.661 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8307e-05 s, 0.0 kB/s 2023-10-09T06:33:45.662 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:45.662 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1056 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:45.717 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:45.718 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:45.718 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.338e-05 s, 0.0 kB/s 2023-10-09T06:33:45.718 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:45.718 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1057 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:45.773 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:45.773 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:45.773 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1802e-05 s, 0.0 kB/s 2023-10-09T06:33:45.774 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:45.774 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1058 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:45.829 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:45.829 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:45.829 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8945e-05 s, 0.0 kB/s 2023-10-09T06:33:45.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:45.830 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1059 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:45.885 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:45.886 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:45.886 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9004e-05 s, 0.0 kB/s 2023-10-09T06:33:45.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:45.886 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1060 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:45.941 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:45.941 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:45.941 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8236e-05 s, 0.0 kB/s 2023-10-09T06:33:45.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:45.942 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1061 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:45.997 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:45.997 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:45.997 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1165e-05 s, 0.0 kB/s 2023-10-09T06:33:45.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:45.997 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1062 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:46.053 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:46.053 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:46.053 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1056e-05 s, 0.0 kB/s 2023-10-09T06:33:46.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:46.053 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1063 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:46.109 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:46.109 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:46.109 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8215e-05 s, 0.0 kB/s 2023-10-09T06:33:46.109 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:46.110 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1064 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:46.165 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:46.165 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:46.165 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9825e-05 s, 0.0 kB/s 2023-10-09T06:33:46.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:46.165 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1065 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:46.221 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:46.221 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:46.221 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7121e-05 s, 0.0 kB/s 2023-10-09T06:33:46.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:46.222 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1066 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:46.277 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:46.277 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:46.278 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9488e-05 s, 0.0 kB/s 2023-10-09T06:33:46.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:46.278 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1067 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:46.333 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:46.333 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:46.333 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.882e-05 s, 0.0 kB/s 2023-10-09T06:33:46.333 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:46.334 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1068 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:46.389 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:46.389 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:46.389 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8478e-05 s, 0.0 kB/s 2023-10-09T06:33:46.389 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:46.389 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1069 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:46.443 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:46.444 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:46.444 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7231e-05 s, 0.0 kB/s 2023-10-09T06:33:46.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:46.444 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1070 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:46.499 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:46.499 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:46.499 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.148e-05 s, 0.0 kB/s 2023-10-09T06:33:46.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:46.499 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1071 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:46.555 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:46.555 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:46.555 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9597e-05 s, 0.0 kB/s 2023-10-09T06:33:46.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:46.555 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1072 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:46.611 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:46.611 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:46.612 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1096e-05 s, 0.0 kB/s 2023-10-09T06:33:46.612 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:46.612 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1073 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:46.667 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:46.667 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:46.667 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8274e-05 s, 0.0 kB/s 2023-10-09T06:33:46.667 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:46.668 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1074 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:46.726 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:46.726 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:46.726 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.026e-05 s, 0.0 kB/s 2023-10-09T06:33:46.726 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:46.726 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1075 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:46.782 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:46.783 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:46.783 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2136e-05 s, 0.0 kB/s 2023-10-09T06:33:46.783 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:46.783 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1076 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:46.839 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:46.839 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:46.839 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0448e-05 s, 0.0 kB/s 2023-10-09T06:33:46.840 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:46.840 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1077 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:46.895 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:46.895 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:46.895 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0573e-05 s, 0.0 kB/s 2023-10-09T06:33:46.896 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:46.896 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1078 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:46.952 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:46.952 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:46.952 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1699e-05 s, 0.0 kB/s 2023-10-09T06:33:46.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:46.953 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1079 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:47.008 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:47.008 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:47.008 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0632e-05 s, 0.0 kB/s 2023-10-09T06:33:47.009 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:47.009 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1080 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:47.065 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:47.065 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:47.065 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1964e-05 s, 0.0 kB/s 2023-10-09T06:33:47.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:47.066 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1081 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:47.121 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:47.121 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:47.121 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0192e-05 s, 0.0 kB/s 2023-10-09T06:33:47.122 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:47.122 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1082 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:47.177 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:47.177 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:47.177 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1026e-05 s, 0.0 kB/s 2023-10-09T06:33:47.178 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:47.178 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1083 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:47.233 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:47.233 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:47.233 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2176e-05 s, 0.0 kB/s 2023-10-09T06:33:47.233 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:47.233 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1084 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:47.289 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:47.289 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:47.289 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2621e-05 s, 0.0 kB/s 2023-10-09T06:33:47.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:47.290 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1085 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:47.346 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:47.346 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:47.346 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1332e-05 s, 0.0 kB/s 2023-10-09T06:33:47.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:47.347 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1086 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:47.403 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:47.403 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:47.403 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1061e-05 s, 0.0 kB/s 2023-10-09T06:33:47.404 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:47.404 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1087 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:47.459 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:47.459 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:47.459 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1218e-05 s, 0.0 kB/s 2023-10-09T06:33:47.460 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:47.460 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1088 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:47.515 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:47.515 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:47.516 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0197e-05 s, 0.0 kB/s 2023-10-09T06:33:47.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:47.516 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1089 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:47.571 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:47.572 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:47.572 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0893e-05 s, 0.0 kB/s 2023-10-09T06:33:47.572 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:47.572 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1090 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:47.627 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:47.627 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:47.627 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2041e-05 s, 0.0 kB/s 2023-10-09T06:33:47.627 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:47.627 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1091 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:47.682 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:47.682 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:47.683 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8837e-05 s, 0.0 kB/s 2023-10-09T06:33:47.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:47.684 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1092 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:47.739 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:47.739 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:47.739 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1932e-05 s, 0.0 kB/s 2023-10-09T06:33:47.740 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:47.740 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1093 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:47.795 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:47.795 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:47.795 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.016e-05 s, 0.0 kB/s 2023-10-09T06:33:47.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:47.795 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1094 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:47.851 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:47.851 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:47.851 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7792e-05 s, 0.0 kB/s 2023-10-09T06:33:47.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:47.851 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1095 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:47.907 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:47.907 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:47.907 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0849e-05 s, 0.0 kB/s 2023-10-09T06:33:47.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:47.908 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1096 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:47.963 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:47.963 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:47.963 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8625e-05 s, 0.0 kB/s 2023-10-09T06:33:47.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:47.963 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1097 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:48.018 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:48.018 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:48.018 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7356e-05 s, 0.0 kB/s 2023-10-09T06:33:48.018 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:48.018 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1098 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:48.074 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:48.074 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:48.074 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2702e-05 s, 0.0 kB/s 2023-10-09T06:33:48.075 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:48.075 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1099 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:48.130 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:48.130 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:48.131 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0729e-05 s, 0.0 kB/s 2023-10-09T06:33:48.131 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:48.131 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1100 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:48.186 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:48.187 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:48.187 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2716e-05 s, 0.0 kB/s 2023-10-09T06:33:48.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:48.187 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1101 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:48.242 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:48.243 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:48.243 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9834e-05 s, 0.0 kB/s 2023-10-09T06:33:48.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:48.243 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1102 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:48.298 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:48.299 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:48.299 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.04e-05 s, 0.0 kB/s 2023-10-09T06:33:48.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:48.299 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1103 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:48.355 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:48.355 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:48.355 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3252e-05 s, 0.0 kB/s 2023-10-09T06:33:48.356 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:48.356 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1104 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:48.412 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:48.412 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:48.412 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9993e-05 s, 0.0 kB/s 2023-10-09T06:33:48.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:48.412 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1105 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:48.466 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:48.467 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:48.467 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0001e-05 s, 0.0 kB/s 2023-10-09T06:33:48.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:48.468 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1106 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:48.523 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:48.523 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:48.523 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9688e-05 s, 0.0 kB/s 2023-10-09T06:33:48.524 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:48.524 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1107 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:48.579 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:48.579 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:48.579 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0315e-05 s, 0.0 kB/s 2023-10-09T06:33:48.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:48.580 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1108 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:48.636 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:48.636 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:48.636 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0589e-05 s, 0.0 kB/s 2023-10-09T06:33:48.637 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:48.637 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1109 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:48.692 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:48.692 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:48.693 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8917e-05 s, 0.0 kB/s 2023-10-09T06:33:48.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:48.693 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1110 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:48.748 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:48.748 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:48.748 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1823e-05 s, 0.0 kB/s 2023-10-09T06:33:48.749 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:48.749 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1111 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:48.804 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:48.804 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:48.804 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0572e-05 s, 0.0 kB/s 2023-10-09T06:33:48.804 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:48.804 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1112 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:48.861 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:48.861 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:48.861 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0735e-05 s, 0.0 kB/s 2023-10-09T06:33:48.861 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:48.861 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1113 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:48.923 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:48.923 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:48.924 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1573e-05 s, 0.0 kB/s 2023-10-09T06:33:48.924 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:48.924 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1114 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:48.979 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:48.979 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:48.979 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0651e-05 s, 0.0 kB/s 2023-10-09T06:33:48.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:48.979 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1115 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:49.035 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:49.035 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:49.035 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8658e-05 s, 0.0 kB/s 2023-10-09T06:33:49.035 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:49.036 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1116 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:49.091 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:49.091 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:49.091 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0137e-05 s, 0.0 kB/s 2023-10-09T06:33:49.091 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:49.092 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1117 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:49.147 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:49.147 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:49.147 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9241e-05 s, 0.0 kB/s 2023-10-09T06:33:49.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:49.148 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1118 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:49.203 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:49.203 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:49.203 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.73e-05 s, 0.0 kB/s 2023-10-09T06:33:49.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:49.204 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1119 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:49.259 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:49.259 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:49.259 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6961e-05 s, 0.0 kB/s 2023-10-09T06:33:49.260 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:49.260 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1120 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:49.314 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:49.315 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:49.315 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0914e-05 s, 0.0 kB/s 2023-10-09T06:33:49.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:49.315 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1121 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:49.370 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:49.371 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:49.371 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.858e-05 s, 0.0 kB/s 2023-10-09T06:33:49.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:49.371 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1122 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:49.426 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:49.426 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:49.426 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9299e-05 s, 0.0 kB/s 2023-10-09T06:33:49.426 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:49.427 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1123 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:49.482 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:49.482 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:49.483 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0039e-05 s, 0.0 kB/s 2023-10-09T06:33:49.483 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:49.483 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1124 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:49.538 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:49.538 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:49.538 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0174e-05 s, 0.0 kB/s 2023-10-09T06:33:49.539 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:49.539 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1125 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:49.594 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:49.594 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:49.594 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.2283e-05 s, 0.0 kB/s 2023-10-09T06:33:49.594 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:49.595 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1126 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:49.650 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:49.650 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:49.650 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8125e-05 s, 0.0 kB/s 2023-10-09T06:33:49.651 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:49.651 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1127 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:49.706 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:49.707 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:49.707 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.735e-05 s, 0.0 kB/s 2023-10-09T06:33:49.707 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:49.707 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1128 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:49.762 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:49.762 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:49.762 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8009e-05 s, 0.0 kB/s 2023-10-09T06:33:49.763 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:49.763 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1129 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:49.818 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:49.818 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:49.818 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1659e-05 s, 0.0 kB/s 2023-10-09T06:33:49.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:49.818 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1130 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:49.873 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:49.873 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:49.873 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.764e-05 s, 0.0 kB/s 2023-10-09T06:33:49.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:49.873 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1131 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:49.929 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:49.929 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:49.929 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0368e-05 s, 0.0 kB/s 2023-10-09T06:33:49.929 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:49.929 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1132 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:49.985 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:49.985 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:49.985 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7584e-05 s, 0.0 kB/s 2023-10-09T06:33:49.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:49.986 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1133 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:50.041 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:50.041 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:50.041 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9538e-05 s, 0.0 kB/s 2023-10-09T06:33:50.041 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:50.042 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1134 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:50.096 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:50.096 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:50.096 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1639e-05 s, 0.0 kB/s 2023-10-09T06:33:50.097 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:50.097 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1135 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:50.152 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:50.152 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:50.152 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7038e-05 s, 0.0 kB/s 2023-10-09T06:33:50.152 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:50.153 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1136 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:50.208 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:50.208 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:50.208 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.026e-05 s, 0.0 kB/s 2023-10-09T06:33:50.208 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:50.208 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1137 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:50.262 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:50.262 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:50.263 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7678e-05 s, 0.0 kB/s 2023-10-09T06:33:50.263 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:50.264 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1138 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:50.319 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:50.319 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:50.319 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3409e-05 s, 0.0 kB/s 2023-10-09T06:33:50.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:50.319 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1139 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:50.374 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:50.375 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:50.375 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.27e-05 s, 0.0 kB/s 2023-10-09T06:33:50.375 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:50.375 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1140 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:50.430 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:50.430 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:50.430 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9926e-05 s, 0.0 kB/s 2023-10-09T06:33:50.430 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:50.430 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1141 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:50.485 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:50.485 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:50.485 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8851e-05 s, 0.0 kB/s 2023-10-09T06:33:50.485 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:50.485 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1142 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:50.541 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:50.541 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:50.542 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1917e-05 s, 0.0 kB/s 2023-10-09T06:33:50.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:50.542 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1143 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:50.597 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:50.597 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:50.597 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1349e-05 s, 0.0 kB/s 2023-10-09T06:33:50.597 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:50.598 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1144 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:50.652 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:50.653 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:50.653 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.206e-05 s, 0.0 kB/s 2023-10-09T06:33:50.653 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:50.653 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1145 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:50.709 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:50.709 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:50.709 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0681e-05 s, 0.0 kB/s 2023-10-09T06:33:50.709 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:50.709 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1146 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:50.764 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:50.765 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:50.765 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.781e-05 s, 0.0 kB/s 2023-10-09T06:33:50.765 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:50.765 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1147 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:50.820 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:50.820 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:50.820 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2807e-05 s, 0.0 kB/s 2023-10-09T06:33:50.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:50.821 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1148 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:50.876 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:50.876 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:50.876 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1321e-05 s, 0.0 kB/s 2023-10-09T06:33:50.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:50.877 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1149 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:50.932 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:50.932 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:50.932 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0715e-05 s, 0.0 kB/s 2023-10-09T06:33:50.932 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:50.932 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1150 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:50.988 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:50.988 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:50.988 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8231e-05 s, 0.0 kB/s 2023-10-09T06:33:50.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:50.989 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1151 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:51.045 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:51.045 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:51.045 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2428e-05 s, 0.0 kB/s 2023-10-09T06:33:51.045 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:51.046 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1152 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:51.101 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:51.101 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:51.102 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.054e-05 s, 0.0 kB/s 2023-10-09T06:33:51.102 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:51.102 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1153 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:51.157 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:51.157 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:51.157 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3111e-05 s, 0.0 kB/s 2023-10-09T06:33:51.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:51.157 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1154 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:51.214 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:51.214 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:51.214 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0366e-05 s, 0.0 kB/s 2023-10-09T06:33:51.214 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:51.215 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1155 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:51.269 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:51.269 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:51.269 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8029e-05 s, 0.0 kB/s 2023-10-09T06:33:51.270 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:51.270 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1156 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:51.325 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:51.325 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:51.325 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0398e-05 s, 0.0 kB/s 2023-10-09T06:33:51.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:51.326 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1157 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:51.381 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:51.381 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:51.381 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0681e-05 s, 0.0 kB/s 2023-10-09T06:33:51.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:51.381 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1158 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:51.437 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:51.437 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:51.437 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9241e-05 s, 0.0 kB/s 2023-10-09T06:33:51.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:51.437 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1159 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:51.493 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:51.493 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:51.493 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.932e-05 s, 0.0 kB/s 2023-10-09T06:33:51.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:51.494 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1160 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:51.550 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:51.550 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:51.550 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9957e-05 s, 0.0 kB/s 2023-10-09T06:33:51.551 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:51.551 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1161 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:51.607 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:51.607 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:51.607 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2353e-05 s, 0.0 kB/s 2023-10-09T06:33:51.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:51.608 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1162 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:51.662 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:51.663 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:51.663 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.094e-05 s, 0.0 kB/s 2023-10-09T06:33:51.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:51.663 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1163 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:51.718 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:51.719 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:51.719 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9364e-05 s, 0.0 kB/s 2023-10-09T06:33:51.719 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:51.719 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1164 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:51.774 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:51.774 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:51.774 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1285e-05 s, 0.0 kB/s 2023-10-09T06:33:51.775 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:51.775 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1165 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:51.830 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:51.830 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:51.830 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0894e-05 s, 0.0 kB/s 2023-10-09T06:33:51.831 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:51.831 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1166 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:51.886 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:51.887 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:51.887 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0064e-05 s, 0.0 kB/s 2023-10-09T06:33:51.889 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:51.889 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1167 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:51.942 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:51.942 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:51.942 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8979e-05 s, 0.0 kB/s 2023-10-09T06:33:51.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:51.943 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1168 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:51.998 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:51.998 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:51.998 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1666e-05 s, 0.0 kB/s 2023-10-09T06:33:51.998 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:51.999 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1169 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:52.054 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:52.054 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:52.054 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0892e-05 s, 0.0 kB/s 2023-10-09T06:33:52.054 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:52.054 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1170 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:52.109 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:52.109 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:52.109 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7895e-05 s, 0.0 kB/s 2023-10-09T06:33:52.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:52.110 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1171 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:52.164 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:52.165 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:52.165 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8208e-05 s, 0.0 kB/s 2023-10-09T06:33:52.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:52.165 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1172 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:52.220 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:52.220 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:52.220 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9135e-05 s, 0.0 kB/s 2023-10-09T06:33:52.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:52.220 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1173 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:52.275 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:52.275 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:52.275 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.7945e-05 s, 0.0 kB/s 2023-10-09T06:33:52.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:52.276 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1174 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:52.331 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:52.332 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:52.332 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8244e-05 s, 0.0 kB/s 2023-10-09T06:33:52.332 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:52.332 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1175 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:52.387 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:52.387 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:52.388 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9374e-05 s, 0.0 kB/s 2023-10-09T06:33:52.388 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:52.388 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1176 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:52.443 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:52.443 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:52.443 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2072e-05 s, 0.0 kB/s 2023-10-09T06:33:52.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:52.444 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1177 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:52.498 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:52.499 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:52.499 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0085e-05 s, 0.0 kB/s 2023-10-09T06:33:52.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:52.499 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1178 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:52.554 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:52.554 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:52.555 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7835e-05 s, 0.0 kB/s 2023-10-09T06:33:52.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:52.555 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1179 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:52.610 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:52.610 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:52.610 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8449e-05 s, 0.0 kB/s 2023-10-09T06:33:52.610 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:52.610 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1180 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:52.664 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:52.665 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:52.665 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8242e-05 s, 0.0 kB/s 2023-10-09T06:33:52.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:52.665 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1181 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:52.720 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:52.720 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:52.720 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1728e-05 s, 0.0 kB/s 2023-10-09T06:33:52.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:52.721 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1182 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:52.776 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:52.776 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:52.776 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8096e-05 s, 0.0 kB/s 2023-10-09T06:33:52.777 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:52.777 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1183 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:52.832 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:52.832 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:52.832 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9431e-05 s, 0.0 kB/s 2023-10-09T06:33:52.832 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:52.833 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1184 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:52.887 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:52.888 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:52.888 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3133e-05 s, 0.0 kB/s 2023-10-09T06:33:52.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:52.888 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1185 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:52.943 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:52.943 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:52.943 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.5294e-05 s, 0.0 kB/s 2023-10-09T06:33:52.943 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:52.943 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1186 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:52.999 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:52.999 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:52.999 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9247e-05 s, 0.0 kB/s 2023-10-09T06:33:52.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:52.999 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1187 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:53.053 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:53.054 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:53.054 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1934e-05 s, 0.0 kB/s 2023-10-09T06:33:53.054 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:53.054 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1188 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:53.109 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:53.109 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:53.109 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9598e-05 s, 0.0 kB/s 2023-10-09T06:33:53.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:53.110 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1189 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:53.165 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:53.165 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:53.165 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.029e-05 s, 0.0 kB/s 2023-10-09T06:33:53.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:53.166 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1190 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:53.221 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:53.221 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:53.221 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.0351e-05 s, 0.0 kB/s 2023-10-09T06:33:53.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:53.222 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1191 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:53.277 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:53.277 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:53.277 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0522e-05 s, 0.0 kB/s 2023-10-09T06:33:53.277 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:53.277 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1192 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:53.333 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:53.333 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:53.333 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9259e-05 s, 0.0 kB/s 2023-10-09T06:33:53.333 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:53.334 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1193 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:53.388 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:53.389 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:53.389 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9064e-05 s, 0.0 kB/s 2023-10-09T06:33:53.390 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:53.390 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1194 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:53.445 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:53.445 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:53.445 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7867e-05 s, 0.0 kB/s 2023-10-09T06:33:53.445 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:53.445 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1195 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:53.500 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:53.500 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:53.500 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.8116e-05 s, 0.0 kB/s 2023-10-09T06:33:53.502 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:53.502 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1196 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:53.556 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:53.556 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:53.556 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8913e-05 s, 0.0 kB/s 2023-10-09T06:33:53.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:53.557 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1197 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:53.612 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:53.612 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:53.612 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4433e-05 s, 0.0 kB/s 2023-10-09T06:33:53.613 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:53.613 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1198 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:53.668 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:53.669 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:53.669 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8666e-05 s, 0.0 kB/s 2023-10-09T06:33:53.669 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:53.669 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1199 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:53.724 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:53.725 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:53.725 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0981e-05 s, 0.0 kB/s 2023-10-09T06:33:53.725 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:53.725 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1200 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:53.780 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:53.780 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:53.780 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7563e-05 s, 0.0 kB/s 2023-10-09T06:33:53.780 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:53.780 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1201 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:53.835 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:53.835 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:53.835 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0712e-05 s, 0.0 kB/s 2023-10-09T06:33:53.835 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:53.835 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1202 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:53.890 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:53.890 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:53.890 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8534e-05 s, 0.0 kB/s 2023-10-09T06:33:53.890 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:53.890 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1203 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:53.946 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:53.946 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:53.946 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8255e-05 s, 0.0 kB/s 2023-10-09T06:33:53.946 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:53.947 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1204 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:54.002 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:54.003 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:54.003 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9926e-05 s, 0.0 kB/s 2023-10-09T06:33:54.003 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:54.003 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1205 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:54.058 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:54.090 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:54.090 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8026e-05 s, 0.0 kB/s 2023-10-09T06:33:54.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:54.090 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1206 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:54.113 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:54.113 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:54.114 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7874e-05 s, 0.0 kB/s 2023-10-09T06:33:54.114 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:54.114 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1207 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:54.169 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:54.169 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:54.169 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.939e-05 s, 0.0 kB/s 2023-10-09T06:33:54.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:54.170 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1208 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:54.225 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:54.225 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:54.225 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8469e-05 s, 0.0 kB/s 2023-10-09T06:33:54.225 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:54.225 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1209 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:54.281 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:54.281 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:54.281 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9561e-05 s, 0.0 kB/s 2023-10-09T06:33:54.281 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:54.282 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1210 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:54.337 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:54.337 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:54.337 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0246e-05 s, 0.0 kB/s 2023-10-09T06:33:54.338 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:54.338 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1211 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:54.400 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:54.400 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:54.400 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3369e-05 s, 0.0 kB/s 2023-10-09T06:33:54.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:54.400 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1212 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:54.456 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:54.456 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:54.456 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.024e-05 s, 0.0 kB/s 2023-10-09T06:33:54.456 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:54.456 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1213 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:54.512 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:54.512 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:54.512 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7312e-05 s, 0.0 kB/s 2023-10-09T06:33:54.512 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:54.512 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1214 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:54.569 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:54.569 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:54.569 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3683e-05 s, 0.0 kB/s 2023-10-09T06:33:54.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:54.569 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1215 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:54.624 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:54.624 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:54.624 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0136e-05 s, 0.0 kB/s 2023-10-09T06:33:54.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:54.625 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1216 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:54.680 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:54.680 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:54.680 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.772e-05 s, 0.0 kB/s 2023-10-09T06:33:54.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:54.680 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1217 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:54.736 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:54.736 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:54.736 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1213e-05 s, 0.0 kB/s 2023-10-09T06:33:54.736 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:54.737 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1218 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:54.792 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:54.792 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:54.792 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.875e-05 s, 0.0 kB/s 2023-10-09T06:33:54.792 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:54.792 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1219 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:54.846 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:54.847 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:54.847 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0513e-05 s, 0.0 kB/s 2023-10-09T06:33:54.847 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:54.847 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1220 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:54.902 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:54.902 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:54.902 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4734e-05 s, 0.0 kB/s 2023-10-09T06:33:54.902 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:54.902 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1221 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:54.957 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:54.957 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:54.957 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9729e-05 s, 0.0 kB/s 2023-10-09T06:33:54.957 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:54.958 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1222 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:55.013 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:55.013 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:55.013 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7028e-05 s, 0.0 kB/s 2023-10-09T06:33:55.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:55.013 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1223 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:55.068 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:55.068 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:55.069 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9307e-05 s, 0.0 kB/s 2023-10-09T06:33:55.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:55.070 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1224 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:55.125 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:55.125 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:55.125 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0885e-05 s, 0.0 kB/s 2023-10-09T06:33:55.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:55.125 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1225 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:55.181 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:55.181 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:55.181 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.994e-05 s, 0.0 kB/s 2023-10-09T06:33:55.181 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:55.182 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1226 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:55.237 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:55.237 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:55.237 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9608e-05 s, 0.0 kB/s 2023-10-09T06:33:55.237 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:55.237 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1227 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:55.293 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:55.293 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:55.293 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8399e-05 s, 0.0 kB/s 2023-10-09T06:33:55.294 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:55.294 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1228 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:55.349 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:55.349 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:55.349 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2973e-05 s, 0.0 kB/s 2023-10-09T06:33:55.350 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:55.350 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1229 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:55.405 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:55.406 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:55.406 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0165e-05 s, 0.0 kB/s 2023-10-09T06:33:55.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:55.406 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1230 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:55.461 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:55.461 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:55.461 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8953e-05 s, 0.0 kB/s 2023-10-09T06:33:55.461 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:55.462 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1231 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:55.516 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:55.516 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:55.516 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7139e-05 s, 0.0 kB/s 2023-10-09T06:33:55.517 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:55.517 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1232 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:55.573 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:55.573 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:55.573 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4559e-05 s, 0.0 kB/s 2023-10-09T06:33:55.573 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:55.574 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1233 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:55.628 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:55.629 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:55.629 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2302e-05 s, 0.0 kB/s 2023-10-09T06:33:55.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:55.629 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1234 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:55.684 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:55.684 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:55.684 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.9928e-05 s, 0.0 kB/s 2023-10-09T06:33:55.685 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:55.685 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1235 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:55.741 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:55.741 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:55.741 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2451e-05 s, 0.0 kB/s 2023-10-09T06:33:55.741 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:55.741 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1236 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:55.797 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:55.798 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:55.798 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1942e-05 s, 0.0 kB/s 2023-10-09T06:33:55.798 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:55.798 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1237 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:55.853 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:55.853 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:55.854 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.262e-05 s, 0.0 kB/s 2023-10-09T06:33:55.854 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:55.854 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1238 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:55.909 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:55.910 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:55.910 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0111e-05 s, 0.0 kB/s 2023-10-09T06:33:55.910 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:55.910 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1239 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:55.965 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:55.965 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:55.965 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.214e-05 s, 0.0 kB/s 2023-10-09T06:33:55.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:55.966 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1240 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:56.021 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:56.021 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:56.021 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0599e-05 s, 0.0 kB/s 2023-10-09T06:33:56.021 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:56.022 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1241 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:56.077 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:56.077 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:56.078 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1431e-05 s, 0.0 kB/s 2023-10-09T06:33:56.078 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:56.078 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1242 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:56.133 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:56.133 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:56.133 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9946e-05 s, 0.0 kB/s 2023-10-09T06:33:56.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:56.133 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1243 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:56.187 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:56.188 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:56.188 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0256e-05 s, 0.0 kB/s 2023-10-09T06:33:56.188 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:56.188 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1244 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:56.243 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:56.243 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:56.243 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0315e-05 s, 0.0 kB/s 2023-10-09T06:33:56.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:56.243 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1245 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:56.299 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:56.299 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:56.299 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2523e-05 s, 0.0 kB/s 2023-10-09T06:33:56.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:56.299 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1246 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:56.354 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:56.354 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:56.355 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9964e-05 s, 0.0 kB/s 2023-10-09T06:33:56.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:56.357 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1247 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:56.412 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:56.412 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:56.412 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0199e-05 s, 0.0 kB/s 2023-10-09T06:33:56.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:56.412 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1248 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:56.467 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:56.467 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:56.467 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8643e-05 s, 0.0 kB/s 2023-10-09T06:33:56.467 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:56.468 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1249 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:56.523 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:56.523 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:56.523 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0787e-05 s, 0.0 kB/s 2023-10-09T06:33:56.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:56.524 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1250 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:56.579 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:56.579 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:56.579 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2223e-05 s, 0.0 kB/s 2023-10-09T06:33:56.580 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:56.580 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1251 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:56.634 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:56.634 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:56.635 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.979e-05 s, 0.0 kB/s 2023-10-09T06:33:56.635 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:56.636 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1252 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:56.691 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:56.692 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:56.692 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2074e-05 s, 0.0 kB/s 2023-10-09T06:33:56.692 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:56.692 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1253 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:56.748 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:56.748 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:56.748 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.6498e-05 s, 0.0 kB/s 2023-10-09T06:33:56.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:56.748 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1254 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:56.803 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:56.804 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:56.804 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9847e-05 s, 0.0 kB/s 2023-10-09T06:33:56.804 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:56.804 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1255 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:56.858 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:56.859 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:56.859 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0254e-05 s, 0.0 kB/s 2023-10-09T06:33:56.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:56.860 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1256 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:56.915 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:56.915 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:56.915 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.922e-05 s, 0.0 kB/s 2023-10-09T06:33:56.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:56.916 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1257 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:56.971 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:56.971 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:56.971 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2315e-05 s, 0.0 kB/s 2023-10-09T06:33:56.972 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:56.972 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1258 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:57.027 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:57.027 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:57.027 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1372e-05 s, 0.0 kB/s 2023-10-09T06:33:57.027 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:57.027 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1259 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:57.083 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:57.083 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:57.083 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9216e-05 s, 0.0 kB/s 2023-10-09T06:33:57.083 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:57.084 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1260 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:57.139 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:57.139 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:57.139 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0133e-05 s, 0.0 kB/s 2023-10-09T06:33:57.140 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:57.140 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1261 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:57.196 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:57.196 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:57.196 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9701e-05 s, 0.0 kB/s 2023-10-09T06:33:57.197 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:57.197 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1262 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:57.252 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:57.252 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:57.253 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8928e-05 s, 0.0 kB/s 2023-10-09T06:33:57.253 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:57.253 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1263 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:57.308 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:57.308 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:57.308 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.915e-05 s, 0.0 kB/s 2023-10-09T06:33:57.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:57.308 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1264 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:57.364 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:57.364 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:57.364 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2246e-05 s, 0.0 kB/s 2023-10-09T06:33:57.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:57.364 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1265 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:57.421 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:57.421 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:57.422 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2324e-05 s, 0.0 kB/s 2023-10-09T06:33:57.422 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:57.422 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1266 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:57.477 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:57.477 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:57.477 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1611e-05 s, 0.0 kB/s 2023-10-09T06:33:57.478 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:57.478 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1267 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:57.533 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:57.533 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:57.534 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.966e-05 s, 0.0 kB/s 2023-10-09T06:33:57.534 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:57.534 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1268 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:57.590 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:57.591 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:57.591 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3281e-05 s, 0.0 kB/s 2023-10-09T06:33:57.591 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:57.591 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1269 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:57.646 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:57.647 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:57.647 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0848e-05 s, 0.0 kB/s 2023-10-09T06:33:57.647 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:57.647 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1270 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:57.702 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:57.702 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:57.702 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0734e-05 s, 0.0 kB/s 2023-10-09T06:33:57.703 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:57.703 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1271 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:57.758 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:57.759 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:57.759 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0034e-05 s, 0.0 kB/s 2023-10-09T06:33:57.759 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:57.759 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1272 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:57.814 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:57.815 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:57.815 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1613e-05 s, 0.0 kB/s 2023-10-09T06:33:57.815 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:57.815 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1273 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:57.870 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:57.870 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:57.870 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.016e-05 s, 0.0 kB/s 2023-10-09T06:33:57.871 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:57.871 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1274 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:57.926 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:57.926 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:57.926 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9248e-05 s, 0.0 kB/s 2023-10-09T06:33:57.927 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:57.927 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1275 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:57.982 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:57.983 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:57.983 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8645e-05 s, 0.0 kB/s 2023-10-09T06:33:57.983 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:57.983 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1276 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:58.038 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:58.038 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:58.038 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3769e-05 s, 0.0 kB/s 2023-10-09T06:33:58.039 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:58.039 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1277 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:58.094 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:58.094 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:58.094 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0895e-05 s, 0.0 kB/s 2023-10-09T06:33:58.094 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:58.094 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1278 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:58.150 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:58.150 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:58.150 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0821e-05 s, 0.0 kB/s 2023-10-09T06:33:58.151 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:58.151 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1279 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:58.206 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:58.206 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:58.207 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1754e-05 s, 0.0 kB/s 2023-10-09T06:33:58.207 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:58.207 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1280 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:58.262 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:58.262 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:58.262 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9605e-05 s, 0.0 kB/s 2023-10-09T06:33:58.262 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:58.262 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1281 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:58.318 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:58.318 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:58.318 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9279e-05 s, 0.0 kB/s 2023-10-09T06:33:58.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:58.319 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1282 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:58.374 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:58.374 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:58.374 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.069e-05 s, 0.0 kB/s 2023-10-09T06:33:58.374 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:58.374 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1283 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:58.428 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:58.429 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:58.429 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1627e-05 s, 0.0 kB/s 2023-10-09T06:33:58.430 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:58.430 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1284 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:58.484 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:58.485 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:58.485 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0338e-05 s, 0.0 kB/s 2023-10-09T06:33:58.485 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:58.485 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1285 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:58.541 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:58.541 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:58.541 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9658e-05 s, 0.0 kB/s 2023-10-09T06:33:58.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:58.541 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1286 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:58.596 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:58.596 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:58.596 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2507e-05 s, 0.0 kB/s 2023-10-09T06:33:58.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:58.596 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1287 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:58.652 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:58.652 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:58.652 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1498e-05 s, 0.0 kB/s 2023-10-09T06:33:58.653 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:58.653 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1288 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:58.709 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:58.709 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:58.709 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.165e-05 s, 0.0 kB/s 2023-10-09T06:33:58.709 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:58.710 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1289 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:58.765 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:58.765 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:58.766 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1628e-05 s, 0.0 kB/s 2023-10-09T06:33:58.766 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:58.766 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1290 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:58.822 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:58.822 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:58.822 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.215e-05 s, 0.0 kB/s 2023-10-09T06:33:58.823 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:58.823 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1291 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:58.878 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:58.879 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:58.879 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9745e-05 s, 0.0 kB/s 2023-10-09T06:33:58.879 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:58.879 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1292 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:58.934 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:58.934 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:58.935 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.019e-05 s, 0.0 kB/s 2023-10-09T06:33:58.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:58.935 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1293 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:58.990 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:58.991 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:58.991 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0791e-05 s, 0.0 kB/s 2023-10-09T06:33:58.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:58.992 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1294 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:59.046 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:59.046 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:59.046 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9946e-05 s, 0.0 kB/s 2023-10-09T06:33:59.046 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:59.046 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1295 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:59.102 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:59.102 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:59.102 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0388e-05 s, 0.0 kB/s 2023-10-09T06:33:59.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:59.103 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1296 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:59.158 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:59.158 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:59.158 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7599e-05 s, 0.0 kB/s 2023-10-09T06:33:59.158 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:59.159 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1297 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:59.219 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:59.220 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:59.220 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9998e-05 s, 0.0 kB/s 2023-10-09T06:33:59.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:59.221 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1298 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:59.286 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:59.286 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:59.286 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9674e-05 s, 0.0 kB/s 2023-10-09T06:33:59.287 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:59.288 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1299 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:59.358 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:59.359 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:59.359 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4381e-05 s, 0.0 kB/s 2023-10-09T06:33:59.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:59.359 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1300 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:59.418 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:59.418 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:59.418 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0318e-05 s, 0.0 kB/s 2023-10-09T06:33:59.418 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:59.419 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1301 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:59.476 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:59.476 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:59.476 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9687e-05 s, 0.0 kB/s 2023-10-09T06:33:59.476 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:59.477 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1302 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:59.533 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:59.533 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:59.533 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9497e-05 s, 0.0 kB/s 2023-10-09T06:33:59.533 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:59.533 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1303 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:59.590 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:59.590 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:59.590 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3253e-05 s, 0.0 kB/s 2023-10-09T06:33:59.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:59.591 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1304 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:59.688 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:59.688 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:59.688 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 5.3095e-05 s, 0.0 kB/s 2023-10-09T06:33:59.688 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:59.689 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1305 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:59.705 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:59.705 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:59.705 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8537e-05 s, 0.0 kB/s 2023-10-09T06:33:59.706 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:59.706 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1306 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:59.761 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:59.762 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:59.762 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1876e-05 s, 0.0 kB/s 2023-10-09T06:33:59.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:59.762 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1307 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:59.817 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:59.817 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:59.817 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1403e-05 s, 0.0 kB/s 2023-10-09T06:33:59.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:59.818 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1308 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:59.875 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:59.875 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:59.876 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.6038e-05 s, 0.0 kB/s 2023-10-09T06:33:59.876 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:59.876 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1309 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:59.931 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:59.932 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:59.932 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1704e-05 s, 0.0 kB/s 2023-10-09T06:33:59.932 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:59.932 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1310 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:33:59.987 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:33:59.988 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:33:59.988 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0309e-05 s, 0.0 kB/s 2023-10-09T06:33:59.988 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:33:59.988 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1311 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:00.049 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:00.049 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:00.049 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.5172e-05 s, 0.0 kB/s 2023-10-09T06:34:00.050 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:00.050 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1312 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:00.105 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:00.106 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:00.106 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9804e-05 s, 0.0 kB/s 2023-10-09T06:34:00.106 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:00.107 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1313 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:00.161 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:00.161 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:00.161 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9126e-05 s, 0.0 kB/s 2023-10-09T06:34:00.161 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:00.162 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1314 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:00.217 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:00.217 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:00.217 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9527e-05 s, 0.0 kB/s 2023-10-09T06:34:00.217 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:00.217 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1315 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:00.273 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:00.273 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:00.273 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8899e-05 s, 0.0 kB/s 2023-10-09T06:34:00.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:00.273 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1316 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:00.329 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:00.329 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:00.329 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.795e-05 s, 0.0 kB/s 2023-10-09T06:34:00.329 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:00.329 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1317 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:00.385 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:00.386 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:00.386 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0663e-05 s, 0.0 kB/s 2023-10-09T06:34:00.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:00.386 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1318 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:00.441 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:00.441 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:00.441 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0192e-05 s, 0.0 kB/s 2023-10-09T06:34:00.441 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:00.441 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1319 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:00.498 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:00.498 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:00.498 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0631e-05 s, 0.0 kB/s 2023-10-09T06:34:00.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:00.499 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1320 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:00.556 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:00.556 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:00.556 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7856e-05 s, 0.0 kB/s 2023-10-09T06:34:00.556 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:00.556 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1321 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:00.612 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:00.612 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:00.612 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9745e-05 s, 0.0 kB/s 2023-10-09T06:34:00.612 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:00.612 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1322 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:00.668 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:00.668 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:00.668 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7677e-05 s, 0.0 kB/s 2023-10-09T06:34:00.669 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:00.669 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1323 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:00.725 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:00.725 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:00.726 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9561e-05 s, 0.0 kB/s 2023-10-09T06:34:00.726 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:00.726 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1324 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:00.781 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:00.781 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:00.781 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.183e-05 s, 0.0 kB/s 2023-10-09T06:34:00.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:00.782 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1325 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:00.837 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:00.837 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:00.838 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7921e-05 s, 0.0 kB/s 2023-10-09T06:34:00.838 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:00.838 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1326 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:00.893 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:00.893 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:00.893 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9359e-05 s, 0.0 kB/s 2023-10-09T06:34:00.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:00.894 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1327 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:00.949 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:00.950 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:00.950 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9409e-05 s, 0.0 kB/s 2023-10-09T06:34:00.950 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:00.951 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1328 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:01.005 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:01.005 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:01.005 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.973e-05 s, 0.0 kB/s 2023-10-09T06:34:01.005 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:01.006 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1329 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:01.063 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:01.063 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:01.063 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0742e-05 s, 0.0 kB/s 2023-10-09T06:34:01.063 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:01.064 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1330 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:01.119 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:01.120 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:01.120 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2918e-05 s, 0.0 kB/s 2023-10-09T06:34:01.120 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:01.120 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1331 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:01.174 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:01.174 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:01.175 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.4514e-05 s, 0.0 kB/s 2023-10-09T06:34:01.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:01.176 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1332 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:01.231 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:01.231 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:01.231 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.4586e-05 s, 0.0 kB/s 2023-10-09T06:34:01.231 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:01.232 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1333 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:01.287 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:01.287 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:01.287 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0565e-05 s, 0.0 kB/s 2023-10-09T06:34:01.288 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:01.288 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1334 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:01.343 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:01.343 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:01.343 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7456e-05 s, 0.0 kB/s 2023-10-09T06:34:01.344 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:01.344 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1335 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:01.398 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:01.399 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:01.399 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8171e-05 s, 0.0 kB/s 2023-10-09T06:34:01.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:01.400 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1336 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:01.455 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:01.455 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:01.455 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0793e-05 s, 0.0 kB/s 2023-10-09T06:34:01.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:01.456 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1337 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:01.512 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:01.512 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:01.512 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1046e-05 s, 0.0 kB/s 2023-10-09T06:34:01.512 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:01.512 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1338 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:01.568 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:01.568 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:01.568 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2064e-05 s, 0.0 kB/s 2023-10-09T06:34:01.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:01.569 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1339 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:01.625 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:01.625 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:01.625 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0172e-05 s, 0.0 kB/s 2023-10-09T06:34:01.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:01.625 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1340 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:01.681 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:01.682 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:01.682 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0038e-05 s, 0.0 kB/s 2023-10-09T06:34:01.682 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:01.682 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1341 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:01.738 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:01.738 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:01.738 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2474e-05 s, 0.0 kB/s 2023-10-09T06:34:01.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:01.739 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1342 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:01.793 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:01.793 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:01.794 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0867e-05 s, 0.0 kB/s 2023-10-09T06:34:01.794 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:01.794 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1343 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:01.849 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:01.850 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:01.850 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1038e-05 s, 0.0 kB/s 2023-10-09T06:34:01.850 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:01.851 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1344 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:01.905 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:01.905 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:01.905 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0298e-05 s, 0.0 kB/s 2023-10-09T06:34:01.906 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:01.906 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1345 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:01.961 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:01.961 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:01.961 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2965e-05 s, 0.0 kB/s 2023-10-09T06:34:01.961 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:01.962 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1346 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:02.017 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:02.017 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:02.018 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0051e-05 s, 0.0 kB/s 2023-10-09T06:34:02.018 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:02.018 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1347 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:02.073 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:02.074 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:02.074 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0138e-05 s, 0.0 kB/s 2023-10-09T06:34:02.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:02.074 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1348 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:02.129 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:02.129 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:02.129 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3006e-05 s, 0.0 kB/s 2023-10-09T06:34:02.130 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:02.130 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1349 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:02.185 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:02.185 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:02.185 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3264e-05 s, 0.0 kB/s 2023-10-09T06:34:02.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:02.186 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1350 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:02.241 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:02.241 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:02.241 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1487e-05 s, 0.0 kB/s 2023-10-09T06:34:02.241 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:02.242 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1351 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:02.298 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:02.299 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:02.299 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1759e-05 s, 0.0 kB/s 2023-10-09T06:34:02.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:02.299 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1352 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:02.354 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:02.355 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:02.355 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1734e-05 s, 0.0 kB/s 2023-10-09T06:34:02.355 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:02.355 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1353 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:02.410 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:02.410 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:02.410 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2429e-05 s, 0.0 kB/s 2023-10-09T06:34:02.410 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:02.410 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1354 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:02.467 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:02.467 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:02.467 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0862e-05 s, 0.0 kB/s 2023-10-09T06:34:02.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:02.468 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1355 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:02.524 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:02.524 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:02.524 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1633e-05 s, 0.0 kB/s 2023-10-09T06:34:02.524 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:02.524 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1356 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:02.580 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:02.580 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:02.580 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0641e-05 s, 0.0 kB/s 2023-10-09T06:34:02.580 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:02.581 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1357 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:02.636 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:02.636 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:02.636 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1311e-05 s, 0.0 kB/s 2023-10-09T06:34:02.636 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:02.637 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1358 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:02.692 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:02.692 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:02.692 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1731e-05 s, 0.0 kB/s 2023-10-09T06:34:02.692 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:02.692 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1359 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:02.747 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:02.747 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:02.748 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3824e-05 s, 0.0 kB/s 2023-10-09T06:34:02.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:02.748 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1360 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:02.804 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:02.804 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:02.804 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0765e-05 s, 0.0 kB/s 2023-10-09T06:34:02.804 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:02.805 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1361 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:02.859 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:02.860 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:02.860 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2975e-05 s, 0.0 kB/s 2023-10-09T06:34:02.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:02.860 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1362 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:02.915 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:02.915 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:02.915 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2294e-05 s, 0.0 kB/s 2023-10-09T06:34:02.917 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:02.917 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1363 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:02.971 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:02.971 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:02.971 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9404e-05 s, 0.0 kB/s 2023-10-09T06:34:02.972 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:02.972 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1364 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:03.027 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:03.027 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:03.027 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2768e-05 s, 0.0 kB/s 2023-10-09T06:34:03.028 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:03.028 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1365 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:03.083 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:03.083 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:03.083 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8607e-05 s, 0.0 kB/s 2023-10-09T06:34:03.083 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:03.084 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1366 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:03.140 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:03.141 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:03.141 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1848e-05 s, 0.0 kB/s 2023-10-09T06:34:03.141 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:03.141 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1367 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:03.196 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:03.196 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:03.196 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9088e-05 s, 0.0 kB/s 2023-10-09T06:34:03.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:03.197 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1368 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:03.252 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:03.253 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:03.253 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0126e-05 s, 0.0 kB/s 2023-10-09T06:34:03.253 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:03.253 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1369 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:03.308 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:03.309 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:03.309 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3579e-05 s, 0.0 kB/s 2023-10-09T06:34:03.309 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:03.309 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1370 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:03.364 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:03.364 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:03.364 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1988e-05 s, 0.0 kB/s 2023-10-09T06:34:03.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:03.365 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1371 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:03.419 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:03.420 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:03.420 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8908e-05 s, 0.0 kB/s 2023-10-09T06:34:03.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:03.420 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1372 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:03.475 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:03.475 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:03.475 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8467e-05 s, 0.0 kB/s 2023-10-09T06:34:03.475 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:03.476 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1373 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:03.532 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:03.532 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:03.532 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9656e-05 s, 0.0 kB/s 2023-10-09T06:34:03.532 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:03.532 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1374 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:03.587 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:03.588 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:03.588 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2956e-05 s, 0.0 kB/s 2023-10-09T06:34:03.588 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:03.588 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1375 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:03.644 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:03.644 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:03.644 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8649e-05 s, 0.0 kB/s 2023-10-09T06:34:03.644 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:03.645 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1376 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:03.698 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:03.699 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:03.699 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8937e-05 s, 0.0 kB/s 2023-10-09T06:34:03.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:03.699 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1377 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:03.754 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:03.754 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:03.755 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2258e-05 s, 0.0 kB/s 2023-10-09T06:34:03.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:03.755 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1378 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:03.810 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:03.810 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:03.810 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1403e-05 s, 0.0 kB/s 2023-10-09T06:34:03.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:03.811 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1379 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:03.866 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:03.866 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:03.866 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.103e-05 s, 0.0 kB/s 2023-10-09T06:34:03.866 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:03.867 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1380 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:03.922 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:03.922 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:03.923 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2128e-05 s, 0.0 kB/s 2023-10-09T06:34:03.923 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:03.923 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1381 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:03.978 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:03.978 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:03.978 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0569e-05 s, 0.0 kB/s 2023-10-09T06:34:03.978 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:03.979 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1382 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:04.034 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:04.034 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:04.034 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.785e-05 s, 0.0 kB/s 2023-10-09T06:34:04.035 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:04.035 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1383 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:04.090 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:04.090 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:04.090 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2062e-05 s, 0.0 kB/s 2023-10-09T06:34:04.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:04.090 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1384 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:04.145 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:04.145 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:04.145 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.823e-05 s, 0.0 kB/s 2023-10-09T06:34:04.145 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:04.145 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1385 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:04.201 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:04.201 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:04.201 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0967e-05 s, 0.0 kB/s 2023-10-09T06:34:04.202 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:04.202 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1386 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:04.257 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:04.257 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:04.257 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1541e-05 s, 0.0 kB/s 2023-10-09T06:34:04.257 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:04.257 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1387 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:04.313 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:04.313 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:04.313 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9359e-05 s, 0.0 kB/s 2023-10-09T06:34:04.314 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:04.314 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1388 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:04.368 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:04.368 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:04.368 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8704e-05 s, 0.0 kB/s 2023-10-09T06:34:04.369 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:04.369 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1389 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:04.423 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:04.424 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:04.424 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0435e-05 s, 0.0 kB/s 2023-10-09T06:34:04.424 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:04.424 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1390 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:04.479 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:04.479 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:04.479 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1152e-05 s, 0.0 kB/s 2023-10-09T06:34:04.479 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:04.479 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1391 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:04.535 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:04.535 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:04.535 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2487e-05 s, 0.0 kB/s 2023-10-09T06:34:04.536 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:04.536 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1392 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:04.591 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:04.591 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:04.591 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7443e-05 s, 0.0 kB/s 2023-10-09T06:34:04.591 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:04.591 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1393 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:04.646 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:04.646 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:04.646 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.5418e-05 s, 0.0 kB/s 2023-10-09T06:34:04.647 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:04.647 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1394 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:04.702 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:04.702 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:04.702 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7678e-05 s, 0.0 kB/s 2023-10-09T06:34:04.703 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:04.703 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1395 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:04.758 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:04.759 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:04.759 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.1844e-05 s, 0.0 kB/s 2023-10-09T06:34:04.759 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:04.759 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1396 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:04.814 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:04.814 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:04.814 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0849e-05 s, 0.0 kB/s 2023-10-09T06:34:04.815 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:04.815 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1397 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:04.870 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:04.870 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:04.870 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0293e-05 s, 0.0 kB/s 2023-10-09T06:34:04.871 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:04.871 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1398 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:04.926 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:04.926 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:04.926 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1525e-05 s, 0.0 kB/s 2023-10-09T06:34:04.927 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:04.927 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1399 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:04.982 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:04.982 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:04.982 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2871e-05 s, 0.0 kB/s 2023-10-09T06:34:04.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:04.982 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1400 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:05.038 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:05.038 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:05.038 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3234e-05 s, 0.0 kB/s 2023-10-09T06:34:05.038 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:05.038 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1401 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:05.093 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:05.093 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:05.093 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8814e-05 s, 0.0 kB/s 2023-10-09T06:34:05.093 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:05.094 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1402 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:05.149 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:05.149 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:05.149 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0635e-05 s, 0.0 kB/s 2023-10-09T06:34:05.149 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:05.150 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1403 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:05.205 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:05.205 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:05.205 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9442e-05 s, 0.0 kB/s 2023-10-09T06:34:05.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:05.205 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1404 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:05.260 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:05.260 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:05.260 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9004e-05 s, 0.0 kB/s 2023-10-09T06:34:05.260 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:05.261 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1405 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:05.316 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:05.316 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:05.316 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8056e-05 s, 0.0 kB/s 2023-10-09T06:34:05.317 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:05.317 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1406 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:05.372 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:05.372 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:05.372 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9285e-05 s, 0.0 kB/s 2023-10-09T06:34:05.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:05.372 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1407 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:05.428 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:05.428 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:05.428 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1456e-05 s, 0.0 kB/s 2023-10-09T06:34:05.428 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:05.429 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1408 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:05.484 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:05.484 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:05.484 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9675e-05 s, 0.0 kB/s 2023-10-09T06:34:05.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:05.484 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1409 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:05.539 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:05.540 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:05.540 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9032e-05 s, 0.0 kB/s 2023-10-09T06:34:05.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:05.540 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1410 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:05.596 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:05.597 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:05.597 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8418e-05 s, 0.0 kB/s 2023-10-09T06:34:05.597 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:05.597 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1411 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:05.652 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:05.652 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:05.652 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.4766e-05 s, 0.0 kB/s 2023-10-09T06:34:05.653 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:05.653 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1412 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:05.708 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:05.708 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:05.708 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0832e-05 s, 0.0 kB/s 2023-10-09T06:34:05.708 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:05.709 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1413 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:05.764 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:05.765 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:05.765 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9451e-05 s, 0.0 kB/s 2023-10-09T06:34:05.765 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:05.765 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1414 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:05.820 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:05.820 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:05.821 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9279e-05 s, 0.0 kB/s 2023-10-09T06:34:05.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:05.821 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1415 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:05.876 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:05.876 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:05.877 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2213e-05 s, 0.0 kB/s 2023-10-09T06:34:05.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:05.877 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1416 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:05.932 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:05.932 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:05.932 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9826e-05 s, 0.0 kB/s 2023-10-09T06:34:05.932 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:05.933 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1417 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:05.988 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:05.988 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:05.988 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9702e-05 s, 0.0 kB/s 2023-10-09T06:34:05.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:05.989 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1418 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:06.045 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:06.045 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:06.045 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3346e-05 s, 0.0 kB/s 2023-10-09T06:34:06.045 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:06.046 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1419 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:06.101 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:06.101 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:06.102 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9714e-05 s, 0.0 kB/s 2023-10-09T06:34:06.102 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:06.102 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1420 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:06.157 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:06.157 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:06.157 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8142e-05 s, 0.0 kB/s 2023-10-09T06:34:06.158 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:06.158 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1421 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:06.213 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:06.213 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:06.213 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.9998e-05 s, 0.0 kB/s 2023-10-09T06:34:06.213 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:06.214 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1422 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:06.269 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:06.269 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:06.269 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0889e-05 s, 0.0 kB/s 2023-10-09T06:34:06.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:06.270 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1423 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:06.325 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:06.325 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:06.325 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0474e-05 s, 0.0 kB/s 2023-10-09T06:34:06.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:06.326 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1424 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:06.381 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:06.381 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:06.381 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9474e-05 s, 0.0 kB/s 2023-10-09T06:34:06.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:06.382 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1425 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:06.436 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:06.436 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:06.437 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1311e-05 s, 0.0 kB/s 2023-10-09T06:34:06.438 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:06.438 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1426 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:06.493 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:06.493 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:06.493 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9001e-05 s, 0.0 kB/s 2023-10-09T06:34:06.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:06.494 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1427 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:06.549 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:06.550 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:06.550 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2877e-05 s, 0.0 kB/s 2023-10-09T06:34:06.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:06.550 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1428 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:06.606 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:06.607 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:06.607 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9195e-05 s, 0.0 kB/s 2023-10-09T06:34:06.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:06.607 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1429 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:06.661 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:06.661 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:06.661 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0795e-05 s, 0.0 kB/s 2023-10-09T06:34:06.662 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:06.663 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1430 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:06.718 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:06.718 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:06.718 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1175e-05 s, 0.0 kB/s 2023-10-09T06:34:06.719 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:06.719 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1431 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:06.774 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:06.774 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:06.775 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7998e-05 s, 0.0 kB/s 2023-10-09T06:34:06.775 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:06.775 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1432 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:06.830 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:06.830 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:06.830 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8303e-05 s, 0.0 kB/s 2023-10-09T06:34:06.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:06.830 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1433 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:06.886 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:06.886 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:06.886 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9064e-05 s, 0.0 kB/s 2023-10-09T06:34:06.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:06.886 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1434 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:06.941 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:06.941 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:06.941 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2169e-05 s, 0.0 kB/s 2023-10-09T06:34:06.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:06.942 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1435 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:06.997 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:06.997 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:06.997 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9218e-05 s, 0.0 kB/s 2023-10-09T06:34:06.998 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:06.998 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1436 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:07.053 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:07.053 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:07.053 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8395e-05 s, 0.0 kB/s 2023-10-09T06:34:07.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:07.054 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1437 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:07.109 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:07.109 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:07.109 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.915e-05 s, 0.0 kB/s 2023-10-09T06:34:07.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:07.110 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1438 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:07.165 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:07.165 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:07.165 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8989e-05 s, 0.0 kB/s 2023-10-09T06:34:07.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:07.165 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1439 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:07.220 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:07.220 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:07.220 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0158e-05 s, 0.0 kB/s 2023-10-09T06:34:07.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:07.221 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1440 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:07.276 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:07.276 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:07.276 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8944e-05 s, 0.0 kB/s 2023-10-09T06:34:07.276 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:07.277 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1441 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:07.332 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:07.332 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:07.333 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0532e-05 s, 0.0 kB/s 2023-10-09T06:34:07.333 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:07.333 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1442 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:07.388 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:07.388 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:07.388 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2358e-05 s, 0.0 kB/s 2023-10-09T06:34:07.389 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:07.389 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1443 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:07.444 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:07.444 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:07.444 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.914e-05 s, 0.0 kB/s 2023-10-09T06:34:07.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:07.444 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1444 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:07.499 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:07.499 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:07.499 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9884e-05 s, 0.0 kB/s 2023-10-09T06:34:07.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:07.500 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1445 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:07.556 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:07.557 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:07.557 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9845e-05 s, 0.0 kB/s 2023-10-09T06:34:07.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:07.557 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1446 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:07.612 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:07.612 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:07.612 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8866e-05 s, 0.0 kB/s 2023-10-09T06:34:07.613 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:07.613 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1447 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:07.669 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:07.669 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:07.669 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1131e-05 s, 0.0 kB/s 2023-10-09T06:34:07.670 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:07.670 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1448 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:07.725 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:07.725 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:07.726 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7893e-05 s, 0.0 kB/s 2023-10-09T06:34:07.726 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:07.726 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1449 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:07.781 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:07.781 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:07.782 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9533e-05 s, 0.0 kB/s 2023-10-09T06:34:07.782 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:07.782 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1450 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:07.837 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:07.837 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:07.837 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2808e-05 s, 0.0 kB/s 2023-10-09T06:34:07.837 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:07.837 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1451 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:07.893 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:07.893 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:07.893 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0725e-05 s, 0.0 kB/s 2023-10-09T06:34:07.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:07.894 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1452 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:07.948 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:07.949 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:07.949 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8153e-05 s, 0.0 kB/s 2023-10-09T06:34:07.949 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:07.949 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1453 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:08.004 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:08.004 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:08.004 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2283e-05 s, 0.0 kB/s 2023-10-09T06:34:08.004 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:08.004 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1454 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:08.059 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:08.059 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:08.059 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0984e-05 s, 0.0 kB/s 2023-10-09T06:34:08.059 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:08.059 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1455 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:08.115 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:08.115 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:08.115 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9468e-05 s, 0.0 kB/s 2023-10-09T06:34:08.115 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:08.115 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1456 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:08.170 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:08.170 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:08.170 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2107e-05 s, 0.0 kB/s 2023-10-09T06:34:08.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:08.170 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1457 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:08.225 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:08.225 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:08.225 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2968e-05 s, 0.0 kB/s 2023-10-09T06:34:08.225 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:08.225 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1458 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:08.281 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:08.281 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:08.281 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.938e-05 s, 0.0 kB/s 2023-10-09T06:34:08.282 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:08.282 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1459 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:08.337 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:08.337 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:08.337 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9345e-05 s, 0.0 kB/s 2023-10-09T06:34:08.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:08.338 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1460 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:08.393 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:08.393 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:08.393 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9897e-05 s, 0.0 kB/s 2023-10-09T06:34:08.393 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:08.393 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1461 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:08.448 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:08.448 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:08.448 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8107e-05 s, 0.0 kB/s 2023-10-09T06:34:08.449 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:08.449 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1462 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:08.504 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:08.504 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:08.504 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.062e-05 s, 0.0 kB/s 2023-10-09T06:34:08.504 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:08.505 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1463 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:08.561 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:08.561 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:08.561 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2479e-05 s, 0.0 kB/s 2023-10-09T06:34:08.561 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:08.561 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1464 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:08.616 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:08.616 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:08.616 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0232e-05 s, 0.0 kB/s 2023-10-09T06:34:08.616 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:08.616 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1465 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:08.672 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:08.672 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:08.672 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8603e-05 s, 0.0 kB/s 2023-10-09T06:34:08.672 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:08.672 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1466 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:08.726 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:08.727 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:08.727 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3406e-05 s, 0.0 kB/s 2023-10-09T06:34:08.727 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:08.727 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1467 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:08.782 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:08.782 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:08.782 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8614e-05 s, 0.0 kB/s 2023-10-09T06:34:08.782 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:08.782 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1468 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:08.838 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:08.838 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:08.838 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1131e-05 s, 0.0 kB/s 2023-10-09T06:34:08.839 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:08.839 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1469 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:08.894 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:08.894 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:08.894 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9699e-05 s, 0.0 kB/s 2023-10-09T06:34:08.894 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:08.895 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1470 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:08.949 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:08.950 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:08.950 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1825e-05 s, 0.0 kB/s 2023-10-09T06:34:08.950 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:08.950 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1471 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:09.005 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:09.005 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:09.005 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7108e-05 s, 0.0 kB/s 2023-10-09T06:34:09.005 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:09.005 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1472 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:09.061 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:09.061 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:09.061 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0755e-05 s, 0.0 kB/s 2023-10-09T06:34:09.061 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:09.062 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1473 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:09.117 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:09.117 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:09.117 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9259e-05 s, 0.0 kB/s 2023-10-09T06:34:09.117 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:09.118 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1474 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:09.173 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:09.173 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:09.173 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1709e-05 s, 0.0 kB/s 2023-10-09T06:34:09.173 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:09.173 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1475 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:09.227 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:09.227 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:09.228 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7989e-05 s, 0.0 kB/s 2023-10-09T06:34:09.228 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:09.228 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1476 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:09.283 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:09.283 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:09.283 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9185e-05 s, 0.0 kB/s 2023-10-09T06:34:09.283 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:09.283 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1477 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:09.339 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:09.339 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:09.339 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.6403e-05 s, 0.0 kB/s 2023-10-09T06:34:09.339 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:09.339 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1478 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:09.393 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:09.394 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:09.394 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0884e-05 s, 0.0 kB/s 2023-10-09T06:34:09.394 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:09.394 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1479 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:09.449 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:09.449 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:09.449 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9847e-05 s, 0.0 kB/s 2023-10-09T06:34:09.449 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:09.449 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1480 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:09.504 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:09.504 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:09.504 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1551e-05 s, 0.0 kB/s 2023-10-09T06:34:09.504 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:09.504 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1481 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:09.559 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:09.559 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:09.559 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9465e-05 s, 0.0 kB/s 2023-10-09T06:34:09.559 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:09.559 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1482 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:09.615 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:09.615 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:09.615 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2533e-05 s, 0.0 kB/s 2023-10-09T06:34:09.615 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:09.616 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1483 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:09.671 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:09.671 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:09.671 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8395e-05 s, 0.0 kB/s 2023-10-09T06:34:09.672 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:09.672 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1484 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:09.727 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:09.727 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:09.727 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1899e-05 s, 0.0 kB/s 2023-10-09T06:34:09.728 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:09.728 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1485 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:09.783 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:09.783 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:09.783 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.244e-05 s, 0.0 kB/s 2023-10-09T06:34:09.783 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:09.784 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1486 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:09.838 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:09.838 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:09.839 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0157e-05 s, 0.0 kB/s 2023-10-09T06:34:09.839 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:09.840 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1487 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:09.895 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:09.895 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:09.895 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9042e-05 s, 0.0 kB/s 2023-10-09T06:34:09.895 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:09.895 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1488 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:09.950 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:09.951 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:09.951 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8982e-05 s, 0.0 kB/s 2023-10-09T06:34:09.951 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:09.952 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1489 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:10.007 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:10.007 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:10.007 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8289e-05 s, 0.0 kB/s 2023-10-09T06:34:10.007 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:10.007 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1490 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:10.063 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:10.063 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:10.063 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8954e-05 s, 0.0 kB/s 2023-10-09T06:34:10.063 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:10.064 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1491 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:10.118 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:10.119 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:10.119 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8123e-05 s, 0.0 kB/s 2023-10-09T06:34:10.120 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:10.120 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1492 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:10.174 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:10.174 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:10.174 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6583e-05 s, 0.0 kB/s 2023-10-09T06:34:10.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:10.175 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1493 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:10.230 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:10.231 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:10.231 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7796e-05 s, 0.0 kB/s 2023-10-09T06:34:10.231 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:10.231 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1494 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:10.286 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:10.286 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:10.286 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1889e-05 s, 0.0 kB/s 2023-10-09T06:34:10.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:10.286 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1495 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:10.342 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:10.342 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:10.342 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9594e-05 s, 0.0 kB/s 2023-10-09T06:34:10.342 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:10.343 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1496 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:10.397 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:10.397 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:10.397 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9544e-05 s, 0.0 kB/s 2023-10-09T06:34:10.398 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:10.398 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1497 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:10.454 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:10.454 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:10.454 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9863e-05 s, 0.0 kB/s 2023-10-09T06:34:10.454 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:10.455 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1498 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:10.509 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:10.510 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:10.510 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3824e-05 s, 0.0 kB/s 2023-10-09T06:34:10.510 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:10.510 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1499 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:10.565 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:10.565 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:10.565 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.4375e-05 s, 0.0 kB/s 2023-10-09T06:34:10.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:10.565 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1500 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:10.620 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:10.620 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:10.620 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9876e-05 s, 0.0 kB/s 2023-10-09T06:34:10.620 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:10.621 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1501 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:10.675 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:10.676 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:10.676 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.853e-05 s, 0.0 kB/s 2023-10-09T06:34:10.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:10.677 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1502 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:10.732 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:10.732 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:10.732 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0085e-05 s, 0.0 kB/s 2023-10-09T06:34:10.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:10.733 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1503 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:10.788 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:10.788 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:10.788 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.056e-05 s, 0.0 kB/s 2023-10-09T06:34:10.789 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:10.789 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1504 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:10.844 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:10.844 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:10.844 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9898e-05 s, 0.0 kB/s 2023-10-09T06:34:10.844 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:10.844 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1505 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:10.900 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:10.900 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:10.900 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0453e-05 s, 0.0 kB/s 2023-10-09T06:34:10.900 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:10.900 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1506 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:10.966 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:10.966 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:10.967 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.7173e-05 s, 0.0 kB/s 2023-10-09T06:34:10.967 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:10.967 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1507 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:11.024 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:11.024 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:11.024 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4525e-05 s, 0.0 kB/s 2023-10-09T06:34:11.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:11.025 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1508 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:11.080 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:11.080 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:11.081 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0675e-05 s, 0.0 kB/s 2023-10-09T06:34:11.081 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:11.081 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1509 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:11.136 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:11.136 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:11.136 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8632e-05 s, 0.0 kB/s 2023-10-09T06:34:11.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:11.136 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1510 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:11.191 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:11.191 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:11.191 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.952e-05 s, 0.0 kB/s 2023-10-09T06:34:11.192 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:11.192 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1511 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:11.247 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:11.247 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:11.247 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1341e-05 s, 0.0 kB/s 2023-10-09T06:34:11.247 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:11.247 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1512 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:11.302 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:11.303 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:11.303 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.83e-05 s, 0.0 kB/s 2023-10-09T06:34:11.303 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:11.303 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1513 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:11.358 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:11.359 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:11.359 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7912e-05 s, 0.0 kB/s 2023-10-09T06:34:11.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:11.359 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1514 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:11.415 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:11.415 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:11.415 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1799e-05 s, 0.0 kB/s 2023-10-09T06:34:11.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:11.416 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1515 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:11.470 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:11.470 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:11.470 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9801e-05 s, 0.0 kB/s 2023-10-09T06:34:11.470 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:11.470 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1516 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:11.525 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:11.525 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:11.525 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0884e-05 s, 0.0 kB/s 2023-10-09T06:34:11.525 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:11.526 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1517 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:11.581 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:11.581 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:11.581 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.829e-05 s, 0.0 kB/s 2023-10-09T06:34:11.582 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:11.582 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1518 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:11.637 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:11.637 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:11.638 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1159e-05 s, 0.0 kB/s 2023-10-09T06:34:11.638 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:11.638 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1519 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:11.693 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:11.693 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:11.693 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8034e-05 s, 0.0 kB/s 2023-10-09T06:34:11.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:11.694 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1520 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:11.749 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:11.749 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:11.749 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0775e-05 s, 0.0 kB/s 2023-10-09T06:34:11.750 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:11.750 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1521 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:11.805 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:11.805 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:11.805 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4143e-05 s, 0.0 kB/s 2023-10-09T06:34:11.805 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:11.805 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1522 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:11.861 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:11.861 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:11.861 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1341e-05 s, 0.0 kB/s 2023-10-09T06:34:11.861 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:11.862 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1523 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:11.917 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:11.917 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:11.917 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7633e-05 s, 0.0 kB/s 2023-10-09T06:34:11.917 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:11.917 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1524 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:11.973 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:11.973 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:11.973 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9941e-05 s, 0.0 kB/s 2023-10-09T06:34:11.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:11.974 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1525 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:12.029 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:12.029 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:12.029 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0969e-05 s, 0.0 kB/s 2023-10-09T06:34:12.029 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:12.029 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1526 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:12.084 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:12.084 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:12.084 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1146e-05 s, 0.0 kB/s 2023-10-09T06:34:12.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:12.085 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1527 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:12.141 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:12.141 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:12.141 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9815e-05 s, 0.0 kB/s 2023-10-09T06:34:12.141 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:12.141 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1528 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:12.197 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:12.197 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:12.197 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0788e-05 s, 0.0 kB/s 2023-10-09T06:34:12.197 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:12.197 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1529 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:12.253 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:12.253 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:12.253 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.096e-05 s, 0.0 kB/s 2023-10-09T06:34:12.253 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:12.253 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1530 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:12.308 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:12.308 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:12.308 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3051e-05 s, 0.0 kB/s 2023-10-09T06:34:12.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:12.308 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1531 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:12.363 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:12.364 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:12.364 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0509e-05 s, 0.0 kB/s 2023-10-09T06:34:12.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:12.365 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1532 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:12.420 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:12.420 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:12.421 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2605e-05 s, 0.0 kB/s 2023-10-09T06:34:12.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:12.421 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1533 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:12.475 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:12.476 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:12.476 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0955e-05 s, 0.0 kB/s 2023-10-09T06:34:12.476 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:12.476 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1534 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:12.533 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:12.533 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:12.533 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8369e-05 s, 0.0 kB/s 2023-10-09T06:34:12.534 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:12.534 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1535 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:12.589 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:12.589 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:12.589 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.749e-05 s, 0.0 kB/s 2023-10-09T06:34:12.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:12.590 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1536 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:12.645 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:12.645 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:12.645 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8959e-05 s, 0.0 kB/s 2023-10-09T06:34:12.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:12.646 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1537 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:12.701 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:12.701 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:12.701 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9827e-05 s, 0.0 kB/s 2023-10-09T06:34:12.702 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:12.702 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1538 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:12.757 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:12.757 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:12.757 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7546e-05 s, 0.0 kB/s 2023-10-09T06:34:12.758 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:12.758 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1539 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:12.813 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:12.813 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:12.813 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3267e-05 s, 0.0 kB/s 2023-10-09T06:34:12.814 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:12.814 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1540 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:12.870 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:12.870 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:12.870 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1578e-05 s, 0.0 kB/s 2023-10-09T06:34:12.870 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:12.870 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1541 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:12.926 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:12.926 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:12.926 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7781e-05 s, 0.0 kB/s 2023-10-09T06:34:12.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:12.927 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1542 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:12.981 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:12.981 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:12.981 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9676e-05 s, 0.0 kB/s 2023-10-09T06:34:12.981 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:12.982 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1543 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:13.037 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:13.037 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:13.037 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2055e-05 s, 0.0 kB/s 2023-10-09T06:34:13.037 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:13.037 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1544 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:13.092 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:13.092 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:13.092 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2339e-05 s, 0.0 kB/s 2023-10-09T06:34:13.092 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:13.093 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1545 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:13.148 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:13.148 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:13.148 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1274e-05 s, 0.0 kB/s 2023-10-09T06:34:13.148 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:13.149 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1546 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:13.204 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:13.204 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:13.204 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0242e-05 s, 0.0 kB/s 2023-10-09T06:34:13.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:13.204 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1547 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:13.260 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:13.260 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:13.260 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9152e-05 s, 0.0 kB/s 2023-10-09T06:34:13.261 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:13.261 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1548 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:13.316 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:13.316 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:13.316 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9552e-05 s, 0.0 kB/s 2023-10-09T06:34:13.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:13.316 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1549 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:13.370 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:13.371 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:13.371 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9024e-05 s, 0.0 kB/s 2023-10-09T06:34:13.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:13.371 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1550 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:13.384 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:34:13.413 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:34:13.426 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:13.426 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:13.426 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0186e-05 s, 0.0 kB/s 2023-10-09T06:34:13.426 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:13.426 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1551 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:13.481 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:13.481 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:13.481 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.199e-05 s, 0.0 kB/s 2023-10-09T06:34:13.482 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:13.482 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1552 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:13.537 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:13.537 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:13.537 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9503e-05 s, 0.0 kB/s 2023-10-09T06:34:13.537 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:13.538 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1553 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:13.593 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:13.593 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:13.593 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9151e-05 s, 0.0 kB/s 2023-10-09T06:34:13.594 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:13.594 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1554 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:13.650 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:13.650 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:13.650 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1503e-05 s, 0.0 kB/s 2023-10-09T06:34:13.650 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:13.650 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1555 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:13.705 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:13.705 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:13.705 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6953e-05 s, 0.0 kB/s 2023-10-09T06:34:13.705 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:13.705 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1556 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:13.760 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:13.760 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:13.760 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9055e-05 s, 0.0 kB/s 2023-10-09T06:34:13.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:13.761 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1557 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:13.816 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:13.816 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:13.816 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9374e-05 s, 0.0 kB/s 2023-10-09T06:34:13.817 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:13.817 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1558 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:13.872 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:13.872 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:13.872 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9784e-05 s, 0.0 kB/s 2023-10-09T06:34:13.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:13.873 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1559 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:13.928 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:13.928 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:13.928 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7922e-05 s, 0.0 kB/s 2023-10-09T06:34:13.928 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:13.928 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1560 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:13.983 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:13.984 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:13.984 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.968e-05 s, 0.0 kB/s 2023-10-09T06:34:13.984 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:13.984 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1561 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:14.039 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:14.039 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:14.039 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8653e-05 s, 0.0 kB/s 2023-10-09T06:34:14.040 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:14.040 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1562 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:14.095 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:14.095 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:14.095 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8169e-05 s, 0.0 kB/s 2023-10-09T06:34:14.095 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:14.096 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1563 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:14.150 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:14.150 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:14.150 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0136e-05 s, 0.0 kB/s 2023-10-09T06:34:14.151 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:14.151 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1564 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:14.207 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:14.207 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:14.207 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2656e-05 s, 0.0 kB/s 2023-10-09T06:34:14.207 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:14.207 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1565 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:14.263 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:14.263 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:14.263 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7891e-05 s, 0.0 kB/s 2023-10-09T06:34:14.263 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:14.263 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1566 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:14.318 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:14.319 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:14.319 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8938e-05 s, 0.0 kB/s 2023-10-09T06:34:14.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:14.319 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1567 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:14.374 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:14.374 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:14.374 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3762e-05 s, 0.0 kB/s 2023-10-09T06:34:14.374 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:14.374 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1568 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:14.428 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:14.429 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:14.429 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.827e-05 s, 0.0 kB/s 2023-10-09T06:34:14.429 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:14.429 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1569 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:14.484 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:14.484 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:14.484 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0397e-05 s, 0.0 kB/s 2023-10-09T06:34:14.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:14.484 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1570 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:14.541 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:14.541 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:14.541 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2237e-05 s, 0.0 kB/s 2023-10-09T06:34:14.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:14.541 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1571 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:14.596 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:14.596 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:14.596 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9282e-05 s, 0.0 kB/s 2023-10-09T06:34:14.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:14.596 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1572 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:14.652 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:14.652 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:14.652 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1821e-05 s, 0.0 kB/s 2023-10-09T06:34:14.653 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:14.653 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1573 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:14.708 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:14.708 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:14.708 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.956e-05 s, 0.0 kB/s 2023-10-09T06:34:14.708 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:14.708 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1574 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:14.764 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:14.764 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:14.764 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0306e-05 s, 0.0 kB/s 2023-10-09T06:34:14.765 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:14.765 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1575 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:14.820 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:14.820 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:14.820 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0038e-05 s, 0.0 kB/s 2023-10-09T06:34:14.820 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:14.820 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1576 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:14.876 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:14.876 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:14.876 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7937e-05 s, 0.0 kB/s 2023-10-09T06:34:14.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:14.877 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1577 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:14.932 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:14.932 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:14.932 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9818e-05 s, 0.0 kB/s 2023-10-09T06:34:14.932 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:14.932 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1578 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:14.987 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:14.987 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:14.987 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2631e-05 s, 0.0 kB/s 2023-10-09T06:34:14.987 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:14.987 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1579 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:15.043 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:15.043 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:15.043 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8707e-05 s, 0.0 kB/s 2023-10-09T06:34:15.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:15.043 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1580 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:15.098 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:15.098 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:15.098 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0298e-05 s, 0.0 kB/s 2023-10-09T06:34:15.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:15.098 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1581 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:15.153 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:15.153 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:15.153 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8216e-05 s, 0.0 kB/s 2023-10-09T06:34:15.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:15.154 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1582 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:15.209 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:15.209 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:15.209 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.6607e-05 s, 0.0 kB/s 2023-10-09T06:34:15.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:15.209 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1583 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:15.264 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:15.264 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:15.264 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1175e-05 s, 0.0 kB/s 2023-10-09T06:34:15.264 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:15.264 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1584 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:15.320 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:15.320 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:15.320 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6859e-05 s, 0.0 kB/s 2023-10-09T06:34:15.320 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:15.320 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1585 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:15.376 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:15.376 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:15.376 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9385e-05 s, 0.0 kB/s 2023-10-09T06:34:15.376 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:15.376 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1586 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:15.432 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:15.432 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:15.432 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0103e-05 s, 0.0 kB/s 2023-10-09T06:34:15.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:15.432 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1587 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:15.488 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:15.488 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:15.488 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2818e-05 s, 0.0 kB/s 2023-10-09T06:34:15.488 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:15.488 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1588 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:15.543 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:15.543 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:15.543 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.323e-05 s, 0.0 kB/s 2023-10-09T06:34:15.543 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:15.544 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1589 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:15.599 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:15.599 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:15.599 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2207e-05 s, 0.0 kB/s 2023-10-09T06:34:15.599 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:15.599 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1590 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:15.654 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:15.655 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:15.655 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1609e-05 s, 0.0 kB/s 2023-10-09T06:34:15.655 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:15.655 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1591 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:15.709 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:15.710 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:15.710 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0942e-05 s, 0.0 kB/s 2023-10-09T06:34:15.710 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:15.710 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1592 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:15.765 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:15.765 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:15.765 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7455e-05 s, 0.0 kB/s 2023-10-09T06:34:15.765 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:15.765 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1593 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:15.821 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:15.821 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:15.821 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9549e-05 s, 0.0 kB/s 2023-10-09T06:34:15.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:15.821 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1594 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:15.876 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:15.876 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:15.876 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1596e-05 s, 0.0 kB/s 2023-10-09T06:34:15.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:15.877 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1595 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:15.932 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:15.932 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:15.932 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1318e-05 s, 0.0 kB/s 2023-10-09T06:34:15.932 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:15.932 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1596 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:15.987 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:15.987 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:15.987 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8154e-05 s, 0.0 kB/s 2023-10-09T06:34:15.987 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:15.987 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1597 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:16.043 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:16.043 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:16.043 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9881e-05 s, 0.0 kB/s 2023-10-09T06:34:16.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:16.044 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1598 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:16.099 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:16.099 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:16.099 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8972e-05 s, 0.0 kB/s 2023-10-09T06:34:16.100 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:16.100 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1599 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:16.155 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:16.155 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:16.155 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0955e-05 s, 0.0 kB/s 2023-10-09T06:34:16.155 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:16.155 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1600 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:16.210 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:16.210 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:16.210 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.4722e-05 s, 0.0 kB/s 2023-10-09T06:34:16.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:16.211 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1601 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:16.266 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:16.266 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:16.266 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9128e-05 s, 0.0 kB/s 2023-10-09T06:34:16.266 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:16.266 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1602 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:16.322 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:16.322 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:16.322 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1284e-05 s, 0.0 kB/s 2023-10-09T06:34:16.323 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:16.323 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1603 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:16.378 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:16.378 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:16.378 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.988e-05 s, 0.0 kB/s 2023-10-09T06:34:16.378 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:16.379 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1604 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:16.434 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:16.434 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:16.434 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1284e-05 s, 0.0 kB/s 2023-10-09T06:34:16.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:16.434 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1605 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:16.490 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:16.490 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:16.490 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9359e-05 s, 0.0 kB/s 2023-10-09T06:34:16.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:16.491 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1606 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:16.585 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:16.585 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:16.585 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.7493e-05 s, 0.0 kB/s 2023-10-09T06:34:16.585 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:16.585 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1607 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:16.601 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:16.601 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:16.601 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0207e-05 s, 0.0 kB/s 2023-10-09T06:34:16.601 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:16.601 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1608 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:16.657 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:16.657 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:16.657 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2686e-05 s, 0.0 kB/s 2023-10-09T06:34:16.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:16.658 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1609 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:16.713 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:16.713 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:16.714 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2964e-05 s, 0.0 kB/s 2023-10-09T06:34:16.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:16.714 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1610 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:16.769 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:16.769 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:16.769 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7836e-05 s, 0.0 kB/s 2023-10-09T06:34:16.770 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:16.770 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1611 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:16.825 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:16.825 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:16.825 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1627e-05 s, 0.0 kB/s 2023-10-09T06:34:16.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:16.826 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1612 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:16.881 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:16.881 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:16.881 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0365e-05 s, 0.0 kB/s 2023-10-09T06:34:16.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:16.881 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1613 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:16.936 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:16.936 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:16.936 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9128e-05 s, 0.0 kB/s 2023-10-09T06:34:16.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:16.936 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1614 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:16.990 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:16.991 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:16.991 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9893e-05 s, 0.0 kB/s 2023-10-09T06:34:16.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:16.991 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1615 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:17.046 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:17.046 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:17.047 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0511e-05 s, 0.0 kB/s 2023-10-09T06:34:17.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:17.047 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1616 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:17.102 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:17.102 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:17.102 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1363e-05 s, 0.0 kB/s 2023-10-09T06:34:17.102 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:17.102 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1617 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:17.158 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:17.158 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:17.158 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9992e-05 s, 0.0 kB/s 2023-10-09T06:34:17.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:17.159 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1618 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:17.214 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:17.214 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:17.214 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0336e-05 s, 0.0 kB/s 2023-10-09T06:34:17.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:17.215 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1619 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:17.270 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:17.271 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:17.271 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9385e-05 s, 0.0 kB/s 2023-10-09T06:34:17.271 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:17.271 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1620 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:17.326 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:17.326 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:17.326 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7707e-05 s, 0.0 kB/s 2023-10-09T06:34:17.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:17.327 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1621 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:17.382 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:17.384 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:17.384 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9481e-05 s, 0.0 kB/s 2023-10-09T06:34:17.385 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:17.386 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1622 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:17.437 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:17.437 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:17.438 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9165e-05 s, 0.0 kB/s 2023-10-09T06:34:17.438 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:17.439 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1623 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:17.493 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:17.494 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:17.494 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7443e-05 s, 0.0 kB/s 2023-10-09T06:34:17.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:17.495 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1624 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:17.550 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:17.550 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:17.550 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9958e-05 s, 0.0 kB/s 2023-10-09T06:34:17.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:17.551 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1625 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:17.606 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:17.606 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:17.606 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0888e-05 s, 0.0 kB/s 2023-10-09T06:34:17.606 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:17.606 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1626 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:17.661 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:17.661 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:17.661 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.2046e-05 s, 0.0 kB/s 2023-10-09T06:34:17.662 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:17.662 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1627 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:17.717 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:17.717 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:17.717 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9075e-05 s, 0.0 kB/s 2023-10-09T06:34:17.718 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:17.718 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1628 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:17.773 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:17.773 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:17.773 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2006e-05 s, 0.0 kB/s 2023-10-09T06:34:17.774 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:17.774 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1629 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:17.830 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:17.830 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:17.830 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7668e-05 s, 0.0 kB/s 2023-10-09T06:34:17.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:17.831 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1630 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:17.886 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:17.886 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:17.886 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9499e-05 s, 0.0 kB/s 2023-10-09T06:34:17.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:17.886 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1631 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:17.942 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:17.942 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:17.942 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2168e-05 s, 0.0 kB/s 2023-10-09T06:34:17.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:17.942 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1632 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:17.997 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:17.997 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:17.997 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4355e-05 s, 0.0 kB/s 2023-10-09T06:34:17.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:17.997 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1633 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:18.053 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:18.053 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:18.053 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2781e-05 s, 0.0 kB/s 2023-10-09T06:34:18.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:18.053 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1634 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:18.109 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:18.109 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:18.109 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6889e-05 s, 0.0 kB/s 2023-10-09T06:34:18.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:18.110 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1635 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:18.165 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:18.165 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:18.165 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.789e-05 s, 0.0 kB/s 2023-10-09T06:34:18.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:18.165 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1636 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:18.221 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:18.221 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:18.221 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.028e-05 s, 0.0 kB/s 2023-10-09T06:34:18.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:18.221 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1637 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:18.278 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:18.278 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:18.278 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9456e-05 s, 0.0 kB/s 2023-10-09T06:34:18.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:18.279 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1638 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:18.334 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:18.334 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:18.334 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.9272e-05 s, 0.0 kB/s 2023-10-09T06:34:18.334 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:18.334 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1639 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:18.389 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:18.389 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:18.389 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0697e-05 s, 0.0 kB/s 2023-10-09T06:34:18.390 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:18.390 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1640 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:18.445 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:18.445 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:18.445 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2253e-05 s, 0.0 kB/s 2023-10-09T06:34:18.446 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:18.446 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1641 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:18.501 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:18.501 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:18.501 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8698e-05 s, 0.0 kB/s 2023-10-09T06:34:18.501 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:18.501 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1642 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:18.557 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:18.557 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:18.557 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9619e-05 s, 0.0 kB/s 2023-10-09T06:34:18.558 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:18.558 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1643 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:18.613 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:18.613 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:18.613 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8069e-05 s, 0.0 kB/s 2023-10-09T06:34:18.613 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:18.613 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1644 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:18.669 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:18.669 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:18.669 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.851e-05 s, 0.0 kB/s 2023-10-09T06:34:18.669 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:18.669 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1645 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:18.725 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:18.725 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:18.725 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2584e-05 s, 0.0 kB/s 2023-10-09T06:34:18.725 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:18.725 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1646 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:18.781 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:18.781 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:18.781 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.13e-05 s, 0.0 kB/s 2023-10-09T06:34:18.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:18.781 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1647 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:18.836 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:18.836 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:18.836 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8904e-05 s, 0.0 kB/s 2023-10-09T06:34:18.837 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:18.837 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1648 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:18.895 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:18.895 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:18.895 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9208e-05 s, 0.0 kB/s 2023-10-09T06:34:18.895 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:18.896 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1649 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:18.951 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:18.951 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:18.951 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2144e-05 s, 0.0 kB/s 2023-10-09T06:34:18.951 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:18.952 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1650 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:19.007 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:19.007 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:19.007 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.093e-05 s, 0.0 kB/s 2023-10-09T06:34:19.008 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:19.008 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1651 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:19.063 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:19.063 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:19.063 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8057e-05 s, 0.0 kB/s 2023-10-09T06:34:19.064 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:19.064 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1652 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:19.119 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:19.119 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:19.119 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0952e-05 s, 0.0 kB/s 2023-10-09T06:34:19.119 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:19.119 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1653 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:19.175 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:19.175 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:19.175 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0784e-05 s, 0.0 kB/s 2023-10-09T06:34:19.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:19.175 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1654 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:19.230 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:19.230 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:19.230 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2112e-05 s, 0.0 kB/s 2023-10-09T06:34:19.230 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:19.231 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1655 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:19.286 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:19.286 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:19.286 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9474e-05 s, 0.0 kB/s 2023-10-09T06:34:19.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:19.287 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1656 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:19.342 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:19.342 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:19.342 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1242e-05 s, 0.0 kB/s 2023-10-09T06:34:19.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:19.343 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1657 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:19.398 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:19.398 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:19.398 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2449e-05 s, 0.0 kB/s 2023-10-09T06:34:19.398 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:19.399 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1658 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:19.453 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:19.453 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:19.453 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1766e-05 s, 0.0 kB/s 2023-10-09T06:34:19.453 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:19.453 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1659 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:19.509 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:19.509 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:19.509 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1337e-05 s, 0.0 kB/s 2023-10-09T06:34:19.509 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:19.509 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1660 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:19.564 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:19.564 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:19.564 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9337e-05 s, 0.0 kB/s 2023-10-09T06:34:19.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:19.564 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1661 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:19.619 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:19.619 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:19.619 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1431e-05 s, 0.0 kB/s 2023-10-09T06:34:19.619 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:19.619 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1662 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:19.675 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:19.675 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:19.675 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2107e-05 s, 0.0 kB/s 2023-10-09T06:34:19.675 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:19.676 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1663 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:19.732 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:19.732 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:19.732 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.7088e-05 s, 0.0 kB/s 2023-10-09T06:34:19.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:19.732 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1664 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:19.788 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:19.788 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:19.788 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1551e-05 s, 0.0 kB/s 2023-10-09T06:34:19.789 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:19.789 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1665 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:19.844 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:19.844 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:19.844 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0348e-05 s, 0.0 kB/s 2023-10-09T06:34:19.844 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:19.845 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1666 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:19.900 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:19.900 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:19.900 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.096e-05 s, 0.0 kB/s 2023-10-09T06:34:19.900 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:19.900 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1667 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:19.955 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:19.955 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:19.955 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1645e-05 s, 0.0 kB/s 2023-10-09T06:34:19.955 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:19.955 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1668 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:20.011 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:20.011 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:20.011 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1649e-05 s, 0.0 kB/s 2023-10-09T06:34:20.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:20.012 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1669 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:20.066 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:20.067 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:20.067 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9762e-05 s, 0.0 kB/s 2023-10-09T06:34:20.067 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:20.067 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1670 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:20.122 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:20.123 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:20.123 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8243e-05 s, 0.0 kB/s 2023-10-09T06:34:20.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:20.123 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1671 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:20.178 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:20.178 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:20.178 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0859e-05 s, 0.0 kB/s 2023-10-09T06:34:20.179 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:20.179 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1672 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:20.234 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:20.235 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:20.235 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1081e-05 s, 0.0 kB/s 2023-10-09T06:34:20.235 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:20.235 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1673 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:20.289 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:20.290 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:20.290 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8509e-05 s, 0.0 kB/s 2023-10-09T06:34:20.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:20.290 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1674 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:20.345 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:20.345 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:20.345 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1779e-05 s, 0.0 kB/s 2023-10-09T06:34:20.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:20.346 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1675 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:20.401 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:20.401 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:20.401 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0713e-05 s, 0.0 kB/s 2023-10-09T06:34:20.401 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:20.401 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1676 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:20.456 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:20.456 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:20.456 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.5411e-05 s, 0.0 kB/s 2023-10-09T06:34:20.457 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:20.457 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1677 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:20.513 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:20.513 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:20.513 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8714e-05 s, 0.0 kB/s 2023-10-09T06:34:20.514 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:20.514 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1678 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:20.569 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:20.569 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:20.569 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9602e-05 s, 0.0 kB/s 2023-10-09T06:34:20.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:20.569 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1679 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:20.624 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:20.624 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:20.624 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8674e-05 s, 0.0 kB/s 2023-10-09T06:34:20.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:20.624 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1680 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:20.680 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:20.681 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:20.681 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1225e-05 s, 0.0 kB/s 2023-10-09T06:34:20.681 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:20.681 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1681 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:20.736 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:20.736 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:20.736 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.246e-05 s, 0.0 kB/s 2023-10-09T06:34:20.736 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:20.736 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1682 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:20.792 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:20.792 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:20.792 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1392e-05 s, 0.0 kB/s 2023-10-09T06:34:20.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:20.793 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1683 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:20.848 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:20.848 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:20.848 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9146e-05 s, 0.0 kB/s 2023-10-09T06:34:20.848 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:20.848 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1684 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:20.904 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:20.904 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:20.904 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3465e-05 s, 0.0 kB/s 2023-10-09T06:34:20.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:20.905 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1685 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:20.960 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:20.960 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:20.960 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1087e-05 s, 0.0 kB/s 2023-10-09T06:34:20.961 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:20.961 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1686 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:21.017 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:21.017 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:21.017 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8595e-05 s, 0.0 kB/s 2023-10-09T06:34:21.017 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:21.017 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1687 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:21.072 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:21.072 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:21.072 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1964e-05 s, 0.0 kB/s 2023-10-09T06:34:21.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:21.072 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1688 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:21.127 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:21.128 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:21.128 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0584e-05 s, 0.0 kB/s 2023-10-09T06:34:21.128 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:21.128 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1689 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:21.183 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:21.183 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:21.183 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4199e-05 s, 0.0 kB/s 2023-10-09T06:34:21.183 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:21.183 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1690 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:21.237 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:21.238 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:21.238 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.878e-05 s, 0.0 kB/s 2023-10-09T06:34:21.238 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:21.238 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1691 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:21.293 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:21.293 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:21.293 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1396e-05 s, 0.0 kB/s 2023-10-09T06:34:21.293 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:21.294 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1692 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:21.349 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:21.349 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:21.349 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9958e-05 s, 0.0 kB/s 2023-10-09T06:34:21.350 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:21.350 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1693 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:21.405 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:21.405 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:21.405 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9319e-05 s, 0.0 kB/s 2023-10-09T06:34:21.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:21.406 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1694 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:21.461 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:21.461 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:21.461 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1301e-05 s, 0.0 kB/s 2023-10-09T06:34:21.461 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:21.462 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1695 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:21.517 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:21.517 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:21.517 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3063e-05 s, 0.0 kB/s 2023-10-09T06:34:21.517 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:21.517 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1696 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:21.573 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:21.573 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:21.573 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8482e-05 s, 0.0 kB/s 2023-10-09T06:34:21.574 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:21.574 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1697 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:21.629 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:21.629 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:21.629 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1331e-05 s, 0.0 kB/s 2023-10-09T06:34:21.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:21.630 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1698 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:21.697 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:21.697 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:21.697 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.8307e-05 s, 0.0 kB/s 2023-10-09T06:34:21.698 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:21.698 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1699 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:21.753 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:21.754 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:21.754 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8157e-05 s, 0.0 kB/s 2023-10-09T06:34:21.754 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:21.754 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1700 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:21.809 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:21.810 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:21.810 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0829e-05 s, 0.0 kB/s 2023-10-09T06:34:21.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:21.810 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1701 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:21.865 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:21.865 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:21.865 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8652e-05 s, 0.0 kB/s 2023-10-09T06:34:21.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:21.865 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1702 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:21.921 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:21.921 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:21.921 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.86e-05 s, 0.0 kB/s 2023-10-09T06:34:21.921 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:21.921 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1703 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:21.977 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:21.978 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:21.978 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2291e-05 s, 0.0 kB/s 2023-10-09T06:34:21.978 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:21.978 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1704 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:22.032 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:22.033 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:22.033 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.875e-05 s, 0.0 kB/s 2023-10-09T06:34:22.033 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:22.033 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1705 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:22.088 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:22.088 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:22.088 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.864e-05 s, 0.0 kB/s 2023-10-09T06:34:22.089 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:22.089 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1706 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:22.144 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:22.144 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:22.144 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2544e-05 s, 0.0 kB/s 2023-10-09T06:34:22.144 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:22.144 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1707 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:22.200 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:22.200 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:22.200 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.973e-05 s, 0.0 kB/s 2023-10-09T06:34:22.200 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:22.200 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1708 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:22.255 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:22.255 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:22.255 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1848e-05 s, 0.0 kB/s 2023-10-09T06:34:22.255 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:22.255 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1709 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:22.311 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:22.311 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:22.311 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3399e-05 s, 0.0 kB/s 2023-10-09T06:34:22.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:22.312 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1710 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:22.367 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:22.367 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:22.367 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.119e-05 s, 0.0 kB/s 2023-10-09T06:34:22.368 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:22.368 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1711 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:22.423 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:22.424 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:22.424 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.189e-05 s, 0.0 kB/s 2023-10-09T06:34:22.424 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:22.424 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1712 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:22.479 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:22.479 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:22.479 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3582e-05 s, 0.0 kB/s 2023-10-09T06:34:22.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:22.480 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1713 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:22.535 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:22.535 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:22.535 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1973e-05 s, 0.0 kB/s 2023-10-09T06:34:22.535 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:22.536 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1714 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:22.591 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:22.591 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:22.591 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2229e-05 s, 0.0 kB/s 2023-10-09T06:34:22.592 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:22.592 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1715 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:22.646 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:22.646 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:22.646 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.5507e-05 s, 0.0 kB/s 2023-10-09T06:34:22.647 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:22.647 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1716 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:22.703 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:22.704 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:22.704 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9411e-05 s, 0.0 kB/s 2023-10-09T06:34:22.704 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:22.704 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1717 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:22.760 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:22.760 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:22.760 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1125e-05 s, 0.0 kB/s 2023-10-09T06:34:22.760 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:22.760 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1718 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:22.815 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:22.815 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:22.815 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1491e-05 s, 0.0 kB/s 2023-10-09T06:34:22.816 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:22.816 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1719 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:22.871 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:22.871 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:22.871 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0715e-05 s, 0.0 kB/s 2023-10-09T06:34:22.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:22.872 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1720 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:22.927 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:22.927 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:22.927 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8311e-05 s, 0.0 kB/s 2023-10-09T06:34:22.928 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:22.928 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1721 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:22.983 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:22.983 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:22.984 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7198e-05 s, 0.0 kB/s 2023-10-09T06:34:22.984 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:22.984 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1722 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:23.039 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:23.039 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:23.039 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9574e-05 s, 0.0 kB/s 2023-10-09T06:34:23.039 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:23.039 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1723 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:23.094 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:23.094 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:23.094 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7423e-05 s, 0.0 kB/s 2023-10-09T06:34:23.094 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:23.095 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1724 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:23.150 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:23.150 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:23.150 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.95e-05 s, 0.0 kB/s 2023-10-09T06:34:23.150 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:23.150 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1725 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:23.206 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:23.206 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:23.206 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7667e-05 s, 0.0 kB/s 2023-10-09T06:34:23.206 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:23.206 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1726 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:23.262 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:23.262 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:23.262 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0251e-05 s, 0.0 kB/s 2023-10-09T06:34:23.262 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:23.262 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1727 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:23.317 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:23.317 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:23.317 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9353e-05 s, 0.0 kB/s 2023-10-09T06:34:23.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:23.318 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1728 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:23.373 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:23.373 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:23.373 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0206e-05 s, 0.0 kB/s 2023-10-09T06:34:23.373 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:23.374 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1729 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:23.428 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:23.428 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:23.428 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0442e-05 s, 0.0 kB/s 2023-10-09T06:34:23.429 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:23.429 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1730 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:23.485 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:23.485 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:23.485 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8878e-05 s, 0.0 kB/s 2023-10-09T06:34:23.485 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:23.485 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1731 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:23.540 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:23.540 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:23.540 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.929e-05 s, 0.0 kB/s 2023-10-09T06:34:23.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:23.541 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1732 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:23.596 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:23.596 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:23.596 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0755e-05 s, 0.0 kB/s 2023-10-09T06:34:23.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:23.596 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1733 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:23.652 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:23.652 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:23.652 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0396e-05 s, 0.0 kB/s 2023-10-09T06:34:23.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:23.653 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1734 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:23.709 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:23.709 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:23.709 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.972e-05 s, 0.0 kB/s 2023-10-09T06:34:23.709 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:23.709 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1735 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:23.765 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:23.765 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:23.765 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.003e-05 s, 0.0 kB/s 2023-10-09T06:34:23.765 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:23.766 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1736 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:23.821 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:23.821 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:23.821 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0036e-05 s, 0.0 kB/s 2023-10-09T06:34:23.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:23.821 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1737 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:23.877 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:23.877 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:23.877 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.415e-05 s, 0.0 kB/s 2023-10-09T06:34:23.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:23.877 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1738 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:23.932 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:23.932 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:23.932 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8196e-05 s, 0.0 kB/s 2023-10-09T06:34:23.932 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:23.932 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1739 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:23.987 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:23.987 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:23.987 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9039e-05 s, 0.0 kB/s 2023-10-09T06:34:23.987 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:23.987 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1740 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:24.043 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:24.043 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:24.043 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.117e-05 s, 0.0 kB/s 2023-10-09T06:34:24.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:24.043 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1741 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:24.098 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:24.098 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:24.099 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0179e-05 s, 0.0 kB/s 2023-10-09T06:34:24.099 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:24.100 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1742 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:24.155 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:24.155 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:24.155 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.925e-05 s, 0.0 kB/s 2023-10-09T06:34:24.155 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:24.155 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1743 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:24.210 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:24.210 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:24.210 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1059e-05 s, 0.0 kB/s 2023-10-09T06:34:24.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:24.210 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1744 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:24.265 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:24.265 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:24.265 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9597e-05 s, 0.0 kB/s 2023-10-09T06:34:24.265 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:24.266 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1745 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:24.321 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:24.321 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:24.321 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8203e-05 s, 0.0 kB/s 2023-10-09T06:34:24.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:24.321 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1746 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:24.375 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:24.376 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:24.376 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.7307e-05 s, 0.0 kB/s 2023-10-09T06:34:24.376 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:24.376 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1747 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:24.431 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:24.431 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:24.431 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.95e-05 s, 0.0 kB/s 2023-10-09T06:34:24.431 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:24.431 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1748 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:24.486 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:24.486 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:24.486 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9058e-05 s, 0.0 kB/s 2023-10-09T06:34:24.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:24.487 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1749 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:24.542 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:24.542 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:24.542 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.99e-05 s, 0.0 kB/s 2023-10-09T06:34:24.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:24.543 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1750 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:24.598 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:24.598 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:24.598 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8294e-05 s, 0.0 kB/s 2023-10-09T06:34:24.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:24.598 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1751 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:24.654 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:24.654 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:24.654 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9125e-05 s, 0.0 kB/s 2023-10-09T06:34:24.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:24.654 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1752 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:24.710 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:24.710 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:24.710 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9459e-05 s, 0.0 kB/s 2023-10-09T06:34:24.710 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:24.711 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1753 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:24.766 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:24.766 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:24.766 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.941e-05 s, 0.0 kB/s 2023-10-09T06:34:24.767 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:24.767 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1754 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:24.821 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:24.822 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:24.822 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1443e-05 s, 0.0 kB/s 2023-10-09T06:34:24.822 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:24.822 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1755 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:24.876 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:24.877 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:24.877 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8253e-05 s, 0.0 kB/s 2023-10-09T06:34:24.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:24.877 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1756 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:24.932 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:24.932 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:24.932 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3341e-05 s, 0.0 kB/s 2023-10-09T06:34:24.932 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:24.932 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1757 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:24.987 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:24.987 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:24.987 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7687e-05 s, 0.0 kB/s 2023-10-09T06:34:24.988 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:24.988 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1758 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:25.043 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:25.111 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:25.111 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0128e-05 s, 0.0 kB/s 2023-10-09T06:34:25.111 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:25.111 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1759 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:25.127 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:25.127 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:25.127 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9668e-05 s, 0.0 kB/s 2023-10-09T06:34:25.127 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:25.128 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1760 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:25.183 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:25.183 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:25.183 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8447e-05 s, 0.0 kB/s 2023-10-09T06:34:25.183 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:25.183 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1761 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:25.238 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:25.238 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:25.238 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7529e-05 s, 0.0 kB/s 2023-10-09T06:34:25.239 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:25.239 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1762 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:25.293 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:25.294 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:25.294 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9663e-05 s, 0.0 kB/s 2023-10-09T06:34:25.294 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:25.295 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1763 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:25.349 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:25.349 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:25.349 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1151e-05 s, 0.0 kB/s 2023-10-09T06:34:25.350 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:25.350 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1764 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:25.405 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:25.405 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:25.405 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1259e-05 s, 0.0 kB/s 2023-10-09T06:34:25.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:25.406 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1765 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:25.461 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:25.461 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:25.462 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.837e-05 s, 0.0 kB/s 2023-10-09T06:34:25.462 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:25.462 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1766 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:25.517 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:25.517 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:25.517 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9843e-05 s, 0.0 kB/s 2023-10-09T06:34:25.518 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:25.518 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1767 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:25.573 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:25.574 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:25.574 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9167e-05 s, 0.0 kB/s 2023-10-09T06:34:25.574 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:25.574 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1768 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:25.629 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:25.629 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:25.629 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2251e-05 s, 0.0 kB/s 2023-10-09T06:34:25.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:25.629 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1769 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:25.685 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:25.685 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:25.685 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8475e-05 s, 0.0 kB/s 2023-10-09T06:34:25.686 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:25.686 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1770 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:25.742 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:25.742 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:25.742 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8657e-05 s, 0.0 kB/s 2023-10-09T06:34:25.742 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:25.743 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1771 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:25.797 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:25.797 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:25.797 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.204e-05 s, 0.0 kB/s 2023-10-09T06:34:25.798 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:25.798 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1772 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:25.853 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:25.853 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:25.853 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0309e-05 s, 0.0 kB/s 2023-10-09T06:34:25.854 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:25.854 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1773 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:25.909 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:25.910 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:25.910 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0033e-05 s, 0.0 kB/s 2023-10-09T06:34:25.910 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:25.910 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1774 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:25.965 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:25.965 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:25.965 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.844e-05 s, 0.0 kB/s 2023-10-09T06:34:25.965 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:25.965 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1775 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:26.021 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:26.021 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:26.021 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1735e-05 s, 0.0 kB/s 2023-10-09T06:34:26.021 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:26.021 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1776 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:26.078 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:26.079 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:26.079 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1404e-05 s, 0.0 kB/s 2023-10-09T06:34:26.079 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:26.079 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1777 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:26.134 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:26.134 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:26.134 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8231e-05 s, 0.0 kB/s 2023-10-09T06:34:26.135 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:26.135 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1778 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:26.190 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:26.190 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:26.190 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0311e-05 s, 0.0 kB/s 2023-10-09T06:34:26.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:26.190 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1779 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:26.245 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:26.245 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:26.245 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.6008e-05 s, 0.0 kB/s 2023-10-09T06:34:26.245 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:26.245 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1780 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:26.300 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:26.300 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:26.300 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.004e-05 s, 0.0 kB/s 2023-10-09T06:34:26.300 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:26.300 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1781 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:26.356 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:26.356 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:26.356 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2902e-05 s, 0.0 kB/s 2023-10-09T06:34:26.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:26.357 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1782 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:26.412 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:26.412 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:26.412 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8303e-05 s, 0.0 kB/s 2023-10-09T06:34:26.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:26.412 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1783 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:26.467 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:26.467 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:26.467 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.182e-05 s, 0.0 kB/s 2023-10-09T06:34:26.467 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:26.468 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1784 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:26.522 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:26.523 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:26.523 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0028e-05 s, 0.0 kB/s 2023-10-09T06:34:26.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:26.523 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1785 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:26.578 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:26.578 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:26.578 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8689e-05 s, 0.0 kB/s 2023-10-09T06:34:26.578 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:26.578 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1786 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:26.633 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:26.633 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:26.633 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8924e-05 s, 0.0 kB/s 2023-10-09T06:34:26.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:26.634 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1787 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:26.689 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:26.689 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:26.689 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2237e-05 s, 0.0 kB/s 2023-10-09T06:34:26.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:26.689 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1788 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:26.745 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:26.745 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:26.745 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1514e-05 s, 0.0 kB/s 2023-10-09T06:34:26.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:26.746 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1789 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:26.820 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:26.820 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:26.821 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.9218e-05 s, 0.0 kB/s 2023-10-09T06:34:26.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:26.821 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1790 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:26.876 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:26.876 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:26.876 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8958e-05 s, 0.0 kB/s 2023-10-09T06:34:26.876 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:26.877 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1791 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:26.932 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:26.932 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:26.932 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8173e-05 s, 0.0 kB/s 2023-10-09T06:34:26.932 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:26.932 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1792 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:26.988 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:26.988 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:26.988 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1661e-05 s, 0.0 kB/s 2023-10-09T06:34:26.988 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:26.988 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1793 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:27.044 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:27.044 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:27.044 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9952e-05 s, 0.0 kB/s 2023-10-09T06:34:27.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:27.044 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1794 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:27.099 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:27.099 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:27.099 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.7391e-05 s, 0.0 kB/s 2023-10-09T06:34:27.099 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:27.099 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1795 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:27.154 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:27.154 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:27.154 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8846e-05 s, 0.0 kB/s 2023-10-09T06:34:27.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:27.155 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1796 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:27.210 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:27.210 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:27.210 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7408e-05 s, 0.0 kB/s 2023-10-09T06:34:27.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:27.210 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1797 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:27.266 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:27.266 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:27.266 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.7335e-05 s, 0.0 kB/s 2023-10-09T06:34:27.266 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:27.266 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1798 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:27.321 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:27.321 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:27.321 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6904e-05 s, 0.0 kB/s 2023-10-09T06:34:27.322 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:27.322 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1799 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:27.377 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:27.377 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:27.377 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9404e-05 s, 0.0 kB/s 2023-10-09T06:34:27.378 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:27.378 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1800 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:27.433 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:27.433 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:27.433 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8997e-05 s, 0.0 kB/s 2023-10-09T06:34:27.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:27.434 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1801 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:27.488 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:27.489 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:27.489 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.292e-05 s, 0.0 kB/s 2023-10-09T06:34:27.489 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:27.489 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1802 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:27.544 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:27.544 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:27.544 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7443e-05 s, 0.0 kB/s 2023-10-09T06:34:27.544 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:27.544 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1803 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:27.599 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:27.599 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:27.599 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9071e-05 s, 0.0 kB/s 2023-10-09T06:34:27.599 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:27.599 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1804 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:27.653 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:27.653 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:27.653 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2649e-05 s, 0.0 kB/s 2023-10-09T06:34:27.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:27.654 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1805 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:27.710 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:27.710 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:27.710 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0494e-05 s, 0.0 kB/s 2023-10-09T06:34:27.710 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:27.712 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1806 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:27.766 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:27.766 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:27.766 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.833e-05 s, 0.0 kB/s 2023-10-09T06:34:27.766 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:27.766 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1807 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:27.821 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:27.821 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:27.821 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9476e-05 s, 0.0 kB/s 2023-10-09T06:34:27.822 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:27.822 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1808 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:27.877 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:27.877 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:27.877 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9692e-05 s, 0.0 kB/s 2023-10-09T06:34:27.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:27.877 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1809 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:27.933 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:27.933 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:27.933 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.942e-05 s, 0.0 kB/s 2023-10-09T06:34:27.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:27.934 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1810 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:27.988 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:27.989 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:27.989 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0916e-05 s, 0.0 kB/s 2023-10-09T06:34:27.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:27.989 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1811 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:28.044 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:28.044 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:28.044 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1263e-05 s, 0.0 kB/s 2023-10-09T06:34:28.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:28.044 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1812 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:28.100 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:28.100 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:28.100 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9228e-05 s, 0.0 kB/s 2023-10-09T06:34:28.100 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:28.101 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1813 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:28.156 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:28.156 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:28.156 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9742e-05 s, 0.0 kB/s 2023-10-09T06:34:28.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:28.157 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1814 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:28.212 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:28.213 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:28.213 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0818e-05 s, 0.0 kB/s 2023-10-09T06:34:28.213 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:28.213 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1815 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:28.269 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:28.269 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:28.269 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.09e-05 s, 0.0 kB/s 2023-10-09T06:34:28.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:28.269 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1816 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:28.325 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:28.325 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:28.325 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8341e-05 s, 0.0 kB/s 2023-10-09T06:34:28.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:28.326 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1817 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:28.380 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:28.381 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:28.381 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0705e-05 s, 0.0 kB/s 2023-10-09T06:34:28.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:28.381 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1818 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:28.437 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:28.437 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:28.437 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0548e-05 s, 0.0 kB/s 2023-10-09T06:34:28.438 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:28.438 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1819 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:28.493 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:28.493 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:28.493 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.876e-05 s, 0.0 kB/s 2023-10-09T06:34:28.493 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:28.493 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1820 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:28.549 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:28.550 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:28.550 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1579e-05 s, 0.0 kB/s 2023-10-09T06:34:28.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:28.550 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1821 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:28.605 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:28.605 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:28.606 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9639e-05 s, 0.0 kB/s 2023-10-09T06:34:28.606 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:28.606 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1822 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:28.661 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:28.661 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:28.661 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9348e-05 s, 0.0 kB/s 2023-10-09T06:34:28.661 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:28.661 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1823 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:28.716 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:28.716 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:28.716 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.08e-05 s, 0.0 kB/s 2023-10-09T06:34:28.716 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:28.717 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1824 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:28.773 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:28.773 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:28.773 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1981e-05 s, 0.0 kB/s 2023-10-09T06:34:28.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:28.774 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1825 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:28.829 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:28.829 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:28.829 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7482e-05 s, 0.0 kB/s 2023-10-09T06:34:28.829 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:28.830 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1826 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:28.885 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:28.885 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:28.885 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8337e-05 s, 0.0 kB/s 2023-10-09T06:34:28.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:28.886 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1827 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:28.941 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:28.941 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:28.941 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0201e-05 s, 0.0 kB/s 2023-10-09T06:34:28.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:28.942 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1828 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:28.996 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:28.996 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:28.996 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8679e-05 s, 0.0 kB/s 2023-10-09T06:34:28.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:28.998 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1829 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:29.053 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:29.053 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:29.053 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.853e-05 s, 0.0 kB/s 2023-10-09T06:34:29.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:29.054 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1830 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:29.109 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:29.109 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:29.109 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0365e-05 s, 0.0 kB/s 2023-10-09T06:34:29.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:29.110 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1831 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:29.167 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:29.167 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:29.167 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1811e-05 s, 0.0 kB/s 2023-10-09T06:34:29.167 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:29.168 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1832 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:29.223 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:29.223 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:29.223 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9127e-05 s, 0.0 kB/s 2023-10-09T06:34:29.223 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:29.224 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1833 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:29.279 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:29.279 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:29.279 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9201e-05 s, 0.0 kB/s 2023-10-09T06:34:29.280 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:29.280 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1834 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:29.335 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:29.335 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:29.335 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.912e-05 s, 0.0 kB/s 2023-10-09T06:34:29.335 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:29.336 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1835 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:29.391 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:29.391 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:29.391 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8223e-05 s, 0.0 kB/s 2023-10-09T06:34:29.392 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:29.392 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1836 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:29.447 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:29.447 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:29.447 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0768e-05 s, 0.0 kB/s 2023-10-09T06:34:29.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:29.448 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1837 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:29.503 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:29.503 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:29.503 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0703e-05 s, 0.0 kB/s 2023-10-09T06:34:29.504 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:29.504 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1838 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:29.559 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:29.559 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:29.559 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.951e-05 s, 0.0 kB/s 2023-10-09T06:34:29.560 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:29.560 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1839 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:29.615 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:29.615 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:29.615 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7564e-05 s, 0.0 kB/s 2023-10-09T06:34:29.616 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:29.616 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1840 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:29.671 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:29.671 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:29.671 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3759e-05 s, 0.0 kB/s 2023-10-09T06:34:29.672 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:29.672 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1841 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:29.727 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:29.727 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:29.727 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9201e-05 s, 0.0 kB/s 2023-10-09T06:34:29.728 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:29.728 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1842 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:29.784 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:29.784 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:29.784 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1561e-05 s, 0.0 kB/s 2023-10-09T06:34:29.784 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:29.785 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1843 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:29.840 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:29.841 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:29.841 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0881e-05 s, 0.0 kB/s 2023-10-09T06:34:29.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:29.841 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1844 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:29.896 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:29.896 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:29.897 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2035e-05 s, 0.0 kB/s 2023-10-09T06:34:29.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:29.897 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1845 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:29.952 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:29.952 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:29.952 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2275e-05 s, 0.0 kB/s 2023-10-09T06:34:29.952 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:29.953 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1846 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:30.008 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:30.008 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:30.008 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7102e-05 s, 0.0 kB/s 2023-10-09T06:34:30.009 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:30.009 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1847 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:30.064 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:30.064 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:30.064 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.982e-05 s, 0.0 kB/s 2023-10-09T06:34:30.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:30.065 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1848 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:30.120 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:30.120 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:30.121 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9889e-05 s, 0.0 kB/s 2023-10-09T06:34:30.121 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:30.121 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1849 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:30.182 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:30.182 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:30.183 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.169e-05 s, 0.0 kB/s 2023-10-09T06:34:30.183 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:30.183 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1850 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:30.243 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:30.243 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:30.243 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0534e-05 s, 0.0 kB/s 2023-10-09T06:34:30.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:30.244 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1851 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:30.300 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:30.301 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:30.301 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7117e-05 s, 0.0 kB/s 2023-10-09T06:34:30.301 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:30.301 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1852 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:30.360 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:30.360 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:30.361 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9791e-05 s, 0.0 kB/s 2023-10-09T06:34:30.361 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:30.361 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1853 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:30.425 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:30.426 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:30.426 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3248e-05 s, 0.0 kB/s 2023-10-09T06:34:30.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:30.427 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1854 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:30.483 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:30.483 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:30.484 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3387e-05 s, 0.0 kB/s 2023-10-09T06:34:30.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:30.484 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1855 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:30.540 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:30.540 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:30.540 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8232e-05 s, 0.0 kB/s 2023-10-09T06:34:30.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:30.541 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1856 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:30.598 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:30.598 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:30.598 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0957e-05 s, 0.0 kB/s 2023-10-09T06:34:30.599 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:30.599 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1857 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:30.655 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:30.655 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:30.655 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0971e-05 s, 0.0 kB/s 2023-10-09T06:34:30.655 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:30.655 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1858 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:30.713 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:30.713 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:30.713 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1619e-05 s, 0.0 kB/s 2023-10-09T06:34:30.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:30.714 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1859 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:30.770 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:30.770 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:30.770 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1134e-05 s, 0.0 kB/s 2023-10-09T06:34:30.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:30.771 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1860 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:30.827 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:30.827 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:30.827 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9831e-05 s, 0.0 kB/s 2023-10-09T06:34:30.827 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:30.827 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1861 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:30.883 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:30.883 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:30.883 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2976e-05 s, 0.0 kB/s 2023-10-09T06:34:30.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:30.884 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1862 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:30.939 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:30.939 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:30.939 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2342e-05 s, 0.0 kB/s 2023-10-09T06:34:30.939 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:30.940 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1863 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:30.996 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:30.996 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:30.996 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.125e-05 s, 0 B/s 2023-10-09T06:34:30.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:30.997 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1864 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:31.053 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:31.053 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:31.053 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2048e-05 s, 0.0 kB/s 2023-10-09T06:34:31.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:31.053 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1865 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:31.109 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:31.110 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:31.110 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3532e-05 s, 0.0 kB/s 2023-10-09T06:34:31.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:31.110 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1866 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:31.165 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:31.166 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:31.166 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0431e-05 s, 0.0 kB/s 2023-10-09T06:34:31.166 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:31.166 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1867 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:31.221 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:31.221 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:31.222 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0412e-05 s, 0.0 kB/s 2023-10-09T06:34:31.222 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:31.222 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1868 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:31.277 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:31.277 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:31.277 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9781e-05 s, 0.0 kB/s 2023-10-09T06:34:31.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:31.278 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1869 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:31.332 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:31.333 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:31.333 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1765e-05 s, 0.0 kB/s 2023-10-09T06:34:31.333 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:31.334 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1870 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:31.389 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:31.389 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:31.389 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 5.1716e-05 s, 0.0 kB/s 2023-10-09T06:34:31.389 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:31.389 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1871 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:31.444 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:31.444 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:31.445 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8549e-05 s, 0.0 kB/s 2023-10-09T06:34:31.445 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:31.445 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1872 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:31.499 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:31.499 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:31.499 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8285e-05 s, 0.0 kB/s 2023-10-09T06:34:31.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:31.500 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1873 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:31.556 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:31.556 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:31.557 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1567e-05 s, 0.0 kB/s 2023-10-09T06:34:31.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:31.557 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1874 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:31.612 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:31.612 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:31.612 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7854e-05 s, 0.0 kB/s 2023-10-09T06:34:31.613 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:31.613 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1875 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:31.668 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:31.668 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:31.668 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9659e-05 s, 0.0 kB/s 2023-10-09T06:34:31.668 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:31.668 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1876 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:31.724 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:31.724 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:31.724 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8501e-05 s, 0.0 kB/s 2023-10-09T06:34:31.724 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:31.724 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1877 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:31.780 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:31.781 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:31.781 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1009e-05 s, 0.0 kB/s 2023-10-09T06:34:31.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:31.781 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1878 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:31.837 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:31.837 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:31.837 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3171e-05 s, 0.0 kB/s 2023-10-09T06:34:31.837 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:31.837 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1879 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:31.892 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:31.892 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:31.893 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.106e-05 s, 0.0 kB/s 2023-10-09T06:34:31.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:31.893 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1880 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:31.948 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:31.948 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:31.948 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1002e-05 s, 0.0 kB/s 2023-10-09T06:34:31.948 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:31.949 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1881 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:32.004 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:32.004 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:32.004 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3175e-05 s, 0.0 kB/s 2023-10-09T06:34:32.004 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:32.004 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1882 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:32.059 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:32.059 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:32.059 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0627e-05 s, 0.0 kB/s 2023-10-09T06:34:32.059 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:32.059 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1883 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:32.115 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:32.115 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:32.116 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1064e-05 s, 0.0 kB/s 2023-10-09T06:34:32.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:32.116 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1884 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:32.171 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:32.171 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:32.171 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.116e-05 s, 0.0 kB/s 2023-10-09T06:34:32.171 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:32.172 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1885 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:32.226 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:32.227 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:32.227 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0834e-05 s, 0.0 kB/s 2023-10-09T06:34:32.228 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:32.228 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1886 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:32.283 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:32.283 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:32.283 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0637e-05 s, 0.0 kB/s 2023-10-09T06:34:32.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:32.284 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1887 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:32.339 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:32.340 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:32.340 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9016e-05 s, 0.0 kB/s 2023-10-09T06:34:32.340 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:32.340 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1888 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:32.395 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:32.396 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:32.396 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.272e-05 s, 0.0 kB/s 2023-10-09T06:34:32.396 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:32.396 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1889 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:32.451 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:32.451 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:32.452 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2482e-05 s, 0.0 kB/s 2023-10-09T06:34:32.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:32.452 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1890 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:32.507 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:32.507 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:32.507 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8601e-05 s, 0.0 kB/s 2023-10-09T06:34:32.507 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:32.507 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1891 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:32.563 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:32.563 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:32.564 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2576e-05 s, 0.0 kB/s 2023-10-09T06:34:32.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:32.564 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1892 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:32.618 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:32.619 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:32.619 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.113e-05 s, 0.0 kB/s 2023-10-09T06:34:32.619 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:32.619 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1893 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:32.674 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:32.674 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:32.674 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1993e-05 s, 0.0 kB/s 2023-10-09T06:34:32.675 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:32.675 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1894 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:32.730 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:32.730 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:32.730 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0241e-05 s, 0.0 kB/s 2023-10-09T06:34:32.730 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:32.730 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1895 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:32.785 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:32.785 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:32.785 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1809e-05 s, 0.0 kB/s 2023-10-09T06:34:32.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:32.786 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1896 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:32.841 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:32.842 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:32.842 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.082e-05 s, 0.0 kB/s 2023-10-09T06:34:32.842 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:32.842 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1897 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:32.897 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:32.897 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:32.897 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7718e-05 s, 0.0 kB/s 2023-10-09T06:34:32.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:32.897 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1898 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:32.992 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:32.992 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:32.992 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.0115e-05 s, 0.0 kB/s 2023-10-09T06:34:32.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:32.992 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1899 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:33.008 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:33.008 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:33.008 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1886e-05 s, 0.0 kB/s 2023-10-09T06:34:33.008 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:33.009 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1900 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:33.064 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:33.065 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:33.065 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0051e-05 s, 0.0 kB/s 2023-10-09T06:34:33.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:33.065 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1901 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:33.120 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:33.120 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:33.121 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1984e-05 s, 0.0 kB/s 2023-10-09T06:34:33.121 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:33.121 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1902 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:33.176 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:33.176 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:33.176 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9145e-05 s, 0.0 kB/s 2023-10-09T06:34:33.177 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:33.177 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1903 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:33.232 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:33.233 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:33.233 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0732e-05 s, 0.0 kB/s 2023-10-09T06:34:33.233 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:33.233 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1904 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:33.288 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:33.288 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:33.288 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0514e-05 s, 0.0 kB/s 2023-10-09T06:34:33.288 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:33.289 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1905 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:33.344 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:33.344 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:33.345 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0694e-05 s, 0.0 kB/s 2023-10-09T06:34:33.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:33.345 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1906 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:33.400 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:33.400 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:33.400 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9221e-05 s, 0.0 kB/s 2023-10-09T06:34:33.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:33.401 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1907 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:33.455 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:33.456 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:33.456 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0022e-05 s, 0.0 kB/s 2023-10-09T06:34:33.457 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:33.457 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1908 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:33.512 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:33.512 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:33.512 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7371e-05 s, 0.0 kB/s 2023-10-09T06:34:33.512 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:33.513 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1909 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:33.569 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:33.569 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:33.569 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0201e-05 s, 0.0 kB/s 2023-10-09T06:34:33.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:33.569 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1910 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:33.624 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:33.625 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:33.625 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0849e-05 s, 0.0 kB/s 2023-10-09T06:34:33.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:33.625 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1911 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:33.681 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:33.682 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:33.682 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1236e-05 s, 0.0 kB/s 2023-10-09T06:34:33.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:33.683 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1912 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:33.738 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:33.738 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:33.738 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.8525e-05 s, 0.0 kB/s 2023-10-09T06:34:33.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:33.739 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1913 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:33.795 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:33.795 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:33.795 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9153e-05 s, 0.0 kB/s 2023-10-09T06:34:33.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:33.795 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1914 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:33.851 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:33.851 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:33.851 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.7478e-05 s, 0.0 kB/s 2023-10-09T06:34:33.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:33.851 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1915 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:33.906 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:33.906 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:33.906 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8338e-05 s, 0.0 kB/s 2023-10-09T06:34:33.906 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:33.907 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1916 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:33.962 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:33.963 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:33.963 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0738e-05 s, 0.0 kB/s 2023-10-09T06:34:33.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:33.963 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1917 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:34.018 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:34.018 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:34.018 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0183e-05 s, 0.0 kB/s 2023-10-09T06:34:34.019 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:34.019 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1918 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:34.074 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:34.075 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:34.075 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9556e-05 s, 0.0 kB/s 2023-10-09T06:34:34.075 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:34.075 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1919 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:34.130 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:34.130 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:34.130 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7931e-05 s, 0.0 kB/s 2023-10-09T06:34:34.130 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:34.131 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1920 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:34.186 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:34.186 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:34.186 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0343e-05 s, 0.0 kB/s 2023-10-09T06:34:34.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:34.187 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1921 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:34.242 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:34.242 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:34.242 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2739e-05 s, 0.0 kB/s 2023-10-09T06:34:34.242 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:34.243 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1922 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:34.298 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:34.299 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:34.299 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8874e-05 s, 0.0 kB/s 2023-10-09T06:34:34.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:34.299 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1923 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:34.354 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:34.354 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:34.354 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2711e-05 s, 0.0 kB/s 2023-10-09T06:34:34.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:34.355 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1924 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:34.410 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:34.410 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:34.411 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4093e-05 s, 0.0 kB/s 2023-10-09T06:34:34.411 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:34.411 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1925 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:34.466 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:34.466 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:34.466 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3227e-05 s, 0.0 kB/s 2023-10-09T06:34:34.467 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:34.467 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1926 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:34.522 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:34.522 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:34.522 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0356e-05 s, 0.0 kB/s 2023-10-09T06:34:34.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:34.523 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1927 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:34.579 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:34.579 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:34.579 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9285e-05 s, 0.0 kB/s 2023-10-09T06:34:34.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:34.580 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1928 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:34.635 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:34.635 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:34.635 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9534e-05 s, 0.0 kB/s 2023-10-09T06:34:34.635 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:34.635 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1929 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:34.690 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:34.690 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:34.690 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9556e-05 s, 0.0 kB/s 2023-10-09T06:34:34.691 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:34.691 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1930 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:34.746 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:34.746 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:34.746 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4781e-05 s, 0.0 kB/s 2023-10-09T06:34:34.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:34.746 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1931 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:34.801 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:34.801 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:34.802 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1965e-05 s, 0.0 kB/s 2023-10-09T06:34:34.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:34.802 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1932 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:34.858 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:34.858 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:34.858 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3011e-05 s, 0.0 kB/s 2023-10-09T06:34:34.858 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:34.859 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1933 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:34.914 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:34.914 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:34.914 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0353e-05 s, 0.0 kB/s 2023-10-09T06:34:34.914 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:34.914 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1934 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:34.968 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:34.969 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:34.969 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7587e-05 s, 0.0 kB/s 2023-10-09T06:34:34.969 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:34.969 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1935 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:35.024 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:35.024 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:35.024 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4196e-05 s, 0.0 kB/s 2023-10-09T06:34:35.024 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:35.024 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1936 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:35.079 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:35.080 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:35.080 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8326e-05 s, 0.0 kB/s 2023-10-09T06:34:35.080 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:35.080 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1937 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:35.134 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:35.135 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:35.135 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8309e-05 s, 0.0 kB/s 2023-10-09T06:34:35.135 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:35.135 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1938 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:35.189 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:35.190 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:35.190 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7706e-05 s, 0.0 kB/s 2023-10-09T06:34:35.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:35.190 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1939 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:35.245 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:35.245 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:35.245 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2205e-05 s, 0.0 kB/s 2023-10-09T06:34:35.246 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:35.246 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1940 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:35.302 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:35.302 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:35.302 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1201e-05 s, 0.0 kB/s 2023-10-09T06:34:35.302 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:35.302 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1941 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:35.358 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:35.358 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:35.358 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2631e-05 s, 0.0 kB/s 2023-10-09T06:34:35.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:35.359 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1942 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:35.414 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:35.414 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:35.415 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2304e-05 s, 0.0 kB/s 2023-10-09T06:34:35.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:35.415 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1943 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:35.470 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:35.471 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:35.471 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0137e-05 s, 0.0 kB/s 2023-10-09T06:34:35.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:35.471 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1944 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:35.526 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:35.526 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:35.526 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7426e-05 s, 0.0 kB/s 2023-10-09T06:34:35.527 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:35.527 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1945 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:35.582 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:35.582 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:35.582 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0961e-05 s, 0.0 kB/s 2023-10-09T06:34:35.582 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:35.582 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1946 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:35.639 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:35.639 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:35.639 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1479e-05 s, 0.0 kB/s 2023-10-09T06:34:35.639 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:35.639 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1947 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:35.694 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:35.694 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:35.694 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3605e-05 s, 0.0 kB/s 2023-10-09T06:34:35.694 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:35.694 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1948 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:35.750 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:35.750 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:35.750 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1355e-05 s, 0.0 kB/s 2023-10-09T06:34:35.750 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:35.750 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1949 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:35.805 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:35.805 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:35.805 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.179e-05 s, 0.0 kB/s 2023-10-09T06:34:35.806 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:35.806 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1950 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:35.861 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:35.861 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:35.861 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0724e-05 s, 0.0 kB/s 2023-10-09T06:34:35.862 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:35.862 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1951 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:35.917 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:35.917 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:35.918 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.821e-05 s, 0.0 kB/s 2023-10-09T06:34:35.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:35.918 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1952 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:35.973 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:35.973 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:35.974 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0312e-05 s, 0.0 kB/s 2023-10-09T06:34:35.974 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:35.974 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1953 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:36.029 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:36.029 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:36.029 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8955e-05 s, 0.0 kB/s 2023-10-09T06:34:36.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:36.030 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1954 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:36.085 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:36.085 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:36.085 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2957e-05 s, 0.0 kB/s 2023-10-09T06:34:36.086 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:36.086 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1955 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:36.141 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:36.142 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:36.142 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0238e-05 s, 0.0 kB/s 2023-10-09T06:34:36.142 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:36.142 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1956 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:36.196 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:36.197 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:36.197 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1243e-05 s, 0.0 kB/s 2023-10-09T06:34:36.197 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:36.197 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1957 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:36.252 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:36.252 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:36.252 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.122e-05 s, 0.0 kB/s 2023-10-09T06:34:36.253 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:36.253 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1958 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:36.308 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:36.308 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:36.308 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9709e-05 s, 0.0 kB/s 2023-10-09T06:34:36.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:36.308 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1959 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:36.364 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:36.364 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:36.364 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1231e-05 s, 0.0 kB/s 2023-10-09T06:34:36.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:36.364 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1960 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:36.419 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:36.419 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:36.419 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1196e-05 s, 0.0 kB/s 2023-10-09T06:34:36.419 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:36.419 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1961 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:36.473 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:36.473 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:36.474 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2754e-05 s, 0.0 kB/s 2023-10-09T06:34:36.474 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:36.474 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1962 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:36.528 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:36.529 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:36.529 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8409e-05 s, 0.0 kB/s 2023-10-09T06:34:36.529 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:36.529 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1963 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:36.584 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:36.584 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:36.584 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2114e-05 s, 0.0 kB/s 2023-10-09T06:34:36.585 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:36.585 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1964 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:36.640 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:36.640 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:36.640 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9479e-05 s, 0.0 kB/s 2023-10-09T06:34:36.640 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:36.640 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1965 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:36.696 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:36.696 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:36.696 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1993e-05 s, 0.0 kB/s 2023-10-09T06:34:36.697 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:36.697 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1966 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:36.752 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:36.752 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:36.752 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1928e-05 s, 0.0 kB/s 2023-10-09T06:34:36.753 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:36.753 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1967 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:36.808 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:36.808 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:36.808 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.882e-05 s, 0.0 kB/s 2023-10-09T06:34:36.808 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:36.809 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1968 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:36.864 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:36.864 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:36.864 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0263e-05 s, 0.0 kB/s 2023-10-09T06:34:36.864 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:36.864 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1969 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:36.920 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:36.920 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:36.920 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1303e-05 s, 0.0 kB/s 2023-10-09T06:34:36.921 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:36.921 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1970 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:36.975 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:36.976 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:36.976 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3414e-05 s, 0.0 kB/s 2023-10-09T06:34:36.976 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:36.976 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1971 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:37.031 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:37.031 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:37.032 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1983e-05 s, 0.0 kB/s 2023-10-09T06:34:37.032 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:37.032 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1972 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:37.087 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:37.087 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:37.088 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.984e-05 s, 0.0 kB/s 2023-10-09T06:34:37.088 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:37.088 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1973 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:37.143 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:37.144 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:37.144 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.097e-05 s, 0.0 kB/s 2023-10-09T06:34:37.144 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:37.144 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1974 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:37.199 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:37.199 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:37.199 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.025e-05 s, 0.0 kB/s 2023-10-09T06:34:37.200 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:37.200 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1975 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:37.255 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:37.255 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:37.255 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0503e-05 s, 0.0 kB/s 2023-10-09T06:34:37.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:37.256 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1976 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:37.311 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:37.311 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:37.311 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0303e-05 s, 0.0 kB/s 2023-10-09T06:34:37.311 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:37.312 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1977 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:37.367 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:37.367 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:37.367 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9934e-05 s, 0.0 kB/s 2023-10-09T06:34:37.367 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:37.368 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1978 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:37.423 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:37.423 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:37.423 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2857e-05 s, 0.0 kB/s 2023-10-09T06:34:37.423 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:37.424 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1979 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:37.479 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:37.479 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:37.479 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1398e-05 s, 0.0 kB/s 2023-10-09T06:34:37.479 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:37.480 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1980 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:37.535 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:37.535 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:37.535 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9986e-05 s, 0.0 kB/s 2023-10-09T06:34:37.535 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:37.535 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1981 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:37.590 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:37.590 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:37.590 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.105e-05 s, 0.0 kB/s 2023-10-09T06:34:37.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:37.590 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1982 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:37.645 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:37.645 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:37.645 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4522e-05 s, 0.0 kB/s 2023-10-09T06:34:37.646 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:37.646 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1983 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:37.701 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:37.701 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:37.701 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9432e-05 s, 0.0 kB/s 2023-10-09T06:34:37.701 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:37.701 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1984 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:37.757 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:37.757 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:37.757 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3668e-05 s, 0.0 kB/s 2023-10-09T06:34:37.758 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:37.758 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1985 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:37.813 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:37.813 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:37.813 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9196e-05 s, 0.0 kB/s 2023-10-09T06:34:37.814 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:37.814 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1986 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:37.869 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:37.869 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:37.869 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9942e-05 s, 0.0 kB/s 2023-10-09T06:34:37.869 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:37.870 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1987 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:37.925 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:37.925 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:37.925 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0547e-05 s, 0.0 kB/s 2023-10-09T06:34:37.925 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:37.925 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1988 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:37.981 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:37.981 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:37.981 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9042e-05 s, 0.0 kB/s 2023-10-09T06:34:37.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:37.982 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1989 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:38.037 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:38.037 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:38.037 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9195e-05 s, 0.0 kB/s 2023-10-09T06:34:38.037 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:38.037 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1990 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:38.093 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:38.093 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:38.093 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9643e-05 s, 0.0 kB/s 2023-10-09T06:34:38.094 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:38.094 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1991 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:38.149 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:38.149 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:38.149 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9122e-05 s, 0.0 kB/s 2023-10-09T06:34:38.149 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:38.149 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1992 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:38.205 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:38.205 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:38.205 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0947e-05 s, 0.0 kB/s 2023-10-09T06:34:38.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:38.205 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1993 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:38.260 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:38.260 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:38.260 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8517e-05 s, 0.0 kB/s 2023-10-09T06:34:38.260 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:38.261 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1994 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:38.315 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:38.316 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:38.316 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.836e-05 s, 0.0 kB/s 2023-10-09T06:34:38.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:38.316 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1995 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:38.371 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:38.371 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:38.371 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0425e-05 s, 0.0 kB/s 2023-10-09T06:34:38.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:38.371 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1996 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:38.426 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:38.427 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:38.427 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8227e-05 s, 0.0 kB/s 2023-10-09T06:34:38.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:38.427 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1997 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:38.481 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:38.481 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:38.482 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8794e-05 s, 0.0 kB/s 2023-10-09T06:34:38.482 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:38.482 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1998 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:38.537 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:38.537 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:38.537 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0306e-05 s, 0.0 kB/s 2023-10-09T06:34:38.538 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:38.538 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.1999 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:38.602 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:38.602 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:38.603 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4799e-05 s, 0.0 kB/s 2023-10-09T06:34:38.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:38.603 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2000 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:38.658 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:38.658 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:38.658 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.875e-05 s, 0.0 kB/s 2023-10-09T06:34:38.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:38.658 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2001 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:38.713 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:38.714 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:38.714 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.972e-05 s, 0.0 kB/s 2023-10-09T06:34:38.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:38.714 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2002 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:38.769 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:38.769 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:38.769 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9638e-05 s, 0.0 kB/s 2023-10-09T06:34:38.770 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:38.770 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2003 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:38.825 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:38.825 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:38.825 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9129e-05 s, 0.0 kB/s 2023-10-09T06:34:38.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:38.825 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2004 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:38.880 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:38.880 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:38.881 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1045e-05 s, 0.0 kB/s 2023-10-09T06:34:38.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:38.881 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2005 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:38.937 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:38.937 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:38.938 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8561e-05 s, 0.0 kB/s 2023-10-09T06:34:38.938 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:38.938 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2006 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:38.994 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:38.994 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:38.994 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1143e-05 s, 0.0 kB/s 2023-10-09T06:34:38.994 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:38.994 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2007 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:39.049 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:39.049 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:39.050 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1414e-05 s, 0.0 kB/s 2023-10-09T06:34:39.050 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:39.050 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2008 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:39.104 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:39.105 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:39.105 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.9163e-05 s, 0.0 kB/s 2023-10-09T06:34:39.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:39.105 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2009 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:39.160 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:39.160 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:39.160 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9578e-05 s, 0.0 kB/s 2023-10-09T06:34:39.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:39.160 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2010 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:39.214 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:39.215 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:39.215 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0379e-05 s, 0.0 kB/s 2023-10-09T06:34:39.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:39.216 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2011 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:39.271 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:39.271 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:39.271 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8995e-05 s, 0.0 kB/s 2023-10-09T06:34:39.271 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:39.272 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2012 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:39.327 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:39.327 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:39.327 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0471e-05 s, 0.0 kB/s 2023-10-09T06:34:39.327 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:39.328 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2013 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:39.382 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:39.382 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:39.382 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7759e-05 s, 0.0 kB/s 2023-10-09T06:34:39.383 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:39.383 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2014 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:39.438 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:39.438 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:39.439 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9228e-05 s, 0.0 kB/s 2023-10-09T06:34:39.439 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:39.439 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2015 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:39.494 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:39.494 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:39.494 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9186e-05 s, 0.0 kB/s 2023-10-09T06:34:39.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:39.494 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2016 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:39.550 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:39.550 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:39.550 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.158e-05 s, 0.0 kB/s 2023-10-09T06:34:39.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:39.551 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2017 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:39.606 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:39.606 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:39.606 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0271e-05 s, 0.0 kB/s 2023-10-09T06:34:39.606 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:39.607 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2018 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:39.662 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:39.662 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:39.662 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7772e-05 s, 0.0 kB/s 2023-10-09T06:34:39.662 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:39.662 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2019 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:39.716 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:39.716 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:39.716 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2236e-05 s, 0.0 kB/s 2023-10-09T06:34:39.717 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:39.717 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2020 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:39.773 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:39.774 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:39.774 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.409e-05 s, 0.0 kB/s 2023-10-09T06:34:39.774 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:39.774 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2021 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:39.829 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:39.829 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:39.829 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2599e-05 s, 0.0 kB/s 2023-10-09T06:34:39.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:39.830 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2022 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:39.885 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:39.885 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:39.886 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1779e-05 s, 0.0 kB/s 2023-10-09T06:34:39.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:39.886 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2023 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:39.941 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:39.941 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:39.941 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3245e-05 s, 0.0 kB/s 2023-10-09T06:34:39.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:39.942 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2024 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:39.996 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:39.996 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:39.996 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0081e-05 s, 0.0 kB/s 2023-10-09T06:34:39.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:39.997 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2025 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:40.053 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:40.053 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:40.053 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8849e-05 s, 0.0 kB/s 2023-10-09T06:34:40.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:40.053 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2026 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:40.107 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:40.108 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:40.108 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7021e-05 s, 0.0 kB/s 2023-10-09T06:34:40.108 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:40.108 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2027 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:40.163 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:40.163 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:40.163 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8458e-05 s, 0.0 kB/s 2023-10-09T06:34:40.163 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:40.163 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2028 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:40.219 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:40.219 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:40.220 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1735e-05 s, 0.0 kB/s 2023-10-09T06:34:40.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:40.220 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2029 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:40.274 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:40.274 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:40.274 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1811e-05 s, 0.0 kB/s 2023-10-09T06:34:40.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:40.275 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2030 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:40.330 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:40.330 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:40.331 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8506e-05 s, 0.0 kB/s 2023-10-09T06:34:40.331 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:40.331 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2031 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:40.386 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:40.386 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:40.387 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9888e-05 s, 0.0 kB/s 2023-10-09T06:34:40.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:40.387 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2032 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:40.441 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:40.442 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:40.442 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8861e-05 s, 0.0 kB/s 2023-10-09T06:34:40.442 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:40.442 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2033 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:40.496 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:40.497 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:40.497 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2723e-05 s, 0.0 kB/s 2023-10-09T06:34:40.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:40.497 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2034 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:40.552 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:40.552 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:40.552 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9701e-05 s, 0.0 kB/s 2023-10-09T06:34:40.552 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:40.552 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2035 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:40.607 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:40.607 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:40.607 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0422e-05 s, 0.0 kB/s 2023-10-09T06:34:40.608 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:40.608 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2036 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:40.663 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:40.663 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:40.663 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9251e-05 s, 0.0 kB/s 2023-10-09T06:34:40.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:40.664 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2037 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:40.718 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:40.719 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:40.719 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0506e-05 s, 0.0 kB/s 2023-10-09T06:34:40.719 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:40.719 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2038 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:40.775 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:40.775 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:40.775 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1452e-05 s, 0.0 kB/s 2023-10-09T06:34:40.775 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:40.776 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2039 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:40.831 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:40.831 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:40.831 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2132e-05 s, 0.0 kB/s 2023-10-09T06:34:40.832 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:40.832 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2040 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:40.888 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:40.888 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:40.889 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9255e-05 s, 0.0 kB/s 2023-10-09T06:34:40.889 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:40.889 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2041 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:40.944 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:40.944 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:40.945 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1119e-05 s, 0.0 kB/s 2023-10-09T06:34:40.945 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:40.945 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2042 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:41.000 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:41.001 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:41.001 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0756e-05 s, 0.0 kB/s 2023-10-09T06:34:41.001 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:41.001 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2043 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:41.057 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:41.057 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:41.057 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0368e-05 s, 0.0 kB/s 2023-10-09T06:34:41.058 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:41.059 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2044 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:41.113 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:41.113 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:41.113 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4987e-05 s, 0.0 kB/s 2023-10-09T06:34:41.114 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:41.114 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2045 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:41.170 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:41.170 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:41.170 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9905e-05 s, 0.0 kB/s 2023-10-09T06:34:41.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:41.170 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2046 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:41.226 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:41.226 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:41.226 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7302e-05 s, 0.0 kB/s 2023-10-09T06:34:41.226 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:41.226 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2047 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:41.281 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:41.281 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:41.281 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1504e-05 s, 0.0 kB/s 2023-10-09T06:34:41.281 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:41.281 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2048 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:41.336 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:41.337 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:41.337 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9935e-05 s, 0.0 kB/s 2023-10-09T06:34:41.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:41.337 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2049 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:41.392 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:41.392 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:41.392 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.527e-05 s, 0.0 kB/s 2023-10-09T06:34:41.392 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:41.393 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2050 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:41.448 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:41.448 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:41.448 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8793e-05 s, 0.0 kB/s 2023-10-09T06:34:41.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:41.448 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2051 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:41.504 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:41.504 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:41.505 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1631e-05 s, 0.0 kB/s 2023-10-09T06:34:41.505 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:41.505 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2052 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:41.560 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:41.560 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:41.560 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8762e-05 s, 0.0 kB/s 2023-10-09T06:34:41.560 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:41.561 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2053 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:41.616 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:41.616 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:41.616 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9471e-05 s, 0.0 kB/s 2023-10-09T06:34:41.616 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:41.617 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2054 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:41.672 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:41.672 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:41.672 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9452e-05 s, 0.0 kB/s 2023-10-09T06:34:41.673 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:41.673 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2055 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:41.727 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:41.728 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:41.728 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1742e-05 s, 0.0 kB/s 2023-10-09T06:34:41.729 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:41.729 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2056 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:41.784 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:41.785 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:41.785 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.847e-05 s, 0.0 kB/s 2023-10-09T06:34:41.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:41.785 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2057 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:41.840 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:41.840 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:41.840 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3871e-05 s, 0.0 kB/s 2023-10-09T06:34:41.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:41.841 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2058 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:41.896 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:41.897 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:41.897 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0588e-05 s, 0.0 kB/s 2023-10-09T06:34:41.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:41.897 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2059 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:41.952 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:41.952 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:41.953 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2131e-05 s, 0.0 kB/s 2023-10-09T06:34:41.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:41.953 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2060 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:42.008 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:42.008 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:42.008 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.946e-05 s, 0.0 kB/s 2023-10-09T06:34:42.008 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:42.009 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2061 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:42.064 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:42.064 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:42.064 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3772e-05 s, 0.0 kB/s 2023-10-09T06:34:42.064 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:42.064 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2062 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:42.119 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:42.119 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:42.119 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7429e-05 s, 0.0 kB/s 2023-10-09T06:34:42.120 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:42.120 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2063 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:42.175 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:42.175 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:42.175 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.208e-05 s, 0.0 kB/s 2023-10-09T06:34:42.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:42.175 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2064 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:42.230 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:42.230 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:42.230 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.354e-05 s, 0.0 kB/s 2023-10-09T06:34:42.230 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:42.231 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2065 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:42.286 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:42.286 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:42.286 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.5589e-05 s, 0.0 kB/s 2023-10-09T06:34:42.287 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:42.287 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2066 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:42.341 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:42.342 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:42.342 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0378e-05 s, 0.0 kB/s 2023-10-09T06:34:42.342 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:42.342 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2067 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:42.398 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:42.398 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:42.398 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1567e-05 s, 0.0 kB/s 2023-10-09T06:34:42.399 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:42.399 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2068 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:42.454 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:42.454 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:42.454 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9663e-05 s, 0.0 kB/s 2023-10-09T06:34:42.454 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:42.455 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2069 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:42.509 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:42.510 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:42.510 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6883e-05 s, 0.0 kB/s 2023-10-09T06:34:42.510 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:42.510 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2070 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:42.564 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:42.565 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:42.565 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0877e-05 s, 0.0 kB/s 2023-10-09T06:34:42.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:42.565 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2071 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:42.620 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:42.620 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:42.620 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.7288e-05 s, 0.0 kB/s 2023-10-09T06:34:42.620 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:42.621 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2072 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:42.676 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:42.676 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:42.676 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9269e-05 s, 0.0 kB/s 2023-10-09T06:34:42.676 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:42.676 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2073 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:42.731 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:42.732 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:42.732 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0338e-05 s, 0.0 kB/s 2023-10-09T06:34:42.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:42.733 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2074 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:42.787 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:42.788 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:42.788 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.34e-05 s, 0.0 kB/s 2023-10-09T06:34:42.789 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:42.789 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2075 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:42.844 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:42.845 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:42.845 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9561e-05 s, 0.0 kB/s 2023-10-09T06:34:42.845 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:42.845 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2076 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:42.900 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:42.900 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:42.900 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9839e-05 s, 0.0 kB/s 2023-10-09T06:34:42.900 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:42.901 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2077 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:42.956 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:42.956 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:42.956 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1181e-05 s, 0.0 kB/s 2023-10-09T06:34:42.956 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:42.956 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2078 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:43.012 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:43.012 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:43.012 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8815e-05 s, 0.0 kB/s 2023-10-09T06:34:43.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:43.013 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2079 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:43.068 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:43.068 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:43.068 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8099e-05 s, 0.0 kB/s 2023-10-09T06:34:43.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:43.069 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2080 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:43.124 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:43.124 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:43.124 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0322e-05 s, 0.0 kB/s 2023-10-09T06:34:43.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:43.124 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2081 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:43.180 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:43.180 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:43.180 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2484e-05 s, 0.0 kB/s 2023-10-09T06:34:43.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:43.180 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2082 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:43.237 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:43.238 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:43.238 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.046e-05 s, 0.0 kB/s 2023-10-09T06:34:43.238 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:43.238 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2083 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:43.293 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:43.293 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:43.293 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1803e-05 s, 0.0 kB/s 2023-10-09T06:34:43.294 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:43.294 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2084 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:43.349 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:43.349 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:43.349 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8216e-05 s, 0.0 kB/s 2023-10-09T06:34:43.349 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:43.349 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2085 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:43.405 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:43.405 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:43.405 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8688e-05 s, 0.0 kB/s 2023-10-09T06:34:43.405 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:43.405 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2086 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:43.460 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:43.460 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:43.460 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7365e-05 s, 0.0 kB/s 2023-10-09T06:34:43.460 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:43.460 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2087 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:43.480 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:34:43.503 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:34:43.516 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:43.516 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:43.516 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9914e-05 s, 0.0 kB/s 2023-10-09T06:34:43.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:43.517 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2088 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:43.572 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:43.572 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:43.573 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2971e-05 s, 0.0 kB/s 2023-10-09T06:34:43.573 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:43.573 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2089 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:43.628 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:43.628 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:43.628 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9257e-05 s, 0.0 kB/s 2023-10-09T06:34:43.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:43.629 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2090 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:43.718 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:43.719 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:43.719 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.2877e-05 s, 0.0 kB/s 2023-10-09T06:34:43.719 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:43.719 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2091 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:43.735 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:43.735 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:43.735 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9491e-05 s, 0.0 kB/s 2023-10-09T06:34:43.735 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:43.735 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2092 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:43.791 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:43.791 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:43.791 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.358e-05 s, 0.0 kB/s 2023-10-09T06:34:43.792 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:43.792 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2093 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:43.847 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:43.847 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:43.847 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0155e-05 s, 0.0 kB/s 2023-10-09T06:34:43.847 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:43.847 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2094 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:43.903 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:43.903 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:43.903 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0962e-05 s, 0.0 kB/s 2023-10-09T06:34:43.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:43.904 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2095 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:43.959 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:43.959 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:43.959 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0997e-05 s, 0.0 kB/s 2023-10-09T06:34:43.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:43.959 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2096 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:44.015 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:44.015 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:44.015 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1251e-05 s, 0.0 kB/s 2023-10-09T06:34:44.015 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:44.015 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2097 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:44.071 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:44.071 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:44.071 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3141e-05 s, 0.0 kB/s 2023-10-09T06:34:44.071 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:44.071 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2098 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:44.127 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:44.127 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:44.127 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.8148e-05 s, 0.0 kB/s 2023-10-09T06:34:44.127 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:44.127 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2099 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:44.181 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:44.181 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:44.181 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0967e-05 s, 0.0 kB/s 2023-10-09T06:34:44.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:44.182 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2100 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:44.238 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:44.238 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:44.238 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9916e-05 s, 0.0 kB/s 2023-10-09T06:34:44.238 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:44.239 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2101 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:44.294 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:44.294 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:44.294 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2693e-05 s, 0.0 kB/s 2023-10-09T06:34:44.294 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:44.294 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2102 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:44.350 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:44.350 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:44.350 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2574e-05 s, 0.0 kB/s 2023-10-09T06:34:44.350 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:44.350 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2103 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:44.406 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:44.406 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:44.406 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9687e-05 s, 0.0 kB/s 2023-10-09T06:34:44.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:44.406 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2104 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:44.462 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:44.462 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:44.462 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1626e-05 s, 0.0 kB/s 2023-10-09T06:34:44.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:44.463 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2105 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:44.518 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:44.518 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:44.518 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9413e-05 s, 0.0 kB/s 2023-10-09T06:34:44.518 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:44.518 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2106 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:44.572 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:44.572 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:44.572 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1664e-05 s, 0.0 kB/s 2023-10-09T06:34:44.573 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:44.573 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2107 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:44.629 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:44.629 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:44.629 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.915e-05 s, 0.0 kB/s 2023-10-09T06:34:44.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:44.629 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2108 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:44.683 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:44.683 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:44.683 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9921e-05 s, 0.0 kB/s 2023-10-09T06:34:44.684 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:44.684 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2109 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:44.739 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:44.739 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:44.739 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9717e-05 s, 0.0 kB/s 2023-10-09T06:34:44.739 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:44.739 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2110 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:44.795 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:44.795 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:44.796 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.392e-05 s, 0.0 kB/s 2023-10-09T06:34:44.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:44.796 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2111 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:44.851 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:44.851 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:44.851 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9899e-05 s, 0.0 kB/s 2023-10-09T06:34:44.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:44.851 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2112 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:44.907 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:44.907 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:44.907 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8858e-05 s, 0.0 kB/s 2023-10-09T06:34:44.908 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:44.908 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2113 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:44.963 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:44.963 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:44.963 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2644e-05 s, 0.0 kB/s 2023-10-09T06:34:44.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:44.963 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2114 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:45.019 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:45.019 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:45.019 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9566e-05 s, 0.0 kB/s 2023-10-09T06:34:45.019 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:45.019 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2115 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:45.075 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:45.075 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:45.075 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0197e-05 s, 0.0 kB/s 2023-10-09T06:34:45.075 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:45.075 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2116 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:45.130 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:45.130 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:45.130 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9468e-05 s, 0.0 kB/s 2023-10-09T06:34:45.131 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:45.131 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2117 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:45.186 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:45.186 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:45.186 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7868e-05 s, 0.0 kB/s 2023-10-09T06:34:45.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:45.187 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2118 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:45.242 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:45.242 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:45.242 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8123e-05 s, 0.0 kB/s 2023-10-09T06:34:45.242 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:45.242 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2119 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:45.298 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:45.298 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:45.298 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1866e-05 s, 0.0 kB/s 2023-10-09T06:34:45.298 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:45.298 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2120 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:45.353 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:45.353 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:45.353 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.063e-05 s, 0.0 kB/s 2023-10-09T06:34:45.353 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:45.353 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2121 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:45.407 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:45.408 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:45.408 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3139e-05 s, 0.0 kB/s 2023-10-09T06:34:45.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:45.408 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2122 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:45.463 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:45.463 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:45.463 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6919e-05 s, 0.0 kB/s 2023-10-09T06:34:45.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:45.463 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2123 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:45.519 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:45.519 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:45.519 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8295e-05 s, 0.0 kB/s 2023-10-09T06:34:45.519 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:45.519 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2124 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:45.574 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:45.574 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:45.574 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0581e-05 s, 0.0 kB/s 2023-10-09T06:34:45.574 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:45.574 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2125 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:45.628 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:45.629 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:45.629 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0609e-05 s, 0.0 kB/s 2023-10-09T06:34:45.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:45.629 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2126 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:45.684 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:45.684 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:45.684 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7447e-05 s, 0.0 kB/s 2023-10-09T06:34:45.685 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:45.685 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2127 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:45.740 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:45.740 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:45.740 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9637e-05 s, 0.0 kB/s 2023-10-09T06:34:45.740 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:45.741 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2128 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:45.801 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:45.801 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:45.801 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2308e-05 s, 0.0 kB/s 2023-10-09T06:34:45.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:45.802 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2129 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:45.887 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:45.887 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:45.888 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3592e-05 s, 0.0 kB/s 2023-10-09T06:34:45.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:45.888 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2130 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:45.953 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:45.953 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:45.953 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1498e-05 s, 0.0 kB/s 2023-10-09T06:34:45.955 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:45.955 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2131 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:46.018 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:46.019 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:46.019 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1408e-05 s, 0.0 kB/s 2023-10-09T06:34:46.019 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:46.019 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2132 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:46.077 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:46.077 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:46.077 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0516e-05 s, 0.0 kB/s 2023-10-09T06:34:46.078 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:46.078 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2133 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:46.139 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:46.140 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:46.140 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8811e-05 s, 0.0 kB/s 2023-10-09T06:34:46.140 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:46.140 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2134 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:46.198 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:46.198 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:46.198 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8205e-05 s, 0.0 kB/s 2023-10-09T06:34:46.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:46.199 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2135 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:46.260 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:46.261 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:46.261 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1502e-05 s, 0.0 kB/s 2023-10-09T06:34:46.261 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:46.262 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2136 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:46.319 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:46.320 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:46.320 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1312e-05 s, 0.0 kB/s 2023-10-09T06:34:46.320 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:46.320 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2137 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:46.375 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:46.375 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:46.375 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9746e-05 s, 0.0 kB/s 2023-10-09T06:34:46.375 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:46.376 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2138 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:46.432 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:46.432 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:46.432 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0781e-05 s, 0.0 kB/s 2023-10-09T06:34:46.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:46.433 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2139 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:46.489 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:46.489 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:46.489 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1107e-05 s, 0.0 kB/s 2023-10-09T06:34:46.489 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:46.489 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2140 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:46.545 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:46.546 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:46.546 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0396e-05 s, 0.0 kB/s 2023-10-09T06:34:46.546 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:46.546 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2141 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:46.602 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:46.602 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:46.602 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.835e-05 s, 0.0 kB/s 2023-10-09T06:34:46.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:46.603 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2142 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:46.658 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:46.659 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:46.659 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1207e-05 s, 0.0 kB/s 2023-10-09T06:34:46.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:46.659 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2143 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:46.714 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:46.714 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:46.714 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4173e-05 s, 0.0 kB/s 2023-10-09T06:34:46.717 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:46.717 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2144 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:46.771 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:46.772 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:46.772 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1571e-05 s, 0.0 kB/s 2023-10-09T06:34:46.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:46.772 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2145 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:46.827 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:46.828 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:46.828 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2326e-05 s, 0.0 kB/s 2023-10-09T06:34:46.828 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:46.828 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2146 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:46.883 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:46.883 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:46.883 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1991e-05 s, 0.0 kB/s 2023-10-09T06:34:46.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:46.884 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2147 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:46.940 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:46.940 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:46.940 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3261e-05 s, 0.0 kB/s 2023-10-09T06:34:46.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:46.941 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2148 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:46.996 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:46.996 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:46.996 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2951e-05 s, 0.0 kB/s 2023-10-09T06:34:46.996 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:46.997 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2149 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:47.052 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:47.052 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:47.052 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0596e-05 s, 0.0 kB/s 2023-10-09T06:34:47.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:47.052 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2150 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:47.107 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:47.107 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:47.107 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1469e-05 s, 0.0 kB/s 2023-10-09T06:34:47.108 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:47.108 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2151 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:47.164 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:47.164 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:47.164 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.961e-05 s, 0.0 kB/s 2023-10-09T06:34:47.164 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:47.164 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2152 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:47.218 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:47.219 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:47.219 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0282e-05 s, 0.0 kB/s 2023-10-09T06:34:47.219 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:47.220 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2153 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:47.274 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:47.275 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:47.275 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9172e-05 s, 0.0 kB/s 2023-10-09T06:34:47.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:47.275 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2154 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:47.330 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:47.330 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:47.330 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0877e-05 s, 0.0 kB/s 2023-10-09T06:34:47.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:47.330 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2155 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:47.385 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:47.385 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:47.385 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0541e-05 s, 0.0 kB/s 2023-10-09T06:34:47.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:47.386 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2156 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:47.440 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:47.440 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:47.440 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1244e-05 s, 0.0 kB/s 2023-10-09T06:34:47.441 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:47.441 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2157 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:47.496 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:47.496 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:47.496 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9979e-05 s, 0.0 kB/s 2023-10-09T06:34:47.496 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:47.497 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2158 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:47.552 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:47.552 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:47.552 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3456e-05 s, 0.0 kB/s 2023-10-09T06:34:47.553 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:47.553 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2159 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:47.607 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:47.607 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:47.608 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0789e-05 s, 0.0 kB/s 2023-10-09T06:34:47.608 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:47.609 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2160 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:47.664 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:47.664 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:47.664 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9854e-05 s, 0.0 kB/s 2023-10-09T06:34:47.664 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:47.664 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2161 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:47.719 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:47.719 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:47.719 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.993e-05 s, 0.0 kB/s 2023-10-09T06:34:47.719 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:47.719 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2162 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:47.774 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:47.775 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:47.775 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2307e-05 s, 0.0 kB/s 2023-10-09T06:34:47.775 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:47.775 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2163 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:47.830 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:47.830 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:47.830 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2118e-05 s, 0.0 kB/s 2023-10-09T06:34:47.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:47.830 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2164 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:47.885 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:47.885 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:47.885 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.966e-05 s, 0.0 kB/s 2023-10-09T06:34:47.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:47.886 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2165 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:47.941 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:47.941 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:47.941 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0957e-05 s, 0.0 kB/s 2023-10-09T06:34:47.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:47.942 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2166 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:47.997 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:47.997 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:47.997 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1026e-05 s, 0.0 kB/s 2023-10-09T06:34:47.998 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:47.998 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2167 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:48.053 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:48.053 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:48.053 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3887e-05 s, 0.0 kB/s 2023-10-09T06:34:48.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:48.053 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2168 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:48.107 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:48.107 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:48.107 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0298e-05 s, 0.0 kB/s 2023-10-09T06:34:48.108 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:48.109 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2169 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:48.164 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:48.164 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:48.164 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.074e-05 s, 0.0 kB/s 2023-10-09T06:34:48.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:48.165 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2170 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:48.220 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:48.220 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:48.220 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1348e-05 s, 0.0 kB/s 2023-10-09T06:34:48.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:48.220 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2171 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:48.276 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:48.276 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:48.276 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0824e-05 s, 0.0 kB/s 2023-10-09T06:34:48.276 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:48.276 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2172 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:48.331 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:48.332 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:48.332 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8171e-05 s, 0.0 kB/s 2023-10-09T06:34:48.333 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:48.333 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2173 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:48.387 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:48.387 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:48.387 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.7446e-05 s, 0.0 kB/s 2023-10-09T06:34:48.388 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:48.388 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2174 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:48.444 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:48.444 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:48.444 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2279e-05 s, 0.0 kB/s 2023-10-09T06:34:48.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:48.444 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2175 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:48.499 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:48.499 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:48.499 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8125e-05 s, 0.0 kB/s 2023-10-09T06:34:48.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:48.499 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2176 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:48.554 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:48.554 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:48.554 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8984e-05 s, 0.0 kB/s 2023-10-09T06:34:48.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:48.554 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2177 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:48.610 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:48.610 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:48.610 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1559e-05 s, 0.0 kB/s 2023-10-09T06:34:48.610 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:48.610 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2178 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:48.665 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:48.666 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:48.666 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8713e-05 s, 0.0 kB/s 2023-10-09T06:34:48.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:48.666 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2179 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:48.721 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:48.721 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:48.721 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1468e-05 s, 0.0 kB/s 2023-10-09T06:34:48.722 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:48.722 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2180 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:48.777 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:48.777 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:48.777 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0078e-05 s, 0.0 kB/s 2023-10-09T06:34:48.778 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:48.779 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2181 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:48.837 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:48.837 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:48.837 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3686e-05 s, 0.0 kB/s 2023-10-09T06:34:48.838 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:48.838 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2182 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:48.893 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:48.893 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:48.893 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8579e-05 s, 0.0 kB/s 2023-10-09T06:34:48.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:48.894 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2183 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:48.948 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:48.949 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:48.949 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4745e-05 s, 0.0 kB/s 2023-10-09T06:34:48.949 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:48.949 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2184 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:49.004 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:49.004 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:49.005 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9506e-05 s, 0.0 kB/s 2023-10-09T06:34:49.005 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:49.005 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2185 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:49.060 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:49.060 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:49.060 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9236e-05 s, 0.0 kB/s 2023-10-09T06:34:49.060 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:49.061 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2186 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:49.116 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:49.116 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:49.116 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8542e-05 s, 0.0 kB/s 2023-10-09T06:34:49.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:49.116 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2187 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:49.172 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:49.172 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:49.172 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8257e-05 s, 0.0 kB/s 2023-10-09T06:34:49.173 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:49.173 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2188 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:49.228 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:49.228 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:49.229 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6927e-05 s, 0.0 kB/s 2023-10-09T06:34:49.229 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:49.229 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2189 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:49.284 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:49.284 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:49.284 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2413e-05 s, 0.0 kB/s 2023-10-09T06:34:49.285 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:49.285 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2190 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:49.340 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:49.341 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:49.341 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9842e-05 s, 0.0 kB/s 2023-10-09T06:34:49.341 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:49.341 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2191 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:49.396 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:49.396 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:49.396 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0073e-05 s, 0.0 kB/s 2023-10-09T06:34:49.396 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:49.397 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2192 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:49.452 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:49.452 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:49.453 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2251e-05 s, 0.0 kB/s 2023-10-09T06:34:49.453 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:49.453 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2193 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:49.508 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:49.508 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:49.508 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7381e-05 s, 0.0 kB/s 2023-10-09T06:34:49.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:49.508 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2194 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:49.563 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:49.563 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:49.563 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.796e-05 s, 0.0 kB/s 2023-10-09T06:34:49.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:49.566 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2195 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:49.618 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:49.619 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:49.619 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.048e-05 s, 0.0 kB/s 2023-10-09T06:34:49.619 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:49.620 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2196 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:49.674 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:49.674 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:49.674 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8976e-05 s, 0.0 kB/s 2023-10-09T06:34:49.675 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:49.675 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2197 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:49.731 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:49.731 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:49.731 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9725e-05 s, 0.0 kB/s 2023-10-09T06:34:49.731 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:49.732 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2198 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:49.787 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:49.787 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:49.787 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3896e-05 s, 0.0 kB/s 2023-10-09T06:34:49.788 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:49.788 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2199 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:49.843 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:49.843 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:49.843 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.999e-05 s, 0.0 kB/s 2023-10-09T06:34:49.843 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:49.843 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2200 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:49.899 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:49.899 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:49.899 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9263e-05 s, 0.0 kB/s 2023-10-09T06:34:49.899 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:49.899 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2201 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:49.955 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:49.955 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:49.955 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9036e-05 s, 0.0 kB/s 2023-10-09T06:34:49.956 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:49.956 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2202 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:50.011 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:50.011 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:50.012 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1976e-05 s, 0.0 kB/s 2023-10-09T06:34:50.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:50.012 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2203 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:50.067 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:50.067 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:50.067 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8889e-05 s, 0.0 kB/s 2023-10-09T06:34:50.067 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:50.068 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2204 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:50.123 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:50.123 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:50.123 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0347e-05 s, 0.0 kB/s 2023-10-09T06:34:50.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:50.124 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2205 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:50.179 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:50.179 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:50.179 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8076e-05 s, 0.0 kB/s 2023-10-09T06:34:50.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:50.180 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2206 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:50.234 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:50.234 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:50.235 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0164e-05 s, 0.0 kB/s 2023-10-09T06:34:50.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:50.236 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2207 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:50.291 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:50.291 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:50.291 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9841e-05 s, 0.0 kB/s 2023-10-09T06:34:50.291 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:50.291 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2208 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:50.346 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:50.346 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:50.346 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2343e-05 s, 0.0 kB/s 2023-10-09T06:34:50.346 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:50.346 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2209 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:50.402 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:50.402 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:50.403 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7463e-05 s, 0.0 kB/s 2023-10-09T06:34:50.403 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:50.403 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2210 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:50.458 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:50.458 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:50.458 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9036e-05 s, 0.0 kB/s 2023-10-09T06:34:50.458 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:50.459 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2211 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:50.514 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:50.514 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:50.515 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0051e-05 s, 0.0 kB/s 2023-10-09T06:34:50.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:50.515 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2212 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:50.570 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:50.570 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:50.570 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0633e-05 s, 0.0 kB/s 2023-10-09T06:34:50.570 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:50.571 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2213 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:50.626 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:50.626 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:50.626 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.849e-05 s, 0.0 kB/s 2023-10-09T06:34:50.627 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:50.627 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2214 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:50.682 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:50.682 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:50.682 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7884e-05 s, 0.0 kB/s 2023-10-09T06:34:50.682 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:50.682 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2215 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:50.738 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:50.738 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:50.738 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8721e-05 s, 0.0 kB/s 2023-10-09T06:34:50.739 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:50.739 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2216 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:50.794 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:50.794 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:50.794 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0875e-05 s, 0.0 kB/s 2023-10-09T06:34:50.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:50.795 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2217 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:50.850 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:50.850 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:50.850 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0676e-05 s, 0.0 kB/s 2023-10-09T06:34:50.850 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:50.850 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2218 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:50.906 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:50.906 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:50.906 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0343e-05 s, 0.0 kB/s 2023-10-09T06:34:50.906 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:50.906 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2219 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:50.962 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:50.963 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:50.963 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8848e-05 s, 0.0 kB/s 2023-10-09T06:34:50.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:50.963 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2220 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:51.019 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:51.019 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:51.019 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7689e-05 s, 0.0 kB/s 2023-10-09T06:34:51.019 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:51.019 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2221 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:51.074 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:51.074 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:51.074 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0753e-05 s, 0.0 kB/s 2023-10-09T06:34:51.075 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:51.075 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2222 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:51.130 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:51.130 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:51.131 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2937e-05 s, 0.0 kB/s 2023-10-09T06:34:51.131 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:51.131 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2223 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:51.198 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:51.198 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:51.198 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1247e-05 s, 0.0 kB/s 2023-10-09T06:34:51.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:51.199 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2224 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:51.263 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:51.263 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:51.263 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.249e-05 s, 0.0 kB/s 2023-10-09T06:34:51.264 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:51.264 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2225 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:51.319 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:51.320 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:51.320 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8424e-05 s, 0.0 kB/s 2023-10-09T06:34:51.320 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:51.320 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2226 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:51.379 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:51.379 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:51.380 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0096e-05 s, 0.0 kB/s 2023-10-09T06:34:51.380 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:51.380 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2227 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:51.443 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:51.443 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:51.443 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.7082e-05 s, 0.0 kB/s 2023-10-09T06:34:51.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:51.444 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2228 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:51.505 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:51.505 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:51.506 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9562e-05 s, 0.0 kB/s 2023-10-09T06:34:51.506 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:51.506 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2229 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:51.564 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:51.564 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:51.565 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8069e-05 s, 0.0 kB/s 2023-10-09T06:34:51.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:51.565 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2230 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:51.623 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:51.623 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:51.623 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4273e-05 s, 0.0 kB/s 2023-10-09T06:34:51.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:51.624 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2231 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:51.680 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:51.680 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:51.680 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8749e-05 s, 0.0 kB/s 2023-10-09T06:34:51.681 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:51.681 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2232 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:51.737 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:51.737 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:51.737 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8553e-05 s, 0.0 kB/s 2023-10-09T06:34:51.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:51.738 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2233 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:51.793 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:51.794 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:51.794 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1134e-05 s, 0.0 kB/s 2023-10-09T06:34:51.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:51.795 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2234 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:51.854 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:51.854 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:51.854 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7691e-05 s, 0.0 kB/s 2023-10-09T06:34:51.854 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:51.854 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2235 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:51.912 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:51.912 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:51.913 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0862e-05 s, 0.0 kB/s 2023-10-09T06:34:51.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:51.913 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2236 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:51.968 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:51.968 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:51.968 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1239e-05 s, 0.0 kB/s 2023-10-09T06:34:51.969 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:51.969 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2237 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:52.023 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:52.024 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:52.024 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9714e-05 s, 0.0 kB/s 2023-10-09T06:34:52.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:52.025 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2238 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:52.080 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:52.080 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:52.080 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9554e-05 s, 0.0 kB/s 2023-10-09T06:34:52.081 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:52.081 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2239 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:52.136 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:52.136 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:52.136 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8339e-05 s, 0.0 kB/s 2023-10-09T06:34:52.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:52.136 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2240 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:52.191 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:52.192 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:52.192 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8489e-05 s, 0.0 kB/s 2023-10-09T06:34:52.192 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:52.192 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2241 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:52.247 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:52.247 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:52.247 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0279e-05 s, 0.0 kB/s 2023-10-09T06:34:52.248 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:52.248 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2242 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:52.303 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:52.303 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:52.304 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8214e-05 s, 0.0 kB/s 2023-10-09T06:34:52.304 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:52.304 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2243 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:52.359 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:52.359 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:52.359 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0203e-05 s, 0.0 kB/s 2023-10-09T06:34:52.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:52.360 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2244 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:52.415 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:52.415 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:52.415 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2626e-05 s, 0.0 kB/s 2023-10-09T06:34:52.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:52.415 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2245 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:52.471 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:52.471 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:52.471 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7837e-05 s, 0.0 kB/s 2023-10-09T06:34:52.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:52.471 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2246 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:52.527 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:52.527 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:52.527 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8781e-05 s, 0.0 kB/s 2023-10-09T06:34:52.527 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:52.527 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2247 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:52.582 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:52.582 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:52.582 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9089e-05 s, 0.0 kB/s 2023-10-09T06:34:52.582 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:52.582 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2248 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:52.637 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:52.637 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:52.637 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.736e-05 s, 0.0 kB/s 2023-10-09T06:34:52.637 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:52.638 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2249 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:52.693 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:52.693 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:52.693 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7421e-05 s, 0.0 kB/s 2023-10-09T06:34:52.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:52.693 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2250 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:52.748 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:52.749 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:52.749 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8585e-05 s, 0.0 kB/s 2023-10-09T06:34:52.749 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:52.750 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2251 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:52.805 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:52.805 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:52.805 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0179e-05 s, 0.0 kB/s 2023-10-09T06:34:52.806 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:52.806 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2252 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:52.861 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:52.861 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:52.861 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.996e-05 s, 0.0 kB/s 2023-10-09T06:34:52.861 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:52.862 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2253 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:52.917 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:52.917 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:52.917 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9618e-05 s, 0.0 kB/s 2023-10-09T06:34:52.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:52.918 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2254 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:52.973 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:52.973 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:52.973 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.901e-05 s, 0.0 kB/s 2023-10-09T06:34:52.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:52.973 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2255 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:53.028 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:53.028 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:53.029 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7741e-05 s, 0.0 kB/s 2023-10-09T06:34:53.029 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:53.029 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2256 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:53.084 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:53.084 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:53.084 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9053e-05 s, 0.0 kB/s 2023-10-09T06:34:53.084 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:53.085 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2257 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:53.139 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:53.139 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:53.139 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1563e-05 s, 0.0 kB/s 2023-10-09T06:34:53.140 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:53.140 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2258 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:53.196 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:53.196 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:53.196 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7899e-05 s, 0.0 kB/s 2023-10-09T06:34:53.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:53.196 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2259 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:53.251 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:53.251 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:53.251 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7419e-05 s, 0.0 kB/s 2023-10-09T06:34:53.251 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:53.251 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2260 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:53.306 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:53.306 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:53.306 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9422e-05 s, 0.0 kB/s 2023-10-09T06:34:53.306 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:53.307 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2261 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:53.362 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:53.362 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:53.362 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2484e-05 s, 0.0 kB/s 2023-10-09T06:34:53.362 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:53.362 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2262 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:53.417 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:53.417 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:53.417 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8578e-05 s, 0.0 kB/s 2023-10-09T06:34:53.417 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:53.418 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2263 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:53.473 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:53.473 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:53.473 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0707e-05 s, 0.0 kB/s 2023-10-09T06:34:53.474 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:53.474 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2264 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:53.529 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:53.529 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:53.529 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8626e-05 s, 0.0 kB/s 2023-10-09T06:34:53.530 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:53.530 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2265 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:53.585 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:53.585 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:53.585 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9624e-05 s, 0.0 kB/s 2023-10-09T06:34:53.585 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:53.585 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2266 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:53.640 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:53.641 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:53.641 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9679e-05 s, 0.0 kB/s 2023-10-09T06:34:53.641 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:53.641 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2267 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:53.696 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:53.696 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:53.696 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0771e-05 s, 0.0 kB/s 2023-10-09T06:34:53.697 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:53.697 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2268 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:53.752 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:53.752 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:53.752 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0926e-05 s, 0.0 kB/s 2023-10-09T06:34:53.753 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:53.753 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2269 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:53.808 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:53.809 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:53.809 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4137e-05 s, 0.0 kB/s 2023-10-09T06:34:53.809 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:53.809 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2270 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:53.864 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:53.864 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:53.864 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0065e-05 s, 0.0 kB/s 2023-10-09T06:34:53.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:53.865 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2271 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:53.920 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:53.920 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:53.920 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9229e-05 s, 0.0 kB/s 2023-10-09T06:34:53.921 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:53.921 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2272 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:53.976 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:53.976 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:53.976 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0731e-05 s, 0.0 kB/s 2023-10-09T06:34:53.977 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:53.977 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2273 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:54.032 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:54.032 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:54.032 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0065e-05 s, 0.0 kB/s 2023-10-09T06:34:54.033 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:54.033 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2274 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:54.088 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:54.089 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:54.089 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0609e-05 s, 0.0 kB/s 2023-10-09T06:34:54.089 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:54.089 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2275 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:54.144 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:54.144 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:54.144 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7281e-05 s, 0.0 kB/s 2023-10-09T06:34:54.144 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:54.144 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2276 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:54.198 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:54.198 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:54.199 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9785e-05 s, 0.0 kB/s 2023-10-09T06:34:54.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:54.200 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2277 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:54.255 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:54.255 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:54.255 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2231e-05 s, 0.0 kB/s 2023-10-09T06:34:54.255 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:54.256 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2278 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:54.311 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:54.311 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:54.311 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7706e-05 s, 0.0 kB/s 2023-10-09T06:34:54.311 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:54.311 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2279 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:54.366 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:54.366 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:54.366 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7749e-05 s, 0.0 kB/s 2023-10-09T06:34:54.366 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:54.366 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2280 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:54.420 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:54.420 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:54.421 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8027e-05 s, 0.0 kB/s 2023-10-09T06:34:54.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:54.422 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2281 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:54.477 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:54.477 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:54.477 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9913e-05 s, 0.0 kB/s 2023-10-09T06:34:54.477 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:54.477 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2282 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:54.532 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:54.532 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:54.532 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1251e-05 s, 0.0 kB/s 2023-10-09T06:34:54.533 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:54.533 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2283 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:54.588 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:54.589 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:54.589 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1822e-05 s, 0.0 kB/s 2023-10-09T06:34:54.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:54.589 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2284 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:54.644 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:54.644 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:54.644 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0658e-05 s, 0.0 kB/s 2023-10-09T06:34:54.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:54.645 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2285 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:54.700 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:54.700 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:54.700 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1025e-05 s, 0.0 kB/s 2023-10-09T06:34:54.700 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:54.701 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2286 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:54.756 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:54.756 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:54.756 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.092e-05 s, 0.0 kB/s 2023-10-09T06:34:54.756 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:54.757 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2287 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:54.812 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:54.812 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:54.812 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9951e-05 s, 0.0 kB/s 2023-10-09T06:34:54.813 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:54.813 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2288 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:54.868 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:54.868 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:54.868 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8419e-05 s, 0.0 kB/s 2023-10-09T06:34:54.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:54.868 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2289 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:54.923 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:54.923 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:54.923 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0552e-05 s, 0.0 kB/s 2023-10-09T06:34:54.923 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:54.923 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2290 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:54.991 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:54.992 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:54.992 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.5183e-05 s, 0.0 kB/s 2023-10-09T06:34:54.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:54.992 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2291 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:55.047 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:55.048 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:55.048 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.282e-05 s, 0.0 kB/s 2023-10-09T06:34:55.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:55.048 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2292 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:55.102 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:55.103 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:55.103 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6517e-05 s, 0.0 kB/s 2023-10-09T06:34:55.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:55.103 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2293 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:55.158 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:55.158 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:55.158 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.994e-05 s, 0.0 kB/s 2023-10-09T06:34:55.158 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:55.158 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2294 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:55.213 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:55.213 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:55.213 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0414e-05 s, 0.0 kB/s 2023-10-09T06:34:55.213 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:55.214 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2295 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:55.269 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:55.269 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:55.269 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8619e-05 s, 0.0 kB/s 2023-10-09T06:34:55.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:55.269 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2296 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:55.324 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:55.324 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:55.324 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0023e-05 s, 0.0 kB/s 2023-10-09T06:34:55.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:55.324 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2297 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:55.380 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:55.380 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:55.380 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1e-05 s, 0.0 kB/s 2023-10-09T06:34:55.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:55.381 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2298 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:55.436 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:55.436 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:55.436 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7699e-05 s, 0.0 kB/s 2023-10-09T06:34:55.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:55.437 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2299 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:55.492 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:55.492 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:55.492 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1279e-05 s, 0.0 kB/s 2023-10-09T06:34:55.493 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:55.493 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2300 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:55.548 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:55.548 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:55.549 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1573e-05 s, 0.0 kB/s 2023-10-09T06:34:55.549 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:55.549 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2301 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:55.604 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:55.605 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:55.605 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1508e-05 s, 0.0 kB/s 2023-10-09T06:34:55.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:55.605 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2302 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:55.660 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:55.660 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:55.660 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.991e-05 s, 0.0 kB/s 2023-10-09T06:34:55.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:55.660 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2303 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:55.716 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:55.716 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:55.716 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2471e-05 s, 0.0 kB/s 2023-10-09T06:34:55.716 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:55.716 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2304 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:55.772 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:55.772 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:55.772 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.184e-05 s, 0.0 kB/s 2023-10-09T06:34:55.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:55.772 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2305 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:55.828 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:55.828 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:55.828 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3037e-05 s, 0.0 kB/s 2023-10-09T06:34:55.829 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:55.829 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2306 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:55.884 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:55.884 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:55.885 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.212e-05 s, 0.0 kB/s 2023-10-09T06:34:55.885 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:55.885 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2307 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:55.940 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:55.940 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:55.941 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2257e-05 s, 0.0 kB/s 2023-10-09T06:34:55.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:55.941 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2308 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:55.996 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:55.996 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:55.996 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.95e-05 s, 0.0 kB/s 2023-10-09T06:34:55.996 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:55.997 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2309 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:56.053 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:56.053 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:56.053 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2475e-05 s, 0.0 kB/s 2023-10-09T06:34:56.054 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:56.054 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2310 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:56.111 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:56.112 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:56.112 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8991e-05 s, 0.0 kB/s 2023-10-09T06:34:56.112 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:56.112 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2311 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:56.167 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:56.168 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:56.168 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0632e-05 s, 0.0 kB/s 2023-10-09T06:34:56.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:56.168 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2312 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:56.223 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:56.223 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:56.223 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.799e-05 s, 0.0 kB/s 2023-10-09T06:34:56.223 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:56.223 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2313 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:56.279 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:56.279 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:56.279 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2353e-05 s, 0.0 kB/s 2023-10-09T06:34:56.279 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:56.279 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2314 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:56.335 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:56.335 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:56.335 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1985e-05 s, 0.0 kB/s 2023-10-09T06:34:56.335 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:56.336 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2315 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:56.391 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:56.391 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:56.391 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3299e-05 s, 0.0 kB/s 2023-10-09T06:34:56.391 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:56.391 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2316 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:56.447 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:56.447 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:56.447 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3823e-05 s, 0.0 kB/s 2023-10-09T06:34:56.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:56.448 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2317 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:56.503 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:56.503 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:56.503 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1804e-05 s, 0.0 kB/s 2023-10-09T06:34:56.504 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:56.504 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2318 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:56.559 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:56.559 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:56.559 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.768e-05 s, 0.0 kB/s 2023-10-09T06:34:56.560 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:56.560 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2319 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:56.615 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:56.615 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:56.615 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9237e-05 s, 0.0 kB/s 2023-10-09T06:34:56.616 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:56.616 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2320 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:56.671 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:56.671 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:56.671 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0979e-05 s, 0.0 kB/s 2023-10-09T06:34:56.671 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:56.672 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2321 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:56.727 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:56.727 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:56.727 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.981e-05 s, 0.0 kB/s 2023-10-09T06:34:56.728 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:56.728 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2322 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:56.784 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:56.785 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:56.785 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1636e-05 s, 0.0 kB/s 2023-10-09T06:34:56.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:56.785 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2323 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:56.840 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:56.840 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:56.840 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1948e-05 s, 0.0 kB/s 2023-10-09T06:34:56.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:56.841 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2324 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:56.896 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:56.896 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:56.896 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4242e-05 s, 0.0 kB/s 2023-10-09T06:34:56.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:56.897 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2325 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:56.952 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:56.952 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:56.953 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2702e-05 s, 0.0 kB/s 2023-10-09T06:34:56.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:56.953 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2326 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:57.008 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:57.009 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:57.009 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0291e-05 s, 0.0 kB/s 2023-10-09T06:34:57.009 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:57.009 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2327 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:57.064 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:57.065 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:57.065 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8007e-05 s, 0.0 kB/s 2023-10-09T06:34:57.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:57.065 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2328 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:57.121 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:57.121 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:57.121 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7831e-05 s, 0.0 kB/s 2023-10-09T06:34:57.121 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:57.122 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2329 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:57.177 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:57.177 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:57.177 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9003e-05 s, 0.0 kB/s 2023-10-09T06:34:57.177 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:57.177 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2330 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:57.233 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:57.233 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:57.233 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9862e-05 s, 0.0 kB/s 2023-10-09T06:34:57.233 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:57.234 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2331 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:57.289 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:57.289 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:57.289 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1408e-05 s, 0.0 kB/s 2023-10-09T06:34:57.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:57.290 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2332 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:57.345 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:57.345 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:57.345 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.893e-05 s, 0.0 kB/s 2023-10-09T06:34:57.346 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:57.346 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2333 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:57.401 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:57.401 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:57.401 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1628e-05 s, 0.0 kB/s 2023-10-09T06:34:57.401 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:57.402 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2334 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:57.457 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:57.458 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:57.458 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9936e-05 s, 0.0 kB/s 2023-10-09T06:34:57.458 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:57.458 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2335 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:57.513 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:57.513 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:57.513 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2892e-05 s, 0.0 kB/s 2023-10-09T06:34:57.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:57.514 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2336 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:57.569 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:57.569 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:57.569 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9241e-05 s, 0.0 kB/s 2023-10-09T06:34:57.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:57.569 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2337 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:57.625 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:57.625 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:57.625 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9926e-05 s, 0.0 kB/s 2023-10-09T06:34:57.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:57.626 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2338 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:57.681 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:57.681 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:57.682 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2875e-05 s, 0.0 kB/s 2023-10-09T06:34:57.682 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:57.682 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2339 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:57.737 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:57.737 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:57.737 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8424e-05 s, 0.0 kB/s 2023-10-09T06:34:57.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:57.738 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2340 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:57.795 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:57.795 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:57.795 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3739e-05 s, 0.0 kB/s 2023-10-09T06:34:57.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:57.795 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2341 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:57.850 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:57.850 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:57.850 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0115e-05 s, 0.0 kB/s 2023-10-09T06:34:57.850 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:57.851 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2342 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:57.906 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:57.907 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:57.907 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1237e-05 s, 0.0 kB/s 2023-10-09T06:34:57.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:57.907 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2343 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:57.962 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:57.963 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:57.963 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0165e-05 s, 0.0 kB/s 2023-10-09T06:34:57.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:57.963 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2344 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:58.018 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:58.018 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:58.018 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9761e-05 s, 0.0 kB/s 2023-10-09T06:34:58.019 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:58.019 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2345 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:58.075 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:58.075 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:58.075 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.6122e-05 s, 0.0 kB/s 2023-10-09T06:34:58.075 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:58.075 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2346 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:58.131 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:58.131 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:58.131 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0602e-05 s, 0.0 kB/s 2023-10-09T06:34:58.132 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:58.132 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2347 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:58.187 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:58.187 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:58.187 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9465e-05 s, 0.0 kB/s 2023-10-09T06:34:58.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:58.187 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2348 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:58.243 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:58.243 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:58.243 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2944e-05 s, 0.0 kB/s 2023-10-09T06:34:58.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:58.244 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2349 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:58.299 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:58.299 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:58.299 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8393e-05 s, 0.0 kB/s 2023-10-09T06:34:58.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:58.300 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2350 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:58.355 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:58.355 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:58.356 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0209e-05 s, 0.0 kB/s 2023-10-09T06:34:58.356 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:58.356 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2351 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:58.411 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:58.411 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:58.412 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9902e-05 s, 0.0 kB/s 2023-10-09T06:34:58.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:58.412 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2352 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:58.467 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:58.467 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:58.467 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2307e-05 s, 0.0 kB/s 2023-10-09T06:34:58.467 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:58.467 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2353 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:58.523 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:58.524 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:58.524 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8529e-05 s, 0.0 kB/s 2023-10-09T06:34:58.524 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:58.524 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2354 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:58.579 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:58.579 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:58.580 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0085e-05 s, 0.0 kB/s 2023-10-09T06:34:58.580 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:58.580 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2355 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:58.635 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:58.635 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:58.635 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0966e-05 s, 0.0 kB/s 2023-10-09T06:34:58.635 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:58.636 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2356 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:58.691 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:58.691 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:58.691 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0275e-05 s, 0.0 kB/s 2023-10-09T06:34:58.691 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:58.691 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2357 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:58.746 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:58.746 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:58.747 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9618e-05 s, 0.0 kB/s 2023-10-09T06:34:58.747 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:58.747 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2358 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:58.802 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:58.802 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:58.802 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.963e-05 s, 0.0 kB/s 2023-10-09T06:34:58.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:58.803 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2359 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:58.857 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:58.858 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:58.858 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1565e-05 s, 0.0 kB/s 2023-10-09T06:34:58.858 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:58.858 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2360 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:58.913 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:58.913 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:58.913 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8726e-05 s, 0.0 kB/s 2023-10-09T06:34:58.914 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:58.914 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2361 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:58.969 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:58.969 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:58.969 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9912e-05 s, 0.0 kB/s 2023-10-09T06:34:58.969 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:58.969 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2362 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:59.026 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:59.026 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:59.026 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8425e-05 s, 0.0 kB/s 2023-10-09T06:34:59.027 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:59.027 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2363 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:59.082 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:59.082 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:59.082 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9002e-05 s, 0.0 kB/s 2023-10-09T06:34:59.083 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:59.083 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2364 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:59.139 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:59.139 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:59.139 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8582e-05 s, 0.0 kB/s 2023-10-09T06:34:59.139 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:59.139 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2365 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:59.195 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:59.195 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:59.195 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.799e-05 s, 0.0 kB/s 2023-10-09T06:34:59.195 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:59.195 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2366 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:59.250 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:59.250 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:59.250 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9121e-05 s, 0.0 kB/s 2023-10-09T06:34:59.250 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:59.250 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2367 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:59.306 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:59.306 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:59.306 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2943e-05 s, 0.0 kB/s 2023-10-09T06:34:59.306 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:59.306 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2368 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:59.362 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:59.362 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:59.362 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7948e-05 s, 0.0 kB/s 2023-10-09T06:34:59.363 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:59.363 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2369 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:59.418 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:59.418 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:59.418 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9856e-05 s, 0.0 kB/s 2023-10-09T06:34:59.418 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:59.419 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2370 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:59.474 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:59.474 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:59.474 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2942e-05 s, 0.0 kB/s 2023-10-09T06:34:59.474 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:59.474 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2371 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:59.530 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:59.530 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:59.530 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9648e-05 s, 0.0 kB/s 2023-10-09T06:34:59.531 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:59.531 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2372 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:59.586 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:59.586 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:59.586 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1294e-05 s, 0.0 kB/s 2023-10-09T06:34:59.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:59.587 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2373 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:59.642 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:59.642 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:59.642 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2368e-05 s, 0.0 kB/s 2023-10-09T06:34:59.642 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:59.642 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2374 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:59.698 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:59.698 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:59.699 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7404e-05 s, 0.0 kB/s 2023-10-09T06:34:59.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:59.700 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2375 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:59.754 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:59.754 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:59.755 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1726e-05 s, 0.0 kB/s 2023-10-09T06:34:59.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:59.755 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2376 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:59.810 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:59.810 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:59.811 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2367e-05 s, 0.0 kB/s 2023-10-09T06:34:59.811 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:59.811 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2377 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:59.866 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:59.867 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:59.867 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.892e-05 s, 0.0 kB/s 2023-10-09T06:34:59.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:59.868 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2378 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:59.922 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:59.922 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:59.923 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2355e-05 s, 0.0 kB/s 2023-10-09T06:34:59.923 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:59.923 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2379 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:34:59.978 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:34:59.978 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:34:59.978 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0589e-05 s, 0.0 kB/s 2023-10-09T06:34:59.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:34:59.979 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2380 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:00.034 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:00.034 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:00.034 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2754e-05 s, 0.0 kB/s 2023-10-09T06:35:00.034 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:00.035 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2381 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:00.090 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:00.091 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:00.091 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0604e-05 s, 0.0 kB/s 2023-10-09T06:35:00.091 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:00.091 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2382 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:00.146 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:00.146 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:00.146 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9419e-05 s, 0.0 kB/s 2023-10-09T06:35:00.146 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:00.146 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2383 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:00.202 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:00.202 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:00.203 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0287e-05 s, 0.0 kB/s 2023-10-09T06:35:00.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:00.203 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2384 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:00.259 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:00.259 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:00.259 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.041e-05 s, 0.0 kB/s 2023-10-09T06:35:00.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:00.259 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2385 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:00.315 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:00.315 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:00.315 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9425e-05 s, 0.0 kB/s 2023-10-09T06:35:00.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:00.316 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2386 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:00.371 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:00.371 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:00.371 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.904e-05 s, 0.0 kB/s 2023-10-09T06:35:00.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:00.372 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2387 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:00.427 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:00.427 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:00.427 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9608e-05 s, 0.0 kB/s 2023-10-09T06:35:00.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:00.427 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2388 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:00.482 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:00.482 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:00.482 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.329e-05 s, 0.0 kB/s 2023-10-09T06:35:00.483 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:00.483 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2389 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:00.538 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:00.539 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:00.539 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0771e-05 s, 0.0 kB/s 2023-10-09T06:35:00.539 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:00.539 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2390 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:00.620 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:00.620 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:00.620 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.2964e-05 s, 0.0 kB/s 2023-10-09T06:35:00.621 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:00.621 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2391 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:00.676 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:00.676 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:00.676 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0111e-05 s, 0.0 kB/s 2023-10-09T06:35:00.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:00.677 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2392 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:00.732 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:00.732 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:00.732 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.145e-05 s, 0.0 kB/s 2023-10-09T06:35:00.733 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:00.733 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2393 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:00.788 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:00.789 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:00.789 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2445e-05 s, 0.0 kB/s 2023-10-09T06:35:00.789 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:00.789 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2394 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:00.844 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:00.844 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:00.844 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2223e-05 s, 0.0 kB/s 2023-10-09T06:35:00.844 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:00.844 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2395 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:00.899 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:00.900 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:00.900 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9621e-05 s, 0.0 kB/s 2023-10-09T06:35:00.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:00.901 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2396 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:00.956 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:00.956 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:00.957 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2439e-05 s, 0.0 kB/s 2023-10-09T06:35:00.957 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:00.957 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2397 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:01.012 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:01.012 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:01.012 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8474e-05 s, 0.0 kB/s 2023-10-09T06:35:01.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:01.013 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2398 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:01.068 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:01.069 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:01.069 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1753e-05 s, 0.0 kB/s 2023-10-09T06:35:01.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:01.070 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2399 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:01.126 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:01.126 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:01.126 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0658e-05 s, 0.0 kB/s 2023-10-09T06:35:01.126 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:01.126 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2400 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:01.181 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:01.181 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:01.181 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.9268e-05 s, 0.0 kB/s 2023-10-09T06:35:01.181 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:01.181 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2401 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:01.237 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:01.237 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:01.237 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1961e-05 s, 0.0 kB/s 2023-10-09T06:35:01.237 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:01.238 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2402 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:01.293 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:01.294 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:01.294 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3519e-05 s, 0.0 kB/s 2023-10-09T06:35:01.294 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:01.294 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2403 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:01.349 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:01.350 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:01.350 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0982e-05 s, 0.0 kB/s 2023-10-09T06:35:01.350 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:01.350 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2404 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:01.406 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:01.406 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:01.406 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1482e-05 s, 0.0 kB/s 2023-10-09T06:35:01.407 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:01.407 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2405 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:01.461 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:01.462 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:01.462 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0289e-05 s, 0.0 kB/s 2023-10-09T06:35:01.462 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:01.462 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2406 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:01.517 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:01.517 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:01.517 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2501e-05 s, 0.0 kB/s 2023-10-09T06:35:01.518 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:01.518 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2407 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:01.573 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:01.574 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:01.574 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0823e-05 s, 0.0 kB/s 2023-10-09T06:35:01.574 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:01.574 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2408 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:01.629 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:01.629 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:01.629 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3053e-05 s, 0.0 kB/s 2023-10-09T06:35:01.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:01.629 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2409 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:01.684 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:01.684 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:01.685 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0953e-05 s, 0.0 kB/s 2023-10-09T06:35:01.685 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:01.685 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2410 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:01.740 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:01.740 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:01.741 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2443e-05 s, 0.0 kB/s 2023-10-09T06:35:01.741 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:01.741 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2411 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:01.796 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:01.796 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:01.797 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0569e-05 s, 0.0 kB/s 2023-10-09T06:35:01.797 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:01.797 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2412 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:01.852 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:01.852 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:01.852 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9899e-05 s, 0.0 kB/s 2023-10-09T06:35:01.853 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:01.853 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2413 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:01.908 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:01.908 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:01.909 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9029e-05 s, 0.0 kB/s 2023-10-09T06:35:01.909 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:01.909 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2414 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:01.965 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:01.965 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:01.965 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8802e-05 s, 0.0 kB/s 2023-10-09T06:35:01.965 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:01.965 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2415 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:02.021 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:02.021 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:02.021 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8771e-05 s, 0.0 kB/s 2023-10-09T06:35:02.021 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:02.021 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2416 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:02.076 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:02.077 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:02.077 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3605e-05 s, 0.0 kB/s 2023-10-09T06:35:02.077 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:02.077 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2417 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:02.133 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:02.133 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:02.133 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0979e-05 s, 0.0 kB/s 2023-10-09T06:35:02.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:02.133 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2418 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:02.188 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:02.188 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:02.189 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0461e-05 s, 0.0 kB/s 2023-10-09T06:35:02.189 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:02.189 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2419 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:02.244 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:02.244 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:02.245 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1312e-05 s, 0.0 kB/s 2023-10-09T06:35:02.245 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:02.245 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2420 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:02.300 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:02.300 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:02.301 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.016e-05 s, 0.0 kB/s 2023-10-09T06:35:02.301 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:02.301 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2421 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:02.356 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:02.356 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:02.356 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1282e-05 s, 0.0 kB/s 2023-10-09T06:35:02.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:02.357 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2422 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:02.412 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:02.412 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:02.412 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0029e-05 s, 0.0 kB/s 2023-10-09T06:35:02.413 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:02.413 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2423 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:02.468 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:02.468 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:02.468 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8806e-05 s, 0.0 kB/s 2023-10-09T06:35:02.469 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:02.469 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2424 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:02.524 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:02.524 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:02.524 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2423e-05 s, 0.0 kB/s 2023-10-09T06:35:02.525 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:02.525 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2425 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:02.580 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:02.580 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:02.580 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2296e-05 s, 0.0 kB/s 2023-10-09T06:35:02.581 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:02.581 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2426 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:02.636 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:02.636 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:02.636 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9744e-05 s, 0.0 kB/s 2023-10-09T06:35:02.636 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:02.637 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2427 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:02.692 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:02.692 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:02.692 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8449e-05 s, 0.0 kB/s 2023-10-09T06:35:02.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:02.693 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2428 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:02.748 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:02.748 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:02.748 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0448e-05 s, 0.0 kB/s 2023-10-09T06:35:02.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:02.749 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2429 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:02.804 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:02.805 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:02.805 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1942e-05 s, 0.0 kB/s 2023-10-09T06:35:02.805 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:02.805 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2430 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:02.860 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:02.860 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:02.860 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1275e-05 s, 0.0 kB/s 2023-10-09T06:35:02.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:02.860 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2431 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:02.915 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:02.915 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:02.916 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0457e-05 s, 0.0 kB/s 2023-10-09T06:35:02.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:02.916 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2432 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:02.971 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:02.971 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:02.971 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1563e-05 s, 0.0 kB/s 2023-10-09T06:35:02.972 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:02.972 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2433 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:03.027 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:03.027 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:03.027 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8888e-05 s, 0.0 kB/s 2023-10-09T06:35:03.027 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:03.027 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2434 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:03.083 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:03.083 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:03.083 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9669e-05 s, 0.0 kB/s 2023-10-09T06:35:03.084 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:03.084 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2435 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:03.140 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:03.140 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:03.140 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9922e-05 s, 0.0 kB/s 2023-10-09T06:35:03.140 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:03.140 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2436 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:03.195 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:03.196 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:03.196 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8335e-05 s, 0.0 kB/s 2023-10-09T06:35:03.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:03.196 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2437 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:03.250 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:03.250 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:03.251 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8148e-05 s, 0.0 kB/s 2023-10-09T06:35:03.251 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:03.252 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2438 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:03.306 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:03.307 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:03.307 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9224e-05 s, 0.0 kB/s 2023-10-09T06:35:03.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:03.308 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2439 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:03.362 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:03.362 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:03.362 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1081e-05 s, 0.0 kB/s 2023-10-09T06:35:03.363 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:03.363 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2440 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:03.419 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:03.419 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:03.419 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8651e-05 s, 0.0 kB/s 2023-10-09T06:35:03.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:03.420 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2441 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:03.475 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:03.475 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:03.475 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1237e-05 s, 0.0 kB/s 2023-10-09T06:35:03.475 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:03.476 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2442 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:03.531 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:03.532 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:03.532 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9449e-05 s, 0.0 kB/s 2023-10-09T06:35:03.533 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:03.533 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2443 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:03.589 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:03.589 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:03.589 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3675e-05 s, 0.0 kB/s 2023-10-09T06:35:03.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:03.589 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2444 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:03.644 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:03.644 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:03.644 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0985e-05 s, 0.0 kB/s 2023-10-09T06:35:03.644 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:03.644 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2445 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:03.700 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:03.700 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:03.700 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1978e-05 s, 0.0 kB/s 2023-10-09T06:35:03.700 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:03.701 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2446 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:03.756 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:03.756 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:03.756 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0443e-05 s, 0.0 kB/s 2023-10-09T06:35:03.756 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:03.756 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2447 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:03.811 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:03.811 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:03.812 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4059e-05 s, 0.0 kB/s 2023-10-09T06:35:03.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:03.812 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2448 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:03.867 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:03.867 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:03.867 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9167e-05 s, 0.0 kB/s 2023-10-09T06:35:03.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:03.868 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2449 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:03.923 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:03.923 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:03.924 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0003e-05 s, 0.0 kB/s 2023-10-09T06:35:03.924 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:03.924 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2450 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:03.979 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:03.979 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:03.979 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9613e-05 s, 0.0 kB/s 2023-10-09T06:35:03.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:03.980 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2451 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:04.034 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:04.034 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:04.035 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9835e-05 s, 0.0 kB/s 2023-10-09T06:35:04.035 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:04.036 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2452 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:04.090 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:04.090 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:04.091 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.092e-05 s, 0.0 kB/s 2023-10-09T06:35:04.092 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:04.092 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2453 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:04.148 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:04.148 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:04.148 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9478e-05 s, 0.0 kB/s 2023-10-09T06:35:04.148 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:04.149 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2454 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:04.204 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:04.204 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:04.204 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1819e-05 s, 0.0 kB/s 2023-10-09T06:35:04.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:04.205 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2455 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:04.260 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:04.260 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:04.260 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9037e-05 s, 0.0 kB/s 2023-10-09T06:35:04.261 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:04.261 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2456 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:04.316 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:04.316 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:04.316 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1394e-05 s, 0.0 kB/s 2023-10-09T06:35:04.317 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:04.317 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2457 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:04.372 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:04.372 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:04.372 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.277e-05 s, 0.0 kB/s 2023-10-09T06:35:04.373 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:04.373 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2458 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:04.428 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:04.428 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:04.428 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9411e-05 s, 0.0 kB/s 2023-10-09T06:35:04.428 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:04.429 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2459 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:04.484 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:04.484 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:04.484 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.989e-05 s, 0.0 kB/s 2023-10-09T06:35:04.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:04.485 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2460 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:04.541 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:04.541 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:04.542 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2961e-05 s, 0.0 kB/s 2023-10-09T06:35:04.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:04.542 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2461 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:04.597 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:04.597 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:04.597 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1045e-05 s, 0.0 kB/s 2023-10-09T06:35:04.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:04.598 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2462 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:04.653 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:04.653 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:04.653 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.907e-05 s, 0.0 kB/s 2023-10-09T06:35:04.653 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:04.653 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2463 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:04.709 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:04.709 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:04.709 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9427e-05 s, 0.0 kB/s 2023-10-09T06:35:04.709 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:04.709 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2464 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:04.765 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:04.765 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:04.765 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1757e-05 s, 0.0 kB/s 2023-10-09T06:35:04.766 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:04.766 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2465 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:04.821 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:04.822 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:04.822 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2243e-05 s, 0.0 kB/s 2023-10-09T06:35:04.822 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:04.822 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2466 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:04.877 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:04.877 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:04.877 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.043e-05 s, 0.0 kB/s 2023-10-09T06:35:04.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:04.878 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2467 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:04.933 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:04.933 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:04.933 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7689e-05 s, 0.0 kB/s 2023-10-09T06:35:04.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:04.934 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2468 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:04.989 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:04.989 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:04.989 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0329e-05 s, 0.0 kB/s 2023-10-09T06:35:04.990 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:04.990 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2469 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:05.045 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:05.045 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:05.045 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2893e-05 s, 0.0 kB/s 2023-10-09T06:35:05.046 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:05.046 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2470 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:05.101 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:05.101 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:05.101 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0753e-05 s, 0.0 kB/s 2023-10-09T06:35:05.102 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:05.102 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2471 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:05.158 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:05.159 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:05.159 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0599e-05 s, 0.0 kB/s 2023-10-09T06:35:05.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:05.159 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2472 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:05.214 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:05.214 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:05.214 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.05e-05 s, 0.0 kB/s 2023-10-09T06:35:05.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:05.215 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2473 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:05.269 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:05.269 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:05.269 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.0321e-05 s, 0.0 kB/s 2023-10-09T06:35:05.270 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:05.270 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2474 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:05.326 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:05.326 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:05.326 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8543e-05 s, 0.0 kB/s 2023-10-09T06:35:05.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:05.327 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2475 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:05.382 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:05.382 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:05.383 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0308e-05 s, 0.0 kB/s 2023-10-09T06:35:05.383 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:05.383 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2476 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:05.439 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:05.440 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:05.440 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7283e-05 s, 0.0 kB/s 2023-10-09T06:35:05.440 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:05.441 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2477 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:05.494 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:05.494 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:05.494 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0411e-05 s, 0.0 kB/s 2023-10-09T06:35:05.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:05.495 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2478 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:05.550 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:05.551 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:05.551 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1381e-05 s, 0.0 kB/s 2023-10-09T06:35:05.551 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:05.551 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2479 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:05.606 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:05.606 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:05.606 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4661e-05 s, 0.0 kB/s 2023-10-09T06:35:05.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:05.607 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2480 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:05.662 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:05.662 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:05.662 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7663e-05 s, 0.0 kB/s 2023-10-09T06:35:05.662 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:05.663 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2481 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:05.747 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:05.748 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:05.748 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 5.5646e-05 s, 0.0 kB/s 2023-10-09T06:35:05.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:05.748 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2482 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:05.763 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:05.763 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:05.763 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 5.4211e-05 s, 0.0 kB/s 2023-10-09T06:35:05.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:05.764 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2483 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:05.820 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:05.820 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:05.820 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9001e-05 s, 0.0 kB/s 2023-10-09T06:35:05.820 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:05.820 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2484 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:05.876 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:05.876 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:05.877 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0018e-05 s, 0.0 kB/s 2023-10-09T06:35:05.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:05.878 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2485 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:05.932 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:05.932 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:05.932 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.426e-05 s, 0.0 kB/s 2023-10-09T06:35:05.933 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:05.933 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2486 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:05.989 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:05.989 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:05.989 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3651e-05 s, 0.0 kB/s 2023-10-09T06:35:05.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:05.989 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2487 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:06.046 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:06.046 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:06.046 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0178e-05 s, 0.0 kB/s 2023-10-09T06:35:06.046 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:06.046 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2488 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:06.101 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:06.101 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:06.101 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1453e-05 s, 0.0 kB/s 2023-10-09T06:35:06.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:06.101 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2489 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:06.157 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:06.158 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:06.158 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.074e-05 s, 0.0 kB/s 2023-10-09T06:35:06.158 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:06.158 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2490 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:06.213 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:06.213 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:06.213 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9869e-05 s, 0.0 kB/s 2023-10-09T06:35:06.214 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:06.214 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2491 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:06.269 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:06.269 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:06.269 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.02e-05 s, 0.0 kB/s 2023-10-09T06:35:06.270 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:06.270 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2492 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:06.325 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:06.325 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:06.326 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9048e-05 s, 0.0 kB/s 2023-10-09T06:35:06.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:06.326 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2493 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:06.381 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:06.381 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:06.381 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0362e-05 s, 0.0 kB/s 2023-10-09T06:35:06.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:06.382 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2494 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:06.436 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:06.437 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:06.437 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.847e-05 s, 0.0 kB/s 2023-10-09T06:35:06.438 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:06.438 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2495 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:06.493 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:06.493 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:06.493 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0818e-05 s, 0.0 kB/s 2023-10-09T06:35:06.493 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:06.494 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2496 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:06.549 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:06.549 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:06.549 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3533e-05 s, 0.0 kB/s 2023-10-09T06:35:06.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:06.550 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2497 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:06.605 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:06.605 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:06.605 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2221e-05 s, 0.0 kB/s 2023-10-09T06:35:06.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:06.606 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2498 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:06.661 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:06.661 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:06.661 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8689e-05 s, 0.0 kB/s 2023-10-09T06:35:06.662 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:06.662 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2499 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:06.717 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:06.717 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:06.717 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1493e-05 s, 0.0 kB/s 2023-10-09T06:35:06.718 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:06.718 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2500 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:06.773 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:06.773 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:06.773 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1226e-05 s, 0.0 kB/s 2023-10-09T06:35:06.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:06.773 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2501 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:06.829 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:06.829 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:06.830 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7887e-05 s, 0.0 kB/s 2023-10-09T06:35:06.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:06.830 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2502 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:06.885 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:06.886 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:06.886 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2161e-05 s, 0.0 kB/s 2023-10-09T06:35:06.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:06.886 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2503 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:06.941 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:06.941 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:06.941 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7862e-05 s, 0.0 kB/s 2023-10-09T06:35:06.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:06.942 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2504 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:06.997 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:06.998 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:06.998 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0033e-05 s, 0.0 kB/s 2023-10-09T06:35:06.998 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:06.998 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2505 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:07.053 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:07.054 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:07.054 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.245e-05 s, 0.0 kB/s 2023-10-09T06:35:07.054 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:07.054 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2506 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:07.109 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:07.109 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:07.109 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8316e-05 s, 0.0 kB/s 2023-10-09T06:35:07.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:07.110 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2507 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:07.165 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:07.166 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:07.166 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0543e-05 s, 0.0 kB/s 2023-10-09T06:35:07.166 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:07.166 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2508 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:07.221 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:07.221 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:07.221 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.716e-05 s, 0.0 kB/s 2023-10-09T06:35:07.222 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:07.222 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2509 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:07.277 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:07.277 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:07.277 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0753e-05 s, 0.0 kB/s 2023-10-09T06:35:07.277 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:07.277 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2510 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:07.332 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:07.332 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:07.332 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1143e-05 s, 0.0 kB/s 2023-10-09T06:35:07.332 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:07.332 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2511 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:07.388 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:07.388 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:07.388 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1835e-05 s, 0.0 kB/s 2023-10-09T06:35:07.388 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:07.388 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2512 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:07.443 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:07.443 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:07.444 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9207e-05 s, 0.0 kB/s 2023-10-09T06:35:07.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:07.444 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2513 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:07.499 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:07.499 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:07.500 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9685e-05 s, 0.0 kB/s 2023-10-09T06:35:07.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:07.500 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2514 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:07.555 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:07.555 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:07.555 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1081e-05 s, 0.0 kB/s 2023-10-09T06:35:07.556 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:07.556 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2515 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:07.611 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:07.611 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:07.611 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.104e-05 s, 0.0 kB/s 2023-10-09T06:35:07.612 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:07.612 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2516 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:07.667 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:07.667 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:07.667 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1128e-05 s, 0.0 kB/s 2023-10-09T06:35:07.668 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:07.668 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2517 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:07.723 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:07.723 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:07.723 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.791e-05 s, 0.0 kB/s 2023-10-09T06:35:07.724 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:07.724 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2518 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:07.778 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:07.779 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:07.779 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8833e-05 s, 0.0 kB/s 2023-10-09T06:35:07.779 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:07.780 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2519 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:07.835 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:07.835 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:07.835 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8732e-05 s, 0.0 kB/s 2023-10-09T06:35:07.836 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:07.836 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2520 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:07.892 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:07.892 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:07.892 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0561e-05 s, 0.0 kB/s 2023-10-09T06:35:07.892 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:07.892 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2521 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:07.948 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:07.948 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:07.948 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.06e-05 s, 0.0 kB/s 2023-10-09T06:35:07.948 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:07.949 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2522 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:08.004 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:08.005 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:08.005 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.208e-05 s, 0.0 kB/s 2023-10-09T06:35:08.005 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:08.005 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2523 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:08.060 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:08.060 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:08.060 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3418e-05 s, 0.0 kB/s 2023-10-09T06:35:08.061 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:08.061 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2524 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:08.116 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:08.116 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:08.116 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9379e-05 s, 0.0 kB/s 2023-10-09T06:35:08.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:08.117 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2525 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:08.172 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:08.172 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:08.172 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8113e-05 s, 0.0 kB/s 2023-10-09T06:35:08.173 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:08.173 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2526 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:08.228 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:08.228 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:08.228 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1518e-05 s, 0.0 kB/s 2023-10-09T06:35:08.229 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:08.229 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2527 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:08.283 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:08.283 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:08.284 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9834e-05 s, 0.0 kB/s 2023-10-09T06:35:08.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:08.285 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2528 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:08.340 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:08.340 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:08.340 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8548e-05 s, 0.0 kB/s 2023-10-09T06:35:08.340 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:08.340 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2529 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:08.396 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:08.396 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:08.396 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9732e-05 s, 0.0 kB/s 2023-10-09T06:35:08.396 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:08.397 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2530 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:08.452 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:08.452 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:08.452 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.6681e-05 s, 0.0 kB/s 2023-10-09T06:35:08.453 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:08.453 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2531 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:08.508 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:08.508 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:08.508 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8998e-05 s, 0.0 kB/s 2023-10-09T06:35:08.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:08.508 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2532 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:08.563 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:08.563 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:08.564 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0804e-05 s, 0.0 kB/s 2023-10-09T06:35:08.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:08.564 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2533 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:08.619 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:08.619 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:08.619 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8349e-05 s, 0.0 kB/s 2023-10-09T06:35:08.619 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:08.620 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2534 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:08.675 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:08.675 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:08.675 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2451e-05 s, 0.0 kB/s 2023-10-09T06:35:08.675 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:08.675 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2535 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:08.730 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:08.730 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:08.730 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9205e-05 s, 0.0 kB/s 2023-10-09T06:35:08.730 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:08.730 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2536 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:08.786 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:08.786 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:08.786 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0398e-05 s, 0.0 kB/s 2023-10-09T06:35:08.786 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:08.786 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2537 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:08.841 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:08.841 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:08.842 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1823e-05 s, 0.0 kB/s 2023-10-09T06:35:08.842 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:08.842 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2538 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:08.897 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:08.897 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:08.897 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1145e-05 s, 0.0 kB/s 2023-10-09T06:35:08.898 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:08.898 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2539 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:08.953 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:08.953 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:08.953 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.261e-05 s, 0.0 kB/s 2023-10-09T06:35:08.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:08.953 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2540 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:09.009 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:09.009 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:09.009 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8557e-05 s, 0.0 kB/s 2023-10-09T06:35:09.009 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:09.010 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2541 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:09.065 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:09.065 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:09.066 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9035e-05 s, 0.0 kB/s 2023-10-09T06:35:09.066 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:09.066 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2542 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:09.121 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:09.121 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:09.121 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.831e-05 s, 0.0 kB/s 2023-10-09T06:35:09.122 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:09.122 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2543 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:09.177 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:09.177 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:09.177 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0552e-05 s, 0.0 kB/s 2023-10-09T06:35:09.178 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:09.178 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2544 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:09.233 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:09.233 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:09.233 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.213e-05 s, 0.0 kB/s 2023-10-09T06:35:09.233 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:09.233 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2545 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:09.289 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:09.289 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:09.289 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8501e-05 s, 0.0 kB/s 2023-10-09T06:35:09.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:09.290 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2546 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:09.345 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:09.345 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:09.346 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0669e-05 s, 0.0 kB/s 2023-10-09T06:35:09.346 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:09.346 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2547 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:09.401 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:09.401 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:09.401 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9369e-05 s, 0.0 kB/s 2023-10-09T06:35:09.401 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:09.402 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2548 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:09.457 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:09.457 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:09.457 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1524e-05 s, 0.0 kB/s 2023-10-09T06:35:09.457 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:09.458 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2549 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:09.513 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:09.513 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:09.513 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3049e-05 s, 0.0 kB/s 2023-10-09T06:35:09.514 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:09.514 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2550 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:09.569 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:09.570 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:09.570 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0551e-05 s, 0.0 kB/s 2023-10-09T06:35:09.570 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:09.570 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2551 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:09.625 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:09.626 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:09.626 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9156e-05 s, 0.0 kB/s 2023-10-09T06:35:09.626 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:09.626 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2552 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:09.681 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:09.681 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:09.681 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.749e-05 s, 0.0 kB/s 2023-10-09T06:35:09.681 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:09.681 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2553 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:09.737 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:09.737 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:09.737 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1151e-05 s, 0.0 kB/s 2023-10-09T06:35:09.737 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:09.737 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2554 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:09.793 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:09.793 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:09.793 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9836e-05 s, 0.0 kB/s 2023-10-09T06:35:09.794 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:09.794 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2555 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:09.849 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:09.849 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:09.849 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.889e-05 s, 0.0 kB/s 2023-10-09T06:35:09.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:09.849 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2556 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:09.905 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:09.905 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:09.905 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1281e-05 s, 0.0 kB/s 2023-10-09T06:35:09.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:09.906 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2557 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:09.961 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:09.961 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:09.961 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9862e-05 s, 0.0 kB/s 2023-10-09T06:35:09.961 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:09.961 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2558 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:10.017 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:10.018 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:10.018 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1676e-05 s, 0.0 kB/s 2023-10-09T06:35:10.018 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:10.018 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2559 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:10.073 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:10.074 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:10.074 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1659e-05 s, 0.0 kB/s 2023-10-09T06:35:10.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:10.074 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2560 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:10.129 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:10.129 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:10.129 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0716e-05 s, 0.0 kB/s 2023-10-09T06:35:10.130 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:10.130 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2561 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:10.185 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:10.186 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:10.186 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8884e-05 s, 0.0 kB/s 2023-10-09T06:35:10.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:10.186 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2562 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:10.241 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:10.241 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:10.241 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9777e-05 s, 0.0 kB/s 2023-10-09T06:35:10.242 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:10.242 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2563 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:10.297 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:10.297 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:10.297 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8963e-05 s, 0.0 kB/s 2023-10-09T06:35:10.297 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:10.297 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2564 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:10.353 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:10.353 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:10.353 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8912e-05 s, 0.0 kB/s 2023-10-09T06:35:10.353 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:10.354 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2565 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:10.409 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:10.409 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:10.409 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0039e-05 s, 0.0 kB/s 2023-10-09T06:35:10.410 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:10.410 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2566 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:10.465 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:10.465 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:10.465 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9417e-05 s, 0.0 kB/s 2023-10-09T06:35:10.465 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:10.466 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2567 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:10.522 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:10.522 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:10.522 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8451e-05 s, 0.0 kB/s 2023-10-09T06:35:10.522 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:10.523 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2568 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:10.578 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:10.578 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:10.578 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2503e-05 s, 0.0 kB/s 2023-10-09T06:35:10.578 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:10.578 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2569 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:10.633 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:10.633 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:10.633 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2076e-05 s, 0.0 kB/s 2023-10-09T06:35:10.634 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:10.634 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2570 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:10.690 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:10.690 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:10.690 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8003e-05 s, 0.0 kB/s 2023-10-09T06:35:10.690 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:10.690 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2571 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:10.746 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:10.746 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:10.746 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9426e-05 s, 0.0 kB/s 2023-10-09T06:35:10.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:10.747 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2572 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:10.802 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:10.802 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:10.802 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1366e-05 s, 0.0 kB/s 2023-10-09T06:35:10.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:10.803 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2573 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:10.859 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:10.859 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:10.859 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.844e-05 s, 0.0 kB/s 2023-10-09T06:35:10.859 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:10.859 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2574 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:10.916 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:10.917 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:10.917 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1919e-05 s, 0.0 kB/s 2023-10-09T06:35:10.917 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:10.917 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2575 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:10.972 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:10.973 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:10.973 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1066e-05 s, 0.0 kB/s 2023-10-09T06:35:10.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:10.973 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2576 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:11.029 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:11.029 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:11.029 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2161e-05 s, 0.0 kB/s 2023-10-09T06:35:11.029 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:11.029 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2577 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:11.086 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:11.086 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:11.086 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.103e-05 s, 0.0 kB/s 2023-10-09T06:35:11.087 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:11.087 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2578 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:11.143 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:11.143 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:11.143 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9649e-05 s, 0.0 kB/s 2023-10-09T06:35:11.144 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:11.144 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2579 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:11.200 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:11.200 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:11.200 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1323e-05 s, 0.0 kB/s 2023-10-09T06:35:11.200 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:11.200 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2580 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:11.256 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:11.256 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:11.256 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8944e-05 s, 0.0 kB/s 2023-10-09T06:35:11.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:11.256 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2581 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:11.314 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:11.314 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:11.314 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.5085e-05 s, 0.0 kB/s 2023-10-09T06:35:11.314 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:11.315 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2582 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:11.370 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:11.371 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:11.371 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8065e-05 s, 0.0 kB/s 2023-10-09T06:35:11.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:11.371 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2583 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:11.426 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:11.426 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:11.427 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1454e-05 s, 0.0 kB/s 2023-10-09T06:35:11.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:11.427 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2584 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:11.482 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:11.482 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:11.482 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3206e-05 s, 0.0 kB/s 2023-10-09T06:35:11.483 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:11.485 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2585 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:11.539 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:11.540 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:11.540 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2331e-05 s, 0.0 kB/s 2023-10-09T06:35:11.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:11.540 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2586 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:11.595 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:11.596 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:11.596 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8882e-05 s, 0.0 kB/s 2023-10-09T06:35:11.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:11.596 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2587 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:11.651 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:11.651 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:11.652 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.216e-05 s, 0.0 kB/s 2023-10-09T06:35:11.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:11.652 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2588 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:11.707 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:11.707 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:11.707 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3283e-05 s, 0.0 kB/s 2023-10-09T06:35:11.707 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:11.708 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2589 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:11.764 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:11.764 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:11.764 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1216e-05 s, 0.0 kB/s 2023-10-09T06:35:11.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:11.764 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2590 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:11.820 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:11.820 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:11.821 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4545e-05 s, 0.0 kB/s 2023-10-09T06:35:11.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:11.821 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2591 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:11.876 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:11.876 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:11.876 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0371e-05 s, 0.0 kB/s 2023-10-09T06:35:11.876 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:11.876 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2592 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:11.934 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:11.934 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:11.934 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.116e-05 s, 0.0 kB/s 2023-10-09T06:35:11.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:11.934 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2593 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:11.992 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:11.992 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:11.992 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2345e-05 s, 0.0 kB/s 2023-10-09T06:35:11.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:11.992 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2594 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:12.053 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:12.053 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:12.054 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1623e-05 s, 0.0 kB/s 2023-10-09T06:35:12.054 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:12.054 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2595 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:12.115 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:12.115 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:12.115 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0074e-05 s, 0.0 kB/s 2023-10-09T06:35:12.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:12.116 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2596 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:12.178 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:12.178 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:12.178 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8957e-05 s, 0.0 kB/s 2023-10-09T06:35:12.179 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:12.179 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2597 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:12.248 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:12.248 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:12.249 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.202e-05 s, 0.0 kB/s 2023-10-09T06:35:12.249 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:12.249 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2598 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:12.314 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:12.314 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:12.314 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0334e-05 s, 0.0 kB/s 2023-10-09T06:35:12.314 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:12.314 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2599 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:12.379 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:12.379 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:12.379 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.005e-05 s, 0.0 kB/s 2023-10-09T06:35:12.380 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:12.380 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2600 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:12.436 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:12.436 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:12.437 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4012e-05 s, 0.0 kB/s 2023-10-09T06:35:12.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:12.437 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2601 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:12.495 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:12.495 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:12.495 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3629e-05 s, 0.0 kB/s 2023-10-09T06:35:12.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:12.496 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2602 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:12.552 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:12.552 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:12.553 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0748e-05 s, 0.0 kB/s 2023-10-09T06:35:12.553 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:12.553 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2603 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:12.609 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:12.610 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:12.610 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1038e-05 s, 0.0 kB/s 2023-10-09T06:35:12.610 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:12.611 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2604 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:12.667 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:12.667 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:12.667 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8715e-05 s, 0.0 kB/s 2023-10-09T06:35:12.668 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:12.668 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2605 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:12.724 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:12.724 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:12.724 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0366e-05 s, 0.0 kB/s 2023-10-09T06:35:12.724 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:12.724 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2606 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:12.780 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:12.780 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:12.780 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.882e-05 s, 0.0 kB/s 2023-10-09T06:35:12.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:12.781 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2607 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:12.836 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:12.836 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:12.836 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1768e-05 s, 0.0 kB/s 2023-10-09T06:35:12.837 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:12.837 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2608 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:12.893 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:12.893 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:12.894 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3021e-05 s, 0.0 kB/s 2023-10-09T06:35:12.894 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:12.894 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2609 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:12.953 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:12.953 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:12.953 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1785e-05 s, 0.0 kB/s 2023-10-09T06:35:12.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:12.953 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2610 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:13.010 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:13.010 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:13.010 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0066e-05 s, 0.0 kB/s 2023-10-09T06:35:13.011 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:13.011 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2611 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:13.066 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:13.067 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:13.067 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2177e-05 s, 0.0 kB/s 2023-10-09T06:35:13.067 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:13.067 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2612 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:13.125 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:13.125 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:13.125 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.3729e-05 s, 0.0 kB/s 2023-10-09T06:35:13.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:13.125 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2613 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:13.181 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:13.181 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:13.181 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1018e-05 s, 0.0 kB/s 2023-10-09T06:35:13.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:13.182 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2614 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:13.239 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:13.239 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:13.239 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2361e-05 s, 0.0 kB/s 2023-10-09T06:35:13.239 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:13.239 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2615 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:13.295 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:13.296 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:13.296 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2455e-05 s, 0.0 kB/s 2023-10-09T06:35:13.296 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:13.296 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2616 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:13.351 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:13.351 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:13.351 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8558e-05 s, 0.0 kB/s 2023-10-09T06:35:13.352 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:13.352 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2617 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:13.408 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:13.408 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:13.408 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8346e-05 s, 0.0 kB/s 2023-10-09T06:35:13.409 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:13.409 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2618 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:13.464 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:13.464 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:13.464 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1601e-05 s, 0.0 kB/s 2023-10-09T06:35:13.464 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:13.464 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2619 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:13.520 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:13.520 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:13.520 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0089e-05 s, 0.0 kB/s 2023-10-09T06:35:13.520 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:13.520 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2620 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:13.575 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:13.576 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:13.576 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1221e-05 s, 0.0 kB/s 2023-10-09T06:35:13.576 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:13.576 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2621 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:13.585 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:35:13.618 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:35:13.631 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:13.631 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:13.631 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0963e-05 s, 0.0 kB/s 2023-10-09T06:35:13.631 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:13.632 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2622 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:13.728 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:13.728 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:13.728 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1651e-05 s, 0.0 kB/s 2023-10-09T06:35:13.728 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:13.728 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2623 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:13.784 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:13.784 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:13.784 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2139e-05 s, 0.0 kB/s 2023-10-09T06:35:13.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:13.785 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2624 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:13.840 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:13.840 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:13.840 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2007e-05 s, 0.0 kB/s 2023-10-09T06:35:13.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:13.841 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2625 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:13.896 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:13.896 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:13.896 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2069e-05 s, 0.0 kB/s 2023-10-09T06:35:13.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:13.897 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2626 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:13.952 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:13.952 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:13.952 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2539e-05 s, 0.0 kB/s 2023-10-09T06:35:13.952 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:13.953 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2627 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:14.008 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:14.008 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:14.009 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.219e-05 s, 0.0 kB/s 2023-10-09T06:35:14.009 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:14.009 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2628 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:14.064 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:14.064 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:14.064 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8262e-05 s, 0.0 kB/s 2023-10-09T06:35:14.064 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:14.065 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2629 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:14.120 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:14.120 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:14.120 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9741e-05 s, 0.0 kB/s 2023-10-09T06:35:14.120 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:14.120 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2630 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:14.176 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:14.176 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:14.176 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0919e-05 s, 0.0 kB/s 2023-10-09T06:35:14.176 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:14.177 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2631 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:14.232 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:14.232 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:14.232 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8677e-05 s, 0.0 kB/s 2023-10-09T06:35:14.233 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:14.233 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2632 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:14.288 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:14.288 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:14.288 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.912e-05 s, 0.0 kB/s 2023-10-09T06:35:14.288 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:14.288 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2633 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:14.343 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:14.344 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:14.344 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0274e-05 s, 0.0 kB/s 2023-10-09T06:35:14.344 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:14.344 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2634 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:14.399 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:14.399 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:14.399 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0739e-05 s, 0.0 kB/s 2023-10-09T06:35:14.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:14.400 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2635 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:14.455 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:14.455 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:14.455 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0614e-05 s, 0.0 kB/s 2023-10-09T06:35:14.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:14.456 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2636 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:14.511 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:14.511 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:14.511 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1107e-05 s, 0.0 kB/s 2023-10-09T06:35:14.511 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:14.512 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2637 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:14.568 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:14.568 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:14.568 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.34e-05 s, 0.0 kB/s 2023-10-09T06:35:14.568 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:14.569 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2638 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:14.624 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:14.625 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:14.625 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.001e-05 s, 0.0 kB/s 2023-10-09T06:35:14.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:14.625 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2639 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:14.680 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:14.680 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:14.680 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2152e-05 s, 0.0 kB/s 2023-10-09T06:35:14.681 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:14.681 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2640 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:14.737 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:14.737 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:14.737 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3089e-05 s, 0.0 kB/s 2023-10-09T06:35:14.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:14.738 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2641 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:14.794 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:14.794 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:14.794 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2277e-05 s, 0.0 kB/s 2023-10-09T06:35:14.794 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:14.794 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2642 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:14.849 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:14.850 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:14.850 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0242e-05 s, 0.0 kB/s 2023-10-09T06:35:14.850 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:14.850 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2643 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:14.905 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:14.905 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:14.905 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9147e-05 s, 0.0 kB/s 2023-10-09T06:35:14.906 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:14.906 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2644 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:14.961 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:14.961 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:14.961 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9763e-05 s, 0.0 kB/s 2023-10-09T06:35:14.962 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:14.962 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2645 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:15.018 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:15.018 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:15.018 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2089e-05 s, 0.0 kB/s 2023-10-09T06:35:15.018 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:15.018 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2646 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:15.074 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:15.074 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:15.074 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1159e-05 s, 0.0 kB/s 2023-10-09T06:35:15.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:15.074 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2647 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:15.129 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:15.130 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:15.130 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2058e-05 s, 0.0 kB/s 2023-10-09T06:35:15.130 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:15.130 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2648 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:15.185 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:15.185 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:15.185 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9562e-05 s, 0.0 kB/s 2023-10-09T06:35:15.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:15.186 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2649 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:15.242 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:15.242 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:15.242 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2e-05 s, 0 B/s 2023-10-09T06:35:15.242 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:15.242 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2650 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:15.297 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:15.297 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:15.298 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4405e-05 s, 0.0 kB/s 2023-10-09T06:35:15.298 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:15.298 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2651 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:15.354 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:15.354 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:15.354 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3295e-05 s, 0.0 kB/s 2023-10-09T06:35:15.355 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:15.355 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2652 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:15.411 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:15.411 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:15.411 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3925e-05 s, 0.0 kB/s 2023-10-09T06:35:15.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:15.412 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2653 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:15.467 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:15.467 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:15.467 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9172e-05 s, 0.0 kB/s 2023-10-09T06:35:15.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:15.468 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2654 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:15.523 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:15.523 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:15.523 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.6349e-05 s, 0.0 kB/s 2023-10-09T06:35:15.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:15.524 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2655 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:15.579 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:15.579 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:15.579 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1216e-05 s, 0.0 kB/s 2023-10-09T06:35:15.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:15.580 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2656 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:15.635 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:15.635 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:15.635 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.723e-05 s, 0.0 kB/s 2023-10-09T06:35:15.635 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:15.635 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2657 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:15.691 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:15.691 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:15.692 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1869e-05 s, 0.0 kB/s 2023-10-09T06:35:15.692 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:15.692 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2658 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:15.748 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:15.748 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:15.748 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3581e-05 s, 0.0 kB/s 2023-10-09T06:35:15.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:15.749 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2659 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:15.803 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:15.804 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:15.804 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2716e-05 s, 0.0 kB/s 2023-10-09T06:35:15.804 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:15.804 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2660 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:15.859 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:15.859 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:15.859 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3962e-05 s, 0.0 kB/s 2023-10-09T06:35:15.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:15.860 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2661 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:15.915 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:15.915 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:15.916 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0749e-05 s, 0.0 kB/s 2023-10-09T06:35:15.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:15.916 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2662 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:15.971 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:15.973 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:15.973 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0655e-05 s, 0.0 kB/s 2023-10-09T06:35:15.974 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:15.974 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2663 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:16.027 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:16.028 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:16.028 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4445e-05 s, 0.0 kB/s 2023-10-09T06:35:16.028 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:16.028 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2664 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:16.083 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:16.084 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:16.084 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2618e-05 s, 0.0 kB/s 2023-10-09T06:35:16.084 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:16.084 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2665 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:16.139 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:16.139 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:16.140 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2728e-05 s, 0.0 kB/s 2023-10-09T06:35:16.140 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:16.140 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2666 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:16.195 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:16.195 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:16.195 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7633e-05 s, 0.0 kB/s 2023-10-09T06:35:16.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:16.196 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2667 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:16.253 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:16.253 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:16.253 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2712e-05 s, 0.0 kB/s 2023-10-09T06:35:16.253 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:16.253 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2668 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:16.308 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:16.309 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:16.309 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3482e-05 s, 0.0 kB/s 2023-10-09T06:35:16.309 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:16.309 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2669 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:16.365 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:16.365 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:16.365 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0406e-05 s, 0.0 kB/s 2023-10-09T06:35:16.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:16.365 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2670 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:16.423 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:16.423 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:16.423 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0638e-05 s, 0.0 kB/s 2023-10-09T06:35:16.423 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:16.423 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2671 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:16.479 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:16.479 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:16.479 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2167e-05 s, 0.0 kB/s 2023-10-09T06:35:16.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:16.480 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2672 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:16.536 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:16.536 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:16.536 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.6357e-05 s, 0.0 kB/s 2023-10-09T06:35:16.536 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:16.536 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2673 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:16.592 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:16.593 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:16.593 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2242e-05 s, 0.0 kB/s 2023-10-09T06:35:16.593 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:16.593 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2674 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:16.648 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:16.649 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:16.649 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1355e-05 s, 0.0 kB/s 2023-10-09T06:35:16.649 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:16.649 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2675 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:16.704 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:16.705 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:16.705 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.5945e-05 s, 0.0 kB/s 2023-10-09T06:35:16.705 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:16.705 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2676 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:16.760 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:16.760 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:16.760 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0819e-05 s, 0.0 kB/s 2023-10-09T06:35:16.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:16.761 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2677 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:16.816 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:16.816 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:16.816 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1725e-05 s, 0.0 kB/s 2023-10-09T06:35:16.817 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:16.817 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2678 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:16.872 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:16.873 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:16.873 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.252e-05 s, 0.0 kB/s 2023-10-09T06:35:16.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:16.873 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2679 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:16.928 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:16.928 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:16.928 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2383e-05 s, 0.0 kB/s 2023-10-09T06:35:16.929 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:16.929 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2680 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:17.030 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:17.030 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:17.030 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.39e-05 s, 0.0 kB/s 2023-10-09T06:35:17.031 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:17.031 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2681 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:17.047 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:17.047 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:17.047 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9173e-05 s, 0.0 kB/s 2023-10-09T06:35:17.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:17.048 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2682 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:17.103 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:17.103 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:17.103 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.866e-05 s, 0.0 kB/s 2023-10-09T06:35:17.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:17.104 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2683 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:17.159 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:17.159 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:17.159 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0419e-05 s, 0.0 kB/s 2023-10-09T06:35:17.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:17.160 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2684 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:17.215 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:17.216 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:17.216 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1822e-05 s, 0.0 kB/s 2023-10-09T06:35:17.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:17.216 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2685 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:17.271 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:17.272 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:17.272 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1814e-05 s, 0.0 kB/s 2023-10-09T06:35:17.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:17.272 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2686 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:17.327 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:17.327 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:17.327 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.246e-05 s, 0.0 kB/s 2023-10-09T06:35:17.328 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:17.328 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2687 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:17.384 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:17.384 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:17.384 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1822e-05 s, 0.0 kB/s 2023-10-09T06:35:17.385 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:17.385 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2688 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:17.440 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:17.441 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:17.441 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0514e-05 s, 0.0 kB/s 2023-10-09T06:35:17.441 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:17.441 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2689 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:17.496 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:17.496 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:17.496 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1166e-05 s, 0.0 kB/s 2023-10-09T06:35:17.496 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:17.497 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2690 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:17.552 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:17.553 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:17.553 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9917e-05 s, 0.0 kB/s 2023-10-09T06:35:17.553 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:17.553 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2691 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:17.608 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:17.608 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:17.608 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2486e-05 s, 0.0 kB/s 2023-10-09T06:35:17.608 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:17.608 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2692 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:17.664 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:17.664 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:17.664 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0164e-05 s, 0.0 kB/s 2023-10-09T06:35:17.664 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:17.664 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2693 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:17.719 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:17.719 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:17.719 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2763e-05 s, 0.0 kB/s 2023-10-09T06:35:17.719 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:17.719 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2694 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:17.775 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:17.775 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:17.776 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.846e-05 s, 0.0 kB/s 2023-10-09T06:35:17.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:17.776 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2695 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:17.831 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:17.831 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:17.831 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1184e-05 s, 0.0 kB/s 2023-10-09T06:35:17.831 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:17.831 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2696 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:17.887 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:17.887 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:17.887 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0491e-05 s, 0.0 kB/s 2023-10-09T06:35:17.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:17.888 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2697 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:17.943 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:17.944 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:17.944 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0481e-05 s, 0.0 kB/s 2023-10-09T06:35:17.944 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:17.944 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2698 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:17.999 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:17.999 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:17.999 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1171e-05 s, 0.0 kB/s 2023-10-09T06:35:18.000 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:18.000 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2699 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:18.056 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:18.056 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:18.056 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1487e-05 s, 0.0 kB/s 2023-10-09T06:35:18.056 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:18.056 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2700 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:18.111 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:18.112 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:18.112 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1922e-05 s, 0.0 kB/s 2023-10-09T06:35:18.112 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:18.112 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2701 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:18.167 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:18.168 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:18.168 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9762e-05 s, 0.0 kB/s 2023-10-09T06:35:18.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:18.168 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2702 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:18.223 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:18.223 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:18.223 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8992e-05 s, 0.0 kB/s 2023-10-09T06:35:18.224 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:18.224 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2703 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:18.279 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:18.279 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:18.279 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9322e-05 s, 0.0 kB/s 2023-10-09T06:35:18.280 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:18.280 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2704 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:18.335 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:18.335 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:18.335 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8376e-05 s, 0.0 kB/s 2023-10-09T06:35:18.335 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:18.335 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2705 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:18.391 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:18.391 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:18.391 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2324e-05 s, 0.0 kB/s 2023-10-09T06:35:18.392 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:18.392 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2706 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:18.447 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:18.447 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:18.447 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2773e-05 s, 0.0 kB/s 2023-10-09T06:35:18.447 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:18.448 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2707 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:18.503 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:18.503 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:18.504 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0961e-05 s, 0.0 kB/s 2023-10-09T06:35:18.504 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:18.504 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2708 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:18.561 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:18.561 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:18.561 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0442e-05 s, 0.0 kB/s 2023-10-09T06:35:18.561 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:18.561 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2709 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:18.617 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:18.617 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:18.617 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2048e-05 s, 0.0 kB/s 2023-10-09T06:35:18.618 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:18.618 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2710 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:18.673 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:18.673 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:18.674 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.181e-05 s, 0.0 kB/s 2023-10-09T06:35:18.674 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:18.674 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2711 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:18.729 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:18.729 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:18.729 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0184e-05 s, 0.0 kB/s 2023-10-09T06:35:18.730 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:18.730 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2712 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:18.785 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:18.785 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:18.785 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.055e-05 s, 0.0 kB/s 2023-10-09T06:35:18.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:18.786 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2713 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:18.841 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:18.841 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:18.842 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2264e-05 s, 0.0 kB/s 2023-10-09T06:35:18.842 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:18.842 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2714 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:18.897 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:18.897 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:18.897 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1338e-05 s, 0.0 kB/s 2023-10-09T06:35:18.898 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:18.898 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2715 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:18.953 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:18.953 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:18.953 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9538e-05 s, 0.0 kB/s 2023-10-09T06:35:18.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:18.953 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2716 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:19.009 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:19.009 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:19.009 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.259e-05 s, 0.0 kB/s 2023-10-09T06:35:19.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:19.010 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2717 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:19.065 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:19.065 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:19.065 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9604e-05 s, 0.0 kB/s 2023-10-09T06:35:19.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:19.065 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2718 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:19.121 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:19.121 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:19.121 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9046e-05 s, 0.0 kB/s 2023-10-09T06:35:19.121 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:19.121 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2719 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:19.177 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:19.177 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:19.177 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.123e-05 s, 0.0 kB/s 2023-10-09T06:35:19.178 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:19.178 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2720 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:19.233 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:19.233 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:19.234 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.91e-05 s, 0.0 kB/s 2023-10-09T06:35:19.234 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:19.234 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2721 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:19.289 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:19.289 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:19.289 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8268e-05 s, 0.0 kB/s 2023-10-09T06:35:19.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:19.289 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2722 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:19.345 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:19.345 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:19.345 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0365e-05 s, 0.0 kB/s 2023-10-09T06:35:19.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:19.346 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2723 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:19.401 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:19.402 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:19.402 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8204e-05 s, 0.0 kB/s 2023-10-09T06:35:19.402 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:19.402 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2724 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:19.457 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:19.457 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:19.457 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9937e-05 s, 0.0 kB/s 2023-10-09T06:35:19.457 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:19.457 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2725 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:19.511 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:19.512 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:19.512 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0648e-05 s, 0.0 kB/s 2023-10-09T06:35:19.512 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:19.513 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2726 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:19.568 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:19.568 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:19.568 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1043e-05 s, 0.0 kB/s 2023-10-09T06:35:19.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:19.569 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2727 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:19.624 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:19.624 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:19.624 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1043e-05 s, 0.0 kB/s 2023-10-09T06:35:19.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:19.624 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2728 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:19.679 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:19.679 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:19.679 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7905e-05 s, 0.0 kB/s 2023-10-09T06:35:19.679 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:19.680 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2729 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:19.735 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:19.735 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:19.735 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0179e-05 s, 0.0 kB/s 2023-10-09T06:35:19.735 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:19.735 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2730 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:19.791 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:19.791 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:19.791 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2269e-05 s, 0.0 kB/s 2023-10-09T06:35:19.792 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:19.792 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2731 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:19.847 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:19.847 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:19.847 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8389e-05 s, 0.0 kB/s 2023-10-09T06:35:19.848 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:19.848 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2732 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:19.903 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:19.903 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:19.903 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9036e-05 s, 0.0 kB/s 2023-10-09T06:35:19.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:19.904 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2733 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:19.959 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:19.959 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:19.959 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2367e-05 s, 0.0 kB/s 2023-10-09T06:35:19.960 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:19.960 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2734 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:20.015 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:20.016 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:20.016 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9958e-05 s, 0.0 kB/s 2023-10-09T06:35:20.016 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:20.016 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2735 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:20.071 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:20.072 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:20.072 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8827e-05 s, 0.0 kB/s 2023-10-09T06:35:20.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:20.072 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2736 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:20.127 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:20.128 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:20.128 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2173e-05 s, 0.0 kB/s 2023-10-09T06:35:20.128 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:20.128 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2737 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:20.183 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:20.183 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:20.183 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0538e-05 s, 0.0 kB/s 2023-10-09T06:35:20.183 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:20.183 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2738 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:20.240 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:20.240 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:20.240 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1107e-05 s, 0.0 kB/s 2023-10-09T06:35:20.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:20.241 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2739 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:20.295 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:20.295 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:20.295 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9938e-05 s, 0.0 kB/s 2023-10-09T06:35:20.295 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:20.296 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2740 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:20.351 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:20.351 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:20.351 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1198e-05 s, 0.0 kB/s 2023-10-09T06:35:20.351 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:20.351 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2741 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:20.408 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:20.408 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:20.408 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.172e-05 s, 0.0 kB/s 2023-10-09T06:35:20.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:20.409 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2742 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:20.464 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:20.464 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:20.465 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9404e-05 s, 0.0 kB/s 2023-10-09T06:35:20.465 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:20.465 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2743 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:20.520 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:20.520 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:20.520 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7381e-05 s, 0.0 kB/s 2023-10-09T06:35:20.521 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:20.521 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2744 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:20.576 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:20.577 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:20.577 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0754e-05 s, 0.0 kB/s 2023-10-09T06:35:20.577 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:20.577 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2745 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:20.632 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:20.633 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:20.633 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1356e-05 s, 0.0 kB/s 2023-10-09T06:35:20.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:20.633 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2746 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:20.688 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:20.688 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:20.689 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1143e-05 s, 0.0 kB/s 2023-10-09T06:35:20.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:20.689 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2747 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:20.745 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:20.745 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:20.745 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.904e-05 s, 0.0 kB/s 2023-10-09T06:35:20.745 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:20.745 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2748 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:20.800 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:20.800 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:20.800 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1354e-05 s, 0.0 kB/s 2023-10-09T06:35:20.801 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:20.801 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2749 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:20.856 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:20.856 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:20.856 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1051e-05 s, 0.0 kB/s 2023-10-09T06:35:20.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:20.857 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2750 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:20.912 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:20.912 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:20.912 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0389e-05 s, 0.0 kB/s 2023-10-09T06:35:20.912 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:20.912 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2751 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:20.968 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:20.969 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:20.969 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9931e-05 s, 0.0 kB/s 2023-10-09T06:35:20.969 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:20.969 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2752 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:21.024 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:21.024 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:21.025 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0602e-05 s, 0.0 kB/s 2023-10-09T06:35:21.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:21.025 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2753 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:21.081 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:21.081 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:21.081 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.9583e-05 s, 0.0 kB/s 2023-10-09T06:35:21.081 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:21.081 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2754 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:21.137 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:21.137 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:21.137 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.7764e-05 s, 0.0 kB/s 2023-10-09T06:35:21.138 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:21.138 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2755 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:21.193 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:21.194 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:21.194 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.187e-05 s, 0.0 kB/s 2023-10-09T06:35:21.194 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:21.195 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2756 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:21.249 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:21.250 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:21.250 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0088e-05 s, 0.0 kB/s 2023-10-09T06:35:21.250 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:21.250 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2757 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:21.305 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:21.305 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:21.305 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4133e-05 s, 0.0 kB/s 2023-10-09T06:35:21.306 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:21.306 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2758 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:21.361 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:21.361 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:21.362 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.837e-05 s, 0.0 kB/s 2023-10-09T06:35:21.362 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:21.362 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2759 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:21.417 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:21.417 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:21.417 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9101e-05 s, 0.0 kB/s 2023-10-09T06:35:21.418 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:21.418 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2760 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:21.474 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:21.474 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:21.474 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2652e-05 s, 0.0 kB/s 2023-10-09T06:35:21.475 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:21.475 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2761 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:21.530 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:21.530 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:21.530 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3757e-05 s, 0.0 kB/s 2023-10-09T06:35:21.531 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:21.531 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2762 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:21.586 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:21.587 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:21.587 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3882e-05 s, 0.0 kB/s 2023-10-09T06:35:21.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:21.587 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2763 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:21.642 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:21.642 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:21.642 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0146e-05 s, 0.0 kB/s 2023-10-09T06:35:21.642 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:21.643 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2764 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:21.698 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:21.698 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:21.698 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1099e-05 s, 0.0 kB/s 2023-10-09T06:35:21.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:21.699 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2765 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:21.754 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:21.755 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:21.755 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0398e-05 s, 0.0 kB/s 2023-10-09T06:35:21.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:21.755 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2766 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:21.810 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:21.810 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:21.811 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1633e-05 s, 0.0 kB/s 2023-10-09T06:35:21.811 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:21.811 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2767 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:21.866 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:21.866 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:21.866 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1781e-05 s, 0.0 kB/s 2023-10-09T06:35:21.867 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:21.867 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2768 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:21.922 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:21.922 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:21.922 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8175e-05 s, 0.0 kB/s 2023-10-09T06:35:21.922 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:21.922 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2769 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:21.977 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:21.977 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:21.978 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4831e-05 s, 0.0 kB/s 2023-10-09T06:35:21.978 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:21.978 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2770 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:22.033 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:22.033 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:22.033 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1422e-05 s, 0.0 kB/s 2023-10-09T06:35:22.034 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:22.034 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2771 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:22.089 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:22.090 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:22.090 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3927e-05 s, 0.0 kB/s 2023-10-09T06:35:22.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:22.090 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2772 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:22.145 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:22.145 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:22.145 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1508e-05 s, 0.0 kB/s 2023-10-09T06:35:22.145 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:22.146 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2773 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:22.201 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:22.201 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:22.201 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0935e-05 s, 0.0 kB/s 2023-10-09T06:35:22.202 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:22.202 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2774 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:22.257 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:22.257 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:22.257 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0868e-05 s, 0.0 kB/s 2023-10-09T06:35:22.257 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:22.257 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2775 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:22.312 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:22.312 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:22.312 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9507e-05 s, 0.0 kB/s 2023-10-09T06:35:22.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:22.312 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2776 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:22.368 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:22.368 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:22.368 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8009e-05 s, 0.0 kB/s 2023-10-09T06:35:22.368 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:22.369 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2777 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:22.425 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:22.425 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:22.425 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.992e-05 s, 0.0 kB/s 2023-10-09T06:35:22.425 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:22.426 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2778 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:22.481 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:22.481 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:22.481 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0812e-05 s, 0.0 kB/s 2023-10-09T06:35:22.481 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:22.482 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2779 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:22.537 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:22.538 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:22.538 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2039e-05 s, 0.0 kB/s 2023-10-09T06:35:22.538 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:22.538 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2780 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:22.653 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:22.653 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:22.653 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.7841e-05 s, 0.0 kB/s 2023-10-09T06:35:22.653 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:22.654 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2781 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:22.670 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:22.670 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:22.670 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2003e-05 s, 0.0 kB/s 2023-10-09T06:35:22.670 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:22.670 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2782 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:22.726 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:22.727 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:22.727 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1842e-05 s, 0.0 kB/s 2023-10-09T06:35:22.727 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:22.727 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2783 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:22.782 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:22.782 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:22.782 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.189e-05 s, 0.0 kB/s 2023-10-09T06:35:22.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:22.786 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2784 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:22.838 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:22.839 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:22.839 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0413e-05 s, 0.0 kB/s 2023-10-09T06:35:22.839 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:22.839 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2785 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:22.894 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:22.894 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:22.894 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8318e-05 s, 0.0 kB/s 2023-10-09T06:35:22.895 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:22.895 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2786 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:22.950 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:22.950 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:22.950 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0644e-05 s, 0.0 kB/s 2023-10-09T06:35:22.950 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:22.951 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2787 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:23.006 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:23.006 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:23.007 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8762e-05 s, 0.0 kB/s 2023-10-09T06:35:23.007 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:23.007 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2788 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:23.062 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:23.062 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:23.062 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1414e-05 s, 0.0 kB/s 2023-10-09T06:35:23.062 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:23.062 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2789 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:23.118 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:23.118 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:23.118 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9666e-05 s, 0.0 kB/s 2023-10-09T06:35:23.118 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:23.118 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2790 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:23.174 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:23.174 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:23.174 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.266e-05 s, 0.0 kB/s 2023-10-09T06:35:23.174 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:23.174 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2791 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:23.230 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:23.230 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:23.230 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3344e-05 s, 0.0 kB/s 2023-10-09T06:35:23.231 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:23.231 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2792 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:23.286 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:23.286 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:23.286 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1088e-05 s, 0.0 kB/s 2023-10-09T06:35:23.287 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:23.287 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2793 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:23.342 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:23.342 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:23.342 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1843e-05 s, 0.0 kB/s 2023-10-09T06:35:23.342 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:23.343 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2794 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:23.398 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:23.398 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:23.399 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0136e-05 s, 0.0 kB/s 2023-10-09T06:35:23.399 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:23.399 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2795 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:23.454 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:23.455 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:23.455 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0054e-05 s, 0.0 kB/s 2023-10-09T06:35:23.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:23.455 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2796 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:23.510 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:23.510 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:23.510 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2679e-05 s, 0.0 kB/s 2023-10-09T06:35:23.511 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:23.511 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2797 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:23.566 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:23.567 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:23.567 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0262e-05 s, 0.0 kB/s 2023-10-09T06:35:23.567 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:23.567 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2798 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:23.622 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:23.622 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:23.622 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2614e-05 s, 0.0 kB/s 2023-10-09T06:35:23.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:23.623 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2799 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:23.678 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:23.678 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:23.678 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1305e-05 s, 0.0 kB/s 2023-10-09T06:35:23.679 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:23.679 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2800 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:23.734 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:23.734 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:23.734 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9053e-05 s, 0.0 kB/s 2023-10-09T06:35:23.734 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:23.734 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2801 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:23.790 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:23.790 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:23.791 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0549e-05 s, 0.0 kB/s 2023-10-09T06:35:23.791 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:23.791 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2802 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:23.846 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:23.846 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:23.846 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2099e-05 s, 0.0 kB/s 2023-10-09T06:35:23.847 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:23.847 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2803 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:23.903 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:23.903 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:23.903 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1212e-05 s, 0.0 kB/s 2023-10-09T06:35:23.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:23.904 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2804 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:23.959 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:23.959 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:23.959 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9813e-05 s, 0.0 kB/s 2023-10-09T06:35:23.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:23.959 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2805 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:24.014 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:24.014 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:24.014 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0676e-05 s, 0.0 kB/s 2023-10-09T06:35:24.015 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:24.015 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2806 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:24.070 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:24.070 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:24.071 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.977e-05 s, 0.0 kB/s 2023-10-09T06:35:24.071 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:24.071 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2807 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:24.126 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:24.126 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:24.126 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9967e-05 s, 0.0 kB/s 2023-10-09T06:35:24.126 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:24.127 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2808 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:24.182 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:24.182 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:24.183 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.907e-05 s, 0.0 kB/s 2023-10-09T06:35:24.183 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:24.183 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2809 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:24.238 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:24.238 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:24.238 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.967e-05 s, 0.0 kB/s 2023-10-09T06:35:24.239 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:24.239 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2810 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:24.294 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:24.294 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:24.295 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0669e-05 s, 0.0 kB/s 2023-10-09T06:35:24.295 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:24.295 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2811 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:24.350 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:24.350 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:24.350 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.13e-05 s, 0.0 kB/s 2023-10-09T06:35:24.350 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:24.351 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2812 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:24.407 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:24.407 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:24.407 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.7778e-05 s, 0.0 kB/s 2023-10-09T06:35:24.407 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:24.407 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2813 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:24.462 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:24.463 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:24.463 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1834e-05 s, 0.0 kB/s 2023-10-09T06:35:24.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:24.463 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2814 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:24.518 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:24.518 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:24.518 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2081e-05 s, 0.0 kB/s 2023-10-09T06:35:24.518 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:24.518 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2815 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:24.574 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:24.575 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:24.575 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4206e-05 s, 0.0 kB/s 2023-10-09T06:35:24.576 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:24.576 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2816 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:24.632 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:24.632 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:24.632 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3207e-05 s, 0.0 kB/s 2023-10-09T06:35:24.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:24.633 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2817 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:24.688 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:24.688 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:24.689 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1632e-05 s, 0.0 kB/s 2023-10-09T06:35:24.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:24.689 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2818 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:24.744 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:24.745 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:24.745 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2571e-05 s, 0.0 kB/s 2023-10-09T06:35:24.745 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:24.745 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2819 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:24.800 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:24.801 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:24.801 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.359e-05 s, 0.0 kB/s 2023-10-09T06:35:24.801 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:24.801 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2820 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:24.856 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:24.856 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:24.856 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0181e-05 s, 0.0 kB/s 2023-10-09T06:35:24.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:24.857 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2821 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:24.913 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:24.913 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:24.913 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2481e-05 s, 0.0 kB/s 2023-10-09T06:35:24.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:24.913 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2822 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:24.968 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:24.968 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:24.968 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1712e-05 s, 0.0 kB/s 2023-10-09T06:35:24.969 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:24.969 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2823 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:25.024 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:25.024 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:25.025 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0752e-05 s, 0.0 kB/s 2023-10-09T06:35:25.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:25.025 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2824 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:25.079 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:25.079 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:25.080 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1262e-05 s, 0.0 kB/s 2023-10-09T06:35:25.081 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:25.081 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2825 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:25.136 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:25.136 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:25.136 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2637e-05 s, 0.0 kB/s 2023-10-09T06:35:25.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:25.136 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2826 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:25.192 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:25.192 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:25.192 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1365e-05 s, 0.0 kB/s 2023-10-09T06:35:25.193 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:25.193 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2827 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:25.248 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:25.248 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:25.248 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.067e-05 s, 0.0 kB/s 2023-10-09T06:35:25.248 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:25.248 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2828 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:25.303 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:25.303 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:25.303 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.235e-05 s, 0.0 kB/s 2023-10-09T06:35:25.304 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:25.304 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2829 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:25.359 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:25.359 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:25.359 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8071e-05 s, 0.0 kB/s 2023-10-09T06:35:25.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:25.359 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2830 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:25.415 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:25.415 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:25.415 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7454e-05 s, 0.0 kB/s 2023-10-09T06:35:25.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:25.415 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2831 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:25.472 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:25.472 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:25.472 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1248e-05 s, 0.0 kB/s 2023-10-09T06:35:25.472 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:25.472 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2832 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:25.527 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:25.527 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:25.527 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1619e-05 s, 0.0 kB/s 2023-10-09T06:35:25.528 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:25.528 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2833 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:25.583 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:25.583 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:25.583 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2426e-05 s, 0.0 kB/s 2023-10-09T06:35:25.584 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:25.584 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2834 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:25.639 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:25.640 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:25.640 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1364e-05 s, 0.0 kB/s 2023-10-09T06:35:25.640 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:25.640 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2835 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:25.695 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:25.695 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:25.695 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0127e-05 s, 0.0 kB/s 2023-10-09T06:35:25.696 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:25.696 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2836 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:25.751 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:25.751 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:25.751 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7516e-05 s, 0.0 kB/s 2023-10-09T06:35:25.751 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:25.751 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2837 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:25.805 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:25.805 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:25.805 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.6334e-05 s, 0.0 kB/s 2023-10-09T06:35:25.806 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:25.806 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2838 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:25.862 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:25.862 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:25.862 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7488e-05 s, 0.0 kB/s 2023-10-09T06:35:25.862 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:25.862 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2839 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:25.918 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:25.918 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:25.918 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1558e-05 s, 0.0 kB/s 2023-10-09T06:35:25.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:25.918 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2840 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:25.974 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:25.975 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:25.975 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.348e-05 s, 0.0 kB/s 2023-10-09T06:35:25.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:25.975 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2841 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:26.030 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:26.031 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:26.031 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9587e-05 s, 0.0 kB/s 2023-10-09T06:35:26.031 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:26.031 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2842 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:26.087 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:26.087 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:26.087 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.439e-05 s, 0.0 kB/s 2023-10-09T06:35:26.087 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:26.087 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2843 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:26.143 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:26.144 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:26.144 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1571e-05 s, 0.0 kB/s 2023-10-09T06:35:26.145 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:26.145 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2844 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:26.199 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:26.199 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:26.199 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8546e-05 s, 0.0 kB/s 2023-10-09T06:35:26.200 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:26.200 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2845 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:26.255 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:26.255 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:26.255 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9191e-05 s, 0.0 kB/s 2023-10-09T06:35:26.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:26.256 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2846 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:26.311 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:26.311 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:26.311 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9807e-05 s, 0.0 kB/s 2023-10-09T06:35:26.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:26.312 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2847 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:26.367 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:26.367 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:26.367 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0911e-05 s, 0.0 kB/s 2023-10-09T06:35:26.368 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:26.368 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2848 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:26.423 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:26.423 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:26.423 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7595e-05 s, 0.0 kB/s 2023-10-09T06:35:26.423 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:26.423 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2849 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:26.480 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:26.480 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:26.480 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1326e-05 s, 0.0 kB/s 2023-10-09T06:35:26.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:26.480 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2850 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:26.537 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:26.537 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:26.537 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.361e-05 s, 0.0 kB/s 2023-10-09T06:35:26.538 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:26.538 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2851 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:26.594 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:26.594 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:26.594 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1062e-05 s, 0.0 kB/s 2023-10-09T06:35:26.594 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:26.595 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2852 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:26.650 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:26.650 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:26.650 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1583e-05 s, 0.0 kB/s 2023-10-09T06:35:26.651 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:26.651 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2853 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:26.706 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:26.706 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:26.707 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3397e-05 s, 0.0 kB/s 2023-10-09T06:35:26.707 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:26.707 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2854 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:26.762 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:26.762 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:26.763 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3128e-05 s, 0.0 kB/s 2023-10-09T06:35:26.763 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:26.763 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2855 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:26.818 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:26.818 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:26.818 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1949e-05 s, 0.0 kB/s 2023-10-09T06:35:26.819 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:26.819 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2856 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:26.874 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:26.875 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:26.875 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2483e-05 s, 0.0 kB/s 2023-10-09T06:35:26.875 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:26.875 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2857 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:26.930 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:26.931 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:26.931 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9609e-05 s, 0.0 kB/s 2023-10-09T06:35:26.931 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:26.931 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2858 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:26.986 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:26.987 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:26.987 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8866e-05 s, 0.0 kB/s 2023-10-09T06:35:26.987 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:26.987 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2859 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:27.042 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:27.043 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:27.043 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1511e-05 s, 0.0 kB/s 2023-10-09T06:35:27.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:27.043 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2860 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:27.098 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:27.098 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:27.099 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2579e-05 s, 0.0 kB/s 2023-10-09T06:35:27.099 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:27.100 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2861 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:27.155 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:27.155 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:27.156 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1134e-05 s, 0.0 kB/s 2023-10-09T06:35:27.156 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:27.156 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2862 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:27.211 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:27.212 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:27.212 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0855e-05 s, 0.0 kB/s 2023-10-09T06:35:27.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:27.212 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2863 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:27.267 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:27.267 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:27.268 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0734e-05 s, 0.0 kB/s 2023-10-09T06:35:27.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:27.268 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2864 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:27.323 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:27.324 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:27.324 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0813e-05 s, 0.0 kB/s 2023-10-09T06:35:27.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:27.324 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2865 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:27.380 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:27.380 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:27.380 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1038e-05 s, 0.0 kB/s 2023-10-09T06:35:27.380 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:27.380 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2866 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:27.435 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:27.436 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:27.436 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0946e-05 s, 0.0 kB/s 2023-10-09T06:35:27.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:27.436 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2867 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:27.492 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:27.548 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:27.548 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0169e-05 s, 0.0 kB/s 2023-10-09T06:35:27.549 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:27.549 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2868 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:27.565 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:27.565 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:27.565 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0865e-05 s, 0.0 kB/s 2023-10-09T06:35:27.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:27.565 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2869 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:27.620 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:27.621 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:27.621 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0665e-05 s, 0.0 kB/s 2023-10-09T06:35:27.621 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:27.621 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2870 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:27.676 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:27.677 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:27.677 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.891e-05 s, 0.0 kB/s 2023-10-09T06:35:27.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:27.677 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2871 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:27.757 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:27.758 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:27.758 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.9755e-05 s, 0.0 kB/s 2023-10-09T06:35:27.758 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:27.758 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2872 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:27.813 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:27.814 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:27.814 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0235e-05 s, 0.0 kB/s 2023-10-09T06:35:27.814 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:27.814 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2873 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:27.870 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:27.870 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:27.870 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.331e-05 s, 0.0 kB/s 2023-10-09T06:35:27.870 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:27.870 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2874 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:27.926 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:27.926 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:27.926 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4659e-05 s, 0.0 kB/s 2023-10-09T06:35:27.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:27.926 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2875 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:27.982 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:27.983 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:27.983 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7098e-05 s, 0.0 kB/s 2023-10-09T06:35:27.983 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:27.983 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2876 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:28.038 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:28.039 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:28.039 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3832e-05 s, 0.0 kB/s 2023-10-09T06:35:28.039 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:28.039 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2877 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:28.094 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:28.095 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:28.095 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2919e-05 s, 0.0 kB/s 2023-10-09T06:35:28.095 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:28.095 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2878 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:28.150 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:28.151 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:28.151 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1859e-05 s, 0.0 kB/s 2023-10-09T06:35:28.151 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:28.151 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2879 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:28.207 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:28.207 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:28.207 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2549e-05 s, 0.0 kB/s 2023-10-09T06:35:28.207 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:28.208 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2880 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:28.265 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:28.265 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:28.265 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2609e-05 s, 0.0 kB/s 2023-10-09T06:35:28.265 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:28.265 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2881 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:28.321 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:28.321 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:28.322 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0205e-05 s, 0.0 kB/s 2023-10-09T06:35:28.322 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:28.322 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2882 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:28.377 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:28.378 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:28.378 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9259e-05 s, 0.0 kB/s 2023-10-09T06:35:28.378 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:28.378 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2883 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:28.433 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:28.434 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:28.434 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.122e-05 s, 0.0 kB/s 2023-10-09T06:35:28.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:28.434 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2884 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:28.490 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:28.490 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:28.490 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.182e-05 s, 0.0 kB/s 2023-10-09T06:35:28.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:28.490 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2885 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:28.546 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:28.546 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:28.546 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0464e-05 s, 0.0 kB/s 2023-10-09T06:35:28.546 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:28.546 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2886 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:28.602 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:28.602 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:28.602 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0822e-05 s, 0.0 kB/s 2023-10-09T06:35:28.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:28.603 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2887 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:28.658 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:28.658 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:28.658 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0535e-05 s, 0.0 kB/s 2023-10-09T06:35:28.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:28.659 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2888 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:28.715 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:28.715 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:28.715 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.146e-05 s, 0.0 kB/s 2023-10-09T06:35:28.715 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:28.715 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2889 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:28.770 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:28.770 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:28.771 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9711e-05 s, 0.0 kB/s 2023-10-09T06:35:28.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:28.771 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2890 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:28.826 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:28.827 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:28.827 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1151e-05 s, 0.0 kB/s 2023-10-09T06:35:28.827 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:28.828 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2891 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:28.882 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:28.882 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:28.882 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1923e-05 s, 0.0 kB/s 2023-10-09T06:35:28.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:28.883 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2892 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:28.938 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:28.938 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:28.938 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2045e-05 s, 0.0 kB/s 2023-10-09T06:35:28.938 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:28.939 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2893 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:28.994 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:28.994 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:28.994 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2899e-05 s, 0.0 kB/s 2023-10-09T06:35:28.994 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:28.995 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2894 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:29.050 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:29.050 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:29.051 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8361e-05 s, 0.0 kB/s 2023-10-09T06:35:29.051 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:29.051 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2895 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:29.106 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:29.106 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:29.106 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2429e-05 s, 0.0 kB/s 2023-10-09T06:35:29.106 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:29.107 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2896 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:29.163 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:29.163 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:29.163 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1949e-05 s, 0.0 kB/s 2023-10-09T06:35:29.163 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:29.163 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2897 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:29.220 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:29.220 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:29.220 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1557e-05 s, 0.0 kB/s 2023-10-09T06:35:29.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:29.220 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2898 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:29.275 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:29.276 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:29.276 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.764e-05 s, 0.0 kB/s 2023-10-09T06:35:29.276 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:29.276 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2899 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:29.331 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:29.331 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:29.332 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2583e-05 s, 0.0 kB/s 2023-10-09T06:35:29.332 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:29.332 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2900 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:29.387 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:29.387 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:29.387 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2413e-05 s, 0.0 kB/s 2023-10-09T06:35:29.388 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:29.388 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2901 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:29.444 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:29.444 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:29.444 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3368e-05 s, 0.0 kB/s 2023-10-09T06:35:29.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:29.444 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2902 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:29.499 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:29.499 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:29.499 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.9908e-05 s, 0.0 kB/s 2023-10-09T06:35:29.501 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:29.501 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2903 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:29.556 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:29.556 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:29.556 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2111e-05 s, 0.0 kB/s 2023-10-09T06:35:29.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:29.557 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2904 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:29.612 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:29.612 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:29.613 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9948e-05 s, 0.0 kB/s 2023-10-09T06:35:29.613 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:29.613 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2905 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:29.668 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:29.668 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:29.669 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1104e-05 s, 0.0 kB/s 2023-10-09T06:35:29.669 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:29.669 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2906 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:29.724 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:29.725 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:29.725 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.132e-05 s, 0.0 kB/s 2023-10-09T06:35:29.725 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:29.725 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2907 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:29.780 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:29.780 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:29.780 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3168e-05 s, 0.0 kB/s 2023-10-09T06:35:29.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:29.781 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2908 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:29.836 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:29.836 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:29.836 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0547e-05 s, 0.0 kB/s 2023-10-09T06:35:29.836 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:29.837 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2909 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:29.892 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:29.892 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:29.892 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0248e-05 s, 0.0 kB/s 2023-10-09T06:35:29.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:29.893 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2910 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:29.948 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:29.948 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:29.948 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0069e-05 s, 0.0 kB/s 2023-10-09T06:35:29.949 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:29.949 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2911 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:30.004 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:30.004 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:30.004 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1989e-05 s, 0.0 kB/s 2023-10-09T06:35:30.004 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:30.004 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2912 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:30.060 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:30.060 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:30.060 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0557e-05 s, 0.0 kB/s 2023-10-09T06:35:30.061 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:30.061 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2913 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:30.116 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:30.116 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:30.116 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0744e-05 s, 0.0 kB/s 2023-10-09T06:35:30.117 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:30.117 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2914 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:30.172 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:30.173 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:30.173 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2086e-05 s, 0.0 kB/s 2023-10-09T06:35:30.173 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:30.173 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2915 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:30.228 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:30.228 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:30.228 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1864e-05 s, 0.0 kB/s 2023-10-09T06:35:30.229 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:30.229 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2916 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:30.285 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:30.285 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:30.285 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9521e-05 s, 0.0 kB/s 2023-10-09T06:35:30.285 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:30.285 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2917 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:30.340 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:30.340 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:30.341 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2118e-05 s, 0.0 kB/s 2023-10-09T06:35:30.341 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:30.341 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2918 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:30.396 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:30.397 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:30.397 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9963e-05 s, 0.0 kB/s 2023-10-09T06:35:30.397 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:30.397 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2919 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:30.452 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:30.453 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:30.453 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4683e-05 s, 0.0 kB/s 2023-10-09T06:35:30.453 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:30.453 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2920 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:30.508 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:30.508 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:30.508 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3031e-05 s, 0.0 kB/s 2023-10-09T06:35:30.509 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:30.509 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2921 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:30.565 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:30.565 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:30.565 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2391e-05 s, 0.0 kB/s 2023-10-09T06:35:30.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:30.565 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2922 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:30.621 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:30.621 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:30.622 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7558e-05 s, 0.0 kB/s 2023-10-09T06:35:30.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:30.622 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2923 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:30.677 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:30.677 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:30.677 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9817e-05 s, 0.0 kB/s 2023-10-09T06:35:30.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:30.677 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2924 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:30.733 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:30.734 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:30.734 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1109e-05 s, 0.0 kB/s 2023-10-09T06:35:30.734 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:30.734 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2925 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:30.791 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:30.791 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:30.791 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3349e-05 s, 0.0 kB/s 2023-10-09T06:35:30.791 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:30.791 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2926 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:30.846 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:30.847 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:30.847 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9621e-05 s, 0.0 kB/s 2023-10-09T06:35:30.847 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:30.847 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2927 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:30.902 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:30.902 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:30.902 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1215e-05 s, 0.0 kB/s 2023-10-09T06:35:30.903 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:30.903 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2928 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:30.958 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:30.958 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:30.958 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0537e-05 s, 0.0 kB/s 2023-10-09T06:35:30.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:30.959 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2929 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:31.014 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:31.014 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:31.014 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1432e-05 s, 0.0 kB/s 2023-10-09T06:35:31.015 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:31.015 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2930 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:31.071 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:31.071 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:31.071 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.2958e-05 s, 0.0 kB/s 2023-10-09T06:35:31.071 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:31.071 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2931 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:31.127 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:31.128 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:31.128 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1525e-05 s, 0.0 kB/s 2023-10-09T06:35:31.128 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:31.128 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2932 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:31.183 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:31.183 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:31.183 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8371e-05 s, 0.0 kB/s 2023-10-09T06:35:31.184 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:31.184 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2933 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:31.239 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:31.239 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:31.240 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8613e-05 s, 0.0 kB/s 2023-10-09T06:35:31.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:31.240 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2934 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:31.295 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:31.295 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:31.295 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9334e-05 s, 0.0 kB/s 2023-10-09T06:35:31.295 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:31.295 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2935 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:31.350 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:31.350 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:31.350 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8625e-05 s, 0.0 kB/s 2023-10-09T06:35:31.351 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:31.351 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2936 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:31.406 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:31.406 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:31.406 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0981e-05 s, 0.0 kB/s 2023-10-09T06:35:31.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:31.406 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2937 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:31.462 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:31.462 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:31.462 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9503e-05 s, 0.0 kB/s 2023-10-09T06:35:31.462 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:31.462 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2938 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:31.518 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:31.518 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:31.518 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2365e-05 s, 0.0 kB/s 2023-10-09T06:35:31.518 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:31.518 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2939 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:31.574 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:31.575 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:31.575 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1004e-05 s, 0.0 kB/s 2023-10-09T06:35:31.575 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:31.575 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2940 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:31.630 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:31.630 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:31.630 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2963e-05 s, 0.0 kB/s 2023-10-09T06:35:31.630 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:31.630 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2941 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:31.686 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:31.686 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:31.687 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1398e-05 s, 0.0 kB/s 2023-10-09T06:35:31.687 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:31.687 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2942 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:31.742 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:31.742 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:31.742 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0119e-05 s, 0.0 kB/s 2023-10-09T06:35:31.743 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:31.743 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2943 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:31.797 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:31.798 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:31.798 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.026e-05 s, 0.0 kB/s 2023-10-09T06:35:31.799 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:31.799 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2944 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:31.854 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:31.854 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:31.854 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8646e-05 s, 0.0 kB/s 2023-10-09T06:35:31.855 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:31.855 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2945 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:31.910 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:31.910 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:31.910 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1275e-05 s, 0.0 kB/s 2023-10-09T06:35:31.911 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:31.911 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2946 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:31.966 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:31.966 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:31.967 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0058e-05 s, 0.0 kB/s 2023-10-09T06:35:31.967 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:31.967 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2947 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:32.022 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:32.022 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:32.022 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3652e-05 s, 0.0 kB/s 2023-10-09T06:35:32.022 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:32.023 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2948 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:32.078 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:32.078 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:32.078 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8273e-05 s, 0.0 kB/s 2023-10-09T06:35:32.079 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:32.079 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2949 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:32.134 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:32.134 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:32.134 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2768e-05 s, 0.0 kB/s 2023-10-09T06:35:32.134 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:32.135 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2950 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:32.190 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:32.190 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:32.190 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8282e-05 s, 0.0 kB/s 2023-10-09T06:35:32.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:32.191 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2951 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:32.246 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:32.246 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:32.246 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8768e-05 s, 0.0 kB/s 2023-10-09T06:35:32.246 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:32.246 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2952 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:32.302 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:32.302 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:32.302 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1686e-05 s, 0.0 kB/s 2023-10-09T06:35:32.303 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:32.303 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2953 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:32.358 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:32.359 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:32.359 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1501e-05 s, 0.0 kB/s 2023-10-09T06:35:32.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:32.359 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2954 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:32.414 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:32.414 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:32.414 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1943e-05 s, 0.0 kB/s 2023-10-09T06:35:32.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:32.415 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2955 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:32.471 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:32.471 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:32.471 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2424e-05 s, 0.0 kB/s 2023-10-09T06:35:32.472 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:32.472 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2956 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:32.527 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:32.527 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:32.527 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3839e-05 s, 0.0 kB/s 2023-10-09T06:35:32.528 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:32.528 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2957 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:32.584 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:32.584 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:32.584 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3008e-05 s, 0.0 kB/s 2023-10-09T06:35:32.584 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:32.584 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2958 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:32.640 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:32.640 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:32.641 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9602e-05 s, 0.0 kB/s 2023-10-09T06:35:32.641 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:32.641 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2959 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:32.696 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:32.696 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:32.696 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9338e-05 s, 0.0 kB/s 2023-10-09T06:35:32.697 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:32.697 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2960 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:32.752 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:32.752 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:32.753 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.102e-05 s, 0.0 kB/s 2023-10-09T06:35:32.753 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:32.753 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2961 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:32.808 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:32.808 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:32.809 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2569e-05 s, 0.0 kB/s 2023-10-09T06:35:32.809 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:32.809 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2962 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:32.872 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:32.873 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:32.873 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 4.2007e-05 s, 0.0 kB/s 2023-10-09T06:35:32.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:32.873 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2963 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:32.928 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:32.928 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:32.929 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8554e-05 s, 0.0 kB/s 2023-10-09T06:35:32.929 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:32.929 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2964 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:32.984 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:32.984 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:32.984 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.4586e-05 s, 0.0 kB/s 2023-10-09T06:35:32.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:32.985 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2965 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:33.041 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:33.041 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:33.041 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0582e-05 s, 0.0 kB/s 2023-10-09T06:35:33.041 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:33.042 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2966 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:33.096 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:33.096 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:33.096 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1119e-05 s, 0.0 kB/s 2023-10-09T06:35:33.096 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:33.096 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2967 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:33.152 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:33.152 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:33.152 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2011e-05 s, 0.0 kB/s 2023-10-09T06:35:33.153 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:33.153 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2968 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:33.208 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:33.209 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:33.209 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9558e-05 s, 0.0 kB/s 2023-10-09T06:35:33.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:33.210 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2969 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:33.263 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:33.263 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:33.263 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1131e-05 s, 0.0 kB/s 2023-10-09T06:35:33.263 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:33.264 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2970 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:33.319 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:33.319 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:33.319 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9754e-05 s, 0.0 kB/s 2023-10-09T06:35:33.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:33.319 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2971 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:33.375 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:33.375 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:33.375 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0586e-05 s, 0.0 kB/s 2023-10-09T06:35:33.376 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:33.376 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2972 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:33.431 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:33.431 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:33.431 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1547e-05 s, 0.0 kB/s 2023-10-09T06:35:33.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:33.432 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2973 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:33.487 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:33.487 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:33.487 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0139e-05 s, 0.0 kB/s 2023-10-09T06:35:33.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:33.487 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2974 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:33.543 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:33.544 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:33.544 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.2448e-05 s, 0.0 kB/s 2023-10-09T06:35:33.544 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:33.544 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2975 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:33.600 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:33.600 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:33.600 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.8392e-05 s, 0.0 kB/s 2023-10-09T06:35:33.600 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:33.600 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2976 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:33.655 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:33.655 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:33.656 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1654e-05 s, 0.0 kB/s 2023-10-09T06:35:33.656 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:33.656 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2977 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:33.711 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:33.711 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:33.711 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.9184e-05 s, 0.0 kB/s 2023-10-09T06:35:33.712 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:33.712 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2978 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:33.767 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:33.767 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:33.768 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0483e-05 s, 0.0 kB/s 2023-10-09T06:35:33.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:33.768 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2979 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:33.823 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:33.823 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:33.823 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.3306e-05 s, 0.0 kB/s 2023-10-09T06:35:33.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:33.824 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2980 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:33.880 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:33.880 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:33.880 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0219e-05 s, 0.0 kB/s 2023-10-09T06:35:33.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:33.880 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2981 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:33.935 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:33.935 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:33.935 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8199e-05 s, 0.0 kB/s 2023-10-09T06:35:33.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:33.935 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2982 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:33.991 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:33.991 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:33.991 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0324e-05 s, 0.0 kB/s 2023-10-09T06:35:33.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:33.992 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2983 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:34.047 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:34.047 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:34.047 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9058e-05 s, 0.0 kB/s 2023-10-09T06:35:34.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:34.048 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2984 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:34.103 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:34.103 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:34.103 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8562e-05 s, 0.0 kB/s 2023-10-09T06:35:34.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:34.104 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2985 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:34.159 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:34.159 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:34.159 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.192e-05 s, 0.0 kB/s 2023-10-09T06:35:34.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:34.160 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2986 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:34.215 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:34.215 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:34.215 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7913e-05 s, 0.0 kB/s 2023-10-09T06:35:34.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:34.216 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2987 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:34.270 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:34.270 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:34.271 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.7779e-05 s, 0.0 kB/s 2023-10-09T06:35:34.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:34.272 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2988 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:34.327 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:34.327 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:34.327 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1228e-05 s, 0.0 kB/s 2023-10-09T06:35:34.327 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:34.328 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2989 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:34.383 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:34.383 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:34.384 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0085e-05 s, 0.0 kB/s 2023-10-09T06:35:34.384 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:34.384 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2990 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:34.439 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:34.439 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:34.439 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0399e-05 s, 0.0 kB/s 2023-10-09T06:35:34.439 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:34.440 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2991 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:34.496 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:34.496 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:34.496 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9196e-05 s, 0.0 kB/s 2023-10-09T06:35:34.496 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:34.497 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2992 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:34.551 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:34.552 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:34.552 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1143e-05 s, 0.0 kB/s 2023-10-09T06:35:34.552 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:34.552 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2993 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:34.608 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:34.608 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:34.608 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1752e-05 s, 0.0 kB/s 2023-10-09T06:35:34.608 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:34.608 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2994 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:34.664 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:34.664 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:34.665 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.9992e-05 s, 0.0 kB/s 2023-10-09T06:35:34.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:34.665 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2995 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:34.720 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:34.720 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:34.720 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.993e-05 s, 0.0 kB/s 2023-10-09T06:35:34.720 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:34.720 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2996 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:34.776 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:34.776 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:34.776 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0906e-05 s, 0.0 kB/s 2023-10-09T06:35:34.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:34.777 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2997 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:34.832 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:34.833 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:34.833 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 2.8969e-05 s, 0.0 kB/s 2023-10-09T06:35:34.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:34.833 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2998 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:34.888 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:34.888 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:34.889 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.0242e-05 s, 0.0 kB/s 2023-10-09T06:35:34.889 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:34.889 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000333763/18187d1e-0b8d-46e0-946a-a0d433b334cd/subvolume_file.2999 bs=1M conv=fdatasync count=0 seek=0) 2023-10-09T06:35:34.944 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records in 2023-10-09T06:35:34.944 INFO:teuthology.orchestra.run.smithi163.stderr:0+0 records out 2023-10-09T06:35:34.944 INFO:teuthology.orchestra.run.smithi163.stderr:0 bytes copied, 3.1109e-05 s, 0.0 kB/s 2023-10-09T06:35:34.944 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000333763 2023-10-09T06:35:35.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.102+0000 7fe46f109700 1 Processor -- start 2023-10-09T06:35:35.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.102+0000 7fe46f109700 1 -- start start 2023-10-09T06:35:35.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.103+0000 7fe46f109700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe468159890 0x7fe46815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:35.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.103+0000 7fe46f109700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe468069560 0x7fe4680699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:35.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.103+0000 7fe46f109700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe468069ee0 con 0x7fe468159c60 2023-10-09T06:35:35.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.103+0000 7fe46f109700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe46815a160 con 0x7fe468159890 2023-10-09T06:35:35.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.103+0000 7fe46f109700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4680d13a0 con 0x7fe468069560 2023-10-09T06:35:35.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.103+0000 7fe46cea5700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe468159890 0x7fe46815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:35.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.103+0000 7fe46cea5700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe468159890 0x7fe46815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:50568/0 (socket says 172.21.15.163:50568) 2023-10-09T06:35:35.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.103+0000 7fe46cea5700 1 -- 172.21.15.163:0/3865799485 learned_addr learned my addr 172.21.15.163:0/3865799485 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:35.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.103+0000 7fe46d6a6700 1 --2- 172.21.15.163:0/3865799485 >> v2:172.21.15.181:3300/0 conn(0x7fe468069560 0x7fe4680699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:35.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.104+0000 7fe46cea5700 1 -- 172.21.15.163:0/3865799485 >> v2:172.21.15.181:3300/0 conn(0x7fe468069560 msgr2=0x7fe4680699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:35.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.104+0000 7fe46cea5700 1 --2- 172.21.15.163:0/3865799485 >> v2:172.21.15.181:3300/0 conn(0x7fe468069560 0x7fe4680699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:35.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.104+0000 7fe46cea5700 1 -- 172.21.15.163:0/3865799485 >> v1:172.21.15.181:6789/0 conn(0x7fe468159c60 legacy=0x7fe46815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:35.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.104+0000 7fe46cea5700 1 -- 172.21.15.163:0/3865799485 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46815e140 con 0x7fe468159890 2023-10-09T06:35:35.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.104+0000 7fe46d6a6700 1 --2- 172.21.15.163:0/3865799485 >> v2:172.21.15.181:3300/0 conn(0x7fe468069560 0x7fe4680699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:35:35.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.104+0000 7fe46cea5700 1 --2- 172.21.15.163:0/3865799485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe468159890 0x7fe46815abd0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe46806a890 tx=0x7fe458009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a57ef3c7cefc5e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:35.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.104+0000 7fe4677fe700 1 -- 172.21.15.163:0/3865799485 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45800e070 con 0x7fe468159890 2023-10-09T06:35:35.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.104+0000 7fe4677fe700 1 -- 172.21.15.163:0/3865799485 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe45800ba50 con 0x7fe468159890 2023-10-09T06:35:35.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.104+0000 7fe4677fe700 1 -- 172.21.15.163:0/3865799485 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4580092e0 con 0x7fe468159890 2023-10-09T06:35:35.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.104+0000 7fe46f109700 1 -- 172.21.15.163:0/3865799485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe468159890 msgr2=0x7fe46815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:35.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.104+0000 7fe46f109700 1 --2- 172.21.15.163:0/3865799485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe468159890 0x7fe46815abd0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe46806a890 tx=0x7fe458009840 comp rx=0 tx=0).stop 2023-10-09T06:35:35.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.104+0000 7fe46f109700 1 -- 172.21.15.163:0/3865799485 shutdown_connections 2023-10-09T06:35:35.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.104+0000 7fe46f109700 1 --2- 172.21.15.163:0/3865799485 >> v2:172.21.15.181:3300/0 conn(0x7fe468069560 0x7fe4680699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:35.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.104+0000 7fe46f109700 1 --2- 172.21.15.163:0/3865799485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe468159890 0x7fe46815abd0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:35.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.104+0000 7fe46f109700 1 -- 172.21.15.163:0/3865799485 >> 172.21.15.163:0/3865799485 conn(0x7fe468068e20 msgr2=0x7fe4680d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:35.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.104+0000 7fe46f109700 1 -- 172.21.15.163:0/3865799485 shutdown_connections 2023-10-09T06:35:35.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.104+0000 7fe46f109700 1 -- 172.21.15.163:0/3865799485 wait complete. 2023-10-09T06:35:35.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.106+0000 7fe46f109700 1 Processor -- start 2023-10-09T06:35:35.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.106+0000 7fe46f109700 1 -- start start 2023-10-09T06:35:35.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.106+0000 7fe46f109700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe468069560 0x7fe46816a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:35.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.107+0000 7fe46f109700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe468159890 0x7fe4680cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:35.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.107+0000 7fe46f109700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe468159c60 0x7fe4680cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:35.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.107+0000 7fe46f109700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46816b220 con 0x7fe468159890 2023-10-09T06:35:35.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.107+0000 7fe46f109700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4680cdc80 con 0x7fe468069560 2023-10-09T06:35:35.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.107+0000 7fe46f109700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4680cdf60 con 0x7fe468159c60 2023-10-09T06:35:35.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.107+0000 7fe467fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe468159890 0x7fe4680cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:35.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.107+0000 7fe467fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe468159890 0x7fe4680cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:50580/0 (socket says 172.21.15.163:50580) 2023-10-09T06:35:35.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.107+0000 7fe467fff700 1 -- 172.21.15.163:0/45801413 learned_addr learned my addr 172.21.15.163:0/45801413 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:35.111 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.107+0000 7fe46cea5700 1 --2- 172.21.15.163:0/45801413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe468069560 0x7fe46816a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:35.111 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.107+0000 7fe46d6a6700 1 --2- 172.21.15.163:0/45801413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe468159c60 0x7fe4680cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:35.111 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.107+0000 7fe467fff700 1 -- 172.21.15.163:0/45801413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe468159c60 msgr2=0x7fe4680cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:35.111 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.107+0000 7fe467fff700 1 --2- 172.21.15.163:0/45801413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe468159c60 0x7fe4680cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:35.111 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.107+0000 7fe467fff700 1 -- 172.21.15.163:0/45801413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe468069560 msgr2=0x7fe46816a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:35.111 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.107+0000 7fe467fff700 1 --2- 172.21.15.163:0/45801413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe468069560 0x7fe46816a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:35.111 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.107+0000 7fe467fff700 1 -- 172.21.15.163:0/45801413 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe458009ae0 con 0x7fe468159890 2023-10-09T06:35:35.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.107+0000 7fe467fff700 1 --2- 172.21.15.163:0/45801413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe468159890 0x7fe4680cd0c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00c4e0 tx=0x7fe45c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:35.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.107+0000 7fe464ff9700 1 -- 172.21.15.163:0/45801413 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45c011590 con 0x7fe468159890 2023-10-09T06:35:35.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.107+0000 7fe464ff9700 1 -- 172.21.15.163:0/45801413 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe45c011710 con 0x7fe468159890 2023-10-09T06:35:35.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.107+0000 7fe464ff9700 1 -- 172.21.15.163:0/45801413 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45c011ad0 con 0x7fe468159890 2023-10-09T06:35:35.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.107+0000 7fe46f109700 1 -- 172.21.15.163:0/45801413 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4680ce240 con 0x7fe468159890 2023-10-09T06:35:35.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.107+0000 7fe46f109700 1 -- 172.21.15.163:0/45801413 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4680ce800 con 0x7fe468159890 2023-10-09T06:35:35.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.108+0000 7fe46f109700 1 -- 172.21.15.163:0/45801413 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4680cea20 con 0x7fe468159890 2023-10-09T06:35:35.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.109+0000 7fe464ff9700 1 -- 172.21.15.163:0/45801413 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 8) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fe45c01b3b0 con 0x7fe468159890 2023-10-09T06:35:35.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.109+0000 7fe464ff9700 1 --2- 172.21.15.163:0/45801413 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe454066fc0 0x7fe454069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:35.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.109+0000 7fe464ff9700 1 -- 172.21.15.163:0/45801413 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fe45c083920 con 0x7fe468159890 2023-10-09T06:35:35.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.109+0000 7fe46cea5700 1 --2- 172.21.15.163:0/45801413 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe454066fc0 0x7fe454069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:35.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.110+0000 7fe46cea5700 1 --2- 172.21.15.163:0/45801413 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe454066fc0 0x7fe454069470 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe458009880 tx=0x7fe458018000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:35.115 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.113+0000 7fe464ff9700 1 -- 172.21.15.163:0/45801413 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe45c050b60 con 0x7fe468159890 2023-10-09T06:35:35.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.328+0000 7fe46f109700 1 -- 172.21.15.163:0/45801413 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000333763", "target": ["mon-mgr", ""]}) v1 -- 0x7fe46804c3c0 con 0x7fe454066fc0 2023-10-09T06:35:35.332 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:35:35.332+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:35:35.333 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:35:35.332+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:35:35.333 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:35:35.332+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:35:35.333 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:35:35.332+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:35:35.333 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:35:35.332+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:35:35.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.389+0000 7fe464ff9700 1 -- 172.21.15.163:0/45801413 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe46804c3c0 con 0x7fe454066fc0 2023-10-09T06:35:35.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.392+0000 7fe46f109700 1 -- 172.21.15.163:0/45801413 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe454066fc0 msgr2=0x7fe454069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:35.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.392+0000 7fe46f109700 1 --2- 172.21.15.163:0/45801413 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe454066fc0 0x7fe454069470 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe458009880 tx=0x7fe458018000 comp rx=0 tx=0).stop 2023-10-09T06:35:35.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.392+0000 7fe46f109700 1 -- 172.21.15.163:0/45801413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe468159890 msgr2=0x7fe4680cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:35.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.392+0000 7fe46f109700 1 --2- 172.21.15.163:0/45801413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe468159890 0x7fe4680cd0c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00c4e0 tx=0x7fe45c00c7f0 comp rx=0 tx=0).stop 2023-10-09T06:35:35.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.392+0000 7fe46f109700 1 -- 172.21.15.163:0/45801413 shutdown_connections 2023-10-09T06:35:35.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.392+0000 7fe46f109700 1 --2- 172.21.15.163:0/45801413 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe454066fc0 0x7fe454069470 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:35.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.392+0000 7fe46f109700 1 --2- 172.21.15.163:0/45801413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe468159c60 0x7fe4680cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:35.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.392+0000 7fe46f109700 1 --2- 172.21.15.163:0/45801413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe468069560 0x7fe46816a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:35.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.392+0000 7fe46f109700 1 --2- 172.21.15.163:0/45801413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe468159890 0x7fe4680cd0c0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:35.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.392+0000 7fe46f109700 1 -- 172.21.15.163:0/45801413 >> 172.21.15.163:0/45801413 conn(0x7fe468068e20 msgr2=0x7fe468153490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:35.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.392+0000 7fe46f109700 1 -- 172.21.15.163:0/45801413 shutdown_connections 2023-10-09T06:35:35.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.392+0000 7fe46f109700 1 -- 172.21.15.163:0/45801413 wait complete. 2023-10-09T06:35:35.394 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:35:35.391+0000 7ff95c4ea700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:35:35.394 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:35:35.391+0000 7ff95c4ea700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:35:35.394 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:35:35.391+0000 7ff95c4ea700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:35:35.395 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:35:35.391+0000 7ff95c4ea700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:35:35.395 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:35:35.391+0000 7ff95c4ea700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:35:35.404 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume info cephfs 2023-10-09T06:35:35.406 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:35:35.405+0000 7ff95bce9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:35:35.406 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:35:35.405+0000 7ff95bce9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:35:35.406 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:35:35.405+0000 7ff95bce9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:35:35.407 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:35:35.405+0000 7ff95bce9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:35:35.407 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:35:35.405+0000 7ff95bce9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:35:35.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.524+0000 7f9d5b086700 1 Processor -- start 2023-10-09T06:35:35.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.524+0000 7f9d5b086700 1 -- start start 2023-10-09T06:35:35.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.525+0000 7f9d5b086700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9d54063580 0x7f9d5406e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:35.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.525+0000 7f9d5b086700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9d5406e9b0 0x7f9d540b1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:35.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.525+0000 7f9d5b086700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d5406ee10 con 0x7f9d541575a0 2023-10-09T06:35:35.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.525+0000 7f9d5b086700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d540b1100 con 0x7f9d5406e9b0 2023-10-09T06:35:35.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.525+0000 7f9d5b086700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d540c3ae0 con 0x7f9d54063580 2023-10-09T06:35:35.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.526+0000 7f9d5a885700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9d5406e9b0 0x7f9d540b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:35.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.526+0000 7f9d5a885700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9d5406e9b0 0x7f9d540b1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:50588/0 (socket says 172.21.15.163:50588) 2023-10-09T06:35:35.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.526+0000 7f9d5a885700 1 -- 172.21.15.163:0/3961294753 learned_addr learned my addr 172.21.15.163:0/3961294753 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:35.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.526+0000 7f9d59883700 1 --2- 172.21.15.163:0/3961294753 >> v2:172.21.15.181:3300/0 conn(0x7f9d54063580 0x7f9d5406e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:35.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.526+0000 7f9d5a885700 1 -- 172.21.15.163:0/3961294753 >> v2:172.21.15.181:3300/0 conn(0x7f9d54063580 msgr2=0x7f9d5406e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:35.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.526+0000 7f9d5a885700 1 --2- 172.21.15.163:0/3961294753 >> v2:172.21.15.181:3300/0 conn(0x7f9d54063580 0x7f9d5406e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:35.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.526+0000 7f9d5a885700 1 -- 172.21.15.163:0/3961294753 >> v1:172.21.15.181:6789/0 conn(0x7f9d541575a0 legacy=0x7f9d540cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:35.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.526+0000 7f9d5a885700 1 -- 172.21.15.163:0/3961294753 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d540cbe60 con 0x7f9d5406e9b0 2023-10-09T06:35:35.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.526+0000 7f9d5a885700 1 --2- 172.21.15.163:0/3961294753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d5406e9b0 0x7f9d540b1590 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9d540b1070 tx=0x7f9d4c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1d7dc2e3e2e8df server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:35.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.526+0000 7f9d59082700 1 -- 172.21.15.163:0/3961294753 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c00e070 con 0x7f9d5406e9b0 2023-10-09T06:35:35.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.526+0000 7f9d59082700 1 -- 172.21.15.163:0/3961294753 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4c00ba20 con 0x7f9d5406e9b0 2023-10-09T06:35:35.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.526+0000 7f9d59082700 1 -- 172.21.15.163:0/3961294753 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c0092e0 con 0x7f9d5406e9b0 2023-10-09T06:35:35.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.527+0000 7f9d5b086700 1 -- 172.21.15.163:0/3961294753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d5406e9b0 msgr2=0x7f9d540b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:35.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.527+0000 7f9d5b086700 1 --2- 172.21.15.163:0/3961294753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d5406e9b0 0x7f9d540b1590 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9d540b1070 tx=0x7f9d4c009840 comp rx=0 tx=0).stop 2023-10-09T06:35:35.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.527+0000 7f9d5b086700 1 -- 172.21.15.163:0/3961294753 shutdown_connections 2023-10-09T06:35:35.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.527+0000 7f9d5b086700 1 --2- 172.21.15.163:0/3961294753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d5406e9b0 0x7f9d540b1590 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:35.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.527+0000 7f9d5b086700 1 --2- 172.21.15.163:0/3961294753 >> v2:172.21.15.181:3300/0 conn(0x7f9d54063580 0x7f9d5406e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:35.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.527+0000 7f9d5b086700 1 -- 172.21.15.163:0/3961294753 >> 172.21.15.163:0/3961294753 conn(0x7f9d540c9f70 msgr2=0x7f9d5406dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:35.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.527+0000 7f9d5b086700 1 -- 172.21.15.163:0/3961294753 shutdown_connections 2023-10-09T06:35:35.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.527+0000 7f9d5b086700 1 -- 172.21.15.163:0/3961294753 wait complete. 2023-10-09T06:35:35.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.528+0000 7f9d5b086700 1 Processor -- start 2023-10-09T06:35:35.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.528+0000 7f9d5b086700 1 -- start start 2023-10-09T06:35:35.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.529+0000 7f9d5b086700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d54063580 0x7f9d54149260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:35.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.529+0000 7f9d5b086700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d5406e9b0 0x7f9d541553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:35.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.529+0000 7f9d5b086700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d541575a0 0x7f9d540c1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:35.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.529+0000 7f9d5b086700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d541497a0 con 0x7f9d5406e9b0 2023-10-09T06:35:35.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.529+0000 7f9d5b086700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d540c3e80 con 0x7f9d54063580 2023-10-09T06:35:35.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.529+0000 7f9d5b086700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d540c4160 con 0x7f9d541575a0 2023-10-09T06:35:35.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.529+0000 7f9d59883700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d5406e9b0 0x7f9d541553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:35.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.529+0000 7f9d59883700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d5406e9b0 0x7f9d541553d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:50594/0 (socket says 172.21.15.163:50594) 2023-10-09T06:35:35.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.529+0000 7f9d59883700 1 -- 172.21.15.163:0/2229153105 learned_addr learned my addr 172.21.15.163:0/2229153105 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:35.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.529+0000 7f9d5a084700 1 --2- 172.21.15.163:0/2229153105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d54063580 0x7f9d54149260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:35.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.529+0000 7f9d5a885700 1 --2- 172.21.15.163:0/2229153105 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d541575a0 0x7f9d540c1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:35.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.530+0000 7f9d59883700 1 -- 172.21.15.163:0/2229153105 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d541575a0 msgr2=0x7f9d540c1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:35.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.530+0000 7f9d59883700 1 --2- 172.21.15.163:0/2229153105 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d541575a0 0x7f9d540c1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:35.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.530+0000 7f9d59883700 1 -- 172.21.15.163:0/2229153105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d54063580 msgr2=0x7f9d54149260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:35.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.530+0000 7f9d59883700 1 --2- 172.21.15.163:0/2229153105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d54063580 0x7f9d54149260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:35.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.530+0000 7f9d59883700 1 -- 172.21.15.163:0/2229153105 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4c009ae0 con 0x7f9d5406e9b0 2023-10-09T06:35:35.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.530+0000 7f9d59883700 1 --2- 172.21.15.163:0/2229153105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d5406e9b0 0x7f9d541553d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400ec80 tx=0x7f9d440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:35.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.530+0000 7f9d4a7fc700 1 -- 172.21.15.163:0/2229153105 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4400c850 con 0x7f9d5406e9b0 2023-10-09T06:35:35.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.530+0000 7f9d4a7fc700 1 -- 172.21.15.163:0/2229153105 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4400c9b0 con 0x7f9d5406e9b0 2023-10-09T06:35:35.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.530+0000 7f9d5b086700 1 -- 172.21.15.163:0/2229153105 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d540c4440 con 0x7f9d5406e9b0 2023-10-09T06:35:35.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.530+0000 7f9d4a7fc700 1 -- 172.21.15.163:0/2229153105 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4400cd50 con 0x7f9d5406e9b0 2023-10-09T06:35:35.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.530+0000 7f9d5b086700 1 -- 172.21.15.163:0/2229153105 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d5415a300 con 0x7f9d5406e9b0 2023-10-09T06:35:35.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.531+0000 7f9d5b086700 1 -- 172.21.15.163:0/2229153105 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d5406cfa0 con 0x7f9d5406e9b0 2023-10-09T06:35:35.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.531+0000 7f9d4a7fc700 1 -- 172.21.15.163:0/2229153105 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 8) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f9d44013680 con 0x7f9d5406e9b0 2023-10-09T06:35:35.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.532+0000 7f9d4a7fc700 1 --2- 172.21.15.163:0/2229153105 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9d38066fc0 0x7f9d38069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:35.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.532+0000 7f9d4a7fc700 1 -- 172.21.15.163:0/2229153105 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f9d44084830 con 0x7f9d5406e9b0 2023-10-09T06:35:35.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.532+0000 7f9d5a084700 1 --2- 172.21.15.163:0/2229153105 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9d38066fc0 0x7f9d38069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:35.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.532+0000 7f9d5a084700 1 --2- 172.21.15.163:0/2229153105 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9d38066fc0 0x7f9d38069470 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9d541513f0 tx=0x7f9d50009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:35.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.536+0000 7f9d4a7fc700 1 -- 172.21.15.163:0/2229153105 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d44051a70 con 0x7f9d5406e9b0 2023-10-09T06:35:35.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.765+0000 7f9d5b086700 1 -- 172.21.15.163:0/2229153105 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume info", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d540cc140 con 0x7f9d38066fc0 2023-10-09T06:35:35.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.773+0000 7f9d4a7fc700 1 -- 172.21.15.163:0/2229153105 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+539 (secure 0 0 0) 0x7f9d540cc140 con 0x7f9d38066fc0 2023-10-09T06:35:35.772 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:35:35.773 INFO:teuthology.orchestra.run.smithi163.stdout: "mon_addrs": [ 2023-10-09T06:35:35.773 INFO:teuthology.orchestra.run.smithi163.stdout: "172.21.15.163:6789", 2023-10-09T06:35:35.773 INFO:teuthology.orchestra.run.smithi163.stdout: "172.21.15.181:6789", 2023-10-09T06:35:35.773 INFO:teuthology.orchestra.run.smithi163.stdout: "172.21.15.181:6790" 2023-10-09T06:35:35.773 INFO:teuthology.orchestra.run.smithi163.stdout: ], 2023-10-09T06:35:35.773 INFO:teuthology.orchestra.run.smithi163.stdout: "pending_subvolume_deletions": 1, 2023-10-09T06:35:35.773 INFO:teuthology.orchestra.run.smithi163.stdout: "pools": { 2023-10-09T06:35:35.773 INFO:teuthology.orchestra.run.smithi163.stdout: "data": [ 2023-10-09T06:35:35.773 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:35:35.773 INFO:teuthology.orchestra.run.smithi163.stdout: "avail": 365975896064, 2023-10-09T06:35:35.774 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "cephfs_data", 2023-10-09T06:35:35.774 INFO:teuthology.orchestra.run.smithi163.stdout: "used": 8192 2023-10-09T06:35:35.774 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:35:35.774 INFO:teuthology.orchestra.run.smithi163.stdout: ], 2023-10-09T06:35:35.774 INFO:teuthology.orchestra.run.smithi163.stdout: "metadata": [ 2023-10-09T06:35:35.774 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:35:35.774 INFO:teuthology.orchestra.run.smithi163.stdout: "avail": 365975896064, 2023-10-09T06:35:35.774 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "cephfs_metadata", 2023-10-09T06:35:35.774 INFO:teuthology.orchestra.run.smithi163.stdout: "used": 19881984 2023-10-09T06:35:35.775 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:35:35.775 INFO:teuthology.orchestra.run.smithi163.stdout: ] 2023-10-09T06:35:35.775 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:35:35.775 INFO:teuthology.orchestra.run.smithi163.stdout: "used_size": 145 2023-10-09T06:35:35.775 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:35:35.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.775+0000 7f9d5b086700 1 -- 172.21.15.163:0/2229153105 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9d38066fc0 msgr2=0x7f9d38069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:35.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.775+0000 7f9d5b086700 1 --2- 172.21.15.163:0/2229153105 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9d38066fc0 0x7f9d38069470 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9d541513f0 tx=0x7f9d50009040 comp rx=0 tx=0).stop 2023-10-09T06:35:35.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.775+0000 7f9d5b086700 1 -- 172.21.15.163:0/2229153105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d5406e9b0 msgr2=0x7f9d541553d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:35.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.775+0000 7f9d5b086700 1 --2- 172.21.15.163:0/2229153105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d5406e9b0 0x7f9d541553d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400ec80 tx=0x7f9d440091a0 comp rx=0 tx=0).stop 2023-10-09T06:35:35.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.775+0000 7f9d5b086700 1 -- 172.21.15.163:0/2229153105 shutdown_connections 2023-10-09T06:35:35.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.775+0000 7f9d5b086700 1 --2- 172.21.15.163:0/2229153105 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9d38066fc0 0x7f9d38069470 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:35.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.775+0000 7f9d5b086700 1 --2- 172.21.15.163:0/2229153105 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d541575a0 0x7f9d540c1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:35.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.775+0000 7f9d5b086700 1 --2- 172.21.15.163:0/2229153105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d54063580 0x7f9d54149260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:35.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.775+0000 7f9d5b086700 1 --2- 172.21.15.163:0/2229153105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d5406e9b0 0x7f9d541553d0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:35.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.775+0000 7f9d5b086700 1 -- 172.21.15.163:0/2229153105 >> 172.21.15.163:0/2229153105 conn(0x7f9d540c9f70 msgr2=0x7f9d540b1590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:35.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.775+0000 7f9d5b086700 1 -- 172.21.15.163:0/2229153105 shutdown_connections 2023-10-09T06:35:35.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:35.775+0000 7f9d5b086700 1 -- 172.21.15.163:0/2229153105 wait complete. 2023-10-09T06:35:35.787 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-09T06:35:35.787 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:35:35.824 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:35:35.865 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-09T06:35:35.865 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-09T06:35:35.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:35.865 DEBUG:teuthology.orchestra.run.smithi163:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-09T06:35:36.419 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi163.front.sepia.ceph.com 2023-10-09T06:35:36.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:35:36.419 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-09T06:35:36.435 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-09T06:35:36.435 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:35:36.435 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-09T06:35:36.929 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-09T06:35:36.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:35:36.930 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-09T06:35:36.944 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:35:37.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.083+0000 7f47931f6700 1 Processor -- start 2023-10-09T06:35:37.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.084+0000 7f47931f6700 1 -- start start 2023-10-09T06:35:37.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.084+0000 7f47931f6700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f478c141500 0x7f478c0c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:37.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.084+0000 7f47931f6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f478c06c0a0 0x7f478c0c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:37.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.084+0000 7f47931f6700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f478c15c2a0 con 0x7f478c06bcd0 2023-10-09T06:35:37.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.084+0000 7f47931f6700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f478c0c9880 con 0x7f478c141500 2023-10-09T06:35:37.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.084+0000 7f47931f6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f478c15b690 con 0x7f478c06c0a0 2023-10-09T06:35:37.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.084+0000 7f4790f92700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f478c06bcd0 0x7f478c140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:44934/0 (socket says 172.21.15.163:44934) 2023-10-09T06:35:37.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.084+0000 7f4790f92700 1 -- 172.21.15.163:0/1287025976 learned_addr learned my addr 172.21.15.163:0/1287025976 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:35:37.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.084+0000 7f478bfff700 1 --2- 172.21.15.163:0/1287025976 >> v2:172.21.15.163:3300/0 conn(0x7f478c141500 0x7f478c0c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:37.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.085+0000 7f4791793700 1 --2- 172.21.15.163:0/1287025976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f478c06c0a0 0x7f478c0c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:37.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.085+0000 7f478bfff700 1 -- 172.21.15.163:0/1287025976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f478c06c0a0 msgr2=0x7f478c0c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:37.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.085+0000 7f478bfff700 1 --2- 172.21.15.163:0/1287025976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f478c06c0a0 0x7f478c0c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:37.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.085+0000 7f478bfff700 1 -- 172.21.15.163:0/1287025976 >> v1:172.21.15.163:6789/0 conn(0x7f478c06bcd0 legacy=0x7f478c140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:37.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.085+0000 7f478bfff700 1 -- 172.21.15.163:0/1287025976 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f478c15b970 con 0x7f478c141500 2023-10-09T06:35:37.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.085+0000 7f478bfff700 1 --2- 172.21.15.163:0/1287025976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f478c141500 0x7f478c0c8e00 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f47780098e0 tx=0x7f47780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0346f1d2407ad7f server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:37.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.085+0000 7f478b7fe700 1 -- 172.21.15.163:0/1287025976 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4778009e80 con 0x7f478c141500 2023-10-09T06:35:37.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.085+0000 7f478b7fe700 1 -- 172.21.15.163:0/1287025976 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f477800b860 con 0x7f478c141500 2023-10-09T06:35:37.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.085+0000 7f478b7fe700 1 -- 172.21.15.163:0/1287025976 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477800bbe0 con 0x7f478c141500 2023-10-09T06:35:37.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.086+0000 7f47931f6700 1 -- 172.21.15.163:0/1287025976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f478c141500 msgr2=0x7f478c0c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:37.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.086+0000 7f47931f6700 1 --2- 172.21.15.163:0/1287025976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f478c141500 0x7f478c0c8e00 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f47780098e0 tx=0x7f47780091a0 comp rx=0 tx=0).stop 2023-10-09T06:35:37.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.086+0000 7f47931f6700 1 -- 172.21.15.163:0/1287025976 shutdown_connections 2023-10-09T06:35:37.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.086+0000 7f47931f6700 1 --2- 172.21.15.163:0/1287025976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f478c06c0a0 0x7f478c0c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:37.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.086+0000 7f47931f6700 1 --2- 172.21.15.163:0/1287025976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f478c141500 0x7f478c0c8e00 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:37.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.086+0000 7f47931f6700 1 -- 172.21.15.163:0/1287025976 >> 172.21.15.163:0/1287025976 conn(0x7f478c0ce600 msgr2=0x7f478c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:37.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.086+0000 7f47931f6700 1 -- 172.21.15.163:0/1287025976 shutdown_connections 2023-10-09T06:35:37.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.086+0000 7f47931f6700 1 -- 172.21.15.163:0/1287025976 wait complete. 2023-10-09T06:35:37.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.087+0000 7f47931f6700 1 Processor -- start 2023-10-09T06:35:37.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.087+0000 7f47931f6700 1 -- start start 2023-10-09T06:35:37.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.088+0000 7f47931f6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f478c06bcd0 0x7f478c0c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:37.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.088+0000 7f47931f6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f478c06c0a0 0x7f478c1596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:37.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.088+0000 7f47931f6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f478c141500 0x7f478c159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:37.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.088+0000 7f47931f6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c0b2940 con 0x7f478c06bcd0 2023-10-09T06:35:37.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.088+0000 7f47931f6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c15dc10 con 0x7f478c06c0a0 2023-10-09T06:35:37.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.088+0000 7f47931f6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f478c15d2e0 con 0x7f478c141500 2023-10-09T06:35:37.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.088+0000 7f4790f92700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f478c06bcd0 0x7f478c0c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:37.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.089+0000 7f4790f92700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f478c06bcd0 0x7f478c0c7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:50622/0 (socket says 172.21.15.163:50622) 2023-10-09T06:35:37.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.089+0000 7f4790f92700 1 -- 172.21.15.163:0/1530734689 learned_addr learned my addr 172.21.15.163:0/1530734689 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:37.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.089+0000 7f4791793700 1 --2- 172.21.15.163:0/1530734689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f478c141500 0x7f478c159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:37.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.089+0000 7f478bfff700 1 --2- 172.21.15.163:0/1530734689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f478c06c0a0 0x7f478c1596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:37.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.089+0000 7f4790f92700 1 -- 172.21.15.163:0/1530734689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f478c141500 msgr2=0x7f478c159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:37.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.089+0000 7f4790f92700 1 --2- 172.21.15.163:0/1530734689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f478c141500 0x7f478c159c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:37.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.089+0000 7f4790f92700 1 -- 172.21.15.163:0/1530734689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f478c06c0a0 msgr2=0x7f478c1596f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:37.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.089+0000 7f4790f92700 1 --2- 172.21.15.163:0/1530734689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f478c06c0a0 0x7f478c1596f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:37.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.089+0000 7f4790f92700 1 -- 172.21.15.163:0/1530734689 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4778009420 con 0x7f478c06bcd0 2023-10-09T06:35:37.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.089+0000 7f4790f92700 1 --2- 172.21.15.163:0/1530734689 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f478c06bcd0 0x7f478c0c7e90 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f478000c7c0 tx=0x7f4780007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:37.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.089+0000 7f4788ff9700 1 -- 172.21.15.163:0/1530734689 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4780005b30 con 0x7f478c06bcd0 2023-10-09T06:35:37.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.089+0000 7f4788ff9700 1 -- 172.21.15.163:0/1530734689 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4780005c90 con 0x7f478c06bcd0 2023-10-09T06:35:37.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.089+0000 7f4788ff9700 1 -- 172.21.15.163:0/1530734689 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4780004a80 con 0x7f478c06bcd0 2023-10-09T06:35:37.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.089+0000 7f47931f6700 1 -- 172.21.15.163:0/1530734689 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f478c15d5c0 con 0x7f478c06bcd0 2023-10-09T06:35:37.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.089+0000 7f47931f6700 1 -- 172.21.15.163:0/1530734689 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f478c0c83d0 con 0x7f478c06bcd0 2023-10-09T06:35:37.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.090+0000 7f47931f6700 1 -- 172.21.15.163:0/1530734689 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f478c15d0a0 con 0x7f478c06bcd0 2023-10-09T06:35:37.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.091+0000 7f4788ff9700 1 -- 172.21.15.163:0/1530734689 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f47800070d0 con 0x7f478c06bcd0 2023-10-09T06:35:37.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.091+0000 7f4788ff9700 1 --2- 172.21.15.163:0/1530734689 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4770067170 0x7f4770069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:37.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.092+0000 7f4788ff9700 1 -- 172.21.15.163:0/1530734689 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f4780083ba0 con 0x7f478c06bcd0 2023-10-09T06:35:37.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.092+0000 7f478bfff700 1 --2- 172.21.15.163:0/1530734689 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4770067170 0x7f4770069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:37.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.092+0000 7f478bfff700 1 --2- 172.21.15.163:0/1530734689 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4770067170 0x7f4770069620 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f47780091e0 tx=0x7f4778017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:37.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.099+0000 7f4788ff9700 1 -- 172.21.15.163:0/1530734689 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f478004b5a0 con 0x7f478c06bcd0 2023-10-09T06:35:37.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.347+0000 7f47931f6700 1 -- 172.21.15.163:0/1530734689 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f478c04c3c0 con 0x7f478c06bcd0 2023-10-09T06:35:37.347 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.347+0000 7f4788ff9700 1 -- 172.21.15.163:0/1530734689 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4608 (secure 0 0 0) 0x7f478c15d0a0 con 0x7f478c06bcd0 2023-10-09T06:35:37.347 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:35:37.347 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5407,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3673547262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3673547262},{"type":"v1","addr":"172.21.15.181:6837","nonce":3673547262}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5410,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1705122825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1705122825},{"type":"v1","addr":"172.21.15.181:6836","nonce":1705122825}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/596332711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":596332711},{"type":"v1","addr":"172.21.15.163:6834","nonce":596332711}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:32:30.882925+0000","modified":"2023-10-09T06:32:34.526904+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/3415100865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3415100865},{"type":"v1","addr":"172.21.15.163:6835","nonce":3415100865}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-09T06:35:37.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.350+0000 7f47931f6700 1 -- 172.21.15.163:0/1530734689 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4770067170 msgr2=0x7f4770069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:37.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.350+0000 7f47931f6700 1 --2- 172.21.15.163:0/1530734689 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4770067170 0x7f4770069620 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f47780091e0 tx=0x7f4778017000 comp rx=0 tx=0).stop 2023-10-09T06:35:37.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.350+0000 7f47931f6700 1 -- 172.21.15.163:0/1530734689 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f478c06bcd0 msgr2=0x7f478c0c7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:37.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.350+0000 7f47931f6700 1 --2- 172.21.15.163:0/1530734689 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f478c06bcd0 0x7f478c0c7e90 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f478000c7c0 tx=0x7f4780007f70 comp rx=0 tx=0).stop 2023-10-09T06:35:37.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.350+0000 7f47931f6700 1 -- 172.21.15.163:0/1530734689 shutdown_connections 2023-10-09T06:35:37.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.350+0000 7f47931f6700 1 --2- 172.21.15.163:0/1530734689 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4770067170 0x7f4770069620 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:37.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.350+0000 7f47931f6700 1 --2- 172.21.15.163:0/1530734689 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f478c06bcd0 0x7f478c0c7e90 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:37.351 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.350+0000 7f47931f6700 1 --2- 172.21.15.163:0/1530734689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f478c141500 0x7f478c159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:37.351 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.350+0000 7f47931f6700 1 --2- 172.21.15.163:0/1530734689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f478c06c0a0 0x7f478c1596f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:37.351 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.350+0000 7f47931f6700 1 -- 172.21.15.163:0/1530734689 >> 172.21.15.163:0/1530734689 conn(0x7f478c0ce600 msgr2=0x7f478c0ccc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:37.351 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.350+0000 7f47931f6700 1 -- 172.21.15.163:0/1530734689 shutdown_connections 2023-10-09T06:35:37.351 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.350+0000 7f47931f6700 1 -- 172.21.15.163:0/1530734689 wait complete. 2023-10-09T06:35:37.351 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 41 2023-10-09T06:35:37.362 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:35:37.486 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.486+0000 7f35c23a7700 1 Processor -- start 2023-10-09T06:35:37.486 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.486+0000 7f35c23a7700 1 -- start start 2023-10-09T06:35:37.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.487+0000 7f35c23a7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35bc0cc3c0 0x7f35bc06b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:37.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.487+0000 7f35c23a7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f35bc06bf20 0x7f35bc0b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:37.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.487+0000 7f35c23a7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35bc06c380 con 0x7f35bc1574d0 2023-10-09T06:35:37.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.487+0000 7f35c23a7700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35bc0cd980 con 0x7f35bc06bf20 2023-10-09T06:35:37.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.487+0000 7f35c23a7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc0b16a0 con 0x7f35bc0cc3c0 2023-10-09T06:35:37.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.487+0000 7f35c1ba6700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f35bc06bf20 0x7f35bc0b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:37.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.487+0000 7f35c0ba4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35bc0cc3c0 0x7f35bc06b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:37.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.487+0000 7f35c13a5700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f35bc1574d0 0x7f35bc0cd270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:51336/0 (socket says 172.21.15.163:51336) 2023-10-09T06:35:37.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.487+0000 7f35c1ba6700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f35bc06bf20 0x7f35bc0b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:50632/0 (socket says 172.21.15.163:50632) 2023-10-09T06:35:37.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.487+0000 7f35c13a5700 1 -- 172.21.15.163:0/780386384 learned_addr learned my addr 172.21.15.163:0/780386384 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:35:37.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.487+0000 7f35b3fff700 1 -- 172.21.15.163:0/780386384 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 65885162 0 0) 0x7f35b8001160 con 0x7f35bc1574d0 2023-10-09T06:35:37.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.487+0000 7f35c1ba6700 1 -- 172.21.15.163:0/780386384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35bc0cc3c0 msgr2=0x7f35bc06b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:37.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.487+0000 7f35c1ba6700 1 --2- 172.21.15.163:0/780386384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35bc0cc3c0 0x7f35bc06b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:37.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.487+0000 7f35c1ba6700 1 -- 172.21.15.163:0/780386384 >> v1:172.21.15.181:6789/0 conn(0x7f35bc1574d0 legacy=0x7f35bc0cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:37.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.487+0000 7f35c1ba6700 1 -- 172.21.15.163:0/780386384 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35bc0b1c60 con 0x7f35bc06bf20 2023-10-09T06:35:37.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.488+0000 7f35c1ba6700 1 --2- 172.21.15.163:0/780386384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35bc06bf20 0x7f35bc0b20f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f35b4009e60 tx=0x7f35b4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1aeea90fe9b5813 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:37.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.488+0000 7f35b3fff700 1 -- 172.21.15.163:0/780386384 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b400e070 con 0x7f35bc06bf20 2023-10-09T06:35:37.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.488+0000 7f35b3fff700 1 -- 172.21.15.163:0/780386384 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b400bb60 con 0x7f35bc06bf20 2023-10-09T06:35:37.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.488+0000 7f35b3fff700 1 -- 172.21.15.163:0/780386384 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b4012430 con 0x7f35bc06bf20 2023-10-09T06:35:37.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.488+0000 7f35c23a7700 1 -- 172.21.15.163:0/780386384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35bc06bf20 msgr2=0x7f35bc0b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:37.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.488+0000 7f35c23a7700 1 --2- 172.21.15.163:0/780386384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35bc06bf20 0x7f35bc0b20f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f35b4009e60 tx=0x7f35b4009780 comp rx=0 tx=0).stop 2023-10-09T06:35:37.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.488+0000 7f35c23a7700 1 -- 172.21.15.163:0/780386384 shutdown_connections 2023-10-09T06:35:37.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.488+0000 7f35c23a7700 1 --2- 172.21.15.163:0/780386384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35bc06bf20 0x7f35bc0b20f0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:37.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.488+0000 7f35c23a7700 1 --2- 172.21.15.163:0/780386384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35bc0cc3c0 0x7f35bc06b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:37.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.488+0000 7f35c23a7700 1 -- 172.21.15.163:0/780386384 >> 172.21.15.163:0/780386384 conn(0x7f35bc0ce430 msgr2=0x7f35bc0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:37.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.488+0000 7f35c23a7700 1 -- 172.21.15.163:0/780386384 shutdown_connections 2023-10-09T06:35:37.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.489+0000 7f35c23a7700 1 -- 172.21.15.163:0/780386384 wait complete. 2023-10-09T06:35:37.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.490+0000 7f35c23a7700 1 Processor -- start 2023-10-09T06:35:37.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.490+0000 7f35c23a7700 1 -- start start 2023-10-09T06:35:37.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.490+0000 7f35c23a7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35bc06bf20 0x7f35bc0c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:37.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.490+0000 7f35c23a7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35bc0cc3c0 0x7f35bc148dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:37.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.491+0000 7f35c23a7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35bc1574d0 0x7f35bc153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:37.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.491+0000 7f35c23a7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc169830 con 0x7f35bc1574d0 2023-10-09T06:35:37.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.491+0000 7f35c23a7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc0c9f30 con 0x7f35bc0cc3c0 2023-10-09T06:35:37.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.491+0000 7f35c23a7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc0b05f0 con 0x7f35bc06bf20 2023-10-09T06:35:37.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.491+0000 7f35c1ba6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35bc1574d0 0x7f35bc153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:37.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.491+0000 7f35c1ba6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35bc1574d0 0x7f35bc153b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:50634/0 (socket says 172.21.15.163:50634) 2023-10-09T06:35:37.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.491+0000 7f35c1ba6700 1 -- 172.21.15.163:0/3308117846 learned_addr learned my addr 172.21.15.163:0/3308117846 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:37.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.491+0000 7f35c13a5700 1 --2- 172.21.15.163:0/3308117846 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35bc06bf20 0x7f35bc0c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:37.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.491+0000 7f35c0ba4700 1 --2- 172.21.15.163:0/3308117846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35bc0cc3c0 0x7f35bc148dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:37.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.491+0000 7f35c1ba6700 1 -- 172.21.15.163:0/3308117846 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35bc06bf20 msgr2=0x7f35bc0c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:37.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.491+0000 7f35c1ba6700 1 --2- 172.21.15.163:0/3308117846 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35bc06bf20 0x7f35bc0c99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:37.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.491+0000 7f35c1ba6700 1 -- 172.21.15.163:0/3308117846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35bc0cc3c0 msgr2=0x7f35bc148dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:37.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.491+0000 7f35c1ba6700 1 --2- 172.21.15.163:0/3308117846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35bc0cc3c0 0x7f35bc148dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:37.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.491+0000 7f35c1ba6700 1 -- 172.21.15.163:0/3308117846 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b4009a20 con 0x7f35bc1574d0 2023-10-09T06:35:37.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.491+0000 7f35c1ba6700 1 --2- 172.21.15.163:0/3308117846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35bc1574d0 0x7f35bc153b60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f35b400b6c0 tx=0x7f35b4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:37.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.491+0000 7f35b17fa700 1 -- 172.21.15.163:0/3308117846 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b4027470 con 0x7f35bc1574d0 2023-10-09T06:35:37.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.491+0000 7f35b17fa700 1 -- 172.21.15.163:0/3308117846 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b40275d0 con 0x7f35bc1574d0 2023-10-09T06:35:37.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.491+0000 7f35c23a7700 1 -- 172.21.15.163:0/3308117846 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35bc0b08d0 con 0x7f35bc1574d0 2023-10-09T06:35:37.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.491+0000 7f35b17fa700 1 -- 172.21.15.163:0/3308117846 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b4027990 con 0x7f35bc1574d0 2023-10-09T06:35:37.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.491+0000 7f35c23a7700 1 -- 172.21.15.163:0/3308117846 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35bc0c75e0 con 0x7f35bc1574d0 2023-10-09T06:35:37.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.492+0000 7f35c23a7700 1 -- 172.21.15.163:0/3308117846 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35bc04c3c0 con 0x7f35bc1574d0 2023-10-09T06:35:37.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.493+0000 7f35b17fa700 1 -- 172.21.15.163:0/3308117846 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f35b4024070 con 0x7f35bc1574d0 2023-10-09T06:35:37.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.493+0000 7f35b17fa700 1 --2- 172.21.15.163:0/3308117846 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f35a0067170 0x7f35a0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:37.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.494+0000 7f35b17fa700 1 -- 172.21.15.163:0/3308117846 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f35b4086f40 con 0x7f35bc1574d0 2023-10-09T06:35:37.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.494+0000 7f35c13a5700 1 --2- 172.21.15.163:0/3308117846 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f35a0067170 0x7f35a0069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:37.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.494+0000 7f35c13a5700 1 --2- 172.21.15.163:0/3308117846 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f35a0067170 0x7f35a0069620 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f35b8007fd0 tx=0x7f35b8007f40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:37.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.500+0000 7f35b17fa700 1 -- 172.21.15.163:0/3308117846 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f35b4053ee0 con 0x7f35bc1574d0 2023-10-09T06:35:37.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.717+0000 7f35c23a7700 1 -- 172.21.15.163:0/3308117846 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35bc0adad0 con 0x7f35bc1574d0 2023-10-09T06:35:37.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.718+0000 7f35b17fa700 1 -- 172.21.15.163:0/3308117846 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+15498 (secure 0 0 0) 0x7f35b4057500 con 0x7f35bc1574d0 2023-10-09T06:35:37.718 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:35:37.718 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":65,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:33:25.634897+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:32:28.847438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-09T06:32:29.837643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.1","mappings":[{"from":6,"to":0}]},{"pgid":"11.3c","mappings":[{"from":6,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:35:37.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.720+0000 7f35c23a7700 1 -- 172.21.15.163:0/3308117846 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f35a0067170 msgr2=0x7f35a0069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:37.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.720+0000 7f35c23a7700 1 --2- 172.21.15.163:0/3308117846 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f35a0067170 0x7f35a0069620 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f35b8007fd0 tx=0x7f35b8007f40 comp rx=0 tx=0).stop 2023-10-09T06:35:37.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.720+0000 7f35c23a7700 1 -- 172.21.15.163:0/3308117846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35bc1574d0 msgr2=0x7f35bc153b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:37.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.720+0000 7f35c23a7700 1 --2- 172.21.15.163:0/3308117846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35bc1574d0 0x7f35bc153b60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f35b400b6c0 tx=0x7f35b4019000 comp rx=0 tx=0).stop 2023-10-09T06:35:37.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.720+0000 7f35c23a7700 1 -- 172.21.15.163:0/3308117846 shutdown_connections 2023-10-09T06:35:37.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.720+0000 7f35c23a7700 1 --2- 172.21.15.163:0/3308117846 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f35a0067170 0x7f35a0069620 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:37.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.720+0000 7f35c23a7700 1 --2- 172.21.15.163:0/3308117846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35bc1574d0 0x7f35bc153b60 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:37.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.720+0000 7f35c23a7700 1 --2- 172.21.15.163:0/3308117846 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35bc06bf20 0x7f35bc0c99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:37.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.720+0000 7f35c23a7700 1 --2- 172.21.15.163:0/3308117846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35bc0cc3c0 0x7f35bc148dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:37.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.720+0000 7f35c23a7700 1 -- 172.21.15.163:0/3308117846 >> 172.21.15.163:0/3308117846 conn(0x7f35bc0ce430 msgr2=0x7f35bc0b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:37.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.720+0000 7f35c23a7700 1 -- 172.21.15.163:0/3308117846 shutdown_connections 2023-10-09T06:35:37.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.721+0000 7f35c23a7700 1 -- 172.21.15.163:0/3308117846 wait complete. 2023-10-09T06:35:37.733 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-09T06:32:28.847438+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-10-09T06:32:29.837643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-09T06:35:37.733 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:35:37.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.870+0000 7f9bc2818700 1 Processor -- start 2023-10-09T06:35:37.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.870+0000 7f9bc2818700 1 -- start start 2023-10-09T06:35:37.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.871+0000 7f9bc2818700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9bbc159890 0x7f9bbc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:37.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.871+0000 7f9bc2818700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9bbc159c60 0x7f9bbc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:37.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.871+0000 7f9bc2818700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bbc15a160 con 0x7f9bbc15db10 2023-10-09T06:35:37.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.871+0000 7f9bc2818700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bbc15df50 con 0x7f9bbc159c60 2023-10-09T06:35:37.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.871+0000 7f9bc2818700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bbc0d13a0 con 0x7f9bbc159890 2023-10-09T06:35:37.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.871+0000 7f9bbb7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9bbc159c60 0x7f9bbc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:37.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.871+0000 7f9bbb7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9bbc159c60 0x7f9bbc15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:50648/0 (socket says 172.21.15.163:50648) 2023-10-09T06:35:37.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.871+0000 7f9bbb7fe700 1 -- 172.21.15.163:0/132852953 learned_addr learned my addr 172.21.15.163:0/132852953 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:37.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.871+0000 7f9bbbfff700 1 --2- 172.21.15.163:0/132852953 >> v2:172.21.15.181:3300/0 conn(0x7f9bbc159890 0x7f9bbc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:37.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.872+0000 7f9bbb7fe700 1 -- 172.21.15.163:0/132852953 >> v2:172.21.15.181:3300/0 conn(0x7f9bbc159890 msgr2=0x7f9bbc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:37.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.872+0000 7f9bbb7fe700 1 --2- 172.21.15.163:0/132852953 >> v2:172.21.15.181:3300/0 conn(0x7f9bbc159890 0x7f9bbc15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:37.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.872+0000 7f9bbb7fe700 1 -- 172.21.15.163:0/132852953 >> v1:172.21.15.181:6789/0 conn(0x7f9bbc15db10 legacy=0x7f9bbc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:37.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.872+0000 7f9bbb7fe700 1 -- 172.21.15.163:0/132852953 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bbc15e140 con 0x7f9bbc159c60 2023-10-09T06:35:37.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.872+0000 7f9bbb7fe700 1 --2- 172.21.15.163:0/132852953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bbc159c60 0x7f9bbc15d5d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc0d1090 tx=0x7f9bb0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6bc4c4c8d7bebd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:37.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.872+0000 7f9bbaffd700 1 -- 172.21.15.163:0/132852953 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0009e70 con 0x7f9bbc159c60 2023-10-09T06:35:37.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.872+0000 7f9bbaffd700 1 -- 172.21.15.163:0/132852953 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb000b860 con 0x7f9bbc159c60 2023-10-09T06:35:37.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.872+0000 7f9bbaffd700 1 -- 172.21.15.163:0/132852953 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb000bc00 con 0x7f9bbc159c60 2023-10-09T06:35:37.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.873+0000 7f9bc2818700 1 -- 172.21.15.163:0/132852953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bbc159c60 msgr2=0x7f9bbc15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:37.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.873+0000 7f9bc2818700 1 --2- 172.21.15.163:0/132852953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bbc159c60 0x7f9bbc15d5d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc0d1090 tx=0x7f9bb0009250 comp rx=0 tx=0).stop 2023-10-09T06:35:37.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.873+0000 7f9bc2818700 1 -- 172.21.15.163:0/132852953 shutdown_connections 2023-10-09T06:35:37.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.873+0000 7f9bc2818700 1 --2- 172.21.15.163:0/132852953 >> v2:172.21.15.181:3300/0 conn(0x7f9bbc159890 0x7f9bbc15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:37.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.873+0000 7f9bc2818700 1 --2- 172.21.15.163:0/132852953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bbc159c60 0x7f9bbc15d5d0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:37.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.873+0000 7f9bc2818700 1 -- 172.21.15.163:0/132852953 >> 172.21.15.163:0/132852953 conn(0x7f9bbc068e20 msgr2=0x7f9bbc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:37.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.873+0000 7f9bc2818700 1 -- 172.21.15.163:0/132852953 shutdown_connections 2023-10-09T06:35:37.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.873+0000 7f9bc2818700 1 -- 172.21.15.163:0/132852953 wait complete. 2023-10-09T06:35:37.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.874+0000 7f9bc2818700 1 Processor -- start 2023-10-09T06:35:37.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.874+0000 7f9bc2818700 1 -- start start 2023-10-09T06:35:37.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.875+0000 7f9bc2818700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bbc159890 0x7f9bbc0b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:37.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.875+0000 7f9bc2818700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bbc159c60 0x7f9bbc0b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:37.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.875+0000 7f9bc2818700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bbc15db10 0x7f9bbc0b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:37.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.875+0000 7f9bc2818700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc153e60 con 0x7f9bbc15db10 2023-10-09T06:35:37.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.875+0000 7f9bc2818700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc169f30 con 0x7f9bbc159890 2023-10-09T06:35:37.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.875+0000 7f9bc2818700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc16aa50 con 0x7f9bbc159c60 2023-10-09T06:35:37.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.875+0000 7f9bc0db5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bbc15db10 0x7f9bbc0b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:37.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.875+0000 7f9bc0db5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bbc15db10 0x7f9bbc0b5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:50664/0 (socket says 172.21.15.163:50664) 2023-10-09T06:35:37.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.875+0000 7f9bbbfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bbc159890 0x7f9bbc0b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:37.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.875+0000 7f9bc0db5700 1 -- 172.21.15.163:0/3437093300 learned_addr learned my addr 172.21.15.163:0/3437093300 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:37.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.875+0000 7f9bbb7fe700 1 --2- 172.21.15.163:0/3437093300 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bbc159c60 0x7f9bbc0b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:37.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.875+0000 7f9bc0db5700 1 -- 172.21.15.163:0/3437093300 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bbc159c60 msgr2=0x7f9bbc0b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:37.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.875+0000 7f9bc0db5700 1 --2- 172.21.15.163:0/3437093300 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bbc159c60 0x7f9bbc0b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:37.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.875+0000 7f9bc0db5700 1 -- 172.21.15.163:0/3437093300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bbc159890 msgr2=0x7f9bbc0b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:37.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.875+0000 7f9bc0db5700 1 --2- 172.21.15.163:0/3437093300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bbc159890 0x7f9bbc0b4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:37.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.875+0000 7f9bc0db5700 1 -- 172.21.15.163:0/3437093300 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb00094f0 con 0x7f9bbc15db10 2023-10-09T06:35:37.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.875+0000 7f9bbbfff700 1 --2- 172.21.15.163:0/3437093300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bbc159890 0x7f9bbc0b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:35:37.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.876+0000 7f9bc0db5700 1 --2- 172.21.15.163:0/3437093300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bbc15db10 0x7f9bbc0b5ef0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9bb400c840 tx=0x7f9bb4001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:37.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.876+0000 7f9ba3fff700 1 -- 172.21.15.163:0/3437093300 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4012630 con 0x7f9bbc15db10 2023-10-09T06:35:37.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.876+0000 7f9bc2818700 1 -- 172.21.15.163:0/3437093300 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bbc16a120 con 0x7f9bbc15db10 2023-10-09T06:35:37.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.876+0000 7f9bc2818700 1 -- 172.21.15.163:0/3437093300 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bbc16a6e0 con 0x7f9bbc15db10 2023-10-09T06:35:37.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.876+0000 7f9ba3fff700 1 -- 172.21.15.163:0/3437093300 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb40129a0 con 0x7f9bbc15db10 2023-10-09T06:35:37.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.876+0000 7f9ba3fff700 1 -- 172.21.15.163:0/3437093300 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4012b10 con 0x7f9bbc15db10 2023-10-09T06:35:37.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.877+0000 7f9bc2818700 1 -- 172.21.15.163:0/3437093300 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bbc16ac00 con 0x7f9bbc15db10 2023-10-09T06:35:37.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.878+0000 7f9ba3fff700 1 -- 172.21.15.163:0/3437093300 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f9bb4024420 con 0x7f9bbc15db10 2023-10-09T06:35:37.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.878+0000 7f9ba3fff700 1 --2- 172.21.15.163:0/3437093300 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9ba8067170 0x7f9ba8069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:37.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.878+0000 7f9ba3fff700 1 -- 172.21.15.163:0/3437093300 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f9bb40850d0 con 0x7f9bbc15db10 2023-10-09T06:35:37.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.879+0000 7f9bbbfff700 1 --2- 172.21.15.163:0/3437093300 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9ba8067170 0x7f9ba8069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:37.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.879+0000 7f9bbbfff700 1 --2- 172.21.15.163:0/3437093300 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9ba8067170 0x7f9ba8069620 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9bac011480 tx=0x7f9bac0091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:37.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:37.882+0000 7f9ba3fff700 1 -- 172.21.15.163:0/3437093300 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9bb4052070 con 0x7f9bbc15db10 2023-10-09T06:35:38.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.092+0000 7f9bc2818700 1 -- 172.21.15.163:0/3437093300 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9bbc04c3c0 con 0x7f9bbc15db10 2023-10-09T06:35:38.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.092+0000 7f9ba3fff700 1 -- 172.21.15.163:0/3437093300 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9bb4055690 con 0x7f9bbc15db10 2023-10-09T06:35:38.092 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:35:38.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.094+0000 7f9bc2818700 1 -- 172.21.15.163:0/3437093300 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9ba8067170 msgr2=0x7f9ba8069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:38.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.094+0000 7f9bc2818700 1 --2- 172.21.15.163:0/3437093300 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9ba8067170 0x7f9ba8069620 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9bac011480 tx=0x7f9bac0091a0 comp rx=0 tx=0).stop 2023-10-09T06:35:38.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.094+0000 7f9bc2818700 1 -- 172.21.15.163:0/3437093300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bbc15db10 msgr2=0x7f9bbc0b5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:38.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.094+0000 7f9bc2818700 1 --2- 172.21.15.163:0/3437093300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bbc15db10 0x7f9bbc0b5ef0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9bb400c840 tx=0x7f9bb4001670 comp rx=0 tx=0).stop 2023-10-09T06:35:38.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.094+0000 7f9bc2818700 1 -- 172.21.15.163:0/3437093300 shutdown_connections 2023-10-09T06:35:38.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.094+0000 7f9bc2818700 1 --2- 172.21.15.163:0/3437093300 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9ba8067170 0x7f9ba8069620 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:38.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.094+0000 7f9bc2818700 1 --2- 172.21.15.163:0/3437093300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bbc15db10 0x7f9bbc0b5ef0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:38.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.094+0000 7f9bc2818700 1 --2- 172.21.15.163:0/3437093300 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bbc159890 0x7f9bbc0b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:38.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.094+0000 7f9bc2818700 1 --2- 172.21.15.163:0/3437093300 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bbc159c60 0x7f9bbc0b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:38.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.095+0000 7f9bc2818700 1 -- 172.21.15.163:0/3437093300 >> 172.21.15.163:0/3437093300 conn(0x7f9bbc068e20 msgr2=0x7f9bbc0d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:38.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.095+0000 7f9bc2818700 1 -- 172.21.15.163:0/3437093300 shutdown_connections 2023-10-09T06:35:38.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.095+0000 7f9bc2818700 1 -- 172.21.15.163:0/3437093300 wait complete. 2023-10-09T06:35:38.109 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-09T06:35:38.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.235+0000 7f8a7c6b2700 1 Processor -- start 2023-10-09T06:35:38.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.235+0000 7f8a7c6b2700 1 -- start start 2023-10-09T06:35:38.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.236+0000 7f8a7c6b2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a74159890 0x7f8a7415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:38.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.236+0000 7f8a7c6b2700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a7415a160 con 0x7f8a74069560 2023-10-09T06:35:38.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.236+0000 7f8a7c6b2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a740d13a0 con 0x7f8a74159c60 2023-10-09T06:35:38.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.236+0000 7f8a7c6b2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7415e140 con 0x7f8a74159890 2023-10-09T06:35:38.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.236+0000 7f8a7ac4f700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f8a74069560 0x7f8a740ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:44942/0 (socket says 172.21.15.163:44942) 2023-10-09T06:35:38.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.236+0000 7f8a7ac4f700 1 -- 172.21.15.163:0/2612076962 learned_addr learned my addr 172.21.15.163:0/2612076962 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:35:38.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.236+0000 7f8a7a44e700 1 --2- 172.21.15.163:0/2612076962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a74159890 0x7f8a7415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:38.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.236+0000 7f8a7944c700 1 -- 172.21.15.163:0/2612076962 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2293901275 0 0) 0x7f8a7415a160 con 0x7f8a74069560 2023-10-09T06:35:38.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.236+0000 7f8a7944c700 1 -- 172.21.15.163:0/2612076962 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a60003570 con 0x7f8a74069560 2023-10-09T06:35:38.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.237+0000 7f8a7944c700 1 -- 172.21.15.163:0/2612076962 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3140289728 0 0) 0x7f8a740d13a0 con 0x7f8a74159c60 2023-10-09T06:35:38.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.237+0000 7f8a7944c700 1 -- 172.21.15.163:0/2612076962 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a7415a160 con 0x7f8a74159c60 2023-10-09T06:35:38.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.237+0000 7f8a7944c700 1 -- 172.21.15.163:0/2612076962 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1474604614 0 0) 0x7f8a60003570 con 0x7f8a74069560 2023-10-09T06:35:38.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.237+0000 7f8a7944c700 1 -- 172.21.15.163:0/2612076962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a74159890 msgr2=0x7f8a7415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:38.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.237+0000 7f8a7944c700 1 --2- 172.21.15.163:0/2612076962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a74159890 0x7f8a7415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:38.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.237+0000 7f8a7944c700 1 -- 172.21.15.163:0/2612076962 >> v1:172.21.15.181:6789/0 conn(0x7f8a74159c60 legacy=0x7f8a7415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:38.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.237+0000 7f8a7944c700 1 -- 172.21.15.163:0/2612076962 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7415a3d0 con 0x7f8a74069560 2023-10-09T06:35:38.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.237+0000 7f8a7944c700 1 -- 172.21.15.163:0/2612076962 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f8a6c0017f0 con 0x7f8a74069560 2023-10-09T06:35:38.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.237+0000 7f8a7944c700 1 -- 172.21.15.163:0/2612076962 >> v1:172.21.15.163:6789/0 conn(0x7f8a74069560 legacy=0x7f8a740ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:38.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.237+0000 7f8a7944c700 1 --2- 172.21.15.163:0/2612076962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a74159890 0x7f8a60003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:38.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.237+0000 7f8a7944c700 1 --2- 172.21.15.163:0/2612076962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a60004180 0x7f8a60004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:38.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.237+0000 7f8a7944c700 1 --2- 172.21.15.163:0/2612076962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a60004d50 0x7f8a600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:38.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.237+0000 7f8a7944c700 1 -- 172.21.15.163:0/2612076962 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7415e140 con 0x7f8a60004180 2023-10-09T06:35:38.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.237+0000 7f8a7944c700 1 -- 172.21.15.163:0/2612076962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6000bba0 con 0x7f8a60004d50 2023-10-09T06:35:38.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.237+0000 7f8a7944c700 1 -- 172.21.15.163:0/2612076962 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6000bd60 con 0x7f8a74159890 2023-10-09T06:35:38.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.237+0000 7f8a79c4d700 1 --2- 172.21.15.163:0/2612076962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a60004180 0x7f8a60004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:38.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.237+0000 7f8a7a44e700 1 --2- 172.21.15.163:0/2612076962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a74159890 0x7f8a60003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:38.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.237+0000 7f8a7ac4f700 1 --2- 172.21.15.163:0/2612076962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a60004d50 0x7f8a600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:38.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.237+0000 7f8a79c4d700 1 -- 172.21.15.163:0/2612076962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a74159890 msgr2=0x7f8a60003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:38.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.237+0000 7f8a79c4d700 1 --2- 172.21.15.163:0/2612076962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a74159890 0x7f8a60003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:38.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.237+0000 7f8a79c4d700 1 -- 172.21.15.163:0/2612076962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a60004d50 msgr2=0x7f8a600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:38.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.237+0000 7f8a79c4d700 1 --2- 172.21.15.163:0/2612076962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a60004d50 0x7f8a600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:38.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.237+0000 7f8a79c4d700 1 -- 172.21.15.163:0/2612076962 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a60003570 con 0x7f8a60004180 2023-10-09T06:35:38.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.237+0000 7f8a7a44e700 1 --2- 172.21.15.163:0/2612076962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a74159890 0x7f8a60003c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:35:38.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.237+0000 7f8a79c4d700 1 --2- 172.21.15.163:0/2612076962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a60004180 0x7f8a60004810 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f8a640019e0 tx=0x7f8a64007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf252841e497d66 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:38.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.238+0000 7f8a7944c700 1 -- 172.21.15.163:0/2612076962 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6400c5d0 con 0x7f8a60004180 2023-10-09T06:35:38.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.238+0000 7f8a7944c700 1 -- 172.21.15.163:0/2612076962 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6400c730 con 0x7f8a60004180 2023-10-09T06:35:38.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.238+0000 7f8a7944c700 1 -- 172.21.15.163:0/2612076962 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6400cad0 con 0x7f8a60004180 2023-10-09T06:35:38.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.238+0000 7f8a7c6b2700 1 -- 172.21.15.163:0/2612076962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a60004180 msgr2=0x7f8a60004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:38.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.238+0000 7f8a7c6b2700 1 --2- 172.21.15.163:0/2612076962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a60004180 0x7f8a60004810 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f8a640019e0 tx=0x7f8a64007d30 comp rx=0 tx=0).stop 2023-10-09T06:35:38.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.238+0000 7f8a7ac4f700 1 -- 172.21.15.163:0/2612076962 reap_dead start 2023-10-09T06:35:38.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.238+0000 7f8a7c6b2700 1 -- 172.21.15.163:0/2612076962 shutdown_connections 2023-10-09T06:35:38.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.238+0000 7f8a7c6b2700 1 -- 172.21.15.163:0/2612076962 >> 172.21.15.163:0/2612076962 conn(0x7f8a74068e20 msgr2=0x7f8a740d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:38.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.238+0000 7f8a7c6b2700 1 -- 172.21.15.163:0/2612076962 shutdown_connections 2023-10-09T06:35:38.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.238+0000 7f8a7c6b2700 1 -- 172.21.15.163:0/2612076962 wait complete. 2023-10-09T06:35:38.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.240+0000 7f8a7c6b2700 1 Processor -- start 2023-10-09T06:35:38.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.240+0000 7f8a7c6b2700 1 -- start start 2023-10-09T06:35:38.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.241+0000 7f8a7c6b2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a60004180 0x7f8a740b5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:38.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.241+0000 7f8a7c6b2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a740cf6d0 0x7f8a740cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:38.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.241+0000 7f8a7c6b2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a7416d0d0 0x7f8a7416d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:38.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.241+0000 7f8a7c6b2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a740b0fd0 con 0x7f8a740cf6d0 2023-10-09T06:35:38.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.241+0000 7f8a7c6b2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7416da80 con 0x7f8a60004180 2023-10-09T06:35:38.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.241+0000 7f8a7c6b2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a740c89b0 con 0x7f8a7416d0d0 2023-10-09T06:35:38.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.241+0000 7f8a79c4d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a740cf6d0 0x7f8a740cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:38.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.241+0000 7f8a7a44e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a60004180 0x7f8a740b5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:38.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.241+0000 7f8a79c4d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a740cf6d0 0x7f8a740cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:50684/0 (socket says 172.21.15.163:50684) 2023-10-09T06:35:38.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.241+0000 7f8a79c4d700 1 -- 172.21.15.163:0/2488020911 learned_addr learned my addr 172.21.15.163:0/2488020911 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:38.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.241+0000 7f8a7ac4f700 1 --2- 172.21.15.163:0/2488020911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a7416d0d0 0x7f8a7416d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:38.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.241+0000 7f8a79c4d700 1 -- 172.21.15.163:0/2488020911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a7416d0d0 msgr2=0x7f8a7416d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:38.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.241+0000 7f8a79c4d700 1 --2- 172.21.15.163:0/2488020911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a7416d0d0 0x7f8a7416d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:38.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.241+0000 7f8a79c4d700 1 -- 172.21.15.163:0/2488020911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a60004180 msgr2=0x7f8a740b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:38.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.241+0000 7f8a79c4d700 1 --2- 172.21.15.163:0/2488020911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a60004180 0x7f8a740b5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:38.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.241+0000 7f8a79c4d700 1 -- 172.21.15.163:0/2488020911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a740b60a0 con 0x7f8a740cf6d0 2023-10-09T06:35:38.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.241+0000 7f8a79c4d700 1 --2- 172.21.15.163:0/2488020911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a740cf6d0 0x7f8a740cfb40 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f8a64012a00 tx=0x7f8a64015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:38.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.241+0000 7f8a6a7fc700 1 -- 172.21.15.163:0/2488020911 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6400c5d0 con 0x7f8a740cf6d0 2023-10-09T06:35:38.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.241+0000 7f8a6a7fc700 1 -- 172.21.15.163:0/2488020911 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a64001360 con 0x7f8a740cf6d0 2023-10-09T06:35:38.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.241+0000 7f8a7c6b2700 1 -- 172.21.15.163:0/2488020911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a7416a6a0 con 0x7f8a740cf6d0 2023-10-09T06:35:38.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.241+0000 7f8a6a7fc700 1 -- 172.21.15.163:0/2488020911 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6400ce80 con 0x7f8a740cf6d0 2023-10-09T06:35:38.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.241+0000 7f8a7c6b2700 1 -- 172.21.15.163:0/2488020911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a7416ac60 con 0x7f8a740cf6d0 2023-10-09T06:35:38.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.243+0000 7f8a7c6b2700 1 -- 172.21.15.163:0/2488020911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a740c73d0 con 0x7f8a740cf6d0 2023-10-09T06:35:38.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.243+0000 7f8a6a7fc700 1 -- 172.21.15.163:0/2488020911 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f8a64019070 con 0x7f8a740cf6d0 2023-10-09T06:35:38.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.244+0000 7f8a6a7fc700 1 --2- 172.21.15.163:0/2488020911 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8a58067170 0x7f8a58069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:38.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.244+0000 7f8a6a7fc700 1 -- 172.21.15.163:0/2488020911 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f8a64085c50 con 0x7f8a740cf6d0 2023-10-09T06:35:38.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.244+0000 7f8a7a44e700 1 --2- 172.21.15.163:0/2488020911 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8a58067170 0x7f8a58069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:38.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.244+0000 7f8a7a44e700 1 --2- 172.21.15.163:0/2488020911 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8a58067170 0x7f8a58069620 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8a70006f00 tx=0x7f8a700091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:38.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.248+0000 7f8a6a7fc700 1 -- 172.21.15.163:0/2488020911 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8a6408b020 con 0x7f8a740cf6d0 2023-10-09T06:35:38.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.475+0000 7f8a7c6b2700 1 -- 172.21.15.163:0/2488020911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8a7404c3c0 con 0x7f8a740cf6d0 2023-10-09T06:35:38.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.475+0000 7f8a6a7fc700 1 -- 172.21.15.163:0/2488020911 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8a640170a0 con 0x7f8a740cf6d0 2023-10-09T06:35:38.475 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 32 2023-10-09T06:35:38.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.478+0000 7f8a7c6b2700 1 -- 172.21.15.163:0/2488020911 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8a58067170 msgr2=0x7f8a58069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:38.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.478+0000 7f8a7c6b2700 1 --2- 172.21.15.163:0/2488020911 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8a58067170 0x7f8a58069620 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8a70006f00 tx=0x7f8a700091a0 comp rx=0 tx=0).stop 2023-10-09T06:35:38.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.478+0000 7f8a7c6b2700 1 -- 172.21.15.163:0/2488020911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a740cf6d0 msgr2=0x7f8a740cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:38.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.478+0000 7f8a7c6b2700 1 --2- 172.21.15.163:0/2488020911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a740cf6d0 0x7f8a740cfb40 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f8a64012a00 tx=0x7f8a64015000 comp rx=0 tx=0).stop 2023-10-09T06:35:38.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.478+0000 7f8a7c6b2700 1 -- 172.21.15.163:0/2488020911 shutdown_connections 2023-10-09T06:35:38.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.478+0000 7f8a7c6b2700 1 --2- 172.21.15.163:0/2488020911 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8a58067170 0x7f8a58069620 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:38.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.478+0000 7f8a7c6b2700 1 --2- 172.21.15.163:0/2488020911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a7416d0d0 0x7f8a7416d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:38.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.478+0000 7f8a7c6b2700 1 --2- 172.21.15.163:0/2488020911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a60004180 0x7f8a740b5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:38.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.478+0000 7f8a7c6b2700 1 --2- 172.21.15.163:0/2488020911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a740cf6d0 0x7f8a740cfb40 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:38.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.478+0000 7f8a7c6b2700 1 -- 172.21.15.163:0/2488020911 >> 172.21.15.163:0/2488020911 conn(0x7f8a74068e20 msgr2=0x7f8a740c6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:38.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.478+0000 7f8a7c6b2700 1 -- 172.21.15.163:0/2488020911 shutdown_connections 2023-10-09T06:35:38.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.478+0000 7f8a7c6b2700 1 -- 172.21.15.163:0/2488020911 wait complete. 2023-10-09T06:35:38.490 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-09T06:35:38.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.639+0000 7f24f1a24700 1 Processor -- start 2023-10-09T06:35:38.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.639+0000 7f24f1a24700 1 -- start start 2023-10-09T06:35:38.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.640+0000 7f24f1a24700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24ec1575a0 0x7f24ec0cb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:38.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.640+0000 7f24f1a24700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f24ec063580 0x7f24ec06e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:38.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.640+0000 7f24f1a24700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24ec06ee10 con 0x7f24ec06e9b0 2023-10-09T06:35:38.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.640+0000 7f24f1a24700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24ec0b1100 con 0x7f24ec063580 2023-10-09T06:35:38.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.640+0000 7f24f1a24700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec0c3ae0 con 0x7f24ec1575a0 2023-10-09T06:35:38.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.640+0000 7f24ebfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f24ec063580 0x7f24ec06e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:38.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.640+0000 7f24ebfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f24ec063580 0x7f24ec06e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:37600/0 (socket says 172.21.15.163:37600) 2023-10-09T06:35:38.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.640+0000 7f24ebfff700 1 -- 172.21.15.163:0/498194512 learned_addr learned my addr 172.21.15.163:0/498194512 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:38.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.640+0000 7f24f0a22700 1 --2- 172.21.15.163:0/498194512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24ec1575a0 0x7f24ec0cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:38.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.641+0000 7f24ebfff700 1 -- 172.21.15.163:0/498194512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24ec1575a0 msgr2=0x7f24ec0cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:38.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.641+0000 7f24ebfff700 1 --2- 172.21.15.163:0/498194512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24ec1575a0 0x7f24ec0cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:38.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.641+0000 7f24ebfff700 1 -- 172.21.15.163:0/498194512 >> v1:172.21.15.181:6789/0 conn(0x7f24ec06e9b0 legacy=0x7f24ec0ac9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:38.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.641+0000 7f24ebfff700 1 -- 172.21.15.163:0/498194512 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ec0cbe60 con 0x7f24ec063580 2023-10-09T06:35:38.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.641+0000 7f24f0a22700 1 --2- 172.21.15.163:0/498194512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24ec1575a0 0x7f24ec0cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:35:38.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.641+0000 7f24ebfff700 1 --2- 172.21.15.163:0/498194512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24ec063580 0x7f24ec06e470 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f24d80099b0 tx=0x7f24d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1edb25448f6b7640 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:38.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.641+0000 7f24eb7fe700 1 -- 172.21.15.163:0/498194512 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d800b860 con 0x7f24ec063580 2023-10-09T06:35:38.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.641+0000 7f24eb7fe700 1 -- 172.21.15.163:0/498194512 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d800b9c0 con 0x7f24ec063580 2023-10-09T06:35:38.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.641+0000 7f24eb7fe700 1 -- 172.21.15.163:0/498194512 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d800bd60 con 0x7f24ec063580 2023-10-09T06:35:38.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.642+0000 7f24f1a24700 1 -- 172.21.15.163:0/498194512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24ec063580 msgr2=0x7f24ec06e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:38.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.642+0000 7f24f1a24700 1 --2- 172.21.15.163:0/498194512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24ec063580 0x7f24ec06e470 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f24d80099b0 tx=0x7f24d8009250 comp rx=0 tx=0).stop 2023-10-09T06:35:38.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.642+0000 7f24f1a24700 1 -- 172.21.15.163:0/498194512 shutdown_connections 2023-10-09T06:35:38.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.642+0000 7f24f1a24700 1 --2- 172.21.15.163:0/498194512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24ec1575a0 0x7f24ec0cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:38.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.642+0000 7f24f1a24700 1 --2- 172.21.15.163:0/498194512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24ec063580 0x7f24ec06e470 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:38.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.642+0000 7f24f1a24700 1 -- 172.21.15.163:0/498194512 >> 172.21.15.163:0/498194512 conn(0x7f24ec0c9f70 msgr2=0x7f24ec06dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:38.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.642+0000 7f24f1a24700 1 -- 172.21.15.163:0/498194512 shutdown_connections 2023-10-09T06:35:38.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.642+0000 7f24f1a24700 1 -- 172.21.15.163:0/498194512 wait complete. 2023-10-09T06:35:38.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.643+0000 7f24f1a24700 1 Processor -- start 2023-10-09T06:35:38.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.643+0000 7f24f1a24700 1 -- start start 2023-10-09T06:35:38.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.644+0000 7f24f1a24700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24ec063580 0x7f24ec0c6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:38.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.644+0000 7f24f1a24700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24ec06e9b0 0x7f24ec068d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:38.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.644+0000 7f24f1a24700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24ec1575a0 0x7f24ec06ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:38.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.644+0000 7f24f1a24700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec069410 con 0x7f24ec063580 2023-10-09T06:35:38.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.644+0000 7f24f1a24700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec06bf60 con 0x7f24ec1575a0 2023-10-09T06:35:38.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.644+0000 7f24f1a24700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec06b630 con 0x7f24ec06e9b0 2023-10-09T06:35:38.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.644+0000 7f24f0a22700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24ec063580 0x7f24ec0c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:38.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.644+0000 7f24f0a22700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24ec063580 0x7f24ec0c6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:37602/0 (socket says 172.21.15.163:37602) 2023-10-09T06:35:38.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.644+0000 7f24f0a22700 1 -- 172.21.15.163:0/524730558 learned_addr learned my addr 172.21.15.163:0/524730558 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:38.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.644+0000 7f24f1223700 1 --2- 172.21.15.163:0/524730558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24ec1575a0 0x7f24ec06ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:38.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.644+0000 7f24ebfff700 1 --2- 172.21.15.163:0/524730558 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24ec06e9b0 0x7f24ec068d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:38.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.644+0000 7f24f0a22700 1 -- 172.21.15.163:0/524730558 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24ec06e9b0 msgr2=0x7f24ec068d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:38.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.644+0000 7f24f0a22700 1 --2- 172.21.15.163:0/524730558 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24ec06e9b0 0x7f24ec068d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:38.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.644+0000 7f24f0a22700 1 -- 172.21.15.163:0/524730558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24ec1575a0 msgr2=0x7f24ec06ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:38.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.644+0000 7f24f0a22700 1 --2- 172.21.15.163:0/524730558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24ec1575a0 0x7f24ec06ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:38.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.644+0000 7f24f0a22700 1 -- 172.21.15.163:0/524730558 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d80094f0 con 0x7f24ec063580 2023-10-09T06:35:38.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.644+0000 7f24f0a22700 1 --2- 172.21.15.163:0/524730558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24ec063580 0x7f24ec0c6810 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f24e000d860 tx=0x7f24e0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:38.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.645+0000 7f24e8ff9700 1 -- 172.21.15.163:0/524730558 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e00125f0 con 0x7f24ec063580 2023-10-09T06:35:38.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.645+0000 7f24e8ff9700 1 -- 172.21.15.163:0/524730558 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e0012750 con 0x7f24ec063580 2023-10-09T06:35:38.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.645+0000 7f24e8ff9700 1 -- 172.21.15.163:0/524730558 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e0012af0 con 0x7f24ec063580 2023-10-09T06:35:38.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.645+0000 7f24f1a24700 1 -- 172.21.15.163:0/524730558 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24ec06b910 con 0x7f24ec063580 2023-10-09T06:35:38.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.645+0000 7f24f1a24700 1 -- 172.21.15.163:0/524730558 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24ec06b070 con 0x7f24ec063580 2023-10-09T06:35:38.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.646+0000 7f24f1a24700 1 -- 172.21.15.163:0/524730558 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24ec06cfa0 con 0x7f24ec063580 2023-10-09T06:35:38.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.646+0000 7f24e8ff9700 1 -- 172.21.15.163:0/524730558 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f24e0013480 con 0x7f24ec063580 2023-10-09T06:35:38.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.646+0000 7f24e8ff9700 1 --2- 172.21.15.163:0/524730558 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f24d0067170 0x7f24d0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:38.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.647+0000 7f24e8ff9700 1 -- 172.21.15.163:0/524730558 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f24e0084a60 con 0x7f24ec063580 2023-10-09T06:35:38.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.647+0000 7f24ebfff700 1 --2- 172.21.15.163:0/524730558 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f24d0067170 0x7f24d0069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:38.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.647+0000 7f24ebfff700 1 --2- 172.21.15.163:0/524730558 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f24d0067170 0x7f24d0069620 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f24d80099b0 tx=0x7f24d8017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:38.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.651+0000 7f24e8ff9700 1 -- 172.21.15.163:0/524730558 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f24e0051a00 con 0x7f24ec063580 2023-10-09T06:35:38.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.868+0000 7f24f1a24700 1 -- 172.21.15.163:0/524730558 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f24ec0cc140 con 0x7f24ec063580 2023-10-09T06:35:38.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.869+0000 7f24e8ff9700 1 -- 172.21.15.163:0/524730558 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) v1 ==== 100+0+11 (secure 0 0 0) 0x7f24e0055020 con 0x7f24ec063580 2023-10-09T06:35:38.869 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 64 2023-10-09T06:35:38.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.871+0000 7f24f1a24700 1 -- 172.21.15.163:0/524730558 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f24d0067170 msgr2=0x7f24d0069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:38.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.871+0000 7f24f1a24700 1 --2- 172.21.15.163:0/524730558 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f24d0067170 0x7f24d0069620 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f24d80099b0 tx=0x7f24d8017000 comp rx=0 tx=0).stop 2023-10-09T06:35:38.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.871+0000 7f24f1a24700 1 -- 172.21.15.163:0/524730558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24ec063580 msgr2=0x7f24ec0c6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:38.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.871+0000 7f24f1a24700 1 --2- 172.21.15.163:0/524730558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24ec063580 0x7f24ec0c6810 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f24e000d860 tx=0x7f24e0008040 comp rx=0 tx=0).stop 2023-10-09T06:35:38.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.871+0000 7f24f1a24700 1 -- 172.21.15.163:0/524730558 shutdown_connections 2023-10-09T06:35:38.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.871+0000 7f24f1a24700 1 --2- 172.21.15.163:0/524730558 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f24d0067170 0x7f24d0069620 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:38.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.871+0000 7f24f1a24700 1 --2- 172.21.15.163:0/524730558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24ec063580 0x7f24ec0c6810 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:38.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.871+0000 7f24f1a24700 1 --2- 172.21.15.163:0/524730558 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24ec1575a0 0x7f24ec06ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:38.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.871+0000 7f24f1a24700 1 --2- 172.21.15.163:0/524730558 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24ec06e9b0 0x7f24ec068d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:38.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.871+0000 7f24f1a24700 1 -- 172.21.15.163:0/524730558 >> 172.21.15.163:0/524730558 conn(0x7f24ec0c9f70 msgr2=0x7f24ec0c1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:38.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.871+0000 7f24f1a24700 1 -- 172.21.15.163:0/524730558 shutdown_connections 2023-10-09T06:35:38.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:38.871+0000 7f24f1a24700 1 -- 172.21.15.163:0/524730558 wait complete. 2023-10-09T06:35:38.884 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:35:39.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.018+0000 7f667b410700 1 Processor -- start 2023-10-09T06:35:39.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.019+0000 7f667b410700 1 -- start start 2023-10-09T06:35:39.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.020+0000 7f667b410700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6674159b10 0x7f66740cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:39.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.020+0000 7f667b410700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f667415d730 0x7f667415dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:39.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.020+0000 7f667b410700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6674159ee0 0x7f667415a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:39.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.020+0000 7f667b410700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66740c74b0 con 0x7f667415d730 2023-10-09T06:35:39.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.020+0000 7f667b410700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f667415e0e0 con 0x7f6674159ee0 2023-10-09T06:35:39.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.020+0000 7f667b410700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66740c8700 con 0x7f6674159b10 2023-10-09T06:35:39.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.020+0000 7f66789ab700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f667415d730 0x7f667415dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:39.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.020+0000 7f66789ab700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f667415d730 0x7f667415dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:37618/0 (socket says 172.21.15.163:37618) 2023-10-09T06:35:39.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.020+0000 7f66789ab700 1 -- 172.21.15.163:0/3888692950 learned_addr learned my addr 172.21.15.163:0/3888692950 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:39.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.020+0000 7f66791ac700 1 --2- 172.21.15.163:0/3888692950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6674159b10 0x7f66740cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:39.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.020+0000 7f66799ad700 1 --2- 172.21.15.163:0/3888692950 >> v2:172.21.15.181:3300/0 conn(0x7f6674159ee0 0x7f667415a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:39.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.021+0000 7f66789ab700 1 -- 172.21.15.163:0/3888692950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6674159b10 msgr2=0x7f66740cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:39.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.021+0000 7f66789ab700 1 --2- 172.21.15.163:0/3888692950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6674159b10 0x7f66740cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:39.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.021+0000 7f66789ab700 1 -- 172.21.15.163:0/3888692950 >> v2:172.21.15.181:3300/0 conn(0x7f6674159ee0 msgr2=0x7f667415a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:39.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.021+0000 7f66789ab700 1 --2- 172.21.15.163:0/3888692950 >> v2:172.21.15.181:3300/0 conn(0x7f6674159ee0 0x7f667415a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:39.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.021+0000 7f66789ab700 1 -- 172.21.15.163:0/3888692950 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66740c9070 con 0x7f667415d730 2023-10-09T06:35:39.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.021+0000 7f66789ab700 1 --2- 172.21.15.163:0/3888692950 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f667415d730 0x7f667415dba0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f66740ca830 tx=0x7f6664009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd5e40ce989513c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:39.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.021+0000 7f666bfff700 1 -- 172.21.15.163:0/3888692950 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666400b6d0 con 0x7f667415d730 2023-10-09T06:35:39.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.021+0000 7f666bfff700 1 -- 172.21.15.163:0/3888692950 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f666400b830 con 0x7f667415d730 2023-10-09T06:35:39.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.021+0000 7f666bfff700 1 -- 172.21.15.163:0/3888692950 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666400bbf0 con 0x7f667415d730 2023-10-09T06:35:39.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.022+0000 7f667b410700 1 -- 172.21.15.163:0/3888692950 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f667415d730 msgr2=0x7f667415dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:39.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.022+0000 7f667b410700 1 --2- 172.21.15.163:0/3888692950 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f667415d730 0x7f667415dba0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f66740ca830 tx=0x7f6664009250 comp rx=0 tx=0).stop 2023-10-09T06:35:39.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.022+0000 7f667b410700 1 -- 172.21.15.163:0/3888692950 shutdown_connections 2023-10-09T06:35:39.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.022+0000 7f667b410700 1 --2- 172.21.15.163:0/3888692950 >> v2:172.21.15.181:3300/0 conn(0x7f6674159ee0 0x7f667415a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:39.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.022+0000 7f667b410700 1 --2- 172.21.15.163:0/3888692950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6674159b10 0x7f66740cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:39.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.022+0000 7f667b410700 1 --2- 172.21.15.163:0/3888692950 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f667415d730 0x7f667415dba0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:39.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.022+0000 7f667b410700 1 -- 172.21.15.163:0/3888692950 >> 172.21.15.163:0/3888692950 conn(0x7f66740c9870 msgr2=0x7f66740c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:39.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.022+0000 7f667b410700 1 -- 172.21.15.163:0/3888692950 shutdown_connections 2023-10-09T06:35:39.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.022+0000 7f667b410700 1 -- 172.21.15.163:0/3888692950 wait complete. 2023-10-09T06:35:39.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.023+0000 7f667b410700 1 Processor -- start 2023-10-09T06:35:39.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.023+0000 7f667b410700 1 -- start start 2023-10-09T06:35:39.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.024+0000 7f667b410700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6674159b10 0x7f66740b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:39.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.024+0000 7f667b410700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6674159ee0 0x7f66741479d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:39.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.024+0000 7f667b410700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f667415d730 0x7f667416ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:39.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.024+0000 7f667b410700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6674147f10 con 0x7f667415d730 2023-10-09T06:35:39.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.024+0000 7f667b410700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667416b2a0 con 0x7f6674159b10 2023-10-09T06:35:39.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.024+0000 7f667b410700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66740b1800 con 0x7f6674159ee0 2023-10-09T06:35:39.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.025+0000 7f66799ad700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f667415d730 0x7f667416ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:39.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.025+0000 7f66799ad700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f667415d730 0x7f667416ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:37626/0 (socket says 172.21.15.163:37626) 2023-10-09T06:35:39.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.025+0000 7f66789ab700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6674159ee0 0x7f66741479d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:39.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.025+0000 7f66799ad700 1 -- 172.21.15.163:0/2866977373 learned_addr learned my addr 172.21.15.163:0/2866977373 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:39.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.025+0000 7f66791ac700 1 --2- 172.21.15.163:0/2866977373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6674159b10 0x7f66740b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:39.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.025+0000 7f66789ab700 1 -- 172.21.15.163:0/2866977373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6674159b10 msgr2=0x7f66740b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:39.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.025+0000 7f66789ab700 1 --2- 172.21.15.163:0/2866977373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6674159b10 0x7f66740b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:39.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.025+0000 7f66789ab700 1 -- 172.21.15.163:0/2866977373 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f667415d730 msgr2=0x7f667416ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:39.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.025+0000 7f66789ab700 1 --2- 172.21.15.163:0/2866977373 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f667415d730 0x7f667416ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:39.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.025+0000 7f66789ab700 1 -- 172.21.15.163:0/2866977373 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66640094f0 con 0x7f6674159ee0 2023-10-09T06:35:39.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.025+0000 7f66799ad700 1 --2- 172.21.15.163:0/2866977373 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f667415d730 0x7f667416ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:35:39.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.025+0000 7f66791ac700 1 --2- 172.21.15.163:0/2866977373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6674159b10 0x7f66740b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:35:39.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.025+0000 7f66789ab700 1 --2- 172.21.15.163:0/2866977373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6674159ee0 0x7f66741479d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6674064900 tx=0x7f6664017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:39.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.025+0000 7f66697fa700 1 -- 172.21.15.163:0/2866977373 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6664000d10 con 0x7f6674159ee0 2023-10-09T06:35:39.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.025+0000 7f66697fa700 1 -- 172.21.15.163:0/2866977373 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f666400bbb0 con 0x7f6674159ee0 2023-10-09T06:35:39.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.025+0000 7f66697fa700 1 -- 172.21.15.163:0/2866977373 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6664011650 con 0x7f6674159ee0 2023-10-09T06:35:39.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.025+0000 7f667b410700 1 -- 172.21.15.163:0/2866977373 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66740b1ae0 con 0x7f6674159ee0 2023-10-09T06:35:39.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.025+0000 7f667b410700 1 -- 172.21.15.163:0/2866977373 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66740b20a0 con 0x7f6674159ee0 2023-10-09T06:35:39.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.026+0000 7f667b410700 1 -- 172.21.15.163:0/2866977373 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66740afc30 con 0x7f6674159ee0 2023-10-09T06:35:39.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.027+0000 7f66697fa700 1 -- 172.21.15.163:0/2866977373 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f6664011a40 con 0x7f6674159ee0 2023-10-09T06:35:39.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.027+0000 7f66697fa700 1 --2- 172.21.15.163:0/2866977373 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6658067170 0x7f6658069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:39.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.027+0000 7f66697fa700 1 -- 172.21.15.163:0/2866977373 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f6664085e50 con 0x7f6674159ee0 2023-10-09T06:35:39.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.028+0000 7f66791ac700 1 --2- 172.21.15.163:0/2866977373 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6658067170 0x7f6658069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:39.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.028+0000 7f66791ac700 1 --2- 172.21.15.163:0/2866977373 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6658067170 0x7f6658069620 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6670011480 tx=0x7f66700091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:39.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.033+0000 7f66697fa700 1 -- 172.21.15.163:0/2866977373 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f66740afc30 con 0x7f6674159ee0 2023-10-09T06:35:39.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.320+0000 7f667b410700 1 -- 172.21.15.163:0/2866977373 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66740afc30 con 0x7f6674159ee0 2023-10-09T06:35:39.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.320+0000 7f66697fa700 1 -- 172.21.15.163:0/2866977373 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4608 (secure 0 0 0) 0x7f66740afc30 con 0x7f6674159ee0 2023-10-09T06:35:39.321 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:35:39.321 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5407,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3673547262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3673547262},{"type":"v1","addr":"172.21.15.181:6837","nonce":3673547262}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5410,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1705122825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1705122825},{"type":"v1","addr":"172.21.15.181:6836","nonce":1705122825}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/596332711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":596332711},{"type":"v1","addr":"172.21.15.163:6834","nonce":596332711}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:32:30.882925+0000","modified":"2023-10-09T06:32:34.526904+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/3415100865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3415100865},{"type":"v1","addr":"172.21.15.163:6835","nonce":3415100865}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-09T06:35:39.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.322+0000 7f667b410700 1 -- 172.21.15.163:0/2866977373 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6658067170 msgr2=0x7f6658069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:39.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.322+0000 7f667b410700 1 --2- 172.21.15.163:0/2866977373 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6658067170 0x7f6658069620 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6670011480 tx=0x7f66700091a0 comp rx=0 tx=0).stop 2023-10-09T06:35:39.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.322+0000 7f667b410700 1 -- 172.21.15.163:0/2866977373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6674159ee0 msgr2=0x7f66741479d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:39.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.322+0000 7f667b410700 1 --2- 172.21.15.163:0/2866977373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6674159ee0 0x7f66741479d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6674064900 tx=0x7f6664017000 comp rx=0 tx=0).stop 2023-10-09T06:35:39.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.322+0000 7f667b410700 1 -- 172.21.15.163:0/2866977373 shutdown_connections 2023-10-09T06:35:39.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.322+0000 7f667b410700 1 --2- 172.21.15.163:0/2866977373 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6658067170 0x7f6658069620 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:39.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.322+0000 7f667b410700 1 --2- 172.21.15.163:0/2866977373 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f667415d730 0x7f667416ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:39.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.322+0000 7f667b410700 1 --2- 172.21.15.163:0/2866977373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6674159b10 0x7f66740b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:39.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.322+0000 7f667b410700 1 --2- 172.21.15.163:0/2866977373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6674159ee0 0x7f66741479d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:39.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.322+0000 7f667b410700 1 -- 172.21.15.163:0/2866977373 >> 172.21.15.163:0/2866977373 conn(0x7f66740c9870 msgr2=0x7f66741556c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:39.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.322+0000 7f667b410700 1 -- 172.21.15.163:0/2866977373 shutdown_connections 2023-10-09T06:35:39.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.322+0000 7f667b410700 1 -- 172.21.15.163:0/2866977373 wait complete. 2023-10-09T06:35:39.324 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 41 2023-10-09T06:35:39.336 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:35:39.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.459+0000 7f30a7513700 1 Processor -- start 2023-10-09T06:35:39.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.460+0000 7f30a7513700 1 -- start start 2023-10-09T06:35:39.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.461+0000 7f30a7513700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f30a0069560 0x7f30a00699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:39.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.461+0000 7f30a7513700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30a0069ee0 con 0x7f30a0159c60 2023-10-09T06:35:39.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.461+0000 7f30a7513700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30a015a160 con 0x7f30a0159890 2023-10-09T06:35:39.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.461+0000 7f30a7513700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30a00d13a0 con 0x7f30a0069560 2023-10-09T06:35:39.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.461+0000 7f30a4aae700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f30a0159c60 0x7f30a015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:58104/0 (socket says 172.21.15.163:58104) 2023-10-09T06:35:39.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.461+0000 7f30a4aae700 1 -- 172.21.15.163:0/2932099245 learned_addr learned my addr 172.21.15.163:0/2932099245 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:35:39.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.461+0000 7f30a5ab0700 1 --2- 172.21.15.163:0/2932099245 >> v2:172.21.15.181:3300/0 conn(0x7f30a0069560 0x7f30a00699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:39.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.461+0000 7f3097fff700 1 -- 172.21.15.163:0/2932099245 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1424300357 0 0) 0x7f30a0069ee0 con 0x7f30a0159c60 2023-10-09T06:35:39.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.461+0000 7f3097fff700 1 -- 172.21.15.163:0/2932099245 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f308c003570 con 0x7f30a0159c60 2023-10-09T06:35:39.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.461+0000 7f3097fff700 1 -- 172.21.15.163:0/2932099245 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172534957 0 0) 0x7f30a015a160 con 0x7f30a0159890 2023-10-09T06:35:39.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.461+0000 7f3097fff700 1 -- 172.21.15.163:0/2932099245 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30a0069ee0 con 0x7f30a0159890 2023-10-09T06:35:39.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.461+0000 7f30a5ab0700 1 -- 172.21.15.163:0/2932099245 >> v1:172.21.15.181:6789/0 conn(0x7f30a0159890 legacy=0x7f30a015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:39.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.461+0000 7f30a5ab0700 1 -- 172.21.15.163:0/2932099245 >> v1:172.21.15.163:6789/0 conn(0x7f30a0159c60 legacy=0x7f30a015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:39.463 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.461+0000 7f30a5ab0700 1 -- 172.21.15.163:0/2932099245 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a015e140 con 0x7f30a0069560 2023-10-09T06:35:39.463 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.461+0000 7f30a5ab0700 1 --2- 172.21.15.163:0/2932099245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a0069560 0x7f30a00699a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3098009e60 tx=0x7f3098009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7f9bc5a39962fb0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:39.463 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.461+0000 7f3097fff700 1 -- 172.21.15.163:0/2932099245 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309800e070 con 0x7f30a0069560 2023-10-09T06:35:39.463 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.461+0000 7f3097fff700 1 -- 172.21.15.163:0/2932099245 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f309800bab0 con 0x7f30a0069560 2023-10-09T06:35:39.463 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.461+0000 7f3097fff700 1 -- 172.21.15.163:0/2932099245 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309800be70 con 0x7f30a0069560 2023-10-09T06:35:39.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.462+0000 7f30a7513700 1 -- 172.21.15.163:0/2932099245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a0069560 msgr2=0x7f30a00699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:39.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.462+0000 7f30a7513700 1 --2- 172.21.15.163:0/2932099245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a0069560 0x7f30a00699a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3098009e60 tx=0x7f3098009780 comp rx=0 tx=0).stop 2023-10-09T06:35:39.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.462+0000 7f30a7513700 1 -- 172.21.15.163:0/2932099245 shutdown_connections 2023-10-09T06:35:39.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.462+0000 7f30a7513700 1 --2- 172.21.15.163:0/2932099245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a0069560 0x7f30a00699a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:39.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.462+0000 7f30a7513700 1 -- 172.21.15.163:0/2932099245 >> 172.21.15.163:0/2932099245 conn(0x7f30a0068e20 msgr2=0x7f30a00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:39.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.462+0000 7f30a7513700 1 -- 172.21.15.163:0/2932099245 shutdown_connections 2023-10-09T06:35:39.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.462+0000 7f30a7513700 1 -- 172.21.15.163:0/2932099245 wait complete. 2023-10-09T06:35:39.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.464+0000 7f30a7513700 1 Processor -- start 2023-10-09T06:35:39.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.464+0000 7f30a7513700 1 -- start start 2023-10-09T06:35:39.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.465+0000 7f30a7513700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a0069560 0x7f30a01524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:39.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.465+0000 7f30a7513700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30a0159890 0x7f30a00ce530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:39.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.465+0000 7f30a7513700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30a0159c60 0x7f30a00cea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:39.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.465+0000 7f30a7513700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a00cd260 con 0x7f30a0159c60 2023-10-09T06:35:39.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.465+0000 7f30a7513700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a00cd540 con 0x7f30a0069560 2023-10-09T06:35:39.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.465+0000 7f30a7513700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a00cd820 con 0x7f30a0159890 2023-10-09T06:35:39.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.465+0000 7f30a5ab0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30a0159c60 0x7f30a00cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:39.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.465+0000 7f30a5ab0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30a0159c60 0x7f30a00cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:37638/0 (socket says 172.21.15.163:37638) 2023-10-09T06:35:39.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.465+0000 7f30a5ab0700 1 -- 172.21.15.163:0/3085507546 learned_addr learned my addr 172.21.15.163:0/3085507546 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:39.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.465+0000 7f30a52af700 1 --2- 172.21.15.163:0/3085507546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a0069560 0x7f30a01524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:39.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.465+0000 7f30a4aae700 1 --2- 172.21.15.163:0/3085507546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30a0159890 0x7f30a00ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:39.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.466+0000 7f30a52af700 1 -- 172.21.15.163:0/3085507546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30a0159890 msgr2=0x7f30a00ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:39.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.466+0000 7f30a52af700 1 --2- 172.21.15.163:0/3085507546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30a0159890 0x7f30a00ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:39.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.466+0000 7f30a52af700 1 -- 172.21.15.163:0/3085507546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30a0159c60 msgr2=0x7f30a00cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:39.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.466+0000 7f30a52af700 1 --2- 172.21.15.163:0/3085507546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30a0159c60 0x7f30a00cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:39.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.466+0000 7f30a52af700 1 -- 172.21.15.163:0/3085507546 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3098009a20 con 0x7f30a0069560 2023-10-09T06:35:39.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.466+0000 7f30a52af700 1 --2- 172.21.15.163:0/3085507546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a0069560 0x7f30a01524f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f309c00c560 tx=0x7f309c007d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:39.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.466+0000 7f30957fa700 1 -- 172.21.15.163:0/3085507546 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c011590 con 0x7f30a0069560 2023-10-09T06:35:39.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.466+0000 7f30957fa700 1 -- 172.21.15.163:0/3085507546 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f309c0116f0 con 0x7f30a0069560 2023-10-09T06:35:39.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.466+0000 7f30a7513700 1 -- 172.21.15.163:0/3085507546 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a00cdb00 con 0x7f30a0069560 2023-10-09T06:35:39.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.466+0000 7f30957fa700 1 -- 172.21.15.163:0/3085507546 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c011ab0 con 0x7f30a0069560 2023-10-09T06:35:39.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.466+0000 7f30a7513700 1 -- 172.21.15.163:0/3085507546 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a00ce000 con 0x7f30a0069560 2023-10-09T06:35:39.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.467+0000 7f30a7513700 1 -- 172.21.15.163:0/3085507546 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30a00cde70 con 0x7f30a0069560 2023-10-09T06:35:39.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.468+0000 7f30957fa700 1 -- 172.21.15.163:0/3085507546 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f309c01c460 con 0x7f30a0069560 2023-10-09T06:35:39.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.468+0000 7f30957fa700 1 --2- 172.21.15.163:0/3085507546 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3084067170 0x7f3084069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:39.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.468+0000 7f30957fa700 1 -- 172.21.15.163:0/3085507546 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f309c084470 con 0x7f30a0069560 2023-10-09T06:35:39.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.468+0000 7f30a4aae700 1 --2- 172.21.15.163:0/3085507546 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3084067170 0x7f3084069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:39.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.469+0000 7f30a4aae700 1 --2- 172.21.15.163:0/3085507546 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3084067170 0x7f3084069620 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3090007fd0 tx=0x7f3090009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:39.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.473+0000 7f30957fa700 1 -- 172.21.15.163:0/3085507546 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f309c051410 con 0x7f30a0069560 2023-10-09T06:35:39.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.668+0000 7f30a7513700 1 -- 172.21.15.163:0/3085507546 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30a00ce220 con 0x7f30a0069560 2023-10-09T06:35:39.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.669+0000 7f30957fa700 1 -- 172.21.15.163:0/3085507546 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+15498 (secure 0 0 0) 0x7f309c054a30 con 0x7f30a0069560 2023-10-09T06:35:39.669 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:35:39.669 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":65,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:33:25.634897+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:32:28.847438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-09T06:32:29.837643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.1","mappings":[{"from":6,"to":0}]},{"pgid":"11.3c","mappings":[{"from":6,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:35:39.671 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.671+0000 7f30a7513700 1 -- 172.21.15.163:0/3085507546 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3084067170 msgr2=0x7f3084069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:39.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.671+0000 7f30a7513700 1 --2- 172.21.15.163:0/3085507546 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3084067170 0x7f3084069620 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3090007fd0 tx=0x7f3090009040 comp rx=0 tx=0).stop 2023-10-09T06:35:39.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.671+0000 7f30a7513700 1 -- 172.21.15.163:0/3085507546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a0069560 msgr2=0x7f30a01524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:39.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.671+0000 7f30a7513700 1 --2- 172.21.15.163:0/3085507546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a0069560 0x7f30a01524f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f309c00c560 tx=0x7f309c007d50 comp rx=0 tx=0).stop 2023-10-09T06:35:39.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.672+0000 7f30a7513700 1 -- 172.21.15.163:0/3085507546 shutdown_connections 2023-10-09T06:35:39.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.672+0000 7f30a7513700 1 --2- 172.21.15.163:0/3085507546 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3084067170 0x7f3084069620 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:39.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.672+0000 7f30a7513700 1 --2- 172.21.15.163:0/3085507546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30a0159c60 0x7f30a00cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:39.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.672+0000 7f30a7513700 1 --2- 172.21.15.163:0/3085507546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30a0069560 0x7f30a01524f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:39.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.672+0000 7f30a7513700 1 --2- 172.21.15.163:0/3085507546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30a0159890 0x7f30a00ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:39.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.672+0000 7f30a7513700 1 -- 172.21.15.163:0/3085507546 >> 172.21.15.163:0/3085507546 conn(0x7f30a0068e20 msgr2=0x7f30a00b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:39.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.672+0000 7f30a7513700 1 -- 172.21.15.163:0/3085507546 shutdown_connections 2023-10-09T06:35:39.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.672+0000 7f30a7513700 1 -- 172.21.15.163:0/3085507546 wait complete. 2023-10-09T06:35:39.684 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-09T06:35:39.684 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:35:39.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.803+0000 7f674b0df700 1 Processor -- start 2023-10-09T06:35:39.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.803+0000 7f674b0df700 1 -- start start 2023-10-09T06:35:39.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.804+0000 7f674b0df700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6744143050 0x7f6744159c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:39.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.804+0000 7f674b0df700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f674406b9e0 0x7f674406be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:39.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.804+0000 7f674b0df700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67440b3810 0x7f67440b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:39.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.804+0000 7f674b0df700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f674406c390 con 0x7f6744143050 2023-10-09T06:35:39.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.804+0000 7f674b0df700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6744143420 con 0x7f674406b9e0 2023-10-09T06:35:39.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.804+0000 7f674b0df700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674406a1e0 con 0x7f67440b3810 2023-10-09T06:35:39.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.804+0000 7f674a0dd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6744143050 0x7f6744159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:39.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.804+0000 7f674a0dd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6744143050 0x7f6744159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:37644/0 (socket says 172.21.15.163:37644) 2023-10-09T06:35:39.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.804+0000 7f674a0dd700 1 -- 172.21.15.163:0/1926622400 learned_addr learned my addr 172.21.15.163:0/1926622400 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:39.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.804+0000 7f67498dc700 1 --2- 172.21.15.163:0/1926622400 >> v2:172.21.15.181:3300/0 conn(0x7f674406b9e0 0x7f674406be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:39.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.804+0000 7f674a8de700 1 --2- 172.21.15.163:0/1926622400 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67440b3810 0x7f67440b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:39.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.804+0000 7f674a0dd700 1 -- 172.21.15.163:0/1926622400 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67440b3810 msgr2=0x7f67440b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:39.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.804+0000 7f674a0dd700 1 --2- 172.21.15.163:0/1926622400 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67440b3810 0x7f67440b3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:39.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.804+0000 7f674a0dd700 1 -- 172.21.15.163:0/1926622400 >> v2:172.21.15.181:3300/0 conn(0x7f674406b9e0 msgr2=0x7f674406be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:39.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.804+0000 7f674a0dd700 1 --2- 172.21.15.163:0/1926622400 >> v2:172.21.15.181:3300/0 conn(0x7f674406b9e0 0x7f674406be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:39.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.804+0000 7f674a0dd700 1 -- 172.21.15.163:0/1926622400 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67440695d0 con 0x7f6744143050 2023-10-09T06:35:39.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.805+0000 7f674a0dd700 1 --2- 172.21.15.163:0/1926622400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6744143050 0x7f6744159c90 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f674406ca20 tx=0x7f6740009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af81fa62beb58930 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:39.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.805+0000 7f67490db700 1 -- 172.21.15.163:0/1926622400 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6740005c10 con 0x7f6744143050 2023-10-09T06:35:39.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.805+0000 7f67490db700 1 -- 172.21.15.163:0/1926622400 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6740005d70 con 0x7f6744143050 2023-10-09T06:35:39.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.805+0000 7f67490db700 1 -- 172.21.15.163:0/1926622400 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6740006280 con 0x7f6744143050 2023-10-09T06:35:39.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.805+0000 7f674b0df700 1 -- 172.21.15.163:0/1926622400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6744143050 msgr2=0x7f6744159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:39.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.805+0000 7f674b0df700 1 --2- 172.21.15.163:0/1926622400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6744143050 0x7f6744159c90 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f674406ca20 tx=0x7f6740009220 comp rx=0 tx=0).stop 2023-10-09T06:35:39.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.805+0000 7f674b0df700 1 -- 172.21.15.163:0/1926622400 shutdown_connections 2023-10-09T06:35:39.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.805+0000 7f674b0df700 1 --2- 172.21.15.163:0/1926622400 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67440b3810 0x7f67440b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:39.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.805+0000 7f674b0df700 1 --2- 172.21.15.163:0/1926622400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6744143050 0x7f6744159c90 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:39.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.805+0000 7f674b0df700 1 --2- 172.21.15.163:0/1926622400 >> v2:172.21.15.181:3300/0 conn(0x7f674406b9e0 0x7f674406be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:39.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.805+0000 7f674b0df700 1 -- 172.21.15.163:0/1926622400 >> 172.21.15.163:0/1926622400 conn(0x7f67440ce430 msgr2=0x7f67440ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:39.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.805+0000 7f674b0df700 1 -- 172.21.15.163:0/1926622400 shutdown_connections 2023-10-09T06:35:39.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.805+0000 7f674b0df700 1 -- 172.21.15.163:0/1926622400 wait complete. 2023-10-09T06:35:39.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.807+0000 7f674b0df700 1 Processor -- start 2023-10-09T06:35:39.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.807+0000 7f674b0df700 1 -- start start 2023-10-09T06:35:39.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.807+0000 7f674b0df700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f674406b9e0 0x7f674414bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:39.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.807+0000 7f674b0df700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67440b3810 0x7f6744139740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:39.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.807+0000 7f674b0df700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6744143050 0x7f674413b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:39.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.807+0000 7f674b0df700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674413a9a0 con 0x7f67440b3810 2023-10-09T06:35:39.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.807+0000 7f674b0df700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674415c050 con 0x7f674406b9e0 2023-10-09T06:35:39.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.807+0000 7f674b0df700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674414c430 con 0x7f6744143050 2023-10-09T06:35:39.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.807+0000 7f67498dc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67440b3810 0x7f6744139740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:39.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.807+0000 7f67498dc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67440b3810 0x7f6744139740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:37660/0 (socket says 172.21.15.163:37660) 2023-10-09T06:35:39.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.807+0000 7f67498dc700 1 -- 172.21.15.163:0/1597974726 learned_addr learned my addr 172.21.15.163:0/1597974726 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:39.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.807+0000 7f674a0dd700 1 --2- 172.21.15.163:0/1597974726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f674406b9e0 0x7f674414bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:39.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.807+0000 7f674a8de700 1 --2- 172.21.15.163:0/1597974726 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6744143050 0x7f674413b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:39.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.808+0000 7f67498dc700 1 -- 172.21.15.163:0/1597974726 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6744143050 msgr2=0x7f674413b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:39.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.808+0000 7f67498dc700 1 --2- 172.21.15.163:0/1597974726 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6744143050 0x7f674413b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:39.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.808+0000 7f67498dc700 1 -- 172.21.15.163:0/1597974726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f674406b9e0 msgr2=0x7f674414bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:39.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.808+0000 7f67498dc700 1 --2- 172.21.15.163:0/1597974726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f674406b9e0 0x7f674414bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:39.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.808+0000 7f67498dc700 1 -- 172.21.15.163:0/1597974726 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6740009480 con 0x7f67440b3810 2023-10-09T06:35:39.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.808+0000 7f67498dc700 1 --2- 172.21.15.163:0/1597974726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67440b3810 0x7f6744139740 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f673400ecb0 tx=0x7f67340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:39.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.808+0000 7f673a7fc700 1 -- 172.21.15.163:0/1597974726 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673400c850 con 0x7f67440b3810 2023-10-09T06:35:39.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.808+0000 7f673a7fc700 1 -- 172.21.15.163:0/1597974726 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673400c9b0 con 0x7f67440b3810 2023-10-09T06:35:39.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.808+0000 7f674b0df700 1 -- 172.21.15.163:0/1597974726 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f674413a100 con 0x7f67440b3810 2023-10-09T06:35:39.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.808+0000 7f673a7fc700 1 -- 172.21.15.163:0/1597974726 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673400cd50 con 0x7f67440b3810 2023-10-09T06:35:39.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.808+0000 7f674b0df700 1 -- 172.21.15.163:0/1597974726 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f674413a6c0 con 0x7f67440b3810 2023-10-09T06:35:39.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.809+0000 7f674b0df700 1 -- 172.21.15.163:0/1597974726 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f674415c200 con 0x7f67440b3810 2023-10-09T06:35:39.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.809+0000 7f673a7fc700 1 -- 172.21.15.163:0/1597974726 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f6734013680 con 0x7f67440b3810 2023-10-09T06:35:39.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.810+0000 7f673a7fc700 1 --2- 172.21.15.163:0/1597974726 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6728067170 0x7f6728069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:39.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.810+0000 7f673a7fc700 1 -- 172.21.15.163:0/1597974726 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f6734084b10 con 0x7f67440b3810 2023-10-09T06:35:39.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.810+0000 7f674a0dd700 1 --2- 172.21.15.163:0/1597974726 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6728067170 0x7f6728069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:39.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.810+0000 7f674a0dd700 1 --2- 172.21.15.163:0/1597974726 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6728067170 0x7f6728069620 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6740007fd0 tx=0x7f6740016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:39.815 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:39.816+0000 7f673a7fc700 1 -- 172.21.15.163:0/1597974726 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f674415c200 con 0x7f67440b3810 2023-10-09T06:35:40.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.061+0000 7f674b0df700 1 -- 172.21.15.163:0/1597974726 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f674415c1c0 con 0x7f67440b3810 2023-10-09T06:35:40.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.062+0000 7f673a7fc700 1 -- 172.21.15.163:0/1597974726 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4608 (secure 0 0 0) 0x7f6734051ab0 con 0x7f67440b3810 2023-10-09T06:35:40.062 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:35:40.062 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5407,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3673547262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3673547262},{"type":"v1","addr":"172.21.15.181:6837","nonce":3673547262}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5410,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1705122825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1705122825},{"type":"v1","addr":"172.21.15.181:6836","nonce":1705122825}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/596332711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":596332711},{"type":"v1","addr":"172.21.15.163:6834","nonce":596332711}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:32:30.882925+0000","modified":"2023-10-09T06:32:34.526904+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/3415100865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3415100865},{"type":"v1","addr":"172.21.15.163:6835","nonce":3415100865}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-09T06:35:40.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.064+0000 7f674b0df700 1 -- 172.21.15.163:0/1597974726 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6728067170 msgr2=0x7f6728069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:40.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.064+0000 7f674b0df700 1 --2- 172.21.15.163:0/1597974726 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6728067170 0x7f6728069620 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6740007fd0 tx=0x7f6740016000 comp rx=0 tx=0).stop 2023-10-09T06:35:40.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.064+0000 7f674b0df700 1 -- 172.21.15.163:0/1597974726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67440b3810 msgr2=0x7f6744139740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:40.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.064+0000 7f674b0df700 1 --2- 172.21.15.163:0/1597974726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67440b3810 0x7f6744139740 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f673400ecb0 tx=0x7f67340091a0 comp rx=0 tx=0).stop 2023-10-09T06:35:40.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.064+0000 7f674b0df700 1 -- 172.21.15.163:0/1597974726 shutdown_connections 2023-10-09T06:35:40.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.064+0000 7f674b0df700 1 --2- 172.21.15.163:0/1597974726 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6728067170 0x7f6728069620 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:40.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.064+0000 7f674b0df700 1 --2- 172.21.15.163:0/1597974726 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6744143050 0x7f674413b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:40.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.064+0000 7f674b0df700 1 --2- 172.21.15.163:0/1597974726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f674406b9e0 0x7f674414bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:40.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.064+0000 7f674b0df700 1 --2- 172.21.15.163:0/1597974726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67440b3810 0x7f6744139740 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:40.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.064+0000 7f674b0df700 1 -- 172.21.15.163:0/1597974726 >> 172.21.15.163:0/1597974726 conn(0x7f67440ce430 msgr2=0x7f67440adec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:40.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.064+0000 7f674b0df700 1 -- 172.21.15.163:0/1597974726 shutdown_connections 2023-10-09T06:35:40.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.064+0000 7f674b0df700 1 -- 172.21.15.163:0/1597974726 wait complete. 2023-10-09T06:35:40.065 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 41 2023-10-09T06:35:40.076 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:35:40.194 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.194+0000 7f8f6b665700 1 Processor -- start 2023-10-09T06:35:40.194 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.195+0000 7f8f6b665700 1 -- start start 2023-10-09T06:35:40.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.195+0000 7f8f6b665700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8f64155c10 0x7f8f64155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:40.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.195+0000 7f8f6b665700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8f640ac230 0x7f8f640ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:40.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.195+0000 7f8f6b665700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f641568a0 0x7f8f64156c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:40.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.195+0000 7f8f6b665700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f640ad930 con 0x7f8f64155c10 2023-10-09T06:35:40.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.195+0000 7f8f6b665700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f640acd20 con 0x7f8f640ac230 2023-10-09T06:35:40.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.195+0000 7f8f6b665700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f640ad000 con 0x7f8f641568a0 2023-10-09T06:35:40.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.196+0000 7f8f68c00700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8f640ac230 0x7f8f640ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:40.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.196+0000 7f8f69401700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8f64155c10 0x7f8f64155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:40.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.196+0000 7f8f68c00700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8f640ac230 0x7f8f640ac6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:37054/0 (socket says 172.21.15.163:37054) 2023-10-09T06:35:40.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.196+0000 7f8f68c00700 1 -- 172.21.15.163:0/990888030 learned_addr learned my addr 172.21.15.163:0/990888030 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:40.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.196+0000 7f8f69c02700 1 --2- 172.21.15.163:0/990888030 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f641568a0 0x7f8f64156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:40.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.196+0000 7f8f69401700 1 -- 172.21.15.163:0/990888030 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f641568a0 msgr2=0x7f8f64156c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:40.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.196+0000 7f8f69401700 1 --2- 172.21.15.163:0/990888030 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f641568a0 0x7f8f64156c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:40.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.196+0000 7f8f69401700 1 -- 172.21.15.163:0/990888030 >> v2:172.21.15.181:3300/0 conn(0x7f8f640ac230 msgr2=0x7f8f640ac6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:40.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.196+0000 7f8f69401700 1 --2- 172.21.15.163:0/990888030 >> v2:172.21.15.181:3300/0 conn(0x7f8f640ac230 0x7f8f640ac6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:40.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.196+0000 7f8f69401700 1 -- 172.21.15.163:0/990888030 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f640ad2e0 con 0x7f8f64155c10 2023-10-09T06:35:40.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.196+0000 7f8f69401700 1 --2- 172.21.15.163:0/990888030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f64155c10 0x7f8f64155fe0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8f64065bb0 tx=0x7f8f6000b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da89701300178f25 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:40.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.196+0000 7f8f5bfff700 1 -- 172.21.15.163:0/990888030 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6000da70 con 0x7f8f64155c10 2023-10-09T06:35:40.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.196+0000 7f8f5bfff700 1 -- 172.21.15.163:0/990888030 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6000dbd0 con 0x7f8f64155c10 2023-10-09T06:35:40.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.196+0000 7f8f5bfff700 1 -- 172.21.15.163:0/990888030 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f60019480 con 0x7f8f64155c10 2023-10-09T06:35:40.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.197+0000 7f8f6b665700 1 -- 172.21.15.163:0/990888030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f64155c10 msgr2=0x7f8f64155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:40.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.197+0000 7f8f6b665700 1 --2- 172.21.15.163:0/990888030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f64155c10 0x7f8f64155fe0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8f64065bb0 tx=0x7f8f6000b430 comp rx=0 tx=0).stop 2023-10-09T06:35:40.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.197+0000 7f8f6b665700 1 -- 172.21.15.163:0/990888030 shutdown_connections 2023-10-09T06:35:40.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.197+0000 7f8f6b665700 1 --2- 172.21.15.163:0/990888030 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f641568a0 0x7f8f64156c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:40.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.197+0000 7f8f6b665700 1 --2- 172.21.15.163:0/990888030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f64155c10 0x7f8f64155fe0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:40.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.197+0000 7f8f6b665700 1 --2- 172.21.15.163:0/990888030 >> v2:172.21.15.181:3300/0 conn(0x7f8f640ac230 0x7f8f640ac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:40.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.197+0000 7f8f6b665700 1 -- 172.21.15.163:0/990888030 >> 172.21.15.163:0/990888030 conn(0x7f8f6406a320 msgr2=0x7f8f640cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:40.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.197+0000 7f8f6b665700 1 -- 172.21.15.163:0/990888030 shutdown_connections 2023-10-09T06:35:40.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.197+0000 7f8f6b665700 1 -- 172.21.15.163:0/990888030 wait complete. 2023-10-09T06:35:40.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.198+0000 7f8f6b665700 1 Processor -- start 2023-10-09T06:35:40.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.198+0000 7f8f6b665700 1 -- start start 2023-10-09T06:35:40.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.199+0000 7f8f6b665700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f640ac230 0x7f8f641697c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:40.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.199+0000 7f8f6b665700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f64155c10 0x7f8f64151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:40.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.199+0000 7f8f6b665700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f641568a0 0x7f8f64159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:40.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.199+0000 7f8f6b665700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f640ca9a0 con 0x7f8f640ac230 2023-10-09T06:35:40.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.199+0000 7f8f6b665700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f64169d00 con 0x7f8f64155c10 2023-10-09T06:35:40.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.199+0000 7f8f6b665700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f640aae40 con 0x7f8f641568a0 2023-10-09T06:35:40.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.199+0000 7f8f69401700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f640ac230 0x7f8f641697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:40.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.199+0000 7f8f69401700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f640ac230 0x7f8f641697c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:37684/0 (socket says 172.21.15.163:37684) 2023-10-09T06:35:40.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.199+0000 7f8f69401700 1 -- 172.21.15.163:0/3714255396 learned_addr learned my addr 172.21.15.163:0/3714255396 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:40.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.199+0000 7f8f68c00700 1 --2- 172.21.15.163:0/3714255396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f64155c10 0x7f8f64151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:40.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.200+0000 7f8f69c02700 1 --2- 172.21.15.163:0/3714255396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f641568a0 0x7f8f64159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:40.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.200+0000 7f8f69401700 1 -- 172.21.15.163:0/3714255396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f641568a0 msgr2=0x7f8f64159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:40.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.200+0000 7f8f69401700 1 --2- 172.21.15.163:0/3714255396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f641568a0 0x7f8f64159c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:40.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.200+0000 7f8f69401700 1 -- 172.21.15.163:0/3714255396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f64155c10 msgr2=0x7f8f64151720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:40.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.200+0000 7f8f69401700 1 --2- 172.21.15.163:0/3714255396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f64155c10 0x7f8f64151720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:40.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.200+0000 7f8f69401700 1 -- 172.21.15.163:0/3714255396 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6000b6d0 con 0x7f8f640ac230 2023-10-09T06:35:40.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.200+0000 7f8f69c02700 1 --2- 172.21.15.163:0/3714255396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f641568a0 0x7f8f64159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:35:40.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.200+0000 7f8f69401700 1 --2- 172.21.15.163:0/3714255396 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f640ac230 0x7f8f641697c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000b2c0 tx=0x7f8f6001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:40.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.200+0000 7f8f597fa700 1 -- 172.21.15.163:0/3714255396 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f60020700 con 0x7f8f640ac230 2023-10-09T06:35:40.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.200+0000 7f8f597fa700 1 -- 172.21.15.163:0/3714255396 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f60020880 con 0x7f8f640ac230 2023-10-09T06:35:40.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.200+0000 7f8f6b665700 1 -- 172.21.15.163:0/3714255396 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f640ca6c0 con 0x7f8f640ac230 2023-10-09T06:35:40.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.200+0000 7f8f597fa700 1 -- 172.21.15.163:0/3714255396 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f60020c40 con 0x7f8f640ac230 2023-10-09T06:35:40.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.200+0000 7f8f6b665700 1 -- 172.21.15.163:0/3714255396 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f640ca370 con 0x7f8f640ac230 2023-10-09T06:35:40.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.201+0000 7f8f6b665700 1 -- 172.21.15.163:0/3714255396 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f640aaca0 con 0x7f8f640ac230 2023-10-09T06:35:40.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.201+0000 7f8f597fa700 1 -- 172.21.15.163:0/3714255396 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f8f6002b430 con 0x7f8f640ac230 2023-10-09T06:35:40.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.202+0000 7f8f597fa700 1 --2- 172.21.15.163:0/3714255396 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8f48067170 0x7f8f48069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:40.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.202+0000 7f8f597fa700 1 -- 172.21.15.163:0/3714255396 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f8f60026030 con 0x7f8f640ac230 2023-10-09T06:35:40.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.202+0000 7f8f68c00700 1 --2- 172.21.15.163:0/3714255396 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8f48067170 0x7f8f48069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:40.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.202+0000 7f8f68c00700 1 --2- 172.21.15.163:0/3714255396 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8f48067170 0x7f8f48069620 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8f64154b90 tx=0x7f8f54009250 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:40.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.206+0000 7f8f597fa700 1 -- 172.21.15.163:0/3714255396 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8f640aaca0 con 0x7f8f640ac230 2023-10-09T06:35:40.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.442+0000 7f8f6b665700 1 -- 172.21.15.163:0/3714255396 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f640aaca0 con 0x7f8f640ac230 2023-10-09T06:35:40.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.442+0000 7f8f597fa700 1 -- 172.21.15.163:0/3714255396 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4608 (secure 0 0 0) 0x7f8f640aaca0 con 0x7f8f640ac230 2023-10-09T06:35:40.443 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:35:40.443 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5407,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3673547262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3673547262},{"type":"v1","addr":"172.21.15.181:6837","nonce":3673547262}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5410,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1705122825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1705122825},{"type":"v1","addr":"172.21.15.181:6836","nonce":1705122825}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/596332711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":596332711},{"type":"v1","addr":"172.21.15.163:6834","nonce":596332711}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:32:30.882925+0000","modified":"2023-10-09T06:32:34.526904+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/3415100865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3415100865},{"type":"v1","addr":"172.21.15.163:6835","nonce":3415100865}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-09T06:35:40.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.444+0000 7f8f6b665700 1 -- 172.21.15.163:0/3714255396 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8f48067170 msgr2=0x7f8f48069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:40.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.444+0000 7f8f6b665700 1 --2- 172.21.15.163:0/3714255396 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8f48067170 0x7f8f48069620 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8f64154b90 tx=0x7f8f54009250 comp rx=0 tx=0).stop 2023-10-09T06:35:40.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.445+0000 7f8f6b665700 1 -- 172.21.15.163:0/3714255396 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f640ac230 msgr2=0x7f8f641697c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:40.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.445+0000 7f8f6b665700 1 --2- 172.21.15.163:0/3714255396 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f640ac230 0x7f8f641697c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000b2c0 tx=0x7f8f6001b000 comp rx=0 tx=0).stop 2023-10-09T06:35:40.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.445+0000 7f8f6b665700 1 -- 172.21.15.163:0/3714255396 shutdown_connections 2023-10-09T06:35:40.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.445+0000 7f8f6b665700 1 --2- 172.21.15.163:0/3714255396 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8f48067170 0x7f8f48069620 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:40.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.445+0000 7f8f6b665700 1 --2- 172.21.15.163:0/3714255396 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f640ac230 0x7f8f641697c0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:40.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.445+0000 7f8f6b665700 1 --2- 172.21.15.163:0/3714255396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f641568a0 0x7f8f64159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:40.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.445+0000 7f8f6b665700 1 --2- 172.21.15.163:0/3714255396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f64155c10 0x7f8f64151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:40.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.445+0000 7f8f6b665700 1 -- 172.21.15.163:0/3714255396 >> 172.21.15.163:0/3714255396 conn(0x7f8f6406a320 msgr2=0x7f8f6406ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:40.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.445+0000 7f8f6b665700 1 -- 172.21.15.163:0/3714255396 shutdown_connections 2023-10-09T06:35:40.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.445+0000 7f8f6b665700 1 -- 172.21.15.163:0/3714255396 wait complete. 2023-10-09T06:35:40.446 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 41 2023-10-09T06:35:40.457 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:35:40.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.583+0000 7fbaff59e700 1 Processor -- start 2023-10-09T06:35:40.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.583+0000 7fbaff59e700 1 -- start start 2023-10-09T06:35:40.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.584+0000 7fbaff59e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbb000b3f20 0x7fbb00159c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:40.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.584+0000 7fbaff59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb00143420 con 0x7fbb0006c0f0 2023-10-09T06:35:40.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.584+0000 7fbaff59e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb0006a1e0 con 0x7fbb00143050 2023-10-09T06:35:40.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.584+0000 7fbaff59e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb000695d0 con 0x7fbb000b3f20 2023-10-09T06:35:40.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.584+0000 7fbafdd9b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fbb0006c0f0 0x7fbb000b3810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:58110/0 (socket says 172.21.15.163:58110) 2023-10-09T06:35:40.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.584+0000 7fbafdd9b700 1 -- 172.21.15.163:0/2891796164 learned_addr learned my addr 172.21.15.163:0/2891796164 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:35:40.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.584+0000 7fbafed9d700 1 --2- 172.21.15.163:0/2891796164 >> v2:172.21.15.181:3300/0 conn(0x7fbb000b3f20 0x7fbb00159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:40.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.584+0000 7fbafd59a700 1 -- 172.21.15.163:0/2891796164 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2407352083 0 0) 0x7fbb00143420 con 0x7fbb0006c0f0 2023-10-09T06:35:40.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.584+0000 7fbafd59a700 1 -- 172.21.15.163:0/2891796164 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbae8003570 con 0x7fbb0006c0f0 2023-10-09T06:35:40.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.584+0000 7fbafd59a700 1 -- 172.21.15.163:0/2891796164 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 464871933 0 0) 0x7fbb0006a1e0 con 0x7fbb00143050 2023-10-09T06:35:40.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.584+0000 7fbafd59a700 1 -- 172.21.15.163:0/2891796164 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb00143420 con 0x7fbb00143050 2023-10-09T06:35:40.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.584+0000 7fbafed9d700 1 -- 172.21.15.163:0/2891796164 >> v1:172.21.15.181:6789/0 conn(0x7fbb00143050 legacy=0x7fbb0006b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:40.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.584+0000 7fbafed9d700 1 -- 172.21.15.163:0/2891796164 >> v1:172.21.15.163:6789/0 conn(0x7fbb0006c0f0 legacy=0x7fbb000b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:40.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.584+0000 7fbafed9d700 1 -- 172.21.15.163:0/2891796164 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb000698b0 con 0x7fbb000b3f20 2023-10-09T06:35:40.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.584+0000 7fbafed9d700 1 --2- 172.21.15.163:0/2891796164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb000b3f20 0x7fbb00159c90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8009e60 tx=0x7fbaf8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=981fbeef37e42751 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:40.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.585+0000 7fbafd59a700 1 -- 172.21.15.163:0/2891796164 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf800e070 con 0x7fbb000b3f20 2023-10-09T06:35:40.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.585+0000 7fbafd59a700 1 -- 172.21.15.163:0/2891796164 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf800ba70 con 0x7fbb000b3f20 2023-10-09T06:35:40.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.585+0000 7fbafd59a700 1 -- 172.21.15.163:0/2891796164 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf800be10 con 0x7fbb000b3f20 2023-10-09T06:35:40.588 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.585+0000 7fbaff59e700 1 -- 172.21.15.163:0/2891796164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb000b3f20 msgr2=0x7fbb00159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:40.588 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.585+0000 7fbaff59e700 1 --2- 172.21.15.163:0/2891796164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb000b3f20 0x7fbb00159c90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8009e60 tx=0x7fbaf8009780 comp rx=0 tx=0).stop 2023-10-09T06:35:40.588 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.585+0000 7fbaff59e700 1 -- 172.21.15.163:0/2891796164 shutdown_connections 2023-10-09T06:35:40.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.585+0000 7fbaff59e700 1 --2- 172.21.15.163:0/2891796164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb000b3f20 0x7fbb00159c90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:40.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.585+0000 7fbaff59e700 1 -- 172.21.15.163:0/2891796164 >> 172.21.15.163:0/2891796164 conn(0x7fbb000ce430 msgr2=0x7fbb000ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:40.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.585+0000 7fbaff59e700 1 -- 172.21.15.163:0/2891796164 shutdown_connections 2023-10-09T06:35:40.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.585+0000 7fbaff59e700 1 -- 172.21.15.163:0/2891796164 wait complete. 2023-10-09T06:35:40.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.586+0000 7fbaff59e700 1 Processor -- start 2023-10-09T06:35:40.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.586+0000 7fbaff59e700 1 -- start start 2023-10-09T06:35:40.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.587+0000 7fbaff59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb0006c0f0 0x7fbb00156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:40.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.587+0000 7fbaff59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb000b3f20 0x7fbb0013ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:40.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.587+0000 7fbaff59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb00143050 0x7fbb0013f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:40.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.587+0000 7fbaff59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0016a760 con 0x7fbb00143050 2023-10-09T06:35:40.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.587+0000 7fbaff59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb000af410 con 0x7fbb0006c0f0 2023-10-09T06:35:40.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.587+0000 7fbaff59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb000aeae0 con 0x7fbb000b3f20 2023-10-09T06:35:40.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.587+0000 7fbafed9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb00143050 0x7fbb0013f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:40.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.587+0000 7fbafed9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb00143050 0x7fbb0013f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:37698/0 (socket says 172.21.15.163:37698) 2023-10-09T06:35:40.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.587+0000 7fbafed9d700 1 -- 172.21.15.163:0/2430728145 learned_addr learned my addr 172.21.15.163:0/2430728145 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:40.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.587+0000 7fbafe59c700 1 --2- 172.21.15.163:0/2430728145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb0006c0f0 0x7fbb00156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:40.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.587+0000 7fbafdd9b700 1 --2- 172.21.15.163:0/2430728145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb000b3f20 0x7fbb0013ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:40.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.587+0000 7fbafed9d700 1 -- 172.21.15.163:0/2430728145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb000b3f20 msgr2=0x7fbb0013ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:40.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.587+0000 7fbafed9d700 1 --2- 172.21.15.163:0/2430728145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb000b3f20 0x7fbb0013ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:40.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.587+0000 7fbafed9d700 1 -- 172.21.15.163:0/2430728145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb0006c0f0 msgr2=0x7fbb00156f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:40.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.587+0000 7fbafed9d700 1 --2- 172.21.15.163:0/2430728145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb0006c0f0 0x7fbb00156f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:40.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.587+0000 7fbafed9d700 1 -- 172.21.15.163:0/2430728145 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf8009a20 con 0x7fbb00143050 2023-10-09T06:35:40.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.588+0000 7fbafed9d700 1 --2- 172.21.15.163:0/2430728145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb00143050 0x7fbb0013f010 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800b610 tx=0x7fbaf8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:40.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.588+0000 7fbaee7fc700 1 -- 172.21.15.163:0/2430728145 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8012b20 con 0x7fbb00143050 2023-10-09T06:35:40.591 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.588+0000 7fbaee7fc700 1 -- 172.21.15.163:0/2430728145 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf8021070 con 0x7fbb00143050 2023-10-09T06:35:40.591 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.588+0000 7fbaff59e700 1 -- 172.21.15.163:0/2430728145 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb000aedc0 con 0x7fbb00143050 2023-10-09T06:35:40.591 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.588+0000 7fbaee7fc700 1 -- 172.21.15.163:0/2430728145 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8026430 con 0x7fbb00143050 2023-10-09T06:35:40.591 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.588+0000 7fbaff59e700 1 -- 172.21.15.163:0/2430728145 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb00157490 con 0x7fbb00143050 2023-10-09T06:35:40.592 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.589+0000 7fbaff59e700 1 -- 172.21.15.163:0/2430728145 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb000af5a0 con 0x7fbb00143050 2023-10-09T06:35:40.592 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.589+0000 7fbaee7fc700 1 -- 172.21.15.163:0/2430728145 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fbaf8023030 con 0x7fbb00143050 2023-10-09T06:35:40.592 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.590+0000 7fbaee7fc700 1 --2- 172.21.15.163:0/2430728145 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbae0067170 0x7fbae0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:40.592 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.590+0000 7fbaee7fc700 1 -- 172.21.15.163:0/2430728145 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fbaf80868c0 con 0x7fbb00143050 2023-10-09T06:35:40.592 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.590+0000 7fbafe59c700 1 --2- 172.21.15.163:0/2430728145 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbae0067170 0x7fbae0069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:40.592 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.590+0000 7fbafe59c700 1 --2- 172.21.15.163:0/2430728145 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbae0067170 0x7fbae0069620 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0001230 tx=0x7fbaf0007cd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:40.593 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.594+0000 7fbaee7fc700 1 -- 172.21.15.163:0/2430728145 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbaf8053860 con 0x7fbb00143050 2023-10-09T06:35:40.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.801+0000 7fbaff59e700 1 -- 172.21.15.163:0/2430728145 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb0004c3c0 con 0x7fbb00143050 2023-10-09T06:35:40.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.801+0000 7fbaee7fc700 1 -- 172.21.15.163:0/2430728145 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+15498 (secure 0 0 0) 0x7fbaf8056e80 con 0x7fbb00143050 2023-10-09T06:35:40.802 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:35:40.802 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":65,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:33:25.634897+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:32:28.847438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-09T06:32:29.837643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.1","mappings":[{"from":6,"to":0}]},{"pgid":"11.3c","mappings":[{"from":6,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:35:40.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.803+0000 7fbaff59e700 1 -- 172.21.15.163:0/2430728145 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbae0067170 msgr2=0x7fbae0069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:40.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.803+0000 7fbaff59e700 1 --2- 172.21.15.163:0/2430728145 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbae0067170 0x7fbae0069620 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0001230 tx=0x7fbaf0007cd0 comp rx=0 tx=0).stop 2023-10-09T06:35:40.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.803+0000 7fbaff59e700 1 -- 172.21.15.163:0/2430728145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb00143050 msgr2=0x7fbb0013f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:40.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.803+0000 7fbaff59e700 1 --2- 172.21.15.163:0/2430728145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb00143050 0x7fbb0013f010 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800b610 tx=0x7fbaf8018000 comp rx=0 tx=0).stop 2023-10-09T06:35:40.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.804+0000 7fbaff59e700 1 -- 172.21.15.163:0/2430728145 shutdown_connections 2023-10-09T06:35:40.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.804+0000 7fbaff59e700 1 --2- 172.21.15.163:0/2430728145 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbae0067170 0x7fbae0069620 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:40.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.804+0000 7fbaff59e700 1 --2- 172.21.15.163:0/2430728145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb00143050 0x7fbb0013f010 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:40.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.804+0000 7fbaff59e700 1 --2- 172.21.15.163:0/2430728145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb0006c0f0 0x7fbb00156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:40.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.804+0000 7fbaff59e700 1 --2- 172.21.15.163:0/2430728145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb000b3f20 0x7fbb0013ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:40.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.804+0000 7fbaff59e700 1 -- 172.21.15.163:0/2430728145 >> 172.21.15.163:0/2430728145 conn(0x7fbb000ce430 msgr2=0x7fbb000ade30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:40.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.804+0000 7fbaff59e700 1 -- 172.21.15.163:0/2430728145 shutdown_connections 2023-10-09T06:35:40.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.804+0000 7fbaff59e700 1 -- 172.21.15.163:0/2430728145 wait complete. 2023-10-09T06:35:40.817 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-09T06:35:40.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.936+0000 7fad3c812700 1 Processor -- start 2023-10-09T06:35:40.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.936+0000 7fad3c812700 1 -- start start 2023-10-09T06:35:40.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.937+0000 7fad3c812700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fad381598d0 0x7fad3815aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:40.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.937+0000 7fad3c812700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad3815af50 con 0x7fad380c7bd0 2023-10-09T06:35:40.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.937+0000 7fad3c812700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad3806e980 con 0x7fad3806bbe0 2023-10-09T06:35:40.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.937+0000 7fad3c812700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad380ce180 con 0x7fad381598d0 2023-10-09T06:35:40.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.937+0000 7fad36ffd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fad380c7bd0 0x7fad3806ee10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:58118/0 (socket says 172.21.15.163:58118) 2023-10-09T06:35:40.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.937+0000 7fad36ffd700 1 -- 172.21.15.163:0/4042155533 learned_addr learned my addr 172.21.15.163:0/4042155533 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:35:40.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.937+0000 7fad377fe700 1 --2- 172.21.15.163:0/4042155533 >> v2:172.21.15.181:3300/0 conn(0x7fad381598d0 0x7fad3815aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:40.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.937+0000 7fad367fc700 1 -- 172.21.15.163:0/4042155533 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 306770633 0 0) 0x7fad2c001110 con 0x7fad380c7bd0 2023-10-09T06:35:40.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.937+0000 7fad367fc700 1 -- 172.21.15.163:0/4042155533 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad20003570 con 0x7fad380c7bd0 2023-10-09T06:35:40.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.938+0000 7fad367fc700 1 -- 172.21.15.163:0/4042155533 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2062706912 0 0) 0x7fad3806e980 con 0x7fad3806bbe0 2023-10-09T06:35:40.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.938+0000 7fad367fc700 1 -- 172.21.15.163:0/4042155533 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad2c001110 con 0x7fad3806bbe0 2023-10-09T06:35:40.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.938+0000 7fad377fe700 1 -- 172.21.15.163:0/4042155533 >> v1:172.21.15.181:6789/0 conn(0x7fad3806bbe0 legacy=0x7fad38068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:40.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.938+0000 7fad377fe700 1 -- 172.21.15.163:0/4042155533 >> v1:172.21.15.163:6789/0 conn(0x7fad380c7bd0 legacy=0x7fad3806ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:40.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.938+0000 7fad377fe700 1 -- 172.21.15.163:0/4042155533 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad380c80f0 con 0x7fad381598d0 2023-10-09T06:35:40.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.938+0000 7fad377fe700 1 --2- 172.21.15.163:0/4042155533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad381598d0 0x7fad3815aa10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fad280098e0 tx=0x7fad280091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=199df1c33a61563 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:40.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.938+0000 7fad367fc700 1 -- 172.21.15.163:0/4042155533 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad28011630 con 0x7fad381598d0 2023-10-09T06:35:40.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.938+0000 7fad367fc700 1 -- 172.21.15.163:0/4042155533 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad28011790 con 0x7fad381598d0 2023-10-09T06:35:40.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.938+0000 7fad367fc700 1 -- 172.21.15.163:0/4042155533 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad28011b30 con 0x7fad381598d0 2023-10-09T06:35:40.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.939+0000 7fad3c812700 1 -- 172.21.15.163:0/4042155533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad381598d0 msgr2=0x7fad3815aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:40.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.939+0000 7fad3c812700 1 --2- 172.21.15.163:0/4042155533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad381598d0 0x7fad3815aa10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fad280098e0 tx=0x7fad280091a0 comp rx=0 tx=0).stop 2023-10-09T06:35:40.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.939+0000 7fad3c812700 1 -- 172.21.15.163:0/4042155533 shutdown_connections 2023-10-09T06:35:40.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.939+0000 7fad3c812700 1 --2- 172.21.15.163:0/4042155533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad381598d0 0x7fad3815aa10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:40.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.939+0000 7fad3c812700 1 -- 172.21.15.163:0/4042155533 >> 172.21.15.163:0/4042155533 conn(0x7fad3806b490 msgr2=0x7fad380cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:40.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.939+0000 7fad3c812700 1 -- 172.21.15.163:0/4042155533 shutdown_connections 2023-10-09T06:35:40.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.939+0000 7fad3c812700 1 -- 172.21.15.163:0/4042155533 wait complete. 2023-10-09T06:35:40.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.940+0000 7fad3c812700 1 Processor -- start 2023-10-09T06:35:40.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.940+0000 7fad3c812700 1 -- start start 2023-10-09T06:35:40.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.941+0000 7fad3c812700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad3806bbe0 0x7fad38155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:40.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.941+0000 7fad3c812700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad380c7bd0 0x7fad380c5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:40.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.941+0000 7fad3c812700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad381598d0 0x7fad38169d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:40.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.941+0000 7fad3c812700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad380afbd0 con 0x7fad381598d0 2023-10-09T06:35:40.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.941+0000 7fad3c812700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad381556d0 con 0x7fad3806bbe0 2023-10-09T06:35:40.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.941+0000 7fad3c812700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad380c43b0 con 0x7fad380c7bd0 2023-10-09T06:35:40.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.941+0000 7fad377fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad3806bbe0 0x7fad38155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:40.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.941+0000 7fad36ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad380c7bd0 0x7fad380c5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:40.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.941+0000 7fad37fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad381598d0 0x7fad38169d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:40.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.941+0000 7fad377fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad3806bbe0 0x7fad38155190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:37100/0 (socket says 172.21.15.163:37100) 2023-10-09T06:35:40.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.941+0000 7fad377fe700 1 -- 172.21.15.163:0/3881871768 learned_addr learned my addr 172.21.15.163:0/3881871768 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:40.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.941+0000 7fad37fff700 1 -- 172.21.15.163:0/3881871768 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad380c7bd0 msgr2=0x7fad380c5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:40.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.941+0000 7fad37fff700 1 --2- 172.21.15.163:0/3881871768 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad380c7bd0 0x7fad380c5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:40.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.941+0000 7fad37fff700 1 -- 172.21.15.163:0/3881871768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad3806bbe0 msgr2=0x7fad38155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:40.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.941+0000 7fad37fff700 1 --2- 172.21.15.163:0/3881871768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad3806bbe0 0x7fad38155190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:40.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.941+0000 7fad37fff700 1 -- 172.21.15.163:0/3881871768 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad28009420 con 0x7fad381598d0 2023-10-09T06:35:40.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.941+0000 7fad377fe700 1 --2- 172.21.15.163:0/3881871768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad3806bbe0 0x7fad38155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:35:40.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.941+0000 7fad37fff700 1 --2- 172.21.15.163:0/3881871768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad381598d0 0x7fad38169d00 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fad300012e0 tx=0x7fad30002730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:40.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.941+0000 7fad1f7fe700 1 -- 172.21.15.163:0/3881871768 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad30012630 con 0x7fad381598d0 2023-10-09T06:35:40.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.941+0000 7fad1f7fe700 1 -- 172.21.15.163:0/3881871768 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad30012790 con 0x7fad381598d0 2023-10-09T06:35:40.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.941+0000 7fad1f7fe700 1 -- 172.21.15.163:0/3881871768 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad30012b50 con 0x7fad381598d0 2023-10-09T06:35:40.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.941+0000 7fad3c812700 1 -- 172.21.15.163:0/3881871768 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad380c4690 con 0x7fad381598d0 2023-10-09T06:35:40.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.942+0000 7fad3c812700 1 -- 172.21.15.163:0/3881871768 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad380c4c50 con 0x7fad381598d0 2023-10-09T06:35:40.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.943+0000 7fad3c812700 1 -- 172.21.15.163:0/3881871768 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad3804c3c0 con 0x7fad381598d0 2023-10-09T06:35:40.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.943+0000 7fad1f7fe700 1 -- 172.21.15.163:0/3881871768 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fad3001b400 con 0x7fad381598d0 2023-10-09T06:35:40.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.943+0000 7fad1f7fe700 1 --2- 172.21.15.163:0/3881871768 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fad24067170 0x7fad24069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:40.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.943+0000 7fad1f7fe700 1 -- 172.21.15.163:0/3881871768 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5763+0+0 (secure 0 0 0) 0x7fad30084b30 con 0x7fad381598d0 2023-10-09T06:35:40.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.943+0000 7fad377fe700 1 --2- 172.21.15.163:0/3881871768 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fad24067170 0x7fad24069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:40.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.944+0000 7fad377fe700 1 --2- 172.21.15.163:0/3881871768 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fad24067170 0x7fad24069620 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fad28011fd0 tx=0x7fad28017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:40.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:40.950+0000 7fad1f7fe700 1 -- 172.21.15.163:0/3881871768 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad30051a00 con 0x7fad381598d0 2023-10-09T06:35:41.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.226+0000 7fad3c812700 1 -- 172.21.15.163:0/3881871768 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fad38069010 con 0x7fad381598d0 2023-10-09T06:35:41.227 INFO:tasks.ceph.mon.a.smithi163.stderr:2023-10-09T06:35:41.227+0000 7ff251c04700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-09T06:35:41.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.233+0000 7fad1f7fe700 1 -- 172.21.15.163:0/3881871768 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v42) v1 ==== 161+0+0 (secure 0 0 0) 0x7fad30016020 con 0x7fad381598d0 2023-10-09T06:35:41.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.235+0000 7fad3c812700 1 -- 172.21.15.163:0/3881871768 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fad24067170 msgr2=0x7fad24069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:41.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.235+0000 7fad3c812700 1 --2- 172.21.15.163:0/3881871768 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fad24067170 0x7fad24069620 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fad28011fd0 tx=0x7fad28017000 comp rx=0 tx=0).stop 2023-10-09T06:35:41.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.235+0000 7fad3c812700 1 -- 172.21.15.163:0/3881871768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad381598d0 msgr2=0x7fad38169d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:41.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.235+0000 7fad3c812700 1 --2- 172.21.15.163:0/3881871768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad381598d0 0x7fad38169d00 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fad300012e0 tx=0x7fad30002730 comp rx=0 tx=0).stop 2023-10-09T06:35:41.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.235+0000 7fad3c812700 1 -- 172.21.15.163:0/3881871768 shutdown_connections 2023-10-09T06:35:41.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.235+0000 7fad3c812700 1 --2- 172.21.15.163:0/3881871768 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fad24067170 0x7fad24069620 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:41.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.235+0000 7fad3c812700 1 --2- 172.21.15.163:0/3881871768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad381598d0 0x7fad38169d00 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:41.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.235+0000 7fad3c812700 1 --2- 172.21.15.163:0/3881871768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad3806bbe0 0x7fad38155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:41.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.235+0000 7fad3c812700 1 --2- 172.21.15.163:0/3881871768 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad380c7bd0 0x7fad380c5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:41.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.235+0000 7fad3c812700 1 -- 172.21.15.163:0/3881871768 >> 172.21.15.163:0/3881871768 conn(0x7fad3806b490 msgr2=0x7fad38156080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:41.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.235+0000 7fad3c812700 1 -- 172.21.15.163:0/3881871768 shutdown_connections 2023-10-09T06:35:41.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.235+0000 7fad3c812700 1 -- 172.21.15.163:0/3881871768 wait complete. 2023-10-09T06:35:41.237 INFO:teuthology.orchestra.run.smithi163.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-09T06:35:41.249 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-09T06:35:41.268 INFO:tasks.ceph.mds.c.smithi163.stdout:starting mds.c at 2023-10-09T06:35:41.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.365+0000 7f15b0611700 1 Processor -- start 2023-10-09T06:35:41.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.365+0000 7f15b0611700 1 -- start start 2023-10-09T06:35:41.374 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.366+0000 7f15b0611700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f15a806af10 0x7f15a8157da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:41.374 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.366+0000 7f15b0611700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f15a81582e0 0x7f15a80ad220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:41.374 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.366+0000 7f15b0611700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a8158880 con 0x7f15a806ab40 2023-10-09T06:35:41.374 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.366+0000 7f15b0611700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a8158e40 con 0x7f15a806af10 2023-10-09T06:35:41.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.366+0000 7f15b0611700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a8158b60 con 0x7f15a81582e0 2023-10-09T06:35:41.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.366+0000 7f15adbac700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f15a806af10 0x7f15a8157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:41.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.367+0000 7f15adbac700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f15a806af10 0x7f15a8157da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:37736/0 (socket says 172.21.15.163:37736) 2023-10-09T06:35:41.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.367+0000 7f15adbac700 1 -- 172.21.15.163:0/3736425289 learned_addr learned my addr 172.21.15.163:0/3736425289 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:41.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.367+0000 7f15aebae700 1 --2- 172.21.15.163:0/3736425289 >> v2:172.21.15.181:3300/0 conn(0x7f15a81582e0 0x7f15a80ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:41.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.367+0000 7f15adbac700 1 -- 172.21.15.163:0/3736425289 >> v2:172.21.15.181:3300/0 conn(0x7f15a81582e0 msgr2=0x7f15a80ad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:41.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.367+0000 7f15adbac700 1 --2- 172.21.15.163:0/3736425289 >> v2:172.21.15.181:3300/0 conn(0x7f15a81582e0 0x7f15a80ad220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:41.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.367+0000 7f15adbac700 1 -- 172.21.15.163:0/3736425289 >> v1:172.21.15.181:6789/0 conn(0x7f15a806ab40 legacy=0x7f15a815bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:41.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.367+0000 7f15adbac700 1 -- 172.21.15.163:0/3736425289 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a815b5a0 con 0x7f15a806af10 2023-10-09T06:35:41.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.367+0000 7f15adbac700 1 --2- 172.21.15.163:0/3736425289 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15a806af10 0x7f15a8157da0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f15a80ae350 tx=0x7f1598009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829068eb2370e666 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:41.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.367+0000 7f15ad3ab700 1 -- 172.21.15.163:0/3736425289 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159800b820 con 0x7f15a806af10 2023-10-09T06:35:41.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.367+0000 7f15ad3ab700 1 -- 172.21.15.163:0/3736425289 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159800b980 con 0x7f15a806af10 2023-10-09T06:35:41.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.367+0000 7f15ad3ab700 1 -- 172.21.15.163:0/3736425289 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159800bd20 con 0x7f15a806af10 2023-10-09T06:35:41.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.368+0000 7f15b0611700 1 -- 172.21.15.163:0/3736425289 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15a806af10 msgr2=0x7f15a8157da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:41.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.368+0000 7f15b0611700 1 --2- 172.21.15.163:0/3736425289 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15a806af10 0x7f15a8157da0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f15a80ae350 tx=0x7f1598009250 comp rx=0 tx=0).stop 2023-10-09T06:35:41.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.368+0000 7f15b0611700 1 -- 172.21.15.163:0/3736425289 shutdown_connections 2023-10-09T06:35:41.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.368+0000 7f15b0611700 1 --2- 172.21.15.163:0/3736425289 >> v2:172.21.15.181:3300/0 conn(0x7f15a81582e0 0x7f15a80ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:41.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.368+0000 7f15b0611700 1 --2- 172.21.15.163:0/3736425289 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15a806af10 0x7f15a8157da0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:41.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.368+0000 7f15b0611700 1 -- 172.21.15.163:0/3736425289 >> 172.21.15.163:0/3736425289 conn(0x7f15a80cd640 msgr2=0x7f15a80cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:41.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.368+0000 7f15b0611700 1 -- 172.21.15.163:0/3736425289 shutdown_connections 2023-10-09T06:35:41.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.368+0000 7f15b0611700 1 -- 172.21.15.163:0/3736425289 wait complete. 2023-10-09T06:35:41.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.369+0000 7f15b0611700 1 Processor -- start 2023-10-09T06:35:41.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.369+0000 7f15b0611700 1 -- start start 2023-10-09T06:35:41.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.369+0000 7f15b0611700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15a806ab40 0x7f15a8150af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:41.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.369+0000 7f15b0611700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15a806af10 0x7f15a80c19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:41.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.370+0000 7f15b0611700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15a81582e0 0x7f15a80c3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:41.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.370+0000 7f15b0611700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a80c2320 con 0x7f15a806af10 2023-10-09T06:35:41.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.370+0000 7f15b0611700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a8169060 con 0x7f15a806ab40 2023-10-09T06:35:41.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.370+0000 7f15b0611700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a8169340 con 0x7f15a81582e0 2023-10-09T06:35:41.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.370+0000 7f15adbac700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15a806af10 0x7f15a80c19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:41.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.370+0000 7f15adbac700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15a806af10 0x7f15a80c19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:37752/0 (socket says 172.21.15.163:37752) 2023-10-09T06:35:41.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.370+0000 7f15adbac700 1 -- 172.21.15.163:0/2760056109 learned_addr learned my addr 172.21.15.163:0/2760056109 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:41.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.370+0000 7f15ae3ad700 1 --2- 172.21.15.163:0/2760056109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15a806ab40 0x7f15a8150af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:41.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.370+0000 7f15aebae700 1 --2- 172.21.15.163:0/2760056109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15a81582e0 0x7f15a80c3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:41.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.370+0000 7f15adbac700 1 -- 172.21.15.163:0/2760056109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15a81582e0 msgr2=0x7f15a80c3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:41.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.370+0000 7f15adbac700 1 --2- 172.21.15.163:0/2760056109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15a81582e0 0x7f15a80c3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:41.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.370+0000 7f15adbac700 1 -- 172.21.15.163:0/2760056109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15a806ab40 msgr2=0x7f15a8150af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:41.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.370+0000 7f15adbac700 1 --2- 172.21.15.163:0/2760056109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15a806ab40 0x7f15a8150af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:41.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.370+0000 7f15adbac700 1 -- 172.21.15.163:0/2760056109 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15980094f0 con 0x7f15a806af10 2023-10-09T06:35:41.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.370+0000 7f15adbac700 1 --2- 172.21.15.163:0/2760056109 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15a806af10 0x7f15a80c19b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f15980030c0 tx=0x7f1598016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:41.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.370+0000 7f159e7fc700 1 -- 172.21.15.163:0/2760056109 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159801b470 con 0x7f15a806af10 2023-10-09T06:35:41.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.370+0000 7f159e7fc700 1 -- 172.21.15.163:0/2760056109 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159801b5f0 con 0x7f15a806af10 2023-10-09T06:35:41.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.370+0000 7f15b0611700 1 -- 172.21.15.163:0/2760056109 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a8169620 con 0x7f15a806af10 2023-10-09T06:35:41.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.370+0000 7f159e7fc700 1 -- 172.21.15.163:0/2760056109 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159801b9b0 con 0x7f15a806af10 2023-10-09T06:35:41.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.370+0000 7f15b0611700 1 -- 172.21.15.163:0/2760056109 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a8151030 con 0x7f15a806af10 2023-10-09T06:35:41.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.371+0000 7f15b0611700 1 -- 172.21.15.163:0/2760056109 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a804c3c0 con 0x7f15a806af10 2023-10-09T06:35:41.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.372+0000 7f159e7fc700 1 -- 172.21.15.163:0/2760056109 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f159801c450 con 0x7f15a806af10 2023-10-09T06:35:41.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.372+0000 7f159e7fc700 1 --2- 172.21.15.163:0/2760056109 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f158c067170 0x7f158c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:41.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.372+0000 7f159e7fc700 1 -- 172.21.15.163:0/2760056109 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5849+0+0 (secure 0 0 0) 0x7f1598086510 con 0x7f15a806af10 2023-10-09T06:35:41.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.372+0000 7f15ae3ad700 1 --2- 172.21.15.163:0/2760056109 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f158c067170 0x7f158c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:41.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.372+0000 7f15ae3ad700 1 --2- 172.21.15.163:0/2760056109 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f158c067170 0x7f158c069620 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f15a4007fd0 tx=0x7f15a40026a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:41.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.376+0000 7f159e7fc700 1 -- 172.21.15.163:0/2760056109 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1598053460 con 0x7f15a806af10 2023-10-09T06:35:41.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:41.604+0000 7f15b0611700 1 -- 172.21.15.163:0/2760056109 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f15a806b7b0 con 0x7f15a806af10 2023-10-09T06:35:42.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.231+0000 7f159e7fc700 1 -- 172.21.15.163:0/2760056109 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v43) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1598056a80 con 0x7f15a806af10 2023-10-09T06:35:42.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.234+0000 7f15b0611700 1 -- 172.21.15.163:0/2760056109 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f158c067170 msgr2=0x7f158c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:42.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.234+0000 7f15b0611700 1 --2- 172.21.15.163:0/2760056109 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f158c067170 0x7f158c069620 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f15a4007fd0 tx=0x7f15a40026a0 comp rx=0 tx=0).stop 2023-10-09T06:35:42.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.234+0000 7f15b0611700 1 -- 172.21.15.163:0/2760056109 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15a806af10 msgr2=0x7f15a80c19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:42.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.234+0000 7f15b0611700 1 --2- 172.21.15.163:0/2760056109 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15a806af10 0x7f15a80c19b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f15980030c0 tx=0x7f1598016000 comp rx=0 tx=0).stop 2023-10-09T06:35:42.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.234+0000 7f15b0611700 1 -- 172.21.15.163:0/2760056109 shutdown_connections 2023-10-09T06:35:42.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.234+0000 7f15b0611700 1 --2- 172.21.15.163:0/2760056109 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f158c067170 0x7f158c069620 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:42.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.234+0000 7f15b0611700 1 --2- 172.21.15.163:0/2760056109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15a81582e0 0x7f15a80c3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:42.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.234+0000 7f15b0611700 1 --2- 172.21.15.163:0/2760056109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15a806ab40 0x7f15a8150af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:42.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.234+0000 7f15b0611700 1 --2- 172.21.15.163:0/2760056109 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15a806af10 0x7f15a80c19b0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:42.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.234+0000 7f15b0611700 1 -- 172.21.15.163:0/2760056109 >> 172.21.15.163:0/2760056109 conn(0x7f15a80cd640 msgr2=0x7f15a815a390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:42.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.234+0000 7f15b0611700 1 -- 172.21.15.163:0/2760056109 shutdown_connections 2023-10-09T06:35:42.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.234+0000 7f15b0611700 1 -- 172.21.15.163:0/2760056109 wait complete. 2023-10-09T06:35:42.250 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-09T06:35:42.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.414+0000 7f233884a700 1 Processor -- start 2023-10-09T06:35:42.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.414+0000 7f233884a700 1 -- start start 2023-10-09T06:35:42.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.415+0000 7f233884a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23340cd910 0x7f23340ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:42.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.415+0000 7f233884a700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23340b5b00 con 0x7f2334159ec0 2023-10-09T06:35:42.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.415+0000 7f233884a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23340b5da0 con 0x7f23340b55a0 2023-10-09T06:35:42.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.415+0000 7f233884a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23340b6080 con 0x7f23340cd910 2023-10-09T06:35:42.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.415+0000 7f233259c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f2334159ec0 0x7f23340b4e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:58132/0 (socket says 172.21.15.163:58132) 2023-10-09T06:35:42.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.415+0000 7f233259c700 1 -- 172.21.15.163:0/1934528519 learned_addr learned my addr 172.21.15.163:0/1934528519 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:35:42.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.416+0000 7f233159a700 1 -- 172.21.15.163:0/1934528519 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1518338332 0 0) 0x7f23340b5b00 con 0x7f2334159ec0 2023-10-09T06:35:42.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.416+0000 7f233159a700 1 -- 172.21.15.163:0/1934528519 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f231c003570 con 0x7f2334159ec0 2023-10-09T06:35:42.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.416+0000 7f2331d9b700 1 --2- 172.21.15.163:0/1934528519 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23340cd910 0x7f23340ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:42.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.416+0000 7f233159a700 1 -- 172.21.15.163:0/1934528519 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3780475851 0 0) 0x7f231c003570 con 0x7f2334159ec0 2023-10-09T06:35:42.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.416+0000 7f233159a700 1 -- 172.21.15.163:0/1934528519 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23340cd910 msgr2=0x7f23340ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:42.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.416+0000 7f233159a700 1 --2- 172.21.15.163:0/1934528519 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23340cd910 0x7f23340ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:42.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.416+0000 7f233159a700 1 -- 172.21.15.163:0/1934528519 >> v1:172.21.15.181:6789/0 conn(0x7f23340b55a0 legacy=0x7f23340ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:42.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.416+0000 7f233159a700 1 -- 172.21.15.163:0/1934528519 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23340cc100 con 0x7f2334159ec0 2023-10-09T06:35:42.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.416+0000 7f233159a700 1 -- 172.21.15.163:0/1934528519 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f2324001970 con 0x7f2334159ec0 2023-10-09T06:35:42.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.416+0000 7f233159a700 1 -- 172.21.15.163:0/1934528519 >> v1:172.21.15.163:6789/0 conn(0x7f2334159ec0 legacy=0x7f23340b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:42.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.416+0000 7f233159a700 1 --2- 172.21.15.163:0/1934528519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f231c003a90 0x7f231c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:42.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.416+0000 7f233159a700 1 --2- 172.21.15.163:0/1934528519 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f231c004810 0x7f231c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:42.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.416+0000 7f233159a700 1 --2- 172.21.15.163:0/1934528519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f231c005180 0x7f231c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:42.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.416+0000 7f233159a700 1 -- 172.21.15.163:0/1934528519 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23340b6080 con 0x7f231c005180 2023-10-09T06:35:42.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.416+0000 7f233159a700 1 -- 172.21.15.163:0/1934528519 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231c00bfa0 con 0x7f231c003a90 2023-10-09T06:35:42.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.416+0000 7f233159a700 1 -- 172.21.15.163:0/1934528519 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f231c00c0e0 con 0x7f231c004810 2023-10-09T06:35:42.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.416+0000 7f2332d9d700 1 --2- 172.21.15.163:0/1934528519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f231c005180 0x7f231c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:42.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.416+0000 7f233259c700 1 --2- 172.21.15.163:0/1934528519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f231c003a90 0x7f231c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:42.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.416+0000 7f2331d9b700 1 --2- 172.21.15.163:0/1934528519 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f231c004810 0x7f231c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:42.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.416+0000 7f2332d9d700 1 -- 172.21.15.163:0/1934528519 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f231c004810 msgr2=0x7f231c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:42.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.416+0000 7f2332d9d700 1 --2- 172.21.15.163:0/1934528519 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f231c004810 0x7f231c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:42.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.416+0000 7f2332d9d700 1 -- 172.21.15.163:0/1934528519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f231c003a90 msgr2=0x7f231c003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:42.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.416+0000 7f2332d9d700 1 --2- 172.21.15.163:0/1934528519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f231c003a90 0x7f231c003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:42.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.416+0000 7f2332d9d700 1 -- 172.21.15.163:0/1934528519 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f231c003570 con 0x7f231c005180 2023-10-09T06:35:42.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.417+0000 7f2332d9d700 1 --2- 172.21.15.163:0/1934528519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f231c005180 0x7f231c00ba60 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f232c0019a0 tx=0x7f232c007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed36f82a32e4ff99 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:42.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.417+0000 7f233159a700 1 -- 172.21.15.163:0/1934528519 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c00c5d0 con 0x7f231c005180 2023-10-09T06:35:42.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.417+0000 7f233159a700 1 -- 172.21.15.163:0/1934528519 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232c00c750 con 0x7f231c005180 2023-10-09T06:35:42.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.417+0000 7f233159a700 1 -- 172.21.15.163:0/1934528519 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c00cb10 con 0x7f231c005180 2023-10-09T06:35:42.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.417+0000 7f233884a700 1 -- 172.21.15.163:0/1934528519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f231c005180 msgr2=0x7f231c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:42.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.417+0000 7f233884a700 1 --2- 172.21.15.163:0/1934528519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f231c005180 0x7f231c00ba60 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f232c0019a0 tx=0x7f232c007d10 comp rx=0 tx=0).stop 2023-10-09T06:35:42.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.417+0000 7f2332d9d700 1 -- 172.21.15.163:0/1934528519 reap_dead start 2023-10-09T06:35:42.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.417+0000 7f233884a700 1 -- 172.21.15.163:0/1934528519 shutdown_connections 2023-10-09T06:35:42.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.417+0000 7f233884a700 1 -- 172.21.15.163:0/1934528519 >> 172.21.15.163:0/1934528519 conn(0x7f23340999f0 msgr2=0x7f23340b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:42.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.417+0000 7f233884a700 1 -- 172.21.15.163:0/1934528519 shutdown_connections 2023-10-09T06:35:42.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.417+0000 7f233884a700 1 -- 172.21.15.163:0/1934528519 wait complete. 2023-10-09T06:35:42.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.418+0000 7f233884a700 1 Processor -- start 2023-10-09T06:35:42.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.419+0000 7f233884a700 1 -- start start 2023-10-09T06:35:42.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.419+0000 7f233884a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f231c005180 0x7f23341580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:42.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.419+0000 7f233884a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f233409a460 0x7f23340ad280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:42.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.419+0000 7f233884a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2334157290 0x7f23340c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:42.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.419+0000 7f233884a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233409d280 con 0x7f231c005180 2023-10-09T06:35:42.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.419+0000 7f233884a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2334157ec0 con 0x7f233409a460 2023-10-09T06:35:42.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.419+0000 7f233884a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23340ad7c0 con 0x7f2334157290 2023-10-09T06:35:42.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.419+0000 7f233259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f231c005180 0x7f23341580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:42.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.419+0000 7f233259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f231c005180 0x7f23341580c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:37774/0 (socket says 172.21.15.163:37774) 2023-10-09T06:35:42.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.419+0000 7f233259c700 1 -- 172.21.15.163:0/2874239867 learned_addr learned my addr 172.21.15.163:0/2874239867 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:42.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.419+0000 7f2332d9d700 1 --2- 172.21.15.163:0/2874239867 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2334157290 0x7f23340c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:42.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.419+0000 7f2331d9b700 1 --2- 172.21.15.163:0/2874239867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f233409a460 0x7f23340ad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:42.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.420+0000 7f233259c700 1 -- 172.21.15.163:0/2874239867 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2334157290 msgr2=0x7f23340c98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:42.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.420+0000 7f233259c700 1 --2- 172.21.15.163:0/2874239867 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2334157290 0x7f23340c98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:42.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.420+0000 7f233259c700 1 -- 172.21.15.163:0/2874239867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f233409a460 msgr2=0x7f23340ad280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:42.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.420+0000 7f233259c700 1 --2- 172.21.15.163:0/2874239867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f233409a460 0x7f23340ad280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:42.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.420+0000 7f233259c700 1 -- 172.21.15.163:0/2874239867 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2334157c20 con 0x7f231c005180 2023-10-09T06:35:42.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.420+0000 7f2331d9b700 1 --2- 172.21.15.163:0/2874239867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f233409a460 0x7f23340ad280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:35:42.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.420+0000 7f233259c700 1 --2- 172.21.15.163:0/2874239867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f231c005180 0x7f23341580c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f232400ff10 tx=0x7f2324009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:42.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.420+0000 7f23227fc700 1 -- 172.21.15.163:0/2874239867 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2324005d80 con 0x7f231c005180 2023-10-09T06:35:42.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.420+0000 7f23227fc700 1 -- 172.21.15.163:0/2874239867 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232400d9d0 con 0x7f231c005180 2023-10-09T06:35:42.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.420+0000 7f23227fc700 1 -- 172.21.15.163:0/2874239867 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232400dd50 con 0x7f231c005180 2023-10-09T06:35:42.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.420+0000 7f233884a700 1 -- 172.21.15.163:0/2874239867 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f233409cc30 con 0x7f231c005180 2023-10-09T06:35:42.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.420+0000 7f233884a700 1 -- 172.21.15.163:0/2874239867 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f233406ae00 con 0x7f231c005180 2023-10-09T06:35:42.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.421+0000 7f233884a700 1 -- 172.21.15.163:0/2874239867 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2334063c90 con 0x7f231c005180 2023-10-09T06:35:42.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.421+0000 7f23227fc700 1 -- 172.21.15.163:0/2874239867 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f23240072e0 con 0x7f231c005180 2023-10-09T06:35:42.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.422+0000 7f23227fc700 1 --2- 172.21.15.163:0/2874239867 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f23140671a0 0x7f2314069650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:42.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.422+0000 7f23227fc700 1 -- 172.21.15.163:0/2874239867 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5849+0+0 (secure 0 0 0) 0x7f2324084e60 con 0x7f231c005180 2023-10-09T06:35:42.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.422+0000 7f2331d9b700 1 --2- 172.21.15.163:0/2874239867 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f23140671a0 0x7f2314069650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:42.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.422+0000 7f2331d9b700 1 --2- 172.21.15.163:0/2874239867 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f23140671a0 0x7f2314069650 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f23340b2440 tx=0x7f2328008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:42.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.426+0000 7f23227fc700 1 -- 172.21.15.163:0/2874239867 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2324051db0 con 0x7f231c005180 2023-10-09T06:35:42.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:42.612+0000 7f233884a700 1 -- 172.21.15.163:0/2874239867 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f23340718b0 con 0x7f231c005180 2023-10-09T06:35:43.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.579+0000 7f23227fc700 1 -- 172.21.15.163:0/2874239867 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v67) v1 ==== 181+0+0 (secure 0 0 0) 0x7f23240553d0 con 0x7f231c005180 2023-10-09T06:35:43.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.582+0000 7f233884a700 1 -- 172.21.15.163:0/2874239867 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f23140671a0 msgr2=0x7f2314069650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:43.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.582+0000 7f233884a700 1 --2- 172.21.15.163:0/2874239867 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f23140671a0 0x7f2314069650 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f23340b2440 tx=0x7f2328008040 comp rx=0 tx=0).stop 2023-10-09T06:35:43.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.582+0000 7f233884a700 1 -- 172.21.15.163:0/2874239867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f231c005180 msgr2=0x7f23341580c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:43.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.582+0000 7f233884a700 1 --2- 172.21.15.163:0/2874239867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f231c005180 0x7f23341580c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f232400ff10 tx=0x7f2324009040 comp rx=0 tx=0).stop 2023-10-09T06:35:43.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.583+0000 7f233884a700 1 -- 172.21.15.163:0/2874239867 shutdown_connections 2023-10-09T06:35:43.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.583+0000 7f233884a700 1 --2- 172.21.15.163:0/2874239867 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f23140671a0 0x7f2314069650 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:43.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.583+0000 7f233884a700 1 --2- 172.21.15.163:0/2874239867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f231c005180 0x7f23341580c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:43.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.583+0000 7f233884a700 1 --2- 172.21.15.163:0/2874239867 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2334157290 0x7f23340c98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:43.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.583+0000 7f233884a700 1 --2- 172.21.15.163:0/2874239867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f233409a460 0x7f23340ad280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:43.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.583+0000 7f233884a700 1 -- 172.21.15.163:0/2874239867 >> 172.21.15.163:0/2874239867 conn(0x7f23340999f0 msgr2=0x7f23340cc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:43.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.583+0000 7f233884a700 1 -- 172.21.15.163:0/2874239867 shutdown_connections 2023-10-09T06:35:43.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.583+0000 7f233884a700 1 -- 172.21.15.163:0/2874239867 wait complete. 2023-10-09T06:35:43.587 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_metadata' removed 2023-10-09T06:35:43.600 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-09T06:35:43.674 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:35:43.677 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:35:43.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.724+0000 7fbad08c3700 1 Processor -- start 2023-10-09T06:35:43.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.724+0000 7fbad08c3700 1 -- start start 2023-10-09T06:35:43.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.725+0000 7fbad08c3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbacc0b2100 0x7fbacc068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:43.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.725+0000 7fbad08c3700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbacc0688d0 0x7fbacc06e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:43.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.725+0000 7fbad08c3700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbacc0ae6d0 con 0x7fbacc157740 2023-10-09T06:35:43.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.725+0000 7fbad08c3700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbacc0acc00 con 0x7fbacc0688d0 2023-10-09T06:35:43.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.725+0000 7fbad08c3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbacc06d6a0 con 0x7fbacc0b2100 2023-10-09T06:35:43.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.725+0000 7fbaca59c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fbacc157740 0x7fbacc0adfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:58134/0 (socket says 172.21.15.163:58134) 2023-10-09T06:35:43.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.725+0000 7fbaca59c700 1 -- 172.21.15.163:0/3356410167 learned_addr learned my addr 172.21.15.163:0/3356410167 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:35:43.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.725+0000 7fbacad9d700 1 --2- 172.21.15.163:0/3356410167 >> v2:172.21.15.163:3300/0 conn(0x7fbacc0688d0 0x7fbacc06e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:43.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.726+0000 7fbac9d9b700 1 -- 172.21.15.163:0/3356410167 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1994019274 0 0) 0x7fbabc001110 con 0x7fbacc157740 2023-10-09T06:35:43.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.726+0000 7fbac9d9b700 1 -- 172.21.15.163:0/3356410167 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbab8003570 con 0x7fbacc157740 2023-10-09T06:35:43.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.726+0000 7fbac1d9b700 1 --2- 172.21.15.163:0/3356410167 >> v2:172.21.15.181:3300/0 conn(0x7fbacc0b2100 0x7fbacc068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:43.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.726+0000 7fbacad9d700 1 -- 172.21.15.163:0/3356410167 >> v2:172.21.15.181:3300/0 conn(0x7fbacc0b2100 msgr2=0x7fbacc068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:43.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.726+0000 7fbacad9d700 1 --2- 172.21.15.163:0/3356410167 >> v2:172.21.15.181:3300/0 conn(0x7fbacc0b2100 0x7fbacc068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:43.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.726+0000 7fbacad9d700 1 -- 172.21.15.163:0/3356410167 >> v1:172.21.15.163:6789/0 conn(0x7fbacc157740 legacy=0x7fbacc0adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:43.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.726+0000 7fbacad9d700 1 -- 172.21.15.163:0/3356410167 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbacc068d30 con 0x7fbacc0688d0 2023-10-09T06:35:43.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.726+0000 7fbacad9d700 1 --2- 172.21.15.163:0/3356410167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbacc0688d0 0x7fbacc06e0b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fbacc06d7e0 tx=0x7fbac4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c127e5bf40f800 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:43.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.726+0000 7fbac9d9b700 1 -- 172.21.15.163:0/3356410167 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac4016070 con 0x7fbacc0688d0 2023-10-09T06:35:43.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.726+0000 7fbac9d9b700 1 -- 172.21.15.163:0/3356410167 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac400bb30 con 0x7fbacc0688d0 2023-10-09T06:35:43.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.726+0000 7fbac9d9b700 1 -- 172.21.15.163:0/3356410167 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac400e070 con 0x7fbacc0688d0 2023-10-09T06:35:43.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.726+0000 7fbad08c3700 1 -- 172.21.15.163:0/3356410167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbacc0688d0 msgr2=0x7fbacc06e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:43.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.726+0000 7fbad08c3700 1 --2- 172.21.15.163:0/3356410167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbacc0688d0 0x7fbacc06e0b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fbacc06d7e0 tx=0x7fbac4009780 comp rx=0 tx=0).stop 2023-10-09T06:35:43.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.726+0000 7fbad08c3700 1 -- 172.21.15.163:0/3356410167 shutdown_connections 2023-10-09T06:35:43.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.726+0000 7fbad08c3700 1 --2- 172.21.15.163:0/3356410167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbacc0688d0 0x7fbacc06e0b0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:43.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.726+0000 7fbad08c3700 1 --2- 172.21.15.163:0/3356410167 >> v2:172.21.15.181:3300/0 conn(0x7fbacc0b2100 0x7fbacc068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:43.730 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.726+0000 7fbad08c3700 1 -- 172.21.15.163:0/3356410167 >> 172.21.15.163:0/3356410167 conn(0x7fbacc0c8f90 msgr2=0x7fbacc0c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:43.730 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.727+0000 7fbad08c3700 1 -- 172.21.15.163:0/3356410167 shutdown_connections 2023-10-09T06:35:43.730 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.727+0000 7fbad08c3700 1 -- 172.21.15.163:0/3356410167 wait complete. 2023-10-09T06:35:43.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.728+0000 7fbad08c3700 1 Processor -- start 2023-10-09T06:35:43.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.728+0000 7fbad08c3700 1 -- start start 2023-10-09T06:35:43.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.728+0000 7fbad08c3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbacc0688d0 0x7fbacc15c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:43.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.728+0000 7fbad08c3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbacc0b2100 0x7fbacc0cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:43.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.728+0000 7fbad08c3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbacc157740 0x7fbacc155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:43.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.728+0000 7fbad08c3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc0c47b0 con 0x7fbacc157740 2023-10-09T06:35:43.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.728+0000 7fbad08c3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc16af40 con 0x7fbacc0688d0 2023-10-09T06:35:43.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.728+0000 7fbad08c3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc16b220 con 0x7fbacc0b2100 2023-10-09T06:35:43.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.729+0000 7fbacad9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbacc157740 0x7fbacc155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:43.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.729+0000 7fbacad9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbacc157740 0x7fbacc155170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:37786/0 (socket says 172.21.15.163:37786) 2023-10-09T06:35:43.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.729+0000 7fbacad9d700 1 -- 172.21.15.163:0/2980501832 learned_addr learned my addr 172.21.15.163:0/2980501832 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:43.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.729+0000 7fbac1d9b700 1 --2- 172.21.15.163:0/2980501832 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbacc0b2100 0x7fbacc0cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:43.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.729+0000 7fbaca59c700 1 --2- 172.21.15.163:0/2980501832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbacc0688d0 0x7fbacc15c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:43.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.729+0000 7fbacad9d700 1 -- 172.21.15.163:0/2980501832 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbacc0b2100 msgr2=0x7fbacc0cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:43.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.729+0000 7fbacad9d700 1 --2- 172.21.15.163:0/2980501832 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbacc0b2100 0x7fbacc0cabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:43.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.729+0000 7fbacad9d700 1 -- 172.21.15.163:0/2980501832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbacc0688d0 msgr2=0x7fbacc15c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:43.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.729+0000 7fbacad9d700 1 --2- 172.21.15.163:0/2980501832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbacc0688d0 0x7fbacc15c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:43.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.729+0000 7fbacad9d700 1 -- 172.21.15.163:0/2980501832 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac4009a20 con 0x7fbacc157740 2023-10-09T06:35:43.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.729+0000 7fbaca59c700 1 --2- 172.21.15.163:0/2980501832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbacc0688d0 0x7fbacc15c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:35:43.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.729+0000 7fbacad9d700 1 --2- 172.21.15.163:0/2980501832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbacc157740 0x7fbacc155170 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0b1ad0 tx=0x7fbac4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:43.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.729+0000 7fbac2ffd700 1 -- 172.21.15.163:0/2980501832 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac4027470 con 0x7fbacc157740 2023-10-09T06:35:43.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.729+0000 7fbac2ffd700 1 -- 172.21.15.163:0/2980501832 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac4027610 con 0x7fbacc157740 2023-10-09T06:35:43.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.729+0000 7fbad08c3700 1 -- 172.21.15.163:0/2980501832 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbacc16b500 con 0x7fbacc157740 2023-10-09T06:35:43.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.729+0000 7fbac2ffd700 1 -- 172.21.15.163:0/2980501832 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac40279d0 con 0x7fbacc157740 2023-10-09T06:35:43.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.729+0000 7fbad08c3700 1 -- 172.21.15.163:0/2980501832 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbacc155eb0 con 0x7fbacc157740 2023-10-09T06:35:43.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.730+0000 7fbad08c3700 1 -- 172.21.15.163:0/2980501832 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbacc0718b0 con 0x7fbacc157740 2023-10-09T06:35:43.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.730+0000 7fbac2ffd700 1 -- 172.21.15.163:0/2980501832 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fbac4024070 con 0x7fbacc157740 2023-10-09T06:35:43.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.731+0000 7fbac2ffd700 1 --2- 172.21.15.163:0/2980501832 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbaac067170 0x7fbaac069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:43.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.731+0000 7fbac2ffd700 1 -- 172.21.15.163:0/2980501832 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5411+0+0 (secure 0 0 0) 0x7fbac4086e10 con 0x7fbacc157740 2023-10-09T06:35:43.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.731+0000 7fbaca59c700 1 --2- 172.21.15.163:0/2980501832 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbaac067170 0x7fbaac069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:43.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.732+0000 7fbaca59c700 1 --2- 172.21.15.163:0/2980501832 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbaac067170 0x7fbaac069620 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbabc001270 tx=0x7fbabc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:43.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.735+0000 7fbac2ffd700 1 -- 172.21.15.163:0/2980501832 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbac4053f10 con 0x7fbacc157740 2023-10-09T06:35:43.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:43.921+0000 7fbad08c3700 1 -- 172.21.15.163:0/2980501832 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbacc068d30 con 0x7fbacc157740 2023-10-09T06:35:44.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.589+0000 7fbac2ffd700 1 -- 172.21.15.163:0/2980501832 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v68) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbacc068d30 con 0x7fbacc157740 2023-10-09T06:35:44.593 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.592+0000 7fbad08c3700 1 -- 172.21.15.163:0/2980501832 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbaac067170 msgr2=0x7fbaac069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:44.594 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.592+0000 7fbad08c3700 1 --2- 172.21.15.163:0/2980501832 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbaac067170 0x7fbaac069620 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbabc001270 tx=0x7fbabc009040 comp rx=0 tx=0).stop 2023-10-09T06:35:44.594 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.592+0000 7fbad08c3700 1 -- 172.21.15.163:0/2980501832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbacc157740 msgr2=0x7fbacc155170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:44.594 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.592+0000 7fbad08c3700 1 --2- 172.21.15.163:0/2980501832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbacc157740 0x7fbacc155170 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0b1ad0 tx=0x7fbac4018000 comp rx=0 tx=0).stop 2023-10-09T06:35:44.594 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.593+0000 7fbad08c3700 1 -- 172.21.15.163:0/2980501832 shutdown_connections 2023-10-09T06:35:44.594 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.593+0000 7fbad08c3700 1 --2- 172.21.15.163:0/2980501832 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbaac067170 0x7fbaac069620 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:44.594 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.593+0000 7fbad08c3700 1 --2- 172.21.15.163:0/2980501832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbacc157740 0x7fbacc155170 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:44.594 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.593+0000 7fbad08c3700 1 --2- 172.21.15.163:0/2980501832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbacc0688d0 0x7fbacc15c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:44.594 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.593+0000 7fbad08c3700 1 --2- 172.21.15.163:0/2980501832 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbacc0b2100 0x7fbacc0cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:44.594 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.593+0000 7fbad08c3700 1 -- 172.21.15.163:0/2980501832 >> 172.21.15.163:0/2980501832 conn(0x7fbacc0c8f90 msgr2=0x7fbacc0b12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:44.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.593+0000 7fbad08c3700 1 -- 172.21.15.163:0/2980501832 shutdown_connections 2023-10-09T06:35:44.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.593+0000 7fbad08c3700 1 -- 172.21.15.163:0/2980501832 wait complete. 2023-10-09T06:35:44.595 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_data' removed 2023-10-09T06:35:44.607 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-09T06:35:44.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.723+0000 7f3340435700 1 Processor -- start 2023-10-09T06:35:44.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.723+0000 7f3340435700 1 -- start start 2023-10-09T06:35:44.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.724+0000 7f3340435700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f333806bcd0 0x7f33380caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:44.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.724+0000 7f3340435700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f333806c0a0 0x7f333815a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:44.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.724+0000 7f3340435700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f333815df50 0x7f333815e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:44.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.724+0000 7f3340435700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f333815d7e0 con 0x7f333815df50 2023-10-09T06:35:44.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.724+0000 7f3340435700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f333815dac0 con 0x7f333806bcd0 2023-10-09T06:35:44.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.724+0000 7f3340435700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333815aa30 con 0x7f333806c0a0 2023-10-09T06:35:44.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.725+0000 7f333e9d2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f333815df50 0x7f333815e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:44.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.725+0000 7f333e9d2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f333815df50 0x7f333815e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:37802/0 (socket says 172.21.15.163:37802) 2023-10-09T06:35:44.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.725+0000 7f333e9d2700 1 -- 172.21.15.163:0/2231765044 learned_addr learned my addr 172.21.15.163:0/2231765044 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:44.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.725+0000 7f333e1d1700 1 --2- 172.21.15.163:0/2231765044 >> v2:172.21.15.181:3300/0 conn(0x7f333806bcd0 0x7f33380caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:44.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.725+0000 7f333d9d0700 1 --2- 172.21.15.163:0/2231765044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f333806c0a0 0x7f333815a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:44.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.725+0000 7f333e9d2700 1 -- 172.21.15.163:0/2231765044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f333806c0a0 msgr2=0x7f333815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:44.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.725+0000 7f333e9d2700 1 --2- 172.21.15.163:0/2231765044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f333806c0a0 0x7f333815a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:44.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.725+0000 7f333e9d2700 1 -- 172.21.15.163:0/2231765044 >> v2:172.21.15.181:3300/0 conn(0x7f333806bcd0 msgr2=0x7f33380caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:44.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.725+0000 7f333e9d2700 1 --2- 172.21.15.163:0/2231765044 >> v2:172.21.15.181:3300/0 conn(0x7f333806bcd0 0x7f33380caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:44.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.725+0000 7f333e9d2700 1 -- 172.21.15.163:0/2231765044 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333815ad10 con 0x7f333815df50 2023-10-09T06:35:44.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.725+0000 7f333e9d2700 1 --2- 172.21.15.163:0/2231765044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f333815df50 0x7f333815e3c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f33380691a0 tx=0x7f3330009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddbd285f6459fe0b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:44.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.725+0000 7f333d1cf700 1 -- 172.21.15.163:0/2231765044 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333000e070 con 0x7f333815df50 2023-10-09T06:35:44.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.725+0000 7f333d1cf700 1 -- 172.21.15.163:0/2231765044 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333000ba50 con 0x7f333815df50 2023-10-09T06:35:44.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.725+0000 7f333d1cf700 1 -- 172.21.15.163:0/2231765044 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33300092e0 con 0x7f333815df50 2023-10-09T06:35:44.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.726+0000 7f3340435700 1 -- 172.21.15.163:0/2231765044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f333815df50 msgr2=0x7f333815e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:44.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.726+0000 7f3340435700 1 --2- 172.21.15.163:0/2231765044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f333815df50 0x7f333815e3c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f33380691a0 tx=0x7f3330009840 comp rx=0 tx=0).stop 2023-10-09T06:35:44.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.726+0000 7f3340435700 1 -- 172.21.15.163:0/2231765044 shutdown_connections 2023-10-09T06:35:44.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.726+0000 7f3340435700 1 --2- 172.21.15.163:0/2231765044 >> v2:172.21.15.181:3300/0 conn(0x7f333806bcd0 0x7f33380caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:44.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.726+0000 7f3340435700 1 --2- 172.21.15.163:0/2231765044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f333815df50 0x7f333815e3c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:44.730 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.726+0000 7f3340435700 1 --2- 172.21.15.163:0/2231765044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f333806c0a0 0x7f333815a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:44.730 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.726+0000 7f3340435700 1 -- 172.21.15.163:0/2231765044 >> 172.21.15.163:0/2231765044 conn(0x7f33380ce600 msgr2=0x7f33380cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:44.730 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.726+0000 7f3340435700 1 -- 172.21.15.163:0/2231765044 shutdown_connections 2023-10-09T06:35:44.730 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.726+0000 7f3340435700 1 -- 172.21.15.163:0/2231765044 wait complete. 2023-10-09T06:35:44.730 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.727+0000 7f3340435700 1 Processor -- start 2023-10-09T06:35:44.730 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.727+0000 7f3340435700 1 -- start start 2023-10-09T06:35:44.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.728+0000 7f3340435700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f333806bcd0 0x7f33380b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:44.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.728+0000 7f3340435700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f333806c0a0 0x7f3338156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:44.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.728+0000 7f3340435700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f333815df50 0x7f3338169ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:44.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.728+0000 7f3340435700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33380b4120 con 0x7f333806bcd0 2023-10-09T06:35:44.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.728+0000 7f3340435700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33380c6b90 con 0x7f333815df50 2023-10-09T06:35:44.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.728+0000 7f3340435700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33380c6e70 con 0x7f333806c0a0 2023-10-09T06:35:44.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.728+0000 7f333e1d1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f333806bcd0 0x7f33380b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:44.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.728+0000 7f333e1d1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f333806bcd0 0x7f33380b13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:37810/0 (socket says 172.21.15.163:37810) 2023-10-09T06:35:44.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.728+0000 7f333e1d1700 1 -- 172.21.15.163:0/3940099704 learned_addr learned my addr 172.21.15.163:0/3940099704 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:44.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.728+0000 7f333e9d2700 1 --2- 172.21.15.163:0/3940099704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f333815df50 0x7f3338169ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:44.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.728+0000 7f333d9d0700 1 --2- 172.21.15.163:0/3940099704 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f333806c0a0 0x7f3338156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:44.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.728+0000 7f333e1d1700 1 -- 172.21.15.163:0/3940099704 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f333806c0a0 msgr2=0x7f3338156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:44.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.728+0000 7f333e1d1700 1 --2- 172.21.15.163:0/3940099704 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f333806c0a0 0x7f3338156320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:44.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.728+0000 7f333e1d1700 1 -- 172.21.15.163:0/3940099704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f333815df50 msgr2=0x7f3338169ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:44.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.728+0000 7f333e1d1700 1 --2- 172.21.15.163:0/3940099704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f333815df50 0x7f3338169ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:44.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.728+0000 7f333e1d1700 1 -- 172.21.15.163:0/3940099704 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3330009ae0 con 0x7f333806bcd0 2023-10-09T06:35:44.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.728+0000 7f333e1d1700 1 --2- 172.21.15.163:0/3940099704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f333806bcd0 0x7f33380b13a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f333400d860 tx=0x7f3334008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:44.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.728+0000 7f332e7fc700 1 -- 172.21.15.163:0/3940099704 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3334009610 con 0x7f333806bcd0 2023-10-09T06:35:44.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.728+0000 7f332e7fc700 1 -- 172.21.15.163:0/3940099704 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3334009770 con 0x7f333806bcd0 2023-10-09T06:35:44.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.728+0000 7f3340435700 1 -- 172.21.15.163:0/3940099704 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33380c7150 con 0x7f333806bcd0 2023-10-09T06:35:44.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.728+0000 7f332e7fc700 1 -- 172.21.15.163:0/3940099704 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3334009b30 con 0x7f333806bcd0 2023-10-09T06:35:44.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.728+0000 7f3340435700 1 -- 172.21.15.163:0/3940099704 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33380c7710 con 0x7f333806bcd0 2023-10-09T06:35:44.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.729+0000 7f3340435700 1 -- 172.21.15.163:0/3940099704 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33380c7a80 con 0x7f333806bcd0 2023-10-09T06:35:44.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.730+0000 7f332e7fc700 1 -- 172.21.15.163:0/3940099704 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f333401e460 con 0x7f333806bcd0 2023-10-09T06:35:44.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.730+0000 7f332e7fc700 1 --2- 172.21.15.163:0/3940099704 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f331c067170 0x7f331c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:44.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.730+0000 7f332e7fc700 1 -- 172.21.15.163:0/3940099704 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f3334084bc0 con 0x7f333806bcd0 2023-10-09T06:35:44.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.730+0000 7f333d9d0700 1 --2- 172.21.15.163:0/3940099704 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f331c067170 0x7f331c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:44.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.731+0000 7f333d9d0700 1 --2- 172.21.15.163:0/3940099704 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f331c067170 0x7f331c069620 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f333816d1c0 tx=0x7f3328008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:44.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.734+0000 7f332e7fc700 1 -- 172.21.15.163:0/3940099704 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3334051ea0 con 0x7f333806bcd0 2023-10-09T06:35:44.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.916+0000 7f3340435700 1 -- 172.21.15.163:0/3940099704 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f333804c3c0 con 0x7f333806bcd0 2023-10-09T06:35:44.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.920+0000 7f332e7fc700 1 -- 172.21.15.163:0/3940099704 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f33340554c0 con 0x7f333806bcd0 2023-10-09T06:35:44.923 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.922+0000 7f3340435700 1 -- 172.21.15.163:0/3940099704 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f331c067170 msgr2=0x7f331c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:44.923 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.922+0000 7f3340435700 1 --2- 172.21.15.163:0/3940099704 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f331c067170 0x7f331c069620 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f333816d1c0 tx=0x7f3328008040 comp rx=0 tx=0).stop 2023-10-09T06:35:44.923 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.922+0000 7f3340435700 1 -- 172.21.15.163:0/3940099704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f333806bcd0 msgr2=0x7f33380b13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:44.923 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.922+0000 7f3340435700 1 --2- 172.21.15.163:0/3940099704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f333806bcd0 0x7f33380b13a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f333400d860 tx=0x7f3334008040 comp rx=0 tx=0).stop 2023-10-09T06:35:44.923 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.922+0000 7f3340435700 1 -- 172.21.15.163:0/3940099704 shutdown_connections 2023-10-09T06:35:44.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.922+0000 7f3340435700 1 --2- 172.21.15.163:0/3940099704 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f331c067170 0x7f331c069620 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:44.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.922+0000 7f3340435700 1 --2- 172.21.15.163:0/3940099704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f333806bcd0 0x7f33380b13a0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:44.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.922+0000 7f3340435700 1 --2- 172.21.15.163:0/3940099704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f333815df50 0x7f3338169ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:44.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.922+0000 7f3340435700 1 --2- 172.21.15.163:0/3940099704 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f333806c0a0 0x7f3338156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:44.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.922+0000 7f3340435700 1 -- 172.21.15.163:0/3940099704 >> 172.21.15.163:0/3940099704 conn(0x7f33380ce600 msgr2=0x7f3338157200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:44.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.922+0000 7f3340435700 1 -- 172.21.15.163:0/3940099704 shutdown_connections 2023-10-09T06:35:44.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:44.922+0000 7f3340435700 1 -- 172.21.15.163:0/3940099704 wait complete. 2023-10-09T06:35:44.935 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions' 2023-10-09T06:35:45.049 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.048+0000 7fec22082700 1 Processor -- start 2023-10-09T06:35:45.049 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.049+0000 7fec22082700 1 -- start start 2023-10-09T06:35:45.052 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.049+0000 7fec22082700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fec1c1597f0 0x7fec1c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:45.052 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.049+0000 7fec22082700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec1c0b11d0 con 0x7fec1c159420 2023-10-09T06:35:45.052 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.049+0000 7fec22082700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec1c0b14b0 con 0x7fec1c15d230 2023-10-09T06:35:45.052 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.049+0000 7fec22082700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec1c0b1790 con 0x7fec1c1597f0 2023-10-09T06:35:45.052 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.049+0000 7fec2087f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fec1c1597f0 0x7fec1c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:45.052 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.050+0000 7fec21080700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fec1c159420 0x7fec1c0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:58144/0 (socket says 172.21.15.163:58144) 2023-10-09T06:35:45.052 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.050+0000 7fec21080700 1 -- 172.21.15.163:0/899024263 learned_addr learned my addr 172.21.15.163:0/899024263 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:35:45.052 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.050+0000 7fec2087f700 1 -- 172.21.15.163:0/899024263 >> v1:172.21.15.181:6789/0 conn(0x7fec1c15d230 legacy=0x7fec1c15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:45.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.050+0000 7fec2087f700 1 -- 172.21.15.163:0/899024263 >> v1:172.21.15.163:6789/0 conn(0x7fec1c159420 legacy=0x7fec1c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:45.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.050+0000 7fec2087f700 1 -- 172.21.15.163:0/899024263 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1c0b1a70 con 0x7fec1c1597f0 2023-10-09T06:35:45.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.050+0000 7fec13fff700 1 -- 172.21.15.163:0/899024263 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2480586480 0 0) 0x7fec1c0b11d0 con 0x7fec1c159420 2023-10-09T06:35:45.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.050+0000 7fec2087f700 1 --2- 172.21.15.163:0/899024263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec1c1597f0 0x7fec1c0b2270 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fec0c0098e0 tx=0x7fec0c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c2305e40e620f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:45.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.050+0000 7fec13fff700 1 -- 172.21.15.163:0/899024263 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c009e80 con 0x7fec1c1597f0 2023-10-09T06:35:45.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.050+0000 7fec13fff700 1 -- 172.21.15.163:0/899024263 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec0c00b820 con 0x7fec1c1597f0 2023-10-09T06:35:45.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.050+0000 7fec13fff700 1 -- 172.21.15.163:0/899024263 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c00bba0 con 0x7fec1c1597f0 2023-10-09T06:35:45.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.051+0000 7fec22082700 1 -- 172.21.15.163:0/899024263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec1c1597f0 msgr2=0x7fec1c0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:45.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.051+0000 7fec22082700 1 --2- 172.21.15.163:0/899024263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec1c1597f0 0x7fec1c0b2270 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fec0c0098e0 tx=0x7fec0c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:35:45.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.051+0000 7fec22082700 1 -- 172.21.15.163:0/899024263 shutdown_connections 2023-10-09T06:35:45.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.051+0000 7fec22082700 1 --2- 172.21.15.163:0/899024263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec1c1597f0 0x7fec1c0b2270 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:45.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.051+0000 7fec22082700 1 -- 172.21.15.163:0/899024263 >> 172.21.15.163:0/899024263 conn(0x7fec1c0d0660 msgr2=0x7fec1c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:45.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.051+0000 7fec22082700 1 -- 172.21.15.163:0/899024263 shutdown_connections 2023-10-09T06:35:45.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.051+0000 7fec22082700 1 -- 172.21.15.163:0/899024263 wait complete. 2023-10-09T06:35:45.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.052+0000 7fec22082700 1 Processor -- start 2023-10-09T06:35:45.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.052+0000 7fec22082700 1 -- start start 2023-10-09T06:35:45.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.053+0000 7fec22082700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec1c159420 0x7fec1c1541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:45.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.053+0000 7fec22082700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec1c1597f0 0x7fec1c0c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:45.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.053+0000 7fec22082700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec1c15d230 0x7fec1c069880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:45.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.053+0000 7fec22082700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c0c6c60 con 0x7fec1c1597f0 2023-10-09T06:35:45.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.053+0000 7fec22082700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c0c5de0 con 0x7fec1c15d230 2023-10-09T06:35:45.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.053+0000 7fec22082700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c0c60c0 con 0x7fec1c159420 2023-10-09T06:35:45.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.053+0000 7fec2087f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec1c1597f0 0x7fec1c0c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:45.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.053+0000 7fec2087f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec1c1597f0 0x7fec1c0c5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:37830/0 (socket says 172.21.15.163:37830) 2023-10-09T06:35:45.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.053+0000 7fec2087f700 1 -- 172.21.15.163:0/4079117248 learned_addr learned my addr 172.21.15.163:0/4079117248 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:45.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.053+0000 7fec21881700 1 --2- 172.21.15.163:0/4079117248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec1c15d230 0x7fec1c069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:45.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.053+0000 7fec21080700 1 --2- 172.21.15.163:0/4079117248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec1c159420 0x7fec1c1541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:45.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.053+0000 7fec2087f700 1 -- 172.21.15.163:0/4079117248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec1c159420 msgr2=0x7fec1c1541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:45.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.053+0000 7fec2087f700 1 --2- 172.21.15.163:0/4079117248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec1c159420 0x7fec1c1541d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:45.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.053+0000 7fec2087f700 1 -- 172.21.15.163:0/4079117248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec1c15d230 msgr2=0x7fec1c069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:45.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.053+0000 7fec2087f700 1 --2- 172.21.15.163:0/4079117248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec1c15d230 0x7fec1c069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:45.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.053+0000 7fec2087f700 1 -- 172.21.15.163:0/4079117248 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec0c009420 con 0x7fec1c1597f0 2023-10-09T06:35:45.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.053+0000 7fec21881700 1 --2- 172.21.15.163:0/4079117248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec1c15d230 0x7fec1c069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:35:45.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.053+0000 7fec2087f700 1 --2- 172.21.15.163:0/4079117248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec1c1597f0 0x7fec1c0c5400 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fec0c0030c0 tx=0x7fec0c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:45.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.053+0000 7fec117fa700 1 -- 172.21.15.163:0/4079117248 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c00ba10 con 0x7fec1c1597f0 2023-10-09T06:35:45.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.053+0000 7fec117fa700 1 -- 172.21.15.163:0/4079117248 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec0c020070 con 0x7fec1c1597f0 2023-10-09T06:35:45.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.053+0000 7fec22082700 1 -- 172.21.15.163:0/4079117248 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec1c0c63a0 con 0x7fec1c1597f0 2023-10-09T06:35:45.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.053+0000 7fec117fa700 1 -- 172.21.15.163:0/4079117248 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0c01b6a0 con 0x7fec1c1597f0 2023-10-09T06:35:45.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.053+0000 7fec22082700 1 -- 172.21.15.163:0/4079117248 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec1c16af90 con 0x7fec1c1597f0 2023-10-09T06:35:45.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.054+0000 7fec22082700 1 -- 172.21.15.163:0/4079117248 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec1c04c3c0 con 0x7fec1c1597f0 2023-10-09T06:35:45.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.055+0000 7fec117fa700 1 -- 172.21.15.163:0/4079117248 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fec0c022030 con 0x7fec1c1597f0 2023-10-09T06:35:45.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.055+0000 7fec117fa700 1 --2- 172.21.15.163:0/4079117248 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fec00067170 0x7fec00069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:45.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.055+0000 7fec117fa700 1 -- 172.21.15.163:0/4079117248 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fec0c086610 con 0x7fec1c1597f0 2023-10-09T06:35:45.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.055+0000 7fec21080700 1 --2- 172.21.15.163:0/4079117248 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fec00067170 0x7fec00069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:45.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.056+0000 7fec21080700 1 --2- 172.21.15.163:0/4079117248 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fec00067170 0x7fec00069620 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fec18001560 tx=0x7fec18009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:45.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.059+0000 7fec117fa700 1 -- 172.21.15.163:0/4079117248 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec0c0538d0 con 0x7fec1c1597f0 2023-10-09T06:35:45.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.316+0000 7fec22082700 1 -- 172.21.15.163:0/4079117248 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions"]} v 0) v1 -- 0x7fec1c0c6da0 con 0x7fec1c1597f0 2023-10-09T06:35:45.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.585+0000 7fec117fa700 1 -- 172.21.15.163:0/4079117248 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions"]}]=0 v238) v1 ==== 158+0+0 (secure 0 0 0) 0x7fec0c056ef0 con 0x7fec1c1597f0 2023-10-09T06:35:45.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.588+0000 7fec22082700 1 -- 172.21.15.163:0/4079117248 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fec00067170 msgr2=0x7fec00069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:45.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.588+0000 7fec22082700 1 --2- 172.21.15.163:0/4079117248 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fec00067170 0x7fec00069620 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fec18001560 tx=0x7fec18009040 comp rx=0 tx=0).stop 2023-10-09T06:35:45.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.588+0000 7fec22082700 1 -- 172.21.15.163:0/4079117248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec1c1597f0 msgr2=0x7fec1c0c5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:45.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.588+0000 7fec22082700 1 --2- 172.21.15.163:0/4079117248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec1c1597f0 0x7fec1c0c5400 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fec0c0030c0 tx=0x7fec0c016000 comp rx=0 tx=0).stop 2023-10-09T06:35:45.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.588+0000 7fec22082700 1 -- 172.21.15.163:0/4079117248 shutdown_connections 2023-10-09T06:35:45.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.588+0000 7fec22082700 1 --2- 172.21.15.163:0/4079117248 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fec00067170 0x7fec00069620 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:45.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.588+0000 7fec22082700 1 --2- 172.21.15.163:0/4079117248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec1c15d230 0x7fec1c069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:45.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.588+0000 7fec22082700 1 --2- 172.21.15.163:0/4079117248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec1c159420 0x7fec1c1541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:45.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.588+0000 7fec22082700 1 --2- 172.21.15.163:0/4079117248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec1c1597f0 0x7fec1c0c5400 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:45.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.588+0000 7fec22082700 1 -- 172.21.15.163:0/4079117248 >> 172.21.15.163:0/4079117248 conn(0x7fec1c0d0660 msgr2=0x7fec1c155080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:45.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.588+0000 7fec22082700 1 -- 172.21.15.163:0/4079117248 shutdown_connections 2023-10-09T06:35:45.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.588+0000 7fec22082700 1 -- 172.21.15.163:0/4079117248 wait complete. 2023-10-09T06:35:45.601 INFO:tasks.cephfs_test_runner:Tests the pending_subvolume_deletions in 'fs volume info' command ... ok 2023-10-09T06:35:45.601 INFO:tasks.cephfs_test_runner:Starting test: test_volume_info_with_human_readable_flag (tasks.cephfs.test_volumes.TestVolumes) Tests the 'fs volume info --human_readable' command 2023-10-09T06:35:45.601 INFO:tasks.cephfs_test_runner:test_volume_info_with_human_readable_flag (tasks.cephfs.test_volumes.TestVolumes) 2023-10-09T06:35:45.602 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag' 2023-10-09T06:35:45.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.716+0000 7ff84357a700 1 Processor -- start 2023-10-09T06:35:45.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.716+0000 7ff84357a700 1 -- start start 2023-10-09T06:35:45.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.717+0000 7ff84357a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff83c1597f0 0x7ff83c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:45.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.717+0000 7ff84357a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff83c15d230 0x7ff83c15d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:45.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.717+0000 7ff84357a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff83c15dbb0 con 0x7ff83c159420 2023-10-09T06:35:45.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.717+0000 7ff84357a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff83c0b11d0 con 0x7ff83c1597f0 2023-10-09T06:35:45.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.717+0000 7ff84357a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff83c0b14b0 con 0x7ff83c15d230 2023-10-09T06:35:45.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.717+0000 7ff841d77700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff83c1597f0 0x7ff83c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:45.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.718+0000 7ff841d77700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff83c1597f0 0x7ff83c0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:37218/0 (socket says 172.21.15.163:37218) 2023-10-09T06:35:45.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.718+0000 7ff841d77700 1 -- 172.21.15.163:0/2321943626 learned_addr learned my addr 172.21.15.163:0/2321943626 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:45.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.718+0000 7ff842578700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff83c159420 0x7ff83c0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:47722/0 (socket says 172.21.15.163:47722) 2023-10-09T06:35:45.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.718+0000 7ff842d79700 1 --2- 172.21.15.163:0/2321943626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff83c15d230 0x7ff83c15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:45.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.718+0000 7ff841d77700 1 -- 172.21.15.163:0/2321943626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff83c15d230 msgr2=0x7ff83c15d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:45.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.718+0000 7ff841d77700 1 --2- 172.21.15.163:0/2321943626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff83c15d230 0x7ff83c15d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:45.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.718+0000 7ff841d77700 1 -- 172.21.15.163:0/2321943626 >> v1:172.21.15.181:6789/0 conn(0x7ff83c159420 legacy=0x7ff83c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:45.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.718+0000 7ff841d77700 1 -- 172.21.15.163:0/2321943626 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff83c0b1790 con 0x7ff83c1597f0 2023-10-09T06:35:45.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.718+0000 7ff841d77700 1 --2- 172.21.15.163:0/2321943626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff83c1597f0 0x7ff83c0b2270 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff82c0099b0 tx=0x7ff82c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b43ee7324a86582 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:45.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.718+0000 7ff841576700 1 -- 172.21.15.163:0/2321943626 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c009ea0 con 0x7ff83c1597f0 2023-10-09T06:35:45.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.718+0000 7ff841576700 1 -- 172.21.15.163:0/2321943626 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82c00b860 con 0x7ff83c1597f0 2023-10-09T06:35:45.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.718+0000 7ff841576700 1 -- 172.21.15.163:0/2321943626 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c00bc00 con 0x7ff83c1597f0 2023-10-09T06:35:45.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.719+0000 7ff84357a700 1 -- 172.21.15.163:0/2321943626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff83c1597f0 msgr2=0x7ff83c0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:45.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.719+0000 7ff84357a700 1 --2- 172.21.15.163:0/2321943626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff83c1597f0 0x7ff83c0b2270 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff82c0099b0 tx=0x7ff82c009250 comp rx=0 tx=0).stop 2023-10-09T06:35:45.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.719+0000 7ff84357a700 1 -- 172.21.15.163:0/2321943626 shutdown_connections 2023-10-09T06:35:45.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.719+0000 7ff84357a700 1 --2- 172.21.15.163:0/2321943626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff83c15d230 0x7ff83c15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:45.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.719+0000 7ff84357a700 1 --2- 172.21.15.163:0/2321943626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff83c1597f0 0x7ff83c0b2270 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:45.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.719+0000 7ff84357a700 1 -- 172.21.15.163:0/2321943626 >> 172.21.15.163:0/2321943626 conn(0x7ff83c0d0660 msgr2=0x7ff83c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:45.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.719+0000 7ff84357a700 1 -- 172.21.15.163:0/2321943626 shutdown_connections 2023-10-09T06:35:45.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.719+0000 7ff84357a700 1 -- 172.21.15.163:0/2321943626 wait complete. 2023-10-09T06:35:45.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.720+0000 7ff84357a700 1 Processor -- start 2023-10-09T06:35:45.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.720+0000 7ff84357a700 1 -- start start 2023-10-09T06:35:45.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.721+0000 7ff84357a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff83c159420 0x7ff83c15a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:45.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.721+0000 7ff84357a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff83c1597f0 0x7ff83c152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:45.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.721+0000 7ff84357a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff83c15d230 0x7ff83c14e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:45.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.721+0000 7ff84357a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83c155810 con 0x7ff83c159420 2023-10-09T06:35:45.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.721+0000 7ff84357a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83c152dc0 con 0x7ff83c1597f0 2023-10-09T06:35:45.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.721+0000 7ff84357a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff83c14ead0 con 0x7ff83c15d230 2023-10-09T06:35:45.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.721+0000 7ff842578700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff83c159420 0x7ff83c15a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:45.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.721+0000 7ff842578700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff83c159420 0x7ff83c15a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:37836/0 (socket says 172.21.15.163:37836) 2023-10-09T06:35:45.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.721+0000 7ff842578700 1 -- 172.21.15.163:0/3981889174 learned_addr learned my addr 172.21.15.163:0/3981889174 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:45.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.721+0000 7ff841d77700 1 --2- 172.21.15.163:0/3981889174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff83c1597f0 0x7ff83c152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:45.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.721+0000 7ff842d79700 1 --2- 172.21.15.163:0/3981889174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff83c15d230 0x7ff83c14e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:45.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.721+0000 7ff842578700 1 -- 172.21.15.163:0/3981889174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff83c15d230 msgr2=0x7ff83c14e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:45.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.721+0000 7ff842578700 1 --2- 172.21.15.163:0/3981889174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff83c15d230 0x7ff83c14e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:45.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.721+0000 7ff842578700 1 -- 172.21.15.163:0/3981889174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff83c1597f0 msgr2=0x7ff83c152880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:45.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.721+0000 7ff842578700 1 --2- 172.21.15.163:0/3981889174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff83c1597f0 0x7ff83c152880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:45.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.721+0000 7ff842578700 1 -- 172.21.15.163:0/3981889174 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82c0094f0 con 0x7ff83c159420 2023-10-09T06:35:45.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.721+0000 7ff841d77700 1 --2- 172.21.15.163:0/3981889174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff83c1597f0 0x7ff83c152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:35:45.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.721+0000 7ff842578700 1 --2- 172.21.15.163:0/3981889174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff83c159420 0x7ff83c15a200 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff83800cd80 tx=0x7ff838001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:45.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.721+0000 7ff8327fc700 1 -- 172.21.15.163:0/3981889174 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff838005df0 con 0x7ff83c159420 2023-10-09T06:35:45.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.721+0000 7ff8327fc700 1 -- 172.21.15.163:0/3981889174 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff83800a8b0 con 0x7ff83c159420 2023-10-09T06:35:45.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.721+0000 7ff84357a700 1 -- 172.21.15.163:0/3981889174 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff83c155a60 con 0x7ff83c159420 2023-10-09T06:35:45.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.721+0000 7ff8327fc700 1 -- 172.21.15.163:0/3981889174 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83800ac70 con 0x7ff83c159420 2023-10-09T06:35:45.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.721+0000 7ff84357a700 1 -- 172.21.15.163:0/3981889174 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff83c156020 con 0x7ff83c159420 2023-10-09T06:35:45.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.722+0000 7ff84357a700 1 -- 172.21.15.163:0/3981889174 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff83c04c3c0 con 0x7ff83c159420 2023-10-09T06:35:45.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.723+0000 7ff8327fc700 1 -- 172.21.15.163:0/3981889174 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7ff838007300 con 0x7ff83c159420 2023-10-09T06:35:45.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.723+0000 7ff8327fc700 1 --2- 172.21.15.163:0/3981889174 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff820067170 0x7ff820069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:45.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.723+0000 7ff8327fc700 1 -- 172.21.15.163:0/3981889174 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff838084600 con 0x7ff83c159420 2023-10-09T06:35:45.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.723+0000 7ff841d77700 1 --2- 172.21.15.163:0/3981889174 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff820067170 0x7ff820069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:45.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.724+0000 7ff841d77700 1 --2- 172.21.15.163:0/3981889174 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff820067170 0x7ff820069620 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff82c0099b0 tx=0x7ff82c017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:45.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.727+0000 7ff8327fc700 1 -- 172.21.15.163:0/3981889174 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff8380518e0 con 0x7ff83c159420 2023-10-09T06:35:45.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:45.988+0000 7ff84357a700 1 -- 172.21.15.163:0/3981889174 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag"]} v 0) v1 -- 0x7ff83c0b1a70 con 0x7ff83c159420 2023-10-09T06:35:46.592 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.592+0000 7ff8327fc700 1 -- 172.21.15.163:0/3981889174 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag"]}]=0 v239) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff838054f00 con 0x7ff83c159420 2023-10-09T06:35:46.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.594+0000 7ff84357a700 1 -- 172.21.15.163:0/3981889174 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff820067170 msgr2=0x7ff820069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:46.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.594+0000 7ff84357a700 1 --2- 172.21.15.163:0/3981889174 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff820067170 0x7ff820069620 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff82c0099b0 tx=0x7ff82c017000 comp rx=0 tx=0).stop 2023-10-09T06:35:46.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.594+0000 7ff84357a700 1 -- 172.21.15.163:0/3981889174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff83c159420 msgr2=0x7ff83c15a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:46.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.594+0000 7ff84357a700 1 --2- 172.21.15.163:0/3981889174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff83c159420 0x7ff83c15a200 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff83800cd80 tx=0x7ff838001b90 comp rx=0 tx=0).stop 2023-10-09T06:35:46.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.594+0000 7ff84357a700 1 -- 172.21.15.163:0/3981889174 shutdown_connections 2023-10-09T06:35:46.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.594+0000 7ff84357a700 1 --2- 172.21.15.163:0/3981889174 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff820067170 0x7ff820069620 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:46.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.594+0000 7ff84357a700 1 --2- 172.21.15.163:0/3981889174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff83c159420 0x7ff83c15a200 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:46.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.594+0000 7ff84357a700 1 --2- 172.21.15.163:0/3981889174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff83c15d230 0x7ff83c14e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:46.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.594+0000 7ff84357a700 1 --2- 172.21.15.163:0/3981889174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff83c1597f0 0x7ff83c152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:46.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.594+0000 7ff84357a700 1 -- 172.21.15.163:0/3981889174 >> 172.21.15.163:0/3981889174 conn(0x7ff83c0d0660 msgr2=0x7ff83c0d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:46.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.594+0000 7ff84357a700 1 -- 172.21.15.163:0/3981889174 shutdown_connections 2023-10-09T06:35:46.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.594+0000 7ff84357a700 1 -- 172.21.15.163:0/3981889174 wait complete. 2023-10-09T06:35:46.607 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-09T06:35:46.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.724+0000 7f437d3e0700 1 Processor -- start 2023-10-09T06:35:46.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.725+0000 7f437d3e0700 1 -- start start 2023-10-09T06:35:46.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.725+0000 7f437d3e0700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4378159c60 0x7f437815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:46.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.725+0000 7f437d3e0700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f437815a160 con 0x7f437815db10 2023-10-09T06:35:46.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.725+0000 7f437d3e0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43780d13a0 con 0x7f4378159890 2023-10-09T06:35:46.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.725+0000 7f437d3e0700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f437815df50 con 0x7f4378159c60 2023-10-09T06:35:46.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.726+0000 7f436e7fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4378159c60 0x7f437815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:46.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.726+0000 7f43777fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f437815db10 0x7f4378069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:58158/0 (socket says 172.21.15.163:58158) 2023-10-09T06:35:46.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.726+0000 7f43777fe700 1 -- 172.21.15.163:0/3651851897 learned_addr learned my addr 172.21.15.163:0/3651851897 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:35:46.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.726+0000 7f436e7fc700 1 -- 172.21.15.163:0/3651851897 >> v1:172.21.15.181:6789/0 conn(0x7f4378159890 legacy=0x7f437815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:46.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.726+0000 7f436e7fc700 1 -- 172.21.15.163:0/3651851897 >> v1:172.21.15.163:6789/0 conn(0x7f437815db10 legacy=0x7f4378069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:46.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.726+0000 7f436e7fc700 1 -- 172.21.15.163:0/3651851897 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f437815e140 con 0x7f4378159c60 2023-10-09T06:35:46.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.726+0000 7f43767fc700 1 -- 172.21.15.163:0/3651851897 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3351401792 0 0) 0x7f437815a160 con 0x7f437815db10 2023-10-09T06:35:46.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.726+0000 7f436e7fc700 1 --2- 172.21.15.163:0/3651851897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4378159c60 0x7f437815d5d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f43600098e0 tx=0x7f43600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b059f0f0004d3775 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:46.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.726+0000 7f43767fc700 1 -- 172.21.15.163:0/3651851897 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360009e80 con 0x7f4378159c60 2023-10-09T06:35:46.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.726+0000 7f43767fc700 1 -- 172.21.15.163:0/3651851897 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436000b860 con 0x7f4378159c60 2023-10-09T06:35:46.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.726+0000 7f43767fc700 1 -- 172.21.15.163:0/3651851897 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436000bbe0 con 0x7f4378159c60 2023-10-09T06:35:46.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.727+0000 7f437d3e0700 1 -- 172.21.15.163:0/3651851897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4378159c60 msgr2=0x7f437815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:46.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.727+0000 7f437d3e0700 1 --2- 172.21.15.163:0/3651851897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4378159c60 0x7f437815d5d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f43600098e0 tx=0x7f43600091a0 comp rx=0 tx=0).stop 2023-10-09T06:35:46.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.727+0000 7f437d3e0700 1 -- 172.21.15.163:0/3651851897 shutdown_connections 2023-10-09T06:35:46.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.727+0000 7f437d3e0700 1 --2- 172.21.15.163:0/3651851897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4378159c60 0x7f437815d5d0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:46.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.727+0000 7f437d3e0700 1 -- 172.21.15.163:0/3651851897 >> 172.21.15.163:0/3651851897 conn(0x7f4378068e20 msgr2=0x7f43780d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:46.730 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.727+0000 7f437d3e0700 1 -- 172.21.15.163:0/3651851897 shutdown_connections 2023-10-09T06:35:46.730 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.727+0000 7f437d3e0700 1 -- 172.21.15.163:0/3651851897 wait complete. 2023-10-09T06:35:46.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.728+0000 7f437d3e0700 1 Processor -- start 2023-10-09T06:35:46.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.728+0000 7f437d3e0700 1 -- start start 2023-10-09T06:35:46.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.729+0000 7f437d3e0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4378159890 0x7f43780cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:46.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.729+0000 7f437d3e0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f437815db10 0x7f437815b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:46.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.729+0000 7f437d3e0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43780ce000 0x7f437815ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:46.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.729+0000 7f437d3e0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43780b1490 con 0x7f43780ce000 2023-10-09T06:35:46.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.729+0000 7f437d3e0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43780b0810 con 0x7f437815db10 2023-10-09T06:35:46.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.729+0000 7f437d3e0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43780b0af0 con 0x7f4378159890 2023-10-09T06:35:46.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.729+0000 7f43777fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43780ce000 0x7f437815ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:46.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.729+0000 7f43777fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43780ce000 0x7f437815ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:37854/0 (socket says 172.21.15.163:37854) 2023-10-09T06:35:46.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.729+0000 7f43777fe700 1 -- 172.21.15.163:0/4191868844 learned_addr learned my addr 172.21.15.163:0/4191868844 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:46.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.729+0000 7f436e7fc700 1 --2- 172.21.15.163:0/4191868844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f437815db10 0x7f437815b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:46.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.729+0000 7f4376ffd700 1 --2- 172.21.15.163:0/4191868844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4378159890 0x7f43780cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:46.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.729+0000 7f43777fe700 1 -- 172.21.15.163:0/4191868844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4378159890 msgr2=0x7f43780cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:46.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.729+0000 7f43777fe700 1 --2- 172.21.15.163:0/4191868844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4378159890 0x7f43780cdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:46.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.729+0000 7f43777fe700 1 -- 172.21.15.163:0/4191868844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f437815db10 msgr2=0x7f437815b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:46.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.729+0000 7f43777fe700 1 --2- 172.21.15.163:0/4191868844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f437815db10 0x7f437815b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:46.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.729+0000 7f43777fe700 1 -- 172.21.15.163:0/4191868844 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4360009420 con 0x7f43780ce000 2023-10-09T06:35:46.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.729+0000 7f436e7fc700 1 --2- 172.21.15.163:0/4191868844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f437815db10 0x7f437815b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:35:46.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.729+0000 7f4376ffd700 1 --2- 172.21.15.163:0/4191868844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4378159890 0x7f43780cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:35:46.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.729+0000 7f43777fe700 1 --2- 172.21.15.163:0/4191868844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43780ce000 0x7f437815ba10 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f437000cbf0 tx=0x7f43700019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:46.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.730+0000 7f436f7fe700 1 -- 172.21.15.163:0/4191868844 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437000a9c0 con 0x7f43780ce000 2023-10-09T06:35:46.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.730+0000 7f436f7fe700 1 -- 172.21.15.163:0/4191868844 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f437000ab40 con 0x7f43780ce000 2023-10-09T06:35:46.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.730+0000 7f437d3e0700 1 -- 172.21.15.163:0/4191868844 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43780b0dd0 con 0x7f43780ce000 2023-10-09T06:35:46.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.730+0000 7f436f7fe700 1 -- 172.21.15.163:0/4191868844 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43700070f0 con 0x7f43780ce000 2023-10-09T06:35:46.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.730+0000 7f437d3e0700 1 -- 172.21.15.163:0/4191868844 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43780b5fd0 con 0x7f43780ce000 2023-10-09T06:35:46.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.731+0000 7f437d3e0700 1 -- 172.21.15.163:0/4191868844 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43780b15d0 con 0x7f43780ce000 2023-10-09T06:35:46.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.731+0000 7f436f7fe700 1 -- 172.21.15.163:0/4191868844 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f4370007520 con 0x7f43780ce000 2023-10-09T06:35:46.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.731+0000 7f436f7fe700 1 --2- 172.21.15.163:0/4191868844 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f43580671f0 0x7f43580696a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:46.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.731+0000 7f436f7fe700 1 -- 172.21.15.163:0/4191868844 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f4370084940 con 0x7f43780ce000 2023-10-09T06:35:46.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.732+0000 7f4376ffd700 1 --2- 172.21.15.163:0/4191868844 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f43580671f0 0x7f43580696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:46.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.732+0000 7f4376ffd700 1 --2- 172.21.15.163:0/4191868844 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f43580671f0 0x7f43580696a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4368001840 tx=0x7f4368009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:46.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.735+0000 7f436f7fe700 1 -- 172.21.15.163:0/4191868844 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4370051c00 con 0x7f43780ce000 2023-10-09T06:35:46.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.920+0000 7f437d3e0700 1 -- 172.21.15.163:0/4191868844 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f437804c3c0 con 0x7f43780ce000 2023-10-09T06:35:46.925 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.924+0000 7f436f7fe700 1 -- 172.21.15.163:0/4191868844 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4370055220 con 0x7f43780ce000 2023-10-09T06:35:46.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.926+0000 7f437d3e0700 1 -- 172.21.15.163:0/4191868844 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f43580671f0 msgr2=0x7f43580696a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:46.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.926+0000 7f437d3e0700 1 --2- 172.21.15.163:0/4191868844 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f43580671f0 0x7f43580696a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4368001840 tx=0x7f4368009040 comp rx=0 tx=0).stop 2023-10-09T06:35:46.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.926+0000 7f437d3e0700 1 -- 172.21.15.163:0/4191868844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43780ce000 msgr2=0x7f437815ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:46.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.926+0000 7f437d3e0700 1 --2- 172.21.15.163:0/4191868844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43780ce000 0x7f437815ba10 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f437000cbf0 tx=0x7f43700019b0 comp rx=0 tx=0).stop 2023-10-09T06:35:46.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.926+0000 7f437d3e0700 1 -- 172.21.15.163:0/4191868844 shutdown_connections 2023-10-09T06:35:46.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.926+0000 7f437d3e0700 1 --2- 172.21.15.163:0/4191868844 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f43580671f0 0x7f43580696a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:46.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.926+0000 7f437d3e0700 1 --2- 172.21.15.163:0/4191868844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43780ce000 0x7f437815ba10 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:46.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.926+0000 7f437d3e0700 1 --2- 172.21.15.163:0/4191868844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4378159890 0x7f43780cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:46.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.926+0000 7f437d3e0700 1 --2- 172.21.15.163:0/4191868844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f437815db10 0x7f437815b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:46.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.926+0000 7f437d3e0700 1 -- 172.21.15.163:0/4191868844 >> 172.21.15.163:0/4191868844 conn(0x7f4378068e20 msgr2=0x7f4378155560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:46.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.926+0000 7f437d3e0700 1 -- 172.21.15.163:0/4191868844 shutdown_connections 2023-10-09T06:35:46.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:46.926+0000 7f437d3e0700 1 -- 172.21.15.163:0/4191868844 wait complete. 2023-10-09T06:35:46.939 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-09T06:35:46.940 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:35:46.978 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:35:47.024 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-09T06:35:47.025 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-09T06:35:47.025 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:35:47.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.138+0000 7f721bee4700 1 Processor -- start 2023-10-09T06:35:47.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.139+0000 7f721bee4700 1 -- start start 2023-10-09T06:35:47.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.139+0000 7f721bee4700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f721406bcd0 0x7f72140caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:47.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.139+0000 7f721bee4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f721406c0a0 0x7f721415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:47.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.139+0000 7f721bee4700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f721415d7e0 con 0x7f721415df50 2023-10-09T06:35:47.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.139+0000 7f721bee4700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f721415dac0 con 0x7f721406bcd0 2023-10-09T06:35:47.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.139+0000 7f721bee4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f721415aa30 con 0x7f721406c0a0 2023-10-09T06:35:47.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.140+0000 7f7219c80700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f721406bcd0 0x7f72140caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:47.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.140+0000 7f721a481700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f721415df50 0x7f72140c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:58162/0 (socket says 172.21.15.163:58162) 2023-10-09T06:35:47.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.140+0000 7f721a481700 1 -- 172.21.15.163:0/3891199700 learned_addr learned my addr 172.21.15.163:0/3891199700 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:35:47.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.140+0000 7f7219c80700 1 -- 172.21.15.163:0/3891199700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f721406c0a0 msgr2=0x7f721415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:47.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.140+0000 7f7219c80700 1 --2- 172.21.15.163:0/3891199700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f721406c0a0 0x7f721415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:47.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.140+0000 7f7219c80700 1 -- 172.21.15.163:0/3891199700 >> v1:172.21.15.163:6789/0 conn(0x7f721415df50 legacy=0x7f72140c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:47.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.140+0000 7f7219c80700 1 -- 172.21.15.163:0/3891199700 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721415ad10 con 0x7f721406bcd0 2023-10-09T06:35:47.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.140+0000 7f7218c7e700 1 -- 172.21.15.163:0/3891199700 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 300037431 0 0) 0x7f721415d7e0 con 0x7f721415df50 2023-10-09T06:35:47.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.140+0000 7f7219c80700 1 --2- 172.21.15.163:0/3891199700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f721406bcd0 0x7f72140caa90 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f7210009e60 tx=0x7f7210009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54374f7825a8bdab server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:47.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.140+0000 7f7218c7e700 1 -- 172.21.15.163:0/3891199700 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721000e070 con 0x7f721406bcd0 2023-10-09T06:35:47.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.140+0000 7f7218c7e700 1 -- 172.21.15.163:0/3891199700 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721000bc30 con 0x7f721406bcd0 2023-10-09T06:35:47.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.140+0000 7f7218c7e700 1 -- 172.21.15.163:0/3891199700 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7210012430 con 0x7f721406bcd0 2023-10-09T06:35:47.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.141+0000 7f721bee4700 1 -- 172.21.15.163:0/3891199700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f721406bcd0 msgr2=0x7f72140caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:47.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.141+0000 7f721bee4700 1 --2- 172.21.15.163:0/3891199700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f721406bcd0 0x7f72140caa90 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f7210009e60 tx=0x7f7210009780 comp rx=0 tx=0).stop 2023-10-09T06:35:47.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.141+0000 7f721bee4700 1 -- 172.21.15.163:0/3891199700 shutdown_connections 2023-10-09T06:35:47.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.141+0000 7f721bee4700 1 --2- 172.21.15.163:0/3891199700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f721406bcd0 0x7f72140caa90 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:47.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.141+0000 7f721bee4700 1 --2- 172.21.15.163:0/3891199700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f721406c0a0 0x7f721415a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:47.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.141+0000 7f721bee4700 1 -- 172.21.15.163:0/3891199700 >> 172.21.15.163:0/3891199700 conn(0x7f72140ce600 msgr2=0x7f72140cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:47.144 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.141+0000 7f721bee4700 1 -- 172.21.15.163:0/3891199700 shutdown_connections 2023-10-09T06:35:47.144 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.141+0000 7f721bee4700 1 -- 172.21.15.163:0/3891199700 wait complete. 2023-10-09T06:35:47.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.142+0000 7f721bee4700 1 Processor -- start 2023-10-09T06:35:47.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.142+0000 7f721bee4700 1 -- start start 2023-10-09T06:35:47.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.142+0000 7f721bee4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f721406bcd0 0x7f721416ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:47.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.142+0000 7f721bee4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f721406c0a0 0x7f7214157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:47.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.142+0000 7f721bee4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f721415df50 0x7f7214154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:47.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.143+0000 7f721bee4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7214158ed0 con 0x7f721415df50 2023-10-09T06:35:47.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.143+0000 7f721bee4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72140b2890 con 0x7f721406c0a0 2023-10-09T06:35:47.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.143+0000 7f721bee4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7214158630 con 0x7f721406bcd0 2023-10-09T06:35:47.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.143+0000 7f721a481700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f721415df50 0x7f7214154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:47.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.143+0000 7f721a481700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f721415df50 0x7f7214154a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:37862/0 (socket says 172.21.15.163:37862) 2023-10-09T06:35:47.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.143+0000 7f721a481700 1 -- 172.21.15.163:0/3146583516 learned_addr learned my addr 172.21.15.163:0/3146583516 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:47.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.143+0000 7f7219c80700 1 --2- 172.21.15.163:0/3146583516 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f721406bcd0 0x7f721416ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:47.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.143+0000 7f721947f700 1 --2- 172.21.15.163:0/3146583516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f721406c0a0 0x7f7214157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:47.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.143+0000 7f721a481700 1 -- 172.21.15.163:0/3146583516 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f721406bcd0 msgr2=0x7f721416ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:47.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.143+0000 7f721a481700 1 --2- 172.21.15.163:0/3146583516 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f721406bcd0 0x7f721416ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:47.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.143+0000 7f721a481700 1 -- 172.21.15.163:0/3146583516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f721406c0a0 msgr2=0x7f7214157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:47.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.143+0000 7f721a481700 1 --2- 172.21.15.163:0/3146583516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f721406c0a0 0x7f7214157de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:47.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.143+0000 7f721a481700 1 -- 172.21.15.163:0/3146583516 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7210009a20 con 0x7f721415df50 2023-10-09T06:35:47.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.143+0000 7f721a481700 1 --2- 172.21.15.163:0/3146583516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f721415df50 0x7f7214154a60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f720c00c7e0 tx=0x7f720c007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:47.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.143+0000 7f7209ffb700 1 -- 172.21.15.163:0/3146583516 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c005c00 con 0x7f721415df50 2023-10-09T06:35:47.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.143+0000 7f7209ffb700 1 -- 172.21.15.163:0/3146583516 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720c005d80 con 0x7f721415df50 2023-10-09T06:35:47.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.143+0000 7f721bee4700 1 -- 172.21.15.163:0/3146583516 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7214158910 con 0x7f721415df50 2023-10-09T06:35:47.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.143+0000 7f7209ffb700 1 -- 172.21.15.163:0/3146583516 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c007320 con 0x7f721415df50 2023-10-09T06:35:47.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.144+0000 7f721bee4700 1 -- 172.21.15.163:0/3146583516 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7214158320 con 0x7f721415df50 2023-10-09T06:35:47.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.145+0000 7f721bee4700 1 -- 172.21.15.163:0/3146583516 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72140b2a40 con 0x7f721415df50 2023-10-09T06:35:47.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.145+0000 7f7209ffb700 1 -- 172.21.15.163:0/3146583516 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f720c01b3b0 con 0x7f721415df50 2023-10-09T06:35:47.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.145+0000 7f7209ffb700 1 --2- 172.21.15.163:0/3146583516 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f71f8067170 0x7f71f8069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:47.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.145+0000 7f7209ffb700 1 -- 172.21.15.163:0/3146583516 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f720c083990 con 0x7f721415df50 2023-10-09T06:35:47.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.145+0000 7f7219c80700 1 --2- 172.21.15.163:0/3146583516 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f71f8067170 0x7f71f8069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:47.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.146+0000 7f7219c80700 1 --2- 172.21.15.163:0/3146583516 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f71f8067170 0x7f71f8069620 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7210009e60 tx=0x7f7210019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:47.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.149+0000 7f7209ffb700 1 -- 172.21.15.163:0/3146583516 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72140b2a40 con 0x7f721415df50 2023-10-09T06:35:47.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.375+0000 7f721bee4700 1 -- 172.21.15.163:0/3146583516 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72140b2a00 con 0x7f721415df50 2023-10-09T06:35:47.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.375+0000 7f7209ffb700 1 -- 172.21.15.163:0/3146583516 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+3538 (secure 0 0 0) 0x7f720c017020 con 0x7f721415df50 2023-10-09T06:35:47.377 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:35:47.377 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":43,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5407,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3673547262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3673547262},{"type":"v1","addr":"172.21.15.181:6837","nonce":3673547262}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5410,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1705122825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1705122825},{"type":"v1","addr":"172.21.15.181:6836","nonce":1705122825}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":5745,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/596332711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":596332711},{"type":"v1","addr":"172.21.15.163:6834","nonce":596332711}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6099,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1357297958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1357297958},{"type":"v1","addr":"172.21.15.163:6835","nonce":1357297958}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[]} 2023-10-09T06:35:47.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.378+0000 7f721bee4700 1 -- 172.21.15.163:0/3146583516 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f71f8067170 msgr2=0x7f71f8069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:47.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.378+0000 7f721bee4700 1 --2- 172.21.15.163:0/3146583516 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f71f8067170 0x7f71f8069620 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7210009e60 tx=0x7f7210019000 comp rx=0 tx=0).stop 2023-10-09T06:35:47.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.378+0000 7f721bee4700 1 -- 172.21.15.163:0/3146583516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f721415df50 msgr2=0x7f7214154a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:47.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.378+0000 7f721bee4700 1 --2- 172.21.15.163:0/3146583516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f721415df50 0x7f7214154a60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f720c00c7e0 tx=0x7f720c007d90 comp rx=0 tx=0).stop 2023-10-09T06:35:47.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.378+0000 7f721bee4700 1 -- 172.21.15.163:0/3146583516 shutdown_connections 2023-10-09T06:35:47.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.378+0000 7f721bee4700 1 --2- 172.21.15.163:0/3146583516 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f71f8067170 0x7f71f8069620 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:47.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.378+0000 7f721bee4700 1 --2- 172.21.15.163:0/3146583516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f721415df50 0x7f7214154a60 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:47.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.378+0000 7f721bee4700 1 --2- 172.21.15.163:0/3146583516 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f721406bcd0 0x7f721416ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:47.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.378+0000 7f721bee4700 1 --2- 172.21.15.163:0/3146583516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f721406c0a0 0x7f7214157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:47.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.378+0000 7f721bee4700 1 -- 172.21.15.163:0/3146583516 >> 172.21.15.163:0/3146583516 conn(0x7f72140ce600 msgr2=0x7f721406cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:47.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.378+0000 7f721bee4700 1 -- 172.21.15.163:0/3146583516 shutdown_connections 2023-10-09T06:35:47.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.378+0000 7f721bee4700 1 -- 172.21.15.163:0/3146583516 wait complete. 2023-10-09T06:35:47.380 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 43 2023-10-09T06:35:47.391 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-09T06:35:47.391 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-09T06:35:47.392 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-09T06:35:47.392 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:35:47.392 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-09T06:35:47.392 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-09T06:35:47.392 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-09T06:35:47.392 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:35:47.393 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-09T06:35:47.393 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-09T06:35:47.393 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-09T06:35:47.393 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:35:47.394 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-09T06:35:47.394 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-09T06:35:47.394 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-09T06:35:47.394 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:35:47.400 INFO:tasks.ceph.mds.a:Stopped 2023-10-09T06:35:47.400 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-09T06:35:47.401 INFO:tasks.ceph.mds.c:Stopped 2023-10-09T06:35:47.401 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-09T06:35:47.405 INFO:tasks.ceph.mds.b:Stopped 2023-10-09T06:35:47.405 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-09T06:35:47.406 INFO:tasks.ceph.mds.d:Stopped 2023-10-09T06:35:47.406 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-09T06:35:47.444 INFO:tasks.ceph.mds.a:Started 2023-10-09T06:35:47.444 DEBUG:teuthology.parallel:result is None 2023-10-09T06:35:47.445 INFO:tasks.ceph.mds.c:Started 2023-10-09T06:35:47.445 DEBUG:teuthology.parallel:result is None 2023-10-09T06:35:47.447 INFO:tasks.ceph.mds.b:Started 2023-10-09T06:35:47.447 DEBUG:teuthology.parallel:result is None 2023-10-09T06:35:47.448 INFO:tasks.ceph.mds.d:Started 2023-10-09T06:35:47.448 DEBUG:teuthology.parallel:result is None 2023-10-09T06:35:47.448 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-09T06:35:47.516 INFO:tasks.ceph.mds.c.smithi163.stdout:starting mds.c at 2023-10-09T06:35:47.517 INFO:tasks.ceph.mds.a.smithi163.stdout:starting mds.a at 2023-10-09T06:35:47.529 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-09T06:35:47.529 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-09T06:35:47.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.569+0000 7f079ffff700 1 Processor -- start 2023-10-09T06:35:47.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.569+0000 7f079ffff700 1 -- start start 2023-10-09T06:35:47.572 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.570+0000 7f079ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f07a0159420 0x7f07a00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:47.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.570+0000 7f079ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07a015d230 0x7f07a015d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:47.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.570+0000 7f079ffff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07a00b2980 con 0x7f07a01597f0 2023-10-09T06:35:47.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.570+0000 7f079ffff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a015dbb0 con 0x7f07a0159420 2023-10-09T06:35:47.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.570+0000 7f079ffff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a00b11d0 con 0x7f07a015d230 2023-10-09T06:35:47.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.570+0000 7f079effd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f07a0159420 0x7f07a00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:47.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.570+0000 7f079f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07a015d230 0x7f07a015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:47.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.570+0000 7f079e7fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f07a01597f0 0x7f07a00b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:47760/0 (socket says 172.21.15.163:47760) 2023-10-09T06:35:47.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.570+0000 7f079e7fc700 1 -- 172.21.15.163:0/3543311557 learned_addr learned my addr 172.21.15.163:0/3543311557 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:35:47.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.571+0000 7f079effd700 1 -- 172.21.15.163:0/3543311557 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07a015d230 msgr2=0x7f07a015d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:47.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.571+0000 7f079effd700 1 --2- 172.21.15.163:0/3543311557 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07a015d230 0x7f07a015d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:47.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.571+0000 7f079effd700 1 -- 172.21.15.163:0/3543311557 >> v1:172.21.15.181:6789/0 conn(0x7f07a01597f0 legacy=0x7f07a00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:47.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.571+0000 7f079effd700 1 -- 172.21.15.163:0/3543311557 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a00b14b0 con 0x7f07a0159420 2023-10-09T06:35:47.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.571+0000 7f079dffb700 1 -- 172.21.15.163:0/3543311557 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 912019567 0 0) 0x7f07a00b2980 con 0x7f07a01597f0 2023-10-09T06:35:47.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.571+0000 7f079effd700 1 --2- 172.21.15.163:0/3543311557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a0159420 0x7f07a00b5f40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f07900098e0 tx=0x7f07900091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=385687521042d0ca server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:47.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.571+0000 7f079dffb700 1 -- 172.21.15.163:0/3543311557 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079000b640 con 0x7f07a0159420 2023-10-09T06:35:47.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.571+0000 7f079dffb700 1 -- 172.21.15.163:0/3543311557 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079000b7a0 con 0x7f07a0159420 2023-10-09T06:35:47.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.571+0000 7f079dffb700 1 -- 172.21.15.163:0/3543311557 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079000bb40 con 0x7f07a0159420 2023-10-09T06:35:47.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.571+0000 7f079ffff700 1 -- 172.21.15.163:0/3543311557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a0159420 msgr2=0x7f07a00b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:47.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.571+0000 7f079ffff700 1 --2- 172.21.15.163:0/3543311557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a0159420 0x7f07a00b5f40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f07900098e0 tx=0x7f07900091a0 comp rx=0 tx=0).stop 2023-10-09T06:35:47.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.571+0000 7f079ffff700 1 -- 172.21.15.163:0/3543311557 shutdown_connections 2023-10-09T06:35:47.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.571+0000 7f079ffff700 1 --2- 172.21.15.163:0/3543311557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a0159420 0x7f07a00b5f40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:47.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.571+0000 7f079ffff700 1 --2- 172.21.15.163:0/3543311557 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07a015d230 0x7f07a015d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:47.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.571+0000 7f079ffff700 1 -- 172.21.15.163:0/3543311557 >> 172.21.15.163:0/3543311557 conn(0x7f07a00d0660 msgr2=0x7f07a00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:47.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.572+0000 7f079ffff700 1 -- 172.21.15.163:0/3543311557 shutdown_connections 2023-10-09T06:35:47.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.572+0000 7f079ffff700 1 -- 172.21.15.163:0/3543311557 wait complete. 2023-10-09T06:35:47.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.573+0000 7f079ffff700 1 Processor -- start 2023-10-09T06:35:47.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.573+0000 7f079ffff700 1 -- start start 2023-10-09T06:35:47.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.573+0000 7f079ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07a0159420 0x7f07a015c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:47.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.573+0000 7f079ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a01597f0 0x7f07a0148fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:47.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.573+0000 7f079ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a015d230 0x7f07a00c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:47.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.573+0000 7f079ffff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a00c8840 con 0x7f07a015d230 2023-10-09T06:35:47.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.573+0000 7f079ffff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a0169ea0 con 0x7f07a01597f0 2023-10-09T06:35:47.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.573+0000 7f079ffff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a015ac20 con 0x7f07a0159420 2023-10-09T06:35:47.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.574+0000 7f079f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a015d230 0x7f07a00c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:47.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.574+0000 7f079f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a015d230 0x7f07a00c6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:37906/0 (socket says 172.21.15.163:37906) 2023-10-09T06:35:47.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.574+0000 7f079f7fe700 1 -- 172.21.15.163:0/709292394 learned_addr learned my addr 172.21.15.163:0/709292394 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:47.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.574+0000 7f079e7fc700 1 --2- 172.21.15.163:0/709292394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a01597f0 0x7f07a0148fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:47.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.574+0000 7f079effd700 1 --2- 172.21.15.163:0/709292394 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07a0159420 0x7f07a015c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:47.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.574+0000 7f079f7fe700 1 -- 172.21.15.163:0/709292394 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07a0159420 msgr2=0x7f07a015c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:47.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.574+0000 7f079f7fe700 1 --2- 172.21.15.163:0/709292394 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07a0159420 0x7f07a015c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:47.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.574+0000 7f079f7fe700 1 -- 172.21.15.163:0/709292394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a01597f0 msgr2=0x7f07a0148fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:47.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.574+0000 7f079f7fe700 1 --2- 172.21.15.163:0/709292394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a01597f0 0x7f07a0148fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:47.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.574+0000 7f079f7fe700 1 -- 172.21.15.163:0/709292394 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0790009420 con 0x7f07a015d230 2023-10-09T06:35:47.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.574+0000 7f079f7fe700 1 --2- 172.21.15.163:0/709292394 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a015d230 0x7f07a00c6e20 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f0798009df0 tx=0x7f079800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:47.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.574+0000 7f0786ffd700 1 -- 172.21.15.163:0/709292394 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079800cb50 con 0x7f07a015d230 2023-10-09T06:35:47.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.574+0000 7f0786ffd700 1 -- 172.21.15.163:0/709292394 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079800f040 con 0x7f07a015d230 2023-10-09T06:35:47.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.574+0000 7f079ffff700 1 -- 172.21.15.163:0/709292394 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a015af00 con 0x7f07a015d230 2023-10-09T06:35:47.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.574+0000 7f0786ffd700 1 -- 172.21.15.163:0/709292394 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0798014480 con 0x7f07a015d230 2023-10-09T06:35:47.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.574+0000 7f079ffff700 1 -- 172.21.15.163:0/709292394 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a015b4c0 con 0x7f07a015d230 2023-10-09T06:35:47.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.575+0000 7f079ffff700 1 -- 172.21.15.163:0/709292394 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a015cca0 con 0x7f07a015d230 2023-10-09T06:35:47.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.576+0000 7f0786ffd700 1 -- 172.21.15.163:0/709292394 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f0798014830 con 0x7f07a015d230 2023-10-09T06:35:47.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.576+0000 7f0786ffd700 1 --2- 172.21.15.163:0/709292394 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f078c0671a0 0x7f078c069650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:47.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.576+0000 7f0786ffd700 1 -- 172.21.15.163:0/709292394 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0798084f40 con 0x7f07a015d230 2023-10-09T06:35:47.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.576+0000 7f079effd700 1 --2- 172.21.15.163:0/709292394 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f078c0671a0 0x7f078c069650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:47.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.577+0000 7f079effd700 1 --2- 172.21.15.163:0/709292394 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f078c0671a0 0x7f078c069650 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f07900093f0 tx=0x7f0790017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:47.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.580+0000 7f0786ffd700 1 -- 172.21.15.163:0/709292394 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0798052110 con 0x7f07a015d230 2023-10-09T06:35:47.608 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-10-09T06:35:47.608 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-10-09T06:35:47.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:47.780+0000 7f079ffff700 1 -- 172.21.15.163:0/709292394 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f07a004c3c0 con 0x7f07a015d230 2023-10-09T06:35:48.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.607+0000 7f0786ffd700 1 -- 172.21.15.163:0/709292394 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v73) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0798055730 con 0x7f07a015d230 2023-10-09T06:35:48.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.610+0000 7f079ffff700 1 -- 172.21.15.163:0/709292394 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f078c0671a0 msgr2=0x7f078c069650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:48.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.610+0000 7f079ffff700 1 --2- 172.21.15.163:0/709292394 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f078c0671a0 0x7f078c069650 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f07900093f0 tx=0x7f0790017000 comp rx=0 tx=0).stop 2023-10-09T06:35:48.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.610+0000 7f079ffff700 1 -- 172.21.15.163:0/709292394 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a015d230 msgr2=0x7f07a00c6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:48.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.610+0000 7f079ffff700 1 --2- 172.21.15.163:0/709292394 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a015d230 0x7f07a00c6e20 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f0798009df0 tx=0x7f079800b040 comp rx=0 tx=0).stop 2023-10-09T06:35:48.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.610+0000 7f079ffff700 1 -- 172.21.15.163:0/709292394 shutdown_connections 2023-10-09T06:35:48.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.610+0000 7f079ffff700 1 --2- 172.21.15.163:0/709292394 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f078c0671a0 0x7f078c069650 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:48.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.610+0000 7f079ffff700 1 --2- 172.21.15.163:0/709292394 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a015d230 0x7f07a00c6e20 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:48.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.610+0000 7f079ffff700 1 --2- 172.21.15.163:0/709292394 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07a0159420 0x7f07a015c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:48.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.610+0000 7f079ffff700 1 --2- 172.21.15.163:0/709292394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a01597f0 0x7f07a0148fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:48.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.610+0000 7f079ffff700 1 -- 172.21.15.163:0/709292394 >> 172.21.15.163:0/709292394 conn(0x7f07a00d0660 msgr2=0x7f07a00ccf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:48.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.610+0000 7f079ffff700 1 -- 172.21.15.163:0/709292394 shutdown_connections 2023-10-09T06:35:48.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.610+0000 7f079ffff700 1 -- 172.21.15.163:0/709292394 wait complete. 2023-10-09T06:35:48.612 INFO:teuthology.orchestra.run.smithi163.stderr: removed all blocklist entries 2023-10-09T06:35:48.623 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-09T06:35:48.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.738+0000 7f8bbf6aa700 1 Processor -- start 2023-10-09T06:35:48.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.738+0000 7f8bbf6aa700 1 -- start start 2023-10-09T06:35:48.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.739+0000 7f8bbf6aa700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8bb8159890 0x7f8bb815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:48.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.739+0000 7f8bbf6aa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bb8069560 0x7f8bb80699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:48.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.739+0000 7f8bbf6aa700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb8069ee0 con 0x7f8bb8159c60 2023-10-09T06:35:48.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.739+0000 7f8bbf6aa700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb815a160 con 0x7f8bb8159890 2023-10-09T06:35:48.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.739+0000 7f8bbf6aa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb80d13a0 con 0x7f8bb8069560 2023-10-09T06:35:48.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.739+0000 7f8bbd446700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8bb8159890 0x7f8bb815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:48.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.739+0000 7f8bbdc47700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bb8069560 0x7f8bb80699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:48.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.739+0000 7f8bbcc45700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f8bb8159c60 0x7f8bb815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:34636/0 (socket says 172.21.15.163:34636) 2023-10-09T06:35:48.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.739+0000 7f8bbcc45700 1 -- 172.21.15.163:0/80455705 learned_addr learned my addr 172.21.15.163:0/80455705 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:35:48.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.740+0000 7f8bbdc47700 1 -- 172.21.15.163:0/80455705 >> v2:172.21.15.181:3300/0 conn(0x7f8bb8159890 msgr2=0x7f8bb815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:48.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.740+0000 7f8bbdc47700 1 --2- 172.21.15.163:0/80455705 >> v2:172.21.15.181:3300/0 conn(0x7f8bb8159890 0x7f8bb815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:48.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.740+0000 7f8bbdc47700 1 -- 172.21.15.163:0/80455705 >> v1:172.21.15.181:6789/0 conn(0x7f8bb8159c60 legacy=0x7f8bb815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:48.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.740+0000 7f8baffff700 1 -- 172.21.15.163:0/80455705 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1626432234 0 0) 0x7f8bb8069ee0 con 0x7f8bb8159c60 2023-10-09T06:35:48.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.740+0000 7f8bbdc47700 1 -- 172.21.15.163:0/80455705 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb815e140 con 0x7f8bb8069560 2023-10-09T06:35:48.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.740+0000 7f8bbd446700 1 --2- 172.21.15.163:0/80455705 >> v2:172.21.15.181:3300/0 conn(0x7f8bb8159890 0x7f8bb815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:35:48.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.740+0000 7f8bbdc47700 1 --2- 172.21.15.163:0/80455705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bb8069560 0x7f8bb80699a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009e60 tx=0x7f8bb0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=373f13053ef80c0f server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:48.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.740+0000 7f8baffff700 1 -- 172.21.15.163:0/80455705 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb000e070 con 0x7f8bb8069560 2023-10-09T06:35:48.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.740+0000 7f8baffff700 1 -- 172.21.15.163:0/80455705 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb000bb60 con 0x7f8bb8069560 2023-10-09T06:35:48.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.740+0000 7f8baffff700 1 -- 172.21.15.163:0/80455705 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0012430 con 0x7f8bb8069560 2023-10-09T06:35:48.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.740+0000 7f8bbf6aa700 1 -- 172.21.15.163:0/80455705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bb8069560 msgr2=0x7f8bb80699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:48.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.740+0000 7f8bbf6aa700 1 --2- 172.21.15.163:0/80455705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bb8069560 0x7f8bb80699a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009e60 tx=0x7f8bb0009780 comp rx=0 tx=0).stop 2023-10-09T06:35:48.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.740+0000 7f8bbf6aa700 1 -- 172.21.15.163:0/80455705 shutdown_connections 2023-10-09T06:35:48.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.740+0000 7f8bbf6aa700 1 --2- 172.21.15.163:0/80455705 >> v2:172.21.15.181:3300/0 conn(0x7f8bb8159890 0x7f8bb815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:48.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.740+0000 7f8bbf6aa700 1 --2- 172.21.15.163:0/80455705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bb8069560 0x7f8bb80699a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:48.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.741+0000 7f8bbf6aa700 1 -- 172.21.15.163:0/80455705 >> 172.21.15.163:0/80455705 conn(0x7f8bb8068e20 msgr2=0x7f8bb80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:48.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.741+0000 7f8bbf6aa700 1 -- 172.21.15.163:0/80455705 shutdown_connections 2023-10-09T06:35:48.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.741+0000 7f8bbf6aa700 1 -- 172.21.15.163:0/80455705 wait complete. 2023-10-09T06:35:48.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.742+0000 7f8bbf6aa700 1 Processor -- start 2023-10-09T06:35:48.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.742+0000 7f8bbf6aa700 1 -- start start 2023-10-09T06:35:48.747 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.743+0000 7f8bbf6aa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bb8069560 0x7f8bb81551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:48.747 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.743+0000 7f8bbf6aa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bb8159890 0x7f8bb815c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:48.747 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.743+0000 7f8bbf6aa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bb8159c60 0x7f8bb815c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:48.747 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.743+0000 7f8bbf6aa700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb80b3620 con 0x7f8bb8159890 2023-10-09T06:35:48.748 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.743+0000 7f8bbf6aa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb80b3870 con 0x7f8bb8159c60 2023-10-09T06:35:48.748 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.743+0000 7f8bbf6aa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb80b3b50 con 0x7f8bb8069560 2023-10-09T06:35:48.748 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.743+0000 7f8bbcc45700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bb8159890 0x7f8bb815c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:48.748 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.743+0000 7f8bbcc45700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bb8159890 0x7f8bb815c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55964/0 (socket says 172.21.15.163:55964) 2023-10-09T06:35:48.748 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.743+0000 7f8bbcc45700 1 -- 172.21.15.163:0/2407172273 learned_addr learned my addr 172.21.15.163:0/2407172273 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:48.748 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.743+0000 7f8bbd446700 1 --2- 172.21.15.163:0/2407172273 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bb8069560 0x7f8bb81551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:48.748 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.743+0000 7f8bbdc47700 1 --2- 172.21.15.163:0/2407172273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bb8159c60 0x7f8bb815c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:48.748 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.743+0000 7f8bbcc45700 1 -- 172.21.15.163:0/2407172273 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bb8069560 msgr2=0x7f8bb81551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:48.748 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.743+0000 7f8bbcc45700 1 --2- 172.21.15.163:0/2407172273 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bb8069560 0x7f8bb81551d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:48.748 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.743+0000 7f8bbcc45700 1 -- 172.21.15.163:0/2407172273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bb8159c60 msgr2=0x7f8bb815c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:48.748 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.743+0000 7f8bbcc45700 1 --2- 172.21.15.163:0/2407172273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bb8159c60 0x7f8bb815c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:48.749 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.743+0000 7f8bbcc45700 1 -- 172.21.15.163:0/2407172273 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb4009420 con 0x7f8bb8159890 2023-10-09T06:35:48.749 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.743+0000 7f8bbcc45700 1 --2- 172.21.15.163:0/2407172273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bb8159890 0x7f8bb815c4a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800c7e0 tx=0x7f8ba8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:48.749 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.743+0000 7f8bad7fa700 1 -- 172.21.15.163:0/2407172273 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8005b30 con 0x7f8bb8159890 2023-10-09T06:35:48.749 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.743+0000 7f8bbf6aa700 1 -- 172.21.15.163:0/2407172273 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb0009a20 con 0x7f8bb8159890 2023-10-09T06:35:48.749 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.743+0000 7f8bad7fa700 1 -- 172.21.15.163:0/2407172273 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba8005c90 con 0x7f8bb8159890 2023-10-09T06:35:48.749 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.743+0000 7f8bad7fa700 1 -- 172.21.15.163:0/2407172273 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8007300 con 0x7f8bb8159890 2023-10-09T06:35:48.749 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.743+0000 7f8bbf6aa700 1 -- 172.21.15.163:0/2407172273 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb80b4110 con 0x7f8bb8159890 2023-10-09T06:35:48.749 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.744+0000 7f8bbf6aa700 1 -- 172.21.15.163:0/2407172273 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb804c3c0 con 0x7f8bb8159890 2023-10-09T06:35:48.749 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.745+0000 7f8bad7fa700 1 -- 172.21.15.163:0/2407172273 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f8ba801b3b0 con 0x7f8bb8159890 2023-10-09T06:35:48.749 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.745+0000 7f8bad7fa700 1 --2- 172.21.15.163:0/2407172273 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8b9c067170 0x7f8b9c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:48.749 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.745+0000 7f8bad7fa700 1 -- 172.21.15.163:0/2407172273 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8ba8083b80 con 0x7f8bb8159890 2023-10-09T06:35:48.750 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.745+0000 7f8bbd446700 1 --2- 172.21.15.163:0/2407172273 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8b9c067170 0x7f8b9c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:48.750 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.746+0000 7f8bbd446700 1 --2- 172.21.15.163:0/2407172273 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8b9c067170 0x7f8b9c069620 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4009f40 tx=0x7f8bb40091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:48.750 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:48.749+0000 7f8bad7fa700 1 -- 172.21.15.163:0/2407172273 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ba8050de0 con 0x7f8bb8159890 2023-10-09T06:35:49.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.008+0000 7f8bbf6aa700 1 -- 172.21.15.163:0/2407172273 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f8bb806a5b0 con 0x7f8bb8159890 2023-10-09T06:35:49.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.009+0000 7f8bad7fa700 1 -- 172.21.15.163:0/2407172273 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f8ba8017020 con 0x7f8bb8159890 2023-10-09T06:35:49.010 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:35:49.010 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:35:49.010 INFO:teuthology.orchestra.run.smithi163.stdout: "auth_dump": [ 2023-10-09T06:35:49.010 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:35:49.010 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.a", 2023-10-09T06:35:49.010 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNltXtwDxAAlnrWRsfnZDLEmsJVowh3Mg==", 2023-10-09T06:35:49.011 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:35:49.011 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:35:49.011 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:35:49.011 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:35:49.011 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:35:49.011 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:35:49.011 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:35:49.011 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:35:49.011 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.b", 2023-10-09T06:35:49.011 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlTdGhGhAAXuldy0HrFd+44sSu/EV3qA==", 2023-10-09T06:35:49.011 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:35:49.012 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:35:49.012 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:35:49.012 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:35:49.012 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:35:49.012 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:35:49.012 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:35:49.012 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:35:49.012 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.c", 2023-10-09T06:35:49.012 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlGis7FhAAYG76TiOFX8QkTwjNX2+Ncw==", 2023-10-09T06:35:49.012 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:35:49.012 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:35:49.012 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:35:49.013 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:35:49.013 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:35:49.013 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:35:49.013 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:35:49.013 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:35:49.013 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.d", 2023-10-09T06:35:49.013 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlKhRuIRAAfhG5Gpc7klXJloYXepOhsg==", 2023-10-09T06:35:49.013 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:35:49.013 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:35:49.013 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:35:49.013 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:35:49.013 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:35:49.013 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:35:49.014 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:35:49.014 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:35:49.014 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.0", 2023-10-09T06:35:49.014 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDcnSNlrQeFOxAA+s6UitrTn5TK1N7kgySoXw==", 2023-10-09T06:35:49.014 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:35:49.014 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:35:49.014 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:35:49.014 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:35:49.014 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:35:49.014 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:35:49.014 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:35:49.015 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.1", 2023-10-09T06:35:49.015 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDfnSNlleccORAAnB+lNUFblivdoY44JfELxA==", 2023-10-09T06:35:49.015 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:35:49.015 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:35:49.015 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:35:49.015 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:35:49.015 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:35:49.015 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:35:49.015 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:35:49.015 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.2", 2023-10-09T06:35:49.015 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDinSNlFoB4NxAAT0+trRBij5rqgc+F0eWUfQ==", 2023-10-09T06:35:49.015 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:35:49.015 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:35:49.016 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:35:49.016 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:35:49.016 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:35:49.016 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:35:49.016 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:35:49.016 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.3", 2023-10-09T06:35:49.016 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDlnSNlRRQ6NhAABoBtrFjSvQujolq0Pxrd1Q==", 2023-10-09T06:35:49.016 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:35:49.016 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:35:49.016 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:35:49.016 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:35:49.016 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:35:49.017 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:35:49.017 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:35:49.017 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.4", 2023-10-09T06:35:49.017 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDrnSNl308uOxAA5OJyPLGQmVSJqha5d+nKKQ==", 2023-10-09T06:35:49.017 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:35:49.017 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:35:49.017 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:35:49.017 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:35:49.017 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:35:49.017 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:35:49.017 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:35:49.017 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.5", 2023-10-09T06:35:49.017 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDunSNlHnK+OBAAU7l4dCJ82SbGSzBEm8Axzw==", 2023-10-09T06:35:49.018 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:35:49.018 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:35:49.018 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:35:49.018 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:35:49.018 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:35:49.018 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:35:49.018 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:35:49.018 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.6", 2023-10-09T06:35:49.018 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDxnSNlwUSLOBAAN5N+F5ho+Co8pBh4Rz+lXQ==", 2023-10-09T06:35:49.018 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:35:49.018 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:35:49.018 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:35:49.019 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:35:49.019 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:35:49.019 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:35:49.019 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:35:49.019 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.7", 2023-10-09T06:35:49.019 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQD0nSNl2gxGORAAoE8OLGEifcEx/km8rfSV9g==", 2023-10-09T06:35:49.019 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:35:49.019 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:35:49.019 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:35:49.019 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:35:49.019 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:35:49.019 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:35:49.019 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:35:49.020 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.0", 2023-10-09T06:35:49.020 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlnxY5JRAApr6czcpMdQYYtXQ9yg2q6A==", 2023-10-09T06:35:49.020 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:35:49.020 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:35:49.020 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow r", 2023-10-09T06:35:49.020 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-09T06:35:49.020 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:35:49.020 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:35:49.020 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:35:49.020 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.1", 2023-10-09T06:35:49.020 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlk6ybLRAAuShqTDAtymGdxN8KBdZDDA==", 2023-10-09T06:35:49.020 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:35:49.020 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:35:49.021 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:35:49.021 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:35:49.021 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:35:49.021 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:35:49.021 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:35:49.021 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:35:49.021 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.2", 2023-10-09T06:35:49.021 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlYyOmKhAAQmpr+mGfJH4GUa28dtlJwA==", 2023-10-09T06:35:49.021 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:35:49.021 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:35:49.021 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:35:49.021 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:35:49.022 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:35:49.022 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:35:49.022 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:35:49.022 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:35:49.022 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.3", 2023-10-09T06:35:49.022 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlD2H1MhAA4zvaXY/Gd0kndzi+QSwp7w==", 2023-10-09T06:35:49.022 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:35:49.022 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:35:49.023 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:35:49.023 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:35:49.023 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:35:49.023 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:35:49.023 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:35:49.023 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:35:49.023 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.admin", 2023-10-09T06:35:49.023 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDYnSNlsHH4LxAA85xeFEP1CUwoblaePYk5zA==", 2023-10-09T06:35:49.023 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:35:49.023 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:35:49.024 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:35:49.024 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:35:49.024 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:35:49.024 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:35:49.024 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:35:49.024 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:35:49.024 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mgr.x", 2023-10-09T06:35:49.024 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNl2sloDBAA0FoWN9iovCl6783o3/btiA==", 2023-10-09T06:35:49.024 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:35:49.024 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:35:49.024 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile mgr", 2023-10-09T06:35:49.024 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:35:49.024 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:35:49.025 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:35:49.025 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:35:49.025 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mgr.y", 2023-10-09T06:35:49.025 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlcFWVCBAAR2dKPg01ciTiNpSX0JIKFg==", 2023-10-09T06:35:49.025 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:35:49.025 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:35:49.025 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile mgr", 2023-10-09T06:35:49.025 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:35:49.025 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:35:49.025 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:35:49.025 INFO:teuthology.orchestra.run.smithi163.stdout: ] 2023-10-09T06:35:49.025 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:35:49.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.011+0000 7f8bbf6aa700 1 -- 172.21.15.163:0/2407172273 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8b9c067170 msgr2=0x7f8b9c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:49.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.011+0000 7f8bbf6aa700 1 --2- 172.21.15.163:0/2407172273 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8b9c067170 0x7f8b9c069620 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4009f40 tx=0x7f8bb40091a0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.011+0000 7f8bbf6aa700 1 -- 172.21.15.163:0/2407172273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bb8159890 msgr2=0x7f8bb815c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:49.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.011+0000 7f8bbf6aa700 1 --2- 172.21.15.163:0/2407172273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bb8159890 0x7f8bb815c4a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800c7e0 tx=0x7f8ba8007df0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.011+0000 7f8bbf6aa700 1 -- 172.21.15.163:0/2407172273 shutdown_connections 2023-10-09T06:35:49.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.011+0000 7f8bbf6aa700 1 --2- 172.21.15.163:0/2407172273 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8b9c067170 0x7f8b9c069620 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.011+0000 7f8bbf6aa700 1 --2- 172.21.15.163:0/2407172273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bb8159c60 0x7f8bb815c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.011+0000 7f8bbf6aa700 1 --2- 172.21.15.163:0/2407172273 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bb8069560 0x7f8bb81551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.011+0000 7f8bbf6aa700 1 --2- 172.21.15.163:0/2407172273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bb8159890 0x7f8bb815c4a0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.011+0000 7f8bbf6aa700 1 -- 172.21.15.163:0/2407172273 >> 172.21.15.163:0/2407172273 conn(0x7f8bb8068e20 msgr2=0x7f8bb80b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:49.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.011+0000 7f8bbf6aa700 1 -- 172.21.15.163:0/2407172273 shutdown_connections 2023-10-09T06:35:49.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.011+0000 7f8bbf6aa700 1 -- 172.21.15.163:0/2407172273 wait complete. 2023-10-09T06:35:49.029 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:35:49.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.142+0000 7f2dcd399700 1 Processor -- start 2023-10-09T06:35:49.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.142+0000 7f2dcd399700 1 -- start start 2023-10-09T06:35:49.145 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.143+0000 7f2dcd399700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2dc8159890 0x7f2dc815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:49.145 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.143+0000 7f2dcd399700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2dc8069560 0x7f2dc80699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:49.145 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.143+0000 7f2dcd399700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dc8069ee0 con 0x7f2dc8159c60 2023-10-09T06:35:49.145 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.143+0000 7f2dcd399700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dc815a160 con 0x7f2dc8159890 2023-10-09T06:35:49.145 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.143+0000 7f2dcd399700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dc80d13a0 con 0x7f2dc8069560 2023-10-09T06:35:49.145 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.143+0000 7f2dc6ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2dc8159890 0x7f2dc815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:49.145 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.143+0000 7f2dc6ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2dc8159890 0x7f2dc815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55976/0 (socket says 172.21.15.163:55976) 2023-10-09T06:35:49.145 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.143+0000 7f2dc6ffd700 1 -- 172.21.15.163:0/2191350914 learned_addr learned my addr 172.21.15.163:0/2191350914 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:49.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.143+0000 7f2dc77fe700 1 --2- 172.21.15.163:0/2191350914 >> v2:172.21.15.181:3300/0 conn(0x7f2dc8069560 0x7f2dc80699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:49.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.143+0000 7f2dc6ffd700 1 -- 172.21.15.163:0/2191350914 >> v2:172.21.15.181:3300/0 conn(0x7f2dc8069560 msgr2=0x7f2dc80699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:49.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.143+0000 7f2dc6ffd700 1 --2- 172.21.15.163:0/2191350914 >> v2:172.21.15.181:3300/0 conn(0x7f2dc8069560 0x7f2dc80699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.143+0000 7f2dc6ffd700 1 -- 172.21.15.163:0/2191350914 >> v1:172.21.15.181:6789/0 conn(0x7f2dc8159c60 legacy=0x7f2dc815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:49.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.143+0000 7f2dc6ffd700 1 -- 172.21.15.163:0/2191350914 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc815e140 con 0x7f2dc8159890 2023-10-09T06:35:49.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.143+0000 7f2dc6ffd700 1 --2- 172.21.15.163:0/2191350914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2dc8159890 0x7f2dc815abd0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2dc806a890 tx=0x7f2dc0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49fe530d0804fb41 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:49.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.143+0000 7f2dc5ffb700 1 -- 172.21.15.163:0/2191350914 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0009e70 con 0x7f2dc8159890 2023-10-09T06:35:49.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.144+0000 7f2dc5ffb700 1 -- 172.21.15.163:0/2191350914 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc000b860 con 0x7f2dc8159890 2023-10-09T06:35:49.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.144+0000 7f2dc5ffb700 1 -- 172.21.15.163:0/2191350914 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc000bc00 con 0x7f2dc8159890 2023-10-09T06:35:49.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.144+0000 7f2dcd399700 1 -- 172.21.15.163:0/2191350914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2dc8159890 msgr2=0x7f2dc815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:49.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.144+0000 7f2dcd399700 1 --2- 172.21.15.163:0/2191350914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2dc8159890 0x7f2dc815abd0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2dc806a890 tx=0x7f2dc0009250 comp rx=0 tx=0).stop 2023-10-09T06:35:49.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.144+0000 7f2dcd399700 1 -- 172.21.15.163:0/2191350914 shutdown_connections 2023-10-09T06:35:49.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.144+0000 7f2dcd399700 1 --2- 172.21.15.163:0/2191350914 >> v2:172.21.15.181:3300/0 conn(0x7f2dc8069560 0x7f2dc80699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.144+0000 7f2dcd399700 1 --2- 172.21.15.163:0/2191350914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2dc8159890 0x7f2dc815abd0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.144+0000 7f2dcd399700 1 -- 172.21.15.163:0/2191350914 >> 172.21.15.163:0/2191350914 conn(0x7f2dc8068e20 msgr2=0x7f2dc80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:49.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.144+0000 7f2dcd399700 1 -- 172.21.15.163:0/2191350914 shutdown_connections 2023-10-09T06:35:49.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.144+0000 7f2dcd399700 1 -- 172.21.15.163:0/2191350914 wait complete. 2023-10-09T06:35:49.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.145+0000 7f2dcd399700 1 Processor -- start 2023-10-09T06:35:49.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.145+0000 7f2dcd399700 1 -- start start 2023-10-09T06:35:49.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.146+0000 7f2dcd399700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2dc8069560 0x7f2dc80c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:49.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.146+0000 7f2dcd399700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc8159890 0x7f2dc80b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:49.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.146+0000 7f2dcd399700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dc8159c60 0x7f2dc816d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:49.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.146+0000 7f2dcd399700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc80b64c0 con 0x7f2dc8069560 2023-10-09T06:35:49.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.146+0000 7f2dcd399700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc80b4430 con 0x7f2dc8159890 2023-10-09T06:35:49.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.146+0000 7f2dcd399700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc80b4710 con 0x7f2dc8159c60 2023-10-09T06:35:49.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.146+0000 7f2dc6ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2dc8069560 0x7f2dc80c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:49.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.146+0000 7f2dc6ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2dc8069560 0x7f2dc80c7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55988/0 (socket says 172.21.15.163:55988) 2023-10-09T06:35:49.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.146+0000 7f2dc6ffd700 1 -- 172.21.15.163:0/2418309897 learned_addr learned my addr 172.21.15.163:0/2418309897 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:49.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.146+0000 7f2dc67fc700 1 --2- 172.21.15.163:0/2418309897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc8159890 0x7f2dc80b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:49.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.147+0000 7f2dc77fe700 1 --2- 172.21.15.163:0/2418309897 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dc8159c60 0x7f2dc816d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:49.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.147+0000 7f2dc67fc700 1 -- 172.21.15.163:0/2418309897 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dc8159c60 msgr2=0x7f2dc816d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:49.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.147+0000 7f2dc67fc700 1 --2- 172.21.15.163:0/2418309897 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dc8159c60 0x7f2dc816d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.147+0000 7f2dc67fc700 1 -- 172.21.15.163:0/2418309897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2dc8069560 msgr2=0x7f2dc80c7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:49.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.147+0000 7f2dc67fc700 1 --2- 172.21.15.163:0/2418309897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2dc8069560 0x7f2dc80c7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.147+0000 7f2dc67fc700 1 -- 172.21.15.163:0/2418309897 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc00094f0 con 0x7f2dc8159890 2023-10-09T06:35:49.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.147+0000 7f2dc67fc700 1 --2- 172.21.15.163:0/2418309897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc8159890 0x7f2dc80b5c70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc00c490 tx=0x7f2dbc007ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:49.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.147+0000 7f2daeffd700 1 -- 172.21.15.163:0/2418309897 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc011590 con 0x7f2dc8159890 2023-10-09T06:35:49.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.147+0000 7f2daeffd700 1 -- 172.21.15.163:0/2418309897 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dbc011710 con 0x7f2dc8159890 2023-10-09T06:35:49.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.147+0000 7f2daeffd700 1 -- 172.21.15.163:0/2418309897 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc011ad0 con 0x7f2dc8159890 2023-10-09T06:35:49.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.147+0000 7f2dcd399700 1 -- 172.21.15.163:0/2418309897 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dc80b49f0 con 0x7f2dc8159890 2023-10-09T06:35:49.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.148+0000 7f2dcd399700 1 -- 172.21.15.163:0/2418309897 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc80b61b0 con 0x7f2dc8159890 2023-10-09T06:35:49.152 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.149+0000 7f2dcd399700 1 -- 172.21.15.163:0/2418309897 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dc80b4f10 con 0x7f2dc8159890 2023-10-09T06:35:49.152 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.149+0000 7f2daeffd700 1 -- 172.21.15.163:0/2418309897 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f2dbc01b3b0 con 0x7f2dc8159890 2023-10-09T06:35:49.152 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.149+0000 7f2daeffd700 1 --2- 172.21.15.163:0/2418309897 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2db4067170 0x7f2db4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:49.152 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.150+0000 7f2daeffd700 1 -- 172.21.15.163:0/2418309897 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f2dbc083850 con 0x7f2dc8159890 2023-10-09T06:35:49.152 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.150+0000 7f2dc6ffd700 1 --2- 172.21.15.163:0/2418309897 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2db4067170 0x7f2db4069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:49.152 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.150+0000 7f2dc6ffd700 1 --2- 172.21.15.163:0/2418309897 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2db4067170 0x7f2db4069620 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0006e80 tx=0x7f2dc0017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:49.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.154+0000 7f2daeffd700 1 -- 172.21.15.163:0/2418309897 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2dc80b4f10 con 0x7f2dc8159890 2023-10-09T06:35:49.336 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.335+0000 7f2dcd399700 1 -- 172.21.15.163:0/2418309897 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2dc80b4f10 con 0x7f2dc8159890 2023-10-09T06:35:49.337 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.336+0000 7f2daeffd700 1 -- 172.21.15.163:0/2418309897 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+12183 (secure 0 0 0) 0x7f2dc80b4f10 con 0x7f2dc8159890 2023-10-09T06:35:49.337 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:35:49.337 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":73,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:35:48.606112+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:35:49.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.338+0000 7f2dcd399700 1 -- 172.21.15.163:0/2418309897 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2db4067170 msgr2=0x7f2db4069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:49.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.338+0000 7f2dcd399700 1 --2- 172.21.15.163:0/2418309897 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2db4067170 0x7f2db4069620 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0006e80 tx=0x7f2dc0017000 comp rx=0 tx=0).stop 2023-10-09T06:35:49.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.339+0000 7f2dcd399700 1 -- 172.21.15.163:0/2418309897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc8159890 msgr2=0x7f2dc80b5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:49.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.339+0000 7f2dcd399700 1 --2- 172.21.15.163:0/2418309897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc8159890 0x7f2dc80b5c70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc00c490 tx=0x7f2dbc007ec0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.339+0000 7f2dcd399700 1 -- 172.21.15.163:0/2418309897 shutdown_connections 2023-10-09T06:35:49.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.339+0000 7f2dcd399700 1 --2- 172.21.15.163:0/2418309897 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2db4067170 0x7f2db4069620 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.339+0000 7f2dcd399700 1 --2- 172.21.15.163:0/2418309897 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2dc8069560 0x7f2dc80c7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.339+0000 7f2dcd399700 1 --2- 172.21.15.163:0/2418309897 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dc8159c60 0x7f2dc816d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.339+0000 7f2dcd399700 1 --2- 172.21.15.163:0/2418309897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dc8159890 0x7f2dc80b5c70 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.341 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.339+0000 7f2dcd399700 1 -- 172.21.15.163:0/2418309897 >> 172.21.15.163:0/2418309897 conn(0x7f2dc8068e20 msgr2=0x7f2dc81534c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:49.341 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.339+0000 7f2dcd399700 1 -- 172.21.15.163:0/2418309897 shutdown_connections 2023-10-09T06:35:49.341 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.339+0000 7f2dcd399700 1 -- 172.21.15.163:0/2418309897 wait complete. 2023-10-09T06:35:49.352 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-09T06:35:49.352 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:35:49.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.469+0000 7f7e619ef700 1 Processor -- start 2023-10-09T06:35:49.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.469+0000 7f7e619ef700 1 -- start start 2023-10-09T06:35:49.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.470+0000 7f7e619ef700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7e5c159c10 0x7f7e5c15b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:49.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.470+0000 7f7e619ef700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e5c15c910 0x7f7e5c15cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:49.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.470+0000 7f7e619ef700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e5c15c480 con 0x7f7e5c0b5ea0 2023-10-09T06:35:49.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.470+0000 7f7e619ef700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e5c0b2a40 con 0x7f7e5c159c10 2023-10-09T06:35:49.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.470+0000 7f7e619ef700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c15d290 con 0x7f7e5c15c910 2023-10-09T06:35:49.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.470+0000 7f7e5affd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7e5c159c10 0x7f7e5c15b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:49.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.470+0000 7f7e5affd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7e5c159c10 0x7f7e5c15b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55998/0 (socket says 172.21.15.163:55998) 2023-10-09T06:35:49.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.470+0000 7f7e5affd700 1 -- 172.21.15.163:0/254487112 learned_addr learned my addr 172.21.15.163:0/254487112 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:49.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.470+0000 7f7e5affd700 1 -- 172.21.15.163:0/254487112 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e5c15c910 msgr2=0x7f7e5c15cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:49.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.470+0000 7f7e5affd700 1 --2- 172.21.15.163:0/254487112 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e5c15c910 0x7f7e5c15cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.470+0000 7f7e5affd700 1 -- 172.21.15.163:0/254487112 >> v1:172.21.15.181:6789/0 conn(0x7f7e5c0b5ea0 legacy=0x7f7e5c068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:49.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.470+0000 7f7e5affd700 1 -- 172.21.15.163:0/254487112 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5c15b5d0 con 0x7f7e5c159c10 2023-10-09T06:35:49.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.470+0000 7f7e5affd700 1 --2- 172.21.15.163:0/254487112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e5c159c10 0x7f7e5c15b090 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c0099b0 tx=0x7f7e4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d0a644bb6554801 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:49.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.470+0000 7f7e59ffb700 1 -- 172.21.15.163:0/254487112 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c00b820 con 0x7f7e5c159c10 2023-10-09T06:35:49.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.470+0000 7f7e59ffb700 1 -- 172.21.15.163:0/254487112 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e4c00b980 con 0x7f7e5c159c10 2023-10-09T06:35:49.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.470+0000 7f7e59ffb700 1 -- 172.21.15.163:0/254487112 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c00bd20 con 0x7f7e5c159c10 2023-10-09T06:35:49.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.471+0000 7f7e619ef700 1 -- 172.21.15.163:0/254487112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e5c159c10 msgr2=0x7f7e5c15b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:49.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.471+0000 7f7e619ef700 1 --2- 172.21.15.163:0/254487112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e5c159c10 0x7f7e5c15b090 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c0099b0 tx=0x7f7e4c009250 comp rx=0 tx=0).stop 2023-10-09T06:35:49.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.471+0000 7f7e619ef700 1 -- 172.21.15.163:0/254487112 shutdown_connections 2023-10-09T06:35:49.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.471+0000 7f7e619ef700 1 --2- 172.21.15.163:0/254487112 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e5c15c910 0x7f7e5c15cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.471+0000 7f7e619ef700 1 --2- 172.21.15.163:0/254487112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e5c159c10 0x7f7e5c15b090 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.471+0000 7f7e619ef700 1 -- 172.21.15.163:0/254487112 >> 172.21.15.163:0/254487112 conn(0x7f7e5c099a20 msgr2=0x7f7e5c0af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:49.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.471+0000 7f7e619ef700 1 -- 172.21.15.163:0/254487112 shutdown_connections 2023-10-09T06:35:49.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.471+0000 7f7e619ef700 1 -- 172.21.15.163:0/254487112 wait complete. 2023-10-09T06:35:49.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.472+0000 7f7e619ef700 1 Processor -- start 2023-10-09T06:35:49.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.472+0000 7f7e619ef700 1 -- start start 2023-10-09T06:35:49.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.473+0000 7f7e619ef700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e5c0b5ea0 0x7f7e5c16a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:49.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.473+0000 7f7e619ef700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e5c159c10 0x7f7e5c166c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:49.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.473+0000 7f7e619ef700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e5c15c910 0x7f7e5c0c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:49.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.473+0000 7f7e619ef700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c16a7c0 con 0x7f7e5c0b5ea0 2023-10-09T06:35:49.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.473+0000 7f7e619ef700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c0c65c0 con 0x7f7e5c15c910 2023-10-09T06:35:49.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.473+0000 7f7e619ef700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c06a2b0 con 0x7f7e5c159c10 2023-10-09T06:35:49.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.473+0000 7f7e5affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e5c0b5ea0 0x7f7e5c16a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:49.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.473+0000 7f7e5affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e5c0b5ea0 0x7f7e5c16a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56006/0 (socket says 172.21.15.163:56006) 2023-10-09T06:35:49.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.473+0000 7f7e5affd700 1 -- 172.21.15.163:0/1553944032 learned_addr learned my addr 172.21.15.163:0/1553944032 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:49.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.473+0000 7f7e5a7fc700 1 --2- 172.21.15.163:0/1553944032 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e5c159c10 0x7f7e5c166c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:49.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.473+0000 7f7e5b7fe700 1 --2- 172.21.15.163:0/1553944032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e5c15c910 0x7f7e5c0c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:49.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.473+0000 7f7e5affd700 1 -- 172.21.15.163:0/1553944032 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e5c159c10 msgr2=0x7f7e5c166c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:49.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.473+0000 7f7e5affd700 1 --2- 172.21.15.163:0/1553944032 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e5c159c10 0x7f7e5c166c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.473+0000 7f7e5affd700 1 -- 172.21.15.163:0/1553944032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e5c15c910 msgr2=0x7f7e5c0c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:49.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.473+0000 7f7e5affd700 1 --2- 172.21.15.163:0/1553944032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e5c15c910 0x7f7e5c0c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.473+0000 7f7e5affd700 1 -- 172.21.15.163:0/1553944032 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e4c0094f0 con 0x7f7e5c0b5ea0 2023-10-09T06:35:49.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.473+0000 7f7e5b7fe700 1 --2- 172.21.15.163:0/1553944032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e5c15c910 0x7f7e5c0c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:35:49.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.474+0000 7f7e5affd700 1 --2- 172.21.15.163:0/1553944032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e5c0b5ea0 0x7f7e5c16a280 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c0090b0 tx=0x7f7e4c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:49.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.474+0000 7f7e42ffd700 1 -- 172.21.15.163:0/1553944032 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c01a460 con 0x7f7e5c0b5ea0 2023-10-09T06:35:49.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.474+0000 7f7e42ffd700 1 -- 172.21.15.163:0/1553944032 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e4c01a5c0 con 0x7f7e5c0b5ea0 2023-10-09T06:35:49.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.474+0000 7f7e42ffd700 1 -- 172.21.15.163:0/1553944032 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c01a960 con 0x7f7e5c0b5ea0 2023-10-09T06:35:49.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.474+0000 7f7e619ef700 1 -- 172.21.15.163:0/1553944032 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e5c0c59b0 con 0x7f7e5c0b5ea0 2023-10-09T06:35:49.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.474+0000 7f7e619ef700 1 -- 172.21.15.163:0/1553944032 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e5c0c5f70 con 0x7f7e5c0b5ea0 2023-10-09T06:35:49.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.475+0000 7f7e619ef700 1 -- 172.21.15.163:0/1553944032 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e5c04c3c0 con 0x7f7e5c0b5ea0 2023-10-09T06:35:49.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.475+0000 7f7e42ffd700 1 -- 172.21.15.163:0/1553944032 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f7e4c01ad50 con 0x7f7e5c0b5ea0 2023-10-09T06:35:49.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.476+0000 7f7e42ffd700 1 --2- 172.21.15.163:0/1553944032 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7e48067170 0x7f7e48069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:49.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.476+0000 7f7e42ffd700 1 -- 172.21.15.163:0/1553944032 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f7e4c085f80 con 0x7f7e5c0b5ea0 2023-10-09T06:35:49.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.476+0000 7f7e5a7fc700 1 --2- 172.21.15.163:0/1553944032 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7e48067170 0x7f7e48069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:49.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.476+0000 7f7e5a7fc700 1 --2- 172.21.15.163:0/1553944032 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7e48067170 0x7f7e48069620 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7e50006fd0 tx=0x7f7e50008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:49.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.480+0000 7f7e42ffd700 1 -- 172.21.15.163:0/1553944032 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7e4c0b75a0 con 0x7f7e5c0b5ea0 2023-10-09T06:35:49.662 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.661+0000 7f7e619ef700 1 -- 172.21.15.163:0/1553944032 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7e5c15b870 con 0x7f7e5c0b5ea0 2023-10-09T06:35:49.662 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.661+0000 7f7e42ffd700 1 -- 172.21.15.163:0/1553944032 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7e4c02a020 con 0x7f7e5c0b5ea0 2023-10-09T06:35:49.663 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:35:49.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.664+0000 7f7e619ef700 1 -- 172.21.15.163:0/1553944032 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7e48067170 msgr2=0x7f7e48069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:49.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.664+0000 7f7e619ef700 1 --2- 172.21.15.163:0/1553944032 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7e48067170 0x7f7e48069620 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7e50006fd0 tx=0x7f7e50008040 comp rx=0 tx=0).stop 2023-10-09T06:35:49.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.664+0000 7f7e619ef700 1 -- 172.21.15.163:0/1553944032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e5c0b5ea0 msgr2=0x7f7e5c16a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:49.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.664+0000 7f7e619ef700 1 --2- 172.21.15.163:0/1553944032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e5c0b5ea0 0x7f7e5c16a280 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c0090b0 tx=0x7f7e4c016000 comp rx=0 tx=0).stop 2023-10-09T06:35:49.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.664+0000 7f7e619ef700 1 -- 172.21.15.163:0/1553944032 shutdown_connections 2023-10-09T06:35:49.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.664+0000 7f7e619ef700 1 --2- 172.21.15.163:0/1553944032 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7e48067170 0x7f7e48069620 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.664+0000 7f7e619ef700 1 --2- 172.21.15.163:0/1553944032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e5c0b5ea0 0x7f7e5c16a280 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.664+0000 7f7e619ef700 1 --2- 172.21.15.163:0/1553944032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e5c15c910 0x7f7e5c0c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.664+0000 7f7e619ef700 1 --2- 172.21.15.163:0/1553944032 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e5c159c10 0x7f7e5c166c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.664+0000 7f7e619ef700 1 -- 172.21.15.163:0/1553944032 >> 172.21.15.163:0/1553944032 conn(0x7f7e5c099a20 msgr2=0x7f7e5c1592e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:49.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.664+0000 7f7e619ef700 1 -- 172.21.15.163:0/1553944032 shutdown_connections 2023-10-09T06:35:49.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.664+0000 7f7e619ef700 1 -- 172.21.15.163:0/1553944032 wait complete. 2023-10-09T06:35:49.677 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-09T06:35:49.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.790+0000 7f37bbd14700 1 Processor -- start 2023-10-09T06:35:49.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.790+0000 7f37bbd14700 1 -- start start 2023-10-09T06:35:49.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.791+0000 7f37bbd14700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37b406bcd0 0x7f37b40caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:49.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.791+0000 7f37bbd14700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f37b406c0a0 0x7f37b415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:49.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.791+0000 7f37bbd14700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f37b415df50 0x7f37b415e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:49.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.791+0000 7f37bbd14700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37b415d7e0 con 0x7f37b415df50 2023-10-09T06:35:49.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.791+0000 7f37bbd14700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37b415dac0 con 0x7f37b406c0a0 2023-10-09T06:35:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.791+0000 7f37bbd14700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b415aa30 con 0x7f37b406bcd0 2023-10-09T06:35:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.791+0000 7f37ba2b1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f37b415df50 0x7f37b415e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.791+0000 7f37ba2b1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f37b415df50 0x7f37b415e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56008/0 (socket says 172.21.15.163:56008) 2023-10-09T06:35:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.792+0000 7f37ba2b1700 1 -- 172.21.15.163:0/3604411709 learned_addr learned my addr 172.21.15.163:0/3604411709 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.792+0000 7f37b9ab0700 1 --2- 172.21.15.163:0/3604411709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37b406bcd0 0x7f37b40caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.792+0000 7f37b92af700 1 --2- 172.21.15.163:0/3604411709 >> v2:172.21.15.181:3300/0 conn(0x7f37b406c0a0 0x7f37b415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.792+0000 7f37ba2b1700 1 -- 172.21.15.163:0/3604411709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37b406bcd0 msgr2=0x7f37b40caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.792+0000 7f37ba2b1700 1 --2- 172.21.15.163:0/3604411709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37b406bcd0 0x7f37b40caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.792+0000 7f37ba2b1700 1 -- 172.21.15.163:0/3604411709 >> v2:172.21.15.181:3300/0 conn(0x7f37b406c0a0 msgr2=0x7f37b415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:49.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.792+0000 7f37ba2b1700 1 --2- 172.21.15.163:0/3604411709 >> v2:172.21.15.181:3300/0 conn(0x7f37b406c0a0 0x7f37b415a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.792+0000 7f37ba2b1700 1 -- 172.21.15.163:0/3604411709 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b415ad10 con 0x7f37b415df50 2023-10-09T06:35:49.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.792+0000 7f37b92af700 1 --2- 172.21.15.163:0/3604411709 >> v2:172.21.15.181:3300/0 conn(0x7f37b406c0a0 0x7f37b415a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:35:49.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.792+0000 7f37ba2b1700 1 --2- 172.21.15.163:0/3604411709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37b415df50 0x7f37b415e3b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f37b40c5660 tx=0x7f37ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a327853ae29c26f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:49.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.792+0000 7f37b8aae700 1 -- 172.21.15.163:0/3604411709 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac00e070 con 0x7f37b415df50 2023-10-09T06:35:49.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.792+0000 7f37b8aae700 1 -- 172.21.15.163:0/3604411709 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ac00ba50 con 0x7f37b415df50 2023-10-09T06:35:49.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.792+0000 7f37b8aae700 1 -- 172.21.15.163:0/3604411709 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac0092e0 con 0x7f37b415df50 2023-10-09T06:35:49.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.793+0000 7f37bbd14700 1 -- 172.21.15.163:0/3604411709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37b415df50 msgr2=0x7f37b415e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:49.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.793+0000 7f37bbd14700 1 --2- 172.21.15.163:0/3604411709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37b415df50 0x7f37b415e3b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f37b40c5660 tx=0x7f37ac009840 comp rx=0 tx=0).stop 2023-10-09T06:35:49.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.793+0000 7f37bbd14700 1 -- 172.21.15.163:0/3604411709 shutdown_connections 2023-10-09T06:35:49.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.793+0000 7f37bbd14700 1 --2- 172.21.15.163:0/3604411709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37b415df50 0x7f37b415e3b0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.793+0000 7f37bbd14700 1 --2- 172.21.15.163:0/3604411709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37b406bcd0 0x7f37b40caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.793+0000 7f37bbd14700 1 --2- 172.21.15.163:0/3604411709 >> v2:172.21.15.181:3300/0 conn(0x7f37b406c0a0 0x7f37b415a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.793+0000 7f37bbd14700 1 -- 172.21.15.163:0/3604411709 >> 172.21.15.163:0/3604411709 conn(0x7f37b40ce600 msgr2=0x7f37b40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:49.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.793+0000 7f37bbd14700 1 -- 172.21.15.163:0/3604411709 shutdown_connections 2023-10-09T06:35:49.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.793+0000 7f37bbd14700 1 -- 172.21.15.163:0/3604411709 wait complete. 2023-10-09T06:35:49.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.794+0000 7f37bbd14700 1 Processor -- start 2023-10-09T06:35:49.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.794+0000 7f37bbd14700 1 -- start start 2023-10-09T06:35:49.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.794+0000 7f37bbd14700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37b406bcd0 0x7f37b40b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:49.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.794+0000 7f37bbd14700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37b406c0a0 0x7f37b416da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:49.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.795+0000 7f37bbd14700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37b415df50 0x7f37b406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:49.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.795+0000 7f37bbd14700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b40b4250 con 0x7f37b415df50 2023-10-09T06:35:49.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.795+0000 7f37bbd14700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b416d590 con 0x7f37b406c0a0 2023-10-09T06:35:49.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.795+0000 7f37bbd14700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b40b39b0 con 0x7f37b406bcd0 2023-10-09T06:35:49.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.795+0000 7f37ba2b1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37b415df50 0x7f37b406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:49.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.795+0000 7f37ba2b1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37b415df50 0x7f37b406d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56016/0 (socket says 172.21.15.163:56016) 2023-10-09T06:35:49.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.795+0000 7f37b9ab0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37b406bcd0 0x7f37b40b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:49.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.795+0000 7f37ba2b1700 1 -- 172.21.15.163:0/799579523 learned_addr learned my addr 172.21.15.163:0/799579523 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:49.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.795+0000 7f37b92af700 1 --2- 172.21.15.163:0/799579523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37b406c0a0 0x7f37b416da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:49.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.795+0000 7f37ba2b1700 1 -- 172.21.15.163:0/799579523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37b406bcd0 msgr2=0x7f37b40b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:49.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.795+0000 7f37ba2b1700 1 --2- 172.21.15.163:0/799579523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37b406bcd0 0x7f37b40b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.795+0000 7f37ba2b1700 1 -- 172.21.15.163:0/799579523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37b406c0a0 msgr2=0x7f37b416da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:49.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.795+0000 7f37ba2b1700 1 --2- 172.21.15.163:0/799579523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37b406c0a0 0x7f37b416da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.795+0000 7f37ba2b1700 1 -- 172.21.15.163:0/799579523 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac009ae0 con 0x7f37b415df50 2023-10-09T06:35:49.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.795+0000 7f37ba2b1700 1 --2- 172.21.15.163:0/799579523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37b415df50 0x7f37b406d660 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f37ac009fd0 tx=0x7f37ac018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:49.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.795+0000 7f37a9ffb700 1 -- 172.21.15.163:0/799579523 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac0092e0 con 0x7f37b415df50 2023-10-09T06:35:49.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.795+0000 7f37a9ffb700 1 -- 172.21.15.163:0/799579523 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ac007d80 con 0x7f37b415df50 2023-10-09T06:35:49.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.795+0000 7f37a9ffb700 1 -- 172.21.15.163:0/799579523 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac01d670 con 0x7f37b415df50 2023-10-09T06:35:49.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.795+0000 7f37bbd14700 1 -- 172.21.15.163:0/799579523 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37b40b3c90 con 0x7f37b415df50 2023-10-09T06:35:49.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.796+0000 7f37bbd14700 1 -- 172.21.15.163:0/799579523 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b40b1f20 con 0x7f37b415df50 2023-10-09T06:35:49.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.796+0000 7f37bbd14700 1 -- 172.21.15.163:0/799579523 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37b416d3f0 con 0x7f37b415df50 2023-10-09T06:35:49.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.797+0000 7f37a9ffb700 1 -- 172.21.15.163:0/799579523 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f37ac003050 con 0x7f37b415df50 2023-10-09T06:35:49.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.797+0000 7f37a9ffb700 1 --2- 172.21.15.163:0/799579523 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3798067170 0x7f3798069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:49.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.797+0000 7f37a9ffb700 1 -- 172.21.15.163:0/799579523 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f37ac086330 con 0x7f37b415df50 2023-10-09T06:35:49.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.797+0000 7f37b9ab0700 1 --2- 172.21.15.163:0/799579523 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3798067170 0x7f3798069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:49.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.798+0000 7f37b9ab0700 1 --2- 172.21.15.163:0/799579523 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3798067170 0x7f3798069620 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f37b0006fd0 tx=0x7f37b0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:49.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.801+0000 7f37a9ffb700 1 -- 172.21.15.163:0/799579523 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37b416d3f0 con 0x7f37b415df50 2023-10-09T06:35:49.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.987+0000 7f37bbd14700 1 -- 172.21.15.163:0/799579523 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f37b416d3f0 con 0x7f37b415df50 2023-10-09T06:35:49.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.988+0000 7f37a9ffb700 1 -- 172.21.15.163:0/799579523 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v73) v1 ==== 84+0+65 (secure 0 0 0) 0x7f37b416d3f0 con 0x7f37b415df50 2023-10-09T06:35:49.989 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:35:49.989 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:35:49.989 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:35:49.989 INFO:teuthology.orchestra.run.smithi163.stdout: "poolnum": 1, 2023-10-09T06:35:49.989 INFO:teuthology.orchestra.run.smithi163.stdout: "poolname": ".mgr" 2023-10-09T06:35:49.989 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:35:49.989 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:35:49.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.989+0000 7f37bbd14700 1 -- 172.21.15.163:0/799579523 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3798067170 msgr2=0x7f3798069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:49.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.989+0000 7f37bbd14700 1 --2- 172.21.15.163:0/799579523 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3798067170 0x7f3798069620 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f37b0006fd0 tx=0x7f37b0008040 comp rx=0 tx=0).stop 2023-10-09T06:35:49.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.989+0000 7f37bbd14700 1 -- 172.21.15.163:0/799579523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37b415df50 msgr2=0x7f37b406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:49.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.989+0000 7f37bbd14700 1 --2- 172.21.15.163:0/799579523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37b415df50 0x7f37b406d660 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f37ac009fd0 tx=0x7f37ac018000 comp rx=0 tx=0).stop 2023-10-09T06:35:49.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.989+0000 7f37bbd14700 1 -- 172.21.15.163:0/799579523 shutdown_connections 2023-10-09T06:35:49.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.989+0000 7f37bbd14700 1 --2- 172.21.15.163:0/799579523 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3798067170 0x7f3798069620 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.989+0000 7f37bbd14700 1 --2- 172.21.15.163:0/799579523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37b415df50 0x7f37b406d660 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.989+0000 7f37bbd14700 1 --2- 172.21.15.163:0/799579523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37b406bcd0 0x7f37b40b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.989+0000 7f37bbd14700 1 --2- 172.21.15.163:0/799579523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37b406c0a0 0x7f37b416da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:49.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.989+0000 7f37bbd14700 1 -- 172.21.15.163:0/799579523 >> 172.21.15.163:0/799579523 conn(0x7f37b40ce600 msgr2=0x7f37b40c8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:49.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.990+0000 7f37bbd14700 1 -- 172.21.15.163:0/799579523 shutdown_connections 2023-10-09T06:35:49.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:49.990+0000 7f37bbd14700 1 -- 172.21.15.163:0/799579523 wait complete. 2023-10-09T06:35:50.002 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-09T06:35:50.003 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-09T06:35:50.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.120+0000 7f277a5dd700 1 Processor -- start 2023-10-09T06:35:50.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.120+0000 7f277a5dd700 1 -- start start 2023-10-09T06:35:50.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.121+0000 7f277a5dd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2774159ec0 0x7f27740b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:50.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.121+0000 7f277a5dd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f27740b53d0 0x7f27740cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:50.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.121+0000 7f277a5dd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27740cce40 0x7f27740cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:50.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.121+0000 7f277a5dd700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27740b5b00 con 0x7f2774159ec0 2023-10-09T06:35:50.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.121+0000 7f277a5dd700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27740b5850 con 0x7f27740b53d0 2023-10-09T06:35:50.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.121+0000 7f277a5dd700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27740b5da0 con 0x7f27740cce40 2023-10-09T06:35:50.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.121+0000 7f2773fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2774159ec0 0x7f27740b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:50.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.121+0000 7f2773fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2774159ec0 0x7f27740b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56022/0 (socket says 172.21.15.163:56022) 2023-10-09T06:35:50.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.121+0000 7f2773fff700 1 -- 172.21.15.163:0/1495451055 learned_addr learned my addr 172.21.15.163:0/1495451055 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:50.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.121+0000 7f2778b7a700 1 --2- 172.21.15.163:0/1495451055 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27740cce40 0x7f27740cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:50.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.121+0000 7f27737fe700 1 --2- 172.21.15.163:0/1495451055 >> v2:172.21.15.181:3300/0 conn(0x7f27740b53d0 0x7f27740cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:50.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.121+0000 7f2773fff700 1 -- 172.21.15.163:0/1495451055 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27740cce40 msgr2=0x7f27740cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:50.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.121+0000 7f2773fff700 1 --2- 172.21.15.163:0/1495451055 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27740cce40 0x7f27740cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:50.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.121+0000 7f2773fff700 1 -- 172.21.15.163:0/1495451055 >> v2:172.21.15.181:3300/0 conn(0x7f27740b53d0 msgr2=0x7f27740cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:50.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.121+0000 7f2773fff700 1 --2- 172.21.15.163:0/1495451055 >> v2:172.21.15.181:3300/0 conn(0x7f27740b53d0 0x7f27740cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:50.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.121+0000 7f2773fff700 1 -- 172.21.15.163:0/1495451055 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27740b6080 con 0x7f2774159ec0 2023-10-09T06:35:50.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.121+0000 7f2773fff700 1 --2- 172.21.15.163:0/1495451055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2774159ec0 0x7f27740b4e90 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f27740b64d0 tx=0x7f2764009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6c6f3c744b05e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:50.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.121+0000 7f2772ffd700 1 -- 172.21.15.163:0/1495451055 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2764005c10 con 0x7f2774159ec0 2023-10-09T06:35:50.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.122+0000 7f2772ffd700 1 -- 172.21.15.163:0/1495451055 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2764005d70 con 0x7f2774159ec0 2023-10-09T06:35:50.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.122+0000 7f2772ffd700 1 -- 172.21.15.163:0/1495451055 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276400b880 con 0x7f2774159ec0 2023-10-09T06:35:50.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.122+0000 7f277a5dd700 1 -- 172.21.15.163:0/1495451055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2774159ec0 msgr2=0x7f27740b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:50.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.122+0000 7f277a5dd700 1 --2- 172.21.15.163:0/1495451055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2774159ec0 0x7f27740b4e90 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f27740b64d0 tx=0x7f2764009220 comp rx=0 tx=0).stop 2023-10-09T06:35:50.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.122+0000 7f277a5dd700 1 -- 172.21.15.163:0/1495451055 shutdown_connections 2023-10-09T06:35:50.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.122+0000 7f277a5dd700 1 --2- 172.21.15.163:0/1495451055 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27740cce40 0x7f27740cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:50.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.122+0000 7f277a5dd700 1 --2- 172.21.15.163:0/1495451055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2774159ec0 0x7f27740b4e90 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:50.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.122+0000 7f277a5dd700 1 --2- 172.21.15.163:0/1495451055 >> v2:172.21.15.181:3300/0 conn(0x7f27740b53d0 0x7f27740cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:50.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.122+0000 7f277a5dd700 1 -- 172.21.15.163:0/1495451055 >> 172.21.15.163:0/1495451055 conn(0x7f27740999f0 msgr2=0x7f27740b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:50.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.122+0000 7f277a5dd700 1 -- 172.21.15.163:0/1495451055 shutdown_connections 2023-10-09T06:35:50.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.122+0000 7f277a5dd700 1 -- 172.21.15.163:0/1495451055 wait complete. 2023-10-09T06:35:50.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.123+0000 7f277a5dd700 1 Processor -- start 2023-10-09T06:35:50.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.123+0000 7f277a5dd700 1 -- start start 2023-10-09T06:35:50.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.124+0000 7f277a5dd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27740b53d0 0x7f27741450b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:50.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.124+0000 7f277a5dd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27740cce40 0x7f2774158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:50.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.124+0000 7f277a5dd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2774159ec0 0x7f2774159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:50.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.124+0000 7f277a5dd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277409d2c0 con 0x7f2774159ec0 2023-10-09T06:35:50.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.124+0000 7f277a5dd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277409d5a0 con 0x7f27740b53d0 2023-10-09T06:35:50.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.124+0000 7f277a5dd700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f277409d880 con 0x7f27740cce40 2023-10-09T06:35:50.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.124+0000 7f2778b7a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2774159ec0 0x7f2774159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:50.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.124+0000 7f2778b7a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2774159ec0 0x7f2774159370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56028/0 (socket says 172.21.15.163:56028) 2023-10-09T06:35:50.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.124+0000 7f2778b7a700 1 -- 172.21.15.163:0/1469145395 learned_addr learned my addr 172.21.15.163:0/1469145395 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:50.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.124+0000 7f27737fe700 1 --2- 172.21.15.163:0/1469145395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27740cce40 0x7f2774158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:50.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.124+0000 7f2773fff700 1 --2- 172.21.15.163:0/1469145395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27740b53d0 0x7f27741450b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:50.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.124+0000 7f2778b7a700 1 -- 172.21.15.163:0/1469145395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27740cce40 msgr2=0x7f2774158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:50.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.124+0000 7f2778b7a700 1 --2- 172.21.15.163:0/1469145395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27740cce40 0x7f2774158e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:50.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.124+0000 7f2778b7a700 1 -- 172.21.15.163:0/1469145395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27740b53d0 msgr2=0x7f27741450b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:50.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.124+0000 7f2778b7a700 1 --2- 172.21.15.163:0/1469145395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27740b53d0 0x7f27741450b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:50.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.124+0000 7f2778b7a700 1 -- 172.21.15.163:0/1469145395 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2764009480 con 0x7f2774159ec0 2023-10-09T06:35:50.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.125+0000 7f2778b7a700 1 --2- 172.21.15.163:0/1469145395 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2774159ec0 0x7f2774159370 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f276c009fa0 tx=0x7f276c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:50.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.125+0000 7f275bfff700 1 -- 172.21.15.163:0/1469145395 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c00ed20 con 0x7f2774159ec0 2023-10-09T06:35:50.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.125+0000 7f275bfff700 1 -- 172.21.15.163:0/1469145395 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276c00ee80 con 0x7f2774159ec0 2023-10-09T06:35:50.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.125+0000 7f277a5dd700 1 -- 172.21.15.163:0/1469145395 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f277409db60 con 0x7f2774159ec0 2023-10-09T06:35:50.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.125+0000 7f275bfff700 1 -- 172.21.15.163:0/1469145395 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c014670 con 0x7f2774159ec0 2023-10-09T06:35:50.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.125+0000 7f277a5dd700 1 -- 172.21.15.163:0/1469145395 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27741455f0 con 0x7f2774159ec0 2023-10-09T06:35:50.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.126+0000 7f277a5dd700 1 -- 172.21.15.163:0/1469145395 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27740718b0 con 0x7f2774159ec0 2023-10-09T06:35:50.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.126+0000 7f275bfff700 1 -- 172.21.15.163:0/1469145395 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f276c0149e0 con 0x7f2774159ec0 2023-10-09T06:35:50.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.126+0000 7f275bfff700 1 --2- 172.21.15.163:0/1469145395 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2760067170 0x7f2760069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:50.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.126+0000 7f275bfff700 1 -- 172.21.15.163:0/1469145395 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f276c084d20 con 0x7f2774159ec0 2023-10-09T06:35:50.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.127+0000 7f2773fff700 1 --2- 172.21.15.163:0/1469145395 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2760067170 0x7f2760069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:50.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.127+0000 7f2773fff700 1 --2- 172.21.15.163:0/1469145395 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2760067170 0x7f2760069620 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2764009260 tx=0x7f2764016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:50.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.131+0000 7f275bfff700 1 -- 172.21.15.163:0/1469145395 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f276c052050 con 0x7f2774159ec0 2023-10-09T06:35:50.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.313+0000 7f277a5dd700 1 -- 172.21.15.163:0/1469145395 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2774063c40 con 0x7f2774159ec0 2023-10-09T06:35:50.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.616+0000 7f275bfff700 1 -- 172.21.15.163:0/1469145395 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v74) v1 ==== 138+0+0 (secure 0 0 0) 0x7f276c055670 con 0x7f2774159ec0 2023-10-09T06:35:50.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.620+0000 7f277a5dd700 1 -- 172.21.15.163:0/1469145395 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2760067170 msgr2=0x7f2760069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:50.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.620+0000 7f277a5dd700 1 --2- 172.21.15.163:0/1469145395 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2760067170 0x7f2760069620 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2764009260 tx=0x7f2764016000 comp rx=0 tx=0).stop 2023-10-09T06:35:50.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.620+0000 7f277a5dd700 1 -- 172.21.15.163:0/1469145395 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2774159ec0 msgr2=0x7f2774159370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:50.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.620+0000 7f277a5dd700 1 --2- 172.21.15.163:0/1469145395 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2774159ec0 0x7f2774159370 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f276c009fa0 tx=0x7f276c009900 comp rx=0 tx=0).stop 2023-10-09T06:35:50.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.620+0000 7f277a5dd700 1 -- 172.21.15.163:0/1469145395 shutdown_connections 2023-10-09T06:35:50.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.620+0000 7f277a5dd700 1 --2- 172.21.15.163:0/1469145395 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2760067170 0x7f2760069620 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:50.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.620+0000 7f277a5dd700 1 --2- 172.21.15.163:0/1469145395 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2774159ec0 0x7f2774159370 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:50.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.620+0000 7f277a5dd700 1 --2- 172.21.15.163:0/1469145395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27740b53d0 0x7f27741450b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:50.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.620+0000 7f277a5dd700 1 --2- 172.21.15.163:0/1469145395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27740cce40 0x7f2774158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:50.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.620+0000 7f277a5dd700 1 -- 172.21.15.163:0/1469145395 >> 172.21.15.163:0/1469145395 conn(0x7f27740999f0 msgr2=0x7f27740c5510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:50.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.620+0000 7f277a5dd700 1 -- 172.21.15.163:0/1469145395 shutdown_connections 2023-10-09T06:35:50.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.620+0000 7f277a5dd700 1 -- 172.21.15.163:0/1469145395 wait complete. 2023-10-09T06:35:50.622 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_metadata' created 2023-10-09T06:35:50.635 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-09T06:35:50.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.756+0000 7f6acd247700 1 Processor -- start 2023-10-09T06:35:50.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.756+0000 7f6acd247700 1 -- start start 2023-10-09T06:35:50.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.757+0000 7f6acd247700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6ac806bcd0 0x7f6ac80c9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:50.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.757+0000 7f6acd247700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ac815b790 con 0x7f6ac81414c0 2023-10-09T06:35:50.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.757+0000 7f6acd247700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ac815ba70 con 0x7f6ac8158d40 2023-10-09T06:35:50.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.757+0000 7f6acd247700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ac815bd50 con 0x7f6ac806bcd0 2023-10-09T06:35:50.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.757+0000 7f6ac6d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6ac806bcd0 0x7f6ac80c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:50.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.757+0000 7f6ac759e700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f6ac81414c0 0x7f6ac80cc290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57684/0 (socket says 172.21.15.163:57684) 2023-10-09T06:35:50.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.757+0000 7f6ac759e700 1 -- 172.21.15.163:0/4078783070 learned_addr learned my addr 172.21.15.163:0/4078783070 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:35:50.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.757+0000 7f6ac6d9d700 1 -- 172.21.15.163:0/4078783070 >> v1:172.21.15.181:6789/0 conn(0x7f6ac8158d40 legacy=0x7f6ac8140db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:50.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.757+0000 7f6ac6d9d700 1 -- 172.21.15.163:0/4078783070 >> v1:172.21.15.163:6789/0 conn(0x7f6ac81414c0 legacy=0x7f6ac80cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:50.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.757+0000 7f6ac6d9d700 1 -- 172.21.15.163:0/4078783070 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac815c030 con 0x7f6ac806bcd0 2023-10-09T06:35:50.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.757+0000 7f6ac659c700 1 -- 172.21.15.163:0/4078783070 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3772895788 0 0) 0x7f6ac815b790 con 0x7f6ac81414c0 2023-10-09T06:35:50.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.757+0000 7f6ac6d9d700 1 --2- 172.21.15.163:0/4078783070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ac806bcd0 0x7f6ac80c9230 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80098e0 tx=0x7f6ab80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af9e322f376b4fea server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:50.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.757+0000 7f6ac659c700 1 -- 172.21.15.163:0/4078783070 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab8009e80 con 0x7f6ac806bcd0 2023-10-09T06:35:50.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.757+0000 7f6ac659c700 1 -- 172.21.15.163:0/4078783070 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab800b820 con 0x7f6ac806bcd0 2023-10-09T06:35:50.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.757+0000 7f6ac659c700 1 -- 172.21.15.163:0/4078783070 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab800bba0 con 0x7f6ac806bcd0 2023-10-09T06:35:50.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.758+0000 7f6acd247700 1 -- 172.21.15.163:0/4078783070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ac806bcd0 msgr2=0x7f6ac80c9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:50.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.758+0000 7f6acd247700 1 --2- 172.21.15.163:0/4078783070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ac806bcd0 0x7f6ac80c9230 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80098e0 tx=0x7f6ab80091a0 comp rx=0 tx=0).stop 2023-10-09T06:35:50.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.758+0000 7f6acd247700 1 -- 172.21.15.163:0/4078783070 shutdown_connections 2023-10-09T06:35:50.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.758+0000 7f6acd247700 1 --2- 172.21.15.163:0/4078783070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ac806bcd0 0x7f6ac80c9230 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:50.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.758+0000 7f6acd247700 1 -- 172.21.15.163:0/4078783070 >> 172.21.15.163:0/4078783070 conn(0x7f6ac80ce5c0 msgr2=0x7f6ac80ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:50.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.758+0000 7f6acd247700 1 -- 172.21.15.163:0/4078783070 shutdown_connections 2023-10-09T06:35:50.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.758+0000 7f6acd247700 1 -- 172.21.15.163:0/4078783070 wait complete. 2023-10-09T06:35:50.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.759+0000 7f6acd247700 1 Processor -- start 2023-10-09T06:35:50.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.759+0000 7f6acd247700 1 -- start start 2023-10-09T06:35:50.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.760+0000 7f6acd247700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ac806bcd0 0x7f6ac8156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:50.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.760+0000 7f6acd247700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ac81414c0 0x7f6ac80c6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:50.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.760+0000 7f6acd247700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ac8158d40 0x7f6ac81580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:50.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.760+0000 7f6acd247700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac81406b0 con 0x7f6ac8158d40 2023-10-09T06:35:50.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.760+0000 7f6acd247700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac813f820 con 0x7f6ac806bcd0 2023-10-09T06:35:50.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.760+0000 7f6acd247700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac8156fb0 con 0x7f6ac81414c0 2023-10-09T06:35:50.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.760+0000 7f6ac759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ac8158d40 0x7f6ac81580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:50.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.760+0000 7f6ac759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ac8158d40 0x7f6ac81580b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56044/0 (socket says 172.21.15.163:56044) 2023-10-09T06:35:50.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.760+0000 7f6ac759e700 1 -- 172.21.15.163:0/1607553608 learned_addr learned my addr 172.21.15.163:0/1607553608 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:50.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.760+0000 7f6ac6d9d700 1 --2- 172.21.15.163:0/1607553608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ac806bcd0 0x7f6ac8156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:50.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.760+0000 7f6abffff700 1 --2- 172.21.15.163:0/1607553608 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ac81414c0 0x7f6ac80c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:50.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.760+0000 7f6ac759e700 1 -- 172.21.15.163:0/1607553608 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ac81414c0 msgr2=0x7f6ac80c6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:50.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.760+0000 7f6ac759e700 1 --2- 172.21.15.163:0/1607553608 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ac81414c0 0x7f6ac80c6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:50.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.760+0000 7f6ac759e700 1 -- 172.21.15.163:0/1607553608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ac806bcd0 msgr2=0x7f6ac8156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:50.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.760+0000 7f6ac759e700 1 --2- 172.21.15.163:0/1607553608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ac806bcd0 0x7f6ac8156a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:50.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.760+0000 7f6ac759e700 1 -- 172.21.15.163:0/1607553608 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab8009420 con 0x7f6ac8158d40 2023-10-09T06:35:50.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.760+0000 7f6abffff700 1 --2- 172.21.15.163:0/1607553608 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ac81414c0 0x7f6ac80c6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:35:50.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.760+0000 7f6ac759e700 1 --2- 172.21.15.163:0/1607553608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ac8158d40 0x7f6ac81580b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000ccc0 tx=0x7f6ac0001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:50.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.760+0000 7f6abeffd700 1 -- 172.21.15.163:0/1607553608 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac000ab10 con 0x7f6ac8158d40 2023-10-09T06:35:50.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.760+0000 7f6abeffd700 1 -- 172.21.15.163:0/1607553608 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac000ac70 con 0x7f6ac8158d40 2023-10-09T06:35:50.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.760+0000 7f6acd247700 1 -- 172.21.15.163:0/1607553608 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ac813fa30 con 0x7f6ac8158d40 2023-10-09T06:35:50.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.761+0000 7f6abeffd700 1 -- 172.21.15.163:0/1607553608 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac00070f0 con 0x7f6ac8158d40 2023-10-09T06:35:50.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.761+0000 7f6acd247700 1 -- 172.21.15.163:0/1607553608 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ac813fff0 con 0x7f6ac8158d40 2023-10-09T06:35:50.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.761+0000 7f6acd247700 1 -- 172.21.15.163:0/1607553608 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ac81407f0 con 0x7f6ac8158d40 2023-10-09T06:35:50.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.762+0000 7f6abeffd700 1 -- 172.21.15.163:0/1607553608 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f6ac0007520 con 0x7f6ac8158d40 2023-10-09T06:35:50.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.762+0000 7f6abeffd700 1 --2- 172.21.15.163:0/1607553608 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6aa8067170 0x7f6aa8069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:50.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.762+0000 7f6abeffd700 1 -- 172.21.15.163:0/1607553608 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f6ac0084ab0 con 0x7f6ac8158d40 2023-10-09T06:35:50.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.763+0000 7f6ac6d9d700 1 --2- 172.21.15.163:0/1607553608 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6aa8067170 0x7f6aa8069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:50.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.763+0000 7f6ac6d9d700 1 --2- 172.21.15.163:0/1607553608 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6aa8067170 0x7f6aa8069620 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80030c0 tx=0x7f6ab8016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:50.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.767+0000 7f6abeffd700 1 -- 172.21.15.163:0/1607553608 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ac81407f0 con 0x7f6ac8158d40 2023-10-09T06:35:50.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:50.951+0000 7f6acd247700 1 -- 172.21.15.163:0/1607553608 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6ac81407f0 con 0x7f6ac8158d40 2023-10-09T06:35:51.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.620+0000 7f6abeffd700 1 -- 172.21.15.163:0/1607553608 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v75) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6ac81407f0 con 0x7f6ac8158d40 2023-10-09T06:35:51.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.625+0000 7f6acd247700 1 -- 172.21.15.163:0/1607553608 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6aa8067170 msgr2=0x7f6aa8069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:51.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.625+0000 7f6acd247700 1 --2- 172.21.15.163:0/1607553608 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6aa8067170 0x7f6aa8069620 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80030c0 tx=0x7f6ab8016000 comp rx=0 tx=0).stop 2023-10-09T06:35:51.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.625+0000 7f6acd247700 1 -- 172.21.15.163:0/1607553608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ac8158d40 msgr2=0x7f6ac81580b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:51.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.625+0000 7f6acd247700 1 --2- 172.21.15.163:0/1607553608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ac8158d40 0x7f6ac81580b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000ccc0 tx=0x7f6ac0001a20 comp rx=0 tx=0).stop 2023-10-09T06:35:51.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.630+0000 7f6acd247700 1 -- 172.21.15.163:0/1607553608 shutdown_connections 2023-10-09T06:35:51.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.630+0000 7f6acd247700 1 --2- 172.21.15.163:0/1607553608 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6aa8067170 0x7f6aa8069620 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:51.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.630+0000 7f6acd247700 1 --2- 172.21.15.163:0/1607553608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ac8158d40 0x7f6ac81580b0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:51.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.630+0000 7f6acd247700 1 --2- 172.21.15.163:0/1607553608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ac806bcd0 0x7f6ac8156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:51.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.630+0000 7f6acd247700 1 --2- 172.21.15.163:0/1607553608 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ac81414c0 0x7f6ac80c6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:51.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.630+0000 7f6acd247700 1 -- 172.21.15.163:0/1607553608 >> 172.21.15.163:0/1607553608 conn(0x7f6ac80ce5c0 msgr2=0x7f6ac8157800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:51.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.631+0000 7f6acd247700 1 -- 172.21.15.163:0/1607553608 shutdown_connections 2023-10-09T06:35:51.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.631+0000 7f6acd247700 1 -- 172.21.15.163:0/1607553608 wait complete. 2023-10-09T06:35:51.634 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_data' created 2023-10-09T06:35:51.649 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-09T06:35:51.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.766+0000 7f1a41789700 1 Processor -- start 2023-10-09T06:35:51.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.766+0000 7f1a41789700 1 -- start start 2023-10-09T06:35:51.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.767+0000 7f1a41789700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1a3c159c60 0x7f1a3c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:51.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.767+0000 7f1a41789700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1a3c15db10 0x7f1a3c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:51.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.767+0000 7f1a41789700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a3c15a160 con 0x7f1a3c159890 2023-10-09T06:35:51.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.767+0000 7f1a41789700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a3c15df50 con 0x7f1a3c159c60 2023-10-09T06:35:51.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.767+0000 7f1a41789700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a3c0d13a0 con 0x7f1a3c15db10 2023-10-09T06:35:51.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.767+0000 7f1a327fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1a3c159c60 0x7f1a3c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:51.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.767+0000 7f1a327fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1a3c159c60 0x7f1a3c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56046/0 (socket says 172.21.15.163:56046) 2023-10-09T06:35:51.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.767+0000 7f1a327fc700 1 -- 172.21.15.163:0/249537796 learned_addr learned my addr 172.21.15.163:0/249537796 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:51.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.767+0000 7f1a3b7fe700 1 --2- 172.21.15.163:0/249537796 >> v2:172.21.15.181:3300/0 conn(0x7f1a3c15db10 0x7f1a3c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:51.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.767+0000 7f1a327fc700 1 -- 172.21.15.163:0/249537796 >> v2:172.21.15.181:3300/0 conn(0x7f1a3c15db10 msgr2=0x7f1a3c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:51.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.767+0000 7f1a327fc700 1 --2- 172.21.15.163:0/249537796 >> v2:172.21.15.181:3300/0 conn(0x7f1a3c15db10 0x7f1a3c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:51.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.768+0000 7f1a327fc700 1 -- 172.21.15.163:0/249537796 >> v1:172.21.15.181:6789/0 conn(0x7f1a3c159890 legacy=0x7f1a3c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:51.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.768+0000 7f1a327fc700 1 -- 172.21.15.163:0/249537796 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3c15e140 con 0x7f1a3c159c60 2023-10-09T06:35:51.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.768+0000 7f1a3b7fe700 1 --2- 172.21.15.163:0/249537796 >> v2:172.21.15.181:3300/0 conn(0x7f1a3c15db10 0x7f1a3c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:35:51.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.768+0000 7f1a327fc700 1 --2- 172.21.15.163:0/249537796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a3c159c60 0x7f1a3c15d5d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c06a890 tx=0x7f1a24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49283cf93cd65c9d server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:51.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.768+0000 7f1a3a7fc700 1 -- 172.21.15.163:0/249537796 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2400b820 con 0x7f1a3c159c60 2023-10-09T06:35:51.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.768+0000 7f1a3a7fc700 1 -- 172.21.15.163:0/249537796 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2400b980 con 0x7f1a3c159c60 2023-10-09T06:35:51.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.768+0000 7f1a3a7fc700 1 -- 172.21.15.163:0/249537796 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2400bd20 con 0x7f1a3c159c60 2023-10-09T06:35:51.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.768+0000 7f1a41789700 1 -- 172.21.15.163:0/249537796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a3c159c60 msgr2=0x7f1a3c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:51.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.768+0000 7f1a41789700 1 --2- 172.21.15.163:0/249537796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a3c159c60 0x7f1a3c15d5d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c06a890 tx=0x7f1a24009250 comp rx=0 tx=0).stop 2023-10-09T06:35:51.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.768+0000 7f1a41789700 1 -- 172.21.15.163:0/249537796 shutdown_connections 2023-10-09T06:35:51.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.768+0000 7f1a41789700 1 --2- 172.21.15.163:0/249537796 >> v2:172.21.15.181:3300/0 conn(0x7f1a3c15db10 0x7f1a3c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:51.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.768+0000 7f1a41789700 1 --2- 172.21.15.163:0/249537796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a3c159c60 0x7f1a3c15d5d0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:51.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.768+0000 7f1a41789700 1 -- 172.21.15.163:0/249537796 >> 172.21.15.163:0/249537796 conn(0x7f1a3c068e20 msgr2=0x7f1a3c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:51.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.768+0000 7f1a41789700 1 -- 172.21.15.163:0/249537796 shutdown_connections 2023-10-09T06:35:51.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.768+0000 7f1a41789700 1 -- 172.21.15.163:0/249537796 wait complete. 2023-10-09T06:35:51.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.770+0000 7f1a41789700 1 Processor -- start 2023-10-09T06:35:51.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.770+0000 7f1a41789700 1 -- start start 2023-10-09T06:35:51.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.770+0000 7f1a41789700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a3c159890 0x7f1a3c0b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:51.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.770+0000 7f1a41789700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a3c159c60 0x7f1a3c14c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:51.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.770+0000 7f1a41789700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a3c15db10 0x7f1a3c153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:51.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.770+0000 7f1a41789700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c0b1e10 con 0x7f1a3c159890 2023-10-09T06:35:51.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.770+0000 7f1a41789700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c0b0580 con 0x7f1a3c15db10 2023-10-09T06:35:51.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.770+0000 7f1a41789700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c14c5e0 con 0x7f1a3c159c60 2023-10-09T06:35:51.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.770+0000 7f1a3affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a3c159890 0x7f1a3c0b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:51.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.770+0000 7f1a3affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a3c159890 0x7f1a3c0b18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56052/0 (socket says 172.21.15.163:56052) 2023-10-09T06:35:51.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.770+0000 7f1a3affd700 1 -- 172.21.15.163:0/2604439270 learned_addr learned my addr 172.21.15.163:0/2604439270 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:51.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.771+0000 7f1a327fc700 1 --2- 172.21.15.163:0/2604439270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a3c159c60 0x7f1a3c14c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:51.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.771+0000 7f1a3b7fe700 1 --2- 172.21.15.163:0/2604439270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a3c15db10 0x7f1a3c153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:51.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.771+0000 7f1a3affd700 1 -- 172.21.15.163:0/2604439270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a3c159c60 msgr2=0x7f1a3c14c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:51.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.771+0000 7f1a3affd700 1 --2- 172.21.15.163:0/2604439270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a3c159c60 0x7f1a3c14c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:51.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.771+0000 7f1a3affd700 1 -- 172.21.15.163:0/2604439270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a3c15db10 msgr2=0x7f1a3c153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:51.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.771+0000 7f1a3affd700 1 --2- 172.21.15.163:0/2604439270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a3c15db10 0x7f1a3c153930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:51.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.771+0000 7f1a3affd700 1 -- 172.21.15.163:0/2604439270 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a240094f0 con 0x7f1a3c159890 2023-10-09T06:35:51.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.771+0000 7f1a3b7fe700 1 --2- 172.21.15.163:0/2604439270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a3c15db10 0x7f1a3c153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:35:51.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.771+0000 7f1a3affd700 1 --2- 172.21.15.163:0/2604439270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a3c159890 0x7f1a3c0b18d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00c5d0 tx=0x7f1a2c007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:51.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.771+0000 7f1a337fe700 1 -- 172.21.15.163:0/2604439270 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c011590 con 0x7f1a3c159890 2023-10-09T06:35:51.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.771+0000 7f1a337fe700 1 -- 172.21.15.163:0/2604439270 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2c011710 con 0x7f1a3c159890 2023-10-09T06:35:51.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.771+0000 7f1a41789700 1 -- 172.21.15.163:0/2604439270 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a3c0b0790 con 0x7f1a3c159890 2023-10-09T06:35:51.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.771+0000 7f1a337fe700 1 -- 172.21.15.163:0/2604439270 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c011ad0 con 0x7f1a3c159890 2023-10-09T06:35:51.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.771+0000 7f1a41789700 1 -- 172.21.15.163:0/2604439270 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a3c0cf9c0 con 0x7f1a3c159890 2023-10-09T06:35:51.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.772+0000 7f1a41789700 1 -- 172.21.15.163:0/2604439270 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a3c04c3c0 con 0x7f1a3c159890 2023-10-09T06:35:51.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.773+0000 7f1a337fe700 1 -- 172.21.15.163:0/2604439270 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f1a2c01c460 con 0x7f1a3c159890 2023-10-09T06:35:51.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.773+0000 7f1a337fe700 1 --2- 172.21.15.163:0/2604439270 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a1c067170 0x7f1a1c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:51.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.773+0000 7f1a337fe700 1 -- 172.21.15.163:0/2604439270 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f1a2c084530 con 0x7f1a3c159890 2023-10-09T06:35:51.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.773+0000 7f1a327fc700 1 --2- 172.21.15.163:0/2604439270 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a1c067170 0x7f1a1c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:51.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.774+0000 7f1a327fc700 1 --2- 172.21.15.163:0/2604439270 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a1c067170 0x7f1a1c069620 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c06a890 tx=0x7f1a24016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:51.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:51.777+0000 7f1a337fe700 1 -- 172.21.15.163:0/2604439270 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a2c051570 con 0x7f1a3c159890 2023-10-09T06:35:52.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.000+0000 7f1a41789700 1 -- 172.21.15.163:0/2604439270 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1a3c06a5b0 con 0x7f1a3c159890 2023-10-09T06:35:52.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.649+0000 7f1a337fe700 1 -- 172.21.15.163:0/2604439270 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v45) v1 ==== 174+0+0 (secure 0 0 0) 0x7f1a2c054b90 con 0x7f1a3c159890 2023-10-09T06:35:52.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.657+0000 7f1a41789700 1 -- 172.21.15.163:0/2604439270 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a1c067170 msgr2=0x7f1a1c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:52.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.657+0000 7f1a41789700 1 --2- 172.21.15.163:0/2604439270 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a1c067170 0x7f1a1c069620 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c06a890 tx=0x7f1a24016000 comp rx=0 tx=0).stop 2023-10-09T06:35:52.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.657+0000 7f1a41789700 1 -- 172.21.15.163:0/2604439270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a3c159890 msgr2=0x7f1a3c0b18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:52.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.657+0000 7f1a41789700 1 --2- 172.21.15.163:0/2604439270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a3c159890 0x7f1a3c0b18d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00c5d0 tx=0x7f1a2c007f60 comp rx=0 tx=0).stop 2023-10-09T06:35:52.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.662+0000 7f1a41789700 1 -- 172.21.15.163:0/2604439270 shutdown_connections 2023-10-09T06:35:52.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.662+0000 7f1a41789700 1 --2- 172.21.15.163:0/2604439270 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1a1c067170 0x7f1a1c069620 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:52.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.662+0000 7f1a41789700 1 --2- 172.21.15.163:0/2604439270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1a3c159890 0x7f1a3c0b18d0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:52.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.662+0000 7f1a41789700 1 --2- 172.21.15.163:0/2604439270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a3c15db10 0x7f1a3c153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:52.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.662+0000 7f1a41789700 1 --2- 172.21.15.163:0/2604439270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a3c159c60 0x7f1a3c14c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:52.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.662+0000 7f1a41789700 1 -- 172.21.15.163:0/2604439270 >> 172.21.15.163:0/2604439270 conn(0x7f1a3c068e20 msgr2=0x7f1a3c14d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:52.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.662+0000 7f1a41789700 1 -- 172.21.15.163:0/2604439270 shutdown_connections 2023-10-09T06:35:52.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.662+0000 7f1a41789700 1 -- 172.21.15.163:0/2604439270 wait complete. 2023-10-09T06:35:52.665 INFO:teuthology.orchestra.run.smithi163.stderr:new fs with metadata pool 12 and data pool 13 2023-10-09T06:35:52.678 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:35:52.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.792+0000 7f27a0161700 1 Processor -- start 2023-10-09T06:35:52.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.792+0000 7f27a0161700 1 -- start start 2023-10-09T06:35:52.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.793+0000 7f27a0161700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f279806bcd0 0x7f27980caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:52.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.793+0000 7f27a0161700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f279815df50 0x7f279815e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:52.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.793+0000 7f27a0161700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f279815d7e0 con 0x7f279806c0a0 2023-10-09T06:35:52.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.793+0000 7f27a0161700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f279815dac0 con 0x7f279815df50 2023-10-09T06:35:52.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.793+0000 7f27a0161700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279815aa30 con 0x7f279806bcd0 2023-10-09T06:35:52.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.793+0000 7f279d6fc700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f279806c0a0 0x7f279815a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57686/0 (socket says 172.21.15.163:57686) 2023-10-09T06:35:52.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.793+0000 7f279d6fc700 1 -- 172.21.15.163:0/327341264 learned_addr learned my addr 172.21.15.163:0/327341264 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:35:52.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.794+0000 7f279defd700 1 --2- 172.21.15.163:0/327341264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f279806bcd0 0x7f27980caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:52.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.794+0000 7f279cefb700 1 -- 172.21.15.163:0/327341264 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2777819590 0 0) 0x7f279815d7e0 con 0x7f279806c0a0 2023-10-09T06:35:52.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.794+0000 7f279cefb700 1 -- 172.21.15.163:0/327341264 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2784003570 con 0x7f279806c0a0 2023-10-09T06:35:52.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.794+0000 7f279e6fe700 1 --2- 172.21.15.163:0/327341264 >> v2:172.21.15.181:3300/0 conn(0x7f279815df50 0x7f279815e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:52.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.794+0000 7f279cefb700 1 -- 172.21.15.163:0/327341264 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 492995801 0 0) 0x7f2784003570 con 0x7f279806c0a0 2023-10-09T06:35:52.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.794+0000 7f279cefb700 1 -- 172.21.15.163:0/327341264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f279806bcd0 msgr2=0x7f27980caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:52.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.794+0000 7f279cefb700 1 --2- 172.21.15.163:0/327341264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f279806bcd0 0x7f27980caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:52.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.794+0000 7f279cefb700 1 -- 172.21.15.163:0/327341264 >> v2:172.21.15.181:3300/0 conn(0x7f279815df50 msgr2=0x7f279815e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:52.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.794+0000 7f279cefb700 1 --2- 172.21.15.163:0/327341264 >> v2:172.21.15.181:3300/0 conn(0x7f279815df50 0x7f279815e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:52.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.794+0000 7f279cefb700 1 -- 172.21.15.163:0/327341264 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f279815ad10 con 0x7f279806c0a0 2023-10-09T06:35:52.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.794+0000 7f279cefb700 1 -- 172.21.15.163:0/327341264 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f2788001470 con 0x7f279806c0a0 2023-10-09T06:35:52.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.794+0000 7f279cefb700 1 -- 172.21.15.163:0/327341264 >> v1:172.21.15.163:6789/0 conn(0x7f279806c0a0 legacy=0x7f279815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:52.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.794+0000 7f279cefb700 1 --2- 172.21.15.163:0/327341264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f279806bcd0 0x7f2784003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:52.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.794+0000 7f279cefb700 1 --2- 172.21.15.163:0/327341264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2784003fe0 0x7f2784004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:52.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.794+0000 7f279cefb700 1 --2- 172.21.15.163:0/327341264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2784004d50 0x7f27840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:52.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.794+0000 7f279cefb700 1 -- 172.21.15.163:0/327341264 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279815dac0 con 0x7f2784004d50 2023-10-09T06:35:52.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.794+0000 7f279cefb700 1 -- 172.21.15.163:0/327341264 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279815aa30 con 0x7f2784003fe0 2023-10-09T06:35:52.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.794+0000 7f279cefb700 1 -- 172.21.15.163:0/327341264 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278400bb40 con 0x7f279806bcd0 2023-10-09T06:35:52.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.794+0000 7f279e6fe700 1 --2- 172.21.15.163:0/327341264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2784004d50 0x7f27840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:52.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.794+0000 7f279defd700 1 --2- 172.21.15.163:0/327341264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f279806bcd0 0x7f2784003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:52.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.794+0000 7f279d6fc700 1 --2- 172.21.15.163:0/327341264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2784003fe0 0x7f2784004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:52.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.794+0000 7f279e6fe700 1 -- 172.21.15.163:0/327341264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f279806bcd0 msgr2=0x7f2784003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:52.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.794+0000 7f279e6fe700 1 --2- 172.21.15.163:0/327341264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f279806bcd0 0x7f2784003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:52.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.794+0000 7f279e6fe700 1 -- 172.21.15.163:0/327341264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2784003fe0 msgr2=0x7f2784004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:52.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.794+0000 7f279e6fe700 1 --2- 172.21.15.163:0/327341264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2784003fe0 0x7f2784004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:52.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.795+0000 7f279e6fe700 1 -- 172.21.15.163:0/327341264 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2784003570 con 0x7f2784004d50 2023-10-09T06:35:52.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.795+0000 7f279e6fe700 1 --2- 172.21.15.163:0/327341264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2784004d50 0x7f27840051c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f27900094d0 tx=0x7f279000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a6573318b597ae server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:52.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.795+0000 7f279cefb700 1 -- 172.21.15.163:0/327341264 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279000ea90 con 0x7f2784004d50 2023-10-09T06:35:52.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.795+0000 7f279cefb700 1 -- 172.21.15.163:0/327341264 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279000f040 con 0x7f2784004d50 2023-10-09T06:35:52.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.795+0000 7f279cefb700 1 -- 172.21.15.163:0/327341264 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279000ee20 con 0x7f2784004d50 2023-10-09T06:35:52.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.795+0000 7f27a0161700 1 -- 172.21.15.163:0/327341264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2784004d50 msgr2=0x7f27840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:52.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.795+0000 7f27a0161700 1 --2- 172.21.15.163:0/327341264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2784004d50 0x7f27840051c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f27900094d0 tx=0x7f279000b040 comp rx=0 tx=0).stop 2023-10-09T06:35:52.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.795+0000 7f279e6fe700 1 -- 172.21.15.163:0/327341264 reap_dead start 2023-10-09T06:35:52.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.795+0000 7f27a0161700 1 -- 172.21.15.163:0/327341264 shutdown_connections 2023-10-09T06:35:52.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.795+0000 7f27a0161700 1 -- 172.21.15.163:0/327341264 >> 172.21.15.163:0/327341264 conn(0x7f27980ce600 msgr2=0x7f27980cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:52.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.795+0000 7f27a0161700 1 -- 172.21.15.163:0/327341264 shutdown_connections 2023-10-09T06:35:52.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.795+0000 7f27a0161700 1 -- 172.21.15.163:0/327341264 wait complete. 2023-10-09T06:35:52.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.796+0000 7f27a0161700 1 Processor -- start 2023-10-09T06:35:52.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.797+0000 7f27a0161700 1 -- start start 2023-10-09T06:35:52.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.797+0000 7f27a0161700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2798154410 0x7f2798154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:52.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.797+0000 7f27a0161700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f279815a240 0x7f27980c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:52.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.797+0000 7f27a0161700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27980c43b0 0x7f2798158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:52.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.797+0000 7f27a0161700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27980b2a20 con 0x7f2798154410 2023-10-09T06:35:52.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.797+0000 7f27a0161700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27980c4820 con 0x7f279815a240 2023-10-09T06:35:52.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.797+0000 7f27a0161700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27980b2bb0 con 0x7f27980c43b0 2023-10-09T06:35:52.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.797+0000 7f279defd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2798154410 0x7f2798154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:52.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.797+0000 7f279defd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2798154410 0x7f2798154cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56076/0 (socket says 172.21.15.163:56076) 2023-10-09T06:35:52.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.797+0000 7f279defd700 1 -- 172.21.15.163:0/3642983129 learned_addr learned my addr 172.21.15.163:0/3642983129 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:52.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.797+0000 7f279d6fc700 1 --2- 172.21.15.163:0/3642983129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f279815a240 0x7f27980c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:52.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.797+0000 7f279e6fe700 1 --2- 172.21.15.163:0/3642983129 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27980c43b0 0x7f2798158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:52.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.798+0000 7f279defd700 1 -- 172.21.15.163:0/3642983129 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27980c43b0 msgr2=0x7f2798158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:52.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.798+0000 7f279defd700 1 --2- 172.21.15.163:0/3642983129 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27980c43b0 0x7f2798158300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:52.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.798+0000 7f279defd700 1 -- 172.21.15.163:0/3642983129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f279815a240 msgr2=0x7f27980c3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:52.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.798+0000 7f279defd700 1 --2- 172.21.15.163:0/3642983129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f279815a240 0x7f27980c3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:52.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.798+0000 7f279defd700 1 -- 172.21.15.163:0/3642983129 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2798156660 con 0x7f2798154410 2023-10-09T06:35:52.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.798+0000 7f279defd700 1 --2- 172.21.15.163:0/3642983129 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2798154410 0x7f2798154cf0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2794006ac0 tx=0x7f27940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:52.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.798+0000 7f278dffb700 1 -- 172.21.15.163:0/3642983129 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279400f940 con 0x7f2798154410 2023-10-09T06:35:52.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.798+0000 7f278dffb700 1 -- 172.21.15.163:0/3642983129 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279400faa0 con 0x7f2798154410 2023-10-09T06:35:52.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.798+0000 7f278dffb700 1 -- 172.21.15.163:0/3642983129 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279400fe20 con 0x7f2798154410 2023-10-09T06:35:52.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.798+0000 7f27a0161700 1 -- 172.21.15.163:0/3642983129 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27981552d0 con 0x7f2798154410 2023-10-09T06:35:52.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.798+0000 7f27a0161700 1 -- 172.21.15.163:0/3642983129 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2798155890 con 0x7f2798154410 2023-10-09T06:35:52.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.799+0000 7f27a0161700 1 -- 172.21.15.163:0/3642983129 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27980648d0 con 0x7f2798154410 2023-10-09T06:35:52.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.799+0000 7f278dffb700 1 -- 172.21.15.163:0/3642983129 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f2794014650 con 0x7f2798154410 2023-10-09T06:35:52.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.800+0000 7f278dffb700 1 --2- 172.21.15.163:0/3642983129 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f277c067170 0x7f277c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:52.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.800+0000 7f278dffb700 1 -- 172.21.15.163:0/3642983129 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2794018030 con 0x7f2798154410 2023-10-09T06:35:52.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.800+0000 7f279d6fc700 1 --2- 172.21.15.163:0/3642983129 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f277c067170 0x7f277c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:52.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.800+0000 7f279d6fc700 1 --2- 172.21.15.163:0/3642983129 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f277c067170 0x7f277c069620 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f27880010e0 tx=0x7f2788001e90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:52.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.804+0000 7f278dffb700 1 -- 172.21.15.163:0/3642983129 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f27940521c0 con 0x7f2798154410 2023-10-09T06:35:52.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.989+0000 7f27a0161700 1 -- 172.21.15.163:0/3642983129 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f279815a610 con 0x7f2798154410 2023-10-09T06:35:52.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.989+0000 7f278dffb700 1 -- 172.21.15.163:0/3642983129 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+15414 (secure 0 0 0) 0x7f27940557e0 con 0x7f2798154410 2023-10-09T06:35:52.990 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:35:52.990 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":76,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:35:52.002379+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:35:50.332005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-09T06:35:50.969956+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:35:52.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.991+0000 7f27a0161700 1 -- 172.21.15.163:0/3642983129 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f277c067170 msgr2=0x7f277c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:52.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.991+0000 7f27a0161700 1 --2- 172.21.15.163:0/3642983129 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f277c067170 0x7f277c069620 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f27880010e0 tx=0x7f2788001e90 comp rx=0 tx=0).stop 2023-10-09T06:35:52.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.991+0000 7f27a0161700 1 -- 172.21.15.163:0/3642983129 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2798154410 msgr2=0x7f2798154cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:52.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.991+0000 7f27a0161700 1 --2- 172.21.15.163:0/3642983129 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2798154410 0x7f2798154cf0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2794006ac0 tx=0x7f27940091a0 comp rx=0 tx=0).stop 2023-10-09T06:35:52.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.992+0000 7f27a0161700 1 -- 172.21.15.163:0/3642983129 shutdown_connections 2023-10-09T06:35:52.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.992+0000 7f27a0161700 1 --2- 172.21.15.163:0/3642983129 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f277c067170 0x7f277c069620 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:52.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.992+0000 7f27a0161700 1 --2- 172.21.15.163:0/3642983129 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2798154410 0x7f2798154cf0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:52.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.992+0000 7f27a0161700 1 --2- 172.21.15.163:0/3642983129 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27980c43b0 0x7f2798158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:52.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.992+0000 7f27a0161700 1 --2- 172.21.15.163:0/3642983129 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f279815a240 0x7f27980c3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:52.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.992+0000 7f27a0161700 1 -- 172.21.15.163:0/3642983129 >> 172.21.15.163:0/3642983129 conn(0x7f27980ce600 msgr2=0x7f279815bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:52.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.992+0000 7f27a0161700 1 -- 172.21.15.163:0/3642983129 shutdown_connections 2023-10-09T06:35:52.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:52.992+0000 7f27a0161700 1 -- 172.21.15.163:0/3642983129 wait complete. 2023-10-09T06:35:53.005 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:35:53.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.119+0000 7fec3bc81700 1 Processor -- start 2023-10-09T06:35:53.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.119+0000 7fec3bc81700 1 -- start start 2023-10-09T06:35:53.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.120+0000 7fec3bc81700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fec34159c60 0x7fec3415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:53.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.120+0000 7fec3bc81700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fec3415db10 0x7fec34069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:53.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.120+0000 7fec3bc81700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec3415a160 con 0x7fec34159890 2023-10-09T06:35:53.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.120+0000 7fec3bc81700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec3415df50 con 0x7fec3415db10 2023-10-09T06:35:53.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.120+0000 7fec3bc81700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec340d13a0 con 0x7fec34159c60 2023-10-09T06:35:53.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.120+0000 7fec3a21e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fec3415db10 0x7fec34069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:53.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.120+0000 7fec39a1d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fec34159890 0x7fec3415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57688/0 (socket says 172.21.15.163:57688) 2023-10-09T06:35:53.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.120+0000 7fec39a1d700 1 -- 172.21.15.163:0/3529480333 learned_addr learned my addr 172.21.15.163:0/3529480333 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:35:53.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.121+0000 7fec3921c700 1 --2- 172.21.15.163:0/3529480333 >> v2:172.21.15.181:3300/0 conn(0x7fec34159c60 0x7fec3415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:53.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.121+0000 7fec3a21e700 1 -- 172.21.15.163:0/3529480333 >> v2:172.21.15.181:3300/0 conn(0x7fec34159c60 msgr2=0x7fec3415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:53.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.121+0000 7fec3a21e700 1 --2- 172.21.15.163:0/3529480333 >> v2:172.21.15.181:3300/0 conn(0x7fec34159c60 0x7fec3415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:53.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.121+0000 7fec3a21e700 1 -- 172.21.15.163:0/3529480333 >> v1:172.21.15.163:6789/0 conn(0x7fec34159890 legacy=0x7fec3415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:53.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.121+0000 7fec38a1b700 1 -- 172.21.15.163:0/3529480333 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 175709020 0 0) 0x7fec3415a160 con 0x7fec34159890 2023-10-09T06:35:53.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.121+0000 7fec3a21e700 1 -- 172.21.15.163:0/3529480333 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3415e140 con 0x7fec3415db10 2023-10-09T06:35:53.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.121+0000 7fec3a21e700 1 --2- 172.21.15.163:0/3529480333 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec3415db10 0x7fec34069560 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fec340b67f0 tx=0x7fec2c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4254105d30ad9c84 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:53.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.121+0000 7fec38a1b700 1 -- 172.21.15.163:0/3529480333 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c00e070 con 0x7fec3415db10 2023-10-09T06:35:53.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.121+0000 7fec38a1b700 1 -- 172.21.15.163:0/3529480333 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2c00bc30 con 0x7fec3415db10 2023-10-09T06:35:53.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.121+0000 7fec38a1b700 1 -- 172.21.15.163:0/3529480333 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c012430 con 0x7fec3415db10 2023-10-09T06:35:53.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.121+0000 7fec3bc81700 1 -- 172.21.15.163:0/3529480333 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec3415db10 msgr2=0x7fec34069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:53.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.121+0000 7fec3bc81700 1 --2- 172.21.15.163:0/3529480333 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec3415db10 0x7fec34069560 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fec340b67f0 tx=0x7fec2c009780 comp rx=0 tx=0).stop 2023-10-09T06:35:53.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.121+0000 7fec3bc81700 1 -- 172.21.15.163:0/3529480333 shutdown_connections 2023-10-09T06:35:53.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.121+0000 7fec3bc81700 1 --2- 172.21.15.163:0/3529480333 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec3415db10 0x7fec34069560 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:53.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.121+0000 7fec3bc81700 1 --2- 172.21.15.163:0/3529480333 >> v2:172.21.15.181:3300/0 conn(0x7fec34159c60 0x7fec3415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:53.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.121+0000 7fec3bc81700 1 -- 172.21.15.163:0/3529480333 >> 172.21.15.163:0/3529480333 conn(0x7fec34068e20 msgr2=0x7fec340d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:53.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.122+0000 7fec3bc81700 1 -- 172.21.15.163:0/3529480333 shutdown_connections 2023-10-09T06:35:53.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.122+0000 7fec3bc81700 1 -- 172.21.15.163:0/3529480333 wait complete. 2023-10-09T06:35:53.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.123+0000 7fec3bc81700 1 Processor -- start 2023-10-09T06:35:53.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.123+0000 7fec3bc81700 1 -- start start 2023-10-09T06:35:53.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.123+0000 7fec3bc81700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec34159890 0x7fec340cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:53.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.123+0000 7fec3bc81700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec34159c60 0x7fec340cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:53.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.123+0000 7fec3bc81700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec3415db10 0x7fec34148e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:53.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.123+0000 7fec3bc81700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec340b0540 con 0x7fec3415db10 2023-10-09T06:35:53.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.123+0000 7fec3bc81700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3416d490 con 0x7fec34159890 2023-10-09T06:35:53.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.123+0000 7fec3bc81700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec3416d770 con 0x7fec34159c60 2023-10-09T06:35:53.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.123+0000 7fec3a21e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec3415db10 0x7fec34148e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:53.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.123+0000 7fec3a21e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec3415db10 0x7fec34148e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56098/0 (socket says 172.21.15.163:56098) 2023-10-09T06:35:53.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.123+0000 7fec3a21e700 1 -- 172.21.15.163:0/2475309041 learned_addr learned my addr 172.21.15.163:0/2475309041 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:53.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.123+0000 7fec39a1d700 1 --2- 172.21.15.163:0/2475309041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec34159890 0x7fec340cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:53.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.123+0000 7fec3921c700 1 --2- 172.21.15.163:0/2475309041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec34159c60 0x7fec340cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:53.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.124+0000 7fec3a21e700 1 -- 172.21.15.163:0/2475309041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec34159c60 msgr2=0x7fec340cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:53.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.124+0000 7fec3a21e700 1 --2- 172.21.15.163:0/2475309041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec34159c60 0x7fec340cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:53.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.124+0000 7fec3a21e700 1 -- 172.21.15.163:0/2475309041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec34159890 msgr2=0x7fec340cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:53.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.124+0000 7fec3a21e700 1 --2- 172.21.15.163:0/2475309041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec34159890 0x7fec340cf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:53.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.124+0000 7fec3a21e700 1 -- 172.21.15.163:0/2475309041 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2c009a20 con 0x7fec3415db10 2023-10-09T06:35:53.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.124+0000 7fec3a21e700 1 --2- 172.21.15.163:0/2475309041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec3415db10 0x7fec34148e00 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fec34070e20 tx=0x7fec2c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:53.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.124+0000 7fec29ffb700 1 -- 172.21.15.163:0/2475309041 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c027470 con 0x7fec3415db10 2023-10-09T06:35:53.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.124+0000 7fec29ffb700 1 -- 172.21.15.163:0/2475309041 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2c0275d0 con 0x7fec3415db10 2023-10-09T06:35:53.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.124+0000 7fec3bc81700 1 -- 172.21.15.163:0/2475309041 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec3416da50 con 0x7fec3415db10 2023-10-09T06:35:53.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.124+0000 7fec29ffb700 1 -- 172.21.15.163:0/2475309041 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c027990 con 0x7fec3415db10 2023-10-09T06:35:53.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.124+0000 7fec3bc81700 1 -- 172.21.15.163:0/2475309041 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec340cfce0 con 0x7fec3415db10 2023-10-09T06:35:53.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.125+0000 7fec3bc81700 1 -- 172.21.15.163:0/2475309041 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec340b06f0 con 0x7fec3415db10 2023-10-09T06:35:53.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.126+0000 7fec29ffb700 1 -- 172.21.15.163:0/2475309041 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fec2c024070 con 0x7fec3415db10 2023-10-09T06:35:53.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.126+0000 7fec29ffb700 1 --2- 172.21.15.163:0/2475309041 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fec18067170 0x7fec18069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:53.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.126+0000 7fec29ffb700 1 -- 172.21.15.163:0/2475309041 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fec2c086ef0 con 0x7fec3415db10 2023-10-09T06:35:53.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.126+0000 7fec39a1d700 1 --2- 172.21.15.163:0/2475309041 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fec18067170 0x7fec18069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:53.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.127+0000 7fec39a1d700 1 --2- 172.21.15.163:0/2475309041 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fec18067170 0x7fec18069620 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fec30007fd0 tx=0x7fec30007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:53.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.130+0000 7fec29ffb700 1 -- 172.21.15.163:0/2475309041 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec2c053f80 con 0x7fec3415db10 2023-10-09T06:35:53.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.316+0000 7fec3bc81700 1 -- 172.21.15.163:0/2475309041 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec3404c3c0 con 0x7fec3415db10 2023-10-09T06:35:53.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.317+0000 7fec29ffb700 1 -- 172.21.15.163:0/2475309041 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+15414 (secure 0 0 0) 0x7fec2c0575a0 con 0x7fec3415db10 2023-10-09T06:35:53.317 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:35:53.318 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":76,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:35:52.002379+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:35:50.332005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-09T06:35:50.969956+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:35:53.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.319+0000 7fec3bc81700 1 -- 172.21.15.163:0/2475309041 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fec18067170 msgr2=0x7fec18069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:53.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.319+0000 7fec3bc81700 1 --2- 172.21.15.163:0/2475309041 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fec18067170 0x7fec18069620 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fec30007fd0 tx=0x7fec30007d30 comp rx=0 tx=0).stop 2023-10-09T06:35:53.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.319+0000 7fec3bc81700 1 -- 172.21.15.163:0/2475309041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec3415db10 msgr2=0x7fec34148e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:53.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.319+0000 7fec3bc81700 1 --2- 172.21.15.163:0/2475309041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec3415db10 0x7fec34148e00 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fec34070e20 tx=0x7fec2c019000 comp rx=0 tx=0).stop 2023-10-09T06:35:53.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.319+0000 7fec3bc81700 1 -- 172.21.15.163:0/2475309041 shutdown_connections 2023-10-09T06:35:53.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.319+0000 7fec3bc81700 1 --2- 172.21.15.163:0/2475309041 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fec18067170 0x7fec18069620 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:53.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.319+0000 7fec3bc81700 1 --2- 172.21.15.163:0/2475309041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec3415db10 0x7fec34148e00 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:53.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.319+0000 7fec3bc81700 1 --2- 172.21.15.163:0/2475309041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec34159890 0x7fec340cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:53.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.319+0000 7fec3bc81700 1 --2- 172.21.15.163:0/2475309041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec34159c60 0x7fec340cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:53.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.319+0000 7fec3bc81700 1 -- 172.21.15.163:0/2475309041 >> 172.21.15.163:0/2475309041 conn(0x7fec34068e20 msgr2=0x7fec341552f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:53.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.319+0000 7fec3bc81700 1 -- 172.21.15.163:0/2475309041 shutdown_connections 2023-10-09T06:35:53.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.319+0000 7fec3bc81700 1 -- 172.21.15.163:0/2475309041 wait complete. 2023-10-09T06:35:53.332 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-09T06:35:53.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.448+0000 7fd4e4147700 1 Processor -- start 2023-10-09T06:35:53.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.448+0000 7fd4e4147700 1 -- start start 2023-10-09T06:35:53.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.449+0000 7fd4e4147700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd4dc06c0a0 0x7fd4dc0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:53.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.449+0000 7fd4e4147700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd4dc15df50 0x7fd4dc15e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:53.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.449+0000 7fd4e4147700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4dc15d7e0 con 0x7fd4dc06bcd0 2023-10-09T06:35:53.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.449+0000 7fd4e4147700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4dc15dac0 con 0x7fd4dc06c0a0 2023-10-09T06:35:53.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.449+0000 7fd4e4147700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4dc15aa30 con 0x7fd4dc15df50 2023-10-09T06:35:53.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.449+0000 7fd4e16e2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd4dc06c0a0 0x7fd4dc0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:53.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.449+0000 7fd4e16e2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd4dc06c0a0 0x7fd4dc0caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56102/0 (socket says 172.21.15.163:56102) 2023-10-09T06:35:53.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.449+0000 7fd4e16e2700 1 -- 172.21.15.163:0/776689065 learned_addr learned my addr 172.21.15.163:0/776689065 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:53.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.450+0000 7fd4e26e4700 1 --2- 172.21.15.163:0/776689065 >> v2:172.21.15.181:3300/0 conn(0x7fd4dc15df50 0x7fd4dc15e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:53.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.450+0000 7fd4e16e2700 1 -- 172.21.15.163:0/776689065 >> v2:172.21.15.181:3300/0 conn(0x7fd4dc15df50 msgr2=0x7fd4dc15e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:53.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.450+0000 7fd4e16e2700 1 --2- 172.21.15.163:0/776689065 >> v2:172.21.15.181:3300/0 conn(0x7fd4dc15df50 0x7fd4dc15e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:53.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.450+0000 7fd4e16e2700 1 -- 172.21.15.163:0/776689065 >> v1:172.21.15.181:6789/0 conn(0x7fd4dc06bcd0 legacy=0x7fd4dc15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:53.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.450+0000 7fd4e16e2700 1 -- 172.21.15.163:0/776689065 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4dc15ad10 con 0x7fd4dc06c0a0 2023-10-09T06:35:53.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.450+0000 7fd4e16e2700 1 --2- 172.21.15.163:0/776689065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4dc06c0a0 0x7fd4dc0caa90 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc15d720 tx=0x7fd4cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1fdd4a8d752353 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:53.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.450+0000 7fd4e0ee1700 1 -- 172.21.15.163:0/776689065 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc00b820 con 0x7fd4dc06c0a0 2023-10-09T06:35:53.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.450+0000 7fd4e0ee1700 1 -- 172.21.15.163:0/776689065 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4cc00b980 con 0x7fd4dc06c0a0 2023-10-09T06:35:53.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.450+0000 7fd4e0ee1700 1 -- 172.21.15.163:0/776689065 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc00bd40 con 0x7fd4dc06c0a0 2023-10-09T06:35:53.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.451+0000 7fd4e4147700 1 -- 172.21.15.163:0/776689065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4dc06c0a0 msgr2=0x7fd4dc0caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:53.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.451+0000 7fd4e4147700 1 --2- 172.21.15.163:0/776689065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4dc06c0a0 0x7fd4dc0caa90 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc15d720 tx=0x7fd4cc009250 comp rx=0 tx=0).stop 2023-10-09T06:35:53.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.451+0000 7fd4e4147700 1 -- 172.21.15.163:0/776689065 shutdown_connections 2023-10-09T06:35:53.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.451+0000 7fd4e4147700 1 --2- 172.21.15.163:0/776689065 >> v2:172.21.15.181:3300/0 conn(0x7fd4dc15df50 0x7fd4dc15e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:53.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.451+0000 7fd4e4147700 1 --2- 172.21.15.163:0/776689065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4dc06c0a0 0x7fd4dc0caa90 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:53.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.451+0000 7fd4e4147700 1 -- 172.21.15.163:0/776689065 >> 172.21.15.163:0/776689065 conn(0x7fd4dc0ce600 msgr2=0x7fd4dc0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:53.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.451+0000 7fd4e4147700 1 -- 172.21.15.163:0/776689065 shutdown_connections 2023-10-09T06:35:53.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.451+0000 7fd4e4147700 1 -- 172.21.15.163:0/776689065 wait complete. 2023-10-09T06:35:53.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.452+0000 7fd4e4147700 1 Processor -- start 2023-10-09T06:35:53.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.452+0000 7fd4e4147700 1 -- start start 2023-10-09T06:35:53.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.452+0000 7fd4e4147700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4dc06bcd0 0x7fd4dc0b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:53.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.452+0000 7fd4e4147700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4dc06c0a0 0x7fd4dc150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:53.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.452+0000 7fd4e4147700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4dc15df50 0x7fd4dc0b2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:53.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.452+0000 7fd4e4147700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc1514f0 con 0x7fd4dc15df50 2023-10-09T06:35:53.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.452+0000 7fd4e4147700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc0b2b70 con 0x7fd4dc06bcd0 2023-10-09T06:35:53.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.452+0000 7fd4e4147700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc0b2da0 con 0x7fd4dc06c0a0 2023-10-09T06:35:53.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.453+0000 7fd4e26e4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4dc15df50 0x7fd4dc0b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:53.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.453+0000 7fd4e26e4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4dc15df50 0x7fd4dc0b2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56118/0 (socket says 172.21.15.163:56118) 2023-10-09T06:35:53.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.453+0000 7fd4e26e4700 1 -- 172.21.15.163:0/3111872992 learned_addr learned my addr 172.21.15.163:0/3111872992 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:53.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.453+0000 7fd4e1ee3700 1 --2- 172.21.15.163:0/3111872992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4dc06bcd0 0x7fd4dc0b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:53.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.453+0000 7fd4e16e2700 1 --2- 172.21.15.163:0/3111872992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4dc06c0a0 0x7fd4dc150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:53.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.453+0000 7fd4e26e4700 1 -- 172.21.15.163:0/3111872992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4dc06c0a0 msgr2=0x7fd4dc150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:53.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.453+0000 7fd4e26e4700 1 --2- 172.21.15.163:0/3111872992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4dc06c0a0 0x7fd4dc150fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:53.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.453+0000 7fd4e26e4700 1 -- 172.21.15.163:0/3111872992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4dc06bcd0 msgr2=0x7fd4dc0b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:53.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.453+0000 7fd4e26e4700 1 --2- 172.21.15.163:0/3111872992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4dc06bcd0 0x7fd4dc0b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:53.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.453+0000 7fd4e26e4700 1 -- 172.21.15.163:0/3111872992 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc0094f0 con 0x7fd4dc15df50 2023-10-09T06:35:53.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.453+0000 7fd4e16e2700 1 --2- 172.21.15.163:0/3111872992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4dc06c0a0 0x7fd4dc150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:35:53.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.453+0000 7fd4e26e4700 1 --2- 172.21.15.163:0/3111872992 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4dc15df50 0x7fd4dc0b2630 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400d860 tx=0x7fd4d4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:53.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.453+0000 7fd4d1ffb700 1 -- 172.21.15.163:0/3111872992 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d40125f0 con 0x7fd4dc15df50 2023-10-09T06:35:53.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.453+0000 7fd4d1ffb700 1 -- 172.21.15.163:0/3111872992 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d4012770 con 0x7fd4dc15df50 2023-10-09T06:35:53.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.453+0000 7fd4e4147700 1 -- 172.21.15.163:0/3111872992 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4dc0b3080 con 0x7fd4dc15df50 2023-10-09T06:35:53.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.453+0000 7fd4d1ffb700 1 -- 172.21.15.163:0/3111872992 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4012b30 con 0x7fd4dc15df50 2023-10-09T06:35:53.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.453+0000 7fd4e4147700 1 -- 172.21.15.163:0/3111872992 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4dc0b3640 con 0x7fd4dc15df50 2023-10-09T06:35:53.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.454+0000 7fd4e4147700 1 -- 172.21.15.163:0/3111872992 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4dc04c3c0 con 0x7fd4dc15df50 2023-10-09T06:35:53.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.455+0000 7fd4d1ffb700 1 -- 172.21.15.163:0/3111872992 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fd4d401e460 con 0x7fd4dc15df50 2023-10-09T06:35:53.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.455+0000 7fd4d1ffb700 1 --2- 172.21.15.163:0/3111872992 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd4c0067170 0x7fd4c0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:53.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.455+0000 7fd4d1ffb700 1 -- 172.21.15.163:0/3111872992 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd4d4084e20 con 0x7fd4dc15df50 2023-10-09T06:35:53.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.455+0000 7fd4e1ee3700 1 --2- 172.21.15.163:0/3111872992 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd4c0067170 0x7fd4c0069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:53.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.456+0000 7fd4e1ee3700 1 --2- 172.21.15.163:0/3111872992 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd4c0067170 0x7fd4c0069620 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8007560 tx=0x7fd4d8009760 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:53.460 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.459+0000 7fd4d1ffb700 1 -- 172.21.15.163:0/3111872992 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd4d4055890 con 0x7fd4dc15df50 2023-10-09T06:35:53.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:53.688+0000 7fd4e4147700 1 -- 172.21.15.163:0/3111872992 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd4dc15aff0 con 0x7fd4dc15df50 2023-10-09T06:35:54.322 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-10-09T06:35:54.322 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-10-09T06:35:54.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.654+0000 7fd4d1ffb700 1 -- 172.21.15.163:0/3111872992 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v47) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd4d4055420 con 0x7fd4dc15df50 2023-10-09T06:35:54.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.657+0000 7fd4e4147700 1 -- 172.21.15.163:0/3111872992 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd4c0067170 msgr2=0x7fd4c0069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:54.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.657+0000 7fd4e4147700 1 --2- 172.21.15.163:0/3111872992 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd4c0067170 0x7fd4c0069620 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8007560 tx=0x7fd4d8009760 comp rx=0 tx=0).stop 2023-10-09T06:35:54.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.657+0000 7fd4e4147700 1 -- 172.21.15.163:0/3111872992 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4dc15df50 msgr2=0x7fd4dc0b2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:54.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.657+0000 7fd4e4147700 1 --2- 172.21.15.163:0/3111872992 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4dc15df50 0x7fd4dc0b2630 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400d860 tx=0x7fd4d4008040 comp rx=0 tx=0).stop 2023-10-09T06:35:54.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.657+0000 7fd4e4147700 1 -- 172.21.15.163:0/3111872992 shutdown_connections 2023-10-09T06:35:54.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.657+0000 7fd4e4147700 1 --2- 172.21.15.163:0/3111872992 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd4c0067170 0x7fd4c0069620 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:54.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.657+0000 7fd4e4147700 1 --2- 172.21.15.163:0/3111872992 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4dc15df50 0x7fd4dc0b2630 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:54.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.657+0000 7fd4e4147700 1 --2- 172.21.15.163:0/3111872992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4dc06bcd0 0x7fd4dc0b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:54.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.657+0000 7fd4e4147700 1 --2- 172.21.15.163:0/3111872992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4dc06c0a0 0x7fd4dc150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:54.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.657+0000 7fd4e4147700 1 -- 172.21.15.163:0/3111872992 >> 172.21.15.163:0/3111872992 conn(0x7fd4dc0ce600 msgr2=0x7fd4dc15c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:54.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.657+0000 7fd4e4147700 1 -- 172.21.15.163:0/3111872992 shutdown_connections 2023-10-09T06:35:54.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.657+0000 7fd4e4147700 1 -- 172.21.15.163:0/3111872992 wait complete. 2023-10-09T06:35:54.673 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-09T06:35:54.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.795+0000 7fc3b9320700 1 Processor -- start 2023-10-09T06:35:54.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.795+0000 7fc3b9320700 1 -- start start 2023-10-09T06:35:54.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.796+0000 7fc3b9320700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3b40b4e70 0x7fc3b40ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:54.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.796+0000 7fc3b9320700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc3b40b16d0 0x7fc3b40b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:54.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.796+0000 7fc3b9320700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc3b415d240 0x7fc3b415d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:54.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.796+0000 7fc3b9320700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3b40b2080 con 0x7fc3b415d240 2023-10-09T06:35:54.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.796+0000 7fc3b9320700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3b40cfb00 con 0x7fc3b40b16d0 2023-10-09T06:35:54.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.796+0000 7fc3b9320700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b40b5c40 con 0x7fc3b40b4e70 2023-10-09T06:35:54.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.796+0000 7fc3b8b1f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc3b415d240 0x7fc3b415d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:54.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.796+0000 7fc3b8b1f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc3b415d240 0x7fc3b415d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56124/0 (socket says 172.21.15.163:56124) 2023-10-09T06:35:54.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.796+0000 7fc3b8b1f700 1 -- 172.21.15.163:0/929386276 learned_addr learned my addr 172.21.15.163:0/929386276 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:54.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.796+0000 7fc3b3fff700 1 --2- 172.21.15.163:0/929386276 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3b40b4e70 0x7fc3b40ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:54.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.796+0000 7fc3b37fe700 1 --2- 172.21.15.163:0/929386276 >> v2:172.21.15.181:3300/0 conn(0x7fc3b40b16d0 0x7fc3b40b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:54.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.796+0000 7fc3b8b1f700 1 -- 172.21.15.163:0/929386276 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3b40b4e70 msgr2=0x7fc3b40ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:54.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.796+0000 7fc3b8b1f700 1 --2- 172.21.15.163:0/929386276 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3b40b4e70 0x7fc3b40ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:54.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.796+0000 7fc3b8b1f700 1 -- 172.21.15.163:0/929386276 >> v2:172.21.15.181:3300/0 conn(0x7fc3b40b16d0 msgr2=0x7fc3b40b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:54.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.796+0000 7fc3b8b1f700 1 --2- 172.21.15.163:0/929386276 >> v2:172.21.15.181:3300/0 conn(0x7fc3b40b16d0 0x7fc3b40b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:54.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.796+0000 7fc3b8b1f700 1 -- 172.21.15.163:0/929386276 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b40b5f20 con 0x7fc3b415d240 2023-10-09T06:35:54.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.796+0000 7fc3b37fe700 1 --2- 172.21.15.163:0/929386276 >> v2:172.21.15.181:3300/0 conn(0x7fc3b40b16d0 0x7fc3b40b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:35:54.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.797+0000 7fc3b8b1f700 1 --2- 172.21.15.163:0/929386276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3b415d240 0x7fc3b415d6c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4089440 tx=0x7fc3ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d2429664dbf6c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:54.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.797+0000 7fc3b2ffd700 1 -- 172.21.15.163:0/929386276 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac00e070 con 0x7fc3b415d240 2023-10-09T06:35:54.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.797+0000 7fc3b2ffd700 1 -- 172.21.15.163:0/929386276 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ac00ba50 con 0x7fc3b415d240 2023-10-09T06:35:54.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.797+0000 7fc3b2ffd700 1 -- 172.21.15.163:0/929386276 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac0092e0 con 0x7fc3b415d240 2023-10-09T06:35:54.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.797+0000 7fc3b9320700 1 -- 172.21.15.163:0/929386276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3b415d240 msgr2=0x7fc3b415d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:54.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.797+0000 7fc3b9320700 1 --2- 172.21.15.163:0/929386276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3b415d240 0x7fc3b415d6c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4089440 tx=0x7fc3ac009840 comp rx=0 tx=0).stop 2023-10-09T06:35:54.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.797+0000 7fc3b9320700 1 -- 172.21.15.163:0/929386276 shutdown_connections 2023-10-09T06:35:54.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.797+0000 7fc3b9320700 1 --2- 172.21.15.163:0/929386276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3b415d240 0x7fc3b415d6c0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:54.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.797+0000 7fc3b9320700 1 --2- 172.21.15.163:0/929386276 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3b40b4e70 0x7fc3b40ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:54.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.797+0000 7fc3b9320700 1 --2- 172.21.15.163:0/929386276 >> v2:172.21.15.181:3300/0 conn(0x7fc3b40b16d0 0x7fc3b40b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:54.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.797+0000 7fc3b9320700 1 -- 172.21.15.163:0/929386276 >> 172.21.15.163:0/929386276 conn(0x7fc3b40d10c0 msgr2=0x7fc3b40affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:54.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.797+0000 7fc3b9320700 1 -- 172.21.15.163:0/929386276 shutdown_connections 2023-10-09T06:35:54.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.797+0000 7fc3b9320700 1 -- 172.21.15.163:0/929386276 wait complete. 2023-10-09T06:35:54.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.798+0000 7fc3b9320700 1 Processor -- start 2023-10-09T06:35:54.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.799+0000 7fc3b9320700 1 -- start start 2023-10-09T06:35:54.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.799+0000 7fc3b9320700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3b40b16d0 0x7fc3b414b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:54.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.799+0000 7fc3b9320700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3b40b4e70 0x7fc3b40cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:54.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.799+0000 7fc3b9320700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3b415d240 0x7fc3b415f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:54.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.799+0000 7fc3b9320700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b40c8090 con 0x7fc3b415d240 2023-10-09T06:35:54.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.799+0000 7fc3b9320700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b40c7410 con 0x7fc3b40b4e70 2023-10-09T06:35:54.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.799+0000 7fc3b9320700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b40c76f0 con 0x7fc3b40b16d0 2023-10-09T06:35:54.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.799+0000 7fc3b8b1f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3b415d240 0x7fc3b415f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:54.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.799+0000 7fc3b8b1f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3b415d240 0x7fc3b415f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56132/0 (socket says 172.21.15.163:56132) 2023-10-09T06:35:54.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.799+0000 7fc3b8b1f700 1 -- 172.21.15.163:0/3742195282 learned_addr learned my addr 172.21.15.163:0/3742195282 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:54.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.800+0000 7fc3b3fff700 1 --2- 172.21.15.163:0/3742195282 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3b40b16d0 0x7fc3b414b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:54.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.800+0000 7fc3b8b1f700 1 -- 172.21.15.163:0/3742195282 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3b40b16d0 msgr2=0x7fc3b414b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:54.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.800+0000 7fc3b8b1f700 1 --2- 172.21.15.163:0/3742195282 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3b40b16d0 0x7fc3b414b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:54.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.800+0000 7fc3b8b1f700 1 -- 172.21.15.163:0/3742195282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3b40b4e70 msgr2=0x7fc3b40cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:54.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.800+0000 7fc3b37fe700 1 --2- 172.21.15.163:0/3742195282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3b40b4e70 0x7fc3b40cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:54.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.800+0000 7fc3b8b1f700 1 --2- 172.21.15.163:0/3742195282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3b40b4e70 0x7fc3b40cba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:54.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.800+0000 7fc3b8b1f700 1 -- 172.21.15.163:0/3742195282 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ac009ae0 con 0x7fc3b415d240 2023-10-09T06:35:54.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.800+0000 7fc3b3fff700 1 --2- 172.21.15.163:0/3742195282 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3b40b16d0 0x7fc3b414b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:35:54.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.800+0000 7fc3b8b1f700 1 --2- 172.21.15.163:0/3742195282 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3b415d240 0x7fc3b415f790 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac009fd0 tx=0x7fc3ac018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:54.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.800+0000 7fc39bfff700 1 -- 172.21.15.163:0/3742195282 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac0094e0 con 0x7fc3b415d240 2023-10-09T06:35:54.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.800+0000 7fc39bfff700 1 -- 172.21.15.163:0/3742195282 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ac0032f0 con 0x7fc3b415d240 2023-10-09T06:35:54.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.800+0000 7fc3b9320700 1 -- 172.21.15.163:0/3742195282 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b40c79d0 con 0x7fc3b415d240 2023-10-09T06:35:54.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.800+0000 7fc39bfff700 1 -- 172.21.15.163:0/3742195282 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac025690 con 0x7fc3b415d240 2023-10-09T06:35:54.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.800+0000 7fc3b9320700 1 -- 172.21.15.163:0/3742195282 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b40c70e0 con 0x7fc3b415d240 2023-10-09T06:35:54.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.801+0000 7fc3b9320700 1 -- 172.21.15.163:0/3742195282 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b40c81d0 con 0x7fc3b415d240 2023-10-09T06:35:54.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.801+0000 7fc39bfff700 1 -- 172.21.15.163:0/3742195282 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fc3ac003040 con 0x7fc3b415d240 2023-10-09T06:35:54.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.802+0000 7fc39bfff700 1 --2- 172.21.15.163:0/3742195282 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc3a0067170 0x7fc3a0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:54.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.802+0000 7fc39bfff700 1 -- 172.21.15.163:0/3742195282 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc3ac0865d0 con 0x7fc3b415d240 2023-10-09T06:35:54.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.802+0000 7fc3b3fff700 1 --2- 172.21.15.163:0/3742195282 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc3a0067170 0x7fc3a0069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:54.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.802+0000 7fc3b3fff700 1 --2- 172.21.15.163:0/3742195282 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc3a0067170 0x7fc3a0069620 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4006fd0 tx=0x7fc3a4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:54.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:54.806+0000 7fc39bfff700 1 -- 172.21.15.163:0/3742195282 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc3b40c81d0 con 0x7fc3b415d240 2023-10-09T06:35:55.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.029+0000 7fc3b9320700 1 -- 172.21.15.163:0/3742195282 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc3b40c81d0 con 0x7fc3b415d240 2023-10-09T06:35:55.662 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.661+0000 7fc39bfff700 1 -- 172.21.15.163:0/3742195282 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v48) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc3b40c81d0 con 0x7fc3b415d240 2023-10-09T06:35:55.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.664+0000 7fc3b9320700 1 -- 172.21.15.163:0/3742195282 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc3a0067170 msgr2=0x7fc3a0069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:55.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.664+0000 7fc3b9320700 1 --2- 172.21.15.163:0/3742195282 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc3a0067170 0x7fc3a0069620 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4006fd0 tx=0x7fc3a4008040 comp rx=0 tx=0).stop 2023-10-09T06:35:55.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.664+0000 7fc3b9320700 1 -- 172.21.15.163:0/3742195282 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3b415d240 msgr2=0x7fc3b415f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:55.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.664+0000 7fc3b9320700 1 --2- 172.21.15.163:0/3742195282 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3b415d240 0x7fc3b415f790 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac009fd0 tx=0x7fc3ac018000 comp rx=0 tx=0).stop 2023-10-09T06:35:55.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.664+0000 7fc3b9320700 1 -- 172.21.15.163:0/3742195282 shutdown_connections 2023-10-09T06:35:55.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.664+0000 7fc3b9320700 1 --2- 172.21.15.163:0/3742195282 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc3a0067170 0x7fc3a0069620 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:55.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.664+0000 7fc3b9320700 1 --2- 172.21.15.163:0/3742195282 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3b415d240 0x7fc3b415f790 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:55.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.664+0000 7fc3b9320700 1 --2- 172.21.15.163:0/3742195282 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3b40b16d0 0x7fc3b414b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:55.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.664+0000 7fc3b9320700 1 --2- 172.21.15.163:0/3742195282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3b40b4e70 0x7fc3b40cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:55.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.664+0000 7fc3b9320700 1 -- 172.21.15.163:0/3742195282 >> 172.21.15.163:0/3742195282 conn(0x7fc3b40d10c0 msgr2=0x7fc3b4068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:55.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.664+0000 7fc3b9320700 1 -- 172.21.15.163:0/3742195282 shutdown_connections 2023-10-09T06:35:55.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.664+0000 7fc3b9320700 1 -- 172.21.15.163:0/3742195282 wait complete. 2023-10-09T06:35:55.678 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:35:55.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.792+0000 7fef6c32b700 1 Processor -- start 2023-10-09T06:35:55.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.792+0000 7fef6c32b700 1 -- start start 2023-10-09T06:35:55.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.793+0000 7fef6c32b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fef64159c60 0x7fef6415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:55.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.793+0000 7fef6c32b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef6415a160 con 0x7fef6415db10 2023-10-09T06:35:55.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.793+0000 7fef6c32b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef640d13a0 con 0x7fef64159890 2023-10-09T06:35:55.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.793+0000 7fef6c32b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef6415df50 con 0x7fef64159c60 2023-10-09T06:35:55.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.793+0000 7fef6a8c8700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fef6415db10 0x7fef64069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57696/0 (socket says 172.21.15.163:57696) 2023-10-09T06:35:55.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.793+0000 7fef6a8c8700 1 -- 172.21.15.163:0/1878481978 learned_addr learned my addr 172.21.15.163:0/1878481978 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:35:55.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.794+0000 7fef698c6700 1 --2- 172.21.15.163:0/1878481978 >> v2:172.21.15.181:3300/0 conn(0x7fef64159c60 0x7fef6415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:55.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.794+0000 7fef690c5700 1 -- 172.21.15.163:0/1878481978 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3996767979 0 0) 0x7fef6415a160 con 0x7fef6415db10 2023-10-09T06:35:55.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.794+0000 7fef690c5700 1 -- 172.21.15.163:0/1878481978 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef50003570 con 0x7fef6415db10 2023-10-09T06:35:55.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.794+0000 7fef690c5700 1 -- 172.21.15.163:0/1878481978 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2562877157 0 0) 0x7fef50003570 con 0x7fef6415db10 2023-10-09T06:35:55.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.794+0000 7fef690c5700 1 -- 172.21.15.163:0/1878481978 >> v2:172.21.15.181:3300/0 conn(0x7fef64159c60 msgr2=0x7fef6415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:55.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.794+0000 7fef690c5700 1 --2- 172.21.15.163:0/1878481978 >> v2:172.21.15.181:3300/0 conn(0x7fef64159c60 0x7fef6415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:55.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.794+0000 7fef690c5700 1 -- 172.21.15.163:0/1878481978 >> v1:172.21.15.181:6789/0 conn(0x7fef64159890 legacy=0x7fef6415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:55.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.794+0000 7fef690c5700 1 -- 172.21.15.163:0/1878481978 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6415e140 con 0x7fef6415db10 2023-10-09T06:35:55.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.794+0000 7fef690c5700 1 -- 172.21.15.163:0/1878481978 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fef5c0017f0 con 0x7fef6415db10 2023-10-09T06:35:55.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.794+0000 7fef690c5700 1 -- 172.21.15.163:0/1878481978 >> v1:172.21.15.163:6789/0 conn(0x7fef6415db10 legacy=0x7fef64069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:55.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.794+0000 7fef690c5700 1 --2- 172.21.15.163:0/1878481978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef50003a60 0x7fef50003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:55.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.794+0000 7fef690c5700 1 --2- 172.21.15.163:0/1878481978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef50004810 0x7fef50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:55.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.794+0000 7fef690c5700 1 --2- 172.21.15.163:0/1878481978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef500051c0 0x7fef5000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:55.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.794+0000 7fef690c5700 1 -- 172.21.15.163:0/1878481978 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5000c0a0 con 0x7fef50003a60 2023-10-09T06:35:55.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.794+0000 7fef690c5700 1 -- 172.21.15.163:0/1878481978 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5000c210 con 0x7fef50004810 2023-10-09T06:35:55.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.794+0000 7fef690c5700 1 -- 172.21.15.163:0/1878481978 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef5000c3d0 con 0x7fef500051c0 2023-10-09T06:35:55.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.794+0000 7fef6a0c7700 1 --2- 172.21.15.163:0/1878481978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef50003a60 0x7fef50003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:55.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.794+0000 7fef698c6700 1 --2- 172.21.15.163:0/1878481978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef50004810 0x7fef50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:55.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.794+0000 7fef6a8c8700 1 --2- 172.21.15.163:0/1878481978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef500051c0 0x7fef5000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:55.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.794+0000 7fef6a0c7700 1 -- 172.21.15.163:0/1878481978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef500051c0 msgr2=0x7fef5000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:55.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.794+0000 7fef6a0c7700 1 --2- 172.21.15.163:0/1878481978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef500051c0 0x7fef5000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:55.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.794+0000 7fef6a0c7700 1 -- 172.21.15.163:0/1878481978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef50004810 msgr2=0x7fef50004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:55.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.794+0000 7fef6a0c7700 1 --2- 172.21.15.163:0/1878481978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef50004810 0x7fef50004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:55.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.794+0000 7fef6a0c7700 1 -- 172.21.15.163:0/1878481978 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef50003570 con 0x7fef50003a60 2023-10-09T06:35:55.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.794+0000 7fef6a8c8700 1 --2- 172.21.15.163:0/1878481978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef500051c0 0x7fef5000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:35:55.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.794+0000 7fef6a8c8700 1 -- 172.21.15.163:0/1878481978 reap_dead start 2023-10-09T06:35:55.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.794+0000 7fef6a0c7700 1 --2- 172.21.15.163:0/1878481978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef50003a60 0x7fef50003e70 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fef600018b0 tx=0x7fef60007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f576449ff951842b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:55.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.795+0000 7fef690c5700 1 -- 172.21.15.163:0/1878481978 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef60012420 con 0x7fef50003a60 2023-10-09T06:35:55.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.795+0000 7fef690c5700 1 -- 172.21.15.163:0/1878481978 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef600125a0 con 0x7fef50003a60 2023-10-09T06:35:55.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.795+0000 7fef690c5700 1 -- 172.21.15.163:0/1878481978 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef60012960 con 0x7fef50003a60 2023-10-09T06:35:55.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.795+0000 7fef6c32b700 1 -- 172.21.15.163:0/1878481978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef50003a60 msgr2=0x7fef50003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:55.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.795+0000 7fef6c32b700 1 --2- 172.21.15.163:0/1878481978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef50003a60 0x7fef50003e70 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fef600018b0 tx=0x7fef60007d30 comp rx=0 tx=0).stop 2023-10-09T06:35:55.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.795+0000 7fef6c32b700 1 -- 172.21.15.163:0/1878481978 shutdown_connections 2023-10-09T06:35:55.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.795+0000 7fef6c32b700 1 --2- 172.21.15.163:0/1878481978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef50003a60 0x7fef50003e70 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:55.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.795+0000 7fef6c32b700 1 -- 172.21.15.163:0/1878481978 >> 172.21.15.163:0/1878481978 conn(0x7fef64068e20 msgr2=0x7fef640d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:55.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.795+0000 7fef6c32b700 1 -- 172.21.15.163:0/1878481978 shutdown_connections 2023-10-09T06:35:55.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.795+0000 7fef6c32b700 1 -- 172.21.15.163:0/1878481978 wait complete. 2023-10-09T06:35:55.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.796+0000 7fef6c32b700 1 Processor -- start 2023-10-09T06:35:55.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.796+0000 7fef6c32b700 1 -- start start 2023-10-09T06:35:55.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.797+0000 7fef6c32b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef50003a60 0x7fef640cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:55.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.797+0000 7fef6c32b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef640b2980 0x7fef6416cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:55.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.797+0000 7fef6c32b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef64148e90 0x7fef6416d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:55.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.797+0000 7fef6c32b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef640b45f0 con 0x7fef640b2980 2023-10-09T06:35:55.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.797+0000 7fef6c32b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef640b5da0 con 0x7fef50003a60 2023-10-09T06:35:55.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.797+0000 7fef6c32b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef640b6040 con 0x7fef64148e90 2023-10-09T06:35:55.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.797+0000 7fef698c6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef640b2980 0x7fef6416cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:55.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.797+0000 7fef698c6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef640b2980 0x7fef6416cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56144/0 (socket says 172.21.15.163:56144) 2023-10-09T06:35:55.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.797+0000 7fef698c6700 1 -- 172.21.15.163:0/3057598864 learned_addr learned my addr 172.21.15.163:0/3057598864 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:55.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.797+0000 7fef6a8c8700 1 --2- 172.21.15.163:0/3057598864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef64148e90 0x7fef6416d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:55.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.797+0000 7fef6a0c7700 1 --2- 172.21.15.163:0/3057598864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef50003a60 0x7fef640cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:55.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.797+0000 7fef698c6700 1 -- 172.21.15.163:0/3057598864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef64148e90 msgr2=0x7fef6416d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:55.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.797+0000 7fef698c6700 1 --2- 172.21.15.163:0/3057598864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef64148e90 0x7fef6416d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:55.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.797+0000 7fef698c6700 1 -- 172.21.15.163:0/3057598864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef50003a60 msgr2=0x7fef640cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:55.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.797+0000 7fef698c6700 1 --2- 172.21.15.163:0/3057598864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef50003a60 0x7fef640cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:55.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.797+0000 7fef698c6700 1 -- 172.21.15.163:0/3057598864 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef640b6320 con 0x7fef640b2980 2023-10-09T06:35:55.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.797+0000 7fef6a0c7700 1 --2- 172.21.15.163:0/3057598864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef50003a60 0x7fef640cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:35:55.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.798+0000 7fef698c6700 1 --2- 172.21.15.163:0/3057598864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef640b2980 0x7fef6416cce0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fef54011d40 tx=0x7fef540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:55.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.798+0000 7fef5a7fc700 1 -- 172.21.15.163:0/3057598864 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5400fac0 con 0x7fef640b2980 2023-10-09T06:35:55.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.798+0000 7fef5a7fc700 1 -- 172.21.15.163:0/3057598864 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5400fc20 con 0x7fef640b2980 2023-10-09T06:35:55.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.798+0000 7fef6c32b700 1 -- 172.21.15.163:0/3057598864 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef640b4280 con 0x7fef640b2980 2023-10-09T06:35:55.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.798+0000 7fef5a7fc700 1 -- 172.21.15.163:0/3057598864 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef54014440 con 0x7fef640b2980 2023-10-09T06:35:55.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.798+0000 7fef6c32b700 1 -- 172.21.15.163:0/3057598864 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef640c98c0 con 0x7fef640b2980 2023-10-09T06:35:55.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.799+0000 7fef6c32b700 1 -- 172.21.15.163:0/3057598864 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef640b4760 con 0x7fef640b2980 2023-10-09T06:35:55.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.799+0000 7fef5a7fc700 1 -- 172.21.15.163:0/3057598864 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fef54014870 con 0x7fef640b2980 2023-10-09T06:35:55.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.799+0000 7fef5a7fc700 1 --2- 172.21.15.163:0/3057598864 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fef48067170 0x7fef48069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:55.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.799+0000 7fef5a7fc700 1 -- 172.21.15.163:0/3057598864 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fef54018030 con 0x7fef640b2980 2023-10-09T06:35:55.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.800+0000 7fef6a0c7700 1 --2- 172.21.15.163:0/3057598864 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fef48067170 0x7fef48069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:55.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.800+0000 7fef6a0c7700 1 --2- 172.21.15.163:0/3057598864 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fef48067170 0x7fef48069620 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fef60001690 tx=0x7fef60015000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:55.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:55.803+0000 7fef5a7fc700 1 -- 172.21.15.163:0/3057598864 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fef54052260 con 0x7fef640b2980 2023-10-09T06:35:56.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.027+0000 7fef6c32b700 1 -- 172.21.15.163:0/3057598864 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef6404c3c0 con 0x7fef640b2980 2023-10-09T06:35:56.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.028+0000 7fef5a7fc700 1 -- 172.21.15.163:0/3057598864 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4608 (secure 0 0 0) 0x7fef54055880 con 0x7fef640b2980 2023-10-09T06:35:56.029 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:35:56.029 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1946291256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1946291256},{"type":"v1","addr":"172.21.15.181:6837","nonce":1946291256}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5749,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1366951098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1366951098},{"type":"v1","addr":"172.21.15.181:6836","nonce":1366951098}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3619934997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3619934997},{"type":"v1","addr":"172.21.15.163:6834","nonce":3619934997}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:35:52.002698+0000","modified":"2023-10-09T06:35:55.653628+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/234029086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":234029086},{"type":"v1","addr":"172.21.15.163:6835","nonce":234029086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-09T06:35:56.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.030+0000 7fef6c32b700 1 -- 172.21.15.163:0/3057598864 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fef48067170 msgr2=0x7fef48069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:56.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.030+0000 7fef6c32b700 1 --2- 172.21.15.163:0/3057598864 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fef48067170 0x7fef48069620 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fef60001690 tx=0x7fef60015000 comp rx=0 tx=0).stop 2023-10-09T06:35:56.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.030+0000 7fef6c32b700 1 -- 172.21.15.163:0/3057598864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef640b2980 msgr2=0x7fef6416cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:56.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.030+0000 7fef6c32b700 1 --2- 172.21.15.163:0/3057598864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef640b2980 0x7fef6416cce0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fef54011d40 tx=0x7fef540091a0 comp rx=0 tx=0).stop 2023-10-09T06:35:56.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.030+0000 7fef6c32b700 1 -- 172.21.15.163:0/3057598864 shutdown_connections 2023-10-09T06:35:56.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.030+0000 7fef6c32b700 1 --2- 172.21.15.163:0/3057598864 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fef48067170 0x7fef48069620 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:56.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.030+0000 7fef6c32b700 1 --2- 172.21.15.163:0/3057598864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef64148e90 0x7fef6416d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:56.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.030+0000 7fef6c32b700 1 --2- 172.21.15.163:0/3057598864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef50003a60 0x7fef640cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:56.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.030+0000 7fef6c32b700 1 --2- 172.21.15.163:0/3057598864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef640b2980 0x7fef6416cce0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:56.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.030+0000 7fef6c32b700 1 -- 172.21.15.163:0/3057598864 >> 172.21.15.163:0/3057598864 conn(0x7fef64068e20 msgr2=0x7fef64149c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:56.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.030+0000 7fef6c32b700 1 -- 172.21.15.163:0/3057598864 shutdown_connections 2023-10-09T06:35:56.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.030+0000 7fef6c32b700 1 -- 172.21.15.163:0/3057598864 wait complete. 2023-10-09T06:35:56.032 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 48 2023-10-09T06:35:56.043 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:35:56.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.158+0000 7f9a0e909700 1 Processor -- start 2023-10-09T06:35:56.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.158+0000 7f9a0e909700 1 -- start start 2023-10-09T06:35:56.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.159+0000 7f9a0e909700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a081597f0 0x7f9a080b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:56.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.159+0000 7f9a0e909700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9a0815d230 0x7f9a0815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:56.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.159+0000 7f9a0e909700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a0815dbd0 con 0x7f9a08159420 2023-10-09T06:35:56.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.159+0000 7f9a0e909700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a080b11d0 con 0x7f9a0815d230 2023-10-09T06:35:56.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.159+0000 7f9a0e909700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a080b14b0 con 0x7f9a081597f0 2023-10-09T06:35:56.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.159+0000 7f9a0e108700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9a0815d230 0x7f9a0815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:56.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.159+0000 7f9a0e108700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9a0815d230 0x7f9a0815d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56160/0 (socket says 172.21.15.163:56160) 2023-10-09T06:35:56.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.159+0000 7f9a0e108700 1 -- 172.21.15.163:0/1053515507 learned_addr learned my addr 172.21.15.163:0/1053515507 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:56.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.159+0000 7f9a0d106700 1 --2- 172.21.15.163:0/1053515507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a081597f0 0x7f9a080b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:56.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.159+0000 7f9a0e108700 1 -- 172.21.15.163:0/1053515507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a081597f0 msgr2=0x7f9a080b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:56.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.159+0000 7f9a0e108700 1 --2- 172.21.15.163:0/1053515507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a081597f0 0x7f9a080b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:56.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.159+0000 7f9a0e108700 1 -- 172.21.15.163:0/1053515507 >> v1:172.21.15.181:6789/0 conn(0x7f9a08159420 legacy=0x7f9a080b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:56.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.159+0000 7f9a0e108700 1 -- 172.21.15.163:0/1053515507 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a080b1790 con 0x7f9a0815d230 2023-10-09T06:35:56.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.159+0000 7f9a0d106700 1 --2- 172.21.15.163:0/1053515507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a081597f0 0x7f9a080b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:35:56.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.159+0000 7f9a0e108700 1 --2- 172.21.15.163:0/1053515507 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0815d230 0x7f9a0815d690 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f9a00009f20 tx=0x7f9a00009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a72b22b6e53e6690 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:56.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.160+0000 7f9a0c905700 1 -- 172.21.15.163:0/1053515507 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0000e070 con 0x7f9a0815d230 2023-10-09T06:35:56.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.160+0000 7f9a0c905700 1 -- 172.21.15.163:0/1053515507 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0000ba80 con 0x7f9a0815d230 2023-10-09T06:35:56.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.160+0000 7f9a0c905700 1 -- 172.21.15.163:0/1053515507 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a000092e0 con 0x7f9a0815d230 2023-10-09T06:35:56.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.160+0000 7f9a0e909700 1 -- 172.21.15.163:0/1053515507 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0815d230 msgr2=0x7f9a0815d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:56.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.160+0000 7f9a0e909700 1 --2- 172.21.15.163:0/1053515507 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0815d230 0x7f9a0815d690 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f9a00009f20 tx=0x7f9a00009840 comp rx=0 tx=0).stop 2023-10-09T06:35:56.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.160+0000 7f9a0e909700 1 -- 172.21.15.163:0/1053515507 shutdown_connections 2023-10-09T06:35:56.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.160+0000 7f9a0e909700 1 --2- 172.21.15.163:0/1053515507 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0815d230 0x7f9a0815d690 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:56.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.160+0000 7f9a0e909700 1 --2- 172.21.15.163:0/1053515507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a081597f0 0x7f9a080b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:56.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.160+0000 7f9a0e909700 1 -- 172.21.15.163:0/1053515507 >> 172.21.15.163:0/1053515507 conn(0x7f9a080d0660 msgr2=0x7f9a080d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:56.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.160+0000 7f9a0e909700 1 -- 172.21.15.163:0/1053515507 shutdown_connections 2023-10-09T06:35:56.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.160+0000 7f9a0e909700 1 -- 172.21.15.163:0/1053515507 wait complete. 2023-10-09T06:35:56.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.161+0000 7f9a0e909700 1 Processor -- start 2023-10-09T06:35:56.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.161+0000 7f9a0e909700 1 -- start start 2023-10-09T06:35:56.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.162+0000 7f9a0e909700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a08159420 0x7f9a08157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:56.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.162+0000 7f9a0e909700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a081597f0 0x7f9a080c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:56.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.162+0000 7f9a0e909700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0815d230 0x7f9a080b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:56.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.162+0000 7f9a0e909700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a080c9280 con 0x7f9a0815d230 2023-10-09T06:35:56.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.162+0000 7f9a0e909700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a080c9410 con 0x7f9a08159420 2023-10-09T06:35:56.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.162+0000 7f9a0e909700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a081576c0 con 0x7f9a081597f0 2023-10-09T06:35:56.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.162+0000 7f9a0e108700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0815d230 0x7f9a080b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:56.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.162+0000 7f9a0e108700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0815d230 0x7f9a080b5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56174/0 (socket says 172.21.15.163:56174) 2023-10-09T06:35:56.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.162+0000 7f9a0e108700 1 -- 172.21.15.163:0/2896082410 learned_addr learned my addr 172.21.15.163:0/2896082410 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:56.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.162+0000 7f9a0e108700 1 -- 172.21.15.163:0/2896082410 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a081597f0 msgr2=0x7f9a080c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:56.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.162+0000 7f9a0d106700 1 --2- 172.21.15.163:0/2896082410 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a081597f0 0x7f9a080c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:56.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.162+0000 7f9a0e108700 1 --2- 172.21.15.163:0/2896082410 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a081597f0 0x7f9a080c6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:56.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.162+0000 7f9a0e108700 1 -- 172.21.15.163:0/2896082410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a08159420 msgr2=0x7f9a08157180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:56.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.162+0000 7f9a0d907700 1 --2- 172.21.15.163:0/2896082410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a08159420 0x7f9a08157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:56.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.163+0000 7f9a0e108700 1 --2- 172.21.15.163:0/2896082410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a08159420 0x7f9a08157180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:56.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.163+0000 7f9a0e108700 1 -- 172.21.15.163:0/2896082410 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a00009ae0 con 0x7f9a0815d230 2023-10-09T06:35:56.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.163+0000 7f9a0d907700 1 --2- 172.21.15.163:0/2896082410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a08159420 0x7f9a08157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:35:56.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.163+0000 7f9a0e108700 1 --2- 172.21.15.163:0/2896082410 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0815d230 0x7f9a080b5a90 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9a00008950 tx=0x7f9a00018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:56.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.163+0000 7f99fdffb700 1 -- 172.21.15.163:0/2896082410 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a00008ea0 con 0x7f9a0815d230 2023-10-09T06:35:56.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.163+0000 7f99fdffb700 1 -- 172.21.15.163:0/2896082410 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a000032f0 con 0x7f9a0815d230 2023-10-09T06:35:56.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.163+0000 7f9a0e909700 1 -- 172.21.15.163:0/2896082410 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a080c73d0 con 0x7f9a0815d230 2023-10-09T06:35:56.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.163+0000 7f99fdffb700 1 -- 172.21.15.163:0/2896082410 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0001c660 con 0x7f9a0815d230 2023-10-09T06:35:56.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.163+0000 7f9a0e909700 1 -- 172.21.15.163:0/2896082410 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a080c7c70 con 0x7f9a0815d230 2023-10-09T06:35:56.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.164+0000 7f9a0e909700 1 -- 172.21.15.163:0/2896082410 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a080c95c0 con 0x7f9a0815d230 2023-10-09T06:35:56.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.164+0000 7f99fdffb700 1 -- 172.21.15.163:0/2896082410 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f9a00003040 con 0x7f9a0815d230 2023-10-09T06:35:56.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.165+0000 7f99fdffb700 1 --2- 172.21.15.163:0/2896082410 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f99ec067170 0x7f99ec069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:56.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.165+0000 7f99fdffb700 1 -- 172.21.15.163:0/2896082410 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9a00086240 con 0x7f9a0815d230 2023-10-09T06:35:56.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.165+0000 7f9a0d907700 1 --2- 172.21.15.163:0/2896082410 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f99ec067170 0x7f99ec069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:56.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.165+0000 7f9a0d907700 1 --2- 172.21.15.163:0/2896082410 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f99ec067170 0x7f99ec069620 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9a04001840 tx=0x7f9a04009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:56.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.169+0000 7f99fdffb700 1 -- 172.21.15.163:0/2896082410 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a00053110 con 0x7f9a0815d230 2023-10-09T06:35:56.357 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.356+0000 7f9a0e909700 1 -- 172.21.15.163:0/2896082410 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a080b1a70 con 0x7f9a0815d230 2023-10-09T06:35:56.357 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.357+0000 7f99fdffb700 1 -- 172.21.15.163:0/2896082410 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+15401 (secure 0 0 0) 0x7f9a00056730 con 0x7f9a0815d230 2023-10-09T06:35:56.358 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:35:56.358 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":77,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:35:53.645012+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:35:50.332005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-09T06:35:50.969956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:35:56.360 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.359+0000 7f9a0e909700 1 -- 172.21.15.163:0/2896082410 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f99ec067170 msgr2=0x7f99ec069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:56.360 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.359+0000 7f9a0e909700 1 --2- 172.21.15.163:0/2896082410 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f99ec067170 0x7f99ec069620 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9a04001840 tx=0x7f9a04009040 comp rx=0 tx=0).stop 2023-10-09T06:35:56.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.359+0000 7f9a0e909700 1 -- 172.21.15.163:0/2896082410 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0815d230 msgr2=0x7f9a080b5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:56.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.359+0000 7f9a0e909700 1 --2- 172.21.15.163:0/2896082410 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0815d230 0x7f9a080b5a90 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9a00008950 tx=0x7f9a00018000 comp rx=0 tx=0).stop 2023-10-09T06:35:56.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.359+0000 7f9a0e909700 1 -- 172.21.15.163:0/2896082410 shutdown_connections 2023-10-09T06:35:56.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.359+0000 7f9a0e909700 1 --2- 172.21.15.163:0/2896082410 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f99ec067170 0x7f99ec069620 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:56.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.359+0000 7f9a0e909700 1 --2- 172.21.15.163:0/2896082410 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0815d230 0x7f9a080b5a90 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:56.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.359+0000 7f9a0e909700 1 --2- 172.21.15.163:0/2896082410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a08159420 0x7f9a08157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:56.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.359+0000 7f9a0e909700 1 --2- 172.21.15.163:0/2896082410 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a081597f0 0x7f9a080c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:56.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.359+0000 7f9a0e909700 1 -- 172.21.15.163:0/2896082410 >> 172.21.15.163:0/2896082410 conn(0x7f9a080d0660 msgr2=0x7f9a080ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:56.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.359+0000 7f9a0e909700 1 -- 172.21.15.163:0/2896082410 shutdown_connections 2023-10-09T06:35:56.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.359+0000 7f9a0e909700 1 -- 172.21.15.163:0/2896082410 wait complete. 2023-10-09T06:35:56.373 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-09T06:35:56.373 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:35:56.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.488+0000 7f39d17b6700 1 Processor -- start 2023-10-09T06:35:56.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.488+0000 7f39d17b6700 1 -- start start 2023-10-09T06:35:56.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.489+0000 7f39d17b6700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f39cc159890 0x7f39cc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:56.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.489+0000 7f39d17b6700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39cc15a160 con 0x7f39cc159c60 2023-10-09T06:35:56.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.489+0000 7f39d17b6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39cc0d13a0 con 0x7f39cc069560 2023-10-09T06:35:56.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.489+0000 7f39d17b6700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39cc15e140 con 0x7f39cc159890 2023-10-09T06:35:56.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.489+0000 7f39caffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f39cc159890 0x7f39cc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:56.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.489+0000 7f39c3fff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f39cc159c60 0x7f39cc15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57700/0 (socket says 172.21.15.163:57700) 2023-10-09T06:35:56.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.489+0000 7f39c3fff700 1 -- 172.21.15.163:0/975327000 learned_addr learned my addr 172.21.15.163:0/975327000 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:35:56.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.490+0000 7f39caffd700 1 -- 172.21.15.163:0/975327000 >> v1:172.21.15.181:6789/0 conn(0x7f39cc069560 legacy=0x7f39cc0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:56.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.490+0000 7f39caffd700 1 -- 172.21.15.163:0/975327000 >> v1:172.21.15.163:6789/0 conn(0x7f39cc159c60 legacy=0x7f39cc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:56.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.490+0000 7f39caffd700 1 -- 172.21.15.163:0/975327000 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39cc15a3d0 con 0x7f39cc159890 2023-10-09T06:35:56.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.490+0000 7f39caffd700 1 --2- 172.21.15.163:0/975327000 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39cc159890 0x7f39cc15abd0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0098e0 tx=0x7f39bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d5e71a1318d780 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:56.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.490+0000 7f39ca7fc700 1 -- 172.21.15.163:0/975327000 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc009e80 con 0x7f39cc159890 2023-10-09T06:35:56.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.490+0000 7f39ca7fc700 1 -- 172.21.15.163:0/975327000 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39bc00b820 con 0x7f39cc159890 2023-10-09T06:35:56.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.490+0000 7f39ca7fc700 1 -- 172.21.15.163:0/975327000 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc00bba0 con 0x7f39cc159890 2023-10-09T06:35:56.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.490+0000 7f39d17b6700 1 -- 172.21.15.163:0/975327000 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39cc159890 msgr2=0x7f39cc15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:56.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.490+0000 7f39d17b6700 1 --2- 172.21.15.163:0/975327000 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39cc159890 0x7f39cc15abd0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0098e0 tx=0x7f39bc0091a0 comp rx=0 tx=0).stop 2023-10-09T06:35:56.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.490+0000 7f39d17b6700 1 -- 172.21.15.163:0/975327000 shutdown_connections 2023-10-09T06:35:56.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.490+0000 7f39d17b6700 1 --2- 172.21.15.163:0/975327000 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39cc159890 0x7f39cc15abd0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:56.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.490+0000 7f39d17b6700 1 -- 172.21.15.163:0/975327000 >> 172.21.15.163:0/975327000 conn(0x7f39cc068e20 msgr2=0x7f39cc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:56.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.490+0000 7f39d17b6700 1 -- 172.21.15.163:0/975327000 shutdown_connections 2023-10-09T06:35:56.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.491+0000 7f39d17b6700 1 -- 172.21.15.163:0/975327000 wait complete. 2023-10-09T06:35:56.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.492+0000 7f39d17b6700 1 Processor -- start 2023-10-09T06:35:56.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.492+0000 7f39d17b6700 1 -- start start 2023-10-09T06:35:56.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.492+0000 7f39d17b6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39cc069560 0x7f39cc1480f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:56.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.492+0000 7f39d17b6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39cc159890 0x7f39cc15cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:56.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.492+0000 7f39d17b6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39cc159c60 0x7f39cc15d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:56.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.492+0000 7f39d17b6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc155170 con 0x7f39cc159c60 2023-10-09T06:35:56.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.492+0000 7f39d17b6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc0b36b0 con 0x7f39cc069560 2023-10-09T06:35:56.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.492+0000 7f39d17b6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc0c6560 con 0x7f39cc159890 2023-10-09T06:35:56.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.492+0000 7f39cb7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39cc159c60 0x7f39cc15d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:56.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.492+0000 7f39cb7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39cc159c60 0x7f39cc15d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56186/0 (socket says 172.21.15.163:56186) 2023-10-09T06:35:56.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.492+0000 7f39cb7fe700 1 -- 172.21.15.163:0/3273757113 learned_addr learned my addr 172.21.15.163:0/3273757113 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:56.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.493+0000 7f39c3fff700 1 --2- 172.21.15.163:0/3273757113 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39cc159890 0x7f39cc15cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:56.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.493+0000 7f39caffd700 1 --2- 172.21.15.163:0/3273757113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39cc069560 0x7f39cc1480f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:56.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.493+0000 7f39cb7fe700 1 -- 172.21.15.163:0/3273757113 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39cc159890 msgr2=0x7f39cc15cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:56.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.493+0000 7f39cb7fe700 1 --2- 172.21.15.163:0/3273757113 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39cc159890 0x7f39cc15cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:56.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.493+0000 7f39cb7fe700 1 -- 172.21.15.163:0/3273757113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39cc069560 msgr2=0x7f39cc1480f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:56.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.493+0000 7f39cb7fe700 1 --2- 172.21.15.163:0/3273757113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39cc069560 0x7f39cc1480f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:56.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.493+0000 7f39cb7fe700 1 -- 172.21.15.163:0/3273757113 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39bc009420 con 0x7f39cc159c60 2023-10-09T06:35:56.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.493+0000 7f39caffd700 1 --2- 172.21.15.163:0/3273757113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39cc069560 0x7f39cc1480f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:35:56.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.493+0000 7f39cb7fe700 1 --2- 172.21.15.163:0/3273757113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39cc159c60 0x7f39cc15d4a0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f39c400bc30 tx=0x7f39c4009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:56.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.493+0000 7f39c2ffd700 1 -- 172.21.15.163:0/3273757113 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c400c900 con 0x7f39cc159c60 2023-10-09T06:35:56.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.493+0000 7f39c2ffd700 1 -- 172.21.15.163:0/3273757113 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c400caa0 con 0x7f39cc159c60 2023-10-09T06:35:56.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.493+0000 7f39d17b6700 1 -- 172.21.15.163:0/3273757113 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39cc0c6840 con 0x7f39cc159c60 2023-10-09T06:35:56.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.493+0000 7f39c2ffd700 1 -- 172.21.15.163:0/3273757113 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c400ce60 con 0x7f39cc159c60 2023-10-09T06:35:56.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.493+0000 7f39d17b6700 1 -- 172.21.15.163:0/3273757113 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39cc154b20 con 0x7f39cc159c60 2023-10-09T06:35:56.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.494+0000 7f39d17b6700 1 -- 172.21.15.163:0/3273757113 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39cc0b2210 con 0x7f39cc159c60 2023-10-09T06:35:56.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.494+0000 7f39c2ffd700 1 -- 172.21.15.163:0/3273757113 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f39c4014620 con 0x7f39cc159c60 2023-10-09T06:35:56.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.495+0000 7f39c2ffd700 1 --2- 172.21.15.163:0/3273757113 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f39ac067170 0x7f39ac069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:56.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.495+0000 7f39c2ffd700 1 -- 172.21.15.163:0/3273757113 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f39c4084850 con 0x7f39cc159c60 2023-10-09T06:35:56.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.495+0000 7f39caffd700 1 --2- 172.21.15.163:0/3273757113 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f39ac067170 0x7f39ac069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:56.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.495+0000 7f39caffd700 1 --2- 172.21.15.163:0/3273757113 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f39ac067170 0x7f39ac069620 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0030c0 tx=0x7f39bc016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:56.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.499+0000 7f39c2ffd700 1 -- 172.21.15.163:0/3273757113 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f39c404c5a0 con 0x7f39cc159c60 2023-10-09T06:35:56.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.681+0000 7f39d17b6700 1 -- 172.21.15.163:0/3273757113 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f39cc04c3c0 con 0x7f39ac067170 2023-10-09T06:35:56.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.689+0000 7f39c2ffd700 1 -- 172.21.15.163:0/3273757113 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289484 (secure 0 0 0) 0x7f39cc04c3c0 con 0x7f39ac067170 2023-10-09T06:35:56.693 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:35:56.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.692+0000 7f39d17b6700 1 -- 172.21.15.163:0/3273757113 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f39ac067170 msgr2=0x7f39ac069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:56.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.692+0000 7f39d17b6700 1 --2- 172.21.15.163:0/3273757113 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f39ac067170 0x7f39ac069620 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0030c0 tx=0x7f39bc016000 comp rx=0 tx=0).stop 2023-10-09T06:35:56.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.692+0000 7f39d17b6700 1 -- 172.21.15.163:0/3273757113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39cc159c60 msgr2=0x7f39cc15d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:56.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.692+0000 7f39d17b6700 1 --2- 172.21.15.163:0/3273757113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39cc159c60 0x7f39cc15d4a0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f39c400bc30 tx=0x7f39c4009820 comp rx=0 tx=0).stop 2023-10-09T06:35:56.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.692+0000 7f39d17b6700 1 -- 172.21.15.163:0/3273757113 shutdown_connections 2023-10-09T06:35:56.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.692+0000 7f39d17b6700 1 --2- 172.21.15.163:0/3273757113 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f39ac067170 0x7f39ac069620 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:56.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.692+0000 7f39d17b6700 1 --2- 172.21.15.163:0/3273757113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39cc159c60 0x7f39cc15d4a0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:56.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.692+0000 7f39d17b6700 1 --2- 172.21.15.163:0/3273757113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39cc069560 0x7f39cc1480f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:56.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.692+0000 7f39d17b6700 1 --2- 172.21.15.163:0/3273757113 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39cc159890 0x7f39cc15cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:56.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.692+0000 7f39d17b6700 1 -- 172.21.15.163:0/3273757113 >> 172.21.15.163:0/3273757113 conn(0x7f39cc068e20 msgr2=0x7f39cc15bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:56.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.692+0000 7f39d17b6700 1 -- 172.21.15.163:0/3273757113 shutdown_connections 2023-10-09T06:35:56.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.693+0000 7f39d17b6700 1 -- 172.21.15.163:0/3273757113 wait complete. 2023-10-09T06:35:56.697 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:35:56.705 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2023-10-09T06:35:56.376308+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2395112,"kb_used_data":8616,"kb_used_omap":0,"kb_used_meta":2386432,"kb_avail":752579608,"statfs":{"total":773094113280,"available":770641518592,"internally_reserved":0,"allocated":8822784,"data_stored":4447732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2443706368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.759281"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.657970+0000","last_change":"2023-10-09T06:35:52.650315+0000","last_active":"2023-10-09T06:35:52.657970+0000","last_peered":"2023-10-09T06:35:52.657970+0000","last_clean":"2023-10-09T06:35:52.657970+0000","last_became_active":"2023-10-09T06:35:52.650047+0000","last_became_peered":"2023-10-09T06:35:52.650047+0000","last_unstale":"2023-10-09T06:35:52.657970+0000","last_undegraded":"2023-10-09T06:35:52.657970+0000","last_fullsized":"2023-10-09T06:35:52.657970+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:10:47.085979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.659857+0000","last_change":"2023-10-09T06:35:52.653970+0000","last_active":"2023-10-09T06:35:52.659857+0000","last_peered":"2023-10-09T06:35:52.659857+0000","last_clean":"2023-10-09T06:35:52.659857+0000","last_became_active":"2023-10-09T06:35:52.653711+0000","last_became_peered":"2023-10-09T06:35:52.653711+0000","last_unstale":"2023-10-09T06:35:52.659857+0000","last_undegraded":"2023-10-09T06:35:52.659857+0000","last_fullsized":"2023-10-09T06:35:52.659857+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:07:54.245113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.650819+0000","last_change":"2023-10-09T06:35:52.648946+0000","last_active":"2023-10-09T06:35:52.650819+0000","last_peered":"2023-10-09T06:35:52.650819+0000","last_clean":"2023-10-09T06:35:52.650819+0000","last_became_active":"2023-10-09T06:35:52.648066+0000","last_became_peered":"2023-10-09T06:35:52.648066+0000","last_unstale":"2023-10-09T06:35:52.650819+0000","last_undegraded":"2023-10-09T06:35:52.650819+0000","last_fullsized":"2023-10-09T06:35:52.650819+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:35:12.091172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.660835+0000","last_change":"2023-10-09T06:35:52.657178+0000","last_active":"2023-10-09T06:35:52.660835+0000","last_peered":"2023-10-09T06:35:52.660835+0000","last_clean":"2023-10-09T06:35:52.660835+0000","last_became_active":"2023-10-09T06:35:52.656906+0000","last_became_peered":"2023-10-09T06:35:52.656906+0000","last_unstale":"2023-10-09T06:35:52.660835+0000","last_undegraded":"2023-10-09T06:35:52.660835+0000","last_fullsized":"2023-10-09T06:35:52.660835+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:13:27.952312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.657517+0000","last_change":"2023-10-09T06:35:52.651381+0000","last_active":"2023-10-09T06:35:52.657517+0000","last_peered":"2023-10-09T06:35:52.657517+0000","last_clean":"2023-10-09T06:35:52.657517+0000","last_became_active":"2023-10-09T06:35:52.651091+0000","last_became_peered":"2023-10-09T06:35:52.651091+0000","last_unstale":"2023-10-09T06:35:52.657517+0000","last_undegraded":"2023-10-09T06:35:52.657517+0000","last_fullsized":"2023-10-09T06:35:52.657517+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:05:30.191217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.659103+0000","last_change":"2023-10-09T06:35:52.653182+0000","last_active":"2023-10-09T06:35:52.659103+0000","last_peered":"2023-10-09T06:35:52.659103+0000","last_clean":"2023-10-09T06:35:52.659103+0000","last_became_active":"2023-10-09T06:35:52.652927+0000","last_became_peered":"2023-10-09T06:35:52.652927+0000","last_unstale":"2023-10-09T06:35:52.659103+0000","last_undegraded":"2023-10-09T06:35:52.659103+0000","last_fullsized":"2023-10-09T06:35:52.659103+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:21:30.385965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.660222+0000","last_change":"2023-10-09T06:35:52.653565+0000","last_active":"2023-10-09T06:35:52.660222+0000","last_peered":"2023-10-09T06:35:52.660222+0000","last_clean":"2023-10-09T06:35:52.660222+0000","last_became_active":"2023-10-09T06:35:52.653300+0000","last_became_peered":"2023-10-09T06:35:52.653300+0000","last_unstale":"2023-10-09T06:35:52.660222+0000","last_undegraded":"2023-10-09T06:35:52.660222+0000","last_fullsized":"2023-10-09T06:35:52.660222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:42:44.459965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.660042+0000","last_change":"2023-10-09T06:35:52.659248+0000","last_active":"2023-10-09T06:35:52.660042+0000","last_peered":"2023-10-09T06:35:52.660042+0000","last_clean":"2023-10-09T06:35:52.660042+0000","last_became_active":"2023-10-09T06:35:52.658957+0000","last_became_peered":"2023-10-09T06:35:52.658957+0000","last_unstale":"2023-10-09T06:35:52.660042+0000","last_undegraded":"2023-10-09T06:35:52.660042+0000","last_fullsized":"2023-10-09T06:35:52.660042+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:10:47.085979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.653723+0000","last_change":"2023-10-09T06:35:52.649500+0000","last_active":"2023-10-09T06:35:52.653723+0000","last_peered":"2023-10-09T06:35:52.653723+0000","last_clean":"2023-10-09T06:35:52.653723+0000","last_became_active":"2023-10-09T06:35:52.649219+0000","last_became_peered":"2023-10-09T06:35:52.649219+0000","last_unstale":"2023-10-09T06:35:52.653723+0000","last_undegraded":"2023-10-09T06:35:52.653723+0000","last_fullsized":"2023-10-09T06:35:52.653723+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:59:03.103633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.661146+0000","last_change":"2023-10-09T06:35:52.653146+0000","last_active":"2023-10-09T06:35:52.661146+0000","last_peered":"2023-10-09T06:35:52.661146+0000","last_clean":"2023-10-09T06:35:52.661146+0000","last_became_active":"2023-10-09T06:35:52.652893+0000","last_became_peered":"2023-10-09T06:35:52.652893+0000","last_unstale":"2023-10-09T06:35:52.661146+0000","last_undegraded":"2023-10-09T06:35:52.661146+0000","last_fullsized":"2023-10-09T06:35:52.661146+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:35:42.110847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.657635+0000","last_change":"2023-10-09T06:35:52.653309+0000","last_active":"2023-10-09T06:35:52.657635+0000","last_peered":"2023-10-09T06:35:52.657635+0000","last_clean":"2023-10-09T06:35:52.657635+0000","last_became_active":"2023-10-09T06:35:52.653033+0000","last_became_peered":"2023-10-09T06:35:52.653033+0000","last_unstale":"2023-10-09T06:35:52.657635+0000","last_undegraded":"2023-10-09T06:35:52.657635+0000","last_fullsized":"2023-10-09T06:35:52.657635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:21:30.385965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.651942+0000","last_change":"2023-10-09T06:35:52.647755+0000","last_active":"2023-10-09T06:35:52.651942+0000","last_peered":"2023-10-09T06:35:52.651942+0000","last_clean":"2023-10-09T06:35:52.651942+0000","last_became_active":"2023-10-09T06:35:52.647239+0000","last_became_peered":"2023-10-09T06:35:52.647239+0000","last_unstale":"2023-10-09T06:35:52.651942+0000","last_undegraded":"2023-10-09T06:35:52.651942+0000","last_fullsized":"2023-10-09T06:35:52.651942+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:21:30.385965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.655858+0000","last_change":"2023-10-09T06:35:52.652392+0000","last_active":"2023-10-09T06:35:52.655858+0000","last_peered":"2023-10-09T06:35:52.655858+0000","last_clean":"2023-10-09T06:35:52.655858+0000","last_became_active":"2023-10-09T06:35:52.652092+0000","last_became_peered":"2023-10-09T06:35:52.652092+0000","last_unstale":"2023-10-09T06:35:52.655858+0000","last_undegraded":"2023-10-09T06:35:52.655858+0000","last_fullsized":"2023-10-09T06:35:52.655858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:33:54.388524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.664234+0000","last_change":"2023-10-09T06:35:52.659657+0000","last_active":"2023-10-09T06:35:52.664234+0000","last_peered":"2023-10-09T06:35:52.664234+0000","last_clean":"2023-10-09T06:35:52.664234+0000","last_became_active":"2023-10-09T06:35:52.659001+0000","last_became_peered":"2023-10-09T06:35:52.659001+0000","last_unstale":"2023-10-09T06:35:52.664234+0000","last_undegraded":"2023-10-09T06:35:52.664234+0000","last_fullsized":"2023-10-09T06:35:52.664234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:57:30.555787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.653548+0000","last_change":"2023-10-09T06:35:52.650985+0000","last_active":"2023-10-09T06:35:52.653548+0000","last_peered":"2023-10-09T06:35:52.653548+0000","last_clean":"2023-10-09T06:35:52.653548+0000","last_became_active":"2023-10-09T06:35:52.650638+0000","last_became_peered":"2023-10-09T06:35:52.650638+0000","last_unstale":"2023-10-09T06:35:52.653548+0000","last_undegraded":"2023-10-09T06:35:52.653548+0000","last_fullsized":"2023-10-09T06:35:52.653548+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:13:56.638985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.657154+0000","last_change":"2023-10-09T06:35:52.651163+0000","last_active":"2023-10-09T06:35:52.657154+0000","last_peered":"2023-10-09T06:35:52.657154+0000","last_clean":"2023-10-09T06:35:52.657154+0000","last_became_active":"2023-10-09T06:35:52.650078+0000","last_became_peered":"2023-10-09T06:35:52.650078+0000","last_unstale":"2023-10-09T06:35:52.657154+0000","last_undegraded":"2023-10-09T06:35:52.657154+0000","last_fullsized":"2023-10-09T06:35:52.657154+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:10:47.085979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.661365+0000","last_change":"2023-10-09T06:35:52.658022+0000","last_active":"2023-10-09T06:35:52.661365+0000","last_peered":"2023-10-09T06:35:52.661365+0000","last_clean":"2023-10-09T06:35:52.661365+0000","last_became_active":"2023-10-09T06:35:52.653730+0000","last_became_peered":"2023-10-09T06:35:52.653730+0000","last_unstale":"2023-10-09T06:35:52.661365+0000","last_undegraded":"2023-10-09T06:35:52.661365+0000","last_fullsized":"2023-10-09T06:35:52.661365+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:59:03.103633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.660755+0000","last_change":"2023-10-09T06:35:52.654777+0000","last_active":"2023-10-09T06:35:52.660755+0000","last_peered":"2023-10-09T06:35:52.660755+0000","last_clean":"2023-10-09T06:35:52.660755+0000","last_became_active":"2023-10-09T06:35:52.654498+0000","last_became_peered":"2023-10-09T06:35:52.654498+0000","last_unstale":"2023-10-09T06:35:52.660755+0000","last_undegraded":"2023-10-09T06:35:52.660755+0000","last_fullsized":"2023-10-09T06:35:52.660755+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:26:24.468235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.665768+0000","last_change":"2023-10-09T06:35:52.649349+0000","last_active":"2023-10-09T06:35:52.665768+0000","last_peered":"2023-10-09T06:35:52.665768+0000","last_clean":"2023-10-09T06:35:52.665768+0000","last_became_active":"2023-10-09T06:35:52.649112+0000","last_became_peered":"2023-10-09T06:35:52.649112+0000","last_unstale":"2023-10-09T06:35:52.665768+0000","last_undegraded":"2023-10-09T06:35:52.665768+0000","last_fullsized":"2023-10-09T06:35:52.665768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:04:11.040070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.669110+0000","last_change":"2023-10-09T06:35:52.667954+0000","last_active":"2023-10-09T06:35:52.669110+0000","last_peered":"2023-10-09T06:35:52.669110+0000","last_clean":"2023-10-09T06:35:52.669110+0000","last_became_active":"2023-10-09T06:35:52.667656+0000","last_became_peered":"2023-10-09T06:35:52.667656+0000","last_unstale":"2023-10-09T06:35:52.669110+0000","last_undegraded":"2023-10-09T06:35:52.669110+0000","last_fullsized":"2023-10-09T06:35:52.669110+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:24:19.715831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.669152+0000","last_change":"2023-10-09T06:35:52.668017+0000","last_active":"2023-10-09T06:35:52.669152+0000","last_peered":"2023-10-09T06:35:52.669152+0000","last_clean":"2023-10-09T06:35:52.669152+0000","last_became_active":"2023-10-09T06:35:52.667618+0000","last_became_peered":"2023-10-09T06:35:52.667618+0000","last_unstale":"2023-10-09T06:35:52.669152+0000","last_undegraded":"2023-10-09T06:35:52.669152+0000","last_fullsized":"2023-10-09T06:35:52.669152+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:59:37.178211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.668133+0000","last_change":"2023-10-09T06:35:52.666822+0000","last_active":"2023-10-09T06:35:52.668133+0000","last_peered":"2023-10-09T06:35:52.668133+0000","last_clean":"2023-10-09T06:35:52.668133+0000","last_became_active":"2023-10-09T06:35:52.666508+0000","last_became_peered":"2023-10-09T06:35:52.666508+0000","last_unstale":"2023-10-09T06:35:52.668133+0000","last_undegraded":"2023-10-09T06:35:52.668133+0000","last_fullsized":"2023-10-09T06:35:52.668133+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:59:03.103633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.647107+0000","last_change":"2023-10-09T06:35:52.641370+0000","last_active":"2023-10-09T06:35:52.647107+0000","last_peered":"2023-10-09T06:35:52.647107+0000","last_clean":"2023-10-09T06:35:52.647107+0000","last_became_active":"2023-10-09T06:35:52.641054+0000","last_became_peered":"2023-10-09T06:35:52.641054+0000","last_unstale":"2023-10-09T06:35:52.647107+0000","last_undegraded":"2023-10-09T06:35:52.647107+0000","last_fullsized":"2023-10-09T06:35:52.647107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:55:43.724447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.671278+0000","last_change":"2023-10-09T06:35:52.669011+0000","last_active":"2023-10-09T06:35:52.671278+0000","last_peered":"2023-10-09T06:35:52.671278+0000","last_clean":"2023-10-09T06:35:52.671278+0000","last_became_active":"2023-10-09T06:35:52.668804+0000","last_became_peered":"2023-10-09T06:35:52.668804+0000","last_unstale":"2023-10-09T06:35:52.671278+0000","last_undegraded":"2023-10-09T06:35:52.671278+0000","last_fullsized":"2023-10-09T06:35:52.671278+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:10:50.929917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.656310+0000","last_change":"2023-10-09T06:35:52.651591+0000","last_active":"2023-10-09T06:35:52.656310+0000","last_peered":"2023-10-09T06:35:52.656310+0000","last_clean":"2023-10-09T06:35:52.656310+0000","last_became_active":"2023-10-09T06:35:52.651222+0000","last_became_peered":"2023-10-09T06:35:52.651222+0000","last_unstale":"2023-10-09T06:35:52.656310+0000","last_undegraded":"2023-10-09T06:35:52.656310+0000","last_fullsized":"2023-10-09T06:35:52.656310+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:04:11.040070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.661400+0000","last_change":"2023-10-09T06:35:52.658434+0000","last_active":"2023-10-09T06:35:52.661400+0000","last_peered":"2023-10-09T06:35:52.661400+0000","last_clean":"2023-10-09T06:35:52.661400+0000","last_became_active":"2023-10-09T06:35:52.658018+0000","last_became_peered":"2023-10-09T06:35:52.658018+0000","last_unstale":"2023-10-09T06:35:52.661400+0000","last_undegraded":"2023-10-09T06:35:52.661400+0000","last_fullsized":"2023-10-09T06:35:52.661400+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:16:36.606784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.661306+0000","last_change":"2023-10-09T06:35:52.659376+0000","last_active":"2023-10-09T06:35:52.661306+0000","last_peered":"2023-10-09T06:35:52.661306+0000","last_clean":"2023-10-09T06:35:52.661306+0000","last_became_active":"2023-10-09T06:35:52.659085+0000","last_became_peered":"2023-10-09T06:35:52.659085+0000","last_unstale":"2023-10-09T06:35:52.661306+0000","last_undegraded":"2023-10-09T06:35:52.661306+0000","last_fullsized":"2023-10-09T06:35:52.661306+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:21:30.385965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.659204+0000","last_change":"2023-10-09T06:35:52.652059+0000","last_active":"2023-10-09T06:35:52.659204+0000","last_peered":"2023-10-09T06:35:52.659204+0000","last_clean":"2023-10-09T06:35:52.659204+0000","last_became_active":"2023-10-09T06:35:52.651773+0000","last_became_peered":"2023-10-09T06:35:52.651773+0000","last_unstale":"2023-10-09T06:35:52.659204+0000","last_undegraded":"2023-10-09T06:35:52.659204+0000","last_fullsized":"2023-10-09T06:35:52.659204+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:13:52.904899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.657898+0000","last_change":"2023-10-09T06:35:52.650110+0000","last_active":"2023-10-09T06:35:52.657898+0000","last_peered":"2023-10-09T06:35:52.657898+0000","last_clean":"2023-10-09T06:35:52.657898+0000","last_became_active":"2023-10-09T06:35:52.648802+0000","last_became_peered":"2023-10-09T06:35:52.648802+0000","last_unstale":"2023-10-09T06:35:52.657898+0000","last_undegraded":"2023-10-09T06:35:52.657898+0000","last_fullsized":"2023-10-09T06:35:52.657898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:10:47.085979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.662182+0000","last_change":"2023-10-09T06:35:52.658596+0000","last_active":"2023-10-09T06:35:52.662182+0000","last_peered":"2023-10-09T06:35:52.662182+0000","last_clean":"2023-10-09T06:35:52.662182+0000","last_became_active":"2023-10-09T06:35:52.658144+0000","last_became_peered":"2023-10-09T06:35:52.658144+0000","last_unstale":"2023-10-09T06:35:52.662182+0000","last_undegraded":"2023-10-09T06:35:52.662182+0000","last_fullsized":"2023-10-09T06:35:52.662182+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:43:22.600613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.656185+0000","last_change":"2023-10-09T06:35:52.650675+0000","last_active":"2023-10-09T06:35:52.656185+0000","last_peered":"2023-10-09T06:35:52.656185+0000","last_clean":"2023-10-09T06:35:52.656185+0000","last_became_active":"2023-10-09T06:35:52.649111+0000","last_became_peered":"2023-10-09T06:35:52.649111+0000","last_unstale":"2023-10-09T06:35:52.656185+0000","last_undegraded":"2023-10-09T06:35:52.656185+0000","last_fullsized":"2023-10-09T06:35:52.656185+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:55:43.724447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.661848+0000","last_change":"2023-10-09T06:35:52.655334+0000","last_active":"2023-10-09T06:35:52.661848+0000","last_peered":"2023-10-09T06:35:52.661848+0000","last_clean":"2023-10-09T06:35:52.661848+0000","last_became_active":"2023-10-09T06:35:52.654976+0000","last_became_peered":"2023-10-09T06:35:52.654976+0000","last_unstale":"2023-10-09T06:35:52.661848+0000","last_undegraded":"2023-10-09T06:35:52.661848+0000","last_fullsized":"2023-10-09T06:35:52.661848+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:38:23.450418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.658181+0000","last_change":"2023-10-09T06:35:52.650391+0000","last_active":"2023-10-09T06:35:52.658181+0000","last_peered":"2023-10-09T06:35:52.658181+0000","last_clean":"2023-10-09T06:35:52.658181+0000","last_became_active":"2023-10-09T06:35:52.650095+0000","last_became_peered":"2023-10-09T06:35:52.650095+0000","last_unstale":"2023-10-09T06:35:52.658181+0000","last_undegraded":"2023-10-09T06:35:52.658181+0000","last_fullsized":"2023-10-09T06:35:52.658181+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:35:42.110847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"76'4","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.663903+0000","last_change":"2023-10-09T06:35:51.628579+0000","last_active":"2023-10-09T06:35:52.663903+0000","last_peered":"2023-10-09T06:35:52.663903+0000","last_clean":"2023-10-09T06:35:52.663903+0000","last_became_active":"2023-10-09T06:35:51.628237+0000","last_became_peered":"2023-10-09T06:35:51.628237+0000","last_unstale":"2023-10-09T06:35:52.663903+0000","last_undegraded":"2023-10-09T06:35:52.663903+0000","last_fullsized":"2023-10-09T06:35:52.663903+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:51:31.112094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.647349+0000","last_change":"2023-10-09T06:35:52.642018+0000","last_active":"2023-10-09T06:35:52.647349+0000","last_peered":"2023-10-09T06:35:52.647349+0000","last_clean":"2023-10-09T06:35:52.647349+0000","last_became_active":"2023-10-09T06:35:52.641757+0000","last_became_peered":"2023-10-09T06:35:52.641757+0000","last_unstale":"2023-10-09T06:35:52.647349+0000","last_undegraded":"2023-10-09T06:35:52.647349+0000","last_fullsized":"2023-10-09T06:35:52.647349+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:05:30.191217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.664819+0000","last_change":"2023-10-09T06:35:51.634019+0000","last_active":"2023-10-09T06:35:52.664819+0000","last_peered":"2023-10-09T06:35:52.664819+0000","last_clean":"2023-10-09T06:35:52.664819+0000","last_became_active":"2023-10-09T06:35:51.633736+0000","last_became_peered":"2023-10-09T06:35:51.633736+0000","last_unstale":"2023-10-09T06:35:52.664819+0000","last_undegraded":"2023-10-09T06:35:52.664819+0000","last_fullsized":"2023-10-09T06:35:52.664819+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:39:23.129225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.661468+0000","last_change":"2023-10-09T06:35:52.655477+0000","last_active":"2023-10-09T06:35:52.661468+0000","last_peered":"2023-10-09T06:35:52.661468+0000","last_clean":"2023-10-09T06:35:52.661468+0000","last_became_active":"2023-10-09T06:35:52.654965+0000","last_became_peered":"2023-10-09T06:35:52.654965+0000","last_unstale":"2023-10-09T06:35:52.661468+0000","last_undegraded":"2023-10-09T06:35:52.661468+0000","last_fullsized":"2023-10-09T06:35:52.661468+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:19:47.218422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.667926+0000","last_change":"2023-10-09T06:35:51.638920+0000","last_active":"2023-10-09T06:35:52.667926+0000","last_peered":"2023-10-09T06:35:52.667926+0000","last_clean":"2023-10-09T06:35:52.667926+0000","last_became_active":"2023-10-09T06:35:51.638548+0000","last_became_peered":"2023-10-09T06:35:51.638548+0000","last_unstale":"2023-10-09T06:35:52.667926+0000","last_undegraded":"2023-10-09T06:35:52.667926+0000","last_fullsized":"2023-10-09T06:35:52.667926+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:55:17.450357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.659055+0000","last_change":"2023-10-09T06:35:52.654497+0000","last_active":"2023-10-09T06:35:52.659055+0000","last_peered":"2023-10-09T06:35:52.659055+0000","last_clean":"2023-10-09T06:35:52.659055+0000","last_became_active":"2023-10-09T06:35:52.654166+0000","last_became_peered":"2023-10-09T06:35:52.654166+0000","last_unstale":"2023-10-09T06:35:52.659055+0000","last_undegraded":"2023-10-09T06:35:52.659055+0000","last_fullsized":"2023-10-09T06:35:52.659055+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:39:29.529534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.667988+0000","last_change":"2023-10-09T06:35:51.639411+0000","last_active":"2023-10-09T06:35:52.667988+0000","last_peered":"2023-10-09T06:35:52.667988+0000","last_clean":"2023-10-09T06:35:52.667988+0000","last_became_active":"2023-10-09T06:35:51.638841+0000","last_became_peered":"2023-10-09T06:35:51.638841+0000","last_unstale":"2023-10-09T06:35:52.667988+0000","last_undegraded":"2023-10-09T06:35:52.667988+0000","last_fullsized":"2023-10-09T06:35:52.667988+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:19:46.211210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.662164+0000","last_change":"2023-10-09T06:35:52.658425+0000","last_active":"2023-10-09T06:35:52.662164+0000","last_peered":"2023-10-09T06:35:52.662164+0000","last_clean":"2023-10-09T06:35:52.662164+0000","last_became_active":"2023-10-09T06:35:52.656477+0000","last_became_peered":"2023-10-09T06:35:52.656477+0000","last_unstale":"2023-10-09T06:35:52.662164+0000","last_undegraded":"2023-10-09T06:35:52.662164+0000","last_fullsized":"2023-10-09T06:35:52.662164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:55:43.724447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.663305+0000","last_change":"2023-10-09T06:35:51.637653+0000","last_active":"2023-10-09T06:35:52.663305+0000","last_peered":"2023-10-09T06:35:52.663305+0000","last_clean":"2023-10-09T06:35:52.663305+0000","last_became_active":"2023-10-09T06:35:51.637228+0000","last_became_peered":"2023-10-09T06:35:51.637228+0000","last_unstale":"2023-10-09T06:35:52.663305+0000","last_undegraded":"2023-10-09T06:35:52.663305+0000","last_fullsized":"2023-10-09T06:35:52.663305+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:41:00.450594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.661108+0000","last_change":"2023-10-09T06:35:52.657387+0000","last_active":"2023-10-09T06:35:52.661108+0000","last_peered":"2023-10-09T06:35:52.661108+0000","last_clean":"2023-10-09T06:35:52.661108+0000","last_became_active":"2023-10-09T06:35:52.653930+0000","last_became_peered":"2023-10-09T06:35:52.653930+0000","last_unstale":"2023-10-09T06:35:52.661108+0000","last_undegraded":"2023-10-09T06:35:52.661108+0000","last_fullsized":"2023-10-09T06:35:52.661108+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:07:54.245113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.665047+0000","last_change":"2023-10-09T06:35:51.634538+0000","last_active":"2023-10-09T06:35:52.665047+0000","last_peered":"2023-10-09T06:35:52.665047+0000","last_clean":"2023-10-09T06:35:52.665047+0000","last_became_active":"2023-10-09T06:35:51.634240+0000","last_became_peered":"2023-10-09T06:35:51.634240+0000","last_unstale":"2023-10-09T06:35:52.665047+0000","last_undegraded":"2023-10-09T06:35:52.665047+0000","last_fullsized":"2023-10-09T06:35:52.665047+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:19:09.292729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.650847+0000","last_change":"2023-10-09T06:35:52.648848+0000","last_active":"2023-10-09T06:35:52.650847+0000","last_peered":"2023-10-09T06:35:52.650847+0000","last_clean":"2023-10-09T06:35:52.650847+0000","last_became_active":"2023-10-09T06:35:52.648551+0000","last_became_peered":"2023-10-09T06:35:52.648551+0000","last_unstale":"2023-10-09T06:35:52.650847+0000","last_undegraded":"2023-10-09T06:35:52.650847+0000","last_fullsized":"2023-10-09T06:35:52.650847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:28:42.829846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.639169+0000","last_change":"2023-10-09T06:35:51.635731+0000","last_active":"2023-10-09T06:35:52.639169+0000","last_peered":"2023-10-09T06:35:52.639169+0000","last_clean":"2023-10-09T06:35:52.639169+0000","last_became_active":"2023-10-09T06:35:51.635504+0000","last_became_peered":"2023-10-09T06:35:51.635504+0000","last_unstale":"2023-10-09T06:35:52.639169+0000","last_undegraded":"2023-10-09T06:35:52.639169+0000","last_fullsized":"2023-10-09T06:35:52.639169+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:38:22.443207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.665389+0000","last_change":"2023-10-09T06:35:52.660517+0000","last_active":"2023-10-09T06:35:52.665389+0000","last_peered":"2023-10-09T06:35:52.665389+0000","last_clean":"2023-10-09T06:35:52.665389+0000","last_became_active":"2023-10-09T06:35:52.660219+0000","last_became_peered":"2023-10-09T06:35:52.660219+0000","last_unstale":"2023-10-09T06:35:52.665389+0000","last_undegraded":"2023-10-09T06:35:52.665389+0000","last_fullsized":"2023-10-09T06:35:52.665389+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:31:21.904859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.649514+0000","last_change":"2023-10-09T06:35:51.636565+0000","last_active":"2023-10-09T06:35:52.649514+0000","last_peered":"2023-10-09T06:35:52.649514+0000","last_clean":"2023-10-09T06:35:52.649514+0000","last_became_active":"2023-10-09T06:35:51.636248+0000","last_became_peered":"2023-10-09T06:35:51.636248+0000","last_unstale":"2023-10-09T06:35:52.649514+0000","last_undegraded":"2023-10-09T06:35:52.649514+0000","last_fullsized":"2023-10-09T06:35:52.649514+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:39:23.129225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.661927+0000","last_change":"2023-10-09T06:35:52.655665+0000","last_active":"2023-10-09T06:35:52.661927+0000","last_peered":"2023-10-09T06:35:52.661927+0000","last_clean":"2023-10-09T06:35:52.661927+0000","last_became_active":"2023-10-09T06:35:52.655385+0000","last_became_peered":"2023-10-09T06:35:52.655385+0000","last_unstale":"2023-10-09T06:35:52.661927+0000","last_undegraded":"2023-10-09T06:35:52.661927+0000","last_fullsized":"2023-10-09T06:35:52.661927+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:11:29.107451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.639536+0000","last_change":"2023-10-09T06:35:51.641020+0000","last_active":"2023-10-09T06:35:52.639536+0000","last_peered":"2023-10-09T06:35:52.639536+0000","last_clean":"2023-10-09T06:35:52.639536+0000","last_became_active":"2023-10-09T06:35:51.640740+0000","last_became_peered":"2023-10-09T06:35:51.640740+0000","last_unstale":"2023-10-09T06:35:52.639536+0000","last_undegraded":"2023-10-09T06:35:52.639536+0000","last_fullsized":"2023-10-09T06:35:52.639536+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:32:58.351321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.660333+0000","last_change":"2023-10-09T06:35:52.653016+0000","last_active":"2023-10-09T06:35:52.660333+0000","last_peered":"2023-10-09T06:35:52.660333+0000","last_clean":"2023-10-09T06:35:52.660333+0000","last_became_active":"2023-10-09T06:35:52.652720+0000","last_became_peered":"2023-10-09T06:35:52.652720+0000","last_unstale":"2023-10-09T06:35:52.660333+0000","last_undegraded":"2023-10-09T06:35:52.660333+0000","last_fullsized":"2023-10-09T06:35:52.660333+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:55:43.724447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.639319+0000","last_change":"2023-10-09T06:35:51.634208+0000","last_active":"2023-10-09T06:35:52.639319+0000","last_peered":"2023-10-09T06:35:52.639319+0000","last_clean":"2023-10-09T06:35:52.639319+0000","last_became_active":"2023-10-09T06:35:51.633353+0000","last_became_peered":"2023-10-09T06:35:51.633353+0000","last_unstale":"2023-10-09T06:35:52.639319+0000","last_undegraded":"2023-10-09T06:35:52.639319+0000","last_fullsized":"2023-10-09T06:35:52.639319+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:57:42.823722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.665653+0000","last_change":"2023-10-09T06:35:52.648101+0000","last_active":"2023-10-09T06:35:52.665653+0000","last_peered":"2023-10-09T06:35:52.665653+0000","last_clean":"2023-10-09T06:35:52.665653+0000","last_became_active":"2023-10-09T06:35:52.647826+0000","last_became_peered":"2023-10-09T06:35:52.647826+0000","last_unstale":"2023-10-09T06:35:52.665653+0000","last_undegraded":"2023-10-09T06:35:52.665653+0000","last_fullsized":"2023-10-09T06:35:52.665653+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:43:22.600613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"76'3","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.664087+0000","last_change":"2023-10-09T06:35:51.636707+0000","last_active":"2023-10-09T06:35:52.664087+0000","last_peered":"2023-10-09T06:35:52.664087+0000","last_clean":"2023-10-09T06:35:52.664087+0000","last_became_active":"2023-10-09T06:35:51.636459+0000","last_became_peered":"2023-10-09T06:35:51.636459+0000","last_unstale":"2023-10-09T06:35:52.664087+0000","last_undegraded":"2023-10-09T06:35:52.664087+0000","last_fullsized":"2023-10-09T06:35:52.664087+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:44:17.924698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.667652+0000","last_change":"2023-10-09T06:35:52.666318+0000","last_active":"2023-10-09T06:35:52.667652+0000","last_peered":"2023-10-09T06:35:52.667652+0000","last_clean":"2023-10-09T06:35:52.667652+0000","last_became_active":"2023-10-09T06:35:52.666076+0000","last_became_peered":"2023-10-09T06:35:52.666076+0000","last_unstale":"2023-10-09T06:35:52.667652+0000","last_undegraded":"2023-10-09T06:35:52.667652+0000","last_fullsized":"2023-10-09T06:35:52.667652+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:05:23.290057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"76'6","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.667626+0000","last_change":"2023-10-09T06:35:51.633767+0000","last_active":"2023-10-09T06:35:52.667626+0000","last_peered":"2023-10-09T06:35:52.667626+0000","last_clean":"2023-10-09T06:35:52.667626+0000","last_became_active":"2023-10-09T06:35:51.633518+0000","last_became_peered":"2023-10-09T06:35:51.633518+0000","last_unstale":"2023-10-09T06:35:52.667626+0000","last_undegraded":"2023-10-09T06:35:52.667626+0000","last_fullsized":"2023-10-09T06:35:52.667626+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:02:15.246012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.667468+0000","last_change":"2023-10-09T06:35:52.657558+0000","last_active":"2023-10-09T06:35:52.667468+0000","last_peered":"2023-10-09T06:35:52.667468+0000","last_clean":"2023-10-09T06:35:52.667468+0000","last_became_active":"2023-10-09T06:35:52.657320+0000","last_became_peered":"2023-10-09T06:35:52.657320+0000","last_unstale":"2023-10-09T06:35:52.667468+0000","last_undegraded":"2023-10-09T06:35:52.667468+0000","last_fullsized":"2023-10-09T06:35:52.667468+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:59:03.103633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.668040+0000","last_change":"2023-10-09T06:35:51.639595+0000","last_active":"2023-10-09T06:35:52.668040+0000","last_peered":"2023-10-09T06:35:52.668040+0000","last_clean":"2023-10-09T06:35:52.668040+0000","last_became_active":"2023-10-09T06:35:51.639343+0000","last_became_peered":"2023-10-09T06:35:51.639343+0000","last_unstale":"2023-10-09T06:35:52.668040+0000","last_undegraded":"2023-10-09T06:35:52.668040+0000","last_fullsized":"2023-10-09T06:35:52.668040+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:30:15.468084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.659722+0000","last_change":"2023-10-09T06:35:52.656891+0000","last_active":"2023-10-09T06:35:52.659722+0000","last_peered":"2023-10-09T06:35:52.659722+0000","last_clean":"2023-10-09T06:35:52.659722+0000","last_became_active":"2023-10-09T06:35:52.656573+0000","last_became_peered":"2023-10-09T06:35:52.656573+0000","last_unstale":"2023-10-09T06:35:52.659722+0000","last_undegraded":"2023-10-09T06:35:52.659722+0000","last_fullsized":"2023-10-09T06:35:52.659722+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:18:29.136434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.650850+0000","last_change":"2023-10-09T06:35:51.635878+0000","last_active":"2023-10-09T06:35:52.650850+0000","last_peered":"2023-10-09T06:35:52.650850+0000","last_clean":"2023-10-09T06:35:52.650850+0000","last_became_active":"2023-10-09T06:35:51.635650+0000","last_became_peered":"2023-10-09T06:35:51.635650+0000","last_unstale":"2023-10-09T06:35:52.650850+0000","last_undegraded":"2023-10-09T06:35:52.650850+0000","last_fullsized":"2023-10-09T06:35:52.650850+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:39:28.522323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.663433+0000","last_change":"2023-10-09T06:35:52.658000+0000","last_active":"2023-10-09T06:35:52.663433+0000","last_peered":"2023-10-09T06:35:52.663433+0000","last_clean":"2023-10-09T06:35:52.663433+0000","last_became_active":"2023-10-09T06:35:52.657583+0000","last_became_peered":"2023-10-09T06:35:52.657583+0000","last_unstale":"2023-10-09T06:35:52.663433+0000","last_undegraded":"2023-10-09T06:35:52.663433+0000","last_fullsized":"2023-10-09T06:35:52.663433+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:05:30.191217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.645827+0000","last_change":"2023-10-09T06:35:51.634875+0000","last_active":"2023-10-09T06:35:52.645827+0000","last_peered":"2023-10-09T06:35:52.645827+0000","last_clean":"2023-10-09T06:35:52.645827+0000","last_became_active":"2023-10-09T06:35:51.634619+0000","last_became_peered":"2023-10-09T06:35:51.634619+0000","last_unstale":"2023-10-09T06:35:52.645827+0000","last_undegraded":"2023-10-09T06:35:52.645827+0000","last_fullsized":"2023-10-09T06:35:52.645827+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:12:07.625373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.660013+0000","last_change":"2023-10-09T06:35:52.654229+0000","last_active":"2023-10-09T06:35:52.660013+0000","last_peered":"2023-10-09T06:35:52.660013+0000","last_clean":"2023-10-09T06:35:52.660013+0000","last_became_active":"2023-10-09T06:35:52.653791+0000","last_became_peered":"2023-10-09T06:35:52.653791+0000","last_unstale":"2023-10-09T06:35:52.660013+0000","last_undegraded":"2023-10-09T06:35:52.660013+0000","last_fullsized":"2023-10-09T06:35:52.660013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:05:23.290057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"76'8","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.670492+0000","last_change":"2023-10-09T06:35:51.637899+0000","last_active":"2023-10-09T06:35:52.670492+0000","last_peered":"2023-10-09T06:35:52.670492+0000","last_clean":"2023-10-09T06:35:52.670492+0000","last_became_active":"2023-10-09T06:35:51.637646+0000","last_became_peered":"2023-10-09T06:35:51.637646+0000","last_unstale":"2023-10-09T06:35:52.670492+0000","last_undegraded":"2023-10-09T06:35:52.670492+0000","last_fullsized":"2023-10-09T06:35:52.670492+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:13:51.897687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.655810+0000","last_change":"2023-10-09T06:35:52.652375+0000","last_active":"2023-10-09T06:35:52.655810+0000","last_peered":"2023-10-09T06:35:52.655810+0000","last_clean":"2023-10-09T06:35:52.655810+0000","last_became_active":"2023-10-09T06:35:52.652056+0000","last_became_peered":"2023-10-09T06:35:52.652056+0000","last_unstale":"2023-10-09T06:35:52.655810+0000","last_undegraded":"2023-10-09T06:35:52.655810+0000","last_fullsized":"2023-10-09T06:35:52.655810+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:09:58.474455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.662490+0000","last_change":"2023-10-09T06:35:51.632482+0000","last_active":"2023-10-09T06:35:52.662490+0000","last_peered":"2023-10-09T06:35:52.662490+0000","last_clean":"2023-10-09T06:35:52.662490+0000","last_became_active":"2023-10-09T06:35:51.632098+0000","last_became_peered":"2023-10-09T06:35:51.632098+0000","last_unstale":"2023-10-09T06:35:52.662490+0000","last_undegraded":"2023-10-09T06:35:52.662490+0000","last_fullsized":"2023-10-09T06:35:52.662490+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:39:33.814116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.647441+0000","last_change":"2023-10-09T06:35:52.645484+0000","last_active":"2023-10-09T06:35:52.647441+0000","last_peered":"2023-10-09T06:35:52.647441+0000","last_clean":"2023-10-09T06:35:52.647441+0000","last_became_active":"2023-10-09T06:35:52.645210+0000","last_became_peered":"2023-10-09T06:35:52.645210+0000","last_unstale":"2023-10-09T06:35:52.647441+0000","last_undegraded":"2023-10-09T06:35:52.647441+0000","last_fullsized":"2023-10-09T06:35:52.647441+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:14:06.354004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.667723+0000","last_change":"2023-10-09T06:35:51.637794+0000","last_active":"2023-10-09T06:35:52.667723+0000","last_peered":"2023-10-09T06:35:52.667723+0000","last_clean":"2023-10-09T06:35:52.667723+0000","last_became_active":"2023-10-09T06:35:51.637509+0000","last_became_peered":"2023-10-09T06:35:51.637509+0000","last_unstale":"2023-10-09T06:35:52.667723+0000","last_undegraded":"2023-10-09T06:35:52.667723+0000","last_fullsized":"2023-10-09T06:35:52.667723+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:02:15.246012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.661279+0000","last_change":"2023-10-09T06:35:52.658640+0000","last_active":"2023-10-09T06:35:52.661279+0000","last_peered":"2023-10-09T06:35:52.661279+0000","last_clean":"2023-10-09T06:35:52.661279+0000","last_became_active":"2023-10-09T06:35:52.658323+0000","last_became_peered":"2023-10-09T06:35:52.658323+0000","last_unstale":"2023-10-09T06:35:52.661279+0000","last_undegraded":"2023-10-09T06:35:52.661279+0000","last_fullsized":"2023-10-09T06:35:52.661279+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:14:06.354004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.640191+0000","last_change":"2023-10-09T06:35:51.639419+0000","last_active":"2023-10-09T06:35:52.640191+0000","last_peered":"2023-10-09T06:35:52.640191+0000","last_clean":"2023-10-09T06:35:52.640191+0000","last_became_active":"2023-10-09T06:35:51.638955+0000","last_became_peered":"2023-10-09T06:35:51.638955+0000","last_unstale":"2023-10-09T06:35:52.640191+0000","last_undegraded":"2023-10-09T06:35:52.640191+0000","last_fullsized":"2023-10-09T06:35:52.640191+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:23:54.185514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.664286+0000","last_change":"2023-10-09T06:35:52.660013+0000","last_active":"2023-10-09T06:35:52.664286+0000","last_peered":"2023-10-09T06:35:52.664286+0000","last_clean":"2023-10-09T06:35:52.664286+0000","last_became_active":"2023-10-09T06:35:52.659275+0000","last_became_peered":"2023-10-09T06:35:52.659275+0000","last_unstale":"2023-10-09T06:35:52.664286+0000","last_undegraded":"2023-10-09T06:35:52.664286+0000","last_fullsized":"2023-10-09T06:35:52.664286+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:02:06.434139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.665059+0000","last_change":"2023-10-09T06:35:51.631719+0000","last_active":"2023-10-09T06:35:52.665059+0000","last_peered":"2023-10-09T06:35:52.665059+0000","last_clean":"2023-10-09T06:35:52.665059+0000","last_became_active":"2023-10-09T06:35:51.629672+0000","last_became_peered":"2023-10-09T06:35:51.629672+0000","last_unstale":"2023-10-09T06:35:52.665059+0000","last_undegraded":"2023-10-09T06:35:52.665059+0000","last_fullsized":"2023-10-09T06:35:52.665059+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:00:36.962657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.651846+0000","last_change":"2023-10-09T06:35:52.649501+0000","last_active":"2023-10-09T06:35:52.651846+0000","last_peered":"2023-10-09T06:35:52.651846+0000","last_clean":"2023-10-09T06:35:52.651846+0000","last_became_active":"2023-10-09T06:35:52.649011+0000","last_became_peered":"2023-10-09T06:35:52.649011+0000","last_unstale":"2023-10-09T06:35:52.651846+0000","last_undegraded":"2023-10-09T06:35:52.651846+0000","last_fullsized":"2023-10-09T06:35:52.651846+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:43:22.600613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.646933+0000","last_change":"2023-10-09T06:35:51.632607+0000","last_active":"2023-10-09T06:35:52.646933+0000","last_peered":"2023-10-09T06:35:52.646933+0000","last_clean":"2023-10-09T06:35:52.646933+0000","last_became_active":"2023-10-09T06:35:51.632358+0000","last_became_peered":"2023-10-09T06:35:51.632358+0000","last_unstale":"2023-10-09T06:35:52.646933+0000","last_undegraded":"2023-10-09T06:35:52.646933+0000","last_fullsized":"2023-10-09T06:35:52.646933+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:55:21.838020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.658378+0000","last_change":"2023-10-09T06:35:52.653004+0000","last_active":"2023-10-09T06:35:52.658378+0000","last_peered":"2023-10-09T06:35:52.658378+0000","last_clean":"2023-10-09T06:35:52.658378+0000","last_became_active":"2023-10-09T06:35:52.652792+0000","last_became_peered":"2023-10-09T06:35:52.652792+0000","last_unstale":"2023-10-09T06:35:52.658378+0000","last_undegraded":"2023-10-09T06:35:52.658378+0000","last_fullsized":"2023-10-09T06:35:52.658378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:03:03.287437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.647486+0000","last_change":"2023-10-09T06:35:51.629251+0000","last_active":"2023-10-09T06:35:52.647486+0000","last_peered":"2023-10-09T06:35:52.647486+0000","last_clean":"2023-10-09T06:35:52.647486+0000","last_became_active":"2023-10-09T06:35:51.628991+0000","last_became_peered":"2023-10-09T06:35:51.628991+0000","last_unstale":"2023-10-09T06:35:52.647486+0000","last_undegraded":"2023-10-09T06:35:52.647486+0000","last_fullsized":"2023-10-09T06:35:52.647486+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:54:04.391468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.659130+0000","last_change":"2023-10-09T06:35:52.652700+0000","last_active":"2023-10-09T06:35:52.659130+0000","last_peered":"2023-10-09T06:35:52.659130+0000","last_clean":"2023-10-09T06:35:52.659130+0000","last_became_active":"2023-10-09T06:35:52.652425+0000","last_became_peered":"2023-10-09T06:35:52.652425+0000","last_unstale":"2023-10-09T06:35:52.659130+0000","last_undegraded":"2023-10-09T06:35:52.659130+0000","last_fullsized":"2023-10-09T06:35:52.659130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:05:23.290057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.648569+0000","last_change":"2023-10-09T06:35:51.628116+0000","last_active":"2023-10-09T06:35:52.648569+0000","last_peered":"2023-10-09T06:35:52.648569+0000","last_clean":"2023-10-09T06:35:52.648569+0000","last_became_active":"2023-10-09T06:35:51.627896+0000","last_became_peered":"2023-10-09T06:35:51.627896+0000","last_unstale":"2023-10-09T06:35:52.648569+0000","last_undegraded":"2023-10-09T06:35:52.648569+0000","last_fullsized":"2023-10-09T06:35:52.648569+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:16:35.599572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.658200+0000","last_change":"2023-10-09T06:35:52.653706+0000","last_active":"2023-10-09T06:35:52.658200+0000","last_peered":"2023-10-09T06:35:52.658200+0000","last_clean":"2023-10-09T06:35:52.658200+0000","last_became_active":"2023-10-09T06:35:52.653462+0000","last_became_peered":"2023-10-09T06:35:52.653462+0000","last_unstale":"2023-10-09T06:35:52.658200+0000","last_undegraded":"2023-10-09T06:35:52.658200+0000","last_fullsized":"2023-10-09T06:35:52.658200+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:04:11.040070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.663273+0000","last_change":"2023-10-09T06:35:51.635409+0000","last_active":"2023-10-09T06:35:52.663273+0000","last_peered":"2023-10-09T06:35:52.663273+0000","last_clean":"2023-10-09T06:35:52.663273+0000","last_became_active":"2023-10-09T06:35:51.635161+0000","last_became_peered":"2023-10-09T06:35:51.635161+0000","last_unstale":"2023-10-09T06:35:52.663273+0000","last_undegraded":"2023-10-09T06:35:52.663273+0000","last_fullsized":"2023-10-09T06:35:52.663273+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:26:30.104488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.650752+0000","last_change":"2023-10-09T06:35:52.648325+0000","last_active":"2023-10-09T06:35:52.650752+0000","last_peered":"2023-10-09T06:35:52.650752+0000","last_clean":"2023-10-09T06:35:52.650752+0000","last_became_active":"2023-10-09T06:35:52.648032+0000","last_became_peered":"2023-10-09T06:35:52.648032+0000","last_unstale":"2023-10-09T06:35:52.650752+0000","last_undegraded":"2023-10-09T06:35:52.650752+0000","last_fullsized":"2023-10-09T06:35:52.650752+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:31:21.904859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.650387+0000","last_change":"2023-10-09T06:35:51.638562+0000","last_active":"2023-10-09T06:35:52.650387+0000","last_peered":"2023-10-09T06:35:52.650387+0000","last_clean":"2023-10-09T06:35:52.650387+0000","last_became_active":"2023-10-09T06:35:51.638285+0000","last_became_peered":"2023-10-09T06:35:51.638285+0000","last_unstale":"2023-10-09T06:35:52.650387+0000","last_undegraded":"2023-10-09T06:35:52.650387+0000","last_fullsized":"2023-10-09T06:35:52.650387+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:47:56.832137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.661375+0000","last_change":"2023-10-09T06:35:52.655279+0000","last_active":"2023-10-09T06:35:52.661375+0000","last_peered":"2023-10-09T06:35:52.661375+0000","last_clean":"2023-10-09T06:35:52.661375+0000","last_became_active":"2023-10-09T06:35:52.654501+0000","last_became_peered":"2023-10-09T06:35:52.654501+0000","last_unstale":"2023-10-09T06:35:52.661375+0000","last_undegraded":"2023-10-09T06:35:52.661375+0000","last_fullsized":"2023-10-09T06:35:52.661375+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:55:18.457568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.639445+0000","last_change":"2023-10-09T06:35:51.630872+0000","last_active":"2023-10-09T06:35:52.639445+0000","last_peered":"2023-10-09T06:35:52.639445+0000","last_clean":"2023-10-09T06:35:52.639445+0000","last_became_active":"2023-10-09T06:35:51.630150+0000","last_became_peered":"2023-10-09T06:35:51.630150+0000","last_unstale":"2023-10-09T06:35:52.639445+0000","last_undegraded":"2023-10-09T06:35:52.639445+0000","last_fullsized":"2023-10-09T06:35:52.639445+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:30:26.813152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.659288+0000","last_change":"2023-10-09T06:35:52.653099+0000","last_active":"2023-10-09T06:35:52.659288+0000","last_peered":"2023-10-09T06:35:52.659288+0000","last_clean":"2023-10-09T06:35:52.659288+0000","last_became_active":"2023-10-09T06:35:52.652892+0000","last_became_peered":"2023-10-09T06:35:52.652892+0000","last_unstale":"2023-10-09T06:35:52.659288+0000","last_undegraded":"2023-10-09T06:35:52.659288+0000","last_fullsized":"2023-10-09T06:35:52.659288+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:43:22.600613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.639300+0000","last_change":"2023-10-09T06:35:51.634017+0000","last_active":"2023-10-09T06:35:52.639300+0000","last_peered":"2023-10-09T06:35:52.639300+0000","last_clean":"2023-10-09T06:35:52.639300+0000","last_became_active":"2023-10-09T06:35:51.633775+0000","last_became_peered":"2023-10-09T06:35:51.633775+0000","last_unstale":"2023-10-09T06:35:52.639300+0000","last_undegraded":"2023-10-09T06:35:52.639300+0000","last_fullsized":"2023-10-09T06:35:52.639300+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:11:49.746928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.652066+0000","last_change":"2023-10-09T06:35:52.649658+0000","last_active":"2023-10-09T06:35:52.652066+0000","last_peered":"2023-10-09T06:35:52.652066+0000","last_clean":"2023-10-09T06:35:52.652066+0000","last_became_active":"2023-10-09T06:35:52.649204+0000","last_became_peered":"2023-10-09T06:35:52.649204+0000","last_unstale":"2023-10-09T06:35:52.652066+0000","last_undegraded":"2023-10-09T06:35:52.652066+0000","last_fullsized":"2023-10-09T06:35:52.652066+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:04:11.040070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.639749+0000","last_change":"2023-10-09T06:35:51.638604+0000","last_active":"2023-10-09T06:35:52.639749+0000","last_peered":"2023-10-09T06:35:52.639749+0000","last_clean":"2023-10-09T06:35:52.639749+0000","last_became_active":"2023-10-09T06:35:51.638391+0000","last_became_peered":"2023-10-09T06:35:51.638391+0000","last_unstale":"2023-10-09T06:35:52.639749+0000","last_undegraded":"2023-10-09T06:35:52.639749+0000","last_fullsized":"2023-10-09T06:35:52.639749+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:14:05.346793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.652130+0000","last_change":"2023-10-09T06:35:52.645140+0000","last_active":"2023-10-09T06:35:52.652130+0000","last_peered":"2023-10-09T06:35:52.652130+0000","last_clean":"2023-10-09T06:35:52.652130+0000","last_became_active":"2023-10-09T06:35:52.644876+0000","last_became_peered":"2023-10-09T06:35:52.644876+0000","last_unstale":"2023-10-09T06:35:52.652130+0000","last_undegraded":"2023-10-09T06:35:52.652130+0000","last_fullsized":"2023-10-09T06:35:52.652130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:05:23.290057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.649542+0000","last_change":"2023-10-09T06:35:51.634273+0000","last_active":"2023-10-09T06:35:52.649542+0000","last_peered":"2023-10-09T06:35:52.649542+0000","last_clean":"2023-10-09T06:35:52.649542+0000","last_became_active":"2023-10-09T06:35:51.634000+0000","last_became_peered":"2023-10-09T06:35:51.634000+0000","last_unstale":"2023-10-09T06:35:52.649542+0000","last_undegraded":"2023-10-09T06:35:52.649542+0000","last_fullsized":"2023-10-09T06:35:52.649542+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:04:26.655638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.660049+0000","last_change":"2023-10-09T06:35:52.659092+0000","last_active":"2023-10-09T06:35:52.660049+0000","last_peered":"2023-10-09T06:35:52.660049+0000","last_clean":"2023-10-09T06:35:52.660049+0000","last_became_active":"2023-10-09T06:35:52.658805+0000","last_became_peered":"2023-10-09T06:35:52.658805+0000","last_unstale":"2023-10-09T06:35:52.660049+0000","last_undegraded":"2023-10-09T06:35:52.660049+0000","last_fullsized":"2023-10-09T06:35:52.660049+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:00:37.969869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.638980+0000","last_change":"2023-10-09T06:35:51.633529+0000","last_active":"2023-10-09T06:35:52.638980+0000","last_peered":"2023-10-09T06:35:52.638980+0000","last_clean":"2023-10-09T06:35:52.638980+0000","last_became_active":"2023-10-09T06:35:51.633269+0000","last_became_peered":"2023-10-09T06:35:51.633269+0000","last_unstale":"2023-10-09T06:35:52.638980+0000","last_undegraded":"2023-10-09T06:35:52.638980+0000","last_fullsized":"2023-10-09T06:35:52.638980+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:41:00.450594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.662379+0000","last_change":"2023-10-09T06:35:52.656014+0000","last_active":"2023-10-09T06:35:52.662379+0000","last_peered":"2023-10-09T06:35:52.662379+0000","last_clean":"2023-10-09T06:35:52.662379+0000","last_became_active":"2023-10-09T06:35:52.655749+0000","last_became_peered":"2023-10-09T06:35:52.655749+0000","last_unstale":"2023-10-09T06:35:52.662379+0000","last_undegraded":"2023-10-09T06:35:52.662379+0000","last_fullsized":"2023-10-09T06:35:52.662379+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:38:30.691298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.649684+0000","last_change":"2023-10-09T06:35:51.636358+0000","last_active":"2023-10-09T06:35:52.649684+0000","last_peered":"2023-10-09T06:35:52.649684+0000","last_clean":"2023-10-09T06:35:52.649684+0000","last_became_active":"2023-10-09T06:35:51.636091+0000","last_became_peered":"2023-10-09T06:35:51.636091+0000","last_unstale":"2023-10-09T06:35:52.649684+0000","last_undegraded":"2023-10-09T06:35:52.649684+0000","last_fullsized":"2023-10-09T06:35:52.649684+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:11:49.746928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.659616+0000","last_change":"2023-10-09T06:35:52.654058+0000","last_active":"2023-10-09T06:35:52.659616+0000","last_peered":"2023-10-09T06:35:52.659616+0000","last_clean":"2023-10-09T06:35:52.659616+0000","last_became_active":"2023-10-09T06:35:52.653766+0000","last_became_peered":"2023-10-09T06:35:52.653766+0000","last_unstale":"2023-10-09T06:35:52.659616+0000","last_undegraded":"2023-10-09T06:35:52.659616+0000","last_fullsized":"2023-10-09T06:35:52.659616+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:35:42.110847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.663353+0000","last_change":"2023-10-09T06:35:51.637316+0000","last_active":"2023-10-09T06:35:52.663353+0000","last_peered":"2023-10-09T06:35:52.663353+0000","last_clean":"2023-10-09T06:35:52.663353+0000","last_became_active":"2023-10-09T06:35:51.637048+0000","last_became_peered":"2023-10-09T06:35:51.637048+0000","last_unstale":"2023-10-09T06:35:52.663353+0000","last_undegraded":"2023-10-09T06:35:52.663353+0000","last_fullsized":"2023-10-09T06:35:52.663353+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:35:41.103636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":211,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.642958+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:35:52.642958+0000","last_peered":"2023-10-09T06:35:52.642958+0000","last_clean":"2023-10-09T06:35:52.642958+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:35:52.642958+0000","last_undegraded":"2023-10-09T06:35:52.642958+0000","last_fullsized":"2023-10-09T06:35:52.642958+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574917,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298796,"kb_used_data":1508,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073044,"statfs":{"total":96636764160,"available":96330797056,"internally_reserved":0,"allocated":1544192,"data_stored":998236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:33 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.50700000000000001,"15min":0.503},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.66300000000000003,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.34100000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65700000000000003,"5min":0.753,"15min":0.753},"last":0.442}]},{"osd":1,"last update":"Mon Oct 9 06:35:33 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.50900000000000001,"15min":0.52100000000000002},"min":{"1min":0.36099999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.63700000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52300000000000002,"15min":0.52400000000000002},"min":{"1min":0.32900000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.66100000000000003,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.58399999999999996}]},{"osd":2,"last update":"Mon Oct 9 06:35:33 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.52100000000000002,"15min":0.50900000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65700000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52900000000000003,"15min":0.54500000000000004},"min":{"1min":0.36799999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.69099999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.50800000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:35:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.53500000000000003,"15min":0.55300000000000005},"min":{"1min":0.36099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56399999999999995,"15min":0.59799999999999998},"min":{"1min":0.42299999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82099999999999995,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.46600000000000003}]},{"osd":4,"last update":"Mon Oct 9 06:35:33 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.48199999999999998,"15min":0.52400000000000002},"min":{"1min":0.307,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.69999999999999996,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56899999999999995,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60099999999999998}]},{"osd":5,"last update":"Mon Oct 9 06:35:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52700000000000002,"15min":0.57699999999999996},"min":{"1min":0.34399999999999997,"5min":0.307,"15min":0.307},"max":{"1min":0.85799999999999998,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58699999999999997,"15min":0.621},"min":{"1min":0.41699999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.74099999999999999,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.64100000000000001}]},{"osd":6,"last update":"Mon Oct 9 06:35:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55100000000000005,"15min":0.59799999999999998},"min":{"1min":0.378,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.78100000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.62},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55300000000000005,"15min":0.61799999999999999},"min":{"1min":0.35399999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.79900000000000004,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.628}]}]},{"osd":6,"up_from":13,"seq":55834574916,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073616,"statfs":{"total":96636764160,"available":96331382784,"internally_reserved":0,"allocated":958464,"data_stored":408464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.495,"15min":0.501},"min":{"1min":0.35399999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.32200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.49099999999999999}]},{"osd":1,"last update":"Mon Oct 9 06:35:36 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.495,"15min":0.51300000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.72499999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.502},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52600000000000002,"15min":0.53200000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.36199999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:35:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51000000000000001,"15min":0.51500000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69499999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.51300000000000001,"15min":0.53300000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.67500000000000004,"5min":0.752,"15min":0.752},"last":0.54700000000000004}]},{"osd":3,"last update":"Mon Oct 9 06:35:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53500000000000003,"15min":0.54800000000000004},"min":{"1min":0.40000000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70899999999999996,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59399999999999997,"15min":0.71799999999999997},"min":{"1min":0.372,"5min":0.37,"15min":0.37},"max":{"1min":0.73499999999999999,"5min":4.7510000000000003,"15min":4.7510000000000003},"last":0.39900000000000002}]},{"osd":4,"last update":"Mon Oct 9 06:35:36 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46800000000000003,"15min":0.47999999999999998},"min":{"1min":0.29799999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.67500000000000004,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.316},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.57199999999999995,"15min":0.58499999999999996},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.76700000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.53800000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:35:36 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.52700000000000002,"15min":0.53300000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.755,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57799999999999996,"15min":0.59999999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78000000000000003,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.57699999999999996}]},{"osd":7,"last update":"Mon Oct 9 06:35:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56499999999999995,"15min":0.59099999999999997},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.434},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.52200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302320,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069520,"statfs":{"total":96636764160,"available":96327188480,"internally_reserved":0,"allocated":958464,"data_stored":407992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.30399999999999999,"15min":0.308},"min":{"1min":0.20200000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34899999999999998,"15min":0.35999999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.53600000000000003,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.38200000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34300000000000003,"15min":0.35099999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.38400000000000001,"15min":0.38600000000000001},"min":{"1min":0.29699999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.57499999999999996,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.441}]},{"osd":3,"last update":"Mon Oct 9 06:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.36099999999999999,"15min":0.36299999999999999},"min":{"1min":0.247,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.41299999999999998,"15min":0.57899999999999996},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.54800000000000004,"5min":5.8529999999999998,"15min":5.8529999999999998},"last":0.34499999999999997}]},{"osd":4,"last update":"Mon Oct 9 06:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.42299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58199999999999996,"15min":0.58599999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.63}]},{"osd":5,"last update":"Mon Oct 9 06:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.62,"15min":0.77800000000000002},"min":{"1min":0.37,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.747,"5min":5.9459999999999997,"15min":5.9459999999999997},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60199999999999998,"15min":0.61199999999999999},"min":{"1min":0.47499999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.92700000000000005,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.64300000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.59599999999999997,"15min":0.748},"min":{"1min":0.39100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73799999999999999,"5min":5.9160000000000004,"15min":5.9160000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.42699999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91000000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.52100000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.89500000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.624,"15min":0.79000000000000004},"min":{"1min":0.40699999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80900000000000005,"5min":5.9729999999999999,"15min":5.9729999999999999},"last":0.61299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574918,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302892,"kb_used_data":1508,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068948,"statfs":{"total":96636764160,"available":96326602752,"internally_reserved":0,"allocated":1544192,"data_stored":998200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.308,"15min":0.317},"min":{"1min":0.214,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.47599999999999998,"5min":0.498,"15min":0.498},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.33700000000000002,"15min":0.34699999999999998},"min":{"1min":0.23100000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.53600000000000003,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.32400000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.35299999999999998,"15min":0.36299999999999999},"min":{"1min":0.252,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.31},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38100000000000001,"15min":0.38900000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.56499999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.375}]},{"osd":3,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35599999999999998,"15min":0.39700000000000002},"min":{"1min":0.27300000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51200000000000001,"5min":1.554,"15min":1.554},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.37,"5min":0.378,"15min":0.40400000000000003},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.58099999999999996,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.35699999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.36599999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70199999999999996,"5min":1.212,"15min":1.212},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61199999999999999,"15min":0.75800000000000001},"min":{"1min":0.41799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82499999999999996,"5min":5.484,"15min":5.484},"last":0.52900000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.61499999999999999,"15min":0.76300000000000001},"min":{"1min":0.37,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.69999999999999996,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57299999999999995,"15min":0.57599999999999996},"min":{"1min":0.40899999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84499999999999997,"5min":1.113,"15min":1.113},"last":0.51400000000000001}]},{"osd":6,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59499999999999997,"15min":0.60699999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.72899999999999998,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.42799999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83899999999999997,"5min":1.169,"15min":1.169},"last":0.629}]},{"osd":7,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.39700000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.85499999999999998,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.628,"15min":0.79000000000000004},"min":{"1min":0.42699999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.71299999999999997,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.58599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574919,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298216,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073624,"statfs":{"total":96636764160,"available":96331390976,"internally_reserved":0,"allocated":950272,"data_stored":408352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.32200000000000001,"15min":0.35999999999999999},"min":{"1min":0.218,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.50800000000000001,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.40100000000000002,"15min":0.60999999999999999},"min":{"1min":0.216,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.53800000000000003,"5min":7.2699999999999996,"15min":7.2699999999999996},"last":0.45400000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.32800000000000001,"15min":0.34899999999999998},"min":{"1min":0.22600000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.497,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.45100000000000001,"15min":0.66600000000000004},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.57599999999999996,"5min":7.327,"15min":7.327},"last":0.51500000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.376,"15min":0.39400000000000002},"min":{"1min":0.28299999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.52900000000000003,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.374,"15min":0.39500000000000002},"min":{"1min":0.25900000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.48399999999999999,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.497}]},{"osd":4,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53700000000000003,"15min":0.54100000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.73299999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.628,"15min":0.83299999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83199999999999996,"5min":7.2389999999999999,"15min":7.2389999999999999},"last":0.66600000000000004}]},{"osd":5,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.69899999999999995,"15min":0.98999999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.747,"5min":10.890000000000001,"15min":10.890000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59099999999999997,"15min":0.60199999999999998},"min":{"1min":0.39800000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.78200000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.78200000000000003}]},{"osd":6,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.64300000000000002,"15min":0.84099999999999997},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77300000000000002,"5min":7.2999999999999998,"15min":7.2999999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57199999999999995,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77000000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.73599999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.67200000000000004,"15min":0.873},"min":{"1min":0.38600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.79900000000000004,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.61599999999999999,"15min":0.628},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79300000000000004,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.77200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574918,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073616,"statfs":{"total":96636764160,"available":96331382784,"internally_reserved":0,"allocated":958464,"data_stored":409334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.30199999999999999,"5min":0.35699999999999998,"15min":0.32700000000000001},"min":{"1min":0.23599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.38300000000000001,"5min":5.8700000000000001,"15min":5.8700000000000001},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.38600000000000001,"15min":0.36599999999999999},"min":{"1min":0.224,"5min":0.222,"15min":0.222},"max":{"1min":0.46200000000000002,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.40000000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.375,"15min":0.35699999999999998},"min":{"1min":0.23899999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.47699999999999998,"5min":4.0469999999999997,"15min":4.0469999999999997},"last":0.375},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40000000000000002,"15min":0.38700000000000001},"min":{"1min":0.27100000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.48699999999999999,"5min":4.1509999999999998,"15min":4.1509999999999998},"last":0.45100000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.373,"15min":0.374},"min":{"1min":0.24399999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.499,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.439},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.38900000000000001,"15min":0.38500000000000001},"min":{"1min":0.26200000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.51500000000000001,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.48399999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58099999999999996,"15min":0.57099999999999995},"min":{"1min":0.372,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80100000000000005,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57299999999999995,"15min":0.56200000000000006},"min":{"1min":0.36599999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68899999999999995,"5min":3.0710000000000002,"15min":3.0710000000000002},"last":0.70399999999999996}]},{"osd":5,"last update":"Mon Oct 9 06:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60999999999999999,"15min":0.58599999999999997},"min":{"1min":0.38200000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.746,"5min":6.2300000000000004,"15min":6.2300000000000004},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.60199999999999998,"15min":0.59199999999999997},"min":{"1min":0.41499999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72299999999999998,"5min":3.0219999999999998,"15min":3.0219999999999998},"last":0.76800000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.60599999999999998,"15min":0.58899999999999997},"min":{"1min":0.42899999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.71199999999999997,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.67900000000000005,"15min":0.64800000000000002},"min":{"1min":0.46400000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.79300000000000004,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.78500000000000003}]},{"osd":7,"last update":"Mon Oct 9 06:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63100000000000001,"15min":0.60999999999999999},"min":{"1min":0.496,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77900000000000003,"5min":4.2629999999999999,"15min":4.2629999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.64700000000000002,"15min":0.61099999999999999},"min":{"1min":0.40500000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.73599999999999999,"5min":7.3650000000000002,"15min":7.3650000000000002},"last":0.72299999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574917,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298212,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073628,"statfs":{"total":96636764160,"available":96331395072,"internally_reserved":0,"allocated":946176,"data_stored":407798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.53200000000000003,"15min":0.56000000000000005},"min":{"1min":0.34499999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.63900000000000001,"5min":1.083,"15min":1.083},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.33300000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.97799999999999998,"5min":1.054,"15min":1.054},"last":0.48999999999999999}]},{"osd":1,"last update":"Mon Oct 9 06:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.65700000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51100000000000001,"15min":0.495},"min":{"1min":0.42099999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.62,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.42099999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52600000000000002,"15min":0.54200000000000004},"min":{"1min":0.35299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71499999999999997,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.438},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.57099999999999995,"15min":0.59899999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.65100000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.55800000000000005}]},{"osd":3,"last update":"Mon Oct 9 06:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54000000000000004,"15min":0.55700000000000005},"min":{"1min":0.39200000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.66700000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.501},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57599999999999996,"15min":0.59099999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.78500000000000003,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.51000000000000001}]},{"osd":5,"last update":"Mon Oct 9 06:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52300000000000002,"15min":0.56100000000000005},"min":{"1min":0.36199999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.76500000000000001,"5min":0.998,"15min":0.998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58799999999999997,"15min":0.61399999999999999},"min":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.751,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.46700000000000003}]},{"osd":6,"last update":"Mon Oct 9 06:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.56999999999999995},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57399999999999995,"15min":0.60199999999999998},"min":{"1min":0.41499999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.45300000000000001}]},{"osd":7,"last update":"Mon Oct 9 06:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54900000000000004,"15min":0.57499999999999996},"min":{"1min":0.33600000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.83199999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.371},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60899999999999999,"15min":0.65100000000000002},"min":{"1min":0.40500000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98999999999999999,"5min":1.175,"15min":1.175},"last":0.34999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574917,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298228,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073612,"statfs":{"total":96636764160,"available":96331378688,"internally_reserved":0,"allocated":962560,"data_stored":409356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.498,"15min":0.50800000000000001},"min":{"1min":0.30599999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.60499999999999998,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.501},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.51300000000000001,"15min":0.51200000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.621,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.45700000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.50900000000000001,"15min":0.51000000000000001},"min":{"1min":0.33000000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.67500000000000004,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.376},{"interface":"front","average":{"1min":0.496,"5min":0.52100000000000002,"15min":0.53100000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64700000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.46999999999999997}]},{"osd":2,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54600000000000004,"15min":0.54400000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.69399999999999995,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58099999999999996,"15min":0.58799999999999997},"min":{"1min":0.40999999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.71699999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.48099999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53900000000000003,"15min":0.54200000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.66100000000000003,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55600000000000005,"15min":0.56399999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.69999999999999996,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.437}]},{"osd":4,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.49399999999999999,"15min":0.496},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68300000000000005,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.71999999999999997,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.35299999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.36099999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72099999999999997,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.55500000000000005,"15min":0.55800000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73799999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.42599999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78100000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.35199999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.84499999999999997,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.54800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:35:56.714 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:35:56.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.860+0000 7fbab14ec700 1 Processor -- start 2023-10-09T06:35:56.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.860+0000 7fbab14ec700 1 -- start start 2023-10-09T06:35:56.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.861+0000 7fbab14ec700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbaac159890 0x7fbaac15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:56.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.861+0000 7fbab14ec700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbaac069560 0x7fbaac0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:56.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.861+0000 7fbab14ec700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaac15a160 con 0x7fbaac159c60 2023-10-09T06:35:56.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.861+0000 7fbab14ec700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaac069f20 con 0x7fbaac069560 2023-10-09T06:35:56.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.861+0000 7fbab14ec700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaac0d13a0 con 0x7fbaac159890 2023-10-09T06:35:56.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.861+0000 7fbaab7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbaac069560 0x7fbaac0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:56.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.861+0000 7fbaa27fc700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fbaac159c60 0x7fbaac15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57714/0 (socket says 172.21.15.163:57714) 2023-10-09T06:35:56.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.861+0000 7fbaa27fc700 1 -- 172.21.15.163:0/4286066326 learned_addr learned my addr 172.21.15.163:0/4286066326 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:35:56.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.862+0000 7fbaaaffd700 1 --2- 172.21.15.163:0/4286066326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbaac159890 0x7fbaac15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:56.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.862+0000 7fbaab7fe700 1 -- 172.21.15.163:0/4286066326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbaac159890 msgr2=0x7fbaac15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:56.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.862+0000 7fbaab7fe700 1 --2- 172.21.15.163:0/4286066326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbaac159890 0x7fbaac15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:56.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.862+0000 7fbaab7fe700 1 -- 172.21.15.163:0/4286066326 >> v1:172.21.15.163:6789/0 conn(0x7fbaac159c60 legacy=0x7fbaac15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:56.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.862+0000 7fbaab7fe700 1 -- 172.21.15.163:0/4286066326 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaac15e140 con 0x7fbaac069560 2023-10-09T06:35:56.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.862+0000 7fbaaa7fc700 1 -- 172.21.15.163:0/4286066326 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 588887218 0 0) 0x7fbaac15a160 con 0x7fbaac159c60 2023-10-09T06:35:56.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.862+0000 7fbaab7fe700 1 --2- 172.21.15.163:0/4286066326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaac069560 0x7fbaac0699e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4009e60 tx=0x7fbaa4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7bb9f2bfeb23ff6 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:56.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.862+0000 7fbaaa7fc700 1 -- 172.21.15.163:0/4286066326 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa400e070 con 0x7fbaac069560 2023-10-09T06:35:56.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.862+0000 7fbaaa7fc700 1 -- 172.21.15.163:0/4286066326 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa400bc30 con 0x7fbaac069560 2023-10-09T06:35:56.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.862+0000 7fbaaa7fc700 1 -- 172.21.15.163:0/4286066326 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4012430 con 0x7fbaac069560 2023-10-09T06:35:56.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.863+0000 7fbab14ec700 1 -- 172.21.15.163:0/4286066326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaac069560 msgr2=0x7fbaac0699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:56.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.863+0000 7fbab14ec700 1 --2- 172.21.15.163:0/4286066326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaac069560 0x7fbaac0699e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4009e60 tx=0x7fbaa4009780 comp rx=0 tx=0).stop 2023-10-09T06:35:56.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.863+0000 7fbab14ec700 1 -- 172.21.15.163:0/4286066326 shutdown_connections 2023-10-09T06:35:56.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.863+0000 7fbab14ec700 1 --2- 172.21.15.163:0/4286066326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaac069560 0x7fbaac0699e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:56.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.863+0000 7fbab14ec700 1 --2- 172.21.15.163:0/4286066326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbaac159890 0x7fbaac15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:56.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.863+0000 7fbab14ec700 1 -- 172.21.15.163:0/4286066326 >> 172.21.15.163:0/4286066326 conn(0x7fbaac068e20 msgr2=0x7fbaac0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:56.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.863+0000 7fbab14ec700 1 -- 172.21.15.163:0/4286066326 shutdown_connections 2023-10-09T06:35:56.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.863+0000 7fbab14ec700 1 -- 172.21.15.163:0/4286066326 wait complete. 2023-10-09T06:35:56.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.864+0000 7fbab14ec700 1 Processor -- start 2023-10-09T06:35:56.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.864+0000 7fbab14ec700 1 -- start start 2023-10-09T06:35:56.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.865+0000 7fbab14ec700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbaac069560 0x7fbaac15be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:56.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.865+0000 7fbab14ec700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaac159890 0x7fbaac0c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:56.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.865+0000 7fbab14ec700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaac159c60 0x7fbaac0c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:56.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.865+0000 7fbab14ec700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaac0cd940 con 0x7fbaac159c60 2023-10-09T06:35:56.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.865+0000 7fbab14ec700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaac0cdc00 con 0x7fbaac159890 2023-10-09T06:35:56.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.865+0000 7fbab14ec700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaac15c8c0 con 0x7fbaac069560 2023-10-09T06:35:56.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.865+0000 7fbaab7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaac159c60 0x7fbaac0c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:56.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.865+0000 7fbaab7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaac159c60 0x7fbaac0c6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56204/0 (socket says 172.21.15.163:56204) 2023-10-09T06:35:56.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.865+0000 7fbaab7fe700 1 -- 172.21.15.163:0/3167836959 learned_addr learned my addr 172.21.15.163:0/3167836959 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:56.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.865+0000 7fbaaaffd700 1 --2- 172.21.15.163:0/3167836959 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbaac069560 0x7fbaac15be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:56.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.865+0000 7fbaa27fc700 1 --2- 172.21.15.163:0/3167836959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaac159890 0x7fbaac0c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:56.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.865+0000 7fbaab7fe700 1 -- 172.21.15.163:0/3167836959 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbaac069560 msgr2=0x7fbaac15be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:56.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.865+0000 7fbaab7fe700 1 --2- 172.21.15.163:0/3167836959 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbaac069560 0x7fbaac15be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:56.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.865+0000 7fbaab7fe700 1 -- 172.21.15.163:0/3167836959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaac159890 msgr2=0x7fbaac0c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:56.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.865+0000 7fbaab7fe700 1 --2- 172.21.15.163:0/3167836959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaac159890 0x7fbaac0c6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:56.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.865+0000 7fbaab7fe700 1 -- 172.21.15.163:0/3167836959 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa4009a20 con 0x7fbaac159c60 2023-10-09T06:35:56.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.865+0000 7fbaaaffd700 1 --2- 172.21.15.163:0/3167836959 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbaac069560 0x7fbaac15be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:35:56.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.865+0000 7fbaa27fc700 1 --2- 172.21.15.163:0/3167836959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaac159890 0x7fbaac0c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:35:56.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.865+0000 7fbaab7fe700 1 --2- 172.21.15.163:0/3167836959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaac159c60 0x7fbaac0c6c50 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4013000 tx=0x7fbaa4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:56.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.865+0000 7fbaa37fe700 1 -- 172.21.15.163:0/3167836959 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4012ea0 con 0x7fbaac159c60 2023-10-09T06:35:56.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.865+0000 7fbaa37fe700 1 -- 172.21.15.163:0/3167836959 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa401e630 con 0x7fbaac159c60 2023-10-09T06:35:56.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.865+0000 7fbab14ec700 1 -- 172.21.15.163:0/3167836959 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaac15cba0 con 0x7fbaac159c60 2023-10-09T06:35:56.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.865+0000 7fbaa37fe700 1 -- 172.21.15.163:0/3167836959 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa401e9d0 con 0x7fbaac159c60 2023-10-09T06:35:56.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.865+0000 7fbab14ec700 1 -- 172.21.15.163:0/3167836959 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaac15d160 con 0x7fbaac159c60 2023-10-09T06:35:56.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.866+0000 7fbab14ec700 1 -- 172.21.15.163:0/3167836959 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaac04c3c0 con 0x7fbaac159c60 2023-10-09T06:35:56.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.867+0000 7fbaa37fe700 1 -- 172.21.15.163:0/3167836959 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fbaa4024070 con 0x7fbaac159c60 2023-10-09T06:35:56.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.867+0000 7fbaa37fe700 1 --2- 172.21.15.163:0/3167836959 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fba8c067170 0x7fba8c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:56.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.867+0000 7fbaa37fe700 1 -- 172.21.15.163:0/3167836959 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbaa40871a0 con 0x7fbaac159c60 2023-10-09T06:35:56.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.867+0000 7fbaaaffd700 1 --2- 172.21.15.163:0/3167836959 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fba8c067170 0x7fba8c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:56.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.868+0000 7fbaaaffd700 1 --2- 172.21.15.163:0/3167836959 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fba8c067170 0x7fba8c069620 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fba9c006fd0 tx=0x7fba9c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:56.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:56.871+0000 7fbaa37fe700 1 -- 172.21.15.163:0/3167836959 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbaa4054180 con 0x7fbaac159c60 2023-10-09T06:35:57.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.050+0000 7fbab14ec700 1 -- 172.21.15.163:0/3167836959 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbaac0cdd40 con 0x7fba8c067170 2023-10-09T06:35:57.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.058+0000 7fbaa37fe700 1 -- 172.21.15.163:0/3167836959 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289484 (secure 0 0 0) 0x7fbaac0cdd40 con 0x7fba8c067170 2023-10-09T06:35:57.063 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:35:57.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.061+0000 7fbab14ec700 1 -- 172.21.15.163:0/3167836959 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fba8c067170 msgr2=0x7fba8c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:57.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.061+0000 7fbab14ec700 1 --2- 172.21.15.163:0/3167836959 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fba8c067170 0x7fba8c069620 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fba9c006fd0 tx=0x7fba9c008040 comp rx=0 tx=0).stop 2023-10-09T06:35:57.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.061+0000 7fbab14ec700 1 -- 172.21.15.163:0/3167836959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaac159c60 msgr2=0x7fbaac0c6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:57.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.061+0000 7fbab14ec700 1 --2- 172.21.15.163:0/3167836959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaac159c60 0x7fbaac0c6c50 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4013000 tx=0x7fbaa4019000 comp rx=0 tx=0).stop 2023-10-09T06:35:57.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.061+0000 7fbab14ec700 1 -- 172.21.15.163:0/3167836959 shutdown_connections 2023-10-09T06:35:57.066 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.061+0000 7fbab14ec700 1 --2- 172.21.15.163:0/3167836959 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fba8c067170 0x7fba8c069620 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:57.066 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.061+0000 7fbab14ec700 1 --2- 172.21.15.163:0/3167836959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaac159c60 0x7fbaac0c6c50 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:57.066 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.061+0000 7fbab14ec700 1 --2- 172.21.15.163:0/3167836959 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbaac069560 0x7fbaac15be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:57.066 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.061+0000 7fbab14ec700 1 --2- 172.21.15.163:0/3167836959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaac159890 0x7fbaac0c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:57.066 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.061+0000 7fbab14ec700 1 -- 172.21.15.163:0/3167836959 >> 172.21.15.163:0/3167836959 conn(0x7fbaac068e20 msgr2=0x7fbaac157720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:57.066 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.061+0000 7fbab14ec700 1 -- 172.21.15.163:0/3167836959 shutdown_connections 2023-10-09T06:35:57.066 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.061+0000 7fbab14ec700 1 -- 172.21.15.163:0/3167836959 wait complete. 2023-10-09T06:35:57.066 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:35:57.075 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2023-10-09T06:35:56.376308+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2395112,"kb_used_data":8616,"kb_used_omap":0,"kb_used_meta":2386432,"kb_avail":752579608,"statfs":{"total":773094113280,"available":770641518592,"internally_reserved":0,"allocated":8822784,"data_stored":4447732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2443706368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.759281"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.657970+0000","last_change":"2023-10-09T06:35:52.650315+0000","last_active":"2023-10-09T06:35:52.657970+0000","last_peered":"2023-10-09T06:35:52.657970+0000","last_clean":"2023-10-09T06:35:52.657970+0000","last_became_active":"2023-10-09T06:35:52.650047+0000","last_became_peered":"2023-10-09T06:35:52.650047+0000","last_unstale":"2023-10-09T06:35:52.657970+0000","last_undegraded":"2023-10-09T06:35:52.657970+0000","last_fullsized":"2023-10-09T06:35:52.657970+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:10:47.085979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.659857+0000","last_change":"2023-10-09T06:35:52.653970+0000","last_active":"2023-10-09T06:35:52.659857+0000","last_peered":"2023-10-09T06:35:52.659857+0000","last_clean":"2023-10-09T06:35:52.659857+0000","last_became_active":"2023-10-09T06:35:52.653711+0000","last_became_peered":"2023-10-09T06:35:52.653711+0000","last_unstale":"2023-10-09T06:35:52.659857+0000","last_undegraded":"2023-10-09T06:35:52.659857+0000","last_fullsized":"2023-10-09T06:35:52.659857+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:07:54.245113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.650819+0000","last_change":"2023-10-09T06:35:52.648946+0000","last_active":"2023-10-09T06:35:52.650819+0000","last_peered":"2023-10-09T06:35:52.650819+0000","last_clean":"2023-10-09T06:35:52.650819+0000","last_became_active":"2023-10-09T06:35:52.648066+0000","last_became_peered":"2023-10-09T06:35:52.648066+0000","last_unstale":"2023-10-09T06:35:52.650819+0000","last_undegraded":"2023-10-09T06:35:52.650819+0000","last_fullsized":"2023-10-09T06:35:52.650819+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:35:12.091172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.660835+0000","last_change":"2023-10-09T06:35:52.657178+0000","last_active":"2023-10-09T06:35:52.660835+0000","last_peered":"2023-10-09T06:35:52.660835+0000","last_clean":"2023-10-09T06:35:52.660835+0000","last_became_active":"2023-10-09T06:35:52.656906+0000","last_became_peered":"2023-10-09T06:35:52.656906+0000","last_unstale":"2023-10-09T06:35:52.660835+0000","last_undegraded":"2023-10-09T06:35:52.660835+0000","last_fullsized":"2023-10-09T06:35:52.660835+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:13:27.952312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.657517+0000","last_change":"2023-10-09T06:35:52.651381+0000","last_active":"2023-10-09T06:35:52.657517+0000","last_peered":"2023-10-09T06:35:52.657517+0000","last_clean":"2023-10-09T06:35:52.657517+0000","last_became_active":"2023-10-09T06:35:52.651091+0000","last_became_peered":"2023-10-09T06:35:52.651091+0000","last_unstale":"2023-10-09T06:35:52.657517+0000","last_undegraded":"2023-10-09T06:35:52.657517+0000","last_fullsized":"2023-10-09T06:35:52.657517+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:05:30.191217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.659103+0000","last_change":"2023-10-09T06:35:52.653182+0000","last_active":"2023-10-09T06:35:52.659103+0000","last_peered":"2023-10-09T06:35:52.659103+0000","last_clean":"2023-10-09T06:35:52.659103+0000","last_became_active":"2023-10-09T06:35:52.652927+0000","last_became_peered":"2023-10-09T06:35:52.652927+0000","last_unstale":"2023-10-09T06:35:52.659103+0000","last_undegraded":"2023-10-09T06:35:52.659103+0000","last_fullsized":"2023-10-09T06:35:52.659103+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:21:30.385965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.660222+0000","last_change":"2023-10-09T06:35:52.653565+0000","last_active":"2023-10-09T06:35:52.660222+0000","last_peered":"2023-10-09T06:35:52.660222+0000","last_clean":"2023-10-09T06:35:52.660222+0000","last_became_active":"2023-10-09T06:35:52.653300+0000","last_became_peered":"2023-10-09T06:35:52.653300+0000","last_unstale":"2023-10-09T06:35:52.660222+0000","last_undegraded":"2023-10-09T06:35:52.660222+0000","last_fullsized":"2023-10-09T06:35:52.660222+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:42:44.459965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.660042+0000","last_change":"2023-10-09T06:35:52.659248+0000","last_active":"2023-10-09T06:35:52.660042+0000","last_peered":"2023-10-09T06:35:52.660042+0000","last_clean":"2023-10-09T06:35:52.660042+0000","last_became_active":"2023-10-09T06:35:52.658957+0000","last_became_peered":"2023-10-09T06:35:52.658957+0000","last_unstale":"2023-10-09T06:35:52.660042+0000","last_undegraded":"2023-10-09T06:35:52.660042+0000","last_fullsized":"2023-10-09T06:35:52.660042+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:10:47.085979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.653723+0000","last_change":"2023-10-09T06:35:52.649500+0000","last_active":"2023-10-09T06:35:52.653723+0000","last_peered":"2023-10-09T06:35:52.653723+0000","last_clean":"2023-10-09T06:35:52.653723+0000","last_became_active":"2023-10-09T06:35:52.649219+0000","last_became_peered":"2023-10-09T06:35:52.649219+0000","last_unstale":"2023-10-09T06:35:52.653723+0000","last_undegraded":"2023-10-09T06:35:52.653723+0000","last_fullsized":"2023-10-09T06:35:52.653723+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:59:03.103633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.661146+0000","last_change":"2023-10-09T06:35:52.653146+0000","last_active":"2023-10-09T06:35:52.661146+0000","last_peered":"2023-10-09T06:35:52.661146+0000","last_clean":"2023-10-09T06:35:52.661146+0000","last_became_active":"2023-10-09T06:35:52.652893+0000","last_became_peered":"2023-10-09T06:35:52.652893+0000","last_unstale":"2023-10-09T06:35:52.661146+0000","last_undegraded":"2023-10-09T06:35:52.661146+0000","last_fullsized":"2023-10-09T06:35:52.661146+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:35:42.110847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.657635+0000","last_change":"2023-10-09T06:35:52.653309+0000","last_active":"2023-10-09T06:35:52.657635+0000","last_peered":"2023-10-09T06:35:52.657635+0000","last_clean":"2023-10-09T06:35:52.657635+0000","last_became_active":"2023-10-09T06:35:52.653033+0000","last_became_peered":"2023-10-09T06:35:52.653033+0000","last_unstale":"2023-10-09T06:35:52.657635+0000","last_undegraded":"2023-10-09T06:35:52.657635+0000","last_fullsized":"2023-10-09T06:35:52.657635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:21:30.385965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.651942+0000","last_change":"2023-10-09T06:35:52.647755+0000","last_active":"2023-10-09T06:35:52.651942+0000","last_peered":"2023-10-09T06:35:52.651942+0000","last_clean":"2023-10-09T06:35:52.651942+0000","last_became_active":"2023-10-09T06:35:52.647239+0000","last_became_peered":"2023-10-09T06:35:52.647239+0000","last_unstale":"2023-10-09T06:35:52.651942+0000","last_undegraded":"2023-10-09T06:35:52.651942+0000","last_fullsized":"2023-10-09T06:35:52.651942+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:21:30.385965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.655858+0000","last_change":"2023-10-09T06:35:52.652392+0000","last_active":"2023-10-09T06:35:52.655858+0000","last_peered":"2023-10-09T06:35:52.655858+0000","last_clean":"2023-10-09T06:35:52.655858+0000","last_became_active":"2023-10-09T06:35:52.652092+0000","last_became_peered":"2023-10-09T06:35:52.652092+0000","last_unstale":"2023-10-09T06:35:52.655858+0000","last_undegraded":"2023-10-09T06:35:52.655858+0000","last_fullsized":"2023-10-09T06:35:52.655858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:33:54.388524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.664234+0000","last_change":"2023-10-09T06:35:52.659657+0000","last_active":"2023-10-09T06:35:52.664234+0000","last_peered":"2023-10-09T06:35:52.664234+0000","last_clean":"2023-10-09T06:35:52.664234+0000","last_became_active":"2023-10-09T06:35:52.659001+0000","last_became_peered":"2023-10-09T06:35:52.659001+0000","last_unstale":"2023-10-09T06:35:52.664234+0000","last_undegraded":"2023-10-09T06:35:52.664234+0000","last_fullsized":"2023-10-09T06:35:52.664234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:57:30.555787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.653548+0000","last_change":"2023-10-09T06:35:52.650985+0000","last_active":"2023-10-09T06:35:52.653548+0000","last_peered":"2023-10-09T06:35:52.653548+0000","last_clean":"2023-10-09T06:35:52.653548+0000","last_became_active":"2023-10-09T06:35:52.650638+0000","last_became_peered":"2023-10-09T06:35:52.650638+0000","last_unstale":"2023-10-09T06:35:52.653548+0000","last_undegraded":"2023-10-09T06:35:52.653548+0000","last_fullsized":"2023-10-09T06:35:52.653548+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:13:56.638985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.657154+0000","last_change":"2023-10-09T06:35:52.651163+0000","last_active":"2023-10-09T06:35:52.657154+0000","last_peered":"2023-10-09T06:35:52.657154+0000","last_clean":"2023-10-09T06:35:52.657154+0000","last_became_active":"2023-10-09T06:35:52.650078+0000","last_became_peered":"2023-10-09T06:35:52.650078+0000","last_unstale":"2023-10-09T06:35:52.657154+0000","last_undegraded":"2023-10-09T06:35:52.657154+0000","last_fullsized":"2023-10-09T06:35:52.657154+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:10:47.085979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.661365+0000","last_change":"2023-10-09T06:35:52.658022+0000","last_active":"2023-10-09T06:35:52.661365+0000","last_peered":"2023-10-09T06:35:52.661365+0000","last_clean":"2023-10-09T06:35:52.661365+0000","last_became_active":"2023-10-09T06:35:52.653730+0000","last_became_peered":"2023-10-09T06:35:52.653730+0000","last_unstale":"2023-10-09T06:35:52.661365+0000","last_undegraded":"2023-10-09T06:35:52.661365+0000","last_fullsized":"2023-10-09T06:35:52.661365+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:59:03.103633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.660755+0000","last_change":"2023-10-09T06:35:52.654777+0000","last_active":"2023-10-09T06:35:52.660755+0000","last_peered":"2023-10-09T06:35:52.660755+0000","last_clean":"2023-10-09T06:35:52.660755+0000","last_became_active":"2023-10-09T06:35:52.654498+0000","last_became_peered":"2023-10-09T06:35:52.654498+0000","last_unstale":"2023-10-09T06:35:52.660755+0000","last_undegraded":"2023-10-09T06:35:52.660755+0000","last_fullsized":"2023-10-09T06:35:52.660755+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:26:24.468235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.665768+0000","last_change":"2023-10-09T06:35:52.649349+0000","last_active":"2023-10-09T06:35:52.665768+0000","last_peered":"2023-10-09T06:35:52.665768+0000","last_clean":"2023-10-09T06:35:52.665768+0000","last_became_active":"2023-10-09T06:35:52.649112+0000","last_became_peered":"2023-10-09T06:35:52.649112+0000","last_unstale":"2023-10-09T06:35:52.665768+0000","last_undegraded":"2023-10-09T06:35:52.665768+0000","last_fullsized":"2023-10-09T06:35:52.665768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:04:11.040070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.669110+0000","last_change":"2023-10-09T06:35:52.667954+0000","last_active":"2023-10-09T06:35:52.669110+0000","last_peered":"2023-10-09T06:35:52.669110+0000","last_clean":"2023-10-09T06:35:52.669110+0000","last_became_active":"2023-10-09T06:35:52.667656+0000","last_became_peered":"2023-10-09T06:35:52.667656+0000","last_unstale":"2023-10-09T06:35:52.669110+0000","last_undegraded":"2023-10-09T06:35:52.669110+0000","last_fullsized":"2023-10-09T06:35:52.669110+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:24:19.715831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.669152+0000","last_change":"2023-10-09T06:35:52.668017+0000","last_active":"2023-10-09T06:35:52.669152+0000","last_peered":"2023-10-09T06:35:52.669152+0000","last_clean":"2023-10-09T06:35:52.669152+0000","last_became_active":"2023-10-09T06:35:52.667618+0000","last_became_peered":"2023-10-09T06:35:52.667618+0000","last_unstale":"2023-10-09T06:35:52.669152+0000","last_undegraded":"2023-10-09T06:35:52.669152+0000","last_fullsized":"2023-10-09T06:35:52.669152+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:59:37.178211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.668133+0000","last_change":"2023-10-09T06:35:52.666822+0000","last_active":"2023-10-09T06:35:52.668133+0000","last_peered":"2023-10-09T06:35:52.668133+0000","last_clean":"2023-10-09T06:35:52.668133+0000","last_became_active":"2023-10-09T06:35:52.666508+0000","last_became_peered":"2023-10-09T06:35:52.666508+0000","last_unstale":"2023-10-09T06:35:52.668133+0000","last_undegraded":"2023-10-09T06:35:52.668133+0000","last_fullsized":"2023-10-09T06:35:52.668133+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:59:03.103633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.647107+0000","last_change":"2023-10-09T06:35:52.641370+0000","last_active":"2023-10-09T06:35:52.647107+0000","last_peered":"2023-10-09T06:35:52.647107+0000","last_clean":"2023-10-09T06:35:52.647107+0000","last_became_active":"2023-10-09T06:35:52.641054+0000","last_became_peered":"2023-10-09T06:35:52.641054+0000","last_unstale":"2023-10-09T06:35:52.647107+0000","last_undegraded":"2023-10-09T06:35:52.647107+0000","last_fullsized":"2023-10-09T06:35:52.647107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:55:43.724447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.671278+0000","last_change":"2023-10-09T06:35:52.669011+0000","last_active":"2023-10-09T06:35:52.671278+0000","last_peered":"2023-10-09T06:35:52.671278+0000","last_clean":"2023-10-09T06:35:52.671278+0000","last_became_active":"2023-10-09T06:35:52.668804+0000","last_became_peered":"2023-10-09T06:35:52.668804+0000","last_unstale":"2023-10-09T06:35:52.671278+0000","last_undegraded":"2023-10-09T06:35:52.671278+0000","last_fullsized":"2023-10-09T06:35:52.671278+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:10:50.929917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.656310+0000","last_change":"2023-10-09T06:35:52.651591+0000","last_active":"2023-10-09T06:35:52.656310+0000","last_peered":"2023-10-09T06:35:52.656310+0000","last_clean":"2023-10-09T06:35:52.656310+0000","last_became_active":"2023-10-09T06:35:52.651222+0000","last_became_peered":"2023-10-09T06:35:52.651222+0000","last_unstale":"2023-10-09T06:35:52.656310+0000","last_undegraded":"2023-10-09T06:35:52.656310+0000","last_fullsized":"2023-10-09T06:35:52.656310+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:04:11.040070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.661400+0000","last_change":"2023-10-09T06:35:52.658434+0000","last_active":"2023-10-09T06:35:52.661400+0000","last_peered":"2023-10-09T06:35:52.661400+0000","last_clean":"2023-10-09T06:35:52.661400+0000","last_became_active":"2023-10-09T06:35:52.658018+0000","last_became_peered":"2023-10-09T06:35:52.658018+0000","last_unstale":"2023-10-09T06:35:52.661400+0000","last_undegraded":"2023-10-09T06:35:52.661400+0000","last_fullsized":"2023-10-09T06:35:52.661400+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:16:36.606784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.661306+0000","last_change":"2023-10-09T06:35:52.659376+0000","last_active":"2023-10-09T06:35:52.661306+0000","last_peered":"2023-10-09T06:35:52.661306+0000","last_clean":"2023-10-09T06:35:52.661306+0000","last_became_active":"2023-10-09T06:35:52.659085+0000","last_became_peered":"2023-10-09T06:35:52.659085+0000","last_unstale":"2023-10-09T06:35:52.661306+0000","last_undegraded":"2023-10-09T06:35:52.661306+0000","last_fullsized":"2023-10-09T06:35:52.661306+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:21:30.385965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.659204+0000","last_change":"2023-10-09T06:35:52.652059+0000","last_active":"2023-10-09T06:35:52.659204+0000","last_peered":"2023-10-09T06:35:52.659204+0000","last_clean":"2023-10-09T06:35:52.659204+0000","last_became_active":"2023-10-09T06:35:52.651773+0000","last_became_peered":"2023-10-09T06:35:52.651773+0000","last_unstale":"2023-10-09T06:35:52.659204+0000","last_undegraded":"2023-10-09T06:35:52.659204+0000","last_fullsized":"2023-10-09T06:35:52.659204+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:13:52.904899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.657898+0000","last_change":"2023-10-09T06:35:52.650110+0000","last_active":"2023-10-09T06:35:52.657898+0000","last_peered":"2023-10-09T06:35:52.657898+0000","last_clean":"2023-10-09T06:35:52.657898+0000","last_became_active":"2023-10-09T06:35:52.648802+0000","last_became_peered":"2023-10-09T06:35:52.648802+0000","last_unstale":"2023-10-09T06:35:52.657898+0000","last_undegraded":"2023-10-09T06:35:52.657898+0000","last_fullsized":"2023-10-09T06:35:52.657898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:10:47.085979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.662182+0000","last_change":"2023-10-09T06:35:52.658596+0000","last_active":"2023-10-09T06:35:52.662182+0000","last_peered":"2023-10-09T06:35:52.662182+0000","last_clean":"2023-10-09T06:35:52.662182+0000","last_became_active":"2023-10-09T06:35:52.658144+0000","last_became_peered":"2023-10-09T06:35:52.658144+0000","last_unstale":"2023-10-09T06:35:52.662182+0000","last_undegraded":"2023-10-09T06:35:52.662182+0000","last_fullsized":"2023-10-09T06:35:52.662182+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:43:22.600613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.656185+0000","last_change":"2023-10-09T06:35:52.650675+0000","last_active":"2023-10-09T06:35:52.656185+0000","last_peered":"2023-10-09T06:35:52.656185+0000","last_clean":"2023-10-09T06:35:52.656185+0000","last_became_active":"2023-10-09T06:35:52.649111+0000","last_became_peered":"2023-10-09T06:35:52.649111+0000","last_unstale":"2023-10-09T06:35:52.656185+0000","last_undegraded":"2023-10-09T06:35:52.656185+0000","last_fullsized":"2023-10-09T06:35:52.656185+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:55:43.724447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.661848+0000","last_change":"2023-10-09T06:35:52.655334+0000","last_active":"2023-10-09T06:35:52.661848+0000","last_peered":"2023-10-09T06:35:52.661848+0000","last_clean":"2023-10-09T06:35:52.661848+0000","last_became_active":"2023-10-09T06:35:52.654976+0000","last_became_peered":"2023-10-09T06:35:52.654976+0000","last_unstale":"2023-10-09T06:35:52.661848+0000","last_undegraded":"2023-10-09T06:35:52.661848+0000","last_fullsized":"2023-10-09T06:35:52.661848+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:38:23.450418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.658181+0000","last_change":"2023-10-09T06:35:52.650391+0000","last_active":"2023-10-09T06:35:52.658181+0000","last_peered":"2023-10-09T06:35:52.658181+0000","last_clean":"2023-10-09T06:35:52.658181+0000","last_became_active":"2023-10-09T06:35:52.650095+0000","last_became_peered":"2023-10-09T06:35:52.650095+0000","last_unstale":"2023-10-09T06:35:52.658181+0000","last_undegraded":"2023-10-09T06:35:52.658181+0000","last_fullsized":"2023-10-09T06:35:52.658181+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:35:42.110847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"76'4","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.663903+0000","last_change":"2023-10-09T06:35:51.628579+0000","last_active":"2023-10-09T06:35:52.663903+0000","last_peered":"2023-10-09T06:35:52.663903+0000","last_clean":"2023-10-09T06:35:52.663903+0000","last_became_active":"2023-10-09T06:35:51.628237+0000","last_became_peered":"2023-10-09T06:35:51.628237+0000","last_unstale":"2023-10-09T06:35:52.663903+0000","last_undegraded":"2023-10-09T06:35:52.663903+0000","last_fullsized":"2023-10-09T06:35:52.663903+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:51:31.112094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.647349+0000","last_change":"2023-10-09T06:35:52.642018+0000","last_active":"2023-10-09T06:35:52.647349+0000","last_peered":"2023-10-09T06:35:52.647349+0000","last_clean":"2023-10-09T06:35:52.647349+0000","last_became_active":"2023-10-09T06:35:52.641757+0000","last_became_peered":"2023-10-09T06:35:52.641757+0000","last_unstale":"2023-10-09T06:35:52.647349+0000","last_undegraded":"2023-10-09T06:35:52.647349+0000","last_fullsized":"2023-10-09T06:35:52.647349+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:05:30.191217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.664819+0000","last_change":"2023-10-09T06:35:51.634019+0000","last_active":"2023-10-09T06:35:52.664819+0000","last_peered":"2023-10-09T06:35:52.664819+0000","last_clean":"2023-10-09T06:35:52.664819+0000","last_became_active":"2023-10-09T06:35:51.633736+0000","last_became_peered":"2023-10-09T06:35:51.633736+0000","last_unstale":"2023-10-09T06:35:52.664819+0000","last_undegraded":"2023-10-09T06:35:52.664819+0000","last_fullsized":"2023-10-09T06:35:52.664819+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:39:23.129225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.661468+0000","last_change":"2023-10-09T06:35:52.655477+0000","last_active":"2023-10-09T06:35:52.661468+0000","last_peered":"2023-10-09T06:35:52.661468+0000","last_clean":"2023-10-09T06:35:52.661468+0000","last_became_active":"2023-10-09T06:35:52.654965+0000","last_became_peered":"2023-10-09T06:35:52.654965+0000","last_unstale":"2023-10-09T06:35:52.661468+0000","last_undegraded":"2023-10-09T06:35:52.661468+0000","last_fullsized":"2023-10-09T06:35:52.661468+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:19:47.218422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.667926+0000","last_change":"2023-10-09T06:35:51.638920+0000","last_active":"2023-10-09T06:35:52.667926+0000","last_peered":"2023-10-09T06:35:52.667926+0000","last_clean":"2023-10-09T06:35:52.667926+0000","last_became_active":"2023-10-09T06:35:51.638548+0000","last_became_peered":"2023-10-09T06:35:51.638548+0000","last_unstale":"2023-10-09T06:35:52.667926+0000","last_undegraded":"2023-10-09T06:35:52.667926+0000","last_fullsized":"2023-10-09T06:35:52.667926+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:55:17.450357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.659055+0000","last_change":"2023-10-09T06:35:52.654497+0000","last_active":"2023-10-09T06:35:52.659055+0000","last_peered":"2023-10-09T06:35:52.659055+0000","last_clean":"2023-10-09T06:35:52.659055+0000","last_became_active":"2023-10-09T06:35:52.654166+0000","last_became_peered":"2023-10-09T06:35:52.654166+0000","last_unstale":"2023-10-09T06:35:52.659055+0000","last_undegraded":"2023-10-09T06:35:52.659055+0000","last_fullsized":"2023-10-09T06:35:52.659055+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:39:29.529534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.667988+0000","last_change":"2023-10-09T06:35:51.639411+0000","last_active":"2023-10-09T06:35:52.667988+0000","last_peered":"2023-10-09T06:35:52.667988+0000","last_clean":"2023-10-09T06:35:52.667988+0000","last_became_active":"2023-10-09T06:35:51.638841+0000","last_became_peered":"2023-10-09T06:35:51.638841+0000","last_unstale":"2023-10-09T06:35:52.667988+0000","last_undegraded":"2023-10-09T06:35:52.667988+0000","last_fullsized":"2023-10-09T06:35:52.667988+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:19:46.211210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.662164+0000","last_change":"2023-10-09T06:35:52.658425+0000","last_active":"2023-10-09T06:35:52.662164+0000","last_peered":"2023-10-09T06:35:52.662164+0000","last_clean":"2023-10-09T06:35:52.662164+0000","last_became_active":"2023-10-09T06:35:52.656477+0000","last_became_peered":"2023-10-09T06:35:52.656477+0000","last_unstale":"2023-10-09T06:35:52.662164+0000","last_undegraded":"2023-10-09T06:35:52.662164+0000","last_fullsized":"2023-10-09T06:35:52.662164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:55:43.724447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.663305+0000","last_change":"2023-10-09T06:35:51.637653+0000","last_active":"2023-10-09T06:35:52.663305+0000","last_peered":"2023-10-09T06:35:52.663305+0000","last_clean":"2023-10-09T06:35:52.663305+0000","last_became_active":"2023-10-09T06:35:51.637228+0000","last_became_peered":"2023-10-09T06:35:51.637228+0000","last_unstale":"2023-10-09T06:35:52.663305+0000","last_undegraded":"2023-10-09T06:35:52.663305+0000","last_fullsized":"2023-10-09T06:35:52.663305+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:41:00.450594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.661108+0000","last_change":"2023-10-09T06:35:52.657387+0000","last_active":"2023-10-09T06:35:52.661108+0000","last_peered":"2023-10-09T06:35:52.661108+0000","last_clean":"2023-10-09T06:35:52.661108+0000","last_became_active":"2023-10-09T06:35:52.653930+0000","last_became_peered":"2023-10-09T06:35:52.653930+0000","last_unstale":"2023-10-09T06:35:52.661108+0000","last_undegraded":"2023-10-09T06:35:52.661108+0000","last_fullsized":"2023-10-09T06:35:52.661108+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:07:54.245113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.665047+0000","last_change":"2023-10-09T06:35:51.634538+0000","last_active":"2023-10-09T06:35:52.665047+0000","last_peered":"2023-10-09T06:35:52.665047+0000","last_clean":"2023-10-09T06:35:52.665047+0000","last_became_active":"2023-10-09T06:35:51.634240+0000","last_became_peered":"2023-10-09T06:35:51.634240+0000","last_unstale":"2023-10-09T06:35:52.665047+0000","last_undegraded":"2023-10-09T06:35:52.665047+0000","last_fullsized":"2023-10-09T06:35:52.665047+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:19:09.292729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.650847+0000","last_change":"2023-10-09T06:35:52.648848+0000","last_active":"2023-10-09T06:35:52.650847+0000","last_peered":"2023-10-09T06:35:52.650847+0000","last_clean":"2023-10-09T06:35:52.650847+0000","last_became_active":"2023-10-09T06:35:52.648551+0000","last_became_peered":"2023-10-09T06:35:52.648551+0000","last_unstale":"2023-10-09T06:35:52.650847+0000","last_undegraded":"2023-10-09T06:35:52.650847+0000","last_fullsized":"2023-10-09T06:35:52.650847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:28:42.829846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.639169+0000","last_change":"2023-10-09T06:35:51.635731+0000","last_active":"2023-10-09T06:35:52.639169+0000","last_peered":"2023-10-09T06:35:52.639169+0000","last_clean":"2023-10-09T06:35:52.639169+0000","last_became_active":"2023-10-09T06:35:51.635504+0000","last_became_peered":"2023-10-09T06:35:51.635504+0000","last_unstale":"2023-10-09T06:35:52.639169+0000","last_undegraded":"2023-10-09T06:35:52.639169+0000","last_fullsized":"2023-10-09T06:35:52.639169+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:38:22.443207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.665389+0000","last_change":"2023-10-09T06:35:52.660517+0000","last_active":"2023-10-09T06:35:52.665389+0000","last_peered":"2023-10-09T06:35:52.665389+0000","last_clean":"2023-10-09T06:35:52.665389+0000","last_became_active":"2023-10-09T06:35:52.660219+0000","last_became_peered":"2023-10-09T06:35:52.660219+0000","last_unstale":"2023-10-09T06:35:52.665389+0000","last_undegraded":"2023-10-09T06:35:52.665389+0000","last_fullsized":"2023-10-09T06:35:52.665389+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:31:21.904859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.649514+0000","last_change":"2023-10-09T06:35:51.636565+0000","last_active":"2023-10-09T06:35:52.649514+0000","last_peered":"2023-10-09T06:35:52.649514+0000","last_clean":"2023-10-09T06:35:52.649514+0000","last_became_active":"2023-10-09T06:35:51.636248+0000","last_became_peered":"2023-10-09T06:35:51.636248+0000","last_unstale":"2023-10-09T06:35:52.649514+0000","last_undegraded":"2023-10-09T06:35:52.649514+0000","last_fullsized":"2023-10-09T06:35:52.649514+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:39:23.129225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.661927+0000","last_change":"2023-10-09T06:35:52.655665+0000","last_active":"2023-10-09T06:35:52.661927+0000","last_peered":"2023-10-09T06:35:52.661927+0000","last_clean":"2023-10-09T06:35:52.661927+0000","last_became_active":"2023-10-09T06:35:52.655385+0000","last_became_peered":"2023-10-09T06:35:52.655385+0000","last_unstale":"2023-10-09T06:35:52.661927+0000","last_undegraded":"2023-10-09T06:35:52.661927+0000","last_fullsized":"2023-10-09T06:35:52.661927+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:11:29.107451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.639536+0000","last_change":"2023-10-09T06:35:51.641020+0000","last_active":"2023-10-09T06:35:52.639536+0000","last_peered":"2023-10-09T06:35:52.639536+0000","last_clean":"2023-10-09T06:35:52.639536+0000","last_became_active":"2023-10-09T06:35:51.640740+0000","last_became_peered":"2023-10-09T06:35:51.640740+0000","last_unstale":"2023-10-09T06:35:52.639536+0000","last_undegraded":"2023-10-09T06:35:52.639536+0000","last_fullsized":"2023-10-09T06:35:52.639536+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:32:58.351321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.660333+0000","last_change":"2023-10-09T06:35:52.653016+0000","last_active":"2023-10-09T06:35:52.660333+0000","last_peered":"2023-10-09T06:35:52.660333+0000","last_clean":"2023-10-09T06:35:52.660333+0000","last_became_active":"2023-10-09T06:35:52.652720+0000","last_became_peered":"2023-10-09T06:35:52.652720+0000","last_unstale":"2023-10-09T06:35:52.660333+0000","last_undegraded":"2023-10-09T06:35:52.660333+0000","last_fullsized":"2023-10-09T06:35:52.660333+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:55:43.724447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.639319+0000","last_change":"2023-10-09T06:35:51.634208+0000","last_active":"2023-10-09T06:35:52.639319+0000","last_peered":"2023-10-09T06:35:52.639319+0000","last_clean":"2023-10-09T06:35:52.639319+0000","last_became_active":"2023-10-09T06:35:51.633353+0000","last_became_peered":"2023-10-09T06:35:51.633353+0000","last_unstale":"2023-10-09T06:35:52.639319+0000","last_undegraded":"2023-10-09T06:35:52.639319+0000","last_fullsized":"2023-10-09T06:35:52.639319+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:57:42.823722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.665653+0000","last_change":"2023-10-09T06:35:52.648101+0000","last_active":"2023-10-09T06:35:52.665653+0000","last_peered":"2023-10-09T06:35:52.665653+0000","last_clean":"2023-10-09T06:35:52.665653+0000","last_became_active":"2023-10-09T06:35:52.647826+0000","last_became_peered":"2023-10-09T06:35:52.647826+0000","last_unstale":"2023-10-09T06:35:52.665653+0000","last_undegraded":"2023-10-09T06:35:52.665653+0000","last_fullsized":"2023-10-09T06:35:52.665653+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:43:22.600613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"76'3","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.664087+0000","last_change":"2023-10-09T06:35:51.636707+0000","last_active":"2023-10-09T06:35:52.664087+0000","last_peered":"2023-10-09T06:35:52.664087+0000","last_clean":"2023-10-09T06:35:52.664087+0000","last_became_active":"2023-10-09T06:35:51.636459+0000","last_became_peered":"2023-10-09T06:35:51.636459+0000","last_unstale":"2023-10-09T06:35:52.664087+0000","last_undegraded":"2023-10-09T06:35:52.664087+0000","last_fullsized":"2023-10-09T06:35:52.664087+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:44:17.924698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.667652+0000","last_change":"2023-10-09T06:35:52.666318+0000","last_active":"2023-10-09T06:35:52.667652+0000","last_peered":"2023-10-09T06:35:52.667652+0000","last_clean":"2023-10-09T06:35:52.667652+0000","last_became_active":"2023-10-09T06:35:52.666076+0000","last_became_peered":"2023-10-09T06:35:52.666076+0000","last_unstale":"2023-10-09T06:35:52.667652+0000","last_undegraded":"2023-10-09T06:35:52.667652+0000","last_fullsized":"2023-10-09T06:35:52.667652+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:05:23.290057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"76'6","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.667626+0000","last_change":"2023-10-09T06:35:51.633767+0000","last_active":"2023-10-09T06:35:52.667626+0000","last_peered":"2023-10-09T06:35:52.667626+0000","last_clean":"2023-10-09T06:35:52.667626+0000","last_became_active":"2023-10-09T06:35:51.633518+0000","last_became_peered":"2023-10-09T06:35:51.633518+0000","last_unstale":"2023-10-09T06:35:52.667626+0000","last_undegraded":"2023-10-09T06:35:52.667626+0000","last_fullsized":"2023-10-09T06:35:52.667626+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:02:15.246012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.667468+0000","last_change":"2023-10-09T06:35:52.657558+0000","last_active":"2023-10-09T06:35:52.667468+0000","last_peered":"2023-10-09T06:35:52.667468+0000","last_clean":"2023-10-09T06:35:52.667468+0000","last_became_active":"2023-10-09T06:35:52.657320+0000","last_became_peered":"2023-10-09T06:35:52.657320+0000","last_unstale":"2023-10-09T06:35:52.667468+0000","last_undegraded":"2023-10-09T06:35:52.667468+0000","last_fullsized":"2023-10-09T06:35:52.667468+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:59:03.103633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.668040+0000","last_change":"2023-10-09T06:35:51.639595+0000","last_active":"2023-10-09T06:35:52.668040+0000","last_peered":"2023-10-09T06:35:52.668040+0000","last_clean":"2023-10-09T06:35:52.668040+0000","last_became_active":"2023-10-09T06:35:51.639343+0000","last_became_peered":"2023-10-09T06:35:51.639343+0000","last_unstale":"2023-10-09T06:35:52.668040+0000","last_undegraded":"2023-10-09T06:35:52.668040+0000","last_fullsized":"2023-10-09T06:35:52.668040+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:30:15.468084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.659722+0000","last_change":"2023-10-09T06:35:52.656891+0000","last_active":"2023-10-09T06:35:52.659722+0000","last_peered":"2023-10-09T06:35:52.659722+0000","last_clean":"2023-10-09T06:35:52.659722+0000","last_became_active":"2023-10-09T06:35:52.656573+0000","last_became_peered":"2023-10-09T06:35:52.656573+0000","last_unstale":"2023-10-09T06:35:52.659722+0000","last_undegraded":"2023-10-09T06:35:52.659722+0000","last_fullsized":"2023-10-09T06:35:52.659722+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:18:29.136434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.650850+0000","last_change":"2023-10-09T06:35:51.635878+0000","last_active":"2023-10-09T06:35:52.650850+0000","last_peered":"2023-10-09T06:35:52.650850+0000","last_clean":"2023-10-09T06:35:52.650850+0000","last_became_active":"2023-10-09T06:35:51.635650+0000","last_became_peered":"2023-10-09T06:35:51.635650+0000","last_unstale":"2023-10-09T06:35:52.650850+0000","last_undegraded":"2023-10-09T06:35:52.650850+0000","last_fullsized":"2023-10-09T06:35:52.650850+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:39:28.522323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.663433+0000","last_change":"2023-10-09T06:35:52.658000+0000","last_active":"2023-10-09T06:35:52.663433+0000","last_peered":"2023-10-09T06:35:52.663433+0000","last_clean":"2023-10-09T06:35:52.663433+0000","last_became_active":"2023-10-09T06:35:52.657583+0000","last_became_peered":"2023-10-09T06:35:52.657583+0000","last_unstale":"2023-10-09T06:35:52.663433+0000","last_undegraded":"2023-10-09T06:35:52.663433+0000","last_fullsized":"2023-10-09T06:35:52.663433+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:05:30.191217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.645827+0000","last_change":"2023-10-09T06:35:51.634875+0000","last_active":"2023-10-09T06:35:52.645827+0000","last_peered":"2023-10-09T06:35:52.645827+0000","last_clean":"2023-10-09T06:35:52.645827+0000","last_became_active":"2023-10-09T06:35:51.634619+0000","last_became_peered":"2023-10-09T06:35:51.634619+0000","last_unstale":"2023-10-09T06:35:52.645827+0000","last_undegraded":"2023-10-09T06:35:52.645827+0000","last_fullsized":"2023-10-09T06:35:52.645827+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:12:07.625373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.660013+0000","last_change":"2023-10-09T06:35:52.654229+0000","last_active":"2023-10-09T06:35:52.660013+0000","last_peered":"2023-10-09T06:35:52.660013+0000","last_clean":"2023-10-09T06:35:52.660013+0000","last_became_active":"2023-10-09T06:35:52.653791+0000","last_became_peered":"2023-10-09T06:35:52.653791+0000","last_unstale":"2023-10-09T06:35:52.660013+0000","last_undegraded":"2023-10-09T06:35:52.660013+0000","last_fullsized":"2023-10-09T06:35:52.660013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:05:23.290057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"76'8","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.670492+0000","last_change":"2023-10-09T06:35:51.637899+0000","last_active":"2023-10-09T06:35:52.670492+0000","last_peered":"2023-10-09T06:35:52.670492+0000","last_clean":"2023-10-09T06:35:52.670492+0000","last_became_active":"2023-10-09T06:35:51.637646+0000","last_became_peered":"2023-10-09T06:35:51.637646+0000","last_unstale":"2023-10-09T06:35:52.670492+0000","last_undegraded":"2023-10-09T06:35:52.670492+0000","last_fullsized":"2023-10-09T06:35:52.670492+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:13:51.897687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.655810+0000","last_change":"2023-10-09T06:35:52.652375+0000","last_active":"2023-10-09T06:35:52.655810+0000","last_peered":"2023-10-09T06:35:52.655810+0000","last_clean":"2023-10-09T06:35:52.655810+0000","last_became_active":"2023-10-09T06:35:52.652056+0000","last_became_peered":"2023-10-09T06:35:52.652056+0000","last_unstale":"2023-10-09T06:35:52.655810+0000","last_undegraded":"2023-10-09T06:35:52.655810+0000","last_fullsized":"2023-10-09T06:35:52.655810+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:09:58.474455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.662490+0000","last_change":"2023-10-09T06:35:51.632482+0000","last_active":"2023-10-09T06:35:52.662490+0000","last_peered":"2023-10-09T06:35:52.662490+0000","last_clean":"2023-10-09T06:35:52.662490+0000","last_became_active":"2023-10-09T06:35:51.632098+0000","last_became_peered":"2023-10-09T06:35:51.632098+0000","last_unstale":"2023-10-09T06:35:52.662490+0000","last_undegraded":"2023-10-09T06:35:52.662490+0000","last_fullsized":"2023-10-09T06:35:52.662490+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:39:33.814116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.647441+0000","last_change":"2023-10-09T06:35:52.645484+0000","last_active":"2023-10-09T06:35:52.647441+0000","last_peered":"2023-10-09T06:35:52.647441+0000","last_clean":"2023-10-09T06:35:52.647441+0000","last_became_active":"2023-10-09T06:35:52.645210+0000","last_became_peered":"2023-10-09T06:35:52.645210+0000","last_unstale":"2023-10-09T06:35:52.647441+0000","last_undegraded":"2023-10-09T06:35:52.647441+0000","last_fullsized":"2023-10-09T06:35:52.647441+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:14:06.354004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.667723+0000","last_change":"2023-10-09T06:35:51.637794+0000","last_active":"2023-10-09T06:35:52.667723+0000","last_peered":"2023-10-09T06:35:52.667723+0000","last_clean":"2023-10-09T06:35:52.667723+0000","last_became_active":"2023-10-09T06:35:51.637509+0000","last_became_peered":"2023-10-09T06:35:51.637509+0000","last_unstale":"2023-10-09T06:35:52.667723+0000","last_undegraded":"2023-10-09T06:35:52.667723+0000","last_fullsized":"2023-10-09T06:35:52.667723+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:02:15.246012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.661279+0000","last_change":"2023-10-09T06:35:52.658640+0000","last_active":"2023-10-09T06:35:52.661279+0000","last_peered":"2023-10-09T06:35:52.661279+0000","last_clean":"2023-10-09T06:35:52.661279+0000","last_became_active":"2023-10-09T06:35:52.658323+0000","last_became_peered":"2023-10-09T06:35:52.658323+0000","last_unstale":"2023-10-09T06:35:52.661279+0000","last_undegraded":"2023-10-09T06:35:52.661279+0000","last_fullsized":"2023-10-09T06:35:52.661279+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:14:06.354004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.640191+0000","last_change":"2023-10-09T06:35:51.639419+0000","last_active":"2023-10-09T06:35:52.640191+0000","last_peered":"2023-10-09T06:35:52.640191+0000","last_clean":"2023-10-09T06:35:52.640191+0000","last_became_active":"2023-10-09T06:35:51.638955+0000","last_became_peered":"2023-10-09T06:35:51.638955+0000","last_unstale":"2023-10-09T06:35:52.640191+0000","last_undegraded":"2023-10-09T06:35:52.640191+0000","last_fullsized":"2023-10-09T06:35:52.640191+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:23:54.185514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.664286+0000","last_change":"2023-10-09T06:35:52.660013+0000","last_active":"2023-10-09T06:35:52.664286+0000","last_peered":"2023-10-09T06:35:52.664286+0000","last_clean":"2023-10-09T06:35:52.664286+0000","last_became_active":"2023-10-09T06:35:52.659275+0000","last_became_peered":"2023-10-09T06:35:52.659275+0000","last_unstale":"2023-10-09T06:35:52.664286+0000","last_undegraded":"2023-10-09T06:35:52.664286+0000","last_fullsized":"2023-10-09T06:35:52.664286+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:02:06.434139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.665059+0000","last_change":"2023-10-09T06:35:51.631719+0000","last_active":"2023-10-09T06:35:52.665059+0000","last_peered":"2023-10-09T06:35:52.665059+0000","last_clean":"2023-10-09T06:35:52.665059+0000","last_became_active":"2023-10-09T06:35:51.629672+0000","last_became_peered":"2023-10-09T06:35:51.629672+0000","last_unstale":"2023-10-09T06:35:52.665059+0000","last_undegraded":"2023-10-09T06:35:52.665059+0000","last_fullsized":"2023-10-09T06:35:52.665059+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:00:36.962657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.651846+0000","last_change":"2023-10-09T06:35:52.649501+0000","last_active":"2023-10-09T06:35:52.651846+0000","last_peered":"2023-10-09T06:35:52.651846+0000","last_clean":"2023-10-09T06:35:52.651846+0000","last_became_active":"2023-10-09T06:35:52.649011+0000","last_became_peered":"2023-10-09T06:35:52.649011+0000","last_unstale":"2023-10-09T06:35:52.651846+0000","last_undegraded":"2023-10-09T06:35:52.651846+0000","last_fullsized":"2023-10-09T06:35:52.651846+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:43:22.600613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.646933+0000","last_change":"2023-10-09T06:35:51.632607+0000","last_active":"2023-10-09T06:35:52.646933+0000","last_peered":"2023-10-09T06:35:52.646933+0000","last_clean":"2023-10-09T06:35:52.646933+0000","last_became_active":"2023-10-09T06:35:51.632358+0000","last_became_peered":"2023-10-09T06:35:51.632358+0000","last_unstale":"2023-10-09T06:35:52.646933+0000","last_undegraded":"2023-10-09T06:35:52.646933+0000","last_fullsized":"2023-10-09T06:35:52.646933+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:55:21.838020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.658378+0000","last_change":"2023-10-09T06:35:52.653004+0000","last_active":"2023-10-09T06:35:52.658378+0000","last_peered":"2023-10-09T06:35:52.658378+0000","last_clean":"2023-10-09T06:35:52.658378+0000","last_became_active":"2023-10-09T06:35:52.652792+0000","last_became_peered":"2023-10-09T06:35:52.652792+0000","last_unstale":"2023-10-09T06:35:52.658378+0000","last_undegraded":"2023-10-09T06:35:52.658378+0000","last_fullsized":"2023-10-09T06:35:52.658378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:03:03.287437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.647486+0000","last_change":"2023-10-09T06:35:51.629251+0000","last_active":"2023-10-09T06:35:52.647486+0000","last_peered":"2023-10-09T06:35:52.647486+0000","last_clean":"2023-10-09T06:35:52.647486+0000","last_became_active":"2023-10-09T06:35:51.628991+0000","last_became_peered":"2023-10-09T06:35:51.628991+0000","last_unstale":"2023-10-09T06:35:52.647486+0000","last_undegraded":"2023-10-09T06:35:52.647486+0000","last_fullsized":"2023-10-09T06:35:52.647486+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:54:04.391468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.659130+0000","last_change":"2023-10-09T06:35:52.652700+0000","last_active":"2023-10-09T06:35:52.659130+0000","last_peered":"2023-10-09T06:35:52.659130+0000","last_clean":"2023-10-09T06:35:52.659130+0000","last_became_active":"2023-10-09T06:35:52.652425+0000","last_became_peered":"2023-10-09T06:35:52.652425+0000","last_unstale":"2023-10-09T06:35:52.659130+0000","last_undegraded":"2023-10-09T06:35:52.659130+0000","last_fullsized":"2023-10-09T06:35:52.659130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:05:23.290057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.648569+0000","last_change":"2023-10-09T06:35:51.628116+0000","last_active":"2023-10-09T06:35:52.648569+0000","last_peered":"2023-10-09T06:35:52.648569+0000","last_clean":"2023-10-09T06:35:52.648569+0000","last_became_active":"2023-10-09T06:35:51.627896+0000","last_became_peered":"2023-10-09T06:35:51.627896+0000","last_unstale":"2023-10-09T06:35:52.648569+0000","last_undegraded":"2023-10-09T06:35:52.648569+0000","last_fullsized":"2023-10-09T06:35:52.648569+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:16:35.599572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.658200+0000","last_change":"2023-10-09T06:35:52.653706+0000","last_active":"2023-10-09T06:35:52.658200+0000","last_peered":"2023-10-09T06:35:52.658200+0000","last_clean":"2023-10-09T06:35:52.658200+0000","last_became_active":"2023-10-09T06:35:52.653462+0000","last_became_peered":"2023-10-09T06:35:52.653462+0000","last_unstale":"2023-10-09T06:35:52.658200+0000","last_undegraded":"2023-10-09T06:35:52.658200+0000","last_fullsized":"2023-10-09T06:35:52.658200+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:04:11.040070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.663273+0000","last_change":"2023-10-09T06:35:51.635409+0000","last_active":"2023-10-09T06:35:52.663273+0000","last_peered":"2023-10-09T06:35:52.663273+0000","last_clean":"2023-10-09T06:35:52.663273+0000","last_became_active":"2023-10-09T06:35:51.635161+0000","last_became_peered":"2023-10-09T06:35:51.635161+0000","last_unstale":"2023-10-09T06:35:52.663273+0000","last_undegraded":"2023-10-09T06:35:52.663273+0000","last_fullsized":"2023-10-09T06:35:52.663273+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:26:30.104488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.650752+0000","last_change":"2023-10-09T06:35:52.648325+0000","last_active":"2023-10-09T06:35:52.650752+0000","last_peered":"2023-10-09T06:35:52.650752+0000","last_clean":"2023-10-09T06:35:52.650752+0000","last_became_active":"2023-10-09T06:35:52.648032+0000","last_became_peered":"2023-10-09T06:35:52.648032+0000","last_unstale":"2023-10-09T06:35:52.650752+0000","last_undegraded":"2023-10-09T06:35:52.650752+0000","last_fullsized":"2023-10-09T06:35:52.650752+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:31:21.904859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.650387+0000","last_change":"2023-10-09T06:35:51.638562+0000","last_active":"2023-10-09T06:35:52.650387+0000","last_peered":"2023-10-09T06:35:52.650387+0000","last_clean":"2023-10-09T06:35:52.650387+0000","last_became_active":"2023-10-09T06:35:51.638285+0000","last_became_peered":"2023-10-09T06:35:51.638285+0000","last_unstale":"2023-10-09T06:35:52.650387+0000","last_undegraded":"2023-10-09T06:35:52.650387+0000","last_fullsized":"2023-10-09T06:35:52.650387+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:47:56.832137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.661375+0000","last_change":"2023-10-09T06:35:52.655279+0000","last_active":"2023-10-09T06:35:52.661375+0000","last_peered":"2023-10-09T06:35:52.661375+0000","last_clean":"2023-10-09T06:35:52.661375+0000","last_became_active":"2023-10-09T06:35:52.654501+0000","last_became_peered":"2023-10-09T06:35:52.654501+0000","last_unstale":"2023-10-09T06:35:52.661375+0000","last_undegraded":"2023-10-09T06:35:52.661375+0000","last_fullsized":"2023-10-09T06:35:52.661375+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:55:18.457568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.639445+0000","last_change":"2023-10-09T06:35:51.630872+0000","last_active":"2023-10-09T06:35:52.639445+0000","last_peered":"2023-10-09T06:35:52.639445+0000","last_clean":"2023-10-09T06:35:52.639445+0000","last_became_active":"2023-10-09T06:35:51.630150+0000","last_became_peered":"2023-10-09T06:35:51.630150+0000","last_unstale":"2023-10-09T06:35:52.639445+0000","last_undegraded":"2023-10-09T06:35:52.639445+0000","last_fullsized":"2023-10-09T06:35:52.639445+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:30:26.813152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.659288+0000","last_change":"2023-10-09T06:35:52.653099+0000","last_active":"2023-10-09T06:35:52.659288+0000","last_peered":"2023-10-09T06:35:52.659288+0000","last_clean":"2023-10-09T06:35:52.659288+0000","last_became_active":"2023-10-09T06:35:52.652892+0000","last_became_peered":"2023-10-09T06:35:52.652892+0000","last_unstale":"2023-10-09T06:35:52.659288+0000","last_undegraded":"2023-10-09T06:35:52.659288+0000","last_fullsized":"2023-10-09T06:35:52.659288+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:43:22.600613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.639300+0000","last_change":"2023-10-09T06:35:51.634017+0000","last_active":"2023-10-09T06:35:52.639300+0000","last_peered":"2023-10-09T06:35:52.639300+0000","last_clean":"2023-10-09T06:35:52.639300+0000","last_became_active":"2023-10-09T06:35:51.633775+0000","last_became_peered":"2023-10-09T06:35:51.633775+0000","last_unstale":"2023-10-09T06:35:52.639300+0000","last_undegraded":"2023-10-09T06:35:52.639300+0000","last_fullsized":"2023-10-09T06:35:52.639300+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:11:49.746928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.652066+0000","last_change":"2023-10-09T06:35:52.649658+0000","last_active":"2023-10-09T06:35:52.652066+0000","last_peered":"2023-10-09T06:35:52.652066+0000","last_clean":"2023-10-09T06:35:52.652066+0000","last_became_active":"2023-10-09T06:35:52.649204+0000","last_became_peered":"2023-10-09T06:35:52.649204+0000","last_unstale":"2023-10-09T06:35:52.652066+0000","last_undegraded":"2023-10-09T06:35:52.652066+0000","last_fullsized":"2023-10-09T06:35:52.652066+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:04:11.040070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.639749+0000","last_change":"2023-10-09T06:35:51.638604+0000","last_active":"2023-10-09T06:35:52.639749+0000","last_peered":"2023-10-09T06:35:52.639749+0000","last_clean":"2023-10-09T06:35:52.639749+0000","last_became_active":"2023-10-09T06:35:51.638391+0000","last_became_peered":"2023-10-09T06:35:51.638391+0000","last_unstale":"2023-10-09T06:35:52.639749+0000","last_undegraded":"2023-10-09T06:35:52.639749+0000","last_fullsized":"2023-10-09T06:35:52.639749+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:14:05.346793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.652130+0000","last_change":"2023-10-09T06:35:52.645140+0000","last_active":"2023-10-09T06:35:52.652130+0000","last_peered":"2023-10-09T06:35:52.652130+0000","last_clean":"2023-10-09T06:35:52.652130+0000","last_became_active":"2023-10-09T06:35:52.644876+0000","last_became_peered":"2023-10-09T06:35:52.644876+0000","last_unstale":"2023-10-09T06:35:52.652130+0000","last_undegraded":"2023-10-09T06:35:52.652130+0000","last_fullsized":"2023-10-09T06:35:52.652130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:05:23.290057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.649542+0000","last_change":"2023-10-09T06:35:51.634273+0000","last_active":"2023-10-09T06:35:52.649542+0000","last_peered":"2023-10-09T06:35:52.649542+0000","last_clean":"2023-10-09T06:35:52.649542+0000","last_became_active":"2023-10-09T06:35:51.634000+0000","last_became_peered":"2023-10-09T06:35:51.634000+0000","last_unstale":"2023-10-09T06:35:52.649542+0000","last_undegraded":"2023-10-09T06:35:52.649542+0000","last_fullsized":"2023-10-09T06:35:52.649542+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:04:26.655638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.660049+0000","last_change":"2023-10-09T06:35:52.659092+0000","last_active":"2023-10-09T06:35:52.660049+0000","last_peered":"2023-10-09T06:35:52.660049+0000","last_clean":"2023-10-09T06:35:52.660049+0000","last_became_active":"2023-10-09T06:35:52.658805+0000","last_became_peered":"2023-10-09T06:35:52.658805+0000","last_unstale":"2023-10-09T06:35:52.660049+0000","last_undegraded":"2023-10-09T06:35:52.660049+0000","last_fullsized":"2023-10-09T06:35:52.660049+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:00:37.969869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.638980+0000","last_change":"2023-10-09T06:35:51.633529+0000","last_active":"2023-10-09T06:35:52.638980+0000","last_peered":"2023-10-09T06:35:52.638980+0000","last_clean":"2023-10-09T06:35:52.638980+0000","last_became_active":"2023-10-09T06:35:51.633269+0000","last_became_peered":"2023-10-09T06:35:51.633269+0000","last_unstale":"2023-10-09T06:35:52.638980+0000","last_undegraded":"2023-10-09T06:35:52.638980+0000","last_fullsized":"2023-10-09T06:35:52.638980+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:41:00.450594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.662379+0000","last_change":"2023-10-09T06:35:52.656014+0000","last_active":"2023-10-09T06:35:52.662379+0000","last_peered":"2023-10-09T06:35:52.662379+0000","last_clean":"2023-10-09T06:35:52.662379+0000","last_became_active":"2023-10-09T06:35:52.655749+0000","last_became_peered":"2023-10-09T06:35:52.655749+0000","last_unstale":"2023-10-09T06:35:52.662379+0000","last_undegraded":"2023-10-09T06:35:52.662379+0000","last_fullsized":"2023-10-09T06:35:52.662379+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:38:30.691298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.649684+0000","last_change":"2023-10-09T06:35:51.636358+0000","last_active":"2023-10-09T06:35:52.649684+0000","last_peered":"2023-10-09T06:35:52.649684+0000","last_clean":"2023-10-09T06:35:52.649684+0000","last_became_active":"2023-10-09T06:35:51.636091+0000","last_became_peered":"2023-10-09T06:35:51.636091+0000","last_unstale":"2023-10-09T06:35:52.649684+0000","last_undegraded":"2023-10-09T06:35:52.649684+0000","last_fullsized":"2023-10-09T06:35:52.649684+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:11:49.746928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.659616+0000","last_change":"2023-10-09T06:35:52.654058+0000","last_active":"2023-10-09T06:35:52.659616+0000","last_peered":"2023-10-09T06:35:52.659616+0000","last_clean":"2023-10-09T06:35:52.659616+0000","last_became_active":"2023-10-09T06:35:52.653766+0000","last_became_peered":"2023-10-09T06:35:52.653766+0000","last_unstale":"2023-10-09T06:35:52.659616+0000","last_undegraded":"2023-10-09T06:35:52.659616+0000","last_fullsized":"2023-10-09T06:35:52.659616+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:51.616286+0000","last_clean_scrub_stamp":"2023-10-09T06:35:51.616286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:35:42.110847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.663353+0000","last_change":"2023-10-09T06:35:51.637316+0000","last_active":"2023-10-09T06:35:52.663353+0000","last_peered":"2023-10-09T06:35:52.663353+0000","last_clean":"2023-10-09T06:35:52.663353+0000","last_became_active":"2023-10-09T06:35:51.637048+0000","last_became_peered":"2023-10-09T06:35:51.637048+0000","last_unstale":"2023-10-09T06:35:52.663353+0000","last_undegraded":"2023-10-09T06:35:52.663353+0000","last_fullsized":"2023-10-09T06:35:52.663353+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:35:50.609074+0000","last_clean_scrub_stamp":"2023-10-09T06:35:50.609074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:35:41.103636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":211,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-09T06:35:52.642958+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:35:52.642958+0000","last_peered":"2023-10-09T06:35:52.642958+0000","last_clean":"2023-10-09T06:35:52.642958+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:35:52.642958+0000","last_undegraded":"2023-10-09T06:35:52.642958+0000","last_fullsized":"2023-10-09T06:35:52.642958+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574917,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298796,"kb_used_data":1508,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073044,"statfs":{"total":96636764160,"available":96330797056,"internally_reserved":0,"allocated":1544192,"data_stored":998236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:33 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.50700000000000001,"15min":0.503},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.66300000000000003,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.34100000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65700000000000003,"5min":0.753,"15min":0.753},"last":0.442}]},{"osd":1,"last update":"Mon Oct 9 06:35:33 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.50900000000000001,"15min":0.52100000000000002},"min":{"1min":0.36099999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.63700000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52300000000000002,"15min":0.52400000000000002},"min":{"1min":0.32900000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.66100000000000003,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.58399999999999996}]},{"osd":2,"last update":"Mon Oct 9 06:35:33 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.52100000000000002,"15min":0.50900000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65700000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52900000000000003,"15min":0.54500000000000004},"min":{"1min":0.36799999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.69099999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.50800000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:35:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.53500000000000003,"15min":0.55300000000000005},"min":{"1min":0.36099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56399999999999995,"15min":0.59799999999999998},"min":{"1min":0.42299999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82099999999999995,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.46600000000000003}]},{"osd":4,"last update":"Mon Oct 9 06:35:33 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.48199999999999998,"15min":0.52400000000000002},"min":{"1min":0.307,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.69999999999999996,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56899999999999995,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60099999999999998}]},{"osd":5,"last update":"Mon Oct 9 06:35:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52700000000000002,"15min":0.57699999999999996},"min":{"1min":0.34399999999999997,"5min":0.307,"15min":0.307},"max":{"1min":0.85799999999999998,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58699999999999997,"15min":0.621},"min":{"1min":0.41699999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.74099999999999999,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.64100000000000001}]},{"osd":6,"last update":"Mon Oct 9 06:35:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55100000000000005,"15min":0.59799999999999998},"min":{"1min":0.378,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.78100000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.62},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55300000000000005,"15min":0.61799999999999999},"min":{"1min":0.35399999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.79900000000000004,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.628}]}]},{"osd":6,"up_from":13,"seq":55834574916,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073616,"statfs":{"total":96636764160,"available":96331382784,"internally_reserved":0,"allocated":958464,"data_stored":408464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.495,"15min":0.501},"min":{"1min":0.35399999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.32200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.49099999999999999}]},{"osd":1,"last update":"Mon Oct 9 06:35:36 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.495,"15min":0.51300000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.72499999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.502},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52600000000000002,"15min":0.53200000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.36199999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:35:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51000000000000001,"15min":0.51500000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69499999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.51300000000000001,"15min":0.53300000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.67500000000000004,"5min":0.752,"15min":0.752},"last":0.54700000000000004}]},{"osd":3,"last update":"Mon Oct 9 06:35:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53500000000000003,"15min":0.54800000000000004},"min":{"1min":0.40000000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70899999999999996,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59399999999999997,"15min":0.71799999999999997},"min":{"1min":0.372,"5min":0.37,"15min":0.37},"max":{"1min":0.73499999999999999,"5min":4.7510000000000003,"15min":4.7510000000000003},"last":0.39900000000000002}]},{"osd":4,"last update":"Mon Oct 9 06:35:36 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46800000000000003,"15min":0.47999999999999998},"min":{"1min":0.29799999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.67500000000000004,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.316},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.57199999999999995,"15min":0.58499999999999996},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.76700000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.53800000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:35:36 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.52700000000000002,"15min":0.53300000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.755,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57799999999999996,"15min":0.59999999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78000000000000003,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.57699999999999996}]},{"osd":7,"last update":"Mon Oct 9 06:35:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56499999999999995,"15min":0.59099999999999997},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.434},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.52200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302320,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069520,"statfs":{"total":96636764160,"available":96327188480,"internally_reserved":0,"allocated":958464,"data_stored":407992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.30399999999999999,"15min":0.308},"min":{"1min":0.20200000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34899999999999998,"15min":0.35999999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.53600000000000003,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.38200000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34300000000000003,"15min":0.35099999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.38400000000000001,"15min":0.38600000000000001},"min":{"1min":0.29699999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.57499999999999996,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.441}]},{"osd":3,"last update":"Mon Oct 9 06:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.36099999999999999,"15min":0.36299999999999999},"min":{"1min":0.247,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.41299999999999998,"15min":0.57899999999999996},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.54800000000000004,"5min":5.8529999999999998,"15min":5.8529999999999998},"last":0.34499999999999997}]},{"osd":4,"last update":"Mon Oct 9 06:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.42299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58199999999999996,"15min":0.58599999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.63}]},{"osd":5,"last update":"Mon Oct 9 06:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.62,"15min":0.77800000000000002},"min":{"1min":0.37,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.747,"5min":5.9459999999999997,"15min":5.9459999999999997},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60199999999999998,"15min":0.61199999999999999},"min":{"1min":0.47499999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.92700000000000005,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.64300000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.59599999999999997,"15min":0.748},"min":{"1min":0.39100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73799999999999999,"5min":5.9160000000000004,"15min":5.9160000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.42699999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91000000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.52100000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.89500000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.624,"15min":0.79000000000000004},"min":{"1min":0.40699999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80900000000000005,"5min":5.9729999999999999,"15min":5.9729999999999999},"last":0.61299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574918,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302892,"kb_used_data":1508,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068948,"statfs":{"total":96636764160,"available":96326602752,"internally_reserved":0,"allocated":1544192,"data_stored":998200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.308,"15min":0.317},"min":{"1min":0.214,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.47599999999999998,"5min":0.498,"15min":0.498},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.33700000000000002,"15min":0.34699999999999998},"min":{"1min":0.23100000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.53600000000000003,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.32400000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.35299999999999998,"15min":0.36299999999999999},"min":{"1min":0.252,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.31},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38100000000000001,"15min":0.38900000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.56499999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.375}]},{"osd":3,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35599999999999998,"15min":0.39700000000000002},"min":{"1min":0.27300000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51200000000000001,"5min":1.554,"15min":1.554},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.37,"5min":0.378,"15min":0.40400000000000003},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.58099999999999996,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.35699999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.36599999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70199999999999996,"5min":1.212,"15min":1.212},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61199999999999999,"15min":0.75800000000000001},"min":{"1min":0.41799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82499999999999996,"5min":5.484,"15min":5.484},"last":0.52900000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.61499999999999999,"15min":0.76300000000000001},"min":{"1min":0.37,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.69999999999999996,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57299999999999995,"15min":0.57599999999999996},"min":{"1min":0.40899999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84499999999999997,"5min":1.113,"15min":1.113},"last":0.51400000000000001}]},{"osd":6,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59499999999999997,"15min":0.60699999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.72899999999999998,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.42799999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83899999999999997,"5min":1.169,"15min":1.169},"last":0.629}]},{"osd":7,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.39700000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.85499999999999998,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.628,"15min":0.79000000000000004},"min":{"1min":0.42699999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.71299999999999997,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.58599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574919,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298216,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073624,"statfs":{"total":96636764160,"available":96331390976,"internally_reserved":0,"allocated":950272,"data_stored":408352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.32200000000000001,"15min":0.35999999999999999},"min":{"1min":0.218,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.50800000000000001,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.40100000000000002,"15min":0.60999999999999999},"min":{"1min":0.216,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.53800000000000003,"5min":7.2699999999999996,"15min":7.2699999999999996},"last":0.45400000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.32800000000000001,"15min":0.34899999999999998},"min":{"1min":0.22600000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.497,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.45100000000000001,"15min":0.66600000000000004},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.57599999999999996,"5min":7.327,"15min":7.327},"last":0.51500000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.376,"15min":0.39400000000000002},"min":{"1min":0.28299999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.52900000000000003,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.374,"15min":0.39500000000000002},"min":{"1min":0.25900000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.48399999999999999,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.497}]},{"osd":4,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53700000000000003,"15min":0.54100000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.73299999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.628,"15min":0.83299999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83199999999999996,"5min":7.2389999999999999,"15min":7.2389999999999999},"last":0.66600000000000004}]},{"osd":5,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.69899999999999995,"15min":0.98999999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.747,"5min":10.890000000000001,"15min":10.890000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59099999999999997,"15min":0.60199999999999998},"min":{"1min":0.39800000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.78200000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.78200000000000003}]},{"osd":6,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.64300000000000002,"15min":0.84099999999999997},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77300000000000002,"5min":7.2999999999999998,"15min":7.2999999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57199999999999995,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77000000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.73599999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.67200000000000004,"15min":0.873},"min":{"1min":0.38600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.79900000000000004,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.61599999999999999,"15min":0.628},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79300000000000004,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.77200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574918,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073616,"statfs":{"total":96636764160,"available":96331382784,"internally_reserved":0,"allocated":958464,"data_stored":409334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.30199999999999999,"5min":0.35699999999999998,"15min":0.32700000000000001},"min":{"1min":0.23599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.38300000000000001,"5min":5.8700000000000001,"15min":5.8700000000000001},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.38600000000000001,"15min":0.36599999999999999},"min":{"1min":0.224,"5min":0.222,"15min":0.222},"max":{"1min":0.46200000000000002,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.40000000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.375,"15min":0.35699999999999998},"min":{"1min":0.23899999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.47699999999999998,"5min":4.0469999999999997,"15min":4.0469999999999997},"last":0.375},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40000000000000002,"15min":0.38700000000000001},"min":{"1min":0.27100000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.48699999999999999,"5min":4.1509999999999998,"15min":4.1509999999999998},"last":0.45100000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.373,"15min":0.374},"min":{"1min":0.24399999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.499,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.439},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.38900000000000001,"15min":0.38500000000000001},"min":{"1min":0.26200000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.51500000000000001,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.48399999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58099999999999996,"15min":0.57099999999999995},"min":{"1min":0.372,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80100000000000005,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57299999999999995,"15min":0.56200000000000006},"min":{"1min":0.36599999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68899999999999995,"5min":3.0710000000000002,"15min":3.0710000000000002},"last":0.70399999999999996}]},{"osd":5,"last update":"Mon Oct 9 06:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60999999999999999,"15min":0.58599999999999997},"min":{"1min":0.38200000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.746,"5min":6.2300000000000004,"15min":6.2300000000000004},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.60199999999999998,"15min":0.59199999999999997},"min":{"1min":0.41499999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72299999999999998,"5min":3.0219999999999998,"15min":3.0219999999999998},"last":0.76800000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.60599999999999998,"15min":0.58899999999999997},"min":{"1min":0.42899999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.71199999999999997,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.67900000000000005,"15min":0.64800000000000002},"min":{"1min":0.46400000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.79300000000000004,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.78500000000000003}]},{"osd":7,"last update":"Mon Oct 9 06:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63100000000000001,"15min":0.60999999999999999},"min":{"1min":0.496,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77900000000000003,"5min":4.2629999999999999,"15min":4.2629999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.64700000000000002,"15min":0.61099999999999999},"min":{"1min":0.40500000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.73599999999999999,"5min":7.3650000000000002,"15min":7.3650000000000002},"last":0.72299999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574917,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298212,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073628,"statfs":{"total":96636764160,"available":96331395072,"internally_reserved":0,"allocated":946176,"data_stored":407798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.53200000000000003,"15min":0.56000000000000005},"min":{"1min":0.34499999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.63900000000000001,"5min":1.083,"15min":1.083},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.33300000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.97799999999999998,"5min":1.054,"15min":1.054},"last":0.48999999999999999}]},{"osd":1,"last update":"Mon Oct 9 06:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.65700000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51100000000000001,"15min":0.495},"min":{"1min":0.42099999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.62,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.42099999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52600000000000002,"15min":0.54200000000000004},"min":{"1min":0.35299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71499999999999997,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.438},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.57099999999999995,"15min":0.59899999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.65100000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.55800000000000005}]},{"osd":3,"last update":"Mon Oct 9 06:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54000000000000004,"15min":0.55700000000000005},"min":{"1min":0.39200000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.66700000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.501},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57599999999999996,"15min":0.59099999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.78500000000000003,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.51000000000000001}]},{"osd":5,"last update":"Mon Oct 9 06:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52300000000000002,"15min":0.56100000000000005},"min":{"1min":0.36199999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.76500000000000001,"5min":0.998,"15min":0.998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58799999999999997,"15min":0.61399999999999999},"min":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.751,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.46700000000000003}]},{"osd":6,"last update":"Mon Oct 9 06:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.56999999999999995},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57399999999999995,"15min":0.60199999999999998},"min":{"1min":0.41499999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.45300000000000001}]},{"osd":7,"last update":"Mon Oct 9 06:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54900000000000004,"15min":0.57499999999999996},"min":{"1min":0.33600000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.83199999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.371},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60899999999999999,"15min":0.65100000000000002},"min":{"1min":0.40500000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98999999999999999,"5min":1.175,"15min":1.175},"last":0.34999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574917,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298228,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073612,"statfs":{"total":96636764160,"available":96331378688,"internally_reserved":0,"allocated":962560,"data_stored":409356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.498,"15min":0.50800000000000001},"min":{"1min":0.30599999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.60499999999999998,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.501},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.51300000000000001,"15min":0.51200000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.621,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.45700000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.50900000000000001,"15min":0.51000000000000001},"min":{"1min":0.33000000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.67500000000000004,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.376},{"interface":"front","average":{"1min":0.496,"5min":0.52100000000000002,"15min":0.53100000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64700000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.46999999999999997}]},{"osd":2,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54600000000000004,"15min":0.54400000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.69399999999999995,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58099999999999996,"15min":0.58799999999999997},"min":{"1min":0.40999999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.71699999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.48099999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53900000000000003,"15min":0.54200000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.66100000000000003,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55600000000000005,"15min":0.56399999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.69999999999999996,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.437}]},{"osd":4,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.49399999999999999,"15min":0.496},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68300000000000005,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.71999999999999997,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.35299999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.36099999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72099999999999997,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.55500000000000005,"15min":0.55800000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73799999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.42599999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78100000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.35199999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.84499999999999997,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.54800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:35:57.081 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-09T06:35:57.081 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-09T06:35:57.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.237+0000 7f64a83ed700 1 Processor -- start 2023-10-09T06:35:57.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.237+0000 7f64a83ed700 1 -- start start 2023-10-09T06:35:57.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.238+0000 7f64a83ed700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64a0069c70 0x7f64a00cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:57.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.238+0000 7f64a83ed700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f64a00af320 0x7f64a00cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:57.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.238+0000 7f64a83ed700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64a00ce380 con 0x7f64a0159950 2023-10-09T06:35:57.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.238+0000 7f64a83ed700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64a00cec20 con 0x7f64a00af320 2023-10-09T06:35:57.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.238+0000 7f64a83ed700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a00afa90 con 0x7f64a0069c70 2023-10-09T06:35:57.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.238+0000 7f64a5988700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64a0069c70 0x7f64a00cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:57.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.238+0000 7f64a698a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f64a00af320 0x7f64a00cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:57.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.238+0000 7f64a5988700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64a0069c70 0x7f64a00cbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:45836/0 (socket says 172.21.15.163:45836) 2023-10-09T06:35:57.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.238+0000 7f64a698a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f64a00af320 0x7f64a00cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:54010/0 (socket says 172.21.15.163:54010) 2023-10-09T06:35:57.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.238+0000 7f64a5988700 1 -- 172.21.15.163:0/3112811631 learned_addr learned my addr 172.21.15.163:0/3112811631 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:57.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.239+0000 7f64a698a700 1 -- 172.21.15.163:0/3112811631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64a0069c70 msgr2=0x7f64a00cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:57.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.239+0000 7f64a698a700 1 --2- 172.21.15.163:0/3112811631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64a0069c70 0x7f64a00cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:57.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.239+0000 7f64a698a700 1 -- 172.21.15.163:0/3112811631 >> v1:172.21.15.181:6789/0 conn(0x7f64a0159950 legacy=0x7f64a0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:57.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.239+0000 7f64a698a700 1 -- 172.21.15.163:0/3112811631 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a00ce5d0 con 0x7f64a00af320 2023-10-09T06:35:57.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.239+0000 7f64a5187700 1 -- 172.21.15.163:0/3112811631 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3801064253 0 0) 0x7f64a00ce380 con 0x7f64a0159950 2023-10-09T06:35:57.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.239+0000 7f64a5988700 1 --2- 172.21.15.163:0/3112811631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64a0069c70 0x7f64a00cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:35:57.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.239+0000 7f64a698a700 1 --2- 172.21.15.163:0/3112811631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64a00af320 0x7f64a00cf0b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6498009e60 tx=0x7f6498009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cde0abc3fada5653 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:57.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.239+0000 7f64a5187700 1 -- 172.21.15.163:0/3112811631 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649800e070 con 0x7f64a00af320 2023-10-09T06:35:57.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.239+0000 7f64a5187700 1 -- 172.21.15.163:0/3112811631 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649800bb60 con 0x7f64a00af320 2023-10-09T06:35:57.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.239+0000 7f64a5187700 1 -- 172.21.15.163:0/3112811631 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6498012430 con 0x7f64a00af320 2023-10-09T06:35:57.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.240+0000 7f64a83ed700 1 -- 172.21.15.163:0/3112811631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64a00af320 msgr2=0x7f64a00cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:57.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.240+0000 7f64a83ed700 1 --2- 172.21.15.163:0/3112811631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64a00af320 0x7f64a00cf0b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6498009e60 tx=0x7f6498009780 comp rx=0 tx=0).stop 2023-10-09T06:35:57.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.240+0000 7f64a83ed700 1 -- 172.21.15.163:0/3112811631 shutdown_connections 2023-10-09T06:35:57.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.240+0000 7f64a83ed700 1 --2- 172.21.15.163:0/3112811631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64a00af320 0x7f64a00cf0b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:57.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.240+0000 7f64a83ed700 1 --2- 172.21.15.163:0/3112811631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64a0069c70 0x7f64a00cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:57.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.240+0000 7f64a83ed700 1 -- 172.21.15.163:0/3112811631 >> 172.21.15.163:0/3112811631 conn(0x7f64a0068e00 msgr2=0x7f64a00d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:57.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.240+0000 7f64a83ed700 1 -- 172.21.15.163:0/3112811631 shutdown_connections 2023-10-09T06:35:57.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.240+0000 7f64a83ed700 1 -- 172.21.15.163:0/3112811631 wait complete. 2023-10-09T06:35:57.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.241+0000 7f64a83ed700 1 Processor -- start 2023-10-09T06:35:57.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.241+0000 7f64a83ed700 1 -- start start 2023-10-09T06:35:57.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.241+0000 7f64a83ed700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64a0069c70 0x7f64a00b2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:57.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.242+0000 7f64a83ed700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64a00af320 0x7f64a00b2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:57.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.242+0000 7f64a83ed700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64a0159950 0x7f64a015b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:57.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.242+0000 7f64a83ed700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a015e000 con 0x7f64a0159950 2023-10-09T06:35:57.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.242+0000 7f64a83ed700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a015d6d0 con 0x7f64a0069c70 2023-10-09T06:35:57.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.242+0000 7f64a83ed700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a015d9b0 con 0x7f64a00af320 2023-10-09T06:35:57.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.242+0000 7f64a698a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64a0159950 0x7f64a015b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:57.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.242+0000 7f64a698a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64a0159950 0x7f64a015b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56206/0 (socket says 172.21.15.163:56206) 2023-10-09T06:35:57.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.242+0000 7f64a698a700 1 -- 172.21.15.163:0/3856295175 learned_addr learned my addr 172.21.15.163:0/3856295175 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:57.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.242+0000 7f64a5988700 1 --2- 172.21.15.163:0/3856295175 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64a00af320 0x7f64a00b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:57.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.242+0000 7f64a6189700 1 --2- 172.21.15.163:0/3856295175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64a0069c70 0x7f64a00b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:57.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.242+0000 7f64a698a700 1 -- 172.21.15.163:0/3856295175 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64a00af320 msgr2=0x7f64a00b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:57.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.242+0000 7f64a698a700 1 --2- 172.21.15.163:0/3856295175 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64a00af320 0x7f64a00b2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:57.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.242+0000 7f64a698a700 1 -- 172.21.15.163:0/3856295175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64a0069c70 msgr2=0x7f64a00b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:57.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.242+0000 7f64a698a700 1 --2- 172.21.15.163:0/3856295175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64a0069c70 0x7f64a00b2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:57.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.242+0000 7f64a698a700 1 -- 172.21.15.163:0/3856295175 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64900094f0 con 0x7f64a0159950 2023-10-09T06:35:57.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.242+0000 7f64a698a700 1 --2- 172.21.15.163:0/3856295175 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64a0159950 0x7f64a015b320 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f64a0070e20 tx=0x7f6498019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:57.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.242+0000 7f64967fc700 1 -- 172.21.15.163:0/3856295175 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6498027470 con 0x7f64a0159950 2023-10-09T06:35:57.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.242+0000 7f64967fc700 1 -- 172.21.15.163:0/3856295175 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64980275f0 con 0x7f64a0159950 2023-10-09T06:35:57.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.242+0000 7f64a83ed700 1 -- 172.21.15.163:0/3856295175 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6498009a20 con 0x7f64a0159950 2023-10-09T06:35:57.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.242+0000 7f64967fc700 1 -- 172.21.15.163:0/3856295175 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64980279b0 con 0x7f64a0159950 2023-10-09T06:35:57.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.242+0000 7f64a83ed700 1 -- 172.21.15.163:0/3856295175 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a015d380 con 0x7f64a0159950 2023-10-09T06:35:57.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.243+0000 7f64a83ed700 1 -- 172.21.15.163:0/3856295175 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64a015e1b0 con 0x7f64a0159950 2023-10-09T06:35:57.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.244+0000 7f64967fc700 1 -- 172.21.15.163:0/3856295175 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f6498024070 con 0x7f64a0159950 2023-10-09T06:35:57.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.244+0000 7f64967fc700 1 --2- 172.21.15.163:0/3856295175 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6484067170 0x7f6484069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:57.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.244+0000 7f64967fc700 1 -- 172.21.15.163:0/3856295175 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6498086fc0 con 0x7f64a0159950 2023-10-09T06:35:57.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.244+0000 7f64a6189700 1 --2- 172.21.15.163:0/3856295175 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6484067170 0x7f6484069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:57.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.245+0000 7f64a6189700 1 --2- 172.21.15.163:0/3856295175 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6484067170 0x7f6484069620 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f649c0011e0 tx=0x7f649c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:57.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.249+0000 7f64967fc700 1 -- 172.21.15.163:0/3856295175 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f64980b85a0 con 0x7f64a0159950 2023-10-09T06:35:57.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.510+0000 7f64a83ed700 1 -- 172.21.15.163:0/3856295175 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f64a00718b0 con 0x7f64a0159950 2023-10-09T06:35:57.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.513+0000 7f64967fc700 1 -- 172.21.15.163:0/3856295175 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+0 (secure 0 0 0) 0x7f6498053fe0 con 0x7f64a0159950 2023-10-09T06:35:57.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.516+0000 7f64a83ed700 1 -- 172.21.15.163:0/3856295175 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6484067170 msgr2=0x7f6484069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:57.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.516+0000 7f64a83ed700 1 --2- 172.21.15.163:0/3856295175 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6484067170 0x7f6484069620 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f649c0011e0 tx=0x7f649c009040 comp rx=0 tx=0).stop 2023-10-09T06:35:57.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.516+0000 7f64a83ed700 1 -- 172.21.15.163:0/3856295175 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64a0159950 msgr2=0x7f64a015b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:57.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.516+0000 7f64a83ed700 1 --2- 172.21.15.163:0/3856295175 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64a0159950 0x7f64a015b320 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f64a0070e20 tx=0x7f6498019000 comp rx=0 tx=0).stop 2023-10-09T06:35:57.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.516+0000 7f64a83ed700 1 -- 172.21.15.163:0/3856295175 shutdown_connections 2023-10-09T06:35:57.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.516+0000 7f64a83ed700 1 --2- 172.21.15.163:0/3856295175 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6484067170 0x7f6484069620 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:57.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.516+0000 7f64a83ed700 1 --2- 172.21.15.163:0/3856295175 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64a0159950 0x7f64a015b320 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:57.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.516+0000 7f64a83ed700 1 --2- 172.21.15.163:0/3856295175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64a0069c70 0x7f64a00b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:57.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.516+0000 7f64a83ed700 1 --2- 172.21.15.163:0/3856295175 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64a00af320 0x7f64a00b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:57.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.516+0000 7f64a83ed700 1 -- 172.21.15.163:0/3856295175 >> 172.21.15.163:0/3856295175 conn(0x7f64a0068e00 msgr2=0x7f64a0069530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:57.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.516+0000 7f64a83ed700 1 -- 172.21.15.163:0/3856295175 shutdown_connections 2023-10-09T06:35:57.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.516+0000 7f64a83ed700 1 -- 172.21.15.163:0/3856295175 wait complete. 2023-10-09T06:35:57.518 INFO:teuthology.orchestra.run.smithi163.stderr:updated caps for client.0 2023-10-09T06:35:57.529 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:35:57.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.644+0000 7fe455f7d700 1 Processor -- start 2023-10-09T06:35:57.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.644+0000 7fe455f7d700 1 -- start start 2023-10-09T06:35:57.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.645+0000 7fe455f7d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe450159890 0x7fe45015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:57.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.645+0000 7fe455f7d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe45015a160 con 0x7fe450159c60 2023-10-09T06:35:57.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.645+0000 7fe455f7d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4500d13a0 con 0x7fe450069560 2023-10-09T06:35:57.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.645+0000 7fe455f7d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe45015e140 con 0x7fe450159890 2023-10-09T06:35:57.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.645+0000 7fe44effd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fe450159c60 0x7fe45015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57722/0 (socket says 172.21.15.163:57722) 2023-10-09T06:35:57.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.645+0000 7fe44effd700 1 -- 172.21.15.163:0/1651727298 learned_addr learned my addr 172.21.15.163:0/1651727298 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:35:57.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.645+0000 7fe44f7fe700 1 --2- 172.21.15.163:0/1651727298 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe450159890 0x7fe45015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:57.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.645+0000 7fe44e7fc700 1 -- 172.21.15.163:0/1651727298 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1210498294 0 0) 0x7fe45015a160 con 0x7fe450159c60 2023-10-09T06:35:57.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.645+0000 7fe44e7fc700 1 -- 172.21.15.163:0/1651727298 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe43c003570 con 0x7fe450159c60 2023-10-09T06:35:57.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.646+0000 7fe44e7fc700 1 -- 172.21.15.163:0/1651727298 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 611882227 0 0) 0x7fe4500d13a0 con 0x7fe450069560 2023-10-09T06:35:57.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.646+0000 7fe44e7fc700 1 -- 172.21.15.163:0/1651727298 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe45015a160 con 0x7fe450069560 2023-10-09T06:35:57.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.646+0000 7fe44e7fc700 1 -- 172.21.15.163:0/1651727298 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 720309595 0 0) 0x7fe43c003570 con 0x7fe450159c60 2023-10-09T06:35:57.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.646+0000 7fe44e7fc700 1 -- 172.21.15.163:0/1651727298 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe450159890 msgr2=0x7fe45015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:57.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.646+0000 7fe44e7fc700 1 --2- 172.21.15.163:0/1651727298 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe450159890 0x7fe45015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:57.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.646+0000 7fe44e7fc700 1 -- 172.21.15.163:0/1651727298 >> v1:172.21.15.181:6789/0 conn(0x7fe450069560 legacy=0x7fe4500ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:57.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.646+0000 7fe44e7fc700 1 -- 172.21.15.163:0/1651727298 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45015a3d0 con 0x7fe450159c60 2023-10-09T06:35:57.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.646+0000 7fe44e7fc700 1 -- 172.21.15.163:0/1651727298 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fe438001470 con 0x7fe450159c60 2023-10-09T06:35:57.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.646+0000 7fe44e7fc700 1 -- 172.21.15.163:0/1651727298 >> v1:172.21.15.163:6789/0 conn(0x7fe450159c60 legacy=0x7fe45015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:57.651 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.646+0000 7fe44e7fc700 1 --2- 172.21.15.163:0/1651727298 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe450159890 0x7fe43c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:57.651 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.646+0000 7fe44e7fc700 1 --2- 172.21.15.163:0/1651727298 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe43c004180 0x7fe43c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:57.651 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.646+0000 7fe44e7fc700 1 --2- 172.21.15.163:0/1651727298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe43c004d50 0x7fe43c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:57.651 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.646+0000 7fe44e7fc700 1 -- 172.21.15.163:0/1651727298 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45015e140 con 0x7fe43c004180 2023-10-09T06:35:57.651 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.646+0000 7fe44e7fc700 1 -- 172.21.15.163:0/1651727298 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43c00bba0 con 0x7fe43c004d50 2023-10-09T06:35:57.651 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.646+0000 7fe44e7fc700 1 -- 172.21.15.163:0/1651727298 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe43c00bd60 con 0x7fe450159890 2023-10-09T06:35:57.651 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.646+0000 7fe44effd700 1 --2- 172.21.15.163:0/1651727298 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe43c004180 0x7fe43c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:57.651 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.646+0000 7fe44f7fe700 1 --2- 172.21.15.163:0/1651727298 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe450159890 0x7fe43c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:57.651 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.646+0000 7fe44ffff700 1 --2- 172.21.15.163:0/1651727298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe43c004d50 0x7fe43c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:57.651 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.646+0000 7fe44effd700 1 -- 172.21.15.163:0/1651727298 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe450159890 msgr2=0x7fe43c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:57.651 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.646+0000 7fe44effd700 1 --2- 172.21.15.163:0/1651727298 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe450159890 0x7fe43c003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:57.651 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.646+0000 7fe44effd700 1 -- 172.21.15.163:0/1651727298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe43c004d50 msgr2=0x7fe43c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:57.651 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.646+0000 7fe44effd700 1 --2- 172.21.15.163:0/1651727298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe43c004d50 0x7fe43c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:57.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.646+0000 7fe44effd700 1 -- 172.21.15.163:0/1651727298 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe43c003570 con 0x7fe43c004180 2023-10-09T06:35:57.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.646+0000 7fe44ffff700 1 --2- 172.21.15.163:0/1651727298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe43c004d50 0x7fe43c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:35:57.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.646+0000 7fe44effd700 1 --2- 172.21.15.163:0/1651727298 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe43c004180 0x7fe43c004810 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe43800c2a0 tx=0x7fe438001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d04298ce4787f5b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:57.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.646+0000 7fe44e7fc700 1 -- 172.21.15.163:0/1651727298 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43800cb70 con 0x7fe43c004180 2023-10-09T06:35:57.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.646+0000 7fe44e7fc700 1 -- 172.21.15.163:0/1651727298 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe43800ccf0 con 0x7fe43c004180 2023-10-09T06:35:57.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.646+0000 7fe44e7fc700 1 -- 172.21.15.163:0/1651727298 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe438005b70 con 0x7fe43c004180 2023-10-09T06:35:57.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.647+0000 7fe455f7d700 1 -- 172.21.15.163:0/1651727298 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe43c004180 msgr2=0x7fe43c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:57.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.647+0000 7fe455f7d700 1 --2- 172.21.15.163:0/1651727298 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe43c004180 0x7fe43c004810 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe43800c2a0 tx=0x7fe438001e10 comp rx=0 tx=0).stop 2023-10-09T06:35:57.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.647+0000 7fe44ffff700 1 -- 172.21.15.163:0/1651727298 reap_dead start 2023-10-09T06:35:57.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.647+0000 7fe455f7d700 1 -- 172.21.15.163:0/1651727298 shutdown_connections 2023-10-09T06:35:57.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.647+0000 7fe455f7d700 1 -- 172.21.15.163:0/1651727298 >> 172.21.15.163:0/1651727298 conn(0x7fe450068e20 msgr2=0x7fe4500d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:57.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.647+0000 7fe455f7d700 1 -- 172.21.15.163:0/1651727298 shutdown_connections 2023-10-09T06:35:57.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.647+0000 7fe455f7d700 1 -- 172.21.15.163:0/1651727298 wait complete. 2023-10-09T06:35:57.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.648+0000 7fe455f7d700 1 Processor -- start 2023-10-09T06:35:57.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.648+0000 7fe455f7d700 1 -- start start 2023-10-09T06:35:57.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.649+0000 7fe455f7d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe43c004180 0x7fe450151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:57.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.649+0000 7fe455f7d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe450155260 0x7fe4501556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:57.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.649+0000 7fe455f7d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4500c5d00 0x7fe4500c6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:57.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.649+0000 7fe455f7d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe450152290 con 0x7fe43c004180 2023-10-09T06:35:57.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.649+0000 7fe455f7d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4500c66b0 con 0x7fe4500c5d00 2023-10-09T06:35:57.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.649+0000 7fe455f7d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4501537c0 con 0x7fe450155260 2023-10-09T06:35:57.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.649+0000 7fe44f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe43c004180 0x7fe450151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:57.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.649+0000 7fe44f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe43c004180 0x7fe450151d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56216/0 (socket says 172.21.15.163:56216) 2023-10-09T06:35:57.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.649+0000 7fe44f7fe700 1 -- 172.21.15.163:0/3447027263 learned_addr learned my addr 172.21.15.163:0/3447027263 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:57.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.649+0000 7fe44f7fe700 1 -- 172.21.15.163:0/3447027263 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe450155260 msgr2=0x7fe4501556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:57.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.649+0000 7fe44f7fe700 1 --2- 172.21.15.163:0/3447027263 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe450155260 0x7fe4501556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:57.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.649+0000 7fe44f7fe700 1 -- 172.21.15.163:0/3447027263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4500c5d00 msgr2=0x7fe4500c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:57.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.649+0000 7fe44ffff700 1 --2- 172.21.15.163:0/3447027263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4500c5d00 0x7fe4500c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:57.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.649+0000 7fe44f7fe700 1 --2- 172.21.15.163:0/3447027263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4500c5d00 0x7fe4500c6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:57.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.650+0000 7fe44f7fe700 1 -- 172.21.15.163:0/3447027263 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe450153a60 con 0x7fe43c004180 2023-10-09T06:35:57.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.650+0000 7fe44f7fe700 1 --2- 172.21.15.163:0/3447027263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe43c004180 0x7fe450151d50 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe440011fb0 tx=0x7fe44000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:57.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.650+0000 7fe4477fe700 1 -- 172.21.15.163:0/3447027263 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44000e040 con 0x7fe43c004180 2023-10-09T06:35:57.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.650+0000 7fe4477fe700 1 -- 172.21.15.163:0/3447027263 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44000f040 con 0x7fe43c004180 2023-10-09T06:35:57.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.650+0000 7fe4477fe700 1 -- 172.21.15.163:0/3447027263 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe440015430 con 0x7fe43c004180 2023-10-09T06:35:57.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.650+0000 7fe455f7d700 1 -- 172.21.15.163:0/3447027263 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe450153d40 con 0x7fe43c004180 2023-10-09T06:35:57.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.650+0000 7fe455f7d700 1 -- 172.21.15.163:0/3447027263 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe450157340 con 0x7fe43c004180 2023-10-09T06:35:57.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.651+0000 7fe455f7d700 1 -- 172.21.15.163:0/3447027263 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe450154390 con 0x7fe43c004180 2023-10-09T06:35:57.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.651+0000 7fe4477fe700 1 -- 172.21.15.163:0/3447027263 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fe4400157e0 con 0x7fe43c004180 2023-10-09T06:35:57.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.651+0000 7fe4477fe700 1 --2- 172.21.15.163:0/3447027263 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe430067170 0x7fe430069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:57.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.652+0000 7fe4477fe700 1 -- 172.21.15.163:0/3447027263 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe4400855f0 con 0x7fe43c004180 2023-10-09T06:35:57.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.652+0000 7fe44effd700 1 --2- 172.21.15.163:0/3447027263 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe430067170 0x7fe430069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:57.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.652+0000 7fe44effd700 1 --2- 172.21.15.163:0/3447027263 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe430067170 0x7fe430069620 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe438000c00 tx=0x7fe438015000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:57.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.656+0000 7fe4477fe700 1 -- 172.21.15.163:0/3447027263 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe440055f50 con 0x7fe43c004180 2023-10-09T06:35:57.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.881+0000 7fe455f7d700 1 -- 172.21.15.163:0/3447027263 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4501541e0 con 0x7fe43c004180 2023-10-09T06:35:57.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.882+0000 7fe4477fe700 1 -- 172.21.15.163:0/3447027263 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4608 (secure 0 0 0) 0x7fe440055ae0 con 0x7fe43c004180 2023-10-09T06:35:57.883 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:35:57.883 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1946291256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1946291256},{"type":"v1","addr":"172.21.15.181:6837","nonce":1946291256}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5749,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1366951098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1366951098},{"type":"v1","addr":"172.21.15.181:6836","nonce":1366951098}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3619934997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3619934997},{"type":"v1","addr":"172.21.15.163:6834","nonce":3619934997}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:35:52.002698+0000","modified":"2023-10-09T06:35:55.653628+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/234029086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":234029086},{"type":"v1","addr":"172.21.15.163:6835","nonce":234029086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-09T06:35:57.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.884+0000 7fe455f7d700 1 -- 172.21.15.163:0/3447027263 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe430067170 msgr2=0x7fe430069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:57.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.884+0000 7fe455f7d700 1 --2- 172.21.15.163:0/3447027263 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe430067170 0x7fe430069620 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe438000c00 tx=0x7fe438015000 comp rx=0 tx=0).stop 2023-10-09T06:35:57.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.884+0000 7fe455f7d700 1 -- 172.21.15.163:0/3447027263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe43c004180 msgr2=0x7fe450151d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:57.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.884+0000 7fe455f7d700 1 --2- 172.21.15.163:0/3447027263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe43c004180 0x7fe450151d50 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe440011fb0 tx=0x7fe44000b040 comp rx=0 tx=0).stop 2023-10-09T06:35:57.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.884+0000 7fe455f7d700 1 -- 172.21.15.163:0/3447027263 shutdown_connections 2023-10-09T06:35:57.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.884+0000 7fe455f7d700 1 --2- 172.21.15.163:0/3447027263 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe430067170 0x7fe430069620 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:57.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.884+0000 7fe455f7d700 1 --2- 172.21.15.163:0/3447027263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe43c004180 0x7fe450151d50 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:57.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.884+0000 7fe455f7d700 1 --2- 172.21.15.163:0/3447027263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4500c5d00 0x7fe4500c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:57.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.884+0000 7fe455f7d700 1 --2- 172.21.15.163:0/3447027263 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe450155260 0x7fe4501556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:57.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.884+0000 7fe455f7d700 1 -- 172.21.15.163:0/3447027263 >> 172.21.15.163:0/3447027263 conn(0x7fe450068e20 msgr2=0x7fe45015d430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:57.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.884+0000 7fe455f7d700 1 -- 172.21.15.163:0/3447027263 shutdown_connections 2023-10-09T06:35:57.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:57.884+0000 7fe455f7d700 1 -- 172.21.15.163:0/3447027263 wait complete. 2023-10-09T06:35:57.886 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 48 2023-10-09T06:35:57.929 INFO:tasks.ceph.mds.a.smithi163.stderr:2023-10-09T06:35:57.929+0000 7f730e082700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-09T06:35:58.299 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-09T06:35:52.002698+0000', 'modified': '2023-10-09T06:35:55.653628+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6171}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6171': {'gid': 6171, 'name': 'a', 'rank': 0, 'incarnation': 45, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.163:6835/234029086', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6833', 'nonce': 234029086}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 234029086}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-09T06:35:58.299 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-09T06:35:58.300 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-09T06:35:58.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.428+0000 7f2a80fe8700 1 Processor -- start 2023-10-09T06:35:58.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.428+0000 7f2a80fe8700 1 -- start start 2023-10-09T06:35:58.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.429+0000 7f2a80fe8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a7c068920 0x7f2a7c06f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:58.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.429+0000 7f2a80fe8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2a7c06f640 0x7f2a7c159510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:58.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.429+0000 7f2a80fe8700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a7c06ec70 con 0x7f2a7c068550 2023-10-09T06:35:58.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.429+0000 7f2a80fe8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a7c068d90 con 0x7f2a7c06f640 2023-10-09T06:35:58.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.429+0000 7f2a80fe8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c0b3430 con 0x7f2a7c068920 2023-10-09T06:35:58.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.429+0000 7f2a7a59c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f2a7c068550 0x7f2a7c15e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:57724/0 (socket says 172.21.15.163:57724) 2023-10-09T06:35:58.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.429+0000 7f2a7a59c700 1 -- 172.21.15.163:0/500886929 learned_addr learned my addr 172.21.15.163:0/500886929 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:35:58.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.430+0000 7f2a7ad9d700 1 --2- 172.21.15.163:0/500886929 >> v2:172.21.15.181:3300/0 conn(0x7f2a7c06f640 0x7f2a7c159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:58.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.430+0000 7f2a71d9b700 1 --2- 172.21.15.163:0/500886929 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a7c068920 0x7f2a7c06f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:58.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.430+0000 7f2a79d9b700 1 -- 172.21.15.163:0/500886929 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 574708147 0 0) 0x7f2a7c06ec70 con 0x7f2a7c068550 2023-10-09T06:35:58.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.430+0000 7f2a79d9b700 1 -- 172.21.15.163:0/500886929 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a68003570 con 0x7f2a7c068550 2023-10-09T06:35:58.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.430+0000 7f2a79d9b700 1 -- 172.21.15.163:0/500886929 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2353316817 0 0) 0x7f2a68003570 con 0x7f2a7c068550 2023-10-09T06:35:58.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.430+0000 7f2a79d9b700 1 -- 172.21.15.163:0/500886929 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a7c068920 msgr2=0x7f2a7c06f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:58.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.430+0000 7f2a79d9b700 1 --2- 172.21.15.163:0/500886929 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a7c068920 0x7f2a7c06f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.430+0000 7f2a79d9b700 1 -- 172.21.15.163:0/500886929 >> v2:172.21.15.181:3300/0 conn(0x7f2a7c06f640 msgr2=0x7f2a7c159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:58.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.430+0000 7f2a79d9b700 1 --2- 172.21.15.163:0/500886929 >> v2:172.21.15.181:3300/0 conn(0x7f2a7c06f640 0x7f2a7c159510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.430+0000 7f2a79d9b700 1 -- 172.21.15.163:0/500886929 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a7c0b2e70 con 0x7f2a7c068550 2023-10-09T06:35:58.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.430+0000 7f2a79d9b700 1 -- 172.21.15.163:0/500886929 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f2a6c001470 con 0x7f2a7c068550 2023-10-09T06:35:58.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.430+0000 7f2a79d9b700 1 -- 172.21.15.163:0/500886929 >> v1:172.21.15.163:6789/0 conn(0x7f2a7c068550 legacy=0x7f2a7c15e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:58.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.430+0000 7f2a79d9b700 1 --2- 172.21.15.163:0/500886929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a68003a60 0x7f2a68003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:58.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.430+0000 7f2a79d9b700 1 --2- 172.21.15.163:0/500886929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a68004810 0x7f2a68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:58.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.430+0000 7f2a79d9b700 1 --2- 172.21.15.163:0/500886929 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a7c068920 0x7f2a680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:58.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.430+0000 7f2a79d9b700 1 -- 172.21.15.163:0/500886929 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c068d90 con 0x7f2a68003a60 2023-10-09T06:35:58.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.430+0000 7f2a79d9b700 1 -- 172.21.15.163:0/500886929 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6800ba60 con 0x7f2a68004810 2023-10-09T06:35:58.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.430+0000 7f2a79d9b700 1 -- 172.21.15.163:0/500886929 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6800bba0 con 0x7f2a7c068920 2023-10-09T06:35:58.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.430+0000 7f2a7ad9d700 1 --2- 172.21.15.163:0/500886929 >> v2:172.21.15.181:3300/0 conn(0x7f2a7c06f640 0x7f2a7c159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:35:58.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.430+0000 7f2a7a59c700 1 --2- 172.21.15.163:0/500886929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a68003a60 0x7f2a68003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:58.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.430+0000 7f2a71d9b700 1 --2- 172.21.15.163:0/500886929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a68004810 0x7f2a68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:58.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.430+0000 7f2a7ad9d700 1 --2- 172.21.15.163:0/500886929 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a7c068920 0x7f2a680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:58.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.430+0000 7f2a7a59c700 1 -- 172.21.15.163:0/500886929 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a7c068920 msgr2=0x7f2a680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:58.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.430+0000 7f2a7a59c700 1 --2- 172.21.15.163:0/500886929 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a7c068920 0x7f2a680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.430+0000 7f2a7a59c700 1 -- 172.21.15.163:0/500886929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a68004810 msgr2=0x7f2a68004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:58.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.430+0000 7f2a7a59c700 1 --2- 172.21.15.163:0/500886929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a68004810 0x7f2a68004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.430+0000 7f2a7a59c700 1 -- 172.21.15.163:0/500886929 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a68003570 con 0x7f2a68003a60 2023-10-09T06:35:58.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.431+0000 7f2a7a59c700 1 --2- 172.21.15.163:0/500886929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a68003a60 0x7f2a68003e70 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00c3f0 tx=0x7f2a7c0b2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e6df5137034b32 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:58.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.431+0000 7f2a79d9b700 1 -- 172.21.15.163:0/500886929 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c00cc80 con 0x7f2a68003a60 2023-10-09T06:35:58.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.431+0000 7f2a79d9b700 1 -- 172.21.15.163:0/500886929 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a6c00cde0 con 0x7f2a68003a60 2023-10-09T06:35:58.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.431+0000 7f2a79d9b700 1 -- 172.21.15.163:0/500886929 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c005b70 con 0x7f2a68003a60 2023-10-09T06:35:58.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.431+0000 7f2a80fe8700 1 -- 172.21.15.163:0/500886929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a68003a60 msgr2=0x7f2a68003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:58.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.431+0000 7f2a80fe8700 1 --2- 172.21.15.163:0/500886929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a68003a60 0x7f2a68003e70 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00c3f0 tx=0x7f2a7c0b2040 comp rx=0 tx=0).stop 2023-10-09T06:35:58.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.431+0000 7f2a7ad9d700 1 -- 172.21.15.163:0/500886929 reap_dead start 2023-10-09T06:35:58.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.431+0000 7f2a80fe8700 1 -- 172.21.15.163:0/500886929 shutdown_connections 2023-10-09T06:35:58.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.431+0000 7f2a80fe8700 1 -- 172.21.15.163:0/500886929 >> 172.21.15.163:0/500886929 conn(0x7f2a7c0cd480 msgr2=0x7f2a7c0cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:58.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.431+0000 7f2a80fe8700 1 -- 172.21.15.163:0/500886929 shutdown_connections 2023-10-09T06:35:58.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.431+0000 7f2a80fe8700 1 -- 172.21.15.163:0/500886929 wait complete. 2023-10-09T06:35:58.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.432+0000 7f2a80fe8700 1 Processor -- start 2023-10-09T06:35:58.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.433+0000 7f2a80fe8700 1 -- start start 2023-10-09T06:35:58.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.433+0000 7f2a80fe8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a68003a60 0x7f2a7c169da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:58.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.433+0000 7f2a80fe8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a7c0ae660 0x7f2a7c0ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:58.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.433+0000 7f2a80fe8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a7c14f8b0 0x7f2a7c1579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:58.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.433+0000 7f2a80fe8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c0aec00 con 0x7f2a7c0ae660 2023-10-09T06:35:58.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.433+0000 7f2a80fe8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c0aeee0 con 0x7f2a7c14f8b0 2023-10-09T06:35:58.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.433+0000 7f2a80fe8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c0af1c0 con 0x7f2a68003a60 2023-10-09T06:35:58.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.433+0000 7f2a71d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a7c0ae660 0x7f2a7c0ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:58.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.433+0000 7f2a71d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a7c0ae660 0x7f2a7c0ad890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56236/0 (socket says 172.21.15.163:56236) 2023-10-09T06:35:58.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.433+0000 7f2a71d9b700 1 -- 172.21.15.163:0/3847971211 learned_addr learned my addr 172.21.15.163:0/3847971211 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:58.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.433+0000 7f2a7a59c700 1 --2- 172.21.15.163:0/3847971211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a68003a60 0x7f2a7c169da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:58.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.434+0000 7f2a7ad9d700 1 --2- 172.21.15.163:0/3847971211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a7c14f8b0 0x7f2a7c1579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:58.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.434+0000 7f2a71d9b700 1 -- 172.21.15.163:0/3847971211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a68003a60 msgr2=0x7f2a7c169da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:58.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.434+0000 7f2a71d9b700 1 --2- 172.21.15.163:0/3847971211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a68003a60 0x7f2a7c169da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.434+0000 7f2a71d9b700 1 -- 172.21.15.163:0/3847971211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a7c14f8b0 msgr2=0x7f2a7c1579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:58.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.434+0000 7f2a71d9b700 1 --2- 172.21.15.163:0/3847971211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a7c14f8b0 0x7f2a7c1579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.434+0000 7f2a71d9b700 1 -- 172.21.15.163:0/3847971211 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a7c0af4a0 con 0x7f2a7c0ae660 2023-10-09T06:35:58.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.434+0000 7f2a71d9b700 1 --2- 172.21.15.163:0/3847971211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a7c0ae660 0x7f2a7c0ad890 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f2a64011d40 tx=0x7f2a640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:58.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.434+0000 7f2a72ffd700 1 -- 172.21.15.163:0/3847971211 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6400f990 con 0x7f2a7c0ae660 2023-10-09T06:35:58.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.434+0000 7f2a72ffd700 1 -- 172.21.15.163:0/3847971211 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a6400faf0 con 0x7f2a7c0ae660 2023-10-09T06:35:58.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.434+0000 7f2a72ffd700 1 -- 172.21.15.163:0/3847971211 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6400feb0 con 0x7f2a7c0ae660 2023-10-09T06:35:58.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.434+0000 7f2a80fe8700 1 -- 172.21.15.163:0/3847971211 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a7c0addd0 con 0x7f2a7c0ae660 2023-10-09T06:35:58.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.434+0000 7f2a80fe8700 1 -- 172.21.15.163:0/3847971211 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a7c14fef0 con 0x7f2a7c0ae660 2023-10-09T06:35:58.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.436+0000 7f2a72ffd700 1 -- 172.21.15.163:0/3847971211 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f2a64014650 con 0x7f2a7c0ae660 2023-10-09T06:35:58.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.436+0000 7f2a72ffd700 1 --2- 172.21.15.163:0/3847971211 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2a5c067170 0x7f2a5c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:58.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.436+0000 7f2a72ffd700 1 -- 172.21.15.163:0/3847971211 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2a64018030 con 0x7f2a7c0ae660 2023-10-09T06:35:58.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.436+0000 7f2a7a59c700 1 --2- 172.21.15.163:0/3847971211 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2a5c067170 0x7f2a5c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:58.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.437+0000 7f2a7a59c700 1 --2- 172.21.15.163:0/3847971211 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2a5c067170 0x7f2a5c069620 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c007d10 tx=0x7f2a6c015000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:58.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.438+0000 7f2a8a2e0b80 1 Processor -- start 2023-10-09T06:35:58.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.438+0000 7f2a8a2e0b80 1 -- start start 2023-10-09T06:35:58.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.439+0000 7f2a8a2e0b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afef9c7a50 0x55afef9c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:58.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.439+0000 7f2a8a2e0b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55afefa11d00 0x55afef9f0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:58.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.439+0000 7f2a8a2e0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afef9f0f60 0x55afef9c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:58.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.439+0000 7f2a8a2e0b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afef91ea70 con 0x55afef9c7a50 2023-10-09T06:35:58.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.439+0000 7f2a8a2e0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afef91c270 con 0x55afef9f0f60 2023-10-09T06:35:58.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.439+0000 7f2a8a2e0b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55afef918060 con 0x55afefa11d00 2023-10-09T06:35:58.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.439+0000 7f2a7a59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afef9c7a50 0x55afef9c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:58.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.439+0000 7f2a7a59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afef9c7a50 0x55afef9c7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56242/0 (socket says 172.21.15.163:56242) 2023-10-09T06:35:58.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.439+0000 7f2a7a59c700 1 -- 172.21.15.163:0/3299406307 learned_addr learned my addr 172.21.15.163:0/3299406307 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:58.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.439+0000 7f2a71d9b700 1 --2- 172.21.15.163:0/3299406307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55afefa11d00 0x55afef9f0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:58.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.439+0000 7f2a7ad9d700 1 --2- 172.21.15.163:0/3299406307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afef9f0f60 0x55afef9c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:58.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.439+0000 7f2a7a59c700 1 -- 172.21.15.163:0/3299406307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55afefa11d00 msgr2=0x55afef9f0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:58.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.439+0000 7f2a7a59c700 1 --2- 172.21.15.163:0/3299406307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55afefa11d00 0x55afef9f0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.439+0000 7f2a7a59c700 1 -- 172.21.15.163:0/3299406307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afef9f0f60 msgr2=0x55afef9c3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:58.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.439+0000 7f2a7a59c700 1 --2- 172.21.15.163:0/3299406307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afef9f0f60 0x55afef9c3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.439+0000 7f2a7a59c700 1 -- 172.21.15.163:0/3299406307 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afef7b2270 con 0x55afef9c7a50 2023-10-09T06:35:58.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.440+0000 7f2a7a59c700 1 --2- 172.21.15.163:0/3299406307 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afef9c7a50 0x55afef9c7e20 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c007b80 tx=0x7f2a6c021070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b97f02ad9d21d72 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:58.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.440+0000 7f2a5b7fe700 1 -- 172.21.15.163:0/3299406307 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c000c30 con 0x55afef9c7a50 2023-10-09T06:35:58.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.440+0000 7f2a5b7fe700 1 -- 172.21.15.163:0/3299406307 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a6c023070 con 0x55afef9c7a50 2023-10-09T06:35:58.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.440+0000 7f2a5b7fe700 1 -- 172.21.15.163:0/3299406307 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c001710 con 0x55afef9c7a50 2023-10-09T06:35:58.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.440+0000 7f2a8a2e0b80 1 -- 172.21.15.163:0/3299406307 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afef9c7a50 msgr2=0x55afef9c7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:58.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.440+0000 7f2a8a2e0b80 1 --2- 172.21.15.163:0/3299406307 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afef9c7a50 0x55afef9c7e20 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c007b80 tx=0x7f2a6c021070 comp rx=0 tx=0).stop 2023-10-09T06:35:58.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.441+0000 7f2a8a2e0b80 1 -- 172.21.15.163:0/3299406307 shutdown_connections 2023-10-09T06:35:58.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.441+0000 7f2a8a2e0b80 1 --2- 172.21.15.163:0/3299406307 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afef9c7a50 0x55afef9c7e20 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.441+0000 7f2a8a2e0b80 1 --2- 172.21.15.163:0/3299406307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afef9f0f60 0x55afef9c3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.441+0000 7f2a8a2e0b80 1 --2- 172.21.15.163:0/3299406307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55afefa11d00 0x55afef9f0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.441+0000 7f2a8a2e0b80 1 -- 172.21.15.163:0/3299406307 >> 172.21.15.163:0/3299406307 conn(0x55afef9c9370 msgr2=0x55afef9c9740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:58.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.441+0000 7f2a8a2e0b80 1 -- 172.21.15.163:0/3299406307 shutdown_connections 2023-10-09T06:35:58.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.441+0000 7f2a8a2e0b80 1 -- 172.21.15.163:0/3299406307 wait complete. 2023-10-09T06:35:58.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.441+0000 7f2a8a2e0b80 1 -- start start 2023-10-09T06:35:58.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.441+0000 7f2a8a2e0b80 1 Processor -- start 2023-10-09T06:35:58.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.442+0000 7f2a8a2e0b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55afef9c7a50 0x55afefa09de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:58.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.442+0000 7f2a8a2e0b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afef9f0f60 0x55afefa06430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:58.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.442+0000 7f2a8a2e0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afefa11d00 0x55afef9ed3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:58.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.442+0000 7f2a8a2e0b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afef8c3890 con 0x55afef9f0f60 2023-10-09T06:35:58.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.442+0000 7f2a8a2e0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afef8deea0 con 0x55afefa11d00 2023-10-09T06:35:58.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.442+0000 7f2a8a2e0b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55afef79ae30 con 0x55afef9c7a50 2023-10-09T06:35:58.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.442+0000 7f2a8a2e0b80 10 client.0 _ll_register_callbacks cb 0x55afef98b110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-09T06:35:58.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.442+0000 7f2a71d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afef9f0f60 0x55afefa06430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:58.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.442+0000 7f2a71d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afef9f0f60 0x55afefa06430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56256/0 (socket says 172.21.15.163:56256) 2023-10-09T06:35:58.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.442+0000 7f2a71d9b700 1 -- 172.21.15.163:0/733153438 learned_addr learned my addr 172.21.15.163:0/733153438 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:58.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.442+0000 7f2a7a59c700 1 --2- 172.21.15.163:0/733153438 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55afef9c7a50 0x55afefa09de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:58.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.442+0000 7f2a7ad9d700 1 --2- 172.21.15.163:0/733153438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afefa11d00 0x55afef9ed3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:58.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.443+0000 7f2a71d9b700 1 -- 172.21.15.163:0/733153438 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55afef9c7a50 msgr2=0x55afefa09de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:58.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.443+0000 7f2a71d9b700 1 --2- 172.21.15.163:0/733153438 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55afef9c7a50 0x55afefa09de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.443+0000 7f2a71d9b700 1 -- 172.21.15.163:0/733153438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afefa11d00 msgr2=0x55afef9ed3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:58.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.443+0000 7f2a71d9b700 1 --2- 172.21.15.163:0/733153438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afefa11d00 0x55afef9ed3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.443+0000 7f2a71d9b700 1 -- 172.21.15.163:0/733153438 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a68003570 con 0x55afef9f0f60 2023-10-09T06:35:58.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.443+0000 7f2a71d9b700 1 --2- 172.21.15.163:0/733153438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afef9f0f60 0x55afefa06430 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f2a64000c00 tx=0x7f2a64008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a665a84820af32 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:58.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.443+0000 7f2a58ff9700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-10-09T06:35:58.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.443+0000 7f2a58ff9700 1 -- 172.21.15.163:0/733153438 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6405d1a0 con 0x55afef9f0f60 2023-10-09T06:35:58.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.443+0000 7f2a58ff9700 1 -- 172.21.15.163:0/733153438 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2a6405b050 con 0x55afef9f0f60 2023-10-09T06:35:58.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.443+0000 7f2a8a2e0b80 1 -- 172.21.15.163:0/733153438 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55afef9bcb70 con 0x55afef9f0f60 2023-10-09T06:35:58.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.443+0000 7f2a58ff9700 1 -- 172.21.15.163:0/733153438 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a640852e0 con 0x55afef9f0f60 2023-10-09T06:35:58.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.443+0000 7f2a8a2e0b80 10 client.6309 fetch_fsmap learned FSMap version 48 2023-10-09T06:35:58.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.443+0000 7f2a8a2e0b80 1 -- 172.21.15.163:0/733153438 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55afef754fe0 con 0x55afef9f0f60 2023-10-09T06:35:58.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.443+0000 7f2a58ff9700 1 -- 172.21.15.163:0/733153438 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55afef9bcb70 con 0x55afef9f0f60 2023-10-09T06:35:58.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.443+0000 7f2a58ff9700 1 -- 172.21.15.163:0/733153438 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 48) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f2a640556f0 con 0x55afef9f0f60 2023-10-09T06:35:58.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.443+0000 7f2a8a2e0b80 10 client.6309 fetch_fsmap finished waiting for FSMap version 48 2023-10-09T06:35:58.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.443+0000 7f2a8a2e0b80 10 client.6309 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-09T06:35:58.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.443+0000 7f2a8a2e0b80 20 client.6309 populate_metadata read hostname 'smithi163' 2023-10-09T06:35:58.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.443+0000 7f2a8a2e0b80 1 --2- 172.21.15.163:0/733153438 >> [v2:172.21.15.163:6833/234029086,v1:172.21.15.163:6835/234029086] conn(0x55afef9f6e80 0x55afef9f7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:58.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.443+0000 7f2a8a2e0b80 4 client.6309 mds_command: new command op to 6171 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-09T06:35:58.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.443+0000 7f2a8a2e0b80 1 -- 172.21.15.163:0/733153438 --> [v2:172.21.15.163:6833/234029086,v1:172.21.15.163:6835/234029086] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55afef999aa0 con 0x55afef9f6e80 2023-10-09T06:35:58.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.444+0000 7f2a7a59c700 1 --2- 172.21.15.163:0/733153438 >> [v2:172.21.15.163:6833/234029086,v1:172.21.15.163:6835/234029086] conn(0x55afef9f6e80 0x55afef9f7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:58.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.444+0000 7f2a58ff9700 10 client.6309 ms_handle_connect on v2:172.21.15.163:6833/234029086 2023-10-09T06:35:58.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.444+0000 7f2a7a59c700 1 --2- 172.21.15.163:0/733153438 >> [v2:172.21.15.163:6833/234029086,v1:172.21.15.163:6835/234029086] conn(0x55afef9f6e80 0x55afef9f7250 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1525ee4005e1f5d8 server_cookie=b0e8c67f40df6203 in_seq=0 out_seq=0 2023-10-09T06:35:58.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.444+0000 7f2a58ff9700 1 -- 172.21.15.163:0/733153438 <== mds.0 v2:172.21.15.163:6833/234029086 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55afef999aa0 con 0x55afef9f6e80 2023-10-09T06:35:58.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.444+0000 7f2a58ff9700 10 client.6309 handle_command_reply: tid=0 2023-10-09T06:35:58.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.444+0000 7f2a8a2e0b80 1 client.6309 shutdown 2023-10-09T06:35:58.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.444+0000 7f2a8a2e0b80 1 -- 172.21.15.163:0/733153438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afef9f0f60 msgr2=0x55afefa06430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:58.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.444+0000 7f2a8a2e0b80 1 --2- 172.21.15.163:0/733153438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afef9f0f60 0x55afefa06430 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f2a64000c00 tx=0x7f2a64008040 comp rx=0 tx=0).stop 2023-10-09T06:35:58.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.445+0000 7f2a8a2e0b80 1 -- 172.21.15.163:0/733153438 shutdown_connections 2023-10-09T06:35:58.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.445+0000 7f2a8a2e0b80 1 --2- 172.21.15.163:0/733153438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afefa11d00 0x55afef9ed3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.445+0000 7f2a8a2e0b80 1 --2- 172.21.15.163:0/733153438 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55afef9c7a50 0x55afefa09de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.445+0000 7f2a8a2e0b80 1 --2- 172.21.15.163:0/733153438 >> [v2:172.21.15.163:6833/234029086,v1:172.21.15.163:6835/234029086] conn(0x55afef9f6e80 0x55afef9f7250 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.445+0000 7f2a8a2e0b80 1 --2- 172.21.15.163:0/733153438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afef9f0f60 0x55afefa06430 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.445+0000 7f2a8a2e0b80 1 -- 172.21.15.163:0/733153438 >> 172.21.15.163:0/733153438 conn(0x55afef9c9370 msgr2=0x55afefa0d550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:58.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.445+0000 7f2a58ff9700 0 client.6309 ms_handle_reset on v2:172.21.15.163:6833/234029086 2023-10-09T06:35:58.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.445+0000 7f2a8a2e0b80 1 -- 172.21.15.163:0/733153438 shutdown_connections 2023-10-09T06:35:58.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.445+0000 7f2a8a2e0b80 1 -- 172.21.15.163:0/733153438 wait complete. 2023-10-09T06:35:58.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.445+0000 7f2a8a2e0b80 20 client.6309 trim_cache size 0 max 16384 2023-10-09T06:35:58.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.461+0000 7f2a8a2e0b80 1 Processor -- start 2023-10-09T06:35:58.463 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.461+0000 7f2a8a2e0b80 1 -- start start 2023-10-09T06:35:58.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.462+0000 7f2a8a2e0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afef9c7a50 0x55afefa08620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:58.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.462+0000 7f2a8a2e0b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55afef9f0f60 0x55afefa08b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:58.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.462+0000 7f2a8a2e0b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afefa11d00 0x55afef9eb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:58.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.462+0000 7f2a8a2e0b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afef9c35f0 con 0x55afefa11d00 2023-10-09T06:35:58.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.462+0000 7f2a8a2e0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afefa06fa0 con 0x55afef9c7a50 2023-10-09T06:35:58.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.462+0000 7f2a8a2e0b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55afef9c7330 con 0x55afef9f0f60 2023-10-09T06:35:58.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.462+0000 7f2a7ad9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afefa11d00 0x55afef9eb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:58.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.462+0000 7f2a7ad9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afefa11d00 0x55afef9eb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56272/0 (socket says 172.21.15.163:56272) 2023-10-09T06:35:58.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.462+0000 7f2a7ad9d700 1 -- 172.21.15.163:0/4161691076 learned_addr learned my addr 172.21.15.163:0/4161691076 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:58.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.462+0000 7f2a7ad9d700 1 -- 172.21.15.163:0/4161691076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55afef9f0f60 msgr2=0x55afefa08b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:58.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.462+0000 7f2a7ad9d700 1 --2- 172.21.15.163:0/4161691076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55afef9f0f60 0x55afefa08b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.462+0000 7f2a7ad9d700 1 -- 172.21.15.163:0/4161691076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afef9c7a50 msgr2=0x55afefa08620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:58.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.462+0000 7f2a7ad9d700 1 --2- 172.21.15.163:0/4161691076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afef9c7a50 0x55afefa08620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.462+0000 7f2a7ad9d700 1 -- 172.21.15.163:0/4161691076 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afef8e6780 con 0x55afefa11d00 2023-10-09T06:35:58.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.463+0000 7f2a7ad9d700 1 --2- 172.21.15.163:0/4161691076 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afefa11d00 0x55afef9eb8a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0aceb0 tx=0x7f2a7400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2bfe5577dd76a0c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:58.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.463+0000 7f2a5bfff700 1 -- 172.21.15.163:0/4161691076 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7400e070 con 0x55afefa11d00 2023-10-09T06:35:58.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.463+0000 7f2a5bfff700 1 -- 172.21.15.163:0/4161691076 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7400f040 con 0x55afefa11d00 2023-10-09T06:35:58.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.463+0000 7f2a5bfff700 1 -- 172.21.15.163:0/4161691076 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a740199e0 con 0x55afefa11d00 2023-10-09T06:35:58.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.463+0000 7f2a8a2e0b80 1 -- 172.21.15.163:0/4161691076 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afefa11d00 msgr2=0x55afef9eb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:58.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.463+0000 7f2a8a2e0b80 1 --2- 172.21.15.163:0/4161691076 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afefa11d00 0x55afef9eb8a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0aceb0 tx=0x7f2a7400b040 comp rx=0 tx=0).stop 2023-10-09T06:35:58.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.463+0000 7f2a8a2e0b80 1 -- 172.21.15.163:0/4161691076 shutdown_connections 2023-10-09T06:35:58.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.463+0000 7f2a8a2e0b80 1 --2- 172.21.15.163:0/4161691076 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afefa11d00 0x55afef9eb8a0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.463+0000 7f2a8a2e0b80 1 --2- 172.21.15.163:0/4161691076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afef9c7a50 0x55afefa08620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.463+0000 7f2a8a2e0b80 1 --2- 172.21.15.163:0/4161691076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55afef9f0f60 0x55afefa08b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.463+0000 7f2a8a2e0b80 1 -- 172.21.15.163:0/4161691076 >> 172.21.15.163:0/4161691076 conn(0x55afef9c9370 msgr2=0x55afefa09dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:58.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.463+0000 7f2a8a2e0b80 1 -- 172.21.15.163:0/4161691076 shutdown_connections 2023-10-09T06:35:58.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.463+0000 7f2a8a2e0b80 1 -- 172.21.15.163:0/4161691076 wait complete. 2023-10-09T06:35:58.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.464+0000 7f2a8a2e0b80 1 -- start start 2023-10-09T06:35:58.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.464+0000 7f2a8a2e0b80 1 Processor -- start 2023-10-09T06:35:58.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.465+0000 7f2a8a2e0b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afef9c7a50 0x55afef9eb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:58.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.465+0000 7f2a8a2e0b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afef9f0f60 0x55afefa08620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:58.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.465+0000 7f2a8a2e0b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55afefa11d00 0x55afefa08b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:58.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.465+0000 7f2a8a2e0b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afef9ec4b0 con 0x55afef9f0f60 2023-10-09T06:35:58.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.465+0000 7f2a8a2e0b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afefa74ca0 con 0x55afef9c7a50 2023-10-09T06:35:58.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.465+0000 7f2a8a2e0b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55afefa754b0 con 0x55afefa11d00 2023-10-09T06:35:58.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.465+0000 7f2a8a2e0b80 10 client.0 _ll_register_callbacks cb 0x55afef98b110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-09T06:35:58.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.465+0000 7f2a71d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afef9f0f60 0x55afefa08620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:58.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.465+0000 7f2a71d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afef9f0f60 0x55afefa08620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56284/0 (socket says 172.21.15.163:56284) 2023-10-09T06:35:58.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.465+0000 7f2a71d9b700 1 -- 172.21.15.163:0/574429023 learned_addr learned my addr 172.21.15.163:0/574429023 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:58.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.465+0000 7f2a7ad9d700 1 --2- 172.21.15.163:0/574429023 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55afefa11d00 0x55afefa08b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:58.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.465+0000 7f2a71d9b700 1 -- 172.21.15.163:0/574429023 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55afefa11d00 msgr2=0x55afefa08b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:58.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.465+0000 7f2a7a59c700 1 --2- 172.21.15.163:0/574429023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afef9c7a50 0x55afef9eb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:58.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.465+0000 7f2a71d9b700 1 --2- 172.21.15.163:0/574429023 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55afefa11d00 0x55afefa08b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.465+0000 7f2a71d9b700 1 -- 172.21.15.163:0/574429023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afef9c7a50 msgr2=0x55afef9eb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:58.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.465+0000 7f2a71d9b700 1 --2- 172.21.15.163:0/574429023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afef9c7a50 0x55afef9eb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.465+0000 7f2a71d9b700 1 -- 172.21.15.163:0/574429023 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a74009a20 con 0x55afef9f0f60 2023-10-09T06:35:58.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.465+0000 7f2a71d9b700 1 --2- 172.21.15.163:0/574429023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afef9f0f60 0x55afefa08620 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f2a64056190 tx=0x7f2a6401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ddc0771b96bea6c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:58.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.465+0000 7f2a58ff9700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-10-09T06:35:58.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.466+0000 7f2a58ff9700 1 -- 172.21.15.163:0/574429023 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6405d1a0 con 0x55afef9f0f60 2023-10-09T06:35:58.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.466+0000 7f2a58ff9700 1 -- 172.21.15.163:0/574429023 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2a6405b050 con 0x55afef9f0f60 2023-10-09T06:35:58.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.466+0000 7f2a8a2e0b80 1 -- 172.21.15.163:0/574429023 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a6c023070 con 0x55afef9f0f60 2023-10-09T06:35:58.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.466+0000 7f2a58ff9700 1 -- 172.21.15.163:0/574429023 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a640556f0 con 0x55afef9f0f60 2023-10-09T06:35:58.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.466+0000 7f2a8a2e0b80 10 client.6315 fetch_fsmap learned FSMap version 48 2023-10-09T06:35:58.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.466+0000 7f2a8a2e0b80 1 -- 172.21.15.163:0/574429023 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55afef8cd470 con 0x55afef9f0f60 2023-10-09T06:35:58.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.466+0000 7f2a58ff9700 1 -- 172.21.15.163:0/574429023 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a64070080 con 0x55afef9f0f60 2023-10-09T06:35:58.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.466+0000 7f2a58ff9700 1 -- 172.21.15.163:0/574429023 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 48) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f2a6405b420 con 0x55afef9f0f60 2023-10-09T06:35:58.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.466+0000 7f2a8a2e0b80 10 client.6315 fetch_fsmap finished waiting for FSMap version 48 2023-10-09T06:35:58.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.466+0000 7f2a8a2e0b80 10 client.6315 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-09T06:35:58.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.466+0000 7f2a8a2e0b80 20 client.6315 populate_metadata read hostname 'smithi163' 2023-10-09T06:35:58.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.466+0000 7f2a8a2e0b80 1 --2- 172.21.15.163:0/574429023 >> [v2:172.21.15.163:6833/234029086,v1:172.21.15.163:6835/234029086] conn(0x55afefa746d0 0x55afefa272f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:58.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.466+0000 7f2a8a2e0b80 4 client.6315 mds_command: new command op to 6171 tid=0[{"prefix": "status"}] 2023-10-09T06:35:58.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.466+0000 7f2a8a2e0b80 1 -- 172.21.15.163:0/574429023 --> [v2:172.21.15.163:6833/234029086,v1:172.21.15.163:6835/234029086] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55afef8bae00 con 0x55afefa746d0 2023-10-09T06:35:58.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.466+0000 7f2a7a59c700 1 --2- 172.21.15.163:0/574429023 >> [v2:172.21.15.163:6833/234029086,v1:172.21.15.163:6835/234029086] conn(0x55afefa746d0 0x55afefa272f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:58.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.466+0000 7f2a58ff9700 10 client.6315 ms_handle_connect on v2:172.21.15.163:6833/234029086 2023-10-09T06:35:58.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.466+0000 7f2a7a59c700 1 --2- 172.21.15.163:0/574429023 >> [v2:172.21.15.163:6833/234029086,v1:172.21.15.163:6835/234029086] conn(0x55afefa746d0 0x55afefa272f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a22ff5a56c93243f server_cookie=185d900aebc19ed3 in_seq=0 out_seq=0 2023-10-09T06:35:58.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.467+0000 7f2a58ff9700 1 -- 172.21.15.163:0/574429023 <== mds.0 v2:172.21.15.163:6833/234029086 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55afef8bae00 con 0x55afefa746d0 2023-10-09T06:35:58.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.467+0000 7f2a58ff9700 10 client.6315 handle_command_reply: tid=0 2023-10-09T06:35:58.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.467+0000 7f2a8a2e0b80 1 client.6315 shutdown 2023-10-09T06:35:58.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.467+0000 7f2a8a2e0b80 1 -- 172.21.15.163:0/574429023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afef9f0f60 msgr2=0x55afefa08620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:58.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.467+0000 7f2a8a2e0b80 1 --2- 172.21.15.163:0/574429023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afef9f0f60 0x55afefa08620 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f2a64056190 tx=0x7f2a6401f000 comp rx=0 tx=0).stop 2023-10-09T06:35:58.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.467+0000 7f2a8a2e0b80 1 -- 172.21.15.163:0/574429023 shutdown_connections 2023-10-09T06:35:58.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.467+0000 7f2a8a2e0b80 1 --2- 172.21.15.163:0/574429023 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55afefa11d00 0x55afefa08b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.467+0000 7f2a8a2e0b80 1 --2- 172.21.15.163:0/574429023 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55afef9c7a50 0x55afef9eb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.467+0000 7f2a8a2e0b80 1 --2- 172.21.15.163:0/574429023 >> [v2:172.21.15.163:6833/234029086,v1:172.21.15.163:6835/234029086] conn(0x55afefa746d0 0x55afefa272f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.467+0000 7f2a8a2e0b80 1 --2- 172.21.15.163:0/574429023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55afef9f0f60 0x55afefa08620 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.467+0000 7f2a58ff9700 0 client.6315 ms_handle_reset on v2:172.21.15.163:6833/234029086 2023-10-09T06:35:58.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.467+0000 7f2a8a2e0b80 1 -- 172.21.15.163:0/574429023 >> 172.21.15.163:0/574429023 conn(0x55afef9c9370 msgr2=0x55afefa09d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:58.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.467+0000 7f2a8a2e0b80 1 -- 172.21.15.163:0/574429023 shutdown_connections 2023-10-09T06:35:58.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.467+0000 7f2a8a2e0b80 1 -- 172.21.15.163:0/574429023 wait complete. 2023-10-09T06:35:58.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.467+0000 7f2a8a2e0b80 20 client.6315 trim_cache size 0 max 16384 2023-10-09T06:35:58.783 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:35:58.783 INFO:teuthology.orchestra.run.smithi163.stdout: "cluster_fsid": "2520b8f7-539f-4bae-9f17-1fce66f66ddd", 2023-10-09T06:35:58.783 INFO:teuthology.orchestra.run.smithi163.stdout: "whoami": 0, 2023-10-09T06:35:58.783 INFO:teuthology.orchestra.run.smithi163.stdout: "id": 6171, 2023-10-09T06:35:58.783 INFO:teuthology.orchestra.run.smithi163.stdout: "want_state": "up:active", 2023-10-09T06:35:58.784 INFO:teuthology.orchestra.run.smithi163.stdout: "state": "up:active", 2023-10-09T06:35:58.784 INFO:teuthology.orchestra.run.smithi163.stdout: "fs_name": "cephfs", 2023-10-09T06:35:58.784 INFO:teuthology.orchestra.run.smithi163.stdout: "rank_uptime": 5.8226613990000002, 2023-10-09T06:35:58.784 INFO:teuthology.orchestra.run.smithi163.stdout: "mdsmap_epoch": 48, 2023-10-09T06:35:58.784 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch": 77, 2023-10-09T06:35:58.784 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch_barrier": 77, 2023-10-09T06:35:58.784 INFO:teuthology.orchestra.run.smithi163.stdout: "uptime": 10.949701464 2023-10-09T06:35:58.784 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:35:58.786 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.467+0000 7f2a80fe8700 1 -- 172.21.15.163:0/3847971211 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2a5c067170 msgr2=0x7f2a5c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:58.786 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.467+0000 7f2a80fe8700 1 --2- 172.21.15.163:0/3847971211 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2a5c067170 0x7f2a5c069620 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c007d10 tx=0x7f2a6c015000 comp rx=0 tx=0).stop 2023-10-09T06:35:58.787 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.467+0000 7f2a80fe8700 1 -- 172.21.15.163:0/3847971211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a7c0ae660 msgr2=0x7f2a7c0ad890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:58.787 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.467+0000 7f2a80fe8700 1 --2- 172.21.15.163:0/3847971211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a7c0ae660 0x7f2a7c0ad890 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f2a64011d40 tx=0x7f2a640091a0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.787 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.468+0000 7f2a80fe8700 1 -- 172.21.15.163:0/3847971211 shutdown_connections 2023-10-09T06:35:58.787 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.468+0000 7f2a80fe8700 1 --2- 172.21.15.163:0/3847971211 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2a5c067170 0x7f2a5c069620 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.787 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.468+0000 7f2a80fe8700 1 --2- 172.21.15.163:0/3847971211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a7c14f8b0 0x7f2a7c1579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.787 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.468+0000 7f2a80fe8700 1 --2- 172.21.15.163:0/3847971211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a68003a60 0x7f2a7c169da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.787 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.468+0000 7f2a80fe8700 1 --2- 172.21.15.163:0/3847971211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a7c0ae660 0x7f2a7c0ad890 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.787 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.468+0000 7f2a80fe8700 1 -- 172.21.15.163:0/3847971211 >> 172.21.15.163:0/3847971211 conn(0x7f2a7c0cd480 msgr2=0x7f2a7c0b0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:58.787 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.468+0000 7f2a80fe8700 1 -- 172.21.15.163:0/3847971211 shutdown_connections 2023-10-09T06:35:58.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.468+0000 7f2a80fe8700 1 -- 172.21.15.163:0/3847971211 wait complete. 2023-10-09T06:35:58.788 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-09T06:35:58.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.902+0000 7f6419d56700 1 Processor -- start 2023-10-09T06:35:58.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.902+0000 7f6419d56700 1 -- start start 2023-10-09T06:35:58.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.903+0000 7f6419d56700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64141599c0 0x7f6414069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:58.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.903+0000 7f6419d56700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f64140cbd00 0x7f64140ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:58.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.903+0000 7f6419d56700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f641406a220 con 0x7f6414069aa0 2023-10-09T06:35:58.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.903+0000 7f6419d56700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6414069f40 con 0x7f64140cbd00 2023-10-09T06:35:58.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.903+0000 7f6419d56700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f641415a0b0 con 0x7f64141599c0 2023-10-09T06:35:58.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.904+0000 7f64137fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64141599c0 0x7f6414069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:58.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.904+0000 7f6413fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f64140cbd00 0x7f64140ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:58.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.904+0000 7f6412ffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6414069aa0 0x7f64140cb5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:57218/0 (socket says 172.21.15.163:57218) 2023-10-09T06:35:58.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.904+0000 7f6412ffd700 1 -- 172.21.15.163:0/4118771865 learned_addr learned my addr 172.21.15.163:0/4118771865 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:35:58.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.904+0000 7f6413fff700 1 -- 172.21.15.163:0/4118771865 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64141599c0 msgr2=0x7f6414069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:58.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.904+0000 7f6413fff700 1 --2- 172.21.15.163:0/4118771865 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64141599c0 0x7f6414069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.904+0000 7f6413fff700 1 -- 172.21.15.163:0/4118771865 >> v1:172.21.15.181:6789/0 conn(0x7f6414069aa0 legacy=0x7f64140cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:58.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.904+0000 7f64127fc700 1 -- 172.21.15.163:0/4118771865 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3487836191 0 0) 0x7f641406a220 con 0x7f6414069aa0 2023-10-09T06:35:58.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.904+0000 7f6413fff700 1 -- 172.21.15.163:0/4118771865 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f641415a390 con 0x7f64140cbd00 2023-10-09T06:35:58.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.904+0000 7f64137fe700 1 --2- 172.21.15.163:0/4118771865 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64141599c0 0x7f6414069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:35:58.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.904+0000 7f6413fff700 1 --2- 172.21.15.163:0/4118771865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64140cbd00 0x7f64140ce2e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f640c0098e0 tx=0x7f640c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c833dd9b6967ff4c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:58.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.904+0000 7f64127fc700 1 -- 172.21.15.163:0/4118771865 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c00b640 con 0x7f64140cbd00 2023-10-09T06:35:58.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.904+0000 7f64127fc700 1 -- 172.21.15.163:0/4118771865 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f640c00b7a0 con 0x7f64140cbd00 2023-10-09T06:35:58.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.904+0000 7f64127fc700 1 -- 172.21.15.163:0/4118771865 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c00bb40 con 0x7f64140cbd00 2023-10-09T06:35:58.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.905+0000 7f6419d56700 1 -- 172.21.15.163:0/4118771865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64140cbd00 msgr2=0x7f64140ce2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:58.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.905+0000 7f6419d56700 1 --2- 172.21.15.163:0/4118771865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64140cbd00 0x7f64140ce2e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f640c0098e0 tx=0x7f640c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.905+0000 7f6419d56700 1 -- 172.21.15.163:0/4118771865 shutdown_connections 2023-10-09T06:35:58.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.905+0000 7f6419d56700 1 --2- 172.21.15.163:0/4118771865 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64140cbd00 0x7f64140ce2e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.905+0000 7f6419d56700 1 --2- 172.21.15.163:0/4118771865 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64141599c0 0x7f6414069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.905+0000 7f6419d56700 1 -- 172.21.15.163:0/4118771865 >> 172.21.15.163:0/4118771865 conn(0x7f6414068e10 msgr2=0x7f64140d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:58.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.905+0000 7f6419d56700 1 -- 172.21.15.163:0/4118771865 shutdown_connections 2023-10-09T06:35:58.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.905+0000 7f6419d56700 1 -- 172.21.15.163:0/4118771865 wait complete. 2023-10-09T06:35:58.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.906+0000 7f6419d56700 1 Processor -- start 2023-10-09T06:35:58.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.906+0000 7f6419d56700 1 -- start start 2023-10-09T06:35:58.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.907+0000 7f6419d56700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6414069aa0 0x7f64140b60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:58.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.907+0000 7f6419d56700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64140cbd00 0x7f64140b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:58.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.907+0000 7f6419d56700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64141599c0 0x7f641414d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:58.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.907+0000 7f6419d56700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641415b520 con 0x7f64141599c0 2023-10-09T06:35:58.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.907+0000 7f6419d56700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64140b4550 con 0x7f64140cbd00 2023-10-09T06:35:58.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.907+0000 7f6419d56700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64140b4830 con 0x7f6414069aa0 2023-10-09T06:35:58.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.907+0000 7f6413fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64141599c0 0x7f641414d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:58.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.907+0000 7f6413fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64141599c0 0x7f641414d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40418/0 (socket says 172.21.15.163:40418) 2023-10-09T06:35:58.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.907+0000 7f6413fff700 1 -- 172.21.15.163:0/2172131920 learned_addr learned my addr 172.21.15.163:0/2172131920 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:58.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.907+0000 7f64137fe700 1 --2- 172.21.15.163:0/2172131920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6414069aa0 0x7f64140b60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:58.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.907+0000 7f6412ffd700 1 --2- 172.21.15.163:0/2172131920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64140cbd00 0x7f64140b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:58.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.907+0000 7f6413fff700 1 -- 172.21.15.163:0/2172131920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6414069aa0 msgr2=0x7f64140b60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:58.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.907+0000 7f6413fff700 1 --2- 172.21.15.163:0/2172131920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6414069aa0 0x7f64140b60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.907+0000 7f6413fff700 1 -- 172.21.15.163:0/2172131920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64140cbd00 msgr2=0x7f64140b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:58.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.907+0000 7f6413fff700 1 --2- 172.21.15.163:0/2172131920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64140cbd00 0x7f64140b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:58.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.907+0000 7f6413fff700 1 -- 172.21.15.163:0/2172131920 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6404009a20 con 0x7f64141599c0 2023-10-09T06:35:58.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.907+0000 7f6412ffd700 1 --2- 172.21.15.163:0/2172131920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64140cbd00 0x7f64140b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:35:58.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.908+0000 7f6413fff700 1 --2- 172.21.15.163:0/2172131920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64141599c0 0x7f641414d860 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f640c013000 tx=0x7f640c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:58.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.908+0000 7f63fb7fe700 1 -- 172.21.15.163:0/2172131920 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c011420 con 0x7f64141599c0 2023-10-09T06:35:58.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.908+0000 7f63fb7fe700 1 -- 172.21.15.163:0/2172131920 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f640c0115a0 con 0x7f64141599c0 2023-10-09T06:35:58.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.908+0000 7f6419d56700 1 -- 172.21.15.163:0/2172131920 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f640c009420 con 0x7f64141599c0 2023-10-09T06:35:58.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.908+0000 7f63fb7fe700 1 -- 172.21.15.163:0/2172131920 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c011960 con 0x7f64141599c0 2023-10-09T06:35:58.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.908+0000 7f6419d56700 1 -- 172.21.15.163:0/2172131920 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64140b2890 con 0x7f64141599c0 2023-10-09T06:35:58.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.909+0000 7f6419d56700 1 -- 172.21.15.163:0/2172131920 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f641415ba00 con 0x7f64141599c0 2023-10-09T06:35:58.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.909+0000 7f63fb7fe700 1 -- 172.21.15.163:0/2172131920 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f640c011d10 con 0x7f64141599c0 2023-10-09T06:35:58.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.909+0000 7f63fb7fe700 1 --2- 172.21.15.163:0/2172131920 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6400067170 0x7f6400069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:58.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.910+0000 7f63fb7fe700 1 -- 172.21.15.163:0/2172131920 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f640c0861e0 con 0x7f64141599c0 2023-10-09T06:35:58.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.910+0000 7f64137fe700 1 --2- 172.21.15.163:0/2172131920 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6400067170 0x7f6400069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:58.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.910+0000 7f64137fe700 1 --2- 172.21.15.163:0/2172131920 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6400067170 0x7f6400069620 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f640400f650 tx=0x7f640400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:58.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:58.914+0000 7f63fb7fe700 1 -- 172.21.15.163:0/2172131920 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f641415ba00 con 0x7f64141599c0 2023-10-09T06:35:59.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.139+0000 7f6419d56700 1 -- 172.21.15.163:0/2172131920 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f641415ba00 con 0x7f64141599c0 2023-10-09T06:35:59.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.140+0000 7f63fb7fe700 1 -- 172.21.15.163:0/2172131920 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v48) v1 ==== 53+0+73 (secure 0 0 0) 0x7f641415ba00 con 0x7f64141599c0 2023-10-09T06:35:59.142 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-09T06:35:59.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.142+0000 7f6419d56700 1 -- 172.21.15.163:0/2172131920 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6400067170 msgr2=0x7f6400069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:59.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.142+0000 7f6419d56700 1 --2- 172.21.15.163:0/2172131920 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6400067170 0x7f6400069620 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f640400f650 tx=0x7f640400b040 comp rx=0 tx=0).stop 2023-10-09T06:35:59.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.142+0000 7f6419d56700 1 -- 172.21.15.163:0/2172131920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64141599c0 msgr2=0x7f641414d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:59.144 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.142+0000 7f6419d56700 1 --2- 172.21.15.163:0/2172131920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64141599c0 0x7f641414d860 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f640c013000 tx=0x7f640c017000 comp rx=0 tx=0).stop 2023-10-09T06:35:59.144 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.142+0000 7f6419d56700 1 -- 172.21.15.163:0/2172131920 shutdown_connections 2023-10-09T06:35:59.144 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.142+0000 7f6419d56700 1 --2- 172.21.15.163:0/2172131920 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6400067170 0x7f6400069620 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:59.144 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.142+0000 7f6419d56700 1 --2- 172.21.15.163:0/2172131920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64141599c0 0x7f641414d860 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:59.144 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.142+0000 7f6419d56700 1 --2- 172.21.15.163:0/2172131920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6414069aa0 0x7f64140b60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:59.144 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.142+0000 7f6419d56700 1 --2- 172.21.15.163:0/2172131920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64140cbd00 0x7f64140b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:59.144 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.142+0000 7f6419d56700 1 -- 172.21.15.163:0/2172131920 >> 172.21.15.163:0/2172131920 conn(0x7f6414068e10 msgr2=0x7f64140ce3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:59.144 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.142+0000 7f6419d56700 1 -- 172.21.15.163:0/2172131920 shutdown_connections 2023-10-09T06:35:59.144 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.142+0000 7f6419d56700 1 -- 172.21.15.163:0/2172131920 wait complete. 2023-10-09T06:35:59.155 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-09T06:35:59.155 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-09T06:35:59.156 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi163.front.sepia.ceph.com 2023-10-09T06:35:59.156 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-09T06:35:59.156 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-09T06:35:59.156 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-09T06:35:59.156 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-09T06:35:59.156 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-09T06:35:59.157 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:35:59.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:35:59.157 DEBUG:teuthology.orchestra.run.smithi163:> ip addr 2023-10-09T06:35:59.173 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:35:59.174 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:35:59.174 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:35:59.174 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:35:59.174 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2023-10-09T06:35:59.174 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:35:59.175 INFO:teuthology.orchestra.run.smithi163.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:35:59.175 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:06 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:35:59.175 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f0 2023-10-09T06:35:59.175 INFO:teuthology.orchestra.run.smithi163.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:35:59.175 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:07 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:35:59.175 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f1 2023-10-09T06:35:59.175 INFO:teuthology.orchestra.run.smithi163.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:35:59.175 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5a brd ff:ff:ff:ff:ff:ff 2023-10-09T06:35:59.175 INFO:teuthology.orchestra.run.smithi163.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-09T06:35:59.175 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5b brd ff:ff:ff:ff:ff:ff 2023-10-09T06:35:59.176 INFO:teuthology.orchestra.run.smithi163.stdout: inet 172.21.15.163/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-09T06:35:59.176 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft 42164sec preferred_lft 42164sec 2023-10-09T06:35:59.176 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::ec4:7aff:fe88:7d5b/64 scope link 2023-10-09T06:35:59.176 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:35:59.176 INFO:teuthology.orchestra.run.smithi163.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-09T06:35:59.176 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 2e:01:84:b9:f4:8e brd ff:ff:ff:ff:ff:ff 2023-10-09T06:35:59.176 INFO:teuthology.orchestra.run.smithi163.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-09T06:35:59.176 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:35:59.176 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::d03c:93ff:fe9e:c8cb/64 scope link 2023-10-09T06:35:59.176 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:35:59.177 INFO:teuthology.orchestra.run.smithi163.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:35:59.177 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 2e:01:84:b9:f4:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-09T06:35:59.177 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::2c01:84ff:feb9:f48e/64 scope link 2023-10-09T06:35:59.177 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:35:59.177 INFO:teuthology.orchestra.run.smithi163.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:35:59.177 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether c6:88:3e:d7:c1:56 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-09T06:35:59.177 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::c488:3eff:fed7:c156/64 scope link 2023-10-09T06:35:59.178 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:35:59.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:35:59.178 DEBUG:teuthology.orchestra.run.smithi163:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-09T06:35:59.237 INFO:teuthology.orchestra.run.smithi163.stdout:1 2023-10-09T06:35:59.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:35:59.238 DEBUG:teuthology.orchestra.run.smithi163:> route 2023-10-09T06:35:59.295 INFO:teuthology.orchestra.run.smithi163.stdout:Kernel IP routing table 2023-10-09T06:35:59.295 INFO:teuthology.orchestra.run.smithi163.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-09T06:35:59.295 INFO:teuthology.orchestra.run.smithi163.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-09T06:35:59.295 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-09T06:35:59.296 INFO:teuthology.orchestra.run.smithi163.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-09T06:35:59.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:35:59.296 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-09T06:35:59.296 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-09T06:35:59.296 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-09T06:35:59.296 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-09T06:35:59.296 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-09T06:35:59.296 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-09T06:35:59.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:35:59.397 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2023-10-09T06:35:59.413 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-09T06:35:59.413 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-09T06:35:59.413 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:35:59.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:35:59.413 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set brx.0 up 2023-10-09T06:35:59.478 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:35:59.479 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-09T06:35:59.534 INFO:teuthology.orchestra.run.smithi163.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:35:59.534 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:35:59.534 DEBUG:teuthology.orchestra.run.smithi163:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-09T06:35:59.591 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-09T06:35:59.592 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-09T06:35:59.592 DEBUG:teuthology.orchestra.run.smithi163:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:35:59.671 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:35:59.671 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:35:59.691 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:35:59.691 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:35:59.691 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:35:59.691 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with new device syntax: 0@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:35:59.691 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:35:59.691 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:35:59.691 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:35:59.691 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:35:59.714 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:35:59.714 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:35:59.714 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:35:59.715 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with new device syntax: 0@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:35:59.715 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:35:59.715 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:35:59.715 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:35:59.715 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:35:59.715 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790" 2023-10-09T06:35:59.715 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790:/ 2023-10-09T06:35:59.715 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=2520b8f7-539f-4bae-9f17-1fce66f66ddd" will pass to kernel 2023-10-09T06:35:59.716 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-09T06:35:59.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:35:59.717 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-09T06:35:59.782 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-09T06:35:59.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.936+0000 7f7ccec95700 1 Processor -- start 2023-10-09T06:35:59.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.936+0000 7f7ccec95700 1 -- start start 2023-10-09T06:35:59.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.937+0000 7f7ccec95700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7cc8068920 0x7f7cc806f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:59.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.937+0000 7f7ccec95700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cc806f640 0x7f7cc8159510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:59.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.937+0000 7f7ccec95700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cc806ec70 con 0x7f7cc8068550 2023-10-09T06:35:59.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.937+0000 7f7ccec95700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cc8068d90 con 0x7f7cc8068920 2023-10-09T06:35:59.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.937+0000 7f7ccec95700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc80b3430 con 0x7f7cc806f640 2023-10-09T06:35:59.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.937+0000 7f7cc7fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7cc8068920 0x7f7cc806f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:59.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.937+0000 7f7ccca31700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f7cc8068550 0x7f7cc815e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60612/0 (socket says 172.21.15.163:60612) 2023-10-09T06:35:59.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.937+0000 7f7ccca31700 1 -- 172.21.15.163:0/2001748582 learned_addr learned my addr 172.21.15.163:0/2001748582 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:35:59.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.937+0000 7f7cc7fff700 1 -- 172.21.15.163:0/2001748582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cc806f640 msgr2=0x7f7cc8159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:35:59.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.937+0000 7f7ccd232700 1 --2- 172.21.15.163:0/2001748582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cc806f640 0x7f7cc8159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:59.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.937+0000 7f7cc7fff700 1 --2- 172.21.15.163:0/2001748582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cc806f640 0x7f7cc8159510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:59.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.937+0000 7f7cc77fe700 1 -- 172.21.15.163:0/2001748582 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936037851 0 0) 0x7f7cc806ec70 con 0x7f7cc8068550 2023-10-09T06:35:59.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.937+0000 7f7cc7fff700 1 -- 172.21.15.163:0/2001748582 >> v1:172.21.15.163:6789/0 conn(0x7f7cc8068550 legacy=0x7f7cc815e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:59.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.937+0000 7f7cc7fff700 1 -- 172.21.15.163:0/2001748582 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc80b2e70 con 0x7f7cc8068920 2023-10-09T06:35:59.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.937+0000 7f7ccd232700 1 --2- 172.21.15.163:0/2001748582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cc806f640 0x7f7cc8159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:35:59.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.937+0000 7f7cc7fff700 1 --2- 172.21.15.163:0/2001748582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cc8068920 0x7f7cc806f100 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4009910 tx=0x7f7cb40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6d5c99c400a4d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:59.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.938+0000 7f7cc77fe700 1 -- 172.21.15.163:0/2001748582 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb400b820 con 0x7f7cc8068920 2023-10-09T06:35:59.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.938+0000 7f7cc77fe700 1 -- 172.21.15.163:0/2001748582 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb400b980 con 0x7f7cc8068920 2023-10-09T06:35:59.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.938+0000 7f7cc77fe700 1 -- 172.21.15.163:0/2001748582 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb400bd20 con 0x7f7cc8068920 2023-10-09T06:35:59.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.938+0000 7f7ccec95700 1 -- 172.21.15.163:0/2001748582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cc8068920 msgr2=0x7f7cc806f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:59.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.938+0000 7f7ccec95700 1 --2- 172.21.15.163:0/2001748582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cc8068920 0x7f7cc806f100 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4009910 tx=0x7f7cb40091a0 comp rx=0 tx=0).stop 2023-10-09T06:35:59.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.938+0000 7f7ccec95700 1 -- 172.21.15.163:0/2001748582 shutdown_connections 2023-10-09T06:35:59.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.938+0000 7f7ccec95700 1 --2- 172.21.15.163:0/2001748582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cc806f640 0x7f7cc8159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:59.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.938+0000 7f7ccec95700 1 --2- 172.21.15.163:0/2001748582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cc8068920 0x7f7cc806f100 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:59.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.938+0000 7f7ccec95700 1 -- 172.21.15.163:0/2001748582 >> 172.21.15.163:0/2001748582 conn(0x7f7cc80cd480 msgr2=0x7f7cc80cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:35:59.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.938+0000 7f7ccec95700 1 -- 172.21.15.163:0/2001748582 shutdown_connections 2023-10-09T06:35:59.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.938+0000 7f7ccec95700 1 -- 172.21.15.163:0/2001748582 wait complete. 2023-10-09T06:35:59.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.939+0000 7f7ccec95700 1 Processor -- start 2023-10-09T06:35:59.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.939+0000 7f7ccec95700 1 -- start start 2023-10-09T06:35:59.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.940+0000 7f7ccec95700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cc8068550 0x7f7cc813f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:59.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.940+0000 7f7ccec95700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cc8068920 0x7f7cc80ae780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:59.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.940+0000 7f7ccec95700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cc806f640 0x7f7cc80aa040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:59.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.940+0000 7f7ccec95700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc814f700 con 0x7f7cc8068920 2023-10-09T06:35:59.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.940+0000 7f7ccec95700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc80a98d0 con 0x7f7cc806f640 2023-10-09T06:35:59.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.940+0000 7f7ccec95700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc80aad60 con 0x7f7cc8068550 2023-10-09T06:35:59.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.940+0000 7f7cc7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cc8068920 0x7f7cc80ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:59.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.940+0000 7f7cc7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cc8068920 0x7f7cc80ae780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40432/0 (socket says 172.21.15.163:40432) 2023-10-09T06:35:59.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.940+0000 7f7cc7fff700 1 -- 172.21.15.163:0/3745963638 learned_addr learned my addr 172.21.15.163:0/3745963638 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:35:59.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.940+0000 7f7ccd232700 1 --2- 172.21.15.163:0/3745963638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cc806f640 0x7f7cc80aa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:59.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.940+0000 7f7ccca31700 1 --2- 172.21.15.163:0/3745963638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cc8068550 0x7f7cc813f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:59.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.940+0000 7f7cc7fff700 1 -- 172.21.15.163:0/3745963638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cc8068550 msgr2=0x7f7cc813f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:59.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.940+0000 7f7cc7fff700 1 --2- 172.21.15.163:0/3745963638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cc8068550 0x7f7cc813f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:59.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.940+0000 7f7cc7fff700 1 -- 172.21.15.163:0/3745963638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cc806f640 msgr2=0x7f7cc80aa040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:35:59.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.940+0000 7f7cc7fff700 1 --2- 172.21.15.163:0/3745963638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cc806f640 0x7f7cc80aa040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:35:59.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.940+0000 7f7cc7fff700 1 -- 172.21.15.163:0/3745963638 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb4009420 con 0x7f7cc8068920 2023-10-09T06:35:59.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.940+0000 7f7ccca31700 1 --2- 172.21.15.163:0/3745963638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cc8068550 0x7f7cc813f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:35:59.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.941+0000 7f7cc7fff700 1 --2- 172.21.15.163:0/3745963638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cc8068920 0x7f7cc80ae780 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7cb400b3c0 tx=0x7f7cb4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:59.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.941+0000 7f7cc4ff9700 1 -- 172.21.15.163:0/3745963638 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4024440 con 0x7f7cc8068920 2023-10-09T06:35:59.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.941+0000 7f7cc4ff9700 1 -- 172.21.15.163:0/3745963638 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb40245a0 con 0x7f7cc8068920 2023-10-09T06:35:59.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.941+0000 7f7ccec95700 1 -- 172.21.15.163:0/3745963638 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cc80ab040 con 0x7f7cc8068920 2023-10-09T06:35:59.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.941+0000 7f7cc4ff9700 1 -- 172.21.15.163:0/3745963638 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4024960 con 0x7f7cc8068920 2023-10-09T06:35:59.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.941+0000 7f7ccec95700 1 -- 172.21.15.163:0/3745963638 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cc80a9b20 con 0x7f7cc8068920 2023-10-09T06:35:59.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.942+0000 7f7ccec95700 1 -- 172.21.15.163:0/3745963638 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cc804c3c0 con 0x7f7cc8068920 2023-10-09T06:35:59.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.942+0000 7f7cc4ff9700 1 -- 172.21.15.163:0/3745963638 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f7cb401a420 con 0x7f7cc8068920 2023-10-09T06:35:59.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.942+0000 7f7cc4ff9700 1 --2- 172.21.15.163:0/3745963638 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7cac067170 0x7f7cac069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:35:59.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.943+0000 7f7cc4ff9700 1 -- 172.21.15.163:0/3745963638 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7cb40868b0 con 0x7f7cc8068920 2023-10-09T06:35:59.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.943+0000 7f7ccca31700 1 --2- 172.21.15.163:0/3745963638 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7cac067170 0x7f7cac069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:35:59.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.943+0000 7f7ccca31700 1 --2- 172.21.15.163:0/3745963638 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7cac067170 0x7f7cac069620 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc0018d0 tx=0x7f7cbc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:35:59.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:35:59.947+0000 7f7cc4ff9700 1 -- 172.21.15.163:0/3745963638 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7cb4053890 con 0x7f7cc8068920 2023-10-09T06:36:00.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.172+0000 7f7ccec95700 1 -- 172.21.15.163:0/3745963638 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7cc80b3150 con 0x7f7cc8068920 2023-10-09T06:36:00.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.172+0000 7f7cc4ff9700 1 -- 172.21.15.163:0/3745963638 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v48) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7cb4056eb0 con 0x7f7cc8068920 2023-10-09T06:36:00.173 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-09T06:36:00.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.174+0000 7f7ccec95700 1 -- 172.21.15.163:0/3745963638 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7cac067170 msgr2=0x7f7cac069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:00.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.174+0000 7f7ccec95700 1 --2- 172.21.15.163:0/3745963638 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7cac067170 0x7f7cac069620 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc0018d0 tx=0x7f7cbc009040 comp rx=0 tx=0).stop 2023-10-09T06:36:00.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.174+0000 7f7ccec95700 1 -- 172.21.15.163:0/3745963638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cc8068920 msgr2=0x7f7cc80ae780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:00.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.174+0000 7f7ccec95700 1 --2- 172.21.15.163:0/3745963638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cc8068920 0x7f7cc80ae780 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7cb400b3c0 tx=0x7f7cb4017000 comp rx=0 tx=0).stop 2023-10-09T06:36:00.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.174+0000 7f7ccec95700 1 -- 172.21.15.163:0/3745963638 shutdown_connections 2023-10-09T06:36:00.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.174+0000 7f7ccec95700 1 --2- 172.21.15.163:0/3745963638 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7cac067170 0x7f7cac069620 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:00.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.174+0000 7f7ccec95700 1 --2- 172.21.15.163:0/3745963638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cc806f640 0x7f7cc80aa040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:00.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.174+0000 7f7ccec95700 1 --2- 172.21.15.163:0/3745963638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cc8068550 0x7f7cc813f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:00.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.174+0000 7f7ccec95700 1 --2- 172.21.15.163:0/3745963638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cc8068920 0x7f7cc80ae780 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:00.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.174+0000 7f7ccec95700 1 -- 172.21.15.163:0/3745963638 >> 172.21.15.163:0/3745963638 conn(0x7f7cc80cd480 msgr2=0x7f7cc80b0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:00.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.174+0000 7f7ccec95700 1 -- 172.21.15.163:0/3745963638 shutdown_connections 2023-10-09T06:36:00.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.174+0000 7f7ccec95700 1 -- 172.21.15.163:0/3745963638 wait complete. 2023-10-09T06:36:00.187 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-09T06:36:00.187 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-09T06:36:00.187 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-09T06:36:00.188 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-09T06:36:00.188 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-09T06:36:00.188 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-09T06:36:00.188 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-09T06:36:00.188 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-09T06:36:00.188 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:36:00.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:00.188 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-09T06:36:00.206 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:36:00.207 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:36:00.207 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:36:00.207 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:00.207 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-09T06:36:00.207 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:00.207 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:36:00.207 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:36:00.207 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2023-10-09T06:36:00.207 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:36:00.207 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:36:00.207 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2023-10-09T06:36:00.208 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:36:00.208 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-09T06:36:00.208 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-09T06:36:00.208 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-09T06:36:00.208 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-09T06:36:00.208 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42140sec preferred_lft 42140sec 2023-10-09T06:36:00.208 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2023-10-09T06:36:00.208 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:00.208 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-09T06:36:00.208 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 72:62:e6:72:7c:2e brd ff:ff:ff:ff:ff:ff 2023-10-09T06:36:00.208 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-09T06:36:00.208 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:00.208 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::e89a:4cff:fefb:7198/64 scope link 2023-10-09T06:36:00.209 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:00.209 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:36:00.209 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 72:62:e6:72:7c:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-09T06:36:00.209 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::7062:e6ff:fe72:7c2e/64 scope link 2023-10-09T06:36:00.209 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:00.209 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:36:00.209 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether fa:4e:a3:9d:5a:5f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-09T06:36:00.209 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::f84e:a3ff:fe9d:5a5f/64 scope link 2023-10-09T06:36:00.209 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:00.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:00.210 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-09T06:36:00.274 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-09T06:36:00.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:00.275 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-09T06:36:00.334 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-09T06:36:00.335 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-09T06:36:00.335 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-09T06:36:00.335 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-09T06:36:00.335 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-09T06:36:00.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:00.335 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-09T06:36:00.335 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-09T06:36:00.336 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-09T06:36:00.336 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-09T06:36:00.336 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-09T06:36:00.336 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-09T06:36:00.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:00.441 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-09T06:36:00.457 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-09T06:36:00.457 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-09T06:36:00.457 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:36:00.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:00.457 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-09T06:36:00.522 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:36:00.522 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-09T06:36:00.577 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:36:00.578 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:36:00.578 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-09T06:36:00.631 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-09T06:36:00.632 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-09T06:36:00.632 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:36:00.712 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:36:00.712 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:36:00.730 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:36:00.731 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:36:00.731 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:36:00.731 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 1@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:36:00.731 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:36:00.731 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:36:00.731 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:36:00.732 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:36:00.754 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:36:00.754 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:36:00.754 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:36:00.754 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 1@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:36:00.754 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:36:00.755 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:36:00.755 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:36:00.755 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:36:00.755 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790" 2023-10-09T06:36:00.755 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790:/ 2023-10-09T06:36:00.755 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=2520b8f7-539f-4bae-9f17-1fce66f66ddd" will pass to kernel 2023-10-09T06:36:00.757 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-09T06:36:00.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:00.757 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-09T06:36:00.820 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-09T06:36:00.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.938+0000 7fc99a867700 1 Processor -- start 2023-10-09T06:36:00.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.938+0000 7fc99a867700 1 -- start start 2023-10-09T06:36:00.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.939+0000 7fc99a867700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc9940cd910 0x7fc9940ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:00.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.939+0000 7fc99a867700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc9940b55a0 0x7fc9940ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:00.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.939+0000 7fc99a867700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9940b5b00 con 0x7fc994159ec0 2023-10-09T06:36:00.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.939+0000 7fc99a867700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9940b5da0 con 0x7fc9940cd910 2023-10-09T06:36:00.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.939+0000 7fc99a867700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9940b6080 con 0x7fc9940b55a0 2023-10-09T06:36:00.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.939+0000 7fc9937fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc9940cd910 0x7fc9940ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:00.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.939+0000 7fc9937fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc9940cd910 0x7fc9940ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40434/0 (socket says 172.21.15.163:40434) 2023-10-09T06:36:00.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.939+0000 7fc9937fe700 1 -- 172.21.15.163:0/1607608954 learned_addr learned my addr 172.21.15.163:0/1607608954 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:00.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.939+0000 7fc998e04700 1 --2- 172.21.15.163:0/1607608954 >> v2:172.21.15.181:3300/0 conn(0x7fc9940b55a0 0x7fc9940ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:00.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.940+0000 7fc9937fe700 1 -- 172.21.15.163:0/1607608954 >> v2:172.21.15.181:3300/0 conn(0x7fc9940b55a0 msgr2=0x7fc9940ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:00.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.940+0000 7fc9937fe700 1 --2- 172.21.15.163:0/1607608954 >> v2:172.21.15.181:3300/0 conn(0x7fc9940b55a0 0x7fc9940ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:00.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.940+0000 7fc9937fe700 1 -- 172.21.15.163:0/1607608954 >> v1:172.21.15.181:6789/0 conn(0x7fc994159ec0 legacy=0x7fc9940b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:00.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.940+0000 7fc9937fe700 1 -- 172.21.15.163:0/1607608954 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9940cc100 con 0x7fc9940cd910 2023-10-09T06:36:00.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.940+0000 7fc9937fe700 1 --2- 172.21.15.163:0/1607608954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9940cd910 0x7fc9940ce340 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc9880099b0 tx=0x7fc988009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6390eff672b22f21 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:00.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.940+0000 7fc992ffd700 1 -- 172.21.15.163:0/1607608954 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98800b820 con 0x7fc9940cd910 2023-10-09T06:36:00.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.940+0000 7fc992ffd700 1 -- 172.21.15.163:0/1607608954 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98800b980 con 0x7fc9940cd910 2023-10-09T06:36:00.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.940+0000 7fc992ffd700 1 -- 172.21.15.163:0/1607608954 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98800bd20 con 0x7fc9940cd910 2023-10-09T06:36:00.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.940+0000 7fc99a867700 1 -- 172.21.15.163:0/1607608954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9940cd910 msgr2=0x7fc9940ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:00.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.940+0000 7fc99a867700 1 --2- 172.21.15.163:0/1607608954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9940cd910 0x7fc9940ce340 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc9880099b0 tx=0x7fc988009250 comp rx=0 tx=0).stop 2023-10-09T06:36:00.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.940+0000 7fc99a867700 1 -- 172.21.15.163:0/1607608954 shutdown_connections 2023-10-09T06:36:00.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.940+0000 7fc99a867700 1 --2- 172.21.15.163:0/1607608954 >> v2:172.21.15.181:3300/0 conn(0x7fc9940b55a0 0x7fc9940ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:00.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.940+0000 7fc99a867700 1 --2- 172.21.15.163:0/1607608954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9940cd910 0x7fc9940ce340 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:00.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.940+0000 7fc99a867700 1 -- 172.21.15.163:0/1607608954 >> 172.21.15.163:0/1607608954 conn(0x7fc9940999f0 msgr2=0x7fc9940b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:00.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.940+0000 7fc99a867700 1 -- 172.21.15.163:0/1607608954 shutdown_connections 2023-10-09T06:36:00.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.940+0000 7fc99a867700 1 -- 172.21.15.163:0/1607608954 wait complete. 2023-10-09T06:36:00.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.941+0000 7fc99a867700 1 Processor -- start 2023-10-09T06:36:00.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.942+0000 7fc99a867700 1 -- start start 2023-10-09T06:36:00.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.942+0000 7fc99a867700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9940b55a0 0x7fc9941518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:00.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.942+0000 7fc99a867700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc994159ec0 0x7fc99414e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:00.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.942+0000 7fc99a867700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc994152250 0x7fc9940c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:00.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.942+0000 7fc99a867700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99414eb40 con 0x7fc994152250 2023-10-09T06:36:00.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.942+0000 7fc99a867700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc994152cb0 con 0x7fc994159ec0 2023-10-09T06:36:00.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.942+0000 7fc99a867700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc994152f50 con 0x7fc9940b55a0 2023-10-09T06:36:00.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.943+0000 7fc998e04700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc994152250 0x7fc9940c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:00.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.943+0000 7fc998e04700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc994152250 0x7fc9940c8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40450/0 (socket says 172.21.15.163:40450) 2023-10-09T06:36:00.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.943+0000 7fc998e04700 1 -- 172.21.15.163:0/2447228785 learned_addr learned my addr 172.21.15.163:0/2447228785 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:00.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.943+0000 7fc9937fe700 1 --2- 172.21.15.163:0/2447228785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc994159ec0 0x7fc99414e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:00.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.943+0000 7fc993fff700 1 --2- 172.21.15.163:0/2447228785 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9940b55a0 0x7fc9941518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:00.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.943+0000 7fc998e04700 1 -- 172.21.15.163:0/2447228785 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9940b55a0 msgr2=0x7fc9941518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:00.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.943+0000 7fc998e04700 1 --2- 172.21.15.163:0/2447228785 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9940b55a0 0x7fc9941518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:00.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.943+0000 7fc998e04700 1 -- 172.21.15.163:0/2447228785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc994159ec0 msgr2=0x7fc99414e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:00.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.943+0000 7fc998e04700 1 --2- 172.21.15.163:0/2447228785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc994159ec0 0x7fc99414e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:00.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.943+0000 7fc998e04700 1 -- 172.21.15.163:0/2447228785 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9880094f0 con 0x7fc994152250 2023-10-09T06:36:00.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.943+0000 7fc998e04700 1 --2- 172.21.15.163:0/2447228785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc994152250 0x7fc9940c8a30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc98c007f20 tx=0x7fc98c009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:00.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.943+0000 7fc97bfff700 1 -- 172.21.15.163:0/2447228785 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c00eb90 con 0x7fc994152250 2023-10-09T06:36:00.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.943+0000 7fc97bfff700 1 -- 172.21.15.163:0/2447228785 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98c00ed10 con 0x7fc994152250 2023-10-09T06:36:00.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.943+0000 7fc99a867700 1 -- 172.21.15.163:0/2447228785 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc994153230 con 0x7fc994152250 2023-10-09T06:36:00.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.943+0000 7fc97bfff700 1 -- 172.21.15.163:0/2447228785 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c014670 con 0x7fc994152250 2023-10-09T06:36:00.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.943+0000 7fc99a867700 1 -- 172.21.15.163:0/2447228785 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc994155550 con 0x7fc994152250 2023-10-09T06:36:00.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.944+0000 7fc99a867700 1 -- 172.21.15.163:0/2447228785 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9940b4100 con 0x7fc994152250 2023-10-09T06:36:00.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.945+0000 7fc97bfff700 1 -- 172.21.15.163:0/2447228785 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fc98c0149e0 con 0x7fc994152250 2023-10-09T06:36:00.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.945+0000 7fc97bfff700 1 --2- 172.21.15.163:0/2447228785 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc980067170 0x7fc980069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:00.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.945+0000 7fc97bfff700 1 -- 172.21.15.163:0/2447228785 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc98c084e70 con 0x7fc994152250 2023-10-09T06:36:00.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.945+0000 7fc993fff700 1 --2- 172.21.15.163:0/2447228785 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc980067170 0x7fc980069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:00.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.945+0000 7fc993fff700 1 --2- 172.21.15.163:0/2447228785 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc980067170 0x7fc980069620 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc984007fd0 tx=0x7fc984007eb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:00.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:00.949+0000 7fc97bfff700 1 -- 172.21.15.163:0/2447228785 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc98c051e50 con 0x7fc994152250 2023-10-09T06:36:01.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.176+0000 7fc99a867700 1 -- 172.21.15.163:0/2447228785 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc994063c90 con 0x7fc994152250 2023-10-09T06:36:01.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.687+0000 7fc97bfff700 1 -- 172.21.15.163:0/2447228785 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v49) v1 ==== 136+0+0 (secure 0 0 0) 0x7fc98c055470 con 0x7fc994152250 2023-10-09T06:36:01.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.689+0000 7fc99a867700 1 -- 172.21.15.163:0/2447228785 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc980067170 msgr2=0x7fc980069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:01.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.689+0000 7fc99a867700 1 --2- 172.21.15.163:0/2447228785 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc980067170 0x7fc980069620 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc984007fd0 tx=0x7fc984007eb0 comp rx=0 tx=0).stop 2023-10-09T06:36:01.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.689+0000 7fc99a867700 1 -- 172.21.15.163:0/2447228785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc994152250 msgr2=0x7fc9940c8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:01.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.689+0000 7fc99a867700 1 --2- 172.21.15.163:0/2447228785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc994152250 0x7fc9940c8a30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc98c007f20 tx=0x7fc98c009860 comp rx=0 tx=0).stop 2023-10-09T06:36:01.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.689+0000 7fc99a867700 1 -- 172.21.15.163:0/2447228785 shutdown_connections 2023-10-09T06:36:01.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.689+0000 7fc99a867700 1 --2- 172.21.15.163:0/2447228785 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc980067170 0x7fc980069620 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:01.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.689+0000 7fc99a867700 1 --2- 172.21.15.163:0/2447228785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc994152250 0x7fc9940c8a30 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:01.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.689+0000 7fc99a867700 1 --2- 172.21.15.163:0/2447228785 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9940b55a0 0x7fc9941518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:01.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.689+0000 7fc99a867700 1 --2- 172.21.15.163:0/2447228785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc994159ec0 0x7fc99414e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:01.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.689+0000 7fc99a867700 1 -- 172.21.15.163:0/2447228785 >> 172.21.15.163:0/2447228785 conn(0x7fc9940999f0 msgr2=0x7fc9941592d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:01.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.689+0000 7fc99a867700 1 -- 172.21.15.163:0/2447228785 shutdown_connections 2023-10-09T06:36:01.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.690+0000 7fc99a867700 1 -- 172.21.15.163:0/2447228785 wait complete. 2023-10-09T06:36:01.703 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-09T06:36:01.819 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.819+0000 7fc01027a700 1 Processor -- start 2023-10-09T06:36:01.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.819+0000 7fc01027a700 1 -- start start 2023-10-09T06:36:01.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.820+0000 7fc01027a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc008159890 0x7fc00815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:01.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.820+0000 7fc01027a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc008069560 0x7fc0080699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:01.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.820+0000 7fc01027a700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc00815a160 con 0x7fc008159c60 2023-10-09T06:36:01.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.820+0000 7fc01027a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc008069f20 con 0x7fc008069560 2023-10-09T06:36:01.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.820+0000 7fc01027a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0080d13a0 con 0x7fc008159890 2023-10-09T06:36:01.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.820+0000 7fc00e817700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc008069560 0x7fc0080699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:01.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.820+0000 7fc00e817700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc008069560 0x7fc0080699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40458/0 (socket says 172.21.15.163:40458) 2023-10-09T06:36:01.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.820+0000 7fc00d815700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fc008159c60 0x7fc00815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60618/0 (socket says 172.21.15.163:60618) 2023-10-09T06:36:01.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.820+0000 7fc00e817700 1 -- 172.21.15.163:0/1869945251 learned_addr learned my addr 172.21.15.163:0/1869945251 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:01.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.820+0000 7fc00e016700 1 --2- 172.21.15.163:0/1869945251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc008159890 0x7fc00815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:01.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.820+0000 7fc00e817700 1 -- 172.21.15.163:0/1869945251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc008159890 msgr2=0x7fc00815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:01.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.820+0000 7fc00e817700 1 --2- 172.21.15.163:0/1869945251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc008159890 0x7fc00815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:01.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.820+0000 7fc00e817700 1 -- 172.21.15.163:0/1869945251 >> v1:172.21.15.163:6789/0 conn(0x7fc008159c60 legacy=0x7fc00815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:01.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.820+0000 7fc00e817700 1 -- 172.21.15.163:0/1869945251 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00815e140 con 0x7fc008069560 2023-10-09T06:36:01.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.820+0000 7fc00d014700 1 -- 172.21.15.163:0/1869945251 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 876971881 0 0) 0x7fc00815a160 con 0x7fc008159c60 2023-10-09T06:36:01.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.820+0000 7fc00e817700 1 --2- 172.21.15.163:0/1869945251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc008069560 0x7fc0080699e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc0000099b0 tx=0x7fc000009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557fb0018dfb033b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:01.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.820+0000 7fc00d014700 1 -- 172.21.15.163:0/1869945251 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00000b860 con 0x7fc008069560 2023-10-09T06:36:01.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.820+0000 7fc00d014700 1 -- 172.21.15.163:0/1869945251 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00000b9c0 con 0x7fc008069560 2023-10-09T06:36:01.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.820+0000 7fc00d014700 1 -- 172.21.15.163:0/1869945251 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00000bd60 con 0x7fc008069560 2023-10-09T06:36:01.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.821+0000 7fc01027a700 1 -- 172.21.15.163:0/1869945251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc008069560 msgr2=0x7fc0080699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:01.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.821+0000 7fc01027a700 1 --2- 172.21.15.163:0/1869945251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc008069560 0x7fc0080699e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc0000099b0 tx=0x7fc000009250 comp rx=0 tx=0).stop 2023-10-09T06:36:01.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.821+0000 7fc01027a700 1 -- 172.21.15.163:0/1869945251 shutdown_connections 2023-10-09T06:36:01.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.821+0000 7fc01027a700 1 --2- 172.21.15.163:0/1869945251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc008069560 0x7fc0080699e0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:01.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.821+0000 7fc01027a700 1 --2- 172.21.15.163:0/1869945251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc008159890 0x7fc00815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:01.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.821+0000 7fc01027a700 1 -- 172.21.15.163:0/1869945251 >> 172.21.15.163:0/1869945251 conn(0x7fc008068e20 msgr2=0x7fc0080d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:01.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.821+0000 7fc01027a700 1 -- 172.21.15.163:0/1869945251 shutdown_connections 2023-10-09T06:36:01.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.821+0000 7fc01027a700 1 -- 172.21.15.163:0/1869945251 wait complete. 2023-10-09T06:36:01.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.822+0000 7fc01027a700 1 Processor -- start 2023-10-09T06:36:01.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.822+0000 7fc01027a700 1 -- start start 2023-10-09T06:36:01.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.823+0000 7fc01027a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc008069560 0x7fc00815c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:01.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.823+0000 7fc01027a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc008159890 0x7fc008153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:01.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.823+0000 7fc01027a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc008159c60 0x7fc00816a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:01.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.823+0000 7fc01027a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0080b3590 con 0x7fc008069560 2023-10-09T06:36:01.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.823+0000 7fc01027a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0080b3870 con 0x7fc008159c60 2023-10-09T06:36:01.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.823+0000 7fc01027a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc00816a040 con 0x7fc008159890 2023-10-09T06:36:01.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.823+0000 7fc00e016700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc008069560 0x7fc00815c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:01.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.823+0000 7fc00e016700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc008069560 0x7fc00815c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40470/0 (socket says 172.21.15.163:40470) 2023-10-09T06:36:01.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.823+0000 7fc00e016700 1 -- 172.21.15.163:0/3392358496 learned_addr learned my addr 172.21.15.163:0/3392358496 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:01.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.823+0000 7fc00d815700 1 --2- 172.21.15.163:0/3392358496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc008159890 0x7fc008153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:01.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.823+0000 7fc00e817700 1 --2- 172.21.15.163:0/3392358496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc008159c60 0x7fc00816a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:01.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.823+0000 7fc00e016700 1 -- 172.21.15.163:0/3392358496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc008159890 msgr2=0x7fc008153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:01.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.823+0000 7fc00e016700 1 --2- 172.21.15.163:0/3392358496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc008159890 0x7fc008153f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:01.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.823+0000 7fc00e016700 1 -- 172.21.15.163:0/3392358496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc008159c60 msgr2=0x7fc00816a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:01.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.823+0000 7fc00e016700 1 --2- 172.21.15.163:0/3392358496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc008159c60 0x7fc00816a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:01.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.823+0000 7fc00e016700 1 -- 172.21.15.163:0/3392358496 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0000094f0 con 0x7fc008069560 2023-10-09T06:36:01.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.823+0000 7fc00d815700 1 --2- 172.21.15.163:0/3392358496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc008159890 0x7fc008153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:01.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.823+0000 7fc00e817700 1 --2- 172.21.15.163:0/3392358496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc008159c60 0x7fc00816a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:01.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.823+0000 7fc00e016700 1 --2- 172.21.15.163:0/3392358496 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc008069560 0x7fc00815c790 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc00400e1c0 tx=0x7fc004009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:01.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.824+0000 7fbffe7fc700 1 -- 172.21.15.163:0/3392358496 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00400ee60 con 0x7fc008069560 2023-10-09T06:36:01.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.824+0000 7fbffe7fc700 1 -- 172.21.15.163:0/3392358496 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc004014420 con 0x7fc008069560 2023-10-09T06:36:01.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.824+0000 7fbffe7fc700 1 -- 172.21.15.163:0/3392358496 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0040147c0 con 0x7fc008069560 2023-10-09T06:36:01.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.824+0000 7fc01027a700 1 -- 172.21.15.163:0/3392358496 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc00816a320 con 0x7fc008069560 2023-10-09T06:36:01.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.824+0000 7fc01027a700 1 -- 172.21.15.163:0/3392358496 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0081697f0 con 0x7fc008069560 2023-10-09T06:36:01.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.824+0000 7fc01027a700 1 -- 172.21.15.163:0/3392358496 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0080b3b50 con 0x7fc008069560 2023-10-09T06:36:01.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.825+0000 7fbffe7fc700 1 -- 172.21.15.163:0/3392358496 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fc004014b70 con 0x7fc008069560 2023-10-09T06:36:01.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.825+0000 7fbffe7fc700 1 --2- 172.21.15.163:0/3392358496 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbfec067170 0x7fbfec069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:01.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.825+0000 7fbffe7fc700 1 -- 172.21.15.163:0/3392358496 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc004084fe0 con 0x7fc008069560 2023-10-09T06:36:01.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.825+0000 7fc00d815700 1 --2- 172.21.15.163:0/3392358496 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbfec067170 0x7fbfec069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:01.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.826+0000 7fc00d815700 1 --2- 172.21.15.163:0/3392358496 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbfec067170 0x7fbfec069620 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc008155000 tx=0x7fbff8007cb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:01.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:01.829+0000 7fbffe7fc700 1 -- 172.21.15.163:0/3392358496 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc004051fc0 con 0x7fc008069560 2023-10-09T06:36:02.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.038+0000 7fc01027a700 1 -- 172.21.15.163:0/3392358496 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0080cac50 con 0x7fbfec067170 2023-10-09T06:36:02.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.041+0000 7fbffe7fc700 1 -- 172.21.15.163:0/3392358496 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fc0080cac50 con 0x7fbfec067170 2023-10-09T06:36:02.041 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:36:02.041 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:02.041 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "cephfs" 2023-10-09T06:36:02.041 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:02.041 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:36:02.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.042+0000 7fc01027a700 1 -- 172.21.15.163:0/3392358496 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbfec067170 msgr2=0x7fbfec069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:02.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.042+0000 7fc01027a700 1 --2- 172.21.15.163:0/3392358496 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbfec067170 0x7fbfec069620 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc008155000 tx=0x7fbff8007cb0 comp rx=0 tx=0).stop 2023-10-09T06:36:02.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.042+0000 7fc01027a700 1 -- 172.21.15.163:0/3392358496 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc008069560 msgr2=0x7fc00815c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:02.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.042+0000 7fc01027a700 1 --2- 172.21.15.163:0/3392358496 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc008069560 0x7fc00815c790 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc00400e1c0 tx=0x7fc004009920 comp rx=0 tx=0).stop 2023-10-09T06:36:02.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.042+0000 7fc01027a700 1 -- 172.21.15.163:0/3392358496 shutdown_connections 2023-10-09T06:36:02.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.042+0000 7fc01027a700 1 --2- 172.21.15.163:0/3392358496 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbfec067170 0x7fbfec069620 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:02.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.042+0000 7fc01027a700 1 --2- 172.21.15.163:0/3392358496 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc008069560 0x7fc00815c790 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:02.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.042+0000 7fc01027a700 1 --2- 172.21.15.163:0/3392358496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc008159c60 0x7fc00816a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:02.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.042+0000 7fc01027a700 1 --2- 172.21.15.163:0/3392358496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc008159890 0x7fc008153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:02.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.042+0000 7fc01027a700 1 -- 172.21.15.163:0/3392358496 >> 172.21.15.163:0/3392358496 conn(0x7fc008068e20 msgr2=0x7fc008155180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:02.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.042+0000 7fc01027a700 1 -- 172.21.15.163:0/3392358496 shutdown_connections 2023-10-09T06:36:02.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.042+0000 7fc01027a700 1 -- 172.21.15.163:0/3392358496 wait complete. 2023-10-09T06:36:02.056 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-09T06:36:02.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.172+0000 7f3c00335700 1 Processor -- start 2023-10-09T06:36:02.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.172+0000 7f3c00335700 1 -- start start 2023-10-09T06:36:02.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.173+0000 7f3c00335700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3bf8069560 0x7f3bf80699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:02.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.173+0000 7f3c00335700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf8069ee0 con 0x7f3bf8159c60 2023-10-09T06:36:02.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.173+0000 7f3c00335700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf815a160 con 0x7f3bf8159890 2023-10-09T06:36:02.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.173+0000 7f3c00335700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bf80d13a0 con 0x7f3bf8069560 2023-10-09T06:36:02.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.173+0000 7f3bfd8d0700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f3bf8159c60 0x7f3bf815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60632/0 (socket says 172.21.15.163:60632) 2023-10-09T06:36:02.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.173+0000 7f3bfd8d0700 1 -- 172.21.15.163:0/3393924109 learned_addr learned my addr 172.21.15.163:0/3393924109 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:02.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.173+0000 7f3bfe8d2700 1 --2- 172.21.15.163:0/3393924109 >> v2:172.21.15.181:3300/0 conn(0x7f3bf8069560 0x7f3bf80699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:02.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.173+0000 7f3bfd0cf700 1 -- 172.21.15.163:0/3393924109 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 723842318 0 0) 0x7f3bf8069ee0 con 0x7f3bf8159c60 2023-10-09T06:36:02.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.173+0000 7f3bfd0cf700 1 -- 172.21.15.163:0/3393924109 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3be4003570 con 0x7f3bf8159c60 2023-10-09T06:36:02.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.173+0000 7f3bfd0cf700 1 -- 172.21.15.163:0/3393924109 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 616471904 0 0) 0x7f3bf815a160 con 0x7f3bf8159890 2023-10-09T06:36:02.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.173+0000 7f3bfe8d2700 1 -- 172.21.15.163:0/3393924109 >> v1:172.21.15.181:6789/0 conn(0x7f3bf8159890 legacy=0x7f3bf815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:02.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.173+0000 7f3bfe8d2700 1 -- 172.21.15.163:0/3393924109 >> v1:172.21.15.163:6789/0 conn(0x7f3bf8159c60 legacy=0x7f3bf815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:02.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.173+0000 7f3bfe8d2700 1 -- 172.21.15.163:0/3393924109 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf815e140 con 0x7f3bf8069560 2023-10-09T06:36:02.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.173+0000 7f3bfe8d2700 1 --2- 172.21.15.163:0/3393924109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bf8069560 0x7f3bf80699a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0009e60 tx=0x7f3bf0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ec17e5d0799d7bc server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:02.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.174+0000 7f3bfd0cf700 1 -- 172.21.15.163:0/3393924109 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf000e070 con 0x7f3bf8069560 2023-10-09T06:36:02.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.174+0000 7f3bfd0cf700 1 -- 172.21.15.163:0/3393924109 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bf000bb60 con 0x7f3bf8069560 2023-10-09T06:36:02.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.174+0000 7f3bfd0cf700 1 -- 172.21.15.163:0/3393924109 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf0012430 con 0x7f3bf8069560 2023-10-09T06:36:02.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.174+0000 7f3c00335700 1 -- 172.21.15.163:0/3393924109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bf8069560 msgr2=0x7f3bf80699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:02.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.174+0000 7f3c00335700 1 --2- 172.21.15.163:0/3393924109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bf8069560 0x7f3bf80699a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0009e60 tx=0x7f3bf0009780 comp rx=0 tx=0).stop 2023-10-09T06:36:02.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.174+0000 7f3c00335700 1 -- 172.21.15.163:0/3393924109 shutdown_connections 2023-10-09T06:36:02.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.174+0000 7f3c00335700 1 --2- 172.21.15.163:0/3393924109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bf8069560 0x7f3bf80699a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:02.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.174+0000 7f3c00335700 1 -- 172.21.15.163:0/3393924109 >> 172.21.15.163:0/3393924109 conn(0x7f3bf8068e20 msgr2=0x7f3bf80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:02.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.174+0000 7f3c00335700 1 -- 172.21.15.163:0/3393924109 shutdown_connections 2023-10-09T06:36:02.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.174+0000 7f3c00335700 1 -- 172.21.15.163:0/3393924109 wait complete. 2023-10-09T06:36:02.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.175+0000 7f3c00335700 1 Processor -- start 2023-10-09T06:36:02.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.175+0000 7f3c00335700 1 -- start start 2023-10-09T06:36:02.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.176+0000 7f3c00335700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bf8069560 0x7f3bf80ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:02.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.176+0000 7f3c00335700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bf8159890 0x7f3bf81472f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:02.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.176+0000 7f3c00335700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bf8159c60 0x7f3bf815b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:02.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.176+0000 7f3c00335700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf80cfec0 con 0x7f3bf8069560 2023-10-09T06:36:02.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.176+0000 7f3c00335700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf8147830 con 0x7f3bf8159c60 2023-10-09T06:36:02.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.176+0000 7f3c00335700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf80c9e30 con 0x7f3bf8159890 2023-10-09T06:36:02.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.176+0000 7f3bfe0d1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bf8069560 0x7f3bf80ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:02.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.176+0000 7f3bfe0d1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bf8069560 0x7f3bf80ca2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40486/0 (socket says 172.21.15.163:40486) 2023-10-09T06:36:02.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.176+0000 7f3bfe0d1700 1 -- 172.21.15.163:0/1456097592 learned_addr learned my addr 172.21.15.163:0/1456097592 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:02.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.176+0000 7f3bfe8d2700 1 --2- 172.21.15.163:0/1456097592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bf8159c60 0x7f3bf815b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:02.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.176+0000 7f3bfd8d0700 1 --2- 172.21.15.163:0/1456097592 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bf8159890 0x7f3bf81472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:02.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.177+0000 7f3bfe0d1700 1 -- 172.21.15.163:0/1456097592 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bf8159890 msgr2=0x7f3bf81472f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:02.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.177+0000 7f3bfe0d1700 1 --2- 172.21.15.163:0/1456097592 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bf8159890 0x7f3bf81472f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:02.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.177+0000 7f3bfe0d1700 1 -- 172.21.15.163:0/1456097592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bf8159c60 msgr2=0x7f3bf815b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:02.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.177+0000 7f3bfe0d1700 1 --2- 172.21.15.163:0/1456097592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bf8159c60 0x7f3bf815b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:02.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.177+0000 7f3bfe0d1700 1 -- 172.21.15.163:0/1456097592 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf0009a20 con 0x7f3bf8069560 2023-10-09T06:36:02.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.177+0000 7f3bfe0d1700 1 --2- 172.21.15.163:0/1456097592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bf8069560 0x7f3bf80ca2c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f3bf400c7e0 tx=0x7f3bf4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:02.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.177+0000 7f3bee7fc700 1 -- 172.21.15.163:0/1456097592 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4012700 con 0x7f3bf8069560 2023-10-09T06:36:02.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.177+0000 7f3bee7fc700 1 -- 172.21.15.163:0/1456097592 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bf4012880 con 0x7f3bf8069560 2023-10-09T06:36:02.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.177+0000 7f3c00335700 1 -- 172.21.15.163:0/1456097592 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf80cf900 con 0x7f3bf8069560 2023-10-09T06:36:02.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.177+0000 7f3bee7fc700 1 -- 172.21.15.163:0/1456097592 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4012c40 con 0x7f3bf8069560 2023-10-09T06:36:02.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.177+0000 7f3c00335700 1 -- 172.21.15.163:0/1456097592 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf80c9ac0 con 0x7f3bf8069560 2023-10-09T06:36:02.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.178+0000 7f3c00335700 1 -- 172.21.15.163:0/1456097592 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf80ca110 con 0x7f3bf8069560 2023-10-09T06:36:02.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.178+0000 7f3bee7fc700 1 -- 172.21.15.163:0/1456097592 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f3bf401b400 con 0x7f3bf8069560 2023-10-09T06:36:02.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.179+0000 7f3bee7fc700 1 --2- 172.21.15.163:0/1456097592 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3bdc067170 0x7f3bdc069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:02.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.179+0000 7f3bee7fc700 1 -- 172.21.15.163:0/1456097592 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3bf4084450 con 0x7f3bf8069560 2023-10-09T06:36:02.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.179+0000 7f3bfd8d0700 1 --2- 172.21.15.163:0/1456097592 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3bdc067170 0x7f3bdc069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:02.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.179+0000 7f3bfd8d0700 1 --2- 172.21.15.163:0/1456097592 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3bdc067170 0x7f3bdc069620 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3bf816d170 tx=0x7f3be8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:02.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.183+0000 7f3bee7fc700 1 -- 172.21.15.163:0/1456097592 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3bf4051430 con 0x7f3bf8069560 2023-10-09T06:36:02.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.365+0000 7f3c00335700 1 -- 172.21.15.163:0/1456097592 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3bf804c3c0 con 0x7f3bf8069560 2023-10-09T06:36:02.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.366+0000 7f3bee7fc700 1 -- 172.21.15.163:0/1456097592 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3bf4054a50 con 0x7f3bf8069560 2023-10-09T06:36:02.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.368+0000 7f3c00335700 1 -- 172.21.15.163:0/1456097592 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3bdc067170 msgr2=0x7f3bdc069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:02.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.368+0000 7f3c00335700 1 --2- 172.21.15.163:0/1456097592 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3bdc067170 0x7f3bdc069620 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3bf816d170 tx=0x7f3be8009040 comp rx=0 tx=0).stop 2023-10-09T06:36:02.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.368+0000 7f3c00335700 1 -- 172.21.15.163:0/1456097592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bf8069560 msgr2=0x7f3bf80ca2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:02.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.368+0000 7f3c00335700 1 --2- 172.21.15.163:0/1456097592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bf8069560 0x7f3bf80ca2c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f3bf400c7e0 tx=0x7f3bf4007df0 comp rx=0 tx=0).stop 2023-10-09T06:36:02.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.368+0000 7f3c00335700 1 -- 172.21.15.163:0/1456097592 shutdown_connections 2023-10-09T06:36:02.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.368+0000 7f3c00335700 1 --2- 172.21.15.163:0/1456097592 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3bdc067170 0x7f3bdc069620 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:02.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.368+0000 7f3c00335700 1 --2- 172.21.15.163:0/1456097592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bf8069560 0x7f3bf80ca2c0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:02.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.368+0000 7f3c00335700 1 --2- 172.21.15.163:0/1456097592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bf8159c60 0x7f3bf815b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:02.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.368+0000 7f3c00335700 1 --2- 172.21.15.163:0/1456097592 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bf8159890 0x7f3bf81472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:02.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.368+0000 7f3c00335700 1 -- 172.21.15.163:0/1456097592 >> 172.21.15.163:0/1456097592 conn(0x7f3bf8068e20 msgr2=0x7f3bf80cadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:02.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.368+0000 7f3c00335700 1 -- 172.21.15.163:0/1456097592 shutdown_connections 2023-10-09T06:36:02.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.368+0000 7f3c00335700 1 -- 172.21.15.163:0/1456097592 wait complete. 2023-10-09T06:36:02.381 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs group_0000000001000982 2023-10-09T06:36:02.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.497+0000 7f2fc4918700 1 Processor -- start 2023-10-09T06:36:02.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.497+0000 7f2fc4918700 1 -- start start 2023-10-09T06:36:02.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.498+0000 7f2fc4918700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2fc0069560 0x7f2fc00699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:02.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.498+0000 7f2fc4918700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fc0069ee0 con 0x7f2fc0159890 2023-10-09T06:36:02.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.498+0000 7f2fc4918700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fc015a160 con 0x7f2fc0159c60 2023-10-09T06:36:02.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.498+0000 7f2fc4918700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc00d13a0 con 0x7f2fc0069560 2023-10-09T06:36:02.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.498+0000 7f2fbed9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2fc0069560 0x7f2fc00699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:02.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.498+0000 7f2fbe59c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f2fc0159890 0x7f2fc015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60642/0 (socket says 172.21.15.163:60642) 2023-10-09T06:36:02.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.498+0000 7f2fbed9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2fc0069560 0x7f2fc00699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40500/0 (socket says 172.21.15.163:40500) 2023-10-09T06:36:02.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.498+0000 7f2fbe59c700 1 -- 172.21.15.163:0/335658349 learned_addr learned my addr 172.21.15.163:0/335658349 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:02.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.498+0000 7f2fbed9d700 1 -- 172.21.15.163:0/335658349 >> v1:172.21.15.181:6789/0 conn(0x7f2fc0159c60 legacy=0x7f2fc015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:02.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.498+0000 7f2fbed9d700 1 -- 172.21.15.163:0/335658349 >> v1:172.21.15.163:6789/0 conn(0x7f2fc0159890 legacy=0x7f2fc015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:02.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.498+0000 7f2fbed9d700 1 -- 172.21.15.163:0/335658349 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc015e140 con 0x7f2fc0069560 2023-10-09T06:36:02.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.498+0000 7f2fbdd9b700 1 -- 172.21.15.163:0/335658349 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 148315213 0 0) 0x7f2fc0069ee0 con 0x7f2fc0159890 2023-10-09T06:36:02.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.499+0000 7f2fbed9d700 1 --2- 172.21.15.163:0/335658349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fc0069560 0x7f2fc00699a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8009e60 tx=0x7f2fb8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c344ffc0c3baa9 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:02.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.499+0000 7f2fbdd9b700 1 -- 172.21.15.163:0/335658349 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb800e070 con 0x7f2fc0069560 2023-10-09T06:36:02.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.499+0000 7f2fbdd9b700 1 -- 172.21.15.163:0/335658349 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb800bc30 con 0x7f2fc0069560 2023-10-09T06:36:02.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.499+0000 7f2fbdd9b700 1 -- 172.21.15.163:0/335658349 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8012430 con 0x7f2fc0069560 2023-10-09T06:36:02.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.499+0000 7f2fc4918700 1 -- 172.21.15.163:0/335658349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fc0069560 msgr2=0x7f2fc00699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:02.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.499+0000 7f2fc4918700 1 --2- 172.21.15.163:0/335658349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fc0069560 0x7f2fc00699a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8009e60 tx=0x7f2fb8009780 comp rx=0 tx=0).stop 2023-10-09T06:36:02.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.499+0000 7f2fc4918700 1 -- 172.21.15.163:0/335658349 shutdown_connections 2023-10-09T06:36:02.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.499+0000 7f2fc4918700 1 --2- 172.21.15.163:0/335658349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fc0069560 0x7f2fc00699a0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:02.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.499+0000 7f2fc4918700 1 -- 172.21.15.163:0/335658349 >> 172.21.15.163:0/335658349 conn(0x7f2fc0068e20 msgr2=0x7f2fc00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:02.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.499+0000 7f2fc4918700 1 -- 172.21.15.163:0/335658349 shutdown_connections 2023-10-09T06:36:02.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.499+0000 7f2fc4918700 1 -- 172.21.15.163:0/335658349 wait complete. 2023-10-09T06:36:02.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.501+0000 7f2fc4918700 1 Processor -- start 2023-10-09T06:36:02.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.501+0000 7f2fc4918700 1 -- start start 2023-10-09T06:36:02.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.501+0000 7f2fc4918700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fc0069560 0x7f2fc0149340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:02.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.501+0000 7f2fc4918700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fc0159890 0x7f2fc00b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:02.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.501+0000 7f2fc4918700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fc0159c60 0x7f2fc00c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:02.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.501+0000 7f2fc4918700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc0149880 con 0x7f2fc0159890 2023-10-09T06:36:02.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.501+0000 7f2fc4918700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc00cd0c0 con 0x7f2fc0069560 2023-10-09T06:36:02.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.501+0000 7f2fc4918700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc00c8320 con 0x7f2fc0159c60 2023-10-09T06:36:02.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.501+0000 7f2fb5d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fc0159890 0x7f2fc00b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:02.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.501+0000 7f2fb5d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fc0159890 0x7f2fc00b3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40506/0 (socket says 172.21.15.163:40506) 2023-10-09T06:36:02.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.501+0000 7f2fb5d9b700 1 -- 172.21.15.163:0/3818854112 learned_addr learned my addr 172.21.15.163:0/3818854112 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:02.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.502+0000 7f2fbe59c700 1 --2- 172.21.15.163:0/3818854112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fc0069560 0x7f2fc0149340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:02.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.502+0000 7f2fbed9d700 1 --2- 172.21.15.163:0/3818854112 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fc0159c60 0x7f2fc00c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:02.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.502+0000 7f2fb5d9b700 1 -- 172.21.15.163:0/3818854112 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fc0159c60 msgr2=0x7f2fc00c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:02.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.502+0000 7f2fb5d9b700 1 --2- 172.21.15.163:0/3818854112 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fc0159c60 0x7f2fc00c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:02.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.502+0000 7f2fb5d9b700 1 -- 172.21.15.163:0/3818854112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fc0069560 msgr2=0x7f2fc0149340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:02.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.502+0000 7f2fb5d9b700 1 --2- 172.21.15.163:0/3818854112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fc0069560 0x7f2fc0149340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:02.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.502+0000 7f2fb5d9b700 1 -- 172.21.15.163:0/3818854112 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb8009a20 con 0x7f2fc0159890 2023-10-09T06:36:02.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.502+0000 7f2fbed9d700 1 --2- 172.21.15.163:0/3818854112 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fc0159c60 0x7f2fc00c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:02.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.502+0000 7f2fb5d9b700 1 --2- 172.21.15.163:0/3818854112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fc0159890 0x7f2fc00b3b30 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800c590 tx=0x7f2fa8007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:02.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.502+0000 7f2fb6ffd700 1 -- 172.21.15.163:0/3818854112 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8011590 con 0x7f2fc0159890 2023-10-09T06:36:02.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.502+0000 7f2fb6ffd700 1 -- 172.21.15.163:0/3818854112 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa8011730 con 0x7f2fc0159890 2023-10-09T06:36:02.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.502+0000 7f2fc4918700 1 -- 172.21.15.163:0/3818854112 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc00c8600 con 0x7f2fc0159890 2023-10-09T06:36:02.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.502+0000 7f2fb6ffd700 1 -- 172.21.15.163:0/3818854112 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8011af0 con 0x7f2fc0159890 2023-10-09T06:36:02.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.502+0000 7f2fc4918700 1 -- 172.21.15.163:0/3818854112 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc00cd310 con 0x7f2fc0159890 2023-10-09T06:36:02.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.503+0000 7f2fc4918700 1 -- 172.21.15.163:0/3818854112 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fc00cd530 con 0x7f2fc0159890 2023-10-09T06:36:02.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.503+0000 7f2fb6ffd700 1 -- 172.21.15.163:0/3818854112 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f2fa8012420 con 0x7f2fc0159890 2023-10-09T06:36:02.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.504+0000 7f2fb6ffd700 1 --2- 172.21.15.163:0/3818854112 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2fa0067170 0x7f2fa0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:02.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.504+0000 7f2fb6ffd700 1 -- 172.21.15.163:0/3818854112 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2fa8083b10 con 0x7f2fc0159890 2023-10-09T06:36:02.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.504+0000 7f2fbe59c700 1 --2- 172.21.15.163:0/3818854112 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2fa0067170 0x7f2fa0069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:02.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.504+0000 7f2fbe59c700 1 --2- 172.21.15.163:0/3818854112 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2fa0067170 0x7f2fa0069620 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0007fd0 tx=0x7f2fb0007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:02.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.508+0000 7f2fb6ffd700 1 -- 172.21.15.163:0/3818854112 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2fa804b5a0 con 0x7f2fc0159890 2023-10-09T06:36:02.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.720+0000 7f2fc4918700 1 -- 172.21.15.163:0/3818854112 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "group_0000000001000982", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fc004c3c0 con 0x7f2fa0067170 2023-10-09T06:36:02.727 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:36:02.727+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:36:02.727 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:36:02.727+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:36:02.727 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:36:02.727+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:36:02.727 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:36:02.727+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:36:02.727 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:36:02.727+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:36:02.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.741+0000 7f2fb6ffd700 1 -- 172.21.15.163:0/3818854112 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2fc004c3c0 con 0x7f2fa0067170 2023-10-09T06:36:02.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.743+0000 7f2fc4918700 1 -- 172.21.15.163:0/3818854112 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2fa0067170 msgr2=0x7f2fa0069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:02.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.743+0000 7f2fc4918700 1 --2- 172.21.15.163:0/3818854112 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2fa0067170 0x7f2fa0069620 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0007fd0 tx=0x7f2fb0007d30 comp rx=0 tx=0).stop 2023-10-09T06:36:02.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.743+0000 7f2fc4918700 1 -- 172.21.15.163:0/3818854112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fc0159890 msgr2=0x7f2fc00b3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:02.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.743+0000 7f2fc4918700 1 --2- 172.21.15.163:0/3818854112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fc0159890 0x7f2fc00b3b30 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800c590 tx=0x7f2fa8007f30 comp rx=0 tx=0).stop 2023-10-09T06:36:02.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.743+0000 7f2fc4918700 1 -- 172.21.15.163:0/3818854112 shutdown_connections 2023-10-09T06:36:02.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.743+0000 7f2fc4918700 1 --2- 172.21.15.163:0/3818854112 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2fa0067170 0x7f2fa0069620 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:02.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.743+0000 7f2fc4918700 1 --2- 172.21.15.163:0/3818854112 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fc0159c60 0x7f2fc00c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:02.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.743+0000 7f2fc4918700 1 --2- 172.21.15.163:0/3818854112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fc0069560 0x7f2fc0149340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:02.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.743+0000 7f2fc4918700 1 --2- 172.21.15.163:0/3818854112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fc0159890 0x7f2fc00b3b30 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:02.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.743+0000 7f2fc4918700 1 -- 172.21.15.163:0/3818854112 >> 172.21.15.163:0/3818854112 conn(0x7f2fc0068e20 msgr2=0x7f2fc01553e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:02.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.743+0000 7f2fc4918700 1 -- 172.21.15.163:0/3818854112 shutdown_connections 2023-10-09T06:36:02.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.743+0000 7f2fc4918700 1 -- 172.21.15.163:0/3818854112 wait complete. 2023-10-09T06:36:02.756 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume info cephfs --human_readable 2023-10-09T06:36:02.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.872+0000 7f57fd77b700 1 Processor -- start 2023-10-09T06:36:02.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.872+0000 7f57fd77b700 1 -- start start 2023-10-09T06:36:02.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.873+0000 7f57fd77b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f57f8159c60 0x7f57f815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:02.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.873+0000 7f57fd77b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f815a160 con 0x7f57f815db10 2023-10-09T06:36:02.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.873+0000 7f57fd77b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f80d13a0 con 0x7f57f8159890 2023-10-09T06:36:02.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.873+0000 7f57fd77b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57f815df50 con 0x7f57f8159c60 2023-10-09T06:36:02.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.873+0000 7f57f77fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f57f815db10 0x7f57f8069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60658/0 (socket says 172.21.15.163:60658) 2023-10-09T06:36:02.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.873+0000 7f57f77fe700 1 -- 172.21.15.163:0/670590296 learned_addr learned my addr 172.21.15.163:0/670590296 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:02.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.873+0000 7f57ee7fc700 1 --2- 172.21.15.163:0/670590296 >> v2:172.21.15.181:3300/0 conn(0x7f57f8159c60 0x7f57f815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:02.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.873+0000 7f57f67fc700 1 -- 172.21.15.163:0/670590296 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1096882951 0 0) 0x7f57f815a160 con 0x7f57f815db10 2023-10-09T06:36:02.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.873+0000 7f57f67fc700 1 -- 172.21.15.163:0/670590296 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57e4003570 con 0x7f57f815db10 2023-10-09T06:36:02.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.874+0000 7f57f67fc700 1 -- 172.21.15.163:0/670590296 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2331778973 0 0) 0x7f57e4003570 con 0x7f57f815db10 2023-10-09T06:36:02.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.874+0000 7f57ee7fc700 1 -- 172.21.15.163:0/670590296 >> v1:172.21.15.181:6789/0 conn(0x7f57f8159890 legacy=0x7f57f815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:02.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.874+0000 7f57ee7fc700 1 -- 172.21.15.163:0/670590296 >> v1:172.21.15.163:6789/0 conn(0x7f57f815db10 legacy=0x7f57f8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:02.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.874+0000 7f57ee7fc700 1 -- 172.21.15.163:0/670590296 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f815e140 con 0x7f57f8159c60 2023-10-09T06:36:02.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.874+0000 7f57ee7fc700 1 --2- 172.21.15.163:0/670590296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57f8159c60 0x7f57f815d5d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f57e00098e0 tx=0x7f57e00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eebfdbab30e1c776 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:02.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.874+0000 7f57f67fc700 1 -- 172.21.15.163:0/670590296 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f00017f0 con 0x7f57f8159c60 2023-10-09T06:36:02.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.874+0000 7f57f67fc700 1 -- 172.21.15.163:0/670590296 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e0011420 con 0x7f57f8159c60 2023-10-09T06:36:02.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.874+0000 7f57f67fc700 1 -- 172.21.15.163:0/670590296 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e00117c0 con 0x7f57f8159c60 2023-10-09T06:36:02.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.875+0000 7f57fd77b700 1 -- 172.21.15.163:0/670590296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57f8159c60 msgr2=0x7f57f815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:02.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.875+0000 7f57fd77b700 1 --2- 172.21.15.163:0/670590296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57f8159c60 0x7f57f815d5d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f57e00098e0 tx=0x7f57e00091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:02.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.875+0000 7f57fd77b700 1 -- 172.21.15.163:0/670590296 shutdown_connections 2023-10-09T06:36:02.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.875+0000 7f57fd77b700 1 --2- 172.21.15.163:0/670590296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57f8159c60 0x7f57f815d5d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:02.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.875+0000 7f57fd77b700 1 -- 172.21.15.163:0/670590296 >> 172.21.15.163:0/670590296 conn(0x7f57f8068e20 msgr2=0x7f57f80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:02.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.875+0000 7f57fd77b700 1 -- 172.21.15.163:0/670590296 shutdown_connections 2023-10-09T06:36:02.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.875+0000 7f57fd77b700 1 -- 172.21.15.163:0/670590296 wait complete. 2023-10-09T06:36:02.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.876+0000 7f57fd77b700 1 Processor -- start 2023-10-09T06:36:02.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.876+0000 7f57fd77b700 1 -- start start 2023-10-09T06:36:02.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.877+0000 7f57fd77b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57f8159890 0x7f57f80ce320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:02.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.877+0000 7f57fd77b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57f8159c60 0x7f57f80b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:02.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.877+0000 7f57fd77b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57f815db10 0x7f57f81452f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:02.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.877+0000 7f57fd77b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f80b3e20 con 0x7f57f8159c60 2023-10-09T06:36:02.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.877+0000 7f57fd77b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f8145830 con 0x7f57f8159890 2023-10-09T06:36:02.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.877+0000 7f57fd77b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f80cf610 con 0x7f57f815db10 2023-10-09T06:36:02.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.877+0000 7f57ee7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57f8159c60 0x7f57f80b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:02.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.877+0000 7f57ee7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57f8159c60 0x7f57f80b38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40514/0 (socket says 172.21.15.163:40514) 2023-10-09T06:36:02.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.877+0000 7f57ee7fc700 1 -- 172.21.15.163:0/292725156 learned_addr learned my addr 172.21.15.163:0/292725156 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:02.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.877+0000 7f57f6ffd700 1 --2- 172.21.15.163:0/292725156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57f8159890 0x7f57f80ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:02.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.877+0000 7f57f77fe700 1 --2- 172.21.15.163:0/292725156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57f815db10 0x7f57f81452f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:02.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.877+0000 7f57ee7fc700 1 -- 172.21.15.163:0/292725156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57f815db10 msgr2=0x7f57f81452f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:02.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.877+0000 7f57ee7fc700 1 --2- 172.21.15.163:0/292725156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57f815db10 0x7f57f81452f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:02.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.877+0000 7f57ee7fc700 1 -- 172.21.15.163:0/292725156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57f8159890 msgr2=0x7f57f80ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:02.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.877+0000 7f57ee7fc700 1 --2- 172.21.15.163:0/292725156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57f8159890 0x7f57f80ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:02.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.877+0000 7f57ee7fc700 1 -- 172.21.15.163:0/292725156 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e0009420 con 0x7f57f8159c60 2023-10-09T06:36:02.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.877+0000 7f57ee7fc700 1 --2- 172.21.15.163:0/292725156 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57f8159c60 0x7f57f80b38e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f57f8070e20 tx=0x7f57e0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:02.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.877+0000 7f57ef7fe700 1 -- 172.21.15.163:0/292725156 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e0010a80 con 0x7f57f8159c60 2023-10-09T06:36:02.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.877+0000 7f57ef7fe700 1 -- 172.21.15.163:0/292725156 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e0010c00 con 0x7f57f8159c60 2023-10-09T06:36:02.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.877+0000 7f57fd77b700 1 -- 172.21.15.163:0/292725156 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f80cea00 con 0x7f57f8159c60 2023-10-09T06:36:02.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.877+0000 7f57ef7fe700 1 -- 172.21.15.163:0/292725156 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e001d430 con 0x7f57f8159c60 2023-10-09T06:36:02.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.878+0000 7f57fd77b700 1 -- 172.21.15.163:0/292725156 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f80cefc0 con 0x7f57f8159c60 2023-10-09T06:36:02.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.878+0000 7f57fd77b700 1 -- 172.21.15.163:0/292725156 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f804c3c0 con 0x7f57f8159c60 2023-10-09T06:36:02.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.879+0000 7f57ef7fe700 1 -- 172.21.15.163:0/292725156 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f57e001d7a0 con 0x7f57f8159c60 2023-10-09T06:36:02.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.879+0000 7f57ef7fe700 1 --2- 172.21.15.163:0/292725156 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f57d8067170 0x7f57d8069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:02.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.879+0000 7f57ef7fe700 1 -- 172.21.15.163:0/292725156 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f57e0086590 con 0x7f57f8159c60 2023-10-09T06:36:02.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.879+0000 7f57f6ffd700 1 --2- 172.21.15.163:0/292725156 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f57d8067170 0x7f57d8069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:02.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.880+0000 7f57f6ffd700 1 --2- 172.21.15.163:0/292725156 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f57d8067170 0x7f57d8069620 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f57e8003ae0 tx=0x7f57e8007ec0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:02.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:02.883+0000 7f57ef7fe700 1 -- 172.21.15.163:0/292725156 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f57e0053570 con 0x7f57f8159c60 2023-10-09T06:36:03.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.092+0000 7f57fd77b700 1 -- 172.21.15.163:0/292725156 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume info", "vol_name": "cephfs", "human_readable": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f57f80cf750 con 0x7f57d8067170 2023-10-09T06:36:03.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.095+0000 7f57ef7fe700 1 -- 172.21.15.163:0/292725156 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+535 (secure 0 0 0) 0x7f57f80cf750 con 0x7f57d8067170 2023-10-09T06:36:03.096 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:36:03.096 INFO:teuthology.orchestra.run.smithi163.stdout: "mon_addrs": [ 2023-10-09T06:36:03.096 INFO:teuthology.orchestra.run.smithi163.stdout: "172.21.15.163:6789", 2023-10-09T06:36:03.096 INFO:teuthology.orchestra.run.smithi163.stdout: "172.21.15.181:6789", 2023-10-09T06:36:03.096 INFO:teuthology.orchestra.run.smithi163.stdout: "172.21.15.181:6790" 2023-10-09T06:36:03.096 INFO:teuthology.orchestra.run.smithi163.stdout: ], 2023-10-09T06:36:03.096 INFO:teuthology.orchestra.run.smithi163.stdout: "pending_subvolume_deletions": 0, 2023-10-09T06:36:03.096 INFO:teuthology.orchestra.run.smithi163.stdout: "pools": { 2023-10-09T06:36:03.097 INFO:teuthology.orchestra.run.smithi163.stdout: "data": [ 2023-10-09T06:36:03.097 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:03.097 INFO:teuthology.orchestra.run.smithi163.stdout: "avail": " 340G", 2023-10-09T06:36:03.097 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "cephfs_data", 2023-10-09T06:36:03.097 INFO:teuthology.orchestra.run.smithi163.stdout: "used": " 0 " 2023-10-09T06:36:03.097 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:03.097 INFO:teuthology.orchestra.run.smithi163.stdout: ], 2023-10-09T06:36:03.097 INFO:teuthology.orchestra.run.smithi163.stdout: "metadata": [ 2023-10-09T06:36:03.097 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:03.097 INFO:teuthology.orchestra.run.smithi163.stdout: "avail": " 340G", 2023-10-09T06:36:03.097 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "cephfs_metadata", 2023-10-09T06:36:03.097 INFO:teuthology.orchestra.run.smithi163.stdout: "used": "64.0k" 2023-10-09T06:36:03.097 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:03.098 INFO:teuthology.orchestra.run.smithi163.stdout: ] 2023-10-09T06:36:03.098 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:03.098 INFO:teuthology.orchestra.run.smithi163.stdout: "used_size": " 0 " 2023-10-09T06:36:03.098 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:36:03.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.097+0000 7f57fd77b700 1 -- 172.21.15.163:0/292725156 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f57d8067170 msgr2=0x7f57d8069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:03.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.097+0000 7f57fd77b700 1 --2- 172.21.15.163:0/292725156 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f57d8067170 0x7f57d8069620 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f57e8003ae0 tx=0x7f57e8007ec0 comp rx=0 tx=0).stop 2023-10-09T06:36:03.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.097+0000 7f57fd77b700 1 -- 172.21.15.163:0/292725156 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57f8159c60 msgr2=0x7f57f80b38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:03.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.097+0000 7f57fd77b700 1 --2- 172.21.15.163:0/292725156 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57f8159c60 0x7f57f80b38e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f57f8070e20 tx=0x7f57e0017000 comp rx=0 tx=0).stop 2023-10-09T06:36:03.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.097+0000 7f57fd77b700 1 -- 172.21.15.163:0/292725156 shutdown_connections 2023-10-09T06:36:03.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.097+0000 7f57fd77b700 1 --2- 172.21.15.163:0/292725156 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f57d8067170 0x7f57d8069620 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:03.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.097+0000 7f57fd77b700 1 --2- 172.21.15.163:0/292725156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57f815db10 0x7f57f81452f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:03.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.097+0000 7f57fd77b700 1 --2- 172.21.15.163:0/292725156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57f8159890 0x7f57f80ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:03.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.097+0000 7f57fd77b700 1 --2- 172.21.15.163:0/292725156 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57f8159c60 0x7f57f80b38e0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:03.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.097+0000 7f57fd77b700 1 -- 172.21.15.163:0/292725156 >> 172.21.15.163:0/292725156 conn(0x7f57f8068e20 msgr2=0x7f57f81546d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:03.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.097+0000 7f57fd77b700 1 -- 172.21.15.163:0/292725156 shutdown_connections 2023-10-09T06:36:03.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.097+0000 7f57fd77b700 1 -- 172.21.15.163:0/292725156 wait complete. 2023-10-09T06:36:03.110 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-09T06:36:03.110 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:36:03.151 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:36:03.200 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-09T06:36:03.200 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-09T06:36:03.201 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:36:03.201 DEBUG:teuthology.orchestra.run.smithi163:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-09T06:36:03.241 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi163.front.sepia.ceph.com 2023-10-09T06:36:03.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:03.242 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-09T06:36:03.296 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-09T06:36:03.297 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:36:03.297 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-09T06:36:03.340 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-09T06:36:03.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:03.340 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-09T06:36:03.399 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:36:03.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.513+0000 7f68b1728700 1 Processor -- start 2023-10-09T06:36:03.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.513+0000 7f68b1728700 1 -- start start 2023-10-09T06:36:03.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.513+0000 7f68b1728700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f68ac159890 0x7f68ac15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:03.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.513+0000 7f68b1728700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f68ac159c60 0x7f68ac15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:03.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.513+0000 7f68b1728700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68ac15a160 con 0x7f68ac15db10 2023-10-09T06:36:03.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.513+0000 7f68b1728700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68ac15df50 con 0x7f68ac159c60 2023-10-09T06:36:03.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.513+0000 7f68b1728700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68ac0d13a0 con 0x7f68ac159890 2023-10-09T06:36:03.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.514+0000 7f68aa7fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f68ac159c60 0x7f68ac15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:03.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.514+0000 7f68aa7fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f68ac159c60 0x7f68ac15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40530/0 (socket says 172.21.15.163:40530) 2023-10-09T06:36:03.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.514+0000 7f68aa7fc700 1 -- 172.21.15.163:0/3604799349 learned_addr learned my addr 172.21.15.163:0/3604799349 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:03.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.514+0000 7f68aaffd700 1 --2- 172.21.15.163:0/3604799349 >> v2:172.21.15.181:3300/0 conn(0x7f68ac159890 0x7f68ac15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:03.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.514+0000 7f68aa7fc700 1 -- 172.21.15.163:0/3604799349 >> v2:172.21.15.181:3300/0 conn(0x7f68ac159890 msgr2=0x7f68ac15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:03.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.514+0000 7f68aa7fc700 1 --2- 172.21.15.163:0/3604799349 >> v2:172.21.15.181:3300/0 conn(0x7f68ac159890 0x7f68ac15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:03.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.514+0000 7f68aa7fc700 1 -- 172.21.15.163:0/3604799349 >> v1:172.21.15.181:6789/0 conn(0x7f68ac15db10 legacy=0x7f68ac069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:03.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.514+0000 7f68aa7fc700 1 -- 172.21.15.163:0/3604799349 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ac15e140 con 0x7f68ac159c60 2023-10-09T06:36:03.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.514+0000 7f68aa7fc700 1 --2- 172.21.15.163:0/3604799349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68ac159c60 0x7f68ac15d5d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0d1090 tx=0x7f68a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5598ec697f2fb165 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:03.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.514+0000 7f68a9ffb700 1 -- 172.21.15.163:0/3604799349 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a000b820 con 0x7f68ac159c60 2023-10-09T06:36:03.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.514+0000 7f68a9ffb700 1 -- 172.21.15.163:0/3604799349 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a000b980 con 0x7f68ac159c60 2023-10-09T06:36:03.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.514+0000 7f68a9ffb700 1 -- 172.21.15.163:0/3604799349 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a000bd20 con 0x7f68ac159c60 2023-10-09T06:36:03.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.515+0000 7f68b1728700 1 -- 172.21.15.163:0/3604799349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68ac159c60 msgr2=0x7f68ac15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:03.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.515+0000 7f68b1728700 1 --2- 172.21.15.163:0/3604799349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68ac159c60 0x7f68ac15d5d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0d1090 tx=0x7f68a0009250 comp rx=0 tx=0).stop 2023-10-09T06:36:03.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.515+0000 7f68b1728700 1 -- 172.21.15.163:0/3604799349 shutdown_connections 2023-10-09T06:36:03.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.515+0000 7f68b1728700 1 --2- 172.21.15.163:0/3604799349 >> v2:172.21.15.181:3300/0 conn(0x7f68ac159890 0x7f68ac15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:03.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.515+0000 7f68b1728700 1 --2- 172.21.15.163:0/3604799349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68ac159c60 0x7f68ac15d5d0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:03.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.515+0000 7f68b1728700 1 -- 172.21.15.163:0/3604799349 >> 172.21.15.163:0/3604799349 conn(0x7f68ac068e20 msgr2=0x7f68ac0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:03.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.515+0000 7f68b1728700 1 -- 172.21.15.163:0/3604799349 shutdown_connections 2023-10-09T06:36:03.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.515+0000 7f68b1728700 1 -- 172.21.15.163:0/3604799349 wait complete. 2023-10-09T06:36:03.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.516+0000 7f68b1728700 1 Processor -- start 2023-10-09T06:36:03.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.516+0000 7f68b1728700 1 -- start start 2023-10-09T06:36:03.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.517+0000 7f68b1728700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68ac159890 0x7f68ac147420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:03.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.517+0000 7f68b1728700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68ac159c60 0x7f68ac0cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:03.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.517+0000 7f68b1728700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68ac15db10 0x7f68ac0cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:03.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.517+0000 7f68b1728700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac0cfe20 con 0x7f68ac159890 2023-10-09T06:36:03.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.517+0000 7f68b1728700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac0c9600 con 0x7f68ac159c60 2023-10-09T06:36:03.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.517+0000 7f68b1728700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac0cf860 con 0x7f68ac15db10 2023-10-09T06:36:03.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.517+0000 7f68aaffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68ac159890 0x7f68ac147420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:03.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.517+0000 7f68aaffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68ac159890 0x7f68ac147420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40544/0 (socket says 172.21.15.163:40544) 2023-10-09T06:36:03.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.517+0000 7f68aaffd700 1 -- 172.21.15.163:0/4281089846 learned_addr learned my addr 172.21.15.163:0/4281089846 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:03.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.517+0000 7f68aa7fc700 1 --2- 172.21.15.163:0/4281089846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68ac159c60 0x7f68ac0cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:03.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.517+0000 7f68ab7fe700 1 --2- 172.21.15.163:0/4281089846 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68ac15db10 0x7f68ac0cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:03.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.517+0000 7f68aaffd700 1 -- 172.21.15.163:0/4281089846 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68ac15db10 msgr2=0x7f68ac0cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:03.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.517+0000 7f68aaffd700 1 --2- 172.21.15.163:0/4281089846 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68ac15db10 0x7f68ac0cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:03.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.517+0000 7f68aaffd700 1 -- 172.21.15.163:0/4281089846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68ac159c60 msgr2=0x7f68ac0cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:03.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.517+0000 7f68aaffd700 1 --2- 172.21.15.163:0/4281089846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68ac159c60 0x7f68ac0cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:03.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.517+0000 7f68aaffd700 1 -- 172.21.15.163:0/4281089846 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a00094f0 con 0x7f68ac159890 2023-10-09T06:36:03.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.517+0000 7f68aaffd700 1 --2- 172.21.15.163:0/4281089846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68ac159890 0x7f68ac147420 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f689c00c7d0 tx=0x7f689c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:03.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.518+0000 7f6892ffd700 1 -- 172.21.15.163:0/4281089846 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c011600 con 0x7f68ac159890 2023-10-09T06:36:03.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.518+0000 7f6892ffd700 1 -- 172.21.15.163:0/4281089846 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689c011760 con 0x7f68ac159890 2023-10-09T06:36:03.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.518+0000 7f6892ffd700 1 -- 172.21.15.163:0/4281089846 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c011b20 con 0x7f68ac159890 2023-10-09T06:36:03.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.518+0000 7f68b1728700 1 -- 172.21.15.163:0/4281089846 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68ac0cfb40 con 0x7f68ac159890 2023-10-09T06:36:03.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.518+0000 7f68b1728700 1 -- 172.21.15.163:0/4281089846 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68ac0cd600 con 0x7f68ac159890 2023-10-09T06:36:03.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.519+0000 7f68b1728700 1 -- 172.21.15.163:0/4281089846 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68ac0c90f0 con 0x7f68ac159890 2023-10-09T06:36:03.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.519+0000 7f6892ffd700 1 -- 172.21.15.163:0/4281089846 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f689c0193f0 con 0x7f68ac159890 2023-10-09T06:36:03.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.519+0000 7f6892ffd700 1 --2- 172.21.15.163:0/4281089846 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6898067170 0x7f6898069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:03.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.520+0000 7f6892ffd700 1 -- 172.21.15.163:0/4281089846 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f689c084c90 con 0x7f68ac159890 2023-10-09T06:36:03.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.520+0000 7f68aa7fc700 1 --2- 172.21.15.163:0/4281089846 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6898067170 0x7f6898069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:03.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.520+0000 7f68aa7fc700 1 --2- 172.21.15.163:0/4281089846 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6898067170 0x7f6898069620 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0d1090 tx=0x7f68a0016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:03.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.524+0000 7f6892ffd700 1 -- 172.21.15.163:0/4281089846 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68ac0c90f0 con 0x7f68ac159890 2023-10-09T06:36:03.750 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.749+0000 7f68b1728700 1 -- 172.21.15.163:0/4281089846 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68ac0c90f0 con 0x7f68ac159890 2023-10-09T06:36:03.750 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.749+0000 7f6892ffd700 1 -- 172.21.15.163:0/4281089846 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4608 (secure 0 0 0) 0x7f68ac0c90f0 con 0x7f68ac159890 2023-10-09T06:36:03.751 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:03.751 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1946291256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1946291256},{"type":"v1","addr":"172.21.15.181:6837","nonce":1946291256}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5749,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1366951098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1366951098},{"type":"v1","addr":"172.21.15.181:6836","nonce":1366951098}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3619934997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3619934997},{"type":"v1","addr":"172.21.15.163:6834","nonce":3619934997}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:35:52.002698+0000","modified":"2023-10-09T06:35:55.653628+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/234029086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":234029086},{"type":"v1","addr":"172.21.15.163:6835","nonce":234029086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-09T06:36:03.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.752+0000 7f68b1728700 1 -- 172.21.15.163:0/4281089846 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6898067170 msgr2=0x7f6898069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:03.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.752+0000 7f68b1728700 1 --2- 172.21.15.163:0/4281089846 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6898067170 0x7f6898069620 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0d1090 tx=0x7f68a0016000 comp rx=0 tx=0).stop 2023-10-09T06:36:03.754 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.752+0000 7f68b1728700 1 -- 172.21.15.163:0/4281089846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68ac159890 msgr2=0x7f68ac147420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:03.754 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.752+0000 7f68b1728700 1 --2- 172.21.15.163:0/4281089846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68ac159890 0x7f68ac147420 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f689c00c7d0 tx=0x7f689c008040 comp rx=0 tx=0).stop 2023-10-09T06:36:03.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.752+0000 7f68b1728700 1 -- 172.21.15.163:0/4281089846 shutdown_connections 2023-10-09T06:36:03.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.752+0000 7f68b1728700 1 --2- 172.21.15.163:0/4281089846 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6898067170 0x7f6898069620 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:03.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.752+0000 7f68b1728700 1 --2- 172.21.15.163:0/4281089846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68ac159890 0x7f68ac147420 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:03.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.752+0000 7f68b1728700 1 --2- 172.21.15.163:0/4281089846 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68ac15db10 0x7f68ac0cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:03.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.752+0000 7f68b1728700 1 --2- 172.21.15.163:0/4281089846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68ac159c60 0x7f68ac0cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:03.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.752+0000 7f68b1728700 1 -- 172.21.15.163:0/4281089846 >> 172.21.15.163:0/4281089846 conn(0x7f68ac068e20 msgr2=0x7f68ac069530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:03.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.752+0000 7f68b1728700 1 -- 172.21.15.163:0/4281089846 shutdown_connections 2023-10-09T06:36:03.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.752+0000 7f68b1728700 1 -- 172.21.15.163:0/4281089846 wait complete. 2023-10-09T06:36:03.756 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 49 2023-10-09T06:36:03.765 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:36:03.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.879+0000 7f46e59a4700 1 Processor -- start 2023-10-09T06:36:03.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.879+0000 7f46e59a4700 1 -- start start 2023-10-09T06:36:03.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.880+0000 7f46e59a4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46e0141500 0x7f46e00c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:03.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.880+0000 7f46e59a4700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f46e006c0a0 0x7f46e00c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:03.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.880+0000 7f46e59a4700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46e015c2a0 con 0x7f46e006bcd0 2023-10-09T06:36:03.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.880+0000 7f46e59a4700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e00c9880 con 0x7f46e006c0a0 2023-10-09T06:36:03.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.880+0000 7f46e59a4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e015b690 con 0x7f46e0141500 2023-10-09T06:36:03.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.880+0000 7f46deffd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f46e006bcd0 0x7f46e0140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60666/0 (socket says 172.21.15.163:60666) 2023-10-09T06:36:03.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.880+0000 7f46deffd700 1 -- 172.21.15.163:0/3929391986 learned_addr learned my addr 172.21.15.163:0/3929391986 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:03.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.880+0000 7f46df7fe700 1 --2- 172.21.15.163:0/3929391986 >> v2:172.21.15.163:3300/0 conn(0x7f46e006c0a0 0x7f46e00c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:03.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.880+0000 7f46de7fc700 1 -- 172.21.15.163:0/3929391986 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1412332922 0 0) 0x7f46e015c2a0 con 0x7f46e006bcd0 2023-10-09T06:36:03.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.880+0000 7f46de7fc700 1 -- 172.21.15.163:0/3929391986 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46cc003570 con 0x7f46e006bcd0 2023-10-09T06:36:03.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.880+0000 7f46d67fc700 1 --2- 172.21.15.163:0/3929391986 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46e0141500 0x7f46e00c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:03.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.881+0000 7f46df7fe700 1 -- 172.21.15.163:0/3929391986 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46e0141500 msgr2=0x7f46e00c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:03.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.881+0000 7f46df7fe700 1 --2- 172.21.15.163:0/3929391986 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46e0141500 0x7f46e00c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:03.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.881+0000 7f46df7fe700 1 -- 172.21.15.163:0/3929391986 >> v1:172.21.15.163:6789/0 conn(0x7f46e006bcd0 legacy=0x7f46e0140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:03.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.881+0000 7f46df7fe700 1 -- 172.21.15.163:0/3929391986 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e015b970 con 0x7f46e006c0a0 2023-10-09T06:36:03.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.881+0000 7f46df7fe700 1 --2- 172.21.15.163:0/3929391986 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46e006c0a0 0x7f46e00c9340 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009e60 tx=0x7f46d8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47402a046e0dc27c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:03.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.881+0000 7f46de7fc700 1 -- 172.21.15.163:0/3929391986 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d800e070 con 0x7f46e006c0a0 2023-10-09T06:36:03.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.881+0000 7f46de7fc700 1 -- 172.21.15.163:0/3929391986 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d800bb60 con 0x7f46e006c0a0 2023-10-09T06:36:03.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.881+0000 7f46de7fc700 1 -- 172.21.15.163:0/3929391986 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d8012430 con 0x7f46e006c0a0 2023-10-09T06:36:03.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.881+0000 7f46e59a4700 1 -- 172.21.15.163:0/3929391986 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46e006c0a0 msgr2=0x7f46e00c9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:03.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.881+0000 7f46e59a4700 1 --2- 172.21.15.163:0/3929391986 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46e006c0a0 0x7f46e00c9340 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009e60 tx=0x7f46d8009780 comp rx=0 tx=0).stop 2023-10-09T06:36:03.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.881+0000 7f46e59a4700 1 -- 172.21.15.163:0/3929391986 shutdown_connections 2023-10-09T06:36:03.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.881+0000 7f46e59a4700 1 --2- 172.21.15.163:0/3929391986 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46e006c0a0 0x7f46e00c9340 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:03.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.881+0000 7f46e59a4700 1 --2- 172.21.15.163:0/3929391986 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46e0141500 0x7f46e00c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:03.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.881+0000 7f46e59a4700 1 -- 172.21.15.163:0/3929391986 >> 172.21.15.163:0/3929391986 conn(0x7f46e00ce600 msgr2=0x7f46e00cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:03.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.881+0000 7f46e59a4700 1 -- 172.21.15.163:0/3929391986 shutdown_connections 2023-10-09T06:36:03.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.881+0000 7f46e59a4700 1 -- 172.21.15.163:0/3929391986 wait complete. 2023-10-09T06:36:03.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.883+0000 7f46e59a4700 1 Processor -- start 2023-10-09T06:36:03.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.883+0000 7f46e59a4700 1 -- start start 2023-10-09T06:36:03.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.883+0000 7f46e59a4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46e006bcd0 0x7f46e013c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:03.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.883+0000 7f46e59a4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46e006c0a0 0x7f46e00c7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:03.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.883+0000 7f46e59a4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46e0141500 0x7f46e00c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:03.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.883+0000 7f46e59a4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e013c860 con 0x7f46e0141500 2023-10-09T06:36:03.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.883+0000 7f46e59a4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e00b1fb0 con 0x7f46e006c0a0 2023-10-09T06:36:03.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.883+0000 7f46e59a4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e00b13a0 con 0x7f46e006bcd0 2023-10-09T06:36:03.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.883+0000 7f46df7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46e0141500 0x7f46e00c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:03.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.884+0000 7f46df7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46e0141500 0x7f46e00c7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40556/0 (socket says 172.21.15.163:40556) 2023-10-09T06:36:03.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.884+0000 7f46df7fe700 1 -- 172.21.15.163:0/645217798 learned_addr learned my addr 172.21.15.163:0/645217798 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:03.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.884+0000 7f46deffd700 1 --2- 172.21.15.163:0/645217798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46e006bcd0 0x7f46e013c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:03.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.884+0000 7f46df7fe700 1 -- 172.21.15.163:0/645217798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46e006bcd0 msgr2=0x7f46e013c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:03.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.884+0000 7f46df7fe700 1 --2- 172.21.15.163:0/645217798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46e006bcd0 0x7f46e013c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:03.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.884+0000 7f46df7fe700 1 -- 172.21.15.163:0/645217798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46e006c0a0 msgr2=0x7f46e00c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:03.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.884+0000 7f46df7fe700 1 --2- 172.21.15.163:0/645217798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46e006c0a0 0x7f46e00c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:03.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.884+0000 7f46df7fe700 1 -- 172.21.15.163:0/645217798 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d8009a20 con 0x7f46e0141500 2023-10-09T06:36:03.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.884+0000 7f46df7fe700 1 --2- 172.21.15.163:0/645217798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46e0141500 0x7f46e00c7570 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f46e0069220 tx=0x7f46d8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:03.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.884+0000 7f46d77fe700 1 -- 172.21.15.163:0/645217798 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d8012e80 con 0x7f46e0141500 2023-10-09T06:36:03.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.884+0000 7f46d77fe700 1 -- 172.21.15.163:0/645217798 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d801d630 con 0x7f46e0141500 2023-10-09T06:36:03.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.884+0000 7f46d77fe700 1 -- 172.21.15.163:0/645217798 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d801d9f0 con 0x7f46e0141500 2023-10-09T06:36:03.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.884+0000 7f46e59a4700 1 -- 172.21.15.163:0/645217798 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e00b1680 con 0x7f46e0141500 2023-10-09T06:36:03.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.884+0000 7f46e59a4700 1 -- 172.21.15.163:0/645217798 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e00b1c40 con 0x7f46e0141500 2023-10-09T06:36:03.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.885+0000 7f46e59a4700 1 -- 172.21.15.163:0/645217798 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e00b2160 con 0x7f46e0141500 2023-10-09T06:36:03.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.885+0000 7f46d77fe700 1 -- 172.21.15.163:0/645217798 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f46d8024070 con 0x7f46e0141500 2023-10-09T06:36:03.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.886+0000 7f46d77fe700 1 --2- 172.21.15.163:0/645217798 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f46c0067170 0x7f46c0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:03.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.886+0000 7f46d77fe700 1 -- 172.21.15.163:0/645217798 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f46d8087110 con 0x7f46e0141500 2023-10-09T06:36:03.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.886+0000 7f46deffd700 1 --2- 172.21.15.163:0/645217798 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f46c0067170 0x7f46c0069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:03.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.886+0000 7f46deffd700 1 --2- 172.21.15.163:0/645217798 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f46c0067170 0x7f46c0069620 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f46d00058d0 tx=0x7f46d0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:03.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:03.890+0000 7f46d77fe700 1 -- 172.21.15.163:0/645217798 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f46e00b2160 con 0x7f46e0141500 2023-10-09T06:36:04.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.073+0000 7f46e59a4700 1 -- 172.21.15.163:0/645217798 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46e00b2120 con 0x7f46e0141500 2023-10-09T06:36:04.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.073+0000 7f46d77fe700 1 -- 172.21.15.163:0/645217798 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+15401 (secure 0 0 0) 0x7f46d80540f0 con 0x7f46e0141500 2023-10-09T06:36:04.074 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:04.075 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":77,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:35:53.645012+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:35:50.332005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-09T06:35:50.969956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:36:04.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.075+0000 7f46e59a4700 1 -- 172.21.15.163:0/645217798 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f46c0067170 msgr2=0x7f46c0069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:04.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.075+0000 7f46e59a4700 1 --2- 172.21.15.163:0/645217798 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f46c0067170 0x7f46c0069620 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f46d00058d0 tx=0x7f46d0009040 comp rx=0 tx=0).stop 2023-10-09T06:36:04.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.075+0000 7f46e59a4700 1 -- 172.21.15.163:0/645217798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46e0141500 msgr2=0x7f46e00c7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:04.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.075+0000 7f46e59a4700 1 --2- 172.21.15.163:0/645217798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46e0141500 0x7f46e00c7570 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f46e0069220 tx=0x7f46d8019000 comp rx=0 tx=0).stop 2023-10-09T06:36:04.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.075+0000 7f46e59a4700 1 -- 172.21.15.163:0/645217798 shutdown_connections 2023-10-09T06:36:04.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.075+0000 7f46e59a4700 1 --2- 172.21.15.163:0/645217798 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f46c0067170 0x7f46c0069620 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.075+0000 7f46e59a4700 1 --2- 172.21.15.163:0/645217798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46e0141500 0x7f46e00c7570 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.075+0000 7f46e59a4700 1 --2- 172.21.15.163:0/645217798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46e006bcd0 0x7f46e013c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.075+0000 7f46e59a4700 1 --2- 172.21.15.163:0/645217798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46e006c0a0 0x7f46e00c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.078 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.075+0000 7f46e59a4700 1 -- 172.21.15.163:0/645217798 >> 172.21.15.163:0/645217798 conn(0x7f46e00ce600 msgr2=0x7f46e00c33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:04.078 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.075+0000 7f46e59a4700 1 -- 172.21.15.163:0/645217798 shutdown_connections 2023-10-09T06:36:04.078 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.075+0000 7f46e59a4700 1 -- 172.21.15.163:0/645217798 wait complete. 2023-10-09T06:36:04.088 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-09T06:35:50.332005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-10-09T06:35:50.969956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-09T06:36:04.089 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:36:04.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.205+0000 7fd5386d6700 1 Processor -- start 2023-10-09T06:36:04.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.205+0000 7fd5386d6700 1 -- start start 2023-10-09T06:36:04.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.206+0000 7fd5386d6700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd530159890 0x7fd53015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:04.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.206+0000 7fd5386d6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd530069560 0x7fd5300699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:04.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.206+0000 7fd5386d6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd530069ee0 con 0x7fd530159c60 2023-10-09T06:36:04.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.206+0000 7fd5386d6700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd53015a160 con 0x7fd530159890 2023-10-09T06:36:04.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.206+0000 7fd5386d6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5300d13a0 con 0x7fd530069560 2023-10-09T06:36:04.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.206+0000 7fd536472700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd530159890 0x7fd53015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:04.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.206+0000 7fd536472700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd530159890 0x7fd53015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40564/0 (socket says 172.21.15.163:40564) 2023-10-09T06:36:04.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.206+0000 7fd536472700 1 -- 172.21.15.163:0/847419811 learned_addr learned my addr 172.21.15.163:0/847419811 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:04.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.207+0000 7fd536c73700 1 --2- 172.21.15.163:0/847419811 >> v2:172.21.15.181:3300/0 conn(0x7fd530069560 0x7fd5300699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:04.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.207+0000 7fd536472700 1 -- 172.21.15.163:0/847419811 >> v2:172.21.15.181:3300/0 conn(0x7fd530069560 msgr2=0x7fd5300699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:04.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.207+0000 7fd536472700 1 --2- 172.21.15.163:0/847419811 >> v2:172.21.15.181:3300/0 conn(0x7fd530069560 0x7fd5300699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.207+0000 7fd536472700 1 -- 172.21.15.163:0/847419811 >> v1:172.21.15.181:6789/0 conn(0x7fd530159c60 legacy=0x7fd53015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:04.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.207+0000 7fd536472700 1 -- 172.21.15.163:0/847419811 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53015e140 con 0x7fd530159890 2023-10-09T06:36:04.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.207+0000 7fd536472700 1 --2- 172.21.15.163:0/847419811 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd530159890 0x7fd53015abd0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd53006a890 tx=0x7fd52c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7040ddfda51d84 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:04.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.207+0000 7fd535470700 1 -- 172.21.15.163:0/847419811 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c00b860 con 0x7fd530159890 2023-10-09T06:36:04.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.207+0000 7fd535470700 1 -- 172.21.15.163:0/847419811 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52c00b9c0 con 0x7fd530159890 2023-10-09T06:36:04.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.207+0000 7fd535470700 1 -- 172.21.15.163:0/847419811 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c00bd60 con 0x7fd530159890 2023-10-09T06:36:04.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.208+0000 7fd5386d6700 1 -- 172.21.15.163:0/847419811 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd530159890 msgr2=0x7fd53015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:04.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.208+0000 7fd5386d6700 1 --2- 172.21.15.163:0/847419811 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd530159890 0x7fd53015abd0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd53006a890 tx=0x7fd52c009250 comp rx=0 tx=0).stop 2023-10-09T06:36:04.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.208+0000 7fd5386d6700 1 -- 172.21.15.163:0/847419811 shutdown_connections 2023-10-09T06:36:04.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.208+0000 7fd5386d6700 1 --2- 172.21.15.163:0/847419811 >> v2:172.21.15.181:3300/0 conn(0x7fd530069560 0x7fd5300699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.208+0000 7fd5386d6700 1 --2- 172.21.15.163:0/847419811 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd530159890 0x7fd53015abd0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.208+0000 7fd5386d6700 1 -- 172.21.15.163:0/847419811 >> 172.21.15.163:0/847419811 conn(0x7fd530068e20 msgr2=0x7fd5300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:04.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.208+0000 7fd5386d6700 1 -- 172.21.15.163:0/847419811 shutdown_connections 2023-10-09T06:36:04.211 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.208+0000 7fd5386d6700 1 -- 172.21.15.163:0/847419811 wait complete. 2023-10-09T06:36:04.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.209+0000 7fd5386d6700 1 Processor -- start 2023-10-09T06:36:04.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.209+0000 7fd5386d6700 1 -- start start 2023-10-09T06:36:04.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.209+0000 7fd5386d6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd530069560 0x7fd5300b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:04.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.210+0000 7fd5386d6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd530159890 0x7fd5300c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:04.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.210+0000 7fd5386d6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd530159c60 0x7fd530152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:04.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.210+0000 7fd5386d6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300b36c0 con 0x7fd530159c60 2023-10-09T06:36:04.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.210+0000 7fd5386d6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300c87a0 con 0x7fd530159890 2023-10-09T06:36:04.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.210+0000 7fd5386d6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300c8a70 con 0x7fd530069560 2023-10-09T06:36:04.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.210+0000 7fd536c73700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd530159c60 0x7fd530152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:04.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.210+0000 7fd536c73700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd530159c60 0x7fd530152430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40580/0 (socket says 172.21.15.163:40580) 2023-10-09T06:36:04.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.210+0000 7fd536c73700 1 -- 172.21.15.163:0/3088530624 learned_addr learned my addr 172.21.15.163:0/3088530624 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:04.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.210+0000 7fd536472700 1 --2- 172.21.15.163:0/3088530624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd530069560 0x7fd5300b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:04.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.210+0000 7fd535c71700 1 --2- 172.21.15.163:0/3088530624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd530159890 0x7fd5300c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:04.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.210+0000 7fd536c73700 1 -- 172.21.15.163:0/3088530624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd530069560 msgr2=0x7fd5300b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:04.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.210+0000 7fd536c73700 1 --2- 172.21.15.163:0/3088530624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd530069560 0x7fd5300b60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.210+0000 7fd536c73700 1 -- 172.21.15.163:0/3088530624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd530159890 msgr2=0x7fd5300c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:04.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.210+0000 7fd536c73700 1 --2- 172.21.15.163:0/3088530624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd530159890 0x7fd5300c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.210+0000 7fd536c73700 1 -- 172.21.15.163:0/3088530624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52c0094f0 con 0x7fd530159c60 2023-10-09T06:36:04.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.210+0000 7fd535c71700 1 --2- 172.21.15.163:0/3088530624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd530159890 0x7fd5300c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:04.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.210+0000 7fd536472700 1 --2- 172.21.15.163:0/3088530624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd530069560 0x7fd5300b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:04.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.210+0000 7fd536c73700 1 --2- 172.21.15.163:0/3088530624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd530159c60 0x7fd530152430 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd528009fa0 tx=0x7fd528009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:04.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.210+0000 7fd5267fc700 1 -- 172.21.15.163:0/3088530624 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52800ed20 con 0x7fd530159c60 2023-10-09T06:36:04.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.210+0000 7fd5267fc700 1 -- 172.21.15.163:0/3088530624 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52800ee80 con 0x7fd530159c60 2023-10-09T06:36:04.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.210+0000 7fd5386d6700 1 -- 172.21.15.163:0/3088530624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5300c8d50 con 0x7fd530159c60 2023-10-09T06:36:04.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.210+0000 7fd5267fc700 1 -- 172.21.15.163:0/3088530624 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528014670 con 0x7fd530159c60 2023-10-09T06:36:04.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.210+0000 7fd5386d6700 1 -- 172.21.15.163:0/3088530624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5300c9310 con 0x7fd530159c60 2023-10-09T06:36:04.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.211+0000 7fd5386d6700 1 -- 172.21.15.163:0/3088530624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5300b3870 con 0x7fd530159c60 2023-10-09T06:36:04.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.212+0000 7fd5267fc700 1 -- 172.21.15.163:0/3088530624 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fd5280149e0 con 0x7fd530159c60 2023-10-09T06:36:04.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.212+0000 7fd5267fc700 1 --2- 172.21.15.163:0/3088530624 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd514067170 0x7fd514069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:04.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.212+0000 7fd5267fc700 1 -- 172.21.15.163:0/3088530624 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd528085000 con 0x7fd530159c60 2023-10-09T06:36:04.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.212+0000 7fd536472700 1 --2- 172.21.15.163:0/3088530624 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd514067170 0x7fd514069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:04.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.213+0000 7fd536472700 1 --2- 172.21.15.163:0/3088530624 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd514067170 0x7fd514069620 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd53006a890 tx=0x7fd52c017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:04.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.216+0000 7fd5267fc700 1 -- 172.21.15.163:0/3088530624 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd528051fe0 con 0x7fd530159c60 2023-10-09T06:36:04.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.398+0000 7fd5386d6700 1 -- 172.21.15.163:0/3088530624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd53004c3c0 con 0x7fd530159c60 2023-10-09T06:36:04.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.398+0000 7fd5267fc700 1 -- 172.21.15.163:0/3088530624 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd528055600 con 0x7fd530159c60 2023-10-09T06:36:04.399 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:36:04.401 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.400+0000 7fd5386d6700 1 -- 172.21.15.163:0/3088530624 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd514067170 msgr2=0x7fd514069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:04.401 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.400+0000 7fd5386d6700 1 --2- 172.21.15.163:0/3088530624 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd514067170 0x7fd514069620 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd53006a890 tx=0x7fd52c017000 comp rx=0 tx=0).stop 2023-10-09T06:36:04.401 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.400+0000 7fd5386d6700 1 -- 172.21.15.163:0/3088530624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd530159c60 msgr2=0x7fd530152430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:04.402 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.400+0000 7fd5386d6700 1 --2- 172.21.15.163:0/3088530624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd530159c60 0x7fd530152430 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd528009fa0 tx=0x7fd528009900 comp rx=0 tx=0).stop 2023-10-09T06:36:04.402 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.400+0000 7fd5386d6700 1 -- 172.21.15.163:0/3088530624 shutdown_connections 2023-10-09T06:36:04.402 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.400+0000 7fd5386d6700 1 --2- 172.21.15.163:0/3088530624 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd514067170 0x7fd514069620 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.402 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.400+0000 7fd5386d6700 1 --2- 172.21.15.163:0/3088530624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd530159c60 0x7fd530152430 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.402 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.400+0000 7fd5386d6700 1 --2- 172.21.15.163:0/3088530624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd530069560 0x7fd5300b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.402 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.400+0000 7fd5386d6700 1 --2- 172.21.15.163:0/3088530624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd530159890 0x7fd5300c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.402 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.400+0000 7fd5386d6700 1 -- 172.21.15.163:0/3088530624 >> 172.21.15.163:0/3088530624 conn(0x7fd530068e20 msgr2=0x7fd530153390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:04.402 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.400+0000 7fd5386d6700 1 -- 172.21.15.163:0/3088530624 shutdown_connections 2023-10-09T06:36:04.402 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.400+0000 7fd5386d6700 1 -- 172.21.15.163:0/3088530624 wait complete. 2023-10-09T06:36:04.413 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-09T06:36:04.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.530+0000 7f5ed359e700 1 Processor -- start 2023-10-09T06:36:04.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.530+0000 7f5ed359e700 1 -- start start 2023-10-09T06:36:04.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.531+0000 7f5ed359e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5ed40b16d0 0x7f5ed40b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:04.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.531+0000 7f5ed359e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5ed40b33b0 0x7f5ed40b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:04.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.531+0000 7f5ed359e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed40b2080 con 0x7f5ed40b4e70 2023-10-09T06:36:04.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.531+0000 7f5ed359e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ed40cfb00 con 0x7f5ed40b16d0 2023-10-09T06:36:04.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.531+0000 7f5ed359e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ed40b5c40 con 0x7f5ed40b33b0 2023-10-09T06:36:04.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.531+0000 7f5ec9d9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5ed40b16d0 0x7f5ed40b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:04.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.531+0000 7f5ed259c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f5ed40b4e70 0x7f5ed40ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60678/0 (socket says 172.21.15.163:60678) 2023-10-09T06:36:04.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.531+0000 7f5ed259c700 1 -- 172.21.15.163:0/501991937 learned_addr learned my addr 172.21.15.163:0/501991937 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:04.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.531+0000 7f5ed2d9d700 1 --2- 172.21.15.163:0/501991937 >> v2:172.21.15.181:3300/0 conn(0x7f5ed40b33b0 0x7f5ed40b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:04.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.531+0000 7f5ec9d9b700 1 -- 172.21.15.163:0/501991937 >> v2:172.21.15.181:3300/0 conn(0x7f5ed40b33b0 msgr2=0x7f5ed40b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:04.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.531+0000 7f5ec9d9b700 1 --2- 172.21.15.163:0/501991937 >> v2:172.21.15.181:3300/0 conn(0x7f5ed40b33b0 0x7f5ed40b37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.531+0000 7f5ec9d9b700 1 -- 172.21.15.163:0/501991937 >> v1:172.21.15.163:6789/0 conn(0x7f5ed40b4e70 legacy=0x7f5ed40ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:04.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.531+0000 7f5ec9d9b700 1 -- 172.21.15.163:0/501991937 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed40b5f20 con 0x7f5ed40b16d0 2023-10-09T06:36:04.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.531+0000 7f5ed1d9b700 1 -- 172.21.15.163:0/501991937 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2871920832 0 0) 0x7f5ed40b2080 con 0x7f5ed40b4e70 2023-10-09T06:36:04.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.532+0000 7f5ec9d9b700 1 --2- 172.21.15.163:0/501991937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ed40b16d0 0x7f5ed40b1b40 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4071380 tx=0x7f5ebc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9bad150a91f6e0f server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:04.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.532+0000 7f5ed1d9b700 1 -- 172.21.15.163:0/501991937 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc011630 con 0x7f5ed40b16d0 2023-10-09T06:36:04.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.532+0000 7f5ed1d9b700 1 -- 172.21.15.163:0/501991937 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ebc011790 con 0x7f5ed40b16d0 2023-10-09T06:36:04.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.532+0000 7f5ed1d9b700 1 -- 172.21.15.163:0/501991937 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc011b30 con 0x7f5ed40b16d0 2023-10-09T06:36:04.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.532+0000 7f5ed359e700 1 -- 172.21.15.163:0/501991937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ed40b16d0 msgr2=0x7f5ed40b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:04.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.532+0000 7f5ed359e700 1 --2- 172.21.15.163:0/501991937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ed40b16d0 0x7f5ed40b1b40 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4071380 tx=0x7f5ebc0091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.532+0000 7f5ed359e700 1 -- 172.21.15.163:0/501991937 shutdown_connections 2023-10-09T06:36:04.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.532+0000 7f5ed359e700 1 --2- 172.21.15.163:0/501991937 >> v2:172.21.15.181:3300/0 conn(0x7f5ed40b33b0 0x7f5ed40b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.532+0000 7f5ed359e700 1 --2- 172.21.15.163:0/501991937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ed40b16d0 0x7f5ed40b1b40 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.532+0000 7f5ed359e700 1 -- 172.21.15.163:0/501991937 >> 172.21.15.163:0/501991937 conn(0x7f5ed40d10c0 msgr2=0x7f5ed40affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:04.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.532+0000 7f5ed359e700 1 -- 172.21.15.163:0/501991937 shutdown_connections 2023-10-09T06:36:04.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.532+0000 7f5ed359e700 1 -- 172.21.15.163:0/501991937 wait complete. 2023-10-09T06:36:04.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.533+0000 7f5ed359e700 1 Processor -- start 2023-10-09T06:36:04.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.534+0000 7f5ed359e700 1 -- start start 2023-10-09T06:36:04.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.534+0000 7f5ed359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ed40b16d0 0x7f5ed414aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:04.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.534+0000 7f5ed359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ed40b33b0 0x7f5ed414af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:04.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.534+0000 7f5ed359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ed40b4e70 0x7f5ed416d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:04.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.534+0000 7f5ed359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed40a9b10 con 0x7f5ed40b16d0 2023-10-09T06:36:04.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.534+0000 7f5ed359e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed40a8f00 con 0x7f5ed40b33b0 2023-10-09T06:36:04.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.534+0000 7f5ed359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed40a91e0 con 0x7f5ed40b4e70 2023-10-09T06:36:04.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.534+0000 7f5ed259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ed40b16d0 0x7f5ed414aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:04.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.534+0000 7f5ed259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ed40b16d0 0x7f5ed414aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40604/0 (socket says 172.21.15.163:40604) 2023-10-09T06:36:04.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.534+0000 7f5ed259c700 1 -- 172.21.15.163:0/3736923488 learned_addr learned my addr 172.21.15.163:0/3736923488 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:04.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.535+0000 7f5ed2d9d700 1 --2- 172.21.15.163:0/3736923488 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ed40b4e70 0x7f5ed416d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:04.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.535+0000 7f5ec9d9b700 1 --2- 172.21.15.163:0/3736923488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ed40b33b0 0x7f5ed414af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:04.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.535+0000 7f5ed259c700 1 -- 172.21.15.163:0/3736923488 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ed40b4e70 msgr2=0x7f5ed416d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:04.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.535+0000 7f5ed259c700 1 --2- 172.21.15.163:0/3736923488 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ed40b4e70 0x7f5ed416d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.535+0000 7f5ed259c700 1 -- 172.21.15.163:0/3736923488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ed40b33b0 msgr2=0x7f5ed414af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:04.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.535+0000 7f5ed259c700 1 --2- 172.21.15.163:0/3736923488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ed40b33b0 0x7f5ed414af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.535+0000 7f5ed259c700 1 -- 172.21.15.163:0/3736923488 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ebc009420 con 0x7f5ed40b16d0 2023-10-09T06:36:04.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.535+0000 7f5ed2d9d700 1 --2- 172.21.15.163:0/3736923488 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ed40b4e70 0x7f5ed416d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:04.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.535+0000 7f5ec9d9b700 1 --2- 172.21.15.163:0/3736923488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ed40b33b0 0x7f5ed414af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:04.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.535+0000 7f5ed259c700 1 --2- 172.21.15.163:0/3736923488 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ed40b16d0 0x7f5ed414aa10 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400c7e0 tx=0x7f5ec4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:04.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.535+0000 7f5ecaffd700 1 -- 172.21.15.163:0/3736923488 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4005b30 con 0x7f5ed40b16d0 2023-10-09T06:36:04.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.535+0000 7f5ecaffd700 1 -- 172.21.15.163:0/3736923488 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec4005c90 con 0x7f5ed40b16d0 2023-10-09T06:36:04.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.535+0000 7f5ed359e700 1 -- 172.21.15.163:0/3736923488 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed40a94c0 con 0x7f5ed40b16d0 2023-10-09T06:36:04.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.535+0000 7f5ecaffd700 1 -- 172.21.15.163:0/3736923488 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4007300 con 0x7f5ed40b16d0 2023-10-09T06:36:04.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.535+0000 7f5ed359e700 1 -- 172.21.15.163:0/3736923488 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed414b5a0 con 0x7f5ed40b16d0 2023-10-09T06:36:04.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.536+0000 7f5ed359e700 1 -- 172.21.15.163:0/3736923488 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed40aaf70 con 0x7f5ed40b16d0 2023-10-09T06:36:04.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.536+0000 7f5ecaffd700 1 -- 172.21.15.163:0/3736923488 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f5ec401b400 con 0x7f5ed40b16d0 2023-10-09T06:36:04.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.537+0000 7f5ecaffd700 1 --2- 172.21.15.163:0/3736923488 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5eb4067170 0x7f5eb4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:04.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.537+0000 7f5ecaffd700 1 -- 172.21.15.163:0/3736923488 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5ec4084490 con 0x7f5ed40b16d0 2023-10-09T06:36:04.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.537+0000 7f5ec9d9b700 1 --2- 172.21.15.163:0/3736923488 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5eb4067170 0x7f5eb4069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:04.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.537+0000 7f5ec9d9b700 1 --2- 172.21.15.163:0/3736923488 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5eb4067170 0x7f5eb4069620 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0093f0 tx=0x7f5ebc017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:04.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.541+0000 7f5ecaffd700 1 -- 172.21.15.163:0/3736923488 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ec4051420 con 0x7f5ed40b16d0 2023-10-09T06:36:04.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.713+0000 7f5ecaffd700 1 -- 172.21.15.163:0/3736923488 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f5ec40517f0 con 0x7f5ed40b16d0 2023-10-09T06:36:04.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.728+0000 7f5ed359e700 1 -- 172.21.15.163:0/3736923488 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5ed404bb80 con 0x7f5ed40b16d0 2023-10-09T06:36:04.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.728+0000 7f5ecaffd700 1 -- 172.21.15.163:0/3736923488 <== mon.0 v2:172.21.15.163:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5ec4054eb0 con 0x7f5ed40b16d0 2023-10-09T06:36:04.729 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 32 2023-10-09T06:36:04.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.731+0000 7f5ed359e700 1 -- 172.21.15.163:0/3736923488 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5eb4067170 msgr2=0x7f5eb4069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:04.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.731+0000 7f5ed359e700 1 --2- 172.21.15.163:0/3736923488 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5eb4067170 0x7f5eb4069620 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0093f0 tx=0x7f5ebc017000 comp rx=0 tx=0).stop 2023-10-09T06:36:04.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.731+0000 7f5ed359e700 1 -- 172.21.15.163:0/3736923488 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ed40b16d0 msgr2=0x7f5ed414aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:04.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.731+0000 7f5ed359e700 1 --2- 172.21.15.163:0/3736923488 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ed40b16d0 0x7f5ed414aa10 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400c7e0 tx=0x7f5ec4007df0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.731+0000 7f5ed359e700 1 -- 172.21.15.163:0/3736923488 shutdown_connections 2023-10-09T06:36:04.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.731+0000 7f5ed359e700 1 --2- 172.21.15.163:0/3736923488 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5eb4067170 0x7f5eb4069620 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.731+0000 7f5ed359e700 1 --2- 172.21.15.163:0/3736923488 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ed40b16d0 0x7f5ed414aa10 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.731+0000 7f5ed359e700 1 --2- 172.21.15.163:0/3736923488 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ed40b4e70 0x7f5ed416d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.731+0000 7f5ed359e700 1 --2- 172.21.15.163:0/3736923488 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ed40b33b0 0x7f5ed414af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.731+0000 7f5ed359e700 1 -- 172.21.15.163:0/3736923488 >> 172.21.15.163:0/3736923488 conn(0x7f5ed40d10c0 msgr2=0x7f5ed40b0780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:04.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.731+0000 7f5ed359e700 1 -- 172.21.15.163:0/3736923488 shutdown_connections 2023-10-09T06:36:04.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.731+0000 7f5ed359e700 1 -- 172.21.15.163:0/3736923488 wait complete. 2023-10-09T06:36:04.744 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-09T06:36:04.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.861+0000 7fbe77fff700 1 Processor -- start 2023-10-09T06:36:04.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.861+0000 7fbe77fff700 1 -- start start 2023-10-09T06:36:04.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.862+0000 7fbe77fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbe7806eb80 0x7fbe780b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:04.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.862+0000 7fbe77fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe780b27b0 con 0x7fbe780b0d20 2023-10-09T06:36:04.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.862+0000 7fbe77fff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe780b1de0 con 0x7fbe78159a50 2023-10-09T06:36:04.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.862+0000 7fbe77fff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe780c9180 con 0x7fbe7806eb80 2023-10-09T06:36:04.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.862+0000 7fbe6e7fc700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fbe780b0d20 0x7fbe7806e470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60686/0 (socket says 172.21.15.163:60686) 2023-10-09T06:36:04.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.862+0000 7fbe6e7fc700 1 -- 172.21.15.163:0/2916690349 learned_addr learned my addr 172.21.15.163:0/2916690349 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:04.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.862+0000 7fbe767fc700 1 -- 172.21.15.163:0/2916690349 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2204350451 0 0) 0x7fbe60001110 con 0x7fbe780b0d20 2023-10-09T06:36:04.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.863+0000 7fbe767fc700 1 -- 172.21.15.163:0/2916690349 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe64003570 con 0x7fbe780b0d20 2023-10-09T06:36:04.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.863+0000 7fbe777fe700 1 --2- 172.21.15.163:0/2916690349 >> v2:172.21.15.181:3300/0 conn(0x7fbe7806eb80 0x7fbe780b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:04.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.863+0000 7fbe767fc700 1 -- 172.21.15.163:0/2916690349 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 5017991 0 0) 0x7fbe64003570 con 0x7fbe780b0d20 2023-10-09T06:36:04.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.863+0000 7fbe767fc700 1 -- 172.21.15.163:0/2916690349 >> v2:172.21.15.181:3300/0 conn(0x7fbe7806eb80 msgr2=0x7fbe780b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:04.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.863+0000 7fbe767fc700 1 --2- 172.21.15.163:0/2916690349 >> v2:172.21.15.181:3300/0 conn(0x7fbe7806eb80 0x7fbe780b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.863+0000 7fbe767fc700 1 -- 172.21.15.163:0/2916690349 >> v1:172.21.15.181:6789/0 conn(0x7fbe78159a50 legacy=0x7fbe780b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:04.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.863+0000 7fbe767fc700 1 -- 172.21.15.163:0/2916690349 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe780b11d0 con 0x7fbe780b0d20 2023-10-09T06:36:04.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.863+0000 7fbe767fc700 1 -- 172.21.15.163:0/2916690349 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fbe60001600 con 0x7fbe780b0d20 2023-10-09T06:36:04.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.863+0000 7fbe767fc700 1 -- 172.21.15.163:0/2916690349 >> v1:172.21.15.163:6789/0 conn(0x7fbe780b0d20 legacy=0x7fbe7806e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:04.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.863+0000 7fbe767fc700 1 --2- 172.21.15.163:0/2916690349 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe64003a60 0x7fbe64003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:04.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.863+0000 7fbe767fc700 1 --2- 172.21.15.163:0/2916690349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe64004810 0x7fbe64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:04.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.863+0000 7fbe767fc700 1 --2- 172.21.15.163:0/2916690349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe640051c0 0x7fbe6400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:04.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.863+0000 7fbe767fc700 1 -- 172.21.15.163:0/2916690349 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe780c9180 con 0x7fbe640051c0 2023-10-09T06:36:04.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.863+0000 7fbe767fc700 1 -- 172.21.15.163:0/2916690349 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6400c0d0 con 0x7fbe64004810 2023-10-09T06:36:04.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.863+0000 7fbe767fc700 1 -- 172.21.15.163:0/2916690349 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6400c290 con 0x7fbe64003a60 2023-10-09T06:36:04.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.863+0000 7fbe777fe700 1 --2- 172.21.15.163:0/2916690349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe640051c0 0x7fbe6400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:04.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.863+0000 7fbe6e7fc700 1 --2- 172.21.15.163:0/2916690349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe64004810 0x7fbe64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:04.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.863+0000 7fbe76ffd700 1 --2- 172.21.15.163:0/2916690349 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe64003a60 0x7fbe64003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:04.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.863+0000 7fbe777fe700 1 -- 172.21.15.163:0/2916690349 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe64003a60 msgr2=0x7fbe64003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:04.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.863+0000 7fbe777fe700 1 --2- 172.21.15.163:0/2916690349 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe64003a60 0x7fbe64003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.863+0000 7fbe777fe700 1 -- 172.21.15.163:0/2916690349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe64004810 msgr2=0x7fbe64004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:04.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.863+0000 7fbe777fe700 1 --2- 172.21.15.163:0/2916690349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe64004810 0x7fbe64004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.863+0000 7fbe777fe700 1 -- 172.21.15.163:0/2916690349 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe64003570 con 0x7fbe640051c0 2023-10-09T06:36:04.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.863+0000 7fbe777fe700 1 -- 172.21.15.163:0/2916690349 reap_dead start 2023-10-09T06:36:04.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.863+0000 7fbe777fe700 1 --2- 172.21.15.163:0/2916690349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe640051c0 0x7fbe6400ba60 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fbe7806ef80 tx=0x7fbe7000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131ab227aee20b2a server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:04.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.863+0000 7fbe767fc700 1 -- 172.21.15.163:0/2916690349 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7000ea90 con 0x7fbe640051c0 2023-10-09T06:36:04.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.864+0000 7fbe767fc700 1 -- 172.21.15.163:0/2916690349 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7000f040 con 0x7fbe640051c0 2023-10-09T06:36:04.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.864+0000 7fbe767fc700 1 -- 172.21.15.163:0/2916690349 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7000ee40 con 0x7fbe640051c0 2023-10-09T06:36:04.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.864+0000 7fbe77fff700 1 -- 172.21.15.163:0/2916690349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe640051c0 msgr2=0x7fbe6400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:04.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.864+0000 7fbe77fff700 1 --2- 172.21.15.163:0/2916690349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe640051c0 0x7fbe6400ba60 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fbe7806ef80 tx=0x7fbe7000b040 comp rx=0 tx=0).stop 2023-10-09T06:36:04.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.864+0000 7fbe77fff700 1 -- 172.21.15.163:0/2916690349 shutdown_connections 2023-10-09T06:36:04.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.864+0000 7fbe77fff700 1 --2- 172.21.15.163:0/2916690349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe640051c0 0x7fbe6400ba60 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.864+0000 7fbe77fff700 1 -- 172.21.15.163:0/2916690349 >> 172.21.15.163:0/2916690349 conn(0x7fbe780c9f70 msgr2=0x7fbe7806dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:04.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.864+0000 7fbe77fff700 1 -- 172.21.15.163:0/2916690349 shutdown_connections 2023-10-09T06:36:04.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.864+0000 7fbe77fff700 1 -- 172.21.15.163:0/2916690349 wait complete. 2023-10-09T06:36:04.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.865+0000 7fbe77fff700 1 Processor -- start 2023-10-09T06:36:04.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.865+0000 7fbe77fff700 1 -- start start 2023-10-09T06:36:04.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.866+0000 7fbe77fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe640051c0 0x7fbe7814ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:04.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.866+0000 7fbe77fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe7814d390 0x7fbe78153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:04.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.866+0000 7fbe77fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe78154350 0x7fbe78165d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:04.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.866+0000 7fbe77fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe781547b0 con 0x7fbe78154350 2023-10-09T06:36:04.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.866+0000 7fbe77fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7814d860 con 0x7fbe640051c0 2023-10-09T06:36:04.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.866+0000 7fbe77fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe78168ef0 con 0x7fbe7814d390 2023-10-09T06:36:04.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.866+0000 7fbe777fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe78154350 0x7fbe78165d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:04.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.866+0000 7fbe777fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe78154350 0x7fbe78165d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40620/0 (socket says 172.21.15.163:40620) 2023-10-09T06:36:04.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.866+0000 7fbe777fe700 1 -- 172.21.15.163:0/3982393188 learned_addr learned my addr 172.21.15.163:0/3982393188 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:04.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.866+0000 7fbe76ffd700 1 --2- 172.21.15.163:0/3982393188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe640051c0 0x7fbe7814ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:04.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.866+0000 7fbe6e7fc700 1 --2- 172.21.15.163:0/3982393188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe7814d390 0x7fbe78153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:04.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.866+0000 7fbe777fe700 1 -- 172.21.15.163:0/3982393188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe7814d390 msgr2=0x7fbe78153e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:04.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.866+0000 7fbe777fe700 1 --2- 172.21.15.163:0/3982393188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe7814d390 0x7fbe78153e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.866+0000 7fbe777fe700 1 -- 172.21.15.163:0/3982393188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe640051c0 msgr2=0x7fbe7814ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:04.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.866+0000 7fbe777fe700 1 --2- 172.21.15.163:0/3982393188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe640051c0 0x7fbe7814ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:04.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.866+0000 7fbe777fe700 1 -- 172.21.15.163:0/3982393188 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe781691d0 con 0x7fbe78154350 2023-10-09T06:36:04.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.867+0000 7fbe777fe700 1 --2- 172.21.15.163:0/3982393188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe78154350 0x7fbe78165d40 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fbe7806db40 tx=0x7fbe70019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:04.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.867+0000 7fbe6f7fe700 1 -- 172.21.15.163:0/3982393188 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7001d660 con 0x7fbe78154350 2023-10-09T06:36:04.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.867+0000 7fbe6f7fe700 1 -- 172.21.15.163:0/3982393188 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe70022070 con 0x7fbe78154350 2023-10-09T06:36:04.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.867+0000 7fbe77fff700 1 -- 172.21.15.163:0/3982393188 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe781694b0 con 0x7fbe78154350 2023-10-09T06:36:04.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.867+0000 7fbe6f7fe700 1 -- 172.21.15.163:0/3982393188 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7001da10 con 0x7fbe78154350 2023-10-09T06:36:04.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.867+0000 7fbe77fff700 1 -- 172.21.15.163:0/3982393188 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe78166280 con 0x7fbe78154350 2023-10-09T06:36:04.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.868+0000 7fbe77fff700 1 -- 172.21.15.163:0/3982393188 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe7806cfa0 con 0x7fbe78154350 2023-10-09T06:36:04.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.868+0000 7fbe6f7fe700 1 -- 172.21.15.163:0/3982393188 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fbe70003020 con 0x7fbe78154350 2023-10-09T06:36:04.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.868+0000 7fbe6f7fe700 1 --2- 172.21.15.163:0/3982393188 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbe58067200 0x7fbe580696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:04.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.869+0000 7fbe6f7fe700 1 -- 172.21.15.163:0/3982393188 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbe70024070 con 0x7fbe78154350 2023-10-09T06:36:04.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.869+0000 7fbe76ffd700 1 --2- 172.21.15.163:0/3982393188 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbe58067200 0x7fbe580696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:04.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.869+0000 7fbe76ffd700 1 --2- 172.21.15.163:0/3982393188 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbe58067200 0x7fbe580696b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbe68000c00 tx=0x7fbe68009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:04.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:04.873+0000 7fbe6f7fe700 1 -- 172.21.15.163:0/3982393188 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbe70053ea0 con 0x7fbe78154350 2023-10-09T06:36:05.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.054+0000 7fbe77fff700 1 -- 172.21.15.163:0/3982393188 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbe780b14b0 con 0x7fbe78154350 2023-10-09T06:36:05.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.055+0000 7fbe6f7fe700 1 -- 172.21.15.163:0/3982393188 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbe700180a0 con 0x7fbe78154350 2023-10-09T06:36:05.056 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 64 2023-10-09T06:36:05.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.057+0000 7fbe77fff700 1 -- 172.21.15.163:0/3982393188 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbe58067200 msgr2=0x7fbe580696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:05.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.057+0000 7fbe77fff700 1 --2- 172.21.15.163:0/3982393188 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbe58067200 0x7fbe580696b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbe68000c00 tx=0x7fbe68009040 comp rx=0 tx=0).stop 2023-10-09T06:36:05.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.057+0000 7fbe77fff700 1 -- 172.21.15.163:0/3982393188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe78154350 msgr2=0x7fbe78165d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:05.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.057+0000 7fbe77fff700 1 --2- 172.21.15.163:0/3982393188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe78154350 0x7fbe78165d40 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fbe7806db40 tx=0x7fbe70019000 comp rx=0 tx=0).stop 2023-10-09T06:36:05.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.057+0000 7fbe77fff700 1 -- 172.21.15.163:0/3982393188 shutdown_connections 2023-10-09T06:36:05.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.057+0000 7fbe77fff700 1 --2- 172.21.15.163:0/3982393188 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbe58067200 0x7fbe580696b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.057+0000 7fbe77fff700 1 --2- 172.21.15.163:0/3982393188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe78154350 0x7fbe78165d40 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.057+0000 7fbe77fff700 1 --2- 172.21.15.163:0/3982393188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe640051c0 0x7fbe7814ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.057+0000 7fbe77fff700 1 --2- 172.21.15.163:0/3982393188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe7814d390 0x7fbe78153e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.057+0000 7fbe77fff700 1 -- 172.21.15.163:0/3982393188 >> 172.21.15.163:0/3982393188 conn(0x7fbe780c9f70 msgr2=0x7fbe78154d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:05.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.057+0000 7fbe77fff700 1 -- 172.21.15.163:0/3982393188 shutdown_connections 2023-10-09T06:36:05.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.057+0000 7fbe77fff700 1 -- 172.21.15.163:0/3982393188 wait complete. 2023-10-09T06:36:05.070 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:36:05.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.184+0000 7f4ed2a52700 1 Processor -- start 2023-10-09T06:36:05.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.184+0000 7f4ed2a52700 1 -- start start 2023-10-09T06:36:05.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.185+0000 7f4ed2a52700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ecc159c60 0x7f4ecc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:05.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.185+0000 7f4ed2a52700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4ecc15db10 0x7f4ecc069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:05.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.185+0000 7f4ed2a52700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ecc15a160 con 0x7f4ecc159890 2023-10-09T06:36:05.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.185+0000 7f4ed2a52700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ecc0d13a0 con 0x7f4ecc15db10 2023-10-09T06:36:05.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.185+0000 7f4ed2a52700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc15e140 con 0x7f4ecc159c60 2023-10-09T06:36:05.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.185+0000 7f4ed0fef700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4ecc15db10 0x7f4ecc069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:05.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.185+0000 7f4ecb7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ecc159c60 0x7f4ecc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:05.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.185+0000 7f4ecb7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ecc159c60 0x7f4ecc15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:40000/0 (socket says 172.21.15.163:40000) 2023-10-09T06:36:05.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.185+0000 7f4ed0fef700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4ecc15db10 0x7f4ecc069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:49060/0 (socket says 172.21.15.163:49060) 2023-10-09T06:36:05.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.185+0000 7f4ecbfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4ecc159890 0x7f4ecc15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:57316/0 (socket says 172.21.15.163:57316) 2023-10-09T06:36:05.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.185+0000 7f4ecb7fe700 1 -- 172.21.15.163:0/2179672888 learned_addr learned my addr 172.21.15.163:0/2179672888 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:05.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.186+0000 7f4ecb7fe700 1 -- 172.21.15.163:0/2179672888 >> v2:172.21.15.181:3300/0 conn(0x7f4ecc15db10 msgr2=0x7f4ecc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:05.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.186+0000 7f4ecb7fe700 1 --2- 172.21.15.163:0/2179672888 >> v2:172.21.15.181:3300/0 conn(0x7f4ecc15db10 0x7f4ecc069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.186+0000 7f4ecb7fe700 1 -- 172.21.15.163:0/2179672888 >> v1:172.21.15.181:6789/0 conn(0x7f4ecc159890 legacy=0x7f4ecc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:05.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.186+0000 7f4ecb7fe700 1 -- 172.21.15.163:0/2179672888 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ecc15a3d0 con 0x7f4ecc159c60 2023-10-09T06:36:05.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.186+0000 7f4ed0fef700 1 --2- 172.21.15.163:0/2179672888 >> v2:172.21.15.181:3300/0 conn(0x7f4ecc15db10 0x7f4ecc069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:36:05.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.186+0000 7f4ecb7fe700 1 --2- 172.21.15.163:0/2179672888 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ecc159c60 0x7f4ecc15d5d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00099b0 tx=0x7f4ec0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=719197afd4b6a59d server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:05.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.186+0000 7f4ecaffd700 1 -- 172.21.15.163:0/2179672888 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0009ea0 con 0x7f4ecc159c60 2023-10-09T06:36:05.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.186+0000 7f4ecaffd700 1 -- 172.21.15.163:0/2179672888 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec000b860 con 0x7f4ecc159c60 2023-10-09T06:36:05.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.186+0000 7f4ecaffd700 1 -- 172.21.15.163:0/2179672888 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec000bc00 con 0x7f4ecc159c60 2023-10-09T06:36:05.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.187+0000 7f4ed2a52700 1 -- 172.21.15.163:0/2179672888 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ecc159c60 msgr2=0x7f4ecc15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:05.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.187+0000 7f4ed2a52700 1 --2- 172.21.15.163:0/2179672888 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ecc159c60 0x7f4ecc15d5d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00099b0 tx=0x7f4ec0009250 comp rx=0 tx=0).stop 2023-10-09T06:36:05.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.187+0000 7f4ed2a52700 1 -- 172.21.15.163:0/2179672888 shutdown_connections 2023-10-09T06:36:05.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.187+0000 7f4ed2a52700 1 --2- 172.21.15.163:0/2179672888 >> v2:172.21.15.181:3300/0 conn(0x7f4ecc15db10 0x7f4ecc069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.187+0000 7f4ed2a52700 1 --2- 172.21.15.163:0/2179672888 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ecc159c60 0x7f4ecc15d5d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.187+0000 7f4ed2a52700 1 -- 172.21.15.163:0/2179672888 >> 172.21.15.163:0/2179672888 conn(0x7f4ecc068e20 msgr2=0x7f4ecc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:05.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.187+0000 7f4ed2a52700 1 -- 172.21.15.163:0/2179672888 shutdown_connections 2023-10-09T06:36:05.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.187+0000 7f4ed2a52700 1 -- 172.21.15.163:0/2179672888 wait complete. 2023-10-09T06:36:05.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.188+0000 7f4ed2a52700 1 Processor -- start 2023-10-09T06:36:05.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.188+0000 7f4ed2a52700 1 -- start start 2023-10-09T06:36:05.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.188+0000 7f4ed2a52700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ecc159890 0x7f4ecc16a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:05.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.188+0000 7f4ed2a52700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ecc159c60 0x7f4ecc16cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:05.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.189+0000 7f4ed2a52700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ecc15db10 0x7f4ecc152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:05.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.189+0000 7f4ed2a52700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc0cff40 con 0x7f4ecc159c60 2023-10-09T06:36:05.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.189+0000 7f4ed2a52700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc16ac60 con 0x7f4ecc15db10 2023-10-09T06:36:05.194 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.189+0000 7f4ed2a52700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc0cf330 con 0x7f4ecc159890 2023-10-09T06:36:05.194 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.189+0000 7f4ecb7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ecc159c60 0x7f4ecc16cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:05.194 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.189+0000 7f4ecb7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ecc159c60 0x7f4ecc16cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40630/0 (socket says 172.21.15.163:40630) 2023-10-09T06:36:05.194 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.189+0000 7f4ecb7fe700 1 -- 172.21.15.163:0/4158451887 learned_addr learned my addr 172.21.15.163:0/4158451887 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:05.194 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.189+0000 7f4ed0fef700 1 --2- 172.21.15.163:0/4158451887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ecc15db10 0x7f4ecc152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:05.194 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.189+0000 7f4ecbfff700 1 --2- 172.21.15.163:0/4158451887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ecc159890 0x7f4ecc16a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:05.194 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.189+0000 7f4ecb7fe700 1 -- 172.21.15.163:0/4158451887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ecc159890 msgr2=0x7f4ecc16a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:05.194 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.189+0000 7f4ecb7fe700 1 --2- 172.21.15.163:0/4158451887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ecc159890 0x7f4ecc16a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.189+0000 7f4ecb7fe700 1 -- 172.21.15.163:0/4158451887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ecc15db10 msgr2=0x7f4ecc152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:05.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.189+0000 7f4ecb7fe700 1 --2- 172.21.15.163:0/4158451887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ecc15db10 0x7f4ecc152860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.189+0000 7f4ecb7fe700 1 -- 172.21.15.163:0/4158451887 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec4009a20 con 0x7f4ecc159c60 2023-10-09T06:36:05.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.189+0000 7f4ecb7fe700 1 --2- 172.21.15.163:0/4158451887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ecc159c60 0x7f4ecc16cab0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc070e20 tx=0x7f4ec0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:05.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.189+0000 7f4eb3fff700 1 -- 172.21.15.163:0/4158451887 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec000bbe0 con 0x7f4ecc159c60 2023-10-09T06:36:05.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.189+0000 7f4eb3fff700 1 -- 172.21.15.163:0/4158451887 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec0011420 con 0x7f4ecc159c60 2023-10-09T06:36:05.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.189+0000 7f4ed2a52700 1 -- 172.21.15.163:0/4158451887 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec00094f0 con 0x7f4ecc159c60 2023-10-09T06:36:05.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.189+0000 7f4eb3fff700 1 -- 172.21.15.163:0/4158451887 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec00117c0 con 0x7f4ecc159c60 2023-10-09T06:36:05.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.189+0000 7f4ed2a52700 1 -- 172.21.15.163:0/4158451887 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ecc0cf8f0 con 0x7f4ecc159c60 2023-10-09T06:36:05.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.190+0000 7f4ed2a52700 1 -- 172.21.15.163:0/4158451887 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ecc0cd0c0 con 0x7f4ecc159c60 2023-10-09T06:36:05.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.191+0000 7f4eb3fff700 1 -- 172.21.15.163:0/4158451887 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f4ec0011bf0 con 0x7f4ecc159c60 2023-10-09T06:36:05.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.191+0000 7f4eb3fff700 1 --2- 172.21.15.163:0/4158451887 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4eb8067200 0x7f4eb80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:05.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.191+0000 7f4eb3fff700 1 -- 172.21.15.163:0/4158451887 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4ec0086120 con 0x7f4ecc159c60 2023-10-09T06:36:05.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.191+0000 7f4ecbfff700 1 --2- 172.21.15.163:0/4158451887 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4eb8067200 0x7f4eb80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:05.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.192+0000 7f4ecbfff700 1 --2- 172.21.15.163:0/4158451887 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4eb8067200 0x7f4eb80696b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc004fd0 tx=0x7f4ebc007eb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:05.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.195+0000 7f4eb3fff700 1 -- 172.21.15.163:0/4158451887 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ecc0cd0c0 con 0x7f4ecc159c60 2023-10-09T06:36:05.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.420+0000 7f4ed2a52700 1 -- 172.21.15.163:0/4158451887 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ecc0cd0c0 con 0x7f4ecc159c60 2023-10-09T06:36:05.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.421+0000 7f4eb3fff700 1 -- 172.21.15.163:0/4158451887 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4608 (secure 0 0 0) 0x7f4ecc0cd0c0 con 0x7f4ecc159c60 2023-10-09T06:36:05.422 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:05.422 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1946291256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1946291256},{"type":"v1","addr":"172.21.15.181:6837","nonce":1946291256}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5749,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1366951098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1366951098},{"type":"v1","addr":"172.21.15.181:6836","nonce":1366951098}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3619934997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3619934997},{"type":"v1","addr":"172.21.15.163:6834","nonce":3619934997}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:35:52.002698+0000","modified":"2023-10-09T06:35:55.653628+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/234029086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":234029086},{"type":"v1","addr":"172.21.15.163:6835","nonce":234029086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-09T06:36:05.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.423+0000 7f4ed2a52700 1 -- 172.21.15.163:0/4158451887 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4eb8067200 msgr2=0x7f4eb80696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:05.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.423+0000 7f4ed2a52700 1 --2- 172.21.15.163:0/4158451887 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4eb8067200 0x7f4eb80696b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc004fd0 tx=0x7f4ebc007eb0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.423+0000 7f4ed2a52700 1 -- 172.21.15.163:0/4158451887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ecc159c60 msgr2=0x7f4ecc16cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:05.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.423+0000 7f4ed2a52700 1 --2- 172.21.15.163:0/4158451887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ecc159c60 0x7f4ecc16cab0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc070e20 tx=0x7f4ec0017000 comp rx=0 tx=0).stop 2023-10-09T06:36:05.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.423+0000 7f4ed2a52700 1 -- 172.21.15.163:0/4158451887 shutdown_connections 2023-10-09T06:36:05.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.423+0000 7f4ed2a52700 1 --2- 172.21.15.163:0/4158451887 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4eb8067200 0x7f4eb80696b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.423+0000 7f4ed2a52700 1 --2- 172.21.15.163:0/4158451887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ecc15db10 0x7f4ecc152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.423+0000 7f4ed2a52700 1 --2- 172.21.15.163:0/4158451887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ecc159890 0x7f4ecc16a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.423+0000 7f4ed2a52700 1 --2- 172.21.15.163:0/4158451887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ecc159c60 0x7f4ecc16cab0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.423+0000 7f4ed2a52700 1 -- 172.21.15.163:0/4158451887 >> 172.21.15.163:0/4158451887 conn(0x7f4ecc068e20 msgr2=0x7f4ecc1588c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:05.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.423+0000 7f4ed2a52700 1 -- 172.21.15.163:0/4158451887 shutdown_connections 2023-10-09T06:36:05.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.423+0000 7f4ed2a52700 1 -- 172.21.15.163:0/4158451887 wait complete. 2023-10-09T06:36:05.425 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 49 2023-10-09T06:36:05.436 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:36:05.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.551+0000 7fadd359e700 1 Processor -- start 2023-10-09T06:36:05.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.551+0000 7fadd359e700 1 -- start start 2023-10-09T06:36:05.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.552+0000 7fadd359e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fadd415d230 0x7fadd415d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:05.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.552+0000 7fadd359e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadd40b2980 con 0x7fadd4159420 2023-10-09T06:36:05.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.552+0000 7fadd359e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadd415db70 con 0x7fadd41597f0 2023-10-09T06:36:05.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.552+0000 7fadd359e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadd40b11d0 con 0x7fadd415d230 2023-10-09T06:36:05.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.552+0000 7fadd259c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fadd4159420 0x7fadd40b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60688/0 (socket says 172.21.15.163:60688) 2023-10-09T06:36:05.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.552+0000 7fadd259c700 1 -- 172.21.15.163:0/407441872 learned_addr learned my addr 172.21.15.163:0/407441872 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:05.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.552+0000 7fadd159a700 1 -- 172.21.15.163:0/407441872 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2231598446 0 0) 0x7fadd40b2980 con 0x7fadd4159420 2023-10-09T06:36:05.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.552+0000 7fadd159a700 1 -- 172.21.15.163:0/407441872 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadbc003570 con 0x7fadd4159420 2023-10-09T06:36:05.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.552+0000 7fadd2d9d700 1 --2- 172.21.15.163:0/407441872 >> v2:172.21.15.181:3300/0 conn(0x7fadd415d230 0x7fadd415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:05.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.552+0000 7fadd159a700 1 -- 172.21.15.163:0/407441872 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1714538020 0 0) 0x7fadbc003570 con 0x7fadd4159420 2023-10-09T06:36:05.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.552+0000 7fadd159a700 1 -- 172.21.15.163:0/407441872 >> v2:172.21.15.181:3300/0 conn(0x7fadd415d230 msgr2=0x7fadd415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:05.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.552+0000 7fadd159a700 1 --2- 172.21.15.163:0/407441872 >> v2:172.21.15.181:3300/0 conn(0x7fadd415d230 0x7fadd415d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.552+0000 7fadd159a700 1 -- 172.21.15.163:0/407441872 >> v1:172.21.15.181:6789/0 conn(0x7fadd41597f0 legacy=0x7fadd40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:05.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.552+0000 7fadd159a700 1 -- 172.21.15.163:0/407441872 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd40b14b0 con 0x7fadd4159420 2023-10-09T06:36:05.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.552+0000 7fadd2d9d700 1 --2- 172.21.15.163:0/407441872 >> v2:172.21.15.181:3300/0 conn(0x7fadd415d230 0x7fadd415d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:05.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.552+0000 7fadd159a700 1 -- 172.21.15.163:0/407441872 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fadcc001470 con 0x7fadd4159420 2023-10-09T06:36:05.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.553+0000 7fadd159a700 1 -- 172.21.15.163:0/407441872 >> v1:172.21.15.163:6789/0 conn(0x7fadd4159420 legacy=0x7fadd40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:05.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.553+0000 7fadd159a700 1 --2- 172.21.15.163:0/407441872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadbc003a60 0x7fadbc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:05.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.553+0000 7fadd159a700 1 --2- 172.21.15.163:0/407441872 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadbc004810 0x7fadbc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:05.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.553+0000 7fadd159a700 1 --2- 172.21.15.163:0/407441872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadbc0052e0 0x7fadbc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:05.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.553+0000 7fadd159a700 1 -- 172.21.15.163:0/407441872 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd40b11d0 con 0x7fadbc0052e0 2023-10-09T06:36:05.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.553+0000 7fadd159a700 1 -- 172.21.15.163:0/407441872 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc00c190 con 0x7fadbc003a60 2023-10-09T06:36:05.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.553+0000 7fadd159a700 1 -- 172.21.15.163:0/407441872 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc00c350 con 0x7fadbc004810 2023-10-09T06:36:05.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.553+0000 7fadd2d9d700 1 --2- 172.21.15.163:0/407441872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadbc0052e0 0x7fadbc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:05.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.553+0000 7fadd259c700 1 --2- 172.21.15.163:0/407441872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadbc003a60 0x7fadbc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:05.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.553+0000 7fadd1d9b700 1 --2- 172.21.15.163:0/407441872 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadbc004810 0x7fadbc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:05.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.553+0000 7fadd2d9d700 1 -- 172.21.15.163:0/407441872 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadbc004810 msgr2=0x7fadbc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:05.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.553+0000 7fadd2d9d700 1 --2- 172.21.15.163:0/407441872 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadbc004810 0x7fadbc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.553+0000 7fadd2d9d700 1 -- 172.21.15.163:0/407441872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadbc003a60 msgr2=0x7fadbc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:05.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.553+0000 7fadd2d9d700 1 --2- 172.21.15.163:0/407441872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadbc003a60 0x7fadbc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.553+0000 7fadd2d9d700 1 -- 172.21.15.163:0/407441872 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadbc003570 con 0x7fadbc0052e0 2023-10-09T06:36:05.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.553+0000 7fadd1d9b700 1 --2- 172.21.15.163:0/407441872 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadbc004810 0x7fadbc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:05.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.553+0000 7fadd2d9d700 1 -- 172.21.15.163:0/407441872 reap_dead start 2023-10-09T06:36:05.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.553+0000 7fadd2d9d700 1 --2- 172.21.15.163:0/407441872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadbc0052e0 0x7fadbc00ba60 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fadc4007580 tx=0x7fadc40097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5128d93b5d3243e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:05.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.553+0000 7fadd159a700 1 -- 172.21.15.163:0/407441872 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc4009ae0 con 0x7fadbc0052e0 2023-10-09T06:36:05.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.553+0000 7fadd159a700 1 -- 172.21.15.163:0/407441872 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc4007dd0 con 0x7fadbc0052e0 2023-10-09T06:36:05.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.553+0000 7fadd159a700 1 -- 172.21.15.163:0/407441872 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc4014650 con 0x7fadbc0052e0 2023-10-09T06:36:05.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.554+0000 7fadd359e700 1 -- 172.21.15.163:0/407441872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadbc0052e0 msgr2=0x7fadbc00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:05.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.554+0000 7fadd359e700 1 --2- 172.21.15.163:0/407441872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadbc0052e0 0x7fadbc00ba60 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fadc4007580 tx=0x7fadc40097d0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.554+0000 7fadd359e700 1 -- 172.21.15.163:0/407441872 shutdown_connections 2023-10-09T06:36:05.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.554+0000 7fadd359e700 1 --2- 172.21.15.163:0/407441872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadbc0052e0 0x7fadbc00ba60 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.554+0000 7fadd359e700 1 -- 172.21.15.163:0/407441872 >> 172.21.15.163:0/407441872 conn(0x7fadd40d0660 msgr2=0x7fadd40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:05.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.554+0000 7fadd359e700 1 -- 172.21.15.163:0/407441872 shutdown_connections 2023-10-09T06:36:05.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.554+0000 7fadd359e700 1 -- 172.21.15.163:0/407441872 wait complete. 2023-10-09T06:36:05.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.555+0000 7fadd359e700 1 Processor -- start 2023-10-09T06:36:05.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.555+0000 7fadd359e700 1 -- start start 2023-10-09T06:36:05.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.556+0000 7fadd359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadbc0052e0 0x7fadd40c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:05.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.556+0000 7fadd359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadd40c56c0 0x7fadd4154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:05.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.556+0000 7fadd359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadd4155c30 0x7fadd4156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:05.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.556+0000 7fadd359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd40c9a30 con 0x7fadd40c56c0 2023-10-09T06:36:05.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.556+0000 7fadd359e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd41565c0 con 0x7fadd4155c30 2023-10-09T06:36:05.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.556+0000 7fadd359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd40c5f10 con 0x7fadbc0052e0 2023-10-09T06:36:05.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.556+0000 7fadd1d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadd40c56c0 0x7fadd4154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:05.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.556+0000 7fadd1d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadd40c56c0 0x7fadd4154840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40646/0 (socket says 172.21.15.163:40646) 2023-10-09T06:36:05.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.556+0000 7fadd1d9b700 1 -- 172.21.15.163:0/3191049450 learned_addr learned my addr 172.21.15.163:0/3191049450 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:05.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.556+0000 7fadd259c700 1 --2- 172.21.15.163:0/3191049450 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadbc0052e0 0x7fadd40c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:05.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.556+0000 7fadd2d9d700 1 --2- 172.21.15.163:0/3191049450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadd4155c30 0x7fadd4156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:05.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.556+0000 7fadd1d9b700 1 -- 172.21.15.163:0/3191049450 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadbc0052e0 msgr2=0x7fadd40c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:05.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.556+0000 7fadd1d9b700 1 --2- 172.21.15.163:0/3191049450 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadbc0052e0 0x7fadd40c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.556+0000 7fadd1d9b700 1 -- 172.21.15.163:0/3191049450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadd4155c30 msgr2=0x7fadd4156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:05.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.556+0000 7fadd1d9b700 1 --2- 172.21.15.163:0/3191049450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadd4155c30 0x7fadd4156080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.556+0000 7fadd1d9b700 1 -- 172.21.15.163:0/3191049450 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd40c61f0 con 0x7fadd40c56c0 2023-10-09T06:36:05.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.556+0000 7fadd2d9d700 1 --2- 172.21.15.163:0/3191049450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadd4155c30 0x7fadd4156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:05.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.556+0000 7fadd1d9b700 1 --2- 172.21.15.163:0/3191049450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadd40c56c0 0x7fadd4154840 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fadc800e580 tx=0x7fadc8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:05.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.556+0000 7fadc27fc700 1 -- 172.21.15.163:0/3191049450 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc8011460 con 0x7fadd40c56c0 2023-10-09T06:36:05.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.556+0000 7fadc27fc700 1 -- 172.21.15.163:0/3191049450 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc8010040 con 0x7fadd40c56c0 2023-10-09T06:36:05.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.556+0000 7fadd359e700 1 -- 172.21.15.163:0/3191049450 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd40c64d0 con 0x7fadd40c56c0 2023-10-09T06:36:05.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.556+0000 7fadc27fc700 1 -- 172.21.15.163:0/3191049450 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc80117f0 con 0x7fadd40c56c0 2023-10-09T06:36:05.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.556+0000 7fadd359e700 1 -- 172.21.15.163:0/3191049450 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd4154d80 con 0x7fadd40c56c0 2023-10-09T06:36:05.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.557+0000 7fadd359e700 1 -- 172.21.15.163:0/3191049450 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd40c9800 con 0x7fadd40c56c0 2023-10-09T06:36:05.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.558+0000 7fadc27fc700 1 -- 172.21.15.163:0/3191049450 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fadc8011be0 con 0x7fadd40c56c0 2023-10-09T06:36:05.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.558+0000 7fadc27fc700 1 --2- 172.21.15.163:0/3191049450 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fadb4067200 0x7fadb40696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:05.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.558+0000 7fadc27fc700 1 -- 172.21.15.163:0/3191049450 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fadc8083c20 con 0x7fadd40c56c0 2023-10-09T06:36:05.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.559+0000 7fadd259c700 1 --2- 172.21.15.163:0/3191049450 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fadb4067200 0x7fadb40696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:05.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.559+0000 7fadd259c700 1 --2- 172.21.15.163:0/3191049450 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fadb4067200 0x7fadb40696b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fadcc0015a0 tx=0x7fadd415dd00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:05.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.562+0000 7fadc27fc700 1 -- 172.21.15.163:0/3191049450 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fadc804d5a0 con 0x7fadd40c56c0 2023-10-09T06:36:05.750 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.749+0000 7fadd359e700 1 -- 172.21.15.163:0/3191049450 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fadd404c3c0 con 0x7fadd40c56c0 2023-10-09T06:36:05.750 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.750+0000 7fadc27fc700 1 -- 172.21.15.163:0/3191049450 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+15401 (secure 0 0 0) 0x7fadd40c9800 con 0x7fadd40c56c0 2023-10-09T06:36:05.751 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:05.751 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":77,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:35:53.645012+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:35:50.332005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-09T06:35:50.969956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:36:05.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.752+0000 7fadd359e700 1 -- 172.21.15.163:0/3191049450 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fadb4067200 msgr2=0x7fadb40696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:05.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.752+0000 7fadd359e700 1 --2- 172.21.15.163:0/3191049450 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fadb4067200 0x7fadb40696b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fadcc0015a0 tx=0x7fadd415dd00 comp rx=0 tx=0).stop 2023-10-09T06:36:05.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.752+0000 7fadd359e700 1 -- 172.21.15.163:0/3191049450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadd40c56c0 msgr2=0x7fadd4154840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:05.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.752+0000 7fadd359e700 1 --2- 172.21.15.163:0/3191049450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadd40c56c0 0x7fadd4154840 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fadc800e580 tx=0x7fadc8009040 comp rx=0 tx=0).stop 2023-10-09T06:36:05.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.752+0000 7fadd359e700 1 -- 172.21.15.163:0/3191049450 shutdown_connections 2023-10-09T06:36:05.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.752+0000 7fadd359e700 1 --2- 172.21.15.163:0/3191049450 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fadb4067200 0x7fadb40696b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.752+0000 7fadd359e700 1 --2- 172.21.15.163:0/3191049450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadd4155c30 0x7fadd4156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.752+0000 7fadd359e700 1 --2- 172.21.15.163:0/3191049450 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadbc0052e0 0x7fadd40c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.754 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.752+0000 7fadd359e700 1 --2- 172.21.15.163:0/3191049450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadd40c56c0 0x7fadd4154840 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.754 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.752+0000 7fadd359e700 1 -- 172.21.15.163:0/3191049450 >> 172.21.15.163:0/3191049450 conn(0x7fadd40d0660 msgr2=0x7fadd41551b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:05.754 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.752+0000 7fadd359e700 1 -- 172.21.15.163:0/3191049450 shutdown_connections 2023-10-09T06:36:05.754 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.752+0000 7fadd359e700 1 -- 172.21.15.163:0/3191049450 wait complete. 2023-10-09T06:36:05.765 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-09T06:36:05.766 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:36:05.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.879+0000 7fcf3fb77700 1 Processor -- start 2023-10-09T06:36:05.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.879+0000 7fcf3fb77700 1 -- start start 2023-10-09T06:36:05.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.880+0000 7fcf3fb77700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcf3806c0a0 0x7fcf380caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:05.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.880+0000 7fcf3fb77700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf3815d7e0 con 0x7fcf3815df50 2023-10-09T06:36:05.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.880+0000 7fcf3fb77700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf3815dac0 con 0x7fcf3806bcd0 2023-10-09T06:36:05.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.880+0000 7fcf3fb77700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3815aa30 con 0x7fcf3806c0a0 2023-10-09T06:36:05.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.880+0000 7fcf3e114700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fcf3815df50 0x7fcf380c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60696/0 (socket says 172.21.15.163:60696) 2023-10-09T06:36:05.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.880+0000 7fcf3e114700 1 -- 172.21.15.163:0/746639312 learned_addr learned my addr 172.21.15.163:0/746639312 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:05.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.881+0000 7fcf3d112700 1 --2- 172.21.15.163:0/746639312 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcf3806c0a0 0x7fcf380caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:05.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.881+0000 7fcf3c911700 1 -- 172.21.15.163:0/746639312 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4075869140 0 0) 0x7fcf3815d7e0 con 0x7fcf3815df50 2023-10-09T06:36:05.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.881+0000 7fcf3c911700 1 -- 172.21.15.163:0/746639312 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf24003570 con 0x7fcf3815df50 2023-10-09T06:36:05.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.881+0000 7fcf3c911700 1 -- 172.21.15.163:0/746639312 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1128858424 0 0) 0x7fcf24003570 con 0x7fcf3815df50 2023-10-09T06:36:05.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.881+0000 7fcf3c911700 1 -- 172.21.15.163:0/746639312 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcf3806c0a0 msgr2=0x7fcf380caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:05.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.881+0000 7fcf3c911700 1 --2- 172.21.15.163:0/746639312 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcf3806c0a0 0x7fcf380caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.881+0000 7fcf3c911700 1 -- 172.21.15.163:0/746639312 >> v1:172.21.15.181:6789/0 conn(0x7fcf3806bcd0 legacy=0x7fcf3815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:05.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.881+0000 7fcf3c911700 1 -- 172.21.15.163:0/746639312 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3815ad10 con 0x7fcf3815df50 2023-10-09T06:36:05.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.881+0000 7fcf3c911700 1 -- 172.21.15.163:0/746639312 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fcf30001470 con 0x7fcf3815df50 2023-10-09T06:36:05.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.881+0000 7fcf3c911700 1 -- 172.21.15.163:0/746639312 >> v1:172.21.15.163:6789/0 conn(0x7fcf3815df50 legacy=0x7fcf380c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:05.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.881+0000 7fcf3c911700 1 --2- 172.21.15.163:0/746639312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf24003a90 0x7fcf24003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:05.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.881+0000 7fcf3c911700 1 --2- 172.21.15.163:0/746639312 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcf3806c0a0 0x7fcf24004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:05.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.881+0000 7fcf3c911700 1 --2- 172.21.15.163:0/746639312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf24004d50 0x7fcf240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:05.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.881+0000 7fcf3c911700 1 -- 172.21.15.163:0/746639312 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3815aa30 con 0x7fcf24004d50 2023-10-09T06:36:05.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.881+0000 7fcf3c911700 1 -- 172.21.15.163:0/746639312 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2400bb70 con 0x7fcf24003a90 2023-10-09T06:36:05.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.881+0000 7fcf3c911700 1 -- 172.21.15.163:0/746639312 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2400bd30 con 0x7fcf3806c0a0 2023-10-09T06:36:05.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.881+0000 7fcf3e114700 1 --2- 172.21.15.163:0/746639312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf24004d50 0x7fcf240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:05.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.881+0000 7fcf3d112700 1 --2- 172.21.15.163:0/746639312 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcf3806c0a0 0x7fcf24004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:05.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.881+0000 7fcf3d913700 1 --2- 172.21.15.163:0/746639312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf24003a90 0x7fcf24003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:05.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.881+0000 7fcf3e114700 1 -- 172.21.15.163:0/746639312 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcf3806c0a0 msgr2=0x7fcf24004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:05.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.881+0000 7fcf3e114700 1 --2- 172.21.15.163:0/746639312 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcf3806c0a0 0x7fcf24004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.881+0000 7fcf3e114700 1 -- 172.21.15.163:0/746639312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf24003a90 msgr2=0x7fcf24003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:05.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.881+0000 7fcf3e114700 1 --2- 172.21.15.163:0/746639312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf24003a90 0x7fcf24003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.881+0000 7fcf3e114700 1 -- 172.21.15.163:0/746639312 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf24003570 con 0x7fcf24004d50 2023-10-09T06:36:05.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.882+0000 7fcf3e114700 1 --2- 172.21.15.163:0/746639312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf24004d50 0x7fcf240051c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000c390 tx=0x7fcf380ce4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce81152a3f280073 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:05.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.882+0000 7fcf3c911700 1 -- 172.21.15.163:0/746639312 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3000cc60 con 0x7fcf24004d50 2023-10-09T06:36:05.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.882+0000 7fcf3c911700 1 -- 172.21.15.163:0/746639312 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3000ce00 con 0x7fcf24004d50 2023-10-09T06:36:05.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.882+0000 7fcf3c911700 1 -- 172.21.15.163:0/746639312 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf30005b70 con 0x7fcf24004d50 2023-10-09T06:36:05.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.882+0000 7fcf3fb77700 1 -- 172.21.15.163:0/746639312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf24004d50 msgr2=0x7fcf240051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:05.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.882+0000 7fcf3fb77700 1 --2- 172.21.15.163:0/746639312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf24004d50 0x7fcf240051c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000c390 tx=0x7fcf380ce4d0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.882+0000 7fcf3e114700 1 -- 172.21.15.163:0/746639312 reap_dead start 2023-10-09T06:36:05.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.882+0000 7fcf3fb77700 1 -- 172.21.15.163:0/746639312 shutdown_connections 2023-10-09T06:36:05.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.882+0000 7fcf3fb77700 1 -- 172.21.15.163:0/746639312 >> 172.21.15.163:0/746639312 conn(0x7fcf380ce600 msgr2=0x7fcf380cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:05.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.882+0000 7fcf3fb77700 1 -- 172.21.15.163:0/746639312 shutdown_connections 2023-10-09T06:36:05.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.882+0000 7fcf3fb77700 1 -- 172.21.15.163:0/746639312 wait complete. 2023-10-09T06:36:05.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.883+0000 7fcf3fb77700 1 Processor -- start 2023-10-09T06:36:05.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.883+0000 7fcf3fb77700 1 -- start start 2023-10-09T06:36:05.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.884+0000 7fcf3fb77700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf24004d50 0x7fcf380b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:05.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.884+0000 7fcf3fb77700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcf3816a2e0 0x7fcf38158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:05.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.884+0000 7fcf3fb77700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf3806a3c0 0x7fcf3806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:05.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.884+0000 7fcf3fb77700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3806ad80 con 0x7fcf24004d50 2023-10-09T06:36:05.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.884+0000 7fcf3fb77700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3816ae10 con 0x7fcf3806a3c0 2023-10-09T06:36:05.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.884+0000 7fcf3fb77700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3816b0f0 con 0x7fcf3816a2e0 2023-10-09T06:36:05.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.884+0000 7fcf3d913700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf24004d50 0x7fcf380b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:05.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.884+0000 7fcf3d913700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf24004d50 0x7fcf380b18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40670/0 (socket says 172.21.15.163:40670) 2023-10-09T06:36:05.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.884+0000 7fcf3d913700 1 -- 172.21.15.163:0/802312792 learned_addr learned my addr 172.21.15.163:0/802312792 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:05.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.884+0000 7fcf3e114700 1 --2- 172.21.15.163:0/802312792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf3806a3c0 0x7fcf3806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:05.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.884+0000 7fcf3d112700 1 --2- 172.21.15.163:0/802312792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcf3816a2e0 0x7fcf38158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:05.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.884+0000 7fcf3d913700 1 -- 172.21.15.163:0/802312792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcf3816a2e0 msgr2=0x7fcf38158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:05.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.885+0000 7fcf3d913700 1 --2- 172.21.15.163:0/802312792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcf3816a2e0 0x7fcf38158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.885+0000 7fcf3d913700 1 -- 172.21.15.163:0/802312792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf3806a3c0 msgr2=0x7fcf3806a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:05.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.885+0000 7fcf3d913700 1 --2- 172.21.15.163:0/802312792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf3806a3c0 0x7fcf3806a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:05.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.885+0000 7fcf3d913700 1 -- 172.21.15.163:0/802312792 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3816b3d0 con 0x7fcf24004d50 2023-10-09T06:36:05.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.885+0000 7fcf3e114700 1 --2- 172.21.15.163:0/802312792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf3806a3c0 0x7fcf3806a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:05.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.885+0000 7fcf3d112700 1 --2- 172.21.15.163:0/802312792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcf3816a2e0 0x7fcf38158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:05.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.885+0000 7fcf3d913700 1 --2- 172.21.15.163:0/802312792 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf24004d50 0x7fcf380b18e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fcf3400ebc0 tx=0x7fcf340016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:05.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.885+0000 7fcf2dffb700 1 -- 172.21.15.163:0/802312792 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3400c890 con 0x7fcf24004d50 2023-10-09T06:36:05.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.885+0000 7fcf2dffb700 1 -- 172.21.15.163:0/802312792 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3400c9f0 con 0x7fcf24004d50 2023-10-09T06:36:05.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.885+0000 7fcf3fb77700 1 -- 172.21.15.163:0/802312792 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf3816b6b0 con 0x7fcf24004d50 2023-10-09T06:36:05.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.885+0000 7fcf2dffb700 1 -- 172.21.15.163:0/802312792 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3400cdb0 con 0x7fcf24004d50 2023-10-09T06:36:05.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.885+0000 7fcf3fb77700 1 -- 172.21.15.163:0/802312792 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf38158660 con 0x7fcf24004d50 2023-10-09T06:36:05.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.886+0000 7fcf3fb77700 1 -- 172.21.15.163:0/802312792 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf3816bbd0 con 0x7fcf24004d50 2023-10-09T06:36:05.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.886+0000 7fcf2dffb700 1 -- 172.21.15.163:0/802312792 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fcf34015640 con 0x7fcf24004d50 2023-10-09T06:36:05.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.887+0000 7fcf2dffb700 1 --2- 172.21.15.163:0/802312792 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcf1c067200 0x7fcf1c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:05.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.887+0000 7fcf2dffb700 1 -- 172.21.15.163:0/802312792 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcf34085200 con 0x7fcf24004d50 2023-10-09T06:36:05.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.887+0000 7fcf3d112700 1 --2- 172.21.15.163:0/802312792 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcf1c067200 0x7fcf1c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:05.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.888+0000 7fcf3d112700 1 --2- 172.21.15.163:0/802312792 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcf1c067200 0x7fcf1c0696b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcf28006f00 tx=0x7fcf28008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:05.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:05.891+0000 7fcf2dffb700 1 -- 172.21.15.163:0/802312792 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcf3816bbd0 con 0x7fcf24004d50 2023-10-09T06:36:06.115 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.114+0000 7fcf3fb77700 1 -- 172.21.15.163:0/802312792 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf3816bbd0 con 0x7fcf24004d50 2023-10-09T06:36:06.116 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.115+0000 7fcf2dffb700 1 -- 172.21.15.163:0/802312792 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4608 (secure 0 0 0) 0x7fcf3816bbd0 con 0x7fcf24004d50 2023-10-09T06:36:06.116 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:06.116 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1946291256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1946291256},{"type":"v1","addr":"172.21.15.181:6837","nonce":1946291256}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5749,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1366951098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1366951098},{"type":"v1","addr":"172.21.15.181:6836","nonce":1366951098}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3619934997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3619934997},{"type":"v1","addr":"172.21.15.163:6834","nonce":3619934997}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:35:52.002698+0000","modified":"2023-10-09T06:35:55.653628+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/234029086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":234029086},{"type":"v1","addr":"172.21.15.163:6835","nonce":234029086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-09T06:36:06.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.117+0000 7fcf3fb77700 1 -- 172.21.15.163:0/802312792 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcf1c067200 msgr2=0x7fcf1c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:06.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.117+0000 7fcf3fb77700 1 --2- 172.21.15.163:0/802312792 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcf1c067200 0x7fcf1c0696b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcf28006f00 tx=0x7fcf28008040 comp rx=0 tx=0).stop 2023-10-09T06:36:06.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.117+0000 7fcf3fb77700 1 -- 172.21.15.163:0/802312792 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf24004d50 msgr2=0x7fcf380b18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:06.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.117+0000 7fcf3fb77700 1 --2- 172.21.15.163:0/802312792 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf24004d50 0x7fcf380b18e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fcf3400ebc0 tx=0x7fcf340016d0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.117+0000 7fcf3fb77700 1 -- 172.21.15.163:0/802312792 shutdown_connections 2023-10-09T06:36:06.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.117+0000 7fcf3fb77700 1 --2- 172.21.15.163:0/802312792 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcf1c067200 0x7fcf1c0696b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.117+0000 7fcf3fb77700 1 --2- 172.21.15.163:0/802312792 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf24004d50 0x7fcf380b18e0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.117+0000 7fcf3fb77700 1 --2- 172.21.15.163:0/802312792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf3806a3c0 0x7fcf3806a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.117+0000 7fcf3fb77700 1 --2- 172.21.15.163:0/802312792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcf3816a2e0 0x7fcf38158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.117+0000 7fcf3fb77700 1 -- 172.21.15.163:0/802312792 >> 172.21.15.163:0/802312792 conn(0x7fcf380ce600 msgr2=0x7fcf38159430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:06.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.117+0000 7fcf3fb77700 1 -- 172.21.15.163:0/802312792 shutdown_connections 2023-10-09T06:36:06.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.117+0000 7fcf3fb77700 1 -- 172.21.15.163:0/802312792 wait complete. 2023-10-09T06:36:06.120 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 49 2023-10-09T06:36:06.130 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:36:06.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.244+0000 7f55c2695700 1 Processor -- start 2023-10-09T06:36:06.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.244+0000 7f55c2695700 1 -- start start 2023-10-09T06:36:06.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.245+0000 7f55c2695700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f55bc159c60 0x7f55bc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:06.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.245+0000 7f55c2695700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55bc15db10 0x7f55bc069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:06.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.245+0000 7f55c2695700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55bc15a160 con 0x7f55bc159890 2023-10-09T06:36:06.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.245+0000 7f55c2695700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55bc15df50 con 0x7f55bc159c60 2023-10-09T06:36:06.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.245+0000 7f55c2695700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc0d13a0 con 0x7f55bc15db10 2023-10-09T06:36:06.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.245+0000 7f55bbfff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f55bc159890 0x7f55bc15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60704/0 (socket says 172.21.15.163:60704) 2023-10-09T06:36:06.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.245+0000 7f55bbfff700 1 -- 172.21.15.163:0/1395101759 learned_addr learned my addr 172.21.15.163:0/1395101759 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:06.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.246+0000 7f55baffd700 1 -- 172.21.15.163:0/1395101759 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3533909468 0 0) 0x7f55bc15a160 con 0x7f55bc159890 2023-10-09T06:36:06.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.246+0000 7f55baffd700 1 -- 172.21.15.163:0/1395101759 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55a4003570 con 0x7f55bc159890 2023-10-09T06:36:06.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.246+0000 7f55c0c32700 1 --2- 172.21.15.163:0/1395101759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55bc15db10 0x7f55bc069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:06.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.246+0000 7f55bb7fe700 1 --2- 172.21.15.163:0/1395101759 >> v2:172.21.15.181:3300/0 conn(0x7f55bc159c60 0x7f55bc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:06.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.246+0000 7f55baffd700 1 -- 172.21.15.163:0/1395101759 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1610226911 0 0) 0x7f55a4003570 con 0x7f55bc159890 2023-10-09T06:36:06.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.246+0000 7f55baffd700 1 -- 172.21.15.163:0/1395101759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55bc15db10 msgr2=0x7f55bc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:06.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.246+0000 7f55baffd700 1 --2- 172.21.15.163:0/1395101759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55bc15db10 0x7f55bc069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.246+0000 7f55baffd700 1 -- 172.21.15.163:0/1395101759 >> v2:172.21.15.181:3300/0 conn(0x7f55bc159c60 msgr2=0x7f55bc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:06.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.246+0000 7f55baffd700 1 --2- 172.21.15.163:0/1395101759 >> v2:172.21.15.181:3300/0 conn(0x7f55bc159c60 0x7f55bc15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.246+0000 7f55baffd700 1 -- 172.21.15.163:0/1395101759 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55bc15e140 con 0x7f55bc159890 2023-10-09T06:36:06.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.246+0000 7f55baffd700 1 -- 172.21.15.163:0/1395101759 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f55ac001470 con 0x7f55bc159890 2023-10-09T06:36:06.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.246+0000 7f55baffd700 1 -- 172.21.15.163:0/1395101759 >> v1:172.21.15.163:6789/0 conn(0x7f55bc159890 legacy=0x7f55bc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:06.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.246+0000 7f55baffd700 1 --2- 172.21.15.163:0/1395101759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55a4003ae0 0x7f55a4003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:06.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.246+0000 7f55baffd700 1 --2- 172.21.15.163:0/1395101759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55a4004810 0x7f55a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:06.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.246+0000 7f55baffd700 1 --2- 172.21.15.163:0/1395101759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55bc15db10 0x7f55a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:06.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.246+0000 7f55baffd700 1 -- 172.21.15.163:0/1395101759 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc0d13a0 con 0x7f55a4004810 2023-10-09T06:36:06.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.246+0000 7f55baffd700 1 -- 172.21.15.163:0/1395101759 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a400bac0 con 0x7f55a4003ae0 2023-10-09T06:36:06.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.246+0000 7f55baffd700 1 -- 172.21.15.163:0/1395101759 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a400bc80 con 0x7f55bc15db10 2023-10-09T06:36:06.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.246+0000 7f55bb7fe700 1 --2- 172.21.15.163:0/1395101759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55a4004810 0x7f55a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:06.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.246+0000 7f55bbfff700 1 --2- 172.21.15.163:0/1395101759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55a4003ae0 0x7f55a4003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:06.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.246+0000 7f55c0c32700 1 --2- 172.21.15.163:0/1395101759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55bc15db10 0x7f55a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:06.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.246+0000 7f55bb7fe700 1 -- 172.21.15.163:0/1395101759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55bc15db10 msgr2=0x7f55a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:06.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.246+0000 7f55bb7fe700 1 --2- 172.21.15.163:0/1395101759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55bc15db10 0x7f55a40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.246+0000 7f55bb7fe700 1 -- 172.21.15.163:0/1395101759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55a4003ae0 msgr2=0x7f55a4003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:06.256 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.246+0000 7f55bb7fe700 1 --2- 172.21.15.163:0/1395101759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55a4003ae0 0x7f55a4003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.256 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.246+0000 7f55bb7fe700 1 -- 172.21.15.163:0/1395101759 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a4003570 con 0x7f55a4004810 2023-10-09T06:36:06.256 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.247+0000 7f55bb7fe700 1 --2- 172.21.15.163:0/1395101759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55a4004810 0x7f55a4004c80 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f55b0009c00 tx=0x7f55b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32536bc9bc463d50 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:06.256 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.247+0000 7f55baffd700 1 -- 172.21.15.163:0/1395101759 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b000e710 con 0x7f55a4004810 2023-10-09T06:36:06.256 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.247+0000 7f55baffd700 1 -- 172.21.15.163:0/1395101759 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b000e870 con 0x7f55a4004810 2023-10-09T06:36:06.256 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.247+0000 7f55baffd700 1 -- 172.21.15.163:0/1395101759 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b000ec30 con 0x7f55a4004810 2023-10-09T06:36:06.256 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.247+0000 7f55c2695700 1 -- 172.21.15.163:0/1395101759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55a4004810 msgr2=0x7f55a4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:06.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.247+0000 7f55c2695700 1 --2- 172.21.15.163:0/1395101759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55a4004810 0x7f55a4004c80 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f55b0009c00 tx=0x7f55b00091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.247+0000 7f55c0c32700 1 -- 172.21.15.163:0/1395101759 reap_dead start 2023-10-09T06:36:06.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.247+0000 7f55c2695700 1 -- 172.21.15.163:0/1395101759 shutdown_connections 2023-10-09T06:36:06.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.247+0000 7f55c2695700 1 -- 172.21.15.163:0/1395101759 >> 172.21.15.163:0/1395101759 conn(0x7f55bc068e20 msgr2=0x7f55bc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:06.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.247+0000 7f55c2695700 1 -- 172.21.15.163:0/1395101759 shutdown_connections 2023-10-09T06:36:06.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.247+0000 7f55c2695700 1 -- 172.21.15.163:0/1395101759 wait complete. 2023-10-09T06:36:06.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.248+0000 7f55c2695700 1 Processor -- start 2023-10-09T06:36:06.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.248+0000 7f55c2695700 1 -- start start 2023-10-09T06:36:06.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.249+0000 7f55c2695700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55a4004810 0x7f55bc151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:06.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.249+0000 7f55c2695700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55bc15ec60 0x7f55bc14c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:06.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.249+0000 7f55c2695700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55bc0cdd80 0x7f55bc0ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:06.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.249+0000 7f55c2695700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc0b36c0 con 0x7f55a4004810 2023-10-09T06:36:06.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.249+0000 7f55c2695700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc0ce730 con 0x7f55bc15ec60 2023-10-09T06:36:06.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.249+0000 7f55c2695700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc15f0d0 con 0x7f55bc0cdd80 2023-10-09T06:36:06.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.249+0000 7f55bbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55a4004810 0x7f55bc151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:06.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.249+0000 7f55bbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55a4004810 0x7f55bc151c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40694/0 (socket says 172.21.15.163:40694) 2023-10-09T06:36:06.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.249+0000 7f55bbfff700 1 -- 172.21.15.163:0/3945435621 learned_addr learned my addr 172.21.15.163:0/3945435621 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:06.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.249+0000 7f55bb7fe700 1 --2- 172.21.15.163:0/3945435621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55bc15ec60 0x7f55bc14c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:06.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.249+0000 7f55c0c32700 1 --2- 172.21.15.163:0/3945435621 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55bc0cdd80 0x7f55bc0ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:06.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.249+0000 7f55bbfff700 1 -- 172.21.15.163:0/3945435621 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55bc0cdd80 msgr2=0x7f55bc0ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:06.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.250+0000 7f55bbfff700 1 --2- 172.21.15.163:0/3945435621 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55bc0cdd80 0x7f55bc0ce1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.250+0000 7f55bbfff700 1 -- 172.21.15.163:0/3945435621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55bc15ec60 msgr2=0x7f55bc14c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:06.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.250+0000 7f55bbfff700 1 --2- 172.21.15.163:0/3945435621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55bc15ec60 0x7f55bc14c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.250+0000 7f55bbfff700 1 -- 172.21.15.163:0/3945435621 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55bc1522e0 con 0x7f55a4004810 2023-10-09T06:36:06.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.250+0000 7f55bb7fe700 1 --2- 172.21.15.163:0/3945435621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55bc15ec60 0x7f55bc14c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:36:06.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.250+0000 7f55bbfff700 1 --2- 172.21.15.163:0/3945435621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55a4004810 0x7f55bc151c70 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f55ac007310 tx=0x7f55ac001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:06.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.250+0000 7f55a3fff700 1 -- 172.21.15.163:0/3945435621 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac018030 con 0x7f55a4004810 2023-10-09T06:36:06.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.250+0000 7f55a3fff700 1 -- 172.21.15.163:0/3945435621 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ac00f8d0 con 0x7f55a4004810 2023-10-09T06:36:06.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.250+0000 7f55a3fff700 1 -- 172.21.15.163:0/3945435621 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac00fc90 con 0x7f55a4004810 2023-10-09T06:36:06.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.250+0000 7f55c2695700 1 -- 172.21.15.163:0/3945435621 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55bc1525c0 con 0x7f55a4004810 2023-10-09T06:36:06.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.250+0000 7f55c2695700 1 -- 172.21.15.163:0/3945435621 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55bc152b80 con 0x7f55a4004810 2023-10-09T06:36:06.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.251+0000 7f55c2695700 1 -- 172.21.15.163:0/3945435621 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55bc04c3c0 con 0x7f55a4004810 2023-10-09T06:36:06.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.251+0000 7f55a3fff700 1 -- 172.21.15.163:0/3945435621 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f55ac01b070 con 0x7f55a4004810 2023-10-09T06:36:06.261 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.251+0000 7f55a3fff700 1 --2- 172.21.15.163:0/3945435621 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f55a8067230 0x7f55a80696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:06.261 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.252+0000 7f55a3fff700 1 -- 172.21.15.163:0/3945435621 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f55ac019070 con 0x7f55a4004810 2023-10-09T06:36:06.261 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.252+0000 7f55bb7fe700 1 --2- 172.21.15.163:0/3945435621 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f55a8067230 0x7f55a80696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:06.261 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.252+0000 7f55bb7fe700 1 --2- 172.21.15.163:0/3945435621 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f55a8067230 0x7f55a80696e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f55bc0b2660 tx=0x7f55b001a000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:06.261 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.256+0000 7f55a3fff700 1 -- 172.21.15.163:0/3945435621 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55ac0517d0 con 0x7f55a4004810 2023-10-09T06:36:06.481 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.480+0000 7f55c2695700 1 -- 172.21.15.163:0/3945435621 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55bc0cdb60 con 0x7f55a4004810 2023-10-09T06:36:06.481 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.481+0000 7f55a3fff700 1 -- 172.21.15.163:0/3945435621 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4608 (secure 0 0 0) 0x7f55ac054df0 con 0x7f55a4004810 2023-10-09T06:36:06.482 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:06.482 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1946291256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1946291256},{"type":"v1","addr":"172.21.15.181:6837","nonce":1946291256}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5749,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1366951098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1366951098},{"type":"v1","addr":"172.21.15.181:6836","nonce":1366951098}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3619934997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3619934997},{"type":"v1","addr":"172.21.15.163:6834","nonce":3619934997}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:35:52.002698+0000","modified":"2023-10-09T06:35:55.653628+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/234029086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":234029086},{"type":"v1","addr":"172.21.15.163:6835","nonce":234029086}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-09T06:36:06.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.483+0000 7f55c2695700 1 -- 172.21.15.163:0/3945435621 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f55a8067230 msgr2=0x7f55a80696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:06.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.483+0000 7f55c2695700 1 --2- 172.21.15.163:0/3945435621 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f55a8067230 0x7f55a80696e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f55bc0b2660 tx=0x7f55b001a000 comp rx=0 tx=0).stop 2023-10-09T06:36:06.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.483+0000 7f55c2695700 1 -- 172.21.15.163:0/3945435621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55a4004810 msgr2=0x7f55bc151c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:06.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.483+0000 7f55c2695700 1 --2- 172.21.15.163:0/3945435621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55a4004810 0x7f55bc151c70 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f55ac007310 tx=0x7f55ac001dc0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.483+0000 7f55c2695700 1 -- 172.21.15.163:0/3945435621 shutdown_connections 2023-10-09T06:36:06.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.483+0000 7f55c2695700 1 --2- 172.21.15.163:0/3945435621 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f55a8067230 0x7f55a80696e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.483+0000 7f55c2695700 1 --2- 172.21.15.163:0/3945435621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55a4004810 0x7f55bc151c70 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.483+0000 7f55c2695700 1 --2- 172.21.15.163:0/3945435621 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55bc0cdd80 0x7f55bc0ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.483+0000 7f55c2695700 1 --2- 172.21.15.163:0/3945435621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55bc15ec60 0x7f55bc14c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.483+0000 7f55c2695700 1 -- 172.21.15.163:0/3945435621 >> 172.21.15.163:0/3945435621 conn(0x7f55bc068e20 msgr2=0x7f55bc15aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:06.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.483+0000 7f55c2695700 1 -- 172.21.15.163:0/3945435621 shutdown_connections 2023-10-09T06:36:06.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.483+0000 7f55c2695700 1 -- 172.21.15.163:0/3945435621 wait complete. 2023-10-09T06:36:06.485 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 49 2023-10-09T06:36:06.496 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:36:06.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.610+0000 7f19cb3fe700 1 Processor -- start 2023-10-09T06:36:06.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.610+0000 7f19cb3fe700 1 -- start start 2023-10-09T06:36:06.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.611+0000 7f19cb3fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19c4159420 0x7f19c40b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:06.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.611+0000 7f19cb3fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f19c415d230 0x7f19c415d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:06.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.611+0000 7f19cb3fe700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19c415dbd0 con 0x7f19c41597f0 2023-10-09T06:36:06.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.611+0000 7f19cb3fe700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c40b2980 con 0x7f19c415d230 2023-10-09T06:36:06.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.611+0000 7f19cb3fe700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c40b11d0 con 0x7f19c4159420 2023-10-09T06:36:06.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.611+0000 7f19cabfd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f19c415d230 0x7f19c415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:06.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.611+0000 7f19ca3fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19c4159420 0x7f19c40b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:06.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.611+0000 7f19c9bfb700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f19c41597f0 0x7f19c40b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:57346/0 (socket says 172.21.15.163:57346) 2023-10-09T06:36:06.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.611+0000 7f19cabfd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f19c415d230 0x7f19c415d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:49144/0 (socket says 172.21.15.163:49144) 2023-10-09T06:36:06.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.611+0000 7f19ca3fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19c4159420 0x7f19c40b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:40092/0 (socket says 172.21.15.163:40092) 2023-10-09T06:36:06.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.611+0000 7f19c9bfb700 1 -- 172.21.15.163:0/3609769293 learned_addr learned my addr 172.21.15.163:0/3609769293 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:06.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.612+0000 7f19cabfd700 1 -- 172.21.15.163:0/3609769293 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19c4159420 msgr2=0x7f19c40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:06.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.612+0000 7f19cabfd700 1 --2- 172.21.15.163:0/3609769293 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19c4159420 0x7f19c40b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.612+0000 7f19cabfd700 1 -- 172.21.15.163:0/3609769293 >> v1:172.21.15.181:6789/0 conn(0x7f19c41597f0 legacy=0x7f19c40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:06.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.612+0000 7f19cabfd700 1 -- 172.21.15.163:0/3609769293 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c40b14b0 con 0x7f19c415d230 2023-10-09T06:36:06.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.612+0000 7f19c93fa700 1 -- 172.21.15.163:0/3609769293 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 650852808 0 0) 0x7f19b4001110 con 0x7f19c41597f0 2023-10-09T06:36:06.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.612+0000 7f19ca3fc700 1 --2- 172.21.15.163:0/3609769293 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19c4159420 0x7f19c40b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:36:06.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.612+0000 7f19cabfd700 1 --2- 172.21.15.163:0/3609769293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19c415d230 0x7f19c415d690 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f19bc0098e0 tx=0x7f19bc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c8a0529808a6b7b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:06.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.612+0000 7f19c93fa700 1 -- 172.21.15.163:0/3609769293 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19bc00b640 con 0x7f19c415d230 2023-10-09T06:36:06.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.612+0000 7f19c93fa700 1 -- 172.21.15.163:0/3609769293 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19bc00b7a0 con 0x7f19c415d230 2023-10-09T06:36:06.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.612+0000 7f19c93fa700 1 -- 172.21.15.163:0/3609769293 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19bc00bb40 con 0x7f19c415d230 2023-10-09T06:36:06.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.613+0000 7f19cb3fe700 1 -- 172.21.15.163:0/3609769293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19c415d230 msgr2=0x7f19c415d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:06.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.613+0000 7f19cb3fe700 1 --2- 172.21.15.163:0/3609769293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19c415d230 0x7f19c415d690 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f19bc0098e0 tx=0x7f19bc0091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.613+0000 7f19cb3fe700 1 -- 172.21.15.163:0/3609769293 shutdown_connections 2023-10-09T06:36:06.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.613+0000 7f19cb3fe700 1 --2- 172.21.15.163:0/3609769293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19c415d230 0x7f19c415d690 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.613+0000 7f19cb3fe700 1 --2- 172.21.15.163:0/3609769293 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19c4159420 0x7f19c40b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.613+0000 7f19cb3fe700 1 -- 172.21.15.163:0/3609769293 >> 172.21.15.163:0/3609769293 conn(0x7f19c40d0660 msgr2=0x7f19c40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:06.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.613+0000 7f19cb3fe700 1 -- 172.21.15.163:0/3609769293 shutdown_connections 2023-10-09T06:36:06.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.613+0000 7f19cb3fe700 1 -- 172.21.15.163:0/3609769293 wait complete. 2023-10-09T06:36:06.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.614+0000 7f19cb3fe700 1 Processor -- start 2023-10-09T06:36:06.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.614+0000 7f19cb3fe700 1 -- start start 2023-10-09T06:36:06.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.614+0000 7f19cb3fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19c4159420 0x7f19c415c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:06.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.615+0000 7f19cb3fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19c41597f0 0x7f19c4159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:06.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.615+0000 7f19cb3fe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19c415d230 0x7f19c40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:06.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.615+0000 7f19cb3fe700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c416a060 con 0x7f19c4159420 2023-10-09T06:36:06.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.615+0000 7f19cb3fe700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c416a340 con 0x7f19c415d230 2023-10-09T06:36:06.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.615+0000 7f19cb3fe700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c416a620 con 0x7f19c41597f0 2023-10-09T06:36:06.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.615+0000 7f19ca3fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19c4159420 0x7f19c415c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:06.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.615+0000 7f19ca3fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19c4159420 0x7f19c415c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40704/0 (socket says 172.21.15.163:40704) 2023-10-09T06:36:06.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.615+0000 7f19ca3fc700 1 -- 172.21.15.163:0/854193940 learned_addr learned my addr 172.21.15.163:0/854193940 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:06.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.615+0000 7f19c9bfb700 1 --2- 172.21.15.163:0/854193940 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19c41597f0 0x7f19c4159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:06.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.615+0000 7f19cabfd700 1 --2- 172.21.15.163:0/854193940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19c415d230 0x7f19c40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:06.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.615+0000 7f19ca3fc700 1 -- 172.21.15.163:0/854193940 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19c41597f0 msgr2=0x7f19c4159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:06.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.615+0000 7f19ca3fc700 1 --2- 172.21.15.163:0/854193940 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19c41597f0 0x7f19c4159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.615+0000 7f19ca3fc700 1 -- 172.21.15.163:0/854193940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19c415d230 msgr2=0x7f19c40b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:06.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.615+0000 7f19ca3fc700 1 --2- 172.21.15.163:0/854193940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19c415d230 0x7f19c40b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.615+0000 7f19ca3fc700 1 -- 172.21.15.163:0/854193940 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c0009a20 con 0x7f19c4159420 2023-10-09T06:36:06.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.615+0000 7f19ca3fc700 1 --2- 172.21.15.163:0/854193940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19c4159420 0x7f19c415c870 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f19c000c170 tx=0x7f19c000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:06.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.615+0000 7f19ba7fc700 1 -- 172.21.15.163:0/854193940 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c000ccb0 con 0x7f19c4159420 2023-10-09T06:36:06.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.615+0000 7f19ba7fc700 1 -- 172.21.15.163:0/854193940 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c000f040 con 0x7f19c4159420 2023-10-09T06:36:06.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.615+0000 7f19cb3fe700 1 -- 172.21.15.163:0/854193940 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19bc009420 con 0x7f19c4159420 2023-10-09T06:36:06.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.615+0000 7f19ba7fc700 1 -- 172.21.15.163:0/854193940 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c0014480 con 0x7f19c4159420 2023-10-09T06:36:06.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.615+0000 7f19cb3fe700 1 -- 172.21.15.163:0/854193940 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c415aff0 con 0x7f19c4159420 2023-10-09T06:36:06.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.616+0000 7f19cb3fe700 1 -- 172.21.15.163:0/854193940 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19c404c3c0 con 0x7f19c4159420 2023-10-09T06:36:06.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.617+0000 7f19ba7fc700 1 -- 172.21.15.163:0/854193940 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f19c00147f0 con 0x7f19c4159420 2023-10-09T06:36:06.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.617+0000 7f19ba7fc700 1 --2- 172.21.15.163:0/854193940 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f19a8067200 0x7f19a80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:06.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.617+0000 7f19ba7fc700 1 -- 172.21.15.163:0/854193940 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f19c00857d0 con 0x7f19c4159420 2023-10-09T06:36:06.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.617+0000 7f19c9bfb700 1 --2- 172.21.15.163:0/854193940 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f19a8067200 0x7f19a80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:06.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.618+0000 7f19c9bfb700 1 --2- 172.21.15.163:0/854193940 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f19a8067200 0x7f19a80696b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f19b4006f30 tx=0x7f19b4007b90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:06.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.621+0000 7f19ba7fc700 1 -- 172.21.15.163:0/854193940 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f19c00525e0 con 0x7f19c4159420 2023-10-09T06:36:06.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.807+0000 7f19cb3fe700 1 -- 172.21.15.163:0/854193940 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19c40b1790 con 0x7f19c4159420 2023-10-09T06:36:06.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.808+0000 7f19ba7fc700 1 -- 172.21.15.163:0/854193940 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+15401 (secure 0 0 0) 0x7f19c0055c00 con 0x7f19c4159420 2023-10-09T06:36:06.808 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:06.809 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":77,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:35:53.645012+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:35:50.332005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-09T06:35:50.969956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:36:06.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.810+0000 7f19cb3fe700 1 -- 172.21.15.163:0/854193940 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f19a8067200 msgr2=0x7f19a80696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:06.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.810+0000 7f19cb3fe700 1 --2- 172.21.15.163:0/854193940 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f19a8067200 0x7f19a80696b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f19b4006f30 tx=0x7f19b4007b90 comp rx=0 tx=0).stop 2023-10-09T06:36:06.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.810+0000 7f19cb3fe700 1 -- 172.21.15.163:0/854193940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19c4159420 msgr2=0x7f19c415c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:06.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.810+0000 7f19cb3fe700 1 --2- 172.21.15.163:0/854193940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19c4159420 0x7f19c415c870 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f19c000c170 tx=0x7f19c000b040 comp rx=0 tx=0).stop 2023-10-09T06:36:06.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.810+0000 7f19cb3fe700 1 -- 172.21.15.163:0/854193940 shutdown_connections 2023-10-09T06:36:06.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.810+0000 7f19cb3fe700 1 --2- 172.21.15.163:0/854193940 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f19a8067200 0x7f19a80696b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.810+0000 7f19cb3fe700 1 --2- 172.21.15.163:0/854193940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19c4159420 0x7f19c415c870 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.810+0000 7f19cb3fe700 1 --2- 172.21.15.163:0/854193940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19c415d230 0x7f19c40b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.810+0000 7f19cb3fe700 1 --2- 172.21.15.163:0/854193940 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19c41597f0 0x7f19c4159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.810+0000 7f19cb3fe700 1 -- 172.21.15.163:0/854193940 >> 172.21.15.163:0/854193940 conn(0x7f19c40d0660 msgr2=0x7f19c40d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:06.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.810+0000 7f19cb3fe700 1 -- 172.21.15.163:0/854193940 shutdown_connections 2023-10-09T06:36:06.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.810+0000 7f19cb3fe700 1 -- 172.21.15.163:0/854193940 wait complete. 2023-10-09T06:36:06.824 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-09T06:36:06.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.939+0000 7ffb12d96700 1 Processor -- start 2023-10-09T06:36:06.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.939+0000 7ffb12d96700 1 -- start start 2023-10-09T06:36:06.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.940+0000 7ffb12d96700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb0c1597f0 0x7ffb0c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:06.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.940+0000 7ffb12d96700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ffb0c15d230 0x7ffb0c15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:06.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.940+0000 7ffb12d96700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb0c15dbd0 con 0x7ffb0c159420 2023-10-09T06:36:06.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.940+0000 7ffb12d96700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb0c0b11d0 con 0x7ffb0c15d230 2023-10-09T06:36:06.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.940+0000 7ffb12d96700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c0b14b0 con 0x7ffb0c1597f0 2023-10-09T06:36:06.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.940+0000 7ffb12595700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ffb0c15d230 0x7ffb0c15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:06.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.940+0000 7ffb12595700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ffb0c15d230 0x7ffb0c15d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40706/0 (socket says 172.21.15.163:40706) 2023-10-09T06:36:06.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.940+0000 7ffb12595700 1 -- 172.21.15.163:0/312284525 learned_addr learned my addr 172.21.15.163:0/312284525 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:06.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.940+0000 7ffb11593700 1 --2- 172.21.15.163:0/312284525 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb0c1597f0 0x7ffb0c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:06.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.940+0000 7ffb12595700 1 -- 172.21.15.163:0/312284525 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb0c1597f0 msgr2=0x7ffb0c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:06.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.940+0000 7ffb12595700 1 --2- 172.21.15.163:0/312284525 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb0c1597f0 0x7ffb0c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.940+0000 7ffb12595700 1 -- 172.21.15.163:0/312284525 >> v1:172.21.15.181:6789/0 conn(0x7ffb0c159420 legacy=0x7ffb0c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:06.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.940+0000 7ffb12595700 1 -- 172.21.15.163:0/312284525 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0c0b1790 con 0x7ffb0c15d230 2023-10-09T06:36:06.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.940+0000 7ffb12595700 1 --2- 172.21.15.163:0/312284525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb0c15d230 0x7ffb0c15d690 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ffb04009f20 tx=0x7ffb04009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae66d1bb07241ef server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:06.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.941+0000 7ffb10d92700 1 -- 172.21.15.163:0/312284525 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0400e070 con 0x7ffb0c15d230 2023-10-09T06:36:06.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.941+0000 7ffb10d92700 1 -- 172.21.15.163:0/312284525 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0400b9d0 con 0x7ffb0c15d230 2023-10-09T06:36:06.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.941+0000 7ffb10d92700 1 -- 172.21.15.163:0/312284525 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb040092e0 con 0x7ffb0c15d230 2023-10-09T06:36:06.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.941+0000 7ffb12d96700 1 -- 172.21.15.163:0/312284525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb0c15d230 msgr2=0x7ffb0c15d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:06.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.941+0000 7ffb12d96700 1 --2- 172.21.15.163:0/312284525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb0c15d230 0x7ffb0c15d690 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ffb04009f20 tx=0x7ffb04009840 comp rx=0 tx=0).stop 2023-10-09T06:36:06.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.941+0000 7ffb12d96700 1 -- 172.21.15.163:0/312284525 shutdown_connections 2023-10-09T06:36:06.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.941+0000 7ffb12d96700 1 --2- 172.21.15.163:0/312284525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb0c15d230 0x7ffb0c15d690 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.941+0000 7ffb12d96700 1 --2- 172.21.15.163:0/312284525 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb0c1597f0 0x7ffb0c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.941+0000 7ffb12d96700 1 -- 172.21.15.163:0/312284525 >> 172.21.15.163:0/312284525 conn(0x7ffb0c0d0660 msgr2=0x7ffb0c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:06.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.941+0000 7ffb12d96700 1 -- 172.21.15.163:0/312284525 shutdown_connections 2023-10-09T06:36:06.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.941+0000 7ffb12d96700 1 -- 172.21.15.163:0/312284525 wait complete. 2023-10-09T06:36:06.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.942+0000 7ffb12d96700 1 Processor -- start 2023-10-09T06:36:06.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.942+0000 7ffb12d96700 1 -- start start 2023-10-09T06:36:06.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.943+0000 7ffb12d96700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb0c159420 0x7ffb0c16a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:06.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.943+0000 7ffb12d96700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb0c1597f0 0x7ffb0c16c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:06.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.943+0000 7ffb12d96700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb0c15d230 0x7ffb0c15aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:06.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.943+0000 7ffb12d96700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c16ca90 con 0x7ffb0c1597f0 2023-10-09T06:36:06.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.943+0000 7ffb12d96700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c15b000 con 0x7ffb0c15d230 2023-10-09T06:36:06.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.943+0000 7ffb12d96700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c0b51b0 con 0x7ffb0c159420 2023-10-09T06:36:06.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.943+0000 7ffb11593700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb0c1597f0 0x7ffb0c16c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:06.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.943+0000 7ffb11593700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb0c1597f0 0x7ffb0c16c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40720/0 (socket says 172.21.15.163:40720) 2023-10-09T06:36:06.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.943+0000 7ffb11593700 1 -- 172.21.15.163:0/3426344478 learned_addr learned my addr 172.21.15.163:0/3426344478 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:06.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.943+0000 7ffb11593700 1 -- 172.21.15.163:0/3426344478 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb0c159420 msgr2=0x7ffb0c16a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:06.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.943+0000 7ffb11593700 1 --2- 172.21.15.163:0/3426344478 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb0c159420 0x7ffb0c16a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.943+0000 7ffb11593700 1 -- 172.21.15.163:0/3426344478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb0c15d230 msgr2=0x7ffb0c15aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:06.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.943+0000 7ffb11593700 1 --2- 172.21.15.163:0/3426344478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb0c15d230 0x7ffb0c15aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:06.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.943+0000 7ffb11593700 1 -- 172.21.15.163:0/3426344478 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb04009ae0 con 0x7ffb0c1597f0 2023-10-09T06:36:06.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.944+0000 7ffb11593700 1 --2- 172.21.15.163:0/3426344478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb0c1597f0 0x7ffb0c16c550 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00ecb0 tx=0x7ffafc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:06.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.944+0000 7ffb01ffb700 1 -- 172.21.15.163:0/3426344478 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc00c9d0 con 0x7ffb0c1597f0 2023-10-09T06:36:06.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.944+0000 7ffb01ffb700 1 -- 172.21.15.163:0/3426344478 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffafc00cb30 con 0x7ffb0c1597f0 2023-10-09T06:36:06.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.944+0000 7ffb12d96700 1 -- 172.21.15.163:0/3426344478 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb0c15b7f0 con 0x7ffb0c1597f0 2023-10-09T06:36:06.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.944+0000 7ffb01ffb700 1 -- 172.21.15.163:0/3426344478 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc00ced0 con 0x7ffb0c1597f0 2023-10-09T06:36:06.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.944+0000 7ffb12d96700 1 -- 172.21.15.163:0/3426344478 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb0c15b510 con 0x7ffb0c1597f0 2023-10-09T06:36:06.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.945+0000 7ffb12d96700 1 -- 172.21.15.163:0/3426344478 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb0c0d1180 con 0x7ffb0c1597f0 2023-10-09T06:36:06.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.945+0000 7ffb01ffb700 1 -- 172.21.15.163:0/3426344478 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7ffafc013680 con 0x7ffb0c1597f0 2023-10-09T06:36:06.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.946+0000 7ffb01ffb700 1 --2- 172.21.15.163:0/3426344478 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffaf0067200 0x7ffaf00696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:06.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.946+0000 7ffb01ffb700 1 -- 172.21.15.163:0/3426344478 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ffafc084d40 con 0x7ffb0c1597f0 2023-10-09T06:36:06.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.946+0000 7ffb11d94700 1 --2- 172.21.15.163:0/3426344478 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffaf0067200 0x7ffaf00696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:06.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.947+0000 7ffb11d94700 1 --2- 172.21.15.163:0/3426344478 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffaf0067200 0x7ffaf00696b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c069be0 tx=0x7ffb08007b40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:06.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:06.950+0000 7ffb01ffb700 1 -- 172.21.15.163:0/3426344478 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffafc051c60 con 0x7ffb0c1597f0 2023-10-09T06:36:07.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.172+0000 7ffb12d96700 1 -- 172.21.15.163:0/3426344478 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ffb0c04c410 con 0x7ffb0c1597f0 2023-10-09T06:36:07.174 INFO:tasks.ceph.mon.a.smithi163.stderr:2023-10-09T06:36:07.172+0000 7ff251c04700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-09T06:36:07.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.176+0000 7ffb01ffb700 1 -- 172.21.15.163:0/3426344478 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) v1 ==== 161+0+0 (secure 0 0 0) 0x7ffafc055280 con 0x7ffb0c1597f0 2023-10-09T06:36:07.178 INFO:tasks.ceph.mds.a.smithi163.stderr: -1763> 2023-10-09T06:35:57.929+0000 7f730e082700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-09T06:36:07.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.179+0000 7ffb12d96700 1 -- 172.21.15.163:0/3426344478 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffaf0067200 msgr2=0x7ffaf00696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:07.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.179+0000 7ffb12d96700 1 --2- 172.21.15.163:0/3426344478 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffaf0067200 0x7ffaf00696b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c069be0 tx=0x7ffb08007b40 comp rx=0 tx=0).stop 2023-10-09T06:36:07.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.179+0000 7ffb12d96700 1 -- 172.21.15.163:0/3426344478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb0c1597f0 msgr2=0x7ffb0c16c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:07.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.179+0000 7ffb12d96700 1 --2- 172.21.15.163:0/3426344478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb0c1597f0 0x7ffb0c16c550 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00ecb0 tx=0x7ffafc0091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:07.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.179+0000 7ffb12d96700 1 -- 172.21.15.163:0/3426344478 shutdown_connections 2023-10-09T06:36:07.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.179+0000 7ffb12d96700 1 --2- 172.21.15.163:0/3426344478 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffaf0067200 0x7ffaf00696b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:07.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.179+0000 7ffb12d96700 1 --2- 172.21.15.163:0/3426344478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb0c15d230 0x7ffb0c15aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:07.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.179+0000 7ffb12d96700 1 --2- 172.21.15.163:0/3426344478 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb0c159420 0x7ffb0c16a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:07.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.179+0000 7ffb12d96700 1 --2- 172.21.15.163:0/3426344478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb0c1597f0 0x7ffb0c16c550 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:07.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.179+0000 7ffb12d96700 1 -- 172.21.15.163:0/3426344478 >> 172.21.15.163:0/3426344478 conn(0x7ffb0c0d0660 msgr2=0x7ffb0c157220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:07.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.179+0000 7ffb12d96700 1 -- 172.21.15.163:0/3426344478 shutdown_connections 2023-10-09T06:36:07.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.179+0000 7ffb12d96700 1 -- 172.21.15.163:0/3426344478 wait complete. 2023-10-09T06:36:07.182 INFO:teuthology.orchestra.run.smithi163.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-09T06:36:07.193 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-09T06:36:07.201 INFO:tasks.ceph.mds.a.smithi163.stdout:starting mds.a at 2023-10-09T06:36:07.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.310+0000 7f41df655700 1 Processor -- start 2023-10-09T06:36:07.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.310+0000 7f41df655700 1 -- start start 2023-10-09T06:36:07.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.311+0000 7f41df655700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41d8159c60 0x7f41d815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:07.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.311+0000 7f41df655700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f41d815db10 0x7f41d8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:07.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.311+0000 7f41df655700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d815a160 con 0x7f41d8159890 2023-10-09T06:36:07.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.311+0000 7f41df655700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41d80d13a0 con 0x7f41d815db10 2023-10-09T06:36:07.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.311+0000 7f41df655700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d815e140 con 0x7f41d8159c60 2023-10-09T06:36:07.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.311+0000 7f41ddbf2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f41d815db10 0x7f41d8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:07.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.311+0000 7f41ddbf2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f41d815db10 0x7f41d8069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40756/0 (socket says 172.21.15.163:40756) 2023-10-09T06:36:07.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.311+0000 7f41ddbf2700 1 -- 172.21.15.163:0/1714741505 learned_addr learned my addr 172.21.15.163:0/1714741505 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:07.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.311+0000 7f41dcbf0700 1 --2- 172.21.15.163:0/1714741505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41d8159c60 0x7f41d815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:07.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.311+0000 7f41ddbf2700 1 -- 172.21.15.163:0/1714741505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41d8159c60 msgr2=0x7f41d815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:07.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.311+0000 7f41ddbf2700 1 --2- 172.21.15.163:0/1714741505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41d8159c60 0x7f41d815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:07.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.311+0000 7f41ddbf2700 1 -- 172.21.15.163:0/1714741505 >> v1:172.21.15.181:6789/0 conn(0x7f41d8159890 legacy=0x7f41d815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:07.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.311+0000 7f41ddbf2700 1 -- 172.21.15.163:0/1714741505 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d815a3d0 con 0x7f41d815db10 2023-10-09T06:36:07.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.312+0000 7f41ddbf2700 1 --2- 172.21.15.163:0/1714741505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41d815db10 0x7f41d8069560 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f41d00099b0 tx=0x7f41d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12af87d0393d4897 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:07.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.312+0000 7f41cffff700 1 -- 172.21.15.163:0/1714741505 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d000b860 con 0x7f41d815db10 2023-10-09T06:36:07.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.312+0000 7f41cffff700 1 -- 172.21.15.163:0/1714741505 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d000b9c0 con 0x7f41d815db10 2023-10-09T06:36:07.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.312+0000 7f41cffff700 1 -- 172.21.15.163:0/1714741505 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d000bd60 con 0x7f41d815db10 2023-10-09T06:36:07.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.312+0000 7f41df655700 1 -- 172.21.15.163:0/1714741505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41d815db10 msgr2=0x7f41d8069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:07.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.312+0000 7f41df655700 1 --2- 172.21.15.163:0/1714741505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41d815db10 0x7f41d8069560 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f41d00099b0 tx=0x7f41d0009250 comp rx=0 tx=0).stop 2023-10-09T06:36:07.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.312+0000 7f41df655700 1 -- 172.21.15.163:0/1714741505 shutdown_connections 2023-10-09T06:36:07.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.312+0000 7f41df655700 1 --2- 172.21.15.163:0/1714741505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41d815db10 0x7f41d8069560 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:07.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.312+0000 7f41df655700 1 --2- 172.21.15.163:0/1714741505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41d8159c60 0x7f41d815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:07.319 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.312+0000 7f41df655700 1 -- 172.21.15.163:0/1714741505 >> 172.21.15.163:0/1714741505 conn(0x7f41d8068e20 msgr2=0x7f41d80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:07.319 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.313+0000 7f41df655700 1 -- 172.21.15.163:0/1714741505 shutdown_connections 2023-10-09T06:36:07.319 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.313+0000 7f41df655700 1 -- 172.21.15.163:0/1714741505 wait complete. 2023-10-09T06:36:07.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.314+0000 7f41df655700 1 Processor -- start 2023-10-09T06:36:07.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.314+0000 7f41df655700 1 -- start start 2023-10-09T06:36:07.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.314+0000 7f41df655700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41d8159890 0x7f41d816a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:07.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.314+0000 7f41df655700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41d8159c60 0x7f41d81518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:07.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.314+0000 7f41df655700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41d815db10 0x7f41d80b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:07.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.315+0000 7f41df655700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d80b0ad0 con 0x7f41d8159890 2023-10-09T06:36:07.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.315+0000 7f41df655700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d80b0db0 con 0x7f41d815db10 2023-10-09T06:36:07.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.315+0000 7f41df655700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d80b1090 con 0x7f41d8159c60 2023-10-09T06:36:07.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.315+0000 7f41dd3f1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41d8159890 0x7f41d816a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:07.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.315+0000 7f41dd3f1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41d8159890 0x7f41d816a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40772/0 (socket says 172.21.15.163:40772) 2023-10-09T06:36:07.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.315+0000 7f41dd3f1700 1 -- 172.21.15.163:0/166395466 learned_addr learned my addr 172.21.15.163:0/166395466 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:07.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.315+0000 7f41dcbf0700 1 --2- 172.21.15.163:0/166395466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41d8159c60 0x7f41d81518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:07.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.315+0000 7f41ddbf2700 1 --2- 172.21.15.163:0/166395466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41d815db10 0x7f41d80b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:07.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.315+0000 7f41dd3f1700 1 -- 172.21.15.163:0/166395466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41d8159c60 msgr2=0x7f41d81518f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:07.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.315+0000 7f41dd3f1700 1 --2- 172.21.15.163:0/166395466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41d8159c60 0x7f41d81518f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:07.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.315+0000 7f41dd3f1700 1 -- 172.21.15.163:0/166395466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41d815db10 msgr2=0x7f41d80b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:07.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.315+0000 7f41dd3f1700 1 --2- 172.21.15.163:0/166395466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41d815db10 0x7f41d80b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:07.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.315+0000 7f41dd3f1700 1 -- 172.21.15.163:0/166395466 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d00094f0 con 0x7f41d8159890 2023-10-09T06:36:07.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.315+0000 7f41dd3f1700 1 --2- 172.21.15.163:0/166395466 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41d8159890 0x7f41d816a950 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f41d400cb10 tx=0x7f41d40017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:07.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.315+0000 7f41cd7fa700 1 -- 172.21.15.163:0/166395466 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d400ce20 con 0x7f41d8159890 2023-10-09T06:36:07.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.315+0000 7f41cd7fa700 1 -- 172.21.15.163:0/166395466 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d400a7e0 con 0x7f41d8159890 2023-10-09T06:36:07.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.315+0000 7f41cd7fa700 1 -- 172.21.15.163:0/166395466 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d400aba0 con 0x7f41d8159890 2023-10-09T06:36:07.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.315+0000 7f41df655700 1 -- 172.21.15.163:0/166395466 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41d80b1370 con 0x7f41d8159890 2023-10-09T06:36:07.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.315+0000 7f41df655700 1 -- 172.21.15.163:0/166395466 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d80cfe40 con 0x7f41d8159890 2023-10-09T06:36:07.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.316+0000 7f41df655700 1 -- 172.21.15.163:0/166395466 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41d804c3c0 con 0x7f41d8159890 2023-10-09T06:36:07.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.317+0000 7f41cd7fa700 1 -- 172.21.15.163:0/166395466 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f41d4003030 con 0x7f41d8159890 2023-10-09T06:36:07.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.317+0000 7f41cd7fa700 1 --2- 172.21.15.163:0/166395466 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f41bc067200 0x7f41bc0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:07.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.317+0000 7f41cd7fa700 1 -- 172.21.15.163:0/166395466 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f41d4084960 con 0x7f41d8159890 2023-10-09T06:36:07.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.317+0000 7f41dcbf0700 1 --2- 172.21.15.163:0/166395466 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f41bc067200 0x7f41bc0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:07.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.318+0000 7f41dcbf0700 1 --2- 172.21.15.163:0/166395466 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f41bc067200 0x7f41bc0696b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f41c8006fd0 tx=0x7f41c8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:07.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.321+0000 7f41cd7fa700 1 -- 172.21.15.163:0/166395466 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f41d4051830 con 0x7f41d8159890 2023-10-09T06:36:07.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:07.546+0000 7f41df655700 1 -- 172.21.15.163:0/166395466 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f41d806a5b0 con 0x7f41d8159890 2023-10-09T06:36:08.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.177+0000 7f41cd7fa700 1 -- 172.21.15.163:0/166395466 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v51) v1 ==== 104+0+0 (secure 0 0 0) 0x7f41d4054e50 con 0x7f41d8159890 2023-10-09T06:36:08.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.179+0000 7f41df655700 1 -- 172.21.15.163:0/166395466 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f41bc067200 msgr2=0x7f41bc0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:08.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.179+0000 7f41df655700 1 --2- 172.21.15.163:0/166395466 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f41bc067200 0x7f41bc0696b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f41c8006fd0 tx=0x7f41c8008040 comp rx=0 tx=0).stop 2023-10-09T06:36:08.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.179+0000 7f41df655700 1 -- 172.21.15.163:0/166395466 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41d8159890 msgr2=0x7f41d816a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:08.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.179+0000 7f41df655700 1 --2- 172.21.15.163:0/166395466 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41d8159890 0x7f41d816a950 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f41d400cb10 tx=0x7f41d40017b0 comp rx=0 tx=0).stop 2023-10-09T06:36:08.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.179+0000 7f41df655700 1 -- 172.21.15.163:0/166395466 shutdown_connections 2023-10-09T06:36:08.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.179+0000 7f41df655700 1 --2- 172.21.15.163:0/166395466 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f41bc067200 0x7f41bc0696b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:08.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.179+0000 7f41df655700 1 --2- 172.21.15.163:0/166395466 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41d8159890 0x7f41d816a950 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:08.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.179+0000 7f41df655700 1 --2- 172.21.15.163:0/166395466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41d815db10 0x7f41d80b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:08.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.179+0000 7f41df655700 1 --2- 172.21.15.163:0/166395466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41d8159c60 0x7f41d81518f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:08.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.179+0000 7f41df655700 1 -- 172.21.15.163:0/166395466 >> 172.21.15.163:0/166395466 conn(0x7f41d8068e20 msgr2=0x7f41d8153070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:08.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.179+0000 7f41df655700 1 -- 172.21.15.163:0/166395466 shutdown_connections 2023-10-09T06:36:08.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.179+0000 7f41df655700 1 -- 172.21.15.163:0/166395466 wait complete. 2023-10-09T06:36:08.194 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-09T06:36:08.355 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.354+0000 7f8d8e615700 1 Processor -- start 2023-10-09T06:36:08.355 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.354+0000 7f8d8e615700 1 -- start start 2023-10-09T06:36:08.359 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.355+0000 7f8d8e615700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d880b0790 0x7f8d880cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:08.359 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.355+0000 7f8d8e615700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d880cfa00 con 0x7f8d881591e0 2023-10-09T06:36:08.359 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.355+0000 7f8d8e615700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d880cb580 con 0x7f8d880b03c0 2023-10-09T06:36:08.360 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.355+0000 7f8d8e615700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d880cf160 con 0x7f8d880b0790 2023-10-09T06:36:08.360 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.355+0000 7f8d8de14700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f8d881591e0 0x7f8d88159620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60724/0 (socket says 172.21.15.163:60724) 2023-10-09T06:36:08.360 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.355+0000 7f8d8de14700 1 -- 172.21.15.163:0/3744495679 learned_addr learned my addr 172.21.15.163:0/3744495679 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:08.360 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.355+0000 7f8d8ce12700 1 --2- 172.21.15.163:0/3744495679 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d880b0790 0x7f8d880cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:08.360 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.355+0000 7f8d7ffff700 1 -- 172.21.15.163:0/3744495679 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1738742389 0 0) 0x7f8d880cfa00 con 0x7f8d881591e0 2023-10-09T06:36:08.360 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.355+0000 7f8d7ffff700 1 -- 172.21.15.163:0/3744495679 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d74003570 con 0x7f8d881591e0 2023-10-09T06:36:08.360 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.355+0000 7f8d7ffff700 1 -- 172.21.15.163:0/3744495679 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2932571609 0 0) 0x7f8d74003570 con 0x7f8d881591e0 2023-10-09T06:36:08.360 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.355+0000 7f8d7ffff700 1 -- 172.21.15.163:0/3744495679 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d880b0790 msgr2=0x7f8d880cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:08.360 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.355+0000 7f8d7ffff700 1 --2- 172.21.15.163:0/3744495679 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d880b0790 0x7f8d880cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:08.360 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.355+0000 7f8d7ffff700 1 -- 172.21.15.163:0/3744495679 >> v1:172.21.15.181:6789/0 conn(0x7f8d880b03c0 legacy=0x7f8d880b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:08.360 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.355+0000 7f8d7ffff700 1 -- 172.21.15.163:0/3744495679 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d880cf440 con 0x7f8d881591e0 2023-10-09T06:36:08.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.356+0000 7f8d7ffff700 1 -- 172.21.15.163:0/3744495679 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f8d800017f0 con 0x7f8d881591e0 2023-10-09T06:36:08.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.356+0000 7f8d7ffff700 1 -- 172.21.15.163:0/3744495679 >> v1:172.21.15.163:6789/0 conn(0x7f8d881591e0 legacy=0x7f8d88159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:08.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.356+0000 7f8d7ffff700 1 --2- 172.21.15.163:0/3744495679 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d880b0790 0x7f8d74003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:08.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.356+0000 7f8d7ffff700 1 --2- 172.21.15.163:0/3744495679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d74004010 0x7f8d74004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:08.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.356+0000 7f8d7ffff700 1 --2- 172.21.15.163:0/3744495679 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d74004d50 0x7f8d740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:08.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.356+0000 7f8d7ffff700 1 -- 172.21.15.163:0/3744495679 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d880cf160 con 0x7f8d74004d50 2023-10-09T06:36:08.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.356+0000 7f8d7ffff700 1 -- 172.21.15.163:0/3744495679 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7400bbf0 con 0x7f8d74004010 2023-10-09T06:36:08.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.356+0000 7f8d7ffff700 1 -- 172.21.15.163:0/3744495679 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7400bdb0 con 0x7f8d880b0790 2023-10-09T06:36:08.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.356+0000 7f8d8de14700 1 --2- 172.21.15.163:0/3744495679 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d74004d50 0x7f8d740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:08.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.356+0000 7f8d8d613700 1 --2- 172.21.15.163:0/3744495679 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d880b0790 0x7f8d74003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:08.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.356+0000 7f8d8ce12700 1 --2- 172.21.15.163:0/3744495679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d74004010 0x7f8d74004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:08.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.356+0000 7f8d8de14700 1 -- 172.21.15.163:0/3744495679 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d880b0790 msgr2=0x7f8d74003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:08.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.356+0000 7f8d8de14700 1 --2- 172.21.15.163:0/3744495679 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d880b0790 0x7f8d74003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:08.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.356+0000 7f8d8de14700 1 -- 172.21.15.163:0/3744495679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d74004010 msgr2=0x7f8d74004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:08.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.356+0000 7f8d8de14700 1 --2- 172.21.15.163:0/3744495679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d74004010 0x7f8d74004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:08.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.356+0000 7f8d8de14700 1 -- 172.21.15.163:0/3744495679 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d74003570 con 0x7f8d74004d50 2023-10-09T06:36:08.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.356+0000 7f8d8ce12700 1 --2- 172.21.15.163:0/3744495679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d74004010 0x7f8d74004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:08.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.356+0000 7f8d8de14700 1 --2- 172.21.15.163:0/3744495679 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d74004d50 0x7f8d740051c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8d80002250 tx=0x7f8d880b0c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6835ae10179833c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:08.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.356+0000 7f8d7ffff700 1 -- 172.21.15.163:0/3744495679 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d800070d0 con 0x7f8d74004d50 2023-10-09T06:36:08.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.356+0000 7f8d7ffff700 1 -- 172.21.15.163:0/3744495679 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d80007230 con 0x7f8d74004d50 2023-10-09T06:36:08.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.356+0000 7f8d7ffff700 1 -- 172.21.15.163:0/3744495679 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d800075f0 con 0x7f8d74004d50 2023-10-09T06:36:08.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.357+0000 7f8d8e615700 1 -- 172.21.15.163:0/3744495679 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d74004d50 msgr2=0x7f8d740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:08.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.357+0000 7f8d8e615700 1 --2- 172.21.15.163:0/3744495679 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d74004d50 0x7f8d740051c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8d80002250 tx=0x7f8d880b0c40 comp rx=0 tx=0).stop 2023-10-09T06:36:08.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.357+0000 7f8d8de14700 1 -- 172.21.15.163:0/3744495679 reap_dead start 2023-10-09T06:36:08.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.357+0000 7f8d8e615700 1 -- 172.21.15.163:0/3744495679 shutdown_connections 2023-10-09T06:36:08.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.357+0000 7f8d8e615700 1 -- 172.21.15.163:0/3744495679 >> 172.21.15.163:0/3744495679 conn(0x7f8d88063230 msgr2=0x7f8d88063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:08.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.357+0000 7f8d8e615700 1 -- 172.21.15.163:0/3744495679 shutdown_connections 2023-10-09T06:36:08.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.357+0000 7f8d8e615700 1 -- 172.21.15.163:0/3744495679 wait complete. 2023-10-09T06:36:08.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.358+0000 7f8d8e615700 1 Processor -- start 2023-10-09T06:36:08.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.358+0000 7f8d8e615700 1 -- start start 2023-10-09T06:36:08.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.359+0000 7f8d8e615700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d74004d50 0x7f8d880c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:08.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.359+0000 7f8d8e615700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d8815c0a0 0x7f8d880c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:08.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.359+0000 7f8d8e615700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d8815c510 0x7f8d880ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:08.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.359+0000 7f8d8e615700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d880c3ff0 con 0x7f8d8815c510 2023-10-09T06:36:08.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.359+0000 7f8d8e615700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d880c4240 con 0x7f8d8815c0a0 2023-10-09T06:36:08.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.359+0000 7f8d8e615700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d880c4520 con 0x7f8d74004d50 2023-10-09T06:36:08.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.359+0000 7f8d8de14700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d8815c510 0x7f8d880ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:08.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.359+0000 7f8d8de14700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d8815c510 0x7f8d880ca300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:40800/0 (socket says 172.21.15.163:40800) 2023-10-09T06:36:08.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.359+0000 7f8d8de14700 1 -- 172.21.15.163:0/20246359 learned_addr learned my addr 172.21.15.163:0/20246359 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:08.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.359+0000 7f8d8d613700 1 --2- 172.21.15.163:0/20246359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d74004d50 0x7f8d880c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:08.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.359+0000 7f8d8de14700 1 -- 172.21.15.163:0/20246359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d74004d50 msgr2=0x7f8d880c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:08.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.359+0000 7f8d8de14700 1 --2- 172.21.15.163:0/20246359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d74004d50 0x7f8d880c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:08.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.359+0000 7f8d8ce12700 1 --2- 172.21.15.163:0/20246359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d8815c0a0 0x7f8d880c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:08.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.359+0000 7f8d8de14700 1 -- 172.21.15.163:0/20246359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d8815c0a0 msgr2=0x7f8d880c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:08.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.359+0000 7f8d8de14700 1 --2- 172.21.15.163:0/20246359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d8815c0a0 0x7f8d880c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:08.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.359+0000 7f8d8de14700 1 -- 172.21.15.163:0/20246359 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d880c4800 con 0x7f8d8815c510 2023-10-09T06:36:08.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.359+0000 7f8d8d613700 1 --2- 172.21.15.163:0/20246359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d74004d50 0x7f8d880c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:36:08.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.359+0000 7f8d8de14700 1 --2- 172.21.15.163:0/20246359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d8815c510 0x7f8d880ca300 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8d80012000 tx=0x7f8d80015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:08.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.360+0000 7f8d7d7fa700 1 -- 172.21.15.163:0/20246359 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d800075b0 con 0x7f8d8815c510 2023-10-09T06:36:08.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.360+0000 7f8d7d7fa700 1 -- 172.21.15.163:0/20246359 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d8001e450 con 0x7f8d8815c510 2023-10-09T06:36:08.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.360+0000 7f8d8e615700 1 -- 172.21.15.163:0/20246359 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d880c4ae0 con 0x7f8d8815c510 2023-10-09T06:36:08.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.360+0000 7f8d7d7fa700 1 -- 172.21.15.163:0/20246359 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8001e810 con 0x7f8d8815c510 2023-10-09T06:36:08.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.360+0000 7f8d8e615700 1 -- 172.21.15.163:0/20246359 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8815a920 con 0x7f8d8815c510 2023-10-09T06:36:08.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.360+0000 7f8d8e615700 1 -- 172.21.15.163:0/20246359 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d880c4cd0 con 0x7f8d8815c510 2023-10-09T06:36:08.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.361+0000 7f8d7d7fa700 1 -- 172.21.15.163:0/20246359 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f8d8001ec40 con 0x7f8d8815c510 2023-10-09T06:36:08.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.361+0000 7f8d7d7fa700 1 --2- 172.21.15.163:0/20246359 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8d6c067230 0x7f8d6c0696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:08.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.361+0000 7f8d7d7fa700 1 -- 172.21.15.163:0/20246359 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f8d80086d50 con 0x7f8d8815c510 2023-10-09T06:36:08.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.362+0000 7f8d8d613700 1 --2- 172.21.15.163:0/20246359 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8d6c067230 0x7f8d6c0696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:08.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.362+0000 7f8d8d613700 1 --2- 172.21.15.163:0/20246359 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8d6c067230 0x7f8d6c0696e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8d84000c00 tx=0x7f8d84009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:08.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.365+0000 7f8d7d7fa700 1 -- 172.21.15.163:0/20246359 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8d80053c20 con 0x7f8d8815c510 2023-10-09T06:36:08.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:08.549+0000 7f8d8e615700 1 -- 172.21.15.163:0/20246359 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8d8804c3c0 con 0x7f8d8815c510 2023-10-09T06:36:09.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.523+0000 7f8d7d7fa700 1 -- 172.21.15.163:0/20246359 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v79) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8d800160d0 con 0x7f8d8815c510 2023-10-09T06:36:09.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.527+0000 7f8d8e615700 1 -- 172.21.15.163:0/20246359 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8d6c067230 msgr2=0x7f8d6c0696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:09.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.527+0000 7f8d8e615700 1 --2- 172.21.15.163:0/20246359 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8d6c067230 0x7f8d6c0696e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8d84000c00 tx=0x7f8d84009040 comp rx=0 tx=0).stop 2023-10-09T06:36:09.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.527+0000 7f8d8e615700 1 -- 172.21.15.163:0/20246359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d8815c510 msgr2=0x7f8d880ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:09.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.527+0000 7f8d8e615700 1 --2- 172.21.15.163:0/20246359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d8815c510 0x7f8d880ca300 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8d80012000 tx=0x7f8d80015000 comp rx=0 tx=0).stop 2023-10-09T06:36:09.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.527+0000 7f8d8e615700 1 -- 172.21.15.163:0/20246359 shutdown_connections 2023-10-09T06:36:09.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.527+0000 7f8d8e615700 1 --2- 172.21.15.163:0/20246359 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8d6c067230 0x7f8d6c0696e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:09.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.527+0000 7f8d8e615700 1 --2- 172.21.15.163:0/20246359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d8815c510 0x7f8d880ca300 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:09.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.527+0000 7f8d8e615700 1 --2- 172.21.15.163:0/20246359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d74004d50 0x7f8d880c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:09.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.527+0000 7f8d8e615700 1 --2- 172.21.15.163:0/20246359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d8815c0a0 0x7f8d880c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:09.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.527+0000 7f8d8e615700 1 -- 172.21.15.163:0/20246359 >> 172.21.15.163:0/20246359 conn(0x7f8d88063230 msgr2=0x7f8d88158970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:09.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.527+0000 7f8d8e615700 1 -- 172.21.15.163:0/20246359 shutdown_connections 2023-10-09T06:36:09.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.527+0000 7f8d8e615700 1 -- 172.21.15.163:0/20246359 wait complete. 2023-10-09T06:36:09.529 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_metadata' removed 2023-10-09T06:36:09.543 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-09T06:36:09.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.666+0000 7fc50f59e700 1 Processor -- start 2023-10-09T06:36:09.667 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.666+0000 7fc50f59e700 1 -- start start 2023-10-09T06:36:09.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.667+0000 7fc50f59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc5100b03c0 0x7fc5100cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:09.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.667+0000 7fc50f59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5100cfa00 con 0x7fc5101591e0 2023-10-09T06:36:09.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.667+0000 7fc50f59e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5100cb580 con 0x7fc5100b0790 2023-10-09T06:36:09.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.667+0000 7fc50f59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5100cf160 con 0x7fc5100b03c0 2023-10-09T06:36:09.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.667+0000 7fc50e59c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc5100b03c0 0x7fc5100cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:09.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.667+0000 7fc50ed9d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fc5101591e0 0x7fc510159600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:45446/0 (socket says 172.21.15.163:45446) 2023-10-09T06:36:09.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.667+0000 7fc50ed9d700 1 -- 172.21.15.163:0/173882781 learned_addr learned my addr 172.21.15.163:0/173882781 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:09.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.667+0000 7fc50e59c700 1 -- 172.21.15.163:0/173882781 >> v1:172.21.15.181:6789/0 conn(0x7fc5100b0790 legacy=0x7fc5100b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:09.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.667+0000 7fc50e59c700 1 -- 172.21.15.163:0/173882781 >> v1:172.21.15.163:6789/0 conn(0x7fc5101591e0 legacy=0x7fc510159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:09.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.667+0000 7fc50e59c700 1 -- 172.21.15.163:0/173882781 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5100cf440 con 0x7fc5100b03c0 2023-10-09T06:36:09.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.667+0000 7fc50d59a700 1 -- 172.21.15.163:0/173882781 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4175088659 0 0) 0x7fc5100cfa00 con 0x7fc5101591e0 2023-10-09T06:36:09.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.667+0000 7fc50e59c700 1 --2- 172.21.15.163:0/173882781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5100b03c0 0x7fc5100cea70 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fc508009910 tx=0x7fc5080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39fe025c3ea2c306 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:09.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.667+0000 7fc50d59a700 1 -- 172.21.15.163:0/173882781 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50800b7e0 con 0x7fc5100b03c0 2023-10-09T06:36:09.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.667+0000 7fc50d59a700 1 -- 172.21.15.163:0/173882781 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50800b940 con 0x7fc5100b03c0 2023-10-09T06:36:09.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.667+0000 7fc50d59a700 1 -- 172.21.15.163:0/173882781 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50800bce0 con 0x7fc5100b03c0 2023-10-09T06:36:09.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.668+0000 7fc50f59e700 1 -- 172.21.15.163:0/173882781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5100b03c0 msgr2=0x7fc5100cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:09.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.668+0000 7fc50f59e700 1 --2- 172.21.15.163:0/173882781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5100b03c0 0x7fc5100cea70 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fc508009910 tx=0x7fc5080091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:09.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.668+0000 7fc50f59e700 1 -- 172.21.15.163:0/173882781 shutdown_connections 2023-10-09T06:36:09.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.668+0000 7fc50f59e700 1 --2- 172.21.15.163:0/173882781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5100b03c0 0x7fc5100cea70 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:09.671 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.668+0000 7fc50f59e700 1 -- 172.21.15.163:0/173882781 >> 172.21.15.163:0/173882781 conn(0x7fc510063230 msgr2=0x7fc510063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:09.671 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.668+0000 7fc50f59e700 1 -- 172.21.15.163:0/173882781 shutdown_connections 2023-10-09T06:36:09.671 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.668+0000 7fc50f59e700 1 -- 172.21.15.163:0/173882781 wait complete. 2023-10-09T06:36:09.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.669+0000 7fc50f59e700 1 Processor -- start 2023-10-09T06:36:09.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.669+0000 7fc50f59e700 1 -- start start 2023-10-09T06:36:09.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.670+0000 7fc50f59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5100b03c0 0x7fc510151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:09.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.670+0000 7fc50f59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5100b0790 0x7fc5100c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:09.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.670+0000 7fc50f59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5101591e0 0x7fc5100b44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:09.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.670+0000 7fc50f59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5101524b0 con 0x7fc5100b03c0 2023-10-09T06:36:09.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.670+0000 7fc50f59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5100c90c0 con 0x7fc5100b0790 2023-10-09T06:36:09.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.670+0000 7fc50f59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5100c93a0 con 0x7fc5101591e0 2023-10-09T06:36:09.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.670+0000 7fc50e59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5100b03c0 0x7fc510151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:09.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.670+0000 7fc50e59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5100b03c0 0x7fc510151f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35470/0 (socket says 172.21.15.163:35470) 2023-10-09T06:36:09.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.670+0000 7fc50e59c700 1 -- 172.21.15.163:0/2064216057 learned_addr learned my addr 172.21.15.163:0/2064216057 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:09.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.670+0000 7fc50e59c700 1 -- 172.21.15.163:0/2064216057 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5101591e0 msgr2=0x7fc5100b44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:09.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.670+0000 7fc50e59c700 1 --2- 172.21.15.163:0/2064216057 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5101591e0 0x7fc5100b44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:09.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.670+0000 7fc50e59c700 1 -- 172.21.15.163:0/2064216057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5100b0790 msgr2=0x7fc5100c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:09.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.670+0000 7fc50e59c700 1 --2- 172.21.15.163:0/2064216057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5100b0790 0x7fc5100c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:09.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.670+0000 7fc50e59c700 1 -- 172.21.15.163:0/2064216057 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc508009420 con 0x7fc5100b03c0 2023-10-09T06:36:09.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.670+0000 7fc50e59c700 1 --2- 172.21.15.163:0/2064216057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5100b03c0 0x7fc510151f70 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc5080093f0 tx=0x7fc508016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:09.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.670+0000 7fc4fe7fc700 1 -- 172.21.15.163:0/2064216057 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc508011ca0 con 0x7fc5100b03c0 2023-10-09T06:36:09.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.671+0000 7fc4fe7fc700 1 -- 172.21.15.163:0/2064216057 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc508020070 con 0x7fc5100b03c0 2023-10-09T06:36:09.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.671+0000 7fc4fe7fc700 1 -- 172.21.15.163:0/2064216057 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50801b6a0 con 0x7fc5100b03c0 2023-10-09T06:36:09.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.671+0000 7fc50f59e700 1 -- 172.21.15.163:0/2064216057 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5100c9680 con 0x7fc5100b03c0 2023-10-09T06:36:09.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.671+0000 7fc50f59e700 1 -- 172.21.15.163:0/2064216057 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5100c9c40 con 0x7fc5100b03c0 2023-10-09T06:36:09.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.672+0000 7fc50f59e700 1 -- 172.21.15.163:0/2064216057 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5100c3f70 con 0x7fc5100b03c0 2023-10-09T06:36:09.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.672+0000 7fc4fe7fc700 1 -- 172.21.15.163:0/2064216057 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fc508022030 con 0x7fc5100b03c0 2023-10-09T06:36:09.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.672+0000 7fc4fe7fc700 1 --2- 172.21.15.163:0/2064216057 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc4f0067200 0x7fc4f00696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:09.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.672+0000 7fc4fe7fc700 1 -- 172.21.15.163:0/2064216057 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5353+0+0 (secure 0 0 0) 0x7fc5080867b0 con 0x7fc5100b03c0 2023-10-09T06:36:09.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.673+0000 7fc50dd9b700 1 --2- 172.21.15.163:0/2064216057 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc4f0067200 0x7fc4f00696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:09.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.673+0000 7fc50dd9b700 1 --2- 172.21.15.163:0/2064216057 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc4f0067200 0x7fc4f00696b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc504007fd0 tx=0x7fc504007b20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:09.677 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.676+0000 7fc4fe7fc700 1 -- 172.21.15.163:0/2064216057 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc508053830 con 0x7fc5100b03c0 2023-10-09T06:36:09.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:09.861+0000 7fc50f59e700 1 -- 172.21.15.163:0/2064216057 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc5100c3f70 con 0x7fc5100b03c0 2023-10-09T06:36:10.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.727+0000 7fc4fe7fc700 1 -- 172.21.15.163:0/2064216057 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v80) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc508056e50 con 0x7fc5100b03c0 2023-10-09T06:36:10.731 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.730+0000 7fc50f59e700 1 -- 172.21.15.163:0/2064216057 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc4f0067200 msgr2=0x7fc4f00696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:10.731 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.730+0000 7fc50f59e700 1 --2- 172.21.15.163:0/2064216057 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc4f0067200 0x7fc4f00696b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc504007fd0 tx=0x7fc504007b20 comp rx=0 tx=0).stop 2023-10-09T06:36:10.731 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.730+0000 7fc50f59e700 1 -- 172.21.15.163:0/2064216057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5100b03c0 msgr2=0x7fc510151f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:10.731 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.730+0000 7fc50f59e700 1 --2- 172.21.15.163:0/2064216057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5100b03c0 0x7fc510151f70 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc5080093f0 tx=0x7fc508016000 comp rx=0 tx=0).stop 2023-10-09T06:36:10.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.731+0000 7fc50f59e700 1 -- 172.21.15.163:0/2064216057 shutdown_connections 2023-10-09T06:36:10.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.731+0000 7fc50f59e700 1 --2- 172.21.15.163:0/2064216057 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc4f0067200 0x7fc4f00696b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:10.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.731+0000 7fc50f59e700 1 --2- 172.21.15.163:0/2064216057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5100b03c0 0x7fc510151f70 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:10.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.731+0000 7fc50f59e700 1 --2- 172.21.15.163:0/2064216057 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5101591e0 0x7fc5100b44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:10.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.731+0000 7fc50f59e700 1 --2- 172.21.15.163:0/2064216057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5100b0790 0x7fc5100c85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:10.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.731+0000 7fc50f59e700 1 -- 172.21.15.163:0/2064216057 >> 172.21.15.163:0/2064216057 conn(0x7fc510063230 msgr2=0x7fc510068420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:10.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.731+0000 7fc50f59e700 1 -- 172.21.15.163:0/2064216057 shutdown_connections 2023-10-09T06:36:10.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.731+0000 7fc50f59e700 1 -- 172.21.15.163:0/2064216057 wait complete. 2023-10-09T06:36:10.733 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_data' removed 2023-10-09T06:36:10.748 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-09T06:36:10.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.869+0000 7f0913071700 1 Processor -- start 2023-10-09T06:36:10.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.869+0000 7f0913071700 1 -- start start 2023-10-09T06:36:10.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.870+0000 7f0913071700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f090c159890 0x7f090c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:10.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.870+0000 7f0913071700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f090c069560 0x7f090c0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:10.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.870+0000 7f0913071700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f090c15a160 con 0x7f090c159c60 2023-10-09T06:36:10.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.870+0000 7f0913071700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f090c069f20 con 0x7f090c069560 2023-10-09T06:36:10.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.870+0000 7f0913071700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f090c0d13a0 con 0x7f090c159890 2023-10-09T06:36:10.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.870+0000 7f091160e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f090c069560 0x7f090c0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:10.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.870+0000 7f0910e0d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f090c159890 0x7f090c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:10.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.870+0000 7f091160e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f090c069560 0x7f090c0699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:33572/0 (socket says 172.21.15.163:33572) 2023-10-09T06:36:10.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.870+0000 7f0910e0d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f090c159890 0x7f090c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:35582/0 (socket says 172.21.15.163:35582) 2023-10-09T06:36:10.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.870+0000 7f0903fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f090c159c60 0x7f090c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:48436/0 (socket says 172.21.15.163:48436) 2023-10-09T06:36:10.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.870+0000 7f091160e700 1 -- 172.21.15.163:0/116386816 learned_addr learned my addr 172.21.15.163:0/116386816 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:10.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.871+0000 7f091160e700 1 -- 172.21.15.163:0/116386816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f090c159890 msgr2=0x7f090c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:10.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.871+0000 7f091160e700 1 --2- 172.21.15.163:0/116386816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f090c159890 0x7f090c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:10.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.871+0000 7f091160e700 1 -- 172.21.15.163:0/116386816 >> v1:172.21.15.181:6789/0 conn(0x7f090c159c60 legacy=0x7f090c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:10.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.871+0000 7f091160e700 1 -- 172.21.15.163:0/116386816 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090c15e140 con 0x7f090c069560 2023-10-09T06:36:10.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.871+0000 7f09037fe700 1 -- 172.21.15.163:0/116386816 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3260463283 0 0) 0x7f090c15a160 con 0x7f090c159c60 2023-10-09T06:36:10.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.871+0000 7f0910e0d700 1 --2- 172.21.15.163:0/116386816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f090c159890 0x7f090c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:36:10.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.871+0000 7f091160e700 1 --2- 172.21.15.163:0/116386816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f090c069560 0x7f090c0699e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0904009f20 tx=0x7f0904009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe338671d8b509d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:10.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.871+0000 7f09037fe700 1 -- 172.21.15.163:0/116386816 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090400e070 con 0x7f090c069560 2023-10-09T06:36:10.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.871+0000 7f09037fe700 1 -- 172.21.15.163:0/116386816 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090400b9d0 con 0x7f090c069560 2023-10-09T06:36:10.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.871+0000 7f09037fe700 1 -- 172.21.15.163:0/116386816 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09040092e0 con 0x7f090c069560 2023-10-09T06:36:10.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.872+0000 7f0913071700 1 -- 172.21.15.163:0/116386816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f090c069560 msgr2=0x7f090c0699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:10.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.872+0000 7f0913071700 1 --2- 172.21.15.163:0/116386816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f090c069560 0x7f090c0699e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0904009f20 tx=0x7f0904009840 comp rx=0 tx=0).stop 2023-10-09T06:36:10.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.872+0000 7f0913071700 1 -- 172.21.15.163:0/116386816 shutdown_connections 2023-10-09T06:36:10.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.872+0000 7f0913071700 1 --2- 172.21.15.163:0/116386816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f090c069560 0x7f090c0699e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:10.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.872+0000 7f0913071700 1 --2- 172.21.15.163:0/116386816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f090c159890 0x7f090c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:10.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.872+0000 7f0913071700 1 -- 172.21.15.163:0/116386816 >> 172.21.15.163:0/116386816 conn(0x7f090c068e20 msgr2=0x7f090c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:10.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.872+0000 7f0913071700 1 -- 172.21.15.163:0/116386816 shutdown_connections 2023-10-09T06:36:10.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.872+0000 7f0913071700 1 -- 172.21.15.163:0/116386816 wait complete. 2023-10-09T06:36:10.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.873+0000 7f0913071700 1 Processor -- start 2023-10-09T06:36:10.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.873+0000 7f0913071700 1 -- start start 2023-10-09T06:36:10.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.874+0000 7f0913071700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f090c069560 0x7f090c0cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:10.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.874+0000 7f0913071700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f090c159890 0x7f090c14d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:10.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.874+0000 7f0913071700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f090c159c60 0x7f090c0c6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:10.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.874+0000 7f0913071700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c0ce5e0 con 0x7f090c159890 2023-10-09T06:36:10.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.874+0000 7f0913071700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c15d6e0 con 0x7f090c069560 2023-10-09T06:36:10.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.874+0000 7f0913071700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f090c0ce010 con 0x7f090c159c60 2023-10-09T06:36:10.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.874+0000 7f0903fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f090c159890 0x7f090c14d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:10.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.874+0000 7f0903fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f090c159890 0x7f090c14d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35480/0 (socket says 172.21.15.163:35480) 2023-10-09T06:36:10.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.874+0000 7f0903fff700 1 -- 172.21.15.163:0/196974516 learned_addr learned my addr 172.21.15.163:0/196974516 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:10.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.874+0000 7f0910e0d700 1 --2- 172.21.15.163:0/196974516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f090c069560 0x7f090c0cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:10.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.874+0000 7f091160e700 1 --2- 172.21.15.163:0/196974516 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f090c159c60 0x7f090c0c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:10.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.874+0000 7f0903fff700 1 -- 172.21.15.163:0/196974516 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f090c159c60 msgr2=0x7f090c0c6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:10.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.874+0000 7f0903fff700 1 --2- 172.21.15.163:0/196974516 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f090c159c60 0x7f090c0c6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:10.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.874+0000 7f0903fff700 1 -- 172.21.15.163:0/196974516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f090c069560 msgr2=0x7f090c0cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:10.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.874+0000 7f0903fff700 1 --2- 172.21.15.163:0/196974516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f090c069560 0x7f090c0cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:10.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.874+0000 7f0903fff700 1 -- 172.21.15.163:0/196974516 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0908009420 con 0x7f090c159890 2023-10-09T06:36:10.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.874+0000 7f0903fff700 1 --2- 172.21.15.163:0/196974516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f090c159890 0x7f090c14d330 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00c510 tx=0x7f08fc00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:10.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.874+0000 7f0900ff9700 1 -- 172.21.15.163:0/196974516 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc005a70 con 0x7f090c159890 2023-10-09T06:36:10.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.875+0000 7f0913071700 1 -- 172.21.15.163:0/196974516 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0904009ae0 con 0x7f090c159890 2023-10-09T06:36:10.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.875+0000 7f0913071700 1 -- 172.21.15.163:0/196974516 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f090c15cdb0 con 0x7f090c159890 2023-10-09T06:36:10.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.875+0000 7f0900ff9700 1 -- 172.21.15.163:0/196974516 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08fc001130 con 0x7f090c159890 2023-10-09T06:36:10.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.875+0000 7f0900ff9700 1 -- 172.21.15.163:0/196974516 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc005de0 con 0x7f090c159890 2023-10-09T06:36:10.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.875+0000 7f0913071700 1 -- 172.21.15.163:0/196974516 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ec0052b0 con 0x7f090c159890 2023-10-09T06:36:10.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.876+0000 7f0900ff9700 1 -- 172.21.15.163:0/196974516 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f08fc0072e0 con 0x7f090c159890 2023-10-09T06:36:10.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.876+0000 7f0900ff9700 1 --2- 172.21.15.163:0/196974516 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f08f8067200 0x7f08f80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:10.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.876+0000 7f0900ff9700 1 -- 172.21.15.163:0/196974516 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f08fc0838e0 con 0x7f090c159890 2023-10-09T06:36:10.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.876+0000 7f0910e0d700 1 --2- 172.21.15.163:0/196974516 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f08f8067200 0x7f08f80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:10.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.877+0000 7f0910e0d700 1 --2- 172.21.15.163:0/196974516 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f08f8067200 0x7f08f80696b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f090c0b1a30 tx=0x7f09080091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:10.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:10.880+0000 7f0900ff9700 1 -- 172.21.15.163:0/196974516 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f08fc04b5a0 con 0x7f090c159890 2023-10-09T06:36:11.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.063+0000 7f0913071700 1 -- 172.21.15.163:0/196974516 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f08ec005ae0 con 0x7f090c159890 2023-10-09T06:36:11.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.066+0000 7f0900ff9700 1 -- 172.21.15.163:0/196974516 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f08fc050bb0 con 0x7f090c159890 2023-10-09T06:36:11.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.069+0000 7f0913071700 1 -- 172.21.15.163:0/196974516 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f08f8067200 msgr2=0x7f08f80696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:11.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.069+0000 7f0913071700 1 --2- 172.21.15.163:0/196974516 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f08f8067200 0x7f08f80696b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f090c0b1a30 tx=0x7f09080091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:11.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.069+0000 7f0913071700 1 -- 172.21.15.163:0/196974516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f090c159890 msgr2=0x7f090c14d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:11.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.069+0000 7f0913071700 1 --2- 172.21.15.163:0/196974516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f090c159890 0x7f090c14d330 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00c510 tx=0x7f08fc00c820 comp rx=0 tx=0).stop 2023-10-09T06:36:11.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.069+0000 7f0913071700 1 -- 172.21.15.163:0/196974516 shutdown_connections 2023-10-09T06:36:11.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.069+0000 7f0913071700 1 --2- 172.21.15.163:0/196974516 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f08f8067200 0x7f08f80696b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:11.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.069+0000 7f0913071700 1 --2- 172.21.15.163:0/196974516 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f090c159c60 0x7f090c0c6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:11.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.069+0000 7f0913071700 1 --2- 172.21.15.163:0/196974516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f090c069560 0x7f090c0cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:11.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.069+0000 7f0913071700 1 --2- 172.21.15.163:0/196974516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f090c159890 0x7f090c14d330 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:11.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.069+0000 7f0913071700 1 -- 172.21.15.163:0/196974516 >> 172.21.15.163:0/196974516 conn(0x7f090c068e20 msgr2=0x7f090c0d0920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:11.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.069+0000 7f0913071700 1 -- 172.21.15.163:0/196974516 shutdown_connections 2023-10-09T06:36:11.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.069+0000 7f0913071700 1 -- 172.21.15.163:0/196974516 wait complete. 2023-10-09T06:36:11.082 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag' 2023-10-09T06:36:11.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.196+0000 7f4937ab0700 1 Processor -- start 2023-10-09T06:36:11.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.197+0000 7f4937ab0700 1 -- start start 2023-10-09T06:36:11.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.197+0000 7f4937ab0700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4930159a40 0x7f49300c9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:11.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.197+0000 7f4937ab0700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49300c90e0 con 0x7f493015d7b0 2023-10-09T06:36:11.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.197+0000 7f4937ab0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f493015dc40 con 0x7f4930159e10 2023-10-09T06:36:11.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.197+0000 7f4937ab0700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f493015df20 con 0x7f4930159a40 2023-10-09T06:36:11.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.197+0000 7f493584c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4930159a40 0x7f49300c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:11.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.198+0000 7f493504b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f493015d7b0 0x7f49300c82c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:45450/0 (socket says 172.21.15.163:45450) 2023-10-09T06:36:11.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.198+0000 7f493504b700 1 -- 172.21.15.163:0/2334041575 learned_addr learned my addr 172.21.15.163:0/2334041575 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:11.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.198+0000 7f493584c700 1 -- 172.21.15.163:0/2334041575 >> v1:172.21.15.181:6789/0 conn(0x7f4930159e10 legacy=0x7f49300c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:11.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.198+0000 7f493584c700 1 -- 172.21.15.163:0/2334041575 >> v1:172.21.15.163:6789/0 conn(0x7f493015d7b0 legacy=0x7f49300c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:11.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.198+0000 7f493584c700 1 -- 172.21.15.163:0/2334041575 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493015e200 con 0x7f4930159a40 2023-10-09T06:36:11.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.198+0000 7f493484a700 1 -- 172.21.15.163:0/2334041575 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 101656598 0 0) 0x7f49300c90e0 con 0x7f493015d7b0 2023-10-09T06:36:11.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.198+0000 7f493584c700 1 --2- 172.21.15.163:0/2334041575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4930159a40 0x7f49300c9510 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f492c0098e0 tx=0x7f492c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb6d704f0e31e32 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:11.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.198+0000 7f493484a700 1 -- 172.21.15.163:0/2334041575 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c009e80 con 0x7f4930159a40 2023-10-09T06:36:11.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.198+0000 7f493484a700 1 -- 172.21.15.163:0/2334041575 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492c00b8f0 con 0x7f4930159a40 2023-10-09T06:36:11.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.198+0000 7f493484a700 1 -- 172.21.15.163:0/2334041575 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c00bc70 con 0x7f4930159a40 2023-10-09T06:36:11.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.199+0000 7f4937ab0700 1 -- 172.21.15.163:0/2334041575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4930159a40 msgr2=0x7f49300c9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:11.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.199+0000 7f4937ab0700 1 --2- 172.21.15.163:0/2334041575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4930159a40 0x7f49300c9510 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f492c0098e0 tx=0x7f492c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:11.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.199+0000 7f4937ab0700 1 -- 172.21.15.163:0/2334041575 shutdown_connections 2023-10-09T06:36:11.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.199+0000 7f4937ab0700 1 --2- 172.21.15.163:0/2334041575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4930159a40 0x7f49300c9510 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:11.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.199+0000 7f4937ab0700 1 -- 172.21.15.163:0/2334041575 >> 172.21.15.163:0/2334041575 conn(0x7f493006b450 msgr2=0x7f49300cd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:11.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.199+0000 7f4937ab0700 1 -- 172.21.15.163:0/2334041575 shutdown_connections 2023-10-09T06:36:11.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.199+0000 7f4937ab0700 1 -- 172.21.15.163:0/2334041575 wait complete. 2023-10-09T06:36:11.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.200+0000 7f4937ab0700 1 Processor -- start 2023-10-09T06:36:11.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.200+0000 7f4937ab0700 1 -- start start 2023-10-09T06:36:11.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.201+0000 7f4937ab0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4930159a40 0x7f49300c7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:11.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.201+0000 7f4937ab0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4930159e10 0x7f49300c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:11.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.201+0000 7f4937ab0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f493015d7b0 0x7f493016d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:11.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.201+0000 7f4937ab0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49300c4c30 con 0x7f493015d7b0 2023-10-09T06:36:11.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.201+0000 7f4937ab0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49300c4e60 con 0x7f4930159a40 2023-10-09T06:36:11.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.201+0000 7f4937ab0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493015bb10 con 0x7f4930159e10 2023-10-09T06:36:11.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.201+0000 7f493604d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f493015d7b0 0x7f493016d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:11.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.201+0000 7f493604d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f493015d7b0 0x7f493016d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35502/0 (socket says 172.21.15.163:35502) 2023-10-09T06:36:11.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.201+0000 7f493604d700 1 -- 172.21.15.163:0/59403371 learned_addr learned my addr 172.21.15.163:0/59403371 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:11.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.201+0000 7f493584c700 1 --2- 172.21.15.163:0/59403371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4930159a40 0x7f49300c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:11.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.201+0000 7f493504b700 1 --2- 172.21.15.163:0/59403371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4930159e10 0x7f49300c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:11.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.201+0000 7f493604d700 1 -- 172.21.15.163:0/59403371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4930159e10 msgr2=0x7f49300c46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:11.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.201+0000 7f493604d700 1 --2- 172.21.15.163:0/59403371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4930159e10 0x7f49300c46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:11.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.201+0000 7f493604d700 1 -- 172.21.15.163:0/59403371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4930159a40 msgr2=0x7f49300c7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:11.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.201+0000 7f493604d700 1 --2- 172.21.15.163:0/59403371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4930159a40 0x7f49300c7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:11.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.201+0000 7f493604d700 1 -- 172.21.15.163:0/59403371 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c009420 con 0x7f493015d7b0 2023-10-09T06:36:11.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.201+0000 7f493504b700 1 --2- 172.21.15.163:0/59403371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4930159e10 0x7f49300c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:11.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.201+0000 7f493584c700 1 --2- 172.21.15.163:0/59403371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4930159a40 0x7f49300c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:11.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.201+0000 7f493604d700 1 --2- 172.21.15.163:0/59403371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f493015d7b0 0x7f493016d590 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f492800c940 tx=0x7f49280016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:11.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.201+0000 7f4925ffb700 1 -- 172.21.15.163:0/59403371 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492800a7a0 con 0x7f493015d7b0 2023-10-09T06:36:11.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.201+0000 7f4925ffb700 1 -- 172.21.15.163:0/59403371 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492800a900 con 0x7f493015d7b0 2023-10-09T06:36:11.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.201+0000 7f4937ab0700 1 -- 172.21.15.163:0/59403371 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f493015b1e0 con 0x7f493015d7b0 2023-10-09T06:36:11.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.201+0000 7f4925ffb700 1 -- 172.21.15.163:0/59403371 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492800acc0 con 0x7f493015d7b0 2023-10-09T06:36:11.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.202+0000 7f4937ab0700 1 -- 172.21.15.163:0/59403371 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f493015b7a0 con 0x7f493015d7b0 2023-10-09T06:36:11.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.202+0000 7f4937ab0700 1 -- 172.21.15.163:0/59403371 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f493004c3c0 con 0x7f493015d7b0 2023-10-09T06:36:11.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.203+0000 7f4925ffb700 1 -- 172.21.15.163:0/59403371 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f4928012630 con 0x7f493015d7b0 2023-10-09T06:36:11.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.203+0000 7f4925ffb700 1 --2- 172.21.15.163:0/59403371 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4914067200 0x7f49140696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:11.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.203+0000 7f4925ffb700 1 -- 172.21.15.163:0/59403371 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f4928084990 con 0x7f493015d7b0 2023-10-09T06:36:11.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.204+0000 7f493584c700 1 --2- 172.21.15.163:0/59403371 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4914067200 0x7f49140696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:11.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.204+0000 7f493584c700 1 --2- 172.21.15.163:0/59403371 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4914067200 0x7f49140696b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f492c0093f0 tx=0x7f492c017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:11.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.208+0000 7f4925ffb700 1 -- 172.21.15.163:0/59403371 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4928051b70 con 0x7f493015d7b0 2023-10-09T06:36:11.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.473+0000 7f4937ab0700 1 -- 172.21.15.163:0/59403371 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag"]} v 0) v1 -- 0x7f493015bc50 con 0x7f493015d7b0 2023-10-09T06:36:11.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.728+0000 7f4925ffb700 1 -- 172.21.15.163:0/59403371 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag"]}]=0 v264) v1 ==== 158+0+0 (secure 0 0 0) 0x7f4928016020 con 0x7f493015d7b0 2023-10-09T06:36:11.731 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.731+0000 7f4937ab0700 1 -- 172.21.15.163:0/59403371 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4914067200 msgr2=0x7f49140696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:11.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.731+0000 7f4937ab0700 1 --2- 172.21.15.163:0/59403371 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4914067200 0x7f49140696b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f492c0093f0 tx=0x7f492c017000 comp rx=0 tx=0).stop 2023-10-09T06:36:11.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.731+0000 7f4937ab0700 1 -- 172.21.15.163:0/59403371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f493015d7b0 msgr2=0x7f493016d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:11.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.731+0000 7f4937ab0700 1 --2- 172.21.15.163:0/59403371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f493015d7b0 0x7f493016d590 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f492800c940 tx=0x7f49280016f0 comp rx=0 tx=0).stop 2023-10-09T06:36:11.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.731+0000 7f4937ab0700 1 -- 172.21.15.163:0/59403371 shutdown_connections 2023-10-09T06:36:11.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.731+0000 7f4937ab0700 1 --2- 172.21.15.163:0/59403371 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4914067200 0x7f49140696b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:11.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.731+0000 7f4937ab0700 1 --2- 172.21.15.163:0/59403371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f493015d7b0 0x7f493016d590 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:11.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.731+0000 7f4937ab0700 1 --2- 172.21.15.163:0/59403371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4930159a40 0x7f49300c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:11.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.731+0000 7f4937ab0700 1 --2- 172.21.15.163:0/59403371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4930159e10 0x7f49300c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:11.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.731+0000 7f4937ab0700 1 -- 172.21.15.163:0/59403371 >> 172.21.15.163:0/59403371 conn(0x7f493006b450 msgr2=0x7f4930152f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:11.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.731+0000 7f4937ab0700 1 -- 172.21.15.163:0/59403371 shutdown_connections 2023-10-09T06:36:11.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.731+0000 7f4937ab0700 1 -- 172.21.15.163:0/59403371 wait complete. 2023-10-09T06:36:11.744 INFO:tasks.cephfs_test_runner:Tests the 'fs volume info --human_readable' command ... ok 2023-10-09T06:36:11.744 INFO:tasks.cephfs_test_runner:Starting test: test_volume_info_with_human_readable_flag_without_subvolumegroup (tasks.cephfs.test_volumes.TestVolumes) Tests the 'fs volume info --human_readable' command without subvolume group 2023-10-09T06:36:11.744 INFO:tasks.cephfs_test_runner:test_volume_info_with_human_readable_flag_without_subvolumegroup (tasks.cephfs.test_volumes.TestVolumes) 2023-10-09T06:36:11.744 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup' 2023-10-09T06:36:11.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.859+0000 7f7f98bda700 1 Processor -- start 2023-10-09T06:36:11.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.859+0000 7f7f98bda700 1 -- start start 2023-10-09T06:36:11.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.860+0000 7f7f98bda700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7f94159890 0x7f7f9415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:11.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.860+0000 7f7f98bda700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7f94069560 0x7f7f940699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:11.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.860+0000 7f7f98bda700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f94069ee0 con 0x7f7f94159c60 2023-10-09T06:36:11.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.860+0000 7f7f98bda700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f9415a160 con 0x7f7f94159890 2023-10-09T06:36:11.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.860+0000 7f7f98bda700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f940d13a0 con 0x7f7f94069560 2023-10-09T06:36:11.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.860+0000 7f7f9259c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7f94159890 0x7f7f9415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:11.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.860+0000 7f7f9259c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7f94159890 0x7f7f9415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35512/0 (socket says 172.21.15.163:35512) 2023-10-09T06:36:11.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.860+0000 7f7f9259c700 1 -- 172.21.15.163:0/686395042 learned_addr learned my addr 172.21.15.163:0/686395042 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:11.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.860+0000 7f7f92d9d700 1 --2- 172.21.15.163:0/686395042 >> v2:172.21.15.181:3300/0 conn(0x7f7f94069560 0x7f7f940699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:11.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.860+0000 7f7f9259c700 1 -- 172.21.15.163:0/686395042 >> v2:172.21.15.181:3300/0 conn(0x7f7f94069560 msgr2=0x7f7f940699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:11.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.860+0000 7f7f9259c700 1 --2- 172.21.15.163:0/686395042 >> v2:172.21.15.181:3300/0 conn(0x7f7f94069560 0x7f7f940699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:11.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.860+0000 7f7f9259c700 1 -- 172.21.15.163:0/686395042 >> v1:172.21.15.181:6789/0 conn(0x7f7f94159c60 legacy=0x7f7f9415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:11.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.860+0000 7f7f9259c700 1 -- 172.21.15.163:0/686395042 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9415e140 con 0x7f7f94159890 2023-10-09T06:36:11.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.860+0000 7f7f9259c700 1 --2- 172.21.15.163:0/686395042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f94159890 0x7f7f9415abd0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f7f9406a890 tx=0x7f7f8c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d666c6d4934683 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:11.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.860+0000 7f7f9159a700 1 -- 172.21.15.163:0/686395042 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c00e070 con 0x7f7f94159890 2023-10-09T06:36:11.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.860+0000 7f7f9159a700 1 -- 172.21.15.163:0/686395042 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8c00ba10 con 0x7f7f94159890 2023-10-09T06:36:11.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.860+0000 7f7f9159a700 1 -- 172.21.15.163:0/686395042 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c0092e0 con 0x7f7f94159890 2023-10-09T06:36:11.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.861+0000 7f7f98bda700 1 -- 172.21.15.163:0/686395042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f94159890 msgr2=0x7f7f9415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:11.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.861+0000 7f7f98bda700 1 --2- 172.21.15.163:0/686395042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f94159890 0x7f7f9415abd0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f7f9406a890 tx=0x7f7f8c009840 comp rx=0 tx=0).stop 2023-10-09T06:36:11.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.861+0000 7f7f98bda700 1 -- 172.21.15.163:0/686395042 shutdown_connections 2023-10-09T06:36:11.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.861+0000 7f7f98bda700 1 --2- 172.21.15.163:0/686395042 >> v2:172.21.15.181:3300/0 conn(0x7f7f94069560 0x7f7f940699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:11.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.861+0000 7f7f98bda700 1 --2- 172.21.15.163:0/686395042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f94159890 0x7f7f9415abd0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:11.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.861+0000 7f7f98bda700 1 -- 172.21.15.163:0/686395042 >> 172.21.15.163:0/686395042 conn(0x7f7f94068e20 msgr2=0x7f7f940d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:11.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.861+0000 7f7f98bda700 1 -- 172.21.15.163:0/686395042 shutdown_connections 2023-10-09T06:36:11.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.861+0000 7f7f98bda700 1 -- 172.21.15.163:0/686395042 wait complete. 2023-10-09T06:36:11.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.862+0000 7f7f98bda700 1 Processor -- start 2023-10-09T06:36:11.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.862+0000 7f7f98bda700 1 -- start start 2023-10-09T06:36:11.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.863+0000 7f7f98bda700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f94069560 0x7f7f9416a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:11.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.863+0000 7f7f98bda700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f94159890 0x7f7f94152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:11.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.863+0000 7f7f98bda700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f94159c60 0x7f7f940c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:11.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.863+0000 7f7f98bda700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f940c8a50 con 0x7f7f94159c60 2023-10-09T06:36:11.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.863+0000 7f7f98bda700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9416aae0 con 0x7f7f94069560 2023-10-09T06:36:11.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.863+0000 7f7f98bda700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f940c7e40 con 0x7f7f94159890 2023-10-09T06:36:11.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.863+0000 7f7f92d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f94159c60 0x7f7f940c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:11.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.863+0000 7f7f92d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f94159c60 0x7f7f940c97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35528/0 (socket says 172.21.15.163:35528) 2023-10-09T06:36:11.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.863+0000 7f7f92d9d700 1 -- 172.21.15.163:0/2903983559 learned_addr learned my addr 172.21.15.163:0/2903983559 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:11.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.863+0000 7f7f9259c700 1 --2- 172.21.15.163:0/2903983559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f94069560 0x7f7f9416a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:11.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.863+0000 7f7f91d9b700 1 --2- 172.21.15.163:0/2903983559 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f94159890 0x7f7f94152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:11.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.863+0000 7f7f92d9d700 1 -- 172.21.15.163:0/2903983559 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f94159890 msgr2=0x7f7f94152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:11.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.863+0000 7f7f92d9d700 1 --2- 172.21.15.163:0/2903983559 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f94159890 0x7f7f94152320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:11.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.863+0000 7f7f92d9d700 1 -- 172.21.15.163:0/2903983559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f94069560 msgr2=0x7f7f9416a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:11.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.863+0000 7f7f92d9d700 1 --2- 172.21.15.163:0/2903983559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f94069560 0x7f7f9416a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:11.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.863+0000 7f7f92d9d700 1 -- 172.21.15.163:0/2903983559 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c009ae0 con 0x7f7f94159c60 2023-10-09T06:36:11.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.863+0000 7f7f91d9b700 1 --2- 172.21.15.163:0/2903983559 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f94159890 0x7f7f94152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:11.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.863+0000 7f7f92d9d700 1 --2- 172.21.15.163:0/2903983559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f94159c60 0x7f7f940c97a0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f7f8400c880 tx=0x7f7f84008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:11.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.864+0000 7f7f8a7fc700 1 -- 172.21.15.163:0/2903983559 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f84011600 con 0x7f7f94159c60 2023-10-09T06:36:11.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.864+0000 7f7f8a7fc700 1 -- 172.21.15.163:0/2903983559 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f84011760 con 0x7f7f94159c60 2023-10-09T06:36:11.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.864+0000 7f7f98bda700 1 -- 172.21.15.163:0/2903983559 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f940c8120 con 0x7f7f94159c60 2023-10-09T06:36:11.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.864+0000 7f7f8a7fc700 1 -- 172.21.15.163:0/2903983559 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f84011ae0 con 0x7f7f94159c60 2023-10-09T06:36:11.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.864+0000 7f7f98bda700 1 -- 172.21.15.163:0/2903983559 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f940c86e0 con 0x7f7f94159c60 2023-10-09T06:36:11.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.864+0000 7f7f98bda700 1 -- 172.21.15.163:0/2903983559 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f9404c3c0 con 0x7f7f94159c60 2023-10-09T06:36:11.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.865+0000 7f7f8a7fc700 1 -- 172.21.15.163:0/2903983559 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f7f840193f0 con 0x7f7f94159c60 2023-10-09T06:36:11.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.865+0000 7f7f8a7fc700 1 --2- 172.21.15.163:0/2903983559 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7f74067200 0x7f7f740696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:11.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.865+0000 7f7f8a7fc700 1 -- 172.21.15.163:0/2903983559 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f7f84084a80 con 0x7f7f94159c60 2023-10-09T06:36:11.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.865+0000 7f7f9259c700 1 --2- 172.21.15.163:0/2903983559 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7f74067200 0x7f7f740696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:11.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.866+0000 7f7f9259c700 1 --2- 172.21.15.163:0/2903983559 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7f74067200 0x7f7f740696b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c009880 tx=0x7f7f8c017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:11.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:11.869+0000 7f7f8a7fc700 1 -- 172.21.15.163:0/2903983559 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f84086e60 con 0x7f7f94159c60 2023-10-09T06:36:12.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.129+0000 7f7f98bda700 1 -- 172.21.15.163:0/2903983559 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup"]} v 0) v1 -- 0x7f7f9406a5b0 con 0x7f7f94159c60 2023-10-09T06:36:12.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.733+0000 7f7f8a7fc700 1 -- 172.21.15.163:0/2903983559 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup"]}]=0 v265) v1 ==== 184+0+0 (secure 0 0 0) 0x7f7f84051b90 con 0x7f7f94159c60 2023-10-09T06:36:12.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.735+0000 7f7f98bda700 1 -- 172.21.15.163:0/2903983559 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7f74067200 msgr2=0x7f7f740696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:12.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.735+0000 7f7f98bda700 1 --2- 172.21.15.163:0/2903983559 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7f74067200 0x7f7f740696b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c009880 tx=0x7f7f8c017000 comp rx=0 tx=0).stop 2023-10-09T06:36:12.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.735+0000 7f7f98bda700 1 -- 172.21.15.163:0/2903983559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f94159c60 msgr2=0x7f7f940c97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:12.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.735+0000 7f7f98bda700 1 --2- 172.21.15.163:0/2903983559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f94159c60 0x7f7f940c97a0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f7f8400c880 tx=0x7f7f84008040 comp rx=0 tx=0).stop 2023-10-09T06:36:12.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.735+0000 7f7f98bda700 1 -- 172.21.15.163:0/2903983559 shutdown_connections 2023-10-09T06:36:12.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.735+0000 7f7f98bda700 1 --2- 172.21.15.163:0/2903983559 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7f74067200 0x7f7f740696b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:12.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.735+0000 7f7f98bda700 1 --2- 172.21.15.163:0/2903983559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f94159c60 0x7f7f940c97a0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:12.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.735+0000 7f7f98bda700 1 --2- 172.21.15.163:0/2903983559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f94069560 0x7f7f9416a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:12.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.735+0000 7f7f98bda700 1 --2- 172.21.15.163:0/2903983559 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f94159890 0x7f7f94152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:12.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.735+0000 7f7f98bda700 1 -- 172.21.15.163:0/2903983559 >> 172.21.15.163:0/2903983559 conn(0x7f7f94068e20 msgr2=0x7f7f940b1420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:12.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.735+0000 7f7f98bda700 1 -- 172.21.15.163:0/2903983559 shutdown_connections 2023-10-09T06:36:12.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.735+0000 7f7f98bda700 1 -- 172.21.15.163:0/2903983559 wait complete. 2023-10-09T06:36:12.749 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-09T06:36:12.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.867+0000 7f223ffb5700 1 Processor -- start 2023-10-09T06:36:12.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.867+0000 7f223ffb5700 1 -- start start 2023-10-09T06:36:12.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.868+0000 7f223ffb5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2238159c60 0x7f223815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:12.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.868+0000 7f223ffb5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f223815db10 0x7f2238069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:12.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.868+0000 7f223ffb5700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f223815a160 con 0x7f2238159890 2023-10-09T06:36:12.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.868+0000 7f223ffb5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22380d13a0 con 0x7f223815db10 2023-10-09T06:36:12.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.868+0000 7f223ffb5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f223815e140 con 0x7f2238159c60 2023-10-09T06:36:12.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.868+0000 7f223dd51700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f2238159890 0x7f223815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:45460/0 (socket says 172.21.15.163:45460) 2023-10-09T06:36:12.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.868+0000 7f223dd51700 1 -- 172.21.15.163:0/2500713761 learned_addr learned my addr 172.21.15.163:0/2500713761 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:12.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.868+0000 7f223d550700 1 --2- 172.21.15.163:0/2500713761 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2238159c60 0x7f223815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:12.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.868+0000 7f223e552700 1 --2- 172.21.15.163:0/2500713761 >> v2:172.21.15.181:3300/0 conn(0x7f223815db10 0x7f2238069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:12.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.868+0000 7f223cd4f700 1 -- 172.21.15.163:0/2500713761 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3008438884 0 0) 0x7f223815a160 con 0x7f2238159890 2023-10-09T06:36:12.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.868+0000 7f223cd4f700 1 -- 172.21.15.163:0/2500713761 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2224003570 con 0x7f2238159890 2023-10-09T06:36:12.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.868+0000 7f223cd4f700 1 -- 172.21.15.163:0/2500713761 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 622095723 0 0) 0x7f2224003570 con 0x7f2238159890 2023-10-09T06:36:12.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.868+0000 7f223cd4f700 1 -- 172.21.15.163:0/2500713761 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2238159c60 msgr2=0x7f223815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:12.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.868+0000 7f223cd4f700 1 --2- 172.21.15.163:0/2500713761 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2238159c60 0x7f223815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:12.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.868+0000 7f223cd4f700 1 -- 172.21.15.163:0/2500713761 >> v2:172.21.15.181:3300/0 conn(0x7f223815db10 msgr2=0x7f2238069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:12.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.868+0000 7f223cd4f700 1 --2- 172.21.15.163:0/2500713761 >> v2:172.21.15.181:3300/0 conn(0x7f223815db10 0x7f2238069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:12.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.868+0000 7f223cd4f700 1 -- 172.21.15.163:0/2500713761 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223815a3d0 con 0x7f2238159890 2023-10-09T06:36:12.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.868+0000 7f223cd4f700 1 -- 172.21.15.163:0/2500713761 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f22340017f0 con 0x7f2238159890 2023-10-09T06:36:12.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.868+0000 7f223cd4f700 1 -- 172.21.15.163:0/2500713761 >> v1:172.21.15.163:6789/0 conn(0x7f2238159890 legacy=0x7f223815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:12.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.868+0000 7f223cd4f700 1 --2- 172.21.15.163:0/2500713761 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2224003a60 0x7f2224003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:12.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.869+0000 7f223cd4f700 1 --2- 172.21.15.163:0/2500713761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2224004810 0x7f2224004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:12.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.869+0000 7f223cd4f700 1 --2- 172.21.15.163:0/2500713761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22240051c0 0x7f222400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:12.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.869+0000 7f223cd4f700 1 -- 172.21.15.163:0/2500713761 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22380d13a0 con 0x7f22240051c0 2023-10-09T06:36:12.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.869+0000 7f223cd4f700 1 -- 172.21.15.163:0/2500713761 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223815e140 con 0x7f2224004810 2023-10-09T06:36:12.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.869+0000 7f223cd4f700 1 -- 172.21.15.163:0/2500713761 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f222400bfa0 con 0x7f2224003a60 2023-10-09T06:36:12.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.869+0000 7f223d550700 1 --2- 172.21.15.163:0/2500713761 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2238159c60 0x7f223815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:36:12.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.869+0000 7f223e552700 1 --2- 172.21.15.163:0/2500713761 >> v2:172.21.15.181:3300/0 conn(0x7f223815db10 0x7f2238069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:36:12.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.869+0000 7f223dd51700 1 --2- 172.21.15.163:0/2500713761 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2224003a60 0x7f2224003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:12.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.869+0000 7f223e552700 1 --2- 172.21.15.163:0/2500713761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22240051c0 0x7f222400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:12.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.869+0000 7f223d550700 1 --2- 172.21.15.163:0/2500713761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2224004810 0x7f2224004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:12.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.869+0000 7f223dd51700 1 -- 172.21.15.163:0/2500713761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2224004810 msgr2=0x7f2224004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:12.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.869+0000 7f223dd51700 1 --2- 172.21.15.163:0/2500713761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2224004810 0x7f2224004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:12.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.869+0000 7f223dd51700 1 -- 172.21.15.163:0/2500713761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22240051c0 msgr2=0x7f222400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:12.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.869+0000 7f223dd51700 1 --2- 172.21.15.163:0/2500713761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22240051c0 0x7f222400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:12.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.869+0000 7f223dd51700 1 -- 172.21.15.163:0/2500713761 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2224003570 con 0x7f2224003a60 2023-10-09T06:36:12.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.869+0000 7f223e552700 1 --2- 172.21.15.163:0/2500713761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22240051c0 0x7f222400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:36:12.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.869+0000 7f223dd51700 1 --2- 172.21.15.163:0/2500713761 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2224003a60 0x7f2224003e70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f223400a970 tx=0x7f2234002210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec64737e3c26a109 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:12.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.869+0000 7f223cd4f700 1 -- 172.21.15.163:0/2500713761 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22340070d0 con 0x7f2224003a60 2023-10-09T06:36:12.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.869+0000 7f223cd4f700 1 -- 172.21.15.163:0/2500713761 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2234007270 con 0x7f2224003a60 2023-10-09T06:36:12.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.869+0000 7f223cd4f700 1 -- 172.21.15.163:0/2500713761 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2234007630 con 0x7f2224003a60 2023-10-09T06:36:12.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.870+0000 7f223ffb5700 1 -- 172.21.15.163:0/2500713761 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2224003a60 msgr2=0x7f2224003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:12.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.870+0000 7f223ffb5700 1 --2- 172.21.15.163:0/2500713761 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2224003a60 0x7f2224003e70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f223400a970 tx=0x7f2234002210 comp rx=0 tx=0).stop 2023-10-09T06:36:12.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.870+0000 7f223e552700 1 -- 172.21.15.163:0/2500713761 reap_dead start 2023-10-09T06:36:12.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.870+0000 7f223ffb5700 1 -- 172.21.15.163:0/2500713761 shutdown_connections 2023-10-09T06:36:12.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.870+0000 7f223ffb5700 1 -- 172.21.15.163:0/2500713761 >> 172.21.15.163:0/2500713761 conn(0x7f2238068e20 msgr2=0x7f22380d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:12.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.870+0000 7f223ffb5700 1 -- 172.21.15.163:0/2500713761 shutdown_connections 2023-10-09T06:36:12.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.870+0000 7f223ffb5700 1 -- 172.21.15.163:0/2500713761 wait complete. 2023-10-09T06:36:12.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.871+0000 7f223ffb5700 1 Processor -- start 2023-10-09T06:36:12.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.871+0000 7f223ffb5700 1 -- start start 2023-10-09T06:36:12.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.872+0000 7f223ffb5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2224003a60 0x7f2238144360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:12.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.872+0000 7f223ffb5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22380b5f00 0x7f22380b4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:12.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.872+0000 7f223ffb5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22380c9b50 0x7f22380c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:12.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.872+0000 7f223ffb5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22380ca530 con 0x7f22380b5f00 2023-10-09T06:36:12.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.872+0000 7f223ffb5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22380b4480 con 0x7f22380c9b50 2023-10-09T06:36:12.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.872+0000 7f223ffb5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22380b3590 con 0x7f2224003a60 2023-10-09T06:36:12.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.872+0000 7f223d550700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22380b5f00 0x7f22380b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:12.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.872+0000 7f223d550700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22380b5f00 0x7f22380b4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35536/0 (socket says 172.21.15.163:35536) 2023-10-09T06:36:12.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.872+0000 7f223d550700 1 -- 172.21.15.163:0/1260031038 learned_addr learned my addr 172.21.15.163:0/1260031038 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:12.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.872+0000 7f223dd51700 1 --2- 172.21.15.163:0/1260031038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2224003a60 0x7f2238144360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:12.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.872+0000 7f223e552700 1 --2- 172.21.15.163:0/1260031038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22380c9b50 0x7f22380c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:12.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.872+0000 7f223d550700 1 -- 172.21.15.163:0/1260031038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2224003a60 msgr2=0x7f2238144360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:12.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.872+0000 7f223d550700 1 --2- 172.21.15.163:0/1260031038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2224003a60 0x7f2238144360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:12.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.872+0000 7f223d550700 1 -- 172.21.15.163:0/1260031038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22380c9b50 msgr2=0x7f22380c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:12.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.872+0000 7f223d550700 1 --2- 172.21.15.163:0/1260031038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22380c9b50 0x7f22380c9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:12.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.872+0000 7f223d550700 1 -- 172.21.15.163:0/1260031038 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22380b3870 con 0x7f22380b5f00 2023-10-09T06:36:12.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.872+0000 7f223d550700 1 --2- 172.21.15.163:0/1260031038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22380b5f00 0x7f22380b4910 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f2228011d20 tx=0x7f22280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:12.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.872+0000 7f222dffb700 1 -- 172.21.15.163:0/1260031038 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2228011ed0 con 0x7f22380b5f00 2023-10-09T06:36:12.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.873+0000 7f222dffb700 1 -- 172.21.15.163:0/1260031038 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222800fcd0 con 0x7f22380b5f00 2023-10-09T06:36:12.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.873+0000 7f222dffb700 1 -- 172.21.15.163:0/1260031038 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2228014670 con 0x7f22380b5f00 2023-10-09T06:36:12.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.873+0000 7f223ffb5700 1 -- 172.21.15.163:0/1260031038 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22380b3b50 con 0x7f22380b5f00 2023-10-09T06:36:12.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.873+0000 7f223ffb5700 1 -- 172.21.15.163:0/1260031038 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22380b4110 con 0x7f22380b5f00 2023-10-09T06:36:12.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.874+0000 7f223ffb5700 1 -- 172.21.15.163:0/1260031038 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22380b45f0 con 0x7f22380b5f00 2023-10-09T06:36:12.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.874+0000 7f222dffb700 1 -- 172.21.15.163:0/1260031038 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f2228014a20 con 0x7f22380b5f00 2023-10-09T06:36:12.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.874+0000 7f222dffb700 1 --2- 172.21.15.163:0/1260031038 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f221c067200 0x7f221c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:12.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.874+0000 7f222dffb700 1 -- 172.21.15.163:0/1260031038 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2228018070 con 0x7f22380b5f00 2023-10-09T06:36:12.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.875+0000 7f223dd51700 1 --2- 172.21.15.163:0/1260031038 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f221c067200 0x7f221c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:12.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.875+0000 7f223dd51700 1 --2- 172.21.15.163:0/1260031038 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f221c067200 0x7f221c0696b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f223400a970 tx=0x7f2234015000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:12.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:12.879+0000 7f222dffb700 1 -- 172.21.15.163:0/1260031038 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2228052540 con 0x7f22380b5f00 2023-10-09T06:36:13.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.060+0000 7f223ffb5700 1 -- 172.21.15.163:0/1260031038 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f223804c3c0 con 0x7f22380b5f00 2023-10-09T06:36:13.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.064+0000 7f222dffb700 1 -- 172.21.15.163:0/1260031038 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2228055b60 con 0x7f22380b5f00 2023-10-09T06:36:13.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.066+0000 7f223ffb5700 1 -- 172.21.15.163:0/1260031038 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f221c067200 msgr2=0x7f221c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:13.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.066+0000 7f223ffb5700 1 --2- 172.21.15.163:0/1260031038 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f221c067200 0x7f221c0696b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f223400a970 tx=0x7f2234015000 comp rx=0 tx=0).stop 2023-10-09T06:36:13.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.066+0000 7f223ffb5700 1 -- 172.21.15.163:0/1260031038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22380b5f00 msgr2=0x7f22380b4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:13.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.066+0000 7f223ffb5700 1 --2- 172.21.15.163:0/1260031038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22380b5f00 0x7f22380b4910 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f2228011d20 tx=0x7f22280091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:13.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.066+0000 7f223ffb5700 1 -- 172.21.15.163:0/1260031038 shutdown_connections 2023-10-09T06:36:13.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.066+0000 7f223ffb5700 1 --2- 172.21.15.163:0/1260031038 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f221c067200 0x7f221c0696b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:13.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.066+0000 7f223ffb5700 1 --2- 172.21.15.163:0/1260031038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22380c9b50 0x7f22380c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:13.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.066+0000 7f223ffb5700 1 --2- 172.21.15.163:0/1260031038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2224003a60 0x7f2238144360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:13.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.066+0000 7f223ffb5700 1 --2- 172.21.15.163:0/1260031038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22380b5f00 0x7f22380b4910 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:13.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.066+0000 7f223ffb5700 1 -- 172.21.15.163:0/1260031038 >> 172.21.15.163:0/1260031038 conn(0x7f2238068e20 msgr2=0x7f22380b1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:13.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.066+0000 7f223ffb5700 1 -- 172.21.15.163:0/1260031038 shutdown_connections 2023-10-09T06:36:13.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.066+0000 7f223ffb5700 1 -- 172.21.15.163:0/1260031038 wait complete. 2023-10-09T06:36:13.079 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-09T06:36:13.079 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:36:13.117 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:36:13.168 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-09T06:36:13.169 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-09T06:36:13.169 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:36:13.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.283+0000 7f4032be8700 1 Processor -- start 2023-10-09T06:36:13.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.283+0000 7f4032be8700 1 -- start start 2023-10-09T06:36:13.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.284+0000 7f4032be8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f402c159890 0x7f402c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:13.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.284+0000 7f4032be8700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f402c15a160 con 0x7f402c159c60 2023-10-09T06:36:13.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.284+0000 7f4032be8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f402c0d13a0 con 0x7f402c069560 2023-10-09T06:36:13.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.284+0000 7f4032be8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f402c15e140 con 0x7f402c159890 2023-10-09T06:36:13.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.284+0000 7f402bfff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f402c159c60 0x7f402c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:45476/0 (socket says 172.21.15.163:45476) 2023-10-09T06:36:13.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.284+0000 7f402bfff700 1 -- 172.21.15.163:0/663397663 learned_addr learned my addr 172.21.15.163:0/663397663 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:13.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.284+0000 7f4030984700 1 --2- 172.21.15.163:0/663397663 >> v2:172.21.15.181:3300/0 conn(0x7f402c159890 0x7f402c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:13.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.284+0000 7f402b7fe700 1 -- 172.21.15.163:0/663397663 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 351582232 0 0) 0x7f402c15a160 con 0x7f402c159c60 2023-10-09T06:36:13.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.284+0000 7f402b7fe700 1 -- 172.21.15.163:0/663397663 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f401c003570 con 0x7f402c159c60 2023-10-09T06:36:13.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.284+0000 7f4030984700 1 -- 172.21.15.163:0/663397663 >> v1:172.21.15.181:6789/0 conn(0x7f402c069560 legacy=0x7f402c0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:13.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.284+0000 7f402b7fe700 1 -- 172.21.15.163:0/663397663 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 363168135 0 0) 0x7f401c003570 con 0x7f402c159c60 2023-10-09T06:36:13.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.284+0000 7f4030984700 1 -- 172.21.15.163:0/663397663 >> v1:172.21.15.163:6789/0 conn(0x7f402c159c60 legacy=0x7f402c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:13.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.284+0000 7f4030984700 1 -- 172.21.15.163:0/663397663 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f402c15a3d0 con 0x7f402c159890 2023-10-09T06:36:13.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.284+0000 7f4030984700 1 --2- 172.21.15.163:0/663397663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f402c159890 0x7f402c15abd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f40200098e0 tx=0x7f40200091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a77b6fe493f2fe89 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:13.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.285+0000 7f402b7fe700 1 -- 172.21.15.163:0/663397663 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4018001470 con 0x7f402c159890 2023-10-09T06:36:13.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.285+0000 7f402b7fe700 1 -- 172.21.15.163:0/663397663 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4020011420 con 0x7f402c159890 2023-10-09T06:36:13.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.285+0000 7f402b7fe700 1 -- 172.21.15.163:0/663397663 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40200117c0 con 0x7f402c159890 2023-10-09T06:36:13.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.285+0000 7f4032be8700 1 -- 172.21.15.163:0/663397663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f402c159890 msgr2=0x7f402c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:13.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.285+0000 7f4032be8700 1 --2- 172.21.15.163:0/663397663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f402c159890 0x7f402c15abd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f40200098e0 tx=0x7f40200091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:13.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.285+0000 7f4032be8700 1 -- 172.21.15.163:0/663397663 shutdown_connections 2023-10-09T06:36:13.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.285+0000 7f4032be8700 1 --2- 172.21.15.163:0/663397663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f402c159890 0x7f402c15abd0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:13.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.285+0000 7f4032be8700 1 -- 172.21.15.163:0/663397663 >> 172.21.15.163:0/663397663 conn(0x7f402c068e20 msgr2=0x7f402c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:13.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.285+0000 7f4032be8700 1 -- 172.21.15.163:0/663397663 shutdown_connections 2023-10-09T06:36:13.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.285+0000 7f4032be8700 1 -- 172.21.15.163:0/663397663 wait complete. 2023-10-09T06:36:13.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.286+0000 7f4032be8700 1 Processor -- start 2023-10-09T06:36:13.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.286+0000 7f4032be8700 1 -- start start 2023-10-09T06:36:13.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.287+0000 7f4032be8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f402c069560 0x7f402c151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:13.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.287+0000 7f4032be8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f402c159890 0x7f402c15c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:13.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.287+0000 7f4032be8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f402c159c60 0x7f402c14d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:13.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.287+0000 7f4032be8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402c151b00 con 0x7f402c159c60 2023-10-09T06:36:13.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.287+0000 7f4032be8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402c151de0 con 0x7f402c069560 2023-10-09T06:36:13.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.287+0000 7f4032be8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f402c1520c0 con 0x7f402c159890 2023-10-09T06:36:13.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.287+0000 7f4031185700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f402c159c60 0x7f402c14d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:13.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.287+0000 7f4031185700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f402c159c60 0x7f402c14d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35538/0 (socket says 172.21.15.163:35538) 2023-10-09T06:36:13.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.287+0000 7f4031185700 1 -- 172.21.15.163:0/3447376590 learned_addr learned my addr 172.21.15.163:0/3447376590 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:13.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.287+0000 7f402bfff700 1 --2- 172.21.15.163:0/3447376590 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f402c159890 0x7f402c15c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:13.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.287+0000 7f4030984700 1 --2- 172.21.15.163:0/3447376590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f402c069560 0x7f402c151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:13.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.288+0000 7f4031185700 1 -- 172.21.15.163:0/3447376590 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f402c159890 msgr2=0x7f402c15c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:13.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.288+0000 7f4031185700 1 --2- 172.21.15.163:0/3447376590 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f402c159890 0x7f402c15c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:13.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.288+0000 7f4031185700 1 -- 172.21.15.163:0/3447376590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f402c069560 msgr2=0x7f402c151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:13.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.288+0000 7f4031185700 1 --2- 172.21.15.163:0/3447376590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f402c069560 0x7f402c151400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:13.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.288+0000 7f4031185700 1 -- 172.21.15.163:0/3447376590 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4020009420 con 0x7f402c159c60 2023-10-09T06:36:13.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.288+0000 7f4030984700 1 --2- 172.21.15.163:0/3447376590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f402c069560 0x7f402c151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:13.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.288+0000 7f4031185700 1 --2- 172.21.15.163:0/3447376590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f402c159c60 0x7f402c14d1b0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f402400c8e0 tx=0x7f4024001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:13.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.288+0000 7f4028ff9700 1 -- 172.21.15.163:0/3447376590 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402400ce80 con 0x7f402c159c60 2023-10-09T06:36:13.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.288+0000 7f4028ff9700 1 -- 172.21.15.163:0/3447376590 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402400a870 con 0x7f402c159c60 2023-10-09T06:36:13.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.288+0000 7f4028ff9700 1 -- 172.21.15.163:0/3447376590 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402400ac10 con 0x7f402c159c60 2023-10-09T06:36:13.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.288+0000 7f4032be8700 1 -- 172.21.15.163:0/3447376590 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f402c1523a0 con 0x7f402c159c60 2023-10-09T06:36:13.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.288+0000 7f4032be8700 1 -- 172.21.15.163:0/3447376590 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f402c14d6f0 con 0x7f402c159c60 2023-10-09T06:36:13.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.289+0000 7f4032be8700 1 -- 172.21.15.163:0/3447376590 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f402c1528c0 con 0x7f402c159c60 2023-10-09T06:36:13.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.289+0000 7f4028ff9700 1 -- 172.21.15.163:0/3447376590 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f4024012420 con 0x7f402c159c60 2023-10-09T06:36:13.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.290+0000 7f4028ff9700 1 --2- 172.21.15.163:0/3447376590 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4010067200 0x7f40100696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:13.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.290+0000 7f4028ff9700 1 -- 172.21.15.163:0/3447376590 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f4024084a00 con 0x7f402c159c60 2023-10-09T06:36:13.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.290+0000 7f4030984700 1 --2- 172.21.15.163:0/3447376590 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4010067200 0x7f40100696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:13.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.290+0000 7f4030984700 1 --2- 172.21.15.163:0/3447376590 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4010067200 0x7f40100696b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f40200091e0 tx=0x7f4020016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:13.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.294+0000 7f4028ff9700 1 -- 172.21.15.163:0/3447376590 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f402c1528c0 con 0x7f402c159c60 2023-10-09T06:36:13.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.516+0000 7f4032be8700 1 -- 172.21.15.163:0/3447376590 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f402c1528c0 con 0x7f402c159c60 2023-10-09T06:36:13.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.517+0000 7f4028ff9700 1 -- 172.21.15.163:0/3447376590 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+3541 (secure 0 0 0) 0x7f402c1528c0 con 0x7f402c159c60 2023-10-09T06:36:13.518 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:13.518 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":51,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1946291256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1946291256},{"type":"v1","addr":"172.21.15.181:6837","nonce":1946291256}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":5749,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1366951098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1366951098},{"type":"v1","addr":"172.21.15.181:6836","nonce":1366951098}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3619934997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3619934997},{"type":"v1","addr":"172.21.15.163:6834","nonce":3619934997}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6444,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/4134903085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":4134903085},{"type":"v1","addr":"172.21.15.163:6835","nonce":4134903085}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[]} 2023-10-09T06:36:13.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.519+0000 7f4032be8700 1 -- 172.21.15.163:0/3447376590 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4010067200 msgr2=0x7f40100696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:13.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.519+0000 7f4032be8700 1 --2- 172.21.15.163:0/3447376590 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4010067200 0x7f40100696b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f40200091e0 tx=0x7f4020016000 comp rx=0 tx=0).stop 2023-10-09T06:36:13.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.519+0000 7f4032be8700 1 -- 172.21.15.163:0/3447376590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f402c159c60 msgr2=0x7f402c14d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:13.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.519+0000 7f4032be8700 1 --2- 172.21.15.163:0/3447376590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f402c159c60 0x7f402c14d1b0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f402400c8e0 tx=0x7f4024001710 comp rx=0 tx=0).stop 2023-10-09T06:36:13.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.519+0000 7f4032be8700 1 -- 172.21.15.163:0/3447376590 shutdown_connections 2023-10-09T06:36:13.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.519+0000 7f4032be8700 1 --2- 172.21.15.163:0/3447376590 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4010067200 0x7f40100696b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:13.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.519+0000 7f4032be8700 1 --2- 172.21.15.163:0/3447376590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f402c159c60 0x7f402c14d1b0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:13.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.519+0000 7f4032be8700 1 --2- 172.21.15.163:0/3447376590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f402c069560 0x7f402c151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:13.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.519+0000 7f4032be8700 1 --2- 172.21.15.163:0/3447376590 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f402c159890 0x7f402c15c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:13.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.519+0000 7f4032be8700 1 -- 172.21.15.163:0/3447376590 >> 172.21.15.163:0/3447376590 conn(0x7f402c068e20 msgr2=0x7f402c0cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:13.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.519+0000 7f4032be8700 1 -- 172.21.15.163:0/3447376590 shutdown_connections 2023-10-09T06:36:13.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.519+0000 7f4032be8700 1 -- 172.21.15.163:0/3447376590 wait complete. 2023-10-09T06:36:13.521 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 51 2023-10-09T06:36:13.532 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-09T06:36:13.532 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-09T06:36:13.533 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-09T06:36:13.533 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:36:13.533 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-09T06:36:13.533 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-09T06:36:13.533 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-09T06:36:13.533 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:36:13.534 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-09T06:36:13.534 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-09T06:36:13.534 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-09T06:36:13.534 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:36:13.534 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-09T06:36:13.534 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-09T06:36:13.534 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-09T06:36:13.534 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:36:13.541 INFO:tasks.ceph.mds.a:Stopped 2023-10-09T06:36:13.541 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-09T06:36:13.542 INFO:tasks.ceph.mds.c:Stopped 2023-10-09T06:36:13.542 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-09T06:36:13.544 INFO:tasks.ceph.mds.b:Stopped 2023-10-09T06:36:13.544 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-09T06:36:13.545 INFO:tasks.ceph.mds.d:Stopped 2023-10-09T06:36:13.545 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-09T06:36:13.584 INFO:tasks.ceph.mds.a:Started 2023-10-09T06:36:13.584 DEBUG:teuthology.parallel:result is None 2023-10-09T06:36:13.585 INFO:tasks.ceph.mds.c:Started 2023-10-09T06:36:13.585 DEBUG:teuthology.parallel:result is None 2023-10-09T06:36:13.587 INFO:tasks.ceph.mds.b:Started 2023-10-09T06:36:13.587 DEBUG:teuthology.parallel:result is None 2023-10-09T06:36:13.587 INFO:tasks.ceph.mds.d:Started 2023-10-09T06:36:13.588 DEBUG:teuthology.parallel:result is None 2023-10-09T06:36:13.588 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-09T06:36:13.657 INFO:tasks.ceph.mds.c.smithi163.stdout:starting mds.c at 2023-10-09T06:36:13.658 INFO:tasks.ceph.mds.a.smithi163.stdout:starting mds.a at 2023-10-09T06:36:13.666 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-09T06:36:13.667 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-09T06:36:13.667 INFO:tasks.ceph.mds.b.smithi181.stderr:2023-10-09T06:36:13.666+0000 7f1a289e2700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-09T06:36:13.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.708+0000 7f90bf1ef700 1 Processor -- start 2023-10-09T06:36:13.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.708+0000 7f90bf1ef700 1 -- start start 2023-10-09T06:36:13.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.709+0000 7f90bf1ef700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f90b8143050 0x7f90b8159c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:13.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.709+0000 7f90bf1ef700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f90b80b3f20 0x7f90b806be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:13.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.709+0000 7f90bf1ef700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b806c390 con 0x7f90b806b9e0 2023-10-09T06:36:13.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.709+0000 7f90bf1ef700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90b8143420 con 0x7f90b80b3f20 2023-10-09T06:36:13.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.709+0000 7f90bf1ef700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90b806a1e0 con 0x7f90b8143050 2023-10-09T06:36:13.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.709+0000 7f90be9ee700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f90b80b3f20 0x7f90b806be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:13.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.709+0000 7f90be9ee700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f90b80b3f20 0x7f90b806be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35584/0 (socket says 172.21.15.163:35584) 2023-10-09T06:36:13.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.709+0000 7f90be9ee700 1 -- 172.21.15.163:0/4019213030 learned_addr learned my addr 172.21.15.163:0/4019213030 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:13.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.709+0000 7f90be1ed700 1 --2- 172.21.15.163:0/4019213030 >> v2:172.21.15.181:3300/0 conn(0x7f90b8143050 0x7f90b8159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:13.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.709+0000 7f90be9ee700 1 -- 172.21.15.163:0/4019213030 >> v2:172.21.15.181:3300/0 conn(0x7f90b8143050 msgr2=0x7f90b8159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:13.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.709+0000 7f90be9ee700 1 --2- 172.21.15.163:0/4019213030 >> v2:172.21.15.181:3300/0 conn(0x7f90b8143050 0x7f90b8159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:13.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.709+0000 7f90be9ee700 1 -- 172.21.15.163:0/4019213030 >> v1:172.21.15.181:6789/0 conn(0x7f90b806b9e0 legacy=0x7f90b80b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:13.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.709+0000 7f90be9ee700 1 -- 172.21.15.163:0/4019213030 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b80695d0 con 0x7f90b80b3f20 2023-10-09T06:36:13.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.709+0000 7f90be9ee700 1 --2- 172.21.15.163:0/4019213030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90b80b3f20 0x7f90b806be50 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f90b806eec0 tx=0x7f90b0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37532e230ea7fb2a server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:13.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.710+0000 7f90bd1eb700 1 -- 172.21.15.163:0/4019213030 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b000e070 con 0x7f90b80b3f20 2023-10-09T06:36:13.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.710+0000 7f90bd1eb700 1 -- 172.21.15.163:0/4019213030 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b000ba20 con 0x7f90b80b3f20 2023-10-09T06:36:13.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.710+0000 7f90bd1eb700 1 -- 172.21.15.163:0/4019213030 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b00092e0 con 0x7f90b80b3f20 2023-10-09T06:36:13.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.710+0000 7f90bf1ef700 1 -- 172.21.15.163:0/4019213030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90b80b3f20 msgr2=0x7f90b806be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:13.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.710+0000 7f90bf1ef700 1 --2- 172.21.15.163:0/4019213030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90b80b3f20 0x7f90b806be50 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f90b806eec0 tx=0x7f90b0009840 comp rx=0 tx=0).stop 2023-10-09T06:36:13.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.710+0000 7f90bf1ef700 1 -- 172.21.15.163:0/4019213030 shutdown_connections 2023-10-09T06:36:13.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.710+0000 7f90bf1ef700 1 --2- 172.21.15.163:0/4019213030 >> v2:172.21.15.181:3300/0 conn(0x7f90b8143050 0x7f90b8159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:13.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.710+0000 7f90bf1ef700 1 --2- 172.21.15.163:0/4019213030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90b80b3f20 0x7f90b806be50 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:13.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.710+0000 7f90bf1ef700 1 -- 172.21.15.163:0/4019213030 >> 172.21.15.163:0/4019213030 conn(0x7f90b80ce430 msgr2=0x7f90b80ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:13.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.710+0000 7f90bf1ef700 1 -- 172.21.15.163:0/4019213030 shutdown_connections 2023-10-09T06:36:13.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.710+0000 7f90bf1ef700 1 -- 172.21.15.163:0/4019213030 wait complete. 2023-10-09T06:36:13.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.711+0000 7f90bf1ef700 1 Processor -- start 2023-10-09T06:36:13.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.711+0000 7f90bf1ef700 1 -- start start 2023-10-09T06:36:13.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.712+0000 7f90bf1ef700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90b806b9e0 0x7f90b806d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:13.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.712+0000 7f90bf1ef700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90b80b3f20 0x7f90b815bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:13.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.712+0000 7f90bf1ef700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90b8143050 0x7f90b8138980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:13.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.712+0000 7f90bf1ef700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b815c770 con 0x7f90b806b9e0 2023-10-09T06:36:13.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.712+0000 7f90bf1ef700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b815ca50 con 0x7f90b8143050 2023-10-09T06:36:13.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.712+0000 7f90bf1ef700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b815cd30 con 0x7f90b80b3f20 2023-10-09T06:36:13.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.712+0000 7f90be1ed700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90b806b9e0 0x7f90b806d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:13.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.712+0000 7f90be1ed700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90b806b9e0 0x7f90b806d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35596/0 (socket says 172.21.15.163:35596) 2023-10-09T06:36:13.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.712+0000 7f90be1ed700 1 -- 172.21.15.163:0/4230529518 learned_addr learned my addr 172.21.15.163:0/4230529518 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:13.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.712+0000 7f90be9ee700 1 --2- 172.21.15.163:0/4230529518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90b8143050 0x7f90b8138980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:13.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.712+0000 7f90bd9ec700 1 --2- 172.21.15.163:0/4230529518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90b80b3f20 0x7f90b815bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:13.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.712+0000 7f90be1ed700 1 -- 172.21.15.163:0/4230529518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90b80b3f20 msgr2=0x7f90b815bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:13.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.712+0000 7f90be1ed700 1 --2- 172.21.15.163:0/4230529518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90b80b3f20 0x7f90b815bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:13.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.712+0000 7f90be1ed700 1 -- 172.21.15.163:0/4230529518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90b8143050 msgr2=0x7f90b8138980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:13.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.712+0000 7f90be1ed700 1 --2- 172.21.15.163:0/4230529518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90b8143050 0x7f90b8138980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:13.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.712+0000 7f90be1ed700 1 -- 172.21.15.163:0/4230529518 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b0009ae0 con 0x7f90b806b9e0 2023-10-09T06:36:13.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.712+0000 7f90bd9ec700 1 --2- 172.21.15.163:0/4230529518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90b80b3f20 0x7f90b815bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:36:13.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.713+0000 7f90be1ed700 1 --2- 172.21.15.163:0/4230529518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90b806b9e0 0x7f90b806d1d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f90b400ec80 tx=0x7f90b4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:13.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.713+0000 7f90ae7fc700 1 -- 172.21.15.163:0/4230529518 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b400c9d0 con 0x7f90b806b9e0 2023-10-09T06:36:13.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.713+0000 7f90ae7fc700 1 -- 172.21.15.163:0/4230529518 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b400cb30 con 0x7f90b806b9e0 2023-10-09T06:36:13.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.713+0000 7f90bf1ef700 1 -- 172.21.15.163:0/4230529518 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b815d010 con 0x7f90b806b9e0 2023-10-09T06:36:13.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.713+0000 7f90ae7fc700 1 -- 172.21.15.163:0/4230529518 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b4013470 con 0x7f90b806b9e0 2023-10-09T06:36:13.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.713+0000 7f90bf1ef700 1 -- 172.21.15.163:0/4230529518 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b8158810 con 0x7f90b806b9e0 2023-10-09T06:36:13.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.714+0000 7f90bf1ef700 1 -- 172.21.15.163:0/4230529518 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90b804c3c0 con 0x7f90b806b9e0 2023-10-09T06:36:13.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.714+0000 7f90ae7fc700 1 -- 172.21.15.163:0/4230529518 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f90b4013860 con 0x7f90b806b9e0 2023-10-09T06:36:13.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.714+0000 7f90ae7fc700 1 --2- 172.21.15.163:0/4230529518 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f909c067200 0x7f909c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:13.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.714+0000 7f90ae7fc700 1 -- 172.21.15.163:0/4230529518 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f90b4084dc0 con 0x7f90b806b9e0 2023-10-09T06:36:13.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.715+0000 7f90bd9ec700 1 --2- 172.21.15.163:0/4230529518 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f909c067200 0x7f909c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:13.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.715+0000 7f90bd9ec700 1 --2- 172.21.15.163:0/4230529518 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f909c067200 0x7f909c0696b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f90b81577c0 tx=0x7f90a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:13.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.719+0000 7f90ae7fc700 1 -- 172.21.15.163:0/4230529518 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90b4051fe0 con 0x7f90b806b9e0 2023-10-09T06:36:13.762 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:36:13.764 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:36:13.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:13.927+0000 7f90bf1ef700 1 -- 172.21.15.163:0/4230529518 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f90b80adad0 con 0x7f90b806b9e0 2023-10-09T06:36:14.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.762+0000 7f90ae7fc700 1 -- 172.21.15.163:0/4230529518 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v85) v1 ==== 97+0+0 (secure 0 0 0) 0x7f90b4055600 con 0x7f90b806b9e0 2023-10-09T06:36:14.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.765+0000 7f90bf1ef700 1 -- 172.21.15.163:0/4230529518 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f909c067200 msgr2=0x7f909c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:14.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.765+0000 7f90bf1ef700 1 --2- 172.21.15.163:0/4230529518 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f909c067200 0x7f909c0696b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f90b81577c0 tx=0x7f90a8009040 comp rx=0 tx=0).stop 2023-10-09T06:36:14.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.765+0000 7f90bf1ef700 1 -- 172.21.15.163:0/4230529518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90b806b9e0 msgr2=0x7f90b806d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:14.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.765+0000 7f90bf1ef700 1 --2- 172.21.15.163:0/4230529518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90b806b9e0 0x7f90b806d1d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f90b400ec80 tx=0x7f90b4009120 comp rx=0 tx=0).stop 2023-10-09T06:36:14.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.765+0000 7f90bf1ef700 1 -- 172.21.15.163:0/4230529518 shutdown_connections 2023-10-09T06:36:14.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.765+0000 7f90bf1ef700 1 --2- 172.21.15.163:0/4230529518 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f909c067200 0x7f909c0696b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:14.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.765+0000 7f90bf1ef700 1 --2- 172.21.15.163:0/4230529518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90b806b9e0 0x7f90b806d1d0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:14.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.765+0000 7f90bf1ef700 1 --2- 172.21.15.163:0/4230529518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90b8143050 0x7f90b8138980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:14.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.765+0000 7f90bf1ef700 1 --2- 172.21.15.163:0/4230529518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90b80b3f20 0x7f90b815bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:14.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.765+0000 7f90bf1ef700 1 -- 172.21.15.163:0/4230529518 >> 172.21.15.163:0/4230529518 conn(0x7f90b80ce430 msgr2=0x7f90b8159c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:14.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.765+0000 7f90bf1ef700 1 -- 172.21.15.163:0/4230529518 shutdown_connections 2023-10-09T06:36:14.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.765+0000 7f90bf1ef700 1 -- 172.21.15.163:0/4230529518 wait complete. 2023-10-09T06:36:14.767 INFO:teuthology.orchestra.run.smithi163.stderr: removed all blocklist entries 2023-10-09T06:36:14.777 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-09T06:36:14.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.891+0000 7fa76aa1b700 1 Processor -- start 2023-10-09T06:36:14.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.891+0000 7fa76aa1b700 1 -- start start 2023-10-09T06:36:14.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.892+0000 7fa76aa1b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa764156690 0x7fa7640ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:14.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.892+0000 7fa76aa1b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7640af480 0x7fa7640694a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:14.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.892+0000 7fa76aa1b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa764157e30 con 0x7fa764156a60 2023-10-09T06:36:14.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.892+0000 7fa76aa1b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa76406aa00 con 0x7fa764156690 2023-10-09T06:36:14.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.892+0000 7fa76aa1b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7640699e0 con 0x7fa7640af480 2023-10-09T06:36:14.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.892+0000 7fa763fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa764156690 0x7fa7640ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:14.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.892+0000 7fa763fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa764156690 0x7fa7640ca9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35612/0 (socket says 172.21.15.163:35612) 2023-10-09T06:36:14.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.892+0000 7fa763fff700 1 -- 172.21.15.163:0/2629051663 learned_addr learned my addr 172.21.15.163:0/2629051663 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:14.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.893+0000 7fa7637fe700 1 --2- 172.21.15.163:0/2629051663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7640af480 0x7fa7640694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:14.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.893+0000 7fa763fff700 1 -- 172.21.15.163:0/2629051663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7640af480 msgr2=0x7fa7640694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:14.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.893+0000 7fa763fff700 1 --2- 172.21.15.163:0/2629051663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7640af480 0x7fa7640694a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:14.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.893+0000 7fa763fff700 1 -- 172.21.15.163:0/2629051663 >> v1:172.21.15.181:6789/0 conn(0x7fa764156a60 legacy=0x7fa7640c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:14.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.893+0000 7fa763fff700 1 -- 172.21.15.163:0/2629051663 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7641578b0 con 0x7fa764156690 2023-10-09T06:36:14.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.893+0000 7fa763fff700 1 --2- 172.21.15.163:0/2629051663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa764156690 0x7fa7640ca9d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa7540099b0 tx=0x7fa754009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb5b8aa200815de server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:14.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.893+0000 7fa762ffd700 1 -- 172.21.15.163:0/2629051663 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75400b860 con 0x7fa764156690 2023-10-09T06:36:14.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.893+0000 7fa762ffd700 1 -- 172.21.15.163:0/2629051663 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75400b9c0 con 0x7fa764156690 2023-10-09T06:36:14.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.893+0000 7fa762ffd700 1 -- 172.21.15.163:0/2629051663 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75400bd80 con 0x7fa764156690 2023-10-09T06:36:14.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.894+0000 7fa76aa1b700 1 -- 172.21.15.163:0/2629051663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa764156690 msgr2=0x7fa7640ca9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:14.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.894+0000 7fa76aa1b700 1 --2- 172.21.15.163:0/2629051663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa764156690 0x7fa7640ca9d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa7540099b0 tx=0x7fa754009250 comp rx=0 tx=0).stop 2023-10-09T06:36:14.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.894+0000 7fa76aa1b700 1 -- 172.21.15.163:0/2629051663 shutdown_connections 2023-10-09T06:36:14.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.894+0000 7fa76aa1b700 1 --2- 172.21.15.163:0/2629051663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa764156690 0x7fa7640ca9d0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:14.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.894+0000 7fa76aa1b700 1 --2- 172.21.15.163:0/2629051663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7640af480 0x7fa7640694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:14.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.894+0000 7fa76aa1b700 1 -- 172.21.15.163:0/2629051663 >> 172.21.15.163:0/2629051663 conn(0x7fa7640c9870 msgr2=0x7fa7640c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:14.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.894+0000 7fa76aa1b700 1 -- 172.21.15.163:0/2629051663 shutdown_connections 2023-10-09T06:36:14.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.894+0000 7fa76aa1b700 1 -- 172.21.15.163:0/2629051663 wait complete. 2023-10-09T06:36:14.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.896+0000 7fa76aa1b700 1 Processor -- start 2023-10-09T06:36:14.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.896+0000 7fa76aa1b700 1 -- start start 2023-10-09T06:36:14.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.897+0000 7fa76aa1b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7640af480 0x7fa7640c3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:14.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.897+0000 7fa76aa1b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa764156690 0x7fa76413e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:14.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.897+0000 7fa76aa1b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa764156a60 0x7fa7640c1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:14.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.897+0000 7fa76aa1b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7640c3a80 con 0x7fa7640af480 2023-10-09T06:36:14.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.897+0000 7fa76aa1b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7640c30b0 con 0x7fa764156a60 2023-10-09T06:36:14.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.897+0000 7fa76aa1b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7640c24a0 con 0x7fa764156690 2023-10-09T06:36:14.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.897+0000 7fa763fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7640af480 0x7fa7640c3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:14.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.897+0000 7fa763fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7640af480 0x7fa7640c3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35622/0 (socket says 172.21.15.163:35622) 2023-10-09T06:36:14.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.897+0000 7fa763fff700 1 -- 172.21.15.163:0/1238763866 learned_addr learned my addr 172.21.15.163:0/1238763866 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:14.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.897+0000 7fa7637fe700 1 --2- 172.21.15.163:0/1238763866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa764156690 0x7fa76413e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:14.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.897+0000 7fa768fb8700 1 --2- 172.21.15.163:0/1238763866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa764156a60 0x7fa7640c1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:14.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.897+0000 7fa763fff700 1 -- 172.21.15.163:0/1238763866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa764156690 msgr2=0x7fa76413e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:14.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.897+0000 7fa763fff700 1 --2- 172.21.15.163:0/1238763866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa764156690 0x7fa76413e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:14.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.897+0000 7fa763fff700 1 -- 172.21.15.163:0/1238763866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa764156a60 msgr2=0x7fa7640c1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:14.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.897+0000 7fa763fff700 1 --2- 172.21.15.163:0/1238763866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa764156a60 0x7fa7640c1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:14.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.897+0000 7fa763fff700 1 -- 172.21.15.163:0/1238763866 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7540094f0 con 0x7fa7640af480 2023-10-09T06:36:14.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.897+0000 7fa763fff700 1 --2- 172.21.15.163:0/1238763866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7640af480 0x7fa7640c3540 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa754006fd0 tx=0x7fa754017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:14.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.897+0000 7fa74bfff700 1 -- 172.21.15.163:0/1238763866 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75401f070 con 0x7fa7640af480 2023-10-09T06:36:14.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.897+0000 7fa74bfff700 1 -- 172.21.15.163:0/1238763866 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa754011d70 con 0x7fa7640af480 2023-10-09T06:36:14.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.897+0000 7fa76aa1b700 1 -- 172.21.15.163:0/1238763866 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7640c2780 con 0x7fa7640af480 2023-10-09T06:36:14.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.897+0000 7fa74bfff700 1 -- 172.21.15.163:0/1238763866 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75401a650 con 0x7fa7640af480 2023-10-09T06:36:14.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.897+0000 7fa76aa1b700 1 -- 172.21.15.163:0/1238763866 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7640c2d40 con 0x7fa7640af480 2023-10-09T06:36:14.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.897+0000 7fa76aa1b700 1 -- 172.21.15.163:0/1238763866 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa76404c3c0 con 0x7fa7640af480 2023-10-09T06:36:14.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.898+0000 7fa74bfff700 1 -- 172.21.15.163:0/1238763866 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fa75401aa00 con 0x7fa7640af480 2023-10-09T06:36:14.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.898+0000 7fa74bfff700 1 --2- 172.21.15.163:0/1238763866 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa75006b610 0x7fa75006dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:14.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.898+0000 7fa74bfff700 1 -- 172.21.15.163:0/1238763866 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fa754085a30 con 0x7fa7640af480 2023-10-09T06:36:14.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.899+0000 7fa7637fe700 1 --2- 172.21.15.163:0/1238763866 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa75006b610 0x7fa75006dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:14.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.899+0000 7fa7637fe700 1 --2- 172.21.15.163:0/1238763866 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa75006b610 0x7fa75006dac0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa764152870 tx=0x7fa758008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:14.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:14.903+0000 7fa74bfff700 1 -- 172.21.15.163:0/1238763866 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa754052d20 con 0x7fa7640af480 2023-10-09T06:36:15.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.162+0000 7fa76aa1b700 1 -- 172.21.15.163:0/1238763866 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa7640c6f70 con 0x7fa7640af480 2023-10-09T06:36:15.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.162+0000 7fa74bfff700 1 -- 172.21.15.163:0/1238763866 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa754028090 con 0x7fa7640af480 2023-10-09T06:36:15.163 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:15.163 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:36:15.163 INFO:teuthology.orchestra.run.smithi163.stdout: "auth_dump": [ 2023-10-09T06:36:15.164 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:15.164 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.a", 2023-10-09T06:36:15.164 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNltXtwDxAAlnrWRsfnZDLEmsJVowh3Mg==", 2023-10-09T06:36:15.164 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:15.164 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:36:15.164 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:36:15.164 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:36:15.164 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:15.164 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:15.164 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:15.164 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:15.165 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.b", 2023-10-09T06:36:15.165 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlTdGhGhAAXuldy0HrFd+44sSu/EV3qA==", 2023-10-09T06:36:15.165 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:15.165 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:36:15.165 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:36:15.165 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:36:15.165 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:15.165 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:15.165 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:15.165 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:15.166 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.c", 2023-10-09T06:36:15.166 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlGis7FhAAYG76TiOFX8QkTwjNX2+Ncw==", 2023-10-09T06:36:15.166 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:15.166 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:36:15.166 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:36:15.166 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:36:15.166 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:15.166 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:15.166 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:15.166 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:15.166 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.d", 2023-10-09T06:36:15.167 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlKhRuIRAAfhG5Gpc7klXJloYXepOhsg==", 2023-10-09T06:36:15.167 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:15.167 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:36:15.167 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:36:15.167 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:36:15.167 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:15.167 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:15.167 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:15.167 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:15.167 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.0", 2023-10-09T06:36:15.167 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDcnSNlrQeFOxAA+s6UitrTn5TK1N7kgySoXw==", 2023-10-09T06:36:15.167 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:15.168 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:36:15.168 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:36:15.168 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:15.168 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:15.168 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:15.168 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:15.168 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.1", 2023-10-09T06:36:15.168 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDfnSNlleccORAAnB+lNUFblivdoY44JfELxA==", 2023-10-09T06:36:15.168 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:15.168 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:36:15.168 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:36:15.168 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:15.169 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:15.169 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:15.169 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:15.169 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.2", 2023-10-09T06:36:15.169 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDinSNlFoB4NxAAT0+trRBij5rqgc+F0eWUfQ==", 2023-10-09T06:36:15.169 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:15.169 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:36:15.169 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:36:15.169 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:15.169 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:15.169 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:15.169 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:15.170 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.3", 2023-10-09T06:36:15.170 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDlnSNlRRQ6NhAABoBtrFjSvQujolq0Pxrd1Q==", 2023-10-09T06:36:15.170 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:15.170 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:36:15.170 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:36:15.170 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:15.170 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:15.170 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:15.170 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:15.170 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.4", 2023-10-09T06:36:15.170 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDrnSNl308uOxAA5OJyPLGQmVSJqha5d+nKKQ==", 2023-10-09T06:36:15.171 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:15.171 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:36:15.171 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:36:15.171 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:15.171 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:15.171 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:15.171 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:15.171 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.5", 2023-10-09T06:36:15.172 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDunSNlHnK+OBAAU7l4dCJ82SbGSzBEm8Axzw==", 2023-10-09T06:36:15.172 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:15.172 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:36:15.172 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:36:15.172 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:15.172 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:15.172 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:15.172 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:15.172 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.6", 2023-10-09T06:36:15.173 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDxnSNlwUSLOBAAN5N+F5ho+Co8pBh4Rz+lXQ==", 2023-10-09T06:36:15.173 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:15.173 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:36:15.173 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:36:15.173 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:15.173 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:15.173 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:15.173 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:15.173 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.7", 2023-10-09T06:36:15.173 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQD0nSNl2gxGORAAoE8OLGEifcEx/km8rfSV9g==", 2023-10-09T06:36:15.173 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:15.173 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:36:15.174 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:36:15.174 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:15.174 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:15.174 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:15.174 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:15.174 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.0", 2023-10-09T06:36:15.174 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlnxY5JRAApr6czcpMdQYYtXQ9yg2q6A==", 2023-10-09T06:36:15.174 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:15.174 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:36:15.175 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow r", 2023-10-09T06:36:15.175 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-09T06:36:15.175 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:15.175 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:15.175 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:15.175 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.1", 2023-10-09T06:36:15.175 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlk6ybLRAAuShqTDAtymGdxN8KBdZDDA==", 2023-10-09T06:36:15.175 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:15.175 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:36:15.175 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:36:15.175 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:36:15.175 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:36:15.176 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:15.176 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:15.176 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:15.176 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.2", 2023-10-09T06:36:15.176 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlYyOmKhAAQmpr+mGfJH4GUa28dtlJwA==", 2023-10-09T06:36:15.176 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:15.176 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:36:15.176 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:36:15.177 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:36:15.177 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:36:15.177 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:15.177 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:15.177 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:15.177 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.3", 2023-10-09T06:36:15.177 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlD2H1MhAA4zvaXY/Gd0kndzi+QSwp7w==", 2023-10-09T06:36:15.177 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:15.177 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:36:15.177 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:36:15.178 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:36:15.178 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:36:15.178 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:15.178 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:15.178 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:15.178 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.admin", 2023-10-09T06:36:15.179 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDYnSNlsHH4LxAA85xeFEP1CUwoblaePYk5zA==", 2023-10-09T06:36:15.179 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:15.179 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:36:15.179 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:36:15.179 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:36:15.179 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:15.179 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:15.179 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:15.179 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:15.179 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mgr.x", 2023-10-09T06:36:15.179 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNl2sloDBAA0FoWN9iovCl6783o3/btiA==", 2023-10-09T06:36:15.180 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:15.180 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:36:15.180 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile mgr", 2023-10-09T06:36:15.180 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:15.180 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:15.180 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:15.181 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:15.181 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mgr.y", 2023-10-09T06:36:15.181 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlcFWVCBAAR2dKPg01ciTiNpSX0JIKFg==", 2023-10-09T06:36:15.181 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:15.181 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:36:15.181 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile mgr", 2023-10-09T06:36:15.181 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:15.182 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:15.182 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:15.182 INFO:teuthology.orchestra.run.smithi163.stdout: ] 2023-10-09T06:36:15.182 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:36:15.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.164+0000 7fa76aa1b700 1 -- 172.21.15.163:0/1238763866 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa75006b610 msgr2=0x7fa75006dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:15.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.164+0000 7fa76aa1b700 1 --2- 172.21.15.163:0/1238763866 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa75006b610 0x7fa75006dac0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa764152870 tx=0x7fa758008040 comp rx=0 tx=0).stop 2023-10-09T06:36:15.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.165+0000 7fa76aa1b700 1 -- 172.21.15.163:0/1238763866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7640af480 msgr2=0x7fa7640c3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:15.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.165+0000 7fa76aa1b700 1 --2- 172.21.15.163:0/1238763866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7640af480 0x7fa7640c3540 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa754006fd0 tx=0x7fa754017000 comp rx=0 tx=0).stop 2023-10-09T06:36:15.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.165+0000 7fa76aa1b700 1 -- 172.21.15.163:0/1238763866 shutdown_connections 2023-10-09T06:36:15.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.165+0000 7fa76aa1b700 1 --2- 172.21.15.163:0/1238763866 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa75006b610 0x7fa75006dac0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.165+0000 7fa76aa1b700 1 --2- 172.21.15.163:0/1238763866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7640af480 0x7fa7640c3540 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.165+0000 7fa76aa1b700 1 --2- 172.21.15.163:0/1238763866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa764156a60 0x7fa7640c1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.165+0000 7fa76aa1b700 1 --2- 172.21.15.163:0/1238763866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa764156690 0x7fa76413e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.165+0000 7fa76aa1b700 1 -- 172.21.15.163:0/1238763866 >> 172.21.15.163:0/1238763866 conn(0x7fa7640c9870 msgr2=0x7fa764158320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:15.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.165+0000 7fa76aa1b700 1 -- 172.21.15.163:0/1238763866 shutdown_connections 2023-10-09T06:36:15.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.165+0000 7fa76aa1b700 1 -- 172.21.15.163:0/1238763866 wait complete. 2023-10-09T06:36:15.187 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:36:15.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.302+0000 7ff2626ef700 1 Processor -- start 2023-10-09T06:36:15.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.302+0000 7ff2626ef700 1 -- start start 2023-10-09T06:36:15.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.303+0000 7ff2626ef700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff25c159420 0x7ff25c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:15.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.303+0000 7ff2626ef700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff25c15d230 0x7ff25c15d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:15.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.303+0000 7ff2626ef700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff25c0b2980 con 0x7ff25c1597f0 2023-10-09T06:36:15.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.303+0000 7ff2626ef700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff25c15dbb0 con 0x7ff25c159420 2023-10-09T06:36:15.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.303+0000 7ff2626ef700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c0b11d0 con 0x7ff25c15d230 2023-10-09T06:36:15.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.303+0000 7ff2616ed700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff25c159420 0x7ff25c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:15.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.303+0000 7ff260eec700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff25c1597f0 0x7ff25c0b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:45494/0 (socket says 172.21.15.163:45494) 2023-10-09T06:36:15.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.303+0000 7ff260eec700 1 -- 172.21.15.163:0/2113643344 learned_addr learned my addr 172.21.15.163:0/2113643344 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:15.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.303+0000 7ff2616ed700 1 -- 172.21.15.163:0/2113643344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff25c15d230 msgr2=0x7ff25c15d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:15.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.303+0000 7ff261eee700 1 --2- 172.21.15.163:0/2113643344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff25c15d230 0x7ff25c15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:15.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.303+0000 7ff253fff700 1 -- 172.21.15.163:0/2113643344 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1512472010 0 0) 0x7ff25c0b2980 con 0x7ff25c1597f0 2023-10-09T06:36:15.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.303+0000 7ff2616ed700 1 --2- 172.21.15.163:0/2113643344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff25c15d230 0x7ff25c15d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.303+0000 7ff2616ed700 1 -- 172.21.15.163:0/2113643344 >> v1:172.21.15.163:6789/0 conn(0x7ff25c1597f0 legacy=0x7ff25c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:15.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.303+0000 7ff2616ed700 1 -- 172.21.15.163:0/2113643344 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25c0b14b0 con 0x7ff25c159420 2023-10-09T06:36:15.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.303+0000 7ff261eee700 1 --2- 172.21.15.163:0/2113643344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff25c15d230 0x7ff25c15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:36:15.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.303+0000 7ff2616ed700 1 --2- 172.21.15.163:0/2113643344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff25c159420 0x7ff25c0b5f40 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ff2580098e0 tx=0x7ff2580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efcc856e7969e58d server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:15.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.304+0000 7ff253fff700 1 -- 172.21.15.163:0/2113643344 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff258009e80 con 0x7ff25c159420 2023-10-09T06:36:15.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.304+0000 7ff253fff700 1 -- 172.21.15.163:0/2113643344 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25800b860 con 0x7ff25c159420 2023-10-09T06:36:15.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.304+0000 7ff253fff700 1 -- 172.21.15.163:0/2113643344 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25800bbe0 con 0x7ff25c159420 2023-10-09T06:36:15.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.304+0000 7ff2626ef700 1 -- 172.21.15.163:0/2113643344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff25c159420 msgr2=0x7ff25c0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:15.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.304+0000 7ff2626ef700 1 --2- 172.21.15.163:0/2113643344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff25c159420 0x7ff25c0b5f40 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ff2580098e0 tx=0x7ff2580091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.304+0000 7ff2626ef700 1 -- 172.21.15.163:0/2113643344 shutdown_connections 2023-10-09T06:36:15.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.304+0000 7ff2626ef700 1 --2- 172.21.15.163:0/2113643344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff25c15d230 0x7ff25c15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.304+0000 7ff2626ef700 1 --2- 172.21.15.163:0/2113643344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff25c159420 0x7ff25c0b5f40 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.304+0000 7ff2626ef700 1 -- 172.21.15.163:0/2113643344 >> 172.21.15.163:0/2113643344 conn(0x7ff25c0d0660 msgr2=0x7ff25c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:15.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.304+0000 7ff2626ef700 1 -- 172.21.15.163:0/2113643344 shutdown_connections 2023-10-09T06:36:15.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.304+0000 7ff2626ef700 1 -- 172.21.15.163:0/2113643344 wait complete. 2023-10-09T06:36:15.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.305+0000 7ff2626ef700 1 Processor -- start 2023-10-09T06:36:15.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.305+0000 7ff2626ef700 1 -- start start 2023-10-09T06:36:15.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.306+0000 7ff2626ef700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff25c159420 0x7ff25c0c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:15.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.306+0000 7ff2626ef700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff25c1597f0 0x7ff25c14a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:15.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.306+0000 7ff2626ef700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff25c15d230 0x7ff25c068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:15.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.306+0000 7ff2626ef700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c156920 con 0x7ff25c15d230 2023-10-09T06:36:15.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.306+0000 7ff2626ef700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c0c6ea0 con 0x7ff25c159420 2023-10-09T06:36:15.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.306+0000 7ff2626ef700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c0c6290 con 0x7ff25c1597f0 2023-10-09T06:36:15.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.306+0000 7ff261eee700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff25c15d230 0x7ff25c068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:15.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.306+0000 7ff261eee700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff25c15d230 0x7ff25c068390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35642/0 (socket says 172.21.15.163:35642) 2023-10-09T06:36:15.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.306+0000 7ff261eee700 1 -- 172.21.15.163:0/3862803256 learned_addr learned my addr 172.21.15.163:0/3862803256 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:15.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.307+0000 7ff260eec700 1 --2- 172.21.15.163:0/3862803256 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff25c1597f0 0x7ff25c14a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:15.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.307+0000 7ff2616ed700 1 --2- 172.21.15.163:0/3862803256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff25c159420 0x7ff25c0c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:15.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.307+0000 7ff261eee700 1 -- 172.21.15.163:0/3862803256 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff25c1597f0 msgr2=0x7ff25c14a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:15.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.307+0000 7ff261eee700 1 --2- 172.21.15.163:0/3862803256 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff25c1597f0 0x7ff25c14a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.307+0000 7ff261eee700 1 -- 172.21.15.163:0/3862803256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff25c159420 msgr2=0x7ff25c0c5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:15.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.307+0000 7ff261eee700 1 --2- 172.21.15.163:0/3862803256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff25c159420 0x7ff25c0c5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.307+0000 7ff261eee700 1 -- 172.21.15.163:0/3862803256 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff258009420 con 0x7ff25c15d230 2023-10-09T06:36:15.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.307+0000 7ff2616ed700 1 --2- 172.21.15.163:0/3862803256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff25c159420 0x7ff25c0c5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:15.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.307+0000 7ff261eee700 1 --2- 172.21.15.163:0/3862803256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff25c15d230 0x7ff25c068390 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff25400cef0 tx=0x7ff254007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:15.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.307+0000 7ff2517fa700 1 -- 172.21.15.163:0/3862803256 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25400add0 con 0x7ff25c15d230 2023-10-09T06:36:15.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.307+0000 7ff2517fa700 1 -- 172.21.15.163:0/3862803256 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff254016420 con 0x7ff25c15d230 2023-10-09T06:36:15.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.307+0000 7ff2517fa700 1 -- 172.21.15.163:0/3862803256 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2540167e0 con 0x7ff25c15d230 2023-10-09T06:36:15.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.307+0000 7ff2626ef700 1 -- 172.21.15.163:0/3862803256 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff25c0c6570 con 0x7ff25c15d230 2023-10-09T06:36:15.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.307+0000 7ff2626ef700 1 -- 172.21.15.163:0/3862803256 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff25c0c6b30 con 0x7ff25c15d230 2023-10-09T06:36:15.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.308+0000 7ff2626ef700 1 -- 172.21.15.163:0/3862803256 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff25c0c7050 con 0x7ff25c15d230 2023-10-09T06:36:15.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.308+0000 7ff2517fa700 1 -- 172.21.15.163:0/3862803256 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7ff254016bd0 con 0x7ff25c15d230 2023-10-09T06:36:15.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.309+0000 7ff2517fa700 1 --2- 172.21.15.163:0/3862803256 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff240067200 0x7ff2400696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:15.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.309+0000 7ff2517fa700 1 -- 172.21.15.163:0/3862803256 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff254012070 con 0x7ff25c15d230 2023-10-09T06:36:15.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.309+0000 7ff2616ed700 1 --2- 172.21.15.163:0/3862803256 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff240067200 0x7ff2400696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:15.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.309+0000 7ff2616ed700 1 --2- 172.21.15.163:0/3862803256 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff240067200 0x7ff2400696b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff2580091e0 tx=0x7ff258017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:15.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.313+0000 7ff2517fa700 1 -- 172.21.15.163:0/3862803256 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff25404e5a0 con 0x7ff25c15d230 2023-10-09T06:36:15.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.496+0000 7ff2626ef700 1 -- 172.21.15.163:0/3862803256 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff25c15dd50 con 0x7ff25c15d230 2023-10-09T06:36:15.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.496+0000 7ff2517fa700 1 -- 172.21.15.163:0/3862803256 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+12183 (secure 0 0 0) 0x7ff25c0c7050 con 0x7ff25c15d230 2023-10-09T06:36:15.496 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:15.496 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":85,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:36:14.752311+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:36:15.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.498+0000 7ff2626ef700 1 -- 172.21.15.163:0/3862803256 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff240067200 msgr2=0x7ff2400696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:15.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.498+0000 7ff2626ef700 1 --2- 172.21.15.163:0/3862803256 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff240067200 0x7ff2400696b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff2580091e0 tx=0x7ff258017000 comp rx=0 tx=0).stop 2023-10-09T06:36:15.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.498+0000 7ff2626ef700 1 -- 172.21.15.163:0/3862803256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff25c15d230 msgr2=0x7ff25c068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:15.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.498+0000 7ff2626ef700 1 --2- 172.21.15.163:0/3862803256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff25c15d230 0x7ff25c068390 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff25400cef0 tx=0x7ff254007860 comp rx=0 tx=0).stop 2023-10-09T06:36:15.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.498+0000 7ff2626ef700 1 -- 172.21.15.163:0/3862803256 shutdown_connections 2023-10-09T06:36:15.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.498+0000 7ff2626ef700 1 --2- 172.21.15.163:0/3862803256 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff240067200 0x7ff2400696b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.498+0000 7ff2626ef700 1 --2- 172.21.15.163:0/3862803256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff25c15d230 0x7ff25c068390 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.498+0000 7ff2626ef700 1 --2- 172.21.15.163:0/3862803256 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff25c159420 0x7ff25c0c5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.498+0000 7ff2626ef700 1 --2- 172.21.15.163:0/3862803256 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff25c1597f0 0x7ff25c14a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.498+0000 7ff2626ef700 1 -- 172.21.15.163:0/3862803256 >> 172.21.15.163:0/3862803256 conn(0x7ff25c0d0660 msgr2=0x7ff25c158280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:15.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.498+0000 7ff2626ef700 1 -- 172.21.15.163:0/3862803256 shutdown_connections 2023-10-09T06:36:15.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.498+0000 7ff2626ef700 1 -- 172.21.15.163:0/3862803256 wait complete. 2023-10-09T06:36:15.511 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-09T06:36:15.511 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:36:15.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.628+0000 7f67ad973700 1 Processor -- start 2023-10-09T06:36:15.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.628+0000 7f67ad973700 1 -- start start 2023-10-09T06:36:15.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.629+0000 7f67ad973700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f67a806bcd0 0x7f67a80caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:15.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.629+0000 7f67ad973700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f67a806c0a0 0x7f67a815a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:15.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.629+0000 7f67ad973700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67a815d7e0 con 0x7f67a815df50 2023-10-09T06:36:15.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.629+0000 7f67ad973700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a815dac0 con 0x7f67a806bcd0 2023-10-09T06:36:15.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.629+0000 7f67ad973700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a815aa30 con 0x7f67a806c0a0 2023-10-09T06:36:15.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.629+0000 7f67a6ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f67a806bcd0 0x7f67a80caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:15.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.629+0000 7f67a77fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f67a815df50 0x7f67a80c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:45502/0 (socket says 172.21.15.163:45502) 2023-10-09T06:36:15.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.629+0000 7f67a77fe700 1 -- 172.21.15.163:0/1279416700 learned_addr learned my addr 172.21.15.163:0/1279416700 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:15.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.630+0000 7f679e7fc700 1 --2- 172.21.15.163:0/1279416700 >> v2:172.21.15.181:3300/0 conn(0x7f67a806c0a0 0x7f67a815a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:15.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.630+0000 7f67a6ffd700 1 -- 172.21.15.163:0/1279416700 >> v2:172.21.15.181:3300/0 conn(0x7f67a806c0a0 msgr2=0x7f67a815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:15.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.630+0000 7f67a6ffd700 1 --2- 172.21.15.163:0/1279416700 >> v2:172.21.15.181:3300/0 conn(0x7f67a806c0a0 0x7f67a815a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.630+0000 7f67a6ffd700 1 -- 172.21.15.163:0/1279416700 >> v1:172.21.15.163:6789/0 conn(0x7f67a815df50 legacy=0x7f67a80c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:15.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.630+0000 7f67a6ffd700 1 -- 172.21.15.163:0/1279416700 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a815ad10 con 0x7f67a806bcd0 2023-10-09T06:36:15.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.630+0000 7f67a67fc700 1 -- 172.21.15.163:0/1279416700 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2258629304 0 0) 0x7f67a815d7e0 con 0x7f67a815df50 2023-10-09T06:36:15.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.630+0000 7f679e7fc700 1 --2- 172.21.15.163:0/1279416700 >> v2:172.21.15.181:3300/0 conn(0x7f67a806c0a0 0x7f67a815a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:15.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.630+0000 7f67a6ffd700 1 --2- 172.21.15.163:0/1279416700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67a806bcd0 0x7f67a80caa90 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f67a806b4e0 tx=0x7f67980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3e980d2fe74e84 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:15.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.630+0000 7f67a67fc700 1 -- 172.21.15.163:0/1279416700 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679800b6b0 con 0x7f67a806bcd0 2023-10-09T06:36:15.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.630+0000 7f67a67fc700 1 -- 172.21.15.163:0/1279416700 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679800b810 con 0x7f67a806bcd0 2023-10-09T06:36:15.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.630+0000 7f67a67fc700 1 -- 172.21.15.163:0/1279416700 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679800bbb0 con 0x7f67a806bcd0 2023-10-09T06:36:15.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.630+0000 7f67ad973700 1 -- 172.21.15.163:0/1279416700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67a806bcd0 msgr2=0x7f67a80caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:15.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.630+0000 7f67ad973700 1 --2- 172.21.15.163:0/1279416700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67a806bcd0 0x7f67a80caa90 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f67a806b4e0 tx=0x7f67980091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.631+0000 7f67ad973700 1 -- 172.21.15.163:0/1279416700 shutdown_connections 2023-10-09T06:36:15.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.631+0000 7f67ad973700 1 --2- 172.21.15.163:0/1279416700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67a806bcd0 0x7f67a80caa90 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.631+0000 7f67ad973700 1 --2- 172.21.15.163:0/1279416700 >> v2:172.21.15.181:3300/0 conn(0x7f67a806c0a0 0x7f67a815a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.631+0000 7f67ad973700 1 -- 172.21.15.163:0/1279416700 >> 172.21.15.163:0/1279416700 conn(0x7f67a80ce600 msgr2=0x7f67a80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:15.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.631+0000 7f67ad973700 1 -- 172.21.15.163:0/1279416700 shutdown_connections 2023-10-09T06:36:15.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.631+0000 7f67ad973700 1 -- 172.21.15.163:0/1279416700 wait complete. 2023-10-09T06:36:15.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.632+0000 7f67ad973700 1 Processor -- start 2023-10-09T06:36:15.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.632+0000 7f67ad973700 1 -- start start 2023-10-09T06:36:15.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.633+0000 7f67ad973700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67a806bcd0 0x7f67a81411f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:15.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.633+0000 7f67ad973700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67a806c0a0 0x7f67a8154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:15.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.633+0000 7f67ad973700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67a815df50 0x7f67a8154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:15.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.633+0000 7f67ad973700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a80b1ae0 con 0x7f67a806bcd0 2023-10-09T06:36:15.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.633+0000 7f67ad973700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a815a530 con 0x7f67a806c0a0 2023-10-09T06:36:15.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.633+0000 7f67ad973700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a80b14d0 con 0x7f67a815df50 2023-10-09T06:36:15.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.633+0000 7f67a6ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67a806bcd0 0x7f67a81411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:15.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.633+0000 7f67a6ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67a806bcd0 0x7f67a81411f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35664/0 (socket says 172.21.15.163:35664) 2023-10-09T06:36:15.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.633+0000 7f67a6ffd700 1 -- 172.21.15.163:0/1054685783 learned_addr learned my addr 172.21.15.163:0/1054685783 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:15.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.633+0000 7f679e7fc700 1 --2- 172.21.15.163:0/1054685783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67a806c0a0 0x7f67a8154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:15.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.633+0000 7f67a77fe700 1 --2- 172.21.15.163:0/1054685783 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67a815df50 0x7f67a8154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:15.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.633+0000 7f67a6ffd700 1 -- 172.21.15.163:0/1054685783 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67a815df50 msgr2=0x7f67a8154b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:15.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.633+0000 7f67a6ffd700 1 --2- 172.21.15.163:0/1054685783 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67a815df50 0x7f67a8154b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.633+0000 7f67a6ffd700 1 -- 172.21.15.163:0/1054685783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67a806c0a0 msgr2=0x7f67a8154610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:15.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.633+0000 7f67a6ffd700 1 --2- 172.21.15.163:0/1054685783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67a806c0a0 0x7f67a8154610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.633+0000 7f67a6ffd700 1 -- 172.21.15.163:0/1054685783 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6798009420 con 0x7f67a806bcd0 2023-10-09T06:36:15.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.633+0000 7f67a77fe700 1 --2- 172.21.15.163:0/1054685783 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67a815df50 0x7f67a8154b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:15.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.633+0000 7f67a6ffd700 1 --2- 172.21.15.163:0/1054685783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67a806bcd0 0x7f67a81411f0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f67980093f0 tx=0x7f6798017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:15.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.633+0000 7f679f7fe700 1 -- 172.21.15.163:0/1054685783 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679800b980 con 0x7f67a806bcd0 2023-10-09T06:36:15.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.633+0000 7f679f7fe700 1 -- 172.21.15.163:0/1054685783 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679800bb00 con 0x7f67a806bcd0 2023-10-09T06:36:15.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.633+0000 7f67ad973700 1 -- 172.21.15.163:0/1054685783 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a815a250 con 0x7f67a806bcd0 2023-10-09T06:36:15.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.633+0000 7f679f7fe700 1 -- 172.21.15.163:0/1054685783 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6798011670 con 0x7f67a806bcd0 2023-10-09T06:36:15.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.633+0000 7f67ad973700 1 -- 172.21.15.163:0/1054685783 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a8141730 con 0x7f67a806bcd0 2023-10-09T06:36:15.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.634+0000 7f67ad973700 1 -- 172.21.15.163:0/1054685783 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a804c3c0 con 0x7f67a806bcd0 2023-10-09T06:36:15.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.635+0000 7f679f7fe700 1 -- 172.21.15.163:0/1054685783 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f6798011aa0 con 0x7f67a806bcd0 2023-10-09T06:36:15.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.635+0000 7f679f7fe700 1 --2- 172.21.15.163:0/1054685783 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6788067200 0x7f67880696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:15.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.635+0000 7f679f7fe700 1 -- 172.21.15.163:0/1054685783 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6798085ce0 con 0x7f67a806bcd0 2023-10-09T06:36:15.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.635+0000 7f679e7fc700 1 --2- 172.21.15.163:0/1054685783 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6788067200 0x7f67880696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:15.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.636+0000 7f679e7fc700 1 --2- 172.21.15.163:0/1054685783 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6788067200 0x7f67880696b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f67a80b4520 tx=0x7f6790008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:15.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.639+0000 7f679f7fe700 1 -- 172.21.15.163:0/1054685783 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6798052f50 con 0x7f67a806bcd0 2023-10-09T06:36:15.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.823+0000 7f67ad973700 1 -- 172.21.15.163:0/1054685783 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f67a815aff0 con 0x7f67a806bcd0 2023-10-09T06:36:15.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.823+0000 7f679f7fe700 1 -- 172.21.15.163:0/1054685783 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6798015020 con 0x7f67a806bcd0 2023-10-09T06:36:15.823 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:36:15.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.826+0000 7f67ad973700 1 -- 172.21.15.163:0/1054685783 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6788067200 msgr2=0x7f67880696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:15.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.826+0000 7f67ad973700 1 --2- 172.21.15.163:0/1054685783 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6788067200 0x7f67880696b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f67a80b4520 tx=0x7f6790008040 comp rx=0 tx=0).stop 2023-10-09T06:36:15.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.826+0000 7f67ad973700 1 -- 172.21.15.163:0/1054685783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67a806bcd0 msgr2=0x7f67a81411f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:15.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.826+0000 7f67ad973700 1 --2- 172.21.15.163:0/1054685783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67a806bcd0 0x7f67a81411f0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f67980093f0 tx=0x7f6798017000 comp rx=0 tx=0).stop 2023-10-09T06:36:15.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.826+0000 7f67ad973700 1 -- 172.21.15.163:0/1054685783 shutdown_connections 2023-10-09T06:36:15.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.826+0000 7f67ad973700 1 --2- 172.21.15.163:0/1054685783 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6788067200 0x7f67880696b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.826+0000 7f67ad973700 1 --2- 172.21.15.163:0/1054685783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67a806bcd0 0x7f67a81411f0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.826+0000 7f67ad973700 1 --2- 172.21.15.163:0/1054685783 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67a815df50 0x7f67a8154b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.826+0000 7f67ad973700 1 --2- 172.21.15.163:0/1054685783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67a806c0a0 0x7f67a8154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.826+0000 7f67ad973700 1 -- 172.21.15.163:0/1054685783 >> 172.21.15.163:0/1054685783 conn(0x7f67a80ce600 msgr2=0x7f67a815c500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:15.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.826+0000 7f67ad973700 1 -- 172.21.15.163:0/1054685783 shutdown_connections 2023-10-09T06:36:15.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.826+0000 7f67ad973700 1 -- 172.21.15.163:0/1054685783 wait complete. 2023-10-09T06:36:15.838 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-09T06:36:15.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.952+0000 7f62acc2b700 1 Processor -- start 2023-10-09T06:36:15.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.952+0000 7f62acc2b700 1 -- start start 2023-10-09T06:36:15.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.953+0000 7f62acc2b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f62a81597f0 0x7f62a80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:15.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.953+0000 7f62acc2b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f62a815d230 0x7f62a815d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:15.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.953+0000 7f62acc2b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62a815db70 con 0x7f62a8159420 2023-10-09T06:36:15.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.953+0000 7f62acc2b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a80b11d0 con 0x7f62a815d230 2023-10-09T06:36:15.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.953+0000 7f62acc2b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a80b14b0 con 0x7f62a81597f0 2023-10-09T06:36:15.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.953+0000 7f62a7fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f62a815d230 0x7f62a815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:15.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.953+0000 7f62a77fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f62a8159420 0x7f62a80b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:45518/0 (socket says 172.21.15.163:45518) 2023-10-09T06:36:15.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.953+0000 7f62a77fe700 1 -- 172.21.15.163:0/238149095 learned_addr learned my addr 172.21.15.163:0/238149095 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:15.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.953+0000 7f62a6ffd700 1 --2- 172.21.15.163:0/238149095 >> v2:172.21.15.181:3300/0 conn(0x7f62a81597f0 0x7f62a80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:15.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.953+0000 7f62a7fff700 1 -- 172.21.15.163:0/238149095 >> v2:172.21.15.181:3300/0 conn(0x7f62a81597f0 msgr2=0x7f62a80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:15.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.954+0000 7f62a7fff700 1 --2- 172.21.15.163:0/238149095 >> v2:172.21.15.181:3300/0 conn(0x7f62a81597f0 0x7f62a80b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.954+0000 7f62a7fff700 1 -- 172.21.15.163:0/238149095 >> v1:172.21.15.163:6789/0 conn(0x7f62a8159420 legacy=0x7f62a80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:15.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.954+0000 7f62a67fc700 1 -- 172.21.15.163:0/238149095 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1429444909 0 0) 0x7f62a815db70 con 0x7f62a8159420 2023-10-09T06:36:15.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.954+0000 7f62a7fff700 1 -- 172.21.15.163:0/238149095 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a80b1790 con 0x7f62a815d230 2023-10-09T06:36:15.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.954+0000 7f62a6ffd700 1 --2- 172.21.15.163:0/238149095 >> v2:172.21.15.181:3300/0 conn(0x7f62a81597f0 0x7f62a80b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:36:15.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.954+0000 7f62a7fff700 1 --2- 172.21.15.163:0/238149095 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62a815d230 0x7f62a815d630 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f62a8089750 tx=0x7f62a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e856cb4993212034 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:15.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.954+0000 7f62a67fc700 1 -- 172.21.15.163:0/238149095 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a000b6b0 con 0x7f62a815d230 2023-10-09T06:36:15.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.954+0000 7f62a67fc700 1 -- 172.21.15.163:0/238149095 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a000b810 con 0x7f62a815d230 2023-10-09T06:36:15.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.954+0000 7f62a67fc700 1 -- 172.21.15.163:0/238149095 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a000bbb0 con 0x7f62a815d230 2023-10-09T06:36:15.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.954+0000 7f62acc2b700 1 -- 172.21.15.163:0/238149095 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62a815d230 msgr2=0x7f62a815d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:15.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.954+0000 7f62acc2b700 1 --2- 172.21.15.163:0/238149095 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62a815d230 0x7f62a815d630 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f62a8089750 tx=0x7f62a00091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.954+0000 7f62acc2b700 1 -- 172.21.15.163:0/238149095 shutdown_connections 2023-10-09T06:36:15.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.954+0000 7f62acc2b700 1 --2- 172.21.15.163:0/238149095 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62a815d230 0x7f62a815d630 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.954+0000 7f62acc2b700 1 --2- 172.21.15.163:0/238149095 >> v2:172.21.15.181:3300/0 conn(0x7f62a81597f0 0x7f62a80b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.954+0000 7f62acc2b700 1 -- 172.21.15.163:0/238149095 >> 172.21.15.163:0/238149095 conn(0x7f62a80d0660 msgr2=0x7f62a80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:15.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.954+0000 7f62acc2b700 1 -- 172.21.15.163:0/238149095 shutdown_connections 2023-10-09T06:36:15.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.954+0000 7f62acc2b700 1 -- 172.21.15.163:0/238149095 wait complete. 2023-10-09T06:36:15.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.956+0000 7f62acc2b700 1 Processor -- start 2023-10-09T06:36:15.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.956+0000 7f62acc2b700 1 -- start start 2023-10-09T06:36:15.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.956+0000 7f62acc2b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62a8159420 0x7f62a8153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:15.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.956+0000 7f62acc2b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a81597f0 0x7f62a80b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:15.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.956+0000 7f62acc2b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a815d230 0x7f62a814fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:15.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.956+0000 7f62acc2b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a8154d30 con 0x7f62a8159420 2023-10-09T06:36:15.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.956+0000 7f62acc2b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a8152030 con 0x7f62a81597f0 2023-10-09T06:36:15.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.956+0000 7f62acc2b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a8154250 con 0x7f62a815d230 2023-10-09T06:36:15.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.956+0000 7f62a77fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62a8159420 0x7f62a8153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:15.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.956+0000 7f62a77fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62a8159420 0x7f62a8153d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35686/0 (socket says 172.21.15.163:35686) 2023-10-09T06:36:15.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.956+0000 7f62a77fe700 1 -- 172.21.15.163:0/170630608 learned_addr learned my addr 172.21.15.163:0/170630608 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:15.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.957+0000 7f62a7fff700 1 --2- 172.21.15.163:0/170630608 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a815d230 0x7f62a814fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:15.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.957+0000 7f62a6ffd700 1 --2- 172.21.15.163:0/170630608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a81597f0 0x7f62a80b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:15.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.957+0000 7f62a77fe700 1 -- 172.21.15.163:0/170630608 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a815d230 msgr2=0x7f62a814fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:15.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.957+0000 7f62a77fe700 1 --2- 172.21.15.163:0/170630608 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a815d230 0x7f62a814fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.957+0000 7f62a77fe700 1 -- 172.21.15.163:0/170630608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a81597f0 msgr2=0x7f62a80b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:15.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.957+0000 7f62a77fe700 1 --2- 172.21.15.163:0/170630608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a81597f0 0x7f62a80b4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:15.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.957+0000 7f62a77fe700 1 -- 172.21.15.163:0/170630608 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a0009420 con 0x7f62a8159420 2023-10-09T06:36:15.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.957+0000 7f62a6ffd700 1 --2- 172.21.15.163:0/170630608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a81597f0 0x7f62a80b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:36:15.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.957+0000 7f62a77fe700 1 --2- 172.21.15.163:0/170630608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62a8159420 0x7f62a8153d10 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f629800cd80 tx=0x7f6298001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:15.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.957+0000 7f628f7fe700 1 -- 172.21.15.163:0/170630608 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629800aba0 con 0x7f62a8159420 2023-10-09T06:36:15.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.957+0000 7f628f7fe700 1 -- 172.21.15.163:0/170630608 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629800ad00 con 0x7f62a8159420 2023-10-09T06:36:15.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.957+0000 7f628f7fe700 1 -- 172.21.15.163:0/170630608 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6298007300 con 0x7f62a8159420 2023-10-09T06:36:15.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.957+0000 7f62acc2b700 1 -- 172.21.15.163:0/170630608 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a8151420 con 0x7f62a8159420 2023-10-09T06:36:15.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.957+0000 7f62acc2b700 1 -- 172.21.15.163:0/170630608 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a81519e0 con 0x7f62a8159420 2023-10-09T06:36:15.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.958+0000 7f62acc2b700 1 -- 172.21.15.163:0/170630608 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a804c3c0 con 0x7f62a8159420 2023-10-09T06:36:15.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.958+0000 7f628f7fe700 1 -- 172.21.15.163:0/170630608 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f62980076f0 con 0x7f62a8159420 2023-10-09T06:36:15.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.959+0000 7f628f7fe700 1 --2- 172.21.15.163:0/170630608 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6294067200 0x7f62940696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:15.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.959+0000 7f628f7fe700 1 -- 172.21.15.163:0/170630608 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6298084c30 con 0x7f62a8159420 2023-10-09T06:36:15.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.959+0000 7f62a6ffd700 1 --2- 172.21.15.163:0/170630608 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6294067200 0x7f62940696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:15.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.959+0000 7f62a6ffd700 1 --2- 172.21.15.163:0/170630608 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6294067200 0x7f62940696b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f62a814fd20 tx=0x7f629c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:15.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:15.963+0000 7f628f7fe700 1 -- 172.21.15.163:0/170630608 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6298051f20 con 0x7f62a8159420 2023-10-09T06:36:16.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.149+0000 7f62acc2b700 1 -- 172.21.15.163:0/170630608 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f62a80b1a70 con 0x7f62a8159420 2023-10-09T06:36:16.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.149+0000 7f628f7fe700 1 -- 172.21.15.163:0/170630608 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v85) v1 ==== 84+0+65 (secure 0 0 0) 0x7f6298055540 con 0x7f62a8159420 2023-10-09T06:36:16.149 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:16.149 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:36:16.149 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:16.150 INFO:teuthology.orchestra.run.smithi163.stdout: "poolnum": 1, 2023-10-09T06:36:16.150 INFO:teuthology.orchestra.run.smithi163.stdout: "poolname": ".mgr" 2023-10-09T06:36:16.150 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:16.150 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:36:16.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.151+0000 7f62acc2b700 1 -- 172.21.15.163:0/170630608 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6294067200 msgr2=0x7f62940696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:16.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.151+0000 7f62acc2b700 1 --2- 172.21.15.163:0/170630608 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6294067200 0x7f62940696b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f62a814fd20 tx=0x7f629c008040 comp rx=0 tx=0).stop 2023-10-09T06:36:16.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.151+0000 7f62acc2b700 1 -- 172.21.15.163:0/170630608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62a8159420 msgr2=0x7f62a8153d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:16.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.151+0000 7f62acc2b700 1 --2- 172.21.15.163:0/170630608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62a8159420 0x7f62a8153d10 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f629800cd80 tx=0x7f6298001ae0 comp rx=0 tx=0).stop 2023-10-09T06:36:16.152 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.151+0000 7f62acc2b700 1 -- 172.21.15.163:0/170630608 shutdown_connections 2023-10-09T06:36:16.152 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.151+0000 7f62acc2b700 1 --2- 172.21.15.163:0/170630608 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6294067200 0x7f62940696b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:16.152 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.151+0000 7f62acc2b700 1 --2- 172.21.15.163:0/170630608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62a8159420 0x7f62a8153d10 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:16.152 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.151+0000 7f62acc2b700 1 --2- 172.21.15.163:0/170630608 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a815d230 0x7f62a814fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:16.152 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.151+0000 7f62acc2b700 1 --2- 172.21.15.163:0/170630608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62a81597f0 0x7f62a80b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:16.152 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.151+0000 7f62acc2b700 1 -- 172.21.15.163:0/170630608 >> 172.21.15.163:0/170630608 conn(0x7f62a80d0660 msgr2=0x7f62a8157070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:16.152 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.151+0000 7f62acc2b700 1 -- 172.21.15.163:0/170630608 shutdown_connections 2023-10-09T06:36:16.152 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.151+0000 7f62acc2b700 1 -- 172.21.15.163:0/170630608 wait complete. 2023-10-09T06:36:16.163 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-09T06:36:16.164 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-09T06:36:16.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.282+0000 7efe013b8700 1 Processor -- start 2023-10-09T06:36:16.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.282+0000 7efe013b8700 1 -- start start 2023-10-09T06:36:16.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.283+0000 7efe013b8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdfc0b0790 0x7efdfc0cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:16.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.283+0000 7efe013b8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efdfc1591e0 0x7efdfc159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:16.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.283+0000 7efe013b8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdfc0cfa00 con 0x7efdfc0b03c0 2023-10-09T06:36:16.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.283+0000 7efe013b8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdfc0cb580 con 0x7efdfc1591e0 2023-10-09T06:36:16.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.283+0000 7efe013b8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc0cf160 con 0x7efdfc0b0790 2023-10-09T06:36:16.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.283+0000 7efe00bb7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efdfc1591e0 0x7efdfc159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:16.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.283+0000 7efdfb7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdfc0b0790 0x7efdfc0cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:16.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.284+0000 7efe00bb7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efdfc1591e0 0x7efdfc159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:33776/0 (socket says 172.21.15.163:33776) 2023-10-09T06:36:16.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.284+0000 7efe00bb7700 1 -- 172.21.15.163:0/544210510 learned_addr learned my addr 172.21.15.163:0/544210510 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:16.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.284+0000 7efdfb7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdfc0b0790 0x7efdfc0cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:35706/0 (socket says 172.21.15.163:35706) 2023-10-09T06:36:16.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.284+0000 7efdfbfff700 1 --1- 172.21.15.163:0/544210510 >> v1:172.21.15.181:6789/0 conn(0x7efdfc0b03c0 0x7efdfc0b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:48514/0 (socket says 172.21.15.163:48514) 2023-10-09T06:36:16.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.284+0000 7efdfb7fe700 1 -- 172.21.15.163:0/544210510 >> v2:172.21.15.181:3300/0 conn(0x7efdfc1591e0 msgr2=0x7efdfc159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:16.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.284+0000 7efdfb7fe700 1 --2- 172.21.15.163:0/544210510 >> v2:172.21.15.181:3300/0 conn(0x7efdfc1591e0 0x7efdfc159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:16.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.284+0000 7efdfb7fe700 1 -- 172.21.15.163:0/544210510 >> v1:172.21.15.181:6789/0 conn(0x7efdfc0b03c0 legacy=0x7efdfc0b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:16.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.284+0000 7efdfb7fe700 1 -- 172.21.15.163:0/544210510 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc0cf440 con 0x7efdfc0b0790 2023-10-09T06:36:16.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.284+0000 7efe00bb7700 1 --2- 172.21.15.163:0/544210510 >> v2:172.21.15.181:3300/0 conn(0x7efdfc1591e0 0x7efdfc159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:36:16.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.284+0000 7efdfb7fe700 1 --2- 172.21.15.163:0/544210510 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdfc0b0790 0x7efdfc0cea70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efdf00098e0 tx=0x7efdf00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46ff0b588888e081 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:16.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.284+0000 7efdfaffd700 1 -- 172.21.15.163:0/544210510 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf000b640 con 0x7efdfc0b0790 2023-10-09T06:36:16.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.284+0000 7efdfaffd700 1 -- 172.21.15.163:0/544210510 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf000b7a0 con 0x7efdfc0b0790 2023-10-09T06:36:16.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.284+0000 7efdfaffd700 1 -- 172.21.15.163:0/544210510 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf000bb40 con 0x7efdfc0b0790 2023-10-09T06:36:16.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.285+0000 7efe013b8700 1 -- 172.21.15.163:0/544210510 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdfc0b0790 msgr2=0x7efdfc0cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:16.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.285+0000 7efe013b8700 1 --2- 172.21.15.163:0/544210510 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdfc0b0790 0x7efdfc0cea70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efdf00098e0 tx=0x7efdf00091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:16.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.285+0000 7efe013b8700 1 -- 172.21.15.163:0/544210510 shutdown_connections 2023-10-09T06:36:16.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.285+0000 7efe013b8700 1 --2- 172.21.15.163:0/544210510 >> v2:172.21.15.181:3300/0 conn(0x7efdfc1591e0 0x7efdfc159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:16.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.285+0000 7efe013b8700 1 --2- 172.21.15.163:0/544210510 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdfc0b0790 0x7efdfc0cea70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:16.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.285+0000 7efe013b8700 1 -- 172.21.15.163:0/544210510 >> 172.21.15.163:0/544210510 conn(0x7efdfc063230 msgr2=0x7efdfc063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:16.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.285+0000 7efe013b8700 1 -- 172.21.15.163:0/544210510 shutdown_connections 2023-10-09T06:36:16.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.285+0000 7efe013b8700 1 -- 172.21.15.163:0/544210510 wait complete. 2023-10-09T06:36:16.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.286+0000 7efe013b8700 1 Processor -- start 2023-10-09T06:36:16.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.286+0000 7efe013b8700 1 -- start start 2023-10-09T06:36:16.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.287+0000 7efe013b8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdfc0b03c0 0x7efdfc15b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:16.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.287+0000 7efe013b8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdfc0b0790 0x7efdfc16d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:16.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.287+0000 7efe013b8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdfc1591e0 0x7efdfc0cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:16.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.287+0000 7efe013b8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc16e340 con 0x7efdfc0b0790 2023-10-09T06:36:16.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.287+0000 7efe013b8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc15bca0 con 0x7efdfc1591e0 2023-10-09T06:36:16.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.287+0000 7efe013b8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc16dd80 con 0x7efdfc0b03c0 2023-10-09T06:36:16.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.287+0000 7efdfb7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdfc0b0790 0x7efdfc16d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:16.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.287+0000 7efdfb7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdfc0b0790 0x7efdfc16d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35694/0 (socket says 172.21.15.163:35694) 2023-10-09T06:36:16.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.287+0000 7efdfb7fe700 1 -- 172.21.15.163:0/3115140760 learned_addr learned my addr 172.21.15.163:0/3115140760 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:16.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.287+0000 7efdfbfff700 1 --2- 172.21.15.163:0/3115140760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdfc0b03c0 0x7efdfc15b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:16.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.287+0000 7efe00bb7700 1 --2- 172.21.15.163:0/3115140760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdfc1591e0 0x7efdfc0cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:16.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.287+0000 7efdfb7fe700 1 -- 172.21.15.163:0/3115140760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdfc0b03c0 msgr2=0x7efdfc15b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:16.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.287+0000 7efdfb7fe700 1 --2- 172.21.15.163:0/3115140760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdfc0b03c0 0x7efdfc15b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:16.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.287+0000 7efdfb7fe700 1 -- 172.21.15.163:0/3115140760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdfc1591e0 msgr2=0x7efdfc0cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:16.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.287+0000 7efdfb7fe700 1 --2- 172.21.15.163:0/3115140760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdfc1591e0 0x7efdfc0cbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:16.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.287+0000 7efdfb7fe700 1 -- 172.21.15.163:0/3115140760 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf4009ae0 con 0x7efdfc0b0790 2023-10-09T06:36:16.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.287+0000 7efe00bb7700 1 --2- 172.21.15.163:0/3115140760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdfc1591e0 0x7efdfc0cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:36:16.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.287+0000 7efdfb7fe700 1 --2- 172.21.15.163:0/3115140760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdfc0b0790 0x7efdfc16d3a0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0cf2a0 tx=0x7efdf0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:16.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.288+0000 7efde3fff700 1 -- 172.21.15.163:0/3115140760 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf001bcd0 con 0x7efdfc0b0790 2023-10-09T06:36:16.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.288+0000 7efde3fff700 1 -- 172.21.15.163:0/3115140760 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf001be30 con 0x7efdfc0b0790 2023-10-09T06:36:16.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.288+0000 7efe013b8700 1 -- 172.21.15.163:0/3115140760 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf0009420 con 0x7efdfc0b0790 2023-10-09T06:36:16.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.288+0000 7efde3fff700 1 -- 172.21.15.163:0/3115140760 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf0011670 con 0x7efdfc0b0790 2023-10-09T06:36:16.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.288+0000 7efe013b8700 1 -- 172.21.15.163:0/3115140760 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdfc0c5000 con 0x7efdfc0b0790 2023-10-09T06:36:16.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.289+0000 7efe013b8700 1 -- 172.21.15.163:0/3115140760 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdfc0c4e60 con 0x7efdfc0b0790 2023-10-09T06:36:16.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.289+0000 7efde3fff700 1 -- 172.21.15.163:0/3115140760 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7efdf0011a20 con 0x7efdfc0b0790 2023-10-09T06:36:16.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.289+0000 7efde3fff700 1 --2- 172.21.15.163:0/3115140760 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7efde8067200 0x7efde80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:16.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.290+0000 7efde3fff700 1 -- 172.21.15.163:0/3115140760 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 4853+0+0 (secure 0 0 0) 0x7efdf0085b10 con 0x7efdfc0b0790 2023-10-09T06:36:16.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.290+0000 7efdfbfff700 1 --2- 172.21.15.163:0/3115140760 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7efde8067200 0x7efde80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:16.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.290+0000 7efdfbfff700 1 --2- 172.21.15.163:0/3115140760 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7efde8067200 0x7efde80696b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7efdec007fd0 tx=0x7efdec007d70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:16.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.294+0000 7efde3fff700 1 -- 172.21.15.163:0/3115140760 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efdfc0c4e60 con 0x7efdfc0b0790 2023-10-09T06:36:16.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.479+0000 7efe013b8700 1 -- 172.21.15.163:0/3115140760 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7efdfc0c4e60 con 0x7efdfc0b0790 2023-10-09T06:36:16.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.769+0000 7efde3fff700 1 -- 172.21.15.163:0/3115140760 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v86) v1 ==== 138+0+0 (secure 0 0 0) 0x7efdfc0c4e60 con 0x7efdfc0b0790 2023-10-09T06:36:16.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.773+0000 7efe013b8700 1 -- 172.21.15.163:0/3115140760 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7efde8067200 msgr2=0x7efde80696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:16.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.773+0000 7efe013b8700 1 --2- 172.21.15.163:0/3115140760 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7efde8067200 0x7efde80696b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7efdec007fd0 tx=0x7efdec007d70 comp rx=0 tx=0).stop 2023-10-09T06:36:16.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.773+0000 7efe013b8700 1 -- 172.21.15.163:0/3115140760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdfc0b0790 msgr2=0x7efdfc16d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:16.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.773+0000 7efe013b8700 1 --2- 172.21.15.163:0/3115140760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdfc0b0790 0x7efdfc16d3a0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0cf2a0 tx=0x7efdf0017000 comp rx=0 tx=0).stop 2023-10-09T06:36:16.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.774+0000 7efe013b8700 1 -- 172.21.15.163:0/3115140760 shutdown_connections 2023-10-09T06:36:16.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.774+0000 7efe013b8700 1 --2- 172.21.15.163:0/3115140760 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7efde8067200 0x7efde80696b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:16.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.774+0000 7efe013b8700 1 --2- 172.21.15.163:0/3115140760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdfc1591e0 0x7efdfc0cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:16.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.774+0000 7efe013b8700 1 --2- 172.21.15.163:0/3115140760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdfc0b03c0 0x7efdfc15b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:16.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.774+0000 7efe013b8700 1 --2- 172.21.15.163:0/3115140760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdfc0b0790 0x7efdfc16d3a0 secure :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0cf2a0 tx=0x7efdf0017000 comp rx=0 tx=0).stop 2023-10-09T06:36:16.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.774+0000 7efe013b8700 1 -- 172.21.15.163:0/3115140760 >> 172.21.15.163:0/3115140760 conn(0x7efdfc063230 msgr2=0x7efdfc1587d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:16.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.775+0000 7efe013b8700 1 -- 172.21.15.163:0/3115140760 shutdown_connections 2023-10-09T06:36:16.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.775+0000 7efe013b8700 1 -- 172.21.15.163:0/3115140760 wait complete. 2023-10-09T06:36:16.775 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_metadata' created 2023-10-09T06:36:16.787 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-09T06:36:16.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.909+0000 7f2695289700 1 Processor -- start 2023-10-09T06:36:16.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.909+0000 7f2695289700 1 -- start start 2023-10-09T06:36:16.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.910+0000 7f2695289700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26900c9700 0x7f26900caef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:16.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.910+0000 7f2695289700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26900adf00 con 0x7f26900cb430 2023-10-09T06:36:16.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.910+0000 7f2695289700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26900cda80 con 0x7f26900b0360 2023-10-09T06:36:16.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.910+0000 7f2695289700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26900cd890 con 0x7f26900c9700 2023-10-09T06:36:16.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.910+0000 7f2694a88700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f26900cb430 0x7f26900ae390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:45530/0 (socket says 172.21.15.163:45530) 2023-10-09T06:36:16.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.910+0000 7f2694a88700 1 -- 172.21.15.163:0/3072799501 learned_addr learned my addr 172.21.15.163:0/3072799501 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:16.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.910+0000 7f268f7fe700 1 --2- 172.21.15.163:0/3072799501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26900c9700 0x7f26900caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:16.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.910+0000 7f268effd700 1 -- 172.21.15.163:0/3072799501 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1969362349 0 0) 0x7f26900adf00 con 0x7f26900cb430 2023-10-09T06:36:16.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.910+0000 7f268effd700 1 -- 172.21.15.163:0/3072799501 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2678003570 con 0x7f26900cb430 2023-10-09T06:36:16.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.910+0000 7f268effd700 1 -- 172.21.15.163:0/3072799501 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3856605377 0 0) 0x7f2678003570 con 0x7f26900cb430 2023-10-09T06:36:16.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.911+0000 7f268effd700 1 -- 172.21.15.163:0/3072799501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26900c9700 msgr2=0x7f26900caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:16.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.911+0000 7f268effd700 1 --2- 172.21.15.163:0/3072799501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26900c9700 0x7f26900caef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:16.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.911+0000 7f268effd700 1 -- 172.21.15.163:0/3072799501 >> v1:172.21.15.181:6789/0 conn(0x7f26900b0360 legacy=0x7f26900cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:16.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.911+0000 7f268effd700 1 -- 172.21.15.163:0/3072799501 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26900cdd60 con 0x7f26900cb430 2023-10-09T06:36:16.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.911+0000 7f268effd700 1 -- 172.21.15.163:0/3072799501 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f26880017f0 con 0x7f26900cb430 2023-10-09T06:36:16.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.911+0000 7f268effd700 1 -- 172.21.15.163:0/3072799501 >> v1:172.21.15.163:6789/0 conn(0x7f26900cb430 legacy=0x7f26900ae390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:16.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.911+0000 7f268effd700 1 --2- 172.21.15.163:0/3072799501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26900c9700 0x7f2678003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:16.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.911+0000 7f268effd700 1 --2- 172.21.15.163:0/3072799501 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2678003fd0 0x7f2678004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:16.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.911+0000 7f268effd700 1 --2- 172.21.15.163:0/3072799501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2678004d50 0x7f26780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:16.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.911+0000 7f268effd700 1 -- 172.21.15.163:0/3072799501 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267800bb10 con 0x7f2678003fd0 2023-10-09T06:36:16.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.911+0000 7f268effd700 1 -- 172.21.15.163:0/3072799501 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267800bc80 con 0x7f2678004d50 2023-10-09T06:36:16.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.911+0000 7f268effd700 1 -- 172.21.15.163:0/3072799501 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f267800be40 con 0x7f26900c9700 2023-10-09T06:36:16.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.911+0000 7f268f7fe700 1 --2- 172.21.15.163:0/3072799501 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2678003fd0 0x7f2678004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:16.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.911+0000 7f268ffff700 1 --2- 172.21.15.163:0/3072799501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26900c9700 0x7f2678003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:16.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.911+0000 7f2694a88700 1 --2- 172.21.15.163:0/3072799501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2678004d50 0x7f26780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:16.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.911+0000 7f268f7fe700 1 -- 172.21.15.163:0/3072799501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26900c9700 msgr2=0x7f2678003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:16.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.911+0000 7f268f7fe700 1 --2- 172.21.15.163:0/3072799501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26900c9700 0x7f2678003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:16.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.911+0000 7f268f7fe700 1 -- 172.21.15.163:0/3072799501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2678004d50 msgr2=0x7f26780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:16.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.911+0000 7f268f7fe700 1 --2- 172.21.15.163:0/3072799501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2678004d50 0x7f26780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:16.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.911+0000 7f268f7fe700 1 -- 172.21.15.163:0/3072799501 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2678003570 con 0x7f2678003fd0 2023-10-09T06:36:16.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.911+0000 7f2694a88700 1 --2- 172.21.15.163:0/3072799501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2678004d50 0x7f26780051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:16.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.911+0000 7f268f7fe700 1 --2- 172.21.15.163:0/3072799501 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2678003fd0 0x7f2678004810 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2684009c00 tx=0x7f26840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b53998c2dbda5cc server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:16.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.911+0000 7f268effd700 1 -- 172.21.15.163:0/3072799501 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268400e710 con 0x7f2678003fd0 2023-10-09T06:36:16.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.911+0000 7f268effd700 1 -- 172.21.15.163:0/3072799501 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268400e870 con 0x7f2678003fd0 2023-10-09T06:36:16.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.911+0000 7f268effd700 1 -- 172.21.15.163:0/3072799501 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268400ec30 con 0x7f2678003fd0 2023-10-09T06:36:16.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.912+0000 7f2695289700 1 -- 172.21.15.163:0/3072799501 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2678003fd0 msgr2=0x7f2678004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:16.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.912+0000 7f2695289700 1 --2- 172.21.15.163:0/3072799501 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2678003fd0 0x7f2678004810 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2684009c00 tx=0x7f26840091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:16.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.912+0000 7f2694a88700 1 -- 172.21.15.163:0/3072799501 reap_dead start 2023-10-09T06:36:16.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.912+0000 7f2695289700 1 -- 172.21.15.163:0/3072799501 shutdown_connections 2023-10-09T06:36:16.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.912+0000 7f2695289700 1 -- 172.21.15.163:0/3072799501 >> 172.21.15.163:0/3072799501 conn(0x7f269006baa0 msgr2=0x7f269006bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:16.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.912+0000 7f2695289700 1 -- 172.21.15.163:0/3072799501 shutdown_connections 2023-10-09T06:36:16.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.912+0000 7f2695289700 1 -- 172.21.15.163:0/3072799501 wait complete. 2023-10-09T06:36:16.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.913+0000 7f2695289700 1 Processor -- start 2023-10-09T06:36:16.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.913+0000 7f2695289700 1 -- start start 2023-10-09T06:36:16.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.914+0000 7f2695289700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2690069ac0 0x7f26900c5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:16.919 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.914+0000 7f2695289700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26900c5c80 0x7f2690069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:16.919 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.914+0000 7f2695289700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f269006a3d0 0x7f269006a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:16.919 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.914+0000 7f2695289700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2690146c50 con 0x7f269006a3d0 2023-10-09T06:36:16.919 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.914+0000 7f2695289700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269006ad40 con 0x7f26900c5c80 2023-10-09T06:36:16.919 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.914+0000 7f2695289700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2690158370 con 0x7f2690069ac0 2023-10-09T06:36:16.919 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.914+0000 7f2694a88700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f269006a3d0 0x7f269006a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:16.919 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.914+0000 7f2694a88700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f269006a3d0 0x7f269006a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35704/0 (socket says 172.21.15.163:35704) 2023-10-09T06:36:16.919 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.914+0000 7f2694a88700 1 -- 172.21.15.163:0/940144953 learned_addr learned my addr 172.21.15.163:0/940144953 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:16.919 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.914+0000 7f268f7fe700 1 --2- 172.21.15.163:0/940144953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26900c5c80 0x7f2690069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:16.919 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.914+0000 7f268ffff700 1 --2- 172.21.15.163:0/940144953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2690069ac0 0x7f26900c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:16.919 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.914+0000 7f2694a88700 1 -- 172.21.15.163:0/940144953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2690069ac0 msgr2=0x7f26900c5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:16.919 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.914+0000 7f2694a88700 1 --2- 172.21.15.163:0/940144953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2690069ac0 0x7f26900c5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:16.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.914+0000 7f2694a88700 1 -- 172.21.15.163:0/940144953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26900c5c80 msgr2=0x7f2690069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:16.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.914+0000 7f2694a88700 1 --2- 172.21.15.163:0/940144953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26900c5c80 0x7f2690069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:16.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.914+0000 7f2694a88700 1 -- 172.21.15.163:0/940144953 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26901585c0 con 0x7f269006a3d0 2023-10-09T06:36:16.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.915+0000 7f2694a88700 1 --2- 172.21.15.163:0/940144953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f269006a3d0 0x7f269006a800 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f2688002250 tx=0x7f26880013c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:16.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.915+0000 7f2677fff700 1 -- 172.21.15.163:0/940144953 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268800edc0 con 0x7f269006a3d0 2023-10-09T06:36:16.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.915+0000 7f2677fff700 1 -- 172.21.15.163:0/940144953 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2688013420 con 0x7f269006a3d0 2023-10-09T06:36:16.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.915+0000 7f2677fff700 1 -- 172.21.15.163:0/940144953 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26880137e0 con 0x7f269006a3d0 2023-10-09T06:36:16.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.915+0000 7f2695289700 1 -- 172.21.15.163:0/940144953 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26901588a0 con 0x7f269006a3d0 2023-10-09T06:36:16.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.915+0000 7f2695289700 1 -- 172.21.15.163:0/940144953 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2690158e60 con 0x7f269006a3d0 2023-10-09T06:36:16.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.916+0000 7f2695289700 1 -- 172.21.15.163:0/940144953 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2690159050 con 0x7f269006a3d0 2023-10-09T06:36:16.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.916+0000 7f2677fff700 1 -- 172.21.15.163:0/940144953 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f2688013b50 con 0x7f269006a3d0 2023-10-09T06:36:16.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.916+0000 7f2677fff700 1 --2- 172.21.15.163:0/940144953 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f267c067280 0x7f267c069730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:16.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.917+0000 7f2677fff700 1 -- 172.21.15.163:0/940144953 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f2688085040 con 0x7f269006a3d0 2023-10-09T06:36:16.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.917+0000 7f268ffff700 1 --2- 172.21.15.163:0/940144953 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f267c067280 0x7f267c069730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:16.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.917+0000 7f268ffff700 1 --2- 172.21.15.163:0/940144953 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f267c067280 0x7f267c069730 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2680000f60 tx=0x7f2680009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:16.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:16.921+0000 7f2677fff700 1 -- 172.21.15.163:0/940144953 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2688052100 con 0x7f269006a3d0 2023-10-09T06:36:17.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.104+0000 7f2695289700 1 -- 172.21.15.163:0/940144953 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f269004c3c0 con 0x7f269006a3d0 2023-10-09T06:36:17.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.775+0000 7f2677fff700 1 -- 172.21.15.163:0/940144953 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v87) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2688055720 con 0x7f269006a3d0 2023-10-09T06:36:17.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.782+0000 7f2695289700 1 -- 172.21.15.163:0/940144953 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f267c067280 msgr2=0x7f267c069730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:17.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.782+0000 7f2695289700 1 --2- 172.21.15.163:0/940144953 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f267c067280 0x7f267c069730 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2680000f60 tx=0x7f2680009040 comp rx=0 tx=0).stop 2023-10-09T06:36:17.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.782+0000 7f2695289700 1 -- 172.21.15.163:0/940144953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f269006a3d0 msgr2=0x7f269006a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:17.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.782+0000 7f2695289700 1 --2- 172.21.15.163:0/940144953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f269006a3d0 0x7f269006a800 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f2688002250 tx=0x7f26880013c0 comp rx=0 tx=0).stop 2023-10-09T06:36:17.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.785+0000 7f2695289700 1 -- 172.21.15.163:0/940144953 shutdown_connections 2023-10-09T06:36:17.785 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.785+0000 7f2695289700 1 --2- 172.21.15.163:0/940144953 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f267c067280 0x7f267c069730 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:17.785 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.785+0000 7f2695289700 1 --2- 172.21.15.163:0/940144953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f269006a3d0 0x7f269006a800 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:17.785 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.785+0000 7f2695289700 1 --2- 172.21.15.163:0/940144953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2690069ac0 0x7f26900c5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:17.785 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.785+0000 7f2695289700 1 --2- 172.21.15.163:0/940144953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26900c5c80 0x7f2690069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:17.785 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.785+0000 7f2695289700 1 -- 172.21.15.163:0/940144953 >> 172.21.15.163:0/940144953 conn(0x7f269006baa0 msgr2=0x7f26900b3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:17.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.789+0000 7f2695289700 1 -- 172.21.15.163:0/940144953 shutdown_connections 2023-10-09T06:36:17.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.789+0000 7f2695289700 1 -- 172.21.15.163:0/940144953 wait complete. 2023-10-09T06:36:17.793 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_data' created 2023-10-09T06:36:17.806 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-09T06:36:17.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.924+0000 7f4991167700 1 Processor -- start 2023-10-09T06:36:17.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.924+0000 7f4991167700 1 -- start start 2023-10-09T06:36:17.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.925+0000 7f4991167700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f498c159890 0x7f498c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:17.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.925+0000 7f4991167700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f498c15a160 con 0x7f498c159c60 2023-10-09T06:36:17.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.925+0000 7f4991167700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f498c0d13a0 con 0x7f498c069560 2023-10-09T06:36:17.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.925+0000 7f4991167700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498c15e140 con 0x7f498c159890 2023-10-09T06:36:17.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.925+0000 7f498259c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f498c159c60 0x7f498c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:45540/0 (socket says 172.21.15.163:45540) 2023-10-09T06:36:17.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.925+0000 7f498259c700 1 -- 172.21.15.163:0/2599367673 learned_addr learned my addr 172.21.15.163:0/2599367673 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:17.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.925+0000 7f498a59c700 1 -- 172.21.15.163:0/2599367673 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3993741372 0 0) 0x7f498c15a160 con 0x7f498c159c60 2023-10-09T06:36:17.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.926+0000 7f498ad9d700 1 --2- 172.21.15.163:0/2599367673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f498c159890 0x7f498c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:17.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.926+0000 7f498a59c700 1 -- 172.21.15.163:0/2599367673 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4978003570 con 0x7f498c159c60 2023-10-09T06:36:17.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.926+0000 7f498a59c700 1 -- 172.21.15.163:0/2599367673 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1910628374 0 0) 0x7f4978003570 con 0x7f498c159c60 2023-10-09T06:36:17.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.926+0000 7f498a59c700 1 -- 172.21.15.163:0/2599367673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f498c159890 msgr2=0x7f498c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:17.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.926+0000 7f498a59c700 1 --2- 172.21.15.163:0/2599367673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f498c159890 0x7f498c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:17.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.926+0000 7f498a59c700 1 -- 172.21.15.163:0/2599367673 >> v1:172.21.15.181:6789/0 conn(0x7f498c069560 legacy=0x7f498c0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:17.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.926+0000 7f498a59c700 1 -- 172.21.15.163:0/2599367673 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498c15a3d0 con 0x7f498c159c60 2023-10-09T06:36:17.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.926+0000 7f498a59c700 1 -- 172.21.15.163:0/2599367673 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f4974001470 con 0x7f498c159c60 2023-10-09T06:36:17.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.926+0000 7f498ad9d700 1 --2- 172.21.15.163:0/2599367673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f498c159890 0x7f498c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:17.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.926+0000 7f498a59c700 1 -- 172.21.15.163:0/2599367673 >> v1:172.21.15.163:6789/0 conn(0x7f498c159c60 legacy=0x7f498c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:17.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.926+0000 7f498a59c700 1 --2- 172.21.15.163:0/2599367673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4978003a90 0x7f4978003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:17.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.926+0000 7f498a59c700 1 --2- 172.21.15.163:0/2599367673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4978004810 0x7f4978004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:17.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.926+0000 7f498a59c700 1 --2- 172.21.15.163:0/2599367673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f498c159890 0x7f4978005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:17.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.926+0000 7f498a59c700 1 -- 172.21.15.163:0/2599367673 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c15e140 con 0x7f4978004810 2023-10-09T06:36:17.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.926+0000 7f498a59c700 1 -- 172.21.15.163:0/2599367673 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497800bbf0 con 0x7f4978003a90 2023-10-09T06:36:17.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.926+0000 7f498a59c700 1 -- 172.21.15.163:0/2599367673 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f497800bdb0 con 0x7f498c159890 2023-10-09T06:36:17.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.926+0000 7f498259c700 1 --2- 172.21.15.163:0/2599367673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4978004810 0x7f4978004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:17.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.926+0000 7f498b59e700 1 --2- 172.21.15.163:0/2599367673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f498c159890 0x7f4978005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:17.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.926+0000 7f498ad9d700 1 --2- 172.21.15.163:0/2599367673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4978003a90 0x7f4978003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:17.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.926+0000 7f498259c700 1 -- 172.21.15.163:0/2599367673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f498c159890 msgr2=0x7f4978005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:17.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.926+0000 7f498259c700 1 --2- 172.21.15.163:0/2599367673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f498c159890 0x7f4978005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:17.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.926+0000 7f498259c700 1 -- 172.21.15.163:0/2599367673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4978003a90 msgr2=0x7f4978003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:17.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.926+0000 7f498259c700 1 --2- 172.21.15.163:0/2599367673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4978003a90 0x7f4978003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:17.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.926+0000 7f498259c700 1 -- 172.21.15.163:0/2599367673 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4978003570 con 0x7f4978004810 2023-10-09T06:36:17.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.926+0000 7f498259c700 1 --2- 172.21.15.163:0/2599367673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4978004810 0x7f4978004c80 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f497400c390 tx=0x7f4974001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ffde8c96d863247 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:17.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.927+0000 7f498a59c700 1 -- 172.21.15.163:0/2599367673 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497400cc60 con 0x7f4978004810 2023-10-09T06:36:17.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.927+0000 7f498a59c700 1 -- 172.21.15.163:0/2599367673 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497400cdc0 con 0x7f4978004810 2023-10-09T06:36:17.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.927+0000 7f498a59c700 1 -- 172.21.15.163:0/2599367673 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4974005b70 con 0x7f4978004810 2023-10-09T06:36:17.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.927+0000 7f4991167700 1 -- 172.21.15.163:0/2599367673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4978004810 msgr2=0x7f4978004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:17.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.927+0000 7f4991167700 1 --2- 172.21.15.163:0/2599367673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4978004810 0x7f4978004c80 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f497400c390 tx=0x7f4974001e50 comp rx=0 tx=0).stop 2023-10-09T06:36:17.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.927+0000 7f498b59e700 1 -- 172.21.15.163:0/2599367673 reap_dead start 2023-10-09T06:36:17.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.927+0000 7f4991167700 1 -- 172.21.15.163:0/2599367673 shutdown_connections 2023-10-09T06:36:17.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.927+0000 7f4991167700 1 -- 172.21.15.163:0/2599367673 >> 172.21.15.163:0/2599367673 conn(0x7f498c068e20 msgr2=0x7f498c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:17.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.927+0000 7f4991167700 1 -- 172.21.15.163:0/2599367673 shutdown_connections 2023-10-09T06:36:17.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.927+0000 7f4991167700 1 -- 172.21.15.163:0/2599367673 wait complete. 2023-10-09T06:36:17.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.928+0000 7f4991167700 1 Processor -- start 2023-10-09T06:36:17.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.928+0000 7f4991167700 1 -- start start 2023-10-09T06:36:17.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.929+0000 7f4991167700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4978004810 0x7f498c152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:17.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.929+0000 7f4991167700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f498c158750 0x7f498c1564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:17.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.929+0000 7f4991167700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f498c0b5740 0x7f498c0b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:17.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.929+0000 7f4991167700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c156e60 con 0x7f498c158750 2023-10-09T06:36:17.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.929+0000 7f4991167700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c157140 con 0x7f4978004810 2023-10-09T06:36:17.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.929+0000 7f4991167700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498c157420 con 0x7f498c0b5740 2023-10-09T06:36:17.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.929+0000 7f498259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f498c158750 0x7f498c1564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:17.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.929+0000 7f498259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f498c158750 0x7f498c1564f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:35708/0 (socket says 172.21.15.163:35708) 2023-10-09T06:36:17.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.929+0000 7f498259c700 1 -- 172.21.15.163:0/1035182178 learned_addr learned my addr 172.21.15.163:0/1035182178 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:17.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.929+0000 7f498ad9d700 1 --2- 172.21.15.163:0/1035182178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4978004810 0x7f498c152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:17.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.929+0000 7f498b59e700 1 --2- 172.21.15.163:0/1035182178 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f498c0b5740 0x7f498c0b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:17.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.929+0000 7f498259c700 1 -- 172.21.15.163:0/1035182178 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f498c0b5740 msgr2=0x7f498c0b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:17.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.929+0000 7f498259c700 1 --2- 172.21.15.163:0/1035182178 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f498c0b5740 0x7f498c0b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:17.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.929+0000 7f498259c700 1 -- 172.21.15.163:0/1035182178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4978004810 msgr2=0x7f498c152210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:17.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.929+0000 7f498259c700 1 --2- 172.21.15.163:0/1035182178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4978004810 0x7f498c152210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:17.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.929+0000 7f498259c700 1 -- 172.21.15.163:0/1035182178 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498c157700 con 0x7f498c158750 2023-10-09T06:36:17.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.930+0000 7f498259c700 1 --2- 172.21.15.163:0/1035182178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f498c158750 0x7f498c1564f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4974001e90 tx=0x7f4974015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:17.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.930+0000 7f49837fe700 1 -- 172.21.15.163:0/1035182178 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4974001580 con 0x7f498c158750 2023-10-09T06:36:17.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.930+0000 7f49837fe700 1 -- 172.21.15.163:0/1035182178 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49740072e0 con 0x7f498c158750 2023-10-09T06:36:17.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.930+0000 7f4991167700 1 -- 172.21.15.163:0/1035182178 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f498c1579e0 con 0x7f498c158750 2023-10-09T06:36:17.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.930+0000 7f49837fe700 1 -- 172.21.15.163:0/1035182178 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49740076c0 con 0x7f498c158750 2023-10-09T06:36:17.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.930+0000 7f4991167700 1 -- 172.21.15.163:0/1035182178 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f498c0b6120 con 0x7f498c158750 2023-10-09T06:36:17.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.931+0000 7f4991167700 1 -- 172.21.15.163:0/1035182178 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f498c04c3c0 con 0x7f498c158750 2023-10-09T06:36:17.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.931+0000 7f49837fe700 1 -- 172.21.15.163:0/1035182178 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f4974019070 con 0x7f498c158750 2023-10-09T06:36:17.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.932+0000 7f49837fe700 1 --2- 172.21.15.163:0/1035182178 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f496c067200 0x7f496c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:17.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.932+0000 7f49837fe700 1 -- 172.21.15.163:0/1035182178 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f49740864d0 con 0x7f498c158750 2023-10-09T06:36:17.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.932+0000 7f498ad9d700 1 --2- 172.21.15.163:0/1035182178 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f496c067200 0x7f496c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:17.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.932+0000 7f498ad9d700 1 --2- 172.21.15.163:0/1035182178 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f496c067200 0x7f496c0696b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f497c006fd0 tx=0x7f497c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:17.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:17.936+0000 7f49837fe700 1 -- 172.21.15.163:0/1035182178 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f49740534d0 con 0x7f498c158750 2023-10-09T06:36:18.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.162+0000 7f4991167700 1 -- 172.21.15.163:0/1035182178 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f498c0b0120 con 0x7f498c158750 2023-10-09T06:36:18.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.778+0000 7f49837fe700 1 -- 172.21.15.163:0/1035182178 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v53) v1 ==== 174+0+0 (secure 0 0 0) 0x7f49740220c0 con 0x7f498c158750 2023-10-09T06:36:18.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.781+0000 7f4991167700 1 -- 172.21.15.163:0/1035182178 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f496c067200 msgr2=0x7f496c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:18.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.781+0000 7f4991167700 1 --2- 172.21.15.163:0/1035182178 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f496c067200 0x7f496c0696b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f497c006fd0 tx=0x7f497c008040 comp rx=0 tx=0).stop 2023-10-09T06:36:18.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.781+0000 7f4991167700 1 -- 172.21.15.163:0/1035182178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f498c158750 msgr2=0x7f498c1564f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:18.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.781+0000 7f4991167700 1 --2- 172.21.15.163:0/1035182178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f498c158750 0x7f498c1564f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4974001e90 tx=0x7f4974015000 comp rx=0 tx=0).stop 2023-10-09T06:36:18.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.782+0000 7f4991167700 1 -- 172.21.15.163:0/1035182178 shutdown_connections 2023-10-09T06:36:18.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.782+0000 7f4991167700 1 --2- 172.21.15.163:0/1035182178 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f496c067200 0x7f496c0696b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:18.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.782+0000 7f4991167700 1 --2- 172.21.15.163:0/1035182178 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f498c0b5740 0x7f498c0b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:18.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.782+0000 7f4991167700 1 --2- 172.21.15.163:0/1035182178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4978004810 0x7f498c152210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:18.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.782+0000 7f4991167700 1 --2- 172.21.15.163:0/1035182178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f498c158750 0x7f498c1564f0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:18.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.782+0000 7f4991167700 1 -- 172.21.15.163:0/1035182178 >> 172.21.15.163:0/1035182178 conn(0x7f498c068e20 msgr2=0x7f498c15d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:18.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.782+0000 7f4991167700 1 -- 172.21.15.163:0/1035182178 shutdown_connections 2023-10-09T06:36:18.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.782+0000 7f4991167700 1 -- 172.21.15.163:0/1035182178 wait complete. 2023-10-09T06:36:18.783 INFO:teuthology.orchestra.run.smithi163.stderr:new fs with metadata pool 14 and data pool 15 2023-10-09T06:36:18.796 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:36:18.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.910+0000 7f53ba3d2700 1 Processor -- start 2023-10-09T06:36:18.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.910+0000 7f53ba3d2700 1 -- start start 2023-10-09T06:36:18.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.911+0000 7f53ba3d2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53b406c0a0 0x7f53b40caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:18.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.911+0000 7f53ba3d2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f53b415df50 0x7f53b415e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:18.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.911+0000 7f53ba3d2700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b415d7e0 con 0x7f53b406bcd0 2023-10-09T06:36:18.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.911+0000 7f53ba3d2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b415dac0 con 0x7f53b415df50 2023-10-09T06:36:18.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.911+0000 7f53ba3d2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b415aa30 con 0x7f53b406c0a0 2023-10-09T06:36:18.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.911+0000 7f53b3fff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f53b406bcd0 0x7f53b415a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33628/0 (socket says 172.21.15.163:33628) 2023-10-09T06:36:18.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.911+0000 7f53b3fff700 1 -- 172.21.15.163:0/4188346635 learned_addr learned my addr 172.21.15.163:0/4188346635 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:18.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.912+0000 7f53b2ffd700 1 -- 172.21.15.163:0/4188346635 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2899876695 0 0) 0x7f53b415d7e0 con 0x7f53b406bcd0 2023-10-09T06:36:18.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.912+0000 7f53b2ffd700 1 -- 172.21.15.163:0/4188346635 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f539c003570 con 0x7f53b406bcd0 2023-10-09T06:36:18.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.912+0000 7f53b37fe700 1 --2- 172.21.15.163:0/4188346635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53b406c0a0 0x7f53b40caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:18.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.912+0000 7f53b896f700 1 --2- 172.21.15.163:0/4188346635 >> v2:172.21.15.181:3300/0 conn(0x7f53b415df50 0x7f53b415e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:18.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.912+0000 7f53b2ffd700 1 -- 172.21.15.163:0/4188346635 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2959187093 0 0) 0x7f539c003570 con 0x7f53b406bcd0 2023-10-09T06:36:18.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.912+0000 7f53b2ffd700 1 -- 172.21.15.163:0/4188346635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53b406c0a0 msgr2=0x7f53b40caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:18.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.912+0000 7f53b2ffd700 1 --2- 172.21.15.163:0/4188346635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53b406c0a0 0x7f53b40caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:18.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.912+0000 7f53b2ffd700 1 -- 172.21.15.163:0/4188346635 >> v2:172.21.15.181:3300/0 conn(0x7f53b415df50 msgr2=0x7f53b415e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:18.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.912+0000 7f53b2ffd700 1 --2- 172.21.15.163:0/4188346635 >> v2:172.21.15.181:3300/0 conn(0x7f53b415df50 0x7f53b415e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:18.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.912+0000 7f53b2ffd700 1 -- 172.21.15.163:0/4188346635 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b415ad10 con 0x7f53b406bcd0 2023-10-09T06:36:18.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.912+0000 7f53b2ffd700 1 -- 172.21.15.163:0/4188346635 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f53a4001470 con 0x7f53b406bcd0 2023-10-09T06:36:18.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.912+0000 7f53b2ffd700 1 -- 172.21.15.163:0/4188346635 >> v1:172.21.15.163:6789/0 conn(0x7f53b406bcd0 legacy=0x7f53b415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:18.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.912+0000 7f53b2ffd700 1 --2- 172.21.15.163:0/4188346635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f539c003a60 0x7f539c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:18.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.912+0000 7f53b2ffd700 1 --2- 172.21.15.163:0/4188346635 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f539c004810 0x7f539c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:18.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.912+0000 7f53b2ffd700 1 --2- 172.21.15.163:0/4188346635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53b406c0a0 0x7f539c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:18.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.912+0000 7f53b2ffd700 1 -- 172.21.15.163:0/4188346635 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b415dac0 con 0x7f539c004810 2023-10-09T06:36:18.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.912+0000 7f53b2ffd700 1 -- 172.21.15.163:0/4188346635 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b415aa30 con 0x7f539c003a60 2023-10-09T06:36:18.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.912+0000 7f53b2ffd700 1 -- 172.21.15.163:0/4188346635 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f539c00bc20 con 0x7f53b406c0a0 2023-10-09T06:36:18.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.912+0000 7f53b896f700 1 --2- 172.21.15.163:0/4188346635 >> v2:172.21.15.181:3300/0 conn(0x7f53b415df50 0x7f53b415e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:18.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.912+0000 7f53b37fe700 1 --2- 172.21.15.163:0/4188346635 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f539c004810 0x7f539c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:18.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.912+0000 7f53b3fff700 1 --2- 172.21.15.163:0/4188346635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f539c003a60 0x7f539c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:18.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.912+0000 7f53b896f700 1 --2- 172.21.15.163:0/4188346635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53b406c0a0 0x7f539c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:18.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.912+0000 7f53b37fe700 1 -- 172.21.15.163:0/4188346635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53b406c0a0 msgr2=0x7f539c005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:18.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.912+0000 7f53b37fe700 1 --2- 172.21.15.163:0/4188346635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53b406c0a0 0x7f539c005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:18.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.912+0000 7f53b37fe700 1 -- 172.21.15.163:0/4188346635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f539c003a60 msgr2=0x7f539c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:18.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.912+0000 7f53b37fe700 1 --2- 172.21.15.163:0/4188346635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f539c003a60 0x7f539c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:18.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.912+0000 7f53b37fe700 1 -- 172.21.15.163:0/4188346635 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f539c003570 con 0x7f539c004810 2023-10-09T06:36:18.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.912+0000 7f53b37fe700 1 --2- 172.21.15.163:0/4188346635 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f539c004810 0x7f539c004c80 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f53a800bb90 tx=0x7f53a8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4e82d839a6fdd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:18.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.913+0000 7f53b2ffd700 1 -- 172.21.15.163:0/4188346635 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a800d5d0 con 0x7f539c004810 2023-10-09T06:36:18.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.913+0000 7f53b2ffd700 1 -- 172.21.15.163:0/4188346635 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a800d730 con 0x7f539c004810 2023-10-09T06:36:18.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.913+0000 7f53b2ffd700 1 -- 172.21.15.163:0/4188346635 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a800daf0 con 0x7f539c004810 2023-10-09T06:36:18.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.913+0000 7f53ba3d2700 1 -- 172.21.15.163:0/4188346635 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f539c004810 msgr2=0x7f539c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:18.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.913+0000 7f53ba3d2700 1 --2- 172.21.15.163:0/4188346635 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f539c004810 0x7f539c004c80 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f53a800bb90 tx=0x7f53a8008040 comp rx=0 tx=0).stop 2023-10-09T06:36:18.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.913+0000 7f53b896f700 1 -- 172.21.15.163:0/4188346635 reap_dead start 2023-10-09T06:36:18.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.913+0000 7f53ba3d2700 1 -- 172.21.15.163:0/4188346635 shutdown_connections 2023-10-09T06:36:18.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.913+0000 7f53ba3d2700 1 -- 172.21.15.163:0/4188346635 >> 172.21.15.163:0/4188346635 conn(0x7f53b40ce600 msgr2=0x7f53b40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:18.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.913+0000 7f53ba3d2700 1 -- 172.21.15.163:0/4188346635 shutdown_connections 2023-10-09T06:36:18.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.913+0000 7f53ba3d2700 1 -- 172.21.15.163:0/4188346635 wait complete. 2023-10-09T06:36:18.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.914+0000 7f53ba3d2700 1 Processor -- start 2023-10-09T06:36:18.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.915+0000 7f53ba3d2700 1 -- start start 2023-10-09T06:36:18.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.915+0000 7f53ba3d2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f539c004810 0x7f53b416a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:18.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.915+0000 7f53ba3d2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53b40c56b0 0x7f53b40c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:18.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.915+0000 7f53ba3d2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53b4157f30 0x7f53b41583a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:18.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.915+0000 7f53ba3d2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b416ae00 con 0x7f53b4157f30 2023-10-09T06:36:18.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.915+0000 7f53ba3d2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b40c7180 con 0x7f539c004810 2023-10-09T06:36:18.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.915+0000 7f53ba3d2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b40c6060 con 0x7f53b40c56b0 2023-10-09T06:36:18.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.915+0000 7f53b896f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53b4157f30 0x7f53b41583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:18.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.915+0000 7f53b896f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53b4157f30 0x7f53b41583a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:54806/0 (socket says 172.21.15.163:54806) 2023-10-09T06:36:18.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.915+0000 7f53b896f700 1 -- 172.21.15.163:0/1248196839 learned_addr learned my addr 172.21.15.163:0/1248196839 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:18.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.916+0000 7f53b37fe700 1 --2- 172.21.15.163:0/1248196839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53b40c56b0 0x7f53b40c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:18.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.916+0000 7f53b3fff700 1 --2- 172.21.15.163:0/1248196839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f539c004810 0x7f53b416a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:18.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.916+0000 7f53b896f700 1 -- 172.21.15.163:0/1248196839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53b40c56b0 msgr2=0x7f53b40c5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:18.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.916+0000 7f53b896f700 1 --2- 172.21.15.163:0/1248196839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53b40c56b0 0x7f53b40c5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:18.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.916+0000 7f53b896f700 1 -- 172.21.15.163:0/1248196839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f539c004810 msgr2=0x7f53b416a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:18.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.916+0000 7f53b896f700 1 --2- 172.21.15.163:0/1248196839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f539c004810 0x7f53b416a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:18.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.916+0000 7f53b896f700 1 -- 172.21.15.163:0/1248196839 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b40c76a0 con 0x7f53b4157f30 2023-10-09T06:36:18.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.916+0000 7f53b896f700 1 --2- 172.21.15.163:0/1248196839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53b4157f30 0x7f53b41583a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f53b4157db0 tx=0x7f53ac0097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:18.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.916+0000 7f539bfff700 1 -- 172.21.15.163:0/1248196839 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac01b070 con 0x7f53b4157f30 2023-10-09T06:36:18.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.916+0000 7f539bfff700 1 -- 172.21.15.163:0/1248196839 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53ac010c30 con 0x7f53b4157f30 2023-10-09T06:36:18.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.916+0000 7f53ba3d2700 1 -- 172.21.15.163:0/1248196839 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b4158c60 con 0x7f53b4157f30 2023-10-09T06:36:18.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.916+0000 7f539bfff700 1 -- 172.21.15.163:0/1248196839 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac017430 con 0x7f53b4157f30 2023-10-09T06:36:18.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.916+0000 7f53ba3d2700 1 -- 172.21.15.163:0/1248196839 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b4159220 con 0x7f53b4157f30 2023-10-09T06:36:18.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.917+0000 7f53ba3d2700 1 -- 172.21.15.163:0/1248196839 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53b404c3c0 con 0x7f53b4157f30 2023-10-09T06:36:18.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.917+0000 7f539bfff700 1 -- 172.21.15.163:0/1248196839 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f53ac003020 con 0x7f53b4157f30 2023-10-09T06:36:18.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.918+0000 7f539bfff700 1 --2- 172.21.15.163:0/1248196839 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f53a0067200 0x7f53a00696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:18.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.918+0000 7f539bfff700 1 -- 172.21.15.163:0/1248196839 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f53ac0853a0 con 0x7f53b4157f30 2023-10-09T06:36:18.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.918+0000 7f53b3fff700 1 --2- 172.21.15.163:0/1248196839 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f53a0067200 0x7f53a00696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:18.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.918+0000 7f53b3fff700 1 --2- 172.21.15.163:0/1248196839 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f53a0067200 0x7f53a00696b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f53a4007ed0 tx=0x7f53a4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:18.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:18.922+0000 7f539bfff700 1 -- 172.21.15.163:0/1248196839 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f53ac0522c0 con 0x7f53b4157f30 2023-10-09T06:36:19.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.103+0000 7f53ba3d2700 1 -- 172.21.15.163:0/1248196839 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53b40c7880 con 0x7f53b4157f30 2023-10-09T06:36:19.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.104+0000 7f539bfff700 1 -- 172.21.15.163:0/1248196839 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+15414 (secure 0 0 0) 0x7f53ac0558e0 con 0x7f53b4157f30 2023-10-09T06:36:19.104 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:19.104 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":88,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:36:18.163106+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:36:16.497125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-09T06:36:17.121589+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:36:19.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.106+0000 7f53ba3d2700 1 -- 172.21.15.163:0/1248196839 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f53a0067200 msgr2=0x7f53a00696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:19.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.106+0000 7f53ba3d2700 1 --2- 172.21.15.163:0/1248196839 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f53a0067200 0x7f53a00696b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f53a4007ed0 tx=0x7f53a4009040 comp rx=0 tx=0).stop 2023-10-09T06:36:19.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.106+0000 7f53ba3d2700 1 -- 172.21.15.163:0/1248196839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53b4157f30 msgr2=0x7f53b41583a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:19.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.106+0000 7f53ba3d2700 1 --2- 172.21.15.163:0/1248196839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53b4157f30 0x7f53b41583a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f53b4157db0 tx=0x7f53ac0097d0 comp rx=0 tx=0).stop 2023-10-09T06:36:19.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.106+0000 7f53ba3d2700 1 -- 172.21.15.163:0/1248196839 shutdown_connections 2023-10-09T06:36:19.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.106+0000 7f53ba3d2700 1 --2- 172.21.15.163:0/1248196839 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f53a0067200 0x7f53a00696b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:19.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.106+0000 7f53ba3d2700 1 --2- 172.21.15.163:0/1248196839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53b4157f30 0x7f53b41583a0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:19.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.106+0000 7f53ba3d2700 1 --2- 172.21.15.163:0/1248196839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f539c004810 0x7f53b416a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:19.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.106+0000 7f53ba3d2700 1 --2- 172.21.15.163:0/1248196839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53b40c56b0 0x7f53b40c5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:19.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.106+0000 7f53ba3d2700 1 -- 172.21.15.163:0/1248196839 >> 172.21.15.163:0/1248196839 conn(0x7f53b40ce600 msgr2=0x7f53b4157180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:19.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.106+0000 7f53ba3d2700 1 -- 172.21.15.163:0/1248196839 shutdown_connections 2023-10-09T06:36:19.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.106+0000 7f53ba3d2700 1 -- 172.21.15.163:0/1248196839 wait complete. 2023-10-09T06:36:19.119 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:36:19.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.233+0000 7f4261eb6700 1 Processor -- start 2023-10-09T06:36:19.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.233+0000 7f4261eb6700 1 -- start start 2023-10-09T06:36:19.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.234+0000 7f4261eb6700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f425c159890 0x7f425c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:19.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.234+0000 7f4261eb6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f425c159c60 0x7f425c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:19.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.234+0000 7f4261eb6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f425c15a160 con 0x7f425c15db10 2023-10-09T06:36:19.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.234+0000 7f4261eb6700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f425c0d13a0 con 0x7f425c159890 2023-10-09T06:36:19.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.234+0000 7f4261eb6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425c15e140 con 0x7f425c159c60 2023-10-09T06:36:19.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.234+0000 7f425b7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f425c159890 0x7f425c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:19.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.234+0000 7f425b7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f425c159890 0x7f425c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:54810/0 (socket says 172.21.15.163:54810) 2023-10-09T06:36:19.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.234+0000 7f425b7fe700 1 -- 172.21.15.163:0/573710532 learned_addr learned my addr 172.21.15.163:0/573710532 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:19.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.234+0000 7f425affd700 1 --2- 172.21.15.163:0/573710532 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f425c159c60 0x7f425c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:19.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.235+0000 7f425b7fe700 1 -- 172.21.15.163:0/573710532 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f425c159c60 msgr2=0x7f425c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:19.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.235+0000 7f425b7fe700 1 --2- 172.21.15.163:0/573710532 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f425c159c60 0x7f425c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:19.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.235+0000 7f425b7fe700 1 -- 172.21.15.163:0/573710532 >> v1:172.21.15.181:6789/0 conn(0x7f425c15db10 legacy=0x7f425c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:19.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.235+0000 7f425b7fe700 1 -- 172.21.15.163:0/573710532 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c15a3d0 con 0x7f425c159890 2023-10-09T06:36:19.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.235+0000 7f425b7fe700 1 --2- 172.21.15.163:0/573710532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f425c159890 0x7f425c15abd0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f424c0099b0 tx=0x7f424c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=760de8173dcadea2 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:19.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.235+0000 7f425a7fc700 1 -- 172.21.15.163:0/573710532 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c00b820 con 0x7f425c159890 2023-10-09T06:36:19.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.235+0000 7f425a7fc700 1 -- 172.21.15.163:0/573710532 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424c00b980 con 0x7f425c159890 2023-10-09T06:36:19.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.235+0000 7f425a7fc700 1 -- 172.21.15.163:0/573710532 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c00bd20 con 0x7f425c159890 2023-10-09T06:36:19.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.235+0000 7f4261eb6700 1 -- 172.21.15.163:0/573710532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f425c159890 msgr2=0x7f425c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:19.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.235+0000 7f4261eb6700 1 --2- 172.21.15.163:0/573710532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f425c159890 0x7f425c15abd0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f424c0099b0 tx=0x7f424c009250 comp rx=0 tx=0).stop 2023-10-09T06:36:19.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.235+0000 7f4261eb6700 1 -- 172.21.15.163:0/573710532 shutdown_connections 2023-10-09T06:36:19.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.235+0000 7f4261eb6700 1 --2- 172.21.15.163:0/573710532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f425c159890 0x7f425c15abd0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:19.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.235+0000 7f4261eb6700 1 --2- 172.21.15.163:0/573710532 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f425c159c60 0x7f425c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:19.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.235+0000 7f4261eb6700 1 -- 172.21.15.163:0/573710532 >> 172.21.15.163:0/573710532 conn(0x7f425c068e20 msgr2=0x7f425c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:19.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.235+0000 7f4261eb6700 1 -- 172.21.15.163:0/573710532 shutdown_connections 2023-10-09T06:36:19.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.235+0000 7f4261eb6700 1 -- 172.21.15.163:0/573710532 wait complete. 2023-10-09T06:36:19.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.237+0000 7f4261eb6700 1 Processor -- start 2023-10-09T06:36:19.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.237+0000 7f4261eb6700 1 -- start start 2023-10-09T06:36:19.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.237+0000 7f4261eb6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f425c159890 0x7f425c0cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:19.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.237+0000 7f4261eb6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f425c159c60 0x7f425c0c9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:19.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.237+0000 7f4261eb6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f425c15db10 0x7f425c0b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:19.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.237+0000 7f4261eb6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c0c7a40 con 0x7f425c159890 2023-10-09T06:36:19.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.237+0000 7f4261eb6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c1526c0 con 0x7f425c15db10 2023-10-09T06:36:19.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.237+0000 7f4261eb6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425c0c9e90 con 0x7f425c159c60 2023-10-09T06:36:19.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.238+0000 7f425b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f425c159890 0x7f425c0cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:19.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.238+0000 7f425b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f425c159890 0x7f425c0cfa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:54816/0 (socket says 172.21.15.163:54816) 2023-10-09T06:36:19.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.238+0000 7f425b7fe700 1 -- 172.21.15.163:0/4037770778 learned_addr learned my addr 172.21.15.163:0/4037770778 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:19.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.238+0000 7f425bfff700 1 --2- 172.21.15.163:0/4037770778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f425c15db10 0x7f425c0b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:19.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.238+0000 7f425b7fe700 1 -- 172.21.15.163:0/4037770778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f425c159c60 msgr2=0x7f425c0c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:19.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.238+0000 7f425affd700 1 --2- 172.21.15.163:0/4037770778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f425c159c60 0x7f425c0c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:19.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.238+0000 7f425b7fe700 1 --2- 172.21.15.163:0/4037770778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f425c159c60 0x7f425c0c9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:19.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.238+0000 7f425b7fe700 1 -- 172.21.15.163:0/4037770778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f425c15db10 msgr2=0x7f425c0b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:19.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.238+0000 7f425b7fe700 1 --2- 172.21.15.163:0/4037770778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f425c15db10 0x7f425c0b07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:19.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.238+0000 7f425b7fe700 1 -- 172.21.15.163:0/4037770778 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f424c0094f0 con 0x7f425c159890 2023-10-09T06:36:19.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.238+0000 7f425b7fe700 1 --2- 172.21.15.163:0/4037770778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f425c159890 0x7f425c0cfa60 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f424c009290 tx=0x7f424c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:19.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.238+0000 7f42437fe700 1 -- 172.21.15.163:0/4037770778 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c011d70 con 0x7f425c159890 2023-10-09T06:36:19.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.238+0000 7f42437fe700 1 -- 172.21.15.163:0/4037770778 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424c020070 con 0x7f425c159890 2023-10-09T06:36:19.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.238+0000 7f4261eb6700 1 -- 172.21.15.163:0/4037770778 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f425c0b3b00 con 0x7f425c159890 2023-10-09T06:36:19.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.238+0000 7f42437fe700 1 -- 172.21.15.163:0/4037770778 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c01b6a0 con 0x7f425c159890 2023-10-09T06:36:19.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.238+0000 7f4261eb6700 1 -- 172.21.15.163:0/4037770778 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f425c0b40c0 con 0x7f425c159890 2023-10-09T06:36:19.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.239+0000 7f4261eb6700 1 -- 172.21.15.163:0/4037770778 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f425c04c3c0 con 0x7f425c159890 2023-10-09T06:36:19.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.240+0000 7f42437fe700 1 -- 172.21.15.163:0/4037770778 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f424c022030 con 0x7f425c159890 2023-10-09T06:36:19.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.240+0000 7f42437fe700 1 --2- 172.21.15.163:0/4037770778 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4248067200 0x7f42480696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:19.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.240+0000 7f42437fe700 1 -- 172.21.15.163:0/4037770778 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f424c086a30 con 0x7f425c159890 2023-10-09T06:36:19.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.240+0000 7f425affd700 1 --2- 172.21.15.163:0/4037770778 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4248067200 0x7f42480696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:19.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.241+0000 7f425affd700 1 --2- 172.21.15.163:0/4037770778 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4248067200 0x7f42480696b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f425c16cf80 tx=0x7f4250008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:19.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.244+0000 7f42437fe700 1 -- 172.21.15.163:0/4037770778 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f424c053870 con 0x7f425c159890 2023-10-09T06:36:19.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.429+0000 7f4261eb6700 1 -- 172.21.15.163:0/4037770778 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f425c04c3c0 con 0x7f425c159890 2023-10-09T06:36:19.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.429+0000 7f42437fe700 1 -- 172.21.15.163:0/4037770778 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+15414 (secure 0 0 0) 0x7f424c056e90 con 0x7f425c159890 2023-10-09T06:36:19.430 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:19.430 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":88,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:36:18.163106+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:36:16.497125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-09T06:36:17.121589+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:36:19.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.431+0000 7f4261eb6700 1 -- 172.21.15.163:0/4037770778 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4248067200 msgr2=0x7f42480696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:19.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.431+0000 7f4261eb6700 1 --2- 172.21.15.163:0/4037770778 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4248067200 0x7f42480696b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f425c16cf80 tx=0x7f4250008040 comp rx=0 tx=0).stop 2023-10-09T06:36:19.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.431+0000 7f4261eb6700 1 -- 172.21.15.163:0/4037770778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f425c159890 msgr2=0x7f425c0cfa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:19.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.431+0000 7f4261eb6700 1 --2- 172.21.15.163:0/4037770778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f425c159890 0x7f425c0cfa60 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f424c009290 tx=0x7f424c016000 comp rx=0 tx=0).stop 2023-10-09T06:36:19.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.431+0000 7f4261eb6700 1 -- 172.21.15.163:0/4037770778 shutdown_connections 2023-10-09T06:36:19.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.431+0000 7f4261eb6700 1 --2- 172.21.15.163:0/4037770778 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4248067200 0x7f42480696b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:19.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.431+0000 7f4261eb6700 1 --2- 172.21.15.163:0/4037770778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f425c159890 0x7f425c0cfa60 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:19.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.431+0000 7f4261eb6700 1 --2- 172.21.15.163:0/4037770778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f425c15db10 0x7f425c0b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:19.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.431+0000 7f4261eb6700 1 --2- 172.21.15.163:0/4037770778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f425c159c60 0x7f425c0c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:19.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.431+0000 7f4261eb6700 1 -- 172.21.15.163:0/4037770778 >> 172.21.15.163:0/4037770778 conn(0x7f425c068e20 msgr2=0x7f425c0b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:19.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.432+0000 7f4261eb6700 1 -- 172.21.15.163:0/4037770778 shutdown_connections 2023-10-09T06:36:19.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.432+0000 7f4261eb6700 1 -- 172.21.15.163:0/4037770778 wait complete. 2023-10-09T06:36:19.444 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-09T06:36:19.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.563+0000 7f3e20110700 1 Processor -- start 2023-10-09T06:36:19.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.563+0000 7f3e20110700 1 -- start start 2023-10-09T06:36:19.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.564+0000 7f3e20110700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3e18159c10 0x7f3e1815b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:19.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.564+0000 7f3e20110700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e180b5ea0 0x7f3e18068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:19.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.564+0000 7f3e20110700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e1815c480 con 0x7f3e18068bb0 2023-10-09T06:36:19.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.564+0000 7f3e20110700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e180b2a40 con 0x7f3e18159c10 2023-10-09T06:36:19.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.564+0000 7f3e20110700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1815b5d0 con 0x7f3e180b5ea0 2023-10-09T06:36:19.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.564+0000 7f3e1deac700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3e18159c10 0x7f3e1815b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:19.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.564+0000 7f3e1deac700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3e18159c10 0x7f3e1815b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:54824/0 (socket says 172.21.15.163:54824) 2023-10-09T06:36:19.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.564+0000 7f3e1deac700 1 -- 172.21.15.163:0/2730185207 learned_addr learned my addr 172.21.15.163:0/2730185207 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:19.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.565+0000 7f3e1d6ab700 1 --2- 172.21.15.163:0/2730185207 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e180b5ea0 0x7f3e18068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:19.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.565+0000 7f3e1deac700 1 -- 172.21.15.163:0/2730185207 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e180b5ea0 msgr2=0x7f3e18068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:19.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.565+0000 7f3e1deac700 1 --2- 172.21.15.163:0/2730185207 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e180b5ea0 0x7f3e18068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:19.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.565+0000 7f3e1deac700 1 -- 172.21.15.163:0/2730185207 >> v1:172.21.15.181:6789/0 conn(0x7f3e18068bb0 legacy=0x7f3e1815c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:19.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.565+0000 7f3e1deac700 1 -- 172.21.15.163:0/2730185207 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e1815b870 con 0x7f3e18159c10 2023-10-09T06:36:19.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.565+0000 7f3e1deac700 1 --2- 172.21.15.163:0/2730185207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e18159c10 0x7f3e1815b090 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f3e140099b0 tx=0x7f3e14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e741e1ca8f99a362 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:19.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.565+0000 7f3e1ceaa700 1 -- 172.21.15.163:0/2730185207 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e14009ea0 con 0x7f3e18159c10 2023-10-09T06:36:19.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.565+0000 7f3e1ceaa700 1 -- 172.21.15.163:0/2730185207 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e1400b860 con 0x7f3e18159c10 2023-10-09T06:36:19.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.565+0000 7f3e1ceaa700 1 -- 172.21.15.163:0/2730185207 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1400bc00 con 0x7f3e18159c10 2023-10-09T06:36:19.569 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.566+0000 7f3e20110700 1 -- 172.21.15.163:0/2730185207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e18159c10 msgr2=0x7f3e1815b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:19.569 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.566+0000 7f3e20110700 1 --2- 172.21.15.163:0/2730185207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e18159c10 0x7f3e1815b090 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f3e140099b0 tx=0x7f3e14009250 comp rx=0 tx=0).stop 2023-10-09T06:36:19.569 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.566+0000 7f3e20110700 1 -- 172.21.15.163:0/2730185207 shutdown_connections 2023-10-09T06:36:19.569 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.566+0000 7f3e20110700 1 --2- 172.21.15.163:0/2730185207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e18159c10 0x7f3e1815b090 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:19.569 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.566+0000 7f3e20110700 1 --2- 172.21.15.163:0/2730185207 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e180b5ea0 0x7f3e18068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:19.569 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.566+0000 7f3e20110700 1 -- 172.21.15.163:0/2730185207 >> 172.21.15.163:0/2730185207 conn(0x7f3e18099a20 msgr2=0x7f3e180af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:19.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.566+0000 7f3e20110700 1 -- 172.21.15.163:0/2730185207 shutdown_connections 2023-10-09T06:36:19.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.566+0000 7f3e20110700 1 -- 172.21.15.163:0/2730185207 wait complete. 2023-10-09T06:36:19.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.567+0000 7f3e20110700 1 Processor -- start 2023-10-09T06:36:19.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.567+0000 7f3e20110700 1 -- start start 2023-10-09T06:36:19.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.567+0000 7f3e20110700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e18068bb0 0x7f3e18169b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:19.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.567+0000 7f3e20110700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e180b5ea0 0x7f3e1814e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:19.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.567+0000 7f3e20110700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e18159c10 0x7f3e1806a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:19.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.568+0000 7f3e20110700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e180b0370 con 0x7f3e18159c10 2023-10-09T06:36:19.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.568+0000 7f3e20110700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e180c56c0 con 0x7f3e18068bb0 2023-10-09T06:36:19.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.568+0000 7f3e20110700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e180c59a0 con 0x7f3e180b5ea0 2023-10-09T06:36:19.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.568+0000 7f3e1e6ad700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e18159c10 0x7f3e1806a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:19.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.568+0000 7f3e1e6ad700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e18159c10 0x7f3e1806a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:54836/0 (socket says 172.21.15.163:54836) 2023-10-09T06:36:19.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.568+0000 7f3e1e6ad700 1 -- 172.21.15.163:0/1022498234 learned_addr learned my addr 172.21.15.163:0/1022498234 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:19.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.568+0000 7f3e1deac700 1 --2- 172.21.15.163:0/1022498234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e18068bb0 0x7f3e18169b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:19.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.568+0000 7f3e1d6ab700 1 --2- 172.21.15.163:0/1022498234 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e180b5ea0 0x7f3e1814e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:19.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.568+0000 7f3e1e6ad700 1 -- 172.21.15.163:0/1022498234 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e180b5ea0 msgr2=0x7f3e1814e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:19.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.568+0000 7f3e1e6ad700 1 --2- 172.21.15.163:0/1022498234 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e180b5ea0 0x7f3e1814e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:19.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.568+0000 7f3e1e6ad700 1 -- 172.21.15.163:0/1022498234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e18068bb0 msgr2=0x7f3e18169b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:19.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.568+0000 7f3e1e6ad700 1 --2- 172.21.15.163:0/1022498234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e18068bb0 0x7f3e18169b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:19.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.568+0000 7f3e1e6ad700 1 -- 172.21.15.163:0/1022498234 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e140094f0 con 0x7f3e18159c10 2023-10-09T06:36:19.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.568+0000 7f3e1e6ad700 1 --2- 172.21.15.163:0/1022498234 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e18159c10 0x7f3e1806a070 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000c8e0 tx=0x7f3e10001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:19.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.568+0000 7f3e0dffb700 1 -- 172.21.15.163:0/1022498234 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e10012630 con 0x7f3e18159c10 2023-10-09T06:36:19.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.568+0000 7f3e0dffb700 1 -- 172.21.15.163:0/1022498234 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e10012790 con 0x7f3e18159c10 2023-10-09T06:36:19.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.568+0000 7f3e20110700 1 -- 172.21.15.163:0/1022498234 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e180c5c80 con 0x7f3e18159c10 2023-10-09T06:36:19.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.568+0000 7f3e0dffb700 1 -- 172.21.15.163:0/1022498234 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e10012b50 con 0x7f3e18159c10 2023-10-09T06:36:19.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.568+0000 7f3e20110700 1 -- 172.21.15.163:0/1022498234 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e1816a0d0 con 0x7f3e18159c10 2023-10-09T06:36:19.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.569+0000 7f3e20110700 1 -- 172.21.15.163:0/1022498234 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e180c61a0 con 0x7f3e18159c10 2023-10-09T06:36:19.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.570+0000 7f3e0dffb700 1 -- 172.21.15.163:0/1022498234 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f3e1001a3a0 con 0x7f3e18159c10 2023-10-09T06:36:19.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.570+0000 7f3e0dffb700 1 --2- 172.21.15.163:0/1022498234 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3dfc067200 0x7f3dfc0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:19.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.570+0000 7f3e0dffb700 1 -- 172.21.15.163:0/1022498234 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3e10084ab0 con 0x7f3e18159c10 2023-10-09T06:36:19.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.570+0000 7f3e1deac700 1 --2- 172.21.15.163:0/1022498234 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3dfc067200 0x7f3dfc0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:19.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.571+0000 7f3e1deac700 1 --2- 172.21.15.163:0/1022498234 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3dfc067200 0x7f3dfc0696b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3e140099b0 tx=0x7f3e14017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:19.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.574+0000 7f3e0dffb700 1 -- 172.21.15.163:0/1022498234 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3e100519d0 con 0x7f3e18159c10 2023-10-09T06:36:19.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:19.803+0000 7f3e20110700 1 -- 172.21.15.163:0/1022498234 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3e1804c3c0 con 0x7f3e18159c10 2023-10-09T06:36:20.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.794+0000 7f3e0dffb700 1 -- 172.21.15.163:0/1022498234 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v56) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3e10054ff0 con 0x7f3e18159c10 2023-10-09T06:36:20.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.796+0000 7f3e20110700 1 -- 172.21.15.163:0/1022498234 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3dfc067200 msgr2=0x7f3dfc0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:20.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.796+0000 7f3e20110700 1 --2- 172.21.15.163:0/1022498234 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3dfc067200 0x7f3dfc0696b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3e140099b0 tx=0x7f3e14017000 comp rx=0 tx=0).stop 2023-10-09T06:36:20.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.796+0000 7f3e20110700 1 -- 172.21.15.163:0/1022498234 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e18159c10 msgr2=0x7f3e1806a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:20.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.796+0000 7f3e20110700 1 --2- 172.21.15.163:0/1022498234 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e18159c10 0x7f3e1806a070 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000c8e0 tx=0x7f3e10001710 comp rx=0 tx=0).stop 2023-10-09T06:36:20.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.796+0000 7f3e20110700 1 -- 172.21.15.163:0/1022498234 shutdown_connections 2023-10-09T06:36:20.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.796+0000 7f3e20110700 1 --2- 172.21.15.163:0/1022498234 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3dfc067200 0x7f3dfc0696b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:20.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.796+0000 7f3e20110700 1 --2- 172.21.15.163:0/1022498234 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e18159c10 0x7f3e1806a070 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:20.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.796+0000 7f3e20110700 1 --2- 172.21.15.163:0/1022498234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e18068bb0 0x7f3e18169b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:20.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.796+0000 7f3e20110700 1 --2- 172.21.15.163:0/1022498234 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e180b5ea0 0x7f3e1814e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:20.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.796+0000 7f3e20110700 1 -- 172.21.15.163:0/1022498234 >> 172.21.15.163:0/1022498234 conn(0x7f3e18099a20 msgr2=0x7f3e18157f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:20.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.797+0000 7f3e20110700 1 -- 172.21.15.163:0/1022498234 shutdown_connections 2023-10-09T06:36:20.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.797+0000 7f3e20110700 1 -- 172.21.15.163:0/1022498234 wait complete. 2023-10-09T06:36:20.810 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-09T06:36:20.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.931+0000 7f06513be700 1 Processor -- start 2023-10-09T06:36:20.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.931+0000 7f06513be700 1 -- start start 2023-10-09T06:36:20.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.932+0000 7f06513be700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f064c068390 0x7f064c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:20.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.932+0000 7f06513be700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f064c0c2bd0 0x7f064c068760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:20.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.932+0000 7f06513be700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f064c15b110 con 0x7f064c0c8ac0 2023-10-09T06:36:20.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.932+0000 7f06513be700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f064c0c33b0 con 0x7f064c068390 2023-10-09T06:36:20.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.932+0000 7f06513be700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f064c0c8630 con 0x7f064c0c2bd0 2023-10-09T06:36:20.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.932+0000 7f064b7fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f064c0c8ac0 0x7f064c06e6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33642/0 (socket says 172.21.15.163:33642) 2023-10-09T06:36:20.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.932+0000 7f064b7fe700 1 -- 172.21.15.163:0/3815724916 learned_addr learned my addr 172.21.15.163:0/3815724916 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:20.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.932+0000 7f064affd700 1 --2- 172.21.15.163:0/3815724916 >> v2:172.21.15.181:3300/0 conn(0x7f064c068390 0x7f064c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:20.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.932+0000 7f064a7fc700 1 --2- 172.21.15.163:0/3815724916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f064c0c2bd0 0x7f064c068760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:20.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.932+0000 7f0649ffb700 1 -- 172.21.15.163:0/3815724916 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4008986708 0 0) 0x7f0644001490 con 0x7f064c0c8ac0 2023-10-09T06:36:20.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.933+0000 7f0649ffb700 1 -- 172.21.15.163:0/3815724916 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0634003570 con 0x7f064c0c8ac0 2023-10-09T06:36:20.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.933+0000 7f064affd700 1 -- 172.21.15.163:0/3815724916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f064c0c2bd0 msgr2=0x7f064c068760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:20.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.933+0000 7f064affd700 1 --2- 172.21.15.163:0/3815724916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f064c0c2bd0 0x7f064c068760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:20.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.933+0000 7f064affd700 1 -- 172.21.15.163:0/3815724916 >> v1:172.21.15.163:6789/0 conn(0x7f064c0c8ac0 legacy=0x7f064c06e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:20.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.933+0000 7f0649ffb700 1 -- 172.21.15.163:0/3815724916 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3860036450 0 0) 0x7f0634003570 con 0x7f064c0c8ac0 2023-10-09T06:36:20.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.933+0000 7f064affd700 1 -- 172.21.15.163:0/3815724916 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f064c0c7d40 con 0x7f064c068390 2023-10-09T06:36:20.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.933+0000 7f064affd700 1 --2- 172.21.15.163:0/3815724916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f064c068390 0x7f064c15abd0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f063c0098e0 tx=0x7f063c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1b62d69b7e9742f server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:20.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.933+0000 7f0649ffb700 1 -- 172.21.15.163:0/3815724916 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0644001980 con 0x7f064c068390 2023-10-09T06:36:20.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.933+0000 7f0649ffb700 1 -- 172.21.15.163:0/3815724916 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063c011420 con 0x7f064c068390 2023-10-09T06:36:20.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.933+0000 7f0649ffb700 1 -- 172.21.15.163:0/3815724916 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c0117c0 con 0x7f064c068390 2023-10-09T06:36:20.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.934+0000 7f06513be700 1 -- 172.21.15.163:0/3815724916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f064c068390 msgr2=0x7f064c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:20.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.934+0000 7f06513be700 1 --2- 172.21.15.163:0/3815724916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f064c068390 0x7f064c15abd0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f063c0098e0 tx=0x7f063c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:20.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.934+0000 7f06513be700 1 -- 172.21.15.163:0/3815724916 shutdown_connections 2023-10-09T06:36:20.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.934+0000 7f06513be700 1 --2- 172.21.15.163:0/3815724916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f064c068390 0x7f064c15abd0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:20.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.934+0000 7f06513be700 1 --2- 172.21.15.163:0/3815724916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f064c0c2bd0 0x7f064c068760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:20.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.934+0000 7f06513be700 1 -- 172.21.15.163:0/3815724916 >> 172.21.15.163:0/3815724916 conn(0x7f064c069f30 msgr2=0x7f064c06a960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:20.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.934+0000 7f06513be700 1 -- 172.21.15.163:0/3815724916 shutdown_connections 2023-10-09T06:36:20.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.934+0000 7f06513be700 1 -- 172.21.15.163:0/3815724916 wait complete. 2023-10-09T06:36:20.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.935+0000 7f06513be700 1 Processor -- start 2023-10-09T06:36:20.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.935+0000 7f06513be700 1 -- start start 2023-10-09T06:36:20.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.936+0000 7f06513be700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f064c068390 0x7f064c0ada40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:20.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.936+0000 7f06513be700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f064c0c2bd0 0x7f064c155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:20.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.936+0000 7f06513be700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f064c0c8ac0 0x7f064c158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:20.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.936+0000 7f06513be700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064c0acd50 con 0x7f064c068390 2023-10-09T06:36:20.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.936+0000 7f06513be700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064c0ad5b0 con 0x7f064c0c2bd0 2023-10-09T06:36:20.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.936+0000 7f06513be700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f064c16b1d0 con 0x7f064c0c8ac0 2023-10-09T06:36:20.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.936+0000 7f064affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f064c068390 0x7f064c0ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:20.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.936+0000 7f064affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f064c068390 0x7f064c0ada40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:54838/0 (socket says 172.21.15.163:54838) 2023-10-09T06:36:20.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.936+0000 7f064affd700 1 -- 172.21.15.163:0/139787976 learned_addr learned my addr 172.21.15.163:0/139787976 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:20.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.936+0000 7f064a7fc700 1 --2- 172.21.15.163:0/139787976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f064c0c2bd0 0x7f064c155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:20.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.936+0000 7f064b7fe700 1 --2- 172.21.15.163:0/139787976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f064c0c8ac0 0x7f064c158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:20.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.936+0000 7f064affd700 1 -- 172.21.15.163:0/139787976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f064c0c8ac0 msgr2=0x7f064c158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:20.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.936+0000 7f064affd700 1 --2- 172.21.15.163:0/139787976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f064c0c8ac0 0x7f064c158380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:20.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.936+0000 7f064affd700 1 -- 172.21.15.163:0/139787976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f064c0c2bd0 msgr2=0x7f064c155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:20.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.936+0000 7f064affd700 1 --2- 172.21.15.163:0/139787976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f064c0c2bd0 0x7f064c155d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:20.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.936+0000 7f064affd700 1 -- 172.21.15.163:0/139787976 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f063c009420 con 0x7f064c068390 2023-10-09T06:36:20.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.936+0000 7f064affd700 1 --2- 172.21.15.163:0/139787976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f064c068390 0x7f064c0ada40 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f063c0093f0 tx=0x7f063c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:20.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.936+0000 7f0632ffd700 1 -- 172.21.15.163:0/139787976 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c01cbe0 con 0x7f064c068390 2023-10-09T06:36:20.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.936+0000 7f0632ffd700 1 -- 172.21.15.163:0/139787976 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063c01cd40 con 0x7f064c068390 2023-10-09T06:36:20.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.936+0000 7f06513be700 1 -- 172.21.15.163:0/139787976 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f064c0acf60 con 0x7f064c068390 2023-10-09T06:36:20.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.936+0000 7f0632ffd700 1 -- 172.21.15.163:0/139787976 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c01a680 con 0x7f064c068390 2023-10-09T06:36:20.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.936+0000 7f06513be700 1 -- 172.21.15.163:0/139787976 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f064c156250 con 0x7f064c068390 2023-10-09T06:36:20.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.937+0000 7f06513be700 1 -- 172.21.15.163:0/139787976 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f064c0ad890 con 0x7f064c068390 2023-10-09T06:36:20.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.938+0000 7f0632ffd700 1 -- 172.21.15.163:0/139787976 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f063c01aa70 con 0x7f064c068390 2023-10-09T06:36:20.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.938+0000 7f0632ffd700 1 --2- 172.21.15.163:0/139787976 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0638067200 0x7f06380696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:20.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.938+0000 7f0632ffd700 1 -- 172.21.15.163:0/139787976 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f063c086680 con 0x7f064c068390 2023-10-09T06:36:20.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.938+0000 7f064a7fc700 1 --2- 172.21.15.163:0/139787976 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0638067200 0x7f06380696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:20.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.939+0000 7f064a7fc700 1 --2- 172.21.15.163:0/139787976 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0638067200 0x7f06380696b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f064c0ce3e0 tx=0x7f0640008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:20.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:20.942+0000 7f0632ffd700 1 -- 172.21.15.163:0/139787976 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f063c0535a0 con 0x7f064c068390 2023-10-09T06:36:21.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.167+0000 7f06513be700 1 -- 172.21.15.163:0/139787976 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f064c04c3c0 con 0x7f064c068390 2023-10-09T06:36:21.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.800+0000 7f0632ffd700 1 -- 172.21.15.163:0/139787976 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v57) v1 ==== 122+0+0 (secure 0 0 0) 0x7f063c056bc0 con 0x7f064c068390 2023-10-09T06:36:21.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.802+0000 7f06513be700 1 -- 172.21.15.163:0/139787976 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0638067200 msgr2=0x7f06380696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:21.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.802+0000 7f06513be700 1 --2- 172.21.15.163:0/139787976 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0638067200 0x7f06380696b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f064c0ce3e0 tx=0x7f0640008040 comp rx=0 tx=0).stop 2023-10-09T06:36:21.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.802+0000 7f06513be700 1 -- 172.21.15.163:0/139787976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f064c068390 msgr2=0x7f064c0ada40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:21.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.802+0000 7f06513be700 1 --2- 172.21.15.163:0/139787976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f064c068390 0x7f064c0ada40 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f063c0093f0 tx=0x7f063c016000 comp rx=0 tx=0).stop 2023-10-09T06:36:21.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.802+0000 7f06513be700 1 -- 172.21.15.163:0/139787976 shutdown_connections 2023-10-09T06:36:21.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.802+0000 7f06513be700 1 --2- 172.21.15.163:0/139787976 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0638067200 0x7f06380696b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:21.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.802+0000 7f06513be700 1 --2- 172.21.15.163:0/139787976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f064c068390 0x7f064c0ada40 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:21.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.802+0000 7f06513be700 1 --2- 172.21.15.163:0/139787976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f064c0c8ac0 0x7f064c158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:21.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.802+0000 7f06513be700 1 --2- 172.21.15.163:0/139787976 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f064c0c2bd0 0x7f064c155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:21.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.802+0000 7f06513be700 1 -- 172.21.15.163:0/139787976 >> 172.21.15.163:0/139787976 conn(0x7f064c069f30 msgr2=0x7f064c0b2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:21.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.802+0000 7f06513be700 1 -- 172.21.15.163:0/139787976 shutdown_connections 2023-10-09T06:36:21.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.802+0000 7f06513be700 1 -- 172.21.15.163:0/139787976 wait complete. 2023-10-09T06:36:21.814 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:36:21.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.929+0000 7fcb06120700 1 Processor -- start 2023-10-09T06:36:21.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.929+0000 7fcb06120700 1 -- start start 2023-10-09T06:36:21.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.930+0000 7fcb06120700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcb00159420 0x7fcb000b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:21.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.930+0000 7fcb06120700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcb001597f0 0x7fcb000b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:21.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.930+0000 7fcb06120700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb0015d230 0x7fcb0015d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:21.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.930+0000 7fcb06120700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb0015dbb0 con 0x7fcb001597f0 2023-10-09T06:36:21.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.930+0000 7fcb06120700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb000b11d0 con 0x7fcb00159420 2023-10-09T06:36:21.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.930+0000 7fcb06120700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb000b14b0 con 0x7fcb0015d230 2023-10-09T06:36:21.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.930+0000 7fcb0491d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcb001597f0 0x7fcb000b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:21.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.930+0000 7fcb0491d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcb001597f0 0x7fcb000b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:54850/0 (socket says 172.21.15.163:54850) 2023-10-09T06:36:21.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.930+0000 7fcb0491d700 1 -- 172.21.15.163:0/1917138740 learned_addr learned my addr 172.21.15.163:0/1917138740 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:21.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.930+0000 7fcb0591f700 1 --2- 172.21.15.163:0/1917138740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb0015d230 0x7fcb0015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:21.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.930+0000 7fcb0511e700 1 --2- 172.21.15.163:0/1917138740 >> v2:172.21.15.181:3300/0 conn(0x7fcb00159420 0x7fcb000b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:21.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.930+0000 7fcb0491d700 1 -- 172.21.15.163:0/1917138740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb0015d230 msgr2=0x7fcb0015d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:21.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.930+0000 7fcb0491d700 1 --2- 172.21.15.163:0/1917138740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb0015d230 0x7fcb0015d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:21.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.930+0000 7fcb0491d700 1 -- 172.21.15.163:0/1917138740 >> v2:172.21.15.181:3300/0 conn(0x7fcb00159420 msgr2=0x7fcb000b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:21.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.930+0000 7fcb0491d700 1 --2- 172.21.15.163:0/1917138740 >> v2:172.21.15.181:3300/0 conn(0x7fcb00159420 0x7fcb000b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:21.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.930+0000 7fcb0491d700 1 -- 172.21.15.163:0/1917138740 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb000b1790 con 0x7fcb001597f0 2023-10-09T06:36:21.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.930+0000 7fcb0591f700 1 --2- 172.21.15.163:0/1917138740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb0015d230 0x7fcb0015d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:21.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.931+0000 7fcb0491d700 1 --2- 172.21.15.163:0/1917138740 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb001597f0 0x7fcb000b2270 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fcb000b2f20 tx=0x7fcaf0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc8390f6717f70 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:21.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.931+0000 7fcaf7fff700 1 -- 172.21.15.163:0/1917138740 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf000b860 con 0x7fcb001597f0 2023-10-09T06:36:21.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.931+0000 7fcaf7fff700 1 -- 172.21.15.163:0/1917138740 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf000b9c0 con 0x7fcb001597f0 2023-10-09T06:36:21.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.931+0000 7fcaf7fff700 1 -- 172.21.15.163:0/1917138740 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf000bd60 con 0x7fcb001597f0 2023-10-09T06:36:21.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.931+0000 7fcb06120700 1 -- 172.21.15.163:0/1917138740 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb001597f0 msgr2=0x7fcb000b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:21.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.931+0000 7fcb06120700 1 --2- 172.21.15.163:0/1917138740 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb001597f0 0x7fcb000b2270 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fcb000b2f20 tx=0x7fcaf0009250 comp rx=0 tx=0).stop 2023-10-09T06:36:21.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.931+0000 7fcb06120700 1 -- 172.21.15.163:0/1917138740 shutdown_connections 2023-10-09T06:36:21.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.931+0000 7fcb06120700 1 --2- 172.21.15.163:0/1917138740 >> v2:172.21.15.181:3300/0 conn(0x7fcb00159420 0x7fcb000b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:21.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.931+0000 7fcb06120700 1 --2- 172.21.15.163:0/1917138740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb0015d230 0x7fcb0015d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:21.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.931+0000 7fcb06120700 1 --2- 172.21.15.163:0/1917138740 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb001597f0 0x7fcb000b2270 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:21.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.931+0000 7fcb06120700 1 -- 172.21.15.163:0/1917138740 >> 172.21.15.163:0/1917138740 conn(0x7fcb000d0660 msgr2=0x7fcb000d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:21.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.931+0000 7fcb06120700 1 -- 172.21.15.163:0/1917138740 shutdown_connections 2023-10-09T06:36:21.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.931+0000 7fcb06120700 1 -- 172.21.15.163:0/1917138740 wait complete. 2023-10-09T06:36:21.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.932+0000 7fcb06120700 1 Processor -- start 2023-10-09T06:36:21.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.932+0000 7fcb06120700 1 -- start start 2023-10-09T06:36:21.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.933+0000 7fcb06120700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb00159420 0x7fcb00155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:21.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.933+0000 7fcb06120700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb001597f0 0x7fcb000c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:21.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.933+0000 7fcb06120700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb0015d230 0x7fcb0016cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:21.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.933+0000 7fcb06120700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0015aca0 con 0x7fcb0015d230 2023-10-09T06:36:21.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.933+0000 7fcb06120700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb000c8470 con 0x7fcb001597f0 2023-10-09T06:36:21.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.933+0000 7fcb06120700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0015af10 con 0x7fcb00159420 2023-10-09T06:36:21.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.933+0000 7fcb0591f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb0015d230 0x7fcb0016cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:21.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.933+0000 7fcb0591f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb0015d230 0x7fcb0016cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:54852/0 (socket says 172.21.15.163:54852) 2023-10-09T06:36:21.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.933+0000 7fcb0591f700 1 -- 172.21.15.163:0/39989362 learned_addr learned my addr 172.21.15.163:0/39989362 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:21.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.934+0000 7fcb0511e700 1 --2- 172.21.15.163:0/39989362 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb00159420 0x7fcb00155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:21.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.934+0000 7fcb0491d700 1 --2- 172.21.15.163:0/39989362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb001597f0 0x7fcb000c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:21.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.934+0000 7fcb0591f700 1 -- 172.21.15.163:0/39989362 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb00159420 msgr2=0x7fcb00155330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:21.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.934+0000 7fcb0591f700 1 --2- 172.21.15.163:0/39989362 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb00159420 0x7fcb00155330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:21.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.934+0000 7fcb0591f700 1 -- 172.21.15.163:0/39989362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb001597f0 msgr2=0x7fcb000c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:21.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.934+0000 7fcb0591f700 1 --2- 172.21.15.163:0/39989362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb001597f0 0x7fcb000c7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:21.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.934+0000 7fcb0591f700 1 -- 172.21.15.163:0/39989362 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf00094f0 con 0x7fcb0015d230 2023-10-09T06:36:21.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.934+0000 7fcb0511e700 1 --2- 172.21.15.163:0/39989362 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb00159420 0x7fcb00155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:36:21.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.934+0000 7fcb0591f700 1 --2- 172.21.15.163:0/39989362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb0015d230 0x7fcb0016cf60 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fcaf800d860 tx=0x7fcaf8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:21.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.934+0000 7fcaf57fa700 1 -- 172.21.15.163:0/39989362 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf80125f0 con 0x7fcb0015d230 2023-10-09T06:36:21.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.934+0000 7fcaf57fa700 1 -- 172.21.15.163:0/39989362 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf8012750 con 0x7fcb0015d230 2023-10-09T06:36:21.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.934+0000 7fcb06120700 1 -- 172.21.15.163:0/39989362 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb0015b1f0 con 0x7fcb0015d230 2023-10-09T06:36:21.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.934+0000 7fcaf57fa700 1 -- 172.21.15.163:0/39989362 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8012ad0 con 0x7fcb0015d230 2023-10-09T06:36:21.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.934+0000 7fcb06120700 1 -- 172.21.15.163:0/39989362 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb000c8750 con 0x7fcb0015d230 2023-10-09T06:36:21.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.935+0000 7fcb06120700 1 -- 172.21.15.163:0/39989362 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb0015b380 con 0x7fcb0015d230 2023-10-09T06:36:21.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.935+0000 7fcaf57fa700 1 -- 172.21.15.163:0/39989362 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fcaf8014430 con 0x7fcb0015d230 2023-10-09T06:36:21.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.936+0000 7fcaf57fa700 1 --2- 172.21.15.163:0/39989362 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcae4067200 0x7fcae40696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:21.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.936+0000 7fcaf57fa700 1 -- 172.21.15.163:0/39989362 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcaf8084d10 con 0x7fcb0015d230 2023-10-09T06:36:21.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.936+0000 7fcb0511e700 1 --2- 172.21.15.163:0/39989362 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcae4067200 0x7fcae40696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:21.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.936+0000 7fcb0511e700 1 --2- 172.21.15.163:0/39989362 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcae4067200 0x7fcae40696b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcafc007540 tx=0x7fcafc00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:21.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:21.940+0000 7fcaf57fa700 1 -- 172.21.15.163:0/39989362 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcaf80556c0 con 0x7fcb0015d230 2023-10-09T06:36:22.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.162+0000 7fcb06120700 1 -- 172.21.15.163:0/39989362 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb0004c3c0 con 0x7fcb0015d230 2023-10-09T06:36:22.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.163+0000 7fcaf57fa700 1 -- 172.21.15.163:0/39989362 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fcaf8055250 con 0x7fcb0015d230 2023-10-09T06:36:22.163 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:22.163 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5501,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/862991395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":862991395},{"type":"v1","addr":"172.21.15.181:6836","nonce":862991395}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/699275566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":699275566},{"type":"v1","addr":"172.21.15.181:6837","nonce":699275566}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6504,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3664732208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3664732208},{"type":"v1","addr":"172.21.15.163:6834","nonce":3664732208}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:36:18.163421+0000","modified":"2023-10-09T06:36:21.793136+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6507},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6507":{"gid":6507,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/2687247401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2687247401},{"type":"v1","addr":"172.21.15.163:6835","nonce":2687247401}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-09T06:36:22.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.165+0000 7fcb06120700 1 -- 172.21.15.163:0/39989362 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcae4067200 msgr2=0x7fcae40696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:22.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.165+0000 7fcb06120700 1 --2- 172.21.15.163:0/39989362 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcae4067200 0x7fcae40696b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcafc007540 tx=0x7fcafc00b040 comp rx=0 tx=0).stop 2023-10-09T06:36:22.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.165+0000 7fcb06120700 1 -- 172.21.15.163:0/39989362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb0015d230 msgr2=0x7fcb0016cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:22.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.165+0000 7fcb06120700 1 --2- 172.21.15.163:0/39989362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb0015d230 0x7fcb0016cf60 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fcaf800d860 tx=0x7fcaf8008040 comp rx=0 tx=0).stop 2023-10-09T06:36:22.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.165+0000 7fcb06120700 1 -- 172.21.15.163:0/39989362 shutdown_connections 2023-10-09T06:36:22.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.165+0000 7fcb06120700 1 --2- 172.21.15.163:0/39989362 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcae4067200 0x7fcae40696b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:22.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.165+0000 7fcb06120700 1 --2- 172.21.15.163:0/39989362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb0015d230 0x7fcb0016cf60 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:22.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.165+0000 7fcb06120700 1 --2- 172.21.15.163:0/39989362 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb00159420 0x7fcb00155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:22.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.165+0000 7fcb06120700 1 --2- 172.21.15.163:0/39989362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb001597f0 0x7fcb000c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:22.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.165+0000 7fcb06120700 1 -- 172.21.15.163:0/39989362 >> 172.21.15.163:0/39989362 conn(0x7fcb000d0660 msgr2=0x7fcb000b56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:22.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.165+0000 7fcb06120700 1 -- 172.21.15.163:0/39989362 shutdown_connections 2023-10-09T06:36:22.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.165+0000 7fcb06120700 1 -- 172.21.15.163:0/39989362 wait complete. 2023-10-09T06:36:22.166 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 57 2023-10-09T06:36:22.177 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:36:22.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.292+0000 7f4825f6d700 1 Processor -- start 2023-10-09T06:36:22.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.292+0000 7f4825f6d700 1 -- start start 2023-10-09T06:36:22.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.293+0000 7f4825f6d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4820159890 0x7f482015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:22.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.293+0000 7f4825f6d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4820159c60 0x7f482015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:22.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.293+0000 7f4825f6d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f482015db10 0x7f4820069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:22.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.293+0000 7f4825f6d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f482015df50 con 0x7f4820159890 2023-10-09T06:36:22.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.293+0000 7f4825f6d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f482015a160 con 0x7f4820159c60 2023-10-09T06:36:22.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.293+0000 7f4825f6d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48200d13a0 con 0x7f482015db10 2023-10-09T06:36:22.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.293+0000 7f481f7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4820159890 0x7f482015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:22.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.293+0000 7f481f7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4820159890 0x7f482015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:54860/0 (socket says 172.21.15.163:54860) 2023-10-09T06:36:22.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.293+0000 7f481f7fe700 1 -- 172.21.15.163:0/1656582832 learned_addr learned my addr 172.21.15.163:0/1656582832 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:22.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.293+0000 7f481ffff700 1 --2- 172.21.15.163:0/1656582832 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f482015db10 0x7f4820069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:22.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.293+0000 7f481effd700 1 --2- 172.21.15.163:0/1656582832 >> v2:172.21.15.181:3300/0 conn(0x7f4820159c60 0x7f482015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:22.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.294+0000 7f481f7fe700 1 -- 172.21.15.163:0/1656582832 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f482015db10 msgr2=0x7f4820069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:22.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.294+0000 7f481f7fe700 1 --2- 172.21.15.163:0/1656582832 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f482015db10 0x7f4820069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:22.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.294+0000 7f481f7fe700 1 -- 172.21.15.163:0/1656582832 >> v2:172.21.15.181:3300/0 conn(0x7f4820159c60 msgr2=0x7f482015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:22.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.294+0000 7f481f7fe700 1 --2- 172.21.15.163:0/1656582832 >> v2:172.21.15.181:3300/0 conn(0x7f4820159c60 0x7f482015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:22.303 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.294+0000 7f481f7fe700 1 -- 172.21.15.163:0/1656582832 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482015e140 con 0x7f4820159890 2023-10-09T06:36:22.303 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.294+0000 7f481f7fe700 1 --2- 172.21.15.163:0/1656582832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4820159890 0x7f482015abd0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f48200b67f0 tx=0x7f4810009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ca5ba0ed634030 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:22.303 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.294+0000 7f481e7fc700 1 -- 172.21.15.163:0/1656582832 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481000e070 con 0x7f4820159890 2023-10-09T06:36:22.303 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.294+0000 7f481e7fc700 1 -- 172.21.15.163:0/1656582832 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481000b970 con 0x7f4820159890 2023-10-09T06:36:22.303 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.294+0000 7f481e7fc700 1 -- 172.21.15.163:0/1656582832 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48100092e0 con 0x7f4820159890 2023-10-09T06:36:22.303 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.294+0000 7f4825f6d700 1 -- 172.21.15.163:0/1656582832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4820159890 msgr2=0x7f482015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:22.303 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.294+0000 7f4825f6d700 1 --2- 172.21.15.163:0/1656582832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4820159890 0x7f482015abd0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f48200b67f0 tx=0x7f4810009840 comp rx=0 tx=0).stop 2023-10-09T06:36:22.303 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.294+0000 7f4825f6d700 1 -- 172.21.15.163:0/1656582832 shutdown_connections 2023-10-09T06:36:22.303 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.294+0000 7f4825f6d700 1 --2- 172.21.15.163:0/1656582832 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f482015db10 0x7f4820069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:22.303 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.294+0000 7f4825f6d700 1 --2- 172.21.15.163:0/1656582832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4820159890 0x7f482015abd0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:22.303 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.294+0000 7f4825f6d700 1 --2- 172.21.15.163:0/1656582832 >> v2:172.21.15.181:3300/0 conn(0x7f4820159c60 0x7f482015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:22.303 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.294+0000 7f4825f6d700 1 -- 172.21.15.163:0/1656582832 >> 172.21.15.163:0/1656582832 conn(0x7f4820068e20 msgr2=0x7f48200d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:22.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.294+0000 7f4825f6d700 1 -- 172.21.15.163:0/1656582832 shutdown_connections 2023-10-09T06:36:22.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.294+0000 7f4825f6d700 1 -- 172.21.15.163:0/1656582832 wait complete. 2023-10-09T06:36:22.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.296+0000 7f4825f6d700 1 Processor -- start 2023-10-09T06:36:22.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.296+0000 7f4825f6d700 1 -- start start 2023-10-09T06:36:22.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.296+0000 7f4825f6d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4820159890 0x7f48201583a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:22.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.296+0000 7f4825f6d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4820159c60 0x7f48200c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:22.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.296+0000 7f4825f6d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f482015db10 0x7f48200c8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:22.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.296+0000 7f4825f6d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482016cf00 con 0x7f482015db10 2023-10-09T06:36:22.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.296+0000 7f4825f6d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4820152940 con 0x7f4820159c60 2023-10-09T06:36:22.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.296+0000 7f4825f6d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48200b3630 con 0x7f4820159890 2023-10-09T06:36:22.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.296+0000 7f481ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f482015db10 0x7f48200c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:22.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.297+0000 7f481f7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4820159890 0x7f48201583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:22.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.297+0000 7f481ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f482015db10 0x7f48200c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:54876/0 (socket says 172.21.15.163:54876) 2023-10-09T06:36:22.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.297+0000 7f481ffff700 1 -- 172.21.15.163:0/2634711922 learned_addr learned my addr 172.21.15.163:0/2634711922 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:22.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.297+0000 7f481effd700 1 --2- 172.21.15.163:0/2634711922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4820159c60 0x7f48200c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:22.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.297+0000 7f481ffff700 1 -- 172.21.15.163:0/2634711922 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4820159890 msgr2=0x7f48201583a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:22.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.297+0000 7f481ffff700 1 --2- 172.21.15.163:0/2634711922 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4820159890 0x7f48201583a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:22.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.297+0000 7f481ffff700 1 -- 172.21.15.163:0/2634711922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4820159c60 msgr2=0x7f48200c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:22.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.297+0000 7f481ffff700 1 --2- 172.21.15.163:0/2634711922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4820159c60 0x7f48200c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:22.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.297+0000 7f481ffff700 1 -- 172.21.15.163:0/2634711922 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4810009ae0 con 0x7f482015db10 2023-10-09T06:36:22.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.297+0000 7f481ffff700 1 --2- 172.21.15.163:0/2634711922 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f482015db10 0x7f48200c8710 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f481800e8d0 tx=0x7f48180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:22.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.297+0000 7f48077fe700 1 -- 172.21.15.163:0/2634711922 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4818013680 con 0x7f482015db10 2023-10-09T06:36:22.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.297+0000 7f48077fe700 1 -- 172.21.15.163:0/2634711922 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48180137e0 con 0x7f482015db10 2023-10-09T06:36:22.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.297+0000 7f4825f6d700 1 -- 172.21.15.163:0/2634711922 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48200b3910 con 0x7f482015db10 2023-10-09T06:36:22.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.297+0000 7f48077fe700 1 -- 172.21.15.163:0/2634711922 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4818013ba0 con 0x7f482015db10 2023-10-09T06:36:22.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.297+0000 7f4825f6d700 1 -- 172.21.15.163:0/2634711922 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48200b3ed0 con 0x7f482015db10 2023-10-09T06:36:22.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.298+0000 7f4825f6d700 1 -- 172.21.15.163:0/2634711922 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f482004c3c0 con 0x7f482015db10 2023-10-09T06:36:22.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.299+0000 7f48077fe700 1 -- 172.21.15.163:0/2634711922 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f48180123d0 con 0x7f482015db10 2023-10-09T06:36:22.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.299+0000 7f48077fe700 1 --2- 172.21.15.163:0/2634711922 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f480c067200 0x7f480c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:22.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.299+0000 7f48077fe700 1 -- 172.21.15.163:0/2634711922 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f48180849c0 con 0x7f482015db10 2023-10-09T06:36:22.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.299+0000 7f481f7fe700 1 --2- 172.21.15.163:0/2634711922 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f480c067200 0x7f480c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:22.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.300+0000 7f481f7fe700 1 --2- 172.21.15.163:0/2634711922 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f480c067200 0x7f480c0696b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4820089500 tx=0x7f4810018000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:22.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.303+0000 7f48077fe700 1 -- 172.21.15.163:0/2634711922 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48180518e0 con 0x7f482015db10 2023-10-09T06:36:22.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.487+0000 7f4825f6d700 1 -- 172.21.15.163:0/2634711922 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f482006a5b0 con 0x7f482015db10 2023-10-09T06:36:22.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.488+0000 7f48077fe700 1 -- 172.21.15.163:0/2634711922 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+15401 (secure 0 0 0) 0x7f4818017090 con 0x7f482015db10 2023-10-09T06:36:22.488 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:22.488 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":89,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:36:19.522776+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:36:16.497125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-09T06:36:17.121589+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:36:22.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.490+0000 7f4825f6d700 1 -- 172.21.15.163:0/2634711922 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f480c067200 msgr2=0x7f480c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:22.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.490+0000 7f4825f6d700 1 --2- 172.21.15.163:0/2634711922 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f480c067200 0x7f480c0696b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4820089500 tx=0x7f4810018000 comp rx=0 tx=0).stop 2023-10-09T06:36:22.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.490+0000 7f4825f6d700 1 -- 172.21.15.163:0/2634711922 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f482015db10 msgr2=0x7f48200c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:22.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.490+0000 7f4825f6d700 1 --2- 172.21.15.163:0/2634711922 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f482015db10 0x7f48200c8710 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f481800e8d0 tx=0x7f48180091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:22.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.490+0000 7f4825f6d700 1 -- 172.21.15.163:0/2634711922 shutdown_connections 2023-10-09T06:36:22.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.490+0000 7f4825f6d700 1 --2- 172.21.15.163:0/2634711922 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f480c067200 0x7f480c0696b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:22.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.490+0000 7f4825f6d700 1 --2- 172.21.15.163:0/2634711922 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f482015db10 0x7f48200c8710 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:22.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.490+0000 7f4825f6d700 1 --2- 172.21.15.163:0/2634711922 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4820159890 0x7f48201583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:22.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.490+0000 7f4825f6d700 1 --2- 172.21.15.163:0/2634711922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4820159c60 0x7f48200c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:22.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.490+0000 7f4825f6d700 1 -- 172.21.15.163:0/2634711922 >> 172.21.15.163:0/2634711922 conn(0x7f4820068e20 msgr2=0x7f48200b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:22.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.490+0000 7f4825f6d700 1 -- 172.21.15.163:0/2634711922 shutdown_connections 2023-10-09T06:36:22.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.491+0000 7f4825f6d700 1 -- 172.21.15.163:0/2634711922 wait complete. 2023-10-09T06:36:22.503 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-09T06:36:22.503 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:36:22.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.618+0000 7f6876e15700 1 Processor -- start 2023-10-09T06:36:22.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.618+0000 7f6876e15700 1 -- start start 2023-10-09T06:36:22.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.619+0000 7f6876e15700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6870159420 0x7f68700b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:22.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.619+0000 7f6876e15700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f687015d230 0x7f687015d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:22.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.619+0000 7f6876e15700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68700b2980 con 0x7f68701597f0 2023-10-09T06:36:22.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.619+0000 7f6876e15700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f687015dbb0 con 0x7f6870159420 2023-10-09T06:36:22.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.619+0000 7f6876e15700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68700b11d0 con 0x7f687015d230 2023-10-09T06:36:22.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.619+0000 7f6875e13700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6870159420 0x7f68700b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:22.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.619+0000 7f6875e13700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6870159420 0x7f68700b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:54888/0 (socket says 172.21.15.163:54888) 2023-10-09T06:36:22.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.619+0000 7f6875e13700 1 -- 172.21.15.163:0/79228097 learned_addr learned my addr 172.21.15.163:0/79228097 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:22.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.619+0000 7f6876614700 1 --2- 172.21.15.163:0/79228097 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f687015d230 0x7f687015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:22.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.619+0000 7f6875e13700 1 -- 172.21.15.163:0/79228097 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f687015d230 msgr2=0x7f687015d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:22.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.619+0000 7f6875e13700 1 --2- 172.21.15.163:0/79228097 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f687015d230 0x7f687015d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:22.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.619+0000 7f6875e13700 1 -- 172.21.15.163:0/79228097 >> v1:172.21.15.181:6789/0 conn(0x7f68701597f0 legacy=0x7f68700b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:22.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.619+0000 7f6875e13700 1 -- 172.21.15.163:0/79228097 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68700b14b0 con 0x7f6870159420 2023-10-09T06:36:22.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.619+0000 7f6875e13700 1 --2- 172.21.15.163:0/79228097 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6870159420 0x7f68700b5f40 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f68680099b0 tx=0x7f6868009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87257d9f40210321 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:22.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.620+0000 7f6874e11700 1 -- 172.21.15.163:0/79228097 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686800b860 con 0x7f6870159420 2023-10-09T06:36:22.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.620+0000 7f6874e11700 1 -- 172.21.15.163:0/79228097 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686800b9c0 con 0x7f6870159420 2023-10-09T06:36:22.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.620+0000 7f6874e11700 1 -- 172.21.15.163:0/79228097 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686800bd60 con 0x7f6870159420 2023-10-09T06:36:22.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.620+0000 7f6876e15700 1 -- 172.21.15.163:0/79228097 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6870159420 msgr2=0x7f68700b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:22.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.620+0000 7f6876e15700 1 --2- 172.21.15.163:0/79228097 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6870159420 0x7f68700b5f40 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f68680099b0 tx=0x7f6868009250 comp rx=0 tx=0).stop 2023-10-09T06:36:22.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.620+0000 7f6876e15700 1 -- 172.21.15.163:0/79228097 shutdown_connections 2023-10-09T06:36:22.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.620+0000 7f6876e15700 1 --2- 172.21.15.163:0/79228097 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f687015d230 0x7f687015d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:22.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.620+0000 7f6876e15700 1 --2- 172.21.15.163:0/79228097 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6870159420 0x7f68700b5f40 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:22.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.620+0000 7f6876e15700 1 -- 172.21.15.163:0/79228097 >> 172.21.15.163:0/79228097 conn(0x7f68700d0660 msgr2=0x7f68700d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:22.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.620+0000 7f6876e15700 1 -- 172.21.15.163:0/79228097 shutdown_connections 2023-10-09T06:36:22.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.620+0000 7f6876e15700 1 -- 172.21.15.163:0/79228097 wait complete. 2023-10-09T06:36:22.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.621+0000 7f6876e15700 1 Processor -- start 2023-10-09T06:36:22.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.621+0000 7f6876e15700 1 -- start start 2023-10-09T06:36:22.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.622+0000 7f6876e15700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6870159420 0x7f68701551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:22.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.622+0000 7f6876e15700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68701597f0 0x7f68700cf950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:22.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.622+0000 7f6876e15700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f687015d230 0x7f68700cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:22.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.622+0000 7f6876e15700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68700c6ba0 con 0x7f6870159420 2023-10-09T06:36:22.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.622+0000 7f6876e15700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68700c9ba0 con 0x7f68701597f0 2023-10-09T06:36:22.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.622+0000 7f6876e15700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68700c9870 con 0x7f687015d230 2023-10-09T06:36:22.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.622+0000 7f6875e13700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6870159420 0x7f68701551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:22.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.622+0000 7f6875e13700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6870159420 0x7f68701551e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:54900/0 (socket says 172.21.15.163:54900) 2023-10-09T06:36:22.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.622+0000 7f6875e13700 1 -- 172.21.15.163:0/4257510322 learned_addr learned my addr 172.21.15.163:0/4257510322 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:22.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.622+0000 7f6875612700 1 --2- 172.21.15.163:0/4257510322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68701597f0 0x7f68700cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:22.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.622+0000 7f6876614700 1 --2- 172.21.15.163:0/4257510322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f687015d230 0x7f68700cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:22.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.622+0000 7f6875e13700 1 -- 172.21.15.163:0/4257510322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f687015d230 msgr2=0x7f68700cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:22.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.622+0000 7f6875e13700 1 --2- 172.21.15.163:0/4257510322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f687015d230 0x7f68700cfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:22.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.622+0000 7f6875e13700 1 -- 172.21.15.163:0/4257510322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68701597f0 msgr2=0x7f68700cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:22.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.622+0000 7f6875e13700 1 --2- 172.21.15.163:0/4257510322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68701597f0 0x7f68700cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:22.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.622+0000 7f6875e13700 1 -- 172.21.15.163:0/4257510322 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68680094f0 con 0x7f6870159420 2023-10-09T06:36:22.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.622+0000 7f6875612700 1 --2- 172.21.15.163:0/4257510322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68701597f0 0x7f68700cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:36:22.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.623+0000 7f6875e13700 1 --2- 172.21.15.163:0/4257510322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6870159420 0x7f68701551e0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f6868006fd0 tx=0x7f6868017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:22.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.623+0000 7f6865ffb700 1 -- 172.21.15.163:0/4257510322 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6868024440 con 0x7f6870159420 2023-10-09T06:36:22.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.623+0000 7f6865ffb700 1 -- 172.21.15.163:0/4257510322 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68680245c0 con 0x7f6870159420 2023-10-09T06:36:22.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.623+0000 7f6876e15700 1 -- 172.21.15.163:0/4257510322 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68700c6300 con 0x7f6870159420 2023-10-09T06:36:22.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.623+0000 7f6865ffb700 1 -- 172.21.15.163:0/4257510322 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6868024980 con 0x7f6870159420 2023-10-09T06:36:22.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.623+0000 7f6876e15700 1 -- 172.21.15.163:0/4257510322 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68700c68c0 con 0x7f6870159420 2023-10-09T06:36:22.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.624+0000 7f6876e15700 1 -- 172.21.15.163:0/4257510322 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f687004c3c0 con 0x7f6870159420 2023-10-09T06:36:22.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.624+0000 7f6865ffb700 1 -- 172.21.15.163:0/4257510322 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f686801a420 con 0x7f6870159420 2023-10-09T06:36:22.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.625+0000 7f6865ffb700 1 --2- 172.21.15.163:0/4257510322 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6854067200 0x7f68540696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:22.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.625+0000 7f6865ffb700 1 -- 172.21.15.163:0/4257510322 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6868086800 con 0x7f6870159420 2023-10-09T06:36:22.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.625+0000 7f6875612700 1 --2- 172.21.15.163:0/4257510322 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6854067200 0x7f68540696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:22.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.625+0000 7f6875612700 1 --2- 172.21.15.163:0/4257510322 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6854067200 0x7f68540696b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f68701586f0 tx=0x7f6860009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:22.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.629+0000 7f6865ffb700 1 -- 172.21.15.163:0/4257510322 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6868053720 con 0x7f6870159420 2023-10-09T06:36:22.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.808+0000 7f6876e15700 1 -- 172.21.15.163:0/4257510322 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f68700b1790 con 0x7f6854067200 2023-10-09T06:36:22.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.816+0000 7f6865ffb700 1 -- 172.21.15.163:0/4257510322 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289479 (secure 0 0 0) 0x7f68700b1790 con 0x7f6854067200 2023-10-09T06:36:22.821 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:22.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.819+0000 7f6876e15700 1 -- 172.21.15.163:0/4257510322 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6854067200 msgr2=0x7f68540696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:22.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.819+0000 7f6876e15700 1 --2- 172.21.15.163:0/4257510322 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6854067200 0x7f68540696b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f68701586f0 tx=0x7f6860009040 comp rx=0 tx=0).stop 2023-10-09T06:36:22.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.819+0000 7f6876e15700 1 -- 172.21.15.163:0/4257510322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6870159420 msgr2=0x7f68701551e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:22.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.819+0000 7f6876e15700 1 --2- 172.21.15.163:0/4257510322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6870159420 0x7f68701551e0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f6868006fd0 tx=0x7f6868017000 comp rx=0 tx=0).stop 2023-10-09T06:36:22.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.819+0000 7f6876e15700 1 -- 172.21.15.163:0/4257510322 shutdown_connections 2023-10-09T06:36:22.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.819+0000 7f6876e15700 1 --2- 172.21.15.163:0/4257510322 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6854067200 0x7f68540696b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:22.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.819+0000 7f6876e15700 1 --2- 172.21.15.163:0/4257510322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6870159420 0x7f68701551e0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:22.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.819+0000 7f6876e15700 1 --2- 172.21.15.163:0/4257510322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f687015d230 0x7f68700cfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:22.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.820+0000 7f6876e15700 1 --2- 172.21.15.163:0/4257510322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68701597f0 0x7f68700cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:22.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.820+0000 7f6876e15700 1 -- 172.21.15.163:0/4257510322 >> 172.21.15.163:0/4257510322 conn(0x7f68700d0660 msgr2=0x7f6870156000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:22.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.820+0000 7f6876e15700 1 -- 172.21.15.163:0/4257510322 shutdown_connections 2023-10-09T06:36:22.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.820+0000 7f6876e15700 1 -- 172.21.15.163:0/4257510322 wait complete. 2023-10-09T06:36:22.825 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:36:22.831 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":258,"stamp":"2023-10-09T06:36:22.382567+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":97,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2400400,"kb_used_data":9808,"kb_used_omap":0,"kb_used_meta":2390528,"kb_avail":752574320,"statfs":{"total":773094113280,"available":770636103680,"internally_reserved":0,"allocated":10043392,"data_stored":4982124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2447900672},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.614481"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.546092+0000","last_change":"2023-10-09T06:36:18.541429+0000","last_active":"2023-10-09T06:36:18.546092+0000","last_peered":"2023-10-09T06:36:18.546092+0000","last_clean":"2023-10-09T06:36:18.546092+0000","last_became_active":"2023-10-09T06:36:18.541161+0000","last_became_peered":"2023-10-09T06:36:18.541161+0000","last_unstale":"2023-10-09T06:36:18.546092+0000","last_undegraded":"2023-10-09T06:36:18.546092+0000","last_fullsized":"2023-10-09T06:36:18.546092+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:25:20.676670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.543457+0000","last_change":"2023-10-09T06:36:18.540749+0000","last_active":"2023-10-09T06:36:18.543457+0000","last_peered":"2023-10-09T06:36:18.543457+0000","last_clean":"2023-10-09T06:36:18.543457+0000","last_became_active":"2023-10-09T06:36:18.540490+0000","last_became_peered":"2023-10-09T06:36:18.540490+0000","last_unstale":"2023-10-09T06:36:18.543457+0000","last_undegraded":"2023-10-09T06:36:18.543457+0000","last_fullsized":"2023-10-09T06:36:18.543457+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:50:33.268079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.546205+0000","last_change":"2023-10-09T06:36:18.541957+0000","last_active":"2023-10-09T06:36:18.546205+0000","last_peered":"2023-10-09T06:36:18.546205+0000","last_clean":"2023-10-09T06:36:18.546205+0000","last_became_active":"2023-10-09T06:36:18.541687+0000","last_became_peered":"2023-10-09T06:36:18.541687+0000","last_unstale":"2023-10-09T06:36:18.546205+0000","last_undegraded":"2023-10-09T06:36:18.546205+0000","last_fullsized":"2023-10-09T06:36:18.546205+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:45:50.359758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.539263+0000","last_change":"2023-10-09T06:36:18.534478+0000","last_active":"2023-10-09T06:36:18.539263+0000","last_peered":"2023-10-09T06:36:18.539263+0000","last_clean":"2023-10-09T06:36:18.539263+0000","last_became_active":"2023-10-09T06:36:18.534225+0000","last_became_peered":"2023-10-09T06:36:18.534225+0000","last_unstale":"2023-10-09T06:36:18.539263+0000","last_undegraded":"2023-10-09T06:36:18.539263+0000","last_fullsized":"2023-10-09T06:36:18.539263+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:38:02.641521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.539419+0000","last_change":"2023-10-09T06:36:18.534316+0000","last_active":"2023-10-09T06:36:18.539419+0000","last_peered":"2023-10-09T06:36:18.539419+0000","last_clean":"2023-10-09T06:36:18.539419+0000","last_became_active":"2023-10-09T06:36:18.534035+0000","last_became_peered":"2023-10-09T06:36:18.534035+0000","last_unstale":"2023-10-09T06:36:18.539419+0000","last_undegraded":"2023-10-09T06:36:18.539419+0000","last_fullsized":"2023-10-09T06:36:18.539419+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:15:59.454989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.547827+0000","last_change":"2023-10-09T06:36:18.545526+0000","last_active":"2023-10-09T06:36:18.547827+0000","last_peered":"2023-10-09T06:36:18.547827+0000","last_clean":"2023-10-09T06:36:18.547827+0000","last_became_active":"2023-10-09T06:36:18.545191+0000","last_became_peered":"2023-10-09T06:36:18.545191+0000","last_unstale":"2023-10-09T06:36:18.547827+0000","last_undegraded":"2023-10-09T06:36:18.547827+0000","last_fullsized":"2023-10-09T06:36:18.547827+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:25:40.992746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.544355+0000","last_change":"2023-10-09T06:36:18.543171+0000","last_active":"2023-10-09T06:36:18.544355+0000","last_peered":"2023-10-09T06:36:18.544355+0000","last_clean":"2023-10-09T06:36:18.544355+0000","last_became_active":"2023-10-09T06:36:18.542688+0000","last_became_peered":"2023-10-09T06:36:18.542688+0000","last_unstale":"2023-10-09T06:36:18.544355+0000","last_undegraded":"2023-10-09T06:36:18.544355+0000","last_fullsized":"2023-10-09T06:36:18.544355+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:46:35.313608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.546016+0000","last_change":"2023-10-09T06:36:18.541168+0000","last_active":"2023-10-09T06:36:18.546016+0000","last_peered":"2023-10-09T06:36:18.546016+0000","last_clean":"2023-10-09T06:36:18.546016+0000","last_became_active":"2023-10-09T06:36:18.540929+0000","last_became_peered":"2023-10-09T06:36:18.540929+0000","last_unstale":"2023-10-09T06:36:18.546016+0000","last_undegraded":"2023-10-09T06:36:18.546016+0000","last_fullsized":"2023-10-09T06:36:18.546016+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:38:02.641521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.540321+0000","last_change":"2023-10-09T06:36:18.538212+0000","last_active":"2023-10-09T06:36:18.540321+0000","last_peered":"2023-10-09T06:36:18.540321+0000","last_clean":"2023-10-09T06:36:18.540321+0000","last_became_active":"2023-10-09T06:36:18.537861+0000","last_became_peered":"2023-10-09T06:36:18.537861+0000","last_unstale":"2023-10-09T06:36:18.540321+0000","last_undegraded":"2023-10-09T06:36:18.540321+0000","last_fullsized":"2023-10-09T06:36:18.540321+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:34:03.599407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.539212+0000","last_change":"2023-10-09T06:36:18.534506+0000","last_active":"2023-10-09T06:36:18.539212+0000","last_peered":"2023-10-09T06:36:18.539212+0000","last_clean":"2023-10-09T06:36:18.539212+0000","last_became_active":"2023-10-09T06:36:18.534169+0000","last_became_peered":"2023-10-09T06:36:18.534169+0000","last_unstale":"2023-10-09T06:36:18.539212+0000","last_undegraded":"2023-10-09T06:36:18.539212+0000","last_fullsized":"2023-10-09T06:36:18.539212+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:37:50.129692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.545696+0000","last_change":"2023-10-09T06:36:18.540876+0000","last_active":"2023-10-09T06:36:18.545696+0000","last_peered":"2023-10-09T06:36:18.545696+0000","last_clean":"2023-10-09T06:36:18.545696+0000","last_became_active":"2023-10-09T06:36:18.540616+0000","last_became_peered":"2023-10-09T06:36:18.540616+0000","last_unstale":"2023-10-09T06:36:18.545696+0000","last_undegraded":"2023-10-09T06:36:18.545696+0000","last_fullsized":"2023-10-09T06:36:18.545696+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:58:03.063521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.540857+0000","last_change":"2023-10-09T06:36:18.536276+0000","last_active":"2023-10-09T06:36:18.540857+0000","last_peered":"2023-10-09T06:36:18.540857+0000","last_clean":"2023-10-09T06:36:18.540857+0000","last_became_active":"2023-10-09T06:36:18.535997+0000","last_became_peered":"2023-10-09T06:36:18.535997+0000","last_unstale":"2023-10-09T06:36:18.540857+0000","last_undegraded":"2023-10-09T06:36:18.540857+0000","last_fullsized":"2023-10-09T06:36:18.540857+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:48:11.177371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.546105+0000","last_change":"2023-10-09T06:36:18.539474+0000","last_active":"2023-10-09T06:36:18.546105+0000","last_peered":"2023-10-09T06:36:18.546105+0000","last_clean":"2023-10-09T06:36:18.546105+0000","last_became_active":"2023-10-09T06:36:18.539176+0000","last_became_peered":"2023-10-09T06:36:18.539176+0000","last_unstale":"2023-10-09T06:36:18.546105+0000","last_undegraded":"2023-10-09T06:36:18.546105+0000","last_fullsized":"2023-10-09T06:36:18.546105+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:47:18.417758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.537602+0000","last_change":"2023-10-09T06:36:18.529730+0000","last_active":"2023-10-09T06:36:18.537602+0000","last_peered":"2023-10-09T06:36:18.537602+0000","last_clean":"2023-10-09T06:36:18.537602+0000","last_became_active":"2023-10-09T06:36:18.529321+0000","last_became_peered":"2023-10-09T06:36:18.529321+0000","last_unstale":"2023-10-09T06:36:18.537602+0000","last_undegraded":"2023-10-09T06:36:18.537602+0000","last_fullsized":"2023-10-09T06:36:18.537602+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:14:27.092915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.543405+0000","last_change":"2023-10-09T06:36:18.535984+0000","last_active":"2023-10-09T06:36:18.543405+0000","last_peered":"2023-10-09T06:36:18.543405+0000","last_clean":"2023-10-09T06:36:18.543405+0000","last_became_active":"2023-10-09T06:36:18.535728+0000","last_became_peered":"2023-10-09T06:36:18.535728+0000","last_unstale":"2023-10-09T06:36:18.543405+0000","last_undegraded":"2023-10-09T06:36:18.543405+0000","last_fullsized":"2023-10-09T06:36:18.543405+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:46:35.313608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.547718+0000","last_change":"2023-10-09T06:36:18.544591+0000","last_active":"2023-10-09T06:36:18.547718+0000","last_peered":"2023-10-09T06:36:18.547718+0000","last_clean":"2023-10-09T06:36:18.547718+0000","last_became_active":"2023-10-09T06:36:18.544091+0000","last_became_peered":"2023-10-09T06:36:18.544091+0000","last_unstale":"2023-10-09T06:36:18.547718+0000","last_undegraded":"2023-10-09T06:36:18.547718+0000","last_fullsized":"2023-10-09T06:36:18.547718+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:35:52.976193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.541701+0000","last_change":"2023-10-09T06:36:18.537299+0000","last_active":"2023-10-09T06:36:18.541701+0000","last_peered":"2023-10-09T06:36:18.541701+0000","last_clean":"2023-10-09T06:36:18.541701+0000","last_became_active":"2023-10-09T06:36:18.537051+0000","last_became_peered":"2023-10-09T06:36:18.537051+0000","last_unstale":"2023-10-09T06:36:18.541701+0000","last_undegraded":"2023-10-09T06:36:18.541701+0000","last_fullsized":"2023-10-09T06:36:18.541701+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:13:52.813781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.538831+0000","last_change":"2023-10-09T06:36:18.534109+0000","last_active":"2023-10-09T06:36:18.538831+0000","last_peered":"2023-10-09T06:36:18.538831+0000","last_clean":"2023-10-09T06:36:18.538831+0000","last_became_active":"2023-10-09T06:36:18.533797+0000","last_became_peered":"2023-10-09T06:36:18.533797+0000","last_unstale":"2023-10-09T06:36:18.538831+0000","last_undegraded":"2023-10-09T06:36:18.538831+0000","last_fullsized":"2023-10-09T06:36:18.538831+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:53:53.083376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.537158+0000","last_change":"2023-10-09T06:36:18.534001+0000","last_active":"2023-10-09T06:36:18.537158+0000","last_peered":"2023-10-09T06:36:18.537158+0000","last_clean":"2023-10-09T06:36:18.537158+0000","last_became_active":"2023-10-09T06:36:18.533751+0000","last_became_peered":"2023-10-09T06:36:18.533751+0000","last_unstale":"2023-10-09T06:36:18.537158+0000","last_undegraded":"2023-10-09T06:36:18.537158+0000","last_fullsized":"2023-10-09T06:36:18.537158+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:31:48.056171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.540878+0000","last_change":"2023-10-09T06:36:18.538080+0000","last_active":"2023-10-09T06:36:18.540878+0000","last_peered":"2023-10-09T06:36:18.540878+0000","last_clean":"2023-10-09T06:36:18.540878+0000","last_became_active":"2023-10-09T06:36:18.537790+0000","last_became_peered":"2023-10-09T06:36:18.537790+0000","last_unstale":"2023-10-09T06:36:18.540878+0000","last_undegraded":"2023-10-09T06:36:18.540878+0000","last_fullsized":"2023-10-09T06:36:18.540878+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:36:26.717698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.543644+0000","last_change":"2023-10-09T06:36:18.541076+0000","last_active":"2023-10-09T06:36:18.543644+0000","last_peered":"2023-10-09T06:36:18.543644+0000","last_clean":"2023-10-09T06:36:18.543644+0000","last_became_active":"2023-10-09T06:36:18.540802+0000","last_became_peered":"2023-10-09T06:36:18.540802+0000","last_unstale":"2023-10-09T06:36:18.543644+0000","last_undegraded":"2023-10-09T06:36:18.543644+0000","last_fullsized":"2023-10-09T06:36:18.543644+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:17:02.758096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.546050+0000","last_change":"2023-10-09T06:36:18.542358+0000","last_active":"2023-10-09T06:36:18.546050+0000","last_peered":"2023-10-09T06:36:18.546050+0000","last_clean":"2023-10-09T06:36:18.546050+0000","last_became_active":"2023-10-09T06:36:18.542090+0000","last_became_peered":"2023-10-09T06:36:18.542090+0000","last_unstale":"2023-10-09T06:36:18.546050+0000","last_undegraded":"2023-10-09T06:36:18.546050+0000","last_fullsized":"2023-10-09T06:36:18.546050+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:25:35.071015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.543855+0000","last_change":"2023-10-09T06:36:18.540804+0000","last_active":"2023-10-09T06:36:18.543855+0000","last_peered":"2023-10-09T06:36:18.543855+0000","last_clean":"2023-10-09T06:36:18.543855+0000","last_became_active":"2023-10-09T06:36:18.540557+0000","last_became_peered":"2023-10-09T06:36:18.540557+0000","last_unstale":"2023-10-09T06:36:18.543855+0000","last_undegraded":"2023-10-09T06:36:18.543855+0000","last_fullsized":"2023-10-09T06:36:18.543855+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:35:52.976193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.544734+0000","last_change":"2023-10-09T06:36:18.543081+0000","last_active":"2023-10-09T06:36:18.544734+0000","last_peered":"2023-10-09T06:36:18.544734+0000","last_clean":"2023-10-09T06:36:18.544734+0000","last_became_active":"2023-10-09T06:36:18.542799+0000","last_became_peered":"2023-10-09T06:36:18.542799+0000","last_unstale":"2023-10-09T06:36:18.544734+0000","last_undegraded":"2023-10-09T06:36:18.544734+0000","last_fullsized":"2023-10-09T06:36:18.544734+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:50:33.268079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.539540+0000","last_change":"2023-10-09T06:36:18.536990+0000","last_active":"2023-10-09T06:36:18.539540+0000","last_peered":"2023-10-09T06:36:18.539540+0000","last_clean":"2023-10-09T06:36:18.539540+0000","last_became_active":"2023-10-09T06:36:18.536697+0000","last_became_peered":"2023-10-09T06:36:18.536697+0000","last_unstale":"2023-10-09T06:36:18.539540+0000","last_undegraded":"2023-10-09T06:36:18.539540+0000","last_fullsized":"2023-10-09T06:36:18.539540+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:36:26.717698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.543419+0000","last_change":"2023-10-09T06:36:18.539886+0000","last_active":"2023-10-09T06:36:18.543419+0000","last_peered":"2023-10-09T06:36:18.543419+0000","last_clean":"2023-10-09T06:36:18.543419+0000","last_became_active":"2023-10-09T06:36:18.539643+0000","last_became_peered":"2023-10-09T06:36:18.539643+0000","last_unstale":"2023-10-09T06:36:18.543419+0000","last_undegraded":"2023-10-09T06:36:18.543419+0000","last_fullsized":"2023-10-09T06:36:18.543419+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:18:46.793953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.547711+0000","last_change":"2023-10-09T06:36:18.544727+0000","last_active":"2023-10-09T06:36:18.547711+0000","last_peered":"2023-10-09T06:36:18.547711+0000","last_clean":"2023-10-09T06:36:18.547711+0000","last_became_active":"2023-10-09T06:36:18.543753+0000","last_became_peered":"2023-10-09T06:36:18.543753+0000","last_unstale":"2023-10-09T06:36:18.547711+0000","last_undegraded":"2023-10-09T06:36:18.547711+0000","last_fullsized":"2023-10-09T06:36:18.547711+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:47:18.417758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.544139+0000","last_change":"2023-10-09T06:36:18.538657+0000","last_active":"2023-10-09T06:36:18.544139+0000","last_peered":"2023-10-09T06:36:18.544139+0000","last_clean":"2023-10-09T06:36:18.544139+0000","last_became_active":"2023-10-09T06:36:18.538151+0000","last_became_peered":"2023-10-09T06:36:18.538151+0000","last_unstale":"2023-10-09T06:36:18.544139+0000","last_undegraded":"2023-10-09T06:36:18.544139+0000","last_fullsized":"2023-10-09T06:36:18.544139+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:32:10.068899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.541358+0000","last_change":"2023-10-09T06:36:18.536121+0000","last_active":"2023-10-09T06:36:18.541358+0000","last_peered":"2023-10-09T06:36:18.541358+0000","last_clean":"2023-10-09T06:36:18.541358+0000","last_became_active":"2023-10-09T06:36:18.535859+0000","last_became_peered":"2023-10-09T06:36:18.535859+0000","last_unstale":"2023-10-09T06:36:18.541358+0000","last_undegraded":"2023-10-09T06:36:18.541358+0000","last_fullsized":"2023-10-09T06:36:18.541358+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:25:40.992746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.541414+0000","last_change":"2023-10-09T06:36:18.534555+0000","last_active":"2023-10-09T06:36:18.541414+0000","last_peered":"2023-10-09T06:36:18.541414+0000","last_clean":"2023-10-09T06:36:18.541414+0000","last_became_active":"2023-10-09T06:36:18.534157+0000","last_became_peered":"2023-10-09T06:36:18.534157+0000","last_unstale":"2023-10-09T06:36:18.541414+0000","last_undegraded":"2023-10-09T06:36:18.541414+0000","last_fullsized":"2023-10-09T06:36:18.541414+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:58:12.693109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.546146+0000","last_change":"2023-10-09T06:36:18.539980+0000","last_active":"2023-10-09T06:36:18.546146+0000","last_peered":"2023-10-09T06:36:18.546146+0000","last_clean":"2023-10-09T06:36:18.546146+0000","last_became_active":"2023-10-09T06:36:18.539733+0000","last_became_peered":"2023-10-09T06:36:18.539733+0000","last_unstale":"2023-10-09T06:36:18.546146+0000","last_undegraded":"2023-10-09T06:36:18.546146+0000","last_fullsized":"2023-10-09T06:36:18.546146+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:18:46.793953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.541005+0000","last_change":"2023-10-09T06:36:18.535952+0000","last_active":"2023-10-09T06:36:18.541005+0000","last_peered":"2023-10-09T06:36:18.541005+0000","last_clean":"2023-10-09T06:36:18.541005+0000","last_became_active":"2023-10-09T06:36:18.535676+0000","last_became_peered":"2023-10-09T06:36:18.535676+0000","last_unstale":"2023-10-09T06:36:18.541005+0000","last_undegraded":"2023-10-09T06:36:18.541005+0000","last_fullsized":"2023-10-09T06:36:18.541005+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:07:24.866847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.548753+0000","last_change":"2023-10-09T06:36:18.543941+0000","last_active":"2023-10-09T06:36:18.548753+0000","last_peered":"2023-10-09T06:36:18.548753+0000","last_clean":"2023-10-09T06:36:18.548753+0000","last_became_active":"2023-10-09T06:36:18.543663+0000","last_became_peered":"2023-10-09T06:36:18.543663+0000","last_unstale":"2023-10-09T06:36:18.548753+0000","last_undegraded":"2023-10-09T06:36:18.548753+0000","last_fullsized":"2023-10-09T06:36:18.548753+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:01:04.121181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"88'4","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.785772+0000","last_change":"2023-10-09T06:36:17.789045+0000","last_active":"2023-10-09T06:36:18.785772+0000","last_peered":"2023-10-09T06:36:18.785772+0000","last_clean":"2023-10-09T06:36:18.785772+0000","last_became_active":"2023-10-09T06:36:17.788789+0000","last_became_peered":"2023-10-09T06:36:17.788789+0000","last_unstale":"2023-10-09T06:36:18.785772+0000","last_undegraded":"2023-10-09T06:36:18.785772+0000","last_fullsized":"2023-10-09T06:36:18.785772+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:37:49.125201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.546157+0000","last_change":"2023-10-09T06:36:18.540062+0000","last_active":"2023-10-09T06:36:18.546157+0000","last_peered":"2023-10-09T06:36:18.546157+0000","last_clean":"2023-10-09T06:36:18.546157+0000","last_became_active":"2023-10-09T06:36:18.539812+0000","last_became_peered":"2023-10-09T06:36:18.539812+0000","last_unstale":"2023-10-09T06:36:18.546157+0000","last_undegraded":"2023-10-09T06:36:18.546157+0000","last_fullsized":"2023-10-09T06:36:18.546157+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:59:36.007453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"88'1","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.785749+0000","last_change":"2023-10-09T06:36:17.787086+0000","last_active":"2023-10-09T06:36:18.785749+0000","last_peered":"2023-10-09T06:36:18.785749+0000","last_clean":"2023-10-09T06:36:18.785749+0000","last_became_active":"2023-10-09T06:36:17.786801+0000","last_became_peered":"2023-10-09T06:36:17.786801+0000","last_unstale":"2023-10-09T06:36:18.785749+0000","last_undegraded":"2023-10-09T06:36:18.785749+0000","last_fullsized":"2023-10-09T06:36:18.785749+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:36:25.713208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.542327+0000","last_change":"2023-10-09T06:36:18.539481+0000","last_active":"2023-10-09T06:36:18.542327+0000","last_peered":"2023-10-09T06:36:18.542327+0000","last_clean":"2023-10-09T06:36:18.542327+0000","last_became_active":"2023-10-09T06:36:18.539242+0000","last_became_peered":"2023-10-09T06:36:18.539242+0000","last_unstale":"2023-10-09T06:36:18.542327+0000","last_undegraded":"2023-10-09T06:36:18.542327+0000","last_fullsized":"2023-10-09T06:36:18.542327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:07:55.608736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.790465+0000","last_change":"2023-10-09T06:36:17.790681+0000","last_active":"2023-10-09T06:36:18.790465+0000","last_peered":"2023-10-09T06:36:18.790465+0000","last_clean":"2023-10-09T06:36:18.790465+0000","last_became_active":"2023-10-09T06:36:17.789335+0000","last_became_peered":"2023-10-09T06:36:17.789335+0000","last_unstale":"2023-10-09T06:36:18.790465+0000","last_undegraded":"2023-10-09T06:36:18.790465+0000","last_fullsized":"2023-10-09T06:36:18.790465+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:43:52.206102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.547106+0000","last_change":"2023-10-09T06:36:18.546318+0000","last_active":"2023-10-09T06:36:18.547106+0000","last_peered":"2023-10-09T06:36:18.547106+0000","last_clean":"2023-10-09T06:36:18.547106+0000","last_became_active":"2023-10-09T06:36:18.546028+0000","last_became_peered":"2023-10-09T06:36:18.546028+0000","last_unstale":"2023-10-09T06:36:18.547106+0000","last_undegraded":"2023-10-09T06:36:18.547106+0000","last_fullsized":"2023-10-09T06:36:18.547106+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:45:50.359758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.785760+0000","last_change":"2023-10-09T06:36:17.792815+0000","last_active":"2023-10-09T06:36:18.785760+0000","last_peered":"2023-10-09T06:36:18.785760+0000","last_clean":"2023-10-09T06:36:18.785760+0000","last_became_active":"2023-10-09T06:36:17.792546+0000","last_became_peered":"2023-10-09T06:36:17.792546+0000","last_unstale":"2023-10-09T06:36:18.785760+0000","last_undegraded":"2023-10-09T06:36:18.785760+0000","last_fullsized":"2023-10-09T06:36:18.785760+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:13:55.927806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.547689+0000","last_change":"2023-10-09T06:36:18.541248+0000","last_active":"2023-10-09T06:36:18.547689+0000","last_peered":"2023-10-09T06:36:18.547689+0000","last_clean":"2023-10-09T06:36:18.547689+0000","last_became_active":"2023-10-09T06:36:18.540823+0000","last_became_peered":"2023-10-09T06:36:18.540823+0000","last_unstale":"2023-10-09T06:36:18.547689+0000","last_undegraded":"2023-10-09T06:36:18.547689+0000","last_fullsized":"2023-10-09T06:36:18.547689+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:13:56.932296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"88'1","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.784223+0000","last_change":"2023-10-09T06:36:17.783698+0000","last_active":"2023-10-09T06:36:18.784223+0000","last_peered":"2023-10-09T06:36:18.784223+0000","last_clean":"2023-10-09T06:36:18.784223+0000","last_became_active":"2023-10-09T06:36:17.783443+0000","last_became_peered":"2023-10-09T06:36:17.783443+0000","last_unstale":"2023-10-09T06:36:18.784223+0000","last_undegraded":"2023-10-09T06:36:18.784223+0000","last_fullsized":"2023-10-09T06:36:18.784223+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:02:51.159150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.544310+0000","last_change":"2023-10-09T06:36:18.542927+0000","last_active":"2023-10-09T06:36:18.544310+0000","last_peered":"2023-10-09T06:36:18.544310+0000","last_clean":"2023-10-09T06:36:18.544310+0000","last_became_active":"2023-10-09T06:36:18.542636+0000","last_became_peered":"2023-10-09T06:36:18.542636+0000","last_unstale":"2023-10-09T06:36:18.544310+0000","last_undegraded":"2023-10-09T06:36:18.544310+0000","last_fullsized":"2023-10-09T06:36:18.544310+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:38:02.641521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"88'1","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.785624+0000","last_change":"2023-10-09T06:36:17.791534+0000","last_active":"2023-10-09T06:36:18.785624+0000","last_peered":"2023-10-09T06:36:18.785624+0000","last_clean":"2023-10-09T06:36:18.785624+0000","last_became_active":"2023-10-09T06:36:17.791247+0000","last_became_peered":"2023-10-09T06:36:17.791247+0000","last_unstale":"2023-10-09T06:36:18.785624+0000","last_undegraded":"2023-10-09T06:36:18.785624+0000","last_fullsized":"2023-10-09T06:36:18.785624+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:32:09.064409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.543501+0000","last_change":"2023-10-09T06:36:18.536143+0000","last_active":"2023-10-09T06:36:18.543501+0000","last_peered":"2023-10-09T06:36:18.543501+0000","last_clean":"2023-10-09T06:36:18.543501+0000","last_became_active":"2023-10-09T06:36:18.535861+0000","last_became_peered":"2023-10-09T06:36:18.535861+0000","last_unstale":"2023-10-09T06:36:18.543501+0000","last_undegraded":"2023-10-09T06:36:18.543501+0000","last_fullsized":"2023-10-09T06:36:18.543501+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:25:40.992746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.526884+0000","last_change":"2023-10-09T06:36:17.790108+0000","last_active":"2023-10-09T06:36:18.526884+0000","last_peered":"2023-10-09T06:36:18.526884+0000","last_clean":"2023-10-09T06:36:18.526884+0000","last_became_active":"2023-10-09T06:36:17.789670+0000","last_became_peered":"2023-10-09T06:36:17.789670+0000","last_unstale":"2023-10-09T06:36:18.526884+0000","last_undegraded":"2023-10-09T06:36:18.526884+0000","last_fullsized":"2023-10-09T06:36:18.526884+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:01:10.473684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.542340+0000","last_change":"2023-10-09T06:36:18.541208+0000","last_active":"2023-10-09T06:36:18.542340+0000","last_peered":"2023-10-09T06:36:18.542340+0000","last_clean":"2023-10-09T06:36:18.542340+0000","last_became_active":"2023-10-09T06:36:18.540906+0000","last_became_peered":"2023-10-09T06:36:18.540906+0000","last_unstale":"2023-10-09T06:36:18.542340+0000","last_undegraded":"2023-10-09T06:36:18.542340+0000","last_fullsized":"2023-10-09T06:36:18.542340+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:35:51.857807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.527982+0000","last_change":"2023-10-09T06:36:17.787232+0000","last_active":"2023-10-09T06:36:18.527982+0000","last_peered":"2023-10-09T06:36:18.527982+0000","last_clean":"2023-10-09T06:36:18.527982+0000","last_became_active":"2023-10-09T06:36:17.786928+0000","last_became_peered":"2023-10-09T06:36:17.786928+0000","last_unstale":"2023-10-09T06:36:18.527982+0000","last_undegraded":"2023-10-09T06:36:18.527982+0000","last_fullsized":"2023-10-09T06:36:18.527982+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:18:25.639233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.543441+0000","last_change":"2023-10-09T06:36:18.540166+0000","last_active":"2023-10-09T06:36:18.543441+0000","last_peered":"2023-10-09T06:36:18.543441+0000","last_clean":"2023-10-09T06:36:18.543441+0000","last_became_active":"2023-10-09T06:36:18.539225+0000","last_became_peered":"2023-10-09T06:36:18.539225+0000","last_unstale":"2023-10-09T06:36:18.543441+0000","last_undegraded":"2023-10-09T06:36:18.543441+0000","last_fullsized":"2023-10-09T06:36:18.543441+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:47:18.417758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.528512+0000","last_change":"2023-10-09T06:36:17.784692+0000","last_active":"2023-10-09T06:36:18.528512+0000","last_peered":"2023-10-09T06:36:18.528512+0000","last_clean":"2023-10-09T06:36:18.528512+0000","last_became_active":"2023-10-09T06:36:17.784451+0000","last_became_peered":"2023-10-09T06:36:17.784451+0000","last_unstale":"2023-10-09T06:36:18.528512+0000","last_undegraded":"2023-10-09T06:36:18.528512+0000","last_fullsized":"2023-10-09T06:36:18.528512+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:01:09.038441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.545654+0000","last_change":"2023-10-09T06:36:18.543166+0000","last_active":"2023-10-09T06:36:18.545654+0000","last_peered":"2023-10-09T06:36:18.545654+0000","last_clean":"2023-10-09T06:36:18.545654+0000","last_became_active":"2023-10-09T06:36:18.541454+0000","last_became_peered":"2023-10-09T06:36:18.541454+0000","last_unstale":"2023-10-09T06:36:18.545654+0000","last_undegraded":"2023-10-09T06:36:18.545654+0000","last_fullsized":"2023-10-09T06:36:18.545654+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:43:53.210593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.529024+0000","last_change":"2023-10-09T06:36:17.793119+0000","last_active":"2023-10-09T06:36:18.529024+0000","last_peered":"2023-10-09T06:36:18.529024+0000","last_clean":"2023-10-09T06:36:18.529024+0000","last_became_active":"2023-10-09T06:36:17.792867+0000","last_became_peered":"2023-10-09T06:36:17.792867+0000","last_unstale":"2023-10-09T06:36:18.529024+0000","last_undegraded":"2023-10-09T06:36:18.529024+0000","last_fullsized":"2023-10-09T06:36:18.529024+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:45:49.355268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.544162+0000","last_change":"2023-10-09T06:36:18.538828+0000","last_active":"2023-10-09T06:36:18.544162+0000","last_peered":"2023-10-09T06:36:18.544162+0000","last_clean":"2023-10-09T06:36:18.544162+0000","last_became_active":"2023-10-09T06:36:18.538465+0000","last_became_peered":"2023-10-09T06:36:18.538465+0000","last_unstale":"2023-10-09T06:36:18.544162+0000","last_undegraded":"2023-10-09T06:36:18.544162+0000","last_fullsized":"2023-10-09T06:36:18.544162+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:48:11.177371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"88'3","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.787110+0000","last_change":"2023-10-09T06:36:17.792731+0000","last_active":"2023-10-09T06:36:18.787110+0000","last_peered":"2023-10-09T06:36:18.787110+0000","last_clean":"2023-10-09T06:36:18.787110+0000","last_became_active":"2023-10-09T06:36:17.792268+0000","last_became_peered":"2023-10-09T06:36:17.792268+0000","last_unstale":"2023-10-09T06:36:18.787110+0000","last_undegraded":"2023-10-09T06:36:18.787110+0000","last_fullsized":"2023-10-09T06:36:18.787110+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:32:09.064409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.547730+0000","last_change":"2023-10-09T06:36:18.544894+0000","last_active":"2023-10-09T06:36:18.547730+0000","last_peered":"2023-10-09T06:36:18.547730+0000","last_clean":"2023-10-09T06:36:18.547730+0000","last_became_active":"2023-10-09T06:36:18.544172+0000","last_became_peered":"2023-10-09T06:36:18.544172+0000","last_unstale":"2023-10-09T06:36:18.547730+0000","last_undegraded":"2023-10-09T06:36:18.547730+0000","last_fullsized":"2023-10-09T06:36:18.547730+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:13:52.813781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"88'6","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.797787+0000","last_change":"2023-10-09T06:36:17.792248+0000","last_active":"2023-10-09T06:36:18.797787+0000","last_peered":"2023-10-09T06:36:18.797787+0000","last_clean":"2023-10-09T06:36:18.797787+0000","last_became_active":"2023-10-09T06:36:17.791859+0000","last_became_peered":"2023-10-09T06:36:17.791859+0000","last_unstale":"2023-10-09T06:36:18.797787+0000","last_undegraded":"2023-10-09T06:36:18.797787+0000","last_fullsized":"2023-10-09T06:36:18.797787+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:25:34.425692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.546175+0000","last_change":"2023-10-09T06:36:18.542359+0000","last_active":"2023-10-09T06:36:18.546175+0000","last_peered":"2023-10-09T06:36:18.546175+0000","last_clean":"2023-10-09T06:36:18.546175+0000","last_became_active":"2023-10-09T06:36:18.542099+0000","last_became_peered":"2023-10-09T06:36:18.542099+0000","last_unstale":"2023-10-09T06:36:18.546175+0000","last_undegraded":"2023-10-09T06:36:18.546175+0000","last_fullsized":"2023-10-09T06:36:18.546175+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:25:35.071015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.785802+0000","last_change":"2023-10-09T06:36:17.790092+0000","last_active":"2023-10-09T06:36:18.785802+0000","last_peered":"2023-10-09T06:36:18.785802+0000","last_clean":"2023-10-09T06:36:18.785802+0000","last_became_active":"2023-10-09T06:36:17.789772+0000","last_became_peered":"2023-10-09T06:36:17.789772+0000","last_unstale":"2023-10-09T06:36:18.785802+0000","last_undegraded":"2023-10-09T06:36:18.785802+0000","last_fullsized":"2023-10-09T06:36:18.785802+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:36:07.257669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.542264+0000","last_change":"2023-10-09T06:36:18.538233+0000","last_active":"2023-10-09T06:36:18.542264+0000","last_peered":"2023-10-09T06:36:18.542264+0000","last_clean":"2023-10-09T06:36:18.542264+0000","last_became_active":"2023-10-09T06:36:18.537926+0000","last_became_peered":"2023-10-09T06:36:18.537926+0000","last_unstale":"2023-10-09T06:36:18.542264+0000","last_undegraded":"2023-10-09T06:36:18.542264+0000","last_fullsized":"2023-10-09T06:36:18.542264+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:48:11.177371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.525374+0000","last_change":"2023-10-09T06:36:17.787958+0000","last_active":"2023-10-09T06:36:18.525374+0000","last_peered":"2023-10-09T06:36:18.525374+0000","last_clean":"2023-10-09T06:36:18.525374+0000","last_became_active":"2023-10-09T06:36:17.787701+0000","last_became_peered":"2023-10-09T06:36:17.787701+0000","last_unstale":"2023-10-09T06:36:18.525374+0000","last_undegraded":"2023-10-09T06:36:18.525374+0000","last_fullsized":"2023-10-09T06:36:18.525374+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:21:55.532787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.545777+0000","last_change":"2023-10-09T06:36:18.543187+0000","last_active":"2023-10-09T06:36:18.545777+0000","last_peered":"2023-10-09T06:36:18.545777+0000","last_clean":"2023-10-09T06:36:18.545777+0000","last_became_active":"2023-10-09T06:36:18.541974+0000","last_became_peered":"2023-10-09T06:36:18.541974+0000","last_unstale":"2023-10-09T06:36:18.545777+0000","last_undegraded":"2023-10-09T06:36:18.545777+0000","last_fullsized":"2023-10-09T06:36:18.545777+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:25:20.676670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.530417+0000","last_change":"2023-10-09T06:36:17.789901+0000","last_active":"2023-10-09T06:36:18.530417+0000","last_peered":"2023-10-09T06:36:18.530417+0000","last_clean":"2023-10-09T06:36:18.530417+0000","last_became_active":"2023-10-09T06:36:17.789637+0000","last_became_peered":"2023-10-09T06:36:17.789637+0000","last_unstale":"2023-10-09T06:36:18.530417+0000","last_undegraded":"2023-10-09T06:36:18.530417+0000","last_fullsized":"2023-10-09T06:36:18.530417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:04:48.386913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.534651+0000","last_change":"2023-10-09T06:36:18.532913+0000","last_active":"2023-10-09T06:36:18.534651+0000","last_peered":"2023-10-09T06:36:18.534651+0000","last_clean":"2023-10-09T06:36:18.534651+0000","last_became_active":"2023-10-09T06:36:18.532670+0000","last_became_peered":"2023-10-09T06:36:18.532670+0000","last_unstale":"2023-10-09T06:36:18.534651+0000","last_undegraded":"2023-10-09T06:36:18.534651+0000","last_fullsized":"2023-10-09T06:36:18.534651+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:14:22.790297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"88'8","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.788094+0000","last_change":"2023-10-09T06:36:17.787273+0000","last_active":"2023-10-09T06:36:18.788094+0000","last_peered":"2023-10-09T06:36:18.788094+0000","last_clean":"2023-10-09T06:36:18.788094+0000","last_became_active":"2023-10-09T06:36:17.787000+0000","last_became_peered":"2023-10-09T06:36:17.787000+0000","last_unstale":"2023-10-09T06:36:18.788094+0000","last_undegraded":"2023-10-09T06:36:18.788094+0000","last_fullsized":"2023-10-09T06:36:18.788094+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:43:46.558569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.539259+0000","last_change":"2023-10-09T06:36:18.533573+0000","last_active":"2023-10-09T06:36:18.539259+0000","last_peered":"2023-10-09T06:36:18.539259+0000","last_clean":"2023-10-09T06:36:18.539259+0000","last_became_active":"2023-10-09T06:36:18.533249+0000","last_became_peered":"2023-10-09T06:36:18.533249+0000","last_unstale":"2023-10-09T06:36:18.539259+0000","last_undegraded":"2023-10-09T06:36:18.539259+0000","last_fullsized":"2023-10-09T06:36:18.539259+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:34:03.599407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.786103+0000","last_change":"2023-10-09T06:36:17.791080+0000","last_active":"2023-10-09T06:36:18.786103+0000","last_peered":"2023-10-09T06:36:18.786103+0000","last_clean":"2023-10-09T06:36:18.786103+0000","last_became_active":"2023-10-09T06:36:17.790800+0000","last_became_peered":"2023-10-09T06:36:17.790800+0000","last_unstale":"2023-10-09T06:36:18.786103+0000","last_undegraded":"2023-10-09T06:36:18.786103+0000","last_fullsized":"2023-10-09T06:36:18.786103+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:36:22.684838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.546061+0000","last_change":"2023-10-09T06:36:18.540939+0000","last_active":"2023-10-09T06:36:18.546061+0000","last_peered":"2023-10-09T06:36:18.546061+0000","last_clean":"2023-10-09T06:36:18.546061+0000","last_became_active":"2023-10-09T06:36:18.540653+0000","last_became_peered":"2023-10-09T06:36:18.540653+0000","last_unstale":"2023-10-09T06:36:18.546061+0000","last_undegraded":"2023-10-09T06:36:18.546061+0000","last_fullsized":"2023-10-09T06:36:18.546061+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:02:12.638507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"88'1","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.784927+0000","last_change":"2023-10-09T06:36:17.785484+0000","last_active":"2023-10-09T06:36:18.784927+0000","last_peered":"2023-10-09T06:36:18.784927+0000","last_clean":"2023-10-09T06:36:18.784927+0000","last_became_active":"2023-10-09T06:36:17.785101+0000","last_became_peered":"2023-10-09T06:36:17.785101+0000","last_unstale":"2023-10-09T06:36:18.784927+0000","last_undegraded":"2023-10-09T06:36:18.784927+0000","last_fullsized":"2023-10-09T06:36:18.784927+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:36:22.684838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.547690+0000","last_change":"2023-10-09T06:36:18.544498+0000","last_active":"2023-10-09T06:36:18.547690+0000","last_peered":"2023-10-09T06:36:18.547690+0000","last_clean":"2023-10-09T06:36:18.547690+0000","last_became_active":"2023-10-09T06:36:18.543371+0000","last_became_peered":"2023-10-09T06:36:18.543371+0000","last_unstale":"2023-10-09T06:36:18.547690+0000","last_undegraded":"2023-10-09T06:36:18.547690+0000","last_fullsized":"2023-10-09T06:36:18.547690+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:18:01.852339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.532429+0000","last_change":"2023-10-09T06:36:17.792800+0000","last_active":"2023-10-09T06:36:18.532429+0000","last_peered":"2023-10-09T06:36:18.532429+0000","last_clean":"2023-10-09T06:36:18.532429+0000","last_became_active":"2023-10-09T06:36:17.792526+0000","last_became_peered":"2023-10-09T06:36:17.792526+0000","last_unstale":"2023-10-09T06:36:18.532429+0000","last_undegraded":"2023-10-09T06:36:18.532429+0000","last_fullsized":"2023-10-09T06:36:18.532429+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:21:51.581268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.541571+0000","last_change":"2023-10-09T06:36:18.536770+0000","last_active":"2023-10-09T06:36:18.541571+0000","last_peered":"2023-10-09T06:36:18.541571+0000","last_clean":"2023-10-09T06:36:18.541571+0000","last_became_active":"2023-10-09T06:36:18.536464+0000","last_became_peered":"2023-10-09T06:36:18.536464+0000","last_unstale":"2023-10-09T06:36:18.541571+0000","last_undegraded":"2023-10-09T06:36:18.541571+0000","last_fullsized":"2023-10-09T06:36:18.541571+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:35:38.242484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.790605+0000","last_change":"2023-10-09T06:36:17.791898+0000","last_active":"2023-10-09T06:36:18.790605+0000","last_peered":"2023-10-09T06:36:18.790605+0000","last_clean":"2023-10-09T06:36:18.790605+0000","last_became_active":"2023-10-09T06:36:17.791560+0000","last_became_peered":"2023-10-09T06:36:17.791560+0000","last_unstale":"2023-10-09T06:36:18.790605+0000","last_undegraded":"2023-10-09T06:36:18.790605+0000","last_fullsized":"2023-10-09T06:36:18.790605+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:35:50.853317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.540041+0000","last_change":"2023-10-09T06:36:18.535564+0000","last_active":"2023-10-09T06:36:18.540041+0000","last_peered":"2023-10-09T06:36:18.540041+0000","last_clean":"2023-10-09T06:36:18.540041+0000","last_became_active":"2023-10-09T06:36:18.535299+0000","last_became_peered":"2023-10-09T06:36:18.535299+0000","last_unstale":"2023-10-09T06:36:18.540041+0000","last_undegraded":"2023-10-09T06:36:18.540041+0000","last_fullsized":"2023-10-09T06:36:18.540041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:46:16.269549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.534300+0000","last_change":"2023-10-09T06:36:17.790420+0000","last_active":"2023-10-09T06:36:18.534300+0000","last_peered":"2023-10-09T06:36:18.534300+0000","last_clean":"2023-10-09T06:36:18.534300+0000","last_became_active":"2023-10-09T06:36:17.790228+0000","last_became_peered":"2023-10-09T06:36:17.790228+0000","last_unstale":"2023-10-09T06:36:18.534300+0000","last_undegraded":"2023-10-09T06:36:18.534300+0000","last_fullsized":"2023-10-09T06:36:18.534300+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:01:09.038441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.537322+0000","last_change":"2023-10-09T06:36:18.534119+0000","last_active":"2023-10-09T06:36:18.537322+0000","last_peered":"2023-10-09T06:36:18.537322+0000","last_clean":"2023-10-09T06:36:18.537322+0000","last_became_active":"2023-10-09T06:36:18.533898+0000","last_became_peered":"2023-10-09T06:36:18.533898+0000","last_unstale":"2023-10-09T06:36:18.537322+0000","last_undegraded":"2023-10-09T06:36:18.537322+0000","last_fullsized":"2023-10-09T06:36:18.537322+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:34:20.539836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.534524+0000","last_change":"2023-10-09T06:36:17.787612+0000","last_active":"2023-10-09T06:36:18.534524+0000","last_peered":"2023-10-09T06:36:18.534524+0000","last_clean":"2023-10-09T06:36:18.534524+0000","last_became_active":"2023-10-09T06:36:17.787335+0000","last_became_peered":"2023-10-09T06:36:17.787335+0000","last_unstale":"2023-10-09T06:36:18.534524+0000","last_undegraded":"2023-10-09T06:36:18.534524+0000","last_fullsized":"2023-10-09T06:36:18.534524+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:25:34.425692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.539505+0000","last_change":"2023-10-09T06:36:18.536670+0000","last_active":"2023-10-09T06:36:18.539505+0000","last_peered":"2023-10-09T06:36:18.539505+0000","last_clean":"2023-10-09T06:36:18.539505+0000","last_became_active":"2023-10-09T06:36:18.536058+0000","last_became_peered":"2023-10-09T06:36:18.536058+0000","last_unstale":"2023-10-09T06:36:18.539505+0000","last_undegraded":"2023-10-09T06:36:18.539505+0000","last_fullsized":"2023-10-09T06:36:18.539505+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:35:51.857807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.532714+0000","last_change":"2023-10-09T06:36:17.788967+0000","last_active":"2023-10-09T06:36:18.532714+0000","last_peered":"2023-10-09T06:36:18.532714+0000","last_clean":"2023-10-09T06:36:18.532714+0000","last_became_active":"2023-10-09T06:36:17.788685+0000","last_became_peered":"2023-10-09T06:36:17.788685+0000","last_unstale":"2023-10-09T06:36:18.532714+0000","last_undegraded":"2023-10-09T06:36:18.532714+0000","last_fullsized":"2023-10-09T06:36:18.532714+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:25:19.672180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.546037+0000","last_change":"2023-10-09T06:36:18.541923+0000","last_active":"2023-10-09T06:36:18.546037+0000","last_peered":"2023-10-09T06:36:18.546037+0000","last_clean":"2023-10-09T06:36:18.546037+0000","last_became_active":"2023-10-09T06:36:18.541630+0000","last_became_peered":"2023-10-09T06:36:18.541630+0000","last_unstale":"2023-10-09T06:36:18.546037+0000","last_undegraded":"2023-10-09T06:36:18.546037+0000","last_fullsized":"2023-10-09T06:36:18.546037+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:46:35.313608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.783556+0000","last_change":"2023-10-09T06:36:17.792603+0000","last_active":"2023-10-09T06:36:18.783556+0000","last_peered":"2023-10-09T06:36:18.783556+0000","last_clean":"2023-10-09T06:36:18.783556+0000","last_became_active":"2023-10-09T06:36:17.792288+0000","last_became_peered":"2023-10-09T06:36:17.792288+0000","last_unstale":"2023-10-09T06:36:18.783556+0000","last_undegraded":"2023-10-09T06:36:18.783556+0000","last_fullsized":"2023-10-09T06:36:18.783556+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:36:22.684838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.538476+0000","last_change":"2023-10-09T06:36:18.533485+0000","last_active":"2023-10-09T06:36:18.538476+0000","last_peered":"2023-10-09T06:36:18.538476+0000","last_clean":"2023-10-09T06:36:18.538476+0000","last_became_active":"2023-10-09T06:36:18.533150+0000","last_became_peered":"2023-10-09T06:36:18.533150+0000","last_unstale":"2023-10-09T06:36:18.538476+0000","last_undegraded":"2023-10-09T06:36:18.538476+0000","last_fullsized":"2023-10-09T06:36:18.538476+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:36:26.717698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.532318+0000","last_change":"2023-10-09T06:36:17.791125+0000","last_active":"2023-10-09T06:36:18.532318+0000","last_peered":"2023-10-09T06:36:18.532318+0000","last_clean":"2023-10-09T06:36:18.532318+0000","last_became_active":"2023-10-09T06:36:17.789317+0000","last_became_peered":"2023-10-09T06:36:17.789317+0000","last_unstale":"2023-10-09T06:36:18.532318+0000","last_undegraded":"2023-10-09T06:36:18.532318+0000","last_fullsized":"2023-10-09T06:36:18.532318+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:25:34.425692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.543420+0000","last_change":"2023-10-09T06:36:18.540876+0000","last_active":"2023-10-09T06:36:18.543420+0000","last_peered":"2023-10-09T06:36:18.543420+0000","last_clean":"2023-10-09T06:36:18.543420+0000","last_became_active":"2023-10-09T06:36:18.540636+0000","last_became_peered":"2023-10-09T06:36:18.540636+0000","last_unstale":"2023-10-09T06:36:18.543420+0000","last_undegraded":"2023-10-09T06:36:18.543420+0000","last_fullsized":"2023-10-09T06:36:18.543420+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:34:03.599407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.532332+0000","last_change":"2023-10-09T06:36:17.793201+0000","last_active":"2023-10-09T06:36:18.532332+0000","last_peered":"2023-10-09T06:36:18.532332+0000","last_clean":"2023-10-09T06:36:18.532332+0000","last_became_active":"2023-10-09T06:36:17.792955+0000","last_became_peered":"2023-10-09T06:36:17.792955+0000","last_unstale":"2023-10-09T06:36:18.532332+0000","last_undegraded":"2023-10-09T06:36:18.532332+0000","last_fullsized":"2023-10-09T06:36:18.532332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:15:58.450498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.539402+0000","last_change":"2023-10-09T06:36:18.535541+0000","last_active":"2023-10-09T06:36:18.539402+0000","last_peered":"2023-10-09T06:36:18.539402+0000","last_clean":"2023-10-09T06:36:18.539402+0000","last_became_active":"2023-10-09T06:36:18.535246+0000","last_became_peered":"2023-10-09T06:36:18.535246+0000","last_unstale":"2023-10-09T06:36:18.539402+0000","last_undegraded":"2023-10-09T06:36:18.539402+0000","last_fullsized":"2023-10-09T06:36:18.539402+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:25:20.676670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.526944+0000","last_change":"2023-10-09T06:36:17.784824+0000","last_active":"2023-10-09T06:36:18.526944+0000","last_peered":"2023-10-09T06:36:18.526944+0000","last_clean":"2023-10-09T06:36:18.526944+0000","last_became_active":"2023-10-09T06:36:17.784571+0000","last_became_peered":"2023-10-09T06:36:17.784571+0000","last_unstale":"2023-10-09T06:36:18.526944+0000","last_undegraded":"2023-10-09T06:36:18.526944+0000","last_fullsized":"2023-10-09T06:36:18.526944+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:02:15.826669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.541393+0000","last_change":"2023-10-09T06:36:18.534396+0000","last_active":"2023-10-09T06:36:18.541393+0000","last_peered":"2023-10-09T06:36:18.541393+0000","last_clean":"2023-10-09T06:36:18.541393+0000","last_became_active":"2023-10-09T06:36:18.533843+0000","last_became_peered":"2023-10-09T06:36:18.533843+0000","last_unstale":"2023-10-09T06:36:18.541393+0000","last_undegraded":"2023-10-09T06:36:18.541393+0000","last_fullsized":"2023-10-09T06:36:18.541393+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:58:18.392433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.532532+0000","last_change":"2023-10-09T06:36:17.790768+0000","last_active":"2023-10-09T06:36:18.532532+0000","last_peered":"2023-10-09T06:36:18.532532+0000","last_clean":"2023-10-09T06:36:18.532532+0000","last_became_active":"2023-10-09T06:36:17.790602+0000","last_became_peered":"2023-10-09T06:36:17.790602+0000","last_unstale":"2023-10-09T06:36:18.532532+0000","last_undegraded":"2023-10-09T06:36:18.532532+0000","last_fullsized":"2023-10-09T06:36:18.532532+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:45:49.355268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.542140+0000","last_change":"2023-10-09T06:36:18.540890+0000","last_active":"2023-10-09T06:36:18.542140+0000","last_peered":"2023-10-09T06:36:18.542140+0000","last_clean":"2023-10-09T06:36:18.542140+0000","last_became_active":"2023-10-09T06:36:18.540591+0000","last_became_peered":"2023-10-09T06:36:18.540591+0000","last_unstale":"2023-10-09T06:36:18.542140+0000","last_undegraded":"2023-10-09T06:36:18.542140+0000","last_fullsized":"2023-10-09T06:36:18.542140+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:32:10.068899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.526766+0000","last_change":"2023-10-09T06:36:17.786310+0000","last_active":"2023-10-09T06:36:18.526766+0000","last_peered":"2023-10-09T06:36:18.526766+0000","last_clean":"2023-10-09T06:36:18.526766+0000","last_became_active":"2023-10-09T06:36:17.785631+0000","last_became_peered":"2023-10-09T06:36:17.785631+0000","last_unstale":"2023-10-09T06:36:18.526766+0000","last_undegraded":"2023-10-09T06:36:18.526766+0000","last_fullsized":"2023-10-09T06:36:18.526766+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:44:49.435194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.537306+0000","last_change":"2023-10-09T06:36:18.533231+0000","last_active":"2023-10-09T06:36:18.537306+0000","last_peered":"2023-10-09T06:36:18.537306+0000","last_clean":"2023-10-09T06:36:18.537306+0000","last_became_active":"2023-10-09T06:36:18.532977+0000","last_became_peered":"2023-10-09T06:36:18.532977+0000","last_unstale":"2023-10-09T06:36:18.537306+0000","last_undegraded":"2023-10-09T06:36:18.537306+0000","last_fullsized":"2023-10-09T06:36:18.537306+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:11:55.258763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.533336+0000","last_change":"2023-10-09T06:36:17.785559+0000","last_active":"2023-10-09T06:36:18.533336+0000","last_peered":"2023-10-09T06:36:18.533336+0000","last_clean":"2023-10-09T06:36:18.533336+0000","last_became_active":"2023-10-09T06:36:17.785290+0000","last_became_peered":"2023-10-09T06:36:17.785290+0000","last_unstale":"2023-10-09T06:36:18.533336+0000","last_undegraded":"2023-10-09T06:36:18.533336+0000","last_fullsized":"2023-10-09T06:36:18.533336+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:25:19.672180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.539523+0000","last_change":"2023-10-09T06:36:18.536856+0000","last_active":"2023-10-09T06:36:18.539523+0000","last_peered":"2023-10-09T06:36:18.539523+0000","last_clean":"2023-10-09T06:36:18.539523+0000","last_became_active":"2023-10-09T06:36:18.536403+0000","last_became_peered":"2023-10-09T06:36:18.536403+0000","last_unstale":"2023-10-09T06:36:18.539523+0000","last_undegraded":"2023-10-09T06:36:18.539523+0000","last_fullsized":"2023-10-09T06:36:18.539523+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:45:50.359758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.524897+0000","last_change":"2023-10-09T06:36:17.788850+0000","last_active":"2023-10-09T06:36:18.524897+0000","last_peered":"2023-10-09T06:36:18.524897+0000","last_clean":"2023-10-09T06:36:18.524897+0000","last_became_active":"2023-10-09T06:36:17.788604+0000","last_became_peered":"2023-10-09T06:36:17.788604+0000","last_unstale":"2023-10-09T06:36:18.524897+0000","last_undegraded":"2023-10-09T06:36:18.524897+0000","last_fullsized":"2023-10-09T06:36:18.524897+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:05:55.338038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.540075+0000","last_change":"2023-10-09T06:36:18.536117+0000","last_active":"2023-10-09T06:36:18.540075+0000","last_peered":"2023-10-09T06:36:18.540075+0000","last_clean":"2023-10-09T06:36:18.540075+0000","last_became_active":"2023-10-09T06:36:18.534990+0000","last_became_peered":"2023-10-09T06:36:18.534990+0000","last_unstale":"2023-10-09T06:36:18.540075+0000","last_undegraded":"2023-10-09T06:36:18.540075+0000","last_fullsized":"2023-10-09T06:36:18.540075+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:29:08.981159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"88'1","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.796295+0000","last_change":"2023-10-09T06:36:17.786883+0000","last_active":"2023-10-09T06:36:18.796295+0000","last_peered":"2023-10-09T06:36:18.796295+0000","last_clean":"2023-10-09T06:36:18.796295+0000","last_became_active":"2023-10-09T06:36:17.786607+0000","last_became_peered":"2023-10-09T06:36:17.786607+0000","last_unstale":"2023-10-09T06:36:18.796295+0000","last_undegraded":"2023-10-09T06:36:18.796295+0000","last_fullsized":"2023-10-09T06:36:18.796295+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:34:02.594917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":221,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.535858+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:36:18.535858+0000","last_peered":"2023-10-09T06:36:18.535858+0000","last_clean":"2023-10-09T06:36:18.535858+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:36:18.535858+0000","last_undegraded":"2023-10-09T06:36:18.535858+0000","last_fullsized":"2023-10-09T06:36:18.535858+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574922,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298940,"kb_used_data":1652,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072900,"statfs":{"total":96636764160,"available":96330649600,"internally_reserved":0,"allocated":1691648,"data_stored":1060361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:33 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.50700000000000001,"15min":0.503},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.66300000000000003,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.34100000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65700000000000003,"5min":0.753,"15min":0.753},"last":0.68000000000000005}]},{"osd":1,"last update":"Mon Oct 9 06:35:33 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.50900000000000001,"15min":0.52100000000000002},"min":{"1min":0.36099999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.63700000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52300000000000002,"15min":0.52400000000000002},"min":{"1min":0.32900000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.66100000000000003,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.62}]},{"osd":2,"last update":"Mon Oct 9 06:35:33 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.52100000000000002,"15min":0.50900000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65700000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52900000000000003,"15min":0.54500000000000004},"min":{"1min":0.36799999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.69099999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.59699999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:35:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.53500000000000003,"15min":0.55300000000000005},"min":{"1min":0.36099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56399999999999995,"15min":0.59799999999999998},"min":{"1min":0.42299999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82099999999999995,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.66200000000000003}]},{"osd":4,"last update":"Mon Oct 9 06:35:33 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.48199999999999998,"15min":0.52400000000000002},"min":{"1min":0.307,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.69999999999999996,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56899999999999995,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.85999999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:35:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52700000000000002,"15min":0.57699999999999996},"min":{"1min":0.34399999999999997,"5min":0.307,"15min":0.307},"max":{"1min":0.85799999999999998,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58699999999999997,"15min":0.621},"min":{"1min":0.41699999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.74099999999999999,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.79300000000000004}]},{"osd":6,"last update":"Mon Oct 9 06:35:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55100000000000005,"15min":0.59799999999999998},"min":{"1min":0.378,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.78100000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55300000000000005,"15min":0.61799999999999999},"min":{"1min":0.35399999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.79900000000000004,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.77800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574921,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302468,"kb_used_data":1084,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069372,"statfs":{"total":96636764160,"available":96327036928,"internally_reserved":0,"allocated":1110016,"data_stored":476793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.495,"15min":0.501},"min":{"1min":0.35399999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.32200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.47599999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:35:36 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.495,"15min":0.51300000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.72499999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52600000000000002,"15min":0.53200000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.61399999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:35:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51000000000000001,"15min":0.51500000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69499999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.495},{"interface":"front","average":{"1min":0.5,"5min":0.51300000000000001,"15min":0.53300000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.67500000000000004,"5min":0.752,"15min":0.752},"last":0.44700000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:35:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53500000000000003,"15min":0.54800000000000004},"min":{"1min":0.40000000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70899999999999996,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59399999999999997,"15min":0.71799999999999997},"min":{"1min":0.372,"5min":0.37,"15min":0.37},"max":{"1min":0.73499999999999999,"5min":4.7510000000000003,"15min":4.7510000000000003},"last":0.41699999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:35:36 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46800000000000003,"15min":0.47999999999999998},"min":{"1min":0.29799999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.67500000000000004,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.57199999999999995,"15min":0.58499999999999996},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.76700000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.59599999999999997}]},{"osd":5,"last update":"Mon Oct 9 06:35:36 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.52700000000000002,"15min":0.53300000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.755,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.318},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57799999999999996,"15min":0.59999999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78000000000000003,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.51200000000000001}]},{"osd":7,"last update":"Mon Oct 9 06:35:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56499999999999995,"15min":0.59099999999999997},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.55700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302460,"kb_used_data":1076,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069380,"statfs":{"total":96636764160,"available":96327045120,"internally_reserved":0,"allocated":1101824,"data_stored":476169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.30399999999999999,"15min":0.308},"min":{"1min":0.20200000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"last":0.19700000000000001},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34899999999999998,"15min":0.35999999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.53600000000000003,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.26300000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34300000000000003,"15min":0.35099999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.251},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.38400000000000001,"15min":0.38600000000000001},"min":{"1min":0.29699999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.57499999999999996,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.33300000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.36099999999999999,"15min":0.36299999999999999},"min":{"1min":0.247,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.41299999999999998,"15min":0.57899999999999996},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.54800000000000004,"5min":5.8529999999999998,"15min":5.8529999999999998},"last":0.30399999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.42299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58199999999999996,"15min":0.58599999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.37}]},{"osd":5,"last update":"Mon Oct 9 06:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.62,"15min":0.77800000000000002},"min":{"1min":0.37,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.747,"5min":5.9459999999999997,"15min":5.9459999999999997},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60199999999999998,"15min":0.61199999999999999},"min":{"1min":0.47499999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.92700000000000005,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.443}]},{"osd":6,"last update":"Mon Oct 9 06:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.59599999999999997,"15min":0.748},"min":{"1min":0.39100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73799999999999999,"5min":5.9160000000000004,"15min":5.9160000000000004},"last":0.38},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.42699999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91000000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.35699999999999998}]},{"osd":7,"last update":"Mon Oct 9 06:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.89500000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.433},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.624,"15min":0.79000000000000004},"min":{"1min":0.40699999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80900000000000005,"5min":5.9729999999999999,"15min":5.9729999999999999},"last":0.40400000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303044,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068796,"statfs":{"total":96636764160,"available":96326447104,"internally_reserved":0,"allocated":1699840,"data_stored":1067091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.308,"15min":0.317},"min":{"1min":0.214,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.47599999999999998,"5min":0.498,"15min":0.498},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.33700000000000002,"15min":0.34699999999999998},"min":{"1min":0.23100000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.53600000000000003,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.25}]},{"osd":2,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.35299999999999998,"15min":0.36299999999999999},"min":{"1min":0.252,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38100000000000001,"15min":0.38900000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.56499999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.29499999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35599999999999998,"15min":0.39700000000000002},"min":{"1min":0.27300000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51200000000000001,"5min":1.554,"15min":1.554},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.378,"15min":0.40400000000000003},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.58099999999999996,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.311}]},{"osd":4,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.36599999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70199999999999996,"5min":1.212,"15min":1.212},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61199999999999999,"15min":0.75800000000000001},"min":{"1min":0.41799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82499999999999996,"5min":5.484,"15min":5.484},"last":0.376}]},{"osd":5,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.61499999999999999,"15min":0.76300000000000001},"min":{"1min":0.37,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.69999999999999996,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57299999999999995,"15min":0.57599999999999996},"min":{"1min":0.40899999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84499999999999997,"5min":1.113,"15min":1.113},"last":0.47599999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59499999999999997,"15min":0.60699999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.72899999999999998,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.42799999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83899999999999997,"5min":1.169,"15min":1.169},"last":0.45700000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.39700000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.85499999999999998,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.628,"15min":0.79000000000000004},"min":{"1min":0.42699999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.71299999999999997,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.53400000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298368,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073472,"statfs":{"total":96636764160,"available":96331235328,"internally_reserved":0,"allocated":1105920,"data_stored":476203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.32200000000000001,"15min":0.35999999999999999},"min":{"1min":0.218,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.50800000000000001,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.311},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.40100000000000002,"15min":0.60999999999999999},"min":{"1min":0.216,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.53800000000000003,"5min":7.2699999999999996,"15min":7.2699999999999996},"last":0.26100000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.32800000000000001,"15min":0.34899999999999998},"min":{"1min":0.22600000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.497,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.371,"5min":0.45100000000000001,"15min":0.66600000000000004},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.57599999999999996,"5min":7.327,"15min":7.327},"last":0.376}]},{"osd":3,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.376,"15min":0.39400000000000002},"min":{"1min":0.28299999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.52900000000000003,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.374,"15min":0.39500000000000002},"min":{"1min":0.25900000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.48399999999999999,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.35699999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53700000000000003,"15min":0.54100000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.73299999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.628,"15min":0.83299999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83199999999999996,"5min":7.2389999999999999,"15min":7.2389999999999999},"last":0.60899999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.69899999999999995,"15min":0.98999999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.747,"5min":10.890000000000001,"15min":10.890000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59099999999999997,"15min":0.60199999999999998},"min":{"1min":0.39800000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.78200000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.84499999999999997}]},{"osd":6,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.64300000000000002,"15min":0.84099999999999997},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77300000000000002,"5min":7.2999999999999998,"15min":7.2999999999999998},"last":0.753},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57199999999999995,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77000000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.623}]},{"osd":7,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.67200000000000004,"15min":0.873},"min":{"1min":0.38600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.79900000000000004,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.61599999999999999,"15min":0.628},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79300000000000004,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.57799999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574923,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298368,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073472,"statfs":{"total":96636764160,"available":96331235328,"internally_reserved":0,"allocated":1105920,"data_stored":471439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.30199999999999999,"5min":0.35699999999999998,"15min":0.32700000000000001},"min":{"1min":0.23599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.38300000000000001,"5min":5.8700000000000001,"15min":5.8700000000000001},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.38600000000000001,"15min":0.36599999999999999},"min":{"1min":0.224,"5min":0.222,"15min":0.222},"max":{"1min":0.46200000000000002,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.38400000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.375,"15min":0.35699999999999998},"min":{"1min":0.23899999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.47699999999999998,"5min":4.0469999999999997,"15min":4.0469999999999997},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40000000000000002,"15min":0.38700000000000001},"min":{"1min":0.27100000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.48699999999999999,"5min":4.1509999999999998,"15min":4.1509999999999998},"last":0.36099999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.373,"15min":0.374},"min":{"1min":0.24399999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.499,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.38900000000000001,"15min":0.38500000000000001},"min":{"1min":0.26200000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.51500000000000001,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.375}]},{"osd":4,"last update":"Mon Oct 9 06:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58099999999999996,"15min":0.57099999999999995},"min":{"1min":0.372,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80100000000000005,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57299999999999995,"15min":0.56200000000000006},"min":{"1min":0.36599999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68899999999999995,"5min":3.0710000000000002,"15min":3.0710000000000002},"last":0.42199999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60999999999999999,"15min":0.58599999999999997},"min":{"1min":0.38200000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.746,"5min":6.2300000000000004,"15min":6.2300000000000004},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.60199999999999998,"15min":0.59199999999999997},"min":{"1min":0.41499999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72299999999999998,"5min":3.0219999999999998,"15min":3.0219999999999998},"last":0.51700000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.60599999999999998,"15min":0.58899999999999997},"min":{"1min":0.42899999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.71199999999999997,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.504},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.67900000000000005,"15min":0.64800000000000002},"min":{"1min":0.46400000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.79300000000000004,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.52500000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63100000000000001,"15min":0.60999999999999999},"min":{"1min":0.496,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77900000000000003,"5min":4.2629999999999999,"15min":4.2629999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.64700000000000002,"15min":0.61099999999999999},"min":{"1min":0.40500000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.73599999999999999,"5min":7.3650000000000002,"15min":7.3650000000000002},"last":0.53600000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574922,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298384,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073456,"statfs":{"total":96636764160,"available":96331218944,"internally_reserved":0,"allocated":1122304,"data_stored":477829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.53200000000000003,"15min":0.56000000000000005},"min":{"1min":0.34499999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.63900000000000001,"5min":1.083,"15min":1.083},"last":0.621},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.33300000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.97799999999999998,"5min":1.054,"15min":1.054},"last":0.61099999999999999}]},{"osd":1,"last update":"Mon Oct 9 06:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.65700000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51100000000000001,"15min":0.495},"min":{"1min":0.42099999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.62,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.63300000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52600000000000002,"15min":0.54200000000000004},"min":{"1min":0.35299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71499999999999997,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.57099999999999995,"15min":0.59899999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.65100000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.55900000000000005}]},{"osd":3,"last update":"Mon Oct 9 06:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54000000000000004,"15min":0.55700000000000005},"min":{"1min":0.39200000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.66700000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57599999999999996,"15min":0.59099999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.78500000000000003,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.51400000000000001}]},{"osd":5,"last update":"Mon Oct 9 06:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52300000000000002,"15min":0.56100000000000005},"min":{"1min":0.36199999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.76500000000000001,"5min":0.998,"15min":0.998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58799999999999997,"15min":0.61399999999999999},"min":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.751,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.59299999999999997}]},{"osd":6,"last update":"Mon Oct 9 06:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.56999999999999995},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57399999999999995,"15min":0.60199999999999998},"min":{"1min":0.41499999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.67800000000000005}]},{"osd":7,"last update":"Mon Oct 9 06:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54900000000000004,"15min":0.57499999999999996},"min":{"1min":0.33600000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.83199999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60899999999999999,"15min":0.65100000000000002},"min":{"1min":0.40500000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98999999999999999,"5min":1.175,"15min":1.175},"last":0.66200000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574922,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298368,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073472,"statfs":{"total":96636764160,"available":96331235328,"internally_reserved":0,"allocated":1105920,"data_stored":476239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.498,"15min":0.50800000000000001},"min":{"1min":0.30599999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.60499999999999998,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.314},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.51300000000000001,"15min":0.51200000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.621,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.60899999999999999}]},{"osd":1,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.50900000000000001,"15min":0.51000000000000001},"min":{"1min":0.33000000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.67500000000000004,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.52100000000000002,"15min":0.53100000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64700000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.36599999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54600000000000004,"15min":0.54400000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.69399999999999995,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.495},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58099999999999996,"15min":0.58799999999999997},"min":{"1min":0.40999999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.71699999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.55800000000000005}]},{"osd":3,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53900000000000003,"15min":0.54200000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.66100000000000003,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55600000000000005,"15min":0.56399999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.69999999999999996,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.59099999999999997}]},{"osd":4,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.49399999999999999,"15min":0.496},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68300000000000005,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.71999999999999997,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.68700000000000006}]},{"osd":6,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.36099999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72099999999999997,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.625},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.55500000000000005,"15min":0.55800000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73799999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.66300000000000003}]},{"osd":7,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78100000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.35199999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.84499999999999997,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.81299999999999994}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:36:22.839 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:36:22.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.987+0000 7f202f177700 1 Processor -- start 2023-10-09T06:36:22.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.987+0000 7f202f177700 1 -- start start 2023-10-09T06:36:22.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.988+0000 7f202f177700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2028069560 0x7f20280699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:22.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.988+0000 7f202f177700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2028069ee0 con 0x7f2028159890 2023-10-09T06:36:22.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.988+0000 7f202f177700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f202815a160 con 0x7f2028159c60 2023-10-09T06:36:22.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.988+0000 7f202f177700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20280d13a0 con 0x7f2028069560 2023-10-09T06:36:22.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.988+0000 7f202d714700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2028069560 0x7f20280699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:22.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.988+0000 7f202cf13700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f2028159890 0x7f202815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33648/0 (socket says 172.21.15.163:33648) 2023-10-09T06:36:22.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.988+0000 7f202cf13700 1 -- 172.21.15.163:0/1238166235 learned_addr learned my addr 172.21.15.163:0/1238166235 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:22.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.988+0000 7f202d714700 1 -- 172.21.15.163:0/1238166235 >> v1:172.21.15.181:6789/0 conn(0x7f2028159c60 legacy=0x7f202815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:22.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.988+0000 7f202d714700 1 -- 172.21.15.163:0/1238166235 >> v1:172.21.15.163:6789/0 conn(0x7f2028159890 legacy=0x7f202815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:22.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.988+0000 7f202d714700 1 -- 172.21.15.163:0/1238166235 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202815e140 con 0x7f2028069560 2023-10-09T06:36:22.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.988+0000 7f201f7fe700 1 -- 172.21.15.163:0/1238166235 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1851884772 0 0) 0x7f2028069ee0 con 0x7f2028159890 2023-10-09T06:36:22.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.988+0000 7f202d714700 1 --2- 172.21.15.163:0/1238166235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2028069560 0x7f20280699a0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f20200098e0 tx=0x7f20200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e409b652e6fb7c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:22.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.988+0000 7f201f7fe700 1 -- 172.21.15.163:0/1238166235 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2020009e80 con 0x7f2028069560 2023-10-09T06:36:22.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.988+0000 7f201f7fe700 1 -- 172.21.15.163:0/1238166235 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202000b820 con 0x7f2028069560 2023-10-09T06:36:22.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.988+0000 7f201f7fe700 1 -- 172.21.15.163:0/1238166235 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202000bba0 con 0x7f2028069560 2023-10-09T06:36:22.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.989+0000 7f202f177700 1 -- 172.21.15.163:0/1238166235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2028069560 msgr2=0x7f20280699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:22.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.989+0000 7f202f177700 1 --2- 172.21.15.163:0/1238166235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2028069560 0x7f20280699a0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f20200098e0 tx=0x7f20200091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:22.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.989+0000 7f202f177700 1 -- 172.21.15.163:0/1238166235 shutdown_connections 2023-10-09T06:36:22.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.989+0000 7f202f177700 1 --2- 172.21.15.163:0/1238166235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2028069560 0x7f20280699a0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:22.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.989+0000 7f202f177700 1 -- 172.21.15.163:0/1238166235 >> 172.21.15.163:0/1238166235 conn(0x7f2028068e20 msgr2=0x7f20280d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:22.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.989+0000 7f202f177700 1 -- 172.21.15.163:0/1238166235 shutdown_connections 2023-10-09T06:36:22.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.989+0000 7f202f177700 1 -- 172.21.15.163:0/1238166235 wait complete. 2023-10-09T06:36:22.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.990+0000 7f202f177700 1 Processor -- start 2023-10-09T06:36:22.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.990+0000 7f202f177700 1 -- start start 2023-10-09T06:36:22.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.991+0000 7f202f177700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2028069560 0x7f20281534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:22.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.991+0000 7f202f177700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2028159890 0x7f20280b5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:22.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.991+0000 7f202f177700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2028159c60 0x7f2028154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:22.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.991+0000 7f202f177700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20280cd0c0 con 0x7f2028159890 2023-10-09T06:36:22.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.991+0000 7f202f177700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20280cd960 con 0x7f2028069560 2023-10-09T06:36:22.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.991+0000 7f202f177700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2028153a30 con 0x7f2028159c60 2023-10-09T06:36:22.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.991+0000 7f201ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2028159890 0x7f20280b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:22.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.991+0000 7f201ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2028159890 0x7f20280b5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:54918/0 (socket says 172.21.15.163:54918) 2023-10-09T06:36:22.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.991+0000 7f201ffff700 1 -- 172.21.15.163:0/1275833771 learned_addr learned my addr 172.21.15.163:0/1275833771 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:22.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.991+0000 7f202cf13700 1 --2- 172.21.15.163:0/1275833771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2028069560 0x7f20281534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:22.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.991+0000 7f202d714700 1 --2- 172.21.15.163:0/1275833771 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2028159c60 0x7f2028154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:22.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.991+0000 7f201ffff700 1 -- 172.21.15.163:0/1275833771 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2028159c60 msgr2=0x7f2028154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:22.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.991+0000 7f201ffff700 1 --2- 172.21.15.163:0/1275833771 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2028159c60 0x7f2028154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:22.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.991+0000 7f201ffff700 1 -- 172.21.15.163:0/1275833771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2028069560 msgr2=0x7f20281534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:22.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.991+0000 7f201ffff700 1 --2- 172.21.15.163:0/1275833771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2028069560 0x7f20281534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:22.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.991+0000 7f201ffff700 1 -- 172.21.15.163:0/1275833771 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2020009420 con 0x7f2028159890 2023-10-09T06:36:22.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.991+0000 7f202d714700 1 --2- 172.21.15.163:0/1275833771 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2028159c60 0x7f2028154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:36:22.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.991+0000 7f202cf13700 1 --2- 172.21.15.163:0/1275833771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2028069560 0x7f20281534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:36:22.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.991+0000 7f201ffff700 1 --2- 172.21.15.163:0/1275833771 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2028159890 0x7f20280b5980 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f201800b610 tx=0x7f2018009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:22.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.992+0000 7f201cff9700 1 -- 172.21.15.163:0/1275833771 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201800f510 con 0x7f2028159890 2023-10-09T06:36:22.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.992+0000 7f201cff9700 1 -- 172.21.15.163:0/1275833771 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201800f670 con 0x7f2028159890 2023-10-09T06:36:22.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.992+0000 7f202f177700 1 -- 172.21.15.163:0/1275833771 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20280b62c0 con 0x7f2028159890 2023-10-09T06:36:22.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.992+0000 7f201cff9700 1 -- 172.21.15.163:0/1275833771 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201800fa30 con 0x7f2028159890 2023-10-09T06:36:22.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.992+0000 7f202f177700 1 -- 172.21.15.163:0/1275833771 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20280cd5f0 con 0x7f2028159890 2023-10-09T06:36:22.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.993+0000 7f202f177700 1 -- 172.21.15.163:0/1275833771 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20280cdad0 con 0x7f2028159890 2023-10-09T06:36:22.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.993+0000 7f201cff9700 1 -- 172.21.15.163:0/1275833771 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f201801a400 con 0x7f2028159890 2023-10-09T06:36:22.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.993+0000 7f201cff9700 1 --2- 172.21.15.163:0/1275833771 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2014067200 0x7f20140696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:22.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.993+0000 7f201cff9700 1 -- 172.21.15.163:0/1275833771 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f20180844b0 con 0x7f2028159890 2023-10-09T06:36:22.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.994+0000 7f202cf13700 1 --2- 172.21.15.163:0/1275833771 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2014067200 0x7f20140696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:22.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.994+0000 7f202cf13700 1 --2- 172.21.15.163:0/1275833771 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2014067200 0x7f20140696b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f202400d270 tx=0x7f2024009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:22.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:22.997+0000 7f201cff9700 1 -- 172.21.15.163:0/1275833771 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2018051450 con 0x7f2028159890 2023-10-09T06:36:23.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.174+0000 7f202f177700 1 -- 172.21.15.163:0/1275833771 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f202804c3c0 con 0x7f2014067200 2023-10-09T06:36:23.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.182+0000 7f201cff9700 1 -- 172.21.15.163:0/1275833771 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289479 (secure 0 0 0) 0x7f202804c3c0 con 0x7f2014067200 2023-10-09T06:36:23.185 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:23.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.185+0000 7f202f177700 1 -- 172.21.15.163:0/1275833771 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2014067200 msgr2=0x7f20140696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:23.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.185+0000 7f202f177700 1 --2- 172.21.15.163:0/1275833771 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2014067200 0x7f20140696b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f202400d270 tx=0x7f2024009040 comp rx=0 tx=0).stop 2023-10-09T06:36:23.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.185+0000 7f202f177700 1 -- 172.21.15.163:0/1275833771 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2028159890 msgr2=0x7f20280b5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:23.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.185+0000 7f202f177700 1 --2- 172.21.15.163:0/1275833771 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2028159890 0x7f20280b5980 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f201800b610 tx=0x7f2018009120 comp rx=0 tx=0).stop 2023-10-09T06:36:23.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.185+0000 7f202f177700 1 -- 172.21.15.163:0/1275833771 shutdown_connections 2023-10-09T06:36:23.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.185+0000 7f202f177700 1 --2- 172.21.15.163:0/1275833771 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2014067200 0x7f20140696b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:23.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.185+0000 7f202f177700 1 --2- 172.21.15.163:0/1275833771 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2028159c60 0x7f2028154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:23.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.185+0000 7f202f177700 1 --2- 172.21.15.163:0/1275833771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2028069560 0x7f20281534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:23.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.185+0000 7f202f177700 1 --2- 172.21.15.163:0/1275833771 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2028159890 0x7f20280b5980 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:23.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.185+0000 7f202f177700 1 -- 172.21.15.163:0/1275833771 >> 172.21.15.163:0/1275833771 conn(0x7f2028068e20 msgr2=0x7f20280d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:23.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.186+0000 7f202f177700 1 -- 172.21.15.163:0/1275833771 shutdown_connections 2023-10-09T06:36:23.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.186+0000 7f202f177700 1 -- 172.21.15.163:0/1275833771 wait complete. 2023-10-09T06:36:23.188 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:36:23.197 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":258,"stamp":"2023-10-09T06:36:22.382567+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":97,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2400400,"kb_used_data":9808,"kb_used_omap":0,"kb_used_meta":2390528,"kb_avail":752574320,"statfs":{"total":773094113280,"available":770636103680,"internally_reserved":0,"allocated":10043392,"data_stored":4982124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2447900672},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.614481"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.546092+0000","last_change":"2023-10-09T06:36:18.541429+0000","last_active":"2023-10-09T06:36:18.546092+0000","last_peered":"2023-10-09T06:36:18.546092+0000","last_clean":"2023-10-09T06:36:18.546092+0000","last_became_active":"2023-10-09T06:36:18.541161+0000","last_became_peered":"2023-10-09T06:36:18.541161+0000","last_unstale":"2023-10-09T06:36:18.546092+0000","last_undegraded":"2023-10-09T06:36:18.546092+0000","last_fullsized":"2023-10-09T06:36:18.546092+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:25:20.676670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.543457+0000","last_change":"2023-10-09T06:36:18.540749+0000","last_active":"2023-10-09T06:36:18.543457+0000","last_peered":"2023-10-09T06:36:18.543457+0000","last_clean":"2023-10-09T06:36:18.543457+0000","last_became_active":"2023-10-09T06:36:18.540490+0000","last_became_peered":"2023-10-09T06:36:18.540490+0000","last_unstale":"2023-10-09T06:36:18.543457+0000","last_undegraded":"2023-10-09T06:36:18.543457+0000","last_fullsized":"2023-10-09T06:36:18.543457+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:50:33.268079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.546205+0000","last_change":"2023-10-09T06:36:18.541957+0000","last_active":"2023-10-09T06:36:18.546205+0000","last_peered":"2023-10-09T06:36:18.546205+0000","last_clean":"2023-10-09T06:36:18.546205+0000","last_became_active":"2023-10-09T06:36:18.541687+0000","last_became_peered":"2023-10-09T06:36:18.541687+0000","last_unstale":"2023-10-09T06:36:18.546205+0000","last_undegraded":"2023-10-09T06:36:18.546205+0000","last_fullsized":"2023-10-09T06:36:18.546205+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:45:50.359758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.539263+0000","last_change":"2023-10-09T06:36:18.534478+0000","last_active":"2023-10-09T06:36:18.539263+0000","last_peered":"2023-10-09T06:36:18.539263+0000","last_clean":"2023-10-09T06:36:18.539263+0000","last_became_active":"2023-10-09T06:36:18.534225+0000","last_became_peered":"2023-10-09T06:36:18.534225+0000","last_unstale":"2023-10-09T06:36:18.539263+0000","last_undegraded":"2023-10-09T06:36:18.539263+0000","last_fullsized":"2023-10-09T06:36:18.539263+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:38:02.641521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.539419+0000","last_change":"2023-10-09T06:36:18.534316+0000","last_active":"2023-10-09T06:36:18.539419+0000","last_peered":"2023-10-09T06:36:18.539419+0000","last_clean":"2023-10-09T06:36:18.539419+0000","last_became_active":"2023-10-09T06:36:18.534035+0000","last_became_peered":"2023-10-09T06:36:18.534035+0000","last_unstale":"2023-10-09T06:36:18.539419+0000","last_undegraded":"2023-10-09T06:36:18.539419+0000","last_fullsized":"2023-10-09T06:36:18.539419+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:15:59.454989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.547827+0000","last_change":"2023-10-09T06:36:18.545526+0000","last_active":"2023-10-09T06:36:18.547827+0000","last_peered":"2023-10-09T06:36:18.547827+0000","last_clean":"2023-10-09T06:36:18.547827+0000","last_became_active":"2023-10-09T06:36:18.545191+0000","last_became_peered":"2023-10-09T06:36:18.545191+0000","last_unstale":"2023-10-09T06:36:18.547827+0000","last_undegraded":"2023-10-09T06:36:18.547827+0000","last_fullsized":"2023-10-09T06:36:18.547827+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:25:40.992746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.544355+0000","last_change":"2023-10-09T06:36:18.543171+0000","last_active":"2023-10-09T06:36:18.544355+0000","last_peered":"2023-10-09T06:36:18.544355+0000","last_clean":"2023-10-09T06:36:18.544355+0000","last_became_active":"2023-10-09T06:36:18.542688+0000","last_became_peered":"2023-10-09T06:36:18.542688+0000","last_unstale":"2023-10-09T06:36:18.544355+0000","last_undegraded":"2023-10-09T06:36:18.544355+0000","last_fullsized":"2023-10-09T06:36:18.544355+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:46:35.313608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.546016+0000","last_change":"2023-10-09T06:36:18.541168+0000","last_active":"2023-10-09T06:36:18.546016+0000","last_peered":"2023-10-09T06:36:18.546016+0000","last_clean":"2023-10-09T06:36:18.546016+0000","last_became_active":"2023-10-09T06:36:18.540929+0000","last_became_peered":"2023-10-09T06:36:18.540929+0000","last_unstale":"2023-10-09T06:36:18.546016+0000","last_undegraded":"2023-10-09T06:36:18.546016+0000","last_fullsized":"2023-10-09T06:36:18.546016+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:38:02.641521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.540321+0000","last_change":"2023-10-09T06:36:18.538212+0000","last_active":"2023-10-09T06:36:18.540321+0000","last_peered":"2023-10-09T06:36:18.540321+0000","last_clean":"2023-10-09T06:36:18.540321+0000","last_became_active":"2023-10-09T06:36:18.537861+0000","last_became_peered":"2023-10-09T06:36:18.537861+0000","last_unstale":"2023-10-09T06:36:18.540321+0000","last_undegraded":"2023-10-09T06:36:18.540321+0000","last_fullsized":"2023-10-09T06:36:18.540321+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:34:03.599407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.539212+0000","last_change":"2023-10-09T06:36:18.534506+0000","last_active":"2023-10-09T06:36:18.539212+0000","last_peered":"2023-10-09T06:36:18.539212+0000","last_clean":"2023-10-09T06:36:18.539212+0000","last_became_active":"2023-10-09T06:36:18.534169+0000","last_became_peered":"2023-10-09T06:36:18.534169+0000","last_unstale":"2023-10-09T06:36:18.539212+0000","last_undegraded":"2023-10-09T06:36:18.539212+0000","last_fullsized":"2023-10-09T06:36:18.539212+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:37:50.129692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.545696+0000","last_change":"2023-10-09T06:36:18.540876+0000","last_active":"2023-10-09T06:36:18.545696+0000","last_peered":"2023-10-09T06:36:18.545696+0000","last_clean":"2023-10-09T06:36:18.545696+0000","last_became_active":"2023-10-09T06:36:18.540616+0000","last_became_peered":"2023-10-09T06:36:18.540616+0000","last_unstale":"2023-10-09T06:36:18.545696+0000","last_undegraded":"2023-10-09T06:36:18.545696+0000","last_fullsized":"2023-10-09T06:36:18.545696+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:58:03.063521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.540857+0000","last_change":"2023-10-09T06:36:18.536276+0000","last_active":"2023-10-09T06:36:18.540857+0000","last_peered":"2023-10-09T06:36:18.540857+0000","last_clean":"2023-10-09T06:36:18.540857+0000","last_became_active":"2023-10-09T06:36:18.535997+0000","last_became_peered":"2023-10-09T06:36:18.535997+0000","last_unstale":"2023-10-09T06:36:18.540857+0000","last_undegraded":"2023-10-09T06:36:18.540857+0000","last_fullsized":"2023-10-09T06:36:18.540857+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:48:11.177371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.546105+0000","last_change":"2023-10-09T06:36:18.539474+0000","last_active":"2023-10-09T06:36:18.546105+0000","last_peered":"2023-10-09T06:36:18.546105+0000","last_clean":"2023-10-09T06:36:18.546105+0000","last_became_active":"2023-10-09T06:36:18.539176+0000","last_became_peered":"2023-10-09T06:36:18.539176+0000","last_unstale":"2023-10-09T06:36:18.546105+0000","last_undegraded":"2023-10-09T06:36:18.546105+0000","last_fullsized":"2023-10-09T06:36:18.546105+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:47:18.417758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.537602+0000","last_change":"2023-10-09T06:36:18.529730+0000","last_active":"2023-10-09T06:36:18.537602+0000","last_peered":"2023-10-09T06:36:18.537602+0000","last_clean":"2023-10-09T06:36:18.537602+0000","last_became_active":"2023-10-09T06:36:18.529321+0000","last_became_peered":"2023-10-09T06:36:18.529321+0000","last_unstale":"2023-10-09T06:36:18.537602+0000","last_undegraded":"2023-10-09T06:36:18.537602+0000","last_fullsized":"2023-10-09T06:36:18.537602+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:14:27.092915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.543405+0000","last_change":"2023-10-09T06:36:18.535984+0000","last_active":"2023-10-09T06:36:18.543405+0000","last_peered":"2023-10-09T06:36:18.543405+0000","last_clean":"2023-10-09T06:36:18.543405+0000","last_became_active":"2023-10-09T06:36:18.535728+0000","last_became_peered":"2023-10-09T06:36:18.535728+0000","last_unstale":"2023-10-09T06:36:18.543405+0000","last_undegraded":"2023-10-09T06:36:18.543405+0000","last_fullsized":"2023-10-09T06:36:18.543405+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:46:35.313608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.547718+0000","last_change":"2023-10-09T06:36:18.544591+0000","last_active":"2023-10-09T06:36:18.547718+0000","last_peered":"2023-10-09T06:36:18.547718+0000","last_clean":"2023-10-09T06:36:18.547718+0000","last_became_active":"2023-10-09T06:36:18.544091+0000","last_became_peered":"2023-10-09T06:36:18.544091+0000","last_unstale":"2023-10-09T06:36:18.547718+0000","last_undegraded":"2023-10-09T06:36:18.547718+0000","last_fullsized":"2023-10-09T06:36:18.547718+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:35:52.976193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.541701+0000","last_change":"2023-10-09T06:36:18.537299+0000","last_active":"2023-10-09T06:36:18.541701+0000","last_peered":"2023-10-09T06:36:18.541701+0000","last_clean":"2023-10-09T06:36:18.541701+0000","last_became_active":"2023-10-09T06:36:18.537051+0000","last_became_peered":"2023-10-09T06:36:18.537051+0000","last_unstale":"2023-10-09T06:36:18.541701+0000","last_undegraded":"2023-10-09T06:36:18.541701+0000","last_fullsized":"2023-10-09T06:36:18.541701+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:13:52.813781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.538831+0000","last_change":"2023-10-09T06:36:18.534109+0000","last_active":"2023-10-09T06:36:18.538831+0000","last_peered":"2023-10-09T06:36:18.538831+0000","last_clean":"2023-10-09T06:36:18.538831+0000","last_became_active":"2023-10-09T06:36:18.533797+0000","last_became_peered":"2023-10-09T06:36:18.533797+0000","last_unstale":"2023-10-09T06:36:18.538831+0000","last_undegraded":"2023-10-09T06:36:18.538831+0000","last_fullsized":"2023-10-09T06:36:18.538831+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:53:53.083376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.537158+0000","last_change":"2023-10-09T06:36:18.534001+0000","last_active":"2023-10-09T06:36:18.537158+0000","last_peered":"2023-10-09T06:36:18.537158+0000","last_clean":"2023-10-09T06:36:18.537158+0000","last_became_active":"2023-10-09T06:36:18.533751+0000","last_became_peered":"2023-10-09T06:36:18.533751+0000","last_unstale":"2023-10-09T06:36:18.537158+0000","last_undegraded":"2023-10-09T06:36:18.537158+0000","last_fullsized":"2023-10-09T06:36:18.537158+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:31:48.056171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.540878+0000","last_change":"2023-10-09T06:36:18.538080+0000","last_active":"2023-10-09T06:36:18.540878+0000","last_peered":"2023-10-09T06:36:18.540878+0000","last_clean":"2023-10-09T06:36:18.540878+0000","last_became_active":"2023-10-09T06:36:18.537790+0000","last_became_peered":"2023-10-09T06:36:18.537790+0000","last_unstale":"2023-10-09T06:36:18.540878+0000","last_undegraded":"2023-10-09T06:36:18.540878+0000","last_fullsized":"2023-10-09T06:36:18.540878+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:36:26.717698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.543644+0000","last_change":"2023-10-09T06:36:18.541076+0000","last_active":"2023-10-09T06:36:18.543644+0000","last_peered":"2023-10-09T06:36:18.543644+0000","last_clean":"2023-10-09T06:36:18.543644+0000","last_became_active":"2023-10-09T06:36:18.540802+0000","last_became_peered":"2023-10-09T06:36:18.540802+0000","last_unstale":"2023-10-09T06:36:18.543644+0000","last_undegraded":"2023-10-09T06:36:18.543644+0000","last_fullsized":"2023-10-09T06:36:18.543644+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:17:02.758096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.546050+0000","last_change":"2023-10-09T06:36:18.542358+0000","last_active":"2023-10-09T06:36:18.546050+0000","last_peered":"2023-10-09T06:36:18.546050+0000","last_clean":"2023-10-09T06:36:18.546050+0000","last_became_active":"2023-10-09T06:36:18.542090+0000","last_became_peered":"2023-10-09T06:36:18.542090+0000","last_unstale":"2023-10-09T06:36:18.546050+0000","last_undegraded":"2023-10-09T06:36:18.546050+0000","last_fullsized":"2023-10-09T06:36:18.546050+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:25:35.071015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.543855+0000","last_change":"2023-10-09T06:36:18.540804+0000","last_active":"2023-10-09T06:36:18.543855+0000","last_peered":"2023-10-09T06:36:18.543855+0000","last_clean":"2023-10-09T06:36:18.543855+0000","last_became_active":"2023-10-09T06:36:18.540557+0000","last_became_peered":"2023-10-09T06:36:18.540557+0000","last_unstale":"2023-10-09T06:36:18.543855+0000","last_undegraded":"2023-10-09T06:36:18.543855+0000","last_fullsized":"2023-10-09T06:36:18.543855+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:35:52.976193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.544734+0000","last_change":"2023-10-09T06:36:18.543081+0000","last_active":"2023-10-09T06:36:18.544734+0000","last_peered":"2023-10-09T06:36:18.544734+0000","last_clean":"2023-10-09T06:36:18.544734+0000","last_became_active":"2023-10-09T06:36:18.542799+0000","last_became_peered":"2023-10-09T06:36:18.542799+0000","last_unstale":"2023-10-09T06:36:18.544734+0000","last_undegraded":"2023-10-09T06:36:18.544734+0000","last_fullsized":"2023-10-09T06:36:18.544734+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:50:33.268079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.539540+0000","last_change":"2023-10-09T06:36:18.536990+0000","last_active":"2023-10-09T06:36:18.539540+0000","last_peered":"2023-10-09T06:36:18.539540+0000","last_clean":"2023-10-09T06:36:18.539540+0000","last_became_active":"2023-10-09T06:36:18.536697+0000","last_became_peered":"2023-10-09T06:36:18.536697+0000","last_unstale":"2023-10-09T06:36:18.539540+0000","last_undegraded":"2023-10-09T06:36:18.539540+0000","last_fullsized":"2023-10-09T06:36:18.539540+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:36:26.717698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.543419+0000","last_change":"2023-10-09T06:36:18.539886+0000","last_active":"2023-10-09T06:36:18.543419+0000","last_peered":"2023-10-09T06:36:18.543419+0000","last_clean":"2023-10-09T06:36:18.543419+0000","last_became_active":"2023-10-09T06:36:18.539643+0000","last_became_peered":"2023-10-09T06:36:18.539643+0000","last_unstale":"2023-10-09T06:36:18.543419+0000","last_undegraded":"2023-10-09T06:36:18.543419+0000","last_fullsized":"2023-10-09T06:36:18.543419+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:18:46.793953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.547711+0000","last_change":"2023-10-09T06:36:18.544727+0000","last_active":"2023-10-09T06:36:18.547711+0000","last_peered":"2023-10-09T06:36:18.547711+0000","last_clean":"2023-10-09T06:36:18.547711+0000","last_became_active":"2023-10-09T06:36:18.543753+0000","last_became_peered":"2023-10-09T06:36:18.543753+0000","last_unstale":"2023-10-09T06:36:18.547711+0000","last_undegraded":"2023-10-09T06:36:18.547711+0000","last_fullsized":"2023-10-09T06:36:18.547711+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:47:18.417758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.544139+0000","last_change":"2023-10-09T06:36:18.538657+0000","last_active":"2023-10-09T06:36:18.544139+0000","last_peered":"2023-10-09T06:36:18.544139+0000","last_clean":"2023-10-09T06:36:18.544139+0000","last_became_active":"2023-10-09T06:36:18.538151+0000","last_became_peered":"2023-10-09T06:36:18.538151+0000","last_unstale":"2023-10-09T06:36:18.544139+0000","last_undegraded":"2023-10-09T06:36:18.544139+0000","last_fullsized":"2023-10-09T06:36:18.544139+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:32:10.068899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.541358+0000","last_change":"2023-10-09T06:36:18.536121+0000","last_active":"2023-10-09T06:36:18.541358+0000","last_peered":"2023-10-09T06:36:18.541358+0000","last_clean":"2023-10-09T06:36:18.541358+0000","last_became_active":"2023-10-09T06:36:18.535859+0000","last_became_peered":"2023-10-09T06:36:18.535859+0000","last_unstale":"2023-10-09T06:36:18.541358+0000","last_undegraded":"2023-10-09T06:36:18.541358+0000","last_fullsized":"2023-10-09T06:36:18.541358+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:25:40.992746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.541414+0000","last_change":"2023-10-09T06:36:18.534555+0000","last_active":"2023-10-09T06:36:18.541414+0000","last_peered":"2023-10-09T06:36:18.541414+0000","last_clean":"2023-10-09T06:36:18.541414+0000","last_became_active":"2023-10-09T06:36:18.534157+0000","last_became_peered":"2023-10-09T06:36:18.534157+0000","last_unstale":"2023-10-09T06:36:18.541414+0000","last_undegraded":"2023-10-09T06:36:18.541414+0000","last_fullsized":"2023-10-09T06:36:18.541414+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:58:12.693109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.546146+0000","last_change":"2023-10-09T06:36:18.539980+0000","last_active":"2023-10-09T06:36:18.546146+0000","last_peered":"2023-10-09T06:36:18.546146+0000","last_clean":"2023-10-09T06:36:18.546146+0000","last_became_active":"2023-10-09T06:36:18.539733+0000","last_became_peered":"2023-10-09T06:36:18.539733+0000","last_unstale":"2023-10-09T06:36:18.546146+0000","last_undegraded":"2023-10-09T06:36:18.546146+0000","last_fullsized":"2023-10-09T06:36:18.546146+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:18:46.793953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.541005+0000","last_change":"2023-10-09T06:36:18.535952+0000","last_active":"2023-10-09T06:36:18.541005+0000","last_peered":"2023-10-09T06:36:18.541005+0000","last_clean":"2023-10-09T06:36:18.541005+0000","last_became_active":"2023-10-09T06:36:18.535676+0000","last_became_peered":"2023-10-09T06:36:18.535676+0000","last_unstale":"2023-10-09T06:36:18.541005+0000","last_undegraded":"2023-10-09T06:36:18.541005+0000","last_fullsized":"2023-10-09T06:36:18.541005+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:07:24.866847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.548753+0000","last_change":"2023-10-09T06:36:18.543941+0000","last_active":"2023-10-09T06:36:18.548753+0000","last_peered":"2023-10-09T06:36:18.548753+0000","last_clean":"2023-10-09T06:36:18.548753+0000","last_became_active":"2023-10-09T06:36:18.543663+0000","last_became_peered":"2023-10-09T06:36:18.543663+0000","last_unstale":"2023-10-09T06:36:18.548753+0000","last_undegraded":"2023-10-09T06:36:18.548753+0000","last_fullsized":"2023-10-09T06:36:18.548753+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:01:04.121181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"88'4","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.785772+0000","last_change":"2023-10-09T06:36:17.789045+0000","last_active":"2023-10-09T06:36:18.785772+0000","last_peered":"2023-10-09T06:36:18.785772+0000","last_clean":"2023-10-09T06:36:18.785772+0000","last_became_active":"2023-10-09T06:36:17.788789+0000","last_became_peered":"2023-10-09T06:36:17.788789+0000","last_unstale":"2023-10-09T06:36:18.785772+0000","last_undegraded":"2023-10-09T06:36:18.785772+0000","last_fullsized":"2023-10-09T06:36:18.785772+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:37:49.125201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.546157+0000","last_change":"2023-10-09T06:36:18.540062+0000","last_active":"2023-10-09T06:36:18.546157+0000","last_peered":"2023-10-09T06:36:18.546157+0000","last_clean":"2023-10-09T06:36:18.546157+0000","last_became_active":"2023-10-09T06:36:18.539812+0000","last_became_peered":"2023-10-09T06:36:18.539812+0000","last_unstale":"2023-10-09T06:36:18.546157+0000","last_undegraded":"2023-10-09T06:36:18.546157+0000","last_fullsized":"2023-10-09T06:36:18.546157+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:59:36.007453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"88'1","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.785749+0000","last_change":"2023-10-09T06:36:17.787086+0000","last_active":"2023-10-09T06:36:18.785749+0000","last_peered":"2023-10-09T06:36:18.785749+0000","last_clean":"2023-10-09T06:36:18.785749+0000","last_became_active":"2023-10-09T06:36:17.786801+0000","last_became_peered":"2023-10-09T06:36:17.786801+0000","last_unstale":"2023-10-09T06:36:18.785749+0000","last_undegraded":"2023-10-09T06:36:18.785749+0000","last_fullsized":"2023-10-09T06:36:18.785749+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:36:25.713208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.542327+0000","last_change":"2023-10-09T06:36:18.539481+0000","last_active":"2023-10-09T06:36:18.542327+0000","last_peered":"2023-10-09T06:36:18.542327+0000","last_clean":"2023-10-09T06:36:18.542327+0000","last_became_active":"2023-10-09T06:36:18.539242+0000","last_became_peered":"2023-10-09T06:36:18.539242+0000","last_unstale":"2023-10-09T06:36:18.542327+0000","last_undegraded":"2023-10-09T06:36:18.542327+0000","last_fullsized":"2023-10-09T06:36:18.542327+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:07:55.608736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.790465+0000","last_change":"2023-10-09T06:36:17.790681+0000","last_active":"2023-10-09T06:36:18.790465+0000","last_peered":"2023-10-09T06:36:18.790465+0000","last_clean":"2023-10-09T06:36:18.790465+0000","last_became_active":"2023-10-09T06:36:17.789335+0000","last_became_peered":"2023-10-09T06:36:17.789335+0000","last_unstale":"2023-10-09T06:36:18.790465+0000","last_undegraded":"2023-10-09T06:36:18.790465+0000","last_fullsized":"2023-10-09T06:36:18.790465+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:43:52.206102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.547106+0000","last_change":"2023-10-09T06:36:18.546318+0000","last_active":"2023-10-09T06:36:18.547106+0000","last_peered":"2023-10-09T06:36:18.547106+0000","last_clean":"2023-10-09T06:36:18.547106+0000","last_became_active":"2023-10-09T06:36:18.546028+0000","last_became_peered":"2023-10-09T06:36:18.546028+0000","last_unstale":"2023-10-09T06:36:18.547106+0000","last_undegraded":"2023-10-09T06:36:18.547106+0000","last_fullsized":"2023-10-09T06:36:18.547106+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:45:50.359758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.785760+0000","last_change":"2023-10-09T06:36:17.792815+0000","last_active":"2023-10-09T06:36:18.785760+0000","last_peered":"2023-10-09T06:36:18.785760+0000","last_clean":"2023-10-09T06:36:18.785760+0000","last_became_active":"2023-10-09T06:36:17.792546+0000","last_became_peered":"2023-10-09T06:36:17.792546+0000","last_unstale":"2023-10-09T06:36:18.785760+0000","last_undegraded":"2023-10-09T06:36:18.785760+0000","last_fullsized":"2023-10-09T06:36:18.785760+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:13:55.927806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.547689+0000","last_change":"2023-10-09T06:36:18.541248+0000","last_active":"2023-10-09T06:36:18.547689+0000","last_peered":"2023-10-09T06:36:18.547689+0000","last_clean":"2023-10-09T06:36:18.547689+0000","last_became_active":"2023-10-09T06:36:18.540823+0000","last_became_peered":"2023-10-09T06:36:18.540823+0000","last_unstale":"2023-10-09T06:36:18.547689+0000","last_undegraded":"2023-10-09T06:36:18.547689+0000","last_fullsized":"2023-10-09T06:36:18.547689+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:13:56.932296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"88'1","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.784223+0000","last_change":"2023-10-09T06:36:17.783698+0000","last_active":"2023-10-09T06:36:18.784223+0000","last_peered":"2023-10-09T06:36:18.784223+0000","last_clean":"2023-10-09T06:36:18.784223+0000","last_became_active":"2023-10-09T06:36:17.783443+0000","last_became_peered":"2023-10-09T06:36:17.783443+0000","last_unstale":"2023-10-09T06:36:18.784223+0000","last_undegraded":"2023-10-09T06:36:18.784223+0000","last_fullsized":"2023-10-09T06:36:18.784223+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:02:51.159150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.544310+0000","last_change":"2023-10-09T06:36:18.542927+0000","last_active":"2023-10-09T06:36:18.544310+0000","last_peered":"2023-10-09T06:36:18.544310+0000","last_clean":"2023-10-09T06:36:18.544310+0000","last_became_active":"2023-10-09T06:36:18.542636+0000","last_became_peered":"2023-10-09T06:36:18.542636+0000","last_unstale":"2023-10-09T06:36:18.544310+0000","last_undegraded":"2023-10-09T06:36:18.544310+0000","last_fullsized":"2023-10-09T06:36:18.544310+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:38:02.641521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"88'1","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.785624+0000","last_change":"2023-10-09T06:36:17.791534+0000","last_active":"2023-10-09T06:36:18.785624+0000","last_peered":"2023-10-09T06:36:18.785624+0000","last_clean":"2023-10-09T06:36:18.785624+0000","last_became_active":"2023-10-09T06:36:17.791247+0000","last_became_peered":"2023-10-09T06:36:17.791247+0000","last_unstale":"2023-10-09T06:36:18.785624+0000","last_undegraded":"2023-10-09T06:36:18.785624+0000","last_fullsized":"2023-10-09T06:36:18.785624+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:32:09.064409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.543501+0000","last_change":"2023-10-09T06:36:18.536143+0000","last_active":"2023-10-09T06:36:18.543501+0000","last_peered":"2023-10-09T06:36:18.543501+0000","last_clean":"2023-10-09T06:36:18.543501+0000","last_became_active":"2023-10-09T06:36:18.535861+0000","last_became_peered":"2023-10-09T06:36:18.535861+0000","last_unstale":"2023-10-09T06:36:18.543501+0000","last_undegraded":"2023-10-09T06:36:18.543501+0000","last_fullsized":"2023-10-09T06:36:18.543501+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:25:40.992746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.526884+0000","last_change":"2023-10-09T06:36:17.790108+0000","last_active":"2023-10-09T06:36:18.526884+0000","last_peered":"2023-10-09T06:36:18.526884+0000","last_clean":"2023-10-09T06:36:18.526884+0000","last_became_active":"2023-10-09T06:36:17.789670+0000","last_became_peered":"2023-10-09T06:36:17.789670+0000","last_unstale":"2023-10-09T06:36:18.526884+0000","last_undegraded":"2023-10-09T06:36:18.526884+0000","last_fullsized":"2023-10-09T06:36:18.526884+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:01:10.473684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.542340+0000","last_change":"2023-10-09T06:36:18.541208+0000","last_active":"2023-10-09T06:36:18.542340+0000","last_peered":"2023-10-09T06:36:18.542340+0000","last_clean":"2023-10-09T06:36:18.542340+0000","last_became_active":"2023-10-09T06:36:18.540906+0000","last_became_peered":"2023-10-09T06:36:18.540906+0000","last_unstale":"2023-10-09T06:36:18.542340+0000","last_undegraded":"2023-10-09T06:36:18.542340+0000","last_fullsized":"2023-10-09T06:36:18.542340+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:35:51.857807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.527982+0000","last_change":"2023-10-09T06:36:17.787232+0000","last_active":"2023-10-09T06:36:18.527982+0000","last_peered":"2023-10-09T06:36:18.527982+0000","last_clean":"2023-10-09T06:36:18.527982+0000","last_became_active":"2023-10-09T06:36:17.786928+0000","last_became_peered":"2023-10-09T06:36:17.786928+0000","last_unstale":"2023-10-09T06:36:18.527982+0000","last_undegraded":"2023-10-09T06:36:18.527982+0000","last_fullsized":"2023-10-09T06:36:18.527982+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:18:25.639233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.543441+0000","last_change":"2023-10-09T06:36:18.540166+0000","last_active":"2023-10-09T06:36:18.543441+0000","last_peered":"2023-10-09T06:36:18.543441+0000","last_clean":"2023-10-09T06:36:18.543441+0000","last_became_active":"2023-10-09T06:36:18.539225+0000","last_became_peered":"2023-10-09T06:36:18.539225+0000","last_unstale":"2023-10-09T06:36:18.543441+0000","last_undegraded":"2023-10-09T06:36:18.543441+0000","last_fullsized":"2023-10-09T06:36:18.543441+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:47:18.417758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.528512+0000","last_change":"2023-10-09T06:36:17.784692+0000","last_active":"2023-10-09T06:36:18.528512+0000","last_peered":"2023-10-09T06:36:18.528512+0000","last_clean":"2023-10-09T06:36:18.528512+0000","last_became_active":"2023-10-09T06:36:17.784451+0000","last_became_peered":"2023-10-09T06:36:17.784451+0000","last_unstale":"2023-10-09T06:36:18.528512+0000","last_undegraded":"2023-10-09T06:36:18.528512+0000","last_fullsized":"2023-10-09T06:36:18.528512+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:01:09.038441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.545654+0000","last_change":"2023-10-09T06:36:18.543166+0000","last_active":"2023-10-09T06:36:18.545654+0000","last_peered":"2023-10-09T06:36:18.545654+0000","last_clean":"2023-10-09T06:36:18.545654+0000","last_became_active":"2023-10-09T06:36:18.541454+0000","last_became_peered":"2023-10-09T06:36:18.541454+0000","last_unstale":"2023-10-09T06:36:18.545654+0000","last_undegraded":"2023-10-09T06:36:18.545654+0000","last_fullsized":"2023-10-09T06:36:18.545654+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:43:53.210593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.529024+0000","last_change":"2023-10-09T06:36:17.793119+0000","last_active":"2023-10-09T06:36:18.529024+0000","last_peered":"2023-10-09T06:36:18.529024+0000","last_clean":"2023-10-09T06:36:18.529024+0000","last_became_active":"2023-10-09T06:36:17.792867+0000","last_became_peered":"2023-10-09T06:36:17.792867+0000","last_unstale":"2023-10-09T06:36:18.529024+0000","last_undegraded":"2023-10-09T06:36:18.529024+0000","last_fullsized":"2023-10-09T06:36:18.529024+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:45:49.355268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.544162+0000","last_change":"2023-10-09T06:36:18.538828+0000","last_active":"2023-10-09T06:36:18.544162+0000","last_peered":"2023-10-09T06:36:18.544162+0000","last_clean":"2023-10-09T06:36:18.544162+0000","last_became_active":"2023-10-09T06:36:18.538465+0000","last_became_peered":"2023-10-09T06:36:18.538465+0000","last_unstale":"2023-10-09T06:36:18.544162+0000","last_undegraded":"2023-10-09T06:36:18.544162+0000","last_fullsized":"2023-10-09T06:36:18.544162+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:48:11.177371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"88'3","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.787110+0000","last_change":"2023-10-09T06:36:17.792731+0000","last_active":"2023-10-09T06:36:18.787110+0000","last_peered":"2023-10-09T06:36:18.787110+0000","last_clean":"2023-10-09T06:36:18.787110+0000","last_became_active":"2023-10-09T06:36:17.792268+0000","last_became_peered":"2023-10-09T06:36:17.792268+0000","last_unstale":"2023-10-09T06:36:18.787110+0000","last_undegraded":"2023-10-09T06:36:18.787110+0000","last_fullsized":"2023-10-09T06:36:18.787110+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:32:09.064409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.547730+0000","last_change":"2023-10-09T06:36:18.544894+0000","last_active":"2023-10-09T06:36:18.547730+0000","last_peered":"2023-10-09T06:36:18.547730+0000","last_clean":"2023-10-09T06:36:18.547730+0000","last_became_active":"2023-10-09T06:36:18.544172+0000","last_became_peered":"2023-10-09T06:36:18.544172+0000","last_unstale":"2023-10-09T06:36:18.547730+0000","last_undegraded":"2023-10-09T06:36:18.547730+0000","last_fullsized":"2023-10-09T06:36:18.547730+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:13:52.813781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"88'6","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.797787+0000","last_change":"2023-10-09T06:36:17.792248+0000","last_active":"2023-10-09T06:36:18.797787+0000","last_peered":"2023-10-09T06:36:18.797787+0000","last_clean":"2023-10-09T06:36:18.797787+0000","last_became_active":"2023-10-09T06:36:17.791859+0000","last_became_peered":"2023-10-09T06:36:17.791859+0000","last_unstale":"2023-10-09T06:36:18.797787+0000","last_undegraded":"2023-10-09T06:36:18.797787+0000","last_fullsized":"2023-10-09T06:36:18.797787+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:25:34.425692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.546175+0000","last_change":"2023-10-09T06:36:18.542359+0000","last_active":"2023-10-09T06:36:18.546175+0000","last_peered":"2023-10-09T06:36:18.546175+0000","last_clean":"2023-10-09T06:36:18.546175+0000","last_became_active":"2023-10-09T06:36:18.542099+0000","last_became_peered":"2023-10-09T06:36:18.542099+0000","last_unstale":"2023-10-09T06:36:18.546175+0000","last_undegraded":"2023-10-09T06:36:18.546175+0000","last_fullsized":"2023-10-09T06:36:18.546175+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:25:35.071015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.785802+0000","last_change":"2023-10-09T06:36:17.790092+0000","last_active":"2023-10-09T06:36:18.785802+0000","last_peered":"2023-10-09T06:36:18.785802+0000","last_clean":"2023-10-09T06:36:18.785802+0000","last_became_active":"2023-10-09T06:36:17.789772+0000","last_became_peered":"2023-10-09T06:36:17.789772+0000","last_unstale":"2023-10-09T06:36:18.785802+0000","last_undegraded":"2023-10-09T06:36:18.785802+0000","last_fullsized":"2023-10-09T06:36:18.785802+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:36:07.257669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.542264+0000","last_change":"2023-10-09T06:36:18.538233+0000","last_active":"2023-10-09T06:36:18.542264+0000","last_peered":"2023-10-09T06:36:18.542264+0000","last_clean":"2023-10-09T06:36:18.542264+0000","last_became_active":"2023-10-09T06:36:18.537926+0000","last_became_peered":"2023-10-09T06:36:18.537926+0000","last_unstale":"2023-10-09T06:36:18.542264+0000","last_undegraded":"2023-10-09T06:36:18.542264+0000","last_fullsized":"2023-10-09T06:36:18.542264+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:48:11.177371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.525374+0000","last_change":"2023-10-09T06:36:17.787958+0000","last_active":"2023-10-09T06:36:18.525374+0000","last_peered":"2023-10-09T06:36:18.525374+0000","last_clean":"2023-10-09T06:36:18.525374+0000","last_became_active":"2023-10-09T06:36:17.787701+0000","last_became_peered":"2023-10-09T06:36:17.787701+0000","last_unstale":"2023-10-09T06:36:18.525374+0000","last_undegraded":"2023-10-09T06:36:18.525374+0000","last_fullsized":"2023-10-09T06:36:18.525374+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:21:55.532787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.545777+0000","last_change":"2023-10-09T06:36:18.543187+0000","last_active":"2023-10-09T06:36:18.545777+0000","last_peered":"2023-10-09T06:36:18.545777+0000","last_clean":"2023-10-09T06:36:18.545777+0000","last_became_active":"2023-10-09T06:36:18.541974+0000","last_became_peered":"2023-10-09T06:36:18.541974+0000","last_unstale":"2023-10-09T06:36:18.545777+0000","last_undegraded":"2023-10-09T06:36:18.545777+0000","last_fullsized":"2023-10-09T06:36:18.545777+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:25:20.676670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.530417+0000","last_change":"2023-10-09T06:36:17.789901+0000","last_active":"2023-10-09T06:36:18.530417+0000","last_peered":"2023-10-09T06:36:18.530417+0000","last_clean":"2023-10-09T06:36:18.530417+0000","last_became_active":"2023-10-09T06:36:17.789637+0000","last_became_peered":"2023-10-09T06:36:17.789637+0000","last_unstale":"2023-10-09T06:36:18.530417+0000","last_undegraded":"2023-10-09T06:36:18.530417+0000","last_fullsized":"2023-10-09T06:36:18.530417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:04:48.386913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.534651+0000","last_change":"2023-10-09T06:36:18.532913+0000","last_active":"2023-10-09T06:36:18.534651+0000","last_peered":"2023-10-09T06:36:18.534651+0000","last_clean":"2023-10-09T06:36:18.534651+0000","last_became_active":"2023-10-09T06:36:18.532670+0000","last_became_peered":"2023-10-09T06:36:18.532670+0000","last_unstale":"2023-10-09T06:36:18.534651+0000","last_undegraded":"2023-10-09T06:36:18.534651+0000","last_fullsized":"2023-10-09T06:36:18.534651+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:14:22.790297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"88'8","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.788094+0000","last_change":"2023-10-09T06:36:17.787273+0000","last_active":"2023-10-09T06:36:18.788094+0000","last_peered":"2023-10-09T06:36:18.788094+0000","last_clean":"2023-10-09T06:36:18.788094+0000","last_became_active":"2023-10-09T06:36:17.787000+0000","last_became_peered":"2023-10-09T06:36:17.787000+0000","last_unstale":"2023-10-09T06:36:18.788094+0000","last_undegraded":"2023-10-09T06:36:18.788094+0000","last_fullsized":"2023-10-09T06:36:18.788094+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:43:46.558569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.539259+0000","last_change":"2023-10-09T06:36:18.533573+0000","last_active":"2023-10-09T06:36:18.539259+0000","last_peered":"2023-10-09T06:36:18.539259+0000","last_clean":"2023-10-09T06:36:18.539259+0000","last_became_active":"2023-10-09T06:36:18.533249+0000","last_became_peered":"2023-10-09T06:36:18.533249+0000","last_unstale":"2023-10-09T06:36:18.539259+0000","last_undegraded":"2023-10-09T06:36:18.539259+0000","last_fullsized":"2023-10-09T06:36:18.539259+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:34:03.599407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.786103+0000","last_change":"2023-10-09T06:36:17.791080+0000","last_active":"2023-10-09T06:36:18.786103+0000","last_peered":"2023-10-09T06:36:18.786103+0000","last_clean":"2023-10-09T06:36:18.786103+0000","last_became_active":"2023-10-09T06:36:17.790800+0000","last_became_peered":"2023-10-09T06:36:17.790800+0000","last_unstale":"2023-10-09T06:36:18.786103+0000","last_undegraded":"2023-10-09T06:36:18.786103+0000","last_fullsized":"2023-10-09T06:36:18.786103+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:36:22.684838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.546061+0000","last_change":"2023-10-09T06:36:18.540939+0000","last_active":"2023-10-09T06:36:18.546061+0000","last_peered":"2023-10-09T06:36:18.546061+0000","last_clean":"2023-10-09T06:36:18.546061+0000","last_became_active":"2023-10-09T06:36:18.540653+0000","last_became_peered":"2023-10-09T06:36:18.540653+0000","last_unstale":"2023-10-09T06:36:18.546061+0000","last_undegraded":"2023-10-09T06:36:18.546061+0000","last_fullsized":"2023-10-09T06:36:18.546061+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:02:12.638507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"88'1","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.784927+0000","last_change":"2023-10-09T06:36:17.785484+0000","last_active":"2023-10-09T06:36:18.784927+0000","last_peered":"2023-10-09T06:36:18.784927+0000","last_clean":"2023-10-09T06:36:18.784927+0000","last_became_active":"2023-10-09T06:36:17.785101+0000","last_became_peered":"2023-10-09T06:36:17.785101+0000","last_unstale":"2023-10-09T06:36:18.784927+0000","last_undegraded":"2023-10-09T06:36:18.784927+0000","last_fullsized":"2023-10-09T06:36:18.784927+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:36:22.684838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.547690+0000","last_change":"2023-10-09T06:36:18.544498+0000","last_active":"2023-10-09T06:36:18.547690+0000","last_peered":"2023-10-09T06:36:18.547690+0000","last_clean":"2023-10-09T06:36:18.547690+0000","last_became_active":"2023-10-09T06:36:18.543371+0000","last_became_peered":"2023-10-09T06:36:18.543371+0000","last_unstale":"2023-10-09T06:36:18.547690+0000","last_undegraded":"2023-10-09T06:36:18.547690+0000","last_fullsized":"2023-10-09T06:36:18.547690+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:18:01.852339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.532429+0000","last_change":"2023-10-09T06:36:17.792800+0000","last_active":"2023-10-09T06:36:18.532429+0000","last_peered":"2023-10-09T06:36:18.532429+0000","last_clean":"2023-10-09T06:36:18.532429+0000","last_became_active":"2023-10-09T06:36:17.792526+0000","last_became_peered":"2023-10-09T06:36:17.792526+0000","last_unstale":"2023-10-09T06:36:18.532429+0000","last_undegraded":"2023-10-09T06:36:18.532429+0000","last_fullsized":"2023-10-09T06:36:18.532429+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:21:51.581268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.541571+0000","last_change":"2023-10-09T06:36:18.536770+0000","last_active":"2023-10-09T06:36:18.541571+0000","last_peered":"2023-10-09T06:36:18.541571+0000","last_clean":"2023-10-09T06:36:18.541571+0000","last_became_active":"2023-10-09T06:36:18.536464+0000","last_became_peered":"2023-10-09T06:36:18.536464+0000","last_unstale":"2023-10-09T06:36:18.541571+0000","last_undegraded":"2023-10-09T06:36:18.541571+0000","last_fullsized":"2023-10-09T06:36:18.541571+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:35:38.242484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.790605+0000","last_change":"2023-10-09T06:36:17.791898+0000","last_active":"2023-10-09T06:36:18.790605+0000","last_peered":"2023-10-09T06:36:18.790605+0000","last_clean":"2023-10-09T06:36:18.790605+0000","last_became_active":"2023-10-09T06:36:17.791560+0000","last_became_peered":"2023-10-09T06:36:17.791560+0000","last_unstale":"2023-10-09T06:36:18.790605+0000","last_undegraded":"2023-10-09T06:36:18.790605+0000","last_fullsized":"2023-10-09T06:36:18.790605+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:35:50.853317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.540041+0000","last_change":"2023-10-09T06:36:18.535564+0000","last_active":"2023-10-09T06:36:18.540041+0000","last_peered":"2023-10-09T06:36:18.540041+0000","last_clean":"2023-10-09T06:36:18.540041+0000","last_became_active":"2023-10-09T06:36:18.535299+0000","last_became_peered":"2023-10-09T06:36:18.535299+0000","last_unstale":"2023-10-09T06:36:18.540041+0000","last_undegraded":"2023-10-09T06:36:18.540041+0000","last_fullsized":"2023-10-09T06:36:18.540041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:46:16.269549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.534300+0000","last_change":"2023-10-09T06:36:17.790420+0000","last_active":"2023-10-09T06:36:18.534300+0000","last_peered":"2023-10-09T06:36:18.534300+0000","last_clean":"2023-10-09T06:36:18.534300+0000","last_became_active":"2023-10-09T06:36:17.790228+0000","last_became_peered":"2023-10-09T06:36:17.790228+0000","last_unstale":"2023-10-09T06:36:18.534300+0000","last_undegraded":"2023-10-09T06:36:18.534300+0000","last_fullsized":"2023-10-09T06:36:18.534300+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:01:09.038441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.537322+0000","last_change":"2023-10-09T06:36:18.534119+0000","last_active":"2023-10-09T06:36:18.537322+0000","last_peered":"2023-10-09T06:36:18.537322+0000","last_clean":"2023-10-09T06:36:18.537322+0000","last_became_active":"2023-10-09T06:36:18.533898+0000","last_became_peered":"2023-10-09T06:36:18.533898+0000","last_unstale":"2023-10-09T06:36:18.537322+0000","last_undegraded":"2023-10-09T06:36:18.537322+0000","last_fullsized":"2023-10-09T06:36:18.537322+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:34:20.539836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.534524+0000","last_change":"2023-10-09T06:36:17.787612+0000","last_active":"2023-10-09T06:36:18.534524+0000","last_peered":"2023-10-09T06:36:18.534524+0000","last_clean":"2023-10-09T06:36:18.534524+0000","last_became_active":"2023-10-09T06:36:17.787335+0000","last_became_peered":"2023-10-09T06:36:17.787335+0000","last_unstale":"2023-10-09T06:36:18.534524+0000","last_undegraded":"2023-10-09T06:36:18.534524+0000","last_fullsized":"2023-10-09T06:36:18.534524+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:25:34.425692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.539505+0000","last_change":"2023-10-09T06:36:18.536670+0000","last_active":"2023-10-09T06:36:18.539505+0000","last_peered":"2023-10-09T06:36:18.539505+0000","last_clean":"2023-10-09T06:36:18.539505+0000","last_became_active":"2023-10-09T06:36:18.536058+0000","last_became_peered":"2023-10-09T06:36:18.536058+0000","last_unstale":"2023-10-09T06:36:18.539505+0000","last_undegraded":"2023-10-09T06:36:18.539505+0000","last_fullsized":"2023-10-09T06:36:18.539505+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:35:51.857807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.532714+0000","last_change":"2023-10-09T06:36:17.788967+0000","last_active":"2023-10-09T06:36:18.532714+0000","last_peered":"2023-10-09T06:36:18.532714+0000","last_clean":"2023-10-09T06:36:18.532714+0000","last_became_active":"2023-10-09T06:36:17.788685+0000","last_became_peered":"2023-10-09T06:36:17.788685+0000","last_unstale":"2023-10-09T06:36:18.532714+0000","last_undegraded":"2023-10-09T06:36:18.532714+0000","last_fullsized":"2023-10-09T06:36:18.532714+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:25:19.672180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.546037+0000","last_change":"2023-10-09T06:36:18.541923+0000","last_active":"2023-10-09T06:36:18.546037+0000","last_peered":"2023-10-09T06:36:18.546037+0000","last_clean":"2023-10-09T06:36:18.546037+0000","last_became_active":"2023-10-09T06:36:18.541630+0000","last_became_peered":"2023-10-09T06:36:18.541630+0000","last_unstale":"2023-10-09T06:36:18.546037+0000","last_undegraded":"2023-10-09T06:36:18.546037+0000","last_fullsized":"2023-10-09T06:36:18.546037+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:46:35.313608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.783556+0000","last_change":"2023-10-09T06:36:17.792603+0000","last_active":"2023-10-09T06:36:18.783556+0000","last_peered":"2023-10-09T06:36:18.783556+0000","last_clean":"2023-10-09T06:36:18.783556+0000","last_became_active":"2023-10-09T06:36:17.792288+0000","last_became_peered":"2023-10-09T06:36:17.792288+0000","last_unstale":"2023-10-09T06:36:18.783556+0000","last_undegraded":"2023-10-09T06:36:18.783556+0000","last_fullsized":"2023-10-09T06:36:18.783556+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:36:22.684838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.538476+0000","last_change":"2023-10-09T06:36:18.533485+0000","last_active":"2023-10-09T06:36:18.538476+0000","last_peered":"2023-10-09T06:36:18.538476+0000","last_clean":"2023-10-09T06:36:18.538476+0000","last_became_active":"2023-10-09T06:36:18.533150+0000","last_became_peered":"2023-10-09T06:36:18.533150+0000","last_unstale":"2023-10-09T06:36:18.538476+0000","last_undegraded":"2023-10-09T06:36:18.538476+0000","last_fullsized":"2023-10-09T06:36:18.538476+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:36:26.717698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.532318+0000","last_change":"2023-10-09T06:36:17.791125+0000","last_active":"2023-10-09T06:36:18.532318+0000","last_peered":"2023-10-09T06:36:18.532318+0000","last_clean":"2023-10-09T06:36:18.532318+0000","last_became_active":"2023-10-09T06:36:17.789317+0000","last_became_peered":"2023-10-09T06:36:17.789317+0000","last_unstale":"2023-10-09T06:36:18.532318+0000","last_undegraded":"2023-10-09T06:36:18.532318+0000","last_fullsized":"2023-10-09T06:36:18.532318+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:25:34.425692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.543420+0000","last_change":"2023-10-09T06:36:18.540876+0000","last_active":"2023-10-09T06:36:18.543420+0000","last_peered":"2023-10-09T06:36:18.543420+0000","last_clean":"2023-10-09T06:36:18.543420+0000","last_became_active":"2023-10-09T06:36:18.540636+0000","last_became_peered":"2023-10-09T06:36:18.540636+0000","last_unstale":"2023-10-09T06:36:18.543420+0000","last_undegraded":"2023-10-09T06:36:18.543420+0000","last_fullsized":"2023-10-09T06:36:18.543420+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:34:03.599407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.532332+0000","last_change":"2023-10-09T06:36:17.793201+0000","last_active":"2023-10-09T06:36:18.532332+0000","last_peered":"2023-10-09T06:36:18.532332+0000","last_clean":"2023-10-09T06:36:18.532332+0000","last_became_active":"2023-10-09T06:36:17.792955+0000","last_became_peered":"2023-10-09T06:36:17.792955+0000","last_unstale":"2023-10-09T06:36:18.532332+0000","last_undegraded":"2023-10-09T06:36:18.532332+0000","last_fullsized":"2023-10-09T06:36:18.532332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:15:58.450498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.539402+0000","last_change":"2023-10-09T06:36:18.535541+0000","last_active":"2023-10-09T06:36:18.539402+0000","last_peered":"2023-10-09T06:36:18.539402+0000","last_clean":"2023-10-09T06:36:18.539402+0000","last_became_active":"2023-10-09T06:36:18.535246+0000","last_became_peered":"2023-10-09T06:36:18.535246+0000","last_unstale":"2023-10-09T06:36:18.539402+0000","last_undegraded":"2023-10-09T06:36:18.539402+0000","last_fullsized":"2023-10-09T06:36:18.539402+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:25:20.676670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.526944+0000","last_change":"2023-10-09T06:36:17.784824+0000","last_active":"2023-10-09T06:36:18.526944+0000","last_peered":"2023-10-09T06:36:18.526944+0000","last_clean":"2023-10-09T06:36:18.526944+0000","last_became_active":"2023-10-09T06:36:17.784571+0000","last_became_peered":"2023-10-09T06:36:17.784571+0000","last_unstale":"2023-10-09T06:36:18.526944+0000","last_undegraded":"2023-10-09T06:36:18.526944+0000","last_fullsized":"2023-10-09T06:36:18.526944+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:02:15.826669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.541393+0000","last_change":"2023-10-09T06:36:18.534396+0000","last_active":"2023-10-09T06:36:18.541393+0000","last_peered":"2023-10-09T06:36:18.541393+0000","last_clean":"2023-10-09T06:36:18.541393+0000","last_became_active":"2023-10-09T06:36:18.533843+0000","last_became_peered":"2023-10-09T06:36:18.533843+0000","last_unstale":"2023-10-09T06:36:18.541393+0000","last_undegraded":"2023-10-09T06:36:18.541393+0000","last_fullsized":"2023-10-09T06:36:18.541393+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:58:18.392433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.532532+0000","last_change":"2023-10-09T06:36:17.790768+0000","last_active":"2023-10-09T06:36:18.532532+0000","last_peered":"2023-10-09T06:36:18.532532+0000","last_clean":"2023-10-09T06:36:18.532532+0000","last_became_active":"2023-10-09T06:36:17.790602+0000","last_became_peered":"2023-10-09T06:36:17.790602+0000","last_unstale":"2023-10-09T06:36:18.532532+0000","last_undegraded":"2023-10-09T06:36:18.532532+0000","last_fullsized":"2023-10-09T06:36:18.532532+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:45:49.355268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.542140+0000","last_change":"2023-10-09T06:36:18.540890+0000","last_active":"2023-10-09T06:36:18.542140+0000","last_peered":"2023-10-09T06:36:18.542140+0000","last_clean":"2023-10-09T06:36:18.542140+0000","last_became_active":"2023-10-09T06:36:18.540591+0000","last_became_peered":"2023-10-09T06:36:18.540591+0000","last_unstale":"2023-10-09T06:36:18.542140+0000","last_undegraded":"2023-10-09T06:36:18.542140+0000","last_fullsized":"2023-10-09T06:36:18.542140+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:32:10.068899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.526766+0000","last_change":"2023-10-09T06:36:17.786310+0000","last_active":"2023-10-09T06:36:18.526766+0000","last_peered":"2023-10-09T06:36:18.526766+0000","last_clean":"2023-10-09T06:36:18.526766+0000","last_became_active":"2023-10-09T06:36:17.785631+0000","last_became_peered":"2023-10-09T06:36:17.785631+0000","last_unstale":"2023-10-09T06:36:18.526766+0000","last_undegraded":"2023-10-09T06:36:18.526766+0000","last_fullsized":"2023-10-09T06:36:18.526766+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:44:49.435194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.537306+0000","last_change":"2023-10-09T06:36:18.533231+0000","last_active":"2023-10-09T06:36:18.537306+0000","last_peered":"2023-10-09T06:36:18.537306+0000","last_clean":"2023-10-09T06:36:18.537306+0000","last_became_active":"2023-10-09T06:36:18.532977+0000","last_became_peered":"2023-10-09T06:36:18.532977+0000","last_unstale":"2023-10-09T06:36:18.537306+0000","last_undegraded":"2023-10-09T06:36:18.537306+0000","last_fullsized":"2023-10-09T06:36:18.537306+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:11:55.258763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.533336+0000","last_change":"2023-10-09T06:36:17.785559+0000","last_active":"2023-10-09T06:36:18.533336+0000","last_peered":"2023-10-09T06:36:18.533336+0000","last_clean":"2023-10-09T06:36:18.533336+0000","last_became_active":"2023-10-09T06:36:17.785290+0000","last_became_peered":"2023-10-09T06:36:17.785290+0000","last_unstale":"2023-10-09T06:36:18.533336+0000","last_undegraded":"2023-10-09T06:36:18.533336+0000","last_fullsized":"2023-10-09T06:36:18.533336+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:25:19.672180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.539523+0000","last_change":"2023-10-09T06:36:18.536856+0000","last_active":"2023-10-09T06:36:18.539523+0000","last_peered":"2023-10-09T06:36:18.539523+0000","last_clean":"2023-10-09T06:36:18.539523+0000","last_became_active":"2023-10-09T06:36:18.536403+0000","last_became_peered":"2023-10-09T06:36:18.536403+0000","last_unstale":"2023-10-09T06:36:18.539523+0000","last_undegraded":"2023-10-09T06:36:18.539523+0000","last_fullsized":"2023-10-09T06:36:18.539523+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:45:50.359758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.524897+0000","last_change":"2023-10-09T06:36:17.788850+0000","last_active":"2023-10-09T06:36:18.524897+0000","last_peered":"2023-10-09T06:36:18.524897+0000","last_clean":"2023-10-09T06:36:18.524897+0000","last_became_active":"2023-10-09T06:36:17.788604+0000","last_became_peered":"2023-10-09T06:36:17.788604+0000","last_unstale":"2023-10-09T06:36:18.524897+0000","last_undegraded":"2023-10-09T06:36:18.524897+0000","last_fullsized":"2023-10-09T06:36:18.524897+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:05:55.338038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.540075+0000","last_change":"2023-10-09T06:36:18.536117+0000","last_active":"2023-10-09T06:36:18.540075+0000","last_peered":"2023-10-09T06:36:18.540075+0000","last_clean":"2023-10-09T06:36:18.540075+0000","last_became_active":"2023-10-09T06:36:18.534990+0000","last_became_peered":"2023-10-09T06:36:18.534990+0000","last_unstale":"2023-10-09T06:36:18.540075+0000","last_undegraded":"2023-10-09T06:36:18.540075+0000","last_fullsized":"2023-10-09T06:36:18.540075+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:17.767598+0000","last_clean_scrub_stamp":"2023-10-09T06:36:17.767598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:29:08.981159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"88'1","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.796295+0000","last_change":"2023-10-09T06:36:17.786883+0000","last_active":"2023-10-09T06:36:18.796295+0000","last_peered":"2023-10-09T06:36:18.796295+0000","last_clean":"2023-10-09T06:36:18.796295+0000","last_became_active":"2023-10-09T06:36:17.786607+0000","last_became_peered":"2023-10-09T06:36:17.786607+0000","last_unstale":"2023-10-09T06:36:18.796295+0000","last_undegraded":"2023-10-09T06:36:18.796295+0000","last_fullsized":"2023-10-09T06:36:18.796295+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:16.763107+0000","last_clean_scrub_stamp":"2023-10-09T06:36:16.763107+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:34:02.594917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":221,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-09T06:36:18.535858+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:36:18.535858+0000","last_peered":"2023-10-09T06:36:18.535858+0000","last_clean":"2023-10-09T06:36:18.535858+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:36:18.535858+0000","last_undegraded":"2023-10-09T06:36:18.535858+0000","last_fullsized":"2023-10-09T06:36:18.535858+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574922,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298940,"kb_used_data":1652,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072900,"statfs":{"total":96636764160,"available":96330649600,"internally_reserved":0,"allocated":1691648,"data_stored":1060361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:33 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.50700000000000001,"15min":0.503},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.66300000000000003,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.34100000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65700000000000003,"5min":0.753,"15min":0.753},"last":0.68000000000000005}]},{"osd":1,"last update":"Mon Oct 9 06:35:33 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.50900000000000001,"15min":0.52100000000000002},"min":{"1min":0.36099999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.63700000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.52300000000000002,"15min":0.52400000000000002},"min":{"1min":0.32900000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.66100000000000003,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.62}]},{"osd":2,"last update":"Mon Oct 9 06:35:33 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.52100000000000002,"15min":0.50900000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65700000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52900000000000003,"15min":0.54500000000000004},"min":{"1min":0.36799999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.69099999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.59699999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:35:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.53500000000000003,"15min":0.55300000000000005},"min":{"1min":0.36099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56399999999999995,"15min":0.59799999999999998},"min":{"1min":0.42299999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82099999999999995,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.66200000000000003}]},{"osd":4,"last update":"Mon Oct 9 06:35:33 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.48199999999999998,"15min":0.52400000000000002},"min":{"1min":0.307,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.69999999999999996,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56899999999999995,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.85999999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:35:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52700000000000002,"15min":0.57699999999999996},"min":{"1min":0.34399999999999997,"5min":0.307,"15min":0.307},"max":{"1min":0.85799999999999998,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58699999999999997,"15min":0.621},"min":{"1min":0.41699999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.74099999999999999,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.79300000000000004}]},{"osd":6,"last update":"Mon Oct 9 06:35:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55100000000000005,"15min":0.59799999999999998},"min":{"1min":0.378,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.78100000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55300000000000005,"15min":0.61799999999999999},"min":{"1min":0.35399999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.79900000000000004,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.77800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574921,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302468,"kb_used_data":1084,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069372,"statfs":{"total":96636764160,"available":96327036928,"internally_reserved":0,"allocated":1110016,"data_stored":476793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.495,"15min":0.501},"min":{"1min":0.35399999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.32200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.47599999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:35:36 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.495,"15min":0.51300000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.72499999999999998,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52600000000000002,"15min":0.53200000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.61399999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:35:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51000000000000001,"15min":0.51500000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69499999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.495},{"interface":"front","average":{"1min":0.5,"5min":0.51300000000000001,"15min":0.53300000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.67500000000000004,"5min":0.752,"15min":0.752},"last":0.44700000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:35:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53500000000000003,"15min":0.54800000000000004},"min":{"1min":0.40000000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70899999999999996,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59399999999999997,"15min":0.71799999999999997},"min":{"1min":0.372,"5min":0.37,"15min":0.37},"max":{"1min":0.73499999999999999,"5min":4.7510000000000003,"15min":4.7510000000000003},"last":0.41699999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:35:36 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46800000000000003,"15min":0.47999999999999998},"min":{"1min":0.29799999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.67500000000000004,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.57199999999999995,"15min":0.58499999999999996},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.76700000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.59599999999999997}]},{"osd":5,"last update":"Mon Oct 9 06:35:36 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.52700000000000002,"15min":0.53300000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.755,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.318},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57799999999999996,"15min":0.59999999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78000000000000003,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.51200000000000001}]},{"osd":7,"last update":"Mon Oct 9 06:35:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56499999999999995,"15min":0.59099999999999997},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.55700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302460,"kb_used_data":1076,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069380,"statfs":{"total":96636764160,"available":96327045120,"internally_reserved":0,"allocated":1101824,"data_stored":476169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.30399999999999999,"15min":0.308},"min":{"1min":0.20200000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"last":0.19700000000000001},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34899999999999998,"15min":0.35999999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.53600000000000003,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.26300000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34300000000000003,"15min":0.35099999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.251},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.38400000000000001,"15min":0.38600000000000001},"min":{"1min":0.29699999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.57499999999999996,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.33300000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.36099999999999999,"15min":0.36299999999999999},"min":{"1min":0.247,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.41299999999999998,"15min":0.57899999999999996},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.54800000000000004,"5min":5.8529999999999998,"15min":5.8529999999999998},"last":0.30399999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.42299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.58199999999999996,"15min":0.58599999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.37}]},{"osd":5,"last update":"Mon Oct 9 06:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.62,"15min":0.77800000000000002},"min":{"1min":0.37,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.747,"5min":5.9459999999999997,"15min":5.9459999999999997},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60199999999999998,"15min":0.61199999999999999},"min":{"1min":0.47499999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.92700000000000005,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.443}]},{"osd":6,"last update":"Mon Oct 9 06:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.59599999999999997,"15min":0.748},"min":{"1min":0.39100000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73799999999999999,"5min":5.9160000000000004,"15min":5.9160000000000004},"last":0.38},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.42699999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91000000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.35699999999999998}]},{"osd":7,"last update":"Mon Oct 9 06:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56999999999999995,"15min":0.58999999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.89500000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.433},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.624,"15min":0.79000000000000004},"min":{"1min":0.40699999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80900000000000005,"5min":5.9729999999999999,"15min":5.9729999999999999},"last":0.40400000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303044,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068796,"statfs":{"total":96636764160,"available":96326447104,"internally_reserved":0,"allocated":1699840,"data_stored":1067091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.308,"15min":0.317},"min":{"1min":0.214,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.47599999999999998,"5min":0.498,"15min":0.498},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.33700000000000002,"15min":0.34699999999999998},"min":{"1min":0.23100000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.53600000000000003,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.25}]},{"osd":2,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.35299999999999998,"15min":0.36299999999999999},"min":{"1min":0.252,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38100000000000001,"15min":0.38900000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.56499999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.29499999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35599999999999998,"15min":0.39700000000000002},"min":{"1min":0.27300000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51200000000000001,"5min":1.554,"15min":1.554},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.378,"15min":0.40400000000000003},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.58099999999999996,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.311}]},{"osd":4,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.36599999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70199999999999996,"5min":1.212,"15min":1.212},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61199999999999999,"15min":0.75800000000000001},"min":{"1min":0.41799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82499999999999996,"5min":5.484,"15min":5.484},"last":0.376}]},{"osd":5,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.61499999999999999,"15min":0.76300000000000001},"min":{"1min":0.37,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.69999999999999996,"5min":5.5449999999999999,"15min":5.5449999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57299999999999995,"15min":0.57599999999999996},"min":{"1min":0.40899999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84499999999999997,"5min":1.113,"15min":1.113},"last":0.47599999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59499999999999997,"15min":0.60699999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.72899999999999998,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.42799999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83899999999999997,"5min":1.169,"15min":1.169},"last":0.45700000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.39700000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.85499999999999998,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.628,"15min":0.79000000000000004},"min":{"1min":0.42699999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.71299999999999997,"5min":5.5800000000000001,"15min":5.5800000000000001},"last":0.53400000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298368,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073472,"statfs":{"total":96636764160,"available":96331235328,"internally_reserved":0,"allocated":1105920,"data_stored":476203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.32200000000000001,"15min":0.35999999999999999},"min":{"1min":0.218,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.50800000000000001,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.311},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.40100000000000002,"15min":0.60999999999999999},"min":{"1min":0.216,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.53800000000000003,"5min":7.2699999999999996,"15min":7.2699999999999996},"last":0.26100000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.32800000000000001,"15min":0.34899999999999998},"min":{"1min":0.22600000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.497,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.371,"5min":0.45100000000000001,"15min":0.66600000000000004},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.57599999999999996,"5min":7.327,"15min":7.327},"last":0.376}]},{"osd":3,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.376,"15min":0.39400000000000002},"min":{"1min":0.28299999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.52900000000000003,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.374,"15min":0.39500000000000002},"min":{"1min":0.25900000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.48399999999999999,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.35699999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53700000000000003,"15min":0.54100000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.73299999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.628,"15min":0.83299999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83199999999999996,"5min":7.2389999999999999,"15min":7.2389999999999999},"last":0.60899999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.69899999999999995,"15min":0.98999999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.747,"5min":10.890000000000001,"15min":10.890000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59099999999999997,"15min":0.60199999999999998},"min":{"1min":0.39800000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.78200000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.84499999999999997}]},{"osd":6,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.64300000000000002,"15min":0.84099999999999997},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77300000000000002,"5min":7.2999999999999998,"15min":7.2999999999999998},"last":0.753},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57199999999999995,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77000000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.623}]},{"osd":7,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.67200000000000004,"15min":0.873},"min":{"1min":0.38600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.79900000000000004,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.61599999999999999,"15min":0.628},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79300000000000004,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.57799999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574923,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298368,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073472,"statfs":{"total":96636764160,"available":96331235328,"internally_reserved":0,"allocated":1105920,"data_stored":471439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.30199999999999999,"5min":0.35699999999999998,"15min":0.32700000000000001},"min":{"1min":0.23599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.38300000000000001,"5min":5.8700000000000001,"15min":5.8700000000000001},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.38600000000000001,"15min":0.36599999999999999},"min":{"1min":0.224,"5min":0.222,"15min":0.222},"max":{"1min":0.46200000000000002,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.38400000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.375,"15min":0.35699999999999998},"min":{"1min":0.23899999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.47699999999999998,"5min":4.0469999999999997,"15min":4.0469999999999997},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40000000000000002,"15min":0.38700000000000001},"min":{"1min":0.27100000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.48699999999999999,"5min":4.1509999999999998,"15min":4.1509999999999998},"last":0.36099999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.373,"15min":0.374},"min":{"1min":0.24399999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.499,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.38900000000000001,"15min":0.38500000000000001},"min":{"1min":0.26200000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.51500000000000001,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.375}]},{"osd":4,"last update":"Mon Oct 9 06:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58099999999999996,"15min":0.57099999999999995},"min":{"1min":0.372,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80100000000000005,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57299999999999995,"15min":0.56200000000000006},"min":{"1min":0.36599999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68899999999999995,"5min":3.0710000000000002,"15min":3.0710000000000002},"last":0.42199999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60999999999999999,"15min":0.58599999999999997},"min":{"1min":0.38200000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.746,"5min":6.2300000000000004,"15min":6.2300000000000004},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.60199999999999998,"15min":0.59199999999999997},"min":{"1min":0.41499999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72299999999999998,"5min":3.0219999999999998,"15min":3.0219999999999998},"last":0.51700000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.60599999999999998,"15min":0.58899999999999997},"min":{"1min":0.42899999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.71199999999999997,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.504},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.67900000000000005,"15min":0.64800000000000002},"min":{"1min":0.46400000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.79300000000000004,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.52500000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:35:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63100000000000001,"15min":0.60999999999999999},"min":{"1min":0.496,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77900000000000003,"5min":4.2629999999999999,"15min":4.2629999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.64700000000000002,"15min":0.61099999999999999},"min":{"1min":0.40500000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.73599999999999999,"5min":7.3650000000000002,"15min":7.3650000000000002},"last":0.53600000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574922,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298384,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073456,"statfs":{"total":96636764160,"available":96331218944,"internally_reserved":0,"allocated":1122304,"data_stored":477829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.53200000000000003,"15min":0.56000000000000005},"min":{"1min":0.34499999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.63900000000000001,"5min":1.083,"15min":1.083},"last":0.621},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.33300000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.97799999999999998,"5min":1.054,"15min":1.054},"last":0.61099999999999999}]},{"osd":1,"last update":"Mon Oct 9 06:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.65700000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51100000000000001,"15min":0.495},"min":{"1min":0.42099999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.62,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.63300000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52600000000000002,"15min":0.54200000000000004},"min":{"1min":0.35299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71499999999999997,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.57099999999999995,"15min":0.59899999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.65100000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.55900000000000005}]},{"osd":3,"last update":"Mon Oct 9 06:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54000000000000004,"15min":0.55700000000000005},"min":{"1min":0.39200000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.66700000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57599999999999996,"15min":0.59099999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.78500000000000003,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.51400000000000001}]},{"osd":5,"last update":"Mon Oct 9 06:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52300000000000002,"15min":0.56100000000000005},"min":{"1min":0.36199999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.76500000000000001,"5min":0.998,"15min":0.998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58799999999999997,"15min":0.61399999999999999},"min":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.751,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.59299999999999997}]},{"osd":6,"last update":"Mon Oct 9 06:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.56999999999999995},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57399999999999995,"15min":0.60199999999999998},"min":{"1min":0.41499999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.67800000000000005}]},{"osd":7,"last update":"Mon Oct 9 06:35:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54900000000000004,"15min":0.57499999999999996},"min":{"1min":0.33600000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.83199999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60899999999999999,"15min":0.65100000000000002},"min":{"1min":0.40500000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.98999999999999999,"5min":1.175,"15min":1.175},"last":0.66200000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574922,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298368,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073472,"statfs":{"total":96636764160,"available":96331235328,"internally_reserved":0,"allocated":1105920,"data_stored":476239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.498,"15min":0.50800000000000001},"min":{"1min":0.30599999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.60499999999999998,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.314},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.51300000000000001,"15min":0.51200000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.621,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.60899999999999999}]},{"osd":1,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.50900000000000001,"15min":0.51000000000000001},"min":{"1min":0.33000000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.67500000000000004,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.52100000000000002,"15min":0.53100000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64700000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.36599999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54600000000000004,"15min":0.54400000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.69399999999999995,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.495},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.58099999999999996,"15min":0.58799999999999997},"min":{"1min":0.40999999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.71699999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.55800000000000005}]},{"osd":3,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53900000000000003,"15min":0.54200000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.66100000000000003,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55600000000000005,"15min":0.56399999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.69999999999999996,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.59099999999999997}]},{"osd":4,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.49399999999999999,"15min":0.496},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68300000000000005,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.71999999999999997,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.68700000000000006}]},{"osd":6,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.36099999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72099999999999997,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.625},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.55500000000000005,"15min":0.55800000000000005},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73799999999999999,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.66300000000000003}]},{"osd":7,"last update":"Mon Oct 9 06:35:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78100000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.35199999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.84499999999999997,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.81299999999999994}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:36:23.204 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-09T06:36:23.204 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-09T06:36:23.359 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.359+0000 7f0c5175b700 1 Processor -- start 2023-10-09T06:36:23.359 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.359+0000 7f0c5175b700 1 -- start start 2023-10-09T06:36:23.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.360+0000 7f0c5175b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c4c15d930 0x7f0c4c1645c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:23.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.360+0000 7f0c5175b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c4c0b10f0 con 0x7f0c4c0cf480 2023-10-09T06:36:23.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.360+0000 7f0c5175b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c4c15e000 con 0x7f0c4c1595d0 2023-10-09T06:36:23.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.360+0000 7f0c5175b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c0cf990 con 0x7f0c4c15d930 2023-10-09T06:36:23.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.360+0000 7f0c4b7fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f0c4c0cf480 0x7f0c4c15d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33656/0 (socket says 172.21.15.163:33656) 2023-10-09T06:36:23.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.360+0000 7f0c4b7fe700 1 -- 172.21.15.163:0/2711466335 learned_addr learned my addr 172.21.15.163:0/2711466335 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:23.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.360+0000 7f0c50f5a700 1 --2- 172.21.15.163:0/2711466335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c4c15d930 0x7f0c4c1645c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:23.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.360+0000 7f0c4affd700 1 -- 172.21.15.163:0/2711466335 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4147226189 0 0) 0x7f0c4c0b10f0 con 0x7f0c4c0cf480 2023-10-09T06:36:23.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.360+0000 7f0c4affd700 1 -- 172.21.15.163:0/2711466335 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c34003570 con 0x7f0c4c0cf480 2023-10-09T06:36:23.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.361+0000 7f0c4affd700 1 -- 172.21.15.163:0/2711466335 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2514937045 0 0) 0x7f0c4c15e000 con 0x7f0c4c1595d0 2023-10-09T06:36:23.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.361+0000 7f0c4affd700 1 -- 172.21.15.163:0/2711466335 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c4c0b10f0 con 0x7f0c4c1595d0 2023-10-09T06:36:23.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.361+0000 7f0c4affd700 1 -- 172.21.15.163:0/2711466335 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2547323607 0 0) 0x7f0c34003570 con 0x7f0c4c0cf480 2023-10-09T06:36:23.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.361+0000 7f0c50f5a700 1 -- 172.21.15.163:0/2711466335 >> v1:172.21.15.181:6789/0 conn(0x7f0c4c1595d0 legacy=0x7f0c4c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:23.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.361+0000 7f0c50f5a700 1 -- 172.21.15.163:0/2711466335 >> v1:172.21.15.163:6789/0 conn(0x7f0c4c0cf480 legacy=0x7f0c4c15d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:23.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.361+0000 7f0c50f5a700 1 -- 172.21.15.163:0/2711466335 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4c0cfc70 con 0x7f0c4c15d930 2023-10-09T06:36:23.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.361+0000 7f0c50f5a700 1 --2- 172.21.15.163:0/2711466335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c4c15d930 0x7f0c4c1645c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0c44009e90 tx=0x7f0c44009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=593cbd826089f15b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:23.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.361+0000 7f0c4affd700 1 -- 172.21.15.163:0/2711466335 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4400e070 con 0x7f0c4c15d930 2023-10-09T06:36:23.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.361+0000 7f0c4affd700 1 -- 172.21.15.163:0/2711466335 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c4400b870 con 0x7f0c4c15d930 2023-10-09T06:36:23.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.361+0000 7f0c4affd700 1 -- 172.21.15.163:0/2711466335 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c40001470 con 0x7f0c4c15d930 2023-10-09T06:36:23.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.361+0000 7f0c5175b700 1 -- 172.21.15.163:0/2711466335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c4c15d930 msgr2=0x7f0c4c1645c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:23.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.361+0000 7f0c5175b700 1 --2- 172.21.15.163:0/2711466335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c4c15d930 0x7f0c4c1645c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0c44009e90 tx=0x7f0c44009780 comp rx=0 tx=0).stop 2023-10-09T06:36:23.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.362+0000 7f0c5175b700 1 -- 172.21.15.163:0/2711466335 shutdown_connections 2023-10-09T06:36:23.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.362+0000 7f0c5175b700 1 --2- 172.21.15.163:0/2711466335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c4c15d930 0x7f0c4c1645c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:23.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.362+0000 7f0c5175b700 1 -- 172.21.15.163:0/2711466335 >> 172.21.15.163:0/2711466335 conn(0x7f0c4c0d05f0 msgr2=0x7f0c4c0d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:23.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.362+0000 7f0c5175b700 1 -- 172.21.15.163:0/2711466335 shutdown_connections 2023-10-09T06:36:23.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.362+0000 7f0c5175b700 1 -- 172.21.15.163:0/2711466335 wait complete. 2023-10-09T06:36:23.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.363+0000 7f0c5175b700 1 Processor -- start 2023-10-09T06:36:23.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.363+0000 7f0c5175b700 1 -- start start 2023-10-09T06:36:23.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.364+0000 7f0c5175b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c4c0cf480 0x7f0c4c169eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:23.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.364+0000 7f0c5175b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c4c1595d0 0x7f0c4c14d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:23.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.364+0000 7f0c5175b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c4c15d930 0x7f0c4c14db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:23.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.364+0000 7f0c5175b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c16aee0 con 0x7f0c4c1595d0 2023-10-09T06:36:23.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.364+0000 7f0c5175b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c15b2e0 con 0x7f0c4c0cf480 2023-10-09T06:36:23.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.364+0000 7f0c5175b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c16a3f0 con 0x7f0c4c15d930 2023-10-09T06:36:23.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.364+0000 7f0c4b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c4c1595d0 0x7f0c4c14d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:23.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.364+0000 7f0c4b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c4c1595d0 0x7f0c4c14d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:54932/0 (socket says 172.21.15.163:54932) 2023-10-09T06:36:23.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.364+0000 7f0c4b7fe700 1 -- 172.21.15.163:0/215148646 learned_addr learned my addr 172.21.15.163:0/215148646 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:23.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.364+0000 7f0c4bfff700 1 --2- 172.21.15.163:0/215148646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c4c0cf480 0x7f0c4c169eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:23.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.364+0000 7f0c50f5a700 1 --2- 172.21.15.163:0/215148646 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c4c15d930 0x7f0c4c14db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:23.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.364+0000 7f0c4b7fe700 1 -- 172.21.15.163:0/215148646 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c4c15d930 msgr2=0x7f0c4c14db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:23.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.364+0000 7f0c4b7fe700 1 --2- 172.21.15.163:0/215148646 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c4c15d930 0x7f0c4c14db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:23.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.364+0000 7f0c4b7fe700 1 -- 172.21.15.163:0/215148646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c4c0cf480 msgr2=0x7f0c4c169eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:23.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.364+0000 7f0c4b7fe700 1 --2- 172.21.15.163:0/215148646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c4c0cf480 0x7f0c4c169eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:23.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.364+0000 7f0c4b7fe700 1 -- 172.21.15.163:0/215148646 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c44009a20 con 0x7f0c4c1595d0 2023-10-09T06:36:23.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.364+0000 7f0c4b7fe700 1 --2- 172.21.15.163:0/215148646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c4c1595d0 0x7f0c4c14d380 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000c840 tx=0x7f0c40001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:23.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.364+0000 7f0c33fff700 1 -- 172.21.15.163:0/215148646 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c40005b30 con 0x7f0c4c1595d0 2023-10-09T06:36:23.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.364+0000 7f0c33fff700 1 -- 172.21.15.163:0/215148646 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c40005c90 con 0x7f0c4c1595d0 2023-10-09T06:36:23.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.364+0000 7f0c5175b700 1 -- 172.21.15.163:0/215148646 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c4c15d500 con 0x7f0c4c1595d0 2023-10-09T06:36:23.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.364+0000 7f0c33fff700 1 -- 172.21.15.163:0/215148646 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c40007340 con 0x7f0c4c1595d0 2023-10-09T06:36:23.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.364+0000 7f0c5175b700 1 -- 172.21.15.163:0/215148646 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c4c16a920 con 0x7f0c4c1595d0 2023-10-09T06:36:23.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.365+0000 7f0c5175b700 1 -- 172.21.15.163:0/215148646 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c4c15b470 con 0x7f0c4c1595d0 2023-10-09T06:36:23.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.366+0000 7f0c33fff700 1 -- 172.21.15.163:0/215148646 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f0c40012400 con 0x7f0c4c1595d0 2023-10-09T06:36:23.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.366+0000 7f0c33fff700 1 --2- 172.21.15.163:0/215148646 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0c38067200 0x7f0c380696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:23.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.366+0000 7f0c33fff700 1 -- 172.21.15.163:0/215148646 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0c40083e10 con 0x7f0c4c1595d0 2023-10-09T06:36:23.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.366+0000 7f0c4bfff700 1 --2- 172.21.15.163:0/215148646 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0c38067200 0x7f0c380696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:23.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.367+0000 7f0c4bfff700 1 --2- 172.21.15.163:0/215148646 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0c38067200 0x7f0c380696b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c003ae0 tx=0x7f0c3c007d40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:23.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.370+0000 7f0c33fff700 1 -- 172.21.15.163:0/215148646 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c4004b5a0 con 0x7f0c4c1595d0 2023-10-09T06:36:23.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.630+0000 7f0c5175b700 1 -- 172.21.15.163:0/215148646 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f0c4c04c3c0 con 0x7f0c4c1595d0 2023-10-09T06:36:23.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.634+0000 7f0c33fff700 1 -- 172.21.15.163:0/215148646 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+0 (secure 0 0 0) 0x7f0c40017070 con 0x7f0c4c1595d0 2023-10-09T06:36:23.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.636+0000 7f0c5175b700 1 -- 172.21.15.163:0/215148646 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0c38067200 msgr2=0x7f0c380696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:23.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.636+0000 7f0c5175b700 1 --2- 172.21.15.163:0/215148646 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0c38067200 0x7f0c380696b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c003ae0 tx=0x7f0c3c007d40 comp rx=0 tx=0).stop 2023-10-09T06:36:23.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.636+0000 7f0c5175b700 1 -- 172.21.15.163:0/215148646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c4c1595d0 msgr2=0x7f0c4c14d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:23.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.636+0000 7f0c5175b700 1 --2- 172.21.15.163:0/215148646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c4c1595d0 0x7f0c4c14d380 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000c840 tx=0x7f0c40001980 comp rx=0 tx=0).stop 2023-10-09T06:36:23.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.636+0000 7f0c5175b700 1 -- 172.21.15.163:0/215148646 shutdown_connections 2023-10-09T06:36:23.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.636+0000 7f0c5175b700 1 --2- 172.21.15.163:0/215148646 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0c38067200 0x7f0c380696b0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:23.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.636+0000 7f0c5175b700 1 --2- 172.21.15.163:0/215148646 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c4c15d930 0x7f0c4c14db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:23.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.636+0000 7f0c5175b700 1 --2- 172.21.15.163:0/215148646 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c4c0cf480 0x7f0c4c169eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:23.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.636+0000 7f0c5175b700 1 --2- 172.21.15.163:0/215148646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c4c1595d0 0x7f0c4c14d380 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:23.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.636+0000 7f0c5175b700 1 -- 172.21.15.163:0/215148646 >> 172.21.15.163:0/215148646 conn(0x7f0c4c0d05f0 msgr2=0x7f0c4c153f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:23.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.636+0000 7f0c5175b700 1 -- 172.21.15.163:0/215148646 shutdown_connections 2023-10-09T06:36:23.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.636+0000 7f0c5175b700 1 -- 172.21.15.163:0/215148646 wait complete. 2023-10-09T06:36:23.638 INFO:teuthology.orchestra.run.smithi163.stderr:updated caps for client.0 2023-10-09T06:36:23.648 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:36:23.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.763+0000 7f3315881700 1 Processor -- start 2023-10-09T06:36:23.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.763+0000 7f3315881700 1 -- start start 2023-10-09T06:36:23.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.764+0000 7f3315881700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3310159890 0x7f331015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:23.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.764+0000 7f3315881700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3310159c60 0x7f331015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:23.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.764+0000 7f3315881700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f331015a160 con 0x7f331015db10 2023-10-09T06:36:23.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.764+0000 7f3315881700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f331015df50 con 0x7f3310159890 2023-10-09T06:36:23.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.764+0000 7f3315881700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33100d13a0 con 0x7f3310159c60 2023-10-09T06:36:23.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.764+0000 7f330effd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3310159890 0x7f331015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:23.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.764+0000 7f330f7fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f331015db10 0x7f3310069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33670/0 (socket says 172.21.15.163:33670) 2023-10-09T06:36:23.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.764+0000 7f330f7fe700 1 -- 172.21.15.163:0/1693353106 learned_addr learned my addr 172.21.15.163:0/1693353106 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:23.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.764+0000 7f330effd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3310159890 0x7f331015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:54942/0 (socket says 172.21.15.163:54942) 2023-10-09T06:36:23.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.764+0000 7f330effd700 1 -- 172.21.15.163:0/1693353106 >> v2:172.21.15.181:3300/0 conn(0x7f3310159c60 msgr2=0x7f331015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:23.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.764+0000 7f330e7fc700 1 --2- 172.21.15.163:0/1693353106 >> v2:172.21.15.181:3300/0 conn(0x7f3310159c60 0x7f331015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:23.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.764+0000 7f330effd700 1 --2- 172.21.15.163:0/1693353106 >> v2:172.21.15.181:3300/0 conn(0x7f3310159c60 0x7f331015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:23.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.764+0000 7f330dffb700 1 -- 172.21.15.163:0/1693353106 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3703658376 0 0) 0x7f331015a160 con 0x7f331015db10 2023-10-09T06:36:23.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.764+0000 7f330effd700 1 -- 172.21.15.163:0/1693353106 >> v1:172.21.15.163:6789/0 conn(0x7f331015db10 legacy=0x7f3310069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:23.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.764+0000 7f330effd700 1 -- 172.21.15.163:0/1693353106 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f331015e140 con 0x7f3310159890 2023-10-09T06:36:23.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.764+0000 7f330effd700 1 --2- 172.21.15.163:0/1693353106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3310159890 0x7f331015abd0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f33100b67f0 tx=0x7f3308009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de1cafa2e254967 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:23.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.764+0000 7f330dffb700 1 -- 172.21.15.163:0/1693353106 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330800e070 con 0x7f3310159890 2023-10-09T06:36:23.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.764+0000 7f330dffb700 1 -- 172.21.15.163:0/1693353106 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330800bc30 con 0x7f3310159890 2023-10-09T06:36:23.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.764+0000 7f330dffb700 1 -- 172.21.15.163:0/1693353106 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308012430 con 0x7f3310159890 2023-10-09T06:36:23.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.765+0000 7f3315881700 1 -- 172.21.15.163:0/1693353106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3310159890 msgr2=0x7f331015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:23.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.765+0000 7f3315881700 1 --2- 172.21.15.163:0/1693353106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3310159890 0x7f331015abd0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f33100b67f0 tx=0x7f3308009780 comp rx=0 tx=0).stop 2023-10-09T06:36:23.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.765+0000 7f3315881700 1 -- 172.21.15.163:0/1693353106 shutdown_connections 2023-10-09T06:36:23.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.765+0000 7f3315881700 1 --2- 172.21.15.163:0/1693353106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3310159890 0x7f331015abd0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:23.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.765+0000 7f3315881700 1 --2- 172.21.15.163:0/1693353106 >> v2:172.21.15.181:3300/0 conn(0x7f3310159c60 0x7f331015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:23.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.765+0000 7f3315881700 1 -- 172.21.15.163:0/1693353106 >> 172.21.15.163:0/1693353106 conn(0x7f3310068e20 msgr2=0x7f33100d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:23.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.765+0000 7f3315881700 1 -- 172.21.15.163:0/1693353106 shutdown_connections 2023-10-09T06:36:23.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.765+0000 7f3315881700 1 -- 172.21.15.163:0/1693353106 wait complete. 2023-10-09T06:36:23.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.766+0000 7f3315881700 1 Processor -- start 2023-10-09T06:36:23.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.766+0000 7f3315881700 1 -- start start 2023-10-09T06:36:23.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.767+0000 7f3315881700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3310159890 0x7f33100b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:23.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.767+0000 7f3315881700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3310159c60 0x7f331016b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:23.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.767+0000 7f3315881700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f331015db10 0x7f33100cf300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:23.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.767+0000 7f3315881700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33100cde30 con 0x7f3310159c60 2023-10-09T06:36:23.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.767+0000 7f3315881700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33100cd260 con 0x7f331015db10 2023-10-09T06:36:23.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.767+0000 7f3315881700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33100cd500 con 0x7f3310159890 2023-10-09T06:36:23.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.767+0000 7f330e7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3310159c60 0x7f331016b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:23.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.767+0000 7f330e7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3310159c60 0x7f331016b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:54954/0 (socket says 172.21.15.163:54954) 2023-10-09T06:36:23.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.767+0000 7f330e7fc700 1 -- 172.21.15.163:0/1110102764 learned_addr learned my addr 172.21.15.163:0/1110102764 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:23.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.767+0000 7f330effd700 1 --2- 172.21.15.163:0/1110102764 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3310159890 0x7f33100b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:23.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.767+0000 7f330f7fe700 1 --2- 172.21.15.163:0/1110102764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f331015db10 0x7f33100cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:23.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.768+0000 7f330e7fc700 1 -- 172.21.15.163:0/1110102764 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3310159890 msgr2=0x7f33100b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:23.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.768+0000 7f330e7fc700 1 --2- 172.21.15.163:0/1110102764 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3310159890 0x7f33100b58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:23.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.768+0000 7f330e7fc700 1 -- 172.21.15.163:0/1110102764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f331015db10 msgr2=0x7f33100cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:23.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.768+0000 7f330e7fc700 1 --2- 172.21.15.163:0/1110102764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f331015db10 0x7f33100cf300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:23.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.768+0000 7f330e7fc700 1 -- 172.21.15.163:0/1110102764 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3308009a20 con 0x7f3310159c60 2023-10-09T06:36:23.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.768+0000 7f330e7fc700 1 --2- 172.21.15.163:0/1110102764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3310159c60 0x7f331016b360 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f330400c7d0 tx=0x7f3304008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:23.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.768+0000 7f32f6ffd700 1 -- 172.21.15.163:0/1110102764 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3304011600 con 0x7f3310159c60 2023-10-09T06:36:23.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.768+0000 7f32f6ffd700 1 -- 172.21.15.163:0/1110102764 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3304011760 con 0x7f3310159c60 2023-10-09T06:36:23.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.768+0000 7f3315881700 1 -- 172.21.15.163:0/1110102764 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33100cd7e0 con 0x7f3310159c60 2023-10-09T06:36:23.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.768+0000 7f32f6ffd700 1 -- 172.21.15.163:0/1110102764 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3304011b20 con 0x7f3310159c60 2023-10-09T06:36:23.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.768+0000 7f3315881700 1 -- 172.21.15.163:0/1110102764 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33100cf840 con 0x7f3310159c60 2023-10-09T06:36:23.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.769+0000 7f3315881700 1 -- 172.21.15.163:0/1110102764 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33100cfac0 con 0x7f3310159c60 2023-10-09T06:36:23.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.769+0000 7f32f6ffd700 1 -- 172.21.15.163:0/1110102764 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f33040193f0 con 0x7f3310159c60 2023-10-09T06:36:23.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.770+0000 7f32f6ffd700 1 --2- 172.21.15.163:0/1110102764 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f32fc067200 0x7f32fc0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:23.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.770+0000 7f32f6ffd700 1 -- 172.21.15.163:0/1110102764 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3304084db0 con 0x7f3310159c60 2023-10-09T06:36:23.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.770+0000 7f330effd700 1 --2- 172.21.15.163:0/1110102764 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f32fc067200 0x7f32fc0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:23.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.770+0000 7f330effd700 1 --2- 172.21.15.163:0/1110102764 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f32fc067200 0x7f32fc0696b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f33100b67f0 tx=0x7f3308019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:23.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.774+0000 7f32f6ffd700 1 -- 172.21.15.163:0/1110102764 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3304055650 con 0x7f3310159c60 2023-10-09T06:36:23.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.999+0000 7f3315881700 1 -- 172.21.15.163:0/1110102764 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f331004c3c0 con 0x7f3310159c60 2023-10-09T06:36:23.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:23.999+0000 7f32f6ffd700 1 -- 172.21.15.163:0/1110102764 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f33040551e0 con 0x7f3310159c60 2023-10-09T06:36:23.999 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:23.999 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5501,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/862991395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":862991395},{"type":"v1","addr":"172.21.15.181:6836","nonce":862991395}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/699275566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":699275566},{"type":"v1","addr":"172.21.15.181:6837","nonce":699275566}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6504,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3664732208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3664732208},{"type":"v1","addr":"172.21.15.163:6834","nonce":3664732208}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:36:18.163421+0000","modified":"2023-10-09T06:36:21.793136+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6507},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6507":{"gid":6507,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/2687247401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2687247401},{"type":"v1","addr":"172.21.15.163:6835","nonce":2687247401}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-09T06:36:24.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.001+0000 7f3315881700 1 -- 172.21.15.163:0/1110102764 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f32fc067200 msgr2=0x7f32fc0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:24.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.001+0000 7f3315881700 1 --2- 172.21.15.163:0/1110102764 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f32fc067200 0x7f32fc0696b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f33100b67f0 tx=0x7f3308019000 comp rx=0 tx=0).stop 2023-10-09T06:36:24.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.001+0000 7f3315881700 1 -- 172.21.15.163:0/1110102764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3310159c60 msgr2=0x7f331016b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:24.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.001+0000 7f3315881700 1 --2- 172.21.15.163:0/1110102764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3310159c60 0x7f331016b360 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f330400c7d0 tx=0x7f3304008040 comp rx=0 tx=0).stop 2023-10-09T06:36:24.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.001+0000 7f3315881700 1 -- 172.21.15.163:0/1110102764 shutdown_connections 2023-10-09T06:36:24.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.001+0000 7f3315881700 1 --2- 172.21.15.163:0/1110102764 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f32fc067200 0x7f32fc0696b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.001+0000 7f3315881700 1 --2- 172.21.15.163:0/1110102764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f331015db10 0x7f33100cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.001+0000 7f3315881700 1 --2- 172.21.15.163:0/1110102764 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3310159890 0x7f33100b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.001+0000 7f3315881700 1 --2- 172.21.15.163:0/1110102764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3310159c60 0x7f331016b360 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.001+0000 7f3315881700 1 -- 172.21.15.163:0/1110102764 >> 172.21.15.163:0/1110102764 conn(0x7f3310068e20 msgr2=0x7f3310152fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:24.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.001+0000 7f3315881700 1 -- 172.21.15.163:0/1110102764 shutdown_connections 2023-10-09T06:36:24.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.001+0000 7f3315881700 1 -- 172.21.15.163:0/1110102764 wait complete. 2023-10-09T06:36:24.003 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 57 2023-10-09T06:36:24.416 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-09T06:36:18.163421+0000', 'modified': '2023-10-09T06:36:21.793136+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6507}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6507': {'gid': 6507, 'name': 'a', 'rank': 0, 'incarnation': 53, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.163:6835/2687247401', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6833', 'nonce': 2687247401}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 2687247401}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-09T06:36:24.416 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-09T06:36:24.416 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-09T06:36:24.528 INFO:tasks.ceph.mds.a.smithi163.stderr:2023-10-09T06:36:24.527+0000 7f7bf3de8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-09T06:36:24.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.537+0000 7f0e3c45a700 1 Processor -- start 2023-10-09T06:36:24.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.537+0000 7f0e3c45a700 1 -- start start 2023-10-09T06:36:24.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.538+0000 7f0e3c45a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e34069560 0x7f0e340699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:24.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.538+0000 7f0e3c45a700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e34069ee0 con 0x7f0e34159c60 2023-10-09T06:36:24.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.538+0000 7f0e3c45a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e3415a160 con 0x7f0e34159890 2023-10-09T06:36:24.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.538+0000 7f0e3c45a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e340d13a0 con 0x7f0e34069560 2023-10-09T06:36:24.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.538+0000 7f0e399f5700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f0e34159c60 0x7f0e3415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33682/0 (socket says 172.21.15.163:33682) 2023-10-09T06:36:24.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.538+0000 7f0e399f5700 1 -- 172.21.15.163:0/4199559170 learned_addr learned my addr 172.21.15.163:0/4199559170 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:24.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.539+0000 7f0e3a9f7700 1 --2- 172.21.15.163:0/4199559170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e34069560 0x7f0e340699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:24.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.539+0000 7f0e391f4700 1 -- 172.21.15.163:0/4199559170 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1069386825 0 0) 0x7f0e34069ee0 con 0x7f0e34159c60 2023-10-09T06:36:24.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.539+0000 7f0e391f4700 1 -- 172.21.15.163:0/4199559170 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e20003570 con 0x7f0e34159c60 2023-10-09T06:36:24.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.539+0000 7f0e391f4700 1 -- 172.21.15.163:0/4199559170 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3535382685 0 0) 0x7f0e3415a160 con 0x7f0e34159890 2023-10-09T06:36:24.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.539+0000 7f0e391f4700 1 -- 172.21.15.163:0/4199559170 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e34069ee0 con 0x7f0e34159890 2023-10-09T06:36:24.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.539+0000 7f0e391f4700 1 -- 172.21.15.163:0/4199559170 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1666463599 0 0) 0x7f0e20003570 con 0x7f0e34159c60 2023-10-09T06:36:24.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.539+0000 7f0e391f4700 1 -- 172.21.15.163:0/4199559170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e34069560 msgr2=0x7f0e340699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:24.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.539+0000 7f0e391f4700 1 --2- 172.21.15.163:0/4199559170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e34069560 0x7f0e340699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.539+0000 7f0e391f4700 1 -- 172.21.15.163:0/4199559170 >> v1:172.21.15.181:6789/0 conn(0x7f0e34159890 legacy=0x7f0e3415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:24.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.539+0000 7f0e391f4700 1 -- 172.21.15.163:0/4199559170 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3415e140 con 0x7f0e34159c60 2023-10-09T06:36:24.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.539+0000 7f0e391f4700 1 -- 172.21.15.163:0/4199559170 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f0e24001470 con 0x7f0e34159c60 2023-10-09T06:36:24.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.539+0000 7f0e391f4700 1 -- 172.21.15.163:0/4199559170 >> v1:172.21.15.163:6789/0 conn(0x7f0e34159c60 legacy=0x7f0e3415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:24.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.539+0000 7f0e391f4700 1 --2- 172.21.15.163:0/4199559170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e20003c20 0x7f0e20004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:24.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.539+0000 7f0e391f4700 1 --2- 172.21.15.163:0/4199559170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e34069560 0x7f0e20004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:24.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.539+0000 7f0e391f4700 1 --2- 172.21.15.163:0/4199559170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e20004d50 0x7f0e200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:24.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.539+0000 7f0e391f4700 1 -- 172.21.15.163:0/4199559170 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e340d13a0 con 0x7f0e20004d50 2023-10-09T06:36:24.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.539+0000 7f0e391f4700 1 -- 172.21.15.163:0/4199559170 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2000bb70 con 0x7f0e20003c20 2023-10-09T06:36:24.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.539+0000 7f0e391f4700 1 -- 172.21.15.163:0/4199559170 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2000bd30 con 0x7f0e34069560 2023-10-09T06:36:24.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.539+0000 7f0e3a9f7700 1 --2- 172.21.15.163:0/4199559170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e20004d50 0x7f0e200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:24.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.539+0000 7f0e399f5700 1 --2- 172.21.15.163:0/4199559170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e34069560 0x7f0e20004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:24.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.539+0000 7f0e3a1f6700 1 --2- 172.21.15.163:0/4199559170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e20003c20 0x7f0e20004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:24.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.539+0000 7f0e3a9f7700 1 -- 172.21.15.163:0/4199559170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e34069560 msgr2=0x7f0e20004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:24.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.539+0000 7f0e3a9f7700 1 --2- 172.21.15.163:0/4199559170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e34069560 0x7f0e20004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.539+0000 7f0e3a9f7700 1 -- 172.21.15.163:0/4199559170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e20003c20 msgr2=0x7f0e20004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:24.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.539+0000 7f0e3a9f7700 1 --2- 172.21.15.163:0/4199559170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e20003c20 0x7f0e20004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.539+0000 7f0e3a9f7700 1 -- 172.21.15.163:0/4199559170 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e20003570 con 0x7f0e20004d50 2023-10-09T06:36:24.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.540+0000 7f0e3a9f7700 1 --2- 172.21.15.163:0/4199559170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e20004d50 0x7f0e200051c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009c00 tx=0x7f0e2c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5771d14c850205e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:24.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.540+0000 7f0e391f4700 1 -- 172.21.15.163:0/4199559170 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c00e710 con 0x7f0e20004d50 2023-10-09T06:36:24.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.540+0000 7f0e391f4700 1 -- 172.21.15.163:0/4199559170 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2c00e870 con 0x7f0e20004d50 2023-10-09T06:36:24.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.540+0000 7f0e391f4700 1 -- 172.21.15.163:0/4199559170 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c00ec30 con 0x7f0e20004d50 2023-10-09T06:36:24.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.540+0000 7f0e3c45a700 1 -- 172.21.15.163:0/4199559170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e20004d50 msgr2=0x7f0e200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:24.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.540+0000 7f0e3c45a700 1 --2- 172.21.15.163:0/4199559170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e20004d50 0x7f0e200051c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009c00 tx=0x7f0e2c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.540+0000 7f0e3a9f7700 1 -- 172.21.15.163:0/4199559170 reap_dead start 2023-10-09T06:36:24.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.540+0000 7f0e3c45a700 1 -- 172.21.15.163:0/4199559170 shutdown_connections 2023-10-09T06:36:24.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.540+0000 7f0e3c45a700 1 -- 172.21.15.163:0/4199559170 >> 172.21.15.163:0/4199559170 conn(0x7f0e34068e20 msgr2=0x7f0e340d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:24.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.540+0000 7f0e3c45a700 1 -- 172.21.15.163:0/4199559170 shutdown_connections 2023-10-09T06:36:24.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.540+0000 7f0e3c45a700 1 -- 172.21.15.163:0/4199559170 wait complete. 2023-10-09T06:36:24.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.542+0000 7f0e3c45a700 1 Processor -- start 2023-10-09T06:36:24.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.542+0000 7f0e3c45a700 1 -- start start 2023-10-09T06:36:24.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.542+0000 7f0e3c45a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e20004d50 0x7f0e340b5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:24.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.542+0000 7f0e3c45a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e3415c650 0x7f0e3415cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:24.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.542+0000 7f0e3c45a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e34169ef0 0x7f0e3416a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:24.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.542+0000 7f0e3c45a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e340b6420 con 0x7f0e3415c650 2023-10-09T06:36:24.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.542+0000 7f0e3c45a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3415d000 con 0x7f0e20004d50 2023-10-09T06:36:24.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.542+0000 7f0e3c45a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e34153240 con 0x7f0e34169ef0 2023-10-09T06:36:24.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.542+0000 7f0e399f5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e3415c650 0x7f0e3415cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:24.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.542+0000 7f0e399f5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e3415c650 0x7f0e3415cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:54970/0 (socket says 172.21.15.163:54970) 2023-10-09T06:36:24.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.542+0000 7f0e399f5700 1 -- 172.21.15.163:0/3083183219 learned_addr learned my addr 172.21.15.163:0/3083183219 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:24.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.542+0000 7f0e3a9f7700 1 --2- 172.21.15.163:0/3083183219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e34169ef0 0x7f0e3416a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:24.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.542+0000 7f0e3a1f6700 1 --2- 172.21.15.163:0/3083183219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e20004d50 0x7f0e340b5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:24.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.543+0000 7f0e399f5700 1 -- 172.21.15.163:0/3083183219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e34169ef0 msgr2=0x7f0e3416a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:24.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.543+0000 7f0e399f5700 1 --2- 172.21.15.163:0/3083183219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e34169ef0 0x7f0e3416a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.543+0000 7f0e399f5700 1 -- 172.21.15.163:0/3083183219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e20004d50 msgr2=0x7f0e340b5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:24.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.543+0000 7f0e399f5700 1 --2- 172.21.15.163:0/3083183219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e20004d50 0x7f0e340b5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.543+0000 7f0e399f5700 1 -- 172.21.15.163:0/3083183219 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e341521a0 con 0x7f0e3415c650 2023-10-09T06:36:24.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.543+0000 7f0e3a1f6700 1 --2- 172.21.15.163:0/3083183219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e20004d50 0x7f0e340b5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:24.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.543+0000 7f0e399f5700 1 --2- 172.21.15.163:0/3083183219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e3415c650 0x7f0e3415cac0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0e240073c0 tx=0x7f0e24001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:24.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.543+0000 7f0e2a7fc700 1 -- 172.21.15.163:0/3083183219 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24018030 con 0x7f0e3415c650 2023-10-09T06:36:24.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.543+0000 7f0e2a7fc700 1 -- 172.21.15.163:0/3083183219 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2400f6c0 con 0x7f0e3415c650 2023-10-09T06:36:24.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.543+0000 7f0e3c45a700 1 -- 172.21.15.163:0/3083183219 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e34152480 con 0x7f0e3415c650 2023-10-09T06:36:24.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.543+0000 7f0e2a7fc700 1 -- 172.21.15.163:0/3083183219 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2400faa0 con 0x7f0e3415c650 2023-10-09T06:36:24.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.543+0000 7f0e3c45a700 1 -- 172.21.15.163:0/3083183219 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e34152a40 con 0x7f0e3415c650 2023-10-09T06:36:24.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.544+0000 7f0e2a7fc700 1 -- 172.21.15.163:0/3083183219 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f0e2401b070 con 0x7f0e3415c650 2023-10-09T06:36:24.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.545+0000 7f0e2a7fc700 1 --2- 172.21.15.163:0/3083183219 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0e18067230 0x7f0e180696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:24.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.545+0000 7f0e2a7fc700 1 -- 172.21.15.163:0/3083183219 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0e24019070 con 0x7f0e3415c650 2023-10-09T06:36:24.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.545+0000 7f0e3a1f6700 1 --2- 172.21.15.163:0/3083183219 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0e18067230 0x7f0e180696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:24.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.545+0000 7f0e3a1f6700 1 --2- 172.21.15.163:0/3083183219 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0e18067230 0x7f0e180696e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0e300013d0 tx=0x7f0e30009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:24.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.547+0000 7f0e45752b80 1 Processor -- start 2023-10-09T06:36:24.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.547+0000 7f0e45752b80 1 -- start start 2023-10-09T06:36:24.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.547+0000 7f0e45752b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069efb9a50 0x56069efb9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:24.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.547+0000 7f0e45752b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56069f003d00 0x56069efe2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:24.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.547+0000 7f0e45752b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56069efe2f60 0x56069efb5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:24.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.547+0000 7f0e45752b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069ef10a70 con 0x56069f003d00 2023-10-09T06:36:24.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.547+0000 7f0e45752b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069ef0e270 con 0x56069efb9a50 2023-10-09T06:36:24.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.547+0000 7f0e45752b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x56069ef0a060 con 0x56069efe2f60 2023-10-09T06:36:24.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.547+0000 7f0e399f5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56069f003d00 0x56069efe2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:24.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.547+0000 7f0e399f5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56069f003d00 0x56069efe2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:54980/0 (socket says 172.21.15.163:54980) 2023-10-09T06:36:24.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.548+0000 7f0e399f5700 1 -- 172.21.15.163:0/794577426 learned_addr learned my addr 172.21.15.163:0/794577426 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:24.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.548+0000 7f0e3a1f6700 1 --2- 172.21.15.163:0/794577426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069efb9a50 0x56069efb9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:24.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.548+0000 7f0e3a9f7700 1 --2- 172.21.15.163:0/794577426 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56069efe2f60 0x56069efb5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:24.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.548+0000 7f0e399f5700 1 -- 172.21.15.163:0/794577426 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56069efe2f60 msgr2=0x56069efb5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:24.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.548+0000 7f0e399f5700 1 --2- 172.21.15.163:0/794577426 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56069efe2f60 0x56069efb5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.548+0000 7f0e399f5700 1 -- 172.21.15.163:0/794577426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069efb9a50 msgr2=0x56069efb9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:24.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.548+0000 7f0e399f5700 1 --2- 172.21.15.163:0/794577426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069efb9a50 0x56069efb9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.548+0000 7f0e399f5700 1 -- 172.21.15.163:0/794577426 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56069eda4270 con 0x56069f003d00 2023-10-09T06:36:24.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.548+0000 7f0e399f5700 1 --2- 172.21.15.163:0/794577426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56069f003d00 0x56069efe2a20 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f0e24054f80 tx=0x7f0e24009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9ea27a4900154e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:24.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.548+0000 7f0e177fe700 1 -- 172.21.15.163:0/794577426 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24055280 con 0x56069f003d00 2023-10-09T06:36:24.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.548+0000 7f0e177fe700 1 -- 172.21.15.163:0/794577426 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e240584d0 con 0x56069f003d00 2023-10-09T06:36:24.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.548+0000 7f0e177fe700 1 -- 172.21.15.163:0/794577426 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2405c220 con 0x56069f003d00 2023-10-09T06:36:24.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.549+0000 7f0e45752b80 1 -- 172.21.15.163:0/794577426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56069f003d00 msgr2=0x56069efe2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:24.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.549+0000 7f0e45752b80 1 --2- 172.21.15.163:0/794577426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56069f003d00 0x56069efe2a20 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f0e24054f80 tx=0x7f0e24009040 comp rx=0 tx=0).stop 2023-10-09T06:36:24.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.549+0000 7f0e45752b80 1 -- 172.21.15.163:0/794577426 shutdown_connections 2023-10-09T06:36:24.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.549+0000 7f0e45752b80 1 --2- 172.21.15.163:0/794577426 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56069efe2f60 0x56069efb5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.549+0000 7f0e45752b80 1 --2- 172.21.15.163:0/794577426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069efb9a50 0x56069efb9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.549+0000 7f0e45752b80 1 --2- 172.21.15.163:0/794577426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56069f003d00 0x56069efe2a20 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.549+0000 7f0e45752b80 1 -- 172.21.15.163:0/794577426 >> 172.21.15.163:0/794577426 conn(0x56069efbb370 msgr2=0x56069efbb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:24.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.549+0000 7f0e45752b80 1 -- 172.21.15.163:0/794577426 shutdown_connections 2023-10-09T06:36:24.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.549+0000 7f0e45752b80 1 -- 172.21.15.163:0/794577426 wait complete. 2023-10-09T06:36:24.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.549+0000 7f0e45752b80 1 -- start start 2023-10-09T06:36:24.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.549+0000 7f0e45752b80 1 Processor -- start 2023-10-09T06:36:24.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.550+0000 7f0e45752b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56069efb9a50 0x56069f012f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:24.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.550+0000 7f0e45752b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56069efe2f60 0x56069f002900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:24.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.550+0000 7f0e45752b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069f003d00 0x56069eff9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:24.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.550+0000 7f0e45752b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069eeb5890 con 0x56069efe2f60 2023-10-09T06:36:24.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.550+0000 7f0e45752b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069eed0ea0 con 0x56069f003d00 2023-10-09T06:36:24.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.550+0000 7f0e45752b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x56069ed8ce30 con 0x56069efb9a50 2023-10-09T06:36:24.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.550+0000 7f0e45752b80 10 client.0 _ll_register_callbacks cb 0x56069ef7d110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-09T06:36:24.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.550+0000 7f0e399f5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56069efe2f60 0x56069f002900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:24.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.550+0000 7f0e399f5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56069efe2f60 0x56069f002900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:54994/0 (socket says 172.21.15.163:54994) 2023-10-09T06:36:24.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.550+0000 7f0e399f5700 1 -- 172.21.15.163:0/3538087425 learned_addr learned my addr 172.21.15.163:0/3538087425 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:24.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.551+0000 7f0e3a9f7700 1 --2- 172.21.15.163:0/3538087425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069f003d00 0x56069eff9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:24.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.551+0000 7f0e3a1f6700 1 --2- 172.21.15.163:0/3538087425 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56069efb9a50 0x56069f012f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:24.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.551+0000 7f0e399f5700 1 -- 172.21.15.163:0/3538087425 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56069efb9a50 msgr2=0x56069f012f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:24.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.551+0000 7f0e399f5700 1 --2- 172.21.15.163:0/3538087425 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56069efb9a50 0x56069f012f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.551+0000 7f0e399f5700 1 -- 172.21.15.163:0/3538087425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069f003d00 msgr2=0x56069eff9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:24.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.551+0000 7f0e399f5700 1 --2- 172.21.15.163:0/3538087425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069f003d00 0x56069eff9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.551+0000 7f0e399f5700 1 -- 172.21.15.163:0/3538087425 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e34152a40 con 0x56069efe2f60 2023-10-09T06:36:24.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.551+0000 7f0e399f5700 1 --2- 172.21.15.163:0/3538087425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56069efe2f60 0x56069f002900 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f0e24054f80 tx=0x7f0e24068000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6bab7995df06e17 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:24.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.551+0000 7f0e14ff9700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-10-09T06:36:24.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.551+0000 7f0e14ff9700 1 -- 172.21.15.163:0/3538087425 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24070070 con 0x56069efe2f60 2023-10-09T06:36:24.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.551+0000 7f0e14ff9700 1 -- 172.21.15.163:0/3538087425 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0e2405c220 con 0x56069efe2f60 2023-10-09T06:36:24.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.551+0000 7f0e45752b80 1 -- 172.21.15.163:0/3538087425 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56069efaeb70 con 0x56069efe2f60 2023-10-09T06:36:24.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.551+0000 7f0e14ff9700 1 -- 172.21.15.163:0/3538087425 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e24054980 con 0x56069efe2f60 2023-10-09T06:36:24.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.551+0000 7f0e45752b80 10 client.6651 fetch_fsmap learned FSMap version 57 2023-10-09T06:36:24.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.551+0000 7f0e45752b80 1 -- 172.21.15.163:0/3538087425 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x56069ed46fe0 con 0x56069efe2f60 2023-10-09T06:36:24.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.551+0000 7f0e14ff9700 1 -- 172.21.15.163:0/3538087425 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56069efaeb70 con 0x56069efe2f60 2023-10-09T06:36:24.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.551+0000 7f0e14ff9700 1 -- 172.21.15.163:0/3538087425 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 57) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f0e2405bd90 con 0x56069efe2f60 2023-10-09T06:36:24.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.551+0000 7f0e45752b80 10 client.6651 fetch_fsmap finished waiting for FSMap version 57 2023-10-09T06:36:24.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.551+0000 7f0e45752b80 10 client.6651 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-09T06:36:24.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.551+0000 7f0e45752b80 20 client.6651 populate_metadata read hostname 'smithi163' 2023-10-09T06:36:24.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.551+0000 7f0e45752b80 1 --2- 172.21.15.163:0/3538087425 >> [v2:172.21.15.163:6833/2687247401,v1:172.21.15.163:6835/2687247401] conn(0x56069effd7f0 0x56069effdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:24.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.551+0000 7f0e45752b80 4 client.6651 mds_command: new command op to 6507 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-09T06:36:24.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.551+0000 7f0e45752b80 1 -- 172.21.15.163:0/3538087425 --> [v2:172.21.15.163:6833/2687247401,v1:172.21.15.163:6835/2687247401] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56069ef8baa0 con 0x56069effd7f0 2023-10-09T06:36:24.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.551+0000 7f0e3a1f6700 1 --2- 172.21.15.163:0/3538087425 >> [v2:172.21.15.163:6833/2687247401,v1:172.21.15.163:6835/2687247401] conn(0x56069effd7f0 0x56069effdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:24.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.552+0000 7f0e3a1f6700 1 --2- 172.21.15.163:0/3538087425 >> [v2:172.21.15.163:6833/2687247401,v1:172.21.15.163:6835/2687247401] conn(0x56069effd7f0 0x56069effdbc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=762eaabb0adccc86 server_cookie=73cce0054fdd511e in_seq=0 out_seq=0 2023-10-09T06:36:24.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.552+0000 7f0e14ff9700 10 client.6651 ms_handle_connect on v2:172.21.15.163:6833/2687247401 2023-10-09T06:36:24.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.552+0000 7f0e14ff9700 1 -- 172.21.15.163:0/3538087425 <== mds.0 v2:172.21.15.163:6833/2687247401 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x56069ef8baa0 con 0x56069effd7f0 2023-10-09T06:36:24.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.552+0000 7f0e14ff9700 10 client.6651 handle_command_reply: tid=0 2023-10-09T06:36:24.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.552+0000 7f0e45752b80 1 client.6651 shutdown 2023-10-09T06:36:24.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.552+0000 7f0e45752b80 1 -- 172.21.15.163:0/3538087425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56069efe2f60 msgr2=0x56069f002900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:24.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.552+0000 7f0e45752b80 1 --2- 172.21.15.163:0/3538087425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56069efe2f60 0x56069f002900 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f0e24054f80 tx=0x7f0e24068000 comp rx=0 tx=0).stop 2023-10-09T06:36:24.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.552+0000 7f0e45752b80 1 -- 172.21.15.163:0/3538087425 shutdown_connections 2023-10-09T06:36:24.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.552+0000 7f0e45752b80 1 --2- 172.21.15.163:0/3538087425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069f003d00 0x56069eff9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.552+0000 7f0e45752b80 1 --2- 172.21.15.163:0/3538087425 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56069efb9a50 0x56069f012f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.552+0000 7f0e45752b80 1 --2- 172.21.15.163:0/3538087425 >> [v2:172.21.15.163:6833/2687247401,v1:172.21.15.163:6835/2687247401] conn(0x56069effd7f0 0x56069effdbc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.552+0000 7f0e45752b80 1 --2- 172.21.15.163:0/3538087425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56069efe2f60 0x56069f002900 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.552+0000 7f0e14ff9700 0 client.6651 ms_handle_reset on v2:172.21.15.163:6833/2687247401 2023-10-09T06:36:24.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.552+0000 7f0e45752b80 1 -- 172.21.15.163:0/3538087425 >> 172.21.15.163:0/3538087425 conn(0x56069efbb370 msgr2=0x56069efb6980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:24.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.553+0000 7f0e45752b80 1 -- 172.21.15.163:0/3538087425 shutdown_connections 2023-10-09T06:36:24.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.553+0000 7f0e45752b80 1 -- 172.21.15.163:0/3538087425 wait complete. 2023-10-09T06:36:24.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.553+0000 7f0e45752b80 20 client.6651 trim_cache size 0 max 16384 2023-10-09T06:36:24.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.569+0000 7f0e45752b80 1 Processor -- start 2023-10-09T06:36:24.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.569+0000 7f0e45752b80 1 -- start start 2023-10-09T06:36:24.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.570+0000 7f0e45752b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56069efb9a50 0x56069f05fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:24.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.570+0000 7f0e45752b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56069efe2f60 0x56069efb5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:24.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.570+0000 7f0e45752b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069f003d00 0x56069efb5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:24.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.570+0000 7f0e45752b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069f001d10 con 0x56069efe2f60 2023-10-09T06:36:24.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.570+0000 7f0e45752b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069efb54f0 con 0x56069f003d00 2023-10-09T06:36:24.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.570+0000 7f0e45752b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x56069eeb5810 con 0x56069efb9a50 2023-10-09T06:36:24.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.570+0000 7f0e399f5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56069efe2f60 0x56069efb5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:24.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.570+0000 7f0e399f5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56069efe2f60 0x56069efb5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55002/0 (socket says 172.21.15.163:55002) 2023-10-09T06:36:24.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.570+0000 7f0e399f5700 1 -- 172.21.15.163:0/831592917 learned_addr learned my addr 172.21.15.163:0/831592917 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:24.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.570+0000 7f0e3a1f6700 1 --2- 172.21.15.163:0/831592917 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56069efb9a50 0x56069f05fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:24.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.570+0000 7f0e399f5700 1 -- 172.21.15.163:0/831592917 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56069efb9a50 msgr2=0x56069f05fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:24.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.570+0000 7f0e3a9f7700 1 --2- 172.21.15.163:0/831592917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069f003d00 0x56069efb5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:24.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.570+0000 7f0e399f5700 1 --2- 172.21.15.163:0/831592917 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56069efb9a50 0x56069f05fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.570+0000 7f0e399f5700 1 -- 172.21.15.163:0/831592917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069f003d00 msgr2=0x56069efb5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:24.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.570+0000 7f0e399f5700 1 --2- 172.21.15.163:0/831592917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069f003d00 0x56069efb5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.570+0000 7f0e399f5700 1 -- 172.21.15.163:0/831592917 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56069eed8780 con 0x56069efe2f60 2023-10-09T06:36:24.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.571+0000 7f0e399f5700 1 --2- 172.21.15.163:0/831592917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56069efe2f60 0x56069efb5920 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f0e240016b0 tx=0x7f0e24070000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9fce8d40e438b6e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:24.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.571+0000 7f0e17fff700 1 -- 172.21.15.163:0/831592917 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24054980 con 0x56069efe2f60 2023-10-09T06:36:24.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.571+0000 7f0e17fff700 1 -- 172.21.15.163:0/831592917 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2405c220 con 0x56069efe2f60 2023-10-09T06:36:24.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.571+0000 7f0e17fff700 1 -- 172.21.15.163:0/831592917 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2405bd90 con 0x56069efe2f60 2023-10-09T06:36:24.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.571+0000 7f0e45752b80 1 -- 172.21.15.163:0/831592917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56069efe2f60 msgr2=0x56069efb5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:24.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.571+0000 7f0e45752b80 1 --2- 172.21.15.163:0/831592917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56069efe2f60 0x56069efb5920 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f0e240016b0 tx=0x7f0e24070000 comp rx=0 tx=0).stop 2023-10-09T06:36:24.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.571+0000 7f0e45752b80 1 -- 172.21.15.163:0/831592917 shutdown_connections 2023-10-09T06:36:24.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.571+0000 7f0e45752b80 1 --2- 172.21.15.163:0/831592917 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069f003d00 0x56069efb5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.571+0000 7f0e45752b80 1 --2- 172.21.15.163:0/831592917 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56069efb9a50 0x56069f05fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.571+0000 7f0e45752b80 1 --2- 172.21.15.163:0/831592917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56069efe2f60 0x56069efb5920 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.571+0000 7f0e45752b80 1 -- 172.21.15.163:0/831592917 >> 172.21.15.163:0/831592917 conn(0x56069efbb370 msgr2=0x56069f0026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:24.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.571+0000 7f0e45752b80 1 -- 172.21.15.163:0/831592917 shutdown_connections 2023-10-09T06:36:24.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.571+0000 7f0e45752b80 1 -- 172.21.15.163:0/831592917 wait complete. 2023-10-09T06:36:24.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.572+0000 7f0e45752b80 1 -- start start 2023-10-09T06:36:24.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.572+0000 7f0e45752b80 1 Processor -- start 2023-10-09T06:36:24.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.573+0000 7f0e45752b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56069efb9a50 0x56069f05ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:24.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.573+0000 7f0e45752b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069efe2f60 0x56069effdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:24.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.573+0000 7f0e45752b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56069f003d00 0x56069efb58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:24.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.573+0000 7f0e45752b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069f0670e0 con 0x56069f003d00 2023-10-09T06:36:24.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.573+0000 7f0e45752b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069effe2c0 con 0x56069efe2f60 2023-10-09T06:36:24.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.573+0000 7f0e45752b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x56069eff9220 con 0x56069efb9a50 2023-10-09T06:36:24.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.573+0000 7f0e45752b80 10 client.0 _ll_register_callbacks cb 0x56069ef7d110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-09T06:36:24.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.573+0000 7f0e3a9f7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56069f003d00 0x56069efb58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:24.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.573+0000 7f0e3a9f7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56069f003d00 0x56069efb58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55010/0 (socket says 172.21.15.163:55010) 2023-10-09T06:36:24.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.573+0000 7f0e3a9f7700 1 -- 172.21.15.163:0/3116558390 learned_addr learned my addr 172.21.15.163:0/3116558390 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:24.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.573+0000 7f0e3a1f6700 1 --2- 172.21.15.163:0/3116558390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56069efb9a50 0x56069f05ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:24.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.573+0000 7f0e399f5700 1 --2- 172.21.15.163:0/3116558390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069efe2f60 0x56069effdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:24.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.573+0000 7f0e3a9f7700 1 -- 172.21.15.163:0/3116558390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56069efb9a50 msgr2=0x56069f05ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:24.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.573+0000 7f0e3a9f7700 1 --2- 172.21.15.163:0/3116558390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56069efb9a50 0x56069f05ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.573+0000 7f0e3a9f7700 1 -- 172.21.15.163:0/3116558390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069efe2f60 msgr2=0x56069effdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:24.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.573+0000 7f0e3a9f7700 1 --2- 172.21.15.163:0/3116558390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069efe2f60 0x56069effdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.573+0000 7f0e3a9f7700 1 -- 172.21.15.163:0/3116558390 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56069ed46fe0 con 0x56069f003d00 2023-10-09T06:36:24.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.573+0000 7f0e3a9f7700 1 --2- 172.21.15.163:0/3116558390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56069f003d00 0x56069efb58d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000bb40 tx=0x7f0e2c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824a7473431ed36e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:24.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.573+0000 7f0e14ff9700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-10-09T06:36:24.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.573+0000 7f0e14ff9700 1 -- 172.21.15.163:0/3116558390 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c022040 con 0x56069f003d00 2023-10-09T06:36:24.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.574+0000 7f0e45752b80 1 -- 172.21.15.163:0/3116558390 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e240584d0 con 0x56069f003d00 2023-10-09T06:36:24.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.574+0000 7f0e14ff9700 1 -- 172.21.15.163:0/3116558390 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0e2c000e40 con 0x56069f003d00 2023-10-09T06:36:24.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.574+0000 7f0e14ff9700 1 -- 172.21.15.163:0/3116558390 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e2c0095b0 con 0x56069f003d00 2023-10-09T06:36:24.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.574+0000 7f0e45752b80 10 client.6657 fetch_fsmap learned FSMap version 57 2023-10-09T06:36:24.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.574+0000 7f0e45752b80 1 -- 172.21.15.163:0/3116558390 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x56069eebf470 con 0x56069f003d00 2023-10-09T06:36:24.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.574+0000 7f0e14ff9700 1 -- 172.21.15.163:0/3116558390 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e240584d0 con 0x56069f003d00 2023-10-09T06:36:24.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.574+0000 7f0e14ff9700 1 -- 172.21.15.163:0/3116558390 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 57) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f0e2c011070 con 0x56069f003d00 2023-10-09T06:36:24.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.574+0000 7f0e45752b80 10 client.6657 fetch_fsmap finished waiting for FSMap version 57 2023-10-09T06:36:24.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.574+0000 7f0e45752b80 10 client.6657 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-09T06:36:24.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.574+0000 7f0e45752b80 20 client.6657 populate_metadata read hostname 'smithi163' 2023-10-09T06:36:24.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.574+0000 7f0e45752b80 1 --2- 172.21.15.163:0/3116558390 >> [v2:172.21.15.163:6833/2687247401,v1:172.21.15.163:6835/2687247401] conn(0x56069f0193d0 0x56069f0197a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:24.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.574+0000 7f0e45752b80 4 client.6657 mds_command: new command op to 6507 tid=0[{"prefix": "status"}] 2023-10-09T06:36:24.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.574+0000 7f0e45752b80 1 -- 172.21.15.163:0/3116558390 --> [v2:172.21.15.163:6833/2687247401,v1:172.21.15.163:6835/2687247401] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56069eeace00 con 0x56069f0193d0 2023-10-09T06:36:24.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.574+0000 7f0e3a1f6700 1 --2- 172.21.15.163:0/3116558390 >> [v2:172.21.15.163:6833/2687247401,v1:172.21.15.163:6835/2687247401] conn(0x56069f0193d0 0x56069f0197a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:24.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.574+0000 7f0e14ff9700 10 client.6657 ms_handle_connect on v2:172.21.15.163:6833/2687247401 2023-10-09T06:36:24.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.574+0000 7f0e3a1f6700 1 --2- 172.21.15.163:0/3116558390 >> [v2:172.21.15.163:6833/2687247401,v1:172.21.15.163:6835/2687247401] conn(0x56069f0193d0 0x56069f0197a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2fc5cb540672649 server_cookie=c671a15d8271abc in_seq=0 out_seq=0 2023-10-09T06:36:24.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.574+0000 7f0e14ff9700 1 -- 172.21.15.163:0/3116558390 <== mds.0 v2:172.21.15.163:6833/2687247401 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x56069eeace00 con 0x56069f0193d0 2023-10-09T06:36:24.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.574+0000 7f0e14ff9700 10 client.6657 handle_command_reply: tid=0 2023-10-09T06:36:24.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.575+0000 7f0e45752b80 1 client.6657 shutdown 2023-10-09T06:36:24.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.575+0000 7f0e45752b80 1 -- 172.21.15.163:0/3116558390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56069f003d00 msgr2=0x56069efb58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:24.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.575+0000 7f0e45752b80 1 --2- 172.21.15.163:0/3116558390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56069f003d00 0x56069efb58d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000bb40 tx=0x7f0e2c01a000 comp rx=0 tx=0).stop 2023-10-09T06:36:24.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.575+0000 7f0e45752b80 1 -- 172.21.15.163:0/3116558390 shutdown_connections 2023-10-09T06:36:24.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.575+0000 7f0e45752b80 1 --2- 172.21.15.163:0/3116558390 >> [v2:172.21.15.163:6833/2687247401,v1:172.21.15.163:6835/2687247401] conn(0x56069f0193d0 0x56069f0197a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.575+0000 7f0e45752b80 1 --2- 172.21.15.163:0/3116558390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56069f003d00 0x56069efb58d0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.575+0000 7f0e45752b80 1 --2- 172.21.15.163:0/3116558390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56069efb9a50 0x56069f05ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.575+0000 7f0e14ff9700 0 client.6657 ms_handle_reset on v2:172.21.15.163:6833/2687247401 2023-10-09T06:36:24.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.575+0000 7f0e45752b80 1 --2- 172.21.15.163:0/3116558390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56069efe2f60 0x56069effdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.575+0000 7f0e45752b80 1 -- 172.21.15.163:0/3116558390 >> 172.21.15.163:0/3116558390 conn(0x56069efbb370 msgr2=0x56069efe9360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:24.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.575+0000 7f0e45752b80 1 -- 172.21.15.163:0/3116558390 shutdown_connections 2023-10-09T06:36:24.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.575+0000 7f0e45752b80 1 -- 172.21.15.163:0/3116558390 wait complete. 2023-10-09T06:36:24.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.575+0000 7f0e45752b80 20 client.6657 trim_cache size 0 max 16384 2023-10-09T06:36:24.586 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:36:24.586 INFO:teuthology.orchestra.run.smithi163.stdout: "cluster_fsid": "2520b8f7-539f-4bae-9f17-1fce66f66ddd", 2023-10-09T06:36:24.586 INFO:teuthology.orchestra.run.smithi163.stdout: "whoami": 0, 2023-10-09T06:36:24.586 INFO:teuthology.orchestra.run.smithi163.stdout: "id": 6507, 2023-10-09T06:36:24.586 INFO:teuthology.orchestra.run.smithi163.stdout: "want_state": "up:active", 2023-10-09T06:36:24.586 INFO:teuthology.orchestra.run.smithi163.stdout: "state": "up:active", 2023-10-09T06:36:24.587 INFO:teuthology.orchestra.run.smithi163.stdout: "fs_name": "cephfs", 2023-10-09T06:36:24.587 INFO:teuthology.orchestra.run.smithi163.stdout: "rank_uptime": 5.7991235579999998, 2023-10-09T06:36:24.587 INFO:teuthology.orchestra.run.smithi163.stdout: "mdsmap_epoch": 57, 2023-10-09T06:36:24.587 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch": 89, 2023-10-09T06:36:24.587 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch_barrier": 89, 2023-10-09T06:36:24.587 INFO:teuthology.orchestra.run.smithi163.stdout: "uptime": 10.916014292 2023-10-09T06:36:24.587 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:36:24.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.575+0000 7f0e3c45a700 1 -- 172.21.15.163:0/3083183219 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0e18067230 msgr2=0x7f0e180696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:24.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.575+0000 7f0e3c45a700 1 --2- 172.21.15.163:0/3083183219 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0e18067230 0x7f0e180696e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0e300013d0 tx=0x7f0e30009040 comp rx=0 tx=0).stop 2023-10-09T06:36:24.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.575+0000 7f0e3c45a700 1 -- 172.21.15.163:0/3083183219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e3415c650 msgr2=0x7f0e3415cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:24.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.575+0000 7f0e3c45a700 1 --2- 172.21.15.163:0/3083183219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e3415c650 0x7f0e3415cac0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0e240073c0 tx=0x7f0e24001e10 comp rx=0 tx=0).stop 2023-10-09T06:36:24.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.575+0000 7f0e3c45a700 1 -- 172.21.15.163:0/3083183219 shutdown_connections 2023-10-09T06:36:24.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.575+0000 7f0e3c45a700 1 --2- 172.21.15.163:0/3083183219 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0e18067230 0x7f0e180696e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.575+0000 7f0e3c45a700 1 --2- 172.21.15.163:0/3083183219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e34169ef0 0x7f0e3416a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.575+0000 7f0e3c45a700 1 --2- 172.21.15.163:0/3083183219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e20004d50 0x7f0e340b5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.575+0000 7f0e3c45a700 1 --2- 172.21.15.163:0/3083183219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e3415c650 0x7f0e3415cac0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.575+0000 7f0e3c45a700 1 -- 172.21.15.163:0/3083183219 >> 172.21.15.163:0/3083183219 conn(0x7f0e34068e20 msgr2=0x7f0e340b2990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:24.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.575+0000 7f0e3c45a700 1 -- 172.21.15.163:0/3083183219 shutdown_connections 2023-10-09T06:36:24.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.575+0000 7f0e3c45a700 1 -- 172.21.15.163:0/3083183219 wait complete. 2023-10-09T06:36:24.591 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-09T06:36:24.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.704+0000 7fe0a7140700 1 Processor -- start 2023-10-09T06:36:24.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.704+0000 7fe0a7140700 1 -- start start 2023-10-09T06:36:24.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.705+0000 7fe0a7140700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe0a00aca80 0x7fe0a006aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:24.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.705+0000 7fe0a7140700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0a0069d60 0x7fe0a00c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:24.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.705+0000 7fe0a7140700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0a006bdb0 con 0x7fe0a00c81f0 2023-10-09T06:36:24.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.705+0000 7fe0a7140700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0a00c8670 con 0x7fe0a00aca80 2023-10-09T06:36:24.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.705+0000 7fe0a7140700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a00cb370 con 0x7fe0a0069d60 2023-10-09T06:36:24.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.706+0000 7fe0a4edc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe0a00aca80 0x7fe0a006aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:24.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.706+0000 7fe097fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0a0069d60 0x7fe0a00c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:24.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.706+0000 7fe0a4edc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe0a00aca80 0x7fe0a006aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:34734/0 (socket says 172.21.15.163:34734) 2023-10-09T06:36:24.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.706+0000 7fe0a4edc700 1 -- 172.21.15.163:0/2801490290 learned_addr learned my addr 172.21.15.163:0/2801490290 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:24.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.706+0000 7fe0a4edc700 1 -- 172.21.15.163:0/2801490290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0a0069d60 msgr2=0x7fe0a00c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:24.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.706+0000 7fe0a4edc700 1 --2- 172.21.15.163:0/2801490290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0a0069d60 0x7fe0a00c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.706+0000 7fe0a4edc700 1 -- 172.21.15.163:0/2801490290 >> v1:172.21.15.181:6789/0 conn(0x7fe0a00c81f0 legacy=0x7fe0a0068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:24.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.706+0000 7fe0a4edc700 1 -- 172.21.15.163:0/2801490290 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a0159f60 con 0x7fe0a00aca80 2023-10-09T06:36:24.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.706+0000 7fe0a4edc700 1 --2- 172.21.15.163:0/2801490290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0a00aca80 0x7fe0a006aba0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe09c009f70 tx=0x7fe09c009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=805ed336217ffed4 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:24.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.706+0000 7fe0977fe700 1 -- 172.21.15.163:0/2801490290 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09c00e070 con 0x7fe0a00aca80 2023-10-09T06:36:24.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.706+0000 7fe0977fe700 1 -- 172.21.15.163:0/2801490290 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09c00ba20 con 0x7fe0a00aca80 2023-10-09T06:36:24.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.706+0000 7fe0977fe700 1 -- 172.21.15.163:0/2801490290 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09c0092e0 con 0x7fe0a00aca80 2023-10-09T06:36:24.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.707+0000 7fe0a7140700 1 -- 172.21.15.163:0/2801490290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0a00aca80 msgr2=0x7fe0a006aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:24.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.707+0000 7fe0a7140700 1 --2- 172.21.15.163:0/2801490290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0a00aca80 0x7fe0a006aba0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe09c009f70 tx=0x7fe09c009840 comp rx=0 tx=0).stop 2023-10-09T06:36:24.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.707+0000 7fe0a7140700 1 -- 172.21.15.163:0/2801490290 shutdown_connections 2023-10-09T06:36:24.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.707+0000 7fe0a7140700 1 --2- 172.21.15.163:0/2801490290 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0a00aca80 0x7fe0a006aba0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.707+0000 7fe0a7140700 1 --2- 172.21.15.163:0/2801490290 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0a0069d60 0x7fe0a00c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.707+0000 7fe0a7140700 1 -- 172.21.15.163:0/2801490290 >> 172.21.15.163:0/2801490290 conn(0x7fe0a00cd670 msgr2=0x7fe0a00cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:24.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.707+0000 7fe0a7140700 1 -- 172.21.15.163:0/2801490290 shutdown_connections 2023-10-09T06:36:24.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.707+0000 7fe0a7140700 1 -- 172.21.15.163:0/2801490290 wait complete. 2023-10-09T06:36:24.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.708+0000 7fe0a7140700 1 Processor -- start 2023-10-09T06:36:24.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.708+0000 7fe0a7140700 1 -- start start 2023-10-09T06:36:24.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.709+0000 7fe0a7140700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0a0069d60 0x7fe0a0152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:24.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.709+0000 7fe0a7140700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0a00aca80 0x7fe0a00b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:24.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.709+0000 7fe0a7140700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0a00c81f0 0x7fe0a00af100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:24.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.709+0000 7fe0a7140700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a00ad3f0 con 0x7fe0a00c81f0 2023-10-09T06:36:24.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.709+0000 7fe0a7140700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a00ad6d0 con 0x7fe0a00aca80 2023-10-09T06:36:24.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.709+0000 7fe0a7140700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a00b2ab0 con 0x7fe0a0069d60 2023-10-09T06:36:24.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.709+0000 7fe0a56dd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0a00c81f0 0x7fe0a00af100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:24.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.709+0000 7fe0a56dd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0a00c81f0 0x7fe0a00af100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55020/0 (socket says 172.21.15.163:55020) 2023-10-09T06:36:24.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.709+0000 7fe0a56dd700 1 -- 172.21.15.163:0/288714588 learned_addr learned my addr 172.21.15.163:0/288714588 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:24.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.709+0000 7fe097fff700 1 --2- 172.21.15.163:0/288714588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0a00aca80 0x7fe0a00b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:24.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.709+0000 7fe0a4edc700 1 --2- 172.21.15.163:0/288714588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0a0069d60 0x7fe0a0152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:24.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.710+0000 7fe0a56dd700 1 -- 172.21.15.163:0/288714588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0a0069d60 msgr2=0x7fe0a0152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:24.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.710+0000 7fe0a56dd700 1 --2- 172.21.15.163:0/288714588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0a0069d60 0x7fe0a0152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.710+0000 7fe0a56dd700 1 -- 172.21.15.163:0/288714588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0a00aca80 msgr2=0x7fe0a00b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:24.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.710+0000 7fe0a56dd700 1 --2- 172.21.15.163:0/288714588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0a00aca80 0x7fe0a00b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.710+0000 7fe0a56dd700 1 -- 172.21.15.163:0/288714588 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe09c009ae0 con 0x7fe0a00c81f0 2023-10-09T06:36:24.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.710+0000 7fe0a56dd700 1 --2- 172.21.15.163:0/288714588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0a00c81f0 0x7fe0a00af100 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe09800c4f0 tx=0x7fe098007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:24.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.710+0000 7fe094ff9700 1 -- 172.21.15.163:0/288714588 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe098011590 con 0x7fe0a00c81f0 2023-10-09T06:36:24.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.710+0000 7fe094ff9700 1 -- 172.21.15.163:0/288714588 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe098011710 con 0x7fe0a00c81f0 2023-10-09T06:36:24.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.710+0000 7fe0a7140700 1 -- 172.21.15.163:0/288714588 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a0152c00 con 0x7fe0a00c81f0 2023-10-09T06:36:24.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.710+0000 7fe094ff9700 1 -- 172.21.15.163:0/288714588 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe098011ad0 con 0x7fe0a00c81f0 2023-10-09T06:36:24.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.710+0000 7fe0a7140700 1 -- 172.21.15.163:0/288714588 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a00b3010 con 0x7fe0a00c81f0 2023-10-09T06:36:24.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.711+0000 7fe0a7140700 1 -- 172.21.15.163:0/288714588 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0a004c3c0 con 0x7fe0a00c81f0 2023-10-09T06:36:24.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.711+0000 7fe094ff9700 1 -- 172.21.15.163:0/288714588 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fe09801b3b0 con 0x7fe0a00c81f0 2023-10-09T06:36:24.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.712+0000 7fe094ff9700 1 --2- 172.21.15.163:0/288714588 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe08c067200 0x7fe08c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:24.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.712+0000 7fe094ff9700 1 -- 172.21.15.163:0/288714588 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe098083d90 con 0x7fe0a00c81f0 2023-10-09T06:36:24.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.712+0000 7fe0a4edc700 1 --2- 172.21.15.163:0/288714588 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe08c067200 0x7fe08c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:24.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.712+0000 7fe0a4edc700 1 --2- 172.21.15.163:0/288714588 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe08c067200 0x7fe08c0696b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe09c009880 tx=0x7fe09c018000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:24.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.716+0000 7fe094ff9700 1 -- 172.21.15.163:0/288714588 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe098087030 con 0x7fe0a00c81f0 2023-10-09T06:36:24.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.945+0000 7fe0a7140700 1 -- 172.21.15.163:0/288714588 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe0a00ad810 con 0x7fe0a00c81f0 2023-10-09T06:36:24.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.946+0000 7fe094ff9700 1 -- 172.21.15.163:0/288714588 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v57) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe098017020 con 0x7fe0a00c81f0 2023-10-09T06:36:24.948 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-09T06:36:24.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.948+0000 7fe0a7140700 1 -- 172.21.15.163:0/288714588 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe08c067200 msgr2=0x7fe08c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:24.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.948+0000 7fe0a7140700 1 --2- 172.21.15.163:0/288714588 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe08c067200 0x7fe08c0696b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe09c009880 tx=0x7fe09c018000 comp rx=0 tx=0).stop 2023-10-09T06:36:24.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.948+0000 7fe0a7140700 1 -- 172.21.15.163:0/288714588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0a00c81f0 msgr2=0x7fe0a00af100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:24.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.948+0000 7fe0a7140700 1 --2- 172.21.15.163:0/288714588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0a00c81f0 0x7fe0a00af100 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe09800c4f0 tx=0x7fe098007f50 comp rx=0 tx=0).stop 2023-10-09T06:36:24.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.948+0000 7fe0a7140700 1 -- 172.21.15.163:0/288714588 shutdown_connections 2023-10-09T06:36:24.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.948+0000 7fe0a7140700 1 --2- 172.21.15.163:0/288714588 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe08c067200 0x7fe08c0696b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.948+0000 7fe0a7140700 1 --2- 172.21.15.163:0/288714588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0a00c81f0 0x7fe0a00af100 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.948+0000 7fe0a7140700 1 --2- 172.21.15.163:0/288714588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0a0069d60 0x7fe0a0152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.948+0000 7fe0a7140700 1 --2- 172.21.15.163:0/288714588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0a00aca80 0x7fe0a00b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:24.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.948+0000 7fe0a7140700 1 -- 172.21.15.163:0/288714588 >> 172.21.15.163:0/288714588 conn(0x7fe0a00cd670 msgr2=0x7fe0a00ab7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:24.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.948+0000 7fe0a7140700 1 -- 172.21.15.163:0/288714588 shutdown_connections 2023-10-09T06:36:24.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:24.948+0000 7fe0a7140700 1 -- 172.21.15.163:0/288714588 wait complete. 2023-10-09T06:36:24.961 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-09T06:36:24.961 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-09T06:36:24.961 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi163.front.sepia.ceph.com 2023-10-09T06:36:24.962 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-09T06:36:24.962 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-09T06:36:24.962 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-09T06:36:24.962 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-09T06:36:24.962 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-09T06:36:24.962 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:36:24.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:24.962 DEBUG:teuthology.orchestra.run.smithi163:> ip addr 2023-10-09T06:36:24.983 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:36:24.983 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:36:24.984 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:36:24.984 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:24.984 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2023-10-09T06:36:24.984 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:24.985 INFO:teuthology.orchestra.run.smithi163.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:36:24.985 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:06 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:36:24.985 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f0 2023-10-09T06:36:24.985 INFO:teuthology.orchestra.run.smithi163.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:36:24.985 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:07 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:36:24.986 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f1 2023-10-09T06:36:24.986 INFO:teuthology.orchestra.run.smithi163.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:36:24.986 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5a brd ff:ff:ff:ff:ff:ff 2023-10-09T06:36:24.986 INFO:teuthology.orchestra.run.smithi163.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-09T06:36:24.986 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5b brd ff:ff:ff:ff:ff:ff 2023-10-09T06:36:24.986 INFO:teuthology.orchestra.run.smithi163.stdout: inet 172.21.15.163/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-09T06:36:24.986 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft 42138sec preferred_lft 42138sec 2023-10-09T06:36:24.986 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::ec4:7aff:fe88:7d5b/64 scope link 2023-10-09T06:36:24.986 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:24.987 INFO:teuthology.orchestra.run.smithi163.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-09T06:36:24.987 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 2e:01:84:b9:f4:8e brd ff:ff:ff:ff:ff:ff 2023-10-09T06:36:24.987 INFO:teuthology.orchestra.run.smithi163.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-09T06:36:24.987 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:24.987 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::d03c:93ff:fe9e:c8cb/64 scope link 2023-10-09T06:36:24.987 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:24.987 INFO:teuthology.orchestra.run.smithi163.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:36:24.987 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 2e:01:84:b9:f4:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-09T06:36:24.987 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::2c01:84ff:feb9:f48e/64 scope link 2023-10-09T06:36:24.987 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:24.988 INFO:teuthology.orchestra.run.smithi163.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:36:24.988 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether c6:88:3e:d7:c1:56 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-09T06:36:24.988 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::c488:3eff:fed7:c156/64 scope link 2023-10-09T06:36:24.988 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:24.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:24.989 DEBUG:teuthology.orchestra.run.smithi163:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-09T06:36:25.050 INFO:teuthology.orchestra.run.smithi163.stdout:1 2023-10-09T06:36:25.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:25.052 DEBUG:teuthology.orchestra.run.smithi163:> route 2023-10-09T06:36:25.108 INFO:teuthology.orchestra.run.smithi163.stdout:Kernel IP routing table 2023-10-09T06:36:25.109 INFO:teuthology.orchestra.run.smithi163.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-09T06:36:25.109 INFO:teuthology.orchestra.run.smithi163.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-09T06:36:25.109 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-09T06:36:25.109 INFO:teuthology.orchestra.run.smithi163.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-09T06:36:25.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:25.109 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-09T06:36:25.110 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-09T06:36:25.110 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-09T06:36:25.110 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-09T06:36:25.110 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-09T06:36:25.110 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-09T06:36:25.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:25.213 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2023-10-09T06:36:25.229 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-09T06:36:25.229 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-09T06:36:25.229 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:36:25.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:25.230 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set brx.0 up 2023-10-09T06:36:25.294 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:36:25.294 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-09T06:36:25.348 INFO:teuthology.orchestra.run.smithi163.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:36:25.348 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:36:25.349 DEBUG:teuthology.orchestra.run.smithi163:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-09T06:36:25.403 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-09T06:36:25.403 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-09T06:36:25.403 DEBUG:teuthology.orchestra.run.smithi163:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:36:25.483 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:36:25.628 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:36:25.629 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:36:25.629 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:36:25.629 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:36:25.629 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with new device syntax: 0@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:36:25.629 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:36:25.630 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:36:25.630 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:36:25.630 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:36:25.630 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:36:25.630 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:36:25.630 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:36:25.630 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with new device syntax: 0@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:36:25.630 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:36:25.630 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:36:25.631 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:36:25.631 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:36:25.631 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790" 2023-10-09T06:36:25.631 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790:/ 2023-10-09T06:36:25.631 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=2520b8f7-539f-4bae-9f17-1fce66f66ddd" will pass to kernel 2023-10-09T06:36:25.632 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-09T06:36:25.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:25.632 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-09T06:36:25.695 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-09T06:36:25.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.950+0000 7fb20759e700 1 Processor -- start 2023-10-09T06:36:25.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.950+0000 7fb20759e700 1 -- start start 2023-10-09T06:36:25.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.951+0000 7fb20759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2081597f0 0x7fb2080b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:25.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.951+0000 7fb20759e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2080b11d0 con 0x7fb20815d230 2023-10-09T06:36:25.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.951+0000 7fb20759e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2080b14b0 con 0x7fb208159420 2023-10-09T06:36:25.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.951+0000 7fb20759e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2080b1790 con 0x7fb2081597f0 2023-10-09T06:36:25.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.951+0000 7fb206d9d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb20815d230 0x7fb2080b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33684/0 (socket says 172.21.15.163:33684) 2023-10-09T06:36:25.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.951+0000 7fb206d9d700 1 -- 172.21.15.163:0/3898422126 learned_addr learned my addr 172.21.15.163:0/3898422126 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:25.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.951+0000 7fb1fdd9b700 1 --2- 172.21.15.163:0/3898422126 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2081597f0 0x7fb2080b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:25.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.951+0000 7fb205d9b700 1 -- 172.21.15.163:0/3898422126 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4008549776 0 0) 0x7fb2080b11d0 con 0x7fb20815d230 2023-10-09T06:36:25.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.951+0000 7fb205d9b700 1 -- 172.21.15.163:0/3898422126 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1f4003570 con 0x7fb20815d230 2023-10-09T06:36:25.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.951+0000 7fb205d9b700 1 -- 172.21.15.163:0/3898422126 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 862610350 0 0) 0x7fb1f4003570 con 0x7fb20815d230 2023-10-09T06:36:25.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.951+0000 7fb205d9b700 1 -- 172.21.15.163:0/3898422126 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2081597f0 msgr2=0x7fb2080b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:25.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.951+0000 7fb205d9b700 1 --2- 172.21.15.163:0/3898422126 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2081597f0 0x7fb2080b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:25.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.951+0000 7fb205d9b700 1 -- 172.21.15.163:0/3898422126 >> v1:172.21.15.181:6789/0 conn(0x7fb208159420 legacy=0x7fb2080b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:25.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.951+0000 7fb205d9b700 1 -- 172.21.15.163:0/3898422126 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2080b1a70 con 0x7fb20815d230 2023-10-09T06:36:25.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.952+0000 7fb205d9b700 1 -- 172.21.15.163:0/3898422126 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fb2000017f0 con 0x7fb20815d230 2023-10-09T06:36:25.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.952+0000 7fb205d9b700 1 -- 172.21.15.163:0/3898422126 >> v1:172.21.15.163:6789/0 conn(0x7fb20815d230 legacy=0x7fb2080b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:25.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.952+0000 7fb205d9b700 1 --2- 172.21.15.163:0/3898422126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1f4003a90 0x7fb1f4003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:25.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.952+0000 7fb205d9b700 1 --2- 172.21.15.163:0/3898422126 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2081597f0 0x7fb1f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:25.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.952+0000 7fb205d9b700 1 --2- 172.21.15.163:0/3898422126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1f4004d50 0x7fb1f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:25.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.952+0000 7fb205d9b700 1 -- 172.21.15.163:0/3898422126 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2080b1790 con 0x7fb1f4003a90 2023-10-09T06:36:25.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.952+0000 7fb205d9b700 1 -- 172.21.15.163:0/3898422126 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f400bb10 con 0x7fb1f4004d50 2023-10-09T06:36:25.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.952+0000 7fb205d9b700 1 -- 172.21.15.163:0/3898422126 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f400bcd0 con 0x7fb2081597f0 2023-10-09T06:36:25.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.952+0000 7fb20659c700 1 --2- 172.21.15.163:0/3898422126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1f4003a90 0x7fb1f4003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:25.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.952+0000 7fb1fdd9b700 1 --2- 172.21.15.163:0/3898422126 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2081597f0 0x7fb1f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:25.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.952+0000 7fb206d9d700 1 --2- 172.21.15.163:0/3898422126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1f4004d50 0x7fb1f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:25.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.952+0000 7fb20659c700 1 -- 172.21.15.163:0/3898422126 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2081597f0 msgr2=0x7fb1f4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:25.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.952+0000 7fb20659c700 1 --2- 172.21.15.163:0/3898422126 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2081597f0 0x7fb1f4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:25.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.952+0000 7fb20659c700 1 -- 172.21.15.163:0/3898422126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1f4004d50 msgr2=0x7fb1f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:25.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.952+0000 7fb20659c700 1 --2- 172.21.15.163:0/3898422126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1f4004d50 0x7fb1f40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:25.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.952+0000 7fb20659c700 1 -- 172.21.15.163:0/3898422126 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f4003570 con 0x7fb1f4003a90 2023-10-09T06:36:25.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.952+0000 7fb20659c700 1 --2- 172.21.15.163:0/3898422126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1f4003a90 0x7fb1f4003e60 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb1f80019a0 tx=0x7fb1f8007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3861d85660e264 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:25.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.952+0000 7fb205d9b700 1 -- 172.21.15.163:0/3898422126 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f800c5d0 con 0x7fb1f4003a90 2023-10-09T06:36:25.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.952+0000 7fb205d9b700 1 -- 172.21.15.163:0/3898422126 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f800c730 con 0x7fb1f4003a90 2023-10-09T06:36:25.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.952+0000 7fb205d9b700 1 -- 172.21.15.163:0/3898422126 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f800cad0 con 0x7fb1f4003a90 2023-10-09T06:36:25.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.953+0000 7fb20759e700 1 -- 172.21.15.163:0/3898422126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1f4003a90 msgr2=0x7fb1f4003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:25.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.953+0000 7fb20759e700 1 --2- 172.21.15.163:0/3898422126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1f4003a90 0x7fb1f4003e60 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb1f80019a0 tx=0x7fb1f8007d10 comp rx=0 tx=0).stop 2023-10-09T06:36:25.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.953+0000 7fb206d9d700 1 -- 172.21.15.163:0/3898422126 reap_dead start 2023-10-09T06:36:25.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.953+0000 7fb20759e700 1 -- 172.21.15.163:0/3898422126 shutdown_connections 2023-10-09T06:36:25.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.953+0000 7fb20759e700 1 -- 172.21.15.163:0/3898422126 >> 172.21.15.163:0/3898422126 conn(0x7fb2080d0660 msgr2=0x7fb2080d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:25.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.953+0000 7fb20759e700 1 -- 172.21.15.163:0/3898422126 shutdown_connections 2023-10-09T06:36:25.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.953+0000 7fb20759e700 1 -- 172.21.15.163:0/3898422126 wait complete. 2023-10-09T06:36:25.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.954+0000 7fb20759e700 1 Processor -- start 2023-10-09T06:36:25.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.954+0000 7fb20759e700 1 -- start start 2023-10-09T06:36:25.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.955+0000 7fb20759e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1f4003a90 0x7fb208150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:25.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.955+0000 7fb20759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb20813ac70 0x7fb208151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:25.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.955+0000 7fb20759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb20814d3b0 0x7fb2080b4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:25.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.955+0000 7fb20759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20814fef0 con 0x7fb20813ac70 2023-10-09T06:36:25.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.955+0000 7fb20759e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb208152070 con 0x7fb1f4003a90 2023-10-09T06:36:25.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.955+0000 7fb20759e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb208150470 con 0x7fb20814d3b0 2023-10-09T06:36:25.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.955+0000 7fb1fdd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb20813ac70 0x7fb208151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:25.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.955+0000 7fb1fdd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb20813ac70 0x7fb208151140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55046/0 (socket says 172.21.15.163:55046) 2023-10-09T06:36:25.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.955+0000 7fb1fdd9b700 1 -- 172.21.15.163:0/1172644475 learned_addr learned my addr 172.21.15.163:0/1172644475 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:25.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.955+0000 7fb206d9d700 1 --2- 172.21.15.163:0/1172644475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb20814d3b0 0x7fb2080b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:25.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.955+0000 7fb20659c700 1 --2- 172.21.15.163:0/1172644475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1f4003a90 0x7fb208150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:25.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.955+0000 7fb1fdd9b700 1 -- 172.21.15.163:0/1172644475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb20814d3b0 msgr2=0x7fb2080b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:25.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.955+0000 7fb1fdd9b700 1 --2- 172.21.15.163:0/1172644475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb20814d3b0 0x7fb2080b4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:25.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.955+0000 7fb1fdd9b700 1 -- 172.21.15.163:0/1172644475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1f4003a90 msgr2=0x7fb208150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:25.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.955+0000 7fb1fdd9b700 1 --2- 172.21.15.163:0/1172644475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1f4003a90 0x7fb208150900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:25.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.955+0000 7fb1fdd9b700 1 -- 172.21.15.163:0/1172644475 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb208151ab0 con 0x7fb20813ac70 2023-10-09T06:36:25.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.955+0000 7fb1fdd9b700 1 --2- 172.21.15.163:0/1172644475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb20813ac70 0x7fb208151140 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0011d40 tx=0x7fb1f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:25.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.956+0000 7fb1feffd700 1 -- 172.21.15.163:0/1172644475 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f000fa60 con 0x7fb20813ac70 2023-10-09T06:36:25.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.956+0000 7fb1feffd700 1 -- 172.21.15.163:0/1172644475 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f000fbc0 con 0x7fb20813ac70 2023-10-09T06:36:25.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.956+0000 7fb1feffd700 1 -- 172.21.15.163:0/1172644475 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0014440 con 0x7fb20813ac70 2023-10-09T06:36:25.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.956+0000 7fb20759e700 1 -- 172.21.15.163:0/1172644475 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb208151d90 con 0x7fb20813ac70 2023-10-09T06:36:25.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.956+0000 7fb20759e700 1 -- 172.21.15.163:0/1172644475 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb208150e40 con 0x7fb20813ac70 2023-10-09T06:36:25.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.956+0000 7fb20759e700 1 -- 172.21.15.163:0/1172644475 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb20804c3c0 con 0x7fb20813ac70 2023-10-09T06:36:25.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.957+0000 7fb1feffd700 1 -- 172.21.15.163:0/1172644475 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fb1f0014a80 con 0x7fb20813ac70 2023-10-09T06:36:25.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.957+0000 7fb1feffd700 1 --2- 172.21.15.163:0/1172644475 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb1e8067200 0x7fb1e80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:25.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.957+0000 7fb1feffd700 1 -- 172.21.15.163:0/1172644475 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb1f0018030 con 0x7fb20813ac70 2023-10-09T06:36:25.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.958+0000 7fb20659c700 1 --2- 172.21.15.163:0/1172644475 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb1e8067200 0x7fb1e80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:25.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.958+0000 7fb20659c700 1 --2- 172.21.15.163:0/1172644475 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb1e8067200 0x7fb1e80696b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb2080b2bf0 tx=0x7fb1f8015000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:25.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:25.961+0000 7fb1feffd700 1 -- 172.21.15.163:0/1172644475 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb1f0052550 con 0x7fb20813ac70 2023-10-09T06:36:26.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.186+0000 7fb20759e700 1 -- 172.21.15.163:0/1172644475 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb2081505b0 con 0x7fb20813ac70 2023-10-09T06:36:26.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.187+0000 7fb1feffd700 1 -- 172.21.15.163:0/1172644475 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v57) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb1f0055b70 con 0x7fb20813ac70 2023-10-09T06:36:26.188 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-09T06:36:26.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.189+0000 7fb20759e700 1 -- 172.21.15.163:0/1172644475 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb1e8067200 msgr2=0x7fb1e80696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:26.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.189+0000 7fb20759e700 1 --2- 172.21.15.163:0/1172644475 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb1e8067200 0x7fb1e80696b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb2080b2bf0 tx=0x7fb1f8015000 comp rx=0 tx=0).stop 2023-10-09T06:36:26.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.189+0000 7fb20759e700 1 -- 172.21.15.163:0/1172644475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb20813ac70 msgr2=0x7fb208151140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:26.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.189+0000 7fb20759e700 1 --2- 172.21.15.163:0/1172644475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb20813ac70 0x7fb208151140 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0011d40 tx=0x7fb1f00091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:26.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.189+0000 7fb20759e700 1 -- 172.21.15.163:0/1172644475 shutdown_connections 2023-10-09T06:36:26.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.189+0000 7fb20759e700 1 --2- 172.21.15.163:0/1172644475 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb1e8067200 0x7fb1e80696b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:26.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.189+0000 7fb20759e700 1 --2- 172.21.15.163:0/1172644475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb20814d3b0 0x7fb2080b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:26.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.189+0000 7fb20759e700 1 --2- 172.21.15.163:0/1172644475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1f4003a90 0x7fb208150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:26.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.189+0000 7fb20759e700 1 --2- 172.21.15.163:0/1172644475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb20813ac70 0x7fb208151140 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:26.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.189+0000 7fb20759e700 1 -- 172.21.15.163:0/1172644475 >> 172.21.15.163:0/1172644475 conn(0x7fb2080d0660 msgr2=0x7fb2080cf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:26.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.189+0000 7fb20759e700 1 -- 172.21.15.163:0/1172644475 shutdown_connections 2023-10-09T06:36:26.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.189+0000 7fb20759e700 1 -- 172.21.15.163:0/1172644475 wait complete. 2023-10-09T06:36:26.202 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-09T06:36:26.203 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-09T06:36:26.203 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-09T06:36:26.203 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-09T06:36:26.203 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-09T06:36:26.203 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-09T06:36:26.203 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-09T06:36:26.203 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-09T06:36:26.203 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:36:26.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:26.204 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-09T06:36:26.222 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:36:26.222 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:36:26.222 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:36:26.222 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:26.223 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-09T06:36:26.223 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:26.223 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:36:26.223 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:36:26.223 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2023-10-09T06:36:26.223 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:36:26.223 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:36:26.224 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2023-10-09T06:36:26.224 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:36:26.224 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-09T06:36:26.224 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-09T06:36:26.224 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-09T06:36:26.224 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-09T06:36:26.224 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42114sec preferred_lft 42114sec 2023-10-09T06:36:26.224 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2023-10-09T06:36:26.224 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:26.225 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-09T06:36:26.225 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 72:62:e6:72:7c:2e brd ff:ff:ff:ff:ff:ff 2023-10-09T06:36:26.225 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-09T06:36:26.225 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:26.225 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::e89a:4cff:fefb:7198/64 scope link 2023-10-09T06:36:26.225 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:26.225 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:36:26.225 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 72:62:e6:72:7c:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-09T06:36:26.225 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::7062:e6ff:fe72:7c2e/64 scope link 2023-10-09T06:36:26.225 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:26.225 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:36:26.225 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether fa:4e:a3:9d:5a:5f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-09T06:36:26.226 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::f84e:a3ff:fe9d:5a5f/64 scope link 2023-10-09T06:36:26.226 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:26.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:26.226 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-09T06:36:26.290 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-09T06:36:26.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:26.291 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-09T06:36:26.350 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-09T06:36:26.351 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-09T06:36:26.351 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-09T06:36:26.351 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-09T06:36:26.351 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-09T06:36:26.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:26.352 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-09T06:36:26.352 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-09T06:36:26.352 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-09T06:36:26.352 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-09T06:36:26.352 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-09T06:36:26.352 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-09T06:36:26.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:26.458 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-09T06:36:26.473 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-09T06:36:26.473 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-09T06:36:26.474 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:36:26.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:26.474 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-09T06:36:26.540 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:36:26.540 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-09T06:36:26.596 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:36:26.596 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:36:26.596 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-09T06:36:26.651 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-09T06:36:26.652 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-09T06:36:26.652 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:36:26.732 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:36:26.732 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:36:26.750 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:36:26.751 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:36:26.751 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:36:26.751 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 1@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:36:26.751 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:36:26.751 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:36:26.751 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:36:26.752 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:36:26.773 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:36:26.773 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:36:26.773 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:36:26.773 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 1@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:36:26.773 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:36:26.774 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:36:26.774 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:36:26.774 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:36:26.774 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790" 2023-10-09T06:36:26.775 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790:/ 2023-10-09T06:36:26.775 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=2520b8f7-539f-4bae-9f17-1fce66f66ddd" will pass to kernel 2023-10-09T06:36:26.776 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-09T06:36:26.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:26.776 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-09T06:36:26.840 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-09T06:36:26.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.959+0000 7fa08201c700 1 Processor -- start 2023-10-09T06:36:26.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.960+0000 7fa08201c700 1 -- start start 2023-10-09T06:36:26.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.960+0000 7fa08201c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa07c0cb560 0x7fa07c0cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:26.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.960+0000 7fa08201c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa07c069ec0 0x7fa07c15ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:26.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.960+0000 7fa08201c700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa07c0ccde0 con 0x7fa07c0cc970 2023-10-09T06:36:26.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.960+0000 7fa08201c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa07c06aa30 con 0x7fa07c069ec0 2023-10-09T06:36:26.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.960+0000 7fa08201c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c159db0 con 0x7fa07c0cb560 2023-10-09T06:36:26.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.960+0000 7fa080819700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fa07c0cc970 0x7fa07c0697b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33700/0 (socket says 172.21.15.163:33700) 2023-10-09T06:36:26.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.961+0000 7fa080819700 1 -- 172.21.15.163:0/1598453206 learned_addr learned my addr 172.21.15.163:0/1598453206 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:26.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.961+0000 7fa08101a700 1 --2- 172.21.15.163:0/1598453206 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa07c0cb560 0x7fa07c0cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:26.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.961+0000 7fa08181b700 1 --2- 172.21.15.163:0/1598453206 >> v2:172.21.15.181:3300/0 conn(0x7fa07c069ec0 0x7fa07c15ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:26.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.961+0000 7fa073fff700 1 -- 172.21.15.163:0/1598453206 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3719397 0 0) 0x7fa07c0ccde0 con 0x7fa07c0cc970 2023-10-09T06:36:26.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.961+0000 7fa073fff700 1 -- 172.21.15.163:0/1598453206 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa068003570 con 0x7fa07c0cc970 2023-10-09T06:36:26.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.961+0000 7fa073fff700 1 -- 172.21.15.163:0/1598453206 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2435518555 0 0) 0x7fa068003570 con 0x7fa07c0cc970 2023-10-09T06:36:26.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.961+0000 7fa073fff700 1 -- 172.21.15.163:0/1598453206 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa07c0cb560 msgr2=0x7fa07c0cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:26.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.961+0000 7fa073fff700 1 --2- 172.21.15.163:0/1598453206 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa07c0cb560 0x7fa07c0cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:26.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.961+0000 7fa073fff700 1 -- 172.21.15.163:0/1598453206 >> v2:172.21.15.181:3300/0 conn(0x7fa07c069ec0 msgr2=0x7fa07c15ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:26.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.961+0000 7fa073fff700 1 --2- 172.21.15.163:0/1598453206 >> v2:172.21.15.181:3300/0 conn(0x7fa07c069ec0 0x7fa07c15ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:26.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.961+0000 7fa073fff700 1 -- 172.21.15.163:0/1598453206 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07c06a470 con 0x7fa07c0cc970 2023-10-09T06:36:26.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.961+0000 7fa073fff700 1 -- 172.21.15.163:0/1598453206 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fa06c001470 con 0x7fa07c0cc970 2023-10-09T06:36:26.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.961+0000 7fa073fff700 1 -- 172.21.15.163:0/1598453206 >> v1:172.21.15.163:6789/0 conn(0x7fa07c0cc970 legacy=0x7fa07c0697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:26.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.961+0000 7fa073fff700 1 --2- 172.21.15.163:0/1598453206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa068003a60 0x7fa068003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:26.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.961+0000 7fa073fff700 1 --2- 172.21.15.163:0/1598453206 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa07c0cb560 0x7fa068004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:26.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.961+0000 7fa073fff700 1 --2- 172.21.15.163:0/1598453206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa068004d50 0x7fa0680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:26.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.961+0000 7fa073fff700 1 -- 172.21.15.163:0/1598453206 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c06aa30 con 0x7fa068004d50 2023-10-09T06:36:26.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.961+0000 7fa073fff700 1 -- 172.21.15.163:0/1598453206 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c159db0 con 0x7fa068003a60 2023-10-09T06:36:26.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.961+0000 7fa073fff700 1 -- 172.21.15.163:0/1598453206 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa06800ba60 con 0x7fa07c0cb560 2023-10-09T06:36:26.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.961+0000 7fa08181b700 1 --2- 172.21.15.163:0/1598453206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa068004d50 0x7fa0680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:26.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.961+0000 7fa08101a700 1 --2- 172.21.15.163:0/1598453206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa068003a60 0x7fa068003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:26.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.961+0000 7fa080819700 1 --2- 172.21.15.163:0/1598453206 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa07c0cb560 0x7fa068004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:26.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.961+0000 7fa08181b700 1 -- 172.21.15.163:0/1598453206 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa07c0cb560 msgr2=0x7fa068004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:26.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.961+0000 7fa08181b700 1 --2- 172.21.15.163:0/1598453206 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa07c0cb560 0x7fa068004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:26.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.961+0000 7fa08181b700 1 -- 172.21.15.163:0/1598453206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa068003a60 msgr2=0x7fa068003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:26.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.961+0000 7fa08181b700 1 --2- 172.21.15.163:0/1598453206 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa068003a60 0x7fa068003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:26.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.961+0000 7fa08181b700 1 -- 172.21.15.163:0/1598453206 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa068003570 con 0x7fa068004d50 2023-10-09T06:36:26.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.962+0000 7fa08181b700 1 --2- 172.21.15.163:0/1598453206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa068004d50 0x7fa0680051c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fa074009c00 tx=0x7fa0740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208b9d6dd9eb2ed5 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:26.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.962+0000 7fa073fff700 1 -- 172.21.15.163:0/1598453206 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07400e6b0 con 0x7fa068004d50 2023-10-09T06:36:26.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.962+0000 7fa073fff700 1 -- 172.21.15.163:0/1598453206 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa07400e810 con 0x7fa068004d50 2023-10-09T06:36:26.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.962+0000 7fa073fff700 1 -- 172.21.15.163:0/1598453206 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07400ebd0 con 0x7fa068004d50 2023-10-09T06:36:26.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.962+0000 7fa08201c700 1 -- 172.21.15.163:0/1598453206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa068004d50 msgr2=0x7fa0680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:26.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.962+0000 7fa08201c700 1 --2- 172.21.15.163:0/1598453206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa068004d50 0x7fa0680051c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fa074009c00 tx=0x7fa0740091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:26.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.962+0000 7fa08181b700 1 -- 172.21.15.163:0/1598453206 reap_dead start 2023-10-09T06:36:26.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.962+0000 7fa08201c700 1 -- 172.21.15.163:0/1598453206 shutdown_connections 2023-10-09T06:36:26.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.962+0000 7fa08201c700 1 -- 172.21.15.163:0/1598453206 >> 172.21.15.163:0/1598453206 conn(0x7fa07c06baf0 msgr2=0x7fa07c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:26.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.962+0000 7fa08201c700 1 -- 172.21.15.163:0/1598453206 shutdown_connections 2023-10-09T06:36:26.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.962+0000 7fa08201c700 1 -- 172.21.15.163:0/1598453206 wait complete. 2023-10-09T06:36:26.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.963+0000 7fa08201c700 1 Processor -- start 2023-10-09T06:36:26.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.963+0000 7fa08201c700 1 -- start start 2023-10-09T06:36:26.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.964+0000 7fa08201c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa07c169bc0 0x7fa07c169f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:26.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.964+0000 7fa08201c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa07c0c9ad0 0x7fa07c15a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:26.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.964+0000 7fa08201c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa07c0c9f40 0x7fa07c15a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:26.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.964+0000 7fa08201c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c148ca0 con 0x7fa07c0c9ad0 2023-10-09T06:36:26.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.964+0000 7fa08201c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c0a98d0 con 0x7fa07c169bc0 2023-10-09T06:36:26.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.964+0000 7fa08201c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c0aa170 con 0x7fa07c0c9f40 2023-10-09T06:36:26.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.964+0000 7fa080819700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa07c0c9ad0 0x7fa07c15a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:26.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.964+0000 7fa080819700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa07c0c9ad0 0x7fa07c15a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55062/0 (socket says 172.21.15.163:55062) 2023-10-09T06:36:26.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.964+0000 7fa080819700 1 -- 172.21.15.163:0/3004832112 learned_addr learned my addr 172.21.15.163:0/3004832112 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:26.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.964+0000 7fa08181b700 1 --2- 172.21.15.163:0/3004832112 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa07c0c9f40 0x7fa07c15a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:26.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.965+0000 7fa08101a700 1 --2- 172.21.15.163:0/3004832112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa07c169bc0 0x7fa07c169f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:26.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.965+0000 7fa080819700 1 -- 172.21.15.163:0/3004832112 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa07c0c9f40 msgr2=0x7fa07c15a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:26.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.965+0000 7fa080819700 1 --2- 172.21.15.163:0/3004832112 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa07c0c9f40 0x7fa07c15a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:26.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.965+0000 7fa080819700 1 -- 172.21.15.163:0/3004832112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa07c169bc0 msgr2=0x7fa07c169f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:26.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.965+0000 7fa080819700 1 --2- 172.21.15.163:0/3004832112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa07c169bc0 0x7fa07c169f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:26.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.965+0000 7fa080819700 1 -- 172.21.15.163:0/3004832112 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07c16a4d0 con 0x7fa07c0c9ad0 2023-10-09T06:36:26.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.965+0000 7fa080819700 1 --2- 172.21.15.163:0/3004832112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa07c0c9ad0 0x7fa07c15a490 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0073f0 tx=0x7fa06c001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:26.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.965+0000 7fa0717fa700 1 -- 172.21.15.163:0/3004832112 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c018030 con 0x7fa07c0c9ad0 2023-10-09T06:36:26.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.965+0000 7fa0717fa700 1 -- 172.21.15.163:0/3004832112 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06c00f8d0 con 0x7fa07c0c9ad0 2023-10-09T06:36:26.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.965+0000 7fa08201c700 1 -- 172.21.15.163:0/3004832112 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa07c16a700 con 0x7fa07c0c9ad0 2023-10-09T06:36:26.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.965+0000 7fa0717fa700 1 -- 172.21.15.163:0/3004832112 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c00fc70 con 0x7fa07c0c9ad0 2023-10-09T06:36:26.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.965+0000 7fa08201c700 1 -- 172.21.15.163:0/3004832112 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa07c0a9b20 con 0x7fa07c0c9ad0 2023-10-09T06:36:26.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.966+0000 7fa08201c700 1 -- 172.21.15.163:0/3004832112 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa07c04c3c0 con 0x7fa07c0c9ad0 2023-10-09T06:36:26.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.966+0000 7fa0717fa700 1 -- 172.21.15.163:0/3004832112 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fa06c01b070 con 0x7fa07c0c9ad0 2023-10-09T06:36:26.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.967+0000 7fa0717fa700 1 --2- 172.21.15.163:0/3004832112 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa060067200 0x7fa0600696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:26.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.967+0000 7fa0717fa700 1 -- 172.21.15.163:0/3004832112 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa06c019070 con 0x7fa07c0c9ad0 2023-10-09T06:36:26.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.967+0000 7fa08101a700 1 --2- 172.21.15.163:0/3004832112 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa060067200 0x7fa0600696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:26.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.967+0000 7fa08101a700 1 --2- 172.21.15.163:0/3004832112 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa060067200 0x7fa0600696b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa07c1583e0 tx=0x7fa07800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:26.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:26.971+0000 7fa0717fa700 1 -- 172.21.15.163:0/3004832112 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa06c0518f0 con 0x7fa07c0c9ad0 2023-10-09T06:36:27.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.194+0000 7fa08201c700 1 -- 172.21.15.163:0/3004832112 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa07c065910 con 0x7fa07c0c9ad0 2023-10-09T06:36:27.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.826+0000 7fa0717fa700 1 -- 172.21.15.163:0/3004832112 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v58) v1 ==== 136+0+0 (secure 0 0 0) 0x7fa06c054f10 con 0x7fa07c0c9ad0 2023-10-09T06:36:27.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.829+0000 7fa08201c700 1 -- 172.21.15.163:0/3004832112 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa060067200 msgr2=0x7fa0600696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:27.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.829+0000 7fa08201c700 1 --2- 172.21.15.163:0/3004832112 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa060067200 0x7fa0600696b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa07c1583e0 tx=0x7fa07800b040 comp rx=0 tx=0).stop 2023-10-09T06:36:27.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.829+0000 7fa08201c700 1 -- 172.21.15.163:0/3004832112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa07c0c9ad0 msgr2=0x7fa07c15a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:27.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.829+0000 7fa08201c700 1 --2- 172.21.15.163:0/3004832112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa07c0c9ad0 0x7fa07c15a490 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0073f0 tx=0x7fa06c001e90 comp rx=0 tx=0).stop 2023-10-09T06:36:27.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.829+0000 7fa08201c700 1 -- 172.21.15.163:0/3004832112 shutdown_connections 2023-10-09T06:36:27.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.829+0000 7fa08201c700 1 --2- 172.21.15.163:0/3004832112 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa060067200 0x7fa0600696b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:27.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.829+0000 7fa08201c700 1 --2- 172.21.15.163:0/3004832112 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa07c0c9f40 0x7fa07c15a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:27.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.829+0000 7fa08201c700 1 --2- 172.21.15.163:0/3004832112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa07c169bc0 0x7fa07c169f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:27.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.829+0000 7fa08201c700 1 --2- 172.21.15.163:0/3004832112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa07c0c9ad0 0x7fa07c15a490 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:27.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.829+0000 7fa08201c700 1 -- 172.21.15.163:0/3004832112 >> 172.21.15.163:0/3004832112 conn(0x7fa07c06baf0 msgr2=0x7fa07c0c4650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:27.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.829+0000 7fa08201c700 1 -- 172.21.15.163:0/3004832112 shutdown_connections 2023-10-09T06:36:27.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.829+0000 7fa08201c700 1 -- 172.21.15.163:0/3004832112 wait complete. 2023-10-09T06:36:27.843 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-09T06:36:27.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.960+0000 7ff4b4d96700 1 Processor -- start 2023-10-09T06:36:27.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.960+0000 7ff4b4d96700 1 -- start start 2023-10-09T06:36:27.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.961+0000 7ff4b4d96700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4b0159890 0x7ff4b015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:27.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.961+0000 7ff4b4d96700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff4b0159c60 0x7ff4b015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:27.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.961+0000 7ff4b4d96700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b015a160 con 0x7ff4b015db10 2023-10-09T06:36:27.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.961+0000 7ff4b4d96700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b00d13a0 con 0x7ff4b0159c60 2023-10-09T06:36:27.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.961+0000 7ff4b4d96700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b015e140 con 0x7ff4b0159890 2023-10-09T06:36:27.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.961+0000 7ff4ae59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4b0159890 0x7ff4b015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:27.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.961+0000 7ff4add9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff4b0159c60 0x7ff4b015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:27.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.961+0000 7ff4aed9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff4b015db10 0x7ff4b0069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:42002/0 (socket says 172.21.15.163:42002) 2023-10-09T06:36:27.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.961+0000 7ff4add9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff4b0159c60 0x7ff4b015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:34776/0 (socket says 172.21.15.163:34776) 2023-10-09T06:36:27.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.961+0000 7ff4aed9d700 1 -- 172.21.15.163:0/1388426932 learned_addr learned my addr 172.21.15.163:0/1388426932 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:27.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.961+0000 7ff4ae59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4b0159890 0x7ff4b015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:57614/0 (socket says 172.21.15.163:57614) 2023-10-09T06:36:27.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.961+0000 7ff4ae59c700 1 -- 172.21.15.163:0/1388426932 >> v2:172.21.15.181:3300/0 conn(0x7ff4b0159c60 msgr2=0x7ff4b015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:27.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.961+0000 7ff4ae59c700 1 --2- 172.21.15.163:0/1388426932 >> v2:172.21.15.181:3300/0 conn(0x7ff4b0159c60 0x7ff4b015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:27.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.961+0000 7ff4ae59c700 1 -- 172.21.15.163:0/1388426932 >> v1:172.21.15.181:6789/0 conn(0x7ff4b015db10 legacy=0x7ff4b0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:27.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.961+0000 7ff4ad59a700 1 -- 172.21.15.163:0/1388426932 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3614009052 0 0) 0x7ff4b015a160 con 0x7ff4b015db10 2023-10-09T06:36:27.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.961+0000 7ff4ae59c700 1 -- 172.21.15.163:0/1388426932 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b015a3d0 con 0x7ff4b0159890 2023-10-09T06:36:27.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.961+0000 7ff4add9b700 1 --2- 172.21.15.163:0/1388426932 >> v2:172.21.15.181:3300/0 conn(0x7ff4b0159c60 0x7ff4b015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:36:27.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.961+0000 7ff4ae59c700 1 --2- 172.21.15.163:0/1388426932 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4b0159890 0x7ff4b015abd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009e60 tx=0x7ff4a0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48aa9ad64e8d94fe server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:27.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.962+0000 7ff4ad59a700 1 -- 172.21.15.163:0/1388426932 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a000e070 con 0x7ff4b0159890 2023-10-09T06:36:27.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.962+0000 7ff4ad59a700 1 -- 172.21.15.163:0/1388426932 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a000bb60 con 0x7ff4b0159890 2023-10-09T06:36:27.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.962+0000 7ff4ad59a700 1 -- 172.21.15.163:0/1388426932 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0012430 con 0x7ff4b0159890 2023-10-09T06:36:27.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.962+0000 7ff4b4d96700 1 -- 172.21.15.163:0/1388426932 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4b0159890 msgr2=0x7ff4b015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:27.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.962+0000 7ff4b4d96700 1 --2- 172.21.15.163:0/1388426932 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4b0159890 0x7ff4b015abd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009e60 tx=0x7ff4a0009780 comp rx=0 tx=0).stop 2023-10-09T06:36:27.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.962+0000 7ff4b4d96700 1 -- 172.21.15.163:0/1388426932 shutdown_connections 2023-10-09T06:36:27.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.962+0000 7ff4b4d96700 1 --2- 172.21.15.163:0/1388426932 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4b0159890 0x7ff4b015abd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:27.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.962+0000 7ff4b4d96700 1 --2- 172.21.15.163:0/1388426932 >> v2:172.21.15.181:3300/0 conn(0x7ff4b0159c60 0x7ff4b015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:27.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.962+0000 7ff4b4d96700 1 -- 172.21.15.163:0/1388426932 >> 172.21.15.163:0/1388426932 conn(0x7ff4b0068e20 msgr2=0x7ff4b00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:27.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.962+0000 7ff4b4d96700 1 -- 172.21.15.163:0/1388426932 shutdown_connections 2023-10-09T06:36:27.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.962+0000 7ff4b4d96700 1 -- 172.21.15.163:0/1388426932 wait complete. 2023-10-09T06:36:27.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.963+0000 7ff4b4d96700 1 Processor -- start 2023-10-09T06:36:27.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.963+0000 7ff4b4d96700 1 -- start start 2023-10-09T06:36:27.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.964+0000 7ff4b4d96700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4b0159890 0x7ff4b0158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:27.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.964+0000 7ff4b4d96700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4b0159c60 0x7ff4b00c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:27.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.964+0000 7ff4b4d96700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4b015db10 0x7ff4b00ce760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:27.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.964+0000 7ff4b4d96700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b00c7de0 con 0x7ff4b015db10 2023-10-09T06:36:27.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.964+0000 7ff4b4d96700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b0158c10 con 0x7ff4b0159890 2023-10-09T06:36:27.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.964+0000 7ff4b4d96700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b016d490 con 0x7ff4b0159c60 2023-10-09T06:36:27.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.964+0000 7ff4aed9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4b015db10 0x7ff4b00ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:27.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.964+0000 7ff4aed9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4b015db10 0x7ff4b00ce760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55074/0 (socket says 172.21.15.163:55074) 2023-10-09T06:36:27.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.964+0000 7ff4aed9d700 1 -- 172.21.15.163:0/1641728831 learned_addr learned my addr 172.21.15.163:0/1641728831 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:27.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.964+0000 7ff4ae59c700 1 --2- 172.21.15.163:0/1641728831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4b0159890 0x7ff4b0158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:27.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.964+0000 7ff4add9b700 1 --2- 172.21.15.163:0/1641728831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4b0159c60 0x7ff4b00c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:27.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.964+0000 7ff4aed9d700 1 -- 172.21.15.163:0/1641728831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4b0159c60 msgr2=0x7ff4b00c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:27.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.965+0000 7ff4aed9d700 1 --2- 172.21.15.163:0/1641728831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4b0159c60 0x7ff4b00c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:27.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.965+0000 7ff4aed9d700 1 -- 172.21.15.163:0/1641728831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4b0159890 msgr2=0x7ff4b0158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:27.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.965+0000 7ff4aed9d700 1 --2- 172.21.15.163:0/1641728831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4b0159890 0x7ff4b0158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:27.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.965+0000 7ff4aed9d700 1 -- 172.21.15.163:0/1641728831 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a4009420 con 0x7ff4b015db10 2023-10-09T06:36:27.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.965+0000 7ff4aed9d700 1 --2- 172.21.15.163:0/1641728831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4b015db10 0x7ff4b00ce760 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7ff4a800c530 tx=0x7ff4a8007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:27.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.965+0000 7ff49e7fc700 1 -- 172.21.15.163:0/1641728831 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8005b30 con 0x7ff4b015db10 2023-10-09T06:36:27.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.965+0000 7ff49e7fc700 1 -- 172.21.15.163:0/1641728831 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a8005c90 con 0x7ff4b015db10 2023-10-09T06:36:27.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.965+0000 7ff4b4d96700 1 -- 172.21.15.163:0/1641728831 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a0009a20 con 0x7ff4b015db10 2023-10-09T06:36:27.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.965+0000 7ff49e7fc700 1 -- 172.21.15.163:0/1641728831 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8007320 con 0x7ff4b015db10 2023-10-09T06:36:27.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.965+0000 7ff4b4d96700 1 -- 172.21.15.163:0/1641728831 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b016da50 con 0x7ff4b015db10 2023-10-09T06:36:27.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.966+0000 7ff4b4d96700 1 -- 172.21.15.163:0/1641728831 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b004c3c0 con 0x7ff4b015db10 2023-10-09T06:36:27.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.966+0000 7ff49e7fc700 1 -- 172.21.15.163:0/1641728831 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7ff4a801b3b0 con 0x7ff4b015db10 2023-10-09T06:36:27.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.966+0000 7ff49e7fc700 1 --2- 172.21.15.163:0/1641728831 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff490067200 0x7ff4900696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:27.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.967+0000 7ff49e7fc700 1 -- 172.21.15.163:0/1641728831 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff4a8083d00 con 0x7ff4b015db10 2023-10-09T06:36:27.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.967+0000 7ff4ae59c700 1 --2- 172.21.15.163:0/1641728831 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff490067200 0x7ff4900696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:27.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.967+0000 7ff4ae59c700 1 --2- 172.21.15.163:0/1641728831 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff490067200 0x7ff4900696b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff4a00097c0 tx=0x7ff4a0019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:27.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:27.971+0000 7ff49e7fc700 1 -- 172.21.15.163:0/1641728831 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff4a8050c20 con 0x7ff4b015db10 2023-10-09T06:36:28.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.177+0000 7ff4b4d96700 1 -- 172.21.15.163:0/1641728831 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4b006a5b0 con 0x7ff490067200 2023-10-09T06:36:28.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.179+0000 7ff49e7fc700 1 -- 172.21.15.163:0/1641728831 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7ff4b006a5b0 con 0x7ff490067200 2023-10-09T06:36:28.180 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:36:28.180 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:28.180 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "cephfs" 2023-10-09T06:36:28.180 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:28.180 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:36:28.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.181+0000 7ff4b4d96700 1 -- 172.21.15.163:0/1641728831 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff490067200 msgr2=0x7ff4900696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:28.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.181+0000 7ff4b4d96700 1 --2- 172.21.15.163:0/1641728831 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff490067200 0x7ff4900696b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff4a00097c0 tx=0x7ff4a0019000 comp rx=0 tx=0).stop 2023-10-09T06:36:28.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.181+0000 7ff4b4d96700 1 -- 172.21.15.163:0/1641728831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4b015db10 msgr2=0x7ff4b00ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:28.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.181+0000 7ff4b4d96700 1 --2- 172.21.15.163:0/1641728831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4b015db10 0x7ff4b00ce760 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7ff4a800c530 tx=0x7ff4a8007f90 comp rx=0 tx=0).stop 2023-10-09T06:36:28.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.181+0000 7ff4b4d96700 1 -- 172.21.15.163:0/1641728831 shutdown_connections 2023-10-09T06:36:28.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.181+0000 7ff4b4d96700 1 --2- 172.21.15.163:0/1641728831 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff490067200 0x7ff4900696b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:28.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.181+0000 7ff4b4d96700 1 --2- 172.21.15.163:0/1641728831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4b015db10 0x7ff4b00ce760 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:28.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.181+0000 7ff4b4d96700 1 --2- 172.21.15.163:0/1641728831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4b0159890 0x7ff4b0158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:28.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.181+0000 7ff4b4d96700 1 --2- 172.21.15.163:0/1641728831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4b0159c60 0x7ff4b00c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:28.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.181+0000 7ff4b4d96700 1 -- 172.21.15.163:0/1641728831 >> 172.21.15.163:0/1641728831 conn(0x7ff4b0068e20 msgr2=0x7ff4b015b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:28.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.181+0000 7ff4b4d96700 1 -- 172.21.15.163:0/1641728831 shutdown_connections 2023-10-09T06:36:28.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.181+0000 7ff4b4d96700 1 -- 172.21.15.163:0/1641728831 wait complete. 2023-10-09T06:36:28.194 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-09T06:36:28.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.311+0000 7fb980979700 1 Processor -- start 2023-10-09T06:36:28.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.311+0000 7fb980979700 1 -- start start 2023-10-09T06:36:28.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.312+0000 7fb980979700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb97c0b4e70 0x7fb97c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:28.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.312+0000 7fb980979700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb97c0b16d0 0x7fb97c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:28.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.312+0000 7fb980979700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb97c0b2080 con 0x7fb97c15d240 2023-10-09T06:36:28.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.312+0000 7fb980979700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb97c0cfb00 con 0x7fb97c0b16d0 2023-10-09T06:36:28.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.312+0000 7fb980979700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb97c0b5c40 con 0x7fb97c0b4e70 2023-10-09T06:36:28.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.312+0000 7fb97affd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb97c0b16d0 0x7fb97c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:28.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.312+0000 7fb97affd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb97c0b16d0 0x7fb97c0b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55076/0 (socket says 172.21.15.163:55076) 2023-10-09T06:36:28.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.312+0000 7fb97affd700 1 -- 172.21.15.163:0/2484949677 learned_addr learned my addr 172.21.15.163:0/2484949677 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:28.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.312+0000 7fb97b7fe700 1 --2- 172.21.15.163:0/2484949677 >> v2:172.21.15.181:3300/0 conn(0x7fb97c0b4e70 0x7fb97c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:28.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.312+0000 7fb97affd700 1 -- 172.21.15.163:0/2484949677 >> v2:172.21.15.181:3300/0 conn(0x7fb97c0b4e70 msgr2=0x7fb97c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:28.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.312+0000 7fb97affd700 1 --2- 172.21.15.163:0/2484949677 >> v2:172.21.15.181:3300/0 conn(0x7fb97c0b4e70 0x7fb97c0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:28.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.312+0000 7fb97affd700 1 -- 172.21.15.163:0/2484949677 >> v1:172.21.15.181:6789/0 conn(0x7fb97c15d240 legacy=0x7fb97c15d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:28.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.312+0000 7fb97affd700 1 -- 172.21.15.163:0/2484949677 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97c0b5f20 con 0x7fb97c0b16d0 2023-10-09T06:36:28.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.312+0000 7fb97b7fe700 1 --2- 172.21.15.163:0/2484949677 >> v2:172.21.15.181:3300/0 conn(0x7fb97c0b4e70 0x7fb97c0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:36:28.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.312+0000 7fb97affd700 1 --2- 172.21.15.163:0/2484949677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb97c0b16d0 0x7fb97c0b1b40 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fb97c064910 tx=0x7fb970009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2e6ec12819c977 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:28.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.313+0000 7fb97a7fc700 1 -- 172.21.15.163:0/2484949677 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97000b820 con 0x7fb97c0b16d0 2023-10-09T06:36:28.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.313+0000 7fb97a7fc700 1 -- 172.21.15.163:0/2484949677 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97000b980 con 0x7fb97c0b16d0 2023-10-09T06:36:28.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.313+0000 7fb97a7fc700 1 -- 172.21.15.163:0/2484949677 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97000bd20 con 0x7fb97c0b16d0 2023-10-09T06:36:28.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.313+0000 7fb980979700 1 -- 172.21.15.163:0/2484949677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb97c0b16d0 msgr2=0x7fb97c0b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:28.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.313+0000 7fb980979700 1 --2- 172.21.15.163:0/2484949677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb97c0b16d0 0x7fb97c0b1b40 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fb97c064910 tx=0x7fb970009250 comp rx=0 tx=0).stop 2023-10-09T06:36:28.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.313+0000 7fb980979700 1 -- 172.21.15.163:0/2484949677 shutdown_connections 2023-10-09T06:36:28.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.313+0000 7fb980979700 1 --2- 172.21.15.163:0/2484949677 >> v2:172.21.15.181:3300/0 conn(0x7fb97c0b4e70 0x7fb97c0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:28.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.313+0000 7fb980979700 1 --2- 172.21.15.163:0/2484949677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb97c0b16d0 0x7fb97c0b1b40 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:28.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.313+0000 7fb980979700 1 -- 172.21.15.163:0/2484949677 >> 172.21.15.163:0/2484949677 conn(0x7fb97c0d10c0 msgr2=0x7fb97c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:28.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.313+0000 7fb980979700 1 -- 172.21.15.163:0/2484949677 shutdown_connections 2023-10-09T06:36:28.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.313+0000 7fb980979700 1 -- 172.21.15.163:0/2484949677 wait complete. 2023-10-09T06:36:28.319 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.314+0000 7fb980979700 1 Processor -- start 2023-10-09T06:36:28.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.314+0000 7fb980979700 1 -- start start 2023-10-09T06:36:28.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.315+0000 7fb980979700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb97c0b16d0 0x7fb97c16ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:28.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.315+0000 7fb980979700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb97c0b4e70 0x7fb97c0b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:28.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.315+0000 7fb980979700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb97c15d240 0x7fb97c0b4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:28.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.315+0000 7fb980979700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c16b280 con 0x7fb97c15d240 2023-10-09T06:36:28.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.315+0000 7fb980979700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c16b080 con 0x7fb97c0b16d0 2023-10-09T06:36:28.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.315+0000 7fb980979700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c158850 con 0x7fb97c0b4e70 2023-10-09T06:36:28.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.315+0000 7fb97bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb97c15d240 0x7fb97c0b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:28.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.315+0000 7fb97bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb97c15d240 0x7fb97c0b4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55090/0 (socket says 172.21.15.163:55090) 2023-10-09T06:36:28.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.315+0000 7fb97bfff700 1 -- 172.21.15.163:0/35154752 learned_addr learned my addr 172.21.15.163:0/35154752 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:28.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.315+0000 7fb97affd700 1 --2- 172.21.15.163:0/35154752 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb97c0b4e70 0x7fb97c0b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:28.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.315+0000 7fb97b7fe700 1 --2- 172.21.15.163:0/35154752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb97c0b16d0 0x7fb97c16ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:28.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.316+0000 7fb97bfff700 1 -- 172.21.15.163:0/35154752 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb97c0b4e70 msgr2=0x7fb97c0b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:28.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.316+0000 7fb97bfff700 1 --2- 172.21.15.163:0/35154752 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb97c0b4e70 0x7fb97c0b3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:28.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.316+0000 7fb97bfff700 1 -- 172.21.15.163:0/35154752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb97c0b16d0 msgr2=0x7fb97c16ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:28.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.316+0000 7fb97bfff700 1 --2- 172.21.15.163:0/35154752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb97c0b16d0 0x7fb97c16ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:28.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.316+0000 7fb97bfff700 1 -- 172.21.15.163:0/35154752 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9700094f0 con 0x7fb97c15d240 2023-10-09T06:36:28.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.316+0000 7fb97b7fe700 1 --2- 172.21.15.163:0/35154752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb97c0b16d0 0x7fb97c16ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:36:28.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.316+0000 7fb97bfff700 1 --2- 172.21.15.163:0/35154752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb97c15d240 0x7fb97c0b4160 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fb97400b4e0 tx=0x7fb974006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:28.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.316+0000 7fb9637fe700 1 -- 172.21.15.163:0/35154752 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97400f5e0 con 0x7fb97c15d240 2023-10-09T06:36:28.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.316+0000 7fb9637fe700 1 -- 172.21.15.163:0/35154752 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97400f740 con 0x7fb97c15d240 2023-10-09T06:36:28.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.316+0000 7fb980979700 1 -- 172.21.15.163:0/35154752 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb97c16c9c0 con 0x7fb97c15d240 2023-10-09T06:36:28.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.316+0000 7fb9637fe700 1 -- 172.21.15.163:0/35154752 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97400fb00 con 0x7fb97c15d240 2023-10-09T06:36:28.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.316+0000 7fb980979700 1 -- 172.21.15.163:0/35154752 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb97c16ded0 con 0x7fb97c15d240 2023-10-09T06:36:28.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.317+0000 7fb980979700 1 -- 172.21.15.163:0/35154752 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb97c089600 con 0x7fb97c15d240 2023-10-09T06:36:28.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.317+0000 7fb9637fe700 1 -- 172.21.15.163:0/35154752 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fb97401a400 con 0x7fb97c15d240 2023-10-09T06:36:28.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.318+0000 7fb9637fe700 1 --2- 172.21.15.163:0/35154752 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb968067200 0x7fb9680696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:28.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.318+0000 7fb9637fe700 1 -- 172.21.15.163:0/35154752 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb9740845b0 con 0x7fb97c15d240 2023-10-09T06:36:28.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.318+0000 7fb97b7fe700 1 --2- 172.21.15.163:0/35154752 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb968067200 0x7fb9680696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:28.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.318+0000 7fb97b7fe700 1 --2- 172.21.15.163:0/35154752 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb968067200 0x7fb9680696b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb96c00d640 tx=0x7fb96c007670 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:28.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.322+0000 7fb9637fe700 1 -- 172.21.15.163:0/35154752 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb974051550 con 0x7fb97c15d240 2023-10-09T06:36:28.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.507+0000 7fb980979700 1 -- 172.21.15.163:0/35154752 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb97c16b3c0 con 0x7fb97c15d240 2023-10-09T06:36:28.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.508+0000 7fb9637fe700 1 -- 172.21.15.163:0/35154752 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb974054b70 con 0x7fb97c15d240 2023-10-09T06:36:28.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.510+0000 7fb980979700 1 -- 172.21.15.163:0/35154752 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb968067200 msgr2=0x7fb9680696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:28.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.510+0000 7fb980979700 1 --2- 172.21.15.163:0/35154752 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb968067200 0x7fb9680696b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb96c00d640 tx=0x7fb96c007670 comp rx=0 tx=0).stop 2023-10-09T06:36:28.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.510+0000 7fb980979700 1 -- 172.21.15.163:0/35154752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb97c15d240 msgr2=0x7fb97c0b4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:28.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.510+0000 7fb980979700 1 --2- 172.21.15.163:0/35154752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb97c15d240 0x7fb97c0b4160 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fb97400b4e0 tx=0x7fb974006f90 comp rx=0 tx=0).stop 2023-10-09T06:36:28.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.510+0000 7fb980979700 1 -- 172.21.15.163:0/35154752 shutdown_connections 2023-10-09T06:36:28.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.510+0000 7fb980979700 1 --2- 172.21.15.163:0/35154752 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb968067200 0x7fb9680696b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:28.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.510+0000 7fb980979700 1 --2- 172.21.15.163:0/35154752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb97c15d240 0x7fb97c0b4160 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:28.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.510+0000 7fb980979700 1 --2- 172.21.15.163:0/35154752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb97c0b16d0 0x7fb97c16ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:28.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.510+0000 7fb980979700 1 --2- 172.21.15.163:0/35154752 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb97c0b4e70 0x7fb97c0b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:28.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.510+0000 7fb980979700 1 -- 172.21.15.163:0/35154752 >> 172.21.15.163:0/35154752 conn(0x7fb97c0d10c0 msgr2=0x7fb97c0b54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:28.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.510+0000 7fb980979700 1 -- 172.21.15.163:0/35154752 shutdown_connections 2023-10-09T06:36:28.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.510+0000 7fb980979700 1 -- 172.21.15.163:0/35154752 wait complete. 2023-10-09T06:36:28.523 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume info cephfs --human_readable 2023-10-09T06:36:28.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.640+0000 7f407c9a2700 1 Processor -- start 2023-10-09T06:36:28.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.640+0000 7f407c9a2700 1 -- start start 2023-10-09T06:36:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.641+0000 7f407c9a2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4078159a60 0x7f40780c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.641+0000 7f407c9a2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f40780cb230 0x7f40780cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.641+0000 7f407c9a2700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40780c9cb0 con 0x7f40780ccc00 2023-10-09T06:36:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.641+0000 7f407c9a2700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40780cbbd0 con 0x7f40780cb230 2023-10-09T06:36:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.641+0000 7f407c9a2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407806dfc0 con 0x7f4078159a60 2023-10-09T06:36:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.641+0000 7f4077fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f40780cb230 0x7f40780cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.641+0000 7f4076ffd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f40780ccc00 0x7f4078069ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53446/0 (socket says 172.21.15.163:53446) 2023-10-09T06:36:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.641+0000 7f4076ffd700 1 -- 172.21.15.163:0/1601333949 learned_addr learned my addr 172.21.15.163:0/1601333949 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:28.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.642+0000 7f40777fe700 1 --2- 172.21.15.163:0/1601333949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4078159a60 0x7f40780c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:28.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.642+0000 7f4077fff700 1 -- 172.21.15.163:0/1601333949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4078159a60 msgr2=0x7f40780c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:28.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.642+0000 7f4077fff700 1 --2- 172.21.15.163:0/1601333949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4078159a60 0x7f40780c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:28.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.642+0000 7f4077fff700 1 -- 172.21.15.163:0/1601333949 >> v1:172.21.15.163:6789/0 conn(0x7f40780ccc00 legacy=0x7f4078069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:28.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.642+0000 7f40767fc700 1 -- 172.21.15.163:0/1601333949 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2817696938 0 0) 0x7f40780c9cb0 con 0x7f40780ccc00 2023-10-09T06:36:28.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.642+0000 7f4077fff700 1 -- 172.21.15.163:0/1601333949 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40780c9f30 con 0x7f40780cb230 2023-10-09T06:36:28.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.642+0000 7f4077fff700 1 --2- 172.21.15.163:0/1601333949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40780cb230 0x7f40780cb690 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f4070009910 tx=0x7f40700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b643fe64fad80058 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:28.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.642+0000 7f40767fc700 1 -- 172.21.15.163:0/1601333949 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407000b6d0 con 0x7f40780cb230 2023-10-09T06:36:28.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.642+0000 7f40767fc700 1 -- 172.21.15.163:0/1601333949 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407000b830 con 0x7f40780cb230 2023-10-09T06:36:28.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.642+0000 7f40767fc700 1 -- 172.21.15.163:0/1601333949 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407000bbf0 con 0x7f40780cb230 2023-10-09T06:36:28.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.643+0000 7f407c9a2700 1 -- 172.21.15.163:0/1601333949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40780cb230 msgr2=0x7f40780cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:28.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.643+0000 7f407c9a2700 1 --2- 172.21.15.163:0/1601333949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40780cb230 0x7f40780cb690 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f4070009910 tx=0x7f40700091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:28.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.643+0000 7f407c9a2700 1 -- 172.21.15.163:0/1601333949 shutdown_connections 2023-10-09T06:36:28.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.643+0000 7f407c9a2700 1 --2- 172.21.15.163:0/1601333949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40780cb230 0x7f40780cb690 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:28.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.643+0000 7f407c9a2700 1 --2- 172.21.15.163:0/1601333949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4078159a60 0x7f40780c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:28.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.643+0000 7f407c9a2700 1 -- 172.21.15.163:0/1601333949 >> 172.21.15.163:0/1601333949 conn(0x7f407806baa0 msgr2=0x7f407806bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:28.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.643+0000 7f407c9a2700 1 -- 172.21.15.163:0/1601333949 shutdown_connections 2023-10-09T06:36:28.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.643+0000 7f407c9a2700 1 -- 172.21.15.163:0/1601333949 wait complete. 2023-10-09T06:36:28.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.644+0000 7f407c9a2700 1 Processor -- start 2023-10-09T06:36:28.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.644+0000 7f407c9a2700 1 -- start start 2023-10-09T06:36:28.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.645+0000 7f407c9a2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40780cb230 0x7f4078157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:28.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.645+0000 7f407c9a2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40780ccc00 0x7f4078154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:28.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.645+0000 7f407c9a2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4078159a60 0x7f40780b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:28.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.645+0000 7f407c9a2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40780b1b10 con 0x7f40780ccc00 2023-10-09T06:36:28.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.645+0000 7f407c9a2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4078157820 con 0x7f40780cb230 2023-10-09T06:36:28.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.645+0000 7f407c9a2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4078157b00 con 0x7f4078159a60 2023-10-09T06:36:28.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.645+0000 7f4076ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40780ccc00 0x7f4078154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:28.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.645+0000 7f4076ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40780ccc00 0x7f4078154c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59560/0 (socket says 172.21.15.163:59560) 2023-10-09T06:36:28.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.645+0000 7f4076ffd700 1 -- 172.21.15.163:0/4219331345 learned_addr learned my addr 172.21.15.163:0/4219331345 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:28.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.645+0000 7f4077fff700 1 --2- 172.21.15.163:0/4219331345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4078159a60 0x7f40780b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:28.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.645+0000 7f40777fe700 1 --2- 172.21.15.163:0/4219331345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40780cb230 0x7f4078157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:28.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.645+0000 7f4076ffd700 1 -- 172.21.15.163:0/4219331345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4078159a60 msgr2=0x7f40780b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:28.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.645+0000 7f4076ffd700 1 --2- 172.21.15.163:0/4219331345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4078159a60 0x7f40780b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:28.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.645+0000 7f4076ffd700 1 -- 172.21.15.163:0/4219331345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40780cb230 msgr2=0x7f4078157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:28.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.645+0000 7f4076ffd700 1 --2- 172.21.15.163:0/4219331345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40780cb230 0x7f4078157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:28.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.645+0000 7f4076ffd700 1 -- 172.21.15.163:0/4219331345 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4070009420 con 0x7f40780ccc00 2023-10-09T06:36:28.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.645+0000 7f4076ffd700 1 --2- 172.21.15.163:0/4219331345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40780ccc00 0x7f4078154c90 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f406000c7e0 tx=0x7f4060007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:28.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.645+0000 7f406f7fe700 1 -- 172.21.15.163:0/4219331345 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4060005b30 con 0x7f40780ccc00 2023-10-09T06:36:28.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.645+0000 7f406f7fe700 1 -- 172.21.15.163:0/4219331345 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4060005c90 con 0x7f40780ccc00 2023-10-09T06:36:28.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.645+0000 7f407c9a2700 1 -- 172.21.15.163:0/4219331345 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4078157de0 con 0x7f40780ccc00 2023-10-09T06:36:28.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.645+0000 7f406f7fe700 1 -- 172.21.15.163:0/4219331345 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4060007300 con 0x7f40780ccc00 2023-10-09T06:36:28.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.645+0000 7f407c9a2700 1 -- 172.21.15.163:0/4219331345 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40781551d0 con 0x7f40780ccc00 2023-10-09T06:36:28.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.646+0000 7f407c9a2700 1 -- 172.21.15.163:0/4219331345 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40780b1c80 con 0x7f40780ccc00 2023-10-09T06:36:28.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.646+0000 7f406f7fe700 1 -- 172.21.15.163:0/4219331345 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f406001b400 con 0x7f40780ccc00 2023-10-09T06:36:28.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.647+0000 7f406f7fe700 1 --2- 172.21.15.163:0/4219331345 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4058067200 0x7f40580696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:28.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.647+0000 7f406f7fe700 1 -- 172.21.15.163:0/4219331345 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4060084460 con 0x7f40780ccc00 2023-10-09T06:36:28.651 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.647+0000 7f40777fe700 1 --2- 172.21.15.163:0/4219331345 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4058067200 0x7f40580696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:28.651 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.647+0000 7f40777fe700 1 --2- 172.21.15.163:0/4219331345 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4058067200 0x7f40580696b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4068007600 tx=0x7f406800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:28.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.651+0000 7f406f7fe700 1 -- 172.21.15.163:0/4219331345 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4060051400 con 0x7f40780ccc00 2023-10-09T06:36:28.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.860+0000 7f407c9a2700 1 -- 172.21.15.163:0/4219331345 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume info", "vol_name": "cephfs", "human_readable": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f407806ec10 con 0x7f4058067200 2023-10-09T06:36:28.863 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:36:28.862+0000 7ff9594e4700 -1 client.6366 remove_session_caps still has dirty|flushing caps on 0x10000000001.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2023-10-09T06:36:02.736614+0000 mtime=2023-10-09T06:36:02.736614+0000 ctime=2023-10-09T06:36:02.741946+0000 change_attr=2 caps=- flushing_caps=Ax COMPLETE parents=0x10000000000.head["group_0000000001000982"] has_dir_layout 0x55f9b12f4600) 2023-10-09T06:36:28.867 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:36:28.867+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:36:28.868 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:36:28.867+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:36:28.868 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:36:28.867+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:36:28.868 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:36:28.867+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:36:28.868 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:36:28.867+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:36:28.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.873+0000 7f406f7fe700 1 -- 172.21.15.163:0/4219331345 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+471 (secure 0 0 0) 0x7f407806ec10 con 0x7f4058067200 2023-10-09T06:36:28.874 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:36:28.874 INFO:teuthology.orchestra.run.smithi163.stdout: "mon_addrs": [ 2023-10-09T06:36:28.874 INFO:teuthology.orchestra.run.smithi163.stdout: "172.21.15.163:6789", 2023-10-09T06:36:28.874 INFO:teuthology.orchestra.run.smithi163.stdout: "172.21.15.181:6789", 2023-10-09T06:36:28.874 INFO:teuthology.orchestra.run.smithi163.stdout: "172.21.15.181:6790" 2023-10-09T06:36:28.874 INFO:teuthology.orchestra.run.smithi163.stdout: ], 2023-10-09T06:36:28.875 INFO:teuthology.orchestra.run.smithi163.stdout: "pools": { 2023-10-09T06:36:28.875 INFO:teuthology.orchestra.run.smithi163.stdout: "data": [ 2023-10-09T06:36:28.875 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:28.875 INFO:teuthology.orchestra.run.smithi163.stdout: "avail": " 340G", 2023-10-09T06:36:28.875 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "cephfs_data", 2023-10-09T06:36:28.875 INFO:teuthology.orchestra.run.smithi163.stdout: "used": " 0 " 2023-10-09T06:36:28.875 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:28.875 INFO:teuthology.orchestra.run.smithi163.stdout: ], 2023-10-09T06:36:28.876 INFO:teuthology.orchestra.run.smithi163.stdout: "metadata": [ 2023-10-09T06:36:28.876 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:28.876 INFO:teuthology.orchestra.run.smithi163.stdout: "avail": " 340G", 2023-10-09T06:36:28.876 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "cephfs_metadata", 2023-10-09T06:36:28.876 INFO:teuthology.orchestra.run.smithi163.stdout: "used": "64.0k" 2023-10-09T06:36:28.876 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:28.876 INFO:teuthology.orchestra.run.smithi163.stdout: ] 2023-10-09T06:36:28.876 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:28.877 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:36:28.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.875+0000 7f407c9a2700 1 -- 172.21.15.163:0/4219331345 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4058067200 msgr2=0x7f40580696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:28.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.875+0000 7f407c9a2700 1 --2- 172.21.15.163:0/4219331345 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4058067200 0x7f40580696b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4068007600 tx=0x7f406800b040 comp rx=0 tx=0).stop 2023-10-09T06:36:28.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.875+0000 7f407c9a2700 1 -- 172.21.15.163:0/4219331345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40780ccc00 msgr2=0x7f4078154c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:28.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.875+0000 7f407c9a2700 1 --2- 172.21.15.163:0/4219331345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40780ccc00 0x7f4078154c90 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f406000c7e0 tx=0x7f4060007df0 comp rx=0 tx=0).stop 2023-10-09T06:36:28.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.875+0000 7f407c9a2700 1 -- 172.21.15.163:0/4219331345 shutdown_connections 2023-10-09T06:36:28.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.875+0000 7f407c9a2700 1 --2- 172.21.15.163:0/4219331345 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4058067200 0x7f40580696b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:28.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.875+0000 7f407c9a2700 1 --2- 172.21.15.163:0/4219331345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4078159a60 0x7f40780b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:28.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.875+0000 7f407c9a2700 1 --2- 172.21.15.163:0/4219331345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40780cb230 0x7f4078157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:28.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.875+0000 7f407c9a2700 1 --2- 172.21.15.163:0/4219331345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40780ccc00 0x7f4078154c90 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:28.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.875+0000 7f407c9a2700 1 -- 172.21.15.163:0/4219331345 >> 172.21.15.163:0/4219331345 conn(0x7f407806baa0 msgr2=0x7f40780cefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:28.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.875+0000 7f407c9a2700 1 -- 172.21.15.163:0/4219331345 shutdown_connections 2023-10-09T06:36:28.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:28.875+0000 7f407c9a2700 1 -- 172.21.15.163:0/4219331345 wait complete. 2023-10-09T06:36:28.888 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-09T06:36:28.889 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:36:28.927 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:36:28.968 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-09T06:36:28.969 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-09T06:36:28.969 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:36:28.969 DEBUG:teuthology.orchestra.run.smithi163:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-09T06:36:29.006 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi163.front.sepia.ceph.com 2023-10-09T06:36:29.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:29.006 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-09T06:36:29.062 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-09T06:36:29.062 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:36:29.062 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-09T06:36:29.104 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-09T06:36:29.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:29.105 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-09T06:36:29.162 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:36:29.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.276+0000 7fbc32d7c700 1 Processor -- start 2023-10-09T06:36:29.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.276+0000 7fbc32d7c700 1 -- start start 2023-10-09T06:36:29.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.277+0000 7fbc32d7c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbc2c159890 0x7fbc2c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:29.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.277+0000 7fbc32d7c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbc2c069560 0x7fbc2c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:29.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.277+0000 7fbc32d7c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc2c069ee0 con 0x7fbc2c159c60 2023-10-09T06:36:29.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.277+0000 7fbc32d7c700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc2c15a160 con 0x7fbc2c159890 2023-10-09T06:36:29.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.277+0000 7fbc32d7c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc2c0d13a0 con 0x7fbc2c069560 2023-10-09T06:36:29.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.277+0000 7fbc30b18700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbc2c159890 0x7fbc2c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:29.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.277+0000 7fbc30b18700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbc2c159890 0x7fbc2c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59564/0 (socket says 172.21.15.163:59564) 2023-10-09T06:36:29.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.277+0000 7fbc30b18700 1 -- 172.21.15.163:0/1278229476 learned_addr learned my addr 172.21.15.163:0/1278229476 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:29.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.277+0000 7fbc31319700 1 --2- 172.21.15.163:0/1278229476 >> v2:172.21.15.181:3300/0 conn(0x7fbc2c069560 0x7fbc2c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:29.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.278+0000 7fbc30b18700 1 -- 172.21.15.163:0/1278229476 >> v2:172.21.15.181:3300/0 conn(0x7fbc2c069560 msgr2=0x7fbc2c0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:29.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.278+0000 7fbc30b18700 1 --2- 172.21.15.163:0/1278229476 >> v2:172.21.15.181:3300/0 conn(0x7fbc2c069560 0x7fbc2c0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:29.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.278+0000 7fbc30b18700 1 -- 172.21.15.163:0/1278229476 >> v1:172.21.15.181:6789/0 conn(0x7fbc2c159c60 legacy=0x7fbc2c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:29.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.278+0000 7fbc30b18700 1 -- 172.21.15.163:0/1278229476 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2c15e140 con 0x7fbc2c159890 2023-10-09T06:36:29.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.278+0000 7fbc30b18700 1 --2- 172.21.15.163:0/1278229476 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc2c159890 0x7fbc2c15abd0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c06a890 tx=0x7fbc28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ebc1958c35956e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:29.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.278+0000 7fbc237fe700 1 -- 172.21.15.163:0/1278229476 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2800b860 con 0x7fbc2c159890 2023-10-09T06:36:29.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.278+0000 7fbc237fe700 1 -- 172.21.15.163:0/1278229476 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2800b9c0 con 0x7fbc2c159890 2023-10-09T06:36:29.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.278+0000 7fbc237fe700 1 -- 172.21.15.163:0/1278229476 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2800bd60 con 0x7fbc2c159890 2023-10-09T06:36:29.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.278+0000 7fbc32d7c700 1 -- 172.21.15.163:0/1278229476 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc2c159890 msgr2=0x7fbc2c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:29.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.278+0000 7fbc32d7c700 1 --2- 172.21.15.163:0/1278229476 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc2c159890 0x7fbc2c15abd0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c06a890 tx=0x7fbc28009250 comp rx=0 tx=0).stop 2023-10-09T06:36:29.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.278+0000 7fbc32d7c700 1 -- 172.21.15.163:0/1278229476 shutdown_connections 2023-10-09T06:36:29.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.278+0000 7fbc32d7c700 1 --2- 172.21.15.163:0/1278229476 >> v2:172.21.15.181:3300/0 conn(0x7fbc2c069560 0x7fbc2c0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:29.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.278+0000 7fbc32d7c700 1 --2- 172.21.15.163:0/1278229476 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc2c159890 0x7fbc2c15abd0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:29.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.278+0000 7fbc32d7c700 1 -- 172.21.15.163:0/1278229476 >> 172.21.15.163:0/1278229476 conn(0x7fbc2c068e20 msgr2=0x7fbc2c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:29.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.278+0000 7fbc32d7c700 1 -- 172.21.15.163:0/1278229476 shutdown_connections 2023-10-09T06:36:29.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.278+0000 7fbc32d7c700 1 -- 172.21.15.163:0/1278229476 wait complete. 2023-10-09T06:36:29.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.279+0000 7fbc32d7c700 1 Processor -- start 2023-10-09T06:36:29.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.280+0000 7fbc32d7c700 1 -- start start 2023-10-09T06:36:29.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.280+0000 7fbc32d7c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc2c069560 0x7fbc2c0b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:29.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.280+0000 7fbc32d7c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc2c159890 0x7fbc2c16d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:29.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.280+0000 7fbc32d7c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc2c159c60 0x7fbc2c16d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:29.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.280+0000 7fbc32d7c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c0b4df0 con 0x7fbc2c159c60 2023-10-09T06:36:29.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.280+0000 7fbc32d7c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c0b44c0 con 0x7fbc2c159890 2023-10-09T06:36:29.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.280+0000 7fbc32d7c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c0b47a0 con 0x7fbc2c069560 2023-10-09T06:36:29.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.280+0000 7fbc31319700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc2c159c60 0x7fbc2c16d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:29.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.281+0000 7fbc31319700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc2c159c60 0x7fbc2c16d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59574/0 (socket says 172.21.15.163:59574) 2023-10-09T06:36:29.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.281+0000 7fbc31319700 1 -- 172.21.15.163:0/1542083663 learned_addr learned my addr 172.21.15.163:0/1542083663 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:29.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.281+0000 7fbc23fff700 1 --2- 172.21.15.163:0/1542083663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc2c159890 0x7fbc2c16d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:29.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.281+0000 7fbc30b18700 1 --2- 172.21.15.163:0/1542083663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc2c069560 0x7fbc2c0b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:29.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.281+0000 7fbc31319700 1 -- 172.21.15.163:0/1542083663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc2c069560 msgr2=0x7fbc2c0b5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:29.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.281+0000 7fbc31319700 1 --2- 172.21.15.163:0/1542083663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc2c069560 0x7fbc2c0b5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:29.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.281+0000 7fbc31319700 1 -- 172.21.15.163:0/1542083663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc2c159890 msgr2=0x7fbc2c16d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:29.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.281+0000 7fbc31319700 1 --2- 172.21.15.163:0/1542083663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc2c159890 0x7fbc2c16d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:29.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.281+0000 7fbc31319700 1 -- 172.21.15.163:0/1542083663 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc280094f0 con 0x7fbc2c159c60 2023-10-09T06:36:29.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.281+0000 7fbc23fff700 1 --2- 172.21.15.163:0/1542083663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc2c159890 0x7fbc2c16d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:29.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.281+0000 7fbc31319700 1 --2- 172.21.15.163:0/1542083663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc2c159c60 0x7fbc2c16d890 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009fa0 tx=0x7fbc24009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:29.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.281+0000 7fbc20ff9700 1 -- 172.21.15.163:0/1542083663 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2400ed20 con 0x7fbc2c159c60 2023-10-09T06:36:29.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.281+0000 7fbc20ff9700 1 -- 172.21.15.163:0/1542083663 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2400ee80 con 0x7fbc2c159c60 2023-10-09T06:36:29.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.281+0000 7fbc32d7c700 1 -- 172.21.15.163:0/1542083663 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc2c0b4a80 con 0x7fbc2c159c60 2023-10-09T06:36:29.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.281+0000 7fbc20ff9700 1 -- 172.21.15.163:0/1542083663 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc24014670 con 0x7fbc2c159c60 2023-10-09T06:36:29.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.281+0000 7fbc32d7c700 1 -- 172.21.15.163:0/1542083663 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc2c143bc0 con 0x7fbc2c159c60 2023-10-09T06:36:29.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.282+0000 7fbc32d7c700 1 -- 172.21.15.163:0/1542083663 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc2c0b4fa0 con 0x7fbc2c159c60 2023-10-09T06:36:29.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.283+0000 7fbc20ff9700 1 -- 172.21.15.163:0/1542083663 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fbc240149e0 con 0x7fbc2c159c60 2023-10-09T06:36:29.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.283+0000 7fbc20ff9700 1 --2- 172.21.15.163:0/1542083663 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbc18067200 0x7fbc180696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:29.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.283+0000 7fbc20ff9700 1 -- 172.21.15.163:0/1542083663 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbc24085120 con 0x7fbc2c159c60 2023-10-09T06:36:29.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.283+0000 7fbc30b18700 1 --2- 172.21.15.163:0/1542083663 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbc18067200 0x7fbc180696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:29.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.284+0000 7fbc30b18700 1 --2- 172.21.15.163:0/1542083663 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbc18067200 0x7fbc180696b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c06a890 tx=0x7fbc28017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:29.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.287+0000 7fbc20ff9700 1 -- 172.21.15.163:0/1542083663 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc2c0b4fa0 con 0x7fbc2c159c60 2023-10-09T06:36:29.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.512+0000 7fbc32d7c700 1 -- 172.21.15.163:0/1542083663 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc2c04c3c0 con 0x7fbc2c159c60 2023-10-09T06:36:29.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.512+0000 7fbc20ff9700 1 -- 172.21.15.163:0/1542083663 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fbc24052040 con 0x7fbc2c159c60 2023-10-09T06:36:29.514 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:29.514 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":58,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5501,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/862991395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":862991395},{"type":"v1","addr":"172.21.15.181:6836","nonce":862991395}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/699275566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":699275566},{"type":"v1","addr":"172.21.15.181:6837","nonce":699275566}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6504,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3664732208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3664732208},{"type":"v1","addr":"172.21.15.163:6834","nonce":3664732208}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:36:18.163421+0000","modified":"2023-10-09T06:36:21.793136+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6507},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6507":{"gid":6507,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/2687247401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2687247401},{"type":"v1","addr":"172.21.15.163:6835","nonce":2687247401}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-09T06:36:29.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.514+0000 7fbc32d7c700 1 -- 172.21.15.163:0/1542083663 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbc18067200 msgr2=0x7fbc180696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:29.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.514+0000 7fbc32d7c700 1 --2- 172.21.15.163:0/1542083663 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbc18067200 0x7fbc180696b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c06a890 tx=0x7fbc28017000 comp rx=0 tx=0).stop 2023-10-09T06:36:29.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.514+0000 7fbc32d7c700 1 -- 172.21.15.163:0/1542083663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc2c159c60 msgr2=0x7fbc2c16d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:29.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.514+0000 7fbc32d7c700 1 --2- 172.21.15.163:0/1542083663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc2c159c60 0x7fbc2c16d890 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009fa0 tx=0x7fbc24009900 comp rx=0 tx=0).stop 2023-10-09T06:36:29.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.514+0000 7fbc32d7c700 1 -- 172.21.15.163:0/1542083663 shutdown_connections 2023-10-09T06:36:29.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.514+0000 7fbc32d7c700 1 --2- 172.21.15.163:0/1542083663 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbc18067200 0x7fbc180696b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:29.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.514+0000 7fbc32d7c700 1 --2- 172.21.15.163:0/1542083663 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc2c159c60 0x7fbc2c16d890 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:29.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.514+0000 7fbc32d7c700 1 --2- 172.21.15.163:0/1542083663 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc2c069560 0x7fbc2c0b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:29.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.514+0000 7fbc32d7c700 1 --2- 172.21.15.163:0/1542083663 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc2c159890 0x7fbc2c16d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:29.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.515+0000 7fbc32d7c700 1 -- 172.21.15.163:0/1542083663 >> 172.21.15.163:0/1542083663 conn(0x7fbc2c068e20 msgr2=0x7fbc2c1534c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:29.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.515+0000 7fbc32d7c700 1 -- 172.21.15.163:0/1542083663 shutdown_connections 2023-10-09T06:36:29.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.515+0000 7fbc32d7c700 1 -- 172.21.15.163:0/1542083663 wait complete. 2023-10-09T06:36:29.520 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 58 2023-10-09T06:36:29.528 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:36:29.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.643+0000 7f308bfff700 1 Processor -- start 2023-10-09T06:36:29.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.643+0000 7f308bfff700 1 -- start start 2023-10-09T06:36:29.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.644+0000 7f308bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f308c1574d0 0x7f308c0cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:29.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.644+0000 7f308bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f308c0cc3c0 0x7f308c06b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:29.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.644+0000 7f308bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f308c06bf20 0x7f308c06cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:29.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.644+0000 7f308bfff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f308c0b16a0 con 0x7f308c1574d0 2023-10-09T06:36:29.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.644+0000 7f308bfff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f308c0b1c60 con 0x7f308c0cc3c0 2023-10-09T06:36:29.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.644+0000 7f308bfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f308c06c360 con 0x7f308c06bf20 2023-10-09T06:36:29.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.644+0000 7f308affd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f308c1574d0 0x7f308c0cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:29.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.644+0000 7f308affd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f308c1574d0 0x7f308c0cd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59578/0 (socket says 172.21.15.163:59578) 2023-10-09T06:36:29.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.644+0000 7f308affd700 1 -- 172.21.15.163:0/154357643 learned_addr learned my addr 172.21.15.163:0/154357643 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:29.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.644+0000 7f308a7fc700 1 --2- 172.21.15.163:0/154357643 >> v2:172.21.15.181:3300/0 conn(0x7f308c0cc3c0 0x7f308c06b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:29.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.644+0000 7f308b7fe700 1 --2- 172.21.15.163:0/154357643 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f308c06bf20 0x7f308c06cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:29.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.644+0000 7f308affd700 1 -- 172.21.15.163:0/154357643 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f308c06bf20 msgr2=0x7f308c06cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:29.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.644+0000 7f308affd700 1 --2- 172.21.15.163:0/154357643 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f308c06bf20 0x7f308c06cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:29.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.644+0000 7f308affd700 1 -- 172.21.15.163:0/154357643 >> v2:172.21.15.181:3300/0 conn(0x7f308c0cc3c0 msgr2=0x7f308c06b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:29.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.644+0000 7f308affd700 1 --2- 172.21.15.163:0/154357643 >> v2:172.21.15.181:3300/0 conn(0x7f308c0cc3c0 0x7f308c06b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:29.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.644+0000 7f308affd700 1 -- 172.21.15.163:0/154357643 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308c1578a0 con 0x7f308c1574d0 2023-10-09T06:36:29.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.644+0000 7f308a7fc700 1 --2- 172.21.15.163:0/154357643 >> v2:172.21.15.181:3300/0 conn(0x7f308c0cc3c0 0x7f308c06b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:29.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.644+0000 7f308affd700 1 --2- 172.21.15.163:0/154357643 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f308c1574d0 0x7f308c0cd270 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f308c06eec0 tx=0x7f307c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98ae7b316ba4dd7 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:29.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.645+0000 7f3089ffb700 1 -- 172.21.15.163:0/154357643 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c005c10 con 0x7f308c1574d0 2023-10-09T06:36:29.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.645+0000 7f3089ffb700 1 -- 172.21.15.163:0/154357643 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307c005d70 con 0x7f308c1574d0 2023-10-09T06:36:29.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.645+0000 7f3089ffb700 1 -- 172.21.15.163:0/154357643 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c00b880 con 0x7f308c1574d0 2023-10-09T06:36:29.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.645+0000 7f308bfff700 1 -- 172.21.15.163:0/154357643 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f308c1574d0 msgr2=0x7f308c0cd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:29.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.645+0000 7f308bfff700 1 --2- 172.21.15.163:0/154357643 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f308c1574d0 0x7f308c0cd270 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f308c06eec0 tx=0x7f307c009220 comp rx=0 tx=0).stop 2023-10-09T06:36:29.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.645+0000 7f308bfff700 1 -- 172.21.15.163:0/154357643 shutdown_connections 2023-10-09T06:36:29.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.645+0000 7f308bfff700 1 --2- 172.21.15.163:0/154357643 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f308c06bf20 0x7f308c06cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:29.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.645+0000 7f308bfff700 1 --2- 172.21.15.163:0/154357643 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f308c1574d0 0x7f308c0cd270 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:29.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.645+0000 7f308bfff700 1 --2- 172.21.15.163:0/154357643 >> v2:172.21.15.181:3300/0 conn(0x7f308c0cc3c0 0x7f308c06b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:29.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.645+0000 7f308bfff700 1 -- 172.21.15.163:0/154357643 >> 172.21.15.163:0/154357643 conn(0x7f308c0ce430 msgr2=0x7f308c0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:29.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.645+0000 7f308bfff700 1 -- 172.21.15.163:0/154357643 shutdown_connections 2023-10-09T06:36:29.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.645+0000 7f308bfff700 1 -- 172.21.15.163:0/154357643 wait complete. 2023-10-09T06:36:29.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.646+0000 7f308bfff700 1 Processor -- start 2023-10-09T06:36:29.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.646+0000 7f308bfff700 1 -- start start 2023-10-09T06:36:29.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.647+0000 7f308bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f308c06bf20 0x7f308c14a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:29.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.647+0000 7f308bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f308c0cc3c0 0x7f308c169450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:29.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.647+0000 7f308bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f308c1574d0 0x7f308c15bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:29.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.647+0000 7f308bfff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308c0b04a0 con 0x7f308c06bf20 2023-10-09T06:36:29.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.647+0000 7f308bfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308c15a3a0 con 0x7f308c0cc3c0 2023-10-09T06:36:29.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.647+0000 7f308bfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f308c0b01c0 con 0x7f308c1574d0 2023-10-09T06:36:29.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.647+0000 7f308affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f308c06bf20 0x7f308c14a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:29.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.647+0000 7f308affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f308c06bf20 0x7f308c14a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59586/0 (socket says 172.21.15.163:59586) 2023-10-09T06:36:29.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.647+0000 7f308affd700 1 -- 172.21.15.163:0/4281666661 learned_addr learned my addr 172.21.15.163:0/4281666661 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:29.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.647+0000 7f308affd700 1 -- 172.21.15.163:0/4281666661 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f308c1574d0 msgr2=0x7f308c15bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:29.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.647+0000 7f308affd700 1 --2- 172.21.15.163:0/4281666661 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f308c1574d0 0x7f308c15bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:29.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.647+0000 7f308affd700 1 -- 172.21.15.163:0/4281666661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f308c0cc3c0 msgr2=0x7f308c169450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:29.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.647+0000 7f308affd700 1 --2- 172.21.15.163:0/4281666661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f308c0cc3c0 0x7f308c169450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:29.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.647+0000 7f308affd700 1 -- 172.21.15.163:0/4281666661 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307c009480 con 0x7f308c06bf20 2023-10-09T06:36:29.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.648+0000 7f308affd700 1 --2- 172.21.15.163:0/4281666661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f308c06bf20 0x7f308c14a340 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f308c06edf0 tx=0x7f307c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:29.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.648+0000 7f3072ffd700 1 -- 172.21.15.163:0/4281666661 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c006b60 con 0x7f308c06bf20 2023-10-09T06:36:29.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.648+0000 7f3072ffd700 1 -- 172.21.15.163:0/4281666661 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307c011af0 con 0x7f308c06bf20 2023-10-09T06:36:29.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.648+0000 7f308bfff700 1 -- 172.21.15.163:0/4281666661 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f308c159d50 con 0x7f308c06bf20 2023-10-09T06:36:29.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.648+0000 7f3072ffd700 1 -- 172.21.15.163:0/4281666661 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c01c410 con 0x7f308c06bf20 2023-10-09T06:36:29.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.648+0000 7f308bfff700 1 -- 172.21.15.163:0/4281666661 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f308c15c4d0 con 0x7f308c06bf20 2023-10-09T06:36:29.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.649+0000 7f308bfff700 1 -- 172.21.15.163:0/4281666661 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f308c06b9e0 con 0x7f308c06bf20 2023-10-09T06:36:29.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.649+0000 7f3072ffd700 1 -- 172.21.15.163:0/4281666661 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f307c01c800 con 0x7f308c06bf20 2023-10-09T06:36:29.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.650+0000 7f3072ffd700 1 --2- 172.21.15.163:0/4281666661 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3078067200 0x7f30780696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:29.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.650+0000 7f3072ffd700 1 -- 172.21.15.163:0/4281666661 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f307c086330 con 0x7f308c06bf20 2023-10-09T06:36:29.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.650+0000 7f308a7fc700 1 --2- 172.21.15.163:0/4281666661 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3078067200 0x7f30780696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:29.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.650+0000 7f308a7fc700 1 --2- 172.21.15.163:0/4281666661 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3078067200 0x7f30780696b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3080006fd0 tx=0x7f3080008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:29.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.654+0000 7f3072ffd700 1 -- 172.21.15.163:0/4281666661 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f307c053290 con 0x7f308c06bf20 2023-10-09T06:36:29.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.840+0000 7f308bfff700 1 -- 172.21.15.163:0/4281666661 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f308c04c3c0 con 0x7f308c06bf20 2023-10-09T06:36:29.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.841+0000 7f3072ffd700 1 -- 172.21.15.163:0/4281666661 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+15401 (secure 0 0 0) 0x7f307c0568b0 con 0x7f308c06bf20 2023-10-09T06:36:29.842 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:29.842 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":89,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:36:19.522776+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:36:16.497125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-09T06:36:17.121589+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:36:29.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.843+0000 7f308bfff700 1 -- 172.21.15.163:0/4281666661 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3078067200 msgr2=0x7f30780696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:29.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.843+0000 7f308bfff700 1 --2- 172.21.15.163:0/4281666661 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3078067200 0x7f30780696b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3080006fd0 tx=0x7f3080008040 comp rx=0 tx=0).stop 2023-10-09T06:36:29.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.843+0000 7f308bfff700 1 -- 172.21.15.163:0/4281666661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f308c06bf20 msgr2=0x7f308c14a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:29.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.843+0000 7f308bfff700 1 --2- 172.21.15.163:0/4281666661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f308c06bf20 0x7f308c14a340 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f308c06edf0 tx=0x7f307c016000 comp rx=0 tx=0).stop 2023-10-09T06:36:29.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.843+0000 7f308bfff700 1 -- 172.21.15.163:0/4281666661 shutdown_connections 2023-10-09T06:36:29.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.843+0000 7f308bfff700 1 --2- 172.21.15.163:0/4281666661 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3078067200 0x7f30780696b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:29.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.843+0000 7f308bfff700 1 --2- 172.21.15.163:0/4281666661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f308c06bf20 0x7f308c14a340 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:29.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.843+0000 7f308bfff700 1 --2- 172.21.15.163:0/4281666661 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f308c1574d0 0x7f308c15bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:29.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.843+0000 7f308bfff700 1 --2- 172.21.15.163:0/4281666661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f308c0cc3c0 0x7f308c169450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:29.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.843+0000 7f308bfff700 1 -- 172.21.15.163:0/4281666661 >> 172.21.15.163:0/4281666661 conn(0x7f308c0ce430 msgr2=0x7f308c14cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:29.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.843+0000 7f308bfff700 1 -- 172.21.15.163:0/4281666661 shutdown_connections 2023-10-09T06:36:29.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.843+0000 7f308bfff700 1 -- 172.21.15.163:0/4281666661 wait complete. 2023-10-09T06:36:29.856 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-09T06:36:16.497125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-10-09T06:36:17.121589+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-09T06:36:29.857 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:36:29.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.973+0000 7f446d16c700 1 Processor -- start 2023-10-09T06:36:29.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.973+0000 7f446d16c700 1 -- start start 2023-10-09T06:36:29.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.973+0000 7f446d16c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4468069560 0x7f44680699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:29.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.973+0000 7f446d16c700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4468069ee0 con 0x7f4468159890 2023-10-09T06:36:29.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.973+0000 7f446d16c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f446815a160 con 0x7f4468159c60 2023-10-09T06:36:29.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.973+0000 7f446d16c700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44680d13a0 con 0x7f4468069560 2023-10-09T06:36:29.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.974+0000 7f446759e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4468069560 0x7f44680699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:29.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.974+0000 7f4466d9d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f4468159890 0x7f446815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53458/0 (socket says 172.21.15.163:53458) 2023-10-09T06:36:29.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.974+0000 7f446759e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4468069560 0x7f44680699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59600/0 (socket says 172.21.15.163:59600) 2023-10-09T06:36:29.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.974+0000 7f4466d9d700 1 -- 172.21.15.163:0/2070776621 learned_addr learned my addr 172.21.15.163:0/2070776621 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:29.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.974+0000 7f446759e700 1 -- 172.21.15.163:0/2070776621 >> v1:172.21.15.181:6789/0 conn(0x7f4468159c60 legacy=0x7f446815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:29.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.974+0000 7f446759e700 1 -- 172.21.15.163:0/2070776621 >> v1:172.21.15.163:6789/0 conn(0x7f4468159890 legacy=0x7f446815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:29.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.974+0000 7f446759e700 1 -- 172.21.15.163:0/2070776621 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446815e140 con 0x7f4468069560 2023-10-09T06:36:29.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.974+0000 7f446659c700 1 -- 172.21.15.163:0/2070776621 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1706954104 0 0) 0x7f4468069ee0 con 0x7f4468159890 2023-10-09T06:36:29.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.974+0000 7f446759e700 1 --2- 172.21.15.163:0/2070776621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4468069560 0x7f44680699a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f4460009e60 tx=0x7f4460009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c80bf479f1981b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:29.977 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.974+0000 7f446659c700 1 -- 172.21.15.163:0/2070776621 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446000e070 con 0x7f4468069560 2023-10-09T06:36:29.977 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.974+0000 7f446659c700 1 -- 172.21.15.163:0/2070776621 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446000bbf0 con 0x7f4468069560 2023-10-09T06:36:29.977 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.974+0000 7f446659c700 1 -- 172.21.15.163:0/2070776621 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4460012430 con 0x7f4468069560 2023-10-09T06:36:29.977 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.975+0000 7f446d16c700 1 -- 172.21.15.163:0/2070776621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4468069560 msgr2=0x7f44680699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:29.977 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.975+0000 7f446d16c700 1 --2- 172.21.15.163:0/2070776621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4468069560 0x7f44680699a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f4460009e60 tx=0x7f4460009780 comp rx=0 tx=0).stop 2023-10-09T06:36:29.977 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.975+0000 7f446d16c700 1 -- 172.21.15.163:0/2070776621 shutdown_connections 2023-10-09T06:36:29.977 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.975+0000 7f446d16c700 1 --2- 172.21.15.163:0/2070776621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4468069560 0x7f44680699a0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:29.977 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.975+0000 7f446d16c700 1 -- 172.21.15.163:0/2070776621 >> 172.21.15.163:0/2070776621 conn(0x7f4468068e20 msgr2=0x7f44680d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:29.977 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.975+0000 7f446d16c700 1 -- 172.21.15.163:0/2070776621 shutdown_connections 2023-10-09T06:36:29.977 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.975+0000 7f446d16c700 1 -- 172.21.15.163:0/2070776621 wait complete. 2023-10-09T06:36:29.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.976+0000 7f446d16c700 1 Processor -- start 2023-10-09T06:36:29.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.976+0000 7f446d16c700 1 -- start start 2023-10-09T06:36:29.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.977+0000 7f446d16c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4468069560 0x7f446815ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:29.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.977+0000 7f446d16c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4468159890 0x7f446814b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:29.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.977+0000 7f446d16c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4468159c60 0x7f4468157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:29.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.977+0000 7f446d16c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44680cf010 con 0x7f4468159890 2023-10-09T06:36:29.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.977+0000 7f446d16c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446815d3a0 con 0x7f4468069560 2023-10-09T06:36:29.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.977+0000 7f446d16c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44680ce400 con 0x7f4468159c60 2023-10-09T06:36:29.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.977+0000 7f445e59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4468159890 0x7f446814b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:29.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.977+0000 7f445e59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4468159890 0x7f446814b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59614/0 (socket says 172.21.15.163:59614) 2023-10-09T06:36:29.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.977+0000 7f445e59c700 1 -- 172.21.15.163:0/2878066591 learned_addr learned my addr 172.21.15.163:0/2878066591 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:29.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.977+0000 7f4466d9d700 1 --2- 172.21.15.163:0/2878066591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4468069560 0x7f446815ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:29.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.977+0000 7f446759e700 1 --2- 172.21.15.163:0/2878066591 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4468159c60 0x7f4468157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:29.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.977+0000 7f445e59c700 1 -- 172.21.15.163:0/2878066591 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4468159c60 msgr2=0x7f4468157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:29.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.977+0000 7f445e59c700 1 --2- 172.21.15.163:0/2878066591 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4468159c60 0x7f4468157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:29.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.977+0000 7f445e59c700 1 -- 172.21.15.163:0/2878066591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4468069560 msgr2=0x7f446815ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:29.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.977+0000 7f445e59c700 1 --2- 172.21.15.163:0/2878066591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4468069560 0x7f446815ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:29.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.977+0000 7f445e59c700 1 -- 172.21.15.163:0/2878066591 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4460009a20 con 0x7f4468159890 2023-10-09T06:36:29.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.977+0000 7f445e59c700 1 --2- 172.21.15.163:0/2878066591 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4468159890 0x7f446814b000 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f445000b5e0 tx=0x7f4450009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:29.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.977+0000 7f445f7fe700 1 -- 172.21.15.163:0/2878066591 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445000f510 con 0x7f4468159890 2023-10-09T06:36:29.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.977+0000 7f445f7fe700 1 -- 172.21.15.163:0/2878066591 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445000f670 con 0x7f4468159890 2023-10-09T06:36:29.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.977+0000 7f446d16c700 1 -- 172.21.15.163:0/2878066591 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44680ce6e0 con 0x7f4468159890 2023-10-09T06:36:29.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.978+0000 7f445f7fe700 1 -- 172.21.15.163:0/2878066591 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445000fa30 con 0x7f4468159890 2023-10-09T06:36:29.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.978+0000 7f446d16c700 1 -- 172.21.15.163:0/2878066591 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44680ceca0 con 0x7f4468159890 2023-10-09T06:36:29.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.978+0000 7f446d16c700 1 -- 172.21.15.163:0/2878066591 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44680cf180 con 0x7f4468159890 2023-10-09T06:36:29.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.979+0000 7f445f7fe700 1 -- 172.21.15.163:0/2878066591 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f445001a400 con 0x7f4468159890 2023-10-09T06:36:29.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.979+0000 7f445f7fe700 1 --2- 172.21.15.163:0/2878066591 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4448067200 0x7f44480696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:29.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.979+0000 7f445f7fe700 1 -- 172.21.15.163:0/2878066591 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4450084320 con 0x7f4468159890 2023-10-09T06:36:29.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.980+0000 7f4466d9d700 1 --2- 172.21.15.163:0/2878066591 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4448067200 0x7f44480696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:29.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.980+0000 7f4466d9d700 1 --2- 172.21.15.163:0/2878066591 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4448067200 0x7f44480696b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4458007fd0 tx=0x7f4458007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:29.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:29.983+0000 7f445f7fe700 1 -- 172.21.15.163:0/2878066591 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f44500512c0 con 0x7f4468159890 2023-10-09T06:36:30.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.164+0000 7f446d16c700 1 -- 172.21.15.163:0/2878066591 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f446804c3c0 con 0x7f4468159890 2023-10-09T06:36:30.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.164+0000 7f445f7fe700 1 -- 172.21.15.163:0/2878066591 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f44500548e0 con 0x7f4468159890 2023-10-09T06:36:30.165 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:36:30.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.166+0000 7f446d16c700 1 -- 172.21.15.163:0/2878066591 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4448067200 msgr2=0x7f44480696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:30.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.166+0000 7f446d16c700 1 --2- 172.21.15.163:0/2878066591 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4448067200 0x7f44480696b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4458007fd0 tx=0x7f4458007d30 comp rx=0 tx=0).stop 2023-10-09T06:36:30.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.166+0000 7f446d16c700 1 -- 172.21.15.163:0/2878066591 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4468159890 msgr2=0x7f446814b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:30.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.166+0000 7f446d16c700 1 --2- 172.21.15.163:0/2878066591 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4468159890 0x7f446814b000 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f445000b5e0 tx=0x7f4450009120 comp rx=0 tx=0).stop 2023-10-09T06:36:30.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.166+0000 7f446d16c700 1 -- 172.21.15.163:0/2878066591 shutdown_connections 2023-10-09T06:36:30.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.166+0000 7f446d16c700 1 --2- 172.21.15.163:0/2878066591 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4448067200 0x7f44480696b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:30.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.166+0000 7f446d16c700 1 --2- 172.21.15.163:0/2878066591 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4468159c60 0x7f4468157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:30.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.166+0000 7f446d16c700 1 --2- 172.21.15.163:0/2878066591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4468069560 0x7f446815ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:30.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.166+0000 7f446d16c700 1 --2- 172.21.15.163:0/2878066591 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4468159890 0x7f446814b000 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:30.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.166+0000 7f446d16c700 1 -- 172.21.15.163:0/2878066591 >> 172.21.15.163:0/2878066591 conn(0x7f4468068e20 msgr2=0x7f44680c6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:30.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.166+0000 7f446d16c700 1 -- 172.21.15.163:0/2878066591 shutdown_connections 2023-10-09T06:36:30.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.166+0000 7f446d16c700 1 -- 172.21.15.163:0/2878066591 wait complete. 2023-10-09T06:36:30.179 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-09T06:36:30.298 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.297+0000 7f5ec8105700 1 Processor -- start 2023-10-09T06:36:30.298 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.297+0000 7f5ec8105700 1 -- start start 2023-10-09T06:36:30.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.298+0000 7f5ec8105700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5ec00b5ea0 0x7f5ec0068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:30.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.298+0000 7f5ec8105700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ec0068bb0 0x7f5ec015c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:30.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.298+0000 7f5ec8105700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ec015c480 con 0x7f5ec0159c10 2023-10-09T06:36:30.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.298+0000 7f5ec8105700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ec00b2a40 con 0x7f5ec00b5ea0 2023-10-09T06:36:30.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.298+0000 7f5ec8105700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec015b870 con 0x7f5ec0068bb0 2023-10-09T06:36:30.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.298+0000 7f5ec56a0700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5ec00b5ea0 0x7f5ec0068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:30.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.298+0000 7f5ec5ea1700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f5ec0159c10 0x7f5ec015b090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53474/0 (socket says 172.21.15.163:53474) 2023-10-09T06:36:30.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.298+0000 7f5ec5ea1700 1 -- 172.21.15.163:0/3314213086 learned_addr learned my addr 172.21.15.163:0/3314213086 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:30.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.298+0000 7f5ec66a2700 1 --2- 172.21.15.163:0/3314213086 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ec0068bb0 0x7f5ec015c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:30.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.298+0000 7f5ec56a0700 1 -- 172.21.15.163:0/3314213086 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ec0068bb0 msgr2=0x7f5ec015c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:30.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.298+0000 7f5ec56a0700 1 --2- 172.21.15.163:0/3314213086 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ec0068bb0 0x7f5ec015c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:30.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.298+0000 7f5ec56a0700 1 -- 172.21.15.163:0/3314213086 >> v1:172.21.15.163:6789/0 conn(0x7f5ec0159c10 legacy=0x7f5ec015b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:30.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.298+0000 7f5ec4e9f700 1 -- 172.21.15.163:0/3314213086 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 708558242 0 0) 0x7f5ec015c480 con 0x7f5ec0159c10 2023-10-09T06:36:30.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.298+0000 7f5ec56a0700 1 -- 172.21.15.163:0/3314213086 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec015bb50 con 0x7f5ec00b5ea0 2023-10-09T06:36:30.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.298+0000 7f5ec56a0700 1 --2- 172.21.15.163:0/3314213086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ec00b5ea0 0x7f5ec0068670 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00098e0 tx=0x7f5eb00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f0c129a2741e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:30.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.299+0000 7f5ec4e9f700 1 -- 172.21.15.163:0/3314213086 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0009e80 con 0x7f5ec00b5ea0 2023-10-09T06:36:30.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.299+0000 7f5ec4e9f700 1 -- 172.21.15.163:0/3314213086 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb000b860 con 0x7f5ec00b5ea0 2023-10-09T06:36:30.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.299+0000 7f5ec4e9f700 1 -- 172.21.15.163:0/3314213086 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb000bbe0 con 0x7f5ec00b5ea0 2023-10-09T06:36:30.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.299+0000 7f5ec8105700 1 -- 172.21.15.163:0/3314213086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ec00b5ea0 msgr2=0x7f5ec0068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:30.303 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.299+0000 7f5ec8105700 1 --2- 172.21.15.163:0/3314213086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ec00b5ea0 0x7f5ec0068670 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00098e0 tx=0x7f5eb00091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:30.303 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.299+0000 7f5ec8105700 1 -- 172.21.15.163:0/3314213086 shutdown_connections 2023-10-09T06:36:30.303 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.299+0000 7f5ec8105700 1 --2- 172.21.15.163:0/3314213086 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ec0068bb0 0x7f5ec015c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:30.303 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.299+0000 7f5ec8105700 1 --2- 172.21.15.163:0/3314213086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ec00b5ea0 0x7f5ec0068670 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:30.303 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.299+0000 7f5ec8105700 1 -- 172.21.15.163:0/3314213086 >> 172.21.15.163:0/3314213086 conn(0x7f5ec0099a20 msgr2=0x7f5ec00af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:30.303 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.299+0000 7f5ec8105700 1 -- 172.21.15.163:0/3314213086 shutdown_connections 2023-10-09T06:36:30.303 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.299+0000 7f5ec8105700 1 -- 172.21.15.163:0/3314213086 wait complete. 2023-10-09T06:36:30.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.300+0000 7f5ec8105700 1 Processor -- start 2023-10-09T06:36:30.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.300+0000 7f5ec8105700 1 -- start start 2023-10-09T06:36:30.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.301+0000 7f5ec8105700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ec0068bb0 0x7f5ec01486e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:30.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.301+0000 7f5ec8105700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ec00b5ea0 0x7f5ec006a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:30.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.301+0000 7f5ec8105700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ec0159c10 0x7f5ec006a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:30.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.301+0000 7f5ec8105700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec014bea0 con 0x7f5ec0159c10 2023-10-09T06:36:30.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.301+0000 7f5ec8105700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec014b290 con 0x7f5ec00b5ea0 2023-10-09T06:36:30.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.301+0000 7f5ec8105700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec014b570 con 0x7f5ec0068bb0 2023-10-09T06:36:30.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.301+0000 7f5ec66a2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ec0159c10 0x7f5ec006a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:30.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.301+0000 7f5ec66a2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ec0159c10 0x7f5ec006a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59618/0 (socket says 172.21.15.163:59618) 2023-10-09T06:36:30.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.301+0000 7f5ec66a2700 1 -- 172.21.15.163:0/2392236972 learned_addr learned my addr 172.21.15.163:0/2392236972 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:30.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.301+0000 7f5ec56a0700 1 --2- 172.21.15.163:0/2392236972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ec00b5ea0 0x7f5ec006a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:30.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.301+0000 7f5ec5ea1700 1 --2- 172.21.15.163:0/2392236972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ec0068bb0 0x7f5ec01486e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:30.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.301+0000 7f5ec66a2700 1 -- 172.21.15.163:0/2392236972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ec0068bb0 msgr2=0x7f5ec01486e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:30.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.301+0000 7f5ec66a2700 1 --2- 172.21.15.163:0/2392236972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ec0068bb0 0x7f5ec01486e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:30.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.301+0000 7f5ec66a2700 1 -- 172.21.15.163:0/2392236972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ec00b5ea0 msgr2=0x7f5ec006a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:30.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.301+0000 7f5ec66a2700 1 --2- 172.21.15.163:0/2392236972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ec00b5ea0 0x7f5ec006a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:30.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.301+0000 7f5ec66a2700 1 -- 172.21.15.163:0/2392236972 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb0009420 con 0x7f5ec0159c10 2023-10-09T06:36:30.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.302+0000 7f5ec66a2700 1 --2- 172.21.15.163:0/2392236972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ec0159c10 0x7f5ec006a600 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800ceb0 tx=0x7f5eb8007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:30.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.302+0000 7f5eb5ffb700 1 -- 172.21.15.163:0/2392236972 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb800ae40 con 0x7f5ec0159c10 2023-10-09T06:36:30.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.302+0000 7f5eb5ffb700 1 -- 172.21.15.163:0/2392236972 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb8016430 con 0x7f5ec0159c10 2023-10-09T06:36:30.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.302+0000 7f5ec8105700 1 -- 172.21.15.163:0/2392236972 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec014b850 con 0x7f5ec0159c10 2023-10-09T06:36:30.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.302+0000 7f5eb5ffb700 1 -- 172.21.15.163:0/2392236972 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb80167d0 con 0x7f5ec0159c10 2023-10-09T06:36:30.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.302+0000 7f5ec8105700 1 -- 172.21.15.163:0/2392236972 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec0148c20 con 0x7f5ec0159c10 2023-10-09T06:36:30.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.303+0000 7f5ec8105700 1 -- 172.21.15.163:0/2392236972 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec004c3c0 con 0x7f5ec0159c10 2023-10-09T06:36:30.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.303+0000 7f5eb5ffb700 1 -- 172.21.15.163:0/2392236972 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f5eb8003020 con 0x7f5ec0159c10 2023-10-09T06:36:30.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.304+0000 7f5eb5ffb700 1 --2- 172.21.15.163:0/2392236972 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5ea4067200 0x7f5ea40696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:30.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.304+0000 7f5eb5ffb700 1 -- 172.21.15.163:0/2392236972 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5eb8012070 con 0x7f5ec0159c10 2023-10-09T06:36:30.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.304+0000 7f5ec5ea1700 1 --2- 172.21.15.163:0/2392236972 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5ea4067200 0x7f5ea40696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:30.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.304+0000 7f5ec5ea1700 1 --2- 172.21.15.163:0/2392236972 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5ea4067200 0x7f5ea40696b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00c8f0 tx=0x7f5ebc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:30.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.308+0000 7f5eb5ffb700 1 -- 172.21.15.163:0/2392236972 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5eb8052850 con 0x7f5ec0159c10 2023-10-09T06:36:30.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.489+0000 7f5ec8105700 1 -- 172.21.15.163:0/2392236972 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5ec014bfe0 con 0x7f5ec0159c10 2023-10-09T06:36:30.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.490+0000 7f5eb5ffb700 1 -- 172.21.15.163:0/2392236972 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5eb8055e70 con 0x7f5ec0159c10 2023-10-09T06:36:30.491 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 32 2023-10-09T06:36:30.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.492+0000 7f5ec8105700 1 -- 172.21.15.163:0/2392236972 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5ea4067200 msgr2=0x7f5ea40696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:30.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.492+0000 7f5ec8105700 1 --2- 172.21.15.163:0/2392236972 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5ea4067200 0x7f5ea40696b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00c8f0 tx=0x7f5ebc009040 comp rx=0 tx=0).stop 2023-10-09T06:36:30.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.492+0000 7f5ec8105700 1 -- 172.21.15.163:0/2392236972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ec0159c10 msgr2=0x7f5ec006a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:30.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.492+0000 7f5ec8105700 1 --2- 172.21.15.163:0/2392236972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ec0159c10 0x7f5ec006a600 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800ceb0 tx=0x7f5eb8007870 comp rx=0 tx=0).stop 2023-10-09T06:36:30.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.492+0000 7f5ec8105700 1 -- 172.21.15.163:0/2392236972 shutdown_connections 2023-10-09T06:36:30.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.492+0000 7f5ec8105700 1 --2- 172.21.15.163:0/2392236972 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5ea4067200 0x7f5ea40696b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:30.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.492+0000 7f5ec8105700 1 --2- 172.21.15.163:0/2392236972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ec0159c10 0x7f5ec006a600 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:30.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.492+0000 7f5ec8105700 1 --2- 172.21.15.163:0/2392236972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ec0068bb0 0x7f5ec01486e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:30.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.492+0000 7f5ec8105700 1 --2- 172.21.15.163:0/2392236972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ec00b5ea0 0x7f5ec006a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:30.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.492+0000 7f5ec8105700 1 -- 172.21.15.163:0/2392236972 >> 172.21.15.163:0/2392236972 conn(0x7f5ec0099a20 msgr2=0x7f5ec00b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:30.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.492+0000 7f5ec8105700 1 -- 172.21.15.163:0/2392236972 shutdown_connections 2023-10-09T06:36:30.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.492+0000 7f5ec8105700 1 -- 172.21.15.163:0/2392236972 wait complete. 2023-10-09T06:36:30.504 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-09T06:36:30.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.622+0000 7fa688d7c700 1 Processor -- start 2023-10-09T06:36:30.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.622+0000 7fa688d7c700 1 -- start start 2023-10-09T06:36:30.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.623+0000 7fa688d7c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa684159c60 0x7fa68415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:30.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.623+0000 7fa688d7c700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa68415a160 con 0x7fa68415db10 2023-10-09T06:36:30.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.623+0000 7fa688d7c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6840d13a0 con 0x7fa684159890 2023-10-09T06:36:30.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.623+0000 7fa688d7c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa68415df50 con 0x7fa684159c60 2023-10-09T06:36:30.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.623+0000 7fa682d9d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fa68415db10 0x7fa684069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53490/0 (socket says 172.21.15.163:53490) 2023-10-09T06:36:30.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.623+0000 7fa682d9d700 1 -- 172.21.15.163:0/3095631233 learned_addr learned my addr 172.21.15.163:0/3095631233 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:30.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.623+0000 7fa681d9b700 1 --2- 172.21.15.163:0/3095631233 >> v2:172.21.15.181:3300/0 conn(0x7fa684159c60 0x7fa68415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:30.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.624+0000 7fa68159a700 1 -- 172.21.15.163:0/3095631233 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1152657921 0 0) 0x7fa68415a160 con 0x7fa68415db10 2023-10-09T06:36:30.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.624+0000 7fa68159a700 1 -- 172.21.15.163:0/3095631233 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa66c003570 con 0x7fa68415db10 2023-10-09T06:36:30.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.624+0000 7fa681d9b700 1 -- 172.21.15.163:0/3095631233 >> v1:172.21.15.181:6789/0 conn(0x7fa684159890 legacy=0x7fa68415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:30.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.624+0000 7fa68159a700 1 -- 172.21.15.163:0/3095631233 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3091883898 0 0) 0x7fa6840d13a0 con 0x7fa684159890 2023-10-09T06:36:30.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.624+0000 7fa681d9b700 1 -- 172.21.15.163:0/3095631233 >> v1:172.21.15.163:6789/0 conn(0x7fa68415db10 legacy=0x7fa684069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:30.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.624+0000 7fa681d9b700 1 -- 172.21.15.163:0/3095631233 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa68415e140 con 0x7fa684159c60 2023-10-09T06:36:30.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.624+0000 7fa681d9b700 1 --2- 172.21.15.163:0/3095631233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa684159c60 0x7fa68415d5d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa6780098e0 tx=0x7fa6780091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3106d545ab1cec2d server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:30.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.624+0000 7fa68159a700 1 -- 172.21.15.163:0/3095631233 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67c001470 con 0x7fa684159c60 2023-10-09T06:36:30.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.624+0000 7fa68159a700 1 -- 172.21.15.163:0/3095631233 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa678011420 con 0x7fa684159c60 2023-10-09T06:36:30.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.624+0000 7fa68159a700 1 -- 172.21.15.163:0/3095631233 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6780117c0 con 0x7fa684159c60 2023-10-09T06:36:30.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.625+0000 7fa688d7c700 1 -- 172.21.15.163:0/3095631233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa684159c60 msgr2=0x7fa68415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:30.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.625+0000 7fa688d7c700 1 --2- 172.21.15.163:0/3095631233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa684159c60 0x7fa68415d5d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa6780098e0 tx=0x7fa6780091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:30.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.625+0000 7fa688d7c700 1 -- 172.21.15.163:0/3095631233 shutdown_connections 2023-10-09T06:36:30.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.625+0000 7fa688d7c700 1 --2- 172.21.15.163:0/3095631233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa684159c60 0x7fa68415d5d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:30.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.625+0000 7fa688d7c700 1 -- 172.21.15.163:0/3095631233 >> 172.21.15.163:0/3095631233 conn(0x7fa684068e20 msgr2=0x7fa6840d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:30.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.625+0000 7fa688d7c700 1 -- 172.21.15.163:0/3095631233 shutdown_connections 2023-10-09T06:36:30.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.625+0000 7fa688d7c700 1 -- 172.21.15.163:0/3095631233 wait complete. 2023-10-09T06:36:30.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.626+0000 7fa688d7c700 1 Processor -- start 2023-10-09T06:36:30.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.626+0000 7fa688d7c700 1 -- start start 2023-10-09T06:36:30.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.627+0000 7fa688d7c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa684159890 0x7fa684148300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:30.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.627+0000 7fa688d7c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa684159c60 0x7fa68416cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:30.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.627+0000 7fa688d7c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa68415db10 0x7fa684158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:30.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.627+0000 7fa688d7c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa684148840 con 0x7fa684159890 2023-10-09T06:36:30.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.627+0000 7fa688d7c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6840b0a90 con 0x7fa684159c60 2023-10-09T06:36:30.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.627+0000 7fa688d7c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6840b1050 con 0x7fa68415db10 2023-10-09T06:36:30.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.627+0000 7fa68259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa684159890 0x7fa684148300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:30.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.627+0000 7fa68259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa684159890 0x7fa684148300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59622/0 (socket says 172.21.15.163:59622) 2023-10-09T06:36:30.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.627+0000 7fa68259c700 1 -- 172.21.15.163:0/3122302553 learned_addr learned my addr 172.21.15.163:0/3122302553 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:30.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.627+0000 7fa682d9d700 1 --2- 172.21.15.163:0/3122302553 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa68415db10 0x7fa684158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:30.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.627+0000 7fa681d9b700 1 --2- 172.21.15.163:0/3122302553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa684159c60 0x7fa68416cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:30.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.627+0000 7fa68259c700 1 -- 172.21.15.163:0/3122302553 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa68415db10 msgr2=0x7fa684158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:30.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.627+0000 7fa68259c700 1 --2- 172.21.15.163:0/3122302553 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa68415db10 0x7fa684158390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:30.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.627+0000 7fa68259c700 1 -- 172.21.15.163:0/3122302553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa684159c60 msgr2=0x7fa68416cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:30.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.627+0000 7fa68259c700 1 --2- 172.21.15.163:0/3122302553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa684159c60 0x7fa68416cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:30.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.627+0000 7fa68259c700 1 -- 172.21.15.163:0/3122302553 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa678009420 con 0x7fa684159890 2023-10-09T06:36:30.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.627+0000 7fa68259c700 1 --2- 172.21.15.163:0/3122302553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa684159890 0x7fa684148300 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fa67400cb90 tx=0x7fa6740017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:30.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.627+0000 7fa6727fc700 1 -- 172.21.15.163:0/3122302553 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6740020b0 con 0x7fa684159890 2023-10-09T06:36:30.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.627+0000 7fa6727fc700 1 -- 172.21.15.163:0/3122302553 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67400a7a0 con 0x7fa684159890 2023-10-09T06:36:30.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.627+0000 7fa688d7c700 1 -- 172.21.15.163:0/3122302553 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa68416d9e0 con 0x7fa684159890 2023-10-09T06:36:30.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.627+0000 7fa6727fc700 1 -- 172.21.15.163:0/3122302553 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67400ab60 con 0x7fa684159890 2023-10-09T06:36:30.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.627+0000 7fa688d7c700 1 -- 172.21.15.163:0/3122302553 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6840b0ce0 con 0x7fa684159890 2023-10-09T06:36:30.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.628+0000 7fa688d7c700 1 -- 172.21.15.163:0/3122302553 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa68404c3c0 con 0x7fa684159890 2023-10-09T06:36:30.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.629+0000 7fa6727fc700 1 -- 172.21.15.163:0/3122302553 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fa674003030 con 0x7fa684159890 2023-10-09T06:36:30.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.629+0000 7fa6727fc700 1 --2- 172.21.15.163:0/3122302553 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa664067200 0x7fa6640696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:30.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.629+0000 7fa6727fc700 1 -- 172.21.15.163:0/3122302553 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa6740849e0 con 0x7fa684159890 2023-10-09T06:36:30.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.629+0000 7fa681d9b700 1 --2- 172.21.15.163:0/3122302553 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa664067200 0x7fa6640696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:30.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.630+0000 7fa681d9b700 1 --2- 172.21.15.163:0/3122302553 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa664067200 0x7fa6640696b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa6780030c0 tx=0x7fa678017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:30.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.633+0000 7fa6727fc700 1 -- 172.21.15.163:0/3122302553 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa674051900 con 0x7fa684159890 2023-10-09T06:36:30.814 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.813+0000 7fa688d7c700 1 -- 172.21.15.163:0/3122302553 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa6840b0ea0 con 0x7fa684159890 2023-10-09T06:36:30.815 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.814+0000 7fa6727fc700 1 -- 172.21.15.163:0/3122302553 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa674054f20 con 0x7fa684159890 2023-10-09T06:36:30.815 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 64 2023-10-09T06:36:30.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.816+0000 7fa688d7c700 1 -- 172.21.15.163:0/3122302553 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa664067200 msgr2=0x7fa6640696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:30.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.816+0000 7fa688d7c700 1 --2- 172.21.15.163:0/3122302553 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa664067200 0x7fa6640696b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa6780030c0 tx=0x7fa678017000 comp rx=0 tx=0).stop 2023-10-09T06:36:30.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.816+0000 7fa688d7c700 1 -- 172.21.15.163:0/3122302553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa684159890 msgr2=0x7fa684148300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:30.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.816+0000 7fa688d7c700 1 --2- 172.21.15.163:0/3122302553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa684159890 0x7fa684148300 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fa67400cb90 tx=0x7fa6740017a0 comp rx=0 tx=0).stop 2023-10-09T06:36:30.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.816+0000 7fa688d7c700 1 -- 172.21.15.163:0/3122302553 shutdown_connections 2023-10-09T06:36:30.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.816+0000 7fa688d7c700 1 --2- 172.21.15.163:0/3122302553 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa664067200 0x7fa6640696b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:30.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.816+0000 7fa688d7c700 1 --2- 172.21.15.163:0/3122302553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa684159890 0x7fa684148300 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:30.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.816+0000 7fa688d7c700 1 --2- 172.21.15.163:0/3122302553 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa68415db10 0x7fa684158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:30.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.816+0000 7fa688d7c700 1 --2- 172.21.15.163:0/3122302553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa684159c60 0x7fa68416cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:30.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.816+0000 7fa688d7c700 1 -- 172.21.15.163:0/3122302553 >> 172.21.15.163:0/3122302553 conn(0x7fa684068e20 msgr2=0x7fa684156e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:30.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.816+0000 7fa688d7c700 1 -- 172.21.15.163:0/3122302553 shutdown_connections 2023-10-09T06:36:30.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.816+0000 7fa688d7c700 1 -- 172.21.15.163:0/3122302553 wait complete. 2023-10-09T06:36:30.829 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:36:30.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.943+0000 7fa3609f8700 1 Processor -- start 2023-10-09T06:36:30.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.943+0000 7fa3609f8700 1 -- start start 2023-10-09T06:36:30.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.944+0000 7fa3609f8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa35c159890 0x7fa35c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:30.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.944+0000 7fa3609f8700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa35c15a160 con 0x7fa35c159c60 2023-10-09T06:36:30.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.944+0000 7fa3609f8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa35c0d13a0 con 0x7fa35c069560 2023-10-09T06:36:30.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.944+0000 7fa3609f8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa35c15e140 con 0x7fa35c159890 2023-10-09T06:36:30.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.944+0000 7fa351d9b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fa35c159c60 0x7fa35c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53506/0 (socket says 172.21.15.163:53506) 2023-10-09T06:36:30.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.944+0000 7fa351d9b700 1 -- 172.21.15.163:0/841627072 learned_addr learned my addr 172.21.15.163:0/841627072 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:30.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.944+0000 7fa35a59c700 1 --2- 172.21.15.163:0/841627072 >> v2:172.21.15.181:3300/0 conn(0x7fa35c159890 0x7fa35c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:30.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.944+0000 7fa359d9b700 1 -- 172.21.15.163:0/841627072 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2364775086 0 0) 0x7fa35c15a160 con 0x7fa35c159c60 2023-10-09T06:36:30.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.944+0000 7fa359d9b700 1 -- 172.21.15.163:0/841627072 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa348003570 con 0x7fa35c159c60 2023-10-09T06:36:30.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.945+0000 7fa359d9b700 1 -- 172.21.15.163:0/841627072 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2536803856 0 0) 0x7fa35c0d13a0 con 0x7fa35c069560 2023-10-09T06:36:30.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.945+0000 7fa359d9b700 1 -- 172.21.15.163:0/841627072 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa35c15a160 con 0x7fa35c069560 2023-10-09T06:36:30.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.945+0000 7fa359d9b700 1 -- 172.21.15.163:0/841627072 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 581593841 0 0) 0x7fa348003570 con 0x7fa35c159c60 2023-10-09T06:36:30.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.945+0000 7fa359d9b700 1 -- 172.21.15.163:0/841627072 >> v2:172.21.15.181:3300/0 conn(0x7fa35c159890 msgr2=0x7fa35c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:30.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.945+0000 7fa359d9b700 1 --2- 172.21.15.163:0/841627072 >> v2:172.21.15.181:3300/0 conn(0x7fa35c159890 0x7fa35c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:30.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.945+0000 7fa359d9b700 1 -- 172.21.15.163:0/841627072 >> v1:172.21.15.181:6789/0 conn(0x7fa35c069560 legacy=0x7fa35c0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:30.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.945+0000 7fa359d9b700 1 -- 172.21.15.163:0/841627072 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c15a3d0 con 0x7fa35c159c60 2023-10-09T06:36:30.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.945+0000 7fa359d9b700 1 -- 172.21.15.163:0/841627072 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fa344001470 con 0x7fa35c159c60 2023-10-09T06:36:30.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.945+0000 7fa359d9b700 1 -- 172.21.15.163:0/841627072 >> v1:172.21.15.163:6789/0 conn(0x7fa35c159c60 legacy=0x7fa35c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:30.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.945+0000 7fa359d9b700 1 --2- 172.21.15.163:0/841627072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa348003bf0 0x7fa348003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:30.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.945+0000 7fa359d9b700 1 --2- 172.21.15.163:0/841627072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa348004810 0x7fa348004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:30.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.945+0000 7fa359d9b700 1 --2- 172.21.15.163:0/841627072 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3480051c0 0x7fa34800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:30.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.945+0000 7fa359d9b700 1 -- 172.21.15.163:0/841627072 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c15e140 con 0x7fa348004810 2023-10-09T06:36:30.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.945+0000 7fa359d9b700 1 -- 172.21.15.163:0/841627072 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34800c0b0 con 0x7fa348003bf0 2023-10-09T06:36:30.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.945+0000 7fa359d9b700 1 -- 172.21.15.163:0/841627072 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa34800c270 con 0x7fa3480051c0 2023-10-09T06:36:30.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.945+0000 7fa351d9b700 1 --2- 172.21.15.163:0/841627072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa348004810 0x7fa348004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:30.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.945+0000 7fa35a59c700 1 --2- 172.21.15.163:0/841627072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa348003bf0 0x7fa348003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:30.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.945+0000 7fa35ad9d700 1 --2- 172.21.15.163:0/841627072 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3480051c0 0x7fa34800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:30.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.945+0000 7fa351d9b700 1 -- 172.21.15.163:0/841627072 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3480051c0 msgr2=0x7fa34800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:30.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.945+0000 7fa351d9b700 1 --2- 172.21.15.163:0/841627072 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3480051c0 0x7fa34800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:30.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.945+0000 7fa351d9b700 1 -- 172.21.15.163:0/841627072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa348003bf0 msgr2=0x7fa348003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:30.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.945+0000 7fa351d9b700 1 --2- 172.21.15.163:0/841627072 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa348003bf0 0x7fa348003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:30.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.945+0000 7fa35ad9d700 1 -- 172.21.15.163:0/841627072 reap_dead start 2023-10-09T06:36:30.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.945+0000 7fa351d9b700 1 -- 172.21.15.163:0/841627072 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa348003570 con 0x7fa348004810 2023-10-09T06:36:30.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.945+0000 7fa351d9b700 1 --2- 172.21.15.163:0/841627072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa348004810 0x7fa348004c80 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fa34400c2d0 tx=0x7fa344001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fe4a8e3daaeb6d server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:30.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.945+0000 7fa359d9b700 1 -- 172.21.15.163:0/841627072 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34400cba0 con 0x7fa348004810 2023-10-09T06:36:30.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.945+0000 7fa359d9b700 1 -- 172.21.15.163:0/841627072 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa34400cd20 con 0x7fa348004810 2023-10-09T06:36:30.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.945+0000 7fa359d9b700 1 -- 172.21.15.163:0/841627072 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa344005b70 con 0x7fa348004810 2023-10-09T06:36:30.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.946+0000 7fa3609f8700 1 -- 172.21.15.163:0/841627072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa348004810 msgr2=0x7fa348004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:30.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.946+0000 7fa3609f8700 1 --2- 172.21.15.163:0/841627072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa348004810 0x7fa348004c80 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fa34400c2d0 tx=0x7fa344001e10 comp rx=0 tx=0).stop 2023-10-09T06:36:30.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.946+0000 7fa3609f8700 1 -- 172.21.15.163:0/841627072 shutdown_connections 2023-10-09T06:36:30.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.946+0000 7fa3609f8700 1 --2- 172.21.15.163:0/841627072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa348004810 0x7fa348004c80 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:30.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.946+0000 7fa3609f8700 1 -- 172.21.15.163:0/841627072 >> 172.21.15.163:0/841627072 conn(0x7fa35c068e20 msgr2=0x7fa35c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:30.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.946+0000 7fa3609f8700 1 -- 172.21.15.163:0/841627072 shutdown_connections 2023-10-09T06:36:30.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.946+0000 7fa3609f8700 1 -- 172.21.15.163:0/841627072 wait complete. 2023-10-09T06:36:30.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.947+0000 7fa3609f8700 1 Processor -- start 2023-10-09T06:36:30.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.947+0000 7fa3609f8700 1 -- start start 2023-10-09T06:36:30.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.948+0000 7fa3609f8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa348004810 0x7fa35c149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:30.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.948+0000 7fa3609f8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa35c0b2140 0x7fa35c0b25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:30.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.948+0000 7fa3609f8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa35c0cf480 0x7fa35c0cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:30.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.948+0000 7fa3609f8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c156e00 con 0x7fa35c0b2140 2023-10-09T06:36:30.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.948+0000 7fa3609f8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c0b2af0 con 0x7fa348004810 2023-10-09T06:36:30.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.948+0000 7fa3609f8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c157dd0 con 0x7fa35c0cf480 2023-10-09T06:36:30.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.948+0000 7fa351d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa35c0b2140 0x7fa35c0b25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:30.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.948+0000 7fa351d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa35c0b2140 0x7fa35c0b25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59638/0 (socket says 172.21.15.163:59638) 2023-10-09T06:36:30.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.948+0000 7fa351d9b700 1 -- 172.21.15.163:0/276157427 learned_addr learned my addr 172.21.15.163:0/276157427 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:30.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.948+0000 7fa35ad9d700 1 --2- 172.21.15.163:0/276157427 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa35c0cf480 0x7fa35c0cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:30.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.948+0000 7fa35a59c700 1 --2- 172.21.15.163:0/276157427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa348004810 0x7fa35c149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:30.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.948+0000 7fa351d9b700 1 -- 172.21.15.163:0/276157427 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa35c0cf480 msgr2=0x7fa35c0cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:30.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.948+0000 7fa351d9b700 1 --2- 172.21.15.163:0/276157427 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa35c0cf480 0x7fa35c0cf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:30.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.948+0000 7fa351d9b700 1 -- 172.21.15.163:0/276157427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa348004810 msgr2=0x7fa35c149dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:30.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.948+0000 7fa351d9b700 1 --2- 172.21.15.163:0/276157427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa348004810 0x7fa35c149dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:30.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.948+0000 7fa351d9b700 1 -- 172.21.15.163:0/276157427 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c155da0 con 0x7fa35c0b2140 2023-10-09T06:36:30.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.948+0000 7fa35ad9d700 1 --2- 172.21.15.163:0/276157427 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa35c0cf480 0x7fa35c0cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:36:30.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.948+0000 7fa35a59c700 1 --2- 172.21.15.163:0/276157427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa348004810 0x7fa35c149dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:36:30.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.949+0000 7fa351d9b700 1 --2- 172.21.15.163:0/276157427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa35c0b2140 0x7fa35c0b25b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fa344001e50 tx=0x7fa344015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:30.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.949+0000 7fa352ffd700 1 -- 172.21.15.163:0/276157427 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3440070d0 con 0x7fa35c0b2140 2023-10-09T06:36:30.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.949+0000 7fa352ffd700 1 -- 172.21.15.163:0/276157427 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa344007250 con 0x7fa35c0b2140 2023-10-09T06:36:30.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.949+0000 7fa3609f8700 1 -- 172.21.15.163:0/276157427 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa35c156040 con 0x7fa35c0b2140 2023-10-09T06:36:30.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.949+0000 7fa352ffd700 1 -- 172.21.15.163:0/276157427 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa344007610 con 0x7fa35c0b2140 2023-10-09T06:36:30.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.949+0000 7fa3609f8700 1 -- 172.21.15.163:0/276157427 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa35c156600 con 0x7fa35c0b2140 2023-10-09T06:36:30.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.950+0000 7fa3609f8700 1 -- 172.21.15.163:0/276157427 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa35c04c3c0 con 0x7fa35c0b2140 2023-10-09T06:36:30.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.950+0000 7fa352ffd700 1 -- 172.21.15.163:0/276157427 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fa34401e430 con 0x7fa35c0b2140 2023-10-09T06:36:30.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.950+0000 7fa352ffd700 1 --2- 172.21.15.163:0/276157427 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa33c067280 0x7fa33c069730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:30.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.950+0000 7fa352ffd700 1 -- 172.21.15.163:0/276157427 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa344085650 con 0x7fa35c0b2140 2023-10-09T06:36:30.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.950+0000 7fa35a59c700 1 --2- 172.21.15.163:0/276157427 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa33c067280 0x7fa33c069730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:30.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.951+0000 7fa35a59c700 1 --2- 172.21.15.163:0/276157427 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa33c067280 0x7fa33c069730 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0093f0 tx=0x7fa34c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:30.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:30.955+0000 7fa352ffd700 1 -- 172.21.15.163:0/276157427 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa344052590 con 0x7fa35c0b2140 2023-10-09T06:36:31.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.179+0000 7fa3609f8700 1 -- 172.21.15.163:0/276157427 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa35c06a5b0 con 0x7fa35c0b2140 2023-10-09T06:36:31.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.180+0000 7fa352ffd700 1 -- 172.21.15.163:0/276157427 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fa344017090 con 0x7fa35c0b2140 2023-10-09T06:36:31.181 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:31.181 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":58,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5501,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/862991395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":862991395},{"type":"v1","addr":"172.21.15.181:6836","nonce":862991395}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/699275566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":699275566},{"type":"v1","addr":"172.21.15.181:6837","nonce":699275566}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6504,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3664732208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3664732208},{"type":"v1","addr":"172.21.15.163:6834","nonce":3664732208}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:36:18.163421+0000","modified":"2023-10-09T06:36:21.793136+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6507},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6507":{"gid":6507,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/2687247401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2687247401},{"type":"v1","addr":"172.21.15.163:6835","nonce":2687247401}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-09T06:36:31.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.182+0000 7fa3609f8700 1 -- 172.21.15.163:0/276157427 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa33c067280 msgr2=0x7fa33c069730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:31.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.182+0000 7fa3609f8700 1 --2- 172.21.15.163:0/276157427 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa33c067280 0x7fa33c069730 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0093f0 tx=0x7fa34c008040 comp rx=0 tx=0).stop 2023-10-09T06:36:31.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.182+0000 7fa3609f8700 1 -- 172.21.15.163:0/276157427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa35c0b2140 msgr2=0x7fa35c0b25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:31.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.182+0000 7fa3609f8700 1 --2- 172.21.15.163:0/276157427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa35c0b2140 0x7fa35c0b25b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fa344001e50 tx=0x7fa344015000 comp rx=0 tx=0).stop 2023-10-09T06:36:31.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.182+0000 7fa3609f8700 1 -- 172.21.15.163:0/276157427 shutdown_connections 2023-10-09T06:36:31.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.182+0000 7fa3609f8700 1 --2- 172.21.15.163:0/276157427 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa33c067280 0x7fa33c069730 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:31.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.182+0000 7fa3609f8700 1 --2- 172.21.15.163:0/276157427 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa35c0cf480 0x7fa35c0cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:31.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.182+0000 7fa3609f8700 1 --2- 172.21.15.163:0/276157427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa348004810 0x7fa35c149dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:31.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.182+0000 7fa3609f8700 1 --2- 172.21.15.163:0/276157427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa35c0b2140 0x7fa35c0b25b0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:31.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.182+0000 7fa3609f8700 1 -- 172.21.15.163:0/276157427 >> 172.21.15.163:0/276157427 conn(0x7fa35c068e20 msgr2=0x7fa35c1535a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:31.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.182+0000 7fa3609f8700 1 -- 172.21.15.163:0/276157427 shutdown_connections 2023-10-09T06:36:31.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.182+0000 7fa3609f8700 1 -- 172.21.15.163:0/276157427 wait complete. 2023-10-09T06:36:31.184 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 58 2023-10-09T06:36:31.195 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:36:31.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.311+0000 7f620d599700 1 Processor -- start 2023-10-09T06:36:31.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.311+0000 7f620d599700 1 -- start start 2023-10-09T06:36:31.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.312+0000 7f620d599700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6208159890 0x7f620815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:31.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.312+0000 7f620d599700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6208159c60 0x7f620815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:31.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.312+0000 7f620d599700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f620815db10 0x7f6208069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:31.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.312+0000 7f620d599700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f620815a160 con 0x7f620815db10 2023-10-09T06:36:31.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.312+0000 7f620d599700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62080d13a0 con 0x7f6208159c60 2023-10-09T06:36:31.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.312+0000 7f620d599700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f620815e140 con 0x7f6208159890 2023-10-09T06:36:31.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.312+0000 7f62077fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f620815db10 0x7f6208069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:31.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.312+0000 7f62077fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f620815db10 0x7f6208069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59642/0 (socket says 172.21.15.163:59642) 2023-10-09T06:36:31.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.312+0000 7f62077fe700 1 -- 172.21.15.163:0/3598546769 learned_addr learned my addr 172.21.15.163:0/3598546769 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:31.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.312+0000 7f6206ffd700 1 --2- 172.21.15.163:0/3598546769 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6208159890 0x7f620815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:31.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.312+0000 7f62067fc700 1 --2- 172.21.15.163:0/3598546769 >> v2:172.21.15.181:3300/0 conn(0x7f6208159c60 0x7f620815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:31.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.312+0000 7f62077fe700 1 -- 172.21.15.163:0/3598546769 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6208159890 msgr2=0x7f620815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:31.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.312+0000 7f62077fe700 1 --2- 172.21.15.163:0/3598546769 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6208159890 0x7f620815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:31.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.312+0000 7f62077fe700 1 -- 172.21.15.163:0/3598546769 >> v2:172.21.15.181:3300/0 conn(0x7f6208159c60 msgr2=0x7f620815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:31.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.312+0000 7f62077fe700 1 --2- 172.21.15.163:0/3598546769 >> v2:172.21.15.181:3300/0 conn(0x7f6208159c60 0x7f620815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:31.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.312+0000 7f62077fe700 1 -- 172.21.15.163:0/3598546769 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620815a3d0 con 0x7f620815db10 2023-10-09T06:36:31.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.312+0000 7f62067fc700 1 --2- 172.21.15.163:0/3598546769 >> v2:172.21.15.181:3300/0 conn(0x7f6208159c60 0x7f620815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:31.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.312+0000 7f62077fe700 1 --2- 172.21.15.163:0/3598546769 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f620815db10 0x7f6208069560 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f62080650f0 tx=0x7f61f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8476d5ef99d5b926 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:31.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.312+0000 7f6205ffb700 1 -- 172.21.15.163:0/3598546769 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f8009e70 con 0x7f620815db10 2023-10-09T06:36:31.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.312+0000 7f6205ffb700 1 -- 172.21.15.163:0/3598546769 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f800b860 con 0x7f620815db10 2023-10-09T06:36:31.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.312+0000 7f6205ffb700 1 -- 172.21.15.163:0/3598546769 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f800bc00 con 0x7f620815db10 2023-10-09T06:36:31.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.313+0000 7f620d599700 1 -- 172.21.15.163:0/3598546769 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f620815db10 msgr2=0x7f6208069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:31.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.313+0000 7f620d599700 1 --2- 172.21.15.163:0/3598546769 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f620815db10 0x7f6208069560 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f62080650f0 tx=0x7f61f8009250 comp rx=0 tx=0).stop 2023-10-09T06:36:31.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.313+0000 7f620d599700 1 -- 172.21.15.163:0/3598546769 shutdown_connections 2023-10-09T06:36:31.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.313+0000 7f620d599700 1 --2- 172.21.15.163:0/3598546769 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f620815db10 0x7f6208069560 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:31.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.313+0000 7f620d599700 1 --2- 172.21.15.163:0/3598546769 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6208159890 0x7f620815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:31.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.313+0000 7f620d599700 1 --2- 172.21.15.163:0/3598546769 >> v2:172.21.15.181:3300/0 conn(0x7f6208159c60 0x7f620815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:31.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.313+0000 7f620d599700 1 -- 172.21.15.163:0/3598546769 >> 172.21.15.163:0/3598546769 conn(0x7f6208068e20 msgr2=0x7f62080d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:31.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.313+0000 7f620d599700 1 -- 172.21.15.163:0/3598546769 shutdown_connections 2023-10-09T06:36:31.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.313+0000 7f620d599700 1 -- 172.21.15.163:0/3598546769 wait complete. 2023-10-09T06:36:31.319 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.314+0000 7f620d599700 1 Processor -- start 2023-10-09T06:36:31.319 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.314+0000 7f620d599700 1 -- start start 2023-10-09T06:36:31.319 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.315+0000 7f620d599700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6208159890 0x7f6208152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:31.319 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.315+0000 7f620d599700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6208159c60 0x7f620814c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:31.319 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.315+0000 7f620d599700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f620815db10 0x7f620815d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:31.319 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.315+0000 7f620d599700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6208152870 con 0x7f620815db10 2023-10-09T06:36:31.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.315+0000 7f620d599700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6208152b50 con 0x7f6208159890 2023-10-09T06:36:31.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.315+0000 7f620d599700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6208152e30 con 0x7f6208159c60 2023-10-09T06:36:31.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.315+0000 7f62077fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f620815db10 0x7f620815d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:31.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.315+0000 7f62077fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f620815db10 0x7f620815d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59646/0 (socket says 172.21.15.163:59646) 2023-10-09T06:36:31.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.315+0000 7f62077fe700 1 -- 172.21.15.163:0/4205154945 learned_addr learned my addr 172.21.15.163:0/4205154945 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:31.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.315+0000 7f62067fc700 1 --2- 172.21.15.163:0/4205154945 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6208159c60 0x7f620814c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:31.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.315+0000 7f6206ffd700 1 --2- 172.21.15.163:0/4205154945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6208159890 0x7f6208152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:31.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.315+0000 7f62077fe700 1 -- 172.21.15.163:0/4205154945 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6208159c60 msgr2=0x7f620814c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:31.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.315+0000 7f62077fe700 1 --2- 172.21.15.163:0/4205154945 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6208159c60 0x7f620814c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:31.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.315+0000 7f62077fe700 1 -- 172.21.15.163:0/4205154945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6208159890 msgr2=0x7f6208152260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:31.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.315+0000 7f62077fe700 1 --2- 172.21.15.163:0/4205154945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6208159890 0x7f6208152260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:31.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.315+0000 7f62077fe700 1 -- 172.21.15.163:0/4205154945 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f80094f0 con 0x7f620815db10 2023-10-09T06:36:31.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.315+0000 7f62067fc700 1 --2- 172.21.15.163:0/4205154945 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6208159c60 0x7f620814c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:36:31.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.316+0000 7f62077fe700 1 --2- 172.21.15.163:0/4205154945 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f620815db10 0x7f620815d380 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f61f80094c0 tx=0x7f61f8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:31.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.316+0000 7f61eeffd700 1 -- 172.21.15.163:0/4205154945 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f8026480 con 0x7f620815db10 2023-10-09T06:36:31.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.316+0000 7f61eeffd700 1 -- 172.21.15.163:0/4205154945 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f801f070 con 0x7f620815db10 2023-10-09T06:36:31.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.316+0000 7f620d599700 1 -- 172.21.15.163:0/4205154945 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6208153110 con 0x7f620815db10 2023-10-09T06:36:31.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.316+0000 7f61eeffd700 1 -- 172.21.15.163:0/4205154945 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f8026830 con 0x7f620815db10 2023-10-09T06:36:31.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.316+0000 7f620d599700 1 -- 172.21.15.163:0/4205154945 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f620815d8c0 con 0x7f620815db10 2023-10-09T06:36:31.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.317+0000 7f620d599700 1 -- 172.21.15.163:0/4205154945 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f620804c3c0 con 0x7f620815db10 2023-10-09T06:36:31.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.317+0000 7f61eeffd700 1 -- 172.21.15.163:0/4205154945 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f61f8026c20 con 0x7f620815db10 2023-10-09T06:36:31.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.318+0000 7f61eeffd700 1 --2- 172.21.15.163:0/4205154945 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f61f4067290 0x7f61f4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:31.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.318+0000 7f61eeffd700 1 -- 172.21.15.163:0/4205154945 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f61f8021070 con 0x7f620815db10 2023-10-09T06:36:31.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.318+0000 7f6206ffd700 1 --2- 172.21.15.163:0/4205154945 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f61f4067290 0x7f61f4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:31.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.318+0000 7f6206ffd700 1 --2- 172.21.15.163:0/4205154945 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f61f4067290 0x7f61f4069740 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f62000076c0 tx=0x7f620000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:31.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.322+0000 7f61eeffd700 1 -- 172.21.15.163:0/4205154945 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61f80538b0 con 0x7f620815db10 2023-10-09T06:36:31.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.505+0000 7f620d599700 1 -- 172.21.15.163:0/4205154945 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f620806a5b0 con 0x7f620815db10 2023-10-09T06:36:31.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.506+0000 7f61eeffd700 1 -- 172.21.15.163:0/4205154945 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+15401 (secure 0 0 0) 0x7f61f8015020 con 0x7f620815db10 2023-10-09T06:36:31.507 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:31.507 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":89,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:36:19.522776+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:36:16.497125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-09T06:36:17.121589+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:36:31.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.508+0000 7f620d599700 1 -- 172.21.15.163:0/4205154945 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f61f4067290 msgr2=0x7f61f4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:31.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.508+0000 7f620d599700 1 --2- 172.21.15.163:0/4205154945 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f61f4067290 0x7f61f4069740 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f62000076c0 tx=0x7f620000b040 comp rx=0 tx=0).stop 2023-10-09T06:36:31.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.508+0000 7f620d599700 1 -- 172.21.15.163:0/4205154945 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f620815db10 msgr2=0x7f620815d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:31.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.508+0000 7f620d599700 1 --2- 172.21.15.163:0/4205154945 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f620815db10 0x7f620815d380 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f61f80094c0 tx=0x7f61f8017000 comp rx=0 tx=0).stop 2023-10-09T06:36:31.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.508+0000 7f620d599700 1 -- 172.21.15.163:0/4205154945 shutdown_connections 2023-10-09T06:36:31.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.508+0000 7f620d599700 1 --2- 172.21.15.163:0/4205154945 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f61f4067290 0x7f61f4069740 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:31.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.508+0000 7f620d599700 1 --2- 172.21.15.163:0/4205154945 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f620815db10 0x7f620815d380 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:31.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.508+0000 7f620d599700 1 --2- 172.21.15.163:0/4205154945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6208159890 0x7f6208152260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:31.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.508+0000 7f620d599700 1 --2- 172.21.15.163:0/4205154945 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6208159c60 0x7f620814c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:31.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.508+0000 7f620d599700 1 -- 172.21.15.163:0/4205154945 >> 172.21.15.163:0/4205154945 conn(0x7f6208068e20 msgr2=0x7f62080b19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:31.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.508+0000 7f620d599700 1 -- 172.21.15.163:0/4205154945 shutdown_connections 2023-10-09T06:36:31.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.508+0000 7f620d599700 1 -- 172.21.15.163:0/4205154945 wait complete. 2023-10-09T06:36:31.522 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-09T06:36:31.522 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:36:31.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.635+0000 7f474fad8700 1 Processor -- start 2023-10-09T06:36:31.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.635+0000 7f474fad8700 1 -- start start 2023-10-09T06:36:31.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.636+0000 7f474fad8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4748159890 0x7f474815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:31.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.636+0000 7f474fad8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4748069560 0x7f47480699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:31.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.636+0000 7f474fad8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4748069ee0 con 0x7f4748159c60 2023-10-09T06:36:31.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.636+0000 7f474fad8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f474815a160 con 0x7f4748159890 2023-10-09T06:36:31.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.636+0000 7f474fad8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47480d13a0 con 0x7f4748069560 2023-10-09T06:36:31.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.636+0000 7f474d874700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4748159890 0x7f474815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:31.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.636+0000 7f474e075700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4748069560 0x7f47480699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:31.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.636+0000 7f474e075700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4748069560 0x7f47480699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:35104/0 (socket says 172.21.15.163:35104) 2023-10-09T06:36:31.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.636+0000 7f474d874700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4748159890 0x7f474815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:36572/0 (socket says 172.21.15.163:36572) 2023-10-09T06:36:31.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.636+0000 7f474e075700 1 -- 172.21.15.163:0/346274590 learned_addr learned my addr 172.21.15.163:0/346274590 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:31.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.636+0000 7f474d073700 1 --1- 172.21.15.163:0/346274590 >> v1:172.21.15.181:6789/0 conn(0x7f4748159c60 0x7f474815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:45936/0 (socket says 172.21.15.163:45936) 2023-10-09T06:36:31.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.637+0000 7f474e075700 1 -- 172.21.15.163:0/346274590 >> v2:172.21.15.181:3300/0 conn(0x7f4748159890 msgr2=0x7f474815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:31.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.637+0000 7f474e075700 1 --2- 172.21.15.163:0/346274590 >> v2:172.21.15.181:3300/0 conn(0x7f4748159890 0x7f474815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:31.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.637+0000 7f474e075700 1 -- 172.21.15.163:0/346274590 >> v1:172.21.15.181:6789/0 conn(0x7f4748159c60 legacy=0x7f474815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:31.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.637+0000 7f474c872700 1 -- 172.21.15.163:0/346274590 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1484927050 0 0) 0x7f4748069ee0 con 0x7f4748159c60 2023-10-09T06:36:31.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.637+0000 7f474e075700 1 -- 172.21.15.163:0/346274590 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f474815e140 con 0x7f4748069560 2023-10-09T06:36:31.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.637+0000 7f474e075700 1 --2- 172.21.15.163:0/346274590 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4748069560 0x7f47480699a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4740009f20 tx=0x7f4740009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94445cd8574803b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:31.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.637+0000 7f474c872700 1 -- 172.21.15.163:0/346274590 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474000e070 con 0x7f4748069560 2023-10-09T06:36:31.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.637+0000 7f474c872700 1 -- 172.21.15.163:0/346274590 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f474000b9d0 con 0x7f4748069560 2023-10-09T06:36:31.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.637+0000 7f474c872700 1 -- 172.21.15.163:0/346274590 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47400092e0 con 0x7f4748069560 2023-10-09T06:36:31.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.638+0000 7f474fad8700 1 -- 172.21.15.163:0/346274590 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4748069560 msgr2=0x7f47480699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:31.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.638+0000 7f474fad8700 1 --2- 172.21.15.163:0/346274590 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4748069560 0x7f47480699a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4740009f20 tx=0x7f4740009840 comp rx=0 tx=0).stop 2023-10-09T06:36:31.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.638+0000 7f474fad8700 1 -- 172.21.15.163:0/346274590 shutdown_connections 2023-10-09T06:36:31.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.638+0000 7f474fad8700 1 --2- 172.21.15.163:0/346274590 >> v2:172.21.15.181:3300/0 conn(0x7f4748159890 0x7f474815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:31.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.638+0000 7f474fad8700 1 --2- 172.21.15.163:0/346274590 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4748069560 0x7f47480699a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:31.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.638+0000 7f474fad8700 1 -- 172.21.15.163:0/346274590 >> 172.21.15.163:0/346274590 conn(0x7f4748068e20 msgr2=0x7f47480d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:31.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.638+0000 7f474fad8700 1 -- 172.21.15.163:0/346274590 shutdown_connections 2023-10-09T06:36:31.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.638+0000 7f474fad8700 1 -- 172.21.15.163:0/346274590 wait complete. 2023-10-09T06:36:31.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.639+0000 7f474fad8700 1 Processor -- start 2023-10-09T06:36:31.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.639+0000 7f474fad8700 1 -- start start 2023-10-09T06:36:31.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.639+0000 7f474fad8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4748069560 0x7f4748140e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:31.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.639+0000 7f474fad8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4748159890 0x7f47480b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:31.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.639+0000 7f474fad8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4748159c60 0x7f47480d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:31.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.640+0000 7f474fad8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47480b3e90 con 0x7f4748069560 2023-10-09T06:36:31.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.640+0000 7f474fad8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47480ce4a0 con 0x7f4748159890 2023-10-09T06:36:31.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.640+0000 7f474fad8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47480cd890 con 0x7f4748159c60 2023-10-09T06:36:31.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.640+0000 7f474d874700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4748069560 0x7f4748140e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:31.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.640+0000 7f474d874700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4748069560 0x7f4748140e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59650/0 (socket says 172.21.15.163:59650) 2023-10-09T06:36:31.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.640+0000 7f474d874700 1 -- 172.21.15.163:0/3810403662 learned_addr learned my addr 172.21.15.163:0/3810403662 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:31.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.640+0000 7f474e075700 1 --2- 172.21.15.163:0/3810403662 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4748159c60 0x7f47480d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:31.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.640+0000 7f474d073700 1 --2- 172.21.15.163:0/3810403662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4748159890 0x7f47480b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:31.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.640+0000 7f474d874700 1 -- 172.21.15.163:0/3810403662 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4748159c60 msgr2=0x7f47480d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:31.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.640+0000 7f474d874700 1 --2- 172.21.15.163:0/3810403662 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4748159c60 0x7f47480d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:31.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.640+0000 7f474d874700 1 -- 172.21.15.163:0/3810403662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4748159890 msgr2=0x7f47480b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:31.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.640+0000 7f474d874700 1 --2- 172.21.15.163:0/3810403662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4748159890 0x7f47480b5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:31.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.640+0000 7f474d874700 1 -- 172.21.15.163:0/3810403662 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4740009ae0 con 0x7f4748069560 2023-10-09T06:36:31.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.640+0000 7f474d874700 1 --2- 172.21.15.163:0/3810403662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4748069560 0x7f4748140e70 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f474400ec80 tx=0x7f47440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:31.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.640+0000 7f473dffb700 1 -- 172.21.15.163:0/3810403662 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474400c850 con 0x7f4748069560 2023-10-09T06:36:31.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.640+0000 7f473dffb700 1 -- 172.21.15.163:0/3810403662 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f474400c9d0 con 0x7f4748069560 2023-10-09T06:36:31.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.640+0000 7f474fad8700 1 -- 172.21.15.163:0/3810403662 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47480cdb70 con 0x7f4748069560 2023-10-09T06:36:31.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.640+0000 7f473dffb700 1 -- 172.21.15.163:0/3810403662 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474400cd90 con 0x7f4748069560 2023-10-09T06:36:31.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.640+0000 7f474fad8700 1 -- 172.21.15.163:0/3810403662 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47480ce130 con 0x7f4748069560 2023-10-09T06:36:31.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.641+0000 7f474fad8700 1 -- 172.21.15.163:0/3810403662 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4748141470 con 0x7f4748069560 2023-10-09T06:36:31.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.642+0000 7f473dffb700 1 -- 172.21.15.163:0/3810403662 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f4744013680 con 0x7f4748069560 2023-10-09T06:36:31.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.642+0000 7f473dffb700 1 --2- 172.21.15.163:0/3810403662 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f472c067310 0x7f472c0697c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:31.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.642+0000 7f473dffb700 1 -- 172.21.15.163:0/3810403662 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4744084d80 con 0x7f4748069560 2023-10-09T06:36:31.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.642+0000 7f474d073700 1 --2- 172.21.15.163:0/3810403662 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f472c067310 0x7f472c0697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:31.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.643+0000 7f474d073700 1 --2- 172.21.15.163:0/3810403662 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f472c067310 0x7f472c0697c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4748147d70 tx=0x7f4738007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:31.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.646+0000 7f473dffb700 1 -- 172.21.15.163:0/3810403662 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4744051ba0 con 0x7f4748069560 2023-10-09T06:36:31.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.869+0000 7f474fad8700 1 -- 172.21.15.163:0/3810403662 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f474804c3c0 con 0x7f4748069560 2023-10-09T06:36:31.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.870+0000 7f473dffb700 1 -- 172.21.15.163:0/3810403662 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f47440551c0 con 0x7f4748069560 2023-10-09T06:36:31.871 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:31.871 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":58,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5501,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/862991395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":862991395},{"type":"v1","addr":"172.21.15.181:6836","nonce":862991395}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/699275566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":699275566},{"type":"v1","addr":"172.21.15.181:6837","nonce":699275566}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6504,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3664732208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3664732208},{"type":"v1","addr":"172.21.15.163:6834","nonce":3664732208}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:36:18.163421+0000","modified":"2023-10-09T06:36:21.793136+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6507},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6507":{"gid":6507,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/2687247401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2687247401},{"type":"v1","addr":"172.21.15.163:6835","nonce":2687247401}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-09T06:36:31.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.872+0000 7f474fad8700 1 -- 172.21.15.163:0/3810403662 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f472c067310 msgr2=0x7f472c0697c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:31.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.872+0000 7f474fad8700 1 --2- 172.21.15.163:0/3810403662 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f472c067310 0x7f472c0697c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4748147d70 tx=0x7f4738007d50 comp rx=0 tx=0).stop 2023-10-09T06:36:31.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.872+0000 7f474fad8700 1 -- 172.21.15.163:0/3810403662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4748069560 msgr2=0x7f4748140e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:31.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.872+0000 7f474fad8700 1 --2- 172.21.15.163:0/3810403662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4748069560 0x7f4748140e70 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f474400ec80 tx=0x7f47440091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:31.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.872+0000 7f474fad8700 1 -- 172.21.15.163:0/3810403662 shutdown_connections 2023-10-09T06:36:31.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.872+0000 7f474fad8700 1 --2- 172.21.15.163:0/3810403662 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f472c067310 0x7f472c0697c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:31.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.872+0000 7f474fad8700 1 --2- 172.21.15.163:0/3810403662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4748069560 0x7f4748140e70 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:31.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.872+0000 7f474fad8700 1 --2- 172.21.15.163:0/3810403662 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4748159c60 0x7f47480d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:31.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.872+0000 7f474fad8700 1 --2- 172.21.15.163:0/3810403662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4748159890 0x7f47480b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:31.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.872+0000 7f474fad8700 1 -- 172.21.15.163:0/3810403662 >> 172.21.15.163:0/3810403662 conn(0x7f4748068e20 msgr2=0x7f4748069930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:31.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.872+0000 7f474fad8700 1 -- 172.21.15.163:0/3810403662 shutdown_connections 2023-10-09T06:36:31.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.872+0000 7f474fad8700 1 -- 172.21.15.163:0/3810403662 wait complete. 2023-10-09T06:36:31.874 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 58 2023-10-09T06:36:31.885 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:36:31.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.998+0000 7f6f5a9ec700 1 Processor -- start 2023-10-09T06:36:31.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.998+0000 7f6f5a9ec700 1 -- start start 2023-10-09T06:36:32.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.999+0000 7f6f5a9ec700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6f54159420 0x7f6f540b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:32.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.999+0000 7f6f5a9ec700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6f5415d230 0x7f6f5415d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:32.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.999+0000 7f6f5a9ec700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f540b2980 con 0x7f6f541597f0 2023-10-09T06:36:32.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.999+0000 7f6f5a9ec700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f5415db70 con 0x7f6f5415d230 2023-10-09T06:36:32.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.999+0000 7f6f5a9ec700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f540b11d0 con 0x7f6f54159420 2023-10-09T06:36:32.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.999+0000 7f6f5a1eb700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6f5415d230 0x7f6f5415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:32.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.999+0000 7f6f5a1eb700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6f5415d230 0x7f6f5415d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59658/0 (socket says 172.21.15.163:59658) 2023-10-09T06:36:32.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.999+0000 7f6f5a1eb700 1 -- 172.21.15.163:0/749631654 learned_addr learned my addr 172.21.15.163:0/749631654 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:32.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.999+0000 7f6f599ea700 1 --2- 172.21.15.163:0/749631654 >> v2:172.21.15.181:3300/0 conn(0x7f6f54159420 0x7f6f540b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:32.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.999+0000 7f6f5a1eb700 1 -- 172.21.15.163:0/749631654 >> v2:172.21.15.181:3300/0 conn(0x7f6f54159420 msgr2=0x7f6f540b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:32.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.999+0000 7f6f5a1eb700 1 --2- 172.21.15.163:0/749631654 >> v2:172.21.15.181:3300/0 conn(0x7f6f54159420 0x7f6f540b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:32.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.999+0000 7f6f5a1eb700 1 -- 172.21.15.163:0/749631654 >> v1:172.21.15.181:6789/0 conn(0x7f6f541597f0 legacy=0x7f6f540b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:32.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:31.999+0000 7f6f5a1eb700 1 -- 172.21.15.163:0/749631654 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f540b14b0 con 0x7f6f5415d230 2023-10-09T06:36:32.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.000+0000 7f6f5a1eb700 1 --2- 172.21.15.163:0/749631654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f5415d230 0x7f6f5415d630 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f6f540b2f20 tx=0x7f6f4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae46d78ca0c9f411 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:32.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.000+0000 7f6f589e8700 1 -- 172.21.15.163:0/749631654 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c00b820 con 0x7f6f5415d230 2023-10-09T06:36:32.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.000+0000 7f6f589e8700 1 -- 172.21.15.163:0/749631654 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f4c00b980 con 0x7f6f5415d230 2023-10-09T06:36:32.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.000+0000 7f6f589e8700 1 -- 172.21.15.163:0/749631654 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c00bd20 con 0x7f6f5415d230 2023-10-09T06:36:32.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.000+0000 7f6f5a9ec700 1 -- 172.21.15.163:0/749631654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f5415d230 msgr2=0x7f6f5415d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:32.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.000+0000 7f6f5a9ec700 1 --2- 172.21.15.163:0/749631654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f5415d230 0x7f6f5415d630 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f6f540b2f20 tx=0x7f6f4c009250 comp rx=0 tx=0).stop 2023-10-09T06:36:32.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.000+0000 7f6f5a9ec700 1 -- 172.21.15.163:0/749631654 shutdown_connections 2023-10-09T06:36:32.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.000+0000 7f6f5a9ec700 1 --2- 172.21.15.163:0/749631654 >> v2:172.21.15.181:3300/0 conn(0x7f6f54159420 0x7f6f540b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:32.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.000+0000 7f6f5a9ec700 1 --2- 172.21.15.163:0/749631654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f5415d230 0x7f6f5415d630 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:32.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.000+0000 7f6f5a9ec700 1 -- 172.21.15.163:0/749631654 >> 172.21.15.163:0/749631654 conn(0x7f6f540d0660 msgr2=0x7f6f540d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:32.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.000+0000 7f6f5a9ec700 1 -- 172.21.15.163:0/749631654 shutdown_connections 2023-10-09T06:36:32.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.000+0000 7f6f5a9ec700 1 -- 172.21.15.163:0/749631654 wait complete. 2023-10-09T06:36:32.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.002+0000 7f6f5a9ec700 1 Processor -- start 2023-10-09T06:36:32.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.002+0000 7f6f5a9ec700 1 -- start start 2023-10-09T06:36:32.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.002+0000 7f6f5a9ec700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f54159420 0x7f6f541550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:32.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.002+0000 7f6f5a9ec700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f541597f0 0x7f6f540b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:32.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.002+0000 7f6f5a9ec700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f5415d230 0x7f6f540b5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:32.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.002+0000 7f6f5a9ec700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f541555e0 con 0x7f6f541597f0 2023-10-09T06:36:32.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.002+0000 7f6f5a9ec700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f540cbab0 con 0x7f6f5415d230 2023-10-09T06:36:32.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.002+0000 7f6f5a9ec700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f540cc2c0 con 0x7f6f54159420 2023-10-09T06:36:32.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.002+0000 7f6f591e9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f541597f0 0x7f6f540b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:32.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.002+0000 7f6f591e9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f541597f0 0x7f6f540b4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59672/0 (socket says 172.21.15.163:59672) 2023-10-09T06:36:32.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.002+0000 7f6f591e9700 1 -- 172.21.15.163:0/830617760 learned_addr learned my addr 172.21.15.163:0/830617760 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:32.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.003+0000 7f6f5a1eb700 1 --2- 172.21.15.163:0/830617760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f5415d230 0x7f6f540b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:32.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.003+0000 7f6f599ea700 1 --2- 172.21.15.163:0/830617760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f54159420 0x7f6f541550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:32.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.003+0000 7f6f591e9700 1 -- 172.21.15.163:0/830617760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f54159420 msgr2=0x7f6f541550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:32.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.003+0000 7f6f591e9700 1 --2- 172.21.15.163:0/830617760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f54159420 0x7f6f541550a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:32.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.003+0000 7f6f591e9700 1 -- 172.21.15.163:0/830617760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f5415d230 msgr2=0x7f6f540b5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:32.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.003+0000 7f6f591e9700 1 --2- 172.21.15.163:0/830617760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f5415d230 0x7f6f540b5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:32.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.003+0000 7f6f591e9700 1 -- 172.21.15.163:0/830617760 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f4c0094f0 con 0x7f6f541597f0 2023-10-09T06:36:32.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.003+0000 7f6f5a1eb700 1 --2- 172.21.15.163:0/830617760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f5415d230 0x7f6f540b5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:36:32.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.003+0000 7f6f591e9700 1 --2- 172.21.15.163:0/830617760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f541597f0 0x7f6f540b4e20 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f6f4400c590 tx=0x7f6f44007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:32.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.003+0000 7f6f49ffb700 1 -- 172.21.15.163:0/830617760 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f44011590 con 0x7f6f541597f0 2023-10-09T06:36:32.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.003+0000 7f6f49ffb700 1 -- 172.21.15.163:0/830617760 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f44011730 con 0x7f6f541597f0 2023-10-09T06:36:32.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.003+0000 7f6f5a9ec700 1 -- 172.21.15.163:0/830617760 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f540c5970 con 0x7f6f541597f0 2023-10-09T06:36:32.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.003+0000 7f6f49ffb700 1 -- 172.21.15.163:0/830617760 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f44011af0 con 0x7f6f541597f0 2023-10-09T06:36:32.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.003+0000 7f6f5a9ec700 1 -- 172.21.15.163:0/830617760 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f540cbfe0 con 0x7f6f541597f0 2023-10-09T06:36:32.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.004+0000 7f6f5a9ec700 1 -- 172.21.15.163:0/830617760 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f5404c3c0 con 0x7f6f541597f0 2023-10-09T06:36:32.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.004+0000 7f6f49ffb700 1 -- 172.21.15.163:0/830617760 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f6f4401c460 con 0x7f6f541597f0 2023-10-09T06:36:32.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.005+0000 7f6f49ffb700 1 --2- 172.21.15.163:0/830617760 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6f38067290 0x7f6f38069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:32.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.005+0000 7f6f49ffb700 1 -- 172.21.15.163:0/830617760 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6f44084860 con 0x7f6f541597f0 2023-10-09T06:36:32.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.005+0000 7f6f599ea700 1 --2- 172.21.15.163:0/830617760 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6f38067290 0x7f6f38069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:32.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.005+0000 7f6f599ea700 1 --2- 172.21.15.163:0/830617760 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6f38067290 0x7f6f38069740 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6f5416c8d0 tx=0x7f6f50007530 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:32.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.009+0000 7f6f49ffb700 1 -- 172.21.15.163:0/830617760 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f44051680 con 0x7f6f541597f0 2023-10-09T06:36:32.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.232+0000 7f6f5a9ec700 1 -- 172.21.15.163:0/830617760 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f540b1790 con 0x7f6f541597f0 2023-10-09T06:36:32.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.233+0000 7f6f49ffb700 1 -- 172.21.15.163:0/830617760 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f6f44054ca0 con 0x7f6f541597f0 2023-10-09T06:36:32.234 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:32.234 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":58,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5501,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/862991395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":862991395},{"type":"v1","addr":"172.21.15.181:6836","nonce":862991395}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/699275566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":699275566},{"type":"v1","addr":"172.21.15.181:6837","nonce":699275566}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6504,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3664732208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3664732208},{"type":"v1","addr":"172.21.15.163:6834","nonce":3664732208}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:36:18.163421+0000","modified":"2023-10-09T06:36:21.793136+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6507},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6507":{"gid":6507,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/2687247401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2687247401},{"type":"v1","addr":"172.21.15.163:6835","nonce":2687247401}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-09T06:36:32.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.235+0000 7f6f5a9ec700 1 -- 172.21.15.163:0/830617760 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6f38067290 msgr2=0x7f6f38069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:32.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.235+0000 7f6f5a9ec700 1 --2- 172.21.15.163:0/830617760 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6f38067290 0x7f6f38069740 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6f5416c8d0 tx=0x7f6f50007530 comp rx=0 tx=0).stop 2023-10-09T06:36:32.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.235+0000 7f6f5a9ec700 1 -- 172.21.15.163:0/830617760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f541597f0 msgr2=0x7f6f540b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:32.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.235+0000 7f6f5a9ec700 1 --2- 172.21.15.163:0/830617760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f541597f0 0x7f6f540b4e20 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f6f4400c590 tx=0x7f6f44007f30 comp rx=0 tx=0).stop 2023-10-09T06:36:32.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.235+0000 7f6f5a9ec700 1 -- 172.21.15.163:0/830617760 shutdown_connections 2023-10-09T06:36:32.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.235+0000 7f6f5a9ec700 1 --2- 172.21.15.163:0/830617760 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6f38067290 0x7f6f38069740 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:32.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.235+0000 7f6f5a9ec700 1 --2- 172.21.15.163:0/830617760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f5415d230 0x7f6f540b5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:32.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.235+0000 7f6f5a9ec700 1 --2- 172.21.15.163:0/830617760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f54159420 0x7f6f541550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:32.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.235+0000 7f6f5a9ec700 1 --2- 172.21.15.163:0/830617760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f541597f0 0x7f6f540b4e20 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:32.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.235+0000 7f6f5a9ec700 1 -- 172.21.15.163:0/830617760 >> 172.21.15.163:0/830617760 conn(0x7f6f540d0660 msgr2=0x7f6f54069890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:32.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.235+0000 7f6f5a9ec700 1 -- 172.21.15.163:0/830617760 shutdown_connections 2023-10-09T06:36:32.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.235+0000 7f6f5a9ec700 1 -- 172.21.15.163:0/830617760 wait complete. 2023-10-09T06:36:32.237 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 58 2023-10-09T06:36:32.248 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:36:32.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.362+0000 7fbcdd796700 1 Processor -- start 2023-10-09T06:36:32.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.362+0000 7fbcdd796700 1 -- start start 2023-10-09T06:36:32.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.363+0000 7fbcdd796700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbcd8159420 0x7fbcd80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:32.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.363+0000 7fbcdd796700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcd80b2980 con 0x7fbcd81597f0 2023-10-09T06:36:32.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.363+0000 7fbcdd796700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcd80b11d0 con 0x7fbcd815d230 2023-10-09T06:36:32.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.363+0000 7fbcdd796700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcd80b14b0 con 0x7fbcd8159420 2023-10-09T06:36:32.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.363+0000 7fbcd77fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fbcd81597f0 0x7fbcd80b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53522/0 (socket says 172.21.15.163:53522) 2023-10-09T06:36:32.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.363+0000 7fbcd77fe700 1 -- 172.21.15.163:0/2259974876 learned_addr learned my addr 172.21.15.163:0/2259974876 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:32.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.363+0000 7fbcd7fff700 1 --2- 172.21.15.163:0/2259974876 >> v2:172.21.15.181:3300/0 conn(0x7fbcd8159420 0x7fbcd80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:32.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.363+0000 7fbcd6ffd700 1 -- 172.21.15.163:0/2259974876 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2198924438 0 0) 0x7fbcd80b2980 con 0x7fbcd81597f0 2023-10-09T06:36:32.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.363+0000 7fbcd6ffd700 1 -- 172.21.15.163:0/2259974876 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcc0003570 con 0x7fbcd81597f0 2023-10-09T06:36:32.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.363+0000 7fbcd7fff700 1 -- 172.21.15.163:0/2259974876 >> v1:172.21.15.181:6789/0 conn(0x7fbcd815d230 legacy=0x7fbcd815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:32.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.363+0000 7fbcd7fff700 1 -- 172.21.15.163:0/2259974876 >> v1:172.21.15.163:6789/0 conn(0x7fbcd81597f0 legacy=0x7fbcd80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:32.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.363+0000 7fbcd6ffd700 1 -- 172.21.15.163:0/2259974876 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2093869383 0 0) 0x7fbcc0003570 con 0x7fbcd81597f0 2023-10-09T06:36:32.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.363+0000 7fbcd7fff700 1 -- 172.21.15.163:0/2259974876 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd80b1790 con 0x7fbcd8159420 2023-10-09T06:36:32.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.364+0000 7fbcd7fff700 1 --2- 172.21.15.163:0/2259974876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcd8159420 0x7fbcd80b5f40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbcc80098e0 tx=0x7fbcc80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6960fe2618189d52 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:32.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.364+0000 7fbcd6ffd700 1 -- 172.21.15.163:0/2259974876 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc001470 con 0x7fbcd8159420 2023-10-09T06:36:32.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.364+0000 7fbcd6ffd700 1 -- 172.21.15.163:0/2259974876 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc8011420 con 0x7fbcd8159420 2023-10-09T06:36:32.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.364+0000 7fbcd6ffd700 1 -- 172.21.15.163:0/2259974876 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc80117c0 con 0x7fbcd8159420 2023-10-09T06:36:32.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.364+0000 7fbcdd796700 1 -- 172.21.15.163:0/2259974876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcd8159420 msgr2=0x7fbcd80b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:32.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.364+0000 7fbcdd796700 1 --2- 172.21.15.163:0/2259974876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcd8159420 0x7fbcd80b5f40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbcc80098e0 tx=0x7fbcc80091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:32.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.364+0000 7fbcdd796700 1 -- 172.21.15.163:0/2259974876 shutdown_connections 2023-10-09T06:36:32.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.364+0000 7fbcdd796700 1 --2- 172.21.15.163:0/2259974876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcd8159420 0x7fbcd80b5f40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:32.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.364+0000 7fbcdd796700 1 -- 172.21.15.163:0/2259974876 >> 172.21.15.163:0/2259974876 conn(0x7fbcd80d0660 msgr2=0x7fbcd80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:32.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.364+0000 7fbcdd796700 1 -- 172.21.15.163:0/2259974876 shutdown_connections 2023-10-09T06:36:32.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.364+0000 7fbcdd796700 1 -- 172.21.15.163:0/2259974876 wait complete. 2023-10-09T06:36:32.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.365+0000 7fbcdd796700 1 Processor -- start 2023-10-09T06:36:32.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.366+0000 7fbcdd796700 1 -- start start 2023-10-09T06:36:32.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.366+0000 7fbcdd796700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcd8159420 0x7fbcd815c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:32.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.366+0000 7fbcdd796700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcd81597f0 0x7fbcd80c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:32.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.366+0000 7fbcdd796700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcd815d230 0x7fbcd80c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:32.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.366+0000 7fbcdd796700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd815c2a0 con 0x7fbcd815d230 2023-10-09T06:36:32.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.366+0000 7fbcdd796700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd815b690 con 0x7fbcd81597f0 2023-10-09T06:36:32.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.366+0000 7fbcdd796700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd815b970 con 0x7fbcd8159420 2023-10-09T06:36:32.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.366+0000 7fbcdcf95700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcd815d230 0x7fbcd80c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:32.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.366+0000 7fbcdcf95700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcd815d230 0x7fbcd80c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59674/0 (socket says 172.21.15.163:59674) 2023-10-09T06:36:32.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.366+0000 7fbcdcf95700 1 -- 172.21.15.163:0/1667750940 learned_addr learned my addr 172.21.15.163:0/1667750940 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:32.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.366+0000 7fbcd77fe700 1 --2- 172.21.15.163:0/1667750940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcd81597f0 0x7fbcd80c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:32.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.367+0000 7fbcd7fff700 1 --2- 172.21.15.163:0/1667750940 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcd8159420 0x7fbcd815c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:32.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.367+0000 7fbcdcf95700 1 -- 172.21.15.163:0/1667750940 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcd8159420 msgr2=0x7fbcd815c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:32.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.367+0000 7fbcdcf95700 1 --2- 172.21.15.163:0/1667750940 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcd8159420 0x7fbcd815c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:32.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.367+0000 7fbcdcf95700 1 -- 172.21.15.163:0/1667750940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcd81597f0 msgr2=0x7fbcd80c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:32.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.367+0000 7fbcdcf95700 1 --2- 172.21.15.163:0/1667750940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcd81597f0 0x7fbcd80c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:32.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.367+0000 7fbcdcf95700 1 -- 172.21.15.163:0/1667750940 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc8009420 con 0x7fbcd815d230 2023-10-09T06:36:32.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.367+0000 7fbcd77fe700 1 --2- 172.21.15.163:0/1667750940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcd81597f0 0x7fbcd80c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:32.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.367+0000 7fbcd7fff700 1 --2- 172.21.15.163:0/1667750940 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcd8159420 0x7fbcd815c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:32.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.367+0000 7fbcdcf95700 1 --2- 172.21.15.163:0/1667750940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcd815d230 0x7fbcd80c8180 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000c810 tx=0x7fbcd0001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:32.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.367+0000 7fbcbffff700 1 -- 172.21.15.163:0/1667750940 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0012630 con 0x7fbcd815d230 2023-10-09T06:36:32.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.367+0000 7fbcbffff700 1 -- 172.21.15.163:0/1667750940 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd00127b0 con 0x7fbcd815d230 2023-10-09T06:36:32.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.367+0000 7fbcdd796700 1 -- 172.21.15.163:0/1667750940 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd815bc50 con 0x7fbcd815d230 2023-10-09T06:36:32.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.367+0000 7fbcbffff700 1 -- 172.21.15.163:0/1667750940 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0012b70 con 0x7fbcd815d230 2023-10-09T06:36:32.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.367+0000 7fbcdd796700 1 -- 172.21.15.163:0/1667750940 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd80cc050 con 0x7fbcd815d230 2023-10-09T06:36:32.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.368+0000 7fbcdd796700 1 -- 172.21.15.163:0/1667750940 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcd815c410 con 0x7fbcd815d230 2023-10-09T06:36:32.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.368+0000 7fbcbffff700 1 -- 172.21.15.163:0/1667750940 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fbcd0003030 con 0x7fbcd815d230 2023-10-09T06:36:32.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.369+0000 7fbcbffff700 1 --2- 172.21.15.163:0/1667750940 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbcc4067290 0x7fbcc4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:32.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.369+0000 7fbcbffff700 1 -- 172.21.15.163:0/1667750940 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbcd0084b60 con 0x7fbcd815d230 2023-10-09T06:36:32.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.369+0000 7fbcd7fff700 1 --2- 172.21.15.163:0/1667750940 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbcc4067290 0x7fbcc4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:32.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.369+0000 7fbcd7fff700 1 --2- 172.21.15.163:0/1667750940 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbcc4067290 0x7fbcc4069740 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fbcc80091e0 tx=0x7fbcc8016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:32.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.373+0000 7fbcbffff700 1 -- 172.21.15.163:0/1667750940 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbcd00518c0 con 0x7fbcd815d230 2023-10-09T06:36:32.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.558+0000 7fbcdd796700 1 -- 172.21.15.163:0/1667750940 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbcd804c3c0 con 0x7fbcd815d230 2023-10-09T06:36:32.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.559+0000 7fbcbffff700 1 -- 172.21.15.163:0/1667750940 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+15401 (secure 0 0 0) 0x7fbcd0054ee0 con 0x7fbcd815d230 2023-10-09T06:36:32.560 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:32.560 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":89,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:36:19.522776+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:36:16.497125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-09T06:36:17.121589+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:36:32.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.561+0000 7fbcdd796700 1 -- 172.21.15.163:0/1667750940 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbcc4067290 msgr2=0x7fbcc4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:32.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.561+0000 7fbcdd796700 1 --2- 172.21.15.163:0/1667750940 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbcc4067290 0x7fbcc4069740 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fbcc80091e0 tx=0x7fbcc8016000 comp rx=0 tx=0).stop 2023-10-09T06:36:32.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.561+0000 7fbcdd796700 1 -- 172.21.15.163:0/1667750940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcd815d230 msgr2=0x7fbcd80c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:32.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.561+0000 7fbcdd796700 1 --2- 172.21.15.163:0/1667750940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcd815d230 0x7fbcd80c8180 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000c810 tx=0x7fbcd0001650 comp rx=0 tx=0).stop 2023-10-09T06:36:32.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.561+0000 7fbcdd796700 1 -- 172.21.15.163:0/1667750940 shutdown_connections 2023-10-09T06:36:32.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.561+0000 7fbcdd796700 1 --2- 172.21.15.163:0/1667750940 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbcc4067290 0x7fbcc4069740 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:32.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.561+0000 7fbcdd796700 1 --2- 172.21.15.163:0/1667750940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcd815d230 0x7fbcd80c8180 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:32.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.561+0000 7fbcdd796700 1 --2- 172.21.15.163:0/1667750940 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcd8159420 0x7fbcd815c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:32.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.561+0000 7fbcdd796700 1 --2- 172.21.15.163:0/1667750940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcd81597f0 0x7fbcd80c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:32.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.561+0000 7fbcdd796700 1 -- 172.21.15.163:0/1667750940 >> 172.21.15.163:0/1667750940 conn(0x7fbcd80d0660 msgr2=0x7fbcd80d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:32.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.561+0000 7fbcdd796700 1 -- 172.21.15.163:0/1667750940 shutdown_connections 2023-10-09T06:36:32.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.561+0000 7fbcdd796700 1 -- 172.21.15.163:0/1667750940 wait complete. 2023-10-09T06:36:32.574 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-09T06:36:32.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.688+0000 7fa78e843700 1 Processor -- start 2023-10-09T06:36:32.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.688+0000 7fa78e843700 1 -- start start 2023-10-09T06:36:32.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.689+0000 7fa78e843700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa788159420 0x7fa7880b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:32.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.689+0000 7fa78e843700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa7881597f0 0x7fa7880b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:32.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.689+0000 7fa78e843700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7880b11d0 con 0x7fa78815d230 2023-10-09T06:36:32.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.689+0000 7fa78e843700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7880b14b0 con 0x7fa788159420 2023-10-09T06:36:32.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.689+0000 7fa78e843700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7880b1790 con 0x7fa7881597f0 2023-10-09T06:36:32.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.689+0000 7fa78d841700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa788159420 0x7fa7880b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:32.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.689+0000 7fa78e042700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fa78815d230 0x7fa78815d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53532/0 (socket says 172.21.15.163:53532) 2023-10-09T06:36:32.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.689+0000 7fa78e042700 1 -- 172.21.15.163:0/710791005 learned_addr learned my addr 172.21.15.163:0/710791005 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:32.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.689+0000 7fa78d040700 1 --2- 172.21.15.163:0/710791005 >> v2:172.21.15.181:3300/0 conn(0x7fa7881597f0 0x7fa7880b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:32.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.689+0000 7fa78d841700 1 -- 172.21.15.163:0/710791005 >> v2:172.21.15.181:3300/0 conn(0x7fa7881597f0 msgr2=0x7fa7880b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:32.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.689+0000 7fa78d841700 1 --2- 172.21.15.163:0/710791005 >> v2:172.21.15.181:3300/0 conn(0x7fa7881597f0 0x7fa7880b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:32.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.689+0000 7fa78d841700 1 -- 172.21.15.163:0/710791005 >> v1:172.21.15.163:6789/0 conn(0x7fa78815d230 legacy=0x7fa78815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:32.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.689+0000 7fa78d841700 1 -- 172.21.15.163:0/710791005 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7880b1a70 con 0x7fa788159420 2023-10-09T06:36:32.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.689+0000 7fa78c83f700 1 -- 172.21.15.163:0/710791005 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1598663006 0 0) 0x7fa7880b11d0 con 0x7fa78815d230 2023-10-09T06:36:32.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.689+0000 7fa78d040700 1 --2- 172.21.15.163:0/710791005 >> v2:172.21.15.181:3300/0 conn(0x7fa7881597f0 0x7fa7880b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:32.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.690+0000 7fa78d841700 1 --2- 172.21.15.163:0/710791005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa788159420 0x7fa7880b5f40 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fa788089750 tx=0x7fa7840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7632cab9286d479c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:32.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.690+0000 7fa78c83f700 1 -- 172.21.15.163:0/710791005 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78400b6b0 con 0x7fa788159420 2023-10-09T06:36:32.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.690+0000 7fa78c83f700 1 -- 172.21.15.163:0/710791005 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78400b810 con 0x7fa788159420 2023-10-09T06:36:32.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.690+0000 7fa78c83f700 1 -- 172.21.15.163:0/710791005 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78400bbb0 con 0x7fa788159420 2023-10-09T06:36:32.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.690+0000 7fa78e843700 1 -- 172.21.15.163:0/710791005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa788159420 msgr2=0x7fa7880b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:32.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.690+0000 7fa78e843700 1 --2- 172.21.15.163:0/710791005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa788159420 0x7fa7880b5f40 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fa788089750 tx=0x7fa7840091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:32.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.690+0000 7fa78e843700 1 -- 172.21.15.163:0/710791005 shutdown_connections 2023-10-09T06:36:32.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.690+0000 7fa78e843700 1 --2- 172.21.15.163:0/710791005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa788159420 0x7fa7880b5f40 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:32.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.690+0000 7fa78e843700 1 --2- 172.21.15.163:0/710791005 >> v2:172.21.15.181:3300/0 conn(0x7fa7881597f0 0x7fa7880b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:32.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.690+0000 7fa78e843700 1 -- 172.21.15.163:0/710791005 >> 172.21.15.163:0/710791005 conn(0x7fa7880d0660 msgr2=0x7fa7880d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:32.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.690+0000 7fa78e843700 1 -- 172.21.15.163:0/710791005 shutdown_connections 2023-10-09T06:36:32.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.690+0000 7fa78e843700 1 -- 172.21.15.163:0/710791005 wait complete. 2023-10-09T06:36:32.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.691+0000 7fa78e843700 1 Processor -- start 2023-10-09T06:36:32.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.691+0000 7fa78e843700 1 -- start start 2023-10-09T06:36:32.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.692+0000 7fa78e843700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa788159420 0x7fa7880c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:32.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.692+0000 7fa78e843700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7881597f0 0x7fa7881437e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:32.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.692+0000 7fa78e843700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa78815d230 0x7fa788156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:32.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.692+0000 7fa78e843700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7880cafa0 con 0x7fa788159420 2023-10-09T06:36:32.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.692+0000 7fa78e843700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7880ca390 con 0x7fa78815d230 2023-10-09T06:36:32.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.692+0000 7fa78e843700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7880ca670 con 0x7fa7881597f0 2023-10-09T06:36:32.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.692+0000 7fa78d841700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa788159420 0x7fa7880c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:32.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.692+0000 7fa78d841700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa788159420 0x7fa7880c9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59688/0 (socket says 172.21.15.163:59688) 2023-10-09T06:36:32.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.692+0000 7fa78d841700 1 -- 172.21.15.163:0/3271664038 learned_addr learned my addr 172.21.15.163:0/3271664038 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:32.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.692+0000 7fa78e042700 1 --2- 172.21.15.163:0/3271664038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa78815d230 0x7fa788156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:32.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.692+0000 7fa78d040700 1 --2- 172.21.15.163:0/3271664038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7881597f0 0x7fa7881437e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:32.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.692+0000 7fa78d841700 1 -- 172.21.15.163:0/3271664038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7881597f0 msgr2=0x7fa7881437e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:32.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.692+0000 7fa78d841700 1 --2- 172.21.15.163:0/3271664038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7881597f0 0x7fa7881437e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:32.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.692+0000 7fa78d841700 1 -- 172.21.15.163:0/3271664038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa78815d230 msgr2=0x7fa788156b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:32.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.692+0000 7fa78d841700 1 --2- 172.21.15.163:0/3271664038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa78815d230 0x7fa788156b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:32.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.692+0000 7fa78d841700 1 -- 172.21.15.163:0/3271664038 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa784009420 con 0x7fa788159420 2023-10-09T06:36:32.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.693+0000 7fa78d841700 1 --2- 172.21.15.163:0/3271664038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa788159420 0x7fa7880c9af0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fa7840091e0 tx=0x7fa784017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:32.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.693+0000 7fa77dffb700 1 -- 172.21.15.163:0/3271664038 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78400bb90 con 0x7fa788159420 2023-10-09T06:36:32.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.693+0000 7fa77dffb700 1 -- 172.21.15.163:0/3271664038 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa784011420 con 0x7fa788159420 2023-10-09T06:36:32.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.693+0000 7fa78e843700 1 -- 172.21.15.163:0/3271664038 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7880ca950 con 0x7fa788159420 2023-10-09T06:36:32.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.693+0000 7fa77dffb700 1 -- 172.21.15.163:0/3271664038 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7840117c0 con 0x7fa788159420 2023-10-09T06:36:32.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.693+0000 7fa78e843700 1 -- 172.21.15.163:0/3271664038 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7880ca030 con 0x7fa788159420 2023-10-09T06:36:32.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.694+0000 7fa78e843700 1 -- 172.21.15.163:0/3271664038 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa78804c3c0 con 0x7fa788159420 2023-10-09T06:36:32.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.694+0000 7fa77dffb700 1 -- 172.21.15.163:0/3271664038 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fa784011bb0 con 0x7fa788159420 2023-10-09T06:36:32.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.695+0000 7fa77dffb700 1 --2- 172.21.15.163:0/3271664038 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa76c067290 0x7fa76c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:32.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.695+0000 7fa77dffb700 1 -- 172.21.15.163:0/3271664038 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa784086410 con 0x7fa788159420 2023-10-09T06:36:32.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.695+0000 7fa78d040700 1 --2- 172.21.15.163:0/3271664038 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa76c067290 0x7fa76c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:32.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.695+0000 7fa78d040700 1 --2- 172.21.15.163:0/3271664038 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa76c067290 0x7fa76c069740 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa778006f00 tx=0x7fa778008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:32.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.699+0000 7fa77dffb700 1 -- 172.21.15.163:0/3271664038 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa784053270 con 0x7fa788159420 2023-10-09T06:36:32.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.926+0000 7fa78e843700 1 -- 172.21.15.163:0/3271664038 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa7880b1d50 con 0x7fa788159420 2023-10-09T06:36:32.927 INFO:tasks.ceph.mon.a.smithi163.stderr:2023-10-09T06:36:32.926+0000 7ff251c04700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-09T06:36:32.933 INFO:tasks.ceph.mds.a.smithi163.stderr: -796> 2023-10-09T06:36:24.527+0000 7f7bf3de8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-09T06:36:32.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.934+0000 7fa77dffb700 1 -- 172.21.15.163:0/3271664038 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v59) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa784011e60 con 0x7fa788159420 2023-10-09T06:36:32.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.936+0000 7fa78e843700 1 -- 172.21.15.163:0/3271664038 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa76c067290 msgr2=0x7fa76c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:32.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.936+0000 7fa78e843700 1 --2- 172.21.15.163:0/3271664038 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa76c067290 0x7fa76c069740 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa778006f00 tx=0x7fa778008040 comp rx=0 tx=0).stop 2023-10-09T06:36:32.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.936+0000 7fa78e843700 1 -- 172.21.15.163:0/3271664038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa788159420 msgr2=0x7fa7880c9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:32.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.936+0000 7fa78e843700 1 --2- 172.21.15.163:0/3271664038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa788159420 0x7fa7880c9af0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fa7840091e0 tx=0x7fa784017000 comp rx=0 tx=0).stop 2023-10-09T06:36:32.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.936+0000 7fa78e843700 1 -- 172.21.15.163:0/3271664038 shutdown_connections 2023-10-09T06:36:32.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.936+0000 7fa78e843700 1 --2- 172.21.15.163:0/3271664038 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa76c067290 0x7fa76c069740 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:32.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.936+0000 7fa78e843700 1 --2- 172.21.15.163:0/3271664038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa788159420 0x7fa7880c9af0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:32.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.936+0000 7fa78e843700 1 --2- 172.21.15.163:0/3271664038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa78815d230 0x7fa788156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:32.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.936+0000 7fa78e843700 1 --2- 172.21.15.163:0/3271664038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7881597f0 0x7fa7881437e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:32.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.936+0000 7fa78e843700 1 -- 172.21.15.163:0/3271664038 >> 172.21.15.163:0/3271664038 conn(0x7fa7880d0660 msgr2=0x7fa7880cfcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:32.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.937+0000 7fa78e843700 1 -- 172.21.15.163:0/3271664038 shutdown_connections 2023-10-09T06:36:32.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:32.937+0000 7fa78e843700 1 -- 172.21.15.163:0/3271664038 wait complete. 2023-10-09T06:36:32.939 INFO:teuthology.orchestra.run.smithi163.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-09T06:36:32.950 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-09T06:36:32.954 INFO:tasks.ceph.mds.a.smithi163.stdout:starting mds.a at 2023-10-09T06:36:33.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.066+0000 7f93bf389700 1 Processor -- start 2023-10-09T06:36:33.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.066+0000 7f93bf389700 1 -- start start 2023-10-09T06:36:33.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.067+0000 7f93bf389700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f93b80c91e0 0x7f93b80cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:33.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.067+0000 7f93bf389700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f93b806ae10 0x7f93b806b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:33.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.067+0000 7f93bf389700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93b80cc210 con 0x7f93b806aa40 2023-10-09T06:36:33.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.067+0000 7f93bf389700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b80cc4f0 con 0x7f93b806ae10 2023-10-09T06:36:33.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.067+0000 7f93bf389700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b80cc7d0 con 0x7f93b80c91e0 2023-10-09T06:36:33.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.067+0000 7f93bd926700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f93b806ae10 0x7f93b806b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:33.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.067+0000 7f93bd926700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f93b806ae10 0x7f93b806b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59720/0 (socket says 172.21.15.163:59720) 2023-10-09T06:36:33.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.067+0000 7f93bd926700 1 -- 172.21.15.163:0/528870371 learned_addr learned my addr 172.21.15.163:0/528870371 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:33.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.067+0000 7f93bc924700 1 --2- 172.21.15.163:0/528870371 >> v2:172.21.15.181:3300/0 conn(0x7f93b80c91e0 0x7f93b80cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:33.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.067+0000 7f93bd926700 1 -- 172.21.15.163:0/528870371 >> v2:172.21.15.181:3300/0 conn(0x7f93b80c91e0 msgr2=0x7f93b80cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:33.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.067+0000 7f93bd926700 1 --2- 172.21.15.163:0/528870371 >> v2:172.21.15.181:3300/0 conn(0x7f93b80c91e0 0x7f93b80cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:33.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.067+0000 7f93bd926700 1 -- 172.21.15.163:0/528870371 >> v1:172.21.15.181:6789/0 conn(0x7f93b806aa40 legacy=0x7f93b806bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:33.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.067+0000 7f93bd926700 1 -- 172.21.15.163:0/528870371 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b80ccab0 con 0x7f93b806ae10 2023-10-09T06:36:33.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.068+0000 7f93bd926700 1 --2- 172.21.15.163:0/528870371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93b806ae10 0x7f93b806b270 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f93b00099b0 tx=0x7f93b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a7beba4f19ba7 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:33.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.068+0000 7f93b7fff700 1 -- 172.21.15.163:0/528870371 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0009ea0 con 0x7f93b806ae10 2023-10-09T06:36:33.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.068+0000 7f93b7fff700 1 -- 172.21.15.163:0/528870371 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b000b820 con 0x7f93b806ae10 2023-10-09T06:36:33.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.068+0000 7f93b7fff700 1 -- 172.21.15.163:0/528870371 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b000bbc0 con 0x7f93b806ae10 2023-10-09T06:36:33.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.068+0000 7f93bf389700 1 -- 172.21.15.163:0/528870371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93b806ae10 msgr2=0x7f93b806b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:33.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.068+0000 7f93bf389700 1 --2- 172.21.15.163:0/528870371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93b806ae10 0x7f93b806b270 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f93b00099b0 tx=0x7f93b0009250 comp rx=0 tx=0).stop 2023-10-09T06:36:33.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.068+0000 7f93bf389700 1 -- 172.21.15.163:0/528870371 shutdown_connections 2023-10-09T06:36:33.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.068+0000 7f93bf389700 1 --2- 172.21.15.163:0/528870371 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93b806ae10 0x7f93b806b270 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:33.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.068+0000 7f93bf389700 1 --2- 172.21.15.163:0/528870371 >> v2:172.21.15.181:3300/0 conn(0x7f93b80c91e0 0x7f93b80cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:33.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.068+0000 7f93bf389700 1 -- 172.21.15.163:0/528870371 >> 172.21.15.163:0/528870371 conn(0x7f93b80cd670 msgr2=0x7f93b80cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:33.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.068+0000 7f93bf389700 1 -- 172.21.15.163:0/528870371 shutdown_connections 2023-10-09T06:36:33.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.068+0000 7f93bf389700 1 -- 172.21.15.163:0/528870371 wait complete. 2023-10-09T06:36:33.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.069+0000 7f93bf389700 1 Processor -- start 2023-10-09T06:36:33.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.069+0000 7f93bf389700 1 -- start start 2023-10-09T06:36:33.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.070+0000 7f93bf389700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93b806aa40 0x7f93b814e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:33.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.070+0000 7f93bf389700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93b806ae10 0x7f93b8166240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:33.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.070+0000 7f93bf389700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93b80c91e0 0x7f93b80c34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:33.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.070+0000 7f93bf389700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b814e900 con 0x7f93b80c91e0 2023-10-09T06:36:33.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.070+0000 7f93bf389700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b8151b00 con 0x7f93b806ae10 2023-10-09T06:36:33.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.070+0000 7f93bf389700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b8166780 con 0x7f93b806aa40 2023-10-09T06:36:33.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.070+0000 7f93bd926700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93b80c91e0 0x7f93b80c34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:33.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.070+0000 7f93bd926700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93b80c91e0 0x7f93b80c34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59724/0 (socket says 172.21.15.163:59724) 2023-10-09T06:36:33.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.070+0000 7f93bd926700 1 -- 172.21.15.163:0/2745372856 learned_addr learned my addr 172.21.15.163:0/2745372856 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:33.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.071+0000 7f93bc924700 1 --2- 172.21.15.163:0/2745372856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93b806ae10 0x7f93b8166240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:33.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.071+0000 7f93bd125700 1 --2- 172.21.15.163:0/2745372856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93b806aa40 0x7f93b814e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:33.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.071+0000 7f93bd926700 1 -- 172.21.15.163:0/2745372856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93b806aa40 msgr2=0x7f93b814e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:33.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.071+0000 7f93bd926700 1 --2- 172.21.15.163:0/2745372856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93b806aa40 0x7f93b814e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:33.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.071+0000 7f93bd926700 1 -- 172.21.15.163:0/2745372856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93b806ae10 msgr2=0x7f93b8166240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:33.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.071+0000 7f93bd926700 1 --2- 172.21.15.163:0/2745372856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93b806ae10 0x7f93b8166240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:33.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.071+0000 7f93bd926700 1 -- 172.21.15.163:0/2745372856 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b00094f0 con 0x7f93b80c91e0 2023-10-09T06:36:33.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.071+0000 7f93bd125700 1 --2- 172.21.15.163:0/2745372856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93b806aa40 0x7f93b814e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:33.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.071+0000 7f93bd926700 1 --2- 172.21.15.163:0/2745372856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93b80c91e0 0x7f93b80c34e0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f93b806c6a0 tx=0x7f93b0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:33.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.071+0000 7f93b57fa700 1 -- 172.21.15.163:0/2745372856 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0011420 con 0x7f93b80c91e0 2023-10-09T06:36:33.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.071+0000 7f93b57fa700 1 -- 172.21.15.163:0/2745372856 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b00115a0 con 0x7f93b80c91e0 2023-10-09T06:36:33.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.071+0000 7f93bf389700 1 -- 172.21.15.163:0/2745372856 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b8150ef0 con 0x7f93b80c91e0 2023-10-09T06:36:33.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.071+0000 7f93b57fa700 1 -- 172.21.15.163:0/2745372856 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0011960 con 0x7f93b80c91e0 2023-10-09T06:36:33.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.071+0000 7f93bf389700 1 -- 172.21.15.163:0/2745372856 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b81514b0 con 0x7f93b80c91e0 2023-10-09T06:36:33.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.072+0000 7f93bf389700 1 -- 172.21.15.163:0/2745372856 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93b804c3c0 con 0x7f93b80c91e0 2023-10-09T06:36:33.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.072+0000 7f93b57fa700 1 -- 172.21.15.163:0/2745372856 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f93b00253f0 con 0x7f93b80c91e0 2023-10-09T06:36:33.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.073+0000 7f93b57fa700 1 --2- 172.21.15.163:0/2745372856 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f93a4067290 0x7f93a4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:33.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.073+0000 7f93b57fa700 1 -- 172.21.15.163:0/2745372856 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f93b0086580 con 0x7f93b80c91e0 2023-10-09T06:36:33.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.073+0000 7f93bd125700 1 --2- 172.21.15.163:0/2745372856 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f93a4067290 0x7f93a4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:33.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.073+0000 7f93bd125700 1 --2- 172.21.15.163:0/2745372856 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f93a4067290 0x7f93a4069740 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f93b8157570 tx=0x7f93a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:33.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.077+0000 7f93b57fa700 1 -- 172.21.15.163:0/2745372856 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f93b0053390 con 0x7f93b80c91e0 2023-10-09T06:36:33.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.299+0000 7f93bf389700 1 -- 172.21.15.163:0/2745372856 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f93b80c7cb0 con 0x7f93b80c91e0 2023-10-09T06:36:33.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.932+0000 7f93b57fa700 1 -- 172.21.15.163:0/2745372856 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v60) v1 ==== 104+0+0 (secure 0 0 0) 0x7f93b00569b0 con 0x7f93b80c91e0 2023-10-09T06:36:33.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.935+0000 7f93bf389700 1 -- 172.21.15.163:0/2745372856 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f93a4067290 msgr2=0x7f93a4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:33.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.935+0000 7f93bf389700 1 --2- 172.21.15.163:0/2745372856 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f93a4067290 0x7f93a4069740 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f93b8157570 tx=0x7f93a8009040 comp rx=0 tx=0).stop 2023-10-09T06:36:33.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.935+0000 7f93bf389700 1 -- 172.21.15.163:0/2745372856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93b80c91e0 msgr2=0x7f93b80c34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:33.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.935+0000 7f93bf389700 1 --2- 172.21.15.163:0/2745372856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93b80c91e0 0x7f93b80c34e0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f93b806c6a0 tx=0x7f93b0017000 comp rx=0 tx=0).stop 2023-10-09T06:36:33.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.935+0000 7f93bf389700 1 -- 172.21.15.163:0/2745372856 shutdown_connections 2023-10-09T06:36:33.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.935+0000 7f93bf389700 1 --2- 172.21.15.163:0/2745372856 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f93a4067290 0x7f93a4069740 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:33.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.935+0000 7f93bf389700 1 --2- 172.21.15.163:0/2745372856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93b80c91e0 0x7f93b80c34e0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:33.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.935+0000 7f93bf389700 1 --2- 172.21.15.163:0/2745372856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93b806aa40 0x7f93b814e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:33.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.935+0000 7f93bf389700 1 --2- 172.21.15.163:0/2745372856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93b806ae10 0x7f93b8166240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:33.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.935+0000 7f93bf389700 1 -- 172.21.15.163:0/2745372856 >> 172.21.15.163:0/2745372856 conn(0x7f93b80cd670 msgr2=0x7f93b8150430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:33.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.935+0000 7f93bf389700 1 -- 172.21.15.163:0/2745372856 shutdown_connections 2023-10-09T06:36:33.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:33.935+0000 7f93bf389700 1 -- 172.21.15.163:0/2745372856 wait complete. 2023-10-09T06:36:33.948 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-09T06:36:34.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.106+0000 7f5eb56b2700 1 Processor -- start 2023-10-09T06:36:34.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.106+0000 7f5eb56b2700 1 -- start start 2023-10-09T06:36:34.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.106+0000 7f5eb56b2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5eb0159ec0 0x7f5eb00b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:34.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.106+0000 7f5eb56b2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5eb00b53d0 0x7f5eb00cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:34.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.106+0000 7f5eb56b2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eb00b5b00 con 0x7f5eb00cce40 2023-10-09T06:36:34.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.106+0000 7f5eb56b2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb00b5da0 con 0x7f5eb00b53d0 2023-10-09T06:36:34.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.106+0000 7f5eb56b2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb00b6080 con 0x7f5eb0159ec0 2023-10-09T06:36:34.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.107+0000 7f5eae7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5eb00b53d0 0x7f5eb00cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:34.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.107+0000 7f5eaeffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5eb0159ec0 0x7f5eb00b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:34.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.107+0000 7f5eaf7fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f5eb00cce40 0x7f5eb00cd280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:45964/0 (socket says 172.21.15.163:45964) 2023-10-09T06:36:34.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.107+0000 7f5eae7fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5eb00b53d0 0x7f5eb00cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:36660/0 (socket says 172.21.15.163:36660) 2023-10-09T06:36:34.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.107+0000 7f5eaeffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5eb0159ec0 0x7f5eb00b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:35174/0 (socket says 172.21.15.163:35174) 2023-10-09T06:36:34.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.107+0000 7f5eaf7fe700 1 -- 172.21.15.163:0/1785530953 learned_addr learned my addr 172.21.15.163:0/1785530953 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:34.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.107+0000 7f5eaeffd700 1 -- 172.21.15.163:0/1785530953 >> v2:172.21.15.181:3300/0 conn(0x7f5eb00b53d0 msgr2=0x7f5eb00cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:34.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.107+0000 7f5eaeffd700 1 --2- 172.21.15.163:0/1785530953 >> v2:172.21.15.181:3300/0 conn(0x7f5eb00b53d0 0x7f5eb00cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:34.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.107+0000 7f5eaeffd700 1 -- 172.21.15.163:0/1785530953 >> v1:172.21.15.181:6789/0 conn(0x7f5eb00cce40 legacy=0x7f5eb00cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:34.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.107+0000 7f5eadffb700 1 -- 172.21.15.163:0/1785530953 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 687310250 0 0) 0x7f5eb00b5b00 con 0x7f5eb00cce40 2023-10-09T06:36:34.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.107+0000 7f5eaeffd700 1 -- 172.21.15.163:0/1785530953 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb00cc100 con 0x7f5eb0159ec0 2023-10-09T06:36:34.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.107+0000 7f5eae7fc700 1 --2- 172.21.15.163:0/1785530953 >> v2:172.21.15.181:3300/0 conn(0x7f5eb00b53d0 0x7f5eb00cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:36:34.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.108+0000 7f5eaeffd700 1 --2- 172.21.15.163:0/1785530953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5eb0159ec0 0x7f5eb00b4e90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009eb0 tx=0x7f5ea8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7dd9ae40c543bb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:34.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.108+0000 7f5eadffb700 1 -- 172.21.15.163:0/1785530953 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea800e070 con 0x7f5eb0159ec0 2023-10-09T06:36:34.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.108+0000 7f5eadffb700 1 -- 172.21.15.163:0/1785530953 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea800bc30 con 0x7f5eb0159ec0 2023-10-09T06:36:34.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.108+0000 7f5eadffb700 1 -- 172.21.15.163:0/1785530953 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8012430 con 0x7f5eb0159ec0 2023-10-09T06:36:34.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.108+0000 7f5eb56b2700 1 -- 172.21.15.163:0/1785530953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5eb0159ec0 msgr2=0x7f5eb00b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:34.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.108+0000 7f5eb56b2700 1 --2- 172.21.15.163:0/1785530953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5eb0159ec0 0x7f5eb00b4e90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009eb0 tx=0x7f5ea8009780 comp rx=0 tx=0).stop 2023-10-09T06:36:34.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.108+0000 7f5eb56b2700 1 -- 172.21.15.163:0/1785530953 shutdown_connections 2023-10-09T06:36:34.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.108+0000 7f5eb56b2700 1 --2- 172.21.15.163:0/1785530953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5eb0159ec0 0x7f5eb00b4e90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:34.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.108+0000 7f5eb56b2700 1 --2- 172.21.15.163:0/1785530953 >> v2:172.21.15.181:3300/0 conn(0x7f5eb00b53d0 0x7f5eb00cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:34.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.108+0000 7f5eb56b2700 1 -- 172.21.15.163:0/1785530953 >> 172.21.15.163:0/1785530953 conn(0x7f5eb00999f0 msgr2=0x7f5eb00b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:34.115 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.108+0000 7f5eb56b2700 1 -- 172.21.15.163:0/1785530953 shutdown_connections 2023-10-09T06:36:34.115 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.108+0000 7f5eb56b2700 1 -- 172.21.15.163:0/1785530953 wait complete. 2023-10-09T06:36:34.115 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.110+0000 7f5eb56b2700 1 Processor -- start 2023-10-09T06:36:34.115 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.110+0000 7f5eb56b2700 1 -- start start 2023-10-09T06:36:34.115 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.110+0000 7f5eb56b2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5eb00b53d0 0x7f5eb0068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:34.115 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.110+0000 7f5eb56b2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5eb00cce40 0x7f5eb0158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:34.115 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.110+0000 7f5eb56b2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5eb0159ec0 0x7f5eb0143db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:34.115 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.110+0000 7f5eb56b2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb0155f60 con 0x7f5eb0159ec0 2023-10-09T06:36:34.115 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.110+0000 7f5eb56b2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb00688d0 con 0x7f5eb00cce40 2023-10-09T06:36:34.116 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.110+0000 7f5eb56b2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb016a790 con 0x7f5eb00b53d0 2023-10-09T06:36:34.116 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.110+0000 7f5eaf7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5eb0159ec0 0x7f5eb0143db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:34.116 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.110+0000 7f5eaf7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5eb0159ec0 0x7f5eb0143db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59730/0 (socket says 172.21.15.163:59730) 2023-10-09T06:36:34.116 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.110+0000 7f5eaf7fe700 1 -- 172.21.15.163:0/167918812 learned_addr learned my addr 172.21.15.163:0/167918812 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:34.116 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.111+0000 7f5eaeffd700 1 --2- 172.21.15.163:0/167918812 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5eb00b53d0 0x7f5eb0068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:34.116 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.111+0000 7f5eae7fc700 1 --2- 172.21.15.163:0/167918812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5eb00cce40 0x7f5eb0158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:34.116 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.111+0000 7f5eaf7fe700 1 -- 172.21.15.163:0/167918812 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5eb00b53d0 msgr2=0x7f5eb0068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:34.116 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.111+0000 7f5eaf7fe700 1 --2- 172.21.15.163:0/167918812 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5eb00b53d0 0x7f5eb0068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:34.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.111+0000 7f5eaf7fe700 1 -- 172.21.15.163:0/167918812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5eb00cce40 msgr2=0x7f5eb0158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:34.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.111+0000 7f5eaf7fe700 1 --2- 172.21.15.163:0/167918812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5eb00cce40 0x7f5eb0158060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:34.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.111+0000 7f5eaf7fe700 1 -- 172.21.15.163:0/167918812 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e98009420 con 0x7f5eb0159ec0 2023-10-09T06:36:34.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.111+0000 7f5eae7fc700 1 --2- 172.21.15.163:0/167918812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5eb00cce40 0x7f5eb0158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:34.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.111+0000 7f5eaf7fe700 1 --2- 172.21.15.163:0/167918812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5eb0159ec0 0x7f5eb0143db0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f5ea000c5d0 tx=0x7f5ea0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:34.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.111+0000 7f5ea6ffd700 1 -- 172.21.15.163:0/167918812 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0005b30 con 0x7f5eb0159ec0 2023-10-09T06:36:34.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.111+0000 7f5ea6ffd700 1 -- 172.21.15.163:0/167918812 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea0005c90 con 0x7f5eb0159ec0 2023-10-09T06:36:34.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.111+0000 7f5eb56b2700 1 -- 172.21.15.163:0/167918812 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea8009a20 con 0x7f5eb0159ec0 2023-10-09T06:36:34.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.111+0000 7f5ea6ffd700 1 -- 172.21.15.163:0/167918812 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0007320 con 0x7f5eb0159ec0 2023-10-09T06:36:34.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.111+0000 7f5eb56b2700 1 -- 172.21.15.163:0/167918812 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb01561f0 con 0x7f5eb0159ec0 2023-10-09T06:36:34.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.112+0000 7f5eb56b2700 1 -- 172.21.15.163:0/167918812 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb0063c90 con 0x7f5eb0159ec0 2023-10-09T06:36:34.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.112+0000 7f5ea6ffd700 1 -- 172.21.15.163:0/167918812 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f5ea0007690 con 0x7f5eb0159ec0 2023-10-09T06:36:34.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.113+0000 7f5ea6ffd700 1 --2- 172.21.15.163:0/167918812 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5e90067290 0x7f5e90069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:34.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.113+0000 7f5ea6ffd700 1 -- 172.21.15.163:0/167918812 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f5ea0084060 con 0x7f5eb0159ec0 2023-10-09T06:36:34.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.113+0000 7f5eaeffd700 1 --2- 172.21.15.163:0/167918812 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5e90067290 0x7f5e90069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:34.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.113+0000 7f5eaeffd700 1 --2- 172.21.15.163:0/167918812 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5e90067290 0x7f5e90069740 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009eb0 tx=0x7f5ea8019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:34.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.117+0000 7f5ea6ffd700 1 -- 172.21.15.163:0/167918812 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ea0050e70 con 0x7f5eb0159ec0 2023-10-09T06:36:34.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.301+0000 7f5eb56b2700 1 -- 172.21.15.163:0/167918812 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5eb00718b0 con 0x7f5eb0159ec0 2023-10-09T06:36:34.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.941+0000 7f5ea6ffd700 1 -- 172.21.15.163:0/167918812 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5ea0017020 con 0x7f5eb0159ec0 2023-10-09T06:36:34.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.944+0000 7f5eb56b2700 1 -- 172.21.15.163:0/167918812 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5e90067290 msgr2=0x7f5e90069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:34.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.944+0000 7f5eb56b2700 1 --2- 172.21.15.163:0/167918812 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5e90067290 0x7f5e90069740 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009eb0 tx=0x7f5ea8019000 comp rx=0 tx=0).stop 2023-10-09T06:36:34.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.944+0000 7f5eb56b2700 1 -- 172.21.15.163:0/167918812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5eb0159ec0 msgr2=0x7f5eb0143db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:34.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.944+0000 7f5eb56b2700 1 --2- 172.21.15.163:0/167918812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5eb0159ec0 0x7f5eb0143db0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f5ea000c5d0 tx=0x7f5ea0007d30 comp rx=0 tx=0).stop 2023-10-09T06:36:34.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.944+0000 7f5eb56b2700 1 -- 172.21.15.163:0/167918812 shutdown_connections 2023-10-09T06:36:34.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.944+0000 7f5eb56b2700 1 --2- 172.21.15.163:0/167918812 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5e90067290 0x7f5e90069740 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:34.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.944+0000 7f5eb56b2700 1 --2- 172.21.15.163:0/167918812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5eb0159ec0 0x7f5eb0143db0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:34.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.944+0000 7f5eb56b2700 1 --2- 172.21.15.163:0/167918812 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5eb00b53d0 0x7f5eb0068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:34.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.944+0000 7f5eb56b2700 1 --2- 172.21.15.163:0/167918812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5eb00cce40 0x7f5eb0158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:34.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.944+0000 7f5eb56b2700 1 -- 172.21.15.163:0/167918812 >> 172.21.15.163:0/167918812 conn(0x7f5eb00999f0 msgr2=0x7f5eb00cf540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:34.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.944+0000 7f5eb56b2700 1 -- 172.21.15.163:0/167918812 shutdown_connections 2023-10-09T06:36:34.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:34.944+0000 7f5eb56b2700 1 -- 172.21.15.163:0/167918812 wait complete. 2023-10-09T06:36:34.948 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_metadata' removed 2023-10-09T06:36:34.959 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-09T06:36:35.078 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.077+0000 7fa273015700 1 Processor -- start 2023-10-09T06:36:35.078 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.077+0000 7fa273015700 1 -- start start 2023-10-09T06:36:35.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.078+0000 7fa273015700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa26c159ec0 0x7fa26c0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:35.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.078+0000 7fa273015700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa26c0b53d0 0x7fa26c0cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:35.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.078+0000 7fa273015700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa26c0cce40 0x7fa26c0cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:35.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.078+0000 7fa273015700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa26c0b5b00 con 0x7fa26c0cce40 2023-10-09T06:36:35.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.078+0000 7fa273015700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa26c0b5850 con 0x7fa26c159ec0 2023-10-09T06:36:35.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.078+0000 7fa273015700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c0b5da0 con 0x7fa26c0b53d0 2023-10-09T06:36:35.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.078+0000 7fa2715b2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa26c0cce40 0x7fa26c0cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:35.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.078+0000 7fa2715b2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa26c0cce40 0x7fa26c0cd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59742/0 (socket says 172.21.15.163:59742) 2023-10-09T06:36:35.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.078+0000 7fa2715b2700 1 -- 172.21.15.163:0/4126767048 learned_addr learned my addr 172.21.15.163:0/4126767048 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:35.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.078+0000 7fa270db1700 1 --2- 172.21.15.163:0/4126767048 >> v2:172.21.15.181:3300/0 conn(0x7fa26c159ec0 0x7fa26c0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:35.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.078+0000 7fa26bfff700 1 --2- 172.21.15.163:0/4126767048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa26c0b53d0 0x7fa26c0cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:35.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.078+0000 7fa2715b2700 1 -- 172.21.15.163:0/4126767048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa26c0b53d0 msgr2=0x7fa26c0cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:35.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.078+0000 7fa2715b2700 1 --2- 172.21.15.163:0/4126767048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa26c0b53d0 0x7fa26c0cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:35.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.078+0000 7fa2715b2700 1 -- 172.21.15.163:0/4126767048 >> v2:172.21.15.181:3300/0 conn(0x7fa26c159ec0 msgr2=0x7fa26c0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:35.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.078+0000 7fa2715b2700 1 --2- 172.21.15.163:0/4126767048 >> v2:172.21.15.181:3300/0 conn(0x7fa26c159ec0 0x7fa26c0b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:35.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.078+0000 7fa2715b2700 1 -- 172.21.15.163:0/4126767048 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa26c0b6080 con 0x7fa26c0cce40 2023-10-09T06:36:35.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.078+0000 7fa2715b2700 1 --2- 172.21.15.163:0/4126767048 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa26c0cce40 0x7fa26c0cd2a0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa26c089120 tx=0x7fa254009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a036f096ed014088 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:35.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.079+0000 7fa26b7fe700 1 -- 172.21.15.163:0/4126767048 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25400b860 con 0x7fa26c0cce40 2023-10-09T06:36:35.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.079+0000 7fa26b7fe700 1 -- 172.21.15.163:0/4126767048 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25400b9c0 con 0x7fa26c0cce40 2023-10-09T06:36:35.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.079+0000 7fa26b7fe700 1 -- 172.21.15.163:0/4126767048 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25400bd60 con 0x7fa26c0cce40 2023-10-09T06:36:35.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.079+0000 7fa273015700 1 -- 172.21.15.163:0/4126767048 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa26c0cce40 msgr2=0x7fa26c0cd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:35.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.079+0000 7fa273015700 1 --2- 172.21.15.163:0/4126767048 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa26c0cce40 0x7fa26c0cd2a0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa26c089120 tx=0x7fa254009250 comp rx=0 tx=0).stop 2023-10-09T06:36:35.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.079+0000 7fa273015700 1 -- 172.21.15.163:0/4126767048 shutdown_connections 2023-10-09T06:36:35.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.079+0000 7fa273015700 1 --2- 172.21.15.163:0/4126767048 >> v2:172.21.15.181:3300/0 conn(0x7fa26c159ec0 0x7fa26c0b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:35.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.079+0000 7fa273015700 1 --2- 172.21.15.163:0/4126767048 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa26c0cce40 0x7fa26c0cd2a0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:35.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.079+0000 7fa273015700 1 --2- 172.21.15.163:0/4126767048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa26c0b53d0 0x7fa26c0cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:35.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.079+0000 7fa273015700 1 -- 172.21.15.163:0/4126767048 >> 172.21.15.163:0/4126767048 conn(0x7fa26c0999f0 msgr2=0x7fa26c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:35.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.079+0000 7fa273015700 1 -- 172.21.15.163:0/4126767048 shutdown_connections 2023-10-09T06:36:35.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.079+0000 7fa273015700 1 -- 172.21.15.163:0/4126767048 wait complete. 2023-10-09T06:36:35.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.080+0000 7fa273015700 1 Processor -- start 2023-10-09T06:36:35.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.080+0000 7fa273015700 1 -- start start 2023-10-09T06:36:35.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.081+0000 7fa273015700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa26c0b53d0 0x7fa26c069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:35.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.081+0000 7fa273015700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa26c0cce40 0x7fa26c150160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:35.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.081+0000 7fa273015700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa26c159ec0 0x7fa26c0b3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:35.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.081+0000 7fa273015700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c1546b0 con 0x7fa26c159ec0 2023-10-09T06:36:35.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.081+0000 7fa273015700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c154990 con 0x7fa26c0cce40 2023-10-09T06:36:35.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.081+0000 7fa273015700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c154c70 con 0x7fa26c0b53d0 2023-10-09T06:36:35.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.081+0000 7fa2715b2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa26c159ec0 0x7fa26c0b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:35.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.081+0000 7fa2715b2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa26c159ec0 0x7fa26c0b3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59756/0 (socket says 172.21.15.163:59756) 2023-10-09T06:36:35.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.081+0000 7fa2715b2700 1 -- 172.21.15.163:0/4043805285 learned_addr learned my addr 172.21.15.163:0/4043805285 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:35.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.081+0000 7fa26bfff700 1 --2- 172.21.15.163:0/4043805285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa26c0cce40 0x7fa26c150160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:35.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.081+0000 7fa270db1700 1 --2- 172.21.15.163:0/4043805285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa26c0b53d0 0x7fa26c069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:35.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.082+0000 7fa2715b2700 1 -- 172.21.15.163:0/4043805285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa26c0b53d0 msgr2=0x7fa26c069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:35.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.082+0000 7fa2715b2700 1 --2- 172.21.15.163:0/4043805285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa26c0b53d0 0x7fa26c069d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:35.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.082+0000 7fa2715b2700 1 -- 172.21.15.163:0/4043805285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa26c0cce40 msgr2=0x7fa26c150160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:35.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.082+0000 7fa2715b2700 1 --2- 172.21.15.163:0/4043805285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa26c0cce40 0x7fa26c150160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:35.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.082+0000 7fa2715b2700 1 -- 172.21.15.163:0/4043805285 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2540094f0 con 0x7fa26c159ec0 2023-10-09T06:36:35.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.082+0000 7fa26bfff700 1 --2- 172.21.15.163:0/4043805285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa26c0cce40 0x7fa26c150160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:35.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.082+0000 7fa2715b2700 1 --2- 172.21.15.163:0/4043805285 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa26c159ec0 0x7fa26c0b3530 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fa26c089390 tx=0x7fa254017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:35.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.082+0000 7fa268ff9700 1 -- 172.21.15.163:0/4043805285 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25401a630 con 0x7fa26c159ec0 2023-10-09T06:36:35.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.082+0000 7fa268ff9700 1 -- 172.21.15.163:0/4043805285 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25401a7b0 con 0x7fa26c159ec0 2023-10-09T06:36:35.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.082+0000 7fa273015700 1 -- 172.21.15.163:0/4043805285 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa26c154f50 con 0x7fa26c159ec0 2023-10-09T06:36:35.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.082+0000 7fa268ff9700 1 -- 172.21.15.163:0/4043805285 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25401ab70 con 0x7fa26c159ec0 2023-10-09T06:36:35.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.082+0000 7fa273015700 1 -- 172.21.15.163:0/4043805285 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa26c06a260 con 0x7fa26c159ec0 2023-10-09T06:36:35.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.083+0000 7fa273015700 1 -- 172.21.15.163:0/4043805285 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa26c063c90 con 0x7fa26c159ec0 2023-10-09T06:36:35.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.083+0000 7fa268ff9700 1 -- 172.21.15.163:0/4043805285 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fa25402d440 con 0x7fa26c159ec0 2023-10-09T06:36:35.090 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.084+0000 7fa268ff9700 1 --2- 172.21.15.163:0/4043805285 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa24c067290 0x7fa24c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:35.090 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.084+0000 7fa268ff9700 1 -- 172.21.15.163:0/4043805285 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5353+0+0 (secure 0 0 0) 0x7fa2540865d0 con 0x7fa26c159ec0 2023-10-09T06:36:35.090 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.084+0000 7fa270db1700 1 --2- 172.21.15.163:0/4043805285 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa24c067290 0x7fa24c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:35.090 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.084+0000 7fa270db1700 1 --2- 172.21.15.163:0/4043805285 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa24c067290 0x7fa24c069740 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa264006aa0 tx=0x7fa264008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:35.090 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.088+0000 7fa268ff9700 1 -- 172.21.15.163:0/4043805285 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa254053610 con 0x7fa26c159ec0 2023-10-09T06:36:35.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.271+0000 7fa273015700 1 -- 172.21.15.163:0/4043805285 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa26c0718b0 con 0x7fa26c159ec0 2023-10-09T06:36:35.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.948+0000 7fa268ff9700 1 -- 172.21.15.163:0/4043805285 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa254056c30 con 0x7fa26c159ec0 2023-10-09T06:36:35.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.951+0000 7fa273015700 1 -- 172.21.15.163:0/4043805285 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa24c067290 msgr2=0x7fa24c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:35.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.951+0000 7fa273015700 1 --2- 172.21.15.163:0/4043805285 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa24c067290 0x7fa24c069740 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa264006aa0 tx=0x7fa264008040 comp rx=0 tx=0).stop 2023-10-09T06:36:35.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.951+0000 7fa273015700 1 -- 172.21.15.163:0/4043805285 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa26c159ec0 msgr2=0x7fa26c0b3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:35.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.951+0000 7fa273015700 1 --2- 172.21.15.163:0/4043805285 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa26c159ec0 0x7fa26c0b3530 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fa26c089390 tx=0x7fa254017000 comp rx=0 tx=0).stop 2023-10-09T06:36:35.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.951+0000 7fa273015700 1 -- 172.21.15.163:0/4043805285 shutdown_connections 2023-10-09T06:36:35.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.951+0000 7fa273015700 1 --2- 172.21.15.163:0/4043805285 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa24c067290 0x7fa24c069740 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:35.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.951+0000 7fa273015700 1 --2- 172.21.15.163:0/4043805285 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa26c159ec0 0x7fa26c0b3530 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:35.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.951+0000 7fa273015700 1 --2- 172.21.15.163:0/4043805285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa26c0b53d0 0x7fa26c069d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:35.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.951+0000 7fa273015700 1 --2- 172.21.15.163:0/4043805285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa26c0cce40 0x7fa26c150160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:35.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.951+0000 7fa273015700 1 -- 172.21.15.163:0/4043805285 >> 172.21.15.163:0/4043805285 conn(0x7fa26c0999f0 msgr2=0x7fa26c1520e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:35.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.951+0000 7fa273015700 1 -- 172.21.15.163:0/4043805285 shutdown_connections 2023-10-09T06:36:35.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:35.951+0000 7fa273015700 1 -- 172.21.15.163:0/4043805285 wait complete. 2023-10-09T06:36:35.953 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_data' removed 2023-10-09T06:36:35.965 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-09T06:36:36.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.082+0000 7f49b180a700 1 Processor -- start 2023-10-09T06:36:36.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.082+0000 7f49b180a700 1 -- start start 2023-10-09T06:36:36.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.082+0000 7f49b180a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49ac159890 0x7f49ac15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:36.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.082+0000 7f49b180a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f49ac069560 0x7f49ac0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:36.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.082+0000 7f49b180a700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49ac15a160 con 0x7f49ac159c60 2023-10-09T06:36:36.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.082+0000 7f49b180a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49ac069f20 con 0x7f49ac069560 2023-10-09T06:36:36.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.082+0000 7f49b180a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac0d13a0 con 0x7f49ac159890 2023-10-09T06:36:36.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.083+0000 7f49ab7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f49ac069560 0x7f49ac0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:36.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.083+0000 7f49aa7fc700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f49ac159c60 0x7f49ac15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53556/0 (socket says 172.21.15.163:53556) 2023-10-09T06:36:36.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.083+0000 7f49aa7fc700 1 -- 172.21.15.163:0/2289340047 learned_addr learned my addr 172.21.15.163:0/2289340047 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:36.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.083+0000 7f49aaffd700 1 --2- 172.21.15.163:0/2289340047 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49ac159890 0x7f49ac15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:36.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.083+0000 7f49a9ffb700 1 -- 172.21.15.163:0/2289340047 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2056665532 0 0) 0x7f49ac15a160 con 0x7f49ac159c60 2023-10-09T06:36:36.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.083+0000 7f49a9ffb700 1 -- 172.21.15.163:0/2289340047 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4994003570 con 0x7f49ac159c60 2023-10-09T06:36:36.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.083+0000 7f49ab7fe700 1 -- 172.21.15.163:0/2289340047 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49ac159890 msgr2=0x7f49ac15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:36.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.083+0000 7f49ab7fe700 1 --2- 172.21.15.163:0/2289340047 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49ac159890 0x7f49ac15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:36.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.083+0000 7f49ab7fe700 1 -- 172.21.15.163:0/2289340047 >> v1:172.21.15.163:6789/0 conn(0x7f49ac159c60 legacy=0x7f49ac15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:36.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.083+0000 7f49ab7fe700 1 -- 172.21.15.163:0/2289340047 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ac15e140 con 0x7f49ac069560 2023-10-09T06:36:36.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.083+0000 7f49ab7fe700 1 --2- 172.21.15.163:0/2289340047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49ac069560 0x7f49ac0699e0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f499c0098e0 tx=0x7f499c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09764ae844c2edd server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:36.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.083+0000 7f49a9ffb700 1 -- 172.21.15.163:0/2289340047 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c00b640 con 0x7f49ac069560 2023-10-09T06:36:36.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.083+0000 7f49a9ffb700 1 -- 172.21.15.163:0/2289340047 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499c00b7a0 con 0x7f49ac069560 2023-10-09T06:36:36.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.083+0000 7f49a9ffb700 1 -- 172.21.15.163:0/2289340047 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c00bb40 con 0x7f49ac069560 2023-10-09T06:36:36.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.084+0000 7f49b180a700 1 -- 172.21.15.163:0/2289340047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49ac069560 msgr2=0x7f49ac0699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:36.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.084+0000 7f49b180a700 1 --2- 172.21.15.163:0/2289340047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49ac069560 0x7f49ac0699e0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f499c0098e0 tx=0x7f499c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:36.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.084+0000 7f49b180a700 1 -- 172.21.15.163:0/2289340047 shutdown_connections 2023-10-09T06:36:36.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.084+0000 7f49b180a700 1 --2- 172.21.15.163:0/2289340047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49ac069560 0x7f49ac0699e0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:36.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.084+0000 7f49b180a700 1 --2- 172.21.15.163:0/2289340047 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49ac159890 0x7f49ac15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:36.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.084+0000 7f49b180a700 1 -- 172.21.15.163:0/2289340047 >> 172.21.15.163:0/2289340047 conn(0x7f49ac068e20 msgr2=0x7f49ac0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:36.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.084+0000 7f49b180a700 1 -- 172.21.15.163:0/2289340047 shutdown_connections 2023-10-09T06:36:36.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.084+0000 7f49b180a700 1 -- 172.21.15.163:0/2289340047 wait complete. 2023-10-09T06:36:36.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.085+0000 7f49b180a700 1 Processor -- start 2023-10-09T06:36:36.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.085+0000 7f49b180a700 1 -- start start 2023-10-09T06:36:36.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.086+0000 7f49b180a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49ac069560 0x7f49ac0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:36.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.086+0000 7f49b180a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49ac159890 0x7f49ac155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:36.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.086+0000 7f49b180a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49ac159c60 0x7f49ac15ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:36.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.086+0000 7f49b180a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac15e380 con 0x7f49ac159c60 2023-10-09T06:36:36.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.086+0000 7f49b180a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac0b3ad0 con 0x7f49ac159890 2023-10-09T06:36:36.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.086+0000 7f49b180a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac0b5ff0 con 0x7f49ac069560 2023-10-09T06:36:36.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.086+0000 7f49ab7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49ac159c60 0x7f49ac15ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:36.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.086+0000 7f49ab7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49ac159c60 0x7f49ac15ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59772/0 (socket says 172.21.15.163:59772) 2023-10-09T06:36:36.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.086+0000 7f49ab7fe700 1 -- 172.21.15.163:0/3303238428 learned_addr learned my addr 172.21.15.163:0/3303238428 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:36.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.086+0000 7f49aa7fc700 1 --2- 172.21.15.163:0/3303238428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49ac159890 0x7f49ac155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:36.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.086+0000 7f49aaffd700 1 --2- 172.21.15.163:0/3303238428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49ac069560 0x7f49ac0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:36.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.086+0000 7f49ab7fe700 1 -- 172.21.15.163:0/3303238428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49ac069560 msgr2=0x7f49ac0b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:36.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.086+0000 7f49ab7fe700 1 --2- 172.21.15.163:0/3303238428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49ac069560 0x7f49ac0b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:36.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.086+0000 7f49ab7fe700 1 -- 172.21.15.163:0/3303238428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49ac159890 msgr2=0x7f49ac155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:36.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.086+0000 7f49ab7fe700 1 --2- 172.21.15.163:0/3303238428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49ac159890 0x7f49ac155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:36.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.086+0000 7f49ab7fe700 1 -- 172.21.15.163:0/3303238428 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499c009420 con 0x7f49ac159c60 2023-10-09T06:36:36.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.086+0000 7f49aaffd700 1 --2- 172.21.15.163:0/3303238428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49ac069560 0x7f49ac0b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:36.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.086+0000 7f49ab7fe700 1 --2- 172.21.15.163:0/3303238428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49ac159c60 0x7f49ac15ed90 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f49ac0899c0 tx=0x7f499c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:36.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.086+0000 7f4992ffd700 1 -- 172.21.15.163:0/3303238428 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c011420 con 0x7f49ac159c60 2023-10-09T06:36:36.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.087+0000 7f4992ffd700 1 -- 172.21.15.163:0/3303238428 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499c011580 con 0x7f49ac159c60 2023-10-09T06:36:36.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.087+0000 7f49b180a700 1 -- 172.21.15.163:0/3303238428 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49ac0b62d0 con 0x7f49ac159c60 2023-10-09T06:36:36.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.087+0000 7f4992ffd700 1 -- 172.21.15.163:0/3303238428 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c011940 con 0x7f49ac159c60 2023-10-09T06:36:36.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.087+0000 7f49b180a700 1 -- 172.21.15.163:0/3303238428 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49ac15e870 con 0x7f49ac159c60 2023-10-09T06:36:36.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.087+0000 7f49b180a700 1 -- 172.21.15.163:0/3303238428 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49ac0b01a0 con 0x7f49ac159c60 2023-10-09T06:36:36.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.088+0000 7f4992ffd700 1 -- 172.21.15.163:0/3303238428 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f499c011cb0 con 0x7f49ac159c60 2023-10-09T06:36:36.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.088+0000 7f4992ffd700 1 --2- 172.21.15.163:0/3303238428 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4998067290 0x7f4998069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:36.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.088+0000 7f4992ffd700 1 -- 172.21.15.163:0/3303238428 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f499c086360 con 0x7f49ac159c60 2023-10-09T06:36:36.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.088+0000 7f49aaffd700 1 --2- 172.21.15.163:0/3303238428 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4998067290 0x7f4998069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:36.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.089+0000 7f49aaffd700 1 --2- 172.21.15.163:0/3303238428 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4998067290 0x7f4998069740 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f49a4006fd0 tx=0x7f49a4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:36.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.092+0000 7f4992ffd700 1 -- 172.21.15.163:0/3303238428 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f499c0534c0 con 0x7f49ac159c60 2023-10-09T06:36:36.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.275+0000 7f49b180a700 1 -- 172.21.15.163:0/3303238428 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f49ac04c3c0 con 0x7f49ac159c60 2023-10-09T06:36:36.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.279+0000 7f4992ffd700 1 -- 172.21.15.163:0/3303238428 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f499c056ae0 con 0x7f49ac159c60 2023-10-09T06:36:36.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.281+0000 7f49b180a700 1 -- 172.21.15.163:0/3303238428 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4998067290 msgr2=0x7f4998069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:36.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.281+0000 7f49b180a700 1 --2- 172.21.15.163:0/3303238428 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4998067290 0x7f4998069740 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f49a4006fd0 tx=0x7f49a4008040 comp rx=0 tx=0).stop 2023-10-09T06:36:36.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.281+0000 7f49b180a700 1 -- 172.21.15.163:0/3303238428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49ac159c60 msgr2=0x7f49ac15ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:36.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.281+0000 7f49b180a700 1 --2- 172.21.15.163:0/3303238428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49ac159c60 0x7f49ac15ed90 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f49ac0899c0 tx=0x7f499c017000 comp rx=0 tx=0).stop 2023-10-09T06:36:36.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.281+0000 7f49b180a700 1 -- 172.21.15.163:0/3303238428 shutdown_connections 2023-10-09T06:36:36.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.281+0000 7f49b180a700 1 --2- 172.21.15.163:0/3303238428 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4998067290 0x7f4998069740 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:36.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.281+0000 7f49b180a700 1 --2- 172.21.15.163:0/3303238428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49ac159c60 0x7f49ac15ed90 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:36.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.281+0000 7f49b180a700 1 --2- 172.21.15.163:0/3303238428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49ac069560 0x7f49ac0b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:36.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.281+0000 7f49b180a700 1 --2- 172.21.15.163:0/3303238428 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49ac159890 0x7f49ac155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:36.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.281+0000 7f49b180a700 1 -- 172.21.15.163:0/3303238428 >> 172.21.15.163:0/3303238428 conn(0x7f49ac068e20 msgr2=0x7f49ac0c5590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:36.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.281+0000 7f49b180a700 1 -- 172.21.15.163:0/3303238428 shutdown_connections 2023-10-09T06:36:36.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.281+0000 7f49b180a700 1 -- 172.21.15.163:0/3303238428 wait complete. 2023-10-09T06:36:36.294 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup' 2023-10-09T06:36:36.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.408+0000 7fc783d3f700 1 Processor -- start 2023-10-09T06:36:36.409 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.408+0000 7fc783d3f700 1 -- start start 2023-10-09T06:36:36.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.408+0000 7fc783d3f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc77c159890 0x7fc77c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:36.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.409+0000 7fc783d3f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc77c15a160 con 0x7fc77c159c60 2023-10-09T06:36:36.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.409+0000 7fc783d3f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc77c0d13a0 con 0x7fc77c069560 2023-10-09T06:36:36.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.409+0000 7fc783d3f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c15e140 con 0x7fc77c159890 2023-10-09T06:36:36.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.409+0000 7fc7812da700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fc77c159c60 0x7fc77c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53566/0 (socket says 172.21.15.163:53566) 2023-10-09T06:36:36.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.409+0000 7fc7812da700 1 -- 172.21.15.163:0/1629193994 learned_addr learned my addr 172.21.15.163:0/1629193994 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:36.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.409+0000 7fc781adb700 1 --2- 172.21.15.163:0/1629193994 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc77c159890 0x7fc77c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:36.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.409+0000 7fc780ad9700 1 -- 172.21.15.163:0/1629193994 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4112333110 0 0) 0x7fc77c15a160 con 0x7fc77c159c60 2023-10-09T06:36:36.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.409+0000 7fc780ad9700 1 -- 172.21.15.163:0/1629193994 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc768003570 con 0x7fc77c159c60 2023-10-09T06:36:36.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.409+0000 7fc780ad9700 1 -- 172.21.15.163:0/1629193994 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2273881500 0 0) 0x7fc768003570 con 0x7fc77c159c60 2023-10-09T06:36:36.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.409+0000 7fc780ad9700 1 -- 172.21.15.163:0/1629193994 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc77c159890 msgr2=0x7fc77c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:36.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.409+0000 7fc780ad9700 1 --2- 172.21.15.163:0/1629193994 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc77c159890 0x7fc77c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:36.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.409+0000 7fc780ad9700 1 -- 172.21.15.163:0/1629193994 >> v1:172.21.15.181:6789/0 conn(0x7fc77c069560 legacy=0x7fc77c0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:36.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.409+0000 7fc780ad9700 1 -- 172.21.15.163:0/1629193994 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77c15a3d0 con 0x7fc77c159c60 2023-10-09T06:36:36.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.409+0000 7fc780ad9700 1 -- 172.21.15.163:0/1629193994 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fc76c001470 con 0x7fc77c159c60 2023-10-09T06:36:36.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.409+0000 7fc780ad9700 1 -- 172.21.15.163:0/1629193994 >> v1:172.21.15.163:6789/0 conn(0x7fc77c159c60 legacy=0x7fc77c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:36.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.409+0000 7fc780ad9700 1 --2- 172.21.15.163:0/1629193994 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc77c159890 0x7fc768003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:36.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.409+0000 7fc780ad9700 1 --2- 172.21.15.163:0/1629193994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc768003fd0 0x7fc768004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:36.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.409+0000 7fc780ad9700 1 --2- 172.21.15.163:0/1629193994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc768004d50 0x7fc7680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:36.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.409+0000 7fc780ad9700 1 -- 172.21.15.163:0/1629193994 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c15e140 con 0x7fc768003fd0 2023-10-09T06:36:36.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.409+0000 7fc780ad9700 1 -- 172.21.15.163:0/1629193994 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76800bb40 con 0x7fc768004d50 2023-10-09T06:36:36.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.409+0000 7fc780ad9700 1 -- 172.21.15.163:0/1629193994 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc76800bd00 con 0x7fc77c159890 2023-10-09T06:36:36.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.409+0000 7fc7812da700 1 --2- 172.21.15.163:0/1629193994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc768003fd0 0x7fc768004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:36.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.409+0000 7fc781adb700 1 --2- 172.21.15.163:0/1629193994 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc77c159890 0x7fc768003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:36.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.409+0000 7fc7822dc700 1 --2- 172.21.15.163:0/1629193994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc768004d50 0x7fc7680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:36.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.410+0000 7fc7812da700 1 -- 172.21.15.163:0/1629193994 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc77c159890 msgr2=0x7fc768003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:36.416 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.410+0000 7fc7812da700 1 --2- 172.21.15.163:0/1629193994 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc77c159890 0x7fc768003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:36.416 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.410+0000 7fc7812da700 1 -- 172.21.15.163:0/1629193994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc768004d50 msgr2=0x7fc7680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:36.416 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.410+0000 7fc7812da700 1 --2- 172.21.15.163:0/1629193994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc768004d50 0x7fc7680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:36.416 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.410+0000 7fc7812da700 1 -- 172.21.15.163:0/1629193994 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc768003570 con 0x7fc768003fd0 2023-10-09T06:36:36.416 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.410+0000 7fc7812da700 1 --2- 172.21.15.163:0/1629193994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc768003fd0 0x7fc768004810 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00c390 tx=0x7fc76c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6607b513efe544af server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:36.416 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.410+0000 7fc780ad9700 1 -- 172.21.15.163:0/1629193994 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76c00cc60 con 0x7fc768003fd0 2023-10-09T06:36:36.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.410+0000 7fc780ad9700 1 -- 172.21.15.163:0/1629193994 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc76c00cdc0 con 0x7fc768003fd0 2023-10-09T06:36:36.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.410+0000 7fc780ad9700 1 -- 172.21.15.163:0/1629193994 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76c005b50 con 0x7fc768003fd0 2023-10-09T06:36:36.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.410+0000 7fc783d3f700 1 -- 172.21.15.163:0/1629193994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc768003fd0 msgr2=0x7fc768004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:36.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.410+0000 7fc783d3f700 1 --2- 172.21.15.163:0/1629193994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc768003fd0 0x7fc768004810 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00c390 tx=0x7fc76c001e70 comp rx=0 tx=0).stop 2023-10-09T06:36:36.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.410+0000 7fc7822dc700 1 -- 172.21.15.163:0/1629193994 reap_dead start 2023-10-09T06:36:36.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.410+0000 7fc783d3f700 1 -- 172.21.15.163:0/1629193994 shutdown_connections 2023-10-09T06:36:36.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.410+0000 7fc783d3f700 1 -- 172.21.15.163:0/1629193994 >> 172.21.15.163:0/1629193994 conn(0x7fc77c068e20 msgr2=0x7fc77c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:36.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.411+0000 7fc783d3f700 1 -- 172.21.15.163:0/1629193994 shutdown_connections 2023-10-09T06:36:36.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.411+0000 7fc783d3f700 1 -- 172.21.15.163:0/1629193994 wait complete. 2023-10-09T06:36:36.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.412+0000 7fc783d3f700 1 Processor -- start 2023-10-09T06:36:36.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.412+0000 7fc783d3f700 1 -- start start 2023-10-09T06:36:36.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.412+0000 7fc783d3f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc768003fd0 0x7fc77c144150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:36.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.412+0000 7fc783d3f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc77c0ce740 0x7fc77c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:36.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.412+0000 7fc783d3f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc77c155d90 0x7fc77c156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:36.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.412+0000 7fc783d3f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c0b2520 con 0x7fc77c155d90 2023-10-09T06:36:36.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.412+0000 7fc783d3f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c0cfe50 con 0x7fc77c0ce740 2023-10-09T06:36:36.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.412+0000 7fc783d3f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c0cf520 con 0x7fc768003fd0 2023-10-09T06:36:36.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.412+0000 7fc7822dc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc77c155d90 0x7fc77c156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:36.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.412+0000 7fc7822dc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc77c155d90 0x7fc77c156200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59800/0 (socket says 172.21.15.163:59800) 2023-10-09T06:36:36.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.413+0000 7fc7822dc700 1 -- 172.21.15.163:0/999322447 learned_addr learned my addr 172.21.15.163:0/999322447 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:36.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.413+0000 7fc781adb700 1 --2- 172.21.15.163:0/999322447 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc768003fd0 0x7fc77c144150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:36.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.413+0000 7fc7812da700 1 --2- 172.21.15.163:0/999322447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc77c0ce740 0x7fc77c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:36.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.413+0000 7fc7822dc700 1 -- 172.21.15.163:0/999322447 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc768003fd0 msgr2=0x7fc77c144150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:36.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.413+0000 7fc7822dc700 1 --2- 172.21.15.163:0/999322447 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc768003fd0 0x7fc77c144150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:36.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.413+0000 7fc7822dc700 1 -- 172.21.15.163:0/999322447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc77c0ce740 msgr2=0x7fc77c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:36.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.413+0000 7fc7822dc700 1 --2- 172.21.15.163:0/999322447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc77c0ce740 0x7fc77c0cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:36.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.413+0000 7fc7822dc700 1 -- 172.21.15.163:0/999322447 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77c0cf800 con 0x7fc77c155d90 2023-10-09T06:36:36.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.413+0000 7fc7822dc700 1 --2- 172.21.15.163:0/999322447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc77c155d90 0x7fc77c156200 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0d03b0 tx=0x7fc774007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:36.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.413+0000 7fc771ffb700 1 -- 172.21.15.163:0/999322447 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77400f8a0 con 0x7fc77c155d90 2023-10-09T06:36:36.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.413+0000 7fc771ffb700 1 -- 172.21.15.163:0/999322447 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77400fa00 con 0x7fc77c155d90 2023-10-09T06:36:36.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.413+0000 7fc771ffb700 1 -- 172.21.15.163:0/999322447 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77400fdc0 con 0x7fc77c155d90 2023-10-09T06:36:36.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.413+0000 7fc783d3f700 1 -- 172.21.15.163:0/999322447 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc77c0cfae0 con 0x7fc77c155d90 2023-10-09T06:36:36.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.413+0000 7fc783d3f700 1 -- 172.21.15.163:0/999322447 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77c151b20 con 0x7fc77c155d90 2023-10-09T06:36:36.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.414+0000 7fc783d3f700 1 -- 172.21.15.163:0/999322447 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc77c04c3c0 con 0x7fc77c155d90 2023-10-09T06:36:36.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.414+0000 7fc771ffb700 1 -- 172.21.15.163:0/999322447 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fc774013600 con 0x7fc77c155d90 2023-10-09T06:36:36.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.415+0000 7fc771ffb700 1 --2- 172.21.15.163:0/999322447 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc760067290 0x7fc760069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:36.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.415+0000 7fc771ffb700 1 -- 172.21.15.163:0/999322447 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fc7740847a0 con 0x7fc77c155d90 2023-10-09T06:36:36.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.415+0000 7fc781adb700 1 --2- 172.21.15.163:0/999322447 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc760067290 0x7fc760069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:36.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.415+0000 7fc781adb700 1 --2- 172.21.15.163:0/999322447 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc760067290 0x7fc760069740 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc778009fd0 tx=0x7fc77800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:36.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.419+0000 7fc771ffb700 1 -- 172.21.15.163:0/999322447 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc774051900 con 0x7fc77c155d90 2023-10-09T06:36:36.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.686+0000 7fc783d3f700 1 -- 172.21.15.163:0/999322447 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup"]} v 0) v1 -- 0x7fc77c0c6cc0 con 0x7fc77c155d90 2023-10-09T06:36:36.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.949+0000 7fc771ffb700 1 -- 172.21.15.163:0/999322447 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup"]}]=0 v289) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc774054f20 con 0x7fc77c155d90 2023-10-09T06:36:36.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.952+0000 7fc783d3f700 1 -- 172.21.15.163:0/999322447 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc760067290 msgr2=0x7fc760069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:36.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.952+0000 7fc783d3f700 1 --2- 172.21.15.163:0/999322447 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc760067290 0x7fc760069740 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc778009fd0 tx=0x7fc77800b040 comp rx=0 tx=0).stop 2023-10-09T06:36:36.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.952+0000 7fc783d3f700 1 -- 172.21.15.163:0/999322447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc77c155d90 msgr2=0x7fc77c156200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:36.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.952+0000 7fc783d3f700 1 --2- 172.21.15.163:0/999322447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc77c155d90 0x7fc77c156200 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0d03b0 tx=0x7fc774007d10 comp rx=0 tx=0).stop 2023-10-09T06:36:36.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.952+0000 7fc783d3f700 1 -- 172.21.15.163:0/999322447 shutdown_connections 2023-10-09T06:36:36.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.952+0000 7fc783d3f700 1 --2- 172.21.15.163:0/999322447 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc760067290 0x7fc760069740 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:36.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.952+0000 7fc783d3f700 1 --2- 172.21.15.163:0/999322447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc77c155d90 0x7fc77c156200 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:36.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.952+0000 7fc783d3f700 1 --2- 172.21.15.163:0/999322447 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc768003fd0 0x7fc77c144150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:36.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.952+0000 7fc783d3f700 1 --2- 172.21.15.163:0/999322447 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc77c0ce740 0x7fc77c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:36.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.952+0000 7fc783d3f700 1 -- 172.21.15.163:0/999322447 >> 172.21.15.163:0/999322447 conn(0x7fc77c068e20 msgr2=0x7fc77c15c570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:36.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.952+0000 7fc783d3f700 1 -- 172.21.15.163:0/999322447 shutdown_connections 2023-10-09T06:36:36.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:36.952+0000 7fc783d3f700 1 -- 172.21.15.163:0/999322447 wait complete. 2023-10-09T06:36:36.965 INFO:tasks.cephfs_test_runner:Tests the 'fs volume info --human_readable' command without subvolume group ... ok 2023-10-09T06:36:36.965 INFO:tasks.cephfs_test_runner:Starting test: test_volume_info_without_subvolumegroup (tasks.cephfs.test_volumes.TestVolumes) Tests the 'fs volume info' command without subvolume group 2023-10-09T06:36:36.966 INFO:tasks.cephfs_test_runner:test_volume_info_without_subvolumegroup (tasks.cephfs.test_volumes.TestVolumes) 2023-10-09T06:36:36.966 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup' 2023-10-09T06:36:37.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.082+0000 7f5216e4d700 1 Processor -- start 2023-10-09T06:36:37.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.082+0000 7f5216e4d700 1 -- start start 2023-10-09T06:36:37.090 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.082+0000 7f5216e4d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5210156690 0x7f52100ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:37.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.082+0000 7f5216e4d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5210156a60 0x7f52100694a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:37.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.082+0000 7f5216e4d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5210157e30 con 0x7f52100af480 2023-10-09T06:36:37.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.082+0000 7f5216e4d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f521006aa00 con 0x7f5210156a60 2023-10-09T06:36:37.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.082+0000 7f5216e4d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52100699e0 con 0x7f5210156690 2023-10-09T06:36:37.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.083+0000 7f5207fff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f52100af480 0x7f52100c86c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53582/0 (socket says 172.21.15.163:53582) 2023-10-09T06:36:37.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.083+0000 7f5207fff700 1 -- 172.21.15.163:0/163183403 learned_addr learned my addr 172.21.15.163:0/163183403 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:37.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.083+0000 7f52077fe700 1 -- 172.21.15.163:0/163183403 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1618683561 0 0) 0x7f5210157e30 con 0x7f52100af480 2023-10-09T06:36:37.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.083+0000 7f52077fe700 1 -- 172.21.15.163:0/163183403 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51f8003570 con 0x7f52100af480 2023-10-09T06:36:37.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.083+0000 7f5214be9700 1 --2- 172.21.15.163:0/163183403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5210156690 0x7f52100ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:37.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.083+0000 7f52153ea700 1 --2- 172.21.15.163:0/163183403 >> v2:172.21.15.181:3300/0 conn(0x7f5210156a60 0x7f52100694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:37.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.083+0000 7f52077fe700 1 -- 172.21.15.163:0/163183403 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1533648223 0 0) 0x7f51f8003570 con 0x7f52100af480 2023-10-09T06:36:37.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.083+0000 7f52077fe700 1 -- 172.21.15.163:0/163183403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5210156690 msgr2=0x7f52100ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:37.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.083+0000 7f52077fe700 1 --2- 172.21.15.163:0/163183403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5210156690 0x7f52100ca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:37.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.083+0000 7f52077fe700 1 -- 172.21.15.163:0/163183403 >> v2:172.21.15.181:3300/0 conn(0x7f5210156a60 msgr2=0x7f52100694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:37.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.083+0000 7f52077fe700 1 --2- 172.21.15.163:0/163183403 >> v2:172.21.15.181:3300/0 conn(0x7f5210156a60 0x7f52100694a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:37.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.083+0000 7f52077fe700 1 -- 172.21.15.163:0/163183403 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52101578b0 con 0x7f52100af480 2023-10-09T06:36:37.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.083+0000 7f52077fe700 1 -- 172.21.15.163:0/163183403 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f5200001470 con 0x7f52100af480 2023-10-09T06:36:37.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.083+0000 7f52077fe700 1 -- 172.21.15.163:0/163183403 >> v1:172.21.15.163:6789/0 conn(0x7f52100af480 legacy=0x7f52100c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:37.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.083+0000 7f52077fe700 1 --2- 172.21.15.163:0/163183403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5210156690 0x7f51f8003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:37.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.083+0000 7f52077fe700 1 --2- 172.21.15.163:0/163183403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51f8004090 0x7f51f8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:37.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.083+0000 7f52077fe700 1 --2- 172.21.15.163:0/163183403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51f8004d50 0x7f51f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:37.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.083+0000 7f52077fe700 1 -- 172.21.15.163:0/163183403 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521006aa00 con 0x7f51f8004d50 2023-10-09T06:36:37.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.083+0000 7f52077fe700 1 -- 172.21.15.163:0/163183403 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52100699e0 con 0x7f51f8004090 2023-10-09T06:36:37.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.083+0000 7f52077fe700 1 -- 172.21.15.163:0/163183403 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f800bca0 con 0x7f5210156690 2023-10-09T06:36:37.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.083+0000 7f52153ea700 1 --2- 172.21.15.163:0/163183403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51f8004d50 0x7f51f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:37.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.083+0000 7f5214be9700 1 --2- 172.21.15.163:0/163183403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5210156690 0x7f51f8003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:37.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.083+0000 7f5207fff700 1 --2- 172.21.15.163:0/163183403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51f8004090 0x7f51f8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:37.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.084+0000 7f52153ea700 1 -- 172.21.15.163:0/163183403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5210156690 msgr2=0x7f51f8003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:37.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.084+0000 7f52153ea700 1 --2- 172.21.15.163:0/163183403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5210156690 0x7f51f8003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:37.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.084+0000 7f52153ea700 1 -- 172.21.15.163:0/163183403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51f8004090 msgr2=0x7f51f8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:37.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.084+0000 7f52153ea700 1 --2- 172.21.15.163:0/163183403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51f8004090 0x7f51f8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:37.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.084+0000 7f52153ea700 1 -- 172.21.15.163:0/163183403 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f8003570 con 0x7f51f8004d50 2023-10-09T06:36:37.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.084+0000 7f5207fff700 1 --2- 172.21.15.163:0/163183403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51f8004090 0x7f51f8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:37.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.084+0000 7f52153ea700 1 --2- 172.21.15.163:0/163183403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51f8004d50 0x7f51f80051c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f520800b5f0 tx=0x7f5208008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9106010dfe8cce server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:37.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.084+0000 7f52077fe700 1 -- 172.21.15.163:0/163183403 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520800d5d0 con 0x7f51f8004d50 2023-10-09T06:36:37.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.084+0000 7f52077fe700 1 -- 172.21.15.163:0/163183403 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520800d730 con 0x7f51f8004d50 2023-10-09T06:36:37.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.084+0000 7f52077fe700 1 -- 172.21.15.163:0/163183403 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520800daf0 con 0x7f51f8004d50 2023-10-09T06:36:37.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.084+0000 7f5216e4d700 1 -- 172.21.15.163:0/163183403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51f8004d50 msgr2=0x7f51f80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:37.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.084+0000 7f5216e4d700 1 --2- 172.21.15.163:0/163183403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51f8004d50 0x7f51f80051c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f520800b5f0 tx=0x7f5208008040 comp rx=0 tx=0).stop 2023-10-09T06:36:37.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.084+0000 7f52153ea700 1 -- 172.21.15.163:0/163183403 reap_dead start 2023-10-09T06:36:37.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.085+0000 7f5216e4d700 1 -- 172.21.15.163:0/163183403 shutdown_connections 2023-10-09T06:36:37.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.085+0000 7f5216e4d700 1 -- 172.21.15.163:0/163183403 >> 172.21.15.163:0/163183403 conn(0x7f52100c9870 msgr2=0x7f52100c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:37.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.085+0000 7f5216e4d700 1 -- 172.21.15.163:0/163183403 shutdown_connections 2023-10-09T06:36:37.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.085+0000 7f5216e4d700 1 -- 172.21.15.163:0/163183403 wait complete. 2023-10-09T06:36:37.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.086+0000 7f5216e4d700 1 Processor -- start 2023-10-09T06:36:37.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.086+0000 7f5216e4d700 1 -- start start 2023-10-09T06:36:37.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.087+0000 7f5216e4d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52100adec0 0x7f5210153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:37.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.087+0000 7f5216e4d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52100c86c0 0x7f52100c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:37.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.087+0000 7f5216e4d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52101516a0 0x7f5210151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:37.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.087+0000 7f5216e4d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5210152f50 con 0x7f52101516a0 2023-10-09T06:36:37.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.087+0000 7f5216e4d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52100b0f10 con 0x7f52100adec0 2023-10-09T06:36:37.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.087+0000 7f5216e4d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52101538b0 con 0x7f52100c86c0 2023-10-09T06:36:37.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.087+0000 7f52153ea700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52101516a0 0x7f5210151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:37.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.087+0000 7f52153ea700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52101516a0 0x7f5210151b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59816/0 (socket says 172.21.15.163:59816) 2023-10-09T06:36:37.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.087+0000 7f52153ea700 1 -- 172.21.15.163:0/3178607189 learned_addr learned my addr 172.21.15.163:0/3178607189 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:37.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.087+0000 7f5214be9700 1 --2- 172.21.15.163:0/3178607189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52100adec0 0x7f5210153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:37.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.087+0000 7f5207fff700 1 --2- 172.21.15.163:0/3178607189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52100c86c0 0x7f52100c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:37.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.087+0000 7f52153ea700 1 -- 172.21.15.163:0/3178607189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52100c86c0 msgr2=0x7f52100c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:37.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.087+0000 7f52153ea700 1 --2- 172.21.15.163:0/3178607189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52100c86c0 0x7f52100c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:37.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.087+0000 7f52153ea700 1 -- 172.21.15.163:0/3178607189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52100adec0 msgr2=0x7f5210153370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:37.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.087+0000 7f52153ea700 1 --2- 172.21.15.163:0/3178607189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52100adec0 0x7f5210153370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:37.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.087+0000 7f52153ea700 1 -- 172.21.15.163:0/3178607189 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52100c9070 con 0x7f52101516a0 2023-10-09T06:36:37.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.087+0000 7f5207fff700 1 --2- 172.21.15.163:0/3178607189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52100c86c0 0x7f52100c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:37.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.087+0000 7f52153ea700 1 --2- 172.21.15.163:0/3178607189 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52101516a0 0x7f5210151b10 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f5208006fd0 tx=0x7f520801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:37.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.087+0000 7f5204ff9700 1 -- 172.21.15.163:0/3178607189 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520800a070 con 0x7f52101516a0 2023-10-09T06:36:37.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.087+0000 7f5204ff9700 1 -- 172.21.15.163:0/3178607189 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520800dab0 con 0x7f52101516a0 2023-10-09T06:36:37.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.087+0000 7f5216e4d700 1 -- 172.21.15.163:0/3178607189 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5210152050 con 0x7f52101516a0 2023-10-09T06:36:37.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.087+0000 7f5204ff9700 1 -- 172.21.15.163:0/3178607189 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520800d5d0 con 0x7f52101516a0 2023-10-09T06:36:37.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.087+0000 7f5216e4d700 1 -- 172.21.15.163:0/3178607189 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52101525b0 con 0x7f52101516a0 2023-10-09T06:36:37.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.088+0000 7f5216e4d700 1 -- 172.21.15.163:0/3178607189 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5210153090 con 0x7f52101516a0 2023-10-09T06:36:37.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.089+0000 7f5204ff9700 1 -- 172.21.15.163:0/3178607189 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f520801b8f0 con 0x7f52101516a0 2023-10-09T06:36:37.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.089+0000 7f5204ff9700 1 --2- 172.21.15.163:0/3178607189 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f51fc067290 0x7f51fc069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:37.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.089+0000 7f5204ff9700 1 -- 172.21.15.163:0/3178607189 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f5208000e40 con 0x7f52101516a0 2023-10-09T06:36:37.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.089+0000 7f5214be9700 1 --2- 172.21.15.163:0/3178607189 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f51fc067290 0x7f51fc069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:37.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.090+0000 7f5214be9700 1 --2- 172.21.15.163:0/3178607189 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f51fc067290 0x7f51fc069740 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f520c0076e0 tx=0x7f520c0096f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:37.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.093+0000 7f5204ff9700 1 -- 172.21.15.163:0/3178607189 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5210153090 con 0x7f52101516a0 2023-10-09T06:36:37.348 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.347+0000 7f5216e4d700 1 -- 172.21.15.163:0/3178607189 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup"]} v 0) v1 -- 0x7f5210153090 con 0x7f52101516a0 2023-10-09T06:36:37.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.951+0000 7f5204ff9700 1 -- 172.21.15.163:0/3178607189 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup"]}]=0 v290) v1 ==== 159+0+0 (secure 0 0 0) 0x7f5210153090 con 0x7f52101516a0 2023-10-09T06:36:37.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.953+0000 7f5216e4d700 1 -- 172.21.15.163:0/3178607189 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f51fc067290 msgr2=0x7f51fc069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:37.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.953+0000 7f5216e4d700 1 --2- 172.21.15.163:0/3178607189 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f51fc067290 0x7f51fc069740 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f520c0076e0 tx=0x7f520c0096f0 comp rx=0 tx=0).stop 2023-10-09T06:36:37.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.953+0000 7f5216e4d700 1 -- 172.21.15.163:0/3178607189 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52101516a0 msgr2=0x7f5210151b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:37.954 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.953+0000 7f5216e4d700 1 --2- 172.21.15.163:0/3178607189 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52101516a0 0x7f5210151b10 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f5208006fd0 tx=0x7f520801d000 comp rx=0 tx=0).stop 2023-10-09T06:36:37.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.953+0000 7f5216e4d700 1 -- 172.21.15.163:0/3178607189 shutdown_connections 2023-10-09T06:36:37.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.953+0000 7f5216e4d700 1 --2- 172.21.15.163:0/3178607189 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f51fc067290 0x7f51fc069740 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:37.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.953+0000 7f5216e4d700 1 --2- 172.21.15.163:0/3178607189 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52101516a0 0x7f5210151b10 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:37.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.953+0000 7f5216e4d700 1 --2- 172.21.15.163:0/3178607189 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52100adec0 0x7f5210153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:37.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.953+0000 7f5216e4d700 1 --2- 172.21.15.163:0/3178607189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52100c86c0 0x7f52100c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:37.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.953+0000 7f5216e4d700 1 -- 172.21.15.163:0/3178607189 >> 172.21.15.163:0/3178607189 conn(0x7f52100c9870 msgr2=0x7f521014a150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:37.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.953+0000 7f5216e4d700 1 -- 172.21.15.163:0/3178607189 shutdown_connections 2023-10-09T06:36:37.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:37.953+0000 7f5216e4d700 1 -- 172.21.15.163:0/3178607189 wait complete. 2023-10-09T06:36:37.968 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-09T06:36:38.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.087+0000 7ff28766a700 1 Processor -- start 2023-10-09T06:36:38.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.088+0000 7ff28766a700 1 -- start start 2023-10-09T06:36:38.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.088+0000 7ff28766a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff28006bcd0 0x7ff280140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:38.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.088+0000 7ff28766a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff28006c0a0 0x7ff280141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:38.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.088+0000 7ff28766a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff2800c8e00 0x7ff2800c9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:38.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.088+0000 7ff28766a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff28015c2a0 con 0x7ff28006c0a0 2023-10-09T06:36:38.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.088+0000 7ff28766a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff28015b690 con 0x7ff2800c8e00 2023-10-09T06:36:38.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.088+0000 7ff28766a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28015b970 con 0x7ff28006bcd0 2023-10-09T06:36:38.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.089+0000 7ff284c05700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff28006c0a0 0x7ff280141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:38.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.089+0000 7ff284c05700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff28006c0a0 0x7ff280141330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59832/0 (socket says 172.21.15.163:59832) 2023-10-09T06:36:38.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.089+0000 7ff284c05700 1 -- 172.21.15.163:0/3079498637 learned_addr learned my addr 172.21.15.163:0/3079498637 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:38.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.089+0000 7ff285c07700 1 --2- 172.21.15.163:0/3079498637 >> v2:172.21.15.181:3300/0 conn(0x7ff2800c8e00 0x7ff2800c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:38.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.089+0000 7ff285406700 1 --2- 172.21.15.163:0/3079498637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff28006bcd0 0x7ff280140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:38.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.089+0000 7ff284c05700 1 -- 172.21.15.163:0/3079498637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff28006bcd0 msgr2=0x7ff280140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:38.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.089+0000 7ff284c05700 1 --2- 172.21.15.163:0/3079498637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff28006bcd0 0x7ff280140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:38.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.089+0000 7ff284c05700 1 -- 172.21.15.163:0/3079498637 >> v2:172.21.15.181:3300/0 conn(0x7ff2800c8e00 msgr2=0x7ff2800c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:38.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.089+0000 7ff284c05700 1 --2- 172.21.15.163:0/3079498637 >> v2:172.21.15.181:3300/0 conn(0x7ff2800c8e00 0x7ff2800c9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:38.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.089+0000 7ff284c05700 1 -- 172.21.15.163:0/3079498637 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff28015bc50 con 0x7ff28006c0a0 2023-10-09T06:36:38.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.089+0000 7ff285c07700 1 --2- 172.21.15.163:0/3079498637 >> v2:172.21.15.181:3300/0 conn(0x7ff2800c8e00 0x7ff2800c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:38.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.089+0000 7ff285406700 1 --2- 172.21.15.163:0/3079498637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff28006bcd0 0x7ff280140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:38.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.089+0000 7ff284c05700 1 --2- 172.21.15.163:0/3079498637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff28006c0a0 0x7ff280141330 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ff28015b530 tx=0x7ff27c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd1feb8000e3490 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:38.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.089+0000 7ff277fff700 1 -- 172.21.15.163:0/3079498637 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c00b860 con 0x7ff28006c0a0 2023-10-09T06:36:38.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.089+0000 7ff277fff700 1 -- 172.21.15.163:0/3079498637 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27c00b9c0 con 0x7ff28006c0a0 2023-10-09T06:36:38.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.089+0000 7ff277fff700 1 -- 172.21.15.163:0/3079498637 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c00bd60 con 0x7ff28006c0a0 2023-10-09T06:36:38.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.090+0000 7ff28766a700 1 -- 172.21.15.163:0/3079498637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff28006c0a0 msgr2=0x7ff280141330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:38.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.090+0000 7ff28766a700 1 --2- 172.21.15.163:0/3079498637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff28006c0a0 0x7ff280141330 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ff28015b530 tx=0x7ff27c009250 comp rx=0 tx=0).stop 2023-10-09T06:36:38.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.090+0000 7ff28766a700 1 -- 172.21.15.163:0/3079498637 shutdown_connections 2023-10-09T06:36:38.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.090+0000 7ff28766a700 1 --2- 172.21.15.163:0/3079498637 >> v2:172.21.15.181:3300/0 conn(0x7ff2800c8e00 0x7ff2800c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:38.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.090+0000 7ff28766a700 1 --2- 172.21.15.163:0/3079498637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff28006bcd0 0x7ff280140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:38.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.090+0000 7ff28766a700 1 --2- 172.21.15.163:0/3079498637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff28006c0a0 0x7ff280141330 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:38.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.090+0000 7ff28766a700 1 -- 172.21.15.163:0/3079498637 >> 172.21.15.163:0/3079498637 conn(0x7ff2800ce600 msgr2=0x7ff2800cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:38.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.090+0000 7ff28766a700 1 -- 172.21.15.163:0/3079498637 shutdown_connections 2023-10-09T06:36:38.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.090+0000 7ff28766a700 1 -- 172.21.15.163:0/3079498637 wait complete. 2023-10-09T06:36:38.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.091+0000 7ff28766a700 1 Processor -- start 2023-10-09T06:36:38.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.091+0000 7ff28766a700 1 -- start start 2023-10-09T06:36:38.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.092+0000 7ff28766a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff28006bcd0 0x7ff280157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:38.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.092+0000 7ff28766a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff28006c0a0 0x7ff2800c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:38.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.092+0000 7ff28766a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2800c8e00 0x7ff28013db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:38.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.092+0000 7ff28766a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2800c6830 con 0x7ff28006c0a0 2023-10-09T06:36:38.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.092+0000 7ff28766a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2801578a0 con 0x7ff2800c8e00 2023-10-09T06:36:38.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.092+0000 7ff28766a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2800c8010 con 0x7ff28006bcd0 2023-10-09T06:36:38.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.092+0000 7ff284c05700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff28006c0a0 0x7ff2800c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:38.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.092+0000 7ff284c05700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff28006c0a0 0x7ff2800c60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59834/0 (socket says 172.21.15.163:59834) 2023-10-09T06:36:38.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.092+0000 7ff284c05700 1 -- 172.21.15.163:0/1069215211 learned_addr learned my addr 172.21.15.163:0/1069215211 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:38.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.092+0000 7ff285406700 1 --2- 172.21.15.163:0/1069215211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff28006bcd0 0x7ff280157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:38.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.092+0000 7ff285c07700 1 --2- 172.21.15.163:0/1069215211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2800c8e00 0x7ff28013db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:38.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.092+0000 7ff284c05700 1 -- 172.21.15.163:0/1069215211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff28006bcd0 msgr2=0x7ff280157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:38.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.092+0000 7ff284c05700 1 --2- 172.21.15.163:0/1069215211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff28006bcd0 0x7ff280157360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:38.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.092+0000 7ff284c05700 1 -- 172.21.15.163:0/1069215211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2800c8e00 msgr2=0x7ff28013db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:38.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.092+0000 7ff284c05700 1 --2- 172.21.15.163:0/1069215211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2800c8e00 0x7ff28013db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:38.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.092+0000 7ff284c05700 1 -- 172.21.15.163:0/1069215211 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27c0094f0 con 0x7ff28006c0a0 2023-10-09T06:36:38.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.092+0000 7ff284c05700 1 --2- 172.21.15.163:0/1069215211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff28006c0a0 0x7ff2800c60c0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7ff27c009290 tx=0x7ff27c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:38.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.093+0000 7ff2757fa700 1 -- 172.21.15.163:0/1069215211 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c026480 con 0x7ff28006c0a0 2023-10-09T06:36:38.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.093+0000 7ff2757fa700 1 -- 172.21.15.163:0/1069215211 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27c01f070 con 0x7ff28006c0a0 2023-10-09T06:36:38.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.093+0000 7ff2757fa700 1 -- 172.21.15.163:0/1069215211 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c026830 con 0x7ff28006c0a0 2023-10-09T06:36:38.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.093+0000 7ff28766a700 1 -- 172.21.15.163:0/1069215211 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2800c6600 con 0x7ff28006c0a0 2023-10-09T06:36:38.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.093+0000 7ff28766a700 1 -- 172.21.15.163:0/1069215211 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2800b17b0 con 0x7ff28006c0a0 2023-10-09T06:36:38.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.094+0000 7ff28766a700 1 -- 172.21.15.163:0/1069215211 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff28004c3c0 con 0x7ff28006c0a0 2023-10-09T06:36:38.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.094+0000 7ff2757fa700 1 -- 172.21.15.163:0/1069215211 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7ff27c026ba0 con 0x7ff28006c0a0 2023-10-09T06:36:38.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.094+0000 7ff2757fa700 1 --2- 172.21.15.163:0/1069215211 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff264067290 0x7ff264069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:38.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.094+0000 7ff2757fa700 1 -- 172.21.15.163:0/1069215211 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff27c021070 con 0x7ff28006c0a0 2023-10-09T06:36:38.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.095+0000 7ff285406700 1 --2- 172.21.15.163:0/1069215211 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff264067290 0x7ff264069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:38.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.095+0000 7ff285406700 1 --2- 172.21.15.163:0/1069215211 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff264067290 0x7ff264069740 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff270006f00 tx=0x7ff270008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:38.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.098+0000 7ff2757fa700 1 -- 172.21.15.163:0/1069215211 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff27c053780 con 0x7ff28006c0a0 2023-10-09T06:36:38.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.280+0000 7ff28766a700 1 -- 172.21.15.163:0/1069215211 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff2800c41f0 con 0x7ff28006c0a0 2023-10-09T06:36:38.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.283+0000 7ff2757fa700 1 -- 172.21.15.163:0/1069215211 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff27c056da0 con 0x7ff28006c0a0 2023-10-09T06:36:38.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.286+0000 7ff28766a700 1 -- 172.21.15.163:0/1069215211 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff264067290 msgr2=0x7ff264069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:38.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.286+0000 7ff28766a700 1 --2- 172.21.15.163:0/1069215211 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff264067290 0x7ff264069740 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff270006f00 tx=0x7ff270008040 comp rx=0 tx=0).stop 2023-10-09T06:36:38.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.286+0000 7ff28766a700 1 -- 172.21.15.163:0/1069215211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff28006c0a0 msgr2=0x7ff2800c60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:38.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.286+0000 7ff28766a700 1 --2- 172.21.15.163:0/1069215211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff28006c0a0 0x7ff2800c60c0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7ff27c009290 tx=0x7ff27c017000 comp rx=0 tx=0).stop 2023-10-09T06:36:38.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.286+0000 7ff28766a700 1 -- 172.21.15.163:0/1069215211 shutdown_connections 2023-10-09T06:36:38.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.286+0000 7ff28766a700 1 --2- 172.21.15.163:0/1069215211 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff264067290 0x7ff264069740 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:38.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.286+0000 7ff28766a700 1 --2- 172.21.15.163:0/1069215211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2800c8e00 0x7ff28013db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:38.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.286+0000 7ff28766a700 1 --2- 172.21.15.163:0/1069215211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff28006bcd0 0x7ff280157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:38.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.286+0000 7ff28766a700 1 --2- 172.21.15.163:0/1069215211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff28006c0a0 0x7ff2800c60c0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:38.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.286+0000 7ff28766a700 1 -- 172.21.15.163:0/1069215211 >> 172.21.15.163:0/1069215211 conn(0x7ff2800ce600 msgr2=0x7ff280140bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:38.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.286+0000 7ff28766a700 1 -- 172.21.15.163:0/1069215211 shutdown_connections 2023-10-09T06:36:38.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.286+0000 7ff28766a700 1 -- 172.21.15.163:0/1069215211 wait complete. 2023-10-09T06:36:38.299 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-09T06:36:38.299 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:36:38.338 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:36:38.385 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-09T06:36:38.385 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-09T06:36:38.386 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:36:38.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.499+0000 7fb9b530a700 1 Processor -- start 2023-10-09T06:36:38.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.499+0000 7fb9b530a700 1 -- start start 2023-10-09T06:36:38.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.500+0000 7fb9b530a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb9b0157960 0x7fb9b00af250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:38.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.500+0000 7fb9b530a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9b006b970 0x7fb9b0157d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:38.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.500+0000 7fb9b530a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9b006b030 con 0x7fb9b006b4d0 2023-10-09T06:36:38.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.500+0000 7fb9b530a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9b00ac760 con 0x7fb9b0157960 2023-10-09T06:36:38.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.500+0000 7fb9b530a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b00af830 con 0x7fb9b006b970 2023-10-09T06:36:38.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.500+0000 7fb9aeffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb9b0157960 0x7fb9b00af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:38.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.500+0000 7fb9aeffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb9b0157960 0x7fb9b00af250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41186/0 (socket says 172.21.15.163:41186) 2023-10-09T06:36:38.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.500+0000 7fb9aeffd700 1 -- 172.21.15.163:0/4285773637 learned_addr learned my addr 172.21.15.163:0/4285773637 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:38.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.501+0000 7fb9aeffd700 1 -- 172.21.15.163:0/4285773637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9b006b970 msgr2=0x7fb9b0157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:38.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.501+0000 7fb9af7fe700 1 --2- 172.21.15.163:0/4285773637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9b006b970 0x7fb9b0157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:38.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.501+0000 7fb9aeffd700 1 --2- 172.21.15.163:0/4285773637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9b006b970 0x7fb9b0157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:38.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.501+0000 7fb9aeffd700 1 -- 172.21.15.163:0/4285773637 >> v1:172.21.15.181:6789/0 conn(0x7fb9b006b4d0 legacy=0x7fb9b0068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:38.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.501+0000 7fb9aeffd700 1 -- 172.21.15.163:0/4285773637 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b00c7f60 con 0x7fb9b0157960 2023-10-09T06:36:38.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.501+0000 7fb9af7fe700 1 --2- 172.21.15.163:0/4285773637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9b006b970 0x7fb9b0157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:36:38.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.501+0000 7fb9aeffd700 1 --2- 172.21.15.163:0/4285773637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9b0157960 0x7fb9b00af250 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0009f20 tx=0x7fb9a0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1791233014f615b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:38.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.501+0000 7fb9adffb700 1 -- 172.21.15.163:0/4285773637 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a000e070 con 0x7fb9b0157960 2023-10-09T06:36:38.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.501+0000 7fb9adffb700 1 -- 172.21.15.163:0/4285773637 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a000ba40 con 0x7fb9b0157960 2023-10-09T06:36:38.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.501+0000 7fb9adffb700 1 -- 172.21.15.163:0/4285773637 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a00092e0 con 0x7fb9b0157960 2023-10-09T06:36:38.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.502+0000 7fb9b530a700 1 -- 172.21.15.163:0/4285773637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9b0157960 msgr2=0x7fb9b00af250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:38.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.502+0000 7fb9b530a700 1 --2- 172.21.15.163:0/4285773637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9b0157960 0x7fb9b00af250 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0009f20 tx=0x7fb9a0009840 comp rx=0 tx=0).stop 2023-10-09T06:36:38.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.502+0000 7fb9b530a700 1 -- 172.21.15.163:0/4285773637 shutdown_connections 2023-10-09T06:36:38.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.502+0000 7fb9b530a700 1 --2- 172.21.15.163:0/4285773637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9b006b970 0x7fb9b0157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:38.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.502+0000 7fb9b530a700 1 --2- 172.21.15.163:0/4285773637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9b0157960 0x7fb9b00af250 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:38.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.502+0000 7fb9b530a700 1 -- 172.21.15.163:0/4285773637 >> 172.21.15.163:0/4285773637 conn(0x7fb9b00c9870 msgr2=0x7fb9b00c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:38.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.502+0000 7fb9b530a700 1 -- 172.21.15.163:0/4285773637 shutdown_connections 2023-10-09T06:36:38.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.502+0000 7fb9b530a700 1 -- 172.21.15.163:0/4285773637 wait complete. 2023-10-09T06:36:38.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.503+0000 7fb9b530a700 1 Processor -- start 2023-10-09T06:36:38.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.503+0000 7fb9b530a700 1 -- start start 2023-10-09T06:36:38.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.504+0000 7fb9b530a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9b006b4d0 0x7fb9b01549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:38.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.504+0000 7fb9b530a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9b006b970 0x7fb9b00685e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:38.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.504+0000 7fb9b530a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b0157960 0x7fb9b00c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:38.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.504+0000 7fb9b530a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b0153950 con 0x7fb9b006b4d0 2023-10-09T06:36:38.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.504+0000 7fb9b530a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b0153c30 con 0x7fb9b0157960 2023-10-09T06:36:38.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.504+0000 7fb9b530a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b0153f10 con 0x7fb9b006b970 2023-10-09T06:36:38.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.504+0000 7fb9aeffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9b006b4d0 0x7fb9b01549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:38.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.504+0000 7fb9aeffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9b006b4d0 0x7fb9b01549f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41190/0 (socket says 172.21.15.163:41190) 2023-10-09T06:36:38.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.504+0000 7fb9aeffd700 1 -- 172.21.15.163:0/896223528 learned_addr learned my addr 172.21.15.163:0/896223528 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:38.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.504+0000 7fb9af7fe700 1 --2- 172.21.15.163:0/896223528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b0157960 0x7fb9b00c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:38.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.504+0000 7fb9ae7fc700 1 --2- 172.21.15.163:0/896223528 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9b006b970 0x7fb9b00685e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:38.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.504+0000 7fb9aeffd700 1 -- 172.21.15.163:0/896223528 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9b006b970 msgr2=0x7fb9b00685e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:38.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.504+0000 7fb9aeffd700 1 --2- 172.21.15.163:0/896223528 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9b006b970 0x7fb9b00685e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:38.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.504+0000 7fb9aeffd700 1 -- 172.21.15.163:0/896223528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b0157960 msgr2=0x7fb9b00c5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:38.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.504+0000 7fb9aeffd700 1 --2- 172.21.15.163:0/896223528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b0157960 0x7fb9b00c5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:38.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.504+0000 7fb9aeffd700 1 -- 172.21.15.163:0/896223528 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a0009ae0 con 0x7fb9b006b4d0 2023-10-09T06:36:38.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.504+0000 7fb9aeffd700 1 --2- 172.21.15.163:0/896223528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9b006b4d0 0x7fb9b01549f0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fb9b006cac0 tx=0x7fb9a0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:38.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.504+0000 7fb996ffd700 1 -- 172.21.15.163:0/896223528 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a00129a0 con 0x7fb9b006b4d0 2023-10-09T06:36:38.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.504+0000 7fb996ffd700 1 -- 172.21.15.163:0/896223528 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a0012b00 con 0x7fb9b006b4d0 2023-10-09T06:36:38.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.504+0000 7fb9b530a700 1 -- 172.21.15.163:0/896223528 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b01541f0 con 0x7fb9b006b4d0 2023-10-09T06:36:38.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.504+0000 7fb996ffd700 1 -- 172.21.15.163:0/896223528 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a001d420 con 0x7fb9b006b4d0 2023-10-09T06:36:38.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.504+0000 7fb9b530a700 1 -- 172.21.15.163:0/896223528 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b0153600 con 0x7fb9b006b4d0 2023-10-09T06:36:38.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.505+0000 7fb9b530a700 1 -- 172.21.15.163:0/896223528 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9b0154710 con 0x7fb9b006b4d0 2023-10-09T06:36:38.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.506+0000 7fb996ffd700 1 -- 172.21.15.163:0/896223528 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fb9a001d850 con 0x7fb9b006b4d0 2023-10-09T06:36:38.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.506+0000 7fb996ffd700 1 --2- 172.21.15.163:0/896223528 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb99c067290 0x7fb99c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:38.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.506+0000 7fb996ffd700 1 -- 172.21.15.163:0/896223528 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fb9a0085f60 con 0x7fb9b006b4d0 2023-10-09T06:36:38.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.506+0000 7fb9ae7fc700 1 --2- 172.21.15.163:0/896223528 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb99c067290 0x7fb99c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:38.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.507+0000 7fb9ae7fc700 1 --2- 172.21.15.163:0/896223528 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb99c067290 0x7fb99c069740 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0151da0 tx=0x7fb9a4009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:38.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.510+0000 7fb996ffd700 1 -- 172.21.15.163:0/896223528 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb9b0154710 con 0x7fb9b006b4d0 2023-10-09T06:36:38.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.736+0000 7fb9b530a700 1 -- 172.21.15.163:0/896223528 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9b004c3c0 con 0x7fb9b006b4d0 2023-10-09T06:36:38.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.736+0000 7fb996ffd700 1 -- 172.21.15.163:0/896223528 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+3532 (secure 0 0 0) 0x7fb9a002ade0 con 0x7fb9b006b4d0 2023-10-09T06:36:38.737 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:38.738 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":60,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5501,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/862991395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":862991395},{"type":"v1","addr":"172.21.15.181:6836","nonce":862991395}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/699275566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":699275566},{"type":"v1","addr":"172.21.15.181:6837","nonce":699275566}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6504,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3664732208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3664732208},{"type":"v1","addr":"172.21.15.163:6834","nonce":3664732208}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":6780,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/336780329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":336780329},{"type":"v1","addr":"172.21.15.163:6835","nonce":336780329}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[]} 2023-10-09T06:36:38.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.738+0000 7fb9b530a700 1 -- 172.21.15.163:0/896223528 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb99c067290 msgr2=0x7fb99c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:38.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.738+0000 7fb9b530a700 1 --2- 172.21.15.163:0/896223528 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb99c067290 0x7fb99c069740 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0151da0 tx=0x7fb9a4009060 comp rx=0 tx=0).stop 2023-10-09T06:36:38.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.738+0000 7fb9b530a700 1 -- 172.21.15.163:0/896223528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9b006b4d0 msgr2=0x7fb9b01549f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:38.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.738+0000 7fb9b530a700 1 --2- 172.21.15.163:0/896223528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9b006b4d0 0x7fb9b01549f0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fb9b006cac0 tx=0x7fb9a0017000 comp rx=0 tx=0).stop 2023-10-09T06:36:38.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.738+0000 7fb9b530a700 1 -- 172.21.15.163:0/896223528 shutdown_connections 2023-10-09T06:36:38.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.738+0000 7fb9b530a700 1 --2- 172.21.15.163:0/896223528 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb99c067290 0x7fb99c069740 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:38.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.738+0000 7fb9b530a700 1 --2- 172.21.15.163:0/896223528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9b006b4d0 0x7fb9b01549f0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:38.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.738+0000 7fb9b530a700 1 --2- 172.21.15.163:0/896223528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9b0157960 0x7fb9b00c5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:38.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.738+0000 7fb9b530a700 1 --2- 172.21.15.163:0/896223528 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9b006b970 0x7fb9b00685e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:38.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.738+0000 7fb9b530a700 1 -- 172.21.15.163:0/896223528 >> 172.21.15.163:0/896223528 conn(0x7fb9b00c9870 msgr2=0x7fb9b00694a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:38.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.739+0000 7fb9b530a700 1 -- 172.21.15.163:0/896223528 shutdown_connections 2023-10-09T06:36:38.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.739+0000 7fb9b530a700 1 -- 172.21.15.163:0/896223528 wait complete. 2023-10-09T06:36:38.741 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 60 2023-10-09T06:36:38.752 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-09T06:36:38.752 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-09T06:36:38.752 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-09T06:36:38.752 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:36:38.752 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-09T06:36:38.752 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-09T06:36:38.753 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-09T06:36:38.753 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:36:38.753 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-09T06:36:38.753 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-09T06:36:38.753 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-09T06:36:38.753 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:36:38.753 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-09T06:36:38.754 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-09T06:36:38.754 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-09T06:36:38.754 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:36:38.763 INFO:tasks.ceph.mds.c:Stopped 2023-10-09T06:36:38.763 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-09T06:36:38.763 INFO:tasks.ceph.mds.a:Stopped 2023-10-09T06:36:38.764 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-09T06:36:38.765 INFO:tasks.ceph.mds.b:Stopped 2023-10-09T06:36:38.765 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-09T06:36:38.765 INFO:tasks.ceph.mds.d:Stopped 2023-10-09T06:36:38.766 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-09T06:36:38.805 INFO:tasks.ceph.mds.c:Started 2023-10-09T06:36:38.805 DEBUG:teuthology.parallel:result is None 2023-10-09T06:36:38.806 INFO:tasks.ceph.mds.a:Started 2023-10-09T06:36:38.806 DEBUG:teuthology.parallel:result is None 2023-10-09T06:36:38.807 INFO:tasks.ceph.mds.b:Started 2023-10-09T06:36:38.808 DEBUG:teuthology.parallel:result is None 2023-10-09T06:36:38.808 INFO:tasks.ceph.mds.d:Started 2023-10-09T06:36:38.809 DEBUG:teuthology.parallel:result is None 2023-10-09T06:36:38.809 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-09T06:36:38.878 INFO:tasks.ceph.mds.a.smithi163.stdout:starting mds.a at 2023-10-09T06:36:38.878 INFO:tasks.ceph.mds.c.smithi163.stdout:starting mds.c at 2023-10-09T06:36:38.890 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-09T06:36:38.890 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-09T06:36:38.891 INFO:tasks.ceph.mds.d.smithi181.stderr:2023-10-09T06:36:38.890+0000 7fd3c25a0700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-09T06:36:38.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.931+0000 7fcf5d54d700 1 Processor -- start 2023-10-09T06:36:38.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.931+0000 7fcf5d54d700 1 -- start start 2023-10-09T06:36:38.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.931+0000 7fcf5d54d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcf58141500 0x7fcf580c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:38.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.931+0000 7fcf5d54d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf5815c2a0 con 0x7fcf5806c0a0 2023-10-09T06:36:38.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.931+0000 7fcf5d54d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf5815b690 con 0x7fcf5806bcd0 2023-10-09T06:36:38.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.931+0000 7fcf5d54d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf5815b970 con 0x7fcf58141500 2023-10-09T06:36:38.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.932+0000 7fcf577fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fcf5806c0a0 0x7fcf580c9340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:37868/0 (socket says 172.21.15.163:37868) 2023-10-09T06:36:38.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.932+0000 7fcf577fe700 1 -- 172.21.15.163:0/1739356137 learned_addr learned my addr 172.21.15.163:0/1739356137 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:38.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.932+0000 7fcf4ffff700 1 --2- 172.21.15.163:0/1739356137 >> v2:172.21.15.181:3300/0 conn(0x7fcf58141500 0x7fcf580c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:38.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.932+0000 7fcf567fc700 1 -- 172.21.15.163:0/1739356137 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 445283219 0 0) 0x7fcf5815c2a0 con 0x7fcf5806c0a0 2023-10-09T06:36:38.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.932+0000 7fcf567fc700 1 -- 172.21.15.163:0/1739356137 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf44003570 con 0x7fcf5806c0a0 2023-10-09T06:36:38.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.932+0000 7fcf4ffff700 1 -- 172.21.15.163:0/1739356137 >> v1:172.21.15.181:6789/0 conn(0x7fcf5806bcd0 legacy=0x7fcf58140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:38.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.932+0000 7fcf4ffff700 1 -- 172.21.15.163:0/1739356137 >> v1:172.21.15.163:6789/0 conn(0x7fcf5806c0a0 legacy=0x7fcf580c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:38.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.932+0000 7fcf4ffff700 1 -- 172.21.15.163:0/1739356137 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5815bc50 con 0x7fcf58141500 2023-10-09T06:36:38.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.932+0000 7fcf4ffff700 1 --2- 172.21.15.163:0/1739356137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf58141500 0x7fcf580c8e00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fcf400098e0 tx=0x7fcf400091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5738a4d732e59ce server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:38.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.932+0000 7fcf567fc700 1 -- 172.21.15.163:0/1739356137 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4000b640 con 0x7fcf58141500 2023-10-09T06:36:38.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.932+0000 7fcf567fc700 1 -- 172.21.15.163:0/1739356137 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf4000b7a0 con 0x7fcf58141500 2023-10-09T06:36:38.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.932+0000 7fcf567fc700 1 -- 172.21.15.163:0/1739356137 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4000bb40 con 0x7fcf58141500 2023-10-09T06:36:38.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.933+0000 7fcf5d54d700 1 -- 172.21.15.163:0/1739356137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf58141500 msgr2=0x7fcf580c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:38.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.933+0000 7fcf5d54d700 1 --2- 172.21.15.163:0/1739356137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf58141500 0x7fcf580c8e00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fcf400098e0 tx=0x7fcf400091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:38.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.933+0000 7fcf5d54d700 1 -- 172.21.15.163:0/1739356137 shutdown_connections 2023-10-09T06:36:38.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.933+0000 7fcf5d54d700 1 --2- 172.21.15.163:0/1739356137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf58141500 0x7fcf580c8e00 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:38.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.933+0000 7fcf5d54d700 1 -- 172.21.15.163:0/1739356137 >> 172.21.15.163:0/1739356137 conn(0x7fcf580ce600 msgr2=0x7fcf580cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:38.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.933+0000 7fcf5d54d700 1 -- 172.21.15.163:0/1739356137 shutdown_connections 2023-10-09T06:36:38.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.933+0000 7fcf5d54d700 1 -- 172.21.15.163:0/1739356137 wait complete. 2023-10-09T06:36:38.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.934+0000 7fcf5d54d700 1 Processor -- start 2023-10-09T06:36:38.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.934+0000 7fcf5d54d700 1 -- start start 2023-10-09T06:36:38.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.935+0000 7fcf5d54d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcf5806bcd0 0x7fcf5813c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:38.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.935+0000 7fcf5d54d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf5806c0a0 0x7fcf580c6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:38.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.935+0000 7fcf5d54d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf58141500 0x7fcf5816b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:38.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.935+0000 7fcf5d54d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5813f580 con 0x7fcf5806c0a0 2023-10-09T06:36:38.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.935+0000 7fcf5d54d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5816c030 con 0x7fcf58141500 2023-10-09T06:36:38.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.935+0000 7fcf5d54d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5816c310 con 0x7fcf5806bcd0 2023-10-09T06:36:38.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.935+0000 7fcf4ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf5806c0a0 0x7fcf580c6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:38.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.935+0000 7fcf4ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf5806c0a0 0x7fcf580c6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41212/0 (socket says 172.21.15.163:41212) 2023-10-09T06:36:38.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.935+0000 7fcf4ffff700 1 -- 172.21.15.163:0/4290961846 learned_addr learned my addr 172.21.15.163:0/4290961846 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:38.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.935+0000 7fcf56ffd700 1 --2- 172.21.15.163:0/4290961846 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcf5806bcd0 0x7fcf5813c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:38.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.935+0000 7fcf577fe700 1 --2- 172.21.15.163:0/4290961846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf58141500 0x7fcf5816b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:38.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.935+0000 7fcf4ffff700 1 -- 172.21.15.163:0/4290961846 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcf5806bcd0 msgr2=0x7fcf5813c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:38.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.935+0000 7fcf4ffff700 1 --2- 172.21.15.163:0/4290961846 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcf5806bcd0 0x7fcf5813c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:38.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.935+0000 7fcf4ffff700 1 -- 172.21.15.163:0/4290961846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf58141500 msgr2=0x7fcf5816b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:38.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.935+0000 7fcf4ffff700 1 --2- 172.21.15.163:0/4290961846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf58141500 0x7fcf5816b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:38.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.935+0000 7fcf4ffff700 1 -- 172.21.15.163:0/4290961846 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf40009420 con 0x7fcf5806c0a0 2023-10-09T06:36:38.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.936+0000 7fcf4ffff700 1 --2- 172.21.15.163:0/4290961846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf5806c0a0 0x7fcf580c6250 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fcf58069220 tx=0x7fcf40017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:38.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.936+0000 7fcf4effd700 1 -- 172.21.15.163:0/4290961846 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf40000d10 con 0x7fcf5806c0a0 2023-10-09T06:36:38.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.936+0000 7fcf4effd700 1 -- 172.21.15.163:0/4290961846 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf4000bb20 con 0x7fcf5806c0a0 2023-10-09T06:36:38.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.936+0000 7fcf5d54d700 1 -- 172.21.15.163:0/4290961846 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf5813f210 con 0x7fcf5806c0a0 2023-10-09T06:36:38.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.936+0000 7fcf4effd700 1 -- 172.21.15.163:0/4290961846 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf40011650 con 0x7fcf5806c0a0 2023-10-09T06:36:38.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.936+0000 7fcf5d54d700 1 -- 172.21.15.163:0/4290961846 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf5813ee50 con 0x7fcf5806c0a0 2023-10-09T06:36:38.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.937+0000 7fcf5d54d700 1 -- 172.21.15.163:0/4290961846 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf5813f730 con 0x7fcf5806c0a0 2023-10-09T06:36:38.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.937+0000 7fcf4effd700 1 -- 172.21.15.163:0/4290961846 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fcf400119c0 con 0x7fcf5806c0a0 2023-10-09T06:36:38.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.937+0000 7fcf4effd700 1 --2- 172.21.15.163:0/4290961846 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcf38067290 0x7fcf38069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:38.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.938+0000 7fcf4effd700 1 -- 172.21.15.163:0/4290961846 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fcf40085c80 con 0x7fcf5806c0a0 2023-10-09T06:36:38.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.938+0000 7fcf56ffd700 1 --2- 172.21.15.163:0/4290961846 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcf38067290 0x7fcf38069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:38.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.938+0000 7fcf56ffd700 1 --2- 172.21.15.163:0/4290961846 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcf38067290 0x7fcf38069740 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcf580c5fd0 tx=0x7fcf48009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:38.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:38.942+0000 7fcf4effd700 1 -- 172.21.15.163:0/4290961846 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcf5813f730 con 0x7fcf5806c0a0 2023-10-09T06:36:39.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:39.140+0000 7fcf5d54d700 1 -- 172.21.15.163:0/4290961846 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fcf5813f040 con 0x7fcf5806c0a0 2023-10-09T06:36:39.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:39.978+0000 7fcf4effd700 1 -- 172.21.15.163:0/4290961846 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v97) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcf4008c080 con 0x7fcf5806c0a0 2023-10-09T06:36:39.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:39.981+0000 7fcf5d54d700 1 -- 172.21.15.163:0/4290961846 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcf38067290 msgr2=0x7fcf38069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:39.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:39.981+0000 7fcf5d54d700 1 --2- 172.21.15.163:0/4290961846 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcf38067290 0x7fcf38069740 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcf580c5fd0 tx=0x7fcf48009040 comp rx=0 tx=0).stop 2023-10-09T06:36:39.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:39.981+0000 7fcf5d54d700 1 -- 172.21.15.163:0/4290961846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf5806c0a0 msgr2=0x7fcf580c6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:39.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:39.981+0000 7fcf5d54d700 1 --2- 172.21.15.163:0/4290961846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf5806c0a0 0x7fcf580c6250 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fcf58069220 tx=0x7fcf40017000 comp rx=0 tx=0).stop 2023-10-09T06:36:39.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:39.981+0000 7fcf5d54d700 1 -- 172.21.15.163:0/4290961846 shutdown_connections 2023-10-09T06:36:39.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:39.981+0000 7fcf5d54d700 1 --2- 172.21.15.163:0/4290961846 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcf38067290 0x7fcf38069740 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:39.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:39.981+0000 7fcf5d54d700 1 --2- 172.21.15.163:0/4290961846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf58141500 0x7fcf5816b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:39.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:39.981+0000 7fcf5d54d700 1 --2- 172.21.15.163:0/4290961846 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcf5806bcd0 0x7fcf5813c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:39.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:39.981+0000 7fcf5d54d700 1 --2- 172.21.15.163:0/4290961846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf5806c0a0 0x7fcf580c6250 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:39.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:39.981+0000 7fcf5d54d700 1 -- 172.21.15.163:0/4290961846 >> 172.21.15.163:0/4290961846 conn(0x7fcf580ce600 msgr2=0x7fcf580c33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:39.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:39.981+0000 7fcf5d54d700 1 -- 172.21.15.163:0/4290961846 shutdown_connections 2023-10-09T06:36:39.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:39.981+0000 7fcf5d54d700 1 -- 172.21.15.163:0/4290961846 wait complete. 2023-10-09T06:36:39.984 INFO:teuthology.orchestra.run.smithi163.stderr: removed all blocklist entries 2023-10-09T06:36:39.997 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-09T06:36:40.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.116+0000 7f988cb23700 1 Processor -- start 2023-10-09T06:36:40.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.116+0000 7f988cb23700 1 -- start start 2023-10-09T06:36:40.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.117+0000 7f988cb23700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98880caa90 0x7f98880c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:40.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.117+0000 7f988cb23700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f988815d7e0 con 0x7f988806bcd0 2023-10-09T06:36:40.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.117+0000 7f988cb23700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f988815dac0 con 0x7f988806c0a0 2023-10-09T06:36:40.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.117+0000 7f988cb23700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98880caf30 con 0x7f98880caa90 2023-10-09T06:36:40.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.118+0000 7f988659c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f988806bcd0 0x7f988815a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:37880/0 (socket says 172.21.15.163:37880) 2023-10-09T06:36:40.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.118+0000 7f988659c700 1 -- 172.21.15.163:0/2933450530 learned_addr learned my addr 172.21.15.163:0/2933450530 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:40.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.118+0000 7f9886d9d700 1 --2- 172.21.15.163:0/2933450530 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98880caa90 0x7f98880c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:40.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.118+0000 7f988559a700 1 -- 172.21.15.163:0/2933450530 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3048823282 0 0) 0x7f988815d7e0 con 0x7f988806bcd0 2023-10-09T06:36:40.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.118+0000 7f988559a700 1 -- 172.21.15.163:0/2933450530 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9870003570 con 0x7f988806bcd0 2023-10-09T06:36:40.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.118+0000 7f988559a700 1 -- 172.21.15.163:0/2933450530 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3252683057 0 0) 0x7f988815dac0 con 0x7f988806c0a0 2023-10-09T06:36:40.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.118+0000 7f988559a700 1 -- 172.21.15.163:0/2933450530 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f988815d7e0 con 0x7f988806c0a0 2023-10-09T06:36:40.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.118+0000 7f988559a700 1 -- 172.21.15.163:0/2933450530 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 586509876 0 0) 0x7f9870003570 con 0x7f988806bcd0 2023-10-09T06:36:40.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.118+0000 7f988559a700 1 -- 172.21.15.163:0/2933450530 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98880caa90 msgr2=0x7f98880c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:40.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.118+0000 7f988559a700 1 --2- 172.21.15.163:0/2933450530 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98880caa90 0x7f98880c8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:40.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.118+0000 7f988559a700 1 -- 172.21.15.163:0/2933450530 >> v1:172.21.15.181:6789/0 conn(0x7f988806c0a0 legacy=0x7f988815df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:40.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.118+0000 7f988559a700 1 -- 172.21.15.163:0/2933450530 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988815aa30 con 0x7f988806bcd0 2023-10-09T06:36:40.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.118+0000 7f988559a700 1 -- 172.21.15.163:0/2933450530 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f9878001470 con 0x7f988806bcd0 2023-10-09T06:36:40.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.118+0000 7f988559a700 1 -- 172.21.15.163:0/2933450530 >> v1:172.21.15.163:6789/0 conn(0x7f988806bcd0 legacy=0x7f988815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:40.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.118+0000 7f988559a700 1 --2- 172.21.15.163:0/2933450530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9870003c20 0x7f9870004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:40.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.118+0000 7f988559a700 1 --2- 172.21.15.163:0/2933450530 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98880caa90 0x7f9870004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:40.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.118+0000 7f988559a700 1 --2- 172.21.15.163:0/2933450530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9870004d50 0x7f98700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:40.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.118+0000 7f988559a700 1 -- 172.21.15.163:0/2933450530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98880caf30 con 0x7f9870003c20 2023-10-09T06:36:40.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.118+0000 7f988559a700 1 -- 172.21.15.163:0/2933450530 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987000bc40 con 0x7f9870004d50 2023-10-09T06:36:40.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.118+0000 7f988559a700 1 -- 172.21.15.163:0/2933450530 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f987000be00 con 0x7f98880caa90 2023-10-09T06:36:40.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.118+0000 7f988659c700 1 --2- 172.21.15.163:0/2933450530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9870003c20 0x7f9870004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:40.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.118+0000 7f9885d9b700 1 --2- 172.21.15.163:0/2933450530 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98880caa90 0x7f9870004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:40.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.118+0000 7f9886d9d700 1 --2- 172.21.15.163:0/2933450530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9870004d50 0x7f98700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:40.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.118+0000 7f988659c700 1 -- 172.21.15.163:0/2933450530 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98880caa90 msgr2=0x7f9870004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:40.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.118+0000 7f988659c700 1 --2- 172.21.15.163:0/2933450530 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98880caa90 0x7f9870004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:40.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.118+0000 7f988659c700 1 -- 172.21.15.163:0/2933450530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9870004d50 msgr2=0x7f98700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:40.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.118+0000 7f988659c700 1 --2- 172.21.15.163:0/2933450530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9870004d50 0x7f98700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:40.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.119+0000 7f988659c700 1 -- 172.21.15.163:0/2933450530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9870003570 con 0x7f9870003c20 2023-10-09T06:36:40.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.119+0000 7f988659c700 1 --2- 172.21.15.163:0/2933450530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9870003c20 0x7f9870004010 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f987800c1e0 tx=0x7f9878001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b305ff4dd4e3ff server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:40.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.119+0000 7f988559a700 1 -- 172.21.15.163:0/2933450530 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987800cb10 con 0x7f9870003c20 2023-10-09T06:36:40.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.119+0000 7f988559a700 1 -- 172.21.15.163:0/2933450530 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987800cc90 con 0x7f9870003c20 2023-10-09T06:36:40.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.119+0000 7f988559a700 1 -- 172.21.15.163:0/2933450530 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9878005b70 con 0x7f9870003c20 2023-10-09T06:36:40.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.119+0000 7f988cb23700 1 -- 172.21.15.163:0/2933450530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9870003c20 msgr2=0x7f9870004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:40.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.119+0000 7f988cb23700 1 --2- 172.21.15.163:0/2933450530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9870003c20 0x7f9870004010 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f987800c1e0 tx=0x7f9878001d40 comp rx=0 tx=0).stop 2023-10-09T06:36:40.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.119+0000 7f9886d9d700 1 -- 172.21.15.163:0/2933450530 reap_dead start 2023-10-09T06:36:40.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.119+0000 7f988cb23700 1 -- 172.21.15.163:0/2933450530 shutdown_connections 2023-10-09T06:36:40.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.119+0000 7f988cb23700 1 -- 172.21.15.163:0/2933450530 >> 172.21.15.163:0/2933450530 conn(0x7f98880ce600 msgr2=0x7f98880cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:40.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.119+0000 7f988cb23700 1 -- 172.21.15.163:0/2933450530 shutdown_connections 2023-10-09T06:36:40.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.119+0000 7f988cb23700 1 -- 172.21.15.163:0/2933450530 wait complete. 2023-10-09T06:36:40.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.120+0000 7f988cb23700 1 Processor -- start 2023-10-09T06:36:40.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.121+0000 7f988cb23700 1 -- start start 2023-10-09T06:36:40.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.121+0000 7f988cb23700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9870003c20 0x7f988816a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:40.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.121+0000 7f988cb23700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98880b2430 0x7f98880c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:40.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.121+0000 7f988cb23700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98881602d0 0x7f9888160750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:40.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.121+0000 7f988cb23700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988816af20 con 0x7f98881602d0 2023-10-09T06:36:40.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.121+0000 7f988cb23700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9888160c90 con 0x7f9870003c20 2023-10-09T06:36:40.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.121+0000 7f988cb23700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98880b2910 con 0x7f98880b2430 2023-10-09T06:36:40.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.121+0000 7f9886d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98881602d0 0x7f9888160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:40.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.121+0000 7f9886d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98881602d0 0x7f9888160750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41236/0 (socket says 172.21.15.163:41236) 2023-10-09T06:36:40.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.121+0000 7f9886d9d700 1 -- 172.21.15.163:0/660969186 learned_addr learned my addr 172.21.15.163:0/660969186 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:40.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.121+0000 7f988659c700 1 --2- 172.21.15.163:0/660969186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9870003c20 0x7f988816a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:40.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.121+0000 7f9885d9b700 1 --2- 172.21.15.163:0/660969186 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98880b2430 0x7f98880c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:40.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.122+0000 7f9886d9d700 1 -- 172.21.15.163:0/660969186 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98880b2430 msgr2=0x7f98880c4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:40.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.122+0000 7f9886d9d700 1 --2- 172.21.15.163:0/660969186 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98880b2430 0x7f98880c4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:40.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.122+0000 7f9886d9d700 1 -- 172.21.15.163:0/660969186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9870003c20 msgr2=0x7f988816a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:40.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.122+0000 7f9886d9d700 1 --2- 172.21.15.163:0/660969186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9870003c20 0x7f988816a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:40.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.122+0000 7f9886d9d700 1 -- 172.21.15.163:0/660969186 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98880b2bf0 con 0x7f98881602d0 2023-10-09T06:36:40.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.122+0000 7f9885d9b700 1 --2- 172.21.15.163:0/660969186 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98880b2430 0x7f98880c4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:40.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.122+0000 7f9886d9d700 1 --2- 172.21.15.163:0/660969186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98881602d0 0x7f9888160750 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f988806b640 tx=0x7f988000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:40.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.122+0000 7f98767fc700 1 -- 172.21.15.163:0/660969186 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988000e070 con 0x7f98881602d0 2023-10-09T06:36:40.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.122+0000 7f98767fc700 1 -- 172.21.15.163:0/660969186 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9880011bd0 con 0x7f98881602d0 2023-10-09T06:36:40.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.122+0000 7f98767fc700 1 -- 172.21.15.163:0/660969186 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880016470 con 0x7f98881602d0 2023-10-09T06:36:40.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.122+0000 7f988cb23700 1 -- 172.21.15.163:0/660969186 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98880b2ed0 con 0x7f98881602d0 2023-10-09T06:36:40.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.122+0000 7f988cb23700 1 -- 172.21.15.163:0/660969186 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98880b3490 con 0x7f98881602d0 2023-10-09T06:36:40.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.123+0000 7f988cb23700 1 -- 172.21.15.163:0/660969186 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98880b3680 con 0x7f98881602d0 2023-10-09T06:36:40.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.123+0000 7f98767fc700 1 -- 172.21.15.163:0/660969186 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f9880016860 con 0x7f98881602d0 2023-10-09T06:36:40.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.124+0000 7f98767fc700 1 --2- 172.21.15.163:0/660969186 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9868067290 0x7f9868069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:40.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.124+0000 7f98767fc700 1 -- 172.21.15.163:0/660969186 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f98800854b0 con 0x7f98881602d0 2023-10-09T06:36:40.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.124+0000 7f988659c700 1 --2- 172.21.15.163:0/660969186 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9868067290 0x7f9868069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:40.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.124+0000 7f988659c700 1 --2- 172.21.15.163:0/660969186 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9868067290 0x7f9868069740 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9888167b80 tx=0x7f9878015000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:40.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.128+0000 7f98767fc700 1 -- 172.21.15.163:0/660969186 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9880055fe0 con 0x7f98881602d0 2023-10-09T06:36:40.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.382+0000 7f988cb23700 1 -- 172.21.15.163:0/660969186 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f988804c3c0 con 0x7f98881602d0 2023-10-09T06:36:40.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.383+0000 7f98767fc700 1 -- 172.21.15.163:0/660969186 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f9880055b70 con 0x7f98881602d0 2023-10-09T06:36:40.384 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:40.384 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:36:40.384 INFO:teuthology.orchestra.run.smithi163.stdout: "auth_dump": [ 2023-10-09T06:36:40.384 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:40.384 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.a", 2023-10-09T06:36:40.384 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNltXtwDxAAlnrWRsfnZDLEmsJVowh3Mg==", 2023-10-09T06:36:40.384 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:40.385 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:36:40.385 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:36:40.385 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:36:40.385 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:40.385 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:40.385 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:40.386 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:40.386 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.b", 2023-10-09T06:36:40.386 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlTdGhGhAAXuldy0HrFd+44sSu/EV3qA==", 2023-10-09T06:36:40.386 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:40.386 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:36:40.386 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:36:40.386 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:36:40.386 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:40.386 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:40.386 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:40.387 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:40.387 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.c", 2023-10-09T06:36:40.387 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlGis7FhAAYG76TiOFX8QkTwjNX2+Ncw==", 2023-10-09T06:36:40.387 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:40.388 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:36:40.388 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:36:40.388 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:36:40.388 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:40.388 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:40.388 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:40.388 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:40.388 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.d", 2023-10-09T06:36:40.389 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlKhRuIRAAfhG5Gpc7klXJloYXepOhsg==", 2023-10-09T06:36:40.389 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:40.389 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:36:40.389 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:36:40.389 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:36:40.389 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:40.389 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:40.389 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:40.389 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:40.389 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.0", 2023-10-09T06:36:40.389 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDcnSNlrQeFOxAA+s6UitrTn5TK1N7kgySoXw==", 2023-10-09T06:36:40.390 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:40.390 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:36:40.390 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:36:40.390 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:40.390 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:40.390 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:40.390 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:40.390 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.1", 2023-10-09T06:36:40.390 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDfnSNlleccORAAnB+lNUFblivdoY44JfELxA==", 2023-10-09T06:36:40.390 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:40.390 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:36:40.391 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:36:40.391 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:40.391 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:40.391 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:40.391 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:40.391 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.2", 2023-10-09T06:36:40.391 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDinSNlFoB4NxAAT0+trRBij5rqgc+F0eWUfQ==", 2023-10-09T06:36:40.391 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:40.391 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:36:40.392 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:36:40.392 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:40.392 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:40.392 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:40.392 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:40.392 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.3", 2023-10-09T06:36:40.392 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDlnSNlRRQ6NhAABoBtrFjSvQujolq0Pxrd1Q==", 2023-10-09T06:36:40.393 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:40.393 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:36:40.393 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:36:40.393 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:40.393 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:40.393 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:40.393 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:40.394 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.4", 2023-10-09T06:36:40.394 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDrnSNl308uOxAA5OJyPLGQmVSJqha5d+nKKQ==", 2023-10-09T06:36:40.394 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:40.394 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:36:40.394 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:36:40.394 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:40.394 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:40.394 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:40.394 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:40.395 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.5", 2023-10-09T06:36:40.395 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDunSNlHnK+OBAAU7l4dCJ82SbGSzBEm8Axzw==", 2023-10-09T06:36:40.395 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:40.395 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:36:40.395 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:36:40.395 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:40.395 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:40.395 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:40.396 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:40.396 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.6", 2023-10-09T06:36:40.396 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDxnSNlwUSLOBAAN5N+F5ho+Co8pBh4Rz+lXQ==", 2023-10-09T06:36:40.396 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:40.396 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:36:40.396 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:36:40.396 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:40.397 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:40.397 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:40.397 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:40.397 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.7", 2023-10-09T06:36:40.397 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQD0nSNl2gxGORAAoE8OLGEifcEx/km8rfSV9g==", 2023-10-09T06:36:40.397 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:40.397 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:36:40.398 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:36:40.398 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:40.398 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:40.398 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:40.398 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:40.398 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.0", 2023-10-09T06:36:40.398 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlnxY5JRAApr6czcpMdQYYtXQ9yg2q6A==", 2023-10-09T06:36:40.398 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:40.398 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:36:40.398 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow r", 2023-10-09T06:36:40.399 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-09T06:36:40.399 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:40.399 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:40.399 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:40.399 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.1", 2023-10-09T06:36:40.399 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlk6ybLRAAuShqTDAtymGdxN8KBdZDDA==", 2023-10-09T06:36:40.399 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:40.399 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:36:40.400 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:36:40.400 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:36:40.400 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:36:40.400 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:40.400 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:40.400 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:40.400 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.2", 2023-10-09T06:36:40.400 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlYyOmKhAAQmpr+mGfJH4GUa28dtlJwA==", 2023-10-09T06:36:40.401 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:40.401 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:36:40.401 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:36:40.401 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:36:40.401 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:36:40.401 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:40.401 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:40.401 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:40.402 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.3", 2023-10-09T06:36:40.402 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlD2H1MhAA4zvaXY/Gd0kndzi+QSwp7w==", 2023-10-09T06:36:40.402 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:40.402 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:36:40.402 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:36:40.402 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:36:40.402 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:36:40.402 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:40.402 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:40.402 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:40.402 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.admin", 2023-10-09T06:36:40.403 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDYnSNlsHH4LxAA85xeFEP1CUwoblaePYk5zA==", 2023-10-09T06:36:40.403 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:40.403 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:36:40.403 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:36:40.403 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:36:40.403 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:40.403 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:40.403 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:40.403 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:40.403 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mgr.x", 2023-10-09T06:36:40.404 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNl2sloDBAA0FoWN9iovCl6783o3/btiA==", 2023-10-09T06:36:40.404 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:40.404 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:36:40.404 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile mgr", 2023-10-09T06:36:40.404 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:40.404 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:40.405 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:36:40.405 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:40.405 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mgr.y", 2023-10-09T06:36:40.405 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlcFWVCBAAR2dKPg01ciTiNpSX0JIKFg==", 2023-10-09T06:36:40.405 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:36:40.405 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:36:40.405 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile mgr", 2023-10-09T06:36:40.405 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:36:40.405 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:40.405 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:40.406 INFO:teuthology.orchestra.run.smithi163.stdout: ] 2023-10-09T06:36:40.406 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:36:40.407 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.384+0000 7f988cb23700 1 -- 172.21.15.163:0/660969186 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9868067290 msgr2=0x7f9868069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:40.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.384+0000 7f988cb23700 1 --2- 172.21.15.163:0/660969186 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9868067290 0x7f9868069740 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9888167b80 tx=0x7f9878015000 comp rx=0 tx=0).stop 2023-10-09T06:36:40.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.384+0000 7f988cb23700 1 -- 172.21.15.163:0/660969186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98881602d0 msgr2=0x7f9888160750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:40.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.384+0000 7f988cb23700 1 --2- 172.21.15.163:0/660969186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98881602d0 0x7f9888160750 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f988806b640 tx=0x7f988000b040 comp rx=0 tx=0).stop 2023-10-09T06:36:40.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.384+0000 7f988cb23700 1 -- 172.21.15.163:0/660969186 shutdown_connections 2023-10-09T06:36:40.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.384+0000 7f988cb23700 1 --2- 172.21.15.163:0/660969186 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9868067290 0x7f9868069740 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:40.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.384+0000 7f988cb23700 1 --2- 172.21.15.163:0/660969186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98881602d0 0x7f9888160750 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:40.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.385+0000 7f988cb23700 1 --2- 172.21.15.163:0/660969186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9870003c20 0x7f988816a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:40.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.385+0000 7f988cb23700 1 --2- 172.21.15.163:0/660969186 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98880b2430 0x7f98880c4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:40.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.385+0000 7f988cb23700 1 -- 172.21.15.163:0/660969186 >> 172.21.15.163:0/660969186 conn(0x7f98880ce600 msgr2=0x7f98880cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:40.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.385+0000 7f988cb23700 1 -- 172.21.15.163:0/660969186 shutdown_connections 2023-10-09T06:36:40.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.385+0000 7f988cb23700 1 -- 172.21.15.163:0/660969186 wait complete. 2023-10-09T06:36:40.409 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:36:40.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.526+0000 7f6f1a611700 1 Processor -- start 2023-10-09T06:36:40.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.526+0000 7f6f1a611700 1 -- start start 2023-10-09T06:36:40.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.527+0000 7f6f1a611700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6f14159890 0x7f6f1415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:40.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.527+0000 7f6f1a611700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f14069560 0x7f6f140699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:40.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.527+0000 7f6f1a611700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f14069ee0 con 0x7f6f14159c60 2023-10-09T06:36:40.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.527+0000 7f6f1a611700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f1415a160 con 0x7f6f14159890 2023-10-09T06:36:40.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.527+0000 7f6f1a611700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f140d13a0 con 0x7f6f14069560 2023-10-09T06:36:40.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.527+0000 7f6f137fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f6f14159c60 0x7f6f1415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:37886/0 (socket says 172.21.15.163:37886) 2023-10-09T06:36:40.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.527+0000 7f6f137fe700 1 -- 172.21.15.163:0/725816115 learned_addr learned my addr 172.21.15.163:0/725816115 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:40.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.527+0000 7f6f12ffd700 1 -- 172.21.15.163:0/725816115 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2370517621 0 0) 0x7f6f14069ee0 con 0x7f6f14159c60 2023-10-09T06:36:40.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.527+0000 7f6f12ffd700 1 -- 172.21.15.163:0/725816115 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6efc003570 con 0x7f6f14159c60 2023-10-09T06:36:40.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.528+0000 7f6f13fff700 1 --2- 172.21.15.163:0/725816115 >> v2:172.21.15.181:3300/0 conn(0x7f6f14159890 0x7f6f1415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:40.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.528+0000 7f6f18bae700 1 --2- 172.21.15.163:0/725816115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f14069560 0x7f6f140699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:40.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.528+0000 7f6f12ffd700 1 -- 172.21.15.163:0/725816115 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4056431076 0 0) 0x7f6efc003570 con 0x7f6f14159c60 2023-10-09T06:36:40.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.528+0000 7f6f12ffd700 1 -- 172.21.15.163:0/725816115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f14069560 msgr2=0x7f6f140699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:40.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.528+0000 7f6f12ffd700 1 --2- 172.21.15.163:0/725816115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f14069560 0x7f6f140699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:40.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.528+0000 7f6f12ffd700 1 -- 172.21.15.163:0/725816115 >> v2:172.21.15.181:3300/0 conn(0x7f6f14159890 msgr2=0x7f6f1415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:40.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.528+0000 7f6f12ffd700 1 --2- 172.21.15.163:0/725816115 >> v2:172.21.15.181:3300/0 conn(0x7f6f14159890 0x7f6f1415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:40.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.528+0000 7f6f12ffd700 1 -- 172.21.15.163:0/725816115 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1415e140 con 0x7f6f14159c60 2023-10-09T06:36:40.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.528+0000 7f6f12ffd700 1 -- 172.21.15.163:0/725816115 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f6f08001470 con 0x7f6f14159c60 2023-10-09T06:36:40.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.528+0000 7f6f12ffd700 1 -- 172.21.15.163:0/725816115 >> v1:172.21.15.163:6789/0 conn(0x7f6f14159c60 legacy=0x7f6f1415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:40.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.528+0000 7f6f12ffd700 1 --2- 172.21.15.163:0/725816115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f14069560 0x7f6efc003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:40.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.528+0000 7f6f12ffd700 1 --2- 172.21.15.163:0/725816115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6efc003fe0 0x7f6efc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:40.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.528+0000 7f6f12ffd700 1 --2- 172.21.15.163:0/725816115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6efc004d50 0x7f6efc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:40.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.528+0000 7f6f12ffd700 1 -- 172.21.15.163:0/725816115 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1415a160 con 0x7f6efc003fe0 2023-10-09T06:36:40.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.528+0000 7f6f12ffd700 1 -- 172.21.15.163:0/725816115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f140d13a0 con 0x7f6efc004d50 2023-10-09T06:36:40.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.528+0000 7f6f12ffd700 1 -- 172.21.15.163:0/725816115 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc00bb40 con 0x7f6f14069560 2023-10-09T06:36:40.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.528+0000 7f6f137fe700 1 --2- 172.21.15.163:0/725816115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6efc003fe0 0x7f6efc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:40.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.528+0000 7f6f13fff700 1 --2- 172.21.15.163:0/725816115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f14069560 0x7f6efc003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:40.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.528+0000 7f6f18bae700 1 --2- 172.21.15.163:0/725816115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6efc004d50 0x7f6efc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:40.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.528+0000 7f6f137fe700 1 -- 172.21.15.163:0/725816115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f14069560 msgr2=0x7f6efc003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:40.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.528+0000 7f6f137fe700 1 --2- 172.21.15.163:0/725816115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f14069560 0x7f6efc003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:40.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.528+0000 7f6f137fe700 1 -- 172.21.15.163:0/725816115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6efc004d50 msgr2=0x7f6efc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:40.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.528+0000 7f6f137fe700 1 --2- 172.21.15.163:0/725816115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6efc004d50 0x7f6efc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:40.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.528+0000 7f6f137fe700 1 -- 172.21.15.163:0/725816115 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6efc003570 con 0x7f6efc003fe0 2023-10-09T06:36:40.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.528+0000 7f6f137fe700 1 --2- 172.21.15.163:0/725816115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6efc003fe0 0x7f6efc004810 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f6f0800c390 tx=0x7f6f08001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8225ea35086472 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:40.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.528+0000 7f6f12ffd700 1 -- 172.21.15.163:0/725816115 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0800cc60 con 0x7f6efc003fe0 2023-10-09T06:36:40.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.529+0000 7f6f12ffd700 1 -- 172.21.15.163:0/725816115 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0800cde0 con 0x7f6efc003fe0 2023-10-09T06:36:40.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.529+0000 7f6f12ffd700 1 -- 172.21.15.163:0/725816115 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f08005b70 con 0x7f6efc003fe0 2023-10-09T06:36:40.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.529+0000 7f6f1a611700 1 -- 172.21.15.163:0/725816115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6efc003fe0 msgr2=0x7f6efc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:40.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.529+0000 7f6f1a611700 1 --2- 172.21.15.163:0/725816115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6efc003fe0 0x7f6efc004810 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f6f0800c390 tx=0x7f6f08001e90 comp rx=0 tx=0).stop 2023-10-09T06:36:40.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.529+0000 7f6f18bae700 1 -- 172.21.15.163:0/725816115 reap_dead start 2023-10-09T06:36:40.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.529+0000 7f6f1a611700 1 -- 172.21.15.163:0/725816115 shutdown_connections 2023-10-09T06:36:40.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.529+0000 7f6f1a611700 1 -- 172.21.15.163:0/725816115 >> 172.21.15.163:0/725816115 conn(0x7f6f14068e20 msgr2=0x7f6f140d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:40.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.529+0000 7f6f1a611700 1 -- 172.21.15.163:0/725816115 shutdown_connections 2023-10-09T06:36:40.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.529+0000 7f6f1a611700 1 -- 172.21.15.163:0/725816115 wait complete. 2023-10-09T06:36:40.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.530+0000 7f6f1a611700 1 Processor -- start 2023-10-09T06:36:40.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.530+0000 7f6f1a611700 1 -- start start 2023-10-09T06:36:40.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.531+0000 7f6f1a611700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6efc003fe0 0x7f6f14152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:40.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.531+0000 7f6f1a611700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f140c80b0 0x7f6f140c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:40.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.531+0000 7f6f1a611700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f140ce150 0x7f6f140ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:40.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.531+0000 7f6f1a611700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f140cfa90 con 0x7f6efc003fe0 2023-10-09T06:36:40.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.531+0000 7f6f1a611700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f140c8a60 con 0x7f6f140c80b0 2023-10-09T06:36:40.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.531+0000 7f6f1a611700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f140cee80 con 0x7f6f140ce150 2023-10-09T06:36:40.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.531+0000 7f6f13fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6efc003fe0 0x7f6f14152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:40.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.531+0000 7f6f13fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6efc003fe0 0x7f6f14152070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41250/0 (socket says 172.21.15.163:41250) 2023-10-09T06:36:40.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.531+0000 7f6f13fff700 1 -- 172.21.15.163:0/4292909998 learned_addr learned my addr 172.21.15.163:0/4292909998 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:40.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.531+0000 7f6f13fff700 1 -- 172.21.15.163:0/4292909998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f140ce150 msgr2=0x7f6f140ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:40.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.531+0000 7f6f18bae700 1 --2- 172.21.15.163:0/4292909998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f140ce150 0x7f6f140ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:40.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.531+0000 7f6f137fe700 1 --2- 172.21.15.163:0/4292909998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f140c80b0 0x7f6f140c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:40.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.531+0000 7f6f13fff700 1 --2- 172.21.15.163:0/4292909998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f140ce150 0x7f6f140ce5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:40.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.531+0000 7f6f13fff700 1 -- 172.21.15.163:0/4292909998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f140c80b0 msgr2=0x7f6f140c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:40.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.531+0000 7f6f13fff700 1 --2- 172.21.15.163:0/4292909998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f140c80b0 0x7f6f140c8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:40.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.531+0000 7f6f13fff700 1 -- 172.21.15.163:0/4292909998 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f140cf160 con 0x7f6efc003fe0 2023-10-09T06:36:40.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.531+0000 7f6f137fe700 1 --2- 172.21.15.163:0/4292909998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f140c80b0 0x7f6f140c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:36:40.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.532+0000 7f6f13fff700 1 --2- 172.21.15.163:0/4292909998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6efc003fe0 0x7f6f14152070 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f6f04011d70 tx=0x7f6f040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:40.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.532+0000 7f6efbfff700 1 -- 172.21.15.163:0/4292909998 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f04009e20 con 0x7f6efc003fe0 2023-10-09T06:36:40.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.532+0000 7f6efbfff700 1 -- 172.21.15.163:0/4292909998 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0400fcd0 con 0x7f6efc003fe0 2023-10-09T06:36:40.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.532+0000 7f6f1a611700 1 -- 172.21.15.163:0/4292909998 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f140cf440 con 0x7f6efc003fe0 2023-10-09T06:36:40.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.532+0000 7f6efbfff700 1 -- 172.21.15.163:0/4292909998 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f04014650 con 0x7f6efc003fe0 2023-10-09T06:36:40.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.532+0000 7f6f1a611700 1 -- 172.21.15.163:0/4292909998 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f140ceae0 con 0x7f6efc003fe0 2023-10-09T06:36:40.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.533+0000 7f6f1a611700 1 -- 172.21.15.163:0/4292909998 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f1404c3c0 con 0x7f6efc003fe0 2023-10-09T06:36:40.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.533+0000 7f6efbfff700 1 -- 172.21.15.163:0/4292909998 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f6f04014a80 con 0x7f6efc003fe0 2023-10-09T06:36:40.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.533+0000 7f6efbfff700 1 --2- 172.21.15.163:0/4292909998 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6f00067290 0x7f6f00069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:40.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.533+0000 7f6efbfff700 1 -- 172.21.15.163:0/4292909998 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6f04018030 con 0x7f6efc003fe0 2023-10-09T06:36:40.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.534+0000 7f6f137fe700 1 --2- 172.21.15.163:0/4292909998 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6f00067290 0x7f6f00069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:40.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.534+0000 7f6f137fe700 1 --2- 172.21.15.163:0/4292909998 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6f00067290 0x7f6f00069740 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6f141540a0 tx=0x7f6f08015000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:40.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.538+0000 7f6efbfff700 1 -- 172.21.15.163:0/4292909998 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f04052400 con 0x7f6efc003fe0 2023-10-09T06:36:40.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.720+0000 7f6f1a611700 1 -- 172.21.15.163:0/4292909998 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f1406a5b0 con 0x7f6efc003fe0 2023-10-09T06:36:40.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.721+0000 7f6efbfff700 1 -- 172.21.15.163:0/4292909998 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+12183 (secure 0 0 0) 0x7f6f04022090 con 0x7f6efc003fe0 2023-10-09T06:36:40.722 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:40.722 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":97,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:36:39.976992+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:36:40.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.723+0000 7f6f1a611700 1 -- 172.21.15.163:0/4292909998 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6f00067290 msgr2=0x7f6f00069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:40.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.723+0000 7f6f1a611700 1 --2- 172.21.15.163:0/4292909998 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6f00067290 0x7f6f00069740 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6f141540a0 tx=0x7f6f08015000 comp rx=0 tx=0).stop 2023-10-09T06:36:40.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.723+0000 7f6f1a611700 1 -- 172.21.15.163:0/4292909998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6efc003fe0 msgr2=0x7f6f14152070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:40.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.723+0000 7f6f1a611700 1 --2- 172.21.15.163:0/4292909998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6efc003fe0 0x7f6f14152070 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f6f04011d70 tx=0x7f6f040091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:40.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.723+0000 7f6f1a611700 1 -- 172.21.15.163:0/4292909998 shutdown_connections 2023-10-09T06:36:40.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.723+0000 7f6f1a611700 1 --2- 172.21.15.163:0/4292909998 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6f00067290 0x7f6f00069740 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:40.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.723+0000 7f6f1a611700 1 --2- 172.21.15.163:0/4292909998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6efc003fe0 0x7f6f14152070 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:40.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.723+0000 7f6f1a611700 1 --2- 172.21.15.163:0/4292909998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f140ce150 0x7f6f140ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:40.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.723+0000 7f6f1a611700 1 --2- 172.21.15.163:0/4292909998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f140c80b0 0x7f6f140c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:40.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.723+0000 7f6f1a611700 1 -- 172.21.15.163:0/4292909998 >> 172.21.15.163:0/4292909998 conn(0x7f6f14068e20 msgr2=0x7f6f141545f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:40.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.723+0000 7f6f1a611700 1 -- 172.21.15.163:0/4292909998 shutdown_connections 2023-10-09T06:36:40.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.723+0000 7f6f1a611700 1 -- 172.21.15.163:0/4292909998 wait complete. 2023-10-09T06:36:40.736 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-09T06:36:40.737 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:36:40.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.854+0000 7f1766da1700 1 Processor -- start 2023-10-09T06:36:40.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.854+0000 7f1766da1700 1 -- start start 2023-10-09T06:36:40.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.855+0000 7f1766da1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1760157680 0x7f17600cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:40.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.855+0000 7f1766da1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f17600b1450 0x7f17600c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:40.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.855+0000 7f1766da1700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17600cd240 con 0x7f1760157a50 2023-10-09T06:36:40.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.855+0000 7f1766da1700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17600cc700 con 0x7f17600b1450 2023-10-09T06:36:40.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.855+0000 7f1766da1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17600c9a20 con 0x7f1760157680 2023-10-09T06:36:40.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.855+0000 7f17665a0700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f17600b1450 0x7f17600c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:40.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.855+0000 7f176559e700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f1760157a50 0x7f17600c8cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:37892/0 (socket says 172.21.15.163:37892) 2023-10-09T06:36:40.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.855+0000 7f176559e700 1 -- 172.21.15.163:0/918963145 learned_addr learned my addr 172.21.15.163:0/918963145 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:40.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.855+0000 7f1765d9f700 1 --2- 172.21.15.163:0/918963145 >> v2:172.21.15.181:3300/0 conn(0x7f1760157680 0x7f17600cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:40.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.856+0000 7f17665a0700 1 -- 172.21.15.163:0/918963145 >> v2:172.21.15.181:3300/0 conn(0x7f1760157680 msgr2=0x7f17600cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:40.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.856+0000 7f17665a0700 1 --2- 172.21.15.163:0/918963145 >> v2:172.21.15.181:3300/0 conn(0x7f1760157680 0x7f17600cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:40.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.856+0000 7f17665a0700 1 -- 172.21.15.163:0/918963145 >> v1:172.21.15.163:6789/0 conn(0x7f1760157a50 legacy=0x7f17600c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:40.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.856+0000 7f17665a0700 1 -- 172.21.15.163:0/918963145 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17600cc910 con 0x7f17600b1450 2023-10-09T06:36:40.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.856+0000 7f1764d9d700 1 -- 172.21.15.163:0/918963145 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1378888723 0 0) 0x7f17600cd240 con 0x7f1760157a50 2023-10-09T06:36:40.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.856+0000 7f17665a0700 1 --2- 172.21.15.163:0/918963145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17600b1450 0x7f17600c93e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f17600cbf60 tx=0x7f1758009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e964d9b012c972 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:40.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.856+0000 7f1764d9d700 1 -- 172.21.15.163:0/918963145 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175800e070 con 0x7f17600b1450 2023-10-09T06:36:40.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.856+0000 7f1764d9d700 1 -- 172.21.15.163:0/918963145 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175800bb60 con 0x7f17600b1450 2023-10-09T06:36:40.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.856+0000 7f1764d9d700 1 -- 172.21.15.163:0/918963145 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1758012430 con 0x7f17600b1450 2023-10-09T06:36:40.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.856+0000 7f1766da1700 1 -- 172.21.15.163:0/918963145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17600b1450 msgr2=0x7f17600c93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:40.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.856+0000 7f1766da1700 1 --2- 172.21.15.163:0/918963145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17600b1450 0x7f17600c93e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f17600cbf60 tx=0x7f1758009780 comp rx=0 tx=0).stop 2023-10-09T06:36:40.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.856+0000 7f1766da1700 1 -- 172.21.15.163:0/918963145 shutdown_connections 2023-10-09T06:36:40.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.856+0000 7f1766da1700 1 --2- 172.21.15.163:0/918963145 >> v2:172.21.15.181:3300/0 conn(0x7f1760157680 0x7f17600cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:40.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.856+0000 7f1766da1700 1 --2- 172.21.15.163:0/918963145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17600b1450 0x7f17600c93e0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:40.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.856+0000 7f1766da1700 1 -- 172.21.15.163:0/918963145 >> 172.21.15.163:0/918963145 conn(0x7f176006baa0 msgr2=0x7f176006bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:40.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.857+0000 7f1766da1700 1 -- 172.21.15.163:0/918963145 shutdown_connections 2023-10-09T06:36:40.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.857+0000 7f1766da1700 1 -- 172.21.15.163:0/918963145 wait complete. 2023-10-09T06:36:40.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.858+0000 7f1766da1700 1 Processor -- start 2023-10-09T06:36:40.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.858+0000 7f1766da1700 1 -- start start 2023-10-09T06:36:40.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.858+0000 7f1766da1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17600b1450 0x7f1760168620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:40.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.858+0000 7f1766da1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1760157680 0x7f1760150e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:40.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.858+0000 7f1766da1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1760157a50 0x7f17600cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:40.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.858+0000 7f1766da1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1760168b60 con 0x7f1760157680 2023-10-09T06:36:40.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.858+0000 7f1766da1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1760159500 con 0x7f1760157a50 2023-10-09T06:36:40.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.858+0000 7f1766da1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17601597e0 con 0x7f17600b1450 2023-10-09T06:36:40.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.858+0000 7f176559e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1760157680 0x7f1760150e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:40.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.859+0000 7f176559e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1760157680 0x7f1760150e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41270/0 (socket says 172.21.15.163:41270) 2023-10-09T06:36:40.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.859+0000 7f176559e700 1 -- 172.21.15.163:0/3092842134 learned_addr learned my addr 172.21.15.163:0/3092842134 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:40.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.859+0000 7f17665a0700 1 --2- 172.21.15.163:0/3092842134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1760157a50 0x7f17600cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:40.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.859+0000 7f1765d9f700 1 --2- 172.21.15.163:0/3092842134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17600b1450 0x7f1760168620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:40.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.859+0000 7f176559e700 1 -- 172.21.15.163:0/3092842134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17600b1450 msgr2=0x7f1760168620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:40.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.859+0000 7f176559e700 1 --2- 172.21.15.163:0/3092842134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17600b1450 0x7f1760168620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:40.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.859+0000 7f176559e700 1 -- 172.21.15.163:0/3092842134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1760157a50 msgr2=0x7f17600cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:40.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.859+0000 7f176559e700 1 --2- 172.21.15.163:0/3092842134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1760157a50 0x7f17600cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:40.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.859+0000 7f176559e700 1 -- 172.21.15.163:0/3092842134 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1758009a20 con 0x7f1760157680 2023-10-09T06:36:40.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.859+0000 7f1765d9f700 1 --2- 172.21.15.163:0/3092842134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17600b1450 0x7f1760168620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:40.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.859+0000 7f176559e700 1 --2- 172.21.15.163:0/3092842134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1760157680 0x7f1760150e30 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f175000c5e0 tx=0x7f1750000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:40.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.859+0000 7f1755ffb700 1 -- 172.21.15.163:0/3092842134 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1750005b30 con 0x7f1760157680 2023-10-09T06:36:40.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.859+0000 7f1755ffb700 1 -- 172.21.15.163:0/3092842134 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1750005c90 con 0x7f1760157680 2023-10-09T06:36:40.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.859+0000 7f1755ffb700 1 -- 172.21.15.163:0/3092842134 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17500072e0 con 0x7f1760157680 2023-10-09T06:36:40.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.859+0000 7f1766da1700 1 -- 172.21.15.163:0/3092842134 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1760159ac0 con 0x7f1760157680 2023-10-09T06:36:40.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.859+0000 7f1766da1700 1 -- 172.21.15.163:0/3092842134 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176015a080 con 0x7f1760157680 2023-10-09T06:36:40.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.860+0000 7f1766da1700 1 -- 172.21.15.163:0/3092842134 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f176006eba0 con 0x7f1760157680 2023-10-09T06:36:40.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.860+0000 7f1755ffb700 1 -- 172.21.15.163:0/3092842134 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f175001b3b0 con 0x7f1760157680 2023-10-09T06:36:40.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.861+0000 7f1755ffb700 1 --2- 172.21.15.163:0/3092842134 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1744067290 0x7f1744069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:40.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.861+0000 7f1755ffb700 1 -- 172.21.15.163:0/3092842134 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f1750083b70 con 0x7f1760157680 2023-10-09T06:36:40.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.861+0000 7f1765d9f700 1 --2- 172.21.15.163:0/3092842134 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1744067290 0x7f1744069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:40.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.861+0000 7f1765d9f700 1 --2- 172.21.15.163:0/3092842134 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1744067290 0x7f1744069740 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f175c009d80 tx=0x7f175c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:40.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:40.865+0000 7f1755ffb700 1 -- 172.21.15.163:0/3092842134 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f175004b5a0 con 0x7f1760157680 2023-10-09T06:36:41.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.049+0000 7f1766da1700 1 -- 172.21.15.163:0/3092842134 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f17600ccbf0 con 0x7f1760157680 2023-10-09T06:36:41.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.049+0000 7f1755ffb700 1 -- 172.21.15.163:0/3092842134 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) v1 ==== 93+0+10 (secure 0 0 0) 0x7f17500170a0 con 0x7f1760157680 2023-10-09T06:36:41.050 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:36:41.052 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.051+0000 7f1766da1700 1 -- 172.21.15.163:0/3092842134 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1744067290 msgr2=0x7f1744069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:41.052 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.051+0000 7f1766da1700 1 --2- 172.21.15.163:0/3092842134 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1744067290 0x7f1744069740 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f175c009d80 tx=0x7f175c008040 comp rx=0 tx=0).stop 2023-10-09T06:36:41.052 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.051+0000 7f1766da1700 1 -- 172.21.15.163:0/3092842134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1760157680 msgr2=0x7f1760150e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:41.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.051+0000 7f1766da1700 1 --2- 172.21.15.163:0/3092842134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1760157680 0x7f1760150e30 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f175000c5e0 tx=0x7f1750000fe0 comp rx=0 tx=0).stop 2023-10-09T06:36:41.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.051+0000 7f1766da1700 1 -- 172.21.15.163:0/3092842134 shutdown_connections 2023-10-09T06:36:41.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.051+0000 7f1766da1700 1 --2- 172.21.15.163:0/3092842134 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1744067290 0x7f1744069740 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:41.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.051+0000 7f1766da1700 1 --2- 172.21.15.163:0/3092842134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1760157a50 0x7f17600cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:41.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.051+0000 7f1766da1700 1 --2- 172.21.15.163:0/3092842134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17600b1450 0x7f1760168620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:41.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.051+0000 7f1766da1700 1 --2- 172.21.15.163:0/3092842134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1760157680 0x7f1760150e30 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:41.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.051+0000 7f1766da1700 1 -- 172.21.15.163:0/3092842134 >> 172.21.15.163:0/3092842134 conn(0x7f176006baa0 msgr2=0x7f17600cdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:41.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.051+0000 7f1766da1700 1 -- 172.21.15.163:0/3092842134 shutdown_connections 2023-10-09T06:36:41.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.051+0000 7f1766da1700 1 -- 172.21.15.163:0/3092842134 wait complete. 2023-10-09T06:36:41.065 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-09T06:36:41.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.178+0000 7f7e658be700 1 Processor -- start 2023-10-09T06:36:41.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.178+0000 7f7e658be700 1 -- start start 2023-10-09T06:36:41.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.179+0000 7f7e658be700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7e60159420 0x7f7e600b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:41.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.179+0000 7f7e658be700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7e601597f0 0x7f7e600b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:41.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.179+0000 7f7e658be700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e600b11d0 con 0x7f7e6015d230 2023-10-09T06:36:41.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.179+0000 7f7e658be700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e600b14b0 con 0x7f7e60159420 2023-10-09T06:36:41.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.179+0000 7f7e658be700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e600b1790 con 0x7f7e601597f0 2023-10-09T06:36:41.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.179+0000 7f7e648bc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7e60159420 0x7f7e600b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:41.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.179+0000 7f7e648bc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7e60159420 0x7f7e600b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41280/0 (socket says 172.21.15.163:41280) 2023-10-09T06:36:41.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.179+0000 7f7e648bc700 1 -- 172.21.15.163:0/1256508787 learned_addr learned my addr 172.21.15.163:0/1256508787 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:41.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.179+0000 7f7e5ffff700 1 --2- 172.21.15.163:0/1256508787 >> v2:172.21.15.181:3300/0 conn(0x7f7e601597f0 0x7f7e600b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:41.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.179+0000 7f7e648bc700 1 -- 172.21.15.163:0/1256508787 >> v2:172.21.15.181:3300/0 conn(0x7f7e601597f0 msgr2=0x7f7e600b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:41.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.179+0000 7f7e648bc700 1 --2- 172.21.15.163:0/1256508787 >> v2:172.21.15.181:3300/0 conn(0x7f7e601597f0 0x7f7e600b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:41.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.179+0000 7f7e648bc700 1 -- 172.21.15.163:0/1256508787 >> v1:172.21.15.181:6789/0 conn(0x7f7e6015d230 legacy=0x7f7e6015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:41.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.179+0000 7f7e648bc700 1 -- 172.21.15.163:0/1256508787 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e600b1a70 con 0x7f7e60159420 2023-10-09T06:36:41.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.179+0000 7f7e648bc700 1 --2- 172.21.15.163:0/1256508787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e60159420 0x7f7e600b5f40 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f7e60089750 tx=0x7f7e58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe067fdc4152a1f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:41.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.180+0000 7f7e5f7fe700 1 -- 172.21.15.163:0/1256508787 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5800b820 con 0x7f7e60159420 2023-10-09T06:36:41.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.180+0000 7f7e5f7fe700 1 -- 172.21.15.163:0/1256508787 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5800b980 con 0x7f7e60159420 2023-10-09T06:36:41.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.180+0000 7f7e5f7fe700 1 -- 172.21.15.163:0/1256508787 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5800bd20 con 0x7f7e60159420 2023-10-09T06:36:41.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.180+0000 7f7e658be700 1 -- 172.21.15.163:0/1256508787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e60159420 msgr2=0x7f7e600b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:41.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.180+0000 7f7e658be700 1 --2- 172.21.15.163:0/1256508787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e60159420 0x7f7e600b5f40 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f7e60089750 tx=0x7f7e58009250 comp rx=0 tx=0).stop 2023-10-09T06:36:41.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.180+0000 7f7e658be700 1 -- 172.21.15.163:0/1256508787 shutdown_connections 2023-10-09T06:36:41.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.180+0000 7f7e658be700 1 --2- 172.21.15.163:0/1256508787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e60159420 0x7f7e600b5f40 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:41.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.180+0000 7f7e658be700 1 --2- 172.21.15.163:0/1256508787 >> v2:172.21.15.181:3300/0 conn(0x7f7e601597f0 0x7f7e600b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:41.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.180+0000 7f7e658be700 1 -- 172.21.15.163:0/1256508787 >> 172.21.15.163:0/1256508787 conn(0x7f7e600d0660 msgr2=0x7f7e600d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:41.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.180+0000 7f7e658be700 1 -- 172.21.15.163:0/1256508787 shutdown_connections 2023-10-09T06:36:41.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.180+0000 7f7e658be700 1 -- 172.21.15.163:0/1256508787 wait complete. 2023-10-09T06:36:41.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.181+0000 7f7e658be700 1 Processor -- start 2023-10-09T06:36:41.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.181+0000 7f7e658be700 1 -- start start 2023-10-09T06:36:41.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.182+0000 7f7e658be700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e60159420 0x7f7e6015a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:41.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.182+0000 7f7e658be700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e601597f0 0x7f7e6016d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:41.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.182+0000 7f7e658be700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e6015d230 0x7f7e6016d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:41.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.182+0000 7f7e658be700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6016b3c0 con 0x7f7e601597f0 2023-10-09T06:36:41.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.182+0000 7f7e658be700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6015af90 con 0x7f7e60159420 2023-10-09T06:36:41.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.182+0000 7f7e658be700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6015b270 con 0x7f7e6015d230 2023-10-09T06:36:41.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.182+0000 7f7e5ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e601597f0 0x7f7e6016d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:41.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.182+0000 7f7e5ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e601597f0 0x7f7e6016d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41282/0 (socket says 172.21.15.163:41282) 2023-10-09T06:36:41.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.182+0000 7f7e5ffff700 1 -- 172.21.15.163:0/2857902883 learned_addr learned my addr 172.21.15.163:0/2857902883 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:41.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.182+0000 7f7e5ffff700 1 -- 172.21.15.163:0/2857902883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e6015d230 msgr2=0x7f7e6016d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:41.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.182+0000 7f7e648bc700 1 --2- 172.21.15.163:0/2857902883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e60159420 0x7f7e6015a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:41.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.182+0000 7f7e650bd700 1 --2- 172.21.15.163:0/2857902883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e6015d230 0x7f7e6016d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:41.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.183+0000 7f7e5ffff700 1 --2- 172.21.15.163:0/2857902883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e6015d230 0x7f7e6016d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:41.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.183+0000 7f7e5ffff700 1 -- 172.21.15.163:0/2857902883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e60159420 msgr2=0x7f7e6015a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:41.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.183+0000 7f7e5ffff700 1 --2- 172.21.15.163:0/2857902883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e60159420 0x7f7e6015a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:41.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.183+0000 7f7e5ffff700 1 -- 172.21.15.163:0/2857902883 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e580094f0 con 0x7f7e601597f0 2023-10-09T06:36:41.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.183+0000 7f7e5ffff700 1 --2- 172.21.15.163:0/2857902883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e601597f0 0x7f7e6016d3e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000cfc0 tx=0x7f7e50007880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:41.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.183+0000 7f7e5cff9700 1 -- 172.21.15.163:0/2857902883 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e50016430 con 0x7f7e601597f0 2023-10-09T06:36:41.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.183+0000 7f7e5cff9700 1 -- 172.21.15.163:0/2857902883 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e50016590 con 0x7f7e601597f0 2023-10-09T06:36:41.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.183+0000 7f7e658be700 1 -- 172.21.15.163:0/2857902883 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e6015b550 con 0x7f7e601597f0 2023-10-09T06:36:41.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.183+0000 7f7e5cff9700 1 -- 172.21.15.163:0/2857902883 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e50016930 con 0x7f7e601597f0 2023-10-09T06:36:41.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.183+0000 7f7e658be700 1 -- 172.21.15.163:0/2857902883 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6015aba0 con 0x7f7e601597f0 2023-10-09T06:36:41.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.184+0000 7f7e658be700 1 -- 172.21.15.163:0/2857902883 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e6016b530 con 0x7f7e601597f0 2023-10-09T06:36:41.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.184+0000 7f7e5cff9700 1 -- 172.21.15.163:0/2857902883 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f7e50003020 con 0x7f7e601597f0 2023-10-09T06:36:41.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.185+0000 7f7e5cff9700 1 --2- 172.21.15.163:0/2857902883 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7e40067290 0x7f7e40069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:41.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.185+0000 7f7e5cff9700 1 -- 172.21.15.163:0/2857902883 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f7e50012070 con 0x7f7e601597f0 2023-10-09T06:36:41.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.185+0000 7f7e648bc700 1 --2- 172.21.15.163:0/2857902883 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7e40067290 0x7f7e40069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:41.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.185+0000 7f7e648bc700 1 --2- 172.21.15.163:0/2857902883 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7e40067290 0x7f7e40069740 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7e60089750 tx=0x7f7e58016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:41.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.189+0000 7f7e5cff9700 1 -- 172.21.15.163:0/2857902883 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7e50052800 con 0x7f7e601597f0 2023-10-09T06:36:41.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.375+0000 7f7e658be700 1 -- 172.21.15.163:0/2857902883 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7e6004c3c0 con 0x7f7e601597f0 2023-10-09T06:36:41.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.376+0000 7f7e5cff9700 1 -- 172.21.15.163:0/2857902883 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v97) v1 ==== 84+0+65 (secure 0 0 0) 0x7f7e50055e20 con 0x7f7e601597f0 2023-10-09T06:36:41.377 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:41.377 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:36:41.377 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:41.377 INFO:teuthology.orchestra.run.smithi163.stdout: "poolnum": 1, 2023-10-09T06:36:41.377 INFO:teuthology.orchestra.run.smithi163.stdout: "poolname": ".mgr" 2023-10-09T06:36:41.377 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:41.378 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:36:41.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.378+0000 7f7e658be700 1 -- 172.21.15.163:0/2857902883 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7e40067290 msgr2=0x7f7e40069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:41.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.378+0000 7f7e658be700 1 --2- 172.21.15.163:0/2857902883 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7e40067290 0x7f7e40069740 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7e60089750 tx=0x7f7e58016000 comp rx=0 tx=0).stop 2023-10-09T06:36:41.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.378+0000 7f7e658be700 1 -- 172.21.15.163:0/2857902883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e601597f0 msgr2=0x7f7e6016d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:41.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.378+0000 7f7e658be700 1 --2- 172.21.15.163:0/2857902883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e601597f0 0x7f7e6016d3e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000cfc0 tx=0x7f7e50007880 comp rx=0 tx=0).stop 2023-10-09T06:36:41.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.378+0000 7f7e658be700 1 -- 172.21.15.163:0/2857902883 shutdown_connections 2023-10-09T06:36:41.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.378+0000 7f7e658be700 1 --2- 172.21.15.163:0/2857902883 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7e40067290 0x7f7e40069740 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:41.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.378+0000 7f7e658be700 1 --2- 172.21.15.163:0/2857902883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e6015d230 0x7f7e6016d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:41.381 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.378+0000 7f7e658be700 1 --2- 172.21.15.163:0/2857902883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e60159420 0x7f7e6015a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:41.381 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.378+0000 7f7e658be700 1 --2- 172.21.15.163:0/2857902883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e601597f0 0x7f7e6016d3e0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:41.381 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.378+0000 7f7e658be700 1 -- 172.21.15.163:0/2857902883 >> 172.21.15.163:0/2857902883 conn(0x7f7e600d0660 msgr2=0x7f7e60069920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:41.381 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.378+0000 7f7e658be700 1 -- 172.21.15.163:0/2857902883 shutdown_connections 2023-10-09T06:36:41.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.378+0000 7f7e658be700 1 -- 172.21.15.163:0/2857902883 wait complete. 2023-10-09T06:36:41.391 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-09T06:36:41.391 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-09T06:36:41.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.509+0000 7f6ac2e64700 1 Processor -- start 2023-10-09T06:36:41.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.509+0000 7f6ac2e64700 1 -- start start 2023-10-09T06:36:41.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.510+0000 7f6ac2e64700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6abc159ec0 0x7f6abc0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:41.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.510+0000 7f6ac2e64700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6abc0b5b00 con 0x7f6abc0b53d0 2023-10-09T06:36:41.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.510+0000 7f6ac2e64700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6abc0b5850 con 0x7f6abc0cf350 2023-10-09T06:36:41.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.510+0000 7f6ac2e64700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6abc0b5da0 con 0x7f6abc159ec0 2023-10-09T06:36:41.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.510+0000 7f6abbfff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f6abc0b53d0 0x7f6abc0cc900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:37902/0 (socket says 172.21.15.163:37902) 2023-10-09T06:36:41.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.510+0000 7f6abbfff700 1 -- 172.21.15.163:0/4015398257 learned_addr learned my addr 172.21.15.163:0/4015398257 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:41.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.510+0000 7f6ac0c00700 1 --2- 172.21.15.163:0/4015398257 >> v2:172.21.15.181:3300/0 conn(0x7f6abc159ec0 0x7f6abc0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:41.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.511+0000 7f6abb7fe700 1 -- 172.21.15.163:0/4015398257 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2856528429 0 0) 0x7f6abc0b5b00 con 0x7f6abc0b53d0 2023-10-09T06:36:41.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.511+0000 7f6abb7fe700 1 -- 172.21.15.163:0/4015398257 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6aac003570 con 0x7f6abc0b53d0 2023-10-09T06:36:41.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.511+0000 7f6abb7fe700 1 -- 172.21.15.163:0/4015398257 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 141567195 0 0) 0x7f6aac003570 con 0x7f6abc0b53d0 2023-10-09T06:36:41.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.511+0000 7f6ac0c00700 1 -- 172.21.15.163:0/4015398257 >> v1:172.21.15.181:6789/0 conn(0x7f6abc0cf350 legacy=0x7f6abc0cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:41.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.511+0000 7f6ac0c00700 1 -- 172.21.15.163:0/4015398257 >> v1:172.21.15.163:6789/0 conn(0x7f6abc0b53d0 legacy=0x7f6abc0cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:41.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.511+0000 7f6ac0c00700 1 -- 172.21.15.163:0/4015398257 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6abc0b6080 con 0x7f6abc159ec0 2023-10-09T06:36:41.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.511+0000 7f6ac0c00700 1 --2- 172.21.15.163:0/4015398257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6abc159ec0 0x7f6abc0b4e90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6ab00098d0 tx=0x7f6ab0009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be9e804a357acac server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:41.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.511+0000 7f6abb7fe700 1 -- 172.21.15.163:0/4015398257 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8001470 con 0x7f6abc159ec0 2023-10-09T06:36:41.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.511+0000 7f6abb7fe700 1 -- 172.21.15.163:0/4015398257 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab0011420 con 0x7f6abc159ec0 2023-10-09T06:36:41.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.511+0000 7f6abb7fe700 1 -- 172.21.15.163:0/4015398257 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab00117c0 con 0x7f6abc159ec0 2023-10-09T06:36:41.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.512+0000 7f6ac2e64700 1 -- 172.21.15.163:0/4015398257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6abc159ec0 msgr2=0x7f6abc0b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:41.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.512+0000 7f6ac2e64700 1 --2- 172.21.15.163:0/4015398257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6abc159ec0 0x7f6abc0b4e90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6ab00098d0 tx=0x7f6ab0009120 comp rx=0 tx=0).stop 2023-10-09T06:36:41.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.512+0000 7f6ac2e64700 1 -- 172.21.15.163:0/4015398257 shutdown_connections 2023-10-09T06:36:41.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.512+0000 7f6ac2e64700 1 --2- 172.21.15.163:0/4015398257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6abc159ec0 0x7f6abc0b4e90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:41.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.512+0000 7f6ac2e64700 1 -- 172.21.15.163:0/4015398257 >> 172.21.15.163:0/4015398257 conn(0x7f6abc0999f0 msgr2=0x7f6abc0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:41.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.512+0000 7f6ac2e64700 1 -- 172.21.15.163:0/4015398257 shutdown_connections 2023-10-09T06:36:41.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.512+0000 7f6ac2e64700 1 -- 172.21.15.163:0/4015398257 wait complete. 2023-10-09T06:36:41.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.513+0000 7f6ac2e64700 1 Processor -- start 2023-10-09T06:36:41.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.513+0000 7f6ac2e64700 1 -- start start 2023-10-09T06:36:41.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.514+0000 7f6ac2e64700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6abc0b53d0 0x7f6abc09d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:41.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.514+0000 7f6ac2e64700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6abc0cf350 0x7f6abc09a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:41.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.514+0000 7f6ac2e64700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6abc159ec0 0x7f6abc169fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:41.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.514+0000 7f6ac2e64700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc09ab60 con 0x7f6abc0cf350 2023-10-09T06:36:41.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.514+0000 7f6ac2e64700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc0c8c60 con 0x7f6abc159ec0 2023-10-09T06:36:41.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.514+0000 7f6ac2e64700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc09db00 con 0x7f6abc0b53d0 2023-10-09T06:36:41.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.514+0000 7f6abbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6abc0cf350 0x7f6abc09a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:41.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.514+0000 7f6abbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6abc0cf350 0x7f6abc09a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41298/0 (socket says 172.21.15.163:41298) 2023-10-09T06:36:41.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.514+0000 7f6abbfff700 1 -- 172.21.15.163:0/3076533569 learned_addr learned my addr 172.21.15.163:0/3076533569 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:41.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.514+0000 7f6ac1401700 1 --2- 172.21.15.163:0/3076533569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6abc159ec0 0x7f6abc169fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:41.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.514+0000 7f6ac0c00700 1 --2- 172.21.15.163:0/3076533569 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6abc0b53d0 0x7f6abc09d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:41.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.514+0000 7f6abbfff700 1 -- 172.21.15.163:0/3076533569 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6abc0b53d0 msgr2=0x7f6abc09d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:41.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.514+0000 7f6abbfff700 1 --2- 172.21.15.163:0/3076533569 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6abc0b53d0 0x7f6abc09d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:41.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.514+0000 7f6abbfff700 1 -- 172.21.15.163:0/3076533569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6abc159ec0 msgr2=0x7f6abc169fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:41.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.514+0000 7f6abbfff700 1 --2- 172.21.15.163:0/3076533569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6abc159ec0 0x7f6abc169fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:41.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.514+0000 7f6abbfff700 1 -- 172.21.15.163:0/3076533569 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab00093c0 con 0x7f6abc0cf350 2023-10-09T06:36:41.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.514+0000 7f6abbfff700 1 --2- 172.21.15.163:0/3076533569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6abc0cf350 0x7f6abc09a620 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f6aa800cb90 tx=0x7f6aa8001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:41.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.515+0000 7f6ab8ff9700 1 -- 172.21.15.163:0/3076533569 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8007150 con 0x7f6abc0cf350 2023-10-09T06:36:41.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.515+0000 7f6ab8ff9700 1 -- 172.21.15.163:0/3076533569 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa80072d0 con 0x7f6abc0cf350 2023-10-09T06:36:41.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.515+0000 7f6ac2e64700 1 -- 172.21.15.163:0/3076533569 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6abc09dde0 con 0x7f6abc0cf350 2023-10-09T06:36:41.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.515+0000 7f6ab8ff9700 1 -- 172.21.15.163:0/3076533569 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8007690 con 0x7f6abc0cf350 2023-10-09T06:36:41.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.515+0000 7f6ac2e64700 1 -- 172.21.15.163:0/3076533569 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6abc14e850 con 0x7f6abc0cf350 2023-10-09T06:36:41.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.516+0000 7f6ac2e64700 1 -- 172.21.15.163:0/3076533569 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6abc09e300 con 0x7f6abc0cf350 2023-10-09T06:36:41.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.516+0000 7f6ab8ff9700 1 -- 172.21.15.163:0/3076533569 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f6aa8004a40 con 0x7f6abc0cf350 2023-10-09T06:36:41.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.516+0000 7f6ab8ff9700 1 --2- 172.21.15.163:0/3076533569 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6aa0067290 0x7f6aa0069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:41.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.516+0000 7f6ab8ff9700 1 -- 172.21.15.163:0/3076533569 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6aa8083fc0 con 0x7f6abc0cf350 2023-10-09T06:36:41.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.517+0000 7f6ac0c00700 1 --2- 172.21.15.163:0/3076533569 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6aa0067290 0x7f6aa0069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:41.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.517+0000 7f6ac0c00700 1 --2- 172.21.15.163:0/3076533569 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6aa0067290 0x7f6aa0069740 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009390 tx=0x7f6ab0016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:41.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.520+0000 7f6ab8ff9700 1 -- 172.21.15.163:0/3076533569 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6abc09e300 con 0x7f6abc0cf350 2023-10-09T06:36:41.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.704+0000 7f6ac2e64700 1 -- 172.21.15.163:0/3076533569 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6abc09e300 con 0x7f6abc0cf350 2023-10-09T06:36:41.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.989+0000 7f6ab8ff9700 1 -- 172.21.15.163:0/3076533569 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v98) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6abc09e300 con 0x7f6abc0cf350 2023-10-09T06:36:41.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.992+0000 7f6ac2e64700 1 -- 172.21.15.163:0/3076533569 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6aa0067290 msgr2=0x7f6aa0069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:41.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.992+0000 7f6ac2e64700 1 --2- 172.21.15.163:0/3076533569 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6aa0067290 0x7f6aa0069740 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009390 tx=0x7f6ab0016000 comp rx=0 tx=0).stop 2023-10-09T06:36:41.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.992+0000 7f6ac2e64700 1 -- 172.21.15.163:0/3076533569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6abc0cf350 msgr2=0x7f6abc09a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:41.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.992+0000 7f6ac2e64700 1 --2- 172.21.15.163:0/3076533569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6abc0cf350 0x7f6abc09a620 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f6aa800cb90 tx=0x7f6aa8001a30 comp rx=0 tx=0).stop 2023-10-09T06:36:41.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.992+0000 7f6ac2e64700 1 -- 172.21.15.163:0/3076533569 shutdown_connections 2023-10-09T06:36:41.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.992+0000 7f6ac2e64700 1 --2- 172.21.15.163:0/3076533569 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6aa0067290 0x7f6aa0069740 secure :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009390 tx=0x7f6ab0016000 comp rx=0 tx=0).stop 2023-10-09T06:36:41.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.992+0000 7f6ac2e64700 1 --2- 172.21.15.163:0/3076533569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6abc159ec0 0x7f6abc169fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:41.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.992+0000 7f6ac2e64700 1 --2- 172.21.15.163:0/3076533569 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6abc0b53d0 0x7f6abc09d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:41.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.992+0000 7f6ac2e64700 1 --2- 172.21.15.163:0/3076533569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6abc0cf350 0x7f6abc09a620 secure :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f6aa800cb90 tx=0x7f6aa8001a30 comp rx=0 tx=0).stop 2023-10-09T06:36:41.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.992+0000 7f6ac2e64700 1 -- 172.21.15.163:0/3076533569 >> 172.21.15.163:0/3076533569 conn(0x7f6abc0999f0 msgr2=0x7f6abc0c5610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:41.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.994+0000 7f6ac2e64700 1 -- 172.21.15.163:0/3076533569 shutdown_connections 2023-10-09T06:36:41.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:41.995+0000 7f6ac2e64700 1 -- 172.21.15.163:0/3076533569 wait complete. 2023-10-09T06:36:41.997 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_metadata' created 2023-10-09T06:36:42.009 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-09T06:36:42.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.131+0000 7f90b6960700 1 Processor -- start 2023-10-09T06:36:42.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.131+0000 7f90b6960700 1 -- start start 2023-10-09T06:36:42.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.132+0000 7f90b6960700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90b0069920 0x7f90b0157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:42.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.132+0000 7f90b6960700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f90b00af1c0 0x7f90b00cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:42.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.132+0000 7f90b6960700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b0063830 con 0x7f90b00cc4a0 2023-10-09T06:36:42.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.132+0000 7f90b6960700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90b00cc920 con 0x7f90b00af1c0 2023-10-09T06:36:42.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.132+0000 7f90b6960700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b00c90e0 con 0x7f90b0069920 2023-10-09T06:36:42.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.132+0000 7f90b4efd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f90b00cc4a0 0x7f90b015a340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:37916/0 (socket says 172.21.15.163:37916) 2023-10-09T06:36:42.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.132+0000 7f90b4efd700 1 -- 172.21.15.163:0/2171533168 learned_addr learned my addr 172.21.15.163:0/2171533168 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:42.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.133+0000 7f90aeffd700 1 -- 172.21.15.163:0/2171533168 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2284220532 0 0) 0x7f90b0063830 con 0x7f90b00cc4a0 2023-10-09T06:36:42.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.133+0000 7f90aeffd700 1 -- 172.21.15.163:0/2171533168 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9098003570 con 0x7f90b00cc4a0 2023-10-09T06:36:42.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.133+0000 7f90affff700 1 --2- 172.21.15.163:0/2171533168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90b0069920 0x7f90b0157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:42.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.133+0000 7f90af7fe700 1 --2- 172.21.15.163:0/2171533168 >> v2:172.21.15.181:3300/0 conn(0x7f90b00af1c0 0x7f90b00cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:42.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.133+0000 7f90aeffd700 1 -- 172.21.15.163:0/2171533168 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 621145699 0 0) 0x7f9098003570 con 0x7f90b00cc4a0 2023-10-09T06:36:42.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.133+0000 7f90aeffd700 1 -- 172.21.15.163:0/2171533168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90b0069920 msgr2=0x7f90b0157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:42.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.133+0000 7f90aeffd700 1 --2- 172.21.15.163:0/2171533168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90b0069920 0x7f90b0157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:42.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.133+0000 7f90aeffd700 1 -- 172.21.15.163:0/2171533168 >> v2:172.21.15.181:3300/0 conn(0x7f90b00af1c0 msgr2=0x7f90b00cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:42.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.133+0000 7f90aeffd700 1 --2- 172.21.15.163:0/2171533168 >> v2:172.21.15.181:3300/0 conn(0x7f90b00af1c0 0x7f90b00cbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:42.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.133+0000 7f90aeffd700 1 -- 172.21.15.163:0/2171533168 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b0157cb0 con 0x7f90b00cc4a0 2023-10-09T06:36:42.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.133+0000 7f90aeffd700 1 -- 172.21.15.163:0/2171533168 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f90a80017f0 con 0x7f90b00cc4a0 2023-10-09T06:36:42.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.133+0000 7f90aeffd700 1 -- 172.21.15.163:0/2171533168 >> v1:172.21.15.163:6789/0 conn(0x7f90b00cc4a0 legacy=0x7f90b015a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:42.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.133+0000 7f90aeffd700 1 --2- 172.21.15.163:0/2171533168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9098003a60 0x7f9098003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:42.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.133+0000 7f90aeffd700 1 --2- 172.21.15.163:0/2171533168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90b0069920 0x7f9098004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:42.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.133+0000 7f90aeffd700 1 --2- 172.21.15.163:0/2171533168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9098004d50 0x7f90980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:42.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.133+0000 7f90aeffd700 1 -- 172.21.15.163:0/2171533168 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b00cc920 con 0x7f9098004d50 2023-10-09T06:36:42.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.133+0000 7f90aeffd700 1 -- 172.21.15.163:0/2171533168 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b00c90e0 con 0x7f9098003a60 2023-10-09T06:36:42.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.133+0000 7f90aeffd700 1 -- 172.21.15.163:0/2171533168 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f909800ba60 con 0x7f90b0069920 2023-10-09T06:36:42.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.133+0000 7f90b4efd700 1 --2- 172.21.15.163:0/2171533168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9098004d50 0x7f90980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:42.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.133+0000 7f90affff700 1 --2- 172.21.15.163:0/2171533168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9098003a60 0x7f9098003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:42.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.133+0000 7f90af7fe700 1 --2- 172.21.15.163:0/2171533168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90b0069920 0x7f9098004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:42.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.133+0000 7f90b4efd700 1 -- 172.21.15.163:0/2171533168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90b0069920 msgr2=0x7f9098004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:42.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.133+0000 7f90b4efd700 1 --2- 172.21.15.163:0/2171533168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90b0069920 0x7f9098004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:42.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.133+0000 7f90b4efd700 1 -- 172.21.15.163:0/2171533168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9098003a60 msgr2=0x7f9098003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:42.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.133+0000 7f90b4efd700 1 --2- 172.21.15.163:0/2171533168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9098003a60 0x7f9098003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:42.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.133+0000 7f90b4efd700 1 -- 172.21.15.163:0/2171533168 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9098003570 con 0x7f9098004d50 2023-10-09T06:36:42.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.133+0000 7f90b4efd700 1 --2- 172.21.15.163:0/2171533168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9098004d50 0x7f90980051c0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f90a800a970 tx=0x7f90a8002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3393df3a13f9ecf8 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:42.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.134+0000 7f90aeffd700 1 -- 172.21.15.163:0/2171533168 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a8012440 con 0x7f9098004d50 2023-10-09T06:36:42.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.134+0000 7f90aeffd700 1 -- 172.21.15.163:0/2171533168 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a80125c0 con 0x7f9098004d50 2023-10-09T06:36:42.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.134+0000 7f90aeffd700 1 -- 172.21.15.163:0/2171533168 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a8012980 con 0x7f9098004d50 2023-10-09T06:36:42.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.134+0000 7f90b6960700 1 -- 172.21.15.163:0/2171533168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9098004d50 msgr2=0x7f90980051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:42.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.134+0000 7f90b6960700 1 --2- 172.21.15.163:0/2171533168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9098004d50 0x7f90980051c0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f90a800a970 tx=0x7f90a8002210 comp rx=0 tx=0).stop 2023-10-09T06:36:42.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.134+0000 7f90b4efd700 1 -- 172.21.15.163:0/2171533168 reap_dead start 2023-10-09T06:36:42.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.134+0000 7f90b6960700 1 -- 172.21.15.163:0/2171533168 shutdown_connections 2023-10-09T06:36:42.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.134+0000 7f90b6960700 1 -- 172.21.15.163:0/2171533168 >> 172.21.15.163:0/2171533168 conn(0x7f90b00c9870 msgr2=0x7f90b00c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:42.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.134+0000 7f90b6960700 1 -- 172.21.15.163:0/2171533168 shutdown_connections 2023-10-09T06:36:42.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.134+0000 7f90b6960700 1 -- 172.21.15.163:0/2171533168 wait complete. 2023-10-09T06:36:42.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.135+0000 7f90b6960700 1 Processor -- start 2023-10-09T06:36:42.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.136+0000 7f90b6960700 1 -- start start 2023-10-09T06:36:42.144 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.136+0000 7f90b6960700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90b0143900 0x7f90b00ad710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:42.144 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.136+0000 7f90b6960700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90b014ec20 0x7f90b00adc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:42.144 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.136+0000 7f90b6960700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90b0143cd0 0x7f90b00b0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:42.144 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.136+0000 7f90b6960700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b00afff0 con 0x7f90b014ec20 2023-10-09T06:36:42.144 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.136+0000 7f90b6960700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b00afcc0 con 0x7f90b0143900 2023-10-09T06:36:42.145 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.136+0000 7f90b6960700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b00c3f50 con 0x7f90b0143cd0 2023-10-09T06:36:42.145 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.136+0000 7f90af7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90b014ec20 0x7f90b00adc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:42.145 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.136+0000 7f90af7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90b014ec20 0x7f90b00adc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41330/0 (socket says 172.21.15.163:41330) 2023-10-09T06:36:42.145 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.136+0000 7f90af7fe700 1 -- 172.21.15.163:0/3334248782 learned_addr learned my addr 172.21.15.163:0/3334248782 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:42.145 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.136+0000 7f90affff700 1 --2- 172.21.15.163:0/3334248782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90b0143900 0x7f90b00ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:42.145 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.136+0000 7f90b4efd700 1 --2- 172.21.15.163:0/3334248782 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90b0143cd0 0x7f90b00b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:42.145 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.137+0000 7f90af7fe700 1 -- 172.21.15.163:0/3334248782 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90b0143cd0 msgr2=0x7f90b00b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:42.145 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.137+0000 7f90af7fe700 1 --2- 172.21.15.163:0/3334248782 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90b0143cd0 0x7f90b00b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:42.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.137+0000 7f90af7fe700 1 -- 172.21.15.163:0/3334248782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90b0143900 msgr2=0x7f90b00ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:42.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.137+0000 7f90af7fe700 1 --2- 172.21.15.163:0/3334248782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90b0143900 0x7f90b00ad710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:42.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.137+0000 7f90af7fe700 1 -- 172.21.15.163:0/3334248782 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b00c4230 con 0x7f90b014ec20 2023-10-09T06:36:42.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.137+0000 7f90af7fe700 1 --2- 172.21.15.163:0/3334248782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90b014ec20 0x7f90b00adc50 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f90a0006ac0 tx=0x7f90a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:42.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.137+0000 7f9097fff700 1 -- 172.21.15.163:0/3334248782 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a0011ec0 con 0x7f90b014ec20 2023-10-09T06:36:42.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.137+0000 7f9097fff700 1 -- 172.21.15.163:0/3334248782 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a000f940 con 0x7f90b014ec20 2023-10-09T06:36:42.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.137+0000 7f90b6960700 1 -- 172.21.15.163:0/3334248782 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b00c4510 con 0x7f90b014ec20 2023-10-09T06:36:42.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.137+0000 7f9097fff700 1 -- 172.21.15.163:0/3334248782 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a000fcc0 con 0x7f90b014ec20 2023-10-09T06:36:42.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.137+0000 7f90b6960700 1 -- 172.21.15.163:0/3334248782 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b00b0b80 con 0x7f90b014ec20 2023-10-09T06:36:42.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.138+0000 7f90b6960700 1 -- 172.21.15.163:0/3334248782 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90b00718b0 con 0x7f90b014ec20 2023-10-09T06:36:42.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.138+0000 7f9097fff700 1 -- 172.21.15.163:0/3334248782 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f90a0014650 con 0x7f90b014ec20 2023-10-09T06:36:42.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.139+0000 7f9097fff700 1 --2- 172.21.15.163:0/3334248782 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f909c067290 0x7f909c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:42.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.139+0000 7f9097fff700 1 -- 172.21.15.163:0/3334248782 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f90a0018030 con 0x7f90b014ec20 2023-10-09T06:36:42.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.139+0000 7f90affff700 1 --2- 172.21.15.163:0/3334248782 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f909c067290 0x7f909c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:42.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.140+0000 7f90affff700 1 --2- 172.21.15.163:0/3334248782 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f909c067290 0x7f909c069740 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f90a4006f00 tx=0x7f90a40091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:42.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.143+0000 7f9097fff700 1 -- 172.21.15.163:0/3334248782 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90a0055c30 con 0x7f90b014ec20 2023-10-09T06:36:42.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.324+0000 7f90b6960700 1 -- 172.21.15.163:0/3334248782 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f90b00b2c50 con 0x7f90b014ec20 2023-10-09T06:36:42.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:42.996+0000 7f9097fff700 1 -- 172.21.15.163:0/3334248782 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v99) v1 ==== 170+0+0 (secure 0 0 0) 0x7f90a00557c0 con 0x7f90b014ec20 2023-10-09T06:36:43.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.001+0000 7f90b6960700 1 -- 172.21.15.163:0/3334248782 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f909c067290 msgr2=0x7f909c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:43.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.001+0000 7f90b6960700 1 --2- 172.21.15.163:0/3334248782 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f909c067290 0x7f909c069740 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f90a4006f00 tx=0x7f90a40091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:43.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.001+0000 7f90b6960700 1 -- 172.21.15.163:0/3334248782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90b014ec20 msgr2=0x7f90b00adc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:43.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.001+0000 7f90b6960700 1 --2- 172.21.15.163:0/3334248782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90b014ec20 0x7f90b00adc50 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f90a0006ac0 tx=0x7f90a00091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:43.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.002+0000 7f90b6960700 1 -- 172.21.15.163:0/3334248782 shutdown_connections 2023-10-09T06:36:43.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.002+0000 7f90b6960700 1 --2- 172.21.15.163:0/3334248782 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f909c067290 0x7f909c069740 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:43.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.002+0000 7f90b6960700 1 --2- 172.21.15.163:0/3334248782 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90b0143cd0 0x7f90b00b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:43.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.002+0000 7f90b6960700 1 --2- 172.21.15.163:0/3334248782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90b0143900 0x7f90b00ad710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:43.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.002+0000 7f90b6960700 1 --2- 172.21.15.163:0/3334248782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90b014ec20 0x7f90b00adc50 secure :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f90a0006ac0 tx=0x7f90a00091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:43.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.002+0000 7f90b6960700 1 -- 172.21.15.163:0/3334248782 >> 172.21.15.163:0/3334248782 conn(0x7f90b00c9870 msgr2=0x7f90b014f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:43.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.002+0000 7f90b6960700 1 -- 172.21.15.163:0/3334248782 shutdown_connections 2023-10-09T06:36:43.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.002+0000 7f90b6960700 1 -- 172.21.15.163:0/3334248782 wait complete. 2023-10-09T06:36:43.007 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_data' created 2023-10-09T06:36:43.025 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-09T06:36:43.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.143+0000 7ff31dadb700 1 Processor -- start 2023-10-09T06:36:43.144 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.143+0000 7ff31dadb700 1 -- start start 2023-10-09T06:36:43.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.143+0000 7ff31dadb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff31806c0a0 0x7ff3180caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:43.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.143+0000 7ff31dadb700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff31815d7e0 con 0x7ff31806bcd0 2023-10-09T06:36:43.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.143+0000 7ff31dadb700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff31815dac0 con 0x7ff31815df50 2023-10-09T06:36:43.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.143+0000 7ff31dadb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31815aa30 con 0x7ff31806c0a0 2023-10-09T06:36:43.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.144+0000 7ff3177fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff31806bcd0 0x7ff31815a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:37922/0 (socket says 172.21.15.163:37922) 2023-10-09T06:36:43.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.144+0000 7ff3177fe700 1 -- 172.21.15.163:0/2286035152 learned_addr learned my addr 172.21.15.163:0/2286035152 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:43.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.144+0000 7ff316ffd700 1 --2- 172.21.15.163:0/2286035152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff31806c0a0 0x7ff3180caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:43.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.144+0000 7ff3167fc700 1 -- 172.21.15.163:0/2286035152 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3527579906 0 0) 0x7ff31815d7e0 con 0x7ff31806bcd0 2023-10-09T06:36:43.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.144+0000 7ff3167fc700 1 -- 172.21.15.163:0/2286035152 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff300003570 con 0x7ff31806bcd0 2023-10-09T06:36:43.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.144+0000 7ff3167fc700 1 -- 172.21.15.163:0/2286035152 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2110949683 0 0) 0x7ff31815dac0 con 0x7ff31815df50 2023-10-09T06:36:43.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.144+0000 7ff3167fc700 1 -- 172.21.15.163:0/2286035152 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff31815d7e0 con 0x7ff31815df50 2023-10-09T06:36:43.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.144+0000 7ff3167fc700 1 -- 172.21.15.163:0/2286035152 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1230270671 0 0) 0x7ff300003570 con 0x7ff31806bcd0 2023-10-09T06:36:43.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.144+0000 7ff3167fc700 1 -- 172.21.15.163:0/2286035152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff31806c0a0 msgr2=0x7ff3180caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:43.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.144+0000 7ff3167fc700 1 --2- 172.21.15.163:0/2286035152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff31806c0a0 0x7ff3180caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:43.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.144+0000 7ff3167fc700 1 -- 172.21.15.163:0/2286035152 >> v1:172.21.15.181:6789/0 conn(0x7ff31815df50 legacy=0x7ff3180c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:43.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.144+0000 7ff3167fc700 1 -- 172.21.15.163:0/2286035152 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31815ad10 con 0x7ff31806bcd0 2023-10-09T06:36:43.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.144+0000 7ff3167fc700 1 -- 172.21.15.163:0/2286035152 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7ff3080017f0 con 0x7ff31806bcd0 2023-10-09T06:36:43.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.144+0000 7ff3167fc700 1 -- 172.21.15.163:0/2286035152 >> v1:172.21.15.163:6789/0 conn(0x7ff31806bcd0 legacy=0x7ff31815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:43.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.144+0000 7ff3167fc700 1 --2- 172.21.15.163:0/2286035152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff300003c20 0x7ff300004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:43.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.144+0000 7ff3167fc700 1 --2- 172.21.15.163:0/2286035152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff31806c0a0 0x7ff300004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:43.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.144+0000 7ff3167fc700 1 --2- 172.21.15.163:0/2286035152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff300004d50 0x7ff3000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:43.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.144+0000 7ff3167fc700 1 -- 172.21.15.163:0/2286035152 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31815aa30 con 0x7ff300004d50 2023-10-09T06:36:43.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.144+0000 7ff3167fc700 1 -- 172.21.15.163:0/2286035152 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30000bc40 con 0x7ff300003c20 2023-10-09T06:36:43.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.144+0000 7ff3167fc700 1 -- 172.21.15.163:0/2286035152 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff30000be00 con 0x7ff31806c0a0 2023-10-09T06:36:43.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.144+0000 7ff317fff700 1 --2- 172.21.15.163:0/2286035152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff300004d50 0x7ff3000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:43.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.144+0000 7ff3177fe700 1 --2- 172.21.15.163:0/2286035152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff300003c20 0x7ff300004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:43.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.144+0000 7ff316ffd700 1 --2- 172.21.15.163:0/2286035152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff31806c0a0 0x7ff300004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:43.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.145+0000 7ff317fff700 1 -- 172.21.15.163:0/2286035152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff31806c0a0 msgr2=0x7ff300004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:43.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.145+0000 7ff317fff700 1 --2- 172.21.15.163:0/2286035152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff31806c0a0 0x7ff300004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:43.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.145+0000 7ff317fff700 1 -- 172.21.15.163:0/2286035152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff300003c20 msgr2=0x7ff300004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:43.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.145+0000 7ff317fff700 1 --2- 172.21.15.163:0/2286035152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff300003c20 0x7ff300004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:43.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.145+0000 7ff317fff700 1 -- 172.21.15.163:0/2286035152 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff300003570 con 0x7ff300004d50 2023-10-09T06:36:43.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.145+0000 7ff317fff700 1 --2- 172.21.15.163:0/2286035152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff300004d50 0x7ff3000051c0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ff310001a10 tx=0x7ff310007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d16f9a26a7cf8f server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:43.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.145+0000 7ff3167fc700 1 -- 172.21.15.163:0/2286035152 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31000c5d0 con 0x7ff300004d50 2023-10-09T06:36:43.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.145+0000 7ff3167fc700 1 -- 172.21.15.163:0/2286035152 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31000c730 con 0x7ff300004d50 2023-10-09T06:36:43.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.145+0000 7ff3167fc700 1 -- 172.21.15.163:0/2286035152 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31000cad0 con 0x7ff300004d50 2023-10-09T06:36:43.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.145+0000 7ff31dadb700 1 -- 172.21.15.163:0/2286035152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff300004d50 msgr2=0x7ff3000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:43.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.145+0000 7ff31dadb700 1 --2- 172.21.15.163:0/2286035152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff300004d50 0x7ff3000051c0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ff310001a10 tx=0x7ff310007d30 comp rx=0 tx=0).stop 2023-10-09T06:36:43.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.145+0000 7ff317fff700 1 -- 172.21.15.163:0/2286035152 reap_dead start 2023-10-09T06:36:43.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.145+0000 7ff31dadb700 1 -- 172.21.15.163:0/2286035152 shutdown_connections 2023-10-09T06:36:43.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.145+0000 7ff31dadb700 1 -- 172.21.15.163:0/2286035152 >> 172.21.15.163:0/2286035152 conn(0x7ff3180ce600 msgr2=0x7ff3180cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:43.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.146+0000 7ff31dadb700 1 -- 172.21.15.163:0/2286035152 shutdown_connections 2023-10-09T06:36:43.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.146+0000 7ff31dadb700 1 -- 172.21.15.163:0/2286035152 wait complete. 2023-10-09T06:36:43.153 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.147+0000 7ff31dadb700 1 Processor -- start 2023-10-09T06:36:43.153 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.147+0000 7ff31dadb700 1 -- start start 2023-10-09T06:36:43.153 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.147+0000 7ff31dadb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff300004d50 0x7ff31816a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:43.153 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.147+0000 7ff31dadb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3180b13a0 0x7ff3180b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:43.153 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.147+0000 7ff31dadb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff318153790 0x7ff3180cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:43.153 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.147+0000 7ff31dadb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31816ac50 con 0x7ff3180b13a0 2023-10-09T06:36:43.153 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.147+0000 7ff31dadb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31816aa80 con 0x7ff318153790 2023-10-09T06:36:43.154 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.147+0000 7ff31dadb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31816aea0 con 0x7ff300004d50 2023-10-09T06:36:43.154 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.148+0000 7ff316ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3180b13a0 0x7ff3180b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:43.154 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.148+0000 7ff316ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3180b13a0 0x7ff3180b3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41352/0 (socket says 172.21.15.163:41352) 2023-10-09T06:36:43.154 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.148+0000 7ff316ffd700 1 -- 172.21.15.163:0/3364333332 learned_addr learned my addr 172.21.15.163:0/3364333332 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:43.154 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.148+0000 7ff3177fe700 1 --2- 172.21.15.163:0/3364333332 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff300004d50 0x7ff31816a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:43.154 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.148+0000 7ff317fff700 1 --2- 172.21.15.163:0/3364333332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff318153790 0x7ff3180cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:43.154 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.148+0000 7ff316ffd700 1 -- 172.21.15.163:0/3364333332 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff300004d50 msgr2=0x7ff31816a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:43.154 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.148+0000 7ff316ffd700 1 --2- 172.21.15.163:0/3364333332 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff300004d50 0x7ff31816a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:43.154 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.148+0000 7ff316ffd700 1 -- 172.21.15.163:0/3364333332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff318153790 msgr2=0x7ff3180cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:43.154 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.148+0000 7ff316ffd700 1 --2- 172.21.15.163:0/3364333332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff318153790 0x7ff3180cb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:43.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.148+0000 7ff316ffd700 1 -- 172.21.15.163:0/3364333332 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31816b180 con 0x7ff3180b13a0 2023-10-09T06:36:43.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.148+0000 7ff317fff700 1 --2- 172.21.15.163:0/3364333332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff318153790 0x7ff3180cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:43.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.148+0000 7ff316ffd700 1 --2- 172.21.15.163:0/3364333332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3180b13a0 0x7ff3180b3dc0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ff30c011b10 tx=0x7ff30c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:43.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.148+0000 7ff2ff7fe700 1 -- 172.21.15.163:0/3364333332 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c00f9f0 con 0x7ff3180b13a0 2023-10-09T06:36:43.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.148+0000 7ff2ff7fe700 1 -- 172.21.15.163:0/3364333332 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30c00fb50 con 0x7ff3180b13a0 2023-10-09T06:36:43.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.148+0000 7ff31dadb700 1 -- 172.21.15.163:0/3364333332 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff31816b460 con 0x7ff3180b13a0 2023-10-09T06:36:43.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.148+0000 7ff2ff7fe700 1 -- 172.21.15.163:0/3364333332 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c00fed0 con 0x7ff3180b13a0 2023-10-09T06:36:43.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.148+0000 7ff31dadb700 1 -- 172.21.15.163:0/3364333332 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff31816ba20 con 0x7ff3180b13a0 2023-10-09T06:36:43.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.149+0000 7ff31dadb700 1 -- 172.21.15.163:0/3364333332 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff31816bc40 con 0x7ff3180b13a0 2023-10-09T06:36:43.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.150+0000 7ff2ff7fe700 1 -- 172.21.15.163:0/3364333332 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7ff30c014650 con 0x7ff3180b13a0 2023-10-09T06:36:43.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.150+0000 7ff2ff7fe700 1 --2- 172.21.15.163:0/3364333332 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff304067290 0x7ff304069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:43.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.150+0000 7ff2ff7fe700 1 -- 172.21.15.163:0/3364333332 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5605+0+0 (secure 0 0 0) 0x7ff30c018030 con 0x7ff3180b13a0 2023-10-09T06:36:43.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.150+0000 7ff3177fe700 1 --2- 172.21.15.163:0/3364333332 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff304067290 0x7ff304069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:43.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.151+0000 7ff3177fe700 1 --2- 172.21.15.163:0/3364333332 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff304067290 0x7ff304069740 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff308001580 tx=0x7ff308006040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:43.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.154+0000 7ff2ff7fe700 1 -- 172.21.15.163:0/3364333332 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff30c0523b0 con 0x7ff3180b13a0 2023-10-09T06:36:43.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.380+0000 7ff31dadb700 1 -- 172.21.15.163:0/3364333332 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff31804c430 con 0x7ff3180b13a0 2023-10-09T06:36:43.827 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:36:43.830 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:36:43.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:43.998+0000 7ff2ff7fe700 1 -- 172.21.15.163:0/3364333332 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v62) v1 ==== 174+0+0 (secure 0 0 0) 0x7ff31804c430 con 0x7ff3180b13a0 2023-10-09T06:36:44.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.001+0000 7ff31dadb700 1 -- 172.21.15.163:0/3364333332 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff304067290 msgr2=0x7ff304069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:44.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.001+0000 7ff31dadb700 1 --2- 172.21.15.163:0/3364333332 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff304067290 0x7ff304069740 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff308001580 tx=0x7ff308006040 comp rx=0 tx=0).stop 2023-10-09T06:36:44.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.001+0000 7ff31dadb700 1 -- 172.21.15.163:0/3364333332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3180b13a0 msgr2=0x7ff3180b3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:44.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.001+0000 7ff31dadb700 1 --2- 172.21.15.163:0/3364333332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3180b13a0 0x7ff3180b3dc0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ff30c011b10 tx=0x7ff30c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:44.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.001+0000 7ff31dadb700 1 -- 172.21.15.163:0/3364333332 shutdown_connections 2023-10-09T06:36:44.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.001+0000 7ff31dadb700 1 --2- 172.21.15.163:0/3364333332 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff304067290 0x7ff304069740 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:44.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.001+0000 7ff31dadb700 1 --2- 172.21.15.163:0/3364333332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff318153790 0x7ff3180cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:44.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.001+0000 7ff31dadb700 1 --2- 172.21.15.163:0/3364333332 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff300004d50 0x7ff31816a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:44.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.001+0000 7ff31dadb700 1 --2- 172.21.15.163:0/3364333332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3180b13a0 0x7ff3180b3dc0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:44.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.001+0000 7ff31dadb700 1 -- 172.21.15.163:0/3364333332 >> 172.21.15.163:0/3364333332 conn(0x7ff3180ce600 msgr2=0x7ff318159dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:44.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.001+0000 7ff31dadb700 1 -- 172.21.15.163:0/3364333332 shutdown_connections 2023-10-09T06:36:44.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.001+0000 7ff31dadb700 1 -- 172.21.15.163:0/3364333332 wait complete. 2023-10-09T06:36:44.004 INFO:teuthology.orchestra.run.smithi163.stderr:new fs with metadata pool 16 and data pool 17 2023-10-09T06:36:44.019 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:36:44.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.132+0000 7efc56e4d700 1 Processor -- start 2023-10-09T06:36:44.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.132+0000 7efc56e4d700 1 -- start start 2023-10-09T06:36:44.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.133+0000 7efc56e4d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efc501599c0 0x7efc50069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:44.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.133+0000 7efc56e4d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7efc500cbd00 0x7efc500ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:44.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.133+0000 7efc56e4d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc5006a220 con 0x7efc50069aa0 2023-10-09T06:36:44.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.133+0000 7efc56e4d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc50069f40 con 0x7efc500cbd00 2023-10-09T06:36:44.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.133+0000 7efc56e4d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc5015a0b0 con 0x7efc501599c0 2023-10-09T06:36:44.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.133+0000 7efc553ea700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7efc500cbd00 0x7efc500ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:44.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.133+0000 7efc553ea700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7efc500cbd00 0x7efc500ce2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41366/0 (socket says 172.21.15.163:41366) 2023-10-09T06:36:44.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.133+0000 7efc553ea700 1 -- 172.21.15.163:0/1917890682 learned_addr learned my addr 172.21.15.163:0/1917890682 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:44.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.134+0000 7efc54be9700 1 --2- 172.21.15.163:0/1917890682 >> v2:172.21.15.181:3300/0 conn(0x7efc501599c0 0x7efc50069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:44.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.134+0000 7efc553ea700 1 -- 172.21.15.163:0/1917890682 >> v2:172.21.15.181:3300/0 conn(0x7efc501599c0 msgr2=0x7efc50069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:44.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.134+0000 7efc553ea700 1 --2- 172.21.15.163:0/1917890682 >> v2:172.21.15.181:3300/0 conn(0x7efc501599c0 0x7efc50069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:44.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.134+0000 7efc553ea700 1 -- 172.21.15.163:0/1917890682 >> v1:172.21.15.181:6789/0 conn(0x7efc50069aa0 legacy=0x7efc500cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:44.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.134+0000 7efc553ea700 1 -- 172.21.15.163:0/1917890682 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5015a390 con 0x7efc500cbd00 2023-10-09T06:36:44.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.134+0000 7efc553ea700 1 --2- 172.21.15.163:0/1917890682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc500cbd00 0x7efc500ce2e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7efc48009f20 tx=0x7efc48009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7118aefcd2afc863 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:44.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.134+0000 7efc4f7fe700 1 -- 172.21.15.163:0/1917890682 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4800e070 con 0x7efc500cbd00 2023-10-09T06:36:44.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.134+0000 7efc4f7fe700 1 -- 172.21.15.163:0/1917890682 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4800ba00 con 0x7efc500cbd00 2023-10-09T06:36:44.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.134+0000 7efc4f7fe700 1 -- 172.21.15.163:0/1917890682 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc480092e0 con 0x7efc500cbd00 2023-10-09T06:36:44.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.134+0000 7efc56e4d700 1 -- 172.21.15.163:0/1917890682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc500cbd00 msgr2=0x7efc500ce2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:44.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.134+0000 7efc56e4d700 1 --2- 172.21.15.163:0/1917890682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc500cbd00 0x7efc500ce2e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7efc48009f20 tx=0x7efc48009840 comp rx=0 tx=0).stop 2023-10-09T06:36:44.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.134+0000 7efc56e4d700 1 -- 172.21.15.163:0/1917890682 shutdown_connections 2023-10-09T06:36:44.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.134+0000 7efc56e4d700 1 --2- 172.21.15.163:0/1917890682 >> v2:172.21.15.181:3300/0 conn(0x7efc501599c0 0x7efc50069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:44.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.134+0000 7efc56e4d700 1 --2- 172.21.15.163:0/1917890682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc500cbd00 0x7efc500ce2e0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:44.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.135+0000 7efc56e4d700 1 -- 172.21.15.163:0/1917890682 >> 172.21.15.163:0/1917890682 conn(0x7efc50068e10 msgr2=0x7efc500d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:44.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.135+0000 7efc56e4d700 1 -- 172.21.15.163:0/1917890682 shutdown_connections 2023-10-09T06:36:44.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.135+0000 7efc56e4d700 1 -- 172.21.15.163:0/1917890682 wait complete. 2023-10-09T06:36:44.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.136+0000 7efc56e4d700 1 Processor -- start 2023-10-09T06:36:44.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.136+0000 7efc56e4d700 1 -- start start 2023-10-09T06:36:44.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.136+0000 7efc56e4d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc50069aa0 0x7efc500b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:44.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.137+0000 7efc56e4d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc500cbd00 0x7efc50147000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:44.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.137+0000 7efc56e4d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc501599c0 0x7efc500b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:44.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.137+0000 7efc56e4d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc500b49c0 con 0x7efc500cbd00 2023-10-09T06:36:44.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.137+0000 7efc56e4d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc500b3db0 con 0x7efc50069aa0 2023-10-09T06:36:44.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.137+0000 7efc56e4d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc500b4090 con 0x7efc501599c0 2023-10-09T06:36:44.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.137+0000 7efc4ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc500cbd00 0x7efc50147000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:44.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.137+0000 7efc4ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc500cbd00 0x7efc50147000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41376/0 (socket says 172.21.15.163:41376) 2023-10-09T06:36:44.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.137+0000 7efc4ffff700 1 -- 172.21.15.163:0/2949315780 learned_addr learned my addr 172.21.15.163:0/2949315780 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:44.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.137+0000 7efc54be9700 1 --2- 172.21.15.163:0/2949315780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc50069aa0 0x7efc500b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:44.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.137+0000 7efc553ea700 1 --2- 172.21.15.163:0/2949315780 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc501599c0 0x7efc500b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:44.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.137+0000 7efc4ffff700 1 -- 172.21.15.163:0/2949315780 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc501599c0 msgr2=0x7efc500b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:44.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.137+0000 7efc4ffff700 1 --2- 172.21.15.163:0/2949315780 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc501599c0 0x7efc500b5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:44.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.137+0000 7efc4ffff700 1 -- 172.21.15.163:0/2949315780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc50069aa0 msgr2=0x7efc500b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:44.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.137+0000 7efc4ffff700 1 --2- 172.21.15.163:0/2949315780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc50069aa0 0x7efc500b34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:44.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.137+0000 7efc4ffff700 1 -- 172.21.15.163:0/2949315780 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc48009ae0 con 0x7efc500cbd00 2023-10-09T06:36:44.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.137+0000 7efc4ffff700 1 --2- 172.21.15.163:0/2949315780 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc500cbd00 0x7efc50147000 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7efc3800c590 tx=0x7efc38007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:44.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.137+0000 7efc4cff9700 1 -- 172.21.15.163:0/2949315780 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc38011660 con 0x7efc500cbd00 2023-10-09T06:36:44.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.137+0000 7efc4cff9700 1 -- 172.21.15.163:0/2949315780 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc380117c0 con 0x7efc500cbd00 2023-10-09T06:36:44.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.137+0000 7efc56e4d700 1 -- 172.21.15.163:0/2949315780 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc500b4370 con 0x7efc500cbd00 2023-10-09T06:36:44.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.137+0000 7efc4cff9700 1 -- 172.21.15.163:0/2949315780 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc38011b80 con 0x7efc500cbd00 2023-10-09T06:36:44.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.137+0000 7efc56e4d700 1 -- 172.21.15.163:0/2949315780 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc500b39f0 con 0x7efc500cbd00 2023-10-09T06:36:44.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.138+0000 7efc56e4d700 1 -- 172.21.15.163:0/2949315780 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc50064550 con 0x7efc500cbd00 2023-10-09T06:36:44.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.139+0000 7efc4cff9700 1 -- 172.21.15.163:0/2949315780 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7efc3801c460 con 0x7efc500cbd00 2023-10-09T06:36:44.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.139+0000 7efc4cff9700 1 --2- 172.21.15.163:0/2949315780 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7efc30067290 0x7efc30069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:44.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.139+0000 7efc4cff9700 1 -- 172.21.15.163:0/2949315780 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efc38084990 con 0x7efc500cbd00 2023-10-09T06:36:44.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.139+0000 7efc54be9700 1 --2- 172.21.15.163:0/2949315780 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7efc30067290 0x7efc30069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:44.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.140+0000 7efc54be9700 1 --2- 172.21.15.163:0/2949315780 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7efc30067290 0x7efc30069740 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7efc40006fd0 tx=0x7efc40008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:44.144 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.143+0000 7efc4cff9700 1 -- 172.21.15.163:0/2949315780 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efc380517f0 con 0x7efc500cbd00 2023-10-09T06:36:44.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.327+0000 7efc56e4d700 1 -- 172.21.15.163:0/2949315780 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc5004c3c0 con 0x7efc500cbd00 2023-10-09T06:36:44.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.328+0000 7efc4cff9700 1 -- 172.21.15.163:0/2949315780 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+15417 (secure 0 0 0) 0x7efc38054e10 con 0x7efc500cbd00 2023-10-09T06:36:44.329 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:44.329 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":100,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:36:43.381735+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:36:41.723219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-09T06:36:42.342713+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:36:44.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.330+0000 7efc56e4d700 1 -- 172.21.15.163:0/2949315780 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7efc30067290 msgr2=0x7efc30069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:44.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.330+0000 7efc56e4d700 1 --2- 172.21.15.163:0/2949315780 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7efc30067290 0x7efc30069740 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7efc40006fd0 tx=0x7efc40008040 comp rx=0 tx=0).stop 2023-10-09T06:36:44.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.330+0000 7efc56e4d700 1 -- 172.21.15.163:0/2949315780 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc500cbd00 msgr2=0x7efc50147000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:44.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.330+0000 7efc56e4d700 1 --2- 172.21.15.163:0/2949315780 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc500cbd00 0x7efc50147000 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7efc3800c590 tx=0x7efc38007f30 comp rx=0 tx=0).stop 2023-10-09T06:36:44.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.330+0000 7efc56e4d700 1 -- 172.21.15.163:0/2949315780 shutdown_connections 2023-10-09T06:36:44.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.330+0000 7efc56e4d700 1 --2- 172.21.15.163:0/2949315780 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7efc30067290 0x7efc30069740 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:44.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.330+0000 7efc56e4d700 1 --2- 172.21.15.163:0/2949315780 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc501599c0 0x7efc500b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:44.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.330+0000 7efc56e4d700 1 --2- 172.21.15.163:0/2949315780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc50069aa0 0x7efc500b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:44.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.330+0000 7efc56e4d700 1 --2- 172.21.15.163:0/2949315780 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc500cbd00 0x7efc50147000 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:44.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.330+0000 7efc56e4d700 1 -- 172.21.15.163:0/2949315780 >> 172.21.15.163:0/2949315780 conn(0x7efc50068e10 msgr2=0x7efc500c59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:44.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.330+0000 7efc56e4d700 1 -- 172.21.15.163:0/2949315780 shutdown_connections 2023-10-09T06:36:44.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.330+0000 7efc56e4d700 1 -- 172.21.15.163:0/2949315780 wait complete. 2023-10-09T06:36:44.343 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:36:44.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.458+0000 7fa3348e8700 1 Processor -- start 2023-10-09T06:36:44.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.458+0000 7fa3348e8700 1 -- start start 2023-10-09T06:36:44.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.459+0000 7fa3348e8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa330159420 0x7fa3300b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:44.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.459+0000 7fa3348e8700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3300b2980 con 0x7fa3301597f0 2023-10-09T06:36:44.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.459+0000 7fa3348e8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3300b11d0 con 0x7fa33015d230 2023-10-09T06:36:44.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.459+0000 7fa3348e8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3300b14b0 con 0x7fa330159420 2023-10-09T06:36:44.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.459+0000 7fa32effd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fa3301597f0 0x7fa3300b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:37932/0 (socket says 172.21.15.163:37932) 2023-10-09T06:36:44.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.459+0000 7fa32effd700 1 -- 172.21.15.163:0/1005343796 learned_addr learned my addr 172.21.15.163:0/1005343796 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:44.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.459+0000 7fa32f7fe700 1 --2- 172.21.15.163:0/1005343796 >> v2:172.21.15.181:3300/0 conn(0x7fa330159420 0x7fa3300b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:44.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.459+0000 7fa32e7fc700 1 -- 172.21.15.163:0/1005343796 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2167313804 0 0) 0x7fa3300b2980 con 0x7fa3301597f0 2023-10-09T06:36:44.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.459+0000 7fa32e7fc700 1 -- 172.21.15.163:0/1005343796 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa318003570 con 0x7fa3301597f0 2023-10-09T06:36:44.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.460+0000 7fa32e7fc700 1 -- 172.21.15.163:0/1005343796 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 522503593 0 0) 0x7fa318003570 con 0x7fa3301597f0 2023-10-09T06:36:44.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.460+0000 7fa32f7fe700 1 -- 172.21.15.163:0/1005343796 >> v1:172.21.15.181:6789/0 conn(0x7fa33015d230 legacy=0x7fa33015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:44.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.460+0000 7fa32f7fe700 1 -- 172.21.15.163:0/1005343796 >> v1:172.21.15.163:6789/0 conn(0x7fa3301597f0 legacy=0x7fa3300b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:44.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.460+0000 7fa32f7fe700 1 -- 172.21.15.163:0/1005343796 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3300b1790 con 0x7fa330159420 2023-10-09T06:36:44.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.460+0000 7fa32f7fe700 1 --2- 172.21.15.163:0/1005343796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa330159420 0x7fa3300b5f40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa3200098e0 tx=0x7fa3200091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7e5aa0c2e6b5d88 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:44.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.460+0000 7fa32e7fc700 1 -- 172.21.15.163:0/1005343796 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa324001470 con 0x7fa330159420 2023-10-09T06:36:44.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.460+0000 7fa32e7fc700 1 -- 172.21.15.163:0/1005343796 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa320011420 con 0x7fa330159420 2023-10-09T06:36:44.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.460+0000 7fa32e7fc700 1 -- 172.21.15.163:0/1005343796 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3200117c0 con 0x7fa330159420 2023-10-09T06:36:44.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.460+0000 7fa3348e8700 1 -- 172.21.15.163:0/1005343796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa330159420 msgr2=0x7fa3300b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:44.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.460+0000 7fa3348e8700 1 --2- 172.21.15.163:0/1005343796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa330159420 0x7fa3300b5f40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa3200098e0 tx=0x7fa3200091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:44.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.461+0000 7fa3348e8700 1 -- 172.21.15.163:0/1005343796 shutdown_connections 2023-10-09T06:36:44.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.461+0000 7fa3348e8700 1 --2- 172.21.15.163:0/1005343796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa330159420 0x7fa3300b5f40 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:44.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.461+0000 7fa3348e8700 1 -- 172.21.15.163:0/1005343796 >> 172.21.15.163:0/1005343796 conn(0x7fa3300d0660 msgr2=0x7fa3300d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:44.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.461+0000 7fa3348e8700 1 -- 172.21.15.163:0/1005343796 shutdown_connections 2023-10-09T06:36:44.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.461+0000 7fa3348e8700 1 -- 172.21.15.163:0/1005343796 wait complete. 2023-10-09T06:36:44.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.462+0000 7fa3348e8700 1 Processor -- start 2023-10-09T06:36:44.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.462+0000 7fa3348e8700 1 -- start start 2023-10-09T06:36:44.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.462+0000 7fa3348e8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa330159420 0x7fa330155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:44.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.462+0000 7fa3348e8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3301597f0 0x7fa33015aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:44.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.462+0000 7fa3348e8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa33015d230 0x7fa3300b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:44.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.462+0000 7fa3348e8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa330155570 con 0x7fa33015d230 2023-10-09T06:36:44.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.462+0000 7fa3348e8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33015ca20 con 0x7fa330159420 2023-10-09T06:36:44.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.462+0000 7fa3348e8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa33015be10 con 0x7fa3301597f0 2023-10-09T06:36:44.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.463+0000 7fa32ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa33015d230 0x7fa3300b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:44.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.463+0000 7fa32ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa33015d230 0x7fa3300b5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41382/0 (socket says 172.21.15.163:41382) 2023-10-09T06:36:44.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.463+0000 7fa32ffff700 1 -- 172.21.15.163:0/3396673201 learned_addr learned my addr 172.21.15.163:0/3396673201 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:44.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.463+0000 7fa32f7fe700 1 --2- 172.21.15.163:0/3396673201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa330159420 0x7fa330155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:44.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.463+0000 7fa32effd700 1 --2- 172.21.15.163:0/3396673201 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3301597f0 0x7fa33015aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:44.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.463+0000 7fa32ffff700 1 -- 172.21.15.163:0/3396673201 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3301597f0 msgr2=0x7fa33015aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:44.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.463+0000 7fa32ffff700 1 --2- 172.21.15.163:0/3396673201 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3301597f0 0x7fa33015aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:44.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.463+0000 7fa32ffff700 1 -- 172.21.15.163:0/3396673201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa330159420 msgr2=0x7fa330155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:44.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.463+0000 7fa32ffff700 1 --2- 172.21.15.163:0/3396673201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa330159420 0x7fa330155030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:44.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.463+0000 7fa32ffff700 1 -- 172.21.15.163:0/3396673201 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa320009420 con 0x7fa33015d230 2023-10-09T06:36:44.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.463+0000 7fa32f7fe700 1 --2- 172.21.15.163:0/3396673201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa330159420 0x7fa330155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:36:44.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.463+0000 7fa32ffff700 1 --2- 172.21.15.163:0/3396673201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa33015d230 0x7fa3300b5870 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fa32800c810 tx=0x7fa328001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:44.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.463+0000 7fa3177fe700 1 -- 172.21.15.163:0/3396673201 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa328012630 con 0x7fa33015d230 2023-10-09T06:36:44.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.463+0000 7fa3177fe700 1 -- 172.21.15.163:0/3396673201 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3280127b0 con 0x7fa33015d230 2023-10-09T06:36:44.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.463+0000 7fa3348e8700 1 -- 172.21.15.163:0/3396673201 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa33015c0f0 con 0x7fa33015d230 2023-10-09T06:36:44.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.463+0000 7fa3177fe700 1 -- 172.21.15.163:0/3396673201 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa328012b70 con 0x7fa33015d230 2023-10-09T06:36:44.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.463+0000 7fa3348e8700 1 -- 172.21.15.163:0/3396673201 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa33015c6b0 con 0x7fa33015d230 2023-10-09T06:36:44.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.464+0000 7fa3348e8700 1 -- 172.21.15.163:0/3396673201 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa33015cb90 con 0x7fa33015d230 2023-10-09T06:36:44.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.465+0000 7fa3177fe700 1 -- 172.21.15.163:0/3396673201 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fa328003030 con 0x7fa33015d230 2023-10-09T06:36:44.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.465+0000 7fa3177fe700 1 --2- 172.21.15.163:0/3396673201 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa31c067290 0x7fa31c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:44.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.465+0000 7fa3177fe700 1 -- 172.21.15.163:0/3396673201 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa328084b10 con 0x7fa33015d230 2023-10-09T06:36:44.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.465+0000 7fa32f7fe700 1 --2- 172.21.15.163:0/3396673201 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa31c067290 0x7fa31c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:44.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.466+0000 7fa32f7fe700 1 --2- 172.21.15.163:0/3396673201 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa31c067290 0x7fa31c069740 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa32000a010 tx=0x7fa320016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:44.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.469+0000 7fa3177fe700 1 -- 172.21.15.163:0/3396673201 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa328051870 con 0x7fa33015d230 2023-10-09T06:36:44.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.654+0000 7fa3348e8700 1 -- 172.21.15.163:0/3396673201 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa33004c3c0 con 0x7fa33015d230 2023-10-09T06:36:44.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.655+0000 7fa3177fe700 1 -- 172.21.15.163:0/3396673201 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+15404 (secure 0 0 0) 0x7fa328054e90 con 0x7fa33015d230 2023-10-09T06:36:44.656 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:44.656 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":101,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:36:44.526778+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:36:41.723219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-09T06:36:42.342713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:36:44.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.657+0000 7fa3348e8700 1 -- 172.21.15.163:0/3396673201 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa31c067290 msgr2=0x7fa31c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:44.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.657+0000 7fa3348e8700 1 --2- 172.21.15.163:0/3396673201 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa31c067290 0x7fa31c069740 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa32000a010 tx=0x7fa320016000 comp rx=0 tx=0).stop 2023-10-09T06:36:44.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.657+0000 7fa3348e8700 1 -- 172.21.15.163:0/3396673201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa33015d230 msgr2=0x7fa3300b5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:44.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.657+0000 7fa3348e8700 1 --2- 172.21.15.163:0/3396673201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa33015d230 0x7fa3300b5870 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fa32800c810 tx=0x7fa328001650 comp rx=0 tx=0).stop 2023-10-09T06:36:44.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.658+0000 7fa3348e8700 1 -- 172.21.15.163:0/3396673201 shutdown_connections 2023-10-09T06:36:44.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.658+0000 7fa3348e8700 1 --2- 172.21.15.163:0/3396673201 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa31c067290 0x7fa31c069740 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:44.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.658+0000 7fa3348e8700 1 --2- 172.21.15.163:0/3396673201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa33015d230 0x7fa3300b5870 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:44.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.658+0000 7fa3348e8700 1 --2- 172.21.15.163:0/3396673201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa330159420 0x7fa330155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:44.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.658+0000 7fa3348e8700 1 --2- 172.21.15.163:0/3396673201 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3301597f0 0x7fa33015aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:44.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.658+0000 7fa3348e8700 1 -- 172.21.15.163:0/3396673201 >> 172.21.15.163:0/3396673201 conn(0x7fa3300d0660 msgr2=0x7fa330156d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:44.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.658+0000 7fa3348e8700 1 -- 172.21.15.163:0/3396673201 shutdown_connections 2023-10-09T06:36:44.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.658+0000 7fa3348e8700 1 -- 172.21.15.163:0/3396673201 wait complete. 2023-10-09T06:36:44.671 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-09T06:36:44.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.787+0000 7f8bbea7f700 1 Processor -- start 2023-10-09T06:36:44.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.787+0000 7f8bbea7f700 1 -- start start 2023-10-09T06:36:44.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.788+0000 7f8bbea7f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8bb80b1b40 0x7f8bb80b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:44.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.788+0000 7f8bbea7f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb80cfb00 con 0x7f8bb80b16d0 2023-10-09T06:36:44.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.788+0000 7f8bbea7f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb80b5c40 con 0x7f8bb80b4e70 2023-10-09T06:36:44.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.788+0000 7f8bbea7f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb80b5f20 con 0x7f8bb80b1b40 2023-10-09T06:36:44.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.788+0000 7f8bbd27c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f8bb80b16d0 0x7f8bb80b33b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:37946/0 (socket says 172.21.15.163:37946) 2023-10-09T06:36:44.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.788+0000 7f8bbd27c700 1 -- 172.21.15.163:0/617295429 learned_addr learned my addr 172.21.15.163:0/617295429 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:44.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.788+0000 7f8bbe27e700 1 --2- 172.21.15.163:0/617295429 >> v2:172.21.15.181:3300/0 conn(0x7f8bb80b1b40 0x7f8bb80b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:44.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.788+0000 7f8bbca7b700 1 -- 172.21.15.163:0/617295429 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2154508949 0 0) 0x7f8bb80cfb00 con 0x7f8bb80b16d0 2023-10-09T06:36:44.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.788+0000 7f8bbca7b700 1 -- 172.21.15.163:0/617295429 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ba0003570 con 0x7f8bb80b16d0 2023-10-09T06:36:44.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.789+0000 7f8bbca7b700 1 -- 172.21.15.163:0/617295429 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4043364415 0 0) 0x7f8ba0003570 con 0x7f8bb80b16d0 2023-10-09T06:36:44.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.789+0000 7f8bbca7b700 1 -- 172.21.15.163:0/617295429 >> v2:172.21.15.181:3300/0 conn(0x7f8bb80b1b40 msgr2=0x7f8bb80b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:44.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.789+0000 7f8bbca7b700 1 --2- 172.21.15.163:0/617295429 >> v2:172.21.15.181:3300/0 conn(0x7f8bb80b1b40 0x7f8bb80b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:44.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.789+0000 7f8bbca7b700 1 -- 172.21.15.163:0/617295429 >> v1:172.21.15.181:6789/0 conn(0x7f8bb80b4e70 legacy=0x7f8bb80ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:44.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.789+0000 7f8bbca7b700 1 -- 172.21.15.163:0/617295429 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb80b6200 con 0x7f8bb80b16d0 2023-10-09T06:36:44.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.789+0000 7f8bbca7b700 1 -- 172.21.15.163:0/617295429 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f8bac001470 con 0x7f8bb80b16d0 2023-10-09T06:36:44.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.789+0000 7f8bbca7b700 1 -- 172.21.15.163:0/617295429 >> v1:172.21.15.163:6789/0 conn(0x7f8bb80b16d0 legacy=0x7f8bb80b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:44.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.789+0000 7f8bbca7b700 1 --2- 172.21.15.163:0/617295429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ba0003a60 0x7f8ba0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:44.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.789+0000 7f8bbca7b700 1 --2- 172.21.15.163:0/617295429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ba0004810 0x7f8ba0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:44.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.789+0000 7f8bbca7b700 1 --2- 172.21.15.163:0/617295429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ba00052e0 0x7f8ba000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:44.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.789+0000 7f8bbca7b700 1 -- 172.21.15.163:0/617295429 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb80b5f20 con 0x7f8ba0003a60 2023-10-09T06:36:44.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.789+0000 7f8bbca7b700 1 -- 172.21.15.163:0/617295429 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba000c2a0 con 0x7f8ba0004810 2023-10-09T06:36:44.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.789+0000 7f8bbca7b700 1 -- 172.21.15.163:0/617295429 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba000c460 con 0x7f8ba00052e0 2023-10-09T06:36:44.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.789+0000 7f8bbda7d700 1 --2- 172.21.15.163:0/617295429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ba0003a60 0x7f8ba0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:44.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.789+0000 7f8bbd27c700 1 --2- 172.21.15.163:0/617295429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ba0004810 0x7f8ba0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:44.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.789+0000 7f8bbe27e700 1 --2- 172.21.15.163:0/617295429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ba00052e0 0x7f8ba000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:44.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.789+0000 7f8bbda7d700 1 -- 172.21.15.163:0/617295429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ba00052e0 msgr2=0x7f8ba000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:44.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.789+0000 7f8bbda7d700 1 --2- 172.21.15.163:0/617295429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ba00052e0 0x7f8ba000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:44.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.789+0000 7f8bbda7d700 1 -- 172.21.15.163:0/617295429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ba0004810 msgr2=0x7f8ba0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:44.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.789+0000 7f8bbda7d700 1 --2- 172.21.15.163:0/617295429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ba0004810 0x7f8ba0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:44.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.789+0000 7f8bbda7d700 1 -- 172.21.15.163:0/617295429 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba0003570 con 0x7f8ba0003a60 2023-10-09T06:36:44.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.789+0000 7f8bbd27c700 1 --2- 172.21.15.163:0/617295429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ba0004810 0x7f8ba0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:44.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.789+0000 7f8bbe27e700 1 --2- 172.21.15.163:0/617295429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ba00052e0 0x7f8ba000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:44.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.789+0000 7f8bbe27e700 1 -- 172.21.15.163:0/617295429 reap_dead start 2023-10-09T06:36:44.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.789+0000 7f8bbda7d700 1 --2- 172.21.15.163:0/617295429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ba0003a60 0x7f8ba0003e70 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8001fc0 tx=0x7f8ba8001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409f3e78ab010137 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:44.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.789+0000 7f8bbca7b700 1 -- 172.21.15.163:0/617295429 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8001950 con 0x7f8ba0003a60 2023-10-09T06:36:44.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.789+0000 7f8bbca7b700 1 -- 172.21.15.163:0/617295429 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba800c8b0 con 0x7f8ba0003a60 2023-10-09T06:36:44.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.789+0000 7f8bbca7b700 1 -- 172.21.15.163:0/617295429 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba800cc50 con 0x7f8ba0003a60 2023-10-09T06:36:44.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.790+0000 7f8bbea7f700 1 -- 172.21.15.163:0/617295429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ba0003a60 msgr2=0x7f8ba0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:44.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.790+0000 7f8bbea7f700 1 --2- 172.21.15.163:0/617295429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ba0003a60 0x7f8ba0003e70 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8001fc0 tx=0x7f8ba8001640 comp rx=0 tx=0).stop 2023-10-09T06:36:44.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.790+0000 7f8bbea7f700 1 -- 172.21.15.163:0/617295429 shutdown_connections 2023-10-09T06:36:44.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.790+0000 7f8bbea7f700 1 --2- 172.21.15.163:0/617295429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ba0003a60 0x7f8ba0003e70 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:44.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.790+0000 7f8bbea7f700 1 -- 172.21.15.163:0/617295429 >> 172.21.15.163:0/617295429 conn(0x7f8bb80d10c0 msgr2=0x7f8bb80affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:44.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.790+0000 7f8bbea7f700 1 -- 172.21.15.163:0/617295429 shutdown_connections 2023-10-09T06:36:44.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.790+0000 7f8bbea7f700 1 -- 172.21.15.163:0/617295429 wait complete. 2023-10-09T06:36:44.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.791+0000 7f8bbea7f700 1 Processor -- start 2023-10-09T06:36:44.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.791+0000 7f8bbea7f700 1 -- start start 2023-10-09T06:36:44.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.792+0000 7f8bbea7f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ba0003a60 0x7f8bb80c9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:44.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.792+0000 7f8bbea7f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bb80ca040 0x7f8bb8157810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:44.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.792+0000 7f8bbea7f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bb8069590 0x7f8bb80699e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:44.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.792+0000 7f8bbea7f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb8169f10 con 0x7f8bb8069590 2023-10-09T06:36:44.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.792+0000 7f8bbea7f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb8069f20 con 0x7f8bb80ca040 2023-10-09T06:36:44.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.792+0000 7f8bbea7f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb80ca7b0 con 0x7f8ba0003a60 2023-10-09T06:36:44.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.792+0000 7f8bbe27e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bb8069590 0x7f8bb80699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:44.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.792+0000 7f8bbe27e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bb8069590 0x7f8bb80699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41410/0 (socket says 172.21.15.163:41410) 2023-10-09T06:36:44.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.792+0000 7f8bbe27e700 1 -- 172.21.15.163:0/2019905406 learned_addr learned my addr 172.21.15.163:0/2019905406 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:44.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.792+0000 7f8bbda7d700 1 --2- 172.21.15.163:0/2019905406 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ba0003a60 0x7f8bb80c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:44.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.792+0000 7f8bbe27e700 1 -- 172.21.15.163:0/2019905406 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ba0003a60 msgr2=0x7f8bb80c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:44.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.792+0000 7f8bbe27e700 1 --2- 172.21.15.163:0/2019905406 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ba0003a60 0x7f8bb80c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:44.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.792+0000 7f8bbd27c700 1 --2- 172.21.15.163:0/2019905406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bb80ca040 0x7f8bb8157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:44.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.792+0000 7f8bbe27e700 1 -- 172.21.15.163:0/2019905406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bb80ca040 msgr2=0x7f8bb8157810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:44.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.792+0000 7f8bbe27e700 1 --2- 172.21.15.163:0/2019905406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bb80ca040 0x7f8bb8157810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:44.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.792+0000 7f8bbe27e700 1 -- 172.21.15.163:0/2019905406 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb80caa90 con 0x7f8bb8069590 2023-10-09T06:36:44.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.792+0000 7f8bbe27e700 1 --2- 172.21.15.163:0/2019905406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bb8069590 0x7f8bb80699e0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80899c0 tx=0x7f8bb0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:44.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.793+0000 7f8bb5ffb700 1 -- 172.21.15.163:0/2019905406 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0015740 con 0x7f8bb8069590 2023-10-09T06:36:44.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.793+0000 7f8bb5ffb700 1 -- 172.21.15.163:0/2019905406 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb00158a0 con 0x7f8bb8069590 2023-10-09T06:36:44.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.793+0000 7f8bbea7f700 1 -- 172.21.15.163:0/2019905406 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb80cad70 con 0x7f8bb8069590 2023-10-09T06:36:44.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.793+0000 7f8bb5ffb700 1 -- 172.21.15.163:0/2019905406 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0015c60 con 0x7f8bb8069590 2023-10-09T06:36:44.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.793+0000 7f8bbea7f700 1 -- 172.21.15.163:0/2019905406 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb80ca4b0 con 0x7f8bb8069590 2023-10-09T06:36:44.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.794+0000 7f8bbea7f700 1 -- 172.21.15.163:0/2019905406 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb816a080 con 0x7f8bb8069590 2023-10-09T06:36:44.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.794+0000 7f8bb5ffb700 1 -- 172.21.15.163:0/2019905406 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f8bb001f410 con 0x7f8bb8069590 2023-10-09T06:36:44.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.795+0000 7f8bb5ffb700 1 --2- 172.21.15.163:0/2019905406 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8ba4067290 0x7f8ba4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:44.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.795+0000 7f8bb5ffb700 1 -- 172.21.15.163:0/2019905406 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8bb0084730 con 0x7f8bb8069590 2023-10-09T06:36:44.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.795+0000 7f8bbda7d700 1 --2- 172.21.15.163:0/2019905406 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8ba4067290 0x7f8ba4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:44.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.795+0000 7f8bbda7d700 1 --2- 172.21.15.163:0/2019905406 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8ba4067290 0x7f8ba4069740 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8001fc0 tx=0x7f8ba8015000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:44.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:44.799+0000 7f8bb5ffb700 1 -- 172.21.15.163:0/2019905406 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8bb0051610 con 0x7f8bb8069590 2023-10-09T06:36:45.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:45.028+0000 7f8bbea7f700 1 -- 172.21.15.163:0/2019905406 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8bb804bb80 con 0x7f8bb8069590 2023-10-09T06:36:46.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.015+0000 7f8bb5ffb700 1 -- 172.21.15.163:0/2019905406 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v65) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8bb0054c30 con 0x7f8bb8069590 2023-10-09T06:36:46.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.018+0000 7f8bbea7f700 1 -- 172.21.15.163:0/2019905406 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8ba4067290 msgr2=0x7f8ba4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:46.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.018+0000 7f8bbea7f700 1 --2- 172.21.15.163:0/2019905406 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8ba4067290 0x7f8ba4069740 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8001fc0 tx=0x7f8ba8015000 comp rx=0 tx=0).stop 2023-10-09T06:36:46.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.018+0000 7f8bbea7f700 1 -- 172.21.15.163:0/2019905406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bb8069590 msgr2=0x7f8bb80699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:46.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.018+0000 7f8bbea7f700 1 --2- 172.21.15.163:0/2019905406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bb8069590 0x7f8bb80699e0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80899c0 tx=0x7f8bb0008040 comp rx=0 tx=0).stop 2023-10-09T06:36:46.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.018+0000 7f8bbea7f700 1 -- 172.21.15.163:0/2019905406 shutdown_connections 2023-10-09T06:36:46.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.018+0000 7f8bbea7f700 1 --2- 172.21.15.163:0/2019905406 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8ba4067290 0x7f8ba4069740 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:46.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.018+0000 7f8bbea7f700 1 --2- 172.21.15.163:0/2019905406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bb8069590 0x7f8bb80699e0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:46.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.018+0000 7f8bbea7f700 1 --2- 172.21.15.163:0/2019905406 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ba0003a60 0x7f8bb80c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:46.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.018+0000 7f8bbea7f700 1 --2- 172.21.15.163:0/2019905406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bb80ca040 0x7f8bb8157810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:46.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.018+0000 7f8bbea7f700 1 -- 172.21.15.163:0/2019905406 >> 172.21.15.163:0/2019905406 conn(0x7f8bb80d10c0 msgr2=0x7f8bb8159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:46.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.018+0000 7f8bbea7f700 1 -- 172.21.15.163:0/2019905406 shutdown_connections 2023-10-09T06:36:46.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.018+0000 7f8bbea7f700 1 -- 172.21.15.163:0/2019905406 wait complete. 2023-10-09T06:36:46.034 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-09T06:36:46.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.155+0000 7f12bce5b700 1 Processor -- start 2023-10-09T06:36:46.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.155+0000 7f12bce5b700 1 -- start start 2023-10-09T06:36:46.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.156+0000 7f12bce5b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f12b8069560 0x7f12b80699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:46.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.156+0000 7f12bce5b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b8069ee0 con 0x7f12b8159c60 2023-10-09T06:36:46.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.156+0000 7f12bce5b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b815a160 con 0x7f12b8159890 2023-10-09T06:36:46.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.156+0000 7f12bce5b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12b80d13a0 con 0x7f12b8069560 2023-10-09T06:36:46.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.157+0000 7f12b6d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f12b8069560 0x7f12b80699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:46.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.157+0000 7f12b5d9b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f12b8159c60 0x7f12b815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:37960/0 (socket says 172.21.15.163:37960) 2023-10-09T06:36:46.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.157+0000 7f12b5d9b700 1 -- 172.21.15.163:0/2562292666 learned_addr learned my addr 172.21.15.163:0/2562292666 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:46.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.157+0000 7f12b6d9d700 1 -- 172.21.15.163:0/2562292666 >> v1:172.21.15.181:6789/0 conn(0x7f12b8159890 legacy=0x7f12b815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:46.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.157+0000 7f12b6d9d700 1 -- 172.21.15.163:0/2562292666 >> v1:172.21.15.163:6789/0 conn(0x7f12b8159c60 legacy=0x7f12b815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:46.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.157+0000 7f12b6d9d700 1 -- 172.21.15.163:0/2562292666 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b815e140 con 0x7f12b8069560 2023-10-09T06:36:46.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.157+0000 7f12b559a700 1 -- 172.21.15.163:0/2562292666 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3423014754 0 0) 0x7f12b8069ee0 con 0x7f12b8159c60 2023-10-09T06:36:46.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.157+0000 7f12b6d9d700 1 --2- 172.21.15.163:0/2562292666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12b8069560 0x7f12b80699a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f12b00098e0 tx=0x7f12b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=577a1fb4c788ad0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:46.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.157+0000 7f12b559a700 1 -- 172.21.15.163:0/2562292666 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b0009e80 con 0x7f12b8069560 2023-10-09T06:36:46.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.157+0000 7f12b559a700 1 -- 172.21.15.163:0/2562292666 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b000b860 con 0x7f12b8069560 2023-10-09T06:36:46.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.157+0000 7f12b559a700 1 -- 172.21.15.163:0/2562292666 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b000bbe0 con 0x7f12b8069560 2023-10-09T06:36:46.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.158+0000 7f12bce5b700 1 -- 172.21.15.163:0/2562292666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12b8069560 msgr2=0x7f12b80699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:46.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.158+0000 7f12bce5b700 1 --2- 172.21.15.163:0/2562292666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12b8069560 0x7f12b80699a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f12b00098e0 tx=0x7f12b00091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:46.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.158+0000 7f12bce5b700 1 -- 172.21.15.163:0/2562292666 shutdown_connections 2023-10-09T06:36:46.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.158+0000 7f12bce5b700 1 --2- 172.21.15.163:0/2562292666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12b8069560 0x7f12b80699a0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:46.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.158+0000 7f12bce5b700 1 -- 172.21.15.163:0/2562292666 >> 172.21.15.163:0/2562292666 conn(0x7f12b8068e20 msgr2=0x7f12b80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:46.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.158+0000 7f12bce5b700 1 -- 172.21.15.163:0/2562292666 shutdown_connections 2023-10-09T06:36:46.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.158+0000 7f12bce5b700 1 -- 172.21.15.163:0/2562292666 wait complete. 2023-10-09T06:36:46.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.159+0000 7f12bce5b700 1 Processor -- start 2023-10-09T06:36:46.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.159+0000 7f12bce5b700 1 -- start start 2023-10-09T06:36:46.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.159+0000 7f12bce5b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12b8069560 0x7f12b8154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:46.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.159+0000 7f12bce5b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12b8159890 0x7f12b815c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:46.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.159+0000 7f12bce5b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12b8159c60 0x7f12b80ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:46.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.159+0000 7f12bce5b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b80ce930 con 0x7f12b8159890 2023-10-09T06:36:46.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.159+0000 7f12bce5b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b80cfbc0 con 0x7f12b8069560 2023-10-09T06:36:46.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.159+0000 7f12bce5b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b80cf730 con 0x7f12b8159c60 2023-10-09T06:36:46.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.160+0000 7f12b5d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12b8159890 0x7f12b815c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:46.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.160+0000 7f12b5d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12b8159890 0x7f12b815c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41426/0 (socket says 172.21.15.163:41426) 2023-10-09T06:36:46.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.160+0000 7f12b5d9b700 1 -- 172.21.15.163:0/4021004546 learned_addr learned my addr 172.21.15.163:0/4021004546 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:46.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.160+0000 7f12b659c700 1 --2- 172.21.15.163:0/4021004546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12b8069560 0x7f12b8154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:46.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.160+0000 7f12b6d9d700 1 --2- 172.21.15.163:0/4021004546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12b8159c60 0x7f12b80ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:46.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.160+0000 7f12b5d9b700 1 -- 172.21.15.163:0/4021004546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12b8159c60 msgr2=0x7f12b80ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:46.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.160+0000 7f12b5d9b700 1 --2- 172.21.15.163:0/4021004546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12b8159c60 0x7f12b80ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:46.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.160+0000 7f12b5d9b700 1 -- 172.21.15.163:0/4021004546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12b8069560 msgr2=0x7f12b8154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:46.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.160+0000 7f12b5d9b700 1 --2- 172.21.15.163:0/4021004546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12b8069560 0x7f12b8154090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:46.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.160+0000 7f12b5d9b700 1 -- 172.21.15.163:0/4021004546 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b0009420 con 0x7f12b8159890 2023-10-09T06:36:46.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.160+0000 7f12b5d9b700 1 --2- 172.21.15.163:0/4021004546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12b8159890 0x7f12b815c000 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f12a000c5e0 tx=0x7f12a0001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:46.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.160+0000 7f12ae7fc700 1 -- 172.21.15.163:0/4021004546 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a0005b30 con 0x7f12b8159890 2023-10-09T06:36:46.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.160+0000 7f12ae7fc700 1 -- 172.21.15.163:0/4021004546 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a0005c90 con 0x7f12b8159890 2023-10-09T06:36:46.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.160+0000 7f12bce5b700 1 -- 172.21.15.163:0/4021004546 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b80ceb20 con 0x7f12b8159890 2023-10-09T06:36:46.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.160+0000 7f12ae7fc700 1 -- 172.21.15.163:0/4021004546 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a0007300 con 0x7f12b8159890 2023-10-09T06:36:46.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.160+0000 7f12bce5b700 1 -- 172.21.15.163:0/4021004546 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b80cf160 con 0x7f12b8159890 2023-10-09T06:36:46.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.161+0000 7f12bce5b700 1 -- 172.21.15.163:0/4021004546 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b804c3c0 con 0x7f12b8159890 2023-10-09T06:36:46.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.161+0000 7f12ae7fc700 1 -- 172.21.15.163:0/4021004546 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f12a00076b0 con 0x7f12b8159890 2023-10-09T06:36:46.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.162+0000 7f12ae7fc700 1 --2- 172.21.15.163:0/4021004546 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1298067290 0x7f1298069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:46.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.162+0000 7f12ae7fc700 1 -- 172.21.15.163:0/4021004546 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f12a0083ee0 con 0x7f12b8159890 2023-10-09T06:36:46.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.162+0000 7f12b659c700 1 --2- 172.21.15.163:0/4021004546 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1298067290 0x7f1298069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:46.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.162+0000 7f12b659c700 1 --2- 172.21.15.163:0/4021004546 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1298067290 0x7f1298069740 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f12a8007fd0 tx=0x7f12a80016f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:46.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.166+0000 7f12ae7fc700 1 -- 172.21.15.163:0/4021004546 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f12a0050d40 con 0x7f12b8159890 2023-10-09T06:36:46.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:46.391+0000 7f12bce5b700 1 -- 172.21.15.163:0/4021004546 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f12b81545d0 con 0x7f12b8159890 2023-10-09T06:36:47.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.021+0000 7f12ae7fc700 1 -- 172.21.15.163:0/4021004546 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v66) v1 ==== 122+0+0 (secure 0 0 0) 0x7f12a00170c0 con 0x7f12b8159890 2023-10-09T06:36:47.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.024+0000 7f12bce5b700 1 -- 172.21.15.163:0/4021004546 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1298067290 msgr2=0x7f1298069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:47.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.024+0000 7f12bce5b700 1 --2- 172.21.15.163:0/4021004546 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1298067290 0x7f1298069740 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f12a8007fd0 tx=0x7f12a80016f0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.024+0000 7f12bce5b700 1 -- 172.21.15.163:0/4021004546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12b8159890 msgr2=0x7f12b815c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:47.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.024+0000 7f12bce5b700 1 --2- 172.21.15.163:0/4021004546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12b8159890 0x7f12b815c000 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f12a000c5e0 tx=0x7f12a0001070 comp rx=0 tx=0).stop 2023-10-09T06:36:47.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.024+0000 7f12bce5b700 1 -- 172.21.15.163:0/4021004546 shutdown_connections 2023-10-09T06:36:47.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.024+0000 7f12bce5b700 1 --2- 172.21.15.163:0/4021004546 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1298067290 0x7f1298069740 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.024+0000 7f12bce5b700 1 --2- 172.21.15.163:0/4021004546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12b8159c60 0x7f12b80ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.024+0000 7f12bce5b700 1 --2- 172.21.15.163:0/4021004546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12b8069560 0x7f12b8154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.024+0000 7f12bce5b700 1 --2- 172.21.15.163:0/4021004546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12b8159890 0x7f12b815c000 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.024+0000 7f12bce5b700 1 -- 172.21.15.163:0/4021004546 >> 172.21.15.163:0/4021004546 conn(0x7f12b8068e20 msgr2=0x7f12b81576a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:47.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.024+0000 7f12bce5b700 1 -- 172.21.15.163:0/4021004546 shutdown_connections 2023-10-09T06:36:47.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.024+0000 7f12bce5b700 1 -- 172.21.15.163:0/4021004546 wait complete. 2023-10-09T06:36:47.038 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:36:47.152 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.151+0000 7fbbb56d3700 1 Processor -- start 2023-10-09T06:36:47.153 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.152+0000 7fbbb56d3700 1 -- start start 2023-10-09T06:36:47.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.152+0000 7fbbb56d3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbbb0159890 0x7fbbb015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:47.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.152+0000 7fbbb56d3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbb0159c60 0x7fbbb015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:47.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.152+0000 7fbbb56d3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb015a160 con 0x7fbbb015db10 2023-10-09T06:36:47.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.152+0000 7fbbb56d3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb00d13a0 con 0x7fbbb0159890 2023-10-09T06:36:47.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.152+0000 7fbbb56d3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb015e140 con 0x7fbbb0159c60 2023-10-09T06:36:47.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.153+0000 7fbbaeffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbbb0159890 0x7fbbb015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:47.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.153+0000 7fbba67fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbb0159c60 0x7fbbb015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:47.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.153+0000 7fbbaeffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbbb0159890 0x7fbbb015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:57264/0 (socket says 172.21.15.163:57264) 2023-10-09T06:36:47.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.153+0000 7fbbaeffd700 1 -- 172.21.15.163:0/1022363091 learned_addr learned my addr 172.21.15.163:0/1022363091 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:47.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.153+0000 7fbba67fc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbb0159c60 0x7fbbb015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:39414/0 (socket says 172.21.15.163:39414) 2023-10-09T06:36:47.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.153+0000 7fbbaf7fe700 1 --1- 172.21.15.163:0/1022363091 >> v1:172.21.15.181:6789/0 conn(0x7fbbb015db10 0x7fbbb0069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:55122/0 (socket says 172.21.15.163:55122) 2023-10-09T06:36:47.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.153+0000 7fbbaeffd700 1 -- 172.21.15.163:0/1022363091 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbb0159c60 msgr2=0x7fbbb015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:47.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.153+0000 7fbbaeffd700 1 --2- 172.21.15.163:0/1022363091 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbb0159c60 0x7fbbb015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.153+0000 7fbbaeffd700 1 -- 172.21.15.163:0/1022363091 >> v1:172.21.15.181:6789/0 conn(0x7fbbb015db10 legacy=0x7fbbb0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:47.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.153+0000 7fbbaeffd700 1 -- 172.21.15.163:0/1022363091 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb015a3d0 con 0x7fbbb0159890 2023-10-09T06:36:47.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.153+0000 7fbbae7fc700 1 -- 172.21.15.163:0/1022363091 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1582846878 0 0) 0x7fbbb015a160 con 0x7fbbb015db10 2023-10-09T06:36:47.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.153+0000 7fbba67fc700 1 --2- 172.21.15.163:0/1022363091 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbb0159c60 0x7fbbb015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:36:47.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.154+0000 7fbbaeffd700 1 --2- 172.21.15.163:0/1022363091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbb0159890 0x7fbbb015abd0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fbba00099b0 tx=0x7fbba0009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b9474c770ad3cb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:47.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.154+0000 7fbbae7fc700 1 -- 172.21.15.163:0/1022363091 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba0009ea0 con 0x7fbbb0159890 2023-10-09T06:36:47.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.154+0000 7fbbae7fc700 1 -- 172.21.15.163:0/1022363091 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba000b860 con 0x7fbbb0159890 2023-10-09T06:36:47.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.154+0000 7fbbae7fc700 1 -- 172.21.15.163:0/1022363091 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba000bc00 con 0x7fbbb0159890 2023-10-09T06:36:47.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.154+0000 7fbbb56d3700 1 -- 172.21.15.163:0/1022363091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbb0159890 msgr2=0x7fbbb015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:47.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.154+0000 7fbbb56d3700 1 --2- 172.21.15.163:0/1022363091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbb0159890 0x7fbbb015abd0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fbba00099b0 tx=0x7fbba0009250 comp rx=0 tx=0).stop 2023-10-09T06:36:47.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.154+0000 7fbbb56d3700 1 -- 172.21.15.163:0/1022363091 shutdown_connections 2023-10-09T06:36:47.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.154+0000 7fbbb56d3700 1 --2- 172.21.15.163:0/1022363091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbb0159890 0x7fbbb015abd0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.154+0000 7fbbb56d3700 1 --2- 172.21.15.163:0/1022363091 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbb0159c60 0x7fbbb015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.154+0000 7fbbb56d3700 1 -- 172.21.15.163:0/1022363091 >> 172.21.15.163:0/1022363091 conn(0x7fbbb0068e20 msgr2=0x7fbbb00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:47.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.154+0000 7fbbb56d3700 1 -- 172.21.15.163:0/1022363091 shutdown_connections 2023-10-09T06:36:47.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.154+0000 7fbbb56d3700 1 -- 172.21.15.163:0/1022363091 wait complete. 2023-10-09T06:36:47.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.156+0000 7fbbb56d3700 1 Processor -- start 2023-10-09T06:36:47.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.156+0000 7fbbb56d3700 1 -- start start 2023-10-09T06:36:47.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.156+0000 7fbbb56d3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbb0159890 0x7fbbb00b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:47.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.156+0000 7fbbb56d3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbb0159c60 0x7fbbb014d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:47.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.156+0000 7fbbb56d3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbb015db10 0x7fbbb0153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:47.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.156+0000 7fbbb56d3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb00b3de0 con 0x7fbbb0159c60 2023-10-09T06:36:47.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.156+0000 7fbbb56d3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb00b40c0 con 0x7fbbb0159890 2023-10-09T06:36:47.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.156+0000 7fbbb56d3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb00b43a0 con 0x7fbbb015db10 2023-10-09T06:36:47.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.157+0000 7fbba67fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbb0159c60 0x7fbbb014d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:47.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.157+0000 7fbba67fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbb0159c60 0x7fbbb014d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41432/0 (socket says 172.21.15.163:41432) 2023-10-09T06:36:47.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.157+0000 7fbba67fc700 1 -- 172.21.15.163:0/2189917396 learned_addr learned my addr 172.21.15.163:0/2189917396 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:47.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.157+0000 7fbbaeffd700 1 --2- 172.21.15.163:0/2189917396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbb0159890 0x7fbbb00b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:47.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.157+0000 7fbbaf7fe700 1 --2- 172.21.15.163:0/2189917396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbb015db10 0x7fbbb0153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:47.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.157+0000 7fbba67fc700 1 -- 172.21.15.163:0/2189917396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbb015db10 msgr2=0x7fbbb0153850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:47.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.157+0000 7fbba67fc700 1 --2- 172.21.15.163:0/2189917396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbb015db10 0x7fbbb0153850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.157+0000 7fbba67fc700 1 -- 172.21.15.163:0/2189917396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbb0159890 msgr2=0x7fbbb00b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:47.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.157+0000 7fbba67fc700 1 --2- 172.21.15.163:0/2189917396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbb0159890 0x7fbbb00b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.157+0000 7fbba67fc700 1 -- 172.21.15.163:0/2189917396 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb98009420 con 0x7fbbb0159c60 2023-10-09T06:36:47.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.157+0000 7fbbaeffd700 1 --2- 172.21.15.163:0/2189917396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbb0159890 0x7fbbb00b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:47.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.157+0000 7fbbaf7fe700 1 --2- 172.21.15.163:0/2189917396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbb015db10 0x7fbbb0153850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:47.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.157+0000 7fbba67fc700 1 --2- 172.21.15.163:0/2189917396 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbb0159c60 0x7fbbb014d1c0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fbb9800ec80 tx=0x7fbb980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:47.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.157+0000 7fbba77fe700 1 -- 172.21.15.163:0/2189917396 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9800ca60 con 0x7fbbb0159c60 2023-10-09T06:36:47.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.157+0000 7fbba77fe700 1 -- 172.21.15.163:0/2189917396 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9800cbc0 con 0x7fbbb0159c60 2023-10-09T06:36:47.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.157+0000 7fbbb56d3700 1 -- 172.21.15.163:0/2189917396 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbba00094f0 con 0x7fbbb0159c60 2023-10-09T06:36:47.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.157+0000 7fbba77fe700 1 -- 172.21.15.163:0/2189917396 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb98013470 con 0x7fbbb0159c60 2023-10-09T06:36:47.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.157+0000 7fbbb56d3700 1 -- 172.21.15.163:0/2189917396 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb00b49e0 con 0x7fbbb0159c60 2023-10-09T06:36:47.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.158+0000 7fbbb56d3700 1 -- 172.21.15.163:0/2189917396 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb00b3b90 con 0x7fbbb0159c60 2023-10-09T06:36:47.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.159+0000 7fbba77fe700 1 -- 172.21.15.163:0/2189917396 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fbb98013820 con 0x7fbbb0159c60 2023-10-09T06:36:47.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.159+0000 7fbba77fe700 1 --2- 172.21.15.163:0/2189917396 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb90067290 0x7fbb90069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:47.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.159+0000 7fbba77fe700 1 -- 172.21.15.163:0/2189917396 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbb98084d80 con 0x7fbbb0159c60 2023-10-09T06:36:47.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.159+0000 7fbbaeffd700 1 --2- 172.21.15.163:0/2189917396 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb90067290 0x7fbb90069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:47.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.160+0000 7fbbaeffd700 1 --2- 172.21.15.163:0/2189917396 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb90067290 0x7fbb90069740 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fbba00099b0 tx=0x7fbba0017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:47.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.163+0000 7fbba77fe700 1 -- 172.21.15.163:0/2189917396 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb98055670 con 0x7fbbb0159c60 2023-10-09T06:36:47.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.390+0000 7fbbb56d3700 1 -- 172.21.15.163:0/2189917396 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbbb004c3c0 con 0x7fbbb0159c60 2023-10-09T06:36:47.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.390+0000 7fbba77fe700 1 -- 172.21.15.163:0/2189917396 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4611 (secure 0 0 0) 0x7fbb98021090 con 0x7fbbb0159c60 2023-10-09T06:36:47.391 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:47.391 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3452423203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3452423203},{"type":"v1","addr":"172.21.15.181:6836","nonce":3452423203}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6298,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2097756943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2097756943},{"type":"v1","addr":"172.21.15.181:6837","nonce":2097756943}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6831,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3240892979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3240892979},{"type":"v1","addr":"172.21.15.163:6834","nonce":3240892979}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:36:43.382051+0000","modified":"2023-10-09T06:36:47.014979+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6834},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6834":{"gid":6834,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/1684208805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1684208805},{"type":"v1","addr":"172.21.15.163:6835","nonce":1684208805}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-09T06:36:47.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.392+0000 7fbbb56d3700 1 -- 172.21.15.163:0/2189917396 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb90067290 msgr2=0x7fbb90069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:47.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.392+0000 7fbbb56d3700 1 --2- 172.21.15.163:0/2189917396 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb90067290 0x7fbb90069740 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fbba00099b0 tx=0x7fbba0017000 comp rx=0 tx=0).stop 2023-10-09T06:36:47.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.392+0000 7fbbb56d3700 1 -- 172.21.15.163:0/2189917396 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbb0159c60 msgr2=0x7fbbb014d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:47.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.392+0000 7fbbb56d3700 1 --2- 172.21.15.163:0/2189917396 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbb0159c60 0x7fbbb014d1c0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fbb9800ec80 tx=0x7fbb980091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.392+0000 7fbbb56d3700 1 -- 172.21.15.163:0/2189917396 shutdown_connections 2023-10-09T06:36:47.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.392+0000 7fbbb56d3700 1 --2- 172.21.15.163:0/2189917396 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb90067290 0x7fbb90069740 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.392+0000 7fbbb56d3700 1 --2- 172.21.15.163:0/2189917396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbb015db10 0x7fbbb0153850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.392+0000 7fbbb56d3700 1 --2- 172.21.15.163:0/2189917396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbb0159890 0x7fbbb00b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.392+0000 7fbbb56d3700 1 --2- 172.21.15.163:0/2189917396 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbb0159c60 0x7fbbb014d1c0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.392+0000 7fbbb56d3700 1 -- 172.21.15.163:0/2189917396 >> 172.21.15.163:0/2189917396 conn(0x7fbbb0068e20 msgr2=0x7fbbb00c5590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:47.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.392+0000 7fbbb56d3700 1 -- 172.21.15.163:0/2189917396 shutdown_connections 2023-10-09T06:36:47.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.392+0000 7fbbb56d3700 1 -- 172.21.15.163:0/2189917396 wait complete. 2023-10-09T06:36:47.395 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 66 2023-10-09T06:36:47.406 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:36:47.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.520+0000 7f4e99885700 1 Processor -- start 2023-10-09T06:36:47.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.520+0000 7f4e99885700 1 -- start start 2023-10-09T06:36:47.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.521+0000 7f4e99885700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4e94159c60 0x7f4e9415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:47.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.521+0000 7f4e99885700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4e9415db10 0x7f4e94069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:47.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.521+0000 7f4e99885700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e9415a160 con 0x7f4e94159890 2023-10-09T06:36:47.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.521+0000 7f4e99885700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e9415df50 con 0x7f4e9415db10 2023-10-09T06:36:47.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.521+0000 7f4e99885700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e940d13a0 con 0x7f4e94159c60 2023-10-09T06:36:47.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.521+0000 7f4e937fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4e9415db10 0x7f4e94069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:47.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.521+0000 7f4e937fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4e9415db10 0x7f4e94069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41446/0 (socket says 172.21.15.163:41446) 2023-10-09T06:36:47.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.521+0000 7f4e937fe700 1 -- 172.21.15.163:0/3977025093 learned_addr learned my addr 172.21.15.163:0/3977025093 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:47.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.521+0000 7f4e927fc700 1 --2- 172.21.15.163:0/3977025093 >> v2:172.21.15.181:3300/0 conn(0x7f4e94159c60 0x7f4e9415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:47.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.521+0000 7f4e937fe700 1 -- 172.21.15.163:0/3977025093 >> v2:172.21.15.181:3300/0 conn(0x7f4e94159c60 msgr2=0x7f4e9415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:47.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.521+0000 7f4e937fe700 1 --2- 172.21.15.163:0/3977025093 >> v2:172.21.15.181:3300/0 conn(0x7f4e94159c60 0x7f4e9415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.521+0000 7f4e937fe700 1 -- 172.21.15.163:0/3977025093 >> v1:172.21.15.181:6789/0 conn(0x7f4e94159890 legacy=0x7f4e9415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:47.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.521+0000 7f4e937fe700 1 -- 172.21.15.163:0/3977025093 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9415e140 con 0x7f4e9415db10 2023-10-09T06:36:47.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.521+0000 7f4e927fc700 1 --2- 172.21.15.163:0/3977025093 >> v2:172.21.15.181:3300/0 conn(0x7f4e94159c60 0x7f4e9415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:36:47.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.521+0000 7f4e937fe700 1 --2- 172.21.15.163:0/3977025093 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e9415db10 0x7f4e94069560 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f4e940b67f0 tx=0x7f4e8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db71b1119c7cd6ca server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:47.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.521+0000 7f4e91ffb700 1 -- 172.21.15.163:0/3977025093 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c00b820 con 0x7f4e9415db10 2023-10-09T06:36:47.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.521+0000 7f4e91ffb700 1 -- 172.21.15.163:0/3977025093 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8c00b980 con 0x7f4e9415db10 2023-10-09T06:36:47.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.521+0000 7f4e91ffb700 1 -- 172.21.15.163:0/3977025093 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c00bd20 con 0x7f4e9415db10 2023-10-09T06:36:47.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.522+0000 7f4e99885700 1 -- 172.21.15.163:0/3977025093 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e9415db10 msgr2=0x7f4e94069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:47.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.522+0000 7f4e99885700 1 --2- 172.21.15.163:0/3977025093 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e9415db10 0x7f4e94069560 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f4e940b67f0 tx=0x7f4e8c009250 comp rx=0 tx=0).stop 2023-10-09T06:36:47.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.522+0000 7f4e99885700 1 -- 172.21.15.163:0/3977025093 shutdown_connections 2023-10-09T06:36:47.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.522+0000 7f4e99885700 1 --2- 172.21.15.163:0/3977025093 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e9415db10 0x7f4e94069560 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.522+0000 7f4e99885700 1 --2- 172.21.15.163:0/3977025093 >> v2:172.21.15.181:3300/0 conn(0x7f4e94159c60 0x7f4e9415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.522+0000 7f4e99885700 1 -- 172.21.15.163:0/3977025093 >> 172.21.15.163:0/3977025093 conn(0x7f4e94068e20 msgr2=0x7f4e940d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:47.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.522+0000 7f4e99885700 1 -- 172.21.15.163:0/3977025093 shutdown_connections 2023-10-09T06:36:47.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.522+0000 7f4e99885700 1 -- 172.21.15.163:0/3977025093 wait complete. 2023-10-09T06:36:47.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.523+0000 7f4e99885700 1 Processor -- start 2023-10-09T06:36:47.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.523+0000 7f4e99885700 1 -- start start 2023-10-09T06:36:47.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.524+0000 7f4e99885700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e94159890 0x7f4e940b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:47.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.524+0000 7f4e99885700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e94159c60 0x7f4e94169c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:47.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.524+0000 7f4e99885700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e9415db10 0x7f4e9415c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:47.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.524+0000 7f4e99885700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e940b1700 con 0x7f4e94159890 2023-10-09T06:36:47.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.524+0000 7f4e99885700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9415cdd0 con 0x7f4e9415db10 2023-10-09T06:36:47.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.524+0000 7f4e99885700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9416a140 con 0x7f4e94159c60 2023-10-09T06:36:47.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.524+0000 7f4e92ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e94159890 0x7f4e940b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:47.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.524+0000 7f4e92ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e94159890 0x7f4e940b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41462/0 (socket says 172.21.15.163:41462) 2023-10-09T06:36:47.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.524+0000 7f4e92ffd700 1 -- 172.21.15.163:0/120357360 learned_addr learned my addr 172.21.15.163:0/120357360 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:47.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.524+0000 7f4e92ffd700 1 -- 172.21.15.163:0/120357360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e94159c60 msgr2=0x7f4e94169c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:47.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.524+0000 7f4e92ffd700 1 --2- 172.21.15.163:0/120357360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e94159c60 0x7f4e94169c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.524+0000 7f4e92ffd700 1 -- 172.21.15.163:0/120357360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e9415db10 msgr2=0x7f4e9415c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:47.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.524+0000 7f4e92ffd700 1 --2- 172.21.15.163:0/120357360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e9415db10 0x7f4e9415c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.524+0000 7f4e92ffd700 1 -- 172.21.15.163:0/120357360 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8c0094f0 con 0x7f4e94159890 2023-10-09T06:36:47.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.524+0000 7f4e92ffd700 1 --2- 172.21.15.163:0/120357360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e94159890 0x7f4e940b3590 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f4e8400bbb0 tx=0x7f4e84009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:47.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.524+0000 7f4e8affd700 1 -- 172.21.15.163:0/120357360 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e840092e0 con 0x7f4e94159890 2023-10-09T06:36:47.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.525+0000 7f4e8affd700 1 -- 172.21.15.163:0/120357360 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8400c7a0 con 0x7f4e94159890 2023-10-09T06:36:47.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.525+0000 7f4e99885700 1 -- 172.21.15.163:0/120357360 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e940b0dd0 con 0x7f4e94159890 2023-10-09T06:36:47.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.525+0000 7f4e8affd700 1 -- 172.21.15.163:0/120357360 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8400cb40 con 0x7f4e94159890 2023-10-09T06:36:47.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.525+0000 7f4e99885700 1 -- 172.21.15.163:0/120357360 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e940b1390 con 0x7f4e94159890 2023-10-09T06:36:47.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.526+0000 7f4e99885700 1 -- 172.21.15.163:0/120357360 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e940cbf20 con 0x7f4e94159890 2023-10-09T06:36:47.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.526+0000 7f4e8affd700 1 -- 172.21.15.163:0/120357360 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f4e84012070 con 0x7f4e94159890 2023-10-09T06:36:47.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.526+0000 7f4e8affd700 1 --2- 172.21.15.163:0/120357360 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4e74067290 0x7f4e74069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:47.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.526+0000 7f4e8affd700 1 -- 172.21.15.163:0/120357360 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4e84084860 con 0x7f4e94159890 2023-10-09T06:36:47.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.527+0000 7f4e927fc700 1 --2- 172.21.15.163:0/120357360 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4e74067290 0x7f4e74069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:47.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.527+0000 7f4e927fc700 1 --2- 172.21.15.163:0/120357360 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4e74067290 0x7f4e74069740 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4e940b6630 tx=0x7f4e7c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:47.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.530+0000 7f4e8affd700 1 -- 172.21.15.163:0/120357360 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4e840515c0 con 0x7f4e94159890 2023-10-09T06:36:47.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.714+0000 7f4e99885700 1 -- 172.21.15.163:0/120357360 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e9404c3c0 con 0x7f4e94159890 2023-10-09T06:36:47.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.715+0000 7f4e8affd700 1 -- 172.21.15.163:0/120357360 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+15404 (secure 0 0 0) 0x7f4e84054be0 con 0x7f4e94159890 2023-10-09T06:36:47.716 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:47.716 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":101,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:36:44.526778+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:36:41.723219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-09T06:36:42.342713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:36:47.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.717+0000 7f4e99885700 1 -- 172.21.15.163:0/120357360 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4e74067290 msgr2=0x7f4e74069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:47.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.717+0000 7f4e99885700 1 --2- 172.21.15.163:0/120357360 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4e74067290 0x7f4e74069740 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4e940b6630 tx=0x7f4e7c008040 comp rx=0 tx=0).stop 2023-10-09T06:36:47.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.717+0000 7f4e99885700 1 -- 172.21.15.163:0/120357360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e94159890 msgr2=0x7f4e940b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:47.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.717+0000 7f4e99885700 1 --2- 172.21.15.163:0/120357360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e94159890 0x7f4e940b3590 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f4e8400bbb0 tx=0x7f4e84009840 comp rx=0 tx=0).stop 2023-10-09T06:36:47.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.717+0000 7f4e99885700 1 -- 172.21.15.163:0/120357360 shutdown_connections 2023-10-09T06:36:47.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.717+0000 7f4e99885700 1 --2- 172.21.15.163:0/120357360 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4e74067290 0x7f4e74069740 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.717+0000 7f4e99885700 1 --2- 172.21.15.163:0/120357360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e94159890 0x7f4e940b3590 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.717+0000 7f4e99885700 1 --2- 172.21.15.163:0/120357360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e9415db10 0x7f4e9415c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.717+0000 7f4e99885700 1 --2- 172.21.15.163:0/120357360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e94159c60 0x7f4e94169c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.717+0000 7f4e99885700 1 -- 172.21.15.163:0/120357360 >> 172.21.15.163:0/120357360 conn(0x7f4e94068e20 msgr2=0x7f4e941588c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:47.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.717+0000 7f4e99885700 1 -- 172.21.15.163:0/120357360 shutdown_connections 2023-10-09T06:36:47.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.717+0000 7f4e99885700 1 -- 172.21.15.163:0/120357360 wait complete. 2023-10-09T06:36:47.730 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-09T06:36:47.731 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:36:47.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.844+0000 7f55393ea700 1 Processor -- start 2023-10-09T06:36:47.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.844+0000 7f55393ea700 1 -- start start 2023-10-09T06:36:47.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.845+0000 7f55393ea700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f55341578b0 0x7f5534155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:47.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.845+0000 7f55393ea700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55340ad930 con 0x7f5534155c10 2023-10-09T06:36:47.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.845+0000 7f55393ea700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55340acd20 con 0x7f55340af2f0 2023-10-09T06:36:47.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.845+0000 7f55393ea700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55340ad000 con 0x7f55341578b0 2023-10-09T06:36:47.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.845+0000 7f5532ffd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f5534155c10 0x7f55340ac230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:37968/0 (socket says 172.21.15.163:37968) 2023-10-09T06:36:47.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.845+0000 7f5532ffd700 1 -- 172.21.15.163:0/2857427073 learned_addr learned my addr 172.21.15.163:0/2857427073 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:47.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.845+0000 7f55327fc700 1 -- 172.21.15.163:0/2857427073 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3244218764 0 0) 0x7f55340ad930 con 0x7f5534155c10 2023-10-09T06:36:47.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.846+0000 7f552a7fc700 1 --2- 172.21.15.163:0/2857427073 >> v2:172.21.15.181:3300/0 conn(0x7f55341578b0 0x7f5534155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:47.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.846+0000 7f55327fc700 1 -- 172.21.15.163:0/2857427073 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5520003570 con 0x7f5534155c10 2023-10-09T06:36:47.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.846+0000 7f55327fc700 1 -- 172.21.15.163:0/2857427073 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2023637428 0 0) 0x7f5520003570 con 0x7f5534155c10 2023-10-09T06:36:47.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.846+0000 7f55327fc700 1 -- 172.21.15.163:0/2857427073 >> v2:172.21.15.181:3300/0 conn(0x7f55341578b0 msgr2=0x7f5534155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:47.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.846+0000 7f55327fc700 1 --2- 172.21.15.163:0/2857427073 >> v2:172.21.15.181:3300/0 conn(0x7f55341578b0 0x7f5534155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.846+0000 7f55327fc700 1 -- 172.21.15.163:0/2857427073 >> v1:172.21.15.181:6789/0 conn(0x7f55340af2f0 legacy=0x7f55340af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:47.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.846+0000 7f55327fc700 1 -- 172.21.15.163:0/2857427073 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55340ad2e0 con 0x7f5534155c10 2023-10-09T06:36:47.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.846+0000 7f55327fc700 1 -- 172.21.15.163:0/2857427073 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f5524001470 con 0x7f5534155c10 2023-10-09T06:36:47.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.846+0000 7f55327fc700 1 -- 172.21.15.163:0/2857427073 >> v1:172.21.15.163:6789/0 conn(0x7f5534155c10 legacy=0x7f55340ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:47.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.846+0000 7f55327fc700 1 --2- 172.21.15.163:0/2857427073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5520003a60 0x7f5520003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:47.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.846+0000 7f55327fc700 1 --2- 172.21.15.163:0/2857427073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5520004810 0x7f5520004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:47.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.846+0000 7f55327fc700 1 --2- 172.21.15.163:0/2857427073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5520005230 0x7f552000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:47.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.846+0000 7f55327fc700 1 -- 172.21.15.163:0/2857427073 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55340ad000 con 0x7f5520004810 2023-10-09T06:36:47.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.846+0000 7f55327fc700 1 -- 172.21.15.163:0/2857427073 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552000c160 con 0x7f5520005230 2023-10-09T06:36:47.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.846+0000 7f55327fc700 1 -- 172.21.15.163:0/2857427073 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f552000c320 con 0x7f5520003a60 2023-10-09T06:36:47.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.846+0000 7f552a7fc700 1 --2- 172.21.15.163:0/2857427073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5520004810 0x7f5520004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:47.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.846+0000 7f5532ffd700 1 --2- 172.21.15.163:0/2857427073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5520003a60 0x7f5520003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:47.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.846+0000 7f55337fe700 1 --2- 172.21.15.163:0/2857427073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5520005230 0x7f552000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:47.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.846+0000 7f552a7fc700 1 -- 172.21.15.163:0/2857427073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5520003a60 msgr2=0x7f5520003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:47.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.846+0000 7f552a7fc700 1 --2- 172.21.15.163:0/2857427073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5520003a60 0x7f5520003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.846+0000 7f552a7fc700 1 -- 172.21.15.163:0/2857427073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5520005230 msgr2=0x7f552000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:47.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.846+0000 7f552a7fc700 1 --2- 172.21.15.163:0/2857427073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5520005230 0x7f552000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.846+0000 7f552a7fc700 1 -- 172.21.15.163:0/2857427073 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5520003570 con 0x7f5520004810 2023-10-09T06:36:47.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.846+0000 7f55337fe700 1 --2- 172.21.15.163:0/2857427073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5520005230 0x7f552000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:47.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.846+0000 7f5532ffd700 1 --2- 172.21.15.163:0/2857427073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5520003a60 0x7f5520003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:47.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.846+0000 7f55337fe700 1 -- 172.21.15.163:0/2857427073 reap_dead start 2023-10-09T06:36:47.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.846+0000 7f552a7fc700 1 --2- 172.21.15.163:0/2857427073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5520004810 0x7f5520004c80 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f551c00bb90 tx=0x7f551c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876ee104e334af5 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:47.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.847+0000 7f55327fc700 1 -- 172.21.15.163:0/2857427073 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c00d5d0 con 0x7f5520004810 2023-10-09T06:36:47.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.847+0000 7f55327fc700 1 -- 172.21.15.163:0/2857427073 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551c00d730 con 0x7f5520004810 2023-10-09T06:36:47.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.847+0000 7f55327fc700 1 -- 172.21.15.163:0/2857427073 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c00daf0 con 0x7f5520004810 2023-10-09T06:36:47.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.847+0000 7f55393ea700 1 -- 172.21.15.163:0/2857427073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5520004810 msgr2=0x7f5520004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:47.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.847+0000 7f55393ea700 1 --2- 172.21.15.163:0/2857427073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5520004810 0x7f5520004c80 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f551c00bb90 tx=0x7f551c008040 comp rx=0 tx=0).stop 2023-10-09T06:36:47.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.847+0000 7f55393ea700 1 -- 172.21.15.163:0/2857427073 shutdown_connections 2023-10-09T06:36:47.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.847+0000 7f55393ea700 1 --2- 172.21.15.163:0/2857427073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5520004810 0x7f5520004c80 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.847+0000 7f55393ea700 1 -- 172.21.15.163:0/2857427073 >> 172.21.15.163:0/2857427073 conn(0x7f553406a320 msgr2=0x7f55340cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:47.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.847+0000 7f55393ea700 1 -- 172.21.15.163:0/2857427073 shutdown_connections 2023-10-09T06:36:47.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.847+0000 7f55393ea700 1 -- 172.21.15.163:0/2857427073 wait complete. 2023-10-09T06:36:47.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.848+0000 7f55393ea700 1 Processor -- start 2023-10-09T06:36:47.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.848+0000 7f55393ea700 1 -- start start 2023-10-09T06:36:47.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.849+0000 7f55393ea700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5520004810 0x7f5534168570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:47.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.849+0000 7f55393ea700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55340cca00 0x7f5534152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:47.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.849+0000 7f55393ea700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55340a54c0 0x7f55340a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:47.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.849+0000 7f55393ea700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55340a6e70 con 0x7f5520004810 2023-10-09T06:36:47.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.849+0000 7f55393ea700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55340a7cd0 con 0x7f55340a54c0 2023-10-09T06:36:47.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.849+0000 7f55393ea700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55340a5e80 con 0x7f55340cca00 2023-10-09T06:36:47.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.849+0000 7f5532ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5520004810 0x7f5534168570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:47.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.849+0000 7f5532ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5520004810 0x7f5534168570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41488/0 (socket says 172.21.15.163:41488) 2023-10-09T06:36:47.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.849+0000 7f5532ffd700 1 -- 172.21.15.163:0/258888704 learned_addr learned my addr 172.21.15.163:0/258888704 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:47.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.849+0000 7f55337fe700 1 --2- 172.21.15.163:0/258888704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55340a54c0 0x7f55340a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:47.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.849+0000 7f552a7fc700 1 --2- 172.21.15.163:0/258888704 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55340cca00 0x7f5534152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:47.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.849+0000 7f5532ffd700 1 -- 172.21.15.163:0/258888704 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55340cca00 msgr2=0x7f5534152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:47.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.849+0000 7f5532ffd700 1 --2- 172.21.15.163:0/258888704 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55340cca00 0x7f5534152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.849+0000 7f5532ffd700 1 -- 172.21.15.163:0/258888704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55340a54c0 msgr2=0x7f55340a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:47.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.849+0000 7f5532ffd700 1 --2- 172.21.15.163:0/258888704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55340a54c0 0x7f55340a5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:47.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.849+0000 7f5532ffd700 1 -- 172.21.15.163:0/258888704 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5534168ab0 con 0x7f5520004810 2023-10-09T06:36:47.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.850+0000 7f5532ffd700 1 --2- 172.21.15.163:0/258888704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5520004810 0x7f5534168570 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f552400e940 tx=0x7f5524001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:47.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.850+0000 7f552b7fe700 1 -- 172.21.15.163:0/258888704 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552400eee0 con 0x7f5520004810 2023-10-09T06:36:47.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.850+0000 7f552b7fe700 1 -- 172.21.15.163:0/258888704 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f552400c7e0 con 0x7f5520004810 2023-10-09T06:36:47.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.850+0000 7f55393ea700 1 -- 172.21.15.163:0/258888704 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55340a7050 con 0x7f5520004810 2023-10-09T06:36:47.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.850+0000 7f552b7fe700 1 -- 172.21.15.163:0/258888704 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552400cba0 con 0x7f5520004810 2023-10-09T06:36:47.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.850+0000 7f55393ea700 1 -- 172.21.15.163:0/258888704 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55340a73e0 con 0x7f5520004810 2023-10-09T06:36:47.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.851+0000 7f55393ea700 1 -- 172.21.15.163:0/258888704 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55340a7e10 con 0x7f5520004810 2023-10-09T06:36:47.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.851+0000 7f552b7fe700 1 -- 172.21.15.163:0/258888704 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f552401d460 con 0x7f5520004810 2023-10-09T06:36:47.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.851+0000 7f552b7fe700 1 --2- 172.21.15.163:0/258888704 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5514067290 0x7f5514069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:47.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.852+0000 7f552b7fe700 1 -- 172.21.15.163:0/258888704 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5524084900 con 0x7f5520004810 2023-10-09T06:36:47.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.852+0000 7f552a7fc700 1 --2- 172.21.15.163:0/258888704 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5514067290 0x7f5514069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:47.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.852+0000 7f552a7fc700 1 --2- 172.21.15.163:0/258888704 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5514067290 0x7f5514069740 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f551c00bb90 tx=0x7f551c01b000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:47.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:47.856+0000 7f552b7fe700 1 -- 172.21.15.163:0/258888704 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55340a7e10 con 0x7f5520004810 2023-10-09T06:36:48.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.038+0000 7f55393ea700 1 -- 172.21.15.163:0/258888704 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f55340a9390 con 0x7f5514067290 2023-10-09T06:36:48.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.047+0000 7f552b7fe700 1 -- 172.21.15.163:0/258888704 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289907 (secure 0 0 0) 0x7f55340a9390 con 0x7f5514067290 2023-10-09T06:36:48.051 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:48.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.050+0000 7f55393ea700 1 -- 172.21.15.163:0/258888704 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5514067290 msgr2=0x7f5514069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:48.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.050+0000 7f55393ea700 1 --2- 172.21.15.163:0/258888704 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5514067290 0x7f5514069740 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f551c00bb90 tx=0x7f551c01b000 comp rx=0 tx=0).stop 2023-10-09T06:36:48.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.050+0000 7f55393ea700 1 -- 172.21.15.163:0/258888704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5520004810 msgr2=0x7f5534168570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:48.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.050+0000 7f55393ea700 1 --2- 172.21.15.163:0/258888704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5520004810 0x7f5534168570 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f552400e940 tx=0x7f5524001e90 comp rx=0 tx=0).stop 2023-10-09T06:36:48.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.050+0000 7f55393ea700 1 -- 172.21.15.163:0/258888704 shutdown_connections 2023-10-09T06:36:48.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.050+0000 7f55393ea700 1 --2- 172.21.15.163:0/258888704 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5514067290 0x7f5514069740 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:48.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.050+0000 7f55393ea700 1 --2- 172.21.15.163:0/258888704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5520004810 0x7f5534168570 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:48.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.050+0000 7f55393ea700 1 --2- 172.21.15.163:0/258888704 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55340a54c0 0x7f55340a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:48.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.050+0000 7f55393ea700 1 --2- 172.21.15.163:0/258888704 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55340cca00 0x7f5534152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:48.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.050+0000 7f55393ea700 1 -- 172.21.15.163:0/258888704 >> 172.21.15.163:0/258888704 conn(0x7f553406a320 msgr2=0x7f55341534e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:48.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.050+0000 7f55393ea700 1 -- 172.21.15.163:0/258888704 shutdown_connections 2023-10-09T06:36:48.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.050+0000 7f55393ea700 1 -- 172.21.15.163:0/258888704 wait complete. 2023-10-09T06:36:48.054 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:36:48.063 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2023-10-09T06:36:46.388274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2413880,"kb_used_data":11000,"kb_used_omap":0,"kb_used_meta":2402816,"kb_avail":752560840,"statfs":{"total":773094113280,"available":770622300160,"internally_reserved":0,"allocated":11264000,"data_stored":5516516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2460483584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.398669"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551602+0000","last_change":"2023-10-09T06:36:43.547872+0000","last_active":"2023-10-09T06:36:43.551602+0000","last_peered":"2023-10-09T06:36:43.551602+0000","last_clean":"2023-10-09T06:36:43.551602+0000","last_became_active":"2023-10-09T06:36:43.547147+0000","last_became_peered":"2023-10-09T06:36:43.547147+0000","last_unstale":"2023-10-09T06:36:43.551602+0000","last_undegraded":"2023-10-09T06:36:43.551602+0000","last_fullsized":"2023-10-09T06:36:43.551602+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:04:17.924536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.553611+0000","last_change":"2023-10-09T06:36:43.546856+0000","last_active":"2023-10-09T06:36:43.553611+0000","last_peered":"2023-10-09T06:36:43.553611+0000","last_clean":"2023-10-09T06:36:43.553611+0000","last_became_active":"2023-10-09T06:36:43.542794+0000","last_became_peered":"2023-10-09T06:36:43.542794+0000","last_unstale":"2023-10-09T06:36:43.553611+0000","last_undegraded":"2023-10-09T06:36:43.553611+0000","last_fullsized":"2023-10-09T06:36:43.553611+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:10:09.897672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.553501+0000","last_change":"2023-10-09T06:36:43.548848+0000","last_active":"2023-10-09T06:36:43.553501+0000","last_peered":"2023-10-09T06:36:43.553501+0000","last_clean":"2023-10-09T06:36:43.553501+0000","last_became_active":"2023-10-09T06:36:43.548602+0000","last_became_peered":"2023-10-09T06:36:43.548602+0000","last_unstale":"2023-10-09T06:36:43.553501+0000","last_undegraded":"2023-10-09T06:36:43.553501+0000","last_fullsized":"2023-10-09T06:36:43.553501+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:24:32.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.548889+0000","last_change":"2023-10-09T06:36:43.547192+0000","last_active":"2023-10-09T06:36:43.548889+0000","last_peered":"2023-10-09T06:36:43.548889+0000","last_clean":"2023-10-09T06:36:43.548889+0000","last_became_active":"2023-10-09T06:36:43.546933+0000","last_became_peered":"2023-10-09T06:36:43.546933+0000","last_unstale":"2023-10-09T06:36:43.548889+0000","last_undegraded":"2023-10-09T06:36:43.548889+0000","last_fullsized":"2023-10-09T06:36:43.548889+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:22:42.375347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.549076+0000","last_change":"2023-10-09T06:36:43.543257+0000","last_active":"2023-10-09T06:36:43.549076+0000","last_peered":"2023-10-09T06:36:43.549076+0000","last_clean":"2023-10-09T06:36:43.549076+0000","last_became_active":"2023-10-09T06:36:43.543002+0000","last_became_peered":"2023-10-09T06:36:43.543002+0000","last_unstale":"2023-10-09T06:36:43.549076+0000","last_undegraded":"2023-10-09T06:36:43.549076+0000","last_fullsized":"2023-10-09T06:36:43.549076+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:36:18.197746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551041+0000","last_change":"2023-10-09T06:36:43.548543+0000","last_active":"2023-10-09T06:36:43.551041+0000","last_peered":"2023-10-09T06:36:43.551041+0000","last_clean":"2023-10-09T06:36:43.551041+0000","last_became_active":"2023-10-09T06:36:43.548294+0000","last_became_peered":"2023-10-09T06:36:43.548294+0000","last_unstale":"2023-10-09T06:36:43.551041+0000","last_undegraded":"2023-10-09T06:36:43.551041+0000","last_fullsized":"2023-10-09T06:36:43.551041+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:10:02.219425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.549526+0000","last_change":"2023-10-09T06:36:43.546625+0000","last_active":"2023-10-09T06:36:43.549526+0000","last_peered":"2023-10-09T06:36:43.549526+0000","last_clean":"2023-10-09T06:36:43.549526+0000","last_became_active":"2023-10-09T06:36:43.546234+0000","last_became_peered":"2023-10-09T06:36:43.546234+0000","last_unstale":"2023-10-09T06:36:43.549526+0000","last_undegraded":"2023-10-09T06:36:43.549526+0000","last_fullsized":"2023-10-09T06:36:43.549526+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:47:41.770796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551069+0000","last_change":"2023-10-09T06:36:43.546096+0000","last_active":"2023-10-09T06:36:43.551069+0000","last_peered":"2023-10-09T06:36:43.551069+0000","last_clean":"2023-10-09T06:36:43.551069+0000","last_became_active":"2023-10-09T06:36:43.545812+0000","last_became_peered":"2023-10-09T06:36:43.545812+0000","last_unstale":"2023-10-09T06:36:43.551069+0000","last_undegraded":"2023-10-09T06:36:43.551069+0000","last_fullsized":"2023-10-09T06:36:43.551069+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:17:38.218770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551589+0000","last_change":"2023-10-09T06:36:43.547713+0000","last_active":"2023-10-09T06:36:43.551589+0000","last_peered":"2023-10-09T06:36:43.551589+0000","last_clean":"2023-10-09T06:36:43.551589+0000","last_became_active":"2023-10-09T06:36:43.546800+0000","last_became_peered":"2023-10-09T06:36:43.546800+0000","last_unstale":"2023-10-09T06:36:43.551589+0000","last_undegraded":"2023-10-09T06:36:43.551589+0000","last_fullsized":"2023-10-09T06:36:43.551589+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:47:23.297691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.548545+0000","last_change":"2023-10-09T06:36:43.545995+0000","last_active":"2023-10-09T06:36:43.548545+0000","last_peered":"2023-10-09T06:36:43.548545+0000","last_clean":"2023-10-09T06:36:43.548545+0000","last_became_active":"2023-10-09T06:36:43.545730+0000","last_became_peered":"2023-10-09T06:36:43.545730+0000","last_unstale":"2023-10-09T06:36:43.548545+0000","last_undegraded":"2023-10-09T06:36:43.548545+0000","last_fullsized":"2023-10-09T06:36:43.548545+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:42:58.199107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551610+0000","last_change":"2023-10-09T06:36:43.546256+0000","last_active":"2023-10-09T06:36:43.551610+0000","last_peered":"2023-10-09T06:36:43.551610+0000","last_clean":"2023-10-09T06:36:43.551610+0000","last_became_active":"2023-10-09T06:36:43.545944+0000","last_became_peered":"2023-10-09T06:36:43.545944+0000","last_unstale":"2023-10-09T06:36:43.551610+0000","last_undegraded":"2023-10-09T06:36:43.551610+0000","last_fullsized":"2023-10-09T06:36:43.551610+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:53:59.805364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.548854+0000","last_change":"2023-10-09T06:36:43.545645+0000","last_active":"2023-10-09T06:36:43.548854+0000","last_peered":"2023-10-09T06:36:43.548854+0000","last_clean":"2023-10-09T06:36:43.548854+0000","last_became_active":"2023-10-09T06:36:43.545377+0000","last_became_peered":"2023-10-09T06:36:43.545377+0000","last_unstale":"2023-10-09T06:36:43.548854+0000","last_undegraded":"2023-10-09T06:36:43.548854+0000","last_fullsized":"2023-10-09T06:36:43.548854+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:21:45.060458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.550933+0000","last_change":"2023-10-09T06:36:43.546740+0000","last_active":"2023-10-09T06:36:43.550933+0000","last_peered":"2023-10-09T06:36:43.550933+0000","last_clean":"2023-10-09T06:36:43.550933+0000","last_became_active":"2023-10-09T06:36:43.546354+0000","last_became_peered":"2023-10-09T06:36:43.546354+0000","last_unstale":"2023-10-09T06:36:43.550933+0000","last_undegraded":"2023-10-09T06:36:43.550933+0000","last_fullsized":"2023-10-09T06:36:43.550933+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:57:59.240733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.550833+0000","last_change":"2023-10-09T06:36:43.548122+0000","last_active":"2023-10-09T06:36:43.550833+0000","last_peered":"2023-10-09T06:36:43.550833+0000","last_clean":"2023-10-09T06:36:43.550833+0000","last_became_active":"2023-10-09T06:36:43.547844+0000","last_became_peered":"2023-10-09T06:36:43.547844+0000","last_unstale":"2023-10-09T06:36:43.550833+0000","last_undegraded":"2023-10-09T06:36:43.550833+0000","last_fullsized":"2023-10-09T06:36:43.550833+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:46:07.974080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551067+0000","last_change":"2023-10-09T06:36:43.546967+0000","last_active":"2023-10-09T06:36:43.551067+0000","last_peered":"2023-10-09T06:36:43.551067+0000","last_clean":"2023-10-09T06:36:43.551067+0000","last_became_active":"2023-10-09T06:36:43.546719+0000","last_became_peered":"2023-10-09T06:36:43.546719+0000","last_unstale":"2023-10-09T06:36:43.551067+0000","last_undegraded":"2023-10-09T06:36:43.551067+0000","last_fullsized":"2023-10-09T06:36:43.551067+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:35:43.890085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.552451+0000","last_change":"2023-10-09T06:36:43.550265+0000","last_active":"2023-10-09T06:36:43.552451+0000","last_peered":"2023-10-09T06:36:43.552451+0000","last_clean":"2023-10-09T06:36:43.552451+0000","last_became_active":"2023-10-09T06:36:43.550025+0000","last_became_peered":"2023-10-09T06:36:43.550025+0000","last_unstale":"2023-10-09T06:36:43.552451+0000","last_undegraded":"2023-10-09T06:36:43.552451+0000","last_fullsized":"2023-10-09T06:36:43.552451+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:58:26.502785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551484+0000","last_change":"2023-10-09T06:36:43.544799+0000","last_active":"2023-10-09T06:36:43.551484+0000","last_peered":"2023-10-09T06:36:43.551484+0000","last_clean":"2023-10-09T06:36:43.551484+0000","last_became_active":"2023-10-09T06:36:43.544407+0000","last_became_peered":"2023-10-09T06:36:43.544407+0000","last_unstale":"2023-10-09T06:36:43.551484+0000","last_undegraded":"2023-10-09T06:36:43.551484+0000","last_fullsized":"2023-10-09T06:36:43.551484+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:17:22.333508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.549650+0000","last_change":"2023-10-09T06:36:43.544031+0000","last_active":"2023-10-09T06:36:43.549650+0000","last_peered":"2023-10-09T06:36:43.549650+0000","last_clean":"2023-10-09T06:36:43.549650+0000","last_became_active":"2023-10-09T06:36:43.541890+0000","last_became_peered":"2023-10-09T06:36:43.541890+0000","last_unstale":"2023-10-09T06:36:43.549650+0000","last_undegraded":"2023-10-09T06:36:43.549650+0000","last_fullsized":"2023-10-09T06:36:43.549650+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:58:42.420936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.553720+0000","last_change":"2023-10-09T06:36:43.550368+0000","last_active":"2023-10-09T06:36:43.553720+0000","last_peered":"2023-10-09T06:36:43.553720+0000","last_clean":"2023-10-09T06:36:43.553720+0000","last_became_active":"2023-10-09T06:36:43.549880+0000","last_became_peered":"2023-10-09T06:36:43.549880+0000","last_unstale":"2023-10-09T06:36:43.553720+0000","last_undegraded":"2023-10-09T06:36:43.553720+0000","last_fullsized":"2023-10-09T06:36:43.553720+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:02:39.575264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551222+0000","last_change":"2023-10-09T06:36:43.543664+0000","last_active":"2023-10-09T06:36:43.551222+0000","last_peered":"2023-10-09T06:36:43.551222+0000","last_clean":"2023-10-09T06:36:43.551222+0000","last_became_active":"2023-10-09T06:36:43.543258+0000","last_became_peered":"2023-10-09T06:36:43.543258+0000","last_unstale":"2023-10-09T06:36:43.551222+0000","last_undegraded":"2023-10-09T06:36:43.551222+0000","last_fullsized":"2023-10-09T06:36:43.551222+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:14:04.479133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.553681+0000","last_change":"2023-10-09T06:36:43.549119+0000","last_active":"2023-10-09T06:36:43.553681+0000","last_peered":"2023-10-09T06:36:43.553681+0000","last_clean":"2023-10-09T06:36:43.553681+0000","last_became_active":"2023-10-09T06:36:43.548854+0000","last_became_peered":"2023-10-09T06:36:43.548854+0000","last_unstale":"2023-10-09T06:36:43.553681+0000","last_undegraded":"2023-10-09T06:36:43.553681+0000","last_fullsized":"2023-10-09T06:36:43.553681+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:29:33.629874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551066+0000","last_change":"2023-10-09T06:36:43.548070+0000","last_active":"2023-10-09T06:36:43.551066+0000","last_peered":"2023-10-09T06:36:43.551066+0000","last_clean":"2023-10-09T06:36:43.551066+0000","last_became_active":"2023-10-09T06:36:43.547795+0000","last_became_peered":"2023-10-09T06:36:43.547795+0000","last_unstale":"2023-10-09T06:36:43.551066+0000","last_undegraded":"2023-10-09T06:36:43.551066+0000","last_fullsized":"2023-10-09T06:36:43.551066+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:46:07.974080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551293+0000","last_change":"2023-10-09T06:36:43.543870+0000","last_active":"2023-10-09T06:36:43.551293+0000","last_peered":"2023-10-09T06:36:43.551293+0000","last_clean":"2023-10-09T06:36:43.551293+0000","last_became_active":"2023-10-09T06:36:43.543379+0000","last_became_peered":"2023-10-09T06:36:43.543379+0000","last_unstale":"2023-10-09T06:36:43.551293+0000","last_undegraded":"2023-10-09T06:36:43.551293+0000","last_fullsized":"2023-10-09T06:36:43.551293+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:36:16.356718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551564+0000","last_change":"2023-10-09T06:36:43.548755+0000","last_active":"2023-10-09T06:36:43.551564+0000","last_peered":"2023-10-09T06:36:43.551564+0000","last_clean":"2023-10-09T06:36:43.551564+0000","last_became_active":"2023-10-09T06:36:43.548531+0000","last_became_peered":"2023-10-09T06:36:43.548531+0000","last_unstale":"2023-10-09T06:36:43.551564+0000","last_undegraded":"2023-10-09T06:36:43.551564+0000","last_fullsized":"2023-10-09T06:36:43.551564+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:16:37.146078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551462+0000","last_change":"2023-10-09T06:36:43.544787+0000","last_active":"2023-10-09T06:36:43.551462+0000","last_peered":"2023-10-09T06:36:43.551462+0000","last_clean":"2023-10-09T06:36:43.551462+0000","last_became_active":"2023-10-09T06:36:43.544105+0000","last_became_peered":"2023-10-09T06:36:43.544105+0000","last_unstale":"2023-10-09T06:36:43.551462+0000","last_undegraded":"2023-10-09T06:36:43.551462+0000","last_fullsized":"2023-10-09T06:36:43.551462+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:39:47.384191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.552392+0000","last_change":"2023-10-09T06:36:43.549176+0000","last_active":"2023-10-09T06:36:43.552392+0000","last_peered":"2023-10-09T06:36:43.552392+0000","last_clean":"2023-10-09T06:36:43.552392+0000","last_became_active":"2023-10-09T06:36:43.548903+0000","last_became_peered":"2023-10-09T06:36:43.548903+0000","last_unstale":"2023-10-09T06:36:43.552392+0000","last_undegraded":"2023-10-09T06:36:43.552392+0000","last_fullsized":"2023-10-09T06:36:43.552392+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:36:25.109886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.545078+0000","last_change":"2023-10-09T06:36:43.542089+0000","last_active":"2023-10-09T06:36:43.545078+0000","last_peered":"2023-10-09T06:36:43.545078+0000","last_clean":"2023-10-09T06:36:43.545078+0000","last_became_active":"2023-10-09T06:36:43.541796+0000","last_became_peered":"2023-10-09T06:36:43.541796+0000","last_unstale":"2023-10-09T06:36:43.545078+0000","last_undegraded":"2023-10-09T06:36:43.545078+0000","last_fullsized":"2023-10-09T06:36:43.545078+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:29:33.629874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.552882+0000","last_change":"2023-10-09T06:36:43.550593+0000","last_active":"2023-10-09T06:36:43.552882+0000","last_peered":"2023-10-09T06:36:43.552882+0000","last_clean":"2023-10-09T06:36:43.552882+0000","last_became_active":"2023-10-09T06:36:43.550313+0000","last_became_peered":"2023-10-09T06:36:43.550313+0000","last_unstale":"2023-10-09T06:36:43.552882+0000","last_undegraded":"2023-10-09T06:36:43.552882+0000","last_fullsized":"2023-10-09T06:36:43.552882+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:24:32.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.553623+0000","last_change":"2023-10-09T06:36:43.547018+0000","last_active":"2023-10-09T06:36:43.553623+0000","last_peered":"2023-10-09T06:36:43.553623+0000","last_clean":"2023-10-09T06:36:43.553623+0000","last_became_active":"2023-10-09T06:36:43.543878+0000","last_became_peered":"2023-10-09T06:36:43.543878+0000","last_unstale":"2023-10-09T06:36:43.553623+0000","last_undegraded":"2023-10-09T06:36:43.553623+0000","last_fullsized":"2023-10-09T06:36:43.553623+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:32:17.890567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.549548+0000","last_change":"2023-10-09T06:36:43.548486+0000","last_active":"2023-10-09T06:36:43.549548+0000","last_peered":"2023-10-09T06:36:43.549548+0000","last_clean":"2023-10-09T06:36:43.549548+0000","last_became_active":"2023-10-09T06:36:43.548223+0000","last_became_peered":"2023-10-09T06:36:43.548223+0000","last_unstale":"2023-10-09T06:36:43.549548+0000","last_undegraded":"2023-10-09T06:36:43.549548+0000","last_fullsized":"2023-10-09T06:36:43.549548+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:34:01.423040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.548555+0000","last_change":"2023-10-09T06:36:43.542531+0000","last_active":"2023-10-09T06:36:43.548555+0000","last_peered":"2023-10-09T06:36:43.548555+0000","last_clean":"2023-10-09T06:36:43.548555+0000","last_became_active":"2023-10-09T06:36:43.542153+0000","last_became_peered":"2023-10-09T06:36:43.542153+0000","last_unstale":"2023-10-09T06:36:43.548555+0000","last_undegraded":"2023-10-09T06:36:43.548555+0000","last_fullsized":"2023-10-09T06:36:43.548555+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:31.566654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.550972+0000","last_change":"2023-10-09T06:36:43.547018+0000","last_active":"2023-10-09T06:36:43.550972+0000","last_peered":"2023-10-09T06:36:43.550972+0000","last_clean":"2023-10-09T06:36:43.550972+0000","last_became_active":"2023-10-09T06:36:43.546756+0000","last_became_peered":"2023-10-09T06:36:43.546756+0000","last_unstale":"2023-10-09T06:36:43.550972+0000","last_undegraded":"2023-10-09T06:36:43.550972+0000","last_fullsized":"2023-10-09T06:36:43.550972+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:58:42.420936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.545102+0000","last_change":"2023-10-09T06:36:43.544025+0000","last_active":"2023-10-09T06:36:43.545102+0000","last_peered":"2023-10-09T06:36:43.545102+0000","last_clean":"2023-10-09T06:36:43.545102+0000","last_became_active":"2023-10-09T06:36:43.543639+0000","last_became_peered":"2023-10-09T06:36:43.543639+0000","last_unstale":"2023-10-09T06:36:43.545102+0000","last_undegraded":"2023-10-09T06:36:43.545102+0000","last_fullsized":"2023-10-09T06:36:43.545102+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:14:04.479133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"100'4","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.007009+0000","last_change":"2023-10-09T06:36:43.010912+0000","last_active":"2023-10-09T06:36:44.007009+0000","last_peered":"2023-10-09T06:36:44.007009+0000","last_clean":"2023-10-09T06:36:44.007009+0000","last_became_active":"2023-10-09T06:36:43.010662+0000","last_became_peered":"2023-10-09T06:36:43.010662+0000","last_unstale":"2023-10-09T06:36:44.007009+0000","last_undegraded":"2023-10-09T06:36:44.007009+0000","last_fullsized":"2023-10-09T06:36:44.007009+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:34:35.588047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.549903+0000","last_change":"2023-10-09T06:36:43.542881+0000","last_active":"2023-10-09T06:36:43.549903+0000","last_peered":"2023-10-09T06:36:43.549903+0000","last_clean":"2023-10-09T06:36:43.549903+0000","last_became_active":"2023-10-09T06:36:43.542622+0000","last_became_peered":"2023-10-09T06:36:43.542622+0000","last_unstale":"2023-10-09T06:36:43.549903+0000","last_undegraded":"2023-10-09T06:36:43.549903+0000","last_fullsized":"2023-10-09T06:36:43.549903+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:48:16.923502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.006857+0000","last_change":"2023-10-09T06:36:43.011181+0000","last_active":"2023-10-09T06:36:44.006857+0000","last_peered":"2023-10-09T06:36:44.006857+0000","last_clean":"2023-10-09T06:36:44.006857+0000","last_became_active":"2023-10-09T06:36:43.010916+0000","last_became_peered":"2023-10-09T06:36:43.010916+0000","last_unstale":"2023-10-09T06:36:44.006857+0000","last_undegraded":"2023-10-09T06:36:44.006857+0000","last_fullsized":"2023-10-09T06:36:44.006857+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:10:01.213584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551016+0000","last_change":"2023-10-09T06:36:43.547871+0000","last_active":"2023-10-09T06:36:43.551016+0000","last_peered":"2023-10-09T06:36:43.551016+0000","last_clean":"2023-10-09T06:36:43.551016+0000","last_became_active":"2023-10-09T06:36:43.547589+0000","last_became_peered":"2023-10-09T06:36:43.547589+0000","last_unstale":"2023-10-09T06:36:43.551016+0000","last_undegraded":"2023-10-09T06:36:43.551016+0000","last_fullsized":"2023-10-09T06:36:43.551016+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:44:49.663317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.011864+0000","last_change":"2023-10-09T06:36:43.012960+0000","last_active":"2023-10-09T06:36:44.011864+0000","last_peered":"2023-10-09T06:36:44.011864+0000","last_clean":"2023-10-09T06:36:44.011864+0000","last_became_active":"2023-10-09T06:36:43.012657+0000","last_became_peered":"2023-10-09T06:36:43.012657+0000","last_unstale":"2023-10-09T06:36:44.011864+0000","last_undegraded":"2023-10-09T06:36:44.011864+0000","last_fullsized":"2023-10-09T06:36:44.011864+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:25:59.286727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.553690+0000","last_change":"2023-10-09T06:36:43.550145+0000","last_active":"2023-10-09T06:36:43.553690+0000","last_peered":"2023-10-09T06:36:43.553690+0000","last_clean":"2023-10-09T06:36:43.553690+0000","last_became_active":"2023-10-09T06:36:43.549331+0000","last_became_peered":"2023-10-09T06:36:43.549331+0000","last_unstale":"2023-10-09T06:36:43.553690+0000","last_undegraded":"2023-10-09T06:36:43.553690+0000","last_fullsized":"2023-10-09T06:36:43.553690+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:16:37.146078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.011826+0000","last_change":"2023-10-09T06:36:43.016780+0000","last_active":"2023-10-09T06:36:44.011826+0000","last_peered":"2023-10-09T06:36:44.011826+0000","last_clean":"2023-10-09T06:36:44.011826+0000","last_became_active":"2023-10-09T06:36:43.016509+0000","last_became_peered":"2023-10-09T06:36:43.016509+0000","last_unstale":"2023-10-09T06:36:44.011826+0000","last_undegraded":"2023-10-09T06:36:44.011826+0000","last_fullsized":"2023-10-09T06:36:44.011826+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:38:26.857233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551117+0000","last_change":"2023-10-09T06:36:43.547645+0000","last_active":"2023-10-09T06:36:43.551117+0000","last_peered":"2023-10-09T06:36:43.551117+0000","last_clean":"2023-10-09T06:36:43.551117+0000","last_became_active":"2023-10-09T06:36:43.547407+0000","last_became_peered":"2023-10-09T06:36:43.547407+0000","last_unstale":"2023-10-09T06:36:43.551117+0000","last_undegraded":"2023-10-09T06:36:43.551117+0000","last_fullsized":"2023-10-09T06:36:43.551117+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:21:05.012782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.008845+0000","last_change":"2023-10-09T06:36:43.016589+0000","last_active":"2023-10-09T06:36:44.008845+0000","last_peered":"2023-10-09T06:36:44.008845+0000","last_clean":"2023-10-09T06:36:44.008845+0000","last_became_active":"2023-10-09T06:36:43.016310+0000","last_became_peered":"2023-10-09T06:36:43.016310+0000","last_unstale":"2023-10-09T06:36:44.008845+0000","last_undegraded":"2023-10-09T06:36:44.008845+0000","last_fullsized":"2023-10-09T06:36:44.008845+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:29:11.660881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.553576+0000","last_change":"2023-10-09T06:36:43.546574+0000","last_active":"2023-10-09T06:36:43.553576+0000","last_peered":"2023-10-09T06:36:43.553576+0000","last_clean":"2023-10-09T06:36:43.553576+0000","last_became_active":"2023-10-09T06:36:43.543513+0000","last_became_peered":"2023-10-09T06:36:43.543513+0000","last_unstale":"2023-10-09T06:36:43.553576+0000","last_undegraded":"2023-10-09T06:36:43.553576+0000","last_fullsized":"2023-10-09T06:36:43.553576+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:44:49.663317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.007384+0000","last_change":"2023-10-09T06:36:43.015133+0000","last_active":"2023-10-09T06:36:44.007384+0000","last_peered":"2023-10-09T06:36:44.007384+0000","last_clean":"2023-10-09T06:36:44.007384+0000","last_became_active":"2023-10-09T06:36:43.006577+0000","last_became_peered":"2023-10-09T06:36:43.006577+0000","last_unstale":"2023-10-09T06:36:44.007384+0000","last_undegraded":"2023-10-09T06:36:44.007384+0000","last_fullsized":"2023-10-09T06:36:44.007384+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:39:46.378350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.548655+0000","last_change":"2023-10-09T06:36:43.544990+0000","last_active":"2023-10-09T06:36:43.548655+0000","last_peered":"2023-10-09T06:36:43.548655+0000","last_clean":"2023-10-09T06:36:43.548655+0000","last_became_active":"2023-10-09T06:36:43.544732+0000","last_became_peered":"2023-10-09T06:36:43.544732+0000","last_unstale":"2023-10-09T06:36:43.548655+0000","last_undegraded":"2023-10-09T06:36:43.548655+0000","last_fullsized":"2023-10-09T06:36:43.548655+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:22:05.869961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.534992+0000","last_change":"2023-10-09T06:36:43.012712+0000","last_active":"2023-10-09T06:36:43.534992+0000","last_peered":"2023-10-09T06:36:43.534992+0000","last_clean":"2023-10-09T06:36:43.534992+0000","last_became_active":"2023-10-09T06:36:43.012517+0000","last_became_peered":"2023-10-09T06:36:43.012517+0000","last_unstale":"2023-10-09T06:36:43.534992+0000","last_undegraded":"2023-10-09T06:36:43.534992+0000","last_fullsized":"2023-10-09T06:36:43.534992+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:47:36.422842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.549509+0000","last_change":"2023-10-09T06:36:43.546853+0000","last_active":"2023-10-09T06:36:43.549509+0000","last_peered":"2023-10-09T06:36:43.549509+0000","last_clean":"2023-10-09T06:36:43.549509+0000","last_became_active":"2023-10-09T06:36:43.546576+0000","last_became_peered":"2023-10-09T06:36:43.546576+0000","last_unstale":"2023-10-09T06:36:43.549509+0000","last_undegraded":"2023-10-09T06:36:43.549509+0000","last_fullsized":"2023-10-09T06:36:43.549509+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:44:38.197893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.536870+0000","last_change":"2023-10-09T06:36:43.014161+0000","last_active":"2023-10-09T06:36:43.536870+0000","last_peered":"2023-10-09T06:36:43.536870+0000","last_clean":"2023-10-09T06:36:43.536870+0000","last_became_active":"2023-10-09T06:36:43.013804+0000","last_became_peered":"2023-10-09T06:36:43.013804+0000","last_unstale":"2023-10-09T06:36:43.536870+0000","last_undegraded":"2023-10-09T06:36:43.536870+0000","last_fullsized":"2023-10-09T06:36:43.536870+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:10:08.891831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.549988+0000","last_change":"2023-10-09T06:36:43.548114+0000","last_active":"2023-10-09T06:36:43.549988+0000","last_peered":"2023-10-09T06:36:43.549988+0000","last_clean":"2023-10-09T06:36:43.549988+0000","last_became_active":"2023-10-09T06:36:43.547852+0000","last_became_peered":"2023-10-09T06:36:43.547852+0000","last_unstale":"2023-10-09T06:36:43.549988+0000","last_undegraded":"2023-10-09T06:36:43.549988+0000","last_fullsized":"2023-10-09T06:36:43.549988+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:54:24.596769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.530771+0000","last_change":"2023-10-09T06:36:43.013207+0000","last_active":"2023-10-09T06:36:43.530771+0000","last_peered":"2023-10-09T06:36:43.530771+0000","last_clean":"2023-10-09T06:36:43.530771+0000","last_became_active":"2023-10-09T06:36:43.012595+0000","last_became_peered":"2023-10-09T06:36:43.012595+0000","last_unstale":"2023-10-09T06:36:43.530771+0000","last_undegraded":"2023-10-09T06:36:43.530771+0000","last_fullsized":"2023-10-09T06:36:43.530771+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:07:06.655513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.553840+0000","last_change":"2023-10-09T06:36:43.549898+0000","last_active":"2023-10-09T06:36:43.553840+0000","last_peered":"2023-10-09T06:36:43.553840+0000","last_clean":"2023-10-09T06:36:43.553840+0000","last_became_active":"2023-10-09T06:36:43.549644+0000","last_became_peered":"2023-10-09T06:36:43.549644+0000","last_unstale":"2023-10-09T06:36:43.553840+0000","last_undegraded":"2023-10-09T06:36:43.553840+0000","last_fullsized":"2023-10-09T06:36:43.553840+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:36:17.467099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.536527+0000","last_change":"2023-10-09T06:36:43.004963+0000","last_active":"2023-10-09T06:36:43.536527+0000","last_peered":"2023-10-09T06:36:43.536527+0000","last_clean":"2023-10-09T06:36:43.536527+0000","last_became_active":"2023-10-09T06:36:43.004686+0000","last_became_peered":"2023-10-09T06:36:43.004686+0000","last_unstale":"2023-10-09T06:36:43.536527+0000","last_undegraded":"2023-10-09T06:36:43.536527+0000","last_fullsized":"2023-10-09T06:36:43.536527+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:55:06.133663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551031+0000","last_change":"2023-10-09T06:36:43.545220+0000","last_active":"2023-10-09T06:36:43.551031+0000","last_peered":"2023-10-09T06:36:43.551031+0000","last_clean":"2023-10-09T06:36:43.551031+0000","last_became_active":"2023-10-09T06:36:43.544967+0000","last_became_peered":"2023-10-09T06:36:43.544967+0000","last_unstale":"2023-10-09T06:36:43.551031+0000","last_undegraded":"2023-10-09T06:36:43.551031+0000","last_fullsized":"2023-10-09T06:36:43.551031+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:15:38.288431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"100'3","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.011167+0000","last_change":"2023-10-09T06:36:43.015585+0000","last_active":"2023-10-09T06:36:44.011167+0000","last_peered":"2023-10-09T06:36:44.011167+0000","last_clean":"2023-10-09T06:36:44.011167+0000","last_became_active":"2023-10-09T06:36:43.015313+0000","last_became_peered":"2023-10-09T06:36:43.015313+0000","last_unstale":"2023-10-09T06:36:44.011167+0000","last_undegraded":"2023-10-09T06:36:44.011167+0000","last_fullsized":"2023-10-09T06:36:44.011167+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:32:16.884726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.547041+0000","last_change":"2023-10-09T06:36:43.543482+0000","last_active":"2023-10-09T06:36:43.547041+0000","last_peered":"2023-10-09T06:36:43.547041+0000","last_clean":"2023-10-09T06:36:43.547041+0000","last_became_active":"2023-10-09T06:36:43.543211+0000","last_became_peered":"2023-10-09T06:36:43.543211+0000","last_unstale":"2023-10-09T06:36:43.547041+0000","last_undegraded":"2023-10-09T06:36:43.547041+0000","last_fullsized":"2023-10-09T06:36:43.547041+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:44:07.050759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"100'6","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.012879+0000","last_change":"2023-10-09T06:36:43.015604+0000","last_active":"2023-10-09T06:36:44.012879+0000","last_peered":"2023-10-09T06:36:44.012879+0000","last_clean":"2023-10-09T06:36:44.012879+0000","last_became_active":"2023-10-09T06:36:43.015319+0000","last_became_peered":"2023-10-09T06:36:43.015319+0000","last_unstale":"2023-10-09T06:36:44.012879+0000","last_undegraded":"2023-10-09T06:36:44.012879+0000","last_fullsized":"2023-10-09T06:36:44.012879+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:40:22.767109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551048+0000","last_change":"2023-10-09T06:36:43.546579+0000","last_active":"2023-10-09T06:36:43.551048+0000","last_peered":"2023-10-09T06:36:43.551048+0000","last_clean":"2023-10-09T06:36:43.551048+0000","last_became_active":"2023-10-09T06:36:43.546328+0000","last_became_peered":"2023-10-09T06:36:43.546328+0000","last_unstale":"2023-10-09T06:36:43.551048+0000","last_undegraded":"2023-10-09T06:36:43.551048+0000","last_fullsized":"2023-10-09T06:36:43.551048+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:57:06.755705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.013196+0000","last_change":"2023-10-09T06:36:43.012317+0000","last_active":"2023-10-09T06:36:44.013196+0000","last_peered":"2023-10-09T06:36:44.013196+0000","last_clean":"2023-10-09T06:36:44.013196+0000","last_became_active":"2023-10-09T06:36:43.012043+0000","last_became_peered":"2023-10-09T06:36:43.012043+0000","last_unstale":"2023-10-09T06:36:44.013196+0000","last_undegraded":"2023-10-09T06:36:44.013196+0000","last_fullsized":"2023-10-09T06:36:44.013196+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:47:40.764954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.549322+0000","last_change":"2023-10-09T06:36:43.543196+0000","last_active":"2023-10-09T06:36:43.549322+0000","last_peered":"2023-10-09T06:36:43.549322+0000","last_clean":"2023-10-09T06:36:43.549322+0000","last_became_active":"2023-10-09T06:36:43.542943+0000","last_became_peered":"2023-10-09T06:36:43.542943+0000","last_unstale":"2023-10-09T06:36:43.549322+0000","last_undegraded":"2023-10-09T06:36:43.549322+0000","last_fullsized":"2023-10-09T06:36:43.549322+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:21:17.476096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.534629+0000","last_change":"2023-10-09T06:36:43.008335+0000","last_active":"2023-10-09T06:36:43.534629+0000","last_peered":"2023-10-09T06:36:43.534629+0000","last_clean":"2023-10-09T06:36:43.534629+0000","last_became_active":"2023-10-09T06:36:43.008062+0000","last_became_peered":"2023-10-09T06:36:43.008062+0000","last_unstale":"2023-10-09T06:36:43.534629+0000","last_undegraded":"2023-10-09T06:36:43.534629+0000","last_fullsized":"2023-10-09T06:36:43.534629+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:32:01.648799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.549382+0000","last_change":"2023-10-09T06:36:43.544308+0000","last_active":"2023-10-09T06:36:43.549382+0000","last_peered":"2023-10-09T06:36:43.549382+0000","last_clean":"2023-10-09T06:36:43.549382+0000","last_became_active":"2023-10-09T06:36:43.544046+0000","last_became_peered":"2023-10-09T06:36:43.544046+0000","last_unstale":"2023-10-09T06:36:43.549382+0000","last_undegraded":"2023-10-09T06:36:43.549382+0000","last_fullsized":"2023-10-09T06:36:43.549382+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:57:59.240733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"100'8","reported_seq":18,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.013012+0000","last_change":"2023-10-09T06:36:43.011204+0000","last_active":"2023-10-09T06:36:44.013012+0000","last_peered":"2023-10-09T06:36:44.013012+0000","last_clean":"2023-10-09T06:36:44.013012+0000","last_became_active":"2023-10-09T06:36:43.010947+0000","last_became_peered":"2023-10-09T06:36:43.010947+0000","last_unstale":"2023-10-09T06:36:44.013012+0000","last_undegraded":"2023-10-09T06:36:44.013012+0000","last_fullsized":"2023-10-09T06:36:44.013012+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:39:31.971228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.552396+0000","last_change":"2023-10-09T06:36:43.546769+0000","last_active":"2023-10-09T06:36:43.552396+0000","last_peered":"2023-10-09T06:36:43.552396+0000","last_clean":"2023-10-09T06:36:43.552396+0000","last_became_active":"2023-10-09T06:36:43.546105+0000","last_became_peered":"2023-10-09T06:36:43.546105+0000","last_unstale":"2023-10-09T06:36:43.552396+0000","last_undegraded":"2023-10-09T06:36:43.552396+0000","last_fullsized":"2023-10-09T06:36:43.552396+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:50:58.489632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.007041+0000","last_change":"2023-10-09T06:36:43.014201+0000","last_active":"2023-10-09T06:36:44.007041+0000","last_peered":"2023-10-09T06:36:44.007041+0000","last_clean":"2023-10-09T06:36:44.007041+0000","last_became_active":"2023-10-09T06:36:43.013942+0000","last_became_peered":"2023-10-09T06:36:43.013942+0000","last_unstale":"2023-10-09T06:36:44.007041+0000","last_undegraded":"2023-10-09T06:36:44.007041+0000","last_fullsized":"2023-10-09T06:36:44.007041+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:06:42.773417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.552503+0000","last_change":"2023-10-09T06:36:43.549901+0000","last_active":"2023-10-09T06:36:43.552503+0000","last_peered":"2023-10-09T06:36:43.552503+0000","last_clean":"2023-10-09T06:36:43.552503+0000","last_became_active":"2023-10-09T06:36:43.549624+0000","last_became_peered":"2023-10-09T06:36:43.549624+0000","last_unstale":"2023-10-09T06:36:43.552503+0000","last_undegraded":"2023-10-09T06:36:43.552503+0000","last_fullsized":"2023-10-09T06:36:43.552503+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:32:17.890567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.010339+0000","last_change":"2023-10-09T06:36:43.008781+0000","last_active":"2023-10-09T06:36:44.010339+0000","last_peered":"2023-10-09T06:36:44.010339+0000","last_clean":"2023-10-09T06:36:44.010339+0000","last_became_active":"2023-10-09T06:36:43.008519+0000","last_became_peered":"2023-10-09T06:36:43.008519+0000","last_unstale":"2023-10-09T06:36:44.010339+0000","last_undegraded":"2023-10-09T06:36:44.010339+0000","last_fullsized":"2023-10-09T06:36:44.010339+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:19:11.009665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.553638+0000","last_change":"2023-10-09T06:36:43.550221+0000","last_active":"2023-10-09T06:36:43.553638+0000","last_peered":"2023-10-09T06:36:43.553638+0000","last_clean":"2023-10-09T06:36:43.553638+0000","last_became_active":"2023-10-09T06:36:43.550031+0000","last_became_peered":"2023-10-09T06:36:43.550031+0000","last_unstale":"2023-10-09T06:36:43.553638+0000","last_undegraded":"2023-10-09T06:36:43.553638+0000","last_fullsized":"2023-10-09T06:36:43.553638+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:15:38.288431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.009280+0000","last_change":"2023-10-09T06:36:43.007005+0000","last_active":"2023-10-09T06:36:44.009280+0000","last_peered":"2023-10-09T06:36:44.009280+0000","last_clean":"2023-10-09T06:36:44.009280+0000","last_became_active":"2023-10-09T06:36:43.006699+0000","last_became_peered":"2023-10-09T06:36:43.006699+0000","last_unstale":"2023-10-09T06:36:44.009280+0000","last_undegraded":"2023-10-09T06:36:44.009280+0000","last_fullsized":"2023-10-09T06:36:44.009280+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:11:56.498791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.553707+0000","last_change":"2023-10-09T06:36:43.550314+0000","last_active":"2023-10-09T06:36:43.553707+0000","last_peered":"2023-10-09T06:36:43.553707+0000","last_clean":"2023-10-09T06:36:43.553707+0000","last_became_active":"2023-10-09T06:36:43.549601+0000","last_became_peered":"2023-10-09T06:36:43.549601+0000","last_unstale":"2023-10-09T06:36:43.553707+0000","last_undegraded":"2023-10-09T06:36:43.553707+0000","last_fullsized":"2023-10-09T06:36:43.553707+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:57:06.755705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.539696+0000","last_change":"2023-10-09T06:36:43.014847+0000","last_active":"2023-10-09T06:36:43.539696+0000","last_peered":"2023-10-09T06:36:43.539696+0000","last_clean":"2023-10-09T06:36:43.539696+0000","last_became_active":"2023-10-09T06:36:43.014576+0000","last_became_peered":"2023-10-09T06:36:43.014576+0000","last_unstale":"2023-10-09T06:36:43.539696+0000","last_undegraded":"2023-10-09T06:36:43.539696+0000","last_fullsized":"2023-10-09T06:36:43.539696+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:02:57.746909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.550482+0000","last_change":"2023-10-09T06:36:43.544726+0000","last_active":"2023-10-09T06:36:43.550482+0000","last_peered":"2023-10-09T06:36:43.550482+0000","last_clean":"2023-10-09T06:36:43.550482+0000","last_became_active":"2023-10-09T06:36:43.544451+0000","last_became_peered":"2023-10-09T06:36:43.544451+0000","last_unstale":"2023-10-09T06:36:43.550482+0000","last_undegraded":"2023-10-09T06:36:43.550482+0000","last_fullsized":"2023-10-09T06:36:43.550482+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:06:43.779258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.011568+0000","last_change":"2023-10-09T06:36:43.011545+0000","last_active":"2023-10-09T06:36:44.011568+0000","last_peered":"2023-10-09T06:36:44.011568+0000","last_clean":"2023-10-09T06:36:44.011568+0000","last_became_active":"2023-10-09T06:36:43.011292+0000","last_became_peered":"2023-10-09T06:36:43.011292+0000","last_unstale":"2023-10-09T06:36:44.011568+0000","last_undegraded":"2023-10-09T06:36:44.011568+0000","last_fullsized":"2023-10-09T06:36:44.011568+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:44:06.044918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.552401+0000","last_change":"2023-10-09T06:36:43.549783+0000","last_active":"2023-10-09T06:36:43.552401+0000","last_peered":"2023-10-09T06:36:43.552401+0000","last_clean":"2023-10-09T06:36:43.552401+0000","last_became_active":"2023-10-09T06:36:43.549530+0000","last_became_peered":"2023-10-09T06:36:43.549530+0000","last_unstale":"2023-10-09T06:36:43.552401+0000","last_undegraded":"2023-10-09T06:36:43.552401+0000","last_fullsized":"2023-10-09T06:36:43.552401+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:24:32.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.530691+0000","last_change":"2023-10-09T06:36:43.015711+0000","last_active":"2023-10-09T06:36:43.530691+0000","last_peered":"2023-10-09T06:36:43.530691+0000","last_clean":"2023-10-09T06:36:43.530691+0000","last_became_active":"2023-10-09T06:36:43.015465+0000","last_became_peered":"2023-10-09T06:36:43.015465+0000","last_unstale":"2023-10-09T06:36:43.530691+0000","last_undegraded":"2023-10-09T06:36:43.530691+0000","last_fullsized":"2023-10-09T06:36:43.530691+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:54:17.299087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.550508+0000","last_change":"2023-10-09T06:36:43.544228+0000","last_active":"2023-10-09T06:36:43.550508+0000","last_peered":"2023-10-09T06:36:43.550508+0000","last_clean":"2023-10-09T06:36:43.550508+0000","last_became_active":"2023-10-09T06:36:43.543961+0000","last_became_peered":"2023-10-09T06:36:43.543961+0000","last_unstale":"2023-10-09T06:36:43.550508+0000","last_undegraded":"2023-10-09T06:36:43.550508+0000","last_fullsized":"2023-10-09T06:36:43.550508+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:21:17.476096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.536674+0000","last_change":"2023-10-09T06:36:43.011541+0000","last_active":"2023-10-09T06:36:43.536674+0000","last_peered":"2023-10-09T06:36:43.536674+0000","last_clean":"2023-10-09T06:36:43.536674+0000","last_became_active":"2023-10-09T06:36:43.011240+0000","last_became_peered":"2023-10-09T06:36:43.011240+0000","last_unstale":"2023-10-09T06:36:43.536674+0000","last_undegraded":"2023-10-09T06:36:43.536674+0000","last_fullsized":"2023-10-09T06:36:43.536674+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:44:37.192051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.550881+0000","last_change":"2023-10-09T06:36:43.547666+0000","last_active":"2023-10-09T06:36:43.550881+0000","last_peered":"2023-10-09T06:36:43.550881+0000","last_clean":"2023-10-09T06:36:43.550881+0000","last_became_active":"2023-10-09T06:36:43.547420+0000","last_became_peered":"2023-10-09T06:36:43.547420+0000","last_unstale":"2023-10-09T06:36:43.550881+0000","last_undegraded":"2023-10-09T06:36:43.550881+0000","last_fullsized":"2023-10-09T06:36:43.550881+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:28:41.537704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.537079+0000","last_change":"2023-10-09T06:36:43.010741+0000","last_active":"2023-10-09T06:36:43.537079+0000","last_peered":"2023-10-09T06:36:43.537079+0000","last_clean":"2023-10-09T06:36:43.537079+0000","last_became_active":"2023-10-09T06:36:43.008763+0000","last_became_peered":"2023-10-09T06:36:43.008763+0000","last_unstale":"2023-10-09T06:36:43.537079+0000","last_undegraded":"2023-10-09T06:36:43.537079+0000","last_fullsized":"2023-10-09T06:36:43.537079+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:02:38.569423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551759+0000","last_change":"2023-10-09T06:36:43.547654+0000","last_active":"2023-10-09T06:36:43.551759+0000","last_peered":"2023-10-09T06:36:43.551759+0000","last_clean":"2023-10-09T06:36:43.551759+0000","last_became_active":"2023-10-09T06:36:43.547396+0000","last_became_peered":"2023-10-09T06:36:43.547396+0000","last_unstale":"2023-10-09T06:36:43.551759+0000","last_undegraded":"2023-10-09T06:36:43.551759+0000","last_fullsized":"2023-10-09T06:36:43.551759+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:53:59.805364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.007794+0000","last_change":"2023-10-09T06:36:43.012317+0000","last_active":"2023-10-09T06:36:44.007794+0000","last_peered":"2023-10-09T06:36:44.007794+0000","last_clean":"2023-10-09T06:36:44.007794+0000","last_became_active":"2023-10-09T06:36:43.012069+0000","last_became_peered":"2023-10-09T06:36:43.012069+0000","last_unstale":"2023-10-09T06:36:44.007794+0000","last_undegraded":"2023-10-09T06:36:44.007794+0000","last_fullsized":"2023-10-09T06:36:44.007794+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:44:48.657476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.552336+0000","last_change":"2023-10-09T06:36:43.548047+0000","last_active":"2023-10-09T06:36:43.552336+0000","last_peered":"2023-10-09T06:36:43.552336+0000","last_clean":"2023-10-09T06:36:43.552336+0000","last_became_active":"2023-10-09T06:36:43.547458+0000","last_became_peered":"2023-10-09T06:36:43.547458+0000","last_unstale":"2023-10-09T06:36:43.552336+0000","last_undegraded":"2023-10-09T06:36:43.552336+0000","last_fullsized":"2023-10-09T06:36:43.552336+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:56:09.283848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.534568+0000","last_change":"2023-10-09T06:36:43.011625+0000","last_active":"2023-10-09T06:36:43.534568+0000","last_peered":"2023-10-09T06:36:43.534568+0000","last_clean":"2023-10-09T06:36:43.534568+0000","last_became_active":"2023-10-09T06:36:43.011355+0000","last_became_peered":"2023-10-09T06:36:43.011355+0000","last_unstale":"2023-10-09T06:36:43.534568+0000","last_undegraded":"2023-10-09T06:36:43.534568+0000","last_fullsized":"2023-10-09T06:36:43.534568+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:47:36.422842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.548872+0000","last_change":"2023-10-09T06:36:43.546103+0000","last_active":"2023-10-09T06:36:43.548872+0000","last_peered":"2023-10-09T06:36:43.548872+0000","last_clean":"2023-10-09T06:36:43.548872+0000","last_became_active":"2023-10-09T06:36:43.545831+0000","last_became_peered":"2023-10-09T06:36:43.545831+0000","last_unstale":"2023-10-09T06:36:43.548872+0000","last_undegraded":"2023-10-09T06:36:43.548872+0000","last_fullsized":"2023-10-09T06:36:43.548872+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:46:12.350453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.535512+0000","last_change":"2023-10-09T06:36:43.010088+0000","last_active":"2023-10-09T06:36:43.535512+0000","last_peered":"2023-10-09T06:36:43.535512+0000","last_clean":"2023-10-09T06:36:43.535512+0000","last_became_active":"2023-10-09T06:36:43.008670+0000","last_became_peered":"2023-10-09T06:36:43.008670+0000","last_unstale":"2023-10-09T06:36:43.535512+0000","last_undegraded":"2023-10-09T06:36:43.535512+0000","last_fullsized":"2023-10-09T06:36:43.535512+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:42:33.405061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.549440+0000","last_change":"2023-10-09T06:36:43.542158+0000","last_active":"2023-10-09T06:36:43.549440+0000","last_peered":"2023-10-09T06:36:43.549440+0000","last_clean":"2023-10-09T06:36:43.549440+0000","last_became_active":"2023-10-09T06:36:43.541868+0000","last_became_peered":"2023-10-09T06:36:43.541868+0000","last_unstale":"2023-10-09T06:36:43.549440+0000","last_undegraded":"2023-10-09T06:36:43.549440+0000","last_fullsized":"2023-10-09T06:36:43.549440+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:34:01.423040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.538285+0000","last_change":"2023-10-09T06:36:43.014742+0000","last_active":"2023-10-09T06:36:43.538285+0000","last_peered":"2023-10-09T06:36:43.538285+0000","last_clean":"2023-10-09T06:36:43.538285+0000","last_became_active":"2023-10-09T06:36:43.014526+0000","last_became_peered":"2023-10-09T06:36:43.014526+0000","last_unstale":"2023-10-09T06:36:43.538285+0000","last_undegraded":"2023-10-09T06:36:43.538285+0000","last_fullsized":"2023-10-09T06:36:43.538285+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:17:21.327667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551259+0000","last_change":"2023-10-09T06:36:43.542512+0000","last_active":"2023-10-09T06:36:43.551259+0000","last_peered":"2023-10-09T06:36:43.551259+0000","last_clean":"2023-10-09T06:36:43.551259+0000","last_became_active":"2023-10-09T06:36:43.542176+0000","last_became_peered":"2023-10-09T06:36:43.542176+0000","last_unstale":"2023-10-09T06:36:43.551259+0000","last_undegraded":"2023-10-09T06:36:43.551259+0000","last_fullsized":"2023-10-09T06:36:43.551259+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:42:34.410902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.539817+0000","last_change":"2023-10-09T06:36:43.010983+0000","last_active":"2023-10-09T06:36:43.539817+0000","last_peered":"2023-10-09T06:36:43.539817+0000","last_clean":"2023-10-09T06:36:43.539817+0000","last_became_active":"2023-10-09T06:36:43.010695+0000","last_became_peered":"2023-10-09T06:36:43.010695+0000","last_unstale":"2023-10-09T06:36:43.539817+0000","last_undegraded":"2023-10-09T06:36:43.539817+0000","last_fullsized":"2023-10-09T06:36:43.539817+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:40:49.399562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.553602+0000","last_change":"2023-10-09T06:36:43.546811+0000","last_active":"2023-10-09T06:36:43.553602+0000","last_peered":"2023-10-09T06:36:43.553602+0000","last_clean":"2023-10-09T06:36:43.553602+0000","last_became_active":"2023-10-09T06:36:43.542756+0000","last_became_peered":"2023-10-09T06:36:43.542756+0000","last_unstale":"2023-10-09T06:36:43.553602+0000","last_undegraded":"2023-10-09T06:36:43.553602+0000","last_fullsized":"2023-10-09T06:36:43.553602+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:55:07.139504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.535982+0000","last_change":"2023-10-09T06:36:43.011655+0000","last_active":"2023-10-09T06:36:43.535982+0000","last_peered":"2023-10-09T06:36:43.535982+0000","last_clean":"2023-10-09T06:36:43.535982+0000","last_became_active":"2023-10-09T06:36:43.011371+0000","last_became_peered":"2023-10-09T06:36:43.011371+0000","last_unstale":"2023-10-09T06:36:43.535982+0000","last_undegraded":"2023-10-09T06:36:43.535982+0000","last_fullsized":"2023-10-09T06:36:43.535982+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:42:21.170974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551091+0000","last_change":"2023-10-09T06:36:43.547976+0000","last_active":"2023-10-09T06:36:43.551091+0000","last_peered":"2023-10-09T06:36:43.551091+0000","last_clean":"2023-10-09T06:36:43.551091+0000","last_became_active":"2023-10-09T06:36:43.547637+0000","last_became_peered":"2023-10-09T06:36:43.547637+0000","last_unstale":"2023-10-09T06:36:43.551091+0000","last_undegraded":"2023-10-09T06:36:43.551091+0000","last_fullsized":"2023-10-09T06:36:43.551091+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:28:41.537704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.530687+0000","last_change":"2023-10-09T06:36:43.013576+0000","last_active":"2023-10-09T06:36:43.530687+0000","last_peered":"2023-10-09T06:36:43.530687+0000","last_clean":"2023-10-09T06:36:43.530687+0000","last_became_active":"2023-10-09T06:36:43.013271+0000","last_became_peered":"2023-10-09T06:36:43.013271+0000","last_unstale":"2023-10-09T06:36:43.530687+0000","last_undegraded":"2023-10-09T06:36:43.530687+0000","last_fullsized":"2023-10-09T06:36:43.530687+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:07:49.082559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.548575+0000","last_change":"2023-10-09T06:36:43.543417+0000","last_active":"2023-10-09T06:36:43.548575+0000","last_peered":"2023-10-09T06:36:43.548575+0000","last_clean":"2023-10-09T06:36:43.548575+0000","last_became_active":"2023-10-09T06:36:43.542473+0000","last_became_peered":"2023-10-09T06:36:43.542473+0000","last_unstale":"2023-10-09T06:36:43.548575+0000","last_undegraded":"2023-10-09T06:36:43.548575+0000","last_fullsized":"2023-10-09T06:36:43.548575+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:29:51.734644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.536974+0000","last_change":"2023-10-09T06:36:43.009792+0000","last_active":"2023-10-09T06:36:43.536974+0000","last_peered":"2023-10-09T06:36:43.536974+0000","last_clean":"2023-10-09T06:36:43.536974+0000","last_became_active":"2023-10-09T06:36:43.009426+0000","last_became_peered":"2023-10-09T06:36:43.009426+0000","last_unstale":"2023-10-09T06:36:43.536974+0000","last_undegraded":"2023-10-09T06:36:43.536974+0000","last_fullsized":"2023-10-09T06:36:43.536974+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:08:49.168680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551546+0000","last_change":"2023-10-09T06:36:43.548684+0000","last_active":"2023-10-09T06:36:43.551546+0000","last_peered":"2023-10-09T06:36:43.551546+0000","last_clean":"2023-10-09T06:36:43.551546+0000","last_became_active":"2023-10-09T06:36:43.548431+0000","last_became_peered":"2023-10-09T06:36:43.548431+0000","last_unstale":"2023-10-09T06:36:43.551546+0000","last_undegraded":"2023-10-09T06:36:43.551546+0000","last_fullsized":"2023-10-09T06:36:43.551546+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:17:38.218770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.539680+0000","last_change":"2023-10-09T06:36:43.014864+0000","last_active":"2023-10-09T06:36:43.539680+0000","last_peered":"2023-10-09T06:36:43.539680+0000","last_clean":"2023-10-09T06:36:43.539680+0000","last_became_active":"2023-10-09T06:36:43.014566+0000","last_became_peered":"2023-10-09T06:36:43.014566+0000","last_unstale":"2023-10-09T06:36:43.539680+0000","last_undegraded":"2023-10-09T06:36:43.539680+0000","last_fullsized":"2023-10-09T06:36:43.539680+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:40:40.449461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":230,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.539193+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:36:43.539193+0000","last_peered":"2023-10-09T06:36:43.539193+0000","last_clean":"2023-10-09T06:36:43.539193+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:36:43.539193+0000","last_undegraded":"2023-10-09T06:36:43.539193+0000","last_fullsized":"2023-10-09T06:36:43.539193+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574927,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303200,"kb_used_data":1816,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068640,"statfs":{"total":96636764160,"available":96326287360,"internally_reserved":0,"allocated":1859584,"data_stored":1135496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51100000000000001,"15min":0.505},"min":{"1min":0.33400000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68000000000000005,"5min":0.753,"15min":0.753},"last":0.38300000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.50600000000000001,"15min":0.52000000000000002},"min":{"1min":0.38900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":1.2030000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33800000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.64000000000000001,"5min":0.70599999999999996,"15min":0.71999999999999997},"last":0.48799999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52800000000000002,"15min":0.51200000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72899999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52700000000000002,"15min":0.54400000000000004},"min":{"1min":0.38200000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.83099999999999996},"last":0.51500000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53100000000000003,"15min":0.55100000000000005},"min":{"1min":0.35899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.55600000000000005,"15min":0.59499999999999997},"min":{"1min":0.46600000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73599999999999999,"5min":0.82099999999999995,"15min":1.1479999999999999},"last":0.58099999999999996}]},{"osd":4,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46899999999999997,"15min":0.51900000000000002},"min":{"1min":0.30299999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.63700000000000001,"5min":0.69999999999999996,"15min":0.93999999999999995},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56000000000000005,"15min":0.60799999999999998},"min":{"1min":0.35399999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.502}]},{"osd":5,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.57199999999999995},"min":{"1min":0.38800000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.70899999999999996,"5min":0.85799999999999998,"15min":0.91100000000000003},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.58499999999999996,"15min":0.62},"min":{"1min":0.437,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.52900000000000003}]},{"osd":6,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.54600000000000004,"15min":0.59599999999999997},"min":{"1min":0.40200000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.88600000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54500000000000004,"15min":0.61499999999999999},"min":{"1min":0.39700000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.77800000000000002,"5min":0.79900000000000004,"15min":0.84599999999999997},"last":0.54300000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574926,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302616,"kb_used_data":1232,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069224,"statfs":{"total":96636764160,"available":96326885376,"internally_reserved":0,"allocated":1261568,"data_stored":544610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.502,"15min":0.503},"min":{"1min":0.36399999999999999,"5min":0.32000000000000001,"15min":0.30399999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.50800000000000001,"15min":0.51000000000000001},"min":{"1min":0.36299999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.374}]},{"osd":1,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.495,"15min":0.51300000000000001},"min":{"1min":0.35999999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52400000000000002,"15min":0.53100000000000003},"min":{"1min":0.36199999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.437}]},{"osd":2,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51500000000000001,"15min":0.51700000000000002},"min":{"1min":0.41599999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51000000000000001,"15min":0.53200000000000003},"min":{"1min":0.36099999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.628,"5min":0.69399999999999995,"15min":0.752},"last":0.39500000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53200000000000003,"15min":0.54700000000000004},"min":{"1min":0.39300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54200000000000004,"15min":0.70099999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69199999999999995,"5min":0.74399999999999999,"15min":4.7510000000000003},"last":0.504}]},{"osd":4,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46400000000000002,"15min":0.47899999999999998},"min":{"1min":0.28199999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.76400000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56599999999999995,"15min":0.58299999999999996},"min":{"1min":0.29499999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.80800000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.51500000000000001}]},{"osd":5,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.53300000000000003},"min":{"1min":0.318,"5min":0.246,"15min":0.246},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.81200000000000006},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57099999999999995,"15min":0.59699999999999998},"min":{"1min":0.378,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70799999999999996,"5min":0.78000000000000003,"15min":0.86599999999999999},"last":0.54600000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55200000000000005,"15min":0.58599999999999997},"min":{"1min":0.35199999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71299999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.41899999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.73599999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.46000000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302608,"kb_used_data":1224,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069232,"statfs":{"total":96636764160,"available":96326893568,"internally_reserved":0,"allocated":1253376,"data_stored":538376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.31,"15min":0.31},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34599999999999997,"15min":0.35899999999999999},"min":{"1min":0.26300000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.34699999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.34699999999999998,"15min":0.35299999999999998},"min":{"1min":0.24299999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38300000000000001,"15min":0.38600000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.57199999999999995,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.41499999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.36599999999999999,"15min":0.36399999999999999},"min":{"1min":0.27300000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52300000000000002,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.34999999999999998,"15min":0.55800000000000005},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":5.8529999999999998},"last":0.32700000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58099999999999996,"15min":0.57599999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77800000000000002,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.874,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.68200000000000005}]},{"osd":5,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56999999999999995,"15min":0.76100000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83099999999999996,"5min":0.83799999999999997,"15min":5.9459999999999997},"last":0.624},{"interface":"front","average":{"1min":0.63,"5min":0.60399999999999998,"15min":0.61299999999999999},"min":{"1min":0.41899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85699999999999998,"5min":0.92700000000000005,"15min":0.93999999999999995},"last":0.70199999999999996}]},{"osd":6,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.54900000000000004,"15min":0.73299999999999998},"min":{"1min":0.38,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":5.9160000000000004},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83299999999999996,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.63600000000000001}]},{"osd":7,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56799999999999995,"15min":0.58899999999999997},"min":{"1min":0.39300000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.81599999999999995,"5min":0.89500000000000002,"15min":0.97099999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56399999999999995,"15min":0.77000000000000002},"min":{"1min":0.40400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.71899999999999997,"5min":0.80900000000000005,"15min":5.9729999999999999},"last":0.66900000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574928,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303200,"kb_used_data":1816,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068640,"statfs":{"total":96636764160,"available":96326287360,"internally_reserved":0,"allocated":1859584,"data_stored":1130268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.30499999999999999,"15min":0.316},"min":{"1min":0.22600000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.49399999999999999,"5min":0.498,"15min":0.498},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.33200000000000002,"15min":0.34499999999999997},"min":{"1min":0.25,"5min":0.214,"15min":0.214},"max":{"1min":0.48599999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.25600000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.35199999999999998,"15min":0.36299999999999999},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.379,"15min":0.38800000000000001},"min":{"1min":0.29499999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.69399999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.35699999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.33400000000000002,"15min":0.39000000000000001},"min":{"1min":0.246,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.39700000000000002,"5min":0.52900000000000003,"15min":1.554},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.36599999999999999,"15min":0.40000000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.40899999999999997,"5min":0.60899999999999999,"15min":0.79000000000000004},"last":0.377}]},{"osd":4,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.55000000000000004,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70199999999999996,"5min":1.212,"15min":1.212},"last":0.504},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55300000000000005,"15min":0.73899999999999999},"min":{"1min":0.37,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71999999999999997,"5min":0.82499999999999996,"15min":5.484},"last":0.45000000000000001}]},{"osd":5,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.74299999999999999},"min":{"1min":0.39600000000000002,"5min":0.35799999999999998,"15min":0.35499999999999998},"max":{"1min":0.69299999999999995,"5min":0.84399999999999997,"15min":5.5449999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56899999999999995,"15min":0.57499999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.80900000000000005,"5min":0.98699999999999999,"15min":1.113},"last":0.46300000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58999999999999997,"15min":0.60599999999999998},"min":{"1min":0.44400000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.83999999999999997,"5min":1.0269999999999999,"15min":1.0529999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.45700000000000002,"5min":0.41799999999999998,"15min":0.41599999999999998},"max":{"1min":0.755,"5min":1.169,"15min":1.169},"last":0.47099999999999997}]},{"osd":7,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58399999999999996,"15min":0.58999999999999997},"min":{"1min":0.42899999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.76900000000000002,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.432},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56599999999999995,"15min":0.76900000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81599999999999995,"5min":0.873,"15min":5.5800000000000001},"last":0.53300000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574929,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302612,"kb_used_data":1228,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069228,"statfs":{"total":96636764160,"available":96326889472,"internally_reserved":0,"allocated":1257472,"data_stored":544540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.32200000000000001,"15min":0.35999999999999999},"min":{"1min":0.218,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.50800000000000001,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.40100000000000002,"15min":0.60999999999999999},"min":{"1min":0.216,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.53800000000000003,"5min":7.2699999999999996,"15min":7.2699999999999996},"last":0.26900000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.32800000000000001,"15min":0.34899999999999998},"min":{"1min":0.22600000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.497,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.371,"5min":0.45100000000000001,"15min":0.66600000000000004},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.57599999999999996,"5min":7.327,"15min":7.327},"last":0.29999999999999999}]},{"osd":3,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.376,"15min":0.39400000000000002},"min":{"1min":0.28299999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.52900000000000003,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.373},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.374,"15min":0.39500000000000002},"min":{"1min":0.25900000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.48399999999999999,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.318}]},{"osd":4,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53700000000000003,"15min":0.54100000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.73299999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.439},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.628,"15min":0.83299999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83199999999999996,"5min":7.2389999999999999,"15min":7.2389999999999999},"last":0.40300000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.69899999999999995,"15min":0.98999999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.747,"5min":10.890000000000001,"15min":10.890000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59099999999999997,"15min":0.60199999999999998},"min":{"1min":0.39800000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.78200000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.41499999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.64300000000000002,"15min":0.84099999999999997},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77300000000000002,"5min":7.2999999999999998,"15min":7.2999999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57199999999999995,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77000000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.63400000000000001}]},{"osd":7,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.67200000000000004,"15min":0.873},"min":{"1min":0.38600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.79900000000000004,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.61599999999999999,"15min":0.628},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79300000000000004,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.60799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574928,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302628,"kb_used_data":1244,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069212,"statfs":{"total":96636764160,"available":96326873088,"internally_reserved":0,"allocated":1273856,"data_stored":546146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.36099999999999999,"15min":0.32900000000000001},"min":{"1min":0.223,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.46200000000000002,"5min":5.8700000000000001,"15min":5.8700000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.38500000000000001,"15min":0.36599999999999999},"min":{"1min":0.23400000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.502,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.27600000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.379,"15min":0.35799999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.51800000000000002,"5min":4.0469999999999997,"15min":4.0469999999999997},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.40300000000000002,"15min":0.38800000000000001},"min":{"1min":0.28399999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.52300000000000002,"5min":4.1509999999999998,"15min":4.1509999999999998},"last":0.46600000000000003}]},{"osd":2,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.374},"min":{"1min":0.26800000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.56799999999999995,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38900000000000001,"15min":0.38500000000000001},"min":{"1min":0.26800000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.54000000000000004,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.44700000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58199999999999996,"15min":0.57099999999999995},"min":{"1min":0.40600000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.74199999999999999,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.627,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.42199999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77400000000000002,"5min":3.0710000000000002,"15min":3.0710000000000002},"last":0.66600000000000004}]},{"osd":5,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.61599999999999999,"15min":0.58799999999999997},"min":{"1min":0.45700000000000002,"5min":0.34599999999999997,"15min":0.32700000000000001},"max":{"1min":0.69899999999999995,"5min":6.2300000000000004,"15min":6.2300000000000004},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61499999999999999,"15min":0.59699999999999998},"min":{"1min":0.495,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79700000000000004,"5min":3.0219999999999998,"15min":3.0219999999999998},"last":0.53500000000000003}]},{"osd":6,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61599999999999999,"15min":0.59199999999999997},"min":{"1min":0.48199999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.78300000000000003,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68899999999999995,"15min":0.65100000000000002},"min":{"1min":0.46700000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.82499999999999996,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.73999999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64100000000000001,"15min":0.61299999999999999},"min":{"1min":0.44900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85399999999999998,"5min":4.2629999999999999,"15min":4.2629999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.65500000000000003,"15min":0.61299999999999999},"min":{"1min":0.50800000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.77400000000000002,"5min":7.3650000000000002,"15min":7.3650000000000002},"last":0.56699999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574927,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298504,"kb_used_data":1216,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073336,"statfs":{"total":96636764160,"available":96331096064,"internally_reserved":0,"allocated":1245184,"data_stored":538252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.51400000000000001,"15min":0.55400000000000005},"min":{"1min":0.28100000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.65100000000000002,"5min":0.78800000000000003,"15min":1.083},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.52600000000000002,"15min":0.51700000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.67800000000000005,"5min":0.97799999999999998,"15min":1.054},"last":0.63300000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.71399999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.50600000000000001,"15min":0.49399999999999999},"min":{"1min":0.31900000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.67100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.35399999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52100000000000002,"15min":0.54100000000000004},"min":{"1min":0.35999999999999999,"5min":0.33700000000000002,"15min":0.32900000000000001},"max":{"1min":0.69399999999999995,"5min":0.82099999999999995,"15min":1.0249999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55500000000000005,"15min":0.59299999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.68799999999999994,"5min":0.78300000000000003,"15min":0.93799999999999994},"last":0.56999999999999995}]},{"osd":3,"last update":"Mon Oct 9 06:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.52600000000000002,"15min":0.55200000000000005},"min":{"1min":0.32200000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70199999999999996,"5min":0.77500000000000002,"15min":0.98499999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56999999999999995,"15min":0.58899999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73599999999999999,"5min":0.86199999999999999,"15min":1.1319999999999999},"last":0.53100000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51300000000000001,"15min":0.55700000000000005},"min":{"1min":0.30399999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.74099999999999999,"5min":0.78000000000000003,"15min":0.998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.60999999999999999},"min":{"1min":0.377,"5min":0.36299999999999999,"15min":0.35099999999999998},"max":{"1min":0.70999999999999996,"5min":0.80000000000000004,"15min":0.97999999999999998},"last":0.47799999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53900000000000003,"15min":0.56899999999999995},"min":{"1min":0.34699999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80200000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56699999999999995,"15min":0.59999999999999998},"min":{"1min":0.38700000000000001,"5min":0.33500000000000002,"15min":0.32700000000000001},"max":{"1min":0.77900000000000003,"5min":1.01,"15min":1.01},"last":0.65300000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54000000000000004,"15min":0.57199999999999995},"min":{"1min":0.35599999999999998,"5min":0.33600000000000002,"15min":0.28299999999999997},"max":{"1min":0.753,"5min":0.85599999999999998,"15min":0.90000000000000002},"last":0.442},{"interface":"front","average":{"1min":0.62,"5min":0.59799999999999998,"15min":0.64800000000000002},"min":{"1min":0.34999999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79500000000000004,"5min":0.98999999999999999,"15min":1.175},"last":0.59999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574927,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298512,"kb_used_data":1224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073328,"statfs":{"total":96636764160,"available":96331087872,"internally_reserved":0,"allocated":1253376,"data_stored":538828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.496,"15min":0.50700000000000001},"min":{"1min":0.314,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.64100000000000001,"5min":0.71399999999999997,"15min":0.86599999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.51900000000000002,"15min":0.51400000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.44400000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.50800000000000001,"15min":0.51000000000000001},"min":{"1min":0.34899999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.61699999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.52100000000000002,"15min":0.53100000000000003},"min":{"1min":0.30599999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73899999999999999,"5min":0.754,"15min":0.81599999999999995},"last":0.46700000000000003}]},{"osd":2,"last update":"Mon Oct 9 06:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.33100000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.70699999999999996,"5min":0.73299999999999998,"15min":0.83999999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57899999999999996,"15min":0.58699999999999997},"min":{"1min":0.46500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72799999999999998,"5min":0.745,"15min":0.79900000000000004},"last":0.64100000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.41099999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.69799999999999995,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55500000000000005,"15min":0.56399999999999995},"min":{"1min":0.39800000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.71099999999999997,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.70299999999999996}]},{"osd":4,"last update":"Mon Oct 9 06:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.495,"15min":0.496},"min":{"1min":0.29999999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54200000000000004,"15min":0.54500000000000004},"min":{"1min":0.35299999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.72299999999999998,"5min":0.76300000000000001,"15min":0.80700000000000005},"last":0.71699999999999997}]},{"osd":6,"last update":"Mon Oct 9 06:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55600000000000005,"15min":0.53400000000000003},"min":{"1min":0.376,"5min":0.35699999999999998,"15min":0.32600000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.83199999999999996},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.42599999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.66800000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.39600000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76900000000000002,"5min":0.84399999999999997,"15min":0.85599999999999998},"last":0.749},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.442,"5min":0.35199999999999998,"15min":0.314},"max":{"1min":0.81399999999999995,"5min":0.84499999999999997,"15min":0.90600000000000003},"last":0.76400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:36:48.071 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:36:48.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.216+0000 7f3290dbc700 1 Processor -- start 2023-10-09T06:36:48.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.216+0000 7f3290dbc700 1 -- start start 2023-10-09T06:36:48.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.217+0000 7f3290dbc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f328c159c60 0x7f328c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:48.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.217+0000 7f3290dbc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f328c15db10 0x7f328c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:48.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.217+0000 7f3290dbc700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f328c15a160 con 0x7f328c159890 2023-10-09T06:36:48.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.217+0000 7f3290dbc700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f328c15df50 con 0x7f328c159c60 2023-10-09T06:36:48.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.217+0000 7f3290dbc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328c0d13a0 con 0x7f328c15db10 2023-10-09T06:36:48.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.217+0000 7f328a59c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f328c159890 0x7f328c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:37982/0 (socket says 172.21.15.163:37982) 2023-10-09T06:36:48.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.217+0000 7f328a59c700 1 -- 172.21.15.163:0/3102283022 learned_addr learned my addr 172.21.15.163:0/3102283022 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:48.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.217+0000 7f3289d9b700 1 --2- 172.21.15.163:0/3102283022 >> v2:172.21.15.181:3300/0 conn(0x7f328c159c60 0x7f328c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:48.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.217+0000 7f328ad9d700 1 --2- 172.21.15.163:0/3102283022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f328c15db10 0x7f328c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:48.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.217+0000 7f328959a700 1 -- 172.21.15.163:0/3102283022 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2998679526 0 0) 0x7f328c15a160 con 0x7f328c159890 2023-10-09T06:36:48.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.218+0000 7f328959a700 1 -- 172.21.15.163:0/3102283022 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3278003570 con 0x7f328c159890 2023-10-09T06:36:48.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.218+0000 7f328959a700 1 -- 172.21.15.163:0/3102283022 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 202092362 0 0) 0x7f3278003570 con 0x7f328c159890 2023-10-09T06:36:48.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.218+0000 7f328959a700 1 -- 172.21.15.163:0/3102283022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f328c15db10 msgr2=0x7f328c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:48.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.218+0000 7f328959a700 1 --2- 172.21.15.163:0/3102283022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f328c15db10 0x7f328c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:48.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.218+0000 7f328959a700 1 -- 172.21.15.163:0/3102283022 >> v2:172.21.15.181:3300/0 conn(0x7f328c159c60 msgr2=0x7f328c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:48.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.218+0000 7f328959a700 1 --2- 172.21.15.163:0/3102283022 >> v2:172.21.15.181:3300/0 conn(0x7f328c159c60 0x7f328c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:48.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.218+0000 7f328959a700 1 -- 172.21.15.163:0/3102283022 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328c15e140 con 0x7f328c159890 2023-10-09T06:36:48.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.218+0000 7f328959a700 1 -- 172.21.15.163:0/3102283022 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f327c0017f0 con 0x7f328c159890 2023-10-09T06:36:48.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.218+0000 7f328959a700 1 -- 172.21.15.163:0/3102283022 >> v1:172.21.15.163:6789/0 conn(0x7f328c159890 legacy=0x7f328c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:48.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.218+0000 7f328959a700 1 --2- 172.21.15.163:0/3102283022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f328c15db10 0x7f3278003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:48.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.218+0000 7f328959a700 1 --2- 172.21.15.163:0/3102283022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3278003fe0 0x7f3278004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:48.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.218+0000 7f328959a700 1 --2- 172.21.15.163:0/3102283022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3278004d50 0x7f32780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:48.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.218+0000 7f328959a700 1 -- 172.21.15.163:0/3102283022 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c0d13a0 con 0x7f3278003fe0 2023-10-09T06:36:48.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.218+0000 7f328959a700 1 -- 172.21.15.163:0/3102283022 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327800ba60 con 0x7f3278004d50 2023-10-09T06:36:48.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.218+0000 7f328959a700 1 -- 172.21.15.163:0/3102283022 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f327800bbd0 con 0x7f328c15db10 2023-10-09T06:36:48.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.218+0000 7f3289d9b700 1 --2- 172.21.15.163:0/3102283022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3278003fe0 0x7f3278004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:48.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.218+0000 7f328a59c700 1 --2- 172.21.15.163:0/3102283022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f328c15db10 0x7f3278003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:48.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.218+0000 7f328ad9d700 1 --2- 172.21.15.163:0/3102283022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3278004d50 0x7f32780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:48.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.218+0000 7f3289d9b700 1 -- 172.21.15.163:0/3102283022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f328c15db10 msgr2=0x7f3278003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:48.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.218+0000 7f3289d9b700 1 --2- 172.21.15.163:0/3102283022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f328c15db10 0x7f3278003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:48.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.218+0000 7f3289d9b700 1 -- 172.21.15.163:0/3102283022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3278004d50 msgr2=0x7f32780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:48.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.218+0000 7f3289d9b700 1 --2- 172.21.15.163:0/3102283022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3278004d50 0x7f32780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:48.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.218+0000 7f3289d9b700 1 -- 172.21.15.163:0/3102283022 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3278003570 con 0x7f3278003fe0 2023-10-09T06:36:48.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.218+0000 7f328ad9d700 1 --2- 172.21.15.163:0/3102283022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3278004d50 0x7f32780051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:48.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.218+0000 7f3289d9b700 1 --2- 172.21.15.163:0/3102283022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3278003fe0 0x7f3278004810 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f3274009c00 tx=0x7f32740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d175d84fd9c4de78 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:48.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.219+0000 7f328959a700 1 -- 172.21.15.163:0/3102283022 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327400e6b0 con 0x7f3278003fe0 2023-10-09T06:36:48.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.219+0000 7f328959a700 1 -- 172.21.15.163:0/3102283022 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327400e810 con 0x7f3278003fe0 2023-10-09T06:36:48.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.219+0000 7f328959a700 1 -- 172.21.15.163:0/3102283022 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327400ebd0 con 0x7f3278003fe0 2023-10-09T06:36:48.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.219+0000 7f3290dbc700 1 -- 172.21.15.163:0/3102283022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3278003fe0 msgr2=0x7f3278004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:48.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.219+0000 7f3290dbc700 1 --2- 172.21.15.163:0/3102283022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3278003fe0 0x7f3278004810 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f3274009c00 tx=0x7f32740091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:48.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.219+0000 7f328ad9d700 1 -- 172.21.15.163:0/3102283022 reap_dead start 2023-10-09T06:36:48.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.219+0000 7f3290dbc700 1 -- 172.21.15.163:0/3102283022 shutdown_connections 2023-10-09T06:36:48.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.219+0000 7f3290dbc700 1 -- 172.21.15.163:0/3102283022 >> 172.21.15.163:0/3102283022 conn(0x7f328c068e20 msgr2=0x7f328c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:48.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.219+0000 7f3290dbc700 1 -- 172.21.15.163:0/3102283022 shutdown_connections 2023-10-09T06:36:48.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.219+0000 7f3290dbc700 1 -- 172.21.15.163:0/3102283022 wait complete. 2023-10-09T06:36:48.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.220+0000 7f3290dbc700 1 Processor -- start 2023-10-09T06:36:48.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.220+0000 7f3290dbc700 1 -- start start 2023-10-09T06:36:48.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.221+0000 7f3290dbc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3278003fe0 0x7f328c0b0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:48.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.221+0000 7f3290dbc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f328c0c9f50 0x7f328c150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:48.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.221+0000 7f3290dbc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f328c0cf8e0 0x7f328c158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:48.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.221+0000 7f3290dbc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c15b4d0 con 0x7f328c0cf8e0 2023-10-09T06:36:48.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.221+0000 7f3290dbc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c151280 con 0x7f328c0c9f50 2023-10-09T06:36:48.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.221+0000 7f3290dbc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328c0c78a0 con 0x7f3278003fe0 2023-10-09T06:36:48.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.221+0000 7f328ad9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f328c0cf8e0 0x7f328c158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:48.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.221+0000 7f328ad9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f328c0cf8e0 0x7f328c158340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41514/0 (socket says 172.21.15.163:41514) 2023-10-09T06:36:48.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.221+0000 7f328ad9d700 1 -- 172.21.15.163:0/2372995998 learned_addr learned my addr 172.21.15.163:0/2372995998 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:48.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.221+0000 7f328a59c700 1 --2- 172.21.15.163:0/2372995998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3278003fe0 0x7f328c0b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:48.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.221+0000 7f3289d9b700 1 --2- 172.21.15.163:0/2372995998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f328c0c9f50 0x7f328c150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:48.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.221+0000 7f328ad9d700 1 -- 172.21.15.163:0/2372995998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3278003fe0 msgr2=0x7f328c0b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:48.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.221+0000 7f328ad9d700 1 --2- 172.21.15.163:0/2372995998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3278003fe0 0x7f328c0b0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:48.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.221+0000 7f328ad9d700 1 -- 172.21.15.163:0/2372995998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f328c0c9f50 msgr2=0x7f328c150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:48.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.221+0000 7f328ad9d700 1 --2- 172.21.15.163:0/2372995998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f328c0c9f50 0x7f328c150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:48.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.221+0000 7f328ad9d700 1 -- 172.21.15.163:0/2372995998 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328c0ca4b0 con 0x7f328c0cf8e0 2023-10-09T06:36:48.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.222+0000 7f328ad9d700 1 --2- 172.21.15.163:0/2372995998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f328c0cf8e0 0x7f328c158340 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f328c0cadc0 tx=0x7f3284008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:48.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.222+0000 7f32827fc700 1 -- 172.21.15.163:0/2372995998 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3284011ed0 con 0x7f328c0cf8e0 2023-10-09T06:36:48.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.222+0000 7f32827fc700 1 -- 172.21.15.163:0/2372995998 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3284015670 con 0x7f328c0cf8e0 2023-10-09T06:36:48.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.222+0000 7f3290dbc700 1 -- 172.21.15.163:0/2372995998 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f328c0c7af0 con 0x7f328c0cf8e0 2023-10-09T06:36:48.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.222+0000 7f32827fc700 1 -- 172.21.15.163:0/2372995998 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3284015a10 con 0x7f328c0cf8e0 2023-10-09T06:36:48.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.222+0000 7f3290dbc700 1 -- 172.21.15.163:0/2372995998 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f328c0cfd80 con 0x7f328c0cf8e0 2023-10-09T06:36:48.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.223+0000 7f3290dbc700 1 -- 172.21.15.163:0/2372995998 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f328c04c3c0 con 0x7f328c0cf8e0 2023-10-09T06:36:48.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.223+0000 7f32827fc700 1 -- 172.21.15.163:0/2372995998 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f328401e400 con 0x7f328c0cf8e0 2023-10-09T06:36:48.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.223+0000 7f32827fc700 1 --2- 172.21.15.163:0/2372995998 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f326c067290 0x7f326c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:48.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.223+0000 7f32827fc700 1 -- 172.21.15.163:0/2372995998 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3284084ca0 con 0x7f328c0cf8e0 2023-10-09T06:36:48.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.224+0000 7f328a59c700 1 --2- 172.21.15.163:0/2372995998 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f326c067290 0x7f326c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:48.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.224+0000 7f328a59c700 1 --2- 172.21.15.163:0/2372995998 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f326c067290 0x7f326c069740 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f327c0019a0 tx=0x7f327c006040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:48.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.228+0000 7f32827fc700 1 -- 172.21.15.163:0/2372995998 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3284051ac0 con 0x7f328c0cf8e0 2023-10-09T06:36:48.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.407+0000 7f3290dbc700 1 -- 172.21.15.163:0/2372995998 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f328c06a5b0 con 0x7f326c067290 2023-10-09T06:36:48.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.416+0000 7f32827fc700 1 -- 172.21.15.163:0/2372995998 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289907 (secure 0 0 0) 0x7f328c06a5b0 con 0x7f326c067290 2023-10-09T06:36:48.420 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:48.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.419+0000 7f3290dbc700 1 -- 172.21.15.163:0/2372995998 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f326c067290 msgr2=0x7f326c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:48.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.419+0000 7f3290dbc700 1 --2- 172.21.15.163:0/2372995998 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f326c067290 0x7f326c069740 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f327c0019a0 tx=0x7f327c006040 comp rx=0 tx=0).stop 2023-10-09T06:36:48.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.419+0000 7f3290dbc700 1 -- 172.21.15.163:0/2372995998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f328c0cf8e0 msgr2=0x7f328c158340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:48.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.419+0000 7f3290dbc700 1 --2- 172.21.15.163:0/2372995998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f328c0cf8e0 0x7f328c158340 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f328c0cadc0 tx=0x7f3284008040 comp rx=0 tx=0).stop 2023-10-09T06:36:48.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.419+0000 7f3290dbc700 1 -- 172.21.15.163:0/2372995998 shutdown_connections 2023-10-09T06:36:48.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.419+0000 7f3290dbc700 1 --2- 172.21.15.163:0/2372995998 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f326c067290 0x7f326c069740 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:48.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.419+0000 7f3290dbc700 1 --2- 172.21.15.163:0/2372995998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f328c0cf8e0 0x7f328c158340 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:48.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.419+0000 7f3290dbc700 1 --2- 172.21.15.163:0/2372995998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3278003fe0 0x7f328c0b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:48.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.419+0000 7f3290dbc700 1 --2- 172.21.15.163:0/2372995998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f328c0c9f50 0x7f328c150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:48.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.419+0000 7f3290dbc700 1 -- 172.21.15.163:0/2372995998 >> 172.21.15.163:0/2372995998 conn(0x7f328c068e20 msgr2=0x7f328c152c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:48.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.419+0000 7f3290dbc700 1 -- 172.21.15.163:0/2372995998 shutdown_connections 2023-10-09T06:36:48.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.419+0000 7f3290dbc700 1 -- 172.21.15.163:0/2372995998 wait complete. 2023-10-09T06:36:48.423 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:36:48.432 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2023-10-09T06:36:48.388907+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2413880,"kb_used_data":11000,"kb_used_omap":0,"kb_used_meta":2402816,"kb_avail":752560840,"statfs":{"total":773094113280,"available":770622300160,"internally_reserved":0,"allocated":11264000,"data_stored":5516516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2460483584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001444"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551602+0000","last_change":"2023-10-09T06:36:43.547872+0000","last_active":"2023-10-09T06:36:43.551602+0000","last_peered":"2023-10-09T06:36:43.551602+0000","last_clean":"2023-10-09T06:36:43.551602+0000","last_became_active":"2023-10-09T06:36:43.547147+0000","last_became_peered":"2023-10-09T06:36:43.547147+0000","last_unstale":"2023-10-09T06:36:43.551602+0000","last_undegraded":"2023-10-09T06:36:43.551602+0000","last_fullsized":"2023-10-09T06:36:43.551602+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:04:17.924536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.553611+0000","last_change":"2023-10-09T06:36:43.546856+0000","last_active":"2023-10-09T06:36:43.553611+0000","last_peered":"2023-10-09T06:36:43.553611+0000","last_clean":"2023-10-09T06:36:43.553611+0000","last_became_active":"2023-10-09T06:36:43.542794+0000","last_became_peered":"2023-10-09T06:36:43.542794+0000","last_unstale":"2023-10-09T06:36:43.553611+0000","last_undegraded":"2023-10-09T06:36:43.553611+0000","last_fullsized":"2023-10-09T06:36:43.553611+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:10:09.897672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.553501+0000","last_change":"2023-10-09T06:36:43.548848+0000","last_active":"2023-10-09T06:36:43.553501+0000","last_peered":"2023-10-09T06:36:43.553501+0000","last_clean":"2023-10-09T06:36:43.553501+0000","last_became_active":"2023-10-09T06:36:43.548602+0000","last_became_peered":"2023-10-09T06:36:43.548602+0000","last_unstale":"2023-10-09T06:36:43.553501+0000","last_undegraded":"2023-10-09T06:36:43.553501+0000","last_fullsized":"2023-10-09T06:36:43.553501+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:24:32.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.548889+0000","last_change":"2023-10-09T06:36:43.547192+0000","last_active":"2023-10-09T06:36:43.548889+0000","last_peered":"2023-10-09T06:36:43.548889+0000","last_clean":"2023-10-09T06:36:43.548889+0000","last_became_active":"2023-10-09T06:36:43.546933+0000","last_became_peered":"2023-10-09T06:36:43.546933+0000","last_unstale":"2023-10-09T06:36:43.548889+0000","last_undegraded":"2023-10-09T06:36:43.548889+0000","last_fullsized":"2023-10-09T06:36:43.548889+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:22:42.375347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.549076+0000","last_change":"2023-10-09T06:36:43.543257+0000","last_active":"2023-10-09T06:36:43.549076+0000","last_peered":"2023-10-09T06:36:43.549076+0000","last_clean":"2023-10-09T06:36:43.549076+0000","last_became_active":"2023-10-09T06:36:43.543002+0000","last_became_peered":"2023-10-09T06:36:43.543002+0000","last_unstale":"2023-10-09T06:36:43.549076+0000","last_undegraded":"2023-10-09T06:36:43.549076+0000","last_fullsized":"2023-10-09T06:36:43.549076+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:36:18.197746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551041+0000","last_change":"2023-10-09T06:36:43.548543+0000","last_active":"2023-10-09T06:36:43.551041+0000","last_peered":"2023-10-09T06:36:43.551041+0000","last_clean":"2023-10-09T06:36:43.551041+0000","last_became_active":"2023-10-09T06:36:43.548294+0000","last_became_peered":"2023-10-09T06:36:43.548294+0000","last_unstale":"2023-10-09T06:36:43.551041+0000","last_undegraded":"2023-10-09T06:36:43.551041+0000","last_fullsized":"2023-10-09T06:36:43.551041+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:10:02.219425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.549526+0000","last_change":"2023-10-09T06:36:43.546625+0000","last_active":"2023-10-09T06:36:43.549526+0000","last_peered":"2023-10-09T06:36:43.549526+0000","last_clean":"2023-10-09T06:36:43.549526+0000","last_became_active":"2023-10-09T06:36:43.546234+0000","last_became_peered":"2023-10-09T06:36:43.546234+0000","last_unstale":"2023-10-09T06:36:43.549526+0000","last_undegraded":"2023-10-09T06:36:43.549526+0000","last_fullsized":"2023-10-09T06:36:43.549526+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:47:41.770796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551069+0000","last_change":"2023-10-09T06:36:43.546096+0000","last_active":"2023-10-09T06:36:43.551069+0000","last_peered":"2023-10-09T06:36:43.551069+0000","last_clean":"2023-10-09T06:36:43.551069+0000","last_became_active":"2023-10-09T06:36:43.545812+0000","last_became_peered":"2023-10-09T06:36:43.545812+0000","last_unstale":"2023-10-09T06:36:43.551069+0000","last_undegraded":"2023-10-09T06:36:43.551069+0000","last_fullsized":"2023-10-09T06:36:43.551069+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:17:38.218770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551589+0000","last_change":"2023-10-09T06:36:43.547713+0000","last_active":"2023-10-09T06:36:43.551589+0000","last_peered":"2023-10-09T06:36:43.551589+0000","last_clean":"2023-10-09T06:36:43.551589+0000","last_became_active":"2023-10-09T06:36:43.546800+0000","last_became_peered":"2023-10-09T06:36:43.546800+0000","last_unstale":"2023-10-09T06:36:43.551589+0000","last_undegraded":"2023-10-09T06:36:43.551589+0000","last_fullsized":"2023-10-09T06:36:43.551589+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:47:23.297691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.548545+0000","last_change":"2023-10-09T06:36:43.545995+0000","last_active":"2023-10-09T06:36:43.548545+0000","last_peered":"2023-10-09T06:36:43.548545+0000","last_clean":"2023-10-09T06:36:43.548545+0000","last_became_active":"2023-10-09T06:36:43.545730+0000","last_became_peered":"2023-10-09T06:36:43.545730+0000","last_unstale":"2023-10-09T06:36:43.548545+0000","last_undegraded":"2023-10-09T06:36:43.548545+0000","last_fullsized":"2023-10-09T06:36:43.548545+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:42:58.199107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551610+0000","last_change":"2023-10-09T06:36:43.546256+0000","last_active":"2023-10-09T06:36:43.551610+0000","last_peered":"2023-10-09T06:36:43.551610+0000","last_clean":"2023-10-09T06:36:43.551610+0000","last_became_active":"2023-10-09T06:36:43.545944+0000","last_became_peered":"2023-10-09T06:36:43.545944+0000","last_unstale":"2023-10-09T06:36:43.551610+0000","last_undegraded":"2023-10-09T06:36:43.551610+0000","last_fullsized":"2023-10-09T06:36:43.551610+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:53:59.805364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.548854+0000","last_change":"2023-10-09T06:36:43.545645+0000","last_active":"2023-10-09T06:36:43.548854+0000","last_peered":"2023-10-09T06:36:43.548854+0000","last_clean":"2023-10-09T06:36:43.548854+0000","last_became_active":"2023-10-09T06:36:43.545377+0000","last_became_peered":"2023-10-09T06:36:43.545377+0000","last_unstale":"2023-10-09T06:36:43.548854+0000","last_undegraded":"2023-10-09T06:36:43.548854+0000","last_fullsized":"2023-10-09T06:36:43.548854+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:21:45.060458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.550933+0000","last_change":"2023-10-09T06:36:43.546740+0000","last_active":"2023-10-09T06:36:43.550933+0000","last_peered":"2023-10-09T06:36:43.550933+0000","last_clean":"2023-10-09T06:36:43.550933+0000","last_became_active":"2023-10-09T06:36:43.546354+0000","last_became_peered":"2023-10-09T06:36:43.546354+0000","last_unstale":"2023-10-09T06:36:43.550933+0000","last_undegraded":"2023-10-09T06:36:43.550933+0000","last_fullsized":"2023-10-09T06:36:43.550933+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:57:59.240733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.550833+0000","last_change":"2023-10-09T06:36:43.548122+0000","last_active":"2023-10-09T06:36:43.550833+0000","last_peered":"2023-10-09T06:36:43.550833+0000","last_clean":"2023-10-09T06:36:43.550833+0000","last_became_active":"2023-10-09T06:36:43.547844+0000","last_became_peered":"2023-10-09T06:36:43.547844+0000","last_unstale":"2023-10-09T06:36:43.550833+0000","last_undegraded":"2023-10-09T06:36:43.550833+0000","last_fullsized":"2023-10-09T06:36:43.550833+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:46:07.974080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551067+0000","last_change":"2023-10-09T06:36:43.546967+0000","last_active":"2023-10-09T06:36:43.551067+0000","last_peered":"2023-10-09T06:36:43.551067+0000","last_clean":"2023-10-09T06:36:43.551067+0000","last_became_active":"2023-10-09T06:36:43.546719+0000","last_became_peered":"2023-10-09T06:36:43.546719+0000","last_unstale":"2023-10-09T06:36:43.551067+0000","last_undegraded":"2023-10-09T06:36:43.551067+0000","last_fullsized":"2023-10-09T06:36:43.551067+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:35:43.890085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.552451+0000","last_change":"2023-10-09T06:36:43.550265+0000","last_active":"2023-10-09T06:36:43.552451+0000","last_peered":"2023-10-09T06:36:43.552451+0000","last_clean":"2023-10-09T06:36:43.552451+0000","last_became_active":"2023-10-09T06:36:43.550025+0000","last_became_peered":"2023-10-09T06:36:43.550025+0000","last_unstale":"2023-10-09T06:36:43.552451+0000","last_undegraded":"2023-10-09T06:36:43.552451+0000","last_fullsized":"2023-10-09T06:36:43.552451+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:58:26.502785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551484+0000","last_change":"2023-10-09T06:36:43.544799+0000","last_active":"2023-10-09T06:36:43.551484+0000","last_peered":"2023-10-09T06:36:43.551484+0000","last_clean":"2023-10-09T06:36:43.551484+0000","last_became_active":"2023-10-09T06:36:43.544407+0000","last_became_peered":"2023-10-09T06:36:43.544407+0000","last_unstale":"2023-10-09T06:36:43.551484+0000","last_undegraded":"2023-10-09T06:36:43.551484+0000","last_fullsized":"2023-10-09T06:36:43.551484+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:17:22.333508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.549650+0000","last_change":"2023-10-09T06:36:43.544031+0000","last_active":"2023-10-09T06:36:43.549650+0000","last_peered":"2023-10-09T06:36:43.549650+0000","last_clean":"2023-10-09T06:36:43.549650+0000","last_became_active":"2023-10-09T06:36:43.541890+0000","last_became_peered":"2023-10-09T06:36:43.541890+0000","last_unstale":"2023-10-09T06:36:43.549650+0000","last_undegraded":"2023-10-09T06:36:43.549650+0000","last_fullsized":"2023-10-09T06:36:43.549650+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:58:42.420936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.553720+0000","last_change":"2023-10-09T06:36:43.550368+0000","last_active":"2023-10-09T06:36:43.553720+0000","last_peered":"2023-10-09T06:36:43.553720+0000","last_clean":"2023-10-09T06:36:43.553720+0000","last_became_active":"2023-10-09T06:36:43.549880+0000","last_became_peered":"2023-10-09T06:36:43.549880+0000","last_unstale":"2023-10-09T06:36:43.553720+0000","last_undegraded":"2023-10-09T06:36:43.553720+0000","last_fullsized":"2023-10-09T06:36:43.553720+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:02:39.575264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551222+0000","last_change":"2023-10-09T06:36:43.543664+0000","last_active":"2023-10-09T06:36:43.551222+0000","last_peered":"2023-10-09T06:36:43.551222+0000","last_clean":"2023-10-09T06:36:43.551222+0000","last_became_active":"2023-10-09T06:36:43.543258+0000","last_became_peered":"2023-10-09T06:36:43.543258+0000","last_unstale":"2023-10-09T06:36:43.551222+0000","last_undegraded":"2023-10-09T06:36:43.551222+0000","last_fullsized":"2023-10-09T06:36:43.551222+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:14:04.479133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.553681+0000","last_change":"2023-10-09T06:36:43.549119+0000","last_active":"2023-10-09T06:36:43.553681+0000","last_peered":"2023-10-09T06:36:43.553681+0000","last_clean":"2023-10-09T06:36:43.553681+0000","last_became_active":"2023-10-09T06:36:43.548854+0000","last_became_peered":"2023-10-09T06:36:43.548854+0000","last_unstale":"2023-10-09T06:36:43.553681+0000","last_undegraded":"2023-10-09T06:36:43.553681+0000","last_fullsized":"2023-10-09T06:36:43.553681+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:29:33.629874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551066+0000","last_change":"2023-10-09T06:36:43.548070+0000","last_active":"2023-10-09T06:36:43.551066+0000","last_peered":"2023-10-09T06:36:43.551066+0000","last_clean":"2023-10-09T06:36:43.551066+0000","last_became_active":"2023-10-09T06:36:43.547795+0000","last_became_peered":"2023-10-09T06:36:43.547795+0000","last_unstale":"2023-10-09T06:36:43.551066+0000","last_undegraded":"2023-10-09T06:36:43.551066+0000","last_fullsized":"2023-10-09T06:36:43.551066+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:46:07.974080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551293+0000","last_change":"2023-10-09T06:36:43.543870+0000","last_active":"2023-10-09T06:36:43.551293+0000","last_peered":"2023-10-09T06:36:43.551293+0000","last_clean":"2023-10-09T06:36:43.551293+0000","last_became_active":"2023-10-09T06:36:43.543379+0000","last_became_peered":"2023-10-09T06:36:43.543379+0000","last_unstale":"2023-10-09T06:36:43.551293+0000","last_undegraded":"2023-10-09T06:36:43.551293+0000","last_fullsized":"2023-10-09T06:36:43.551293+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:36:16.356718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551564+0000","last_change":"2023-10-09T06:36:43.548755+0000","last_active":"2023-10-09T06:36:43.551564+0000","last_peered":"2023-10-09T06:36:43.551564+0000","last_clean":"2023-10-09T06:36:43.551564+0000","last_became_active":"2023-10-09T06:36:43.548531+0000","last_became_peered":"2023-10-09T06:36:43.548531+0000","last_unstale":"2023-10-09T06:36:43.551564+0000","last_undegraded":"2023-10-09T06:36:43.551564+0000","last_fullsized":"2023-10-09T06:36:43.551564+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:16:37.146078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551462+0000","last_change":"2023-10-09T06:36:43.544787+0000","last_active":"2023-10-09T06:36:43.551462+0000","last_peered":"2023-10-09T06:36:43.551462+0000","last_clean":"2023-10-09T06:36:43.551462+0000","last_became_active":"2023-10-09T06:36:43.544105+0000","last_became_peered":"2023-10-09T06:36:43.544105+0000","last_unstale":"2023-10-09T06:36:43.551462+0000","last_undegraded":"2023-10-09T06:36:43.551462+0000","last_fullsized":"2023-10-09T06:36:43.551462+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:39:47.384191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.552392+0000","last_change":"2023-10-09T06:36:43.549176+0000","last_active":"2023-10-09T06:36:43.552392+0000","last_peered":"2023-10-09T06:36:43.552392+0000","last_clean":"2023-10-09T06:36:43.552392+0000","last_became_active":"2023-10-09T06:36:43.548903+0000","last_became_peered":"2023-10-09T06:36:43.548903+0000","last_unstale":"2023-10-09T06:36:43.552392+0000","last_undegraded":"2023-10-09T06:36:43.552392+0000","last_fullsized":"2023-10-09T06:36:43.552392+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:36:25.109886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.545078+0000","last_change":"2023-10-09T06:36:43.542089+0000","last_active":"2023-10-09T06:36:43.545078+0000","last_peered":"2023-10-09T06:36:43.545078+0000","last_clean":"2023-10-09T06:36:43.545078+0000","last_became_active":"2023-10-09T06:36:43.541796+0000","last_became_peered":"2023-10-09T06:36:43.541796+0000","last_unstale":"2023-10-09T06:36:43.545078+0000","last_undegraded":"2023-10-09T06:36:43.545078+0000","last_fullsized":"2023-10-09T06:36:43.545078+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:29:33.629874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.552882+0000","last_change":"2023-10-09T06:36:43.550593+0000","last_active":"2023-10-09T06:36:43.552882+0000","last_peered":"2023-10-09T06:36:43.552882+0000","last_clean":"2023-10-09T06:36:43.552882+0000","last_became_active":"2023-10-09T06:36:43.550313+0000","last_became_peered":"2023-10-09T06:36:43.550313+0000","last_unstale":"2023-10-09T06:36:43.552882+0000","last_undegraded":"2023-10-09T06:36:43.552882+0000","last_fullsized":"2023-10-09T06:36:43.552882+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:24:32.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.553623+0000","last_change":"2023-10-09T06:36:43.547018+0000","last_active":"2023-10-09T06:36:43.553623+0000","last_peered":"2023-10-09T06:36:43.553623+0000","last_clean":"2023-10-09T06:36:43.553623+0000","last_became_active":"2023-10-09T06:36:43.543878+0000","last_became_peered":"2023-10-09T06:36:43.543878+0000","last_unstale":"2023-10-09T06:36:43.553623+0000","last_undegraded":"2023-10-09T06:36:43.553623+0000","last_fullsized":"2023-10-09T06:36:43.553623+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:32:17.890567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.549548+0000","last_change":"2023-10-09T06:36:43.548486+0000","last_active":"2023-10-09T06:36:43.549548+0000","last_peered":"2023-10-09T06:36:43.549548+0000","last_clean":"2023-10-09T06:36:43.549548+0000","last_became_active":"2023-10-09T06:36:43.548223+0000","last_became_peered":"2023-10-09T06:36:43.548223+0000","last_unstale":"2023-10-09T06:36:43.549548+0000","last_undegraded":"2023-10-09T06:36:43.549548+0000","last_fullsized":"2023-10-09T06:36:43.549548+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:34:01.423040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.548555+0000","last_change":"2023-10-09T06:36:43.542531+0000","last_active":"2023-10-09T06:36:43.548555+0000","last_peered":"2023-10-09T06:36:43.548555+0000","last_clean":"2023-10-09T06:36:43.548555+0000","last_became_active":"2023-10-09T06:36:43.542153+0000","last_became_peered":"2023-10-09T06:36:43.542153+0000","last_unstale":"2023-10-09T06:36:43.548555+0000","last_undegraded":"2023-10-09T06:36:43.548555+0000","last_fullsized":"2023-10-09T06:36:43.548555+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:31.566654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.550972+0000","last_change":"2023-10-09T06:36:43.547018+0000","last_active":"2023-10-09T06:36:43.550972+0000","last_peered":"2023-10-09T06:36:43.550972+0000","last_clean":"2023-10-09T06:36:43.550972+0000","last_became_active":"2023-10-09T06:36:43.546756+0000","last_became_peered":"2023-10-09T06:36:43.546756+0000","last_unstale":"2023-10-09T06:36:43.550972+0000","last_undegraded":"2023-10-09T06:36:43.550972+0000","last_fullsized":"2023-10-09T06:36:43.550972+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:58:42.420936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.545102+0000","last_change":"2023-10-09T06:36:43.544025+0000","last_active":"2023-10-09T06:36:43.545102+0000","last_peered":"2023-10-09T06:36:43.545102+0000","last_clean":"2023-10-09T06:36:43.545102+0000","last_became_active":"2023-10-09T06:36:43.543639+0000","last_became_peered":"2023-10-09T06:36:43.543639+0000","last_unstale":"2023-10-09T06:36:43.545102+0000","last_undegraded":"2023-10-09T06:36:43.545102+0000","last_fullsized":"2023-10-09T06:36:43.545102+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:14:04.479133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"100'4","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.007009+0000","last_change":"2023-10-09T06:36:43.010912+0000","last_active":"2023-10-09T06:36:44.007009+0000","last_peered":"2023-10-09T06:36:44.007009+0000","last_clean":"2023-10-09T06:36:44.007009+0000","last_became_active":"2023-10-09T06:36:43.010662+0000","last_became_peered":"2023-10-09T06:36:43.010662+0000","last_unstale":"2023-10-09T06:36:44.007009+0000","last_undegraded":"2023-10-09T06:36:44.007009+0000","last_fullsized":"2023-10-09T06:36:44.007009+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:34:35.588047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.549903+0000","last_change":"2023-10-09T06:36:43.542881+0000","last_active":"2023-10-09T06:36:43.549903+0000","last_peered":"2023-10-09T06:36:43.549903+0000","last_clean":"2023-10-09T06:36:43.549903+0000","last_became_active":"2023-10-09T06:36:43.542622+0000","last_became_peered":"2023-10-09T06:36:43.542622+0000","last_unstale":"2023-10-09T06:36:43.549903+0000","last_undegraded":"2023-10-09T06:36:43.549903+0000","last_fullsized":"2023-10-09T06:36:43.549903+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:48:16.923502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.006857+0000","last_change":"2023-10-09T06:36:43.011181+0000","last_active":"2023-10-09T06:36:44.006857+0000","last_peered":"2023-10-09T06:36:44.006857+0000","last_clean":"2023-10-09T06:36:44.006857+0000","last_became_active":"2023-10-09T06:36:43.010916+0000","last_became_peered":"2023-10-09T06:36:43.010916+0000","last_unstale":"2023-10-09T06:36:44.006857+0000","last_undegraded":"2023-10-09T06:36:44.006857+0000","last_fullsized":"2023-10-09T06:36:44.006857+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:10:01.213584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551016+0000","last_change":"2023-10-09T06:36:43.547871+0000","last_active":"2023-10-09T06:36:43.551016+0000","last_peered":"2023-10-09T06:36:43.551016+0000","last_clean":"2023-10-09T06:36:43.551016+0000","last_became_active":"2023-10-09T06:36:43.547589+0000","last_became_peered":"2023-10-09T06:36:43.547589+0000","last_unstale":"2023-10-09T06:36:43.551016+0000","last_undegraded":"2023-10-09T06:36:43.551016+0000","last_fullsized":"2023-10-09T06:36:43.551016+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:44:49.663317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.011864+0000","last_change":"2023-10-09T06:36:43.012960+0000","last_active":"2023-10-09T06:36:44.011864+0000","last_peered":"2023-10-09T06:36:44.011864+0000","last_clean":"2023-10-09T06:36:44.011864+0000","last_became_active":"2023-10-09T06:36:43.012657+0000","last_became_peered":"2023-10-09T06:36:43.012657+0000","last_unstale":"2023-10-09T06:36:44.011864+0000","last_undegraded":"2023-10-09T06:36:44.011864+0000","last_fullsized":"2023-10-09T06:36:44.011864+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:25:59.286727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.553690+0000","last_change":"2023-10-09T06:36:43.550145+0000","last_active":"2023-10-09T06:36:43.553690+0000","last_peered":"2023-10-09T06:36:43.553690+0000","last_clean":"2023-10-09T06:36:43.553690+0000","last_became_active":"2023-10-09T06:36:43.549331+0000","last_became_peered":"2023-10-09T06:36:43.549331+0000","last_unstale":"2023-10-09T06:36:43.553690+0000","last_undegraded":"2023-10-09T06:36:43.553690+0000","last_fullsized":"2023-10-09T06:36:43.553690+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:16:37.146078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.011826+0000","last_change":"2023-10-09T06:36:43.016780+0000","last_active":"2023-10-09T06:36:44.011826+0000","last_peered":"2023-10-09T06:36:44.011826+0000","last_clean":"2023-10-09T06:36:44.011826+0000","last_became_active":"2023-10-09T06:36:43.016509+0000","last_became_peered":"2023-10-09T06:36:43.016509+0000","last_unstale":"2023-10-09T06:36:44.011826+0000","last_undegraded":"2023-10-09T06:36:44.011826+0000","last_fullsized":"2023-10-09T06:36:44.011826+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:38:26.857233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551117+0000","last_change":"2023-10-09T06:36:43.547645+0000","last_active":"2023-10-09T06:36:43.551117+0000","last_peered":"2023-10-09T06:36:43.551117+0000","last_clean":"2023-10-09T06:36:43.551117+0000","last_became_active":"2023-10-09T06:36:43.547407+0000","last_became_peered":"2023-10-09T06:36:43.547407+0000","last_unstale":"2023-10-09T06:36:43.551117+0000","last_undegraded":"2023-10-09T06:36:43.551117+0000","last_fullsized":"2023-10-09T06:36:43.551117+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:21:05.012782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.008845+0000","last_change":"2023-10-09T06:36:43.016589+0000","last_active":"2023-10-09T06:36:44.008845+0000","last_peered":"2023-10-09T06:36:44.008845+0000","last_clean":"2023-10-09T06:36:44.008845+0000","last_became_active":"2023-10-09T06:36:43.016310+0000","last_became_peered":"2023-10-09T06:36:43.016310+0000","last_unstale":"2023-10-09T06:36:44.008845+0000","last_undegraded":"2023-10-09T06:36:44.008845+0000","last_fullsized":"2023-10-09T06:36:44.008845+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:29:11.660881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.553576+0000","last_change":"2023-10-09T06:36:43.546574+0000","last_active":"2023-10-09T06:36:43.553576+0000","last_peered":"2023-10-09T06:36:43.553576+0000","last_clean":"2023-10-09T06:36:43.553576+0000","last_became_active":"2023-10-09T06:36:43.543513+0000","last_became_peered":"2023-10-09T06:36:43.543513+0000","last_unstale":"2023-10-09T06:36:43.553576+0000","last_undegraded":"2023-10-09T06:36:43.553576+0000","last_fullsized":"2023-10-09T06:36:43.553576+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:44:49.663317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.007384+0000","last_change":"2023-10-09T06:36:43.015133+0000","last_active":"2023-10-09T06:36:44.007384+0000","last_peered":"2023-10-09T06:36:44.007384+0000","last_clean":"2023-10-09T06:36:44.007384+0000","last_became_active":"2023-10-09T06:36:43.006577+0000","last_became_peered":"2023-10-09T06:36:43.006577+0000","last_unstale":"2023-10-09T06:36:44.007384+0000","last_undegraded":"2023-10-09T06:36:44.007384+0000","last_fullsized":"2023-10-09T06:36:44.007384+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:39:46.378350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.548655+0000","last_change":"2023-10-09T06:36:43.544990+0000","last_active":"2023-10-09T06:36:43.548655+0000","last_peered":"2023-10-09T06:36:43.548655+0000","last_clean":"2023-10-09T06:36:43.548655+0000","last_became_active":"2023-10-09T06:36:43.544732+0000","last_became_peered":"2023-10-09T06:36:43.544732+0000","last_unstale":"2023-10-09T06:36:43.548655+0000","last_undegraded":"2023-10-09T06:36:43.548655+0000","last_fullsized":"2023-10-09T06:36:43.548655+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:22:05.869961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.534992+0000","last_change":"2023-10-09T06:36:43.012712+0000","last_active":"2023-10-09T06:36:43.534992+0000","last_peered":"2023-10-09T06:36:43.534992+0000","last_clean":"2023-10-09T06:36:43.534992+0000","last_became_active":"2023-10-09T06:36:43.012517+0000","last_became_peered":"2023-10-09T06:36:43.012517+0000","last_unstale":"2023-10-09T06:36:43.534992+0000","last_undegraded":"2023-10-09T06:36:43.534992+0000","last_fullsized":"2023-10-09T06:36:43.534992+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:47:36.422842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.549509+0000","last_change":"2023-10-09T06:36:43.546853+0000","last_active":"2023-10-09T06:36:43.549509+0000","last_peered":"2023-10-09T06:36:43.549509+0000","last_clean":"2023-10-09T06:36:43.549509+0000","last_became_active":"2023-10-09T06:36:43.546576+0000","last_became_peered":"2023-10-09T06:36:43.546576+0000","last_unstale":"2023-10-09T06:36:43.549509+0000","last_undegraded":"2023-10-09T06:36:43.549509+0000","last_fullsized":"2023-10-09T06:36:43.549509+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:44:38.197893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.536870+0000","last_change":"2023-10-09T06:36:43.014161+0000","last_active":"2023-10-09T06:36:43.536870+0000","last_peered":"2023-10-09T06:36:43.536870+0000","last_clean":"2023-10-09T06:36:43.536870+0000","last_became_active":"2023-10-09T06:36:43.013804+0000","last_became_peered":"2023-10-09T06:36:43.013804+0000","last_unstale":"2023-10-09T06:36:43.536870+0000","last_undegraded":"2023-10-09T06:36:43.536870+0000","last_fullsized":"2023-10-09T06:36:43.536870+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:10:08.891831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.549988+0000","last_change":"2023-10-09T06:36:43.548114+0000","last_active":"2023-10-09T06:36:43.549988+0000","last_peered":"2023-10-09T06:36:43.549988+0000","last_clean":"2023-10-09T06:36:43.549988+0000","last_became_active":"2023-10-09T06:36:43.547852+0000","last_became_peered":"2023-10-09T06:36:43.547852+0000","last_unstale":"2023-10-09T06:36:43.549988+0000","last_undegraded":"2023-10-09T06:36:43.549988+0000","last_fullsized":"2023-10-09T06:36:43.549988+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:54:24.596769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.530771+0000","last_change":"2023-10-09T06:36:43.013207+0000","last_active":"2023-10-09T06:36:43.530771+0000","last_peered":"2023-10-09T06:36:43.530771+0000","last_clean":"2023-10-09T06:36:43.530771+0000","last_became_active":"2023-10-09T06:36:43.012595+0000","last_became_peered":"2023-10-09T06:36:43.012595+0000","last_unstale":"2023-10-09T06:36:43.530771+0000","last_undegraded":"2023-10-09T06:36:43.530771+0000","last_fullsized":"2023-10-09T06:36:43.530771+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:07:06.655513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.553840+0000","last_change":"2023-10-09T06:36:43.549898+0000","last_active":"2023-10-09T06:36:43.553840+0000","last_peered":"2023-10-09T06:36:43.553840+0000","last_clean":"2023-10-09T06:36:43.553840+0000","last_became_active":"2023-10-09T06:36:43.549644+0000","last_became_peered":"2023-10-09T06:36:43.549644+0000","last_unstale":"2023-10-09T06:36:43.553840+0000","last_undegraded":"2023-10-09T06:36:43.553840+0000","last_fullsized":"2023-10-09T06:36:43.553840+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:36:17.467099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.536527+0000","last_change":"2023-10-09T06:36:43.004963+0000","last_active":"2023-10-09T06:36:43.536527+0000","last_peered":"2023-10-09T06:36:43.536527+0000","last_clean":"2023-10-09T06:36:43.536527+0000","last_became_active":"2023-10-09T06:36:43.004686+0000","last_became_peered":"2023-10-09T06:36:43.004686+0000","last_unstale":"2023-10-09T06:36:43.536527+0000","last_undegraded":"2023-10-09T06:36:43.536527+0000","last_fullsized":"2023-10-09T06:36:43.536527+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:55:06.133663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551031+0000","last_change":"2023-10-09T06:36:43.545220+0000","last_active":"2023-10-09T06:36:43.551031+0000","last_peered":"2023-10-09T06:36:43.551031+0000","last_clean":"2023-10-09T06:36:43.551031+0000","last_became_active":"2023-10-09T06:36:43.544967+0000","last_became_peered":"2023-10-09T06:36:43.544967+0000","last_unstale":"2023-10-09T06:36:43.551031+0000","last_undegraded":"2023-10-09T06:36:43.551031+0000","last_fullsized":"2023-10-09T06:36:43.551031+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:15:38.288431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"100'3","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.011167+0000","last_change":"2023-10-09T06:36:43.015585+0000","last_active":"2023-10-09T06:36:44.011167+0000","last_peered":"2023-10-09T06:36:44.011167+0000","last_clean":"2023-10-09T06:36:44.011167+0000","last_became_active":"2023-10-09T06:36:43.015313+0000","last_became_peered":"2023-10-09T06:36:43.015313+0000","last_unstale":"2023-10-09T06:36:44.011167+0000","last_undegraded":"2023-10-09T06:36:44.011167+0000","last_fullsized":"2023-10-09T06:36:44.011167+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:32:16.884726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.547041+0000","last_change":"2023-10-09T06:36:43.543482+0000","last_active":"2023-10-09T06:36:43.547041+0000","last_peered":"2023-10-09T06:36:43.547041+0000","last_clean":"2023-10-09T06:36:43.547041+0000","last_became_active":"2023-10-09T06:36:43.543211+0000","last_became_peered":"2023-10-09T06:36:43.543211+0000","last_unstale":"2023-10-09T06:36:43.547041+0000","last_undegraded":"2023-10-09T06:36:43.547041+0000","last_fullsized":"2023-10-09T06:36:43.547041+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:44:07.050759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"100'6","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.012879+0000","last_change":"2023-10-09T06:36:43.015604+0000","last_active":"2023-10-09T06:36:44.012879+0000","last_peered":"2023-10-09T06:36:44.012879+0000","last_clean":"2023-10-09T06:36:44.012879+0000","last_became_active":"2023-10-09T06:36:43.015319+0000","last_became_peered":"2023-10-09T06:36:43.015319+0000","last_unstale":"2023-10-09T06:36:44.012879+0000","last_undegraded":"2023-10-09T06:36:44.012879+0000","last_fullsized":"2023-10-09T06:36:44.012879+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:40:22.767109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551048+0000","last_change":"2023-10-09T06:36:43.546579+0000","last_active":"2023-10-09T06:36:43.551048+0000","last_peered":"2023-10-09T06:36:43.551048+0000","last_clean":"2023-10-09T06:36:43.551048+0000","last_became_active":"2023-10-09T06:36:43.546328+0000","last_became_peered":"2023-10-09T06:36:43.546328+0000","last_unstale":"2023-10-09T06:36:43.551048+0000","last_undegraded":"2023-10-09T06:36:43.551048+0000","last_fullsized":"2023-10-09T06:36:43.551048+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:57:06.755705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.013196+0000","last_change":"2023-10-09T06:36:43.012317+0000","last_active":"2023-10-09T06:36:44.013196+0000","last_peered":"2023-10-09T06:36:44.013196+0000","last_clean":"2023-10-09T06:36:44.013196+0000","last_became_active":"2023-10-09T06:36:43.012043+0000","last_became_peered":"2023-10-09T06:36:43.012043+0000","last_unstale":"2023-10-09T06:36:44.013196+0000","last_undegraded":"2023-10-09T06:36:44.013196+0000","last_fullsized":"2023-10-09T06:36:44.013196+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:47:40.764954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.549322+0000","last_change":"2023-10-09T06:36:43.543196+0000","last_active":"2023-10-09T06:36:43.549322+0000","last_peered":"2023-10-09T06:36:43.549322+0000","last_clean":"2023-10-09T06:36:43.549322+0000","last_became_active":"2023-10-09T06:36:43.542943+0000","last_became_peered":"2023-10-09T06:36:43.542943+0000","last_unstale":"2023-10-09T06:36:43.549322+0000","last_undegraded":"2023-10-09T06:36:43.549322+0000","last_fullsized":"2023-10-09T06:36:43.549322+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:21:17.476096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.534629+0000","last_change":"2023-10-09T06:36:43.008335+0000","last_active":"2023-10-09T06:36:43.534629+0000","last_peered":"2023-10-09T06:36:43.534629+0000","last_clean":"2023-10-09T06:36:43.534629+0000","last_became_active":"2023-10-09T06:36:43.008062+0000","last_became_peered":"2023-10-09T06:36:43.008062+0000","last_unstale":"2023-10-09T06:36:43.534629+0000","last_undegraded":"2023-10-09T06:36:43.534629+0000","last_fullsized":"2023-10-09T06:36:43.534629+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:32:01.648799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.549382+0000","last_change":"2023-10-09T06:36:43.544308+0000","last_active":"2023-10-09T06:36:43.549382+0000","last_peered":"2023-10-09T06:36:43.549382+0000","last_clean":"2023-10-09T06:36:43.549382+0000","last_became_active":"2023-10-09T06:36:43.544046+0000","last_became_peered":"2023-10-09T06:36:43.544046+0000","last_unstale":"2023-10-09T06:36:43.549382+0000","last_undegraded":"2023-10-09T06:36:43.549382+0000","last_fullsized":"2023-10-09T06:36:43.549382+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:57:59.240733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"100'8","reported_seq":18,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.013012+0000","last_change":"2023-10-09T06:36:43.011204+0000","last_active":"2023-10-09T06:36:44.013012+0000","last_peered":"2023-10-09T06:36:44.013012+0000","last_clean":"2023-10-09T06:36:44.013012+0000","last_became_active":"2023-10-09T06:36:43.010947+0000","last_became_peered":"2023-10-09T06:36:43.010947+0000","last_unstale":"2023-10-09T06:36:44.013012+0000","last_undegraded":"2023-10-09T06:36:44.013012+0000","last_fullsized":"2023-10-09T06:36:44.013012+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:39:31.971228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.552396+0000","last_change":"2023-10-09T06:36:43.546769+0000","last_active":"2023-10-09T06:36:43.552396+0000","last_peered":"2023-10-09T06:36:43.552396+0000","last_clean":"2023-10-09T06:36:43.552396+0000","last_became_active":"2023-10-09T06:36:43.546105+0000","last_became_peered":"2023-10-09T06:36:43.546105+0000","last_unstale":"2023-10-09T06:36:43.552396+0000","last_undegraded":"2023-10-09T06:36:43.552396+0000","last_fullsized":"2023-10-09T06:36:43.552396+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:50:58.489632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.007041+0000","last_change":"2023-10-09T06:36:43.014201+0000","last_active":"2023-10-09T06:36:44.007041+0000","last_peered":"2023-10-09T06:36:44.007041+0000","last_clean":"2023-10-09T06:36:44.007041+0000","last_became_active":"2023-10-09T06:36:43.013942+0000","last_became_peered":"2023-10-09T06:36:43.013942+0000","last_unstale":"2023-10-09T06:36:44.007041+0000","last_undegraded":"2023-10-09T06:36:44.007041+0000","last_fullsized":"2023-10-09T06:36:44.007041+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:06:42.773417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.552503+0000","last_change":"2023-10-09T06:36:43.549901+0000","last_active":"2023-10-09T06:36:43.552503+0000","last_peered":"2023-10-09T06:36:43.552503+0000","last_clean":"2023-10-09T06:36:43.552503+0000","last_became_active":"2023-10-09T06:36:43.549624+0000","last_became_peered":"2023-10-09T06:36:43.549624+0000","last_unstale":"2023-10-09T06:36:43.552503+0000","last_undegraded":"2023-10-09T06:36:43.552503+0000","last_fullsized":"2023-10-09T06:36:43.552503+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:32:17.890567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.010339+0000","last_change":"2023-10-09T06:36:43.008781+0000","last_active":"2023-10-09T06:36:44.010339+0000","last_peered":"2023-10-09T06:36:44.010339+0000","last_clean":"2023-10-09T06:36:44.010339+0000","last_became_active":"2023-10-09T06:36:43.008519+0000","last_became_peered":"2023-10-09T06:36:43.008519+0000","last_unstale":"2023-10-09T06:36:44.010339+0000","last_undegraded":"2023-10-09T06:36:44.010339+0000","last_fullsized":"2023-10-09T06:36:44.010339+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:19:11.009665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.553638+0000","last_change":"2023-10-09T06:36:43.550221+0000","last_active":"2023-10-09T06:36:43.553638+0000","last_peered":"2023-10-09T06:36:43.553638+0000","last_clean":"2023-10-09T06:36:43.553638+0000","last_became_active":"2023-10-09T06:36:43.550031+0000","last_became_peered":"2023-10-09T06:36:43.550031+0000","last_unstale":"2023-10-09T06:36:43.553638+0000","last_undegraded":"2023-10-09T06:36:43.553638+0000","last_fullsized":"2023-10-09T06:36:43.553638+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:15:38.288431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.009280+0000","last_change":"2023-10-09T06:36:43.007005+0000","last_active":"2023-10-09T06:36:44.009280+0000","last_peered":"2023-10-09T06:36:44.009280+0000","last_clean":"2023-10-09T06:36:44.009280+0000","last_became_active":"2023-10-09T06:36:43.006699+0000","last_became_peered":"2023-10-09T06:36:43.006699+0000","last_unstale":"2023-10-09T06:36:44.009280+0000","last_undegraded":"2023-10-09T06:36:44.009280+0000","last_fullsized":"2023-10-09T06:36:44.009280+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:11:56.498791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.553707+0000","last_change":"2023-10-09T06:36:43.550314+0000","last_active":"2023-10-09T06:36:43.553707+0000","last_peered":"2023-10-09T06:36:43.553707+0000","last_clean":"2023-10-09T06:36:43.553707+0000","last_became_active":"2023-10-09T06:36:43.549601+0000","last_became_peered":"2023-10-09T06:36:43.549601+0000","last_unstale":"2023-10-09T06:36:43.553707+0000","last_undegraded":"2023-10-09T06:36:43.553707+0000","last_fullsized":"2023-10-09T06:36:43.553707+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:57:06.755705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.539696+0000","last_change":"2023-10-09T06:36:43.014847+0000","last_active":"2023-10-09T06:36:43.539696+0000","last_peered":"2023-10-09T06:36:43.539696+0000","last_clean":"2023-10-09T06:36:43.539696+0000","last_became_active":"2023-10-09T06:36:43.014576+0000","last_became_peered":"2023-10-09T06:36:43.014576+0000","last_unstale":"2023-10-09T06:36:43.539696+0000","last_undegraded":"2023-10-09T06:36:43.539696+0000","last_fullsized":"2023-10-09T06:36:43.539696+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:02:57.746909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.550482+0000","last_change":"2023-10-09T06:36:43.544726+0000","last_active":"2023-10-09T06:36:43.550482+0000","last_peered":"2023-10-09T06:36:43.550482+0000","last_clean":"2023-10-09T06:36:43.550482+0000","last_became_active":"2023-10-09T06:36:43.544451+0000","last_became_peered":"2023-10-09T06:36:43.544451+0000","last_unstale":"2023-10-09T06:36:43.550482+0000","last_undegraded":"2023-10-09T06:36:43.550482+0000","last_fullsized":"2023-10-09T06:36:43.550482+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:06:43.779258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.011568+0000","last_change":"2023-10-09T06:36:43.011545+0000","last_active":"2023-10-09T06:36:44.011568+0000","last_peered":"2023-10-09T06:36:44.011568+0000","last_clean":"2023-10-09T06:36:44.011568+0000","last_became_active":"2023-10-09T06:36:43.011292+0000","last_became_peered":"2023-10-09T06:36:43.011292+0000","last_unstale":"2023-10-09T06:36:44.011568+0000","last_undegraded":"2023-10-09T06:36:44.011568+0000","last_fullsized":"2023-10-09T06:36:44.011568+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:44:06.044918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.552401+0000","last_change":"2023-10-09T06:36:43.549783+0000","last_active":"2023-10-09T06:36:43.552401+0000","last_peered":"2023-10-09T06:36:43.552401+0000","last_clean":"2023-10-09T06:36:43.552401+0000","last_became_active":"2023-10-09T06:36:43.549530+0000","last_became_peered":"2023-10-09T06:36:43.549530+0000","last_unstale":"2023-10-09T06:36:43.552401+0000","last_undegraded":"2023-10-09T06:36:43.552401+0000","last_fullsized":"2023-10-09T06:36:43.552401+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:24:32.354820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.530691+0000","last_change":"2023-10-09T06:36:43.015711+0000","last_active":"2023-10-09T06:36:43.530691+0000","last_peered":"2023-10-09T06:36:43.530691+0000","last_clean":"2023-10-09T06:36:43.530691+0000","last_became_active":"2023-10-09T06:36:43.015465+0000","last_became_peered":"2023-10-09T06:36:43.015465+0000","last_unstale":"2023-10-09T06:36:43.530691+0000","last_undegraded":"2023-10-09T06:36:43.530691+0000","last_fullsized":"2023-10-09T06:36:43.530691+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:54:17.299087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.550508+0000","last_change":"2023-10-09T06:36:43.544228+0000","last_active":"2023-10-09T06:36:43.550508+0000","last_peered":"2023-10-09T06:36:43.550508+0000","last_clean":"2023-10-09T06:36:43.550508+0000","last_became_active":"2023-10-09T06:36:43.543961+0000","last_became_peered":"2023-10-09T06:36:43.543961+0000","last_unstale":"2023-10-09T06:36:43.550508+0000","last_undegraded":"2023-10-09T06:36:43.550508+0000","last_fullsized":"2023-10-09T06:36:43.550508+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:21:17.476096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.536674+0000","last_change":"2023-10-09T06:36:43.011541+0000","last_active":"2023-10-09T06:36:43.536674+0000","last_peered":"2023-10-09T06:36:43.536674+0000","last_clean":"2023-10-09T06:36:43.536674+0000","last_became_active":"2023-10-09T06:36:43.011240+0000","last_became_peered":"2023-10-09T06:36:43.011240+0000","last_unstale":"2023-10-09T06:36:43.536674+0000","last_undegraded":"2023-10-09T06:36:43.536674+0000","last_fullsized":"2023-10-09T06:36:43.536674+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:44:37.192051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.550881+0000","last_change":"2023-10-09T06:36:43.547666+0000","last_active":"2023-10-09T06:36:43.550881+0000","last_peered":"2023-10-09T06:36:43.550881+0000","last_clean":"2023-10-09T06:36:43.550881+0000","last_became_active":"2023-10-09T06:36:43.547420+0000","last_became_peered":"2023-10-09T06:36:43.547420+0000","last_unstale":"2023-10-09T06:36:43.550881+0000","last_undegraded":"2023-10-09T06:36:43.550881+0000","last_fullsized":"2023-10-09T06:36:43.550881+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:28:41.537704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.537079+0000","last_change":"2023-10-09T06:36:43.010741+0000","last_active":"2023-10-09T06:36:43.537079+0000","last_peered":"2023-10-09T06:36:43.537079+0000","last_clean":"2023-10-09T06:36:43.537079+0000","last_became_active":"2023-10-09T06:36:43.008763+0000","last_became_peered":"2023-10-09T06:36:43.008763+0000","last_unstale":"2023-10-09T06:36:43.537079+0000","last_undegraded":"2023-10-09T06:36:43.537079+0000","last_fullsized":"2023-10-09T06:36:43.537079+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:02:38.569423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551759+0000","last_change":"2023-10-09T06:36:43.547654+0000","last_active":"2023-10-09T06:36:43.551759+0000","last_peered":"2023-10-09T06:36:43.551759+0000","last_clean":"2023-10-09T06:36:43.551759+0000","last_became_active":"2023-10-09T06:36:43.547396+0000","last_became_peered":"2023-10-09T06:36:43.547396+0000","last_unstale":"2023-10-09T06:36:43.551759+0000","last_undegraded":"2023-10-09T06:36:43.551759+0000","last_fullsized":"2023-10-09T06:36:43.551759+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:53:59.805364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:44.007794+0000","last_change":"2023-10-09T06:36:43.012317+0000","last_active":"2023-10-09T06:36:44.007794+0000","last_peered":"2023-10-09T06:36:44.007794+0000","last_clean":"2023-10-09T06:36:44.007794+0000","last_became_active":"2023-10-09T06:36:43.012069+0000","last_became_peered":"2023-10-09T06:36:43.012069+0000","last_unstale":"2023-10-09T06:36:44.007794+0000","last_undegraded":"2023-10-09T06:36:44.007794+0000","last_fullsized":"2023-10-09T06:36:44.007794+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:44:48.657476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.552336+0000","last_change":"2023-10-09T06:36:43.548047+0000","last_active":"2023-10-09T06:36:43.552336+0000","last_peered":"2023-10-09T06:36:43.552336+0000","last_clean":"2023-10-09T06:36:43.552336+0000","last_became_active":"2023-10-09T06:36:43.547458+0000","last_became_peered":"2023-10-09T06:36:43.547458+0000","last_unstale":"2023-10-09T06:36:43.552336+0000","last_undegraded":"2023-10-09T06:36:43.552336+0000","last_fullsized":"2023-10-09T06:36:43.552336+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:56:09.283848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.534568+0000","last_change":"2023-10-09T06:36:43.011625+0000","last_active":"2023-10-09T06:36:43.534568+0000","last_peered":"2023-10-09T06:36:43.534568+0000","last_clean":"2023-10-09T06:36:43.534568+0000","last_became_active":"2023-10-09T06:36:43.011355+0000","last_became_peered":"2023-10-09T06:36:43.011355+0000","last_unstale":"2023-10-09T06:36:43.534568+0000","last_undegraded":"2023-10-09T06:36:43.534568+0000","last_fullsized":"2023-10-09T06:36:43.534568+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:47:36.422842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.548872+0000","last_change":"2023-10-09T06:36:43.546103+0000","last_active":"2023-10-09T06:36:43.548872+0000","last_peered":"2023-10-09T06:36:43.548872+0000","last_clean":"2023-10-09T06:36:43.548872+0000","last_became_active":"2023-10-09T06:36:43.545831+0000","last_became_peered":"2023-10-09T06:36:43.545831+0000","last_unstale":"2023-10-09T06:36:43.548872+0000","last_undegraded":"2023-10-09T06:36:43.548872+0000","last_fullsized":"2023-10-09T06:36:43.548872+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:46:12.350453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.535512+0000","last_change":"2023-10-09T06:36:43.010088+0000","last_active":"2023-10-09T06:36:43.535512+0000","last_peered":"2023-10-09T06:36:43.535512+0000","last_clean":"2023-10-09T06:36:43.535512+0000","last_became_active":"2023-10-09T06:36:43.008670+0000","last_became_peered":"2023-10-09T06:36:43.008670+0000","last_unstale":"2023-10-09T06:36:43.535512+0000","last_undegraded":"2023-10-09T06:36:43.535512+0000","last_fullsized":"2023-10-09T06:36:43.535512+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:42:33.405061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.549440+0000","last_change":"2023-10-09T06:36:43.542158+0000","last_active":"2023-10-09T06:36:43.549440+0000","last_peered":"2023-10-09T06:36:43.549440+0000","last_clean":"2023-10-09T06:36:43.549440+0000","last_became_active":"2023-10-09T06:36:43.541868+0000","last_became_peered":"2023-10-09T06:36:43.541868+0000","last_unstale":"2023-10-09T06:36:43.549440+0000","last_undegraded":"2023-10-09T06:36:43.549440+0000","last_fullsized":"2023-10-09T06:36:43.549440+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:34:01.423040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.538285+0000","last_change":"2023-10-09T06:36:43.014742+0000","last_active":"2023-10-09T06:36:43.538285+0000","last_peered":"2023-10-09T06:36:43.538285+0000","last_clean":"2023-10-09T06:36:43.538285+0000","last_became_active":"2023-10-09T06:36:43.014526+0000","last_became_peered":"2023-10-09T06:36:43.014526+0000","last_unstale":"2023-10-09T06:36:43.538285+0000","last_undegraded":"2023-10-09T06:36:43.538285+0000","last_fullsized":"2023-10-09T06:36:43.538285+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:17:21.327667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551259+0000","last_change":"2023-10-09T06:36:43.542512+0000","last_active":"2023-10-09T06:36:43.551259+0000","last_peered":"2023-10-09T06:36:43.551259+0000","last_clean":"2023-10-09T06:36:43.551259+0000","last_became_active":"2023-10-09T06:36:43.542176+0000","last_became_peered":"2023-10-09T06:36:43.542176+0000","last_unstale":"2023-10-09T06:36:43.551259+0000","last_undegraded":"2023-10-09T06:36:43.551259+0000","last_fullsized":"2023-10-09T06:36:43.551259+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:42:34.410902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.539817+0000","last_change":"2023-10-09T06:36:43.010983+0000","last_active":"2023-10-09T06:36:43.539817+0000","last_peered":"2023-10-09T06:36:43.539817+0000","last_clean":"2023-10-09T06:36:43.539817+0000","last_became_active":"2023-10-09T06:36:43.010695+0000","last_became_peered":"2023-10-09T06:36:43.010695+0000","last_unstale":"2023-10-09T06:36:43.539817+0000","last_undegraded":"2023-10-09T06:36:43.539817+0000","last_fullsized":"2023-10-09T06:36:43.539817+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:40:49.399562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.553602+0000","last_change":"2023-10-09T06:36:43.546811+0000","last_active":"2023-10-09T06:36:43.553602+0000","last_peered":"2023-10-09T06:36:43.553602+0000","last_clean":"2023-10-09T06:36:43.553602+0000","last_became_active":"2023-10-09T06:36:43.542756+0000","last_became_peered":"2023-10-09T06:36:43.542756+0000","last_unstale":"2023-10-09T06:36:43.553602+0000","last_undegraded":"2023-10-09T06:36:43.553602+0000","last_fullsized":"2023-10-09T06:36:43.553602+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:55:07.139504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.535982+0000","last_change":"2023-10-09T06:36:43.011655+0000","last_active":"2023-10-09T06:36:43.535982+0000","last_peered":"2023-10-09T06:36:43.535982+0000","last_clean":"2023-10-09T06:36:43.535982+0000","last_became_active":"2023-10-09T06:36:43.011371+0000","last_became_peered":"2023-10-09T06:36:43.011371+0000","last_unstale":"2023-10-09T06:36:43.535982+0000","last_undegraded":"2023-10-09T06:36:43.535982+0000","last_fullsized":"2023-10-09T06:36:43.535982+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:42:21.170974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551091+0000","last_change":"2023-10-09T06:36:43.547976+0000","last_active":"2023-10-09T06:36:43.551091+0000","last_peered":"2023-10-09T06:36:43.551091+0000","last_clean":"2023-10-09T06:36:43.551091+0000","last_became_active":"2023-10-09T06:36:43.547637+0000","last_became_peered":"2023-10-09T06:36:43.547637+0000","last_unstale":"2023-10-09T06:36:43.551091+0000","last_undegraded":"2023-10-09T06:36:43.551091+0000","last_fullsized":"2023-10-09T06:36:43.551091+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:28:41.537704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.530687+0000","last_change":"2023-10-09T06:36:43.013576+0000","last_active":"2023-10-09T06:36:43.530687+0000","last_peered":"2023-10-09T06:36:43.530687+0000","last_clean":"2023-10-09T06:36:43.530687+0000","last_became_active":"2023-10-09T06:36:43.013271+0000","last_became_peered":"2023-10-09T06:36:43.013271+0000","last_unstale":"2023-10-09T06:36:43.530687+0000","last_undegraded":"2023-10-09T06:36:43.530687+0000","last_fullsized":"2023-10-09T06:36:43.530687+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:07:49.082559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.548575+0000","last_change":"2023-10-09T06:36:43.543417+0000","last_active":"2023-10-09T06:36:43.548575+0000","last_peered":"2023-10-09T06:36:43.548575+0000","last_clean":"2023-10-09T06:36:43.548575+0000","last_became_active":"2023-10-09T06:36:43.542473+0000","last_became_peered":"2023-10-09T06:36:43.542473+0000","last_unstale":"2023-10-09T06:36:43.548575+0000","last_undegraded":"2023-10-09T06:36:43.548575+0000","last_fullsized":"2023-10-09T06:36:43.548575+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:29:51.734644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.536974+0000","last_change":"2023-10-09T06:36:43.009792+0000","last_active":"2023-10-09T06:36:43.536974+0000","last_peered":"2023-10-09T06:36:43.536974+0000","last_clean":"2023-10-09T06:36:43.536974+0000","last_became_active":"2023-10-09T06:36:43.009426+0000","last_became_peered":"2023-10-09T06:36:43.009426+0000","last_unstale":"2023-10-09T06:36:43.536974+0000","last_undegraded":"2023-10-09T06:36:43.536974+0000","last_fullsized":"2023-10-09T06:36:43.536974+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:08:49.168680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.551546+0000","last_change":"2023-10-09T06:36:43.548684+0000","last_active":"2023-10-09T06:36:43.551546+0000","last_peered":"2023-10-09T06:36:43.551546+0000","last_clean":"2023-10-09T06:36:43.551546+0000","last_became_active":"2023-10-09T06:36:43.548431+0000","last_became_peered":"2023-10-09T06:36:43.548431+0000","last_unstale":"2023-10-09T06:36:43.551546+0000","last_undegraded":"2023-10-09T06:36:43.551546+0000","last_fullsized":"2023-10-09T06:36:43.551546+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:42.989151+0000","last_clean_scrub_stamp":"2023-10-09T06:36:42.989151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:17:38.218770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.539680+0000","last_change":"2023-10-09T06:36:43.014864+0000","last_active":"2023-10-09T06:36:43.539680+0000","last_peered":"2023-10-09T06:36:43.539680+0000","last_clean":"2023-10-09T06:36:43.539680+0000","last_became_active":"2023-10-09T06:36:43.014566+0000","last_became_peered":"2023-10-09T06:36:43.014566+0000","last_unstale":"2023-10-09T06:36:43.539680+0000","last_undegraded":"2023-10-09T06:36:43.539680+0000","last_fullsized":"2023-10-09T06:36:43.539680+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:36:41.983310+0000","last_clean_scrub_stamp":"2023-10-09T06:36:41.983310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:40:40.449461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":230,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-09T06:36:43.539193+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:36:43.539193+0000","last_peered":"2023-10-09T06:36:43.539193+0000","last_clean":"2023-10-09T06:36:43.539193+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:36:43.539193+0000","last_undegraded":"2023-10-09T06:36:43.539193+0000","last_fullsized":"2023-10-09T06:36:43.539193+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574927,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303200,"kb_used_data":1816,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068640,"statfs":{"total":96636764160,"available":96326287360,"internally_reserved":0,"allocated":1859584,"data_stored":1135496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51100000000000001,"15min":0.505},"min":{"1min":0.33400000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68000000000000005,"5min":0.753,"15min":0.753},"last":0.38300000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.50600000000000001,"15min":0.52000000000000002},"min":{"1min":0.38900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":1.2030000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33800000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.64000000000000001,"5min":0.70599999999999996,"15min":0.71999999999999997},"last":0.48799999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52800000000000002,"15min":0.51200000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72899999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52700000000000002,"15min":0.54400000000000004},"min":{"1min":0.38200000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.83099999999999996},"last":0.51500000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53100000000000003,"15min":0.55100000000000005},"min":{"1min":0.35899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.55600000000000005,"15min":0.59499999999999997},"min":{"1min":0.46600000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73599999999999999,"5min":0.82099999999999995,"15min":1.1479999999999999},"last":0.58099999999999996}]},{"osd":4,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46899999999999997,"15min":0.51900000000000002},"min":{"1min":0.30299999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.63700000000000001,"5min":0.69999999999999996,"15min":0.93999999999999995},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56000000000000005,"15min":0.60799999999999998},"min":{"1min":0.35399999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.502}]},{"osd":5,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.57199999999999995},"min":{"1min":0.38800000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.70899999999999996,"5min":0.85799999999999998,"15min":0.91100000000000003},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.58499999999999996,"15min":0.62},"min":{"1min":0.437,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.52900000000000003}]},{"osd":6,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.54600000000000004,"15min":0.59599999999999997},"min":{"1min":0.40200000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.88600000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54500000000000004,"15min":0.61499999999999999},"min":{"1min":0.39700000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.77800000000000002,"5min":0.79900000000000004,"15min":0.84599999999999997},"last":0.54300000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574926,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302616,"kb_used_data":1232,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069224,"statfs":{"total":96636764160,"available":96326885376,"internally_reserved":0,"allocated":1261568,"data_stored":544610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.502,"15min":0.503},"min":{"1min":0.36399999999999999,"5min":0.32000000000000001,"15min":0.30399999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.50800000000000001,"15min":0.51000000000000001},"min":{"1min":0.36299999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.374}]},{"osd":1,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.495,"15min":0.51300000000000001},"min":{"1min":0.35999999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52400000000000002,"15min":0.53100000000000003},"min":{"1min":0.36199999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.437}]},{"osd":2,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51500000000000001,"15min":0.51700000000000002},"min":{"1min":0.41599999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51000000000000001,"15min":0.53200000000000003},"min":{"1min":0.36099999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.628,"5min":0.69399999999999995,"15min":0.752},"last":0.39500000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53200000000000003,"15min":0.54700000000000004},"min":{"1min":0.39300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54200000000000004,"15min":0.70099999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69199999999999995,"5min":0.74399999999999999,"15min":4.7510000000000003},"last":0.504}]},{"osd":4,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46400000000000002,"15min":0.47899999999999998},"min":{"1min":0.28199999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.76400000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56599999999999995,"15min":0.58299999999999996},"min":{"1min":0.29499999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.80800000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.51500000000000001}]},{"osd":5,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.53300000000000003},"min":{"1min":0.318,"5min":0.246,"15min":0.246},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.81200000000000006},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57099999999999995,"15min":0.59699999999999998},"min":{"1min":0.378,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70799999999999996,"5min":0.78000000000000003,"15min":0.86599999999999999},"last":0.54600000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55200000000000005,"15min":0.58599999999999997},"min":{"1min":0.35199999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71299999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.41899999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.73599999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.46000000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302608,"kb_used_data":1224,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069232,"statfs":{"total":96636764160,"available":96326893568,"internally_reserved":0,"allocated":1253376,"data_stored":538376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.31,"15min":0.31},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34599999999999997,"15min":0.35899999999999999},"min":{"1min":0.26300000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.34699999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.34699999999999998,"15min":0.35299999999999998},"min":{"1min":0.24299999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38300000000000001,"15min":0.38600000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.57199999999999995,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.41499999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.36599999999999999,"15min":0.36399999999999999},"min":{"1min":0.27300000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52300000000000002,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.34999999999999998,"15min":0.55800000000000005},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":5.8529999999999998},"last":0.32700000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58099999999999996,"15min":0.57599999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77800000000000002,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.874,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.68200000000000005}]},{"osd":5,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56999999999999995,"15min":0.76100000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83099999999999996,"5min":0.83799999999999997,"15min":5.9459999999999997},"last":0.624},{"interface":"front","average":{"1min":0.63,"5min":0.60399999999999998,"15min":0.61299999999999999},"min":{"1min":0.41899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85699999999999998,"5min":0.92700000000000005,"15min":0.93999999999999995},"last":0.70199999999999996}]},{"osd":6,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.54900000000000004,"15min":0.73299999999999998},"min":{"1min":0.38,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":5.9160000000000004},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83299999999999996,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.63600000000000001}]},{"osd":7,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56799999999999995,"15min":0.58899999999999997},"min":{"1min":0.39300000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.81599999999999995,"5min":0.89500000000000002,"15min":0.97099999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56399999999999995,"15min":0.77000000000000002},"min":{"1min":0.40400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.71899999999999997,"5min":0.80900000000000005,"15min":5.9729999999999999},"last":0.66900000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574928,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303200,"kb_used_data":1816,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068640,"statfs":{"total":96636764160,"available":96326287360,"internally_reserved":0,"allocated":1859584,"data_stored":1130268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.30499999999999999,"15min":0.316},"min":{"1min":0.22600000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.49399999999999999,"5min":0.498,"15min":0.498},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.33200000000000002,"15min":0.34499999999999997},"min":{"1min":0.25,"5min":0.214,"15min":0.214},"max":{"1min":0.48599999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.25600000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.35199999999999998,"15min":0.36299999999999999},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.379,"15min":0.38800000000000001},"min":{"1min":0.29499999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.69399999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.35699999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.33400000000000002,"15min":0.39000000000000001},"min":{"1min":0.246,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.39700000000000002,"5min":0.52900000000000003,"15min":1.554},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.36599999999999999,"15min":0.40000000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.40899999999999997,"5min":0.60899999999999999,"15min":0.79000000000000004},"last":0.377}]},{"osd":4,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.55000000000000004,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70199999999999996,"5min":1.212,"15min":1.212},"last":0.504},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55300000000000005,"15min":0.73899999999999999},"min":{"1min":0.37,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71999999999999997,"5min":0.82499999999999996,"15min":5.484},"last":0.45000000000000001}]},{"osd":5,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.74299999999999999},"min":{"1min":0.39600000000000002,"5min":0.35799999999999998,"15min":0.35499999999999998},"max":{"1min":0.69299999999999995,"5min":0.84399999999999997,"15min":5.5449999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56899999999999995,"15min":0.57499999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.80900000000000005,"5min":0.98699999999999999,"15min":1.113},"last":0.46300000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58999999999999997,"15min":0.60599999999999998},"min":{"1min":0.44400000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.83999999999999997,"5min":1.0269999999999999,"15min":1.0529999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.45700000000000002,"5min":0.41799999999999998,"15min":0.41599999999999998},"max":{"1min":0.755,"5min":1.169,"15min":1.169},"last":0.47099999999999997}]},{"osd":7,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58399999999999996,"15min":0.58999999999999997},"min":{"1min":0.42899999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.76900000000000002,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.432},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56599999999999995,"15min":0.76900000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81599999999999995,"5min":0.873,"15min":5.5800000000000001},"last":0.53300000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574929,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302612,"kb_used_data":1228,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069228,"statfs":{"total":96636764160,"available":96326889472,"internally_reserved":0,"allocated":1257472,"data_stored":544540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.32200000000000001,"15min":0.35999999999999999},"min":{"1min":0.218,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.50800000000000001,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.40100000000000002,"15min":0.60999999999999999},"min":{"1min":0.216,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.53800000000000003,"5min":7.2699999999999996,"15min":7.2699999999999996},"last":0.26900000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.32800000000000001,"15min":0.34899999999999998},"min":{"1min":0.22600000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.497,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.371,"5min":0.45100000000000001,"15min":0.66600000000000004},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.57599999999999996,"5min":7.327,"15min":7.327},"last":0.29999999999999999}]},{"osd":3,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.376,"15min":0.39400000000000002},"min":{"1min":0.28299999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.52900000000000003,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.373},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.374,"15min":0.39500000000000002},"min":{"1min":0.25900000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.48399999999999999,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.318}]},{"osd":4,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53700000000000003,"15min":0.54100000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.73299999999999998,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.439},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.628,"15min":0.83299999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83199999999999996,"5min":7.2389999999999999,"15min":7.2389999999999999},"last":0.40300000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.69899999999999995,"15min":0.98999999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.747,"5min":10.890000000000001,"15min":10.890000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59099999999999997,"15min":0.60199999999999998},"min":{"1min":0.39800000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.78200000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.41499999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.64300000000000002,"15min":0.84099999999999997},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77300000000000002,"5min":7.2999999999999998,"15min":7.2999999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57199999999999995,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77000000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.63400000000000001}]},{"osd":7,"last update":"Mon Oct 9 06:35:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.67200000000000004,"15min":0.873},"min":{"1min":0.38600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.79900000000000004,"5min":7.1879999999999997,"15min":7.1879999999999997},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.61599999999999999,"15min":0.628},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79300000000000004,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.60799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574928,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302628,"kb_used_data":1244,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069212,"statfs":{"total":96636764160,"available":96326873088,"internally_reserved":0,"allocated":1273856,"data_stored":546146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.36099999999999999,"15min":0.32900000000000001},"min":{"1min":0.223,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.46200000000000002,"5min":5.8700000000000001,"15min":5.8700000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.38500000000000001,"15min":0.36599999999999999},"min":{"1min":0.23400000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.502,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.27600000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.379,"15min":0.35799999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.51800000000000002,"5min":4.0469999999999997,"15min":4.0469999999999997},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.40300000000000002,"15min":0.38800000000000001},"min":{"1min":0.28399999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.52300000000000002,"5min":4.1509999999999998,"15min":4.1509999999999998},"last":0.46600000000000003}]},{"osd":2,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.374},"min":{"1min":0.26800000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.56799999999999995,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38900000000000001,"15min":0.38500000000000001},"min":{"1min":0.26800000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.54000000000000004,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.44700000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58199999999999996,"15min":0.57099999999999995},"min":{"1min":0.40600000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.74199999999999999,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.627,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.42199999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77400000000000002,"5min":3.0710000000000002,"15min":3.0710000000000002},"last":0.66600000000000004}]},{"osd":5,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.61599999999999999,"15min":0.58799999999999997},"min":{"1min":0.45700000000000002,"5min":0.34599999999999997,"15min":0.32700000000000001},"max":{"1min":0.69899999999999995,"5min":6.2300000000000004,"15min":6.2300000000000004},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61499999999999999,"15min":0.59699999999999998},"min":{"1min":0.495,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79700000000000004,"5min":3.0219999999999998,"15min":3.0219999999999998},"last":0.53500000000000003}]},{"osd":6,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61599999999999999,"15min":0.59199999999999997},"min":{"1min":0.48199999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.78300000000000003,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68899999999999995,"15min":0.65100000000000002},"min":{"1min":0.46700000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.82499999999999996,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.73999999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64100000000000001,"15min":0.61299999999999999},"min":{"1min":0.44900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85399999999999998,"5min":4.2629999999999999,"15min":4.2629999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.65500000000000003,"15min":0.61299999999999999},"min":{"1min":0.50800000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.77400000000000002,"5min":7.3650000000000002,"15min":7.3650000000000002},"last":0.56699999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574927,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298504,"kb_used_data":1216,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073336,"statfs":{"total":96636764160,"available":96331096064,"internally_reserved":0,"allocated":1245184,"data_stored":538252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.51400000000000001,"15min":0.55400000000000005},"min":{"1min":0.28100000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.65100000000000002,"5min":0.78800000000000003,"15min":1.083},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.52600000000000002,"15min":0.51700000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.67800000000000005,"5min":0.97799999999999998,"15min":1.054},"last":0.63300000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.71399999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.50600000000000001,"15min":0.49399999999999999},"min":{"1min":0.31900000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.67100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.35399999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52100000000000002,"15min":0.54100000000000004},"min":{"1min":0.35999999999999999,"5min":0.33700000000000002,"15min":0.32900000000000001},"max":{"1min":0.69399999999999995,"5min":0.82099999999999995,"15min":1.0249999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55500000000000005,"15min":0.59299999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.68799999999999994,"5min":0.78300000000000003,"15min":0.93799999999999994},"last":0.56999999999999995}]},{"osd":3,"last update":"Mon Oct 9 06:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.52600000000000002,"15min":0.55200000000000005},"min":{"1min":0.32200000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70199999999999996,"5min":0.77500000000000002,"15min":0.98499999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56999999999999995,"15min":0.58899999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73599999999999999,"5min":0.86199999999999999,"15min":1.1319999999999999},"last":0.53100000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51300000000000001,"15min":0.55700000000000005},"min":{"1min":0.30399999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.74099999999999999,"5min":0.78000000000000003,"15min":0.998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.60999999999999999},"min":{"1min":0.377,"5min":0.36299999999999999,"15min":0.35099999999999998},"max":{"1min":0.70999999999999996,"5min":0.80000000000000004,"15min":0.97999999999999998},"last":0.47799999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53900000000000003,"15min":0.56899999999999995},"min":{"1min":0.34699999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80200000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56699999999999995,"15min":0.59999999999999998},"min":{"1min":0.38700000000000001,"5min":0.33500000000000002,"15min":0.32700000000000001},"max":{"1min":0.77900000000000003,"5min":1.01,"15min":1.01},"last":0.65300000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54000000000000004,"15min":0.57199999999999995},"min":{"1min":0.35599999999999998,"5min":0.33600000000000002,"15min":0.28299999999999997},"max":{"1min":0.753,"5min":0.85599999999999998,"15min":0.90000000000000002},"last":0.442},{"interface":"front","average":{"1min":0.62,"5min":0.59799999999999998,"15min":0.64800000000000002},"min":{"1min":0.34999999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79500000000000004,"5min":0.98999999999999999,"15min":1.175},"last":0.59999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574927,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298512,"kb_used_data":1224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073328,"statfs":{"total":96636764160,"available":96331087872,"internally_reserved":0,"allocated":1253376,"data_stored":538828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.496,"15min":0.50700000000000001},"min":{"1min":0.314,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.64100000000000001,"5min":0.71399999999999997,"15min":0.86599999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.51900000000000002,"15min":0.51400000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.44400000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.50800000000000001,"15min":0.51000000000000001},"min":{"1min":0.34899999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.61699999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.52100000000000002,"15min":0.53100000000000003},"min":{"1min":0.30599999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73899999999999999,"5min":0.754,"15min":0.81599999999999995},"last":0.46700000000000003}]},{"osd":2,"last update":"Mon Oct 9 06:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.33100000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.70699999999999996,"5min":0.73299999999999998,"15min":0.83999999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57899999999999996,"15min":0.58699999999999997},"min":{"1min":0.46500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72799999999999998,"5min":0.745,"15min":0.79900000000000004},"last":0.64100000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.41099999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.69799999999999995,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55500000000000005,"15min":0.56399999999999995},"min":{"1min":0.39800000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.71099999999999997,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.70299999999999996}]},{"osd":4,"last update":"Mon Oct 9 06:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.495,"15min":0.496},"min":{"1min":0.29999999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54200000000000004,"15min":0.54500000000000004},"min":{"1min":0.35299999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.72299999999999998,"5min":0.76300000000000001,"15min":0.80700000000000005},"last":0.71699999999999997}]},{"osd":6,"last update":"Mon Oct 9 06:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55600000000000005,"15min":0.53400000000000003},"min":{"1min":0.376,"5min":0.35699999999999998,"15min":0.32600000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.83199999999999996},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.42599999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.66800000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.39600000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76900000000000002,"5min":0.84399999999999997,"15min":0.85599999999999998},"last":0.749},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.442,"5min":0.35199999999999998,"15min":0.314},"max":{"1min":0.81399999999999995,"5min":0.84499999999999997,"15min":0.90600000000000003},"last":0.76400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:36:48.440 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-09T06:36:48.440 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-09T06:36:48.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.595+0000 7ff5c74d6700 1 Processor -- start 2023-10-09T06:36:48.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.595+0000 7ff5c74d6700 1 -- start start 2023-10-09T06:36:48.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.596+0000 7ff5c74d6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff5c00cf480 0x7ff5c015d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:48.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.596+0000 7ff5c74d6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5c015d760 0x7ff5c00b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:48.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.596+0000 7ff5c74d6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c015dbc0 con 0x7ff5c01595d0 2023-10-09T06:36:48.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.596+0000 7ff5c74d6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c00b10f0 con 0x7ff5c00cf480 2023-10-09T06:36:48.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.596+0000 7ff5c74d6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c015e000 con 0x7ff5c015d760 2023-10-09T06:36:48.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.596+0000 7ff5c5cd3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff5c00cf480 0x7ff5c015d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:48.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.596+0000 7ff5c6cd5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5c015d760 0x7ff5c00b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:48.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.596+0000 7ff5c5cd3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff5c00cf480 0x7ff5c015d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:44496/0 (socket says 172.21.15.163:44496) 2023-10-09T06:36:48.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.596+0000 7ff5c6cd5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5c015d760 0x7ff5c00b1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:54264/0 (socket says 172.21.15.163:54264) 2023-10-09T06:36:48.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.596+0000 7ff5c5cd3700 1 -- 172.21.15.163:0/405445609 learned_addr learned my addr 172.21.15.163:0/405445609 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:48.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.596+0000 7ff5c5cd3700 1 -- 172.21.15.163:0/405445609 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5c015d760 msgr2=0x7ff5c00b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:48.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.596+0000 7ff5c5cd3700 1 --2- 172.21.15.163:0/405445609 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5c015d760 0x7ff5c00b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:48.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.596+0000 7ff5c5cd3700 1 -- 172.21.15.163:0/405445609 >> v1:172.21.15.181:6789/0 conn(0x7ff5c01595d0 legacy=0x7ff5c00ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:48.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.596+0000 7ff5c5cd3700 1 -- 172.21.15.163:0/405445609 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c00cf990 con 0x7ff5c00cf480 2023-10-09T06:36:48.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.597+0000 7ff5c5cd3700 1 --2- 172.21.15.163:0/405445609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5c00cf480 0x7ff5c015d220 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff5b00099b0 tx=0x7ff5b0009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f4ad4f8369dcf9e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:48.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.597+0000 7ff5c54d2700 1 -- 172.21.15.163:0/405445609 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0009ea0 con 0x7ff5c00cf480 2023-10-09T06:36:48.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.597+0000 7ff5c54d2700 1 -- 172.21.15.163:0/405445609 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b000b860 con 0x7ff5c00cf480 2023-10-09T06:36:48.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.597+0000 7ff5c54d2700 1 -- 172.21.15.163:0/405445609 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b000bc00 con 0x7ff5c00cf480 2023-10-09T06:36:48.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.597+0000 7ff5c74d6700 1 -- 172.21.15.163:0/405445609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5c00cf480 msgr2=0x7ff5c015d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:48.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.597+0000 7ff5c74d6700 1 --2- 172.21.15.163:0/405445609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5c00cf480 0x7ff5c015d220 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff5b00099b0 tx=0x7ff5b0009250 comp rx=0 tx=0).stop 2023-10-09T06:36:48.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.597+0000 7ff5c74d6700 1 -- 172.21.15.163:0/405445609 shutdown_connections 2023-10-09T06:36:48.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.597+0000 7ff5c74d6700 1 --2- 172.21.15.163:0/405445609 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5c015d760 0x7ff5c00b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:48.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.597+0000 7ff5c74d6700 1 --2- 172.21.15.163:0/405445609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5c00cf480 0x7ff5c015d220 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:48.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.597+0000 7ff5c74d6700 1 -- 172.21.15.163:0/405445609 >> 172.21.15.163:0/405445609 conn(0x7ff5c00d05f0 msgr2=0x7ff5c00d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:48.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.597+0000 7ff5c74d6700 1 -- 172.21.15.163:0/405445609 shutdown_connections 2023-10-09T06:36:48.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.597+0000 7ff5c74d6700 1 -- 172.21.15.163:0/405445609 wait complete. 2023-10-09T06:36:48.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.598+0000 7ff5c74d6700 1 Processor -- start 2023-10-09T06:36:48.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.598+0000 7ff5c74d6700 1 -- start start 2023-10-09T06:36:48.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.599+0000 7ff5c74d6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5c00cf480 0x7ff5c0158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:48.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.599+0000 7ff5c74d6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c01595d0 0x7ff5c014b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:48.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.599+0000 7ff5c74d6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5c015d760 0x7ff5c015c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:48.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.599+0000 7ff5c74d6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c015b190 con 0x7ff5c01595d0 2023-10-09T06:36:48.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.599+0000 7ff5c74d6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c014dd50 con 0x7ff5c015d760 2023-10-09T06:36:48.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.599+0000 7ff5c74d6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c015b3e0 con 0x7ff5c00cf480 2023-10-09T06:36:48.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.599+0000 7ff5c5cd3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c01595d0 0x7ff5c014b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:48.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.599+0000 7ff5c5cd3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c01595d0 0x7ff5c014b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46074/0 (socket says 172.21.15.163:46074) 2023-10-09T06:36:48.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.599+0000 7ff5c5cd3700 1 -- 172.21.15.163:0/2647674255 learned_addr learned my addr 172.21.15.163:0/2647674255 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:48.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.600+0000 7ff5c6cd5700 1 --2- 172.21.15.163:0/2647674255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5c015d760 0x7ff5c015c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:48.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.600+0000 7ff5c64d4700 1 --2- 172.21.15.163:0/2647674255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5c00cf480 0x7ff5c0158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:48.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.600+0000 7ff5c5cd3700 1 -- 172.21.15.163:0/2647674255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5c00cf480 msgr2=0x7ff5c0158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:48.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.600+0000 7ff5c5cd3700 1 --2- 172.21.15.163:0/2647674255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5c00cf480 0x7ff5c0158730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:48.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.600+0000 7ff5c5cd3700 1 -- 172.21.15.163:0/2647674255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5c015d760 msgr2=0x7ff5c015c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:48.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.600+0000 7ff5c5cd3700 1 --2- 172.21.15.163:0/2647674255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5c015d760 0x7ff5c015c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:48.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.600+0000 7ff5c5cd3700 1 -- 172.21.15.163:0/2647674255 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b00094f0 con 0x7ff5c01595d0 2023-10-09T06:36:48.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.600+0000 7ff5c64d4700 1 --2- 172.21.15.163:0/2647674255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5c00cf480 0x7ff5c0158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:48.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.600+0000 7ff5c6cd5700 1 --2- 172.21.15.163:0/2647674255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5c015d760 0x7ff5c015c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:48.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.600+0000 7ff5c5cd3700 1 --2- 172.21.15.163:0/2647674255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c01595d0 0x7ff5c014b930 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00895f0 tx=0x7ff5b0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:48.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.600+0000 7ff5b67fc700 1 -- 172.21.15.163:0/2647674255 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b000bbe0 con 0x7ff5c01595d0 2023-10-09T06:36:48.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.600+0000 7ff5b67fc700 1 -- 172.21.15.163:0/2647674255 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b0011420 con 0x7ff5c01595d0 2023-10-09T06:36:48.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.600+0000 7ff5c74d6700 1 -- 172.21.15.163:0/2647674255 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c015b6c0 con 0x7ff5c01595d0 2023-10-09T06:36:48.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.600+0000 7ff5b67fc700 1 -- 172.21.15.163:0/2647674255 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b00117c0 con 0x7ff5c01595d0 2023-10-09T06:36:48.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.600+0000 7ff5c74d6700 1 -- 172.21.15.163:0/2647674255 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c015bc80 con 0x7ff5c01595d0 2023-10-09T06:36:48.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.601+0000 7ff5c74d6700 1 -- 172.21.15.163:0/2647674255 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c004c3c0 con 0x7ff5c01595d0 2023-10-09T06:36:48.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.601+0000 7ff5b67fc700 1 -- 172.21.15.163:0/2647674255 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7ff5b0011bf0 con 0x7ff5c01595d0 2023-10-09T06:36:48.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.602+0000 7ff5b67fc700 1 --2- 172.21.15.163:0/2647674255 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff5a4067290 0x7ff5a4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:48.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.602+0000 7ff5b67fc700 1 -- 172.21.15.163:0/2647674255 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff5b0085f60 con 0x7ff5c01595d0 2023-10-09T06:36:48.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.602+0000 7ff5c64d4700 1 --2- 172.21.15.163:0/2647674255 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff5a4067290 0x7ff5a4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:48.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.603+0000 7ff5c64d4700 1 --2- 172.21.15.163:0/2647674255 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff5a4067290 0x7ff5a4069740 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8001850 tx=0x7ff5b8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:48.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.606+0000 7ff5b67fc700 1 -- 172.21.15.163:0/2647674255 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff5b0052e40 con 0x7ff5c01595d0 2023-10-09T06:36:48.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.864+0000 7ff5c74d6700 1 -- 172.21.15.163:0/2647674255 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff5c00b3860 con 0x7ff5c01595d0 2023-10-09T06:36:48.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.868+0000 7ff5b67fc700 1 -- 172.21.15.163:0/2647674255 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7ff5b0056460 con 0x7ff5c01595d0 2023-10-09T06:36:48.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.870+0000 7ff5c74d6700 1 -- 172.21.15.163:0/2647674255 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff5a4067290 msgr2=0x7ff5a4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:48.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.870+0000 7ff5c74d6700 1 --2- 172.21.15.163:0/2647674255 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff5a4067290 0x7ff5a4069740 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8001850 tx=0x7ff5b8009040 comp rx=0 tx=0).stop 2023-10-09T06:36:48.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.870+0000 7ff5c74d6700 1 -- 172.21.15.163:0/2647674255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c01595d0 msgr2=0x7ff5c014b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:48.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.870+0000 7ff5c74d6700 1 --2- 172.21.15.163:0/2647674255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c01595d0 0x7ff5c014b930 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00895f0 tx=0x7ff5b0017000 comp rx=0 tx=0).stop 2023-10-09T06:36:48.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.870+0000 7ff5c74d6700 1 -- 172.21.15.163:0/2647674255 shutdown_connections 2023-10-09T06:36:48.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.870+0000 7ff5c74d6700 1 --2- 172.21.15.163:0/2647674255 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff5a4067290 0x7ff5a4069740 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:48.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.870+0000 7ff5c74d6700 1 --2- 172.21.15.163:0/2647674255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5c015d760 0x7ff5c015c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:48.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.870+0000 7ff5c74d6700 1 --2- 172.21.15.163:0/2647674255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5c00cf480 0x7ff5c0158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:48.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.870+0000 7ff5c74d6700 1 --2- 172.21.15.163:0/2647674255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5c01595d0 0x7ff5c014b930 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:48.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.870+0000 7ff5c74d6700 1 -- 172.21.15.163:0/2647674255 >> 172.21.15.163:0/2647674255 conn(0x7ff5c00d05f0 msgr2=0x7ff5c00693a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:48.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.870+0000 7ff5c74d6700 1 -- 172.21.15.163:0/2647674255 shutdown_connections 2023-10-09T06:36:48.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.870+0000 7ff5c74d6700 1 -- 172.21.15.163:0/2647674255 wait complete. 2023-10-09T06:36:48.872 INFO:teuthology.orchestra.run.smithi163.stderr:updated caps for client.0 2023-10-09T06:36:48.883 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:36:48.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.996+0000 7ff2ce110700 1 Processor -- start 2023-10-09T06:36:48.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.996+0000 7ff2ce110700 1 -- start start 2023-10-09T06:36:49.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.997+0000 7ff2ce110700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2c806c0a0 0x7ff2c80caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.997+0000 7ff2ce110700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2c815d7e0 con 0x7ff2c806bcd0 2023-10-09T06:36:49.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.997+0000 7ff2ce110700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2c815dac0 con 0x7ff2c815df50 2023-10-09T06:36:49.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.997+0000 7ff2ce110700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c815aa30 con 0x7ff2c806c0a0 2023-10-09T06:36:49.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.997+0000 7ff2c77fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff2c806bcd0 0x7ff2c815a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:54322/0 (socket says 172.21.15.163:54322) 2023-10-09T06:36:49.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.997+0000 7ff2c77fe700 1 -- 172.21.15.163:0/3464546027 learned_addr learned my addr 172.21.15.163:0/3464546027 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:49.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.998+0000 7ff2c67fc700 1 -- 172.21.15.163:0/3464546027 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2488732611 0 0) 0x7ff2c815d7e0 con 0x7ff2c806bcd0 2023-10-09T06:36:49.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.998+0000 7ff2c67fc700 1 -- 172.21.15.163:0/3464546027 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2b0003570 con 0x7ff2c806bcd0 2023-10-09T06:36:49.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.998+0000 7ff2c6ffd700 1 --2- 172.21.15.163:0/3464546027 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2c806c0a0 0x7ff2c80caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.998+0000 7ff2c67fc700 1 -- 172.21.15.163:0/3464546027 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 515311382 0 0) 0x7ff2b0003570 con 0x7ff2c806bcd0 2023-10-09T06:36:49.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.998+0000 7ff2c67fc700 1 -- 172.21.15.163:0/3464546027 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2c806c0a0 msgr2=0x7ff2c80caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:49.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.998+0000 7ff2c67fc700 1 --2- 172.21.15.163:0/3464546027 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2c806c0a0 0x7ff2c80caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.998+0000 7ff2c67fc700 1 -- 172.21.15.163:0/3464546027 >> v1:172.21.15.181:6789/0 conn(0x7ff2c815df50 legacy=0x7ff2c80c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:49.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.998+0000 7ff2c67fc700 1 -- 172.21.15.163:0/3464546027 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c815ad10 con 0x7ff2c806bcd0 2023-10-09T06:36:49.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.998+0000 7ff2c67fc700 1 -- 172.21.15.163:0/3464546027 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7ff2b80017f0 con 0x7ff2c806bcd0 2023-10-09T06:36:49.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.998+0000 7ff2c67fc700 1 -- 172.21.15.163:0/3464546027 >> v1:172.21.15.163:6789/0 conn(0x7ff2c806bcd0 legacy=0x7ff2c815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:49.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.998+0000 7ff2c67fc700 1 --2- 172.21.15.163:0/3464546027 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2c806c0a0 0x7ff2b0003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.998+0000 7ff2c67fc700 1 --2- 172.21.15.163:0/3464546027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2b00040c0 0x7ff2b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.998+0000 7ff2c67fc700 1 --2- 172.21.15.163:0/3464546027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2b0004da0 0x7ff2b00051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.998+0000 7ff2c67fc700 1 -- 172.21.15.163:0/3464546027 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c815aa30 con 0x7ff2b00040c0 2023-10-09T06:36:49.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.998+0000 7ff2c67fc700 1 -- 172.21.15.163:0/3464546027 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b000bd10 con 0x7ff2b0004da0 2023-10-09T06:36:49.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.998+0000 7ff2c67fc700 1 -- 172.21.15.163:0/3464546027 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b000bed0 con 0x7ff2c806c0a0 2023-10-09T06:36:49.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.998+0000 7ff2c6ffd700 1 --2- 172.21.15.163:0/3464546027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2b00040c0 0x7ff2b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.998+0000 7ff2c77fe700 1 --2- 172.21.15.163:0/3464546027 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2c806c0a0 0x7ff2b0003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.998+0000 7ff2c7fff700 1 --2- 172.21.15.163:0/3464546027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2b0004da0 0x7ff2b00051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.998+0000 7ff2c6ffd700 1 -- 172.21.15.163:0/3464546027 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2c806c0a0 msgr2=0x7ff2b0003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:49.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.998+0000 7ff2c6ffd700 1 --2- 172.21.15.163:0/3464546027 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2c806c0a0 0x7ff2b0003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.998+0000 7ff2c6ffd700 1 -- 172.21.15.163:0/3464546027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2b0004da0 msgr2=0x7ff2b00051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:49.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.998+0000 7ff2c6ffd700 1 --2- 172.21.15.163:0/3464546027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2b0004da0 0x7ff2b00051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.998+0000 7ff2c6ffd700 1 -- 172.21.15.163:0/3464546027 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b0003570 con 0x7ff2b00040c0 2023-10-09T06:36:49.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.998+0000 7ff2c6ffd700 1 --2- 172.21.15.163:0/3464546027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2b00040c0 0x7ff2b0004810 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc00bb90 tx=0x7ff2bc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31cc487d6e25e328 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:49.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.999+0000 7ff2c67fc700 1 -- 172.21.15.163:0/3464546027 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc00d5d0 con 0x7ff2b00040c0 2023-10-09T06:36:49.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.999+0000 7ff2c67fc700 1 -- 172.21.15.163:0/3464546027 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2bc00d730 con 0x7ff2b00040c0 2023-10-09T06:36:49.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.999+0000 7ff2c67fc700 1 -- 172.21.15.163:0/3464546027 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc00daf0 con 0x7ff2b00040c0 2023-10-09T06:36:49.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.999+0000 7ff2ce110700 1 -- 172.21.15.163:0/3464546027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2b00040c0 msgr2=0x7ff2b0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:49.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.999+0000 7ff2ce110700 1 --2- 172.21.15.163:0/3464546027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2b00040c0 0x7ff2b0004810 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc00bb90 tx=0x7ff2bc008040 comp rx=0 tx=0).stop 2023-10-09T06:36:49.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.999+0000 7ff2c7fff700 1 -- 172.21.15.163:0/3464546027 reap_dead start 2023-10-09T06:36:49.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.999+0000 7ff2ce110700 1 -- 172.21.15.163:0/3464546027 shutdown_connections 2023-10-09T06:36:49.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.999+0000 7ff2ce110700 1 -- 172.21.15.163:0/3464546027 >> 172.21.15.163:0/3464546027 conn(0x7ff2c80ce600 msgr2=0x7ff2c80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:49.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.999+0000 7ff2ce110700 1 -- 172.21.15.163:0/3464546027 shutdown_connections 2023-10-09T06:36:49.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:48.999+0000 7ff2ce110700 1 -- 172.21.15.163:0/3464546027 wait complete. 2023-10-09T06:36:49.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.000+0000 7ff2ce110700 1 Processor -- start 2023-10-09T06:36:49.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.000+0000 7ff2ce110700 1 -- start start 2023-10-09T06:36:49.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.001+0000 7ff2ce110700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2b00040c0 0x7ff2c81549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.001+0000 7ff2ce110700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2c80b3dd0 0x7ff2c806cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.001+0000 7ff2ce110700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2c806aa30 0x7ff2c806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.001+0000 7ff2ce110700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c80b3510 con 0x7ff2b00040c0 2023-10-09T06:36:49.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.001+0000 7ff2ce110700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c8154f10 con 0x7ff2c806aa30 2023-10-09T06:36:49.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.001+0000 7ff2ce110700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c80b3080 con 0x7ff2c80b3dd0 2023-10-09T06:36:49.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.001+0000 7ff2c77fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2b00040c0 0x7ff2c81549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.001+0000 7ff2c77fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2b00040c0 0x7ff2c81549d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46086/0 (socket says 172.21.15.163:46086) 2023-10-09T06:36:49.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.001+0000 7ff2c77fe700 1 -- 172.21.15.163:0/3933453520 learned_addr learned my addr 172.21.15.163:0/3933453520 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:49.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.001+0000 7ff2c7fff700 1 --2- 172.21.15.163:0/3933453520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2c806aa30 0x7ff2c806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.001+0000 7ff2c6ffd700 1 --2- 172.21.15.163:0/3933453520 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2c80b3dd0 0x7ff2c806cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.001+0000 7ff2c77fe700 1 -- 172.21.15.163:0/3933453520 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2c80b3dd0 msgr2=0x7ff2c806cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:49.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.001+0000 7ff2c77fe700 1 --2- 172.21.15.163:0/3933453520 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2c80b3dd0 0x7ff2c806cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.001+0000 7ff2c77fe700 1 -- 172.21.15.163:0/3933453520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2c806aa30 msgr2=0x7ff2c806d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:49.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.001+0000 7ff2c77fe700 1 --2- 172.21.15.163:0/3933453520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2c806aa30 0x7ff2c806d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.002+0000 7ff2c77fe700 1 -- 172.21.15.163:0/3933453520 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c80b2470 con 0x7ff2b00040c0 2023-10-09T06:36:49.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.002+0000 7ff2c7fff700 1 --2- 172.21.15.163:0/3933453520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2c806aa30 0x7ff2c806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:49.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.002+0000 7ff2c77fe700 1 --2- 172.21.15.163:0/3933453520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2b00040c0 0x7ff2c81549d0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8007af0 tx=0x7ff2c80ce4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:49.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.002+0000 7ff2af7fe700 1 -- 172.21.15.163:0/3933453520 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b800ad70 con 0x7ff2b00040c0 2023-10-09T06:36:49.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.002+0000 7ff2af7fe700 1 -- 172.21.15.163:0/3933453520 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b80133f0 con 0x7ff2b00040c0 2023-10-09T06:36:49.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.002+0000 7ff2ce110700 1 -- 172.21.15.163:0/3933453520 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2c80b2750 con 0x7ff2b00040c0 2023-10-09T06:36:49.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.002+0000 7ff2af7fe700 1 -- 172.21.15.163:0/3933453520 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b80137b0 con 0x7ff2b00040c0 2023-10-09T06:36:49.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.002+0000 7ff2ce110700 1 -- 172.21.15.163:0/3933453520 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2c80b2d10 con 0x7ff2b00040c0 2023-10-09T06:36:49.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.003+0000 7ff2ce110700 1 -- 172.21.15.163:0/3933453520 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2c804c3c0 con 0x7ff2b00040c0 2023-10-09T06:36:49.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.003+0000 7ff2af7fe700 1 -- 172.21.15.163:0/3933453520 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7ff2b8013ba0 con 0x7ff2b00040c0 2023-10-09T06:36:49.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.004+0000 7ff2af7fe700 1 --2- 172.21.15.163:0/3933453520 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff2b4067290 0x7ff2b4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.004+0000 7ff2af7fe700 1 -- 172.21.15.163:0/3933453520 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff2b8085890 con 0x7ff2b00040c0 2023-10-09T06:36:49.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.004+0000 7ff2c6ffd700 1 --2- 172.21.15.163:0/3933453520 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff2b4067290 0x7ff2b4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.004+0000 7ff2c6ffd700 1 --2- 172.21.15.163:0/3933453520 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff2b4067290 0x7ff2b4069740 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc00bb90 tx=0x7ff2bc01b000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:49.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.008+0000 7ff2af7fe700 1 -- 172.21.15.163:0/3933453520 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff2b80526f0 con 0x7ff2b00040c0 2023-10-09T06:36:49.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.233+0000 7ff2ce110700 1 -- 172.21.15.163:0/3933453520 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2c815aff0 con 0x7ff2b00040c0 2023-10-09T06:36:49.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.233+0000 7ff2af7fe700 1 -- 172.21.15.163:0/3933453520 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4611 (secure 0 0 0) 0x7ff2b8017090 con 0x7ff2b00040c0 2023-10-09T06:36:49.235 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:49.235 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3452423203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3452423203},{"type":"v1","addr":"172.21.15.181:6836","nonce":3452423203}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6298,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2097756943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2097756943},{"type":"v1","addr":"172.21.15.181:6837","nonce":2097756943}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6831,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3240892979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3240892979},{"type":"v1","addr":"172.21.15.163:6834","nonce":3240892979}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:36:43.382051+0000","modified":"2023-10-09T06:36:47.014979+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6834},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6834":{"gid":6834,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/1684208805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1684208805},{"type":"v1","addr":"172.21.15.163:6835","nonce":1684208805}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-09T06:36:49.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.236+0000 7ff2ce110700 1 -- 172.21.15.163:0/3933453520 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff2b4067290 msgr2=0x7ff2b4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:49.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.236+0000 7ff2ce110700 1 --2- 172.21.15.163:0/3933453520 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff2b4067290 0x7ff2b4069740 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc00bb90 tx=0x7ff2bc01b000 comp rx=0 tx=0).stop 2023-10-09T06:36:49.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.236+0000 7ff2ce110700 1 -- 172.21.15.163:0/3933453520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2b00040c0 msgr2=0x7ff2c81549d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:49.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.236+0000 7ff2ce110700 1 --2- 172.21.15.163:0/3933453520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2b00040c0 0x7ff2c81549d0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8007af0 tx=0x7ff2c80ce4d0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.236+0000 7ff2ce110700 1 -- 172.21.15.163:0/3933453520 shutdown_connections 2023-10-09T06:36:49.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.236+0000 7ff2ce110700 1 --2- 172.21.15.163:0/3933453520 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff2b4067290 0x7ff2b4069740 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.236+0000 7ff2ce110700 1 --2- 172.21.15.163:0/3933453520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2b00040c0 0x7ff2c81549d0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.236+0000 7ff2ce110700 1 --2- 172.21.15.163:0/3933453520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2c806aa30 0x7ff2c806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.236+0000 7ff2ce110700 1 --2- 172.21.15.163:0/3933453520 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2c80b3dd0 0x7ff2c806cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.236+0000 7ff2ce110700 1 -- 172.21.15.163:0/3933453520 >> 172.21.15.163:0/3933453520 conn(0x7ff2c80ce600 msgr2=0x7ff2c815b230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:49.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.236+0000 7ff2ce110700 1 -- 172.21.15.163:0/3933453520 shutdown_connections 2023-10-09T06:36:49.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.236+0000 7ff2ce110700 1 -- 172.21.15.163:0/3933453520 wait complete. 2023-10-09T06:36:49.239 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 66 2023-10-09T06:36:49.532 INFO:tasks.ceph.mds.c.smithi163.stderr:2023-10-09T06:36:49.531+0000 7f669bc50700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-09T06:36:49.651 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-09T06:36:43.382051+0000', 'modified': '2023-10-09T06:36:47.014979+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6834}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6834': {'gid': 6834, 'name': 'c', 'rank': 0, 'incarnation': 62, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.163:6835/1684208805', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6833', 'nonce': 1684208805}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 1684208805}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-09T06:36:49.651 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-09T06:36:49.652 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-09T06:36:49.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.767+0000 7fe8b759e700 1 Processor -- start 2023-10-09T06:36:49.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.767+0000 7fe8b759e700 1 -- start start 2023-10-09T06:36:49.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.768+0000 7fe8b759e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe8b8159420 0x7fe8b80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.768+0000 7fe8b759e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe8b81597f0 0x7fe8b80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.768+0000 7fe8b759e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b80b11d0 con 0x7fe8b815d230 2023-10-09T06:36:49.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.768+0000 7fe8b759e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b80b14b0 con 0x7fe8b81597f0 2023-10-09T06:36:49.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.768+0000 7fe8b759e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b80b1790 con 0x7fe8b8159420 2023-10-09T06:36:49.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.768+0000 7fe8affff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe8b81597f0 0x7fe8b80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.768+0000 7fe8affff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe8b81597f0 0x7fe8b80b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46088/0 (socket says 172.21.15.163:46088) 2023-10-09T06:36:49.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.768+0000 7fe8affff700 1 -- 172.21.15.163:0/1076821801 learned_addr learned my addr 172.21.15.163:0/1076821801 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:49.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.768+0000 7fe8b659c700 1 --2- 172.21.15.163:0/1076821801 >> v2:172.21.15.181:3300/0 conn(0x7fe8b8159420 0x7fe8b80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.768+0000 7fe8affff700 1 -- 172.21.15.163:0/1076821801 >> v2:172.21.15.181:3300/0 conn(0x7fe8b8159420 msgr2=0x7fe8b80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:49.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.768+0000 7fe8affff700 1 --2- 172.21.15.163:0/1076821801 >> v2:172.21.15.181:3300/0 conn(0x7fe8b8159420 0x7fe8b80b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.768+0000 7fe8affff700 1 -- 172.21.15.163:0/1076821801 >> v1:172.21.15.181:6789/0 conn(0x7fe8b815d230 legacy=0x7fe8b815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:49.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.768+0000 7fe8affff700 1 -- 172.21.15.163:0/1076821801 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b80b1a70 con 0x7fe8b81597f0 2023-10-09T06:36:49.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.769+0000 7fe8affff700 1 --2- 172.21.15.163:0/1076821801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8b81597f0 0x7fe8b80b2270 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fe8b80b2f20 tx=0x7fe8a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1878bea2df235c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:49.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.769+0000 7fe8b5d9b700 1 -- 172.21.15.163:0/1076821801 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a000b820 con 0x7fe8b81597f0 2023-10-09T06:36:49.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.769+0000 7fe8b5d9b700 1 -- 172.21.15.163:0/1076821801 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a000b980 con 0x7fe8b81597f0 2023-10-09T06:36:49.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.769+0000 7fe8b5d9b700 1 -- 172.21.15.163:0/1076821801 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a000bd20 con 0x7fe8b81597f0 2023-10-09T06:36:49.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.769+0000 7fe8b759e700 1 -- 172.21.15.163:0/1076821801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8b81597f0 msgr2=0x7fe8b80b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:49.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.769+0000 7fe8b759e700 1 --2- 172.21.15.163:0/1076821801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8b81597f0 0x7fe8b80b2270 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fe8b80b2f20 tx=0x7fe8a0009250 comp rx=0 tx=0).stop 2023-10-09T06:36:49.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.769+0000 7fe8b759e700 1 -- 172.21.15.163:0/1076821801 shutdown_connections 2023-10-09T06:36:49.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.769+0000 7fe8b759e700 1 --2- 172.21.15.163:0/1076821801 >> v2:172.21.15.181:3300/0 conn(0x7fe8b8159420 0x7fe8b80b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.769+0000 7fe8b759e700 1 --2- 172.21.15.163:0/1076821801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8b81597f0 0x7fe8b80b2270 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.769+0000 7fe8b759e700 1 -- 172.21.15.163:0/1076821801 >> 172.21.15.163:0/1076821801 conn(0x7fe8b80d0660 msgr2=0x7fe8b80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:49.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.769+0000 7fe8b759e700 1 -- 172.21.15.163:0/1076821801 shutdown_connections 2023-10-09T06:36:49.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.769+0000 7fe8b759e700 1 -- 172.21.15.163:0/1076821801 wait complete. 2023-10-09T06:36:49.787 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.770+0000 7fe8b759e700 1 Processor -- start 2023-10-09T06:36:49.787 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.770+0000 7fe8b759e700 1 -- start start 2023-10-09T06:36:49.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.771+0000 7fe8b759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8b8159420 0x7fe8b8154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.771+0000 7fe8b759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8b81597f0 0x7fe8b8155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.771+0000 7fe8b759e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8b815d230 0x7fe8b81563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.771+0000 7fe8b759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b80c8520 con 0x7fe8b81597f0 2023-10-09T06:36:49.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.771+0000 7fe8b759e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b815b660 con 0x7fe8b815d230 2023-10-09T06:36:49.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.771+0000 7fe8b759e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b8154cd0 con 0x7fe8b8159420 2023-10-09T06:36:49.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.771+0000 7fe8affff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8b81597f0 0x7fe8b8155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.771+0000 7fe8affff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8b81597f0 0x7fe8b8155e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46104/0 (socket says 172.21.15.163:46104) 2023-10-09T06:36:49.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.771+0000 7fe8affff700 1 -- 172.21.15.163:0/2963592524 learned_addr learned my addr 172.21.15.163:0/2963592524 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:49.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.771+0000 7fe8b6d9d700 1 --2- 172.21.15.163:0/2963592524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8b815d230 0x7fe8b81563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.771+0000 7fe8b659c700 1 --2- 172.21.15.163:0/2963592524 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8b8159420 0x7fe8b8154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.771+0000 7fe8affff700 1 -- 172.21.15.163:0/2963592524 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8b8159420 msgr2=0x7fe8b8154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:49.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.771+0000 7fe8affff700 1 --2- 172.21.15.163:0/2963592524 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8b8159420 0x7fe8b8154790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.771+0000 7fe8affff700 1 -- 172.21.15.163:0/2963592524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8b815d230 msgr2=0x7fe8b81563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:49.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.771+0000 7fe8affff700 1 --2- 172.21.15.163:0/2963592524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8b815d230 0x7fe8b81563a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.771+0000 7fe8affff700 1 -- 172.21.15.163:0/2963592524 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a00094f0 con 0x7fe8b81597f0 2023-10-09T06:36:49.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.772+0000 7fe8affff700 1 --2- 172.21.15.163:0/2963592524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8b81597f0 0x7fe8b8155e60 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8089340 tx=0x7fe8a0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:49.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.772+0000 7fe8ae7fc700 1 -- 172.21.15.163:0/2963592524 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0011d30 con 0x7fe8b81597f0 2023-10-09T06:36:49.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.772+0000 7fe8ae7fc700 1 -- 172.21.15.163:0/2963592524 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a0020070 con 0x7fe8b81597f0 2023-10-09T06:36:49.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.772+0000 7fe8b759e700 1 -- 172.21.15.163:0/2963592524 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b815ad30 con 0x7fe8b81597f0 2023-10-09T06:36:49.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.772+0000 7fe8ae7fc700 1 -- 172.21.15.163:0/2963592524 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a001b6a0 con 0x7fe8b81597f0 2023-10-09T06:36:49.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.772+0000 7fe8b759e700 1 -- 172.21.15.163:0/2963592524 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b815b2f0 con 0x7fe8b81597f0 2023-10-09T06:36:49.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.773+0000 7fe8ae7fc700 1 -- 172.21.15.163:0/2963592524 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fe8a0022030 con 0x7fe8b81597f0 2023-10-09T06:36:49.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.773+0000 7fe8ae7fc700 1 --2- 172.21.15.163:0/2963592524 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe898067290 0x7fe898069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.774+0000 7fe8ae7fc700 1 -- 172.21.15.163:0/2963592524 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe8a0086900 con 0x7fe8b81597f0 2023-10-09T06:36:49.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.774+0000 7fe8b659c700 1 --2- 172.21.15.163:0/2963592524 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe898067290 0x7fe898069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.774+0000 7fe8b659c700 1 --2- 172.21.15.163:0/2963592524 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe898067290 0x7fe898069740 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8006fd0 tx=0x7fe8a8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:49.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.776+0000 7fe8c5bf1b80 1 Processor -- start 2023-10-09T06:36:49.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.776+0000 7fe8c5bf1b80 1 -- start start 2023-10-09T06:36:49.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.776+0000 7fe8c5bf1b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618b5d8a480 0x5618b5d697d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.776+0000 7fe8c5bf1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618b5d3d5f0 0x5618b5d8a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.776+0000 7fe8c5bf1b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5618b5d3fb80 0x5618b5d3fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.776+0000 7fe8c5bf1b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b5c96a70 con 0x5618b5d8a480 2023-10-09T06:36:49.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.776+0000 7fe8c5bf1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b5c94270 con 0x5618b5d3d5f0 2023-10-09T06:36:49.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.776+0000 7fe8c5bf1b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5618b5c90060 con 0x5618b5d3fb80 2023-10-09T06:36:49.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.776+0000 7fe8b659c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618b5d8a480 0x5618b5d697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.776+0000 7fe8b659c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618b5d8a480 0x5618b5d697d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46120/0 (socket says 172.21.15.163:46120) 2023-10-09T06:36:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.776+0000 7fe8b659c700 1 -- 172.21.15.163:0/1322128014 learned_addr learned my addr 172.21.15.163:0/1322128014 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.777+0000 7fe8b659c700 1 -- 172.21.15.163:0/1322128014 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5618b5d3fb80 msgr2=0x5618b5d3fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.777+0000 7fe8b659c700 1 --2- 172.21.15.163:0/1322128014 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5618b5d3fb80 0x5618b5d3fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.777+0000 7fe8b659c700 1 -- 172.21.15.163:0/1322128014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618b5d3d5f0 msgr2=0x5618b5d8a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.777+0000 7fe8b659c700 1 --2- 172.21.15.163:0/1322128014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618b5d3d5f0 0x5618b5d8a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.777+0000 7fe8b659c700 1 -- 172.21.15.163:0/1322128014 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618b5b2a270 con 0x5618b5d8a480 2023-10-09T06:36:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.777+0000 7fe8b659c700 1 --2- 172.21.15.163:0/1322128014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618b5d8a480 0x5618b5d697d0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800fa90 tx=0x7fe8a8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ff3eab7adc31fd server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.777+0000 7fe8977fe700 1 -- 172.21.15.163:0/1322128014 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8024070 con 0x5618b5d8a480 2023-10-09T06:36:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.777+0000 7fe8977fe700 1 -- 172.21.15.163:0/1322128014 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a8019070 con 0x5618b5d8a480 2023-10-09T06:36:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.777+0000 7fe8977fe700 1 -- 172.21.15.163:0/1322128014 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8014b00 con 0x5618b5d8a480 2023-10-09T06:36:49.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.778+0000 7fe8c5bf1b80 1 -- 172.21.15.163:0/1322128014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618b5d8a480 msgr2=0x5618b5d697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:49.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.778+0000 7fe8c5bf1b80 1 --2- 172.21.15.163:0/1322128014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618b5d8a480 0x5618b5d697d0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800fa90 tx=0x7fe8a8016000 comp rx=0 tx=0).stop 2023-10-09T06:36:49.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.778+0000 7fe8c5bf1b80 1 -- 172.21.15.163:0/1322128014 shutdown_connections 2023-10-09T06:36:49.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.778+0000 7fe8c5bf1b80 1 --2- 172.21.15.163:0/1322128014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618b5d8a480 0x5618b5d697d0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.778+0000 7fe8c5bf1b80 1 --2- 172.21.15.163:0/1322128014 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5618b5d3fb80 0x5618b5d3fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.778+0000 7fe8c5bf1b80 1 --2- 172.21.15.163:0/1322128014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618b5d3d5f0 0x5618b5d8a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.778+0000 7fe8c5bf1b80 1 -- 172.21.15.163:0/1322128014 >> 172.21.15.163:0/1322128014 conn(0x5618b5d410c0 msgr2=0x5618b5d418f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:49.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.778+0000 7fe8c5bf1b80 1 -- 172.21.15.163:0/1322128014 shutdown_connections 2023-10-09T06:36:49.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.778+0000 7fe8c5bf1b80 1 -- 172.21.15.163:0/1322128014 wait complete. 2023-10-09T06:36:49.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.778+0000 7fe8c5bf1b80 1 -- start start 2023-10-09T06:36:49.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.778+0000 7fe8c5bf1b80 1 Processor -- start 2023-10-09T06:36:49.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.779+0000 7fe8c5bf1b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5618b5d3d5f0 0x5618b5d85ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.779+0000 7fe8c5bf1b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618b5d3fb80 0x5618b5d9a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.779+0000 7fe8c5bf1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618b5d8a480 0x5618b5d894c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.779+0000 7fe8c5bf1b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b5c3b890 con 0x5618b5d3fb80 2023-10-09T06:36:49.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.779+0000 7fe8c5bf1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b5c56ea0 con 0x5618b5d8a480 2023-10-09T06:36:49.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.779+0000 7fe8c5bf1b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5618b5b12e30 con 0x5618b5d3d5f0 2023-10-09T06:36:49.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.779+0000 7fe8c5bf1b80 10 client.0 _ll_register_callbacks cb 0x5618b5d032c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-09T06:36:49.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.779+0000 7fe8affff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618b5d3fb80 0x5618b5d9a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.779+0000 7fe8affff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618b5d3fb80 0x5618b5d9a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46130/0 (socket says 172.21.15.163:46130) 2023-10-09T06:36:49.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.779+0000 7fe8affff700 1 -- 172.21.15.163:0/984585438 learned_addr learned my addr 172.21.15.163:0/984585438 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:49.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.779+0000 7fe8b659c700 1 --2- 172.21.15.163:0/984585438 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5618b5d3d5f0 0x5618b5d85ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.779+0000 7fe8b6d9d700 1 --2- 172.21.15.163:0/984585438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618b5d8a480 0x5618b5d894c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.780+0000 7fe8affff700 1 -- 172.21.15.163:0/984585438 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5618b5d3d5f0 msgr2=0x5618b5d85ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:49.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.780+0000 7fe8affff700 1 --2- 172.21.15.163:0/984585438 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5618b5d3d5f0 0x5618b5d85ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.780+0000 7fe8affff700 1 -- 172.21.15.163:0/984585438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618b5d8a480 msgr2=0x5618b5d894c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:49.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.780+0000 7fe8affff700 1 --2- 172.21.15.163:0/984585438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618b5d8a480 0x5618b5d894c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.780+0000 7fe8affff700 1 -- 172.21.15.163:0/984585438 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a800e040 con 0x5618b5d3fb80 2023-10-09T06:36:49.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.780+0000 7fe8b659c700 1 --2- 172.21.15.163:0/984585438 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5618b5d3d5f0 0x5618b5d85ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:49.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.780+0000 7fe8affff700 1 --2- 172.21.15.163:0/984585438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618b5d3fb80 0x5618b5d9a1e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0006fd0 tx=0x7fe8a00118f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c023eaa0eb754978 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:49.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.780+0000 7fe894ff9700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-10-09T06:36:49.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.780+0000 7fe894ff9700 1 -- 172.21.15.163:0/984585438 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a006d070 con 0x5618b5d3fb80 2023-10-09T06:36:49.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.780+0000 7fe894ff9700 1 -- 172.21.15.163:0/984585438 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe8a0057420 con 0x5618b5d3fb80 2023-10-09T06:36:49.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.780+0000 7fe8c5bf1b80 1 -- 172.21.15.163:0/984585438 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5618b5d34cd0 con 0x5618b5d3fb80 2023-10-09T06:36:49.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.780+0000 7fe894ff9700 1 -- 172.21.15.163:0/984585438 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8a005a670 con 0x5618b5d3fb80 2023-10-09T06:36:49.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.780+0000 7fe8c5bf1b80 10 client.6969 fetch_fsmap learned FSMap version 66 2023-10-09T06:36:49.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.780+0000 7fe8c5bf1b80 1 -- 172.21.15.163:0/984585438 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x5618b5accfe0 con 0x5618b5d3fb80 2023-10-09T06:36:49.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.780+0000 7fe894ff9700 1 -- 172.21.15.163:0/984585438 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5618b5d34cd0 con 0x5618b5d3fb80 2023-10-09T06:36:49.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.780+0000 7fe894ff9700 1 -- 172.21.15.163:0/984585438 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 66) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe8a0056910 con 0x5618b5d3fb80 2023-10-09T06:36:49.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.780+0000 7fe8c5bf1b80 10 client.6969 fetch_fsmap finished waiting for FSMap version 66 2023-10-09T06:36:49.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.780+0000 7fe8c5bf1b80 10 client.6969 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-09T06:36:49.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.780+0000 7fe8c5bf1b80 20 client.6969 populate_metadata read hostname 'smithi163' 2023-10-09T06:36:49.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.780+0000 7fe8c5bf1b80 1 --2- 172.21.15.163:0/984585438 >> [v2:172.21.15.163:6833/1684208805,v1:172.21.15.163:6835/1684208805] conn(0x5618b5d6e740 0x5618b5d6eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.780+0000 7fe8c5bf1b80 4 client.6969 mds_command: new command op to 6834 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-09T06:36:49.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.780+0000 7fe8c5bf1b80 1 -- 172.21.15.163:0/984585438 --> [v2:172.21.15.163:6833/1684208805,v1:172.21.15.163:6835/1684208805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5618b5d11aa0 con 0x5618b5d6e740 2023-10-09T06:36:49.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.780+0000 7fe8b659c700 1 --2- 172.21.15.163:0/984585438 >> [v2:172.21.15.163:6833/1684208805,v1:172.21.15.163:6835/1684208805] conn(0x5618b5d6e740 0x5618b5d6eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.781+0000 7fe894ff9700 10 client.6969 ms_handle_connect on v2:172.21.15.163:6833/1684208805 2023-10-09T06:36:49.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.781+0000 7fe8b659c700 1 --2- 172.21.15.163:0/984585438 >> [v2:172.21.15.163:6833/1684208805,v1:172.21.15.163:6835/1684208805] conn(0x5618b5d6e740 0x5618b5d6eb10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a8e1f28e7086056 server_cookie=a9347c1a44afe844 in_seq=0 out_seq=0 2023-10-09T06:36:49.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.781+0000 7fe894ff9700 1 -- 172.21.15.163:0/984585438 <== mds.0 v2:172.21.15.163:6833/1684208805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5618b5d11aa0 con 0x5618b5d6e740 2023-10-09T06:36:49.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.781+0000 7fe894ff9700 10 client.6969 handle_command_reply: tid=0 2023-10-09T06:36:49.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.781+0000 7fe8c5bf1b80 1 client.6969 shutdown 2023-10-09T06:36:49.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.781+0000 7fe8c5bf1b80 1 -- 172.21.15.163:0/984585438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618b5d3fb80 msgr2=0x5618b5d9a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:49.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.781+0000 7fe8c5bf1b80 1 --2- 172.21.15.163:0/984585438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618b5d3fb80 0x5618b5d9a1e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0006fd0 tx=0x7fe8a00118f0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.781+0000 7fe8c5bf1b80 1 -- 172.21.15.163:0/984585438 shutdown_connections 2023-10-09T06:36:49.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.781+0000 7fe8c5bf1b80 1 --2- 172.21.15.163:0/984585438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618b5d8a480 0x5618b5d894c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.781+0000 7fe8c5bf1b80 1 --2- 172.21.15.163:0/984585438 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5618b5d3d5f0 0x5618b5d85ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.781+0000 7fe8c5bf1b80 1 --2- 172.21.15.163:0/984585438 >> [v2:172.21.15.163:6833/1684208805,v1:172.21.15.163:6835/1684208805] conn(0x5618b5d6e740 0x5618b5d6eb10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.781+0000 7fe8c5bf1b80 1 --2- 172.21.15.163:0/984585438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618b5d3fb80 0x5618b5d9a1e0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.781+0000 7fe8c5bf1b80 1 -- 172.21.15.163:0/984585438 >> 172.21.15.163:0/984585438 conn(0x5618b5d410c0 msgr2=0x5618b5d418d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:49.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.781+0000 7fe894ff9700 0 client.6969 ms_handle_reset on v2:172.21.15.163:6833/1684208805 2023-10-09T06:36:49.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.782+0000 7fe8c5bf1b80 1 -- 172.21.15.163:0/984585438 shutdown_connections 2023-10-09T06:36:49.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.782+0000 7fe8c5bf1b80 1 -- 172.21.15.163:0/984585438 wait complete. 2023-10-09T06:36:49.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.782+0000 7fe8c5bf1b80 20 client.6969 trim_cache size 0 max 16384 2023-10-09T06:36:49.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.798+0000 7fe8c5bf1b80 1 Processor -- start 2023-10-09T06:36:49.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.799+0000 7fe8c5bf1b80 1 -- start start 2023-10-09T06:36:49.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.799+0000 7fe8c5bf1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618b5d3d5f0 0x5618b5d3cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.799+0000 7fe8c5bf1b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618b5d3fb80 0x5618b5d41c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.799+0000 7fe8c5bf1b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5618b5d8a480 0x5618b5d42140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.799+0000 7fe8c5bf1b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b5d6c420 con 0x5618b5d3fb80 2023-10-09T06:36:49.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.799+0000 7fe8c5bf1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b5d89030 con 0x5618b5d3d5f0 2023-10-09T06:36:49.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.799+0000 7fe8c5bf1b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5618b5d6b2e0 con 0x5618b5d8a480 2023-10-09T06:36:49.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.799+0000 7fe8affff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618b5d3fb80 0x5618b5d41c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.799+0000 7fe8affff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618b5d3fb80 0x5618b5d41c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46140/0 (socket says 172.21.15.163:46140) 2023-10-09T06:36:49.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.799+0000 7fe8affff700 1 -- 172.21.15.163:0/1138281011 learned_addr learned my addr 172.21.15.163:0/1138281011 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:49.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.800+0000 7fe8affff700 1 -- 172.21.15.163:0/1138281011 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5618b5d8a480 msgr2=0x5618b5d42140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:49.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.800+0000 7fe8affff700 1 --2- 172.21.15.163:0/1138281011 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5618b5d8a480 0x5618b5d42140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.800+0000 7fe8affff700 1 -- 172.21.15.163:0/1138281011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618b5d3d5f0 msgr2=0x5618b5d3cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:49.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.800+0000 7fe8affff700 1 --2- 172.21.15.163:0/1138281011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618b5d3d5f0 0x5618b5d3cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.800+0000 7fe8affff700 1 -- 172.21.15.163:0/1138281011 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618b5c5e780 con 0x5618b5d3fb80 2023-10-09T06:36:49.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.800+0000 7fe8affff700 1 --2- 172.21.15.163:0/1138281011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618b5d3fb80 0x5618b5d41c00 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0053920 tx=0x7fe8a0008000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d365b73c29080e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:49.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.800+0000 7fe897fff700 1 -- 172.21.15.163:0/1138281011 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a006d040 con 0x5618b5d3fb80 2023-10-09T06:36:49.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.800+0000 7fe897fff700 1 -- 172.21.15.163:0/1138281011 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a0057420 con 0x5618b5d3fb80 2023-10-09T06:36:49.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.800+0000 7fe897fff700 1 -- 172.21.15.163:0/1138281011 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a005e3c0 con 0x5618b5d3fb80 2023-10-09T06:36:49.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.800+0000 7fe8c5bf1b80 1 -- 172.21.15.163:0/1138281011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618b5d3fb80 msgr2=0x5618b5d41c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:49.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.800+0000 7fe8c5bf1b80 1 --2- 172.21.15.163:0/1138281011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618b5d3fb80 0x5618b5d41c00 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0053920 tx=0x7fe8a0008000 comp rx=0 tx=0).stop 2023-10-09T06:36:49.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.800+0000 7fe8c5bf1b80 1 -- 172.21.15.163:0/1138281011 shutdown_connections 2023-10-09T06:36:49.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.800+0000 7fe8c5bf1b80 1 --2- 172.21.15.163:0/1138281011 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5618b5d8a480 0x5618b5d42140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.800+0000 7fe8c5bf1b80 1 --2- 172.21.15.163:0/1138281011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618b5d3d5f0 0x5618b5d3cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.800+0000 7fe8c5bf1b80 1 --2- 172.21.15.163:0/1138281011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618b5d3fb80 0x5618b5d41c00 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.800+0000 7fe8c5bf1b80 1 -- 172.21.15.163:0/1138281011 >> 172.21.15.163:0/1138281011 conn(0x5618b5d410c0 msgr2=0x5618b5d3ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:49.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.800+0000 7fe8c5bf1b80 1 -- 172.21.15.163:0/1138281011 shutdown_connections 2023-10-09T06:36:49.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.800+0000 7fe8c5bf1b80 1 -- 172.21.15.163:0/1138281011 wait complete. 2023-10-09T06:36:49.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.801+0000 7fe8c5bf1b80 1 -- start start 2023-10-09T06:36:49.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.801+0000 7fe8c5bf1b80 1 Processor -- start 2023-10-09T06:36:49.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.802+0000 7fe8c5bf1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618b5d3d5f0 0x5618b5deb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.802+0000 7fe8c5bf1b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5618b5d3fb80 0x5618b5d69f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.802+0000 7fe8c5bf1b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618b5d8a480 0x5618b5d6a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.802+0000 7fe8c5bf1b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b5d87520 con 0x5618b5d8a480 2023-10-09T06:36:49.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.802+0000 7fe8c5bf1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b5deb9a0 con 0x5618b5d3d5f0 2023-10-09T06:36:49.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.802+0000 7fe8c5bf1b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5618b5d36410 con 0x5618b5d3fb80 2023-10-09T06:36:49.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.802+0000 7fe8c5bf1b80 10 client.0 _ll_register_callbacks cb 0x5618b5d032c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-09T06:36:49.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.802+0000 7fe8b6d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618b5d8a480 0x5618b5d6a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.802+0000 7fe8b6d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618b5d8a480 0x5618b5d6a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46152/0 (socket says 172.21.15.163:46152) 2023-10-09T06:36:49.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.802+0000 7fe8b6d9d700 1 -- 172.21.15.163:0/664727453 learned_addr learned my addr 172.21.15.163:0/664727453 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:49.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.802+0000 7fe8affff700 1 --2- 172.21.15.163:0/664727453 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5618b5d3fb80 0x5618b5d69f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.802+0000 7fe8b6d9d700 1 -- 172.21.15.163:0/664727453 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5618b5d3fb80 msgr2=0x5618b5d69f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:49.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.802+0000 7fe8b6d9d700 1 --2- 172.21.15.163:0/664727453 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5618b5d3fb80 0x5618b5d69f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.802+0000 7fe8b6d9d700 1 -- 172.21.15.163:0/664727453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618b5d3d5f0 msgr2=0x5618b5deb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:49.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.803+0000 7fe8b659c700 1 --2- 172.21.15.163:0/664727453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618b5d3d5f0 0x5618b5deb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.803+0000 7fe8b6d9d700 1 --2- 172.21.15.163:0/664727453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618b5d3d5f0 0x5618b5deb460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.803+0000 7fe8b6d9d700 1 -- 172.21.15.163:0/664727453 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b815b2f0 con 0x5618b5d8a480 2023-10-09T06:36:49.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.803+0000 7fe8b6d9d700 1 --2- 172.21.15.163:0/664727453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618b5d8a480 0x5618b5d6a470 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fe8b815dd40 tx=0x7fe8b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b8e6a8b17d38dd server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:49.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.803+0000 7fe894ff9700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-10-09T06:36:49.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.803+0000 7fe894ff9700 1 -- 172.21.15.163:0/664727453 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b000e050 con 0x5618b5d8a480 2023-10-09T06:36:49.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.803+0000 7fe894ff9700 1 -- 172.21.15.163:0/664727453 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe8b0007a60 con 0x5618b5d8a480 2023-10-09T06:36:49.814 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.803+0000 7fe8c5bf1b80 1 -- 172.21.15.163:0/664727453 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8a8019070 con 0x5618b5d8a480 2023-10-09T06:36:49.814 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.803+0000 7fe894ff9700 1 -- 172.21.15.163:0/664727453 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8b0007e30 con 0x5618b5d8a480 2023-10-09T06:36:49.814 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.803+0000 7fe8c5bf1b80 10 client.6975 fetch_fsmap learned FSMap version 66 2023-10-09T06:36:49.814 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.803+0000 7fe8c5bf1b80 1 -- 172.21.15.163:0/664727453 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x5618b5c45470 con 0x5618b5d8a480 2023-10-09T06:36:49.814 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.803+0000 7fe894ff9700 1 -- 172.21.15.163:0/664727453 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a8019070 con 0x5618b5d8a480 2023-10-09T06:36:49.814 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.803+0000 7fe894ff9700 1 -- 172.21.15.163:0/664727453 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 66) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe8b0015450 con 0x5618b5d8a480 2023-10-09T06:36:49.815 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.803+0000 7fe8c5bf1b80 10 client.6975 fetch_fsmap finished waiting for FSMap version 66 2023-10-09T06:36:49.815 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.803+0000 7fe8c5bf1b80 10 client.6975 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-09T06:36:49.815 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.803+0000 7fe8c5bf1b80 20 client.6975 populate_metadata read hostname 'smithi163' 2023-10-09T06:36:49.815 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.803+0000 7fe8c5bf1b80 1 --2- 172.21.15.163:0/664727453 >> [v2:172.21.15.163:6833/1684208805,v1:172.21.15.163:6835/1684208805] conn(0x5618b5dec390 0x5618b5d97a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.815 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.803+0000 7fe8c5bf1b80 4 client.6975 mds_command: new command op to 6834 tid=0[{"prefix": "status"}] 2023-10-09T06:36:49.815 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.803+0000 7fe8c5bf1b80 1 -- 172.21.15.163:0/664727453 --> [v2:172.21.15.163:6833/1684208805,v1:172.21.15.163:6835/1684208805] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5618b5c32e00 con 0x5618b5dec390 2023-10-09T06:36:49.815 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.803+0000 7fe8b659c700 1 --2- 172.21.15.163:0/664727453 >> [v2:172.21.15.163:6833/1684208805,v1:172.21.15.163:6835/1684208805] conn(0x5618b5dec390 0x5618b5d97a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.815 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.804+0000 7fe894ff9700 10 client.6975 ms_handle_connect on v2:172.21.15.163:6833/1684208805 2023-10-09T06:36:49.815 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.804+0000 7fe8b659c700 1 --2- 172.21.15.163:0/664727453 >> [v2:172.21.15.163:6833/1684208805,v1:172.21.15.163:6835/1684208805] conn(0x5618b5dec390 0x5618b5d97a90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f4dbe58d4e0f27d server_cookie=aa2d8806fdc7a2c6 in_seq=0 out_seq=0 2023-10-09T06:36:49.815 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.804+0000 7fe894ff9700 1 -- 172.21.15.163:0/664727453 <== mds.0 v2:172.21.15.163:6833/1684208805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x5618b5c32e00 con 0x5618b5dec390 2023-10-09T06:36:49.815 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.804+0000 7fe894ff9700 10 client.6975 handle_command_reply: tid=0 2023-10-09T06:36:49.816 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.804+0000 7fe8c5bf1b80 1 client.6975 shutdown 2023-10-09T06:36:49.816 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.804+0000 7fe8c5bf1b80 1 -- 172.21.15.163:0/664727453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618b5d8a480 msgr2=0x5618b5d6a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:49.816 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.804+0000 7fe8c5bf1b80 1 --2- 172.21.15.163:0/664727453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618b5d8a480 0x5618b5d6a470 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fe8b815dd40 tx=0x7fe8b000b040 comp rx=0 tx=0).stop 2023-10-09T06:36:49.816 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.804+0000 7fe8c5bf1b80 1 -- 172.21.15.163:0/664727453 shutdown_connections 2023-10-09T06:36:49.816 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.804+0000 7fe8c5bf1b80 1 --2- 172.21.15.163:0/664727453 >> [v2:172.21.15.163:6833/1684208805,v1:172.21.15.163:6835/1684208805] conn(0x5618b5dec390 0x5618b5d97a90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.816 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.804+0000 7fe8c5bf1b80 1 --2- 172.21.15.163:0/664727453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5618b5d8a480 0x5618b5d6a470 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.816 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.804+0000 7fe8c5bf1b80 1 --2- 172.21.15.163:0/664727453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5618b5d3d5f0 0x5618b5deb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.816 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.804+0000 7fe894ff9700 0 client.6975 ms_handle_reset on v2:172.21.15.163:6833/1684208805 2023-10-09T06:36:49.816 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.804+0000 7fe8c5bf1b80 1 --2- 172.21.15.163:0/664727453 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5618b5d3fb80 0x5618b5d69f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.816 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.804+0000 7fe8c5bf1b80 1 -- 172.21.15.163:0/664727453 >> 172.21.15.163:0/664727453 conn(0x5618b5d410c0 msgr2=0x5618b5d3ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:49.816 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.804+0000 7fe8c5bf1b80 1 -- 172.21.15.163:0/664727453 shutdown_connections 2023-10-09T06:36:49.816 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.804+0000 7fe8c5bf1b80 1 -- 172.21.15.163:0/664727453 wait complete. 2023-10-09T06:36:49.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.804+0000 7fe8c5bf1b80 20 client.6975 trim_cache size 0 max 16384 2023-10-09T06:36:49.817 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:36:49.817 INFO:teuthology.orchestra.run.smithi163.stdout: "cluster_fsid": "2520b8f7-539f-4bae-9f17-1fce66f66ddd", 2023-10-09T06:36:49.817 INFO:teuthology.orchestra.run.smithi163.stdout: "whoami": 0, 2023-10-09T06:36:49.817 INFO:teuthology.orchestra.run.smithi163.stdout: "id": 6834, 2023-10-09T06:36:49.817 INFO:teuthology.orchestra.run.smithi163.stdout: "want_state": "up:active", 2023-10-09T06:36:49.817 INFO:teuthology.orchestra.run.smithi163.stdout: "state": "up:active", 2023-10-09T06:36:49.818 INFO:teuthology.orchestra.run.smithi163.stdout: "fs_name": "cephfs", 2023-10-09T06:36:49.818 INFO:teuthology.orchestra.run.smithi163.stdout: "rank_uptime": 5.8075698999999998, 2023-10-09T06:36:49.818 INFO:teuthology.orchestra.run.smithi163.stdout: "mdsmap_epoch": 66, 2023-10-09T06:36:49.818 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch": 101, 2023-10-09T06:36:49.818 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch_barrier": 101, 2023-10-09T06:36:49.818 INFO:teuthology.orchestra.run.smithi163.stdout: "uptime": 10.924678826999999 2023-10-09T06:36:49.818 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:36:49.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.805+0000 7fe8b759e700 1 -- 172.21.15.163:0/2963592524 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe898067290 msgr2=0x7fe898069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:49.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.805+0000 7fe8b759e700 1 --2- 172.21.15.163:0/2963592524 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe898067290 0x7fe898069740 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8006fd0 tx=0x7fe8a8008040 comp rx=0 tx=0).stop 2023-10-09T06:36:49.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.805+0000 7fe8b759e700 1 -- 172.21.15.163:0/2963592524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8b81597f0 msgr2=0x7fe8b8155e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:49.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.805+0000 7fe8b759e700 1 --2- 172.21.15.163:0/2963592524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8b81597f0 0x7fe8b8155e60 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8089340 tx=0x7fe8a0016000 comp rx=0 tx=0).stop 2023-10-09T06:36:49.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.805+0000 7fe8b759e700 1 -- 172.21.15.163:0/2963592524 shutdown_connections 2023-10-09T06:36:49.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.805+0000 7fe8b759e700 1 --2- 172.21.15.163:0/2963592524 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe898067290 0x7fe898069740 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.805+0000 7fe8b759e700 1 --2- 172.21.15.163:0/2963592524 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8b815d230 0x7fe8b81563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.805+0000 7fe8b759e700 1 --2- 172.21.15.163:0/2963592524 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8b8159420 0x7fe8b8154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.805+0000 7fe8b759e700 1 --2- 172.21.15.163:0/2963592524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8b81597f0 0x7fe8b8155e60 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.805+0000 7fe8b759e700 1 -- 172.21.15.163:0/2963592524 >> 172.21.15.163:0/2963592524 conn(0x7fe8b80d0660 msgr2=0x7fe8b8155570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:49.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.805+0000 7fe8b759e700 1 -- 172.21.15.163:0/2963592524 shutdown_connections 2023-10-09T06:36:49.822 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.805+0000 7fe8b759e700 1 -- 172.21.15.163:0/2963592524 wait complete. 2023-10-09T06:36:49.822 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-09T06:36:49.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.936+0000 7f32eb855700 1 Processor -- start 2023-10-09T06:36:49.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.936+0000 7f32eb855700 1 -- start start 2023-10-09T06:36:49.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.937+0000 7f32eb855700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f32e4159890 0x7f32e415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.937+0000 7f32eb855700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f32e4159c60 0x7f32e415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.937+0000 7f32eb855700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32e415db10 0x7f32e4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.937+0000 7f32eb855700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32e415df50 con 0x7f32e4159c60 2023-10-09T06:36:49.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.937+0000 7f32eb855700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32e415a160 con 0x7f32e4159890 2023-10-09T06:36:49.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.937+0000 7f32eb855700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e40d13a0 con 0x7f32e415db10 2023-10-09T06:36:49.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.937+0000 7f32e8df0700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f32e4159c60 0x7f32e415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.937+0000 7f32e8df0700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f32e4159c60 0x7f32e415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46164/0 (socket says 172.21.15.163:46164) 2023-10-09T06:36:49.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.937+0000 7f32e8df0700 1 -- 172.21.15.163:0/3205293548 learned_addr learned my addr 172.21.15.163:0/3205293548 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:49.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.937+0000 7f32e95f1700 1 --2- 172.21.15.163:0/3205293548 >> v2:172.21.15.181:3300/0 conn(0x7f32e4159890 0x7f32e415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.937+0000 7f32e9df2700 1 --2- 172.21.15.163:0/3205293548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32e415db10 0x7f32e4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.937+0000 7f32e8df0700 1 -- 172.21.15.163:0/3205293548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32e415db10 msgr2=0x7f32e4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:49.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.937+0000 7f32e8df0700 1 --2- 172.21.15.163:0/3205293548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32e415db10 0x7f32e4069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.937+0000 7f32e8df0700 1 -- 172.21.15.163:0/3205293548 >> v2:172.21.15.181:3300/0 conn(0x7f32e4159890 msgr2=0x7f32e415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:49.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.937+0000 7f32e8df0700 1 --2- 172.21.15.163:0/3205293548 >> v2:172.21.15.181:3300/0 conn(0x7f32e4159890 0x7f32e415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.937+0000 7f32e8df0700 1 -- 172.21.15.163:0/3205293548 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e415e140 con 0x7f32e4159c60 2023-10-09T06:36:49.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.937+0000 7f32e8df0700 1 --2- 172.21.15.163:0/3205293548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32e4159c60 0x7f32e415d5d0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f32e40d1090 tx=0x7f32d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316ebbe8f42b425b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:49.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.938+0000 7f32dbfff700 1 -- 172.21.15.163:0/3205293548 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d400b640 con 0x7f32e4159c60 2023-10-09T06:36:49.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.938+0000 7f32dbfff700 1 -- 172.21.15.163:0/3205293548 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d400b7a0 con 0x7f32e4159c60 2023-10-09T06:36:49.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.938+0000 7f32dbfff700 1 -- 172.21.15.163:0/3205293548 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d400bb60 con 0x7f32e4159c60 2023-10-09T06:36:49.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.938+0000 7f32eb855700 1 -- 172.21.15.163:0/3205293548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32e4159c60 msgr2=0x7f32e415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:49.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.938+0000 7f32eb855700 1 --2- 172.21.15.163:0/3205293548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32e4159c60 0x7f32e415d5d0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f32e40d1090 tx=0x7f32d4009250 comp rx=0 tx=0).stop 2023-10-09T06:36:49.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.938+0000 7f32eb855700 1 -- 172.21.15.163:0/3205293548 shutdown_connections 2023-10-09T06:36:49.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.938+0000 7f32eb855700 1 --2- 172.21.15.163:0/3205293548 >> v2:172.21.15.181:3300/0 conn(0x7f32e4159890 0x7f32e415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.938+0000 7f32eb855700 1 --2- 172.21.15.163:0/3205293548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32e415db10 0x7f32e4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.938+0000 7f32eb855700 1 --2- 172.21.15.163:0/3205293548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32e4159c60 0x7f32e415d5d0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.938+0000 7f32eb855700 1 -- 172.21.15.163:0/3205293548 >> 172.21.15.163:0/3205293548 conn(0x7f32e4068e20 msgr2=0x7f32e40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:49.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.938+0000 7f32eb855700 1 -- 172.21.15.163:0/3205293548 shutdown_connections 2023-10-09T06:36:49.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.938+0000 7f32eb855700 1 -- 172.21.15.163:0/3205293548 wait complete. 2023-10-09T06:36:49.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.939+0000 7f32eb855700 1 Processor -- start 2023-10-09T06:36:49.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.939+0000 7f32eb855700 1 -- start start 2023-10-09T06:36:49.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.940+0000 7f32eb855700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32e4159890 0x7f32e40b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.940+0000 7f32eb855700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32e4159c60 0x7f32e40b2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.940+0000 7f32eb855700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32e415db10 0x7f32e40b5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.940+0000 7f32eb855700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e40b05d0 con 0x7f32e4159890 2023-10-09T06:36:49.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.940+0000 7f32eb855700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e40b46a0 con 0x7f32e415db10 2023-10-09T06:36:49.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.940+0000 7f32eb855700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e40b4980 con 0x7f32e4159c60 2023-10-09T06:36:49.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.940+0000 7f32e95f1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32e4159890 0x7f32e40b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.940+0000 7f32e95f1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32e4159890 0x7f32e40b3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46172/0 (socket says 172.21.15.163:46172) 2023-10-09T06:36:49.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.940+0000 7f32e95f1700 1 -- 172.21.15.163:0/3194868750 learned_addr learned my addr 172.21.15.163:0/3194868750 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:49.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.941+0000 7f32e8df0700 1 --2- 172.21.15.163:0/3194868750 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32e4159c60 0x7f32e40b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.941+0000 7f32e9df2700 1 --2- 172.21.15.163:0/3194868750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32e415db10 0x7f32e40b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.941+0000 7f32e95f1700 1 -- 172.21.15.163:0/3194868750 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32e4159c60 msgr2=0x7f32e40b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:49.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.941+0000 7f32e95f1700 1 --2- 172.21.15.163:0/3194868750 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32e4159c60 0x7f32e40b2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.941+0000 7f32e95f1700 1 -- 172.21.15.163:0/3194868750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32e415db10 msgr2=0x7f32e40b5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:49.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.941+0000 7f32e95f1700 1 --2- 172.21.15.163:0/3194868750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32e415db10 0x7f32e40b5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:49.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.941+0000 7f32e95f1700 1 -- 172.21.15.163:0/3194868750 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d40094f0 con 0x7f32e4159890 2023-10-09T06:36:49.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.941+0000 7f32e8df0700 1 --2- 172.21.15.163:0/3194868750 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32e4159c60 0x7f32e40b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:49.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.941+0000 7f32e95f1700 1 --2- 172.21.15.163:0/3194868750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32e4159890 0x7f32e40b3fa0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f32dc009e50 tx=0x7f32dc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:49.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.941+0000 7f32d97fa700 1 -- 172.21.15.163:0/3194868750 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc00cb50 con 0x7f32e4159890 2023-10-09T06:36:49.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.941+0000 7f32d97fa700 1 -- 172.21.15.163:0/3194868750 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32dc00f040 con 0x7f32e4159890 2023-10-09T06:36:49.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.941+0000 7f32eb855700 1 -- 172.21.15.163:0/3194868750 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32e40b07e0 con 0x7f32e4159890 2023-10-09T06:36:49.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.941+0000 7f32d97fa700 1 -- 172.21.15.163:0/3194868750 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc014480 con 0x7f32e4159890 2023-10-09T06:36:49.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.941+0000 7f32eb855700 1 -- 172.21.15.163:0/3194868750 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e40c8560 con 0x7f32e4159890 2023-10-09T06:36:49.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.942+0000 7f32eb855700 1 -- 172.21.15.163:0/3194868750 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32e40b4ba0 con 0x7f32e4159890 2023-10-09T06:36:49.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.943+0000 7f32d97fa700 1 -- 172.21.15.163:0/3194868750 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f32dc0147f0 con 0x7f32e4159890 2023-10-09T06:36:49.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.943+0000 7f32d97fa700 1 --2- 172.21.15.163:0/3194868750 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f32c8067290 0x7f32c8069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:49.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.943+0000 7f32d97fa700 1 -- 172.21.15.163:0/3194868750 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f32dc0854b0 con 0x7f32e4159890 2023-10-09T06:36:49.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.943+0000 7f32e8df0700 1 --2- 172.21.15.163:0/3194868750 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f32c8067290 0x7f32c8069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:49.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.944+0000 7f32e8df0700 1 --2- 172.21.15.163:0/3194868750 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f32c8067290 0x7f32c8069740 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f32e40d1090 tx=0x7f32d4017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:49.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:49.947+0000 7f32d97fa700 1 -- 172.21.15.163:0/3194868750 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32dc052200 con 0x7f32e4159890 2023-10-09T06:36:50.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.170+0000 7f32eb855700 1 -- 172.21.15.163:0/3194868750 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f32e404c3c0 con 0x7f32e4159890 2023-10-09T06:36:50.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.171+0000 7f32d97fa700 1 -- 172.21.15.163:0/3194868750 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v66) v1 ==== 53+0+73 (secure 0 0 0) 0x7f32dc055820 con 0x7f32e4159890 2023-10-09T06:36:50.173 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-09T06:36:50.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.173+0000 7f32eb855700 1 -- 172.21.15.163:0/3194868750 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f32c8067290 msgr2=0x7f32c8069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:50.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.173+0000 7f32eb855700 1 --2- 172.21.15.163:0/3194868750 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f32c8067290 0x7f32c8069740 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f32e40d1090 tx=0x7f32d4017000 comp rx=0 tx=0).stop 2023-10-09T06:36:50.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.173+0000 7f32eb855700 1 -- 172.21.15.163:0/3194868750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32e4159890 msgr2=0x7f32e40b3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:50.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.173+0000 7f32eb855700 1 --2- 172.21.15.163:0/3194868750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32e4159890 0x7f32e40b3fa0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f32dc009e50 tx=0x7f32dc00b040 comp rx=0 tx=0).stop 2023-10-09T06:36:50.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.173+0000 7f32eb855700 1 -- 172.21.15.163:0/3194868750 shutdown_connections 2023-10-09T06:36:50.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.173+0000 7f32eb855700 1 --2- 172.21.15.163:0/3194868750 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f32c8067290 0x7f32c8069740 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:50.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.173+0000 7f32eb855700 1 --2- 172.21.15.163:0/3194868750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32e4159890 0x7f32e40b3fa0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:50.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.173+0000 7f32eb855700 1 --2- 172.21.15.163:0/3194868750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32e415db10 0x7f32e40b5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:50.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.173+0000 7f32eb855700 1 --2- 172.21.15.163:0/3194868750 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32e4159c60 0x7f32e40b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:50.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.173+0000 7f32eb855700 1 -- 172.21.15.163:0/3194868750 >> 172.21.15.163:0/3194868750 conn(0x7f32e4068e20 msgr2=0x7f32e40cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:50.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.173+0000 7f32eb855700 1 -- 172.21.15.163:0/3194868750 shutdown_connections 2023-10-09T06:36:50.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.173+0000 7f32eb855700 1 -- 172.21.15.163:0/3194868750 wait complete. 2023-10-09T06:36:50.186 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-09T06:36:50.186 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-09T06:36:50.186 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi163.front.sepia.ceph.com 2023-10-09T06:36:50.187 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-09T06:36:50.187 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-09T06:36:50.187 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-09T06:36:50.187 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-09T06:36:50.187 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-09T06:36:50.187 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:36:50.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:50.187 DEBUG:teuthology.orchestra.run.smithi163:> ip addr 2023-10-09T06:36:50.203 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:36:50.203 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:36:50.203 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:36:50.203 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:50.203 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2023-10-09T06:36:50.203 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:50.203 INFO:teuthology.orchestra.run.smithi163.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:36:50.203 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:06 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:36:50.203 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f0 2023-10-09T06:36:50.203 INFO:teuthology.orchestra.run.smithi163.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:36:50.204 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:07 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:36:50.204 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f1 2023-10-09T06:36:50.204 INFO:teuthology.orchestra.run.smithi163.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:36:50.204 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5a brd ff:ff:ff:ff:ff:ff 2023-10-09T06:36:50.204 INFO:teuthology.orchestra.run.smithi163.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-09T06:36:50.204 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5b brd ff:ff:ff:ff:ff:ff 2023-10-09T06:36:50.204 INFO:teuthology.orchestra.run.smithi163.stdout: inet 172.21.15.163/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-09T06:36:50.204 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft 42113sec preferred_lft 42113sec 2023-10-09T06:36:50.204 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::ec4:7aff:fe88:7d5b/64 scope link 2023-10-09T06:36:50.204 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:50.204 INFO:teuthology.orchestra.run.smithi163.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-09T06:36:50.205 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 2e:01:84:b9:f4:8e brd ff:ff:ff:ff:ff:ff 2023-10-09T06:36:50.205 INFO:teuthology.orchestra.run.smithi163.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-09T06:36:50.205 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:50.205 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::d03c:93ff:fe9e:c8cb/64 scope link 2023-10-09T06:36:50.205 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:50.205 INFO:teuthology.orchestra.run.smithi163.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:36:50.205 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 2e:01:84:b9:f4:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-09T06:36:50.205 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::2c01:84ff:feb9:f48e/64 scope link 2023-10-09T06:36:50.205 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:50.205 INFO:teuthology.orchestra.run.smithi163.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:36:50.205 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether c6:88:3e:d7:c1:56 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-09T06:36:50.205 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::c488:3eff:fed7:c156/64 scope link 2023-10-09T06:36:50.206 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:50.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:50.206 DEBUG:teuthology.orchestra.run.smithi163:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-09T06:36:50.266 INFO:teuthology.orchestra.run.smithi163.stdout:1 2023-10-09T06:36:50.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:50.270 DEBUG:teuthology.orchestra.run.smithi163:> route 2023-10-09T06:36:50.327 INFO:teuthology.orchestra.run.smithi163.stdout:Kernel IP routing table 2023-10-09T06:36:50.327 INFO:teuthology.orchestra.run.smithi163.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-09T06:36:50.327 INFO:teuthology.orchestra.run.smithi163.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-09T06:36:50.327 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-09T06:36:50.327 INFO:teuthology.orchestra.run.smithi163.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-09T06:36:50.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:50.328 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-09T06:36:50.328 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-09T06:36:50.328 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-09T06:36:50.329 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-09T06:36:50.329 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-09T06:36:50.329 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-09T06:36:50.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:50.427 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2023-10-09T06:36:50.441 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-09T06:36:50.442 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-09T06:36:50.442 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:36:50.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:50.442 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set brx.0 up 2023-10-09T06:36:50.506 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:36:50.506 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-09T06:36:50.561 INFO:teuthology.orchestra.run.smithi163.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:36:50.562 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:36:50.562 DEBUG:teuthology.orchestra.run.smithi163:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-09T06:36:50.616 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-09T06:36:50.616 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-09T06:36:50.616 DEBUG:teuthology.orchestra.run.smithi163:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:36:50.696 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:36:50.696 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:36:50.714 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:36:50.715 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:36:50.715 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:36:50.715 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with new device syntax: 0@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:36:50.715 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:36:50.715 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:36:50.715 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:36:50.715 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:36:50.736 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:36:50.737 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:36:50.737 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:36:50.737 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with new device syntax: 0@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:36:50.737 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:36:50.737 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:36:50.737 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:36:50.737 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:36:50.737 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790" 2023-10-09T06:36:50.737 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790:/ 2023-10-09T06:36:50.737 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=2520b8f7-539f-4bae-9f17-1fce66f66ddd" will pass to kernel 2023-10-09T06:36:50.739 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-09T06:36:50.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:50.739 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-09T06:36:50.804 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-09T06:36:50.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.959+0000 7f5276501700 1 Processor -- start 2023-10-09T06:36:50.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.960+0000 7f5276501700 1 -- start start 2023-10-09T06:36:50.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.960+0000 7f5276501700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5270159890 0x7f527015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:50.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.960+0000 7f5276501700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f527015a160 con 0x7f5270159c60 2023-10-09T06:36:50.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.960+0000 7f5276501700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52700d13a0 con 0x7f5270069560 2023-10-09T06:36:50.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.960+0000 7f5276501700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f527015e140 con 0x7f5270159890 2023-10-09T06:36:50.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.960+0000 7f526ffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5270159890 0x7f527015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:50.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.960+0000 7f526f7fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f5270159c60 0x7f527015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:54330/0 (socket says 172.21.15.163:54330) 2023-10-09T06:36:50.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.961+0000 7f526f7fe700 1 -- 172.21.15.163:0/3538901438 learned_addr learned my addr 172.21.15.163:0/3538901438 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:50.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.961+0000 7f526ffff700 1 -- 172.21.15.163:0/3538901438 >> v1:172.21.15.181:6789/0 conn(0x7f5270069560 legacy=0x7f52700ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:50.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.961+0000 7f526ffff700 1 -- 172.21.15.163:0/3538901438 >> v1:172.21.15.163:6789/0 conn(0x7f5270159c60 legacy=0x7f527015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:50.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.961+0000 7f526ffff700 1 -- 172.21.15.163:0/3538901438 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527015a3d0 con 0x7f5270159890 2023-10-09T06:36:50.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.961+0000 7f526effd700 1 -- 172.21.15.163:0/3538901438 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 802028172 0 0) 0x7f527015a160 con 0x7f5270159c60 2023-10-09T06:36:50.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.961+0000 7f526ffff700 1 --2- 172.21.15.163:0/3538901438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5270159890 0x7f527015abd0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f52600098e0 tx=0x7f52600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf639b30ab613b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:50.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.961+0000 7f526effd700 1 -- 172.21.15.163:0/3538901438 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5260009e80 con 0x7f5270159890 2023-10-09T06:36:50.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.961+0000 7f526effd700 1 -- 172.21.15.163:0/3538901438 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526000b820 con 0x7f5270159890 2023-10-09T06:36:50.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.961+0000 7f526effd700 1 -- 172.21.15.163:0/3538901438 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526000bba0 con 0x7f5270159890 2023-10-09T06:36:50.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.962+0000 7f5276501700 1 -- 172.21.15.163:0/3538901438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5270159890 msgr2=0x7f527015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:50.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.962+0000 7f5276501700 1 --2- 172.21.15.163:0/3538901438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5270159890 0x7f527015abd0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f52600098e0 tx=0x7f52600091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:50.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.962+0000 7f5276501700 1 -- 172.21.15.163:0/3538901438 shutdown_connections 2023-10-09T06:36:50.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.962+0000 7f5276501700 1 --2- 172.21.15.163:0/3538901438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5270159890 0x7f527015abd0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:50.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.962+0000 7f5276501700 1 -- 172.21.15.163:0/3538901438 >> 172.21.15.163:0/3538901438 conn(0x7f5270068e20 msgr2=0x7f52700d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:50.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.962+0000 7f5276501700 1 -- 172.21.15.163:0/3538901438 shutdown_connections 2023-10-09T06:36:50.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.962+0000 7f5276501700 1 -- 172.21.15.163:0/3538901438 wait complete. 2023-10-09T06:36:50.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.963+0000 7f5276501700 1 Processor -- start 2023-10-09T06:36:50.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.963+0000 7f5276501700 1 -- start start 2023-10-09T06:36:50.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.964+0000 7f5276501700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5270069560 0x7f527014d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:50.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.964+0000 7f5276501700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5270159890 0x7f527015d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:50.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.964+0000 7f5276501700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5270159c60 0x7f52700ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:50.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.964+0000 7f5276501700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52700b4850 con 0x7f5270159890 2023-10-09T06:36:50.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.964+0000 7f5276501700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52700b4ac0 con 0x7f5270069560 2023-10-09T06:36:50.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.964+0000 7f5276501700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52700b4da0 con 0x7f5270159c60 2023-10-09T06:36:50.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.964+0000 7f526f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5270159890 0x7f527015d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:50.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.964+0000 7f526f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5270159890 0x7f527015d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46202/0 (socket says 172.21.15.163:46202) 2023-10-09T06:36:50.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.964+0000 7f526f7fe700 1 -- 172.21.15.163:0/2023738291 learned_addr learned my addr 172.21.15.163:0/2023738291 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:50.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.964+0000 7f526ffff700 1 --2- 172.21.15.163:0/2023738291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5270069560 0x7f527014d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:50.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.964+0000 7f5274a9e700 1 --2- 172.21.15.163:0/2023738291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5270159c60 0x7f52700ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:50.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.964+0000 7f526f7fe700 1 -- 172.21.15.163:0/2023738291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5270159c60 msgr2=0x7f52700ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:50.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.964+0000 7f526f7fe700 1 --2- 172.21.15.163:0/2023738291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5270159c60 0x7f52700ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:50.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.964+0000 7f526f7fe700 1 -- 172.21.15.163:0/2023738291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5270069560 msgr2=0x7f527014d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:50.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.964+0000 7f526f7fe700 1 --2- 172.21.15.163:0/2023738291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5270069560 0x7f527014d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:50.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.964+0000 7f526f7fe700 1 -- 172.21.15.163:0/2023738291 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5260009420 con 0x7f5270159890 2023-10-09T06:36:50.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.964+0000 7f5274a9e700 1 --2- 172.21.15.163:0/2023738291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5270159c60 0x7f52700ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:50.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.964+0000 7f526f7fe700 1 --2- 172.21.15.163:0/2023738291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5270159890 0x7f527015d810 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f526400c7e0 tx=0x7f5264007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:50.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.964+0000 7f5257fff700 1 -- 172.21.15.163:0/2023738291 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5264005b30 con 0x7f5270159890 2023-10-09T06:36:50.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.964+0000 7f5257fff700 1 -- 172.21.15.163:0/2023738291 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5264005c90 con 0x7f5270159890 2023-10-09T06:36:50.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.964+0000 7f5276501700 1 -- 172.21.15.163:0/2023738291 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52700b5080 con 0x7f5270159890 2023-10-09T06:36:50.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.964+0000 7f5257fff700 1 -- 172.21.15.163:0/2023738291 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5264007300 con 0x7f5270159890 2023-10-09T06:36:50.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.964+0000 7f5276501700 1 -- 172.21.15.163:0/2023738291 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f527014d930 con 0x7f5270159890 2023-10-09T06:36:50.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.965+0000 7f5276501700 1 -- 172.21.15.163:0/2023738291 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52700b55a0 con 0x7f5270159890 2023-10-09T06:36:50.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.966+0000 7f5257fff700 1 -- 172.21.15.163:0/2023738291 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f52640076b0 con 0x7f5270159890 2023-10-09T06:36:50.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.966+0000 7f5257fff700 1 --2- 172.21.15.163:0/2023738291 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f525c067290 0x7f525c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:50.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.966+0000 7f5257fff700 1 -- 172.21.15.163:0/2023738291 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5264084020 con 0x7f5270159890 2023-10-09T06:36:50.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.966+0000 7f526ffff700 1 --2- 172.21.15.163:0/2023738291 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f525c067290 0x7f525c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:50.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.967+0000 7f526ffff700 1 --2- 172.21.15.163:0/2023738291 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f525c067290 0x7f525c069740 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f52600030c0 tx=0x7f5260016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:50.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:50.970+0000 7f5257fff700 1 -- 172.21.15.163:0/2023738291 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f52700b55a0 con 0x7f5270159890 2023-10-09T06:36:51.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:51.196+0000 7f5276501700 1 -- 172.21.15.163:0/2023738291 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f52700b55a0 con 0x7f5270159890 2023-10-09T06:36:51.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:51.197+0000 7f5257fff700 1 -- 172.21.15.163:0/2023738291 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v66) v1 ==== 53+0+73 (secure 0 0 0) 0x7f52700b55a0 con 0x7f5270159890 2023-10-09T06:36:51.198 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-09T06:36:51.211 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:51.199+0000 7f5276501700 1 -- 172.21.15.163:0/2023738291 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f525c067290 msgr2=0x7f525c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:51.211 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:51.199+0000 7f5276501700 1 --2- 172.21.15.163:0/2023738291 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f525c067290 0x7f525c069740 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f52600030c0 tx=0x7f5260016000 comp rx=0 tx=0).stop 2023-10-09T06:36:51.211 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:51.199+0000 7f5276501700 1 -- 172.21.15.163:0/2023738291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5270159890 msgr2=0x7f527015d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:51.211 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:51.199+0000 7f5276501700 1 --2- 172.21.15.163:0/2023738291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5270159890 0x7f527015d810 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f526400c7e0 tx=0x7f5264007df0 comp rx=0 tx=0).stop 2023-10-09T06:36:51.211 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:51.199+0000 7f5276501700 1 -- 172.21.15.163:0/2023738291 shutdown_connections 2023-10-09T06:36:51.211 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:51.199+0000 7f5276501700 1 --2- 172.21.15.163:0/2023738291 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f525c067290 0x7f525c069740 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:51.211 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:51.199+0000 7f5276501700 1 --2- 172.21.15.163:0/2023738291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5270159c60 0x7f52700ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:51.211 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:51.199+0000 7f5276501700 1 --2- 172.21.15.163:0/2023738291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5270069560 0x7f527014d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:51.212 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:51.199+0000 7f5276501700 1 --2- 172.21.15.163:0/2023738291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5270159890 0x7f527015d810 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:51.212 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:51.199+0000 7f5276501700 1 -- 172.21.15.163:0/2023738291 >> 172.21.15.163:0/2023738291 conn(0x7f5270068e20 msgr2=0x7f52700cf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:51.212 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:51.199+0000 7f5276501700 1 -- 172.21.15.163:0/2023738291 shutdown_connections 2023-10-09T06:36:51.212 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:51.199+0000 7f5276501700 1 -- 172.21.15.163:0/2023738291 wait complete. 2023-10-09T06:36:51.213 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-09T06:36:51.213 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-09T06:36:51.213 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-09T06:36:51.213 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-09T06:36:51.213 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-09T06:36:51.213 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-09T06:36:51.213 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-09T06:36:51.213 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-09T06:36:51.213 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:36:51.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:51.214 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-09T06:36:51.242 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:36:51.242 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:36:51.242 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:36:51.242 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:51.242 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-09T06:36:51.242 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:51.242 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:36:51.242 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:36:51.243 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2023-10-09T06:36:51.243 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:36:51.243 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:36:51.243 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2023-10-09T06:36:51.243 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:36:51.243 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-09T06:36:51.243 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-09T06:36:51.243 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-09T06:36:51.243 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-09T06:36:51.243 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42089sec preferred_lft 42089sec 2023-10-09T06:36:51.243 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2023-10-09T06:36:51.243 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:51.244 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-09T06:36:51.244 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 72:62:e6:72:7c:2e brd ff:ff:ff:ff:ff:ff 2023-10-09T06:36:51.244 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-09T06:36:51.244 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:51.244 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::e89a:4cff:fefb:7198/64 scope link 2023-10-09T06:36:51.244 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:51.244 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:36:51.244 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 72:62:e6:72:7c:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-09T06:36:51.244 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::7062:e6ff:fe72:7c2e/64 scope link 2023-10-09T06:36:51.244 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:51.244 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:36:51.245 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether fa:4e:a3:9d:5a:5f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-09T06:36:51.245 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::f84e:a3ff:fe9d:5a5f/64 scope link 2023-10-09T06:36:51.245 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:36:51.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:51.245 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-09T06:36:51.314 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-09T06:36:51.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:51.316 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-09T06:36:51.386 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-09T06:36:51.386 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-09T06:36:51.386 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-09T06:36:51.386 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-09T06:36:51.386 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-09T06:36:51.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:51.387 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-09T06:36:51.387 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-09T06:36:51.387 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-09T06:36:51.387 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-09T06:36:51.387 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-09T06:36:51.387 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-09T06:36:51.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:51.500 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-09T06:36:51.516 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-09T06:36:51.516 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-09T06:36:51.517 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:36:51.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:51.518 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-09T06:36:51.587 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:36:51.587 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-09T06:36:51.648 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:36:51.648 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:36:51.649 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-09T06:36:51.707 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-09T06:36:51.707 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-09T06:36:51.708 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:36:51.792 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:36:51.792 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:36:51.811 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:36:51.812 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:36:51.812 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:36:51.812 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 1@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:36:51.812 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:36:51.812 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:36:51.812 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:36:51.812 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:36:51.833 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:36:51.833 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:36:51.833 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:36:51.834 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 1@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:36:51.834 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:36:51.834 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:36:51.834 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:36:51.834 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:36:51.834 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790" 2023-10-09T06:36:51.834 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790:/ 2023-10-09T06:36:51.834 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=2520b8f7-539f-4bae-9f17-1fce66f66ddd" will pass to kernel 2023-10-09T06:36:51.835 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-09T06:36:51.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:51.836 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-09T06:36:51.901 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-09T06:36:52.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.024+0000 7ff6c596f700 1 Processor -- start 2023-10-09T06:36:52.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.024+0000 7ff6c596f700 1 -- start start 2023-10-09T06:36:52.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.025+0000 7ff6c596f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff6c0159ec0 0x7ff6c00b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:52.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.025+0000 7ff6c596f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6c00b5b00 con 0x7ff6c00b53d0 2023-10-09T06:36:52.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.025+0000 7ff6c596f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6c00b5850 con 0x7ff6c00cf350 2023-10-09T06:36:52.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.025+0000 7ff6c596f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6c00b5da0 con 0x7ff6c0159ec0 2023-10-09T06:36:52.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.025+0000 7ff6be7fc700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff6c00b53d0 0x7ff6c00cc900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:54332/0 (socket says 172.21.15.163:54332) 2023-10-09T06:36:52.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.025+0000 7ff6be7fc700 1 -- 172.21.15.163:0/3574509986 learned_addr learned my addr 172.21.15.163:0/3574509986 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:52.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.025+0000 7ff6beffd700 1 --2- 172.21.15.163:0/3574509986 >> v2:172.21.15.181:3300/0 conn(0x7ff6c0159ec0 0x7ff6c00b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:52.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.025+0000 7ff6bdffb700 1 -- 172.21.15.163:0/3574509986 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4043588889 0 0) 0x7ff6c00b5b00 con 0x7ff6c00b53d0 2023-10-09T06:36:52.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.025+0000 7ff6bdffb700 1 -- 172.21.15.163:0/3574509986 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6a8003570 con 0x7ff6c00b53d0 2023-10-09T06:36:52.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.026+0000 7ff6bdffb700 1 -- 172.21.15.163:0/3574509986 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3389953515 0 0) 0x7ff6a8003570 con 0x7ff6c00b53d0 2023-10-09T06:36:52.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.026+0000 7ff6beffd700 1 -- 172.21.15.163:0/3574509986 >> v1:172.21.15.181:6789/0 conn(0x7ff6c00cf350 legacy=0x7ff6c00cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:52.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.026+0000 7ff6beffd700 1 -- 172.21.15.163:0/3574509986 >> v1:172.21.15.163:6789/0 conn(0x7ff6c00b53d0 legacy=0x7ff6c00cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:52.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.026+0000 7ff6beffd700 1 -- 172.21.15.163:0/3574509986 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c00b6080 con 0x7ff6c0159ec0 2023-10-09T06:36:52.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.026+0000 7ff6beffd700 1 --2- 172.21.15.163:0/3574509986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6c0159ec0 0x7ff6c00b4e90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0009eb0 tx=0x7ff6b0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bcb9e833afa8145 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:52.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.026+0000 7ff6bdffb700 1 -- 172.21.15.163:0/3574509986 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b000e070 con 0x7ff6c0159ec0 2023-10-09T06:36:52.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.026+0000 7ff6bdffb700 1 -- 172.21.15.163:0/3574509986 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b000b920 con 0x7ff6c0159ec0 2023-10-09T06:36:52.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.026+0000 7ff6bdffb700 1 -- 172.21.15.163:0/3574509986 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4001470 con 0x7ff6c0159ec0 2023-10-09T06:36:52.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.026+0000 7ff6c596f700 1 -- 172.21.15.163:0/3574509986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6c0159ec0 msgr2=0x7ff6c00b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:52.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.026+0000 7ff6c596f700 1 --2- 172.21.15.163:0/3574509986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6c0159ec0 0x7ff6c00b4e90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0009eb0 tx=0x7ff6b0009780 comp rx=0 tx=0).stop 2023-10-09T06:36:52.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.026+0000 7ff6c596f700 1 -- 172.21.15.163:0/3574509986 shutdown_connections 2023-10-09T06:36:52.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.026+0000 7ff6c596f700 1 --2- 172.21.15.163:0/3574509986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6c0159ec0 0x7ff6c00b4e90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:52.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.026+0000 7ff6c596f700 1 -- 172.21.15.163:0/3574509986 >> 172.21.15.163:0/3574509986 conn(0x7ff6c00999f0 msgr2=0x7ff6c00b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:52.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.026+0000 7ff6c596f700 1 -- 172.21.15.163:0/3574509986 shutdown_connections 2023-10-09T06:36:52.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.026+0000 7ff6c596f700 1 -- 172.21.15.163:0/3574509986 wait complete. 2023-10-09T06:36:52.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.028+0000 7ff6c596f700 1 Processor -- start 2023-10-09T06:36:52.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.028+0000 7ff6c596f700 1 -- start start 2023-10-09T06:36:52.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.028+0000 7ff6c596f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6c00b53d0 0x7ff6c00c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:52.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.028+0000 7ff6c596f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6c00cf350 0x7ff6c009ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:52.034 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.028+0000 7ff6c596f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6c0159ec0 0x7ff6c00690d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:52.034 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.028+0000 7ff6c596f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c009a460 con 0x7ff6c00cf350 2023-10-09T06:36:52.034 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.028+0000 7ff6c596f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c00c9a90 con 0x7ff6c00b53d0 2023-10-09T06:36:52.034 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.028+0000 7ff6c596f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c014e8e0 con 0x7ff6c0159ec0 2023-10-09T06:36:52.034 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.028+0000 7ff6be7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6c00cf350 0x7ff6c009ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:52.034 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.029+0000 7ff6be7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6c00cf350 0x7ff6c009ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46204/0 (socket says 172.21.15.163:46204) 2023-10-09T06:36:52.034 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.029+0000 7ff6be7fc700 1 -- 172.21.15.163:0/749976813 learned_addr learned my addr 172.21.15.163:0/749976813 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:52.034 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.029+0000 7ff6beffd700 1 --2- 172.21.15.163:0/749976813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6c00b53d0 0x7ff6c00c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:52.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.029+0000 7ff6bf7fe700 1 --2- 172.21.15.163:0/749976813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6c0159ec0 0x7ff6c00690d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:52.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.029+0000 7ff6be7fc700 1 -- 172.21.15.163:0/749976813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6c0159ec0 msgr2=0x7ff6c00690d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:52.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.029+0000 7ff6be7fc700 1 --2- 172.21.15.163:0/749976813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6c0159ec0 0x7ff6c00690d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:52.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.029+0000 7ff6be7fc700 1 -- 172.21.15.163:0/749976813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6c00b53d0 msgr2=0x7ff6c00c9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:52.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.029+0000 7ff6be7fc700 1 --2- 172.21.15.163:0/749976813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6c00b53d0 0x7ff6c00c9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:52.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.029+0000 7ff6be7fc700 1 -- 172.21.15.163:0/749976813 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b0009a20 con 0x7ff6c00cf350 2023-10-09T06:36:52.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.029+0000 7ff6bf7fe700 1 --2- 172.21.15.163:0/749976813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6c0159ec0 0x7ff6c00690d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:52.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.029+0000 7ff6be7fc700 1 --2- 172.21.15.163:0/749976813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6c00cf350 0x7ff6c009ae70 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400c7f0 tx=0x7ff6b4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:52.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.029+0000 7ff6a6ffd700 1 -- 172.21.15.163:0/749976813 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4007360 con 0x7ff6c00cf350 2023-10-09T06:36:52.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.029+0000 7ff6a6ffd700 1 -- 172.21.15.163:0/749976813 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b40074e0 con 0x7ff6c00cf350 2023-10-09T06:36:52.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.029+0000 7ff6c596f700 1 -- 172.21.15.163:0/749976813 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c009b3b0 con 0x7ff6c00cf350 2023-10-09T06:36:52.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.029+0000 7ff6a6ffd700 1 -- 172.21.15.163:0/749976813 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4004830 con 0x7ff6c00cf350 2023-10-09T06:36:52.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.029+0000 7ff6c596f700 1 -- 172.21.15.163:0/749976813 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c009a990 con 0x7ff6c00cf350 2023-10-09T06:36:52.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.030+0000 7ff6c596f700 1 -- 172.21.15.163:0/749976813 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6c0063c90 con 0x7ff6c00cf350 2023-10-09T06:36:52.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.030+0000 7ff6a6ffd700 1 -- 172.21.15.163:0/749976813 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7ff6b4004bc0 con 0x7ff6c00cf350 2023-10-09T06:36:52.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.031+0000 7ff6a6ffd700 1 --2- 172.21.15.163:0/749976813 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff6ac067290 0x7ff6ac069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:52.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.031+0000 7ff6a6ffd700 1 -- 172.21.15.163:0/749976813 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff6b40841c0 con 0x7ff6c00cf350 2023-10-09T06:36:52.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.031+0000 7ff6beffd700 1 --2- 172.21.15.163:0/749976813 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff6ac067290 0x7ff6ac069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:52.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.031+0000 7ff6beffd700 1 --2- 172.21.15.163:0/749976813 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff6ac067290 0x7ff6ac069740 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff6b00099f0 tx=0x7ff6b0017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:52.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.035+0000 7ff6a6ffd700 1 -- 172.21.15.163:0/749976813 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6b4050fa0 con 0x7ff6c00cf350 2023-10-09T06:36:52.262 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.261+0000 7ff6c596f700 1 -- 172.21.15.163:0/749976813 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff6c00c9c00 con 0x7ff6c00cf350 2023-10-09T06:36:52.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.543+0000 7ff6a6ffd700 1 -- 172.21.15.163:0/749976813 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v67) v1 ==== 136+0+0 (secure 0 0 0) 0x7ff6b40545c0 con 0x7ff6c00cf350 2023-10-09T06:36:52.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.545+0000 7ff6c596f700 1 -- 172.21.15.163:0/749976813 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff6ac067290 msgr2=0x7ff6ac069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:52.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.545+0000 7ff6c596f700 1 --2- 172.21.15.163:0/749976813 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff6ac067290 0x7ff6ac069740 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff6b00099f0 tx=0x7ff6b0017000 comp rx=0 tx=0).stop 2023-10-09T06:36:52.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.545+0000 7ff6c596f700 1 -- 172.21.15.163:0/749976813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6c00cf350 msgr2=0x7ff6c009ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:52.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.545+0000 7ff6c596f700 1 --2- 172.21.15.163:0/749976813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6c00cf350 0x7ff6c009ae70 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400c7f0 tx=0x7ff6b4001a00 comp rx=0 tx=0).stop 2023-10-09T06:36:52.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.545+0000 7ff6c596f700 1 -- 172.21.15.163:0/749976813 shutdown_connections 2023-10-09T06:36:52.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.545+0000 7ff6c596f700 1 --2- 172.21.15.163:0/749976813 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff6ac067290 0x7ff6ac069740 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:52.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.545+0000 7ff6c596f700 1 --2- 172.21.15.163:0/749976813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6c0159ec0 0x7ff6c00690d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:52.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.545+0000 7ff6c596f700 1 --2- 172.21.15.163:0/749976813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6c00b53d0 0x7ff6c00c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:52.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.545+0000 7ff6c596f700 1 --2- 172.21.15.163:0/749976813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6c00cf350 0x7ff6c009ae70 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:52.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.545+0000 7ff6c596f700 1 -- 172.21.15.163:0/749976813 >> 172.21.15.163:0/749976813 conn(0x7ff6c00999f0 msgr2=0x7ff6c00c5610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:52.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.546+0000 7ff6c596f700 1 -- 172.21.15.163:0/749976813 shutdown_connections 2023-10-09T06:36:52.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.546+0000 7ff6c596f700 1 -- 172.21.15.163:0/749976813 wait complete. 2023-10-09T06:36:52.559 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-09T06:36:52.678 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.677+0000 7fa2c0413700 1 Processor -- start 2023-10-09T06:36:52.678 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.677+0000 7fa2c0413700 1 -- start start 2023-10-09T06:36:52.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.678+0000 7fa2c0413700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2b80aca80 0x7fa2b806aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:52.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.678+0000 7fa2c0413700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa2b8069d60 0x7fa2b80c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:52.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.678+0000 7fa2c0413700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b806bdb0 con 0x7fa2b80c81f0 2023-10-09T06:36:52.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.678+0000 7fa2c0413700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2b80c8670 con 0x7fa2b8069d60 2023-10-09T06:36:52.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.678+0000 7fa2c0413700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b80cb370 con 0x7fa2b80aca80 2023-10-09T06:36:52.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.679+0000 7fa2be1af700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2b80aca80 0x7fa2b806aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:52.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.679+0000 7fa2bd9ae700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa2b8069d60 0x7fa2b80c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:52.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.679+0000 7fa2be1af700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2b80aca80 0x7fa2b806aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:54364/0 (socket says 172.21.15.163:54364) 2023-10-09T06:36:52.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.679+0000 7fa2bd9ae700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa2b8069d60 0x7fa2b80c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:44624/0 (socket says 172.21.15.163:44624) 2023-10-09T06:36:52.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.679+0000 7fa2be1af700 1 -- 172.21.15.163:0/2020211255 learned_addr learned my addr 172.21.15.163:0/2020211255 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:52.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.679+0000 7fa2bd9ae700 1 -- 172.21.15.163:0/2020211255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2b80aca80 msgr2=0x7fa2b806aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:52.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.679+0000 7fa2bd9ae700 1 --2- 172.21.15.163:0/2020211255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2b80aca80 0x7fa2b806aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:52.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.679+0000 7fa2bd9ae700 1 -- 172.21.15.163:0/2020211255 >> v1:172.21.15.181:6789/0 conn(0x7fa2b80c81f0 legacy=0x7fa2b8068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:52.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.679+0000 7fa2bd9ae700 1 -- 172.21.15.163:0/2020211255 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b8159f60 con 0x7fa2b8069d60 2023-10-09T06:36:52.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.679+0000 7fa2be1af700 1 --2- 172.21.15.163:0/2020211255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2b80aca80 0x7fa2b806aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:36:52.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.679+0000 7fa2bd9ae700 1 --2- 172.21.15.163:0/2020211255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2b8069d60 0x7fa2b80c7cb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80098e0 tx=0x7fa2a80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9d2ab8c24b199fa server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:52.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.679+0000 7fa2bd1ad700 1 -- 172.21.15.163:0/2020211255 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a800b640 con 0x7fa2b8069d60 2023-10-09T06:36:52.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.680+0000 7fa2bd1ad700 1 -- 172.21.15.163:0/2020211255 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a800b7a0 con 0x7fa2b8069d60 2023-10-09T06:36:52.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.680+0000 7fa2bd1ad700 1 -- 172.21.15.163:0/2020211255 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a800bb40 con 0x7fa2b8069d60 2023-10-09T06:36:52.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.680+0000 7fa2c0413700 1 -- 172.21.15.163:0/2020211255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2b8069d60 msgr2=0x7fa2b80c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:52.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.680+0000 7fa2c0413700 1 --2- 172.21.15.163:0/2020211255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2b8069d60 0x7fa2b80c7cb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80098e0 tx=0x7fa2a80091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:52.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.680+0000 7fa2c0413700 1 -- 172.21.15.163:0/2020211255 shutdown_connections 2023-10-09T06:36:52.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.680+0000 7fa2c0413700 1 --2- 172.21.15.163:0/2020211255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2b80aca80 0x7fa2b806aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:52.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.680+0000 7fa2c0413700 1 --2- 172.21.15.163:0/2020211255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2b8069d60 0x7fa2b80c7cb0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:52.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.680+0000 7fa2c0413700 1 -- 172.21.15.163:0/2020211255 >> 172.21.15.163:0/2020211255 conn(0x7fa2b80cd670 msgr2=0x7fa2b80cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:52.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.680+0000 7fa2c0413700 1 -- 172.21.15.163:0/2020211255 shutdown_connections 2023-10-09T06:36:52.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.680+0000 7fa2c0413700 1 -- 172.21.15.163:0/2020211255 wait complete. 2023-10-09T06:36:52.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.681+0000 7fa2c0413700 1 Processor -- start 2023-10-09T06:36:52.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.681+0000 7fa2c0413700 1 -- start start 2023-10-09T06:36:52.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.682+0000 7fa2c0413700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2b8069d60 0x7fa2b80b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:52.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.682+0000 7fa2c0413700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2b80aca80 0x7fa2b80acef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:52.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.682+0000 7fa2c0413700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2b80c81f0 0x7fa2b80ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:52.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.682+0000 7fa2c0413700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b80afda0 con 0x7fa2b8069d60 2023-10-09T06:36:52.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.682+0000 7fa2c0413700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b80c6990 con 0x7fa2b80c81f0 2023-10-09T06:36:52.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.682+0000 7fa2c0413700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b80c6c70 con 0x7fa2b80aca80 2023-10-09T06:36:52.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.682+0000 7fa2be1af700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2b8069d60 0x7fa2b80b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:52.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.682+0000 7fa2be1af700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2b8069d60 0x7fa2b80b2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46212/0 (socket says 172.21.15.163:46212) 2023-10-09T06:36:52.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.682+0000 7fa2be1af700 1 -- 172.21.15.163:0/4085002667 learned_addr learned my addr 172.21.15.163:0/4085002667 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:52.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.682+0000 7fa2bd9ae700 1 --2- 172.21.15.163:0/4085002667 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2b80aca80 0x7fa2b80acef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:52.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.682+0000 7fa2be9b0700 1 --2- 172.21.15.163:0/4085002667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2b80c81f0 0x7fa2b80ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:52.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.682+0000 7fa2be1af700 1 -- 172.21.15.163:0/4085002667 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2b80aca80 msgr2=0x7fa2b80acef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:52.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.682+0000 7fa2be1af700 1 --2- 172.21.15.163:0/4085002667 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2b80aca80 0x7fa2b80acef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:52.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.682+0000 7fa2be1af700 1 -- 172.21.15.163:0/4085002667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2b80c81f0 msgr2=0x7fa2b80ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:52.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.682+0000 7fa2be1af700 1 --2- 172.21.15.163:0/4085002667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2b80c81f0 0x7fa2b80ad8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:52.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.682+0000 7fa2be1af700 1 -- 172.21.15.163:0/4085002667 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b4009ae0 con 0x7fa2b8069d60 2023-10-09T06:36:52.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.683+0000 7fa2be1af700 1 --2- 172.21.15.163:0/4085002667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2b8069d60 0x7fa2b80b2e60 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400c170 tx=0x7fa2b400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:52.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.683+0000 7fa2ae7fc700 1 -- 172.21.15.163:0/4085002667 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b400cc00 con 0x7fa2b8069d60 2023-10-09T06:36:52.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.683+0000 7fa2ae7fc700 1 -- 172.21.15.163:0/4085002667 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b400f040 con 0x7fa2b8069d60 2023-10-09T06:36:52.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.683+0000 7fa2c0413700 1 -- 172.21.15.163:0/4085002667 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a8009420 con 0x7fa2b8069d60 2023-10-09T06:36:52.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.683+0000 7fa2ae7fc700 1 -- 172.21.15.163:0/4085002667 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4014480 con 0x7fa2b8069d60 2023-10-09T06:36:52.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.683+0000 7fa2c0413700 1 -- 172.21.15.163:0/4085002667 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b815a470 con 0x7fa2b8069d60 2023-10-09T06:36:52.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.684+0000 7fa2c0413700 1 -- 172.21.15.163:0/4085002667 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b815aac0 con 0x7fa2b8069d60 2023-10-09T06:36:52.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.684+0000 7fa2ae7fc700 1 -- 172.21.15.163:0/4085002667 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fa2b40147f0 con 0x7fa2b8069d60 2023-10-09T06:36:52.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.685+0000 7fa2ae7fc700 1 --2- 172.21.15.163:0/4085002667 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa29c067290 0x7fa29c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:52.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.685+0000 7fa2ae7fc700 1 -- 172.21.15.163:0/4085002667 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa2b4085520 con 0x7fa2b8069d60 2023-10-09T06:36:52.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.685+0000 7fa2bd9ae700 1 --2- 172.21.15.163:0/4085002667 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa29c067290 0x7fa29c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:52.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.685+0000 7fa2bd9ae700 1 --2- 172.21.15.163:0/4085002667 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa29c067290 0x7fa29c069740 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa2b806a130 tx=0x7fa2a8017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:52.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.689+0000 7fa2ae7fc700 1 -- 172.21.15.163:0/4085002667 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa2b4052270 con 0x7fa2b8069d60 2023-10-09T06:36:52.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.898+0000 7fa2c0413700 1 -- 172.21.15.163:0/4085002667 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2b804c3c0 con 0x7fa29c067290 2023-10-09T06:36:52.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.899+0000 7fa2ae7fc700 1 -- 172.21.15.163:0/4085002667 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fa2b804c3c0 con 0x7fa29c067290 2023-10-09T06:36:52.900 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:36:52.900 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:52.900 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "cephfs" 2023-10-09T06:36:52.901 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:52.901 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:36:52.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.901+0000 7fa2c0413700 1 -- 172.21.15.163:0/4085002667 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa29c067290 msgr2=0x7fa29c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:52.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.901+0000 7fa2c0413700 1 --2- 172.21.15.163:0/4085002667 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa29c067290 0x7fa29c069740 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa2b806a130 tx=0x7fa2a8017000 comp rx=0 tx=0).stop 2023-10-09T06:36:52.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.901+0000 7fa2c0413700 1 -- 172.21.15.163:0/4085002667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2b8069d60 msgr2=0x7fa2b80b2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:52.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.901+0000 7fa2c0413700 1 --2- 172.21.15.163:0/4085002667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2b8069d60 0x7fa2b80b2e60 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400c170 tx=0x7fa2b400b040 comp rx=0 tx=0).stop 2023-10-09T06:36:52.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.901+0000 7fa2c0413700 1 -- 172.21.15.163:0/4085002667 shutdown_connections 2023-10-09T06:36:52.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.901+0000 7fa2c0413700 1 --2- 172.21.15.163:0/4085002667 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa29c067290 0x7fa29c069740 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:52.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.901+0000 7fa2c0413700 1 --2- 172.21.15.163:0/4085002667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2b8069d60 0x7fa2b80b2e60 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:52.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.901+0000 7fa2c0413700 1 --2- 172.21.15.163:0/4085002667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2b80c81f0 0x7fa2b80ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:52.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.901+0000 7fa2c0413700 1 --2- 172.21.15.163:0/4085002667 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2b80aca80 0x7fa2b80acef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:52.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.901+0000 7fa2c0413700 1 -- 172.21.15.163:0/4085002667 >> 172.21.15.163:0/4085002667 conn(0x7fa2b80cd670 msgr2=0x7fa2b80c29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:52.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.901+0000 7fa2c0413700 1 -- 172.21.15.163:0/4085002667 shutdown_connections 2023-10-09T06:36:52.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:52.901+0000 7fa2c0413700 1 -- 172.21.15.163:0/4085002667 wait complete. 2023-10-09T06:36:52.914 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-09T06:36:53.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.032+0000 7fa7a5f1f700 1 Processor -- start 2023-10-09T06:36:53.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.032+0000 7fa7a5f1f700 1 -- start start 2023-10-09T06:36:53.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.033+0000 7fa7a5f1f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa7a0159890 0x7fa7a015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:53.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.033+0000 7fa7a5f1f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7a015a160 con 0x7fa7a0159c60 2023-10-09T06:36:53.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.033+0000 7fa7a5f1f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7a00d13a0 con 0x7fa7a0069560 2023-10-09T06:36:53.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.033+0000 7fa7a5f1f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7a015e140 con 0x7fa7a0159890 2023-10-09T06:36:53.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.033+0000 7fa79effd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fa7a0159c60 0x7fa7a015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:54340/0 (socket says 172.21.15.163:54340) 2023-10-09T06:36:53.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.033+0000 7fa79effd700 1 -- 172.21.15.163:0/512743635 learned_addr learned my addr 172.21.15.163:0/512743635 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:53.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.033+0000 7fa79f7fe700 1 --2- 172.21.15.163:0/512743635 >> v2:172.21.15.181:3300/0 conn(0x7fa7a0159890 0x7fa7a015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:53.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.034+0000 7fa79e7fc700 1 -- 172.21.15.163:0/512743635 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3967886526 0 0) 0x7fa7a015a160 con 0x7fa7a0159c60 2023-10-09T06:36:53.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.034+0000 7fa79e7fc700 1 -- 172.21.15.163:0/512743635 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa788003570 con 0x7fa7a0159c60 2023-10-09T06:36:53.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.034+0000 7fa79f7fe700 1 -- 172.21.15.163:0/512743635 >> v1:172.21.15.181:6789/0 conn(0x7fa7a0069560 legacy=0x7fa7a00ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:53.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.034+0000 7fa79f7fe700 1 -- 172.21.15.163:0/512743635 >> v1:172.21.15.163:6789/0 conn(0x7fa7a0159c60 legacy=0x7fa7a015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:53.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.034+0000 7fa79f7fe700 1 -- 172.21.15.163:0/512743635 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a015a3d0 con 0x7fa7a0159890 2023-10-09T06:36:53.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.034+0000 7fa79f7fe700 1 --2- 172.21.15.163:0/512743635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7a0159890 0x7fa7a015abd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa7900098e0 tx=0x7fa7900091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97942a292269d386 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:53.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.034+0000 7fa79e7fc700 1 -- 172.21.15.163:0/512743635 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79000b640 con 0x7fa7a0159890 2023-10-09T06:36:53.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.034+0000 7fa79e7fc700 1 -- 172.21.15.163:0/512743635 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa79000b7a0 con 0x7fa7a0159890 2023-10-09T06:36:53.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.034+0000 7fa79e7fc700 1 -- 172.21.15.163:0/512743635 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79000bb40 con 0x7fa7a0159890 2023-10-09T06:36:53.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.035+0000 7fa7a5f1f700 1 -- 172.21.15.163:0/512743635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7a0159890 msgr2=0x7fa7a015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:53.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.035+0000 7fa7a5f1f700 1 --2- 172.21.15.163:0/512743635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7a0159890 0x7fa7a015abd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa7900098e0 tx=0x7fa7900091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:53.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.035+0000 7fa7a5f1f700 1 -- 172.21.15.163:0/512743635 shutdown_connections 2023-10-09T06:36:53.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.035+0000 7fa7a5f1f700 1 --2- 172.21.15.163:0/512743635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7a0159890 0x7fa7a015abd0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:53.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.035+0000 7fa7a5f1f700 1 -- 172.21.15.163:0/512743635 >> 172.21.15.163:0/512743635 conn(0x7fa7a0068e20 msgr2=0x7fa7a00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:53.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.035+0000 7fa7a5f1f700 1 -- 172.21.15.163:0/512743635 shutdown_connections 2023-10-09T06:36:53.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.035+0000 7fa7a5f1f700 1 -- 172.21.15.163:0/512743635 wait complete. 2023-10-09T06:36:53.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.036+0000 7fa7a5f1f700 1 Processor -- start 2023-10-09T06:36:53.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.036+0000 7fa7a5f1f700 1 -- start start 2023-10-09T06:36:53.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.037+0000 7fa7a5f1f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7a0069560 0x7fa7a00cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:53.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.037+0000 7fa7a5f1f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7a0159890 0x7fa7a00c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:53.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.037+0000 7fa7a5f1f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7a0159c60 0x7fa7a00b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:53.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.037+0000 7fa7a5f1f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a015c0d0 con 0x7fa7a0159c60 2023-10-09T06:36:53.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.037+0000 7fa7a5f1f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a015c3b0 con 0x7fa7a0159890 2023-10-09T06:36:53.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.037+0000 7fa7a5f1f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a015c690 con 0x7fa7a0069560 2023-10-09T06:36:53.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.037+0000 7fa79ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7a0159c60 0x7fa7a00b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:53.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.037+0000 7fa79ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7a0159c60 0x7fa7a00b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46224/0 (socket says 172.21.15.163:46224) 2023-10-09T06:36:53.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.037+0000 7fa79ffff700 1 -- 172.21.15.163:0/1609755695 learned_addr learned my addr 172.21.15.163:0/1609755695 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:53.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.037+0000 7fa79f7fe700 1 --2- 172.21.15.163:0/1609755695 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7a0069560 0x7fa7a00cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:53.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.037+0000 7fa79effd700 1 --2- 172.21.15.163:0/1609755695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7a0159890 0x7fa7a00c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:53.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.037+0000 7fa79ffff700 1 -- 172.21.15.163:0/1609755695 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7a0069560 msgr2=0x7fa7a00cf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:53.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.037+0000 7fa79ffff700 1 --2- 172.21.15.163:0/1609755695 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7a0069560 0x7fa7a00cf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:53.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.037+0000 7fa79ffff700 1 -- 172.21.15.163:0/1609755695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7a0159890 msgr2=0x7fa7a00c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:53.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.037+0000 7fa79ffff700 1 --2- 172.21.15.163:0/1609755695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7a0159890 0x7fa7a00c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:53.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.037+0000 7fa79ffff700 1 -- 172.21.15.163:0/1609755695 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa790009420 con 0x7fa7a0159c60 2023-10-09T06:36:53.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.037+0000 7fa79ffff700 1 --2- 172.21.15.163:0/1609755695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7a0159c60 0x7fa7a00b43b0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fa79800c8e0 tx=0x7fa798001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:53.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.037+0000 7fa7877fe700 1 -- 172.21.15.163:0/1609755695 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa798012630 con 0x7fa7a0159c60 2023-10-09T06:36:53.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.038+0000 7fa7877fe700 1 -- 172.21.15.163:0/1609755695 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa798012790 con 0x7fa7a0159c60 2023-10-09T06:36:53.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.038+0000 7fa7a5f1f700 1 -- 172.21.15.163:0/1609755695 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a015c970 con 0x7fa7a0159c60 2023-10-09T06:36:53.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.038+0000 7fa7877fe700 1 -- 172.21.15.163:0/1609755695 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa798012b50 con 0x7fa7a0159c60 2023-10-09T06:36:53.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.038+0000 7fa7a5f1f700 1 -- 172.21.15.163:0/1609755695 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a00cf8e0 con 0x7fa7a0159c60 2023-10-09T06:36:53.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.038+0000 7fa7a5f1f700 1 -- 172.21.15.163:0/1609755695 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a015ce90 con 0x7fa7a0159c60 2023-10-09T06:36:53.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.039+0000 7fa7877fe700 1 -- 172.21.15.163:0/1609755695 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fa79801a3a0 con 0x7fa7a0159c60 2023-10-09T06:36:53.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.039+0000 7fa7877fe700 1 --2- 172.21.15.163:0/1609755695 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa78c067310 0x7fa78c0697c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:53.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.039+0000 7fa7877fe700 1 -- 172.21.15.163:0/1609755695 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa798084d20 con 0x7fa7a0159c60 2023-10-09T06:36:53.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.039+0000 7fa79f7fe700 1 --2- 172.21.15.163:0/1609755695 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa78c067310 0x7fa78c0697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:53.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.040+0000 7fa79f7fe700 1 --2- 172.21.15.163:0/1609755695 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa78c067310 0x7fa78c0697c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa7900093f0 tx=0x7fa790017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:53.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.043+0000 7fa7877fe700 1 -- 172.21.15.163:0/1609755695 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa7a015ce90 con 0x7fa7a0159c60 2023-10-09T06:36:53.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.226+0000 7fa7a5f1f700 1 -- 172.21.15.163:0/1609755695 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa7a015ce90 con 0x7fa7a0159c60 2023-10-09T06:36:53.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.226+0000 7fa7877fe700 1 -- 172.21.15.163:0/1609755695 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa7a015ce90 con 0x7fa7a0159c60 2023-10-09T06:36:53.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.228+0000 7fa7a5f1f700 1 -- 172.21.15.163:0/1609755695 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa78c067310 msgr2=0x7fa78c0697c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:53.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.228+0000 7fa7a5f1f700 1 --2- 172.21.15.163:0/1609755695 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa78c067310 0x7fa78c0697c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa7900093f0 tx=0x7fa790017000 comp rx=0 tx=0).stop 2023-10-09T06:36:53.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.228+0000 7fa7a5f1f700 1 -- 172.21.15.163:0/1609755695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7a0159c60 msgr2=0x7fa7a00b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:53.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.228+0000 7fa7a5f1f700 1 --2- 172.21.15.163:0/1609755695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7a0159c60 0x7fa7a00b43b0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fa79800c8e0 tx=0x7fa798001710 comp rx=0 tx=0).stop 2023-10-09T06:36:53.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.229+0000 7fa7a5f1f700 1 -- 172.21.15.163:0/1609755695 shutdown_connections 2023-10-09T06:36:53.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.229+0000 7fa7a5f1f700 1 --2- 172.21.15.163:0/1609755695 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa78c067310 0x7fa78c0697c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:53.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.229+0000 7fa7a5f1f700 1 --2- 172.21.15.163:0/1609755695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7a0159c60 0x7fa7a00b43b0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:53.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.229+0000 7fa7a5f1f700 1 --2- 172.21.15.163:0/1609755695 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7a0069560 0x7fa7a00cf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:53.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.229+0000 7fa7a5f1f700 1 --2- 172.21.15.163:0/1609755695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7a0159890 0x7fa7a00c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:53.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.229+0000 7fa7a5f1f700 1 -- 172.21.15.163:0/1609755695 >> 172.21.15.163:0/1609755695 conn(0x7fa7a0068e20 msgr2=0x7fa7a00c7170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:53.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.229+0000 7fa7a5f1f700 1 -- 172.21.15.163:0/1609755695 shutdown_connections 2023-10-09T06:36:53.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.229+0000 7fa7a5f1f700 1 -- 172.21.15.163:0/1609755695 wait complete. 2023-10-09T06:36:53.244 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume info cephfs 2023-10-09T06:36:53.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.360+0000 7f798264e700 1 Processor -- start 2023-10-09T06:36:53.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.360+0000 7f798264e700 1 -- start start 2023-10-09T06:36:53.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.361+0000 7f798264e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f797c159890 0x7f797c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:53.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.361+0000 7f798264e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f797c069560 0x7f797c0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:53.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.361+0000 7f798264e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f797c15a160 con 0x7f797c159c60 2023-10-09T06:36:53.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.361+0000 7f798264e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f797c069f20 con 0x7f797c069560 2023-10-09T06:36:53.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.361+0000 7f798264e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f797c0d13a0 con 0x7f797c159890 2023-10-09T06:36:53.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.361+0000 7f797bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f797c159890 0x7f797c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:53.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.361+0000 7f7980beb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f797c069560 0x7f797c0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:53.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.361+0000 7f797bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f797c159890 0x7f797c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:54404/0 (socket says 172.21.15.163:54404) 2023-10-09T06:36:53.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.361+0000 7f7980beb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f797c069560 0x7f797c0699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:44662/0 (socket says 172.21.15.163:44662) 2023-10-09T06:36:53.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.361+0000 7f797bfff700 1 -- 172.21.15.163:0/24318241 learned_addr learned my addr 172.21.15.163:0/24318241 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:53.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.362+0000 7f797bfff700 1 -- 172.21.15.163:0/24318241 >> v2:172.21.15.181:3300/0 conn(0x7f797c069560 msgr2=0x7f797c0699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:53.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.362+0000 7f797bfff700 1 --2- 172.21.15.163:0/24318241 >> v2:172.21.15.181:3300/0 conn(0x7f797c069560 0x7f797c0699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:53.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.362+0000 7f797bfff700 1 -- 172.21.15.163:0/24318241 >> v1:172.21.15.181:6789/0 conn(0x7f797c159c60 legacy=0x7f797c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:53.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.362+0000 7f797bfff700 1 -- 172.21.15.163:0/24318241 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797c15e140 con 0x7f797c159890 2023-10-09T06:36:53.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.362+0000 7f797bfff700 1 --2- 172.21.15.163:0/24318241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f797c159890 0x7f797c15abd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f796c0099b0 tx=0x7f796c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=878b35246a7160a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:53.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.362+0000 7f797affd700 1 -- 172.21.15.163:0/24318241 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c009ea0 con 0x7f797c159890 2023-10-09T06:36:53.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.362+0000 7f797affd700 1 -- 172.21.15.163:0/24318241 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796c00b860 con 0x7f797c159890 2023-10-09T06:36:53.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.362+0000 7f797affd700 1 -- 172.21.15.163:0/24318241 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c00bc00 con 0x7f797c159890 2023-10-09T06:36:53.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.363+0000 7f798264e700 1 -- 172.21.15.163:0/24318241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f797c159890 msgr2=0x7f797c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:53.374 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.363+0000 7f798264e700 1 --2- 172.21.15.163:0/24318241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f797c159890 0x7f797c15abd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f796c0099b0 tx=0x7f796c009250 comp rx=0 tx=0).stop 2023-10-09T06:36:53.374 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.363+0000 7f798264e700 1 -- 172.21.15.163:0/24318241 shutdown_connections 2023-10-09T06:36:53.374 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.363+0000 7f798264e700 1 --2- 172.21.15.163:0/24318241 >> v2:172.21.15.181:3300/0 conn(0x7f797c069560 0x7f797c0699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:53.374 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.363+0000 7f798264e700 1 --2- 172.21.15.163:0/24318241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f797c159890 0x7f797c15abd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:53.374 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.363+0000 7f798264e700 1 -- 172.21.15.163:0/24318241 >> 172.21.15.163:0/24318241 conn(0x7f797c068e20 msgr2=0x7f797c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:53.374 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.363+0000 7f798264e700 1 -- 172.21.15.163:0/24318241 shutdown_connections 2023-10-09T06:36:53.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.363+0000 7f798264e700 1 -- 172.21.15.163:0/24318241 wait complete. 2023-10-09T06:36:53.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.364+0000 7f798264e700 1 Processor -- start 2023-10-09T06:36:53.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.364+0000 7f798264e700 1 -- start start 2023-10-09T06:36:53.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.365+0000 7f798264e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f797c069560 0x7f797c14cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:53.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.365+0000 7f798264e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f797c159890 0x7f797c0ce480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:53.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.365+0000 7f798264e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f797c159c60 0x7f797c0b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:53.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.365+0000 7f798264e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c14d180 con 0x7f797c159c60 2023-10-09T06:36:53.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.365+0000 7f798264e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c0cd100 con 0x7f797c069560 2023-10-09T06:36:53.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.365+0000 7f798264e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f797c0cd3e0 con 0x7f797c159890 2023-10-09T06:36:53.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.365+0000 7f7980beb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f797c159c60 0x7f797c0b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:53.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.365+0000 7f7980beb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f797c159c60 0x7f797c0b44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46240/0 (socket says 172.21.15.163:46240) 2023-10-09T06:36:53.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.365+0000 7f7980beb700 1 -- 172.21.15.163:0/1749792932 learned_addr learned my addr 172.21.15.163:0/1749792932 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:53.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.365+0000 7f797bfff700 1 --2- 172.21.15.163:0/1749792932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f797c069560 0x7f797c14cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:53.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.365+0000 7f797b7fe700 1 --2- 172.21.15.163:0/1749792932 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f797c159890 0x7f797c0ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:53.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.365+0000 7f7980beb700 1 -- 172.21.15.163:0/1749792932 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f797c159890 msgr2=0x7f797c0ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:53.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.365+0000 7f7980beb700 1 --2- 172.21.15.163:0/1749792932 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f797c159890 0x7f797c0ce480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:53.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.365+0000 7f7980beb700 1 -- 172.21.15.163:0/1749792932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f797c069560 msgr2=0x7f797c14cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:53.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.365+0000 7f7980beb700 1 --2- 172.21.15.163:0/1749792932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f797c069560 0x7f797c14cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:53.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.365+0000 7f7980beb700 1 -- 172.21.15.163:0/1749792932 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796c0094f0 con 0x7f797c159c60 2023-10-09T06:36:53.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.365+0000 7f7980beb700 1 --2- 172.21.15.163:0/1749792932 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f797c159c60 0x7f797c0b44b0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f7974009df0 tx=0x7f797400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:53.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.365+0000 7f7963fff700 1 -- 172.21.15.163:0/1749792932 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797400caa0 con 0x7f797c159c60 2023-10-09T06:36:53.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.365+0000 7f7963fff700 1 -- 172.21.15.163:0/1749792932 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797400f040 con 0x7f797c159c60 2023-10-09T06:36:53.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.365+0000 7f7963fff700 1 -- 172.21.15.163:0/1749792932 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797400ce70 con 0x7f797c159c60 2023-10-09T06:36:53.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.365+0000 7f798264e700 1 -- 172.21.15.163:0/1749792932 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f797c0cd6c0 con 0x7f797c159c60 2023-10-09T06:36:53.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.366+0000 7f798264e700 1 -- 172.21.15.163:0/1749792932 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f797c0cdc80 con 0x7f797c159c60 2023-10-09T06:36:53.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.366+0000 7f798264e700 1 -- 172.21.15.163:0/1749792932 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f797c04c3c0 con 0x7f797c159c60 2023-10-09T06:36:53.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.367+0000 7f7963fff700 1 -- 172.21.15.163:0/1749792932 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f7974014690 con 0x7f797c159c60 2023-10-09T06:36:53.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.367+0000 7f7963fff700 1 --2- 172.21.15.163:0/1749792932 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7968067290 0x7f7968069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:53.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.367+0000 7f7963fff700 1 -- 172.21.15.163:0/1749792932 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7974085390 con 0x7f797c159c60 2023-10-09T06:36:53.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.368+0000 7f797bfff700 1 --2- 172.21.15.163:0/1749792932 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7968067290 0x7f7968069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:53.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.368+0000 7f797bfff700 1 --2- 172.21.15.163:0/1749792932 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7968067290 0x7f7968069740 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f796c0099b0 tx=0x7f796c017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:53.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.372+0000 7f7963fff700 1 -- 172.21.15.163:0/1749792932 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7974055b70 con 0x7f797c159c60 2023-10-09T06:36:53.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.585+0000 7f798264e700 1 -- 172.21.15.163:0/1749792932 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume info", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) v1 -- 0x7f797c159050 con 0x7f7968067290 2023-10-09T06:36:53.592 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:36:53.592+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:36:53.592 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:36:53.592+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:36:53.592 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:36:53.592+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:36:53.592 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:36:53.592+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:36:53.592 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:36:53.592+0000 7ff9594e4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-09T06:36:53.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.597+0000 7f7963fff700 1 -- 172.21.15.163:0/1749792932 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+473 (secure 0 0 0) 0x7f797c159050 con 0x7f7968067290 2023-10-09T06:36:53.598 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:36:53.598 INFO:teuthology.orchestra.run.smithi163.stdout: "mon_addrs": [ 2023-10-09T06:36:53.598 INFO:teuthology.orchestra.run.smithi163.stdout: "172.21.15.163:6789", 2023-10-09T06:36:53.599 INFO:teuthology.orchestra.run.smithi163.stdout: "172.21.15.181:6789", 2023-10-09T06:36:53.599 INFO:teuthology.orchestra.run.smithi163.stdout: "172.21.15.181:6790" 2023-10-09T06:36:53.599 INFO:teuthology.orchestra.run.smithi163.stdout: ], 2023-10-09T06:36:53.599 INFO:teuthology.orchestra.run.smithi163.stdout: "pools": { 2023-10-09T06:36:53.599 INFO:teuthology.orchestra.run.smithi163.stdout: "data": [ 2023-10-09T06:36:53.599 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:53.599 INFO:teuthology.orchestra.run.smithi163.stdout: "avail": 365977731072, 2023-10-09T06:36:53.599 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "cephfs_data", 2023-10-09T06:36:53.599 INFO:teuthology.orchestra.run.smithi163.stdout: "used": 0 2023-10-09T06:36:53.599 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:53.599 INFO:teuthology.orchestra.run.smithi163.stdout: ], 2023-10-09T06:36:53.599 INFO:teuthology.orchestra.run.smithi163.stdout: "metadata": [ 2023-10-09T06:36:53.599 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:36:53.600 INFO:teuthology.orchestra.run.smithi163.stdout: "avail": 365977731072, 2023-10-09T06:36:53.600 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "cephfs_metadata", 2023-10-09T06:36:53.600 INFO:teuthology.orchestra.run.smithi163.stdout: "used": 65536 2023-10-09T06:36:53.600 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:53.600 INFO:teuthology.orchestra.run.smithi163.stdout: ] 2023-10-09T06:36:53.600 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:36:53.600 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:36:53.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.599+0000 7f798264e700 1 -- 172.21.15.163:0/1749792932 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7968067290 msgr2=0x7f7968069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:53.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.599+0000 7f798264e700 1 --2- 172.21.15.163:0/1749792932 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7968067290 0x7f7968069740 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f796c0099b0 tx=0x7f796c017000 comp rx=0 tx=0).stop 2023-10-09T06:36:53.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.599+0000 7f798264e700 1 -- 172.21.15.163:0/1749792932 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f797c159c60 msgr2=0x7f797c0b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:53.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.599+0000 7f798264e700 1 --2- 172.21.15.163:0/1749792932 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f797c159c60 0x7f797c0b44b0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f7974009df0 tx=0x7f797400b040 comp rx=0 tx=0).stop 2023-10-09T06:36:53.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.599+0000 7f798264e700 1 -- 172.21.15.163:0/1749792932 shutdown_connections 2023-10-09T06:36:53.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.599+0000 7f798264e700 1 --2- 172.21.15.163:0/1749792932 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7968067290 0x7f7968069740 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:53.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.599+0000 7f798264e700 1 --2- 172.21.15.163:0/1749792932 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f797c159c60 0x7f797c0b44b0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:53.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.599+0000 7f798264e700 1 --2- 172.21.15.163:0/1749792932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f797c069560 0x7f797c14cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:53.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.599+0000 7f798264e700 1 --2- 172.21.15.163:0/1749792932 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f797c159890 0x7f797c0ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:53.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.599+0000 7f798264e700 1 -- 172.21.15.163:0/1749792932 >> 172.21.15.163:0/1749792932 conn(0x7f797c068e20 msgr2=0x7f797c0d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:53.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.600+0000 7f798264e700 1 -- 172.21.15.163:0/1749792932 shutdown_connections 2023-10-09T06:36:53.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:53.600+0000 7f798264e700 1 -- 172.21.15.163:0/1749792932 wait complete. 2023-10-09T06:36:53.612 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-09T06:36:53.613 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:36:53.657 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:36:53.702 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-09T06:36:53.702 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-09T06:36:53.703 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:36:53.703 DEBUG:teuthology.orchestra.run.smithi163:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-09T06:36:53.743 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi163.front.sepia.ceph.com 2023-10-09T06:36:53.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:53.743 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-09T06:36:53.798 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-09T06:36:53.798 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:36:53.798 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-09T06:36:53.842 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-09T06:36:53.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:36:53.843 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-09T06:36:53.900 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:36:54.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.014+0000 7fb7b9411700 1 Processor -- start 2023-10-09T06:36:54.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.014+0000 7fb7b9411700 1 -- start start 2023-10-09T06:36:54.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.015+0000 7fb7b9411700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7b4155c10 0x7fb7b4155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:54.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.015+0000 7fb7b9411700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb7b40ac6d0 0x7fb7b40b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:54.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.015+0000 7fb7b9411700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7b40acac0 con 0x7fb7b40ac230 2023-10-09T06:36:54.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.015+0000 7fb7b9411700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7b40ad930 con 0x7fb7b40ac6d0 2023-10-09T06:36:54.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.015+0000 7fb7b9411700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7b40acd20 con 0x7fb7b4155c10 2023-10-09T06:36:54.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.015+0000 7fb7b37fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb7b40ac6d0 0x7fb7b40b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:54.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.015+0000 7fb7b37fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb7b40ac6d0 0x7fb7b40b0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46242/0 (socket says 172.21.15.163:46242) 2023-10-09T06:36:54.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.015+0000 7fb7b37fe700 1 -- 172.21.15.163:0/2198026691 learned_addr learned my addr 172.21.15.163:0/2198026691 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:54.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.015+0000 7fb7b2ffd700 1 --2- 172.21.15.163:0/2198026691 >> v2:172.21.15.181:3300/0 conn(0x7fb7b4155c10 0x7fb7b4155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:54.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.016+0000 7fb7b37fe700 1 -- 172.21.15.163:0/2198026691 >> v2:172.21.15.181:3300/0 conn(0x7fb7b4155c10 msgr2=0x7fb7b4155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:54.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.016+0000 7fb7b37fe700 1 --2- 172.21.15.163:0/2198026691 >> v2:172.21.15.181:3300/0 conn(0x7fb7b4155c10 0x7fb7b4155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:54.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.016+0000 7fb7b37fe700 1 -- 172.21.15.163:0/2198026691 >> v1:172.21.15.181:6789/0 conn(0x7fb7b40ac230 legacy=0x7fb7b41568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:54.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.016+0000 7fb7b37fe700 1 -- 172.21.15.163:0/2198026691 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b40ad000 con 0x7fb7b40ac6d0 2023-10-09T06:36:54.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.016+0000 7fb7b37fe700 1 --2- 172.21.15.163:0/2198026691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7b40ac6d0 0x7fb7b40b0300 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4063d40 tx=0x7fb7ac00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5774f7e5a2abbacb server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:54.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.016+0000 7fb7b27fc700 1 -- 172.21.15.163:0/2198026691 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac010070 con 0x7fb7b40ac6d0 2023-10-09T06:36:54.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.016+0000 7fb7b27fc700 1 -- 172.21.15.163:0/2198026691 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7ac00dde0 con 0x7fb7b40ac6d0 2023-10-09T06:36:54.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.016+0000 7fb7b27fc700 1 -- 172.21.15.163:0/2198026691 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac00b320 con 0x7fb7b40ac6d0 2023-10-09T06:36:54.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.016+0000 7fb7b9411700 1 -- 172.21.15.163:0/2198026691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7b40ac6d0 msgr2=0x7fb7b40b0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:54.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.016+0000 7fb7b9411700 1 --2- 172.21.15.163:0/2198026691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7b40ac6d0 0x7fb7b40b0300 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4063d40 tx=0x7fb7ac00ba50 comp rx=0 tx=0).stop 2023-10-09T06:36:54.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.017+0000 7fb7b9411700 1 -- 172.21.15.163:0/2198026691 shutdown_connections 2023-10-09T06:36:54.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.017+0000 7fb7b9411700 1 --2- 172.21.15.163:0/2198026691 >> v2:172.21.15.181:3300/0 conn(0x7fb7b4155c10 0x7fb7b4155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:54.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.017+0000 7fb7b9411700 1 --2- 172.21.15.163:0/2198026691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7b40ac6d0 0x7fb7b40b0300 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:54.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.017+0000 7fb7b9411700 1 -- 172.21.15.163:0/2198026691 >> 172.21.15.163:0/2198026691 conn(0x7fb7b406a320 msgr2=0x7fb7b40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:54.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.017+0000 7fb7b9411700 1 -- 172.21.15.163:0/2198026691 shutdown_connections 2023-10-09T06:36:54.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.017+0000 7fb7b9411700 1 -- 172.21.15.163:0/2198026691 wait complete. 2023-10-09T06:36:54.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.018+0000 7fb7b9411700 1 Processor -- start 2023-10-09T06:36:54.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.018+0000 7fb7b9411700 1 -- start start 2023-10-09T06:36:54.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.018+0000 7fb7b9411700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7b40ac230 0x7fb7b414e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:54.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.018+0000 7fb7b9411700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7b40ac6d0 0x7fb7b40cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:54.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.019+0000 7fb7b9411700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7b4155c10 0x7fb7b4157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:54.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.019+0000 7fb7b9411700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b41526e0 con 0x7fb7b40ac230 2023-10-09T06:36:54.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.019+0000 7fb7b9411700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b4166720 con 0x7fb7b4155c10 2023-10-09T06:36:54.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.019+0000 7fb7b9411700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b40cb5d0 con 0x7fb7b40ac6d0 2023-10-09T06:36:54.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.019+0000 7fb7b2ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7b40ac230 0x7fb7b414e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:54.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.019+0000 7fb7b2ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7b40ac230 0x7fb7b414e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46254/0 (socket says 172.21.15.163:46254) 2023-10-09T06:36:54.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.019+0000 7fb7b2ffd700 1 -- 172.21.15.163:0/1529562217 learned_addr learned my addr 172.21.15.163:0/1529562217 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:54.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.019+0000 7fb7b37fe700 1 --2- 172.21.15.163:0/1529562217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7b4155c10 0x7fb7b4157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:54.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.019+0000 7fb7aa7fc700 1 --2- 172.21.15.163:0/1529562217 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7b40ac6d0 0x7fb7b40cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:54.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.019+0000 7fb7b2ffd700 1 -- 172.21.15.163:0/1529562217 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7b40ac6d0 msgr2=0x7fb7b40cb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:54.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.019+0000 7fb7b2ffd700 1 --2- 172.21.15.163:0/1529562217 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7b40ac6d0 0x7fb7b40cb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:54.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.019+0000 7fb7b2ffd700 1 -- 172.21.15.163:0/1529562217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7b4155c10 msgr2=0x7fb7b4157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:54.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.019+0000 7fb7b2ffd700 1 --2- 172.21.15.163:0/1529562217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7b4155c10 0x7fb7b4157d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:54.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.019+0000 7fb7b2ffd700 1 -- 172.21.15.163:0/1529562217 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7ac00bcf0 con 0x7fb7b40ac230 2023-10-09T06:36:54.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.019+0000 7fb7b37fe700 1 --2- 172.21.15.163:0/1529562217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7b4155c10 0x7fb7b4157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:54.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.019+0000 7fb7b2ffd700 1 --2- 172.21.15.163:0/1529562217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7b40ac230 0x7fb7b414e540 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400e930 tx=0x7fb7a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:54.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.019+0000 7fb7ab7fe700 1 -- 172.21.15.163:0/1529562217 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a400eed0 con 0x7fb7b40ac230 2023-10-09T06:36:54.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.019+0000 7fb7ab7fe700 1 -- 172.21.15.163:0/1529562217 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7a400c850 con 0x7fb7b40ac230 2023-10-09T06:36:54.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.019+0000 7fb7b9411700 1 -- 172.21.15.163:0/1529562217 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7b4166440 con 0x7fb7b40ac230 2023-10-09T06:36:54.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.019+0000 7fb7ab7fe700 1 -- 172.21.15.163:0/1529562217 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a400cbf0 con 0x7fb7b40ac230 2023-10-09T06:36:54.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.019+0000 7fb7b9411700 1 -- 172.21.15.163:0/1529562217 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b41505f0 con 0x7fb7b40ac230 2023-10-09T06:36:54.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.020+0000 7fb7b9411700 1 -- 172.21.15.163:0/1529562217 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b4150c40 con 0x7fb7b40ac230 2023-10-09T06:36:54.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.020+0000 7fb7ab7fe700 1 -- 172.21.15.163:0/1529562217 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fb7a4013470 con 0x7fb7b40ac230 2023-10-09T06:36:54.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.021+0000 7fb7ab7fe700 1 --2- 172.21.15.163:0/1529562217 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb794067290 0x7fb794069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:54.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.021+0000 7fb7ab7fe700 1 -- 172.21.15.163:0/1529562217 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb7a4084ca0 con 0x7fb7b40ac230 2023-10-09T06:36:54.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.021+0000 7fb7aa7fc700 1 --2- 172.21.15.163:0/1529562217 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb794067290 0x7fb794069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:54.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.022+0000 7fb7aa7fc700 1 --2- 172.21.15.163:0/1529562217 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb794067290 0x7fb794069740 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb7b406f170 tx=0x7fb79c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:54.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.025+0000 7fb7ab7fe700 1 -- 172.21.15.163:0/1529562217 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb7a4051b00 con 0x7fb7b40ac230 2023-10-09T06:36:54.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.248+0000 7fb7b9411700 1 -- 172.21.15.163:0/1529562217 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7b404c3c0 con 0x7fb7b40ac230 2023-10-09T06:36:54.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.249+0000 7fb7ab7fe700 1 -- 172.21.15.163:0/1529562217 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4611 (secure 0 0 0) 0x7fb7a4055120 con 0x7fb7b40ac230 2023-10-09T06:36:54.250 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:54.250 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3452423203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3452423203},{"type":"v1","addr":"172.21.15.181:6836","nonce":3452423203}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6298,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2097756943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2097756943},{"type":"v1","addr":"172.21.15.181:6837","nonce":2097756943}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6831,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3240892979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3240892979},{"type":"v1","addr":"172.21.15.163:6834","nonce":3240892979}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:36:43.382051+0000","modified":"2023-10-09T06:36:47.014979+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6834},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6834":{"gid":6834,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/1684208805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1684208805},{"type":"v1","addr":"172.21.15.163:6835","nonce":1684208805}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-09T06:36:54.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.251+0000 7fb7b9411700 1 -- 172.21.15.163:0/1529562217 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb794067290 msgr2=0x7fb794069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:54.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.251+0000 7fb7b9411700 1 --2- 172.21.15.163:0/1529562217 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb794067290 0x7fb794069740 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb7b406f170 tx=0x7fb79c009040 comp rx=0 tx=0).stop 2023-10-09T06:36:54.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.251+0000 7fb7b9411700 1 -- 172.21.15.163:0/1529562217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7b40ac230 msgr2=0x7fb7b414e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:54.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.251+0000 7fb7b9411700 1 --2- 172.21.15.163:0/1529562217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7b40ac230 0x7fb7b414e540 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400e930 tx=0x7fb7a40091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:54.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.251+0000 7fb7b9411700 1 -- 172.21.15.163:0/1529562217 shutdown_connections 2023-10-09T06:36:54.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.251+0000 7fb7b9411700 1 --2- 172.21.15.163:0/1529562217 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb794067290 0x7fb794069740 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:54.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.251+0000 7fb7b9411700 1 --2- 172.21.15.163:0/1529562217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7b40ac230 0x7fb7b414e540 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:54.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.251+0000 7fb7b9411700 1 --2- 172.21.15.163:0/1529562217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7b4155c10 0x7fb7b4157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:54.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.251+0000 7fb7b9411700 1 --2- 172.21.15.163:0/1529562217 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7b40ac6d0 0x7fb7b40cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:54.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.251+0000 7fb7b9411700 1 -- 172.21.15.163:0/1529562217 >> 172.21.15.163:0/1529562217 conn(0x7fb7b406a320 msgr2=0x7fb7b414ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:54.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.251+0000 7fb7b9411700 1 -- 172.21.15.163:0/1529562217 shutdown_connections 2023-10-09T06:36:54.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.251+0000 7fb7b9411700 1 -- 172.21.15.163:0/1529562217 wait complete. 2023-10-09T06:36:54.253 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 67 2023-10-09T06:36:54.264 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:36:54.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.378+0000 7fb09863e700 1 Processor -- start 2023-10-09T06:36:54.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.378+0000 7fb09863e700 1 -- start start 2023-10-09T06:36:54.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.379+0000 7fb09863e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb090159890 0x7fb09015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:54.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.379+0000 7fb09863e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb09015a160 con 0x7fb090069560 2023-10-09T06:36:54.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.379+0000 7fb09863e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0900d13a0 con 0x7fb090159c60 2023-10-09T06:36:54.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.379+0000 7fb09863e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb09015e140 con 0x7fb090159890 2023-10-09T06:36:54.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.379+0000 7fb0963da700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb090159890 0x7fb09015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:54.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.379+0000 7fb096bdb700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb090069560 0x7fb0900ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:54346/0 (socket says 172.21.15.163:54346) 2023-10-09T06:36:54.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.380+0000 7fb096bdb700 1 -- 172.21.15.163:0/1731860359 learned_addr learned my addr 172.21.15.163:0/1731860359 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:54.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.380+0000 7fb0963da700 1 -- 172.21.15.163:0/1731860359 >> v1:172.21.15.181:6789/0 conn(0x7fb090159c60 legacy=0x7fb09015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:54.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.380+0000 7fb0963da700 1 -- 172.21.15.163:0/1731860359 >> v1:172.21.15.163:6789/0 conn(0x7fb090069560 legacy=0x7fb0900ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:54.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.380+0000 7fb0963da700 1 -- 172.21.15.163:0/1731860359 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb09015a3d0 con 0x7fb090159890 2023-10-09T06:36:54.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.380+0000 7fb0963da700 1 --2- 172.21.15.163:0/1731860359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb090159890 0x7fb09015abd0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fb08c009e60 tx=0x7fb08c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c140657dd0b4914 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:54.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.380+0000 7fb0953d8700 1 -- 172.21.15.163:0/1731860359 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c00e070 con 0x7fb090159890 2023-10-09T06:36:54.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.380+0000 7fb0953d8700 1 -- 172.21.15.163:0/1731860359 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08c00bbf0 con 0x7fb090159890 2023-10-09T06:36:54.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.380+0000 7fb0953d8700 1 -- 172.21.15.163:0/1731860359 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c012430 con 0x7fb090159890 2023-10-09T06:36:54.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.381+0000 7fb09863e700 1 -- 172.21.15.163:0/1731860359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb090159890 msgr2=0x7fb09015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:54.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.381+0000 7fb09863e700 1 --2- 172.21.15.163:0/1731860359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb090159890 0x7fb09015abd0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fb08c009e60 tx=0x7fb08c009780 comp rx=0 tx=0).stop 2023-10-09T06:36:54.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.381+0000 7fb09863e700 1 -- 172.21.15.163:0/1731860359 shutdown_connections 2023-10-09T06:36:54.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.381+0000 7fb09863e700 1 --2- 172.21.15.163:0/1731860359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb090159890 0x7fb09015abd0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:54.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.381+0000 7fb09863e700 1 -- 172.21.15.163:0/1731860359 >> 172.21.15.163:0/1731860359 conn(0x7fb090068e20 msgr2=0x7fb0900d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:54.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.381+0000 7fb09863e700 1 -- 172.21.15.163:0/1731860359 shutdown_connections 2023-10-09T06:36:54.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.381+0000 7fb09863e700 1 -- 172.21.15.163:0/1731860359 wait complete. 2023-10-09T06:36:54.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.382+0000 7fb09863e700 1 Processor -- start 2023-10-09T06:36:54.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.382+0000 7fb09863e700 1 -- start start 2023-10-09T06:36:54.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.383+0000 7fb09863e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb090069560 0x7fb09014d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:54.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.383+0000 7fb09863e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb090159890 0x7fb09015d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:54.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.383+0000 7fb09863e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb090159c60 0x7fb0901522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:54.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.383+0000 7fb09863e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0900b55b0 con 0x7fb090159890 2023-10-09T06:36:54.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.383+0000 7fb09863e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0900b5120 con 0x7fb090069560 2023-10-09T06:36:54.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.383+0000 7fb09863e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb09014d890 con 0x7fb090159c60 2023-10-09T06:36:54.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.383+0000 7fb095bd9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb090159890 0x7fb09015d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:54.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.383+0000 7fb095bd9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb090159890 0x7fb09015d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46258/0 (socket says 172.21.15.163:46258) 2023-10-09T06:36:54.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.383+0000 7fb095bd9700 1 -- 172.21.15.163:0/3728314490 learned_addr learned my addr 172.21.15.163:0/3728314490 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:54.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.383+0000 7fb0963da700 1 --2- 172.21.15.163:0/3728314490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb090069560 0x7fb09014d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:54.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.383+0000 7fb096bdb700 1 --2- 172.21.15.163:0/3728314490 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb090159c60 0x7fb0901522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:54.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.383+0000 7fb095bd9700 1 -- 172.21.15.163:0/3728314490 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb090159c60 msgr2=0x7fb0901522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:54.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.383+0000 7fb095bd9700 1 --2- 172.21.15.163:0/3728314490 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb090159c60 0x7fb0901522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:54.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.383+0000 7fb095bd9700 1 -- 172.21.15.163:0/3728314490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb090069560 msgr2=0x7fb09014d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:54.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.383+0000 7fb095bd9700 1 --2- 172.21.15.163:0/3728314490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb090069560 0x7fb09014d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:54.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.383+0000 7fb095bd9700 1 -- 172.21.15.163:0/3728314490 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08c009a20 con 0x7fb090159890 2023-10-09T06:36:54.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.383+0000 7fb0963da700 1 --2- 172.21.15.163:0/3728314490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb090069560 0x7fb09014d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:54.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.383+0000 7fb095bd9700 1 --2- 172.21.15.163:0/3728314490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb090159890 0x7fb09015d7d0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fb08000c5d0 tx=0x7fb080007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:54.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.383+0000 7fb0867fc700 1 -- 172.21.15.163:0/3728314490 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080011590 con 0x7fb090159890 2023-10-09T06:36:54.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.383+0000 7fb0867fc700 1 -- 172.21.15.163:0/3728314490 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb080011710 con 0x7fb090159890 2023-10-09T06:36:54.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.383+0000 7fb09863e700 1 -- 172.21.15.163:0/3728314490 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0900b4510 con 0x7fb090159890 2023-10-09T06:36:54.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.383+0000 7fb0867fc700 1 -- 172.21.15.163:0/3728314490 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080011ad0 con 0x7fb090159890 2023-10-09T06:36:54.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.383+0000 7fb09863e700 1 -- 172.21.15.163:0/3728314490 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0900b4ad0 con 0x7fb090159890 2023-10-09T06:36:54.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.384+0000 7fb09863e700 1 -- 172.21.15.163:0/3728314490 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0900b52b0 con 0x7fb090159890 2023-10-09T06:36:54.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.385+0000 7fb0867fc700 1 -- 172.21.15.163:0/3728314490 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fb08001c460 con 0x7fb090159890 2023-10-09T06:36:54.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.385+0000 7fb0867fc700 1 --2- 172.21.15.163:0/3728314490 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb074067290 0x7fb074069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:54.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.385+0000 7fb0867fc700 1 -- 172.21.15.163:0/3728314490 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb080084810 con 0x7fb090159890 2023-10-09T06:36:54.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.385+0000 7fb0963da700 1 --2- 172.21.15.163:0/3728314490 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb074067290 0x7fb074069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:54.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.386+0000 7fb0963da700 1 --2- 172.21.15.163:0/3728314490 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb074067290 0x7fb074069740 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb08c0099f0 tx=0x7fb08c018000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:54.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.389+0000 7fb0867fc700 1 -- 172.21.15.163:0/3728314490 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb080051670 con 0x7fb090159890 2023-10-09T06:36:54.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.574+0000 7fb09863e700 1 -- 172.21.15.163:0/3728314490 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb09004c3c0 con 0x7fb090159890 2023-10-09T06:36:54.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.574+0000 7fb0867fc700 1 -- 172.21.15.163:0/3728314490 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+15404 (secure 0 0 0) 0x7fb080054c90 con 0x7fb090159890 2023-10-09T06:36:54.575 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:54.575 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":101,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:36:44.526778+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:36:41.723219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-09T06:36:42.342713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:36:54.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.577+0000 7fb09863e700 1 -- 172.21.15.163:0/3728314490 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb074067290 msgr2=0x7fb074069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:54.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.577+0000 7fb09863e700 1 --2- 172.21.15.163:0/3728314490 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb074067290 0x7fb074069740 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb08c0099f0 tx=0x7fb08c018000 comp rx=0 tx=0).stop 2023-10-09T06:36:54.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.577+0000 7fb09863e700 1 -- 172.21.15.163:0/3728314490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb090159890 msgr2=0x7fb09015d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:54.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.577+0000 7fb09863e700 1 --2- 172.21.15.163:0/3728314490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb090159890 0x7fb09015d7d0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fb08000c5d0 tx=0x7fb080007f60 comp rx=0 tx=0).stop 2023-10-09T06:36:54.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.577+0000 7fb09863e700 1 -- 172.21.15.163:0/3728314490 shutdown_connections 2023-10-09T06:36:54.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.577+0000 7fb09863e700 1 --2- 172.21.15.163:0/3728314490 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb074067290 0x7fb074069740 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:54.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.577+0000 7fb09863e700 1 --2- 172.21.15.163:0/3728314490 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb090159c60 0x7fb0901522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:54.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.577+0000 7fb09863e700 1 --2- 172.21.15.163:0/3728314490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb090069560 0x7fb09014d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:54.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.577+0000 7fb09863e700 1 --2- 172.21.15.163:0/3728314490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb090159890 0x7fb09015d7d0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:54.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.577+0000 7fb09863e700 1 -- 172.21.15.163:0/3728314490 >> 172.21.15.163:0/3728314490 conn(0x7fb090068e20 msgr2=0x7fb0900b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:54.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.577+0000 7fb09863e700 1 -- 172.21.15.163:0/3728314490 shutdown_connections 2023-10-09T06:36:54.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.577+0000 7fb09863e700 1 -- 172.21.15.163:0/3728314490 wait complete. 2023-10-09T06:36:54.590 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-09T06:36:41.723219+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-10-09T06:36:42.342713+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-09T06:36:54.590 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:36:54.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.708+0000 7fee7ce57700 1 Processor -- start 2023-10-09T06:36:54.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.708+0000 7fee7ce57700 1 -- start start 2023-10-09T06:36:54.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.709+0000 7fee7ce57700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fee780b4e70 0x7fee780ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:54.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.709+0000 7fee7ce57700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee780cfb00 con 0x7fee780b16d0 2023-10-09T06:36:54.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.709+0000 7fee7ce57700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee7815e060 con 0x7fee780b1b40 2023-10-09T06:36:54.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.709+0000 7fee7ce57700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee780b5c40 con 0x7fee780b4e70 2023-10-09T06:36:54.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.709+0000 7fee76ffd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fee780b16d0 0x7fee7815d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:54356/0 (socket says 172.21.15.163:54356) 2023-10-09T06:36:54.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.709+0000 7fee76ffd700 1 -- 172.21.15.163:0/1341851298 learned_addr learned my addr 172.21.15.163:0/1341851298 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:54.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.709+0000 7fee777fe700 1 --2- 172.21.15.163:0/1341851298 >> v2:172.21.15.181:3300/0 conn(0x7fee780b4e70 0x7fee780ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:54.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.709+0000 7fee767fc700 1 -- 172.21.15.163:0/1341851298 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2984068321 0 0) 0x7fee780cfb00 con 0x7fee780b16d0 2023-10-09T06:36:54.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.709+0000 7fee767fc700 1 -- 172.21.15.163:0/1341851298 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee60003570 con 0x7fee780b16d0 2023-10-09T06:36:54.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.710+0000 7fee777fe700 1 -- 172.21.15.163:0/1341851298 >> v1:172.21.15.181:6789/0 conn(0x7fee780b1b40 legacy=0x7fee7815d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:54.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.710+0000 7fee777fe700 1 -- 172.21.15.163:0/1341851298 >> v1:172.21.15.163:6789/0 conn(0x7fee780b16d0 legacy=0x7fee7815d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:54.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.710+0000 7fee777fe700 1 -- 172.21.15.163:0/1341851298 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee780b5f20 con 0x7fee780b4e70 2023-10-09T06:36:54.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.710+0000 7fee777fe700 1 --2- 172.21.15.163:0/1341851298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee780b4e70 0x7fee780ccff0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fee68009e60 tx=0x7fee68009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0238467970a8ed server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:54.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.710+0000 7fee767fc700 1 -- 172.21.15.163:0/1341851298 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6800e070 con 0x7fee780b4e70 2023-10-09T06:36:54.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.710+0000 7fee767fc700 1 -- 172.21.15.163:0/1341851298 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6800bb60 con 0x7fee780b4e70 2023-10-09T06:36:54.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.710+0000 7fee767fc700 1 -- 172.21.15.163:0/1341851298 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee68012430 con 0x7fee780b4e70 2023-10-09T06:36:54.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.711+0000 7fee7ce57700 1 -- 172.21.15.163:0/1341851298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee780b4e70 msgr2=0x7fee780ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:54.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.711+0000 7fee7ce57700 1 --2- 172.21.15.163:0/1341851298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee780b4e70 0x7fee780ccff0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fee68009e60 tx=0x7fee68009780 comp rx=0 tx=0).stop 2023-10-09T06:36:54.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.711+0000 7fee7ce57700 1 -- 172.21.15.163:0/1341851298 shutdown_connections 2023-10-09T06:36:54.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.711+0000 7fee7ce57700 1 --2- 172.21.15.163:0/1341851298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee780b4e70 0x7fee780ccff0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:54.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.711+0000 7fee7ce57700 1 -- 172.21.15.163:0/1341851298 >> 172.21.15.163:0/1341851298 conn(0x7fee780d10c0 msgr2=0x7fee780affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:54.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.711+0000 7fee7ce57700 1 -- 172.21.15.163:0/1341851298 shutdown_connections 2023-10-09T06:36:54.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.711+0000 7fee7ce57700 1 -- 172.21.15.163:0/1341851298 wait complete. 2023-10-09T06:36:54.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.712+0000 7fee7ce57700 1 Processor -- start 2023-10-09T06:36:54.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.712+0000 7fee7ce57700 1 -- start start 2023-10-09T06:36:54.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.713+0000 7fee7ce57700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee780b16d0 0x7fee780caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:54.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.713+0000 7fee7ce57700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee780b1b40 0x7fee780ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:54.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.713+0000 7fee7ce57700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee780b4e70 0x7fee7816d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:54.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.713+0000 7fee7ce57700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee780ca5e0 con 0x7fee780b4e70 2023-10-09T06:36:54.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.713+0000 7fee7ce57700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7816a040 con 0x7fee780b16d0 2023-10-09T06:36:54.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.713+0000 7fee7ce57700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee7816dea0 con 0x7fee780b1b40 2023-10-09T06:36:54.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.713+0000 7fee77fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee780b4e70 0x7fee7816d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:54.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.713+0000 7fee77fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee780b4e70 0x7fee7816d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46260/0 (socket says 172.21.15.163:46260) 2023-10-09T06:36:54.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.713+0000 7fee77fff700 1 -- 172.21.15.163:0/1781298204 learned_addr learned my addr 172.21.15.163:0/1781298204 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:54.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.713+0000 7fee777fe700 1 --2- 172.21.15.163:0/1781298204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee780b16d0 0x7fee780caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:54.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.713+0000 7fee76ffd700 1 --2- 172.21.15.163:0/1781298204 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee780b1b40 0x7fee780ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:54.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.713+0000 7fee77fff700 1 -- 172.21.15.163:0/1781298204 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee780b1b40 msgr2=0x7fee780ae6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:54.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.713+0000 7fee77fff700 1 --2- 172.21.15.163:0/1781298204 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee780b1b40 0x7fee780ae6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:54.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.713+0000 7fee77fff700 1 -- 172.21.15.163:0/1781298204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee780b16d0 msgr2=0x7fee780caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:54.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.713+0000 7fee77fff700 1 --2- 172.21.15.163:0/1781298204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee780b16d0 0x7fee780caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:54.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.713+0000 7fee77fff700 1 -- 172.21.15.163:0/1781298204 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee68009a20 con 0x7fee780b4e70 2023-10-09T06:36:54.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.713+0000 7fee77fff700 1 --2- 172.21.15.163:0/1781298204 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee780b4e70 0x7fee7816d330 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fee7000c560 tx=0x7fee70007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:54.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.713+0000 7fee5f7fe700 1 -- 172.21.15.163:0/1781298204 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee70011590 con 0x7fee780b4e70 2023-10-09T06:36:54.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.713+0000 7fee5f7fe700 1 -- 172.21.15.163:0/1781298204 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee700116f0 con 0x7fee780b4e70 2023-10-09T06:36:54.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.713+0000 7fee7ce57700 1 -- 172.21.15.163:0/1781298204 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee780c9f90 con 0x7fee780b4e70 2023-10-09T06:36:54.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.713+0000 7fee5f7fe700 1 -- 172.21.15.163:0/1781298204 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee70011ab0 con 0x7fee780b4e70 2023-10-09T06:36:54.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.713+0000 7fee7ce57700 1 -- 172.21.15.163:0/1781298204 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee780cafb0 con 0x7fee780b4e70 2023-10-09T06:36:54.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.714+0000 7fee7ce57700 1 -- 172.21.15.163:0/1781298204 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee780ca790 con 0x7fee780b4e70 2023-10-09T06:36:54.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.715+0000 7fee5f7fe700 1 -- 172.21.15.163:0/1781298204 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fee7001b3b0 con 0x7fee780b4e70 2023-10-09T06:36:54.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.715+0000 7fee5f7fe700 1 --2- 172.21.15.163:0/1781298204 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fee64067290 0x7fee64069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:54.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.715+0000 7fee5f7fe700 1 -- 172.21.15.163:0/1781298204 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fee70083e30 con 0x7fee780b4e70 2023-10-09T06:36:54.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.716+0000 7fee777fe700 1 --2- 172.21.15.163:0/1781298204 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fee64067290 0x7fee64069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:54.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.716+0000 7fee777fe700 1 --2- 172.21.15.163:0/1781298204 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fee64067290 0x7fee64069740 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fee680097c0 tx=0x7fee68019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:54.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.719+0000 7fee5f7fe700 1 -- 172.21.15.163:0/1781298204 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee70050c90 con 0x7fee780b4e70 2023-10-09T06:36:54.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.900+0000 7fee7ce57700 1 -- 172.21.15.163:0/1781298204 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fee7804bb80 con 0x7fee780b4e70 2023-10-09T06:36:54.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.901+0000 7fee5f7fe700 1 -- 172.21.15.163:0/1781298204 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) v1 ==== 93+0+10 (secure 0 0 0) 0x7fee70017070 con 0x7fee780b4e70 2023-10-09T06:36:54.901 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:36:54.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.902+0000 7fee7ce57700 1 -- 172.21.15.163:0/1781298204 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fee64067290 msgr2=0x7fee64069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:54.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.902+0000 7fee7ce57700 1 --2- 172.21.15.163:0/1781298204 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fee64067290 0x7fee64069740 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fee680097c0 tx=0x7fee68019000 comp rx=0 tx=0).stop 2023-10-09T06:36:54.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.902+0000 7fee7ce57700 1 -- 172.21.15.163:0/1781298204 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee780b4e70 msgr2=0x7fee7816d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:54.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.902+0000 7fee7ce57700 1 --2- 172.21.15.163:0/1781298204 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee780b4e70 0x7fee7816d330 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fee7000c560 tx=0x7fee70007ef0 comp rx=0 tx=0).stop 2023-10-09T06:36:54.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.903+0000 7fee7ce57700 1 -- 172.21.15.163:0/1781298204 shutdown_connections 2023-10-09T06:36:54.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.903+0000 7fee7ce57700 1 --2- 172.21.15.163:0/1781298204 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fee64067290 0x7fee64069740 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:54.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.903+0000 7fee7ce57700 1 --2- 172.21.15.163:0/1781298204 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee780b4e70 0x7fee7816d330 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:54.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.903+0000 7fee7ce57700 1 --2- 172.21.15.163:0/1781298204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee780b16d0 0x7fee780caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:54.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.903+0000 7fee7ce57700 1 --2- 172.21.15.163:0/1781298204 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee780b1b40 0x7fee780ae6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:54.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.903+0000 7fee7ce57700 1 -- 172.21.15.163:0/1781298204 >> 172.21.15.163:0/1781298204 conn(0x7fee780d10c0 msgr2=0x7fee780b53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:54.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.903+0000 7fee7ce57700 1 -- 172.21.15.163:0/1781298204 shutdown_connections 2023-10-09T06:36:54.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:54.903+0000 7fee7ce57700 1 -- 172.21.15.163:0/1781298204 wait complete. 2023-10-09T06:36:54.916 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-09T06:36:55.034 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.033+0000 7f6b144b3700 1 Processor -- start 2023-10-09T06:36:55.034 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.033+0000 7f6b144b3700 1 -- start start 2023-10-09T06:36:55.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.034+0000 7f6b144b3700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6b0c159890 0x7f6b0c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:55.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.034+0000 7f6b144b3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6b0c159c60 0x7f6b0c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:55.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.034+0000 7f6b144b3700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b0c15a160 con 0x7f6b0c15db10 2023-10-09T06:36:55.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.034+0000 7f6b144b3700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b0c15df50 con 0x7f6b0c159890 2023-10-09T06:36:55.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.034+0000 7f6b144b3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b0c0d13a0 con 0x7f6b0c159c60 2023-10-09T06:36:55.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.034+0000 7f6b1224f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6b0c159890 0x7f6b0c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:55.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.034+0000 7f6b12a50700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f6b0c15db10 0x7f6b0c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:54362/0 (socket says 172.21.15.163:54362) 2023-10-09T06:36:55.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.034+0000 7f6b12a50700 1 -- 172.21.15.163:0/905504730 learned_addr learned my addr 172.21.15.163:0/905504730 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:55.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.035+0000 7f6b11a4e700 1 --2- 172.21.15.163:0/905504730 >> v2:172.21.15.181:3300/0 conn(0x7f6b0c159c60 0x7f6b0c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:55.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.035+0000 7f6b1224f700 1 -- 172.21.15.163:0/905504730 >> v2:172.21.15.181:3300/0 conn(0x7f6b0c159c60 msgr2=0x7f6b0c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:55.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.035+0000 7f6b1224f700 1 --2- 172.21.15.163:0/905504730 >> v2:172.21.15.181:3300/0 conn(0x7f6b0c159c60 0x7f6b0c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.035+0000 7f6b1224f700 1 -- 172.21.15.163:0/905504730 >> v1:172.21.15.163:6789/0 conn(0x7f6b0c15db10 legacy=0x7f6b0c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:55.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.035+0000 7f6b1224f700 1 -- 172.21.15.163:0/905504730 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c15e140 con 0x7f6b0c159890 2023-10-09T06:36:55.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.035+0000 7f6b1124d700 1 -- 172.21.15.163:0/905504730 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2327932698 0 0) 0x7f6b0c15a160 con 0x7f6b0c15db10 2023-10-09T06:36:55.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.035+0000 7f6b11a4e700 1 --2- 172.21.15.163:0/905504730 >> v2:172.21.15.181:3300/0 conn(0x7f6b0c159c60 0x7f6b0c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:36:55.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.035+0000 7f6b1224f700 1 --2- 172.21.15.163:0/905504730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b0c159890 0x7f6b0c15abd0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c0b67f0 tx=0x7f6b080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37b7da9d303ba8f server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:55.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.035+0000 7f6b1124d700 1 -- 172.21.15.163:0/905504730 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0800b6b0 con 0x7f6b0c159890 2023-10-09T06:36:55.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.035+0000 7f6b1124d700 1 -- 172.21.15.163:0/905504730 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0800b810 con 0x7f6b0c159890 2023-10-09T06:36:55.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.035+0000 7f6b1124d700 1 -- 172.21.15.163:0/905504730 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0800bbb0 con 0x7f6b0c159890 2023-10-09T06:36:55.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.035+0000 7f6b144b3700 1 -- 172.21.15.163:0/905504730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b0c159890 msgr2=0x7f6b0c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:55.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.035+0000 7f6b144b3700 1 --2- 172.21.15.163:0/905504730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b0c159890 0x7f6b0c15abd0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c0b67f0 tx=0x7f6b080091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.036+0000 7f6b144b3700 1 -- 172.21.15.163:0/905504730 shutdown_connections 2023-10-09T06:36:55.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.036+0000 7f6b144b3700 1 --2- 172.21.15.163:0/905504730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b0c159890 0x7f6b0c15abd0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.036+0000 7f6b144b3700 1 --2- 172.21.15.163:0/905504730 >> v2:172.21.15.181:3300/0 conn(0x7f6b0c159c60 0x7f6b0c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.036+0000 7f6b144b3700 1 -- 172.21.15.163:0/905504730 >> 172.21.15.163:0/905504730 conn(0x7f6b0c068e20 msgr2=0x7f6b0c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:55.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.036+0000 7f6b144b3700 1 -- 172.21.15.163:0/905504730 shutdown_connections 2023-10-09T06:36:55.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.036+0000 7f6b144b3700 1 -- 172.21.15.163:0/905504730 wait complete. 2023-10-09T06:36:55.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.037+0000 7f6b144b3700 1 Processor -- start 2023-10-09T06:36:55.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.037+0000 7f6b144b3700 1 -- start start 2023-10-09T06:36:55.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.037+0000 7f6b144b3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b0c159890 0x7f6b0c16cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:55.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.037+0000 7f6b144b3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b0c159c60 0x7f6b0c1499d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:55.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.037+0000 7f6b144b3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b0c15db10 0x7f6b0c0cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:55.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.037+0000 7f6b144b3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c0cef60 con 0x7f6b0c159c60 2023-10-09T06:36:55.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.037+0000 7f6b144b3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c16d7e0 con 0x7f6b0c15db10 2023-10-09T06:36:55.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.037+0000 7f6b144b3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c16dac0 con 0x7f6b0c159890 2023-10-09T06:36:55.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.037+0000 7f6b11a4e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b0c159c60 0x7f6b0c1499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:55.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.037+0000 7f6b11a4e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b0c159c60 0x7f6b0c1499d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46272/0 (socket says 172.21.15.163:46272) 2023-10-09T06:36:55.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.037+0000 7f6b11a4e700 1 -- 172.21.15.163:0/3660865520 learned_addr learned my addr 172.21.15.163:0/3660865520 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:55.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.038+0000 7f6b1224f700 1 --2- 172.21.15.163:0/3660865520 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b0c159890 0x7f6b0c16cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:55.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.038+0000 7f6b12a50700 1 --2- 172.21.15.163:0/3660865520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b0c15db10 0x7f6b0c0cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:55.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.038+0000 7f6b11a4e700 1 -- 172.21.15.163:0/3660865520 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b0c159890 msgr2=0x7f6b0c16cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:55.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.038+0000 7f6b11a4e700 1 --2- 172.21.15.163:0/3660865520 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b0c159890 0x7f6b0c16cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.038+0000 7f6b11a4e700 1 -- 172.21.15.163:0/3660865520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b0c15db10 msgr2=0x7f6b0c0cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:55.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.038+0000 7f6b11a4e700 1 --2- 172.21.15.163:0/3660865520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b0c15db10 0x7f6b0c0cf6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.038+0000 7f6b11a4e700 1 -- 172.21.15.163:0/3660865520 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b08009420 con 0x7f6b0c159c60 2023-10-09T06:36:55.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.038+0000 7f6b11a4e700 1 --2- 172.21.15.163:0/3660865520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b0c159c60 0x7f6b0c1499d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00c830 tx=0x7f6afc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:55.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.038+0000 7f6b027fc700 1 -- 172.21.15.163:0/3660865520 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc011600 con 0x7f6b0c159c60 2023-10-09T06:36:55.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.038+0000 7f6b027fc700 1 -- 172.21.15.163:0/3660865520 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6afc011760 con 0x7f6b0c159c60 2023-10-09T06:36:55.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.038+0000 7f6b144b3700 1 -- 172.21.15.163:0/3660865520 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b0c0cf1d0 con 0x7f6b0c159c60 2023-10-09T06:36:55.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.038+0000 7f6b027fc700 1 -- 172.21.15.163:0/3660865520 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc011b20 con 0x7f6b0c159c60 2023-10-09T06:36:55.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.038+0000 7f6b144b3700 1 -- 172.21.15.163:0/3660865520 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b0c149f10 con 0x7f6b0c159c60 2023-10-09T06:36:55.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.039+0000 7f6b144b3700 1 -- 172.21.15.163:0/3660865520 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b0c0cf3c0 con 0x7f6b0c159c60 2023-10-09T06:36:55.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.039+0000 7f6b027fc700 1 -- 172.21.15.163:0/3660865520 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f6afc00a3e0 con 0x7f6b0c159c60 2023-10-09T06:36:55.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.040+0000 7f6b027fc700 1 --2- 172.21.15.163:0/3660865520 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6af0067290 0x7f6af0069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:55.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.040+0000 7f6b027fc700 1 -- 172.21.15.163:0/3660865520 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6afc085050 con 0x7f6b0c159c60 2023-10-09T06:36:55.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.040+0000 7f6b1224f700 1 --2- 172.21.15.163:0/3660865520 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6af0067290 0x7f6af0069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:55.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.040+0000 7f6b1224f700 1 --2- 172.21.15.163:0/3660865520 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6af0067290 0x7f6af0069740 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6b08003080 tx=0x7f6b08017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:55.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.044+0000 7f6b027fc700 1 -- 172.21.15.163:0/3660865520 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6afc051d60 con 0x7f6b0c159c60 2023-10-09T06:36:55.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.225+0000 7f6b144b3700 1 -- 172.21.15.163:0/3660865520 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6b0c04c3c0 con 0x7f6b0c159c60 2023-10-09T06:36:55.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.226+0000 7f6b027fc700 1 -- 172.21.15.163:0/3660865520 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6afc015020 con 0x7f6b0c159c60 2023-10-09T06:36:55.227 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 32 2023-10-09T06:36:55.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.228+0000 7f6b144b3700 1 -- 172.21.15.163:0/3660865520 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6af0067290 msgr2=0x7f6af0069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:55.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.228+0000 7f6b144b3700 1 --2- 172.21.15.163:0/3660865520 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6af0067290 0x7f6af0069740 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6b08003080 tx=0x7f6b08017000 comp rx=0 tx=0).stop 2023-10-09T06:36:55.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.228+0000 7f6b144b3700 1 -- 172.21.15.163:0/3660865520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b0c159c60 msgr2=0x7f6b0c1499d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:55.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.228+0000 7f6b144b3700 1 --2- 172.21.15.163:0/3660865520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b0c159c60 0x7f6b0c1499d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00c830 tx=0x7f6afc008040 comp rx=0 tx=0).stop 2023-10-09T06:36:55.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.228+0000 7f6b144b3700 1 -- 172.21.15.163:0/3660865520 shutdown_connections 2023-10-09T06:36:55.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.228+0000 7f6b144b3700 1 --2- 172.21.15.163:0/3660865520 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6af0067290 0x7f6af0069740 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.228+0000 7f6b144b3700 1 --2- 172.21.15.163:0/3660865520 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b0c15db10 0x7f6b0c0cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.228+0000 7f6b144b3700 1 --2- 172.21.15.163:0/3660865520 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b0c159890 0x7f6b0c16cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.228+0000 7f6b144b3700 1 --2- 172.21.15.163:0/3660865520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b0c159c60 0x7f6b0c1499d0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.228+0000 7f6b144b3700 1 -- 172.21.15.163:0/3660865520 >> 172.21.15.163:0/3660865520 conn(0x7f6b0c068e20 msgr2=0x7f6b0c0c7040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:55.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.228+0000 7f6b144b3700 1 -- 172.21.15.163:0/3660865520 shutdown_connections 2023-10-09T06:36:55.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.228+0000 7f6b144b3700 1 -- 172.21.15.163:0/3660865520 wait complete. 2023-10-09T06:36:55.241 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-09T06:36:55.359 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.358+0000 7fe171ce5700 1 Processor -- start 2023-10-09T06:36:55.359 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.358+0000 7fe171ce5700 1 -- start start 2023-10-09T06:36:55.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.359+0000 7fe171ce5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe16c159890 0x7fe16c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:55.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.359+0000 7fe171ce5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe16c069560 0x7fe16c0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:55.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.359+0000 7fe171ce5700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe16c15a160 con 0x7fe16c159c60 2023-10-09T06:36:55.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.359+0000 7fe171ce5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe16c069f20 con 0x7fe16c069560 2023-10-09T06:36:55.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.359+0000 7fe171ce5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c0d13a0 con 0x7fe16c159890 2023-10-09T06:36:55.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.359+0000 7fe16affd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fe16c159c60 0x7fe16c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:54368/0 (socket says 172.21.15.163:54368) 2023-10-09T06:36:55.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.359+0000 7fe16affd700 1 -- 172.21.15.163:0/231914287 learned_addr learned my addr 172.21.15.163:0/231914287 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:55.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.359+0000 7fe16bfff700 1 --2- 172.21.15.163:0/231914287 >> v2:172.21.15.181:3300/0 conn(0x7fe16c069560 0x7fe16c0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:55.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.359+0000 7fe16b7fe700 1 --2- 172.21.15.163:0/231914287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe16c159890 0x7fe16c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:55.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.359+0000 7fe16a7fc700 1 -- 172.21.15.163:0/231914287 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2348751232 0 0) 0x7fe16c15a160 con 0x7fe16c159c60 2023-10-09T06:36:55.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.359+0000 7fe16a7fc700 1 -- 172.21.15.163:0/231914287 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe154003570 con 0x7fe16c159c60 2023-10-09T06:36:55.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.359+0000 7fe16a7fc700 1 -- 172.21.15.163:0/231914287 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2548758111 0 0) 0x7fe154003570 con 0x7fe16c159c60 2023-10-09T06:36:55.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.359+0000 7fe16a7fc700 1 -- 172.21.15.163:0/231914287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe16c159890 msgr2=0x7fe16c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:55.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.359+0000 7fe16a7fc700 1 --2- 172.21.15.163:0/231914287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe16c159890 0x7fe16c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.359+0000 7fe16a7fc700 1 -- 172.21.15.163:0/231914287 >> v2:172.21.15.181:3300/0 conn(0x7fe16c069560 msgr2=0x7fe16c0699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:55.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.359+0000 7fe16a7fc700 1 --2- 172.21.15.163:0/231914287 >> v2:172.21.15.181:3300/0 conn(0x7fe16c069560 0x7fe16c0699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.359+0000 7fe16a7fc700 1 -- 172.21.15.163:0/231914287 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16c15e140 con 0x7fe16c159c60 2023-10-09T06:36:55.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.359+0000 7fe16a7fc700 1 -- 172.21.15.163:0/231914287 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fe160001470 con 0x7fe16c159c60 2023-10-09T06:36:55.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.360+0000 7fe16a7fc700 1 -- 172.21.15.163:0/231914287 >> v1:172.21.15.163:6789/0 conn(0x7fe16c159c60 legacy=0x7fe16c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:55.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.360+0000 7fe16a7fc700 1 --2- 172.21.15.163:0/231914287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe154003a60 0x7fe154003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:55.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.360+0000 7fe16a7fc700 1 --2- 172.21.15.163:0/231914287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe16c159890 0x7fe154004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:55.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.360+0000 7fe16a7fc700 1 --2- 172.21.15.163:0/231914287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe154004d50 0x7fe1540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:55.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.360+0000 7fe16a7fc700 1 -- 172.21.15.163:0/231914287 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c069f20 con 0x7fe154004d50 2023-10-09T06:36:55.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.360+0000 7fe16a7fc700 1 -- 172.21.15.163:0/231914287 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c0d13a0 con 0x7fe154003a60 2023-10-09T06:36:55.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.360+0000 7fe16a7fc700 1 -- 172.21.15.163:0/231914287 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe15400ba60 con 0x7fe16c159890 2023-10-09T06:36:55.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.360+0000 7fe16bfff700 1 --2- 172.21.15.163:0/231914287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe154004d50 0x7fe1540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:55.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.360+0000 7fe16b7fe700 1 --2- 172.21.15.163:0/231914287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe154003a60 0x7fe154003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:55.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.360+0000 7fe16affd700 1 --2- 172.21.15.163:0/231914287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe16c159890 0x7fe154004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:55.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.360+0000 7fe16bfff700 1 -- 172.21.15.163:0/231914287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe16c159890 msgr2=0x7fe154004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:55.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.360+0000 7fe16bfff700 1 --2- 172.21.15.163:0/231914287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe16c159890 0x7fe154004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.360+0000 7fe16bfff700 1 -- 172.21.15.163:0/231914287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe154003a60 msgr2=0x7fe154003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:55.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.360+0000 7fe16bfff700 1 --2- 172.21.15.163:0/231914287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe154003a60 0x7fe154003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.360+0000 7fe16bfff700 1 -- 172.21.15.163:0/231914287 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe154003570 con 0x7fe154004d50 2023-10-09T06:36:55.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.360+0000 7fe16bfff700 1 --2- 172.21.15.163:0/231914287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe154004d50 0x7fe1540051c0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe164009c00 tx=0x7fe1640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5261fceaf4c0b808 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:55.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.360+0000 7fe16a7fc700 1 -- 172.21.15.163:0/231914287 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16400e6b0 con 0x7fe154004d50 2023-10-09T06:36:55.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.360+0000 7fe16a7fc700 1 -- 172.21.15.163:0/231914287 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16400e810 con 0x7fe154004d50 2023-10-09T06:36:55.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.360+0000 7fe16a7fc700 1 -- 172.21.15.163:0/231914287 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16400ebd0 con 0x7fe154004d50 2023-10-09T06:36:55.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.361+0000 7fe171ce5700 1 -- 172.21.15.163:0/231914287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe154004d50 msgr2=0x7fe1540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:55.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.361+0000 7fe171ce5700 1 --2- 172.21.15.163:0/231914287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe154004d50 0x7fe1540051c0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe164009c00 tx=0x7fe1640091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.361+0000 7fe16bfff700 1 -- 172.21.15.163:0/231914287 reap_dead start 2023-10-09T06:36:55.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.361+0000 7fe171ce5700 1 -- 172.21.15.163:0/231914287 shutdown_connections 2023-10-09T06:36:55.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.361+0000 7fe171ce5700 1 -- 172.21.15.163:0/231914287 >> 172.21.15.163:0/231914287 conn(0x7fe16c068e20 msgr2=0x7fe16c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:55.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.361+0000 7fe171ce5700 1 -- 172.21.15.163:0/231914287 shutdown_connections 2023-10-09T06:36:55.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.361+0000 7fe171ce5700 1 -- 172.21.15.163:0/231914287 wait complete. 2023-10-09T06:36:55.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.362+0000 7fe171ce5700 1 Processor -- start 2023-10-09T06:36:55.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.362+0000 7fe171ce5700 1 -- start start 2023-10-09T06:36:55.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.363+0000 7fe171ce5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe16c151b40 0x7fe16c16a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:55.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.363+0000 7fe171ce5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe16c151f10 0x7fe16c0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:55.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.363+0000 7fe171ce5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe16c0cf210 0x7fe16c0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:55.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.363+0000 7fe171ce5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c0b45a0 con 0x7fe16c151b40 2023-10-09T06:36:55.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.363+0000 7fe171ce5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c0cfbd0 con 0x7fe16c151f10 2023-10-09T06:36:55.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.363+0000 7fe171ce5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c0b3ad0 con 0x7fe16c0cf210 2023-10-09T06:36:55.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.363+0000 7fe16b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe16c151b40 0x7fe16c16a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:55.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.363+0000 7fe16b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe16c151b40 0x7fe16c16a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46286/0 (socket says 172.21.15.163:46286) 2023-10-09T06:36:55.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.363+0000 7fe16b7fe700 1 -- 172.21.15.163:0/1658372337 learned_addr learned my addr 172.21.15.163:0/1658372337 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:55.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.363+0000 7fe16affd700 1 --2- 172.21.15.163:0/1658372337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe16c151f10 0x7fe16c0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:55.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.363+0000 7fe16bfff700 1 --2- 172.21.15.163:0/1658372337 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe16c0cf210 0x7fe16c0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:55.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.363+0000 7fe16b7fe700 1 -- 172.21.15.163:0/1658372337 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe16c0cf210 msgr2=0x7fe16c0cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:55.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.363+0000 7fe16b7fe700 1 --2- 172.21.15.163:0/1658372337 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe16c0cf210 0x7fe16c0cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.363+0000 7fe16b7fe700 1 -- 172.21.15.163:0/1658372337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe16c151f10 msgr2=0x7fe16c0b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:55.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.363+0000 7fe16b7fe700 1 --2- 172.21.15.163:0/1658372337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe16c151f10 0x7fe16c0b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.363+0000 7fe16b7fe700 1 -- 172.21.15.163:0/1658372337 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16c0b4b30 con 0x7fe16c151b40 2023-10-09T06:36:55.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.363+0000 7fe16b7fe700 1 --2- 172.21.15.163:0/1658372337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe16c151b40 0x7fe16c16a5e0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe15c011f30 tx=0x7fe15c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:55.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.363+0000 7fe1537fe700 1 -- 172.21.15.163:0/1658372337 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c00e040 con 0x7fe16c151b40 2023-10-09T06:36:55.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.363+0000 7fe1537fe700 1 -- 172.21.15.163:0/1658372337 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15c00f040 con 0x7fe16c151b40 2023-10-09T06:36:55.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.363+0000 7fe171ce5700 1 -- 172.21.15.163:0/1658372337 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe16c0b3d00 con 0x7fe16c151b40 2023-10-09T06:36:55.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.364+0000 7fe1537fe700 1 -- 172.21.15.163:0/1658372337 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c015430 con 0x7fe16c151b40 2023-10-09T06:36:55.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.364+0000 7fe171ce5700 1 -- 172.21.15.163:0/1658372337 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe16c0b42c0 con 0x7fe16c151b40 2023-10-09T06:36:55.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.364+0000 7fe171ce5700 1 -- 172.21.15.163:0/1658372337 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe16c04c3c0 con 0x7fe16c151b40 2023-10-09T06:36:55.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.365+0000 7fe1537fe700 1 -- 172.21.15.163:0/1658372337 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fe15c015820 con 0x7fe16c151b40 2023-10-09T06:36:55.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.365+0000 7fe1537fe700 1 --2- 172.21.15.163:0/1658372337 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe158067290 0x7fe158069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:55.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.365+0000 7fe1537fe700 1 -- 172.21.15.163:0/1658372337 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe15c085930 con 0x7fe16c151b40 2023-10-09T06:36:55.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.365+0000 7fe16affd700 1 --2- 172.21.15.163:0/1658372337 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe158067290 0x7fe158069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:55.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.366+0000 7fe16affd700 1 --2- 172.21.15.163:0/1658372337 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe158067290 0x7fe158069740 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe16c089750 tx=0x7fe160009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:55.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.369+0000 7fe1537fe700 1 -- 172.21.15.163:0/1658372337 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe15c052580 con 0x7fe16c151b40 2023-10-09T06:36:55.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.551+0000 7fe171ce5700 1 -- 172.21.15.163:0/1658372337 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe16c06a5b0 con 0x7fe16c151b40 2023-10-09T06:36:55.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.551+0000 7fe1537fe700 1 -- 172.21.15.163:0/1658372337 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe15c055ba0 con 0x7fe16c151b40 2023-10-09T06:36:55.552 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 64 2023-10-09T06:36:55.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.553+0000 7fe171ce5700 1 -- 172.21.15.163:0/1658372337 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe158067290 msgr2=0x7fe158069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:55.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.553+0000 7fe171ce5700 1 --2- 172.21.15.163:0/1658372337 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe158067290 0x7fe158069740 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe16c089750 tx=0x7fe160009040 comp rx=0 tx=0).stop 2023-10-09T06:36:55.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.553+0000 7fe171ce5700 1 -- 172.21.15.163:0/1658372337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe16c151b40 msgr2=0x7fe16c16a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:55.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.553+0000 7fe171ce5700 1 --2- 172.21.15.163:0/1658372337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe16c151b40 0x7fe16c16a5e0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe15c011f30 tx=0x7fe15c00b040 comp rx=0 tx=0).stop 2023-10-09T06:36:55.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.553+0000 7fe171ce5700 1 -- 172.21.15.163:0/1658372337 shutdown_connections 2023-10-09T06:36:55.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.553+0000 7fe171ce5700 1 --2- 172.21.15.163:0/1658372337 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe158067290 0x7fe158069740 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.553+0000 7fe171ce5700 1 --2- 172.21.15.163:0/1658372337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe16c151b40 0x7fe16c16a5e0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.553+0000 7fe171ce5700 1 --2- 172.21.15.163:0/1658372337 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe16c0cf210 0x7fe16c0cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.553+0000 7fe171ce5700 1 --2- 172.21.15.163:0/1658372337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe16c151f10 0x7fe16c0b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.553+0000 7fe171ce5700 1 -- 172.21.15.163:0/1658372337 >> 172.21.15.163:0/1658372337 conn(0x7fe16c068e20 msgr2=0x7fe16c15d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:55.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.553+0000 7fe171ce5700 1 -- 172.21.15.163:0/1658372337 shutdown_connections 2023-10-09T06:36:55.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.553+0000 7fe171ce5700 1 -- 172.21.15.163:0/1658372337 wait complete. 2023-10-09T06:36:55.569 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:36:55.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.682+0000 7fcea61a2700 1 Processor -- start 2023-10-09T06:36:55.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.682+0000 7fcea61a2700 1 -- start start 2023-10-09T06:36:55.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.683+0000 7fcea61a2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcea0159890 0x7fcea015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:55.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.683+0000 7fcea61a2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcea0069560 0x7fcea00699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:55.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.683+0000 7fcea61a2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea0069ee0 con 0x7fcea0159c60 2023-10-09T06:36:55.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.683+0000 7fcea61a2700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea015a160 con 0x7fcea0159890 2023-10-09T06:36:55.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.683+0000 7fcea61a2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea00d13a0 con 0x7fcea0069560 2023-10-09T06:36:55.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.683+0000 7fce9f7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcea0159890 0x7fcea015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:55.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.683+0000 7fce9f7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcea0159890 0x7fcea015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46294/0 (socket says 172.21.15.163:46294) 2023-10-09T06:36:55.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.683+0000 7fce9f7fe700 1 -- 172.21.15.163:0/1538953276 learned_addr learned my addr 172.21.15.163:0/1538953276 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:55.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.683+0000 7fce9ffff700 1 --2- 172.21.15.163:0/1538953276 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcea0069560 0x7fcea00699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:55.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.683+0000 7fce9f7fe700 1 -- 172.21.15.163:0/1538953276 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcea0069560 msgr2=0x7fcea00699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:55.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.683+0000 7fce9f7fe700 1 --2- 172.21.15.163:0/1538953276 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcea0069560 0x7fcea00699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.683+0000 7fce9f7fe700 1 -- 172.21.15.163:0/1538953276 >> v1:172.21.15.181:6789/0 conn(0x7fcea0159c60 legacy=0x7fcea015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:55.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.683+0000 7fce9f7fe700 1 -- 172.21.15.163:0/1538953276 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea015e140 con 0x7fcea0159890 2023-10-09T06:36:55.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.684+0000 7fce9f7fe700 1 --2- 172.21.15.163:0/1538953276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcea0159890 0x7fcea015abd0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fce90009f20 tx=0x7fce90009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44c632994d59c37 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:55.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.684+0000 7fce9e7fc700 1 -- 172.21.15.163:0/1538953276 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9000e070 con 0x7fcea0159890 2023-10-09T06:36:55.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.684+0000 7fce9e7fc700 1 -- 172.21.15.163:0/1538953276 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9000ba40 con 0x7fcea0159890 2023-10-09T06:36:55.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.684+0000 7fce9e7fc700 1 -- 172.21.15.163:0/1538953276 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce900092e0 con 0x7fcea0159890 2023-10-09T06:36:55.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.684+0000 7fcea61a2700 1 -- 172.21.15.163:0/1538953276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcea0159890 msgr2=0x7fcea015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:55.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.684+0000 7fcea61a2700 1 --2- 172.21.15.163:0/1538953276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcea0159890 0x7fcea015abd0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fce90009f20 tx=0x7fce90009840 comp rx=0 tx=0).stop 2023-10-09T06:36:55.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.684+0000 7fcea61a2700 1 -- 172.21.15.163:0/1538953276 shutdown_connections 2023-10-09T06:36:55.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.684+0000 7fcea61a2700 1 --2- 172.21.15.163:0/1538953276 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcea0069560 0x7fcea00699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.684+0000 7fcea61a2700 1 --2- 172.21.15.163:0/1538953276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcea0159890 0x7fcea015abd0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.684+0000 7fcea61a2700 1 -- 172.21.15.163:0/1538953276 >> 172.21.15.163:0/1538953276 conn(0x7fcea0068e20 msgr2=0x7fcea00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:55.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.684+0000 7fcea61a2700 1 -- 172.21.15.163:0/1538953276 shutdown_connections 2023-10-09T06:36:55.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.684+0000 7fcea61a2700 1 -- 172.21.15.163:0/1538953276 wait complete. 2023-10-09T06:36:55.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.686+0000 7fcea61a2700 1 Processor -- start 2023-10-09T06:36:55.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.686+0000 7fcea61a2700 1 -- start start 2023-10-09T06:36:55.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.686+0000 7fcea61a2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcea0069560 0x7fcea0154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:55.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.686+0000 7fcea61a2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcea0159890 0x7fcea00c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:55.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.686+0000 7fcea61a2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcea0159c60 0x7fcea00c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:55.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.686+0000 7fcea61a2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea0157840 con 0x7fcea0069560 2023-10-09T06:36:55.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.686+0000 7fcea61a2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea0156c30 con 0x7fcea0159c60 2023-10-09T06:36:55.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.686+0000 7fcea61a2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea0156f10 con 0x7fcea0159890 2023-10-09T06:36:55.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.686+0000 7fce9f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcea0069560 0x7fcea0154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:55.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.686+0000 7fce9f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcea0069560 0x7fcea0154460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46306/0 (socket says 172.21.15.163:46306) 2023-10-09T06:36:55.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.686+0000 7fce9f7fe700 1 -- 172.21.15.163:0/2575856334 learned_addr learned my addr 172.21.15.163:0/2575856334 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:55.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.687+0000 7fce9effd700 1 --2- 172.21.15.163:0/2575856334 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcea0159890 0x7fcea00c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:55.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.687+0000 7fce9ffff700 1 --2- 172.21.15.163:0/2575856334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcea0159c60 0x7fcea00c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:55.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.687+0000 7fce9f7fe700 1 -- 172.21.15.163:0/2575856334 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcea0159890 msgr2=0x7fcea00c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:55.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.687+0000 7fce9f7fe700 1 --2- 172.21.15.163:0/2575856334 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcea0159890 0x7fcea00c6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.687+0000 7fce9f7fe700 1 -- 172.21.15.163:0/2575856334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcea0159c60 msgr2=0x7fcea00c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:55.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.687+0000 7fce9f7fe700 1 --2- 172.21.15.163:0/2575856334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcea0159c60 0x7fcea00c6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.687+0000 7fce9f7fe700 1 -- 172.21.15.163:0/2575856334 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce90009ae0 con 0x7fcea0069560 2023-10-09T06:36:55.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.687+0000 7fce9f7fe700 1 --2- 172.21.15.163:0/2575856334 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcea0069560 0x7fcea0154460 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fcea00899c0 tx=0x7fce90017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:55.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.687+0000 7fce877fe700 1 -- 172.21.15.163:0/2575856334 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce90012aa0 con 0x7fcea0069560 2023-10-09T06:36:55.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.687+0000 7fce877fe700 1 -- 172.21.15.163:0/2575856334 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce90012c00 con 0x7fcea0069560 2023-10-09T06:36:55.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.687+0000 7fcea61a2700 1 -- 172.21.15.163:0/2575856334 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea01571f0 con 0x7fcea0069560 2023-10-09T06:36:55.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.687+0000 7fce877fe700 1 -- 172.21.15.163:0/2575856334 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9001d420 con 0x7fcea0069560 2023-10-09T06:36:55.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.687+0000 7fcea61a2700 1 -- 172.21.15.163:0/2575856334 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea01549a0 con 0x7fcea0069560 2023-10-09T06:36:55.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.688+0000 7fcea61a2700 1 -- 172.21.15.163:0/2575856334 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea004c3c0 con 0x7fcea0069560 2023-10-09T06:36:55.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.688+0000 7fce877fe700 1 -- 172.21.15.163:0/2575856334 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fce9001d810 con 0x7fcea0069560 2023-10-09T06:36:55.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.689+0000 7fce877fe700 1 --2- 172.21.15.163:0/2575856334 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fce8c067320 0x7fce8c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:55.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.689+0000 7fce877fe700 1 -- 172.21.15.163:0/2575856334 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fce900864b0 con 0x7fcea0069560 2023-10-09T06:36:55.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.689+0000 7fce9effd700 1 --2- 172.21.15.163:0/2575856334 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fce8c067320 0x7fce8c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:55.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.689+0000 7fce9effd700 1 --2- 172.21.15.163:0/2575856334 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fce8c067320 0x7fce8c0697d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fcea0158630 tx=0x7fce94009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:55.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.693+0000 7fce877fe700 1 -- 172.21.15.163:0/2575856334 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce90053030 con 0x7fcea0069560 2023-10-09T06:36:55.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.917+0000 7fcea61a2700 1 -- 172.21.15.163:0/2575856334 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcea006a5b0 con 0x7fcea0069560 2023-10-09T06:36:55.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.917+0000 7fce877fe700 1 -- 172.21.15.163:0/2575856334 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4611 (secure 0 0 0) 0x7fce90056650 con 0x7fcea0069560 2023-10-09T06:36:55.918 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:55.918 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3452423203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3452423203},{"type":"v1","addr":"172.21.15.181:6836","nonce":3452423203}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6298,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2097756943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2097756943},{"type":"v1","addr":"172.21.15.181:6837","nonce":2097756943}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6831,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3240892979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3240892979},{"type":"v1","addr":"172.21.15.163:6834","nonce":3240892979}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:36:43.382051+0000","modified":"2023-10-09T06:36:47.014979+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6834},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6834":{"gid":6834,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/1684208805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1684208805},{"type":"v1","addr":"172.21.15.163:6835","nonce":1684208805}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-09T06:36:55.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.919+0000 7fcea61a2700 1 -- 172.21.15.163:0/2575856334 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fce8c067320 msgr2=0x7fce8c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:55.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.919+0000 7fcea61a2700 1 --2- 172.21.15.163:0/2575856334 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fce8c067320 0x7fce8c0697d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fcea0158630 tx=0x7fce94009040 comp rx=0 tx=0).stop 2023-10-09T06:36:55.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.919+0000 7fcea61a2700 1 -- 172.21.15.163:0/2575856334 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcea0069560 msgr2=0x7fcea0154460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:55.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.919+0000 7fcea61a2700 1 --2- 172.21.15.163:0/2575856334 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcea0069560 0x7fcea0154460 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fcea00899c0 tx=0x7fce90017000 comp rx=0 tx=0).stop 2023-10-09T06:36:55.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.919+0000 7fcea61a2700 1 -- 172.21.15.163:0/2575856334 shutdown_connections 2023-10-09T06:36:55.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.919+0000 7fcea61a2700 1 --2- 172.21.15.163:0/2575856334 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fce8c067320 0x7fce8c0697d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.919+0000 7fcea61a2700 1 --2- 172.21.15.163:0/2575856334 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcea0069560 0x7fcea0154460 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.919+0000 7fcea61a2700 1 --2- 172.21.15.163:0/2575856334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcea0159c60 0x7fcea00c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.919+0000 7fcea61a2700 1 --2- 172.21.15.163:0/2575856334 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcea0159890 0x7fcea00c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:55.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.919+0000 7fcea61a2700 1 -- 172.21.15.163:0/2575856334 >> 172.21.15.163:0/2575856334 conn(0x7fcea0068e20 msgr2=0x7fcea01553d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:55.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.919+0000 7fcea61a2700 1 -- 172.21.15.163:0/2575856334 shutdown_connections 2023-10-09T06:36:55.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:55.919+0000 7fcea61a2700 1 -- 172.21.15.163:0/2575856334 wait complete. 2023-10-09T06:36:55.921 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 67 2023-10-09T06:36:55.933 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:36:56.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.047+0000 7f1891dd8700 1 Processor -- start 2023-10-09T06:36:56.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.047+0000 7f1891dd8700 1 -- start start 2023-10-09T06:36:56.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.048+0000 7f1891dd8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f188c159420 0x7f188c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:56.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.048+0000 7f1891dd8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f188c1597f0 0x7f188c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:56.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.048+0000 7f1891dd8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f188c0b11d0 con 0x7f188c15d230 2023-10-09T06:36:56.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.048+0000 7f1891dd8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f188c0b14b0 con 0x7f188c159420 2023-10-09T06:36:56.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.048+0000 7f1891dd8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f188c0b1790 con 0x7f188c1597f0 2023-10-09T06:36:56.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.048+0000 7f188bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f188c1597f0 0x7f188c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:56.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.048+0000 7f1890dd6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f188c159420 0x7f188c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:56.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.048+0000 7f188bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f188c1597f0 0x7f188c0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:54498/0 (socket says 172.21.15.163:54498) 2023-10-09T06:36:56.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.048+0000 7f18915d7700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f188c15d230 0x7f188c0b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:42482/0 (socket says 172.21.15.163:42482) 2023-10-09T06:36:56.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.048+0000 7f1890dd6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f188c159420 0x7f188c0b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:44742/0 (socket says 172.21.15.163:44742) 2023-10-09T06:36:56.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.048+0000 7f188bfff700 1 -- 172.21.15.163:0/2725287068 learned_addr learned my addr 172.21.15.163:0/2725287068 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:56.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.049+0000 7f188bfff700 1 -- 172.21.15.163:0/2725287068 >> v2:172.21.15.181:3300/0 conn(0x7f188c159420 msgr2=0x7f188c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:56.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.049+0000 7f188bfff700 1 --2- 172.21.15.163:0/2725287068 >> v2:172.21.15.181:3300/0 conn(0x7f188c159420 0x7f188c0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.049+0000 7f188bfff700 1 -- 172.21.15.163:0/2725287068 >> v1:172.21.15.181:6789/0 conn(0x7f188c15d230 legacy=0x7f188c0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:56.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.049+0000 7f188bfff700 1 -- 172.21.15.163:0/2725287068 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188c0b1a70 con 0x7f188c1597f0 2023-10-09T06:36:56.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.049+0000 7f1890dd6700 1 --2- 172.21.15.163:0/2725287068 >> v2:172.21.15.181:3300/0 conn(0x7f188c159420 0x7f188c0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:36:56.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.049+0000 7f188bfff700 1 --2- 172.21.15.163:0/2725287068 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f188c1597f0 0x7f188c0b2270 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f187c009f20 tx=0x7f187c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=865d509397371a81 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:56.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.049+0000 7f188b7fe700 1 -- 172.21.15.163:0/2725287068 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c00e070 con 0x7f188c1597f0 2023-10-09T06:36:56.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.049+0000 7f188b7fe700 1 -- 172.21.15.163:0/2725287068 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187c00b9d0 con 0x7f188c1597f0 2023-10-09T06:36:56.052 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.049+0000 7f188b7fe700 1 -- 172.21.15.163:0/2725287068 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c0092e0 con 0x7f188c1597f0 2023-10-09T06:36:56.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.049+0000 7f1891dd8700 1 -- 172.21.15.163:0/2725287068 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f188c1597f0 msgr2=0x7f188c0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:56.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.049+0000 7f1891dd8700 1 --2- 172.21.15.163:0/2725287068 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f188c1597f0 0x7f188c0b2270 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f187c009f20 tx=0x7f187c009840 comp rx=0 tx=0).stop 2023-10-09T06:36:56.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.050+0000 7f1891dd8700 1 -- 172.21.15.163:0/2725287068 shutdown_connections 2023-10-09T06:36:56.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.050+0000 7f1891dd8700 1 --2- 172.21.15.163:0/2725287068 >> v2:172.21.15.181:3300/0 conn(0x7f188c159420 0x7f188c0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.050+0000 7f1891dd8700 1 --2- 172.21.15.163:0/2725287068 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f188c1597f0 0x7f188c0b2270 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.050+0000 7f1891dd8700 1 -- 172.21.15.163:0/2725287068 >> 172.21.15.163:0/2725287068 conn(0x7f188c0d0660 msgr2=0x7f188c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:56.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.050+0000 7f1891dd8700 1 -- 172.21.15.163:0/2725287068 shutdown_connections 2023-10-09T06:36:56.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.050+0000 7f1891dd8700 1 -- 172.21.15.163:0/2725287068 wait complete. 2023-10-09T06:36:56.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.051+0000 7f1891dd8700 1 Processor -- start 2023-10-09T06:36:56.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.051+0000 7f1891dd8700 1 -- start start 2023-10-09T06:36:56.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.052+0000 7f1891dd8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f188c159420 0x7f188c154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:56.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.052+0000 7f1891dd8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f188c1597f0 0x7f188c169eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:56.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.052+0000 7f1891dd8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f188c15d230 0x7f188c16a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:56.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.052+0000 7f1891dd8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c154770 con 0x7f188c15d230 2023-10-09T06:36:56.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.052+0000 7f1891dd8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c0c5400 con 0x7f188c1597f0 2023-10-09T06:36:56.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.052+0000 7f1891dd8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f188c16aa20 con 0x7f188c159420 2023-10-09T06:36:56.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.052+0000 7f18915d7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f188c15d230 0x7f188c16a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:56.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.052+0000 7f18915d7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f188c15d230 0x7f188c16a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46316/0 (socket says 172.21.15.163:46316) 2023-10-09T06:36:56.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.052+0000 7f18915d7700 1 -- 172.21.15.163:0/3867317778 learned_addr learned my addr 172.21.15.163:0/3867317778 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:56.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.052+0000 7f1890dd6700 1 --2- 172.21.15.163:0/3867317778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f188c159420 0x7f188c154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:56.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.052+0000 7f188bfff700 1 --2- 172.21.15.163:0/3867317778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f188c1597f0 0x7f188c169eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:56.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.052+0000 7f18915d7700 1 -- 172.21.15.163:0/3867317778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f188c159420 msgr2=0x7f188c154230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:56.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.052+0000 7f18915d7700 1 --2- 172.21.15.163:0/3867317778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f188c159420 0x7f188c154230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.052+0000 7f18915d7700 1 -- 172.21.15.163:0/3867317778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f188c1597f0 msgr2=0x7f188c169eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:56.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.052+0000 7f18915d7700 1 --2- 172.21.15.163:0/3867317778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f188c1597f0 0x7f188c169eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.052+0000 7f18915d7700 1 -- 172.21.15.163:0/3867317778 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1884009420 con 0x7f188c15d230 2023-10-09T06:36:56.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.052+0000 7f18915d7700 1 --2- 172.21.15.163:0/3867317778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f188c15d230 0x7f188c16a3f0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f187400c4e0 tx=0x7f1874007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:56.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.052+0000 7f1888ff9700 1 -- 172.21.15.163:0/3867317778 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187400ce90 con 0x7f188c15d230 2023-10-09T06:36:56.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.052+0000 7f1888ff9700 1 -- 172.21.15.163:0/3867317778 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1874005a70 con 0x7f188c15d230 2023-10-09T06:36:56.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.052+0000 7f1891dd8700 1 -- 172.21.15.163:0/3867317778 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f187c009ae0 con 0x7f188c15d230 2023-10-09T06:36:56.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.052+0000 7f1888ff9700 1 -- 172.21.15.163:0/3867317778 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1874005e10 con 0x7f188c15d230 2023-10-09T06:36:56.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.052+0000 7f1891dd8700 1 -- 172.21.15.163:0/3867317778 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f188c0c56a0 con 0x7f188c15d230 2023-10-09T06:36:56.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.053+0000 7f1891dd8700 1 -- 172.21.15.163:0/3867317778 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f188c04c3c0 con 0x7f188c15d230 2023-10-09T06:36:56.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.054+0000 7f1888ff9700 1 -- 172.21.15.163:0/3867317778 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f18740072e0 con 0x7f188c15d230 2023-10-09T06:36:56.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.054+0000 7f1888ff9700 1 --2- 172.21.15.163:0/3867317778 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f186c067320 0x7f186c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:56.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.054+0000 7f1888ff9700 1 -- 172.21.15.163:0/3867317778 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1874083f80 con 0x7f188c15d230 2023-10-09T06:36:56.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.054+0000 7f1890dd6700 1 --2- 172.21.15.163:0/3867317778 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f186c067320 0x7f186c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:56.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.055+0000 7f1890dd6700 1 --2- 172.21.15.163:0/3867317778 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f186c067320 0x7f186c0697d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1884009f40 tx=0x7f18840091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:56.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.058+0000 7f1888ff9700 1 -- 172.21.15.163:0/3867317778 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1874050c10 con 0x7f188c15d230 2023-10-09T06:36:56.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.245+0000 7f1891dd8700 1 -- 172.21.15.163:0/3867317778 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f188c0b1d50 con 0x7f188c15d230 2023-10-09T06:36:56.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.246+0000 7f1888ff9700 1 -- 172.21.15.163:0/3867317778 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+15404 (secure 0 0 0) 0x7f1874017020 con 0x7f188c15d230 2023-10-09T06:36:56.247 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:56.247 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":101,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:36:44.526778+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:36:41.723219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-09T06:36:42.342713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:36:56.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.248+0000 7f1891dd8700 1 -- 172.21.15.163:0/3867317778 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f186c067320 msgr2=0x7f186c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:56.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.248+0000 7f1891dd8700 1 --2- 172.21.15.163:0/3867317778 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f186c067320 0x7f186c0697d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1884009f40 tx=0x7f18840091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.248+0000 7f1891dd8700 1 -- 172.21.15.163:0/3867317778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f188c15d230 msgr2=0x7f188c16a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:56.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.248+0000 7f1891dd8700 1 --2- 172.21.15.163:0/3867317778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f188c15d230 0x7f188c16a3f0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f187400c4e0 tx=0x7f1874007f50 comp rx=0 tx=0).stop 2023-10-09T06:36:56.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.248+0000 7f1891dd8700 1 -- 172.21.15.163:0/3867317778 shutdown_connections 2023-10-09T06:36:56.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.248+0000 7f1891dd8700 1 --2- 172.21.15.163:0/3867317778 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f186c067320 0x7f186c0697d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.248+0000 7f1891dd8700 1 --2- 172.21.15.163:0/3867317778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f188c15d230 0x7f188c16a3f0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.248+0000 7f1891dd8700 1 --2- 172.21.15.163:0/3867317778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f188c159420 0x7f188c154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.248+0000 7f1891dd8700 1 --2- 172.21.15.163:0/3867317778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f188c1597f0 0x7f188c169eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.248+0000 7f1891dd8700 1 -- 172.21.15.163:0/3867317778 >> 172.21.15.163:0/3867317778 conn(0x7f188c0d0660 msgr2=0x7f188c0af200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:56.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.248+0000 7f1891dd8700 1 -- 172.21.15.163:0/3867317778 shutdown_connections 2023-10-09T06:36:56.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.248+0000 7f1891dd8700 1 -- 172.21.15.163:0/3867317778 wait complete. 2023-10-09T06:36:56.261 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-09T06:36:56.262 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:36:56.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.376+0000 7f230f1ba700 1 Processor -- start 2023-10-09T06:36:56.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.376+0000 7f230f1ba700 1 -- start start 2023-10-09T06:36:56.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.377+0000 7f230f1ba700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2308159890 0x7f230815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:56.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.377+0000 7f230f1ba700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2308159c60 0x7f230815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:56.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.377+0000 7f230f1ba700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f230815db10 0x7f2308069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:56.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.377+0000 7f230f1ba700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f230815a160 con 0x7f230815db10 2023-10-09T06:36:56.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.377+0000 7f230f1ba700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23080d13a0 con 0x7f2308159890 2023-10-09T06:36:56.381 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.377+0000 7f230f1ba700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f230815e140 con 0x7f2308159c60 2023-10-09T06:36:56.381 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.377+0000 7f230d757700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f230815db10 0x7f2308069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:56.381 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.377+0000 7f230d757700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f230815db10 0x7f2308069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46324/0 (socket says 172.21.15.163:46324) 2023-10-09T06:36:56.381 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.377+0000 7f230d757700 1 -- 172.21.15.163:0/119080416 learned_addr learned my addr 172.21.15.163:0/119080416 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:56.381 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.377+0000 7f22fffff700 1 --2- 172.21.15.163:0/119080416 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2308159c60 0x7f230815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:56.381 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.377+0000 7f230cf56700 1 --2- 172.21.15.163:0/119080416 >> v2:172.21.15.181:3300/0 conn(0x7f2308159890 0x7f230815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:56.381 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.377+0000 7f230d757700 1 -- 172.21.15.163:0/119080416 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2308159c60 msgr2=0x7f230815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:56.381 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.377+0000 7f230d757700 1 --2- 172.21.15.163:0/119080416 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2308159c60 0x7f230815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.381 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.378+0000 7f230d757700 1 -- 172.21.15.163:0/119080416 >> v2:172.21.15.181:3300/0 conn(0x7f2308159890 msgr2=0x7f230815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:56.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.378+0000 7f230d757700 1 --2- 172.21.15.163:0/119080416 >> v2:172.21.15.181:3300/0 conn(0x7f2308159890 0x7f230815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.378+0000 7f230d757700 1 -- 172.21.15.163:0/119080416 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f230815a3d0 con 0x7f230815db10 2023-10-09T06:36:56.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.378+0000 7f230d757700 1 --2- 172.21.15.163:0/119080416 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f230815db10 0x7f2308069560 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f23080d1090 tx=0x7f2300009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1eb98f5bd6380ed server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:56.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.378+0000 7f22ff7fe700 1 -- 172.21.15.163:0/119080416 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230000e070 con 0x7f230815db10 2023-10-09T06:36:56.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.378+0000 7f22ff7fe700 1 -- 172.21.15.163:0/119080416 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f230000ba50 con 0x7f230815db10 2023-10-09T06:36:56.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.378+0000 7f22ff7fe700 1 -- 172.21.15.163:0/119080416 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23000092e0 con 0x7f230815db10 2023-10-09T06:36:56.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.378+0000 7f230f1ba700 1 -- 172.21.15.163:0/119080416 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f230815db10 msgr2=0x7f2308069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:56.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.378+0000 7f230f1ba700 1 --2- 172.21.15.163:0/119080416 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f230815db10 0x7f2308069560 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f23080d1090 tx=0x7f2300009840 comp rx=0 tx=0).stop 2023-10-09T06:36:56.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.378+0000 7f230f1ba700 1 -- 172.21.15.163:0/119080416 shutdown_connections 2023-10-09T06:36:56.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.378+0000 7f230f1ba700 1 --2- 172.21.15.163:0/119080416 >> v2:172.21.15.181:3300/0 conn(0x7f2308159890 0x7f230815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.378+0000 7f230f1ba700 1 --2- 172.21.15.163:0/119080416 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f230815db10 0x7f2308069560 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.378+0000 7f230f1ba700 1 --2- 172.21.15.163:0/119080416 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2308159c60 0x7f230815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.378+0000 7f230f1ba700 1 -- 172.21.15.163:0/119080416 >> 172.21.15.163:0/119080416 conn(0x7f2308068e20 msgr2=0x7f23080d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:56.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.378+0000 7f230f1ba700 1 -- 172.21.15.163:0/119080416 shutdown_connections 2023-10-09T06:36:56.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.378+0000 7f230f1ba700 1 -- 172.21.15.163:0/119080416 wait complete. 2023-10-09T06:36:56.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.380+0000 7f230f1ba700 1 Processor -- start 2023-10-09T06:36:56.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.380+0000 7f230f1ba700 1 -- start start 2023-10-09T06:36:56.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.380+0000 7f230f1ba700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2308159890 0x7f23080cf920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:56.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.380+0000 7f230f1ba700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2308159c60 0x7f23080b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:56.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.381+0000 7f230f1ba700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f230815db10 0x7f23080b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:56.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.381+0000 7f230f1ba700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230816dda0 con 0x7f2308159890 2023-10-09T06:36:56.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.381+0000 7f230f1ba700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2308151f40 con 0x7f230815db10 2023-10-09T06:36:56.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.381+0000 7f230f1ba700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f230816d470 con 0x7f2308159c60 2023-10-09T06:36:56.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.381+0000 7f230cf56700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2308159890 0x7f23080cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:56.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.381+0000 7f230cf56700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2308159890 0x7f23080cf920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46332/0 (socket says 172.21.15.163:46332) 2023-10-09T06:36:56.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.381+0000 7f230cf56700 1 -- 172.21.15.163:0/2740037530 learned_addr learned my addr 172.21.15.163:0/2740037530 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:56.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.381+0000 7f22fffff700 1 --2- 172.21.15.163:0/2740037530 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2308159c60 0x7f23080b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:56.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.381+0000 7f230d757700 1 --2- 172.21.15.163:0/2740037530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f230815db10 0x7f23080b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:56.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.381+0000 7f230cf56700 1 -- 172.21.15.163:0/2740037530 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2308159c60 msgr2=0x7f23080b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:56.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.381+0000 7f230cf56700 1 --2- 172.21.15.163:0/2740037530 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2308159c60 0x7f23080b37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.381+0000 7f230cf56700 1 -- 172.21.15.163:0/2740037530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f230815db10 msgr2=0x7f23080b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:56.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.381+0000 7f230cf56700 1 --2- 172.21.15.163:0/2740037530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f230815db10 0x7f23080b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.381+0000 7f230cf56700 1 -- 172.21.15.163:0/2740037530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2300009ae0 con 0x7f2308159890 2023-10-09T06:36:56.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.381+0000 7f230d757700 1 --2- 172.21.15.163:0/2740037530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f230815db10 0x7f23080b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:56.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.381+0000 7f230cf56700 1 --2- 172.21.15.163:0/2740037530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2308159890 0x7f23080cf920 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f230400d890 tx=0x7f2304008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:56.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.381+0000 7f22fcff9700 1 -- 172.21.15.163:0/2740037530 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23040125f0 con 0x7f2308159890 2023-10-09T06:36:56.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.381+0000 7f22fcff9700 1 -- 172.21.15.163:0/2740037530 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2304012750 con 0x7f2308159890 2023-10-09T06:36:56.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.381+0000 7f230f1ba700 1 -- 172.21.15.163:0/2740037530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f230816d750 con 0x7f2308159890 2023-10-09T06:36:56.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.381+0000 7f22fcff9700 1 -- 172.21.15.163:0/2740037530 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2304012ad0 con 0x7f2308159890 2023-10-09T06:36:56.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.381+0000 7f230f1ba700 1 -- 172.21.15.163:0/2740037530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23080b4390 con 0x7f2308159890 2023-10-09T06:36:56.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.382+0000 7f230f1ba700 1 -- 172.21.15.163:0/2740037530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f230804c3c0 con 0x7f2308159890 2023-10-09T06:36:56.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.383+0000 7f22fcff9700 1 -- 172.21.15.163:0/2740037530 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f230401e460 con 0x7f2308159890 2023-10-09T06:36:56.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.383+0000 7f22fcff9700 1 --2- 172.21.15.163:0/2740037530 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f22f4067320 0x7f22f40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:56.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.383+0000 7f22fcff9700 1 -- 172.21.15.163:0/2740037530 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2304085290 con 0x7f2308159890 2023-10-09T06:36:56.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.383+0000 7f22fffff700 1 --2- 172.21.15.163:0/2740037530 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f22f4067320 0x7f22f40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:56.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.383+0000 7f22fffff700 1 --2- 172.21.15.163:0/2740037530 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f22f4067320 0x7f22f40697d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f23080cad90 tx=0x7f22f8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:56.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.387+0000 7f22fcff9700 1 -- 172.21.15.163:0/2740037530 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2304051f20 con 0x7f2308159890 2023-10-09T06:36:56.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.616+0000 7f230f1ba700 1 -- 172.21.15.163:0/2740037530 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f230806a5b0 con 0x7f2308159890 2023-10-09T06:36:56.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.616+0000 7f22fcff9700 1 -- 172.21.15.163:0/2740037530 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4611 (secure 0 0 0) 0x7f2304055540 con 0x7f2308159890 2023-10-09T06:36:56.617 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:56.617 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3452423203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3452423203},{"type":"v1","addr":"172.21.15.181:6836","nonce":3452423203}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6298,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2097756943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2097756943},{"type":"v1","addr":"172.21.15.181:6837","nonce":2097756943}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6831,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3240892979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3240892979},{"type":"v1","addr":"172.21.15.163:6834","nonce":3240892979}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:36:43.382051+0000","modified":"2023-10-09T06:36:47.014979+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6834},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6834":{"gid":6834,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/1684208805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1684208805},{"type":"v1","addr":"172.21.15.163:6835","nonce":1684208805}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-09T06:36:56.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.618+0000 7f230f1ba700 1 -- 172.21.15.163:0/2740037530 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f22f4067320 msgr2=0x7f22f40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:56.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.618+0000 7f230f1ba700 1 --2- 172.21.15.163:0/2740037530 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f22f4067320 0x7f22f40697d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f23080cad90 tx=0x7f22f8008040 comp rx=0 tx=0).stop 2023-10-09T06:36:56.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.618+0000 7f230f1ba700 1 -- 172.21.15.163:0/2740037530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2308159890 msgr2=0x7f23080cf920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:56.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.618+0000 7f230f1ba700 1 --2- 172.21.15.163:0/2740037530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2308159890 0x7f23080cf920 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f230400d890 tx=0x7f2304008040 comp rx=0 tx=0).stop 2023-10-09T06:36:56.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.618+0000 7f230f1ba700 1 -- 172.21.15.163:0/2740037530 shutdown_connections 2023-10-09T06:36:56.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.618+0000 7f230f1ba700 1 --2- 172.21.15.163:0/2740037530 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f22f4067320 0x7f22f40697d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.618+0000 7f230f1ba700 1 --2- 172.21.15.163:0/2740037530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2308159890 0x7f23080cf920 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.618+0000 7f230f1ba700 1 --2- 172.21.15.163:0/2740037530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f230815db10 0x7f23080b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.618+0000 7f230f1ba700 1 --2- 172.21.15.163:0/2740037530 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2308159c60 0x7f23080b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.618+0000 7f230f1ba700 1 -- 172.21.15.163:0/2740037530 >> 172.21.15.163:0/2740037530 conn(0x7f2308068e20 msgr2=0x7f23080d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:56.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.619+0000 7f230f1ba700 1 -- 172.21.15.163:0/2740037530 shutdown_connections 2023-10-09T06:36:56.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.619+0000 7f230f1ba700 1 -- 172.21.15.163:0/2740037530 wait complete. 2023-10-09T06:36:56.621 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 67 2023-10-09T06:36:56.633 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:36:56.747 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.746+0000 7fb77b4d7700 1 Processor -- start 2023-10-09T06:36:56.747 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.746+0000 7fb77b4d7700 1 -- start start 2023-10-09T06:36:56.749 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.747+0000 7fb77b4d7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7741597f0 0x7fb7740b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:56.749 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.747+0000 7fb77b4d7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb77415d230 0x7fb77415d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:56.749 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.747+0000 7fb77b4d7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb77415dbd0 con 0x7fb774159420 2023-10-09T06:36:56.749 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.747+0000 7fb77b4d7700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7740b11d0 con 0x7fb77415d230 2023-10-09T06:36:56.749 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.747+0000 7fb77b4d7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7740b14b0 con 0x7fb7741597f0 2023-10-09T06:36:56.750 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.747+0000 7fb77acd6700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb77415d230 0x7fb77415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:56.750 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.747+0000 7fb77acd6700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb77415d230 0x7fb77415d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46348/0 (socket says 172.21.15.163:46348) 2023-10-09T06:36:56.750 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.747+0000 7fb77acd6700 1 -- 172.21.15.163:0/2821585893 learned_addr learned my addr 172.21.15.163:0/2821585893 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:56.750 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.747+0000 7fb779cd4700 1 --2- 172.21.15.163:0/2821585893 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7741597f0 0x7fb7740b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:56.750 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.747+0000 7fb77acd6700 1 -- 172.21.15.163:0/2821585893 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7741597f0 msgr2=0x7fb7740b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:56.750 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.747+0000 7fb77acd6700 1 --2- 172.21.15.163:0/2821585893 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7741597f0 0x7fb7740b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.750 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.747+0000 7fb77acd6700 1 -- 172.21.15.163:0/2821585893 >> v1:172.21.15.181:6789/0 conn(0x7fb774159420 legacy=0x7fb7740b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:56.751 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.747+0000 7fb77acd6700 1 -- 172.21.15.163:0/2821585893 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7740b1790 con 0x7fb77415d230 2023-10-09T06:36:56.751 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.748+0000 7fb77acd6700 1 --2- 172.21.15.163:0/2821585893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb77415d230 0x7fb77415d690 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fb76c0099b0 tx=0x7fb76c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8865cb29865a63 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:56.751 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.748+0000 7fb7794d3700 1 -- 172.21.15.163:0/2821585893 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c00b860 con 0x7fb77415d230 2023-10-09T06:36:56.751 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.748+0000 7fb7794d3700 1 -- 172.21.15.163:0/2821585893 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76c00b9c0 con 0x7fb77415d230 2023-10-09T06:36:56.751 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.748+0000 7fb7794d3700 1 -- 172.21.15.163:0/2821585893 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c00bd60 con 0x7fb77415d230 2023-10-09T06:36:56.752 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.748+0000 7fb77b4d7700 1 -- 172.21.15.163:0/2821585893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb77415d230 msgr2=0x7fb77415d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:56.752 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.748+0000 7fb77b4d7700 1 --2- 172.21.15.163:0/2821585893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb77415d230 0x7fb77415d690 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fb76c0099b0 tx=0x7fb76c009250 comp rx=0 tx=0).stop 2023-10-09T06:36:56.752 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.748+0000 7fb77b4d7700 1 -- 172.21.15.163:0/2821585893 shutdown_connections 2023-10-09T06:36:56.752 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.748+0000 7fb77b4d7700 1 --2- 172.21.15.163:0/2821585893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb77415d230 0x7fb77415d690 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.752 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.748+0000 7fb77b4d7700 1 --2- 172.21.15.163:0/2821585893 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7741597f0 0x7fb7740b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.752 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.748+0000 7fb77b4d7700 1 -- 172.21.15.163:0/2821585893 >> 172.21.15.163:0/2821585893 conn(0x7fb7740d0660 msgr2=0x7fb7740d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:56.752 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.748+0000 7fb77b4d7700 1 -- 172.21.15.163:0/2821585893 shutdown_connections 2023-10-09T06:36:56.752 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.748+0000 7fb77b4d7700 1 -- 172.21.15.163:0/2821585893 wait complete. 2023-10-09T06:36:56.754 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.749+0000 7fb77b4d7700 1 Processor -- start 2023-10-09T06:36:56.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.750+0000 7fb77b4d7700 1 -- start start 2023-10-09T06:36:56.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.750+0000 7fb77b4d7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb774159420 0x7fb77416ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:56.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.750+0000 7fb77b4d7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7741597f0 0x7fb774157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:56.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.750+0000 7fb77b4d7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb77415d230 0x7fb774154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:56.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.750+0000 7fb77b4d7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77416a6d0 con 0x7fb7741597f0 2023-10-09T06:36:56.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.750+0000 7fb77b4d7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb774157be0 con 0x7fb774159420 2023-10-09T06:36:56.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.750+0000 7fb77b4d7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb774157ec0 con 0x7fb77415d230 2023-10-09T06:36:56.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.750+0000 7fb779cd4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7741597f0 0x7fb774157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:56.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.750+0000 7fb779cd4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7741597f0 0x7fb774157330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46352/0 (socket says 172.21.15.163:46352) 2023-10-09T06:36:56.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.750+0000 7fb779cd4700 1 -- 172.21.15.163:0/1018592069 learned_addr learned my addr 172.21.15.163:0/1018592069 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:56.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.750+0000 7fb77acd6700 1 --2- 172.21.15.163:0/1018592069 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb77415d230 0x7fb774154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:56.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.750+0000 7fb77a4d5700 1 --2- 172.21.15.163:0/1018592069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb774159420 0x7fb77416ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:56.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.751+0000 7fb779cd4700 1 -- 172.21.15.163:0/1018592069 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb77415d230 msgr2=0x7fb774154250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:56.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.751+0000 7fb779cd4700 1 --2- 172.21.15.163:0/1018592069 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb77415d230 0x7fb774154250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.751+0000 7fb779cd4700 1 -- 172.21.15.163:0/1018592069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb774159420 msgr2=0x7fb77416ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:56.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.751+0000 7fb779cd4700 1 --2- 172.21.15.163:0/1018592069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb774159420 0x7fb77416ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.751+0000 7fb779cd4700 1 -- 172.21.15.163:0/1018592069 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb76c0094f0 con 0x7fb7741597f0 2023-10-09T06:36:56.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.751+0000 7fb77a4d5700 1 --2- 172.21.15.163:0/1018592069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb774159420 0x7fb77416ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:56.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.751+0000 7fb779cd4700 1 --2- 172.21.15.163:0/1018592069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7741597f0 0x7fb774157330 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fb76800d890 tx=0x7fb768008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:56.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.751+0000 7fb7727fc700 1 -- 172.21.15.163:0/1018592069 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7680125f0 con 0x7fb7741597f0 2023-10-09T06:36:56.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.751+0000 7fb7727fc700 1 -- 172.21.15.163:0/1018592069 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb768012770 con 0x7fb7741597f0 2023-10-09T06:36:56.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.751+0000 7fb77b4d7700 1 -- 172.21.15.163:0/1018592069 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb77416a080 con 0x7fb7741597f0 2023-10-09T06:36:56.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.751+0000 7fb7727fc700 1 -- 172.21.15.163:0/1018592069 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb768012b30 con 0x7fb7741597f0 2023-10-09T06:36:56.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.751+0000 7fb77b4d7700 1 -- 172.21.15.163:0/1018592069 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb774157870 con 0x7fb7741597f0 2023-10-09T06:36:56.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.752+0000 7fb77b4d7700 1 -- 172.21.15.163:0/1018592069 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb77416a520 con 0x7fb7741597f0 2023-10-09T06:36:56.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.752+0000 7fb7727fc700 1 -- 172.21.15.163:0/1018592069 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb76801e460 con 0x7fb7741597f0 2023-10-09T06:36:56.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.753+0000 7fb7727fc700 1 --2- 172.21.15.163:0/1018592069 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb760067320 0x7fb7600697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:56.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.753+0000 7fb7727fc700 1 -- 172.21.15.163:0/1018592069 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb7680852e0 con 0x7fb7741597f0 2023-10-09T06:36:56.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.753+0000 7fb77a4d5700 1 --2- 172.21.15.163:0/1018592069 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb760067320 0x7fb7600697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:56.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.753+0000 7fb77a4d5700 1 --2- 172.21.15.163:0/1018592069 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb760067320 0x7fb7600697d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb764007fd0 tx=0x7fb764009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:56.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.757+0000 7fb7727fc700 1 -- 172.21.15.163:0/1018592069 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb768051f70 con 0x7fb7741597f0 2023-10-09T06:36:56.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.980+0000 7fb77b4d7700 1 -- 172.21.15.163:0/1018592069 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb77404c3c0 con 0x7fb7741597f0 2023-10-09T06:36:56.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.981+0000 7fb7727fc700 1 -- 172.21.15.163:0/1018592069 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4611 (secure 0 0 0) 0x7fb768055590 con 0x7fb7741597f0 2023-10-09T06:36:56.982 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:56.982 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3452423203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3452423203},{"type":"v1","addr":"172.21.15.181:6836","nonce":3452423203}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6298,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2097756943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2097756943},{"type":"v1","addr":"172.21.15.181:6837","nonce":2097756943}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6831,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3240892979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3240892979},{"type":"v1","addr":"172.21.15.163:6834","nonce":3240892979}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:36:43.382051+0000","modified":"2023-10-09T06:36:47.014979+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6834},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6834":{"gid":6834,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/1684208805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1684208805},{"type":"v1","addr":"172.21.15.163:6835","nonce":1684208805}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-09T06:36:56.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.983+0000 7fb77b4d7700 1 -- 172.21.15.163:0/1018592069 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb760067320 msgr2=0x7fb7600697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:56.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.983+0000 7fb77b4d7700 1 --2- 172.21.15.163:0/1018592069 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb760067320 0x7fb7600697d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb764007fd0 tx=0x7fb764009040 comp rx=0 tx=0).stop 2023-10-09T06:36:56.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.983+0000 7fb77b4d7700 1 -- 172.21.15.163:0/1018592069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7741597f0 msgr2=0x7fb774157330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:56.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.983+0000 7fb77b4d7700 1 --2- 172.21.15.163:0/1018592069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7741597f0 0x7fb774157330 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fb76800d890 tx=0x7fb768008040 comp rx=0 tx=0).stop 2023-10-09T06:36:56.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.983+0000 7fb77b4d7700 1 -- 172.21.15.163:0/1018592069 shutdown_connections 2023-10-09T06:36:56.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.983+0000 7fb77b4d7700 1 --2- 172.21.15.163:0/1018592069 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb760067320 0x7fb7600697d0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.983+0000 7fb77b4d7700 1 --2- 172.21.15.163:0/1018592069 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb77415d230 0x7fb774154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.983+0000 7fb77b4d7700 1 --2- 172.21.15.163:0/1018592069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb774159420 0x7fb77416ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.983+0000 7fb77b4d7700 1 --2- 172.21.15.163:0/1018592069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7741597f0 0x7fb774157330 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:56.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.983+0000 7fb77b4d7700 1 -- 172.21.15.163:0/1018592069 >> 172.21.15.163:0/1018592069 conn(0x7fb7740d0660 msgr2=0x7fb774155180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:56.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.983+0000 7fb77b4d7700 1 -- 172.21.15.163:0/1018592069 shutdown_connections 2023-10-09T06:36:56.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:56.983+0000 7fb77b4d7700 1 -- 172.21.15.163:0/1018592069 wait complete. 2023-10-09T06:36:56.985 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 67 2023-10-09T06:36:56.996 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:36:57.111 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.110+0000 7f0d6eec8700 1 Processor -- start 2023-10-09T06:36:57.111 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.110+0000 7f0d6eec8700 1 -- start start 2023-10-09T06:36:57.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.111+0000 7f0d6eec8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0d68159890 0x7f0d6815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:57.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.111+0000 7f0d6eec8700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0d68069560 0x7f0d680699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:57.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.111+0000 7f0d6eec8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d68069ee0 con 0x7f0d68159c60 2023-10-09T06:36:57.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.111+0000 7f0d6eec8700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d6815a160 con 0x7f0d68069560 2023-10-09T06:36:57.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.111+0000 7f0d6eec8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d680d13a0 con 0x7f0d68159890 2023-10-09T06:36:57.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.111+0000 7f0d6d465700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0d68069560 0x7f0d680699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:57.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.111+0000 7f0d6d465700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0d68069560 0x7f0d680699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46366/0 (socket says 172.21.15.163:46366) 2023-10-09T06:36:57.115 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.111+0000 7f0d6d465700 1 -- 172.21.15.163:0/3331952042 learned_addr learned my addr 172.21.15.163:0/3331952042 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:57.115 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.111+0000 7f0d6cc64700 1 --2- 172.21.15.163:0/3331952042 >> v2:172.21.15.181:3300/0 conn(0x7f0d68159890 0x7f0d6815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:57.115 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.111+0000 7f0d6d465700 1 -- 172.21.15.163:0/3331952042 >> v2:172.21.15.181:3300/0 conn(0x7f0d68159890 msgr2=0x7f0d6815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:57.115 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.111+0000 7f0d6d465700 1 --2- 172.21.15.163:0/3331952042 >> v2:172.21.15.181:3300/0 conn(0x7f0d68159890 0x7f0d6815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:57.115 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.111+0000 7f0d6d465700 1 -- 172.21.15.163:0/3331952042 >> v1:172.21.15.181:6789/0 conn(0x7f0d68159c60 legacy=0x7f0d6815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:57.115 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.111+0000 7f0d6d465700 1 -- 172.21.15.163:0/3331952042 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6815e140 con 0x7f0d68069560 2023-10-09T06:36:57.115 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.111+0000 7f0d6d465700 1 --2- 172.21.15.163:0/3331952042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d68069560 0x7f0d680699a0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f0d680d1090 tx=0x7f0d60009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb58baeb8f505e87 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:57.115 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.111+0000 7f0d677fe700 1 -- 172.21.15.163:0/3331952042 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6000e070 con 0x7f0d68069560 2023-10-09T06:36:57.115 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.112+0000 7f0d677fe700 1 -- 172.21.15.163:0/3331952042 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6000ba20 con 0x7f0d68069560 2023-10-09T06:36:57.116 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.112+0000 7f0d677fe700 1 -- 172.21.15.163:0/3331952042 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d600092e0 con 0x7f0d68069560 2023-10-09T06:36:57.116 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.112+0000 7f0d6eec8700 1 -- 172.21.15.163:0/3331952042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d68069560 msgr2=0x7f0d680699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:57.116 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.112+0000 7f0d6eec8700 1 --2- 172.21.15.163:0/3331952042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d68069560 0x7f0d680699a0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f0d680d1090 tx=0x7f0d60009840 comp rx=0 tx=0).stop 2023-10-09T06:36:57.116 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.112+0000 7f0d6eec8700 1 -- 172.21.15.163:0/3331952042 shutdown_connections 2023-10-09T06:36:57.116 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.112+0000 7f0d6eec8700 1 --2- 172.21.15.163:0/3331952042 >> v2:172.21.15.181:3300/0 conn(0x7f0d68159890 0x7f0d6815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:57.116 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.112+0000 7f0d6eec8700 1 --2- 172.21.15.163:0/3331952042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d68069560 0x7f0d680699a0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:57.116 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.112+0000 7f0d6eec8700 1 -- 172.21.15.163:0/3331952042 >> 172.21.15.163:0/3331952042 conn(0x7f0d68068e20 msgr2=0x7f0d680d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:57.116 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.112+0000 7f0d6eec8700 1 -- 172.21.15.163:0/3331952042 shutdown_connections 2023-10-09T06:36:57.116 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.112+0000 7f0d6eec8700 1 -- 172.21.15.163:0/3331952042 wait complete. 2023-10-09T06:36:57.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.113+0000 7f0d6eec8700 1 Processor -- start 2023-10-09T06:36:57.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.113+0000 7f0d6eec8700 1 -- start start 2023-10-09T06:36:57.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.114+0000 7f0d6eec8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d68069560 0x7f0d68149310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:57.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.114+0000 7f0d6eec8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d68159890 0x7f0d680cda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:57.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.114+0000 7f0d6eec8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d68159c60 0x7f0d680b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:57.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.114+0000 7f0d6eec8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d68149850 con 0x7f0d68159c60 2023-10-09T06:36:57.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.114+0000 7f0d6eec8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d680c9600 con 0x7f0d68069560 2023-10-09T06:36:57.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.114+0000 7f0d6eec8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d680c89f0 con 0x7f0d68159890 2023-10-09T06:36:57.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.114+0000 7f0d6d465700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d68159c60 0x7f0d680b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:57.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.114+0000 7f0d6d465700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d68159c60 0x7f0d680b3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46382/0 (socket says 172.21.15.163:46382) 2023-10-09T06:36:57.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.114+0000 7f0d6d465700 1 -- 172.21.15.163:0/2076601491 learned_addr learned my addr 172.21.15.163:0/2076601491 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:57.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.114+0000 7f0d67fff700 1 --2- 172.21.15.163:0/2076601491 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d68159890 0x7f0d680cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:57.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.114+0000 7f0d6cc64700 1 --2- 172.21.15.163:0/2076601491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d68069560 0x7f0d68149310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:57.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.114+0000 7f0d6d465700 1 -- 172.21.15.163:0/2076601491 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d68159890 msgr2=0x7f0d680cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:57.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.114+0000 7f0d6d465700 1 --2- 172.21.15.163:0/2076601491 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d68159890 0x7f0d680cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:57.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.114+0000 7f0d6d465700 1 -- 172.21.15.163:0/2076601491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d68069560 msgr2=0x7f0d68149310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:57.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.114+0000 7f0d6d465700 1 --2- 172.21.15.163:0/2076601491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d68069560 0x7f0d68149310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:57.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.114+0000 7f0d6d465700 1 -- 172.21.15.163:0/2076601491 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d60009ae0 con 0x7f0d68159c60 2023-10-09T06:36:57.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.114+0000 7f0d6cc64700 1 --2- 172.21.15.163:0/2076601491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d68069560 0x7f0d68149310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:57.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.115+0000 7f0d6d465700 1 --2- 172.21.15.163:0/2076601491 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d68159c60 0x7f0d680b3df0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009880 tx=0x7f0d60018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:57.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.115+0000 7f0d64ff9700 1 -- 172.21.15.163:0/2076601491 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d60008be0 con 0x7f0d68159c60 2023-10-09T06:36:57.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.115+0000 7f0d64ff9700 1 -- 172.21.15.163:0/2076601491 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d60008d60 con 0x7f0d68159c60 2023-10-09T06:36:57.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.115+0000 7f0d6eec8700 1 -- 172.21.15.163:0/2076601491 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d680c8cd0 con 0x7f0d68159c60 2023-10-09T06:36:57.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.115+0000 7f0d64ff9700 1 -- 172.21.15.163:0/2076601491 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d600266b0 con 0x7f0d68159c60 2023-10-09T06:36:57.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.115+0000 7f0d6eec8700 1 -- 172.21.15.163:0/2076601491 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d680c9290 con 0x7f0d68159c60 2023-10-09T06:36:57.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.116+0000 7f0d6eec8700 1 -- 172.21.15.163:0/2076601491 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d680c97b0 con 0x7f0d68159c60 2023-10-09T06:36:57.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.116+0000 7f0d64ff9700 1 -- 172.21.15.163:0/2076601491 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0d60003040 con 0x7f0d68159c60 2023-10-09T06:36:57.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.116+0000 7f0d64ff9700 1 --2- 172.21.15.163:0/2076601491 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0d4c067320 0x7f0d4c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:57.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.116+0000 7f0d64ff9700 1 -- 172.21.15.163:0/2076601491 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0d60086d90 con 0x7f0d68159c60 2023-10-09T06:36:57.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.117+0000 7f0d6cc64700 1 --2- 172.21.15.163:0/2076601491 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0d4c067320 0x7f0d4c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:57.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.117+0000 7f0d6cc64700 1 --2- 172.21.15.163:0/2076601491 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0d4c067320 0x7f0d4c0697d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c009d80 tx=0x7f0d5c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:57.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.121+0000 7f0d64ff9700 1 -- 172.21.15.163:0/2076601491 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d60053910 con 0x7f0d68159c60 2023-10-09T06:36:57.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.303+0000 7f0d6eec8700 1 -- 172.21.15.163:0/2076601491 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d6804c3c0 con 0x7f0d68159c60 2023-10-09T06:36:57.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.304+0000 7f0d64ff9700 1 -- 172.21.15.163:0/2076601491 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+15404 (secure 0 0 0) 0x7f0d60056f30 con 0x7f0d68159c60 2023-10-09T06:36:57.305 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:36:57.305 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":101,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:36:44.526778+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:36:41.723219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-09T06:36:42.342713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:36:57.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.306+0000 7f0d6eec8700 1 -- 172.21.15.163:0/2076601491 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0d4c067320 msgr2=0x7f0d4c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:57.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.306+0000 7f0d6eec8700 1 --2- 172.21.15.163:0/2076601491 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0d4c067320 0x7f0d4c0697d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c009d80 tx=0x7f0d5c008040 comp rx=0 tx=0).stop 2023-10-09T06:36:57.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.306+0000 7f0d6eec8700 1 -- 172.21.15.163:0/2076601491 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d68159c60 msgr2=0x7f0d680b3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:57.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.306+0000 7f0d6eec8700 1 --2- 172.21.15.163:0/2076601491 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d68159c60 0x7f0d680b3df0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009880 tx=0x7f0d60018000 comp rx=0 tx=0).stop 2023-10-09T06:36:57.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.306+0000 7f0d6eec8700 1 -- 172.21.15.163:0/2076601491 shutdown_connections 2023-10-09T06:36:57.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.306+0000 7f0d6eec8700 1 --2- 172.21.15.163:0/2076601491 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0d4c067320 0x7f0d4c0697d0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:57.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.306+0000 7f0d6eec8700 1 --2- 172.21.15.163:0/2076601491 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d68159c60 0x7f0d680b3df0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:57.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.306+0000 7f0d6eec8700 1 --2- 172.21.15.163:0/2076601491 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d68069560 0x7f0d68149310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:57.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.306+0000 7f0d6eec8700 1 --2- 172.21.15.163:0/2076601491 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d68159890 0x7f0d680cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:57.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.306+0000 7f0d6eec8700 1 -- 172.21.15.163:0/2076601491 >> 172.21.15.163:0/2076601491 conn(0x7f0d68068e20 msgr2=0x7f0d68157f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:57.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.306+0000 7f0d6eec8700 1 -- 172.21.15.163:0/2076601491 shutdown_connections 2023-10-09T06:36:57.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.306+0000 7f0d6eec8700 1 -- 172.21.15.163:0/2076601491 wait complete. 2023-10-09T06:36:57.319 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-09T06:36:57.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.434+0000 7f37ded15700 1 Processor -- start 2023-10-09T06:36:57.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.434+0000 7f37ded15700 1 -- start start 2023-10-09T06:36:57.438 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.435+0000 7f37ded15700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f37d80af170 0x7f37d806e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:57.438 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.435+0000 7f37ded15700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37d806eab0 0x7f37d806bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:57.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.435+0000 7f37ded15700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f37d80690b0 0x7f37d806c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:57.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.435+0000 7f37ded15700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37d806f4f0 con 0x7f37d80af170 2023-10-09T06:36:57.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.435+0000 7f37ded15700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37d806ef30 con 0x7f37d80690b0 2023-10-09T06:36:57.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.435+0000 7f37ded15700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d806f210 con 0x7f37d806eab0 2023-10-09T06:36:57.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.435+0000 7f37ddd13700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f37d80af170 0x7f37d806e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:57.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.435+0000 7f37ddd13700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f37d80af170 0x7f37d806e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46386/0 (socket says 172.21.15.163:46386) 2023-10-09T06:36:57.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.435+0000 7f37ddd13700 1 -- 172.21.15.163:0/1909556627 learned_addr learned my addr 172.21.15.163:0/1909556627 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:57.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.435+0000 7f37de514700 1 --2- 172.21.15.163:0/1909556627 >> v2:172.21.15.181:3300/0 conn(0x7f37d80690b0 0x7f37d806c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:57.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.435+0000 7f37dd512700 1 --2- 172.21.15.163:0/1909556627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37d806eab0 0x7f37d806bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:57.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.435+0000 7f37ddd13700 1 -- 172.21.15.163:0/1909556627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37d806eab0 msgr2=0x7f37d806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:57.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.435+0000 7f37ddd13700 1 --2- 172.21.15.163:0/1909556627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37d806eab0 0x7f37d806bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:57.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.435+0000 7f37ddd13700 1 -- 172.21.15.163:0/1909556627 >> v2:172.21.15.181:3300/0 conn(0x7f37d80690b0 msgr2=0x7f37d806c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:57.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.435+0000 7f37ddd13700 1 --2- 172.21.15.163:0/1909556627 >> v2:172.21.15.181:3300/0 conn(0x7f37d80690b0 0x7f37d806c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:57.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.436+0000 7f37ddd13700 1 -- 172.21.15.163:0/1909556627 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d80b0220 con 0x7f37d80af170 2023-10-09T06:36:57.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.436+0000 7f37ddd13700 1 --2- 172.21.15.163:0/1909556627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37d80af170 0x7f37d806e570 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f37d8063d40 tx=0x7f37d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3d0199d7760b2c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:57.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.436+0000 7f37dcd11700 1 -- 172.21.15.163:0/1909556627 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d400b860 con 0x7f37d80af170 2023-10-09T06:36:57.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.436+0000 7f37dcd11700 1 -- 172.21.15.163:0/1909556627 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d400b9c0 con 0x7f37d80af170 2023-10-09T06:36:57.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.436+0000 7f37dcd11700 1 -- 172.21.15.163:0/1909556627 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d400bd60 con 0x7f37d80af170 2023-10-09T06:36:57.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.436+0000 7f37ded15700 1 -- 172.21.15.163:0/1909556627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37d80af170 msgr2=0x7f37d806e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:57.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.436+0000 7f37ded15700 1 --2- 172.21.15.163:0/1909556627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37d80af170 0x7f37d806e570 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f37d8063d40 tx=0x7f37d4009250 comp rx=0 tx=0).stop 2023-10-09T06:36:57.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.436+0000 7f37ded15700 1 -- 172.21.15.163:0/1909556627 shutdown_connections 2023-10-09T06:36:57.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.436+0000 7f37ded15700 1 --2- 172.21.15.163:0/1909556627 >> v2:172.21.15.181:3300/0 conn(0x7f37d80690b0 0x7f37d806c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:57.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.436+0000 7f37ded15700 1 --2- 172.21.15.163:0/1909556627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37d80af170 0x7f37d806e570 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:57.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.436+0000 7f37ded15700 1 --2- 172.21.15.163:0/1909556627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37d806eab0 0x7f37d806bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:57.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.436+0000 7f37ded15700 1 -- 172.21.15.163:0/1909556627 >> 172.21.15.163:0/1909556627 conn(0x7f37d806b490 msgr2=0x7f37d80cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:57.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.436+0000 7f37ded15700 1 -- 172.21.15.163:0/1909556627 shutdown_connections 2023-10-09T06:36:57.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.436+0000 7f37ded15700 1 -- 172.21.15.163:0/1909556627 wait complete. 2023-10-09T06:36:57.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.437+0000 7f37ded15700 1 Processor -- start 2023-10-09T06:36:57.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.438+0000 7f37ded15700 1 -- start start 2023-10-09T06:36:57.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.438+0000 7f37ded15700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37d80690b0 0x7f37d8155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:57.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.438+0000 7f37ded15700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37d80af170 0x7f37d816b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:57.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.438+0000 7f37ded15700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37d81555a0 0x7f37d816b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:57.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.438+0000 7f37ded15700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d815f560 con 0x7f37d80690b0 2023-10-09T06:36:57.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.438+0000 7f37ded15700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d815c040 con 0x7f37d81555a0 2023-10-09T06:36:57.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.438+0000 7f37ded15700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d815c2d0 con 0x7f37d80af170 2023-10-09T06:36:57.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.438+0000 7f37ddd13700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37d80690b0 0x7f37d8155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:57.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.438+0000 7f37ddd13700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37d80690b0 0x7f37d8155060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46402/0 (socket says 172.21.15.163:46402) 2023-10-09T06:36:57.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.438+0000 7f37ddd13700 1 -- 172.21.15.163:0/1682243323 learned_addr learned my addr 172.21.15.163:0/1682243323 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:57.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.438+0000 7f37dd512700 1 --2- 172.21.15.163:0/1682243323 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37d80af170 0x7f37d816b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:57.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.438+0000 7f37de514700 1 --2- 172.21.15.163:0/1682243323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37d81555a0 0x7f37d816b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:57.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.439+0000 7f37ddd13700 1 -- 172.21.15.163:0/1682243323 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37d80af170 msgr2=0x7f37d816b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:57.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.439+0000 7f37ddd13700 1 --2- 172.21.15.163:0/1682243323 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37d80af170 0x7f37d816b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:57.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.439+0000 7f37ddd13700 1 -- 172.21.15.163:0/1682243323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37d81555a0 msgr2=0x7f37d816b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:57.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.439+0000 7f37ddd13700 1 --2- 172.21.15.163:0/1682243323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37d81555a0 0x7f37d816b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:57.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.439+0000 7f37ddd13700 1 -- 172.21.15.163:0/1682243323 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d40094f0 con 0x7f37d80690b0 2023-10-09T06:36:57.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.439+0000 7f37de514700 1 --2- 172.21.15.163:0/1682243323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37d81555a0 0x7f37d816b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:36:57.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.439+0000 7f37ddd13700 1 --2- 172.21.15.163:0/1682243323 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37d80690b0 0x7f37d8155060 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f37d4006fd0 tx=0x7f37d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:57.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.439+0000 7f37cdffb700 1 -- 172.21.15.163:0/1682243323 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d4011e30 con 0x7f37d80690b0 2023-10-09T06:36:57.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.439+0000 7f37cdffb700 1 -- 172.21.15.163:0/1682243323 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d4024440 con 0x7f37d80690b0 2023-10-09T06:36:57.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.439+0000 7f37cdffb700 1 -- 172.21.15.163:0/1682243323 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d4024800 con 0x7f37d80690b0 2023-10-09T06:36:57.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.439+0000 7f37ded15700 1 -- 172.21.15.163:0/1682243323 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d80cdb40 con 0x7f37d80690b0 2023-10-09T06:36:57.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.439+0000 7f37ded15700 1 -- 172.21.15.163:0/1682243323 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d815f300 con 0x7f37d80690b0 2023-10-09T06:36:57.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.440+0000 7f37ded15700 1 -- 172.21.15.163:0/1682243323 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d804c3c0 con 0x7f37d80690b0 2023-10-09T06:36:57.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.440+0000 7f37cdffb700 1 -- 172.21.15.163:0/1682243323 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f37d401a420 con 0x7f37d80690b0 2023-10-09T06:36:57.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.441+0000 7f37cdffb700 1 --2- 172.21.15.163:0/1682243323 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f37bc067320 0x7f37bc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:57.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.441+0000 7f37cdffb700 1 -- 172.21.15.163:0/1682243323 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f37d4086b20 con 0x7f37d80690b0 2023-10-09T06:36:57.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.441+0000 7f37dd512700 1 --2- 172.21.15.163:0/1682243323 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f37bc067320 0x7f37bc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:57.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.441+0000 7f37dd512700 1 --2- 172.21.15.163:0/1682243323 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f37bc067320 0x7f37bc0697d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f37d80cd030 tx=0x7f37c8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:57.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.445+0000 7f37cdffb700 1 -- 172.21.15.163:0/1682243323 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37d4053830 con 0x7f37d80690b0 2023-10-09T06:36:57.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.672+0000 7f37ded15700 1 -- 172.21.15.163:0/1682243323 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f37d80af5d0 con 0x7f37d80690b0 2023-10-09T06:36:57.673 INFO:tasks.ceph.mon.a.smithi163.stderr:2023-10-09T06:36:57.673+0000 7ff251c04700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-09T06:36:57.679 INFO:tasks.ceph.mds.c.smithi163.stderr: -783> 2023-10-09T06:36:49.531+0000 7f669bc50700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-09T06:36:57.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.680+0000 7f37cdffb700 1 -- 172.21.15.163:0/1682243323 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v68) v1 ==== 161+0+0 (secure 0 0 0) 0x7f37d4015090 con 0x7f37d80690b0 2023-10-09T06:36:57.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.683+0000 7f37ded15700 1 -- 172.21.15.163:0/1682243323 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f37bc067320 msgr2=0x7f37bc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:57.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.683+0000 7f37ded15700 1 --2- 172.21.15.163:0/1682243323 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f37bc067320 0x7f37bc0697d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f37d80cd030 tx=0x7f37c8008040 comp rx=0 tx=0).stop 2023-10-09T06:36:57.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.683+0000 7f37ded15700 1 -- 172.21.15.163:0/1682243323 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37d80690b0 msgr2=0x7f37d8155060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:57.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.683+0000 7f37ded15700 1 --2- 172.21.15.163:0/1682243323 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37d80690b0 0x7f37d8155060 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f37d4006fd0 tx=0x7f37d4017000 comp rx=0 tx=0).stop 2023-10-09T06:36:57.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.683+0000 7f37ded15700 1 -- 172.21.15.163:0/1682243323 shutdown_connections 2023-10-09T06:36:57.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.683+0000 7f37ded15700 1 --2- 172.21.15.163:0/1682243323 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f37bc067320 0x7f37bc0697d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:57.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.683+0000 7f37ded15700 1 --2- 172.21.15.163:0/1682243323 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37d80690b0 0x7f37d8155060 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:57.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.683+0000 7f37ded15700 1 --2- 172.21.15.163:0/1682243323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37d81555a0 0x7f37d816b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:57.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.683+0000 7f37ded15700 1 --2- 172.21.15.163:0/1682243323 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37d80af170 0x7f37d816b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:57.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.683+0000 7f37ded15700 1 -- 172.21.15.163:0/1682243323 >> 172.21.15.163:0/1682243323 conn(0x7f37d806b490 msgr2=0x7f37d8158f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:57.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.683+0000 7f37ded15700 1 -- 172.21.15.163:0/1682243323 shutdown_connections 2023-10-09T06:36:57.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.683+0000 7f37ded15700 1 -- 172.21.15.163:0/1682243323 wait complete. 2023-10-09T06:36:57.685 INFO:teuthology.orchestra.run.smithi163.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-09T06:36:57.696 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-09T06:36:57.701 INFO:tasks.ceph.mds.c.smithi163.stdout:starting mds.c at 2023-10-09T06:36:57.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.812+0000 7fdca67b0700 1 Processor -- start 2023-10-09T06:36:57.814 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.813+0000 7fdca67b0700 1 -- start start 2023-10-09T06:36:57.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.813+0000 7fdca67b0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdca00af480 0x7fdca00ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:57.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.813+0000 7fdca67b0700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca0157e30 con 0x7fdca0156a60 2023-10-09T06:36:57.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.813+0000 7fdca67b0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca01578b0 con 0x7fdca0156690 2023-10-09T06:36:57.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.813+0000 7fdca67b0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdca006aa00 con 0x7fdca00af480 2023-10-09T06:36:57.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.814+0000 7fdca4d4d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fdca0156a60 0x7fdca006ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:54386/0 (socket says 172.21.15.163:54386) 2023-10-09T06:36:57.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.814+0000 7fdca4d4d700 1 -- 172.21.15.163:0/1807982577 learned_addr learned my addr 172.21.15.163:0/1807982577 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:57.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.814+0000 7fdc9f7fe700 1 --2- 172.21.15.163:0/1807982577 >> v2:172.21.15.181:3300/0 conn(0x7fdca00af480 0x7fdca00ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:57.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.814+0000 7fdc9effd700 1 -- 172.21.15.163:0/1807982577 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2257365193 0 0) 0x7fdca0157e30 con 0x7fdca0156a60 2023-10-09T06:36:57.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.814+0000 7fdc9effd700 1 -- 172.21.15.163:0/1807982577 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc88003570 con 0x7fdca0156a60 2023-10-09T06:36:57.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.814+0000 7fdc9effd700 1 -- 172.21.15.163:0/1807982577 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 328592208 0 0) 0x7fdc88003570 con 0x7fdca0156a60 2023-10-09T06:36:57.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.814+0000 7fdc9effd700 1 -- 172.21.15.163:0/1807982577 >> v2:172.21.15.181:3300/0 conn(0x7fdca00af480 msgr2=0x7fdca00ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:57.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.814+0000 7fdc9effd700 1 --2- 172.21.15.163:0/1807982577 >> v2:172.21.15.181:3300/0 conn(0x7fdca00af480 0x7fdca00ca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:57.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.814+0000 7fdc9effd700 1 -- 172.21.15.163:0/1807982577 >> v1:172.21.15.181:6789/0 conn(0x7fdca0156690 legacy=0x7fdca00c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:57.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.814+0000 7fdc9effd700 1 -- 172.21.15.163:0/1807982577 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca0157b50 con 0x7fdca0156a60 2023-10-09T06:36:57.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.814+0000 7fdc9effd700 1 -- 172.21.15.163:0/1807982577 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fdc980017f0 con 0x7fdca0156a60 2023-10-09T06:36:57.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.814+0000 7fdc9effd700 1 -- 172.21.15.163:0/1807982577 >> v1:172.21.15.163:6789/0 conn(0x7fdca0156a60 legacy=0x7fdca006ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:57.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.814+0000 7fdc9effd700 1 --2- 172.21.15.163:0/1807982577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc88003a60 0x7fdc88003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:57.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.814+0000 7fdc9effd700 1 --2- 172.21.15.163:0/1807982577 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc88004810 0x7fdc88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:57.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.814+0000 7fdc9effd700 1 --2- 172.21.15.163:0/1807982577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc880052e0 0x7fdc8800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:57.819 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.814+0000 7fdc9effd700 1 -- 172.21.15.163:0/1807982577 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca006aa00 con 0x7fdc880052e0 2023-10-09T06:36:57.819 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.814+0000 7fdc9effd700 1 -- 172.21.15.163:0/1807982577 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8800c2a0 con 0x7fdc88003a60 2023-10-09T06:36:57.819 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.814+0000 7fdc9effd700 1 -- 172.21.15.163:0/1807982577 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8800c460 con 0x7fdc88004810 2023-10-09T06:36:57.819 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.814+0000 7fdca4d4d700 1 --2- 172.21.15.163:0/1807982577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc880052e0 0x7fdc8800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:57.819 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.814+0000 7fdc9ffff700 1 --2- 172.21.15.163:0/1807982577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc88003a60 0x7fdc88003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:57.819 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.814+0000 7fdc9f7fe700 1 --2- 172.21.15.163:0/1807982577 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc88004810 0x7fdc88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:57.819 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.815+0000 7fdca4d4d700 1 -- 172.21.15.163:0/1807982577 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc88004810 msgr2=0x7fdc88004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:57.819 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.815+0000 7fdca4d4d700 1 --2- 172.21.15.163:0/1807982577 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc88004810 0x7fdc88004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:57.819 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.815+0000 7fdca4d4d700 1 -- 172.21.15.163:0/1807982577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc88003a60 msgr2=0x7fdc88003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:57.819 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.815+0000 7fdca4d4d700 1 --2- 172.21.15.163:0/1807982577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc88003a60 0x7fdc88003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:57.819 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.815+0000 7fdca4d4d700 1 -- 172.21.15.163:0/1807982577 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc88003570 con 0x7fdc880052e0 2023-10-09T06:36:57.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.815+0000 7fdca4d4d700 1 -- 172.21.15.163:0/1807982577 reap_dead start 2023-10-09T06:36:57.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.815+0000 7fdca4d4d700 1 --2- 172.21.15.163:0/1807982577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc880052e0 0x7fdc8800ba60 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800a900 tx=0x7fdca00cccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05b284f9ffd1791 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:57.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.815+0000 7fdc9effd700 1 -- 172.21.15.163:0/1807982577 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc980070d0 con 0x7fdc880052e0 2023-10-09T06:36:57.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.815+0000 7fdc9effd700 1 -- 172.21.15.163:0/1807982577 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc98007230 con 0x7fdc880052e0 2023-10-09T06:36:57.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.815+0000 7fdc9effd700 1 -- 172.21.15.163:0/1807982577 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc980075f0 con 0x7fdc880052e0 2023-10-09T06:36:57.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.815+0000 7fdca67b0700 1 -- 172.21.15.163:0/1807982577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc880052e0 msgr2=0x7fdc8800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:57.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.815+0000 7fdca67b0700 1 --2- 172.21.15.163:0/1807982577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc880052e0 0x7fdc8800ba60 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800a900 tx=0x7fdca00cccd0 comp rx=0 tx=0).stop 2023-10-09T06:36:57.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.815+0000 7fdca67b0700 1 -- 172.21.15.163:0/1807982577 shutdown_connections 2023-10-09T06:36:57.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.815+0000 7fdca67b0700 1 --2- 172.21.15.163:0/1807982577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc880052e0 0x7fdc8800ba60 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:57.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.815+0000 7fdca67b0700 1 -- 172.21.15.163:0/1807982577 >> 172.21.15.163:0/1807982577 conn(0x7fdca00c9870 msgr2=0x7fdca00c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:57.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.815+0000 7fdca67b0700 1 -- 172.21.15.163:0/1807982577 shutdown_connections 2023-10-09T06:36:57.821 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.815+0000 7fdca67b0700 1 -- 172.21.15.163:0/1807982577 wait complete. 2023-10-09T06:36:57.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.816+0000 7fdca67b0700 1 Processor -- start 2023-10-09T06:36:57.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.817+0000 7fdca67b0700 1 -- start start 2023-10-09T06:36:57.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.817+0000 7fdca67b0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc880052e0 0x7fdca0169b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:57.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.817+0000 7fdca67b0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdca015a3c0 0x7fdca015a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:57.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.817+0000 7fdca67b0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdca00c4920 0x7fdca00c4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:57.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.817+0000 7fdca67b0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca015ad70 con 0x7fdc880052e0 2023-10-09T06:36:57.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.817+0000 7fdca67b0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca00c61d0 con 0x7fdca00c4920 2023-10-09T06:36:57.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.817+0000 7fdca67b0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca00c5300 con 0x7fdca015a3c0 2023-10-09T06:36:57.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.817+0000 7fdc9ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc880052e0 0x7fdca0169b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:57.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.817+0000 7fdc9ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc880052e0 0x7fdca0169b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46438/0 (socket says 172.21.15.163:46438) 2023-10-09T06:36:57.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.817+0000 7fdc9ffff700 1 -- 172.21.15.163:0/607762878 learned_addr learned my addr 172.21.15.163:0/607762878 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:57.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.818+0000 7fdca4d4d700 1 --2- 172.21.15.163:0/607762878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdca00c4920 0x7fdca00c4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:57.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.818+0000 7fdc9f7fe700 1 --2- 172.21.15.163:0/607762878 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdca015a3c0 0x7fdca015a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:57.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.818+0000 7fdc9ffff700 1 -- 172.21.15.163:0/607762878 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdca015a3c0 msgr2=0x7fdca015a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:57.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.818+0000 7fdc9ffff700 1 --2- 172.21.15.163:0/607762878 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdca015a3c0 0x7fdca015a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:57.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.818+0000 7fdc9ffff700 1 -- 172.21.15.163:0/607762878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdca00c4920 msgr2=0x7fdca00c4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:57.824 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.818+0000 7fdc9ffff700 1 --2- 172.21.15.163:0/607762878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdca00c4920 0x7fdca00c4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:57.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.818+0000 7fdc9ffff700 1 -- 172.21.15.163:0/607762878 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca00c55c0 con 0x7fdc880052e0 2023-10-09T06:36:57.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.818+0000 7fdc9ffff700 1 --2- 172.21.15.163:0/607762878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc880052e0 0x7fdca0169b50 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fdc90002950 tx=0x7fdc90007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:57.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.818+0000 7fdc87fff700 1 -- 172.21.15.163:0/607762878 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc90002e90 con 0x7fdc880052e0 2023-10-09T06:36:57.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.818+0000 7fdc87fff700 1 -- 172.21.15.163:0/607762878 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9000f8d0 con 0x7fdc880052e0 2023-10-09T06:36:57.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.818+0000 7fdca67b0700 1 -- 172.21.15.163:0/607762878 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca00c58a0 con 0x7fdc880052e0 2023-10-09T06:36:57.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.818+0000 7fdc87fff700 1 -- 172.21.15.163:0/607762878 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9000fc90 con 0x7fdc880052e0 2023-10-09T06:36:57.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.818+0000 7fdca67b0700 1 -- 172.21.15.163:0/607762878 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca00c5e60 con 0x7fdc880052e0 2023-10-09T06:36:57.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.819+0000 7fdca67b0700 1 -- 172.21.15.163:0/607762878 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca004c3c0 con 0x7fdc880052e0 2023-10-09T06:36:57.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.819+0000 7fdc87fff700 1 -- 172.21.15.163:0/607762878 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fdc90006820 con 0x7fdc880052e0 2023-10-09T06:36:57.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.820+0000 7fdc87fff700 1 --2- 172.21.15.163:0/607762878 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdc8c067320 0x7fdc8c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:57.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.820+0000 7fdc87fff700 1 -- 172.21.15.163:0/607762878 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fdc90084e10 con 0x7fdc880052e0 2023-10-09T06:36:57.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.820+0000 7fdc9f7fe700 1 --2- 172.21.15.163:0/607762878 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdc8c067320 0x7fdc8c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:57.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.820+0000 7fdc9f7fe700 1 --2- 172.21.15.163:0/607762878 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdc8c067320 0x7fdc8c0697d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdca00c23a0 tx=0x7fdc94008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:57.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:57.824+0000 7fdc87fff700 1 -- 172.21.15.163:0/607762878 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdc90088030 con 0x7fdc880052e0 2023-10-09T06:36:58.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.050+0000 7fdca67b0700 1 -- 172.21.15.163:0/607762878 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdca006f270 con 0x7fdc880052e0 2023-10-09T06:36:58.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.684+0000 7fdc87fff700 1 -- 172.21.15.163:0/607762878 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v69) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdc90051b00 con 0x7fdc880052e0 2023-10-09T06:36:58.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.686+0000 7fdca67b0700 1 -- 172.21.15.163:0/607762878 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdc8c067320 msgr2=0x7fdc8c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:58.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.686+0000 7fdca67b0700 1 --2- 172.21.15.163:0/607762878 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdc8c067320 0x7fdc8c0697d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdca00c23a0 tx=0x7fdc94008040 comp rx=0 tx=0).stop 2023-10-09T06:36:58.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.686+0000 7fdca67b0700 1 -- 172.21.15.163:0/607762878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc880052e0 msgr2=0x7fdca0169b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:58.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.686+0000 7fdca67b0700 1 --2- 172.21.15.163:0/607762878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc880052e0 0x7fdca0169b50 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fdc90002950 tx=0x7fdc90007d30 comp rx=0 tx=0).stop 2023-10-09T06:36:58.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.686+0000 7fdca67b0700 1 -- 172.21.15.163:0/607762878 shutdown_connections 2023-10-09T06:36:58.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.686+0000 7fdca67b0700 1 --2- 172.21.15.163:0/607762878 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdc8c067320 0x7fdc8c0697d0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:58.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.686+0000 7fdca67b0700 1 --2- 172.21.15.163:0/607762878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc880052e0 0x7fdca0169b50 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:58.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.686+0000 7fdca67b0700 1 --2- 172.21.15.163:0/607762878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdca00c4920 0x7fdca00c4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:58.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.686+0000 7fdca67b0700 1 --2- 172.21.15.163:0/607762878 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdca015a3c0 0x7fdca015a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:58.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.686+0000 7fdca67b0700 1 -- 172.21.15.163:0/607762878 >> 172.21.15.163:0/607762878 conn(0x7fdca00c9870 msgr2=0x7fdca00c23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:58.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.686+0000 7fdca67b0700 1 -- 172.21.15.163:0/607762878 shutdown_connections 2023-10-09T06:36:58.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.686+0000 7fdca67b0700 1 -- 172.21.15.163:0/607762878 wait complete. 2023-10-09T06:36:58.702 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-09T06:36:58.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.863+0000 7fc3fa7f0700 1 Processor -- start 2023-10-09T06:36:58.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.863+0000 7fc3fa7f0700 1 -- start start 2023-10-09T06:36:58.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.863+0000 7fc3fa7f0700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc3f406b000 0x7fc3f40adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:58.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.863+0000 7fc3fa7f0700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3f406a0b0 con 0x7fc3f41564e0 2023-10-09T06:36:58.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.863+0000 7fc3fa7f0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3f40694a0 con 0x7fc3f41568b0 2023-10-09T06:36:58.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.863+0000 7fc3fa7f0700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3f4069780 con 0x7fc3f406b000 2023-10-09T06:36:58.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.863+0000 7fc3f3fff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fc3f41564e0 0x7fc3f406a8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:49848/0 (socket says 172.21.15.163:49848) 2023-10-09T06:36:58.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.863+0000 7fc3f3fff700 1 -- 172.21.15.163:0/1973015270 learned_addr learned my addr 172.21.15.163:0/1973015270 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:58.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.863+0000 7fc3f37fe700 1 --2- 172.21.15.163:0/1973015270 >> v2:172.21.15.163:3300/0 conn(0x7fc3f406b000 0x7fc3f40adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:58.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.863+0000 7fc3f2ffd700 1 -- 172.21.15.163:0/1973015270 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1942628756 0 0) 0x7fc3f406a0b0 con 0x7fc3f41564e0 2023-10-09T06:36:58.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.863+0000 7fc3f2ffd700 1 -- 172.21.15.163:0/1973015270 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3dc003570 con 0x7fc3f41564e0 2023-10-09T06:36:58.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.863+0000 7fc3f37fe700 1 -- 172.21.15.163:0/1973015270 >> v1:172.21.15.181:6789/0 conn(0x7fc3f41568b0 legacy=0x7fc3f406f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:58.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.863+0000 7fc3f37fe700 1 -- 172.21.15.163:0/1973015270 >> v1:172.21.15.163:6789/0 conn(0x7fc3f41564e0 legacy=0x7fc3f406a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:58.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.863+0000 7fc3f37fe700 1 -- 172.21.15.163:0/1973015270 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f4069a60 con 0x7fc3f406b000 2023-10-09T06:36:58.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.863+0000 7fc3f37fe700 1 --2- 172.21.15.163:0/1973015270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3f406b000 0x7fc3f40adfb0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fc3e80098e0 tx=0x7fc3e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83f36f35b67eb31 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:58.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.863+0000 7fc3f2ffd700 1 -- 172.21.15.163:0/1973015270 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8011630 con 0x7fc3f406b000 2023-10-09T06:36:58.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.863+0000 7fc3f2ffd700 1 -- 172.21.15.163:0/1973015270 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e8011790 con 0x7fc3f406b000 2023-10-09T06:36:58.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.863+0000 7fc3f2ffd700 1 -- 172.21.15.163:0/1973015270 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8011b30 con 0x7fc3f406b000 2023-10-09T06:36:58.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.864+0000 7fc3fa7f0700 1 -- 172.21.15.163:0/1973015270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3f406b000 msgr2=0x7fc3f40adfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:58.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.864+0000 7fc3fa7f0700 1 --2- 172.21.15.163:0/1973015270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3f406b000 0x7fc3f40adfb0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fc3e80098e0 tx=0x7fc3e80091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:58.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.864+0000 7fc3fa7f0700 1 -- 172.21.15.163:0/1973015270 shutdown_connections 2023-10-09T06:36:58.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.864+0000 7fc3fa7f0700 1 --2- 172.21.15.163:0/1973015270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3f406b000 0x7fc3f40adfb0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:58.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.864+0000 7fc3fa7f0700 1 -- 172.21.15.163:0/1973015270 >> 172.21.15.163:0/1973015270 conn(0x7fc3f40cd640 msgr2=0x7fc3f40cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:58.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.864+0000 7fc3fa7f0700 1 -- 172.21.15.163:0/1973015270 shutdown_connections 2023-10-09T06:36:58.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.864+0000 7fc3fa7f0700 1 -- 172.21.15.163:0/1973015270 wait complete. 2023-10-09T06:36:58.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.865+0000 7fc3fa7f0700 1 Processor -- start 2023-10-09T06:36:58.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.865+0000 7fc3fa7f0700 1 -- start start 2023-10-09T06:36:58.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.866+0000 7fc3fa7f0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3f406b000 0x7fc3f4167fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:58.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.866+0000 7fc3fa7f0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3f41564e0 0x7fc3f414dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:58.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.866+0000 7fc3fa7f0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3f41568b0 0x7fc3f414e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:58.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.866+0000 7fc3fa7f0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f4156ef0 con 0x7fc3f41564e0 2023-10-09T06:36:58.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.866+0000 7fc3fa7f0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f414eed0 con 0x7fc3f406b000 2023-10-09T06:36:58.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.866+0000 7fc3fa7f0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f4168f00 con 0x7fc3f41568b0 2023-10-09T06:36:58.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.866+0000 7fc3f37fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3f41564e0 0x7fc3f414dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:58.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.866+0000 7fc3f37fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3f41564e0 0x7fc3f414dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51350/0 (socket says 172.21.15.163:51350) 2023-10-09T06:36:58.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.866+0000 7fc3f37fe700 1 -- 172.21.15.163:0/585962027 learned_addr learned my addr 172.21.15.163:0/585962027 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:58.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.866+0000 7fc3f37fe700 1 -- 172.21.15.163:0/585962027 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3f41568b0 msgr2=0x7fc3f414e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:58.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.866+0000 7fc3f3fff700 1 --2- 172.21.15.163:0/585962027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3f406b000 0x7fc3f4167fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:58.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.866+0000 7fc3f37fe700 1 --2- 172.21.15.163:0/585962027 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3f41568b0 0x7fc3f414e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:58.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.866+0000 7fc3f37fe700 1 -- 172.21.15.163:0/585962027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3f406b000 msgr2=0x7fc3f4167fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:58.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.866+0000 7fc3f37fe700 1 --2- 172.21.15.163:0/585962027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3f406b000 0x7fc3f4167fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:58.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.866+0000 7fc3f37fe700 1 -- 172.21.15.163:0/585962027 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e8009420 con 0x7fc3f41564e0 2023-10-09T06:36:58.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.866+0000 7fc3f37fe700 1 --2- 172.21.15.163:0/585962027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3f41564e0 0x7fc3f414dc10 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fc3e80091e0 tx=0x7fc3e8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:58.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.866+0000 7fc3dbfff700 1 -- 172.21.15.163:0/585962027 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8000d10 con 0x7fc3f41564e0 2023-10-09T06:36:58.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.866+0000 7fc3dbfff700 1 -- 172.21.15.163:0/585962027 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e8020070 con 0x7fc3f41564e0 2023-10-09T06:36:58.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.866+0000 7fc3fa7f0700 1 -- 172.21.15.163:0/585962027 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f4168660 con 0x7fc3f41564e0 2023-10-09T06:36:58.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.867+0000 7fc3dbfff700 1 -- 172.21.15.163:0/585962027 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e80276d0 con 0x7fc3f41564e0 2023-10-09T06:36:58.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.867+0000 7fc3fa7f0700 1 -- 172.21.15.163:0/585962027 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f4168c20 con 0x7fc3f41564e0 2023-10-09T06:36:58.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.867+0000 7fc3fa7f0700 1 -- 172.21.15.163:0/585962027 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f4157060 con 0x7fc3f41564e0 2023-10-09T06:36:58.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.868+0000 7fc3dbfff700 1 -- 172.21.15.163:0/585962027 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc3e8027b00 con 0x7fc3f41564e0 2023-10-09T06:36:58.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.868+0000 7fc3dbfff700 1 --2- 172.21.15.163:0/585962027 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc3e0067320 0x7fc3e00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:58.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.868+0000 7fc3dbfff700 1 -- 172.21.15.163:0/585962027 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fc3e8022070 con 0x7fc3f41564e0 2023-10-09T06:36:58.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.869+0000 7fc3f3fff700 1 --2- 172.21.15.163:0/585962027 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc3e0067320 0x7fc3e00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:58.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.869+0000 7fc3f3fff700 1 --2- 172.21.15.163:0/585962027 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc3e0067320 0x7fc3e00697d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc3e40058d0 tx=0x7fc3e4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:58.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:58.873+0000 7fc3dbfff700 1 -- 172.21.15.163:0/585962027 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc3e8053a90 con 0x7fc3f41564e0 2023-10-09T06:36:59.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.061+0000 7fc3fa7f0700 1 -- 172.21.15.163:0/585962027 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc3f40718b0 con 0x7fc3f41564e0 2023-10-09T06:36:59.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.695+0000 7fc3dbfff700 1 -- 172.21.15.163:0/585962027 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v103) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc3e80570b0 con 0x7fc3f41564e0 2023-10-09T06:36:59.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.699+0000 7fc3fa7f0700 1 -- 172.21.15.163:0/585962027 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc3e0067320 msgr2=0x7fc3e00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:59.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.699+0000 7fc3fa7f0700 1 --2- 172.21.15.163:0/585962027 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc3e0067320 0x7fc3e00697d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc3e40058d0 tx=0x7fc3e4009040 comp rx=0 tx=0).stop 2023-10-09T06:36:59.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.699+0000 7fc3fa7f0700 1 -- 172.21.15.163:0/585962027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3f41564e0 msgr2=0x7fc3f414dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:59.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.700+0000 7fc3fa7f0700 1 --2- 172.21.15.163:0/585962027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3f41564e0 0x7fc3f414dc10 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fc3e80091e0 tx=0x7fc3e8016000 comp rx=0 tx=0).stop 2023-10-09T06:36:59.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.700+0000 7fc3fa7f0700 1 -- 172.21.15.163:0/585962027 shutdown_connections 2023-10-09T06:36:59.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.700+0000 7fc3fa7f0700 1 --2- 172.21.15.163:0/585962027 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc3e0067320 0x7fc3e00697d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:59.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.700+0000 7fc3fa7f0700 1 --2- 172.21.15.163:0/585962027 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3f41568b0 0x7fc3f414e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:59.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.700+0000 7fc3fa7f0700 1 --2- 172.21.15.163:0/585962027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3f406b000 0x7fc3f4167fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:59.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.700+0000 7fc3fa7f0700 1 --2- 172.21.15.163:0/585962027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3f41564e0 0x7fc3f414dc10 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:59.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.700+0000 7fc3fa7f0700 1 -- 172.21.15.163:0/585962027 >> 172.21.15.163:0/585962027 conn(0x7fc3f40cd640 msgr2=0x7fc3f40b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:59.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.700+0000 7fc3fa7f0700 1 -- 172.21.15.163:0/585962027 shutdown_connections 2023-10-09T06:36:59.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.700+0000 7fc3fa7f0700 1 -- 172.21.15.163:0/585962027 wait complete. 2023-10-09T06:36:59.701 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_metadata' removed 2023-10-09T06:36:59.714 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-09T06:36:59.836 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.836+0000 7f3b27ef7700 1 Processor -- start 2023-10-09T06:36:59.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.836+0000 7f3b27ef7700 1 -- start start 2023-10-09T06:36:59.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.837+0000 7f3b27ef7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b20159ec0 0x7f3b200b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:59.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.837+0000 7f3b27ef7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3b200b53d0 0x7f3b200cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:59.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.837+0000 7f3b27ef7700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b200b5b00 con 0x7f3b200cce40 2023-10-09T06:36:59.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.837+0000 7f3b27ef7700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b200b5da0 con 0x7f3b200b53d0 2023-10-09T06:36:59.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.837+0000 7f3b27ef7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b200b6080 con 0x7f3b20159ec0 2023-10-09T06:36:59.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.837+0000 7f3b26494700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f3b200cce40 0x7f3b200cd280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:49852/0 (socket says 172.21.15.163:49852) 2023-10-09T06:36:59.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.837+0000 7f3b25492700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3b200b53d0 0x7f3b200cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:59.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.837+0000 7f3b26494700 1 -- 172.21.15.163:0/864439826 learned_addr learned my addr 172.21.15.163:0/864439826 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:36:59.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.837+0000 7f3b25c93700 1 --2- 172.21.15.163:0/864439826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b20159ec0 0x7f3b200b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:59.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.837+0000 7f3b25492700 1 -- 172.21.15.163:0/864439826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b20159ec0 msgr2=0x7f3b200b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:36:59.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.837+0000 7f3b25492700 1 --2- 172.21.15.163:0/864439826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b20159ec0 0x7f3b200b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:59.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.837+0000 7f3b25492700 1 -- 172.21.15.163:0/864439826 >> v1:172.21.15.163:6789/0 conn(0x7f3b200cce40 legacy=0x7f3b200cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:59.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.837+0000 7f3b25492700 1 -- 172.21.15.163:0/864439826 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b200cc100 con 0x7f3b200b53d0 2023-10-09T06:36:59.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.837+0000 7f3b24c91700 1 -- 172.21.15.163:0/864439826 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2717281549 0 0) 0x7f3b200b5b00 con 0x7f3b200cce40 2023-10-09T06:36:59.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.837+0000 7f3b25492700 1 --2- 172.21.15.163:0/864439826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b200b53d0 0x7f3b200cc900 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f3b100098e0 tx=0x7f3b100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c1a588cbb271ca server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:59.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.838+0000 7f3b24c91700 1 -- 172.21.15.163:0/864439826 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b10009e80 con 0x7f3b200b53d0 2023-10-09T06:36:59.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.838+0000 7f3b24c91700 1 -- 172.21.15.163:0/864439826 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1000b860 con 0x7f3b200b53d0 2023-10-09T06:36:59.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.838+0000 7f3b24c91700 1 -- 172.21.15.163:0/864439826 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1000bbe0 con 0x7f3b200b53d0 2023-10-09T06:36:59.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.838+0000 7f3b27ef7700 1 -- 172.21.15.163:0/864439826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b200b53d0 msgr2=0x7f3b200cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:59.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.838+0000 7f3b27ef7700 1 --2- 172.21.15.163:0/864439826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b200b53d0 0x7f3b200cc900 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f3b100098e0 tx=0x7f3b100091a0 comp rx=0 tx=0).stop 2023-10-09T06:36:59.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.838+0000 7f3b27ef7700 1 -- 172.21.15.163:0/864439826 shutdown_connections 2023-10-09T06:36:59.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.838+0000 7f3b27ef7700 1 --2- 172.21.15.163:0/864439826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b20159ec0 0x7f3b200b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:59.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.838+0000 7f3b27ef7700 1 --2- 172.21.15.163:0/864439826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b200b53d0 0x7f3b200cc900 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:59.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.838+0000 7f3b27ef7700 1 -- 172.21.15.163:0/864439826 >> 172.21.15.163:0/864439826 conn(0x7f3b200999f0 msgr2=0x7f3b200b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:36:59.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.838+0000 7f3b27ef7700 1 -- 172.21.15.163:0/864439826 shutdown_connections 2023-10-09T06:36:59.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.838+0000 7f3b27ef7700 1 -- 172.21.15.163:0/864439826 wait complete. 2023-10-09T06:36:59.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.839+0000 7f3b27ef7700 1 Processor -- start 2023-10-09T06:36:59.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.839+0000 7f3b27ef7700 1 -- start start 2023-10-09T06:36:59.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.840+0000 7f3b27ef7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b200b53d0 0x7f3b20144b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:59.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.840+0000 7f3b27ef7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b200cce40 0x7f3b20146890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:59.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.840+0000 7f3b27ef7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b20159ec0 0x7f3b20146dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:59.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.840+0000 7f3b27ef7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b20147660 con 0x7f3b20159ec0 2023-10-09T06:36:59.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.840+0000 7f3b27ef7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b20147940 con 0x7f3b200cce40 2023-10-09T06:36:59.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.840+0000 7f3b27ef7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b20147c20 con 0x7f3b200b53d0 2023-10-09T06:36:59.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.840+0000 7f3b26494700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b20159ec0 0x7f3b20146dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:59.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.840+0000 7f3b26494700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b20159ec0 0x7f3b20146dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51360/0 (socket says 172.21.15.163:51360) 2023-10-09T06:36:59.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.840+0000 7f3b26494700 1 -- 172.21.15.163:0/3780855277 learned_addr learned my addr 172.21.15.163:0/3780855277 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:36:59.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.840+0000 7f3b25492700 1 --2- 172.21.15.163:0/3780855277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b200cce40 0x7f3b20146890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:59.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.840+0000 7f3b25c93700 1 --2- 172.21.15.163:0/3780855277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b200b53d0 0x7f3b20144b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:59.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.840+0000 7f3b26494700 1 -- 172.21.15.163:0/3780855277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b200b53d0 msgr2=0x7f3b20144b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:59.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.840+0000 7f3b26494700 1 --2- 172.21.15.163:0/3780855277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b200b53d0 0x7f3b20144b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:59.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.840+0000 7f3b26494700 1 -- 172.21.15.163:0/3780855277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b200cce40 msgr2=0x7f3b20146890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:36:59.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.840+0000 7f3b26494700 1 --2- 172.21.15.163:0/3780855277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b200cce40 0x7f3b20146890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:36:59.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.840+0000 7f3b26494700 1 -- 172.21.15.163:0/3780855277 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b10009420 con 0x7f3b20159ec0 2023-10-09T06:36:59.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.841+0000 7f3b26494700 1 --2- 172.21.15.163:0/3780855277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b20159ec0 0x7f3b20146dd0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f3b1800cc90 tx=0x7f3b18001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:59.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.841+0000 7f3b15ffb700 1 -- 172.21.15.163:0/3780855277 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1800cea0 con 0x7f3b20159ec0 2023-10-09T06:36:59.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.841+0000 7f3b15ffb700 1 -- 172.21.15.163:0/3780855277 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1800ab70 con 0x7f3b20159ec0 2023-10-09T06:36:59.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.841+0000 7f3b27ef7700 1 -- 172.21.15.163:0/3780855277 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b20147f00 con 0x7f3b20159ec0 2023-10-09T06:36:59.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.841+0000 7f3b15ffb700 1 -- 172.21.15.163:0/3780855277 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b180070f0 con 0x7f3b20159ec0 2023-10-09T06:36:59.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.841+0000 7f3b27ef7700 1 -- 172.21.15.163:0/3780855277 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b20145080 con 0x7f3b20159ec0 2023-10-09T06:36:59.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.842+0000 7f3b27ef7700 1 -- 172.21.15.163:0/3780855277 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b20063c90 con 0x7f3b20159ec0 2023-10-09T06:36:59.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.842+0000 7f3b15ffb700 1 -- 172.21.15.163:0/3780855277 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3b180074a0 con 0x7f3b20159ec0 2023-10-09T06:36:59.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.842+0000 7f3b15ffb700 1 --2- 172.21.15.163:0/3780855277 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3b04067320 0x7f3b040697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:36:59.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.843+0000 7f3b15ffb700 1 -- 172.21.15.163:0/3780855277 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f3b18084f90 con 0x7f3b20159ec0 2023-10-09T06:36:59.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.843+0000 7f3b25c93700 1 --2- 172.21.15.163:0/3780855277 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3b04067320 0x7f3b040697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:36:59.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.843+0000 7f3b25c93700 1 --2- 172.21.15.163:0/3780855277 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3b04067320 0x7f3b040697d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c006520 tx=0x7f3b1c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:36:59.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:36:59.847+0000 7f3b15ffb700 1 -- 172.21.15.163:0/3780855277 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b18051e00 con 0x7f3b20159ec0 2023-10-09T06:37:00.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.030+0000 7f3b27ef7700 1 -- 172.21.15.163:0/3780855277 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3b200718b0 con 0x7f3b20159ec0 2023-10-09T06:37:00.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.702+0000 7f3b15ffb700 1 -- 172.21.15.163:0/3780855277 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v104) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3b18055420 con 0x7f3b20159ec0 2023-10-09T06:37:00.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.706+0000 7f3b27ef7700 1 -- 172.21.15.163:0/3780855277 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3b04067320 msgr2=0x7f3b040697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:00.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.706+0000 7f3b27ef7700 1 --2- 172.21.15.163:0/3780855277 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3b04067320 0x7f3b040697d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c006520 tx=0x7f3b1c008040 comp rx=0 tx=0).stop 2023-10-09T06:37:00.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.706+0000 7f3b27ef7700 1 -- 172.21.15.163:0/3780855277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b20159ec0 msgr2=0x7f3b20146dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:00.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.706+0000 7f3b27ef7700 1 --2- 172.21.15.163:0/3780855277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b20159ec0 0x7f3b20146dd0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f3b1800cc90 tx=0x7f3b18001a80 comp rx=0 tx=0).stop 2023-10-09T06:37:00.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.707+0000 7f3b27ef7700 1 -- 172.21.15.163:0/3780855277 shutdown_connections 2023-10-09T06:37:00.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.707+0000 7f3b27ef7700 1 --2- 172.21.15.163:0/3780855277 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3b04067320 0x7f3b040697d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:00.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.707+0000 7f3b27ef7700 1 --2- 172.21.15.163:0/3780855277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b20159ec0 0x7f3b20146dd0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:00.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.707+0000 7f3b27ef7700 1 --2- 172.21.15.163:0/3780855277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b200b53d0 0x7f3b20144b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:00.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.707+0000 7f3b27ef7700 1 --2- 172.21.15.163:0/3780855277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b200cce40 0x7f3b20146890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:00.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.707+0000 7f3b27ef7700 1 -- 172.21.15.163:0/3780855277 >> 172.21.15.163:0/3780855277 conn(0x7f3b200999f0 msgr2=0x7f3b200ce320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:00.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.707+0000 7f3b27ef7700 1 -- 172.21.15.163:0/3780855277 shutdown_connections 2023-10-09T06:37:00.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.707+0000 7f3b27ef7700 1 -- 172.21.15.163:0/3780855277 wait complete. 2023-10-09T06:37:00.709 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_data' removed 2023-10-09T06:37:00.721 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-09T06:37:00.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.846+0000 7efc9beaa700 1 Processor -- start 2023-10-09T06:37:00.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.846+0000 7efc9beaa700 1 -- start start 2023-10-09T06:37:00.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.847+0000 7efc9beaa700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7efc94069560 0x7efc940699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:00.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.847+0000 7efc9beaa700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc94069ee0 con 0x7efc94159c60 2023-10-09T06:37:00.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.847+0000 7efc9beaa700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc9415a160 con 0x7efc94159890 2023-10-09T06:37:00.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.847+0000 7efc9beaa700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc940d13a0 con 0x7efc94069560 2023-10-09T06:37:00.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.847+0000 7efc9a447700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7efc94069560 0x7efc940699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:00.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.847+0000 7efc99445700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7efc94159c60 0x7efc9415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:49854/0 (socket says 172.21.15.163:49854) 2023-10-09T06:37:00.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.847+0000 7efc99445700 1 -- 172.21.15.163:0/1409885710 learned_addr learned my addr 172.21.15.163:0/1409885710 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:00.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.847+0000 7efc9a447700 1 -- 172.21.15.163:0/1409885710 >> v1:172.21.15.181:6789/0 conn(0x7efc94159890 legacy=0x7efc9415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:00.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.847+0000 7efc9a447700 1 -- 172.21.15.163:0/1409885710 >> v1:172.21.15.163:6789/0 conn(0x7efc94159c60 legacy=0x7efc9415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:00.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.848+0000 7efc9a447700 1 -- 172.21.15.163:0/1409885710 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc9415e140 con 0x7efc94069560 2023-10-09T06:37:00.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.848+0000 7efc98c44700 1 -- 172.21.15.163:0/1409885710 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3449119955 0 0) 0x7efc94069ee0 con 0x7efc94159c60 2023-10-09T06:37:00.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.848+0000 7efc9a447700 1 --2- 172.21.15.163:0/1409885710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc94069560 0x7efc940699a0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7efc8c0098e0 tx=0x7efc8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5178c55b59e8391e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:00.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.848+0000 7efc98c44700 1 -- 172.21.15.163:0/1409885710 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c009e80 con 0x7efc94069560 2023-10-09T06:37:00.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.848+0000 7efc98c44700 1 -- 172.21.15.163:0/1409885710 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8c00b820 con 0x7efc94069560 2023-10-09T06:37:00.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.848+0000 7efc98c44700 1 -- 172.21.15.163:0/1409885710 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c00bba0 con 0x7efc94069560 2023-10-09T06:37:00.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.848+0000 7efc9beaa700 1 -- 172.21.15.163:0/1409885710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc94069560 msgr2=0x7efc940699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:00.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.848+0000 7efc9beaa700 1 --2- 172.21.15.163:0/1409885710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc94069560 0x7efc940699a0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7efc8c0098e0 tx=0x7efc8c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:00.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.848+0000 7efc9beaa700 1 -- 172.21.15.163:0/1409885710 shutdown_connections 2023-10-09T06:37:00.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.848+0000 7efc9beaa700 1 --2- 172.21.15.163:0/1409885710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc94069560 0x7efc940699a0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:00.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.848+0000 7efc9beaa700 1 -- 172.21.15.163:0/1409885710 >> 172.21.15.163:0/1409885710 conn(0x7efc94068e20 msgr2=0x7efc940d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:00.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.848+0000 7efc9beaa700 1 -- 172.21.15.163:0/1409885710 shutdown_connections 2023-10-09T06:37:00.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.848+0000 7efc9beaa700 1 -- 172.21.15.163:0/1409885710 wait complete. 2023-10-09T06:37:00.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.849+0000 7efc9beaa700 1 Processor -- start 2023-10-09T06:37:00.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.850+0000 7efc9beaa700 1 -- start start 2023-10-09T06:37:00.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.850+0000 7efc9beaa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc94069560 0x7efc94152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:00.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.850+0000 7efc9beaa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc94159890 0x7efc9416d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:00.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.850+0000 7efc9beaa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc94159c60 0x7efc940c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:00.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.850+0000 7efc9beaa700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9416a0b0 con 0x7efc94159890 2023-10-09T06:37:00.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.850+0000 7efc9beaa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc941532b0 con 0x7efc94069560 2023-10-09T06:37:00.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.850+0000 7efc9beaa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc94152720 con 0x7efc94159c60 2023-10-09T06:37:00.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.850+0000 7efc99445700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc94159890 0x7efc9416d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:00.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.850+0000 7efc99445700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc94159890 0x7efc9416d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51378/0 (socket says 172.21.15.163:51378) 2023-10-09T06:37:00.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.850+0000 7efc99445700 1 -- 172.21.15.163:0/3498446815 learned_addr learned my addr 172.21.15.163:0/3498446815 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:00.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.851+0000 7efc99c46700 1 --2- 172.21.15.163:0/3498446815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc94069560 0x7efc94152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:00.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.851+0000 7efc99445700 1 -- 172.21.15.163:0/3498446815 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc94159c60 msgr2=0x7efc940c5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:00.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.851+0000 7efc99445700 1 --2- 172.21.15.163:0/3498446815 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc94159c60 0x7efc940c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:00.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.851+0000 7efc99445700 1 -- 172.21.15.163:0/3498446815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc94069560 msgr2=0x7efc94152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:00.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.851+0000 7efc99445700 1 --2- 172.21.15.163:0/3498446815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc94069560 0x7efc94152d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:00.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.851+0000 7efc99445700 1 -- 172.21.15.163:0/3498446815 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc8c009420 con 0x7efc94159890 2023-10-09T06:37:00.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.851+0000 7efc99c46700 1 --2- 172.21.15.163:0/3498446815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc94069560 0x7efc94152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:37:00.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.851+0000 7efc99445700 1 --2- 172.21.15.163:0/3498446815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc94159890 0x7efc9416d6e0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7efc8400c5b0 tx=0x7efc84001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:00.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.851+0000 7efc89ffb700 1 -- 172.21.15.163:0/3498446815 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc84005b30 con 0x7efc94159890 2023-10-09T06:37:00.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.851+0000 7efc89ffb700 1 -- 172.21.15.163:0/3498446815 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc84005c90 con 0x7efc94159890 2023-10-09T06:37:00.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.851+0000 7efc9beaa700 1 -- 172.21.15.163:0/3498446815 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc94151520 con 0x7efc94159890 2023-10-09T06:37:00.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.851+0000 7efc89ffb700 1 -- 172.21.15.163:0/3498446815 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc84007300 con 0x7efc94159890 2023-10-09T06:37:00.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.851+0000 7efc9beaa700 1 -- 172.21.15.163:0/3498446815 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc94151ae0 con 0x7efc94159890 2023-10-09T06:37:00.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.852+0000 7efc9beaa700 1 -- 172.21.15.163:0/3498446815 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc9416a2e0 con 0x7efc94159890 2023-10-09T06:37:00.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.852+0000 7efc89ffb700 1 -- 172.21.15.163:0/3498446815 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7efc8401b3b0 con 0x7efc94159890 2023-10-09T06:37:00.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.853+0000 7efc89ffb700 1 --2- 172.21.15.163:0/3498446815 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7efc78067320 0x7efc780697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:00.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.853+0000 7efc89ffb700 1 -- 172.21.15.163:0/3498446815 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 4939+0+0 (secure 0 0 0) 0x7efc84083ee0 con 0x7efc94159890 2023-10-09T06:37:00.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.853+0000 7efc99c46700 1 --2- 172.21.15.163:0/3498446815 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7efc78067320 0x7efc780697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:00.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.853+0000 7efc99c46700 1 --2- 172.21.15.163:0/3498446815 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7efc78067320 0x7efc780697d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7efc90007fd0 tx=0x7efc90009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:00.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:00.857+0000 7efc89ffb700 1 -- 172.21.15.163:0/3498446815 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efc9416a2e0 con 0x7efc94159890 2023-10-09T06:37:01.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.040+0000 7efc9beaa700 1 -- 172.21.15.163:0/3498446815 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7efc9416a2e0 con 0x7efc94159890 2023-10-09T06:37:01.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.044+0000 7efc89ffb700 1 -- 172.21.15.163:0/3498446815 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7efc9416a2e0 con 0x7efc94159890 2023-10-09T06:37:01.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.046+0000 7efc9beaa700 1 -- 172.21.15.163:0/3498446815 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7efc78067320 msgr2=0x7efc780697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:01.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.046+0000 7efc9beaa700 1 --2- 172.21.15.163:0/3498446815 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7efc78067320 0x7efc780697d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7efc90007fd0 tx=0x7efc90009040 comp rx=0 tx=0).stop 2023-10-09T06:37:01.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.046+0000 7efc9beaa700 1 -- 172.21.15.163:0/3498446815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc94159890 msgr2=0x7efc9416d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:01.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.046+0000 7efc9beaa700 1 --2- 172.21.15.163:0/3498446815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc94159890 0x7efc9416d6e0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7efc8400c5b0 tx=0x7efc84001070 comp rx=0 tx=0).stop 2023-10-09T06:37:01.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.046+0000 7efc9beaa700 1 -- 172.21.15.163:0/3498446815 shutdown_connections 2023-10-09T06:37:01.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.046+0000 7efc9beaa700 1 --2- 172.21.15.163:0/3498446815 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7efc78067320 0x7efc780697d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:01.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.046+0000 7efc9beaa700 1 --2- 172.21.15.163:0/3498446815 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc94159c60 0x7efc940c5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:01.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.046+0000 7efc9beaa700 1 --2- 172.21.15.163:0/3498446815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc94069560 0x7efc94152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:01.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.046+0000 7efc9beaa700 1 --2- 172.21.15.163:0/3498446815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc94159890 0x7efc9416d6e0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:01.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.046+0000 7efc9beaa700 1 -- 172.21.15.163:0/3498446815 >> 172.21.15.163:0/3498446815 conn(0x7efc94068e20 msgr2=0x7efc94069930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:01.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.046+0000 7efc9beaa700 1 -- 172.21.15.163:0/3498446815 shutdown_connections 2023-10-09T06:37:01.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.046+0000 7efc9beaa700 1 -- 172.21.15.163:0/3498446815 wait complete. 2023-10-09T06:37:01.058 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup' 2023-10-09T06:37:01.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.173+0000 7f2096021700 1 Processor -- start 2023-10-09T06:37:01.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.173+0000 7f2096021700 1 -- start start 2023-10-09T06:37:01.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.174+0000 7f2096021700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2090159890 0x7f209015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:01.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.174+0000 7f2096021700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2090159c60 0x7f209015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:01.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.174+0000 7f2096021700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f209015db10 0x7f2090069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:01.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.174+0000 7f2096021700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f209015a160 con 0x7f2090159c60 2023-10-09T06:37:01.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.174+0000 7f2096021700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20900d13a0 con 0x7f209015db10 2023-10-09T06:37:01.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.174+0000 7f2096021700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f209015e140 con 0x7f2090159890 2023-10-09T06:37:01.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.174+0000 7f2086ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2090159c60 0x7f209015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:01.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.174+0000 7f2086ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2090159c60 0x7f209015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51394/0 (socket says 172.21.15.163:51394) 2023-10-09T06:37:01.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.174+0000 7f2086ffd700 1 -- 172.21.15.163:0/2712222532 learned_addr learned my addr 172.21.15.163:0/2712222532 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:01.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.174+0000 7f208ffff700 1 --2- 172.21.15.163:0/2712222532 >> v2:172.21.15.181:3300/0 conn(0x7f209015db10 0x7f2090069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:01.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.174+0000 7f208f7fe700 1 --2- 172.21.15.163:0/2712222532 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2090159890 0x7f209015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:01.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.174+0000 7f2086ffd700 1 -- 172.21.15.163:0/2712222532 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2090159890 msgr2=0x7f209015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:01.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.174+0000 7f2086ffd700 1 --2- 172.21.15.163:0/2712222532 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2090159890 0x7f209015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:01.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.174+0000 7f2086ffd700 1 -- 172.21.15.163:0/2712222532 >> v2:172.21.15.181:3300/0 conn(0x7f209015db10 msgr2=0x7f2090069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:01.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.174+0000 7f2086ffd700 1 --2- 172.21.15.163:0/2712222532 >> v2:172.21.15.181:3300/0 conn(0x7f209015db10 0x7f2090069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:01.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.174+0000 7f2086ffd700 1 -- 172.21.15.163:0/2712222532 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209015a3d0 con 0x7f2090159c60 2023-10-09T06:37:01.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.174+0000 7f208f7fe700 1 --2- 172.21.15.163:0/2712222532 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2090159890 0x7f209015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:01.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.174+0000 7f2086ffd700 1 --2- 172.21.15.163:0/2712222532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2090159c60 0x7f209015d5d0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f2090089b50 tx=0x7f2078009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f475339aa697f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:01.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.175+0000 7f208effd700 1 -- 172.21.15.163:0/2712222532 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2078009e70 con 0x7f2090159c60 2023-10-09T06:37:01.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.175+0000 7f208effd700 1 -- 172.21.15.163:0/2712222532 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207800b860 con 0x7f2090159c60 2023-10-09T06:37:01.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.175+0000 7f208effd700 1 -- 172.21.15.163:0/2712222532 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207800bc00 con 0x7f2090159c60 2023-10-09T06:37:01.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.175+0000 7f2096021700 1 -- 172.21.15.163:0/2712222532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2090159c60 msgr2=0x7f209015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:01.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.175+0000 7f2096021700 1 --2- 172.21.15.163:0/2712222532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2090159c60 0x7f209015d5d0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f2090089b50 tx=0x7f2078009250 comp rx=0 tx=0).stop 2023-10-09T06:37:01.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.175+0000 7f2096021700 1 -- 172.21.15.163:0/2712222532 shutdown_connections 2023-10-09T06:37:01.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.175+0000 7f2096021700 1 --2- 172.21.15.163:0/2712222532 >> v2:172.21.15.181:3300/0 conn(0x7f209015db10 0x7f2090069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:01.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.175+0000 7f2096021700 1 --2- 172.21.15.163:0/2712222532 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2090159890 0x7f209015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:01.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.175+0000 7f2096021700 1 --2- 172.21.15.163:0/2712222532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2090159c60 0x7f209015d5d0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:01.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.175+0000 7f2096021700 1 -- 172.21.15.163:0/2712222532 >> 172.21.15.163:0/2712222532 conn(0x7f2090068e20 msgr2=0x7f20900d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:01.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.175+0000 7f2096021700 1 -- 172.21.15.163:0/2712222532 shutdown_connections 2023-10-09T06:37:01.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.175+0000 7f2096021700 1 -- 172.21.15.163:0/2712222532 wait complete. 2023-10-09T06:37:01.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.176+0000 7f2096021700 1 Processor -- start 2023-10-09T06:37:01.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.176+0000 7f2096021700 1 -- start start 2023-10-09T06:37:01.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.177+0000 7f2096021700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2090159890 0x7f20900b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:01.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.177+0000 7f2096021700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2090159c60 0x7f20901585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:01.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.177+0000 7f2096021700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f209015db10 0x7f2090158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:01.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.177+0000 7f2096021700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209014dc40 con 0x7f209015db10 2023-10-09T06:37:01.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.177+0000 7f2096021700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20900b60d0 con 0x7f2090159c60 2023-10-09T06:37:01.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.177+0000 7f2096021700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f209016a8b0 con 0x7f2090159890 2023-10-09T06:37:01.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.177+0000 7f208ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f209015db10 0x7f2090158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:01.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.177+0000 7f208ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f209015db10 0x7f2090158ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51404/0 (socket says 172.21.15.163:51404) 2023-10-09T06:37:01.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.177+0000 7f208ffff700 1 -- 172.21.15.163:0/287907831 learned_addr learned my addr 172.21.15.163:0/287907831 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:01.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.177+0000 7f2086ffd700 1 --2- 172.21.15.163:0/287907831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2090159c60 0x7f20901585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:01.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.177+0000 7f208f7fe700 1 --2- 172.21.15.163:0/287907831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2090159890 0x7f20900b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:01.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.177+0000 7f208ffff700 1 -- 172.21.15.163:0/287907831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2090159890 msgr2=0x7f20900b5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:01.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.177+0000 7f208ffff700 1 --2- 172.21.15.163:0/287907831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2090159890 0x7f20900b5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:01.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.177+0000 7f208ffff700 1 -- 172.21.15.163:0/287907831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2090159c60 msgr2=0x7f20901585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:01.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.177+0000 7f208ffff700 1 --2- 172.21.15.163:0/287907831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2090159c60 0x7f20901585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:01.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.177+0000 7f208ffff700 1 -- 172.21.15.163:0/287907831 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20780094f0 con 0x7f209015db10 2023-10-09T06:37:01.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.177+0000 7f208f7fe700 1 --2- 172.21.15.163:0/287907831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2090159890 0x7f20900b5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:01.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.178+0000 7f208ffff700 1 --2- 172.21.15.163:0/287907831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f209015db10 0x7f2090158ae0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f2088009df0 tx=0x7f208800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:01.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.178+0000 7f2087fff700 1 -- 172.21.15.163:0/287907831 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208800caf0 con 0x7f209015db10 2023-10-09T06:37:01.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.178+0000 7f2087fff700 1 -- 172.21.15.163:0/287907831 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208800f040 con 0x7f209015db10 2023-10-09T06:37:01.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.178+0000 7f2096021700 1 -- 172.21.15.163:0/287907831 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2090169f80 con 0x7f209015db10 2023-10-09T06:37:01.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.178+0000 7f2087fff700 1 -- 172.21.15.163:0/287907831 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208800cec0 con 0x7f209015db10 2023-10-09T06:37:01.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.178+0000 7f2096021700 1 -- 172.21.15.163:0/287907831 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209016a540 con 0x7f209015db10 2023-10-09T06:37:01.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.179+0000 7f2096021700 1 -- 172.21.15.163:0/287907831 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f209004c3c0 con 0x7f209015db10 2023-10-09T06:37:01.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.179+0000 7f2087fff700 1 -- 172.21.15.163:0/287907831 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2088014690 con 0x7f209015db10 2023-10-09T06:37:01.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.180+0000 7f2087fff700 1 --2- 172.21.15.163:0/287907831 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f20700673a0 0x7f2070069850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:01.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.180+0000 7f2087fff700 1 -- 172.21.15.163:0/287907831 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2088085340 con 0x7f209015db10 2023-10-09T06:37:01.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.180+0000 7f208f7fe700 1 --2- 172.21.15.163:0/287907831 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f20700673a0 0x7f2070069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:01.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.181+0000 7f208f7fe700 1 --2- 172.21.15.163:0/287907831 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f20700673a0 0x7f2070069850 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2080014fd0 tx=0x7f2080008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:01.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.184+0000 7f2087fff700 1 -- 172.21.15.163:0/287907831 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2088055c10 con 0x7f209015db10 2023-10-09T06:37:01.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.441+0000 7f2096021700 1 -- 172.21.15.163:0/287907831 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup"]} v 0) v1 -- 0x7f209016a9f0 con 0x7f209015db10 2023-10-09T06:37:01.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.704+0000 7f2087fff700 1 -- 172.21.15.163:0/287907831 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup"]}]=0 v313) v1 ==== 156+0+0 (secure 0 0 0) 0x7f20880557a0 con 0x7f209015db10 2023-10-09T06:37:01.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.706+0000 7f2096021700 1 -- 172.21.15.163:0/287907831 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f20700673a0 msgr2=0x7f2070069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:01.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.706+0000 7f2096021700 1 --2- 172.21.15.163:0/287907831 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f20700673a0 0x7f2070069850 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2080014fd0 tx=0x7f2080008040 comp rx=0 tx=0).stop 2023-10-09T06:37:01.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.706+0000 7f2096021700 1 -- 172.21.15.163:0/287907831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f209015db10 msgr2=0x7f2090158ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:01.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.706+0000 7f2096021700 1 --2- 172.21.15.163:0/287907831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f209015db10 0x7f2090158ae0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f2088009df0 tx=0x7f208800b040 comp rx=0 tx=0).stop 2023-10-09T06:37:01.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.706+0000 7f2096021700 1 -- 172.21.15.163:0/287907831 shutdown_connections 2023-10-09T06:37:01.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.706+0000 7f2096021700 1 --2- 172.21.15.163:0/287907831 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f20700673a0 0x7f2070069850 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:01.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.706+0000 7f2096021700 1 --2- 172.21.15.163:0/287907831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f209015db10 0x7f2090158ae0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:01.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.706+0000 7f2096021700 1 --2- 172.21.15.163:0/287907831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2090159890 0x7f20900b5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:01.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.706+0000 7f2096021700 1 --2- 172.21.15.163:0/287907831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2090159c60 0x7f20901585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:01.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.706+0000 7f2096021700 1 -- 172.21.15.163:0/287907831 >> 172.21.15.163:0/287907831 conn(0x7f2090068e20 msgr2=0x7f2090154920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:01.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.706+0000 7f2096021700 1 -- 172.21.15.163:0/287907831 shutdown_connections 2023-10-09T06:37:01.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.706+0000 7f2096021700 1 -- 172.21.15.163:0/287907831 wait complete. 2023-10-09T06:37:01.718 INFO:tasks.cephfs_test_runner:Tests the 'fs volume info' command without subvolume group ... ok 2023-10-09T06:37:01.718 INFO:tasks.cephfs_test_runner:Starting test: test_volume_ls (tasks.cephfs.test_volumes.TestVolumes) That the existing and the newly created volumes can be listed and 2023-10-09T06:37:01.719 INFO:tasks.cephfs_test_runner:test_volume_ls (tasks.cephfs.test_volumes.TestVolumes) 2023-10-09T06:37:01.719 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls' 2023-10-09T06:37:01.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.834+0000 7f8cd130f700 1 Processor -- start 2023-10-09T06:37:01.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.834+0000 7f8cd130f700 1 -- start start 2023-10-09T06:37:01.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.835+0000 7f8cd130f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8ccc156680 0x7f8ccc06c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:01.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.835+0000 7f8cd130f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8ccc0af1a0 0x7f8ccc0af620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:01.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.835+0000 7f8cd130f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ccc06be50 con 0x7f8ccc156a50 2023-10-09T06:37:01.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.835+0000 7f8cd130f700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ccc0afb60 con 0x7f8ccc156680 2023-10-09T06:37:01.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.835+0000 7f8cd130f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ccc0c88d0 con 0x7f8ccc0af1a0 2023-10-09T06:37:01.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.835+0000 7f8ccaffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8ccc156680 0x7f8ccc06c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:01.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.835+0000 7f8ccaffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8ccc156680 0x7f8ccc06c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51408/0 (socket says 172.21.15.163:51408) 2023-10-09T06:37:01.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.835+0000 7f8ccaffd700 1 -- 172.21.15.163:0/979254661 learned_addr learned my addr 172.21.15.163:0/979254661 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:01.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.836+0000 7f8ccb7fe700 1 --2- 172.21.15.163:0/979254661 >> v2:172.21.15.181:3300/0 conn(0x7f8ccc0af1a0 0x7f8ccc0af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:01.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.836+0000 7f8ccaffd700 1 -- 172.21.15.163:0/979254661 >> v2:172.21.15.181:3300/0 conn(0x7f8ccc0af1a0 msgr2=0x7f8ccc0af620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:01.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.836+0000 7f8ccaffd700 1 --2- 172.21.15.163:0/979254661 >> v2:172.21.15.181:3300/0 conn(0x7f8ccc0af1a0 0x7f8ccc0af620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:01.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.836+0000 7f8ccaffd700 1 -- 172.21.15.163:0/979254661 >> v1:172.21.15.181:6789/0 conn(0x7f8ccc156a50 legacy=0x7f8ccc0c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:01.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.836+0000 7f8ccaffd700 1 -- 172.21.15.163:0/979254661 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ccc06c3f0 con 0x7f8ccc156680 2023-10-09T06:37:01.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.836+0000 7f8ccaffd700 1 --2- 172.21.15.163:0/979254661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ccc156680 0x7f8ccc06c880 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc0afe30 tx=0x7f8cbc00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b862f2b3eee59f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:01.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.836+0000 7f8cca7fc700 1 -- 172.21.15.163:0/979254661 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc00dba0 con 0x7f8ccc156680 2023-10-09T06:37:01.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.836+0000 7f8cca7fc700 1 -- 172.21.15.163:0/979254661 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cbc00dd00 con 0x7f8ccc156680 2023-10-09T06:37:01.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.836+0000 7f8cca7fc700 1 -- 172.21.15.163:0/979254661 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc0196d0 con 0x7f8ccc156680 2023-10-09T06:37:01.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.836+0000 7f8cd130f700 1 -- 172.21.15.163:0/979254661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ccc156680 msgr2=0x7f8ccc06c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:01.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.837+0000 7f8cd130f700 1 --2- 172.21.15.163:0/979254661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ccc156680 0x7f8ccc06c880 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc0afe30 tx=0x7f8cbc00b4f0 comp rx=0 tx=0).stop 2023-10-09T06:37:01.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.837+0000 7f8cd130f700 1 -- 172.21.15.163:0/979254661 shutdown_connections 2023-10-09T06:37:01.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.837+0000 7f8cd130f700 1 --2- 172.21.15.163:0/979254661 >> v2:172.21.15.181:3300/0 conn(0x7f8ccc0af1a0 0x7f8ccc0af620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:01.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.837+0000 7f8cd130f700 1 --2- 172.21.15.163:0/979254661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ccc156680 0x7f8ccc06c880 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:01.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.837+0000 7f8cd130f700 1 -- 172.21.15.163:0/979254661 >> 172.21.15.163:0/979254661 conn(0x7f8ccc0cdd80 msgr2=0x7f8ccc0c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:01.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.837+0000 7f8cd130f700 1 -- 172.21.15.163:0/979254661 shutdown_connections 2023-10-09T06:37:01.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.837+0000 7f8cd130f700 1 -- 172.21.15.163:0/979254661 wait complete. 2023-10-09T06:37:01.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.838+0000 7f8cd130f700 1 Processor -- start 2023-10-09T06:37:01.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.838+0000 7f8cd130f700 1 -- start start 2023-10-09T06:37:01.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.838+0000 7f8cd130f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ccc0af1a0 0x7f8ccc0b1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:01.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.839+0000 7f8cd130f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ccc156680 0x7f8ccc151450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:01.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.839+0000 7f8cd130f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ccc156a50 0x7f8ccc14cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:01.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.839+0000 7f8cd130f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc151990 con 0x7f8ccc0af1a0 2023-10-09T06:37:01.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.839+0000 7f8cd130f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc16ea90 con 0x7f8ccc156680 2023-10-09T06:37:01.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.839+0000 7f8cd130f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc16df10 con 0x7f8ccc156a50 2023-10-09T06:37:01.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.839+0000 7f8ccaffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ccc0af1a0 0x7f8ccc0b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:01.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.839+0000 7f8ccaffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ccc0af1a0 0x7f8ccc0b1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51418/0 (socket says 172.21.15.163:51418) 2023-10-09T06:37:01.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.839+0000 7f8ccaffd700 1 -- 172.21.15.163:0/1745955438 learned_addr learned my addr 172.21.15.163:0/1745955438 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:01.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.839+0000 7f8cc27fc700 1 --2- 172.21.15.163:0/1745955438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ccc156680 0x7f8ccc151450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:01.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.839+0000 7f8ccb7fe700 1 --2- 172.21.15.163:0/1745955438 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ccc156a50 0x7f8ccc14cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:01.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.839+0000 7f8ccaffd700 1 -- 172.21.15.163:0/1745955438 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ccc156a50 msgr2=0x7f8ccc14cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:01.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.839+0000 7f8ccaffd700 1 --2- 172.21.15.163:0/1745955438 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ccc156a50 0x7f8ccc14cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:01.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.839+0000 7f8ccaffd700 1 -- 172.21.15.163:0/1745955438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ccc156680 msgr2=0x7f8ccc151450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:01.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.839+0000 7f8ccaffd700 1 --2- 172.21.15.163:0/1745955438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ccc156680 0x7f8ccc151450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:01.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.839+0000 7f8ccaffd700 1 -- 172.21.15.163:0/1745955438 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cbc00b790 con 0x7f8ccc0af1a0 2023-10-09T06:37:01.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.839+0000 7f8ccaffd700 1 --2- 172.21.15.163:0/1745955438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ccc0af1a0 0x7f8ccc0b1810 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc06a6a0 tx=0x7f8cbc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:01.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.839+0000 7f8cc37fe700 1 -- 172.21.15.163:0/1745955438 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc010680 con 0x7f8ccc0af1a0 2023-10-09T06:37:01.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.839+0000 7f8cc37fe700 1 -- 172.21.15.163:0/1745955438 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cbc0107e0 con 0x7f8ccc0af1a0 2023-10-09T06:37:01.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.839+0000 7f8cd130f700 1 -- 172.21.15.163:0/1745955438 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ccc16e1f0 con 0x7f8ccc0af1a0 2023-10-09T06:37:01.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.839+0000 7f8cc37fe700 1 -- 172.21.15.163:0/1745955438 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc010ba0 con 0x7f8ccc0af1a0 2023-10-09T06:37:01.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.839+0000 7f8cd130f700 1 -- 172.21.15.163:0/1745955438 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ccc16e7b0 con 0x7f8ccc0af1a0 2023-10-09T06:37:01.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.840+0000 7f8cd130f700 1 -- 172.21.15.163:0/1745955438 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ccc16a5f0 con 0x7f8ccc0af1a0 2023-10-09T06:37:01.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.841+0000 7f8cc37fe700 1 -- 172.21.15.163:0/1745955438 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8cbc0233f0 con 0x7f8ccc0af1a0 2023-10-09T06:37:01.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.841+0000 7f8cc37fe700 1 --2- 172.21.15.163:0/1745955438 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8cac067320 0x7f8cac0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:01.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.841+0000 7f8cc37fe700 1 -- 172.21.15.163:0/1745955438 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f8cbc014070 con 0x7f8ccc0af1a0 2023-10-09T06:37:01.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.841+0000 7f8cc27fc700 1 --2- 172.21.15.163:0/1745955438 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8cac067320 0x7f8cac0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:01.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.841+0000 7f8cc27fc700 1 --2- 172.21.15.163:0/1745955438 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8cac067320 0x7f8cac0697d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc071260 tx=0x7f8cb4007ec0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:01.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:01.845+0000 7f8cc37fe700 1 -- 172.21.15.163:0/1745955438 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8cbc0535c0 con 0x7f8ccc0af1a0 2023-10-09T06:37:02.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.101+0000 7f8cd130f700 1 -- 172.21.15.163:0/1745955438 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls"]} v 0) v1 -- 0x7f8ccc04c3c0 con 0x7f8ccc0af1a0 2023-10-09T06:37:02.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.708+0000 7f8cc37fe700 1 -- 172.21.15.163:0/1745955438 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls"]}]=0 v314) v1 ==== 134+0+0 (secure 0 0 0) 0x7f8cbc056be0 con 0x7f8ccc0af1a0 2023-10-09T06:37:02.710 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.710+0000 7f8cd130f700 1 -- 172.21.15.163:0/1745955438 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8cac067320 msgr2=0x7f8cac0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:02.710 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.710+0000 7f8cd130f700 1 --2- 172.21.15.163:0/1745955438 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8cac067320 0x7f8cac0697d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc071260 tx=0x7f8cb4007ec0 comp rx=0 tx=0).stop 2023-10-09T06:37:02.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.710+0000 7f8cd130f700 1 -- 172.21.15.163:0/1745955438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ccc0af1a0 msgr2=0x7f8ccc0b1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:02.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.710+0000 7f8cd130f700 1 --2- 172.21.15.163:0/1745955438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ccc0af1a0 0x7f8ccc0b1810 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc06a6a0 tx=0x7f8cbc01b000 comp rx=0 tx=0).stop 2023-10-09T06:37:02.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.710+0000 7f8cd130f700 1 -- 172.21.15.163:0/1745955438 shutdown_connections 2023-10-09T06:37:02.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.710+0000 7f8cd130f700 1 --2- 172.21.15.163:0/1745955438 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8cac067320 0x7f8cac0697d0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:02.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.710+0000 7f8cd130f700 1 --2- 172.21.15.163:0/1745955438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ccc0af1a0 0x7f8ccc0b1810 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:02.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.710+0000 7f8cd130f700 1 --2- 172.21.15.163:0/1745955438 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ccc156a50 0x7f8ccc14cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:02.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.710+0000 7f8cd130f700 1 --2- 172.21.15.163:0/1745955438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ccc156680 0x7f8ccc151450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:02.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.710+0000 7f8cd130f700 1 -- 172.21.15.163:0/1745955438 >> 172.21.15.163:0/1745955438 conn(0x7f8ccc0cdd80 msgr2=0x7f8ccc14ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:02.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.710+0000 7f8cd130f700 1 -- 172.21.15.163:0/1745955438 shutdown_connections 2023-10-09T06:37:02.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.710+0000 7f8cd130f700 1 -- 172.21.15.163:0/1745955438 wait complete. 2023-10-09T06:37:02.723 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-09T06:37:02.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.842+0000 7f47400ed700 1 Processor -- start 2023-10-09T06:37:02.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.842+0000 7f47400ed700 1 -- start start 2023-10-09T06:37:02.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.843+0000 7f47400ed700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4738159c60 0x7f473815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:02.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.843+0000 7f47400ed700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f473815db10 0x7f4738069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:02.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.843+0000 7f47400ed700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f473815a160 con 0x7f4738159890 2023-10-09T06:37:02.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.843+0000 7f47400ed700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47380d13a0 con 0x7f473815db10 2023-10-09T06:37:02.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.843+0000 7f47400ed700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473815e140 con 0x7f4738159c60 2023-10-09T06:37:02.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.843+0000 7f473e68a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f473815db10 0x7f4738069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:02.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.843+0000 7f473e68a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f473815db10 0x7f4738069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51428/0 (socket says 172.21.15.163:51428) 2023-10-09T06:37:02.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.843+0000 7f473e68a700 1 -- 172.21.15.163:0/2856044900 learned_addr learned my addr 172.21.15.163:0/2856044900 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:02.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.843+0000 7f473d688700 1 --2- 172.21.15.163:0/2856044900 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4738159c60 0x7f473815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:02.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.844+0000 7f473e68a700 1 -- 172.21.15.163:0/2856044900 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4738159c60 msgr2=0x7f473815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:02.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.844+0000 7f473e68a700 1 --2- 172.21.15.163:0/2856044900 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4738159c60 0x7f473815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:02.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.844+0000 7f473e68a700 1 -- 172.21.15.163:0/2856044900 >> v1:172.21.15.181:6789/0 conn(0x7f4738159890 legacy=0x7f473815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:02.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.844+0000 7f473e68a700 1 -- 172.21.15.163:0/2856044900 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473815a3d0 con 0x7f473815db10 2023-10-09T06:37:02.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.844+0000 7f473e68a700 1 --2- 172.21.15.163:0/2856044900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f473815db10 0x7f4738069560 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f4730009f20 tx=0x7f4730009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a9cfaeb25c87e88 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:02.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.844+0000 7f473ce87700 1 -- 172.21.15.163:0/2856044900 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473000e070 con 0x7f473815db10 2023-10-09T06:37:02.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.844+0000 7f473ce87700 1 -- 172.21.15.163:0/2856044900 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473000ba80 con 0x7f473815db10 2023-10-09T06:37:02.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.844+0000 7f473ce87700 1 -- 172.21.15.163:0/2856044900 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47300092e0 con 0x7f473815db10 2023-10-09T06:37:02.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.844+0000 7f47400ed700 1 -- 172.21.15.163:0/2856044900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f473815db10 msgr2=0x7f4738069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:02.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.844+0000 7f47400ed700 1 --2- 172.21.15.163:0/2856044900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f473815db10 0x7f4738069560 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f4730009f20 tx=0x7f4730009840 comp rx=0 tx=0).stop 2023-10-09T06:37:02.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.844+0000 7f47400ed700 1 -- 172.21.15.163:0/2856044900 shutdown_connections 2023-10-09T06:37:02.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.844+0000 7f47400ed700 1 --2- 172.21.15.163:0/2856044900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f473815db10 0x7f4738069560 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:02.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.844+0000 7f47400ed700 1 --2- 172.21.15.163:0/2856044900 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4738159c60 0x7f473815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:02.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.844+0000 7f47400ed700 1 -- 172.21.15.163:0/2856044900 >> 172.21.15.163:0/2856044900 conn(0x7f4738068e20 msgr2=0x7f47380d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:02.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.844+0000 7f47400ed700 1 -- 172.21.15.163:0/2856044900 shutdown_connections 2023-10-09T06:37:02.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.844+0000 7f47400ed700 1 -- 172.21.15.163:0/2856044900 wait complete. 2023-10-09T06:37:02.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.845+0000 7f47400ed700 1 Processor -- start 2023-10-09T06:37:02.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.846+0000 7f47400ed700 1 -- start start 2023-10-09T06:37:02.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.846+0000 7f47400ed700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4738159890 0x7f473816a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:02.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.846+0000 7f47400ed700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4738159c60 0x7f473814a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:02.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.846+0000 7f47400ed700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f473815db10 0x7f473814bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:02.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.846+0000 7f47400ed700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473814b8e0 con 0x7f4738159890 2023-10-09T06:37:02.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.846+0000 7f47400ed700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473816adc0 con 0x7f473815db10 2023-10-09T06:37:02.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.846+0000 7f47400ed700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473814ac60 con 0x7f4738159c60 2023-10-09T06:37:02.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.846+0000 7f473de89700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4738159890 0x7f473816a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:02.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.846+0000 7f473de89700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4738159890 0x7f473816a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51438/0 (socket says 172.21.15.163:51438) 2023-10-09T06:37:02.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.846+0000 7f473de89700 1 -- 172.21.15.163:0/1671378588 learned_addr learned my addr 172.21.15.163:0/1671378588 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:02.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.846+0000 7f473d688700 1 --2- 172.21.15.163:0/1671378588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4738159c60 0x7f473814a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:02.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.846+0000 7f473e68a700 1 --2- 172.21.15.163:0/1671378588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f473815db10 0x7f473814bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:02.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.847+0000 7f473de89700 1 -- 172.21.15.163:0/1671378588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4738159c60 msgr2=0x7f473814a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:02.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.847+0000 7f473de89700 1 --2- 172.21.15.163:0/1671378588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4738159c60 0x7f473814a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:02.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.847+0000 7f473de89700 1 -- 172.21.15.163:0/1671378588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f473815db10 msgr2=0x7f473814bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:02.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.847+0000 7f473de89700 1 --2- 172.21.15.163:0/1671378588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f473815db10 0x7f473814bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:02.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.847+0000 7f473de89700 1 -- 172.21.15.163:0/1671378588 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4730009ae0 con 0x7f4738159890 2023-10-09T06:37:02.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.847+0000 7f473de89700 1 --2- 172.21.15.163:0/1671378588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4738159890 0x7f473816a880 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f473400c590 tx=0x7f473400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:02.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.847+0000 7f472dffb700 1 -- 172.21.15.163:0/1671378588 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4734011590 con 0x7f4738159890 2023-10-09T06:37:02.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.847+0000 7f472dffb700 1 -- 172.21.15.163:0/1671378588 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47340116f0 con 0x7f4738159890 2023-10-09T06:37:02.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.847+0000 7f47400ed700 1 -- 172.21.15.163:0/1671378588 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f473814af40 con 0x7f4738159890 2023-10-09T06:37:02.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.847+0000 7f472dffb700 1 -- 172.21.15.163:0/1671378588 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4734011ab0 con 0x7f4738159890 2023-10-09T06:37:02.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.847+0000 7f47400ed700 1 -- 172.21.15.163:0/1671378588 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f473814b500 con 0x7f4738159890 2023-10-09T06:37:02.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.848+0000 7f47400ed700 1 -- 172.21.15.163:0/1671378588 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f473804c3c0 con 0x7f4738159890 2023-10-09T06:37:02.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.848+0000 7f472dffb700 1 -- 172.21.15.163:0/1671378588 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f473401b3b0 con 0x7f4738159890 2023-10-09T06:37:02.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.849+0000 7f472dffb700 1 --2- 172.21.15.163:0/1671378588 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f471c067320 0x7f471c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:02.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.849+0000 7f472dffb700 1 -- 172.21.15.163:0/1671378588 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f4734083e80 con 0x7f4738159890 2023-10-09T06:37:02.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.849+0000 7f473d688700 1 --2- 172.21.15.163:0/1671378588 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f471c067320 0x7f471c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:02.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.849+0000 7f473d688700 1 --2- 172.21.15.163:0/1671378588 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f471c067320 0x7f471c0697d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f47380cdc10 tx=0x7f4728008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:02.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:02.853+0000 7f472dffb700 1 -- 172.21.15.163:0/1671378588 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4734050e10 con 0x7f4738159890 2023-10-09T06:37:03.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.036+0000 7f47400ed700 1 -- 172.21.15.163:0/1671378588 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f47380caba0 con 0x7f4738159890 2023-10-09T06:37:03.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.040+0000 7f472dffb700 1 -- 172.21.15.163:0/1671378588 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4734017070 con 0x7f4738159890 2023-10-09T06:37:03.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.042+0000 7f47400ed700 1 -- 172.21.15.163:0/1671378588 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f471c067320 msgr2=0x7f471c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:03.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.042+0000 7f47400ed700 1 --2- 172.21.15.163:0/1671378588 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f471c067320 0x7f471c0697d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f47380cdc10 tx=0x7f4728008040 comp rx=0 tx=0).stop 2023-10-09T06:37:03.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.042+0000 7f47400ed700 1 -- 172.21.15.163:0/1671378588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4738159890 msgr2=0x7f473816a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:03.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.042+0000 7f47400ed700 1 --2- 172.21.15.163:0/1671378588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4738159890 0x7f473816a880 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f473400c590 tx=0x7f473400c950 comp rx=0 tx=0).stop 2023-10-09T06:37:03.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.042+0000 7f47400ed700 1 -- 172.21.15.163:0/1671378588 shutdown_connections 2023-10-09T06:37:03.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.042+0000 7f47400ed700 1 --2- 172.21.15.163:0/1671378588 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f471c067320 0x7f471c0697d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:03.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.042+0000 7f47400ed700 1 --2- 172.21.15.163:0/1671378588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4738159890 0x7f473816a880 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:03.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.042+0000 7f47400ed700 1 --2- 172.21.15.163:0/1671378588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f473815db10 0x7f473814bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:03.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.042+0000 7f47400ed700 1 --2- 172.21.15.163:0/1671378588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4738159c60 0x7f473814a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:03.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.042+0000 7f47400ed700 1 -- 172.21.15.163:0/1671378588 >> 172.21.15.163:0/1671378588 conn(0x7f4738068e20 msgr2=0x7f47381537e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:03.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.042+0000 7f47400ed700 1 -- 172.21.15.163:0/1671378588 shutdown_connections 2023-10-09T06:37:03.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.042+0000 7f47400ed700 1 -- 172.21.15.163:0/1671378588 wait complete. 2023-10-09T06:37:03.054 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-09T06:37:03.054 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:37:03.093 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:37:03.138 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-09T06:37:03.139 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-09T06:37:03.139 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:37:03.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.254+0000 7f6e4f522700 1 Processor -- start 2023-10-09T06:37:03.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.254+0000 7f6e4f522700 1 -- start start 2023-10-09T06:37:03.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.255+0000 7f6e4f522700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6e48159c60 0x7f6e4815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:03.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.255+0000 7f6e4f522700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e4815a160 con 0x7f6e4815db10 2023-10-09T06:37:03.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.255+0000 7f6e4f522700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e480d13a0 con 0x7f6e48159890 2023-10-09T06:37:03.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.255+0000 7f6e4f522700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e4815df50 con 0x7f6e48159c60 2023-10-09T06:37:03.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.255+0000 7f6e4cabd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6e48159c60 0x7f6e4815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:03.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.255+0000 7f6e4dabf700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f6e4815db10 0x7f6e48069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:49862/0 (socket says 172.21.15.163:49862) 2023-10-09T06:37:03.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.255+0000 7f6e4dabf700 1 -- 172.21.15.163:0/3568890791 learned_addr learned my addr 172.21.15.163:0/3568890791 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:03.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.255+0000 7f6e4cabd700 1 -- 172.21.15.163:0/3568890791 >> v1:172.21.15.181:6789/0 conn(0x7f6e48159890 legacy=0x7f6e4815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:03.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.256+0000 7f6e4cabd700 1 -- 172.21.15.163:0/3568890791 >> v1:172.21.15.163:6789/0 conn(0x7f6e4815db10 legacy=0x7f6e48069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:03.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.256+0000 7f6e4cabd700 1 -- 172.21.15.163:0/3568890791 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4815e140 con 0x7f6e48159c60 2023-10-09T06:37:03.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.256+0000 7f6e3ffff700 1 -- 172.21.15.163:0/3568890791 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 302526028 0 0) 0x7f6e4815a160 con 0x7f6e4815db10 2023-10-09T06:37:03.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.256+0000 7f6e4cabd700 1 --2- 172.21.15.163:0/3568890791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e48159c60 0x7f6e4815d5d0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f6e380098e0 tx=0x7f6e380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f36cee8b1deab8 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:03.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.256+0000 7f6e3ffff700 1 -- 172.21.15.163:0/3568890791 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e38009e80 con 0x7f6e48159c60 2023-10-09T06:37:03.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.256+0000 7f6e3ffff700 1 -- 172.21.15.163:0/3568890791 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3800b820 con 0x7f6e48159c60 2023-10-09T06:37:03.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.256+0000 7f6e3ffff700 1 -- 172.21.15.163:0/3568890791 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3800bba0 con 0x7f6e48159c60 2023-10-09T06:37:03.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.256+0000 7f6e4f522700 1 -- 172.21.15.163:0/3568890791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e48159c60 msgr2=0x7f6e4815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:03.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.256+0000 7f6e4f522700 1 --2- 172.21.15.163:0/3568890791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e48159c60 0x7f6e4815d5d0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f6e380098e0 tx=0x7f6e380091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:03.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.256+0000 7f6e4f522700 1 -- 172.21.15.163:0/3568890791 shutdown_connections 2023-10-09T06:37:03.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.256+0000 7f6e4f522700 1 --2- 172.21.15.163:0/3568890791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e48159c60 0x7f6e4815d5d0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:03.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.256+0000 7f6e4f522700 1 -- 172.21.15.163:0/3568890791 >> 172.21.15.163:0/3568890791 conn(0x7f6e48068e20 msgr2=0x7f6e480d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:03.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.256+0000 7f6e4f522700 1 -- 172.21.15.163:0/3568890791 shutdown_connections 2023-10-09T06:37:03.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.256+0000 7f6e4f522700 1 -- 172.21.15.163:0/3568890791 wait complete. 2023-10-09T06:37:03.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.257+0000 7f6e4f522700 1 Processor -- start 2023-10-09T06:37:03.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.257+0000 7f6e4f522700 1 -- start start 2023-10-09T06:37:03.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.258+0000 7f6e4f522700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e48159890 0x7f6e481474b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:03.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.258+0000 7f6e4f522700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e48159c60 0x7f6e480cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:03.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.258+0000 7f6e4f522700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e4815db10 0x7f6e480b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:03.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.258+0000 7f6e4f522700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4816db40 con 0x7f6e4815db10 2023-10-09T06:37:03.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.258+0000 7f6e4f522700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4816cf70 con 0x7f6e48159c60 2023-10-09T06:37:03.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.258+0000 7f6e4f522700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4816d210 con 0x7f6e48159890 2023-10-09T06:37:03.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.258+0000 7f6e4dabf700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e4815db10 0x7f6e480b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:03.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.258+0000 7f6e4dabf700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e4815db10 0x7f6e480b1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51456/0 (socket says 172.21.15.163:51456) 2023-10-09T06:37:03.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.258+0000 7f6e4dabf700 1 -- 172.21.15.163:0/2077903927 learned_addr learned my addr 172.21.15.163:0/2077903927 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:03.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.259+0000 7f6e4cabd700 1 --2- 172.21.15.163:0/2077903927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e48159c60 0x7f6e480cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:03.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.259+0000 7f6e4d2be700 1 --2- 172.21.15.163:0/2077903927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e48159890 0x7f6e481474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:03.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.259+0000 7f6e4dabf700 1 -- 172.21.15.163:0/2077903927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e48159890 msgr2=0x7f6e481474b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:03.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.259+0000 7f6e4dabf700 1 --2- 172.21.15.163:0/2077903927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e48159890 0x7f6e481474b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:03.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.259+0000 7f6e4dabf700 1 -- 172.21.15.163:0/2077903927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e48159c60 msgr2=0x7f6e480cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:03.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.259+0000 7f6e4dabf700 1 --2- 172.21.15.163:0/2077903927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e48159c60 0x7f6e480cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:03.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.259+0000 7f6e4dabf700 1 -- 172.21.15.163:0/2077903927 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e38009420 con 0x7f6e4815db10 2023-10-09T06:37:03.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.259+0000 7f6e4d2be700 1 --2- 172.21.15.163:0/2077903927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e48159890 0x7f6e481474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:03.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.259+0000 7f6e4dabf700 1 --2- 172.21.15.163:0/2077903927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e4815db10 0x7f6e480b1ae0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000c7e0 tx=0x7f6e40007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:03.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.259+0000 7f6e3d7fa700 1 -- 172.21.15.163:0/2077903927 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40005c00 con 0x7f6e4815db10 2023-10-09T06:37:03.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.259+0000 7f6e3d7fa700 1 -- 172.21.15.163:0/2077903927 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e40005d80 con 0x7f6e4815db10 2023-10-09T06:37:03.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.259+0000 7f6e3d7fa700 1 -- 172.21.15.163:0/2077903927 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40007320 con 0x7f6e4815db10 2023-10-09T06:37:03.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.259+0000 7f6e4f522700 1 -- 172.21.15.163:0/2077903927 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4816d4f0 con 0x7f6e4815db10 2023-10-09T06:37:03.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.259+0000 7f6e4f522700 1 -- 172.21.15.163:0/2077903927 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e480cf130 con 0x7f6e4815db10 2023-10-09T06:37:03.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.260+0000 7f6e4f522700 1 -- 172.21.15.163:0/2077903927 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e4816dcb0 con 0x7f6e4815db10 2023-10-09T06:37:03.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.260+0000 7f6e3d7fa700 1 -- 172.21.15.163:0/2077903927 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6e4001b3b0 con 0x7f6e4815db10 2023-10-09T06:37:03.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.261+0000 7f6e3d7fa700 1 --2- 172.21.15.163:0/2077903927 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6e2c067320 0x7f6e2c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:03.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.261+0000 7f6e3d7fa700 1 -- 172.21.15.163:0/2077903927 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f6e40083e70 con 0x7f6e4815db10 2023-10-09T06:37:03.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.261+0000 7f6e4d2be700 1 --2- 172.21.15.163:0/2077903927 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6e2c067320 0x7f6e2c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:03.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.262+0000 7f6e4d2be700 1 --2- 172.21.15.163:0/2077903927 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6e2c067320 0x7f6e2c0697d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6e44001370 tx=0x7f6e44009680 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:03.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.265+0000 7f6e3d7fa700 1 -- 172.21.15.163:0/2077903927 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6e40050e00 con 0x7f6e4815db10 2023-10-09T06:37:03.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.491+0000 7f6e4f522700 1 -- 172.21.15.163:0/2077903927 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e4804c3c0 con 0x7f6e4815db10 2023-10-09T06:37:03.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.491+0000 7f6e3d7fa700 1 -- 172.21.15.163:0/2077903927 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+3538 (secure 0 0 0) 0x7f6e40017070 con 0x7f6e4815db10 2023-10-09T06:37:03.491 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:03.492 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":69,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5714,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3452423203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3452423203},{"type":"v1","addr":"172.21.15.181:6836","nonce":3452423203}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6298,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2097756943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2097756943},{"type":"v1","addr":"172.21.15.181:6837","nonce":2097756943}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":6831,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3240892979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3240892979},{"type":"v1","addr":"172.21.15.163:6834","nonce":3240892979}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":7086,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/448150595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":448150595},{"type":"v1","addr":"172.21.15.163:6835","nonce":448150595}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[]} 2023-10-09T06:37:03.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.493+0000 7f6e4f522700 1 -- 172.21.15.163:0/2077903927 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6e2c067320 msgr2=0x7f6e2c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:03.493 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.493+0000 7f6e4f522700 1 --2- 172.21.15.163:0/2077903927 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6e2c067320 0x7f6e2c0697d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6e44001370 tx=0x7f6e44009680 comp rx=0 tx=0).stop 2023-10-09T06:37:03.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.493+0000 7f6e4f522700 1 -- 172.21.15.163:0/2077903927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e4815db10 msgr2=0x7f6e480b1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:03.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.493+0000 7f6e4f522700 1 --2- 172.21.15.163:0/2077903927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e4815db10 0x7f6e480b1ae0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000c7e0 tx=0x7f6e40007d90 comp rx=0 tx=0).stop 2023-10-09T06:37:03.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.493+0000 7f6e4f522700 1 -- 172.21.15.163:0/2077903927 shutdown_connections 2023-10-09T06:37:03.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.493+0000 7f6e4f522700 1 --2- 172.21.15.163:0/2077903927 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6e2c067320 0x7f6e2c0697d0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:03.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.493+0000 7f6e4f522700 1 --2- 172.21.15.163:0/2077903927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e4815db10 0x7f6e480b1ae0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:03.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.493+0000 7f6e4f522700 1 --2- 172.21.15.163:0/2077903927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e48159890 0x7f6e481474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:03.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.493+0000 7f6e4f522700 1 --2- 172.21.15.163:0/2077903927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e48159c60 0x7f6e480cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:03.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.493+0000 7f6e4f522700 1 -- 172.21.15.163:0/2077903927 >> 172.21.15.163:0/2077903927 conn(0x7f6e48068e20 msgr2=0x7f6e481574d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:03.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.493+0000 7f6e4f522700 1 -- 172.21.15.163:0/2077903927 shutdown_connections 2023-10-09T06:37:03.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.493+0000 7f6e4f522700 1 -- 172.21.15.163:0/2077903927 wait complete. 2023-10-09T06:37:03.495 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 69 2023-10-09T06:37:03.505 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-09T06:37:03.506 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-09T06:37:03.506 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-09T06:37:03.506 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:37:03.506 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-09T06:37:03.506 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-09T06:37:03.506 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-09T06:37:03.507 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:37:03.507 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-09T06:37:03.507 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-09T06:37:03.507 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-09T06:37:03.507 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:37:03.508 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-09T06:37:03.508 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-09T06:37:03.509 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-09T06:37:03.509 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:37:03.514 INFO:tasks.ceph.mds.a:Stopped 2023-10-09T06:37:03.515 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-09T06:37:03.515 INFO:tasks.ceph.mds.c:Stopped 2023-10-09T06:37:03.515 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-09T06:37:03.518 INFO:tasks.ceph.mds.b:Stopped 2023-10-09T06:37:03.518 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-09T06:37:03.519 INFO:tasks.ceph.mds.d:Stopped 2023-10-09T06:37:03.519 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-09T06:37:03.557 INFO:tasks.ceph.mds.a:Started 2023-10-09T06:37:03.557 DEBUG:teuthology.parallel:result is None 2023-10-09T06:37:03.558 INFO:tasks.ceph.mds.c:Started 2023-10-09T06:37:03.558 DEBUG:teuthology.parallel:result is None 2023-10-09T06:37:03.561 INFO:tasks.ceph.mds.b:Started 2023-10-09T06:37:03.562 DEBUG:teuthology.parallel:result is None 2023-10-09T06:37:03.563 INFO:tasks.ceph.mds.d:Started 2023-10-09T06:37:03.564 DEBUG:teuthology.parallel:result is None 2023-10-09T06:37:03.564 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-09T06:37:03.629 INFO:tasks.ceph.mds.a.smithi163.stdout:starting mds.a at 2023-10-09T06:37:03.630 INFO:tasks.ceph.mds.c.smithi163.stdout:starting mds.c at 2023-10-09T06:37:03.645 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-09T06:37:03.645 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-09T06:37:03.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.687+0000 7fc1e8d7c700 1 Processor -- start 2023-10-09T06:37:03.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.687+0000 7fc1e8d7c700 1 -- start start 2023-10-09T06:37:03.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.687+0000 7fc1e8d7c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc1e4159890 0x7fc1e415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:03.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.687+0000 7fc1e8d7c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc1e4159c60 0x7fc1e415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:03.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.687+0000 7fc1e8d7c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1e415a160 con 0x7fc1e415db10 2023-10-09T06:37:03.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.687+0000 7fc1e8d7c700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1e415df50 con 0x7fc1e4159c60 2023-10-09T06:37:03.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.687+0000 7fc1e8d7c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1e40d13a0 con 0x7fc1e4159890 2023-10-09T06:37:03.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.688+0000 7fc1dbfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc1e4159c60 0x7fc1e415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:03.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.688+0000 7fc1dbfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc1e4159c60 0x7fc1e415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51484/0 (socket says 172.21.15.163:51484) 2023-10-09T06:37:03.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.688+0000 7fc1dbfff700 1 -- 172.21.15.163:0/3455509515 learned_addr learned my addr 172.21.15.163:0/3455509515 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:03.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.688+0000 7fc1e259c700 1 --2- 172.21.15.163:0/3455509515 >> v2:172.21.15.181:3300/0 conn(0x7fc1e4159890 0x7fc1e415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:03.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.688+0000 7fc1dbfff700 1 -- 172.21.15.163:0/3455509515 >> v2:172.21.15.181:3300/0 conn(0x7fc1e4159890 msgr2=0x7fc1e415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:03.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.688+0000 7fc1dbfff700 1 --2- 172.21.15.163:0/3455509515 >> v2:172.21.15.181:3300/0 conn(0x7fc1e4159890 0x7fc1e415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:03.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.688+0000 7fc1dbfff700 1 -- 172.21.15.163:0/3455509515 >> v1:172.21.15.181:6789/0 conn(0x7fc1e415db10 legacy=0x7fc1e4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:03.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.688+0000 7fc1dbfff700 1 -- 172.21.15.163:0/3455509515 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e415e140 con 0x7fc1e4159c60 2023-10-09T06:37:03.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.688+0000 7fc1dbfff700 1 --2- 172.21.15.163:0/3455509515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1e4159c60 0x7fc1e415d5d0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40d1090 tx=0x7fc1cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466daecefc29c3ad server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:03.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.688+0000 7fc1e1d9b700 1 -- 172.21.15.163:0/3455509515 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc009e70 con 0x7fc1e4159c60 2023-10-09T06:37:03.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.688+0000 7fc1e1d9b700 1 -- 172.21.15.163:0/3455509515 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1cc00b860 con 0x7fc1e4159c60 2023-10-09T06:37:03.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.688+0000 7fc1e1d9b700 1 -- 172.21.15.163:0/3455509515 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc00bc00 con 0x7fc1e4159c60 2023-10-09T06:37:03.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.689+0000 7fc1e8d7c700 1 -- 172.21.15.163:0/3455509515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1e4159c60 msgr2=0x7fc1e415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:03.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.689+0000 7fc1e8d7c700 1 --2- 172.21.15.163:0/3455509515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1e4159c60 0x7fc1e415d5d0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40d1090 tx=0x7fc1cc009250 comp rx=0 tx=0).stop 2023-10-09T06:37:03.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.689+0000 7fc1e8d7c700 1 -- 172.21.15.163:0/3455509515 shutdown_connections 2023-10-09T06:37:03.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.689+0000 7fc1e8d7c700 1 --2- 172.21.15.163:0/3455509515 >> v2:172.21.15.181:3300/0 conn(0x7fc1e4159890 0x7fc1e415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:03.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.689+0000 7fc1e8d7c700 1 --2- 172.21.15.163:0/3455509515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1e4159c60 0x7fc1e415d5d0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:03.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.689+0000 7fc1e8d7c700 1 -- 172.21.15.163:0/3455509515 >> 172.21.15.163:0/3455509515 conn(0x7fc1e4068e20 msgr2=0x7fc1e40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:03.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.689+0000 7fc1e8d7c700 1 -- 172.21.15.163:0/3455509515 shutdown_connections 2023-10-09T06:37:03.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.689+0000 7fc1e8d7c700 1 -- 172.21.15.163:0/3455509515 wait complete. 2023-10-09T06:37:03.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.690+0000 7fc1e8d7c700 1 Processor -- start 2023-10-09T06:37:03.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.690+0000 7fc1e8d7c700 1 -- start start 2023-10-09T06:37:03.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.691+0000 7fc1e8d7c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1e4159890 0x7fc1e4145a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:03.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.691+0000 7fc1e8d7c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1e4159c60 0x7fc1e4153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:03.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.691+0000 7fc1e8d7c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e415db10 0x7fc1e40c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:03.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.691+0000 7fc1e8d7c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e40b4480 con 0x7fc1e4159890 2023-10-09T06:37:03.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.691+0000 7fc1e8d7c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e4153970 con 0x7fc1e415db10 2023-10-09T06:37:03.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.691+0000 7fc1e8d7c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e40b3590 con 0x7fc1e4159c60 2023-10-09T06:37:03.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.691+0000 7fc1e259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1e4159890 0x7fc1e4145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:03.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.691+0000 7fc1e259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1e4159890 0x7fc1e4145a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51496/0 (socket says 172.21.15.163:51496) 2023-10-09T06:37:03.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.691+0000 7fc1e259c700 1 -- 172.21.15.163:0/1924029546 learned_addr learned my addr 172.21.15.163:0/1924029546 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:03.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.691+0000 7fc1dbfff700 1 --2- 172.21.15.163:0/1924029546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1e4159c60 0x7fc1e4153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:03.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.691+0000 7fc1e2d9d700 1 --2- 172.21.15.163:0/1924029546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e415db10 0x7fc1e40c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:03.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.691+0000 7fc1e259c700 1 -- 172.21.15.163:0/1924029546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1e4159c60 msgr2=0x7fc1e4153430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:03.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.691+0000 7fc1e259c700 1 --2- 172.21.15.163:0/1924029546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1e4159c60 0x7fc1e4153430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:03.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.691+0000 7fc1e259c700 1 -- 172.21.15.163:0/1924029546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e415db10 msgr2=0x7fc1e40c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:03.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.691+0000 7fc1e259c700 1 --2- 172.21.15.163:0/1924029546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e415db10 0x7fc1e40c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:03.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.691+0000 7fc1e259c700 1 -- 172.21.15.163:0/1924029546 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1cc0094f0 con 0x7fc1e4159890 2023-10-09T06:37:03.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.691+0000 7fc1e259c700 1 --2- 172.21.15.163:0/1924029546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1e4159890 0x7fc1e4145a90 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fc1d400ec80 tx=0x7fc1d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:03.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.691+0000 7fc1da7fc700 1 -- 172.21.15.163:0/1924029546 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d400c850 con 0x7fc1e4159890 2023-10-09T06:37:03.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.691+0000 7fc1da7fc700 1 -- 172.21.15.163:0/1924029546 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d400c9b0 con 0x7fc1e4159890 2023-10-09T06:37:03.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.691+0000 7fc1e8d7c700 1 -- 172.21.15.163:0/1924029546 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1e40b3870 con 0x7fc1e4159890 2023-10-09T06:37:03.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.691+0000 7fc1da7fc700 1 -- 172.21.15.163:0/1924029546 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d400cd50 con 0x7fc1e4159890 2023-10-09T06:37:03.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.691+0000 7fc1e8d7c700 1 -- 172.21.15.163:0/1924029546 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e40b3e30 con 0x7fc1e4159890 2023-10-09T06:37:03.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.692+0000 7fc1e8d7c700 1 -- 172.21.15.163:0/1924029546 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1e404c3c0 con 0x7fc1e4159890 2023-10-09T06:37:03.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.693+0000 7fc1da7fc700 1 -- 172.21.15.163:0/1924029546 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc1d4013680 con 0x7fc1e4159890 2023-10-09T06:37:03.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.693+0000 7fc1da7fc700 1 --2- 172.21.15.163:0/1924029546 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc1c4067320 0x7fc1c40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:03.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.693+0000 7fc1da7fc700 1 -- 172.21.15.163:0/1924029546 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fc1d4084ba0 con 0x7fc1e4159890 2023-10-09T06:37:03.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.693+0000 7fc1dbfff700 1 --2- 172.21.15.163:0/1924029546 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc1c4067320 0x7fc1c40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:03.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.694+0000 7fc1dbfff700 1 --2- 172.21.15.163:0/1924029546 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc1c4067320 0x7fc1c40697d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40d1090 tx=0x7fc1cc017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:03.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.697+0000 7fc1da7fc700 1 -- 172.21.15.163:0/1924029546 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc1d4051b30 con 0x7fc1e4159890 2023-10-09T06:37:03.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:03.899+0000 7fc1e8d7c700 1 -- 172.21.15.163:0/1924029546 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc1e415ccc0 con 0x7fc1e4159890 2023-10-09T06:37:04.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.738+0000 7fc1da7fc700 1 -- 172.21.15.163:0/1924029546 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v109) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc1d4055150 con 0x7fc1e4159890 2023-10-09T06:37:04.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.740+0000 7fc1e8d7c700 1 -- 172.21.15.163:0/1924029546 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc1c4067320 msgr2=0x7fc1c40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:04.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.740+0000 7fc1e8d7c700 1 --2- 172.21.15.163:0/1924029546 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc1c4067320 0x7fc1c40697d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40d1090 tx=0x7fc1cc017000 comp rx=0 tx=0).stop 2023-10-09T06:37:04.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.741+0000 7fc1e8d7c700 1 -- 172.21.15.163:0/1924029546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1e4159890 msgr2=0x7fc1e4145a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:04.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.741+0000 7fc1e8d7c700 1 --2- 172.21.15.163:0/1924029546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1e4159890 0x7fc1e4145a90 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fc1d400ec80 tx=0x7fc1d40091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:04.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.741+0000 7fc1e8d7c700 1 -- 172.21.15.163:0/1924029546 shutdown_connections 2023-10-09T06:37:04.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.741+0000 7fc1e8d7c700 1 --2- 172.21.15.163:0/1924029546 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc1c4067320 0x7fc1c40697d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:04.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.741+0000 7fc1e8d7c700 1 --2- 172.21.15.163:0/1924029546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1e4159890 0x7fc1e4145a90 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:04.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.741+0000 7fc1e8d7c700 1 --2- 172.21.15.163:0/1924029546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1e415db10 0x7fc1e40c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:04.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.741+0000 7fc1e8d7c700 1 --2- 172.21.15.163:0/1924029546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1e4159c60 0x7fc1e4153430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:04.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.741+0000 7fc1e8d7c700 1 -- 172.21.15.163:0/1924029546 >> 172.21.15.163:0/1924029546 conn(0x7fc1e4068e20 msgr2=0x7fc1e41543a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:04.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.741+0000 7fc1e8d7c700 1 -- 172.21.15.163:0/1924029546 shutdown_connections 2023-10-09T06:37:04.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.741+0000 7fc1e8d7c700 1 -- 172.21.15.163:0/1924029546 wait complete. 2023-10-09T06:37:04.743 INFO:teuthology.orchestra.run.smithi163.stderr: removed all blocklist entries 2023-10-09T06:37:04.754 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-09T06:37:04.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.872+0000 7f33a84f4700 1 Processor -- start 2023-10-09T06:37:04.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.872+0000 7f33a84f4700 1 -- start start 2023-10-09T06:37:04.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.873+0000 7f33a84f4700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f33a0069560 0x7f33a00699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:04.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.873+0000 7f33a84f4700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a0069ee0 con 0x7f33a0159c60 2023-10-09T06:37:04.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.873+0000 7f33a84f4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a015a160 con 0x7f33a0159890 2023-10-09T06:37:04.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.873+0000 7f33a84f4700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33a00d13a0 con 0x7f33a0069560 2023-10-09T06:37:04.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.873+0000 7f33a6a91700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f33a0069560 0x7f33a00699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:04.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.873+0000 7f33a5a8f700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f33a0159c60 0x7f33a015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:49898/0 (socket says 172.21.15.163:49898) 2023-10-09T06:37:04.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.873+0000 7f33a5a8f700 1 -- 172.21.15.163:0/2643977130 learned_addr learned my addr 172.21.15.163:0/2643977130 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:04.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.873+0000 7f33a6a91700 1 -- 172.21.15.163:0/2643977130 >> v1:172.21.15.181:6789/0 conn(0x7f33a0159890 legacy=0x7f33a015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:04.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.873+0000 7f33a6a91700 1 -- 172.21.15.163:0/2643977130 >> v1:172.21.15.163:6789/0 conn(0x7f33a0159c60 legacy=0x7f33a015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:04.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.873+0000 7f33a6a91700 1 -- 172.21.15.163:0/2643977130 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a015e140 con 0x7f33a0069560 2023-10-09T06:37:04.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.873+0000 7f33a528e700 1 -- 172.21.15.163:0/2643977130 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 279684218 0 0) 0x7f33a0069ee0 con 0x7f33a0159c60 2023-10-09T06:37:04.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.873+0000 7f33a6a91700 1 --2- 172.21.15.163:0/2643977130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33a0069560 0x7f33a00699a0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f3390009e60 tx=0x7f3390009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e625199b82d7b7a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:04.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.873+0000 7f33a528e700 1 -- 172.21.15.163:0/2643977130 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339000e070 con 0x7f33a0069560 2023-10-09T06:37:04.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.873+0000 7f33a528e700 1 -- 172.21.15.163:0/2643977130 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339000bc30 con 0x7f33a0069560 2023-10-09T06:37:04.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.873+0000 7f33a528e700 1 -- 172.21.15.163:0/2643977130 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3390012430 con 0x7f33a0069560 2023-10-09T06:37:04.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.874+0000 7f33a84f4700 1 -- 172.21.15.163:0/2643977130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33a0069560 msgr2=0x7f33a00699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:04.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.874+0000 7f33a84f4700 1 --2- 172.21.15.163:0/2643977130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33a0069560 0x7f33a00699a0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f3390009e60 tx=0x7f3390009780 comp rx=0 tx=0).stop 2023-10-09T06:37:04.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.874+0000 7f33a84f4700 1 -- 172.21.15.163:0/2643977130 shutdown_connections 2023-10-09T06:37:04.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.874+0000 7f33a84f4700 1 --2- 172.21.15.163:0/2643977130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33a0069560 0x7f33a00699a0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:04.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.874+0000 7f33a84f4700 1 -- 172.21.15.163:0/2643977130 >> 172.21.15.163:0/2643977130 conn(0x7f33a0068e20 msgr2=0x7f33a00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:04.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.874+0000 7f33a84f4700 1 -- 172.21.15.163:0/2643977130 shutdown_connections 2023-10-09T06:37:04.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.874+0000 7f33a84f4700 1 -- 172.21.15.163:0/2643977130 wait complete. 2023-10-09T06:37:04.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.875+0000 7f33a84f4700 1 Processor -- start 2023-10-09T06:37:04.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.875+0000 7f33a84f4700 1 -- start start 2023-10-09T06:37:04.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.876+0000 7f33a84f4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33a0069560 0x7f33a01543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:04.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.876+0000 7f33a84f4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33a0159890 0x7f33a015c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:04.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.876+0000 7f33a84f4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33a0159c60 0x7f33a015d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:04.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.876+0000 7f33a84f4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a00ca2f0 con 0x7f33a0159890 2023-10-09T06:37:04.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.876+0000 7f33a84f4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a015c240 con 0x7f33a0069560 2023-10-09T06:37:04.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.876+0000 7f33a84f4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a0154920 con 0x7f33a0159c60 2023-10-09T06:37:04.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.876+0000 7f33a5a8f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33a0159890 0x7f33a015c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:04.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.876+0000 7f33a5a8f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33a0159890 0x7f33a015c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51510/0 (socket says 172.21.15.163:51510) 2023-10-09T06:37:04.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.876+0000 7f33a5a8f700 1 -- 172.21.15.163:0/3797022251 learned_addr learned my addr 172.21.15.163:0/3797022251 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:04.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.876+0000 7f33a5a8f700 1 -- 172.21.15.163:0/3797022251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33a0159c60 msgr2=0x7f33a015d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:04.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.876+0000 7f33a5a8f700 1 --2- 172.21.15.163:0/3797022251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33a0159c60 0x7f33a015d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:04.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.876+0000 7f33a5a8f700 1 -- 172.21.15.163:0/3797022251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33a0069560 msgr2=0x7f33a01543e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:04.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.876+0000 7f33a5a8f700 1 --2- 172.21.15.163:0/3797022251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33a0069560 0x7f33a01543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:04.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.876+0000 7f33a5a8f700 1 -- 172.21.15.163:0/3797022251 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3390009a20 con 0x7f33a0159890 2023-10-09T06:37:04.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.876+0000 7f33a5a8f700 1 --2- 172.21.15.163:0/3797022251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33a0159890 0x7f33a015c6d0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f339400c5e0 tx=0x7f3394001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:04.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.876+0000 7f339e7fc700 1 -- 172.21.15.163:0/3797022251 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3394005b30 con 0x7f33a0159890 2023-10-09T06:37:04.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.877+0000 7f339e7fc700 1 -- 172.21.15.163:0/3797022251 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3394005c90 con 0x7f33a0159890 2023-10-09T06:37:04.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.877+0000 7f339e7fc700 1 -- 172.21.15.163:0/3797022251 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33940072e0 con 0x7f33a0159890 2023-10-09T06:37:04.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.877+0000 7f33a84f4700 1 -- 172.21.15.163:0/3797022251 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a015b670 con 0x7f33a0159890 2023-10-09T06:37:04.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.877+0000 7f33a84f4700 1 -- 172.21.15.163:0/3797022251 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a015bc70 con 0x7f33a0159890 2023-10-09T06:37:04.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.878+0000 7f33a84f4700 1 -- 172.21.15.163:0/3797022251 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a015bff0 con 0x7f33a0159890 2023-10-09T06:37:04.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.878+0000 7f339e7fc700 1 -- 172.21.15.163:0/3797022251 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f339401b3b0 con 0x7f33a0159890 2023-10-09T06:37:04.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.878+0000 7f339e7fc700 1 --2- 172.21.15.163:0/3797022251 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f338c067320 0x7f338c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:04.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.879+0000 7f339e7fc700 1 -- 172.21.15.163:0/3797022251 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f3394083d50 con 0x7f33a0159890 2023-10-09T06:37:04.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.879+0000 7f33a6290700 1 --2- 172.21.15.163:0/3797022251 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f338c067320 0x7f338c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:04.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.879+0000 7f33a6290700 1 --2- 172.21.15.163:0/3797022251 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f338c067320 0x7f338c0697d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3398001660 tx=0x7f3398007b00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:04.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:04.883+0000 7f339e7fc700 1 -- 172.21.15.163:0/3797022251 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f339404b5a0 con 0x7f33a0159890 2023-10-09T06:37:05.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.138+0000 7f33a84f4700 1 -- 172.21.15.163:0/3797022251 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f33a004c3c0 con 0x7f33a0159890 2023-10-09T06:37:05.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.139+0000 7f339e7fc700 1 -- 172.21.15.163:0/3797022251 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3394017020 con 0x7f33a0159890 2023-10-09T06:37:05.139 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:05.140 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:37:05.140 INFO:teuthology.orchestra.run.smithi163.stdout: "auth_dump": [ 2023-10-09T06:37:05.140 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:05.140 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.a", 2023-10-09T06:37:05.140 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNltXtwDxAAlnrWRsfnZDLEmsJVowh3Mg==", 2023-10-09T06:37:05.140 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:05.141 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:37:05.141 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:37:05.141 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:37:05.141 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:05.141 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:05.141 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:05.141 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:05.141 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.b", 2023-10-09T06:37:05.141 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlTdGhGhAAXuldy0HrFd+44sSu/EV3qA==", 2023-10-09T06:37:05.141 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:05.142 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:37:05.142 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:37:05.142 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:37:05.142 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:05.142 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:05.142 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:05.142 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:05.142 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.c", 2023-10-09T06:37:05.142 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlGis7FhAAYG76TiOFX8QkTwjNX2+Ncw==", 2023-10-09T06:37:05.142 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:05.143 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:37:05.143 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:37:05.143 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:37:05.143 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:05.143 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:05.143 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:05.144 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:05.144 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.d", 2023-10-09T06:37:05.144 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlKhRuIRAAfhG5Gpc7klXJloYXepOhsg==", 2023-10-09T06:37:05.144 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:05.144 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:37:05.146 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:37:05.146 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:37:05.146 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:05.146 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:05.146 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:05.146 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:05.146 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.0", 2023-10-09T06:37:05.146 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDcnSNlrQeFOxAA+s6UitrTn5TK1N7kgySoXw==", 2023-10-09T06:37:05.146 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:05.147 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:37:05.150 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:37:05.150 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:05.151 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:05.151 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:05.151 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:05.151 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.1", 2023-10-09T06:37:05.151 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDfnSNlleccORAAnB+lNUFblivdoY44JfELxA==", 2023-10-09T06:37:05.151 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:05.151 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:37:05.151 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:37:05.151 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:05.151 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:05.152 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:05.152 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:05.152 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.2", 2023-10-09T06:37:05.152 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDinSNlFoB4NxAAT0+trRBij5rqgc+F0eWUfQ==", 2023-10-09T06:37:05.152 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:05.152 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:37:05.152 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:37:05.153 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:05.153 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:05.153 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:05.153 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:05.153 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.3", 2023-10-09T06:37:05.153 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDlnSNlRRQ6NhAABoBtrFjSvQujolq0Pxrd1Q==", 2023-10-09T06:37:05.153 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:05.153 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:37:05.153 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:37:05.153 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:05.153 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:05.154 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:05.154 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:05.154 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.4", 2023-10-09T06:37:05.154 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDrnSNl308uOxAA5OJyPLGQmVSJqha5d+nKKQ==", 2023-10-09T06:37:05.154 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:05.154 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:37:05.154 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:37:05.154 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:05.154 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:05.154 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:05.155 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:05.155 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.5", 2023-10-09T06:37:05.155 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDunSNlHnK+OBAAU7l4dCJ82SbGSzBEm8Axzw==", 2023-10-09T06:37:05.155 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:05.155 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:37:05.155 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:37:05.155 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:05.155 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:05.155 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:05.155 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:05.155 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.6", 2023-10-09T06:37:05.155 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDxnSNlwUSLOBAAN5N+F5ho+Co8pBh4Rz+lXQ==", 2023-10-09T06:37:05.155 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:05.156 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:37:05.156 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:37:05.156 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:05.156 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:05.156 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:05.156 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:05.156 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.7", 2023-10-09T06:37:05.156 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQD0nSNl2gxGORAAoE8OLGEifcEx/km8rfSV9g==", 2023-10-09T06:37:05.156 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:05.156 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:37:05.156 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:37:05.157 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:05.157 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:05.157 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:05.157 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:05.157 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.0", 2023-10-09T06:37:05.157 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlnxY5JRAApr6czcpMdQYYtXQ9yg2q6A==", 2023-10-09T06:37:05.157 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:05.157 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:37:05.157 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow r", 2023-10-09T06:37:05.157 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-09T06:37:05.157 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:05.157 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:05.158 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:05.158 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.1", 2023-10-09T06:37:05.158 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlk6ybLRAAuShqTDAtymGdxN8KBdZDDA==", 2023-10-09T06:37:05.158 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:05.158 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:37:05.158 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:37:05.158 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:37:05.158 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:37:05.158 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:05.158 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:05.158 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:05.159 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.2", 2023-10-09T06:37:05.159 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlYyOmKhAAQmpr+mGfJH4GUa28dtlJwA==", 2023-10-09T06:37:05.159 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:05.159 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:37:05.159 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:37:05.159 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:37:05.159 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:37:05.159 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:05.159 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:05.159 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:05.159 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.3", 2023-10-09T06:37:05.159 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlD2H1MhAA4zvaXY/Gd0kndzi+QSwp7w==", 2023-10-09T06:37:05.160 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:05.160 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:37:05.160 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:37:05.160 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:37:05.160 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:37:05.160 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:05.160 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:05.160 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:05.160 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.admin", 2023-10-09T06:37:05.160 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDYnSNlsHH4LxAA85xeFEP1CUwoblaePYk5zA==", 2023-10-09T06:37:05.160 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:05.160 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:37:05.161 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:37:05.161 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:37:05.161 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:05.161 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:05.161 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:05.161 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:05.161 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mgr.x", 2023-10-09T06:37:05.161 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNl2sloDBAA0FoWN9iovCl6783o3/btiA==", 2023-10-09T06:37:05.161 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:05.161 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:37:05.161 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile mgr", 2023-10-09T06:37:05.161 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:05.162 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:05.162 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:05.162 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:05.162 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mgr.y", 2023-10-09T06:37:05.162 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlcFWVCBAAR2dKPg01ciTiNpSX0JIKFg==", 2023-10-09T06:37:05.162 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:05.162 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:37:05.162 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile mgr", 2023-10-09T06:37:05.162 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:05.162 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:05.162 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:05.162 INFO:teuthology.orchestra.run.smithi163.stdout: ] 2023-10-09T06:37:05.163 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:37:05.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.141+0000 7f33a84f4700 1 -- 172.21.15.163:0/3797022251 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f338c067320 msgr2=0x7f338c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:05.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.141+0000 7f33a84f4700 1 --2- 172.21.15.163:0/3797022251 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f338c067320 0x7f338c0697d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3398001660 tx=0x7f3398007b00 comp rx=0 tx=0).stop 2023-10-09T06:37:05.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.141+0000 7f33a84f4700 1 -- 172.21.15.163:0/3797022251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33a0159890 msgr2=0x7f33a015c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:05.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.141+0000 7f33a84f4700 1 --2- 172.21.15.163:0/3797022251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33a0159890 0x7f33a015c6d0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f339400c5e0 tx=0x7f3394001070 comp rx=0 tx=0).stop 2023-10-09T06:37:05.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.141+0000 7f33a84f4700 1 -- 172.21.15.163:0/3797022251 shutdown_connections 2023-10-09T06:37:05.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.141+0000 7f33a84f4700 1 --2- 172.21.15.163:0/3797022251 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f338c067320 0x7f338c0697d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:05.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.141+0000 7f33a84f4700 1 --2- 172.21.15.163:0/3797022251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33a0159c60 0x7f33a015d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:05.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.141+0000 7f33a84f4700 1 --2- 172.21.15.163:0/3797022251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33a0069560 0x7f33a01543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:05.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.141+0000 7f33a84f4700 1 --2- 172.21.15.163:0/3797022251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33a0159890 0x7f33a015c6d0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:05.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.141+0000 7f33a84f4700 1 -- 172.21.15.163:0/3797022251 >> 172.21.15.163:0/3797022251 conn(0x7f33a0068e20 msgr2=0x7f33a00cadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:05.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.141+0000 7f33a84f4700 1 -- 172.21.15.163:0/3797022251 shutdown_connections 2023-10-09T06:37:05.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.141+0000 7f33a84f4700 1 -- 172.21.15.163:0/3797022251 wait complete. 2023-10-09T06:37:05.166 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:37:05.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.280+0000 7f5b72ac6700 1 Processor -- start 2023-10-09T06:37:05.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.280+0000 7f5b72ac6700 1 -- start start 2023-10-09T06:37:05.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.281+0000 7f5b72ac6700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5b6c0b1420 0x7f5b6c0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:05.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.281+0000 7f5b72ac6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b6c0ccf90 0x7f5b6c0c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:05.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.281+0000 7f5b72ac6700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b6c0cc300 con 0x7f5b6c0cbf30 2023-10-09T06:37:05.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.281+0000 7f5b72ac6700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b6c156bf0 con 0x7f5b6c0b1420 2023-10-09T06:37:05.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.281+0000 7f5b72ac6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6c0ae950 con 0x7f5b6c0ccf90 2023-10-09T06:37:05.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.281+0000 7f5b70862700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f5b6c0cbf30 0x7f5b6c0b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:49900/0 (socket says 172.21.15.163:49900) 2023-10-09T06:37:05.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.281+0000 7f5b70862700 1 -- 172.21.15.163:0/3156320266 learned_addr learned my addr 172.21.15.163:0/3156320266 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:05.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.281+0000 7f5b6bfff700 1 --2- 172.21.15.163:0/3156320266 >> v2:172.21.15.163:3300/0 conn(0x7f5b6c0b1420 0x7f5b6c0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:05.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.282+0000 7f5b6b7fe700 1 -- 172.21.15.163:0/3156320266 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2391188638 0 0) 0x7f5b6c0cc300 con 0x7f5b6c0cbf30 2023-10-09T06:37:05.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.282+0000 7f5b71063700 1 --2- 172.21.15.163:0/3156320266 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b6c0ccf90 0x7f5b6c0c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:05.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.282+0000 7f5b6b7fe700 1 -- 172.21.15.163:0/3156320266 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b54003570 con 0x7f5b6c0cbf30 2023-10-09T06:37:05.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.282+0000 7f5b6bfff700 1 -- 172.21.15.163:0/3156320266 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b6c0ccf90 msgr2=0x7f5b6c0c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:05.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.282+0000 7f5b6bfff700 1 --2- 172.21.15.163:0/3156320266 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b6c0ccf90 0x7f5b6c0c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:05.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.282+0000 7f5b6bfff700 1 -- 172.21.15.163:0/3156320266 >> v1:172.21.15.163:6789/0 conn(0x7f5b6c0cbf30 legacy=0x7f5b6c0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:05.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.282+0000 7f5b6bfff700 1 -- 172.21.15.163:0/3156320266 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b6c0ad370 con 0x7f5b6c0b1420 2023-10-09T06:37:05.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.282+0000 7f5b6bfff700 1 --2- 172.21.15.163:0/3156320266 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b6c0b1420 0x7f5b6c0ae410 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009910 tx=0x7f5b600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651c28cd23066237 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:05.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.282+0000 7f5b6b7fe700 1 -- 172.21.15.163:0/3156320266 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6000b640 con 0x7f5b6c0b1420 2023-10-09T06:37:05.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.282+0000 7f5b6b7fe700 1 -- 172.21.15.163:0/3156320266 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b6000b7a0 con 0x7f5b6c0b1420 2023-10-09T06:37:05.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.282+0000 7f5b6b7fe700 1 -- 172.21.15.163:0/3156320266 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6000bb60 con 0x7f5b6c0b1420 2023-10-09T06:37:05.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.282+0000 7f5b72ac6700 1 -- 172.21.15.163:0/3156320266 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b6c0b1420 msgr2=0x7f5b6c0ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:05.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.282+0000 7f5b72ac6700 1 --2- 172.21.15.163:0/3156320266 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b6c0b1420 0x7f5b6c0ae410 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009910 tx=0x7f5b600091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:05.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.283+0000 7f5b72ac6700 1 -- 172.21.15.163:0/3156320266 shutdown_connections 2023-10-09T06:37:05.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.283+0000 7f5b72ac6700 1 --2- 172.21.15.163:0/3156320266 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b6c0ccf90 0x7f5b6c0c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:05.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.283+0000 7f5b72ac6700 1 --2- 172.21.15.163:0/3156320266 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b6c0b1420 0x7f5b6c0ae410 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:05.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.283+0000 7f5b72ac6700 1 -- 172.21.15.163:0/3156320266 >> 172.21.15.163:0/3156320266 conn(0x7f5b6c06a320 msgr2=0x7f5b6c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:05.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.283+0000 7f5b72ac6700 1 -- 172.21.15.163:0/3156320266 shutdown_connections 2023-10-09T06:37:05.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.283+0000 7f5b72ac6700 1 -- 172.21.15.163:0/3156320266 wait complete. 2023-10-09T06:37:05.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.284+0000 7f5b72ac6700 1 Processor -- start 2023-10-09T06:37:05.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.284+0000 7f5b72ac6700 1 -- start start 2023-10-09T06:37:05.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.284+0000 7f5b72ac6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b6c0b1420 0x7f5b6c0aaf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:05.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.284+0000 7f5b72ac6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b6c0cbf30 0x7f5b6c0a8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:05.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.284+0000 7f5b72ac6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b6c0ccf90 0x7f5b6c145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:05.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.284+0000 7f5b72ac6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6c0a9320 con 0x7f5b6c0ccf90 2023-10-09T06:37:05.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.285+0000 7f5b72ac6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6c0a6b80 con 0x7f5b6c0b1420 2023-10-09T06:37:05.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.285+0000 7f5b72ac6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6c146530 con 0x7f5b6c0cbf30 2023-10-09T06:37:05.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.285+0000 7f5b71063700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b6c0ccf90 0x7f5b6c145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:05.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.285+0000 7f5b71063700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b6c0ccf90 0x7f5b6c145ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51530/0 (socket says 172.21.15.163:51530) 2023-10-09T06:37:05.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.285+0000 7f5b71063700 1 -- 172.21.15.163:0/445738473 learned_addr learned my addr 172.21.15.163:0/445738473 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:05.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.285+0000 7f5b6bfff700 1 --2- 172.21.15.163:0/445738473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b6c0cbf30 0x7f5b6c0a8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:05.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.285+0000 7f5b70862700 1 --2- 172.21.15.163:0/445738473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b6c0b1420 0x7f5b6c0aaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:05.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.285+0000 7f5b71063700 1 -- 172.21.15.163:0/445738473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b6c0cbf30 msgr2=0x7f5b6c0a8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:05.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.285+0000 7f5b71063700 1 --2- 172.21.15.163:0/445738473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b6c0cbf30 0x7f5b6c0a8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:05.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.285+0000 7f5b71063700 1 -- 172.21.15.163:0/445738473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b6c0b1420 msgr2=0x7f5b6c0aaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:05.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.285+0000 7f5b71063700 1 --2- 172.21.15.163:0/445738473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b6c0b1420 0x7f5b6c0aaf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:05.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.285+0000 7f5b71063700 1 -- 172.21.15.163:0/445738473 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b60009420 con 0x7f5b6c0ccf90 2023-10-09T06:37:05.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.285+0000 7f5b71063700 1 --2- 172.21.15.163:0/445738473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b6c0ccf90 0x7f5b6c145ff0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f5b6400fba0 tx=0x7f5b64008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:05.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.285+0000 7f5b68ff9700 1 -- 172.21.15.163:0/445738473 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b640097c0 con 0x7f5b6c0ccf90 2023-10-09T06:37:05.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.285+0000 7f5b68ff9700 1 -- 172.21.15.163:0/445738473 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b64009960 con 0x7f5b6c0ccf90 2023-10-09T06:37:05.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.285+0000 7f5b72ac6700 1 -- 172.21.15.163:0/445738473 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b6c0a9550 con 0x7f5b6c0ccf90 2023-10-09T06:37:05.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.285+0000 7f5b68ff9700 1 -- 172.21.15.163:0/445738473 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b64009d20 con 0x7f5b6c0ccf90 2023-10-09T06:37:05.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.285+0000 7f5b72ac6700 1 -- 172.21.15.163:0/445738473 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b6c0a6530 con 0x7f5b6c0ccf90 2023-10-09T06:37:05.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.286+0000 7f5b72ac6700 1 -- 172.21.15.163:0/445738473 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b6c0a6cf0 con 0x7f5b6c0ccf90 2023-10-09T06:37:05.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.287+0000 7f5b68ff9700 1 -- 172.21.15.163:0/445738473 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5b64011660 con 0x7f5b6c0ccf90 2023-10-09T06:37:05.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.287+0000 7f5b68ff9700 1 --2- 172.21.15.163:0/445738473 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5b58067320 0x7f5b580697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:05.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.287+0000 7f5b68ff9700 1 -- 172.21.15.163:0/445738473 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f5b64086fb0 con 0x7f5b6c0ccf90 2023-10-09T06:37:05.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.287+0000 7f5b70862700 1 --2- 172.21.15.163:0/445738473 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5b58067320 0x7f5b580697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:05.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.288+0000 7f5b70862700 1 --2- 172.21.15.163:0/445738473 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5b58067320 0x7f5b580697d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c005c50 tx=0x7f5b5c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:05.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.291+0000 7f5b68ff9700 1 -- 172.21.15.163:0/445738473 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5b64053e80 con 0x7f5b6c0ccf90 2023-10-09T06:37:05.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.474+0000 7f5b72ac6700 1 -- 172.21.15.163:0/445738473 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b6c04c3c0 con 0x7f5b6c0ccf90 2023-10-09T06:37:05.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.475+0000 7f5b68ff9700 1 -- 172.21.15.163:0/445738473 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+12184 (secure 0 0 0) 0x7f5b640574a0 con 0x7f5b6c0ccf90 2023-10-09T06:37:05.474 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:05.475 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":109,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:37:04.733912+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:37:05.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.476+0000 7f5b72ac6700 1 -- 172.21.15.163:0/445738473 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5b58067320 msgr2=0x7f5b580697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:05.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.476+0000 7f5b72ac6700 1 --2- 172.21.15.163:0/445738473 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5b58067320 0x7f5b580697d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c005c50 tx=0x7f5b5c009040 comp rx=0 tx=0).stop 2023-10-09T06:37:05.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.476+0000 7f5b72ac6700 1 -- 172.21.15.163:0/445738473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b6c0ccf90 msgr2=0x7f5b6c145ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:05.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.476+0000 7f5b72ac6700 1 --2- 172.21.15.163:0/445738473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b6c0ccf90 0x7f5b6c145ff0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f5b6400fba0 tx=0x7f5b64008040 comp rx=0 tx=0).stop 2023-10-09T06:37:05.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.477+0000 7f5b72ac6700 1 -- 172.21.15.163:0/445738473 shutdown_connections 2023-10-09T06:37:05.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.477+0000 7f5b72ac6700 1 --2- 172.21.15.163:0/445738473 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5b58067320 0x7f5b580697d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:05.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.477+0000 7f5b72ac6700 1 --2- 172.21.15.163:0/445738473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b6c0ccf90 0x7f5b6c145ff0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:05.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.477+0000 7f5b72ac6700 1 --2- 172.21.15.163:0/445738473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b6c0b1420 0x7f5b6c0aaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:05.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.477+0000 7f5b72ac6700 1 --2- 172.21.15.163:0/445738473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b6c0cbf30 0x7f5b6c0a8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:05.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.477+0000 7f5b72ac6700 1 -- 172.21.15.163:0/445738473 >> 172.21.15.163:0/445738473 conn(0x7f5b6c06a320 msgr2=0x7f5b6c0b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:05.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.477+0000 7f5b72ac6700 1 -- 172.21.15.163:0/445738473 shutdown_connections 2023-10-09T06:37:05.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.477+0000 7f5b72ac6700 1 -- 172.21.15.163:0/445738473 wait complete. 2023-10-09T06:37:05.489 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-09T06:37:05.489 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:37:05.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.607+0000 7f2eb7c75700 1 Processor -- start 2023-10-09T06:37:05.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.608+0000 7f2eb7c75700 1 -- start start 2023-10-09T06:37:05.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.608+0000 7f2eb7c75700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb00caa90 0x7f2eb00c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:05.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.608+0000 7f2eb7c75700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eb015d7e0 con 0x7f2eb006bcd0 2023-10-09T06:37:05.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.608+0000 7f2eb7c75700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eb015dac0 con 0x7f2eb006c0a0 2023-10-09T06:37:05.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.608+0000 7f2eb7c75700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb00caf30 con 0x7f2eb00caa90 2023-10-09T06:37:05.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.609+0000 7f2eb5a11700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f2eb006bcd0 0x7f2eb015a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:49910/0 (socket says 172.21.15.163:49910) 2023-10-09T06:37:05.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.609+0000 7f2eb5a11700 1 -- 172.21.15.163:0/4123814160 learned_addr learned my addr 172.21.15.163:0/4123814160 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:05.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.609+0000 7f2eb6212700 1 --2- 172.21.15.163:0/4123814160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb00caa90 0x7f2eb00c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:05.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.609+0000 7f2eb4a0f700 1 -- 172.21.15.163:0/4123814160 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1025283111 0 0) 0x7f2eb015d7e0 con 0x7f2eb006bcd0 2023-10-09T06:37:05.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.609+0000 7f2eb4a0f700 1 -- 172.21.15.163:0/4123814160 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e9c003570 con 0x7f2eb006bcd0 2023-10-09T06:37:05.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.609+0000 7f2eb4a0f700 1 -- 172.21.15.163:0/4123814160 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 582865364 0 0) 0x7f2eb015dac0 con 0x7f2eb006c0a0 2023-10-09T06:37:05.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.609+0000 7f2eb4a0f700 1 -- 172.21.15.163:0/4123814160 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2eb015d7e0 con 0x7f2eb006c0a0 2023-10-09T06:37:05.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.609+0000 7f2eb4a0f700 1 -- 172.21.15.163:0/4123814160 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 434515500 0 0) 0x7f2e9c003570 con 0x7f2eb006bcd0 2023-10-09T06:37:05.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.609+0000 7f2eb4a0f700 1 -- 172.21.15.163:0/4123814160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb00caa90 msgr2=0x7f2eb00c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:05.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.609+0000 7f2eb4a0f700 1 --2- 172.21.15.163:0/4123814160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb00caa90 0x7f2eb00c8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:05.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.609+0000 7f2eb4a0f700 1 -- 172.21.15.163:0/4123814160 >> v1:172.21.15.181:6789/0 conn(0x7f2eb006c0a0 legacy=0x7f2eb015df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:05.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.609+0000 7f2eb4a0f700 1 -- 172.21.15.163:0/4123814160 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb015aa30 con 0x7f2eb006bcd0 2023-10-09T06:37:05.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.609+0000 7f2eb4a0f700 1 -- 172.21.15.163:0/4123814160 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f2eac001770 con 0x7f2eb006bcd0 2023-10-09T06:37:05.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.609+0000 7f2eb4a0f700 1 -- 172.21.15.163:0/4123814160 >> v1:172.21.15.163:6789/0 conn(0x7f2eb006bcd0 legacy=0x7f2eb015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:05.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.609+0000 7f2eb4a0f700 1 --2- 172.21.15.163:0/4123814160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e9c003c20 0x7f2e9c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:05.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.609+0000 7f2eb4a0f700 1 --2- 172.21.15.163:0/4123814160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e9c004810 0x7f2e9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:05.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.609+0000 7f2eb4a0f700 1 --2- 172.21.15.163:0/4123814160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb00caa90 0x7f2e9c0052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:05.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.609+0000 7f2eb4a0f700 1 -- 172.21.15.163:0/4123814160 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb00caf30 con 0x7f2e9c003c20 2023-10-09T06:37:05.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.609+0000 7f2eb4a0f700 1 -- 172.21.15.163:0/4123814160 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c00bc40 con 0x7f2e9c004810 2023-10-09T06:37:05.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.609+0000 7f2eb4a0f700 1 -- 172.21.15.163:0/4123814160 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c00be00 con 0x7f2eb00caa90 2023-10-09T06:37:05.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.609+0000 7f2eb5a11700 1 --2- 172.21.15.163:0/4123814160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e9c003c20 0x7f2e9c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:05.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.609+0000 7f2eb6212700 1 --2- 172.21.15.163:0/4123814160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb00caa90 0x7f2e9c0052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:05.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.609+0000 7f2eb5210700 1 --2- 172.21.15.163:0/4123814160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e9c004810 0x7f2e9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:05.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.610+0000 7f2eb5a11700 1 -- 172.21.15.163:0/4123814160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb00caa90 msgr2=0x7f2e9c0052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:05.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.610+0000 7f2eb5a11700 1 --2- 172.21.15.163:0/4123814160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb00caa90 0x7f2e9c0052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:05.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.610+0000 7f2eb5a11700 1 -- 172.21.15.163:0/4123814160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e9c004810 msgr2=0x7f2e9c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:05.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.610+0000 7f2eb5a11700 1 --2- 172.21.15.163:0/4123814160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e9c004810 0x7f2e9c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:05.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.610+0000 7f2eb5a11700 1 -- 172.21.15.163:0/4123814160 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c003570 con 0x7f2e9c003c20 2023-10-09T06:37:05.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.610+0000 7f2eb5a11700 1 --2- 172.21.15.163:0/4123814160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e9c003c20 0x7f2e9c004010 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f2eac00a7b0 tx=0x7f2eac002040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd55ebba32961b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:05.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.610+0000 7f2eb4a0f700 1 -- 172.21.15.163:0/4123814160 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac0070d0 con 0x7f2e9c003c20 2023-10-09T06:37:05.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.610+0000 7f2eb4a0f700 1 -- 172.21.15.163:0/4123814160 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eac007250 con 0x7f2e9c003c20 2023-10-09T06:37:05.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.610+0000 7f2eb4a0f700 1 -- 172.21.15.163:0/4123814160 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac007610 con 0x7f2e9c003c20 2023-10-09T06:37:05.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.610+0000 7f2eb7c75700 1 -- 172.21.15.163:0/4123814160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e9c003c20 msgr2=0x7f2e9c004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:05.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.610+0000 7f2eb7c75700 1 --2- 172.21.15.163:0/4123814160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e9c003c20 0x7f2e9c004010 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f2eac00a7b0 tx=0x7f2eac002040 comp rx=0 tx=0).stop 2023-10-09T06:37:05.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.610+0000 7f2eb6212700 1 -- 172.21.15.163:0/4123814160 reap_dead start 2023-10-09T06:37:05.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.610+0000 7f2eb7c75700 1 -- 172.21.15.163:0/4123814160 shutdown_connections 2023-10-09T06:37:05.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.610+0000 7f2eb7c75700 1 -- 172.21.15.163:0/4123814160 >> 172.21.15.163:0/4123814160 conn(0x7f2eb00ce600 msgr2=0x7f2eb00cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:05.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.610+0000 7f2eb7c75700 1 -- 172.21.15.163:0/4123814160 shutdown_connections 2023-10-09T06:37:05.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.610+0000 7f2eb7c75700 1 -- 172.21.15.163:0/4123814160 wait complete. 2023-10-09T06:37:05.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.611+0000 7f2eb7c75700 1 Processor -- start 2023-10-09T06:37:05.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.612+0000 7f2eb7c75700 1 -- start start 2023-10-09T06:37:05.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.612+0000 7f2eb7c75700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e9c003c20 0x7f2eb0159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:05.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.612+0000 7f2eb7c75700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2eb00b30b0 0x7f2eb0154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:05.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.612+0000 7f2eb7c75700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2eb00b3520 0x7f2eb0154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:05.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.612+0000 7f2eb7c75700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb0153600 con 0x7f2eb00b3520 2023-10-09T06:37:05.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.612+0000 7f2eb7c75700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb016ee70 con 0x7f2eb00b30b0 2023-10-09T06:37:05.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.612+0000 7f2eb7c75700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb00b3a90 con 0x7f2e9c003c20 2023-10-09T06:37:05.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.612+0000 7f2eb6212700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2eb00b3520 0x7f2eb0154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:05.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.612+0000 7f2eb6212700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2eb00b3520 0x7f2eb0154da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51542/0 (socket says 172.21.15.163:51542) 2023-10-09T06:37:05.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.612+0000 7f2eb6212700 1 -- 172.21.15.163:0/3506575869 learned_addr learned my addr 172.21.15.163:0/3506575869 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:05.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.612+0000 7f2eb5a11700 1 --2- 172.21.15.163:0/3506575869 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e9c003c20 0x7f2eb0159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:05.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.612+0000 7f2eb5210700 1 --2- 172.21.15.163:0/3506575869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2eb00b30b0 0x7f2eb0154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:05.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.613+0000 7f2eb6212700 1 -- 172.21.15.163:0/3506575869 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e9c003c20 msgr2=0x7f2eb0159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:05.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.613+0000 7f2eb6212700 1 --2- 172.21.15.163:0/3506575869 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e9c003c20 0x7f2eb0159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:05.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.613+0000 7f2eb6212700 1 -- 172.21.15.163:0/3506575869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2eb00b30b0 msgr2=0x7f2eb0154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:05.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.613+0000 7f2eb6212700 1 --2- 172.21.15.163:0/3506575869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2eb00b30b0 0x7f2eb0154860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:05.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.613+0000 7f2eb6212700 1 -- 172.21.15.163:0/3506575869 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb016e820 con 0x7f2eb00b3520 2023-10-09T06:37:05.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.613+0000 7f2eb5210700 1 --2- 172.21.15.163:0/3506575869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2eb00b30b0 0x7f2eb0154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:05.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.613+0000 7f2eb6212700 1 --2- 172.21.15.163:0/3506575869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2eb00b3520 0x7f2eb0154da0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c004790 tx=0x7f2ea80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:05.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.613+0000 7f2ea5ffb700 1 -- 172.21.15.163:0/3506575869 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea800f910 con 0x7f2eb00b3520 2023-10-09T06:37:05.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.613+0000 7f2ea5ffb700 1 -- 172.21.15.163:0/3506575869 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea800fa70 con 0x7f2eb00b3520 2023-10-09T06:37:05.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.613+0000 7f2ea5ffb700 1 -- 172.21.15.163:0/3506575869 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea800fe10 con 0x7f2eb00b3520 2023-10-09T06:37:05.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.613+0000 7f2eb7c75700 1 -- 172.21.15.163:0/3506575869 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb016eb00 con 0x7f2eb00b3520 2023-10-09T06:37:05.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.613+0000 7f2eb7c75700 1 -- 172.21.15.163:0/3506575869 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb016e480 con 0x7f2eb00b3520 2023-10-09T06:37:05.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.614+0000 7f2eb7c75700 1 -- 172.21.15.163:0/3506575869 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eb004c3c0 con 0x7f2eb00b3520 2023-10-09T06:37:05.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.614+0000 7f2ea5ffb700 1 -- 172.21.15.163:0/3506575869 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2ea8014650 con 0x7f2eb00b3520 2023-10-09T06:37:05.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.615+0000 7f2ea5ffb700 1 --2- 172.21.15.163:0/3506575869 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2e94067320 0x7f2e940697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:05.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.615+0000 7f2ea5ffb700 1 -- 172.21.15.163:0/3506575869 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f2ea8085000 con 0x7f2eb00b3520 2023-10-09T06:37:05.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.615+0000 7f2eb5a11700 1 --2- 172.21.15.163:0/3506575869 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2e94067320 0x7f2e940697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:05.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.615+0000 7f2eb5a11700 1 --2- 172.21.15.163:0/3506575869 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2e94067320 0x7f2e940697d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2eac00a110 tx=0x7f2eac016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:05.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.619+0000 7f2ea5ffb700 1 -- 172.21.15.163:0/3506575869 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2ea8052060 con 0x7f2eb00b3520 2023-10-09T06:37:05.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.802+0000 7f2eb7c75700 1 -- 172.21.15.163:0/3506575869 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2eb015ad10 con 0x7f2eb00b3520 2023-10-09T06:37:05.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.802+0000 7f2ea5ffb700 1 -- 172.21.15.163:0/3506575869 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2ea8000e60 con 0x7f2eb00b3520 2023-10-09T06:37:05.803 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:37:05.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.804+0000 7f2eb7c75700 1 -- 172.21.15.163:0/3506575869 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2e94067320 msgr2=0x7f2e940697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:05.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.804+0000 7f2eb7c75700 1 --2- 172.21.15.163:0/3506575869 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2e94067320 0x7f2e940697d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2eac00a110 tx=0x7f2eac016000 comp rx=0 tx=0).stop 2023-10-09T06:37:05.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.804+0000 7f2eb7c75700 1 -- 172.21.15.163:0/3506575869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2eb00b3520 msgr2=0x7f2eb0154da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:05.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.804+0000 7f2eb7c75700 1 --2- 172.21.15.163:0/3506575869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2eb00b3520 0x7f2eb0154da0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c004790 tx=0x7f2ea80091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:05.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.804+0000 7f2eb7c75700 1 -- 172.21.15.163:0/3506575869 shutdown_connections 2023-10-09T06:37:05.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.804+0000 7f2eb7c75700 1 --2- 172.21.15.163:0/3506575869 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2e94067320 0x7f2e940697d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:05.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.804+0000 7f2eb7c75700 1 --2- 172.21.15.163:0/3506575869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2eb00b3520 0x7f2eb0154da0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:05.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.804+0000 7f2eb7c75700 1 --2- 172.21.15.163:0/3506575869 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e9c003c20 0x7f2eb0159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:05.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.804+0000 7f2eb7c75700 1 --2- 172.21.15.163:0/3506575869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2eb00b30b0 0x7f2eb0154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:05.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.804+0000 7f2eb7c75700 1 -- 172.21.15.163:0/3506575869 >> 172.21.15.163:0/3506575869 conn(0x7f2eb00ce600 msgr2=0x7f2eb015bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:05.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.804+0000 7f2eb7c75700 1 -- 172.21.15.163:0/3506575869 shutdown_connections 2023-10-09T06:37:05.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.805+0000 7f2eb7c75700 1 -- 172.21.15.163:0/3506575869 wait complete. 2023-10-09T06:37:05.817 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-09T06:37:05.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.931+0000 7f9a0536d700 1 Processor -- start 2023-10-09T06:37:05.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.931+0000 7f9a0536d700 1 -- start start 2023-10-09T06:37:05.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.932+0000 7f9a0536d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9a001597f0 0x7f9a000b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:05.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.932+0000 7f9a0536d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9a0015d230 0x7f9a0015d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:05.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.932+0000 7f9a0536d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a0015db70 con 0x7f9a00159420 2023-10-09T06:37:05.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.932+0000 7f9a0536d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a000b11d0 con 0x7f9a0015d230 2023-10-09T06:37:05.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.932+0000 7f9a0536d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a000b14b0 con 0x7f9a001597f0 2023-10-09T06:37:05.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.932+0000 7f9a04b6c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9a0015d230 0x7f9a0015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:05.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.932+0000 7f99fffff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f9a00159420 0x7f9a000b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:49924/0 (socket says 172.21.15.163:49924) 2023-10-09T06:37:05.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.932+0000 7f99fffff700 1 -- 172.21.15.163:0/1808728085 learned_addr learned my addr 172.21.15.163:0/1808728085 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:05.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.932+0000 7f99ff7fe700 1 --2- 172.21.15.163:0/1808728085 >> v2:172.21.15.181:3300/0 conn(0x7f9a001597f0 0x7f9a000b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:05.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.932+0000 7f9a04b6c700 1 -- 172.21.15.163:0/1808728085 >> v2:172.21.15.181:3300/0 conn(0x7f9a001597f0 msgr2=0x7f9a000b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:05.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.932+0000 7f9a04b6c700 1 --2- 172.21.15.163:0/1808728085 >> v2:172.21.15.181:3300/0 conn(0x7f9a001597f0 0x7f9a000b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:05.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.932+0000 7f99feffd700 1 -- 172.21.15.163:0/1808728085 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1249928194 0 0) 0x7f9a0015db70 con 0x7f9a00159420 2023-10-09T06:37:05.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.932+0000 7f9a04b6c700 1 -- 172.21.15.163:0/1808728085 >> v1:172.21.15.163:6789/0 conn(0x7f9a00159420 legacy=0x7f9a000b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:05.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.933+0000 7f9a04b6c700 1 -- 172.21.15.163:0/1808728085 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a000b1790 con 0x7f9a0015d230 2023-10-09T06:37:05.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.933+0000 7f99ff7fe700 1 --2- 172.21.15.163:0/1808728085 >> v2:172.21.15.181:3300/0 conn(0x7f9a001597f0 0x7f9a000b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:37:05.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.933+0000 7f9a04b6c700 1 --2- 172.21.15.163:0/1808728085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0015d230 0x7f9a0015d630 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f9a00089750 tx=0x7f99f8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea0aa401193acff server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:05.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.933+0000 7f99feffd700 1 -- 172.21.15.163:0/1808728085 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f800e070 con 0x7f9a0015d230 2023-10-09T06:37:05.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.933+0000 7f99feffd700 1 -- 172.21.15.163:0/1808728085 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f800bc30 con 0x7f9a0015d230 2023-10-09T06:37:05.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.933+0000 7f99feffd700 1 -- 172.21.15.163:0/1808728085 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f8012430 con 0x7f9a0015d230 2023-10-09T06:37:05.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.933+0000 7f9a0536d700 1 -- 172.21.15.163:0/1808728085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0015d230 msgr2=0x7f9a0015d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:05.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.933+0000 7f9a0536d700 1 --2- 172.21.15.163:0/1808728085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0015d230 0x7f9a0015d630 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f9a00089750 tx=0x7f99f8009780 comp rx=0 tx=0).stop 2023-10-09T06:37:05.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.933+0000 7f9a0536d700 1 -- 172.21.15.163:0/1808728085 shutdown_connections 2023-10-09T06:37:05.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.933+0000 7f9a0536d700 1 --2- 172.21.15.163:0/1808728085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0015d230 0x7f9a0015d630 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:05.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.933+0000 7f9a0536d700 1 --2- 172.21.15.163:0/1808728085 >> v2:172.21.15.181:3300/0 conn(0x7f9a001597f0 0x7f9a000b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:05.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.933+0000 7f9a0536d700 1 -- 172.21.15.163:0/1808728085 >> 172.21.15.163:0/1808728085 conn(0x7f9a000d0660 msgr2=0x7f9a000d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:05.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.933+0000 7f9a0536d700 1 -- 172.21.15.163:0/1808728085 shutdown_connections 2023-10-09T06:37:05.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.933+0000 7f9a0536d700 1 -- 172.21.15.163:0/1808728085 wait complete. 2023-10-09T06:37:05.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.934+0000 7f9a0536d700 1 Processor -- start 2023-10-09T06:37:05.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.935+0000 7f9a0536d700 1 -- start start 2023-10-09T06:37:05.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.935+0000 7f9a0536d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a00159420 0x7f9a00153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:05.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.935+0000 7f9a0536d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a001597f0 0x7f9a0014f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:05.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.935+0000 7f9a0536d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0015d230 0x7f9a0015b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:05.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.935+0000 7f9a0536d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a00153810 con 0x7f9a0015d230 2023-10-09T06:37:05.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.935+0000 7f9a0536d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a00153200 con 0x7f9a001597f0 2023-10-09T06:37:05.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.935+0000 7f9a0536d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0015bdc0 con 0x7f9a00159420 2023-10-09T06:37:05.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.936+0000 7f9a04b6c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0015d230 0x7f9a0015b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:05.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.936+0000 7f9a04b6c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0015d230 0x7f9a0015b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51560/0 (socket says 172.21.15.163:51560) 2023-10-09T06:37:05.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.936+0000 7f9a04b6c700 1 -- 172.21.15.163:0/4119775159 learned_addr learned my addr 172.21.15.163:0/4119775159 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:05.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.936+0000 7f99fffff700 1 --2- 172.21.15.163:0/4119775159 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a00159420 0x7f9a00153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:05.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.936+0000 7f99ff7fe700 1 --2- 172.21.15.163:0/4119775159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a001597f0 0x7f9a0014f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:05.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.936+0000 7f9a04b6c700 1 -- 172.21.15.163:0/4119775159 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a00159420 msgr2=0x7f9a00153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:05.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.936+0000 7f9a04b6c700 1 --2- 172.21.15.163:0/4119775159 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a00159420 0x7f9a00153ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:05.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.936+0000 7f9a04b6c700 1 -- 172.21.15.163:0/4119775159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a001597f0 msgr2=0x7f9a0014f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:05.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.936+0000 7f9a04b6c700 1 --2- 172.21.15.163:0/4119775159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a001597f0 0x7f9a0014f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:05.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.936+0000 7f9a04b6c700 1 -- 172.21.15.163:0/4119775159 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f8009a20 con 0x7f9a0015d230 2023-10-09T06:37:05.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.936+0000 7f9a04b6c700 1 --2- 172.21.15.163:0/4119775159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0015d230 0x7f9a0015b450 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f9a00089340 tx=0x7f99f8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:05.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.936+0000 7f99e7fff700 1 -- 172.21.15.163:0/4119775159 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f801d630 con 0x7f9a0015d230 2023-10-09T06:37:05.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.936+0000 7f99e7fff700 1 -- 172.21.15.163:0/4119775159 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f801d790 con 0x7f9a0015d230 2023-10-09T06:37:05.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.936+0000 7f9a0536d700 1 -- 172.21.15.163:0/4119775159 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0015c0a0 con 0x7f9a0015d230 2023-10-09T06:37:05.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.936+0000 7f99e7fff700 1 -- 172.21.15.163:0/4119775159 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f801db30 con 0x7f9a0015d230 2023-10-09T06:37:05.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.936+0000 7f9a0536d700 1 -- 172.21.15.163:0/4119775159 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0014f9e0 con 0x7f9a0015d230 2023-10-09T06:37:05.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.937+0000 7f9a0536d700 1 -- 172.21.15.163:0/4119775159 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a00153660 con 0x7f9a0015d230 2023-10-09T06:37:05.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.937+0000 7f99e7fff700 1 -- 172.21.15.163:0/4119775159 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f99f8024070 con 0x7f9a0015d230 2023-10-09T06:37:05.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.938+0000 7f99e7fff700 1 --2- 172.21.15.163:0/4119775159 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f99ec067320 0x7f99ec0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:05.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.938+0000 7f99e7fff700 1 -- 172.21.15.163:0/4119775159 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f99f8087350 con 0x7f9a0015d230 2023-10-09T06:37:05.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.938+0000 7f99fffff700 1 --2- 172.21.15.163:0/4119775159 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f99ec067320 0x7f99ec0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:05.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.938+0000 7f99fffff700 1 --2- 172.21.15.163:0/4119775159 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f99ec067320 0x7f99ec0697d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f99f0007fd0 tx=0x7f99f0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:05.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:05.942+0000 7f99e7fff700 1 -- 172.21.15.163:0/4119775159 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f99f8054330 con 0x7f9a0015d230 2023-10-09T06:37:06.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.128+0000 7f9a0536d700 1 -- 172.21.15.163:0/4119775159 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9a0015c290 con 0x7f9a0015d230 2023-10-09T06:37:06.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.129+0000 7f99e7fff700 1 -- 172.21.15.163:0/4119775159 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v109) v1 ==== 84+0+65 (secure 0 0 0) 0x7f99f8057950 con 0x7f9a0015d230 2023-10-09T06:37:06.129 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:06.129 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:37:06.129 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:06.129 INFO:teuthology.orchestra.run.smithi163.stdout: "poolnum": 1, 2023-10-09T06:37:06.129 INFO:teuthology.orchestra.run.smithi163.stdout: "poolname": ".mgr" 2023-10-09T06:37:06.129 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:06.129 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:37:06.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.131+0000 7f9a0536d700 1 -- 172.21.15.163:0/4119775159 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f99ec067320 msgr2=0x7f99ec0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:06.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.131+0000 7f9a0536d700 1 --2- 172.21.15.163:0/4119775159 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f99ec067320 0x7f99ec0697d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f99f0007fd0 tx=0x7f99f0009040 comp rx=0 tx=0).stop 2023-10-09T06:37:06.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.131+0000 7f9a0536d700 1 -- 172.21.15.163:0/4119775159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0015d230 msgr2=0x7f9a0015b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:06.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.131+0000 7f9a0536d700 1 --2- 172.21.15.163:0/4119775159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0015d230 0x7f9a0015b450 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f9a00089340 tx=0x7f99f8019000 comp rx=0 tx=0).stop 2023-10-09T06:37:06.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.131+0000 7f9a0536d700 1 -- 172.21.15.163:0/4119775159 shutdown_connections 2023-10-09T06:37:06.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.131+0000 7f9a0536d700 1 --2- 172.21.15.163:0/4119775159 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f99ec067320 0x7f99ec0697d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:06.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.131+0000 7f9a0536d700 1 --2- 172.21.15.163:0/4119775159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0015d230 0x7f9a0015b450 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:06.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.131+0000 7f9a0536d700 1 --2- 172.21.15.163:0/4119775159 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a00159420 0x7f9a00153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:06.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.131+0000 7f9a0536d700 1 --2- 172.21.15.163:0/4119775159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a001597f0 0x7f9a0014f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:06.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.131+0000 7f9a0536d700 1 -- 172.21.15.163:0/4119775159 >> 172.21.15.163:0/4119775159 conn(0x7f9a000d0660 msgr2=0x7f9a00157070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:06.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.131+0000 7f9a0536d700 1 -- 172.21.15.163:0/4119775159 shutdown_connections 2023-10-09T06:37:06.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.131+0000 7f9a0536d700 1 -- 172.21.15.163:0/4119775159 wait complete. 2023-10-09T06:37:06.143 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-09T06:37:06.143 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-09T06:37:06.261 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.261+0000 7ff893d8c700 1 Processor -- start 2023-10-09T06:37:06.261 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.262+0000 7ff893d8c700 1 -- start start 2023-10-09T06:37:06.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.262+0000 7ff893d8c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff88c0cd910 0x7ff88c0ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:06.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.262+0000 7ff893d8c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff88c0b55a0 0x7ff88c0ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:06.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.262+0000 7ff893d8c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff88c0b5b00 con 0x7ff88c159ec0 2023-10-09T06:37:06.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.262+0000 7ff893d8c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff88c0b5da0 con 0x7ff88c0cd910 2023-10-09T06:37:06.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.262+0000 7ff893d8c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c0b6080 con 0x7ff88c0b55a0 2023-10-09T06:37:06.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.263+0000 7ff892329700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff88c0b55a0 0x7ff88c0ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:06.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.263+0000 7ff891327700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff88c0cd910 0x7ff88c0ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:06.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.263+0000 7ff891327700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff88c0cd910 0x7ff88c0ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:53822/0 (socket says 172.21.15.163:53822) 2023-10-09T06:37:06.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.263+0000 7ff892329700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff88c0b55a0 0x7ff88c0ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:36652/0 (socket says 172.21.15.163:36652) 2023-10-09T06:37:06.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.263+0000 7ff891327700 1 -- 172.21.15.163:0/1787155190 learned_addr learned my addr 172.21.15.163:0/1787155190 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:06.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.263+0000 7ff891327700 1 -- 172.21.15.163:0/1787155190 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff88c0b55a0 msgr2=0x7ff88c0ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:06.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.263+0000 7ff891327700 1 --2- 172.21.15.163:0/1787155190 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff88c0b55a0 0x7ff88c0ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:06.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.263+0000 7ff891327700 1 -- 172.21.15.163:0/1787155190 >> v1:172.21.15.181:6789/0 conn(0x7ff88c159ec0 legacy=0x7ff88c0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:06.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.263+0000 7ff891327700 1 -- 172.21.15.163:0/1787155190 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88c0cc100 con 0x7ff88c0cd910 2023-10-09T06:37:06.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.263+0000 7ff890b26700 1 -- 172.21.15.163:0/1787155190 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3703221659 0 0) 0x7ff88c0b5b00 con 0x7ff88c159ec0 2023-10-09T06:37:06.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.263+0000 7ff892329700 1 --2- 172.21.15.163:0/1787155190 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff88c0b55a0 0x7ff88c0ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:37:06.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.263+0000 7ff891327700 1 --2- 172.21.15.163:0/1787155190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff88c0cd910 0x7ff88c0ce340 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0099e0 tx=0x7ff87c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c8cb861e3082353 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:06.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.263+0000 7ff890b26700 1 -- 172.21.15.163:0/1787155190 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c00b820 con 0x7ff88c0cd910 2023-10-09T06:37:06.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.263+0000 7ff890b26700 1 -- 172.21.15.163:0/1787155190 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87c00b980 con 0x7ff88c0cd910 2023-10-09T06:37:06.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.263+0000 7ff890b26700 1 -- 172.21.15.163:0/1787155190 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c00bd20 con 0x7ff88c0cd910 2023-10-09T06:37:06.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.264+0000 7ff893d8c700 1 -- 172.21.15.163:0/1787155190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff88c0cd910 msgr2=0x7ff88c0ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:06.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.264+0000 7ff893d8c700 1 --2- 172.21.15.163:0/1787155190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff88c0cd910 0x7ff88c0ce340 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0099e0 tx=0x7ff87c009250 comp rx=0 tx=0).stop 2023-10-09T06:37:06.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.264+0000 7ff893d8c700 1 -- 172.21.15.163:0/1787155190 shutdown_connections 2023-10-09T06:37:06.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.264+0000 7ff893d8c700 1 --2- 172.21.15.163:0/1787155190 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff88c0b55a0 0x7ff88c0ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:06.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.264+0000 7ff893d8c700 1 --2- 172.21.15.163:0/1787155190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff88c0cd910 0x7ff88c0ce340 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:06.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.264+0000 7ff893d8c700 1 -- 172.21.15.163:0/1787155190 >> 172.21.15.163:0/1787155190 conn(0x7ff88c0999f0 msgr2=0x7ff88c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:06.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.264+0000 7ff893d8c700 1 -- 172.21.15.163:0/1787155190 shutdown_connections 2023-10-09T06:37:06.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.264+0000 7ff893d8c700 1 -- 172.21.15.163:0/1787155190 wait complete. 2023-10-09T06:37:06.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.265+0000 7ff893d8c700 1 Processor -- start 2023-10-09T06:37:06.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.265+0000 7ff893d8c700 1 -- start start 2023-10-09T06:37:06.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.266+0000 7ff893d8c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff88c0b55a0 0x7ff88c16db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:06.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.266+0000 7ff893d8c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff88c159ec0 0x7ff88c169bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:06.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.266+0000 7ff893d8c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff88c157020 0x7ff88c09cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:06.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.266+0000 7ff893d8c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c16ab10 con 0x7ff88c0b55a0 2023-10-09T06:37:06.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.266+0000 7ff893d8c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c16d680 con 0x7ff88c157020 2023-10-09T06:37:06.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.266+0000 7ff893d8c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c16e050 con 0x7ff88c159ec0 2023-10-09T06:37:06.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.266+0000 7ff891b28700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff88c0b55a0 0x7ff88c16db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:06.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.266+0000 7ff891b28700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff88c0b55a0 0x7ff88c16db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51566/0 (socket says 172.21.15.163:51566) 2023-10-09T06:37:06.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.266+0000 7ff891b28700 1 -- 172.21.15.163:0/3735084390 learned_addr learned my addr 172.21.15.163:0/3735084390 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:06.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.266+0000 7ff891327700 1 --2- 172.21.15.163:0/3735084390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff88c159ec0 0x7ff88c169bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:06.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.266+0000 7ff892329700 1 --2- 172.21.15.163:0/3735084390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff88c157020 0x7ff88c09cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:06.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.266+0000 7ff891b28700 1 -- 172.21.15.163:0/3735084390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff88c159ec0 msgr2=0x7ff88c169bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:06.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.266+0000 7ff891b28700 1 --2- 172.21.15.163:0/3735084390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff88c159ec0 0x7ff88c169bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:06.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.266+0000 7ff891b28700 1 -- 172.21.15.163:0/3735084390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff88c157020 msgr2=0x7ff88c09cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:06.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.266+0000 7ff891b28700 1 --2- 172.21.15.163:0/3735084390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff88c157020 0x7ff88c09cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:06.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.266+0000 7ff891b28700 1 -- 172.21.15.163:0/3735084390 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff884009a20 con 0x7ff88c0b55a0 2023-10-09T06:37:06.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.267+0000 7ff891b28700 1 --2- 172.21.15.163:0/3735084390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff88c0b55a0 0x7ff88c16db10 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7ff88800c810 tx=0x7ff888007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:06.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.267+0000 7ff881ffb700 1 -- 172.21.15.163:0/3735084390 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff888011590 con 0x7ff88c0b55a0 2023-10-09T06:37:06.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.267+0000 7ff881ffb700 1 -- 172.21.15.163:0/3735084390 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8880116f0 con 0x7ff88c0b55a0 2023-10-09T06:37:06.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.267+0000 7ff893d8c700 1 -- 172.21.15.163:0/3735084390 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff87c0094f0 con 0x7ff88c0b55a0 2023-10-09T06:37:06.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.267+0000 7ff881ffb700 1 -- 172.21.15.163:0/3735084390 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff888011ab0 con 0x7ff88c0b55a0 2023-10-09T06:37:06.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.267+0000 7ff893d8c700 1 -- 172.21.15.163:0/3735084390 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff88c16a830 con 0x7ff88c0b55a0 2023-10-09T06:37:06.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.268+0000 7ff893d8c700 1 -- 172.21.15.163:0/3735084390 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff88c0c9fd0 con 0x7ff88c0b55a0 2023-10-09T06:37:06.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.268+0000 7ff881ffb700 1 -- 172.21.15.163:0/3735084390 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff88801b3b0 con 0x7ff88c0b55a0 2023-10-09T06:37:06.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.268+0000 7ff881ffb700 1 --2- 172.21.15.163:0/3735084390 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff870067320 0x7ff8700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:06.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.269+0000 7ff881ffb700 1 -- 172.21.15.163:0/3735084390 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff888083de0 con 0x7ff88c0b55a0 2023-10-09T06:37:06.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.269+0000 7ff891327700 1 --2- 172.21.15.163:0/3735084390 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff870067320 0x7ff8700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:06.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.269+0000 7ff891327700 1 --2- 172.21.15.163:0/3735084390 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff870067320 0x7ff8700697d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0099e0 tx=0x7ff87c017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:06.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.273+0000 7ff881ffb700 1 -- 172.21.15.163:0/3735084390 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff888050e80 con 0x7ff88c0b55a0 2023-10-09T06:37:06.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.455+0000 7ff893d8c700 1 -- 172.21.15.163:0/3735084390 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff88c063c90 con 0x7ff88c0b55a0 2023-10-09T06:37:06.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.744+0000 7ff881ffb700 1 -- 172.21.15.163:0/3735084390 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v110) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff888017020 con 0x7ff88c0b55a0 2023-10-09T06:37:06.747 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.747+0000 7ff893d8c700 1 -- 172.21.15.163:0/3735084390 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff870067320 msgr2=0x7ff8700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:06.747 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.747+0000 7ff893d8c700 1 --2- 172.21.15.163:0/3735084390 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff870067320 0x7ff8700697d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0099e0 tx=0x7ff87c017000 comp rx=0 tx=0).stop 2023-10-09T06:37:06.747 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.747+0000 7ff893d8c700 1 -- 172.21.15.163:0/3735084390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff88c0b55a0 msgr2=0x7ff88c16db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:06.747 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.747+0000 7ff893d8c700 1 --2- 172.21.15.163:0/3735084390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff88c0b55a0 0x7ff88c16db10 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7ff88800c810 tx=0x7ff888007dd0 comp rx=0 tx=0).stop 2023-10-09T06:37:06.747 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.747+0000 7ff893d8c700 1 -- 172.21.15.163:0/3735084390 shutdown_connections 2023-10-09T06:37:06.747 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.747+0000 7ff893d8c700 1 --2- 172.21.15.163:0/3735084390 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff870067320 0x7ff8700697d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:06.747 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.747+0000 7ff893d8c700 1 --2- 172.21.15.163:0/3735084390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff88c0b55a0 0x7ff88c16db10 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:06.747 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.747+0000 7ff893d8c700 1 --2- 172.21.15.163:0/3735084390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff88c157020 0x7ff88c09cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:06.748 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.747+0000 7ff893d8c700 1 --2- 172.21.15.163:0/3735084390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff88c159ec0 0x7ff88c169bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:06.748 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.747+0000 7ff893d8c700 1 -- 172.21.15.163:0/3735084390 >> 172.21.15.163:0/3735084390 conn(0x7ff88c0999f0 msgr2=0x7ff88c0c98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:06.748 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.747+0000 7ff893d8c700 1 -- 172.21.15.163:0/3735084390 shutdown_connections 2023-10-09T06:37:06.748 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.747+0000 7ff893d8c700 1 -- 172.21.15.163:0/3735084390 wait complete. 2023-10-09T06:37:06.753 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_metadata' created 2023-10-09T06:37:06.762 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-09T06:37:06.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.883+0000 7f7a9cc7f700 1 Processor -- start 2023-10-09T06:37:06.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.883+0000 7f7a9cc7f700 1 -- start start 2023-10-09T06:37:06.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.884+0000 7f7a9cc7f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a980cbb90 0x7f7a980af320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:06.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.884+0000 7f7a9cc7f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a980af860 con 0x7f7a98069c70 2023-10-09T06:37:06.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.884+0000 7f7a9cc7f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a980cbff0 con 0x7f7a98159950 2023-10-09T06:37:06.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.884+0000 7f7a9cc7f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a980ce380 con 0x7f7a980cbb90 2023-10-09T06:37:06.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.884+0000 7f7a95d9b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f7a98069c70 0x7f7a980cf0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:49932/0 (socket says 172.21.15.163:49932) 2023-10-09T06:37:06.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.884+0000 7f7a95d9b700 1 -- 172.21.15.163:0/2542691218 learned_addr learned my addr 172.21.15.163:0/2542691218 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:06.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.884+0000 7f7a96d9d700 1 --2- 172.21.15.163:0/2542691218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a980cbb90 0x7f7a980af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:06.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.884+0000 7f7a9559a700 1 -- 172.21.15.163:0/2542691218 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 385121580 0 0) 0x7f7a980af860 con 0x7f7a98069c70 2023-10-09T06:37:06.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.884+0000 7f7a9559a700 1 -- 172.21.15.163:0/2542691218 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a80003570 con 0x7f7a98069c70 2023-10-09T06:37:06.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.885+0000 7f7a9559a700 1 -- 172.21.15.163:0/2542691218 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3452600556 0 0) 0x7f7a80003570 con 0x7f7a98069c70 2023-10-09T06:37:06.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.885+0000 7f7a9559a700 1 -- 172.21.15.163:0/2542691218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a980cbb90 msgr2=0x7f7a980af320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:06.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.885+0000 7f7a9559a700 1 --2- 172.21.15.163:0/2542691218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a980cbb90 0x7f7a980af320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:06.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.885+0000 7f7a9559a700 1 -- 172.21.15.163:0/2542691218 >> v1:172.21.15.181:6789/0 conn(0x7f7a98159950 legacy=0x7f7a98069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:06.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.885+0000 7f7a9559a700 1 -- 172.21.15.163:0/2542691218 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a980cec20 con 0x7f7a98069c70 2023-10-09T06:37:06.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.885+0000 7f7a9559a700 1 -- 172.21.15.163:0/2542691218 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f7a8c001470 con 0x7f7a98069c70 2023-10-09T06:37:06.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.885+0000 7f7a9559a700 1 -- 172.21.15.163:0/2542691218 >> v1:172.21.15.163:6789/0 conn(0x7f7a98069c70 legacy=0x7f7a980cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:06.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.885+0000 7f7a9559a700 1 --2- 172.21.15.163:0/2542691218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a80003a90 0x7f7a80003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:06.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.885+0000 7f7a9559a700 1 --2- 172.21.15.163:0/2542691218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a80004810 0x7f7a80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:06.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.885+0000 7f7a9559a700 1 --2- 172.21.15.163:0/2542691218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a980cbb90 0x7f7a800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:06.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.885+0000 7f7a9559a700 1 -- 172.21.15.163:0/2542691218 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a980ce380 con 0x7f7a80004810 2023-10-09T06:37:06.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.885+0000 7f7a9559a700 1 -- 172.21.15.163:0/2542691218 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8000bb10 con 0x7f7a80003a90 2023-10-09T06:37:06.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.885+0000 7f7a9559a700 1 -- 172.21.15.163:0/2542691218 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8000bcd0 con 0x7f7a980cbb90 2023-10-09T06:37:06.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.885+0000 7f7a95d9b700 1 --2- 172.21.15.163:0/2542691218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a80004810 0x7f7a80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:06.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.885+0000 7f7a96d9d700 1 --2- 172.21.15.163:0/2542691218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a980cbb90 0x7f7a800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:06.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.885+0000 7f7a9659c700 1 --2- 172.21.15.163:0/2542691218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a80003a90 0x7f7a80003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:06.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.885+0000 7f7a95d9b700 1 -- 172.21.15.163:0/2542691218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a980cbb90 msgr2=0x7f7a800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:06.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.885+0000 7f7a95d9b700 1 --2- 172.21.15.163:0/2542691218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a980cbb90 0x7f7a800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:06.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.885+0000 7f7a95d9b700 1 -- 172.21.15.163:0/2542691218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a80003a90 msgr2=0x7f7a80003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:06.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.885+0000 7f7a95d9b700 1 --2- 172.21.15.163:0/2542691218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a80003a90 0x7f7a80003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:06.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.885+0000 7f7a95d9b700 1 -- 172.21.15.163:0/2542691218 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a80003570 con 0x7f7a80004810 2023-10-09T06:37:06.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.885+0000 7f7a95d9b700 1 --2- 172.21.15.163:0/2542691218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a80004810 0x7f7a80004c80 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c00c2d0 tx=0x7f7a980ceb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b69bb9a27ba851 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:06.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.886+0000 7f7a9559a700 1 -- 172.21.15.163:0/2542691218 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c00cc00 con 0x7f7a80004810 2023-10-09T06:37:06.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.886+0000 7f7a9559a700 1 -- 172.21.15.163:0/2542691218 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a8c00cd60 con 0x7f7a80004810 2023-10-09T06:37:06.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.886+0000 7f7a9559a700 1 -- 172.21.15.163:0/2542691218 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c005b70 con 0x7f7a80004810 2023-10-09T06:37:06.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.886+0000 7f7a9cc7f700 1 -- 172.21.15.163:0/2542691218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a80004810 msgr2=0x7f7a80004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:06.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.886+0000 7f7a9cc7f700 1 --2- 172.21.15.163:0/2542691218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a80004810 0x7f7a80004c80 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c00c2d0 tx=0x7f7a980ceb90 comp rx=0 tx=0).stop 2023-10-09T06:37:06.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.886+0000 7f7a96d9d700 1 -- 172.21.15.163:0/2542691218 reap_dead start 2023-10-09T06:37:06.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.886+0000 7f7a9cc7f700 1 -- 172.21.15.163:0/2542691218 shutdown_connections 2023-10-09T06:37:06.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.886+0000 7f7a9cc7f700 1 -- 172.21.15.163:0/2542691218 >> 172.21.15.163:0/2542691218 conn(0x7f7a98068e00 msgr2=0x7f7a980d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:06.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.886+0000 7f7a9cc7f700 1 -- 172.21.15.163:0/2542691218 shutdown_connections 2023-10-09T06:37:06.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.886+0000 7f7a9cc7f700 1 -- 172.21.15.163:0/2542691218 wait complete. 2023-10-09T06:37:06.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.887+0000 7f7a9cc7f700 1 Processor -- start 2023-10-09T06:37:06.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.887+0000 7f7a9cc7f700 1 -- start start 2023-10-09T06:37:06.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.888+0000 7f7a9cc7f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a80004810 0x7f7a9815ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:06.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.888+0000 7f7a9cc7f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a980ca110 0x7f7a9815d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:06.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.888+0000 7f7a9cc7f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a981523f0 0x7f7a980b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:06.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.888+0000 7f7a9cc7f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a980c98c0 con 0x7f7a981523f0 2023-10-09T06:37:06.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.888+0000 7f7a9cc7f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9815d7e0 con 0x7f7a980ca110 2023-10-09T06:37:06.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.888+0000 7f7a9cc7f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9815daa0 con 0x7f7a80004810 2023-10-09T06:37:06.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.888+0000 7f7a96d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a981523f0 0x7f7a980b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:06.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.888+0000 7f7a96d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a981523f0 0x7f7a980b55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51582/0 (socket says 172.21.15.163:51582) 2023-10-09T06:37:06.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.888+0000 7f7a96d9d700 1 -- 172.21.15.163:0/3238750721 learned_addr learned my addr 172.21.15.163:0/3238750721 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:06.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.888+0000 7f7a9659c700 1 --2- 172.21.15.163:0/3238750721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a80004810 0x7f7a9815ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:06.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.888+0000 7f7a95d9b700 1 --2- 172.21.15.163:0/3238750721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a980ca110 0x7f7a9815d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:06.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.888+0000 7f7a96d9d700 1 -- 172.21.15.163:0/3238750721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a80004810 msgr2=0x7f7a9815ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:06.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.888+0000 7f7a96d9d700 1 --2- 172.21.15.163:0/3238750721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a80004810 0x7f7a9815ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:06.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.888+0000 7f7a96d9d700 1 -- 172.21.15.163:0/3238750721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a980ca110 msgr2=0x7f7a9815d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:06.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.888+0000 7f7a96d9d700 1 --2- 172.21.15.163:0/3238750721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a980ca110 0x7f7a9815d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:06.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.888+0000 7f7a96d9d700 1 -- 172.21.15.163:0/3238750721 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a98159dc0 con 0x7f7a981523f0 2023-10-09T06:37:06.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.889+0000 7f7a96d9d700 1 --2- 172.21.15.163:0/3238750721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a981523f0 0x7f7a980b55e0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f7a98089b50 tx=0x7f7a900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:06.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.889+0000 7f7a867fc700 1 -- 172.21.15.163:0/3238750721 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a90013750 con 0x7f7a981523f0 2023-10-09T06:37:06.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.889+0000 7f7a867fc700 1 -- 172.21.15.163:0/3238750721 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a900138b0 con 0x7f7a981523f0 2023-10-09T06:37:06.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.889+0000 7f7a867fc700 1 -- 172.21.15.163:0/3238750721 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a90013c50 con 0x7f7a981523f0 2023-10-09T06:37:06.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.889+0000 7f7a9cc7f700 1 -- 172.21.15.163:0/3238750721 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a9815a0a0 con 0x7f7a981523f0 2023-10-09T06:37:06.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.889+0000 7f7a9cc7f700 1 -- 172.21.15.163:0/3238750721 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a9815a660 con 0x7f7a981523f0 2023-10-09T06:37:06.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.890+0000 7f7a9cc7f700 1 -- 172.21.15.163:0/3238750721 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a9815a880 con 0x7f7a981523f0 2023-10-09T06:37:06.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.890+0000 7f7a867fc700 1 -- 172.21.15.163:0/3238750721 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7a90014440 con 0x7f7a981523f0 2023-10-09T06:37:06.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.890+0000 7f7a867fc700 1 --2- 172.21.15.163:0/3238750721 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7a78067320 0x7f7a780697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:06.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.891+0000 7f7a867fc700 1 -- 172.21.15.163:0/3238750721 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f7a90085050 con 0x7f7a981523f0 2023-10-09T06:37:06.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.891+0000 7f7a9659c700 1 --2- 172.21.15.163:0/3238750721 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7a78067320 0x7f7a780697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:06.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.891+0000 7f7a9659c700 1 --2- 172.21.15.163:0/3238750721 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7a78067320 0x7f7a780697d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7a880013d0 tx=0x7f7a88001920 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:06.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:06.895+0000 7f7a867fc700 1 -- 172.21.15.163:0/3238750721 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a90051f40 con 0x7f7a981523f0 2023-10-09T06:37:07.078 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.079+0000 7f7a9cc7f700 1 -- 172.21.15.163:0/3238750721 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7a980718b0 con 0x7f7a981523f0 2023-10-09T06:37:07.747 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.747+0000 7f7a867fc700 1 -- 172.21.15.163:0/3238750721 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v111) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7a90055560 con 0x7f7a981523f0 2023-10-09T06:37:07.750 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.750+0000 7f7a9cc7f700 1 -- 172.21.15.163:0/3238750721 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7a78067320 msgr2=0x7f7a780697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:07.750 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.750+0000 7f7a9cc7f700 1 --2- 172.21.15.163:0/3238750721 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7a78067320 0x7f7a780697d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7a880013d0 tx=0x7f7a88001920 comp rx=0 tx=0).stop 2023-10-09T06:37:07.750 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.750+0000 7f7a9cc7f700 1 -- 172.21.15.163:0/3238750721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a981523f0 msgr2=0x7f7a980b55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:07.750 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.750+0000 7f7a9cc7f700 1 --2- 172.21.15.163:0/3238750721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a981523f0 0x7f7a980b55e0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f7a98089b50 tx=0x7f7a900091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:07.752 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.752+0000 7f7a9cc7f700 1 -- 172.21.15.163:0/3238750721 shutdown_connections 2023-10-09T06:37:07.752 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.752+0000 7f7a9cc7f700 1 --2- 172.21.15.163:0/3238750721 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7a78067320 0x7f7a780697d0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:07.752 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.752+0000 7f7a9cc7f700 1 --2- 172.21.15.163:0/3238750721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a981523f0 0x7f7a980b55e0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:07.752 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.752+0000 7f7a9cc7f700 1 --2- 172.21.15.163:0/3238750721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a80004810 0x7f7a9815ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:07.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.752+0000 7f7a9cc7f700 1 --2- 172.21.15.163:0/3238750721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a980ca110 0x7f7a9815d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:07.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.753+0000 7f7a9cc7f700 1 -- 172.21.15.163:0/3238750721 >> 172.21.15.163:0/3238750721 conn(0x7f7a98068e00 msgr2=0x7f7a98152d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:07.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.753+0000 7f7a9cc7f700 1 -- 172.21.15.163:0/3238750721 shutdown_connections 2023-10-09T06:37:07.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.753+0000 7f7a9cc7f700 1 -- 172.21.15.163:0/3238750721 wait complete. 2023-10-09T06:37:07.757 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_data' created 2023-10-09T06:37:07.774 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-09T06:37:07.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.892+0000 7f073aaf4700 1 Processor -- start 2023-10-09T06:37:07.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.892+0000 7f073aaf4700 1 -- start start 2023-10-09T06:37:07.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.893+0000 7f073aaf4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f07340b16d0 0x7f07340b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:07.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.893+0000 7f073aaf4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07340b33b0 0x7f07340b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:07.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.893+0000 7f073aaf4700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07340b2080 con 0x7f07340b4e70 2023-10-09T06:37:07.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.893+0000 7f073aaf4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07340cfb00 con 0x7f07340b16d0 2023-10-09T06:37:07.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.893+0000 7f073aaf4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07340b5c40 con 0x7f07340b33b0 2023-10-09T06:37:07.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.893+0000 7f0739af2700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f07340b4e70 0x7f07340ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:49946/0 (socket says 172.21.15.163:49946) 2023-10-09T06:37:07.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.893+0000 7f0739af2700 1 -- 172.21.15.163:0/2219701411 learned_addr learned my addr 172.21.15.163:0/2219701411 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:07.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.893+0000 7f07392f1700 1 --2- 172.21.15.163:0/2219701411 >> v2:172.21.15.181:3300/0 conn(0x7f07340b16d0 0x7f07340b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:07.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.893+0000 7f073a2f3700 1 --2- 172.21.15.163:0/2219701411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07340b33b0 0x7f07340b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:07.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.893+0000 7f0738af0700 1 -- 172.21.15.163:0/2219701411 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2785636889 0 0) 0x7f07340b2080 con 0x7f07340b4e70 2023-10-09T06:37:07.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.893+0000 7f0738af0700 1 -- 172.21.15.163:0/2219701411 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0720003570 con 0x7f07340b4e70 2023-10-09T06:37:07.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.894+0000 7f0738af0700 1 -- 172.21.15.163:0/2219701411 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 509648782 0 0) 0x7f0720003570 con 0x7f07340b4e70 2023-10-09T06:37:07.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.894+0000 7f0738af0700 1 -- 172.21.15.163:0/2219701411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07340b33b0 msgr2=0x7f07340b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:07.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.894+0000 7f0738af0700 1 --2- 172.21.15.163:0/2219701411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07340b33b0 0x7f07340b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:07.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.894+0000 7f0738af0700 1 -- 172.21.15.163:0/2219701411 >> v2:172.21.15.181:3300/0 conn(0x7f07340b16d0 msgr2=0x7f07340b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:07.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.894+0000 7f0738af0700 1 --2- 172.21.15.163:0/2219701411 >> v2:172.21.15.181:3300/0 conn(0x7f07340b16d0 0x7f07340b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:07.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.894+0000 7f0738af0700 1 -- 172.21.15.163:0/2219701411 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07340b5f20 con 0x7f07340b4e70 2023-10-09T06:37:07.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.894+0000 7f0738af0700 1 -- 172.21.15.163:0/2219701411 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f0730001470 con 0x7f07340b4e70 2023-10-09T06:37:07.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.894+0000 7f0738af0700 1 -- 172.21.15.163:0/2219701411 >> v1:172.21.15.163:6789/0 conn(0x7f07340b4e70 legacy=0x7f07340ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:07.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.894+0000 7f0738af0700 1 --2- 172.21.15.163:0/2219701411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0720003a60 0x7f0720003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:07.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.894+0000 7f0738af0700 1 --2- 172.21.15.163:0/2219701411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0720004810 0x7f0720004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:07.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.894+0000 7f0738af0700 1 --2- 172.21.15.163:0/2219701411 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07200051c0 0x7f072000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:07.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.894+0000 7f0738af0700 1 -- 172.21.15.163:0/2219701411 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07340cfb00 con 0x7f07200051c0 2023-10-09T06:37:07.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.894+0000 7f0738af0700 1 -- 172.21.15.163:0/2219701411 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07340b5c40 con 0x7f0720004810 2023-10-09T06:37:07.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.894+0000 7f0738af0700 1 -- 172.21.15.163:0/2219701411 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f072000c020 con 0x7f0720003a60 2023-10-09T06:37:07.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.894+0000 7f073a2f3700 1 --2- 172.21.15.163:0/2219701411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07340b33b0 0x7f07340b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:37:07.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.894+0000 7f07392f1700 1 --2- 172.21.15.163:0/2219701411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0720004810 0x7f0720004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:07.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.894+0000 7f073a2f3700 1 --2- 172.21.15.163:0/2219701411 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07200051c0 0x7f072000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:07.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.894+0000 7f0739af2700 1 --2- 172.21.15.163:0/2219701411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0720003a60 0x7f0720003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:07.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.894+0000 7f073a2f3700 1 -- 172.21.15.163:0/2219701411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0720003a60 msgr2=0x7f0720003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:07.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.894+0000 7f073a2f3700 1 --2- 172.21.15.163:0/2219701411 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0720003a60 0x7f0720003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:07.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.894+0000 7f073a2f3700 1 -- 172.21.15.163:0/2219701411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0720004810 msgr2=0x7f0720004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:07.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.894+0000 7f073a2f3700 1 --2- 172.21.15.163:0/2219701411 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0720004810 0x7f0720004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:07.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.894+0000 7f073a2f3700 1 -- 172.21.15.163:0/2219701411 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0720003570 con 0x7f07200051c0 2023-10-09T06:37:07.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.894+0000 7f073a2f3700 1 --2- 172.21.15.163:0/2219701411 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07200051c0 0x7f072000ba60 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f07340b3780 tx=0x7f072c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6545ad47efb47f server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:07.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.894+0000 7f0738af0700 1 -- 172.21.15.163:0/2219701411 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c00eaa0 con 0x7f07200051c0 2023-10-09T06:37:07.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.895+0000 7f0738af0700 1 -- 172.21.15.163:0/2219701411 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072c00f040 con 0x7f07200051c0 2023-10-09T06:37:07.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.895+0000 7f0738af0700 1 -- 172.21.15.163:0/2219701411 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c00ee30 con 0x7f07200051c0 2023-10-09T06:37:07.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.895+0000 7f073aaf4700 1 -- 172.21.15.163:0/2219701411 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07200051c0 msgr2=0x7f072000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:07.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.895+0000 7f073aaf4700 1 --2- 172.21.15.163:0/2219701411 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07200051c0 0x7f072000ba60 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f07340b3780 tx=0x7f072c00b040 comp rx=0 tx=0).stop 2023-10-09T06:37:07.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.895+0000 7f073a2f3700 1 -- 172.21.15.163:0/2219701411 reap_dead start 2023-10-09T06:37:07.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.895+0000 7f073aaf4700 1 -- 172.21.15.163:0/2219701411 shutdown_connections 2023-10-09T06:37:07.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.895+0000 7f073aaf4700 1 -- 172.21.15.163:0/2219701411 >> 172.21.15.163:0/2219701411 conn(0x7f07340d10c0 msgr2=0x7f07340affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:07.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.895+0000 7f073aaf4700 1 -- 172.21.15.163:0/2219701411 shutdown_connections 2023-10-09T06:37:07.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.895+0000 7f073aaf4700 1 -- 172.21.15.163:0/2219701411 wait complete. 2023-10-09T06:37:07.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.896+0000 7f073aaf4700 1 Processor -- start 2023-10-09T06:37:07.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.896+0000 7f073aaf4700 1 -- start start 2023-10-09T06:37:07.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.897+0000 7f073aaf4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0734152e80 0x7f073416a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:07.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.897+0000 7f073aaf4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0734153250 0x7f0734157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:07.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.897+0000 7f073aaf4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f073416d500 0x7f0734157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:07.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.897+0000 7f073aaf4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073416ae70 con 0x7f073416d500 2023-10-09T06:37:07.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.897+0000 7f073aaf4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0734156a30 con 0x7f0734152e80 2023-10-09T06:37:07.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.897+0000 7f073aaf4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07340c6ec0 con 0x7f0734153250 2023-10-09T06:37:07.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.897+0000 7f073a2f3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f073416d500 0x7f0734157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:07.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.897+0000 7f073a2f3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f073416d500 0x7f0734157bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51592/0 (socket says 172.21.15.163:51592) 2023-10-09T06:37:07.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.897+0000 7f073a2f3700 1 -- 172.21.15.163:0/3320412628 learned_addr learned my addr 172.21.15.163:0/3320412628 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:07.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.897+0000 7f07392f1700 1 --2- 172.21.15.163:0/3320412628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0734153250 0x7f0734157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:07.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.897+0000 7f0739af2700 1 --2- 172.21.15.163:0/3320412628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0734152e80 0x7f073416a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:07.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.897+0000 7f073a2f3700 1 -- 172.21.15.163:0/3320412628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0734153250 msgr2=0x7f0734157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:07.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.897+0000 7f073a2f3700 1 --2- 172.21.15.163:0/3320412628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0734153250 0x7f0734157670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:07.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.897+0000 7f073a2f3700 1 -- 172.21.15.163:0/3320412628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0734152e80 msgr2=0x7f073416a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:07.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.897+0000 7f073a2f3700 1 --2- 172.21.15.163:0/3320412628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0734152e80 0x7f073416a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:07.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.897+0000 7f073a2f3700 1 -- 172.21.15.163:0/3320412628 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f073416dac0 con 0x7f073416d500 2023-10-09T06:37:07.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.897+0000 7f07392f1700 1 --2- 172.21.15.163:0/3320412628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0734153250 0x7f0734157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:37:07.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.897+0000 7f073a2f3700 1 --2- 172.21.15.163:0/3320412628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f073416d500 0x7f0734157bb0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f072c0099f0 tx=0x7f072c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:07.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.898+0000 7f0729ffb700 1 -- 172.21.15.163:0/3320412628 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c028470 con 0x7f073416d500 2023-10-09T06:37:07.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.898+0000 7f0729ffb700 1 -- 172.21.15.163:0/3320412628 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072c0285f0 con 0x7f073416d500 2023-10-09T06:37:07.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.898+0000 7f073aaf4700 1 -- 172.21.15.163:0/3320412628 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f073416dda0 con 0x7f073416d500 2023-10-09T06:37:07.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.898+0000 7f0729ffb700 1 -- 172.21.15.163:0/3320412628 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c0289b0 con 0x7f073416d500 2023-10-09T06:37:07.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.898+0000 7f073aaf4700 1 -- 172.21.15.163:0/3320412628 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f073416e360 con 0x7f073416d500 2023-10-09T06:37:07.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.899+0000 7f073aaf4700 1 -- 172.21.15.163:0/3320412628 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0734088990 con 0x7f073416d500 2023-10-09T06:37:07.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.899+0000 7f0729ffb700 1 -- 172.21.15.163:0/3320412628 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f072c025070 con 0x7f073416d500 2023-10-09T06:37:07.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.900+0000 7f0729ffb700 1 --2- 172.21.15.163:0/3320412628 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0718067320 0x7f07180697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:07.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.900+0000 7f0729ffb700 1 -- 172.21.15.163:0/3320412628 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f072c086c80 con 0x7f073416d500 2023-10-09T06:37:07.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.900+0000 7f0739af2700 1 --2- 172.21.15.163:0/3320412628 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0718067320 0x7f07180697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:07.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.900+0000 7f0739af2700 1 --2- 172.21.15.163:0/3320412628 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0718067320 0x7f07180697d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0730004890 tx=0x7f0730009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:07.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:07.904+0000 7f0729ffb700 1 -- 172.21.15.163:0/3320412628 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f072c053930 con 0x7f073416d500 2023-10-09T06:37:08.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.128+0000 7f073aaf4700 1 -- 172.21.15.163:0/3320412628 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0734069b20 con 0x7f073416d500 2023-10-09T06:37:08.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.762+0000 7f0729ffb700 1 -- 172.21.15.163:0/3320412628 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v71) v1 ==== 174+0+0 (secure 0 0 0) 0x7f072c056f50 con 0x7f073416d500 2023-10-09T06:37:08.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.766+0000 7f073aaf4700 1 -- 172.21.15.163:0/3320412628 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0718067320 msgr2=0x7f07180697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:08.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.766+0000 7f073aaf4700 1 --2- 172.21.15.163:0/3320412628 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0718067320 0x7f07180697d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0730004890 tx=0x7f0730009040 comp rx=0 tx=0).stop 2023-10-09T06:37:08.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.766+0000 7f073aaf4700 1 -- 172.21.15.163:0/3320412628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f073416d500 msgr2=0x7f0734157bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:08.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.766+0000 7f073aaf4700 1 --2- 172.21.15.163:0/3320412628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f073416d500 0x7f0734157bb0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f072c0099f0 tx=0x7f072c016000 comp rx=0 tx=0).stop 2023-10-09T06:37:08.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.767+0000 7f073aaf4700 1 -- 172.21.15.163:0/3320412628 shutdown_connections 2023-10-09T06:37:08.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.767+0000 7f073aaf4700 1 --2- 172.21.15.163:0/3320412628 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0718067320 0x7f07180697d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:08.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.767+0000 7f073aaf4700 1 --2- 172.21.15.163:0/3320412628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f073416d500 0x7f0734157bb0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:08.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.767+0000 7f073aaf4700 1 --2- 172.21.15.163:0/3320412628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0734152e80 0x7f073416a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:08.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.767+0000 7f073aaf4700 1 --2- 172.21.15.163:0/3320412628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0734153250 0x7f0734157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:08.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.767+0000 7f073aaf4700 1 -- 172.21.15.163:0/3320412628 >> 172.21.15.163:0/3320412628 conn(0x7f07340d10c0 msgr2=0x7f0734159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:08.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.767+0000 7f073aaf4700 1 -- 172.21.15.163:0/3320412628 shutdown_connections 2023-10-09T06:37:08.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.767+0000 7f073aaf4700 1 -- 172.21.15.163:0/3320412628 wait complete. 2023-10-09T06:37:08.769 INFO:teuthology.orchestra.run.smithi163.stderr:new fs with metadata pool 18 and data pool 19 2023-10-09T06:37:08.782 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:37:08.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.896+0000 7f07a4f7b700 1 Processor -- start 2023-10-09T06:37:08.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.896+0000 7f07a4f7b700 1 -- start start 2023-10-09T06:37:08.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.897+0000 7f07a4f7b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07a0159890 0x7f07a015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:08.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.897+0000 7f07a4f7b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f07a0159c60 0x7f07a015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:08.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.897+0000 7f07a4f7b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07a015a160 con 0x7f07a015db10 2023-10-09T06:37:08.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.897+0000 7f07a4f7b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a00d13a0 con 0x7f07a0159c60 2023-10-09T06:37:08.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.897+0000 7f07a4f7b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a015e140 con 0x7f07a0159890 2023-10-09T06:37:08.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.897+0000 7f0795d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f07a0159c60 0x7f07a015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:08.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.897+0000 7f079e59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07a0159890 0x7f07a015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:08.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.897+0000 7f079ed9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f07a015db10 0x7f07a0069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:34890/0 (socket says 172.21.15.163:34890) 2023-10-09T06:37:08.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.897+0000 7f079ed9d700 1 -- 172.21.15.163:0/609181039 learned_addr learned my addr 172.21.15.163:0/609181039 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:08.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.898+0000 7f0795d9b700 1 -- 172.21.15.163:0/609181039 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07a0159890 msgr2=0x7f07a015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:08.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.898+0000 7f0795d9b700 1 --2- 172.21.15.163:0/609181039 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07a0159890 0x7f07a015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:08.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.898+0000 7f0795d9b700 1 -- 172.21.15.163:0/609181039 >> v1:172.21.15.181:6789/0 conn(0x7f07a015db10 legacy=0x7f07a0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:08.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.898+0000 7f0795d9b700 1 -- 172.21.15.163:0/609181039 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a015a3d0 con 0x7f07a0159c60 2023-10-09T06:37:08.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.898+0000 7f079dd9b700 1 -- 172.21.15.163:0/609181039 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 720055317 0 0) 0x7f07a015a160 con 0x7f07a015db10 2023-10-09T06:37:08.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.898+0000 7f079e59c700 1 --2- 172.21.15.163:0/609181039 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07a0159890 0x7f07a015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:37:08.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.898+0000 7f0795d9b700 1 --2- 172.21.15.163:0/609181039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a0159c60 0x7f07a015d5d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f07880098e0 tx=0x7f07880091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7873e4041f2ffa9a server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:08.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.898+0000 7f079dd9b700 1 -- 172.21.15.163:0/609181039 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078800b640 con 0x7f07a0159c60 2023-10-09T06:37:08.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.898+0000 7f079dd9b700 1 -- 172.21.15.163:0/609181039 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078800b7a0 con 0x7f07a0159c60 2023-10-09T06:37:08.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.898+0000 7f079dd9b700 1 -- 172.21.15.163:0/609181039 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078800bb40 con 0x7f07a0159c60 2023-10-09T06:37:08.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.899+0000 7f07a4f7b700 1 -- 172.21.15.163:0/609181039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a0159c60 msgr2=0x7f07a015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:08.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.899+0000 7f07a4f7b700 1 --2- 172.21.15.163:0/609181039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a0159c60 0x7f07a015d5d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f07880098e0 tx=0x7f07880091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:08.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.899+0000 7f07a4f7b700 1 -- 172.21.15.163:0/609181039 shutdown_connections 2023-10-09T06:37:08.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.899+0000 7f07a4f7b700 1 --2- 172.21.15.163:0/609181039 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07a0159890 0x7f07a015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:08.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.899+0000 7f07a4f7b700 1 --2- 172.21.15.163:0/609181039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a0159c60 0x7f07a015d5d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:08.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.899+0000 7f07a4f7b700 1 -- 172.21.15.163:0/609181039 >> 172.21.15.163:0/609181039 conn(0x7f07a0068e20 msgr2=0x7f07a00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:08.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.899+0000 7f07a4f7b700 1 -- 172.21.15.163:0/609181039 shutdown_connections 2023-10-09T06:37:08.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.899+0000 7f07a4f7b700 1 -- 172.21.15.163:0/609181039 wait complete. 2023-10-09T06:37:08.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.900+0000 7f07a4f7b700 1 Processor -- start 2023-10-09T06:37:08.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.900+0000 7f07a4f7b700 1 -- start start 2023-10-09T06:37:08.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.901+0000 7f07a4f7b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07a0159890 0x7f07a00b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:08.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.901+0000 7f07a4f7b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a0159c60 0x7f07a014c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:08.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.901+0000 7f07a4f7b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a015db10 0x7f07a00cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:08.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.901+0000 7f07a4f7b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a01515a0 con 0x7f07a0159c60 2023-10-09T06:37:08.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.901+0000 7f07a4f7b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a0151840 con 0x7f07a015db10 2023-10-09T06:37:08.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.901+0000 7f07a4f7b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a0151b20 con 0x7f07a0159890 2023-10-09T06:37:08.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.901+0000 7f0795d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a0159c60 0x7f07a014c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:08.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.901+0000 7f0795d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a0159c60 0x7f07a014c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56696/0 (socket says 172.21.15.163:56696) 2023-10-09T06:37:08.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.901+0000 7f0795d9b700 1 -- 172.21.15.163:0/2108580126 learned_addr learned my addr 172.21.15.163:0/2108580126 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:08.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.901+0000 7f079e59c700 1 --2- 172.21.15.163:0/2108580126 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07a0159890 0x7f07a00b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:08.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.901+0000 7f079ed9d700 1 --2- 172.21.15.163:0/2108580126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a015db10 0x7f07a00cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:08.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.901+0000 7f0795d9b700 1 -- 172.21.15.163:0/2108580126 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07a0159890 msgr2=0x7f07a00b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:08.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.901+0000 7f0795d9b700 1 --2- 172.21.15.163:0/2108580126 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07a0159890 0x7f07a00b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:08.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.901+0000 7f0795d9b700 1 -- 172.21.15.163:0/2108580126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a015db10 msgr2=0x7f07a00cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:08.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.901+0000 7f0795d9b700 1 --2- 172.21.15.163:0/2108580126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a015db10 0x7f07a00cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:08.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.901+0000 7f0795d9b700 1 -- 172.21.15.163:0/2108580126 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0790009420 con 0x7f07a0159c60 2023-10-09T06:37:08.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.901+0000 7f079ed9d700 1 --2- 172.21.15.163:0/2108580126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a015db10 0x7f07a00cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:08.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.901+0000 7f0795d9b700 1 --2- 172.21.15.163:0/2108580126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a0159c60 0x7f07a014c600 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f0788013000 tx=0x7f0788017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:08.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.901+0000 7f0796ffd700 1 -- 172.21.15.163:0/2108580126 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0788000d10 con 0x7f07a0159c60 2023-10-09T06:37:08.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.901+0000 7f0796ffd700 1 -- 172.21.15.163:0/2108580126 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078800bb20 con 0x7f07a0159c60 2023-10-09T06:37:08.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.901+0000 7f0796ffd700 1 -- 172.21.15.163:0/2108580126 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0788011650 con 0x7f07a0159c60 2023-10-09T06:37:08.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.901+0000 7f07a4f7b700 1 -- 172.21.15.163:0/2108580126 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0788009420 con 0x7f07a0159c60 2023-10-09T06:37:08.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.902+0000 7f07a4f7b700 1 -- 172.21.15.163:0/2108580126 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a00cdf20 con 0x7f07a0159c60 2023-10-09T06:37:08.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.902+0000 7f07a4f7b700 1 -- 172.21.15.163:0/2108580126 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a004c3c0 con 0x7f07a0159c60 2023-10-09T06:37:08.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.903+0000 7f0796ffd700 1 -- 172.21.15.163:0/2108580126 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0788011a00 con 0x7f07a0159c60 2023-10-09T06:37:08.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.903+0000 7f0796ffd700 1 --2- 172.21.15.163:0/2108580126 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0780067320 0x7f07800697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:08.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.903+0000 7f0796ffd700 1 -- 172.21.15.163:0/2108580126 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0788086280 con 0x7f07a0159c60 2023-10-09T06:37:08.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.903+0000 7f079e59c700 1 --2- 172.21.15.163:0/2108580126 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0780067320 0x7f07800697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:08.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.904+0000 7f079e59c700 1 --2- 172.21.15.163:0/2108580126 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0780067320 0x7f07800697d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0790009f40 tx=0x7f07900091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:08.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:08.908+0000 7f0796ffd700 1 -- 172.21.15.163:0/2108580126 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0788052f10 con 0x7f07a0159c60 2023-10-09T06:37:09.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.091+0000 7f07a4f7b700 1 -- 172.21.15.163:0/2108580126 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07a006a5b0 con 0x7f07a0159c60 2023-10-09T06:37:09.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.092+0000 7f0796ffd700 1 -- 172.21.15.163:0/2108580126 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+15425 (secure 0 0 0) 0x7f0788011cb0 con 0x7f07a0159c60 2023-10-09T06:37:09.093 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:09.093 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":112,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:37:08.128896+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:37:06.472811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-09T06:37:07.096712+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:37:09.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.094+0000 7f07a4f7b700 1 -- 172.21.15.163:0/2108580126 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0780067320 msgr2=0x7f07800697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:09.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.094+0000 7f07a4f7b700 1 --2- 172.21.15.163:0/2108580126 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0780067320 0x7f07800697d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0790009f40 tx=0x7f07900091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:09.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.094+0000 7f07a4f7b700 1 -- 172.21.15.163:0/2108580126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a0159c60 msgr2=0x7f07a014c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:09.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.094+0000 7f07a4f7b700 1 --2- 172.21.15.163:0/2108580126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a0159c60 0x7f07a014c600 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f0788013000 tx=0x7f0788017000 comp rx=0 tx=0).stop 2023-10-09T06:37:09.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.094+0000 7f07a4f7b700 1 -- 172.21.15.163:0/2108580126 shutdown_connections 2023-10-09T06:37:09.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.094+0000 7f07a4f7b700 1 --2- 172.21.15.163:0/2108580126 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0780067320 0x7f07800697d0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:09.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.094+0000 7f07a4f7b700 1 --2- 172.21.15.163:0/2108580126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07a015db10 0x7f07a00cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:09.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.094+0000 7f07a4f7b700 1 --2- 172.21.15.163:0/2108580126 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07a0159890 0x7f07a00b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:09.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.094+0000 7f07a4f7b700 1 --2- 172.21.15.163:0/2108580126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a0159c60 0x7f07a014c600 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:09.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.094+0000 7f07a4f7b700 1 -- 172.21.15.163:0/2108580126 >> 172.21.15.163:0/2108580126 conn(0x7f07a0068e20 msgr2=0x7f07a0154360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:09.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.094+0000 7f07a4f7b700 1 -- 172.21.15.163:0/2108580126 shutdown_connections 2023-10-09T06:37:09.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.095+0000 7f07a4f7b700 1 -- 172.21.15.163:0/2108580126 wait complete. 2023-10-09T06:37:09.108 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:37:09.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.222+0000 7f46a7546700 1 Processor -- start 2023-10-09T06:37:09.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.222+0000 7f46a7546700 1 -- start start 2023-10-09T06:37:09.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.223+0000 7f46a7546700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46a0159420 0x7f46a00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:09.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.223+0000 7f46a7546700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f46a01597f0 0x7f46a00b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:09.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.223+0000 7f46a7546700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f46a015d230 0x7f46a015d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:09.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.223+0000 7f46a7546700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46a015dbd0 con 0x7f46a015d230 2023-10-09T06:37:09.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.223+0000 7f46a7546700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46a00b11d0 con 0x7f46a01597f0 2023-10-09T06:37:09.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.223+0000 7f46a7546700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a00b14b0 con 0x7f46a0159420 2023-10-09T06:37:09.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.223+0000 7f46a6d45700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f46a015d230 0x7f46a015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:09.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.223+0000 7f46a6d45700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f46a015d230 0x7f46a015d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56698/0 (socket says 172.21.15.163:56698) 2023-10-09T06:37:09.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.223+0000 7f46a6d45700 1 -- 172.21.15.163:0/605861415 learned_addr learned my addr 172.21.15.163:0/605861415 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:09.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.223+0000 7f46a6544700 1 --2- 172.21.15.163:0/605861415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46a0159420 0x7f46a00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:09.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.223+0000 7f46a5d43700 1 --2- 172.21.15.163:0/605861415 >> v2:172.21.15.181:3300/0 conn(0x7f46a01597f0 0x7f46a00b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:09.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.224+0000 7f46a6d45700 1 -- 172.21.15.163:0/605861415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46a0159420 msgr2=0x7f46a00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:09.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.224+0000 7f46a6d45700 1 --2- 172.21.15.163:0/605861415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46a0159420 0x7f46a00b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:09.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.224+0000 7f46a6d45700 1 -- 172.21.15.163:0/605861415 >> v2:172.21.15.181:3300/0 conn(0x7f46a01597f0 msgr2=0x7f46a00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:09.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.224+0000 7f46a6d45700 1 --2- 172.21.15.163:0/605861415 >> v2:172.21.15.181:3300/0 conn(0x7f46a01597f0 0x7f46a00b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:09.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.224+0000 7f46a6d45700 1 -- 172.21.15.163:0/605861415 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a00b1790 con 0x7f46a015d230 2023-10-09T06:37:09.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.224+0000 7f46a6544700 1 --2- 172.21.15.163:0/605861415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46a0159420 0x7f46a00b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:37:09.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.224+0000 7f46a5d43700 1 --2- 172.21.15.163:0/605861415 >> v2:172.21.15.181:3300/0 conn(0x7f46a01597f0 0x7f46a00b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:37:09.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.224+0000 7f46a6d45700 1 --2- 172.21.15.163:0/605861415 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46a015d230 0x7f46a015d690 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f46a00d1770 tx=0x7f4698009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e537619b9913b57 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:09.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.224+0000 7f46a5542700 1 -- 172.21.15.163:0/605861415 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469800b640 con 0x7f46a015d230 2023-10-09T06:37:09.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.224+0000 7f46a5542700 1 -- 172.21.15.163:0/605861415 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f469800b7a0 con 0x7f46a015d230 2023-10-09T06:37:09.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.224+0000 7f46a5542700 1 -- 172.21.15.163:0/605861415 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469800bb60 con 0x7f46a015d230 2023-10-09T06:37:09.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.224+0000 7f46a7546700 1 -- 172.21.15.163:0/605861415 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46a015d230 msgr2=0x7f46a015d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:09.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.224+0000 7f46a7546700 1 --2- 172.21.15.163:0/605861415 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46a015d230 0x7f46a015d690 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f46a00d1770 tx=0x7f4698009250 comp rx=0 tx=0).stop 2023-10-09T06:37:09.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.225+0000 7f46a7546700 1 -- 172.21.15.163:0/605861415 shutdown_connections 2023-10-09T06:37:09.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.225+0000 7f46a7546700 1 --2- 172.21.15.163:0/605861415 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46a015d230 0x7f46a015d690 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:09.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.225+0000 7f46a7546700 1 --2- 172.21.15.163:0/605861415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46a0159420 0x7f46a00b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:09.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.225+0000 7f46a7546700 1 --2- 172.21.15.163:0/605861415 >> v2:172.21.15.181:3300/0 conn(0x7f46a01597f0 0x7f46a00b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:09.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.225+0000 7f46a7546700 1 -- 172.21.15.163:0/605861415 >> 172.21.15.163:0/605861415 conn(0x7f46a00d0660 msgr2=0x7f46a00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:09.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.225+0000 7f46a7546700 1 -- 172.21.15.163:0/605861415 shutdown_connections 2023-10-09T06:37:09.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.225+0000 7f46a7546700 1 -- 172.21.15.163:0/605861415 wait complete. 2023-10-09T06:37:09.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.226+0000 7f46a7546700 1 Processor -- start 2023-10-09T06:37:09.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.226+0000 7f46a7546700 1 -- start start 2023-10-09T06:37:09.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.226+0000 7f46a7546700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46a0159420 0x7f46a00695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:09.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.226+0000 7f46a7546700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46a01597f0 0x7f46a0153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:09.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.226+0000 7f46a7546700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46a015d230 0x7f46a0153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:09.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.226+0000 7f46a7546700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a00c55a0 con 0x7f46a01597f0 2023-10-09T06:37:09.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.226+0000 7f46a7546700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a00c5880 con 0x7f46a015d230 2023-10-09T06:37:09.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.226+0000 7f46a7546700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a00c5b60 con 0x7f46a0159420 2023-10-09T06:37:09.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.226+0000 7f46a5d43700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46a01597f0 0x7f46a0153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:09.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.226+0000 7f46a5d43700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46a01597f0 0x7f46a0153430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56714/0 (socket says 172.21.15.163:56714) 2023-10-09T06:37:09.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.226+0000 7f46a5d43700 1 -- 172.21.15.163:0/280376573 learned_addr learned my addr 172.21.15.163:0/280376573 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:09.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.227+0000 7f46a6544700 1 --2- 172.21.15.163:0/280376573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46a0159420 0x7f46a00695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:09.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.227+0000 7f46a6d45700 1 --2- 172.21.15.163:0/280376573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46a015d230 0x7f46a0153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:09.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.227+0000 7f46a5d43700 1 -- 172.21.15.163:0/280376573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46a0159420 msgr2=0x7f46a00695e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:09.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.227+0000 7f46a5d43700 1 --2- 172.21.15.163:0/280376573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46a0159420 0x7f46a00695e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:09.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.227+0000 7f46a5d43700 1 -- 172.21.15.163:0/280376573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46a015d230 msgr2=0x7f46a0153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:09.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.227+0000 7f46a5d43700 1 --2- 172.21.15.163:0/280376573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46a015d230 0x7f46a0153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:09.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.227+0000 7f46a5d43700 1 -- 172.21.15.163:0/280376573 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469c009a20 con 0x7f46a01597f0 2023-10-09T06:37:09.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.227+0000 7f46a6d45700 1 --2- 172.21.15.163:0/280376573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46a015d230 0x7f46a0153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:37:09.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.227+0000 7f46a5d43700 1 --2- 172.21.15.163:0/280376573 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46a01597f0 0x7f46a0153430 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f469000ec80 tx=0x7f46900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:09.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.227+0000 7f46967fc700 1 -- 172.21.15.163:0/280376573 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469000c9d0 con 0x7f46a01597f0 2023-10-09T06:37:09.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.227+0000 7f46967fc700 1 -- 172.21.15.163:0/280376573 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f469000cb30 con 0x7f46a01597f0 2023-10-09T06:37:09.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.227+0000 7f46a7546700 1 -- 172.21.15.163:0/280376573 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4690009420 con 0x7f46a01597f0 2023-10-09T06:37:09.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.227+0000 7f46967fc700 1 -- 172.21.15.163:0/280376573 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469000ceb0 con 0x7f46a01597f0 2023-10-09T06:37:09.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.227+0000 7f46a7546700 1 -- 172.21.15.163:0/280376573 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46980094f0 con 0x7f46a01597f0 2023-10-09T06:37:09.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.228+0000 7f46a7546700 1 -- 172.21.15.163:0/280376573 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a004c3c0 con 0x7f46a01597f0 2023-10-09T06:37:09.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.228+0000 7f46967fc700 1 -- 172.21.15.163:0/280376573 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4690013680 con 0x7f46a01597f0 2023-10-09T06:37:09.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.229+0000 7f46967fc700 1 --2- 172.21.15.163:0/280376573 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4684067320 0x7f46840697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:09.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.229+0000 7f46967fc700 1 -- 172.21.15.163:0/280376573 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4690085090 con 0x7f46a01597f0 2023-10-09T06:37:09.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.229+0000 7f46a6544700 1 --2- 172.21.15.163:0/280376573 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4684067320 0x7f46840697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:09.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.230+0000 7f46a6544700 1 --2- 172.21.15.163:0/280376573 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4684067320 0x7f46840697d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f469c0099f0 tx=0x7f469c00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:09.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.233+0000 7f46967fc700 1 -- 172.21.15.163:0/280376573 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4690051d20 con 0x7f46a01597f0 2023-10-09T06:37:09.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.416+0000 7f46a7546700 1 -- 172.21.15.163:0/280376573 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46a00c56e0 con 0x7f46a01597f0 2023-10-09T06:37:09.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.417+0000 7f46967fc700 1 -- 172.21.15.163:0/280376573 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+15425 (secure 0 0 0) 0x7f46a00c56e0 con 0x7f46a01597f0 2023-10-09T06:37:09.418 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:09.418 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":112,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:37:08.128896+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:37:06.472811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-09T06:37:07.096712+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:37:09.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.419+0000 7f46a7546700 1 -- 172.21.15.163:0/280376573 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4684067320 msgr2=0x7f46840697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:09.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.419+0000 7f46a7546700 1 --2- 172.21.15.163:0/280376573 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4684067320 0x7f46840697d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f469c0099f0 tx=0x7f469c00b040 comp rx=0 tx=0).stop 2023-10-09T06:37:09.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.419+0000 7f46a7546700 1 -- 172.21.15.163:0/280376573 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46a01597f0 msgr2=0x7f46a0153430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:09.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.419+0000 7f46a7546700 1 --2- 172.21.15.163:0/280376573 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46a01597f0 0x7f46a0153430 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f469000ec80 tx=0x7f46900091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:09.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.419+0000 7f46a7546700 1 -- 172.21.15.163:0/280376573 shutdown_connections 2023-10-09T06:37:09.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.419+0000 7f46a7546700 1 --2- 172.21.15.163:0/280376573 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4684067320 0x7f46840697d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:09.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.419+0000 7f46a7546700 1 --2- 172.21.15.163:0/280376573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46a015d230 0x7f46a0153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:09.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.419+0000 7f46a7546700 1 --2- 172.21.15.163:0/280376573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46a0159420 0x7f46a00695e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:09.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.419+0000 7f46a7546700 1 --2- 172.21.15.163:0/280376573 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46a01597f0 0x7f46a0153430 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:09.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.419+0000 7f46a7546700 1 -- 172.21.15.163:0/280376573 >> 172.21.15.163:0/280376573 conn(0x7f46a00d0660 msgr2=0x7f46a0157d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:09.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.419+0000 7f46a7546700 1 -- 172.21.15.163:0/280376573 shutdown_connections 2023-10-09T06:37:09.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.419+0000 7f46a7546700 1 -- 172.21.15.163:0/280376573 wait complete. 2023-10-09T06:37:09.433 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-09T06:37:09.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.551+0000 7f26dc3ab700 1 Processor -- start 2023-10-09T06:37:09.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.551+0000 7f26dc3ab700 1 -- start start 2023-10-09T06:37:09.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.552+0000 7f26dc3ab700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f26d4159890 0x7f26d415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:09.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.552+0000 7f26dc3ab700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26d415a160 con 0x7f26d4069560 2023-10-09T06:37:09.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.552+0000 7f26dc3ab700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26d40d13a0 con 0x7f26d4159c60 2023-10-09T06:37:09.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.552+0000 7f26dc3ab700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26d415e140 con 0x7f26d4159890 2023-10-09T06:37:09.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.552+0000 7f26da147700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f26d4159890 0x7f26d415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:09.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.552+0000 7f26da948700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f26d4069560 0x7f26d40ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:54630/0 (socket says 172.21.15.163:54630) 2023-10-09T06:37:09.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.552+0000 7f26da948700 1 -- 172.21.15.163:0/507941145 learned_addr learned my addr 172.21.15.163:0/507941145 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:09.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.552+0000 7f26da147700 1 -- 172.21.15.163:0/507941145 >> v1:172.21.15.181:6789/0 conn(0x7f26d4159c60 legacy=0x7f26d415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:09.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.552+0000 7f26da147700 1 -- 172.21.15.163:0/507941145 >> v1:172.21.15.163:6789/0 conn(0x7f26d4069560 legacy=0x7f26d40ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:09.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.552+0000 7f26da147700 1 -- 172.21.15.163:0/507941145 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d415a3d0 con 0x7f26d4159890 2023-10-09T06:37:09.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.552+0000 7f26d9145700 1 -- 172.21.15.163:0/507941145 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 23081613 0 0) 0x7f26d415a160 con 0x7f26d4069560 2023-10-09T06:37:09.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.552+0000 7f26da147700 1 --2- 172.21.15.163:0/507941145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26d4159890 0x7f26d415abd0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009e60 tx=0x7f26d0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60ebfcd6c6d05f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:09.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.552+0000 7f26d9145700 1 -- 172.21.15.163:0/507941145 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d000e070 con 0x7f26d4159890 2023-10-09T06:37:09.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.552+0000 7f26d9145700 1 -- 172.21.15.163:0/507941145 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d000bbf0 con 0x7f26d4159890 2023-10-09T06:37:09.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.552+0000 7f26d9145700 1 -- 172.21.15.163:0/507941145 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d0012430 con 0x7f26d4159890 2023-10-09T06:37:09.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.553+0000 7f26dc3ab700 1 -- 172.21.15.163:0/507941145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26d4159890 msgr2=0x7f26d415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:09.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.553+0000 7f26dc3ab700 1 --2- 172.21.15.163:0/507941145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26d4159890 0x7f26d415abd0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009e60 tx=0x7f26d0009780 comp rx=0 tx=0).stop 2023-10-09T06:37:09.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.553+0000 7f26dc3ab700 1 -- 172.21.15.163:0/507941145 shutdown_connections 2023-10-09T06:37:09.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.553+0000 7f26dc3ab700 1 --2- 172.21.15.163:0/507941145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26d4159890 0x7f26d415abd0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:09.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.553+0000 7f26dc3ab700 1 -- 172.21.15.163:0/507941145 >> 172.21.15.163:0/507941145 conn(0x7f26d4068e20 msgr2=0x7f26d40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:09.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.553+0000 7f26dc3ab700 1 -- 172.21.15.163:0/507941145 shutdown_connections 2023-10-09T06:37:09.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.553+0000 7f26dc3ab700 1 -- 172.21.15.163:0/507941145 wait complete. 2023-10-09T06:37:09.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.554+0000 7f26dc3ab700 1 Processor -- start 2023-10-09T06:37:09.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.554+0000 7f26dc3ab700 1 -- start start 2023-10-09T06:37:09.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.555+0000 7f26dc3ab700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26d4069560 0x7f26d40c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:09.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.555+0000 7f26dc3ab700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26d4159890 0x7f26d40ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:09.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.555+0000 7f26dc3ab700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26d4159c60 0x7f26d4169b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:09.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.555+0000 7f26dc3ab700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d416ab80 con 0x7f26d4069560 2023-10-09T06:37:09.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.555+0000 7f26dc3ab700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d416a0a0 con 0x7f26d4159890 2023-10-09T06:37:09.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.555+0000 7f26dc3ab700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d40b4880 con 0x7f26d4159c60 2023-10-09T06:37:09.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.555+0000 7f26da147700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26d4069560 0x7f26d40c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:09.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.555+0000 7f26da147700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26d4069560 0x7f26d40c8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56728/0 (socket says 172.21.15.163:56728) 2023-10-09T06:37:09.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.555+0000 7f26da147700 1 -- 172.21.15.163:0/503803615 learned_addr learned my addr 172.21.15.163:0/503803615 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:09.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.555+0000 7f26d9946700 1 --2- 172.21.15.163:0/503803615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26d4159890 0x7f26d40ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:09.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.555+0000 7f26da948700 1 --2- 172.21.15.163:0/503803615 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26d4159c60 0x7f26d4169b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:09.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.555+0000 7f26da147700 1 -- 172.21.15.163:0/503803615 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26d4159c60 msgr2=0x7f26d4169b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:09.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.555+0000 7f26da147700 1 --2- 172.21.15.163:0/503803615 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26d4159c60 0x7f26d4169b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:09.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.555+0000 7f26da147700 1 -- 172.21.15.163:0/503803615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26d4159890 msgr2=0x7f26d40ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:09.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.555+0000 7f26da147700 1 --2- 172.21.15.163:0/503803615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26d4159890 0x7f26d40ca5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:09.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.555+0000 7f26da147700 1 -- 172.21.15.163:0/503803615 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d0009a20 con 0x7f26d4069560 2023-10-09T06:37:09.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.555+0000 7f26da147700 1 --2- 172.21.15.163:0/503803615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26d4069560 0x7f26d40c8a40 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f26d4070e20 tx=0x7f26d0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:09.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.555+0000 7f26ca7fc700 1 -- 172.21.15.163:0/503803615 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d0027470 con 0x7f26d4069560 2023-10-09T06:37:09.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.555+0000 7f26ca7fc700 1 -- 172.21.15.163:0/503803615 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d0027610 con 0x7f26d4069560 2023-10-09T06:37:09.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.555+0000 7f26dc3ab700 1 -- 172.21.15.163:0/503803615 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d416a2e0 con 0x7f26d4069560 2023-10-09T06:37:09.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.555+0000 7f26ca7fc700 1 -- 172.21.15.163:0/503803615 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d00279d0 con 0x7f26d4069560 2023-10-09T06:37:09.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.556+0000 7f26dc3ab700 1 -- 172.21.15.163:0/503803615 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d416a8a0 con 0x7f26d4069560 2023-10-09T06:37:09.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.556+0000 7f26dc3ab700 1 -- 172.21.15.163:0/503803615 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26d40b4650 con 0x7f26d4069560 2023-10-09T06:37:09.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.557+0000 7f26ca7fc700 1 -- 172.21.15.163:0/503803615 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f26d0024070 con 0x7f26d4069560 2023-10-09T06:37:09.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.557+0000 7f26ca7fc700 1 --2- 172.21.15.163:0/503803615 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f26b8067320 0x7f26b80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:09.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.557+0000 7f26ca7fc700 1 -- 172.21.15.163:0/503803615 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f26d00878b0 con 0x7f26d4069560 2023-10-09T06:37:09.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.558+0000 7f26d9946700 1 --2- 172.21.15.163:0/503803615 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f26b8067320 0x7f26b80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:09.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.558+0000 7f26d9946700 1 --2- 172.21.15.163:0/503803615 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f26b8067320 0x7f26b80697d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f26d40b08a0 tx=0x7f26c40090f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:09.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.561+0000 7f26ca7fc700 1 -- 172.21.15.163:0/503803615 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f26d40b4650 con 0x7f26d4069560 2023-10-09T06:37:09.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:09.792+0000 7f26dc3ab700 1 -- 172.21.15.163:0/503803615 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f26d40b4650 con 0x7f26d4069560 2023-10-09T06:37:10.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.774+0000 7f26ca7fc700 1 -- 172.21.15.163:0/503803615 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v74) v1 ==== 118+0+0 (secure 0 0 0) 0x7f26d40b4650 con 0x7f26d4069560 2023-10-09T06:37:10.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.777+0000 7f26dc3ab700 1 -- 172.21.15.163:0/503803615 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f26b8067320 msgr2=0x7f26b80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:10.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.777+0000 7f26dc3ab700 1 --2- 172.21.15.163:0/503803615 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f26b8067320 0x7f26b80697d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f26d40b08a0 tx=0x7f26c40090f0 comp rx=0 tx=0).stop 2023-10-09T06:37:10.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.777+0000 7f26dc3ab700 1 -- 172.21.15.163:0/503803615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26d4069560 msgr2=0x7f26d40c8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:10.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.777+0000 7f26dc3ab700 1 --2- 172.21.15.163:0/503803615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26d4069560 0x7f26d40c8a40 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f26d4070e20 tx=0x7f26d0018000 comp rx=0 tx=0).stop 2023-10-09T06:37:10.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.777+0000 7f26dc3ab700 1 -- 172.21.15.163:0/503803615 shutdown_connections 2023-10-09T06:37:10.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.777+0000 7f26dc3ab700 1 --2- 172.21.15.163:0/503803615 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f26b8067320 0x7f26b80697d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:10.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.777+0000 7f26dc3ab700 1 --2- 172.21.15.163:0/503803615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26d4069560 0x7f26d40c8a40 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:10.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.777+0000 7f26dc3ab700 1 --2- 172.21.15.163:0/503803615 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26d4159c60 0x7f26d4169b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:10.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.777+0000 7f26dc3ab700 1 --2- 172.21.15.163:0/503803615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26d4159890 0x7f26d40ca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:10.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.777+0000 7f26dc3ab700 1 -- 172.21.15.163:0/503803615 >> 172.21.15.163:0/503803615 conn(0x7f26d4068e20 msgr2=0x7f26d4153060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:10.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.777+0000 7f26dc3ab700 1 -- 172.21.15.163:0/503803615 shutdown_connections 2023-10-09T06:37:10.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.777+0000 7f26dc3ab700 1 -- 172.21.15.163:0/503803615 wait complete. 2023-10-09T06:37:10.791 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-09T06:37:10.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.909+0000 7f0e96c9a700 1 Processor -- start 2023-10-09T06:37:10.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.909+0000 7f0e96c9a700 1 -- start start 2023-10-09T06:37:10.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.910+0000 7f0e96c9a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0e900c9db0 0x7f0e9006a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:10.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.910+0000 7f0e96c9a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0e90157130 0x7f0e9006a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:10.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.910+0000 7f0e96c9a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e9006e150 con 0x7f0e90153900 2023-10-09T06:37:10.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.910+0000 7f0e96c9a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e900c7610 con 0x7f0e900c9db0 2023-10-09T06:37:10.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.910+0000 7f0e96c9a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e900c7800 con 0x7f0e90157130 2023-10-09T06:37:10.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.910+0000 7f0e95497700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0e900c9db0 0x7f0e9006a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:10.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.910+0000 7f0e95497700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0e900c9db0 0x7f0e9006a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56732/0 (socket says 172.21.15.163:56732) 2023-10-09T06:37:10.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.910+0000 7f0e95497700 1 -- 172.21.15.163:0/1863646427 learned_addr learned my addr 172.21.15.163:0/1863646427 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:10.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.910+0000 7f0e96499700 1 --2- 172.21.15.163:0/1863646427 >> v2:172.21.15.181:3300/0 conn(0x7f0e90157130 0x7f0e9006a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:10.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.910+0000 7f0e95497700 1 -- 172.21.15.163:0/1863646427 >> v2:172.21.15.181:3300/0 conn(0x7f0e90157130 msgr2=0x7f0e9006a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:10.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.910+0000 7f0e95497700 1 --2- 172.21.15.163:0/1863646427 >> v2:172.21.15.181:3300/0 conn(0x7f0e90157130 0x7f0e9006a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:10.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.910+0000 7f0e95497700 1 -- 172.21.15.163:0/1863646427 >> v1:172.21.15.181:6789/0 conn(0x7f0e90153900 legacy=0x7f0e900c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:10.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.911+0000 7f0e95497700 1 -- 172.21.15.163:0/1863646427 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e900c7ae0 con 0x7f0e900c9db0 2023-10-09T06:37:10.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.911+0000 7f0e95497700 1 --2- 172.21.15.163:0/1863646427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e900c9db0 0x7f0e9006a000 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f0e900cd2d0 tx=0x7f0e80009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ca72c13d3f8aee server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:10.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.911+0000 7f0e94c96700 1 -- 172.21.15.163:0/1863646427 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e80009e70 con 0x7f0e900c9db0 2023-10-09T06:37:10.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.911+0000 7f0e94c96700 1 -- 172.21.15.163:0/1863646427 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8000b860 con 0x7f0e900c9db0 2023-10-09T06:37:10.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.911+0000 7f0e94c96700 1 -- 172.21.15.163:0/1863646427 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8000bc00 con 0x7f0e900c9db0 2023-10-09T06:37:10.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.911+0000 7f0e96c9a700 1 -- 172.21.15.163:0/1863646427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e900c9db0 msgr2=0x7f0e9006a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:10.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.911+0000 7f0e96c9a700 1 --2- 172.21.15.163:0/1863646427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e900c9db0 0x7f0e9006a000 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f0e900cd2d0 tx=0x7f0e80009250 comp rx=0 tx=0).stop 2023-10-09T06:37:10.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.911+0000 7f0e96c9a700 1 -- 172.21.15.163:0/1863646427 shutdown_connections 2023-10-09T06:37:10.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.911+0000 7f0e96c9a700 1 --2- 172.21.15.163:0/1863646427 >> v2:172.21.15.181:3300/0 conn(0x7f0e90157130 0x7f0e9006a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:10.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.911+0000 7f0e96c9a700 1 --2- 172.21.15.163:0/1863646427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e900c9db0 0x7f0e9006a000 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:10.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.911+0000 7f0e96c9a700 1 -- 172.21.15.163:0/1863646427 >> 172.21.15.163:0/1863646427 conn(0x7f0e900cc1e0 msgr2=0x7f0e900cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:10.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.911+0000 7f0e96c9a700 1 -- 172.21.15.163:0/1863646427 shutdown_connections 2023-10-09T06:37:10.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.911+0000 7f0e96c9a700 1 -- 172.21.15.163:0/1863646427 wait complete. 2023-10-09T06:37:10.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.912+0000 7f0e96c9a700 1 Processor -- start 2023-10-09T06:37:10.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.913+0000 7f0e96c9a700 1 -- start start 2023-10-09T06:37:10.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.913+0000 7f0e96c9a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e900c9db0 0x7f0e9014b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:10.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.913+0000 7f0e96c9a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e90153900 0x7f0e9014f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:10.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.913+0000 7f0e96c9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e90157130 0x7f0e900b0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:10.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.913+0000 7f0e96c9a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9016a0d0 con 0x7f0e90153900 2023-10-09T06:37:10.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.913+0000 7f0e96c9a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e901694c0 con 0x7f0e90157130 2023-10-09T06:37:10.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.913+0000 7f0e96c9a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e901697a0 con 0x7f0e900c9db0 2023-10-09T06:37:10.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.913+0000 7f0e95497700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e90153900 0x7f0e9014f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:10.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.913+0000 7f0e95497700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e90153900 0x7f0e9014f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56738/0 (socket says 172.21.15.163:56738) 2023-10-09T06:37:10.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.913+0000 7f0e95497700 1 -- 172.21.15.163:0/932740912 learned_addr learned my addr 172.21.15.163:0/932740912 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:10.919 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.913+0000 7f0e96499700 1 --2- 172.21.15.163:0/932740912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e90157130 0x7f0e900b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:10.919 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.913+0000 7f0e95c98700 1 --2- 172.21.15.163:0/932740912 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e900c9db0 0x7f0e9014b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:10.919 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.914+0000 7f0e95497700 1 -- 172.21.15.163:0/932740912 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e900c9db0 msgr2=0x7f0e9014b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:10.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.914+0000 7f0e95497700 1 --2- 172.21.15.163:0/932740912 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e900c9db0 0x7f0e9014b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:10.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.914+0000 7f0e95497700 1 -- 172.21.15.163:0/932740912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e90157130 msgr2=0x7f0e900b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:10.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.914+0000 7f0e95497700 1 --2- 172.21.15.163:0/932740912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e90157130 0x7f0e900b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:10.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.914+0000 7f0e95497700 1 -- 172.21.15.163:0/932740912 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e800094f0 con 0x7f0e90153900 2023-10-09T06:37:10.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.914+0000 7f0e95497700 1 --2- 172.21.15.163:0/932740912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e90153900 0x7f0e9014f470 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f0e9006ce80 tx=0x7f0e80017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:10.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.914+0000 7f0e85ffb700 1 -- 172.21.15.163:0/932740912 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8000b9d0 con 0x7f0e90153900 2023-10-09T06:37:10.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.914+0000 7f0e85ffb700 1 -- 172.21.15.163:0/932740912 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8000bb30 con 0x7f0e90153900 2023-10-09T06:37:10.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.914+0000 7f0e85ffb700 1 -- 172.21.15.163:0/932740912 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e80025670 con 0x7f0e90153900 2023-10-09T06:37:10.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.914+0000 7f0e96c9a700 1 -- 172.21.15.163:0/932740912 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e90169a80 con 0x7f0e90153900 2023-10-09T06:37:10.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.914+0000 7f0e96c9a700 1 -- 172.21.15.163:0/932740912 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e9014c9c0 con 0x7f0e90153900 2023-10-09T06:37:10.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.915+0000 7f0e96c9a700 1 -- 172.21.15.163:0/932740912 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e9016a240 con 0x7f0e90153900 2023-10-09T06:37:10.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.917+0000 7f0e85ffb700 1 -- 172.21.15.163:0/932740912 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0e80023070 con 0x7f0e90153900 2023-10-09T06:37:10.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.917+0000 7f0e85ffb700 1 --2- 172.21.15.163:0/932740912 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0e74067320 0x7f0e740697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:10.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.917+0000 7f0e85ffb700 1 -- 172.21.15.163:0/932740912 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0e800865a0 con 0x7f0e90153900 2023-10-09T06:37:10.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.917+0000 7f0e95c98700 1 --2- 172.21.15.163:0/932740912 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0e74067320 0x7f0e740697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:10.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.918+0000 7f0e95c98700 1 --2- 172.21.15.163:0/932740912 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0e74067320 0x7f0e740697d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c007fd0 tx=0x7f0e8c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:10.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:10.920+0000 7f0e85ffb700 1 -- 172.21.15.163:0/932740912 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e800532b0 con 0x7f0e90153900 2023-10-09T06:37:11.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.145+0000 7f0e96c9a700 1 -- 172.21.15.163:0/932740912 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f0e900c7dc0 con 0x7f0e90153900 2023-10-09T06:37:11.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.775+0000 7f0e85ffb700 1 -- 172.21.15.163:0/932740912 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v75) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0e800568d0 con 0x7f0e90153900 2023-10-09T06:37:11.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.778+0000 7f0e96c9a700 1 -- 172.21.15.163:0/932740912 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0e74067320 msgr2=0x7f0e740697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:11.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.778+0000 7f0e96c9a700 1 --2- 172.21.15.163:0/932740912 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0e74067320 0x7f0e740697d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c007fd0 tx=0x7f0e8c009040 comp rx=0 tx=0).stop 2023-10-09T06:37:11.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.778+0000 7f0e96c9a700 1 -- 172.21.15.163:0/932740912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e90153900 msgr2=0x7f0e9014f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:11.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.778+0000 7f0e96c9a700 1 --2- 172.21.15.163:0/932740912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e90153900 0x7f0e9014f470 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f0e9006ce80 tx=0x7f0e80017000 comp rx=0 tx=0).stop 2023-10-09T06:37:11.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.778+0000 7f0e96c9a700 1 -- 172.21.15.163:0/932740912 shutdown_connections 2023-10-09T06:37:11.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.778+0000 7f0e96c9a700 1 --2- 172.21.15.163:0/932740912 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0e74067320 0x7f0e740697d0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:11.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.778+0000 7f0e96c9a700 1 --2- 172.21.15.163:0/932740912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e90157130 0x7f0e900b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:11.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.778+0000 7f0e96c9a700 1 --2- 172.21.15.163:0/932740912 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e900c9db0 0x7f0e9014b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:11.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.778+0000 7f0e96c9a700 1 --2- 172.21.15.163:0/932740912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e90153900 0x7f0e9014f470 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:11.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.778+0000 7f0e96c9a700 1 -- 172.21.15.163:0/932740912 >> 172.21.15.163:0/932740912 conn(0x7f0e900cc1e0 msgr2=0x7f0e900c0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:11.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.778+0000 7f0e96c9a700 1 -- 172.21.15.163:0/932740912 shutdown_connections 2023-10-09T06:37:11.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.778+0000 7f0e96c9a700 1 -- 172.21.15.163:0/932740912 wait complete. 2023-10-09T06:37:11.793 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:37:11.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.911+0000 7f632346e700 1 Processor -- start 2023-10-09T06:37:11.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.911+0000 7f632346e700 1 -- start start 2023-10-09T06:37:11.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.912+0000 7f632346e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f631c159890 0x7f631c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:11.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.912+0000 7f632346e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f631c069560 0x7f631c0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:11.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.912+0000 7f632346e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f631c15a160 con 0x7f631c159c60 2023-10-09T06:37:11.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.912+0000 7f632346e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f631c069f20 con 0x7f631c069560 2023-10-09T06:37:11.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.912+0000 7f632346e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f631c0d13a0 con 0x7f631c159890 2023-10-09T06:37:11.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.912+0000 7f6321a0b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f631c069560 0x7f631c0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:11.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.912+0000 7f6321a0b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f631c069560 0x7f631c0699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56754/0 (socket says 172.21.15.163:56754) 2023-10-09T06:37:11.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.912+0000 7f6321a0b700 1 -- 172.21.15.163:0/3209542123 learned_addr learned my addr 172.21.15.163:0/3209542123 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:11.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.912+0000 7f632120a700 1 --2- 172.21.15.163:0/3209542123 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f631c159890 0x7f631c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:11.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.912+0000 7f6321a0b700 1 -- 172.21.15.163:0/3209542123 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f631c159890 msgr2=0x7f631c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:11.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.912+0000 7f6321a0b700 1 --2- 172.21.15.163:0/3209542123 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f631c159890 0x7f631c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:11.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.912+0000 7f6321a0b700 1 -- 172.21.15.163:0/3209542123 >> v1:172.21.15.181:6789/0 conn(0x7f631c159c60 legacy=0x7f631c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:11.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.912+0000 7f6321a0b700 1 -- 172.21.15.163:0/3209542123 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631c15e140 con 0x7f631c069560 2023-10-09T06:37:11.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.912+0000 7f632120a700 1 --2- 172.21.15.163:0/3209542123 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f631c159890 0x7f631c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:11.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.913+0000 7f6321a0b700 1 --2- 172.21.15.163:0/3209542123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f631c069560 0x7f631c0699e0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f63140099b0 tx=0x7f6314009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff3401e76205db54 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:11.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.913+0000 7f6313fff700 1 -- 172.21.15.163:0/3209542123 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631400b860 con 0x7f631c069560 2023-10-09T06:37:11.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.913+0000 7f6313fff700 1 -- 172.21.15.163:0/3209542123 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631400b9c0 con 0x7f631c069560 2023-10-09T06:37:11.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.913+0000 7f6313fff700 1 -- 172.21.15.163:0/3209542123 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631400bd60 con 0x7f631c069560 2023-10-09T06:37:11.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.913+0000 7f632346e700 1 -- 172.21.15.163:0/3209542123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f631c069560 msgr2=0x7f631c0699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:11.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.913+0000 7f632346e700 1 --2- 172.21.15.163:0/3209542123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f631c069560 0x7f631c0699e0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f63140099b0 tx=0x7f6314009250 comp rx=0 tx=0).stop 2023-10-09T06:37:11.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.913+0000 7f632346e700 1 -- 172.21.15.163:0/3209542123 shutdown_connections 2023-10-09T06:37:11.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.913+0000 7f632346e700 1 --2- 172.21.15.163:0/3209542123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f631c069560 0x7f631c0699e0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:11.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.913+0000 7f632346e700 1 --2- 172.21.15.163:0/3209542123 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f631c159890 0x7f631c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:11.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.913+0000 7f632346e700 1 -- 172.21.15.163:0/3209542123 >> 172.21.15.163:0/3209542123 conn(0x7f631c068e20 msgr2=0x7f631c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:11.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.913+0000 7f632346e700 1 -- 172.21.15.163:0/3209542123 shutdown_connections 2023-10-09T06:37:11.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.913+0000 7f632346e700 1 -- 172.21.15.163:0/3209542123 wait complete. 2023-10-09T06:37:11.919 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.914+0000 7f632346e700 1 Processor -- start 2023-10-09T06:37:11.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.915+0000 7f632346e700 1 -- start start 2023-10-09T06:37:11.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.915+0000 7f632346e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f631c069560 0x7f631c0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:11.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.915+0000 7f632346e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f631c159890 0x7f631c1493d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:11.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.915+0000 7f632346e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f631c159c60 0x7f631c0b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:11.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.915+0000 7f632346e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631c16aaf0 con 0x7f631c159c60 2023-10-09T06:37:11.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.915+0000 7f632346e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631c169c30 con 0x7f631c159890 2023-10-09T06:37:11.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.915+0000 7f632346e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f631c169ee0 con 0x7f631c069560 2023-10-09T06:37:11.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.915+0000 7f6321a0b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f631c159c60 0x7f631c0b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:11.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.915+0000 7f6321a0b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f631c159c60 0x7f631c0b15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56766/0 (socket says 172.21.15.163:56766) 2023-10-09T06:37:11.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.916+0000 7f6321a0b700 1 -- 172.21.15.163:0/3562049831 learned_addr learned my addr 172.21.15.163:0/3562049831 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:11.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.916+0000 7f632120a700 1 --2- 172.21.15.163:0/3562049831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f631c069560 0x7f631c0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:11.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.916+0000 7f6320a09700 1 --2- 172.21.15.163:0/3562049831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f631c159890 0x7f631c1493d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:11.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.916+0000 7f6321a0b700 1 -- 172.21.15.163:0/3562049831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f631c069560 msgr2=0x7f631c0b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:11.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.916+0000 7f6321a0b700 1 --2- 172.21.15.163:0/3562049831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f631c069560 0x7f631c0b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:11.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.916+0000 7f6321a0b700 1 -- 172.21.15.163:0/3562049831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f631c159890 msgr2=0x7f631c1493d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:11.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.916+0000 7f6321a0b700 1 --2- 172.21.15.163:0/3562049831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f631c159890 0x7f631c1493d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:11.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.916+0000 7f6321a0b700 1 -- 172.21.15.163:0/3562049831 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63140094f0 con 0x7f631c159c60 2023-10-09T06:37:11.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.916+0000 7f6321a0b700 1 --2- 172.21.15.163:0/3562049831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f631c159c60 0x7f631c0b15a0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f63140090b0 tx=0x7f6314017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:11.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.916+0000 7f63117fa700 1 -- 172.21.15.163:0/3562049831 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631401a640 con 0x7f631c159c60 2023-10-09T06:37:11.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.916+0000 7f63117fa700 1 -- 172.21.15.163:0/3562049831 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631401f070 con 0x7f631c159c60 2023-10-09T06:37:11.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.916+0000 7f63117fa700 1 -- 172.21.15.163:0/3562049831 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631401aa10 con 0x7f631c159c60 2023-10-09T06:37:11.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.916+0000 7f632346e700 1 -- 172.21.15.163:0/3562049831 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f631c16a1c0 con 0x7f631c159c60 2023-10-09T06:37:11.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.916+0000 7f632346e700 1 -- 172.21.15.163:0/3562049831 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f631c16a780 con 0x7f631c159c60 2023-10-09T06:37:11.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.917+0000 7f632346e700 1 -- 172.21.15.163:0/3562049831 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f631c16d270 con 0x7f631c159c60 2023-10-09T06:37:11.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.917+0000 7f63117fa700 1 -- 172.21.15.163:0/3562049831 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f631401b430 con 0x7f631c159c60 2023-10-09T06:37:11.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.918+0000 7f63117fa700 1 --2- 172.21.15.163:0/3562049831 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6300067320 0x7f63000697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:11.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.918+0000 7f63117fa700 1 -- 172.21.15.163:0/3562049831 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6314021070 con 0x7f631c159c60 2023-10-09T06:37:11.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.918+0000 7f632120a700 1 --2- 172.21.15.163:0/3562049831 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6300067320 0x7f63000697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:11.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.919+0000 7f632120a700 1 --2- 172.21.15.163:0/3562049831 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6300067320 0x7f63000697d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f631c0b2160 tx=0x7f63180098f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:11.923 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:11.922+0000 7f63117fa700 1 -- 172.21.15.163:0/3562049831 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6314053960 con 0x7f631c159c60 2023-10-09T06:37:12.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.147+0000 7f632346e700 1 -- 172.21.15.163:0/3562049831 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f631c04c3c0 con 0x7f631c159c60 2023-10-09T06:37:12.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.147+0000 7f63117fa700 1 -- 172.21.15.163:0/3562049831 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4608 (secure 0 0 0) 0x7f631402b020 con 0x7f631c159c60 2023-10-09T06:37:12.148 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:12.148 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":75,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2821591245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2821591245},{"type":"v1","addr":"172.21.15.181:6836","nonce":2821591245}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":70},{"gid":6592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3075056851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3075056851},{"type":"v1","addr":"172.21.15.181:6837","nonce":3075056851}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":70},{"gid":7158,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/1355789800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1355789800},{"type":"v1","addr":"172.21.15.163:6834","nonce":1355789800}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:37:08.129202+0000","modified":"2023-10-09T06:37:11.774637+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7161},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7161":{"gid":7161,"name":"c","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/571471083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":571471083},{"type":"v1","addr":"172.21.15.163:6835","nonce":571471083}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-09T06:37:12.153 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.149+0000 7f632346e700 1 -- 172.21.15.163:0/3562049831 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6300067320 msgr2=0x7f63000697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:12.153 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.149+0000 7f632346e700 1 --2- 172.21.15.163:0/3562049831 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6300067320 0x7f63000697d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f631c0b2160 tx=0x7f63180098f0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.153 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.149+0000 7f632346e700 1 -- 172.21.15.163:0/3562049831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f631c159c60 msgr2=0x7f631c0b15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:12.153 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.149+0000 7f632346e700 1 --2- 172.21.15.163:0/3562049831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f631c159c60 0x7f631c0b15a0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f63140090b0 tx=0x7f6314017000 comp rx=0 tx=0).stop 2023-10-09T06:37:12.153 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.149+0000 7f632346e700 1 -- 172.21.15.163:0/3562049831 shutdown_connections 2023-10-09T06:37:12.153 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.149+0000 7f632346e700 1 --2- 172.21.15.163:0/3562049831 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6300067320 0x7f63000697d0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.153 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.149+0000 7f632346e700 1 --2- 172.21.15.163:0/3562049831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f631c159c60 0x7f631c0b15a0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.153 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.149+0000 7f632346e700 1 --2- 172.21.15.163:0/3562049831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f631c069560 0x7f631c0b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.154 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.149+0000 7f632346e700 1 --2- 172.21.15.163:0/3562049831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f631c159890 0x7f631c1493d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.154 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.149+0000 7f632346e700 1 -- 172.21.15.163:0/3562049831 >> 172.21.15.163:0/3562049831 conn(0x7f631c068e20 msgr2=0x7f631c155280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:12.154 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.149+0000 7f632346e700 1 -- 172.21.15.163:0/3562049831 shutdown_connections 2023-10-09T06:37:12.154 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.149+0000 7f632346e700 1 -- 172.21.15.163:0/3562049831 wait complete. 2023-10-09T06:37:12.154 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 75 2023-10-09T06:37:12.163 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:37:12.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.277+0000 7f70dbeef700 1 Processor -- start 2023-10-09T06:37:12.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.278+0000 7f70dbeef700 1 -- start start 2023-10-09T06:37:12.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.278+0000 7f70dbeef700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f70d4069d60 0x7f70d40c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:12.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.278+0000 7f70dbeef700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f70d40c81f0 0x7f70d40cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:12.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.278+0000 7f70dbeef700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70d406bdb0 con 0x7f70d40aca80 2023-10-09T06:37:12.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.278+0000 7f70dbeef700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d40cb370 con 0x7f70d40c81f0 2023-10-09T06:37:12.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.278+0000 7f70dbeef700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d4159f60 con 0x7f70d4069d60 2023-10-09T06:37:12.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.278+0000 7f70da48c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f70d40c81f0 0x7f70d40cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:12.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.279+0000 7f70da48c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f70d40c81f0 0x7f70d40cb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56768/0 (socket says 172.21.15.163:56768) 2023-10-09T06:37:12.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.279+0000 7f70da48c700 1 -- 172.21.15.163:0/2406846622 learned_addr learned my addr 172.21.15.163:0/2406846622 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:12.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.279+0000 7f70d948a700 1 --2- 172.21.15.163:0/2406846622 >> v2:172.21.15.181:3300/0 conn(0x7f70d4069d60 0x7f70d40c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:12.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.279+0000 7f70da48c700 1 -- 172.21.15.163:0/2406846622 >> v2:172.21.15.181:3300/0 conn(0x7f70d4069d60 msgr2=0x7f70d40c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:12.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.279+0000 7f70da48c700 1 --2- 172.21.15.163:0/2406846622 >> v2:172.21.15.181:3300/0 conn(0x7f70d4069d60 0x7f70d40c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.279+0000 7f70da48c700 1 -- 172.21.15.163:0/2406846622 >> v1:172.21.15.181:6789/0 conn(0x7f70d40aca80 legacy=0x7f70d406aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:12.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.279+0000 7f70da48c700 1 -- 172.21.15.163:0/2406846622 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d406b2b0 con 0x7f70d40c81f0 2023-10-09T06:37:12.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.279+0000 7f70d948a700 1 --2- 172.21.15.163:0/2406846622 >> v2:172.21.15.181:3300/0 conn(0x7f70d4069d60 0x7f70d40c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:37:12.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.279+0000 7f70da48c700 1 --2- 172.21.15.163:0/2406846622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70d40c81f0 0x7f70d40cb800 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f70d406e540 tx=0x7f70cc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8245e8c86e55f2fb server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:12.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.279+0000 7f70d8c89700 1 -- 172.21.15.163:0/2406846622 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc00e070 con 0x7f70d40c81f0 2023-10-09T06:37:12.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.279+0000 7f70d8c89700 1 -- 172.21.15.163:0/2406846622 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70cc00ba10 con 0x7f70d40c81f0 2023-10-09T06:37:12.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.279+0000 7f70d8c89700 1 -- 172.21.15.163:0/2406846622 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc0092e0 con 0x7f70d40c81f0 2023-10-09T06:37:12.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.280+0000 7f70dbeef700 1 -- 172.21.15.163:0/2406846622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70d40c81f0 msgr2=0x7f70d40cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:12.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.280+0000 7f70dbeef700 1 --2- 172.21.15.163:0/2406846622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70d40c81f0 0x7f70d40cb800 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f70d406e540 tx=0x7f70cc009840 comp rx=0 tx=0).stop 2023-10-09T06:37:12.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.280+0000 7f70dbeef700 1 -- 172.21.15.163:0/2406846622 shutdown_connections 2023-10-09T06:37:12.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.280+0000 7f70dbeef700 1 --2- 172.21.15.163:0/2406846622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70d40c81f0 0x7f70d40cb800 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.280+0000 7f70dbeef700 1 --2- 172.21.15.163:0/2406846622 >> v2:172.21.15.181:3300/0 conn(0x7f70d4069d60 0x7f70d40c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.280+0000 7f70dbeef700 1 -- 172.21.15.163:0/2406846622 >> 172.21.15.163:0/2406846622 conn(0x7f70d40cd670 msgr2=0x7f70d40cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:12.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.280+0000 7f70dbeef700 1 -- 172.21.15.163:0/2406846622 shutdown_connections 2023-10-09T06:37:12.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.280+0000 7f70dbeef700 1 -- 172.21.15.163:0/2406846622 wait complete. 2023-10-09T06:37:12.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.281+0000 7f70dbeef700 1 Processor -- start 2023-10-09T06:37:12.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.281+0000 7f70dbeef700 1 -- start start 2023-10-09T06:37:12.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.281+0000 7f70dbeef700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70d4069d60 0x7f70d414f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:12.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.281+0000 7f70dbeef700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70d40aca80 0x7f70d40c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:12.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.281+0000 7f70dbeef700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70d40c81f0 0x7f70d40b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:12.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.281+0000 7f70dbeef700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d414fb40 con 0x7f70d4069d60 2023-10-09T06:37:12.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.281+0000 7f70dbeef700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d40af930 con 0x7f70d40c81f0 2023-10-09T06:37:12.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.282+0000 7f70dbeef700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d40b1590 con 0x7f70d40aca80 2023-10-09T06:37:12.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.282+0000 7f70d9c8b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70d4069d60 0x7f70d414f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:12.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.282+0000 7f70d9c8b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70d4069d60 0x7f70d414f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56784/0 (socket says 172.21.15.163:56784) 2023-10-09T06:37:12.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.282+0000 7f70d9c8b700 1 -- 172.21.15.163:0/1625012506 learned_addr learned my addr 172.21.15.163:0/1625012506 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:12.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.282+0000 7f70d948a700 1 --2- 172.21.15.163:0/1625012506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70d40aca80 0x7f70d40c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:12.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.282+0000 7f70da48c700 1 --2- 172.21.15.163:0/1625012506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70d40c81f0 0x7f70d40b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:12.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.282+0000 7f70d9c8b700 1 -- 172.21.15.163:0/1625012506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70d40aca80 msgr2=0x7f70d40c52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:12.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.282+0000 7f70d9c8b700 1 --2- 172.21.15.163:0/1625012506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70d40aca80 0x7f70d40c52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.282+0000 7f70d9c8b700 1 -- 172.21.15.163:0/1625012506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70d40c81f0 msgr2=0x7f70d40b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:12.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.282+0000 7f70d9c8b700 1 --2- 172.21.15.163:0/1625012506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70d40c81f0 0x7f70d40b25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.282+0000 7f70d9c8b700 1 -- 172.21.15.163:0/1625012506 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70cc009ae0 con 0x7f70d4069d60 2023-10-09T06:37:12.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.282+0000 7f70d948a700 1 --2- 172.21.15.163:0/1625012506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70d40aca80 0x7f70d40c52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:12.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.282+0000 7f70d9c8b700 1 --2- 172.21.15.163:0/1625012506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70d4069d60 0x7f70d414f600 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f70d000c5e0 tx=0x7f70d00020e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:12.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.282+0000 7f70c9ffb700 1 -- 172.21.15.163:0/1625012506 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d0011590 con 0x7f70d4069d60 2023-10-09T06:37:12.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.282+0000 7f70c9ffb700 1 -- 172.21.15.163:0/1625012506 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d00116f0 con 0x7f70d4069d60 2023-10-09T06:37:12.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.282+0000 7f70c9ffb700 1 -- 172.21.15.163:0/1625012506 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d0011a90 con 0x7f70d4069d60 2023-10-09T06:37:12.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.282+0000 7f70dbeef700 1 -- 172.21.15.163:0/1625012506 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d40b17e0 con 0x7f70d4069d60 2023-10-09T06:37:12.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.282+0000 7f70dbeef700 1 -- 172.21.15.163:0/1625012506 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d40b1da0 con 0x7f70d4069d60 2023-10-09T06:37:12.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.283+0000 7f70dbeef700 1 -- 172.21.15.163:0/1625012506 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d404c3c0 con 0x7f70d4069d60 2023-10-09T06:37:12.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.284+0000 7f70c9ffb700 1 -- 172.21.15.163:0/1625012506 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f70d0012420 con 0x7f70d4069d60 2023-10-09T06:37:12.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.284+0000 7f70c9ffb700 1 --2- 172.21.15.163:0/1625012506 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f70b8067320 0x7f70b80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:12.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.284+0000 7f70c9ffb700 1 -- 172.21.15.163:0/1625012506 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f70d00845e0 con 0x7f70d4069d60 2023-10-09T06:37:12.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.284+0000 7f70d948a700 1 --2- 172.21.15.163:0/1625012506 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f70b8067320 0x7f70b80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:12.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.285+0000 7f70d948a700 1 --2- 172.21.15.163:0/1625012506 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f70b8067320 0x7f70b80697d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f70d40aff30 tx=0x7f70c4006f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:12.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.288+0000 7f70c9ffb700 1 -- 172.21.15.163:0/1625012506 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70d0051370 con 0x7f70d4069d60 2023-10-09T06:37:12.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.473+0000 7f70dbeef700 1 -- 172.21.15.163:0/1625012506 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70d40678d0 con 0x7f70d4069d60 2023-10-09T06:37:12.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.474+0000 7f70c9ffb700 1 -- 172.21.15.163:0/1625012506 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+15412 (secure 0 0 0) 0x7f70d0054990 con 0x7f70d4069d60 2023-10-09T06:37:12.475 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:12.475 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":113,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:37:09.531737+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:37:06.472811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-09T06:37:07.096712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:37:12.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.476+0000 7f70dbeef700 1 -- 172.21.15.163:0/1625012506 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f70b8067320 msgr2=0x7f70b80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:12.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.476+0000 7f70dbeef700 1 --2- 172.21.15.163:0/1625012506 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f70b8067320 0x7f70b80697d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f70d40aff30 tx=0x7f70c4006f90 comp rx=0 tx=0).stop 2023-10-09T06:37:12.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.476+0000 7f70dbeef700 1 -- 172.21.15.163:0/1625012506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70d4069d60 msgr2=0x7f70d414f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:12.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.476+0000 7f70dbeef700 1 --2- 172.21.15.163:0/1625012506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70d4069d60 0x7f70d414f600 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f70d000c5e0 tx=0x7f70d00020e0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.476+0000 7f70dbeef700 1 -- 172.21.15.163:0/1625012506 shutdown_connections 2023-10-09T06:37:12.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.476+0000 7f70dbeef700 1 --2- 172.21.15.163:0/1625012506 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f70b8067320 0x7f70b80697d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.476+0000 7f70dbeef700 1 --2- 172.21.15.163:0/1625012506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70d4069d60 0x7f70d414f600 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.476+0000 7f70dbeef700 1 --2- 172.21.15.163:0/1625012506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70d40c81f0 0x7f70d40b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.476+0000 7f70dbeef700 1 --2- 172.21.15.163:0/1625012506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70d40aca80 0x7f70d40c52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.476+0000 7f70dbeef700 1 -- 172.21.15.163:0/1625012506 >> 172.21.15.163:0/1625012506 conn(0x7f70d40cd670 msgr2=0x7f70d40c3550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:12.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.476+0000 7f70dbeef700 1 -- 172.21.15.163:0/1625012506 shutdown_connections 2023-10-09T06:37:12.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.476+0000 7f70dbeef700 1 -- 172.21.15.163:0/1625012506 wait complete. 2023-10-09T06:37:12.489 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-09T06:37:12.489 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:37:12.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.607+0000 7f6434530700 1 Processor -- start 2023-10-09T06:37:12.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.607+0000 7f6434530700 1 -- start start 2023-10-09T06:37:12.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.608+0000 7f6434530700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f642c159890 0x7f642c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:12.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.608+0000 7f6434530700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f642c069560 0x7f642c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:12.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.608+0000 7f6434530700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f642c069ee0 con 0x7f642c159c60 2023-10-09T06:37:12.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.608+0000 7f6434530700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f642c15a160 con 0x7f642c159890 2023-10-09T06:37:12.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.608+0000 7f6434530700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f642c0d13a0 con 0x7f642c069560 2023-10-09T06:37:12.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.608+0000 7f64322cc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f642c159890 0x7f642c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:12.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.608+0000 7f64322cc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f642c159890 0x7f642c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56798/0 (socket says 172.21.15.163:56798) 2023-10-09T06:37:12.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.608+0000 7f64322cc700 1 -- 172.21.15.163:0/1208503614 learned_addr learned my addr 172.21.15.163:0/1208503614 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:12.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.608+0000 7f6432acd700 1 --2- 172.21.15.163:0/1208503614 >> v2:172.21.15.181:3300/0 conn(0x7f642c069560 0x7f642c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:12.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.609+0000 7f64322cc700 1 -- 172.21.15.163:0/1208503614 >> v2:172.21.15.181:3300/0 conn(0x7f642c069560 msgr2=0x7f642c0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:12.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.609+0000 7f64322cc700 1 --2- 172.21.15.163:0/1208503614 >> v2:172.21.15.181:3300/0 conn(0x7f642c069560 0x7f642c0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.609+0000 7f64322cc700 1 -- 172.21.15.163:0/1208503614 >> v1:172.21.15.181:6789/0 conn(0x7f642c159c60 legacy=0x7f642c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:12.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.609+0000 7f64322cc700 1 -- 172.21.15.163:0/1208503614 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f642c15e140 con 0x7f642c159890 2023-10-09T06:37:12.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.609+0000 7f64322cc700 1 --2- 172.21.15.163:0/1208503614 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f642c159890 0x7f642c15abd0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f642c06a890 tx=0x7f6428009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c7135bb876d197 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:12.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.609+0000 7f64312ca700 1 -- 172.21.15.163:0/1208503614 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642800e070 con 0x7f642c159890 2023-10-09T06:37:12.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.609+0000 7f64312ca700 1 -- 172.21.15.163:0/1208503614 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642800ba20 con 0x7f642c159890 2023-10-09T06:37:12.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.609+0000 7f64312ca700 1 -- 172.21.15.163:0/1208503614 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64280092e0 con 0x7f642c159890 2023-10-09T06:37:12.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.609+0000 7f6434530700 1 -- 172.21.15.163:0/1208503614 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f642c159890 msgr2=0x7f642c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:12.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.609+0000 7f6434530700 1 --2- 172.21.15.163:0/1208503614 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f642c159890 0x7f642c15abd0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f642c06a890 tx=0x7f6428009840 comp rx=0 tx=0).stop 2023-10-09T06:37:12.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.609+0000 7f6434530700 1 -- 172.21.15.163:0/1208503614 shutdown_connections 2023-10-09T06:37:12.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.609+0000 7f6434530700 1 --2- 172.21.15.163:0/1208503614 >> v2:172.21.15.181:3300/0 conn(0x7f642c069560 0x7f642c0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.609+0000 7f6434530700 1 --2- 172.21.15.163:0/1208503614 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f642c159890 0x7f642c15abd0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.610+0000 7f6434530700 1 -- 172.21.15.163:0/1208503614 >> 172.21.15.163:0/1208503614 conn(0x7f642c068e20 msgr2=0x7f642c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:12.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.610+0000 7f6434530700 1 -- 172.21.15.163:0/1208503614 shutdown_connections 2023-10-09T06:37:12.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.610+0000 7f6434530700 1 -- 172.21.15.163:0/1208503614 wait complete. 2023-10-09T06:37:12.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.611+0000 7f6434530700 1 Processor -- start 2023-10-09T06:37:12.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.611+0000 7f6434530700 1 -- start start 2023-10-09T06:37:12.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.611+0000 7f6434530700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f642c069560 0x7f642c0b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:12.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.611+0000 7f6434530700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f642c159890 0x7f642c16d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:12.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.611+0000 7f6434530700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f642c159c60 0x7f642c16d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:12.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.611+0000 7f6434530700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642c0b4df0 con 0x7f642c159890 2023-10-09T06:37:12.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.611+0000 7f6434530700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642c0b44c0 con 0x7f642c069560 2023-10-09T06:37:12.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.611+0000 7f6434530700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f642c0b47a0 con 0x7f642c159c60 2023-10-09T06:37:12.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.611+0000 7f6431acb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f642c159890 0x7f642c16d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:12.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.611+0000 7f6431acb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f642c159890 0x7f642c16d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56800/0 (socket says 172.21.15.163:56800) 2023-10-09T06:37:12.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.611+0000 7f6431acb700 1 -- 172.21.15.163:0/2355145615 learned_addr learned my addr 172.21.15.163:0/2355145615 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:12.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.611+0000 7f64322cc700 1 --2- 172.21.15.163:0/2355145615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f642c069560 0x7f642c0b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:12.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.612+0000 7f6432acd700 1 --2- 172.21.15.163:0/2355145615 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f642c159c60 0x7f642c16d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:12.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.612+0000 7f6431acb700 1 -- 172.21.15.163:0/2355145615 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f642c159c60 msgr2=0x7f642c16d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:12.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.612+0000 7f6431acb700 1 --2- 172.21.15.163:0/2355145615 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f642c159c60 0x7f642c16d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.612+0000 7f6431acb700 1 -- 172.21.15.163:0/2355145615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f642c069560 msgr2=0x7f642c0b5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:12.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.612+0000 7f6431acb700 1 --2- 172.21.15.163:0/2355145615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f642c069560 0x7f642c0b5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.612+0000 7f6431acb700 1 -- 172.21.15.163:0/2355145615 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6428009ae0 con 0x7f642c159890 2023-10-09T06:37:12.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.612+0000 7f6431acb700 1 --2- 172.21.15.163:0/2355145615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f642c159890 0x7f642c16d350 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f641c00c4e0 tx=0x7f641c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:12.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.612+0000 7f64227fc700 1 -- 172.21.15.163:0/2355145615 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c011590 con 0x7f642c159890 2023-10-09T06:37:12.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.612+0000 7f64227fc700 1 -- 172.21.15.163:0/2355145615 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f641c011710 con 0x7f642c159890 2023-10-09T06:37:12.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.612+0000 7f6434530700 1 -- 172.21.15.163:0/2355145615 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f642c0b4a80 con 0x7f642c159890 2023-10-09T06:37:12.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.612+0000 7f64227fc700 1 -- 172.21.15.163:0/2355145615 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c011ad0 con 0x7f642c159890 2023-10-09T06:37:12.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.612+0000 7f6434530700 1 -- 172.21.15.163:0/2355145615 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f642c143bc0 con 0x7f642c159890 2023-10-09T06:37:12.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.613+0000 7f6434530700 1 -- 172.21.15.163:0/2355145615 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f642c04c3c0 con 0x7f642c159890 2023-10-09T06:37:12.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.613+0000 7f64227fc700 1 -- 172.21.15.163:0/2355145615 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f641c01b3b0 con 0x7f642c159890 2023-10-09T06:37:12.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.614+0000 7f64227fc700 1 --2- 172.21.15.163:0/2355145615 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6410067320 0x7f64100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:12.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.614+0000 7f64227fc700 1 -- 172.21.15.163:0/2355145615 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f641c084000 con 0x7f642c159890 2023-10-09T06:37:12.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.614+0000 7f64322cc700 1 --2- 172.21.15.163:0/2355145615 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6410067320 0x7f64100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:12.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.614+0000 7f64322cc700 1 --2- 172.21.15.163:0/2355145615 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6410067320 0x7f64100697d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6428009880 tx=0x7f6428018000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:12.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.618+0000 7f64227fc700 1 -- 172.21.15.163:0/2355145615 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f641c050c90 con 0x7f642c159890 2023-10-09T06:37:12.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.799+0000 7f6434530700 1 -- 172.21.15.163:0/2355145615 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f642c0b4630 con 0x7f6410067320 2023-10-09T06:37:12.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.807+0000 7f64227fc700 1 -- 172.21.15.163:0/2355145615 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290191 (secure 0 0 0) 0x7f642c0b4630 con 0x7f6410067320 2023-10-09T06:37:12.811 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:12.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.811+0000 7f6434530700 1 -- 172.21.15.163:0/2355145615 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6410067320 msgr2=0x7f64100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:12.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.811+0000 7f6434530700 1 --2- 172.21.15.163:0/2355145615 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6410067320 0x7f64100697d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6428009880 tx=0x7f6428018000 comp rx=0 tx=0).stop 2023-10-09T06:37:12.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.811+0000 7f6434530700 1 -- 172.21.15.163:0/2355145615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f642c159890 msgr2=0x7f642c16d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:12.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.811+0000 7f6434530700 1 --2- 172.21.15.163:0/2355145615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f642c159890 0x7f642c16d350 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f641c00c4e0 tx=0x7f641c00c7f0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.811+0000 7f6434530700 1 -- 172.21.15.163:0/2355145615 shutdown_connections 2023-10-09T06:37:12.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.811+0000 7f6434530700 1 --2- 172.21.15.163:0/2355145615 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6410067320 0x7f64100697d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.811+0000 7f6434530700 1 --2- 172.21.15.163:0/2355145615 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f642c159c60 0x7f642c16d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.811+0000 7f6434530700 1 --2- 172.21.15.163:0/2355145615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f642c069560 0x7f642c0b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.811+0000 7f6434530700 1 --2- 172.21.15.163:0/2355145615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f642c159890 0x7f642c16d350 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.811+0000 7f6434530700 1 -- 172.21.15.163:0/2355145615 >> 172.21.15.163:0/2355145615 conn(0x7f642c068e20 msgr2=0x7f642c1534c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:12.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.811+0000 7f6434530700 1 -- 172.21.15.163:0/2355145615 shutdown_connections 2023-10-09T06:37:12.819 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.811+0000 7f6434530700 1 -- 172.21.15.163:0/2355145615 wait complete. 2023-10-09T06:37:12.819 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:37:12.824 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2023-10-09T06:37:12.394152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2419240,"kb_used_data":12264,"kb_used_omap":0,"kb_used_meta":2406912,"kb_avail":752555480,"statfs":{"total":773094113280,"available":770616811520,"internally_reserved":0,"allocated":12558336,"data_stored":6088636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2464677888},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.650831"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.560747+0000","last_change":"2023-10-09T06:37:08.553928+0000","last_active":"2023-10-09T06:37:08.560747+0000","last_peered":"2023-10-09T06:37:08.560747+0000","last_clean":"2023-10-09T06:37:08.560747+0000","last_became_active":"2023-10-09T06:37:08.553656+0000","last_became_peered":"2023-10-09T06:37:08.553656+0000","last_unstale":"2023-10-09T06:37:08.560747+0000","last_undegraded":"2023-10-09T06:37:08.560747+0000","last_fullsized":"2023-10-09T06:37:08.560747+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:57:31.509105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.556688+0000","last_change":"2023-10-09T06:37:08.553111+0000","last_active":"2023-10-09T06:37:08.556688+0000","last_peered":"2023-10-09T06:37:08.556688+0000","last_clean":"2023-10-09T06:37:08.556688+0000","last_became_active":"2023-10-09T06:37:08.552846+0000","last_became_peered":"2023-10-09T06:37:08.552846+0000","last_unstale":"2023-10-09T06:37:08.556688+0000","last_undegraded":"2023-10-09T06:37:08.556688+0000","last_fullsized":"2023-10-09T06:37:08.556688+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:49:02.464075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.560363+0000","last_change":"2023-10-09T06:37:08.555838+0000","last_active":"2023-10-09T06:37:08.560363+0000","last_peered":"2023-10-09T06:37:08.560363+0000","last_clean":"2023-10-09T06:37:08.560363+0000","last_became_active":"2023-10-09T06:37:08.555580+0000","last_became_peered":"2023-10-09T06:37:08.555580+0000","last_unstale":"2023-10-09T06:37:08.560363+0000","last_undegraded":"2023-10-09T06:37:08.560363+0000","last_fullsized":"2023-10-09T06:37:08.560363+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:43:46.874944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.552650+0000","last_change":"2023-10-09T06:37:08.548090+0000","last_active":"2023-10-09T06:37:08.552650+0000","last_peered":"2023-10-09T06:37:08.552650+0000","last_clean":"2023-10-09T06:37:08.552650+0000","last_became_active":"2023-10-09T06:37:08.547819+0000","last_became_peered":"2023-10-09T06:37:08.547819+0000","last_unstale":"2023-10-09T06:37:08.552650+0000","last_undegraded":"2023-10-09T06:37:08.552650+0000","last_fullsized":"2023-10-09T06:37:08.552650+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:24:57.108220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.551203+0000","last_change":"2023-10-09T06:37:08.547420+0000","last_active":"2023-10-09T06:37:08.551203+0000","last_peered":"2023-10-09T06:37:08.551203+0000","last_clean":"2023-10-09T06:37:08.551203+0000","last_became_active":"2023-10-09T06:37:08.547159+0000","last_became_peered":"2023-10-09T06:37:08.547159+0000","last_unstale":"2023-10-09T06:37:08.551203+0000","last_undegraded":"2023-10-09T06:37:08.551203+0000","last_fullsized":"2023-10-09T06:37:08.551203+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:16:58.160581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.554196+0000","last_change":"2023-10-09T06:37:08.548276+0000","last_active":"2023-10-09T06:37:08.554196+0000","last_peered":"2023-10-09T06:37:08.554196+0000","last_clean":"2023-10-09T06:37:08.554196+0000","last_became_active":"2023-10-09T06:37:08.548045+0000","last_became_peered":"2023-10-09T06:37:08.548045+0000","last_unstale":"2023-10-09T06:37:08.554196+0000","last_undegraded":"2023-10-09T06:37:08.554196+0000","last_fullsized":"2023-10-09T06:37:08.554196+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:26:57.580349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.551565+0000","last_change":"2023-10-09T06:37:08.540730+0000","last_active":"2023-10-09T06:37:08.551565+0000","last_peered":"2023-10-09T06:37:08.551565+0000","last_clean":"2023-10-09T06:37:08.551565+0000","last_became_active":"2023-10-09T06:37:08.540473+0000","last_became_peered":"2023-10-09T06:37:08.540473+0000","last_unstale":"2023-10-09T06:37:08.551565+0000","last_undegraded":"2023-10-09T06:37:08.551565+0000","last_fullsized":"2023-10-09T06:37:08.551565+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:23:20.895190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.544637+0000","last_change":"2023-10-09T06:37:08.541018+0000","last_active":"2023-10-09T06:37:08.544637+0000","last_peered":"2023-10-09T06:37:08.544637+0000","last_clean":"2023-10-09T06:37:08.544637+0000","last_became_active":"2023-10-09T06:37:08.540416+0000","last_became_peered":"2023-10-09T06:37:08.540416+0000","last_unstale":"2023-10-09T06:37:08.544637+0000","last_undegraded":"2023-10-09T06:37:08.544637+0000","last_fullsized":"2023-10-09T06:37:08.544637+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:23:41.031871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.557935+0000","last_change":"2023-10-09T06:37:08.553838+0000","last_active":"2023-10-09T06:37:08.557935+0000","last_peered":"2023-10-09T06:37:08.557935+0000","last_clean":"2023-10-09T06:37:08.557935+0000","last_became_active":"2023-10-09T06:37:08.553582+0000","last_became_peered":"2023-10-09T06:37:08.553582+0000","last_unstale":"2023-10-09T06:37:08.557935+0000","last_undegraded":"2023-10-09T06:37:08.557935+0000","last_fullsized":"2023-10-09T06:37:08.557935+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:46:37.103853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.556706+0000","last_change":"2023-10-09T06:37:08.553155+0000","last_active":"2023-10-09T06:37:08.556706+0000","last_peered":"2023-10-09T06:37:08.556706+0000","last_clean":"2023-10-09T06:37:08.556706+0000","last_became_active":"2023-10-09T06:37:08.552833+0000","last_became_peered":"2023-10-09T06:37:08.552833+0000","last_unstale":"2023-10-09T06:37:08.556706+0000","last_undegraded":"2023-10-09T06:37:08.556706+0000","last_fullsized":"2023-10-09T06:37:08.556706+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:26:57.580349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.558205+0000","last_change":"2023-10-09T06:37:08.554294+0000","last_active":"2023-10-09T06:37:08.558205+0000","last_peered":"2023-10-09T06:37:08.558205+0000","last_clean":"2023-10-09T06:37:08.558205+0000","last_became_active":"2023-10-09T06:37:08.554031+0000","last_became_peered":"2023-10-09T06:37:08.554031+0000","last_unstale":"2023-10-09T06:37:08.558205+0000","last_undegraded":"2023-10-09T06:37:08.558205+0000","last_fullsized":"2023-10-09T06:37:08.558205+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:57:35.952883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.549109+0000","last_change":"2023-10-09T06:37:08.547070+0000","last_active":"2023-10-09T06:37:08.549109+0000","last_peered":"2023-10-09T06:37:08.549109+0000","last_clean":"2023-10-09T06:37:08.549109+0000","last_became_active":"2023-10-09T06:37:08.546638+0000","last_became_peered":"2023-10-09T06:37:08.546638+0000","last_unstale":"2023-10-09T06:37:08.549109+0000","last_undegraded":"2023-10-09T06:37:08.549109+0000","last_fullsized":"2023-10-09T06:37:08.549109+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:22:30.623361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.560778+0000","last_change":"2023-10-09T06:37:08.554379+0000","last_active":"2023-10-09T06:37:08.560778+0000","last_peered":"2023-10-09T06:37:08.560778+0000","last_clean":"2023-10-09T06:37:08.560778+0000","last_became_active":"2023-10-09T06:37:08.553391+0000","last_became_peered":"2023-10-09T06:37:08.553391+0000","last_unstale":"2023-10-09T06:37:08.560778+0000","last_undegraded":"2023-10-09T06:37:08.560778+0000","last_fullsized":"2023-10-09T06:37:08.560778+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:46:47.898722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.556569+0000","last_change":"2023-10-09T06:37:08.549708+0000","last_active":"2023-10-09T06:37:08.556569+0000","last_peered":"2023-10-09T06:37:08.556569+0000","last_clean":"2023-10-09T06:37:08.556569+0000","last_became_active":"2023-10-09T06:37:08.549499+0000","last_became_peered":"2023-10-09T06:37:08.549499+0000","last_unstale":"2023-10-09T06:37:08.556569+0000","last_undegraded":"2023-10-09T06:37:08.556569+0000","last_fullsized":"2023-10-09T06:37:08.556569+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:04:40.440444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.553923+0000","last_change":"2023-10-09T06:37:08.549898+0000","last_active":"2023-10-09T06:37:08.553923+0000","last_peered":"2023-10-09T06:37:08.553923+0000","last_clean":"2023-10-09T06:37:08.553923+0000","last_became_active":"2023-10-09T06:37:08.549637+0000","last_became_peered":"2023-10-09T06:37:08.549637+0000","last_unstale":"2023-10-09T06:37:08.553923+0000","last_undegraded":"2023-10-09T06:37:08.553923+0000","last_fullsized":"2023-10-09T06:37:08.553923+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:17:01.899478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.550842+0000","last_change":"2023-10-09T06:37:08.546298+0000","last_active":"2023-10-09T06:37:08.550842+0000","last_peered":"2023-10-09T06:37:08.550842+0000","last_clean":"2023-10-09T06:37:08.550842+0000","last_became_active":"2023-10-09T06:37:08.546041+0000","last_became_peered":"2023-10-09T06:37:08.546041+0000","last_unstale":"2023-10-09T06:37:08.550842+0000","last_undegraded":"2023-10-09T06:37:08.550842+0000","last_fullsized":"2023-10-09T06:37:08.550842+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:18:24.085073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.553192+0000","last_change":"2023-10-09T06:37:08.550852+0000","last_active":"2023-10-09T06:37:08.553192+0000","last_peered":"2023-10-09T06:37:08.553192+0000","last_clean":"2023-10-09T06:37:08.553192+0000","last_became_active":"2023-10-09T06:37:08.550579+0000","last_became_peered":"2023-10-09T06:37:08.550579+0000","last_unstale":"2023-10-09T06:37:08.553192+0000","last_undegraded":"2023-10-09T06:37:08.553192+0000","last_fullsized":"2023-10-09T06:37:08.553192+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:53:19.931007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.554353+0000","last_change":"2023-10-09T06:37:08.550139+0000","last_active":"2023-10-09T06:37:08.554353+0000","last_peered":"2023-10-09T06:37:08.554353+0000","last_clean":"2023-10-09T06:37:08.554353+0000","last_became_active":"2023-10-09T06:37:08.549769+0000","last_became_peered":"2023-10-09T06:37:08.549769+0000","last_unstale":"2023-10-09T06:37:08.554353+0000","last_undegraded":"2023-10-09T06:37:08.554353+0000","last_fullsized":"2023-10-09T06:37:08.554353+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:38:27.210425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.558283+0000","last_change":"2023-10-09T06:37:08.555167+0000","last_active":"2023-10-09T06:37:08.558283+0000","last_peered":"2023-10-09T06:37:08.558283+0000","last_clean":"2023-10-09T06:37:08.558283+0000","last_became_active":"2023-10-09T06:37:08.554886+0000","last_became_peered":"2023-10-09T06:37:08.554886+0000","last_unstale":"2023-10-09T06:37:08.558283+0000","last_undegraded":"2023-10-09T06:37:08.558283+0000","last_fullsized":"2023-10-09T06:37:08.558283+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:45:41.231359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.551789+0000","last_change":"2023-10-09T06:37:08.547292+0000","last_active":"2023-10-09T06:37:08.551789+0000","last_peered":"2023-10-09T06:37:08.551789+0000","last_clean":"2023-10-09T06:37:08.551789+0000","last_became_active":"2023-10-09T06:37:08.546837+0000","last_became_peered":"2023-10-09T06:37:08.546837+0000","last_unstale":"2023-10-09T06:37:08.551789+0000","last_undegraded":"2023-10-09T06:37:08.551789+0000","last_fullsized":"2023-10-09T06:37:08.551789+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:41:55.856641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.551811+0000","last_change":"2023-10-09T06:37:08.550259+0000","last_active":"2023-10-09T06:37:08.551811+0000","last_peered":"2023-10-09T06:37:08.551811+0000","last_clean":"2023-10-09T06:37:08.551811+0000","last_became_active":"2023-10-09T06:37:08.549940+0000","last_became_peered":"2023-10-09T06:37:08.549940+0000","last_unstale":"2023-10-09T06:37:08.551811+0000","last_undegraded":"2023-10-09T06:37:08.551811+0000","last_fullsized":"2023-10-09T06:37:08.551811+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:18:24.085073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.556320+0000","last_change":"2023-10-09T06:37:08.550863+0000","last_active":"2023-10-09T06:37:08.556320+0000","last_peered":"2023-10-09T06:37:08.556320+0000","last_clean":"2023-10-09T06:37:08.556320+0000","last_became_active":"2023-10-09T06:37:08.550592+0000","last_became_peered":"2023-10-09T06:37:08.550592+0000","last_unstale":"2023-10-09T06:37:08.556320+0000","last_undegraded":"2023-10-09T06:37:08.556320+0000","last_fullsized":"2023-10-09T06:37:08.556320+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:00:25.759826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.550395+0000","last_change":"2023-10-09T06:37:08.547200+0000","last_active":"2023-10-09T06:37:08.550395+0000","last_peered":"2023-10-09T06:37:08.550395+0000","last_clean":"2023-10-09T06:37:08.550395+0000","last_became_active":"2023-10-09T06:37:08.546924+0000","last_became_peered":"2023-10-09T06:37:08.546924+0000","last_unstale":"2023-10-09T06:37:08.550395+0000","last_undegraded":"2023-10-09T06:37:08.550395+0000","last_fullsized":"2023-10-09T06:37:08.550395+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:25:32.403134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.556401+0000","last_change":"2023-10-09T06:37:08.549053+0000","last_active":"2023-10-09T06:37:08.556401+0000","last_peered":"2023-10-09T06:37:08.556401+0000","last_clean":"2023-10-09T06:37:08.556401+0000","last_became_active":"2023-10-09T06:37:08.548760+0000","last_became_peered":"2023-10-09T06:37:08.548760+0000","last_unstale":"2023-10-09T06:37:08.556401+0000","last_undegraded":"2023-10-09T06:37:08.556401+0000","last_fullsized":"2023-10-09T06:37:08.556401+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:36:21.796862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.548639+0000","last_change":"2023-10-09T06:37:08.545486+0000","last_active":"2023-10-09T06:37:08.548639+0000","last_peered":"2023-10-09T06:37:08.548639+0000","last_clean":"2023-10-09T06:37:08.548639+0000","last_became_active":"2023-10-09T06:37:08.545242+0000","last_became_peered":"2023-10-09T06:37:08.545242+0000","last_unstale":"2023-10-09T06:37:08.548639+0000","last_undegraded":"2023-10-09T06:37:08.548639+0000","last_fullsized":"2023-10-09T06:37:08.548639+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:06:05.712018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.552158+0000","last_change":"2023-10-09T06:37:08.547689+0000","last_active":"2023-10-09T06:37:08.552158+0000","last_peered":"2023-10-09T06:37:08.552158+0000","last_clean":"2023-10-09T06:37:08.552158+0000","last_became_active":"2023-10-09T06:37:08.547430+0000","last_became_peered":"2023-10-09T06:37:08.547430+0000","last_unstale":"2023-10-09T06:37:08.552158+0000","last_undegraded":"2023-10-09T06:37:08.552158+0000","last_fullsized":"2023-10-09T06:37:08.552158+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:17:34.019897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.560772+0000","last_change":"2023-10-09T06:37:08.554064+0000","last_active":"2023-10-09T06:37:08.560772+0000","last_peered":"2023-10-09T06:37:08.560772+0000","last_clean":"2023-10-09T06:37:08.560772+0000","last_became_active":"2023-10-09T06:37:08.553815+0000","last_became_peered":"2023-10-09T06:37:08.553815+0000","last_unstale":"2023-10-09T06:37:08.560772+0000","last_undegraded":"2023-10-09T06:37:08.560772+0000","last_fullsized":"2023-10-09T06:37:08.560772+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:03:04.328664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.552497+0000","last_change":"2023-10-09T06:37:08.551101+0000","last_active":"2023-10-09T06:37:08.552497+0000","last_peered":"2023-10-09T06:37:08.552497+0000","last_clean":"2023-10-09T06:37:08.552497+0000","last_became_active":"2023-10-09T06:37:08.550850+0000","last_became_peered":"2023-10-09T06:37:08.550850+0000","last_unstale":"2023-10-09T06:37:08.552497+0000","last_undegraded":"2023-10-09T06:37:08.552497+0000","last_fullsized":"2023-10-09T06:37:08.552497+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:22:30.623361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.554473+0000","last_change":"2023-10-09T06:37:08.550777+0000","last_active":"2023-10-09T06:37:08.554473+0000","last_peered":"2023-10-09T06:37:08.554473+0000","last_clean":"2023-10-09T06:37:08.554473+0000","last_became_active":"2023-10-09T06:37:08.550470+0000","last_became_peered":"2023-10-09T06:37:08.550470+0000","last_unstale":"2023-10-09T06:37:08.554473+0000","last_undegraded":"2023-10-09T06:37:08.554473+0000","last_fullsized":"2023-10-09T06:37:08.554473+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:21:42.229496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.556328+0000","last_change":"2023-10-09T06:37:08.550794+0000","last_active":"2023-10-09T06:37:08.556328+0000","last_peered":"2023-10-09T06:37:08.556328+0000","last_clean":"2023-10-09T06:37:08.556328+0000","last_became_active":"2023-10-09T06:37:08.550461+0000","last_became_peered":"2023-10-09T06:37:08.550461+0000","last_unstale":"2023-10-09T06:37:08.556328+0000","last_undegraded":"2023-10-09T06:37:08.556328+0000","last_fullsized":"2023-10-09T06:37:08.556328+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:18:02.972171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.556333+0000","last_change":"2023-10-09T06:37:08.553061+0000","last_active":"2023-10-09T06:37:08.556333+0000","last_peered":"2023-10-09T06:37:08.556333+0000","last_clean":"2023-10-09T06:37:08.556333+0000","last_became_active":"2023-10-09T06:37:08.552322+0000","last_became_peered":"2023-10-09T06:37:08.552322+0000","last_unstale":"2023-10-09T06:37:08.556333+0000","last_undegraded":"2023-10-09T06:37:08.556333+0000","last_fullsized":"2023-10-09T06:37:08.556333+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:41:55.856641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.552357+0000","last_change":"2023-10-09T06:37:08.543114+0000","last_active":"2023-10-09T06:37:08.552357+0000","last_peered":"2023-10-09T06:37:08.552357+0000","last_clean":"2023-10-09T06:37:08.552357+0000","last_became_active":"2023-10-09T06:37:08.542832+0000","last_became_peered":"2023-10-09T06:37:08.542832+0000","last_unstale":"2023-10-09T06:37:08.552357+0000","last_undegraded":"2023-10-09T06:37:08.552357+0000","last_fullsized":"2023-10-09T06:37:08.552357+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:56:38.197098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.554336+0000","last_change":"2023-10-09T06:37:08.550027+0000","last_active":"2023-10-09T06:37:08.554336+0000","last_peered":"2023-10-09T06:37:08.554336+0000","last_clean":"2023-10-09T06:37:08.554336+0000","last_became_active":"2023-10-09T06:37:08.549445+0000","last_became_peered":"2023-10-09T06:37:08.549445+0000","last_unstale":"2023-10-09T06:37:08.554336+0000","last_undegraded":"2023-10-09T06:37:08.554336+0000","last_fullsized":"2023-10-09T06:37:08.554336+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:16:58.160581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"112'4","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.773434+0000","last_change":"2023-10-09T06:37:07.765145+0000","last_active":"2023-10-09T06:37:08.773434+0000","last_peered":"2023-10-09T06:37:08.773434+0000","last_clean":"2023-10-09T06:37:08.773434+0000","last_became_active":"2023-10-09T06:37:07.764771+0000","last_became_peered":"2023-10-09T06:37:07.764771+0000","last_unstale":"2023-10-09T06:37:08.773434+0000","last_undegraded":"2023-10-09T06:37:08.773434+0000","last_fullsized":"2023-10-09T06:37:08.773434+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:41:54.505198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.557786+0000","last_change":"2023-10-09T06:37:08.550447+0000","last_active":"2023-10-09T06:37:08.557786+0000","last_peered":"2023-10-09T06:37:08.557786+0000","last_clean":"2023-10-09T06:37:08.557786+0000","last_became_active":"2023-10-09T06:37:08.550145+0000","last_became_peered":"2023-10-09T06:37:08.550145+0000","last_unstale":"2023-10-09T06:37:08.557786+0000","last_undegraded":"2023-10-09T06:37:08.557786+0000","last_fullsized":"2023-10-09T06:37:08.557786+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:47:45.882993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"112'1","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.771530+0000","last_change":"2023-10-09T06:37:07.758156+0000","last_active":"2023-10-09T06:37:08.771530+0000","last_peered":"2023-10-09T06:37:08.771530+0000","last_clean":"2023-10-09T06:37:08.771530+0000","last_became_active":"2023-10-09T06:37:07.757619+0000","last_became_peered":"2023-10-09T06:37:07.757619+0000","last_unstale":"2023-10-09T06:37:08.771530+0000","last_undegraded":"2023-10-09T06:37:08.771530+0000","last_fullsized":"2023-10-09T06:37:08.771530+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:47:47.047228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.556485+0000","last_change":"2023-10-09T06:37:08.554077+0000","last_active":"2023-10-09T06:37:08.556485+0000","last_peered":"2023-10-09T06:37:08.556485+0000","last_clean":"2023-10-09T06:37:08.556485+0000","last_became_active":"2023-10-09T06:37:08.553640+0000","last_became_peered":"2023-10-09T06:37:08.553640+0000","last_unstale":"2023-10-09T06:37:08.556485+0000","last_undegraded":"2023-10-09T06:37:08.556485+0000","last_fullsized":"2023-10-09T06:37:08.556485+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:43:59.013054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"112'2","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.770994+0000","last_change":"2023-10-09T06:37:07.762798+0000","last_active":"2023-10-09T06:37:08.770994+0000","last_peered":"2023-10-09T06:37:08.770994+0000","last_clean":"2023-10-09T06:37:08.770994+0000","last_became_active":"2023-10-09T06:37:07.762473+0000","last_became_peered":"2023-10-09T06:37:07.762473+0000","last_unstale":"2023-10-09T06:37:08.770994+0000","last_undegraded":"2023-10-09T06:37:08.770994+0000","last_fullsized":"2023-10-09T06:37:08.770994+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:04:06.305373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.551693+0000","last_change":"2023-10-09T06:37:08.548413+0000","last_active":"2023-10-09T06:37:08.551693+0000","last_peered":"2023-10-09T06:37:08.551693+0000","last_clean":"2023-10-09T06:37:08.551693+0000","last_became_active":"2023-10-09T06:37:08.548029+0000","last_became_peered":"2023-10-09T06:37:08.548029+0000","last_unstale":"2023-10-09T06:37:08.551693+0000","last_undegraded":"2023-10-09T06:37:08.551693+0000","last_fullsized":"2023-10-09T06:37:08.551693+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:17:34.019897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"112'2","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.770512+0000","last_change":"2023-10-09T06:37:07.760484+0000","last_active":"2023-10-09T06:37:08.770512+0000","last_peered":"2023-10-09T06:37:08.770512+0000","last_clean":"2023-10-09T06:37:08.770512+0000","last_became_active":"2023-10-09T06:37:07.760195+0000","last_became_peered":"2023-10-09T06:37:07.760195+0000","last_unstale":"2023-10-09T06:37:08.770512+0000","last_undegraded":"2023-10-09T06:37:08.770512+0000","last_fullsized":"2023-10-09T06:37:08.770512+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:25:31.399270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.554312+0000","last_change":"2023-10-09T06:37:08.549926+0000","last_active":"2023-10-09T06:37:08.554312+0000","last_peered":"2023-10-09T06:37:08.554312+0000","last_clean":"2023-10-09T06:37:08.554312+0000","last_became_active":"2023-10-09T06:37:08.549664+0000","last_became_peered":"2023-10-09T06:37:08.549664+0000","last_unstale":"2023-10-09T06:37:08.554312+0000","last_undegraded":"2023-10-09T06:37:08.554312+0000","last_fullsized":"2023-10-09T06:37:08.554312+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:07:45.944631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"112'1","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.772402+0000","last_change":"2023-10-09T06:37:07.764154+0000","last_active":"2023-10-09T06:37:08.772402+0000","last_peered":"2023-10-09T06:37:08.772402+0000","last_clean":"2023-10-09T06:37:08.772402+0000","last_became_active":"2023-10-09T06:37:07.763894+0000","last_became_peered":"2023-10-09T06:37:07.763894+0000","last_unstale":"2023-10-09T06:37:08.772402+0000","last_undegraded":"2023-10-09T06:37:08.772402+0000","last_fullsized":"2023-10-09T06:37:08.772402+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:22:29.619497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.557778+0000","last_change":"2023-10-09T06:37:08.555152+0000","last_active":"2023-10-09T06:37:08.557778+0000","last_peered":"2023-10-09T06:37:08.557778+0000","last_clean":"2023-10-09T06:37:08.557778+0000","last_became_active":"2023-10-09T06:37:08.554661+0000","last_became_peered":"2023-10-09T06:37:08.554661+0000","last_unstale":"2023-10-09T06:37:08.557778+0000","last_undegraded":"2023-10-09T06:37:08.557778+0000","last_fullsized":"2023-10-09T06:37:08.557778+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:04:40.440444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"112'1","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.770726+0000","last_change":"2023-10-09T06:37:07.762196+0000","last_active":"2023-10-09T06:37:08.770726+0000","last_peered":"2023-10-09T06:37:08.770726+0000","last_clean":"2023-10-09T06:37:08.770726+0000","last_became_active":"2023-10-09T06:37:07.761925+0000","last_became_peered":"2023-10-09T06:37:07.761925+0000","last_unstale":"2023-10-09T06:37:08.770726+0000","last_undegraded":"2023-10-09T06:37:08.770726+0000","last_fullsized":"2023-10-09T06:37:08.770726+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:10:44.973825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.545157+0000","last_change":"2023-10-09T06:37:08.541188+0000","last_active":"2023-10-09T06:37:08.545157+0000","last_peered":"2023-10-09T06:37:08.545157+0000","last_clean":"2023-10-09T06:37:08.545157+0000","last_became_active":"2023-10-09T06:37:08.540743+0000","last_became_peered":"2023-10-09T06:37:08.540743+0000","last_unstale":"2023-10-09T06:37:08.545157+0000","last_undegraded":"2023-10-09T06:37:08.545157+0000","last_fullsized":"2023-10-09T06:37:08.545157+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:57:10.867923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.537369+0000","last_change":"2023-10-09T06:37:07.760429+0000","last_active":"2023-10-09T06:37:08.537369+0000","last_peered":"2023-10-09T06:37:08.537369+0000","last_clean":"2023-10-09T06:37:08.537369+0000","last_became_active":"2023-10-09T06:37:07.760185+0000","last_became_peered":"2023-10-09T06:37:07.760185+0000","last_unstale":"2023-10-09T06:37:08.537369+0000","last_undegraded":"2023-10-09T06:37:08.537369+0000","last_fullsized":"2023-10-09T06:37:08.537369+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:22:29.619497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.556640+0000","last_change":"2023-10-09T06:37:08.551322+0000","last_active":"2023-10-09T06:37:08.556640+0000","last_peered":"2023-10-09T06:37:08.556640+0000","last_clean":"2023-10-09T06:37:08.556640+0000","last_became_active":"2023-10-09T06:37:08.551076+0000","last_became_peered":"2023-10-09T06:37:08.551076+0000","last_unstale":"2023-10-09T06:37:08.556640+0000","last_undegraded":"2023-10-09T06:37:08.556640+0000","last_fullsized":"2023-10-09T06:37:08.556640+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:20:06.325842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.542740+0000","last_change":"2023-10-09T06:37:07.765273+0000","last_active":"2023-10-09T06:37:08.542740+0000","last_peered":"2023-10-09T06:37:08.542740+0000","last_clean":"2023-10-09T06:37:08.542740+0000","last_became_active":"2023-10-09T06:37:07.765034+0000","last_became_peered":"2023-10-09T06:37:07.765034+0000","last_unstale":"2023-10-09T06:37:08.542740+0000","last_undegraded":"2023-10-09T06:37:08.542740+0000","last_fullsized":"2023-10-09T06:37:08.542740+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:10:05.886215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.560801+0000","last_change":"2023-10-09T06:37:08.554606+0000","last_active":"2023-10-09T06:37:08.560801+0000","last_peered":"2023-10-09T06:37:08.560801+0000","last_clean":"2023-10-09T06:37:08.560801+0000","last_became_active":"2023-10-09T06:37:08.554376+0000","last_became_peered":"2023-10-09T06:37:08.554376+0000","last_unstale":"2023-10-09T06:37:08.560801+0000","last_undegraded":"2023-10-09T06:37:08.560801+0000","last_fullsized":"2023-10-09T06:37:08.560801+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:36:41.110118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.547115+0000","last_change":"2023-10-09T06:37:07.764425+0000","last_active":"2023-10-09T06:37:08.547115+0000","last_peered":"2023-10-09T06:37:08.547115+0000","last_clean":"2023-10-09T06:37:08.547115+0000","last_became_active":"2023-10-09T06:37:07.764185+0000","last_became_peered":"2023-10-09T06:37:07.764185+0000","last_unstale":"2023-10-09T06:37:08.547115+0000","last_undegraded":"2023-10-09T06:37:08.547115+0000","last_fullsized":"2023-10-09T06:37:08.547115+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:21:28.762319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.552660+0000","last_change":"2023-10-09T06:37:08.548920+0000","last_active":"2023-10-09T06:37:08.552660+0000","last_peered":"2023-10-09T06:37:08.552660+0000","last_clean":"2023-10-09T06:37:08.552660+0000","last_became_active":"2023-10-09T06:37:08.548330+0000","last_became_peered":"2023-10-09T06:37:08.548330+0000","last_unstale":"2023-10-09T06:37:08.552660+0000","last_undegraded":"2023-10-09T06:37:08.552660+0000","last_fullsized":"2023-10-09T06:37:08.552660+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:06:50.653374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.538357+0000","last_change":"2023-10-09T06:37:07.759256+0000","last_active":"2023-10-09T06:37:08.538357+0000","last_peered":"2023-10-09T06:37:08.538357+0000","last_clean":"2023-10-09T06:37:08.538357+0000","last_became_active":"2023-10-09T06:37:07.758957+0000","last_became_peered":"2023-10-09T06:37:07.758957+0000","last_unstale":"2023-10-09T06:37:08.538357+0000","last_undegraded":"2023-10-09T06:37:08.538357+0000","last_fullsized":"2023-10-09T06:37:08.538357+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:26:56.576485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.550902+0000","last_change":"2023-10-09T06:37:08.547584+0000","last_active":"2023-10-09T06:37:08.550902+0000","last_peered":"2023-10-09T06:37:08.550902+0000","last_clean":"2023-10-09T06:37:08.550902+0000","last_became_active":"2023-10-09T06:37:08.547261+0000","last_became_peered":"2023-10-09T06:37:08.547261+0000","last_unstale":"2023-10-09T06:37:08.550902+0000","last_undegraded":"2023-10-09T06:37:08.550902+0000","last_fullsized":"2023-10-09T06:37:08.550902+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:10:54.887525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"112'3","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.773403+0000","last_change":"2023-10-09T06:37:07.763756+0000","last_active":"2023-10-09T06:37:08.773403+0000","last_peered":"2023-10-09T06:37:08.773403+0000","last_clean":"2023-10-09T06:37:08.773403+0000","last_became_active":"2023-10-09T06:37:07.763539+0000","last_became_peered":"2023-10-09T06:37:07.763539+0000","last_unstale":"2023-10-09T06:37:08.773403+0000","last_undegraded":"2023-10-09T06:37:08.773403+0000","last_fullsized":"2023-10-09T06:37:08.773403+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:58:50.252321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.551742+0000","last_change":"2023-10-09T06:37:08.549155+0000","last_active":"2023-10-09T06:37:08.551742+0000","last_peered":"2023-10-09T06:37:08.551742+0000","last_clean":"2023-10-09T06:37:08.551742+0000","last_became_active":"2023-10-09T06:37:08.548679+0000","last_became_peered":"2023-10-09T06:37:08.548679+0000","last_unstale":"2023-10-09T06:37:08.551742+0000","last_undegraded":"2023-10-09T06:37:08.551742+0000","last_fullsized":"2023-10-09T06:37:08.551742+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:55:34.352472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"112'6","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.774034+0000","last_change":"2023-10-09T06:37:07.760289+0000","last_active":"2023-10-09T06:37:08.774034+0000","last_peered":"2023-10-09T06:37:08.774034+0000","last_clean":"2023-10-09T06:37:08.774034+0000","last_became_active":"2023-10-09T06:37:07.759541+0000","last_became_peered":"2023-10-09T06:37:07.759541+0000","last_unstale":"2023-10-09T06:37:08.774034+0000","last_undegraded":"2023-10-09T06:37:08.774034+0000","last_fullsized":"2023-10-09T06:37:08.774034+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:07:07.702870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.560814+0000","last_change":"2023-10-09T06:37:08.554764+0000","last_active":"2023-10-09T06:37:08.560814+0000","last_peered":"2023-10-09T06:37:08.560814+0000","last_clean":"2023-10-09T06:37:08.560814+0000","last_became_active":"2023-10-09T06:37:08.554446+0000","last_became_peered":"2023-10-09T06:37:08.554446+0000","last_unstale":"2023-10-09T06:37:08.560814+0000","last_undegraded":"2023-10-09T06:37:08.560814+0000","last_fullsized":"2023-10-09T06:37:08.560814+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:58:23.994134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.544850+0000","last_change":"2023-10-09T06:37:07.762060+0000","last_active":"2023-10-09T06:37:08.544850+0000","last_peered":"2023-10-09T06:37:08.544850+0000","last_clean":"2023-10-09T06:37:08.544850+0000","last_became_active":"2023-10-09T06:37:07.761733+0000","last_became_peered":"2023-10-09T06:37:07.761733+0000","last_unstale":"2023-10-09T06:37:08.544850+0000","last_undegraded":"2023-10-09T06:37:08.544850+0000","last_fullsized":"2023-10-09T06:37:08.544850+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:10:25.968961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.554536+0000","last_change":"2023-10-09T06:37:08.547068+0000","last_active":"2023-10-09T06:37:08.554536+0000","last_peered":"2023-10-09T06:37:08.554536+0000","last_clean":"2023-10-09T06:37:08.554536+0000","last_became_active":"2023-10-09T06:37:08.546789+0000","last_became_peered":"2023-10-09T06:37:08.546789+0000","last_unstale":"2023-10-09T06:37:08.554536+0000","last_undegraded":"2023-10-09T06:37:08.554536+0000","last_fullsized":"2023-10-09T06:37:08.554536+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:34:01.203730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.540866+0000","last_change":"2023-10-09T06:37:07.768269+0000","last_active":"2023-10-09T06:37:08.540866+0000","last_peered":"2023-10-09T06:37:08.540866+0000","last_clean":"2023-10-09T06:37:08.540866+0000","last_became_active":"2023-10-09T06:37:07.768050+0000","last_became_peered":"2023-10-09T06:37:07.768050+0000","last_unstale":"2023-10-09T06:37:08.540866+0000","last_undegraded":"2023-10-09T06:37:08.540866+0000","last_fullsized":"2023-10-09T06:37:08.540866+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:43:45.871080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.550373+0000","last_change":"2023-10-09T06:37:08.546639+0000","last_active":"2023-10-09T06:37:08.550373+0000","last_peered":"2023-10-09T06:37:08.550373+0000","last_clean":"2023-10-09T06:37:08.550373+0000","last_became_active":"2023-10-09T06:37:08.546336+0000","last_became_peered":"2023-10-09T06:37:08.546336+0000","last_unstale":"2023-10-09T06:37:08.550373+0000","last_undegraded":"2023-10-09T06:37:08.550373+0000","last_fullsized":"2023-10-09T06:37:08.550373+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:06:50.653374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"112'8","reported_seq":18,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.774280+0000","last_change":"2023-10-09T06:37:07.762897+0000","last_active":"2023-10-09T06:37:08.774280+0000","last_peered":"2023-10-09T06:37:08.774280+0000","last_clean":"2023-10-09T06:37:08.774280+0000","last_became_active":"2023-10-09T06:37:07.762652+0000","last_became_peered":"2023-10-09T06:37:07.762652+0000","last_unstale":"2023-10-09T06:37:08.774280+0000","last_undegraded":"2023-10-09T06:37:08.774280+0000","last_fullsized":"2023-10-09T06:37:08.774280+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:22:08.809994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.557748+0000","last_change":"2023-10-09T06:37:08.550231+0000","last_active":"2023-10-09T06:37:08.557748+0000","last_peered":"2023-10-09T06:37:08.557748+0000","last_clean":"2023-10-09T06:37:08.557748+0000","last_became_active":"2023-10-09T06:37:08.549958+0000","last_became_peered":"2023-10-09T06:37:08.549958+0000","last_unstale":"2023-10-09T06:37:08.557748+0000","last_undegraded":"2023-10-09T06:37:08.557748+0000","last_fullsized":"2023-10-09T06:37:08.557748+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:10:06.890078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"112'1","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.771490+0000","last_change":"2023-10-09T06:37:07.765711+0000","last_active":"2023-10-09T06:37:08.771490+0000","last_peered":"2023-10-09T06:37:08.771490+0000","last_clean":"2023-10-09T06:37:08.771490+0000","last_became_active":"2023-10-09T06:37:07.765471+0000","last_became_peered":"2023-10-09T06:37:07.765471+0000","last_unstale":"2023-10-09T06:37:08.771490+0000","last_undegraded":"2023-10-09T06:37:08.771490+0000","last_fullsized":"2023-10-09T06:37:08.771490+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:07:07.702870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.557553+0000","last_change":"2023-10-09T06:37:08.555412+0000","last_active":"2023-10-09T06:37:08.557553+0000","last_peered":"2023-10-09T06:37:08.557553+0000","last_clean":"2023-10-09T06:37:08.557553+0000","last_became_active":"2023-10-09T06:37:08.555140+0000","last_became_peered":"2023-10-09T06:37:08.555140+0000","last_unstale":"2023-10-09T06:37:08.557553+0000","last_undegraded":"2023-10-09T06:37:08.557553+0000","last_fullsized":"2023-10-09T06:37:08.557553+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:46:37.103853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"112'2","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.771307+0000","last_change":"2023-10-09T06:37:07.759053+0000","last_active":"2023-10-09T06:37:08.771307+0000","last_peered":"2023-10-09T06:37:08.771307+0000","last_clean":"2023-10-09T06:37:08.771307+0000","last_became_active":"2023-10-09T06:37:07.758811+0000","last_became_peered":"2023-10-09T06:37:07.758811+0000","last_unstale":"2023-10-09T06:37:08.771307+0000","last_undegraded":"2023-10-09T06:37:08.771307+0000","last_fullsized":"2023-10-09T06:37:08.771307+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:38:28.960341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.551808+0000","last_change":"2023-10-09T06:37:08.548066+0000","last_active":"2023-10-09T06:37:08.551808+0000","last_peered":"2023-10-09T06:37:08.551808+0000","last_clean":"2023-10-09T06:37:08.551808+0000","last_became_active":"2023-10-09T06:37:08.547874+0000","last_became_peered":"2023-10-09T06:37:08.547874+0000","last_unstale":"2023-10-09T06:37:08.551808+0000","last_undegraded":"2023-10-09T06:37:08.551808+0000","last_fullsized":"2023-10-09T06:37:08.551808+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:04:07.309237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"112'1","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.773361+0000","last_change":"2023-10-09T06:37:07.762702+0000","last_active":"2023-10-09T06:37:08.773361+0000","last_peered":"2023-10-09T06:37:08.773361+0000","last_clean":"2023-10-09T06:37:08.773361+0000","last_became_active":"2023-10-09T06:37:07.762456+0000","last_became_peered":"2023-10-09T06:37:07.762456+0000","last_unstale":"2023-10-09T06:37:08.773361+0000","last_undegraded":"2023-10-09T06:37:08.773361+0000","last_fullsized":"2023-10-09T06:37:08.773361+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:25:31.399270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.556656+0000","last_change":"2023-10-09T06:37:08.551582+0000","last_active":"2023-10-09T06:37:08.556656+0000","last_peered":"2023-10-09T06:37:08.556656+0000","last_clean":"2023-10-09T06:37:08.556656+0000","last_became_active":"2023-10-09T06:37:08.551017+0000","last_became_peered":"2023-10-09T06:37:08.551017+0000","last_unstale":"2023-10-09T06:37:08.556656+0000","last_undegraded":"2023-10-09T06:37:08.556656+0000","last_fullsized":"2023-10-09T06:37:08.556656+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:10:45.977689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.545058+0000","last_change":"2023-10-09T06:37:07.757787+0000","last_active":"2023-10-09T06:37:08.545058+0000","last_peered":"2023-10-09T06:37:08.545058+0000","last_clean":"2023-10-09T06:37:08.545058+0000","last_became_active":"2023-10-09T06:37:07.757508+0000","last_became_peered":"2023-10-09T06:37:07.757508+0000","last_unstale":"2023-10-09T06:37:08.545058+0000","last_undegraded":"2023-10-09T06:37:08.545058+0000","last_fullsized":"2023-10-09T06:37:08.545058+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:30:15.484180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.560856+0000","last_change":"2023-10-09T06:37:08.556247+0000","last_active":"2023-10-09T06:37:08.560856+0000","last_peered":"2023-10-09T06:37:08.560856+0000","last_clean":"2023-10-09T06:37:08.560856+0000","last_became_active":"2023-10-09T06:37:08.555910+0000","last_became_peered":"2023-10-09T06:37:08.555910+0000","last_unstale":"2023-10-09T06:37:08.560856+0000","last_undegraded":"2023-10-09T06:37:08.560856+0000","last_fullsized":"2023-10-09T06:37:08.560856+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:29:06.291104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"112'2","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.772662+0000","last_change":"2023-10-09T06:37:07.764433+0000","last_active":"2023-10-09T06:37:08.772662+0000","last_peered":"2023-10-09T06:37:08.772662+0000","last_clean":"2023-10-09T06:37:08.772662+0000","last_became_active":"2023-10-09T06:37:07.764096+0000","last_became_peered":"2023-10-09T06:37:07.764096+0000","last_unstale":"2023-10-09T06:37:08.772662+0000","last_undegraded":"2023-10-09T06:37:08.772662+0000","last_fullsized":"2023-10-09T06:37:08.772662+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:22:08.809994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.551625+0000","last_change":"2023-10-09T06:37:08.541143+0000","last_active":"2023-10-09T06:37:08.551625+0000","last_peered":"2023-10-09T06:37:08.551625+0000","last_clean":"2023-10-09T06:37:08.551625+0000","last_became_active":"2023-10-09T06:37:08.540854+0000","last_became_peered":"2023-10-09T06:37:08.540854+0000","last_unstale":"2023-10-09T06:37:08.551625+0000","last_undegraded":"2023-10-09T06:37:08.551625+0000","last_fullsized":"2023-10-09T06:37:08.551625+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:59:57.832775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.537751+0000","last_change":"2023-10-09T06:37:07.759706+0000","last_active":"2023-10-09T06:37:08.537751+0000","last_peered":"2023-10-09T06:37:08.537751+0000","last_clean":"2023-10-09T06:37:08.537751+0000","last_became_active":"2023-10-09T06:37:07.759519+0000","last_became_peered":"2023-10-09T06:37:07.759519+0000","last_unstale":"2023-10-09T06:37:08.537751+0000","last_undegraded":"2023-10-09T06:37:08.537751+0000","last_fullsized":"2023-10-09T06:37:08.537751+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:49:01.460212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.559033+0000","last_change":"2023-10-09T06:37:08.555663+0000","last_active":"2023-10-09T06:37:08.559033+0000","last_peered":"2023-10-09T06:37:08.559033+0000","last_clean":"2023-10-09T06:37:08.559033+0000","last_became_active":"2023-10-09T06:37:08.555405+0000","last_became_peered":"2023-10-09T06:37:08.555405+0000","last_unstale":"2023-10-09T06:37:08.559033+0000","last_undegraded":"2023-10-09T06:37:08.559033+0000","last_fullsized":"2023-10-09T06:37:08.559033+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:23:20.895190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.537545+0000","last_change":"2023-10-09T06:37:07.758288+0000","last_active":"2023-10-09T06:37:08.537545+0000","last_peered":"2023-10-09T06:37:08.537545+0000","last_clean":"2023-10-09T06:37:08.537545+0000","last_became_active":"2023-10-09T06:37:07.758029+0000","last_became_peered":"2023-10-09T06:37:07.758029+0000","last_unstale":"2023-10-09T06:37:08.537545+0000","last_undegraded":"2023-10-09T06:37:08.537545+0000","last_fullsized":"2023-10-09T06:37:08.537545+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:20:05.321978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.556643+0000","last_change":"2023-10-09T06:37:08.554975+0000","last_active":"2023-10-09T06:37:08.556643+0000","last_peered":"2023-10-09T06:37:08.556643+0000","last_clean":"2023-10-09T06:37:08.556643+0000","last_became_active":"2023-10-09T06:37:08.554688+0000","last_became_peered":"2023-10-09T06:37:08.554688+0000","last_unstale":"2023-10-09T06:37:08.556643+0000","last_undegraded":"2023-10-09T06:37:08.556643+0000","last_fullsized":"2023-10-09T06:37:08.556643+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:38:27.210425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.540916+0000","last_change":"2023-10-09T06:37:07.762931+0000","last_active":"2023-10-09T06:37:08.540916+0000","last_peered":"2023-10-09T06:37:08.540916+0000","last_clean":"2023-10-09T06:37:08.540916+0000","last_became_active":"2023-10-09T06:37:07.762725+0000","last_became_peered":"2023-10-09T06:37:07.762725+0000","last_unstale":"2023-10-09T06:37:08.540916+0000","last_undegraded":"2023-10-09T06:37:08.540916+0000","last_fullsized":"2023-10-09T06:37:08.540916+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:18:53.098466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.553176+0000","last_change":"2023-10-09T06:37:08.548167+0000","last_active":"2023-10-09T06:37:08.553176+0000","last_peered":"2023-10-09T06:37:08.553176+0000","last_clean":"2023-10-09T06:37:08.553176+0000","last_became_active":"2023-10-09T06:37:08.547700+0000","last_became_peered":"2023-10-09T06:37:08.547700+0000","last_unstale":"2023-10-09T06:37:08.553176+0000","last_undegraded":"2023-10-09T06:37:08.553176+0000","last_fullsized":"2023-10-09T06:37:08.553176+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:41:55.509062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"112'2","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.771697+0000","last_change":"2023-10-09T06:37:07.763529+0000","last_active":"2023-10-09T06:37:08.771697+0000","last_peered":"2023-10-09T06:37:08.771697+0000","last_clean":"2023-10-09T06:37:08.771697+0000","last_became_active":"2023-10-09T06:37:07.763261+0000","last_became_peered":"2023-10-09T06:37:07.763261+0000","last_unstale":"2023-10-09T06:37:08.771697+0000","last_undegraded":"2023-10-09T06:37:08.771697+0000","last_fullsized":"2023-10-09T06:37:08.771697+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:40:11.133728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.554668+0000","last_change":"2023-10-09T06:37:08.550020+0000","last_active":"2023-10-09T06:37:08.554668+0000","last_peered":"2023-10-09T06:37:08.554668+0000","last_clean":"2023-10-09T06:37:08.554668+0000","last_became_active":"2023-10-09T06:37:08.549776+0000","last_became_peered":"2023-10-09T06:37:08.549776+0000","last_unstale":"2023-10-09T06:37:08.554668+0000","last_undegraded":"2023-10-09T06:37:08.554668+0000","last_fullsized":"2023-10-09T06:37:08.554668+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:46:32.727481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.543009+0000","last_change":"2023-10-09T06:37:07.764948+0000","last_active":"2023-10-09T06:37:08.543009+0000","last_peered":"2023-10-09T06:37:08.543009+0000","last_clean":"2023-10-09T06:37:08.543009+0000","last_became_active":"2023-10-09T06:37:07.764704+0000","last_became_peered":"2023-10-09T06:37:07.764704+0000","last_unstale":"2023-10-09T06:37:08.543009+0000","last_undegraded":"2023-10-09T06:37:08.543009+0000","last_fullsized":"2023-10-09T06:37:08.543009+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:23:19.891326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.554568+0000","last_change":"2023-10-09T06:37:08.546976+0000","last_active":"2023-10-09T06:37:08.554568+0000","last_peered":"2023-10-09T06:37:08.554568+0000","last_clean":"2023-10-09T06:37:08.554568+0000","last_became_active":"2023-10-09T06:37:08.546709+0000","last_became_peered":"2023-10-09T06:37:08.546709+0000","last_unstale":"2023-10-09T06:37:08.554568+0000","last_undegraded":"2023-10-09T06:37:08.554568+0000","last_fullsized":"2023-10-09T06:37:08.554568+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:40:16.413666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.540575+0000","last_change":"2023-10-09T06:37:07.764780+0000","last_active":"2023-10-09T06:37:08.540575+0000","last_peered":"2023-10-09T06:37:08.540575+0000","last_clean":"2023-10-09T06:37:08.540575+0000","last_became_active":"2023-10-09T06:37:07.764492+0000","last_became_peered":"2023-10-09T06:37:07.764492+0000","last_unstale":"2023-10-09T06:37:08.540575+0000","last_undegraded":"2023-10-09T06:37:08.540575+0000","last_fullsized":"2023-10-09T06:37:08.540575+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:47:44.879129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.553298+0000","last_change":"2023-10-09T06:37:08.548844+0000","last_active":"2023-10-09T06:37:08.553298+0000","last_peered":"2023-10-09T06:37:08.553298+0000","last_clean":"2023-10-09T06:37:08.553298+0000","last_became_active":"2023-10-09T06:37:08.548577+0000","last_became_peered":"2023-10-09T06:37:08.548577+0000","last_unstale":"2023-10-09T06:37:08.553298+0000","last_undegraded":"2023-10-09T06:37:08.553298+0000","last_fullsized":"2023-10-09T06:37:08.553298+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:38:42.567166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.540655+0000","last_change":"2023-10-09T06:37:07.761330+0000","last_active":"2023-10-09T06:37:08.540655+0000","last_peered":"2023-10-09T06:37:08.540655+0000","last_clean":"2023-10-09T06:37:08.540655+0000","last_became_active":"2023-10-09T06:37:07.761051+0000","last_became_peered":"2023-10-09T06:37:07.761051+0000","last_unstale":"2023-10-09T06:37:08.540655+0000","last_undegraded":"2023-10-09T06:37:08.540655+0000","last_fullsized":"2023-10-09T06:37:08.540655+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:38:26.206561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.556451+0000","last_change":"2023-10-09T06:37:08.553941+0000","last_active":"2023-10-09T06:37:08.556451+0000","last_peered":"2023-10-09T06:37:08.556451+0000","last_clean":"2023-10-09T06:37:08.556451+0000","last_became_active":"2023-10-09T06:37:08.553662+0000","last_became_peered":"2023-10-09T06:37:08.553662+0000","last_unstale":"2023-10-09T06:37:08.556451+0000","last_undegraded":"2023-10-09T06:37:08.556451+0000","last_fullsized":"2023-10-09T06:37:08.556451+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:07:45.944631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.539842+0000","last_change":"2023-10-09T06:37:07.765790+0000","last_active":"2023-10-09T06:37:08.539842+0000","last_peered":"2023-10-09T06:37:08.539842+0000","last_clean":"2023-10-09T06:37:08.539842+0000","last_became_active":"2023-10-09T06:37:07.765555+0000","last_became_peered":"2023-10-09T06:37:07.765555+0000","last_unstale":"2023-10-09T06:37:08.539842+0000","last_undegraded":"2023-10-09T06:37:08.539842+0000","last_fullsized":"2023-10-09T06:37:08.539842+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:36:48.859422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.552850+0000","last_change":"2023-10-09T06:37:08.550852+0000","last_active":"2023-10-09T06:37:08.552850+0000","last_peered":"2023-10-09T06:37:08.552850+0000","last_clean":"2023-10-09T06:37:08.552850+0000","last_became_active":"2023-10-09T06:37:08.550574+0000","last_became_peered":"2023-10-09T06:37:08.550574+0000","last_unstale":"2023-10-09T06:37:08.552850+0000","last_undegraded":"2023-10-09T06:37:08.552850+0000","last_fullsized":"2023-10-09T06:37:08.552850+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:25:32.403134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.541516+0000","last_change":"2023-10-09T06:37:07.765156+0000","last_active":"2023-10-09T06:37:08.541516+0000","last_peered":"2023-10-09T06:37:08.541516+0000","last_clean":"2023-10-09T06:37:08.541516+0000","last_became_active":"2023-10-09T06:37:07.764683+0000","last_became_peered":"2023-10-09T06:37:07.764683+0000","last_unstale":"2023-10-09T06:37:08.541516+0000","last_undegraded":"2023-10-09T06:37:08.541516+0000","last_fullsized":"2023-10-09T06:37:08.541516+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:04:39.436580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.549018+0000","last_change":"2023-10-09T06:37:08.542620+0000","last_active":"2023-10-09T06:37:08.549018+0000","last_peered":"2023-10-09T06:37:08.549018+0000","last_clean":"2023-10-09T06:37:08.549018+0000","last_became_active":"2023-10-09T06:37:08.542363+0000","last_became_peered":"2023-10-09T06:37:08.542363+0000","last_unstale":"2023-10-09T06:37:08.549018+0000","last_undegraded":"2023-10-09T06:37:08.549018+0000","last_fullsized":"2023-10-09T06:37:08.549018+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:38:29.964205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.537684+0000","last_change":"2023-10-09T06:37:07.759280+0000","last_active":"2023-10-09T06:37:08.537684+0000","last_peered":"2023-10-09T06:37:08.537684+0000","last_clean":"2023-10-09T06:37:08.537684+0000","last_became_active":"2023-10-09T06:37:07.759028+0000","last_became_peered":"2023-10-09T06:37:07.759028+0000","last_unstale":"2023-10-09T06:37:08.537684+0000","last_undegraded":"2023-10-09T06:37:08.537684+0000","last_fullsized":"2023-10-09T06:37:08.537684+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:34:00.199866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.554549+0000","last_change":"2023-10-09T06:37:08.547159+0000","last_active":"2023-10-09T06:37:08.554549+0000","last_peered":"2023-10-09T06:37:08.554549+0000","last_clean":"2023-10-09T06:37:08.554549+0000","last_became_active":"2023-10-09T06:37:08.546942+0000","last_became_peered":"2023-10-09T06:37:08.546942+0000","last_unstale":"2023-10-09T06:37:08.554549+0000","last_undegraded":"2023-10-09T06:37:08.554549+0000","last_fullsized":"2023-10-09T06:37:08.554549+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:38:20.913074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.540990+0000","last_change":"2023-10-09T06:37:07.756405+0000","last_active":"2023-10-09T06:37:08.540990+0000","last_peered":"2023-10-09T06:37:08.540990+0000","last_clean":"2023-10-09T06:37:08.540990+0000","last_became_active":"2023-10-09T06:37:07.756138+0000","last_became_peered":"2023-10-09T06:37:07.756138+0000","last_unstale":"2023-10-09T06:37:08.540990+0000","last_undegraded":"2023-10-09T06:37:08.540990+0000","last_fullsized":"2023-10-09T06:37:08.540990+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:41:54.852777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.556475+0000","last_change":"2023-10-09T06:37:08.553781+0000","last_active":"2023-10-09T06:37:08.556475+0000","last_peered":"2023-10-09T06:37:08.556475+0000","last_clean":"2023-10-09T06:37:08.556475+0000","last_became_active":"2023-10-09T06:37:08.553466+0000","last_became_peered":"2023-10-09T06:37:08.553466+0000","last_unstale":"2023-10-09T06:37:08.556475+0000","last_undegraded":"2023-10-09T06:37:08.556475+0000","last_fullsized":"2023-10-09T06:37:08.556475+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:04:07.309237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"112'2","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.773385+0000","last_change":"2023-10-09T06:37:07.765120+0000","last_active":"2023-10-09T06:37:08.773385+0000","last_peered":"2023-10-09T06:37:08.773385+0000","last_clean":"2023-10-09T06:37:08.773385+0000","last_became_active":"2023-10-09T06:37:07.764763+0000","last_became_peered":"2023-10-09T06:37:07.764763+0000","last_unstale":"2023-10-09T06:37:08.773385+0000","last_undegraded":"2023-10-09T06:37:08.773385+0000","last_fullsized":"2023-10-09T06:37:08.773385+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:53:18.927143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":240,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.541423+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:37:08.541423+0000","last_peered":"2023-10-09T06:37:08.541423+0000","last_clean":"2023-10-09T06:37:08.541423+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:37:08.541423+0000","last_undegraded":"2023-10-09T06:37:08.541423+0000","last_fullsized":"2023-10-09T06:37:08.541423+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574932,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303368,"kb_used_data":1984,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068472,"statfs":{"total":96636764160,"available":96326115328,"internally_reserved":0,"allocated":2031616,"data_stored":1205425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51100000000000001,"15min":0.505},"min":{"1min":0.33400000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68000000000000005,"5min":0.753,"15min":0.753},"last":0.57099999999999995}]},{"osd":1,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.50600000000000001,"15min":0.52000000000000002},"min":{"1min":0.38900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":1.2030000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33800000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.64000000000000001,"5min":0.70599999999999996,"15min":0.71999999999999997},"last":0.48599999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52800000000000002,"15min":0.51200000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72899999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52700000000000002,"15min":0.54400000000000004},"min":{"1min":0.38200000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.83099999999999996},"last":0.55300000000000005}]},{"osd":3,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53100000000000003,"15min":0.55100000000000005},"min":{"1min":0.35899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.627},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.55600000000000005,"15min":0.59499999999999997},"min":{"1min":0.46600000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73599999999999999,"5min":0.82099999999999995,"15min":1.1479999999999999},"last":0.65400000000000003}]},{"osd":4,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46899999999999997,"15min":0.51900000000000002},"min":{"1min":0.30299999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.63700000000000001,"5min":0.69999999999999996,"15min":0.93999999999999995},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56000000000000005,"15min":0.60799999999999998},"min":{"1min":0.35399999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.68999999999999995}]},{"osd":5,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.57199999999999995},"min":{"1min":0.38800000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.70899999999999996,"5min":0.85799999999999998,"15min":0.91100000000000003},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.58499999999999996,"15min":0.62},"min":{"1min":0.437,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.67100000000000004}]},{"osd":6,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.54600000000000004,"15min":0.59599999999999997},"min":{"1min":0.40200000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.88600000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54500000000000004,"15min":0.61499999999999999},"min":{"1min":0.39700000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.77800000000000002,"5min":0.79900000000000004,"15min":0.84599999999999997},"last":0.71899999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574931,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302776,"kb_used_data":1392,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069064,"statfs":{"total":96636764160,"available":96326721536,"internally_reserved":0,"allocated":1425408,"data_stored":614447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.502,"15min":0.503},"min":{"1min":0.36399999999999999,"5min":0.32000000000000001,"15min":0.30399999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.503,"5min":0.50800000000000001,"15min":0.51000000000000001},"min":{"1min":0.36299999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.60499999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.495,"15min":0.51300000000000001},"min":{"1min":0.35999999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.498},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52400000000000002,"15min":0.53100000000000003},"min":{"1min":0.36199999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.73199999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51500000000000001,"15min":0.51700000000000002},"min":{"1min":0.41599999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51000000000000001,"15min":0.53200000000000003},"min":{"1min":0.36099999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.628,"5min":0.69399999999999995,"15min":0.752},"last":0.64700000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53200000000000003,"15min":0.54700000000000004},"min":{"1min":0.39300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54200000000000004,"15min":0.70099999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69199999999999995,"5min":0.74399999999999999,"15min":4.7510000000000003},"last":0.70799999999999996}]},{"osd":4,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46400000000000002,"15min":0.47899999999999998},"min":{"1min":0.28199999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.76400000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56599999999999995,"15min":0.58299999999999996},"min":{"1min":0.29499999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.80800000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.54900000000000004}]},{"osd":5,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.53300000000000003},"min":{"1min":0.318,"5min":0.246,"15min":0.246},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.81200000000000006},"last":0.372},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57099999999999995,"15min":0.59699999999999998},"min":{"1min":0.378,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70799999999999996,"5min":0.78000000000000003,"15min":0.86599999999999999},"last":0.69499999999999995}]},{"osd":7,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55200000000000005,"15min":0.58599999999999997},"min":{"1min":0.35199999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71299999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.41899999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.73599999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.627}]}]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302764,"kb_used_data":1380,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069076,"statfs":{"total":96636764160,"available":96326733824,"internally_reserved":0,"allocated":1413120,"data_stored":612911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.31,"15min":0.31},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34599999999999997,"15min":0.35899999999999999},"min":{"1min":0.26300000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.318}]},{"osd":2,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.34699999999999998,"15min":0.35299999999999998},"min":{"1min":0.24299999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38300000000000001,"15min":0.38600000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.57199999999999995,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.36399999999999999}]},{"osd":3,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.36599999999999999,"15min":0.36399999999999999},"min":{"1min":0.27300000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52300000000000002,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.34999999999999998,"15min":0.55800000000000005},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":5.8529999999999998},"last":0.34799999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58099999999999996,"15min":0.57599999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77800000000000002,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.874,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.56499999999999995}]},{"osd":5,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56999999999999995,"15min":0.76100000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83099999999999996,"5min":0.83799999999999997,"15min":5.9459999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.60399999999999998,"15min":0.61299999999999999},"min":{"1min":0.41899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85699999999999998,"5min":0.92700000000000005,"15min":0.93999999999999995},"last":0.57599999999999996}]},{"osd":6,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.54900000000000004,"15min":0.73299999999999998},"min":{"1min":0.38,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":5.9160000000000004},"last":0.496},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83299999999999996,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.54200000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56799999999999995,"15min":0.58899999999999997},"min":{"1min":0.39300000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.81599999999999995,"5min":0.89500000000000002,"15min":0.97099999999999997},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56399999999999995,"15min":0.77000000000000002},"min":{"1min":0.40400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.71899999999999997,"5min":0.80900000000000005,"15min":5.9729999999999999},"last":0.58599999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574933,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303344,"kb_used_data":1960,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068496,"statfs":{"total":96636764160,"available":96326139904,"internally_reserved":0,"allocated":2007040,"data_stored":1203279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.30499999999999999,"15min":0.316},"min":{"1min":0.22600000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.49399999999999999,"5min":0.498,"15min":0.498},"last":0.221},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.33200000000000002,"15min":0.34499999999999997},"min":{"1min":0.25,"5min":0.214,"15min":0.214},"max":{"1min":0.48599999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.28499999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.35199999999999998,"15min":0.36299999999999999},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.314},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.379,"15min":0.38800000000000001},"min":{"1min":0.29499999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.69399999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.33500000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.33400000000000002,"15min":0.39000000000000001},"min":{"1min":0.246,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.39700000000000002,"5min":0.52900000000000003,"15min":1.554},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.36599999999999999,"15min":0.40000000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.40899999999999997,"5min":0.60899999999999999,"15min":0.79000000000000004},"last":0.35399999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.55000000000000004,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70199999999999996,"5min":1.212,"15min":1.212},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55300000000000005,"15min":0.73899999999999999},"min":{"1min":0.37,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71999999999999997,"5min":0.82499999999999996,"15min":5.484},"last":0.56799999999999995}]},{"osd":5,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.74299999999999999},"min":{"1min":0.39600000000000002,"5min":0.35799999999999998,"15min":0.35499999999999998},"max":{"1min":0.69299999999999995,"5min":0.84399999999999997,"15min":5.5449999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56899999999999995,"15min":0.57499999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.80900000000000005,"5min":0.98699999999999999,"15min":1.113},"last":0.59699999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58999999999999997,"15min":0.60599999999999998},"min":{"1min":0.44400000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.83999999999999997,"5min":1.0269999999999999,"15min":1.0529999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.45700000000000002,"5min":0.41799999999999998,"15min":0.41599999999999998},"max":{"1min":0.755,"5min":1.169,"15min":1.169},"last":0.55500000000000005}]},{"osd":7,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58399999999999996,"15min":0.58999999999999997},"min":{"1min":0.42899999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.76900000000000002,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56599999999999995,"15min":0.76900000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81599999999999995,"5min":0.873,"15min":5.5800000000000001},"last":0.63400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574934,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302772,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069068,"statfs":{"total":96636764160,"available":96326725632,"internally_reserved":0,"allocated":1421312,"data_stored":613499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.309,"15min":0.35499999999999998},"min":{"1min":0.21199999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.441,"5min":0.50800000000000001,"15min":1.4199999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.32200000000000001,"15min":0.58399999999999996},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.48299999999999998,"5min":0.53800000000000003,"15min":7.2699999999999996},"last":0.28899999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32100000000000001,"15min":0.34699999999999998},"min":{"1min":0.22700000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.41199999999999998,"5min":0.497,"15min":0.65400000000000003},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.371,"15min":0.63900000000000001},"min":{"1min":0.29599999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.51500000000000001,"5min":0.57599999999999996,"15min":7.327},"last":0.33900000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36899999999999999,"15min":0.39100000000000001},"min":{"1min":0.28199999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.48399999999999999,"5min":0.52900000000000003,"15min":0.68600000000000005},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36699999999999999,"15min":0.39300000000000002},"min":{"1min":0.26500000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.51400000000000001,"5min":0.54000000000000004,"15min":0.72699999999999998},"last":0.38600000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.34999999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.79700000000000004,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55300000000000005,"15min":0.80800000000000005},"min":{"1min":0.38600000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72699999999999998,"5min":0.85399999999999998,"15min":7.2389999999999999},"last":0.503}]},{"osd":5,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58799999999999997,"15min":0.95299999999999996},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72499999999999998,"5min":0.83799999999999997,"15min":10.890000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59099999999999997,"15min":0.60199999999999998},"min":{"1min":0.41499999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.55800000000000005}]},{"osd":6,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56799999999999995,"15min":0.81599999999999995},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76700000000000002,"5min":0.89100000000000001,"15min":7.2999999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56999999999999995,"15min":0.57199999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.57299999999999995}]},{"osd":7,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59399999999999997,"15min":0.84699999999999998},"min":{"1min":0.44600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.78800000000000003,"5min":0.87,"15min":7.1879999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60999999999999999,"15min":0.626},"min":{"1min":0.42699999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.90500000000000003},"last":0.52500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574933,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302772,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069068,"statfs":{"total":96636764160,"available":96326725632,"internally_reserved":0,"allocated":1421312,"data_stored":613071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.36099999999999999,"15min":0.32900000000000001},"min":{"1min":0.223,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.46200000000000002,"5min":5.8700000000000001,"15min":5.8700000000000001},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.38500000000000001,"15min":0.36599999999999999},"min":{"1min":0.23400000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.502,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.36799999999999999}]},{"osd":1,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.379,"15min":0.35799999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.51800000000000002,"5min":4.0469999999999997,"15min":4.0469999999999997},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.40300000000000002,"15min":0.38800000000000001},"min":{"1min":0.28399999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.52300000000000002,"5min":4.1509999999999998,"15min":4.1509999999999998},"last":0.31900000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.374},"min":{"1min":0.26800000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.56799999999999995,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38900000000000001,"15min":0.38500000000000001},"min":{"1min":0.26800000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.54000000000000004,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.34499999999999997}]},{"osd":4,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58199999999999996,"15min":0.57099999999999995},"min":{"1min":0.40600000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.74199999999999999,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.42199999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77400000000000002,"5min":3.0710000000000002,"15min":3.0710000000000002},"last":0.65700000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.61599999999999999,"15min":0.58799999999999997},"min":{"1min":0.45700000000000002,"5min":0.34599999999999997,"15min":0.32700000000000001},"max":{"1min":0.69899999999999995,"5min":6.2300000000000004,"15min":6.2300000000000004},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61499999999999999,"15min":0.59699999999999998},"min":{"1min":0.495,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79700000000000004,"5min":3.0219999999999998,"15min":3.0219999999999998},"last":0.58699999999999997}]},{"osd":6,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61599999999999999,"15min":0.59199999999999997},"min":{"1min":0.48199999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.78300000000000003,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68899999999999995,"15min":0.65100000000000002},"min":{"1min":0.46700000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.82499999999999996,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.79400000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64100000000000001,"15min":0.61299999999999999},"min":{"1min":0.44900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85399999999999998,"5min":4.2629999999999999,"15min":4.2629999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.65500000000000003,"15min":0.61299999999999999},"min":{"1min":0.50800000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.77400000000000002,"5min":7.3650000000000002,"15min":7.3650000000000002},"last":0.71699999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302768,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069072,"statfs":{"total":96636764160,"available":96326729728,"internally_reserved":0,"allocated":1417216,"data_stored":612981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.51400000000000001,"15min":0.55400000000000005},"min":{"1min":0.28100000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.65100000000000002,"5min":0.78800000000000003,"15min":1.083},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.52600000000000002,"15min":0.51700000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.67800000000000005,"5min":0.97799999999999998,"15min":1.054},"last":0.54700000000000004}]},{"osd":1,"last update":"Mon Oct 9 06:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.71399999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.50600000000000001,"15min":0.49399999999999999},"min":{"1min":0.31900000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.67100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.45800000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52100000000000002,"15min":0.54100000000000004},"min":{"1min":0.35999999999999999,"5min":0.33700000000000002,"15min":0.32900000000000001},"max":{"1min":0.69399999999999995,"5min":0.82099999999999995,"15min":1.0249999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55500000000000005,"15min":0.59299999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.68799999999999994,"5min":0.78300000000000003,"15min":0.93799999999999994},"last":0.496}]},{"osd":3,"last update":"Mon Oct 9 06:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.52600000000000002,"15min":0.55200000000000005},"min":{"1min":0.32200000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70199999999999996,"5min":0.77500000000000002,"15min":0.98499999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56999999999999995,"15min":0.58899999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73599999999999999,"5min":0.86199999999999999,"15min":1.1319999999999999},"last":0.52900000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51300000000000001,"15min":0.55700000000000005},"min":{"1min":0.30399999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.74099999999999999,"5min":0.78000000000000003,"15min":0.998},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.60999999999999999},"min":{"1min":0.377,"5min":0.36299999999999999,"15min":0.35099999999999998},"max":{"1min":0.70999999999999996,"5min":0.80000000000000004,"15min":0.97999999999999998},"last":0.51700000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53900000000000003,"15min":0.56899999999999995},"min":{"1min":0.34699999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80200000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56699999999999995,"15min":0.59999999999999998},"min":{"1min":0.38700000000000001,"5min":0.33500000000000002,"15min":0.32700000000000001},"max":{"1min":0.77900000000000003,"5min":1.01,"15min":1.01},"last":0.57799999999999996}]},{"osd":7,"last update":"Mon Oct 9 06:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54000000000000004,"15min":0.57199999999999995},"min":{"1min":0.35599999999999998,"5min":0.33600000000000002,"15min":0.28299999999999997},"max":{"1min":0.753,"5min":0.85599999999999998,"15min":0.90000000000000002},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.59799999999999998,"15min":0.64800000000000002},"min":{"1min":0.34999999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79500000000000004,"5min":0.98999999999999999,"15min":1.175},"last":0.59699999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574932,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298676,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073164,"statfs":{"total":96636764160,"available":96330919936,"internally_reserved":0,"allocated":1421312,"data_stored":613023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.496,"15min":0.50700000000000001},"min":{"1min":0.314,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.64100000000000001,"5min":0.71399999999999997,"15min":0.86599999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.51900000000000002,"15min":0.51400000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.42499999999999999}]},{"osd":1,"last update":"Mon Oct 9 06:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.50800000000000001,"15min":0.51000000000000001},"min":{"1min":0.34899999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.61699999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.52100000000000002,"15min":0.53100000000000003},"min":{"1min":0.30599999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73899999999999999,"5min":0.754,"15min":0.81599999999999995},"last":0.44400000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.33100000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.70699999999999996,"5min":0.73299999999999998,"15min":0.83999999999999997},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57899999999999996,"15min":0.58699999999999997},"min":{"1min":0.46500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72799999999999998,"5min":0.745,"15min":0.79900000000000004},"last":0.63100000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.41099999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.69799999999999995,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.503},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55500000000000005,"15min":0.56399999999999995},"min":{"1min":0.39800000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.71099999999999997,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.58899999999999997}]},{"osd":4,"last update":"Mon Oct 9 06:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.495,"15min":0.496},"min":{"1min":0.29999999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54200000000000004,"15min":0.54500000000000004},"min":{"1min":0.35299999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.72299999999999998,"5min":0.76300000000000001,"15min":0.80700000000000005},"last":0.60699999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55600000000000005,"15min":0.53400000000000003},"min":{"1min":0.376,"5min":0.35699999999999998,"15min":0.32600000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.83199999999999996},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.42599999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.62}]},{"osd":7,"last update":"Mon Oct 9 06:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.39600000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76900000000000002,"5min":0.84399999999999997,"15min":0.85599999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.442,"5min":0.35199999999999998,"15min":0.314},"max":{"1min":0.81399999999999995,"5min":0.84499999999999997,"15min":0.90600000000000003},"last":0.69699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:37:12.835 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:37:12.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.981+0000 7f7e1e9a5700 1 Processor -- start 2023-10-09T06:37:12.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.981+0000 7f7e1e9a5700 1 -- start start 2023-10-09T06:37:12.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.982+0000 7f7e1e9a5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7e18159890 0x7f7e1815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:12.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.982+0000 7f7e1e9a5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e18069560 0x7f7e180699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:12.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.982+0000 7f7e1e9a5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e18069ee0 con 0x7f7e18159c60 2023-10-09T06:37:12.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.982+0000 7f7e1e9a5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e1815a160 con 0x7f7e18159890 2023-10-09T06:37:12.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.982+0000 7f7e1e9a5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e180d13a0 con 0x7f7e18069560 2023-10-09T06:37:12.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.982+0000 7f7e17fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7e18159890 0x7f7e1815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:12.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.982+0000 7f7e1cf42700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e18069560 0x7f7e180699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:12.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.982+0000 7f7e17fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7e18159890 0x7f7e1815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:59108/0 (socket says 172.21.15.163:59108) 2023-10-09T06:37:12.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.982+0000 7f7e1cf42700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e18069560 0x7f7e180699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:58982/0 (socket says 172.21.15.163:58982) 2023-10-09T06:37:12.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.982+0000 7f7e177fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7e18159c60 0x7f7e1815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:34940/0 (socket says 172.21.15.163:34940) 2023-10-09T06:37:12.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.982+0000 7f7e17fff700 1 -- 172.21.15.163:0/1573777241 learned_addr learned my addr 172.21.15.163:0/1573777241 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:12.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.982+0000 7f7e17fff700 1 -- 172.21.15.163:0/1573777241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e18069560 msgr2=0x7f7e180699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:12.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.982+0000 7f7e17fff700 1 --2- 172.21.15.163:0/1573777241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e18069560 0x7f7e180699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.983+0000 7f7e17fff700 1 -- 172.21.15.163:0/1573777241 >> v1:172.21.15.181:6789/0 conn(0x7f7e18159c60 legacy=0x7f7e1815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:12.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.983+0000 7f7e17fff700 1 -- 172.21.15.163:0/1573777241 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e1815e140 con 0x7f7e18159890 2023-10-09T06:37:12.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.983+0000 7f7e1cf42700 1 --2- 172.21.15.163:0/1573777241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e18069560 0x7f7e180699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:37:12.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.983+0000 7f7e17fff700 1 --2- 172.21.15.163:0/1573777241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e18159890 0x7f7e1815abd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7e080099b0 tx=0x7f7e08009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77b727f7d46738bb server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:12.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.983+0000 7f7e16ffd700 1 -- 172.21.15.163:0/1573777241 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e08009ea0 con 0x7f7e18159890 2023-10-09T06:37:12.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.983+0000 7f7e16ffd700 1 -- 172.21.15.163:0/1573777241 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e0800b860 con 0x7f7e18159890 2023-10-09T06:37:12.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.983+0000 7f7e16ffd700 1 -- 172.21.15.163:0/1573777241 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0800bc00 con 0x7f7e18159890 2023-10-09T06:37:12.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.983+0000 7f7e1e9a5700 1 -- 172.21.15.163:0/1573777241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e18159890 msgr2=0x7f7e1815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:12.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.983+0000 7f7e1e9a5700 1 --2- 172.21.15.163:0/1573777241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e18159890 0x7f7e1815abd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7e080099b0 tx=0x7f7e08009250 comp rx=0 tx=0).stop 2023-10-09T06:37:12.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.983+0000 7f7e1e9a5700 1 -- 172.21.15.163:0/1573777241 shutdown_connections 2023-10-09T06:37:12.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.983+0000 7f7e1e9a5700 1 --2- 172.21.15.163:0/1573777241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e18159890 0x7f7e1815abd0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.983+0000 7f7e1e9a5700 1 --2- 172.21.15.163:0/1573777241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e18069560 0x7f7e180699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.983+0000 7f7e1e9a5700 1 -- 172.21.15.163:0/1573777241 >> 172.21.15.163:0/1573777241 conn(0x7f7e18068e20 msgr2=0x7f7e180d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:12.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.983+0000 7f7e1e9a5700 1 -- 172.21.15.163:0/1573777241 shutdown_connections 2023-10-09T06:37:12.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.983+0000 7f7e1e9a5700 1 -- 172.21.15.163:0/1573777241 wait complete. 2023-10-09T06:37:12.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.984+0000 7f7e1e9a5700 1 Processor -- start 2023-10-09T06:37:12.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.985+0000 7f7e1e9a5700 1 -- start start 2023-10-09T06:37:12.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.985+0000 7f7e1e9a5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e18069560 0x7f7e18145260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:12.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.985+0000 7f7e1e9a5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e18159890 0x7f7e180cf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:12.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.985+0000 7f7e1e9a5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e18159c60 0x7f7e180cf750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:12.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.985+0000 7f7e1e9a5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e181457a0 con 0x7f7e18159c60 2023-10-09T06:37:12.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.985+0000 7f7e1e9a5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e180c9950 con 0x7f7e18159890 2023-10-09T06:37:12.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.985+0000 7f7e1e9a5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e180c9c30 con 0x7f7e18069560 2023-10-09T06:37:12.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.985+0000 7f7e1cf42700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e18159c60 0x7f7e180cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:12.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.986+0000 7f7e1cf42700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e18159c60 0x7f7e180cf750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56810/0 (socket says 172.21.15.163:56810) 2023-10-09T06:37:12.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.986+0000 7f7e1cf42700 1 -- 172.21.15.163:0/3587459373 learned_addr learned my addr 172.21.15.163:0/3587459373 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:12.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.986+0000 7f7e177fe700 1 --2- 172.21.15.163:0/3587459373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e18159890 0x7f7e180cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:12.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.986+0000 7f7e17fff700 1 --2- 172.21.15.163:0/3587459373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e18069560 0x7f7e18145260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:12.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.986+0000 7f7e1cf42700 1 -- 172.21.15.163:0/3587459373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e18069560 msgr2=0x7f7e18145260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:12.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.986+0000 7f7e1cf42700 1 --2- 172.21.15.163:0/3587459373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e18069560 0x7f7e18145260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.986+0000 7f7e1cf42700 1 -- 172.21.15.163:0/3587459373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e18159890 msgr2=0x7f7e180cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:12.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.986+0000 7f7e1cf42700 1 --2- 172.21.15.163:0/3587459373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e18159890 0x7f7e180cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:12.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.986+0000 7f7e1cf42700 1 -- 172.21.15.163:0/3587459373 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e10009a20 con 0x7f7e18159c60 2023-10-09T06:37:12.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.986+0000 7f7e177fe700 1 --2- 172.21.15.163:0/3587459373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e18159890 0x7f7e180cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:12.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.986+0000 7f7e1cf42700 1 --2- 172.21.15.163:0/3587459373 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e18159c60 0x7f7e180cf750 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009f20 tx=0x7f7e1000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:12.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.986+0000 7f7dfffff700 1 -- 172.21.15.163:0/3587459373 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1000cc80 con 0x7f7e18159c60 2023-10-09T06:37:12.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.986+0000 7f7dfffff700 1 -- 172.21.15.163:0/3587459373 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e1000f040 con 0x7f7e18159c60 2023-10-09T06:37:12.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.986+0000 7f7e1e9a5700 1 -- 172.21.15.163:0/3587459373 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e080094f0 con 0x7f7e18159c60 2023-10-09T06:37:12.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.986+0000 7f7dfffff700 1 -- 172.21.15.163:0/3587459373 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e10014480 con 0x7f7e18159c60 2023-10-09T06:37:12.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.986+0000 7f7e1e9a5700 1 -- 172.21.15.163:0/3587459373 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e180ca1f0 con 0x7f7e18159c60 2023-10-09T06:37:12.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.987+0000 7f7e1e9a5700 1 -- 172.21.15.163:0/3587459373 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e180ca410 con 0x7f7e18159c60 2023-10-09T06:37:12.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.987+0000 7f7dfffff700 1 -- 172.21.15.163:0/3587459373 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7e10003020 con 0x7f7e18159c60 2023-10-09T06:37:12.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.988+0000 7f7dfffff700 1 --2- 172.21.15.163:0/3587459373 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7e04067320 0x7f7e040697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:12.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.988+0000 7f7dfffff700 1 -- 172.21.15.163:0/3587459373 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7e10085700 con 0x7f7e18159c60 2023-10-09T06:37:12.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.988+0000 7f7e17fff700 1 --2- 172.21.15.163:0/3587459373 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7e04067320 0x7f7e040697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:12.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.988+0000 7f7e17fff700 1 --2- 172.21.15.163:0/3587459373 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7e04067320 0x7f7e040697d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7e080099b0 tx=0x7f7e08017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:12.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:12.992+0000 7f7dfffff700 1 -- 172.21.15.163:0/3587459373 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7e10052300 con 0x7f7e18159c60 2023-10-09T06:37:13.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.170+0000 7f7e1e9a5700 1 -- 172.21.15.163:0/3587459373 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7e1804c3c0 con 0x7f7e04067320 2023-10-09T06:37:13.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.179+0000 7f7dfffff700 1 -- 172.21.15.163:0/3587459373 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290191 (secure 0 0 0) 0x7f7e1804c3c0 con 0x7f7e04067320 2023-10-09T06:37:13.183 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:13.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.182+0000 7f7e1e9a5700 1 -- 172.21.15.163:0/3587459373 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7e04067320 msgr2=0x7f7e040697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:13.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.182+0000 7f7e1e9a5700 1 --2- 172.21.15.163:0/3587459373 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7e04067320 0x7f7e040697d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7e080099b0 tx=0x7f7e08017000 comp rx=0 tx=0).stop 2023-10-09T06:37:13.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.182+0000 7f7e1e9a5700 1 -- 172.21.15.163:0/3587459373 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e18159c60 msgr2=0x7f7e180cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:13.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.182+0000 7f7e1e9a5700 1 --2- 172.21.15.163:0/3587459373 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e18159c60 0x7f7e180cf750 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009f20 tx=0x7f7e1000b040 comp rx=0 tx=0).stop 2023-10-09T06:37:13.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.182+0000 7f7e1e9a5700 1 -- 172.21.15.163:0/3587459373 shutdown_connections 2023-10-09T06:37:13.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.182+0000 7f7e1e9a5700 1 --2- 172.21.15.163:0/3587459373 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7e04067320 0x7f7e040697d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:13.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.182+0000 7f7e1e9a5700 1 --2- 172.21.15.163:0/3587459373 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e18159c60 0x7f7e180cf750 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:13.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.182+0000 7f7e1e9a5700 1 --2- 172.21.15.163:0/3587459373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e18069560 0x7f7e18145260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:13.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.182+0000 7f7e1e9a5700 1 --2- 172.21.15.163:0/3587459373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e18159890 0x7f7e180cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:13.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.182+0000 7f7e1e9a5700 1 -- 172.21.15.163:0/3587459373 >> 172.21.15.163:0/3587459373 conn(0x7f7e18068e20 msgr2=0x7f7e18154760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:13.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.182+0000 7f7e1e9a5700 1 -- 172.21.15.163:0/3587459373 shutdown_connections 2023-10-09T06:37:13.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.182+0000 7f7e1e9a5700 1 -- 172.21.15.163:0/3587459373 wait complete. 2023-10-09T06:37:13.186 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:37:13.195 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2023-10-09T06:37:12.394152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2419240,"kb_used_data":12264,"kb_used_omap":0,"kb_used_meta":2406912,"kb_avail":752555480,"statfs":{"total":773094113280,"available":770616811520,"internally_reserved":0,"allocated":12558336,"data_stored":6088636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2464677888},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.650831"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.560747+0000","last_change":"2023-10-09T06:37:08.553928+0000","last_active":"2023-10-09T06:37:08.560747+0000","last_peered":"2023-10-09T06:37:08.560747+0000","last_clean":"2023-10-09T06:37:08.560747+0000","last_became_active":"2023-10-09T06:37:08.553656+0000","last_became_peered":"2023-10-09T06:37:08.553656+0000","last_unstale":"2023-10-09T06:37:08.560747+0000","last_undegraded":"2023-10-09T06:37:08.560747+0000","last_fullsized":"2023-10-09T06:37:08.560747+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:57:31.509105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.556688+0000","last_change":"2023-10-09T06:37:08.553111+0000","last_active":"2023-10-09T06:37:08.556688+0000","last_peered":"2023-10-09T06:37:08.556688+0000","last_clean":"2023-10-09T06:37:08.556688+0000","last_became_active":"2023-10-09T06:37:08.552846+0000","last_became_peered":"2023-10-09T06:37:08.552846+0000","last_unstale":"2023-10-09T06:37:08.556688+0000","last_undegraded":"2023-10-09T06:37:08.556688+0000","last_fullsized":"2023-10-09T06:37:08.556688+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:49:02.464075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.560363+0000","last_change":"2023-10-09T06:37:08.555838+0000","last_active":"2023-10-09T06:37:08.560363+0000","last_peered":"2023-10-09T06:37:08.560363+0000","last_clean":"2023-10-09T06:37:08.560363+0000","last_became_active":"2023-10-09T06:37:08.555580+0000","last_became_peered":"2023-10-09T06:37:08.555580+0000","last_unstale":"2023-10-09T06:37:08.560363+0000","last_undegraded":"2023-10-09T06:37:08.560363+0000","last_fullsized":"2023-10-09T06:37:08.560363+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:43:46.874944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.552650+0000","last_change":"2023-10-09T06:37:08.548090+0000","last_active":"2023-10-09T06:37:08.552650+0000","last_peered":"2023-10-09T06:37:08.552650+0000","last_clean":"2023-10-09T06:37:08.552650+0000","last_became_active":"2023-10-09T06:37:08.547819+0000","last_became_peered":"2023-10-09T06:37:08.547819+0000","last_unstale":"2023-10-09T06:37:08.552650+0000","last_undegraded":"2023-10-09T06:37:08.552650+0000","last_fullsized":"2023-10-09T06:37:08.552650+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:24:57.108220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.551203+0000","last_change":"2023-10-09T06:37:08.547420+0000","last_active":"2023-10-09T06:37:08.551203+0000","last_peered":"2023-10-09T06:37:08.551203+0000","last_clean":"2023-10-09T06:37:08.551203+0000","last_became_active":"2023-10-09T06:37:08.547159+0000","last_became_peered":"2023-10-09T06:37:08.547159+0000","last_unstale":"2023-10-09T06:37:08.551203+0000","last_undegraded":"2023-10-09T06:37:08.551203+0000","last_fullsized":"2023-10-09T06:37:08.551203+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:16:58.160581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.554196+0000","last_change":"2023-10-09T06:37:08.548276+0000","last_active":"2023-10-09T06:37:08.554196+0000","last_peered":"2023-10-09T06:37:08.554196+0000","last_clean":"2023-10-09T06:37:08.554196+0000","last_became_active":"2023-10-09T06:37:08.548045+0000","last_became_peered":"2023-10-09T06:37:08.548045+0000","last_unstale":"2023-10-09T06:37:08.554196+0000","last_undegraded":"2023-10-09T06:37:08.554196+0000","last_fullsized":"2023-10-09T06:37:08.554196+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:26:57.580349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.551565+0000","last_change":"2023-10-09T06:37:08.540730+0000","last_active":"2023-10-09T06:37:08.551565+0000","last_peered":"2023-10-09T06:37:08.551565+0000","last_clean":"2023-10-09T06:37:08.551565+0000","last_became_active":"2023-10-09T06:37:08.540473+0000","last_became_peered":"2023-10-09T06:37:08.540473+0000","last_unstale":"2023-10-09T06:37:08.551565+0000","last_undegraded":"2023-10-09T06:37:08.551565+0000","last_fullsized":"2023-10-09T06:37:08.551565+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:23:20.895190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.544637+0000","last_change":"2023-10-09T06:37:08.541018+0000","last_active":"2023-10-09T06:37:08.544637+0000","last_peered":"2023-10-09T06:37:08.544637+0000","last_clean":"2023-10-09T06:37:08.544637+0000","last_became_active":"2023-10-09T06:37:08.540416+0000","last_became_peered":"2023-10-09T06:37:08.540416+0000","last_unstale":"2023-10-09T06:37:08.544637+0000","last_undegraded":"2023-10-09T06:37:08.544637+0000","last_fullsized":"2023-10-09T06:37:08.544637+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:23:41.031871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.557935+0000","last_change":"2023-10-09T06:37:08.553838+0000","last_active":"2023-10-09T06:37:08.557935+0000","last_peered":"2023-10-09T06:37:08.557935+0000","last_clean":"2023-10-09T06:37:08.557935+0000","last_became_active":"2023-10-09T06:37:08.553582+0000","last_became_peered":"2023-10-09T06:37:08.553582+0000","last_unstale":"2023-10-09T06:37:08.557935+0000","last_undegraded":"2023-10-09T06:37:08.557935+0000","last_fullsized":"2023-10-09T06:37:08.557935+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:46:37.103853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.556706+0000","last_change":"2023-10-09T06:37:08.553155+0000","last_active":"2023-10-09T06:37:08.556706+0000","last_peered":"2023-10-09T06:37:08.556706+0000","last_clean":"2023-10-09T06:37:08.556706+0000","last_became_active":"2023-10-09T06:37:08.552833+0000","last_became_peered":"2023-10-09T06:37:08.552833+0000","last_unstale":"2023-10-09T06:37:08.556706+0000","last_undegraded":"2023-10-09T06:37:08.556706+0000","last_fullsized":"2023-10-09T06:37:08.556706+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:26:57.580349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.558205+0000","last_change":"2023-10-09T06:37:08.554294+0000","last_active":"2023-10-09T06:37:08.558205+0000","last_peered":"2023-10-09T06:37:08.558205+0000","last_clean":"2023-10-09T06:37:08.558205+0000","last_became_active":"2023-10-09T06:37:08.554031+0000","last_became_peered":"2023-10-09T06:37:08.554031+0000","last_unstale":"2023-10-09T06:37:08.558205+0000","last_undegraded":"2023-10-09T06:37:08.558205+0000","last_fullsized":"2023-10-09T06:37:08.558205+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:57:35.952883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.549109+0000","last_change":"2023-10-09T06:37:08.547070+0000","last_active":"2023-10-09T06:37:08.549109+0000","last_peered":"2023-10-09T06:37:08.549109+0000","last_clean":"2023-10-09T06:37:08.549109+0000","last_became_active":"2023-10-09T06:37:08.546638+0000","last_became_peered":"2023-10-09T06:37:08.546638+0000","last_unstale":"2023-10-09T06:37:08.549109+0000","last_undegraded":"2023-10-09T06:37:08.549109+0000","last_fullsized":"2023-10-09T06:37:08.549109+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:22:30.623361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.560778+0000","last_change":"2023-10-09T06:37:08.554379+0000","last_active":"2023-10-09T06:37:08.560778+0000","last_peered":"2023-10-09T06:37:08.560778+0000","last_clean":"2023-10-09T06:37:08.560778+0000","last_became_active":"2023-10-09T06:37:08.553391+0000","last_became_peered":"2023-10-09T06:37:08.553391+0000","last_unstale":"2023-10-09T06:37:08.560778+0000","last_undegraded":"2023-10-09T06:37:08.560778+0000","last_fullsized":"2023-10-09T06:37:08.560778+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:46:47.898722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.556569+0000","last_change":"2023-10-09T06:37:08.549708+0000","last_active":"2023-10-09T06:37:08.556569+0000","last_peered":"2023-10-09T06:37:08.556569+0000","last_clean":"2023-10-09T06:37:08.556569+0000","last_became_active":"2023-10-09T06:37:08.549499+0000","last_became_peered":"2023-10-09T06:37:08.549499+0000","last_unstale":"2023-10-09T06:37:08.556569+0000","last_undegraded":"2023-10-09T06:37:08.556569+0000","last_fullsized":"2023-10-09T06:37:08.556569+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:04:40.440444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.553923+0000","last_change":"2023-10-09T06:37:08.549898+0000","last_active":"2023-10-09T06:37:08.553923+0000","last_peered":"2023-10-09T06:37:08.553923+0000","last_clean":"2023-10-09T06:37:08.553923+0000","last_became_active":"2023-10-09T06:37:08.549637+0000","last_became_peered":"2023-10-09T06:37:08.549637+0000","last_unstale":"2023-10-09T06:37:08.553923+0000","last_undegraded":"2023-10-09T06:37:08.553923+0000","last_fullsized":"2023-10-09T06:37:08.553923+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:17:01.899478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.550842+0000","last_change":"2023-10-09T06:37:08.546298+0000","last_active":"2023-10-09T06:37:08.550842+0000","last_peered":"2023-10-09T06:37:08.550842+0000","last_clean":"2023-10-09T06:37:08.550842+0000","last_became_active":"2023-10-09T06:37:08.546041+0000","last_became_peered":"2023-10-09T06:37:08.546041+0000","last_unstale":"2023-10-09T06:37:08.550842+0000","last_undegraded":"2023-10-09T06:37:08.550842+0000","last_fullsized":"2023-10-09T06:37:08.550842+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:18:24.085073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.553192+0000","last_change":"2023-10-09T06:37:08.550852+0000","last_active":"2023-10-09T06:37:08.553192+0000","last_peered":"2023-10-09T06:37:08.553192+0000","last_clean":"2023-10-09T06:37:08.553192+0000","last_became_active":"2023-10-09T06:37:08.550579+0000","last_became_peered":"2023-10-09T06:37:08.550579+0000","last_unstale":"2023-10-09T06:37:08.553192+0000","last_undegraded":"2023-10-09T06:37:08.553192+0000","last_fullsized":"2023-10-09T06:37:08.553192+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:53:19.931007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.554353+0000","last_change":"2023-10-09T06:37:08.550139+0000","last_active":"2023-10-09T06:37:08.554353+0000","last_peered":"2023-10-09T06:37:08.554353+0000","last_clean":"2023-10-09T06:37:08.554353+0000","last_became_active":"2023-10-09T06:37:08.549769+0000","last_became_peered":"2023-10-09T06:37:08.549769+0000","last_unstale":"2023-10-09T06:37:08.554353+0000","last_undegraded":"2023-10-09T06:37:08.554353+0000","last_fullsized":"2023-10-09T06:37:08.554353+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:38:27.210425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.558283+0000","last_change":"2023-10-09T06:37:08.555167+0000","last_active":"2023-10-09T06:37:08.558283+0000","last_peered":"2023-10-09T06:37:08.558283+0000","last_clean":"2023-10-09T06:37:08.558283+0000","last_became_active":"2023-10-09T06:37:08.554886+0000","last_became_peered":"2023-10-09T06:37:08.554886+0000","last_unstale":"2023-10-09T06:37:08.558283+0000","last_undegraded":"2023-10-09T06:37:08.558283+0000","last_fullsized":"2023-10-09T06:37:08.558283+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:45:41.231359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.551789+0000","last_change":"2023-10-09T06:37:08.547292+0000","last_active":"2023-10-09T06:37:08.551789+0000","last_peered":"2023-10-09T06:37:08.551789+0000","last_clean":"2023-10-09T06:37:08.551789+0000","last_became_active":"2023-10-09T06:37:08.546837+0000","last_became_peered":"2023-10-09T06:37:08.546837+0000","last_unstale":"2023-10-09T06:37:08.551789+0000","last_undegraded":"2023-10-09T06:37:08.551789+0000","last_fullsized":"2023-10-09T06:37:08.551789+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:41:55.856641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.551811+0000","last_change":"2023-10-09T06:37:08.550259+0000","last_active":"2023-10-09T06:37:08.551811+0000","last_peered":"2023-10-09T06:37:08.551811+0000","last_clean":"2023-10-09T06:37:08.551811+0000","last_became_active":"2023-10-09T06:37:08.549940+0000","last_became_peered":"2023-10-09T06:37:08.549940+0000","last_unstale":"2023-10-09T06:37:08.551811+0000","last_undegraded":"2023-10-09T06:37:08.551811+0000","last_fullsized":"2023-10-09T06:37:08.551811+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:18:24.085073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.556320+0000","last_change":"2023-10-09T06:37:08.550863+0000","last_active":"2023-10-09T06:37:08.556320+0000","last_peered":"2023-10-09T06:37:08.556320+0000","last_clean":"2023-10-09T06:37:08.556320+0000","last_became_active":"2023-10-09T06:37:08.550592+0000","last_became_peered":"2023-10-09T06:37:08.550592+0000","last_unstale":"2023-10-09T06:37:08.556320+0000","last_undegraded":"2023-10-09T06:37:08.556320+0000","last_fullsized":"2023-10-09T06:37:08.556320+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:00:25.759826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.550395+0000","last_change":"2023-10-09T06:37:08.547200+0000","last_active":"2023-10-09T06:37:08.550395+0000","last_peered":"2023-10-09T06:37:08.550395+0000","last_clean":"2023-10-09T06:37:08.550395+0000","last_became_active":"2023-10-09T06:37:08.546924+0000","last_became_peered":"2023-10-09T06:37:08.546924+0000","last_unstale":"2023-10-09T06:37:08.550395+0000","last_undegraded":"2023-10-09T06:37:08.550395+0000","last_fullsized":"2023-10-09T06:37:08.550395+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:25:32.403134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.556401+0000","last_change":"2023-10-09T06:37:08.549053+0000","last_active":"2023-10-09T06:37:08.556401+0000","last_peered":"2023-10-09T06:37:08.556401+0000","last_clean":"2023-10-09T06:37:08.556401+0000","last_became_active":"2023-10-09T06:37:08.548760+0000","last_became_peered":"2023-10-09T06:37:08.548760+0000","last_unstale":"2023-10-09T06:37:08.556401+0000","last_undegraded":"2023-10-09T06:37:08.556401+0000","last_fullsized":"2023-10-09T06:37:08.556401+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:36:21.796862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.548639+0000","last_change":"2023-10-09T06:37:08.545486+0000","last_active":"2023-10-09T06:37:08.548639+0000","last_peered":"2023-10-09T06:37:08.548639+0000","last_clean":"2023-10-09T06:37:08.548639+0000","last_became_active":"2023-10-09T06:37:08.545242+0000","last_became_peered":"2023-10-09T06:37:08.545242+0000","last_unstale":"2023-10-09T06:37:08.548639+0000","last_undegraded":"2023-10-09T06:37:08.548639+0000","last_fullsized":"2023-10-09T06:37:08.548639+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:06:05.712018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.552158+0000","last_change":"2023-10-09T06:37:08.547689+0000","last_active":"2023-10-09T06:37:08.552158+0000","last_peered":"2023-10-09T06:37:08.552158+0000","last_clean":"2023-10-09T06:37:08.552158+0000","last_became_active":"2023-10-09T06:37:08.547430+0000","last_became_peered":"2023-10-09T06:37:08.547430+0000","last_unstale":"2023-10-09T06:37:08.552158+0000","last_undegraded":"2023-10-09T06:37:08.552158+0000","last_fullsized":"2023-10-09T06:37:08.552158+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:17:34.019897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.560772+0000","last_change":"2023-10-09T06:37:08.554064+0000","last_active":"2023-10-09T06:37:08.560772+0000","last_peered":"2023-10-09T06:37:08.560772+0000","last_clean":"2023-10-09T06:37:08.560772+0000","last_became_active":"2023-10-09T06:37:08.553815+0000","last_became_peered":"2023-10-09T06:37:08.553815+0000","last_unstale":"2023-10-09T06:37:08.560772+0000","last_undegraded":"2023-10-09T06:37:08.560772+0000","last_fullsized":"2023-10-09T06:37:08.560772+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:03:04.328664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.552497+0000","last_change":"2023-10-09T06:37:08.551101+0000","last_active":"2023-10-09T06:37:08.552497+0000","last_peered":"2023-10-09T06:37:08.552497+0000","last_clean":"2023-10-09T06:37:08.552497+0000","last_became_active":"2023-10-09T06:37:08.550850+0000","last_became_peered":"2023-10-09T06:37:08.550850+0000","last_unstale":"2023-10-09T06:37:08.552497+0000","last_undegraded":"2023-10-09T06:37:08.552497+0000","last_fullsized":"2023-10-09T06:37:08.552497+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:22:30.623361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.554473+0000","last_change":"2023-10-09T06:37:08.550777+0000","last_active":"2023-10-09T06:37:08.554473+0000","last_peered":"2023-10-09T06:37:08.554473+0000","last_clean":"2023-10-09T06:37:08.554473+0000","last_became_active":"2023-10-09T06:37:08.550470+0000","last_became_peered":"2023-10-09T06:37:08.550470+0000","last_unstale":"2023-10-09T06:37:08.554473+0000","last_undegraded":"2023-10-09T06:37:08.554473+0000","last_fullsized":"2023-10-09T06:37:08.554473+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:21:42.229496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.556328+0000","last_change":"2023-10-09T06:37:08.550794+0000","last_active":"2023-10-09T06:37:08.556328+0000","last_peered":"2023-10-09T06:37:08.556328+0000","last_clean":"2023-10-09T06:37:08.556328+0000","last_became_active":"2023-10-09T06:37:08.550461+0000","last_became_peered":"2023-10-09T06:37:08.550461+0000","last_unstale":"2023-10-09T06:37:08.556328+0000","last_undegraded":"2023-10-09T06:37:08.556328+0000","last_fullsized":"2023-10-09T06:37:08.556328+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:18:02.972171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.556333+0000","last_change":"2023-10-09T06:37:08.553061+0000","last_active":"2023-10-09T06:37:08.556333+0000","last_peered":"2023-10-09T06:37:08.556333+0000","last_clean":"2023-10-09T06:37:08.556333+0000","last_became_active":"2023-10-09T06:37:08.552322+0000","last_became_peered":"2023-10-09T06:37:08.552322+0000","last_unstale":"2023-10-09T06:37:08.556333+0000","last_undegraded":"2023-10-09T06:37:08.556333+0000","last_fullsized":"2023-10-09T06:37:08.556333+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:41:55.856641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.552357+0000","last_change":"2023-10-09T06:37:08.543114+0000","last_active":"2023-10-09T06:37:08.552357+0000","last_peered":"2023-10-09T06:37:08.552357+0000","last_clean":"2023-10-09T06:37:08.552357+0000","last_became_active":"2023-10-09T06:37:08.542832+0000","last_became_peered":"2023-10-09T06:37:08.542832+0000","last_unstale":"2023-10-09T06:37:08.552357+0000","last_undegraded":"2023-10-09T06:37:08.552357+0000","last_fullsized":"2023-10-09T06:37:08.552357+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:56:38.197098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.554336+0000","last_change":"2023-10-09T06:37:08.550027+0000","last_active":"2023-10-09T06:37:08.554336+0000","last_peered":"2023-10-09T06:37:08.554336+0000","last_clean":"2023-10-09T06:37:08.554336+0000","last_became_active":"2023-10-09T06:37:08.549445+0000","last_became_peered":"2023-10-09T06:37:08.549445+0000","last_unstale":"2023-10-09T06:37:08.554336+0000","last_undegraded":"2023-10-09T06:37:08.554336+0000","last_fullsized":"2023-10-09T06:37:08.554336+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:16:58.160581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"112'4","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.773434+0000","last_change":"2023-10-09T06:37:07.765145+0000","last_active":"2023-10-09T06:37:08.773434+0000","last_peered":"2023-10-09T06:37:08.773434+0000","last_clean":"2023-10-09T06:37:08.773434+0000","last_became_active":"2023-10-09T06:37:07.764771+0000","last_became_peered":"2023-10-09T06:37:07.764771+0000","last_unstale":"2023-10-09T06:37:08.773434+0000","last_undegraded":"2023-10-09T06:37:08.773434+0000","last_fullsized":"2023-10-09T06:37:08.773434+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:41:54.505198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.557786+0000","last_change":"2023-10-09T06:37:08.550447+0000","last_active":"2023-10-09T06:37:08.557786+0000","last_peered":"2023-10-09T06:37:08.557786+0000","last_clean":"2023-10-09T06:37:08.557786+0000","last_became_active":"2023-10-09T06:37:08.550145+0000","last_became_peered":"2023-10-09T06:37:08.550145+0000","last_unstale":"2023-10-09T06:37:08.557786+0000","last_undegraded":"2023-10-09T06:37:08.557786+0000","last_fullsized":"2023-10-09T06:37:08.557786+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:47:45.882993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"112'1","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.771530+0000","last_change":"2023-10-09T06:37:07.758156+0000","last_active":"2023-10-09T06:37:08.771530+0000","last_peered":"2023-10-09T06:37:08.771530+0000","last_clean":"2023-10-09T06:37:08.771530+0000","last_became_active":"2023-10-09T06:37:07.757619+0000","last_became_peered":"2023-10-09T06:37:07.757619+0000","last_unstale":"2023-10-09T06:37:08.771530+0000","last_undegraded":"2023-10-09T06:37:08.771530+0000","last_fullsized":"2023-10-09T06:37:08.771530+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:47:47.047228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.556485+0000","last_change":"2023-10-09T06:37:08.554077+0000","last_active":"2023-10-09T06:37:08.556485+0000","last_peered":"2023-10-09T06:37:08.556485+0000","last_clean":"2023-10-09T06:37:08.556485+0000","last_became_active":"2023-10-09T06:37:08.553640+0000","last_became_peered":"2023-10-09T06:37:08.553640+0000","last_unstale":"2023-10-09T06:37:08.556485+0000","last_undegraded":"2023-10-09T06:37:08.556485+0000","last_fullsized":"2023-10-09T06:37:08.556485+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:43:59.013054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"112'2","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.770994+0000","last_change":"2023-10-09T06:37:07.762798+0000","last_active":"2023-10-09T06:37:08.770994+0000","last_peered":"2023-10-09T06:37:08.770994+0000","last_clean":"2023-10-09T06:37:08.770994+0000","last_became_active":"2023-10-09T06:37:07.762473+0000","last_became_peered":"2023-10-09T06:37:07.762473+0000","last_unstale":"2023-10-09T06:37:08.770994+0000","last_undegraded":"2023-10-09T06:37:08.770994+0000","last_fullsized":"2023-10-09T06:37:08.770994+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:04:06.305373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.551693+0000","last_change":"2023-10-09T06:37:08.548413+0000","last_active":"2023-10-09T06:37:08.551693+0000","last_peered":"2023-10-09T06:37:08.551693+0000","last_clean":"2023-10-09T06:37:08.551693+0000","last_became_active":"2023-10-09T06:37:08.548029+0000","last_became_peered":"2023-10-09T06:37:08.548029+0000","last_unstale":"2023-10-09T06:37:08.551693+0000","last_undegraded":"2023-10-09T06:37:08.551693+0000","last_fullsized":"2023-10-09T06:37:08.551693+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:17:34.019897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"112'2","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.770512+0000","last_change":"2023-10-09T06:37:07.760484+0000","last_active":"2023-10-09T06:37:08.770512+0000","last_peered":"2023-10-09T06:37:08.770512+0000","last_clean":"2023-10-09T06:37:08.770512+0000","last_became_active":"2023-10-09T06:37:07.760195+0000","last_became_peered":"2023-10-09T06:37:07.760195+0000","last_unstale":"2023-10-09T06:37:08.770512+0000","last_undegraded":"2023-10-09T06:37:08.770512+0000","last_fullsized":"2023-10-09T06:37:08.770512+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:25:31.399270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.554312+0000","last_change":"2023-10-09T06:37:08.549926+0000","last_active":"2023-10-09T06:37:08.554312+0000","last_peered":"2023-10-09T06:37:08.554312+0000","last_clean":"2023-10-09T06:37:08.554312+0000","last_became_active":"2023-10-09T06:37:08.549664+0000","last_became_peered":"2023-10-09T06:37:08.549664+0000","last_unstale":"2023-10-09T06:37:08.554312+0000","last_undegraded":"2023-10-09T06:37:08.554312+0000","last_fullsized":"2023-10-09T06:37:08.554312+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:07:45.944631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"112'1","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.772402+0000","last_change":"2023-10-09T06:37:07.764154+0000","last_active":"2023-10-09T06:37:08.772402+0000","last_peered":"2023-10-09T06:37:08.772402+0000","last_clean":"2023-10-09T06:37:08.772402+0000","last_became_active":"2023-10-09T06:37:07.763894+0000","last_became_peered":"2023-10-09T06:37:07.763894+0000","last_unstale":"2023-10-09T06:37:08.772402+0000","last_undegraded":"2023-10-09T06:37:08.772402+0000","last_fullsized":"2023-10-09T06:37:08.772402+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:22:29.619497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.557778+0000","last_change":"2023-10-09T06:37:08.555152+0000","last_active":"2023-10-09T06:37:08.557778+0000","last_peered":"2023-10-09T06:37:08.557778+0000","last_clean":"2023-10-09T06:37:08.557778+0000","last_became_active":"2023-10-09T06:37:08.554661+0000","last_became_peered":"2023-10-09T06:37:08.554661+0000","last_unstale":"2023-10-09T06:37:08.557778+0000","last_undegraded":"2023-10-09T06:37:08.557778+0000","last_fullsized":"2023-10-09T06:37:08.557778+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:04:40.440444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"112'1","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.770726+0000","last_change":"2023-10-09T06:37:07.762196+0000","last_active":"2023-10-09T06:37:08.770726+0000","last_peered":"2023-10-09T06:37:08.770726+0000","last_clean":"2023-10-09T06:37:08.770726+0000","last_became_active":"2023-10-09T06:37:07.761925+0000","last_became_peered":"2023-10-09T06:37:07.761925+0000","last_unstale":"2023-10-09T06:37:08.770726+0000","last_undegraded":"2023-10-09T06:37:08.770726+0000","last_fullsized":"2023-10-09T06:37:08.770726+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:10:44.973825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.545157+0000","last_change":"2023-10-09T06:37:08.541188+0000","last_active":"2023-10-09T06:37:08.545157+0000","last_peered":"2023-10-09T06:37:08.545157+0000","last_clean":"2023-10-09T06:37:08.545157+0000","last_became_active":"2023-10-09T06:37:08.540743+0000","last_became_peered":"2023-10-09T06:37:08.540743+0000","last_unstale":"2023-10-09T06:37:08.545157+0000","last_undegraded":"2023-10-09T06:37:08.545157+0000","last_fullsized":"2023-10-09T06:37:08.545157+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:57:10.867923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.537369+0000","last_change":"2023-10-09T06:37:07.760429+0000","last_active":"2023-10-09T06:37:08.537369+0000","last_peered":"2023-10-09T06:37:08.537369+0000","last_clean":"2023-10-09T06:37:08.537369+0000","last_became_active":"2023-10-09T06:37:07.760185+0000","last_became_peered":"2023-10-09T06:37:07.760185+0000","last_unstale":"2023-10-09T06:37:08.537369+0000","last_undegraded":"2023-10-09T06:37:08.537369+0000","last_fullsized":"2023-10-09T06:37:08.537369+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:22:29.619497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.556640+0000","last_change":"2023-10-09T06:37:08.551322+0000","last_active":"2023-10-09T06:37:08.556640+0000","last_peered":"2023-10-09T06:37:08.556640+0000","last_clean":"2023-10-09T06:37:08.556640+0000","last_became_active":"2023-10-09T06:37:08.551076+0000","last_became_peered":"2023-10-09T06:37:08.551076+0000","last_unstale":"2023-10-09T06:37:08.556640+0000","last_undegraded":"2023-10-09T06:37:08.556640+0000","last_fullsized":"2023-10-09T06:37:08.556640+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:20:06.325842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.542740+0000","last_change":"2023-10-09T06:37:07.765273+0000","last_active":"2023-10-09T06:37:08.542740+0000","last_peered":"2023-10-09T06:37:08.542740+0000","last_clean":"2023-10-09T06:37:08.542740+0000","last_became_active":"2023-10-09T06:37:07.765034+0000","last_became_peered":"2023-10-09T06:37:07.765034+0000","last_unstale":"2023-10-09T06:37:08.542740+0000","last_undegraded":"2023-10-09T06:37:08.542740+0000","last_fullsized":"2023-10-09T06:37:08.542740+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:10:05.886215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.560801+0000","last_change":"2023-10-09T06:37:08.554606+0000","last_active":"2023-10-09T06:37:08.560801+0000","last_peered":"2023-10-09T06:37:08.560801+0000","last_clean":"2023-10-09T06:37:08.560801+0000","last_became_active":"2023-10-09T06:37:08.554376+0000","last_became_peered":"2023-10-09T06:37:08.554376+0000","last_unstale":"2023-10-09T06:37:08.560801+0000","last_undegraded":"2023-10-09T06:37:08.560801+0000","last_fullsized":"2023-10-09T06:37:08.560801+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:36:41.110118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.547115+0000","last_change":"2023-10-09T06:37:07.764425+0000","last_active":"2023-10-09T06:37:08.547115+0000","last_peered":"2023-10-09T06:37:08.547115+0000","last_clean":"2023-10-09T06:37:08.547115+0000","last_became_active":"2023-10-09T06:37:07.764185+0000","last_became_peered":"2023-10-09T06:37:07.764185+0000","last_unstale":"2023-10-09T06:37:08.547115+0000","last_undegraded":"2023-10-09T06:37:08.547115+0000","last_fullsized":"2023-10-09T06:37:08.547115+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:21:28.762319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.552660+0000","last_change":"2023-10-09T06:37:08.548920+0000","last_active":"2023-10-09T06:37:08.552660+0000","last_peered":"2023-10-09T06:37:08.552660+0000","last_clean":"2023-10-09T06:37:08.552660+0000","last_became_active":"2023-10-09T06:37:08.548330+0000","last_became_peered":"2023-10-09T06:37:08.548330+0000","last_unstale":"2023-10-09T06:37:08.552660+0000","last_undegraded":"2023-10-09T06:37:08.552660+0000","last_fullsized":"2023-10-09T06:37:08.552660+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:06:50.653374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.538357+0000","last_change":"2023-10-09T06:37:07.759256+0000","last_active":"2023-10-09T06:37:08.538357+0000","last_peered":"2023-10-09T06:37:08.538357+0000","last_clean":"2023-10-09T06:37:08.538357+0000","last_became_active":"2023-10-09T06:37:07.758957+0000","last_became_peered":"2023-10-09T06:37:07.758957+0000","last_unstale":"2023-10-09T06:37:08.538357+0000","last_undegraded":"2023-10-09T06:37:08.538357+0000","last_fullsized":"2023-10-09T06:37:08.538357+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:26:56.576485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.550902+0000","last_change":"2023-10-09T06:37:08.547584+0000","last_active":"2023-10-09T06:37:08.550902+0000","last_peered":"2023-10-09T06:37:08.550902+0000","last_clean":"2023-10-09T06:37:08.550902+0000","last_became_active":"2023-10-09T06:37:08.547261+0000","last_became_peered":"2023-10-09T06:37:08.547261+0000","last_unstale":"2023-10-09T06:37:08.550902+0000","last_undegraded":"2023-10-09T06:37:08.550902+0000","last_fullsized":"2023-10-09T06:37:08.550902+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:10:54.887525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"112'3","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.773403+0000","last_change":"2023-10-09T06:37:07.763756+0000","last_active":"2023-10-09T06:37:08.773403+0000","last_peered":"2023-10-09T06:37:08.773403+0000","last_clean":"2023-10-09T06:37:08.773403+0000","last_became_active":"2023-10-09T06:37:07.763539+0000","last_became_peered":"2023-10-09T06:37:07.763539+0000","last_unstale":"2023-10-09T06:37:08.773403+0000","last_undegraded":"2023-10-09T06:37:08.773403+0000","last_fullsized":"2023-10-09T06:37:08.773403+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:58:50.252321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.551742+0000","last_change":"2023-10-09T06:37:08.549155+0000","last_active":"2023-10-09T06:37:08.551742+0000","last_peered":"2023-10-09T06:37:08.551742+0000","last_clean":"2023-10-09T06:37:08.551742+0000","last_became_active":"2023-10-09T06:37:08.548679+0000","last_became_peered":"2023-10-09T06:37:08.548679+0000","last_unstale":"2023-10-09T06:37:08.551742+0000","last_undegraded":"2023-10-09T06:37:08.551742+0000","last_fullsized":"2023-10-09T06:37:08.551742+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:55:34.352472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"112'6","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.774034+0000","last_change":"2023-10-09T06:37:07.760289+0000","last_active":"2023-10-09T06:37:08.774034+0000","last_peered":"2023-10-09T06:37:08.774034+0000","last_clean":"2023-10-09T06:37:08.774034+0000","last_became_active":"2023-10-09T06:37:07.759541+0000","last_became_peered":"2023-10-09T06:37:07.759541+0000","last_unstale":"2023-10-09T06:37:08.774034+0000","last_undegraded":"2023-10-09T06:37:08.774034+0000","last_fullsized":"2023-10-09T06:37:08.774034+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:07:07.702870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.560814+0000","last_change":"2023-10-09T06:37:08.554764+0000","last_active":"2023-10-09T06:37:08.560814+0000","last_peered":"2023-10-09T06:37:08.560814+0000","last_clean":"2023-10-09T06:37:08.560814+0000","last_became_active":"2023-10-09T06:37:08.554446+0000","last_became_peered":"2023-10-09T06:37:08.554446+0000","last_unstale":"2023-10-09T06:37:08.560814+0000","last_undegraded":"2023-10-09T06:37:08.560814+0000","last_fullsized":"2023-10-09T06:37:08.560814+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:58:23.994134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.544850+0000","last_change":"2023-10-09T06:37:07.762060+0000","last_active":"2023-10-09T06:37:08.544850+0000","last_peered":"2023-10-09T06:37:08.544850+0000","last_clean":"2023-10-09T06:37:08.544850+0000","last_became_active":"2023-10-09T06:37:07.761733+0000","last_became_peered":"2023-10-09T06:37:07.761733+0000","last_unstale":"2023-10-09T06:37:08.544850+0000","last_undegraded":"2023-10-09T06:37:08.544850+0000","last_fullsized":"2023-10-09T06:37:08.544850+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:10:25.968961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.554536+0000","last_change":"2023-10-09T06:37:08.547068+0000","last_active":"2023-10-09T06:37:08.554536+0000","last_peered":"2023-10-09T06:37:08.554536+0000","last_clean":"2023-10-09T06:37:08.554536+0000","last_became_active":"2023-10-09T06:37:08.546789+0000","last_became_peered":"2023-10-09T06:37:08.546789+0000","last_unstale":"2023-10-09T06:37:08.554536+0000","last_undegraded":"2023-10-09T06:37:08.554536+0000","last_fullsized":"2023-10-09T06:37:08.554536+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:34:01.203730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.540866+0000","last_change":"2023-10-09T06:37:07.768269+0000","last_active":"2023-10-09T06:37:08.540866+0000","last_peered":"2023-10-09T06:37:08.540866+0000","last_clean":"2023-10-09T06:37:08.540866+0000","last_became_active":"2023-10-09T06:37:07.768050+0000","last_became_peered":"2023-10-09T06:37:07.768050+0000","last_unstale":"2023-10-09T06:37:08.540866+0000","last_undegraded":"2023-10-09T06:37:08.540866+0000","last_fullsized":"2023-10-09T06:37:08.540866+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:43:45.871080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.550373+0000","last_change":"2023-10-09T06:37:08.546639+0000","last_active":"2023-10-09T06:37:08.550373+0000","last_peered":"2023-10-09T06:37:08.550373+0000","last_clean":"2023-10-09T06:37:08.550373+0000","last_became_active":"2023-10-09T06:37:08.546336+0000","last_became_peered":"2023-10-09T06:37:08.546336+0000","last_unstale":"2023-10-09T06:37:08.550373+0000","last_undegraded":"2023-10-09T06:37:08.550373+0000","last_fullsized":"2023-10-09T06:37:08.550373+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:06:50.653374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"112'8","reported_seq":18,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.774280+0000","last_change":"2023-10-09T06:37:07.762897+0000","last_active":"2023-10-09T06:37:08.774280+0000","last_peered":"2023-10-09T06:37:08.774280+0000","last_clean":"2023-10-09T06:37:08.774280+0000","last_became_active":"2023-10-09T06:37:07.762652+0000","last_became_peered":"2023-10-09T06:37:07.762652+0000","last_unstale":"2023-10-09T06:37:08.774280+0000","last_undegraded":"2023-10-09T06:37:08.774280+0000","last_fullsized":"2023-10-09T06:37:08.774280+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:22:08.809994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.557748+0000","last_change":"2023-10-09T06:37:08.550231+0000","last_active":"2023-10-09T06:37:08.557748+0000","last_peered":"2023-10-09T06:37:08.557748+0000","last_clean":"2023-10-09T06:37:08.557748+0000","last_became_active":"2023-10-09T06:37:08.549958+0000","last_became_peered":"2023-10-09T06:37:08.549958+0000","last_unstale":"2023-10-09T06:37:08.557748+0000","last_undegraded":"2023-10-09T06:37:08.557748+0000","last_fullsized":"2023-10-09T06:37:08.557748+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:10:06.890078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"112'1","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.771490+0000","last_change":"2023-10-09T06:37:07.765711+0000","last_active":"2023-10-09T06:37:08.771490+0000","last_peered":"2023-10-09T06:37:08.771490+0000","last_clean":"2023-10-09T06:37:08.771490+0000","last_became_active":"2023-10-09T06:37:07.765471+0000","last_became_peered":"2023-10-09T06:37:07.765471+0000","last_unstale":"2023-10-09T06:37:08.771490+0000","last_undegraded":"2023-10-09T06:37:08.771490+0000","last_fullsized":"2023-10-09T06:37:08.771490+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:07:07.702870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.557553+0000","last_change":"2023-10-09T06:37:08.555412+0000","last_active":"2023-10-09T06:37:08.557553+0000","last_peered":"2023-10-09T06:37:08.557553+0000","last_clean":"2023-10-09T06:37:08.557553+0000","last_became_active":"2023-10-09T06:37:08.555140+0000","last_became_peered":"2023-10-09T06:37:08.555140+0000","last_unstale":"2023-10-09T06:37:08.557553+0000","last_undegraded":"2023-10-09T06:37:08.557553+0000","last_fullsized":"2023-10-09T06:37:08.557553+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:46:37.103853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"112'2","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.771307+0000","last_change":"2023-10-09T06:37:07.759053+0000","last_active":"2023-10-09T06:37:08.771307+0000","last_peered":"2023-10-09T06:37:08.771307+0000","last_clean":"2023-10-09T06:37:08.771307+0000","last_became_active":"2023-10-09T06:37:07.758811+0000","last_became_peered":"2023-10-09T06:37:07.758811+0000","last_unstale":"2023-10-09T06:37:08.771307+0000","last_undegraded":"2023-10-09T06:37:08.771307+0000","last_fullsized":"2023-10-09T06:37:08.771307+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:38:28.960341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.551808+0000","last_change":"2023-10-09T06:37:08.548066+0000","last_active":"2023-10-09T06:37:08.551808+0000","last_peered":"2023-10-09T06:37:08.551808+0000","last_clean":"2023-10-09T06:37:08.551808+0000","last_became_active":"2023-10-09T06:37:08.547874+0000","last_became_peered":"2023-10-09T06:37:08.547874+0000","last_unstale":"2023-10-09T06:37:08.551808+0000","last_undegraded":"2023-10-09T06:37:08.551808+0000","last_fullsized":"2023-10-09T06:37:08.551808+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:04:07.309237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"112'1","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.773361+0000","last_change":"2023-10-09T06:37:07.762702+0000","last_active":"2023-10-09T06:37:08.773361+0000","last_peered":"2023-10-09T06:37:08.773361+0000","last_clean":"2023-10-09T06:37:08.773361+0000","last_became_active":"2023-10-09T06:37:07.762456+0000","last_became_peered":"2023-10-09T06:37:07.762456+0000","last_unstale":"2023-10-09T06:37:08.773361+0000","last_undegraded":"2023-10-09T06:37:08.773361+0000","last_fullsized":"2023-10-09T06:37:08.773361+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:25:31.399270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.556656+0000","last_change":"2023-10-09T06:37:08.551582+0000","last_active":"2023-10-09T06:37:08.556656+0000","last_peered":"2023-10-09T06:37:08.556656+0000","last_clean":"2023-10-09T06:37:08.556656+0000","last_became_active":"2023-10-09T06:37:08.551017+0000","last_became_peered":"2023-10-09T06:37:08.551017+0000","last_unstale":"2023-10-09T06:37:08.556656+0000","last_undegraded":"2023-10-09T06:37:08.556656+0000","last_fullsized":"2023-10-09T06:37:08.556656+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:10:45.977689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.545058+0000","last_change":"2023-10-09T06:37:07.757787+0000","last_active":"2023-10-09T06:37:08.545058+0000","last_peered":"2023-10-09T06:37:08.545058+0000","last_clean":"2023-10-09T06:37:08.545058+0000","last_became_active":"2023-10-09T06:37:07.757508+0000","last_became_peered":"2023-10-09T06:37:07.757508+0000","last_unstale":"2023-10-09T06:37:08.545058+0000","last_undegraded":"2023-10-09T06:37:08.545058+0000","last_fullsized":"2023-10-09T06:37:08.545058+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:30:15.484180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.560856+0000","last_change":"2023-10-09T06:37:08.556247+0000","last_active":"2023-10-09T06:37:08.560856+0000","last_peered":"2023-10-09T06:37:08.560856+0000","last_clean":"2023-10-09T06:37:08.560856+0000","last_became_active":"2023-10-09T06:37:08.555910+0000","last_became_peered":"2023-10-09T06:37:08.555910+0000","last_unstale":"2023-10-09T06:37:08.560856+0000","last_undegraded":"2023-10-09T06:37:08.560856+0000","last_fullsized":"2023-10-09T06:37:08.560856+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:29:06.291104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"112'2","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.772662+0000","last_change":"2023-10-09T06:37:07.764433+0000","last_active":"2023-10-09T06:37:08.772662+0000","last_peered":"2023-10-09T06:37:08.772662+0000","last_clean":"2023-10-09T06:37:08.772662+0000","last_became_active":"2023-10-09T06:37:07.764096+0000","last_became_peered":"2023-10-09T06:37:07.764096+0000","last_unstale":"2023-10-09T06:37:08.772662+0000","last_undegraded":"2023-10-09T06:37:08.772662+0000","last_fullsized":"2023-10-09T06:37:08.772662+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:22:08.809994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.551625+0000","last_change":"2023-10-09T06:37:08.541143+0000","last_active":"2023-10-09T06:37:08.551625+0000","last_peered":"2023-10-09T06:37:08.551625+0000","last_clean":"2023-10-09T06:37:08.551625+0000","last_became_active":"2023-10-09T06:37:08.540854+0000","last_became_peered":"2023-10-09T06:37:08.540854+0000","last_unstale":"2023-10-09T06:37:08.551625+0000","last_undegraded":"2023-10-09T06:37:08.551625+0000","last_fullsized":"2023-10-09T06:37:08.551625+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:59:57.832775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.537751+0000","last_change":"2023-10-09T06:37:07.759706+0000","last_active":"2023-10-09T06:37:08.537751+0000","last_peered":"2023-10-09T06:37:08.537751+0000","last_clean":"2023-10-09T06:37:08.537751+0000","last_became_active":"2023-10-09T06:37:07.759519+0000","last_became_peered":"2023-10-09T06:37:07.759519+0000","last_unstale":"2023-10-09T06:37:08.537751+0000","last_undegraded":"2023-10-09T06:37:08.537751+0000","last_fullsized":"2023-10-09T06:37:08.537751+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:49:01.460212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.559033+0000","last_change":"2023-10-09T06:37:08.555663+0000","last_active":"2023-10-09T06:37:08.559033+0000","last_peered":"2023-10-09T06:37:08.559033+0000","last_clean":"2023-10-09T06:37:08.559033+0000","last_became_active":"2023-10-09T06:37:08.555405+0000","last_became_peered":"2023-10-09T06:37:08.555405+0000","last_unstale":"2023-10-09T06:37:08.559033+0000","last_undegraded":"2023-10-09T06:37:08.559033+0000","last_fullsized":"2023-10-09T06:37:08.559033+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:23:20.895190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.537545+0000","last_change":"2023-10-09T06:37:07.758288+0000","last_active":"2023-10-09T06:37:08.537545+0000","last_peered":"2023-10-09T06:37:08.537545+0000","last_clean":"2023-10-09T06:37:08.537545+0000","last_became_active":"2023-10-09T06:37:07.758029+0000","last_became_peered":"2023-10-09T06:37:07.758029+0000","last_unstale":"2023-10-09T06:37:08.537545+0000","last_undegraded":"2023-10-09T06:37:08.537545+0000","last_fullsized":"2023-10-09T06:37:08.537545+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:20:05.321978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.556643+0000","last_change":"2023-10-09T06:37:08.554975+0000","last_active":"2023-10-09T06:37:08.556643+0000","last_peered":"2023-10-09T06:37:08.556643+0000","last_clean":"2023-10-09T06:37:08.556643+0000","last_became_active":"2023-10-09T06:37:08.554688+0000","last_became_peered":"2023-10-09T06:37:08.554688+0000","last_unstale":"2023-10-09T06:37:08.556643+0000","last_undegraded":"2023-10-09T06:37:08.556643+0000","last_fullsized":"2023-10-09T06:37:08.556643+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:38:27.210425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.540916+0000","last_change":"2023-10-09T06:37:07.762931+0000","last_active":"2023-10-09T06:37:08.540916+0000","last_peered":"2023-10-09T06:37:08.540916+0000","last_clean":"2023-10-09T06:37:08.540916+0000","last_became_active":"2023-10-09T06:37:07.762725+0000","last_became_peered":"2023-10-09T06:37:07.762725+0000","last_unstale":"2023-10-09T06:37:08.540916+0000","last_undegraded":"2023-10-09T06:37:08.540916+0000","last_fullsized":"2023-10-09T06:37:08.540916+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:18:53.098466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.553176+0000","last_change":"2023-10-09T06:37:08.548167+0000","last_active":"2023-10-09T06:37:08.553176+0000","last_peered":"2023-10-09T06:37:08.553176+0000","last_clean":"2023-10-09T06:37:08.553176+0000","last_became_active":"2023-10-09T06:37:08.547700+0000","last_became_peered":"2023-10-09T06:37:08.547700+0000","last_unstale":"2023-10-09T06:37:08.553176+0000","last_undegraded":"2023-10-09T06:37:08.553176+0000","last_fullsized":"2023-10-09T06:37:08.553176+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:41:55.509062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"112'2","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.771697+0000","last_change":"2023-10-09T06:37:07.763529+0000","last_active":"2023-10-09T06:37:08.771697+0000","last_peered":"2023-10-09T06:37:08.771697+0000","last_clean":"2023-10-09T06:37:08.771697+0000","last_became_active":"2023-10-09T06:37:07.763261+0000","last_became_peered":"2023-10-09T06:37:07.763261+0000","last_unstale":"2023-10-09T06:37:08.771697+0000","last_undegraded":"2023-10-09T06:37:08.771697+0000","last_fullsized":"2023-10-09T06:37:08.771697+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:40:11.133728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.554668+0000","last_change":"2023-10-09T06:37:08.550020+0000","last_active":"2023-10-09T06:37:08.554668+0000","last_peered":"2023-10-09T06:37:08.554668+0000","last_clean":"2023-10-09T06:37:08.554668+0000","last_became_active":"2023-10-09T06:37:08.549776+0000","last_became_peered":"2023-10-09T06:37:08.549776+0000","last_unstale":"2023-10-09T06:37:08.554668+0000","last_undegraded":"2023-10-09T06:37:08.554668+0000","last_fullsized":"2023-10-09T06:37:08.554668+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:46:32.727481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.543009+0000","last_change":"2023-10-09T06:37:07.764948+0000","last_active":"2023-10-09T06:37:08.543009+0000","last_peered":"2023-10-09T06:37:08.543009+0000","last_clean":"2023-10-09T06:37:08.543009+0000","last_became_active":"2023-10-09T06:37:07.764704+0000","last_became_peered":"2023-10-09T06:37:07.764704+0000","last_unstale":"2023-10-09T06:37:08.543009+0000","last_undegraded":"2023-10-09T06:37:08.543009+0000","last_fullsized":"2023-10-09T06:37:08.543009+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:23:19.891326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.554568+0000","last_change":"2023-10-09T06:37:08.546976+0000","last_active":"2023-10-09T06:37:08.554568+0000","last_peered":"2023-10-09T06:37:08.554568+0000","last_clean":"2023-10-09T06:37:08.554568+0000","last_became_active":"2023-10-09T06:37:08.546709+0000","last_became_peered":"2023-10-09T06:37:08.546709+0000","last_unstale":"2023-10-09T06:37:08.554568+0000","last_undegraded":"2023-10-09T06:37:08.554568+0000","last_fullsized":"2023-10-09T06:37:08.554568+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:40:16.413666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.540575+0000","last_change":"2023-10-09T06:37:07.764780+0000","last_active":"2023-10-09T06:37:08.540575+0000","last_peered":"2023-10-09T06:37:08.540575+0000","last_clean":"2023-10-09T06:37:08.540575+0000","last_became_active":"2023-10-09T06:37:07.764492+0000","last_became_peered":"2023-10-09T06:37:07.764492+0000","last_unstale":"2023-10-09T06:37:08.540575+0000","last_undegraded":"2023-10-09T06:37:08.540575+0000","last_fullsized":"2023-10-09T06:37:08.540575+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:47:44.879129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.553298+0000","last_change":"2023-10-09T06:37:08.548844+0000","last_active":"2023-10-09T06:37:08.553298+0000","last_peered":"2023-10-09T06:37:08.553298+0000","last_clean":"2023-10-09T06:37:08.553298+0000","last_became_active":"2023-10-09T06:37:08.548577+0000","last_became_peered":"2023-10-09T06:37:08.548577+0000","last_unstale":"2023-10-09T06:37:08.553298+0000","last_undegraded":"2023-10-09T06:37:08.553298+0000","last_fullsized":"2023-10-09T06:37:08.553298+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:38:42.567166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.540655+0000","last_change":"2023-10-09T06:37:07.761330+0000","last_active":"2023-10-09T06:37:08.540655+0000","last_peered":"2023-10-09T06:37:08.540655+0000","last_clean":"2023-10-09T06:37:08.540655+0000","last_became_active":"2023-10-09T06:37:07.761051+0000","last_became_peered":"2023-10-09T06:37:07.761051+0000","last_unstale":"2023-10-09T06:37:08.540655+0000","last_undegraded":"2023-10-09T06:37:08.540655+0000","last_fullsized":"2023-10-09T06:37:08.540655+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:38:26.206561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.556451+0000","last_change":"2023-10-09T06:37:08.553941+0000","last_active":"2023-10-09T06:37:08.556451+0000","last_peered":"2023-10-09T06:37:08.556451+0000","last_clean":"2023-10-09T06:37:08.556451+0000","last_became_active":"2023-10-09T06:37:08.553662+0000","last_became_peered":"2023-10-09T06:37:08.553662+0000","last_unstale":"2023-10-09T06:37:08.556451+0000","last_undegraded":"2023-10-09T06:37:08.556451+0000","last_fullsized":"2023-10-09T06:37:08.556451+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:07:45.944631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.539842+0000","last_change":"2023-10-09T06:37:07.765790+0000","last_active":"2023-10-09T06:37:08.539842+0000","last_peered":"2023-10-09T06:37:08.539842+0000","last_clean":"2023-10-09T06:37:08.539842+0000","last_became_active":"2023-10-09T06:37:07.765555+0000","last_became_peered":"2023-10-09T06:37:07.765555+0000","last_unstale":"2023-10-09T06:37:08.539842+0000","last_undegraded":"2023-10-09T06:37:08.539842+0000","last_fullsized":"2023-10-09T06:37:08.539842+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:36:48.859422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.552850+0000","last_change":"2023-10-09T06:37:08.550852+0000","last_active":"2023-10-09T06:37:08.552850+0000","last_peered":"2023-10-09T06:37:08.552850+0000","last_clean":"2023-10-09T06:37:08.552850+0000","last_became_active":"2023-10-09T06:37:08.550574+0000","last_became_peered":"2023-10-09T06:37:08.550574+0000","last_unstale":"2023-10-09T06:37:08.552850+0000","last_undegraded":"2023-10-09T06:37:08.552850+0000","last_fullsized":"2023-10-09T06:37:08.552850+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:25:32.403134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.541516+0000","last_change":"2023-10-09T06:37:07.765156+0000","last_active":"2023-10-09T06:37:08.541516+0000","last_peered":"2023-10-09T06:37:08.541516+0000","last_clean":"2023-10-09T06:37:08.541516+0000","last_became_active":"2023-10-09T06:37:07.764683+0000","last_became_peered":"2023-10-09T06:37:07.764683+0000","last_unstale":"2023-10-09T06:37:08.541516+0000","last_undegraded":"2023-10-09T06:37:08.541516+0000","last_fullsized":"2023-10-09T06:37:08.541516+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:04:39.436580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.549018+0000","last_change":"2023-10-09T06:37:08.542620+0000","last_active":"2023-10-09T06:37:08.549018+0000","last_peered":"2023-10-09T06:37:08.549018+0000","last_clean":"2023-10-09T06:37:08.549018+0000","last_became_active":"2023-10-09T06:37:08.542363+0000","last_became_peered":"2023-10-09T06:37:08.542363+0000","last_unstale":"2023-10-09T06:37:08.549018+0000","last_undegraded":"2023-10-09T06:37:08.549018+0000","last_fullsized":"2023-10-09T06:37:08.549018+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:38:29.964205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.537684+0000","last_change":"2023-10-09T06:37:07.759280+0000","last_active":"2023-10-09T06:37:08.537684+0000","last_peered":"2023-10-09T06:37:08.537684+0000","last_clean":"2023-10-09T06:37:08.537684+0000","last_became_active":"2023-10-09T06:37:07.759028+0000","last_became_peered":"2023-10-09T06:37:07.759028+0000","last_unstale":"2023-10-09T06:37:08.537684+0000","last_undegraded":"2023-10-09T06:37:08.537684+0000","last_fullsized":"2023-10-09T06:37:08.537684+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:34:00.199866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.554549+0000","last_change":"2023-10-09T06:37:08.547159+0000","last_active":"2023-10-09T06:37:08.554549+0000","last_peered":"2023-10-09T06:37:08.554549+0000","last_clean":"2023-10-09T06:37:08.554549+0000","last_became_active":"2023-10-09T06:37:08.546942+0000","last_became_peered":"2023-10-09T06:37:08.546942+0000","last_unstale":"2023-10-09T06:37:08.554549+0000","last_undegraded":"2023-10-09T06:37:08.554549+0000","last_fullsized":"2023-10-09T06:37:08.554549+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:38:20.913074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.540990+0000","last_change":"2023-10-09T06:37:07.756405+0000","last_active":"2023-10-09T06:37:08.540990+0000","last_peered":"2023-10-09T06:37:08.540990+0000","last_clean":"2023-10-09T06:37:08.540990+0000","last_became_active":"2023-10-09T06:37:07.756138+0000","last_became_peered":"2023-10-09T06:37:07.756138+0000","last_unstale":"2023-10-09T06:37:08.540990+0000","last_undegraded":"2023-10-09T06:37:08.540990+0000","last_fullsized":"2023-10-09T06:37:08.540990+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:41:54.852777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.556475+0000","last_change":"2023-10-09T06:37:08.553781+0000","last_active":"2023-10-09T06:37:08.556475+0000","last_peered":"2023-10-09T06:37:08.556475+0000","last_clean":"2023-10-09T06:37:08.556475+0000","last_became_active":"2023-10-09T06:37:08.553466+0000","last_became_peered":"2023-10-09T06:37:08.553466+0000","last_unstale":"2023-10-09T06:37:08.556475+0000","last_undegraded":"2023-10-09T06:37:08.556475+0000","last_fullsized":"2023-10-09T06:37:08.556475+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:07.742551+0000","last_clean_scrub_stamp":"2023-10-09T06:37:07.742551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:04:07.309237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"112'2","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.773385+0000","last_change":"2023-10-09T06:37:07.765120+0000","last_active":"2023-10-09T06:37:08.773385+0000","last_peered":"2023-10-09T06:37:08.773385+0000","last_clean":"2023-10-09T06:37:08.773385+0000","last_became_active":"2023-10-09T06:37:07.764763+0000","last_became_peered":"2023-10-09T06:37:07.764763+0000","last_unstale":"2023-10-09T06:37:08.773385+0000","last_undegraded":"2023-10-09T06:37:08.773385+0000","last_fullsized":"2023-10-09T06:37:08.773385+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:06.738687+0000","last_clean_scrub_stamp":"2023-10-09T06:37:06.738687+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:53:18.927143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":240,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-09T06:37:08.541423+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:37:08.541423+0000","last_peered":"2023-10-09T06:37:08.541423+0000","last_clean":"2023-10-09T06:37:08.541423+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:37:08.541423+0000","last_undegraded":"2023-10-09T06:37:08.541423+0000","last_fullsized":"2023-10-09T06:37:08.541423+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574932,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303368,"kb_used_data":1984,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068472,"statfs":{"total":96636764160,"available":96326115328,"internally_reserved":0,"allocated":2031616,"data_stored":1205425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51100000000000001,"15min":0.505},"min":{"1min":0.33400000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68000000000000005,"5min":0.753,"15min":0.753},"last":0.57099999999999995}]},{"osd":1,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.50600000000000001,"15min":0.52000000000000002},"min":{"1min":0.38900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":1.2030000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33800000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.64000000000000001,"5min":0.70599999999999996,"15min":0.71999999999999997},"last":0.48599999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52800000000000002,"15min":0.51200000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72899999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52700000000000002,"15min":0.54400000000000004},"min":{"1min":0.38200000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.83099999999999996},"last":0.55300000000000005}]},{"osd":3,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53100000000000003,"15min":0.55100000000000005},"min":{"1min":0.35899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.627},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.55600000000000005,"15min":0.59499999999999997},"min":{"1min":0.46600000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.73599999999999999,"5min":0.82099999999999995,"15min":1.1479999999999999},"last":0.65400000000000003}]},{"osd":4,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46899999999999997,"15min":0.51900000000000002},"min":{"1min":0.30299999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.63700000000000001,"5min":0.69999999999999996,"15min":0.93999999999999995},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56000000000000005,"15min":0.60799999999999998},"min":{"1min":0.35399999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.68999999999999995}]},{"osd":5,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.57199999999999995},"min":{"1min":0.38800000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.70899999999999996,"5min":0.85799999999999998,"15min":0.91100000000000003},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.58499999999999996,"15min":0.62},"min":{"1min":0.437,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.67100000000000004}]},{"osd":6,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.54600000000000004,"15min":0.59599999999999997},"min":{"1min":0.40200000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.88600000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54500000000000004,"15min":0.61499999999999999},"min":{"1min":0.39700000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.77800000000000002,"5min":0.79900000000000004,"15min":0.84599999999999997},"last":0.71899999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574931,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302776,"kb_used_data":1392,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069064,"statfs":{"total":96636764160,"available":96326721536,"internally_reserved":0,"allocated":1425408,"data_stored":614447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.502,"15min":0.503},"min":{"1min":0.36399999999999999,"5min":0.32000000000000001,"15min":0.30399999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.503,"5min":0.50800000000000001,"15min":0.51000000000000001},"min":{"1min":0.36299999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.60499999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.495,"15min":0.51300000000000001},"min":{"1min":0.35999999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.498},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52400000000000002,"15min":0.53100000000000003},"min":{"1min":0.36199999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.73199999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51500000000000001,"15min":0.51700000000000002},"min":{"1min":0.41599999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51000000000000001,"15min":0.53200000000000003},"min":{"1min":0.36099999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.628,"5min":0.69399999999999995,"15min":0.752},"last":0.64700000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53200000000000003,"15min":0.54700000000000004},"min":{"1min":0.39300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54200000000000004,"15min":0.70099999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69199999999999995,"5min":0.74399999999999999,"15min":4.7510000000000003},"last":0.70799999999999996}]},{"osd":4,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46400000000000002,"15min":0.47899999999999998},"min":{"1min":0.28199999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.76400000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56599999999999995,"15min":0.58299999999999996},"min":{"1min":0.29499999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.80800000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.54900000000000004}]},{"osd":5,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.53300000000000003},"min":{"1min":0.318,"5min":0.246,"15min":0.246},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.81200000000000006},"last":0.372},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57099999999999995,"15min":0.59699999999999998},"min":{"1min":0.378,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70799999999999996,"5min":0.78000000000000003,"15min":0.86599999999999999},"last":0.69499999999999995}]},{"osd":7,"last update":"Mon Oct 9 06:36:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55200000000000005,"15min":0.58599999999999997},"min":{"1min":0.35199999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71299999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.41899999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.73599999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.627}]}]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302764,"kb_used_data":1380,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069076,"statfs":{"total":96636764160,"available":96326733824,"internally_reserved":0,"allocated":1413120,"data_stored":612911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.31,"15min":0.31},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34599999999999997,"15min":0.35899999999999999},"min":{"1min":0.26300000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.318}]},{"osd":2,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.34699999999999998,"15min":0.35299999999999998},"min":{"1min":0.24299999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38300000000000001,"15min":0.38600000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.57199999999999995,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.36399999999999999}]},{"osd":3,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.36599999999999999,"15min":0.36399999999999999},"min":{"1min":0.27300000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52300000000000002,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.34999999999999998,"15min":0.55800000000000005},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":5.8529999999999998},"last":0.34799999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58099999999999996,"15min":0.57599999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77800000000000002,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.874,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.56499999999999995}]},{"osd":5,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56999999999999995,"15min":0.76100000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83099999999999996,"5min":0.83799999999999997,"15min":5.9459999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.60399999999999998,"15min":0.61299999999999999},"min":{"1min":0.41899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85699999999999998,"5min":0.92700000000000005,"15min":0.93999999999999995},"last":0.57599999999999996}]},{"osd":6,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.54900000000000004,"15min":0.73299999999999998},"min":{"1min":0.38,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":5.9160000000000004},"last":0.496},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83299999999999996,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.54200000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56799999999999995,"15min":0.58899999999999997},"min":{"1min":0.39300000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.81599999999999995,"5min":0.89500000000000002,"15min":0.97099999999999997},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56399999999999995,"15min":0.77000000000000002},"min":{"1min":0.40400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.71899999999999997,"5min":0.80900000000000005,"15min":5.9729999999999999},"last":0.58599999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574933,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303344,"kb_used_data":1960,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068496,"statfs":{"total":96636764160,"available":96326139904,"internally_reserved":0,"allocated":2007040,"data_stored":1203279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.30499999999999999,"15min":0.316},"min":{"1min":0.22600000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.49399999999999999,"5min":0.498,"15min":0.498},"last":0.221},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.33200000000000002,"15min":0.34499999999999997},"min":{"1min":0.25,"5min":0.214,"15min":0.214},"max":{"1min":0.48599999999999999,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.28499999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.35199999999999998,"15min":0.36299999999999999},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.314},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.379,"15min":0.38800000000000001},"min":{"1min":0.29499999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.69399999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.33500000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.33400000000000002,"15min":0.39000000000000001},"min":{"1min":0.246,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.39700000000000002,"5min":0.52900000000000003,"15min":1.554},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.36599999999999999,"15min":0.40000000000000002},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.40899999999999997,"5min":0.60899999999999999,"15min":0.79000000000000004},"last":0.35399999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.55000000000000004,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70199999999999996,"5min":1.212,"15min":1.212},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55300000000000005,"15min":0.73899999999999999},"min":{"1min":0.37,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71999999999999997,"5min":0.82499999999999996,"15min":5.484},"last":0.56799999999999995}]},{"osd":5,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55500000000000005,"15min":0.74299999999999999},"min":{"1min":0.39600000000000002,"5min":0.35799999999999998,"15min":0.35499999999999998},"max":{"1min":0.69299999999999995,"5min":0.84399999999999997,"15min":5.5449999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56899999999999995,"15min":0.57499999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.80900000000000005,"5min":0.98699999999999999,"15min":1.113},"last":0.59699999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58999999999999997,"15min":0.60599999999999998},"min":{"1min":0.44400000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.83999999999999997,"5min":1.0269999999999999,"15min":1.0529999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.45700000000000002,"5min":0.41799999999999998,"15min":0.41599999999999998},"max":{"1min":0.755,"5min":1.169,"15min":1.169},"last":0.55500000000000005}]},{"osd":7,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58399999999999996,"15min":0.58999999999999997},"min":{"1min":0.42899999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.76900000000000002,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56599999999999995,"15min":0.76900000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81599999999999995,"5min":0.873,"15min":5.5800000000000001},"last":0.63400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574934,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302772,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069068,"statfs":{"total":96636764160,"available":96326725632,"internally_reserved":0,"allocated":1421312,"data_stored":613499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.309,"15min":0.35499999999999998},"min":{"1min":0.21199999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.441,"5min":0.50800000000000001,"15min":1.4199999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.32200000000000001,"15min":0.58399999999999996},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.48299999999999998,"5min":0.53800000000000003,"15min":7.2699999999999996},"last":0.28899999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32100000000000001,"15min":0.34699999999999998},"min":{"1min":0.22700000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.41199999999999998,"5min":0.497,"15min":0.65400000000000003},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.371,"15min":0.63900000000000001},"min":{"1min":0.29599999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.51500000000000001,"5min":0.57599999999999996,"15min":7.327},"last":0.33900000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36899999999999999,"15min":0.39100000000000001},"min":{"1min":0.28199999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.48399999999999999,"5min":0.52900000000000003,"15min":0.68600000000000005},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36699999999999999,"15min":0.39300000000000002},"min":{"1min":0.26500000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.51400000000000001,"5min":0.54000000000000004,"15min":0.72699999999999998},"last":0.38600000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.34999999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.79700000000000004,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55300000000000005,"15min":0.80800000000000005},"min":{"1min":0.38600000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72699999999999998,"5min":0.85399999999999998,"15min":7.2389999999999999},"last":0.503}]},{"osd":5,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58799999999999997,"15min":0.95299999999999996},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72499999999999998,"5min":0.83799999999999997,"15min":10.890000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59099999999999997,"15min":0.60199999999999998},"min":{"1min":0.41499999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.55800000000000005}]},{"osd":6,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56799999999999995,"15min":0.81599999999999995},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76700000000000002,"5min":0.89100000000000001,"15min":7.2999999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56999999999999995,"15min":0.57199999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.57299999999999995}]},{"osd":7,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59399999999999997,"15min":0.84699999999999998},"min":{"1min":0.44600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.78800000000000003,"5min":0.87,"15min":7.1879999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60999999999999999,"15min":0.626},"min":{"1min":0.42699999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.90500000000000003},"last":0.52500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574933,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302772,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069068,"statfs":{"total":96636764160,"available":96326725632,"internally_reserved":0,"allocated":1421312,"data_stored":613071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.36099999999999999,"15min":0.32900000000000001},"min":{"1min":0.223,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.46200000000000002,"5min":5.8700000000000001,"15min":5.8700000000000001},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.38500000000000001,"15min":0.36599999999999999},"min":{"1min":0.23400000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.502,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.36799999999999999}]},{"osd":1,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.379,"15min":0.35799999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.51800000000000002,"5min":4.0469999999999997,"15min":4.0469999999999997},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.40300000000000002,"15min":0.38800000000000001},"min":{"1min":0.28399999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.52300000000000002,"5min":4.1509999999999998,"15min":4.1509999999999998},"last":0.31900000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.374},"min":{"1min":0.26800000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.56799999999999995,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38900000000000001,"15min":0.38500000000000001},"min":{"1min":0.26800000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.54000000000000004,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.34499999999999997}]},{"osd":4,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58199999999999996,"15min":0.57099999999999995},"min":{"1min":0.40600000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.74199999999999999,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.42199999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77400000000000002,"5min":3.0710000000000002,"15min":3.0710000000000002},"last":0.65700000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.61599999999999999,"15min":0.58799999999999997},"min":{"1min":0.45700000000000002,"5min":0.34599999999999997,"15min":0.32700000000000001},"max":{"1min":0.69899999999999995,"5min":6.2300000000000004,"15min":6.2300000000000004},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61499999999999999,"15min":0.59699999999999998},"min":{"1min":0.495,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79700000000000004,"5min":3.0219999999999998,"15min":3.0219999999999998},"last":0.58699999999999997}]},{"osd":6,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61599999999999999,"15min":0.59199999999999997},"min":{"1min":0.48199999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.78300000000000003,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68899999999999995,"15min":0.65100000000000002},"min":{"1min":0.46700000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.82499999999999996,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.79400000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64100000000000001,"15min":0.61299999999999999},"min":{"1min":0.44900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85399999999999998,"5min":4.2629999999999999,"15min":4.2629999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.65500000000000003,"15min":0.61299999999999999},"min":{"1min":0.50800000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.77400000000000002,"5min":7.3650000000000002,"15min":7.3650000000000002},"last":0.71699999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302768,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069072,"statfs":{"total":96636764160,"available":96326729728,"internally_reserved":0,"allocated":1417216,"data_stored":612981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.51400000000000001,"15min":0.55400000000000005},"min":{"1min":0.28100000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.65100000000000002,"5min":0.78800000000000003,"15min":1.083},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.52600000000000002,"15min":0.51700000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.67800000000000005,"5min":0.97799999999999998,"15min":1.054},"last":0.54700000000000004}]},{"osd":1,"last update":"Mon Oct 9 06:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.71399999999999997,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.50600000000000001,"15min":0.49399999999999999},"min":{"1min":0.31900000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.67100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.45800000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52100000000000002,"15min":0.54100000000000004},"min":{"1min":0.35999999999999999,"5min":0.33700000000000002,"15min":0.32900000000000001},"max":{"1min":0.69399999999999995,"5min":0.82099999999999995,"15min":1.0249999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55500000000000005,"15min":0.59299999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.68799999999999994,"5min":0.78300000000000003,"15min":0.93799999999999994},"last":0.496}]},{"osd":3,"last update":"Mon Oct 9 06:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.52600000000000002,"15min":0.55200000000000005},"min":{"1min":0.32200000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70199999999999996,"5min":0.77500000000000002,"15min":0.98499999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56999999999999995,"15min":0.58899999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73599999999999999,"5min":0.86199999999999999,"15min":1.1319999999999999},"last":0.52900000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51300000000000001,"15min":0.55700000000000005},"min":{"1min":0.30399999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.74099999999999999,"5min":0.78000000000000003,"15min":0.998},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.60999999999999999},"min":{"1min":0.377,"5min":0.36299999999999999,"15min":0.35099999999999998},"max":{"1min":0.70999999999999996,"5min":0.80000000000000004,"15min":0.97999999999999998},"last":0.51700000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53900000000000003,"15min":0.56899999999999995},"min":{"1min":0.34699999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.80200000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56699999999999995,"15min":0.59999999999999998},"min":{"1min":0.38700000000000001,"5min":0.33500000000000002,"15min":0.32700000000000001},"max":{"1min":0.77900000000000003,"5min":1.01,"15min":1.01},"last":0.57799999999999996}]},{"osd":7,"last update":"Mon Oct 9 06:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54000000000000004,"15min":0.57199999999999995},"min":{"1min":0.35599999999999998,"5min":0.33600000000000002,"15min":0.28299999999999997},"max":{"1min":0.753,"5min":0.85599999999999998,"15min":0.90000000000000002},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.59799999999999998,"15min":0.64800000000000002},"min":{"1min":0.34999999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79500000000000004,"5min":0.98999999999999999,"15min":1.175},"last":0.59699999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574932,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298676,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073164,"statfs":{"total":96636764160,"available":96330919936,"internally_reserved":0,"allocated":1421312,"data_stored":613023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.496,"15min":0.50700000000000001},"min":{"1min":0.314,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.64100000000000001,"5min":0.71399999999999997,"15min":0.86599999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.51900000000000002,"15min":0.51400000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.42499999999999999}]},{"osd":1,"last update":"Mon Oct 9 06:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.50800000000000001,"15min":0.51000000000000001},"min":{"1min":0.34899999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.61699999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.52100000000000002,"15min":0.53100000000000003},"min":{"1min":0.30599999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73899999999999999,"5min":0.754,"15min":0.81599999999999995},"last":0.44400000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.33100000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.70699999999999996,"5min":0.73299999999999998,"15min":0.83999999999999997},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57899999999999996,"15min":0.58699999999999997},"min":{"1min":0.46500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72799999999999998,"5min":0.745,"15min":0.79900000000000004},"last":0.63100000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.41099999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.69799999999999995,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.503},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55500000000000005,"15min":0.56399999999999995},"min":{"1min":0.39800000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.71099999999999997,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.58899999999999997}]},{"osd":4,"last update":"Mon Oct 9 06:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.495,"15min":0.496},"min":{"1min":0.29999999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54200000000000004,"15min":0.54500000000000004},"min":{"1min":0.35299999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.72299999999999998,"5min":0.76300000000000001,"15min":0.80700000000000005},"last":0.60699999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55600000000000005,"15min":0.53400000000000003},"min":{"1min":0.376,"5min":0.35699999999999998,"15min":0.32600000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.83199999999999996},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.42599999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.62}]},{"osd":7,"last update":"Mon Oct 9 06:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.39600000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76900000000000002,"5min":0.84399999999999997,"15min":0.85599999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.442,"5min":0.35199999999999998,"15min":0.314},"max":{"1min":0.81399999999999995,"5min":0.84499999999999997,"15min":0.90600000000000003},"last":0.69699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:37:13.202 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-09T06:37:13.203 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-09T06:37:13.357 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.356+0000 7f1854e72700 1 Processor -- start 2023-10-09T06:37:13.357 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.356+0000 7f1854e72700 1 -- start start 2023-10-09T06:37:13.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.357+0000 7f1854e72700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f18500cbb90 0x7f18500af320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:13.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.357+0000 7f1854e72700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18500af860 con 0x7f1850159950 2023-10-09T06:37:13.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.357+0000 7f1854e72700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18500ce380 con 0x7f1850069c70 2023-10-09T06:37:13.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.357+0000 7f1854e72700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18500cec20 con 0x7f18500cbb90 2023-10-09T06:37:13.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.357+0000 7f184e59c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f1850159950 0x7f1850069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:54634/0 (socket says 172.21.15.163:54634) 2023-10-09T06:37:13.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.357+0000 7f184e59c700 1 -- 172.21.15.163:0/735494033 learned_addr learned my addr 172.21.15.163:0/735494033 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:13.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.358+0000 7f184ed9d700 1 --2- 172.21.15.163:0/735494033 >> v2:172.21.15.181:3300/0 conn(0x7f18500cbb90 0x7f18500af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:13.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.358+0000 7f184d59a700 1 -- 172.21.15.163:0/735494033 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1712741633 0 0) 0x7f18500af860 con 0x7f1850159950 2023-10-09T06:37:13.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.358+0000 7f184d59a700 1 -- 172.21.15.163:0/735494033 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f183c003570 con 0x7f1850159950 2023-10-09T06:37:13.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.358+0000 7f184d59a700 1 -- 172.21.15.163:0/735494033 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1414020815 0 0) 0x7f183c003570 con 0x7f1850159950 2023-10-09T06:37:13.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.358+0000 7f184d59a700 1 -- 172.21.15.163:0/735494033 >> v2:172.21.15.181:3300/0 conn(0x7f18500cbb90 msgr2=0x7f18500af320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:13.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.358+0000 7f184d59a700 1 --2- 172.21.15.163:0/735494033 >> v2:172.21.15.181:3300/0 conn(0x7f18500cbb90 0x7f18500af320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:13.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.358+0000 7f184d59a700 1 -- 172.21.15.163:0/735494033 >> v1:172.21.15.181:6789/0 conn(0x7f1850069c70 legacy=0x7f18500cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:13.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.358+0000 7f184d59a700 1 -- 172.21.15.163:0/735494033 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18500cbfb0 con 0x7f1850159950 2023-10-09T06:37:13.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.358+0000 7f184d59a700 1 -- 172.21.15.163:0/735494033 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f18400017f0 con 0x7f1850159950 2023-10-09T06:37:13.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.358+0000 7f184d59a700 1 -- 172.21.15.163:0/735494033 >> v1:172.21.15.163:6789/0 conn(0x7f1850159950 legacy=0x7f1850069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:13.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.358+0000 7f184d59a700 1 --2- 172.21.15.163:0/735494033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f183c003a60 0x7f183c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:13.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.358+0000 7f184d59a700 1 --2- 172.21.15.163:0/735494033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f183c004810 0x7f183c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:13.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.358+0000 7f184d59a700 1 --2- 172.21.15.163:0/735494033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f183c0051c0 0x7f183c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:13.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.358+0000 7f184d59a700 1 -- 172.21.15.163:0/735494033 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18500cec20 con 0x7f183c0051c0 2023-10-09T06:37:13.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.358+0000 7f184d59a700 1 -- 172.21.15.163:0/735494033 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183c00c160 con 0x7f183c003a60 2023-10-09T06:37:13.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.358+0000 7f184d59a700 1 -- 172.21.15.163:0/735494033 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f183c00c320 con 0x7f183c004810 2023-10-09T06:37:13.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.358+0000 7f184ed9d700 1 --2- 172.21.15.163:0/735494033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f183c0051c0 0x7f183c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:13.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.358+0000 7f184e59c700 1 --2- 172.21.15.163:0/735494033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f183c003a60 0x7f183c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:13.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.358+0000 7f184dd9b700 1 --2- 172.21.15.163:0/735494033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f183c004810 0x7f183c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:13.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.358+0000 7f184ed9d700 1 -- 172.21.15.163:0/735494033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f183c004810 msgr2=0x7f183c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:13.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.358+0000 7f184ed9d700 1 --2- 172.21.15.163:0/735494033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f183c004810 0x7f183c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:13.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.358+0000 7f184ed9d700 1 -- 172.21.15.163:0/735494033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f183c003a60 msgr2=0x7f183c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:13.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.358+0000 7f184ed9d700 1 --2- 172.21.15.163:0/735494033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f183c003a60 0x7f183c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:13.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.358+0000 7f184ed9d700 1 -- 172.21.15.163:0/735494033 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f183c003570 con 0x7f183c0051c0 2023-10-09T06:37:13.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.359+0000 7f184ed9d700 1 -- 172.21.15.163:0/735494033 reap_dead start 2023-10-09T06:37:13.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.359+0000 7f184ed9d700 1 --2- 172.21.15.163:0/735494033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f183c0051c0 0x7f183c00ba60 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f1848009c00 tx=0x7f18480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5643aa358fdb72 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:13.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.359+0000 7f184d59a700 1 -- 172.21.15.163:0/735494033 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184800e710 con 0x7f183c0051c0 2023-10-09T06:37:13.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.359+0000 7f184d59a700 1 -- 172.21.15.163:0/735494033 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f184800e890 con 0x7f183c0051c0 2023-10-09T06:37:13.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.359+0000 7f184d59a700 1 -- 172.21.15.163:0/735494033 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184800ec50 con 0x7f183c0051c0 2023-10-09T06:37:13.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.359+0000 7f1854e72700 1 -- 172.21.15.163:0/735494033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f183c0051c0 msgr2=0x7f183c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:13.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.359+0000 7f1854e72700 1 --2- 172.21.15.163:0/735494033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f183c0051c0 0x7f183c00ba60 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f1848009c00 tx=0x7f18480091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:13.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.359+0000 7f1854e72700 1 -- 172.21.15.163:0/735494033 shutdown_connections 2023-10-09T06:37:13.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.359+0000 7f1854e72700 1 --2- 172.21.15.163:0/735494033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f183c0051c0 0x7f183c00ba60 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:13.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.359+0000 7f1854e72700 1 -- 172.21.15.163:0/735494033 >> 172.21.15.163:0/735494033 conn(0x7f1850068e00 msgr2=0x7f18500d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:13.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.359+0000 7f1854e72700 1 -- 172.21.15.163:0/735494033 shutdown_connections 2023-10-09T06:37:13.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.359+0000 7f1854e72700 1 -- 172.21.15.163:0/735494033 wait complete. 2023-10-09T06:37:13.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.361+0000 7f1854e72700 1 Processor -- start 2023-10-09T06:37:13.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.361+0000 7f1854e72700 1 -- start start 2023-10-09T06:37:13.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.361+0000 7f1854e72700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f183c0051c0 0x7f1850156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:13.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.361+0000 7f1854e72700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f185015c1c0 0x7f185015c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:13.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.361+0000 7f1854e72700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18500c7b30 0x7f18500c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:13.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.361+0000 7f1854e72700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185015db80 con 0x7f185015c1c0 2023-10-09T06:37:13.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.361+0000 7f1854e72700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185015cf70 con 0x7f18500c7b30 2023-10-09T06:37:13.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.361+0000 7f1854e72700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f185015d250 con 0x7f183c0051c0 2023-10-09T06:37:13.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.361+0000 7f184dd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f185015c1c0 0x7f185015c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:13.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.361+0000 7f184dd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f185015c1c0 0x7f185015c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56836/0 (socket says 172.21.15.163:56836) 2023-10-09T06:37:13.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.361+0000 7f184dd9b700 1 -- 172.21.15.163:0/2194479350 learned_addr learned my addr 172.21.15.163:0/2194479350 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:13.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.361+0000 7f184e59c700 1 --2- 172.21.15.163:0/2194479350 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f183c0051c0 0x7f1850156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:13.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.361+0000 7f184ed9d700 1 --2- 172.21.15.163:0/2194479350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18500c7b30 0x7f18500c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:13.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.362+0000 7f184dd9b700 1 -- 172.21.15.163:0/2194479350 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f183c0051c0 msgr2=0x7f1850156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:13.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.362+0000 7f184dd9b700 1 --2- 172.21.15.163:0/2194479350 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f183c0051c0 0x7f1850156c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:13.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.362+0000 7f184dd9b700 1 -- 172.21.15.163:0/2194479350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18500c7b30 msgr2=0x7f18500c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:13.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.362+0000 7f184dd9b700 1 --2- 172.21.15.163:0/2194479350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18500c7b30 0x7f18500c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:13.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.362+0000 7f184dd9b700 1 -- 172.21.15.163:0/2194479350 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185015d530 con 0x7f185015c1c0 2023-10-09T06:37:13.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.362+0000 7f184dd9b700 1 --2- 172.21.15.163:0/2194479350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f185015c1c0 0x7f185015c630 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f1838002950 tx=0x7f1838007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:13.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.362+0000 7f18467fc700 1 -- 172.21.15.163:0/2194479350 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1838013600 con 0x7f185015c1c0 2023-10-09T06:37:13.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.362+0000 7f18467fc700 1 -- 172.21.15.163:0/2194479350 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1838013760 con 0x7f185015c1c0 2023-10-09T06:37:13.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.362+0000 7f18467fc700 1 -- 172.21.15.163:0/2194479350 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1838013b00 con 0x7f185015c1c0 2023-10-09T06:37:13.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.362+0000 7f1854e72700 1 -- 172.21.15.163:0/2194479350 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f185015d810 con 0x7f185015c1c0 2023-10-09T06:37:13.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.362+0000 7f1854e72700 1 -- 172.21.15.163:0/2194479350 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f185015cb70 con 0x7f185015c1c0 2023-10-09T06:37:13.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.363+0000 7f1854e72700 1 -- 172.21.15.163:0/2194479350 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18500718b0 con 0x7f185015c1c0 2023-10-09T06:37:13.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.363+0000 7f18467fc700 1 -- 172.21.15.163:0/2194479350 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f183801c400 con 0x7f185015c1c0 2023-10-09T06:37:13.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.364+0000 7f18467fc700 1 --2- 172.21.15.163:0/2194479350 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1830067320 0x7f18300697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:13.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.364+0000 7f18467fc700 1 -- 172.21.15.163:0/2194479350 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1838084f30 con 0x7f185015c1c0 2023-10-09T06:37:13.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.364+0000 7f184e59c700 1 --2- 172.21.15.163:0/2194479350 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1830067320 0x7f18300697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:13.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.364+0000 7f184e59c700 1 --2- 172.21.15.163:0/2194479350 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1830067320 0x7f18300697d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1840001920 tx=0x7f1840006040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:13.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.368+0000 7f18467fc700 1 -- 172.21.15.163:0/2194479350 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1838051c80 con 0x7f185015c1c0 2023-10-09T06:37:13.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.624+0000 7f1854e72700 1 -- 172.21.15.163:0/2194479350 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f18500c7910 con 0x7f185015c1c0 2023-10-09T06:37:13.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.630+0000 7f18467fc700 1 -- 172.21.15.163:0/2194479350 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+0 (secure 0 0 0) 0x7f18380552a0 con 0x7f185015c1c0 2023-10-09T06:37:13.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.632+0000 7f1854e72700 1 -- 172.21.15.163:0/2194479350 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1830067320 msgr2=0x7f18300697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:13.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.632+0000 7f1854e72700 1 --2- 172.21.15.163:0/2194479350 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1830067320 0x7f18300697d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1840001920 tx=0x7f1840006040 comp rx=0 tx=0).stop 2023-10-09T06:37:13.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.632+0000 7f1854e72700 1 -- 172.21.15.163:0/2194479350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f185015c1c0 msgr2=0x7f185015c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:13.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.632+0000 7f1854e72700 1 --2- 172.21.15.163:0/2194479350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f185015c1c0 0x7f185015c630 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f1838002950 tx=0x7f1838007d30 comp rx=0 tx=0).stop 2023-10-09T06:37:13.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.632+0000 7f1854e72700 1 -- 172.21.15.163:0/2194479350 shutdown_connections 2023-10-09T06:37:13.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.632+0000 7f1854e72700 1 --2- 172.21.15.163:0/2194479350 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1830067320 0x7f18300697d0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:13.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.632+0000 7f1854e72700 1 --2- 172.21.15.163:0/2194479350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18500c7b30 0x7f18500c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:13.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.632+0000 7f1854e72700 1 --2- 172.21.15.163:0/2194479350 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f183c0051c0 0x7f1850156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:13.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.632+0000 7f1854e72700 1 --2- 172.21.15.163:0/2194479350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f185015c1c0 0x7f185015c630 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:13.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.632+0000 7f1854e72700 1 -- 172.21.15.163:0/2194479350 >> 172.21.15.163:0/2194479350 conn(0x7f1850068e00 msgr2=0x7f18500b04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:13.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.632+0000 7f1854e72700 1 -- 172.21.15.163:0/2194479350 shutdown_connections 2023-10-09T06:37:13.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.632+0000 7f1854e72700 1 -- 172.21.15.163:0/2194479350 wait complete. 2023-10-09T06:37:13.635 INFO:teuthology.orchestra.run.smithi163.stderr:updated caps for client.0 2023-10-09T06:37:13.645 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:37:13.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.759+0000 7f231f36a700 1 Processor -- start 2023-10-09T06:37:13.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.759+0000 7f231f36a700 1 -- start start 2023-10-09T06:37:13.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.760+0000 7f231f36a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2318159420 0x7f23180b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:13.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.760+0000 7f231f36a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f23181597f0 0x7f23180b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:13.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.760+0000 7f231f36a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f231815d230 0x7f231815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:13.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.760+0000 7f231f36a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f231815dbd0 con 0x7f23181597f0 2023-10-09T06:37:13.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.760+0000 7f231f36a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23180b11d0 con 0x7f231815d230 2023-10-09T06:37:13.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.760+0000 7f231f36a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23180b14b0 con 0x7f2318159420 2023-10-09T06:37:13.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.760+0000 7f231db67700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f23181597f0 0x7f23180b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:13.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.760+0000 7f231db67700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f23181597f0 0x7f23180b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56850/0 (socket says 172.21.15.163:56850) 2023-10-09T06:37:13.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.760+0000 7f231db67700 1 -- 172.21.15.163:0/2575318998 learned_addr learned my addr 172.21.15.163:0/2575318998 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:13.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.760+0000 7f231e368700 1 --2- 172.21.15.163:0/2575318998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2318159420 0x7f23180b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:13.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.760+0000 7f231eb69700 1 --2- 172.21.15.163:0/2575318998 >> v2:172.21.15.181:3300/0 conn(0x7f231815d230 0x7f231815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:13.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.761+0000 7f231db67700 1 -- 172.21.15.163:0/2575318998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2318159420 msgr2=0x7f23180b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:13.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.761+0000 7f231db67700 1 --2- 172.21.15.163:0/2575318998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2318159420 0x7f23180b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:13.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.761+0000 7f231db67700 1 -- 172.21.15.163:0/2575318998 >> v2:172.21.15.181:3300/0 conn(0x7f231815d230 msgr2=0x7f231815d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:13.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.761+0000 7f231db67700 1 --2- 172.21.15.163:0/2575318998 >> v2:172.21.15.181:3300/0 conn(0x7f231815d230 0x7f231815d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:13.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.761+0000 7f231db67700 1 -- 172.21.15.163:0/2575318998 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23180b1790 con 0x7f23181597f0 2023-10-09T06:37:13.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.761+0000 7f231db67700 1 --2- 172.21.15.163:0/2575318998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23181597f0 0x7f23180b2270 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f23180d1720 tx=0x7f2308009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db9796780bd6e8f server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:13.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.761+0000 7f231d366700 1 -- 172.21.15.163:0/2575318998 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230800b860 con 0x7f23181597f0 2023-10-09T06:37:13.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.761+0000 7f231d366700 1 -- 172.21.15.163:0/2575318998 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f230800b9c0 con 0x7f23181597f0 2023-10-09T06:37:13.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.761+0000 7f231d366700 1 -- 172.21.15.163:0/2575318998 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230800bd60 con 0x7f23181597f0 2023-10-09T06:37:13.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.761+0000 7f231f36a700 1 -- 172.21.15.163:0/2575318998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23181597f0 msgr2=0x7f23180b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:13.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.761+0000 7f231f36a700 1 --2- 172.21.15.163:0/2575318998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23181597f0 0x7f23180b2270 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f23180d1720 tx=0x7f2308009250 comp rx=0 tx=0).stop 2023-10-09T06:37:13.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.761+0000 7f231f36a700 1 -- 172.21.15.163:0/2575318998 shutdown_connections 2023-10-09T06:37:13.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.761+0000 7f231f36a700 1 --2- 172.21.15.163:0/2575318998 >> v2:172.21.15.181:3300/0 conn(0x7f231815d230 0x7f231815d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:13.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.761+0000 7f231f36a700 1 --2- 172.21.15.163:0/2575318998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2318159420 0x7f23180b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:13.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.761+0000 7f231f36a700 1 --2- 172.21.15.163:0/2575318998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23181597f0 0x7f23180b2270 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:13.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.761+0000 7f231f36a700 1 -- 172.21.15.163:0/2575318998 >> 172.21.15.163:0/2575318998 conn(0x7f23180d0660 msgr2=0x7f23180d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:13.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.762+0000 7f231f36a700 1 -- 172.21.15.163:0/2575318998 shutdown_connections 2023-10-09T06:37:13.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.762+0000 7f231f36a700 1 -- 172.21.15.163:0/2575318998 wait complete. 2023-10-09T06:37:13.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.763+0000 7f231f36a700 1 Processor -- start 2023-10-09T06:37:13.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.763+0000 7f231f36a700 1 -- start start 2023-10-09T06:37:13.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.763+0000 7f231f36a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2318159420 0x7f23180cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:13.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.763+0000 7f231f36a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23181597f0 0x7f2318155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:13.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.763+0000 7f231f36a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f231815d230 0x7f2318156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:13.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.763+0000 7f231f36a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23180c63d0 con 0x7f231815d230 2023-10-09T06:37:13.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.763+0000 7f231f36a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23180cb950 con 0x7f23181597f0 2023-10-09T06:37:13.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.763+0000 7f231f36a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23180cb0a0 con 0x7f2318159420 2023-10-09T06:37:13.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.763+0000 7f231eb69700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f231815d230 0x7f2318156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:13.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.763+0000 7f231eb69700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f231815d230 0x7f2318156070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56860/0 (socket says 172.21.15.163:56860) 2023-10-09T06:37:13.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.763+0000 7f231eb69700 1 -- 172.21.15.163:0/1562443590 learned_addr learned my addr 172.21.15.163:0/1562443590 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:13.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.763+0000 7f231e368700 1 --2- 172.21.15.163:0/1562443590 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2318159420 0x7f23180cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:13.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.763+0000 7f231db67700 1 --2- 172.21.15.163:0/1562443590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23181597f0 0x7f2318155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:13.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.764+0000 7f231eb69700 1 -- 172.21.15.163:0/1562443590 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2318159420 msgr2=0x7f23180cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:13.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.764+0000 7f231eb69700 1 --2- 172.21.15.163:0/1562443590 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2318159420 0x7f23180cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:13.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.764+0000 7f231eb69700 1 -- 172.21.15.163:0/1562443590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23181597f0 msgr2=0x7f2318155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:13.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.764+0000 7f231eb69700 1 --2- 172.21.15.163:0/1562443590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23181597f0 0x7f2318155b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:13.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.764+0000 7f231eb69700 1 -- 172.21.15.163:0/1562443590 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23080094f0 con 0x7f231815d230 2023-10-09T06:37:13.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.764+0000 7f231eb69700 1 --2- 172.21.15.163:0/1562443590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f231815d230 0x7f2318156070 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f2310007f20 tx=0x7f2310009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:13.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.764+0000 7f230e7fc700 1 -- 172.21.15.163:0/1562443590 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231000eb90 con 0x7f231815d230 2023-10-09T06:37:13.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.764+0000 7f230e7fc700 1 -- 172.21.15.163:0/1562443590 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f231000ed10 con 0x7f231815d230 2023-10-09T06:37:13.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.764+0000 7f231f36a700 1 -- 172.21.15.163:0/1562443590 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23180c60f0 con 0x7f231815d230 2023-10-09T06:37:13.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.764+0000 7f230e7fc700 1 -- 172.21.15.163:0/1562443590 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2310014670 con 0x7f231815d230 2023-10-09T06:37:13.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.764+0000 7f231f36a700 1 -- 172.21.15.163:0/1562443590 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23180cb5e0 con 0x7f231815d230 2023-10-09T06:37:13.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.765+0000 7f231f36a700 1 -- 172.21.15.163:0/1562443590 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23180cbae0 con 0x7f231815d230 2023-10-09T06:37:13.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.765+0000 7f230e7fc700 1 -- 172.21.15.163:0/1562443590 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f23100149e0 con 0x7f231815d230 2023-10-09T06:37:13.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.766+0000 7f230e7fc700 1 --2- 172.21.15.163:0/1562443590 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f22fc067320 0x7f22fc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:13.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.766+0000 7f230e7fc700 1 -- 172.21.15.163:0/1562443590 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2310085270 con 0x7f231815d230 2023-10-09T06:37:13.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.766+0000 7f231e368700 1 --2- 172.21.15.163:0/1562443590 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f22fc067320 0x7f22fc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:13.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.767+0000 7f231e368700 1 --2- 172.21.15.163:0/1562443590 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f22fc067320 0x7f22fc0697d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2314006fd0 tx=0x7f2314008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:13.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:13.770+0000 7f230e7fc700 1 -- 172.21.15.163:0/1562443590 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2310051f00 con 0x7f231815d230 2023-10-09T06:37:13.926 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:37:13.929 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:37:14.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.002+0000 7f231f36a700 1 -- 172.21.15.163:0/1562443590 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f231804c3c0 con 0x7f231815d230 2023-10-09T06:37:14.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.003+0000 7f230e7fc700 1 -- 172.21.15.163:0/1562443590 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4608 (secure 0 0 0) 0x7f2310055520 con 0x7f231815d230 2023-10-09T06:37:14.004 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:14.004 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":75,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2821591245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2821591245},{"type":"v1","addr":"172.21.15.181:6836","nonce":2821591245}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":70},{"gid":6592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3075056851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3075056851},{"type":"v1","addr":"172.21.15.181:6837","nonce":3075056851}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":70},{"gid":7158,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/1355789800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1355789800},{"type":"v1","addr":"172.21.15.163:6834","nonce":1355789800}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:37:08.129202+0000","modified":"2023-10-09T06:37:11.774637+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7161},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7161":{"gid":7161,"name":"c","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/571471083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":571471083},{"type":"v1","addr":"172.21.15.163:6835","nonce":571471083}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-09T06:37:14.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.005+0000 7f231f36a700 1 -- 172.21.15.163:0/1562443590 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f22fc067320 msgr2=0x7f22fc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:14.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.005+0000 7f231f36a700 1 --2- 172.21.15.163:0/1562443590 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f22fc067320 0x7f22fc0697d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2314006fd0 tx=0x7f2314008040 comp rx=0 tx=0).stop 2023-10-09T06:37:14.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.005+0000 7f231f36a700 1 -- 172.21.15.163:0/1562443590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f231815d230 msgr2=0x7f2318156070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:14.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.005+0000 7f231f36a700 1 --2- 172.21.15.163:0/1562443590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f231815d230 0x7f2318156070 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f2310007f20 tx=0x7f2310009860 comp rx=0 tx=0).stop 2023-10-09T06:37:14.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.005+0000 7f231f36a700 1 -- 172.21.15.163:0/1562443590 shutdown_connections 2023-10-09T06:37:14.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.005+0000 7f231f36a700 1 --2- 172.21.15.163:0/1562443590 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f22fc067320 0x7f22fc0697d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.005+0000 7f231f36a700 1 --2- 172.21.15.163:0/1562443590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f231815d230 0x7f2318156070 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.005+0000 7f231f36a700 1 --2- 172.21.15.163:0/1562443590 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2318159420 0x7f23180cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.005+0000 7f231f36a700 1 --2- 172.21.15.163:0/1562443590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23181597f0 0x7f2318155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.005+0000 7f231f36a700 1 -- 172.21.15.163:0/1562443590 >> 172.21.15.163:0/1562443590 conn(0x7f23180d0660 msgr2=0x7f23180b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:14.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.005+0000 7f231f36a700 1 -- 172.21.15.163:0/1562443590 shutdown_connections 2023-10-09T06:37:14.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.005+0000 7f231f36a700 1 -- 172.21.15.163:0/1562443590 wait complete. 2023-10-09T06:37:14.007 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 75 2023-10-09T06:37:14.420 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 75, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-09T06:37:08.129202+0000', 'modified': '2023-10-09T06:37:11.774637+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7161}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7161': {'gid': 7161, 'name': 'c', 'rank': 0, 'incarnation': 71, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.163:6835/571471083', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6833', 'nonce': 571471083}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 571471083}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-09T06:37:14.420 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-09T06:37:14.421 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-09T06:37:14.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.535+0000 7f1f96861700 1 Processor -- start 2023-10-09T06:37:14.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.536+0000 7f1f96861700 1 -- start start 2023-10-09T06:37:14.537 INFO:tasks.ceph.mds.c.smithi163.stderr:2023-10-09T06:37:14.536+0000 7fbd9d0b0700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-09T06:37:14.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.536+0000 7f1f96861700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1f90155c10 0x7f1f90155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:14.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.536+0000 7f1f96861700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1f900ac230 0x7f1f900ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:14.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.536+0000 7f1f96861700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f900ad930 con 0x7f1f901568a0 2023-10-09T06:37:14.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.536+0000 7f1f96861700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f900acd20 con 0x7f1f90155c10 2023-10-09T06:37:14.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.536+0000 7f1f96861700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f900ad000 con 0x7f1f900ac230 2023-10-09T06:37:14.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.537+0000 7f1f8ffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1f90155c10 0x7f1f90155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:14.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.537+0000 7f1f94dfe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f1f901568a0 0x7f1f90156c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:54640/0 (socket says 172.21.15.163:54640) 2023-10-09T06:37:14.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.537+0000 7f1f94dfe700 1 -- 172.21.15.163:0/2816607381 learned_addr learned my addr 172.21.15.163:0/2816607381 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:14.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.537+0000 7f1f8f7fe700 1 --2- 172.21.15.163:0/2816607381 >> v2:172.21.15.181:3300/0 conn(0x7f1f900ac230 0x7f1f900ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:14.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.537+0000 7f1f8ffff700 1 -- 172.21.15.163:0/2816607381 >> v2:172.21.15.181:3300/0 conn(0x7f1f900ac230 msgr2=0x7f1f900ac6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:14.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.537+0000 7f1f8ffff700 1 --2- 172.21.15.163:0/2816607381 >> v2:172.21.15.181:3300/0 conn(0x7f1f900ac230 0x7f1f900ac6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.537+0000 7f1f8ffff700 1 -- 172.21.15.163:0/2816607381 >> v1:172.21.15.163:6789/0 conn(0x7f1f901568a0 legacy=0x7f1f90156c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:14.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.537+0000 7f1f8effd700 1 -- 172.21.15.163:0/2816607381 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4031449223 0 0) 0x7f1f900ad930 con 0x7f1f901568a0 2023-10-09T06:37:14.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.537+0000 7f1f8ffff700 1 -- 172.21.15.163:0/2816607381 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f900ad2e0 con 0x7f1f90155c10 2023-10-09T06:37:14.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.537+0000 7f1f8f7fe700 1 --2- 172.21.15.163:0/2816607381 >> v2:172.21.15.181:3300/0 conn(0x7f1f900ac230 0x7f1f900ac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:14.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.537+0000 7f1f8ffff700 1 --2- 172.21.15.163:0/2816607381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f90155c10 0x7f1f90155fe0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f1f90065bb0 tx=0x7f1f800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a84f3b9d32815a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:14.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.537+0000 7f1f8effd700 1 -- 172.21.15.163:0/2816607381 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8000b6b0 con 0x7f1f90155c10 2023-10-09T06:37:14.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.537+0000 7f1f8effd700 1 -- 172.21.15.163:0/2816607381 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8000b810 con 0x7f1f90155c10 2023-10-09T06:37:14.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.537+0000 7f1f8effd700 1 -- 172.21.15.163:0/2816607381 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8000bbb0 con 0x7f1f90155c10 2023-10-09T06:37:14.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.538+0000 7f1f96861700 1 -- 172.21.15.163:0/2816607381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f90155c10 msgr2=0x7f1f90155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:14.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.538+0000 7f1f96861700 1 --2- 172.21.15.163:0/2816607381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f90155c10 0x7f1f90155fe0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f1f90065bb0 tx=0x7f1f800091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.538+0000 7f1f96861700 1 -- 172.21.15.163:0/2816607381 shutdown_connections 2023-10-09T06:37:14.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.538+0000 7f1f96861700 1 --2- 172.21.15.163:0/2816607381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f90155c10 0x7f1f90155fe0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.538+0000 7f1f96861700 1 --2- 172.21.15.163:0/2816607381 >> v2:172.21.15.181:3300/0 conn(0x7f1f900ac230 0x7f1f900ac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.538+0000 7f1f96861700 1 -- 172.21.15.163:0/2816607381 >> 172.21.15.163:0/2816607381 conn(0x7f1f9006a320 msgr2=0x7f1f900cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:14.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.538+0000 7f1f96861700 1 -- 172.21.15.163:0/2816607381 shutdown_connections 2023-10-09T06:37:14.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.538+0000 7f1f96861700 1 -- 172.21.15.163:0/2816607381 wait complete. 2023-10-09T06:37:14.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.539+0000 7f1f96861700 1 Processor -- start 2023-10-09T06:37:14.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.539+0000 7f1f96861700 1 -- start start 2023-10-09T06:37:14.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.540+0000 7f1f96861700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f900ac230 0x7f1f90145950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:14.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.540+0000 7f1f96861700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f90155c10 0x7f1f9014d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:14.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.540+0000 7f1f96861700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f901568a0 0x7f1f900cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:14.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.540+0000 7f1f96861700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f900ca0f0 con 0x7f1f900ac230 2023-10-09T06:37:14.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.540+0000 7f1f96861700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f900c9ae0 con 0x7f1f90155c10 2023-10-09T06:37:14.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.540+0000 7f1f96861700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f90143960 con 0x7f1f901568a0 2023-10-09T06:37:14.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.540+0000 7f1f8ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f900ac230 0x7f1f90145950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:14.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.540+0000 7f1f8ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f900ac230 0x7f1f90145950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56874/0 (socket says 172.21.15.163:56874) 2023-10-09T06:37:14.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.540+0000 7f1f8ffff700 1 -- 172.21.15.163:0/2900386580 learned_addr learned my addr 172.21.15.163:0/2900386580 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:14.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.540+0000 7f1f8ffff700 1 -- 172.21.15.163:0/2900386580 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f901568a0 msgr2=0x7f1f900cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:14.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.540+0000 7f1f8ffff700 1 --2- 172.21.15.163:0/2900386580 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f901568a0 0x7f1f900cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.540+0000 7f1f8f7fe700 1 --2- 172.21.15.163:0/2900386580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f90155c10 0x7f1f9014d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:14.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.540+0000 7f1f8ffff700 1 -- 172.21.15.163:0/2900386580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f90155c10 msgr2=0x7f1f9014d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:14.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.540+0000 7f1f8ffff700 1 --2- 172.21.15.163:0/2900386580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f90155c10 0x7f1f9014d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.540+0000 7f1f8ffff700 1 -- 172.21.15.163:0/2900386580 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f80009420 con 0x7f1f900ac230 2023-10-09T06:37:14.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.540+0000 7f1f8ffff700 1 --2- 172.21.15.163:0/2900386580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f900ac230 0x7f1f90145950 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f1f90064290 tx=0x7f1f80017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:14.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.540+0000 7f1f77fff700 1 -- 172.21.15.163:0/2900386580 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8000bb90 con 0x7f1f900ac230 2023-10-09T06:37:14.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.541+0000 7f1f77fff700 1 -- 172.21.15.163:0/2900386580 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f80011420 con 0x7f1f900ac230 2023-10-09T06:37:14.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.541+0000 7f1f96861700 1 -- 172.21.15.163:0/2900386580 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f900c9d80 con 0x7f1f900ac230 2023-10-09T06:37:14.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.541+0000 7f1f77fff700 1 -- 172.21.15.163:0/2900386580 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f800117a0 con 0x7f1f900ac230 2023-10-09T06:37:14.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.541+0000 7f1f96861700 1 -- 172.21.15.163:0/2900386580 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f901434f0 con 0x7f1f900ac230 2023-10-09T06:37:14.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.542+0000 7f1f77fff700 1 -- 172.21.15.163:0/2900386580 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1f80011b90 con 0x7f1f900ac230 2023-10-09T06:37:14.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.542+0000 7f1f77fff700 1 --2- 172.21.15.163:0/2900386580 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1f7c067320 0x7f1f7c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:14.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.542+0000 7f1f77fff700 1 -- 172.21.15.163:0/2900386580 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1f80086600 con 0x7f1f900ac230 2023-10-09T06:37:14.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.542+0000 7f1f8f7fe700 1 --2- 172.21.15.163:0/2900386580 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1f7c067320 0x7f1f7c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:14.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.543+0000 7f1f8f7fe700 1 --2- 172.21.15.163:0/2900386580 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1f7c067320 0x7f1f7c0697d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1f84006f00 tx=0x7f1f84008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:14.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.544+0000 7f1f9fb59b80 1 Processor -- start 2023-10-09T06:37:14.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.544+0000 7f1f9fb59b80 1 -- start start 2023-10-09T06:37:14.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.545+0000 7f1f9fb59b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4be056a50 0x55c4be056e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:14.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.545+0000 7f1f9fb59b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c4be0a0d00 0x55c4be07fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:14.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.545+0000 7f1f9fb59b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4be07ff60 0x55c4be052cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:14.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.545+0000 7f1f9fb59b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4bdfada70 con 0x55c4be07ff60 2023-10-09T06:37:14.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.545+0000 7f1f9fb59b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4bdfab270 con 0x55c4be056a50 2023-10-09T06:37:14.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.545+0000 7f1f9fb59b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c4bdfa7060 con 0x55c4be0a0d00 2023-10-09T06:37:14.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.545+0000 7f1f94dfe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4be07ff60 0x55c4be052cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:14.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.545+0000 7f1f94dfe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4be07ff60 0x55c4be052cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56882/0 (socket says 172.21.15.163:56882) 2023-10-09T06:37:14.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.545+0000 7f1f94dfe700 1 -- 172.21.15.163:0/2679984949 learned_addr learned my addr 172.21.15.163:0/2679984949 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:14.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.545+0000 7f1f8f7fe700 1 --2- 172.21.15.163:0/2679984949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c4be0a0d00 0x55c4be07fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:14.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.545+0000 7f1f8ffff700 1 --2- 172.21.15.163:0/2679984949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4be056a50 0x55c4be056e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:14.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.545+0000 7f1f94dfe700 1 -- 172.21.15.163:0/2679984949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c4be0a0d00 msgr2=0x55c4be07fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:14.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.545+0000 7f1f94dfe700 1 --2- 172.21.15.163:0/2679984949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c4be0a0d00 0x55c4be07fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.546+0000 7f1f94dfe700 1 -- 172.21.15.163:0/2679984949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4be056a50 msgr2=0x55c4be056e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:14.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.546+0000 7f1f94dfe700 1 --2- 172.21.15.163:0/2679984949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4be056a50 0x55c4be056e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.546+0000 7f1f94dfe700 1 -- 172.21.15.163:0/2679984949 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4bde41270 con 0x55c4be07ff60 2023-10-09T06:37:14.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.546+0000 7f1f94dfe700 1 --2- 172.21.15.163:0/2679984949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4be07ff60 0x55c4be052cf0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f1f88002170 tx=0x7f1f880019e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d96ccd5b86a1e75 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:14.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.546+0000 7f1f757fa700 1 -- 172.21.15.163:0/2679984949 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f88004bb0 con 0x55c4be07ff60 2023-10-09T06:37:14.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.546+0000 7f1f757fa700 1 -- 172.21.15.163:0/2679984949 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f88004d10 con 0x55c4be07ff60 2023-10-09T06:37:14.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.546+0000 7f1f757fa700 1 -- 172.21.15.163:0/2679984949 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f880116c0 con 0x55c4be07ff60 2023-10-09T06:37:14.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.546+0000 7f1f9fb59b80 1 -- 172.21.15.163:0/2679984949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4be07ff60 msgr2=0x55c4be052cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:14.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.546+0000 7f1f9fb59b80 1 --2- 172.21.15.163:0/2679984949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4be07ff60 0x55c4be052cf0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f1f88002170 tx=0x7f1f880019e0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.547+0000 7f1f9fb59b80 1 -- 172.21.15.163:0/2679984949 shutdown_connections 2023-10-09T06:37:14.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.547+0000 7f1f9fb59b80 1 --2- 172.21.15.163:0/2679984949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4be07ff60 0x55c4be052cf0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.547+0000 7f1f9fb59b80 1 --2- 172.21.15.163:0/2679984949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4be056a50 0x55c4be056e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.547+0000 7f1f9fb59b80 1 --2- 172.21.15.163:0/2679984949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c4be0a0d00 0x55c4be07fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.547+0000 7f1f9fb59b80 1 -- 172.21.15.163:0/2679984949 >> 172.21.15.163:0/2679984949 conn(0x55c4be058370 msgr2=0x55c4be058740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:14.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.547+0000 7f1f9fb59b80 1 -- 172.21.15.163:0/2679984949 shutdown_connections 2023-10-09T06:37:14.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.547+0000 7f1f9fb59b80 1 -- 172.21.15.163:0/2679984949 wait complete. 2023-10-09T06:37:14.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.547+0000 7f1f9fb59b80 1 -- start start 2023-10-09T06:37:14.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.547+0000 7f1f9fb59b80 1 Processor -- start 2023-10-09T06:37:14.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.548+0000 7f1f9fb59b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4be056a50 0x55c4be0993a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:14.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.548+0000 7f1f9fb59b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c4be07ff60 0x55c4be09add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:14.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.548+0000 7f1f9fb59b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4be0a0d00 0x55c4be051c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:14.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.548+0000 7f1f9fb59b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4bdf52890 con 0x55c4be0a0d00 2023-10-09T06:37:14.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.548+0000 7f1f9fb59b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4bdf6dea0 con 0x55c4be056a50 2023-10-09T06:37:14.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.548+0000 7f1f9fb59b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c4bde29e30 con 0x55c4be07ff60 2023-10-09T06:37:14.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.548+0000 7f1f9fb59b80 10 client.0 _ll_register_callbacks cb 0x55c4be01a110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-09T06:37:14.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.548+0000 7f1f94dfe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4be0a0d00 0x55c4be051c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:14.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.548+0000 7f1f94dfe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4be0a0d00 0x55c4be051c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56892/0 (socket says 172.21.15.163:56892) 2023-10-09T06:37:14.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.548+0000 7f1f94dfe700 1 -- 172.21.15.163:0/667720255 learned_addr learned my addr 172.21.15.163:0/667720255 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:14.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.548+0000 7f1f8f7fe700 1 --2- 172.21.15.163:0/667720255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c4be07ff60 0x55c4be09add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:14.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.548+0000 7f1f8ffff700 1 --2- 172.21.15.163:0/667720255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4be056a50 0x55c4be0993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:14.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.548+0000 7f1f94dfe700 1 -- 172.21.15.163:0/667720255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c4be07ff60 msgr2=0x55c4be09add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:14.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.548+0000 7f1f94dfe700 1 --2- 172.21.15.163:0/667720255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c4be07ff60 0x55c4be09add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.548+0000 7f1f94dfe700 1 -- 172.21.15.163:0/667720255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4be056a50 msgr2=0x55c4be0993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:14.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.548+0000 7f1f94dfe700 1 --2- 172.21.15.163:0/667720255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4be056a50 0x55c4be0993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.548+0000 7f1f94dfe700 1 -- 172.21.15.163:0/667720255 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f88001cc0 con 0x55c4be0a0d00 2023-10-09T06:37:14.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.548+0000 7f1f8f7fe700 1 --2- 172.21.15.163:0/667720255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c4be07ff60 0x55c4be09add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:14.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.548+0000 7f1f8ffff700 1 --2- 172.21.15.163:0/667720255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4be056a50 0x55c4be0993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:14.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.549+0000 7f1f94dfe700 1 --2- 172.21.15.163:0/667720255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4be0a0d00 0x55c4be051c20 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f1f8800e960 tx=0x7f1f88019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89cdf893bbbfae08 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:14.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.549+0000 7f1f6e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-10-09T06:37:14.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.549+0000 7f1f6e7fc700 1 -- 172.21.15.163:0/667720255 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f88022070 con 0x55c4be0a0d00 2023-10-09T06:37:14.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.549+0000 7f1f6e7fc700 1 -- 172.21.15.163:0/667720255 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1f88011cc0 con 0x55c4be0a0d00 2023-10-09T06:37:14.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.549+0000 7f1f9fb59b80 1 -- 172.21.15.163:0/667720255 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c4be04bb70 con 0x55c4be0a0d00 2023-10-09T06:37:14.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.549+0000 7f1f6e7fc700 1 -- 172.21.15.163:0/667720255 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f88004bb0 con 0x55c4be0a0d00 2023-10-09T06:37:14.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.549+0000 7f1f9fb59b80 10 client.7299 fetch_fsmap learned FSMap version 75 2023-10-09T06:37:14.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.549+0000 7f1f9fb59b80 1 -- 172.21.15.163:0/667720255 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x55c4bdde3fe0 con 0x55c4be0a0d00 2023-10-09T06:37:14.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.549+0000 7f1f6e7fc700 1 -- 172.21.15.163:0/667720255 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c4be04bb70 con 0x55c4be0a0d00 2023-10-09T06:37:14.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.549+0000 7f1f6e7fc700 1 -- 172.21.15.163:0/667720255 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 75) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f1f8801c690 con 0x55c4be0a0d00 2023-10-09T06:37:14.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.549+0000 7f1f9fb59b80 10 client.7299 fetch_fsmap finished waiting for FSMap version 75 2023-10-09T06:37:14.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.549+0000 7f1f9fb59b80 10 client.7299 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-09T06:37:14.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.549+0000 7f1f9fb59b80 20 client.7299 populate_metadata read hostname 'smithi163' 2023-10-09T06:37:14.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.549+0000 7f1f9fb59b80 1 --2- 172.21.15.163:0/667720255 >> [v2:172.21.15.163:6833/571471083,v1:172.21.15.163:6835/571471083] conn(0x55c4be095340 0x55c4be095710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:14.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.549+0000 7f1f9fb59b80 4 client.7299 mds_command: new command op to 7161 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-09T06:37:14.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.549+0000 7f1f9fb59b80 1 -- 172.21.15.163:0/667720255 --> [v2:172.21.15.163:6833/571471083,v1:172.21.15.163:6835/571471083] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c4be028aa0 con 0x55c4be095340 2023-10-09T06:37:14.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.549+0000 7f1f8ffff700 1 --2- 172.21.15.163:0/667720255 >> [v2:172.21.15.163:6833/571471083,v1:172.21.15.163:6835/571471083] conn(0x55c4be095340 0x55c4be095710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:14.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.550+0000 7f1f6e7fc700 10 client.7299 ms_handle_connect on v2:172.21.15.163:6833/571471083 2023-10-09T06:37:14.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.550+0000 7f1f8ffff700 1 --2- 172.21.15.163:0/667720255 >> [v2:172.21.15.163:6833/571471083,v1:172.21.15.163:6835/571471083] conn(0x55c4be095340 0x55c4be095710 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53f75e74dad8898c server_cookie=b369d19e82dd788a in_seq=0 out_seq=0 2023-10-09T06:37:14.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.550+0000 7f1f6e7fc700 1 -- 172.21.15.163:0/667720255 <== mds.0 v2:172.21.15.163:6833/571471083 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55c4be028aa0 con 0x55c4be095340 2023-10-09T06:37:14.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.550+0000 7f1f6e7fc700 10 client.7299 handle_command_reply: tid=0 2023-10-09T06:37:14.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.550+0000 7f1f9fb59b80 1 client.7299 shutdown 2023-10-09T06:37:14.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.550+0000 7f1f9fb59b80 1 -- 172.21.15.163:0/667720255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4be0a0d00 msgr2=0x55c4be051c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:14.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.550+0000 7f1f9fb59b80 1 --2- 172.21.15.163:0/667720255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4be0a0d00 0x55c4be051c20 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f1f8800e960 tx=0x7f1f88019000 comp rx=0 tx=0).stop 2023-10-09T06:37:14.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.550+0000 7f1f9fb59b80 1 -- 172.21.15.163:0/667720255 shutdown_connections 2023-10-09T06:37:14.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.550+0000 7f1f9fb59b80 1 --2- 172.21.15.163:0/667720255 >> [v2:172.21.15.163:6833/571471083,v1:172.21.15.163:6835/571471083] conn(0x55c4be095340 0x55c4be095710 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.550+0000 7f1f9fb59b80 1 --2- 172.21.15.163:0/667720255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4be0a0d00 0x55c4be051c20 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.550+0000 7f1f6e7fc700 0 client.7299 ms_handle_reset on v2:172.21.15.163:6833/571471083 2023-10-09T06:37:14.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.550+0000 7f1f9fb59b80 1 --2- 172.21.15.163:0/667720255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4be056a50 0x55c4be0993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.550+0000 7f1f9fb59b80 1 --2- 172.21.15.163:0/667720255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c4be07ff60 0x55c4be09add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.550+0000 7f1f9fb59b80 1 -- 172.21.15.163:0/667720255 >> 172.21.15.163:0/667720255 conn(0x55c4be058370 msgr2=0x55c4be052c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:14.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.550+0000 7f1f9fb59b80 1 -- 172.21.15.163:0/667720255 shutdown_connections 2023-10-09T06:37:14.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.550+0000 7f1f9fb59b80 1 -- 172.21.15.163:0/667720255 wait complete. 2023-10-09T06:37:14.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.551+0000 7f1f9fb59b80 20 client.7299 trim_cache size 0 max 16384 2023-10-09T06:37:14.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.567+0000 7f1f9fb59b80 1 Processor -- start 2023-10-09T06:37:14.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.567+0000 7f1f9fb59b80 1 -- start start 2023-10-09T06:37:14.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.568+0000 7f1f9fb59b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4be056a50 0x55c4be0aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:14.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.568+0000 7f1f9fb59b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4be07ff60 0x55c4be097770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:14.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.568+0000 7f1f9fb59b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c4be0a0d00 0x55c4be059bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:14.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.568+0000 7f1f9fb59b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4be0529a0 con 0x55c4be056a50 2023-10-09T06:37:14.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.568+0000 7f1f9fb59b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4be059240 con 0x55c4be07ff60 2023-10-09T06:37:14.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.568+0000 7f1f9fb59b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c4be0515a0 con 0x55c4be0a0d00 2023-10-09T06:37:14.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.568+0000 7f1f8ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4be056a50 0x55c4be0aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:14.572 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.568+0000 7f1f8ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4be056a50 0x55c4be0aa4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56894/0 (socket says 172.21.15.163:56894) 2023-10-09T06:37:14.572 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.568+0000 7f1f8ffff700 1 -- 172.21.15.163:0/3468136379 learned_addr learned my addr 172.21.15.163:0/3468136379 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:14.572 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.568+0000 7f1f94dfe700 1 --2- 172.21.15.163:0/3468136379 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c4be0a0d00 0x55c4be059bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:14.572 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.568+0000 7f1f8f7fe700 1 --2- 172.21.15.163:0/3468136379 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4be07ff60 0x55c4be097770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:14.572 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.568+0000 7f1f8ffff700 1 -- 172.21.15.163:0/3468136379 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c4be0a0d00 msgr2=0x55c4be059bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:14.572 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.568+0000 7f1f8ffff700 1 --2- 172.21.15.163:0/3468136379 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c4be0a0d00 0x55c4be059bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.572 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.568+0000 7f1f8ffff700 1 -- 172.21.15.163:0/3468136379 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4be07ff60 msgr2=0x55c4be097770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:14.572 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.568+0000 7f1f8ffff700 1 --2- 172.21.15.163:0/3468136379 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4be07ff60 0x55c4be097770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.572 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.568+0000 7f1f8ffff700 1 -- 172.21.15.163:0/3468136379 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4bdf75780 con 0x55c4be056a50 2023-10-09T06:37:14.572 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.568+0000 7f1f8ffff700 1 --2- 172.21.15.163:0/3468136379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4be056a50 0x55c4be0aa4f0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000b3a0 tx=0x7f1f80059040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cdfbebcb748ed70 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:14.572 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.568+0000 7f1f75ffb700 1 -- 172.21.15.163:0/3468136379 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80056f50 con 0x55c4be056a50 2023-10-09T06:37:14.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.568+0000 7f1f75ffb700 1 -- 172.21.15.163:0/3468136379 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8005a1a0 con 0x55c4be056a50 2023-10-09T06:37:14.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.568+0000 7f1f75ffb700 1 -- 172.21.15.163:0/3468136379 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8005def0 con 0x55c4be056a50 2023-10-09T06:37:14.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.569+0000 7f1f9fb59b80 1 -- 172.21.15.163:0/3468136379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4be056a50 msgr2=0x55c4be0aa4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:14.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.569+0000 7f1f9fb59b80 1 --2- 172.21.15.163:0/3468136379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4be056a50 0x55c4be0aa4f0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000b3a0 tx=0x7f1f80059040 comp rx=0 tx=0).stop 2023-10-09T06:37:14.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.569+0000 7f1f9fb59b80 1 -- 172.21.15.163:0/3468136379 shutdown_connections 2023-10-09T06:37:14.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.569+0000 7f1f9fb59b80 1 --2- 172.21.15.163:0/3468136379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4be056a50 0x55c4be0aa4f0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.569+0000 7f1f9fb59b80 1 --2- 172.21.15.163:0/3468136379 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c4be0a0d00 0x55c4be059bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.569+0000 7f1f9fb59b80 1 --2- 172.21.15.163:0/3468136379 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4be07ff60 0x55c4be097770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.569+0000 7f1f9fb59b80 1 -- 172.21.15.163:0/3468136379 >> 172.21.15.163:0/3468136379 conn(0x55c4be058370 msgr2=0x55c4be04f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:14.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.569+0000 7f1f9fb59b80 1 -- 172.21.15.163:0/3468136379 shutdown_connections 2023-10-09T06:37:14.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.569+0000 7f1f9fb59b80 1 -- 172.21.15.163:0/3468136379 wait complete. 2023-10-09T06:37:14.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.570+0000 7f1f9fb59b80 1 -- start start 2023-10-09T06:37:14.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.570+0000 7f1f9fb59b80 1 Processor -- start 2023-10-09T06:37:14.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.570+0000 7f1f9fb59b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4be056a50 0x55c4be082620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:14.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.570+0000 7f1f9fb59b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4be07ff60 0x55c4be082b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:14.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.570+0000 7f1f9fb59b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c4be0a0d00 0x55c4be0a91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:14.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.570+0000 7f1f9fb59b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4be095cd0 con 0x55c4be056a50 2023-10-09T06:37:14.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.570+0000 7f1f9fb59b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4be096130 con 0x55c4be07ff60 2023-10-09T06:37:14.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.570+0000 7f1f9fb59b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c4be0579f0 con 0x55c4be0a0d00 2023-10-09T06:37:14.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.570+0000 7f1f9fb59b80 10 client.0 _ll_register_callbacks cb 0x55c4be01a110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-09T06:37:14.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.570+0000 7f1f8ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4be056a50 0x55c4be082620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:14.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.571+0000 7f1f8ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4be056a50 0x55c4be082620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56904/0 (socket says 172.21.15.163:56904) 2023-10-09T06:37:14.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.571+0000 7f1f8ffff700 1 -- 172.21.15.163:0/1629837719 learned_addr learned my addr 172.21.15.163:0/1629837719 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:14.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.571+0000 7f1f8f7fe700 1 --2- 172.21.15.163:0/1629837719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4be07ff60 0x55c4be082b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:14.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.571+0000 7f1f94dfe700 1 --2- 172.21.15.163:0/1629837719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c4be0a0d00 0x55c4be0a91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:14.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.571+0000 7f1f8ffff700 1 -- 172.21.15.163:0/1629837719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c4be0a0d00 msgr2=0x55c4be0a91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:14.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.571+0000 7f1f8ffff700 1 --2- 172.21.15.163:0/1629837719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c4be0a0d00 0x55c4be0a91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.571+0000 7f1f8ffff700 1 -- 172.21.15.163:0/1629837719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4be07ff60 msgr2=0x55c4be082b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:14.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.571+0000 7f1f8ffff700 1 --2- 172.21.15.163:0/1629837719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4be07ff60 0x55c4be082b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.571+0000 7f1f8ffff700 1 -- 172.21.15.163:0/1629837719 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f901434f0 con 0x55c4be056a50 2023-10-09T06:37:14.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.571+0000 7f1f8f7fe700 1 --2- 172.21.15.163:0/1629837719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4be07ff60 0x55c4be082b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:14.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.571+0000 7f1f8ffff700 1 --2- 172.21.15.163:0/1629837719 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4be056a50 0x55c4be082620 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000c010 tx=0x7f1f80026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74c55ef8c0e3588 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:14.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.571+0000 7f1f6e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-10-09T06:37:14.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.571+0000 7f1f6e7fc700 1 -- 172.21.15.163:0/1629837719 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8006d180 con 0x55c4be056a50 2023-10-09T06:37:14.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.571+0000 7f1f6e7fc700 1 -- 172.21.15.163:0/1629837719 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1f80011e40 con 0x55c4be056a50 2023-10-09T06:37:14.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.571+0000 7f1f9fb59b80 1 -- 172.21.15.163:0/1629837719 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f88004d10 con 0x55c4be056a50 2023-10-09T06:37:14.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.571+0000 7f1f6e7fc700 1 -- 172.21.15.163:0/1629837719 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f800841a0 con 0x55c4be056a50 2023-10-09T06:37:14.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.571+0000 7f1f9fb59b80 10 client.7305 fetch_fsmap learned FSMap version 75 2023-10-09T06:37:14.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.571+0000 7f1f9fb59b80 1 -- 172.21.15.163:0/1629837719 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x55c4bdf5c470 con 0x55c4be056a50 2023-10-09T06:37:14.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.571+0000 7f1f6e7fc700 1 -- 172.21.15.163:0/1629837719 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8005f070 con 0x55c4be056a50 2023-10-09T06:37:14.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.571+0000 7f1f6e7fc700 1 -- 172.21.15.163:0/1629837719 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 75) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f1f800571b0 con 0x55c4be056a50 2023-10-09T06:37:14.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.571+0000 7f1f9fb59b80 10 client.7305 fetch_fsmap finished waiting for FSMap version 75 2023-10-09T06:37:14.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.571+0000 7f1f9fb59b80 10 client.7305 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-09T06:37:14.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.571+0000 7f1f9fb59b80 20 client.7305 populate_metadata read hostname 'smithi163' 2023-10-09T06:37:14.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.571+0000 7f1f9fb59b80 1 --2- 172.21.15.163:0/1629837719 >> [v2:172.21.15.163:6833/571471083,v1:172.21.15.163:6835/571471083] conn(0x55c4be0830a0 0x55c4be0a96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:14.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.571+0000 7f1f9fb59b80 4 client.7305 mds_command: new command op to 7161 tid=0[{"prefix": "status"}] 2023-10-09T06:37:14.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.571+0000 7f1f9fb59b80 1 -- 172.21.15.163:0/1629837719 --> [v2:172.21.15.163:6833/571471083,v1:172.21.15.163:6835/571471083] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c4bdf49e00 con 0x55c4be0830a0 2023-10-09T06:37:14.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.572+0000 7f1f8f7fe700 1 --2- 172.21.15.163:0/1629837719 >> [v2:172.21.15.163:6833/571471083,v1:172.21.15.163:6835/571471083] conn(0x55c4be0830a0 0x55c4be0a96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:14.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.572+0000 7f1f6e7fc700 10 client.7305 ms_handle_connect on v2:172.21.15.163:6833/571471083 2023-10-09T06:37:14.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.572+0000 7f1f8f7fe700 1 --2- 172.21.15.163:0/1629837719 >> [v2:172.21.15.163:6833/571471083,v1:172.21.15.163:6835/571471083] conn(0x55c4be0830a0 0x55c4be0a96f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3e6798cf0d50f7b server_cookie=80463184ffa3ae91 in_seq=0 out_seq=0 2023-10-09T06:37:14.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.572+0000 7f1f6e7fc700 1 -- 172.21.15.163:0/1629837719 <== mds.0 v2:172.21.15.163:6833/571471083 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55c4bdf49e00 con 0x55c4be0830a0 2023-10-09T06:37:14.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.572+0000 7f1f6e7fc700 10 client.7305 handle_command_reply: tid=0 2023-10-09T06:37:14.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.572+0000 7f1f9fb59b80 1 client.7305 shutdown 2023-10-09T06:37:14.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.572+0000 7f1f9fb59b80 1 -- 172.21.15.163:0/1629837719 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4be056a50 msgr2=0x55c4be082620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:14.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.572+0000 7f1f9fb59b80 1 --2- 172.21.15.163:0/1629837719 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4be056a50 0x55c4be082620 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000c010 tx=0x7f1f80026000 comp rx=0 tx=0).stop 2023-10-09T06:37:14.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.572+0000 7f1f9fb59b80 1 -- 172.21.15.163:0/1629837719 shutdown_connections 2023-10-09T06:37:14.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.572+0000 7f1f9fb59b80 1 --2- 172.21.15.163:0/1629837719 >> [v2:172.21.15.163:6833/571471083,v1:172.21.15.163:6835/571471083] conn(0x55c4be0830a0 0x55c4be0a96f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.572+0000 7f1f9fb59b80 1 --2- 172.21.15.163:0/1629837719 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c4be056a50 0x55c4be082620 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.572+0000 7f1f6e7fc700 0 client.7305 ms_handle_reset on v2:172.21.15.163:6833/571471083 2023-10-09T06:37:14.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.572+0000 7f1f9fb59b80 1 --2- 172.21.15.163:0/1629837719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c4be0a0d00 0x55c4be0a91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.572+0000 7f1f9fb59b80 1 --2- 172.21.15.163:0/1629837719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c4be07ff60 0x55c4be082b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.572+0000 7f1f9fb59b80 1 -- 172.21.15.163:0/1629837719 >> 172.21.15.163:0/1629837719 conn(0x55c4be058370 msgr2=0x55c4be056e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:14.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.572+0000 7f1f9fb59b80 1 -- 172.21.15.163:0/1629837719 shutdown_connections 2023-10-09T06:37:14.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.572+0000 7f1f9fb59b80 1 -- 172.21.15.163:0/1629837719 wait complete. 2023-10-09T06:37:14.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.572+0000 7f1f9fb59b80 20 client.7305 trim_cache size 0 max 16384 2023-10-09T06:37:14.582 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:37:14.582 INFO:teuthology.orchestra.run.smithi163.stdout: "cluster_fsid": "2520b8f7-539f-4bae-9f17-1fce66f66ddd", 2023-10-09T06:37:14.582 INFO:teuthology.orchestra.run.smithi163.stdout: "whoami": 0, 2023-10-09T06:37:14.582 INFO:teuthology.orchestra.run.smithi163.stdout: "id": 7161, 2023-10-09T06:37:14.582 INFO:teuthology.orchestra.run.smithi163.stdout: "want_state": "up:active", 2023-10-09T06:37:14.582 INFO:teuthology.orchestra.run.smithi163.stdout: "state": "up:active", 2023-10-09T06:37:14.582 INFO:teuthology.orchestra.run.smithi163.stdout: "fs_name": "cephfs", 2023-10-09T06:37:14.583 INFO:teuthology.orchestra.run.smithi163.stdout: "rank_uptime": 5.8116591929999997, 2023-10-09T06:37:14.583 INFO:teuthology.orchestra.run.smithi163.stdout: "mdsmap_epoch": 75, 2023-10-09T06:37:14.583 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch": 113, 2023-10-09T06:37:14.583 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch_barrier": 113, 2023-10-09T06:37:14.583 INFO:teuthology.orchestra.run.smithi163.stdout: "uptime": 10.94188359 2023-10-09T06:37:14.583 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:37:14.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.573+0000 7f1f96861700 1 -- 172.21.15.163:0/2900386580 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1f7c067320 msgr2=0x7f1f7c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:14.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.573+0000 7f1f96861700 1 --2- 172.21.15.163:0/2900386580 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1f7c067320 0x7f1f7c0697d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1f84006f00 tx=0x7f1f84008040 comp rx=0 tx=0).stop 2023-10-09T06:37:14.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.573+0000 7f1f96861700 1 -- 172.21.15.163:0/2900386580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f900ac230 msgr2=0x7f1f90145950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:14.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.573+0000 7f1f96861700 1 --2- 172.21.15.163:0/2900386580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f900ac230 0x7f1f90145950 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f1f90064290 tx=0x7f1f80017000 comp rx=0 tx=0).stop 2023-10-09T06:37:14.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.573+0000 7f1f96861700 1 -- 172.21.15.163:0/2900386580 shutdown_connections 2023-10-09T06:37:14.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.573+0000 7f1f96861700 1 --2- 172.21.15.163:0/2900386580 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1f7c067320 0x7f1f7c0697d0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.573+0000 7f1f96861700 1 --2- 172.21.15.163:0/2900386580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f900ac230 0x7f1f90145950 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.573+0000 7f1f96861700 1 --2- 172.21.15.163:0/2900386580 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f901568a0 0x7f1f900cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.573+0000 7f1f96861700 1 --2- 172.21.15.163:0/2900386580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f90155c10 0x7f1f9014d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.573+0000 7f1f96861700 1 -- 172.21.15.163:0/2900386580 >> 172.21.15.163:0/2900386580 conn(0x7f1f9006a320 msgr2=0x7f1f900b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:14.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.573+0000 7f1f96861700 1 -- 172.21.15.163:0/2900386580 shutdown_connections 2023-10-09T06:37:14.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.573+0000 7f1f96861700 1 -- 172.21.15.163:0/2900386580 wait complete. 2023-10-09T06:37:14.586 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-09T06:37:14.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.699+0000 7ff489b78700 1 Processor -- start 2023-10-09T06:37:14.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.699+0000 7ff489b78700 1 -- start start 2023-10-09T06:37:14.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.700+0000 7ff489b78700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4841574d0 0x7ff4840cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:14.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.700+0000 7ff489b78700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff48406c0f0 0x7ff4840b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:14.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.700+0000 7ff489b78700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4840b16a0 con 0x7ff4840cc3c0 2023-10-09T06:37:14.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.700+0000 7ff489b78700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4840b1c60 con 0x7ff48406c0f0 2023-10-09T06:37:14.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.700+0000 7ff489b78700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4841578a0 con 0x7ff4841574d0 2023-10-09T06:37:14.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.700+0000 7ff489377700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff48406c0f0 0x7ff4840b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:14.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.700+0000 7ff489377700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff48406c0f0 0x7ff4840b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56906/0 (socket says 172.21.15.163:56906) 2023-10-09T06:37:14.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.700+0000 7ff489377700 1 -- 172.21.15.163:0/3558756864 learned_addr learned my addr 172.21.15.163:0/3558756864 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:14.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.700+0000 7ff488b76700 1 --2- 172.21.15.163:0/3558756864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4841574d0 0x7ff4840cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:14.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.700+0000 7ff489377700 1 -- 172.21.15.163:0/3558756864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4841574d0 msgr2=0x7ff4840cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:14.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.700+0000 7ff489377700 1 --2- 172.21.15.163:0/3558756864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4841574d0 0x7ff4840cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.700+0000 7ff489377700 1 -- 172.21.15.163:0/3558756864 >> v1:172.21.15.181:6789/0 conn(0x7ff4840cc3c0 legacy=0x7ff48406b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:14.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.700+0000 7ff489377700 1 -- 172.21.15.163:0/3558756864 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4840cdb70 con 0x7ff48406c0f0 2023-10-09T06:37:14.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.700+0000 7ff488b76700 1 --2- 172.21.15.163:0/3558756864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4841574d0 0x7ff4840cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:14.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.700+0000 7ff489377700 1 --2- 172.21.15.163:0/3558756864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff48406c0f0 0x7ff4840b20f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009f20 tx=0x7ff47c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef7df940b0f9d5f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:14.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.701+0000 7ff4837fe700 1 -- 172.21.15.163:0/3558756864 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c00e070 con 0x7ff48406c0f0 2023-10-09T06:37:14.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.701+0000 7ff4837fe700 1 -- 172.21.15.163:0/3558756864 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47c00ba80 con 0x7ff48406c0f0 2023-10-09T06:37:14.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.701+0000 7ff4837fe700 1 -- 172.21.15.163:0/3558756864 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c0092e0 con 0x7ff48406c0f0 2023-10-09T06:37:14.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.701+0000 7ff489b78700 1 -- 172.21.15.163:0/3558756864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff48406c0f0 msgr2=0x7ff4840b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:14.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.701+0000 7ff489b78700 1 --2- 172.21.15.163:0/3558756864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff48406c0f0 0x7ff4840b20f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009f20 tx=0x7ff47c009840 comp rx=0 tx=0).stop 2023-10-09T06:37:14.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.701+0000 7ff489b78700 1 -- 172.21.15.163:0/3558756864 shutdown_connections 2023-10-09T06:37:14.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.701+0000 7ff489b78700 1 --2- 172.21.15.163:0/3558756864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff48406c0f0 0x7ff4840b20f0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.701+0000 7ff489b78700 1 --2- 172.21.15.163:0/3558756864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4841574d0 0x7ff4840cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.701+0000 7ff489b78700 1 -- 172.21.15.163:0/3558756864 >> 172.21.15.163:0/3558756864 conn(0x7ff4840ce430 msgr2=0x7ff4840ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:14.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.701+0000 7ff489b78700 1 -- 172.21.15.163:0/3558756864 shutdown_connections 2023-10-09T06:37:14.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.701+0000 7ff489b78700 1 -- 172.21.15.163:0/3558756864 wait complete. 2023-10-09T06:37:14.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.702+0000 7ff489b78700 1 Processor -- start 2023-10-09T06:37:14.710 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.702+0000 7ff489b78700 1 -- start start 2023-10-09T06:37:14.710 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.703+0000 7ff489b78700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff48406c0f0 0x7ff4840c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:14.710 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.703+0000 7ff489b78700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4840cc3c0 0x7ff48414eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:14.710 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.703+0000 7ff489b78700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4841574d0 0x7ff48414f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:14.710 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.703+0000 7ff489b78700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4840c7ee0 con 0x7ff48406c0f0 2023-10-09T06:37:14.710 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.703+0000 7ff489b78700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4840c81c0 con 0x7ff4840cc3c0 2023-10-09T06:37:14.710 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.703+0000 7ff489b78700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4840c84a0 con 0x7ff4841574d0 2023-10-09T06:37:14.710 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.703+0000 7ff488b76700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff48406c0f0 0x7ff4840c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:14.710 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.703+0000 7ff488b76700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff48406c0f0 0x7ff4840c75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56914/0 (socket says 172.21.15.163:56914) 2023-10-09T06:37:14.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.703+0000 7ff488b76700 1 -- 172.21.15.163:0/3895717354 learned_addr learned my addr 172.21.15.163:0/3895717354 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:14.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.703+0000 7ff489377700 1 --2- 172.21.15.163:0/3895717354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4841574d0 0x7ff48414f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:14.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.703+0000 7ff483fff700 1 --2- 172.21.15.163:0/3895717354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4840cc3c0 0x7ff48414eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:14.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.704+0000 7ff488b76700 1 -- 172.21.15.163:0/3895717354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4841574d0 msgr2=0x7ff48414f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:14.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.704+0000 7ff488b76700 1 --2- 172.21.15.163:0/3895717354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4841574d0 0x7ff48414f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.704+0000 7ff488b76700 1 -- 172.21.15.163:0/3895717354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4840cc3c0 msgr2=0x7ff48414eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:14.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.704+0000 7ff488b76700 1 --2- 172.21.15.163:0/3895717354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4840cc3c0 0x7ff48414eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.704+0000 7ff488b76700 1 -- 172.21.15.163:0/3895717354 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47c009ae0 con 0x7ff48406c0f0 2023-10-09T06:37:14.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.704+0000 7ff488b76700 1 --2- 172.21.15.163:0/3895717354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff48406c0f0 0x7ff4840c75e0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7ff47800d890 tx=0x7ff478008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:14.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.704+0000 7ff480ff9700 1 -- 172.21.15.163:0/3895717354 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff478005a00 con 0x7ff48406c0f0 2023-10-09T06:37:14.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.704+0000 7ff480ff9700 1 -- 172.21.15.163:0/3895717354 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff478005b60 con 0x7ff48406c0f0 2023-10-09T06:37:14.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.704+0000 7ff489b78700 1 -- 172.21.15.163:0/3895717354 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4840c8780 con 0x7ff48406c0f0 2023-10-09T06:37:14.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.704+0000 7ff480ff9700 1 -- 172.21.15.163:0/3895717354 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4780125f0 con 0x7ff48406c0f0 2023-10-09T06:37:14.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.704+0000 7ff489b78700 1 -- 172.21.15.163:0/3895717354 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4840c7b20 con 0x7ff48406c0f0 2023-10-09T06:37:14.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.705+0000 7ff489b78700 1 -- 172.21.15.163:0/3895717354 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff48404c3c0 con 0x7ff48406c0f0 2023-10-09T06:37:14.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.705+0000 7ff480ff9700 1 -- 172.21.15.163:0/3895717354 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff4780129a0 con 0x7ff48406c0f0 2023-10-09T06:37:14.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.706+0000 7ff480ff9700 1 --2- 172.21.15.163:0/3895717354 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff468067320 0x7ff4680697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:14.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.706+0000 7ff480ff9700 1 -- 172.21.15.163:0/3895717354 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff478084d50 con 0x7ff48406c0f0 2023-10-09T06:37:14.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.706+0000 7ff483fff700 1 --2- 172.21.15.163:0/3895717354 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff468067320 0x7ff4680697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:14.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.706+0000 7ff483fff700 1 --2- 172.21.15.163:0/3895717354 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff468067320 0x7ff4680697d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff484156400 tx=0x7ff470007ec0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:14.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.710+0000 7ff480ff9700 1 -- 172.21.15.163:0/3895717354 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff478051a90 con 0x7ff48406c0f0 2023-10-09T06:37:14.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.938+0000 7ff489b78700 1 -- 172.21.15.163:0/3895717354 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff4840cde50 con 0x7ff48406c0f0 2023-10-09T06:37:14.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.938+0000 7ff480ff9700 1 -- 172.21.15.163:0/3895717354 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v75) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff4780550b0 con 0x7ff48406c0f0 2023-10-09T06:37:14.939 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-09T06:37:14.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.940+0000 7ff489b78700 1 -- 172.21.15.163:0/3895717354 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff468067320 msgr2=0x7ff4680697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:14.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.940+0000 7ff489b78700 1 --2- 172.21.15.163:0/3895717354 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff468067320 0x7ff4680697d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff484156400 tx=0x7ff470007ec0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.940+0000 7ff489b78700 1 -- 172.21.15.163:0/3895717354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff48406c0f0 msgr2=0x7ff4840c75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:14.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.940+0000 7ff489b78700 1 --2- 172.21.15.163:0/3895717354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff48406c0f0 0x7ff4840c75e0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7ff47800d890 tx=0x7ff478008040 comp rx=0 tx=0).stop 2023-10-09T06:37:14.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.940+0000 7ff489b78700 1 -- 172.21.15.163:0/3895717354 shutdown_connections 2023-10-09T06:37:14.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.940+0000 7ff489b78700 1 --2- 172.21.15.163:0/3895717354 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff468067320 0x7ff4680697d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.940+0000 7ff489b78700 1 --2- 172.21.15.163:0/3895717354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff48406c0f0 0x7ff4840c75e0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.940+0000 7ff489b78700 1 --2- 172.21.15.163:0/3895717354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4841574d0 0x7ff48414f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.940+0000 7ff489b78700 1 --2- 172.21.15.163:0/3895717354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4840cc3c0 0x7ff48414eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:14.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.940+0000 7ff489b78700 1 -- 172.21.15.163:0/3895717354 >> 172.21.15.163:0/3895717354 conn(0x7ff4840ce430 msgr2=0x7ff4841505b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:14.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.940+0000 7ff489b78700 1 -- 172.21.15.163:0/3895717354 shutdown_connections 2023-10-09T06:37:14.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:14.940+0000 7ff489b78700 1 -- 172.21.15.163:0/3895717354 wait complete. 2023-10-09T06:37:14.953 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-09T06:37:14.953 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-09T06:37:14.953 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi163.front.sepia.ceph.com 2023-10-09T06:37:14.953 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-09T06:37:14.954 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-09T06:37:14.954 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-09T06:37:14.954 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-09T06:37:14.954 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-09T06:37:14.954 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:37:14.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:14.954 DEBUG:teuthology.orchestra.run.smithi163:> ip addr 2023-10-09T06:37:14.969 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:37:14.970 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:37:14.970 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:37:14.970 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:37:14.970 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2023-10-09T06:37:14.970 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:37:14.970 INFO:teuthology.orchestra.run.smithi163.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:37:14.970 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:06 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:37:14.971 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f0 2023-10-09T06:37:14.971 INFO:teuthology.orchestra.run.smithi163.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:37:14.971 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:07 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:37:14.971 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f1 2023-10-09T06:37:14.971 INFO:teuthology.orchestra.run.smithi163.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:37:14.971 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5a brd ff:ff:ff:ff:ff:ff 2023-10-09T06:37:14.971 INFO:teuthology.orchestra.run.smithi163.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-09T06:37:14.972 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5b brd ff:ff:ff:ff:ff:ff 2023-10-09T06:37:14.972 INFO:teuthology.orchestra.run.smithi163.stdout: inet 172.21.15.163/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-09T06:37:14.972 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft 42088sec preferred_lft 42088sec 2023-10-09T06:37:14.972 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::ec4:7aff:fe88:7d5b/64 scope link 2023-10-09T06:37:14.972 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:37:14.972 INFO:teuthology.orchestra.run.smithi163.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-09T06:37:14.972 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 2e:01:84:b9:f4:8e brd ff:ff:ff:ff:ff:ff 2023-10-09T06:37:14.973 INFO:teuthology.orchestra.run.smithi163.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-09T06:37:14.973 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:37:14.973 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::d03c:93ff:fe9e:c8cb/64 scope link 2023-10-09T06:37:14.973 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:37:14.973 INFO:teuthology.orchestra.run.smithi163.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:37:14.973 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 2e:01:84:b9:f4:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-09T06:37:14.973 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::2c01:84ff:feb9:f48e/64 scope link 2023-10-09T06:37:14.974 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:37:14.974 INFO:teuthology.orchestra.run.smithi163.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:37:14.974 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether c6:88:3e:d7:c1:56 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-09T06:37:14.974 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::c488:3eff:fed7:c156/64 scope link 2023-10-09T06:37:14.974 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:37:14.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:14.975 DEBUG:teuthology.orchestra.run.smithi163:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-09T06:37:15.033 INFO:teuthology.orchestra.run.smithi163.stdout:1 2023-10-09T06:37:15.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:15.035 DEBUG:teuthology.orchestra.run.smithi163:> route 2023-10-09T06:37:15.091 INFO:teuthology.orchestra.run.smithi163.stdout:Kernel IP routing table 2023-10-09T06:37:15.091 INFO:teuthology.orchestra.run.smithi163.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-09T06:37:15.091 INFO:teuthology.orchestra.run.smithi163.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-09T06:37:15.091 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-09T06:37:15.091 INFO:teuthology.orchestra.run.smithi163.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-09T06:37:15.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:15.092 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-09T06:37:15.092 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-09T06:37:15.092 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-09T06:37:15.092 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-09T06:37:15.092 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-09T06:37:15.092 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-09T06:37:15.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:15.192 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2023-10-09T06:37:15.206 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-09T06:37:15.207 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-09T06:37:15.207 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:37:15.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:15.207 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set brx.0 up 2023-10-09T06:37:15.271 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:37:15.271 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-09T06:37:15.326 INFO:teuthology.orchestra.run.smithi163.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:37:15.327 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:37:15.327 DEBUG:teuthology.orchestra.run.smithi163:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-09T06:37:15.381 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-09T06:37:15.381 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-09T06:37:15.381 DEBUG:teuthology.orchestra.run.smithi163:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:37:15.460 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:37:15.461 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:37:15.479 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:37:15.479 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:37:15.479 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:37:15.479 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with new device syntax: 0@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:37:15.480 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:37:15.480 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:37:15.480 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:37:15.480 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:37:15.501 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:37:15.502 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:37:15.502 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:37:15.502 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with new device syntax: 0@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:37:15.502 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:37:15.502 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:37:15.502 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:37:15.502 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:37:15.502 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790" 2023-10-09T06:37:15.502 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790:/ 2023-10-09T06:37:15.502 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=2520b8f7-539f-4bae-9f17-1fce66f66ddd" will pass to kernel 2023-10-09T06:37:15.504 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-09T06:37:15.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:15.504 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-09T06:37:15.570 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-09T06:37:15.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.724+0000 7f9c118f8700 1 Processor -- start 2023-10-09T06:37:15.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.724+0000 7f9c118f8700 1 -- start start 2023-10-09T06:37:15.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.725+0000 7f9c118f8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c0c0cbf30 0x7f9c0c0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:15.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.725+0000 7f9c118f8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9c0c0ccf90 0x7f9c0c0b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:15.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.725+0000 7f9c118f8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c0c0cc300 con 0x7f9c0c15a270 2023-10-09T06:37:15.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.725+0000 7f9c118f8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c0c156bf0 con 0x7f9c0c0ccf90 2023-10-09T06:37:15.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.725+0000 7f9c118f8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0c0ae950 con 0x7f9c0c0cbf30 2023-10-09T06:37:15.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.726+0000 7f9c0affd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c0c0cbf30 0x7f9c0c0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:15.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.726+0000 7f9c027fc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9c0c0ccf90 0x7f9c0c0b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:15.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.726+0000 7f9c0affd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c0c0cbf30 0x7f9c0c0ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:59066/0 (socket says 172.21.15.163:59066) 2023-10-09T06:37:15.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.726+0000 7f9c0affd700 1 -- 172.21.15.163:0/4037367253 learned_addr learned my addr 172.21.15.163:0/4037367253 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:15.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.726+0000 7f9c0affd700 1 -- 172.21.15.163:0/4037367253 >> v2:172.21.15.181:3300/0 conn(0x7f9c0c0ccf90 msgr2=0x7f9c0c0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:15.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.726+0000 7f9c0affd700 1 --2- 172.21.15.163:0/4037367253 >> v2:172.21.15.181:3300/0 conn(0x7f9c0c0ccf90 0x7f9c0c0b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:15.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.726+0000 7f9c0affd700 1 -- 172.21.15.163:0/4037367253 >> v1:172.21.15.181:6789/0 conn(0x7f9c0c15a270 legacy=0x7f9c0c0c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:15.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.726+0000 7f9c0affd700 1 -- 172.21.15.163:0/4037367253 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0c0ad370 con 0x7f9c0c0cbf30 2023-10-09T06:37:15.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.726+0000 7f9c0a7fc700 1 -- 172.21.15.163:0/4037367253 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695265136 0 0) 0x7f9c0c0cc300 con 0x7f9c0c15a270 2023-10-09T06:37:15.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.726+0000 7f9c027fc700 1 --2- 172.21.15.163:0/4037367253 >> v2:172.21.15.181:3300/0 conn(0x7f9c0c0ccf90 0x7f9c0c0b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:37:15.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.726+0000 7f9c0affd700 1 --2- 172.21.15.163:0/4037367253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c0c0cbf30 0x7f9c0c0ae410 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0099b0 tx=0x7f9bfc009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c82006243c7e976 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:15.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.726+0000 7f9c0a7fc700 1 -- 172.21.15.163:0/4037367253 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc009ea0 con 0x7f9c0c0cbf30 2023-10-09T06:37:15.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.726+0000 7f9c0a7fc700 1 -- 172.21.15.163:0/4037367253 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bfc00b860 con 0x7f9c0c0cbf30 2023-10-09T06:37:15.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.726+0000 7f9c0a7fc700 1 -- 172.21.15.163:0/4037367253 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc00bc00 con 0x7f9c0c0cbf30 2023-10-09T06:37:15.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.727+0000 7f9c118f8700 1 -- 172.21.15.163:0/4037367253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c0c0cbf30 msgr2=0x7f9c0c0ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:15.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.727+0000 7f9c118f8700 1 --2- 172.21.15.163:0/4037367253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c0c0cbf30 0x7f9c0c0ae410 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0099b0 tx=0x7f9bfc009250 comp rx=0 tx=0).stop 2023-10-09T06:37:15.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.727+0000 7f9c118f8700 1 -- 172.21.15.163:0/4037367253 shutdown_connections 2023-10-09T06:37:15.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.727+0000 7f9c118f8700 1 --2- 172.21.15.163:0/4037367253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c0c0cbf30 0x7f9c0c0ae410 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:15.730 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.727+0000 7f9c118f8700 1 --2- 172.21.15.163:0/4037367253 >> v2:172.21.15.181:3300/0 conn(0x7f9c0c0ccf90 0x7f9c0c0b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:15.730 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.727+0000 7f9c118f8700 1 -- 172.21.15.163:0/4037367253 >> 172.21.15.163:0/4037367253 conn(0x7f9c0c06a320 msgr2=0x7f9c0c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:15.730 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.727+0000 7f9c118f8700 1 -- 172.21.15.163:0/4037367253 shutdown_connections 2023-10-09T06:37:15.730 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.727+0000 7f9c118f8700 1 -- 172.21.15.163:0/4037367253 wait complete. 2023-10-09T06:37:15.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.728+0000 7f9c118f8700 1 Processor -- start 2023-10-09T06:37:15.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.728+0000 7f9c118f8700 1 -- start start 2023-10-09T06:37:15.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.729+0000 7f9c118f8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c0c0cbf30 0x7f9c0c158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:15.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.729+0000 7f9c118f8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c0c0ccf90 0x7f9c0c14f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:15.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.729+0000 7f9c118f8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c0c15a270 0x7f9c0c14fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:15.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.729+0000 7f9c118f8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0c154dc0 con 0x7f9c0c0cbf30 2023-10-09T06:37:15.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.729+0000 7f9c118f8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0c154f50 con 0x7f9c0c15a270 2023-10-09T06:37:15.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.729+0000 7f9c118f8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0c14ff80 con 0x7f9c0c0ccf90 2023-10-09T06:37:15.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.729+0000 7f9c0affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c0c0cbf30 0x7f9c0c158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:15.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.729+0000 7f9c0affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c0c0cbf30 0x7f9c0c158f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56924/0 (socket says 172.21.15.163:56924) 2023-10-09T06:37:15.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.729+0000 7f9c0affd700 1 -- 172.21.15.163:0/3770820801 learned_addr learned my addr 172.21.15.163:0/3770820801 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:15.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.729+0000 7f9c0b7fe700 1 --2- 172.21.15.163:0/3770820801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c0c15a270 0x7f9c0c14fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:15.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.729+0000 7f9c027fc700 1 --2- 172.21.15.163:0/3770820801 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c0c0ccf90 0x7f9c0c14f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:15.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.729+0000 7f9c0affd700 1 -- 172.21.15.163:0/3770820801 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c0c0ccf90 msgr2=0x7f9c0c14f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:15.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.729+0000 7f9c0affd700 1 --2- 172.21.15.163:0/3770820801 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c0c0ccf90 0x7f9c0c14f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:15.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.729+0000 7f9c0affd700 1 -- 172.21.15.163:0/3770820801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c0c15a270 msgr2=0x7f9c0c14fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:15.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.729+0000 7f9c0affd700 1 --2- 172.21.15.163:0/3770820801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c0c15a270 0x7f9c0c14fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:15.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.729+0000 7f9c0affd700 1 -- 172.21.15.163:0/3770820801 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf4009420 con 0x7f9c0c0cbf30 2023-10-09T06:37:15.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.729+0000 7f9c0b7fe700 1 --2- 172.21.15.163:0/3770820801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c0c15a270 0x7f9c0c14fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:15.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.730+0000 7f9c0affd700 1 --2- 172.21.15.163:0/3770820801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c0c0cbf30 0x7f9c0c158f90 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00b3c0 tx=0x7f9bfc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:15.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.730+0000 7f9c037fe700 1 -- 172.21.15.163:0/3770820801 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc00bbe0 con 0x7f9c0c0cbf30 2023-10-09T06:37:15.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.730+0000 7f9c037fe700 1 -- 172.21.15.163:0/3770820801 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bfc011420 con 0x7f9c0c0cbf30 2023-10-09T06:37:15.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.730+0000 7f9c118f8700 1 -- 172.21.15.163:0/3770820801 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bfc0094f0 con 0x7f9c0c0cbf30 2023-10-09T06:37:15.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.730+0000 7f9c037fe700 1 -- 172.21.15.163:0/3770820801 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc0117a0 con 0x7f9c0c0cbf30 2023-10-09T06:37:15.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.730+0000 7f9c118f8700 1 -- 172.21.15.163:0/3770820801 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c0c1504d0 con 0x7f9c0c0cbf30 2023-10-09T06:37:15.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.731+0000 7f9c118f8700 1 -- 172.21.15.163:0/3770820801 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c0c04c3c0 con 0x7f9c0c0cbf30 2023-10-09T06:37:15.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.731+0000 7f9c037fe700 1 -- 172.21.15.163:0/3770820801 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9bfc011b10 con 0x7f9c0c0cbf30 2023-10-09T06:37:15.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.731+0000 7f9c037fe700 1 --2- 172.21.15.163:0/3770820801 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9bec067320 0x7f9bec0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:15.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.732+0000 7f9c037fe700 1 -- 172.21.15.163:0/3770820801 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9bfc086520 con 0x7f9c0c0cbf30 2023-10-09T06:37:15.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.732+0000 7f9c027fc700 1 --2- 172.21.15.163:0/3770820801 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9bec067320 0x7f9bec0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:15.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.732+0000 7f9c027fc700 1 --2- 172.21.15.163:0/3770820801 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9bec067320 0x7f9bec0697d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40093f0 tx=0x7f9bf4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:15.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.736+0000 7f9c037fe700 1 -- 172.21.15.163:0/3770820801 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9bfc0531b0 con 0x7f9c0c0cbf30 2023-10-09T06:37:15.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.964+0000 7f9c118f8700 1 -- 172.21.15.163:0/3770820801 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9c0c06f8a0 con 0x7f9c0c0cbf30 2023-10-09T06:37:15.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.965+0000 7f9c037fe700 1 -- 172.21.15.163:0/3770820801 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v75) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9bfc0567d0 con 0x7f9c0c0cbf30 2023-10-09T06:37:15.966 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-09T06:37:15.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.967+0000 7f9c118f8700 1 -- 172.21.15.163:0/3770820801 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9bec067320 msgr2=0x7f9bec0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:15.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.967+0000 7f9c118f8700 1 --2- 172.21.15.163:0/3770820801 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9bec067320 0x7f9bec0697d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40093f0 tx=0x7f9bf4008040 comp rx=0 tx=0).stop 2023-10-09T06:37:15.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.967+0000 7f9c118f8700 1 -- 172.21.15.163:0/3770820801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c0c0cbf30 msgr2=0x7f9c0c158f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:15.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.967+0000 7f9c118f8700 1 --2- 172.21.15.163:0/3770820801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c0c0cbf30 0x7f9c0c158f90 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00b3c0 tx=0x7f9bfc017000 comp rx=0 tx=0).stop 2023-10-09T06:37:15.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.967+0000 7f9c118f8700 1 -- 172.21.15.163:0/3770820801 shutdown_connections 2023-10-09T06:37:15.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.967+0000 7f9c118f8700 1 --2- 172.21.15.163:0/3770820801 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9bec067320 0x7f9bec0697d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:15.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.967+0000 7f9c118f8700 1 --2- 172.21.15.163:0/3770820801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c0c0cbf30 0x7f9c0c158f90 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:15.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.967+0000 7f9c118f8700 1 --2- 172.21.15.163:0/3770820801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c0c15a270 0x7f9c0c14fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:15.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.967+0000 7f9c118f8700 1 --2- 172.21.15.163:0/3770820801 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c0c0ccf90 0x7f9c0c14f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:15.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.967+0000 7f9c118f8700 1 -- 172.21.15.163:0/3770820801 >> 172.21.15.163:0/3770820801 conn(0x7f9c0c06a320 msgr2=0x7f9c0c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:15.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.967+0000 7f9c118f8700 1 -- 172.21.15.163:0/3770820801 shutdown_connections 2023-10-09T06:37:15.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:15.967+0000 7f9c118f8700 1 -- 172.21.15.163:0/3770820801 wait complete. 2023-10-09T06:37:15.980 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-09T06:37:15.980 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-09T06:37:15.980 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-09T06:37:15.980 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-09T06:37:15.980 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-09T06:37:15.980 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-09T06:37:15.980 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-09T06:37:15.980 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-09T06:37:15.980 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:37:15.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:15.981 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-09T06:37:15.999 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:37:16.000 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:37:16.000 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:37:16.000 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:37:16.000 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-09T06:37:16.000 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:37:16.000 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:37:16.000 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:37:16.000 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2023-10-09T06:37:16.000 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:37:16.000 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:37:16.001 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2023-10-09T06:37:16.001 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:37:16.001 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-09T06:37:16.001 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-09T06:37:16.001 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-09T06:37:16.001 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-09T06:37:16.001 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42064sec preferred_lft 42064sec 2023-10-09T06:37:16.001 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2023-10-09T06:37:16.001 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:37:16.001 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-09T06:37:16.001 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 72:62:e6:72:7c:2e brd ff:ff:ff:ff:ff:ff 2023-10-09T06:37:16.001 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-09T06:37:16.002 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:37:16.002 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::e89a:4cff:fefb:7198/64 scope link 2023-10-09T06:37:16.002 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:37:16.002 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:37:16.002 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 72:62:e6:72:7c:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-09T06:37:16.002 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::7062:e6ff:fe72:7c2e/64 scope link 2023-10-09T06:37:16.002 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:37:16.002 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:37:16.002 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether fa:4e:a3:9d:5a:5f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-09T06:37:16.002 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::f84e:a3ff:fe9d:5a5f/64 scope link 2023-10-09T06:37:16.002 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:37:16.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:16.003 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-09T06:37:16.067 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-09T06:37:16.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:16.069 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-09T06:37:16.126 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-09T06:37:16.126 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-09T06:37:16.127 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-09T06:37:16.127 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-09T06:37:16.127 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-09T06:37:16.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:16.127 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-09T06:37:16.127 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-09T06:37:16.127 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-09T06:37:16.127 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-09T06:37:16.128 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-09T06:37:16.128 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-09T06:37:16.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:16.232 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-09T06:37:16.250 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-09T06:37:16.250 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-09T06:37:16.250 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:37:16.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:16.250 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-09T06:37:16.316 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:37:16.316 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-09T06:37:16.370 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:37:16.371 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:37:16.371 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-09T06:37:16.425 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-09T06:37:16.425 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-09T06:37:16.425 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:37:16.504 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:37:16.504 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:37:16.523 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:37:16.523 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:37:16.523 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:37:16.523 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 1@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:37:16.523 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:37:16.523 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:37:16.523 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:37:16.523 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:37:16.545 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:37:16.545 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:37:16.545 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:37:16.545 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 1@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:37:16.545 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:37:16.545 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:37:16.545 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:37:16.546 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:37:16.546 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790" 2023-10-09T06:37:16.546 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790:/ 2023-10-09T06:37:16.546 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=2520b8f7-539f-4bae-9f17-1fce66f66ddd" will pass to kernel 2023-10-09T06:37:16.547 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-09T06:37:16.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:16.547 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-09T06:37:16.612 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-09T06:37:16.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.733+0000 7f4827f55700 1 Processor -- start 2023-10-09T06:37:16.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.733+0000 7f4827f55700 1 -- start start 2023-10-09T06:37:16.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.734+0000 7f4827f55700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48200cd910 0x7f48200ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:16.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.734+0000 7f4827f55700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48200b5b00 con 0x7f48200b55a0 2023-10-09T06:37:16.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.734+0000 7f4827f55700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48200b5da0 con 0x7f4820159ec0 2023-10-09T06:37:16.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.734+0000 7f4827f55700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48200b6080 con 0x7f48200cd910 2023-10-09T06:37:16.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.734+0000 7f48264f2700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f48200b55a0 0x7f48200ce880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:54652/0 (socket says 172.21.15.163:54652) 2023-10-09T06:37:16.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.734+0000 7f48264f2700 1 -- 172.21.15.163:0/2179078597 learned_addr learned my addr 172.21.15.163:0/2179078597 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:16.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.734+0000 7f48254f0700 1 --2- 172.21.15.163:0/2179078597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48200cd910 0x7f48200ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:16.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.734+0000 7f4824cef700 1 -- 172.21.15.163:0/2179078597 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4260199383 0 0) 0x7f48200b5b00 con 0x7f48200b55a0 2023-10-09T06:37:16.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.734+0000 7f4824cef700 1 -- 172.21.15.163:0/2179078597 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f480c003570 con 0x7f48200b55a0 2023-10-09T06:37:16.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.734+0000 7f4824cef700 1 -- 172.21.15.163:0/2179078597 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 803047429 0 0) 0x7f48200b5da0 con 0x7f4820159ec0 2023-10-09T06:37:16.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.734+0000 7f4824cef700 1 -- 172.21.15.163:0/2179078597 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48200b5b00 con 0x7f4820159ec0 2023-10-09T06:37:16.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.735+0000 7f48254f0700 1 -- 172.21.15.163:0/2179078597 >> v1:172.21.15.181:6789/0 conn(0x7f4820159ec0 legacy=0x7f48200b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:16.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.735+0000 7f48254f0700 1 -- 172.21.15.163:0/2179078597 >> v1:172.21.15.163:6789/0 conn(0x7f48200b55a0 legacy=0x7f48200ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:16.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.735+0000 7f4824cef700 1 -- 172.21.15.163:0/2179078597 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2248870220 0 0) 0x7f480c003570 con 0x7f48200b55a0 2023-10-09T06:37:16.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.735+0000 7f48254f0700 1 -- 172.21.15.163:0/2179078597 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48200cc100 con 0x7f48200cd910 2023-10-09T06:37:16.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.735+0000 7f48254f0700 1 --2- 172.21.15.163:0/2179078597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48200cd910 0x7f48200ce340 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4810009910 tx=0x7f48100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b0bc0b055016882 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:16.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.735+0000 7f4824cef700 1 -- 172.21.15.163:0/2179078597 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c001470 con 0x7f48200cd910 2023-10-09T06:37:16.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.735+0000 7f4824cef700 1 -- 172.21.15.163:0/2179078597 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4810011420 con 0x7f48200cd910 2023-10-09T06:37:16.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.735+0000 7f4824cef700 1 -- 172.21.15.163:0/2179078597 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48100117c0 con 0x7f48200cd910 2023-10-09T06:37:16.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.735+0000 7f4827f55700 1 -- 172.21.15.163:0/2179078597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48200cd910 msgr2=0x7f48200ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:16.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.735+0000 7f4827f55700 1 --2- 172.21.15.163:0/2179078597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48200cd910 0x7f48200ce340 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4810009910 tx=0x7f48100091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:16.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.735+0000 7f4827f55700 1 -- 172.21.15.163:0/2179078597 shutdown_connections 2023-10-09T06:37:16.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.735+0000 7f4827f55700 1 --2- 172.21.15.163:0/2179078597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48200cd910 0x7f48200ce340 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:16.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.735+0000 7f4827f55700 1 -- 172.21.15.163:0/2179078597 >> 172.21.15.163:0/2179078597 conn(0x7f48200999f0 msgr2=0x7f48200b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:16.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.736+0000 7f4827f55700 1 -- 172.21.15.163:0/2179078597 shutdown_connections 2023-10-09T06:37:16.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.736+0000 7f4827f55700 1 -- 172.21.15.163:0/2179078597 wait complete. 2023-10-09T06:37:16.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.737+0000 7f4827f55700 1 Processor -- start 2023-10-09T06:37:16.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.737+0000 7f4827f55700 1 -- start start 2023-10-09T06:37:16.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.737+0000 7f4827f55700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48200b55a0 0x7f482006ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:16.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.737+0000 7f4827f55700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4820159ec0 0x7f482015f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:16.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.737+0000 7f4827f55700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4820069060 0x7f482014adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:16.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.737+0000 7f4827f55700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482015e1f0 con 0x7f4820069060 2023-10-09T06:37:16.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.737+0000 7f4827f55700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482015e490 con 0x7f48200b55a0 2023-10-09T06:37:16.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.737+0000 7f4827f55700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f482015e770 con 0x7f4820159ec0 2023-10-09T06:37:16.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.737+0000 7f48264f2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4820069060 0x7f482014adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:16.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.737+0000 7f48264f2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4820069060 0x7f482014adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56940/0 (socket says 172.21.15.163:56940) 2023-10-09T06:37:16.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.737+0000 7f48264f2700 1 -- 172.21.15.163:0/4001353296 learned_addr learned my addr 172.21.15.163:0/4001353296 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:16.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.737+0000 7f48254f0700 1 --2- 172.21.15.163:0/4001353296 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4820159ec0 0x7f482015f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:16.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.737+0000 7f4825cf1700 1 --2- 172.21.15.163:0/4001353296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48200b55a0 0x7f482006ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:16.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.738+0000 7f48264f2700 1 -- 172.21.15.163:0/4001353296 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4820159ec0 msgr2=0x7f482015f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:16.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.738+0000 7f48264f2700 1 --2- 172.21.15.163:0/4001353296 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4820159ec0 0x7f482015f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:16.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.738+0000 7f48264f2700 1 -- 172.21.15.163:0/4001353296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48200b55a0 msgr2=0x7f482006ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:16.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.738+0000 7f48264f2700 1 --2- 172.21.15.163:0/4001353296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48200b55a0 0x7f482006ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:16.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.738+0000 7f48264f2700 1 -- 172.21.15.163:0/4001353296 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4810009420 con 0x7f4820069060 2023-10-09T06:37:16.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.738+0000 7f48264f2700 1 --2- 172.21.15.163:0/4001353296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4820069060 0x7f482014adb0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f481c00c7f0 tx=0x7f481c001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:16.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.738+0000 7f4815ffb700 1 -- 172.21.15.163:0/4001353296 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c007360 con 0x7f4820069060 2023-10-09T06:37:16.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.738+0000 7f4815ffb700 1 -- 172.21.15.163:0/4001353296 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481c007500 con 0x7f4820069060 2023-10-09T06:37:16.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.738+0000 7f4827f55700 1 -- 172.21.15.163:0/4001353296 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f482015ea50 con 0x7f4820069060 2023-10-09T06:37:16.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.738+0000 7f4815ffb700 1 -- 172.21.15.163:0/4001353296 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481c004a80 con 0x7f4820069060 2023-10-09T06:37:16.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.738+0000 7f4827f55700 1 -- 172.21.15.163:0/4001353296 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48200694c0 con 0x7f4820069060 2023-10-09T06:37:16.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.739+0000 7f4827f55700 1 -- 172.21.15.163:0/4001353296 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4820063c90 con 0x7f4820069060 2023-10-09T06:37:16.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.739+0000 7f4815ffb700 1 -- 172.21.15.163:0/4001353296 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f481c01c3b0 con 0x7f4820069060 2023-10-09T06:37:16.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.740+0000 7f4815ffb700 1 --2- 172.21.15.163:0/4001353296 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4804067320 0x7f48040697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:16.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.740+0000 7f4815ffb700 1 -- 172.21.15.163:0/4001353296 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f481c084380 con 0x7f4820069060 2023-10-09T06:37:16.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.740+0000 7f4825cf1700 1 --2- 172.21.15.163:0/4001353296 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4804067320 0x7f48040697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:16.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.740+0000 7f4825cf1700 1 --2- 172.21.15.163:0/4001353296 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4804067320 0x7f48040697d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f481800f440 tx=0x7f4818002b70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:16.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.744+0000 7f4815ffb700 1 -- 172.21.15.163:0/4001353296 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f481c051090 con 0x7f4820069060 2023-10-09T06:37:16.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:16.974+0000 7f4827f55700 1 -- 172.21.15.163:0/4001353296 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f48200718b0 con 0x7f4820069060 2023-10-09T06:37:17.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.797+0000 7f4815ffb700 1 -- 172.21.15.163:0/4001353296 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v76) v1 ==== 136+0+0 (secure 0 0 0) 0x7f481c0546b0 con 0x7f4820069060 2023-10-09T06:37:17.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.800+0000 7f4827f55700 1 -- 172.21.15.163:0/4001353296 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4804067320 msgr2=0x7f48040697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:17.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.800+0000 7f4827f55700 1 --2- 172.21.15.163:0/4001353296 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4804067320 0x7f48040697d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f481800f440 tx=0x7f4818002b70 comp rx=0 tx=0).stop 2023-10-09T06:37:17.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.800+0000 7f4827f55700 1 -- 172.21.15.163:0/4001353296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4820069060 msgr2=0x7f482014adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:17.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.800+0000 7f4827f55700 1 --2- 172.21.15.163:0/4001353296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4820069060 0x7f482014adb0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f481c00c7f0 tx=0x7f481c001a30 comp rx=0 tx=0).stop 2023-10-09T06:37:17.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.800+0000 7f4827f55700 1 -- 172.21.15.163:0/4001353296 shutdown_connections 2023-10-09T06:37:17.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.800+0000 7f4827f55700 1 --2- 172.21.15.163:0/4001353296 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4804067320 0x7f48040697d0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:17.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.800+0000 7f4827f55700 1 --2- 172.21.15.163:0/4001353296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4820069060 0x7f482014adb0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:17.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.800+0000 7f4827f55700 1 --2- 172.21.15.163:0/4001353296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48200b55a0 0x7f482006ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:17.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.800+0000 7f4827f55700 1 --2- 172.21.15.163:0/4001353296 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4820159ec0 0x7f482015f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:17.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.800+0000 7f4827f55700 1 -- 172.21.15.163:0/4001353296 >> 172.21.15.163:0/4001353296 conn(0x7f48200999f0 msgr2=0x7f482015c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:17.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.800+0000 7f4827f55700 1 -- 172.21.15.163:0/4001353296 shutdown_connections 2023-10-09T06:37:17.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.800+0000 7f4827f55700 1 -- 172.21.15.163:0/4001353296 wait complete. 2023-10-09T06:37:17.815 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-09T06:37:17.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.932+0000 7f2d6ec72700 1 Processor -- start 2023-10-09T06:37:17.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.932+0000 7f2d6ec72700 1 -- start start 2023-10-09T06:37:17.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.933+0000 7f2d6ec72700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2d68159840 0x7f2d680cb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:17.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.933+0000 7f2d6ec72700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2d680cba70 0x7f2d680694a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:17.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.933+0000 7f2d6ec72700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d68159c10 0x7f2d680699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:17.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.933+0000 7f2d6ec72700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d680cfb30 con 0x7f2d680cba70 2023-10-09T06:37:17.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.933+0000 7f2d6ec72700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d680cbf10 con 0x7f2d68159840 2023-10-09T06:37:17.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.933+0000 7f2d6ec72700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d680cf4e0 con 0x7f2d68159c10 2023-10-09T06:37:17.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.933+0000 7f2d67fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2d680cba70 0x7f2d680694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:17.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.933+0000 7f2d67fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2d680cba70 0x7f2d680694a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56956/0 (socket says 172.21.15.163:56956) 2023-10-09T06:37:17.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.933+0000 7f2d67fff700 1 -- 172.21.15.163:0/2404633706 learned_addr learned my addr 172.21.15.163:0/2404633706 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:17.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.933+0000 7f2d6d20f700 1 --2- 172.21.15.163:0/2404633706 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d68159c10 0x7f2d680699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:17.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.933+0000 7f2d6ca0e700 1 --2- 172.21.15.163:0/2404633706 >> v2:172.21.15.181:3300/0 conn(0x7f2d68159840 0x7f2d680cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:17.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.934+0000 7f2d67fff700 1 -- 172.21.15.163:0/2404633706 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d68159c10 msgr2=0x7f2d680699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:17.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.934+0000 7f2d67fff700 1 --2- 172.21.15.163:0/2404633706 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d68159c10 0x7f2d680699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:17.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.934+0000 7f2d67fff700 1 -- 172.21.15.163:0/2404633706 >> v2:172.21.15.181:3300/0 conn(0x7f2d68159840 msgr2=0x7f2d680cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:17.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.934+0000 7f2d67fff700 1 --2- 172.21.15.163:0/2404633706 >> v2:172.21.15.181:3300/0 conn(0x7f2d68159840 0x7f2d680cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:17.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.934+0000 7f2d67fff700 1 -- 172.21.15.163:0/2404633706 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d680cf7c0 con 0x7f2d680cba70 2023-10-09T06:37:17.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.934+0000 7f2d67fff700 1 --2- 172.21.15.163:0/2404633706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d680cba70 0x7f2d680694a0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c0099b0 tx=0x7f2d5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a04665085a28a2c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:17.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.934+0000 7f2d677fe700 1 -- 172.21.15.163:0/2404633706 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c00b820 con 0x7f2d680cba70 2023-10-09T06:37:17.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.934+0000 7f2d677fe700 1 -- 172.21.15.163:0/2404633706 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5c00b980 con 0x7f2d680cba70 2023-10-09T06:37:17.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.934+0000 7f2d677fe700 1 -- 172.21.15.163:0/2404633706 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c00bd40 con 0x7f2d680cba70 2023-10-09T06:37:17.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.934+0000 7f2d6ec72700 1 -- 172.21.15.163:0/2404633706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d680cba70 msgr2=0x7f2d680694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:17.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.934+0000 7f2d6ec72700 1 --2- 172.21.15.163:0/2404633706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d680cba70 0x7f2d680694a0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c0099b0 tx=0x7f2d5c009250 comp rx=0 tx=0).stop 2023-10-09T06:37:17.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.934+0000 7f2d6ec72700 1 -- 172.21.15.163:0/2404633706 shutdown_connections 2023-10-09T06:37:17.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.934+0000 7f2d6ec72700 1 --2- 172.21.15.163:0/2404633706 >> v2:172.21.15.181:3300/0 conn(0x7f2d68159840 0x7f2d680cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:17.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.934+0000 7f2d6ec72700 1 --2- 172.21.15.163:0/2404633706 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d68159c10 0x7f2d680699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:17.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.934+0000 7f2d6ec72700 1 --2- 172.21.15.163:0/2404633706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d680cba70 0x7f2d680694a0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:17.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.934+0000 7f2d6ec72700 1 -- 172.21.15.163:0/2404633706 >> 172.21.15.163:0/2404633706 conn(0x7f2d68068e10 msgr2=0x7f2d680d0760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:17.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.934+0000 7f2d6ec72700 1 -- 172.21.15.163:0/2404633706 shutdown_connections 2023-10-09T06:37:17.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.934+0000 7f2d6ec72700 1 -- 172.21.15.163:0/2404633706 wait complete. 2023-10-09T06:37:17.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.935+0000 7f2d6ec72700 1 Processor -- start 2023-10-09T06:37:17.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.936+0000 7f2d6ec72700 1 -- start start 2023-10-09T06:37:17.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.936+0000 7f2d6ec72700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d680cba70 0x7f2d68155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:17.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.936+0000 7f2d6ec72700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d68159840 0x7f2d68169830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:17.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.936+0000 7f2d6ec72700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d68159c10 0x7f2d68143080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:17.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.936+0000 7f2d6ec72700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d68156530 con 0x7f2d68159840 2023-10-09T06:37:17.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.936+0000 7f2d6ec72700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d681515e0 con 0x7f2d680cba70 2023-10-09T06:37:17.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.936+0000 7f2d6ec72700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d681518c0 con 0x7f2d68159c10 2023-10-09T06:37:17.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.936+0000 7f2d67fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d68159840 0x7f2d68169830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:17.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.936+0000 7f2d67fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d68159840 0x7f2d68169830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56972/0 (socket says 172.21.15.163:56972) 2023-10-09T06:37:17.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.936+0000 7f2d67fff700 1 -- 172.21.15.163:0/4115332101 learned_addr learned my addr 172.21.15.163:0/4115332101 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:17.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.937+0000 7f2d6d20f700 1 --2- 172.21.15.163:0/4115332101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d68159c10 0x7f2d68143080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:17.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.937+0000 7f2d6ca0e700 1 --2- 172.21.15.163:0/4115332101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d680cba70 0x7f2d68155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:17.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.937+0000 7f2d67fff700 1 -- 172.21.15.163:0/4115332101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d68159c10 msgr2=0x7f2d68143080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:17.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.937+0000 7f2d67fff700 1 --2- 172.21.15.163:0/4115332101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d68159c10 0x7f2d68143080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:17.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.937+0000 7f2d67fff700 1 -- 172.21.15.163:0/4115332101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d680cba70 msgr2=0x7f2d68155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:17.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.937+0000 7f2d67fff700 1 --2- 172.21.15.163:0/4115332101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d680cba70 0x7f2d68155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:17.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.937+0000 7f2d67fff700 1 -- 172.21.15.163:0/4115332101 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d5c0094f0 con 0x7f2d68159840 2023-10-09T06:37:17.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.937+0000 7f2d6ca0e700 1 --2- 172.21.15.163:0/4115332101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d680cba70 0x7f2d68155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:17.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.937+0000 7f2d67fff700 1 --2- 172.21.15.163:0/4115332101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d68159840 0x7f2d68169830 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f2d68067c20 tx=0x7f2d5c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:17.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.937+0000 7f2d64ff9700 1 -- 172.21.15.163:0/4115332101 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c01f070 con 0x7f2d68159840 2023-10-09T06:37:17.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.937+0000 7f2d64ff9700 1 -- 172.21.15.163:0/4115332101 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5c002f30 con 0x7f2d68159840 2023-10-09T06:37:17.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.937+0000 7f2d6ec72700 1 -- 172.21.15.163:0/4115332101 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d68151ba0 con 0x7f2d68159840 2023-10-09T06:37:17.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.937+0000 7f2d64ff9700 1 -- 172.21.15.163:0/4115332101 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c011b90 con 0x7f2d68159840 2023-10-09T06:37:17.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.937+0000 7f2d6ec72700 1 -- 172.21.15.163:0/4115332101 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6814f820 con 0x7f2d68159840 2023-10-09T06:37:17.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.938+0000 7f2d6ec72700 1 -- 172.21.15.163:0/4115332101 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d68064550 con 0x7f2d68159840 2023-10-09T06:37:17.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.938+0000 7f2d64ff9700 1 -- 172.21.15.163:0/4115332101 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2d5c023070 con 0x7f2d68159840 2023-10-09T06:37:17.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.939+0000 7f2d64ff9700 1 --2- 172.21.15.163:0/4115332101 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2d54067320 0x7f2d540697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:17.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.939+0000 7f2d64ff9700 1 -- 172.21.15.163:0/4115332101 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2d5c085f50 con 0x7f2d68159840 2023-10-09T06:37:17.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.939+0000 7f2d6ca0e700 1 --2- 172.21.15.163:0/4115332101 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2d54067320 0x7f2d540697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:17.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.939+0000 7f2d6ca0e700 1 --2- 172.21.15.163:0/4115332101 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2d54067320 0x7f2d540697d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2d580075e0 tx=0x7f2d580097e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:17.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:17.943+0000 7f2d64ff9700 1 -- 172.21.15.163:0/4115332101 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d5c052be0 con 0x7f2d68159840 2023-10-09T06:37:18.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.149+0000 7f2d6ec72700 1 -- 172.21.15.163:0/4115332101 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d6804c3c0 con 0x7f2d54067320 2023-10-09T06:37:18.152 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.151+0000 7f2d64ff9700 1 -- 172.21.15.163:0/4115332101 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f2d6804c3c0 con 0x7f2d54067320 2023-10-09T06:37:18.152 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:37:18.152 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:18.152 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "cephfs" 2023-10-09T06:37:18.152 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:18.153 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:37:18.154 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.153+0000 7f2d6ec72700 1 -- 172.21.15.163:0/4115332101 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2d54067320 msgr2=0x7f2d540697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:18.154 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.153+0000 7f2d6ec72700 1 --2- 172.21.15.163:0/4115332101 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2d54067320 0x7f2d540697d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2d580075e0 tx=0x7f2d580097e0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.153+0000 7f2d6ec72700 1 -- 172.21.15.163:0/4115332101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d68159840 msgr2=0x7f2d68169830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:18.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.153+0000 7f2d6ec72700 1 --2- 172.21.15.163:0/4115332101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d68159840 0x7f2d68169830 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f2d68067c20 tx=0x7f2d5c017000 comp rx=0 tx=0).stop 2023-10-09T06:37:18.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.153+0000 7f2d6ec72700 1 -- 172.21.15.163:0/4115332101 shutdown_connections 2023-10-09T06:37:18.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.153+0000 7f2d6ec72700 1 --2- 172.21.15.163:0/4115332101 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2d54067320 0x7f2d540697d0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.153+0000 7f2d6ec72700 1 --2- 172.21.15.163:0/4115332101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d68159c10 0x7f2d68143080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.153+0000 7f2d6ec72700 1 --2- 172.21.15.163:0/4115332101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d680cba70 0x7f2d68155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.153+0000 7f2d6ec72700 1 --2- 172.21.15.163:0/4115332101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d68159840 0x7f2d68169830 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.153+0000 7f2d6ec72700 1 -- 172.21.15.163:0/4115332101 >> 172.21.15.163:0/4115332101 conn(0x7f2d68068e10 msgr2=0x7f2d68158340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:18.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.154+0000 7f2d6ec72700 1 -- 172.21.15.163:0/4115332101 shutdown_connections 2023-10-09T06:37:18.155 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.154+0000 7f2d6ec72700 1 -- 172.21.15.163:0/4115332101 wait complete. 2023-10-09T06:37:18.166 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-09T06:37:18.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.283+0000 7f4541c16700 1 Processor -- start 2023-10-09T06:37:18.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.283+0000 7f4541c16700 1 -- start start 2023-10-09T06:37:18.290 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.284+0000 7f4541c16700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f453c0b16d0 0x7f453c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:18.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.284+0000 7f4541c16700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f453c0b2080 con 0x7f453c0b33b0 2023-10-09T06:37:18.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.284+0000 7f4541c16700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f453c0cfb00 con 0x7f453c0b4e70 2023-10-09T06:37:18.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.284+0000 7f4541c16700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f453c0b5c40 con 0x7f453c0b16d0 2023-10-09T06:37:18.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.284+0000 7f4541415700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f453c0b33b0 0x7f453c0b3830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:54664/0 (socket says 172.21.15.163:54664) 2023-10-09T06:37:18.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.284+0000 7f4541415700 1 -- 172.21.15.163:0/3554521345 learned_addr learned my addr 172.21.15.163:0/3554521345 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:18.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.284+0000 7f4533fff700 1 --2- 172.21.15.163:0/3554521345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f453c0b16d0 0x7f453c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:18.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.284+0000 7f45337fe700 1 -- 172.21.15.163:0/3554521345 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3155366387 0 0) 0x7f453c0b2080 con 0x7f453c0b33b0 2023-10-09T06:37:18.291 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.284+0000 7f45337fe700 1 -- 172.21.15.163:0/3554521345 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4524003570 con 0x7f453c0b33b0 2023-10-09T06:37:18.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.284+0000 7f45337fe700 1 -- 172.21.15.163:0/3554521345 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1283636888 0 0) 0x7f453c0cfb00 con 0x7f453c0b4e70 2023-10-09T06:37:18.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.284+0000 7f45337fe700 1 -- 172.21.15.163:0/3554521345 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f453c0b2080 con 0x7f453c0b4e70 2023-10-09T06:37:18.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.285+0000 7f45337fe700 1 -- 172.21.15.163:0/3554521345 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 969757318 0 0) 0x7f4524003570 con 0x7f453c0b33b0 2023-10-09T06:37:18.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.285+0000 7f45337fe700 1 -- 172.21.15.163:0/3554521345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f453c0b16d0 msgr2=0x7f453c0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:18.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.285+0000 7f45337fe700 1 --2- 172.21.15.163:0/3554521345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f453c0b16d0 0x7f453c0b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.285+0000 7f45337fe700 1 -- 172.21.15.163:0/3554521345 >> v1:172.21.15.181:6789/0 conn(0x7f453c0b4e70 legacy=0x7f453c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:18.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.285+0000 7f45337fe700 1 -- 172.21.15.163:0/3554521345 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453c0b5f20 con 0x7f453c0b33b0 2023-10-09T06:37:18.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.285+0000 7f45337fe700 1 -- 172.21.15.163:0/3554521345 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f45340017f0 con 0x7f453c0b33b0 2023-10-09T06:37:18.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.285+0000 7f45337fe700 1 -- 172.21.15.163:0/3554521345 >> v1:172.21.15.163:6789/0 conn(0x7f453c0b33b0 legacy=0x7f453c0b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:18.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.285+0000 7f45337fe700 1 --2- 172.21.15.163:0/3554521345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4524003c20 0x7f4524004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:18.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.285+0000 7f45337fe700 1 --2- 172.21.15.163:0/3554521345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4524004810 0x7f4524004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:18.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.285+0000 7f45337fe700 1 --2- 172.21.15.163:0/3554521345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4524005210 0x7f452400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:18.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.285+0000 7f45337fe700 1 -- 172.21.15.163:0/3554521345 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453c0b5c40 con 0x7f4524003c20 2023-10-09T06:37:18.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.285+0000 7f45337fe700 1 -- 172.21.15.163:0/3554521345 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452400c160 con 0x7f4524004810 2023-10-09T06:37:18.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.285+0000 7f45337fe700 1 -- 172.21.15.163:0/3554521345 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452400c320 con 0x7f4524005210 2023-10-09T06:37:18.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.285+0000 7f4533fff700 1 --2- 172.21.15.163:0/3554521345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f453c0b16d0 0x7f453c0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:37:18.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.285+0000 7f4540c14700 1 --2- 172.21.15.163:0/3554521345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4524003c20 0x7f4524004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:18.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.285+0000 7f4541415700 1 --2- 172.21.15.163:0/3554521345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4524005210 0x7f452400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:18.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.285+0000 7f4533fff700 1 --2- 172.21.15.163:0/3554521345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4524004810 0x7f4524004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:18.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.285+0000 7f4540c14700 1 -- 172.21.15.163:0/3554521345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4524005210 msgr2=0x7f452400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:18.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.285+0000 7f4540c14700 1 --2- 172.21.15.163:0/3554521345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4524005210 0x7f452400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.285+0000 7f4540c14700 1 -- 172.21.15.163:0/3554521345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4524004810 msgr2=0x7f4524004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:18.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.285+0000 7f4540c14700 1 --2- 172.21.15.163:0/3554521345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4524004810 0x7f4524004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.285+0000 7f4540c14700 1 -- 172.21.15.163:0/3554521345 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4524003570 con 0x7f4524003c20 2023-10-09T06:37:18.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.285+0000 7f4540c14700 1 --2- 172.21.15.163:0/3554521345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4524003c20 0x7f4524004010 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f4538001050 tx=0x7f4538007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53287849bf41ff4a server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:18.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.285+0000 7f45337fe700 1 -- 172.21.15.163:0/3554521345 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453800c5d0 con 0x7f4524003c20 2023-10-09T06:37:18.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.285+0000 7f45337fe700 1 -- 172.21.15.163:0/3554521345 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453800c730 con 0x7f4524003c20 2023-10-09T06:37:18.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.285+0000 7f45337fe700 1 -- 172.21.15.163:0/3554521345 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453800caf0 con 0x7f4524003c20 2023-10-09T06:37:18.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.286+0000 7f4541c16700 1 -- 172.21.15.163:0/3554521345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4524003c20 msgr2=0x7f4524004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:18.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.286+0000 7f4541c16700 1 --2- 172.21.15.163:0/3554521345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4524003c20 0x7f4524004010 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f4538001050 tx=0x7f4538007d10 comp rx=0 tx=0).stop 2023-10-09T06:37:18.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.286+0000 7f4541415700 1 -- 172.21.15.163:0/3554521345 reap_dead start 2023-10-09T06:37:18.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.286+0000 7f4541c16700 1 -- 172.21.15.163:0/3554521345 shutdown_connections 2023-10-09T06:37:18.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.286+0000 7f4541c16700 1 -- 172.21.15.163:0/3554521345 >> 172.21.15.163:0/3554521345 conn(0x7f453c0d10c0 msgr2=0x7f453c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:18.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.286+0000 7f4541c16700 1 -- 172.21.15.163:0/3554521345 shutdown_connections 2023-10-09T06:37:18.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.286+0000 7f4541c16700 1 -- 172.21.15.163:0/3554521345 wait complete. 2023-10-09T06:37:18.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.287+0000 7f4541c16700 1 Processor -- start 2023-10-09T06:37:18.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.287+0000 7f4541c16700 1 -- start start 2023-10-09T06:37:18.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.288+0000 7f4541c16700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4524003c20 0x7f453c148cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:18.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.288+0000 7f4541c16700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f453c1459e0 0x7f453c146dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:18.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.288+0000 7f4541c16700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f453c147300 0x7f453c149660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:18.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.288+0000 7f4541c16700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453c145e50 con 0x7f453c1459e0 2023-10-09T06:37:18.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.288+0000 7f4541c16700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453c147c10 con 0x7f453c147300 2023-10-09T06:37:18.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.288+0000 7f4541c16700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f453c147ef0 con 0x7f4524003c20 2023-10-09T06:37:18.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.288+0000 7f4533fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f453c1459e0 0x7f453c146dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:18.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.288+0000 7f4533fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f453c1459e0 0x7f453c146dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:56994/0 (socket says 172.21.15.163:56994) 2023-10-09T06:37:18.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.288+0000 7f4533fff700 1 -- 172.21.15.163:0/2299061627 learned_addr learned my addr 172.21.15.163:0/2299061627 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:18.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.288+0000 7f4540c14700 1 --2- 172.21.15.163:0/2299061627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4524003c20 0x7f453c148cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:18.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.288+0000 7f4541415700 1 --2- 172.21.15.163:0/2299061627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f453c147300 0x7f453c149660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:18.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.288+0000 7f4533fff700 1 -- 172.21.15.163:0/2299061627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4524003c20 msgr2=0x7f453c148cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:18.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.288+0000 7f4533fff700 1 --2- 172.21.15.163:0/2299061627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4524003c20 0x7f453c148cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.288+0000 7f4533fff700 1 -- 172.21.15.163:0/2299061627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f453c147300 msgr2=0x7f453c149660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:18.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.288+0000 7f4533fff700 1 --2- 172.21.15.163:0/2299061627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f453c147300 0x7f453c149660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.288+0000 7f4533fff700 1 -- 172.21.15.163:0/2299061627 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453c1481d0 con 0x7f453c1459e0 2023-10-09T06:37:18.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.288+0000 7f4533fff700 1 --2- 172.21.15.163:0/2299061627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f453c1459e0 0x7f453c146dc0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f452c011910 tx=0x7f452c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:18.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.288+0000 7f4530ff9700 1 -- 172.21.15.163:0/2299061627 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c011e50 con 0x7f453c1459e0 2023-10-09T06:37:18.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.288+0000 7f4530ff9700 1 -- 172.21.15.163:0/2299061627 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452c00f8e0 con 0x7f453c1459e0 2023-10-09T06:37:18.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.288+0000 7f4541c16700 1 -- 172.21.15.163:0/2299061627 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f453c1484b0 con 0x7f453c1459e0 2023-10-09T06:37:18.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.289+0000 7f4530ff9700 1 -- 172.21.15.163:0/2299061627 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c00fca0 con 0x7f453c1459e0 2023-10-09T06:37:18.298 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.289+0000 7f4541c16700 1 -- 172.21.15.163:0/2299061627 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f453c147770 con 0x7f453c1459e0 2023-10-09T06:37:18.298 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.289+0000 7f4541c16700 1 -- 172.21.15.163:0/2299061627 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f453c148b00 con 0x7f453c1459e0 2023-10-09T06:37:18.298 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.290+0000 7f4530ff9700 1 -- 172.21.15.163:0/2299061627 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f452c015670 con 0x7f453c1459e0 2023-10-09T06:37:18.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.290+0000 7f4530ff9700 1 --2- 172.21.15.163:0/2299061627 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4528067320 0x7f45280697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:18.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.290+0000 7f4530ff9700 1 -- 172.21.15.163:0/2299061627 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f452c085540 con 0x7f453c1459e0 2023-10-09T06:37:18.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.291+0000 7f4540c14700 1 --2- 172.21.15.163:0/2299061627 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4528067320 0x7f45280697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:18.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.291+0000 7f4540c14700 1 --2- 172.21.15.163:0/2299061627 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4528067320 0x7f45280697d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f453c0afb80 tx=0x7f4538015000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:18.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.295+0000 7f4530ff9700 1 -- 172.21.15.163:0/2299061627 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f452c0521d0 con 0x7f453c1459e0 2023-10-09T06:37:18.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.479+0000 7f4541c16700 1 -- 172.21.15.163:0/2299061627 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f453c069840 con 0x7f453c1459e0 2023-10-09T06:37:18.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.479+0000 7f4530ff9700 1 -- 172.21.15.163:0/2299061627 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f452c0557f0 con 0x7f453c1459e0 2023-10-09T06:37:18.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.481+0000 7f4541c16700 1 -- 172.21.15.163:0/2299061627 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4528067320 msgr2=0x7f45280697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:18.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.481+0000 7f4541c16700 1 --2- 172.21.15.163:0/2299061627 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4528067320 0x7f45280697d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f453c0afb80 tx=0x7f4538015000 comp rx=0 tx=0).stop 2023-10-09T06:37:18.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.481+0000 7f4541c16700 1 -- 172.21.15.163:0/2299061627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f453c1459e0 msgr2=0x7f453c146dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:18.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.481+0000 7f4541c16700 1 --2- 172.21.15.163:0/2299061627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f453c1459e0 0x7f453c146dc0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f452c011910 tx=0x7f452c008040 comp rx=0 tx=0).stop 2023-10-09T06:37:18.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.481+0000 7f4541c16700 1 -- 172.21.15.163:0/2299061627 shutdown_connections 2023-10-09T06:37:18.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.481+0000 7f4541c16700 1 --2- 172.21.15.163:0/2299061627 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4528067320 0x7f45280697d0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.481+0000 7f4541c16700 1 --2- 172.21.15.163:0/2299061627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f453c147300 0x7f453c149660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.481+0000 7f4541c16700 1 --2- 172.21.15.163:0/2299061627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4524003c20 0x7f453c148cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.481+0000 7f4541c16700 1 --2- 172.21.15.163:0/2299061627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f453c1459e0 0x7f453c146dc0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.481+0000 7f4541c16700 1 -- 172.21.15.163:0/2299061627 >> 172.21.15.163:0/2299061627 conn(0x7f453c0d10c0 msgr2=0x7f453c069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:18.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.481+0000 7f4541c16700 1 -- 172.21.15.163:0/2299061627 shutdown_connections 2023-10-09T06:37:18.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.481+0000 7f4541c16700 1 -- 172.21.15.163:0/2299061627 wait complete. 2023-10-09T06:37:18.494 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-09T06:37:18.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.609+0000 7f0a34477700 1 Processor -- start 2023-10-09T06:37:18.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.609+0000 7f0a34477700 1 -- start start 2023-10-09T06:37:18.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.610+0000 7f0a34477700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0a2c06aa40 0x7f0a2c0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:18.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.610+0000 7f0a34477700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a2c0c91e0 0x7f0a2c160390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:18.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.610+0000 7f0a34477700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a2c0cc210 con 0x7f0a2c06bac0 2023-10-09T06:37:18.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.610+0000 7f0a34477700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a2c0cc4f0 con 0x7f0a2c06aa40 2023-10-09T06:37:18.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.610+0000 7f0a34477700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2c0cc7d0 con 0x7f0a2c0c91e0 2023-10-09T06:37:18.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.610+0000 7f0a31a12700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f0a2c06bac0 0x7f0a2c06ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:47904/0 (socket says 172.21.15.163:47904) 2023-10-09T06:37:18.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.610+0000 7f0a31a12700 1 -- 172.21.15.163:0/169809254 learned_addr learned my addr 172.21.15.163:0/169809254 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:18.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.611+0000 7f0a31211700 1 -- 172.21.15.163:0/169809254 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 713538566 0 0) 0x7f0a2c0cc210 con 0x7f0a2c06bac0 2023-10-09T06:37:18.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.611+0000 7f0a31211700 1 -- 172.21.15.163:0/169809254 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a18003570 con 0x7f0a2c06bac0 2023-10-09T06:37:18.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.611+0000 7f0a32213700 1 --2- 172.21.15.163:0/169809254 >> v2:172.21.15.181:3300/0 conn(0x7f0a2c06aa40 0x7f0a2c0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:18.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.611+0000 7f0a32a14700 1 --2- 172.21.15.163:0/169809254 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a2c0c91e0 0x7f0a2c160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:18.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.611+0000 7f0a31211700 1 -- 172.21.15.163:0/169809254 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1108798136 0 0) 0x7f0a18003570 con 0x7f0a2c06bac0 2023-10-09T06:37:18.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.611+0000 7f0a31211700 1 -- 172.21.15.163:0/169809254 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a2c0c91e0 msgr2=0x7f0a2c160390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:18.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.611+0000 7f0a31211700 1 --2- 172.21.15.163:0/169809254 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a2c0c91e0 0x7f0a2c160390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.611+0000 7f0a31211700 1 -- 172.21.15.163:0/169809254 >> v2:172.21.15.181:3300/0 conn(0x7f0a2c06aa40 msgr2=0x7f0a2c0cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:18.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.611+0000 7f0a31211700 1 --2- 172.21.15.163:0/169809254 >> v2:172.21.15.181:3300/0 conn(0x7f0a2c06aa40 0x7f0a2c0cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.611+0000 7f0a31211700 1 -- 172.21.15.163:0/169809254 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a2c0ccab0 con 0x7f0a2c06bac0 2023-10-09T06:37:18.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.611+0000 7f0a31211700 1 -- 172.21.15.163:0/169809254 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f0a1c001470 con 0x7f0a2c06bac0 2023-10-09T06:37:18.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.611+0000 7f0a31211700 1 -- 172.21.15.163:0/169809254 >> v1:172.21.15.163:6789/0 conn(0x7f0a2c06bac0 legacy=0x7f0a2c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:18.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.611+0000 7f0a31211700 1 --2- 172.21.15.163:0/169809254 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a2c0c91e0 0x7f0a18003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:18.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.611+0000 7f0a31211700 1 --2- 172.21.15.163:0/169809254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a18003fe0 0x7f0a18004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:18.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.611+0000 7f0a31211700 1 --2- 172.21.15.163:0/169809254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a18004d50 0x7f0a180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:18.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.611+0000 7f0a31211700 1 -- 172.21.15.163:0/169809254 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2c0cc4f0 con 0x7f0a18004d50 2023-10-09T06:37:18.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.611+0000 7f0a31211700 1 -- 172.21.15.163:0/169809254 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2c0cc7d0 con 0x7f0a18003fe0 2023-10-09T06:37:18.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.611+0000 7f0a31211700 1 -- 172.21.15.163:0/169809254 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1800dca0 con 0x7f0a2c0c91e0 2023-10-09T06:37:18.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.611+0000 7f0a32a14700 1 --2- 172.21.15.163:0/169809254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a18004d50 0x7f0a180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:18.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.611+0000 7f0a32213700 1 --2- 172.21.15.163:0/169809254 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a2c0c91e0 0x7f0a18003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:18.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.611+0000 7f0a31a12700 1 --2- 172.21.15.163:0/169809254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a18003fe0 0x7f0a18004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:18.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.611+0000 7f0a32a14700 1 -- 172.21.15.163:0/169809254 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a2c0c91e0 msgr2=0x7f0a18003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:18.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.611+0000 7f0a32a14700 1 --2- 172.21.15.163:0/169809254 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a2c0c91e0 0x7f0a18003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.611+0000 7f0a32a14700 1 -- 172.21.15.163:0/169809254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a18003fe0 msgr2=0x7f0a18004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:18.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.611+0000 7f0a32a14700 1 --2- 172.21.15.163:0/169809254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a18003fe0 0x7f0a18004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.611+0000 7f0a32a14700 1 -- 172.21.15.163:0/169809254 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a18003570 con 0x7f0a18004d50 2023-10-09T06:37:18.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.612+0000 7f0a32a14700 1 --2- 172.21.15.163:0/169809254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a18004d50 0x7f0a180051c0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f0a24009c00 tx=0x7f0a240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1255723d06289117 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:18.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.612+0000 7f0a31211700 1 -- 172.21.15.163:0/169809254 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2400e6b0 con 0x7f0a18004d50 2023-10-09T06:37:18.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.612+0000 7f0a31211700 1 -- 172.21.15.163:0/169809254 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2400e810 con 0x7f0a18004d50 2023-10-09T06:37:18.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.612+0000 7f0a31211700 1 -- 172.21.15.163:0/169809254 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2400ebd0 con 0x7f0a18004d50 2023-10-09T06:37:18.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.612+0000 7f0a34477700 1 -- 172.21.15.163:0/169809254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a18004d50 msgr2=0x7f0a180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:18.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.612+0000 7f0a34477700 1 --2- 172.21.15.163:0/169809254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a18004d50 0x7f0a180051c0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f0a24009c00 tx=0x7f0a240091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.612+0000 7f0a32a14700 1 -- 172.21.15.163:0/169809254 reap_dead start 2023-10-09T06:37:18.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.612+0000 7f0a34477700 1 -- 172.21.15.163:0/169809254 shutdown_connections 2023-10-09T06:37:18.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.612+0000 7f0a34477700 1 -- 172.21.15.163:0/169809254 >> 172.21.15.163:0/169809254 conn(0x7f0a2c0cd670 msgr2=0x7f0a2c0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:18.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.612+0000 7f0a34477700 1 -- 172.21.15.163:0/169809254 shutdown_connections 2023-10-09T06:37:18.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.612+0000 7f0a34477700 1 -- 172.21.15.163:0/169809254 wait complete. 2023-10-09T06:37:18.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.613+0000 7f0a34477700 1 Processor -- start 2023-10-09T06:37:18.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.613+0000 7f0a34477700 1 -- start start 2023-10-09T06:37:18.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.614+0000 7f0a34477700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a18004d50 0x7f0a2c158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:18.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.614+0000 7f0a34477700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a2c0b2970 0x7f0a2c150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:18.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.614+0000 7f0a34477700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a2c1554b0 0x7f0a2c155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:18.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.614+0000 7f0a34477700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2c155e90 con 0x7f0a18004d50 2023-10-09T06:37:18.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.614+0000 7f0a34477700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2c158c00 con 0x7f0a2c0b2970 2023-10-09T06:37:18.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.614+0000 7f0a34477700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2c0aea30 con 0x7f0a2c1554b0 2023-10-09T06:37:18.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.614+0000 7f0a32213700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a18004d50 0x7f0a2c158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:18.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.614+0000 7f0a32213700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a18004d50 0x7f0a2c158060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58538/0 (socket says 172.21.15.163:58538) 2023-10-09T06:37:18.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.614+0000 7f0a32213700 1 -- 172.21.15.163:0/1125957921 learned_addr learned my addr 172.21.15.163:0/1125957921 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:18.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.614+0000 7f0a31a12700 1 --2- 172.21.15.163:0/1125957921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a2c0b2970 0x7f0a2c150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:18.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.614+0000 7f0a32a14700 1 --2- 172.21.15.163:0/1125957921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a2c1554b0 0x7f0a2c155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:18.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.615+0000 7f0a32213700 1 -- 172.21.15.163:0/1125957921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a2c1554b0 msgr2=0x7f0a2c155950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:18.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.615+0000 7f0a32213700 1 --2- 172.21.15.163:0/1125957921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a2c1554b0 0x7f0a2c155950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.615+0000 7f0a32213700 1 -- 172.21.15.163:0/1125957921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a2c0b2970 msgr2=0x7f0a2c150840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:18.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.615+0000 7f0a32213700 1 --2- 172.21.15.163:0/1125957921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a2c0b2970 0x7f0a2c150840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.615+0000 7f0a32213700 1 -- 172.21.15.163:0/1125957921 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a2c158640 con 0x7f0a18004d50 2023-10-09T06:37:18.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.615+0000 7f0a32213700 1 --2- 172.21.15.163:0/1125957921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a18004d50 0x7f0a2c158060 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f0a28011f30 tx=0x7f0a2800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:18.624 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.615+0000 7f0a227fc700 1 -- 172.21.15.163:0/1125957921 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2800e040 con 0x7f0a18004d50 2023-10-09T06:37:18.624 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.615+0000 7f0a227fc700 1 -- 172.21.15.163:0/1125957921 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2800f040 con 0x7f0a18004d50 2023-10-09T06:37:18.624 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.615+0000 7f0a34477700 1 -- 172.21.15.163:0/1125957921 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a2c158920 con 0x7f0a18004d50 2023-10-09T06:37:18.624 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.615+0000 7f0a227fc700 1 -- 172.21.15.163:0/1125957921 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2800fec0 con 0x7f0a18004d50 2023-10-09T06:37:18.624 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.615+0000 7f0a34477700 1 -- 172.21.15.163:0/1125957921 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a2c150d80 con 0x7f0a18004d50 2023-10-09T06:37:18.624 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.616+0000 7f0a34477700 1 -- 172.21.15.163:0/1125957921 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a2c04c3c0 con 0x7f0a18004d50 2023-10-09T06:37:18.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.616+0000 7f0a227fc700 1 -- 172.21.15.163:0/1125957921 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0a28015640 con 0x7f0a18004d50 2023-10-09T06:37:18.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.617+0000 7f0a227fc700 1 --2- 172.21.15.163:0/1125957921 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0a10067320 0x7f0a100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:18.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.617+0000 7f0a227fc700 1 -- 172.21.15.163:0/1125957921 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0a28085920 con 0x7f0a18004d50 2023-10-09T06:37:18.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.617+0000 7f0a31a12700 1 --2- 172.21.15.163:0/1125957921 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0a10067320 0x7f0a100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:18.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.617+0000 7f0a31a12700 1 --2- 172.21.15.163:0/1125957921 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0a10067320 0x7f0a100697d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c007fd0 tx=0x7f0a1c001e90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:18.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.621+0000 7f0a227fc700 1 -- 172.21.15.163:0/1125957921 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a28052560 con 0x7f0a18004d50 2023-10-09T06:37:18.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.838+0000 7f0a34477700 1 -- 172.21.15.163:0/1125957921 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a2c0c7cb0 con 0x7f0a10067320 2023-10-09T06:37:18.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.840+0000 7f0a227fc700 1 -- 172.21.15.163:0/1125957921 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f0a2c0c7cb0 con 0x7f0a10067320 2023-10-09T06:37:18.841 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:37:18.841 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:18.841 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "cephfs" 2023-10-09T06:37:18.841 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:18.842 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:37:18.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.842+0000 7f0a34477700 1 -- 172.21.15.163:0/1125957921 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0a10067320 msgr2=0x7f0a100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:18.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.842+0000 7f0a34477700 1 --2- 172.21.15.163:0/1125957921 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0a10067320 0x7f0a100697d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c007fd0 tx=0x7f0a1c001e90 comp rx=0 tx=0).stop 2023-10-09T06:37:18.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.842+0000 7f0a34477700 1 -- 172.21.15.163:0/1125957921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a18004d50 msgr2=0x7f0a2c158060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:18.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.842+0000 7f0a34477700 1 --2- 172.21.15.163:0/1125957921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a18004d50 0x7f0a2c158060 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f0a28011f30 tx=0x7f0a2800b040 comp rx=0 tx=0).stop 2023-10-09T06:37:18.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.842+0000 7f0a34477700 1 -- 172.21.15.163:0/1125957921 shutdown_connections 2023-10-09T06:37:18.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.842+0000 7f0a34477700 1 --2- 172.21.15.163:0/1125957921 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0a10067320 0x7f0a100697d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.842+0000 7f0a34477700 1 --2- 172.21.15.163:0/1125957921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a18004d50 0x7f0a2c158060 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.842+0000 7f0a34477700 1 --2- 172.21.15.163:0/1125957921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a2c1554b0 0x7f0a2c155950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.842+0000 7f0a34477700 1 --2- 172.21.15.163:0/1125957921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a2c0b2970 0x7f0a2c150840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.842+0000 7f0a34477700 1 -- 172.21.15.163:0/1125957921 >> 172.21.15.163:0/1125957921 conn(0x7f0a2c0cd670 msgr2=0x7f0a2c1526e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:18.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.842+0000 7f0a34477700 1 -- 172.21.15.163:0/1125957921 shutdown_connections 2023-10-09T06:37:18.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.842+0000 7f0a34477700 1 -- 172.21.15.163:0/1125957921 wait complete. 2023-10-09T06:37:18.855 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create volume_0000000000905867 2023-10-09T06:37:18.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.970+0000 7f285876f700 1 Processor -- start 2023-10-09T06:37:18.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.970+0000 7f285876f700 1 -- start start 2023-10-09T06:37:18.978 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.971+0000 7f285876f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f285006bcd0 0x7f2850140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:18.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.971+0000 7f285876f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28500c9510 0x7f2850141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:18.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.971+0000 7f285876f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f285015c2a0 con 0x7f285006c0a0 2023-10-09T06:37:18.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.971+0000 7f285876f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f285015b690 con 0x7f285006bcd0 2023-10-09T06:37:18.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.971+0000 7f285876f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f285015b970 con 0x7f28500c9510 2023-10-09T06:37:18.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.971+0000 7f2855d0a700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f285006c0a0 0x7f28500c8e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:47912/0 (socket says 172.21.15.163:47912) 2023-10-09T06:37:18.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.971+0000 7f2855d0a700 1 -- 172.21.15.163:0/3470365506 learned_addr learned my addr 172.21.15.163:0/3470365506 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:18.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.972+0000 7f285650b700 1 --2- 172.21.15.163:0/3470365506 >> v2:172.21.15.181:3300/0 conn(0x7f285006bcd0 0x7f2850140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:18.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.972+0000 7f2856d0c700 1 --2- 172.21.15.163:0/3470365506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28500c9510 0x7f2850141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:18.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.972+0000 7f2855509700 1 -- 172.21.15.163:0/3470365506 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1099908672 0 0) 0x7f285015c2a0 con 0x7f285006c0a0 2023-10-09T06:37:18.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.972+0000 7f2855509700 1 -- 172.21.15.163:0/3470365506 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f283c003570 con 0x7f285006c0a0 2023-10-09T06:37:18.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.972+0000 7f285650b700 1 -- 172.21.15.163:0/3470365506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28500c9510 msgr2=0x7f2850141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:18.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.972+0000 7f285650b700 1 --2- 172.21.15.163:0/3470365506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28500c9510 0x7f2850141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.972+0000 7f285650b700 1 -- 172.21.15.163:0/3470365506 >> v1:172.21.15.163:6789/0 conn(0x7f285006c0a0 legacy=0x7f28500c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:18.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.972+0000 7f285650b700 1 -- 172.21.15.163:0/3470365506 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f285015bc50 con 0x7f285006bcd0 2023-10-09T06:37:18.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.972+0000 7f2856d0c700 1 --2- 172.21.15.163:0/3470365506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28500c9510 0x7f2850141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:37:18.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.972+0000 7f285650b700 1 --2- 172.21.15.163:0/3470365506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f285006bcd0 0x7f2850140df0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f284c0098e0 tx=0x7f284c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24185a6c65500f8a server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:18.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.972+0000 7f2855509700 1 -- 172.21.15.163:0/3470365506 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c011630 con 0x7f285006bcd0 2023-10-09T06:37:18.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.972+0000 7f2855509700 1 -- 172.21.15.163:0/3470365506 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284c011790 con 0x7f285006bcd0 2023-10-09T06:37:18.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.972+0000 7f2855509700 1 -- 172.21.15.163:0/3470365506 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c011b30 con 0x7f285006bcd0 2023-10-09T06:37:18.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.973+0000 7f285876f700 1 -- 172.21.15.163:0/3470365506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f285006bcd0 msgr2=0x7f2850140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:18.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.973+0000 7f285876f700 1 --2- 172.21.15.163:0/3470365506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f285006bcd0 0x7f2850140df0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f284c0098e0 tx=0x7f284c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.973+0000 7f285876f700 1 -- 172.21.15.163:0/3470365506 shutdown_connections 2023-10-09T06:37:18.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.973+0000 7f285876f700 1 --2- 172.21.15.163:0/3470365506 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f285006bcd0 0x7f2850140df0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.973+0000 7f285876f700 1 --2- 172.21.15.163:0/3470365506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28500c9510 0x7f2850141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.973+0000 7f285876f700 1 -- 172.21.15.163:0/3470365506 >> 172.21.15.163:0/3470365506 conn(0x7f28500ce600 msgr2=0x7f28500cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:18.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.973+0000 7f285876f700 1 -- 172.21.15.163:0/3470365506 shutdown_connections 2023-10-09T06:37:18.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.973+0000 7f285876f700 1 -- 172.21.15.163:0/3470365506 wait complete. 2023-10-09T06:37:18.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.974+0000 7f285876f700 1 Processor -- start 2023-10-09T06:37:18.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.974+0000 7f285876f700 1 -- start start 2023-10-09T06:37:18.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.975+0000 7f285876f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f285006bcd0 0x7f285013fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:18.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.975+0000 7f285876f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f285006c0a0 0x7f285013de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:18.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.975+0000 7f285876f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28500c9510 0x7f28500c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:18.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.975+0000 7f285876f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28500c6db0 con 0x7f28500c9510 2023-10-09T06:37:18.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.975+0000 7f285876f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28500b1d70 con 0x7f285006bcd0 2023-10-09T06:37:18.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.975+0000 7f285876f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28500b1440 con 0x7f285006c0a0 2023-10-09T06:37:18.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.975+0000 7f2856d0c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28500c9510 0x7f28500c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:18.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.975+0000 7f2856d0c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28500c9510 0x7f28500c77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58540/0 (socket says 172.21.15.163:58540) 2023-10-09T06:37:18.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.975+0000 7f2856d0c700 1 -- 172.21.15.163:0/4260609651 learned_addr learned my addr 172.21.15.163:0/4260609651 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:18.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.975+0000 7f285650b700 1 --2- 172.21.15.163:0/4260609651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f285006bcd0 0x7f285013fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:18.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.975+0000 7f2855d0a700 1 --2- 172.21.15.163:0/4260609651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f285006c0a0 0x7f285013de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:18.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.975+0000 7f2856d0c700 1 -- 172.21.15.163:0/4260609651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f285006c0a0 msgr2=0x7f285013de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:18.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.975+0000 7f2856d0c700 1 --2- 172.21.15.163:0/4260609651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f285006c0a0 0x7f285013de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.975+0000 7f2856d0c700 1 -- 172.21.15.163:0/4260609651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f285006bcd0 msgr2=0x7f285013fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:18.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.975+0000 7f2856d0c700 1 --2- 172.21.15.163:0/4260609651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f285006bcd0 0x7f285013fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:18.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.975+0000 7f2856d0c700 1 -- 172.21.15.163:0/4260609651 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2848009a20 con 0x7f28500c9510 2023-10-09T06:37:18.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.975+0000 7f2855d0a700 1 --2- 172.21.15.163:0/4260609651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f285006c0a0 0x7f285013de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:18.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.975+0000 7f2856d0c700 1 --2- 172.21.15.163:0/4260609651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28500c9510 0x7f28500c77c0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f2848009f50 tx=0x7f284800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:18.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.976+0000 7f28467fc700 1 -- 172.21.15.163:0/4260609651 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284800cc00 con 0x7f28500c9510 2023-10-09T06:37:18.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.976+0000 7f28467fc700 1 -- 172.21.15.163:0/4260609651 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284800f040 con 0x7f28500c9510 2023-10-09T06:37:18.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.976+0000 7f285876f700 1 -- 172.21.15.163:0/4260609651 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f284c009420 con 0x7f28500c9510 2023-10-09T06:37:18.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.976+0000 7f28467fc700 1 -- 172.21.15.163:0/4260609651 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2848014480 con 0x7f28500c9510 2023-10-09T06:37:18.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.976+0000 7f285876f700 1 -- 172.21.15.163:0/4260609651 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28500b1a00 con 0x7f28500c9510 2023-10-09T06:37:18.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.976+0000 7f285876f700 1 -- 172.21.15.163:0/4260609651 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28500b1f00 con 0x7f28500c9510 2023-10-09T06:37:18.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.977+0000 7f28467fc700 1 -- 172.21.15.163:0/4260609651 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2848003020 con 0x7f28500c9510 2023-10-09T06:37:18.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.977+0000 7f28467fc700 1 --2- 172.21.15.163:0/4260609651 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2834067320 0x7f28340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:18.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.977+0000 7f28467fc700 1 -- 172.21.15.163:0/4260609651 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2848085690 con 0x7f28500c9510 2023-10-09T06:37:18.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.977+0000 7f285650b700 1 --2- 172.21.15.163:0/4260609651 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2834067320 0x7f28340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:18.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.978+0000 7f285650b700 1 --2- 172.21.15.163:0/4260609651 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2834067320 0x7f28340697d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f284c0091e0 tx=0x7f284c017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:18.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:18.981+0000 7f28467fc700 1 -- 172.21.15.163:0/4260609651 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2848052210 con 0x7f28500c9510 2023-10-09T06:37:19.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:19.189+0000 7f285876f700 1 -- 172.21.15.163:0/4260609651 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "volume_0000000000905867", "target": ["mon-mgr", ""]}) v1 -- 0x7f285004c3c0 con 0x7f2834067320 2023-10-09T06:37:21.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.831+0000 7f28467fc700 1 -- 172.21.15.163:0/4260609651 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) v1 ==== 60+0+0 (secure 0 0 0) 0x7f285004c3c0 con 0x7f2834067320 2023-10-09T06:37:21.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.837+0000 7f285876f700 1 -- 172.21.15.163:0/4260609651 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2834067320 msgr2=0x7f28340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:21.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.837+0000 7f285876f700 1 --2- 172.21.15.163:0/4260609651 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2834067320 0x7f28340697d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f284c0091e0 tx=0x7f284c017000 comp rx=0 tx=0).stop 2023-10-09T06:37:21.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.837+0000 7f285876f700 1 -- 172.21.15.163:0/4260609651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28500c9510 msgr2=0x7f28500c77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:21.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.837+0000 7f285876f700 1 --2- 172.21.15.163:0/4260609651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28500c9510 0x7f28500c77c0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f2848009f50 tx=0x7f284800b040 comp rx=0 tx=0).stop 2023-10-09T06:37:21.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.838+0000 7f285876f700 1 -- 172.21.15.163:0/4260609651 shutdown_connections 2023-10-09T06:37:21.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.838+0000 7f285876f700 1 --2- 172.21.15.163:0/4260609651 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2834067320 0x7f28340697d0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:21.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.838+0000 7f285876f700 1 --2- 172.21.15.163:0/4260609651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28500c9510 0x7f28500c77c0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:21.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.838+0000 7f285876f700 1 --2- 172.21.15.163:0/4260609651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f285006bcd0 0x7f285013fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:21.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.838+0000 7f285876f700 1 --2- 172.21.15.163:0/4260609651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f285006c0a0 0x7f285013de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:21.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.838+0000 7f285876f700 1 -- 172.21.15.163:0/4260609651 >> 172.21.15.163:0/4260609651 conn(0x7f28500ce600 msgr2=0x7f28500c33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:21.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.838+0000 7f285876f700 1 -- 172.21.15.163:0/4260609651 shutdown_connections 2023-10-09T06:37:21.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.838+0000 7f285876f700 1 -- 172.21.15.163:0/4260609651 wait complete. 2023-10-09T06:37:21.841 INFO:teuthology.orchestra.run.smithi163.stderr:Volume created successfully (no MDS daemons created) 2023-10-09T06:37:21.857 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create volume_0000000000905868 2023-10-09T06:37:21.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.978+0000 7fbfa69a4700 1 Processor -- start 2023-10-09T06:37:21.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.978+0000 7fbfa69a4700 1 -- start start 2023-10-09T06:37:21.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.979+0000 7fbfa69a4700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbfa006bcd0 0x7fbfa0140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:21.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.979+0000 7fbfa69a4700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa015c2a0 con 0x7fbfa00c9510 2023-10-09T06:37:21.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.979+0000 7fbfa69a4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa015b690 con 0x7fbfa006c0a0 2023-10-09T06:37:21.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.979+0000 7fbfa69a4700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfa00c9900 con 0x7fbfa006bcd0 2023-10-09T06:37:21.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.979+0000 7fbf9ffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbfa006bcd0 0x7fbfa0140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:21.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.979+0000 7fbf9ffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbfa006bcd0 0x7fbfa0140df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58552/0 (socket says 172.21.15.163:58552) 2023-10-09T06:37:21.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.979+0000 7fbfa4f41700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fbfa00c9510 0x7fbfa00cc1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:47922/0 (socket says 172.21.15.163:47922) 2023-10-09T06:37:21.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.979+0000 7fbf9ffff700 1 -- 172.21.15.163:0/232872538 learned_addr learned my addr 172.21.15.163:0/232872538 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:21.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.980+0000 7fbf9ffff700 1 -- 172.21.15.163:0/232872538 >> v1:172.21.15.181:6789/0 conn(0x7fbfa006c0a0 legacy=0x7fbfa00c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:21.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.980+0000 7fbf9ffff700 1 -- 172.21.15.163:0/232872538 >> v1:172.21.15.163:6789/0 conn(0x7fbfa00c9510 legacy=0x7fbfa00cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:21.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.980+0000 7fbf9ffff700 1 -- 172.21.15.163:0/232872538 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa015b970 con 0x7fbfa006bcd0 2023-10-09T06:37:21.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.980+0000 7fbf9effd700 1 -- 172.21.15.163:0/232872538 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 329968593 0 0) 0x7fbfa015c2a0 con 0x7fbfa00c9510 2023-10-09T06:37:21.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.980+0000 7fbf9ffff700 1 --2- 172.21.15.163:0/232872538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfa006bcd0 0x7fbfa0140df0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fbf900099b0 tx=0x7fbf90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7864ecfc72b0829 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:21.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.980+0000 7fbf9effd700 1 -- 172.21.15.163:0/232872538 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9000b860 con 0x7fbfa006bcd0 2023-10-09T06:37:21.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.980+0000 7fbf9effd700 1 -- 172.21.15.163:0/232872538 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf9000b9c0 con 0x7fbfa006bcd0 2023-10-09T06:37:21.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.980+0000 7fbf9effd700 1 -- 172.21.15.163:0/232872538 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9000bd60 con 0x7fbfa006bcd0 2023-10-09T06:37:21.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.980+0000 7fbfa69a4700 1 -- 172.21.15.163:0/232872538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfa006bcd0 msgr2=0x7fbfa0140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:21.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.980+0000 7fbfa69a4700 1 --2- 172.21.15.163:0/232872538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfa006bcd0 0x7fbfa0140df0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fbf900099b0 tx=0x7fbf90009250 comp rx=0 tx=0).stop 2023-10-09T06:37:21.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.980+0000 7fbfa69a4700 1 -- 172.21.15.163:0/232872538 shutdown_connections 2023-10-09T06:37:21.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.980+0000 7fbfa69a4700 1 --2- 172.21.15.163:0/232872538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfa006bcd0 0x7fbfa0140df0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:21.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.981+0000 7fbfa69a4700 1 -- 172.21.15.163:0/232872538 >> 172.21.15.163:0/232872538 conn(0x7fbfa00ce600 msgr2=0x7fbfa00cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:21.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.981+0000 7fbfa69a4700 1 -- 172.21.15.163:0/232872538 shutdown_connections 2023-10-09T06:37:21.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.981+0000 7fbfa69a4700 1 -- 172.21.15.163:0/232872538 wait complete. 2023-10-09T06:37:21.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.982+0000 7fbfa69a4700 1 Processor -- start 2023-10-09T06:37:21.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.982+0000 7fbfa69a4700 1 -- start start 2023-10-09T06:37:21.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.982+0000 7fbfa69a4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfa006bcd0 0x7fbfa0167d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:21.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.982+0000 7fbfa69a4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfa006c0a0 0x7fbfa0156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:21.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.982+0000 7fbfa69a4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfa00c9510 0x7fbfa00c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:21.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.982+0000 7fbfa69a4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa00c6080 con 0x7fbfa00c9510 2023-10-09T06:37:21.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.982+0000 7fbfa69a4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa013f000 con 0x7fbfa006c0a0 2023-10-09T06:37:21.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.982+0000 7fbfa69a4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa00b03d0 con 0x7fbfa006bcd0 2023-10-09T06:37:21.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.983+0000 7fbfa4f41700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfa00c9510 0x7fbfa00c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:21.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.983+0000 7fbfa4f41700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfa00c9510 0x7fbfa00c7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58558/0 (socket says 172.21.15.163:58558) 2023-10-09T06:37:21.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.983+0000 7fbfa4f41700 1 -- 172.21.15.163:0/704027835 learned_addr learned my addr 172.21.15.163:0/704027835 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:21.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.983+0000 7fbf9f7fe700 1 --2- 172.21.15.163:0/704027835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfa006c0a0 0x7fbfa0156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:21.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.983+0000 7fbf9ffff700 1 --2- 172.21.15.163:0/704027835 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfa006bcd0 0x7fbfa0167d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:21.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.983+0000 7fbfa4f41700 1 -- 172.21.15.163:0/704027835 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfa006bcd0 msgr2=0x7fbfa0167d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:21.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.983+0000 7fbfa4f41700 1 --2- 172.21.15.163:0/704027835 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfa006bcd0 0x7fbfa0167d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:21.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.983+0000 7fbfa4f41700 1 -- 172.21.15.163:0/704027835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfa006c0a0 msgr2=0x7fbfa0156890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:21.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.983+0000 7fbfa4f41700 1 --2- 172.21.15.163:0/704027835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfa006c0a0 0x7fbfa0156890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:21.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.983+0000 7fbfa4f41700 1 -- 172.21.15.163:0/704027835 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf900094f0 con 0x7fbfa00c9510 2023-10-09T06:37:21.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.983+0000 7fbfa4f41700 1 --2- 172.21.15.163:0/704027835 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfa00c9510 0x7fbfa00c7660 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800ccc0 tx=0x7fbf98001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:21.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.983+0000 7fbf87fff700 1 -- 172.21.15.163:0/704027835 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf98005df0 con 0x7fbfa00c9510 2023-10-09T06:37:21.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.983+0000 7fbf87fff700 1 -- 172.21.15.163:0/704027835 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf9800a600 con 0x7fbfa00c9510 2023-10-09T06:37:21.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.983+0000 7fbfa69a4700 1 -- 172.21.15.163:0/704027835 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa00c5750 con 0x7fbfa00c9510 2023-10-09T06:37:21.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.983+0000 7fbf87fff700 1 -- 172.21.15.163:0/704027835 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9800a9c0 con 0x7fbfa00c9510 2023-10-09T06:37:21.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.983+0000 7fbfa69a4700 1 -- 172.21.15.163:0/704027835 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa00c5d10 con 0x7fbfa00c9510 2023-10-09T06:37:21.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.984+0000 7fbfa69a4700 1 -- 172.21.15.163:0/704027835 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa00c6230 con 0x7fbfa00c9510 2023-10-09T06:37:21.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.985+0000 7fbf87fff700 1 -- 172.21.15.163:0/704027835 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbf980070f0 con 0x7fbfa00c9510 2023-10-09T06:37:21.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.985+0000 7fbf87fff700 1 --2- 172.21.15.163:0/704027835 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbf8c067320 0x7fbf8c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:21.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.985+0000 7fbf87fff700 1 -- 172.21.15.163:0/704027835 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fbf98085070 con 0x7fbfa00c9510 2023-10-09T06:37:21.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.985+0000 7fbf9ffff700 1 --2- 172.21.15.163:0/704027835 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbf8c067320 0x7fbf8c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:21.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.986+0000 7fbf9ffff700 1 --2- 172.21.15.163:0/704027835 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbf8c067320 0x7fbf8c0697d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fbf900099b0 tx=0x7fbf90016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:21.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:21.990+0000 7fbf87fff700 1 -- 172.21.15.163:0/704027835 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbfa00c6230 con 0x7fbfa00c9510 2023-10-09T06:37:22.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:22.199+0000 7fbfa69a4700 1 -- 172.21.15.163:0/704027835 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "volume_0000000000905868", "target": ["mon-mgr", ""]}) v1 -- 0x7fbfa004c3c0 con 0x7fbf8c067320 2023-10-09T06:37:24.537 INFO:tasks.ceph.mds.a.smithi163.stderr:2023-10-09T06:37:24.536+0000 7fa2586f9700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-09T06:37:24.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:24.866+0000 7fbf87fff700 1 -- 172.21.15.163:0/704027835 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) v1 ==== 60+0+0 (secure 0 0 0) 0x7fbfa004c3c0 con 0x7fbf8c067320 2023-10-09T06:37:24.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:24.869+0000 7fbfa69a4700 1 -- 172.21.15.163:0/704027835 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbf8c067320 msgr2=0x7fbf8c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:24.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:24.869+0000 7fbfa69a4700 1 --2- 172.21.15.163:0/704027835 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbf8c067320 0x7fbf8c0697d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fbf900099b0 tx=0x7fbf90016000 comp rx=0 tx=0).stop 2023-10-09T06:37:24.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:24.869+0000 7fbfa69a4700 1 -- 172.21.15.163:0/704027835 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfa00c9510 msgr2=0x7fbfa00c7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:24.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:24.869+0000 7fbfa69a4700 1 --2- 172.21.15.163:0/704027835 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfa00c9510 0x7fbfa00c7660 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800ccc0 tx=0x7fbf98001a70 comp rx=0 tx=0).stop 2023-10-09T06:37:24.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:24.869+0000 7fbfa69a4700 1 -- 172.21.15.163:0/704027835 shutdown_connections 2023-10-09T06:37:24.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:24.869+0000 7fbfa69a4700 1 --2- 172.21.15.163:0/704027835 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbf8c067320 0x7fbf8c0697d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:24.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:24.869+0000 7fbfa69a4700 1 --2- 172.21.15.163:0/704027835 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfa00c9510 0x7fbfa00c7660 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:24.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:24.869+0000 7fbfa69a4700 1 --2- 172.21.15.163:0/704027835 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfa006bcd0 0x7fbfa0167d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:24.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:24.869+0000 7fbfa69a4700 1 --2- 172.21.15.163:0/704027835 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfa006c0a0 0x7fbfa0156890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:24.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:24.869+0000 7fbfa69a4700 1 -- 172.21.15.163:0/704027835 >> 172.21.15.163:0/704027835 conn(0x7fbfa00ce600 msgr2=0x7fbfa0157520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:24.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:24.869+0000 7fbfa69a4700 1 -- 172.21.15.163:0/704027835 shutdown_connections 2023-10-09T06:37:24.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:24.869+0000 7fbfa69a4700 1 -- 172.21.15.163:0/704027835 wait complete. 2023-10-09T06:37:24.872 INFO:teuthology.orchestra.run.smithi163.stderr:Volume created successfully (no MDS daemons created) 2023-10-09T06:37:24.884 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-09T06:37:25.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.004+0000 7ff4eacb0700 1 Processor -- start 2023-10-09T06:37:25.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.004+0000 7ff4eacb0700 1 -- start start 2023-10-09T06:37:25.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.005+0000 7ff4eacb0700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff4e4159890 0x7ff4e415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:25.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.005+0000 7ff4eacb0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff4e4159c60 0x7ff4e415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:25.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.005+0000 7ff4eacb0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4e415a160 con 0x7ff4e415db10 2023-10-09T06:37:25.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.005+0000 7ff4eacb0700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e415df50 con 0x7ff4e4159890 2023-10-09T06:37:25.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.005+0000 7ff4eacb0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e40d13a0 con 0x7ff4e4159c60 2023-10-09T06:37:25.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.005+0000 7ff4e8a4c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff4e4159890 0x7ff4e415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:25.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.005+0000 7ff4e8a4c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff4e4159890 0x7ff4e415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58566/0 (socket says 172.21.15.163:58566) 2023-10-09T06:37:25.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.005+0000 7ff4e8a4c700 1 -- 172.21.15.163:0/1148768031 learned_addr learned my addr 172.21.15.163:0/1148768031 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:25.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.005+0000 7ff4e3fff700 1 --2- 172.21.15.163:0/1148768031 >> v2:172.21.15.181:3300/0 conn(0x7ff4e4159c60 0x7ff4e415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:25.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.006+0000 7ff4e8a4c700 1 -- 172.21.15.163:0/1148768031 >> v2:172.21.15.181:3300/0 conn(0x7ff4e4159c60 msgr2=0x7ff4e415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:25.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.006+0000 7ff4e8a4c700 1 --2- 172.21.15.163:0/1148768031 >> v2:172.21.15.181:3300/0 conn(0x7ff4e4159c60 0x7ff4e415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:25.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.006+0000 7ff4e8a4c700 1 -- 172.21.15.163:0/1148768031 >> v1:172.21.15.181:6789/0 conn(0x7ff4e415db10 legacy=0x7ff4e4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:25.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.006+0000 7ff4e8a4c700 1 -- 172.21.15.163:0/1148768031 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e415e140 con 0x7ff4e4159890 2023-10-09T06:37:25.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.006+0000 7ff4e3fff700 1 --2- 172.21.15.163:0/1148768031 >> v2:172.21.15.181:3300/0 conn(0x7ff4e4159c60 0x7ff4e415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:37:25.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.006+0000 7ff4e8a4c700 1 --2- 172.21.15.163:0/1148768031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4e4159890 0x7ff4e415abd0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7ff4e40b67f0 tx=0x7ff4d4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f87af0a2bcc7189 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:25.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.006+0000 7ff4e37fe700 1 -- 172.21.15.163:0/1148768031 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d400e070 con 0x7ff4e4159890 2023-10-09T06:37:25.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.006+0000 7ff4e37fe700 1 -- 172.21.15.163:0/1148768031 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d400ba10 con 0x7ff4e4159890 2023-10-09T06:37:25.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.006+0000 7ff4e37fe700 1 -- 172.21.15.163:0/1148768031 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d40092e0 con 0x7ff4e4159890 2023-10-09T06:37:25.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.006+0000 7ff4eacb0700 1 -- 172.21.15.163:0/1148768031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4e4159890 msgr2=0x7ff4e415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:25.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.006+0000 7ff4eacb0700 1 --2- 172.21.15.163:0/1148768031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4e4159890 0x7ff4e415abd0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7ff4e40b67f0 tx=0x7ff4d4009840 comp rx=0 tx=0).stop 2023-10-09T06:37:25.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.006+0000 7ff4eacb0700 1 -- 172.21.15.163:0/1148768031 shutdown_connections 2023-10-09T06:37:25.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.006+0000 7ff4eacb0700 1 --2- 172.21.15.163:0/1148768031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4e4159890 0x7ff4e415abd0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:25.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.006+0000 7ff4eacb0700 1 --2- 172.21.15.163:0/1148768031 >> v2:172.21.15.181:3300/0 conn(0x7ff4e4159c60 0x7ff4e415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:25.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.006+0000 7ff4eacb0700 1 -- 172.21.15.163:0/1148768031 >> 172.21.15.163:0/1148768031 conn(0x7ff4e4068e20 msgr2=0x7ff4e40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:25.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.006+0000 7ff4eacb0700 1 -- 172.21.15.163:0/1148768031 shutdown_connections 2023-10-09T06:37:25.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.006+0000 7ff4eacb0700 1 -- 172.21.15.163:0/1148768031 wait complete. 2023-10-09T06:37:25.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.007+0000 7ff4eacb0700 1 Processor -- start 2023-10-09T06:37:25.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.008+0000 7ff4eacb0700 1 -- start start 2023-10-09T06:37:25.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.008+0000 7ff4eacb0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4e4159890 0x7ff4e4158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:25.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.008+0000 7ff4eacb0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4e4159c60 0x7ff4e40c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:25.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.008+0000 7ff4eacb0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4e415db10 0x7ff4e40c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:25.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.008+0000 7ff4eacb0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e40b4650 con 0x7ff4e4159890 2023-10-09T06:37:25.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.008+0000 7ff4eacb0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e40c7bd0 con 0x7ff4e415db10 2023-10-09T06:37:25.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.008+0000 7ff4eacb0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e416a890 con 0x7ff4e4159c60 2023-10-09T06:37:25.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.008+0000 7ff4e8a4c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4e4159890 0x7ff4e4158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:25.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.008+0000 7ff4e8a4c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4e4159890 0x7ff4e4158250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58576/0 (socket says 172.21.15.163:58576) 2023-10-09T06:37:25.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.008+0000 7ff4e8a4c700 1 -- 172.21.15.163:0/3987939604 learned_addr learned my addr 172.21.15.163:0/3987939604 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:25.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.009+0000 7ff4e3fff700 1 --2- 172.21.15.163:0/3987939604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4e4159c60 0x7ff4e40c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:25.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.009+0000 7ff4e924d700 1 --2- 172.21.15.163:0/3987939604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4e415db10 0x7ff4e40c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:25.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.009+0000 7ff4e8a4c700 1 -- 172.21.15.163:0/3987939604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4e4159c60 msgr2=0x7ff4e40c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:25.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.009+0000 7ff4e8a4c700 1 --2- 172.21.15.163:0/3987939604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4e4159c60 0x7ff4e40c5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:25.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.009+0000 7ff4e8a4c700 1 -- 172.21.15.163:0/3987939604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4e415db10 msgr2=0x7ff4e40c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:25.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.009+0000 7ff4e8a4c700 1 --2- 172.21.15.163:0/3987939604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4e415db10 0x7ff4e40c82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:25.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.009+0000 7ff4e8a4c700 1 -- 172.21.15.163:0/3987939604 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d4009ae0 con 0x7ff4e4159890 2023-10-09T06:37:25.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.009+0000 7ff4e3fff700 1 --2- 172.21.15.163:0/3987939604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4e4159c60 0x7ff4e40c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:25.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.009+0000 7ff4e8a4c700 1 --2- 172.21.15.163:0/3987939604 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4e4159890 0x7ff4e4158250 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4070e20 tx=0x7ff4d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:25.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.009+0000 7ff4e0ff9700 1 -- 172.21.15.163:0/3987939604 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4012aa0 con 0x7ff4e4159890 2023-10-09T06:37:25.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.009+0000 7ff4e0ff9700 1 -- 172.21.15.163:0/3987939604 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d4012c00 con 0x7ff4e4159890 2023-10-09T06:37:25.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.009+0000 7ff4eacb0700 1 -- 172.21.15.163:0/3987939604 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e4158790 con 0x7ff4e4159890 2023-10-09T06:37:25.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.009+0000 7ff4e0ff9700 1 -- 172.21.15.163:0/3987939604 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d401d420 con 0x7ff4e4159890 2023-10-09T06:37:25.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.009+0000 7ff4eacb0700 1 -- 172.21.15.163:0/3987939604 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e416a240 con 0x7ff4e4159890 2023-10-09T06:37:25.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.010+0000 7ff4eacb0700 1 -- 172.21.15.163:0/3987939604 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4e404c3c0 con 0x7ff4e4159890 2023-10-09T06:37:25.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.010+0000 7ff4e0ff9700 1 -- 172.21.15.163:0/3987939604 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff4d401d810 con 0x7ff4e4159890 2023-10-09T06:37:25.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.011+0000 7ff4e0ff9700 1 --2- 172.21.15.163:0/3987939604 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff4d0067320 0x7ff4d00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:25.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.011+0000 7ff4e0ff9700 1 -- 172.21.15.163:0/3987939604 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 7501+0+0 (secure 0 0 0) 0x7ff4d4086c20 con 0x7ff4e4159890 2023-10-09T06:37:25.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.011+0000 7ff4e3fff700 1 --2- 172.21.15.163:0/3987939604 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff4d0067320 0x7ff4d00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:25.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.011+0000 7ff4e3fff700 1 --2- 172.21.15.163:0/3987939604 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff4d0067320 0x7ff4d00697d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff4e40c8ea0 tx=0x7ff4d8006f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:25.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.015+0000 7ff4e0ff9700 1 -- 172.21.15.163:0/3987939604 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff4d40530a0 con 0x7ff4e4159890 2023-10-09T06:37:25.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.226+0000 7ff4eacb0700 1 -- 172.21.15.163:0/3987939604 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4e406a620 con 0x7ff4d0067320 2023-10-09T06:37:25.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.228+0000 7ff4e0ff9700 1 -- 172.21.15.163:0/3987939604 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+151 (secure 0 0 0) 0x7ff4e406a620 con 0x7ff4d0067320 2023-10-09T06:37:25.228 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:37:25.229 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:25.229 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "cephfs" 2023-10-09T06:37:25.229 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:25.229 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:25.229 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "volume_0000000000905867" 2023-10-09T06:37:25.229 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:25.229 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:25.229 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "volume_0000000000905868" 2023-10-09T06:37:25.229 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:25.229 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:37:25.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.230+0000 7ff4eacb0700 1 -- 172.21.15.163:0/3987939604 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff4d0067320 msgr2=0x7ff4d00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:25.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.230+0000 7ff4eacb0700 1 --2- 172.21.15.163:0/3987939604 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff4d0067320 0x7ff4d00697d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff4e40c8ea0 tx=0x7ff4d8006f90 comp rx=0 tx=0).stop 2023-10-09T06:37:25.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.230+0000 7ff4eacb0700 1 -- 172.21.15.163:0/3987939604 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4e4159890 msgr2=0x7ff4e4158250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:25.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.230+0000 7ff4eacb0700 1 --2- 172.21.15.163:0/3987939604 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4e4159890 0x7ff4e4158250 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4070e20 tx=0x7ff4d4017000 comp rx=0 tx=0).stop 2023-10-09T06:37:25.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.230+0000 7ff4eacb0700 1 -- 172.21.15.163:0/3987939604 shutdown_connections 2023-10-09T06:37:25.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.230+0000 7ff4eacb0700 1 --2- 172.21.15.163:0/3987939604 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff4d0067320 0x7ff4d00697d0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:25.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.230+0000 7ff4eacb0700 1 --2- 172.21.15.163:0/3987939604 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4e4159890 0x7ff4e4158250 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:25.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.230+0000 7ff4eacb0700 1 --2- 172.21.15.163:0/3987939604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4e415db10 0x7ff4e40c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:25.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.230+0000 7ff4eacb0700 1 --2- 172.21.15.163:0/3987939604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4e4159c60 0x7ff4e40c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:25.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.230+0000 7ff4eacb0700 1 -- 172.21.15.163:0/3987939604 >> 172.21.15.163:0/3987939604 conn(0x7ff4e4068e20 msgr2=0x7ff4e4152ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:25.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.230+0000 7ff4eacb0700 1 -- 172.21.15.163:0/3987939604 shutdown_connections 2023-10-09T06:37:25.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.230+0000 7ff4eacb0700 1 -- 172.21.15.163:0/3987939604 wait complete. 2023-10-09T06:37:25.243 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000000905867 --yes-i-really-mean-it 2023-10-09T06:37:25.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.360+0000 7f145d1fa700 1 Processor -- start 2023-10-09T06:37:25.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.360+0000 7f145d1fa700 1 -- start start 2023-10-09T06:37:25.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.360+0000 7f145d1fa700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f145806bcd0 0x7f14580caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:25.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.360+0000 7f145d1fa700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f145806c0a0 0x7f145815a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:25.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.360+0000 7f145d1fa700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f145815d7e0 con 0x7f145815df50 2023-10-09T06:37:25.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.360+0000 7f145d1fa700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f145815dac0 con 0x7f145806c0a0 2023-10-09T06:37:25.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.360+0000 7f145d1fa700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f145815aa30 con 0x7f145806bcd0 2023-10-09T06:37:25.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.361+0000 7f145659c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f145806c0a0 0x7f145815a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:25.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.361+0000 7f145659c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f145806c0a0 0x7f145815a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58578/0 (socket says 172.21.15.163:58578) 2023-10-09T06:37:25.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.361+0000 7f145659c700 1 -- 172.21.15.163:0/1873566978 learned_addr learned my addr 172.21.15.163:0/1873566978 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:25.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.361+0000 7f1456d9d700 1 --2- 172.21.15.163:0/1873566978 >> v2:172.21.15.181:3300/0 conn(0x7f145806bcd0 0x7f14580caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:25.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.361+0000 7f145659c700 1 -- 172.21.15.163:0/1873566978 >> v2:172.21.15.181:3300/0 conn(0x7f145806bcd0 msgr2=0x7f14580caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:25.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.361+0000 7f145659c700 1 --2- 172.21.15.163:0/1873566978 >> v2:172.21.15.181:3300/0 conn(0x7f145806bcd0 0x7f14580caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:25.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.361+0000 7f145659c700 1 -- 172.21.15.163:0/1873566978 >> v1:172.21.15.181:6789/0 conn(0x7f145815df50 legacy=0x7f14580c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:25.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.361+0000 7f145659c700 1 -- 172.21.15.163:0/1873566978 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f145815ad10 con 0x7f145806c0a0 2023-10-09T06:37:25.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.361+0000 7f145659c700 1 --2- 172.21.15.163:0/1873566978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f145806c0a0 0x7f145815a1e0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f14580691a0 tx=0x7f144c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f0ff3e60471ab3 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:25.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.361+0000 7f1455d9b700 1 -- 172.21.15.163:0/1873566978 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c009e70 con 0x7f145806c0a0 2023-10-09T06:37:25.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.361+0000 7f1455d9b700 1 -- 172.21.15.163:0/1873566978 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f144c00b860 con 0x7f145806c0a0 2023-10-09T06:37:25.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.361+0000 7f1455d9b700 1 -- 172.21.15.163:0/1873566978 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c00bc20 con 0x7f145806c0a0 2023-10-09T06:37:25.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.362+0000 7f145d1fa700 1 -- 172.21.15.163:0/1873566978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f145806c0a0 msgr2=0x7f145815a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:25.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.362+0000 7f145d1fa700 1 --2- 172.21.15.163:0/1873566978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f145806c0a0 0x7f145815a1e0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f14580691a0 tx=0x7f144c009250 comp rx=0 tx=0).stop 2023-10-09T06:37:25.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.362+0000 7f145d1fa700 1 -- 172.21.15.163:0/1873566978 shutdown_connections 2023-10-09T06:37:25.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.362+0000 7f145d1fa700 1 --2- 172.21.15.163:0/1873566978 >> v2:172.21.15.181:3300/0 conn(0x7f145806bcd0 0x7f14580caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:25.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.362+0000 7f145d1fa700 1 --2- 172.21.15.163:0/1873566978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f145806c0a0 0x7f145815a1e0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:25.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.362+0000 7f145d1fa700 1 -- 172.21.15.163:0/1873566978 >> 172.21.15.163:0/1873566978 conn(0x7f14580ce600 msgr2=0x7f14580cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:25.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.362+0000 7f145d1fa700 1 -- 172.21.15.163:0/1873566978 shutdown_connections 2023-10-09T06:37:25.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.362+0000 7f145d1fa700 1 -- 172.21.15.163:0/1873566978 wait complete. 2023-10-09T06:37:25.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.363+0000 7f145d1fa700 1 Processor -- start 2023-10-09T06:37:25.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.363+0000 7f145d1fa700 1 -- start start 2023-10-09T06:37:25.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.364+0000 7f145d1fa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f145806bcd0 0x7f1458155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:25.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.364+0000 7f145d1fa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f145806c0a0 0x7f1458169e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:25.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.364+0000 7f145d1fa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f145815df50 0x7f1458147190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:25.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.364+0000 7f145d1fa700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145816d6a0 con 0x7f145806bcd0 2023-10-09T06:37:25.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.364+0000 7f145d1fa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145816a3d0 con 0x7f145806c0a0 2023-10-09T06:37:25.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.364+0000 7f145d1fa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f145816a670 con 0x7f145815df50 2023-10-09T06:37:25.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.364+0000 7f1456d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f145806bcd0 0x7f1458155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:25.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.364+0000 7f1456d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f145806bcd0 0x7f1458155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:58590/0 (socket says 172.21.15.163:58590) 2023-10-09T06:37:25.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.364+0000 7f1456d9d700 1 -- 172.21.15.163:0/3241866041 learned_addr learned my addr 172.21.15.163:0/3241866041 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:25.370 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.364+0000 7f145659c700 1 --2- 172.21.15.163:0/3241866041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f145806c0a0 0x7f1458169e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:25.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.364+0000 7f145759e700 1 --2- 172.21.15.163:0/3241866041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f145815df50 0x7f1458147190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:25.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.364+0000 7f1456d9d700 1 -- 172.21.15.163:0/3241866041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f145815df50 msgr2=0x7f1458147190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:25.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.364+0000 7f1456d9d700 1 --2- 172.21.15.163:0/3241866041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f145815df50 0x7f1458147190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:25.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.364+0000 7f1456d9d700 1 -- 172.21.15.163:0/3241866041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f145806c0a0 msgr2=0x7f1458169e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:25.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.364+0000 7f1456d9d700 1 --2- 172.21.15.163:0/3241866041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f145806c0a0 0x7f1458169e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:25.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.364+0000 7f1456d9d700 1 -- 172.21.15.163:0/3241866041 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f144c0094f0 con 0x7f145806bcd0 2023-10-09T06:37:25.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.364+0000 7f1456d9d700 1 --2- 172.21.15.163:0/3241866041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f145806bcd0 0x7f1458155df0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f144800e930 tx=0x7f14480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:25.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.364+0000 7f143effd700 1 -- 172.21.15.163:0/3241866041 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1448013680 con 0x7f145806bcd0 2023-10-09T06:37:25.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.364+0000 7f143effd700 1 -- 172.21.15.163:0/3241866041 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14480137e0 con 0x7f145806bcd0 2023-10-09T06:37:25.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.364+0000 7f145d1fa700 1 -- 172.21.15.163:0/3241866041 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f145816a950 con 0x7f145806bcd0 2023-10-09T06:37:25.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.364+0000 7f143effd700 1 -- 172.21.15.163:0/3241866041 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1448013ba0 con 0x7f145806bcd0 2023-10-09T06:37:25.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.365+0000 7f145d1fa700 1 -- 172.21.15.163:0/3241866041 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14581476d0 con 0x7f145806bcd0 2023-10-09T06:37:25.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.365+0000 7f145d1fa700 1 -- 172.21.15.163:0/3241866041 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f145804c3c0 con 0x7f145806bcd0 2023-10-09T06:37:25.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.366+0000 7f143effd700 1 -- 172.21.15.163:0/3241866041 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f144800c460 con 0x7f145806bcd0 2023-10-09T06:37:25.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.366+0000 7f143effd700 1 --2- 172.21.15.163:0/3241866041 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1444067320 0x7f14440697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:25.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.366+0000 7f143effd700 1 -- 172.21.15.163:0/3241866041 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 7501+0+0 (secure 0 0 0) 0x7f1448085470 con 0x7f145806bcd0 2023-10-09T06:37:25.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.366+0000 7f145659c700 1 --2- 172.21.15.163:0/3241866041 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1444067320 0x7f14440697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:25.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.367+0000 7f145659c700 1 --2- 172.21.15.163:0/3241866041 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1444067320 0x7f14440697d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f14580691a0 tx=0x7f144c017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:25.373 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.370+0000 7f143effd700 1 -- 172.21.15.163:0/3241866041 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1448051a00 con 0x7f145806bcd0 2023-10-09T06:37:25.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:25.580+0000 7f145d1fa700 1 -- 172.21.15.163:0/3241866041 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "volume_0000000000905867", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7f145816d7e0 con 0x7f1444067320 2023-10-09T06:37:25.586 INFO:tasks.ceph.mon.a.smithi163.stderr:2023-10-09T06:37:25.585+0000 7ff251c04700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-09T06:37:25.590 INFO:tasks.ceph.mds.a.smithi163.stderr: -37> 2023-10-09T06:37:24.536+0000 7fa2586f9700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-09T06:37:25.609 INFO:tasks.ceph.mds.a.smithi163.stdout:starting mds.a at 2023-10-09T06:37:28.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.610+0000 7f143effd700 1 -- 172.21.15.163:0/3241866041 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+110 (secure 0 0 0) 0x7f144c020040 con 0x7f1444067320 2023-10-09T06:37:28.611 INFO:teuthology.orchestra.run.smithi163.stdout:metadata pool: cephfs.volume_0000000000905867.meta data pool: ['cephfs.volume_0000000000905867.data'] removed 2023-10-09T06:37:28.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.613+0000 7f145d1fa700 1 -- 172.21.15.163:0/3241866041 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1444067320 msgr2=0x7f14440697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:28.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.613+0000 7f145d1fa700 1 --2- 172.21.15.163:0/3241866041 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1444067320 0x7f14440697d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f14580691a0 tx=0x7f144c017000 comp rx=0 tx=0).stop 2023-10-09T06:37:28.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.613+0000 7f145d1fa700 1 -- 172.21.15.163:0/3241866041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f145806bcd0 msgr2=0x7f1458155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:28.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.613+0000 7f145d1fa700 1 --2- 172.21.15.163:0/3241866041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f145806bcd0 0x7f1458155df0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f144800e930 tx=0x7f14480091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:28.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.613+0000 7f145d1fa700 1 -- 172.21.15.163:0/3241866041 shutdown_connections 2023-10-09T06:37:28.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.613+0000 7f145d1fa700 1 --2- 172.21.15.163:0/3241866041 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1444067320 0x7f14440697d0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:28.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.613+0000 7f145d1fa700 1 --2- 172.21.15.163:0/3241866041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f145806bcd0 0x7f1458155df0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:28.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.613+0000 7f145d1fa700 1 --2- 172.21.15.163:0/3241866041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f145815df50 0x7f1458147190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:28.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.613+0000 7f145d1fa700 1 --2- 172.21.15.163:0/3241866041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f145806c0a0 0x7f1458169e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:28.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.613+0000 7f145d1fa700 1 -- 172.21.15.163:0/3241866041 >> 172.21.15.163:0/3241866041 conn(0x7f14580ce600 msgr2=0x7f145806a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:28.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.613+0000 7f145d1fa700 1 -- 172.21.15.163:0/3241866041 shutdown_connections 2023-10-09T06:37:28.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.613+0000 7f145d1fa700 1 -- 172.21.15.163:0/3241866041 wait complete. 2023-10-09T06:37:28.628 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000000905868 --yes-i-really-mean-it 2023-10-09T06:37:28.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.752+0000 7f5475a7e700 1 Processor -- start 2023-10-09T06:37:28.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.752+0000 7f5475a7e700 1 -- start start 2023-10-09T06:37:28.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.753+0000 7f5475a7e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5470159c60 0x7f547015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:28.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.753+0000 7f5475a7e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f547015a160 con 0x7f5470159890 2023-10-09T06:37:28.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.753+0000 7f5475a7e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54700d13a0 con 0x7f547015db10 2023-10-09T06:37:28.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.753+0000 7f5475a7e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f547015df50 con 0x7f5470159c60 2023-10-09T06:37:28.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.753+0000 7f546effd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5470159c60 0x7f547015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:28.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.753+0000 7f546effd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5470159c60 0x7f547015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49238/0 (socket says 172.21.15.163:49238) 2023-10-09T06:37:28.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.753+0000 7f546f7fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f5470159890 0x7f547015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:52704/0 (socket says 172.21.15.163:52704) 2023-10-09T06:37:28.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.753+0000 7f546effd700 1 -- 172.21.15.163:0/915591987 learned_addr learned my addr 172.21.15.163:0/915591987 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:28.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.754+0000 7f546effd700 1 -- 172.21.15.163:0/915591987 >> v1:172.21.15.181:6789/0 conn(0x7f547015db10 legacy=0x7f5470069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:28.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.754+0000 7f546effd700 1 -- 172.21.15.163:0/915591987 >> v1:172.21.15.163:6789/0 conn(0x7f5470159890 legacy=0x7f547015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:28.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.754+0000 7f546e7fc700 1 -- 172.21.15.163:0/915591987 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1495422416 0 0) 0x7f547015a160 con 0x7f5470159890 2023-10-09T06:37:28.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.754+0000 7f546effd700 1 -- 172.21.15.163:0/915591987 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547015e140 con 0x7f5470159c60 2023-10-09T06:37:28.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.754+0000 7f546effd700 1 --2- 172.21.15.163:0/915591987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5470159c60 0x7f547015d5d0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f54640099b0 tx=0x7f5464009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52423123a681661 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:28.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.754+0000 7f546e7fc700 1 -- 172.21.15.163:0/915591987 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546400b820 con 0x7f5470159c60 2023-10-09T06:37:28.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.754+0000 7f546e7fc700 1 -- 172.21.15.163:0/915591987 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f546400b980 con 0x7f5470159c60 2023-10-09T06:37:28.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.754+0000 7f546e7fc700 1 -- 172.21.15.163:0/915591987 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546400bd20 con 0x7f5470159c60 2023-10-09T06:37:28.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.754+0000 7f5475a7e700 1 -- 172.21.15.163:0/915591987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5470159c60 msgr2=0x7f547015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:28.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.754+0000 7f5475a7e700 1 --2- 172.21.15.163:0/915591987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5470159c60 0x7f547015d5d0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f54640099b0 tx=0x7f5464009250 comp rx=0 tx=0).stop 2023-10-09T06:37:28.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.754+0000 7f5475a7e700 1 -- 172.21.15.163:0/915591987 shutdown_connections 2023-10-09T06:37:28.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.754+0000 7f5475a7e700 1 --2- 172.21.15.163:0/915591987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5470159c60 0x7f547015d5d0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:28.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.754+0000 7f5475a7e700 1 -- 172.21.15.163:0/915591987 >> 172.21.15.163:0/915591987 conn(0x7f5470068e20 msgr2=0x7f54700d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:28.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.754+0000 7f5475a7e700 1 -- 172.21.15.163:0/915591987 shutdown_connections 2023-10-09T06:37:28.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.754+0000 7f5475a7e700 1 -- 172.21.15.163:0/915591987 wait complete. 2023-10-09T06:37:28.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.756+0000 7f5475a7e700 1 Processor -- start 2023-10-09T06:37:28.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.756+0000 7f5475a7e700 1 -- start start 2023-10-09T06:37:28.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.756+0000 7f5475a7e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5470159890 0x7f54701504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:28.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.756+0000 7f5475a7e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5470159c60 0x7f54700cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:28.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.756+0000 7f5475a7e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f547015db10 0x7f54701584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:28.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.756+0000 7f5475a7e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54700ce610 con 0x7f547015db10 2023-10-09T06:37:28.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.756+0000 7f5475a7e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54700cda00 con 0x7f5470159c60 2023-10-09T06:37:28.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.756+0000 7f5475a7e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54700cdce0 con 0x7f5470159890 2023-10-09T06:37:28.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.757+0000 7f546ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f547015db10 0x7f54701584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:28.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.757+0000 7f546ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f547015db10 0x7f54701584b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49244/0 (socket says 172.21.15.163:49244) 2023-10-09T06:37:28.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.757+0000 7f546ffff700 1 -- 172.21.15.163:0/1102639420 learned_addr learned my addr 172.21.15.163:0/1102639420 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:28.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.757+0000 7f546f7fe700 1 --2- 172.21.15.163:0/1102639420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5470159890 0x7f54701504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:28.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.757+0000 7f546ffff700 1 -- 172.21.15.163:0/1102639420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5470159890 msgr2=0x7f54701504c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:28.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.757+0000 7f546effd700 1 --2- 172.21.15.163:0/1102639420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5470159c60 0x7f54700cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:28.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.757+0000 7f546ffff700 1 --2- 172.21.15.163:0/1102639420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5470159890 0x7f54701504c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:28.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.757+0000 7f546ffff700 1 -- 172.21.15.163:0/1102639420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5470159c60 msgr2=0x7f54700cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:28.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.757+0000 7f546ffff700 1 --2- 172.21.15.163:0/1102639420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5470159c60 0x7f54700cd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:28.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.757+0000 7f546ffff700 1 -- 172.21.15.163:0/1102639420 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54640094f0 con 0x7f547015db10 2023-10-09T06:37:28.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.757+0000 7f546f7fe700 1 --2- 172.21.15.163:0/1102639420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5470159890 0x7f54701504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:37:28.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.757+0000 7f546ffff700 1 --2- 172.21.15.163:0/1102639420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f547015db10 0x7f54701584b0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f546800c5b0 tx=0x7f5468007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:28.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.757+0000 7f54577fe700 1 -- 172.21.15.163:0/1102639420 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5468011590 con 0x7f547015db10 2023-10-09T06:37:28.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.757+0000 7f54577fe700 1 -- 172.21.15.163:0/1102639420 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54680116f0 con 0x7f547015db10 2023-10-09T06:37:28.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.757+0000 7f5475a7e700 1 -- 172.21.15.163:0/1102639420 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54700cdfc0 con 0x7f547015db10 2023-10-09T06:37:28.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.757+0000 7f54577fe700 1 -- 172.21.15.163:0/1102639420 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5468011ab0 con 0x7f547015db10 2023-10-09T06:37:28.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.757+0000 7f5475a7e700 1 -- 172.21.15.163:0/1102639420 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5470150a00 con 0x7f547015db10 2023-10-09T06:37:28.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.758+0000 7f5475a7e700 1 -- 172.21.15.163:0/1102639420 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f547004c3c0 con 0x7f547015db10 2023-10-09T06:37:28.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.758+0000 7f54577fe700 1 -- 172.21.15.163:0/1102639420 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f546801c460 con 0x7f547015db10 2023-10-09T06:37:28.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.759+0000 7f54577fe700 1 --2- 172.21.15.163:0/1102639420 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f545c067320 0x7f545c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:28.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.759+0000 7f54577fe700 1 -- 172.21.15.163:0/1102639420 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6689+0+0 (secure 0 0 0) 0x7f5468084c80 con 0x7f547015db10 2023-10-09T06:37:28.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.759+0000 7f546f7fe700 1 --2- 172.21.15.163:0/1102639420 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f545c067320 0x7f545c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:28.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.760+0000 7f546f7fe700 1 --2- 172.21.15.163:0/1102639420 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f545c067320 0x7f545c0697d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f546000d270 tx=0x7f5460009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:28.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.763+0000 7f54577fe700 1 -- 172.21.15.163:0/1102639420 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5468051530 con 0x7f547015db10 2023-10-09T06:37:28.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:28.972+0000 7f5475a7e700 1 -- 172.21.15.163:0/1102639420 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "volume_0000000000905868", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7f547006a5b0 con 0x7f545c067320 2023-10-09T06:37:28.977 INFO:tasks.ceph.mon.a.smithi163.stderr:2023-10-09T06:37:28.977+0000 7ff251c04700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-09T06:37:29.009 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-09T06:37:31.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:31.992+0000 7f54577fe700 1 -- 172.21.15.163:0/1102639420 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+110 (secure 0 0 0) 0x7f547006a5b0 con 0x7f545c067320 2023-10-09T06:37:31.994 INFO:teuthology.orchestra.run.smithi163.stdout:metadata pool: cephfs.volume_0000000000905868.meta data pool: ['cephfs.volume_0000000000905868.data'] removed 2023-10-09T06:37:31.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:31.995+0000 7f5475a7e700 1 -- 172.21.15.163:0/1102639420 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f545c067320 msgr2=0x7f545c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:31.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:31.995+0000 7f5475a7e700 1 --2- 172.21.15.163:0/1102639420 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f545c067320 0x7f545c0697d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f546000d270 tx=0x7f5460009040 comp rx=0 tx=0).stop 2023-10-09T06:37:31.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:31.995+0000 7f5475a7e700 1 -- 172.21.15.163:0/1102639420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f547015db10 msgr2=0x7f54701584b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:31.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:31.995+0000 7f5475a7e700 1 --2- 172.21.15.163:0/1102639420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f547015db10 0x7f54701584b0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f546800c5b0 tx=0x7f5468007f60 comp rx=0 tx=0).stop 2023-10-09T06:37:31.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:31.995+0000 7f5475a7e700 1 -- 172.21.15.163:0/1102639420 shutdown_connections 2023-10-09T06:37:31.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:31.995+0000 7f5475a7e700 1 --2- 172.21.15.163:0/1102639420 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f545c067320 0x7f545c0697d0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:31.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:31.995+0000 7f5475a7e700 1 --2- 172.21.15.163:0/1102639420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f547015db10 0x7f54701584b0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:31.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:31.995+0000 7f5475a7e700 1 --2- 172.21.15.163:0/1102639420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5470159890 0x7f54701504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:31.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:31.995+0000 7f5475a7e700 1 --2- 172.21.15.163:0/1102639420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5470159c60 0x7f54700cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:31.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:31.995+0000 7f5475a7e700 1 -- 172.21.15.163:0/1102639420 >> 172.21.15.163:0/1102639420 conn(0x7f5470068e20 msgr2=0x7f5470153d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:31.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:31.996+0000 7f5475a7e700 1 -- 172.21.15.163:0/1102639420 shutdown_connections 2023-10-09T06:37:31.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:31.996+0000 7f5475a7e700 1 -- 172.21.15.163:0/1102639420 wait complete. 2023-10-09T06:37:32.012 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-09T06:37:32.012 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:37:32.057 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:37:32.105 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-09T06:37:32.106 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-09T06:37:32.106 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:37:32.106 DEBUG:teuthology.orchestra.run.smithi163:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-09T06:37:32.147 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi163.front.sepia.ceph.com 2023-10-09T06:37:32.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:32.147 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-09T06:37:32.204 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-09T06:37:32.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:37:32.204 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-09T06:37:32.247 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-09T06:37:32.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:32.247 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-09T06:37:32.303 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:37:32.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.419+0000 7f77f9be8700 1 Processor -- start 2023-10-09T06:37:32.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.420+0000 7f77f9be8700 1 -- start start 2023-10-09T06:37:32.494 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.420+0000 7f77f9be8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77f4142ea0 0x7f77f40b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:32.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.420+0000 7f77f9be8700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f40b40f0 con 0x7f77f40ad870 2023-10-09T06:37:32.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.420+0000 7f77f9be8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f4069640 con 0x7f77f406c0f0 2023-10-09T06:37:32.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.420+0000 7f77f9be8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f4069920 con 0x7f77f4142ea0 2023-10-09T06:37:32.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.420+0000 7f77ebfff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f77f40ad870 0x7f77f406b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:52714/0 (socket says 172.21.15.163:52714) 2023-10-09T06:37:32.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.421+0000 7f77ebfff700 1 -- 172.21.15.163:0/3854563847 learned_addr learned my addr 172.21.15.163:0/3854563847 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:32.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.421+0000 7f77f8be6700 1 --2- 172.21.15.163:0/3854563847 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77f4142ea0 0x7f77f40b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:32.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.421+0000 7f77eb7fe700 1 -- 172.21.15.163:0/3854563847 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2174858909 0 0) 0x7f77f40b40f0 con 0x7f77f40ad870 2023-10-09T06:37:32.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.421+0000 7f77eb7fe700 1 -- 172.21.15.163:0/3854563847 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77dc003570 con 0x7f77f40ad870 2023-10-09T06:37:32.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.421+0000 7f77eb7fe700 1 -- 172.21.15.163:0/3854563847 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 306999515 0 0) 0x7f77f4069640 con 0x7f77f406c0f0 2023-10-09T06:37:32.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.421+0000 7f77eb7fe700 1 -- 172.21.15.163:0/3854563847 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77f40b40f0 con 0x7f77f406c0f0 2023-10-09T06:37:32.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.421+0000 7f77f8be6700 1 -- 172.21.15.163:0/3854563847 >> v1:172.21.15.181:6789/0 conn(0x7f77f406c0f0 legacy=0x7f77f406a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:32.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.421+0000 7f77f8be6700 1 -- 172.21.15.163:0/3854563847 >> v1:172.21.15.163:6789/0 conn(0x7f77f40ad870 legacy=0x7f77f406b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:32.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.421+0000 7f77f8be6700 1 -- 172.21.15.163:0/3854563847 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f4069c00 con 0x7f77f4142ea0 2023-10-09T06:37:32.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.421+0000 7f77f8be6700 1 --2- 172.21.15.163:0/3854563847 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77f4142ea0 0x7f77f40b3630 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f77f0009e90 tx=0x7f77f0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33a3621e08706ef0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:32.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.421+0000 7f77eb7fe700 1 -- 172.21.15.163:0/3854563847 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f000e070 con 0x7f77f4142ea0 2023-10-09T06:37:32.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.421+0000 7f77eb7fe700 1 -- 172.21.15.163:0/3854563847 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f000bb80 con 0x7f77f4142ea0 2023-10-09T06:37:32.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.421+0000 7f77eb7fe700 1 -- 172.21.15.163:0/3854563847 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f0012430 con 0x7f77f4142ea0 2023-10-09T06:37:32.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.422+0000 7f77f9be8700 1 -- 172.21.15.163:0/3854563847 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77f4142ea0 msgr2=0x7f77f40b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:32.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.422+0000 7f77f9be8700 1 --2- 172.21.15.163:0/3854563847 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77f4142ea0 0x7f77f40b3630 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f77f0009e90 tx=0x7f77f0009780 comp rx=0 tx=0).stop 2023-10-09T06:37:32.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.422+0000 7f77f9be8700 1 -- 172.21.15.163:0/3854563847 shutdown_connections 2023-10-09T06:37:32.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.422+0000 7f77f9be8700 1 --2- 172.21.15.163:0/3854563847 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77f4142ea0 0x7f77f40b3630 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:32.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.422+0000 7f77f9be8700 1 -- 172.21.15.163:0/3854563847 >> 172.21.15.163:0/3854563847 conn(0x7f77f40ce430 msgr2=0x7f77f40ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:32.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.422+0000 7f77f9be8700 1 -- 172.21.15.163:0/3854563847 shutdown_connections 2023-10-09T06:37:32.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.422+0000 7f77f9be8700 1 -- 172.21.15.163:0/3854563847 wait complete. 2023-10-09T06:37:32.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.423+0000 7f77f9be8700 1 Processor -- start 2023-10-09T06:37:32.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.423+0000 7f77f9be8700 1 -- start start 2023-10-09T06:37:32.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.424+0000 7f77f9be8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77f406c0f0 0x7f77f413e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:32.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.424+0000 7f77f9be8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77f40c78c0 0x7f77f40aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:32.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.424+0000 7f77f9be8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77f40af490 0x7f77f4140390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:32.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.424+0000 7f77f9be8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f40af900 con 0x7f77f40af490 2023-10-09T06:37:32.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.424+0000 7f77f9be8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f4141100 con 0x7f77f40c78c0 2023-10-09T06:37:32.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.424+0000 7f77f9be8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f415c900 con 0x7f77f406c0f0 2023-10-09T06:37:32.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.424+0000 7f77f93e7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77f40af490 0x7f77f4140390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:32.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.424+0000 7f77f93e7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77f40af490 0x7f77f4140390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49260/0 (socket says 172.21.15.163:49260) 2023-10-09T06:37:32.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.424+0000 7f77f93e7700 1 -- 172.21.15.163:0/2624446679 learned_addr learned my addr 172.21.15.163:0/2624446679 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:32.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.424+0000 7f77f8be6700 1 --2- 172.21.15.163:0/2624446679 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77f406c0f0 0x7f77f413e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:32.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.424+0000 7f77ebfff700 1 --2- 172.21.15.163:0/2624446679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77f40c78c0 0x7f77f40aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:32.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.424+0000 7f77f93e7700 1 -- 172.21.15.163:0/2624446679 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77f406c0f0 msgr2=0x7f77f413e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:32.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.424+0000 7f77f93e7700 1 --2- 172.21.15.163:0/2624446679 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77f406c0f0 0x7f77f413e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:32.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.424+0000 7f77f93e7700 1 -- 172.21.15.163:0/2624446679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77f40c78c0 msgr2=0x7f77f40aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:32.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.424+0000 7f77f93e7700 1 --2- 172.21.15.163:0/2624446679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77f40c78c0 0x7f77f40aef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:32.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.424+0000 7f77f93e7700 1 -- 172.21.15.163:0/2624446679 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f0009a20 con 0x7f77f40af490 2023-10-09T06:37:32.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.425+0000 7f77f93e7700 1 --2- 172.21.15.163:0/2624446679 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77f40af490 0x7f77f4140390 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f77ec00c590 tx=0x7f77ec007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:32.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.425+0000 7f77e8ff9700 1 -- 172.21.15.163:0/2624446679 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec016070 con 0x7f77f40af490 2023-10-09T06:37:32.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.425+0000 7f77e8ff9700 1 -- 172.21.15.163:0/2624446679 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ec011590 con 0x7f77f40af490 2023-10-09T06:37:32.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.425+0000 7f77e8ff9700 1 -- 172.21.15.163:0/2624446679 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec011970 con 0x7f77f40af490 2023-10-09T06:37:32.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.425+0000 7f77f9be8700 1 -- 172.21.15.163:0/2624446679 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f415bfd0 con 0x7f77f40af490 2023-10-09T06:37:32.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.425+0000 7f77f9be8700 1 -- 172.21.15.163:0/2624446679 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f415c590 con 0x7f77f40af490 2023-10-09T06:37:32.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.426+0000 7f77f9be8700 1 -- 172.21.15.163:0/2624446679 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f415ca70 con 0x7f77f40af490 2023-10-09T06:37:32.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.426+0000 7f77e8ff9700 1 -- 172.21.15.163:0/2624446679 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f77ec012420 con 0x7f77f40af490 2023-10-09T06:37:32.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.426+0000 7f77e8ff9700 1 --2- 172.21.15.163:0/2624446679 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f77e0067320 0x7f77e00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:32.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.426+0000 7f77e8ff9700 1 -- 172.21.15.163:0/2624446679 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5877+0+0 (secure 0 0 0) 0x7f77ec0848a0 con 0x7f77f40af490 2023-10-09T06:37:32.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.427+0000 7f77f8be6700 1 --2- 172.21.15.163:0/2624446679 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f77e0067320 0x7f77e00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:32.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.427+0000 7f77f8be6700 1 --2- 172.21.15.163:0/2624446679 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f77e0067320 0x7f77e00697d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f77f40cf500 tx=0x7f77f0019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:32.499 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.430+0000 7f77e8ff9700 1 -- 172.21.15.163:0/2624446679 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f77ec051480 con 0x7f77f40af490 2023-10-09T06:37:32.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.659+0000 7f77f9be8700 1 -- 172.21.15.163:0/2624446679 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77f404c3c0 con 0x7f77f40af490 2023-10-09T06:37:32.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.659+0000 7f77e8ff9700 1 -- 172.21.15.163:0/2624446679 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f77ec054aa0 con 0x7f77f40af490 2023-10-09T06:37:32.660 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:32.660 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":85,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2821591245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2821591245},{"type":"v1","addr":"172.21.15.181:6836","nonce":2821591245}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":70},{"gid":6784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1929060525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1929060525},{"type":"v1","addr":"172.21.15.181:6837","nonce":1929060525}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":7371,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/205885744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":205885744},{"type":"v1","addr":"172.21.15.163:6834","nonce":205885744}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:37:08.129202+0000","modified":"2023-10-09T06:37:11.774637+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7161},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7161":{"gid":7161,"name":"c","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/571471083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":571471083},{"type":"v1","addr":"172.21.15.163:6835","nonce":571471083}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-09T06:37:32.662 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.661+0000 7f77f9be8700 1 -- 172.21.15.163:0/2624446679 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f77e0067320 msgr2=0x7f77e00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:32.662 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.661+0000 7f77f9be8700 1 --2- 172.21.15.163:0/2624446679 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f77e0067320 0x7f77e00697d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f77f40cf500 tx=0x7f77f0019000 comp rx=0 tx=0).stop 2023-10-09T06:37:32.662 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.661+0000 7f77f9be8700 1 -- 172.21.15.163:0/2624446679 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77f40af490 msgr2=0x7f77f4140390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:32.662 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.661+0000 7f77f9be8700 1 --2- 172.21.15.163:0/2624446679 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77f40af490 0x7f77f4140390 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f77ec00c590 tx=0x7f77ec007d80 comp rx=0 tx=0).stop 2023-10-09T06:37:32.662 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.661+0000 7f77f9be8700 1 -- 172.21.15.163:0/2624446679 shutdown_connections 2023-10-09T06:37:32.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.661+0000 7f77f9be8700 1 --2- 172.21.15.163:0/2624446679 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f77e0067320 0x7f77e00697d0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:32.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.661+0000 7f77f9be8700 1 --2- 172.21.15.163:0/2624446679 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77f40af490 0x7f77f4140390 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:32.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.661+0000 7f77f9be8700 1 --2- 172.21.15.163:0/2624446679 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77f406c0f0 0x7f77f413e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:32.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.661+0000 7f77f9be8700 1 --2- 172.21.15.163:0/2624446679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77f40c78c0 0x7f77f40aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:32.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.661+0000 7f77f9be8700 1 -- 172.21.15.163:0/2624446679 >> 172.21.15.163:0/2624446679 conn(0x7f77f40ce430 msgr2=0x7f77f4141ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:32.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.661+0000 7f77f9be8700 1 -- 172.21.15.163:0/2624446679 shutdown_connections 2023-10-09T06:37:32.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.661+0000 7f77f9be8700 1 -- 172.21.15.163:0/2624446679 wait complete. 2023-10-09T06:37:32.664 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 85 2023-10-09T06:37:32.674 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:37:32.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.789+0000 7f8521f78700 1 Processor -- start 2023-10-09T06:37:32.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.789+0000 7f8521f78700 1 -- start start 2023-10-09T06:37:32.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.790+0000 7f8521f78700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f851c159c60 0x7f851c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:32.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.790+0000 7f8521f78700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f851c15db10 0x7f851c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:32.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.790+0000 7f8521f78700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f851c15a160 con 0x7f851c159890 2023-10-09T06:37:32.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.790+0000 7f8521f78700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f851c15df50 con 0x7f851c159c60 2023-10-09T06:37:32.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.790+0000 7f8521f78700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f851c0d13a0 con 0x7f851c15db10 2023-10-09T06:37:32.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.790+0000 7f851affd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f851c159c60 0x7f851c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:32.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.790+0000 7f851affd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f851c159c60 0x7f851c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49268/0 (socket says 172.21.15.163:49268) 2023-10-09T06:37:32.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.790+0000 7f851affd700 1 -- 172.21.15.163:0/3951417531 learned_addr learned my addr 172.21.15.163:0/3951417531 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:32.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.790+0000 7f851bfff700 1 --2- 172.21.15.163:0/3951417531 >> v2:172.21.15.181:3300/0 conn(0x7f851c15db10 0x7f851c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:32.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.791+0000 7f851affd700 1 -- 172.21.15.163:0/3951417531 >> v2:172.21.15.181:3300/0 conn(0x7f851c15db10 msgr2=0x7f851c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:32.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.791+0000 7f851affd700 1 --2- 172.21.15.163:0/3951417531 >> v2:172.21.15.181:3300/0 conn(0x7f851c15db10 0x7f851c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:32.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.791+0000 7f851affd700 1 -- 172.21.15.163:0/3951417531 >> v1:172.21.15.181:6789/0 conn(0x7f851c159890 legacy=0x7f851c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:32.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.791+0000 7f851affd700 1 -- 172.21.15.163:0/3951417531 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851c15e140 con 0x7f851c159c60 2023-10-09T06:37:32.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.791+0000 7f851affd700 1 --2- 172.21.15.163:0/3951417531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f851c159c60 0x7f851c15d5d0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f851c06a890 tx=0x7f8510009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ffb293524de417 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:32.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.791+0000 7f851a7fc700 1 -- 172.21.15.163:0/3951417531 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510009e70 con 0x7f851c159c60 2023-10-09T06:37:32.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.791+0000 7f851a7fc700 1 -- 172.21.15.163:0/3951417531 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851000b860 con 0x7f851c159c60 2023-10-09T06:37:32.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.791+0000 7f851a7fc700 1 -- 172.21.15.163:0/3951417531 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851000bc00 con 0x7f851c159c60 2023-10-09T06:37:32.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.791+0000 7f8521f78700 1 -- 172.21.15.163:0/3951417531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f851c159c60 msgr2=0x7f851c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:32.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.791+0000 7f8521f78700 1 --2- 172.21.15.163:0/3951417531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f851c159c60 0x7f851c15d5d0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f851c06a890 tx=0x7f8510009250 comp rx=0 tx=0).stop 2023-10-09T06:37:32.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.791+0000 7f8521f78700 1 -- 172.21.15.163:0/3951417531 shutdown_connections 2023-10-09T06:37:32.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.791+0000 7f8521f78700 1 --2- 172.21.15.163:0/3951417531 >> v2:172.21.15.181:3300/0 conn(0x7f851c15db10 0x7f851c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:32.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.791+0000 7f8521f78700 1 --2- 172.21.15.163:0/3951417531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f851c159c60 0x7f851c15d5d0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:32.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.792+0000 7f8521f78700 1 -- 172.21.15.163:0/3951417531 >> 172.21.15.163:0/3951417531 conn(0x7f851c068e20 msgr2=0x7f851c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:32.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.792+0000 7f8521f78700 1 -- 172.21.15.163:0/3951417531 shutdown_connections 2023-10-09T06:37:32.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.792+0000 7f8521f78700 1 -- 172.21.15.163:0/3951417531 wait complete. 2023-10-09T06:37:32.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.793+0000 7f8521f78700 1 Processor -- start 2023-10-09T06:37:32.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.793+0000 7f8521f78700 1 -- start start 2023-10-09T06:37:32.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.793+0000 7f8521f78700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f851c159890 0x7f851c0c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:32.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.793+0000 7f8521f78700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f851c159c60 0x7f851c140b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:32.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.793+0000 7f8521f78700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f851c15db10 0x7f851c0cf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:32.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.793+0000 7f8521f78700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c0ca970 con 0x7f851c159890 2023-10-09T06:37:32.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.793+0000 7f8521f78700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c0c9df0 con 0x7f851c15db10 2023-10-09T06:37:32.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.793+0000 7f8521f78700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851c0ca0d0 con 0x7f851c159c60 2023-10-09T06:37:32.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.794+0000 7f851b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f851c159890 0x7f851c0c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:32.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.794+0000 7f851b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f851c159890 0x7f851c0c9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49280/0 (socket says 172.21.15.163:49280) 2023-10-09T06:37:32.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.794+0000 7f851b7fe700 1 -- 172.21.15.163:0/546712442 learned_addr learned my addr 172.21.15.163:0/546712442 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:32.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.794+0000 7f851affd700 1 --2- 172.21.15.163:0/546712442 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f851c159c60 0x7f851c140b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:32.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.794+0000 7f851bfff700 1 --2- 172.21.15.163:0/546712442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f851c15db10 0x7f851c0cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:32.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.794+0000 7f851b7fe700 1 -- 172.21.15.163:0/546712442 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f851c159c60 msgr2=0x7f851c140b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:32.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.794+0000 7f851b7fe700 1 --2- 172.21.15.163:0/546712442 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f851c159c60 0x7f851c140b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:32.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.794+0000 7f851b7fe700 1 -- 172.21.15.163:0/546712442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f851c15db10 msgr2=0x7f851c0cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:32.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.794+0000 7f851b7fe700 1 --2- 172.21.15.163:0/546712442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f851c15db10 0x7f851c0cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:32.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.794+0000 7f851b7fe700 1 -- 172.21.15.163:0/546712442 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85100094f0 con 0x7f851c159890 2023-10-09T06:37:32.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.794+0000 7f851bfff700 1 --2- 172.21.15.163:0/546712442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f851c15db10 0x7f851c0cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:32.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.794+0000 7f851affd700 1 --2- 172.21.15.163:0/546712442 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f851c159c60 0x7f851c140b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:32.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.794+0000 7f851b7fe700 1 --2- 172.21.15.163:0/546712442 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f851c159890 0x7f851c0c9510 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f850c00cb10 tx=0x7f850c0017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:32.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.794+0000 7f85037fe700 1 -- 172.21.15.163:0/546712442 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c00a760 con 0x7f851c159890 2023-10-09T06:37:32.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.794+0000 7f85037fe700 1 -- 172.21.15.163:0/546712442 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850c00a8e0 con 0x7f851c159890 2023-10-09T06:37:32.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.794+0000 7f8521f78700 1 -- 172.21.15.163:0/546712442 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f851c0ca3b0 con 0x7f851c159890 2023-10-09T06:37:32.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.794+0000 7f85037fe700 1 -- 172.21.15.163:0/546712442 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c00aca0 con 0x7f851c159890 2023-10-09T06:37:32.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.794+0000 7f8521f78700 1 -- 172.21.15.163:0/546712442 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f851c0c9a50 con 0x7f851c159890 2023-10-09T06:37:32.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.795+0000 7f8521f78700 1 -- 172.21.15.163:0/546712442 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f851c04c3c0 con 0x7f851c159890 2023-10-09T06:37:32.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.795+0000 7f85037fe700 1 -- 172.21.15.163:0/546712442 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f850c003030 con 0x7f851c159890 2023-10-09T06:37:32.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.796+0000 7f85037fe700 1 --2- 172.21.15.163:0/546712442 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8508067320 0x7f85080697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:32.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.796+0000 7f85037fe700 1 -- 172.21.15.163:0/546712442 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5877+0+0 (secure 0 0 0) 0x7f850c084de0 con 0x7f851c159890 2023-10-09T06:37:32.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.796+0000 7f851affd700 1 --2- 172.21.15.163:0/546712442 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8508067320 0x7f85080697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:32.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.796+0000 7f851affd700 1 --2- 172.21.15.163:0/546712442 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8508067320 0x7f85080697d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f851c06a890 tx=0x7f8510017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:32.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.800+0000 7f85037fe700 1 -- 172.21.15.163:0/546712442 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f850c0519c0 con 0x7f851c159890 2023-10-09T06:37:32.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.988+0000 7f8521f78700 1 -- 172.21.15.163:0/546712442 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f851c0b2c50 con 0x7f851c159890 2023-10-09T06:37:32.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.989+0000 7f85037fe700 1 -- 172.21.15.163:0/546712442 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+15675 (secure 0 0 0) 0x7f850c054fe0 con 0x7f851c159890 2023-10-09T06:37:32.990 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:32.990 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":125,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:37:31.984855+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:37:06.472811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-09T06:37:07.096712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6837/3075056851":"2023-10-10T06:37:28.977957+0000","172.21.15.181:6835/3075056851":"2023-10-10T06:37:28.977957+0000","172.21.15.163:6834/1355789800":"2023-10-10T06:37:25.586483+0000","172.21.15.163:6832/1355789800":"2023-10-10T06:37:25.586483+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:37:32.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.991+0000 7f8521f78700 1 -- 172.21.15.163:0/546712442 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8508067320 msgr2=0x7f85080697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:32.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.991+0000 7f8521f78700 1 --2- 172.21.15.163:0/546712442 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8508067320 0x7f85080697d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f851c06a890 tx=0x7f8510017000 comp rx=0 tx=0).stop 2023-10-09T06:37:32.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.991+0000 7f8521f78700 1 -- 172.21.15.163:0/546712442 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f851c159890 msgr2=0x7f851c0c9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:32.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.991+0000 7f8521f78700 1 --2- 172.21.15.163:0/546712442 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f851c159890 0x7f851c0c9510 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f850c00cb10 tx=0x7f850c0017b0 comp rx=0 tx=0).stop 2023-10-09T06:37:32.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.991+0000 7f8521f78700 1 -- 172.21.15.163:0/546712442 shutdown_connections 2023-10-09T06:37:32.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.991+0000 7f8521f78700 1 --2- 172.21.15.163:0/546712442 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8508067320 0x7f85080697d0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:32.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.991+0000 7f8521f78700 1 --2- 172.21.15.163:0/546712442 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f851c159890 0x7f851c0c9510 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:32.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.991+0000 7f8521f78700 1 --2- 172.21.15.163:0/546712442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f851c15db10 0x7f851c0cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:32.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.991+0000 7f8521f78700 1 --2- 172.21.15.163:0/546712442 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f851c159c60 0x7f851c140b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:32.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.991+0000 7f8521f78700 1 -- 172.21.15.163:0/546712442 >> 172.21.15.163:0/546712442 conn(0x7f851c068e20 msgr2=0x7f851c1530f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:32.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.991+0000 7f8521f78700 1 -- 172.21.15.163:0/546712442 shutdown_connections 2023-10-09T06:37:32.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:32.991+0000 7f8521f78700 1 -- 172.21.15.163:0/546712442 wait complete. 2023-10-09T06:37:33.005 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-09T06:37:06.472811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-10-09T06:37:07.096712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-09T06:37:33.005 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:37:33.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.121+0000 7f53c1103700 1 Processor -- start 2023-10-09T06:37:33.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.121+0000 7f53c1103700 1 -- start start 2023-10-09T06:37:33.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.122+0000 7f53c1103700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f53bc159c60 0x7f53bc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:33.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.122+0000 7f53c1103700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53bc15db10 0x7f53bc069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:33.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.122+0000 7f53c1103700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53bc15a160 con 0x7f53bc159890 2023-10-09T06:37:33.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.122+0000 7f53c1103700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53bc15df50 con 0x7f53bc159c60 2023-10-09T06:37:33.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.122+0000 7f53c1103700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc0d13a0 con 0x7f53bc15db10 2023-10-09T06:37:33.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.122+0000 7f53b3fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f53bc159c60 0x7f53bc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:33.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.122+0000 7f53b3fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f53bc159c60 0x7f53bc15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49288/0 (socket says 172.21.15.163:49288) 2023-10-09T06:37:33.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.122+0000 7f53b3fff700 1 -- 172.21.15.163:0/4275695604 learned_addr learned my addr 172.21.15.163:0/4275695604 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:33.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.122+0000 7f53bb59e700 1 --2- 172.21.15.163:0/4275695604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53bc15db10 0x7f53bc069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:33.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.123+0000 7f53b3fff700 1 -- 172.21.15.163:0/4275695604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53bc15db10 msgr2=0x7f53bc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:33.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.123+0000 7f53b3fff700 1 --2- 172.21.15.163:0/4275695604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53bc15db10 0x7f53bc069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:33.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.123+0000 7f53b3fff700 1 -- 172.21.15.163:0/4275695604 >> v1:172.21.15.181:6789/0 conn(0x7f53bc159890 legacy=0x7f53bc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:33.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.123+0000 7f53b3fff700 1 -- 172.21.15.163:0/4275695604 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53bc15e140 con 0x7f53bc159c60 2023-10-09T06:37:33.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.123+0000 7f53ba59c700 1 -- 172.21.15.163:0/4275695604 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 863981690 0 0) 0x7f53bc15a160 con 0x7f53bc159890 2023-10-09T06:37:33.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.123+0000 7f53b3fff700 1 --2- 172.21.15.163:0/4275695604 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53bc159c60 0x7f53bc15d5d0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f53a40099b0 tx=0x7f53a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf94f97e9cf9e89 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:33.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.123+0000 7f53ba59c700 1 -- 172.21.15.163:0/4275695604 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a4009ea0 con 0x7f53bc159c60 2023-10-09T06:37:33.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.123+0000 7f53ba59c700 1 -- 172.21.15.163:0/4275695604 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a400b860 con 0x7f53bc159c60 2023-10-09T06:37:33.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.123+0000 7f53ba59c700 1 -- 172.21.15.163:0/4275695604 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a400bc00 con 0x7f53bc159c60 2023-10-09T06:37:33.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.123+0000 7f53c1103700 1 -- 172.21.15.163:0/4275695604 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53bc159c60 msgr2=0x7f53bc15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:33.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.123+0000 7f53c1103700 1 --2- 172.21.15.163:0/4275695604 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53bc159c60 0x7f53bc15d5d0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f53a40099b0 tx=0x7f53a4009250 comp rx=0 tx=0).stop 2023-10-09T06:37:33.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.123+0000 7f53c1103700 1 -- 172.21.15.163:0/4275695604 shutdown_connections 2023-10-09T06:37:33.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.123+0000 7f53c1103700 1 --2- 172.21.15.163:0/4275695604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53bc15db10 0x7f53bc069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:33.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.123+0000 7f53c1103700 1 --2- 172.21.15.163:0/4275695604 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53bc159c60 0x7f53bc15d5d0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:33.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.123+0000 7f53c1103700 1 -- 172.21.15.163:0/4275695604 >> 172.21.15.163:0/4275695604 conn(0x7f53bc068e20 msgr2=0x7f53bc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:33.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.123+0000 7f53c1103700 1 -- 172.21.15.163:0/4275695604 shutdown_connections 2023-10-09T06:37:33.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.123+0000 7f53c1103700 1 -- 172.21.15.163:0/4275695604 wait complete. 2023-10-09T06:37:33.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.124+0000 7f53c1103700 1 Processor -- start 2023-10-09T06:37:33.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.125+0000 7f53c1103700 1 -- start start 2023-10-09T06:37:33.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.125+0000 7f53c1103700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53bc159890 0x7f53bc0b0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:33.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.125+0000 7f53c1103700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53bc159c60 0x7f53bc150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:33.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.125+0000 7f53c1103700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53bc15db10 0x7f53bc154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:33.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.125+0000 7f53c1103700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc0b0a80 con 0x7f53bc159c60 2023-10-09T06:37:33.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.125+0000 7f53c1103700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc154590 con 0x7f53bc15db10 2023-10-09T06:37:33.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.125+0000 7f53c1103700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc153980 con 0x7f53bc159890 2023-10-09T06:37:33.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.125+0000 7f53b3fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53bc159c60 0x7f53bc150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:33.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.125+0000 7f53b3fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53bc159c60 0x7f53bc150f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49296/0 (socket says 172.21.15.163:49296) 2023-10-09T06:37:33.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.125+0000 7f53b3fff700 1 -- 172.21.15.163:0/813084652 learned_addr learned my addr 172.21.15.163:0/813084652 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:33.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.125+0000 7f53bb59e700 1 --2- 172.21.15.163:0/813084652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53bc15db10 0x7f53bc154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:33.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.125+0000 7f53bad9d700 1 --2- 172.21.15.163:0/813084652 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53bc159890 0x7f53bc0b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:33.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.126+0000 7f53b3fff700 1 -- 172.21.15.163:0/813084652 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53bc159890 msgr2=0x7f53bc0b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:33.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.126+0000 7f53b3fff700 1 --2- 172.21.15.163:0/813084652 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53bc159890 0x7f53bc0b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:33.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.126+0000 7f53b3fff700 1 -- 172.21.15.163:0/813084652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53bc15db10 msgr2=0x7f53bc154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:33.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.126+0000 7f53b3fff700 1 --2- 172.21.15.163:0/813084652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53bc15db10 0x7f53bc154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:33.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.126+0000 7f53b3fff700 1 -- 172.21.15.163:0/813084652 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a40094f0 con 0x7f53bc159c60 2023-10-09T06:37:33.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.126+0000 7f53bad9d700 1 --2- 172.21.15.163:0/813084652 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53bc159890 0x7f53bc0b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:33.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.126+0000 7f53b3fff700 1 --2- 172.21.15.163:0/813084652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53bc159c60 0x7f53bc150f30 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f53bc0899c0 tx=0x7f53a4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:33.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.126+0000 7f53b2ffd700 1 -- 172.21.15.163:0/813084652 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a400bbe0 con 0x7f53bc159c60 2023-10-09T06:37:33.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.126+0000 7f53b2ffd700 1 -- 172.21.15.163:0/813084652 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a4011420 con 0x7f53bc159c60 2023-10-09T06:37:33.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.126+0000 7f53c1103700 1 -- 172.21.15.163:0/813084652 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53bc153c60 con 0x7f53bc159c60 2023-10-09T06:37:33.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.126+0000 7f53b2ffd700 1 -- 172.21.15.163:0/813084652 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a40117e0 con 0x7f53bc159c60 2023-10-09T06:37:33.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.126+0000 7f53c1103700 1 -- 172.21.15.163:0/813084652 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53bc154220 con 0x7f53bc159c60 2023-10-09T06:37:33.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.127+0000 7f53c1103700 1 -- 172.21.15.163:0/813084652 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53bc154720 con 0x7f53bc159c60 2023-10-09T06:37:33.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.127+0000 7f53b2ffd700 1 -- 172.21.15.163:0/813084652 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f53a4011bd0 con 0x7f53bc159c60 2023-10-09T06:37:33.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.128+0000 7f53b2ffd700 1 --2- 172.21.15.163:0/813084652 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f539c067320 0x7f539c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:33.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.128+0000 7f53b2ffd700 1 -- 172.21.15.163:0/813084652 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5877+0+0 (secure 0 0 0) 0x7f53a4086660 con 0x7f53bc159c60 2023-10-09T06:37:33.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.128+0000 7f53bad9d700 1 --2- 172.21.15.163:0/813084652 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f539c067320 0x7f539c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:33.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.129+0000 7f53bad9d700 1 --2- 172.21.15.163:0/813084652 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f539c067320 0x7f539c0697d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0011e0 tx=0x7f53ac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:33.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.132+0000 7f53b2ffd700 1 -- 172.21.15.163:0/813084652 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f53a4053240 con 0x7f53bc159c60 2023-10-09T06:37:33.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.313+0000 7f53c1103700 1 -- 172.21.15.163:0/813084652 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f53bc04c3c0 con 0x7f53bc159c60 2023-10-09T06:37:33.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.313+0000 7f53b2ffd700 1 -- 172.21.15.163:0/813084652 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) v1 ==== 93+0+10 (secure 0 0 0) 0x7f53a4056860 con 0x7f53bc159c60 2023-10-09T06:37:33.315 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:37:33.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.315+0000 7f53c1103700 1 -- 172.21.15.163:0/813084652 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f539c067320 msgr2=0x7f539c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:33.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.315+0000 7f53c1103700 1 --2- 172.21.15.163:0/813084652 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f539c067320 0x7f539c0697d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0011e0 tx=0x7f53ac009040 comp rx=0 tx=0).stop 2023-10-09T06:37:33.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.315+0000 7f53c1103700 1 -- 172.21.15.163:0/813084652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53bc159c60 msgr2=0x7f53bc150f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:33.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.315+0000 7f53c1103700 1 --2- 172.21.15.163:0/813084652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53bc159c60 0x7f53bc150f30 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f53bc0899c0 tx=0x7f53a4017000 comp rx=0 tx=0).stop 2023-10-09T06:37:33.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.315+0000 7f53c1103700 1 -- 172.21.15.163:0/813084652 shutdown_connections 2023-10-09T06:37:33.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.315+0000 7f53c1103700 1 --2- 172.21.15.163:0/813084652 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f539c067320 0x7f539c0697d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:33.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.315+0000 7f53c1103700 1 --2- 172.21.15.163:0/813084652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53bc15db10 0x7f53bc154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:33.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.315+0000 7f53c1103700 1 --2- 172.21.15.163:0/813084652 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53bc159890 0x7f53bc0b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:33.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.315+0000 7f53c1103700 1 --2- 172.21.15.163:0/813084652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53bc159c60 0x7f53bc150f30 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:33.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.316+0000 7f53c1103700 1 -- 172.21.15.163:0/813084652 >> 172.21.15.163:0/813084652 conn(0x7f53bc068e20 msgr2=0x7f53bc152da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:33.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.316+0000 7f53c1103700 1 -- 172.21.15.163:0/813084652 shutdown_connections 2023-10-09T06:37:33.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.316+0000 7f53c1103700 1 -- 172.21.15.163:0/813084652 wait complete. 2023-10-09T06:37:33.328 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-09T06:37:33.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.445+0000 7fee64a25700 1 Processor -- start 2023-10-09T06:37:33.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.445+0000 7fee64a25700 1 -- start start 2023-10-09T06:37:33.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.445+0000 7fee64a25700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fee60159890 0x7fee6015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:33.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.445+0000 7fee64a25700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee60159c60 0x7fee6015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:33.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.446+0000 7fee64a25700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee6015a160 con 0x7fee6015db10 2023-10-09T06:37:33.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.446+0000 7fee64a25700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee600d13a0 con 0x7fee60159890 2023-10-09T06:37:33.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.446+0000 7fee64a25700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6015e140 con 0x7fee60159c60 2023-10-09T06:37:33.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.446+0000 7fee5e59c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fee60159890 0x7fee6015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:33.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.446+0000 7fee5e59c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fee60159890 0x7fee6015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49312/0 (socket says 172.21.15.163:49312) 2023-10-09T06:37:33.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.446+0000 7fee5e59c700 1 -- 172.21.15.163:0/2097403745 learned_addr learned my addr 172.21.15.163:0/2097403745 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:33.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.446+0000 7fee55d9b700 1 --2- 172.21.15.163:0/2097403745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee60159c60 0x7fee6015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:33.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.446+0000 7fee5e59c700 1 -- 172.21.15.163:0/2097403745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee60159c60 msgr2=0x7fee6015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:33.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.446+0000 7fee5e59c700 1 --2- 172.21.15.163:0/2097403745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee60159c60 0x7fee6015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:33.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.446+0000 7fee5e59c700 1 -- 172.21.15.163:0/2097403745 >> v1:172.21.15.181:6789/0 conn(0x7fee6015db10 legacy=0x7fee60069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:33.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.446+0000 7fee5e59c700 1 -- 172.21.15.163:0/2097403745 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6015a3d0 con 0x7fee60159890 2023-10-09T06:37:33.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.446+0000 7fee5e59c700 1 --2- 172.21.15.163:0/2097403745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee60159890 0x7fee6015abd0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fee500099b0 tx=0x7fee50009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae2d3db37fba1b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:33.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.446+0000 7fee5dd9b700 1 -- 172.21.15.163:0/2097403745 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5000b860 con 0x7fee60159890 2023-10-09T06:37:33.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.446+0000 7fee5dd9b700 1 -- 172.21.15.163:0/2097403745 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5000b9c0 con 0x7fee60159890 2023-10-09T06:37:33.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.446+0000 7fee5dd9b700 1 -- 172.21.15.163:0/2097403745 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5000bd60 con 0x7fee60159890 2023-10-09T06:37:33.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.447+0000 7fee64a25700 1 -- 172.21.15.163:0/2097403745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee60159890 msgr2=0x7fee6015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:33.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.447+0000 7fee64a25700 1 --2- 172.21.15.163:0/2097403745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee60159890 0x7fee6015abd0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fee500099b0 tx=0x7fee50009250 comp rx=0 tx=0).stop 2023-10-09T06:37:33.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.447+0000 7fee64a25700 1 -- 172.21.15.163:0/2097403745 shutdown_connections 2023-10-09T06:37:33.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.447+0000 7fee64a25700 1 --2- 172.21.15.163:0/2097403745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee60159890 0x7fee6015abd0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:33.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.447+0000 7fee64a25700 1 --2- 172.21.15.163:0/2097403745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee60159c60 0x7fee6015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:33.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.447+0000 7fee64a25700 1 -- 172.21.15.163:0/2097403745 >> 172.21.15.163:0/2097403745 conn(0x7fee60068e20 msgr2=0x7fee600d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:33.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.447+0000 7fee64a25700 1 -- 172.21.15.163:0/2097403745 shutdown_connections 2023-10-09T06:37:33.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.447+0000 7fee64a25700 1 -- 172.21.15.163:0/2097403745 wait complete. 2023-10-09T06:37:33.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.448+0000 7fee64a25700 1 Processor -- start 2023-10-09T06:37:33.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.448+0000 7fee64a25700 1 -- start start 2023-10-09T06:37:33.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.449+0000 7fee64a25700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee60159890 0x7fee6014c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:33.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.449+0000 7fee64a25700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee60159c60 0x7fee6014f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:33.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.449+0000 7fee64a25700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee6015db10 0x7fee600b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:33.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.449+0000 7fee64a25700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee600b39a0 con 0x7fee6015db10 2023-10-09T06:37:33.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.449+0000 7fee64a25700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee600ce340 con 0x7fee60159c60 2023-10-09T06:37:33.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.449+0000 7fee64a25700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee600b5d90 con 0x7fee60159890 2023-10-09T06:37:33.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.449+0000 7fee5ed9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee6015db10 0x7fee600b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:33.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.449+0000 7fee5ed9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee6015db10 0x7fee600b5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49324/0 (socket says 172.21.15.163:49324) 2023-10-09T06:37:33.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.449+0000 7fee5ed9d700 1 -- 172.21.15.163:0/2646166225 learned_addr learned my addr 172.21.15.163:0/2646166225 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:33.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.449+0000 7fee5e59c700 1 --2- 172.21.15.163:0/2646166225 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee60159890 0x7fee6014c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:33.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.449+0000 7fee55d9b700 1 --2- 172.21.15.163:0/2646166225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee60159c60 0x7fee6014f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:33.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.449+0000 7fee5ed9d700 1 -- 172.21.15.163:0/2646166225 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee60159890 msgr2=0x7fee6014c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:33.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.449+0000 7fee5ed9d700 1 --2- 172.21.15.163:0/2646166225 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee60159890 0x7fee6014c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:33.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.449+0000 7fee5ed9d700 1 -- 172.21.15.163:0/2646166225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee60159c60 msgr2=0x7fee6014f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:33.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.449+0000 7fee5ed9d700 1 --2- 172.21.15.163:0/2646166225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee60159c60 0x7fee6014f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:33.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.449+0000 7fee5ed9d700 1 -- 172.21.15.163:0/2646166225 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee500094f0 con 0x7fee6015db10 2023-10-09T06:37:33.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.449+0000 7fee5ed9d700 1 --2- 172.21.15.163:0/2646166225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee6015db10 0x7fee600b5220 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fee5800c4c0 tx=0x7fee58007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:33.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.449+0000 7fee56ffd700 1 -- 172.21.15.163:0/2646166225 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5800ce70 con 0x7fee6015db10 2023-10-09T06:37:33.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.450+0000 7fee56ffd700 1 -- 172.21.15.163:0/2646166225 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee58011590 con 0x7fee6015db10 2023-10-09T06:37:33.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.450+0000 7fee64a25700 1 -- 172.21.15.163:0/2646166225 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee600b6070 con 0x7fee6015db10 2023-10-09T06:37:33.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.450+0000 7fee56ffd700 1 -- 172.21.15.163:0/2646166225 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee58011970 con 0x7fee6015db10 2023-10-09T06:37:33.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.450+0000 7fee64a25700 1 -- 172.21.15.163:0/2646166225 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee600b3630 con 0x7fee6015db10 2023-10-09T06:37:33.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.450+0000 7fee64a25700 1 -- 172.21.15.163:0/2646166225 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee600b3b50 con 0x7fee6015db10 2023-10-09T06:37:33.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.451+0000 7fee56ffd700 1 -- 172.21.15.163:0/2646166225 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fee58011ce0 con 0x7fee6015db10 2023-10-09T06:37:33.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.451+0000 7fee56ffd700 1 --2- 172.21.15.163:0/2646166225 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fee40067320 0x7fee400697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:33.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.451+0000 7fee56ffd700 1 -- 172.21.15.163:0/2646166225 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5877+0+0 (secure 0 0 0) 0x7fee580840b0 con 0x7fee6015db10 2023-10-09T06:37:33.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.452+0000 7fee5e59c700 1 --2- 172.21.15.163:0/2646166225 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fee40067320 0x7fee400697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:33.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.452+0000 7fee5e59c700 1 --2- 172.21.15.163:0/2646166225 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fee40067320 0x7fee400697d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fee500099b0 tx=0x7fee50017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:33.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.456+0000 7fee56ffd700 1 -- 172.21.15.163:0/2646166225 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee580b55a0 con 0x7fee6015db10 2023-10-09T06:37:33.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.638+0000 7fee64a25700 1 -- 172.21.15.163:0/2646166225 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fee6004c3c0 con 0x7fee6015db10 2023-10-09T06:37:33.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.638+0000 7fee56ffd700 1 -- 172.21.15.163:0/2646166225 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v125) v1 ==== 104+0+11 (secure 0 0 0) 0x7fee580170a0 con 0x7fee6015db10 2023-10-09T06:37:33.640 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 32 2023-10-09T06:37:33.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.640+0000 7fee64a25700 1 -- 172.21.15.163:0/2646166225 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fee40067320 msgr2=0x7fee400697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:33.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.640+0000 7fee64a25700 1 --2- 172.21.15.163:0/2646166225 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fee40067320 0x7fee400697d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fee500099b0 tx=0x7fee50017000 comp rx=0 tx=0).stop 2023-10-09T06:37:33.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.640+0000 7fee64a25700 1 -- 172.21.15.163:0/2646166225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee6015db10 msgr2=0x7fee600b5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:33.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.640+0000 7fee64a25700 1 --2- 172.21.15.163:0/2646166225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee6015db10 0x7fee600b5220 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fee5800c4c0 tx=0x7fee58007f50 comp rx=0 tx=0).stop 2023-10-09T06:37:33.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.640+0000 7fee64a25700 1 -- 172.21.15.163:0/2646166225 shutdown_connections 2023-10-09T06:37:33.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.640+0000 7fee64a25700 1 --2- 172.21.15.163:0/2646166225 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fee40067320 0x7fee400697d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:33.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.640+0000 7fee64a25700 1 --2- 172.21.15.163:0/2646166225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee6015db10 0x7fee600b5220 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:33.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.640+0000 7fee64a25700 1 --2- 172.21.15.163:0/2646166225 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee60159890 0x7fee6014c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:33.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.640+0000 7fee64a25700 1 --2- 172.21.15.163:0/2646166225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee60159c60 0x7fee6014f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:33.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.640+0000 7fee64a25700 1 -- 172.21.15.163:0/2646166225 >> 172.21.15.163:0/2646166225 conn(0x7fee60068e20 msgr2=0x7fee600b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:33.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.640+0000 7fee64a25700 1 -- 172.21.15.163:0/2646166225 shutdown_connections 2023-10-09T06:37:33.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.640+0000 7fee64a25700 1 -- 172.21.15.163:0/2646166225 wait complete. 2023-10-09T06:37:33.653 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-09T06:37:33.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.771+0000 7f3f115f6700 1 Processor -- start 2023-10-09T06:37:33.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.771+0000 7f3f115f6700 1 -- start start 2023-10-09T06:37:33.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.772+0000 7f3f115f6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f0c0b16d0 0x7f3f0c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:33.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.772+0000 7f3f115f6700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0c0b2080 con 0x7f3f0c0b33b0 2023-10-09T06:37:33.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.772+0000 7f3f115f6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0c0cfb00 con 0x7f3f0c0b4e70 2023-10-09T06:37:33.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.772+0000 7f3f115f6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c0b5c40 con 0x7f3f0c0b16d0 2023-10-09T06:37:33.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.772+0000 7f3f10df5700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f3f0c0b33b0 0x7f3f0c0b3830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:52728/0 (socket says 172.21.15.163:52728) 2023-10-09T06:37:33.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.772+0000 7f3f10df5700 1 -- 172.21.15.163:0/2398736818 learned_addr learned my addr 172.21.15.163:0/2398736818 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:33.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.772+0000 7f3f0b7fe700 1 --2- 172.21.15.163:0/2398736818 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f0c0b16d0 0x7f3f0c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:33.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.772+0000 7f3f0affd700 1 -- 172.21.15.163:0/2398736818 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1524723400 0 0) 0x7f3f0c0b2080 con 0x7f3f0c0b33b0 2023-10-09T06:37:33.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.772+0000 7f3f0affd700 1 -- 172.21.15.163:0/2398736818 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ef4003570 con 0x7f3f0c0b33b0 2023-10-09T06:37:33.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.773+0000 7f3f0affd700 1 -- 172.21.15.163:0/2398736818 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1844845153 0 0) 0x7f3f0c0cfb00 con 0x7f3f0c0b4e70 2023-10-09T06:37:33.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.773+0000 7f3f0affd700 1 -- 172.21.15.163:0/2398736818 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f0c0b2080 con 0x7f3f0c0b4e70 2023-10-09T06:37:33.781 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.773+0000 7f3f0affd700 1 -- 172.21.15.163:0/2398736818 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 462710091 0 0) 0x7f3ef4003570 con 0x7f3f0c0b33b0 2023-10-09T06:37:33.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.773+0000 7f3f0affd700 1 -- 172.21.15.163:0/2398736818 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f0c0b16d0 msgr2=0x7f3f0c0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:33.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.773+0000 7f3f0affd700 1 --2- 172.21.15.163:0/2398736818 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f0c0b16d0 0x7f3f0c0b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:33.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.773+0000 7f3f0affd700 1 -- 172.21.15.163:0/2398736818 >> v1:172.21.15.181:6789/0 conn(0x7f3f0c0b4e70 legacy=0x7f3f0c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:33.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.773+0000 7f3f0affd700 1 -- 172.21.15.163:0/2398736818 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c0b5f20 con 0x7f3f0c0b33b0 2023-10-09T06:37:33.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.773+0000 7f3f0affd700 1 -- 172.21.15.163:0/2398736818 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f3f040017f0 con 0x7f3f0c0b33b0 2023-10-09T06:37:33.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.773+0000 7f3f0b7fe700 1 --2- 172.21.15.163:0/2398736818 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f0c0b16d0 0x7f3f0c0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:37:33.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.773+0000 7f3f0affd700 1 -- 172.21.15.163:0/2398736818 >> v1:172.21.15.163:6789/0 conn(0x7f3f0c0b33b0 legacy=0x7f3f0c0b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:33.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.773+0000 7f3f0affd700 1 --2- 172.21.15.163:0/2398736818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ef4003c20 0x7f3ef4004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:33.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.773+0000 7f3f0affd700 1 --2- 172.21.15.163:0/2398736818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ef4004810 0x7f3ef4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:33.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.773+0000 7f3f0affd700 1 --2- 172.21.15.163:0/2398736818 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f0c0b16d0 0x7f3ef4005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:33.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.773+0000 7f3f0affd700 1 -- 172.21.15.163:0/2398736818 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c0b5c40 con 0x7f3ef4003c20 2023-10-09T06:37:33.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.773+0000 7f3f0affd700 1 -- 172.21.15.163:0/2398736818 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef400bbf0 con 0x7f3ef4004810 2023-10-09T06:37:33.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.773+0000 7f3f0affd700 1 -- 172.21.15.163:0/2398736818 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef400bd80 con 0x7f3f0c0b16d0 2023-10-09T06:37:33.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.773+0000 7f3f0bfff700 1 --2- 172.21.15.163:0/2398736818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ef4003c20 0x7f3ef4004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:33.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.773+0000 7f3f10df5700 1 --2- 172.21.15.163:0/2398736818 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f0c0b16d0 0x7f3ef4005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:33.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.773+0000 7f3f0b7fe700 1 --2- 172.21.15.163:0/2398736818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ef4004810 0x7f3ef4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:33.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.773+0000 7f3f0bfff700 1 -- 172.21.15.163:0/2398736818 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f0c0b16d0 msgr2=0x7f3ef4005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:33.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.773+0000 7f3f0bfff700 1 --2- 172.21.15.163:0/2398736818 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f0c0b16d0 0x7f3ef4005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:33.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.773+0000 7f3f0bfff700 1 -- 172.21.15.163:0/2398736818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ef4004810 msgr2=0x7f3ef4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:33.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.773+0000 7f3f0bfff700 1 --2- 172.21.15.163:0/2398736818 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ef4004810 0x7f3ef4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:33.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.773+0000 7f3f0bfff700 1 -- 172.21.15.163:0/2398736818 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f00009420 con 0x7f3ef4003c20 2023-10-09T06:37:33.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.773+0000 7f3f0bfff700 1 --2- 172.21.15.163:0/2398736818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ef4003c20 0x7f3ef4004010 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f3efc001a10 tx=0x7f3efc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2061c33165cc31 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:33.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.773+0000 7f3f0affd700 1 -- 172.21.15.163:0/2398736818 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc00c590 con 0x7f3ef4003c20 2023-10-09T06:37:33.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.773+0000 7f3f0affd700 1 -- 172.21.15.163:0/2398736818 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3efc00c6f0 con 0x7f3ef4003c20 2023-10-09T06:37:33.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.773+0000 7f3f0affd700 1 -- 172.21.15.163:0/2398736818 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc00ca90 con 0x7f3ef4003c20 2023-10-09T06:37:33.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.774+0000 7f3f115f6700 1 -- 172.21.15.163:0/2398736818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ef4003c20 msgr2=0x7f3ef4004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:33.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.774+0000 7f3f115f6700 1 --2- 172.21.15.163:0/2398736818 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ef4003c20 0x7f3ef4004010 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f3efc001a10 tx=0x7f3efc007d30 comp rx=0 tx=0).stop 2023-10-09T06:37:33.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.774+0000 7f3f10df5700 1 -- 172.21.15.163:0/2398736818 reap_dead start 2023-10-09T06:37:33.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.774+0000 7f3f115f6700 1 -- 172.21.15.163:0/2398736818 shutdown_connections 2023-10-09T06:37:33.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.774+0000 7f3f115f6700 1 -- 172.21.15.163:0/2398736818 >> 172.21.15.163:0/2398736818 conn(0x7f3f0c0d10c0 msgr2=0x7f3f0c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:33.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.774+0000 7f3f115f6700 1 -- 172.21.15.163:0/2398736818 shutdown_connections 2023-10-09T06:37:33.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.774+0000 7f3f115f6700 1 -- 172.21.15.163:0/2398736818 wait complete. 2023-10-09T06:37:33.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.775+0000 7f3f115f6700 1 Processor -- start 2023-10-09T06:37:33.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.775+0000 7f3f115f6700 1 -- start start 2023-10-09T06:37:33.784 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.776+0000 7f3f115f6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ef4003c20 0x7f3f0c14c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:33.785 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.776+0000 7f3f115f6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f0c15a300 0x7f3f0c169a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:33.785 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.776+0000 7f3f115f6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f0c0c6f50 0x7f3f0c0c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:33.785 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.776+0000 7f3f115f6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c158730 con 0x7f3f0c15a300 2023-10-09T06:37:33.785 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.776+0000 7f3f115f6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c16aa10 con 0x7f3f0c0c6f50 2023-10-09T06:37:33.785 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.776+0000 7f3f115f6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c0c7aa0 con 0x7f3ef4003c20 2023-10-09T06:37:33.785 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.776+0000 7f3f0b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f0c15a300 0x7f3f0c169a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:33.785 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.776+0000 7f3f0b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f0c15a300 0x7f3f0c169a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49344/0 (socket says 172.21.15.163:49344) 2023-10-09T06:37:33.785 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.776+0000 7f3f0b7fe700 1 -- 172.21.15.163:0/3996969179 learned_addr learned my addr 172.21.15.163:0/3996969179 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:33.785 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.776+0000 7f3f0b7fe700 1 -- 172.21.15.163:0/3996969179 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ef4003c20 msgr2=0x7f3f0c14c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:33.785 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.776+0000 7f3f0b7fe700 1 --2- 172.21.15.163:0/3996969179 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ef4003c20 0x7f3f0c14c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:33.785 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.776+0000 7f3f0b7fe700 1 -- 172.21.15.163:0/3996969179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f0c0c6f50 msgr2=0x7f3f0c0c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:33.785 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.776+0000 7f3f0b7fe700 1 --2- 172.21.15.163:0/3996969179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f0c0c6f50 0x7f3f0c0c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:33.786 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.776+0000 7f3f0b7fe700 1 -- 172.21.15.163:0/3996969179 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c16a170 con 0x7f3f0c15a300 2023-10-09T06:37:33.786 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.777+0000 7f3f0b7fe700 1 --2- 172.21.15.163:0/3996969179 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f0c15a300 0x7f3f0c169a70 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f3f00011da0 tx=0x7f3f000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:33.786 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.777+0000 7f3ef3fff700 1 -- 172.21.15.163:0/3996969179 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0000fc70 con 0x7f3f0c15a300 2023-10-09T06:37:33.786 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.777+0000 7f3ef3fff700 1 -- 172.21.15.163:0/3996969179 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0000fdd0 con 0x7f3f0c15a300 2023-10-09T06:37:33.786 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.777+0000 7f3f115f6700 1 -- 172.21.15.163:0/3996969179 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0c16a450 con 0x7f3f0c15a300 2023-10-09T06:37:33.786 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.777+0000 7f3ef3fff700 1 -- 172.21.15.163:0/3996969179 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f00014690 con 0x7f3f0c15a300 2023-10-09T06:37:33.786 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.777+0000 7f3f115f6700 1 -- 172.21.15.163:0/3996969179 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0c0c9e20 con 0x7f3f0c15a300 2023-10-09T06:37:33.786 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.778+0000 7f3f115f6700 1 -- 172.21.15.163:0/3996969179 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f0c088990 con 0x7f3f0c15a300 2023-10-09T06:37:33.786 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.778+0000 7f3ef3fff700 1 -- 172.21.15.163:0/3996969179 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3f00014a80 con 0x7f3f0c15a300 2023-10-09T06:37:33.786 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.778+0000 7f3ef3fff700 1 --2- 172.21.15.163:0/3996969179 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3ef8067320 0x7f3ef80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:33.787 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.779+0000 7f3ef3fff700 1 -- 172.21.15.163:0/3996969179 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5877+0+0 (secure 0 0 0) 0x7f3f00018030 con 0x7f3f0c15a300 2023-10-09T06:37:33.787 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.779+0000 7f3f0bfff700 1 --2- 172.21.15.163:0/3996969179 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3ef8067320 0x7f3ef80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:33.787 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.779+0000 7f3f0bfff700 1 --2- 172.21.15.163:0/3996969179 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3ef8067320 0x7f3ef80697d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c0b1610 tx=0x7f3efc015000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:33.787 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.783+0000 7f3ef3fff700 1 -- 172.21.15.163:0/3996969179 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3f00052570 con 0x7f3f0c15a300 2023-10-09T06:37:33.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.964+0000 7f3f115f6700 1 -- 172.21.15.163:0/3996969179 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3f0c0c7be0 con 0x7f3f0c15a300 2023-10-09T06:37:33.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.964+0000 7f3ef3fff700 1 -- 172.21.15.163:0/3996969179 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v125) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3f000220a0 con 0x7f3f0c15a300 2023-10-09T06:37:33.967 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 64 2023-10-09T06:37:33.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.967+0000 7f3f115f6700 1 -- 172.21.15.163:0/3996969179 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3ef8067320 msgr2=0x7f3ef80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:33.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.967+0000 7f3f115f6700 1 --2- 172.21.15.163:0/3996969179 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3ef8067320 0x7f3ef80697d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c0b1610 tx=0x7f3efc015000 comp rx=0 tx=0).stop 2023-10-09T06:37:33.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.967+0000 7f3f115f6700 1 -- 172.21.15.163:0/3996969179 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f0c15a300 msgr2=0x7f3f0c169a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:33.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.967+0000 7f3f115f6700 1 --2- 172.21.15.163:0/3996969179 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f0c15a300 0x7f3f0c169a70 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f3f00011da0 tx=0x7f3f000091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:33.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.967+0000 7f3f115f6700 1 -- 172.21.15.163:0/3996969179 shutdown_connections 2023-10-09T06:37:33.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.967+0000 7f3f115f6700 1 --2- 172.21.15.163:0/3996969179 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3ef8067320 0x7f3ef80697d0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:33.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.967+0000 7f3f115f6700 1 --2- 172.21.15.163:0/3996969179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f0c0c6f50 0x7f3f0c0c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:33.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.967+0000 7f3f115f6700 1 --2- 172.21.15.163:0/3996969179 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ef4003c20 0x7f3f0c14c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:33.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.967+0000 7f3f115f6700 1 --2- 172.21.15.163:0/3996969179 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f0c15a300 0x7f3f0c169a70 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:33.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.967+0000 7f3f115f6700 1 -- 172.21.15.163:0/3996969179 >> 172.21.15.163:0/3996969179 conn(0x7f3f0c0d10c0 msgr2=0x7f3f0c158ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:33.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.967+0000 7f3f115f6700 1 -- 172.21.15.163:0/3996969179 shutdown_connections 2023-10-09T06:37:33.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:33.967+0000 7f3f115f6700 1 -- 172.21.15.163:0/3996969179 wait complete. 2023-10-09T06:37:33.980 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:37:34.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.098+0000 7fad328b3700 1 Processor -- start 2023-10-09T06:37:34.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.098+0000 7fad328b3700 1 -- start start 2023-10-09T06:37:34.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.099+0000 7fad328b3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fad2c159c10 0x7fad2c0694a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:34.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.099+0000 7fad328b3700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fad2c0699e0 0x7fad2c0b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:34.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.099+0000 7fad328b3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad2c069e40 con 0x7fad2c159840 2023-10-09T06:37:34.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.099+0000 7fad328b3700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad2c0cfb30 con 0x7fad2c0699e0 2023-10-09T06:37:34.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.099+0000 7fad328b3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad2c0cbc40 con 0x7fad2c159c10 2023-10-09T06:37:34.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.099+0000 7fad30e50700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fad2c0699e0 0x7fad2c0b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:34.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.099+0000 7fad30e50700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fad2c0699e0 0x7fad2c0b05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49346/0 (socket says 172.21.15.163:49346) 2023-10-09T06:37:34.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.099+0000 7fad30e50700 1 -- 172.21.15.163:0/528802502 learned_addr learned my addr 172.21.15.163:0/528802502 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:34.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.099+0000 7fad2b7fe700 1 --2- 172.21.15.163:0/528802502 >> v2:172.21.15.181:3300/0 conn(0x7fad2c159c10 0x7fad2c0694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:34.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.099+0000 7fad30e50700 1 -- 172.21.15.163:0/528802502 >> v2:172.21.15.181:3300/0 conn(0x7fad2c159c10 msgr2=0x7fad2c0694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:34.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.099+0000 7fad30e50700 1 --2- 172.21.15.163:0/528802502 >> v2:172.21.15.181:3300/0 conn(0x7fad2c159c10 0x7fad2c0694a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:34.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.099+0000 7fad30e50700 1 -- 172.21.15.163:0/528802502 >> v1:172.21.15.181:6789/0 conn(0x7fad2c159840 legacy=0x7fad2c0cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:34.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.099+0000 7fad30e50700 1 -- 172.21.15.163:0/528802502 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2c0cbf10 con 0x7fad2c0699e0 2023-10-09T06:37:34.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.099+0000 7fad2b7fe700 1 --2- 172.21.15.163:0/528802502 >> v2:172.21.15.181:3300/0 conn(0x7fad2c159c10 0x7fad2c0694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:37:34.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.099+0000 7fad30e50700 1 --2- 172.21.15.163:0/528802502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad2c0699e0 0x7fad2c0b05e0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fad2c0cad30 tx=0x7fad24009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64dddaf32f2d52f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:34.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.099+0000 7fad2affd700 1 -- 172.21.15.163:0/528802502 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2400e070 con 0x7fad2c0699e0 2023-10-09T06:37:34.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.099+0000 7fad2affd700 1 -- 172.21.15.163:0/528802502 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2400ba10 con 0x7fad2c0699e0 2023-10-09T06:37:34.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.099+0000 7fad2affd700 1 -- 172.21.15.163:0/528802502 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad240092e0 con 0x7fad2c0699e0 2023-10-09T06:37:34.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.100+0000 7fad328b3700 1 -- 172.21.15.163:0/528802502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad2c0699e0 msgr2=0x7fad2c0b05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:34.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.100+0000 7fad328b3700 1 --2- 172.21.15.163:0/528802502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad2c0699e0 0x7fad2c0b05e0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fad2c0cad30 tx=0x7fad24009840 comp rx=0 tx=0).stop 2023-10-09T06:37:34.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.100+0000 7fad328b3700 1 -- 172.21.15.163:0/528802502 shutdown_connections 2023-10-09T06:37:34.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.100+0000 7fad328b3700 1 --2- 172.21.15.163:0/528802502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad2c0699e0 0x7fad2c0b05e0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:34.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.100+0000 7fad328b3700 1 --2- 172.21.15.163:0/528802502 >> v2:172.21.15.181:3300/0 conn(0x7fad2c159c10 0x7fad2c0694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:34.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.100+0000 7fad328b3700 1 -- 172.21.15.163:0/528802502 >> 172.21.15.163:0/528802502 conn(0x7fad2c068e10 msgr2=0x7fad2c0d0760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:34.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.100+0000 7fad328b3700 1 -- 172.21.15.163:0/528802502 shutdown_connections 2023-10-09T06:37:34.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.100+0000 7fad328b3700 1 -- 172.21.15.163:0/528802502 wait complete. 2023-10-09T06:37:34.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.101+0000 7fad328b3700 1 Processor -- start 2023-10-09T06:37:34.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.101+0000 7fad328b3700 1 -- start start 2023-10-09T06:37:34.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.102+0000 7fad328b3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad2c0699e0 0x7fad2c153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:34.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.102+0000 7fad328b3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad2c159840 0x7fad2c0c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:34.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.102+0000 7fad328b3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad2c159c10 0x7fad2c0b25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:34.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.102+0000 7fad328b3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c0c8c00 con 0x7fad2c159c10 2023-10-09T06:37:34.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.102+0000 7fad328b3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c154450 con 0x7fad2c159840 2023-10-09T06:37:34.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.102+0000 7fad328b3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c0c7d50 con 0x7fad2c0699e0 2023-10-09T06:37:34.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.102+0000 7fad30e50700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad2c159c10 0x7fad2c0b25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:34.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.102+0000 7fad30e50700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad2c159c10 0x7fad2c0b25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49348/0 (socket says 172.21.15.163:49348) 2023-10-09T06:37:34.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.102+0000 7fad30e50700 1 -- 172.21.15.163:0/2412103227 learned_addr learned my addr 172.21.15.163:0/2412103227 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:34.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.102+0000 7fad30e50700 1 -- 172.21.15.163:0/2412103227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad2c0699e0 msgr2=0x7fad2c153f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:34.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.102+0000 7fad30e50700 1 --2- 172.21.15.163:0/2412103227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad2c0699e0 0x7fad2c153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:34.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.102+0000 7fad30e50700 1 -- 172.21.15.163:0/2412103227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad2c159840 msgr2=0x7fad2c0c7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:34.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.102+0000 7fad30e50700 1 --2- 172.21.15.163:0/2412103227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad2c159840 0x7fad2c0c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:34.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.102+0000 7fad30e50700 1 -- 172.21.15.163:0/2412103227 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad24009ae0 con 0x7fad2c159c10 2023-10-09T06:37:34.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.102+0000 7fad30e50700 1 --2- 172.21.15.163:0/2412103227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad2c159c10 0x7fad2c0b25b0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fad24009fd0 tx=0x7fad24017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:34.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.103+0000 7fad23fff700 1 -- 172.21.15.163:0/2412103227 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad240092e0 con 0x7fad2c159c10 2023-10-09T06:37:34.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.103+0000 7fad23fff700 1 -- 172.21.15.163:0/2412103227 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2401d420 con 0x7fad2c159c10 2023-10-09T06:37:34.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.103+0000 7fad328b3700 1 -- 172.21.15.163:0/2412103227 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad2c0c7ff0 con 0x7fad2c159c10 2023-10-09T06:37:34.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.103+0000 7fad23fff700 1 -- 172.21.15.163:0/2412103227 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2401d7e0 con 0x7fad2c159c10 2023-10-09T06:37:34.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.103+0000 7fad328b3700 1 -- 172.21.15.163:0/2412103227 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad2c0c85b0 con 0x7fad2c159c10 2023-10-09T06:37:34.108 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.103+0000 7fad328b3700 1 -- 172.21.15.163:0/2412103227 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad2c064550 con 0x7fad2c159c10 2023-10-09T06:37:34.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.104+0000 7fad23fff700 1 -- 172.21.15.163:0/2412103227 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fad2401dbd0 con 0x7fad2c159c10 2023-10-09T06:37:34.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.104+0000 7fad23fff700 1 --2- 172.21.15.163:0/2412103227 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fad0c067320 0x7fad0c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:34.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.104+0000 7fad23fff700 1 -- 172.21.15.163:0/2412103227 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5877+0+0 (secure 0 0 0) 0x7fad240864c0 con 0x7fad2c159c10 2023-10-09T06:37:34.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.104+0000 7fad2bfff700 1 --2- 172.21.15.163:0/2412103227 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fad0c067320 0x7fad0c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:34.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.105+0000 7fad2bfff700 1 --2- 172.21.15.163:0/2412103227 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fad0c067320 0x7fad0c0697d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fad14007fd0 tx=0x7fad14007b20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:34.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.108+0000 7fad23fff700 1 -- 172.21.15.163:0/2412103227 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad24052f90 con 0x7fad2c159c10 2023-10-09T06:37:34.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.333+0000 7fad328b3700 1 -- 172.21.15.163:0/2412103227 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad2c069e40 con 0x7fad2c159c10 2023-10-09T06:37:34.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.333+0000 7fad23fff700 1 -- 172.21.15.163:0/2412103227 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fad2c069e40 con 0x7fad2c159c10 2023-10-09T06:37:34.334 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:34.335 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":85,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2821591245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2821591245},{"type":"v1","addr":"172.21.15.181:6836","nonce":2821591245}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":70},{"gid":6784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1929060525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1929060525},{"type":"v1","addr":"172.21.15.181:6837","nonce":1929060525}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":7371,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/205885744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":205885744},{"type":"v1","addr":"172.21.15.163:6834","nonce":205885744}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:37:08.129202+0000","modified":"2023-10-09T06:37:11.774637+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7161},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7161":{"gid":7161,"name":"c","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/571471083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":571471083},{"type":"v1","addr":"172.21.15.163:6835","nonce":571471083}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-09T06:37:34.337 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.335+0000 7fad328b3700 1 -- 172.21.15.163:0/2412103227 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fad0c067320 msgr2=0x7fad0c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:34.337 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.335+0000 7fad328b3700 1 --2- 172.21.15.163:0/2412103227 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fad0c067320 0x7fad0c0697d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fad14007fd0 tx=0x7fad14007b20 comp rx=0 tx=0).stop 2023-10-09T06:37:34.337 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.335+0000 7fad328b3700 1 -- 172.21.15.163:0/2412103227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad2c159c10 msgr2=0x7fad2c0b25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:34.337 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.335+0000 7fad328b3700 1 --2- 172.21.15.163:0/2412103227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad2c159c10 0x7fad2c0b25b0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fad24009fd0 tx=0x7fad24017000 comp rx=0 tx=0).stop 2023-10-09T06:37:34.337 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.336+0000 7fad328b3700 1 -- 172.21.15.163:0/2412103227 shutdown_connections 2023-10-09T06:37:34.337 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.336+0000 7fad328b3700 1 --2- 172.21.15.163:0/2412103227 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fad0c067320 0x7fad0c0697d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:34.337 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.336+0000 7fad328b3700 1 --2- 172.21.15.163:0/2412103227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad2c159c10 0x7fad2c0b25b0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:34.337 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.336+0000 7fad328b3700 1 --2- 172.21.15.163:0/2412103227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad2c0699e0 0x7fad2c153f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:34.337 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.336+0000 7fad328b3700 1 --2- 172.21.15.163:0/2412103227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad2c159840 0x7fad2c0c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:34.338 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.336+0000 7fad328b3700 1 -- 172.21.15.163:0/2412103227 >> 172.21.15.163:0/2412103227 conn(0x7fad2c068e10 msgr2=0x7fad2c0c68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:34.338 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.336+0000 7fad328b3700 1 -- 172.21.15.163:0/2412103227 shutdown_connections 2023-10-09T06:37:34.338 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.336+0000 7fad328b3700 1 -- 172.21.15.163:0/2412103227 wait complete. 2023-10-09T06:37:34.338 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 85 2023-10-09T06:37:34.349 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:37:34.463 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.462+0000 7f921e0d7700 1 Processor -- start 2023-10-09T06:37:34.463 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.463+0000 7f921e0d7700 1 -- start start 2023-10-09T06:37:34.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.463+0000 7f921e0d7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9218069d60 0x7f92180c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:34.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.463+0000 7f921e0d7700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f921806bdb0 con 0x7f92180c81f0 2023-10-09T06:37:34.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.463+0000 7f921e0d7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92180cb370 con 0x7f92180aca80 2023-10-09T06:37:34.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.463+0000 7f921e0d7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9218159f60 con 0x7f9218069d60 2023-10-09T06:37:34.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.464+0000 7f9217fff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f92180c81f0 0x7f92180cb800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:52732/0 (socket says 172.21.15.163:52732) 2023-10-09T06:37:34.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.464+0000 7f9217fff700 1 -- 172.21.15.163:0/3342776073 learned_addr learned my addr 172.21.15.163:0/3342776073 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:34.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.464+0000 7f9216ffd700 1 --2- 172.21.15.163:0/3342776073 >> v2:172.21.15.181:3300/0 conn(0x7f9218069d60 0x7f92180c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:34.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.464+0000 7f92167fc700 1 -- 172.21.15.163:0/3342776073 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3966617613 0 0) 0x7f921806bdb0 con 0x7f92180c81f0 2023-10-09T06:37:34.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.464+0000 7f92167fc700 1 -- 172.21.15.163:0/3342776073 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9200003570 con 0x7f92180c81f0 2023-10-09T06:37:34.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.464+0000 7f92167fc700 1 -- 172.21.15.163:0/3342776073 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2795559354 0 0) 0x7f9200003570 con 0x7f92180c81f0 2023-10-09T06:37:34.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.464+0000 7f92167fc700 1 -- 172.21.15.163:0/3342776073 >> v2:172.21.15.181:3300/0 conn(0x7f9218069d60 msgr2=0x7f92180c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:34.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.464+0000 7f92167fc700 1 --2- 172.21.15.163:0/3342776073 >> v2:172.21.15.181:3300/0 conn(0x7f9218069d60 0x7f92180c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:34.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.464+0000 7f92167fc700 1 -- 172.21.15.163:0/3342776073 >> v1:172.21.15.181:6789/0 conn(0x7f92180aca80 legacy=0x7f921806aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:34.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.464+0000 7f92167fc700 1 -- 172.21.15.163:0/3342776073 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921806b2b0 con 0x7f92180c81f0 2023-10-09T06:37:34.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.464+0000 7f92167fc700 1 -- 172.21.15.163:0/3342776073 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f9210001470 con 0x7f92180c81f0 2023-10-09T06:37:34.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.464+0000 7f92167fc700 1 -- 172.21.15.163:0/3342776073 >> v1:172.21.15.163:6789/0 conn(0x7f92180c81f0 legacy=0x7f92180cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:34.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.464+0000 7f92167fc700 1 --2- 172.21.15.163:0/3342776073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9200003a60 0x7f9200003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:34.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.464+0000 7f92167fc700 1 --2- 172.21.15.163:0/3342776073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9200004810 0x7f9200004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:34.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.464+0000 7f92167fc700 1 --2- 172.21.15.163:0/3342776073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92000051c0 0x7f920000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:34.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.464+0000 7f92167fc700 1 -- 172.21.15.163:0/3342776073 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9218159f60 con 0x7f9200003a60 2023-10-09T06:37:34.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.464+0000 7f92167fc700 1 -- 172.21.15.163:0/3342776073 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920000c0d0 con 0x7f9200004810 2023-10-09T06:37:34.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.464+0000 7f92167fc700 1 -- 172.21.15.163:0/3342776073 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920000c290 con 0x7f92000051c0 2023-10-09T06:37:34.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.464+0000 7f92177fe700 1 --2- 172.21.15.163:0/3342776073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9200003a60 0x7f9200003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:34.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.464+0000 7f9216ffd700 1 --2- 172.21.15.163:0/3342776073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9200004810 0x7f9200004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:34.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.464+0000 7f9217fff700 1 --2- 172.21.15.163:0/3342776073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92000051c0 0x7f920000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:34.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.465+0000 7f92177fe700 1 -- 172.21.15.163:0/3342776073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92000051c0 msgr2=0x7f920000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:34.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.465+0000 7f92177fe700 1 --2- 172.21.15.163:0/3342776073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92000051c0 0x7f920000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:34.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.465+0000 7f92177fe700 1 -- 172.21.15.163:0/3342776073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9200004810 msgr2=0x7f9200004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:34.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.465+0000 7f92177fe700 1 --2- 172.21.15.163:0/3342776073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9200004810 0x7f9200004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:34.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.465+0000 7f92177fe700 1 -- 172.21.15.163:0/3342776073 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9200003570 con 0x7f9200003a60 2023-10-09T06:37:34.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.465+0000 7f9217fff700 1 --2- 172.21.15.163:0/3342776073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92000051c0 0x7f920000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:34.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.465+0000 7f9217fff700 1 -- 172.21.15.163:0/3342776073 reap_dead start 2023-10-09T06:37:34.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.465+0000 7f92177fe700 1 --2- 172.21.15.163:0/3342776073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9200003a60 0x7f9200003e70 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f920800c2d0 tx=0x7f9208001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a538f46feafcbc server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:34.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.465+0000 7f92167fc700 1 -- 172.21.15.163:0/3342776073 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208001d80 con 0x7f9200003a60 2023-10-09T06:37:34.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.465+0000 7f92167fc700 1 -- 172.21.15.163:0/3342776073 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f920800cba0 con 0x7f9200003a60 2023-10-09T06:37:34.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.465+0000 7f92167fc700 1 -- 172.21.15.163:0/3342776073 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208012400 con 0x7f9200003a60 2023-10-09T06:37:34.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.465+0000 7f921e0d7700 1 -- 172.21.15.163:0/3342776073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9200003a60 msgr2=0x7f9200003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:34.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.465+0000 7f921e0d7700 1 --2- 172.21.15.163:0/3342776073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9200003a60 0x7f9200003e70 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f920800c2d0 tx=0x7f9208001a70 comp rx=0 tx=0).stop 2023-10-09T06:37:34.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.465+0000 7f921e0d7700 1 -- 172.21.15.163:0/3342776073 shutdown_connections 2023-10-09T06:37:34.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.465+0000 7f921e0d7700 1 --2- 172.21.15.163:0/3342776073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9200003a60 0x7f9200003e70 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:34.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.465+0000 7f921e0d7700 1 -- 172.21.15.163:0/3342776073 >> 172.21.15.163:0/3342776073 conn(0x7f92180cd670 msgr2=0x7f92180cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:34.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.465+0000 7f921e0d7700 1 -- 172.21.15.163:0/3342776073 shutdown_connections 2023-10-09T06:37:34.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.465+0000 7f921e0d7700 1 -- 172.21.15.163:0/3342776073 wait complete. 2023-10-09T06:37:34.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.467+0000 7f921e0d7700 1 Processor -- start 2023-10-09T06:37:34.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.467+0000 7f921e0d7700 1 -- start start 2023-10-09T06:37:34.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.467+0000 7f921e0d7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9200003a60 0x7f9218169db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:34.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.467+0000 7f921e0d7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f921815c8d0 0x7f921816a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:34.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.467+0000 7f921e0d7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9218159410 0x7f92180b2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:34.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.467+0000 7f921e0d7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921815c440 con 0x7f9200003a60 2023-10-09T06:37:34.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.467+0000 7f921e0d7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921815b830 con 0x7f921815c8d0 2023-10-09T06:37:34.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.467+0000 7f921e0d7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f921815bb10 con 0x7f9218159410 2023-10-09T06:37:34.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.467+0000 7f92177fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9200003a60 0x7f9218169db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:34.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.467+0000 7f92177fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9200003a60 0x7f9218169db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49360/0 (socket says 172.21.15.163:49360) 2023-10-09T06:37:34.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.467+0000 7f92177fe700 1 -- 172.21.15.163:0/3527438651 learned_addr learned my addr 172.21.15.163:0/3527438651 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:34.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.468+0000 7f9216ffd700 1 --2- 172.21.15.163:0/3527438651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f921815c8d0 0x7f921816a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:34.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.468+0000 7f9217fff700 1 --2- 172.21.15.163:0/3527438651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9218159410 0x7f92180b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:34.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.468+0000 7f92177fe700 1 -- 172.21.15.163:0/3527438651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9218159410 msgr2=0x7f92180b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:34.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.468+0000 7f92177fe700 1 --2- 172.21.15.163:0/3527438651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9218159410 0x7f92180b2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:34.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.468+0000 7f92177fe700 1 -- 172.21.15.163:0/3527438651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f921815c8d0 msgr2=0x7f921816a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:34.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.468+0000 7f92177fe700 1 --2- 172.21.15.163:0/3527438651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f921815c8d0 0x7f921816a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:34.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.468+0000 7f92177fe700 1 -- 172.21.15.163:0/3527438651 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921815bdf0 con 0x7f9200003a60 2023-10-09T06:37:34.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.468+0000 7f92177fe700 1 --2- 172.21.15.163:0/3527438651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9200003a60 0x7f9218169db0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f920800c8c0 tx=0x7f9208016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:34.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.468+0000 7f91ff7fe700 1 -- 172.21.15.163:0/3527438651 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920800cde0 con 0x7f9200003a60 2023-10-09T06:37:34.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.468+0000 7f91ff7fe700 1 -- 172.21.15.163:0/3527438651 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9208021070 con 0x7f9200003a60 2023-10-09T06:37:34.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.468+0000 7f91ff7fe700 1 -- 172.21.15.163:0/3527438651 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208026680 con 0x7f9200003a60 2023-10-09T06:37:34.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.468+0000 7f921e0d7700 1 -- 172.21.15.163:0/3527438651 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f921815c0d0 con 0x7f9200003a60 2023-10-09T06:37:34.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.468+0000 7f921e0d7700 1 -- 172.21.15.163:0/3527438651 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f921815b420 con 0x7f9200003a60 2023-10-09T06:37:34.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.469+0000 7f921e0d7700 1 -- 172.21.15.163:0/3527438651 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f921804c3c0 con 0x7f9200003a60 2023-10-09T06:37:34.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.469+0000 7f91ff7fe700 1 -- 172.21.15.163:0/3527438651 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9208023030 con 0x7f9200003a60 2023-10-09T06:37:34.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.470+0000 7f91ff7fe700 1 --2- 172.21.15.163:0/3527438651 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9204067320 0x7f92040697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:34.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.470+0000 7f91ff7fe700 1 -- 172.21.15.163:0/3527438651 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5877+0+0 (secure 0 0 0) 0x7f9208086cc0 con 0x7f9200003a60 2023-10-09T06:37:34.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.470+0000 7f9216ffd700 1 --2- 172.21.15.163:0/3527438651 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9204067320 0x7f92040697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:34.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.470+0000 7f9216ffd700 1 --2- 172.21.15.163:0/3527438651 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9204067320 0x7f92040697d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f920c006dc0 tx=0x7f920c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:34.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.474+0000 7f91ff7fe700 1 -- 172.21.15.163:0/3527438651 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f920808d080 con 0x7f9200003a60 2023-10-09T06:37:34.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.660+0000 7f921e0d7700 1 -- 172.21.15.163:0/3527438651 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f921815c720 con 0x7f9200003a60 2023-10-09T06:37:34.662 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.661+0000 7f91ff7fe700 1 -- 172.21.15.163:0/3527438651 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+15675 (secure 0 0 0) 0x7f9208057440 con 0x7f9200003a60 2023-10-09T06:37:34.662 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:34.662 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":125,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:37:31.984855+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:37:06.472811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-09T06:37:07.096712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6837/3075056851":"2023-10-10T06:37:28.977957+0000","172.21.15.181:6835/3075056851":"2023-10-10T06:37:28.977957+0000","172.21.15.163:6834/1355789800":"2023-10-10T06:37:25.586483+0000","172.21.15.163:6832/1355789800":"2023-10-10T06:37:25.586483+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:37:34.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.663+0000 7f921e0d7700 1 -- 172.21.15.163:0/3527438651 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9204067320 msgr2=0x7f92040697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:34.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.663+0000 7f921e0d7700 1 --2- 172.21.15.163:0/3527438651 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9204067320 0x7f92040697d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f920c006dc0 tx=0x7f920c008040 comp rx=0 tx=0).stop 2023-10-09T06:37:34.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.663+0000 7f921e0d7700 1 -- 172.21.15.163:0/3527438651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9200003a60 msgr2=0x7f9218169db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:34.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.663+0000 7f921e0d7700 1 --2- 172.21.15.163:0/3527438651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9200003a60 0x7f9218169db0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f920800c8c0 tx=0x7f9208016000 comp rx=0 tx=0).stop 2023-10-09T06:37:34.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.663+0000 7f921e0d7700 1 -- 172.21.15.163:0/3527438651 shutdown_connections 2023-10-09T06:37:34.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.663+0000 7f921e0d7700 1 --2- 172.21.15.163:0/3527438651 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9204067320 0x7f92040697d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:34.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.663+0000 7f921e0d7700 1 --2- 172.21.15.163:0/3527438651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9200003a60 0x7f9218169db0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:34.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.663+0000 7f921e0d7700 1 --2- 172.21.15.163:0/3527438651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9218159410 0x7f92180b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:34.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.663+0000 7f921e0d7700 1 --2- 172.21.15.163:0/3527438651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f921815c8d0 0x7f921816a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:34.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.663+0000 7f921e0d7700 1 -- 172.21.15.163:0/3527438651 >> 172.21.15.163:0/3527438651 conn(0x7f92180cd670 msgr2=0x7f92180cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:34.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.663+0000 7f921e0d7700 1 -- 172.21.15.163:0/3527438651 shutdown_connections 2023-10-09T06:37:34.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.663+0000 7f921e0d7700 1 -- 172.21.15.163:0/3527438651 wait complete. 2023-10-09T06:37:34.676 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-09T06:37:34.677 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:37:34.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.790+0000 7f1963fff700 1 Processor -- start 2023-10-09T06:37:34.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.790+0000 7f1963fff700 1 -- start start 2023-10-09T06:37:34.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.791+0000 7f1963fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1964159420 0x7f19640b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:34.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.791+0000 7f1963fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f19641597f0 0x7f19640b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:34.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.791+0000 7f1963fff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19640b11d0 con 0x7f196415d230 2023-10-09T06:37:34.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.791+0000 7f1963fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19640b14b0 con 0x7f19641597f0 2023-10-09T06:37:34.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.791+0000 7f1963fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19640b1790 con 0x7f1964159420 2023-10-09T06:37:34.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.791+0000 7f195a7fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f19641597f0 0x7f19640b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:34.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.791+0000 7f195a7fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f19641597f0 0x7f19640b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49374/0 (socket says 172.21.15.163:49374) 2023-10-09T06:37:34.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.791+0000 7f195a7fc700 1 -- 172.21.15.163:0/3175567500 learned_addr learned my addr 172.21.15.163:0/3175567500 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:34.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.791+0000 7f1962ffd700 1 --2- 172.21.15.163:0/3175567500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1964159420 0x7f19640b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:34.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.791+0000 7f195a7fc700 1 -- 172.21.15.163:0/3175567500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1964159420 msgr2=0x7f19640b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:34.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.791+0000 7f195a7fc700 1 --2- 172.21.15.163:0/3175567500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1964159420 0x7f19640b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:34.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.791+0000 7f195a7fc700 1 -- 172.21.15.163:0/3175567500 >> v1:172.21.15.181:6789/0 conn(0x7f196415d230 legacy=0x7f19640b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:34.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.791+0000 7f195a7fc700 1 -- 172.21.15.163:0/3175567500 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19640b1a70 con 0x7f19641597f0 2023-10-09T06:37:34.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.791+0000 7f195a7fc700 1 --2- 172.21.15.163:0/3175567500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19641597f0 0x7f19640b2270 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f194c0099b0 tx=0x7f194c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f80340525ade9fe2 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:34.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.792+0000 7f19627fc700 1 -- 172.21.15.163:0/3175567500 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c00b860 con 0x7f19641597f0 2023-10-09T06:37:34.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.792+0000 7f19627fc700 1 -- 172.21.15.163:0/3175567500 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194c00b9c0 con 0x7f19641597f0 2023-10-09T06:37:34.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.792+0000 7f19627fc700 1 -- 172.21.15.163:0/3175567500 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c00bd60 con 0x7f19641597f0 2023-10-09T06:37:34.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.792+0000 7f1963fff700 1 -- 172.21.15.163:0/3175567500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19641597f0 msgr2=0x7f19640b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:34.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.792+0000 7f1963fff700 1 --2- 172.21.15.163:0/3175567500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19641597f0 0x7f19640b2270 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f194c0099b0 tx=0x7f194c009250 comp rx=0 tx=0).stop 2023-10-09T06:37:34.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.792+0000 7f1963fff700 1 -- 172.21.15.163:0/3175567500 shutdown_connections 2023-10-09T06:37:34.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.792+0000 7f1963fff700 1 --2- 172.21.15.163:0/3175567500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1964159420 0x7f19640b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:34.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.792+0000 7f1963fff700 1 --2- 172.21.15.163:0/3175567500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19641597f0 0x7f19640b2270 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:34.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.792+0000 7f1963fff700 1 -- 172.21.15.163:0/3175567500 >> 172.21.15.163:0/3175567500 conn(0x7f19640d0660 msgr2=0x7f19640d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:34.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.792+0000 7f1963fff700 1 -- 172.21.15.163:0/3175567500 shutdown_connections 2023-10-09T06:37:34.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.792+0000 7f1963fff700 1 -- 172.21.15.163:0/3175567500 wait complete. 2023-10-09T06:37:34.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.793+0000 7f1963fff700 1 Processor -- start 2023-10-09T06:37:34.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.793+0000 7f1963fff700 1 -- start start 2023-10-09T06:37:34.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.794+0000 7f1963fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1964159420 0x7f19640c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:34.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.794+0000 7f1963fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19641597f0 0x7f19640c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:34.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.794+0000 7f1963fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f196415d230 0x7f196416aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:34.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.794+0000 7f1963fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196415c790 con 0x7f19641597f0 2023-10-09T06:37:34.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.794+0000 7f1963fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1964169c50 con 0x7f1964159420 2023-10-09T06:37:34.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.794+0000 7f1963fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1964169f30 con 0x7f196415d230 2023-10-09T06:37:34.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.794+0000 7f195a7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19641597f0 0x7f19640c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:34.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.794+0000 7f195a7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19641597f0 0x7f19640c5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49382/0 (socket says 172.21.15.163:49382) 2023-10-09T06:37:34.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.794+0000 7f195a7fc700 1 -- 172.21.15.163:0/1027282174 learned_addr learned my addr 172.21.15.163:0/1027282174 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:34.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.794+0000 7f1962ffd700 1 --2- 172.21.15.163:0/1027282174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1964159420 0x7f19640c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:34.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.795+0000 7f19637fe700 1 --2- 172.21.15.163:0/1027282174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f196415d230 0x7f196416aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:34.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.795+0000 7f195a7fc700 1 -- 172.21.15.163:0/1027282174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f196415d230 msgr2=0x7f196416aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:34.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.795+0000 7f195a7fc700 1 --2- 172.21.15.163:0/1027282174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f196415d230 0x7f196416aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:34.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.795+0000 7f195a7fc700 1 -- 172.21.15.163:0/1027282174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1964159420 msgr2=0x7f19640c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:34.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.795+0000 7f195a7fc700 1 --2- 172.21.15.163:0/1027282174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1964159420 0x7f19640c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:34.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.795+0000 7f195a7fc700 1 -- 172.21.15.163:0/1027282174 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c0094f0 con 0x7f19641597f0 2023-10-09T06:37:34.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.795+0000 7f19637fe700 1 --2- 172.21.15.163:0/1027282174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f196415d230 0x7f196416aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:37:34.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.795+0000 7f195a7fc700 1 --2- 172.21.15.163:0/1027282174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19641597f0 0x7f19640c5940 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f1964089340 tx=0x7f194c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:34.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.795+0000 7f195b7fe700 1 -- 172.21.15.163:0/1027282174 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c024440 con 0x7f19641597f0 2023-10-09T06:37:34.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.795+0000 7f195b7fe700 1 -- 172.21.15.163:0/1027282174 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194c0245c0 con 0x7f19641597f0 2023-10-09T06:37:34.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.795+0000 7f195b7fe700 1 -- 172.21.15.163:0/1027282174 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c024980 con 0x7f19641597f0 2023-10-09T06:37:34.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.795+0000 7f1963fff700 1 -- 172.21.15.163:0/1027282174 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196416a210 con 0x7f19641597f0 2023-10-09T06:37:34.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.795+0000 7f1963fff700 1 -- 172.21.15.163:0/1027282174 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f196416af50 con 0x7f19641597f0 2023-10-09T06:37:34.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.796+0000 7f1963fff700 1 -- 172.21.15.163:0/1027282174 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f196416a730 con 0x7f19641597f0 2023-10-09T06:37:34.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.796+0000 7f195b7fe700 1 -- 172.21.15.163:0/1027282174 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f194c01a420 con 0x7f19641597f0 2023-10-09T06:37:34.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.797+0000 7f195b7fe700 1 --2- 172.21.15.163:0/1027282174 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1944067320 0x7f19440697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:34.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.797+0000 7f195b7fe700 1 -- 172.21.15.163:0/1027282174 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5877+0+0 (secure 0 0 0) 0x7f194c086af0 con 0x7f19641597f0 2023-10-09T06:37:34.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.797+0000 7f1962ffd700 1 --2- 172.21.15.163:0/1027282174 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1944067320 0x7f19440697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:34.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.797+0000 7f1962ffd700 1 --2- 172.21.15.163:0/1027282174 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1944067320 0x7f19440697d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1954006fd0 tx=0x7f1954008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:34.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:34.801+0000 7f195b7fe700 1 -- 172.21.15.163:0/1027282174 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f196416a730 con 0x7f19641597f0 2023-10-09T06:37:35.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.024+0000 7f1963fff700 1 -- 172.21.15.163:0/1027282174 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f196416a730 con 0x7f19641597f0 2023-10-09T06:37:35.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.024+0000 7f195b7fe700 1 -- 172.21.15.163:0/1027282174 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f196416a730 con 0x7f19641597f0 2023-10-09T06:37:35.025 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:35.025 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":85,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2821591245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2821591245},{"type":"v1","addr":"172.21.15.181:6836","nonce":2821591245}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":70},{"gid":6784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1929060525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1929060525},{"type":"v1","addr":"172.21.15.181:6837","nonce":1929060525}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":7371,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/205885744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":205885744},{"type":"v1","addr":"172.21.15.163:6834","nonce":205885744}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:37:08.129202+0000","modified":"2023-10-09T06:37:11.774637+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7161},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7161":{"gid":7161,"name":"c","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/571471083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":571471083},{"type":"v1","addr":"172.21.15.163:6835","nonce":571471083}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-09T06:37:35.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.027+0000 7f1963fff700 1 -- 172.21.15.163:0/1027282174 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1944067320 msgr2=0x7f19440697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:35.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.027+0000 7f1963fff700 1 --2- 172.21.15.163:0/1027282174 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1944067320 0x7f19440697d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1954006fd0 tx=0x7f1954008040 comp rx=0 tx=0).stop 2023-10-09T06:37:35.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.027+0000 7f1963fff700 1 -- 172.21.15.163:0/1027282174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19641597f0 msgr2=0x7f19640c5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:35.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.027+0000 7f1963fff700 1 --2- 172.21.15.163:0/1027282174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19641597f0 0x7f19640c5940 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f1964089340 tx=0x7f194c017000 comp rx=0 tx=0).stop 2023-10-09T06:37:35.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.027+0000 7f1963fff700 1 -- 172.21.15.163:0/1027282174 shutdown_connections 2023-10-09T06:37:35.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.027+0000 7f1963fff700 1 --2- 172.21.15.163:0/1027282174 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1944067320 0x7f19440697d0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:35.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.027+0000 7f1963fff700 1 --2- 172.21.15.163:0/1027282174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f196415d230 0x7f196416aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:35.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.027+0000 7f1963fff700 1 --2- 172.21.15.163:0/1027282174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1964159420 0x7f19640c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:35.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.027+0000 7f1963fff700 1 --2- 172.21.15.163:0/1027282174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19641597f0 0x7f19640c5940 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:35.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.027+0000 7f1963fff700 1 -- 172.21.15.163:0/1027282174 >> 172.21.15.163:0/1027282174 conn(0x7f19640d0660 msgr2=0x7f1964157080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:35.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.027+0000 7f1963fff700 1 -- 172.21.15.163:0/1027282174 shutdown_connections 2023-10-09T06:37:35.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.027+0000 7f1963fff700 1 -- 172.21.15.163:0/1027282174 wait complete. 2023-10-09T06:37:35.029 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 85 2023-10-09T06:37:35.040 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:37:35.154 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.153+0000 7f483ff3e700 1 Processor -- start 2023-10-09T06:37:35.154 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.153+0000 7f483ff3e700 1 -- start start 2023-10-09T06:37:35.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.154+0000 7f483ff3e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4838068920 0x7f483806f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:35.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.154+0000 7f483ff3e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f483806ec70 con 0x7f483806f640 2023-10-09T06:37:35.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.154+0000 7f483ff3e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48380b2e70 con 0x7f4838068550 2023-10-09T06:37:35.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.154+0000 7f483ff3e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4838068d90 con 0x7f4838068920 2023-10-09T06:37:35.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.154+0000 7f483d4d9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4838068920 0x7f483806f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:35.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.154+0000 7f483e4db700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f483806f640 0x7f4838159510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:52742/0 (socket says 172.21.15.163:52742) 2023-10-09T06:37:35.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.154+0000 7f483e4db700 1 -- 172.21.15.163:0/151814737 learned_addr learned my addr 172.21.15.163:0/151814737 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:35.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.155+0000 7f483d4d9700 1 -- 172.21.15.163:0/151814737 >> v1:172.21.15.181:6789/0 conn(0x7f4838068550 legacy=0x7f483815e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:35.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.155+0000 7f483d4d9700 1 -- 172.21.15.163:0/151814737 >> v1:172.21.15.163:6789/0 conn(0x7f483806f640 legacy=0x7f4838159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:35.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.155+0000 7f483d4d9700 1 -- 172.21.15.163:0/151814737 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48380b3150 con 0x7f4838068920 2023-10-09T06:37:35.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.155+0000 7f483ccd8700 1 -- 172.21.15.163:0/151814737 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 203124604 0 0) 0x7f483806ec70 con 0x7f483806f640 2023-10-09T06:37:35.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.155+0000 7f483d4d9700 1 --2- 172.21.15.163:0/151814737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4838068920 0x7f483806f100 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f48280098e0 tx=0x7f48280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93abecc9aa19e65 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:35.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.155+0000 7f483ccd8700 1 -- 172.21.15.163:0/151814737 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4828009e80 con 0x7f4838068920 2023-10-09T06:37:35.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.155+0000 7f483ccd8700 1 -- 172.21.15.163:0/151814737 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482800b820 con 0x7f4838068920 2023-10-09T06:37:35.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.155+0000 7f483ccd8700 1 -- 172.21.15.163:0/151814737 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482800bba0 con 0x7f4838068920 2023-10-09T06:37:35.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.155+0000 7f483ff3e700 1 -- 172.21.15.163:0/151814737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4838068920 msgr2=0x7f483806f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:35.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.155+0000 7f483ff3e700 1 --2- 172.21.15.163:0/151814737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4838068920 0x7f483806f100 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f48280098e0 tx=0x7f48280091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:35.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.155+0000 7f483ff3e700 1 -- 172.21.15.163:0/151814737 shutdown_connections 2023-10-09T06:37:35.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.155+0000 7f483ff3e700 1 --2- 172.21.15.163:0/151814737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4838068920 0x7f483806f100 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:35.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.155+0000 7f483ff3e700 1 -- 172.21.15.163:0/151814737 >> 172.21.15.163:0/151814737 conn(0x7f48380cd480 msgr2=0x7f48380cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:35.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.155+0000 7f483ff3e700 1 -- 172.21.15.163:0/151814737 shutdown_connections 2023-10-09T06:37:35.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.155+0000 7f483ff3e700 1 -- 172.21.15.163:0/151814737 wait complete. 2023-10-09T06:37:35.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.156+0000 7f483ff3e700 1 Processor -- start 2023-10-09T06:37:35.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.157+0000 7f483ff3e700 1 -- start start 2023-10-09T06:37:35.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.157+0000 7f483ff3e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4838068550 0x7f483815ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:35.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.157+0000 7f483ff3e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4838068920 0x7f48380ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:35.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.157+0000 7f483ff3e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f483806f640 0x7f4838151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:35.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.157+0000 7f483ff3e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48380ae890 con 0x7f483806f640 2023-10-09T06:37:35.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.157+0000 7f483ff3e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48380adde0 con 0x7f4838068550 2023-10-09T06:37:35.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.157+0000 7f483ff3e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48380a9b90 con 0x7f4838068920 2023-10-09T06:37:35.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.157+0000 7f483e4db700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f483806f640 0x7f4838151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:35.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.157+0000 7f483e4db700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f483806f640 0x7f4838151d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49402/0 (socket says 172.21.15.163:49402) 2023-10-09T06:37:35.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.157+0000 7f483e4db700 1 -- 172.21.15.163:0/1067681183 learned_addr learned my addr 172.21.15.163:0/1067681183 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:35.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.158+0000 7f483d4d9700 1 --2- 172.21.15.163:0/1067681183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4838068920 0x7f48380ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:35.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.158+0000 7f483dcda700 1 --2- 172.21.15.163:0/1067681183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4838068550 0x7f483815ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:35.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.158+0000 7f483e4db700 1 -- 172.21.15.163:0/1067681183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4838068920 msgr2=0x7f48380ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:35.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.158+0000 7f483e4db700 1 --2- 172.21.15.163:0/1067681183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4838068920 0x7f48380ad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:35.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.158+0000 7f483e4db700 1 -- 172.21.15.163:0/1067681183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4838068550 msgr2=0x7f483815ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:35.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.158+0000 7f483e4db700 1 --2- 172.21.15.163:0/1067681183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4838068550 0x7f483815ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:35.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.158+0000 7f483e4db700 1 -- 172.21.15.163:0/1067681183 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4828009420 con 0x7f483806f640 2023-10-09T06:37:35.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.158+0000 7f483e4db700 1 --2- 172.21.15.163:0/1067681183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f483806f640 0x7f4838151d40 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f483000c7e0 tx=0x7f4830007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:35.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.158+0000 7f482dffb700 1 -- 172.21.15.163:0/1067681183 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4830005c00 con 0x7f483806f640 2023-10-09T06:37:35.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.158+0000 7f482dffb700 1 -- 172.21.15.163:0/1067681183 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4830005d60 con 0x7f483806f640 2023-10-09T06:37:35.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.158+0000 7f482dffb700 1 -- 172.21.15.163:0/1067681183 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4830004a80 con 0x7f483806f640 2023-10-09T06:37:35.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.158+0000 7f483ff3e700 1 -- 172.21.15.163:0/1067681183 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48380adff0 con 0x7f483806f640 2023-10-09T06:37:35.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.158+0000 7f483ff3e700 1 -- 172.21.15.163:0/1067681183 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48380ae5b0 con 0x7f483806f640 2023-10-09T06:37:35.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.159+0000 7f483ff3e700 1 -- 172.21.15.163:0/1067681183 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48380a9960 con 0x7f483806f640 2023-10-09T06:37:35.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.160+0000 7f482dffb700 1 -- 172.21.15.163:0/1067681183 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4830005030 con 0x7f483806f640 2023-10-09T06:37:35.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.160+0000 7f482dffb700 1 --2- 172.21.15.163:0/1067681183 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f481c067320 0x7f481c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:35.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.160+0000 7f482dffb700 1 -- 172.21.15.163:0/1067681183 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5877+0+0 (secure 0 0 0) 0x7f48300841a0 con 0x7f483806f640 2023-10-09T06:37:35.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.160+0000 7f483dcda700 1 --2- 172.21.15.163:0/1067681183 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f481c067320 0x7f481c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:35.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.161+0000 7f483dcda700 1 --2- 172.21.15.163:0/1067681183 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f481c067320 0x7f481c0697d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4834009fd0 tx=0x7f4834009600 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:35.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.164+0000 7f482dffb700 1 -- 172.21.15.163:0/1067681183 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f483004b5a0 con 0x7f483806f640 2023-10-09T06:37:35.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.389+0000 7f483ff3e700 1 -- 172.21.15.163:0/1067681183 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48380a9d00 con 0x7f483806f640 2023-10-09T06:37:35.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.390+0000 7f482dffb700 1 -- 172.21.15.163:0/1067681183 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f48380a9960 con 0x7f483806f640 2023-10-09T06:37:35.391 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:35.391 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":85,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2821591245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2821591245},{"type":"v1","addr":"172.21.15.181:6836","nonce":2821591245}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":70},{"gid":6784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1929060525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1929060525},{"type":"v1","addr":"172.21.15.181:6837","nonce":1929060525}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":7371,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/205885744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":205885744},{"type":"v1","addr":"172.21.15.163:6834","nonce":205885744}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:37:08.129202+0000","modified":"2023-10-09T06:37:11.774637+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7161},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7161":{"gid":7161,"name":"c","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/571471083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":571471083},{"type":"v1","addr":"172.21.15.163:6835","nonce":571471083}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-09T06:37:35.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.392+0000 7f483ff3e700 1 -- 172.21.15.163:0/1067681183 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f481c067320 msgr2=0x7f481c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:35.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.392+0000 7f483ff3e700 1 --2- 172.21.15.163:0/1067681183 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f481c067320 0x7f481c0697d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4834009fd0 tx=0x7f4834009600 comp rx=0 tx=0).stop 2023-10-09T06:37:35.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.392+0000 7f483ff3e700 1 -- 172.21.15.163:0/1067681183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f483806f640 msgr2=0x7f4838151d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:35.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.392+0000 7f483ff3e700 1 --2- 172.21.15.163:0/1067681183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f483806f640 0x7f4838151d40 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f483000c7e0 tx=0x7f4830007d90 comp rx=0 tx=0).stop 2023-10-09T06:37:35.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.392+0000 7f483ff3e700 1 -- 172.21.15.163:0/1067681183 shutdown_connections 2023-10-09T06:37:35.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.392+0000 7f483ff3e700 1 --2- 172.21.15.163:0/1067681183 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f481c067320 0x7f481c0697d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:35.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.392+0000 7f483ff3e700 1 --2- 172.21.15.163:0/1067681183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f483806f640 0x7f4838151d40 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:35.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.392+0000 7f483ff3e700 1 --2- 172.21.15.163:0/1067681183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4838068550 0x7f483815ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:35.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.392+0000 7f483ff3e700 1 --2- 172.21.15.163:0/1067681183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4838068920 0x7f48380ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:35.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.392+0000 7f483ff3e700 1 -- 172.21.15.163:0/1067681183 >> 172.21.15.163:0/1067681183 conn(0x7f48380cd480 msgr2=0x7f48380b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:35.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.392+0000 7f483ff3e700 1 -- 172.21.15.163:0/1067681183 shutdown_connections 2023-10-09T06:37:35.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.392+0000 7f483ff3e700 1 -- 172.21.15.163:0/1067681183 wait complete. 2023-10-09T06:37:35.395 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 85 2023-10-09T06:37:35.405 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:37:35.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.518+0000 7fdfa0b30700 1 Processor -- start 2023-10-09T06:37:35.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.518+0000 7fdfa0b30700 1 -- start start 2023-10-09T06:37:35.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.519+0000 7fdfa0b30700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf9c1597f0 0x7fdf9c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:35.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.519+0000 7fdfa0b30700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdf9c15d230 0x7fdf9c15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:35.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.519+0000 7fdfa0b30700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf9c15dbd0 con 0x7fdf9c159420 2023-10-09T06:37:35.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.519+0000 7fdfa0b30700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf9c0b11d0 con 0x7fdf9c15d230 2023-10-09T06:37:35.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.519+0000 7fdfa0b30700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c0b14b0 con 0x7fdf9c1597f0 2023-10-09T06:37:35.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.519+0000 7fdf9bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdf9c15d230 0x7fdf9c15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:35.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.519+0000 7fdf9bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdf9c15d230 0x7fdf9c15d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49410/0 (socket says 172.21.15.163:49410) 2023-10-09T06:37:35.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.519+0000 7fdf9bfff700 1 -- 172.21.15.163:0/83056749 learned_addr learned my addr 172.21.15.163:0/83056749 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:35.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.519+0000 7fdf9affd700 1 --2- 172.21.15.163:0/83056749 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf9c1597f0 0x7fdf9c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:35.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.519+0000 7fdf9bfff700 1 -- 172.21.15.163:0/83056749 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf9c1597f0 msgr2=0x7fdf9c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:35.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.519+0000 7fdf9bfff700 1 --2- 172.21.15.163:0/83056749 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf9c1597f0 0x7fdf9c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:35.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.519+0000 7fdf9bfff700 1 -- 172.21.15.163:0/83056749 >> v1:172.21.15.181:6789/0 conn(0x7fdf9c159420 legacy=0x7fdf9c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:35.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.519+0000 7fdf9bfff700 1 -- 172.21.15.163:0/83056749 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9c0b1790 con 0x7fdf9c15d230 2023-10-09T06:37:35.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.520+0000 7fdf9bfff700 1 --2- 172.21.15.163:0/83056749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf9c15d230 0x7fdf9c15d690 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fdf940099b0 tx=0x7fdf94009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f20fd70413aea23 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:35.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.520+0000 7fdf9a7fc700 1 -- 172.21.15.163:0/83056749 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94009ea0 con 0x7fdf9c15d230 2023-10-09T06:37:35.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.520+0000 7fdf9a7fc700 1 -- 172.21.15.163:0/83056749 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9400b860 con 0x7fdf9c15d230 2023-10-09T06:37:35.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.520+0000 7fdf9a7fc700 1 -- 172.21.15.163:0/83056749 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9400bc00 con 0x7fdf9c15d230 2023-10-09T06:37:35.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.520+0000 7fdfa0b30700 1 -- 172.21.15.163:0/83056749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf9c15d230 msgr2=0x7fdf9c15d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:35.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.520+0000 7fdfa0b30700 1 --2- 172.21.15.163:0/83056749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf9c15d230 0x7fdf9c15d690 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fdf940099b0 tx=0x7fdf94009250 comp rx=0 tx=0).stop 2023-10-09T06:37:35.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.520+0000 7fdfa0b30700 1 -- 172.21.15.163:0/83056749 shutdown_connections 2023-10-09T06:37:35.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.520+0000 7fdfa0b30700 1 --2- 172.21.15.163:0/83056749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf9c15d230 0x7fdf9c15d690 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:35.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.520+0000 7fdfa0b30700 1 --2- 172.21.15.163:0/83056749 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf9c1597f0 0x7fdf9c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:35.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.520+0000 7fdfa0b30700 1 -- 172.21.15.163:0/83056749 >> 172.21.15.163:0/83056749 conn(0x7fdf9c0d0660 msgr2=0x7fdf9c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:35.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.520+0000 7fdfa0b30700 1 -- 172.21.15.163:0/83056749 shutdown_connections 2023-10-09T06:37:35.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.520+0000 7fdfa0b30700 1 -- 172.21.15.163:0/83056749 wait complete. 2023-10-09T06:37:35.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.522+0000 7fdfa0b30700 1 Processor -- start 2023-10-09T06:37:35.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.522+0000 7fdfa0b30700 1 -- start start 2023-10-09T06:37:35.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.522+0000 7fdfa0b30700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf9c159420 0x7fdf9c16a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:35.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.522+0000 7fdfa0b30700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf9c1597f0 0x7fdf9c16c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:35.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.522+0000 7fdfa0b30700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf9c15d230 0x7fdf9c15aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:35.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.522+0000 7fdfa0b30700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c16ca90 con 0x7fdf9c15d230 2023-10-09T06:37:35.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.522+0000 7fdfa0b30700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c15b000 con 0x7fdf9c159420 2023-10-09T06:37:35.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.522+0000 7fdfa0b30700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c0b51b0 con 0x7fdf9c1597f0 2023-10-09T06:37:35.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.522+0000 7fdf9bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf9c15d230 0x7fdf9c15aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:35.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.522+0000 7fdf9bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf9c15d230 0x7fdf9c15aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49426/0 (socket says 172.21.15.163:49426) 2023-10-09T06:37:35.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.522+0000 7fdf9bfff700 1 -- 172.21.15.163:0/1740729615 learned_addr learned my addr 172.21.15.163:0/1740729615 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:35.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.523+0000 7fdf9affd700 1 --2- 172.21.15.163:0/1740729615 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf9c1597f0 0x7fdf9c16c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:35.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.523+0000 7fdf9b7fe700 1 --2- 172.21.15.163:0/1740729615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf9c159420 0x7fdf9c16a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:35.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.523+0000 7fdf9bfff700 1 -- 172.21.15.163:0/1740729615 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf9c1597f0 msgr2=0x7fdf9c16c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:35.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.523+0000 7fdf9bfff700 1 --2- 172.21.15.163:0/1740729615 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf9c1597f0 0x7fdf9c16c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:35.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.523+0000 7fdf9bfff700 1 -- 172.21.15.163:0/1740729615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf9c159420 msgr2=0x7fdf9c16a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:35.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.523+0000 7fdf9bfff700 1 --2- 172.21.15.163:0/1740729615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf9c159420 0x7fdf9c16a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:35.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.523+0000 7fdf9bfff700 1 -- 172.21.15.163:0/1740729615 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf940094f0 con 0x7fdf9c15d230 2023-10-09T06:37:35.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.523+0000 7fdf9bfff700 1 --2- 172.21.15.163:0/1740729615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf9c15d230 0x7fdf9c15aac0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fdf94006fd0 tx=0x7fdf94017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:35.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.523+0000 7fdf837fe700 1 -- 172.21.15.163:0/1740729615 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94011420 con 0x7fdf9c15d230 2023-10-09T06:37:35.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.523+0000 7fdf837fe700 1 -- 172.21.15.163:0/1740729615 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf940115a0 con 0x7fdf9c15d230 2023-10-09T06:37:35.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.523+0000 7fdfa0b30700 1 -- 172.21.15.163:0/1740729615 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf9c15b7f0 con 0x7fdf9c15d230 2023-10-09T06:37:35.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.523+0000 7fdf837fe700 1 -- 172.21.15.163:0/1740729615 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94011960 con 0x7fdf9c15d230 2023-10-09T06:37:35.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.523+0000 7fdfa0b30700 1 -- 172.21.15.163:0/1740729615 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf9c15b510 con 0x7fdf9c15d230 2023-10-09T06:37:35.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.524+0000 7fdfa0b30700 1 -- 172.21.15.163:0/1740729615 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf9c0d1180 con 0x7fdf9c15d230 2023-10-09T06:37:35.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.524+0000 7fdf837fe700 1 -- 172.21.15.163:0/1740729615 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fdf94011d10 con 0x7fdf9c15d230 2023-10-09T06:37:35.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.525+0000 7fdf837fe700 1 --2- 172.21.15.163:0/1740729615 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdf88067320 0x7fdf880697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:35.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.525+0000 7fdf837fe700 1 -- 172.21.15.163:0/1740729615 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5877+0+0 (secure 0 0 0) 0x7fdf94086690 con 0x7fdf9c15d230 2023-10-09T06:37:35.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.525+0000 7fdf9b7fe700 1 --2- 172.21.15.163:0/1740729615 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdf88067320 0x7fdf880697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:35.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.525+0000 7fdf9b7fe700 1 --2- 172.21.15.163:0/1740729615 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdf88067320 0x7fdf880697d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c007fd0 tx=0x7fdf8c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:35.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.529+0000 7fdf837fe700 1 -- 172.21.15.163:0/1740729615 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdf94053270 con 0x7fdf9c15d230 2023-10-09T06:37:35.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.714+0000 7fdfa0b30700 1 -- 172.21.15.163:0/1740729615 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf9c0af210 con 0x7fdf9c15d230 2023-10-09T06:37:35.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.715+0000 7fdf837fe700 1 -- 172.21.15.163:0/1740729615 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+15675 (secure 0 0 0) 0x7fdf94056890 con 0x7fdf9c15d230 2023-10-09T06:37:35.718 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:35.718 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":125,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:37:31.984855+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:37:06.472811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-09T06:37:07.096712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6837/3075056851":"2023-10-10T06:37:28.977957+0000","172.21.15.181:6835/3075056851":"2023-10-10T06:37:28.977957+0000","172.21.15.163:6834/1355789800":"2023-10-10T06:37:25.586483+0000","172.21.15.163:6832/1355789800":"2023-10-10T06:37:25.586483+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:37:35.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.717+0000 7fdfa0b30700 1 -- 172.21.15.163:0/1740729615 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdf88067320 msgr2=0x7fdf880697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:35.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.717+0000 7fdfa0b30700 1 --2- 172.21.15.163:0/1740729615 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdf88067320 0x7fdf880697d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c007fd0 tx=0x7fdf8c009040 comp rx=0 tx=0).stop 2023-10-09T06:37:35.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.717+0000 7fdfa0b30700 1 -- 172.21.15.163:0/1740729615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf9c15d230 msgr2=0x7fdf9c15aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:35.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.717+0000 7fdfa0b30700 1 --2- 172.21.15.163:0/1740729615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf9c15d230 0x7fdf9c15aac0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fdf94006fd0 tx=0x7fdf94017000 comp rx=0 tx=0).stop 2023-10-09T06:37:35.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.717+0000 7fdfa0b30700 1 -- 172.21.15.163:0/1740729615 shutdown_connections 2023-10-09T06:37:35.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.717+0000 7fdfa0b30700 1 --2- 172.21.15.163:0/1740729615 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdf88067320 0x7fdf880697d0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:35.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.717+0000 7fdfa0b30700 1 --2- 172.21.15.163:0/1740729615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf9c15d230 0x7fdf9c15aac0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:35.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.717+0000 7fdfa0b30700 1 --2- 172.21.15.163:0/1740729615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf9c159420 0x7fdf9c16a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:35.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.717+0000 7fdfa0b30700 1 --2- 172.21.15.163:0/1740729615 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf9c1597f0 0x7fdf9c16c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:35.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.717+0000 7fdfa0b30700 1 -- 172.21.15.163:0/1740729615 >> 172.21.15.163:0/1740729615 conn(0x7fdf9c0d0660 msgr2=0x7fdf9c157220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:35.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.717+0000 7fdfa0b30700 1 -- 172.21.15.163:0/1740729615 shutdown_connections 2023-10-09T06:37:35.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.717+0000 7fdfa0b30700 1 -- 172.21.15.163:0/1740729615 wait complete. 2023-10-09T06:37:35.730 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-09T06:37:35.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.845+0000 7fbfce309700 1 Processor -- start 2023-10-09T06:37:35.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.845+0000 7fbfce309700 1 -- start start 2023-10-09T06:37:35.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.846+0000 7fbfce309700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbfc8159890 0x7fbfc815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:35.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.846+0000 7fbfce309700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfc8159c60 0x7fbfc815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:35.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.846+0000 7fbfce309700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc815a160 con 0x7fbfc815db10 2023-10-09T06:37:35.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.846+0000 7fbfce309700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc80d13a0 con 0x7fbfc8159890 2023-10-09T06:37:35.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.846+0000 7fbfce309700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc815e140 con 0x7fbfc8159c60 2023-10-09T06:37:35.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.846+0000 7fbfc77fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfc8159c60 0x7fbfc815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:35.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.846+0000 7fbfc7fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbfc8159890 0x7fbfc815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:35.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.846+0000 7fbfcc8a6700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbfc815db10 0x7fbfc8069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:49338/0 (socket says 172.21.15.163:49338) 2023-10-09T06:37:35.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.847+0000 7fbfcc8a6700 1 -- 172.21.15.163:0/1362895528 learned_addr learned my addr 172.21.15.163:0/1362895528 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:35.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.847+0000 7fbfc7fff700 1 -- 172.21.15.163:0/1362895528 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfc8159c60 msgr2=0x7fbfc815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:35.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.847+0000 7fbfc7fff700 1 --2- 172.21.15.163:0/1362895528 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfc8159c60 0x7fbfc815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:35.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.847+0000 7fbfc7fff700 1 -- 172.21.15.163:0/1362895528 >> v1:172.21.15.181:6789/0 conn(0x7fbfc815db10 legacy=0x7fbfc8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:35.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.847+0000 7fbfc7fff700 1 -- 172.21.15.163:0/1362895528 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc815a3d0 con 0x7fbfc8159890 2023-10-09T06:37:35.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.847+0000 7fbfc77fe700 1 --2- 172.21.15.163:0/1362895528 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfc8159c60 0x7fbfc815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:37:35.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.847+0000 7fbfc7fff700 1 --2- 172.21.15.163:0/1362895528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc8159890 0x7fbfc815abd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbfb80098e0 tx=0x7fbfb80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48e292cc029e2454 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:35.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.847+0000 7fbfc6ffd700 1 -- 172.21.15.163:0/1362895528 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb800b640 con 0x7fbfc8159890 2023-10-09T06:37:35.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.847+0000 7fbfc6ffd700 1 -- 172.21.15.163:0/1362895528 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb800b7a0 con 0x7fbfc8159890 2023-10-09T06:37:35.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.847+0000 7fbfc6ffd700 1 -- 172.21.15.163:0/1362895528 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb800bb40 con 0x7fbfc8159890 2023-10-09T06:37:35.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.848+0000 7fbfce309700 1 -- 172.21.15.163:0/1362895528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc8159890 msgr2=0x7fbfc815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:35.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.848+0000 7fbfce309700 1 --2- 172.21.15.163:0/1362895528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc8159890 0x7fbfc815abd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbfb80098e0 tx=0x7fbfb80091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:35.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.848+0000 7fbfce309700 1 -- 172.21.15.163:0/1362895528 shutdown_connections 2023-10-09T06:37:35.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.848+0000 7fbfce309700 1 --2- 172.21.15.163:0/1362895528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc8159890 0x7fbfc815abd0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:35.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.848+0000 7fbfce309700 1 --2- 172.21.15.163:0/1362895528 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfc8159c60 0x7fbfc815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:35.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.848+0000 7fbfce309700 1 -- 172.21.15.163:0/1362895528 >> 172.21.15.163:0/1362895528 conn(0x7fbfc8068e20 msgr2=0x7fbfc80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:35.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.848+0000 7fbfce309700 1 -- 172.21.15.163:0/1362895528 shutdown_connections 2023-10-09T06:37:35.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.848+0000 7fbfce309700 1 -- 172.21.15.163:0/1362895528 wait complete. 2023-10-09T06:37:35.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.849+0000 7fbfce309700 1 Processor -- start 2023-10-09T06:37:35.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.849+0000 7fbfce309700 1 -- start start 2023-10-09T06:37:35.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.850+0000 7fbfce309700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc8159890 0x7fbfc81494d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:35.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.850+0000 7fbfce309700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfc8159c60 0x7fbfc814eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:35.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.850+0000 7fbfce309700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfc815db10 0x7fbfc815cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:35.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.850+0000 7fbfce309700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc8150630 con 0x7fbfc815db10 2023-10-09T06:37:35.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.850+0000 7fbfce309700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc8150ac0 con 0x7fbfc8159890 2023-10-09T06:37:35.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.850+0000 7fbfce309700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc814f740 con 0x7fbfc8159c60 2023-10-09T06:37:35.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.850+0000 7fbfcc8a6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfc815db10 0x7fbfc815cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:35.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.850+0000 7fbfcc8a6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfc815db10 0x7fbfc815cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49438/0 (socket says 172.21.15.163:49438) 2023-10-09T06:37:35.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.850+0000 7fbfcc8a6700 1 -- 172.21.15.163:0/2236387948 learned_addr learned my addr 172.21.15.163:0/2236387948 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:35.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.850+0000 7fbfc77fe700 1 --2- 172.21.15.163:0/2236387948 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfc8159c60 0x7fbfc814eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:35.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.850+0000 7fbfc7fff700 1 --2- 172.21.15.163:0/2236387948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc8159890 0x7fbfc81494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:35.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.850+0000 7fbfcc8a6700 1 -- 172.21.15.163:0/2236387948 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfc8159c60 msgr2=0x7fbfc814eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:35.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.850+0000 7fbfcc8a6700 1 --2- 172.21.15.163:0/2236387948 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfc8159c60 0x7fbfc814eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:35.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.850+0000 7fbfcc8a6700 1 -- 172.21.15.163:0/2236387948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc8159890 msgr2=0x7fbfc81494d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:35.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.850+0000 7fbfcc8a6700 1 --2- 172.21.15.163:0/2236387948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc8159890 0x7fbfc81494d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:35.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.850+0000 7fbfcc8a6700 1 -- 172.21.15.163:0/2236387948 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfbc009420 con 0x7fbfc815db10 2023-10-09T06:37:35.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.850+0000 7fbfcc8a6700 1 --2- 172.21.15.163:0/2236387948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfc815db10 0x7fbfc815cd20 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000cbc0 tx=0x7fbfc00019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:35.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.851+0000 7fbfaffff700 1 -- 172.21.15.163:0/2236387948 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc000a7e0 con 0x7fbfc815db10 2023-10-09T06:37:35.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.851+0000 7fbfaffff700 1 -- 172.21.15.163:0/2236387948 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc000a940 con 0x7fbfc815db10 2023-10-09T06:37:35.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.851+0000 7fbfce309700 1 -- 172.21.15.163:0/2236387948 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb8009420 con 0x7fbfc815db10 2023-10-09T06:37:35.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.851+0000 7fbfaffff700 1 -- 172.21.15.163:0/2236387948 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc000acc0 con 0x7fbfc815db10 2023-10-09T06:37:35.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.851+0000 7fbfce309700 1 -- 172.21.15.163:0/2236387948 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc814fd00 con 0x7fbfc815db10 2023-10-09T06:37:35.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.851+0000 7fbfce309700 1 -- 172.21.15.163:0/2236387948 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc81507e0 con 0x7fbfc815db10 2023-10-09T06:37:35.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.852+0000 7fbfaffff700 1 -- 172.21.15.163:0/2236387948 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbfc0007300 con 0x7fbfc815db10 2023-10-09T06:37:35.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.852+0000 7fbfaffff700 1 --2- 172.21.15.163:0/2236387948 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbfb4067320 0x7fbfb40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:35.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.852+0000 7fbfaffff700 1 -- 172.21.15.163:0/2236387948 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5877+0+0 (secure 0 0 0) 0x7fbfc0084f70 con 0x7fbfc815db10 2023-10-09T06:37:35.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.853+0000 7fbfc7fff700 1 --2- 172.21.15.163:0/2236387948 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbfb4067320 0x7fbfb40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:35.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.853+0000 7fbfc7fff700 1 --2- 172.21.15.163:0/2236387948 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbfb4067320 0x7fbfb40697d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbfb80093f0 tx=0x7fbfb8017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:35.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:35.856+0000 7fbfaffff700 1 -- 172.21.15.163:0/2236387948 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbfc81507e0 con 0x7fbfc815db10 2023-10-09T06:37:36.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.080+0000 7fbfce309700 1 -- 172.21.15.163:0/2236387948 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbfc804c3c0 con 0x7fbfc815db10 2023-10-09T06:37:36.081 INFO:tasks.ceph.mon.a.smithi163.stderr:2023-10-09T06:37:36.081+0000 7ff251c04700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-09T06:37:36.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.084+0000 7fbfaffff700 1 -- 172.21.15.163:0/2236387948 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v86) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbfc00555d0 con 0x7fbfc815db10 2023-10-09T06:37:36.086 INFO:tasks.ceph.mds.c.smithi163.stderr: -1393> 2023-10-09T06:37:14.536+0000 7fbd9d0b0700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-09T06:37:36.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.087+0000 7fbfce309700 1 -- 172.21.15.163:0/2236387948 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbfb4067320 msgr2=0x7fbfb40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:36.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.087+0000 7fbfce309700 1 --2- 172.21.15.163:0/2236387948 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbfb4067320 0x7fbfb40697d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbfb80093f0 tx=0x7fbfb8017000 comp rx=0 tx=0).stop 2023-10-09T06:37:36.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.087+0000 7fbfce309700 1 -- 172.21.15.163:0/2236387948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfc815db10 msgr2=0x7fbfc815cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:36.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.087+0000 7fbfce309700 1 --2- 172.21.15.163:0/2236387948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfc815db10 0x7fbfc815cd20 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000cbc0 tx=0x7fbfc00019c0 comp rx=0 tx=0).stop 2023-10-09T06:37:36.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.087+0000 7fbfce309700 1 -- 172.21.15.163:0/2236387948 shutdown_connections 2023-10-09T06:37:36.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.087+0000 7fbfce309700 1 --2- 172.21.15.163:0/2236387948 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbfb4067320 0x7fbfb40697d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:36.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.087+0000 7fbfce309700 1 --2- 172.21.15.163:0/2236387948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfc815db10 0x7fbfc815cd20 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:36.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.087+0000 7fbfce309700 1 --2- 172.21.15.163:0/2236387948 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfc8159890 0x7fbfc81494d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:36.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.087+0000 7fbfce309700 1 --2- 172.21.15.163:0/2236387948 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfc8159c60 0x7fbfc814eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:36.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.087+0000 7fbfce309700 1 -- 172.21.15.163:0/2236387948 >> 172.21.15.163:0/2236387948 conn(0x7fbfc8068e20 msgr2=0x7fbfc814a300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:36.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.087+0000 7fbfce309700 1 -- 172.21.15.163:0/2236387948 shutdown_connections 2023-10-09T06:37:36.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.087+0000 7fbfce309700 1 -- 172.21.15.163:0/2236387948 wait complete. 2023-10-09T06:37:36.089 INFO:teuthology.orchestra.run.smithi163.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-09T06:37:36.104 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-09T06:37:36.111 INFO:tasks.ceph.mds.c.smithi163.stdout:starting mds.c at 2023-10-09T06:37:36.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.219+0000 7fd464b49700 1 Processor -- start 2023-10-09T06:37:36.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.219+0000 7fd464b49700 1 -- start start 2023-10-09T06:37:36.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.220+0000 7fd464b49700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd460159890 0x7fd46015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:36.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.220+0000 7fd464b49700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd46015a160 con 0x7fd460069560 2023-10-09T06:37:36.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.220+0000 7fd464b49700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4600d13a0 con 0x7fd460159c60 2023-10-09T06:37:36.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.220+0000 7fd464b49700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd46015e140 con 0x7fd460159890 2023-10-09T06:37:36.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.220+0000 7fd45ed9d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fd460069560 0x7fd4600ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:52766/0 (socket says 172.21.15.163:52766) 2023-10-09T06:37:36.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.220+0000 7fd45ed9d700 1 -- 172.21.15.163:0/1344898834 learned_addr learned my addr 172.21.15.163:0/1344898834 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:36.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.220+0000 7fd45e59c700 1 --2- 172.21.15.163:0/1344898834 >> v2:172.21.15.181:3300/0 conn(0x7fd460159890 0x7fd46015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:36.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.220+0000 7fd45dd9b700 1 -- 172.21.15.163:0/1344898834 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2302218554 0 0) 0x7fd46015a160 con 0x7fd460069560 2023-10-09T06:37:36.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.220+0000 7fd45dd9b700 1 -- 172.21.15.163:0/1344898834 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd44c003570 con 0x7fd460069560 2023-10-09T06:37:36.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.221+0000 7fd45dd9b700 1 -- 172.21.15.163:0/1344898834 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2692874530 0 0) 0x7fd44c003570 con 0x7fd460069560 2023-10-09T06:37:36.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.221+0000 7fd45dd9b700 1 -- 172.21.15.163:0/1344898834 >> v2:172.21.15.181:3300/0 conn(0x7fd460159890 msgr2=0x7fd46015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:36.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.221+0000 7fd45dd9b700 1 --2- 172.21.15.163:0/1344898834 >> v2:172.21.15.181:3300/0 conn(0x7fd460159890 0x7fd46015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:36.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.221+0000 7fd45dd9b700 1 -- 172.21.15.163:0/1344898834 >> v1:172.21.15.181:6789/0 conn(0x7fd460159c60 legacy=0x7fd46015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:36.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.221+0000 7fd45dd9b700 1 -- 172.21.15.163:0/1344898834 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46015a3d0 con 0x7fd460069560 2023-10-09T06:37:36.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.221+0000 7fd45dd9b700 1 -- 172.21.15.163:0/1344898834 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fd4580017f0 con 0x7fd460069560 2023-10-09T06:37:36.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.221+0000 7fd45dd9b700 1 -- 172.21.15.163:0/1344898834 >> v1:172.21.15.163:6789/0 conn(0x7fd460069560 legacy=0x7fd4600ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:36.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.221+0000 7fd45dd9b700 1 --2- 172.21.15.163:0/1344898834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd44c003a60 0x7fd44c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:36.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.221+0000 7fd45dd9b700 1 --2- 172.21.15.163:0/1344898834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd44c004810 0x7fd44c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:36.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.221+0000 7fd45dd9b700 1 --2- 172.21.15.163:0/1344898834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd44c0051c0 0x7fd44c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:36.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.221+0000 7fd45dd9b700 1 -- 172.21.15.163:0/1344898834 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46015e140 con 0x7fd44c003a60 2023-10-09T06:37:36.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.221+0000 7fd45dd9b700 1 -- 172.21.15.163:0/1344898834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c00c0d0 con 0x7fd44c004810 2023-10-09T06:37:36.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.221+0000 7fd45dd9b700 1 -- 172.21.15.163:0/1344898834 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c00c290 con 0x7fd44c0051c0 2023-10-09T06:37:36.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.221+0000 7fd45e59c700 1 --2- 172.21.15.163:0/1344898834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd44c003a60 0x7fd44c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:36.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.221+0000 7fd455d9b700 1 --2- 172.21.15.163:0/1344898834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd44c004810 0x7fd44c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:36.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.221+0000 7fd45ed9d700 1 --2- 172.21.15.163:0/1344898834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd44c0051c0 0x7fd44c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:36.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.221+0000 7fd45e59c700 1 -- 172.21.15.163:0/1344898834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd44c0051c0 msgr2=0x7fd44c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:36.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.221+0000 7fd45e59c700 1 --2- 172.21.15.163:0/1344898834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd44c0051c0 0x7fd44c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:36.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.221+0000 7fd45e59c700 1 -- 172.21.15.163:0/1344898834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd44c004810 msgr2=0x7fd44c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:36.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.221+0000 7fd45e59c700 1 --2- 172.21.15.163:0/1344898834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd44c004810 0x7fd44c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:36.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.221+0000 7fd45ed9d700 1 -- 172.21.15.163:0/1344898834 reap_dead start 2023-10-09T06:37:36.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.221+0000 7fd45e59c700 1 -- 172.21.15.163:0/1344898834 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44c003570 con 0x7fd44c003a60 2023-10-09T06:37:36.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.221+0000 7fd45e59c700 1 --2- 172.21.15.163:0/1344898834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd44c003a60 0x7fd44c003e70 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fd450009c00 tx=0x7fd4500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1745d1bb78296a server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:36.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.221+0000 7fd45dd9b700 1 -- 172.21.15.163:0/1344898834 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45000e710 con 0x7fd44c003a60 2023-10-09T06:37:36.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.221+0000 7fd45dd9b700 1 -- 172.21.15.163:0/1344898834 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45000e870 con 0x7fd44c003a60 2023-10-09T06:37:36.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.221+0000 7fd45dd9b700 1 -- 172.21.15.163:0/1344898834 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45000ec30 con 0x7fd44c003a60 2023-10-09T06:37:36.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.222+0000 7fd464b49700 1 -- 172.21.15.163:0/1344898834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd44c003a60 msgr2=0x7fd44c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:36.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.222+0000 7fd464b49700 1 --2- 172.21.15.163:0/1344898834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd44c003a60 0x7fd44c003e70 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fd450009c00 tx=0x7fd4500091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:36.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.222+0000 7fd464b49700 1 -- 172.21.15.163:0/1344898834 shutdown_connections 2023-10-09T06:37:36.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.222+0000 7fd464b49700 1 --2- 172.21.15.163:0/1344898834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd44c003a60 0x7fd44c003e70 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:36.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.222+0000 7fd464b49700 1 -- 172.21.15.163:0/1344898834 >> 172.21.15.163:0/1344898834 conn(0x7fd460068e20 msgr2=0x7fd4600d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:36.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.222+0000 7fd464b49700 1 -- 172.21.15.163:0/1344898834 shutdown_connections 2023-10-09T06:37:36.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.222+0000 7fd464b49700 1 -- 172.21.15.163:0/1344898834 wait complete. 2023-10-09T06:37:36.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.223+0000 7fd464b49700 1 Processor -- start 2023-10-09T06:37:36.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.223+0000 7fd464b49700 1 -- start start 2023-10-09T06:37:36.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.224+0000 7fd464b49700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd44c003a60 0x7fd4600cf860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:36.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.224+0000 7fd464b49700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd460149310 0x7fd4600cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:36.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.224+0000 7fd464b49700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4600b1c30 0x7fd4600b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:36.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.224+0000 7fd464b49700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4600b2610 con 0x7fd460149310 2023-10-09T06:37:36.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.224+0000 7fd464b49700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4600ce360 con 0x7fd4600b1c30 2023-10-09T06:37:36.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.224+0000 7fd464b49700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4601566d0 con 0x7fd44c003a60 2023-10-09T06:37:36.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.224+0000 7fd455d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd460149310 0x7fd4600cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:36.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.224+0000 7fd455d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd460149310 0x7fd4600cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49482/0 (socket says 172.21.15.163:49482) 2023-10-09T06:37:36.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.224+0000 7fd455d9b700 1 -- 172.21.15.163:0/13460212 learned_addr learned my addr 172.21.15.163:0/13460212 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:36.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.224+0000 7fd45e59c700 1 --2- 172.21.15.163:0/13460212 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd44c003a60 0x7fd4600cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:36.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.224+0000 7fd45ed9d700 1 --2- 172.21.15.163:0/13460212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4600b1c30 0x7fd4600b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:36.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.224+0000 7fd455d9b700 1 -- 172.21.15.163:0/13460212 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd44c003a60 msgr2=0x7fd4600cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:36.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.224+0000 7fd455d9b700 1 --2- 172.21.15.163:0/13460212 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd44c003a60 0x7fd4600cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:36.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.224+0000 7fd455d9b700 1 -- 172.21.15.163:0/13460212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4600b1c30 msgr2=0x7fd4600b20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:36.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.224+0000 7fd455d9b700 1 --2- 172.21.15.163:0/13460212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4600b1c30 0x7fd4600b20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:36.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.224+0000 7fd455d9b700 1 -- 172.21.15.163:0/13460212 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd460155da0 con 0x7fd460149310 2023-10-09T06:37:36.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.224+0000 7fd455d9b700 1 --2- 172.21.15.163:0/13460212 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd460149310 0x7fd4600cdaa0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fd448002950 tx=0x7fd448007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:36.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.225+0000 7fd456ffd700 1 -- 172.21.15.163:0/13460212 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd448011590 con 0x7fd460149310 2023-10-09T06:37:36.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.225+0000 7fd456ffd700 1 -- 172.21.15.163:0/13460212 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4480116f0 con 0x7fd460149310 2023-10-09T06:37:36.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.225+0000 7fd456ffd700 1 -- 172.21.15.163:0/13460212 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd448011a90 con 0x7fd460149310 2023-10-09T06:37:36.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.225+0000 7fd464b49700 1 -- 172.21.15.163:0/13460212 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd460156080 con 0x7fd460149310 2023-10-09T06:37:36.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.225+0000 7fd464b49700 1 -- 172.21.15.163:0/13460212 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4600cdfe0 con 0x7fd460149310 2023-10-09T06:37:36.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.226+0000 7fd464b49700 1 -- 172.21.15.163:0/13460212 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd460156880 con 0x7fd460149310 2023-10-09T06:37:36.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.226+0000 7fd456ffd700 1 -- 172.21.15.163:0/13460212 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd44801d460 con 0x7fd460149310 2023-10-09T06:37:36.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.226+0000 7fd456ffd700 1 --2- 172.21.15.163:0/13460212 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd440067320 0x7fd4400697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:36.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.226+0000 7fd456ffd700 1 -- 172.21.15.163:0/13460212 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5963+0+0 (secure 0 0 0) 0x7fd448084e00 con 0x7fd460149310 2023-10-09T06:37:36.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.227+0000 7fd45e59c700 1 --2- 172.21.15.163:0/13460212 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd440067320 0x7fd4400697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:36.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.227+0000 7fd45e59c700 1 --2- 172.21.15.163:0/13460212 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd440067320 0x7fd4400697d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd4500091e0 tx=0x7fd45001a000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:36.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.230+0000 7fd456ffd700 1 -- 172.21.15.163:0/13460212 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd460156880 con 0x7fd460149310 2023-10-09T06:37:36.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:36.457+0000 7fd464b49700 1 -- 172.21.15.163:0/13460212 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd460156840 con 0x7fd460149310 2023-10-09T06:37:37.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.087+0000 7fd456ffd700 1 -- 172.21.15.163:0/13460212 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v87) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd448051990 con 0x7fd460149310 2023-10-09T06:37:37.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.089+0000 7fd464b49700 1 -- 172.21.15.163:0/13460212 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd440067320 msgr2=0x7fd4400697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:37.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.089+0000 7fd464b49700 1 --2- 172.21.15.163:0/13460212 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd440067320 0x7fd4400697d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd4500091e0 tx=0x7fd45001a000 comp rx=0 tx=0).stop 2023-10-09T06:37:37.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.089+0000 7fd464b49700 1 -- 172.21.15.163:0/13460212 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd460149310 msgr2=0x7fd4600cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:37.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.089+0000 7fd464b49700 1 --2- 172.21.15.163:0/13460212 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd460149310 0x7fd4600cdaa0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fd448002950 tx=0x7fd448007d30 comp rx=0 tx=0).stop 2023-10-09T06:37:37.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.089+0000 7fd464b49700 1 -- 172.21.15.163:0/13460212 shutdown_connections 2023-10-09T06:37:37.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.089+0000 7fd464b49700 1 --2- 172.21.15.163:0/13460212 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd440067320 0x7fd4400697d0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:37.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.089+0000 7fd464b49700 1 --2- 172.21.15.163:0/13460212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4600b1c30 0x7fd4600b20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:37.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.089+0000 7fd464b49700 1 --2- 172.21.15.163:0/13460212 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd44c003a60 0x7fd4600cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:37.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.089+0000 7fd464b49700 1 --2- 172.21.15.163:0/13460212 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd460149310 0x7fd4600cdaa0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:37.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.089+0000 7fd464b49700 1 -- 172.21.15.163:0/13460212 >> 172.21.15.163:0/13460212 conn(0x7fd460068e20 msgr2=0x7fd460149c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:37.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.090+0000 7fd464b49700 1 -- 172.21.15.163:0/13460212 shutdown_connections 2023-10-09T06:37:37.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.090+0000 7fd464b49700 1 -- 172.21.15.163:0/13460212 wait complete. 2023-10-09T06:37:37.110 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-09T06:37:37.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.269+0000 7f25df890700 1 Processor -- start 2023-10-09T06:37:37.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.270+0000 7f25df890700 1 -- start start 2023-10-09T06:37:37.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.270+0000 7f25df890700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25d8159ec0 0x7f25d80b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:37.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.270+0000 7f25df890700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f25d80b53d0 0x7f25d80cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:37.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.270+0000 7f25df890700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25d80b5b00 con 0x7f25d80cce40 2023-10-09T06:37:37.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.270+0000 7f25df890700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25d80b5da0 con 0x7f25d80b53d0 2023-10-09T06:37:37.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.270+0000 7f25df890700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d80b6080 con 0x7f25d8159ec0 2023-10-09T06:37:37.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.271+0000 7f25dce2b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f25d80b53d0 0x7f25d80cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:37.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.271+0000 7f25dce2b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f25d80b53d0 0x7f25d80cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:49732/0 (socket says 172.21.15.163:49732) 2023-10-09T06:37:37.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.271+0000 7f25dce2b700 1 -- 172.21.15.163:0/3193768372 learned_addr learned my addr 172.21.15.163:0/3193768372 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:37.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.271+0000 7f25dd62c700 1 --2- 172.21.15.163:0/3193768372 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25d8159ec0 0x7f25d80b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:37.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.271+0000 7f25dce2b700 1 -- 172.21.15.163:0/3193768372 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25d8159ec0 msgr2=0x7f25d80b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:37.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.271+0000 7f25dce2b700 1 --2- 172.21.15.163:0/3193768372 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25d8159ec0 0x7f25d80b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:37.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.271+0000 7f25dce2b700 1 -- 172.21.15.163:0/3193768372 >> v1:172.21.15.181:6789/0 conn(0x7f25d80cce40 legacy=0x7f25d80cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:37.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.271+0000 7f25cffff700 1 -- 172.21.15.163:0/3193768372 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 366242192 0 0) 0x7f25d80b5b00 con 0x7f25d80cce40 2023-10-09T06:37:37.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.271+0000 7f25dce2b700 1 -- 172.21.15.163:0/3193768372 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d80cc100 con 0x7f25d80b53d0 2023-10-09T06:37:37.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.271+0000 7f25dce2b700 1 --2- 172.21.15.163:0/3193768372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25d80b53d0 0x7f25d80cc900 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f25c80099b0 tx=0x7f25c8009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5e96cbf6821e395 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:37.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.272+0000 7f25cffff700 1 -- 172.21.15.163:0/3193768372 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c8009ea0 con 0x7f25d80b53d0 2023-10-09T06:37:37.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.272+0000 7f25cffff700 1 -- 172.21.15.163:0/3193768372 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c800b860 con 0x7f25d80b53d0 2023-10-09T06:37:37.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.272+0000 7f25cffff700 1 -- 172.21.15.163:0/3193768372 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c800bc00 con 0x7f25d80b53d0 2023-10-09T06:37:37.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.272+0000 7f25df890700 1 -- 172.21.15.163:0/3193768372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25d80b53d0 msgr2=0x7f25d80cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:37.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.272+0000 7f25df890700 1 --2- 172.21.15.163:0/3193768372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25d80b53d0 0x7f25d80cc900 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f25c80099b0 tx=0x7f25c8009250 comp rx=0 tx=0).stop 2023-10-09T06:37:37.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.272+0000 7f25df890700 1 -- 172.21.15.163:0/3193768372 shutdown_connections 2023-10-09T06:37:37.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.272+0000 7f25df890700 1 --2- 172.21.15.163:0/3193768372 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25d8159ec0 0x7f25d80b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:37.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.272+0000 7f25df890700 1 --2- 172.21.15.163:0/3193768372 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25d80b53d0 0x7f25d80cc900 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:37.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.272+0000 7f25df890700 1 -- 172.21.15.163:0/3193768372 >> 172.21.15.163:0/3193768372 conn(0x7f25d80999f0 msgr2=0x7f25d80b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:37.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.272+0000 7f25df890700 1 -- 172.21.15.163:0/3193768372 shutdown_connections 2023-10-09T06:37:37.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.272+0000 7f25df890700 1 -- 172.21.15.163:0/3193768372 wait complete. 2023-10-09T06:37:37.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.273+0000 7f25df890700 1 Processor -- start 2023-10-09T06:37:37.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.273+0000 7f25df890700 1 -- start start 2023-10-09T06:37:37.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.274+0000 7f25df890700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25d80b53d0 0x7f25d8144b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:37.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.274+0000 7f25df890700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25d80cce40 0x7f25d8146890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:37.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.274+0000 7f25df890700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25d8159ec0 0x7f25d8146dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:37.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.274+0000 7f25df890700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d8147660 con 0x7f25d80cce40 2023-10-09T06:37:37.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.274+0000 7f25df890700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d8147940 con 0x7f25d80b53d0 2023-10-09T06:37:37.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.274+0000 7f25df890700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d8147c20 con 0x7f25d8159ec0 2023-10-09T06:37:37.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.274+0000 7f25dce2b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25d80cce40 0x7f25d8146890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:37.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.274+0000 7f25dce2b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25d80cce40 0x7f25d8146890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49496/0 (socket says 172.21.15.163:49496) 2023-10-09T06:37:37.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.274+0000 7f25dce2b700 1 -- 172.21.15.163:0/494256445 learned_addr learned my addr 172.21.15.163:0/494256445 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:37.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.274+0000 7f25dd62c700 1 --2- 172.21.15.163:0/494256445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25d80b53d0 0x7f25d8144b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:37.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.274+0000 7f25dde2d700 1 --2- 172.21.15.163:0/494256445 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25d8159ec0 0x7f25d8146dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:37.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.275+0000 7f25dce2b700 1 -- 172.21.15.163:0/494256445 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25d8159ec0 msgr2=0x7f25d8146dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:37.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.275+0000 7f25dce2b700 1 --2- 172.21.15.163:0/494256445 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25d8159ec0 0x7f25d8146dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:37.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.275+0000 7f25dce2b700 1 -- 172.21.15.163:0/494256445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25d80b53d0 msgr2=0x7f25d8144b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:37.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.275+0000 7f25dce2b700 1 --2- 172.21.15.163:0/494256445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25d80b53d0 0x7f25d8144b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:37.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.275+0000 7f25dce2b700 1 -- 172.21.15.163:0/494256445 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c80094f0 con 0x7f25d80cce40 2023-10-09T06:37:37.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.275+0000 7f25dce2b700 1 --2- 172.21.15.163:0/494256445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25d80cce40 0x7f25d8146890 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f25d8089390 tx=0x7f25c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:37.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.275+0000 7f25cd7fa700 1 -- 172.21.15.163:0/494256445 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c8000d10 con 0x7f25d80cce40 2023-10-09T06:37:37.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.275+0000 7f25cd7fa700 1 -- 172.21.15.163:0/494256445 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c800bbe0 con 0x7f25d80cce40 2023-10-09T06:37:37.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.275+0000 7f25cd7fa700 1 -- 172.21.15.163:0/494256445 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c8011690 con 0x7f25d80cce40 2023-10-09T06:37:37.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.275+0000 7f25df890700 1 -- 172.21.15.163:0/494256445 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d8147f00 con 0x7f25d80cce40 2023-10-09T06:37:37.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.275+0000 7f25df890700 1 -- 172.21.15.163:0/494256445 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d8145080 con 0x7f25d80cce40 2023-10-09T06:37:37.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.276+0000 7f25df890700 1 -- 172.21.15.163:0/494256445 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25d8063c90 con 0x7f25d80cce40 2023-10-09T06:37:37.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.276+0000 7f25cd7fa700 1 -- 172.21.15.163:0/494256445 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f25c8011ac0 con 0x7f25d80cce40 2023-10-09T06:37:37.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.277+0000 7f25cd7fa700 1 --2- 172.21.15.163:0/494256445 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f25bc067320 0x7f25bc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:37.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.277+0000 7f25cd7fa700 1 -- 172.21.15.163:0/494256445 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f25c8086600 con 0x7f25d80cce40 2023-10-09T06:37:37.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.277+0000 7f25dd62c700 1 --2- 172.21.15.163:0/494256445 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f25bc067320 0x7f25bc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:37.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.277+0000 7f25dd62c700 1 --2- 172.21.15.163:0/494256445 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f25bc067320 0x7f25bc0697d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f25d4009d00 tx=0x7f25d4009120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:37.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.281+0000 7f25cd7fa700 1 -- 172.21.15.163:0/494256445 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25c8053190 con 0x7f25d80cce40 2023-10-09T06:37:37.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:37.464+0000 7f25df890700 1 -- 172.21.15.163:0/494256445 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f25d80718b0 con 0x7f25d80cce40 2023-10-09T06:37:38.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.088+0000 7f25cd7fa700 1 -- 172.21.15.163:0/494256445 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v127) v1 ==== 181+0+0 (secure 0 0 0) 0x7f25c80567b0 con 0x7f25d80cce40 2023-10-09T06:37:38.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.092+0000 7f25df890700 1 -- 172.21.15.163:0/494256445 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f25bc067320 msgr2=0x7f25bc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:38.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.092+0000 7f25df890700 1 --2- 172.21.15.163:0/494256445 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f25bc067320 0x7f25bc0697d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f25d4009d00 tx=0x7f25d4009120 comp rx=0 tx=0).stop 2023-10-09T06:37:38.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.092+0000 7f25df890700 1 -- 172.21.15.163:0/494256445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25d80cce40 msgr2=0x7f25d8146890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:38.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.092+0000 7f25df890700 1 --2- 172.21.15.163:0/494256445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25d80cce40 0x7f25d8146890 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f25d8089390 tx=0x7f25c8017000 comp rx=0 tx=0).stop 2023-10-09T06:37:38.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.092+0000 7f25df890700 1 -- 172.21.15.163:0/494256445 shutdown_connections 2023-10-09T06:37:38.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.092+0000 7f25df890700 1 --2- 172.21.15.163:0/494256445 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f25bc067320 0x7f25bc0697d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:38.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.092+0000 7f25df890700 1 --2- 172.21.15.163:0/494256445 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25d8159ec0 0x7f25d8146dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:38.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.092+0000 7f25df890700 1 --2- 172.21.15.163:0/494256445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25d80b53d0 0x7f25d8144b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:38.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.092+0000 7f25df890700 1 --2- 172.21.15.163:0/494256445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25d80cce40 0x7f25d8146890 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:38.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.092+0000 7f25df890700 1 -- 172.21.15.163:0/494256445 >> 172.21.15.163:0/494256445 conn(0x7f25d80999f0 msgr2=0x7f25d80ce320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:38.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.092+0000 7f25df890700 1 -- 172.21.15.163:0/494256445 shutdown_connections 2023-10-09T06:37:38.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.092+0000 7f25df890700 1 -- 172.21.15.163:0/494256445 wait complete. 2023-10-09T06:37:38.096 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_metadata' removed 2023-10-09T06:37:38.108 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-09T06:37:38.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.234+0000 7f3ddffff700 1 Processor -- start 2023-10-09T06:37:38.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.234+0000 7f3ddffff700 1 -- start start 2023-10-09T06:37:38.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.235+0000 7f3ddffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3de00b03c0 0x7f3de00cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:38.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.235+0000 7f3ddffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3de00b0790 0x7f3de00b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:38.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.235+0000 7f3ddffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3de01591e0 0x7f3de0159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:38.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.235+0000 7f3ddffff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de00cfa00 con 0x7f3de00b03c0 2023-10-09T06:37:38.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.235+0000 7f3ddffff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de00cb580 con 0x7f3de00b0790 2023-10-09T06:37:38.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.235+0000 7f3ddffff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de00b2de0 con 0x7f3de01591e0 2023-10-09T06:37:38.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.235+0000 7f3ddeffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3de00b03c0 0x7f3de00cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:38.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.235+0000 7f3ddeffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3de00b03c0 0x7f3de00cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49502/0 (socket says 172.21.15.163:49502) 2023-10-09T06:37:38.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.235+0000 7f3ddeffd700 1 -- 172.21.15.163:0/2849825895 learned_addr learned my addr 172.21.15.163:0/2849825895 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:38.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.235+0000 7f3ddeffd700 1 -- 172.21.15.163:0/2849825895 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3de01591e0 msgr2=0x7f3de0159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:38.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.235+0000 7f3ddeffd700 1 --2- 172.21.15.163:0/2849825895 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3de01591e0 0x7f3de0159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:38.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.235+0000 7f3ddeffd700 1 -- 172.21.15.163:0/2849825895 >> v2:172.21.15.181:3300/0 conn(0x7f3de00b0790 msgr2=0x7f3de00b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:38.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.235+0000 7f3dde7fc700 1 --2- 172.21.15.163:0/2849825895 >> v2:172.21.15.181:3300/0 conn(0x7f3de00b0790 0x7f3de00b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:38.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.235+0000 7f3ddeffd700 1 --2- 172.21.15.163:0/2849825895 >> v2:172.21.15.181:3300/0 conn(0x7f3de00b0790 0x7f3de00b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:38.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.235+0000 7f3ddeffd700 1 -- 172.21.15.163:0/2849825895 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de00cf160 con 0x7f3de00b03c0 2023-10-09T06:37:38.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.236+0000 7f3ddeffd700 1 --2- 172.21.15.163:0/2849825895 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3de00b03c0 0x7f3de00cea70 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0009f20 tx=0x7f3dd0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2045eac11b574331 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:38.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.236+0000 7f3dddffb700 1 -- 172.21.15.163:0/2849825895 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd000e070 con 0x7f3de00b03c0 2023-10-09T06:37:38.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.236+0000 7f3dddffb700 1 -- 172.21.15.163:0/2849825895 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd000ba40 con 0x7f3de00b03c0 2023-10-09T06:37:38.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.236+0000 7f3dddffb700 1 -- 172.21.15.163:0/2849825895 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd00092e0 con 0x7f3de00b03c0 2023-10-09T06:37:38.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.236+0000 7f3ddffff700 1 -- 172.21.15.163:0/2849825895 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3de00b03c0 msgr2=0x7f3de00cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:38.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.236+0000 7f3ddffff700 1 --2- 172.21.15.163:0/2849825895 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3de00b03c0 0x7f3de00cea70 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0009f20 tx=0x7f3dd0009840 comp rx=0 tx=0).stop 2023-10-09T06:37:38.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.236+0000 7f3ddffff700 1 -- 172.21.15.163:0/2849825895 shutdown_connections 2023-10-09T06:37:38.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.236+0000 7f3ddffff700 1 --2- 172.21.15.163:0/2849825895 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3de01591e0 0x7f3de0159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:38.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.236+0000 7f3ddffff700 1 --2- 172.21.15.163:0/2849825895 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3de00b03c0 0x7f3de00cea70 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:38.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.236+0000 7f3ddffff700 1 --2- 172.21.15.163:0/2849825895 >> v2:172.21.15.181:3300/0 conn(0x7f3de00b0790 0x7f3de00b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:38.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.236+0000 7f3ddffff700 1 -- 172.21.15.163:0/2849825895 >> 172.21.15.163:0/2849825895 conn(0x7f3de0063230 msgr2=0x7f3de0063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:38.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.236+0000 7f3ddffff700 1 -- 172.21.15.163:0/2849825895 shutdown_connections 2023-10-09T06:37:38.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.236+0000 7f3ddffff700 1 -- 172.21.15.163:0/2849825895 wait complete. 2023-10-09T06:37:38.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.238+0000 7f3ddffff700 1 Processor -- start 2023-10-09T06:37:38.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.238+0000 7f3ddffff700 1 -- start start 2023-10-09T06:37:38.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.238+0000 7f3ddffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3de00b03c0 0x7f3de0156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:38.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.238+0000 7f3ddffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3de00b0790 0x7f3de00c9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:38.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.238+0000 7f3ddffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3de01591e0 0x7f3de0169d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:38.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.238+0000 7f3ddffff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de015acb0 con 0x7f3de01591e0 2023-10-09T06:37:38.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.238+0000 7f3ddffff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de015af90 con 0x7f3de00b0790 2023-10-09T06:37:38.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.238+0000 7f3ddffff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de015b270 con 0x7f3de00b03c0 2023-10-09T06:37:38.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.238+0000 7f3ddf7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3de01591e0 0x7f3de0169d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:38.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.238+0000 7f3ddf7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3de01591e0 0x7f3de0169d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49512/0 (socket says 172.21.15.163:49512) 2023-10-09T06:37:38.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.238+0000 7f3ddf7fe700 1 -- 172.21.15.163:0/600362111 learned_addr learned my addr 172.21.15.163:0/600362111 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:38.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.239+0000 7f3ddeffd700 1 --2- 172.21.15.163:0/600362111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3de00b03c0 0x7f3de0156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:38.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.239+0000 7f3dde7fc700 1 --2- 172.21.15.163:0/600362111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3de00b0790 0x7f3de00c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:38.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.239+0000 7f3ddf7fe700 1 -- 172.21.15.163:0/600362111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3de00b03c0 msgr2=0x7f3de0156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:38.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.239+0000 7f3ddf7fe700 1 --2- 172.21.15.163:0/600362111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3de00b03c0 0x7f3de0156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:38.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.239+0000 7f3ddf7fe700 1 -- 172.21.15.163:0/600362111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3de00b0790 msgr2=0x7f3de00c9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:38.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.239+0000 7f3ddf7fe700 1 --2- 172.21.15.163:0/600362111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3de00b0790 0x7f3de00c9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:38.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.239+0000 7f3ddf7fe700 1 -- 172.21.15.163:0/600362111 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd0009ae0 con 0x7f3de01591e0 2023-10-09T06:37:38.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.239+0000 7f3ddeffd700 1 --2- 172.21.15.163:0/600362111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3de00b03c0 0x7f3de0156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:38.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.239+0000 7f3dde7fc700 1 --2- 172.21.15.163:0/600362111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3de00b0790 0x7f3de00c9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:38.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.239+0000 7f3ddf7fe700 1 --2- 172.21.15.163:0/600362111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3de01591e0 0x7f3de0169d50 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f3dd800b590 tx=0x7f3dd8006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:38.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.239+0000 7f3dc6ffd700 1 -- 172.21.15.163:0/600362111 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd800f510 con 0x7f3de01591e0 2023-10-09T06:37:38.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.239+0000 7f3dc6ffd700 1 -- 172.21.15.163:0/600362111 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd800f670 con 0x7f3de01591e0 2023-10-09T06:37:38.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.239+0000 7f3ddffff700 1 -- 172.21.15.163:0/600362111 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de015b550 con 0x7f3de01591e0 2023-10-09T06:37:38.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.239+0000 7f3dc6ffd700 1 -- 172.21.15.163:0/600362111 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd800fa30 con 0x7f3de01591e0 2023-10-09T06:37:38.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.239+0000 7f3ddffff700 1 -- 172.21.15.163:0/600362111 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de0156940 con 0x7f3de01591e0 2023-10-09T06:37:38.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.240+0000 7f3ddffff700 1 -- 172.21.15.163:0/600362111 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de015ba70 con 0x7f3de01591e0 2023-10-09T06:37:38.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.240+0000 7f3dc6ffd700 1 -- 172.21.15.163:0/600362111 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3dd801a400 con 0x7f3de01591e0 2023-10-09T06:37:38.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.241+0000 7f3dc6ffd700 1 --2- 172.21.15.163:0/600362111 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3dcc067320 0x7f3dcc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:38.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.241+0000 7f3dc6ffd700 1 -- 172.21.15.163:0/600362111 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5525+0+0 (secure 0 0 0) 0x7f3dd8084760 con 0x7f3de01591e0 2023-10-09T06:37:38.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.241+0000 7f3ddeffd700 1 --2- 172.21.15.163:0/600362111 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3dcc067320 0x7f3dcc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:38.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.241+0000 7f3ddeffd700 1 --2- 172.21.15.163:0/600362111 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3dcc067320 0x7f3dcc0697d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0009880 tx=0x7f3dd0018000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:38.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.245+0000 7f3dc6ffd700 1 -- 172.21.15.163:0/600362111 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3de015ba70 con 0x7f3de01591e0 2023-10-09T06:37:38.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:38.428+0000 7f3ddffff700 1 -- 172.21.15.163:0/600362111 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3de015ba70 con 0x7f3de01591e0 2023-10-09T06:37:39.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.096+0000 7f3dc6ffd700 1 -- 172.21.15.163:0/600362111 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v128) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3de015ba70 con 0x7f3de01591e0 2023-10-09T06:37:39.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.099+0000 7f3ddffff700 1 -- 172.21.15.163:0/600362111 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3dcc067320 msgr2=0x7f3dcc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:39.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.099+0000 7f3ddffff700 1 --2- 172.21.15.163:0/600362111 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3dcc067320 0x7f3dcc0697d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0009880 tx=0x7f3dd0018000 comp rx=0 tx=0).stop 2023-10-09T06:37:39.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.099+0000 7f3ddffff700 1 -- 172.21.15.163:0/600362111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3de01591e0 msgr2=0x7f3de0169d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:39.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.099+0000 7f3ddffff700 1 --2- 172.21.15.163:0/600362111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3de01591e0 0x7f3de0169d50 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f3dd800b590 tx=0x7f3dd8006f90 comp rx=0 tx=0).stop 2023-10-09T06:37:39.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.099+0000 7f3ddffff700 1 -- 172.21.15.163:0/600362111 shutdown_connections 2023-10-09T06:37:39.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.099+0000 7f3ddffff700 1 --2- 172.21.15.163:0/600362111 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3dcc067320 0x7f3dcc0697d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:39.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.099+0000 7f3ddffff700 1 --2- 172.21.15.163:0/600362111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3de01591e0 0x7f3de0169d50 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:39.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.099+0000 7f3ddffff700 1 --2- 172.21.15.163:0/600362111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3de00b03c0 0x7f3de0156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:39.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.099+0000 7f3ddffff700 1 --2- 172.21.15.163:0/600362111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3de00b0790 0x7f3de00c9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:39.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.099+0000 7f3ddffff700 1 -- 172.21.15.163:0/600362111 >> 172.21.15.163:0/600362111 conn(0x7f3de0063230 msgr2=0x7f3de00b42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:39.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.099+0000 7f3ddffff700 1 -- 172.21.15.163:0/600362111 shutdown_connections 2023-10-09T06:37:39.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.099+0000 7f3ddffff700 1 -- 172.21.15.163:0/600362111 wait complete. 2023-10-09T06:37:39.102 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_data' removed 2023-10-09T06:37:39.113 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-09T06:37:39.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.231+0000 7f2bac876700 1 Processor -- start 2023-10-09T06:37:39.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.231+0000 7f2bac876700 1 -- start start 2023-10-09T06:37:39.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.231+0000 7f2bac876700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2ba8159890 0x7f2ba815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:39.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.231+0000 7f2bac876700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2ba8159c60 0x7f2ba815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:39.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.231+0000 7f2bac876700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ba815a160 con 0x7f2ba815db10 2023-10-09T06:37:39.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.231+0000 7f2bac876700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ba815df50 con 0x7f2ba8159890 2023-10-09T06:37:39.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.231+0000 7f2bac876700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ba80d13a0 con 0x7f2ba8159c60 2023-10-09T06:37:39.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.232+0000 7f2ba659c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2ba8159890 0x7f2ba815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:39.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.232+0000 7f2ba6d9d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f2ba815db10 0x7f2ba8069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:40960/0 (socket says 172.21.15.163:40960) 2023-10-09T06:37:39.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.232+0000 7f2ba6d9d700 1 -- 172.21.15.163:0/2404763409 learned_addr learned my addr 172.21.15.163:0/2404763409 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:39.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.232+0000 7f2ba659c700 1 -- 172.21.15.163:0/2404763409 >> v2:172.21.15.181:3300/0 conn(0x7f2ba8159c60 msgr2=0x7f2ba815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:39.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.232+0000 7f2ba659c700 1 --2- 172.21.15.163:0/2404763409 >> v2:172.21.15.181:3300/0 conn(0x7f2ba8159c60 0x7f2ba815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:39.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.232+0000 7f2ba659c700 1 -- 172.21.15.163:0/2404763409 >> v1:172.21.15.163:6789/0 conn(0x7f2ba815db10 legacy=0x7f2ba8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:39.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.232+0000 7f2ba659c700 1 -- 172.21.15.163:0/2404763409 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba815e140 con 0x7f2ba8159890 2023-10-09T06:37:39.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.232+0000 7f2ba5d9b700 1 -- 172.21.15.163:0/2404763409 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1223188484 0 0) 0x7f2ba815a160 con 0x7f2ba815db10 2023-10-09T06:37:39.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.232+0000 7f2ba659c700 1 --2- 172.21.15.163:0/2404763409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ba8159890 0x7f2ba815abd0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f2ba80b67f0 tx=0x7f2b980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d8ae7091e95973 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:39.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.232+0000 7f2ba5d9b700 1 -- 172.21.15.163:0/2404763409 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9800b6b0 con 0x7f2ba8159890 2023-10-09T06:37:39.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.232+0000 7f2ba5d9b700 1 -- 172.21.15.163:0/2404763409 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b9800b810 con 0x7f2ba8159890 2023-10-09T06:37:39.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.232+0000 7f2ba5d9b700 1 -- 172.21.15.163:0/2404763409 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9800bbb0 con 0x7f2ba8159890 2023-10-09T06:37:39.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.233+0000 7f2bac876700 1 -- 172.21.15.163:0/2404763409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ba8159890 msgr2=0x7f2ba815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:39.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.233+0000 7f2bac876700 1 --2- 172.21.15.163:0/2404763409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ba8159890 0x7f2ba815abd0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f2ba80b67f0 tx=0x7f2b980091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:39.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.233+0000 7f2bac876700 1 -- 172.21.15.163:0/2404763409 shutdown_connections 2023-10-09T06:37:39.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.233+0000 7f2bac876700 1 --2- 172.21.15.163:0/2404763409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ba8159890 0x7f2ba815abd0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:39.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.233+0000 7f2bac876700 1 --2- 172.21.15.163:0/2404763409 >> v2:172.21.15.181:3300/0 conn(0x7f2ba8159c60 0x7f2ba815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:39.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.233+0000 7f2bac876700 1 -- 172.21.15.163:0/2404763409 >> 172.21.15.163:0/2404763409 conn(0x7f2ba8068e20 msgr2=0x7f2ba80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:39.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.233+0000 7f2bac876700 1 -- 172.21.15.163:0/2404763409 shutdown_connections 2023-10-09T06:37:39.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.233+0000 7f2bac876700 1 -- 172.21.15.163:0/2404763409 wait complete. 2023-10-09T06:37:39.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.234+0000 7f2bac876700 1 Processor -- start 2023-10-09T06:37:39.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.234+0000 7f2bac876700 1 -- start start 2023-10-09T06:37:39.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.235+0000 7f2bac876700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ba8159890 0x7f2ba80b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:39.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.235+0000 7f2bac876700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ba8159c60 0x7f2ba8151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:39.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.235+0000 7f2bac876700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ba815db10 0x7f2ba8152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:39.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.235+0000 7f2bac876700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba80b46e0 con 0x7f2ba8159c60 2023-10-09T06:37:39.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.235+0000 7f2bac876700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba80b61b0 con 0x7f2ba8159890 2023-10-09T06:37:39.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.235+0000 7f2bac876700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba80b3ef0 con 0x7f2ba815db10 2023-10-09T06:37:39.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.235+0000 7f2b9dd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ba8159c60 0x7f2ba8151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:39.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.235+0000 7f2b9dd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ba8159c60 0x7f2ba8151d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:48266/0 (socket says 172.21.15.163:48266) 2023-10-09T06:37:39.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.235+0000 7f2b9dd9b700 1 -- 172.21.15.163:0/1254257302 learned_addr learned my addr 172.21.15.163:0/1254257302 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:39.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.235+0000 7f2ba659c700 1 --2- 172.21.15.163:0/1254257302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ba8159890 0x7f2ba80b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:39.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.235+0000 7f2ba6d9d700 1 --2- 172.21.15.163:0/1254257302 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ba815db10 0x7f2ba8152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:39.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.235+0000 7f2b9dd9b700 1 -- 172.21.15.163:0/1254257302 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ba815db10 msgr2=0x7f2ba8152240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:39.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.235+0000 7f2b9dd9b700 1 --2- 172.21.15.163:0/1254257302 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ba815db10 0x7f2ba8152240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:39.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.235+0000 7f2b9dd9b700 1 -- 172.21.15.163:0/1254257302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ba8159890 msgr2=0x7f2ba80b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:39.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.235+0000 7f2b9dd9b700 1 --2- 172.21.15.163:0/1254257302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ba8159890 0x7f2ba80b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:39.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.235+0000 7f2b9dd9b700 1 -- 172.21.15.163:0/1254257302 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b98009420 con 0x7f2ba8159c60 2023-10-09T06:37:39.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.235+0000 7f2b9dd9b700 1 --2- 172.21.15.163:0/1254257302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ba8159c60 0x7f2ba8151d00 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000b810 tx=0x7f2b90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:39.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.235+0000 7f2b9effd700 1 -- 172.21.15.163:0/1254257302 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9000f510 con 0x7f2ba8159c60 2023-10-09T06:37:39.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.236+0000 7f2b9effd700 1 -- 172.21.15.163:0/1254257302 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b9000f690 con 0x7f2ba8159c60 2023-10-09T06:37:39.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.236+0000 7f2bac876700 1 -- 172.21.15.163:0/1254257302 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ba80b5ba0 con 0x7f2ba8159c60 2023-10-09T06:37:39.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.236+0000 7f2b9effd700 1 -- 172.21.15.163:0/1254257302 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9000fa50 con 0x7f2ba8159c60 2023-10-09T06:37:39.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.236+0000 7f2bac876700 1 -- 172.21.15.163:0/1254257302 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ba80b4400 con 0x7f2ba8159c60 2023-10-09T06:37:39.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.236+0000 7f2bac876700 1 -- 172.21.15.163:0/1254257302 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ba804c3c0 con 0x7f2ba8159c60 2023-10-09T06:37:39.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.237+0000 7f2b9effd700 1 -- 172.21.15.163:0/1254257302 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2b9001a400 con 0x7f2ba8159c60 2023-10-09T06:37:39.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.237+0000 7f2b9effd700 1 --2- 172.21.15.163:0/1254257302 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2b88067320 0x7f2b880697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:39.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.237+0000 7f2b9effd700 1 -- 172.21.15.163:0/1254257302 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5111+0+0 (secure 0 0 0) 0x7f2b900845b0 con 0x7f2ba8159c60 2023-10-09T06:37:39.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.237+0000 7f2ba659c700 1 --2- 172.21.15.163:0/1254257302 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2b88067320 0x7f2b880697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:39.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.238+0000 7f2ba659c700 1 --2- 172.21.15.163:0/1254257302 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2b88067320 0x7f2b880697d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2b980093f0 tx=0x7f2b98017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:39.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.242+0000 7f2b9effd700 1 -- 172.21.15.163:0/1254257302 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2b90051510 con 0x7f2ba8159c60 2023-10-09T06:37:39.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.427+0000 7f2bac876700 1 -- 172.21.15.163:0/1254257302 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2ba80b5f00 con 0x7f2ba8159c60 2023-10-09T06:37:39.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.430+0000 7f2b9effd700 1 -- 172.21.15.163:0/1254257302 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2b90054b30 con 0x7f2ba8159c60 2023-10-09T06:37:39.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.432+0000 7f2bac876700 1 -- 172.21.15.163:0/1254257302 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2b88067320 msgr2=0x7f2b880697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:39.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.432+0000 7f2bac876700 1 --2- 172.21.15.163:0/1254257302 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2b88067320 0x7f2b880697d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2b980093f0 tx=0x7f2b98017000 comp rx=0 tx=0).stop 2023-10-09T06:37:39.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.432+0000 7f2bac876700 1 -- 172.21.15.163:0/1254257302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ba8159c60 msgr2=0x7f2ba8151d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:39.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.432+0000 7f2bac876700 1 --2- 172.21.15.163:0/1254257302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ba8159c60 0x7f2ba8151d00 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000b810 tx=0x7f2b90009250 comp rx=0 tx=0).stop 2023-10-09T06:37:39.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.432+0000 7f2bac876700 1 -- 172.21.15.163:0/1254257302 shutdown_connections 2023-10-09T06:37:39.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.432+0000 7f2bac876700 1 --2- 172.21.15.163:0/1254257302 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2b88067320 0x7f2b880697d0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:39.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.432+0000 7f2bac876700 1 --2- 172.21.15.163:0/1254257302 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ba815db10 0x7f2ba8152240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:39.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.432+0000 7f2bac876700 1 --2- 172.21.15.163:0/1254257302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ba8159890 0x7f2ba80b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:39.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.432+0000 7f2bac876700 1 --2- 172.21.15.163:0/1254257302 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ba8159c60 0x7f2ba8151d00 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:39.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.432+0000 7f2bac876700 1 -- 172.21.15.163:0/1254257302 >> 172.21.15.163:0/1254257302 conn(0x7f2ba8068e20 msgr2=0x7f2ba814d350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:39.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.433+0000 7f2bac876700 1 -- 172.21.15.163:0/1254257302 shutdown_connections 2023-10-09T06:37:39.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.433+0000 7f2bac876700 1 -- 172.21.15.163:0/1254257302 wait complete. 2023-10-09T06:37:39.445 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls' 2023-10-09T06:37:39.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.563+0000 7f149fbeb700 1 Processor -- start 2023-10-09T06:37:39.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.564+0000 7f149fbeb700 1 -- start start 2023-10-09T06:37:39.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.564+0000 7f149fbeb700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1498159890 0x7f149815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:39.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.564+0000 7f149fbeb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1498159c60 0x7f149815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:39.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.564+0000 7f149fbeb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f149815db10 0x7f1498069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:39.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.564+0000 7f149fbeb700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f149815a160 con 0x7f1498159890 2023-10-09T06:37:39.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.564+0000 7f149fbeb700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14980d13a0 con 0x7f149815db10 2023-10-09T06:37:39.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.564+0000 7f149fbeb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f149815e140 con 0x7f1498159c60 2023-10-09T06:37:39.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.565+0000 7f149d987700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1498159890 0x7f149815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:39.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.565+0000 7f149d987700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1498159890 0x7f149815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:48278/0 (socket says 172.21.15.163:48278) 2023-10-09T06:37:39.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.565+0000 7f149d987700 1 -- 172.21.15.163:0/1288517633 learned_addr learned my addr 172.21.15.163:0/1288517633 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:39.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.565+0000 7f149d186700 1 --2- 172.21.15.163:0/1288517633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1498159c60 0x7f149815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:39.569 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.565+0000 7f149e188700 1 --2- 172.21.15.163:0/1288517633 >> v2:172.21.15.181:3300/0 conn(0x7f149815db10 0x7f1498069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:39.569 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.565+0000 7f149d987700 1 -- 172.21.15.163:0/1288517633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1498159c60 msgr2=0x7f149815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:39.569 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.565+0000 7f149d987700 1 --2- 172.21.15.163:0/1288517633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1498159c60 0x7f149815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:39.569 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.565+0000 7f149d987700 1 -- 172.21.15.163:0/1288517633 >> v2:172.21.15.181:3300/0 conn(0x7f149815db10 msgr2=0x7f1498069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:39.569 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.565+0000 7f149d987700 1 --2- 172.21.15.163:0/1288517633 >> v2:172.21.15.181:3300/0 conn(0x7f149815db10 0x7f1498069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:39.569 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.565+0000 7f149d987700 1 -- 172.21.15.163:0/1288517633 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149815a3d0 con 0x7f1498159890 2023-10-09T06:37:39.569 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.565+0000 7f149d987700 1 --2- 172.21.15.163:0/1288517633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1498159890 0x7f149815abd0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f1498089b50 tx=0x7f1494009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1791d98a727b1b5d server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:39.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.565+0000 7f149c985700 1 -- 172.21.15.163:0/1288517633 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149400b860 con 0x7f1498159890 2023-10-09T06:37:39.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.565+0000 7f149c985700 1 -- 172.21.15.163:0/1288517633 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149400b9c0 con 0x7f1498159890 2023-10-09T06:37:39.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.565+0000 7f149c985700 1 -- 172.21.15.163:0/1288517633 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149400bd60 con 0x7f1498159890 2023-10-09T06:37:39.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.566+0000 7f149fbeb700 1 -- 172.21.15.163:0/1288517633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1498159890 msgr2=0x7f149815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:39.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.566+0000 7f149fbeb700 1 --2- 172.21.15.163:0/1288517633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1498159890 0x7f149815abd0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f1498089b50 tx=0x7f1494009250 comp rx=0 tx=0).stop 2023-10-09T06:37:39.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.566+0000 7f149fbeb700 1 -- 172.21.15.163:0/1288517633 shutdown_connections 2023-10-09T06:37:39.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.566+0000 7f149fbeb700 1 --2- 172.21.15.163:0/1288517633 >> v2:172.21.15.181:3300/0 conn(0x7f149815db10 0x7f1498069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:39.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.566+0000 7f149fbeb700 1 --2- 172.21.15.163:0/1288517633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1498159890 0x7f149815abd0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:39.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.566+0000 7f149fbeb700 1 --2- 172.21.15.163:0/1288517633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1498159c60 0x7f149815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:39.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.566+0000 7f149fbeb700 1 -- 172.21.15.163:0/1288517633 >> 172.21.15.163:0/1288517633 conn(0x7f1498068e20 msgr2=0x7f14980d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:39.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.566+0000 7f149fbeb700 1 -- 172.21.15.163:0/1288517633 shutdown_connections 2023-10-09T06:37:39.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.566+0000 7f149fbeb700 1 -- 172.21.15.163:0/1288517633 wait complete. 2023-10-09T06:37:39.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.567+0000 7f149fbeb700 1 Processor -- start 2023-10-09T06:37:39.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.567+0000 7f149fbeb700 1 -- start start 2023-10-09T06:37:39.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.568+0000 7f149fbeb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1498159890 0x7f14980b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:39.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.568+0000 7f149fbeb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1498159c60 0x7f14980cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:39.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.568+0000 7f149fbeb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f149815db10 0x7f1498158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:39.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.568+0000 7f149fbeb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14980b4530 con 0x7f1498159890 2023-10-09T06:37:39.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.568+0000 7f149fbeb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14980b5f40 con 0x7f1498159c60 2023-10-09T06:37:39.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.568+0000 7f149fbeb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14980b6220 con 0x7f149815db10 2023-10-09T06:37:39.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.568+0000 7f149d987700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1498159890 0x7f14980b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:39.574 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.568+0000 7f149d987700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1498159890 0x7f14980b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:48294/0 (socket says 172.21.15.163:48294) 2023-10-09T06:37:39.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.568+0000 7f149d987700 1 -- 172.21.15.163:0/778688225 learned_addr learned my addr 172.21.15.163:0/778688225 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:39.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.568+0000 7f149d186700 1 --2- 172.21.15.163:0/778688225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1498159c60 0x7f14980cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:39.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.568+0000 7f149e188700 1 --2- 172.21.15.163:0/778688225 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f149815db10 0x7f1498158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:39.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.568+0000 7f149d987700 1 -- 172.21.15.163:0/778688225 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f149815db10 msgr2=0x7f1498158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:39.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.568+0000 7f149d987700 1 --2- 172.21.15.163:0/778688225 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f149815db10 0x7f1498158560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:39.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.568+0000 7f149d987700 1 -- 172.21.15.163:0/778688225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1498159c60 msgr2=0x7f14980cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:39.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.568+0000 7f149d987700 1 --2- 172.21.15.163:0/778688225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1498159c60 0x7f14980cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:39.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.568+0000 7f149d987700 1 -- 172.21.15.163:0/778688225 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14940094f0 con 0x7f1498159890 2023-10-09T06:37:39.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.568+0000 7f149d987700 1 --2- 172.21.15.163:0/778688225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1498159890 0x7f14980b3f50 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f1498070e20 tx=0x7f1494017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:39.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.568+0000 7f148dffb700 1 -- 172.21.15.163:0/778688225 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1494024440 con 0x7f1498159890 2023-10-09T06:37:39.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.568+0000 7f148dffb700 1 -- 172.21.15.163:0/778688225 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14940245a0 con 0x7f1498159890 2023-10-09T06:37:39.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.568+0000 7f149fbeb700 1 -- 172.21.15.163:0/778688225 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14980ce350 con 0x7f1498159890 2023-10-09T06:37:39.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.568+0000 7f148dffb700 1 -- 172.21.15.163:0/778688225 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1494024960 con 0x7f1498159890 2023-10-09T06:37:39.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.569+0000 7f149fbeb700 1 -- 172.21.15.163:0/778688225 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14980b5b60 con 0x7f1498159890 2023-10-09T06:37:39.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.569+0000 7f149fbeb700 1 -- 172.21.15.163:0/778688225 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f149804c3c0 con 0x7f1498159890 2023-10-09T06:37:39.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.570+0000 7f148dffb700 1 -- 172.21.15.163:0/778688225 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f149401a420 con 0x7f1498159890 2023-10-09T06:37:39.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.570+0000 7f148dffb700 1 --2- 172.21.15.163:0/778688225 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f147c067320 0x7f147c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:39.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.570+0000 7f148dffb700 1 -- 172.21.15.163:0/778688225 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5111+0+0 (secure 0 0 0) 0x7f1494086840 con 0x7f1498159890 2023-10-09T06:37:39.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.570+0000 7f149d186700 1 --2- 172.21.15.163:0/778688225 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f147c067320 0x7f147c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:39.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.571+0000 7f149d186700 1 --2- 172.21.15.163:0/778688225 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f147c067320 0x7f147c0697d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f149816d290 tx=0x7f1488008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:39.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.574+0000 7f148dffb700 1 -- 172.21.15.163:0/778688225 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1494053720 con 0x7f1498159890 2023-10-09T06:37:39.835 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:39.834+0000 7f149fbeb700 1 -- 172.21.15.163:0/778688225 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls"]} v 0) v1 -- 0x7f149806a5b0 con 0x7f1498159890 2023-10-09T06:37:40.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.096+0000 7f148dffb700 1 -- 172.21.15.163:0/778688225 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls"]}]=0 v350) v1 ==== 131+0+0 (secure 0 0 0) 0x7f1494056d40 con 0x7f1498159890 2023-10-09T06:37:40.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.098+0000 7f149fbeb700 1 -- 172.21.15.163:0/778688225 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f147c067320 msgr2=0x7f147c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:40.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.098+0000 7f149fbeb700 1 --2- 172.21.15.163:0/778688225 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f147c067320 0x7f147c0697d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f149816d290 tx=0x7f1488008040 comp rx=0 tx=0).stop 2023-10-09T06:37:40.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.098+0000 7f149fbeb700 1 -- 172.21.15.163:0/778688225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1498159890 msgr2=0x7f14980b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:40.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.098+0000 7f149fbeb700 1 --2- 172.21.15.163:0/778688225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1498159890 0x7f14980b3f50 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f1498070e20 tx=0x7f1494017000 comp rx=0 tx=0).stop 2023-10-09T06:37:40.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.098+0000 7f149fbeb700 1 -- 172.21.15.163:0/778688225 shutdown_connections 2023-10-09T06:37:40.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.098+0000 7f149fbeb700 1 --2- 172.21.15.163:0/778688225 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f147c067320 0x7f147c0697d0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:40.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.098+0000 7f149fbeb700 1 --2- 172.21.15.163:0/778688225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1498159890 0x7f14980b3f50 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:40.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.098+0000 7f149fbeb700 1 --2- 172.21.15.163:0/778688225 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f149815db10 0x7f1498158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:40.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.098+0000 7f149fbeb700 1 --2- 172.21.15.163:0/778688225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1498159c60 0x7f14980cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:40.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.098+0000 7f149fbeb700 1 -- 172.21.15.163:0/778688225 >> 172.21.15.163:0/778688225 conn(0x7f1498068e20 msgr2=0x7f1498153160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:40.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.098+0000 7f149fbeb700 1 -- 172.21.15.163:0/778688225 shutdown_connections 2023-10-09T06:37:40.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.098+0000 7f149fbeb700 1 -- 172.21.15.163:0/778688225 wait complete. 2023-10-09T06:37:40.112 INFO:tasks.cephfs_test_runner:That the existing and the newly created volumes can be listed and ... ok 2023-10-09T06:37:40.112 INFO:tasks.cephfs_test_runner:Starting test: test_volume_rename (tasks.cephfs.test_volumes.TestVolumes) That volume, its file system and pools, can be renamed. 2023-10-09T06:37:40.112 INFO:tasks.cephfs_test_runner:test_volume_rename (tasks.cephfs.test_volumes.TestVolumes) 2023-10-09T06:37:40.113 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename' 2023-10-09T06:37:40.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.228+0000 7f672d2af700 1 Processor -- start 2023-10-09T06:37:40.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.228+0000 7f672d2af700 1 -- start start 2023-10-09T06:37:40.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.228+0000 7f672d2af700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6728159890 0x7f672815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:40.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.229+0000 7f672d2af700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6728069560 0x7f67280699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:40.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.229+0000 7f672d2af700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f672815a160 con 0x7f6728159c60 2023-10-09T06:37:40.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.229+0000 7f672d2af700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6728069f20 con 0x7f6728069560 2023-10-09T06:37:40.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.229+0000 7f672d2af700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67280d13a0 con 0x7f6728159890 2023-10-09T06:37:40.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.229+0000 7f6726ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6728159890 0x7f672815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:40.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.229+0000 7f67277fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6728069560 0x7f67280699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:40.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.229+0000 7f6726ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6728159890 0x7f672815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:48242/0 (socket says 172.21.15.163:48242) 2023-10-09T06:37:40.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.229+0000 7f6726ffd700 1 -- 172.21.15.163:0/1801223063 learned_addr learned my addr 172.21.15.163:0/1801223063 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:40.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.229+0000 7f67277fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6728069560 0x7f67280699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:34828/0 (socket says 172.21.15.163:34828) 2023-10-09T06:37:40.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.229+0000 7f67267fc700 1 --1- 172.21.15.163:0/1801223063 >> v1:172.21.15.181:6789/0 conn(0x7f6728159c60 0x7f672815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:51520/0 (socket says 172.21.15.163:51520) 2023-10-09T06:37:40.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.229+0000 7f67277fe700 1 -- 172.21.15.163:0/1801223063 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6728159890 msgr2=0x7f672815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:40.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.229+0000 7f67277fe700 1 --2- 172.21.15.163:0/1801223063 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6728159890 0x7f672815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:40.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.229+0000 7f67277fe700 1 -- 172.21.15.163:0/1801223063 >> v1:172.21.15.181:6789/0 conn(0x7f6728159c60 legacy=0x7f672815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:40.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.229+0000 7f67277fe700 1 -- 172.21.15.163:0/1801223063 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672815e140 con 0x7f6728069560 2023-10-09T06:37:40.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.229+0000 7f6726ffd700 1 --2- 172.21.15.163:0/1801223063 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6728159890 0x7f672815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:37:40.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.229+0000 7f67277fe700 1 --2- 172.21.15.163:0/1801223063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6728069560 0x7f67280699e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6720009e60 tx=0x7f6720009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3e84238db0a1bcd server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:40.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.230+0000 7f6725ffb700 1 -- 172.21.15.163:0/1801223063 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672000e070 con 0x7f6728069560 2023-10-09T06:37:40.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.230+0000 7f6725ffb700 1 -- 172.21.15.163:0/1801223063 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672000bb60 con 0x7f6728069560 2023-10-09T06:37:40.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.230+0000 7f6725ffb700 1 -- 172.21.15.163:0/1801223063 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6720012430 con 0x7f6728069560 2023-10-09T06:37:40.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.230+0000 7f672d2af700 1 -- 172.21.15.163:0/1801223063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6728069560 msgr2=0x7f67280699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:40.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.230+0000 7f672d2af700 1 --2- 172.21.15.163:0/1801223063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6728069560 0x7f67280699e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6720009e60 tx=0x7f6720009780 comp rx=0 tx=0).stop 2023-10-09T06:37:40.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.230+0000 7f672d2af700 1 -- 172.21.15.163:0/1801223063 shutdown_connections 2023-10-09T06:37:40.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.230+0000 7f672d2af700 1 --2- 172.21.15.163:0/1801223063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6728069560 0x7f67280699e0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:40.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.230+0000 7f672d2af700 1 --2- 172.21.15.163:0/1801223063 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6728159890 0x7f672815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:40.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.230+0000 7f672d2af700 1 -- 172.21.15.163:0/1801223063 >> 172.21.15.163:0/1801223063 conn(0x7f6728068e20 msgr2=0x7f67280d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:40.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.230+0000 7f672d2af700 1 -- 172.21.15.163:0/1801223063 shutdown_connections 2023-10-09T06:37:40.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.230+0000 7f672d2af700 1 -- 172.21.15.163:0/1801223063 wait complete. 2023-10-09T06:37:40.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.231+0000 7f672d2af700 1 Processor -- start 2023-10-09T06:37:40.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.231+0000 7f672d2af700 1 -- start start 2023-10-09T06:37:40.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.232+0000 7f672d2af700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6728069560 0x7f67281493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:40.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.232+0000 7f672d2af700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6728159890 0x7f67280b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:40.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.232+0000 7f672d2af700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6728159c60 0x7f67280b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:40.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.232+0000 7f672d2af700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67280b4d10 con 0x7f6728159c60 2023-10-09T06:37:40.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.232+0000 7f672d2af700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67280b4100 con 0x7f6728159890 2023-10-09T06:37:40.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.232+0000 7f672d2af700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67280b43e0 con 0x7f6728069560 2023-10-09T06:37:40.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.232+0000 7f67277fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6728159c60 0x7f67280b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:40.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.232+0000 7f67277fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6728159c60 0x7f67280b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:48298/0 (socket says 172.21.15.163:48298) 2023-10-09T06:37:40.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.232+0000 7f67277fe700 1 -- 172.21.15.163:0/1821339381 learned_addr learned my addr 172.21.15.163:0/1821339381 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:40.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.232+0000 7f6726ffd700 1 --2- 172.21.15.163:0/1821339381 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6728069560 0x7f67281493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:40.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.233+0000 7f67267fc700 1 --2- 172.21.15.163:0/1821339381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6728159890 0x7f67280b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:40.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.233+0000 7f67277fe700 1 -- 172.21.15.163:0/1821339381 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6728069560 msgr2=0x7f67281493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:40.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.233+0000 7f67277fe700 1 --2- 172.21.15.163:0/1821339381 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6728069560 0x7f67281493a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:40.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.233+0000 7f67277fe700 1 -- 172.21.15.163:0/1821339381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6728159890 msgr2=0x7f67280b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:40.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.233+0000 7f67277fe700 1 --2- 172.21.15.163:0/1821339381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6728159890 0x7f67280b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:40.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.233+0000 7f67277fe700 1 -- 172.21.15.163:0/1821339381 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67180094f0 con 0x7f6728159c60 2023-10-09T06:37:40.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.233+0000 7f6726ffd700 1 --2- 172.21.15.163:0/1821339381 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6728069560 0x7f67281493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:40.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.233+0000 7f67267fc700 1 --2- 172.21.15.163:0/1821339381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6728159890 0x7f67280b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:40.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.233+0000 7f67277fe700 1 --2- 172.21.15.163:0/1821339381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6728159c60 0x7f67280b3590 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f67280899c0 tx=0x7f6720019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:40.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.233+0000 7f670effd700 1 -- 172.21.15.163:0/1821339381 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6720027470 con 0x7f6728159c60 2023-10-09T06:37:40.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.233+0000 7f670effd700 1 -- 172.21.15.163:0/1821339381 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67200275f0 con 0x7f6728159c60 2023-10-09T06:37:40.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.233+0000 7f670effd700 1 -- 172.21.15.163:0/1821339381 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67200279b0 con 0x7f6728159c60 2023-10-09T06:37:40.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.233+0000 7f672d2af700 1 -- 172.21.15.163:0/1821339381 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6720009a20 con 0x7f6728159c60 2023-10-09T06:37:40.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.233+0000 7f672d2af700 1 -- 172.21.15.163:0/1821339381 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67280b49a0 con 0x7f6728159c60 2023-10-09T06:37:40.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.234+0000 7f672d2af700 1 -- 172.21.15.163:0/1821339381 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f672804c3c0 con 0x7f6728159c60 2023-10-09T06:37:40.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.234+0000 7f670effd700 1 -- 172.21.15.163:0/1821339381 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6720024070 con 0x7f6728159c60 2023-10-09T06:37:40.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.235+0000 7f670effd700 1 --2- 172.21.15.163:0/1821339381 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6714067320 0x7f67140697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:40.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.235+0000 7f670effd700 1 -- 172.21.15.163:0/1821339381 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5111+0+0 (secure 0 0 0) 0x7f67200873e0 con 0x7f6728159c60 2023-10-09T06:37:40.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.235+0000 7f6726ffd700 1 --2- 172.21.15.163:0/1821339381 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6714067320 0x7f67140697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:40.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.236+0000 7f6726ffd700 1 --2- 172.21.15.163:0/1821339381 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6714067320 0x7f67140697d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6718006d40 tx=0x7f6718008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:40.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.239+0000 7f670effd700 1 -- 172.21.15.163:0/1821339381 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f67200543c0 con 0x7f6728159c60 2023-10-09T06:37:40.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:40.497+0000 7f672d2af700 1 -- 172.21.15.163:0/1821339381 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename"]} v 0) v1 -- 0x7f672806a5b0 con 0x7f6728159c60 2023-10-09T06:37:41.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.100+0000 7f670effd700 1 -- 172.21.15.163:0/1821339381 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename"]}]=0 v351) v1 ==== 138+0+0 (secure 0 0 0) 0x7f67200579e0 con 0x7f6728159c60 2023-10-09T06:37:41.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.102+0000 7f672d2af700 1 -- 172.21.15.163:0/1821339381 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6714067320 msgr2=0x7f67140697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:41.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.102+0000 7f672d2af700 1 --2- 172.21.15.163:0/1821339381 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6714067320 0x7f67140697d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6718006d40 tx=0x7f6718008040 comp rx=0 tx=0).stop 2023-10-09T06:37:41.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.102+0000 7f672d2af700 1 -- 172.21.15.163:0/1821339381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6728159c60 msgr2=0x7f67280b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:41.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.102+0000 7f672d2af700 1 --2- 172.21.15.163:0/1821339381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6728159c60 0x7f67280b3590 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f67280899c0 tx=0x7f6720019000 comp rx=0 tx=0).stop 2023-10-09T06:37:41.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.102+0000 7f672d2af700 1 -- 172.21.15.163:0/1821339381 shutdown_connections 2023-10-09T06:37:41.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.102+0000 7f672d2af700 1 --2- 172.21.15.163:0/1821339381 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6714067320 0x7f67140697d0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:41.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.102+0000 7f672d2af700 1 --2- 172.21.15.163:0/1821339381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6728159c60 0x7f67280b3590 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:41.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.102+0000 7f672d2af700 1 --2- 172.21.15.163:0/1821339381 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6728069560 0x7f67281493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:41.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.102+0000 7f672d2af700 1 --2- 172.21.15.163:0/1821339381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6728159890 0x7f67280b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:41.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.102+0000 7f672d2af700 1 -- 172.21.15.163:0/1821339381 >> 172.21.15.163:0/1821339381 conn(0x7f6728068e20 msgr2=0x7f6728155280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:41.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.102+0000 7f672d2af700 1 -- 172.21.15.163:0/1821339381 shutdown_connections 2023-10-09T06:37:41.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.102+0000 7f672d2af700 1 -- 172.21.15.163:0/1821339381 wait complete. 2023-10-09T06:37:41.115 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-09T06:37:41.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.232+0000 7f87df265700 1 Processor -- start 2023-10-09T06:37:41.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.232+0000 7f87df265700 1 -- start start 2023-10-09T06:37:41.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.233+0000 7f87df265700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f87d8159930 0x7f87d8159d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:41.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.233+0000 7f87df265700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87d80cd1c0 0x7f87d80cd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:41.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.233+0000 7f87df265700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87d80689e0 con 0x7f87d80cb710 2023-10-09T06:37:41.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.233+0000 7f87df265700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87d80cdb80 con 0x7f87d8159930 2023-10-09T06:37:41.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.233+0000 7f87df265700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d80cbcc0 con 0x7f87d80cd1c0 2023-10-09T06:37:41.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.233+0000 7f87dd001700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f87d8159930 0x7f87d8159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:41.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.233+0000 7f87dc800700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f87d80cb710 0x7f87d80b3ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:51536/0 (socket says 172.21.15.163:51536) 2023-10-09T06:37:41.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.233+0000 7f87dc800700 1 -- 172.21.15.163:0/3521785946 learned_addr learned my addr 172.21.15.163:0/3521785946 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:41.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.233+0000 7f87dd802700 1 --2- 172.21.15.163:0/3521785946 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87d80cd1c0 0x7f87d80cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:41.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.233+0000 7f87cffff700 1 -- 172.21.15.163:0/3521785946 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 459470774 0 0) 0x7f87d80689e0 con 0x7f87d80cb710 2023-10-09T06:37:41.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.233+0000 7f87cffff700 1 -- 172.21.15.163:0/3521785946 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87c4003570 con 0x7f87d80cb710 2023-10-09T06:37:41.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.233+0000 7f87dd001700 1 -- 172.21.15.163:0/3521785946 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87d80cd1c0 msgr2=0x7f87d80cd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:41.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.233+0000 7f87dd001700 1 --2- 172.21.15.163:0/3521785946 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87d80cd1c0 0x7f87d80cd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:41.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.233+0000 7f87dd001700 1 -- 172.21.15.163:0/3521785946 >> v1:172.21.15.181:6789/0 conn(0x7f87d80cb710 legacy=0x7f87d80b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:41.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.233+0000 7f87dd001700 1 -- 172.21.15.163:0/3521785946 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d80cbfa0 con 0x7f87d8159930 2023-10-09T06:37:41.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.234+0000 7f87dd001700 1 --2- 172.21.15.163:0/3521785946 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87d8159930 0x7f87d8159d00 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f87d4009e50 tx=0x7f87d4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa43c336647e0d18 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:41.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.234+0000 7f87cffff700 1 -- 172.21.15.163:0/3521785946 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d400e070 con 0x7f87d8159930 2023-10-09T06:37:41.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.234+0000 7f87cffff700 1 -- 172.21.15.163:0/3521785946 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d400bb40 con 0x7f87d8159930 2023-10-09T06:37:41.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.234+0000 7f87cffff700 1 -- 172.21.15.163:0/3521785946 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d4012430 con 0x7f87d8159930 2023-10-09T06:37:41.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.234+0000 7f87df265700 1 -- 172.21.15.163:0/3521785946 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87d8159930 msgr2=0x7f87d8159d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:41.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.234+0000 7f87df265700 1 --2- 172.21.15.163:0/3521785946 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87d8159930 0x7f87d8159d00 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f87d4009e50 tx=0x7f87d4009780 comp rx=0 tx=0).stop 2023-10-09T06:37:41.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.234+0000 7f87df265700 1 -- 172.21.15.163:0/3521785946 shutdown_connections 2023-10-09T06:37:41.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.234+0000 7f87df265700 1 --2- 172.21.15.163:0/3521785946 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87d80cd1c0 0x7f87d80cd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:41.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.234+0000 7f87df265700 1 --2- 172.21.15.163:0/3521785946 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87d8159930 0x7f87d8159d00 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:41.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.234+0000 7f87df265700 1 -- 172.21.15.163:0/3521785946 >> 172.21.15.163:0/3521785946 conn(0x7f87d80999d0 msgr2=0x7f87d80b3690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:41.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.234+0000 7f87df265700 1 -- 172.21.15.163:0/3521785946 shutdown_connections 2023-10-09T06:37:41.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.234+0000 7f87df265700 1 -- 172.21.15.163:0/3521785946 wait complete. 2023-10-09T06:37:41.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.236+0000 7f87df265700 1 Processor -- start 2023-10-09T06:37:41.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.236+0000 7f87df265700 1 -- start start 2023-10-09T06:37:41.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.236+0000 7f87df265700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87d80cb710 0x7f87d815ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:41.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.236+0000 7f87df265700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d80cd1c0 0x7f87d80c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:41.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.236+0000 7f87df265700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87d8159930 0x7f87d8158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:41.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.236+0000 7f87df265700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d815b4b0 con 0x7f87d80cb710 2023-10-09T06:37:41.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.236+0000 7f87df265700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d815b790 con 0x7f87d80cd1c0 2023-10-09T06:37:41.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.236+0000 7f87df265700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d815ba70 con 0x7f87d8159930 2023-10-09T06:37:41.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.237+0000 7f87dd001700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87d80cb710 0x7f87d815ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:41.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.237+0000 7f87dd001700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87d80cb710 0x7f87d815ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:48316/0 (socket says 172.21.15.163:48316) 2023-10-09T06:37:41.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.237+0000 7f87dd001700 1 -- 172.21.15.163:0/825551370 learned_addr learned my addr 172.21.15.163:0/825551370 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:41.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.237+0000 7f87dd802700 1 --2- 172.21.15.163:0/825551370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87d8159930 0x7f87d8158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:41.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.237+0000 7f87dc800700 1 --2- 172.21.15.163:0/825551370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d80cd1c0 0x7f87d80c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:41.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.237+0000 7f87dd001700 1 -- 172.21.15.163:0/825551370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87d8159930 msgr2=0x7f87d8158c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:41.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.237+0000 7f87dd001700 1 --2- 172.21.15.163:0/825551370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87d8159930 0x7f87d8158c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:41.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.237+0000 7f87dd001700 1 -- 172.21.15.163:0/825551370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d80cd1c0 msgr2=0x7f87d80c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:41.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.237+0000 7f87dd001700 1 --2- 172.21.15.163:0/825551370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d80cd1c0 0x7f87d80c9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:41.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.237+0000 7f87dd001700 1 -- 172.21.15.163:0/825551370 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d40099c0 con 0x7f87d80cb710 2023-10-09T06:37:41.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.237+0000 7f87dc800700 1 --2- 172.21.15.163:0/825551370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d80cd1c0 0x7f87d80c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:41.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.237+0000 7f87dd001700 1 --2- 172.21.15.163:0/825551370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87d80cb710 0x7f87d815ae60 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f87d80891d0 tx=0x7f87d4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:41.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.237+0000 7f87cd7fa700 1 -- 172.21.15.163:0/825551370 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d4012d70 con 0x7f87d80cb710 2023-10-09T06:37:41.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.237+0000 7f87cd7fa700 1 -- 172.21.15.163:0/825551370 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d40263f0 con 0x7f87d80cb710 2023-10-09T06:37:41.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.237+0000 7f87cd7fa700 1 -- 172.21.15.163:0/825551370 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d40267b0 con 0x7f87d80cb710 2023-10-09T06:37:41.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.237+0000 7f87df265700 1 -- 172.21.15.163:0/825551370 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d815bd50 con 0x7f87d80cb710 2023-10-09T06:37:41.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.237+0000 7f87df265700 1 -- 172.21.15.163:0/825551370 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d809ba20 con 0x7f87d80cb710 2023-10-09T06:37:41.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.238+0000 7f87df265700 1 -- 172.21.15.163:0/825551370 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87d815bf70 con 0x7f87d80cb710 2023-10-09T06:37:41.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.238+0000 7f87cd7fa700 1 -- 172.21.15.163:0/825551370 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f87d4026b60 con 0x7f87d80cb710 2023-10-09T06:37:41.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.239+0000 7f87cd7fa700 1 --2- 172.21.15.163:0/825551370 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f87bc067320 0x7f87bc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:41.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.239+0000 7f87cd7fa700 1 -- 172.21.15.163:0/825551370 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5111+0+0 (secure 0 0 0) 0x7f87d4086ae0 con 0x7f87d80cb710 2023-10-09T06:37:41.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.239+0000 7f87dc800700 1 --2- 172.21.15.163:0/825551370 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f87bc067320 0x7f87bc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:41.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.240+0000 7f87dc800700 1 --2- 172.21.15.163:0/825551370 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f87bc067320 0x7f87bc0697d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f87d8160770 tx=0x7f87c8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:41.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.243+0000 7f87cd7fa700 1 -- 172.21.15.163:0/825551370 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f87d40538b0 con 0x7f87d80cb710 2023-10-09T06:37:41.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.428+0000 7f87df265700 1 -- 172.21.15.163:0/825551370 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f87d80cf840 con 0x7f87d80cb710 2023-10-09T06:37:41.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.431+0000 7f87cd7fa700 1 -- 172.21.15.163:0/825551370 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f87d4056ed0 con 0x7f87d80cb710 2023-10-09T06:37:41.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.433+0000 7f87df265700 1 -- 172.21.15.163:0/825551370 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f87bc067320 msgr2=0x7f87bc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:41.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.433+0000 7f87df265700 1 --2- 172.21.15.163:0/825551370 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f87bc067320 0x7f87bc0697d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f87d8160770 tx=0x7f87c8009040 comp rx=0 tx=0).stop 2023-10-09T06:37:41.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.433+0000 7f87df265700 1 -- 172.21.15.163:0/825551370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87d80cb710 msgr2=0x7f87d815ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:41.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.433+0000 7f87df265700 1 --2- 172.21.15.163:0/825551370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87d80cb710 0x7f87d815ae60 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f87d80891d0 tx=0x7f87d4019000 comp rx=0 tx=0).stop 2023-10-09T06:37:41.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.433+0000 7f87df265700 1 -- 172.21.15.163:0/825551370 shutdown_connections 2023-10-09T06:37:41.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.433+0000 7f87df265700 1 --2- 172.21.15.163:0/825551370 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f87bc067320 0x7f87bc0697d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:41.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.433+0000 7f87df265700 1 --2- 172.21.15.163:0/825551370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87d80cb710 0x7f87d815ae60 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:41.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.434+0000 7f87df265700 1 --2- 172.21.15.163:0/825551370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87d8159930 0x7f87d8158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:41.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.434+0000 7f87df265700 1 --2- 172.21.15.163:0/825551370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87d80cd1c0 0x7f87d80c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:41.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.434+0000 7f87df265700 1 -- 172.21.15.163:0/825551370 >> 172.21.15.163:0/825551370 conn(0x7f87d80999d0 msgr2=0x7f87d80690a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:41.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.434+0000 7f87df265700 1 -- 172.21.15.163:0/825551370 shutdown_connections 2023-10-09T06:37:41.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.434+0000 7f87df265700 1 -- 172.21.15.163:0/825551370 wait complete. 2023-10-09T06:37:41.446 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-09T06:37:41.447 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:37:41.487 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:37:41.532 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-09T06:37:41.533 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-09T06:37:41.533 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:37:41.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.646+0000 7f4fc1ac3700 1 Processor -- start 2023-10-09T06:37:41.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.646+0000 7f4fc1ac3700 1 -- start start 2023-10-09T06:37:41.651 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.647+0000 7f4fc1ac3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4fbc06bcd0 0x7f4fbc0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:41.651 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.647+0000 7f4fc1ac3700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fbc15d7e0 con 0x7f4fbc06c0a0 2023-10-09T06:37:41.651 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.647+0000 7f4fc1ac3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fbc15dac0 con 0x7f4fbc15df50 2023-10-09T06:37:41.651 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.647+0000 7f4fc1ac3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fbc15aa30 con 0x7f4fbc06bcd0 2023-10-09T06:37:41.651 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.647+0000 7f4fbaffd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f4fbc06c0a0 0x7f4fbc15a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:40962/0 (socket says 172.21.15.163:40962) 2023-10-09T06:37:41.651 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.647+0000 7f4fbaffd700 1 -- 172.21.15.163:0/2241168928 learned_addr learned my addr 172.21.15.163:0/2241168928 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:41.651 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.648+0000 7f4fbb7fe700 1 --2- 172.21.15.163:0/2241168928 >> v2:172.21.15.181:3300/0 conn(0x7f4fbc06bcd0 0x7f4fbc0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:41.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.648+0000 7f4fba7fc700 1 -- 172.21.15.163:0/2241168928 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3250475030 0 0) 0x7f4fbc15d7e0 con 0x7f4fbc06c0a0 2023-10-09T06:37:41.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.648+0000 7f4fba7fc700 1 -- 172.21.15.163:0/2241168928 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fa4003570 con 0x7f4fbc06c0a0 2023-10-09T06:37:41.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.648+0000 7f4fba7fc700 1 -- 172.21.15.163:0/2241168928 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2675760120 0 0) 0x7f4fbc15dac0 con 0x7f4fbc15df50 2023-10-09T06:37:41.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.648+0000 7f4fba7fc700 1 -- 172.21.15.163:0/2241168928 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fbc15d7e0 con 0x7f4fbc15df50 2023-10-09T06:37:41.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.648+0000 7f4fba7fc700 1 -- 172.21.15.163:0/2241168928 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1561982482 0 0) 0x7f4fa4003570 con 0x7f4fbc06c0a0 2023-10-09T06:37:41.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.648+0000 7f4fba7fc700 1 -- 172.21.15.163:0/2241168928 >> v2:172.21.15.181:3300/0 conn(0x7f4fbc06bcd0 msgr2=0x7f4fbc0caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:41.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.648+0000 7f4fba7fc700 1 --2- 172.21.15.163:0/2241168928 >> v2:172.21.15.181:3300/0 conn(0x7f4fbc06bcd0 0x7f4fbc0caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:41.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.648+0000 7f4fba7fc700 1 -- 172.21.15.163:0/2241168928 >> v1:172.21.15.181:6789/0 conn(0x7f4fbc15df50 legacy=0x7f4fbc0c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:41.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.648+0000 7f4fba7fc700 1 -- 172.21.15.163:0/2241168928 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fbc15ad10 con 0x7f4fbc06c0a0 2023-10-09T06:37:41.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.648+0000 7f4fba7fc700 1 -- 172.21.15.163:0/2241168928 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f4fb0001470 con 0x7f4fbc06c0a0 2023-10-09T06:37:41.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.648+0000 7f4fba7fc700 1 -- 172.21.15.163:0/2241168928 >> v1:172.21.15.163:6789/0 conn(0x7f4fbc06c0a0 legacy=0x7f4fbc15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:41.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.648+0000 7f4fba7fc700 1 --2- 172.21.15.163:0/2241168928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa4003bf0 0x7f4fa4003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:41.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.648+0000 7f4fba7fc700 1 --2- 172.21.15.163:0/2241168928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fa4004810 0x7f4fa4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:41.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.648+0000 7f4fba7fc700 1 --2- 172.21.15.163:0/2241168928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4fa4005230 0x7f4fa400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:41.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.648+0000 7f4fba7fc700 1 -- 172.21.15.163:0/2241168928 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc15aa30 con 0x7f4fa4005230 2023-10-09T06:37:41.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.648+0000 7f4fba7fc700 1 -- 172.21.15.163:0/2241168928 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa400c200 con 0x7f4fa4003bf0 2023-10-09T06:37:41.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.648+0000 7f4fba7fc700 1 -- 172.21.15.163:0/2241168928 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa400c3c0 con 0x7f4fa4004810 2023-10-09T06:37:41.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.648+0000 7f4fbbfff700 1 --2- 172.21.15.163:0/2241168928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4fa4005230 0x7f4fa400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:41.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.648+0000 7f4fbb7fe700 1 --2- 172.21.15.163:0/2241168928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa4003bf0 0x7f4fa4003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:41.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.648+0000 7f4fbaffd700 1 --2- 172.21.15.163:0/2241168928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fa4004810 0x7f4fa4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:41.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.649+0000 7f4fbbfff700 1 -- 172.21.15.163:0/2241168928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fa4004810 msgr2=0x7f4fa4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:41.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.649+0000 7f4fbbfff700 1 --2- 172.21.15.163:0/2241168928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fa4004810 0x7f4fa4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:41.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.649+0000 7f4fbbfff700 1 -- 172.21.15.163:0/2241168928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa4003bf0 msgr2=0x7f4fa4003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:41.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.649+0000 7f4fbbfff700 1 --2- 172.21.15.163:0/2241168928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa4003bf0 0x7f4fa4003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:41.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.649+0000 7f4fbbfff700 1 -- 172.21.15.163:0/2241168928 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa4003570 con 0x7f4fa4005230 2023-10-09T06:37:41.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.649+0000 7f4fbbfff700 1 -- 172.21.15.163:0/2241168928 reap_dead start 2023-10-09T06:37:41.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.649+0000 7f4fbbfff700 1 --2- 172.21.15.163:0/2241168928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4fa4005230 0x7f4fa400ba60 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4001a10 tx=0x7f4fb4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36904beb1c8fee32 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:41.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.649+0000 7f4fba7fc700 1 -- 172.21.15.163:0/2241168928 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb400c5d0 con 0x7f4fa4005230 2023-10-09T06:37:41.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.649+0000 7f4fba7fc700 1 -- 172.21.15.163:0/2241168928 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb400c730 con 0x7f4fa4005230 2023-10-09T06:37:41.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.649+0000 7f4fba7fc700 1 -- 172.21.15.163:0/2241168928 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb400cab0 con 0x7f4fa4005230 2023-10-09T06:37:41.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.649+0000 7f4fc1ac3700 1 -- 172.21.15.163:0/2241168928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4fa4005230 msgr2=0x7f4fa400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:41.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.649+0000 7f4fc1ac3700 1 --2- 172.21.15.163:0/2241168928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4fa4005230 0x7f4fa400ba60 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4001a10 tx=0x7f4fb4007d30 comp rx=0 tx=0).stop 2023-10-09T06:37:41.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.649+0000 7f4fc1ac3700 1 -- 172.21.15.163:0/2241168928 shutdown_connections 2023-10-09T06:37:41.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.649+0000 7f4fc1ac3700 1 --2- 172.21.15.163:0/2241168928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4fa4005230 0x7f4fa400ba60 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:41.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.649+0000 7f4fc1ac3700 1 -- 172.21.15.163:0/2241168928 >> 172.21.15.163:0/2241168928 conn(0x7f4fbc0ce600 msgr2=0x7f4fbc0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:41.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.649+0000 7f4fc1ac3700 1 -- 172.21.15.163:0/2241168928 shutdown_connections 2023-10-09T06:37:41.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.649+0000 7f4fc1ac3700 1 -- 172.21.15.163:0/2241168928 wait complete. 2023-10-09T06:37:41.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.651+0000 7f4fc1ac3700 1 Processor -- start 2023-10-09T06:37:41.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.651+0000 7f4fc1ac3700 1 -- start start 2023-10-09T06:37:41.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.651+0000 7f4fc1ac3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4fa4005230 0x7f4fbc0c7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:41.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.651+0000 7f4fc1ac3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fbc0c5900 0x7f4fbc0b2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:41.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.651+0000 7f4fc1ac3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fbc0c5d70 0x7f4fbc1541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:41.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.651+0000 7f4fc1ac3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc158da0 con 0x7f4fa4005230 2023-10-09T06:37:41.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.651+0000 7f4fc1ac3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc159600 con 0x7f4fbc0c5d70 2023-10-09T06:37:41.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.651+0000 7f4fc1ac3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc0c7e80 con 0x7f4fbc0c5900 2023-10-09T06:37:41.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.651+0000 7f4fbb7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4fa4005230 0x7f4fbc0c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:41.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.651+0000 7f4fbb7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4fa4005230 0x7f4fbc0c7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:48330/0 (socket says 172.21.15.163:48330) 2023-10-09T06:37:41.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.651+0000 7f4fbb7fe700 1 -- 172.21.15.163:0/2961130855 learned_addr learned my addr 172.21.15.163:0/2961130855 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:41.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.652+0000 7f4fbbfff700 1 --2- 172.21.15.163:0/2961130855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fbc0c5d70 0x7f4fbc1541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:41.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.652+0000 7f4fbaffd700 1 --2- 172.21.15.163:0/2961130855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fbc0c5900 0x7f4fbc0b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:41.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.652+0000 7f4fbb7fe700 1 -- 172.21.15.163:0/2961130855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fbc0c5900 msgr2=0x7f4fbc0b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:41.667 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.652+0000 7f4fbb7fe700 1 --2- 172.21.15.163:0/2961130855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fbc0c5900 0x7f4fbc0b2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:41.667 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.652+0000 7f4fbb7fe700 1 -- 172.21.15.163:0/2961130855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fbc0c5d70 msgr2=0x7f4fbc1541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:41.667 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.652+0000 7f4fbb7fe700 1 --2- 172.21.15.163:0/2961130855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fbc0c5d70 0x7f4fbc1541d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:41.667 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.652+0000 7f4fbb7fe700 1 -- 172.21.15.163:0/2961130855 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fbc0b33a0 con 0x7f4fa4005230 2023-10-09T06:37:41.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.652+0000 7f4fbbfff700 1 --2- 172.21.15.163:0/2961130855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fbc0c5d70 0x7f4fbc1541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:41.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.652+0000 7f4fbb7fe700 1 --2- 172.21.15.163:0/2961130855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4fa4005230 0x7f4fbc0c7940 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f4fac009fd0 tx=0x7f4fac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:41.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.652+0000 7f4fa37fe700 1 -- 172.21.15.163:0/2961130855 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fac00e040 con 0x7f4fa4005230 2023-10-09T06:37:41.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.652+0000 7f4fa37fe700 1 -- 172.21.15.163:0/2961130855 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fac00f040 con 0x7f4fa4005230 2023-10-09T06:37:41.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.652+0000 7f4fa37fe700 1 -- 172.21.15.163:0/2961130855 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fac015430 con 0x7f4fa4005230 2023-10-09T06:37:41.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.652+0000 7f4fc1ac3700 1 -- 172.21.15.163:0/2961130855 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fbc158fb0 con 0x7f4fa4005230 2023-10-09T06:37:41.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.652+0000 7f4fc1ac3700 1 -- 172.21.15.163:0/2961130855 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fbc0b3f60 con 0x7f4fa4005230 2023-10-09T06:37:41.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.653+0000 7f4fc1ac3700 1 -- 172.21.15.163:0/2961130855 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fbc04c3c0 con 0x7f4fa4005230 2023-10-09T06:37:41.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.653+0000 7f4fa37fe700 1 -- 172.21.15.163:0/2961130855 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4fac015860 con 0x7f4fa4005230 2023-10-09T06:37:41.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.654+0000 7f4fa37fe700 1 --2- 172.21.15.163:0/2961130855 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4fa8067320 0x7f4fa80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:41.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.654+0000 7f4fa37fe700 1 -- 172.21.15.163:0/2961130855 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5111+0+0 (secure 0 0 0) 0x7f4fac085880 con 0x7f4fa4005230 2023-10-09T06:37:41.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.654+0000 7f4fbaffd700 1 --2- 172.21.15.163:0/2961130855 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4fa8067320 0x7f4fa80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:41.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.654+0000 7f4fbaffd700 1 --2- 172.21.15.163:0/2961130855 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4fa8067320 0x7f4fa80697d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc0b1cb0 tx=0x7f4fb0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:41.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.658+0000 7f4fa37fe700 1 -- 172.21.15.163:0/2961130855 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4fac052650 con 0x7f4fa4005230 2023-10-09T06:37:41.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.883+0000 7f4fc1ac3700 1 -- 172.21.15.163:0/2961130855 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fbc15aff0 con 0x7f4fa4005230 2023-10-09T06:37:41.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.884+0000 7f4fa37fe700 1 -- 172.21.15.163:0/2961130855 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f4fac022de0 con 0x7f4fa4005230 2023-10-09T06:37:41.885 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:41.885 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":87,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2821591245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2821591245},{"type":"v1","addr":"172.21.15.181:6836","nonce":2821591245}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":70},{"gid":6784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1929060525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1929060525},{"type":"v1","addr":"172.21.15.181:6837","nonce":1929060525}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":7371,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/205885744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":205885744},{"type":"v1","addr":"172.21.15.163:6834","nonce":205885744}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":7458,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/993637818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":993637818},{"type":"v1","addr":"172.21.15.163:6835","nonce":993637818}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[]} 2023-10-09T06:37:41.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.885+0000 7f4fc1ac3700 1 -- 172.21.15.163:0/2961130855 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4fa8067320 msgr2=0x7f4fa80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:41.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.885+0000 7f4fc1ac3700 1 --2- 172.21.15.163:0/2961130855 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4fa8067320 0x7f4fa80697d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc0b1cb0 tx=0x7f4fb0009040 comp rx=0 tx=0).stop 2023-10-09T06:37:41.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.885+0000 7f4fc1ac3700 1 -- 172.21.15.163:0/2961130855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4fa4005230 msgr2=0x7f4fbc0c7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:41.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.885+0000 7f4fc1ac3700 1 --2- 172.21.15.163:0/2961130855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4fa4005230 0x7f4fbc0c7940 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f4fac009fd0 tx=0x7f4fac00b040 comp rx=0 tx=0).stop 2023-10-09T06:37:41.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.885+0000 7f4fc1ac3700 1 -- 172.21.15.163:0/2961130855 shutdown_connections 2023-10-09T06:37:41.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.885+0000 7f4fc1ac3700 1 --2- 172.21.15.163:0/2961130855 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4fa8067320 0x7f4fa80697d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:41.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.885+0000 7f4fc1ac3700 1 --2- 172.21.15.163:0/2961130855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4fa4005230 0x7f4fbc0c7940 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:41.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.885+0000 7f4fc1ac3700 1 --2- 172.21.15.163:0/2961130855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fbc0c5d70 0x7f4fbc1541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:41.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.885+0000 7f4fc1ac3700 1 --2- 172.21.15.163:0/2961130855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fbc0c5900 0x7f4fbc0b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:41.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.885+0000 7f4fc1ac3700 1 -- 172.21.15.163:0/2961130855 >> 172.21.15.163:0/2961130855 conn(0x7f4fbc0ce600 msgr2=0x7f4fbc06a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:41.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.886+0000 7f4fc1ac3700 1 -- 172.21.15.163:0/2961130855 shutdown_connections 2023-10-09T06:37:41.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:41.886+0000 7f4fc1ac3700 1 -- 172.21.15.163:0/2961130855 wait complete. 2023-10-09T06:37:41.888 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 87 2023-10-09T06:37:41.899 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-09T06:37:41.899 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-09T06:37:41.899 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-09T06:37:41.899 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:37:41.899 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-09T06:37:41.900 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-09T06:37:41.900 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-09T06:37:41.900 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:37:41.900 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-09T06:37:41.900 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-09T06:37:41.900 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-09T06:37:41.900 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:37:41.901 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-09T06:37:41.901 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-09T06:37:41.901 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-09T06:37:41.901 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:37:41.908 INFO:tasks.ceph.mds.c:Stopped 2023-10-09T06:37:41.908 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-09T06:37:41.909 INFO:tasks.ceph.mds.a:Stopped 2023-10-09T06:37:41.909 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-09T06:37:41.912 INFO:tasks.ceph.mds.b:Stopped 2023-10-09T06:37:41.912 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-09T06:37:41.913 INFO:tasks.ceph.mds.d:Stopped 2023-10-09T06:37:41.913 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-09T06:37:41.951 INFO:tasks.ceph.mds.c:Started 2023-10-09T06:37:41.951 DEBUG:teuthology.parallel:result is None 2023-10-09T06:37:41.952 INFO:tasks.ceph.mds.a:Started 2023-10-09T06:37:41.952 DEBUG:teuthology.parallel:result is None 2023-10-09T06:37:41.955 INFO:tasks.ceph.mds.b:Started 2023-10-09T06:37:41.955 DEBUG:teuthology.parallel:result is None 2023-10-09T06:37:41.956 INFO:tasks.ceph.mds.d:Started 2023-10-09T06:37:41.956 DEBUG:teuthology.parallel:result is None 2023-10-09T06:37:41.956 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-09T06:37:42.025 INFO:tasks.ceph.mds.a.smithi163.stdout:starting mds.a at 2023-10-09T06:37:42.025 INFO:tasks.ceph.mds.c.smithi163.stdout:starting mds.c at 2023-10-09T06:37:42.038 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-09T06:37:42.039 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-09T06:37:42.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.076+0000 7ff8cf09b700 1 Processor -- start 2023-10-09T06:37:42.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.076+0000 7ff8cf09b700 1 -- start start 2023-10-09T06:37:42.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.077+0000 7ff8cf09b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8c80b1c20 0x7ff8c81572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:42.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.077+0000 7ff8cf09b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8c8157c80 con 0x7ff8c8157820 2023-10-09T06:37:42.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.077+0000 7ff8cf09b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8c80cb300 con 0x7ff8c80b1850 2023-10-09T06:37:42.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.077+0000 7ff8cf09b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c806fc30 con 0x7ff8c80b1c20 2023-10-09T06:37:42.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.077+0000 7ff8cd638700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff8c8157820 0x7ff8c80abf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:40994/0 (socket says 172.21.15.163:40994) 2023-10-09T06:37:42.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.077+0000 7ff8cd638700 1 -- 172.21.15.163:0/2781418396 learned_addr learned my addr 172.21.15.163:0/2781418396 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:42.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.078+0000 7ff8c7fff700 1 --2- 172.21.15.163:0/2781418396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8c80b1c20 0x7ff8c81572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:42.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.078+0000 7ff8c77fe700 1 -- 172.21.15.163:0/2781418396 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 315255962 0 0) 0x7ff8c8157c80 con 0x7ff8c8157820 2023-10-09T06:37:42.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.078+0000 7ff8c77fe700 1 -- 172.21.15.163:0/2781418396 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8b8003570 con 0x7ff8c8157820 2023-10-09T06:37:42.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.078+0000 7ff8c77fe700 1 -- 172.21.15.163:0/2781418396 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 439695696 0 0) 0x7ff8c80cb300 con 0x7ff8c80b1850 2023-10-09T06:37:42.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.078+0000 7ff8c77fe700 1 -- 172.21.15.163:0/2781418396 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8c8157c80 con 0x7ff8c80b1850 2023-10-09T06:37:42.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.078+0000 7ff8c7fff700 1 -- 172.21.15.163:0/2781418396 >> v1:172.21.15.181:6789/0 conn(0x7ff8c80b1850 legacy=0x7ff8c80ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:42.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.078+0000 7ff8c7fff700 1 -- 172.21.15.163:0/2781418396 >> v1:172.21.15.163:6789/0 conn(0x7ff8c8157820 legacy=0x7ff8c80abf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:42.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.078+0000 7ff8c7fff700 1 -- 172.21.15.163:0/2781418396 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c80cb5e0 con 0x7ff8c80b1c20 2023-10-09T06:37:42.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.078+0000 7ff8c7fff700 1 --2- 172.21.15.163:0/2781418396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8c80b1c20 0x7ff8c81572e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009910 tx=0x7ff8b40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9cef6f8fe3890f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:42.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.078+0000 7ff8c77fe700 1 -- 172.21.15.163:0/2781418396 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4011630 con 0x7ff8c80b1c20 2023-10-09T06:37:42.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.078+0000 7ff8c77fe700 1 -- 172.21.15.163:0/2781418396 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b4011790 con 0x7ff8c80b1c20 2023-10-09T06:37:42.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.078+0000 7ff8c77fe700 1 -- 172.21.15.163:0/2781418396 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4011b30 con 0x7ff8c80b1c20 2023-10-09T06:37:42.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.079+0000 7ff8cf09b700 1 -- 172.21.15.163:0/2781418396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8c80b1c20 msgr2=0x7ff8c81572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:42.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.079+0000 7ff8cf09b700 1 --2- 172.21.15.163:0/2781418396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8c80b1c20 0x7ff8c81572e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009910 tx=0x7ff8b40091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:42.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.079+0000 7ff8cf09b700 1 -- 172.21.15.163:0/2781418396 shutdown_connections 2023-10-09T06:37:42.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.079+0000 7ff8cf09b700 1 --2- 172.21.15.163:0/2781418396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8c80b1c20 0x7ff8c81572e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:42.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.079+0000 7ff8cf09b700 1 -- 172.21.15.163:0/2781418396 >> 172.21.15.163:0/2781418396 conn(0x7ff8c806a320 msgr2=0x7ff8c80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:42.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.079+0000 7ff8cf09b700 1 -- 172.21.15.163:0/2781418396 shutdown_connections 2023-10-09T06:37:42.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.079+0000 7ff8cf09b700 1 -- 172.21.15.163:0/2781418396 wait complete. 2023-10-09T06:37:42.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.080+0000 7ff8cf09b700 1 Processor -- start 2023-10-09T06:37:42.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.080+0000 7ff8cf09b700 1 -- start start 2023-10-09T06:37:42.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.082+0000 7ff8cf09b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8c80b1850 0x7ff8c813cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:42.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.082+0000 7ff8cf09b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8c80b1c20 0x7ff8c8152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:42.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.082+0000 7ff8cf09b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8c8157820 0x7ff8c80ac9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:42.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.082+0000 7ff8cf09b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c80cc260 con 0x7ff8c8157820 2023-10-09T06:37:42.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.082+0000 7ff8cf09b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c813d4a0 con 0x7ff8c80b1850 2023-10-09T06:37:42.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.082+0000 7ff8cf09b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c816b2f0 con 0x7ff8c80b1c20 2023-10-09T06:37:42.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.082+0000 7ff8cd638700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8c8157820 0x7ff8c80ac9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:42.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.082+0000 7ff8cce37700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8c80b1850 0x7ff8c813cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:42.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.082+0000 7ff8c7fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8c80b1c20 0x7ff8c8152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:42.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.082+0000 7ff8cd638700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8c8157820 0x7ff8c80ac9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:48364/0 (socket says 172.21.15.163:48364) 2023-10-09T06:37:42.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.082+0000 7ff8cd638700 1 -- 172.21.15.163:0/1530553048 learned_addr learned my addr 172.21.15.163:0/1530553048 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:42.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.082+0000 7ff8cd638700 1 -- 172.21.15.163:0/1530553048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8c80b1c20 msgr2=0x7ff8c8152a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:42.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.082+0000 7ff8cd638700 1 --2- 172.21.15.163:0/1530553048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8c80b1c20 0x7ff8c8152a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:42.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.082+0000 7ff8cd638700 1 -- 172.21.15.163:0/1530553048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8c80b1850 msgr2=0x7ff8c813cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:42.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.082+0000 7ff8cd638700 1 --2- 172.21.15.163:0/1530553048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8c80b1850 0x7ff8c813cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:42.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.082+0000 7ff8cd638700 1 -- 172.21.15.163:0/1530553048 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b4009420 con 0x7ff8c8157820 2023-10-09T06:37:42.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.082+0000 7ff8cce37700 1 --2- 172.21.15.163:0/1530553048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8c80b1850 0x7ff8c813cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:37:42.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.082+0000 7ff8cd638700 1 --2- 172.21.15.163:0/1530553048 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8c8157820 0x7ff8c80ac9a0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000cbc0 tx=0x7ff8c0001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:42.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.082+0000 7ff8c4ff9700 1 -- 172.21.15.163:0/1530553048 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c000a7e0 con 0x7ff8c8157820 2023-10-09T06:37:42.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.082+0000 7ff8c4ff9700 1 -- 172.21.15.163:0/1530553048 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c000a940 con 0x7ff8c8157820 2023-10-09T06:37:42.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.082+0000 7ff8cf09b700 1 -- 172.21.15.163:0/1530553048 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c816b5b0 con 0x7ff8c8157820 2023-10-09T06:37:42.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.082+0000 7ff8c4ff9700 1 -- 172.21.15.163:0/1530553048 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c000ad00 con 0x7ff8c8157820 2023-10-09T06:37:42.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.082+0000 7ff8cf09b700 1 -- 172.21.15.163:0/1530553048 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c816bb70 con 0x7ff8c8157820 2023-10-09T06:37:42.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.082+0000 7ff8cf09b700 1 -- 172.21.15.163:0/1530553048 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c804c3c0 con 0x7ff8c8157820 2023-10-09T06:37:42.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.083+0000 7ff8c4ff9700 1 -- 172.21.15.163:0/1530553048 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff8c0007300 con 0x7ff8c8157820 2023-10-09T06:37:42.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.083+0000 7ff8c4ff9700 1 --2- 172.21.15.163:0/1530553048 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff8ac067320 0x7ff8ac0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:42.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.083+0000 7ff8c4ff9700 1 -- 172.21.15.163:0/1530553048 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5111+0+0 (secure 0 0 0) 0x7ff8c0084cc0 con 0x7ff8c8157820 2023-10-09T06:37:42.090 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.083+0000 7ff8cce37700 1 --2- 172.21.15.163:0/1530553048 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff8ac067320 0x7ff8ac0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:42.090 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.084+0000 7ff8cce37700 1 --2- 172.21.15.163:0/1530553048 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff8ac067320 0x7ff8ac0697d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc007fd0 tx=0x7ff8bc007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:42.090 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.088+0000 7ff8c4ff9700 1 -- 172.21.15.163:0/1530553048 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff8c008b080 con 0x7ff8c8157820 2023-10-09T06:37:42.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:42.286+0000 7ff8cf09b700 1 -- 172.21.15.163:0/1530553048 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff8c80cb8c0 con 0x7ff8c8157820 2023-10-09T06:37:43.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.132+0000 7ff8c4ff9700 1 -- 172.21.15.163:0/1530553048 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v133) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff8c0051c20 con 0x7ff8c8157820 2023-10-09T06:37:43.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.135+0000 7ff8cf09b700 1 -- 172.21.15.163:0/1530553048 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff8ac067320 msgr2=0x7ff8ac0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:43.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.135+0000 7ff8cf09b700 1 --2- 172.21.15.163:0/1530553048 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff8ac067320 0x7ff8ac0697d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc007fd0 tx=0x7ff8bc007d50 comp rx=0 tx=0).stop 2023-10-09T06:37:43.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.135+0000 7ff8cf09b700 1 -- 172.21.15.163:0/1530553048 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8c8157820 msgr2=0x7ff8c80ac9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:43.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.135+0000 7ff8cf09b700 1 --2- 172.21.15.163:0/1530553048 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8c8157820 0x7ff8c80ac9a0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000cbc0 tx=0x7ff8c0001a20 comp rx=0 tx=0).stop 2023-10-09T06:37:43.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.135+0000 7ff8cf09b700 1 -- 172.21.15.163:0/1530553048 shutdown_connections 2023-10-09T06:37:43.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.135+0000 7ff8cf09b700 1 --2- 172.21.15.163:0/1530553048 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff8ac067320 0x7ff8ac0697d0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:43.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.135+0000 7ff8cf09b700 1 --2- 172.21.15.163:0/1530553048 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8c8157820 0x7ff8c80ac9a0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:43.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.135+0000 7ff8cf09b700 1 --2- 172.21.15.163:0/1530553048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8c80b1850 0x7ff8c813cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:43.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.135+0000 7ff8cf09b700 1 --2- 172.21.15.163:0/1530553048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8c80b1c20 0x7ff8c8152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:43.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.135+0000 7ff8cf09b700 1 -- 172.21.15.163:0/1530553048 >> 172.21.15.163:0/1530553048 conn(0x7ff8c806a320 msgr2=0x7ff8c8155c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:43.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.135+0000 7ff8cf09b700 1 -- 172.21.15.163:0/1530553048 shutdown_connections 2023-10-09T06:37:43.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.135+0000 7ff8cf09b700 1 -- 172.21.15.163:0/1530553048 wait complete. 2023-10-09T06:37:43.136 INFO:teuthology.orchestra.run.smithi163.stderr: removed all blocklist entries 2023-10-09T06:37:43.148 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-09T06:37:43.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.264+0000 7f5b41a05700 1 Processor -- start 2023-10-09T06:37:43.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.264+0000 7f5b41a05700 1 -- start start 2023-10-09T06:37:43.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.265+0000 7f5b41a05700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5b3c159c60 0x7f5b3c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:43.267 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.265+0000 7f5b41a05700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5b3c15db10 0x7f5b3c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:43.267 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.265+0000 7f5b41a05700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b3c15a160 con 0x7f5b3c159890 2023-10-09T06:37:43.267 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.265+0000 7f5b41a05700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b3c15df50 con 0x7f5b3c159c60 2023-10-09T06:37:43.267 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.265+0000 7f5b41a05700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b3c0d13a0 con 0x7f5b3c15db10 2023-10-09T06:37:43.267 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.265+0000 7f5b327fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5b3c159c60 0x7f5b3c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:43.267 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.265+0000 7f5b327fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5b3c159c60 0x7f5b3c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:48374/0 (socket says 172.21.15.163:48374) 2023-10-09T06:37:43.267 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.265+0000 7f5b327fc700 1 -- 172.21.15.163:0/2387868616 learned_addr learned my addr 172.21.15.163:0/2387868616 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:43.267 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.265+0000 7f5b3b7fe700 1 --2- 172.21.15.163:0/2387868616 >> v2:172.21.15.181:3300/0 conn(0x7f5b3c15db10 0x7f5b3c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:43.267 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.266+0000 7f5b327fc700 1 -- 172.21.15.163:0/2387868616 >> v2:172.21.15.181:3300/0 conn(0x7f5b3c15db10 msgr2=0x7f5b3c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:43.267 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.266+0000 7f5b327fc700 1 --2- 172.21.15.163:0/2387868616 >> v2:172.21.15.181:3300/0 conn(0x7f5b3c15db10 0x7f5b3c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:43.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.266+0000 7f5b327fc700 1 -- 172.21.15.163:0/2387868616 >> v1:172.21.15.181:6789/0 conn(0x7f5b3c159890 legacy=0x7f5b3c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:43.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.266+0000 7f5b327fc700 1 -- 172.21.15.163:0/2387868616 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3c15e140 con 0x7f5b3c159c60 2023-10-09T06:37:43.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.266+0000 7f5b3b7fe700 1 --2- 172.21.15.163:0/2387868616 >> v2:172.21.15.181:3300/0 conn(0x7f5b3c15db10 0x7f5b3c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:43.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.266+0000 7f5b327fc700 1 --2- 172.21.15.163:0/2387868616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b3c159c60 0x7f5b3c15d5d0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c06a890 tx=0x7f5b24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd47771ee8b325c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:43.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.266+0000 7f5b3a7fc700 1 -- 172.21.15.163:0/2387868616 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2400b860 con 0x7f5b3c159c60 2023-10-09T06:37:43.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.266+0000 7f5b3a7fc700 1 -- 172.21.15.163:0/2387868616 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b2400b9c0 con 0x7f5b3c159c60 2023-10-09T06:37:43.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.266+0000 7f5b3a7fc700 1 -- 172.21.15.163:0/2387868616 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2400bd60 con 0x7f5b3c159c60 2023-10-09T06:37:43.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.266+0000 7f5b41a05700 1 -- 172.21.15.163:0/2387868616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b3c159c60 msgr2=0x7f5b3c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:43.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.266+0000 7f5b41a05700 1 --2- 172.21.15.163:0/2387868616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b3c159c60 0x7f5b3c15d5d0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c06a890 tx=0x7f5b24009250 comp rx=0 tx=0).stop 2023-10-09T06:37:43.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.266+0000 7f5b41a05700 1 -- 172.21.15.163:0/2387868616 shutdown_connections 2023-10-09T06:37:43.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.266+0000 7f5b41a05700 1 --2- 172.21.15.163:0/2387868616 >> v2:172.21.15.181:3300/0 conn(0x7f5b3c15db10 0x7f5b3c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:43.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.266+0000 7f5b41a05700 1 --2- 172.21.15.163:0/2387868616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b3c159c60 0x7f5b3c15d5d0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:43.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.266+0000 7f5b41a05700 1 -- 172.21.15.163:0/2387868616 >> 172.21.15.163:0/2387868616 conn(0x7f5b3c068e20 msgr2=0x7f5b3c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:43.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.267+0000 7f5b41a05700 1 -- 172.21.15.163:0/2387868616 shutdown_connections 2023-10-09T06:37:43.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.267+0000 7f5b41a05700 1 -- 172.21.15.163:0/2387868616 wait complete. 2023-10-09T06:37:43.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.268+0000 7f5b41a05700 1 Processor -- start 2023-10-09T06:37:43.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.268+0000 7f5b41a05700 1 -- start start 2023-10-09T06:37:43.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.268+0000 7f5b41a05700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b3c159890 0x7f5b3c14caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:43.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.268+0000 7f5b41a05700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b3c159c60 0x7f5b3c0cf030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:43.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.268+0000 7f5b41a05700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b3c15db10 0x7f5b3c15caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:43.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.269+0000 7f5b41a05700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c16bde0 con 0x7f5b3c159c60 2023-10-09T06:37:43.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.269+0000 7f5b41a05700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c16b1d0 con 0x7f5b3c159890 2023-10-09T06:37:43.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.269+0000 7f5b41a05700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c16b4b0 con 0x7f5b3c15db10 2023-10-09T06:37:43.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.269+0000 7f5b327fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b3c159c60 0x7f5b3c0cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:43.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.269+0000 7f5b327fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b3c159c60 0x7f5b3c0cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:48378/0 (socket says 172.21.15.163:48378) 2023-10-09T06:37:43.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.269+0000 7f5b327fc700 1 -- 172.21.15.163:0/3570944956 learned_addr learned my addr 172.21.15.163:0/3570944956 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:43.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.269+0000 7f5b3affd700 1 --2- 172.21.15.163:0/3570944956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b3c159890 0x7f5b3c14caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:43.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.269+0000 7f5b3b7fe700 1 --2- 172.21.15.163:0/3570944956 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b3c15db10 0x7f5b3c15caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:43.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.269+0000 7f5b327fc700 1 -- 172.21.15.163:0/3570944956 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b3c15db10 msgr2=0x7f5b3c15caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:43.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.269+0000 7f5b327fc700 1 --2- 172.21.15.163:0/3570944956 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b3c15db10 0x7f5b3c15caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:43.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.269+0000 7f5b327fc700 1 -- 172.21.15.163:0/3570944956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b3c159890 msgr2=0x7f5b3c14caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:43.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.269+0000 7f5b327fc700 1 --2- 172.21.15.163:0/3570944956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b3c159890 0x7f5b3c14caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:43.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.269+0000 7f5b327fc700 1 -- 172.21.15.163:0/3570944956 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b240094f0 con 0x7f5b3c159c60 2023-10-09T06:37:43.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.269+0000 7f5b327fc700 1 --2- 172.21.15.163:0/3570944956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b3c159c60 0x7f5b3c0cf030 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f5b24009290 tx=0x7f5b24017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:43.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.269+0000 7f5b337fe700 1 -- 172.21.15.163:0/3570944956 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b24026480 con 0x7f5b3c159c60 2023-10-09T06:37:43.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.269+0000 7f5b337fe700 1 -- 172.21.15.163:0/3570944956 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b2401f070 con 0x7f5b3c159c60 2023-10-09T06:37:43.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.269+0000 7f5b337fe700 1 -- 172.21.15.163:0/3570944956 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b24026850 con 0x7f5b3c159c60 2023-10-09T06:37:43.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.269+0000 7f5b41a05700 1 -- 172.21.15.163:0/3570944956 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b3c16b790 con 0x7f5b3c159c60 2023-10-09T06:37:43.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.269+0000 7f5b41a05700 1 -- 172.21.15.163:0/3570944956 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b3c0b3590 con 0x7f5b3c159c60 2023-10-09T06:37:43.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.270+0000 7f5b41a05700 1 -- 172.21.15.163:0/3570944956 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b3c04c3c0 con 0x7f5b3c159c60 2023-10-09T06:37:43.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.271+0000 7f5b337fe700 1 -- 172.21.15.163:0/3570944956 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5b2401a430 con 0x7f5b3c159c60 2023-10-09T06:37:43.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.271+0000 7f5b337fe700 1 --2- 172.21.15.163:0/3570944956 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5b1c067320 0x7f5b1c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:43.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.271+0000 7f5b337fe700 1 -- 172.21.15.163:0/3570944956 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f5b24021070 con 0x7f5b3c159c60 2023-10-09T06:37:43.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.271+0000 7f5b3affd700 1 --2- 172.21.15.163:0/3570944956 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5b1c067320 0x7f5b1c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:43.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.272+0000 7f5b3affd700 1 --2- 172.21.15.163:0/3570944956 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5b1c067320 0x7f5b1c0697d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c0ce190 tx=0x7f5b2c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:43.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.275+0000 7f5b337fe700 1 -- 172.21.15.163:0/3570944956 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5b24053870 con 0x7f5b3c159c60 2023-10-09T06:37:43.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.529+0000 7f5b41a05700 1 -- 172.21.15.163:0/3570944956 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5b3c0b1aa0 con 0x7f5b3c159c60 2023-10-09T06:37:43.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.529+0000 7f5b337fe700 1 -- 172.21.15.163:0/3570944956 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5b24015090 con 0x7f5b3c159c60 2023-10-09T06:37:43.530 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:43.530 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:37:43.530 INFO:teuthology.orchestra.run.smithi163.stdout: "auth_dump": [ 2023-10-09T06:37:43.530 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:43.530 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.a", 2023-10-09T06:37:43.530 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNltXtwDxAAlnrWRsfnZDLEmsJVowh3Mg==", 2023-10-09T06:37:43.530 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:43.531 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:37:43.531 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:37:43.531 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:37:43.531 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:43.531 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:43.531 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:43.531 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:43.531 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.b", 2023-10-09T06:37:43.531 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlTdGhGhAAXuldy0HrFd+44sSu/EV3qA==", 2023-10-09T06:37:43.532 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:43.532 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:37:43.532 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:37:43.532 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:37:43.532 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:43.532 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:43.532 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:43.532 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:43.532 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.c", 2023-10-09T06:37:43.532 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlGis7FhAAYG76TiOFX8QkTwjNX2+Ncw==", 2023-10-09T06:37:43.533 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:43.533 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:37:43.533 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:37:43.533 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:37:43.533 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:43.533 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:43.533 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:43.533 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:43.533 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.d", 2023-10-09T06:37:43.533 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlKhRuIRAAfhG5Gpc7klXJloYXepOhsg==", 2023-10-09T06:37:43.534 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:43.534 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:37:43.534 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:37:43.534 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:37:43.534 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:43.534 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:43.534 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:43.534 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:43.535 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.0", 2023-10-09T06:37:43.535 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDcnSNlrQeFOxAA+s6UitrTn5TK1N7kgySoXw==", 2023-10-09T06:37:43.535 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:43.535 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:37:43.535 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:37:43.535 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:43.535 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:43.535 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:43.535 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:43.535 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.1", 2023-10-09T06:37:43.535 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDfnSNlleccORAAnB+lNUFblivdoY44JfELxA==", 2023-10-09T06:37:43.536 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:43.536 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:37:43.536 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:37:43.536 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:43.536 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:43.536 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:43.536 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:43.536 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.2", 2023-10-09T06:37:43.536 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDinSNlFoB4NxAAT0+trRBij5rqgc+F0eWUfQ==", 2023-10-09T06:37:43.536 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:43.537 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:37:43.537 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:37:43.537 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:43.537 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:43.537 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:43.537 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:43.537 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.3", 2023-10-09T06:37:43.537 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDlnSNlRRQ6NhAABoBtrFjSvQujolq0Pxrd1Q==", 2023-10-09T06:37:43.537 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:43.537 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:37:43.538 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:37:43.538 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:43.538 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:43.538 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:43.538 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:43.538 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.4", 2023-10-09T06:37:43.538 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDrnSNl308uOxAA5OJyPLGQmVSJqha5d+nKKQ==", 2023-10-09T06:37:43.538 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:43.538 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:37:43.539 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:37:43.539 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:43.539 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:43.539 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:43.539 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:43.539 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.5", 2023-10-09T06:37:43.539 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDunSNlHnK+OBAAU7l4dCJ82SbGSzBEm8Axzw==", 2023-10-09T06:37:43.539 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:43.539 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:37:43.539 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:37:43.539 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:43.540 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:43.540 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:43.540 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:43.540 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.6", 2023-10-09T06:37:43.540 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDxnSNlwUSLOBAAN5N+F5ho+Co8pBh4Rz+lXQ==", 2023-10-09T06:37:43.540 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:43.540 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:37:43.540 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:37:43.540 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:43.540 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:43.541 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:43.541 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:43.541 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.7", 2023-10-09T06:37:43.541 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQD0nSNl2gxGORAAoE8OLGEifcEx/km8rfSV9g==", 2023-10-09T06:37:43.541 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:43.541 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:37:43.541 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:37:43.541 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:43.542 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:43.542 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:43.542 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:43.542 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.0", 2023-10-09T06:37:43.542 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlnxY5JRAApr6czcpMdQYYtXQ9yg2q6A==", 2023-10-09T06:37:43.542 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:43.542 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:37:43.542 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow r", 2023-10-09T06:37:43.542 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-09T06:37:43.542 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:43.543 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:43.543 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:43.543 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.1", 2023-10-09T06:37:43.543 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlk6ybLRAAuShqTDAtymGdxN8KBdZDDA==", 2023-10-09T06:37:43.543 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:43.543 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:37:43.543 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:37:43.543 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:37:43.543 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:37:43.544 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:43.544 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:43.544 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:43.544 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.2", 2023-10-09T06:37:43.544 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlYyOmKhAAQmpr+mGfJH4GUa28dtlJwA==", 2023-10-09T06:37:43.544 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:43.544 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:37:43.544 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:37:43.544 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:37:43.545 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:37:43.545 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:43.545 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:43.545 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:43.545 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.3", 2023-10-09T06:37:43.545 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlD2H1MhAA4zvaXY/Gd0kndzi+QSwp7w==", 2023-10-09T06:37:43.545 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:43.545 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:37:43.545 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:37:43.545 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:37:43.545 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:37:43.546 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:43.546 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:43.546 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:43.546 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.admin", 2023-10-09T06:37:43.546 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDYnSNlsHH4LxAA85xeFEP1CUwoblaePYk5zA==", 2023-10-09T06:37:43.546 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:43.546 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:37:43.546 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:37:43.546 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:37:43.547 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:43.547 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:43.547 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:43.547 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:43.547 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mgr.x", 2023-10-09T06:37:43.547 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNl2sloDBAA0FoWN9iovCl6783o3/btiA==", 2023-10-09T06:37:43.547 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:43.547 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:37:43.547 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile mgr", 2023-10-09T06:37:43.547 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:43.547 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:43.548 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:37:43.548 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:43.548 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mgr.y", 2023-10-09T06:37:43.548 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlcFWVCBAAR2dKPg01ciTiNpSX0JIKFg==", 2023-10-09T06:37:43.548 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:37:43.548 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:37:43.548 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile mgr", 2023-10-09T06:37:43.548 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:37:43.548 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:43.548 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:43.549 INFO:teuthology.orchestra.run.smithi163.stdout: ] 2023-10-09T06:37:43.549 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:37:43.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.531+0000 7f5b41a05700 1 -- 172.21.15.163:0/3570944956 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5b1c067320 msgr2=0x7f5b1c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:43.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.531+0000 7f5b41a05700 1 --2- 172.21.15.163:0/3570944956 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5b1c067320 0x7f5b1c0697d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c0ce190 tx=0x7f5b2c009040 comp rx=0 tx=0).stop 2023-10-09T06:37:43.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.531+0000 7f5b41a05700 1 -- 172.21.15.163:0/3570944956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b3c159c60 msgr2=0x7f5b3c0cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:43.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.531+0000 7f5b41a05700 1 --2- 172.21.15.163:0/3570944956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b3c159c60 0x7f5b3c0cf030 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f5b24009290 tx=0x7f5b24017000 comp rx=0 tx=0).stop 2023-10-09T06:37:43.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.532+0000 7f5b41a05700 1 -- 172.21.15.163:0/3570944956 shutdown_connections 2023-10-09T06:37:43.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.532+0000 7f5b41a05700 1 --2- 172.21.15.163:0/3570944956 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5b1c067320 0x7f5b1c0697d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:43.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.532+0000 7f5b41a05700 1 --2- 172.21.15.163:0/3570944956 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b3c15db10 0x7f5b3c15caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:43.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.532+0000 7f5b41a05700 1 --2- 172.21.15.163:0/3570944956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b3c159890 0x7f5b3c14caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:43.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.532+0000 7f5b41a05700 1 --2- 172.21.15.163:0/3570944956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b3c159c60 0x7f5b3c0cf030 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:43.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.532+0000 7f5b41a05700 1 -- 172.21.15.163:0/3570944956 >> 172.21.15.163:0/3570944956 conn(0x7f5b3c068e20 msgr2=0x7f5b3c0c6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:43.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.532+0000 7f5b41a05700 1 -- 172.21.15.163:0/3570944956 shutdown_connections 2023-10-09T06:37:43.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.532+0000 7f5b41a05700 1 -- 172.21.15.163:0/3570944956 wait complete. 2023-10-09T06:37:43.553 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:37:43.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.668+0000 7f69a561e700 1 Processor -- start 2023-10-09T06:37:43.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.668+0000 7f69a561e700 1 -- start start 2023-10-09T06:37:43.671 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.669+0000 7f69a561e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f69a0155c10 0x7f69a0155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:43.671 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.669+0000 7f69a561e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69a00ac230 0x7f69a00ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:43.671 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.669+0000 7f69a561e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f69a01568a0 0x7f69a0156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:43.671 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.669+0000 7f69a561e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69a00ad930 con 0x7f69a01568a0 2023-10-09T06:37:43.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.669+0000 7f69a561e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69a00acd20 con 0x7f69a0155c10 2023-10-09T06:37:43.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.669+0000 7f69a561e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69a00ad000 con 0x7f69a00ac230 2023-10-09T06:37:43.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.669+0000 7f699f7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f69a01568a0 0x7f69a0156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:43.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.669+0000 7f699f7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f69a01568a0 0x7f69a0156cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:48386/0 (socket says 172.21.15.163:48386) 2023-10-09T06:37:43.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.669+0000 7f699f7fe700 1 -- 172.21.15.163:0/4041481996 learned_addr learned my addr 172.21.15.163:0/4041481996 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:43.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.670+0000 7f699effd700 1 --2- 172.21.15.163:0/4041481996 >> v2:172.21.15.181:3300/0 conn(0x7f69a0155c10 0x7f69a0155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:43.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.670+0000 7f69967fc700 1 --2- 172.21.15.163:0/4041481996 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69a00ac230 0x7f69a00ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:43.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.670+0000 7f699f7fe700 1 -- 172.21.15.163:0/4041481996 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69a00ac230 msgr2=0x7f69a00ac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:43.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.670+0000 7f699f7fe700 1 --2- 172.21.15.163:0/4041481996 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69a00ac230 0x7f69a00ac6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:43.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.670+0000 7f699f7fe700 1 -- 172.21.15.163:0/4041481996 >> v2:172.21.15.181:3300/0 conn(0x7f69a0155c10 msgr2=0x7f69a0155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:43.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.670+0000 7f699f7fe700 1 --2- 172.21.15.163:0/4041481996 >> v2:172.21.15.181:3300/0 conn(0x7f69a0155c10 0x7f69a0155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:43.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.670+0000 7f699f7fe700 1 -- 172.21.15.163:0/4041481996 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a00ad2e0 con 0x7f69a01568a0 2023-10-09T06:37:43.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.670+0000 7f69967fc700 1 --2- 172.21.15.163:0/4041481996 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69a00ac230 0x7f69a00ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:43.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.670+0000 7f699effd700 1 --2- 172.21.15.163:0/4041481996 >> v2:172.21.15.181:3300/0 conn(0x7f69a0155c10 0x7f69a0155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:43.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.670+0000 7f699f7fe700 1 --2- 172.21.15.163:0/4041481996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69a01568a0 0x7f69a0156cc0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f69a0063d40 tx=0x7f6998009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57bdee0a8db78f22 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:43.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.670+0000 7f699e7fc700 1 -- 172.21.15.163:0/4041481996 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699800e070 con 0x7f69a01568a0 2023-10-09T06:37:43.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.670+0000 7f699e7fc700 1 -- 172.21.15.163:0/4041481996 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f699800ba50 con 0x7f69a01568a0 2023-10-09T06:37:43.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.670+0000 7f699e7fc700 1 -- 172.21.15.163:0/4041481996 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69980092e0 con 0x7f69a01568a0 2023-10-09T06:37:43.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.671+0000 7f69a561e700 1 -- 172.21.15.163:0/4041481996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69a01568a0 msgr2=0x7f69a0156cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:43.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.671+0000 7f69a561e700 1 --2- 172.21.15.163:0/4041481996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69a01568a0 0x7f69a0156cc0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f69a0063d40 tx=0x7f6998009840 comp rx=0 tx=0).stop 2023-10-09T06:37:43.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.671+0000 7f69a561e700 1 -- 172.21.15.163:0/4041481996 shutdown_connections 2023-10-09T06:37:43.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.671+0000 7f69a561e700 1 --2- 172.21.15.163:0/4041481996 >> v2:172.21.15.181:3300/0 conn(0x7f69a0155c10 0x7f69a0155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:43.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.671+0000 7f69a561e700 1 --2- 172.21.15.163:0/4041481996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69a01568a0 0x7f69a0156cc0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:43.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.671+0000 7f69a561e700 1 --2- 172.21.15.163:0/4041481996 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69a00ac230 0x7f69a00ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:43.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.671+0000 7f69a561e700 1 -- 172.21.15.163:0/4041481996 >> 172.21.15.163:0/4041481996 conn(0x7f69a006a320 msgr2=0x7f69a00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:43.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.671+0000 7f69a561e700 1 -- 172.21.15.163:0/4041481996 shutdown_connections 2023-10-09T06:37:43.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.671+0000 7f69a561e700 1 -- 172.21.15.163:0/4041481996 wait complete. 2023-10-09T06:37:43.678 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.672+0000 7f69a561e700 1 Processor -- start 2023-10-09T06:37:43.678 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.672+0000 7f69a561e700 1 -- start start 2023-10-09T06:37:43.678 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.673+0000 7f69a561e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69a00ac230 0x7f69a01513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:43.678 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.673+0000 7f69a561e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69a0155c10 0x7f69a0169ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:43.678 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.673+0000 7f69a561e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69a01568a0 0x7f69a00cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:43.678 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.673+0000 7f69a561e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a0156ca0 con 0x7f69a01568a0 2023-10-09T06:37:43.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.673+0000 7f69a561e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a01574d0 con 0x7f69a00ac230 2023-10-09T06:37:43.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.673+0000 7f69a561e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69a00cca10 con 0x7f69a0155c10 2023-10-09T06:37:43.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.673+0000 7f699f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69a01568a0 0x7f69a00cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:43.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.673+0000 7f699f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69a01568a0 0x7f69a00cbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:48396/0 (socket says 172.21.15.163:48396) 2023-10-09T06:37:43.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.673+0000 7f699f7fe700 1 -- 172.21.15.163:0/3529004623 learned_addr learned my addr 172.21.15.163:0/3529004623 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:43.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.673+0000 7f69967fc700 1 --2- 172.21.15.163:0/3529004623 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69a0155c10 0x7f69a0169ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:43.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.673+0000 7f699effd700 1 --2- 172.21.15.163:0/3529004623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69a00ac230 0x7f69a01513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:43.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.673+0000 7f699f7fe700 1 -- 172.21.15.163:0/3529004623 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69a0155c10 msgr2=0x7f69a0169ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:43.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.673+0000 7f699f7fe700 1 --2- 172.21.15.163:0/3529004623 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69a0155c10 0x7f69a0169ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:43.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.673+0000 7f699f7fe700 1 -- 172.21.15.163:0/3529004623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69a00ac230 msgr2=0x7f69a01513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:43.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.673+0000 7f699f7fe700 1 --2- 172.21.15.163:0/3529004623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69a00ac230 0x7f69a01513d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:43.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.673+0000 7f699f7fe700 1 -- 172.21.15.163:0/3529004623 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6998009ae0 con 0x7f69a01568a0 2023-10-09T06:37:43.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.673+0000 7f699f7fe700 1 --2- 172.21.15.163:0/3529004623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69a01568a0 0x7f69a00cbf30 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f69a0064290 tx=0x7f6998018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:43.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.673+0000 7f69977fe700 1 -- 172.21.15.163:0/3529004623 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6998023bb0 con 0x7f69a01568a0 2023-10-09T06:37:43.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.673+0000 7f69977fe700 1 -- 172.21.15.163:0/3529004623 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6998027070 con 0x7f69a01568a0 2023-10-09T06:37:43.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.673+0000 7f69a561e700 1 -- 172.21.15.163:0/3529004623 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69a0156e80 con 0x7f69a01568a0 2023-10-09T06:37:43.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.673+0000 7f69977fe700 1 -- 172.21.15.163:0/3529004623 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6998008950 con 0x7f69a01568a0 2023-10-09T06:37:43.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.674+0000 7f69a561e700 1 -- 172.21.15.163:0/3529004623 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69a00c7ed0 con 0x7f69a01568a0 2023-10-09T06:37:43.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.674+0000 7f69a561e700 1 -- 172.21.15.163:0/3529004623 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69a0157680 con 0x7f69a01568a0 2023-10-09T06:37:43.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.675+0000 7f69977fe700 1 -- 172.21.15.163:0/3529004623 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6998029030 con 0x7f69a01568a0 2023-10-09T06:37:43.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.675+0000 7f69977fe700 1 --2- 172.21.15.163:0/3529004623 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6980067320 0x7f69800697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:43.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.675+0000 7f69977fe700 1 -- 172.21.15.163:0/3529004623 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f699808e880 con 0x7f69a01568a0 2023-10-09T06:37:43.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.675+0000 7f699effd700 1 --2- 172.21.15.163:0/3529004623 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6980067320 0x7f69800697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:43.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.676+0000 7f699effd700 1 --2- 172.21.15.163:0/3529004623 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6980067320 0x7f69800697d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6990006fd0 tx=0x7f6990008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:43.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.680+0000 7f69977fe700 1 -- 172.21.15.163:0/3529004623 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f69a0157680 con 0x7f69a01568a0 2023-10-09T06:37:43.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.864+0000 7f69a561e700 1 -- 172.21.15.163:0/3529004623 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69a0157640 con 0x7f69a01568a0 2023-10-09T06:37:43.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.865+0000 7f69977fe700 1 -- 172.21.15.163:0/3529004623 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+12192 (secure 0 0 0) 0x7f699805b910 con 0x7f69a01568a0 2023-10-09T06:37:43.864 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:43.865 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":133,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:37:43.125355+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:37:43.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.867+0000 7f69a561e700 1 -- 172.21.15.163:0/3529004623 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6980067320 msgr2=0x7f69800697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:43.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.867+0000 7f69a561e700 1 --2- 172.21.15.163:0/3529004623 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6980067320 0x7f69800697d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6990006fd0 tx=0x7f6990008040 comp rx=0 tx=0).stop 2023-10-09T06:37:43.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.867+0000 7f69a561e700 1 -- 172.21.15.163:0/3529004623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69a01568a0 msgr2=0x7f69a00cbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:43.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.867+0000 7f69a561e700 1 --2- 172.21.15.163:0/3529004623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69a01568a0 0x7f69a00cbf30 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f69a0064290 tx=0x7f6998018000 comp rx=0 tx=0).stop 2023-10-09T06:37:43.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.867+0000 7f69a561e700 1 -- 172.21.15.163:0/3529004623 shutdown_connections 2023-10-09T06:37:43.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.867+0000 7f69a561e700 1 --2- 172.21.15.163:0/3529004623 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6980067320 0x7f69800697d0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:43.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.867+0000 7f69a561e700 1 --2- 172.21.15.163:0/3529004623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f69a01568a0 0x7f69a00cbf30 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:43.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.867+0000 7f69a561e700 1 --2- 172.21.15.163:0/3529004623 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69a00ac230 0x7f69a01513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:43.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.867+0000 7f69a561e700 1 --2- 172.21.15.163:0/3529004623 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69a0155c10 0x7f69a0169ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:43.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.867+0000 7f69a561e700 1 -- 172.21.15.163:0/3529004623 >> 172.21.15.163:0/3529004623 conn(0x7f69a006a320 msgr2=0x7f69a006de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:43.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.867+0000 7f69a561e700 1 -- 172.21.15.163:0/3529004623 shutdown_connections 2023-10-09T06:37:43.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.867+0000 7f69a561e700 1 -- 172.21.15.163:0/3529004623 wait complete. 2023-10-09T06:37:43.879 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-09T06:37:43.879 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:37:43.991 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:37:43.994 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:37:43.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.998+0000 7fb6565a0700 1 Processor -- start 2023-10-09T06:37:43.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.998+0000 7fb6565a0700 1 -- start start 2023-10-09T06:37:44.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.999+0000 7fb6565a0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb650159a70 0x7fb6500c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:44.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.999+0000 7fb6565a0700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb65006bfa0 0x7fb6500c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:44.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.999+0000 7fb6565a0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6500ca920 con 0x7fb650068390 2023-10-09T06:37:44.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.999+0000 7fb6565a0700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb65006af00 con 0x7fb65006bfa0 2023-10-09T06:37:44.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.999+0000 7fb6565a0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb65006b1e0 con 0x7fb650159a70 2023-10-09T06:37:44.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.999+0000 7fb654d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb65006bfa0 0x7fb6500c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:44.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.999+0000 7fb654d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb65006bfa0 0x7fb6500c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:48398/0 (socket says 172.21.15.163:48398) 2023-10-09T06:37:44.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.999+0000 7fb654d9d700 1 -- 172.21.15.163:0/298567481 learned_addr learned my addr 172.21.15.163:0/298567481 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:44.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:43.999+0000 7fb65559e700 1 --2- 172.21.15.163:0/298567481 >> v2:172.21.15.181:3300/0 conn(0x7fb650159a70 0x7fb6500c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:44.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.000+0000 7fb654d9d700 1 -- 172.21.15.163:0/298567481 >> v2:172.21.15.181:3300/0 conn(0x7fb650159a70 msgr2=0x7fb6500c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:44.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.000+0000 7fb654d9d700 1 --2- 172.21.15.163:0/298567481 >> v2:172.21.15.181:3300/0 conn(0x7fb650159a70 0x7fb6500c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:44.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.000+0000 7fb654d9d700 1 -- 172.21.15.163:0/298567481 >> v1:172.21.15.181:6789/0 conn(0x7fb650068390 legacy=0x7fb6500caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:44.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.000+0000 7fb654d9d700 1 -- 172.21.15.163:0/298567481 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65006b4c0 con 0x7fb65006bfa0 2023-10-09T06:37:44.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.000+0000 7fb654d9d700 1 --2- 172.21.15.163:0/298567481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb65006bfa0 0x7fb6500c8830 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fb6500640b0 tx=0x7fb640009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794382b41967a8a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:44.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.000+0000 7fb647fff700 1 -- 172.21.15.163:0/298567481 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb640005c10 con 0x7fb65006bfa0 2023-10-09T06:37:44.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.000+0000 7fb647fff700 1 -- 172.21.15.163:0/298567481 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb640005d70 con 0x7fb65006bfa0 2023-10-09T06:37:44.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.000+0000 7fb647fff700 1 -- 172.21.15.163:0/298567481 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb640006260 con 0x7fb65006bfa0 2023-10-09T06:37:44.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.001+0000 7fb6565a0700 1 -- 172.21.15.163:0/298567481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb65006bfa0 msgr2=0x7fb6500c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:44.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.001+0000 7fb6565a0700 1 --2- 172.21.15.163:0/298567481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb65006bfa0 0x7fb6500c8830 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fb6500640b0 tx=0x7fb640009220 comp rx=0 tx=0).stop 2023-10-09T06:37:44.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.001+0000 7fb6565a0700 1 -- 172.21.15.163:0/298567481 shutdown_connections 2023-10-09T06:37:44.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.001+0000 7fb6565a0700 1 --2- 172.21.15.163:0/298567481 >> v2:172.21.15.181:3300/0 conn(0x7fb650159a70 0x7fb6500c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:44.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.001+0000 7fb6565a0700 1 --2- 172.21.15.163:0/298567481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb65006bfa0 0x7fb6500c8830 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:44.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.001+0000 7fb6565a0700 1 -- 172.21.15.163:0/298567481 >> 172.21.15.163:0/298567481 conn(0x7fb6500cddc0 msgr2=0x7fb6500c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:44.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.001+0000 7fb6565a0700 1 -- 172.21.15.163:0/298567481 shutdown_connections 2023-10-09T06:37:44.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.001+0000 7fb6565a0700 1 -- 172.21.15.163:0/298567481 wait complete. 2023-10-09T06:37:44.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.002+0000 7fb6565a0700 1 Processor -- start 2023-10-09T06:37:44.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.002+0000 7fb6565a0700 1 -- start start 2023-10-09T06:37:44.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.003+0000 7fb6565a0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb650068390 0x7fb650147f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:44.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.003+0000 7fb6565a0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb65006bfa0 0x7fb6500b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:44.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.003+0000 7fb6565a0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb650159a70 0x7fb6500c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:44.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.003+0000 7fb6565a0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb650156d30 con 0x7fb650068390 2023-10-09T06:37:44.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.003+0000 7fb6565a0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb650155e40 con 0x7fb650159a70 2023-10-09T06:37:44.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.003+0000 7fb6565a0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb650156120 con 0x7fb65006bfa0 2023-10-09T06:37:44.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.003+0000 7fb65559e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb650068390 0x7fb650147f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:44.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.003+0000 7fb65559e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb650068390 0x7fb650147f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:48406/0 (socket says 172.21.15.163:48406) 2023-10-09T06:37:44.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.003+0000 7fb65559e700 1 -- 172.21.15.163:0/2941288399 learned_addr learned my addr 172.21.15.163:0/2941288399 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:44.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.003+0000 7fb655d9f700 1 --2- 172.21.15.163:0/2941288399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb650159a70 0x7fb6500c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:44.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.003+0000 7fb654d9d700 1 --2- 172.21.15.163:0/2941288399 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb65006bfa0 0x7fb6500b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:44.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.003+0000 7fb65559e700 1 -- 172.21.15.163:0/2941288399 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb65006bfa0 msgr2=0x7fb6500b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:44.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.003+0000 7fb65559e700 1 --2- 172.21.15.163:0/2941288399 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb65006bfa0 0x7fb6500b2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:44.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.003+0000 7fb65559e700 1 -- 172.21.15.163:0/2941288399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb650159a70 msgr2=0x7fb6500c64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:44.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.003+0000 7fb65559e700 1 --2- 172.21.15.163:0/2941288399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb650159a70 0x7fb6500c64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:44.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.003+0000 7fb65559e700 1 -- 172.21.15.163:0/2941288399 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb640009480 con 0x7fb650068390 2023-10-09T06:37:44.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.003+0000 7fb654d9d700 1 --2- 172.21.15.163:0/2941288399 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb65006bfa0 0x7fb6500b2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:44.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.003+0000 7fb65559e700 1 --2- 172.21.15.163:0/2941288399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb650068390 0x7fb650147f40 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00e930 tx=0x7fb64c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:44.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.003+0000 7fb6457fa700 1 -- 172.21.15.163:0/2941288399 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c013680 con 0x7fb650068390 2023-10-09T06:37:44.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.003+0000 7fb6457fa700 1 -- 172.21.15.163:0/2941288399 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64c0137e0 con 0x7fb650068390 2023-10-09T06:37:44.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.003+0000 7fb6565a0700 1 -- 172.21.15.163:0/2941288399 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb650156400 con 0x7fb650068390 2023-10-09T06:37:44.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.003+0000 7fb6457fa700 1 -- 172.21.15.163:0/2941288399 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c013b80 con 0x7fb650068390 2023-10-09T06:37:44.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.003+0000 7fb6565a0700 1 -- 172.21.15.163:0/2941288399 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6501569c0 con 0x7fb650068390 2023-10-09T06:37:44.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.005+0000 7fb6565a0700 1 -- 172.21.15.163:0/2941288399 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb65006d560 con 0x7fb650068390 2023-10-09T06:37:44.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.005+0000 7fb6457fa700 1 -- 172.21.15.163:0/2941288399 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb64c00c460 con 0x7fb650068390 2023-10-09T06:37:44.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.005+0000 7fb6457fa700 1 --2- 172.21.15.163:0/2941288399 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb634067320 0x7fb6340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:44.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.005+0000 7fb6457fa700 1 -- 172.21.15.163:0/2941288399 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fb64c084ad0 con 0x7fb650068390 2023-10-09T06:37:44.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.006+0000 7fb654d9d700 1 --2- 172.21.15.163:0/2941288399 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb634067320 0x7fb6340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:44.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.006+0000 7fb654d9d700 1 --2- 172.21.15.163:0/2941288399 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb634067320 0x7fb6340697d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb640009260 tx=0x7fb640016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:44.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.009+0000 7fb6457fa700 1 -- 172.21.15.163:0/2941288399 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb64c051ab0 con 0x7fb650068390 2023-10-09T06:37:44.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.193+0000 7fb6565a0700 1 -- 172.21.15.163:0/2941288399 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb65006b7a0 con 0x7fb650068390 2023-10-09T06:37:44.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.193+0000 7fb6457fa700 1 -- 172.21.15.163:0/2941288399 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v133) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb64c0550d0 con 0x7fb650068390 2023-10-09T06:37:44.193 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:37:44.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.196+0000 7fb6565a0700 1 -- 172.21.15.163:0/2941288399 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb634067320 msgr2=0x7fb6340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:44.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.196+0000 7fb6565a0700 1 --2- 172.21.15.163:0/2941288399 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb634067320 0x7fb6340697d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb640009260 tx=0x7fb640016000 comp rx=0 tx=0).stop 2023-10-09T06:37:44.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.196+0000 7fb6565a0700 1 -- 172.21.15.163:0/2941288399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb650068390 msgr2=0x7fb650147f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:44.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.196+0000 7fb6565a0700 1 --2- 172.21.15.163:0/2941288399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb650068390 0x7fb650147f40 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00e930 tx=0x7fb64c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:44.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.196+0000 7fb6565a0700 1 -- 172.21.15.163:0/2941288399 shutdown_connections 2023-10-09T06:37:44.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.196+0000 7fb6565a0700 1 --2- 172.21.15.163:0/2941288399 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb634067320 0x7fb6340697d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:44.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.196+0000 7fb6565a0700 1 --2- 172.21.15.163:0/2941288399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb650068390 0x7fb650147f40 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:44.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.196+0000 7fb6565a0700 1 --2- 172.21.15.163:0/2941288399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb650159a70 0x7fb6500c64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:44.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.196+0000 7fb6565a0700 1 --2- 172.21.15.163:0/2941288399 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb65006bfa0 0x7fb6500b2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:44.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.196+0000 7fb6565a0700 1 -- 172.21.15.163:0/2941288399 >> 172.21.15.163:0/2941288399 conn(0x7fb6500cddc0 msgr2=0x7fb6500cc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:44.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.196+0000 7fb6565a0700 1 -- 172.21.15.163:0/2941288399 shutdown_connections 2023-10-09T06:37:44.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.196+0000 7fb6565a0700 1 -- 172.21.15.163:0/2941288399 wait complete. 2023-10-09T06:37:44.208 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-09T06:37:44.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.322+0000 7fecfc6b1700 1 Processor -- start 2023-10-09T06:37:44.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.322+0000 7fecfc6b1700 1 -- start start 2023-10-09T06:37:44.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.323+0000 7fecfc6b1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fecf4159c60 0x7fecf415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:44.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.323+0000 7fecfc6b1700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecf415a160 con 0x7fecf4159890 2023-10-09T06:37:44.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.323+0000 7fecfc6b1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecf40d13a0 con 0x7fecf415db10 2023-10-09T06:37:44.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.323+0000 7fecfc6b1700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecf415df50 con 0x7fecf4159c60 2023-10-09T06:37:44.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.323+0000 7fecf9c4c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fecf4159c60 0x7fecf415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:44.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.323+0000 7fecfa44d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fecf4159890 0x7fecf415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:41000/0 (socket says 172.21.15.163:41000) 2023-10-09T06:37:44.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.323+0000 7fecfa44d700 1 -- 172.21.15.163:0/2446690264 learned_addr learned my addr 172.21.15.163:0/2446690264 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:44.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.324+0000 7fecf9c4c700 1 -- 172.21.15.163:0/2446690264 >> v1:172.21.15.181:6789/0 conn(0x7fecf415db10 legacy=0x7fecf4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:44.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.324+0000 7fecf9c4c700 1 -- 172.21.15.163:0/2446690264 >> v1:172.21.15.163:6789/0 conn(0x7fecf4159890 legacy=0x7fecf415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:44.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.324+0000 7fecf9c4c700 1 -- 172.21.15.163:0/2446690264 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf415e140 con 0x7fecf4159c60 2023-10-09T06:37:44.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.324+0000 7fecf944b700 1 -- 172.21.15.163:0/2446690264 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1417341737 0 0) 0x7fecf415a160 con 0x7fecf4159890 2023-10-09T06:37:44.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.324+0000 7fecf9c4c700 1 --2- 172.21.15.163:0/2446690264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecf4159c60 0x7fecf415d5d0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fece40098e0 tx=0x7fece40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=badfb0667bf03e01 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:44.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.324+0000 7fecf944b700 1 -- 172.21.15.163:0/2446690264 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece4009e80 con 0x7fecf4159c60 2023-10-09T06:37:44.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.324+0000 7fecf944b700 1 -- 172.21.15.163:0/2446690264 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece400b820 con 0x7fecf4159c60 2023-10-09T06:37:44.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.324+0000 7fecf944b700 1 -- 172.21.15.163:0/2446690264 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece400bba0 con 0x7fecf4159c60 2023-10-09T06:37:44.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.324+0000 7fecfc6b1700 1 -- 172.21.15.163:0/2446690264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecf4159c60 msgr2=0x7fecf415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:44.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.324+0000 7fecfc6b1700 1 --2- 172.21.15.163:0/2446690264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecf4159c60 0x7fecf415d5d0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fece40098e0 tx=0x7fece40091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:44.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.325+0000 7fecfc6b1700 1 -- 172.21.15.163:0/2446690264 shutdown_connections 2023-10-09T06:37:44.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.325+0000 7fecfc6b1700 1 --2- 172.21.15.163:0/2446690264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecf4159c60 0x7fecf415d5d0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:44.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.325+0000 7fecfc6b1700 1 -- 172.21.15.163:0/2446690264 >> 172.21.15.163:0/2446690264 conn(0x7fecf4068e20 msgr2=0x7fecf40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:44.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.325+0000 7fecfc6b1700 1 -- 172.21.15.163:0/2446690264 shutdown_connections 2023-10-09T06:37:44.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.325+0000 7fecfc6b1700 1 -- 172.21.15.163:0/2446690264 wait complete. 2023-10-09T06:37:44.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.326+0000 7fecfc6b1700 1 Processor -- start 2023-10-09T06:37:44.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.326+0000 7fecfc6b1700 1 -- start start 2023-10-09T06:37:44.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.326+0000 7fecfc6b1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecf4159890 0x7fecf41501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:44.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.326+0000 7fecfc6b1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecf4159c60 0x7fecf40cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:44.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.326+0000 7fecfc6b1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecf415db10 0x7fecf40c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:44.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.326+0000 7fecfc6b1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf416a6a0 con 0x7fecf415db10 2023-10-09T06:37:44.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.326+0000 7fecfc6b1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf4159060 con 0x7fecf4159890 2023-10-09T06:37:44.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.326+0000 7fecfc6b1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf40cdba0 con 0x7fecf4159c60 2023-10-09T06:37:44.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.327+0000 7fecfac4e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecf415db10 0x7fecf40c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:44.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.327+0000 7fecfac4e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecf415db10 0x7fecf40c67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:48424/0 (socket says 172.21.15.163:48424) 2023-10-09T06:37:44.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.327+0000 7fecfac4e700 1 -- 172.21.15.163:0/4095446907 learned_addr learned my addr 172.21.15.163:0/4095446907 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:44.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.327+0000 7fecfa44d700 1 --2- 172.21.15.163:0/4095446907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecf4159890 0x7fecf41501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:44.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.327+0000 7fecf9c4c700 1 --2- 172.21.15.163:0/4095446907 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecf4159c60 0x7fecf40cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:44.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.327+0000 7fecfac4e700 1 -- 172.21.15.163:0/4095446907 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecf4159c60 msgr2=0x7fecf40cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:44.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.327+0000 7fecfac4e700 1 --2- 172.21.15.163:0/4095446907 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecf4159c60 0x7fecf40cd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:44.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.327+0000 7fecfac4e700 1 -- 172.21.15.163:0/4095446907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecf4159890 msgr2=0x7fecf41501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:44.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.327+0000 7fecfac4e700 1 --2- 172.21.15.163:0/4095446907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecf4159890 0x7fecf41501e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:44.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.327+0000 7fecfac4e700 1 -- 172.21.15.163:0/4095446907 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece4009420 con 0x7fecf415db10 2023-10-09T06:37:44.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.327+0000 7fecfac4e700 1 --2- 172.21.15.163:0/4095446907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecf415db10 0x7fecf40c67b0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fecec00bbb0 tx=0x7fecec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:44.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.327+0000 7fecea7fc700 1 -- 172.21.15.163:0/4095446907 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec0092e0 con 0x7fecf415db10 2023-10-09T06:37:44.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.327+0000 7fecea7fc700 1 -- 172.21.15.163:0/4095446907 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecec00c7a0 con 0x7fecf415db10 2023-10-09T06:37:44.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.327+0000 7fecfc6b1700 1 -- 172.21.15.163:0/4095446907 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecf4169d70 con 0x7fecf415db10 2023-10-09T06:37:44.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.327+0000 7fecea7fc700 1 -- 172.21.15.163:0/4095446907 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec00cb40 con 0x7fecf415db10 2023-10-09T06:37:44.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.327+0000 7fecfc6b1700 1 -- 172.21.15.163:0/4095446907 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf416a330 con 0x7fecf415db10 2023-10-09T06:37:44.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.328+0000 7fecfc6b1700 1 -- 172.21.15.163:0/4095446907 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecf404c3c0 con 0x7fecf415db10 2023-10-09T06:37:44.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.329+0000 7fecea7fc700 1 -- 172.21.15.163:0/4095446907 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fecec012070 con 0x7fecf415db10 2023-10-09T06:37:44.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.329+0000 7fecea7fc700 1 --2- 172.21.15.163:0/4095446907 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fecd8067320 0x7fecd80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:44.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.329+0000 7fecea7fc700 1 -- 172.21.15.163:0/4095446907 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fecec084660 con 0x7fecf415db10 2023-10-09T06:37:44.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.329+0000 7fecfa44d700 1 --2- 172.21.15.163:0/4095446907 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fecd8067320 0x7fecd80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:44.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.330+0000 7fecfa44d700 1 --2- 172.21.15.163:0/4095446907 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fecd8067320 0x7fecd80697d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fecf4157090 tx=0x7fecf0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:44.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.333+0000 7fecea7fc700 1 -- 172.21.15.163:0/4095446907 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fecec051640 con 0x7fecf415db10 2023-10-09T06:37:44.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.517+0000 7fecfc6b1700 1 -- 172.21.15.163:0/4095446907 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fecf406a5b0 con 0x7fecf415db10 2023-10-09T06:37:44.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.518+0000 7fecea7fc700 1 -- 172.21.15.163:0/4095446907 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v133) v1 ==== 84+0+65 (secure 0 0 0) 0x7fecec054c60 con 0x7fecf415db10 2023-10-09T06:37:44.518 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:44.518 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:37:44.518 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:37:44.518 INFO:teuthology.orchestra.run.smithi163.stdout: "poolnum": 1, 2023-10-09T06:37:44.518 INFO:teuthology.orchestra.run.smithi163.stdout: "poolname": ".mgr" 2023-10-09T06:37:44.519 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:37:44.519 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:37:44.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.520+0000 7fecfc6b1700 1 -- 172.21.15.163:0/4095446907 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fecd8067320 msgr2=0x7fecd80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:44.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.520+0000 7fecfc6b1700 1 --2- 172.21.15.163:0/4095446907 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fecd8067320 0x7fecd80697d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fecf4157090 tx=0x7fecf0009040 comp rx=0 tx=0).stop 2023-10-09T06:37:44.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.520+0000 7fecfc6b1700 1 -- 172.21.15.163:0/4095446907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecf415db10 msgr2=0x7fecf40c67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:44.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.520+0000 7fecfc6b1700 1 --2- 172.21.15.163:0/4095446907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecf415db10 0x7fecf40c67b0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fecec00bbb0 tx=0x7fecec009840 comp rx=0 tx=0).stop 2023-10-09T06:37:44.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.520+0000 7fecfc6b1700 1 -- 172.21.15.163:0/4095446907 shutdown_connections 2023-10-09T06:37:44.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.520+0000 7fecfc6b1700 1 --2- 172.21.15.163:0/4095446907 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fecd8067320 0x7fecd80697d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:44.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.520+0000 7fecfc6b1700 1 --2- 172.21.15.163:0/4095446907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecf415db10 0x7fecf40c67b0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:44.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.520+0000 7fecfc6b1700 1 --2- 172.21.15.163:0/4095446907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecf4159890 0x7fecf41501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:44.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.520+0000 7fecfc6b1700 1 --2- 172.21.15.163:0/4095446907 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecf4159c60 0x7fecf40cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:44.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.520+0000 7fecfc6b1700 1 -- 172.21.15.163:0/4095446907 >> 172.21.15.163:0/4095446907 conn(0x7fecf4068e20 msgr2=0x7fecf41570c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:44.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.520+0000 7fecfc6b1700 1 -- 172.21.15.163:0/4095446907 shutdown_connections 2023-10-09T06:37:44.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.520+0000 7fecfc6b1700 1 -- 172.21.15.163:0/4095446907 wait complete. 2023-10-09T06:37:44.532 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-09T06:37:44.532 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-09T06:37:44.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.650+0000 7f618c775700 1 Processor -- start 2023-10-09T06:37:44.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.650+0000 7f618c775700 1 -- start start 2023-10-09T06:37:44.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.651+0000 7f618c775700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6184159ec0 0x7f61840b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:44.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.651+0000 7f618c775700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f61840cf350 0x7f61840cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:44.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.651+0000 7f618c775700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61840b5b00 con 0x7f61840b53d0 2023-10-09T06:37:44.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.651+0000 7f618c775700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61840b5da0 con 0x7f61840cf350 2023-10-09T06:37:44.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.651+0000 7f618c775700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61840b6080 con 0x7f6184159ec0 2023-10-09T06:37:44.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.651+0000 7f618ad12700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f61840cf350 0x7f61840cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:44.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.651+0000 7f6189d10700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f61840b53d0 0x7f61840cc900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:41016/0 (socket says 172.21.15.163:41016) 2023-10-09T06:37:44.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.651+0000 7f6189d10700 1 -- 172.21.15.163:0/4093716852 learned_addr learned my addr 172.21.15.163:0/4093716852 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:44.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.652+0000 7f618a511700 1 --2- 172.21.15.163:0/4093716852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6184159ec0 0x7f61840b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:44.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.652+0000 7f618ad12700 1 -- 172.21.15.163:0/4093716852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6184159ec0 msgr2=0x7f61840b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:44.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.652+0000 7f618ad12700 1 --2- 172.21.15.163:0/4093716852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6184159ec0 0x7f61840b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:44.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.652+0000 7f618ad12700 1 -- 172.21.15.163:0/4093716852 >> v1:172.21.15.163:6789/0 conn(0x7f61840b53d0 legacy=0x7f61840cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:44.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.652+0000 7f618ad12700 1 -- 172.21.15.163:0/4093716852 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61840cc100 con 0x7f61840cf350 2023-10-09T06:37:44.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.652+0000 7f618950f700 1 -- 172.21.15.163:0/4093716852 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2470833854 0 0) 0x7f61840b5b00 con 0x7f61840b53d0 2023-10-09T06:37:44.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.652+0000 7f618ad12700 1 --2- 172.21.15.163:0/4093716852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61840cf350 0x7f61840cd010 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f617c0098e0 tx=0x7f617c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892247b72e9db81b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:44.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.652+0000 7f618950f700 1 -- 172.21.15.163:0/4093716852 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c00b640 con 0x7f61840cf350 2023-10-09T06:37:44.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.652+0000 7f618950f700 1 -- 172.21.15.163:0/4093716852 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617c00b7a0 con 0x7f61840cf350 2023-10-09T06:37:44.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.652+0000 7f618950f700 1 -- 172.21.15.163:0/4093716852 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c00bb40 con 0x7f61840cf350 2023-10-09T06:37:44.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.653+0000 7f618c775700 1 -- 172.21.15.163:0/4093716852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61840cf350 msgr2=0x7f61840cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:44.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.653+0000 7f618c775700 1 --2- 172.21.15.163:0/4093716852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61840cf350 0x7f61840cd010 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f617c0098e0 tx=0x7f617c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:44.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.653+0000 7f618c775700 1 -- 172.21.15.163:0/4093716852 shutdown_connections 2023-10-09T06:37:44.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.653+0000 7f618c775700 1 --2- 172.21.15.163:0/4093716852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61840cf350 0x7f61840cd010 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:44.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.653+0000 7f618c775700 1 --2- 172.21.15.163:0/4093716852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6184159ec0 0x7f61840b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:44.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.653+0000 7f618c775700 1 -- 172.21.15.163:0/4093716852 >> 172.21.15.163:0/4093716852 conn(0x7f61840999f0 msgr2=0x7f61840b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:44.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.653+0000 7f618c775700 1 -- 172.21.15.163:0/4093716852 shutdown_connections 2023-10-09T06:37:44.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.653+0000 7f618c775700 1 -- 172.21.15.163:0/4093716852 wait complete. 2023-10-09T06:37:44.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.654+0000 7f618c775700 1 Processor -- start 2023-10-09T06:37:44.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.654+0000 7f618c775700 1 -- start start 2023-10-09T06:37:44.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.654+0000 7f618c775700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61840b53d0 0x7f61841699b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:44.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.654+0000 7f618c775700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61840cf350 0x7f618409c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:44.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.654+0000 7f618c775700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6184159ec0 0x7f6184156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:44.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.654+0000 7f618c775700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618409a670 con 0x7f61840b53d0 2023-10-09T06:37:44.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.654+0000 7f618c775700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6184169ef0 con 0x7f61840cf350 2023-10-09T06:37:44.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.654+0000 7f618c775700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618409c5c0 con 0x7f6184159ec0 2023-10-09T06:37:44.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.655+0000 7f618a511700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61840b53d0 0x7f61841699b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:44.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.655+0000 7f618a511700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61840b53d0 0x7f61841699b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:48430/0 (socket says 172.21.15.163:48430) 2023-10-09T06:37:44.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.655+0000 7f618a511700 1 -- 172.21.15.163:0/3424109344 learned_addr learned my addr 172.21.15.163:0/3424109344 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:44.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.655+0000 7f618ad12700 1 --2- 172.21.15.163:0/3424109344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6184159ec0 0x7f6184156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:44.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.655+0000 7f6189d10700 1 --2- 172.21.15.163:0/3424109344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61840cf350 0x7f618409c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:44.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.655+0000 7f618a511700 1 -- 172.21.15.163:0/3424109344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6184159ec0 msgr2=0x7f6184156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:44.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.655+0000 7f618a511700 1 --2- 172.21.15.163:0/3424109344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6184159ec0 0x7f6184156e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:44.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.655+0000 7f618a511700 1 -- 172.21.15.163:0/3424109344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61840cf350 msgr2=0x7f618409c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:44.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.655+0000 7f618a511700 1 --2- 172.21.15.163:0/3424109344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61840cf350 0x7f618409c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:44.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.655+0000 7f618a511700 1 -- 172.21.15.163:0/3424109344 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f617c009420 con 0x7f61840b53d0 2023-10-09T06:37:44.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.655+0000 7f618a511700 1 --2- 172.21.15.163:0/3424109344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61840b53d0 0x7f61841699b0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f6180009f70 tx=0x7f618000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:44.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.655+0000 7f617a7fc700 1 -- 172.21.15.163:0/3424109344 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618000ccd0 con 0x7f61840b53d0 2023-10-09T06:37:44.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.655+0000 7f617a7fc700 1 -- 172.21.15.163:0/3424109344 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618000f040 con 0x7f61840b53d0 2023-10-09T06:37:44.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.655+0000 7f618c775700 1 -- 172.21.15.163:0/3424109344 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f618409c860 con 0x7f61840b53d0 2023-10-09T06:37:44.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.655+0000 7f617a7fc700 1 -- 172.21.15.163:0/3424109344 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6180014480 con 0x7f61840b53d0 2023-10-09T06:37:44.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.655+0000 7f618c775700 1 -- 172.21.15.163:0/3424109344 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618409ce20 con 0x7f61840b53d0 2023-10-09T06:37:44.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.656+0000 7f618c775700 1 -- 172.21.15.163:0/3424109344 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f618409a950 con 0x7f61840b53d0 2023-10-09T06:37:44.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.656+0000 7f617a7fc700 1 -- 172.21.15.163:0/3424109344 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f61800147f0 con 0x7f61840b53d0 2023-10-09T06:37:44.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.657+0000 7f617a7fc700 1 --2- 172.21.15.163:0/3424109344 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6168067320 0x7f61680697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:44.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.657+0000 7f617a7fc700 1 -- 172.21.15.163:0/3424109344 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6180085260 con 0x7f61840b53d0 2023-10-09T06:37:44.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.657+0000 7f6189d10700 1 --2- 172.21.15.163:0/3424109344 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6168067320 0x7f61680697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:44.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.657+0000 7f6189d10700 1 --2- 172.21.15.163:0/3424109344 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6168067320 0x7f61680697d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f61840b1f50 tx=0x7f6174009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:44.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.661+0000 7f617a7fc700 1 -- 172.21.15.163:0/3424109344 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6180052130 con 0x7f61840b53d0 2023-10-09T06:37:44.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:44.843+0000 7f618c775700 1 -- 172.21.15.163:0/3424109344 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6184063c90 con 0x7f61840b53d0 2023-10-09T06:37:45.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.144+0000 7f617a7fc700 1 -- 172.21.15.163:0/3424109344 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v134) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6180055750 con 0x7f61840b53d0 2023-10-09T06:37:45.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.147+0000 7f618c775700 1 -- 172.21.15.163:0/3424109344 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6168067320 msgr2=0x7f61680697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:45.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.147+0000 7f618c775700 1 --2- 172.21.15.163:0/3424109344 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6168067320 0x7f61680697d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f61840b1f50 tx=0x7f6174009040 comp rx=0 tx=0).stop 2023-10-09T06:37:45.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.147+0000 7f618c775700 1 -- 172.21.15.163:0/3424109344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61840b53d0 msgr2=0x7f61841699b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:45.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.147+0000 7f618c775700 1 --2- 172.21.15.163:0/3424109344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61840b53d0 0x7f61841699b0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f6180009f70 tx=0x7f618000b040 comp rx=0 tx=0).stop 2023-10-09T06:37:45.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.147+0000 7f618c775700 1 -- 172.21.15.163:0/3424109344 shutdown_connections 2023-10-09T06:37:45.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.147+0000 7f618c775700 1 --2- 172.21.15.163:0/3424109344 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6168067320 0x7f61680697d0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:45.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.147+0000 7f618c775700 1 --2- 172.21.15.163:0/3424109344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61840b53d0 0x7f61841699b0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:45.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.147+0000 7f618c775700 1 --2- 172.21.15.163:0/3424109344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6184159ec0 0x7f6184156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:45.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.147+0000 7f618c775700 1 --2- 172.21.15.163:0/3424109344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61840cf350 0x7f618409c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:45.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.147+0000 7f618c775700 1 -- 172.21.15.163:0/3424109344 >> 172.21.15.163:0/3424109344 conn(0x7f61840999f0 msgr2=0x7f61840c98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:45.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.148+0000 7f618c775700 1 -- 172.21.15.163:0/3424109344 shutdown_connections 2023-10-09T06:37:45.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.148+0000 7f618c775700 1 -- 172.21.15.163:0/3424109344 wait complete. 2023-10-09T06:37:45.149 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_metadata' created 2023-10-09T06:37:45.162 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-09T06:37:45.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.284+0000 7f630faf3700 1 Processor -- start 2023-10-09T06:37:45.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.284+0000 7f630faf3700 1 -- start start 2023-10-09T06:37:45.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.285+0000 7f630faf3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6308159950 0x7f63080cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:45.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.285+0000 7f630faf3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6308069c70 0x7f63080cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:45.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.285+0000 7f630faf3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63080ce380 con 0x7f63080af320 2023-10-09T06:37:45.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.285+0000 7f630faf3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63080cec20 con 0x7f6308159950 2023-10-09T06:37:45.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.285+0000 7f630faf3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63080afa90 con 0x7f6308069c70 2023-10-09T06:37:45.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.285+0000 7f630d88f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6308159950 0x7f63080cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:45.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.285+0000 7f630e090700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6308069c70 0x7f63080cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:45.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.285+0000 7f630d08e700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f63080af320 0x7f6308069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:51604/0 (socket says 172.21.15.163:51604) 2023-10-09T06:37:45.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.285+0000 7f630d08e700 1 -- 172.21.15.163:0/2739928579 learned_addr learned my addr 172.21.15.163:0/2739928579 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:45.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.285+0000 7f630d88f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6308159950 0x7f63080cbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:34990/0 (socket says 172.21.15.163:34990) 2023-10-09T06:37:45.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.285+0000 7f630e090700 1 --2- 172.21.15.163:0/2739928579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6308069c70 0x7f63080cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:48394/0 (socket says 172.21.15.163:48394) 2023-10-09T06:37:45.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.286+0000 7f630d88f700 1 -- 172.21.15.163:0/2739928579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6308069c70 msgr2=0x7f63080cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:45.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.286+0000 7f630d88f700 1 --2- 172.21.15.163:0/2739928579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6308069c70 0x7f63080cf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:45.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.286+0000 7f630d88f700 1 -- 172.21.15.163:0/2739928579 >> v1:172.21.15.181:6789/0 conn(0x7f63080af320 legacy=0x7f6308069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:45.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.286+0000 7f630d88f700 1 -- 172.21.15.163:0/2739928579 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63080ce5d0 con 0x7f6308159950 2023-10-09T06:37:45.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.286+0000 7f630e090700 1 --2- 172.21.15.163:0/2739928579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6308069c70 0x7f63080cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:37:45.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.286+0000 7f630c88d700 1 -- 172.21.15.163:0/2739928579 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1047736880 0 0) 0x7f63080ce380 con 0x7f63080af320 2023-10-09T06:37:45.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.286+0000 7f630d88f700 1 --2- 172.21.15.163:0/2739928579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6308159950 0x7f63080cbb90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f63040098e0 tx=0x7f63040091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b198475e429098f server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:45.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.286+0000 7f630c88d700 1 -- 172.21.15.163:0/2739928579 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630400b640 con 0x7f6308159950 2023-10-09T06:37:45.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.286+0000 7f630c88d700 1 -- 172.21.15.163:0/2739928579 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630400b7a0 con 0x7f6308159950 2023-10-09T06:37:45.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.286+0000 7f630c88d700 1 -- 172.21.15.163:0/2739928579 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630400bb40 con 0x7f6308159950 2023-10-09T06:37:45.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.287+0000 7f630faf3700 1 -- 172.21.15.163:0/2739928579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6308159950 msgr2=0x7f63080cbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:45.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.287+0000 7f630faf3700 1 --2- 172.21.15.163:0/2739928579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6308159950 0x7f63080cbb90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f63040098e0 tx=0x7f63040091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:45.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.287+0000 7f630faf3700 1 -- 172.21.15.163:0/2739928579 shutdown_connections 2023-10-09T06:37:45.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.287+0000 7f630faf3700 1 --2- 172.21.15.163:0/2739928579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6308159950 0x7f63080cbb90 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:45.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.287+0000 7f630faf3700 1 --2- 172.21.15.163:0/2739928579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6308069c70 0x7f63080cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:45.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.287+0000 7f630faf3700 1 -- 172.21.15.163:0/2739928579 >> 172.21.15.163:0/2739928579 conn(0x7f6308068e00 msgr2=0x7f63080d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:45.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.287+0000 7f630faf3700 1 -- 172.21.15.163:0/2739928579 shutdown_connections 2023-10-09T06:37:45.289 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.287+0000 7f630faf3700 1 -- 172.21.15.163:0/2739928579 wait complete. 2023-10-09T06:37:45.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.288+0000 7f630faf3700 1 Processor -- start 2023-10-09T06:37:45.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.288+0000 7f630faf3700 1 -- start start 2023-10-09T06:37:45.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.288+0000 7f630faf3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6308069c70 0x7f63080b5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:45.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.288+0000 7f630faf3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63080af320 0x7f6308159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:45.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.289+0000 7f630faf3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6308159950 0x7f63080cf750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:45.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.289+0000 7f630faf3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630815a260 con 0x7f6308159950 2023-10-09T06:37:45.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.289+0000 7f630faf3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630815a450 con 0x7f6308069c70 2023-10-09T06:37:45.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.289+0000 7f630faf3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f630815a730 con 0x7f63080af320 2023-10-09T06:37:45.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.289+0000 7f630e090700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6308159950 0x7f63080cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:45.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.289+0000 7f630d08e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63080af320 0x7f6308159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:45.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.289+0000 7f630d88f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6308069c70 0x7f63080b5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:45.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.289+0000 7f630e090700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6308159950 0x7f63080cf750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:48436/0 (socket says 172.21.15.163:48436) 2023-10-09T06:37:45.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.289+0000 7f630e090700 1 -- 172.21.15.163:0/843090738 learned_addr learned my addr 172.21.15.163:0/843090738 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:45.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.289+0000 7f630e090700 1 -- 172.21.15.163:0/843090738 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63080af320 msgr2=0x7f6308159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:45.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.289+0000 7f630e090700 1 --2- 172.21.15.163:0/843090738 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63080af320 0x7f6308159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:45.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.289+0000 7f630e090700 1 -- 172.21.15.163:0/843090738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6308069c70 msgr2=0x7f63080b5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:45.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.289+0000 7f630e090700 1 --2- 172.21.15.163:0/843090738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6308069c70 0x7f63080b5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:45.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.289+0000 7f630e090700 1 -- 172.21.15.163:0/843090738 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6300009a20 con 0x7f6308159950 2023-10-09T06:37:45.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.289+0000 7f630d08e700 1 --2- 172.21.15.163:0/843090738 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63080af320 0x7f6308159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:37:45.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.289+0000 7f630e090700 1 --2- 172.21.15.163:0/843090738 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6308159950 0x7f63080cf750 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f6300009f30 tx=0x7f630000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:45.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.289+0000 7f62fdffb700 1 -- 172.21.15.163:0/843090738 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630000cb70 con 0x7f6308159950 2023-10-09T06:37:45.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.289+0000 7f62fdffb700 1 -- 172.21.15.163:0/843090738 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630000f040 con 0x7f6308159950 2023-10-09T06:37:45.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.289+0000 7f630faf3700 1 -- 172.21.15.163:0/843090738 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6304009420 con 0x7f6308159950 2023-10-09T06:37:45.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.289+0000 7f62fdffb700 1 -- 172.21.15.163:0/843090738 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6300014480 con 0x7f6308159950 2023-10-09T06:37:45.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.290+0000 7f630faf3700 1 -- 172.21.15.163:0/843090738 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f630815acf0 con 0x7f6308159950 2023-10-09T06:37:45.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.290+0000 7f630faf3700 1 -- 172.21.15.163:0/843090738 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63080718b0 con 0x7f6308159950 2023-10-09T06:37:45.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.291+0000 7f62fdffb700 1 -- 172.21.15.163:0/843090738 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f63000147f0 con 0x7f6308159950 2023-10-09T06:37:45.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.291+0000 7f62fdffb700 1 --2- 172.21.15.163:0/843090738 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f62ec0673a0 0x7f62ec069850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:45.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.292+0000 7f62fdffb700 1 -- 172.21.15.163:0/843090738 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f6300085890 con 0x7f6308159950 2023-10-09T06:37:45.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.292+0000 7f630d88f700 1 --2- 172.21.15.163:0/843090738 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f62ec0673a0 0x7f62ec069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:45.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.292+0000 7f630d88f700 1 --2- 172.21.15.163:0/843090738 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f62ec0673a0 0x7f62ec069850 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f63040093f0 tx=0x7f6304017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:45.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.296+0000 7f62fdffb700 1 -- 172.21.15.163:0/843090738 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f63000525b0 con 0x7f6308159950 2023-10-09T06:37:45.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:45.478+0000 7f630faf3700 1 -- 172.21.15.163:0/843090738 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f63080ce8b0 con 0x7f6308159950 2023-10-09T06:37:46.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.148+0000 7f62fdffb700 1 -- 172.21.15.163:0/843090738 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v135) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6300055bd0 con 0x7f6308159950 2023-10-09T06:37:46.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.157+0000 7f630faf3700 1 -- 172.21.15.163:0/843090738 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f62ec0673a0 msgr2=0x7f62ec069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:46.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.157+0000 7f630faf3700 1 --2- 172.21.15.163:0/843090738 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f62ec0673a0 0x7f62ec069850 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f63040093f0 tx=0x7f6304017000 comp rx=0 tx=0).stop 2023-10-09T06:37:46.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.157+0000 7f630faf3700 1 -- 172.21.15.163:0/843090738 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6308159950 msgr2=0x7f63080cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:46.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.157+0000 7f630faf3700 1 --2- 172.21.15.163:0/843090738 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6308159950 0x7f63080cf750 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f6300009f30 tx=0x7f630000b040 comp rx=0 tx=0).stop 2023-10-09T06:37:46.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.160+0000 7f630faf3700 1 -- 172.21.15.163:0/843090738 shutdown_connections 2023-10-09T06:37:46.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.160+0000 7f630faf3700 1 --2- 172.21.15.163:0/843090738 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f62ec0673a0 0x7f62ec069850 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:46.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.160+0000 7f630faf3700 1 --2- 172.21.15.163:0/843090738 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6308159950 0x7f63080cf750 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:46.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.160+0000 7f630faf3700 1 --2- 172.21.15.163:0/843090738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6308069c70 0x7f63080b5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:46.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.160+0000 7f630faf3700 1 --2- 172.21.15.163:0/843090738 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63080af320 0x7f6308159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:46.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.160+0000 7f630faf3700 1 -- 172.21.15.163:0/843090738 >> 172.21.15.163:0/843090738 conn(0x7f6308068e00 msgr2=0x7f6308153150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:46.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.163+0000 7f630faf3700 1 -- 172.21.15.163:0/843090738 shutdown_connections 2023-10-09T06:37:46.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.164+0000 7f630faf3700 1 -- 172.21.15.163:0/843090738 wait complete. 2023-10-09T06:37:46.166 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_data' created 2023-10-09T06:37:46.177 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-09T06:37:46.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.295+0000 7fe9625d7700 1 Processor -- start 2023-10-09T06:37:46.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.295+0000 7fe9625d7700 1 -- start start 2023-10-09T06:37:46.298 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.296+0000 7fe9625d7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe95c159890 0x7fe95c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:46.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.296+0000 7fe9625d7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe95c159c60 0x7fe95c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:46.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.296+0000 7fe9625d7700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe95c15a160 con 0x7fe95c15db10 2023-10-09T06:37:46.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.296+0000 7fe9625d7700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe95c15df50 con 0x7fe95c159c60 2023-10-09T06:37:46.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.296+0000 7fe9625d7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe95c0d13a0 con 0x7fe95c159890 2023-10-09T06:37:46.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.296+0000 7fe95b7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe95c159c60 0x7fe95c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:46.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.296+0000 7fe960b74700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fe95c15db10 0x7fe95c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:41026/0 (socket says 172.21.15.163:41026) 2023-10-09T06:37:46.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.296+0000 7fe960b74700 1 -- 172.21.15.163:0/370207796 learned_addr learned my addr 172.21.15.163:0/370207796 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:46.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.296+0000 7fe95bfff700 1 --2- 172.21.15.163:0/370207796 >> v2:172.21.15.181:3300/0 conn(0x7fe95c159890 0x7fe95c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:46.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.297+0000 7fe95b7fe700 1 -- 172.21.15.163:0/370207796 >> v2:172.21.15.181:3300/0 conn(0x7fe95c159890 msgr2=0x7fe95c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:46.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.297+0000 7fe95b7fe700 1 --2- 172.21.15.163:0/370207796 >> v2:172.21.15.181:3300/0 conn(0x7fe95c159890 0x7fe95c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:46.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.297+0000 7fe95b7fe700 1 -- 172.21.15.163:0/370207796 >> v1:172.21.15.163:6789/0 conn(0x7fe95c15db10 legacy=0x7fe95c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:46.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.297+0000 7fe95b7fe700 1 -- 172.21.15.163:0/370207796 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95c15e140 con 0x7fe95c159c60 2023-10-09T06:37:46.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.297+0000 7fe95affd700 1 -- 172.21.15.163:0/370207796 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2728453378 0 0) 0x7fe95c15a160 con 0x7fe95c15db10 2023-10-09T06:37:46.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.297+0000 7fe95b7fe700 1 --2- 172.21.15.163:0/370207796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe95c159c60 0x7fe95c15d5d0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0d1090 tx=0x7fe9500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167173cb8b0cf3ff server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:46.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.297+0000 7fe95affd700 1 -- 172.21.15.163:0/370207796 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe950011630 con 0x7fe95c159c60 2023-10-09T06:37:46.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.297+0000 7fe95affd700 1 -- 172.21.15.163:0/370207796 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe950011790 con 0x7fe95c159c60 2023-10-09T06:37:46.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.297+0000 7fe95affd700 1 -- 172.21.15.163:0/370207796 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe950011b30 con 0x7fe95c159c60 2023-10-09T06:37:46.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.297+0000 7fe9625d7700 1 -- 172.21.15.163:0/370207796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe95c159c60 msgr2=0x7fe95c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:46.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.297+0000 7fe9625d7700 1 --2- 172.21.15.163:0/370207796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe95c159c60 0x7fe95c15d5d0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0d1090 tx=0x7fe9500091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:46.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.297+0000 7fe9625d7700 1 -- 172.21.15.163:0/370207796 shutdown_connections 2023-10-09T06:37:46.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.297+0000 7fe9625d7700 1 --2- 172.21.15.163:0/370207796 >> v2:172.21.15.181:3300/0 conn(0x7fe95c159890 0x7fe95c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:46.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.297+0000 7fe9625d7700 1 --2- 172.21.15.163:0/370207796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe95c159c60 0x7fe95c15d5d0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:46.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.297+0000 7fe9625d7700 1 -- 172.21.15.163:0/370207796 >> 172.21.15.163:0/370207796 conn(0x7fe95c068e20 msgr2=0x7fe95c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:46.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.298+0000 7fe9625d7700 1 -- 172.21.15.163:0/370207796 shutdown_connections 2023-10-09T06:37:46.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.298+0000 7fe9625d7700 1 -- 172.21.15.163:0/370207796 wait complete. 2023-10-09T06:37:46.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.299+0000 7fe9625d7700 1 Processor -- start 2023-10-09T06:37:46.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.299+0000 7fe9625d7700 1 -- start start 2023-10-09T06:37:46.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.299+0000 7fe9625d7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe95c159890 0x7fe95c13f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:46.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.299+0000 7fe9625d7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe95c159c60 0x7fe95c157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:46.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.299+0000 7fe9625d7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe95c15db10 0x7fe95c1580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:46.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.299+0000 7fe9625d7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0cd7b0 con 0x7fe95c159c60 2023-10-09T06:37:46.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.299+0000 7fe9625d7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0b2d50 con 0x7fe95c15db10 2023-10-09T06:37:46.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.299+0000 7fe9625d7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0b2a70 con 0x7fe95c159890 2023-10-09T06:37:46.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.300+0000 7fe95b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe95c159c60 0x7fe95c157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:46.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.300+0000 7fe95b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe95c159c60 0x7fe95c157b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:48466/0 (socket says 172.21.15.163:48466) 2023-10-09T06:37:46.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.300+0000 7fe95b7fe700 1 -- 172.21.15.163:0/2620118625 learned_addr learned my addr 172.21.15.163:0/2620118625 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:46.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.300+0000 7fe960b74700 1 --2- 172.21.15.163:0/2620118625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe95c15db10 0x7fe95c1580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:46.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.300+0000 7fe95bfff700 1 --2- 172.21.15.163:0/2620118625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe95c159890 0x7fe95c13f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:46.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.300+0000 7fe95b7fe700 1 -- 172.21.15.163:0/2620118625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe95c159890 msgr2=0x7fe95c13f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:46.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.300+0000 7fe95b7fe700 1 --2- 172.21.15.163:0/2620118625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe95c159890 0x7fe95c13f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:46.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.300+0000 7fe95b7fe700 1 -- 172.21.15.163:0/2620118625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe95c15db10 msgr2=0x7fe95c1580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:46.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.300+0000 7fe95b7fe700 1 --2- 172.21.15.163:0/2620118625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe95c15db10 0x7fe95c1580c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:46.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.300+0000 7fe95b7fe700 1 -- 172.21.15.163:0/2620118625 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe950009420 con 0x7fe95c159c60 2023-10-09T06:37:46.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.300+0000 7fe95bfff700 1 --2- 172.21.15.163:0/2620118625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe95c159890 0x7fe95c13f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:46.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.300+0000 7fe95b7fe700 1 --2- 172.21.15.163:0/2620118625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe95c159c60 0x7fe95c157b80 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fe95c070e20 tx=0x7fe950017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:46.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.300+0000 7fe943fff700 1 -- 172.21.15.163:0/2620118625 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe950011b10 con 0x7fe95c159c60 2023-10-09T06:37:46.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.300+0000 7fe943fff700 1 -- 172.21.15.163:0/2620118625 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9500253f0 con 0x7fe95c159c60 2023-10-09T06:37:46.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.300+0000 7fe9625d7700 1 -- 172.21.15.163:0/2620118625 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe95c0cd160 con 0x7fe95c159c60 2023-10-09T06:37:46.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.300+0000 7fe943fff700 1 -- 172.21.15.163:0/2620118625 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe950025790 con 0x7fe95c159c60 2023-10-09T06:37:46.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.300+0000 7fe9625d7700 1 -- 172.21.15.163:0/2620118625 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe95c0b1d00 con 0x7fe95c159c60 2023-10-09T06:37:46.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.301+0000 7fe9625d7700 1 -- 172.21.15.163:0/2620118625 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe95c0cd920 con 0x7fe95c159c60 2023-10-09T06:37:46.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.302+0000 7fe943fff700 1 -- 172.21.15.163:0/2620118625 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe950025b80 con 0x7fe95c159c60 2023-10-09T06:37:46.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.302+0000 7fe943fff700 1 --2- 172.21.15.163:0/2620118625 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe948067320 0x7fe9480697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:46.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.302+0000 7fe943fff700 1 -- 172.21.15.163:0/2620118625 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fe950086cf0 con 0x7fe95c159c60 2023-10-09T06:37:46.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.302+0000 7fe95bfff700 1 --2- 172.21.15.163:0/2620118625 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe948067320 0x7fe9480697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:46.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.303+0000 7fe95bfff700 1 --2- 172.21.15.163:0/2620118625 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe948067320 0x7fe9480697d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009d80 tx=0x7fe94c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:46.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.306+0000 7fe943fff700 1 -- 172.21.15.163:0/2620118625 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe950053a60 con 0x7fe95c159c60 2023-10-09T06:37:46.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:46.529+0000 7fe9625d7700 1 -- 172.21.15.163:0/2620118625 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe95c04c3c0 con 0x7fe95c159c60 2023-10-09T06:37:47.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.171+0000 7fe943fff700 1 -- 172.21.15.163:0/2620118625 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 25 v89) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe9500330c0 con 0x7fe95c159c60 2023-10-09T06:37:47.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.175+0000 7fe9625d7700 1 -- 172.21.15.163:0/2620118625 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe948067320 msgr2=0x7fe9480697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:47.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.175+0000 7fe9625d7700 1 --2- 172.21.15.163:0/2620118625 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe948067320 0x7fe9480697d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009d80 tx=0x7fe94c008040 comp rx=0 tx=0).stop 2023-10-09T06:37:47.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.175+0000 7fe9625d7700 1 -- 172.21.15.163:0/2620118625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe95c159c60 msgr2=0x7fe95c157b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:47.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.175+0000 7fe9625d7700 1 --2- 172.21.15.163:0/2620118625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe95c159c60 0x7fe95c157b80 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fe95c070e20 tx=0x7fe950017000 comp rx=0 tx=0).stop 2023-10-09T06:37:47.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.182+0000 7fe9625d7700 1 -- 172.21.15.163:0/2620118625 shutdown_connections 2023-10-09T06:37:47.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.182+0000 7fe9625d7700 1 --2- 172.21.15.163:0/2620118625 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe948067320 0x7fe9480697d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:47.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.182+0000 7fe9625d7700 1 --2- 172.21.15.163:0/2620118625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe95c15db10 0x7fe95c1580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:47.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.182+0000 7fe9625d7700 1 --2- 172.21.15.163:0/2620118625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe95c159890 0x7fe95c13f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:47.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.182+0000 7fe9625d7700 1 --2- 172.21.15.163:0/2620118625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe95c159c60 0x7fe95c157b80 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:47.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.182+0000 7fe9625d7700 1 -- 172.21.15.163:0/2620118625 >> 172.21.15.163:0/2620118625 conn(0x7fe95c068e20 msgr2=0x7fe95c1536e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:47.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.186+0000 7fe9625d7700 1 -- 172.21.15.163:0/2620118625 shutdown_connections 2023-10-09T06:37:47.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.186+0000 7fe9625d7700 1 -- 172.21.15.163:0/2620118625 wait complete. 2023-10-09T06:37:47.186 INFO:teuthology.orchestra.run.smithi163.stderr:new fs with metadata pool 24 and data pool 25 2023-10-09T06:37:47.198 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:37:47.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.313+0000 7f82cb40b700 1 Processor -- start 2023-10-09T06:37:47.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.314+0000 7f82cb40b700 1 -- start start 2023-10-09T06:37:47.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.314+0000 7f82cb40b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f82c40cd1e0 0x7f82c40c9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:47.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.314+0000 7f82cb40b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c415c2a0 con 0x7f82c406bcd0 2023-10-09T06:37:47.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.314+0000 7f82cb40b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c415b690 con 0x7f82c4141500 2023-10-09T06:37:47.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.314+0000 7f82cb40b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c415b970 con 0x7f82c40cd1e0 2023-10-09T06:37:47.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.314+0000 7f82c99a8700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f82c40cd1e0 0x7f82c40c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:47.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.314+0000 7f82c91a7700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f82c406bcd0 0x7f82c4140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:41032/0 (socket says 172.21.15.163:41032) 2023-10-09T06:37:47.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.314+0000 7f82c99a8700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f82c40cd1e0 0x7f82c40c9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:48478/0 (socket says 172.21.15.163:48478) 2023-10-09T06:37:47.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.314+0000 7f82c91a7700 1 -- 172.21.15.163:0/947469562 learned_addr learned my addr 172.21.15.163:0/947469562 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:47.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.315+0000 7f82c99a8700 1 -- 172.21.15.163:0/947469562 >> v1:172.21.15.181:6789/0 conn(0x7f82c4141500 legacy=0x7f82c40c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:47.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.315+0000 7f82c99a8700 1 -- 172.21.15.163:0/947469562 >> v1:172.21.15.163:6789/0 conn(0x7f82c406bcd0 legacy=0x7f82c4140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:47.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.315+0000 7f82c99a8700 1 -- 172.21.15.163:0/947469562 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c415bc50 con 0x7f82c40cd1e0 2023-10-09T06:37:47.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.315+0000 7f82c99a8700 1 --2- 172.21.15.163:0/947469562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82c40cd1e0 0x7f82c40c9510 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f82bc0098e0 tx=0x7f82bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7cd07eb8c559ce4 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:47.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.315+0000 7f82bbfff700 1 -- 172.21.15.163:0/947469562 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc009e80 con 0x7f82c40cd1e0 2023-10-09T06:37:47.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.315+0000 7f82bbfff700 1 -- 172.21.15.163:0/947469562 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82bc00b820 con 0x7f82c40cd1e0 2023-10-09T06:37:47.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.315+0000 7f82bbfff700 1 -- 172.21.15.163:0/947469562 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc00bba0 con 0x7f82c40cd1e0 2023-10-09T06:37:47.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.316+0000 7f82cb40b700 1 -- 172.21.15.163:0/947469562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82c40cd1e0 msgr2=0x7f82c40c9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:47.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.316+0000 7f82cb40b700 1 --2- 172.21.15.163:0/947469562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82c40cd1e0 0x7f82c40c9510 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f82bc0098e0 tx=0x7f82bc0091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:47.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.316+0000 7f82cb40b700 1 -- 172.21.15.163:0/947469562 shutdown_connections 2023-10-09T06:37:47.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.316+0000 7f82cb40b700 1 --2- 172.21.15.163:0/947469562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82c40cd1e0 0x7f82c40c9510 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:47.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.316+0000 7f82cb40b700 1 -- 172.21.15.163:0/947469562 >> 172.21.15.163:0/947469562 conn(0x7f82c40ce600 msgr2=0x7f82c40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:47.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.316+0000 7f82cb40b700 1 -- 172.21.15.163:0/947469562 shutdown_connections 2023-10-09T06:37:47.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.316+0000 7f82cb40b700 1 -- 172.21.15.163:0/947469562 wait complete. 2023-10-09T06:37:47.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.317+0000 7f82cb40b700 1 Processor -- start 2023-10-09T06:37:47.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.317+0000 7f82cb40b700 1 -- start start 2023-10-09T06:37:47.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.318+0000 7f82cb40b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82c406bcd0 0x7f82c406a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:47.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.318+0000 7f82cb40b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c40cd1e0 0x7f82c40b2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:47.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.318+0000 7f82cb40b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82c4141500 0x7f82c415e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:47.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.318+0000 7f82cb40b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c406ae60 con 0x7f82c4141500 2023-10-09T06:37:47.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.318+0000 7f82cb40b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c406a490 con 0x7f82c40cd1e0 2023-10-09T06:37:47.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.318+0000 7f82cb40b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c4069880 con 0x7f82c406bcd0 2023-10-09T06:37:47.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.318+0000 7f82c99a8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82c4141500 0x7f82c415e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:47.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.318+0000 7f82c99a8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82c4141500 0x7f82c415e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:48484/0 (socket says 172.21.15.163:48484) 2023-10-09T06:37:47.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.318+0000 7f82c99a8700 1 -- 172.21.15.163:0/233955018 learned_addr learned my addr 172.21.15.163:0/233955018 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:47.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.318+0000 7f82c99a8700 1 -- 172.21.15.163:0/233955018 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82c406bcd0 msgr2=0x7f82c406a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:47.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.318+0000 7f82c99a8700 1 --2- 172.21.15.163:0/233955018 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82c406bcd0 0x7f82c406a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:47.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.318+0000 7f82c99a8700 1 -- 172.21.15.163:0/233955018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c40cd1e0 msgr2=0x7f82c40b2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:47.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.318+0000 7f82c89a6700 1 --2- 172.21.15.163:0/233955018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c40cd1e0 0x7f82c40b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:47.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.318+0000 7f82c99a8700 1 --2- 172.21.15.163:0/233955018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c40cd1e0 0x7f82c40b2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:47.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.318+0000 7f82c99a8700 1 -- 172.21.15.163:0/233955018 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82bc009420 con 0x7f82c4141500 2023-10-09T06:37:47.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.318+0000 7f82c89a6700 1 --2- 172.21.15.163:0/233955018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c40cd1e0 0x7f82c40b2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:37:47.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.318+0000 7f82c99a8700 1 --2- 172.21.15.163:0/233955018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82c4141500 0x7f82c415e0f0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f82bc0093f0 tx=0x7f82bc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:47.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.318+0000 7f82b97fa700 1 -- 172.21.15.163:0/233955018 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc01b450 con 0x7f82c4141500 2023-10-09T06:37:47.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.318+0000 7f82b97fa700 1 -- 172.21.15.163:0/233955018 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82bc020070 con 0x7f82c4141500 2023-10-09T06:37:47.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.318+0000 7f82cb40b700 1 -- 172.21.15.163:0/233955018 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c4069b60 con 0x7f82c4141500 2023-10-09T06:37:47.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.318+0000 7f82b97fa700 1 -- 172.21.15.163:0/233955018 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc01b820 con 0x7f82c4141500 2023-10-09T06:37:47.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.318+0000 7f82cb40b700 1 -- 172.21.15.163:0/233955018 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c406a120 con 0x7f82c4141500 2023-10-09T06:37:47.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.319+0000 7f82cb40b700 1 -- 172.21.15.163:0/233955018 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c404c3c0 con 0x7f82c4141500 2023-10-09T06:37:47.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.320+0000 7f82b97fa700 1 -- 172.21.15.163:0/233955018 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f82bc022030 con 0x7f82c4141500 2023-10-09T06:37:47.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.320+0000 7f82b97fa700 1 --2- 172.21.15.163:0/233955018 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f82a8067320 0x7f82a80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:47.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.320+0000 7f82b97fa700 1 -- 172.21.15.163:0/233955018 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f82bc086d80 con 0x7f82c4141500 2023-10-09T06:37:47.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.320+0000 7f82c91a7700 1 --2- 172.21.15.163:0/233955018 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f82a8067320 0x7f82a80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:47.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.321+0000 7f82c91a7700 1 --2- 172.21.15.163:0/233955018 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f82a8067320 0x7f82a80697d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f82c00020e0 tx=0x7f82c0001690 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:47.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.324+0000 7f82b97fa700 1 -- 172.21.15.163:0/233955018 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82bc053b20 con 0x7f82c4141500 2023-10-09T06:37:47.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.507+0000 7f82cb40b700 1 -- 172.21.15.163:0/233955018 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82c404c3c0 con 0x7f82c4141500 2023-10-09T06:37:47.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.508+0000 7f82b97fa700 1 -- 172.21.15.163:0/233955018 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+15425 (secure 0 0 0) 0x7f82bc057140 con 0x7f82c4141500 2023-10-09T06:37:47.508 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:47.508 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":136,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:37:46.529636+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:37:44.861841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-10-09T06:37:45.496241+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:37:47.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.510+0000 7f82cb40b700 1 -- 172.21.15.163:0/233955018 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f82a8067320 msgr2=0x7f82a80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:47.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.510+0000 7f82cb40b700 1 --2- 172.21.15.163:0/233955018 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f82a8067320 0x7f82a80697d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f82c00020e0 tx=0x7f82c0001690 comp rx=0 tx=0).stop 2023-10-09T06:37:47.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.510+0000 7f82cb40b700 1 -- 172.21.15.163:0/233955018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82c4141500 msgr2=0x7f82c415e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:47.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.510+0000 7f82cb40b700 1 --2- 172.21.15.163:0/233955018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82c4141500 0x7f82c415e0f0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f82bc0093f0 tx=0x7f82bc016000 comp rx=0 tx=0).stop 2023-10-09T06:37:47.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.510+0000 7f82cb40b700 1 -- 172.21.15.163:0/233955018 shutdown_connections 2023-10-09T06:37:47.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.510+0000 7f82cb40b700 1 --2- 172.21.15.163:0/233955018 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f82a8067320 0x7f82a80697d0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:47.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.510+0000 7f82cb40b700 1 --2- 172.21.15.163:0/233955018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82c4141500 0x7f82c415e0f0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:47.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.510+0000 7f82cb40b700 1 --2- 172.21.15.163:0/233955018 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82c406bcd0 0x7f82c406a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:47.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.510+0000 7f82cb40b700 1 --2- 172.21.15.163:0/233955018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82c40cd1e0 0x7f82c40b2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:47.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.510+0000 7f82cb40b700 1 -- 172.21.15.163:0/233955018 >> 172.21.15.163:0/233955018 conn(0x7f82c40ce600 msgr2=0x7f82c40c3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:47.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.510+0000 7f82cb40b700 1 -- 172.21.15.163:0/233955018 shutdown_connections 2023-10-09T06:37:47.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.510+0000 7f82cb40b700 1 -- 172.21.15.163:0/233955018 wait complete. 2023-10-09T06:37:47.522 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:37:47.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.637+0000 7f019e87f700 1 Processor -- start 2023-10-09T06:37:47.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.637+0000 7f019e87f700 1 -- start start 2023-10-09T06:37:47.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.638+0000 7f019e87f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0198159890 0x7f019815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:47.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.638+0000 7f019e87f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0198069560 0x7f01980699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:47.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.638+0000 7f019e87f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f019815a160 con 0x7f0198159c60 2023-10-09T06:37:47.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.638+0000 7f019e87f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0198069f20 con 0x7f0198069560 2023-10-09T06:37:47.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.638+0000 7f019e87f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01980d13a0 con 0x7f0198159890 2023-10-09T06:37:47.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.638+0000 7f019ce1c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0198069560 0x7f01980699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:47.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.638+0000 7f01977fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f0198159c60 0x7f019815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:51622/0 (socket says 172.21.15.163:51622) 2023-10-09T06:37:47.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.639+0000 7f0197fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0198159890 0x7f019815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:47.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.639+0000 7f01977fe700 1 -- 172.21.15.163:0/1326201225 learned_addr learned my addr 172.21.15.163:0/1326201225 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:47.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.639+0000 7f019ce1c700 1 -- 172.21.15.163:0/1326201225 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0198159890 msgr2=0x7f019815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:47.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.639+0000 7f019ce1c700 1 --2- 172.21.15.163:0/1326201225 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0198159890 0x7f019815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:47.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.639+0000 7f019ce1c700 1 -- 172.21.15.163:0/1326201225 >> v1:172.21.15.181:6789/0 conn(0x7f0198159c60 legacy=0x7f019815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:47.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.639+0000 7f0196ffd700 1 -- 172.21.15.163:0/1326201225 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 572803918 0 0) 0x7f019815a160 con 0x7f0198159c60 2023-10-09T06:37:47.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.639+0000 7f019ce1c700 1 -- 172.21.15.163:0/1326201225 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019815e140 con 0x7f0198069560 2023-10-09T06:37:47.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.639+0000 7f0197fff700 1 --2- 172.21.15.163:0/1326201225 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0198159890 0x7f019815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:37:47.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.639+0000 7f019ce1c700 1 --2- 172.21.15.163:0/1326201225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0198069560 0x7f01980699e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0190009e60 tx=0x7f0190009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af1a4b35121819e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:47.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.639+0000 7f0196ffd700 1 -- 172.21.15.163:0/1326201225 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019000e070 con 0x7f0198069560 2023-10-09T06:37:47.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.639+0000 7f0196ffd700 1 -- 172.21.15.163:0/1326201225 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019000bb60 con 0x7f0198069560 2023-10-09T06:37:47.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.639+0000 7f0196ffd700 1 -- 172.21.15.163:0/1326201225 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0190012430 con 0x7f0198069560 2023-10-09T06:37:47.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.640+0000 7f019e87f700 1 -- 172.21.15.163:0/1326201225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0198069560 msgr2=0x7f01980699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:47.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.640+0000 7f019e87f700 1 --2- 172.21.15.163:0/1326201225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0198069560 0x7f01980699e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0190009e60 tx=0x7f0190009780 comp rx=0 tx=0).stop 2023-10-09T06:37:47.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.640+0000 7f019e87f700 1 -- 172.21.15.163:0/1326201225 shutdown_connections 2023-10-09T06:37:47.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.640+0000 7f019e87f700 1 --2- 172.21.15.163:0/1326201225 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0198069560 0x7f01980699e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:47.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.640+0000 7f019e87f700 1 --2- 172.21.15.163:0/1326201225 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0198159890 0x7f019815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:47.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.640+0000 7f019e87f700 1 -- 172.21.15.163:0/1326201225 >> 172.21.15.163:0/1326201225 conn(0x7f0198068e20 msgr2=0x7f01980d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:47.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.640+0000 7f019e87f700 1 -- 172.21.15.163:0/1326201225 shutdown_connections 2023-10-09T06:37:47.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.640+0000 7f019e87f700 1 -- 172.21.15.163:0/1326201225 wait complete. 2023-10-09T06:37:47.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.641+0000 7f019e87f700 1 Processor -- start 2023-10-09T06:37:47.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.641+0000 7f019e87f700 1 -- start start 2023-10-09T06:37:47.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.642+0000 7f019e87f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0198069560 0x7f01980c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:47.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.642+0000 7f019e87f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0198159890 0x7f01980cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:47.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.642+0000 7f019e87f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0198159c60 0x7f01980cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:47.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.642+0000 7f019e87f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01980b3590 con 0x7f0198159c60 2023-10-09T06:37:47.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.642+0000 7f019e87f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01980c6610 con 0x7f0198069560 2023-10-09T06:37:47.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.642+0000 7f019e87f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0198151440 con 0x7f0198159890 2023-10-09T06:37:47.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.642+0000 7f019ce1c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0198159c60 0x7f01980cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:47.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.642+0000 7f019ce1c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0198159c60 0x7f01980cdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:48496/0 (socket says 172.21.15.163:48496) 2023-10-09T06:37:47.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.642+0000 7f019ce1c700 1 -- 172.21.15.163:0/715323046 learned_addr learned my addr 172.21.15.163:0/715323046 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:47.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.642+0000 7f01977fe700 1 --2- 172.21.15.163:0/715323046 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0198159890 0x7f01980cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:47.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.642+0000 7f0197fff700 1 --2- 172.21.15.163:0/715323046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0198069560 0x7f01980c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:47.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.642+0000 7f019ce1c700 1 -- 172.21.15.163:0/715323046 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0198159890 msgr2=0x7f01980cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:47.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.642+0000 7f019ce1c700 1 --2- 172.21.15.163:0/715323046 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0198159890 0x7f01980cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:47.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.642+0000 7f019ce1c700 1 -- 172.21.15.163:0/715323046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0198069560 msgr2=0x7f01980c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:47.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.642+0000 7f019ce1c700 1 --2- 172.21.15.163:0/715323046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0198069560 0x7f01980c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:47.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.642+0000 7f019ce1c700 1 -- 172.21.15.163:0/715323046 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0188009420 con 0x7f0198159c60 2023-10-09T06:37:47.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.642+0000 7f019ce1c700 1 --2- 172.21.15.163:0/715323046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0198159c60 0x7f01980cdad0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f0190012ad0 tx=0x7f0190019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:47.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.643+0000 7f017ffff700 1 -- 172.21.15.163:0/715323046 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0190012e10 con 0x7f0198159c60 2023-10-09T06:37:47.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.643+0000 7f017ffff700 1 -- 172.21.15.163:0/715323046 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019001e630 con 0x7f0198159c60 2023-10-09T06:37:47.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.643+0000 7f017ffff700 1 -- 172.21.15.163:0/715323046 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019001e9f0 con 0x7f0198159c60 2023-10-09T06:37:47.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.643+0000 7f019e87f700 1 -- 172.21.15.163:0/715323046 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0190009a20 con 0x7f0198159c60 2023-10-09T06:37:47.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.643+0000 7f019e87f700 1 -- 172.21.15.163:0/715323046 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01980c6330 con 0x7f0198159c60 2023-10-09T06:37:47.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.644+0000 7f019e87f700 1 -- 172.21.15.163:0/715323046 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f019804c3c0 con 0x7f0198159c60 2023-10-09T06:37:47.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.644+0000 7f017ffff700 1 -- 172.21.15.163:0/715323046 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0190024070 con 0x7f0198159c60 2023-10-09T06:37:47.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.644+0000 7f017ffff700 1 --2- 172.21.15.163:0/715323046 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0184067320 0x7f01840697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:47.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.644+0000 7f017ffff700 1 -- 172.21.15.163:0/715323046 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0190087400 con 0x7f0198159c60 2023-10-09T06:37:47.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.645+0000 7f0197fff700 1 --2- 172.21.15.163:0/715323046 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0184067320 0x7f01840697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:47.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.645+0000 7f0197fff700 1 --2- 172.21.15.163:0/715323046 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0184067320 0x7f01840697d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f01880093f0 tx=0x7f0188008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:47.651 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.649+0000 7f017ffff700 1 -- 172.21.15.163:0/715323046 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0190054090 con 0x7f0198159c60 2023-10-09T06:37:47.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.831+0000 7f019e87f700 1 -- 172.21.15.163:0/715323046 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0198151580 con 0x7f0198159c60 2023-10-09T06:37:47.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.831+0000 7f017ffff700 1 -- 172.21.15.163:0/715323046 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+15425 (secure 0 0 0) 0x7f01900576b0 con 0x7f0198159c60 2023-10-09T06:37:47.831 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:47.831 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":136,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:37:46.529636+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:37:44.861841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-10-09T06:37:45.496241+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:37:47.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.834+0000 7f019e87f700 1 -- 172.21.15.163:0/715323046 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0184067320 msgr2=0x7f01840697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:47.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.834+0000 7f019e87f700 1 --2- 172.21.15.163:0/715323046 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0184067320 0x7f01840697d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f01880093f0 tx=0x7f0188008040 comp rx=0 tx=0).stop 2023-10-09T06:37:47.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.834+0000 7f019e87f700 1 -- 172.21.15.163:0/715323046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0198159c60 msgr2=0x7f01980cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:47.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.834+0000 7f019e87f700 1 --2- 172.21.15.163:0/715323046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0198159c60 0x7f01980cdad0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f0190012ad0 tx=0x7f0190019000 comp rx=0 tx=0).stop 2023-10-09T06:37:47.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.834+0000 7f019e87f700 1 -- 172.21.15.163:0/715323046 shutdown_connections 2023-10-09T06:37:47.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.834+0000 7f019e87f700 1 --2- 172.21.15.163:0/715323046 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0184067320 0x7f01840697d0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:47.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.834+0000 7f019e87f700 1 --2- 172.21.15.163:0/715323046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0198159c60 0x7f01980cdad0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:47.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.834+0000 7f019e87f700 1 --2- 172.21.15.163:0/715323046 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0198069560 0x7f01980c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:47.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.834+0000 7f019e87f700 1 --2- 172.21.15.163:0/715323046 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0198159890 0x7f01980cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:47.835 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.834+0000 7f019e87f700 1 -- 172.21.15.163:0/715323046 >> 172.21.15.163:0/715323046 conn(0x7f0198068e20 msgr2=0x7f019815d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:47.835 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.834+0000 7f019e87f700 1 -- 172.21.15.163:0/715323046 shutdown_connections 2023-10-09T06:37:47.835 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.834+0000 7f019e87f700 1 -- 172.21.15.163:0/715323046 wait complete. 2023-10-09T06:37:47.846 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-09T06:37:47.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.964+0000 7f549ef7f700 1 Processor -- start 2023-10-09T06:37:47.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.964+0000 7f549ef7f700 1 -- start start 2023-10-09T06:37:47.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.965+0000 7f549ef7f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5498159c60 0x7f549815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:47.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.965+0000 7f549ef7f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f549815db10 0x7f5498069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:47.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.965+0000 7f549ef7f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549815a160 con 0x7f5498159890 2023-10-09T06:37:47.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.965+0000 7f549ef7f700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f549815df50 con 0x7f5498159c60 2023-10-09T06:37:47.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.965+0000 7f549ef7f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54980d13a0 con 0x7f549815db10 2023-10-09T06:37:47.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.965+0000 7f548ffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5498159c60 0x7f549815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:47.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.965+0000 7f548ffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5498159c60 0x7f549815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:48504/0 (socket says 172.21.15.163:48504) 2023-10-09T06:37:47.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.965+0000 7f548ffff700 1 -- 172.21.15.163:0/4294950625 learned_addr learned my addr 172.21.15.163:0/4294950625 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:47.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.965+0000 7f549d51c700 1 --2- 172.21.15.163:0/4294950625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f549815db10 0x7f5498069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:47.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.966+0000 7f548ffff700 1 -- 172.21.15.163:0/4294950625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f549815db10 msgr2=0x7f5498069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:47.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.966+0000 7f548ffff700 1 --2- 172.21.15.163:0/4294950625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f549815db10 0x7f5498069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:47.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.966+0000 7f548ffff700 1 -- 172.21.15.163:0/4294950625 >> v1:172.21.15.181:6789/0 conn(0x7f5498159890 legacy=0x7f549815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:47.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.966+0000 7f548ffff700 1 -- 172.21.15.163:0/4294950625 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549815e140 con 0x7f5498159c60 2023-10-09T06:37:47.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.966+0000 7f549d51c700 1 --2- 172.21.15.163:0/4294950625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f549815db10 0x7f5498069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:37:47.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.966+0000 7f548ffff700 1 --2- 172.21.15.163:0/4294950625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5498159c60 0x7f549815d5d0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f54880099b0 tx=0x7f5488009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c93adadb4b2668 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:47.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.966+0000 7f548f7fe700 1 -- 172.21.15.163:0/4294950625 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548800b860 con 0x7f5498159c60 2023-10-09T06:37:47.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.966+0000 7f548f7fe700 1 -- 172.21.15.163:0/4294950625 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548800b9c0 con 0x7f5498159c60 2023-10-09T06:37:47.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.966+0000 7f548f7fe700 1 -- 172.21.15.163:0/4294950625 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548800bd60 con 0x7f5498159c60 2023-10-09T06:37:47.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.966+0000 7f549ef7f700 1 -- 172.21.15.163:0/4294950625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5498159c60 msgr2=0x7f549815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:47.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.966+0000 7f549ef7f700 1 --2- 172.21.15.163:0/4294950625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5498159c60 0x7f549815d5d0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f54880099b0 tx=0x7f5488009250 comp rx=0 tx=0).stop 2023-10-09T06:37:47.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.966+0000 7f549ef7f700 1 -- 172.21.15.163:0/4294950625 shutdown_connections 2023-10-09T06:37:47.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.966+0000 7f549ef7f700 1 --2- 172.21.15.163:0/4294950625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f549815db10 0x7f5498069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:47.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.966+0000 7f549ef7f700 1 --2- 172.21.15.163:0/4294950625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5498159c60 0x7f549815d5d0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:47.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.966+0000 7f549ef7f700 1 -- 172.21.15.163:0/4294950625 >> 172.21.15.163:0/4294950625 conn(0x7f5498068e20 msgr2=0x7f54980d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:47.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.966+0000 7f549ef7f700 1 -- 172.21.15.163:0/4294950625 shutdown_connections 2023-10-09T06:37:47.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.966+0000 7f549ef7f700 1 -- 172.21.15.163:0/4294950625 wait complete. 2023-10-09T06:37:47.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.968+0000 7f549ef7f700 1 Processor -- start 2023-10-09T06:37:47.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.968+0000 7f549ef7f700 1 -- start start 2023-10-09T06:37:47.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.968+0000 7f549ef7f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5498159890 0x7f549814fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:47.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.968+0000 7f549ef7f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5498159c60 0x7f54980ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:47.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.968+0000 7f549ef7f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f549815db10 0x7f549815cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:47.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.968+0000 7f549ef7f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549816a760 con 0x7f5498159c60 2023-10-09T06:37:47.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.969+0000 7f549ef7f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5498169b20 con 0x7f549815db10 2023-10-09T06:37:47.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.969+0000 7f549ef7f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5498169dc0 con 0x7f5498159890 2023-10-09T06:37:47.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.969+0000 7f548ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5498159c60 0x7f54980ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:47.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.969+0000 7f548ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5498159c60 0x7f54980ce7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:48510/0 (socket says 172.21.15.163:48510) 2023-10-09T06:37:47.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.969+0000 7f548ffff700 1 -- 172.21.15.163:0/1605502723 learned_addr learned my addr 172.21.15.163:0/1605502723 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:47.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.969+0000 7f549d51c700 1 --2- 172.21.15.163:0/1605502723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f549815db10 0x7f549815cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:47.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.969+0000 7f549cd1b700 1 --2- 172.21.15.163:0/1605502723 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5498159890 0x7f549814fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:47.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.969+0000 7f548ffff700 1 -- 172.21.15.163:0/1605502723 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5498159890 msgr2=0x7f549814fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:47.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.969+0000 7f548ffff700 1 --2- 172.21.15.163:0/1605502723 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5498159890 0x7f549814fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:47.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.969+0000 7f548ffff700 1 -- 172.21.15.163:0/1605502723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f549815db10 msgr2=0x7f549815cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:47.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.969+0000 7f548ffff700 1 --2- 172.21.15.163:0/1605502723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f549815db10 0x7f549815cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:47.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.969+0000 7f548ffff700 1 -- 172.21.15.163:0/1605502723 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54880094f0 con 0x7f5498159c60 2023-10-09T06:37:47.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.969+0000 7f549d51c700 1 --2- 172.21.15.163:0/1605502723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f549815db10 0x7f549815cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:47.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.969+0000 7f548ffff700 1 --2- 172.21.15.163:0/1605502723 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5498159c60 0x7f54980ce7c0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f5488009290 tx=0x7f5488017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:47.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.969+0000 7f548cff9700 1 -- 172.21.15.163:0/1605502723 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5488024440 con 0x7f5498159c60 2023-10-09T06:37:47.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.969+0000 7f548cff9700 1 -- 172.21.15.163:0/1605502723 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54880245a0 con 0x7f5498159c60 2023-10-09T06:37:47.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.969+0000 7f548cff9700 1 -- 172.21.15.163:0/1605502723 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5488024940 con 0x7f5498159c60 2023-10-09T06:37:47.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.969+0000 7f549ef7f700 1 -- 172.21.15.163:0/1605502723 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549816a0a0 con 0x7f5498159c60 2023-10-09T06:37:47.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.969+0000 7f549ef7f700 1 -- 172.21.15.163:0/1605502723 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549814ff70 con 0x7f5498159c60 2023-10-09T06:37:47.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.970+0000 7f549ef7f700 1 -- 172.21.15.163:0/1605502723 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f549816a8a0 con 0x7f5498159c60 2023-10-09T06:37:47.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.971+0000 7f548cff9700 1 -- 172.21.15.163:0/1605502723 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f548801a420 con 0x7f5498159c60 2023-10-09T06:37:47.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.971+0000 7f548cff9700 1 --2- 172.21.15.163:0/1605502723 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5484067320 0x7f54840697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:47.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.971+0000 7f548cff9700 1 -- 172.21.15.163:0/1605502723 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5488086bb0 con 0x7f5498159c60 2023-10-09T06:37:47.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.971+0000 7f549cd1b700 1 --2- 172.21.15.163:0/1605502723 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5484067320 0x7f54840697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:47.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.972+0000 7f549cd1b700 1 --2- 172.21.15.163:0/1605502723 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5484067320 0x7f54840697d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5494007fd0 tx=0x7f5494009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:47.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:47.975+0000 7f548cff9700 1 -- 172.21.15.163:0/1605502723 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f549816a8a0 con 0x7f5498159c60 2023-10-09T06:37:48.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:48.203+0000 7f549ef7f700 1 -- 172.21.15.163:0/1605502723 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f549816a8a0 con 0x7f5498159c60 2023-10-09T06:37:49.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.186+0000 7f548cff9700 1 -- 172.21.15.163:0/1605502723 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v92) v1 ==== 118+0+0 (secure 0 0 0) 0x7f549816a8a0 con 0x7f5498159c60 2023-10-09T06:37:49.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.189+0000 7f549ef7f700 1 -- 172.21.15.163:0/1605502723 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5484067320 msgr2=0x7f54840697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:49.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.189+0000 7f549ef7f700 1 --2- 172.21.15.163:0/1605502723 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5484067320 0x7f54840697d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5494007fd0 tx=0x7f5494009040 comp rx=0 tx=0).stop 2023-10-09T06:37:49.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.189+0000 7f549ef7f700 1 -- 172.21.15.163:0/1605502723 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5498159c60 msgr2=0x7f54980ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:49.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.189+0000 7f549ef7f700 1 --2- 172.21.15.163:0/1605502723 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5498159c60 0x7f54980ce7c0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f5488009290 tx=0x7f5488017000 comp rx=0 tx=0).stop 2023-10-09T06:37:49.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.189+0000 7f549ef7f700 1 -- 172.21.15.163:0/1605502723 shutdown_connections 2023-10-09T06:37:49.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.189+0000 7f549ef7f700 1 --2- 172.21.15.163:0/1605502723 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5484067320 0x7f54840697d0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:49.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.189+0000 7f549ef7f700 1 --2- 172.21.15.163:0/1605502723 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f549815db10 0x7f549815cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:49.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.189+0000 7f549ef7f700 1 --2- 172.21.15.163:0/1605502723 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5498159890 0x7f549814fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:49.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.189+0000 7f549ef7f700 1 --2- 172.21.15.163:0/1605502723 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5498159c60 0x7f54980ce7c0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:49.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.189+0000 7f549ef7f700 1 -- 172.21.15.163:0/1605502723 >> 172.21.15.163:0/1605502723 conn(0x7f5498068e20 msgr2=0x7f5498150c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:49.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.189+0000 7f549ef7f700 1 -- 172.21.15.163:0/1605502723 shutdown_connections 2023-10-09T06:37:49.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.189+0000 7f549ef7f700 1 -- 172.21.15.163:0/1605502723 wait complete. 2023-10-09T06:37:49.202 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-09T06:37:49.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.321+0000 7f17c9a52700 1 Processor -- start 2023-10-09T06:37:49.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.321+0000 7f17c9a52700 1 -- start start 2023-10-09T06:37:49.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.322+0000 7f17c9a52700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f17c40c83f0 0x7f17c40c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:49.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.322+0000 7f17c9a52700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f17c41568e0 0x7f17c40ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:49.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.322+0000 7f17c9a52700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17c415b0d0 con 0x7f17c40ccee0 2023-10-09T06:37:49.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.322+0000 7f17c9a52700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c40c7a60 con 0x7f17c40c83f0 2023-10-09T06:37:49.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.322+0000 7f17c9a52700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c415a4c0 con 0x7f17c41568e0 2023-10-09T06:37:49.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.322+0000 7f17c2ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f17c40c83f0 0x7f17c40c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:49.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.322+0000 7f17c2ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f17c40c83f0 0x7f17c40c7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41604/0 (socket says 172.21.15.163:41604) 2023-10-09T06:37:49.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.322+0000 7f17c2ffd700 1 -- 172.21.15.163:0/3591985934 learned_addr learned my addr 172.21.15.163:0/3591985934 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:49.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.323+0000 7f17c37fe700 1 --2- 172.21.15.163:0/3591985934 >> v2:172.21.15.181:3300/0 conn(0x7f17c41568e0 0x7f17c40ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:49.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.323+0000 7f17c2ffd700 1 -- 172.21.15.163:0/3591985934 >> v2:172.21.15.181:3300/0 conn(0x7f17c41568e0 msgr2=0x7f17c40ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:49.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.323+0000 7f17c2ffd700 1 --2- 172.21.15.163:0/3591985934 >> v2:172.21.15.181:3300/0 conn(0x7f17c41568e0 0x7f17c40ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:49.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.323+0000 7f17c2ffd700 1 -- 172.21.15.163:0/3591985934 >> v1:172.21.15.181:6789/0 conn(0x7f17c40ccee0 legacy=0x7f17c41561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:49.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.323+0000 7f17c2ffd700 1 -- 172.21.15.163:0/3591985934 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c415a7a0 con 0x7f17c40c83f0 2023-10-09T06:37:49.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.323+0000 7f17c37fe700 1 --2- 172.21.15.163:0/3591985934 >> v2:172.21.15.181:3300/0 conn(0x7f17c41568e0 0x7f17c40ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:37:49.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.323+0000 7f17c2ffd700 1 --2- 172.21.15.163:0/3591985934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17c40c83f0 0x7f17c40c7520 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f17c406d240 tx=0x7f17b400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b532344711cd1c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:49.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.323+0000 7f17c27fc700 1 -- 172.21.15.163:0/3591985934 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b400dc10 con 0x7f17c40c83f0 2023-10-09T06:37:49.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.323+0000 7f17c27fc700 1 -- 172.21.15.163:0/3591985934 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b400dd70 con 0x7f17c40c83f0 2023-10-09T06:37:49.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.323+0000 7f17c27fc700 1 -- 172.21.15.163:0/3591985934 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b40106c0 con 0x7f17c40c83f0 2023-10-09T06:37:49.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.324+0000 7f17c9a52700 1 -- 172.21.15.163:0/3591985934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17c40c83f0 msgr2=0x7f17c40c7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:49.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.324+0000 7f17c9a52700 1 --2- 172.21.15.163:0/3591985934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17c40c83f0 0x7f17c40c7520 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f17c406d240 tx=0x7f17b400b4f0 comp rx=0 tx=0).stop 2023-10-09T06:37:49.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.324+0000 7f17c9a52700 1 -- 172.21.15.163:0/3591985934 shutdown_connections 2023-10-09T06:37:49.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.324+0000 7f17c9a52700 1 --2- 172.21.15.163:0/3591985934 >> v2:172.21.15.181:3300/0 conn(0x7f17c41568e0 0x7f17c40ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:49.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.324+0000 7f17c9a52700 1 --2- 172.21.15.163:0/3591985934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17c40c83f0 0x7f17c40c7520 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:49.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.324+0000 7f17c9a52700 1 -- 172.21.15.163:0/3591985934 >> 172.21.15.163:0/3591985934 conn(0x7f17c4068e10 msgr2=0x7f17c40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:49.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.324+0000 7f17c9a52700 1 -- 172.21.15.163:0/3591985934 shutdown_connections 2023-10-09T06:37:49.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.324+0000 7f17c9a52700 1 -- 172.21.15.163:0/3591985934 wait complete. 2023-10-09T06:37:49.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.325+0000 7f17c9a52700 1 Processor -- start 2023-10-09T06:37:49.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.325+0000 7f17c9a52700 1 -- start start 2023-10-09T06:37:49.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.326+0000 7f17c9a52700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17c40c83f0 0x7f17c414e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:49.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.326+0000 7f17c9a52700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17c40ccee0 0x7f17c40afc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:49.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.326+0000 7f17c9a52700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17c41568e0 0x7f17c414f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:49.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.326+0000 7f17c9a52700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c40b14a0 con 0x7f17c40c83f0 2023-10-09T06:37:49.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.326+0000 7f17c9a52700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c416a160 con 0x7f17c40ccee0 2023-10-09T06:37:49.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.326+0000 7f17c9a52700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c414ef30 con 0x7f17c41568e0 2023-10-09T06:37:49.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.326+0000 7f17c2ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17c40c83f0 0x7f17c414e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:49.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.326+0000 7f17c2ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17c40c83f0 0x7f17c414e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41608/0 (socket says 172.21.15.163:41608) 2023-10-09T06:37:49.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.326+0000 7f17c2ffd700 1 -- 172.21.15.163:0/4155928003 learned_addr learned my addr 172.21.15.163:0/4155928003 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:49.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.326+0000 7f17c37fe700 1 --2- 172.21.15.163:0/4155928003 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17c41568e0 0x7f17c414f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:49.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.326+0000 7f17ba7fc700 1 --2- 172.21.15.163:0/4155928003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17c40ccee0 0x7f17c40afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:49.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.326+0000 7f17c2ffd700 1 -- 172.21.15.163:0/4155928003 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17c41568e0 msgr2=0x7f17c414f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:49.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.326+0000 7f17c2ffd700 1 --2- 172.21.15.163:0/4155928003 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17c41568e0 0x7f17c414f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:49.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.326+0000 7f17c2ffd700 1 -- 172.21.15.163:0/4155928003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17c40ccee0 msgr2=0x7f17c40afc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:49.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.326+0000 7f17c2ffd700 1 --2- 172.21.15.163:0/4155928003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17c40ccee0 0x7f17c40afc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:49.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.326+0000 7f17c2ffd700 1 -- 172.21.15.163:0/4155928003 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b400b790 con 0x7f17c40c83f0 2023-10-09T06:37:49.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.326+0000 7f17c2ffd700 1 --2- 172.21.15.163:0/4155928003 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17c40c83f0 0x7f17c414e9f0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f17b400b530 tx=0x7f17b401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:49.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.326+0000 7f17bb7fe700 1 -- 172.21.15.163:0/4155928003 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b4018700 con 0x7f17c40c83f0 2023-10-09T06:37:49.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.326+0000 7f17bb7fe700 1 -- 172.21.15.163:0/4155928003 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b4018880 con 0x7f17c40c83f0 2023-10-09T06:37:49.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.326+0000 7f17c9a52700 1 -- 172.21.15.163:0/4155928003 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c40b0150 con 0x7f17c40c83f0 2023-10-09T06:37:49.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.326+0000 7f17bb7fe700 1 -- 172.21.15.163:0/4155928003 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b4018c40 con 0x7f17c40c83f0 2023-10-09T06:37:49.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.326+0000 7f17c9a52700 1 -- 172.21.15.163:0/4155928003 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c40b0760 con 0x7f17c40c83f0 2023-10-09T06:37:49.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.327+0000 7f17c9a52700 1 -- 172.21.15.163:0/4155928003 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c404c3c0 con 0x7f17c40c83f0 2023-10-09T06:37:49.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.328+0000 7f17bb7fe700 1 -- 172.21.15.163:0/4155928003 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f17b40233d0 con 0x7f17c40c83f0 2023-10-09T06:37:49.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.328+0000 7f17bb7fe700 1 --2- 172.21.15.163:0/4155928003 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f17a4067320 0x7f17a40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:49.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.328+0000 7f17bb7fe700 1 -- 172.21.15.163:0/4155928003 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f17b4014070 con 0x7f17c40c83f0 2023-10-09T06:37:49.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.328+0000 7f17ba7fc700 1 --2- 172.21.15.163:0/4155928003 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f17a4067320 0x7f17a40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:49.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.329+0000 7f17ba7fc700 1 --2- 172.21.15.163:0/4155928003 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f17a4067320 0x7f17a40697d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f17c4169120 tx=0x7f17ac007ea0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:49.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.332+0000 7f17bb7fe700 1 -- 172.21.15.163:0/4155928003 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f17b4055800 con 0x7f17c40c83f0 2023-10-09T06:37:49.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:49.557+0000 7f17c9a52700 1 -- 172.21.15.163:0/4155928003 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f17c415aaf0 con 0x7f17c40c83f0 2023-10-09T06:37:50.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.187+0000 7f17bb7fe700 1 -- 172.21.15.163:0/4155928003 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v93) v1 ==== 122+0+0 (secure 0 0 0) 0x7f17b4029020 con 0x7f17c40c83f0 2023-10-09T06:37:50.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.190+0000 7f17c9a52700 1 -- 172.21.15.163:0/4155928003 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f17a4067320 msgr2=0x7f17a40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:50.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.190+0000 7f17c9a52700 1 --2- 172.21.15.163:0/4155928003 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f17a4067320 0x7f17a40697d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f17c4169120 tx=0x7f17ac007ea0 comp rx=0 tx=0).stop 2023-10-09T06:37:50.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.190+0000 7f17c9a52700 1 -- 172.21.15.163:0/4155928003 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17c40c83f0 msgr2=0x7f17c414e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:50.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.190+0000 7f17c9a52700 1 --2- 172.21.15.163:0/4155928003 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17c40c83f0 0x7f17c414e9f0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f17b400b530 tx=0x7f17b401a000 comp rx=0 tx=0).stop 2023-10-09T06:37:50.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.190+0000 7f17c9a52700 1 -- 172.21.15.163:0/4155928003 shutdown_connections 2023-10-09T06:37:50.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.190+0000 7f17c9a52700 1 --2- 172.21.15.163:0/4155928003 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f17a4067320 0x7f17a40697d0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:50.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.190+0000 7f17c9a52700 1 --2- 172.21.15.163:0/4155928003 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17c40c83f0 0x7f17c414e9f0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:50.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.190+0000 7f17c9a52700 1 --2- 172.21.15.163:0/4155928003 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17c41568e0 0x7f17c414f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:50.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.190+0000 7f17c9a52700 1 --2- 172.21.15.163:0/4155928003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17c40ccee0 0x7f17c40afc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:50.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.190+0000 7f17c9a52700 1 -- 172.21.15.163:0/4155928003 >> 172.21.15.163:0/4155928003 conn(0x7f17c4068e10 msgr2=0x7f17c4158d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:50.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.190+0000 7f17c9a52700 1 -- 172.21.15.163:0/4155928003 shutdown_connections 2023-10-09T06:37:50.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.190+0000 7f17c9a52700 1 -- 172.21.15.163:0/4155928003 wait complete. 2023-10-09T06:37:50.202 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:37:50.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.317+0000 7fa44975c700 1 Processor -- start 2023-10-09T06:37:50.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.317+0000 7fa44975c700 1 -- start start 2023-10-09T06:37:50.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.318+0000 7fa44975c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa4440b1e80 0x7fa4440b2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:50.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.318+0000 7fa44975c700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4440cbd00 con 0x7fa44415a020 2023-10-09T06:37:50.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.318+0000 7fa44975c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4440af280 con 0x7fa4440cc190 2023-10-09T06:37:50.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.318+0000 7fa44975c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4440cc630 con 0x7fa4440b1e80 2023-10-09T06:37:50.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.318+0000 7fa4437fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fa44415a020 0x7fa44406e470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33530/0 (socket says 172.21.15.163:33530) 2023-10-09T06:37:50.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.318+0000 7fa4437fe700 1 -- 172.21.15.163:0/589408670 learned_addr learned my addr 172.21.15.163:0/589408670 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:50.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.319+0000 7fa4427fc700 1 -- 172.21.15.163:0/589408670 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2702263266 0 0) 0x7fa4440cbd00 con 0x7fa44415a020 2023-10-09T06:37:50.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.319+0000 7fa4427fc700 1 -- 172.21.15.163:0/589408670 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa430003570 con 0x7fa44415a020 2023-10-09T06:37:50.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.319+0000 7fa442ffd700 1 --2- 172.21.15.163:0/589408670 >> v2:172.21.15.181:3300/0 conn(0x7fa4440b1e80 0x7fa4440b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:50.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.319+0000 7fa4427fc700 1 -- 172.21.15.163:0/589408670 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4135656649 0 0) 0x7fa430003570 con 0x7fa44415a020 2023-10-09T06:37:50.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.319+0000 7fa4427fc700 1 -- 172.21.15.163:0/589408670 >> v2:172.21.15.181:3300/0 conn(0x7fa4440b1e80 msgr2=0x7fa4440b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:50.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.319+0000 7fa4427fc700 1 --2- 172.21.15.163:0/589408670 >> v2:172.21.15.181:3300/0 conn(0x7fa4440b1e80 0x7fa4440b2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:50.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.319+0000 7fa4427fc700 1 -- 172.21.15.163:0/589408670 >> v1:172.21.15.181:6789/0 conn(0x7fa4440cc190 legacy=0x7fa444159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:50.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.319+0000 7fa4427fc700 1 -- 172.21.15.163:0/589408670 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44415a4f0 con 0x7fa44415a020 2023-10-09T06:37:50.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.319+0000 7fa4427fc700 1 -- 172.21.15.163:0/589408670 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fa43c0017f0 con 0x7fa44415a020 2023-10-09T06:37:50.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.319+0000 7fa4427fc700 1 -- 172.21.15.163:0/589408670 >> v1:172.21.15.163:6789/0 conn(0x7fa44415a020 legacy=0x7fa44406e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:50.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.319+0000 7fa4427fc700 1 --2- 172.21.15.163:0/589408670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa430003a60 0x7fa430003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:50.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.319+0000 7fa4427fc700 1 --2- 172.21.15.163:0/589408670 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa430004810 0x7fa430004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:50.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.319+0000 7fa4427fc700 1 --2- 172.21.15.163:0/589408670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4300051c0 0x7fa43000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:50.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.319+0000 7fa4427fc700 1 -- 172.21.15.163:0/589408670 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4440cc630 con 0x7fa4300051c0 2023-10-09T06:37:50.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.319+0000 7fa4427fc700 1 -- 172.21.15.163:0/589408670 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43000c0d0 con 0x7fa430003a60 2023-10-09T06:37:50.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.319+0000 7fa4427fc700 1 -- 172.21.15.163:0/589408670 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa43000c290 con 0x7fa430004810 2023-10-09T06:37:50.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.319+0000 7fa442ffd700 1 --2- 172.21.15.163:0/589408670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa430003a60 0x7fa430003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:50.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.319+0000 7fa4437fe700 1 --2- 172.21.15.163:0/589408670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4300051c0 0x7fa43000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:50.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.319+0000 7fa43bfff700 1 --2- 172.21.15.163:0/589408670 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa430004810 0x7fa430004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:50.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.319+0000 7fa4437fe700 1 -- 172.21.15.163:0/589408670 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa430004810 msgr2=0x7fa430004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:50.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.319+0000 7fa4437fe700 1 --2- 172.21.15.163:0/589408670 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa430004810 0x7fa430004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:50.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.319+0000 7fa4437fe700 1 -- 172.21.15.163:0/589408670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa430003a60 msgr2=0x7fa430003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:50.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.319+0000 7fa4437fe700 1 --2- 172.21.15.163:0/589408670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa430003a60 0x7fa430003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:50.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.319+0000 7fa4437fe700 1 -- 172.21.15.163:0/589408670 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa430003570 con 0x7fa4300051c0 2023-10-09T06:37:50.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.319+0000 7fa442ffd700 1 --2- 172.21.15.163:0/589408670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa430003a60 0x7fa430003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:37:50.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.319+0000 7fa4437fe700 1 -- 172.21.15.163:0/589408670 reap_dead start 2023-10-09T06:37:50.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.320+0000 7fa4437fe700 1 --2- 172.21.15.163:0/589408670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4300051c0 0x7fa43000ba60 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fa44415a420 tx=0x7fa43c0021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee07032fe4ac422b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:50.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.320+0000 7fa4427fc700 1 -- 172.21.15.163:0/589408670 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43c0070d0 con 0x7fa4300051c0 2023-10-09T06:37:50.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.320+0000 7fa4427fc700 1 -- 172.21.15.163:0/589408670 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa43c007230 con 0x7fa4300051c0 2023-10-09T06:37:50.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.320+0000 7fa4427fc700 1 -- 172.21.15.163:0/589408670 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43c0075d0 con 0x7fa4300051c0 2023-10-09T06:37:50.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.320+0000 7fa44975c700 1 -- 172.21.15.163:0/589408670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4300051c0 msgr2=0x7fa43000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:50.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.320+0000 7fa44975c700 1 --2- 172.21.15.163:0/589408670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4300051c0 0x7fa43000ba60 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fa44415a420 tx=0x7fa43c0021d0 comp rx=0 tx=0).stop 2023-10-09T06:37:50.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.320+0000 7fa44975c700 1 -- 172.21.15.163:0/589408670 shutdown_connections 2023-10-09T06:37:50.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.320+0000 7fa44975c700 1 --2- 172.21.15.163:0/589408670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4300051c0 0x7fa43000ba60 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:50.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.320+0000 7fa44975c700 1 -- 172.21.15.163:0/589408670 >> 172.21.15.163:0/589408670 conn(0x7fa4440c9f30 msgr2=0x7fa44406dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:50.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.320+0000 7fa44975c700 1 -- 172.21.15.163:0/589408670 shutdown_connections 2023-10-09T06:37:50.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.320+0000 7fa44975c700 1 -- 172.21.15.163:0/589408670 wait complete. 2023-10-09T06:37:50.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.321+0000 7fa44975c700 1 Processor -- start 2023-10-09T06:37:50.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.321+0000 7fa44975c700 1 -- start start 2023-10-09T06:37:50.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.322+0000 7fa44975c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4300051c0 0x7fa4440abea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:50.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.322+0000 7fa44975c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4440c6660 0x7fa444169a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:50.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.322+0000 7fa44975c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa444169f60 0x7fa4440b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:50.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.322+0000 7fa44975c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44416a9f0 con 0x7fa4440c6660 2023-10-09T06:37:50.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.322+0000 7fa44975c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44416b090 con 0x7fa4300051c0 2023-10-09T06:37:50.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.322+0000 7fa44975c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa44416a430 con 0x7fa444169f60 2023-10-09T06:37:50.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.322+0000 7fa43bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4440c6660 0x7fa444169a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:50.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.322+0000 7fa43bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4440c6660 0x7fa444169a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41624/0 (socket says 172.21.15.163:41624) 2023-10-09T06:37:50.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.322+0000 7fa43bfff700 1 -- 172.21.15.163:0/436356954 learned_addr learned my addr 172.21.15.163:0/436356954 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:50.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.322+0000 7fa442ffd700 1 --2- 172.21.15.163:0/436356954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4300051c0 0x7fa4440abea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:50.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.322+0000 7fa43bfff700 1 -- 172.21.15.163:0/436356954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa444169f60 msgr2=0x7fa4440b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:50.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.322+0000 7fa43bfff700 1 --2- 172.21.15.163:0/436356954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa444169f60 0x7fa4440b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:50.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.322+0000 7fa43bfff700 1 -- 172.21.15.163:0/436356954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4300051c0 msgr2=0x7fa4440abea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:50.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.322+0000 7fa43bfff700 1 --2- 172.21.15.163:0/436356954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4300051c0 0x7fa4440abea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:50.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.322+0000 7fa43bfff700 1 -- 172.21.15.163:0/436356954 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44416a710 con 0x7fa4440c6660 2023-10-09T06:37:50.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.322+0000 7fa442ffd700 1 --2- 172.21.15.163:0/436356954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4300051c0 0x7fa4440abea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:37:50.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.323+0000 7fa43bfff700 1 --2- 172.21.15.163:0/436356954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4440c6660 0x7fa444169a20 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa42c002950 tx=0x7fa42c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:50.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.323+0000 7fa43affd700 1 -- 172.21.15.163:0/436356954 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42c002e90 con 0x7fa4440c6660 2023-10-09T06:37:50.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.323+0000 7fa43affd700 1 -- 172.21.15.163:0/436356954 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa42c00f8d0 con 0x7fa4440c6660 2023-10-09T06:37:50.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.323+0000 7fa44975c700 1 -- 172.21.15.163:0/436356954 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa44416ad20 con 0x7fa4440c6660 2023-10-09T06:37:50.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.323+0000 7fa43affd700 1 -- 172.21.15.163:0/436356954 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42c00fc90 con 0x7fa4440c6660 2023-10-09T06:37:50.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.323+0000 7fa44975c700 1 -- 172.21.15.163:0/436356954 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4440b0d80 con 0x7fa4440c6660 2023-10-09T06:37:50.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.324+0000 7fa44975c700 1 -- 172.21.15.163:0/436356954 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa44416ab60 con 0x7fa4440c6660 2023-10-09T06:37:50.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.324+0000 7fa43affd700 1 -- 172.21.15.163:0/436356954 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa42c006820 con 0x7fa4440c6660 2023-10-09T06:37:50.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.324+0000 7fa43affd700 1 --2- 172.21.15.163:0/436356954 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa424067320 0x7fa4240697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:50.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.324+0000 7fa43affd700 1 -- 172.21.15.163:0/436356954 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa42c084dd0 con 0x7fa4440c6660 2023-10-09T06:37:50.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.325+0000 7fa442ffd700 1 --2- 172.21.15.163:0/436356954 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa424067320 0x7fa4240697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:50.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.325+0000 7fa442ffd700 1 --2- 172.21.15.163:0/436356954 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa424067320 0x7fa4240697d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa4440b1c70 tx=0x7fa434008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:50.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.328+0000 7fa43affd700 1 -- 172.21.15.163:0/436356954 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa42c051a60 con 0x7fa4440c6660 2023-10-09T06:37:50.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.552+0000 7fa44975c700 1 -- 172.21.15.163:0/436356954 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa44416b200 con 0x7fa4440c6660 2023-10-09T06:37:50.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.552+0000 7fa43affd700 1 -- 172.21.15.163:0/436356954 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4613 (secure 0 0 0) 0x7fa42c055080 con 0x7fa4440c6660 2023-10-09T06:37:50.552 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:50.553 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":93,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1554520486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1554520486},{"type":"v1","addr":"172.21.15.181:6837","nonce":1554520486}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1695339871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1695339871},{"type":"v1","addr":"172.21.15.181:6836","nonce":1695339871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":7515,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3668692283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3668692283},{"type":"v1","addr":"172.21.15.163:6834","nonce":3668692283}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:37:46.529939+0000","modified":"2023-10-09T06:37:50.185660+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7518":{"gid":7518,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/3043729326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3043729326},{"type":"v1","addr":"172.21.15.163:6835","nonce":3043729326}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-09T06:37:50.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.554+0000 7fa44975c700 1 -- 172.21.15.163:0/436356954 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa424067320 msgr2=0x7fa4240697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:50.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.554+0000 7fa44975c700 1 --2- 172.21.15.163:0/436356954 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa424067320 0x7fa4240697d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa4440b1c70 tx=0x7fa434008040 comp rx=0 tx=0).stop 2023-10-09T06:37:50.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.554+0000 7fa44975c700 1 -- 172.21.15.163:0/436356954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4440c6660 msgr2=0x7fa444169a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:50.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.554+0000 7fa44975c700 1 --2- 172.21.15.163:0/436356954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4440c6660 0x7fa444169a20 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa42c002950 tx=0x7fa42c007d30 comp rx=0 tx=0).stop 2023-10-09T06:37:50.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.554+0000 7fa44975c700 1 -- 172.21.15.163:0/436356954 shutdown_connections 2023-10-09T06:37:50.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.554+0000 7fa44975c700 1 --2- 172.21.15.163:0/436356954 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa424067320 0x7fa4240697d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:50.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.554+0000 7fa44975c700 1 --2- 172.21.15.163:0/436356954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa444169f60 0x7fa4440b0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:50.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.554+0000 7fa44975c700 1 --2- 172.21.15.163:0/436356954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4300051c0 0x7fa4440abea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:50.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.554+0000 7fa44975c700 1 --2- 172.21.15.163:0/436356954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4440c6660 0x7fa444169a20 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:50.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.554+0000 7fa44975c700 1 -- 172.21.15.163:0/436356954 >> 172.21.15.163:0/436356954 conn(0x7fa4440c9f30 msgr2=0x7fa4440c19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:50.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.554+0000 7fa44975c700 1 -- 172.21.15.163:0/436356954 shutdown_connections 2023-10-09T06:37:50.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.554+0000 7fa44975c700 1 -- 172.21.15.163:0/436356954 wait complete. 2023-10-09T06:37:50.557 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 93 2023-10-09T06:37:50.567 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:37:50.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.680+0000 7f40ac518700 1 Processor -- start 2023-10-09T06:37:50.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.680+0000 7f40ac518700 1 -- start start 2023-10-09T06:37:50.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.681+0000 7f40ac518700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f40a4159890 0x7f40a415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:50.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.681+0000 7f40ac518700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f40a4069560 0x7f40a40699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:50.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.681+0000 7f40ac518700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a4069ee0 con 0x7f40a4159c60 2023-10-09T06:37:50.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.681+0000 7f40ac518700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a415a160 con 0x7f40a4159890 2023-10-09T06:37:50.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.681+0000 7f40ac518700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a40d13a0 con 0x7f40a4069560 2023-10-09T06:37:50.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.682+0000 7f40aa2b4700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f40a4159890 0x7f40a415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:50.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.682+0000 7f40a9ab3700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f40a4159c60 0x7f40a415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33534/0 (socket says 172.21.15.163:33534) 2023-10-09T06:37:50.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.682+0000 7f40a9ab3700 1 -- 172.21.15.163:0/3732252939 learned_addr learned my addr 172.21.15.163:0/3732252939 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:50.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.682+0000 7f40aaab5700 1 --2- 172.21.15.163:0/3732252939 >> v2:172.21.15.181:3300/0 conn(0x7f40a4069560 0x7f40a40699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:50.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.682+0000 7f40aa2b4700 1 -- 172.21.15.163:0/3732252939 >> v2:172.21.15.181:3300/0 conn(0x7f40a4069560 msgr2=0x7f40a40699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:50.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.682+0000 7f40aa2b4700 1 --2- 172.21.15.163:0/3732252939 >> v2:172.21.15.181:3300/0 conn(0x7f40a4069560 0x7f40a40699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:50.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.682+0000 7f40aa2b4700 1 -- 172.21.15.163:0/3732252939 >> v1:172.21.15.163:6789/0 conn(0x7f40a4159c60 legacy=0x7f40a415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:50.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.682+0000 7f40aa2b4700 1 -- 172.21.15.163:0/3732252939 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a415e140 con 0x7f40a4159890 2023-10-09T06:37:50.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.682+0000 7f40a92b2700 1 -- 172.21.15.163:0/3732252939 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2080540006 0 0) 0x7f40a4069ee0 con 0x7f40a4159c60 2023-10-09T06:37:50.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.682+0000 7f40aa2b4700 1 --2- 172.21.15.163:0/3732252939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40a4159890 0x7f40a415abd0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f40a406a890 tx=0x7f4094009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940bacc344ca5e6a server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:50.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.682+0000 7f40a92b2700 1 -- 172.21.15.163:0/3732252939 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409400e070 con 0x7f40a4159890 2023-10-09T06:37:50.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.682+0000 7f40a92b2700 1 -- 172.21.15.163:0/3732252939 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409400bb60 con 0x7f40a4159890 2023-10-09T06:37:50.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.682+0000 7f40a92b2700 1 -- 172.21.15.163:0/3732252939 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4094012430 con 0x7f40a4159890 2023-10-09T06:37:50.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.683+0000 7f40ac518700 1 -- 172.21.15.163:0/3732252939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40a4159890 msgr2=0x7f40a415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:50.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.683+0000 7f40ac518700 1 --2- 172.21.15.163:0/3732252939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40a4159890 0x7f40a415abd0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f40a406a890 tx=0x7f4094009780 comp rx=0 tx=0).stop 2023-10-09T06:37:50.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.683+0000 7f40ac518700 1 -- 172.21.15.163:0/3732252939 shutdown_connections 2023-10-09T06:37:50.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.683+0000 7f40ac518700 1 --2- 172.21.15.163:0/3732252939 >> v2:172.21.15.181:3300/0 conn(0x7f40a4069560 0x7f40a40699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:50.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.683+0000 7f40ac518700 1 --2- 172.21.15.163:0/3732252939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40a4159890 0x7f40a415abd0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:50.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.683+0000 7f40ac518700 1 -- 172.21.15.163:0/3732252939 >> 172.21.15.163:0/3732252939 conn(0x7f40a4068e20 msgr2=0x7f40a40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:50.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.683+0000 7f40ac518700 1 -- 172.21.15.163:0/3732252939 shutdown_connections 2023-10-09T06:37:50.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.683+0000 7f40ac518700 1 -- 172.21.15.163:0/3732252939 wait complete. 2023-10-09T06:37:50.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.684+0000 7f40ac518700 1 Processor -- start 2023-10-09T06:37:50.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.684+0000 7f40ac518700 1 -- start start 2023-10-09T06:37:50.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.685+0000 7f40ac518700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a4069560 0x7f40a4156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:50.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.685+0000 7f40ac518700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40a4159890 0x7f40a40b5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:50.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.685+0000 7f40ac518700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40a4159c60 0x7f40a4153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:50.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.685+0000 7f40ac518700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a40b3fd0 con 0x7f40a4159890 2023-10-09T06:37:50.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.685+0000 7f40ac518700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a40b36a0 con 0x7f40a4069560 2023-10-09T06:37:50.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.685+0000 7f40ac518700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a40b3980 con 0x7f40a4159c60 2023-10-09T06:37:50.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.685+0000 7f40a9ab3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40a4159890 0x7f40a40b5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:50.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.685+0000 7f40a9ab3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40a4159890 0x7f40a40b5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41652/0 (socket says 172.21.15.163:41652) 2023-10-09T06:37:50.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.685+0000 7f40a9ab3700 1 -- 172.21.15.163:0/2225982061 learned_addr learned my addr 172.21.15.163:0/2225982061 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:50.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.685+0000 7f40aa2b4700 1 --2- 172.21.15.163:0/2225982061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a4069560 0x7f40a4156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:50.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.685+0000 7f40aaab5700 1 --2- 172.21.15.163:0/2225982061 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40a4159c60 0x7f40a4153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:50.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.685+0000 7f40a9ab3700 1 -- 172.21.15.163:0/2225982061 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40a4159c60 msgr2=0x7f40a4153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:50.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.685+0000 7f40a9ab3700 1 --2- 172.21.15.163:0/2225982061 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40a4159c60 0x7f40a4153ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:50.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.685+0000 7f40a9ab3700 1 -- 172.21.15.163:0/2225982061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a4069560 msgr2=0x7f40a4156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:50.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.685+0000 7f40a9ab3700 1 --2- 172.21.15.163:0/2225982061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a4069560 0x7f40a4156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:50.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.685+0000 7f40a9ab3700 1 -- 172.21.15.163:0/2225982061 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4094009a20 con 0x7f40a4159890 2023-10-09T06:37:50.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.685+0000 7f40aa2b4700 1 --2- 172.21.15.163:0/2225982061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a4069560 0x7f40a4156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:37:50.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.685+0000 7f40a9ab3700 1 --2- 172.21.15.163:0/2225982061 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40a4159890 0x7f40a40b5bd0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f409800c5b0 tx=0x7f4098001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:50.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.685+0000 7f40a27fc700 1 -- 172.21.15.163:0/2225982061 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098005b30 con 0x7f40a4159890 2023-10-09T06:37:50.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.685+0000 7f40a27fc700 1 -- 172.21.15.163:0/2225982061 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4098005c90 con 0x7f40a4159890 2023-10-09T06:37:50.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.686+0000 7f40ac518700 1 -- 172.21.15.163:0/2225982061 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a40b3c60 con 0x7f40a4159890 2023-10-09T06:37:50.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.686+0000 7f40a27fc700 1 -- 172.21.15.163:0/2225982061 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098007300 con 0x7f40a4159890 2023-10-09T06:37:50.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.686+0000 7f40ac518700 1 -- 172.21.15.163:0/2225982061 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a4156970 con 0x7f40a4159890 2023-10-09T06:37:50.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.686+0000 7f40ac518700 1 -- 172.21.15.163:0/2225982061 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a40b4140 con 0x7f40a4159890 2023-10-09T06:37:50.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.687+0000 7f40a27fc700 1 -- 172.21.15.163:0/2225982061 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f409801b3b0 con 0x7f40a4159890 2023-10-09T06:37:50.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.687+0000 7f40a27fc700 1 --2- 172.21.15.163:0/2225982061 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4090067320 0x7f40900697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:50.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.687+0000 7f40a27fc700 1 -- 172.21.15.163:0/2225982061 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f40980841b0 con 0x7f40a4159890 2023-10-09T06:37:50.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.687+0000 7f40aa2b4700 1 --2- 172.21.15.163:0/2225982061 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4090067320 0x7f40900697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:50.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.688+0000 7f40aa2b4700 1 --2- 172.21.15.163:0/2225982061 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4090067320 0x7f40900697d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f40940097c0 tx=0x7f4094019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:50.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.691+0000 7f40a27fc700 1 -- 172.21.15.163:0/2225982061 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4098050e40 con 0x7f40a4159890 2023-10-09T06:37:50.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.876+0000 7f40ac518700 1 -- 172.21.15.163:0/2225982061 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40a404c3c0 con 0x7f40a4159890 2023-10-09T06:37:50.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.877+0000 7f40a27fc700 1 -- 172.21.15.163:0/2225982061 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+15412 (secure 0 0 0) 0x7f4098054460 con 0x7f40a4159890 2023-10-09T06:37:50.877 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:50.877 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":137,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:37:48.164135+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:37:44.861841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-10-09T06:37:45.496241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:37:50.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.879+0000 7f40ac518700 1 -- 172.21.15.163:0/2225982061 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4090067320 msgr2=0x7f40900697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:50.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.879+0000 7f40ac518700 1 --2- 172.21.15.163:0/2225982061 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4090067320 0x7f40900697d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f40940097c0 tx=0x7f4094019000 comp rx=0 tx=0).stop 2023-10-09T06:37:50.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.879+0000 7f40ac518700 1 -- 172.21.15.163:0/2225982061 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40a4159890 msgr2=0x7f40a40b5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:50.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.879+0000 7f40ac518700 1 --2- 172.21.15.163:0/2225982061 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40a4159890 0x7f40a40b5bd0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f409800c5b0 tx=0x7f4098001070 comp rx=0 tx=0).stop 2023-10-09T06:37:50.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.879+0000 7f40ac518700 1 -- 172.21.15.163:0/2225982061 shutdown_connections 2023-10-09T06:37:50.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.879+0000 7f40ac518700 1 --2- 172.21.15.163:0/2225982061 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4090067320 0x7f40900697d0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:50.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.879+0000 7f40ac518700 1 --2- 172.21.15.163:0/2225982061 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40a4159c60 0x7f40a4153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:50.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.879+0000 7f40ac518700 1 --2- 172.21.15.163:0/2225982061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40a4069560 0x7f40a4156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:50.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.879+0000 7f40ac518700 1 --2- 172.21.15.163:0/2225982061 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40a4159890 0x7f40a40b5bd0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:50.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.879+0000 7f40ac518700 1 -- 172.21.15.163:0/2225982061 >> 172.21.15.163:0/2225982061 conn(0x7f40a4068e20 msgr2=0x7f40a4155290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:50.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.879+0000 7f40ac518700 1 -- 172.21.15.163:0/2225982061 shutdown_connections 2023-10-09T06:37:50.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:50.879+0000 7f40ac518700 1 -- 172.21.15.163:0/2225982061 wait complete. 2023-10-09T06:37:50.892 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-09T06:37:50.892 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:37:51.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.007+0000 7f8ec3fff700 1 Processor -- start 2023-10-09T06:37:51.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.007+0000 7f8ec3fff700 1 -- start start 2023-10-09T06:37:51.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.008+0000 7f8ec3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ec415d230 0x7f8ec415d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:51.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.008+0000 7f8ec3fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ec40b2980 con 0x7f8ec4159420 2023-10-09T06:37:51.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.008+0000 7f8ec3fff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ec415dbb0 con 0x7f8ec41597f0 2023-10-09T06:37:51.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.008+0000 7f8ec3fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec40b11d0 con 0x7f8ec415d230 2023-10-09T06:37:51.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.008+0000 7f8ec2ffd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f8ec4159420 0x7f8ec40b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33548/0 (socket says 172.21.15.163:33548) 2023-10-09T06:37:51.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.008+0000 7f8ec2ffd700 1 -- 172.21.15.163:0/828287172 learned_addr learned my addr 172.21.15.163:0/828287172 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:51.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.008+0000 7f8ec37fe700 1 --2- 172.21.15.163:0/828287172 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ec415d230 0x7f8ec415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:51.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.008+0000 7f8ec1ffb700 1 -- 172.21.15.163:0/828287172 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2803084133 0 0) 0x7f8ec40b2980 con 0x7f8ec4159420 2023-10-09T06:37:51.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.008+0000 7f8ec1ffb700 1 -- 172.21.15.163:0/828287172 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8eac003570 con 0x7f8ec4159420 2023-10-09T06:37:51.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.008+0000 7f8ec1ffb700 1 -- 172.21.15.163:0/828287172 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1907574222 0 0) 0x7f8ec415dbb0 con 0x7f8ec41597f0 2023-10-09T06:37:51.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.008+0000 7f8ec1ffb700 1 -- 172.21.15.163:0/828287172 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ec40b2980 con 0x7f8ec41597f0 2023-10-09T06:37:51.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.008+0000 7f8ec1ffb700 1 -- 172.21.15.163:0/828287172 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3962131312 0 0) 0x7f8eac003570 con 0x7f8ec4159420 2023-10-09T06:37:51.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.008+0000 7f8ec1ffb700 1 -- 172.21.15.163:0/828287172 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ec415d230 msgr2=0x7f8ec415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:51.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.008+0000 7f8ec1ffb700 1 --2- 172.21.15.163:0/828287172 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ec415d230 0x7f8ec415d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:51.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.008+0000 7f8ec1ffb700 1 -- 172.21.15.163:0/828287172 >> v1:172.21.15.181:6789/0 conn(0x7f8ec41597f0 legacy=0x7f8ec40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:51.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.008+0000 7f8ec1ffb700 1 -- 172.21.15.163:0/828287172 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec40b14b0 con 0x7f8ec4159420 2023-10-09T06:37:51.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.008+0000 7f8ec1ffb700 1 -- 172.21.15.163:0/828287172 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f8eb40017f0 con 0x7f8ec4159420 2023-10-09T06:37:51.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.009+0000 7f8ec1ffb700 1 -- 172.21.15.163:0/828287172 >> v1:172.21.15.163:6789/0 conn(0x7f8ec4159420 legacy=0x7f8ec40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:51.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.009+0000 7f8ec1ffb700 1 --2- 172.21.15.163:0/828287172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8eac003c20 0x7f8eac004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:51.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.009+0000 7f8ec1ffb700 1 --2- 172.21.15.163:0/828287172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8eac004810 0x7f8eac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:51.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.009+0000 7f8ec1ffb700 1 --2- 172.21.15.163:0/828287172 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ec415d230 0x7f8eac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:51.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.009+0000 7f8ec1ffb700 1 -- 172.21.15.163:0/828287172 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec40b11d0 con 0x7f8eac003c20 2023-10-09T06:37:51.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.009+0000 7f8ec1ffb700 1 -- 172.21.15.163:0/828287172 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac00bb40 con 0x7f8eac004810 2023-10-09T06:37:51.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.009+0000 7f8ec1ffb700 1 -- 172.21.15.163:0/828287172 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac00bd00 con 0x7f8ec415d230 2023-10-09T06:37:51.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.009+0000 7f8ec2ffd700 1 --2- 172.21.15.163:0/828287172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8eac003c20 0x7f8eac004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:51.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.009+0000 7f8ec27fc700 1 --2- 172.21.15.163:0/828287172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8eac004810 0x7f8eac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:51.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.009+0000 7f8ec37fe700 1 --2- 172.21.15.163:0/828287172 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ec415d230 0x7f8eac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:51.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.009+0000 7f8ec2ffd700 1 -- 172.21.15.163:0/828287172 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ec415d230 msgr2=0x7f8eac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:51.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.009+0000 7f8ec2ffd700 1 --2- 172.21.15.163:0/828287172 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ec415d230 0x7f8eac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:51.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.009+0000 7f8ec2ffd700 1 -- 172.21.15.163:0/828287172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8eac004810 msgr2=0x7f8eac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:51.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.009+0000 7f8ec2ffd700 1 --2- 172.21.15.163:0/828287172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8eac004810 0x7f8eac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:51.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.009+0000 7f8ec2ffd700 1 -- 172.21.15.163:0/828287172 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eac003570 con 0x7f8eac003c20 2023-10-09T06:37:51.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.009+0000 7f8ec2ffd700 1 --2- 172.21.15.163:0/828287172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8eac003c20 0x7f8eac004010 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f8eb400a8f0 tx=0x7f8eb4002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a70d45ae1485014 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:51.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.009+0000 7f8ec1ffb700 1 -- 172.21.15.163:0/828287172 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb40070d0 con 0x7f8eac003c20 2023-10-09T06:37:51.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.009+0000 7f8ec1ffb700 1 -- 172.21.15.163:0/828287172 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb4007250 con 0x7f8eac003c20 2023-10-09T06:37:51.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.009+0000 7f8ec1ffb700 1 -- 172.21.15.163:0/828287172 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4007610 con 0x7f8eac003c20 2023-10-09T06:37:51.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.010+0000 7f8ec3fff700 1 -- 172.21.15.163:0/828287172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8eac003c20 msgr2=0x7f8eac004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:51.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.010+0000 7f8ec3fff700 1 --2- 172.21.15.163:0/828287172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8eac003c20 0x7f8eac004010 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f8eb400a8f0 tx=0x7f8eb4002190 comp rx=0 tx=0).stop 2023-10-09T06:37:51.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.010+0000 7f8ec37fe700 1 -- 172.21.15.163:0/828287172 reap_dead start 2023-10-09T06:37:51.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.010+0000 7f8ec3fff700 1 -- 172.21.15.163:0/828287172 shutdown_connections 2023-10-09T06:37:51.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.010+0000 7f8ec3fff700 1 -- 172.21.15.163:0/828287172 >> 172.21.15.163:0/828287172 conn(0x7f8ec40d0660 msgr2=0x7f8ec40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:51.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.010+0000 7f8ec3fff700 1 -- 172.21.15.163:0/828287172 shutdown_connections 2023-10-09T06:37:51.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.010+0000 7f8ec3fff700 1 -- 172.21.15.163:0/828287172 wait complete. 2023-10-09T06:37:51.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.011+0000 7f8ec3fff700 1 Processor -- start 2023-10-09T06:37:51.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.011+0000 7f8ec3fff700 1 -- start start 2023-10-09T06:37:51.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.012+0000 7f8ec3fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8eac003c20 0x7f8ec414a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:51.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.012+0000 7f8ec3fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ec40cf340 0x7f8ec40c5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:51.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.012+0000 7f8ec3fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ec40c8170 0x7f8ec40c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:51.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.012+0000 7f8ec3fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec40c8b20 con 0x7f8eac003c20 2023-10-09T06:37:51.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.012+0000 7f8ec3fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec414a6d0 con 0x7f8ec40c8170 2023-10-09T06:37:51.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.012+0000 7f8ec3fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec40ca190 con 0x7f8ec40cf340 2023-10-09T06:37:51.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.012+0000 7f8ec2ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8eac003c20 0x7f8ec414a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:51.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.012+0000 7f8ec2ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8eac003c20 0x7f8ec414a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41682/0 (socket says 172.21.15.163:41682) 2023-10-09T06:37:51.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.012+0000 7f8ec2ffd700 1 -- 172.21.15.163:0/1435861278 learned_addr learned my addr 172.21.15.163:0/1435861278 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:51.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.012+0000 7f8ec37fe700 1 --2- 172.21.15.163:0/1435861278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ec40c8170 0x7f8ec40c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:51.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.012+0000 7f8ec27fc700 1 --2- 172.21.15.163:0/1435861278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ec40cf340 0x7f8ec40c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:51.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.012+0000 7f8ec2ffd700 1 -- 172.21.15.163:0/1435861278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ec40cf340 msgr2=0x7f8ec40c5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:51.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.012+0000 7f8ec2ffd700 1 --2- 172.21.15.163:0/1435861278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ec40cf340 0x7f8ec40c5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:51.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.012+0000 7f8ec2ffd700 1 -- 172.21.15.163:0/1435861278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ec40c8170 msgr2=0x7f8ec40c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:51.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.012+0000 7f8ec2ffd700 1 --2- 172.21.15.163:0/1435861278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ec40c8170 0x7f8ec40c85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:51.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.012+0000 7f8ec2ffd700 1 -- 172.21.15.163:0/1435861278 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec40cf890 con 0x7f8eac003c20 2023-10-09T06:37:51.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.012+0000 7f8ec2ffd700 1 --2- 172.21.15.163:0/1435861278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8eac003c20 0x7f8ec414a190 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f8eb400a3b0 tx=0x7f8eb4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:51.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.012+0000 7f8eaaffd700 1 -- 172.21.15.163:0/1435861278 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4004bb0 con 0x7f8eac003c20 2023-10-09T06:37:51.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.012+0000 7f8eaaffd700 1 -- 172.21.15.163:0/1435861278 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb4022070 con 0x7f8eac003c20 2023-10-09T06:37:51.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.012+0000 7f8eaaffd700 1 -- 172.21.15.163:0/1435861278 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4004d30 con 0x7f8eac003c20 2023-10-09T06:37:51.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.012+0000 7f8ec3fff700 1 -- 172.21.15.163:0/1435861278 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ec40cfb70 con 0x7f8eac003c20 2023-10-09T06:37:51.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.012+0000 7f8ec3fff700 1 -- 172.21.15.163:0/1435861278 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ec40d0130 con 0x7f8eac003c20 2023-10-09T06:37:51.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.013+0000 7f8ec3fff700 1 -- 172.21.15.163:0/1435861278 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ec40d02f0 con 0x7f8eac003c20 2023-10-09T06:37:51.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.014+0000 7f8eaaffd700 1 -- 172.21.15.163:0/1435861278 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8eb401b670 con 0x7f8eac003c20 2023-10-09T06:37:51.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.014+0000 7f8eaaffd700 1 --2- 172.21.15.163:0/1435861278 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8eb0067320 0x7f8eb00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:51.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.014+0000 7f8eaaffd700 1 -- 172.21.15.163:0/1435861278 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8eb4086830 con 0x7f8eac003c20 2023-10-09T06:37:51.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.014+0000 7f8ec27fc700 1 --2- 172.21.15.163:0/1435861278 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8eb0067320 0x7f8eb00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:51.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.015+0000 7f8ec27fc700 1 --2- 172.21.15.163:0/1435861278 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8eb0067320 0x7f8eb00697d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8001120 tx=0x7f8eb8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:51.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.018+0000 7f8eaaffd700 1 -- 172.21.15.163:0/1435861278 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8eb40535d0 con 0x7f8eac003c20 2023-10-09T06:37:51.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.199+0000 7f8ec3fff700 1 -- 172.21.15.163:0/1435861278 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ec404c3c0 con 0x7f8eb0067320 2023-10-09T06:37:51.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.207+0000 7f8eaaffd700 1 -- 172.21.15.163:0/1435861278 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+279213 (secure 0 0 0) 0x7f8ec404c3c0 con 0x7f8eb0067320 2023-10-09T06:37:51.211 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:51.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.210+0000 7f8ec3fff700 1 -- 172.21.15.163:0/1435861278 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8eb0067320 msgr2=0x7f8eb00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:51.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.210+0000 7f8ec3fff700 1 --2- 172.21.15.163:0/1435861278 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8eb0067320 0x7f8eb00697d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8001120 tx=0x7f8eb8009040 comp rx=0 tx=0).stop 2023-10-09T06:37:51.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.210+0000 7f8ec3fff700 1 -- 172.21.15.163:0/1435861278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8eac003c20 msgr2=0x7f8ec414a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:51.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.210+0000 7f8ec3fff700 1 --2- 172.21.15.163:0/1435861278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8eac003c20 0x7f8ec414a190 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f8eb400a3b0 tx=0x7f8eb4016000 comp rx=0 tx=0).stop 2023-10-09T06:37:51.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.210+0000 7f8ec3fff700 1 -- 172.21.15.163:0/1435861278 shutdown_connections 2023-10-09T06:37:51.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.210+0000 7f8ec3fff700 1 --2- 172.21.15.163:0/1435861278 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8eb0067320 0x7f8eb00697d0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:51.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.210+0000 7f8ec3fff700 1 --2- 172.21.15.163:0/1435861278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8eac003c20 0x7f8ec414a190 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:51.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.210+0000 7f8ec3fff700 1 --2- 172.21.15.163:0/1435861278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ec40c8170 0x7f8ec40c85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:51.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.210+0000 7f8ec3fff700 1 --2- 172.21.15.163:0/1435861278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ec40cf340 0x7f8ec40c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:51.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.210+0000 7f8ec3fff700 1 -- 172.21.15.163:0/1435861278 >> 172.21.15.163:0/1435861278 conn(0x7f8ec40d0660 msgr2=0x7f8ec40cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:51.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.210+0000 7f8ec3fff700 1 -- 172.21.15.163:0/1435861278 shutdown_connections 2023-10-09T06:37:51.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.210+0000 7f8ec3fff700 1 -- 172.21.15.163:0/1435861278 wait complete. 2023-10-09T06:37:51.214 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:37:51.222 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2023-10-09T06:37:50.402700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2425384,"kb_used_data":14312,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752549336,"statfs":{"total":773094113280,"available":770610520064,"internally_reserved":0,"allocated":14655488,"data_stored":7165976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.259477"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.150485+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:51:33.426962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149695+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:35:38.378414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149092+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:01:46.574508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.146022+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:09:27.400457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.147611+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:37:20.004643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.144979+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:55:12.397428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.147056+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:59:31.649021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.147727+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:30:10.930315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.147575+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:10:02.531578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.147553+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:33:04.749404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149103+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:18:36.804502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.145710+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:34:05.362873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149392+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:00:51.883913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.144958+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:04:18.850838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.145811+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:53:17.680797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.147810+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:34:00.040661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.144992+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:31:32.310904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.145331+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:00:51.883913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.147799+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:22:44.782785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149528+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:11:45.248711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.147628+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:28:29.141519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.148789+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:59:12.303101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149595+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:24:27.771345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.145700+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:19:19.201484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.151603+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:34:05.362873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149469+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:19:19.201484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.144969+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:59:31.649021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149695+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:24:42.401357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149646+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:03:33.845927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.145665+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:42:15.133743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149518+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:16:35.538841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.145331+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:49:26.936922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":256,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-09T06:37:43.180951+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:37:43.180951+0000","last_peered":"2023-10-09T06:37:43.180951+0000","last_clean":"2023-10-09T06:37:43.180951+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:37:43.180951+0000","last_undegraded":"2023-10-09T06:37:43.180951+0000","last_fullsized":"2023-10-09T06:37:43.180951+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574939,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303608,"kb_used_data":2224,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068232,"statfs":{"total":96636764160,"available":96325869568,"internally_reserved":0,"allocated":2277376,"data_stored":1338523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52000000000000002,"15min":0.50700000000000001},"min":{"1min":0.34799999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.37,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67600000000000005,"5min":0.753,"15min":0.753},"last":0.33400000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.503,"15min":0.51900000000000002},"min":{"1min":0.33600000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.74299999999999999,"5min":0.77400000000000002,"15min":1.2030000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.378,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.41099999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53200000000000003,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75800000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.37},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.44,"5min":0.36399999999999999,"15min":0.315},"max":{"1min":0.77900000000000003,"5min":0.80700000000000005,"15min":0.83099999999999996},"last":0.50900000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53800000000000003,"15min":0.55000000000000004},"min":{"1min":0.41199999999999998,"5min":0.35899999999999999,"15min":0.33500000000000002},"max":{"1min":0.68600000000000005,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55600000000000005,"15min":0.59199999999999997},"min":{"1min":0.45300000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.754,"5min":0.82099999999999995,"15min":1.1479999999999999},"last":0.443}]},{"osd":4,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.48199999999999998,"15min":0.51900000000000002},"min":{"1min":0.35599999999999998,"5min":0.30299999999999999,"15min":0.24399999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.93999999999999995},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56599999999999995,"15min":0.60599999999999998},"min":{"1min":0.36799999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76700000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.56799999999999995}]},{"osd":5,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52500000000000002,"15min":0.56999999999999995},"min":{"1min":0.36499999999999999,"5min":0.32500000000000001,"15min":0.307},"max":{"1min":0.81999999999999995,"5min":0.85799999999999998,"15min":0.91100000000000003},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59599999999999997,"15min":0.62},"min":{"1min":0.52900000000000003,"5min":0.41699999999999998,"15min":0.34200000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.54500000000000004}]},{"osd":6,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55700000000000005,"15min":0.59299999999999997},"min":{"1min":0.38600000000000001,"5min":0.33500000000000002,"15min":0.28699999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.88600000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.56100000000000005,"15min":0.61399999999999999},"min":{"1min":0.45100000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.84599999999999997},"last":0.60599999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574938,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":748155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.50800000000000001,"15min":0.502},"min":{"1min":0.35499999999999998,"5min":0.32000000000000001,"15min":0.30399999999999999},"max":{"1min":0.66500000000000004,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.50700000000000001,"15min":0.51000000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70399999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.56799999999999995}]},{"osd":1,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.502,"15min":0.51200000000000001},"min":{"1min":0.33600000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68300000000000005,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52900000000000003,"15min":0.53200000000000003},"min":{"1min":0.38700000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.77100000000000002,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.54100000000000004}]},{"osd":2,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.34499999999999997,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.72399999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51100000000000001,"15min":0.52900000000000003},"min":{"1min":0.39000000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.66600000000000004,"5min":0.67500000000000004,"15min":0.752},"last":0.55400000000000005}]},{"osd":3,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.38900000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66600000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.54300000000000004,"15min":0.68799999999999994},"min":{"1min":0.40400000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.752,"5min":0.752,"15min":4.7510000000000003},"last":0.45900000000000002}]},{"osd":4,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46400000000000002,"15min":0.47699999999999998},"min":{"1min":0.29899999999999999,"5min":0.28199999999999997,"15min":0.27900000000000003},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.76400000000000001},"last":0.436},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56100000000000005,"15min":0.58099999999999996},"min":{"1min":0.41799999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.77200000000000002,"5min":0.81299999999999994,"15min":0.83699999999999997},"last":0.60499999999999998}]},{"osd":5,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.52100000000000002,"15min":0.53000000000000003},"min":{"1min":0.31900000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.66100000000000003,"5min":0.79200000000000004,"15min":0.81200000000000006},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.59499999999999997},"min":{"1min":0.46899999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76000000000000001,"5min":0.78000000000000003,"15min":0.86599999999999999},"last":0.59399999999999997}]},{"osd":7,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55200000000000005,"15min":0.58199999999999996},"min":{"1min":0.41399999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80500000000000005,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58799999999999997,"15min":0.58899999999999997},"min":{"1min":0.46000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78400000000000003,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.57899999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":748155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.316,"15min":0.311},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.503,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.34899999999999998,"15min":0.35799999999999998},"min":{"1min":0.28399999999999997,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.51900000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.44}]},{"osd":2,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.34399999999999997,"15min":0.35099999999999998},"min":{"1min":0.215,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.53700000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.38300000000000001,"15min":0.38500000000000001},"min":{"1min":0.30299999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.55300000000000005,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.47399999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.36499999999999999,"15min":0.36399999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.55800000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.34599999999999997,"15min":0.53500000000000003},"min":{"1min":0.23400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.40400000000000003,"5min":0.56000000000000005,"15min":5.8529999999999998},"last":0.42299999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58299999999999996,"15min":0.57499999999999996},"min":{"1min":0.36199999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78300000000000003,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.40000000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.81100000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.66100000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57699999999999996,"15min":0.74099999999999999},"min":{"1min":0.36499999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88,"5min":0.88,"15min":5.9459999999999997},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61199999999999999,"15min":0.61099999999999999},"min":{"1min":0.38900000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82599999999999996,"5min":0.92700000000000005,"15min":0.93999999999999995},"last":0.67900000000000005}]},{"osd":6,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54900000000000004,"15min":0.71199999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79100000000000004,"5min":0.79600000000000004,"15min":5.9160000000000004},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.379,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84799999999999998,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.77400000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57199999999999995,"15min":0.58599999999999997},"min":{"1min":0.39000000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.80900000000000005,"5min":0.89500000000000002,"15min":0.97099999999999997},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57099999999999995,"15min":0.749},"min":{"1min":0.41399999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":5.9729999999999999},"last":0.60799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303608,"kb_used_data":2224,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068232,"statfs":{"total":96636764160,"available":96325869568,"internally_reserved":0,"allocated":2277376,"data_stored":1338523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.30199999999999999,"15min":0.315},"min":{"1min":0.216,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.433,"5min":0.498,"15min":0.498},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.32100000000000001,"15min":0.34399999999999997},"min":{"1min":0.222,"5min":0.214,"15min":0.214},"max":{"1min":0.497,"5min":0.53600000000000003,"15min":0.94499999999999995},"last":0.39400000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35199999999999998,"15min":0.36199999999999999},"min":{"1min":0.25900000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.50900000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.371,"15min":0.38800000000000001},"min":{"1min":0.30499999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.56499999999999995,"5min":0.69399999999999995,"15min":0.91700000000000004},"last":0.33900000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.32800000000000001,"15min":0.38400000000000001},"min":{"1min":0.23699999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":1.554},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.35999999999999999,"15min":0.39700000000000002},"min":{"1min":0.27700000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54400000000000004,"5min":0.58099999999999996,"15min":0.79000000000000004},"last":0.313}]},{"osd":4,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53600000000000003,"15min":0.55100000000000005},"min":{"1min":0.41199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":1.212},"last":0.62},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.72099999999999997},"min":{"1min":0.42199999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":5.484},"last":0.56499999999999995}]},{"osd":5,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.72499999999999998},"min":{"1min":0.41999999999999998,"5min":0.35799999999999998,"15min":0.35499999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":5.5449999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.83299999999999996,"5min":0.84499999999999997,"15min":1.113},"last":0.67600000000000005}]},{"osd":6,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":1.0529999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59299999999999997,"15min":0.60699999999999998},"min":{"1min":0.45800000000000002,"5min":0.42799999999999999,"15min":0.41599999999999998},"max":{"1min":0.878,"5min":0.878,"15min":1.169},"last":0.65000000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57099999999999995,"15min":0.58899999999999997},"min":{"1min":0.40600000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.78700000000000003,"5min":0.85499999999999998,"15min":1.2390000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56399999999999995,"15min":0.749},"min":{"1min":0.38,"5min":0.375,"15min":0.375},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":5.5800000000000001},"last":0.58199999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":748155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.309,"15min":0.35499999999999998},"min":{"1min":0.21199999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.441,"5min":0.50800000000000001,"15min":1.4199999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.32200000000000001,"15min":0.58399999999999996},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.48299999999999998,"5min":0.53800000000000003,"15min":7.2699999999999996},"last":0.40999999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32100000000000001,"15min":0.34699999999999998},"min":{"1min":0.22700000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.41199999999999998,"5min":0.497,"15min":0.65400000000000003},"last":0.312},{"interface":"front","average":{"1min":0.377,"5min":0.371,"15min":0.63900000000000001},"min":{"1min":0.29599999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.51500000000000001,"5min":0.57599999999999996,"15min":7.327},"last":0.46100000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36899999999999999,"15min":0.39100000000000001},"min":{"1min":0.28199999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.48399999999999999,"5min":0.52900000000000003,"15min":0.68600000000000005},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36699999999999999,"15min":0.39300000000000002},"min":{"1min":0.26500000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.51400000000000001,"5min":0.54000000000000004,"15min":0.72699999999999998},"last":0.44400000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.34999999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.79700000000000004,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55300000000000005,"15min":0.80800000000000005},"min":{"1min":0.38600000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72699999999999998,"5min":0.85399999999999998,"15min":7.2389999999999999},"last":0.73999999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58799999999999997,"15min":0.95299999999999996},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72499999999999998,"5min":0.83799999999999997,"15min":10.890000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59099999999999997,"15min":0.60199999999999998},"min":{"1min":0.41499999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.61399999999999999}]},{"osd":6,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56799999999999995,"15min":0.81599999999999995},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76700000000000002,"5min":0.89100000000000001,"15min":7.2999999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56999999999999995,"15min":0.57199999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.71099999999999997}]},{"osd":7,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59399999999999997,"15min":0.84699999999999998},"min":{"1min":0.44600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.78800000000000003,"5min":0.87,"15min":7.1879999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60999999999999999,"15min":0.626},"min":{"1min":0.42699999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.90500000000000003},"last":0.66000000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":748155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.36099999999999999,"15min":0.32900000000000001},"min":{"1min":0.223,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.46200000000000002,"5min":5.8700000000000001,"15min":5.8700000000000001},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.38500000000000001,"15min":0.36599999999999999},"min":{"1min":0.23400000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.502,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.28299999999999997}]},{"osd":1,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.379,"15min":0.35799999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.51800000000000002,"5min":4.0469999999999997,"15min":4.0469999999999997},"last":0.311},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.40300000000000002,"15min":0.38800000000000001},"min":{"1min":0.28399999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.52300000000000002,"5min":4.1509999999999998,"15min":4.1509999999999998},"last":0.26700000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.374},"min":{"1min":0.26800000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.56799999999999995,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38900000000000001,"15min":0.38500000000000001},"min":{"1min":0.26800000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.54000000000000004,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.36499999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58199999999999996,"15min":0.57099999999999995},"min":{"1min":0.40600000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.74199999999999999,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.42199999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77400000000000002,"5min":3.0710000000000002,"15min":3.0710000000000002},"last":0.42699999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.61599999999999999,"15min":0.58799999999999997},"min":{"1min":0.45700000000000002,"5min":0.34599999999999997,"15min":0.32700000000000001},"max":{"1min":0.69899999999999995,"5min":6.2300000000000004,"15min":6.2300000000000004},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61499999999999999,"15min":0.59699999999999998},"min":{"1min":0.495,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79700000000000004,"5min":3.0219999999999998,"15min":3.0219999999999998},"last":0.53700000000000003}]},{"osd":6,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61599999999999999,"15min":0.59199999999999997},"min":{"1min":0.48199999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.78300000000000003,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68899999999999995,"15min":0.65100000000000002},"min":{"1min":0.46700000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.82499999999999996,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.54500000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64100000000000001,"15min":0.61299999999999999},"min":{"1min":0.44900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85399999999999998,"5min":4.2629999999999999,"15min":4.2629999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.65500000000000003,"15min":0.61299999999999999},"min":{"1min":0.50800000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.77400000000000002,"5min":7.3650000000000002,"15min":7.3650000000000002},"last":0.47099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":748155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.51400000000000001,"15min":0.55300000000000005},"min":{"1min":0.318,"5min":0.28100000000000003,"15min":0.27000000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":1.083},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.35699999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.79900000000000004,"5min":0.97799999999999998,"15min":1.054},"last":0.46100000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.39900000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.501,"5min":0.50700000000000001,"15min":0.49399999999999999},"min":{"1min":0.33800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.76300000000000001,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.42099999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52700000000000002,"15min":0.53900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.32900000000000001},"max":{"1min":0.76600000000000001,"5min":0.82099999999999995,"15min":1.0249999999999999},"last":0.504},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55700000000000005,"15min":0.59099999999999997},"min":{"1min":0.39400000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.93799999999999994},"last":0.39400000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52900000000000003,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.32200000000000001,"15min":0.29199999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.98499999999999999},"last":0.374},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57699999999999996,"15min":0.58799999999999997},"min":{"1min":0.38700000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.1319999999999999},"last":0.55400000000000005}]},{"osd":5,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51600000000000001,"15min":0.55100000000000005},"min":{"1min":0.31900000000000001,"5min":0.30399999999999999,"15min":0.27300000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.998},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58199999999999996,"15min":0.60599999999999998},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.35099999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.97999999999999998},"last":0.52100000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.56799999999999995},"min":{"1min":0.41599999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.76800000000000002,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.59799999999999998},"min":{"1min":0.38700000000000001,"5min":0.34999999999999998,"15min":0.32700000000000001},"max":{"1min":0.85599999999999998,"5min":1.01,"15min":1.01},"last":0.436}]},{"osd":7,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54300000000000004,"15min":0.56999999999999995},"min":{"1min":0.40200000000000002,"5min":0.33600000000000002,"15min":0.28299999999999997},"max":{"1min":0.78900000000000003,"5min":0.85599999999999998,"15min":0.90000000000000002},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.60699999999999998,"15min":0.64500000000000002},"min":{"1min":0.36699999999999999,"5min":0.34999999999999998,"15min":0.32200000000000001},"max":{"1min":0.89000000000000001,"5min":0.98999999999999999,"15min":1.175},"last":0.73299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":748155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.503,"15min":0.50800000000000001},"min":{"1min":0.38800000000000001,"5min":0.29299999999999998,"15min":0.28599999999999998},"max":{"1min":0.67900000000000005,"5min":0.71399999999999997,"15min":0.86599999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51800000000000002,"15min":0.51400000000000001},"min":{"1min":0.42499999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.60599999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.47799999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.38400000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.63900000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52300000000000002,"15min":0.53100000000000003},"min":{"1min":0.43099999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72799999999999998,"5min":0.73899999999999999,"15min":0.81599999999999995},"last":0.50800000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54300000000000004,"15min":0.54400000000000004},"min":{"1min":0.41099999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.63900000000000001,"5min":0.72999999999999998,"15min":0.83999999999999997},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.47199999999999998,"5min":0.40999999999999998,"15min":0.39500000000000002},"max":{"1min":0.72699999999999998,"5min":0.74099999999999999,"15min":0.79900000000000004},"last":0.49399999999999999}]},{"osd":3,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.42399999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.66900000000000004,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.54900000000000004,"15min":0.56000000000000005},"min":{"1min":0.42499999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.63500000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.55100000000000005}]},{"osd":4,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.50600000000000001,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.36199999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.80700000000000005},"last":0.65100000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55300000000000005,"15min":0.53600000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.83199999999999996},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.36199999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77900000000000003,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.56999999999999995}]},{"osd":7,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.57899999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71499999999999997,"5min":0.79500000000000004,"15min":0.85599999999999998},"last":0.628},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.374,"5min":0.35199999999999998,"15min":0.314},"max":{"1min":0.79500000000000004,"5min":0.84499999999999997,"15min":0.90600000000000003},"last":0.58099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:37:51.229 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:37:51.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.378+0000 7f492daa0700 1 Processor -- start 2023-10-09T06:37:51.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.378+0000 7f492daa0700 1 -- start start 2023-10-09T06:37:51.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.379+0000 7f492daa0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4928159890 0x7f492815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:51.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.379+0000 7f492daa0700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492815a160 con 0x7f4928069560 2023-10-09T06:37:51.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.379+0000 7f492daa0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49280d13a0 con 0x7f4928159c60 2023-10-09T06:37:51.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.379+0000 7f492daa0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492815e140 con 0x7f4928159890 2023-10-09T06:37:51.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.379+0000 7f4927fff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f4928069560 0x7f49280ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33550/0 (socket says 172.21.15.163:33550) 2023-10-09T06:37:51.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.379+0000 7f4927fff700 1 -- 172.21.15.163:0/3031582977 learned_addr learned my addr 172.21.15.163:0/3031582977 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:51.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.379+0000 7f49277fe700 1 --2- 172.21.15.163:0/3031582977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4928159890 0x7f492815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:51.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.379+0000 7f49267fc700 1 -- 172.21.15.163:0/3031582977 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1134392712 0 0) 0x7f492815a160 con 0x7f4928069560 2023-10-09T06:37:51.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.379+0000 7f49267fc700 1 -- 172.21.15.163:0/3031582977 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4910003570 con 0x7f4928069560 2023-10-09T06:37:51.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.379+0000 7f49267fc700 1 -- 172.21.15.163:0/3031582977 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 448770003 0 0) 0x7f4910003570 con 0x7f4928069560 2023-10-09T06:37:51.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.379+0000 7f49267fc700 1 -- 172.21.15.163:0/3031582977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4928159890 msgr2=0x7f492815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:51.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.379+0000 7f49267fc700 1 --2- 172.21.15.163:0/3031582977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4928159890 0x7f492815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:51.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.379+0000 7f49267fc700 1 -- 172.21.15.163:0/3031582977 >> v1:172.21.15.181:6789/0 conn(0x7f4928159c60 legacy=0x7f492815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:51.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.379+0000 7f49267fc700 1 -- 172.21.15.163:0/3031582977 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492815a3d0 con 0x7f4928069560 2023-10-09T06:37:51.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.379+0000 7f49267fc700 1 -- 172.21.15.163:0/3031582977 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f49200017f0 con 0x7f4928069560 2023-10-09T06:37:51.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.379+0000 7f49277fe700 1 --2- 172.21.15.163:0/3031582977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4928159890 0x7f492815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:51.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.379+0000 7f49267fc700 1 -- 172.21.15.163:0/3031582977 >> v1:172.21.15.163:6789/0 conn(0x7f4928069560 legacy=0x7f49280ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:51.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.379+0000 7f49267fc700 1 --2- 172.21.15.163:0/3031582977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4910003a90 0x7f4910003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:51.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.379+0000 7f49267fc700 1 --2- 172.21.15.163:0/3031582977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4910004810 0x7f4910004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:51.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.379+0000 7f49267fc700 1 --2- 172.21.15.163:0/3031582977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4928159890 0x7f4910005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:51.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.379+0000 7f49267fc700 1 -- 172.21.15.163:0/3031582977 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492815e140 con 0x7f4910004810 2023-10-09T06:37:51.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.379+0000 7f49267fc700 1 -- 172.21.15.163:0/3031582977 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491000bbf0 con 0x7f4910003a90 2023-10-09T06:37:51.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.379+0000 7f49267fc700 1 -- 172.21.15.163:0/3031582977 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f491000bdb0 con 0x7f4928159890 2023-10-09T06:37:51.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.380+0000 7f4926ffd700 1 --2- 172.21.15.163:0/3031582977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4910004810 0x7f4910004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:51.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.380+0000 7f49277fe700 1 --2- 172.21.15.163:0/3031582977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4910003a90 0x7f4910003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:51.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.380+0000 7f4927fff700 1 --2- 172.21.15.163:0/3031582977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4928159890 0x7f4910005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:51.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.380+0000 7f4926ffd700 1 -- 172.21.15.163:0/3031582977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4928159890 msgr2=0x7f4910005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:51.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.380+0000 7f4926ffd700 1 --2- 172.21.15.163:0/3031582977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4928159890 0x7f4910005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:51.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.380+0000 7f4926ffd700 1 -- 172.21.15.163:0/3031582977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4910003a90 msgr2=0x7f4910003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:51.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.380+0000 7f4926ffd700 1 --2- 172.21.15.163:0/3031582977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4910003a90 0x7f4910003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:51.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.380+0000 7f4926ffd700 1 -- 172.21.15.163:0/3031582977 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4910003570 con 0x7f4910004810 2023-10-09T06:37:51.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.380+0000 7f4926ffd700 1 --2- 172.21.15.163:0/3031582977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4910004810 0x7f4910004c80 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f491c001880 tx=0x7f491c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d388cc3d2611aa1a server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:51.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.380+0000 7f49267fc700 1 -- 172.21.15.163:0/3031582977 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c012420 con 0x7f4910004810 2023-10-09T06:37:51.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.380+0000 7f49267fc700 1 -- 172.21.15.163:0/3031582977 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491c012580 con 0x7f4910004810 2023-10-09T06:37:51.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.380+0000 7f49267fc700 1 -- 172.21.15.163:0/3031582977 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c012920 con 0x7f4910004810 2023-10-09T06:37:51.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.381+0000 7f492daa0700 1 -- 172.21.15.163:0/3031582977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4910004810 msgr2=0x7f4910004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:51.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.381+0000 7f492daa0700 1 --2- 172.21.15.163:0/3031582977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4910004810 0x7f4910004c80 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f491c001880 tx=0x7f491c007d30 comp rx=0 tx=0).stop 2023-10-09T06:37:51.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.381+0000 7f4927fff700 1 -- 172.21.15.163:0/3031582977 reap_dead start 2023-10-09T06:37:51.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.381+0000 7f492daa0700 1 -- 172.21.15.163:0/3031582977 shutdown_connections 2023-10-09T06:37:51.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.381+0000 7f492daa0700 1 -- 172.21.15.163:0/3031582977 >> 172.21.15.163:0/3031582977 conn(0x7f4928068e20 msgr2=0x7f49280d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:51.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.381+0000 7f492daa0700 1 -- 172.21.15.163:0/3031582977 shutdown_connections 2023-10-09T06:37:51.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.381+0000 7f492daa0700 1 -- 172.21.15.163:0/3031582977 wait complete. 2023-10-09T06:37:51.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.382+0000 7f492daa0700 1 Processor -- start 2023-10-09T06:37:51.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.382+0000 7f492daa0700 1 -- start start 2023-10-09T06:37:51.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.383+0000 7f492daa0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4910004810 0x7f49280cf760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:51.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.383+0000 7f492daa0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49280ceb80 0x7f49280c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:51.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.383+0000 7f492daa0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49280b3900 0x7f4928151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:51.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.383+0000 7f492daa0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49280ceff0 con 0x7f49280b3900 2023-10-09T06:37:51.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.383+0000 7f492daa0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49280cfe60 con 0x7f4910004810 2023-10-09T06:37:51.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.383+0000 7f492daa0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49281537c0 con 0x7f49280ceb80 2023-10-09T06:37:51.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.383+0000 7f4927fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49280b3900 0x7f4928151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:51.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.383+0000 7f4927fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49280b3900 0x7f4928151860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41702/0 (socket says 172.21.15.163:41702) 2023-10-09T06:37:51.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.383+0000 7f4927fff700 1 -- 172.21.15.163:0/3882276246 learned_addr learned my addr 172.21.15.163:0/3882276246 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:51.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.383+0000 7f4926ffd700 1 --2- 172.21.15.163:0/3882276246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49280ceb80 0x7f49280c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:51.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.383+0000 7f49277fe700 1 --2- 172.21.15.163:0/3882276246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4910004810 0x7f49280cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:51.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.383+0000 7f4927fff700 1 -- 172.21.15.163:0/3882276246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49280ceb80 msgr2=0x7f49280c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:51.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.383+0000 7f4927fff700 1 --2- 172.21.15.163:0/3882276246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49280ceb80 0x7f49280c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:51.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.383+0000 7f4927fff700 1 -- 172.21.15.163:0/3882276246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4910004810 msgr2=0x7f49280cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:51.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.383+0000 7f4927fff700 1 --2- 172.21.15.163:0/3882276246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4910004810 0x7f49280cf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:51.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.383+0000 7f4927fff700 1 -- 172.21.15.163:0/3882276246 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4928153aa0 con 0x7f49280b3900 2023-10-09T06:37:51.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.383+0000 7f4927fff700 1 --2- 172.21.15.163:0/3882276246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49280b3900 0x7f4928151860 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f49280b67f0 tx=0x7f49200021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:51.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.383+0000 7f490f7fe700 1 -- 172.21.15.163:0/3882276246 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492000ac90 con 0x7f49280b3900 2023-10-09T06:37:51.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.383+0000 7f490f7fe700 1 -- 172.21.15.163:0/3882276246 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492000ae10 con 0x7f49280b3900 2023-10-09T06:37:51.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.383+0000 7f492daa0700 1 -- 172.21.15.163:0/3882276246 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4928153d80 con 0x7f49280b3900 2023-10-09T06:37:51.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.383+0000 7f490f7fe700 1 -- 172.21.15.163:0/3882276246 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4920013640 con 0x7f49280b3900 2023-10-09T06:37:51.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.383+0000 7f492daa0700 1 -- 172.21.15.163:0/3882276246 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49280c7de0 con 0x7f49280b3900 2023-10-09T06:37:51.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.384+0000 7f492daa0700 1 -- 172.21.15.163:0/3882276246 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49281542a0 con 0x7f49280b3900 2023-10-09T06:37:51.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.385+0000 7f490f7fe700 1 -- 172.21.15.163:0/3882276246 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f49200139f0 con 0x7f49280b3900 2023-10-09T06:37:51.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.385+0000 7f490f7fe700 1 --2- 172.21.15.163:0/3882276246 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4914067320 0x7f49140697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:51.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.385+0000 7f490f7fe700 1 -- 172.21.15.163:0/3882276246 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4920085530 con 0x7f49280b3900 2023-10-09T06:37:51.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.385+0000 7f49277fe700 1 --2- 172.21.15.163:0/3882276246 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4914067320 0x7f49140697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:51.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.386+0000 7f49277fe700 1 --2- 172.21.15.163:0/3882276246 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4914067320 0x7f49140697d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4918006fd0 tx=0x7f4918008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:51.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.389+0000 7f490f7fe700 1 -- 172.21.15.163:0/3882276246 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f49281542a0 con 0x7f49280b3900 2023-10-09T06:37:51.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.567+0000 7f492daa0700 1 -- 172.21.15.163:0/3882276246 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f49281542a0 con 0x7f4914067320 2023-10-09T06:37:51.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.575+0000 7f490f7fe700 1 -- 172.21.15.163:0/3882276246 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+279213 (secure 0 0 0) 0x7f49281542a0 con 0x7f4914067320 2023-10-09T06:37:51.578 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:51.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.578+0000 7f492daa0700 1 -- 172.21.15.163:0/3882276246 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4914067320 msgr2=0x7f49140697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:51.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.578+0000 7f492daa0700 1 --2- 172.21.15.163:0/3882276246 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4914067320 0x7f49140697d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4918006fd0 tx=0x7f4918008040 comp rx=0 tx=0).stop 2023-10-09T06:37:51.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.578+0000 7f492daa0700 1 -- 172.21.15.163:0/3882276246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49280b3900 msgr2=0x7f4928151860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:51.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.578+0000 7f492daa0700 1 --2- 172.21.15.163:0/3882276246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49280b3900 0x7f4928151860 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f49280b67f0 tx=0x7f49200021d0 comp rx=0 tx=0).stop 2023-10-09T06:37:51.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.578+0000 7f492daa0700 1 -- 172.21.15.163:0/3882276246 shutdown_connections 2023-10-09T06:37:51.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.578+0000 7f492daa0700 1 --2- 172.21.15.163:0/3882276246 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4914067320 0x7f49140697d0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:51.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.578+0000 7f492daa0700 1 --2- 172.21.15.163:0/3882276246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49280b3900 0x7f4928151860 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:51.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.578+0000 7f492daa0700 1 --2- 172.21.15.163:0/3882276246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4910004810 0x7f49280cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:51.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.578+0000 7f492daa0700 1 --2- 172.21.15.163:0/3882276246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49280ceb80 0x7f49280c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:51.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.578+0000 7f492daa0700 1 -- 172.21.15.163:0/3882276246 >> 172.21.15.163:0/3882276246 conn(0x7f4928068e20 msgr2=0x7f492815d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:51.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.578+0000 7f492daa0700 1 -- 172.21.15.163:0/3882276246 shutdown_connections 2023-10-09T06:37:51.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.579+0000 7f492daa0700 1 -- 172.21.15.163:0/3882276246 wait complete. 2023-10-09T06:37:51.582 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:37:51.591 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2023-10-09T06:37:50.402700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2425384,"kb_used_data":14312,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752549336,"statfs":{"total":773094113280,"available":770610520064,"internally_reserved":0,"allocated":14655488,"data_stored":7165976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.259477"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.150485+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:51:33.426962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149695+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:35:38.378414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149092+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:01:46.574508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.146022+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:09:27.400457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.147611+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:37:20.004643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.144979+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:55:12.397428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.147056+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:59:31.649021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.147727+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:30:10.930315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.147575+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:10:02.531578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.147553+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:33:04.749404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149103+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:18:36.804502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.145710+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:34:05.362873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149392+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:00:51.883913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.144958+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:04:18.850838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.145811+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:53:17.680797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.147810+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:34:00.040661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.144992+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:31:32.310904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.145331+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:00:51.883913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.147799+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:22:44.782785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149528+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:11:45.248711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.147628+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:28:29.141519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.148789+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:59:12.303101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149595+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:24:27.771345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.145700+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:19:19.201484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.151603+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:34:05.362873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149469+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:19:19.201484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.144969+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:59:31.649021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149695+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:24:42.401357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149646+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:03:33.845927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.145665+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:42:15.133743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149518+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:16:35.538841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.145331+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:49:26.936922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":256,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-09T06:37:43.180951+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:37:43.180951+0000","last_peered":"2023-10-09T06:37:43.180951+0000","last_clean":"2023-10-09T06:37:43.180951+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:37:43.180951+0000","last_undegraded":"2023-10-09T06:37:43.180951+0000","last_fullsized":"2023-10-09T06:37:43.180951+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574939,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303608,"kb_used_data":2224,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068232,"statfs":{"total":96636764160,"available":96325869568,"internally_reserved":0,"allocated":2277376,"data_stored":1338523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52000000000000002,"15min":0.50700000000000001},"min":{"1min":0.34799999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.37,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67600000000000005,"5min":0.753,"15min":0.753},"last":0.33400000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.503,"15min":0.51900000000000002},"min":{"1min":0.33600000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.74299999999999999,"5min":0.77400000000000002,"15min":1.2030000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.378,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.41099999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53200000000000003,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75800000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.37},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.44,"5min":0.36399999999999999,"15min":0.315},"max":{"1min":0.77900000000000003,"5min":0.80700000000000005,"15min":0.83099999999999996},"last":0.50900000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53800000000000003,"15min":0.55000000000000004},"min":{"1min":0.41199999999999998,"5min":0.35899999999999999,"15min":0.33500000000000002},"max":{"1min":0.68600000000000005,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55600000000000005,"15min":0.59199999999999997},"min":{"1min":0.45300000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.754,"5min":0.82099999999999995,"15min":1.1479999999999999},"last":0.443}]},{"osd":4,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.48199999999999998,"15min":0.51900000000000002},"min":{"1min":0.35599999999999998,"5min":0.30299999999999999,"15min":0.24399999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.93999999999999995},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56599999999999995,"15min":0.60599999999999998},"min":{"1min":0.36799999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76700000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.56799999999999995}]},{"osd":5,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52500000000000002,"15min":0.56999999999999995},"min":{"1min":0.36499999999999999,"5min":0.32500000000000001,"15min":0.307},"max":{"1min":0.81999999999999995,"5min":0.85799999999999998,"15min":0.91100000000000003},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59599999999999997,"15min":0.62},"min":{"1min":0.52900000000000003,"5min":0.41699999999999998,"15min":0.34200000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.54500000000000004}]},{"osd":6,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55700000000000005,"15min":0.59299999999999997},"min":{"1min":0.38600000000000001,"5min":0.33500000000000002,"15min":0.28699999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.88600000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.56100000000000005,"15min":0.61399999999999999},"min":{"1min":0.45100000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.84599999999999997},"last":0.60599999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574938,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":748155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.50800000000000001,"15min":0.502},"min":{"1min":0.35499999999999998,"5min":0.32000000000000001,"15min":0.30399999999999999},"max":{"1min":0.66500000000000004,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.50700000000000001,"15min":0.51000000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70399999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.56799999999999995}]},{"osd":1,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.502,"15min":0.51200000000000001},"min":{"1min":0.33600000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68300000000000005,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52900000000000003,"15min":0.53200000000000003},"min":{"1min":0.38700000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.77100000000000002,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.54100000000000004}]},{"osd":2,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.34499999999999997,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.72399999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51100000000000001,"15min":0.52900000000000003},"min":{"1min":0.39000000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.66600000000000004,"5min":0.67500000000000004,"15min":0.752},"last":0.55400000000000005}]},{"osd":3,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.38900000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66600000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.54300000000000004,"15min":0.68799999999999994},"min":{"1min":0.40400000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.752,"5min":0.752,"15min":4.7510000000000003},"last":0.45900000000000002}]},{"osd":4,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46400000000000002,"15min":0.47699999999999998},"min":{"1min":0.29899999999999999,"5min":0.28199999999999997,"15min":0.27900000000000003},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.76400000000000001},"last":0.436},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56100000000000005,"15min":0.58099999999999996},"min":{"1min":0.41799999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.77200000000000002,"5min":0.81299999999999994,"15min":0.83699999999999997},"last":0.60499999999999998}]},{"osd":5,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.52100000000000002,"15min":0.53000000000000003},"min":{"1min":0.31900000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.66100000000000003,"5min":0.79200000000000004,"15min":0.81200000000000006},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.59499999999999997},"min":{"1min":0.46899999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76000000000000001,"5min":0.78000000000000003,"15min":0.86599999999999999},"last":0.59399999999999997}]},{"osd":7,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55200000000000005,"15min":0.58199999999999996},"min":{"1min":0.41399999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80500000000000005,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58799999999999997,"15min":0.58899999999999997},"min":{"1min":0.46000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78400000000000003,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.57899999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":748155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.316,"15min":0.311},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.503,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.34899999999999998,"15min":0.35799999999999998},"min":{"1min":0.28399999999999997,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.51900000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.44}]},{"osd":2,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.34399999999999997,"15min":0.35099999999999998},"min":{"1min":0.215,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.53700000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.38300000000000001,"15min":0.38500000000000001},"min":{"1min":0.30299999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.55300000000000005,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.47399999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.36499999999999999,"15min":0.36399999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.55800000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.34599999999999997,"15min":0.53500000000000003},"min":{"1min":0.23400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.40400000000000003,"5min":0.56000000000000005,"15min":5.8529999999999998},"last":0.42299999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58299999999999996,"15min":0.57499999999999996},"min":{"1min":0.36199999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78300000000000003,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.40000000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.81100000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.66100000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57699999999999996,"15min":0.74099999999999999},"min":{"1min":0.36499999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88,"5min":0.88,"15min":5.9459999999999997},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61199999999999999,"15min":0.61099999999999999},"min":{"1min":0.38900000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82599999999999996,"5min":0.92700000000000005,"15min":0.93999999999999995},"last":0.67900000000000005}]},{"osd":6,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54900000000000004,"15min":0.71199999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79100000000000004,"5min":0.79600000000000004,"15min":5.9160000000000004},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.379,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84799999999999998,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.77400000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57199999999999995,"15min":0.58599999999999997},"min":{"1min":0.39000000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.80900000000000005,"5min":0.89500000000000002,"15min":0.97099999999999997},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57099999999999995,"15min":0.749},"min":{"1min":0.41399999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":5.9729999999999999},"last":0.60799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303608,"kb_used_data":2224,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068232,"statfs":{"total":96636764160,"available":96325869568,"internally_reserved":0,"allocated":2277376,"data_stored":1338523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.30199999999999999,"15min":0.315},"min":{"1min":0.216,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.433,"5min":0.498,"15min":0.498},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.32100000000000001,"15min":0.34399999999999997},"min":{"1min":0.222,"5min":0.214,"15min":0.214},"max":{"1min":0.497,"5min":0.53600000000000003,"15min":0.94499999999999995},"last":0.39400000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35199999999999998,"15min":0.36199999999999999},"min":{"1min":0.25900000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.50900000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.371,"15min":0.38800000000000001},"min":{"1min":0.30499999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.56499999999999995,"5min":0.69399999999999995,"15min":0.91700000000000004},"last":0.33900000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.32800000000000001,"15min":0.38400000000000001},"min":{"1min":0.23699999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":1.554},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.35999999999999999,"15min":0.39700000000000002},"min":{"1min":0.27700000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54400000000000004,"5min":0.58099999999999996,"15min":0.79000000000000004},"last":0.313}]},{"osd":4,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53600000000000003,"15min":0.55100000000000005},"min":{"1min":0.41199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":1.212},"last":0.62},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.72099999999999997},"min":{"1min":0.42199999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":5.484},"last":0.56499999999999995}]},{"osd":5,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.72499999999999998},"min":{"1min":0.41999999999999998,"5min":0.35799999999999998,"15min":0.35499999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":5.5449999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.83299999999999996,"5min":0.84499999999999997,"15min":1.113},"last":0.67600000000000005}]},{"osd":6,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":1.0529999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59299999999999997,"15min":0.60699999999999998},"min":{"1min":0.45800000000000002,"5min":0.42799999999999999,"15min":0.41599999999999998},"max":{"1min":0.878,"5min":0.878,"15min":1.169},"last":0.65000000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57099999999999995,"15min":0.58899999999999997},"min":{"1min":0.40600000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.78700000000000003,"5min":0.85499999999999998,"15min":1.2390000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56399999999999995,"15min":0.749},"min":{"1min":0.38,"5min":0.375,"15min":0.375},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":5.5800000000000001},"last":0.58199999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":748155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.309,"15min":0.35499999999999998},"min":{"1min":0.21199999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.441,"5min":0.50800000000000001,"15min":1.4199999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.32200000000000001,"15min":0.58399999999999996},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.48299999999999998,"5min":0.53800000000000003,"15min":7.2699999999999996},"last":0.40999999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32100000000000001,"15min":0.34699999999999998},"min":{"1min":0.22700000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.41199999999999998,"5min":0.497,"15min":0.65400000000000003},"last":0.312},{"interface":"front","average":{"1min":0.377,"5min":0.371,"15min":0.63900000000000001},"min":{"1min":0.29599999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.51500000000000001,"5min":0.57599999999999996,"15min":7.327},"last":0.46100000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36899999999999999,"15min":0.39100000000000001},"min":{"1min":0.28199999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.48399999999999999,"5min":0.52900000000000003,"15min":0.68600000000000005},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36699999999999999,"15min":0.39300000000000002},"min":{"1min":0.26500000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.51400000000000001,"5min":0.54000000000000004,"15min":0.72699999999999998},"last":0.44400000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.34999999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.79700000000000004,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55300000000000005,"15min":0.80800000000000005},"min":{"1min":0.38600000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72699999999999998,"5min":0.85399999999999998,"15min":7.2389999999999999},"last":0.73999999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58799999999999997,"15min":0.95299999999999996},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72499999999999998,"5min":0.83799999999999997,"15min":10.890000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59099999999999997,"15min":0.60199999999999998},"min":{"1min":0.41499999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.61399999999999999}]},{"osd":6,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56799999999999995,"15min":0.81599999999999995},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76700000000000002,"5min":0.89100000000000001,"15min":7.2999999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56999999999999995,"15min":0.57199999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.71099999999999997}]},{"osd":7,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59399999999999997,"15min":0.84699999999999998},"min":{"1min":0.44600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.78800000000000003,"5min":0.87,"15min":7.1879999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60999999999999999,"15min":0.626},"min":{"1min":0.42699999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.90500000000000003},"last":0.66000000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":748155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.36099999999999999,"15min":0.32900000000000001},"min":{"1min":0.223,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.46200000000000002,"5min":5.8700000000000001,"15min":5.8700000000000001},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.38500000000000001,"15min":0.36599999999999999},"min":{"1min":0.23400000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.502,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.28299999999999997}]},{"osd":1,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.379,"15min":0.35799999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.51800000000000002,"5min":4.0469999999999997,"15min":4.0469999999999997},"last":0.311},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.40300000000000002,"15min":0.38800000000000001},"min":{"1min":0.28399999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.52300000000000002,"5min":4.1509999999999998,"15min":4.1509999999999998},"last":0.26700000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.374},"min":{"1min":0.26800000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.56799999999999995,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38900000000000001,"15min":0.38500000000000001},"min":{"1min":0.26800000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.54000000000000004,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.36499999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58199999999999996,"15min":0.57099999999999995},"min":{"1min":0.40600000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.74199999999999999,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.42199999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77400000000000002,"5min":3.0710000000000002,"15min":3.0710000000000002},"last":0.42699999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.61599999999999999,"15min":0.58799999999999997},"min":{"1min":0.45700000000000002,"5min":0.34599999999999997,"15min":0.32700000000000001},"max":{"1min":0.69899999999999995,"5min":6.2300000000000004,"15min":6.2300000000000004},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61499999999999999,"15min":0.59699999999999998},"min":{"1min":0.495,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79700000000000004,"5min":3.0219999999999998,"15min":3.0219999999999998},"last":0.53700000000000003}]},{"osd":6,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61599999999999999,"15min":0.59199999999999997},"min":{"1min":0.48199999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.78300000000000003,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68899999999999995,"15min":0.65100000000000002},"min":{"1min":0.46700000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.82499999999999996,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.54500000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64100000000000001,"15min":0.61299999999999999},"min":{"1min":0.44900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85399999999999998,"5min":4.2629999999999999,"15min":4.2629999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.65500000000000003,"15min":0.61299999999999999},"min":{"1min":0.50800000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.77400000000000002,"5min":7.3650000000000002,"15min":7.3650000000000002},"last":0.47099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":748155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.51400000000000001,"15min":0.55300000000000005},"min":{"1min":0.318,"5min":0.28100000000000003,"15min":0.27000000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":1.083},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.35699999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.79900000000000004,"5min":0.97799999999999998,"15min":1.054},"last":0.46100000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.39900000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.501,"5min":0.50700000000000001,"15min":0.49399999999999999},"min":{"1min":0.33800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.76300000000000001,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.42099999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52700000000000002,"15min":0.53900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.32900000000000001},"max":{"1min":0.76600000000000001,"5min":0.82099999999999995,"15min":1.0249999999999999},"last":0.504},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55700000000000005,"15min":0.59099999999999997},"min":{"1min":0.39400000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.93799999999999994},"last":0.39400000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52900000000000003,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.32200000000000001,"15min":0.29199999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.98499999999999999},"last":0.374},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57699999999999996,"15min":0.58799999999999997},"min":{"1min":0.38700000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.1319999999999999},"last":0.55400000000000005}]},{"osd":5,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51600000000000001,"15min":0.55100000000000005},"min":{"1min":0.31900000000000001,"5min":0.30399999999999999,"15min":0.27300000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.998},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58199999999999996,"15min":0.60599999999999998},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.35099999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.97999999999999998},"last":0.52100000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.56799999999999995},"min":{"1min":0.41599999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.76800000000000002,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.59799999999999998},"min":{"1min":0.38700000000000001,"5min":0.34999999999999998,"15min":0.32700000000000001},"max":{"1min":0.85599999999999998,"5min":1.01,"15min":1.01},"last":0.436}]},{"osd":7,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54300000000000004,"15min":0.56999999999999995},"min":{"1min":0.40200000000000002,"5min":0.33600000000000002,"15min":0.28299999999999997},"max":{"1min":0.78900000000000003,"5min":0.85599999999999998,"15min":0.90000000000000002},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.60699999999999998,"15min":0.64500000000000002},"min":{"1min":0.36699999999999999,"5min":0.34999999999999998,"15min":0.32200000000000001},"max":{"1min":0.89000000000000001,"5min":0.98999999999999999,"15min":1.175},"last":0.73299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":748155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.503,"15min":0.50800000000000001},"min":{"1min":0.38800000000000001,"5min":0.29299999999999998,"15min":0.28599999999999998},"max":{"1min":0.67900000000000005,"5min":0.71399999999999997,"15min":0.86599999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51800000000000002,"15min":0.51400000000000001},"min":{"1min":0.42499999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.60599999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.47799999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.38400000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.63900000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52300000000000002,"15min":0.53100000000000003},"min":{"1min":0.43099999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72799999999999998,"5min":0.73899999999999999,"15min":0.81599999999999995},"last":0.50800000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54300000000000004,"15min":0.54400000000000004},"min":{"1min":0.41099999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.63900000000000001,"5min":0.72999999999999998,"15min":0.83999999999999997},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.47199999999999998,"5min":0.40999999999999998,"15min":0.39500000000000002},"max":{"1min":0.72699999999999998,"5min":0.74099999999999999,"15min":0.79900000000000004},"last":0.49399999999999999}]},{"osd":3,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.42399999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.66900000000000004,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.54900000000000004,"15min":0.56000000000000005},"min":{"1min":0.42499999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.63500000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.55100000000000005}]},{"osd":4,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.50600000000000001,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.36199999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.80700000000000005},"last":0.65100000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55300000000000005,"15min":0.53600000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.83199999999999996},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.36199999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77900000000000003,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.56999999999999995}]},{"osd":7,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.57899999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71499999999999997,"5min":0.79500000000000004,"15min":0.85599999999999998},"last":0.628},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.374,"5min":0.35199999999999998,"15min":0.314},"max":{"1min":0.79500000000000004,"5min":0.84499999999999997,"15min":0.90600000000000003},"last":0.58099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:37:51.598 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:37:51.747 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.747+0000 7f1ca6c71700 1 Processor -- start 2023-10-09T06:37:51.747 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.747+0000 7f1ca6c71700 1 -- start start 2023-10-09T06:37:51.754 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.748+0000 7f1ca6c71700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1ca00aca80 0x7f1ca006aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:51.754 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.748+0000 7f1ca6c71700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ca00c83c0 0x7f1ca00cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:51.754 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.748+0000 7f1ca6c71700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ca006bdb0 con 0x7f1ca0069d60 2023-10-09T06:37:51.754 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.748+0000 7f1ca6c71700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ca00cb370 con 0x7f1ca00aca80 2023-10-09T06:37:51.754 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.748+0000 7f1ca6c71700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca0159f60 con 0x7f1ca00c83c0 2023-10-09T06:37:51.754 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.748+0000 7f1c9ffff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f1ca0069d60 0x7f1ca00c7cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33560/0 (socket says 172.21.15.163:33560) 2023-10-09T06:37:51.754 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.748+0000 7f1c9ffff700 1 -- 172.21.15.163:0/2884732320 learned_addr learned my addr 172.21.15.163:0/2884732320 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:51.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.748+0000 7f1ca520e700 1 --2- 172.21.15.163:0/2884732320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ca00c83c0 0x7f1ca00cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:51.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.748+0000 7f1ca4a0d700 1 --2- 172.21.15.163:0/2884732320 >> v2:172.21.15.181:3300/0 conn(0x7f1ca00aca80 0x7f1ca006aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:51.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.748+0000 7f1c9f7fe700 1 -- 172.21.15.163:0/2884732320 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1819858398 0 0) 0x7f1ca006bdb0 con 0x7f1ca0069d60 2023-10-09T06:37:51.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.748+0000 7f1c9f7fe700 1 -- 172.21.15.163:0/2884732320 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c90003570 con 0x7f1ca0069d60 2023-10-09T06:37:51.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.748+0000 7f1c9f7fe700 1 -- 172.21.15.163:0/2884732320 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3991128339 0 0) 0x7f1c90003570 con 0x7f1ca0069d60 2023-10-09T06:37:51.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.748+0000 7f1c9f7fe700 1 -- 172.21.15.163:0/2884732320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ca00c83c0 msgr2=0x7f1ca00cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:51.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.748+0000 7f1c9f7fe700 1 --2- 172.21.15.163:0/2884732320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ca00c83c0 0x7f1ca00cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:51.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.748+0000 7f1c9f7fe700 1 -- 172.21.15.163:0/2884732320 >> v2:172.21.15.181:3300/0 conn(0x7f1ca00aca80 msgr2=0x7f1ca006aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:51.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.748+0000 7f1c9f7fe700 1 --2- 172.21.15.163:0/2884732320 >> v2:172.21.15.181:3300/0 conn(0x7f1ca00aca80 0x7f1ca006aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:51.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.748+0000 7f1c9f7fe700 1 -- 172.21.15.163:0/2884732320 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca006b510 con 0x7f1ca0069d60 2023-10-09T06:37:51.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.748+0000 7f1c9f7fe700 1 -- 172.21.15.163:0/2884732320 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f1c8c001470 con 0x7f1ca0069d60 2023-10-09T06:37:51.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.749+0000 7f1c9f7fe700 1 -- 172.21.15.163:0/2884732320 >> v1:172.21.15.163:6789/0 conn(0x7f1ca0069d60 legacy=0x7f1ca00c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:51.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.749+0000 7f1c9f7fe700 1 --2- 172.21.15.163:0/2884732320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ca00c83c0 0x7f1c90003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:51.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.749+0000 7f1c9f7fe700 1 --2- 172.21.15.163:0/2884732320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c90003fe0 0x7f1c90004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:51.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.749+0000 7f1c9f7fe700 1 --2- 172.21.15.163:0/2884732320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c90004d50 0x7f1c900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:51.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.749+0000 7f1c9f7fe700 1 -- 172.21.15.163:0/2884732320 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca00cb370 con 0x7f1c90003fe0 2023-10-09T06:37:51.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.749+0000 7f1c9f7fe700 1 -- 172.21.15.163:0/2884732320 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca0159f60 con 0x7f1c90004d50 2023-10-09T06:37:51.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.749+0000 7f1c9f7fe700 1 -- 172.21.15.163:0/2884732320 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9000ba90 con 0x7f1ca00c83c0 2023-10-09T06:37:51.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.749+0000 7f1c9ffff700 1 --2- 172.21.15.163:0/2884732320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c90003fe0 0x7f1c90004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:51.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.749+0000 7f1ca4a0d700 1 --2- 172.21.15.163:0/2884732320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ca00c83c0 0x7f1c90003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:51.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.749+0000 7f1ca520e700 1 --2- 172.21.15.163:0/2884732320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c90004d50 0x7f1c900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:51.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.749+0000 7f1c9ffff700 1 -- 172.21.15.163:0/2884732320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ca00c83c0 msgr2=0x7f1c90003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:51.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.749+0000 7f1c9ffff700 1 --2- 172.21.15.163:0/2884732320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ca00c83c0 0x7f1c90003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:51.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.749+0000 7f1c9ffff700 1 -- 172.21.15.163:0/2884732320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c90004d50 msgr2=0x7f1c900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:51.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.749+0000 7f1c9ffff700 1 --2- 172.21.15.163:0/2884732320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c90004d50 0x7f1c900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:51.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.749+0000 7f1c9ffff700 1 -- 172.21.15.163:0/2884732320 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c90003570 con 0x7f1c90003fe0 2023-10-09T06:37:51.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.749+0000 7f1ca520e700 1 --2- 172.21.15.163:0/2884732320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c90004d50 0x7f1c900051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:51.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.749+0000 7f1c9ffff700 1 --2- 172.21.15.163:0/2884732320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c90003fe0 0x7f1c90004810 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c00c390 tx=0x7f1c8c001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8300c967454f10e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:51.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.749+0000 7f1c9f7fe700 1 -- 172.21.15.163:0/2884732320 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c00cc00 con 0x7f1c90003fe0 2023-10-09T06:37:51.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.749+0000 7f1c9f7fe700 1 -- 172.21.15.163:0/2884732320 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c8c00cda0 con 0x7f1c90003fe0 2023-10-09T06:37:51.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.749+0000 7f1c9f7fe700 1 -- 172.21.15.163:0/2884732320 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c005b70 con 0x7f1c90003fe0 2023-10-09T06:37:51.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.750+0000 7f1ca6c71700 1 -- 172.21.15.163:0/2884732320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c90003fe0 msgr2=0x7f1c90004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:51.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.750+0000 7f1ca6c71700 1 --2- 172.21.15.163:0/2884732320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c90003fe0 0x7f1c90004810 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c00c390 tx=0x7f1c8c001e90 comp rx=0 tx=0).stop 2023-10-09T06:37:51.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.750+0000 7f1ca520e700 1 -- 172.21.15.163:0/2884732320 reap_dead start 2023-10-09T06:37:51.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.750+0000 7f1ca6c71700 1 -- 172.21.15.163:0/2884732320 shutdown_connections 2023-10-09T06:37:51.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.750+0000 7f1ca6c71700 1 -- 172.21.15.163:0/2884732320 >> 172.21.15.163:0/2884732320 conn(0x7f1ca00cd670 msgr2=0x7f1ca00cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:51.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.750+0000 7f1ca6c71700 1 -- 172.21.15.163:0/2884732320 shutdown_connections 2023-10-09T06:37:51.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.750+0000 7f1ca6c71700 1 -- 172.21.15.163:0/2884732320 wait complete. 2023-10-09T06:37:51.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.751+0000 7f1ca6c71700 1 Processor -- start 2023-10-09T06:37:51.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.751+0000 7f1ca6c71700 1 -- start start 2023-10-09T06:37:51.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.752+0000 7f1ca6c71700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c90003fe0 0x7f1ca00af990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:51.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.752+0000 7f1ca6c71700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ca00c7a90 0x7f1ca0156670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:51.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.752+0000 7f1ca6c71700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ca01548d0 0x7f1ca0154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:51.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.752+0000 7f1ca6c71700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca00ae590 con 0x7f1ca01548d0 2023-10-09T06:37:51.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.752+0000 7f1ca6c71700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca016e2b0 con 0x7f1ca00c7a90 2023-10-09T06:37:51.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.752+0000 7f1ca6c71700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca0155280 con 0x7f1c90003fe0 2023-10-09T06:37:51.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.752+0000 7f1ca520e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ca01548d0 0x7f1ca0154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:51.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.752+0000 7f1ca520e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ca01548d0 0x7f1ca0154d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41710/0 (socket says 172.21.15.163:41710) 2023-10-09T06:37:51.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.752+0000 7f1ca520e700 1 -- 172.21.15.163:0/1173840053 learned_addr learned my addr 172.21.15.163:0/1173840053 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:51.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.752+0000 7f1ca4a0d700 1 --2- 172.21.15.163:0/1173840053 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c90003fe0 0x7f1ca00af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:51.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.752+0000 7f1c9ffff700 1 --2- 172.21.15.163:0/1173840053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ca00c7a90 0x7f1ca0156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:51.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.752+0000 7f1ca520e700 1 -- 172.21.15.163:0/1173840053 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c90003fe0 msgr2=0x7f1ca00af990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:51.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.752+0000 7f1ca520e700 1 --2- 172.21.15.163:0/1173840053 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c90003fe0 0x7f1ca00af990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:51.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.752+0000 7f1ca520e700 1 -- 172.21.15.163:0/1173840053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ca00c7a90 msgr2=0x7f1ca0156670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:51.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.752+0000 7f1ca520e700 1 --2- 172.21.15.163:0/1173840053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ca00c7a90 0x7f1ca0156670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:51.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.752+0000 7f1ca520e700 1 -- 172.21.15.163:0/1173840053 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca016d6a0 con 0x7f1ca01548d0 2023-10-09T06:37:51.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.752+0000 7f1ca520e700 1 --2- 172.21.15.163:0/1173840053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ca01548d0 0x7f1ca0154d40 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f1c98009f80 tx=0x7f1c9800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:51.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.753+0000 7f1c9cff9700 1 -- 172.21.15.163:0/1173840053 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9801b070 con 0x7f1ca01548d0 2023-10-09T06:37:51.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.753+0000 7f1c9cff9700 1 -- 172.21.15.163:0/1173840053 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9800f040 con 0x7f1ca01548d0 2023-10-09T06:37:51.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.753+0000 7f1ca6c71700 1 -- 172.21.15.163:0/1173840053 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ca016d980 con 0x7f1ca01548d0 2023-10-09T06:37:51.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.753+0000 7f1c9cff9700 1 -- 172.21.15.163:0/1173840053 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9800e050 con 0x7f1ca01548d0 2023-10-09T06:37:51.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.753+0000 7f1ca6c71700 1 -- 172.21.15.163:0/1173840053 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca016df40 con 0x7f1ca01548d0 2023-10-09T06:37:51.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.754+0000 7f1ca6c71700 1 -- 172.21.15.163:0/1173840053 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ca015e710 con 0x7f1ca01548d0 2023-10-09T06:37:51.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.754+0000 7f1c9cff9700 1 -- 172.21.15.163:0/1173840053 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1c98003020 con 0x7f1ca01548d0 2023-10-09T06:37:51.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.754+0000 7f1c9cff9700 1 --2- 172.21.15.163:0/1173840053 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1c84067320 0x7f1c840697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:51.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.754+0000 7f1c9cff9700 1 -- 172.21.15.163:0/1173840053 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1c980851e0 con 0x7f1ca01548d0 2023-10-09T06:37:51.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.754+0000 7f1ca4a0d700 1 --2- 172.21.15.163:0/1173840053 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1c84067320 0x7f1c840697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:51.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.755+0000 7f1ca4a0d700 1 --2- 172.21.15.163:0/1173840053 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1c84067320 0x7f1c840697d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1c94006aa0 tx=0x7f1c94008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:51.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.758+0000 7f1c9cff9700 1 -- 172.21.15.163:0/1173840053 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ca015e710 con 0x7f1ca01548d0 2023-10-09T06:37:51.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.935+0000 7f1ca6c71700 1 -- 172.21.15.163:0/1173840053 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1ca015e710 con 0x7f1c84067320 2023-10-09T06:37:51.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.944+0000 7f1c9cff9700 1 -- 172.21.15.163:0/1173840053 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+279213 (secure 0 0 0) 0x7f1ca015e710 con 0x7f1c84067320 2023-10-09T06:37:51.947 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:51.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.946+0000 7f1ca6c71700 1 -- 172.21.15.163:0/1173840053 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1c84067320 msgr2=0x7f1c840697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:51.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.946+0000 7f1ca6c71700 1 --2- 172.21.15.163:0/1173840053 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1c84067320 0x7f1c840697d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1c94006aa0 tx=0x7f1c94008040 comp rx=0 tx=0).stop 2023-10-09T06:37:51.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.946+0000 7f1ca6c71700 1 -- 172.21.15.163:0/1173840053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ca01548d0 msgr2=0x7f1ca0154d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:51.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.946+0000 7f1ca6c71700 1 --2- 172.21.15.163:0/1173840053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ca01548d0 0x7f1ca0154d40 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f1c98009f80 tx=0x7f1c9800b040 comp rx=0 tx=0).stop 2023-10-09T06:37:51.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.946+0000 7f1ca6c71700 1 -- 172.21.15.163:0/1173840053 shutdown_connections 2023-10-09T06:37:51.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.946+0000 7f1ca6c71700 1 --2- 172.21.15.163:0/1173840053 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1c84067320 0x7f1c840697d0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:51.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.946+0000 7f1ca6c71700 1 --2- 172.21.15.163:0/1173840053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ca01548d0 0x7f1ca0154d40 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:51.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.946+0000 7f1ca6c71700 1 --2- 172.21.15.163:0/1173840053 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c90003fe0 0x7f1ca00af990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:51.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.946+0000 7f1ca6c71700 1 --2- 172.21.15.163:0/1173840053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ca00c7a90 0x7f1ca0156670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:51.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.947+0000 7f1ca6c71700 1 -- 172.21.15.163:0/1173840053 >> 172.21.15.163:0/1173840053 conn(0x7f1ca00cd670 msgr2=0x7f1ca00c2410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:51.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.947+0000 7f1ca6c71700 1 -- 172.21.15.163:0/1173840053 shutdown_connections 2023-10-09T06:37:51.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:51.947+0000 7f1ca6c71700 1 -- 172.21.15.163:0/1173840053 wait complete. 2023-10-09T06:37:51.951 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:37:51.959 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2023-10-09T06:37:50.402700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2425384,"kb_used_data":14312,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752549336,"statfs":{"total":773094113280,"available":770610520064,"internally_reserved":0,"allocated":14655488,"data_stored":7165976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.259477"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.150485+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:51:33.426962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149695+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:35:38.378414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149092+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:01:46.574508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.146022+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:09:27.400457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.147611+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:37:20.004643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.144979+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:55:12.397428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.147056+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:59:31.649021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.147727+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:30:10.930315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.147575+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:10:02.531578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.147553+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:33:04.749404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149103+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:18:36.804502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.145710+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:34:05.362873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149392+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:00:51.883913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.144958+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:04:18.850838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.145811+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:53:17.680797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.147810+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:34:00.040661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.144992+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:31:32.310904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.145331+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:00:51.883913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.147799+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:22:44.782785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149528+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:11:45.248711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.147628+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:28:29.141519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.148789+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:59:12.303101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149595+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:24:27.771345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.145700+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:19:19.201484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.151603+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:34:05.362873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149469+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:19:19.201484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.144969+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:59:31.649021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149695+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:24:42.401357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149646+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:03:33.845927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.145665+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:42:15.133743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.149518+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:16:35.538841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-09T06:37:46.140387+0000","last_change":"2023-10-09T06:37:46.140387+0000","last_active":"2023-10-09T06:37:46.140387+0000","last_peered":"2023-10-09T06:37:46.140387+0000","last_clean":"2023-10-09T06:37:46.140387+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:46.140387+0000","last_undegraded":"2023-10-09T06:37:46.140387+0000","last_fullsized":"2023-10-09T06:37:46.140387+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:37:45.133411+0000","last_change":"2023-10-09T06:37:45.145331+0000","last_active":"2023-10-09T06:37:45.133411+0000","last_peered":"2023-10-09T06:37:45.133411+0000","last_clean":"2023-10-09T06:37:45.133411+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:37:45.133411+0000","last_undegraded":"2023-10-09T06:37:45.133411+0000","last_fullsized":"2023-10-09T06:37:45.133411+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:49:26.936922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":256,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-09T06:37:43.180951+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:37:43.180951+0000","last_peered":"2023-10-09T06:37:43.180951+0000","last_clean":"2023-10-09T06:37:43.180951+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:37:43.180951+0000","last_undegraded":"2023-10-09T06:37:43.180951+0000","last_fullsized":"2023-10-09T06:37:43.180951+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574939,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303608,"kb_used_data":2224,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068232,"statfs":{"total":96636764160,"available":96325869568,"internally_reserved":0,"allocated":2277376,"data_stored":1338523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52000000000000002,"15min":0.50700000000000001},"min":{"1min":0.34799999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.37,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67600000000000005,"5min":0.753,"15min":0.753},"last":0.33400000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.503,"15min":0.51900000000000002},"min":{"1min":0.33600000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.74299999999999999,"5min":0.77400000000000002,"15min":1.2030000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.378,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.41099999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53200000000000003,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75800000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.37},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.44,"5min":0.36399999999999999,"15min":0.315},"max":{"1min":0.77900000000000003,"5min":0.80700000000000005,"15min":0.83099999999999996},"last":0.50900000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53800000000000003,"15min":0.55000000000000004},"min":{"1min":0.41199999999999998,"5min":0.35899999999999999,"15min":0.33500000000000002},"max":{"1min":0.68600000000000005,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55600000000000005,"15min":0.59199999999999997},"min":{"1min":0.45300000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.754,"5min":0.82099999999999995,"15min":1.1479999999999999},"last":0.443}]},{"osd":4,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.48199999999999998,"15min":0.51900000000000002},"min":{"1min":0.35599999999999998,"5min":0.30299999999999999,"15min":0.24399999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.93999999999999995},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56599999999999995,"15min":0.60599999999999998},"min":{"1min":0.36799999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76700000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.56799999999999995}]},{"osd":5,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52500000000000002,"15min":0.56999999999999995},"min":{"1min":0.36499999999999999,"5min":0.32500000000000001,"15min":0.307},"max":{"1min":0.81999999999999995,"5min":0.85799999999999998,"15min":0.91100000000000003},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59599999999999997,"15min":0.62},"min":{"1min":0.52900000000000003,"5min":0.41699999999999998,"15min":0.34200000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.54500000000000004}]},{"osd":6,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55700000000000005,"15min":0.59299999999999997},"min":{"1min":0.38600000000000001,"5min":0.33500000000000002,"15min":0.28699999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.88600000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.56100000000000005,"15min":0.61399999999999999},"min":{"1min":0.45100000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.84599999999999997},"last":0.60599999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574938,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":748155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.50800000000000001,"15min":0.502},"min":{"1min":0.35499999999999998,"5min":0.32000000000000001,"15min":0.30399999999999999},"max":{"1min":0.66500000000000004,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.50700000000000001,"15min":0.51000000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70399999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.56799999999999995}]},{"osd":1,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.502,"15min":0.51200000000000001},"min":{"1min":0.33600000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68300000000000005,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52900000000000003,"15min":0.53200000000000003},"min":{"1min":0.38700000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.77100000000000002,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.54100000000000004}]},{"osd":2,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.34499999999999997,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.72399999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51100000000000001,"15min":0.52900000000000003},"min":{"1min":0.39000000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.66600000000000004,"5min":0.67500000000000004,"15min":0.752},"last":0.55400000000000005}]},{"osd":3,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.38900000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66600000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.54300000000000004,"15min":0.68799999999999994},"min":{"1min":0.40400000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.752,"5min":0.752,"15min":4.7510000000000003},"last":0.45900000000000002}]},{"osd":4,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46400000000000002,"15min":0.47699999999999998},"min":{"1min":0.29899999999999999,"5min":0.28199999999999997,"15min":0.27900000000000003},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.76400000000000001},"last":0.436},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56100000000000005,"15min":0.58099999999999996},"min":{"1min":0.41799999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.77200000000000002,"5min":0.81299999999999994,"15min":0.83699999999999997},"last":0.60499999999999998}]},{"osd":5,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.52100000000000002,"15min":0.53000000000000003},"min":{"1min":0.31900000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.66100000000000003,"5min":0.79200000000000004,"15min":0.81200000000000006},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.59499999999999997},"min":{"1min":0.46899999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76000000000000001,"5min":0.78000000000000003,"15min":0.86599999999999999},"last":0.59399999999999997}]},{"osd":7,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55200000000000005,"15min":0.58199999999999996},"min":{"1min":0.41399999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80500000000000005,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58799999999999997,"15min":0.58899999999999997},"min":{"1min":0.46000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78400000000000003,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.57899999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":748155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.316,"15min":0.311},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.503,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.34899999999999998,"15min":0.35799999999999998},"min":{"1min":0.28399999999999997,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.51900000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.44}]},{"osd":2,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.34399999999999997,"15min":0.35099999999999998},"min":{"1min":0.215,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.53700000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.38300000000000001,"15min":0.38500000000000001},"min":{"1min":0.30299999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.55300000000000005,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.47399999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.36499999999999999,"15min":0.36399999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.55800000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.34599999999999997,"15min":0.53500000000000003},"min":{"1min":0.23400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.40400000000000003,"5min":0.56000000000000005,"15min":5.8529999999999998},"last":0.42299999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58299999999999996,"15min":0.57499999999999996},"min":{"1min":0.36199999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78300000000000003,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.40000000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.81100000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.66100000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57699999999999996,"15min":0.74099999999999999},"min":{"1min":0.36499999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88,"5min":0.88,"15min":5.9459999999999997},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61199999999999999,"15min":0.61099999999999999},"min":{"1min":0.38900000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82599999999999996,"5min":0.92700000000000005,"15min":0.93999999999999995},"last":0.67900000000000005}]},{"osd":6,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54900000000000004,"15min":0.71199999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79100000000000004,"5min":0.79600000000000004,"15min":5.9160000000000004},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.379,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84799999999999998,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.77400000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57199999999999995,"15min":0.58599999999999997},"min":{"1min":0.39000000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.80900000000000005,"5min":0.89500000000000002,"15min":0.97099999999999997},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57099999999999995,"15min":0.749},"min":{"1min":0.41399999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":5.9729999999999999},"last":0.60799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303608,"kb_used_data":2224,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068232,"statfs":{"total":96636764160,"available":96325869568,"internally_reserved":0,"allocated":2277376,"data_stored":1338523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.30199999999999999,"15min":0.315},"min":{"1min":0.216,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.433,"5min":0.498,"15min":0.498},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.32100000000000001,"15min":0.34399999999999997},"min":{"1min":0.222,"5min":0.214,"15min":0.214},"max":{"1min":0.497,"5min":0.53600000000000003,"15min":0.94499999999999995},"last":0.39400000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35199999999999998,"15min":0.36199999999999999},"min":{"1min":0.25900000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.50900000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.371,"15min":0.38800000000000001},"min":{"1min":0.30499999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.56499999999999995,"5min":0.69399999999999995,"15min":0.91700000000000004},"last":0.33900000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.32800000000000001,"15min":0.38400000000000001},"min":{"1min":0.23699999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":1.554},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.35999999999999999,"15min":0.39700000000000002},"min":{"1min":0.27700000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54400000000000004,"5min":0.58099999999999996,"15min":0.79000000000000004},"last":0.313}]},{"osd":4,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53600000000000003,"15min":0.55100000000000005},"min":{"1min":0.41199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":1.212},"last":0.62},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.72099999999999997},"min":{"1min":0.42199999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":5.484},"last":0.56499999999999995}]},{"osd":5,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.72499999999999998},"min":{"1min":0.41999999999999998,"5min":0.35799999999999998,"15min":0.35499999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":5.5449999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.83299999999999996,"5min":0.84499999999999997,"15min":1.113},"last":0.67600000000000005}]},{"osd":6,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":1.0529999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59299999999999997,"15min":0.60699999999999998},"min":{"1min":0.45800000000000002,"5min":0.42799999999999999,"15min":0.41599999999999998},"max":{"1min":0.878,"5min":0.878,"15min":1.169},"last":0.65000000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57099999999999995,"15min":0.58899999999999997},"min":{"1min":0.40600000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.78700000000000003,"5min":0.85499999999999998,"15min":1.2390000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56399999999999995,"15min":0.749},"min":{"1min":0.38,"5min":0.375,"15min":0.375},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":5.5800000000000001},"last":0.58199999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":748155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.309,"15min":0.35499999999999998},"min":{"1min":0.21199999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.441,"5min":0.50800000000000001,"15min":1.4199999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.32200000000000001,"15min":0.58399999999999996},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.48299999999999998,"5min":0.53800000000000003,"15min":7.2699999999999996},"last":0.40999999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32100000000000001,"15min":0.34699999999999998},"min":{"1min":0.22700000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.41199999999999998,"5min":0.497,"15min":0.65400000000000003},"last":0.312},{"interface":"front","average":{"1min":0.377,"5min":0.371,"15min":0.63900000000000001},"min":{"1min":0.29599999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.51500000000000001,"5min":0.57599999999999996,"15min":7.327},"last":0.46100000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36899999999999999,"15min":0.39100000000000001},"min":{"1min":0.28199999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.48399999999999999,"5min":0.52900000000000003,"15min":0.68600000000000005},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36699999999999999,"15min":0.39300000000000002},"min":{"1min":0.26500000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.51400000000000001,"5min":0.54000000000000004,"15min":0.72699999999999998},"last":0.44400000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.34999999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.79700000000000004,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55300000000000005,"15min":0.80800000000000005},"min":{"1min":0.38600000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72699999999999998,"5min":0.85399999999999998,"15min":7.2389999999999999},"last":0.73999999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58799999999999997,"15min":0.95299999999999996},"min":{"1min":0.39900000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72499999999999998,"5min":0.83799999999999997,"15min":10.890000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59099999999999997,"15min":0.60199999999999998},"min":{"1min":0.41499999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.61399999999999999}]},{"osd":6,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56799999999999995,"15min":0.81599999999999995},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76700000000000002,"5min":0.89100000000000001,"15min":7.2999999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56999999999999995,"15min":0.57199999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.71099999999999997}]},{"osd":7,"last update":"Mon Oct 9 06:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59399999999999997,"15min":0.84699999999999998},"min":{"1min":0.44600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.78800000000000003,"5min":0.87,"15min":7.1879999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60999999999999999,"15min":0.626},"min":{"1min":0.42699999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.90500000000000003},"last":0.66000000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":748155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.36099999999999999,"15min":0.32900000000000001},"min":{"1min":0.223,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.46200000000000002,"5min":5.8700000000000001,"15min":5.8700000000000001},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.38500000000000001,"15min":0.36599999999999999},"min":{"1min":0.23400000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.502,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.28299999999999997}]},{"osd":1,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.379,"15min":0.35799999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.51800000000000002,"5min":4.0469999999999997,"15min":4.0469999999999997},"last":0.311},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.40300000000000002,"15min":0.38800000000000001},"min":{"1min":0.28399999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.52300000000000002,"5min":4.1509999999999998,"15min":4.1509999999999998},"last":0.26700000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.374},"min":{"1min":0.26800000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.56799999999999995,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38900000000000001,"15min":0.38500000000000001},"min":{"1min":0.26800000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.54000000000000004,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.36499999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58199999999999996,"15min":0.57099999999999995},"min":{"1min":0.40600000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.74199999999999999,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.42199999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77400000000000002,"5min":3.0710000000000002,"15min":3.0710000000000002},"last":0.42699999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.61599999999999999,"15min":0.58799999999999997},"min":{"1min":0.45700000000000002,"5min":0.34599999999999997,"15min":0.32700000000000001},"max":{"1min":0.69899999999999995,"5min":6.2300000000000004,"15min":6.2300000000000004},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61499999999999999,"15min":0.59699999999999998},"min":{"1min":0.495,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79700000000000004,"5min":3.0219999999999998,"15min":3.0219999999999998},"last":0.53700000000000003}]},{"osd":6,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61599999999999999,"15min":0.59199999999999997},"min":{"1min":0.48199999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.78300000000000003,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68899999999999995,"15min":0.65100000000000002},"min":{"1min":0.46700000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.82499999999999996,"5min":6.5970000000000004,"15min":6.5970000000000004},"last":0.54500000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:36:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64100000000000001,"15min":0.61299999999999999},"min":{"1min":0.44900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85399999999999998,"5min":4.2629999999999999,"15min":4.2629999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.65500000000000003,"15min":0.61299999999999999},"min":{"1min":0.50800000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.77400000000000002,"5min":7.3650000000000002,"15min":7.3650000000000002},"last":0.47099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":748155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.51400000000000001,"15min":0.55300000000000005},"min":{"1min":0.318,"5min":0.28100000000000003,"15min":0.27000000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":1.083},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.35699999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.79900000000000004,"5min":0.97799999999999998,"15min":1.054},"last":0.46100000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.39900000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.501,"5min":0.50700000000000001,"15min":0.49399999999999999},"min":{"1min":0.33800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.76300000000000001,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.42099999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52700000000000002,"15min":0.53900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.32900000000000001},"max":{"1min":0.76600000000000001,"5min":0.82099999999999995,"15min":1.0249999999999999},"last":0.504},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55700000000000005,"15min":0.59099999999999997},"min":{"1min":0.39400000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.93799999999999994},"last":0.39400000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52900000000000003,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.32200000000000001,"15min":0.29199999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.98499999999999999},"last":0.374},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57699999999999996,"15min":0.58799999999999997},"min":{"1min":0.38700000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.1319999999999999},"last":0.55400000000000005}]},{"osd":5,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51600000000000001,"15min":0.55100000000000005},"min":{"1min":0.31900000000000001,"5min":0.30399999999999999,"15min":0.27300000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.998},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58199999999999996,"15min":0.60599999999999998},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.35099999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.97999999999999998},"last":0.52100000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.56799999999999995},"min":{"1min":0.41599999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.76800000000000002,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.59799999999999998},"min":{"1min":0.38700000000000001,"5min":0.34999999999999998,"15min":0.32700000000000001},"max":{"1min":0.85599999999999998,"5min":1.01,"15min":1.01},"last":0.436}]},{"osd":7,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54300000000000004,"15min":0.56999999999999995},"min":{"1min":0.40200000000000002,"5min":0.33600000000000002,"15min":0.28299999999999997},"max":{"1min":0.78900000000000003,"5min":0.85599999999999998,"15min":0.90000000000000002},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.60699999999999998,"15min":0.64500000000000002},"min":{"1min":0.36699999999999999,"5min":0.34999999999999998,"15min":0.32200000000000001},"max":{"1min":0.89000000000000001,"5min":0.98999999999999999,"15min":1.175},"last":0.73299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":748155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.503,"15min":0.50800000000000001},"min":{"1min":0.38800000000000001,"5min":0.29299999999999998,"15min":0.28599999999999998},"max":{"1min":0.67900000000000005,"5min":0.71399999999999997,"15min":0.86599999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51800000000000002,"15min":0.51400000000000001},"min":{"1min":0.42499999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.60599999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.47799999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.38400000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.63900000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52300000000000002,"15min":0.53100000000000003},"min":{"1min":0.43099999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72799999999999998,"5min":0.73899999999999999,"15min":0.81599999999999995},"last":0.50800000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54300000000000004,"15min":0.54400000000000004},"min":{"1min":0.41099999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.63900000000000001,"5min":0.72999999999999998,"15min":0.83999999999999997},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.47199999999999998,"5min":0.40999999999999998,"15min":0.39500000000000002},"max":{"1min":0.72699999999999998,"5min":0.74099999999999999,"15min":0.79900000000000004},"last":0.49399999999999999}]},{"osd":3,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.42399999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.66900000000000004,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.54900000000000004,"15min":0.56000000000000005},"min":{"1min":0.42499999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.63500000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.55100000000000005}]},{"osd":4,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.50600000000000001,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.36199999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.80700000000000005},"last":0.65100000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55300000000000005,"15min":0.53600000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.83199999999999996},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.36199999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77900000000000003,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.56999999999999995}]},{"osd":7,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.57899999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71499999999999997,"5min":0.79500000000000004,"15min":0.85599999999999998},"last":0.628},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.374,"5min":0.35199999999999998,"15min":0.314},"max":{"1min":0.79500000000000004,"5min":0.84499999999999997,"15min":0.90600000000000003},"last":0.58099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:37:51.965 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.3f is not active+clean 2023-10-09T06:37:51.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.966 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.3e is not active+clean 2023-10-09T06:37:51.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.966 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.3d is not active+clean 2023-10-09T06:37:51.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.966 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.3c is not active+clean 2023-10-09T06:37:51.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.966 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.3b is not active+clean 2023-10-09T06:37:51.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.3a is not active+clean 2023-10-09T06:37:51.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.39 is not active+clean 2023-10-09T06:37:51.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.38 is not active+clean 2023-10-09T06:37:51.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.37 is not active+clean 2023-10-09T06:37:51.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.36 is not active+clean 2023-10-09T06:37:51.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.35 is not active+clean 2023-10-09T06:37:51.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.34 is not active+clean 2023-10-09T06:37:51.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.33 is not active+clean 2023-10-09T06:37:51.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.32 is not active+clean 2023-10-09T06:37:51.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.31 is not active+clean 2023-10-09T06:37:51.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.30 is not active+clean 2023-10-09T06:37:51.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.2f is not active+clean 2023-10-09T06:37:51.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.2e is not active+clean 2023-10-09T06:37:51.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.2d is not active+clean 2023-10-09T06:37:51.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.2c is not active+clean 2023-10-09T06:37:51.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.2b is not active+clean 2023-10-09T06:37:51.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.2a is not active+clean 2023-10-09T06:37:51.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.29 is not active+clean 2023-10-09T06:37:51.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.28 is not active+clean 2023-10-09T06:37:51.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.27 is not active+clean 2023-10-09T06:37:51.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.26 is not active+clean 2023-10-09T06:37:51.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.25 is not active+clean 2023-10-09T06:37:51.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.24 is not active+clean 2023-10-09T06:37:51.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.23 is not active+clean 2023-10-09T06:37:51.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.22 is not active+clean 2023-10-09T06:37:51.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.21 is not active+clean 2023-10-09T06:37:51.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.20 is not active+clean 2023-10-09T06:37:51.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1e is not active+clean 2023-10-09T06:37:51.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1f is not active+clean 2023-10-09T06:37:51.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.150485+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T15:51:33.426962+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-09T06:37:51.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1f is not active+clean 2023-10-09T06:37:51.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1e is not active+clean 2023-10-09T06:37:51.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.149695+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T13:35:38.378414+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-09T06:37:51.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1c is not active+clean 2023-10-09T06:37:51.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1d is not active+clean 2023-10-09T06:37:51.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.149092+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T18:01:46.574508+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-09T06:37:51.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1d is not active+clean 2023-10-09T06:37:51.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1c is not active+clean 2023-10-09T06:37:51.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.146022+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T16:09:27.400457+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-09T06:37:51.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1a is not active+clean 2023-10-09T06:37:51.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1b is not active+clean 2023-10-09T06:37:51.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.147611+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T16:37:20.004643+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-09T06:37:51.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1b is not active+clean 2023-10-09T06:37:51.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1a is not active+clean 2023-10-09T06:37:51.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.144979+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T10:55:12.397428+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-09T06:37:51.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.19 is not active+clean 2023-10-09T06:37:51.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.18 is not active+clean 2023-10-09T06:37:51.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.147056+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T09:59:31.649021+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-09T06:37:51.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.16 is not active+clean 2023-10-09T06:37:51.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.17 is not active+clean 2023-10-09T06:37:51.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.147727+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T10:30:10.930315+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-09T06:37:51.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.17 is not active+clean 2023-10-09T06:37:51.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.16 is not active+clean 2023-10-09T06:37:51.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.147575+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T10:10:02.531578+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-09T06:37:51.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.14 is not active+clean 2023-10-09T06:37:51.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.15 is not active+clean 2023-10-09T06:37:51.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.147553+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T07:33:04.749404+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-09T06:37:51.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.15 is not active+clean 2023-10-09T06:37:51.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.980 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.14 is not active+clean 2023-10-09T06:37:51.980 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.149103+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T14:18:36.804502+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-09T06:37:51.980 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.12 is not active+clean 2023-10-09T06:37:51.980 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.980 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.13 is not active+clean 2023-10-09T06:37:51.981 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.145710+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T14:34:05.362873+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-09T06:37:51.981 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.13 is not active+clean 2023-10-09T06:37:51.981 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.981 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.12 is not active+clean 2023-10-09T06:37:51.981 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.149392+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T15:00:51.883913+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-09T06:37:51.981 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.10 is not active+clean 2023-10-09T06:37:51.981 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.981 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.11 is not active+clean 2023-10-09T06:37:51.981 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.144958+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T07:04:18.850838+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-09T06:37:51.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.11 is not active+clean 2023-10-09T06:37:51.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.10 is not active+clean 2023-10-09T06:37:51.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.145811+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T06:53:17.680797+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-09T06:37:51.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.e is not active+clean 2023-10-09T06:37:51.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.f is not active+clean 2023-10-09T06:37:51.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.147810+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T15:34:00.040661+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-09T06:37:51.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.f is not active+clean 2023-10-09T06:37:51.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.e is not active+clean 2023-10-09T06:37:51.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.144992+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T10:31:32.310904+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-09T06:37:51.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.c is not active+clean 2023-10-09T06:37:51.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.d is not active+clean 2023-10-09T06:37:51.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.145331+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T15:00:51.883913+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-09T06:37:51.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.d is not active+clean 2023-10-09T06:37:51.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.c is not active+clean 2023-10-09T06:37:51.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.147799+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T10:22:44.782785+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-09T06:37:51.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.a is not active+clean 2023-10-09T06:37:51.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.b is not active+clean 2023-10-09T06:37:51.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.149528+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T18:11:45.248711+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-09T06:37:51.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.b is not active+clean 2023-10-09T06:37:51.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.a is not active+clean 2023-10-09T06:37:51.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.147628+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T16:28:29.141519+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-09T06:37:51.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.8 is not active+clean 2023-10-09T06:37:51.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.9 is not active+clean 2023-10-09T06:37:51.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.148789+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T14:59:12.303101+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-09T06:37:51.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.9 is not active+clean 2023-10-09T06:37:51.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.8 is not active+clean 2023-10-09T06:37:51.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.149595+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T15:24:27.771345+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-09T06:37:51.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.6 is not active+clean 2023-10-09T06:37:51.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.7 is not active+clean 2023-10-09T06:37:51.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.145700+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T15:19:19.201484+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-09T06:37:51.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.7 is not active+clean 2023-10-09T06:37:51.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.6 is not active+clean 2023-10-09T06:37:51.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.151603+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T14:34:05.362873+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-09T06:37:51.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.4 is not active+clean 2023-10-09T06:37:51.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.5 is not active+clean 2023-10-09T06:37:51.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.149469+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T15:19:19.201484+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-09T06:37:51.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.5 is not active+clean 2023-10-09T06:37:51.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.4 is not active+clean 2023-10-09T06:37:51.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.144969+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T09:59:31.649021+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-09T06:37:51.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.2 is not active+clean 2023-10-09T06:37:51.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3 is not active+clean 2023-10-09T06:37:51.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.149695+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T17:24:42.401357+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-09T06:37:51.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.3 is not active+clean 2023-10-09T06:37:51.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2 is not active+clean 2023-10-09T06:37:51.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.149646+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T16:03:33.845927+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-09T06:37:51.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.0 is not active+clean 2023-10-09T06:37:51.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1 is not active+clean 2023-10-09T06:37:51.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.145665+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T14:42:15.133743+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-09T06:37:51.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.1 is not active+clean 2023-10-09T06:37:51.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.0 is not active+clean 2023-10-09T06:37:51.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.149518+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T10:16:35.538841+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-09T06:37:51.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 25.18 is not active+clean 2023-10-09T06:37:51.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '25.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-09T06:37:46.140387+0000', 'last_change': '2023-10-09T06:37:46.140387+0000', 'last_active': '2023-10-09T06:37:46.140387+0000', 'last_peered': '2023-10-09T06:37:46.140387+0000', 'last_clean': '2023-10-09T06:37:46.140387+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:46.140387+0000', 'last_undegraded': '2023-10-09T06:37:46.140387+0000', 'last_fullsized': '2023-10-09T06:37:46.140387+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:46.140387+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-09T06:37:51.990 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.19 is not active+clean 2023-10-09T06:37:51.990 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:37:45.133411+0000', 'last_change': '2023-10-09T06:37:45.145331+0000', 'last_active': '2023-10-09T06:37:45.133411+0000', 'last_peered': '2023-10-09T06:37:45.133411+0000', 'last_clean': '2023-10-09T06:37:45.133411+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:37:45.133411+0000', 'last_undegraded': '2023-10-09T06:37:45.133411+0000', 'last_fullsized': '2023-10-09T06:37:45.133411+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'last_clean_scrub_stamp': '2023-10-09T06:37:45.133411+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T15:49:26.936922+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-09T06:37:51.990 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-09T06:37:52.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.114+0000 7f14ba03f700 1 Processor -- start 2023-10-09T06:37:52.115 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.114+0000 7f14ba03f700 1 -- start start 2023-10-09T06:37:52.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.114+0000 7f14ba03f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14b4153f50 0x7f14b415abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:52.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.114+0000 7f14ba03f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f14b4154320 0x7f14b40b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:52.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.114+0000 7f14ba03f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b415b130 con 0x7f14b40b2020 2023-10-09T06:37:52.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.114+0000 7f14ba03f700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b40b2480 con 0x7f14b4154320 2023-10-09T06:37:52.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.114+0000 7f14ba03f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b415db50 con 0x7f14b4153f50 2023-10-09T06:37:52.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.115+0000 7f14b2ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f14b4154320 0x7f14b40b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:52.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.115+0000 7f14b2ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f14b4154320 0x7f14b40b1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41716/0 (socket says 172.21.15.163:41716) 2023-10-09T06:37:52.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.115+0000 7f14b2ffd700 1 -- 172.21.15.163:0/2557776729 learned_addr learned my addr 172.21.15.163:0/2557776729 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:52.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.115+0000 7f14b37fe700 1 --2- 172.21.15.163:0/2557776729 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14b4153f50 0x7f14b415abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:52.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.115+0000 7f14b2ffd700 1 -- 172.21.15.163:0/2557776729 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14b4153f50 msgr2=0x7f14b415abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:52.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.115+0000 7f14b2ffd700 1 --2- 172.21.15.163:0/2557776729 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14b4153f50 0x7f14b415abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:52.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.115+0000 7f14b2ffd700 1 -- 172.21.15.163:0/2557776729 >> v1:172.21.15.181:6789/0 conn(0x7f14b40b2020 legacy=0x7f14b406d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:52.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.115+0000 7f14b2ffd700 1 -- 172.21.15.163:0/2557776729 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b415de30 con 0x7f14b4154320 2023-10-09T06:37:52.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.115+0000 7f14b37fe700 1 --2- 172.21.15.163:0/2557776729 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14b4153f50 0x7f14b415abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:37:52.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.115+0000 7f14b2ffd700 1 --2- 172.21.15.163:0/2557776729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14b4154320 0x7f14b40b1ae0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f14a80099b0 tx=0x7f14a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ea0f555983458d server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:52.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.115+0000 7f14b27fc700 1 -- 172.21.15.163:0/2557776729 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a800b820 con 0x7f14b4154320 2023-10-09T06:37:52.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.115+0000 7f14b27fc700 1 -- 172.21.15.163:0/2557776729 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a800b980 con 0x7f14b4154320 2023-10-09T06:37:52.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.115+0000 7f14b27fc700 1 -- 172.21.15.163:0/2557776729 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a800bd40 con 0x7f14b4154320 2023-10-09T06:37:52.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.116+0000 7f14ba03f700 1 -- 172.21.15.163:0/2557776729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14b4154320 msgr2=0x7f14b40b1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:52.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.116+0000 7f14ba03f700 1 --2- 172.21.15.163:0/2557776729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14b4154320 0x7f14b40b1ae0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f14a80099b0 tx=0x7f14a8009250 comp rx=0 tx=0).stop 2023-10-09T06:37:52.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.116+0000 7f14ba03f700 1 -- 172.21.15.163:0/2557776729 shutdown_connections 2023-10-09T06:37:52.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.116+0000 7f14ba03f700 1 --2- 172.21.15.163:0/2557776729 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14b4153f50 0x7f14b415abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:52.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.116+0000 7f14ba03f700 1 --2- 172.21.15.163:0/2557776729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14b4154320 0x7f14b40b1ae0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:52.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.116+0000 7f14ba03f700 1 -- 172.21.15.163:0/2557776729 >> 172.21.15.163:0/2557776729 conn(0x7f14b406baf0 msgr2=0x7f14b406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:52.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.116+0000 7f14ba03f700 1 -- 172.21.15.163:0/2557776729 shutdown_connections 2023-10-09T06:37:52.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.116+0000 7f14ba03f700 1 -- 172.21.15.163:0/2557776729 wait complete. 2023-10-09T06:37:52.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.117+0000 7f14ba03f700 1 Processor -- start 2023-10-09T06:37:52.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.117+0000 7f14ba03f700 1 -- start start 2023-10-09T06:37:52.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.118+0000 7f14ba03f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14b40b2020 0x7f14b40ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:52.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.118+0000 7f14ba03f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14b4153f50 0x7f14b4142160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:52.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.118+0000 7f14ba03f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14b4154320 0x7f14b40ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:52.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.118+0000 7f14ba03f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b40cad80 con 0x7f14b4153f50 2023-10-09T06:37:52.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.118+0000 7f14ba03f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b40af000 con 0x7f14b4154320 2023-10-09T06:37:52.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.118+0000 7f14ba03f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b40af2e0 con 0x7f14b40b2020 2023-10-09T06:37:52.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.118+0000 7f14b2ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14b4153f50 0x7f14b4142160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:52.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.118+0000 7f14b2ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14b4153f50 0x7f14b4142160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41722/0 (socket says 172.21.15.163:41722) 2023-10-09T06:37:52.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.118+0000 7f14b2ffd700 1 -- 172.21.15.163:0/230520880 learned_addr learned my addr 172.21.15.163:0/230520880 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:52.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.118+0000 7f14b3fff700 1 --2- 172.21.15.163:0/230520880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14b4154320 0x7f14b40ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:52.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.118+0000 7f14b37fe700 1 --2- 172.21.15.163:0/230520880 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14b40b2020 0x7f14b40ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:52.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.118+0000 7f14b2ffd700 1 -- 172.21.15.163:0/230520880 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14b40b2020 msgr2=0x7f14b40ae740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:52.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.118+0000 7f14b2ffd700 1 --2- 172.21.15.163:0/230520880 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14b40b2020 0x7f14b40ae740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:52.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.118+0000 7f14b2ffd700 1 -- 172.21.15.163:0/230520880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14b4154320 msgr2=0x7f14b40ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:52.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.118+0000 7f14b2ffd700 1 --2- 172.21.15.163:0/230520880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14b4154320 0x7f14b40ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:52.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.118+0000 7f14b2ffd700 1 -- 172.21.15.163:0/230520880 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a80094f0 con 0x7f14b4153f50 2023-10-09T06:37:52.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.118+0000 7f14b3fff700 1 --2- 172.21.15.163:0/230520880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14b4154320 0x7f14b40ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:37:52.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.118+0000 7f14b2ffd700 1 --2- 172.21.15.163:0/230520880 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14b4153f50 0x7f14b4142160 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f14b40b46f0 tx=0x7f14a8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:52.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.118+0000 7f149b7fe700 1 -- 172.21.15.163:0/230520880 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a8011c80 con 0x7f14b4153f50 2023-10-09T06:37:52.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.119+0000 7f149b7fe700 1 -- 172.21.15.163:0/230520880 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a8020070 con 0x7f14b4153f50 2023-10-09T06:37:52.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.119+0000 7f14ba03f700 1 -- 172.21.15.163:0/230520880 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b40af5c0 con 0x7f14b4153f50 2023-10-09T06:37:52.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.119+0000 7f149b7fe700 1 -- 172.21.15.163:0/230520880 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a801b6a0 con 0x7f14b4153f50 2023-10-09T06:37:52.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.119+0000 7f14ba03f700 1 -- 172.21.15.163:0/230520880 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b40aec80 con 0x7f14b4153f50 2023-10-09T06:37:52.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.120+0000 7f14ba03f700 1 -- 172.21.15.163:0/230520880 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b40caef0 con 0x7f14b4153f50 2023-10-09T06:37:52.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.120+0000 7f149b7fe700 1 -- 172.21.15.163:0/230520880 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f14a8022030 con 0x7f14b4153f50 2023-10-09T06:37:52.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.120+0000 7f149b7fe700 1 --2- 172.21.15.163:0/230520880 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f14a0067320 0x7f14a00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:52.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.120+0000 7f149b7fe700 1 -- 172.21.15.163:0/230520880 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f14a8086890 con 0x7f14b4153f50 2023-10-09T06:37:52.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.121+0000 7f14b37fe700 1 --2- 172.21.15.163:0/230520880 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f14a0067320 0x7f14a00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:52.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.121+0000 7f14b37fe700 1 --2- 172.21.15.163:0/230520880 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f14a0067320 0x7f14a00697d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f14b40b0640 tx=0x7f14a40075c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:52.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.125+0000 7f149b7fe700 1 -- 172.21.15.163:0/230520880 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f14a8053630 con 0x7f14b4153f50 2023-10-09T06:37:52.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.382+0000 7f14ba03f700 1 -- 172.21.15.163:0/230520880 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f14b415b8e0 con 0x7f14b4153f50 2023-10-09T06:37:52.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.383+0000 7f149b7fe700 1 -- 172.21.15.163:0/230520880 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1534 (secure 0 0 0) 0x7f14a8056c50 con 0x7f14b4153f50 2023-10-09T06:37:52.384 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:52.384 INFO:teuthology.orchestra.run.smithi163.stdout:{"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":453,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":137,"num_osds":8,"num_up_osds":8,"osd_up_since":1696833029,"num_in_osds":8,"osd_in_since":1696833021,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":2483593216,"bytes_avail":770610520064,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":93,"id":12,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":12,"rank":0,"name":"c","status":"up:active","gid":7518}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2023-10-09T06:37:36.399840+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-09T06:37:52.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.385+0000 7f14ba03f700 1 -- 172.21.15.163:0/230520880 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f14a0067320 msgr2=0x7f14a00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:52.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.385+0000 7f14ba03f700 1 --2- 172.21.15.163:0/230520880 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f14a0067320 0x7f14a00697d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f14b40b0640 tx=0x7f14a40075c0 comp rx=0 tx=0).stop 2023-10-09T06:37:52.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.385+0000 7f14ba03f700 1 -- 172.21.15.163:0/230520880 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14b4153f50 msgr2=0x7f14b4142160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:52.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.385+0000 7f14ba03f700 1 --2- 172.21.15.163:0/230520880 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14b4153f50 0x7f14b4142160 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f14b40b46f0 tx=0x7f14a8016000 comp rx=0 tx=0).stop 2023-10-09T06:37:52.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.385+0000 7f14ba03f700 1 -- 172.21.15.163:0/230520880 shutdown_connections 2023-10-09T06:37:52.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.385+0000 7f14ba03f700 1 --2- 172.21.15.163:0/230520880 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f14a0067320 0x7f14a00697d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:52.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.385+0000 7f14ba03f700 1 --2- 172.21.15.163:0/230520880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14b4154320 0x7f14b40ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:52.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.385+0000 7f14ba03f700 1 --2- 172.21.15.163:0/230520880 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14b40b2020 0x7f14b40ae740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:52.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.385+0000 7f14ba03f700 1 --2- 172.21.15.163:0/230520880 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14b4153f50 0x7f14b4142160 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:52.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.385+0000 7f14ba03f700 1 -- 172.21.15.163:0/230520880 >> 172.21.15.163:0/230520880 conn(0x7f14b406baf0 msgr2=0x7f14b41514d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:52.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.385+0000 7f14ba03f700 1 -- 172.21.15.163:0/230520880 shutdown_connections 2023-10-09T06:37:52.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.385+0000 7f14ba03f700 1 -- 172.21.15.163:0/230520880 wait complete. 2023-10-09T06:37:52.398 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-09T06:37:52.398 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:37:52.437 INFO:tasks.ceph.mds.c.smithi163.stderr:2023-10-09T06:37:52.436+0000 7ff249cc3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-09T06:37:52.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.511+0000 7f2ec415f700 1 Processor -- start 2023-10-09T06:37:52.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.512+0000 7f2ec415f700 1 -- start start 2023-10-09T06:37:52.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.512+0000 7f2ec415f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ebc0aca80 0x7f2ebc06aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:52.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.512+0000 7f2ec415f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2ebc0c83c0 0x7f2ebc0cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:52.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.512+0000 7f2ec415f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ebc06bdb0 con 0x7f2ebc069d60 2023-10-09T06:37:52.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.512+0000 7f2ec415f700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ebc0cb370 con 0x7f2ebc0c83c0 2023-10-09T06:37:52.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.512+0000 7f2ec415f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc159f60 con 0x7f2ebc0aca80 2023-10-09T06:37:52.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.512+0000 7f2ec26fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2ebc0c83c0 0x7f2ebc0cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:52.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.513+0000 7f2ec26fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2ebc0c83c0 0x7f2ebc0cb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41724/0 (socket says 172.21.15.163:41724) 2023-10-09T06:37:52.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.513+0000 7f2ec26fc700 1 -- 172.21.15.163:0/3435064075 learned_addr learned my addr 172.21.15.163:0/3435064075 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:52.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.513+0000 7f2ec1efb700 1 --2- 172.21.15.163:0/3435064075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ebc0aca80 0x7f2ebc06aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:52.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.513+0000 7f2ec26fc700 1 -- 172.21.15.163:0/3435064075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ebc0aca80 msgr2=0x7f2ebc06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:52.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.513+0000 7f2ec26fc700 1 --2- 172.21.15.163:0/3435064075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ebc0aca80 0x7f2ebc06aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:52.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.513+0000 7f2ec26fc700 1 -- 172.21.15.163:0/3435064075 >> v1:172.21.15.181:6789/0 conn(0x7f2ebc069d60 legacy=0x7f2ebc0c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:52.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.513+0000 7f2ec26fc700 1 -- 172.21.15.163:0/3435064075 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ebc06b510 con 0x7f2ebc0c83c0 2023-10-09T06:37:52.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.513+0000 7f2ec26fc700 1 --2- 172.21.15.163:0/3435064075 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ebc0c83c0 0x7f2ebc0cb800 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f2eb40099b0 tx=0x7f2eb4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd984e477e18aaf5 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:52.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.513+0000 7f2ec0ef9700 1 -- 172.21.15.163:0/3435064075 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb400b860 con 0x7f2ebc0c83c0 2023-10-09T06:37:52.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.513+0000 7f2ec0ef9700 1 -- 172.21.15.163:0/3435064075 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb400b9c0 con 0x7f2ebc0c83c0 2023-10-09T06:37:52.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.513+0000 7f2ec0ef9700 1 -- 172.21.15.163:0/3435064075 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb400bd60 con 0x7f2ebc0c83c0 2023-10-09T06:37:52.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.514+0000 7f2ec415f700 1 -- 172.21.15.163:0/3435064075 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ebc0c83c0 msgr2=0x7f2ebc0cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:52.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.514+0000 7f2ec415f700 1 --2- 172.21.15.163:0/3435064075 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ebc0c83c0 0x7f2ebc0cb800 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f2eb40099b0 tx=0x7f2eb4009250 comp rx=0 tx=0).stop 2023-10-09T06:37:52.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.514+0000 7f2ec415f700 1 -- 172.21.15.163:0/3435064075 shutdown_connections 2023-10-09T06:37:52.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.514+0000 7f2ec415f700 1 --2- 172.21.15.163:0/3435064075 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ebc0c83c0 0x7f2ebc0cb800 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:52.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.514+0000 7f2ec415f700 1 --2- 172.21.15.163:0/3435064075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ebc0aca80 0x7f2ebc06aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:52.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.514+0000 7f2ec415f700 1 -- 172.21.15.163:0/3435064075 >> 172.21.15.163:0/3435064075 conn(0x7f2ebc0cd670 msgr2=0x7f2ebc0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:52.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.514+0000 7f2ec415f700 1 -- 172.21.15.163:0/3435064075 shutdown_connections 2023-10-09T06:37:52.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.514+0000 7f2ec415f700 1 -- 172.21.15.163:0/3435064075 wait complete. 2023-10-09T06:37:52.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.515+0000 7f2ec415f700 1 Processor -- start 2023-10-09T06:37:52.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.515+0000 7f2ec415f700 1 -- start start 2023-10-09T06:37:52.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.515+0000 7f2ec415f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ebc069d60 0x7f2ebc0c4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:52.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.515+0000 7f2ec415f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ebc0aca80 0x7f2ebc16a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:52.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.516+0000 7f2ec415f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ebc0c83c0 0x7f2ebc145f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:52.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.516+0000 7f2ec415f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc0b2240 con 0x7f2ebc0c83c0 2023-10-09T06:37:52.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.516+0000 7f2ec415f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc16acc0 con 0x7f2ebc069d60 2023-10-09T06:37:52.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.516+0000 7f2ec415f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc158c70 con 0x7f2ebc0aca80 2023-10-09T06:37:52.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.516+0000 7f2ec26fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ebc0c83c0 0x7f2ebc145f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:52.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.516+0000 7f2ec26fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ebc0c83c0 0x7f2ebc145f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41738/0 (socket says 172.21.15.163:41738) 2023-10-09T06:37:52.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.516+0000 7f2ec26fc700 1 -- 172.21.15.163:0/1802383543 learned_addr learned my addr 172.21.15.163:0/1802383543 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:52.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.516+0000 7f2ec1efb700 1 --2- 172.21.15.163:0/1802383543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ebc069d60 0x7f2ebc0c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:52.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.516+0000 7f2ec16fa700 1 --2- 172.21.15.163:0/1802383543 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ebc0aca80 0x7f2ebc16a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:52.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.516+0000 7f2ec26fc700 1 -- 172.21.15.163:0/1802383543 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ebc0aca80 msgr2=0x7f2ebc16a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:52.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.516+0000 7f2ec26fc700 1 --2- 172.21.15.163:0/1802383543 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ebc0aca80 0x7f2ebc16a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:52.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.516+0000 7f2ec26fc700 1 -- 172.21.15.163:0/1802383543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ebc069d60 msgr2=0x7f2ebc0c4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:52.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.516+0000 7f2ec26fc700 1 --2- 172.21.15.163:0/1802383543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ebc069d60 0x7f2ebc0c4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:52.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.516+0000 7f2ec26fc700 1 -- 172.21.15.163:0/1802383543 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb40094f0 con 0x7f2ebc0c83c0 2023-10-09T06:37:52.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.516+0000 7f2ec26fc700 1 --2- 172.21.15.163:0/1802383543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ebc0c83c0 0x7f2ebc145f90 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc06c6a0 tx=0x7f2eb4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:52.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.516+0000 7f2eb1ffb700 1 -- 172.21.15.163:0/1802383543 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4011ea0 con 0x7f2ebc0c83c0 2023-10-09T06:37:52.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.516+0000 7f2eb1ffb700 1 -- 172.21.15.163:0/1802383543 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb401a630 con 0x7f2ebc0c83c0 2023-10-09T06:37:52.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.516+0000 7f2eb1ffb700 1 -- 172.21.15.163:0/1802383543 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb401aa10 con 0x7f2ebc0c83c0 2023-10-09T06:37:52.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.516+0000 7f2ec415f700 1 -- 172.21.15.163:0/1802383543 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ebc1464d0 con 0x7f2ebc0c83c0 2023-10-09T06:37:52.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.516+0000 7f2ec415f700 1 -- 172.21.15.163:0/1802383543 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ebc16db90 con 0x7f2ebc0c83c0 2023-10-09T06:37:52.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.517+0000 7f2ec415f700 1 -- 172.21.15.163:0/1802383543 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ebc0b23f0 con 0x7f2ebc0c83c0 2023-10-09T06:37:52.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.518+0000 7f2eb1ffb700 1 -- 172.21.15.163:0/1802383543 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2eb402d440 con 0x7f2ebc0c83c0 2023-10-09T06:37:52.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.518+0000 7f2eb1ffb700 1 --2- 172.21.15.163:0/1802383543 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2ea0067320 0x7f2ea00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:52.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.518+0000 7f2eb1ffb700 1 -- 172.21.15.163:0/1802383543 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2eb4086650 con 0x7f2ebc0c83c0 2023-10-09T06:37:52.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.519+0000 7f2ec1efb700 1 --2- 172.21.15.163:0/1802383543 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2ea0067320 0x7f2ea00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:52.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.519+0000 7f2ec1efb700 1 --2- 172.21.15.163:0/1802383543 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2ea0067320 0x7f2ea00697d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc0b1ef0 tx=0x7f2eb80098d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:52.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.523+0000 7f2eb1ffb700 1 -- 172.21.15.163:0/1802383543 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2eb40533e0 con 0x7f2ebc0c83c0 2023-10-09T06:37:52.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.701+0000 7f2ec415f700 1 -- 172.21.15.163:0/1802383543 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2ebc04c410 con 0x7f2ea0067320 2023-10-09T06:37:52.710 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.710+0000 7f2eb1ffb700 1 -- 172.21.15.163:0/1802383543 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290264 (secure 0 0 0) 0x7f2ebc04c410 con 0x7f2ea0067320 2023-10-09T06:37:52.714 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:52.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.713+0000 7f2ec415f700 1 -- 172.21.15.163:0/1802383543 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2ea0067320 msgr2=0x7f2ea00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:52.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.713+0000 7f2ec415f700 1 --2- 172.21.15.163:0/1802383543 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2ea0067320 0x7f2ea00697d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc0b1ef0 tx=0x7f2eb80098d0 comp rx=0 tx=0).stop 2023-10-09T06:37:52.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.713+0000 7f2ec415f700 1 -- 172.21.15.163:0/1802383543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ebc0c83c0 msgr2=0x7f2ebc145f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:52.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.713+0000 7f2ec415f700 1 --2- 172.21.15.163:0/1802383543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ebc0c83c0 0x7f2ebc145f90 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc06c6a0 tx=0x7f2eb4017000 comp rx=0 tx=0).stop 2023-10-09T06:37:52.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.713+0000 7f2ec415f700 1 -- 172.21.15.163:0/1802383543 shutdown_connections 2023-10-09T06:37:52.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.713+0000 7f2ec415f700 1 --2- 172.21.15.163:0/1802383543 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2ea0067320 0x7f2ea00697d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:52.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.713+0000 7f2ec415f700 1 --2- 172.21.15.163:0/1802383543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ebc0c83c0 0x7f2ebc145f90 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:52.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.713+0000 7f2ec415f700 1 --2- 172.21.15.163:0/1802383543 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ebc069d60 0x7f2ebc0c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:52.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.713+0000 7f2ec415f700 1 --2- 172.21.15.163:0/1802383543 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ebc0aca80 0x7f2ebc16a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:52.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.713+0000 7f2ec415f700 1 -- 172.21.15.163:0/1802383543 >> 172.21.15.163:0/1802383543 conn(0x7f2ebc0cd670 msgr2=0x7f2ebc0c2410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:52.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.713+0000 7f2ec415f700 1 -- 172.21.15.163:0/1802383543 shutdown_connections 2023-10-09T06:37:52.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:52.713+0000 7f2ec415f700 1 -- 172.21.15.163:0/1802383543 wait complete. 2023-10-09T06:37:52.717 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:37:52.726 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":351,"stamp":"2023-10-09T06:37:52.403136+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2425832,"kb_used_data":14760,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752548888,"statfs":{"total":773094113280,"available":770610061312,"internally_reserved":0,"allocated":15114240,"data_stored":7371540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.259007"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.183830+0000","last_change":"2023-10-09T06:37:47.179494+0000","last_active":"2023-10-09T06:37:47.183830+0000","last_peered":"2023-10-09T06:37:47.183830+0000","last_clean":"2023-10-09T06:37:47.183830+0000","last_became_active":"2023-10-09T06:37:47.179178+0000","last_became_peered":"2023-10-09T06:37:47.179178+0000","last_unstale":"2023-10-09T06:37:47.183830+0000","last_undegraded":"2023-10-09T06:37:47.183830+0000","last_fullsized":"2023-10-09T06:37:47.183830+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:02:47.287524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.174192+0000","last_change":"2023-10-09T06:37:47.170317+0000","last_active":"2023-10-09T06:37:47.174192+0000","last_peered":"2023-10-09T06:37:47.174192+0000","last_clean":"2023-10-09T06:37:47.174192+0000","last_became_active":"2023-10-09T06:37:47.169622+0000","last_became_peered":"2023-10-09T06:37:47.169622+0000","last_unstale":"2023-10-09T06:37:47.174192+0000","last_undegraded":"2023-10-09T06:37:47.174192+0000","last_fullsized":"2023-10-09T06:37:47.174192+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:22:45.789761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.184428+0000","last_change":"2023-10-09T06:37:47.179624+0000","last_active":"2023-10-09T06:37:47.184428+0000","last_peered":"2023-10-09T06:37:47.184428+0000","last_clean":"2023-10-09T06:37:47.184428+0000","last_became_active":"2023-10-09T06:37:47.179336+0000","last_became_peered":"2023-10-09T06:37:47.179336+0000","last_unstale":"2023-10-09T06:37:47.184428+0000","last_undegraded":"2023-10-09T06:37:47.184428+0000","last_fullsized":"2023-10-09T06:37:47.184428+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:31:42.797752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.182655+0000","last_change":"2023-10-09T06:37:47.179465+0000","last_active":"2023-10-09T06:37:47.182655+0000","last_peered":"2023-10-09T06:37:47.182655+0000","last_clean":"2023-10-09T06:37:47.182655+0000","last_became_active":"2023-10-09T06:37:47.178726+0000","last_became_peered":"2023-10-09T06:37:47.178726+0000","last_unstale":"2023-10-09T06:37:47.182655+0000","last_undegraded":"2023-10-09T06:37:47.182655+0000","last_fullsized":"2023-10-09T06:37:47.182655+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:39:40.857018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.181258+0000","last_change":"2023-10-09T06:37:47.180178+0000","last_active":"2023-10-09T06:37:47.181258+0000","last_peered":"2023-10-09T06:37:47.181258+0000","last_clean":"2023-10-09T06:37:47.181258+0000","last_became_active":"2023-10-09T06:37:47.179891+0000","last_became_peered":"2023-10-09T06:37:47.179891+0000","last_unstale":"2023-10-09T06:37:47.181258+0000","last_undegraded":"2023-10-09T06:37:47.181258+0000","last_fullsized":"2023-10-09T06:37:47.181258+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:01:23.258784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.184380+0000","last_change":"2023-10-09T06:37:47.177855+0000","last_active":"2023-10-09T06:37:47.184380+0000","last_peered":"2023-10-09T06:37:47.184380+0000","last_clean":"2023-10-09T06:37:47.184380+0000","last_became_active":"2023-10-09T06:37:47.177194+0000","last_became_peered":"2023-10-09T06:37:47.177194+0000","last_unstale":"2023-10-09T06:37:47.184380+0000","last_undegraded":"2023-10-09T06:37:47.184380+0000","last_fullsized":"2023-10-09T06:37:47.184380+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:02:47.287524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.175545+0000","last_change":"2023-10-09T06:37:47.172833+0000","last_active":"2023-10-09T06:37:47.175545+0000","last_peered":"2023-10-09T06:37:47.175545+0000","last_clean":"2023-10-09T06:37:47.175545+0000","last_became_active":"2023-10-09T06:37:47.172497+0000","last_became_peered":"2023-10-09T06:37:47.172497+0000","last_unstale":"2023-10-09T06:37:47.175545+0000","last_undegraded":"2023-10-09T06:37:47.175545+0000","last_fullsized":"2023-10-09T06:37:47.175545+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:38:04.981588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.177953+0000","last_change":"2023-10-09T06:37:47.172754+0000","last_active":"2023-10-09T06:37:47.177953+0000","last_peered":"2023-10-09T06:37:47.177953+0000","last_clean":"2023-10-09T06:37:47.177953+0000","last_became_active":"2023-10-09T06:37:47.172530+0000","last_became_peered":"2023-10-09T06:37:47.172530+0000","last_unstale":"2023-10-09T06:37:47.177953+0000","last_undegraded":"2023-10-09T06:37:47.177953+0000","last_fullsized":"2023-10-09T06:37:47.177953+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:59:24.334461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.180894+0000","last_change":"2023-10-09T06:37:47.175288+0000","last_active":"2023-10-09T06:37:47.180894+0000","last_peered":"2023-10-09T06:37:47.180894+0000","last_clean":"2023-10-09T06:37:47.180894+0000","last_became_active":"2023-10-09T06:37:47.174830+0000","last_became_peered":"2023-10-09T06:37:47.174830+0000","last_unstale":"2023-10-09T06:37:47.180894+0000","last_undegraded":"2023-10-09T06:37:47.180894+0000","last_fullsized":"2023-10-09T06:37:47.180894+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:30:56.136503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.178069+0000","last_change":"2023-10-09T06:37:47.169865+0000","last_active":"2023-10-09T06:37:47.178069+0000","last_peered":"2023-10-09T06:37:47.178069+0000","last_clean":"2023-10-09T06:37:47.178069+0000","last_became_active":"2023-10-09T06:37:47.169615+0000","last_became_peered":"2023-10-09T06:37:47.169615+0000","last_unstale":"2023-10-09T06:37:47.178069+0000","last_undegraded":"2023-10-09T06:37:47.178069+0000","last_fullsized":"2023-10-09T06:37:47.178069+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:40:16.005552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.180769+0000","last_change":"2023-10-09T06:37:47.177471+0000","last_active":"2023-10-09T06:37:47.180769+0000","last_peered":"2023-10-09T06:37:47.180769+0000","last_clean":"2023-10-09T06:37:47.180769+0000","last_became_active":"2023-10-09T06:37:47.177167+0000","last_became_peered":"2023-10-09T06:37:47.177167+0000","last_unstale":"2023-10-09T06:37:47.180769+0000","last_undegraded":"2023-10-09T06:37:47.180769+0000","last_fullsized":"2023-10-09T06:37:47.180769+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:55:35.647464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.180839+0000","last_change":"2023-10-09T06:37:47.178688+0000","last_active":"2023-10-09T06:37:47.180839+0000","last_peered":"2023-10-09T06:37:47.180839+0000","last_clean":"2023-10-09T06:37:47.180839+0000","last_became_active":"2023-10-09T06:37:47.178433+0000","last_became_peered":"2023-10-09T06:37:47.178433+0000","last_unstale":"2023-10-09T06:37:47.180839+0000","last_undegraded":"2023-10-09T06:37:47.180839+0000","last_fullsized":"2023-10-09T06:37:47.180839+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:26:39.377444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.172274+0000","last_change":"2023-10-09T06:37:47.169996+0000","last_active":"2023-10-09T06:37:47.172274+0000","last_peered":"2023-10-09T06:37:47.172274+0000","last_clean":"2023-10-09T06:37:47.172274+0000","last_became_active":"2023-10-09T06:37:47.169396+0000","last_became_peered":"2023-10-09T06:37:47.169396+0000","last_unstale":"2023-10-09T06:37:47.172274+0000","last_undegraded":"2023-10-09T06:37:47.172274+0000","last_fullsized":"2023-10-09T06:37:47.172274+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:04:59.600452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.179031+0000","last_change":"2023-10-09T06:37:47.173418+0000","last_active":"2023-10-09T06:37:47.179031+0000","last_peered":"2023-10-09T06:37:47.179031+0000","last_clean":"2023-10-09T06:37:47.179031+0000","last_became_active":"2023-10-09T06:37:47.173133+0000","last_became_peered":"2023-10-09T06:37:47.173133+0000","last_unstale":"2023-10-09T06:37:47.179031+0000","last_undegraded":"2023-10-09T06:37:47.179031+0000","last_fullsized":"2023-10-09T06:37:47.179031+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:04:59.600452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.182168+0000","last_change":"2023-10-09T06:37:47.179720+0000","last_active":"2023-10-09T06:37:47.182168+0000","last_peered":"2023-10-09T06:37:47.182168+0000","last_clean":"2023-10-09T06:37:47.182168+0000","last_became_active":"2023-10-09T06:37:47.179094+0000","last_became_peered":"2023-10-09T06:37:47.179094+0000","last_unstale":"2023-10-09T06:37:47.182168+0000","last_undegraded":"2023-10-09T06:37:47.182168+0000","last_fullsized":"2023-10-09T06:37:47.182168+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:30:56.136503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.178270+0000","last_change":"2023-10-09T06:37:47.175094+0000","last_active":"2023-10-09T06:37:47.178270+0000","last_peered":"2023-10-09T06:37:47.178270+0000","last_clean":"2023-10-09T06:37:47.178270+0000","last_became_active":"2023-10-09T06:37:47.174842+0000","last_became_peered":"2023-10-09T06:37:47.174842+0000","last_unstale":"2023-10-09T06:37:47.178270+0000","last_undegraded":"2023-10-09T06:37:47.178270+0000","last_fullsized":"2023-10-09T06:37:47.178270+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:11:04.361133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.187695+0000","last_change":"2023-10-09T06:37:47.181745+0000","last_active":"2023-10-09T06:37:47.187695+0000","last_peered":"2023-10-09T06:37:47.187695+0000","last_clean":"2023-10-09T06:37:47.187695+0000","last_became_active":"2023-10-09T06:37:47.175655+0000","last_became_peered":"2023-10-09T06:37:47.175655+0000","last_unstale":"2023-10-09T06:37:47.187695+0000","last_undegraded":"2023-10-09T06:37:47.187695+0000","last_fullsized":"2023-10-09T06:37:47.187695+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:24:43.408333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.176651+0000","last_change":"2023-10-09T06:37:47.175166+0000","last_active":"2023-10-09T06:37:47.176651+0000","last_peered":"2023-10-09T06:37:47.176651+0000","last_clean":"2023-10-09T06:37:47.176651+0000","last_became_active":"2023-10-09T06:37:47.174847+0000","last_became_peered":"2023-10-09T06:37:47.174847+0000","last_unstale":"2023-10-09T06:37:47.176651+0000","last_undegraded":"2023-10-09T06:37:47.176651+0000","last_fullsized":"2023-10-09T06:37:47.176651+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:30:30.469711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.181764+0000","last_change":"2023-10-09T06:37:47.179072+0000","last_active":"2023-10-09T06:37:47.181764+0000","last_peered":"2023-10-09T06:37:47.181764+0000","last_clean":"2023-10-09T06:37:47.181764+0000","last_became_active":"2023-10-09T06:37:47.178800+0000","last_became_peered":"2023-10-09T06:37:47.178800+0000","last_unstale":"2023-10-09T06:37:47.181764+0000","last_undegraded":"2023-10-09T06:37:47.181764+0000","last_fullsized":"2023-10-09T06:37:47.181764+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:11:04.361133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.182584+0000","last_change":"2023-10-09T06:37:47.175579+0000","last_active":"2023-10-09T06:37:47.182584+0000","last_peered":"2023-10-09T06:37:47.182584+0000","last_clean":"2023-10-09T06:37:47.182584+0000","last_became_active":"2023-10-09T06:37:47.175348+0000","last_became_peered":"2023-10-09T06:37:47.175348+0000","last_unstale":"2023-10-09T06:37:47.182584+0000","last_undegraded":"2023-10-09T06:37:47.182584+0000","last_fullsized":"2023-10-09T06:37:47.182584+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:41:11.780330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.181247+0000","last_change":"2023-10-09T06:37:47.180074+0000","last_active":"2023-10-09T06:37:47.181247+0000","last_peered":"2023-10-09T06:37:47.181247+0000","last_clean":"2023-10-09T06:37:47.181247+0000","last_became_active":"2023-10-09T06:37:47.179800+0000","last_became_peered":"2023-10-09T06:37:47.179800+0000","last_unstale":"2023-10-09T06:37:47.181247+0000","last_undegraded":"2023-10-09T06:37:47.181247+0000","last_fullsized":"2023-10-09T06:37:47.181247+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:42:16.140720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.181696+0000","last_change":"2023-10-09T06:37:47.178389+0000","last_active":"2023-10-09T06:37:47.181696+0000","last_peered":"2023-10-09T06:37:47.181696+0000","last_clean":"2023-10-09T06:37:47.181696+0000","last_became_active":"2023-10-09T06:37:47.178178+0000","last_became_peered":"2023-10-09T06:37:47.178178+0000","last_unstale":"2023-10-09T06:37:47.181696+0000","last_undegraded":"2023-10-09T06:37:47.181696+0000","last_fullsized":"2023-10-09T06:37:47.181696+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:49:25.010643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.175661+0000","last_change":"2023-10-09T06:37:47.171441+0000","last_active":"2023-10-09T06:37:47.175661+0000","last_peered":"2023-10-09T06:37:47.175661+0000","last_clean":"2023-10-09T06:37:47.175661+0000","last_became_active":"2023-10-09T06:37:47.171181+0000","last_became_peered":"2023-10-09T06:37:47.171181+0000","last_unstale":"2023-10-09T06:37:47.175661+0000","last_undegraded":"2023-10-09T06:37:47.175661+0000","last_fullsized":"2023-10-09T06:37:47.175661+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:30:30.469711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.177934+0000","last_change":"2023-10-09T06:37:47.169581+0000","last_active":"2023-10-09T06:37:47.177934+0000","last_peered":"2023-10-09T06:37:47.177934+0000","last_clean":"2023-10-09T06:37:47.177934+0000","last_became_active":"2023-10-09T06:37:47.169186+0000","last_became_peered":"2023-10-09T06:37:47.169186+0000","last_unstale":"2023-10-09T06:37:47.177934+0000","last_undegraded":"2023-10-09T06:37:47.177934+0000","last_fullsized":"2023-10-09T06:37:47.177934+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:49:54.766563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.174021+0000","last_change":"2023-10-09T06:37:47.171633+0000","last_active":"2023-10-09T06:37:47.174021+0000","last_peered":"2023-10-09T06:37:47.174021+0000","last_clean":"2023-10-09T06:37:47.174021+0000","last_became_active":"2023-10-09T06:37:47.171387+0000","last_became_peered":"2023-10-09T06:37:47.171387+0000","last_unstale":"2023-10-09T06:37:47.174021+0000","last_undegraded":"2023-10-09T06:37:47.174021+0000","last_fullsized":"2023-10-09T06:37:47.174021+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:03:12.426555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.171549+0000","last_change":"2023-10-09T06:37:47.169127+0000","last_active":"2023-10-09T06:37:47.171549+0000","last_peered":"2023-10-09T06:37:47.171549+0000","last_clean":"2023-10-09T06:37:47.171549+0000","last_became_active":"2023-10-09T06:37:47.168815+0000","last_became_peered":"2023-10-09T06:37:47.168815+0000","last_unstale":"2023-10-09T06:37:47.171549+0000","last_undegraded":"2023-10-09T06:37:47.171549+0000","last_fullsized":"2023-10-09T06:37:47.171549+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:23:16.450280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.183759+0000","last_change":"2023-10-09T06:37:47.177549+0000","last_active":"2023-10-09T06:37:47.183759+0000","last_peered":"2023-10-09T06:37:47.183759+0000","last_clean":"2023-10-09T06:37:47.183759+0000","last_became_active":"2023-10-09T06:37:47.177263+0000","last_became_peered":"2023-10-09T06:37:47.177263+0000","last_unstale":"2023-10-09T06:37:47.183759+0000","last_undegraded":"2023-10-09T06:37:47.183759+0000","last_fullsized":"2023-10-09T06:37:47.183759+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:14:48.221558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.187920+0000","last_change":"2023-10-09T06:37:47.182473+0000","last_active":"2023-10-09T06:37:47.187920+0000","last_peered":"2023-10-09T06:37:47.187920+0000","last_clean":"2023-10-09T06:37:47.187920+0000","last_became_active":"2023-10-09T06:37:47.175737+0000","last_became_peered":"2023-10-09T06:37:47.175737+0000","last_unstale":"2023-10-09T06:37:47.187920+0000","last_undegraded":"2023-10-09T06:37:47.187920+0000","last_fullsized":"2023-10-09T06:37:47.187920+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:24:28.778321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.181631+0000","last_change":"2023-10-09T06:37:47.179313+0000","last_active":"2023-10-09T06:37:47.181631+0000","last_peered":"2023-10-09T06:37:47.181631+0000","last_clean":"2023-10-09T06:37:47.181631+0000","last_became_active":"2023-10-09T06:37:47.178278+0000","last_became_peered":"2023-10-09T06:37:47.178278+0000","last_unstale":"2023-10-09T06:37:47.181631+0000","last_undegraded":"2023-10-09T06:37:47.181631+0000","last_fullsized":"2023-10-09T06:37:47.181631+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:48:25.266457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.184511+0000","last_change":"2023-10-09T06:37:47.176318+0000","last_active":"2023-10-09T06:37:47.184511+0000","last_peered":"2023-10-09T06:37:47.184511+0000","last_clean":"2023-10-09T06:37:47.184511+0000","last_became_active":"2023-10-09T06:37:47.176056+0000","last_became_peered":"2023-10-09T06:37:47.176056+0000","last_unstale":"2023-10-09T06:37:47.184511+0000","last_undegraded":"2023-10-09T06:37:47.184511+0000","last_fullsized":"2023-10-09T06:37:47.184511+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:26:47.866012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.167706+0000","last_change":"2023-10-09T06:37:47.165039+0000","last_active":"2023-10-09T06:37:47.167706+0000","last_peered":"2023-10-09T06:37:47.167706+0000","last_clean":"2023-10-09T06:37:47.167706+0000","last_became_active":"2023-10-09T06:37:47.164773+0000","last_became_peered":"2023-10-09T06:37:47.164773+0000","last_unstale":"2023-10-09T06:37:47.167706+0000","last_undegraded":"2023-10-09T06:37:47.167706+0000","last_fullsized":"2023-10-09T06:37:47.167706+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:12:00.359900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.180601+0000","last_change":"2023-10-09T06:37:47.173408+0000","last_active":"2023-10-09T06:37:47.180601+0000","last_peered":"2023-10-09T06:37:47.180601+0000","last_clean":"2023-10-09T06:37:47.180601+0000","last_became_active":"2023-10-09T06:37:47.173077+0000","last_became_peered":"2023-10-09T06:37:47.173077+0000","last_unstale":"2023-10-09T06:37:47.180601+0000","last_undegraded":"2023-10-09T06:37:47.180601+0000","last_fullsized":"2023-10-09T06:37:47.180601+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:03:34.852904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.178226+0000","last_change":"2023-10-09T06:37:47.174887+0000","last_active":"2023-10-09T06:37:47.178226+0000","last_peered":"2023-10-09T06:37:47.178226+0000","last_clean":"2023-10-09T06:37:47.178226+0000","last_became_active":"2023-10-09T06:37:47.174598+0000","last_became_peered":"2023-10-09T06:37:47.174598+0000","last_unstale":"2023-10-09T06:37:47.178226+0000","last_undegraded":"2023-10-09T06:37:47.178226+0000","last_fullsized":"2023-10-09T06:37:47.178226+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:40:16.005552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"136'4","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.188324+0000","last_change":"2023-10-09T06:37:46.162496+0000","last_active":"2023-10-09T06:37:47.188324+0000","last_peered":"2023-10-09T06:37:47.188324+0000","last_clean":"2023-10-09T06:37:47.188324+0000","last_became_active":"2023-10-09T06:37:46.161574+0000","last_became_peered":"2023-10-09T06:37:46.161574+0000","last_unstale":"2023-10-09T06:37:47.188324+0000","last_undegraded":"2023-10-09T06:37:47.188324+0000","last_fullsized":"2023-10-09T06:37:47.188324+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:45:54.585264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.184382+0000","last_change":"2023-10-09T06:37:47.181847+0000","last_active":"2023-10-09T06:37:47.184382+0000","last_peered":"2023-10-09T06:37:47.184382+0000","last_clean":"2023-10-09T06:37:47.184382+0000","last_became_active":"2023-10-09T06:37:47.181586+0000","last_became_peered":"2023-10-09T06:37:47.181586+0000","last_unstale":"2023-10-09T06:37:47.184382+0000","last_undegraded":"2023-10-09T06:37:47.184382+0000","last_fullsized":"2023-10-09T06:37:47.184382+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:23:25.125922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.184474+0000","last_change":"2023-10-09T06:37:46.158583+0000","last_active":"2023-10-09T06:37:47.184474+0000","last_peered":"2023-10-09T06:37:47.184474+0000","last_clean":"2023-10-09T06:37:47.184474+0000","last_became_active":"2023-10-09T06:37:46.158300+0000","last_became_peered":"2023-10-09T06:37:46.158300+0000","last_unstale":"2023-10-09T06:37:47.184474+0000","last_undegraded":"2023-10-09T06:37:47.184474+0000","last_fullsized":"2023-10-09T06:37:47.184474+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:35:13.249625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.180578+0000","last_change":"2023-10-09T06:37:47.176951+0000","last_active":"2023-10-09T06:37:47.180578+0000","last_peered":"2023-10-09T06:37:47.180578+0000","last_clean":"2023-10-09T06:37:47.180578+0000","last_became_active":"2023-10-09T06:37:47.176489+0000","last_became_peered":"2023-10-09T06:37:47.176489+0000","last_unstale":"2023-10-09T06:37:47.180578+0000","last_undegraded":"2023-10-09T06:37:47.180578+0000","last_fullsized":"2023-10-09T06:37:47.180578+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:21:09.695263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"136'2","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.184592+0000","last_change":"2023-10-09T06:37:46.160289+0000","last_active":"2023-10-09T06:37:47.184592+0000","last_peered":"2023-10-09T06:37:47.184592+0000","last_clean":"2023-10-09T06:37:47.184592+0000","last_became_active":"2023-10-09T06:37:46.160048+0000","last_became_peered":"2023-10-09T06:37:46.160048+0000","last_unstale":"2023-10-09T06:37:47.184592+0000","last_undegraded":"2023-10-09T06:37:47.184592+0000","last_fullsized":"2023-10-09T06:37:47.184592+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:53:17.680797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.184017+0000","last_change":"2023-10-09T06:37:47.176374+0000","last_active":"2023-10-09T06:37:47.184017+0000","last_peered":"2023-10-09T06:37:47.184017+0000","last_clean":"2023-10-09T06:37:47.184017+0000","last_became_active":"2023-10-09T06:37:47.176091+0000","last_became_peered":"2023-10-09T06:37:47.176091+0000","last_unstale":"2023-10-09T06:37:47.184017+0000","last_undegraded":"2023-10-09T06:37:47.184017+0000","last_fullsized":"2023-10-09T06:37:47.184017+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:26:39.377444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"136'2","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.185061+0000","last_change":"2023-10-09T06:37:46.158542+0000","last_active":"2023-10-09T06:37:47.185061+0000","last_peered":"2023-10-09T06:37:47.185061+0000","last_clean":"2023-10-09T06:37:47.185061+0000","last_became_active":"2023-10-09T06:37:46.158244+0000","last_became_peered":"2023-10-09T06:37:46.158244+0000","last_unstale":"2023-10-09T06:37:47.185061+0000","last_undegraded":"2023-10-09T06:37:47.185061+0000","last_fullsized":"2023-10-09T06:37:47.185061+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:01:22.251808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.181431+0000","last_change":"2023-10-09T06:37:47.177058+0000","last_active":"2023-10-09T06:37:47.181431+0000","last_peered":"2023-10-09T06:37:47.181431+0000","last_clean":"2023-10-09T06:37:47.181431+0000","last_became_active":"2023-10-09T06:37:47.176789+0000","last_became_peered":"2023-10-09T06:37:47.176789+0000","last_unstale":"2023-10-09T06:37:47.181431+0000","last_undegraded":"2023-10-09T06:37:47.181431+0000","last_fullsized":"2023-10-09T06:37:47.181431+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:49:37.191442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.187985+0000","last_change":"2023-10-09T06:37:46.165933+0000","last_active":"2023-10-09T06:37:47.187985+0000","last_peered":"2023-10-09T06:37:47.187985+0000","last_clean":"2023-10-09T06:37:47.187985+0000","last_became_active":"2023-10-09T06:37:46.165686+0000","last_became_peered":"2023-10-09T06:37:46.165686+0000","last_unstale":"2023-10-09T06:37:47.187985+0000","last_undegraded":"2023-10-09T06:37:47.187985+0000","last_fullsized":"2023-10-09T06:37:47.187985+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:15:10.830012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.178161+0000","last_change":"2023-10-09T06:37:47.174344+0000","last_active":"2023-10-09T06:37:47.178161+0000","last_peered":"2023-10-09T06:37:47.178161+0000","last_clean":"2023-10-09T06:37:47.178161+0000","last_became_active":"2023-10-09T06:37:47.174096+0000","last_became_peered":"2023-10-09T06:37:47.174096+0000","last_unstale":"2023-10-09T06:37:47.178161+0000","last_undegraded":"2023-10-09T06:37:47.178161+0000","last_fullsized":"2023-10-09T06:37:47.178161+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:25:23.569565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.186181+0000","last_change":"2023-10-09T06:37:46.162339+0000","last_active":"2023-10-09T06:37:47.186181+0000","last_peered":"2023-10-09T06:37:47.186181+0000","last_clean":"2023-10-09T06:37:47.186181+0000","last_became_active":"2023-10-09T06:37:46.162087+0000","last_became_peered":"2023-10-09T06:37:46.162087+0000","last_unstale":"2023-10-09T06:37:47.186181+0000","last_undegraded":"2023-10-09T06:37:47.186181+0000","last_fullsized":"2023-10-09T06:37:47.186181+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:17:47.143130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.180557+0000","last_change":"2023-10-09T06:37:47.175964+0000","last_active":"2023-10-09T06:37:47.180557+0000","last_peered":"2023-10-09T06:37:47.180557+0000","last_clean":"2023-10-09T06:37:47.180557+0000","last_became_active":"2023-10-09T06:37:47.175682+0000","last_became_peered":"2023-10-09T06:37:47.175682+0000","last_unstale":"2023-10-09T06:37:47.180557+0000","last_undegraded":"2023-10-09T06:37:47.180557+0000","last_fullsized":"2023-10-09T06:37:47.180557+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:12:16.269071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.166048+0000","last_change":"2023-10-09T06:37:46.166420+0000","last_active":"2023-10-09T06:37:47.166048+0000","last_peered":"2023-10-09T06:37:47.166048+0000","last_clean":"2023-10-09T06:37:47.166048+0000","last_became_active":"2023-10-09T06:37:46.166212+0000","last_became_peered":"2023-10-09T06:37:46.166212+0000","last_unstale":"2023-10-09T06:37:47.166048+0000","last_undegraded":"2023-10-09T06:37:47.166048+0000","last_fullsized":"2023-10-09T06:37:47.166048+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:48:22.393545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.181823+0000","last_change":"2023-10-09T06:37:47.176452+0000","last_active":"2023-10-09T06:37:47.181823+0000","last_peered":"2023-10-09T06:37:47.181823+0000","last_clean":"2023-10-09T06:37:47.181823+0000","last_became_active":"2023-10-09T06:37:47.176177+0000","last_became_peered":"2023-10-09T06:37:47.176177+0000","last_unstale":"2023-10-09T06:37:47.181823+0000","last_undegraded":"2023-10-09T06:37:47.181823+0000","last_fullsized":"2023-10-09T06:37:47.181823+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:37:25.126126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.164669+0000","last_change":"2023-10-09T06:37:46.156627+0000","last_active":"2023-10-09T06:37:47.164669+0000","last_peered":"2023-10-09T06:37:47.164669+0000","last_clean":"2023-10-09T06:37:47.164669+0000","last_became_active":"2023-10-09T06:37:46.156313+0000","last_became_peered":"2023-10-09T06:37:46.156313+0000","last_unstale":"2023-10-09T06:37:47.164669+0000","last_undegraded":"2023-10-09T06:37:47.164669+0000","last_fullsized":"2023-10-09T06:37:47.164669+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:39:39.850042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.177799+0000","last_change":"2023-10-09T06:37:47.173116+0000","last_active":"2023-10-09T06:37:47.177799+0000","last_peered":"2023-10-09T06:37:47.177799+0000","last_clean":"2023-10-09T06:37:47.177799+0000","last_became_active":"2023-10-09T06:37:47.172862+0000","last_became_peered":"2023-10-09T06:37:47.172862+0000","last_unstale":"2023-10-09T06:37:47.177799+0000","last_undegraded":"2023-10-09T06:37:47.177799+0000","last_fullsized":"2023-10-09T06:37:47.177799+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:49:25.010643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.168773+0000","last_change":"2023-10-09T06:37:46.157544+0000","last_active":"2023-10-09T06:37:47.168773+0000","last_peered":"2023-10-09T06:37:47.168773+0000","last_clean":"2023-10-09T06:37:47.168773+0000","last_became_active":"2023-10-09T06:37:46.157300+0000","last_became_peered":"2023-10-09T06:37:46.157300+0000","last_unstale":"2023-10-09T06:37:47.168773+0000","last_undegraded":"2023-10-09T06:37:47.168773+0000","last_fullsized":"2023-10-09T06:37:47.168773+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:35:20.412002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.180560+0000","last_change":"2023-10-09T06:37:47.176790+0000","last_active":"2023-10-09T06:37:47.180560+0000","last_peered":"2023-10-09T06:37:47.180560+0000","last_clean":"2023-10-09T06:37:47.180560+0000","last_became_active":"2023-10-09T06:37:47.176119+0000","last_became_peered":"2023-10-09T06:37:47.176119+0000","last_unstale":"2023-10-09T06:37:47.180560+0000","last_undegraded":"2023-10-09T06:37:47.180560+0000","last_fullsized":"2023-10-09T06:37:47.180560+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:36:33.812346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"136'3","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.188047+0000","last_change":"2023-10-09T06:37:46.163808+0000","last_active":"2023-10-09T06:37:47.188047+0000","last_peered":"2023-10-09T06:37:47.188047+0000","last_clean":"2023-10-09T06:37:47.188047+0000","last_became_active":"2023-10-09T06:37:46.163515+0000","last_became_peered":"2023-10-09T06:37:46.163515+0000","last_unstale":"2023-10-09T06:37:47.188047+0000","last_undegraded":"2023-10-09T06:37:47.188047+0000","last_fullsized":"2023-10-09T06:37:47.188047+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:11:07.149616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.181444+0000","last_change":"2023-10-09T06:37:47.176956+0000","last_active":"2023-10-09T06:37:47.181444+0000","last_peered":"2023-10-09T06:37:47.181444+0000","last_clean":"2023-10-09T06:37:47.181444+0000","last_became_active":"2023-10-09T06:37:47.176729+0000","last_became_peered":"2023-10-09T06:37:47.176729+0000","last_unstale":"2023-10-09T06:37:47.181444+0000","last_undegraded":"2023-10-09T06:37:47.181444+0000","last_fullsized":"2023-10-09T06:37:47.181444+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:16:00.779605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"136'6","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.189418+0000","last_change":"2023-10-09T06:37:46.157324+0000","last_active":"2023-10-09T06:37:47.189418+0000","last_peered":"2023-10-09T06:37:47.189418+0000","last_clean":"2023-10-09T06:37:47.189418+0000","last_became_active":"2023-10-09T06:37:46.157046+0000","last_became_peered":"2023-10-09T06:37:46.157046+0000","last_unstale":"2023-10-09T06:37:47.189418+0000","last_undegraded":"2023-10-09T06:37:47.189418+0000","last_fullsized":"2023-10-09T06:37:47.189418+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:49:26.936922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.175688+0000","last_change":"2023-10-09T06:37:47.171370+0000","last_active":"2023-10-09T06:37:47.175688+0000","last_peered":"2023-10-09T06:37:47.175688+0000","last_clean":"2023-10-09T06:37:47.175688+0000","last_became_active":"2023-10-09T06:37:47.171044+0000","last_became_peered":"2023-10-09T06:37:47.171044+0000","last_unstale":"2023-10-09T06:37:47.175688+0000","last_undegraded":"2023-10-09T06:37:47.175688+0000","last_fullsized":"2023-10-09T06:37:47.175688+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:08:34.898844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"136'2","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.187120+0000","last_change":"2023-10-09T06:37:46.165381+0000","last_active":"2023-10-09T06:37:47.187120+0000","last_peered":"2023-10-09T06:37:47.187120+0000","last_clean":"2023-10-09T06:37:47.187120+0000","last_became_active":"2023-10-09T06:37:46.165138+0000","last_became_peered":"2023-10-09T06:37:46.165138+0000","last_unstale":"2023-10-09T06:37:47.187120+0000","last_undegraded":"2023-10-09T06:37:47.187120+0000","last_fullsized":"2023-10-09T06:37:47.187120+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:31:32.310904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.171604+0000","last_change":"2023-10-09T06:37:47.167805+0000","last_active":"2023-10-09T06:37:47.171604+0000","last_peered":"2023-10-09T06:37:47.171604+0000","last_clean":"2023-10-09T06:37:47.171604+0000","last_became_active":"2023-10-09T06:37:47.167541+0000","last_became_peered":"2023-10-09T06:37:47.167541+0000","last_unstale":"2023-10-09T06:37:47.171604+0000","last_undegraded":"2023-10-09T06:37:47.171604+0000","last_fullsized":"2023-10-09T06:37:47.171604+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:42:32.915391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.165457+0000","last_change":"2023-10-09T06:37:46.160443+0000","last_active":"2023-10-09T06:37:47.165457+0000","last_peered":"2023-10-09T06:37:47.165457+0000","last_clean":"2023-10-09T06:37:47.165457+0000","last_became_active":"2023-10-09T06:37:46.160187+0000","last_became_peered":"2023-10-09T06:37:46.160187+0000","last_unstale":"2023-10-09T06:37:47.165457+0000","last_undegraded":"2023-10-09T06:37:47.165457+0000","last_fullsized":"2023-10-09T06:37:47.165457+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:53:17.680797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.178980+0000","last_change":"2023-10-09T06:37:47.174266+0000","last_active":"2023-10-09T06:37:47.178980+0000","last_peered":"2023-10-09T06:37:47.178980+0000","last_clean":"2023-10-09T06:37:47.178980+0000","last_became_active":"2023-10-09T06:37:47.173887+0000","last_became_peered":"2023-10-09T06:37:47.173887+0000","last_unstale":"2023-10-09T06:37:47.178980+0000","last_undegraded":"2023-10-09T06:37:47.178980+0000","last_fullsized":"2023-10-09T06:37:47.178980+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:39:40.857018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.162913+0000","last_change":"2023-10-09T06:37:46.154153+0000","last_active":"2023-10-09T06:37:47.162913+0000","last_peered":"2023-10-09T06:37:47.162913+0000","last_clean":"2023-10-09T06:37:47.162913+0000","last_became_active":"2023-10-09T06:37:46.153874+0000","last_became_peered":"2023-10-09T06:37:46.153874+0000","last_unstale":"2023-10-09T06:37:47.162913+0000","last_undegraded":"2023-10-09T06:37:47.162913+0000","last_fullsized":"2023-10-09T06:37:47.162913+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:07:08.017060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.181250+0000","last_change":"2023-10-09T06:37:47.175380+0000","last_active":"2023-10-09T06:37:47.181250+0000","last_peered":"2023-10-09T06:37:47.181250+0000","last_clean":"2023-10-09T06:37:47.181250+0000","last_became_active":"2023-10-09T06:37:47.173989+0000","last_became_peered":"2023-10-09T06:37:47.173989+0000","last_unstale":"2023-10-09T06:37:47.181250+0000","last_undegraded":"2023-10-09T06:37:47.181250+0000","last_fullsized":"2023-10-09T06:37:47.181250+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:49:18.350262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"136'8","reported_seq":18,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.187541+0000","last_change":"2023-10-09T06:37:46.166498+0000","last_active":"2023-10-09T06:37:47.187541+0000","last_peered":"2023-10-09T06:37:47.187541+0000","last_clean":"2023-10-09T06:37:47.187541+0000","last_became_active":"2023-10-09T06:37:46.166270+0000","last_became_peered":"2023-10-09T06:37:46.166270+0000","last_unstale":"2023-10-09T06:37:47.187541+0000","last_undegraded":"2023-10-09T06:37:47.187541+0000","last_fullsized":"2023-10-09T06:37:47.187541+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:55:12.397428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.178302+0000","last_change":"2023-10-09T06:37:47.172384+0000","last_active":"2023-10-09T06:37:47.178302+0000","last_peered":"2023-10-09T06:37:47.178302+0000","last_clean":"2023-10-09T06:37:47.178302+0000","last_became_active":"2023-10-09T06:37:47.172113+0000","last_became_peered":"2023-10-09T06:37:47.172113+0000","last_unstale":"2023-10-09T06:37:47.178302+0000","last_undegraded":"2023-10-09T06:37:47.178302+0000","last_fullsized":"2023-10-09T06:37:47.178302+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:20:36.794582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.188181+0000","last_change":"2023-10-09T06:37:46.160269+0000","last_active":"2023-10-09T06:37:47.188181+0000","last_peered":"2023-10-09T06:37:47.188181+0000","last_clean":"2023-10-09T06:37:47.188181+0000","last_became_active":"2023-10-09T06:37:46.160041+0000","last_became_peered":"2023-10-09T06:37:46.160041+0000","last_unstale":"2023-10-09T06:37:47.188181+0000","last_undegraded":"2023-10-09T06:37:47.188181+0000","last_fullsized":"2023-10-09T06:37:47.188181+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:18:32.993627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.179995+0000","last_change":"2023-10-09T06:37:47.175110+0000","last_active":"2023-10-09T06:37:47.179995+0000","last_peered":"2023-10-09T06:37:47.179995+0000","last_clean":"2023-10-09T06:37:47.179995+0000","last_became_active":"2023-10-09T06:37:47.174614+0000","last_became_peered":"2023-10-09T06:37:47.174614+0000","last_unstale":"2023-10-09T06:37:47.179995+0000","last_undegraded":"2023-10-09T06:37:47.179995+0000","last_fullsized":"2023-10-09T06:37:47.179995+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:45:22.508406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"136'2","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.185583+0000","last_change":"2023-10-09T06:37:46.155690+0000","last_active":"2023-10-09T06:37:47.185583+0000","last_peered":"2023-10-09T06:37:47.185583+0000","last_clean":"2023-10-09T06:37:47.185583+0000","last_became_active":"2023-10-09T06:37:46.155411+0000","last_became_peered":"2023-10-09T06:37:46.155411+0000","last_unstale":"2023-10-09T06:37:47.185583+0000","last_undegraded":"2023-10-09T06:37:47.185583+0000","last_fullsized":"2023-10-09T06:37:47.185583+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:48:22.393545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.184302+0000","last_change":"2023-10-09T06:37:47.180699+0000","last_active":"2023-10-09T06:37:47.184302+0000","last_peered":"2023-10-09T06:37:47.184302+0000","last_clean":"2023-10-09T06:37:47.184302+0000","last_became_active":"2023-10-09T06:37:47.180421+0000","last_became_peered":"2023-10-09T06:37:47.180421+0000","last_unstale":"2023-10-09T06:37:47.184302+0000","last_undegraded":"2023-10-09T06:37:47.184302+0000","last_fullsized":"2023-10-09T06:37:47.184302+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:48:25.266457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.185915+0000","last_change":"2023-10-09T06:37:46.166713+0000","last_active":"2023-10-09T06:37:47.185915+0000","last_peered":"2023-10-09T06:37:47.185915+0000","last_clean":"2023-10-09T06:37:47.185915+0000","last_became_active":"2023-10-09T06:37:46.166490+0000","last_became_peered":"2023-10-09T06:37:46.166490+0000","last_unstale":"2023-10-09T06:37:47.185915+0000","last_undegraded":"2023-10-09T06:37:47.185915+0000","last_fullsized":"2023-10-09T06:37:47.185915+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:07:08.017060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.174796+0000","last_change":"2023-10-09T06:37:47.172447+0000","last_active":"2023-10-09T06:37:47.174796+0000","last_peered":"2023-10-09T06:37:47.174796+0000","last_clean":"2023-10-09T06:37:47.174796+0000","last_became_active":"2023-10-09T06:37:47.172165+0000","last_became_peered":"2023-10-09T06:37:47.172165+0000","last_unstale":"2023-10-09T06:37:47.174796+0000","last_undegraded":"2023-10-09T06:37:47.174796+0000","last_fullsized":"2023-10-09T06:37:47.174796+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:30:11.937291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.164417+0000","last_change":"2023-10-09T06:37:46.160756+0000","last_active":"2023-10-09T06:37:47.164417+0000","last_peered":"2023-10-09T06:37:47.164417+0000","last_clean":"2023-10-09T06:37:47.164417+0000","last_became_active":"2023-10-09T06:37:46.160492+0000","last_became_peered":"2023-10-09T06:37:46.160492+0000","last_unstale":"2023-10-09T06:37:47.164417+0000","last_undegraded":"2023-10-09T06:37:47.164417+0000","last_fullsized":"2023-10-09T06:37:47.164417+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:03:11.419579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.174987+0000","last_change":"2023-10-09T06:37:47.169100+0000","last_active":"2023-10-09T06:37:47.174987+0000","last_peered":"2023-10-09T06:37:47.174987+0000","last_clean":"2023-10-09T06:37:47.174987+0000","last_became_active":"2023-10-09T06:37:47.168860+0000","last_became_peered":"2023-10-09T06:37:47.168860+0000","last_unstale":"2023-10-09T06:37:47.174987+0000","last_undegraded":"2023-10-09T06:37:47.174987+0000","last_fullsized":"2023-10-09T06:37:47.174987+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:49:54.766563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"136'2","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.187817+0000","last_change":"2023-10-09T06:37:46.161231+0000","last_active":"2023-10-09T06:37:47.187817+0000","last_peered":"2023-10-09T06:37:47.187817+0000","last_clean":"2023-10-09T06:37:47.187817+0000","last_became_active":"2023-10-09T06:37:46.160938+0000","last_became_peered":"2023-10-09T06:37:46.160938+0000","last_unstale":"2023-10-09T06:37:47.187817+0000","last_undegraded":"2023-10-09T06:37:47.187817+0000","last_fullsized":"2023-10-09T06:37:47.187817+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:34:05.362873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.181175+0000","last_change":"2023-10-09T06:37:47.177819+0000","last_active":"2023-10-09T06:37:47.181175+0000","last_peered":"2023-10-09T06:37:47.181175+0000","last_clean":"2023-10-09T06:37:47.181175+0000","last_became_active":"2023-10-09T06:37:47.176509+0000","last_became_peered":"2023-10-09T06:37:47.176509+0000","last_unstale":"2023-10-09T06:37:47.181175+0000","last_undegraded":"2023-10-09T06:37:47.181175+0000","last_fullsized":"2023-10-09T06:37:47.181175+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:31:58.529087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.168299+0000","last_change":"2023-10-09T06:37:46.159376+0000","last_active":"2023-10-09T06:37:47.168299+0000","last_peered":"2023-10-09T06:37:47.168299+0000","last_clean":"2023-10-09T06:37:47.168299+0000","last_became_active":"2023-10-09T06:37:46.159093+0000","last_became_peered":"2023-10-09T06:37:46.159093+0000","last_unstale":"2023-10-09T06:37:47.168299+0000","last_undegraded":"2023-10-09T06:37:47.168299+0000","last_fullsized":"2023-10-09T06:37:47.168299+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:04:38.242742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.180813+0000","last_change":"2023-10-09T06:37:47.178651+0000","last_active":"2023-10-09T06:37:47.180813+0000","last_peered":"2023-10-09T06:37:47.180813+0000","last_clean":"2023-10-09T06:37:47.180813+0000","last_became_active":"2023-10-09T06:37:47.178301+0000","last_became_peered":"2023-10-09T06:37:47.178301+0000","last_unstale":"2023-10-09T06:37:47.180813+0000","last_undegraded":"2023-10-09T06:37:47.180813+0000","last_fullsized":"2023-10-09T06:37:47.180813+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:45:22.508406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.159816+0000","last_change":"2023-10-09T06:37:46.161637+0000","last_active":"2023-10-09T06:37:47.159816+0000","last_peered":"2023-10-09T06:37:47.159816+0000","last_clean":"2023-10-09T06:37:47.159816+0000","last_became_active":"2023-10-09T06:37:46.161327+0000","last_became_peered":"2023-10-09T06:37:46.161327+0000","last_unstale":"2023-10-09T06:37:47.159816+0000","last_undegraded":"2023-10-09T06:37:47.159816+0000","last_fullsized":"2023-10-09T06:37:47.159816+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:00:51.883913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.180692+0000","last_change":"2023-10-09T06:37:47.172441+0000","last_active":"2023-10-09T06:37:47.180692+0000","last_peered":"2023-10-09T06:37:47.180692+0000","last_clean":"2023-10-09T06:37:47.180692+0000","last_became_active":"2023-10-09T06:37:47.172195+0000","last_became_peered":"2023-10-09T06:37:47.172195+0000","last_unstale":"2023-10-09T06:37:47.180692+0000","last_undegraded":"2023-10-09T06:37:47.180692+0000","last_fullsized":"2023-10-09T06:37:47.180692+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:09:28.407433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.165963+0000","last_change":"2023-10-09T06:37:46.152589+0000","last_active":"2023-10-09T06:37:47.165963+0000","last_peered":"2023-10-09T06:37:47.165963+0000","last_clean":"2023-10-09T06:37:47.165963+0000","last_became_active":"2023-10-09T06:37:46.152321+0000","last_became_peered":"2023-10-09T06:37:46.152321+0000","last_unstale":"2023-10-09T06:37:47.165963+0000","last_undegraded":"2023-10-09T06:37:47.165963+0000","last_fullsized":"2023-10-09T06:37:47.165963+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:56:59.799505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.180643+0000","last_change":"2023-10-09T06:37:47.171644+0000","last_active":"2023-10-09T06:37:47.180643+0000","last_peered":"2023-10-09T06:37:47.180643+0000","last_clean":"2023-10-09T06:37:47.180643+0000","last_became_active":"2023-10-09T06:37:47.171443+0000","last_became_peered":"2023-10-09T06:37:47.171443+0000","last_unstale":"2023-10-09T06:37:47.180643+0000","last_undegraded":"2023-10-09T06:37:47.180643+0000","last_fullsized":"2023-10-09T06:37:47.180643+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:33:37.869895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"136'2","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.188359+0000","last_change":"2023-10-09T06:37:46.161611+0000","last_active":"2023-10-09T06:37:47.188359+0000","last_peered":"2023-10-09T06:37:47.188359+0000","last_clean":"2023-10-09T06:37:47.188359+0000","last_became_active":"2023-10-09T06:37:46.161432+0000","last_became_peered":"2023-10-09T06:37:46.161432+0000","last_unstale":"2023-10-09T06:37:47.188359+0000","last_undegraded":"2023-10-09T06:37:47.188359+0000","last_fullsized":"2023-10-09T06:37:47.188359+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:20:25.547981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.181923+0000","last_change":"2023-10-09T06:37:47.177207+0000","last_active":"2023-10-09T06:37:47.181923+0000","last_peered":"2023-10-09T06:37:47.181923+0000","last_clean":"2023-10-09T06:37:47.181923+0000","last_became_active":"2023-10-09T06:37:47.176958+0000","last_became_peered":"2023-10-09T06:37:47.176958+0000","last_unstale":"2023-10-09T06:37:47.181923+0000","last_undegraded":"2023-10-09T06:37:47.181923+0000","last_fullsized":"2023-10-09T06:37:47.181923+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:59:24.334461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.164764+0000","last_change":"2023-10-09T06:37:46.165227+0000","last_active":"2023-10-09T06:37:47.164764+0000","last_peered":"2023-10-09T06:37:47.164764+0000","last_clean":"2023-10-09T06:37:47.164764+0000","last_became_active":"2023-10-09T06:37:46.164959+0000","last_became_peered":"2023-10-09T06:37:46.164959+0000","last_unstale":"2023-10-09T06:37:47.164764+0000","last_undegraded":"2023-10-09T06:37:47.164764+0000","last_fullsized":"2023-10-09T06:37:47.164764+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:59:31.649021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.182525+0000","last_change":"2023-10-09T06:37:47.176879+0000","last_active":"2023-10-09T06:37:47.182525+0000","last_peered":"2023-10-09T06:37:47.182525+0000","last_clean":"2023-10-09T06:37:47.182525+0000","last_became_active":"2023-10-09T06:37:47.176496+0000","last_became_peered":"2023-10-09T06:37:47.176496+0000","last_unstale":"2023-10-09T06:37:47.182525+0000","last_undegraded":"2023-10-09T06:37:47.182525+0000","last_fullsized":"2023-10-09T06:37:47.182525+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:23:25.125922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.162819+0000","last_change":"2023-10-09T06:37:46.165805+0000","last_active":"2023-10-09T06:37:47.162819+0000","last_peered":"2023-10-09T06:37:47.162819+0000","last_clean":"2023-10-09T06:37:47.162819+0000","last_became_active":"2023-10-09T06:37:46.165574+0000","last_became_peered":"2023-10-09T06:37:46.165574+0000","last_unstale":"2023-10-09T06:37:47.162819+0000","last_undegraded":"2023-10-09T06:37:47.162819+0000","last_fullsized":"2023-10-09T06:37:47.162819+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:04:18.850838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.185471+0000","last_change":"2023-10-09T06:37:47.178939+0000","last_active":"2023-10-09T06:37:47.185471+0000","last_peered":"2023-10-09T06:37:47.185471+0000","last_clean":"2023-10-09T06:37:47.185471+0000","last_became_active":"2023-10-09T06:37:47.178711+0000","last_became_peered":"2023-10-09T06:37:47.178711+0000","last_unstale":"2023-10-09T06:37:47.185471+0000","last_undegraded":"2023-10-09T06:37:47.185471+0000","last_fullsized":"2023-10-09T06:37:47.185471+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:49:37.191442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.163047+0000","last_change":"2023-10-09T06:37:46.159095+0000","last_active":"2023-10-09T06:37:47.163047+0000","last_peered":"2023-10-09T06:37:47.163047+0000","last_clean":"2023-10-09T06:37:47.163047+0000","last_became_active":"2023-10-09T06:37:46.158810+0000","last_became_peered":"2023-10-09T06:37:46.158810+0000","last_unstale":"2023-10-09T06:37:47.163047+0000","last_undegraded":"2023-10-09T06:37:47.163047+0000","last_fullsized":"2023-10-09T06:37:47.163047+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:20:25.547981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.183308+0000","last_change":"2023-10-09T06:37:47.178617+0000","last_active":"2023-10-09T06:37:47.183308+0000","last_peered":"2023-10-09T06:37:47.183308+0000","last_clean":"2023-10-09T06:37:47.183308+0000","last_became_active":"2023-10-09T06:37:47.178364+0000","last_became_peered":"2023-10-09T06:37:47.178364+0000","last_unstale":"2023-10-09T06:37:47.183308+0000","last_undegraded":"2023-10-09T06:37:47.183308+0000","last_fullsized":"2023-10-09T06:37:47.183308+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:49:18.350262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.165899+0000","last_change":"2023-10-09T06:37:46.158783+0000","last_active":"2023-10-09T06:37:47.165899+0000","last_peered":"2023-10-09T06:37:47.165899+0000","last_clean":"2023-10-09T06:37:47.165899+0000","last_became_active":"2023-10-09T06:37:46.158466+0000","last_became_peered":"2023-10-09T06:37:46.158466+0000","last_unstale":"2023-10-09T06:37:47.165899+0000","last_undegraded":"2023-10-09T06:37:47.165899+0000","last_fullsized":"2023-10-09T06:37:47.165899+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:55:08.369151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.177809+0000","last_change":"2023-10-09T06:37:47.175805+0000","last_active":"2023-10-09T06:37:47.177809+0000","last_peered":"2023-10-09T06:37:47.177809+0000","last_clean":"2023-10-09T06:37:47.177809+0000","last_became_active":"2023-10-09T06:37:47.175551+0000","last_became_peered":"2023-10-09T06:37:47.175551+0000","last_unstale":"2023-10-09T06:37:47.177809+0000","last_undegraded":"2023-10-09T06:37:47.177809+0000","last_fullsized":"2023-10-09T06:37:47.177809+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:14:48.221558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.165629+0000","last_change":"2023-10-09T06:37:46.154894+0000","last_active":"2023-10-09T06:37:47.165629+0000","last_peered":"2023-10-09T06:37:47.165629+0000","last_clean":"2023-10-09T06:37:47.165629+0000","last_became_active":"2023-10-09T06:37:46.154589+0000","last_became_peered":"2023-10-09T06:37:46.154589+0000","last_unstale":"2023-10-09T06:37:47.165629+0000","last_undegraded":"2023-10-09T06:37:47.165629+0000","last_fullsized":"2023-10-09T06:37:47.165629+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:58:53.461920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.172308+0000","last_change":"2023-10-09T06:37:47.168531+0000","last_active":"2023-10-09T06:37:47.172308+0000","last_peered":"2023-10-09T06:37:47.172308+0000","last_clean":"2023-10-09T06:37:47.172308+0000","last_became_active":"2023-10-09T06:37:47.168206+0000","last_became_peered":"2023-10-09T06:37:47.168206+0000","last_unstale":"2023-10-09T06:37:47.172308+0000","last_undegraded":"2023-10-09T06:37:47.172308+0000","last_fullsized":"2023-10-09T06:37:47.172308+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:22:24.566358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.163873+0000","last_change":"2023-10-09T06:37:46.152044+0000","last_active":"2023-10-09T06:37:47.163873+0000","last_peered":"2023-10-09T06:37:47.163873+0000","last_clean":"2023-10-09T06:37:47.163873+0000","last_became_active":"2023-10-09T06:37:46.151315+0000","last_became_peered":"2023-10-09T06:37:46.151315+0000","last_unstale":"2023-10-09T06:37:47.163873+0000","last_undegraded":"2023-10-09T06:37:47.163873+0000","last_fullsized":"2023-10-09T06:37:47.163873+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:24:27.771345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.180710+0000","last_change":"2023-10-09T06:37:47.173604+0000","last_active":"2023-10-09T06:37:47.180710+0000","last_peered":"2023-10-09T06:37:47.180710+0000","last_clean":"2023-10-09T06:37:47.180710+0000","last_became_active":"2023-10-09T06:37:47.173322+0000","last_became_peered":"2023-10-09T06:37:47.173322+0000","last_unstale":"2023-10-09T06:37:47.180710+0000","last_undegraded":"2023-10-09T06:37:47.180710+0000","last_fullsized":"2023-10-09T06:37:47.180710+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:35:39.385390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.159303+0000","last_change":"2023-10-09T06:37:46.159703+0000","last_active":"2023-10-09T06:37:47.159303+0000","last_peered":"2023-10-09T06:37:47.159303+0000","last_clean":"2023-10-09T06:37:47.159303+0000","last_became_active":"2023-10-09T06:37:46.159455+0000","last_became_peered":"2023-10-09T06:37:46.159455+0000","last_unstale":"2023-10-09T06:37:47.159303+0000","last_undegraded":"2023-10-09T06:37:47.159303+0000","last_fullsized":"2023-10-09T06:37:47.159303+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:19:19.201484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.178102+0000","last_change":"2023-10-09T06:37:47.170391+0000","last_active":"2023-10-09T06:37:47.178102+0000","last_peered":"2023-10-09T06:37:47.178102+0000","last_clean":"2023-10-09T06:37:47.178102+0000","last_became_active":"2023-10-09T06:37:47.170131+0000","last_became_peered":"2023-10-09T06:37:47.170131+0000","last_unstale":"2023-10-09T06:37:47.178102+0000","last_undegraded":"2023-10-09T06:37:47.178102+0000","last_fullsized":"2023-10-09T06:37:47.178102+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:25:23.569565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.171753+0000","last_change":"2023-10-09T06:37:46.159778+0000","last_active":"2023-10-09T06:37:47.171753+0000","last_peered":"2023-10-09T06:37:47.171753+0000","last_clean":"2023-10-09T06:37:47.171753+0000","last_became_active":"2023-10-09T06:37:46.159520+0000","last_became_peered":"2023-10-09T06:37:46.159520+0000","last_unstale":"2023-10-09T06:37:47.171753+0000","last_undegraded":"2023-10-09T06:37:47.171753+0000","last_fullsized":"2023-10-09T06:37:47.171753+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:04:18.850838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":259,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.166310+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:37:47.166310+0000","last_peered":"2023-10-09T06:37:47.166310+0000","last_clean":"2023-10-09T06:37:47.166310+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:37:47.166310+0000","last_undegraded":"2023-10-09T06:37:47.166310+0000","last_fullsized":"2023-10-09T06:37:47.166310+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574940,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303664,"kb_used_data":2280,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068176,"statfs":{"total":96636764160,"available":96325812224,"internally_reserved":0,"allocated":2334720,"data_stored":1363746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52000000000000002,"15min":0.50700000000000001},"min":{"1min":0.34799999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.37,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67600000000000005,"5min":0.753,"15min":0.753},"last":0.438}]},{"osd":1,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.503,"15min":0.51900000000000002},"min":{"1min":0.33600000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.74299999999999999,"5min":0.77400000000000002,"15min":1.2030000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.378,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.55100000000000005}]},{"osd":2,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53200000000000003,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75800000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.44,"5min":0.36399999999999999,"15min":0.315},"max":{"1min":0.77900000000000003,"5min":0.80700000000000005,"15min":0.83099999999999996},"last":0.48099999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53800000000000003,"15min":0.55000000000000004},"min":{"1min":0.41199999999999998,"5min":0.35899999999999999,"15min":0.33500000000000002},"max":{"1min":0.68600000000000005,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55600000000000005,"15min":0.59199999999999997},"min":{"1min":0.45300000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.754,"5min":0.82099999999999995,"15min":1.1479999999999999},"last":0.53500000000000003}]},{"osd":4,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.48199999999999998,"15min":0.51900000000000002},"min":{"1min":0.35599999999999998,"5min":0.30299999999999999,"15min":0.24399999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.93999999999999995},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56599999999999995,"15min":0.60599999999999998},"min":{"1min":0.36799999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76700000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.69099999999999995}]},{"osd":5,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52500000000000002,"15min":0.56999999999999995},"min":{"1min":0.36499999999999999,"5min":0.32500000000000001,"15min":0.307},"max":{"1min":0.81999999999999995,"5min":0.85799999999999998,"15min":0.91100000000000003},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59599999999999997,"15min":0.62},"min":{"1min":0.52900000000000003,"5min":0.41699999999999998,"15min":0.34200000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.71199999999999997}]},{"osd":6,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55700000000000005,"15min":0.59299999999999997},"min":{"1min":0.38600000000000001,"5min":0.33500000000000002,"15min":0.28699999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.88600000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.56100000000000005,"15min":0.61399999999999999},"min":{"1min":0.45100000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.84599999999999997},"last":0.66100000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574939,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303076,"kb_used_data":1692,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068764,"statfs":{"total":96636764160,"available":96326414336,"internally_reserved":0,"allocated":1732608,"data_stored":773274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.50800000000000001,"15min":0.502},"min":{"1min":0.35499999999999998,"5min":0.32000000000000001,"15min":0.30399999999999999},"max":{"1min":0.66500000000000004,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.371},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.50700000000000001,"15min":0.51000000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70399999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.49299999999999999}]},{"osd":1,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.502,"15min":0.51200000000000001},"min":{"1min":0.33600000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68300000000000005,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52900000000000003,"15min":0.53200000000000003},"min":{"1min":0.38700000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.77100000000000002,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.437}]},{"osd":2,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.34499999999999997,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.72399999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51100000000000001,"15min":0.52900000000000003},"min":{"1min":0.39000000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.66600000000000004,"5min":0.67500000000000004,"15min":0.752},"last":0.40100000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.38900000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66600000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.54300000000000004,"15min":0.68799999999999994},"min":{"1min":0.40400000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.752,"5min":0.752,"15min":4.7510000000000003},"last":0.41599999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46400000000000002,"15min":0.47699999999999998},"min":{"1min":0.29899999999999999,"5min":0.28199999999999997,"15min":0.27900000000000003},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.76400000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56100000000000005,"15min":0.58099999999999996},"min":{"1min":0.41799999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.77200000000000002,"5min":0.81299999999999994,"15min":0.83699999999999997},"last":0.58999999999999997}]},{"osd":5,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.52100000000000002,"15min":0.53000000000000003},"min":{"1min":0.31900000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.66100000000000003,"5min":0.79200000000000004,"15min":0.81200000000000006},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.59499999999999997},"min":{"1min":0.46899999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76000000000000001,"5min":0.78000000000000003,"15min":0.86599999999999999},"last":0.54300000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55200000000000005,"15min":0.58199999999999996},"min":{"1min":0.41399999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80500000000000005,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58799999999999997,"15min":0.58899999999999997},"min":{"1min":0.46000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78400000000000003,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.55400000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303088,"kb_used_data":1704,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068752,"statfs":{"total":96636764160,"available":96326402048,"internally_reserved":0,"allocated":1744896,"data_stored":774810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.316,"15min":0.311},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.503,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.309},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.34899999999999998,"15min":0.35799999999999998},"min":{"1min":0.28399999999999997,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.51900000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.31900000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.34399999999999997,"15min":0.35099999999999998},"min":{"1min":0.215,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.53700000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.38300000000000001,"15min":0.38500000000000001},"min":{"1min":0.30299999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.55300000000000005,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.34399999999999997}]},{"osd":3,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.36499999999999999,"15min":0.36399999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.55800000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.34599999999999997,"15min":0.53500000000000003},"min":{"1min":0.23400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.40400000000000003,"5min":0.56000000000000005,"15min":5.8529999999999998},"last":0.35199999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58299999999999996,"15min":0.57499999999999996},"min":{"1min":0.36199999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78300000000000003,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.40000000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.81100000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.52300000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57699999999999996,"15min":0.74099999999999999},"min":{"1min":0.36499999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88,"5min":0.88,"15min":5.9459999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61199999999999999,"15min":0.61099999999999999},"min":{"1min":0.38900000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82599999999999996,"5min":0.92700000000000005,"15min":0.93999999999999995},"last":0.55500000000000005}]},{"osd":6,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54900000000000004,"15min":0.71199999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79100000000000004,"5min":0.79600000000000004,"15min":5.9160000000000004},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.379,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84799999999999998,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.44500000000000001}]},{"osd":7,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57199999999999995,"15min":0.58599999999999997},"min":{"1min":0.39000000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.80900000000000005,"5min":0.89500000000000002,"15min":0.97099999999999997},"last":0.439},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57099999999999995,"15min":0.749},"min":{"1min":0.41399999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":5.9729999999999999},"last":0.53900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574941,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303660,"kb_used_data":2276,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068180,"statfs":{"total":96636764160,"available":96325816320,"internally_reserved":0,"allocated":2330624,"data_stored":1363664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.30199999999999999,"15min":0.315},"min":{"1min":0.216,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.433,"5min":0.498,"15min":0.498},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.32100000000000001,"15min":0.34399999999999997},"min":{"1min":0.222,"5min":0.214,"15min":0.214},"max":{"1min":0.497,"5min":0.53600000000000003,"15min":0.94499999999999995},"last":0.39400000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35199999999999998,"15min":0.36199999999999999},"min":{"1min":0.25900000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.50900000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.371,"15min":0.38800000000000001},"min":{"1min":0.30499999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.56499999999999995,"5min":0.69399999999999995,"15min":0.91700000000000004},"last":0.33900000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.32800000000000001,"15min":0.38400000000000001},"min":{"1min":0.23699999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":1.554},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.35999999999999999,"15min":0.39700000000000002},"min":{"1min":0.27700000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54400000000000004,"5min":0.58099999999999996,"15min":0.79000000000000004},"last":0.313}]},{"osd":4,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53600000000000003,"15min":0.55100000000000005},"min":{"1min":0.41199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":1.212},"last":0.62},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.72099999999999997},"min":{"1min":0.42199999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":5.484},"last":0.56499999999999995}]},{"osd":5,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.72499999999999998},"min":{"1min":0.41999999999999998,"5min":0.35799999999999998,"15min":0.35499999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":5.5449999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.83299999999999996,"5min":0.84499999999999997,"15min":1.113},"last":0.67600000000000005}]},{"osd":6,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":1.0529999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59299999999999997,"15min":0.60699999999999998},"min":{"1min":0.45800000000000002,"5min":0.42799999999999999,"15min":0.41599999999999998},"max":{"1min":0.878,"5min":0.878,"15min":1.169},"last":0.65000000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57099999999999995,"15min":0.58899999999999997},"min":{"1min":0.40600000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.78700000000000003,"5min":0.85499999999999998,"15min":1.2390000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56399999999999995,"15min":0.749},"min":{"1min":0.38,"5min":0.375,"15min":0.375},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":5.5800000000000001},"last":0.58199999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574942,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303088,"kb_used_data":1704,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068752,"statfs":{"total":96636764160,"available":96326402048,"internally_reserved":0,"allocated":1744896,"data_stored":773940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.316,"15min":0.35299999999999998},"min":{"1min":0.23200000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":1.4199999999999999},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.55900000000000005},"min":{"1min":0.27500000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.51600000000000001,"5min":0.53800000000000003,"15min":7.2699999999999996},"last":0.27500000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.317,"15min":0.34399999999999997},"min":{"1min":0.23999999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.65400000000000003},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.372,"5min":0.37,"15min":0.61299999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":7.327},"last":0.314}]},{"osd":3,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.36799999999999999,"15min":0.39100000000000001},"min":{"1min":0.29399999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.68600000000000005},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.36799999999999999,"15min":0.39200000000000002},"min":{"1min":0.27900000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.72699999999999998},"last":0.37}]},{"osd":4,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54500000000000004,"15min":0.54400000000000004},"min":{"1min":0.40799999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.78400000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56200000000000006,"15min":0.78600000000000003},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82699999999999996,"5min":0.84699999999999998,"15min":7.2389999999999999},"last":0.38200000000000001}]},{"osd":5,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.58099999999999996,"15min":0.91500000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82699999999999996,"5min":0.83799999999999997,"15min":10.890000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":0.84299999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.45200000000000001}]},{"osd":6,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56399999999999995,"15min":0.79100000000000004},"min":{"1min":0.42999999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.748,"5min":0.77300000000000002,"15min":7.2999999999999998},"last":0.434},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.45800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.46400000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59299999999999997,"15min":0.82199999999999995},"min":{"1min":0.47499999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":7.1879999999999997},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60399999999999998,"15min":0.624},"min":{"1min":0.49399999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.90500000000000003},"last":0.495}]}]},{"osd":3,"up_from":13,"seq":55834574941,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303088,"kb_used_data":1704,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068752,"statfs":{"total":96636764160,"available":96326402048,"internally_reserved":0,"allocated":1744896,"data_stored":773468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.32200000000000001,"15min":0.33200000000000002},"min":{"1min":0.23100000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":5.8700000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.34699999999999998,"15min":0.36699999999999999},"min":{"1min":0.23999999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":5.6769999999999996},"last":0.56200000000000006}]},{"osd":1,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.35799999999999998,"15min":0.36099999999999999},"min":{"1min":0.26800000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":4.0469999999999997},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.372,"15min":0.38800000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":4.1509999999999998},"last":0.54200000000000004}]},{"osd":2,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.35199999999999998,"15min":0.374},"min":{"1min":0.29999999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.50800000000000001,"5min":0.56799999999999995,"15min":3.1139999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.373,"15min":0.38600000000000001},"min":{"1min":0.28100000000000003,"5min":0.26100000000000001,"15min":0.25600000000000001},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":3.3180000000000001},"last":0.57699999999999996}]},{"osd":4,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55800000000000005,"15min":0.57099999999999995},"min":{"1min":0.38100000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":3.0899999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56499999999999995,"15min":0.56799999999999995},"min":{"1min":0.40300000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76400000000000001,"5min":0.91100000000000003,"15min":3.0710000000000002},"last":0.72299999999999998}]},{"osd":5,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57499999999999996,"15min":0.58899999999999997},"min":{"1min":0.39100000000000001,"5min":0.34599999999999997,"15min":0.32700000000000001},"max":{"1min":0.877,"5min":0.877,"15min":6.2300000000000004},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.441,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79800000000000004,"5min":0.89500000000000002,"15min":3.0219999999999998},"last":0.68400000000000005}]},{"osd":6,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.44500000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":2.6850000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.65200000000000002},"min":{"1min":0.43099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":6.5970000000000004},"last":0.746}]},{"osd":7,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60699999999999998,"15min":0.61399999999999999},"min":{"1min":0.44700000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86099999999999999,"5min":0.93300000000000005,"15min":4.2629999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59999999999999998,"15min":0.61299999999999999},"min":{"1min":0.40500000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.82899999999999996,"5min":0.97199999999999998,"15min":7.3650000000000002},"last":0.69599999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574940,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303092,"kb_used_data":1708,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068748,"statfs":{"total":96636764160,"available":96326397952,"internally_reserved":0,"allocated":1748992,"data_stored":775364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.51400000000000001,"15min":0.55300000000000005},"min":{"1min":0.318,"5min":0.28100000000000003,"15min":0.27000000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":1.083},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.35699999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.79900000000000004,"5min":0.97799999999999998,"15min":1.054},"last":0.68400000000000005}]},{"osd":1,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.39900000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.501,"5min":0.50700000000000001,"15min":0.49399999999999999},"min":{"1min":0.33800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.76300000000000001,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.75700000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52700000000000002,"15min":0.53900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.32900000000000001},"max":{"1min":0.76600000000000001,"5min":0.82099999999999995,"15min":1.0249999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55700000000000005,"15min":0.59099999999999997},"min":{"1min":0.39400000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.93799999999999994},"last":0.623}]},{"osd":3,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52900000000000003,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.32200000000000001,"15min":0.29199999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.98499999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57699999999999996,"15min":0.58799999999999997},"min":{"1min":0.38700000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.1319999999999999},"last":0.77500000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51600000000000001,"15min":0.55100000000000005},"min":{"1min":0.31900000000000001,"5min":0.30399999999999999,"15min":0.27300000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58199999999999996,"15min":0.60599999999999998},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.35099999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.97999999999999998},"last":0.60099999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.56799999999999995},"min":{"1min":0.41599999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.76800000000000002,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.59799999999999998},"min":{"1min":0.38700000000000001,"5min":0.34999999999999998,"15min":0.32700000000000001},"max":{"1min":0.85599999999999998,"5min":1.01,"15min":1.01},"last":0.53700000000000003}]},{"osd":7,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54300000000000004,"15min":0.56999999999999995},"min":{"1min":0.40200000000000002,"5min":0.33600000000000002,"15min":0.28299999999999997},"max":{"1min":0.78900000000000003,"5min":0.85599999999999998,"15min":0.90000000000000002},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.60699999999999998,"15min":0.64500000000000002},"min":{"1min":0.36699999999999999,"5min":0.34999999999999998,"15min":0.32200000000000001},"max":{"1min":0.89000000000000001,"5min":0.98999999999999999,"15min":1.175},"last":0.65900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574940,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303076,"kb_used_data":1692,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068764,"statfs":{"total":96636764160,"available":96326414336,"internally_reserved":0,"allocated":1732608,"data_stored":773274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.503,"15min":0.50800000000000001},"min":{"1min":0.38800000000000001,"5min":0.29299999999999998,"15min":0.28599999999999998},"max":{"1min":0.67900000000000005,"5min":0.71399999999999997,"15min":0.86599999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51800000000000002,"15min":0.51400000000000001},"min":{"1min":0.42499999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.60599999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.52600000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.38400000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.63900000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52300000000000002,"15min":0.53100000000000003},"min":{"1min":0.43099999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72799999999999998,"5min":0.73899999999999999,"15min":0.81599999999999995},"last":0.51200000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54300000000000004,"15min":0.54400000000000004},"min":{"1min":0.41099999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.63900000000000001,"5min":0.72999999999999998,"15min":0.83999999999999997},"last":0.623},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.47199999999999998,"5min":0.40999999999999998,"15min":0.39500000000000002},"max":{"1min":0.72699999999999998,"5min":0.74099999999999999,"15min":0.79900000000000004},"last":0.63300000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.42399999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.66900000000000004,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.54900000000000004,"15min":0.56000000000000005},"min":{"1min":0.42499999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.63500000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.57299999999999995}]},{"osd":4,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.50600000000000001,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.36199999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.80700000000000005},"last":0.47399999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55300000000000005,"15min":0.53600000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.83199999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.36199999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77900000000000003,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.60799999999999998}]},{"osd":7,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.57899999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71499999999999997,"5min":0.79500000000000004,"15min":0.85599999999999998},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.374,"5min":0.35199999999999998,"15min":0.314},"max":{"1min":0.79500000000000004,"5min":0.84499999999999997,"15min":0.90600000000000003},"last":0.53800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:37:55.734 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:37:55.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.862+0000 7f7aeebfb700 1 Processor -- start 2023-10-09T06:37:55.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.862+0000 7f7aeebfb700 1 -- start start 2023-10-09T06:37:55.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.863+0000 7f7aeebfb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7ae81597f0 0x7f7ae80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:55.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.863+0000 7f7aeebfb700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7ae815d230 0x7f7ae815d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:55.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.863+0000 7f7aeebfb700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ae815db70 con 0x7f7ae8159420 2023-10-09T06:37:55.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.863+0000 7f7aeebfb700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae80b11d0 con 0x7f7ae815d230 2023-10-09T06:37:55.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.863+0000 7f7aeebfb700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae80b14b0 con 0x7f7ae81597f0 2023-10-09T06:37:55.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.863+0000 7f7aee3fa700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7ae815d230 0x7f7ae815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:55.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.863+0000 7f7aedbf9700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f7ae8159420 0x7f7ae80b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33562/0 (socket says 172.21.15.163:33562) 2023-10-09T06:37:55.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.863+0000 7f7aedbf9700 1 -- 172.21.15.163:0/918560676 learned_addr learned my addr 172.21.15.163:0/918560676 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:55.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.863+0000 7f7aed3f8700 1 --2- 172.21.15.163:0/918560676 >> v2:172.21.15.181:3300/0 conn(0x7f7ae81597f0 0x7f7ae80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:55.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.863+0000 7f7aee3fa700 1 -- 172.21.15.163:0/918560676 >> v2:172.21.15.181:3300/0 conn(0x7f7ae81597f0 msgr2=0x7f7ae80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:55.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.863+0000 7f7aee3fa700 1 --2- 172.21.15.163:0/918560676 >> v2:172.21.15.181:3300/0 conn(0x7f7ae81597f0 0x7f7ae80b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:55.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.864+0000 7f7aee3fa700 1 -- 172.21.15.163:0/918560676 >> v1:172.21.15.163:6789/0 conn(0x7f7ae8159420 legacy=0x7f7ae80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:55.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.864+0000 7f7aee3fa700 1 -- 172.21.15.163:0/918560676 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae80b1790 con 0x7f7ae815d230 2023-10-09T06:37:55.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.864+0000 7f7aecbf7700 1 -- 172.21.15.163:0/918560676 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2809875630 0 0) 0x7f7ae815db70 con 0x7f7ae8159420 2023-10-09T06:37:55.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.864+0000 7f7aed3f8700 1 --2- 172.21.15.163:0/918560676 >> v2:172.21.15.181:3300/0 conn(0x7f7ae81597f0 0x7f7ae80b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:37:55.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.864+0000 7f7aee3fa700 1 --2- 172.21.15.163:0/918560676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ae815d230 0x7f7ae815d630 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8089750 tx=0x7f7ae00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f18edaa9f085a server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:55.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.864+0000 7f7aecbf7700 1 -- 172.21.15.163:0/918560676 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae000b6b0 con 0x7f7ae815d230 2023-10-09T06:37:55.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.864+0000 7f7aecbf7700 1 -- 172.21.15.163:0/918560676 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae000b810 con 0x7f7ae815d230 2023-10-09T06:37:55.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.864+0000 7f7aecbf7700 1 -- 172.21.15.163:0/918560676 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae000bbb0 con 0x7f7ae815d230 2023-10-09T06:37:55.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.864+0000 7f7aeebfb700 1 -- 172.21.15.163:0/918560676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ae815d230 msgr2=0x7f7ae815d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:55.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.864+0000 7f7aeebfb700 1 --2- 172.21.15.163:0/918560676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ae815d230 0x7f7ae815d630 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8089750 tx=0x7f7ae00091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:55.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.864+0000 7f7aeebfb700 1 -- 172.21.15.163:0/918560676 shutdown_connections 2023-10-09T06:37:55.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.864+0000 7f7aeebfb700 1 --2- 172.21.15.163:0/918560676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ae815d230 0x7f7ae815d630 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:55.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.864+0000 7f7aeebfb700 1 --2- 172.21.15.163:0/918560676 >> v2:172.21.15.181:3300/0 conn(0x7f7ae81597f0 0x7f7ae80b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:55.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.864+0000 7f7aeebfb700 1 -- 172.21.15.163:0/918560676 >> 172.21.15.163:0/918560676 conn(0x7f7ae80d0660 msgr2=0x7f7ae80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:55.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.865+0000 7f7aeebfb700 1 -- 172.21.15.163:0/918560676 shutdown_connections 2023-10-09T06:37:55.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.865+0000 7f7aeebfb700 1 -- 172.21.15.163:0/918560676 wait complete. 2023-10-09T06:37:55.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.866+0000 7f7aeebfb700 1 Processor -- start 2023-10-09T06:37:55.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.866+0000 7f7aeebfb700 1 -- start start 2023-10-09T06:37:55.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.866+0000 7f7aeebfb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ae8159420 0x7f7ae814c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:55.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.866+0000 7f7aeebfb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ae81597f0 0x7f7ae8154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:55.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.866+0000 7f7aeebfb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ae815d230 0x7f7ae8151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:55.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.866+0000 7f7aeebfb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae8150ee0 con 0x7f7ae81597f0 2023-10-09T06:37:55.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.866+0000 7f7aeebfb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae8150260 con 0x7f7ae815d230 2023-10-09T06:37:55.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.866+0000 7f7aeebfb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae8150540 con 0x7f7ae8159420 2023-10-09T06:37:55.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.866+0000 7f7aed3f8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ae81597f0 0x7f7ae8154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:55.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.866+0000 7f7aed3f8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ae81597f0 0x7f7ae8154340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41758/0 (socket says 172.21.15.163:41758) 2023-10-09T06:37:55.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.866+0000 7f7aed3f8700 1 -- 172.21.15.163:0/3793832485 learned_addr learned my addr 172.21.15.163:0/3793832485 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:55.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.867+0000 7f7aee3fa700 1 --2- 172.21.15.163:0/3793832485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ae815d230 0x7f7ae8151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:55.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.867+0000 7f7aedbf9700 1 --2- 172.21.15.163:0/3793832485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ae8159420 0x7f7ae814c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:55.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.867+0000 7f7aed3f8700 1 -- 172.21.15.163:0/3793832485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ae8159420 msgr2=0x7f7ae814c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:55.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.867+0000 7f7aed3f8700 1 --2- 172.21.15.163:0/3793832485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ae8159420 0x7f7ae814c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:55.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.867+0000 7f7aed3f8700 1 -- 172.21.15.163:0/3793832485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ae815d230 msgr2=0x7f7ae8151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:55.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.867+0000 7f7aed3f8700 1 --2- 172.21.15.163:0/3793832485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ae815d230 0x7f7ae8151300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:55.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.867+0000 7f7aed3f8700 1 -- 172.21.15.163:0/3793832485 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae0009420 con 0x7f7ae81597f0 2023-10-09T06:37:55.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.867+0000 7f7aee3fa700 1 --2- 172.21.15.163:0/3793832485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ae815d230 0x7f7ae8151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:37:55.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.867+0000 7f7aed3f8700 1 --2- 172.21.15.163:0/3793832485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ae81597f0 0x7f7ae8154340 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800c880 tx=0x7f7ad8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:55.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.867+0000 7f7addffb700 1 -- 172.21.15.163:0/3793832485 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8011600 con 0x7f7ae81597f0 2023-10-09T06:37:55.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.867+0000 7f7addffb700 1 -- 172.21.15.163:0/3793832485 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad8011760 con 0x7f7ae81597f0 2023-10-09T06:37:55.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.867+0000 7f7aeebfb700 1 -- 172.21.15.163:0/3793832485 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae8150820 con 0x7f7ae81597f0 2023-10-09T06:37:55.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.867+0000 7f7addffb700 1 -- 172.21.15.163:0/3793832485 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8011ae0 con 0x7f7ae81597f0 2023-10-09T06:37:55.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.867+0000 7f7aeebfb700 1 -- 172.21.15.163:0/3793832485 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae814fea0 con 0x7f7ae81597f0 2023-10-09T06:37:55.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.868+0000 7f7aeebfb700 1 -- 172.21.15.163:0/3793832485 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae8151020 con 0x7f7ae81597f0 2023-10-09T06:37:55.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.869+0000 7f7addffb700 1 -- 172.21.15.163:0/3793832485 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7ad80073d0 con 0x7f7ae81597f0 2023-10-09T06:37:55.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.869+0000 7f7addffb700 1 --2- 172.21.15.163:0/3793832485 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7acc067320 0x7f7acc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:55.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.869+0000 7f7addffb700 1 -- 172.21.15.163:0/3793832485 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7ad8084dc0 con 0x7f7ae81597f0 2023-10-09T06:37:55.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.869+0000 7f7aedbf9700 1 --2- 172.21.15.163:0/3793832485 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7acc067320 0x7f7acc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:55.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.870+0000 7f7aedbf9700 1 --2- 172.21.15.163:0/3793832485 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7acc067320 0x7f7acc0697d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4002220 tx=0x7f7ae4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:55.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:55.873+0000 7f7addffb700 1 -- 172.21.15.163:0/3793832485 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ae8151020 con 0x7f7ae81597f0 2023-10-09T06:37:56.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.052+0000 7f7aeebfb700 1 -- 172.21.15.163:0/3793832485 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7ae8151020 con 0x7f7acc067320 2023-10-09T06:37:56.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.061+0000 7f7addffb700 1 -- 172.21.15.163:0/3793832485 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290264 (secure 0 0 0) 0x7f7ae8151020 con 0x7f7acc067320 2023-10-09T06:37:56.066 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:56.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.064+0000 7f7aeebfb700 1 -- 172.21.15.163:0/3793832485 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7acc067320 msgr2=0x7f7acc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:56.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.064+0000 7f7aeebfb700 1 --2- 172.21.15.163:0/3793832485 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7acc067320 0x7f7acc0697d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4002220 tx=0x7f7ae4009040 comp rx=0 tx=0).stop 2023-10-09T06:37:56.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.064+0000 7f7aeebfb700 1 -- 172.21.15.163:0/3793832485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ae81597f0 msgr2=0x7f7ae8154340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:56.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.064+0000 7f7aeebfb700 1 --2- 172.21.15.163:0/3793832485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ae81597f0 0x7f7ae8154340 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800c880 tx=0x7f7ad8008040 comp rx=0 tx=0).stop 2023-10-09T06:37:56.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.064+0000 7f7aeebfb700 1 -- 172.21.15.163:0/3793832485 shutdown_connections 2023-10-09T06:37:56.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.064+0000 7f7aeebfb700 1 --2- 172.21.15.163:0/3793832485 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7acc067320 0x7f7acc0697d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:56.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.064+0000 7f7aeebfb700 1 --2- 172.21.15.163:0/3793832485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ae815d230 0x7f7ae8151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:56.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.064+0000 7f7aeebfb700 1 --2- 172.21.15.163:0/3793832485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ae8159420 0x7f7ae814c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:56.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.064+0000 7f7aeebfb700 1 --2- 172.21.15.163:0/3793832485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ae81597f0 0x7f7ae8154340 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:56.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.064+0000 7f7aeebfb700 1 -- 172.21.15.163:0/3793832485 >> 172.21.15.163:0/3793832485 conn(0x7f7ae80d0660 msgr2=0x7f7ae8069340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:56.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.064+0000 7f7aeebfb700 1 -- 172.21.15.163:0/3793832485 shutdown_connections 2023-10-09T06:37:56.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.064+0000 7f7aeebfb700 1 -- 172.21.15.163:0/3793832485 wait complete. 2023-10-09T06:37:56.070 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:37:56.077 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2023-10-09T06:37:54.403750+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2425832,"kb_used_data":14760,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752548888,"statfs":{"total":773094113280,"available":770610061312,"internally_reserved":0,"allocated":15114240,"data_stored":7371540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002025"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.183830+0000","last_change":"2023-10-09T06:37:47.179494+0000","last_active":"2023-10-09T06:37:47.183830+0000","last_peered":"2023-10-09T06:37:47.183830+0000","last_clean":"2023-10-09T06:37:47.183830+0000","last_became_active":"2023-10-09T06:37:47.179178+0000","last_became_peered":"2023-10-09T06:37:47.179178+0000","last_unstale":"2023-10-09T06:37:47.183830+0000","last_undegraded":"2023-10-09T06:37:47.183830+0000","last_fullsized":"2023-10-09T06:37:47.183830+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:02:47.287524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.174192+0000","last_change":"2023-10-09T06:37:47.170317+0000","last_active":"2023-10-09T06:37:47.174192+0000","last_peered":"2023-10-09T06:37:47.174192+0000","last_clean":"2023-10-09T06:37:47.174192+0000","last_became_active":"2023-10-09T06:37:47.169622+0000","last_became_peered":"2023-10-09T06:37:47.169622+0000","last_unstale":"2023-10-09T06:37:47.174192+0000","last_undegraded":"2023-10-09T06:37:47.174192+0000","last_fullsized":"2023-10-09T06:37:47.174192+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:22:45.789761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.184428+0000","last_change":"2023-10-09T06:37:47.179624+0000","last_active":"2023-10-09T06:37:47.184428+0000","last_peered":"2023-10-09T06:37:47.184428+0000","last_clean":"2023-10-09T06:37:47.184428+0000","last_became_active":"2023-10-09T06:37:47.179336+0000","last_became_peered":"2023-10-09T06:37:47.179336+0000","last_unstale":"2023-10-09T06:37:47.184428+0000","last_undegraded":"2023-10-09T06:37:47.184428+0000","last_fullsized":"2023-10-09T06:37:47.184428+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:31:42.797752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.182655+0000","last_change":"2023-10-09T06:37:47.179465+0000","last_active":"2023-10-09T06:37:47.182655+0000","last_peered":"2023-10-09T06:37:47.182655+0000","last_clean":"2023-10-09T06:37:47.182655+0000","last_became_active":"2023-10-09T06:37:47.178726+0000","last_became_peered":"2023-10-09T06:37:47.178726+0000","last_unstale":"2023-10-09T06:37:47.182655+0000","last_undegraded":"2023-10-09T06:37:47.182655+0000","last_fullsized":"2023-10-09T06:37:47.182655+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:39:40.857018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.181258+0000","last_change":"2023-10-09T06:37:47.180178+0000","last_active":"2023-10-09T06:37:47.181258+0000","last_peered":"2023-10-09T06:37:47.181258+0000","last_clean":"2023-10-09T06:37:47.181258+0000","last_became_active":"2023-10-09T06:37:47.179891+0000","last_became_peered":"2023-10-09T06:37:47.179891+0000","last_unstale":"2023-10-09T06:37:47.181258+0000","last_undegraded":"2023-10-09T06:37:47.181258+0000","last_fullsized":"2023-10-09T06:37:47.181258+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:01:23.258784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.184380+0000","last_change":"2023-10-09T06:37:47.177855+0000","last_active":"2023-10-09T06:37:47.184380+0000","last_peered":"2023-10-09T06:37:47.184380+0000","last_clean":"2023-10-09T06:37:47.184380+0000","last_became_active":"2023-10-09T06:37:47.177194+0000","last_became_peered":"2023-10-09T06:37:47.177194+0000","last_unstale":"2023-10-09T06:37:47.184380+0000","last_undegraded":"2023-10-09T06:37:47.184380+0000","last_fullsized":"2023-10-09T06:37:47.184380+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:02:47.287524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.175545+0000","last_change":"2023-10-09T06:37:47.172833+0000","last_active":"2023-10-09T06:37:47.175545+0000","last_peered":"2023-10-09T06:37:47.175545+0000","last_clean":"2023-10-09T06:37:47.175545+0000","last_became_active":"2023-10-09T06:37:47.172497+0000","last_became_peered":"2023-10-09T06:37:47.172497+0000","last_unstale":"2023-10-09T06:37:47.175545+0000","last_undegraded":"2023-10-09T06:37:47.175545+0000","last_fullsized":"2023-10-09T06:37:47.175545+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:38:04.981588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.177953+0000","last_change":"2023-10-09T06:37:47.172754+0000","last_active":"2023-10-09T06:37:47.177953+0000","last_peered":"2023-10-09T06:37:47.177953+0000","last_clean":"2023-10-09T06:37:47.177953+0000","last_became_active":"2023-10-09T06:37:47.172530+0000","last_became_peered":"2023-10-09T06:37:47.172530+0000","last_unstale":"2023-10-09T06:37:47.177953+0000","last_undegraded":"2023-10-09T06:37:47.177953+0000","last_fullsized":"2023-10-09T06:37:47.177953+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:59:24.334461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.180894+0000","last_change":"2023-10-09T06:37:47.175288+0000","last_active":"2023-10-09T06:37:47.180894+0000","last_peered":"2023-10-09T06:37:47.180894+0000","last_clean":"2023-10-09T06:37:47.180894+0000","last_became_active":"2023-10-09T06:37:47.174830+0000","last_became_peered":"2023-10-09T06:37:47.174830+0000","last_unstale":"2023-10-09T06:37:47.180894+0000","last_undegraded":"2023-10-09T06:37:47.180894+0000","last_fullsized":"2023-10-09T06:37:47.180894+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:30:56.136503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.178069+0000","last_change":"2023-10-09T06:37:47.169865+0000","last_active":"2023-10-09T06:37:47.178069+0000","last_peered":"2023-10-09T06:37:47.178069+0000","last_clean":"2023-10-09T06:37:47.178069+0000","last_became_active":"2023-10-09T06:37:47.169615+0000","last_became_peered":"2023-10-09T06:37:47.169615+0000","last_unstale":"2023-10-09T06:37:47.178069+0000","last_undegraded":"2023-10-09T06:37:47.178069+0000","last_fullsized":"2023-10-09T06:37:47.178069+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:40:16.005552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.180769+0000","last_change":"2023-10-09T06:37:47.177471+0000","last_active":"2023-10-09T06:37:47.180769+0000","last_peered":"2023-10-09T06:37:47.180769+0000","last_clean":"2023-10-09T06:37:47.180769+0000","last_became_active":"2023-10-09T06:37:47.177167+0000","last_became_peered":"2023-10-09T06:37:47.177167+0000","last_unstale":"2023-10-09T06:37:47.180769+0000","last_undegraded":"2023-10-09T06:37:47.180769+0000","last_fullsized":"2023-10-09T06:37:47.180769+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:55:35.647464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.180839+0000","last_change":"2023-10-09T06:37:47.178688+0000","last_active":"2023-10-09T06:37:47.180839+0000","last_peered":"2023-10-09T06:37:47.180839+0000","last_clean":"2023-10-09T06:37:47.180839+0000","last_became_active":"2023-10-09T06:37:47.178433+0000","last_became_peered":"2023-10-09T06:37:47.178433+0000","last_unstale":"2023-10-09T06:37:47.180839+0000","last_undegraded":"2023-10-09T06:37:47.180839+0000","last_fullsized":"2023-10-09T06:37:47.180839+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:26:39.377444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.172274+0000","last_change":"2023-10-09T06:37:47.169996+0000","last_active":"2023-10-09T06:37:47.172274+0000","last_peered":"2023-10-09T06:37:47.172274+0000","last_clean":"2023-10-09T06:37:47.172274+0000","last_became_active":"2023-10-09T06:37:47.169396+0000","last_became_peered":"2023-10-09T06:37:47.169396+0000","last_unstale":"2023-10-09T06:37:47.172274+0000","last_undegraded":"2023-10-09T06:37:47.172274+0000","last_fullsized":"2023-10-09T06:37:47.172274+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:04:59.600452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.179031+0000","last_change":"2023-10-09T06:37:47.173418+0000","last_active":"2023-10-09T06:37:47.179031+0000","last_peered":"2023-10-09T06:37:47.179031+0000","last_clean":"2023-10-09T06:37:47.179031+0000","last_became_active":"2023-10-09T06:37:47.173133+0000","last_became_peered":"2023-10-09T06:37:47.173133+0000","last_unstale":"2023-10-09T06:37:47.179031+0000","last_undegraded":"2023-10-09T06:37:47.179031+0000","last_fullsized":"2023-10-09T06:37:47.179031+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:04:59.600452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.182168+0000","last_change":"2023-10-09T06:37:47.179720+0000","last_active":"2023-10-09T06:37:47.182168+0000","last_peered":"2023-10-09T06:37:47.182168+0000","last_clean":"2023-10-09T06:37:47.182168+0000","last_became_active":"2023-10-09T06:37:47.179094+0000","last_became_peered":"2023-10-09T06:37:47.179094+0000","last_unstale":"2023-10-09T06:37:47.182168+0000","last_undegraded":"2023-10-09T06:37:47.182168+0000","last_fullsized":"2023-10-09T06:37:47.182168+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:30:56.136503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.178270+0000","last_change":"2023-10-09T06:37:47.175094+0000","last_active":"2023-10-09T06:37:47.178270+0000","last_peered":"2023-10-09T06:37:47.178270+0000","last_clean":"2023-10-09T06:37:47.178270+0000","last_became_active":"2023-10-09T06:37:47.174842+0000","last_became_peered":"2023-10-09T06:37:47.174842+0000","last_unstale":"2023-10-09T06:37:47.178270+0000","last_undegraded":"2023-10-09T06:37:47.178270+0000","last_fullsized":"2023-10-09T06:37:47.178270+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:11:04.361133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.187695+0000","last_change":"2023-10-09T06:37:47.181745+0000","last_active":"2023-10-09T06:37:47.187695+0000","last_peered":"2023-10-09T06:37:47.187695+0000","last_clean":"2023-10-09T06:37:47.187695+0000","last_became_active":"2023-10-09T06:37:47.175655+0000","last_became_peered":"2023-10-09T06:37:47.175655+0000","last_unstale":"2023-10-09T06:37:47.187695+0000","last_undegraded":"2023-10-09T06:37:47.187695+0000","last_fullsized":"2023-10-09T06:37:47.187695+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:24:43.408333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.176651+0000","last_change":"2023-10-09T06:37:47.175166+0000","last_active":"2023-10-09T06:37:47.176651+0000","last_peered":"2023-10-09T06:37:47.176651+0000","last_clean":"2023-10-09T06:37:47.176651+0000","last_became_active":"2023-10-09T06:37:47.174847+0000","last_became_peered":"2023-10-09T06:37:47.174847+0000","last_unstale":"2023-10-09T06:37:47.176651+0000","last_undegraded":"2023-10-09T06:37:47.176651+0000","last_fullsized":"2023-10-09T06:37:47.176651+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:30:30.469711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.181764+0000","last_change":"2023-10-09T06:37:47.179072+0000","last_active":"2023-10-09T06:37:47.181764+0000","last_peered":"2023-10-09T06:37:47.181764+0000","last_clean":"2023-10-09T06:37:47.181764+0000","last_became_active":"2023-10-09T06:37:47.178800+0000","last_became_peered":"2023-10-09T06:37:47.178800+0000","last_unstale":"2023-10-09T06:37:47.181764+0000","last_undegraded":"2023-10-09T06:37:47.181764+0000","last_fullsized":"2023-10-09T06:37:47.181764+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:11:04.361133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.182584+0000","last_change":"2023-10-09T06:37:47.175579+0000","last_active":"2023-10-09T06:37:47.182584+0000","last_peered":"2023-10-09T06:37:47.182584+0000","last_clean":"2023-10-09T06:37:47.182584+0000","last_became_active":"2023-10-09T06:37:47.175348+0000","last_became_peered":"2023-10-09T06:37:47.175348+0000","last_unstale":"2023-10-09T06:37:47.182584+0000","last_undegraded":"2023-10-09T06:37:47.182584+0000","last_fullsized":"2023-10-09T06:37:47.182584+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:41:11.780330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.181247+0000","last_change":"2023-10-09T06:37:47.180074+0000","last_active":"2023-10-09T06:37:47.181247+0000","last_peered":"2023-10-09T06:37:47.181247+0000","last_clean":"2023-10-09T06:37:47.181247+0000","last_became_active":"2023-10-09T06:37:47.179800+0000","last_became_peered":"2023-10-09T06:37:47.179800+0000","last_unstale":"2023-10-09T06:37:47.181247+0000","last_undegraded":"2023-10-09T06:37:47.181247+0000","last_fullsized":"2023-10-09T06:37:47.181247+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:42:16.140720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.181696+0000","last_change":"2023-10-09T06:37:47.178389+0000","last_active":"2023-10-09T06:37:47.181696+0000","last_peered":"2023-10-09T06:37:47.181696+0000","last_clean":"2023-10-09T06:37:47.181696+0000","last_became_active":"2023-10-09T06:37:47.178178+0000","last_became_peered":"2023-10-09T06:37:47.178178+0000","last_unstale":"2023-10-09T06:37:47.181696+0000","last_undegraded":"2023-10-09T06:37:47.181696+0000","last_fullsized":"2023-10-09T06:37:47.181696+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:49:25.010643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.175661+0000","last_change":"2023-10-09T06:37:47.171441+0000","last_active":"2023-10-09T06:37:47.175661+0000","last_peered":"2023-10-09T06:37:47.175661+0000","last_clean":"2023-10-09T06:37:47.175661+0000","last_became_active":"2023-10-09T06:37:47.171181+0000","last_became_peered":"2023-10-09T06:37:47.171181+0000","last_unstale":"2023-10-09T06:37:47.175661+0000","last_undegraded":"2023-10-09T06:37:47.175661+0000","last_fullsized":"2023-10-09T06:37:47.175661+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:30:30.469711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.177934+0000","last_change":"2023-10-09T06:37:47.169581+0000","last_active":"2023-10-09T06:37:47.177934+0000","last_peered":"2023-10-09T06:37:47.177934+0000","last_clean":"2023-10-09T06:37:47.177934+0000","last_became_active":"2023-10-09T06:37:47.169186+0000","last_became_peered":"2023-10-09T06:37:47.169186+0000","last_unstale":"2023-10-09T06:37:47.177934+0000","last_undegraded":"2023-10-09T06:37:47.177934+0000","last_fullsized":"2023-10-09T06:37:47.177934+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:49:54.766563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.174021+0000","last_change":"2023-10-09T06:37:47.171633+0000","last_active":"2023-10-09T06:37:47.174021+0000","last_peered":"2023-10-09T06:37:47.174021+0000","last_clean":"2023-10-09T06:37:47.174021+0000","last_became_active":"2023-10-09T06:37:47.171387+0000","last_became_peered":"2023-10-09T06:37:47.171387+0000","last_unstale":"2023-10-09T06:37:47.174021+0000","last_undegraded":"2023-10-09T06:37:47.174021+0000","last_fullsized":"2023-10-09T06:37:47.174021+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:03:12.426555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.171549+0000","last_change":"2023-10-09T06:37:47.169127+0000","last_active":"2023-10-09T06:37:47.171549+0000","last_peered":"2023-10-09T06:37:47.171549+0000","last_clean":"2023-10-09T06:37:47.171549+0000","last_became_active":"2023-10-09T06:37:47.168815+0000","last_became_peered":"2023-10-09T06:37:47.168815+0000","last_unstale":"2023-10-09T06:37:47.171549+0000","last_undegraded":"2023-10-09T06:37:47.171549+0000","last_fullsized":"2023-10-09T06:37:47.171549+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:23:16.450280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.183759+0000","last_change":"2023-10-09T06:37:47.177549+0000","last_active":"2023-10-09T06:37:47.183759+0000","last_peered":"2023-10-09T06:37:47.183759+0000","last_clean":"2023-10-09T06:37:47.183759+0000","last_became_active":"2023-10-09T06:37:47.177263+0000","last_became_peered":"2023-10-09T06:37:47.177263+0000","last_unstale":"2023-10-09T06:37:47.183759+0000","last_undegraded":"2023-10-09T06:37:47.183759+0000","last_fullsized":"2023-10-09T06:37:47.183759+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:14:48.221558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.187920+0000","last_change":"2023-10-09T06:37:47.182473+0000","last_active":"2023-10-09T06:37:47.187920+0000","last_peered":"2023-10-09T06:37:47.187920+0000","last_clean":"2023-10-09T06:37:47.187920+0000","last_became_active":"2023-10-09T06:37:47.175737+0000","last_became_peered":"2023-10-09T06:37:47.175737+0000","last_unstale":"2023-10-09T06:37:47.187920+0000","last_undegraded":"2023-10-09T06:37:47.187920+0000","last_fullsized":"2023-10-09T06:37:47.187920+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:24:28.778321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.181631+0000","last_change":"2023-10-09T06:37:47.179313+0000","last_active":"2023-10-09T06:37:47.181631+0000","last_peered":"2023-10-09T06:37:47.181631+0000","last_clean":"2023-10-09T06:37:47.181631+0000","last_became_active":"2023-10-09T06:37:47.178278+0000","last_became_peered":"2023-10-09T06:37:47.178278+0000","last_unstale":"2023-10-09T06:37:47.181631+0000","last_undegraded":"2023-10-09T06:37:47.181631+0000","last_fullsized":"2023-10-09T06:37:47.181631+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:48:25.266457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.184511+0000","last_change":"2023-10-09T06:37:47.176318+0000","last_active":"2023-10-09T06:37:47.184511+0000","last_peered":"2023-10-09T06:37:47.184511+0000","last_clean":"2023-10-09T06:37:47.184511+0000","last_became_active":"2023-10-09T06:37:47.176056+0000","last_became_peered":"2023-10-09T06:37:47.176056+0000","last_unstale":"2023-10-09T06:37:47.184511+0000","last_undegraded":"2023-10-09T06:37:47.184511+0000","last_fullsized":"2023-10-09T06:37:47.184511+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:26:47.866012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.167706+0000","last_change":"2023-10-09T06:37:47.165039+0000","last_active":"2023-10-09T06:37:47.167706+0000","last_peered":"2023-10-09T06:37:47.167706+0000","last_clean":"2023-10-09T06:37:47.167706+0000","last_became_active":"2023-10-09T06:37:47.164773+0000","last_became_peered":"2023-10-09T06:37:47.164773+0000","last_unstale":"2023-10-09T06:37:47.167706+0000","last_undegraded":"2023-10-09T06:37:47.167706+0000","last_fullsized":"2023-10-09T06:37:47.167706+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:12:00.359900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.180601+0000","last_change":"2023-10-09T06:37:47.173408+0000","last_active":"2023-10-09T06:37:47.180601+0000","last_peered":"2023-10-09T06:37:47.180601+0000","last_clean":"2023-10-09T06:37:47.180601+0000","last_became_active":"2023-10-09T06:37:47.173077+0000","last_became_peered":"2023-10-09T06:37:47.173077+0000","last_unstale":"2023-10-09T06:37:47.180601+0000","last_undegraded":"2023-10-09T06:37:47.180601+0000","last_fullsized":"2023-10-09T06:37:47.180601+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:03:34.852904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.178226+0000","last_change":"2023-10-09T06:37:47.174887+0000","last_active":"2023-10-09T06:37:47.178226+0000","last_peered":"2023-10-09T06:37:47.178226+0000","last_clean":"2023-10-09T06:37:47.178226+0000","last_became_active":"2023-10-09T06:37:47.174598+0000","last_became_peered":"2023-10-09T06:37:47.174598+0000","last_unstale":"2023-10-09T06:37:47.178226+0000","last_undegraded":"2023-10-09T06:37:47.178226+0000","last_fullsized":"2023-10-09T06:37:47.178226+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:40:16.005552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"136'4","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.188324+0000","last_change":"2023-10-09T06:37:46.162496+0000","last_active":"2023-10-09T06:37:47.188324+0000","last_peered":"2023-10-09T06:37:47.188324+0000","last_clean":"2023-10-09T06:37:47.188324+0000","last_became_active":"2023-10-09T06:37:46.161574+0000","last_became_peered":"2023-10-09T06:37:46.161574+0000","last_unstale":"2023-10-09T06:37:47.188324+0000","last_undegraded":"2023-10-09T06:37:47.188324+0000","last_fullsized":"2023-10-09T06:37:47.188324+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:45:54.585264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.184382+0000","last_change":"2023-10-09T06:37:47.181847+0000","last_active":"2023-10-09T06:37:47.184382+0000","last_peered":"2023-10-09T06:37:47.184382+0000","last_clean":"2023-10-09T06:37:47.184382+0000","last_became_active":"2023-10-09T06:37:47.181586+0000","last_became_peered":"2023-10-09T06:37:47.181586+0000","last_unstale":"2023-10-09T06:37:47.184382+0000","last_undegraded":"2023-10-09T06:37:47.184382+0000","last_fullsized":"2023-10-09T06:37:47.184382+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:23:25.125922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.184474+0000","last_change":"2023-10-09T06:37:46.158583+0000","last_active":"2023-10-09T06:37:47.184474+0000","last_peered":"2023-10-09T06:37:47.184474+0000","last_clean":"2023-10-09T06:37:47.184474+0000","last_became_active":"2023-10-09T06:37:46.158300+0000","last_became_peered":"2023-10-09T06:37:46.158300+0000","last_unstale":"2023-10-09T06:37:47.184474+0000","last_undegraded":"2023-10-09T06:37:47.184474+0000","last_fullsized":"2023-10-09T06:37:47.184474+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:35:13.249625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.180578+0000","last_change":"2023-10-09T06:37:47.176951+0000","last_active":"2023-10-09T06:37:47.180578+0000","last_peered":"2023-10-09T06:37:47.180578+0000","last_clean":"2023-10-09T06:37:47.180578+0000","last_became_active":"2023-10-09T06:37:47.176489+0000","last_became_peered":"2023-10-09T06:37:47.176489+0000","last_unstale":"2023-10-09T06:37:47.180578+0000","last_undegraded":"2023-10-09T06:37:47.180578+0000","last_fullsized":"2023-10-09T06:37:47.180578+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:21:09.695263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"136'2","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.184592+0000","last_change":"2023-10-09T06:37:46.160289+0000","last_active":"2023-10-09T06:37:47.184592+0000","last_peered":"2023-10-09T06:37:47.184592+0000","last_clean":"2023-10-09T06:37:47.184592+0000","last_became_active":"2023-10-09T06:37:46.160048+0000","last_became_peered":"2023-10-09T06:37:46.160048+0000","last_unstale":"2023-10-09T06:37:47.184592+0000","last_undegraded":"2023-10-09T06:37:47.184592+0000","last_fullsized":"2023-10-09T06:37:47.184592+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:53:17.680797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.184017+0000","last_change":"2023-10-09T06:37:47.176374+0000","last_active":"2023-10-09T06:37:47.184017+0000","last_peered":"2023-10-09T06:37:47.184017+0000","last_clean":"2023-10-09T06:37:47.184017+0000","last_became_active":"2023-10-09T06:37:47.176091+0000","last_became_peered":"2023-10-09T06:37:47.176091+0000","last_unstale":"2023-10-09T06:37:47.184017+0000","last_undegraded":"2023-10-09T06:37:47.184017+0000","last_fullsized":"2023-10-09T06:37:47.184017+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:26:39.377444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"136'2","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.185061+0000","last_change":"2023-10-09T06:37:46.158542+0000","last_active":"2023-10-09T06:37:47.185061+0000","last_peered":"2023-10-09T06:37:47.185061+0000","last_clean":"2023-10-09T06:37:47.185061+0000","last_became_active":"2023-10-09T06:37:46.158244+0000","last_became_peered":"2023-10-09T06:37:46.158244+0000","last_unstale":"2023-10-09T06:37:47.185061+0000","last_undegraded":"2023-10-09T06:37:47.185061+0000","last_fullsized":"2023-10-09T06:37:47.185061+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:01:22.251808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.181431+0000","last_change":"2023-10-09T06:37:47.177058+0000","last_active":"2023-10-09T06:37:47.181431+0000","last_peered":"2023-10-09T06:37:47.181431+0000","last_clean":"2023-10-09T06:37:47.181431+0000","last_became_active":"2023-10-09T06:37:47.176789+0000","last_became_peered":"2023-10-09T06:37:47.176789+0000","last_unstale":"2023-10-09T06:37:47.181431+0000","last_undegraded":"2023-10-09T06:37:47.181431+0000","last_fullsized":"2023-10-09T06:37:47.181431+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:49:37.191442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.187985+0000","last_change":"2023-10-09T06:37:46.165933+0000","last_active":"2023-10-09T06:37:47.187985+0000","last_peered":"2023-10-09T06:37:47.187985+0000","last_clean":"2023-10-09T06:37:47.187985+0000","last_became_active":"2023-10-09T06:37:46.165686+0000","last_became_peered":"2023-10-09T06:37:46.165686+0000","last_unstale":"2023-10-09T06:37:47.187985+0000","last_undegraded":"2023-10-09T06:37:47.187985+0000","last_fullsized":"2023-10-09T06:37:47.187985+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:15:10.830012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.178161+0000","last_change":"2023-10-09T06:37:47.174344+0000","last_active":"2023-10-09T06:37:47.178161+0000","last_peered":"2023-10-09T06:37:47.178161+0000","last_clean":"2023-10-09T06:37:47.178161+0000","last_became_active":"2023-10-09T06:37:47.174096+0000","last_became_peered":"2023-10-09T06:37:47.174096+0000","last_unstale":"2023-10-09T06:37:47.178161+0000","last_undegraded":"2023-10-09T06:37:47.178161+0000","last_fullsized":"2023-10-09T06:37:47.178161+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:25:23.569565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.186181+0000","last_change":"2023-10-09T06:37:46.162339+0000","last_active":"2023-10-09T06:37:47.186181+0000","last_peered":"2023-10-09T06:37:47.186181+0000","last_clean":"2023-10-09T06:37:47.186181+0000","last_became_active":"2023-10-09T06:37:46.162087+0000","last_became_peered":"2023-10-09T06:37:46.162087+0000","last_unstale":"2023-10-09T06:37:47.186181+0000","last_undegraded":"2023-10-09T06:37:47.186181+0000","last_fullsized":"2023-10-09T06:37:47.186181+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:17:47.143130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.180557+0000","last_change":"2023-10-09T06:37:47.175964+0000","last_active":"2023-10-09T06:37:47.180557+0000","last_peered":"2023-10-09T06:37:47.180557+0000","last_clean":"2023-10-09T06:37:47.180557+0000","last_became_active":"2023-10-09T06:37:47.175682+0000","last_became_peered":"2023-10-09T06:37:47.175682+0000","last_unstale":"2023-10-09T06:37:47.180557+0000","last_undegraded":"2023-10-09T06:37:47.180557+0000","last_fullsized":"2023-10-09T06:37:47.180557+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:12:16.269071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.166048+0000","last_change":"2023-10-09T06:37:46.166420+0000","last_active":"2023-10-09T06:37:47.166048+0000","last_peered":"2023-10-09T06:37:47.166048+0000","last_clean":"2023-10-09T06:37:47.166048+0000","last_became_active":"2023-10-09T06:37:46.166212+0000","last_became_peered":"2023-10-09T06:37:46.166212+0000","last_unstale":"2023-10-09T06:37:47.166048+0000","last_undegraded":"2023-10-09T06:37:47.166048+0000","last_fullsized":"2023-10-09T06:37:47.166048+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:48:22.393545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.181823+0000","last_change":"2023-10-09T06:37:47.176452+0000","last_active":"2023-10-09T06:37:47.181823+0000","last_peered":"2023-10-09T06:37:47.181823+0000","last_clean":"2023-10-09T06:37:47.181823+0000","last_became_active":"2023-10-09T06:37:47.176177+0000","last_became_peered":"2023-10-09T06:37:47.176177+0000","last_unstale":"2023-10-09T06:37:47.181823+0000","last_undegraded":"2023-10-09T06:37:47.181823+0000","last_fullsized":"2023-10-09T06:37:47.181823+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:37:25.126126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.164669+0000","last_change":"2023-10-09T06:37:46.156627+0000","last_active":"2023-10-09T06:37:47.164669+0000","last_peered":"2023-10-09T06:37:47.164669+0000","last_clean":"2023-10-09T06:37:47.164669+0000","last_became_active":"2023-10-09T06:37:46.156313+0000","last_became_peered":"2023-10-09T06:37:46.156313+0000","last_unstale":"2023-10-09T06:37:47.164669+0000","last_undegraded":"2023-10-09T06:37:47.164669+0000","last_fullsized":"2023-10-09T06:37:47.164669+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:39:39.850042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.177799+0000","last_change":"2023-10-09T06:37:47.173116+0000","last_active":"2023-10-09T06:37:47.177799+0000","last_peered":"2023-10-09T06:37:47.177799+0000","last_clean":"2023-10-09T06:37:47.177799+0000","last_became_active":"2023-10-09T06:37:47.172862+0000","last_became_peered":"2023-10-09T06:37:47.172862+0000","last_unstale":"2023-10-09T06:37:47.177799+0000","last_undegraded":"2023-10-09T06:37:47.177799+0000","last_fullsized":"2023-10-09T06:37:47.177799+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:49:25.010643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.168773+0000","last_change":"2023-10-09T06:37:46.157544+0000","last_active":"2023-10-09T06:37:47.168773+0000","last_peered":"2023-10-09T06:37:47.168773+0000","last_clean":"2023-10-09T06:37:47.168773+0000","last_became_active":"2023-10-09T06:37:46.157300+0000","last_became_peered":"2023-10-09T06:37:46.157300+0000","last_unstale":"2023-10-09T06:37:47.168773+0000","last_undegraded":"2023-10-09T06:37:47.168773+0000","last_fullsized":"2023-10-09T06:37:47.168773+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:35:20.412002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.180560+0000","last_change":"2023-10-09T06:37:47.176790+0000","last_active":"2023-10-09T06:37:47.180560+0000","last_peered":"2023-10-09T06:37:47.180560+0000","last_clean":"2023-10-09T06:37:47.180560+0000","last_became_active":"2023-10-09T06:37:47.176119+0000","last_became_peered":"2023-10-09T06:37:47.176119+0000","last_unstale":"2023-10-09T06:37:47.180560+0000","last_undegraded":"2023-10-09T06:37:47.180560+0000","last_fullsized":"2023-10-09T06:37:47.180560+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:36:33.812346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"136'3","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.188047+0000","last_change":"2023-10-09T06:37:46.163808+0000","last_active":"2023-10-09T06:37:47.188047+0000","last_peered":"2023-10-09T06:37:47.188047+0000","last_clean":"2023-10-09T06:37:47.188047+0000","last_became_active":"2023-10-09T06:37:46.163515+0000","last_became_peered":"2023-10-09T06:37:46.163515+0000","last_unstale":"2023-10-09T06:37:47.188047+0000","last_undegraded":"2023-10-09T06:37:47.188047+0000","last_fullsized":"2023-10-09T06:37:47.188047+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:11:07.149616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.181444+0000","last_change":"2023-10-09T06:37:47.176956+0000","last_active":"2023-10-09T06:37:47.181444+0000","last_peered":"2023-10-09T06:37:47.181444+0000","last_clean":"2023-10-09T06:37:47.181444+0000","last_became_active":"2023-10-09T06:37:47.176729+0000","last_became_peered":"2023-10-09T06:37:47.176729+0000","last_unstale":"2023-10-09T06:37:47.181444+0000","last_undegraded":"2023-10-09T06:37:47.181444+0000","last_fullsized":"2023-10-09T06:37:47.181444+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:16:00.779605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"136'6","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.189418+0000","last_change":"2023-10-09T06:37:46.157324+0000","last_active":"2023-10-09T06:37:47.189418+0000","last_peered":"2023-10-09T06:37:47.189418+0000","last_clean":"2023-10-09T06:37:47.189418+0000","last_became_active":"2023-10-09T06:37:46.157046+0000","last_became_peered":"2023-10-09T06:37:46.157046+0000","last_unstale":"2023-10-09T06:37:47.189418+0000","last_undegraded":"2023-10-09T06:37:47.189418+0000","last_fullsized":"2023-10-09T06:37:47.189418+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:49:26.936922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.175688+0000","last_change":"2023-10-09T06:37:47.171370+0000","last_active":"2023-10-09T06:37:47.175688+0000","last_peered":"2023-10-09T06:37:47.175688+0000","last_clean":"2023-10-09T06:37:47.175688+0000","last_became_active":"2023-10-09T06:37:47.171044+0000","last_became_peered":"2023-10-09T06:37:47.171044+0000","last_unstale":"2023-10-09T06:37:47.175688+0000","last_undegraded":"2023-10-09T06:37:47.175688+0000","last_fullsized":"2023-10-09T06:37:47.175688+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:08:34.898844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"136'2","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.187120+0000","last_change":"2023-10-09T06:37:46.165381+0000","last_active":"2023-10-09T06:37:47.187120+0000","last_peered":"2023-10-09T06:37:47.187120+0000","last_clean":"2023-10-09T06:37:47.187120+0000","last_became_active":"2023-10-09T06:37:46.165138+0000","last_became_peered":"2023-10-09T06:37:46.165138+0000","last_unstale":"2023-10-09T06:37:47.187120+0000","last_undegraded":"2023-10-09T06:37:47.187120+0000","last_fullsized":"2023-10-09T06:37:47.187120+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:31:32.310904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.171604+0000","last_change":"2023-10-09T06:37:47.167805+0000","last_active":"2023-10-09T06:37:47.171604+0000","last_peered":"2023-10-09T06:37:47.171604+0000","last_clean":"2023-10-09T06:37:47.171604+0000","last_became_active":"2023-10-09T06:37:47.167541+0000","last_became_peered":"2023-10-09T06:37:47.167541+0000","last_unstale":"2023-10-09T06:37:47.171604+0000","last_undegraded":"2023-10-09T06:37:47.171604+0000","last_fullsized":"2023-10-09T06:37:47.171604+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:42:32.915391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.165457+0000","last_change":"2023-10-09T06:37:46.160443+0000","last_active":"2023-10-09T06:37:47.165457+0000","last_peered":"2023-10-09T06:37:47.165457+0000","last_clean":"2023-10-09T06:37:47.165457+0000","last_became_active":"2023-10-09T06:37:46.160187+0000","last_became_peered":"2023-10-09T06:37:46.160187+0000","last_unstale":"2023-10-09T06:37:47.165457+0000","last_undegraded":"2023-10-09T06:37:47.165457+0000","last_fullsized":"2023-10-09T06:37:47.165457+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:53:17.680797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.178980+0000","last_change":"2023-10-09T06:37:47.174266+0000","last_active":"2023-10-09T06:37:47.178980+0000","last_peered":"2023-10-09T06:37:47.178980+0000","last_clean":"2023-10-09T06:37:47.178980+0000","last_became_active":"2023-10-09T06:37:47.173887+0000","last_became_peered":"2023-10-09T06:37:47.173887+0000","last_unstale":"2023-10-09T06:37:47.178980+0000","last_undegraded":"2023-10-09T06:37:47.178980+0000","last_fullsized":"2023-10-09T06:37:47.178980+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:39:40.857018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.162913+0000","last_change":"2023-10-09T06:37:46.154153+0000","last_active":"2023-10-09T06:37:47.162913+0000","last_peered":"2023-10-09T06:37:47.162913+0000","last_clean":"2023-10-09T06:37:47.162913+0000","last_became_active":"2023-10-09T06:37:46.153874+0000","last_became_peered":"2023-10-09T06:37:46.153874+0000","last_unstale":"2023-10-09T06:37:47.162913+0000","last_undegraded":"2023-10-09T06:37:47.162913+0000","last_fullsized":"2023-10-09T06:37:47.162913+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:07:08.017060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.181250+0000","last_change":"2023-10-09T06:37:47.175380+0000","last_active":"2023-10-09T06:37:47.181250+0000","last_peered":"2023-10-09T06:37:47.181250+0000","last_clean":"2023-10-09T06:37:47.181250+0000","last_became_active":"2023-10-09T06:37:47.173989+0000","last_became_peered":"2023-10-09T06:37:47.173989+0000","last_unstale":"2023-10-09T06:37:47.181250+0000","last_undegraded":"2023-10-09T06:37:47.181250+0000","last_fullsized":"2023-10-09T06:37:47.181250+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:49:18.350262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"136'8","reported_seq":18,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.187541+0000","last_change":"2023-10-09T06:37:46.166498+0000","last_active":"2023-10-09T06:37:47.187541+0000","last_peered":"2023-10-09T06:37:47.187541+0000","last_clean":"2023-10-09T06:37:47.187541+0000","last_became_active":"2023-10-09T06:37:46.166270+0000","last_became_peered":"2023-10-09T06:37:46.166270+0000","last_unstale":"2023-10-09T06:37:47.187541+0000","last_undegraded":"2023-10-09T06:37:47.187541+0000","last_fullsized":"2023-10-09T06:37:47.187541+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:55:12.397428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.178302+0000","last_change":"2023-10-09T06:37:47.172384+0000","last_active":"2023-10-09T06:37:47.178302+0000","last_peered":"2023-10-09T06:37:47.178302+0000","last_clean":"2023-10-09T06:37:47.178302+0000","last_became_active":"2023-10-09T06:37:47.172113+0000","last_became_peered":"2023-10-09T06:37:47.172113+0000","last_unstale":"2023-10-09T06:37:47.178302+0000","last_undegraded":"2023-10-09T06:37:47.178302+0000","last_fullsized":"2023-10-09T06:37:47.178302+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:20:36.794582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.188181+0000","last_change":"2023-10-09T06:37:46.160269+0000","last_active":"2023-10-09T06:37:47.188181+0000","last_peered":"2023-10-09T06:37:47.188181+0000","last_clean":"2023-10-09T06:37:47.188181+0000","last_became_active":"2023-10-09T06:37:46.160041+0000","last_became_peered":"2023-10-09T06:37:46.160041+0000","last_unstale":"2023-10-09T06:37:47.188181+0000","last_undegraded":"2023-10-09T06:37:47.188181+0000","last_fullsized":"2023-10-09T06:37:47.188181+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:18:32.993627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.179995+0000","last_change":"2023-10-09T06:37:47.175110+0000","last_active":"2023-10-09T06:37:47.179995+0000","last_peered":"2023-10-09T06:37:47.179995+0000","last_clean":"2023-10-09T06:37:47.179995+0000","last_became_active":"2023-10-09T06:37:47.174614+0000","last_became_peered":"2023-10-09T06:37:47.174614+0000","last_unstale":"2023-10-09T06:37:47.179995+0000","last_undegraded":"2023-10-09T06:37:47.179995+0000","last_fullsized":"2023-10-09T06:37:47.179995+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:45:22.508406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"136'2","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.185583+0000","last_change":"2023-10-09T06:37:46.155690+0000","last_active":"2023-10-09T06:37:47.185583+0000","last_peered":"2023-10-09T06:37:47.185583+0000","last_clean":"2023-10-09T06:37:47.185583+0000","last_became_active":"2023-10-09T06:37:46.155411+0000","last_became_peered":"2023-10-09T06:37:46.155411+0000","last_unstale":"2023-10-09T06:37:47.185583+0000","last_undegraded":"2023-10-09T06:37:47.185583+0000","last_fullsized":"2023-10-09T06:37:47.185583+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:48:22.393545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.184302+0000","last_change":"2023-10-09T06:37:47.180699+0000","last_active":"2023-10-09T06:37:47.184302+0000","last_peered":"2023-10-09T06:37:47.184302+0000","last_clean":"2023-10-09T06:37:47.184302+0000","last_became_active":"2023-10-09T06:37:47.180421+0000","last_became_peered":"2023-10-09T06:37:47.180421+0000","last_unstale":"2023-10-09T06:37:47.184302+0000","last_undegraded":"2023-10-09T06:37:47.184302+0000","last_fullsized":"2023-10-09T06:37:47.184302+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:48:25.266457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.185915+0000","last_change":"2023-10-09T06:37:46.166713+0000","last_active":"2023-10-09T06:37:47.185915+0000","last_peered":"2023-10-09T06:37:47.185915+0000","last_clean":"2023-10-09T06:37:47.185915+0000","last_became_active":"2023-10-09T06:37:46.166490+0000","last_became_peered":"2023-10-09T06:37:46.166490+0000","last_unstale":"2023-10-09T06:37:47.185915+0000","last_undegraded":"2023-10-09T06:37:47.185915+0000","last_fullsized":"2023-10-09T06:37:47.185915+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:07:08.017060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.174796+0000","last_change":"2023-10-09T06:37:47.172447+0000","last_active":"2023-10-09T06:37:47.174796+0000","last_peered":"2023-10-09T06:37:47.174796+0000","last_clean":"2023-10-09T06:37:47.174796+0000","last_became_active":"2023-10-09T06:37:47.172165+0000","last_became_peered":"2023-10-09T06:37:47.172165+0000","last_unstale":"2023-10-09T06:37:47.174796+0000","last_undegraded":"2023-10-09T06:37:47.174796+0000","last_fullsized":"2023-10-09T06:37:47.174796+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:30:11.937291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.164417+0000","last_change":"2023-10-09T06:37:46.160756+0000","last_active":"2023-10-09T06:37:47.164417+0000","last_peered":"2023-10-09T06:37:47.164417+0000","last_clean":"2023-10-09T06:37:47.164417+0000","last_became_active":"2023-10-09T06:37:46.160492+0000","last_became_peered":"2023-10-09T06:37:46.160492+0000","last_unstale":"2023-10-09T06:37:47.164417+0000","last_undegraded":"2023-10-09T06:37:47.164417+0000","last_fullsized":"2023-10-09T06:37:47.164417+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:03:11.419579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.174987+0000","last_change":"2023-10-09T06:37:47.169100+0000","last_active":"2023-10-09T06:37:47.174987+0000","last_peered":"2023-10-09T06:37:47.174987+0000","last_clean":"2023-10-09T06:37:47.174987+0000","last_became_active":"2023-10-09T06:37:47.168860+0000","last_became_peered":"2023-10-09T06:37:47.168860+0000","last_unstale":"2023-10-09T06:37:47.174987+0000","last_undegraded":"2023-10-09T06:37:47.174987+0000","last_fullsized":"2023-10-09T06:37:47.174987+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:49:54.766563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"136'2","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.187817+0000","last_change":"2023-10-09T06:37:46.161231+0000","last_active":"2023-10-09T06:37:47.187817+0000","last_peered":"2023-10-09T06:37:47.187817+0000","last_clean":"2023-10-09T06:37:47.187817+0000","last_became_active":"2023-10-09T06:37:46.160938+0000","last_became_peered":"2023-10-09T06:37:46.160938+0000","last_unstale":"2023-10-09T06:37:47.187817+0000","last_undegraded":"2023-10-09T06:37:47.187817+0000","last_fullsized":"2023-10-09T06:37:47.187817+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:34:05.362873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.181175+0000","last_change":"2023-10-09T06:37:47.177819+0000","last_active":"2023-10-09T06:37:47.181175+0000","last_peered":"2023-10-09T06:37:47.181175+0000","last_clean":"2023-10-09T06:37:47.181175+0000","last_became_active":"2023-10-09T06:37:47.176509+0000","last_became_peered":"2023-10-09T06:37:47.176509+0000","last_unstale":"2023-10-09T06:37:47.181175+0000","last_undegraded":"2023-10-09T06:37:47.181175+0000","last_fullsized":"2023-10-09T06:37:47.181175+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:31:58.529087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.168299+0000","last_change":"2023-10-09T06:37:46.159376+0000","last_active":"2023-10-09T06:37:47.168299+0000","last_peered":"2023-10-09T06:37:47.168299+0000","last_clean":"2023-10-09T06:37:47.168299+0000","last_became_active":"2023-10-09T06:37:46.159093+0000","last_became_peered":"2023-10-09T06:37:46.159093+0000","last_unstale":"2023-10-09T06:37:47.168299+0000","last_undegraded":"2023-10-09T06:37:47.168299+0000","last_fullsized":"2023-10-09T06:37:47.168299+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:04:38.242742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.180813+0000","last_change":"2023-10-09T06:37:47.178651+0000","last_active":"2023-10-09T06:37:47.180813+0000","last_peered":"2023-10-09T06:37:47.180813+0000","last_clean":"2023-10-09T06:37:47.180813+0000","last_became_active":"2023-10-09T06:37:47.178301+0000","last_became_peered":"2023-10-09T06:37:47.178301+0000","last_unstale":"2023-10-09T06:37:47.180813+0000","last_undegraded":"2023-10-09T06:37:47.180813+0000","last_fullsized":"2023-10-09T06:37:47.180813+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:45:22.508406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.159816+0000","last_change":"2023-10-09T06:37:46.161637+0000","last_active":"2023-10-09T06:37:47.159816+0000","last_peered":"2023-10-09T06:37:47.159816+0000","last_clean":"2023-10-09T06:37:47.159816+0000","last_became_active":"2023-10-09T06:37:46.161327+0000","last_became_peered":"2023-10-09T06:37:46.161327+0000","last_unstale":"2023-10-09T06:37:47.159816+0000","last_undegraded":"2023-10-09T06:37:47.159816+0000","last_fullsized":"2023-10-09T06:37:47.159816+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:00:51.883913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.180692+0000","last_change":"2023-10-09T06:37:47.172441+0000","last_active":"2023-10-09T06:37:47.180692+0000","last_peered":"2023-10-09T06:37:47.180692+0000","last_clean":"2023-10-09T06:37:47.180692+0000","last_became_active":"2023-10-09T06:37:47.172195+0000","last_became_peered":"2023-10-09T06:37:47.172195+0000","last_unstale":"2023-10-09T06:37:47.180692+0000","last_undegraded":"2023-10-09T06:37:47.180692+0000","last_fullsized":"2023-10-09T06:37:47.180692+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:09:28.407433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.165963+0000","last_change":"2023-10-09T06:37:46.152589+0000","last_active":"2023-10-09T06:37:47.165963+0000","last_peered":"2023-10-09T06:37:47.165963+0000","last_clean":"2023-10-09T06:37:47.165963+0000","last_became_active":"2023-10-09T06:37:46.152321+0000","last_became_peered":"2023-10-09T06:37:46.152321+0000","last_unstale":"2023-10-09T06:37:47.165963+0000","last_undegraded":"2023-10-09T06:37:47.165963+0000","last_fullsized":"2023-10-09T06:37:47.165963+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:56:59.799505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.180643+0000","last_change":"2023-10-09T06:37:47.171644+0000","last_active":"2023-10-09T06:37:47.180643+0000","last_peered":"2023-10-09T06:37:47.180643+0000","last_clean":"2023-10-09T06:37:47.180643+0000","last_became_active":"2023-10-09T06:37:47.171443+0000","last_became_peered":"2023-10-09T06:37:47.171443+0000","last_unstale":"2023-10-09T06:37:47.180643+0000","last_undegraded":"2023-10-09T06:37:47.180643+0000","last_fullsized":"2023-10-09T06:37:47.180643+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:33:37.869895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"136'2","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.188359+0000","last_change":"2023-10-09T06:37:46.161611+0000","last_active":"2023-10-09T06:37:47.188359+0000","last_peered":"2023-10-09T06:37:47.188359+0000","last_clean":"2023-10-09T06:37:47.188359+0000","last_became_active":"2023-10-09T06:37:46.161432+0000","last_became_peered":"2023-10-09T06:37:46.161432+0000","last_unstale":"2023-10-09T06:37:47.188359+0000","last_undegraded":"2023-10-09T06:37:47.188359+0000","last_fullsized":"2023-10-09T06:37:47.188359+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:20:25.547981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.181923+0000","last_change":"2023-10-09T06:37:47.177207+0000","last_active":"2023-10-09T06:37:47.181923+0000","last_peered":"2023-10-09T06:37:47.181923+0000","last_clean":"2023-10-09T06:37:47.181923+0000","last_became_active":"2023-10-09T06:37:47.176958+0000","last_became_peered":"2023-10-09T06:37:47.176958+0000","last_unstale":"2023-10-09T06:37:47.181923+0000","last_undegraded":"2023-10-09T06:37:47.181923+0000","last_fullsized":"2023-10-09T06:37:47.181923+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:59:24.334461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.164764+0000","last_change":"2023-10-09T06:37:46.165227+0000","last_active":"2023-10-09T06:37:47.164764+0000","last_peered":"2023-10-09T06:37:47.164764+0000","last_clean":"2023-10-09T06:37:47.164764+0000","last_became_active":"2023-10-09T06:37:46.164959+0000","last_became_peered":"2023-10-09T06:37:46.164959+0000","last_unstale":"2023-10-09T06:37:47.164764+0000","last_undegraded":"2023-10-09T06:37:47.164764+0000","last_fullsized":"2023-10-09T06:37:47.164764+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:59:31.649021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.182525+0000","last_change":"2023-10-09T06:37:47.176879+0000","last_active":"2023-10-09T06:37:47.182525+0000","last_peered":"2023-10-09T06:37:47.182525+0000","last_clean":"2023-10-09T06:37:47.182525+0000","last_became_active":"2023-10-09T06:37:47.176496+0000","last_became_peered":"2023-10-09T06:37:47.176496+0000","last_unstale":"2023-10-09T06:37:47.182525+0000","last_undegraded":"2023-10-09T06:37:47.182525+0000","last_fullsized":"2023-10-09T06:37:47.182525+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:23:25.125922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.162819+0000","last_change":"2023-10-09T06:37:46.165805+0000","last_active":"2023-10-09T06:37:47.162819+0000","last_peered":"2023-10-09T06:37:47.162819+0000","last_clean":"2023-10-09T06:37:47.162819+0000","last_became_active":"2023-10-09T06:37:46.165574+0000","last_became_peered":"2023-10-09T06:37:46.165574+0000","last_unstale":"2023-10-09T06:37:47.162819+0000","last_undegraded":"2023-10-09T06:37:47.162819+0000","last_fullsized":"2023-10-09T06:37:47.162819+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:04:18.850838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.185471+0000","last_change":"2023-10-09T06:37:47.178939+0000","last_active":"2023-10-09T06:37:47.185471+0000","last_peered":"2023-10-09T06:37:47.185471+0000","last_clean":"2023-10-09T06:37:47.185471+0000","last_became_active":"2023-10-09T06:37:47.178711+0000","last_became_peered":"2023-10-09T06:37:47.178711+0000","last_unstale":"2023-10-09T06:37:47.185471+0000","last_undegraded":"2023-10-09T06:37:47.185471+0000","last_fullsized":"2023-10-09T06:37:47.185471+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:49:37.191442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.163047+0000","last_change":"2023-10-09T06:37:46.159095+0000","last_active":"2023-10-09T06:37:47.163047+0000","last_peered":"2023-10-09T06:37:47.163047+0000","last_clean":"2023-10-09T06:37:47.163047+0000","last_became_active":"2023-10-09T06:37:46.158810+0000","last_became_peered":"2023-10-09T06:37:46.158810+0000","last_unstale":"2023-10-09T06:37:47.163047+0000","last_undegraded":"2023-10-09T06:37:47.163047+0000","last_fullsized":"2023-10-09T06:37:47.163047+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:20:25.547981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.183308+0000","last_change":"2023-10-09T06:37:47.178617+0000","last_active":"2023-10-09T06:37:47.183308+0000","last_peered":"2023-10-09T06:37:47.183308+0000","last_clean":"2023-10-09T06:37:47.183308+0000","last_became_active":"2023-10-09T06:37:47.178364+0000","last_became_peered":"2023-10-09T06:37:47.178364+0000","last_unstale":"2023-10-09T06:37:47.183308+0000","last_undegraded":"2023-10-09T06:37:47.183308+0000","last_fullsized":"2023-10-09T06:37:47.183308+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:49:18.350262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.165899+0000","last_change":"2023-10-09T06:37:46.158783+0000","last_active":"2023-10-09T06:37:47.165899+0000","last_peered":"2023-10-09T06:37:47.165899+0000","last_clean":"2023-10-09T06:37:47.165899+0000","last_became_active":"2023-10-09T06:37:46.158466+0000","last_became_peered":"2023-10-09T06:37:46.158466+0000","last_unstale":"2023-10-09T06:37:47.165899+0000","last_undegraded":"2023-10-09T06:37:47.165899+0000","last_fullsized":"2023-10-09T06:37:47.165899+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:55:08.369151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.177809+0000","last_change":"2023-10-09T06:37:47.175805+0000","last_active":"2023-10-09T06:37:47.177809+0000","last_peered":"2023-10-09T06:37:47.177809+0000","last_clean":"2023-10-09T06:37:47.177809+0000","last_became_active":"2023-10-09T06:37:47.175551+0000","last_became_peered":"2023-10-09T06:37:47.175551+0000","last_unstale":"2023-10-09T06:37:47.177809+0000","last_undegraded":"2023-10-09T06:37:47.177809+0000","last_fullsized":"2023-10-09T06:37:47.177809+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:14:48.221558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.165629+0000","last_change":"2023-10-09T06:37:46.154894+0000","last_active":"2023-10-09T06:37:47.165629+0000","last_peered":"2023-10-09T06:37:47.165629+0000","last_clean":"2023-10-09T06:37:47.165629+0000","last_became_active":"2023-10-09T06:37:46.154589+0000","last_became_peered":"2023-10-09T06:37:46.154589+0000","last_unstale":"2023-10-09T06:37:47.165629+0000","last_undegraded":"2023-10-09T06:37:47.165629+0000","last_fullsized":"2023-10-09T06:37:47.165629+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:58:53.461920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.172308+0000","last_change":"2023-10-09T06:37:47.168531+0000","last_active":"2023-10-09T06:37:47.172308+0000","last_peered":"2023-10-09T06:37:47.172308+0000","last_clean":"2023-10-09T06:37:47.172308+0000","last_became_active":"2023-10-09T06:37:47.168206+0000","last_became_peered":"2023-10-09T06:37:47.168206+0000","last_unstale":"2023-10-09T06:37:47.172308+0000","last_undegraded":"2023-10-09T06:37:47.172308+0000","last_fullsized":"2023-10-09T06:37:47.172308+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:22:24.566358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.163873+0000","last_change":"2023-10-09T06:37:46.152044+0000","last_active":"2023-10-09T06:37:47.163873+0000","last_peered":"2023-10-09T06:37:47.163873+0000","last_clean":"2023-10-09T06:37:47.163873+0000","last_became_active":"2023-10-09T06:37:46.151315+0000","last_became_peered":"2023-10-09T06:37:46.151315+0000","last_unstale":"2023-10-09T06:37:47.163873+0000","last_undegraded":"2023-10-09T06:37:47.163873+0000","last_fullsized":"2023-10-09T06:37:47.163873+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:24:27.771345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.180710+0000","last_change":"2023-10-09T06:37:47.173604+0000","last_active":"2023-10-09T06:37:47.180710+0000","last_peered":"2023-10-09T06:37:47.180710+0000","last_clean":"2023-10-09T06:37:47.180710+0000","last_became_active":"2023-10-09T06:37:47.173322+0000","last_became_peered":"2023-10-09T06:37:47.173322+0000","last_unstale":"2023-10-09T06:37:47.180710+0000","last_undegraded":"2023-10-09T06:37:47.180710+0000","last_fullsized":"2023-10-09T06:37:47.180710+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:35:39.385390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.159303+0000","last_change":"2023-10-09T06:37:46.159703+0000","last_active":"2023-10-09T06:37:47.159303+0000","last_peered":"2023-10-09T06:37:47.159303+0000","last_clean":"2023-10-09T06:37:47.159303+0000","last_became_active":"2023-10-09T06:37:46.159455+0000","last_became_peered":"2023-10-09T06:37:46.159455+0000","last_unstale":"2023-10-09T06:37:47.159303+0000","last_undegraded":"2023-10-09T06:37:47.159303+0000","last_fullsized":"2023-10-09T06:37:47.159303+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:19:19.201484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.178102+0000","last_change":"2023-10-09T06:37:47.170391+0000","last_active":"2023-10-09T06:37:47.178102+0000","last_peered":"2023-10-09T06:37:47.178102+0000","last_clean":"2023-10-09T06:37:47.178102+0000","last_became_active":"2023-10-09T06:37:47.170131+0000","last_became_peered":"2023-10-09T06:37:47.170131+0000","last_unstale":"2023-10-09T06:37:47.178102+0000","last_undegraded":"2023-10-09T06:37:47.178102+0000","last_fullsized":"2023-10-09T06:37:47.178102+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:46.140387+0000","last_clean_scrub_stamp":"2023-10-09T06:37:46.140387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:25:23.569565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.171753+0000","last_change":"2023-10-09T06:37:46.159778+0000","last_active":"2023-10-09T06:37:47.171753+0000","last_peered":"2023-10-09T06:37:47.171753+0000","last_clean":"2023-10-09T06:37:47.171753+0000","last_became_active":"2023-10-09T06:37:46.159520+0000","last_became_peered":"2023-10-09T06:37:46.159520+0000","last_unstale":"2023-10-09T06:37:47.171753+0000","last_undegraded":"2023-10-09T06:37:47.171753+0000","last_fullsized":"2023-10-09T06:37:47.171753+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:37:45.133411+0000","last_clean_scrub_stamp":"2023-10-09T06:37:45.133411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:04:18.850838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":259,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-09T06:37:47.166310+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:37:47.166310+0000","last_peered":"2023-10-09T06:37:47.166310+0000","last_clean":"2023-10-09T06:37:47.166310+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:37:47.166310+0000","last_undegraded":"2023-10-09T06:37:47.166310+0000","last_fullsized":"2023-10-09T06:37:47.166310+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574940,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303664,"kb_used_data":2280,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068176,"statfs":{"total":96636764160,"available":96325812224,"internally_reserved":0,"allocated":2334720,"data_stored":1363746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52000000000000002,"15min":0.50700000000000001},"min":{"1min":0.34799999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.37,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67600000000000005,"5min":0.753,"15min":0.753},"last":0.438}]},{"osd":1,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.503,"15min":0.51900000000000002},"min":{"1min":0.33600000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.74299999999999999,"5min":0.77400000000000002,"15min":1.2030000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.378,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.55100000000000005}]},{"osd":2,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53200000000000003,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75800000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.44,"5min":0.36399999999999999,"15min":0.315},"max":{"1min":0.77900000000000003,"5min":0.80700000000000005,"15min":0.83099999999999996},"last":0.48099999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53800000000000003,"15min":0.55000000000000004},"min":{"1min":0.41199999999999998,"5min":0.35899999999999999,"15min":0.33500000000000002},"max":{"1min":0.68600000000000005,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55600000000000005,"15min":0.59199999999999997},"min":{"1min":0.45300000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.754,"5min":0.82099999999999995,"15min":1.1479999999999999},"last":0.53500000000000003}]},{"osd":4,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.48199999999999998,"15min":0.51900000000000002},"min":{"1min":0.35599999999999998,"5min":0.30299999999999999,"15min":0.24399999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.93999999999999995},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56599999999999995,"15min":0.60599999999999998},"min":{"1min":0.36799999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76700000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.69099999999999995}]},{"osd":5,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52500000000000002,"15min":0.56999999999999995},"min":{"1min":0.36499999999999999,"5min":0.32500000000000001,"15min":0.307},"max":{"1min":0.81999999999999995,"5min":0.85799999999999998,"15min":0.91100000000000003},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59599999999999997,"15min":0.62},"min":{"1min":0.52900000000000003,"5min":0.41699999999999998,"15min":0.34200000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.71199999999999997}]},{"osd":6,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55700000000000005,"15min":0.59299999999999997},"min":{"1min":0.38600000000000001,"5min":0.33500000000000002,"15min":0.28699999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.88600000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.56100000000000005,"15min":0.61399999999999999},"min":{"1min":0.45100000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.84599999999999997},"last":0.66100000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574939,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303076,"kb_used_data":1692,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068764,"statfs":{"total":96636764160,"available":96326414336,"internally_reserved":0,"allocated":1732608,"data_stored":773274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.50800000000000001,"15min":0.502},"min":{"1min":0.35499999999999998,"5min":0.32000000000000001,"15min":0.30399999999999999},"max":{"1min":0.66500000000000004,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.371},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.50700000000000001,"15min":0.51000000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70399999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.49299999999999999}]},{"osd":1,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.502,"15min":0.51200000000000001},"min":{"1min":0.33600000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68300000000000005,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52900000000000003,"15min":0.53200000000000003},"min":{"1min":0.38700000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.77100000000000002,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.437}]},{"osd":2,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.34499999999999997,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.72399999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51100000000000001,"15min":0.52900000000000003},"min":{"1min":0.39000000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.66600000000000004,"5min":0.67500000000000004,"15min":0.752},"last":0.40100000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.38900000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66600000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.54300000000000004,"15min":0.68799999999999994},"min":{"1min":0.40400000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.752,"5min":0.752,"15min":4.7510000000000003},"last":0.41599999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46400000000000002,"15min":0.47699999999999998},"min":{"1min":0.29899999999999999,"5min":0.28199999999999997,"15min":0.27900000000000003},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.76400000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56100000000000005,"15min":0.58099999999999996},"min":{"1min":0.41799999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.77200000000000002,"5min":0.81299999999999994,"15min":0.83699999999999997},"last":0.58999999999999997}]},{"osd":5,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.52100000000000002,"15min":0.53000000000000003},"min":{"1min":0.31900000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.66100000000000003,"5min":0.79200000000000004,"15min":0.81200000000000006},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.59499999999999997},"min":{"1min":0.46899999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76000000000000001,"5min":0.78000000000000003,"15min":0.86599999999999999},"last":0.54300000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55200000000000005,"15min":0.58199999999999996},"min":{"1min":0.41399999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80500000000000005,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58799999999999997,"15min":0.58899999999999997},"min":{"1min":0.46000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78400000000000003,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.55400000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303088,"kb_used_data":1704,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068752,"statfs":{"total":96636764160,"available":96326402048,"internally_reserved":0,"allocated":1744896,"data_stored":774810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.316,"15min":0.311},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.503,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.309},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.34899999999999998,"15min":0.35799999999999998},"min":{"1min":0.28399999999999997,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.51900000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.31900000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.34399999999999997,"15min":0.35099999999999998},"min":{"1min":0.215,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.53700000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.38300000000000001,"15min":0.38500000000000001},"min":{"1min":0.30299999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.55300000000000005,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.34399999999999997}]},{"osd":3,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.36499999999999999,"15min":0.36399999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.55800000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.34599999999999997,"15min":0.53500000000000003},"min":{"1min":0.23400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.40400000000000003,"5min":0.56000000000000005,"15min":5.8529999999999998},"last":0.35199999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58299999999999996,"15min":0.57499999999999996},"min":{"1min":0.36199999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78300000000000003,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.40000000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.81100000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.52300000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57699999999999996,"15min":0.74099999999999999},"min":{"1min":0.36499999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88,"5min":0.88,"15min":5.9459999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61199999999999999,"15min":0.61099999999999999},"min":{"1min":0.38900000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82599999999999996,"5min":0.92700000000000005,"15min":0.93999999999999995},"last":0.55500000000000005}]},{"osd":6,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54900000000000004,"15min":0.71199999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79100000000000004,"5min":0.79600000000000004,"15min":5.9160000000000004},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.379,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84799999999999998,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.44500000000000001}]},{"osd":7,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57199999999999995,"15min":0.58599999999999997},"min":{"1min":0.39000000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.80900000000000005,"5min":0.89500000000000002,"15min":0.97099999999999997},"last":0.439},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57099999999999995,"15min":0.749},"min":{"1min":0.41399999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":5.9729999999999999},"last":0.53900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574941,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303660,"kb_used_data":2276,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068180,"statfs":{"total":96636764160,"available":96325816320,"internally_reserved":0,"allocated":2330624,"data_stored":1363664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.30199999999999999,"15min":0.315},"min":{"1min":0.216,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.433,"5min":0.498,"15min":0.498},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.32100000000000001,"15min":0.34399999999999997},"min":{"1min":0.222,"5min":0.214,"15min":0.214},"max":{"1min":0.497,"5min":0.53600000000000003,"15min":0.94499999999999995},"last":0.39400000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35199999999999998,"15min":0.36199999999999999},"min":{"1min":0.25900000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.50900000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.371,"15min":0.38800000000000001},"min":{"1min":0.30499999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.56499999999999995,"5min":0.69399999999999995,"15min":0.91700000000000004},"last":0.33900000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.32800000000000001,"15min":0.38400000000000001},"min":{"1min":0.23699999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":1.554},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.35999999999999999,"15min":0.39700000000000002},"min":{"1min":0.27700000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54400000000000004,"5min":0.58099999999999996,"15min":0.79000000000000004},"last":0.313}]},{"osd":4,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53600000000000003,"15min":0.55100000000000005},"min":{"1min":0.41199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":1.212},"last":0.62},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.72099999999999997},"min":{"1min":0.42199999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":5.484},"last":0.56499999999999995}]},{"osd":5,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.72499999999999998},"min":{"1min":0.41999999999999998,"5min":0.35799999999999998,"15min":0.35499999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":5.5449999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.83299999999999996,"5min":0.84499999999999997,"15min":1.113},"last":0.67600000000000005}]},{"osd":6,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":1.0529999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59299999999999997,"15min":0.60699999999999998},"min":{"1min":0.45800000000000002,"5min":0.42799999999999999,"15min":0.41599999999999998},"max":{"1min":0.878,"5min":0.878,"15min":1.169},"last":0.65000000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57099999999999995,"15min":0.58899999999999997},"min":{"1min":0.40600000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.78700000000000003,"5min":0.85499999999999998,"15min":1.2390000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56399999999999995,"15min":0.749},"min":{"1min":0.38,"5min":0.375,"15min":0.375},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":5.5800000000000001},"last":0.58199999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574942,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303088,"kb_used_data":1704,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068752,"statfs":{"total":96636764160,"available":96326402048,"internally_reserved":0,"allocated":1744896,"data_stored":773940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.316,"15min":0.35299999999999998},"min":{"1min":0.23200000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":1.4199999999999999},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.55900000000000005},"min":{"1min":0.27500000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.51600000000000001,"5min":0.53800000000000003,"15min":7.2699999999999996},"last":0.27500000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.317,"15min":0.34399999999999997},"min":{"1min":0.23999999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.65400000000000003},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.372,"5min":0.37,"15min":0.61299999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":7.327},"last":0.314}]},{"osd":3,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.36799999999999999,"15min":0.39100000000000001},"min":{"1min":0.29399999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.68600000000000005},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.36799999999999999,"15min":0.39200000000000002},"min":{"1min":0.27900000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.72699999999999998},"last":0.37}]},{"osd":4,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54500000000000004,"15min":0.54400000000000004},"min":{"1min":0.40799999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.78400000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56200000000000006,"15min":0.78600000000000003},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82699999999999996,"5min":0.84699999999999998,"15min":7.2389999999999999},"last":0.38200000000000001}]},{"osd":5,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.58099999999999996,"15min":0.91500000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82699999999999996,"5min":0.83799999999999997,"15min":10.890000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":0.84299999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.45200000000000001}]},{"osd":6,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56399999999999995,"15min":0.79100000000000004},"min":{"1min":0.42999999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.748,"5min":0.77300000000000002,"15min":7.2999999999999998},"last":0.434},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.45800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.46400000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59299999999999997,"15min":0.82199999999999995},"min":{"1min":0.47499999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":7.1879999999999997},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60399999999999998,"15min":0.624},"min":{"1min":0.49399999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.90500000000000003},"last":0.495}]}]},{"osd":3,"up_from":13,"seq":55834574941,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303088,"kb_used_data":1704,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068752,"statfs":{"total":96636764160,"available":96326402048,"internally_reserved":0,"allocated":1744896,"data_stored":773468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.32200000000000001,"15min":0.33200000000000002},"min":{"1min":0.23100000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":5.8700000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.34699999999999998,"15min":0.36699999999999999},"min":{"1min":0.23999999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":5.6769999999999996},"last":0.56200000000000006}]},{"osd":1,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.35799999999999998,"15min":0.36099999999999999},"min":{"1min":0.26800000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":4.0469999999999997},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.372,"15min":0.38800000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":4.1509999999999998},"last":0.54200000000000004}]},{"osd":2,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.35199999999999998,"15min":0.374},"min":{"1min":0.29999999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.50800000000000001,"5min":0.56799999999999995,"15min":3.1139999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.373,"15min":0.38600000000000001},"min":{"1min":0.28100000000000003,"5min":0.26100000000000001,"15min":0.25600000000000001},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":3.3180000000000001},"last":0.57699999999999996}]},{"osd":4,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55800000000000005,"15min":0.57099999999999995},"min":{"1min":0.38100000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":3.0899999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56499999999999995,"15min":0.56799999999999995},"min":{"1min":0.40300000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76400000000000001,"5min":0.91100000000000003,"15min":3.0710000000000002},"last":0.72299999999999998}]},{"osd":5,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57499999999999996,"15min":0.58899999999999997},"min":{"1min":0.39100000000000001,"5min":0.34599999999999997,"15min":0.32700000000000001},"max":{"1min":0.877,"5min":0.877,"15min":6.2300000000000004},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.441,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79800000000000004,"5min":0.89500000000000002,"15min":3.0219999999999998},"last":0.68400000000000005}]},{"osd":6,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.44500000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":2.6850000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.65200000000000002},"min":{"1min":0.43099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":6.5970000000000004},"last":0.746}]},{"osd":7,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60699999999999998,"15min":0.61399999999999999},"min":{"1min":0.44700000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86099999999999999,"5min":0.93300000000000005,"15min":4.2629999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59999999999999998,"15min":0.61299999999999999},"min":{"1min":0.40500000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.82899999999999996,"5min":0.97199999999999998,"15min":7.3650000000000002},"last":0.69599999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574940,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303092,"kb_used_data":1708,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068748,"statfs":{"total":96636764160,"available":96326397952,"internally_reserved":0,"allocated":1748992,"data_stored":775364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.51400000000000001,"15min":0.55300000000000005},"min":{"1min":0.318,"5min":0.28100000000000003,"15min":0.27000000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":1.083},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.35699999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.79900000000000004,"5min":0.97799999999999998,"15min":1.054},"last":0.68400000000000005}]},{"osd":1,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.39900000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.501,"5min":0.50700000000000001,"15min":0.49399999999999999},"min":{"1min":0.33800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.76300000000000001,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.75700000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52700000000000002,"15min":0.53900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.32900000000000001},"max":{"1min":0.76600000000000001,"5min":0.82099999999999995,"15min":1.0249999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55700000000000005,"15min":0.59099999999999997},"min":{"1min":0.39400000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.93799999999999994},"last":0.623}]},{"osd":3,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52900000000000003,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.32200000000000001,"15min":0.29199999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.98499999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57699999999999996,"15min":0.58799999999999997},"min":{"1min":0.38700000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.1319999999999999},"last":0.77500000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51600000000000001,"15min":0.55100000000000005},"min":{"1min":0.31900000000000001,"5min":0.30399999999999999,"15min":0.27300000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58199999999999996,"15min":0.60599999999999998},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.35099999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.97999999999999998},"last":0.60099999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.56799999999999995},"min":{"1min":0.41599999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.76800000000000002,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.59799999999999998},"min":{"1min":0.38700000000000001,"5min":0.34999999999999998,"15min":0.32700000000000001},"max":{"1min":0.85599999999999998,"5min":1.01,"15min":1.01},"last":0.53700000000000003}]},{"osd":7,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54300000000000004,"15min":0.56999999999999995},"min":{"1min":0.40200000000000002,"5min":0.33600000000000002,"15min":0.28299999999999997},"max":{"1min":0.78900000000000003,"5min":0.85599999999999998,"15min":0.90000000000000002},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.60699999999999998,"15min":0.64500000000000002},"min":{"1min":0.36699999999999999,"5min":0.34999999999999998,"15min":0.32200000000000001},"max":{"1min":0.89000000000000001,"5min":0.98999999999999999,"15min":1.175},"last":0.65900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574940,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303076,"kb_used_data":1692,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068764,"statfs":{"total":96636764160,"available":96326414336,"internally_reserved":0,"allocated":1732608,"data_stored":773274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.503,"15min":0.50800000000000001},"min":{"1min":0.38800000000000001,"5min":0.29299999999999998,"15min":0.28599999999999998},"max":{"1min":0.67900000000000005,"5min":0.71399999999999997,"15min":0.86599999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51800000000000002,"15min":0.51400000000000001},"min":{"1min":0.42499999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.60599999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.52600000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.38400000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.63900000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52300000000000002,"15min":0.53100000000000003},"min":{"1min":0.43099999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72799999999999998,"5min":0.73899999999999999,"15min":0.81599999999999995},"last":0.51200000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54300000000000004,"15min":0.54400000000000004},"min":{"1min":0.41099999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.63900000000000001,"5min":0.72999999999999998,"15min":0.83999999999999997},"last":0.623},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.47199999999999998,"5min":0.40999999999999998,"15min":0.39500000000000002},"max":{"1min":0.72699999999999998,"5min":0.74099999999999999,"15min":0.79900000000000004},"last":0.63300000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.42399999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.66900000000000004,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.54900000000000004,"15min":0.56000000000000005},"min":{"1min":0.42499999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.63500000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.57299999999999995}]},{"osd":4,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.50600000000000001,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.36199999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.80700000000000005},"last":0.47399999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55300000000000005,"15min":0.53600000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.83199999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.36199999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77900000000000003,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.60799999999999998}]},{"osd":7,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.57899999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71499999999999997,"5min":0.79500000000000004,"15min":0.85599999999999998},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.374,"5min":0.35199999999999998,"15min":0.314},"max":{"1min":0.79500000000000004,"5min":0.84499999999999997,"15min":0.90600000000000003},"last":0.53800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:37:56.084 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-09T06:37:56.084 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-09T06:37:56.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.239+0000 7fa096f77700 1 Processor -- start 2023-10-09T06:37:56.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.239+0000 7fa096f77700 1 -- start start 2023-10-09T06:37:56.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.239+0000 7fa096f77700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa0900cca50 0x7fa0900caab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:56.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.239+0000 7fa096f77700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa0900cce20 0x7fa09015a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:56.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.239+0000 7fa096f77700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa090157c50 con 0x7fa09015df40 2023-10-09T06:37:56.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.239+0000 7fa096f77700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa09015d7d0 con 0x7fa0900cce20 2023-10-09T06:37:56.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.239+0000 7fa096f77700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa09015aa20 con 0x7fa0900cca50 2023-10-09T06:37:56.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.240+0000 7fa087fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa0900cce20 0x7fa09015a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:56.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.240+0000 7fa087fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa0900cce20 0x7fa09015a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41766/0 (socket says 172.21.15.163:41766) 2023-10-09T06:37:56.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.240+0000 7fa087fff700 1 -- 172.21.15.163:0/2792926839 learned_addr learned my addr 172.21.15.163:0/2792926839 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:56.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.240+0000 7fa094d13700 1 --2- 172.21.15.163:0/2792926839 >> v2:172.21.15.181:3300/0 conn(0x7fa0900cca50 0x7fa0900caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:56.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.240+0000 7fa087fff700 1 -- 172.21.15.163:0/2792926839 >> v2:172.21.15.181:3300/0 conn(0x7fa0900cca50 msgr2=0x7fa0900caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:56.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.240+0000 7fa087fff700 1 --2- 172.21.15.163:0/2792926839 >> v2:172.21.15.181:3300/0 conn(0x7fa0900cca50 0x7fa0900caab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:56.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.240+0000 7fa087fff700 1 -- 172.21.15.163:0/2792926839 >> v1:172.21.15.181:6789/0 conn(0x7fa09015df40 legacy=0x7fa09006ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:56.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.240+0000 7fa087fff700 1 -- 172.21.15.163:0/2792926839 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09015ad00 con 0x7fa0900cce20 2023-10-09T06:37:56.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.240+0000 7fa087fff700 1 --2- 172.21.15.163:0/2792926839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0900cce20 0x7fa09015a1d0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fa0900691a0 tx=0x7fa080009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45d3bfb18a831e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:56.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.240+0000 7fa0877fe700 1 -- 172.21.15.163:0/2792926839 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa080009e70 con 0x7fa0900cce20 2023-10-09T06:37:56.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.240+0000 7fa0877fe700 1 -- 172.21.15.163:0/2792926839 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa08000b860 con 0x7fa0900cce20 2023-10-09T06:37:56.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.240+0000 7fa0877fe700 1 -- 172.21.15.163:0/2792926839 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08000bc00 con 0x7fa0900cce20 2023-10-09T06:37:56.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.241+0000 7fa096f77700 1 -- 172.21.15.163:0/2792926839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0900cce20 msgr2=0x7fa09015a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:56.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.241+0000 7fa096f77700 1 --2- 172.21.15.163:0/2792926839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0900cce20 0x7fa09015a1d0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fa0900691a0 tx=0x7fa080009250 comp rx=0 tx=0).stop 2023-10-09T06:37:56.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.241+0000 7fa096f77700 1 -- 172.21.15.163:0/2792926839 shutdown_connections 2023-10-09T06:37:56.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.241+0000 7fa096f77700 1 --2- 172.21.15.163:0/2792926839 >> v2:172.21.15.181:3300/0 conn(0x7fa0900cca50 0x7fa0900caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:56.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.241+0000 7fa096f77700 1 --2- 172.21.15.163:0/2792926839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0900cce20 0x7fa09015a1d0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:56.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.241+0000 7fa096f77700 1 -- 172.21.15.163:0/2792926839 >> 172.21.15.163:0/2792926839 conn(0x7fa0900ce5c0 msgr2=0x7fa0900ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:56.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.241+0000 7fa096f77700 1 -- 172.21.15.163:0/2792926839 shutdown_connections 2023-10-09T06:37:56.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.241+0000 7fa096f77700 1 -- 172.21.15.163:0/2792926839 wait complete. 2023-10-09T06:37:56.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.242+0000 7fa096f77700 1 Processor -- start 2023-10-09T06:37:56.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.242+0000 7fa096f77700 1 -- start start 2023-10-09T06:37:56.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.243+0000 7fa096f77700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0900cca50 0x7fa090144de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:56.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.243+0000 7fa096f77700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0900cce20 0x7fa0900c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:56.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.243+0000 7fa096f77700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa09015df40 0x7fa0900c5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:56.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.243+0000 7fa096f77700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0900b31e0 con 0x7fa09015df40 2023-10-09T06:37:56.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.243+0000 7fa096f77700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0900c5bb0 con 0x7fa0900cce20 2023-10-09T06:37:56.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.243+0000 7fa096f77700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0900c5e50 con 0x7fa0900cca50 2023-10-09T06:37:56.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.243+0000 7fa095514700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa09015df40 0x7fa0900c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:56.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.243+0000 7fa095514700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa09015df40 0x7fa0900c5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41774/0 (socket says 172.21.15.163:41774) 2023-10-09T06:37:56.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.243+0000 7fa095514700 1 -- 172.21.15.163:0/1012057907 learned_addr learned my addr 172.21.15.163:0/1012057907 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:56.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.243+0000 7fa087fff700 1 --2- 172.21.15.163:0/1012057907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0900cce20 0x7fa0900c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:56.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.243+0000 7fa094d13700 1 --2- 172.21.15.163:0/1012057907 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0900cca50 0x7fa090144de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:56.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.243+0000 7fa095514700 1 -- 172.21.15.163:0/1012057907 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0900cca50 msgr2=0x7fa090144de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:56.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.243+0000 7fa095514700 1 --2- 172.21.15.163:0/1012057907 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0900cca50 0x7fa090144de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:56.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.243+0000 7fa095514700 1 -- 172.21.15.163:0/1012057907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0900cce20 msgr2=0x7fa0900c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:56.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.243+0000 7fa095514700 1 --2- 172.21.15.163:0/1012057907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0900cce20 0x7fa0900c9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:56.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.243+0000 7fa095514700 1 -- 172.21.15.163:0/1012057907 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0800094f0 con 0x7fa09015df40 2023-10-09T06:37:56.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.243+0000 7fa087fff700 1 --2- 172.21.15.163:0/1012057907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0900cce20 0x7fa0900c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:37:56.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.243+0000 7fa094d13700 1 --2- 172.21.15.163:0/1012057907 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0900cca50 0x7fa090144de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:37:56.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.243+0000 7fa095514700 1 --2- 172.21.15.163:0/1012057907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa09015df40 0x7fa0900c5670 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fa08800c4c0 tx=0x7fa088007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:56.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.244+0000 7fa084ff9700 1 -- 172.21.15.163:0/1012057907 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa088011590 con 0x7fa09015df40 2023-10-09T06:37:56.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.244+0000 7fa084ff9700 1 -- 172.21.15.163:0/1012057907 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa088011710 con 0x7fa09015df40 2023-10-09T06:37:56.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.244+0000 7fa096f77700 1 -- 172.21.15.163:0/1012057907 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0900c6130 con 0x7fa09015df40 2023-10-09T06:37:56.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.244+0000 7fa084ff9700 1 -- 172.21.15.163:0/1012057907 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa088011ad0 con 0x7fa09015df40 2023-10-09T06:37:56.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.244+0000 7fa096f77700 1 -- 172.21.15.163:0/1012057907 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0900c66f0 con 0x7fa09015df40 2023-10-09T06:37:56.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.244+0000 7fa096f77700 1 -- 172.21.15.163:0/1012057907 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0900b3350 con 0x7fa09015df40 2023-10-09T06:37:56.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.245+0000 7fa084ff9700 1 -- 172.21.15.163:0/1012057907 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa08801b3b0 con 0x7fa09015df40 2023-10-09T06:37:56.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.245+0000 7fa084ff9700 1 --2- 172.21.15.163:0/1012057907 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa07c067320 0x7fa07c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:56.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.245+0000 7fa084ff9700 1 -- 172.21.15.163:0/1012057907 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa088083f30 con 0x7fa09015df40 2023-10-09T06:37:56.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.245+0000 7fa094d13700 1 --2- 172.21.15.163:0/1012057907 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa07c067320 0x7fa07c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:56.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.246+0000 7fa094d13700 1 --2- 172.21.15.163:0/1012057907 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa07c067320 0x7fa07c0697d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa08c007780 tx=0x7fa08c00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:56.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.249+0000 7fa084ff9700 1 -- 172.21.15.163:0/1012057907 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa088050bc0 con 0x7fa09015df40 2023-10-09T06:37:56.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.505+0000 7fa096f77700 1 -- 172.21.15.163:0/1012057907 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa090157c50 con 0x7fa09015df40 2023-10-09T06:37:56.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.508+0000 7fa084ff9700 1 -- 172.21.15.163:0/1012057907 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+0 (secure 0 0 0) 0x7fa090157c50 con 0x7fa09015df40 2023-10-09T06:37:56.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.511+0000 7fa096f77700 1 -- 172.21.15.163:0/1012057907 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa07c067320 msgr2=0x7fa07c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:56.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.511+0000 7fa096f77700 1 --2- 172.21.15.163:0/1012057907 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa07c067320 0x7fa07c0697d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa08c007780 tx=0x7fa08c00b040 comp rx=0 tx=0).stop 2023-10-09T06:37:56.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.511+0000 7fa096f77700 1 -- 172.21.15.163:0/1012057907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa09015df40 msgr2=0x7fa0900c5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:56.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.511+0000 7fa096f77700 1 --2- 172.21.15.163:0/1012057907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa09015df40 0x7fa0900c5670 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fa08800c4c0 tx=0x7fa088007f50 comp rx=0 tx=0).stop 2023-10-09T06:37:56.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.511+0000 7fa096f77700 1 -- 172.21.15.163:0/1012057907 shutdown_connections 2023-10-09T06:37:56.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.511+0000 7fa096f77700 1 --2- 172.21.15.163:0/1012057907 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa07c067320 0x7fa07c0697d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:56.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.511+0000 7fa096f77700 1 --2- 172.21.15.163:0/1012057907 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa09015df40 0x7fa0900c5670 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:56.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.511+0000 7fa096f77700 1 --2- 172.21.15.163:0/1012057907 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0900cca50 0x7fa090144de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:56.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.511+0000 7fa096f77700 1 --2- 172.21.15.163:0/1012057907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0900cce20 0x7fa0900c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:56.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.511+0000 7fa096f77700 1 -- 172.21.15.163:0/1012057907 >> 172.21.15.163:0/1012057907 conn(0x7fa0900ce5c0 msgr2=0x7fa0900c3360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:56.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.511+0000 7fa096f77700 1 -- 172.21.15.163:0/1012057907 shutdown_connections 2023-10-09T06:37:56.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.511+0000 7fa096f77700 1 -- 172.21.15.163:0/1012057907 wait complete. 2023-10-09T06:37:56.514 INFO:teuthology.orchestra.run.smithi163.stderr:updated caps for client.0 2023-10-09T06:37:56.524 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:37:56.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.639+0000 7fd92f787700 1 Processor -- start 2023-10-09T06:37:56.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.639+0000 7fd92f787700 1 -- start start 2023-10-09T06:37:56.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.640+0000 7fd92f787700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd928068550 0x7fd928068920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:56.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.640+0000 7fd92f787700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd92815e010 0x7fd92806f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:56.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.640+0000 7fd92f787700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd92806ec70 con 0x7fd92806f640 2023-10-09T06:37:56.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.640+0000 7fd92f787700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9280b3430 con 0x7fd928068550 2023-10-09T06:37:56.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.640+0000 7fd92f787700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9280b2e70 con 0x7fd92815e010 2023-10-09T06:37:56.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.640+0000 7fd92d523700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd928068550 0x7fd928068920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:56.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.640+0000 7fd92d523700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd928068550 0x7fd928068920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41790/0 (socket says 172.21.15.163:41790) 2023-10-09T06:37:56.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.640+0000 7fd92d523700 1 -- 172.21.15.163:0/2032617826 learned_addr learned my addr 172.21.15.163:0/2032617826 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:56.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.640+0000 7fd92cd22700 1 --2- 172.21.15.163:0/2032617826 >> v2:172.21.15.181:3300/0 conn(0x7fd92815e010 0x7fd92806f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:56.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.640+0000 7fd92d523700 1 -- 172.21.15.163:0/2032617826 >> v2:172.21.15.181:3300/0 conn(0x7fd92815e010 msgr2=0x7fd92806f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:56.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.640+0000 7fd92d523700 1 --2- 172.21.15.163:0/2032617826 >> v2:172.21.15.181:3300/0 conn(0x7fd92815e010 0x7fd92806f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:56.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.640+0000 7fd92d523700 1 -- 172.21.15.163:0/2032617826 >> v1:172.21.15.181:6789/0 conn(0x7fd92806f640 legacy=0x7fd92806a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:56.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.640+0000 7fd92d523700 1 -- 172.21.15.163:0/2032617826 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9280b3150 con 0x7fd928068550 2023-10-09T06:37:56.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.640+0000 7fd92cd22700 1 --2- 172.21.15.163:0/2032617826 >> v2:172.21.15.181:3300/0 conn(0x7fd92815e010 0x7fd92806f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:37:56.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.640+0000 7fd92d523700 1 --2- 172.21.15.163:0/2032617826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd928068550 0x7fd928068920 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fd92806d310 tx=0x7fd924009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffeee4ca3a259ab2 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:56.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.640+0000 7fd91ffff700 1 -- 172.21.15.163:0/2032617826 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92400b820 con 0x7fd928068550 2023-10-09T06:37:56.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.640+0000 7fd91ffff700 1 -- 172.21.15.163:0/2032617826 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92400b980 con 0x7fd928068550 2023-10-09T06:37:56.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.640+0000 7fd91ffff700 1 -- 172.21.15.163:0/2032617826 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92400bd20 con 0x7fd928068550 2023-10-09T06:37:56.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.641+0000 7fd92f787700 1 -- 172.21.15.163:0/2032617826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd928068550 msgr2=0x7fd928068920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:56.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.641+0000 7fd92f787700 1 --2- 172.21.15.163:0/2032617826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd928068550 0x7fd928068920 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fd92806d310 tx=0x7fd924009250 comp rx=0 tx=0).stop 2023-10-09T06:37:56.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.641+0000 7fd92f787700 1 -- 172.21.15.163:0/2032617826 shutdown_connections 2023-10-09T06:37:56.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.641+0000 7fd92f787700 1 --2- 172.21.15.163:0/2032617826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd928068550 0x7fd928068920 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:56.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.641+0000 7fd92f787700 1 --2- 172.21.15.163:0/2032617826 >> v2:172.21.15.181:3300/0 conn(0x7fd92815e010 0x7fd92806f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:56.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.641+0000 7fd92f787700 1 -- 172.21.15.163:0/2032617826 >> 172.21.15.163:0/2032617826 conn(0x7fd9280cd480 msgr2=0x7fd9280cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:56.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.641+0000 7fd92f787700 1 -- 172.21.15.163:0/2032617826 shutdown_connections 2023-10-09T06:37:56.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.641+0000 7fd92f787700 1 -- 172.21.15.163:0/2032617826 wait complete. 2023-10-09T06:37:56.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.642+0000 7fd92f787700 1 Processor -- start 2023-10-09T06:37:56.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.642+0000 7fd92f787700 1 -- start start 2023-10-09T06:37:56.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.643+0000 7fd92f787700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd928068550 0x7fd9281470e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:56.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.643+0000 7fd92f787700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd92806f640 0x7fd92815be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:56.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.643+0000 7fd92f787700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd92815e010 0x7fd92815c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:56.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.643+0000 7fd92f787700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92815a1b0 con 0x7fd92815e010 2023-10-09T06:37:56.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.643+0000 7fd92f787700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd928147620 con 0x7fd92806f640 2023-10-09T06:37:56.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.643+0000 7fd92f787700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9281595a0 con 0x7fd928068550 2023-10-09T06:37:56.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.643+0000 7fd92dd24700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd92815e010 0x7fd92815c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:56.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.643+0000 7fd92dd24700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd92815e010 0x7fd92815c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41802/0 (socket says 172.21.15.163:41802) 2023-10-09T06:37:56.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.643+0000 7fd92dd24700 1 -- 172.21.15.163:0/5238664 learned_addr learned my addr 172.21.15.163:0/5238664 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:56.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.643+0000 7fd92cd22700 1 --2- 172.21.15.163:0/5238664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd92806f640 0x7fd92815be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:56.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.643+0000 7fd92d523700 1 --2- 172.21.15.163:0/5238664 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd928068550 0x7fd9281470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:56.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.643+0000 7fd92dd24700 1 -- 172.21.15.163:0/5238664 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd928068550 msgr2=0x7fd9281470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:56.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.643+0000 7fd92dd24700 1 --2- 172.21.15.163:0/5238664 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd928068550 0x7fd9281470e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:56.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.643+0000 7fd92dd24700 1 -- 172.21.15.163:0/5238664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd92806f640 msgr2=0x7fd92815be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:56.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.643+0000 7fd92dd24700 1 --2- 172.21.15.163:0/5238664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd92806f640 0x7fd92815be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:56.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.643+0000 7fd92dd24700 1 -- 172.21.15.163:0/5238664 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9240094f0 con 0x7fd92815e010 2023-10-09T06:37:56.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.644+0000 7fd92dd24700 1 --2- 172.21.15.163:0/5238664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd92815e010 0x7fd92815c380 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fd92000c930 tx=0x7fd9200026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:56.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.644+0000 7fd91d7fa700 1 -- 172.21.15.163:0/5238664 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92000ce70 con 0x7fd92815e010 2023-10-09T06:37:56.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.644+0000 7fd91d7fa700 1 -- 172.21.15.163:0/5238664 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92000a7e0 con 0x7fd92815e010 2023-10-09T06:37:56.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.644+0000 7fd92f787700 1 -- 172.21.15.163:0/5238664 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd928159880 con 0x7fd92815e010 2023-10-09T06:37:56.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.644+0000 7fd91d7fa700 1 -- 172.21.15.163:0/5238664 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92000aba0 con 0x7fd92815e010 2023-10-09T06:37:56.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.644+0000 7fd92f787700 1 -- 172.21.15.163:0/5238664 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd928159e40 con 0x7fd92815e010 2023-10-09T06:37:56.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.645+0000 7fd92f787700 1 -- 172.21.15.163:0/5238664 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd92804c3c0 con 0x7fd92815e010 2023-10-09T06:37:56.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.645+0000 7fd91d7fa700 1 -- 172.21.15.163:0/5238664 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd920012420 con 0x7fd92815e010 2023-10-09T06:37:56.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.645+0000 7fd91d7fa700 1 --2- 172.21.15.163:0/5238664 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd90c067320 0x7fd90c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:56.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.645+0000 7fd91d7fa700 1 -- 172.21.15.163:0/5238664 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd920085140 con 0x7fd92815e010 2023-10-09T06:37:56.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.646+0000 7fd92d523700 1 --2- 172.21.15.163:0/5238664 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd90c067320 0x7fd90c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:56.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.646+0000 7fd92d523700 1 --2- 172.21.15.163:0/5238664 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd90c067320 0x7fd90c0697d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd92806d310 tx=0x7fd924016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:56.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.649+0000 7fd91d7fa700 1 -- 172.21.15.163:0/5238664 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd920055910 con 0x7fd92815e010 2023-10-09T06:37:56.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.876+0000 7fd92f787700 1 -- 172.21.15.163:0/5238664 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd92806e900 con 0x7fd92815e010 2023-10-09T06:37:56.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.876+0000 7fd91d7fa700 1 -- 172.21.15.163:0/5238664 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4613 (secure 0 0 0) 0x7fd9200554a0 con 0x7fd92815e010 2023-10-09T06:37:56.878 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:37:56.878 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":93,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1554520486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1554520486},{"type":"v1","addr":"172.21.15.181:6837","nonce":1554520486}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1695339871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1695339871},{"type":"v1","addr":"172.21.15.181:6836","nonce":1695339871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":7515,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3668692283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3668692283},{"type":"v1","addr":"172.21.15.163:6834","nonce":3668692283}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:37:46.529939+0000","modified":"2023-10-09T06:37:50.185660+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7518":{"gid":7518,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/3043729326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3043729326},{"type":"v1","addr":"172.21.15.163:6835","nonce":3043729326}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-09T06:37:56.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.878+0000 7fd92f787700 1 -- 172.21.15.163:0/5238664 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd90c067320 msgr2=0x7fd90c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:56.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.878+0000 7fd92f787700 1 --2- 172.21.15.163:0/5238664 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd90c067320 0x7fd90c0697d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd92806d310 tx=0x7fd924016000 comp rx=0 tx=0).stop 2023-10-09T06:37:56.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.878+0000 7fd92f787700 1 -- 172.21.15.163:0/5238664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd92815e010 msgr2=0x7fd92815c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:56.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.878+0000 7fd92f787700 1 --2- 172.21.15.163:0/5238664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd92815e010 0x7fd92815c380 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fd92000c930 tx=0x7fd9200026e0 comp rx=0 tx=0).stop 2023-10-09T06:37:56.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.879+0000 7fd92f787700 1 -- 172.21.15.163:0/5238664 shutdown_connections 2023-10-09T06:37:56.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.879+0000 7fd92f787700 1 --2- 172.21.15.163:0/5238664 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd90c067320 0x7fd90c0697d0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:56.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.879+0000 7fd92f787700 1 --2- 172.21.15.163:0/5238664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd92815e010 0x7fd92815c380 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:56.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.879+0000 7fd92f787700 1 --2- 172.21.15.163:0/5238664 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd928068550 0x7fd9281470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:56.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.879+0000 7fd92f787700 1 --2- 172.21.15.163:0/5238664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd92806f640 0x7fd92815be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:56.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.879+0000 7fd92f787700 1 -- 172.21.15.163:0/5238664 >> 172.21.15.163:0/5238664 conn(0x7fd9280cd480 msgr2=0x7fd928155c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:56.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.879+0000 7fd92f787700 1 -- 172.21.15.163:0/5238664 shutdown_connections 2023-10-09T06:37:56.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:56.879+0000 7fd92f787700 1 -- 172.21.15.163:0/5238664 wait complete. 2023-10-09T06:37:56.881 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 93 2023-10-09T06:37:57.294 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 93, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-09T06:37:46.529939+0000', 'modified': '2023-10-09T06:37:50.185660+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7518}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7518': {'gid': 7518, 'name': 'c', 'rank': 0, 'incarnation': 89, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.163:6835/3043729326', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6833', 'nonce': 3043729326}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 3043729326}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-09T06:37:57.294 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-09T06:37:57.294 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-09T06:37:57.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.410+0000 7f5c5ed6d700 1 Processor -- start 2023-10-09T06:37:57.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.410+0000 7f5c5ed6d700 1 -- start start 2023-10-09T06:37:57.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.411+0000 7f5c5ed6d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c58068550 0x7f5c58068920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:57.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.411+0000 7f5c5ed6d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5c5806f810 0x7f5c58159510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:57.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.411+0000 7f5c5ed6d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c5806ec70 con 0x7f5c5815e010 2023-10-09T06:37:57.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.411+0000 7f5c5ed6d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c580b3430 con 0x7f5c5806f810 2023-10-09T06:37:57.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.411+0000 7f5c5ed6d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5806fc90 con 0x7f5c58068550 2023-10-09T06:37:57.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.411+0000 7f5c5cb09700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c58068550 0x7f5c58068920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:57.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.411+0000 7f5c5d30a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5c5806f810 0x7f5c58159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:57.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.411+0000 7f5c5cb09700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c58068550 0x7f5c58068920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:41100/0 (socket says 172.21.15.163:41100) 2023-10-09T06:37:57.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.411+0000 7f5c5cb09700 1 -- 172.21.15.163:0/2468397670 learned_addr learned my addr 172.21.15.163:0/2468397670 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:57.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.411+0000 7f5c5d30a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5c5806f810 0x7f5c58159510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:34918/0 (socket says 172.21.15.163:34918) 2023-10-09T06:37:57.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.412+0000 7f5c5d30a700 1 -- 172.21.15.163:0/2468397670 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c58068550 msgr2=0x7f5c58068920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:57.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.412+0000 7f5c5d30a700 1 --2- 172.21.15.163:0/2468397670 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c58068550 0x7f5c58068920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.412+0000 7f5c5d30a700 1 -- 172.21.15.163:0/2468397670 >> v1:172.21.15.181:6789/0 conn(0x7f5c5815e010 legacy=0x7f5c5806f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:57.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.412+0000 7f5c5d30a700 1 -- 172.21.15.163:0/2468397670 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c580b2e70 con 0x7f5c5806f810 2023-10-09T06:37:57.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.412+0000 7f5c5cb09700 1 --2- 172.21.15.163:0/2468397670 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c58068550 0x7f5c58068920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:37:57.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.412+0000 7f5c5d30a700 1 --2- 172.21.15.163:0/2468397670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c5806f810 0x7f5c58159510 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009e60 tx=0x7f5c50009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45e648768ba6e40d server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:57.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.412+0000 7f5c4f7fe700 1 -- 172.21.15.163:0/2468397670 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5000e070 con 0x7f5c5806f810 2023-10-09T06:37:57.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.412+0000 7f5c4f7fe700 1 -- 172.21.15.163:0/2468397670 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c5000bb60 con 0x7f5c5806f810 2023-10-09T06:37:57.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.412+0000 7f5c4f7fe700 1 -- 172.21.15.163:0/2468397670 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c50012430 con 0x7f5c5806f810 2023-10-09T06:37:57.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.413+0000 7f5c5ed6d700 1 -- 172.21.15.163:0/2468397670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c5806f810 msgr2=0x7f5c58159510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:57.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.413+0000 7f5c5ed6d700 1 --2- 172.21.15.163:0/2468397670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c5806f810 0x7f5c58159510 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009e60 tx=0x7f5c50009780 comp rx=0 tx=0).stop 2023-10-09T06:37:57.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.413+0000 7f5c5ed6d700 1 -- 172.21.15.163:0/2468397670 shutdown_connections 2023-10-09T06:37:57.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.413+0000 7f5c5ed6d700 1 --2- 172.21.15.163:0/2468397670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c5806f810 0x7f5c58159510 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.413+0000 7f5c5ed6d700 1 --2- 172.21.15.163:0/2468397670 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c58068550 0x7f5c58068920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.413+0000 7f5c5ed6d700 1 -- 172.21.15.163:0/2468397670 >> 172.21.15.163:0/2468397670 conn(0x7f5c580cd480 msgr2=0x7f5c580cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:57.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.413+0000 7f5c5ed6d700 1 -- 172.21.15.163:0/2468397670 shutdown_connections 2023-10-09T06:37:57.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.413+0000 7f5c5ed6d700 1 -- 172.21.15.163:0/2468397670 wait complete. 2023-10-09T06:37:57.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.414+0000 7f5c5ed6d700 1 Processor -- start 2023-10-09T06:37:57.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.414+0000 7f5c5ed6d700 1 -- start start 2023-10-09T06:37:57.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.415+0000 7f5c5ed6d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c58068550 0x7f5c5813bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:57.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.415+0000 7f5c5ed6d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c5806f810 0x7f5c58152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:57.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.415+0000 7f5c5ed6d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c5815e010 0x7f5c58152ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:57.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.415+0000 7f5c5ed6d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c580ab6d0 con 0x7f5c5806f810 2023-10-09T06:37:57.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.415+0000 7f5c5ed6d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c580abf30 con 0x7f5c58068550 2023-10-09T06:37:57.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.415+0000 7f5c5ed6d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5816e280 con 0x7f5c5815e010 2023-10-09T06:37:57.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.415+0000 7f5c4ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c5806f810 0x7f5c58152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:57.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.415+0000 7f5c4ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c5806f810 0x7f5c58152980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41816/0 (socket says 172.21.15.163:41816) 2023-10-09T06:37:57.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.415+0000 7f5c4ffff700 1 -- 172.21.15.163:0/980502495 learned_addr learned my addr 172.21.15.163:0/980502495 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:57.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.415+0000 7f5c5cb09700 1 --2- 172.21.15.163:0/980502495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c58068550 0x7f5c5813bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:57.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.415+0000 7f5c5d30a700 1 --2- 172.21.15.163:0/980502495 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c5815e010 0x7f5c58152ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:57.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.415+0000 7f5c4ffff700 1 -- 172.21.15.163:0/980502495 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c5815e010 msgr2=0x7f5c58152ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:57.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.415+0000 7f5c4ffff700 1 --2- 172.21.15.163:0/980502495 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c5815e010 0x7f5c58152ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.415+0000 7f5c4ffff700 1 -- 172.21.15.163:0/980502495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c58068550 msgr2=0x7f5c5813bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:57.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.415+0000 7f5c4ffff700 1 --2- 172.21.15.163:0/980502495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c58068550 0x7f5c5813bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.415+0000 7f5c4ffff700 1 -- 172.21.15.163:0/980502495 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c540094f0 con 0x7f5c5806f810 2023-10-09T06:37:57.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.415+0000 7f5c4ffff700 1 --2- 172.21.15.163:0/980502495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c5806f810 0x7f5c58152980 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f5c4800c4e0 tx=0x7f5c4800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:57.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.416+0000 7f5c4cff9700 1 -- 172.21.15.163:0/980502495 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48011590 con 0x7f5c5806f810 2023-10-09T06:37:57.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.416+0000 7f5c4cff9700 1 -- 172.21.15.163:0/980502495 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c48011710 con 0x7f5c5806f810 2023-10-09T06:37:57.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.416+0000 7f5c5ed6d700 1 -- 172.21.15.163:0/980502495 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c50009a20 con 0x7f5c5806f810 2023-10-09T06:37:57.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.416+0000 7f5c4cff9700 1 -- 172.21.15.163:0/980502495 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48011ad0 con 0x7f5c5806f810 2023-10-09T06:37:57.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.416+0000 7f5c5ed6d700 1 -- 172.21.15.163:0/980502495 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c580abbc0 con 0x7f5c5806f810 2023-10-09T06:37:57.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.417+0000 7f5c4cff9700 1 -- 172.21.15.163:0/980502495 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5c4801b3b0 con 0x7f5c5806f810 2023-10-09T06:37:57.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.417+0000 7f5c4cff9700 1 --2- 172.21.15.163:0/980502495 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5c44067350 0x7f5c44069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:57.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.417+0000 7f5c4cff9700 1 -- 172.21.15.163:0/980502495 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5c48084060 con 0x7f5c5806f810 2023-10-09T06:37:57.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.418+0000 7f5c5cb09700 1 --2- 172.21.15.163:0/980502495 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5c44067350 0x7f5c44069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:57.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.418+0000 7f5c5cb09700 1 --2- 172.21.15.163:0/980502495 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5c44067350 0x7f5c44069800 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5c54006d20 tx=0x7f5c54009250 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:57.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.419+0000 7f5c68065b80 1 Processor -- start 2023-10-09T06:37:57.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.419+0000 7f5c68065b80 1 -- start start 2023-10-09T06:37:57.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.420+0000 7f5c68065b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a1c328a50 0x558a1c328e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:57.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.420+0000 7f5c68065b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a1c372d00 0x558a1c351a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:57.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.420+0000 7f5c68065b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1c351f60 0x558a1c324cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:57.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.420+0000 7f5c68065b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1c27fa70 con 0x558a1c328a50 2023-10-09T06:37:57.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.420+0000 7f5c68065b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1c27d270 con 0x558a1c351f60 2023-10-09T06:37:57.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.420+0000 7f5c68065b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x558a1c279060 con 0x558a1c372d00 2023-10-09T06:37:57.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.420+0000 7f5c5cb09700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a1c328a50 0x558a1c328e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:57.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.420+0000 7f5c5cb09700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a1c328a50 0x558a1c328e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41830/0 (socket says 172.21.15.163:41830) 2023-10-09T06:37:57.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.420+0000 7f5c5cb09700 1 -- 172.21.15.163:0/3016936811 learned_addr learned my addr 172.21.15.163:0/3016936811 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:57.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.420+0000 7f5c5d30a700 1 --2- 172.21.15.163:0/3016936811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1c351f60 0x558a1c324cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:57.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.420+0000 7f5c4ffff700 1 --2- 172.21.15.163:0/3016936811 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a1c372d00 0x558a1c351a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:57.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.420+0000 7f5c5cb09700 1 -- 172.21.15.163:0/3016936811 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a1c372d00 msgr2=0x558a1c351a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:57.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.420+0000 7f5c5cb09700 1 --2- 172.21.15.163:0/3016936811 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a1c372d00 0x558a1c351a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.420+0000 7f5c5cb09700 1 -- 172.21.15.163:0/3016936811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1c351f60 msgr2=0x558a1c324cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:57.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.420+0000 7f5c5cb09700 1 --2- 172.21.15.163:0/3016936811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1c351f60 0x558a1c324cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.420+0000 7f5c5cb09700 1 -- 172.21.15.163:0/3016936811 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a1c113270 con 0x558a1c328a50 2023-10-09T06:37:57.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.420+0000 7f5c5cb09700 1 --2- 172.21.15.163:0/3016936811 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a1c328a50 0x558a1c328e20 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f5c540144a0 tx=0x7f5c54019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d87d7a4d49de7175 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:57.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.421+0000 7f5c3dffb700 1 -- 172.21.15.163:0/3016936811 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c54029410 con 0x558a1c328a50 2023-10-09T06:37:57.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.421+0000 7f5c3dffb700 1 -- 172.21.15.163:0/3016936811 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c54029570 con 0x558a1c328a50 2023-10-09T06:37:57.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.421+0000 7f5c3dffb700 1 -- 172.21.15.163:0/3016936811 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c540298f0 con 0x558a1c328a50 2023-10-09T06:37:57.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.421+0000 7f5c68065b80 1 -- 172.21.15.163:0/3016936811 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a1c328a50 msgr2=0x558a1c328e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:57.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.421+0000 7f5c68065b80 1 --2- 172.21.15.163:0/3016936811 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a1c328a50 0x558a1c328e20 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f5c540144a0 tx=0x7f5c54019000 comp rx=0 tx=0).stop 2023-10-09T06:37:57.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.421+0000 7f5c68065b80 1 -- 172.21.15.163:0/3016936811 shutdown_connections 2023-10-09T06:37:57.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.421+0000 7f5c68065b80 1 --2- 172.21.15.163:0/3016936811 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a1c328a50 0x558a1c328e20 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.421+0000 7f5c68065b80 1 --2- 172.21.15.163:0/3016936811 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1c351f60 0x558a1c324cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.421+0000 7f5c68065b80 1 --2- 172.21.15.163:0/3016936811 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a1c372d00 0x558a1c351a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.421+0000 7f5c68065b80 1 -- 172.21.15.163:0/3016936811 >> 172.21.15.163:0/3016936811 conn(0x558a1c32a370 msgr2=0x558a1c32a740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:57.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.421+0000 7f5c68065b80 1 -- 172.21.15.163:0/3016936811 shutdown_connections 2023-10-09T06:37:57.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.421+0000 7f5c68065b80 1 -- 172.21.15.163:0/3016936811 wait complete. 2023-10-09T06:37:57.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.422+0000 7f5c68065b80 1 -- start start 2023-10-09T06:37:57.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.422+0000 7f5c68065b80 1 Processor -- start 2023-10-09T06:37:57.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.423+0000 7f5c68065b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a1c328a50 0x558a1c3533d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:57.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.423+0000 7f5c68065b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a1c351f60 0x558a1c37efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:57.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.423+0000 7f5c68065b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1c372d00 0x558a1c354ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:57.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.423+0000 7f5c68065b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1c224890 con 0x558a1c328a50 2023-10-09T06:37:57.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.423+0000 7f5c68065b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1c23fea0 con 0x558a1c372d00 2023-10-09T06:37:57.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.423+0000 7f5c68065b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x558a1c0fbe30 con 0x558a1c351f60 2023-10-09T06:37:57.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.423+0000 7f5c68065b80 10 client.0 _ll_register_callbacks cb 0x558a1c2ec110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-09T06:37:57.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.423+0000 7f5c5cb09700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a1c328a50 0x558a1c3533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:57.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.423+0000 7f5c5cb09700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a1c328a50 0x558a1c3533d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41846/0 (socket says 172.21.15.163:41846) 2023-10-09T06:37:57.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.423+0000 7f5c5cb09700 1 -- 172.21.15.163:0/108334535 learned_addr learned my addr 172.21.15.163:0/108334535 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:57.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.423+0000 7f5c4ffff700 1 --2- 172.21.15.163:0/108334535 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a1c351f60 0x558a1c37efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:57.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.423+0000 7f5c5d30a700 1 --2- 172.21.15.163:0/108334535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1c372d00 0x558a1c354ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:57.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.423+0000 7f5c5cb09700 1 -- 172.21.15.163:0/108334535 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a1c351f60 msgr2=0x558a1c37efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:57.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.423+0000 7f5c5cb09700 1 --2- 172.21.15.163:0/108334535 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a1c351f60 0x558a1c37efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.423+0000 7f5c5cb09700 1 -- 172.21.15.163:0/108334535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1c372d00 msgr2=0x558a1c354ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:57.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.423+0000 7f5c5cb09700 1 --2- 172.21.15.163:0/108334535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1c372d00 0x558a1c354ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.423+0000 7f5c5cb09700 1 -- 172.21.15.163:0/108334535 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c54011c60 con 0x558a1c328a50 2023-10-09T06:37:57.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.423+0000 7f5c5cb09700 1 --2- 172.21.15.163:0/108334535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a1c328a50 0x558a1c3533d0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f5c54009890 tx=0x7f5c54023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf87dc1982ea113c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:57.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.423+0000 7f5c36ffd700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-10-09T06:37:57.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.424+0000 7f5c36ffd700 1 -- 172.21.15.163:0/108334535 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5402e070 con 0x558a1c328a50 2023-10-09T06:37:57.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.424+0000 7f5c36ffd700 1 -- 172.21.15.163:0/108334535 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5c5401a040 con 0x558a1c328a50 2023-10-09T06:37:57.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.424+0000 7f5c68065b80 1 -- 172.21.15.163:0/108334535 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a1c31db70 con 0x558a1c328a50 2023-10-09T06:37:57.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.424+0000 7f5c36ffd700 1 -- 172.21.15.163:0/108334535 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c54029410 con 0x558a1c328a50 2023-10-09T06:37:57.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.424+0000 7f5c68065b80 10 client.7683 fetch_fsmap learned FSMap version 93 2023-10-09T06:37:57.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.424+0000 7f5c68065b80 1 -- 172.21.15.163:0/108334535 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x558a1c0b5fe0 con 0x558a1c328a50 2023-10-09T06:37:57.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.424+0000 7f5c36ffd700 1 -- 172.21.15.163:0/108334535 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558a1c31db70 con 0x558a1c328a50 2023-10-09T06:37:57.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.424+0000 7f5c36ffd700 1 -- 172.21.15.163:0/108334535 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 93) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5c540144d0 con 0x558a1c328a50 2023-10-09T06:37:57.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.424+0000 7f5c68065b80 10 client.7683 fetch_fsmap finished waiting for FSMap version 93 2023-10-09T06:37:57.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.424+0000 7f5c68065b80 10 client.7683 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-09T06:37:57.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.424+0000 7f5c68065b80 20 client.7683 populate_metadata read hostname 'smithi163' 2023-10-09T06:37:57.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.424+0000 7f5c68065b80 1 --2- 172.21.15.163:0/108334535 >> [v2:172.21.15.163:6833/3043729326,v1:172.21.15.163:6835/3043729326] conn(0x558a1c36d210 0x558a1c325c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:57.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.424+0000 7f5c68065b80 4 client.7683 mds_command: new command op to 7518 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-09T06:37:57.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.424+0000 7f5c68065b80 1 -- 172.21.15.163:0/108334535 --> [v2:172.21.15.163:6833/3043729326,v1:172.21.15.163:6835/3043729326] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a1c2faaa0 con 0x558a1c36d210 2023-10-09T06:37:57.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.424+0000 7f5c4ffff700 1 --2- 172.21.15.163:0/108334535 >> [v2:172.21.15.163:6833/3043729326,v1:172.21.15.163:6835/3043729326] conn(0x558a1c36d210 0x558a1c325c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:57.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.424+0000 7f5c4ffff700 1 --2- 172.21.15.163:0/108334535 >> [v2:172.21.15.163:6833/3043729326,v1:172.21.15.163:6835/3043729326] conn(0x558a1c36d210 0x558a1c325c10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bee96982a07959e3 server_cookie=7acacaa1c4513f07 in_seq=0 out_seq=0 2023-10-09T06:37:57.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.424+0000 7f5c36ffd700 10 client.7683 ms_handle_connect on v2:172.21.15.163:6833/3043729326 2023-10-09T06:37:57.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.425+0000 7f5c36ffd700 1 -- 172.21.15.163:0/108334535 <== mds.0 v2:172.21.15.163:6833/3043729326 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x558a1c2faaa0 con 0x558a1c36d210 2023-10-09T06:37:57.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.425+0000 7f5c36ffd700 10 client.7683 handle_command_reply: tid=0 2023-10-09T06:37:57.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.425+0000 7f5c68065b80 1 client.7683 shutdown 2023-10-09T06:37:57.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.425+0000 7f5c68065b80 1 -- 172.21.15.163:0/108334535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a1c328a50 msgr2=0x558a1c3533d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:57.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.425+0000 7f5c68065b80 1 --2- 172.21.15.163:0/108334535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a1c328a50 0x558a1c3533d0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f5c54009890 tx=0x7f5c54023000 comp rx=0 tx=0).stop 2023-10-09T06:37:57.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.425+0000 7f5c68065b80 1 -- 172.21.15.163:0/108334535 shutdown_connections 2023-10-09T06:37:57.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.425+0000 7f5c68065b80 1 --2- 172.21.15.163:0/108334535 >> [v2:172.21.15.163:6833/3043729326,v1:172.21.15.163:6835/3043729326] conn(0x558a1c36d210 0x558a1c325c10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.425+0000 7f5c68065b80 1 --2- 172.21.15.163:0/108334535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a1c328a50 0x558a1c3533d0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.425+0000 7f5c36ffd700 0 client.7683 ms_handle_reset on v2:172.21.15.163:6833/3043729326 2023-10-09T06:37:57.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.425+0000 7f5c68065b80 1 --2- 172.21.15.163:0/108334535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1c372d00 0x558a1c354ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.425+0000 7f5c68065b80 1 --2- 172.21.15.163:0/108334535 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a1c351f60 0x558a1c37efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.425+0000 7f5c68065b80 1 -- 172.21.15.163:0/108334535 >> 172.21.15.163:0/108334535 conn(0x558a1c32a370 msgr2=0x558a1c323c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:57.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.425+0000 7f5c68065b80 1 -- 172.21.15.163:0/108334535 shutdown_connections 2023-10-09T06:37:57.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.425+0000 7f5c68065b80 1 -- 172.21.15.163:0/108334535 wait complete. 2023-10-09T06:37:57.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.425+0000 7f5c68065b80 20 client.7683 trim_cache size 0 max 16384 2023-10-09T06:37:57.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.442+0000 7f5c68065b80 1 Processor -- start 2023-10-09T06:37:57.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.442+0000 7f5c68065b80 1 -- start start 2023-10-09T06:37:57.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.443+0000 7f5c68065b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a1c328a50 0x558a1c323c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:57.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.443+0000 7f5c68065b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1c351f60 0x558a1c324160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:57.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.443+0000 7f5c68065b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a1c372d00 0x558a1c365560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:57.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.443+0000 7f5c68065b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1c36ee10 con 0x558a1c372d00 2023-10-09T06:37:57.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.443+0000 7f5c68065b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1c32b2f0 con 0x558a1c351f60 2023-10-09T06:37:57.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.443+0000 7f5c68065b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x558a1c325cb0 con 0x558a1c328a50 2023-10-09T06:37:57.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.443+0000 7f5c5d30a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a1c372d00 0x558a1c365560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:57.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.443+0000 7f5c5d30a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a1c372d00 0x558a1c365560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41858/0 (socket says 172.21.15.163:41858) 2023-10-09T06:37:57.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.443+0000 7f5c5d30a700 1 -- 172.21.15.163:0/1529681512 learned_addr learned my addr 172.21.15.163:0/1529681512 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:57.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.443+0000 7f5c5cb09700 1 --2- 172.21.15.163:0/1529681512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a1c328a50 0x558a1c323c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:57.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.443+0000 7f5c4ffff700 1 --2- 172.21.15.163:0/1529681512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1c351f60 0x558a1c324160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:57.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.443+0000 7f5c5d30a700 1 -- 172.21.15.163:0/1529681512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a1c328a50 msgr2=0x558a1c323c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:57.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.443+0000 7f5c5d30a700 1 --2- 172.21.15.163:0/1529681512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a1c328a50 0x558a1c323c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.443+0000 7f5c5d30a700 1 -- 172.21.15.163:0/1529681512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1c351f60 msgr2=0x558a1c324160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:57.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.443+0000 7f5c5d30a700 1 --2- 172.21.15.163:0/1529681512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1c351f60 0x558a1c324160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.443+0000 7f5c5d30a700 1 -- 172.21.15.163:0/1529681512 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a1c247780 con 0x558a1c372d00 2023-10-09T06:37:57.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.443+0000 7f5c4ffff700 1 --2- 172.21.15.163:0/1529681512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1c351f60 0x558a1c324160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:37:57.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.443+0000 7f5c5d30a700 1 --2- 172.21.15.163:0/1529681512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a1c372d00 0x558a1c365560 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f5c580ce220 tx=0x7f5c50019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be1ee363c34cb71d server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:57.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.444+0000 7f5c3e7fc700 1 -- 172.21.15.163:0/1529681512 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c500127a0 con 0x558a1c372d00 2023-10-09T06:37:57.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.444+0000 7f5c3e7fc700 1 -- 172.21.15.163:0/1529681512 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c50024040 con 0x558a1c372d00 2023-10-09T06:37:57.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.444+0000 7f5c3e7fc700 1 -- 172.21.15.163:0/1529681512 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c50012b50 con 0x558a1c372d00 2023-10-09T06:37:57.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.444+0000 7f5c68065b80 1 -- 172.21.15.163:0/1529681512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a1c372d00 msgr2=0x558a1c365560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:57.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.444+0000 7f5c68065b80 1 --2- 172.21.15.163:0/1529681512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a1c372d00 0x558a1c365560 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f5c580ce220 tx=0x7f5c50019000 comp rx=0 tx=0).stop 2023-10-09T06:37:57.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.444+0000 7f5c68065b80 1 -- 172.21.15.163:0/1529681512 shutdown_connections 2023-10-09T06:37:57.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.444+0000 7f5c68065b80 1 --2- 172.21.15.163:0/1529681512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a1c372d00 0x558a1c365560 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.444+0000 7f5c68065b80 1 --2- 172.21.15.163:0/1529681512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a1c328a50 0x558a1c323c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.444+0000 7f5c68065b80 1 --2- 172.21.15.163:0/1529681512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1c351f60 0x558a1c324160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.444+0000 7f5c68065b80 1 -- 172.21.15.163:0/1529681512 >> 172.21.15.163:0/1529681512 conn(0x558a1c32a370 msgr2=0x558a1c36e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:57.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.444+0000 7f5c68065b80 1 -- 172.21.15.163:0/1529681512 shutdown_connections 2023-10-09T06:37:57.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.444+0000 7f5c68065b80 1 -- 172.21.15.163:0/1529681512 wait complete. 2023-10-09T06:37:57.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.445+0000 7f5c68065b80 1 -- start start 2023-10-09T06:37:57.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.445+0000 7f5c68065b80 1 Processor -- start 2023-10-09T06:37:57.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.446+0000 7f5c68065b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1c328a50 0x558a1c3d49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:57.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.446+0000 7f5c68065b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a1c351f60 0x558a1c357850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:57.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.446+0000 7f5c68065b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a1c372d00 0x558a1c357d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:57.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.446+0000 7f5c68065b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1c3d4ee0 con 0x558a1c372d00 2023-10-09T06:37:57.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.446+0000 7f5c68065b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a1c36aa70 con 0x558a1c328a50 2023-10-09T06:37:57.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.446+0000 7f5c68065b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x558a1c388fa0 con 0x558a1c351f60 2023-10-09T06:37:57.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.446+0000 7f5c68065b80 10 client.0 _ll_register_callbacks cb 0x558a1c2ec110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-09T06:37:57.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.446+0000 7f5c5d30a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a1c372d00 0x558a1c357d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:57.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.446+0000 7f5c5d30a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a1c372d00 0x558a1c357d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41870/0 (socket says 172.21.15.163:41870) 2023-10-09T06:37:57.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.446+0000 7f5c5d30a700 1 -- 172.21.15.163:0/422868855 learned_addr learned my addr 172.21.15.163:0/422868855 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:57.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.446+0000 7f5c4ffff700 1 --2- 172.21.15.163:0/422868855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a1c351f60 0x558a1c357850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:57.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.446+0000 7f5c5cb09700 1 --2- 172.21.15.163:0/422868855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1c328a50 0x558a1c3d49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:57.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.446+0000 7f5c5d30a700 1 -- 172.21.15.163:0/422868855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a1c351f60 msgr2=0x558a1c357850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:57.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.446+0000 7f5c5d30a700 1 --2- 172.21.15.163:0/422868855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a1c351f60 0x558a1c357850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.446+0000 7f5c5d30a700 1 -- 172.21.15.163:0/422868855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1c328a50 msgr2=0x558a1c3d49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:57.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.446+0000 7f5c5d30a700 1 --2- 172.21.15.163:0/422868855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1c328a50 0x558a1c3d49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.446+0000 7f5c5d30a700 1 -- 172.21.15.163:0/422868855 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c580b2e70 con 0x558a1c372d00 2023-10-09T06:37:57.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.446+0000 7f5c5d30a700 1 --2- 172.21.15.163:0/422868855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a1c372d00 0x558a1c357d90 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x558a1c3249b0 tx=0x7f5c50020070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d929601cf5e203 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:57.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.446+0000 7f5c36ffd700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-10-09T06:37:57.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.446+0000 7f5c36ffd700 1 -- 172.21.15.163:0/422868855 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5001e630 con 0x558a1c372d00 2023-10-09T06:37:57.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.446+0000 7f5c36ffd700 1 -- 172.21.15.163:0/422868855 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5c5001e790 con 0x558a1c372d00 2023-10-09T06:37:57.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.446+0000 7f5c68065b80 1 -- 172.21.15.163:0/422868855 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c54029570 con 0x558a1c372d00 2023-10-09T06:37:57.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.447+0000 7f5c36ffd700 1 -- 172.21.15.163:0/422868855 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c5001eb60 con 0x558a1c372d00 2023-10-09T06:37:57.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.447+0000 7f5c68065b80 10 client.7689 fetch_fsmap learned FSMap version 93 2023-10-09T06:37:57.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.447+0000 7f5c68065b80 1 -- 172.21.15.163:0/422868855 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=2+}) v3 -- 0x558a1c22e470 con 0x558a1c372d00 2023-10-09T06:37:57.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.447+0000 7f5c36ffd700 1 -- 172.21.15.163:0/422868855 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c54029570 con 0x558a1c372d00 2023-10-09T06:37:57.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.447+0000 7f5c36ffd700 1 -- 172.21.15.163:0/422868855 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 93) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5c50027070 con 0x558a1c372d00 2023-10-09T06:37:57.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.447+0000 7f5c68065b80 10 client.7689 fetch_fsmap finished waiting for FSMap version 93 2023-10-09T06:37:57.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.447+0000 7f5c68065b80 10 client.7689 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-09T06:37:57.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.447+0000 7f5c68065b80 20 client.7689 populate_metadata read hostname 'smithi163' 2023-10-09T06:37:57.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.447+0000 7f5c68065b80 1 --2- 172.21.15.163:0/422868855 >> [v2:172.21.15.163:6833/3043729326,v1:172.21.15.163:6835/3043729326] conn(0x558a1c3d5650 0x558a1c3d5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:57.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.447+0000 7f5c68065b80 4 client.7689 mds_command: new command op to 7518 tid=0[{"prefix": "status"}] 2023-10-09T06:37:57.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.447+0000 7f5c68065b80 1 -- 172.21.15.163:0/422868855 --> [v2:172.21.15.163:6833/3043729326,v1:172.21.15.163:6835/3043729326] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558a1c21be00 con 0x558a1c3d5650 2023-10-09T06:37:57.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.447+0000 7f5c5cb09700 1 --2- 172.21.15.163:0/422868855 >> [v2:172.21.15.163:6833/3043729326,v1:172.21.15.163:6835/3043729326] conn(0x558a1c3d5650 0x558a1c3d5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:57.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.447+0000 7f5c36ffd700 10 client.7689 ms_handle_connect on v2:172.21.15.163:6833/3043729326 2023-10-09T06:37:57.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.447+0000 7f5c5cb09700 1 --2- 172.21.15.163:0/422868855 >> [v2:172.21.15.163:6833/3043729326,v1:172.21.15.163:6835/3043729326] conn(0x558a1c3d5650 0x558a1c3d5a20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffbdc6090935f3b6 server_cookie=3e6c17295b19b1d3 in_seq=0 out_seq=0 2023-10-09T06:37:57.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.447+0000 7f5c36ffd700 1 -- 172.21.15.163:0/422868855 <== mds.0 v2:172.21.15.163:6833/3043729326 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x558a1c21be00 con 0x558a1c3d5650 2023-10-09T06:37:57.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.447+0000 7f5c36ffd700 10 client.7689 handle_command_reply: tid=0 2023-10-09T06:37:57.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.447+0000 7f5c68065b80 1 client.7689 shutdown 2023-10-09T06:37:57.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.448+0000 7f5c68065b80 1 -- 172.21.15.163:0/422868855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a1c372d00 msgr2=0x558a1c357d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:57.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.448+0000 7f5c68065b80 1 --2- 172.21.15.163:0/422868855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a1c372d00 0x558a1c357d90 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x558a1c3249b0 tx=0x7f5c50020070 comp rx=0 tx=0).stop 2023-10-09T06:37:57.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.448+0000 7f5c68065b80 1 -- 172.21.15.163:0/422868855 shutdown_connections 2023-10-09T06:37:57.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.448+0000 7f5c68065b80 1 --2- 172.21.15.163:0/422868855 >> [v2:172.21.15.163:6833/3043729326,v1:172.21.15.163:6835/3043729326] conn(0x558a1c3d5650 0x558a1c3d5a20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.448+0000 7f5c68065b80 1 --2- 172.21.15.163:0/422868855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558a1c372d00 0x558a1c357d90 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.448+0000 7f5c68065b80 1 --2- 172.21.15.163:0/422868855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x558a1c328a50 0x558a1c3d49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.448+0000 7f5c36ffd700 0 client.7689 ms_handle_reset on v2:172.21.15.163:6833/3043729326 2023-10-09T06:37:57.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.448+0000 7f5c68065b80 1 --2- 172.21.15.163:0/422868855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x558a1c351f60 0x558a1c357850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.448+0000 7f5c68065b80 1 -- 172.21.15.163:0/422868855 >> 172.21.15.163:0/422868855 conn(0x558a1c32a370 msgr2=0x558a1c32b950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:57.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.448+0000 7f5c68065b80 1 -- 172.21.15.163:0/422868855 shutdown_connections 2023-10-09T06:37:57.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.448+0000 7f5c68065b80 1 -- 172.21.15.163:0/422868855 wait complete. 2023-10-09T06:37:57.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.448+0000 7f5c68065b80 20 client.7689 trim_cache size 0 max 16384 2023-10-09T06:37:57.457 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:37:57.457 INFO:teuthology.orchestra.run.smithi163.stdout: "cluster_fsid": "2520b8f7-539f-4bae-9f17-1fce66f66ddd", 2023-10-09T06:37:57.457 INFO:teuthology.orchestra.run.smithi163.stdout: "whoami": 0, 2023-10-09T06:37:57.457 INFO:teuthology.orchestra.run.smithi163.stdout: "id": 7518, 2023-10-09T06:37:57.457 INFO:teuthology.orchestra.run.smithi163.stdout: "want_state": "up:active", 2023-10-09T06:37:57.457 INFO:teuthology.orchestra.run.smithi163.stdout: "state": "up:active", 2023-10-09T06:37:57.457 INFO:teuthology.orchestra.run.smithi163.stdout: "fs_name": "cephfs", 2023-10-09T06:37:57.457 INFO:teuthology.orchestra.run.smithi163.stdout: "rank_uptime": 10.282135206, 2023-10-09T06:37:57.458 INFO:teuthology.orchestra.run.smithi163.stdout: "mdsmap_epoch": 93, 2023-10-09T06:37:57.458 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch": 137, 2023-10-09T06:37:57.458 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch_barrier": 137, 2023-10-09T06:37:57.458 INFO:teuthology.orchestra.run.smithi163.stdout: "uptime": 15.421830342 2023-10-09T06:37:57.458 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:37:57.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.448+0000 7f5c5ed6d700 1 -- 172.21.15.163:0/980502495 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5c44067350 msgr2=0x7f5c44069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:57.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.448+0000 7f5c5ed6d700 1 --2- 172.21.15.163:0/980502495 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5c44067350 0x7f5c44069800 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5c54006d20 tx=0x7f5c54009250 comp rx=0 tx=0).stop 2023-10-09T06:37:57.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.448+0000 7f5c5ed6d700 1 -- 172.21.15.163:0/980502495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c5806f810 msgr2=0x7f5c58152980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:57.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.448+0000 7f5c5ed6d700 1 --2- 172.21.15.163:0/980502495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c5806f810 0x7f5c58152980 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f5c4800c4e0 tx=0x7f5c4800c7f0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.448+0000 7f5c5ed6d700 1 -- 172.21.15.163:0/980502495 shutdown_connections 2023-10-09T06:37:57.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.448+0000 7f5c5ed6d700 1 --2- 172.21.15.163:0/980502495 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5c44067350 0x7f5c44069800 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.448+0000 7f5c5ed6d700 1 --2- 172.21.15.163:0/980502495 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c5815e010 0x7f5c58152ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.448+0000 7f5c5ed6d700 1 --2- 172.21.15.163:0/980502495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c58068550 0x7f5c5813bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.448+0000 7f5c5ed6d700 1 --2- 172.21.15.163:0/980502495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c5806f810 0x7f5c58152980 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.448+0000 7f5c5ed6d700 1 -- 172.21.15.163:0/980502495 >> 172.21.15.163:0/980502495 conn(0x7f5c580cd480 msgr2=0x7f5c58155fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:57.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.448+0000 7f5c5ed6d700 1 -- 172.21.15.163:0/980502495 shutdown_connections 2023-10-09T06:37:57.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.448+0000 7f5c5ed6d700 1 -- 172.21.15.163:0/980502495 wait complete. 2023-10-09T06:37:57.462 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-09T06:37:57.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.578+0000 7f8b315e8700 1 Processor -- start 2023-10-09T06:37:57.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.578+0000 7f8b315e8700 1 -- start start 2023-10-09T06:37:57.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.579+0000 7f8b315e8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8b2c155740 0x7f8b2c155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:57.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.579+0000 7f8b315e8700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8b2c0b1f40 0x7f8b2c0ae520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:57.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.579+0000 7f8b315e8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b2c06b980 0x7f8b2c0afd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:57.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.579+0000 7f8b315e8700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b2c0b23b0 con 0x7f8b2c0b1f40 2023-10-09T06:37:57.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.579+0000 7f8b315e8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b2c0aeec0 con 0x7f8b2c155740 2023-10-09T06:37:57.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.579+0000 7f8b315e8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c1567e0 con 0x7f8b2c06b980 2023-10-09T06:37:57.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.579+0000 7f8b227fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8b2c0b1f40 0x7f8b2c0ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:57.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.579+0000 7f8b227fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8b2c0b1f40 0x7f8b2c0ae520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41878/0 (socket says 172.21.15.163:41878) 2023-10-09T06:37:57.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.579+0000 7f8b227fc700 1 -- 172.21.15.163:0/8160617 learned_addr learned my addr 172.21.15.163:0/8160617 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:57.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.580+0000 7f8b2b7fe700 1 --2- 172.21.15.163:0/8160617 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b2c06b980 0x7f8b2c0afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:57.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.580+0000 7f8b2affd700 1 --2- 172.21.15.163:0/8160617 >> v2:172.21.15.181:3300/0 conn(0x7f8b2c155740 0x7f8b2c155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:57.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.580+0000 7f8b227fc700 1 -- 172.21.15.163:0/8160617 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b2c06b980 msgr2=0x7f8b2c0afd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:57.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.580+0000 7f8b227fc700 1 --2- 172.21.15.163:0/8160617 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b2c06b980 0x7f8b2c0afd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.580+0000 7f8b227fc700 1 -- 172.21.15.163:0/8160617 >> v2:172.21.15.181:3300/0 conn(0x7f8b2c155740 msgr2=0x7f8b2c155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:57.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.580+0000 7f8b227fc700 1 --2- 172.21.15.163:0/8160617 >> v2:172.21.15.181:3300/0 conn(0x7f8b2c155740 0x7f8b2c155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.580+0000 7f8b227fc700 1 -- 172.21.15.163:0/8160617 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b2c0cc2a0 con 0x7f8b2c0b1f40 2023-10-09T06:37:57.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.580+0000 7f8b227fc700 1 --2- 172.21.15.163:0/8160617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b2c0b1f40 0x7f8b2c0ae520 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0641e0 tx=0x7f8b1400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b960f59e33b9e6a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:57.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.580+0000 7f8b2a7fc700 1 -- 172.21.15.163:0/8160617 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1400dc50 con 0x7f8b2c0b1f40 2023-10-09T06:37:57.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.580+0000 7f8b2a7fc700 1 -- 172.21.15.163:0/8160617 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1400ddb0 con 0x7f8b2c0b1f40 2023-10-09T06:37:57.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.580+0000 7f8b2a7fc700 1 -- 172.21.15.163:0/8160617 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b140196d0 con 0x7f8b2c0b1f40 2023-10-09T06:37:57.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.580+0000 7f8b315e8700 1 -- 172.21.15.163:0/8160617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b2c0b1f40 msgr2=0x7f8b2c0ae520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:57.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.580+0000 7f8b315e8700 1 --2- 172.21.15.163:0/8160617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b2c0b1f40 0x7f8b2c0ae520 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0641e0 tx=0x7f8b1400b4f0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.588 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.581+0000 7f8b315e8700 1 -- 172.21.15.163:0/8160617 shutdown_connections 2023-10-09T06:37:57.588 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.581+0000 7f8b315e8700 1 --2- 172.21.15.163:0/8160617 >> v2:172.21.15.181:3300/0 conn(0x7f8b2c155740 0x7f8b2c155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.588 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.581+0000 7f8b315e8700 1 --2- 172.21.15.163:0/8160617 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b2c06b980 0x7f8b2c0afd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.588 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.581+0000 7f8b315e8700 1 --2- 172.21.15.163:0/8160617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b2c0b1f40 0x7f8b2c0ae520 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.588 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.581+0000 7f8b315e8700 1 -- 172.21.15.163:0/8160617 >> 172.21.15.163:0/8160617 conn(0x7f8b2c06a320 msgr2=0x7f8b2c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:57.588 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.581+0000 7f8b315e8700 1 -- 172.21.15.163:0/8160617 shutdown_connections 2023-10-09T06:37:57.588 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.581+0000 7f8b315e8700 1 -- 172.21.15.163:0/8160617 wait complete. 2023-10-09T06:37:57.588 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.582+0000 7f8b315e8700 1 Processor -- start 2023-10-09T06:37:57.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.582+0000 7f8b315e8700 1 -- start start 2023-10-09T06:37:57.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.582+0000 7f8b315e8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b2c06b980 0x7f8b2c14fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:57.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.582+0000 7f8b315e8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b2c0b1f40 0x7f8b2c16dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:57.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.583+0000 7f8b315e8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b2c155740 0x7f8b2c15aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:57.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.583+0000 7f8b315e8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c0ac080 con 0x7f8b2c06b980 2023-10-09T06:37:57.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.583+0000 7f8b315e8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c150170 con 0x7f8b2c155740 2023-10-09T06:37:57.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.583+0000 7f8b315e8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c16e4e0 con 0x7f8b2c0b1f40 2023-10-09T06:37:57.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.583+0000 7f8b2affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b2c06b980 0x7f8b2c14fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:57.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.583+0000 7f8b2affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b2c06b980 0x7f8b2c14fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41884/0 (socket says 172.21.15.163:41884) 2023-10-09T06:37:57.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.583+0000 7f8b2affd700 1 -- 172.21.15.163:0/1749610930 learned_addr learned my addr 172.21.15.163:0/1749610930 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:57.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.583+0000 7f8b2b7fe700 1 --2- 172.21.15.163:0/1749610930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b2c155740 0x7f8b2c15aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:57.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.583+0000 7f8b227fc700 1 --2- 172.21.15.163:0/1749610930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b2c0b1f40 0x7f8b2c16dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:57.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.583+0000 7f8b2affd700 1 -- 172.21.15.163:0/1749610930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b2c0b1f40 msgr2=0x7f8b2c16dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:57.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.583+0000 7f8b2affd700 1 --2- 172.21.15.163:0/1749610930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b2c0b1f40 0x7f8b2c16dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.583+0000 7f8b2affd700 1 -- 172.21.15.163:0/1749610930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b2c155740 msgr2=0x7f8b2c15aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:57.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.583+0000 7f8b2affd700 1 --2- 172.21.15.163:0/1749610930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b2c155740 0x7f8b2c15aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.583+0000 7f8b2affd700 1 -- 172.21.15.163:0/1749610930 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b1400b790 con 0x7f8b2c06b980 2023-10-09T06:37:57.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.583+0000 7f8b2affd700 1 --2- 172.21.15.163:0/1749610930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b2c06b980 0x7f8b2c14fc30 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00d860 tx=0x7f8b1c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:57.591 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.583+0000 7f8b237fe700 1 -- 172.21.15.163:0/1749610930 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c0125f0 con 0x7f8b2c06b980 2023-10-09T06:37:57.591 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.583+0000 7f8b237fe700 1 -- 172.21.15.163:0/1749610930 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1c012750 con 0x7f8b2c06b980 2023-10-09T06:37:57.591 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.583+0000 7f8b237fe700 1 -- 172.21.15.163:0/1749610930 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c012b10 con 0x7f8b2c06b980 2023-10-09T06:37:57.591 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.583+0000 7f8b315e8700 1 -- 172.21.15.163:0/1749610930 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b2c15a610 con 0x7f8b2c06b980 2023-10-09T06:37:57.591 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.583+0000 7f8b315e8700 1 -- 172.21.15.163:0/1749610930 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b2c159ce0 con 0x7f8b2c06b980 2023-10-09T06:37:57.591 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.584+0000 7f8b315e8700 1 -- 172.21.15.163:0/1749610930 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b2c04c3c0 con 0x7f8b2c06b980 2023-10-09T06:37:57.592 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.585+0000 7f8b237fe700 1 -- 172.21.15.163:0/1749610930 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8b1c013480 con 0x7f8b2c06b980 2023-10-09T06:37:57.592 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.585+0000 7f8b237fe700 1 --2- 172.21.15.163:0/1749610930 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8b0c067320 0x7f8b0c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:57.592 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.585+0000 7f8b237fe700 1 -- 172.21.15.163:0/1749610930 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8b1c084ef0 con 0x7f8b2c06b980 2023-10-09T06:37:57.592 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.585+0000 7f8b227fc700 1 --2- 172.21.15.163:0/1749610930 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8b0c067320 0x7f8b0c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:57.593 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.586+0000 7f8b227fc700 1 --2- 172.21.15.163:0/1749610930 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8b0c067320 0x7f8b0c0697d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0641e0 tx=0x7f8b1401b000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:57.593 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.589+0000 7f8b237fe700 1 -- 172.21.15.163:0/1749610930 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b1c055610 con 0x7f8b2c06b980 2023-10-09T06:37:57.816 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.815+0000 7f8b315e8700 1 -- 172.21.15.163:0/1749610930 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8b2c0ad920 con 0x7f8b2c06b980 2023-10-09T06:37:57.816 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.815+0000 7f8b237fe700 1 -- 172.21.15.163:0/1749610930 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v93) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8b1c0551a0 con 0x7f8b2c06b980 2023-10-09T06:37:57.817 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-09T06:37:57.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.817+0000 7f8b315e8700 1 -- 172.21.15.163:0/1749610930 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8b0c067320 msgr2=0x7f8b0c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:57.819 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.817+0000 7f8b315e8700 1 --2- 172.21.15.163:0/1749610930 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8b0c067320 0x7f8b0c0697d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0641e0 tx=0x7f8b1401b000 comp rx=0 tx=0).stop 2023-10-09T06:37:57.819 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.817+0000 7f8b315e8700 1 -- 172.21.15.163:0/1749610930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b2c06b980 msgr2=0x7f8b2c14fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:57.819 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.817+0000 7f8b315e8700 1 --2- 172.21.15.163:0/1749610930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b2c06b980 0x7f8b2c14fc30 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00d860 tx=0x7f8b1c008040 comp rx=0 tx=0).stop 2023-10-09T06:37:57.819 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.817+0000 7f8b315e8700 1 -- 172.21.15.163:0/1749610930 shutdown_connections 2023-10-09T06:37:57.819 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.817+0000 7f8b315e8700 1 --2- 172.21.15.163:0/1749610930 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8b0c067320 0x7f8b0c0697d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.819 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.817+0000 7f8b315e8700 1 --2- 172.21.15.163:0/1749610930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b2c06b980 0x7f8b2c14fc30 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.817+0000 7f8b315e8700 1 --2- 172.21.15.163:0/1749610930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8b2c155740 0x7f8b2c15aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.817+0000 7f8b315e8700 1 --2- 172.21.15.163:0/1749610930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8b2c0b1f40 0x7f8b2c16dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:57.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.817+0000 7f8b315e8700 1 -- 172.21.15.163:0/1749610930 >> 172.21.15.163:0/1749610930 conn(0x7f8b2c06a320 msgr2=0x7f8b2c06d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:57.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.817+0000 7f8b315e8700 1 -- 172.21.15.163:0/1749610930 shutdown_connections 2023-10-09T06:37:57.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:57.817+0000 7f8b315e8700 1 -- 172.21.15.163:0/1749610930 wait complete. 2023-10-09T06:37:57.830 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-09T06:37:57.830 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-09T06:37:57.831 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi163.front.sepia.ceph.com 2023-10-09T06:37:57.831 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-09T06:37:57.831 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-09T06:37:57.831 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-09T06:37:57.831 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-09T06:37:57.831 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-09T06:37:57.832 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:37:57.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:57.832 DEBUG:teuthology.orchestra.run.smithi163:> ip addr 2023-10-09T06:37:57.848 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:37:57.848 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:37:57.848 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:37:57.848 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:37:57.848 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2023-10-09T06:37:57.849 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:37:57.849 INFO:teuthology.orchestra.run.smithi163.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:37:57.849 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:06 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:37:57.849 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f0 2023-10-09T06:37:57.849 INFO:teuthology.orchestra.run.smithi163.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:37:57.849 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:07 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:37:57.850 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f1 2023-10-09T06:37:57.850 INFO:teuthology.orchestra.run.smithi163.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:37:57.850 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5a brd ff:ff:ff:ff:ff:ff 2023-10-09T06:37:57.850 INFO:teuthology.orchestra.run.smithi163.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-09T06:37:57.850 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5b brd ff:ff:ff:ff:ff:ff 2023-10-09T06:37:57.850 INFO:teuthology.orchestra.run.smithi163.stdout: inet 172.21.15.163/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-09T06:37:57.851 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft 42045sec preferred_lft 42045sec 2023-10-09T06:37:57.851 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::ec4:7aff:fe88:7d5b/64 scope link 2023-10-09T06:37:57.851 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:37:57.851 INFO:teuthology.orchestra.run.smithi163.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-09T06:37:57.851 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 2e:01:84:b9:f4:8e brd ff:ff:ff:ff:ff:ff 2023-10-09T06:37:57.851 INFO:teuthology.orchestra.run.smithi163.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-09T06:37:57.852 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:37:57.852 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::d03c:93ff:fe9e:c8cb/64 scope link 2023-10-09T06:37:57.852 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:37:57.852 INFO:teuthology.orchestra.run.smithi163.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:37:57.852 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 2e:01:84:b9:f4:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-09T06:37:57.852 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::2c01:84ff:feb9:f48e/64 scope link 2023-10-09T06:37:57.852 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:37:57.853 INFO:teuthology.orchestra.run.smithi163.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:37:57.853 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether c6:88:3e:d7:c1:56 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-09T06:37:57.853 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::c488:3eff:fed7:c156/64 scope link 2023-10-09T06:37:57.854 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:37:57.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:57.854 DEBUG:teuthology.orchestra.run.smithi163:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-09T06:37:57.911 INFO:teuthology.orchestra.run.smithi163.stdout:1 2023-10-09T06:37:57.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:57.913 DEBUG:teuthology.orchestra.run.smithi163:> route 2023-10-09T06:37:57.968 INFO:teuthology.orchestra.run.smithi163.stdout:Kernel IP routing table 2023-10-09T06:37:57.969 INFO:teuthology.orchestra.run.smithi163.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-09T06:37:57.969 INFO:teuthology.orchestra.run.smithi163.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-09T06:37:57.969 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-09T06:37:57.969 INFO:teuthology.orchestra.run.smithi163.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-09T06:37:57.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:57.969 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-09T06:37:57.969 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-09T06:37:57.970 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-09T06:37:57.970 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-09T06:37:57.970 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-09T06:37:57.970 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-09T06:37:58.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:58.068 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2023-10-09T06:37:58.082 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-09T06:37:58.083 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-09T06:37:58.083 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:37:58.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:58.083 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set brx.0 up 2023-10-09T06:37:58.147 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:37:58.147 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-09T06:37:58.201 INFO:teuthology.orchestra.run.smithi163.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:37:58.201 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:37:58.201 DEBUG:teuthology.orchestra.run.smithi163:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-09T06:37:58.256 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-09T06:37:58.256 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-09T06:37:58.256 DEBUG:teuthology.orchestra.run.smithi163:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:37:58.335 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:37:58.335 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:37:58.354 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:37:58.354 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:37:58.354 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:37:58.354 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with new device syntax: 0@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:37:58.354 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:37:58.354 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:37:58.354 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:37:58.354 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:37:58.376 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:37:58.376 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:37:58.376 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:37:58.376 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with new device syntax: 0@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:37:58.376 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:37:58.376 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:37:58.376 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:37:58.376 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:37:58.377 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790" 2023-10-09T06:37:58.377 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790:/ 2023-10-09T06:37:58.377 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=2520b8f7-539f-4bae-9f17-1fce66f66ddd" will pass to kernel 2023-10-09T06:37:58.378 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-09T06:37:58.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:58.378 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-09T06:37:58.443 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-09T06:37:58.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.599+0000 7ff4bffff700 1 Processor -- start 2023-10-09T06:37:58.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.599+0000 7ff4bffff700 1 -- start start 2023-10-09T06:37:58.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.600+0000 7ff4bffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff4c0159420 0x7ff4c00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:58.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.600+0000 7ff4bffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4c01597f0 0x7ff4c00b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:58.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.600+0000 7ff4bffff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4c00b11d0 con 0x7ff4c015d230 2023-10-09T06:37:58.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.600+0000 7ff4bffff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4c00b14b0 con 0x7ff4c0159420 2023-10-09T06:37:58.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.600+0000 7ff4bffff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c00b1790 con 0x7ff4c01597f0 2023-10-09T06:37:58.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.600+0000 7ff4bf7fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff4c015d230 0x7ff4c00b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:46938/0 (socket says 172.21.15.163:46938) 2023-10-09T06:37:58.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.600+0000 7ff4bf7fe700 1 -- 172.21.15.163:0/3060543733 learned_addr learned my addr 172.21.15.163:0/3060543733 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:37:58.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.600+0000 7ff4beffd700 1 --2- 172.21.15.163:0/3060543733 >> v2:172.21.15.181:3300/0 conn(0x7ff4c0159420 0x7ff4c00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:58.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.600+0000 7ff4b67fc700 1 --2- 172.21.15.163:0/3060543733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4c01597f0 0x7ff4c00b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:58.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.600+0000 7ff4be7fc700 1 -- 172.21.15.163:0/3060543733 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3449980989 0 0) 0x7ff4c00b11d0 con 0x7ff4c015d230 2023-10-09T06:37:58.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.600+0000 7ff4be7fc700 1 -- 172.21.15.163:0/3060543733 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4ac003570 con 0x7ff4c015d230 2023-10-09T06:37:58.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.600+0000 7ff4be7fc700 1 -- 172.21.15.163:0/3060543733 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2711157840 0 0) 0x7ff4ac003570 con 0x7ff4c015d230 2023-10-09T06:37:58.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.600+0000 7ff4b67fc700 1 -- 172.21.15.163:0/3060543733 >> v2:172.21.15.181:3300/0 conn(0x7ff4c0159420 msgr2=0x7ff4c00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:58.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.601+0000 7ff4b67fc700 1 --2- 172.21.15.163:0/3060543733 >> v2:172.21.15.181:3300/0 conn(0x7ff4c0159420 0x7ff4c00b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:58.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.601+0000 7ff4b67fc700 1 -- 172.21.15.163:0/3060543733 >> v1:172.21.15.163:6789/0 conn(0x7ff4c015d230 legacy=0x7ff4c00b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:58.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.601+0000 7ff4b67fc700 1 -- 172.21.15.163:0/3060543733 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c00b1a70 con 0x7ff4c01597f0 2023-10-09T06:37:58.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.601+0000 7ff4beffd700 1 --2- 172.21.15.163:0/3060543733 >> v2:172.21.15.181:3300/0 conn(0x7ff4c0159420 0x7ff4c00b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:37:58.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.601+0000 7ff4b67fc700 1 --2- 172.21.15.163:0/3060543733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4c01597f0 0x7ff4c00b2270 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80098e0 tx=0x7ff4a80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f175fdbd2d0a152 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:58.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.601+0000 7ff4be7fc700 1 -- 172.21.15.163:0/3060543733 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b80017f0 con 0x7ff4c01597f0 2023-10-09T06:37:58.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.601+0000 7ff4be7fc700 1 -- 172.21.15.163:0/3060543733 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a8011420 con 0x7ff4c01597f0 2023-10-09T06:37:58.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.601+0000 7ff4be7fc700 1 -- 172.21.15.163:0/3060543733 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a80117c0 con 0x7ff4c01597f0 2023-10-09T06:37:58.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.601+0000 7ff4bffff700 1 -- 172.21.15.163:0/3060543733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4c01597f0 msgr2=0x7ff4c00b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:58.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.601+0000 7ff4bffff700 1 --2- 172.21.15.163:0/3060543733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4c01597f0 0x7ff4c00b2270 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80098e0 tx=0x7ff4a80091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:58.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.601+0000 7ff4bffff700 1 -- 172.21.15.163:0/3060543733 shutdown_connections 2023-10-09T06:37:58.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.601+0000 7ff4bffff700 1 --2- 172.21.15.163:0/3060543733 >> v2:172.21.15.181:3300/0 conn(0x7ff4c0159420 0x7ff4c00b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:58.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.601+0000 7ff4bffff700 1 --2- 172.21.15.163:0/3060543733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4c01597f0 0x7ff4c00b2270 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:58.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.601+0000 7ff4bffff700 1 -- 172.21.15.163:0/3060543733 >> 172.21.15.163:0/3060543733 conn(0x7ff4c00d0660 msgr2=0x7ff4c00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:58.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.601+0000 7ff4bffff700 1 -- 172.21.15.163:0/3060543733 shutdown_connections 2023-10-09T06:37:58.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.601+0000 7ff4bffff700 1 -- 172.21.15.163:0/3060543733 wait complete. 2023-10-09T06:37:58.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.603+0000 7ff4bffff700 1 Processor -- start 2023-10-09T06:37:58.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.603+0000 7ff4bffff700 1 -- start start 2023-10-09T06:37:58.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.603+0000 7ff4bffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4c0159420 0x7ff4c0154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:58.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.603+0000 7ff4bffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4c01597f0 0x7ff4c0169e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:58.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.604+0000 7ff4bffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4c015d230 0x7ff4c016a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:58.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.604+0000 7ff4bffff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c01548a0 con 0x7ff4c0159420 2023-10-09T06:37:58.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.604+0000 7ff4bffff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c0169920 con 0x7ff4c015d230 2023-10-09T06:37:58.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.604+0000 7ff4bffff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c015c130 con 0x7ff4c01597f0 2023-10-09T06:37:58.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.604+0000 7ff4beffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4c0159420 0x7ff4c0154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:58.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.604+0000 7ff4beffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4c0159420 0x7ff4c0154360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42118/0 (socket says 172.21.15.163:42118) 2023-10-09T06:37:58.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.604+0000 7ff4beffd700 1 -- 172.21.15.163:0/426573535 learned_addr learned my addr 172.21.15.163:0/426573535 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:58.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.604+0000 7ff4b67fc700 1 --2- 172.21.15.163:0/426573535 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4c01597f0 0x7ff4c0169e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:58.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.604+0000 7ff4bf7fe700 1 --2- 172.21.15.163:0/426573535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4c015d230 0x7ff4c016a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:58.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.604+0000 7ff4beffd700 1 -- 172.21.15.163:0/426573535 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4c01597f0 msgr2=0x7ff4c0169e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:58.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.604+0000 7ff4beffd700 1 --2- 172.21.15.163:0/426573535 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4c01597f0 0x7ff4c0169e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:58.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.604+0000 7ff4beffd700 1 -- 172.21.15.163:0/426573535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4c015d230 msgr2=0x7ff4c016a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:58.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.604+0000 7ff4beffd700 1 --2- 172.21.15.163:0/426573535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4c015d230 0x7ff4c016a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:58.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.604+0000 7ff4beffd700 1 -- 172.21.15.163:0/426573535 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b0009420 con 0x7ff4c0159420 2023-10-09T06:37:58.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.604+0000 7ff4beffd700 1 --2- 172.21.15.163:0/426573535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4c0159420 0x7ff4c0154360 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7ff4b000ec80 tx=0x7ff4b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:58.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.604+0000 7ff4b77fe700 1 -- 172.21.15.163:0/426573535 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b000c920 con 0x7ff4c0159420 2023-10-09T06:37:58.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.604+0000 7ff4b77fe700 1 -- 172.21.15.163:0/426573535 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b000ca80 con 0x7ff4c0159420 2023-10-09T06:37:58.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.604+0000 7ff4bffff700 1 -- 172.21.15.163:0/426573535 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a8009420 con 0x7ff4c0159420 2023-10-09T06:37:58.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.604+0000 7ff4b77fe700 1 -- 172.21.15.163:0/426573535 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b000ce00 con 0x7ff4c0159420 2023-10-09T06:37:58.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.604+0000 7ff4bffff700 1 -- 172.21.15.163:0/426573535 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c015c6f0 con 0x7ff4c0159420 2023-10-09T06:37:58.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.605+0000 7ff4bffff700 1 -- 172.21.15.163:0/426573535 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4c004c3c0 con 0x7ff4c0159420 2023-10-09T06:37:58.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.606+0000 7ff4b77fe700 1 -- 172.21.15.163:0/426573535 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff4b0013680 con 0x7ff4c0159420 2023-10-09T06:37:58.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.606+0000 7ff4b77fe700 1 --2- 172.21.15.163:0/426573535 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff4a0067320 0x7ff4a00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:58.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.606+0000 7ff4b77fe700 1 -- 172.21.15.163:0/426573535 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff4b0085050 con 0x7ff4c0159420 2023-10-09T06:37:58.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.606+0000 7ff4b67fc700 1 --2- 172.21.15.163:0/426573535 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff4a0067320 0x7ff4a00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:58.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.607+0000 7ff4b67fc700 1 --2- 172.21.15.163:0/426573535 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff4a0067320 0x7ff4a00697d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80093f0 tx=0x7ff4a8016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:58.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.611+0000 7ff4b77fe700 1 -- 172.21.15.163:0/426573535 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff4b0051ce0 con 0x7ff4c0159420 2023-10-09T06:37:58.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.833+0000 7ff4bffff700 1 -- 172.21.15.163:0/426573535 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff4c00b1d50 con 0x7ff4c0159420 2023-10-09T06:37:58.835 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.834+0000 7ff4b77fe700 1 -- 172.21.15.163:0/426573535 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v93) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff4b0055300 con 0x7ff4c0159420 2023-10-09T06:37:58.835 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-09T06:37:58.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.836+0000 7ff4bffff700 1 -- 172.21.15.163:0/426573535 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff4a0067320 msgr2=0x7ff4a00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:58.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.836+0000 7ff4bffff700 1 --2- 172.21.15.163:0/426573535 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff4a0067320 0x7ff4a00697d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80093f0 tx=0x7ff4a8016000 comp rx=0 tx=0).stop 2023-10-09T06:37:58.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.836+0000 7ff4bffff700 1 -- 172.21.15.163:0/426573535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4c0159420 msgr2=0x7ff4c0154360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:58.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.836+0000 7ff4bffff700 1 --2- 172.21.15.163:0/426573535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4c0159420 0x7ff4c0154360 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7ff4b000ec80 tx=0x7ff4b00091a0 comp rx=0 tx=0).stop 2023-10-09T06:37:58.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.836+0000 7ff4bffff700 1 -- 172.21.15.163:0/426573535 shutdown_connections 2023-10-09T06:37:58.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.836+0000 7ff4bffff700 1 --2- 172.21.15.163:0/426573535 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff4a0067320 0x7ff4a00697d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:58.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.836+0000 7ff4bffff700 1 --2- 172.21.15.163:0/426573535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4c0159420 0x7ff4c0154360 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:58.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.836+0000 7ff4bffff700 1 --2- 172.21.15.163:0/426573535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4c015d230 0x7ff4c016a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:58.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.836+0000 7ff4bffff700 1 --2- 172.21.15.163:0/426573535 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4c01597f0 0x7ff4c0169e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:58.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.836+0000 7ff4bffff700 1 -- 172.21.15.163:0/426573535 >> 172.21.15.163:0/426573535 conn(0x7ff4c00d0660 msgr2=0x7ff4c00b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:58.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.836+0000 7ff4bffff700 1 -- 172.21.15.163:0/426573535 shutdown_connections 2023-10-09T06:37:58.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:58.836+0000 7ff4bffff700 1 -- 172.21.15.163:0/426573535 wait complete. 2023-10-09T06:37:58.848 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-09T06:37:58.849 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-09T06:37:58.849 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-09T06:37:58.849 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-09T06:37:58.849 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-09T06:37:58.849 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-09T06:37:58.849 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-09T06:37:58.849 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-09T06:37:58.849 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:37:58.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:58.849 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-09T06:37:58.869 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:37:58.870 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:37:58.870 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:37:58.870 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:37:58.870 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-09T06:37:58.870 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:37:58.871 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:37:58.871 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:37:58.871 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2023-10-09T06:37:58.871 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:37:58.871 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:37:58.871 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2023-10-09T06:37:58.872 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:37:58.872 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-09T06:37:58.872 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-09T06:37:58.872 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-09T06:37:58.872 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-09T06:37:58.873 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42021sec preferred_lft 42021sec 2023-10-09T06:37:58.873 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2023-10-09T06:37:58.873 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:37:58.873 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-09T06:37:58.873 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 72:62:e6:72:7c:2e brd ff:ff:ff:ff:ff:ff 2023-10-09T06:37:58.874 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-09T06:37:58.874 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:37:58.874 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::e89a:4cff:fefb:7198/64 scope link 2023-10-09T06:37:58.875 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:37:58.875 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:37:58.875 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 72:62:e6:72:7c:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-09T06:37:58.875 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::7062:e6ff:fe72:7c2e/64 scope link 2023-10-09T06:37:58.875 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:37:58.875 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:37:58.875 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether fa:4e:a3:9d:5a:5f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-09T06:37:58.876 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::f84e:a3ff:fe9d:5a5f/64 scope link 2023-10-09T06:37:58.876 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:37:58.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:58.876 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-09T06:37:58.938 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-09T06:37:58.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:58.939 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-09T06:37:58.998 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-09T06:37:58.998 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-09T06:37:58.998 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-09T06:37:58.998 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-09T06:37:58.999 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-09T06:37:58.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:58.999 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-09T06:37:58.999 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-09T06:37:58.999 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-09T06:37:58.999 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-09T06:37:58.999 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-09T06:37:58.999 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-09T06:37:59.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:59.104 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-09T06:37:59.119 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-09T06:37:59.119 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-09T06:37:59.120 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:37:59.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:59.120 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-09T06:37:59.184 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:37:59.185 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-09T06:37:59.240 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:37:59.240 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:37:59.240 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-09T06:37:59.294 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-09T06:37:59.294 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-09T06:37:59.294 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:37:59.373 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:37:59.374 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:37:59.392 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:37:59.392 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:37:59.392 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:37:59.392 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 1@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:37:59.392 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:37:59.392 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:37:59.393 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:37:59.393 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:37:59.414 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:37:59.414 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:37:59.414 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:37:59.414 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 1@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:37:59.414 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:37:59.414 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:37:59.414 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:37:59.414 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:37:59.415 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790" 2023-10-09T06:37:59.415 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790:/ 2023-10-09T06:37:59.415 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=2520b8f7-539f-4bae-9f17-1fce66f66ddd" will pass to kernel 2023-10-09T06:37:59.416 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-09T06:37:59.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:37:59.416 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-09T06:37:59.480 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-09T06:37:59.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.600+0000 7fe227ca4700 1 Processor -- start 2023-10-09T06:37:59.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.601+0000 7fe227ca4700 1 -- start start 2023-10-09T06:37:59.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.602+0000 7fe227ca4700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe220159ec0 0x7fe2200b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:59.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.602+0000 7fe227ca4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe2200cf350 0x7fe2200cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:59.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.602+0000 7fe227ca4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2200b5b00 con 0x7fe2200b53d0 2023-10-09T06:37:59.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.602+0000 7fe227ca4700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2200b5850 con 0x7fe220159ec0 2023-10-09T06:37:59.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.602+0000 7fe227ca4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2200b5da0 con 0x7fe2200cf350 2023-10-09T06:37:59.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.602+0000 7fe225a40700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe220159ec0 0x7fe2200b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:59.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.602+0000 7fe225a40700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe220159ec0 0x7fe2200b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42132/0 (socket says 172.21.15.163:42132) 2023-10-09T06:37:59.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.602+0000 7fe225a40700 1 -- 172.21.15.163:0/3064683141 learned_addr learned my addr 172.21.15.163:0/3064683141 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:59.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.602+0000 7fe226241700 1 --2- 172.21.15.163:0/3064683141 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe2200cf350 0x7fe2200cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:59.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.602+0000 7fe225a40700 1 -- 172.21.15.163:0/3064683141 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe2200cf350 msgr2=0x7fe2200cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:59.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.602+0000 7fe225a40700 1 --2- 172.21.15.163:0/3064683141 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe2200cf350 0x7fe2200cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:59.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.602+0000 7fe225a40700 1 -- 172.21.15.163:0/3064683141 >> v1:172.21.15.181:6789/0 conn(0x7fe2200b53d0 legacy=0x7fe2200cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:37:59.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.602+0000 7fe225a40700 1 -- 172.21.15.163:0/3064683141 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2200b6080 con 0x7fe220159ec0 2023-10-09T06:37:59.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.602+0000 7fe225a40700 1 --2- 172.21.15.163:0/3064683141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe220159ec0 0x7fe2200b4e90 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009990 tx=0x7fe21c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6093017d247fd72 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:59.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.602+0000 7fe224a3e700 1 -- 172.21.15.163:0/3064683141 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c005c10 con 0x7fe220159ec0 2023-10-09T06:37:59.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.602+0000 7fe224a3e700 1 -- 172.21.15.163:0/3064683141 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe21c005d70 con 0x7fe220159ec0 2023-10-09T06:37:59.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.602+0000 7fe224a3e700 1 -- 172.21.15.163:0/3064683141 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c00b880 con 0x7fe220159ec0 2023-10-09T06:37:59.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.603+0000 7fe227ca4700 1 -- 172.21.15.163:0/3064683141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe220159ec0 msgr2=0x7fe2200b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:59.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.603+0000 7fe227ca4700 1 --2- 172.21.15.163:0/3064683141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe220159ec0 0x7fe2200b4e90 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009990 tx=0x7fe21c009220 comp rx=0 tx=0).stop 2023-10-09T06:37:59.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.603+0000 7fe227ca4700 1 -- 172.21.15.163:0/3064683141 shutdown_connections 2023-10-09T06:37:59.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.603+0000 7fe227ca4700 1 --2- 172.21.15.163:0/3064683141 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe2200cf350 0x7fe2200cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:59.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.603+0000 7fe227ca4700 1 --2- 172.21.15.163:0/3064683141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe220159ec0 0x7fe2200b4e90 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:59.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.603+0000 7fe227ca4700 1 -- 172.21.15.163:0/3064683141 >> 172.21.15.163:0/3064683141 conn(0x7fe2200999f0 msgr2=0x7fe2200b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:37:59.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.603+0000 7fe227ca4700 1 -- 172.21.15.163:0/3064683141 shutdown_connections 2023-10-09T06:37:59.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.603+0000 7fe227ca4700 1 -- 172.21.15.163:0/3064683141 wait complete. 2023-10-09T06:37:59.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.604+0000 7fe227ca4700 1 Processor -- start 2023-10-09T06:37:59.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.604+0000 7fe227ca4700 1 -- start start 2023-10-09T06:37:59.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.605+0000 7fe227ca4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe2200b53d0 0x7fe220152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:59.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.605+0000 7fe227ca4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2200cf350 0x7fe2200c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:59.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.605+0000 7fe227ca4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe220159ec0 0x7fe2200c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:59.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.605+0000 7fe227ca4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe220154c20 con 0x7fe220159ec0 2023-10-09T06:37:59.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.605+0000 7fe227ca4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe220154f00 con 0x7fe2200cf350 2023-10-09T06:37:59.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.605+0000 7fe227ca4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2201551e0 con 0x7fe2200b53d0 2023-10-09T06:37:59.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.605+0000 7fe226241700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe220159ec0 0x7fe2200c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:59.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.605+0000 7fe226241700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe220159ec0 0x7fe2200c9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42134/0 (socket says 172.21.15.163:42134) 2023-10-09T06:37:59.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.605+0000 7fe226241700 1 -- 172.21.15.163:0/1251362172 learned_addr learned my addr 172.21.15.163:0/1251362172 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:37:59.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.605+0000 7fe225a40700 1 --2- 172.21.15.163:0/1251362172 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe2200b53d0 0x7fe220152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:59.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.605+0000 7fe22523f700 1 --2- 172.21.15.163:0/1251362172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2200cf350 0x7fe2200c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:59.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.605+0000 7fe226241700 1 -- 172.21.15.163:0/1251362172 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe2200b53d0 msgr2=0x7fe220152e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:59.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.605+0000 7fe226241700 1 --2- 172.21.15.163:0/1251362172 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe2200b53d0 0x7fe220152e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:59.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.605+0000 7fe226241700 1 -- 172.21.15.163:0/1251362172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2200cf350 msgr2=0x7fe2200c98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:37:59.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.605+0000 7fe226241700 1 --2- 172.21.15.163:0/1251362172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2200cf350 0x7fe2200c98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:37:59.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.605+0000 7fe226241700 1 -- 172.21.15.163:0/1251362172 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe21c009480 con 0x7fe220159ec0 2023-10-09T06:37:59.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.605+0000 7fe226241700 1 --2- 172.21.15.163:0/1251362172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe220159ec0 0x7fe2200c9e00 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fe218009fa0 tx=0x7fe218009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:59.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.605+0000 7fe215ffb700 1 -- 172.21.15.163:0/1251362172 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21800ed20 con 0x7fe220159ec0 2023-10-09T06:37:59.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.605+0000 7fe215ffb700 1 -- 172.21.15.163:0/1251362172 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe21800ee80 con 0x7fe220159ec0 2023-10-09T06:37:59.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.605+0000 7fe227ca4700 1 -- 172.21.15.163:0/1251362172 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2201554c0 con 0x7fe220159ec0 2023-10-09T06:37:59.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.605+0000 7fe215ffb700 1 -- 172.21.15.163:0/1251362172 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe218014670 con 0x7fe220159ec0 2023-10-09T06:37:59.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.606+0000 7fe227ca4700 1 -- 172.21.15.163:0/1251362172 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe220153350 con 0x7fe220159ec0 2023-10-09T06:37:59.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.606+0000 7fe227ca4700 1 -- 172.21.15.163:0/1251362172 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe220153570 con 0x7fe220159ec0 2023-10-09T06:37:59.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.607+0000 7fe215ffb700 1 -- 172.21.15.163:0/1251362172 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe2180149e0 con 0x7fe220159ec0 2023-10-09T06:37:59.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.607+0000 7fe215ffb700 1 --2- 172.21.15.163:0/1251362172 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe204067320 0x7fe2040697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:37:59.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.607+0000 7fe215ffb700 1 -- 172.21.15.163:0/1251362172 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe218085460 con 0x7fe220159ec0 2023-10-09T06:37:59.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.607+0000 7fe225a40700 1 --2- 172.21.15.163:0/1251362172 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe204067320 0x7fe2040697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:37:59.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.608+0000 7fe225a40700 1 --2- 172.21.15.163:0/1251362172 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe204067320 0x7fe2040697d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009260 tx=0x7fe21c016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:37:59.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.612+0000 7fe215ffb700 1 -- 172.21.15.163:0/1251362172 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2180520f0 con 0x7fe220159ec0 2023-10-09T06:37:59.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:37:59.836+0000 7fe227ca4700 1 -- 172.21.15.163:0/1251362172 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe220152960 con 0x7fe220159ec0 2023-10-09T06:38:00.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.559+0000 7fe215ffb700 1 -- 172.21.15.163:0/1251362172 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v94) v1 ==== 136+0+0 (secure 0 0 0) 0x7fe218055710 con 0x7fe220159ec0 2023-10-09T06:38:00.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.562+0000 7fe227ca4700 1 -- 172.21.15.163:0/1251362172 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe204067320 msgr2=0x7fe2040697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:00.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.562+0000 7fe227ca4700 1 --2- 172.21.15.163:0/1251362172 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe204067320 0x7fe2040697d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009260 tx=0x7fe21c016000 comp rx=0 tx=0).stop 2023-10-09T06:38:00.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.562+0000 7fe227ca4700 1 -- 172.21.15.163:0/1251362172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe220159ec0 msgr2=0x7fe2200c9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:00.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.562+0000 7fe227ca4700 1 --2- 172.21.15.163:0/1251362172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe220159ec0 0x7fe2200c9e00 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fe218009fa0 tx=0x7fe218009900 comp rx=0 tx=0).stop 2023-10-09T06:38:00.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.562+0000 7fe227ca4700 1 -- 172.21.15.163:0/1251362172 shutdown_connections 2023-10-09T06:38:00.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.562+0000 7fe227ca4700 1 --2- 172.21.15.163:0/1251362172 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe204067320 0x7fe2040697d0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:00.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.562+0000 7fe227ca4700 1 --2- 172.21.15.163:0/1251362172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe220159ec0 0x7fe2200c9e00 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:00.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.562+0000 7fe227ca4700 1 --2- 172.21.15.163:0/1251362172 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe2200b53d0 0x7fe220152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:00.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.562+0000 7fe227ca4700 1 --2- 172.21.15.163:0/1251362172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2200cf350 0x7fe2200c98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:00.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.562+0000 7fe227ca4700 1 -- 172.21.15.163:0/1251362172 >> 172.21.15.163:0/1251362172 conn(0x7fe2200999f0 msgr2=0x7fe220152250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:00.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.562+0000 7fe227ca4700 1 -- 172.21.15.163:0/1251362172 shutdown_connections 2023-10-09T06:38:00.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.562+0000 7fe227ca4700 1 -- 172.21.15.163:0/1251362172 wait complete. 2023-10-09T06:38:00.576 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-09T06:38:00.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.693+0000 7f07b1ec0700 1 Processor -- start 2023-10-09T06:38:00.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.694+0000 7f07b1ec0700 1 -- start start 2023-10-09T06:38:00.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.694+0000 7f07b1ec0700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f07ac06aa40 0x7f07ac0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:00.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.694+0000 7f07b1ec0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07ac06bac0 0x7f07ac06ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:00.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.694+0000 7f07b1ec0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f07ac06b350 0x7f07ac0c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:00.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.694+0000 7f07b1ec0700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07ac0cc210 con 0x7f07ac06aa40 2023-10-09T06:38:00.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.694+0000 7f07b1ec0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07ac0cc4f0 con 0x7f07ac06b350 2023-10-09T06:38:00.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.694+0000 7f07b1ec0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac0cc7d0 con 0x7f07ac06bac0 2023-10-09T06:38:00.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.694+0000 7f07ab7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f07ac06aa40 0x7f07ac0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:00.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.695+0000 7f07ab7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f07ac06aa40 0x7f07ac0cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42140/0 (socket says 172.21.15.163:42140) 2023-10-09T06:38:00.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.695+0000 7f07ab7fe700 1 -- 172.21.15.163:0/1833474412 learned_addr learned my addr 172.21.15.163:0/1833474412 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:00.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.695+0000 7f07abfff700 1 --2- 172.21.15.163:0/1833474412 >> v2:172.21.15.181:3300/0 conn(0x7f07ac06b350 0x7f07ac0c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:00.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.695+0000 7f07aaffd700 1 --2- 172.21.15.163:0/1833474412 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07ac06bac0 0x7f07ac06ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:00.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.695+0000 7f07ab7fe700 1 -- 172.21.15.163:0/1833474412 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07ac06bac0 msgr2=0x7f07ac06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:00.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.695+0000 7f07ab7fe700 1 --2- 172.21.15.163:0/1833474412 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07ac06bac0 0x7f07ac06ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:00.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.695+0000 7f07ab7fe700 1 -- 172.21.15.163:0/1833474412 >> v2:172.21.15.181:3300/0 conn(0x7f07ac06b350 msgr2=0x7f07ac0c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:00.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.695+0000 7f07ab7fe700 1 --2- 172.21.15.163:0/1833474412 >> v2:172.21.15.181:3300/0 conn(0x7f07ac06b350 0x7f07ac0c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:00.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.695+0000 7f07ab7fe700 1 -- 172.21.15.163:0/1833474412 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ac0ccab0 con 0x7f07ac06aa40 2023-10-09T06:38:00.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.695+0000 7f07ab7fe700 1 --2- 172.21.15.163:0/1833474412 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07ac06aa40 0x7f07ac0cbc00 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0c7c40 tx=0x7f079c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18dda01d18ff7a8a server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:00.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.695+0000 7f07aa7fc700 1 -- 172.21.15.163:0/1833474412 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c00e070 con 0x7f07ac06aa40 2023-10-09T06:38:00.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.695+0000 7f07aa7fc700 1 -- 172.21.15.163:0/1833474412 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079c00b970 con 0x7f07ac06aa40 2023-10-09T06:38:00.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.695+0000 7f07aa7fc700 1 -- 172.21.15.163:0/1833474412 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c0092e0 con 0x7f07ac06aa40 2023-10-09T06:38:00.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.696+0000 7f07b1ec0700 1 -- 172.21.15.163:0/1833474412 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07ac06aa40 msgr2=0x7f07ac0cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:00.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.696+0000 7f07b1ec0700 1 --2- 172.21.15.163:0/1833474412 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07ac06aa40 0x7f07ac0cbc00 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0c7c40 tx=0x7f079c009840 comp rx=0 tx=0).stop 2023-10-09T06:38:00.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.696+0000 7f07b1ec0700 1 -- 172.21.15.163:0/1833474412 shutdown_connections 2023-10-09T06:38:00.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.696+0000 7f07b1ec0700 1 --2- 172.21.15.163:0/1833474412 >> v2:172.21.15.181:3300/0 conn(0x7f07ac06b350 0x7f07ac0c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:00.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.696+0000 7f07b1ec0700 1 --2- 172.21.15.163:0/1833474412 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07ac06aa40 0x7f07ac0cbc00 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:00.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.696+0000 7f07b1ec0700 1 --2- 172.21.15.163:0/1833474412 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07ac06bac0 0x7f07ac06ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:00.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.696+0000 7f07b1ec0700 1 -- 172.21.15.163:0/1833474412 >> 172.21.15.163:0/1833474412 conn(0x7f07ac0cd670 msgr2=0x7f07ac0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:00.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.696+0000 7f07b1ec0700 1 -- 172.21.15.163:0/1833474412 shutdown_connections 2023-10-09T06:38:00.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.696+0000 7f07b1ec0700 1 -- 172.21.15.163:0/1833474412 wait complete. 2023-10-09T06:38:00.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.697+0000 7f07b1ec0700 1 Processor -- start 2023-10-09T06:38:00.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.697+0000 7f07b1ec0700 1 -- start start 2023-10-09T06:38:00.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.698+0000 7f07b1ec0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07ac06aa40 0x7f07ac169c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:00.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.698+0000 7f07b1ec0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07ac06b350 0x7f07ac068da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:00.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.698+0000 7f07b1ec0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07ac06bac0 0x7f07ac0b1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:00.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.698+0000 7f07b1ec0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac1534e0 con 0x7f07ac06aa40 2023-10-09T06:38:00.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.698+0000 7f07b1ec0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac0c5f60 con 0x7f07ac06bac0 2023-10-09T06:38:00.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.698+0000 7f07b1ec0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac0b1740 con 0x7f07ac06b350 2023-10-09T06:38:00.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.698+0000 7f07ab7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07ac06aa40 0x7f07ac169c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:00.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.698+0000 7f07ab7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07ac06aa40 0x7f07ac169c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42150/0 (socket says 172.21.15.163:42150) 2023-10-09T06:38:00.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.698+0000 7f07ab7fe700 1 -- 172.21.15.163:0/2228191429 learned_addr learned my addr 172.21.15.163:0/2228191429 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:00.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.698+0000 7f07ab7fe700 1 -- 172.21.15.163:0/2228191429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07ac06b350 msgr2=0x7f07ac068da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:00.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.698+0000 7f07aaffd700 1 --2- 172.21.15.163:0/2228191429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07ac06b350 0x7f07ac068da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:00.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.698+0000 7f07abfff700 1 --2- 172.21.15.163:0/2228191429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07ac06bac0 0x7f07ac0b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:00.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.698+0000 7f07ab7fe700 1 --2- 172.21.15.163:0/2228191429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07ac06b350 0x7f07ac068da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:00.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.698+0000 7f07ab7fe700 1 -- 172.21.15.163:0/2228191429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07ac06bac0 msgr2=0x7f07ac0b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:00.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.698+0000 7f07ab7fe700 1 --2- 172.21.15.163:0/2228191429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07ac06bac0 0x7f07ac0b1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:00.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.698+0000 7f07ab7fe700 1 -- 172.21.15.163:0/2228191429 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079c009ae0 con 0x7f07ac06aa40 2023-10-09T06:38:00.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.698+0000 7f07ab7fe700 1 --2- 172.21.15.163:0/2228191429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07ac06aa40 0x7f07ac169c80 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f079c012000 tx=0x7f079c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:00.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.698+0000 7f07937fe700 1 -- 172.21.15.163:0/2228191429 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c008950 con 0x7f07ac06aa40 2023-10-09T06:38:00.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.698+0000 7f07937fe700 1 -- 172.21.15.163:0/2228191429 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079c008ad0 con 0x7f07ac06aa40 2023-10-09T06:38:00.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.699+0000 7f07b1ec0700 1 -- 172.21.15.163:0/2228191429 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07ac16a300 con 0x7f07ac06aa40 2023-10-09T06:38:00.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.699+0000 7f07937fe700 1 -- 172.21.15.163:0/2228191429 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c008e90 con 0x7f07ac06aa40 2023-10-09T06:38:00.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.699+0000 7f07b1ec0700 1 -- 172.21.15.163:0/2228191429 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07ac16a8c0 con 0x7f07ac06aa40 2023-10-09T06:38:00.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.699+0000 7f07b1ec0700 1 -- 172.21.15.163:0/2228191429 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07ac16ac00 con 0x7f07ac06aa40 2023-10-09T06:38:00.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.700+0000 7f07937fe700 1 -- 172.21.15.163:0/2228191429 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f079c003040 con 0x7f07ac06aa40 2023-10-09T06:38:00.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.700+0000 7f07937fe700 1 --2- 172.21.15.163:0/2228191429 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0798067320 0x7f07980697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:00.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.700+0000 7f07937fe700 1 -- 172.21.15.163:0/2228191429 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f079c086a70 con 0x7f07ac06aa40 2023-10-09T06:38:00.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.700+0000 7f07aaffd700 1 --2- 172.21.15.163:0/2228191429 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0798067320 0x7f07980697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:00.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.701+0000 7f07aaffd700 1 --2- 172.21.15.163:0/2228191429 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0798067320 0x7f07980697d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f07ac06f8b0 tx=0x7f07a0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:00.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.704+0000 7f07937fe700 1 -- 172.21.15.163:0/2228191429 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f079c0536a0 con 0x7f07ac06aa40 2023-10-09T06:38:00.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.915+0000 7f07b1ec0700 1 -- 172.21.15.163:0/2228191429 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f07ac04c3c0 con 0x7f0798067320 2023-10-09T06:38:00.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.917+0000 7f07937fe700 1 -- 172.21.15.163:0/2228191429 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f07ac04c3c0 con 0x7f0798067320 2023-10-09T06:38:00.918 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:38:00.918 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:00.918 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "cephfs" 2023-10-09T06:38:00.918 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:00.918 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:38:00.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.919+0000 7f07b1ec0700 1 -- 172.21.15.163:0/2228191429 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0798067320 msgr2=0x7f07980697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:00.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.919+0000 7f07b1ec0700 1 --2- 172.21.15.163:0/2228191429 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0798067320 0x7f07980697d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f07ac06f8b0 tx=0x7f07a0008040 comp rx=0 tx=0).stop 2023-10-09T06:38:00.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.919+0000 7f07b1ec0700 1 -- 172.21.15.163:0/2228191429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07ac06aa40 msgr2=0x7f07ac169c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:00.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.919+0000 7f07b1ec0700 1 --2- 172.21.15.163:0/2228191429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07ac06aa40 0x7f07ac169c80 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f079c012000 tx=0x7f079c018000 comp rx=0 tx=0).stop 2023-10-09T06:38:00.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.919+0000 7f07b1ec0700 1 -- 172.21.15.163:0/2228191429 shutdown_connections 2023-10-09T06:38:00.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.919+0000 7f07b1ec0700 1 --2- 172.21.15.163:0/2228191429 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0798067320 0x7f07980697d0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:00.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.919+0000 7f07b1ec0700 1 --2- 172.21.15.163:0/2228191429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07ac06aa40 0x7f07ac169c80 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:00.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.919+0000 7f07b1ec0700 1 --2- 172.21.15.163:0/2228191429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07ac06bac0 0x7f07ac0b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:00.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.919+0000 7f07b1ec0700 1 --2- 172.21.15.163:0/2228191429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07ac06b350 0x7f07ac068da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:00.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.919+0000 7f07b1ec0700 1 -- 172.21.15.163:0/2228191429 >> 172.21.15.163:0/2228191429 conn(0x7f07ac0cd670 msgr2=0x7f07ac152570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:00.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.919+0000 7f07b1ec0700 1 -- 172.21.15.163:0/2228191429 shutdown_connections 2023-10-09T06:38:00.921 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:00.919+0000 7f07b1ec0700 1 -- 172.21.15.163:0/2228191429 wait complete. 2023-10-09T06:38:00.932 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-09T06:38:01.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.049+0000 7f385b3d7700 1 Processor -- start 2023-10-09T06:38:01.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.049+0000 7f385b3d7700 1 -- start start 2023-10-09T06:38:01.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.050+0000 7f385b3d7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3854159c60 0x7f385415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:01.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.050+0000 7f385b3d7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f385415db10 0x7f3854069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:01.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.050+0000 7f385b3d7700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f385415a160 con 0x7f3854159890 2023-10-09T06:38:01.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.050+0000 7f385b3d7700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38540d13a0 con 0x7f385415db10 2023-10-09T06:38:01.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.050+0000 7f385b3d7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385415e140 con 0x7f3854159c60 2023-10-09T06:38:01.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.050+0000 7f3859974700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f385415db10 0x7f3854069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:01.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.050+0000 7f3859974700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f385415db10 0x7f3854069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42164/0 (socket says 172.21.15.163:42164) 2023-10-09T06:38:01.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.050+0000 7f3859173700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f3854159890 0x7f385415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:46942/0 (socket says 172.21.15.163:46942) 2023-10-09T06:38:01.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.050+0000 7f3859974700 1 -- 172.21.15.163:0/3700568497 learned_addr learned my addr 172.21.15.163:0/3700568497 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:01.054 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.051+0000 7f3858972700 1 --2- 172.21.15.163:0/3700568497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3854159c60 0x7f385415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:01.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.051+0000 7f3859974700 1 -- 172.21.15.163:0/3700568497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3854159c60 msgr2=0x7f385415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:01.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.051+0000 7f3859974700 1 --2- 172.21.15.163:0/3700568497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3854159c60 0x7f385415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:01.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.051+0000 7f3859974700 1 -- 172.21.15.163:0/3700568497 >> v1:172.21.15.163:6789/0 conn(0x7f3854159890 legacy=0x7f385415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:01.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.051+0000 7f3859974700 1 -- 172.21.15.163:0/3700568497 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385415a3d0 con 0x7f385415db10 2023-10-09T06:38:01.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.051+0000 7f384bfff700 1 -- 172.21.15.163:0/3700568497 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1690030312 0 0) 0x7f385415a160 con 0x7f3854159890 2023-10-09T06:38:01.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.051+0000 7f3858972700 1 --2- 172.21.15.163:0/3700568497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3854159c60 0x7f385415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:01.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.051+0000 7f3859974700 1 --2- 172.21.15.163:0/3700568497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f385415db10 0x7f3854069560 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f384c009f20 tx=0x7f384c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294f36147992cd19 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:01.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.051+0000 7f384bfff700 1 -- 172.21.15.163:0/3700568497 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c00e070 con 0x7f385415db10 2023-10-09T06:38:01.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.051+0000 7f384bfff700 1 -- 172.21.15.163:0/3700568497 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384c00ba80 con 0x7f385415db10 2023-10-09T06:38:01.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.051+0000 7f384bfff700 1 -- 172.21.15.163:0/3700568497 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c0092e0 con 0x7f385415db10 2023-10-09T06:38:01.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.051+0000 7f385b3d7700 1 -- 172.21.15.163:0/3700568497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f385415db10 msgr2=0x7f3854069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:01.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.051+0000 7f385b3d7700 1 --2- 172.21.15.163:0/3700568497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f385415db10 0x7f3854069560 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f384c009f20 tx=0x7f384c009840 comp rx=0 tx=0).stop 2023-10-09T06:38:01.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.051+0000 7f385b3d7700 1 -- 172.21.15.163:0/3700568497 shutdown_connections 2023-10-09T06:38:01.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.051+0000 7f385b3d7700 1 --2- 172.21.15.163:0/3700568497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f385415db10 0x7f3854069560 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:01.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.052+0000 7f385b3d7700 1 --2- 172.21.15.163:0/3700568497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3854159c60 0x7f385415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:01.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.052+0000 7f385b3d7700 1 -- 172.21.15.163:0/3700568497 >> 172.21.15.163:0/3700568497 conn(0x7f3854068e20 msgr2=0x7f38540d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:01.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.052+0000 7f385b3d7700 1 -- 172.21.15.163:0/3700568497 shutdown_connections 2023-10-09T06:38:01.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.052+0000 7f385b3d7700 1 -- 172.21.15.163:0/3700568497 wait complete. 2023-10-09T06:38:01.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.053+0000 7f385b3d7700 1 Processor -- start 2023-10-09T06:38:01.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.053+0000 7f385b3d7700 1 -- start start 2023-10-09T06:38:01.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.054+0000 7f385b3d7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3854159890 0x7f38540b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:01.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.054+0000 7f385b3d7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3854159c60 0x7f38540b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:01.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.054+0000 7f385b3d7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f385415db10 0x7f38540cd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:01.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.054+0000 7f385b3d7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38540b2550 con 0x7f385415db10 2023-10-09T06:38:01.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.054+0000 7f385b3d7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3854150a80 con 0x7f3854159c60 2023-10-09T06:38:01.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.054+0000 7f385b3d7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38540b0d50 con 0x7f3854159890 2023-10-09T06:38:01.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.054+0000 7f3859974700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f385415db10 0x7f38540cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:01.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.054+0000 7f3859974700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f385415db10 0x7f38540cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42172/0 (socket says 172.21.15.163:42172) 2023-10-09T06:38:01.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.054+0000 7f3859974700 1 -- 172.21.15.163:0/2461864563 learned_addr learned my addr 172.21.15.163:0/2461864563 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:01.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.054+0000 7f3859173700 1 --2- 172.21.15.163:0/2461864563 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3854159890 0x7f38540b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:01.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.054+0000 7f3859974700 1 -- 172.21.15.163:0/2461864563 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3854159890 msgr2=0x7f38540b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:01.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.054+0000 7f3858972700 1 --2- 172.21.15.163:0/2461864563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3854159c60 0x7f38540b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:01.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.054+0000 7f3859974700 1 --2- 172.21.15.163:0/2461864563 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3854159890 0x7f38540b19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:01.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.054+0000 7f3859974700 1 -- 172.21.15.163:0/2461864563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3854159c60 msgr2=0x7f38540b5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:01.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.054+0000 7f3859974700 1 --2- 172.21.15.163:0/2461864563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3854159c60 0x7f38540b5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:01.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.054+0000 7f3859974700 1 -- 172.21.15.163:0/2461864563 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384c009ae0 con 0x7f385415db10 2023-10-09T06:38:01.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.054+0000 7f3859974700 1 --2- 172.21.15.163:0/2461864563 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f385415db10 0x7f38540cd550 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f38540899c0 tx=0x7f384c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:01.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.054+0000 7f38497fa700 1 -- 172.21.15.163:0/2461864563 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c008dd0 con 0x7f385415db10 2023-10-09T06:38:01.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.054+0000 7f38497fa700 1 -- 172.21.15.163:0/2461864563 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384c01c410 con 0x7f385415db10 2023-10-09T06:38:01.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.054+0000 7f385b3d7700 1 -- 172.21.15.163:0/2461864563 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38540b1f90 con 0x7f385415db10 2023-10-09T06:38:01.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.054+0000 7f38497fa700 1 -- 172.21.15.163:0/2461864563 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c01c7d0 con 0x7f385415db10 2023-10-09T06:38:01.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.054+0000 7f385b3d7700 1 -- 172.21.15.163:0/2461864563 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3854150710 con 0x7f385415db10 2023-10-09T06:38:01.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.055+0000 7f385b3d7700 1 -- 172.21.15.163:0/2461864563 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3854150570 con 0x7f385415db10 2023-10-09T06:38:01.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.056+0000 7f38497fa700 1 -- 172.21.15.163:0/2461864563 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f384c003040 con 0x7f385415db10 2023-10-09T06:38:01.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.056+0000 7f38497fa700 1 --2- 172.21.15.163:0/2461864563 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3838067320 0x7f38380697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:01.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.056+0000 7f38497fa700 1 -- 172.21.15.163:0/2461864563 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f384c0867c0 con 0x7f385415db10 2023-10-09T06:38:01.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.056+0000 7f3859173700 1 --2- 172.21.15.163:0/2461864563 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3838067320 0x7f38380697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:01.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.057+0000 7f3859173700 1 --2- 172.21.15.163:0/2461864563 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3838067320 0x7f38380697d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3850007fd0 tx=0x7f3850009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:01.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.060+0000 7f38497fa700 1 -- 172.21.15.163:0/2461864563 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3854150570 con 0x7f385415db10 2023-10-09T06:38:01.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.244+0000 7f385b3d7700 1 -- 172.21.15.163:0/2461864563 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3854150570 con 0x7f385415db10 2023-10-09T06:38:01.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.244+0000 7f38497fa700 1 -- 172.21.15.163:0/2461864563 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3854150570 con 0x7f385415db10 2023-10-09T06:38:01.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.246+0000 7f385b3d7700 1 -- 172.21.15.163:0/2461864563 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3838067320 msgr2=0x7f38380697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:01.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.246+0000 7f385b3d7700 1 --2- 172.21.15.163:0/2461864563 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3838067320 0x7f38380697d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3850007fd0 tx=0x7f3850009040 comp rx=0 tx=0).stop 2023-10-09T06:38:01.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.246+0000 7f385b3d7700 1 -- 172.21.15.163:0/2461864563 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f385415db10 msgr2=0x7f38540cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:01.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.246+0000 7f385b3d7700 1 --2- 172.21.15.163:0/2461864563 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f385415db10 0x7f38540cd550 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f38540899c0 tx=0x7f384c018000 comp rx=0 tx=0).stop 2023-10-09T06:38:01.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.246+0000 7f385b3d7700 1 -- 172.21.15.163:0/2461864563 shutdown_connections 2023-10-09T06:38:01.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.246+0000 7f385b3d7700 1 --2- 172.21.15.163:0/2461864563 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3838067320 0x7f38380697d0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:01.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.246+0000 7f385b3d7700 1 --2- 172.21.15.163:0/2461864563 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f385415db10 0x7f38540cd550 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:01.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.246+0000 7f385b3d7700 1 --2- 172.21.15.163:0/2461864563 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3854159890 0x7f38540b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:01.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.246+0000 7f385b3d7700 1 --2- 172.21.15.163:0/2461864563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3854159c60 0x7f38540b5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:01.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.246+0000 7f385b3d7700 1 -- 172.21.15.163:0/2461864563 >> 172.21.15.163:0/2461864563 conn(0x7f3854068e20 msgr2=0x7f3854156ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:01.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.246+0000 7f385b3d7700 1 -- 172.21.15.163:0/2461864563 shutdown_connections 2023-10-09T06:38:01.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.246+0000 7f385b3d7700 1 -- 172.21.15.163:0/2461864563 wait complete. 2023-10-09T06:38:01.259 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-09T06:38:01.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:38:01.259 DEBUG:teuthology.orchestra.run.smithi163:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-09T06:38:01.297 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi163.front.sepia.ceph.com 2023-10-09T06:38:01.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:01.298 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-09T06:38:01.353 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-09T06:38:01.353 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:38:01.353 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-09T06:38:01.398 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-09T06:38:01.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:01.398 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-09T06:38:01.415 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi163.front.sepia.ceph.com 2023-10-09T06:38:01.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:01.415 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-09T06:38:01.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-09T06:38:01.430 INFO:teuthology.orchestra.run.smithi163.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-09T06:38:01.430 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-09T06:38:01.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:01.431 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-09T06:38:01.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-09T06:38:01.473 INFO:teuthology.orchestra.run.smithi181.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-09T06:38:01.474 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rename cephfs volume_0000000000063288 --yes-i-really-mean-it 2023-10-09T06:38:01.593 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.592+0000 7ff84bfb3700 1 Processor -- start 2023-10-09T06:38:01.593 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.592+0000 7ff84bfb3700 1 -- start start 2023-10-09T06:38:01.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.593+0000 7ff84bfb3700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff844159ec0 0x7ff8440b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:01.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.593+0000 7ff84bfb3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8440b53d0 0x7ff8440cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:01.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.593+0000 7ff84bfb3700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8440b5b00 con 0x7ff8440cce40 2023-10-09T06:38:01.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.593+0000 7ff84bfb3700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8440b5850 con 0x7ff844159ec0 2023-10-09T06:38:01.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.593+0000 7ff84bfb3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8440b5da0 con 0x7ff8440b53d0 2023-10-09T06:38:01.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.593+0000 7ff849d4f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff844159ec0 0x7ff8440b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:01.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.593+0000 7ff84a550700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff8440cce40 0x7ff8440cd2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:46950/0 (socket says 172.21.15.163:46950) 2023-10-09T06:38:01.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.593+0000 7ff84a550700 1 -- 172.21.15.163:0/1116516595 learned_addr learned my addr 172.21.15.163:0/1116516595 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:01.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.594+0000 7ff84954e700 1 --2- 172.21.15.163:0/1116516595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8440b53d0 0x7ff8440cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:01.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.594+0000 7ff848d4d700 1 -- 172.21.15.163:0/1116516595 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3154744289 0 0) 0x7ff8440b5b00 con 0x7ff8440cce40 2023-10-09T06:38:01.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.594+0000 7ff848d4d700 1 -- 172.21.15.163:0/1116516595 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff830003570 con 0x7ff8440cce40 2023-10-09T06:38:01.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.594+0000 7ff849d4f700 1 -- 172.21.15.163:0/1116516595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8440b53d0 msgr2=0x7ff8440cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:01.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.594+0000 7ff849d4f700 1 --2- 172.21.15.163:0/1116516595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8440b53d0 0x7ff8440cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:01.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.594+0000 7ff849d4f700 1 -- 172.21.15.163:0/1116516595 >> v1:172.21.15.163:6789/0 conn(0x7ff8440cce40 legacy=0x7ff8440cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:01.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.594+0000 7ff849d4f700 1 -- 172.21.15.163:0/1116516595 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8440b6080 con 0x7ff844159ec0 2023-10-09T06:38:01.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.594+0000 7ff849d4f700 1 --2- 172.21.15.163:0/1116516595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff844159ec0 0x7ff8440b4e90 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7ff8400098d0 tx=0x7ff840009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=496e5763309e3d89 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:01.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.594+0000 7ff848d4d700 1 -- 172.21.15.163:0/1116516595 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff840005c10 con 0x7ff844159ec0 2023-10-09T06:38:01.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.594+0000 7ff848d4d700 1 -- 172.21.15.163:0/1116516595 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff840005d70 con 0x7ff844159ec0 2023-10-09T06:38:01.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.594+0000 7ff848d4d700 1 -- 172.21.15.163:0/1116516595 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff840006260 con 0x7ff844159ec0 2023-10-09T06:38:01.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.595+0000 7ff84bfb3700 1 -- 172.21.15.163:0/1116516595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff844159ec0 msgr2=0x7ff8440b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:01.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.595+0000 7ff84bfb3700 1 --2- 172.21.15.163:0/1116516595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff844159ec0 0x7ff8440b4e90 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7ff8400098d0 tx=0x7ff840009120 comp rx=0 tx=0).stop 2023-10-09T06:38:01.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.595+0000 7ff84bfb3700 1 -- 172.21.15.163:0/1116516595 shutdown_connections 2023-10-09T06:38:01.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.595+0000 7ff84bfb3700 1 --2- 172.21.15.163:0/1116516595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff844159ec0 0x7ff8440b4e90 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:01.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.595+0000 7ff84bfb3700 1 --2- 172.21.15.163:0/1116516595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8440b53d0 0x7ff8440cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:01.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.595+0000 7ff84bfb3700 1 -- 172.21.15.163:0/1116516595 >> 172.21.15.163:0/1116516595 conn(0x7ff8440999f0 msgr2=0x7ff8440b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:01.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.595+0000 7ff84bfb3700 1 -- 172.21.15.163:0/1116516595 shutdown_connections 2023-10-09T06:38:01.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.595+0000 7ff84bfb3700 1 -- 172.21.15.163:0/1116516595 wait complete. 2023-10-09T06:38:01.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.596+0000 7ff84bfb3700 1 Processor -- start 2023-10-09T06:38:01.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.596+0000 7ff84bfb3700 1 -- start start 2023-10-09T06:38:01.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.596+0000 7ff84bfb3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8440b53d0 0x7ff8441574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:01.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.597+0000 7ff84bfb3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8440cce40 0x7ff84409c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:01.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.597+0000 7ff84bfb3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff844159ec0 0x7ff84409cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:01.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.597+0000 7ff84bfb3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8440ad2f0 con 0x7ff8440b53d0 2023-10-09T06:38:01.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.597+0000 7ff84bfb3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84409ba30 con 0x7ff844159ec0 2023-10-09T06:38:01.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.597+0000 7ff84bfb3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff84409b190 con 0x7ff8440cce40 2023-10-09T06:38:01.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.597+0000 7ff849d4f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8440b53d0 0x7ff8441574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:01.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.597+0000 7ff849d4f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8440b53d0 0x7ff8441574a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42186/0 (socket says 172.21.15.163:42186) 2023-10-09T06:38:01.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.597+0000 7ff849d4f700 1 -- 172.21.15.163:0/2077571231 learned_addr learned my addr 172.21.15.163:0/2077571231 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:01.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.597+0000 7ff84954e700 1 --2- 172.21.15.163:0/2077571231 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8440cce40 0x7ff84409c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:01.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.597+0000 7ff84a550700 1 --2- 172.21.15.163:0/2077571231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff844159ec0 0x7ff84409cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:01.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.597+0000 7ff849d4f700 1 -- 172.21.15.163:0/2077571231 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8440cce40 msgr2=0x7ff84409c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:01.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.597+0000 7ff849d4f700 1 --2- 172.21.15.163:0/2077571231 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8440cce40 0x7ff84409c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:01.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.597+0000 7ff849d4f700 1 -- 172.21.15.163:0/2077571231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff844159ec0 msgr2=0x7ff84409cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:01.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.597+0000 7ff849d4f700 1 --2- 172.21.15.163:0/2077571231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff844159ec0 0x7ff84409cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:01.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.597+0000 7ff849d4f700 1 -- 172.21.15.163:0/2077571231 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8400093c0 con 0x7ff8440b53d0 2023-10-09T06:38:01.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.597+0000 7ff84a550700 1 --2- 172.21.15.163:0/2077571231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff844159ec0 0x7ff84409cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:01.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.597+0000 7ff849d4f700 1 --2- 172.21.15.163:0/2077571231 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8440b53d0 0x7ff8441574a0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7ff840009ec0 tx=0x7ff840016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:01.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.597+0000 7ff839ffb700 1 -- 172.21.15.163:0/2077571231 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84001d910 con 0x7ff8440b53d0 2023-10-09T06:38:01.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.597+0000 7ff839ffb700 1 -- 172.21.15.163:0/2077571231 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff84001da90 con 0x7ff8440b53d0 2023-10-09T06:38:01.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.597+0000 7ff839ffb700 1 -- 172.21.15.163:0/2077571231 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84001de50 con 0x7ff8440b53d0 2023-10-09T06:38:01.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.597+0000 7ff84bfb3700 1 -- 172.21.15.163:0/2077571231 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff84409b470 con 0x7ff8440b53d0 2023-10-09T06:38:01.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.597+0000 7ff84bfb3700 1 -- 172.21.15.163:0/2077571231 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff844069490 con 0x7ff8440b53d0 2023-10-09T06:38:01.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.598+0000 7ff84bfb3700 1 -- 172.21.15.163:0/2077571231 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff844063c90 con 0x7ff8440b53d0 2023-10-09T06:38:01.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.599+0000 7ff839ffb700 1 -- 172.21.15.163:0/2077571231 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff84001b690 con 0x7ff8440b53d0 2023-10-09T06:38:01.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.599+0000 7ff839ffb700 1 --2- 172.21.15.163:0/2077571231 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff828067320 0x7ff8280697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:01.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.599+0000 7ff839ffb700 1 -- 172.21.15.163:0/2077571231 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff840086920 con 0x7ff8440b53d0 2023-10-09T06:38:01.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.599+0000 7ff84954e700 1 --2- 172.21.15.163:0/2077571231 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff828067320 0x7ff8280697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:01.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.600+0000 7ff84954e700 1 --2- 172.21.15.163:0/2077571231 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff828067320 0x7ff8280697d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff8441579e0 tx=0x7ff8340091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:01.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.603+0000 7ff839ffb700 1 -- 172.21.15.163:0/2077571231 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff8400535b0 con 0x7ff8440b53d0 2023-10-09T06:38:01.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:01.812+0000 7ff84bfb3700 1 -- 172.21.15.163:0/2077571231 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume rename", "vol_name": "cephfs", "new_vol_name": "volume_0000000000063288", "yes_i_really_mean_it": true, "target": ["mon-mgr", ""]}) v1 -- 0x7ff8440718b0 con 0x7ff828067320 2023-10-09T06:38:04.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.585+0000 7ff839ffb700 1 -- 172.21.15.163:0/2077571231 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+56 (secure 0 0 0) 0x7ff8440718b0 con 0x7ff828067320 2023-10-09T06:38:04.586 INFO:teuthology.orchestra.run.smithi163.stdout:FS volume 'cephfs' renamed to 'volume_0000000000063288' 2023-10-09T06:38:04.588 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.587+0000 7ff84bfb3700 1 -- 172.21.15.163:0/2077571231 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff828067320 msgr2=0x7ff8280697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:04.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.587+0000 7ff84bfb3700 1 --2- 172.21.15.163:0/2077571231 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff828067320 0x7ff8280697d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff8441579e0 tx=0x7ff8340091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:04.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.587+0000 7ff84bfb3700 1 -- 172.21.15.163:0/2077571231 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8440b53d0 msgr2=0x7ff8441574a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:04.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.587+0000 7ff84bfb3700 1 --2- 172.21.15.163:0/2077571231 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8440b53d0 0x7ff8441574a0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7ff840009ec0 tx=0x7ff840016000 comp rx=0 tx=0).stop 2023-10-09T06:38:04.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.588+0000 7ff84bfb3700 1 -- 172.21.15.163:0/2077571231 shutdown_connections 2023-10-09T06:38:04.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.588+0000 7ff84bfb3700 1 --2- 172.21.15.163:0/2077571231 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff828067320 0x7ff8280697d0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:04.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.588+0000 7ff84bfb3700 1 --2- 172.21.15.163:0/2077571231 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8440b53d0 0x7ff8441574a0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:04.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.588+0000 7ff84bfb3700 1 --2- 172.21.15.163:0/2077571231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff844159ec0 0x7ff84409cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:04.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.588+0000 7ff84bfb3700 1 --2- 172.21.15.163:0/2077571231 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8440cce40 0x7ff84409c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:04.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.588+0000 7ff84bfb3700 1 -- 172.21.15.163:0/2077571231 >> 172.21.15.163:0/2077571231 conn(0x7ff8440999f0 msgr2=0x7ff844159140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:04.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.588+0000 7ff84bfb3700 1 -- 172.21.15.163:0/2077571231 shutdown_connections 2023-10-09T06:38:04.589 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.588+0000 7ff84bfb3700 1 -- 172.21.15.163:0/2077571231 wait complete. 2023-10-09T06:38:04.603 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-09T06:38:04.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.724+0000 7ffa4f708700 1 Processor -- start 2023-10-09T06:38:04.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.724+0000 7ffa4f708700 1 -- start start 2023-10-09T06:38:04.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.725+0000 7ffa4f708700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ffa48159c60 0x7ffa4815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:04.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.725+0000 7ffa4f708700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffa4815db10 0x7ffa48069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:04.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.725+0000 7ffa4f708700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa4815a160 con 0x7ffa48159890 2023-10-09T06:38:04.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.725+0000 7ffa4f708700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa4815df50 con 0x7ffa48159c60 2023-10-09T06:38:04.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.725+0000 7ffa4f708700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa480d13a0 con 0x7ffa4815db10 2023-10-09T06:38:04.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.725+0000 7ffa4cca3700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ffa48159c60 0x7ffa4815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:04.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.725+0000 7ffa4cca3700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ffa48159c60 0x7ffa4815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42188/0 (socket says 172.21.15.163:42188) 2023-10-09T06:38:04.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.725+0000 7ffa4cca3700 1 -- 172.21.15.163:0/1851018735 learned_addr learned my addr 172.21.15.163:0/1851018735 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:04.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.725+0000 7ffa4dca5700 1 --2- 172.21.15.163:0/1851018735 >> v2:172.21.15.181:3300/0 conn(0x7ffa4815db10 0x7ffa48069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:04.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.725+0000 7ffa4cca3700 1 -- 172.21.15.163:0/1851018735 >> v2:172.21.15.181:3300/0 conn(0x7ffa4815db10 msgr2=0x7ffa48069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:04.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.725+0000 7ffa4cca3700 1 --2- 172.21.15.163:0/1851018735 >> v2:172.21.15.181:3300/0 conn(0x7ffa4815db10 0x7ffa48069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:04.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.725+0000 7ffa4cca3700 1 -- 172.21.15.163:0/1851018735 >> v1:172.21.15.181:6789/0 conn(0x7ffa48159890 legacy=0x7ffa4815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:04.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.725+0000 7ffa4cca3700 1 -- 172.21.15.163:0/1851018735 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa4815e140 con 0x7ffa48159c60 2023-10-09T06:38:04.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.726+0000 7ffa4cca3700 1 --2- 172.21.15.163:0/1851018735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa48159c60 0x7ffa4815d5d0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7ffa4806a890 tx=0x7ffa38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82af2da16afee521 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:04.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.726+0000 7ffa3ffff700 1 -- 172.21.15.163:0/1851018735 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3800b860 con 0x7ffa48159c60 2023-10-09T06:38:04.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.726+0000 7ffa3ffff700 1 -- 172.21.15.163:0/1851018735 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3800b9c0 con 0x7ffa48159c60 2023-10-09T06:38:04.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.726+0000 7ffa3ffff700 1 -- 172.21.15.163:0/1851018735 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3800bd60 con 0x7ffa48159c60 2023-10-09T06:38:04.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.726+0000 7ffa4f708700 1 -- 172.21.15.163:0/1851018735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa48159c60 msgr2=0x7ffa4815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:04.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.726+0000 7ffa4f708700 1 --2- 172.21.15.163:0/1851018735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa48159c60 0x7ffa4815d5d0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7ffa4806a890 tx=0x7ffa38009250 comp rx=0 tx=0).stop 2023-10-09T06:38:04.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.726+0000 7ffa4f708700 1 -- 172.21.15.163:0/1851018735 shutdown_connections 2023-10-09T06:38:04.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.726+0000 7ffa4f708700 1 --2- 172.21.15.163:0/1851018735 >> v2:172.21.15.181:3300/0 conn(0x7ffa4815db10 0x7ffa48069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:04.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.726+0000 7ffa4f708700 1 --2- 172.21.15.163:0/1851018735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa48159c60 0x7ffa4815d5d0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:04.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.726+0000 7ffa4f708700 1 -- 172.21.15.163:0/1851018735 >> 172.21.15.163:0/1851018735 conn(0x7ffa48068e20 msgr2=0x7ffa480d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:04.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.726+0000 7ffa4f708700 1 -- 172.21.15.163:0/1851018735 shutdown_connections 2023-10-09T06:38:04.729 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.726+0000 7ffa4f708700 1 -- 172.21.15.163:0/1851018735 wait complete. 2023-10-09T06:38:04.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.727+0000 7ffa4f708700 1 Processor -- start 2023-10-09T06:38:04.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.727+0000 7ffa4f708700 1 -- start start 2023-10-09T06:38:04.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.728+0000 7ffa4f708700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa48159890 0x7ffa480c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:04.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.728+0000 7ffa4f708700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa48159c60 0x7ffa48140b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:04.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.728+0000 7ffa4f708700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa4815db10 0x7ffa480cf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:04.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.728+0000 7ffa4f708700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa480ca970 con 0x7ffa48159c60 2023-10-09T06:38:04.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.728+0000 7ffa4f708700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa480c9df0 con 0x7ffa4815db10 2023-10-09T06:38:04.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.728+0000 7ffa4f708700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa480ca0d0 con 0x7ffa48159890 2023-10-09T06:38:04.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.728+0000 7ffa4cca3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa48159c60 0x7ffa48140b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:04.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.728+0000 7ffa4cca3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa48159c60 0x7ffa48140b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42200/0 (socket says 172.21.15.163:42200) 2023-10-09T06:38:04.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.728+0000 7ffa4cca3700 1 -- 172.21.15.163:0/3819479353 learned_addr learned my addr 172.21.15.163:0/3819479353 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:04.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.729+0000 7ffa4dca5700 1 --2- 172.21.15.163:0/3819479353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa4815db10 0x7ffa480cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:04.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.729+0000 7ffa4d4a4700 1 --2- 172.21.15.163:0/3819479353 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa48159890 0x7ffa480c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:04.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.729+0000 7ffa4cca3700 1 -- 172.21.15.163:0/3819479353 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa48159890 msgr2=0x7ffa480c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:04.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.729+0000 7ffa4cca3700 1 --2- 172.21.15.163:0/3819479353 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa48159890 0x7ffa480c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:04.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.729+0000 7ffa4cca3700 1 -- 172.21.15.163:0/3819479353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa4815db10 msgr2=0x7ffa480cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:04.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.729+0000 7ffa4cca3700 1 --2- 172.21.15.163:0/3819479353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa4815db10 0x7ffa480cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:04.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.729+0000 7ffa4cca3700 1 -- 172.21.15.163:0/3819479353 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa380094f0 con 0x7ffa48159c60 2023-10-09T06:38:04.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.729+0000 7ffa4cca3700 1 --2- 172.21.15.163:0/3819479353 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa48159c60 0x7ffa48140b90 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7ffa48070e20 tx=0x7ffa38017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:04.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.729+0000 7ffa3d7fa700 1 -- 172.21.15.163:0/3819479353 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa38024440 con 0x7ffa48159c60 2023-10-09T06:38:04.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.729+0000 7ffa3d7fa700 1 -- 172.21.15.163:0/3819479353 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa380245a0 con 0x7ffa48159c60 2023-10-09T06:38:04.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.729+0000 7ffa4f708700 1 -- 172.21.15.163:0/3819479353 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa480ca3b0 con 0x7ffa48159c60 2023-10-09T06:38:04.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.729+0000 7ffa3d7fa700 1 -- 172.21.15.163:0/3819479353 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa38024960 con 0x7ffa48159c60 2023-10-09T06:38:04.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.729+0000 7ffa4f708700 1 -- 172.21.15.163:0/3819479353 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa480c9a50 con 0x7ffa48159c60 2023-10-09T06:38:04.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.730+0000 7ffa4f708700 1 -- 172.21.15.163:0/3819479353 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4804c3c0 con 0x7ffa48159c60 2023-10-09T06:38:04.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.730+0000 7ffa3d7fa700 1 -- 172.21.15.163:0/3819479353 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ffa3801a420 con 0x7ffa48159c60 2023-10-09T06:38:04.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.731+0000 7ffa3d7fa700 1 --2- 172.21.15.163:0/3819479353 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffa2c067320 0x7ffa2c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:04.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.731+0000 7ffa3d7fa700 1 -- 172.21.15.163:0/3819479353 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5783+0+0 (secure 0 0 0) 0x7ffa38086c30 con 0x7ffa48159c60 2023-10-09T06:38:04.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.731+0000 7ffa4d4a4700 1 --2- 172.21.15.163:0/3819479353 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffa2c067320 0x7ffa2c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:04.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.732+0000 7ffa4d4a4700 1 --2- 172.21.15.163:0/3819479353 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffa2c067320 0x7ffa2c0697d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ffa44007fd0 tx=0x7ffa44009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:04.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.735+0000 7ffa3d7fa700 1 -- 172.21.15.163:0/3819479353 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa38053870 con 0x7ffa48159c60 2023-10-09T06:38:04.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.946+0000 7ffa4f708700 1 -- 172.21.15.163:0/3819479353 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa480b2c50 con 0x7ffa2c067320 2023-10-09T06:38:04.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.947+0000 7ffa3d7fa700 1 -- 172.21.15.163:0/3819479353 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+58 (secure 0 0 0) 0x7ffa480b2c50 con 0x7ffa2c067320 2023-10-09T06:38:04.948 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:38:04.948 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:04.948 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "volume_0000000000063288" 2023-10-09T06:38:04.949 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:04.949 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:38:04.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.949+0000 7ffa4f708700 1 -- 172.21.15.163:0/3819479353 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffa2c067320 msgr2=0x7ffa2c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:04.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.949+0000 7ffa4f708700 1 --2- 172.21.15.163:0/3819479353 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffa2c067320 0x7ffa2c0697d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ffa44007fd0 tx=0x7ffa44009040 comp rx=0 tx=0).stop 2023-10-09T06:38:04.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.949+0000 7ffa4f708700 1 -- 172.21.15.163:0/3819479353 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa48159c60 msgr2=0x7ffa48140b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:04.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.949+0000 7ffa4f708700 1 --2- 172.21.15.163:0/3819479353 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa48159c60 0x7ffa48140b90 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7ffa48070e20 tx=0x7ffa38017000 comp rx=0 tx=0).stop 2023-10-09T06:38:04.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.950+0000 7ffa4f708700 1 -- 172.21.15.163:0/3819479353 shutdown_connections 2023-10-09T06:38:04.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.950+0000 7ffa4f708700 1 --2- 172.21.15.163:0/3819479353 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffa2c067320 0x7ffa2c0697d0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:04.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.950+0000 7ffa4f708700 1 --2- 172.21.15.163:0/3819479353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa4815db10 0x7ffa480cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:04.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.950+0000 7ffa4f708700 1 --2- 172.21.15.163:0/3819479353 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa48159890 0x7ffa480c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:04.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.950+0000 7ffa4f708700 1 --2- 172.21.15.163:0/3819479353 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa48159c60 0x7ffa48140b90 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:04.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.950+0000 7ffa4f708700 1 -- 172.21.15.163:0/3819479353 >> 172.21.15.163:0/3819479353 conn(0x7ffa48068e20 msgr2=0x7ffa481530f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:04.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.950+0000 7ffa4f708700 1 -- 172.21.15.163:0/3819479353 shutdown_connections 2023-10-09T06:38:04.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:04.950+0000 7ffa4f708700 1 -- 172.21.15.163:0/3819479353 wait complete. 2023-10-09T06:38:04.963 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:38:05.078 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.077+0000 7f8186cd8700 1 Processor -- start 2023-10-09T06:38:05.078 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.077+0000 7f8186cd8700 1 -- start start 2023-10-09T06:38:05.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.078+0000 7f8186cd8700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f818015d730 0x7f818015dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:05.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.078+0000 7f8186cd8700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81800c74b0 con 0x7f8180159b10 2023-10-09T06:38:05.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.078+0000 7f8186cd8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f818015e0e0 con 0x7f818006a410 2023-10-09T06:38:05.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.078+0000 7f8186cd8700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81800c8700 con 0x7f818015d730 2023-10-09T06:38:05.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.078+0000 7f817ffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f818015d730 0x7f818015dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:05.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.078+0000 7f8184a74700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f8180159b10 0x7f81800cb3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:46956/0 (socket says 172.21.15.163:46956) 2023-10-09T06:38:05.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.078+0000 7f8184a74700 1 -- 172.21.15.163:0/4162545875 learned_addr learned my addr 172.21.15.163:0/4162545875 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:05.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.078+0000 7f817ffff700 1 -- 172.21.15.163:0/4162545875 >> v1:172.21.15.181:6789/0 conn(0x7f818006a410 legacy=0x7f818006a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:05.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.078+0000 7f817ffff700 1 -- 172.21.15.163:0/4162545875 >> v1:172.21.15.163:6789/0 conn(0x7f8180159b10 legacy=0x7f81800cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:05.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.078+0000 7f817ffff700 1 -- 172.21.15.163:0/4162545875 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81800c9070 con 0x7f818015d730 2023-10-09T06:38:05.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.078+0000 7f817f7fe700 1 -- 172.21.15.163:0/4162545875 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 551232756 0 0) 0x7f81800c74b0 con 0x7f8180159b10 2023-10-09T06:38:05.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.078+0000 7f817ffff700 1 --2- 172.21.15.163:0/4162545875 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f818015d730 0x7f818015dba0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f81740098e0 tx=0x7f81740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee84e79af4f2c90 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:05.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.079+0000 7f817f7fe700 1 -- 172.21.15.163:0/4162545875 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8174009e80 con 0x7f818015d730 2023-10-09T06:38:05.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.079+0000 7f817f7fe700 1 -- 172.21.15.163:0/4162545875 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817400b8b0 con 0x7f818015d730 2023-10-09T06:38:05.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.079+0000 7f817f7fe700 1 -- 172.21.15.163:0/4162545875 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817400bc30 con 0x7f818015d730 2023-10-09T06:38:05.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.079+0000 7f8186cd8700 1 -- 172.21.15.163:0/4162545875 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f818015d730 msgr2=0x7f818015dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:05.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.079+0000 7f8186cd8700 1 --2- 172.21.15.163:0/4162545875 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f818015d730 0x7f818015dba0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f81740098e0 tx=0x7f81740091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:05.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.079+0000 7f8186cd8700 1 -- 172.21.15.163:0/4162545875 shutdown_connections 2023-10-09T06:38:05.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.079+0000 7f8186cd8700 1 --2- 172.21.15.163:0/4162545875 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f818015d730 0x7f818015dba0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:05.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.079+0000 7f8186cd8700 1 -- 172.21.15.163:0/4162545875 >> 172.21.15.163:0/4162545875 conn(0x7f81800c9870 msgr2=0x7f81800c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:05.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.079+0000 7f8186cd8700 1 -- 172.21.15.163:0/4162545875 shutdown_connections 2023-10-09T06:38:05.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.079+0000 7f8186cd8700 1 -- 172.21.15.163:0/4162545875 wait complete. 2023-10-09T06:38:05.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.080+0000 7f8186cd8700 1 Processor -- start 2023-10-09T06:38:05.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.080+0000 7f8186cd8700 1 -- start start 2023-10-09T06:38:05.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.081+0000 7f8186cd8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f818006a410 0x7f81800c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:05.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.081+0000 7f8186cd8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8180159b10 0x7f81800c4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:05.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.081+0000 7f8186cd8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f818015d730 0x7f81800c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:05.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.081+0000 7f8186cd8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818016b5d0 con 0x7f818006a410 2023-10-09T06:38:05.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.081+0000 7f8186cd8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818016aa50 con 0x7f818015d730 2023-10-09T06:38:05.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.081+0000 7f8186cd8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818016ad30 con 0x7f8180159b10 2023-10-09T06:38:05.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.081+0000 7f8184a74700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f818006a410 0x7f81800c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:05.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.081+0000 7f8184a74700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f818006a410 0x7f81800c6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42220/0 (socket says 172.21.15.163:42220) 2023-10-09T06:38:05.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.081+0000 7f8184a74700 1 -- 172.21.15.163:0/4265200884 learned_addr learned my addr 172.21.15.163:0/4265200884 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:05.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.081+0000 7f817ffff700 1 --2- 172.21.15.163:0/4265200884 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8180159b10 0x7f81800c4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:05.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.081+0000 7f8185275700 1 --2- 172.21.15.163:0/4265200884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f818015d730 0x7f81800c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:05.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.081+0000 7f8184a74700 1 -- 172.21.15.163:0/4265200884 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8180159b10 msgr2=0x7f81800c4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:05.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.081+0000 7f8184a74700 1 --2- 172.21.15.163:0/4265200884 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8180159b10 0x7f81800c4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:05.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.081+0000 7f8184a74700 1 -- 172.21.15.163:0/4265200884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f818015d730 msgr2=0x7f81800c23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:05.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.081+0000 7f8184a74700 1 --2- 172.21.15.163:0/4265200884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f818015d730 0x7f81800c23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:05.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.082+0000 7f8184a74700 1 -- 172.21.15.163:0/4265200884 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8174009420 con 0x7f818006a410 2023-10-09T06:38:05.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.082+0000 7f817ffff700 1 --2- 172.21.15.163:0/4265200884 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8180159b10 0x7f81800c4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:05.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.082+0000 7f8185275700 1 --2- 172.21.15.163:0/4265200884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f818015d730 0x7f81800c23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:05.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.082+0000 7f8184a74700 1 --2- 172.21.15.163:0/4265200884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f818006a410 0x7f81800c6c30 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f817800cd80 tx=0x7f8178001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:05.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.082+0000 7f817cff9700 1 -- 172.21.15.163:0/4265200884 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817800ab10 con 0x7f818006a410 2023-10-09T06:38:05.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.082+0000 7f817cff9700 1 -- 172.21.15.163:0/4265200884 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817800ac70 con 0x7f818006a410 2023-10-09T06:38:05.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.082+0000 7f8186cd8700 1 -- 172.21.15.163:0/4265200884 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f818016b010 con 0x7f818006a410 2023-10-09T06:38:05.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.082+0000 7f817cff9700 1 -- 172.21.15.163:0/4265200884 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81780070f0 con 0x7f818006a410 2023-10-09T06:38:05.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.082+0000 7f8186cd8700 1 -- 172.21.15.163:0/4265200884 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81800c7170 con 0x7f818006a410 2023-10-09T06:38:05.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.083+0000 7f8186cd8700 1 -- 172.21.15.163:0/4265200884 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81800afc30 con 0x7f818006a410 2023-10-09T06:38:05.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.083+0000 7f817cff9700 1 -- 172.21.15.163:0/4265200884 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f81780074e0 con 0x7f818006a410 2023-10-09T06:38:05.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.084+0000 7f817cff9700 1 --2- 172.21.15.163:0/4265200884 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f816c067320 0x7f816c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:05.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.084+0000 7f817cff9700 1 -- 172.21.15.163:0/4265200884 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f8178085230 con 0x7f818006a410 2023-10-09T06:38:05.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.084+0000 7f817ffff700 1 --2- 172.21.15.163:0/4265200884 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f816c067320 0x7f816c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:05.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.084+0000 7f817ffff700 1 --2- 172.21.15.163:0/4265200884 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f816c067320 0x7f816c0697d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f81740030c0 tx=0x7f8174016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:05.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.088+0000 7f817cff9700 1 -- 172.21.15.163:0/4265200884 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8178051ef0 con 0x7f818006a410 2023-10-09T06:38:05.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.308+0000 7f8186cd8700 1 -- 172.21.15.163:0/4265200884 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f818004c3c0 con 0x7f818006a410 2023-10-09T06:38:05.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.309+0000 7f817cff9700 1 -- 172.21.15.163:0/4265200884 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4630 (secure 0 0 0) 0x7f8178055510 con 0x7f818006a410 2023-10-09T06:38:05.310 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:05.310 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":95,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1554520486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1554520486},{"type":"v1","addr":"172.21.15.181:6837","nonce":1554520486}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1695339871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1695339871},{"type":"v1","addr":"172.21.15.181:6836","nonce":1695339871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":7515,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3668692283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3668692283},{"type":"v1","addr":"172.21.15.163:6834","nonce":3668692283}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:37:46.529939+0000","modified":"2023-10-09T06:38:02.561321+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7518":{"gid":7518,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/3043729326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3043729326},{"type":"v1","addr":"172.21.15.163:6835","nonce":3043729326}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"volume_0000000000063288","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-09T06:38:05.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.310+0000 7f8186cd8700 1 -- 172.21.15.163:0/4265200884 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f816c067320 msgr2=0x7f816c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:05.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.310+0000 7f8186cd8700 1 --2- 172.21.15.163:0/4265200884 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f816c067320 0x7f816c0697d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f81740030c0 tx=0x7f8174016000 comp rx=0 tx=0).stop 2023-10-09T06:38:05.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.310+0000 7f8186cd8700 1 -- 172.21.15.163:0/4265200884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f818006a410 msgr2=0x7f81800c6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:05.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.310+0000 7f8186cd8700 1 --2- 172.21.15.163:0/4265200884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f818006a410 0x7f81800c6c30 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f817800cd80 tx=0x7f8178001ae0 comp rx=0 tx=0).stop 2023-10-09T06:38:05.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.310+0000 7f8186cd8700 1 -- 172.21.15.163:0/4265200884 shutdown_connections 2023-10-09T06:38:05.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.310+0000 7f8186cd8700 1 --2- 172.21.15.163:0/4265200884 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f816c067320 0x7f816c0697d0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:05.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.310+0000 7f8186cd8700 1 --2- 172.21.15.163:0/4265200884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f818006a410 0x7f81800c6c30 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:05.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.310+0000 7f8186cd8700 1 --2- 172.21.15.163:0/4265200884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f818015d730 0x7f81800c23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:05.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.310+0000 7f8186cd8700 1 --2- 172.21.15.163:0/4265200884 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8180159b10 0x7f81800c4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:05.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.311+0000 7f8186cd8700 1 -- 172.21.15.163:0/4265200884 >> 172.21.15.163:0/4265200884 conn(0x7f81800c9870 msgr2=0x7f81801556c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:05.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.311+0000 7f8186cd8700 1 -- 172.21.15.163:0/4265200884 shutdown_connections 2023-10-09T06:38:05.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.311+0000 7f8186cd8700 1 -- 172.21.15.163:0/4265200884 wait complete. 2023-10-09T06:38:05.313 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 95 2023-10-09T06:38:05.324 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:38:05.438 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.437+0000 7f63ff833700 1 Processor -- start 2023-10-09T06:38:05.438 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.437+0000 7f63ff833700 1 -- start start 2023-10-09T06:38:05.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.438+0000 7f63ff833700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63f8141500 0x7f63f80c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:05.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.438+0000 7f63ff833700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f63f806c0a0 0x7f63f80c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:05.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.438+0000 7f63ff833700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63f815c2a0 con 0x7f63f806bcd0 2023-10-09T06:38:05.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.438+0000 7f63ff833700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63f80c9880 con 0x7f63f806c0a0 2023-10-09T06:38:05.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.438+0000 7f63ff833700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f815b690 con 0x7f63f8141500 2023-10-09T06:38:05.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.438+0000 7f63fcdce700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63f8141500 0x7f63f80c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:05.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.438+0000 7f63fcdce700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63f8141500 0x7f63f80c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:45750/0 (socket says 172.21.15.163:45750) 2023-10-09T06:38:05.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.438+0000 7f63fddd0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f63f806c0a0 0x7f63f80c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:05.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.438+0000 7f63fcdce700 1 -- 172.21.15.163:0/2972486636 learned_addr learned my addr 172.21.15.163:0/2972486636 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:05.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.439+0000 7f63fcdce700 1 -- 172.21.15.163:0/2972486636 >> v2:172.21.15.181:3300/0 conn(0x7f63f806c0a0 msgr2=0x7f63f80c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:05.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.439+0000 7f63fcdce700 1 --2- 172.21.15.163:0/2972486636 >> v2:172.21.15.181:3300/0 conn(0x7f63f806c0a0 0x7f63f80c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:05.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.439+0000 7f63fcdce700 1 -- 172.21.15.163:0/2972486636 >> v1:172.21.15.181:6789/0 conn(0x7f63f806bcd0 legacy=0x7f63f8140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:05.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.439+0000 7f63fcdce700 1 -- 172.21.15.163:0/2972486636 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f815b970 con 0x7f63f8141500 2023-10-09T06:38:05.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.439+0000 7f63fcdce700 1 --2- 172.21.15.163:0/2972486636 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63f8141500 0x7f63f80c8e00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f63e80099b0 tx=0x7f63e8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6ba10bee85e5304 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:05.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.439+0000 7f63effff700 1 -- 172.21.15.163:0/2972486636 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8009ea0 con 0x7f63f8141500 2023-10-09T06:38:05.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.439+0000 7f63effff700 1 -- 172.21.15.163:0/2972486636 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e800b860 con 0x7f63f8141500 2023-10-09T06:38:05.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.439+0000 7f63effff700 1 -- 172.21.15.163:0/2972486636 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e800bc00 con 0x7f63f8141500 2023-10-09T06:38:05.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.440+0000 7f63ff833700 1 -- 172.21.15.163:0/2972486636 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63f8141500 msgr2=0x7f63f80c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:05.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.440+0000 7f63ff833700 1 --2- 172.21.15.163:0/2972486636 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63f8141500 0x7f63f80c8e00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f63e80099b0 tx=0x7f63e8009250 comp rx=0 tx=0).stop 2023-10-09T06:38:05.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.440+0000 7f63ff833700 1 -- 172.21.15.163:0/2972486636 shutdown_connections 2023-10-09T06:38:05.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.440+0000 7f63ff833700 1 --2- 172.21.15.163:0/2972486636 >> v2:172.21.15.181:3300/0 conn(0x7f63f806c0a0 0x7f63f80c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:05.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.440+0000 7f63ff833700 1 --2- 172.21.15.163:0/2972486636 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63f8141500 0x7f63f80c8e00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:05.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.440+0000 7f63ff833700 1 -- 172.21.15.163:0/2972486636 >> 172.21.15.163:0/2972486636 conn(0x7f63f80ce600 msgr2=0x7f63f80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:05.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.440+0000 7f63ff833700 1 -- 172.21.15.163:0/2972486636 shutdown_connections 2023-10-09T06:38:05.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.440+0000 7f63ff833700 1 -- 172.21.15.163:0/2972486636 wait complete. 2023-10-09T06:38:05.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.441+0000 7f63ff833700 1 Processor -- start 2023-10-09T06:38:05.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.441+0000 7f63ff833700 1 -- start start 2023-10-09T06:38:05.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.442+0000 7f63ff833700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63f806bcd0 0x7f63f80b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:05.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.442+0000 7f63ff833700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63f806c0a0 0x7f63f815d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:05.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.442+0000 7f63ff833700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63f8141500 0x7f63f8155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:05.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.442+0000 7f63ff833700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f8158be0 con 0x7f63f8141500 2023-10-09T06:38:05.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.442+0000 7f63ff833700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f8157fd0 con 0x7f63f806c0a0 2023-10-09T06:38:05.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.442+0000 7f63ff833700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f81582b0 con 0x7f63f806bcd0 2023-10-09T06:38:05.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.442+0000 7f63fddd0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63f8141500 0x7f63f8155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:05.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.442+0000 7f63fddd0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63f8141500 0x7f63f8155f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42236/0 (socket says 172.21.15.163:42236) 2023-10-09T06:38:05.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.442+0000 7f63fddd0700 1 -- 172.21.15.163:0/676035736 learned_addr learned my addr 172.21.15.163:0/676035736 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:05.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.442+0000 7f63fcdce700 1 --2- 172.21.15.163:0/676035736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63f806c0a0 0x7f63f815d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:05.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.442+0000 7f63fd5cf700 1 --2- 172.21.15.163:0/676035736 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63f806bcd0 0x7f63f80b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:05.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.442+0000 7f63fddd0700 1 -- 172.21.15.163:0/676035736 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63f806bcd0 msgr2=0x7f63f80b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:05.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.442+0000 7f63fddd0700 1 --2- 172.21.15.163:0/676035736 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63f806bcd0 0x7f63f80b1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:05.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.442+0000 7f63fddd0700 1 -- 172.21.15.163:0/676035736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63f806c0a0 msgr2=0x7f63f815d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:05.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.442+0000 7f63fddd0700 1 --2- 172.21.15.163:0/676035736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63f806c0a0 0x7f63f815d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:05.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.442+0000 7f63fddd0700 1 -- 172.21.15.163:0/676035736 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e80094f0 con 0x7f63f8141500 2023-10-09T06:38:05.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.442+0000 7f63fddd0700 1 --2- 172.21.15.163:0/676035736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63f8141500 0x7f63f8155f60 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f63f000ea50 tx=0x7f63f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:05.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.443+0000 7f63ed7fa700 1 -- 172.21.15.163:0/676035736 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f000ee90 con 0x7f63f8141500 2023-10-09T06:38:05.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.443+0000 7f63ed7fa700 1 -- 172.21.15.163:0/676035736 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f000c850 con 0x7f63f8141500 2023-10-09T06:38:05.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.443+0000 7f63ff833700 1 -- 172.21.15.163:0/676035736 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f8158590 con 0x7f63f8141500 2023-10-09T06:38:05.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.443+0000 7f63ed7fa700 1 -- 172.21.15.163:0/676035736 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f000cc10 con 0x7f63f8141500 2023-10-09T06:38:05.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.443+0000 7f63ff833700 1 -- 172.21.15.163:0/676035736 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f8157ba0 con 0x7f63f8141500 2023-10-09T06:38:05.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.444+0000 7f63ff833700 1 -- 172.21.15.163:0/676035736 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63f8158d50 con 0x7f63f8141500 2023-10-09T06:38:05.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.444+0000 7f63ed7fa700 1 -- 172.21.15.163:0/676035736 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f63f0013470 con 0x7f63f8141500 2023-10-09T06:38:05.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.444+0000 7f63ed7fa700 1 --2- 172.21.15.163:0/676035736 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f63dc067320 0x7f63dc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:05.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.444+0000 7f63ed7fa700 1 -- 172.21.15.163:0/676035736 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f63f0084ec0 con 0x7f63f8141500 2023-10-09T06:38:05.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.445+0000 7f63fd5cf700 1 --2- 172.21.15.163:0/676035736 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f63dc067320 0x7f63dc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:05.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.445+0000 7f63fd5cf700 1 --2- 172.21.15.163:0/676035736 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f63dc067320 0x7f63dc0697d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f63f4007fd0 tx=0x7f63f4000f40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:05.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.448+0000 7f63ed7fa700 1 -- 172.21.15.163:0/676035736 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f63f0051ac0 con 0x7f63f8141500 2023-10-09T06:38:05.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.636+0000 7f63ff833700 1 -- 172.21.15.163:0/676035736 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63f804c3c0 con 0x7f63f8141500 2023-10-09T06:38:05.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.636+0000 7f63ed7fa700 1 -- 172.21.15.163:0/676035736 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+15490 (secure 0 0 0) 0x7f63f00550e0 con 0x7f63f8141500 2023-10-09T06:38:05.637 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:05.638 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":140,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:38:04.576334+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs.volume_0000000000063288.meta","create_time":"2023-10-09T06:37:44.861841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000063288"}}},{"pool":25,"pool_name":"cephfs.volume_0000000000063288.data","create_time":"2023-10-09T06:37:45.496241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000063288"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:38:05.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.638+0000 7f63ff833700 1 -- 172.21.15.163:0/676035736 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f63dc067320 msgr2=0x7f63dc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:05.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.638+0000 7f63ff833700 1 --2- 172.21.15.163:0/676035736 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f63dc067320 0x7f63dc0697d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f63f4007fd0 tx=0x7f63f4000f40 comp rx=0 tx=0).stop 2023-10-09T06:38:05.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.638+0000 7f63ff833700 1 -- 172.21.15.163:0/676035736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63f8141500 msgr2=0x7f63f8155f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:05.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.638+0000 7f63ff833700 1 --2- 172.21.15.163:0/676035736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63f8141500 0x7f63f8155f60 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f63f000ea50 tx=0x7f63f00091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:05.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.639+0000 7f63ff833700 1 -- 172.21.15.163:0/676035736 shutdown_connections 2023-10-09T06:38:05.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.639+0000 7f63ff833700 1 --2- 172.21.15.163:0/676035736 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f63dc067320 0x7f63dc0697d0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:05.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.639+0000 7f63ff833700 1 --2- 172.21.15.163:0/676035736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63f8141500 0x7f63f8155f60 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:05.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.639+0000 7f63ff833700 1 --2- 172.21.15.163:0/676035736 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63f806bcd0 0x7f63f80b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:05.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.639+0000 7f63ff833700 1 --2- 172.21.15.163:0/676035736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63f806c0a0 0x7f63f815d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:05.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.639+0000 7f63ff833700 1 -- 172.21.15.163:0/676035736 >> 172.21.15.163:0/676035736 conn(0x7f63f80ce600 msgr2=0x7f63f806cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:05.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.639+0000 7f63ff833700 1 -- 172.21.15.163:0/676035736 shutdown_connections 2023-10-09T06:38:05.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.639+0000 7f63ff833700 1 -- 172.21.15.163:0/676035736 wait complete. 2023-10-09T06:38:05.652 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-09T06:38:05.652 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:38:05.692 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:38:05.737 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-09T06:38:05.737 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:38:05.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.851+0000 7f5bbea7d700 1 Processor -- start 2023-10-09T06:38:05.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.851+0000 7f5bbea7d700 1 -- start start 2023-10-09T06:38:05.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.852+0000 7f5bbea7d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5bb81599e0 0x7f5bb8069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:05.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.852+0000 7f5bbea7d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb806a2b0 0x7f5bb806a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:05.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.852+0000 7f5bbea7d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bb80c96a0 con 0x7f5bb8159db0 2023-10-09T06:38:05.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.852+0000 7f5bbea7d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bb80cb9e0 con 0x7f5bb81599e0 2023-10-09T06:38:05.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.852+0000 7f5bbea7d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb80cab60 con 0x7f5bb806a2b0 2023-10-09T06:38:05.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.852+0000 7f5bbc819700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5bb81599e0 0x7f5bb8069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:05.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.852+0000 7f5baffff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f5bb8159db0 0x7f5bb8069080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:46958/0 (socket says 172.21.15.163:46958) 2023-10-09T06:38:05.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.852+0000 7f5baffff700 1 -- 172.21.15.163:0/988611938 learned_addr learned my addr 172.21.15.163:0/988611938 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:05.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.853+0000 7f5bbc819700 1 -- 172.21.15.163:0/988611938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb806a2b0 msgr2=0x7f5bb806a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:05.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.853+0000 7f5bbd01a700 1 --2- 172.21.15.163:0/988611938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb806a2b0 0x7f5bb806a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:05.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.853+0000 7f5baf7fe700 1 -- 172.21.15.163:0/988611938 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2398606850 0 0) 0x7f5bb80c96a0 con 0x7f5bb8159db0 2023-10-09T06:38:05.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.853+0000 7f5bbc819700 1 --2- 172.21.15.163:0/988611938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb806a2b0 0x7f5bb806a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:05.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.853+0000 7f5bbc819700 1 -- 172.21.15.163:0/988611938 >> v1:172.21.15.163:6789/0 conn(0x7f5bb8159db0 legacy=0x7f5bb8069080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:05.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.853+0000 7f5bbc819700 1 -- 172.21.15.163:0/988611938 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb80cb420 con 0x7f5bb81599e0 2023-10-09T06:38:05.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.853+0000 7f5bbd01a700 1 --2- 172.21.15.163:0/988611938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb806a2b0 0x7f5bb806a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:05.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.853+0000 7f5bbc819700 1 --2- 172.21.15.163:0/988611938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bb81599e0 0x7f5bb8069ac0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4009e60 tx=0x7f5bb4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fce834a5ee4478 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:05.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.853+0000 7f5baf7fe700 1 -- 172.21.15.163:0/988611938 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb400e070 con 0x7f5bb81599e0 2023-10-09T06:38:05.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.853+0000 7f5baf7fe700 1 -- 172.21.15.163:0/988611938 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb400bc30 con 0x7f5bb81599e0 2023-10-09T06:38:05.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.853+0000 7f5baf7fe700 1 -- 172.21.15.163:0/988611938 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4012430 con 0x7f5bb81599e0 2023-10-09T06:38:05.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.854+0000 7f5bbea7d700 1 -- 172.21.15.163:0/988611938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bb81599e0 msgr2=0x7f5bb8069ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:05.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.854+0000 7f5bbea7d700 1 --2- 172.21.15.163:0/988611938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bb81599e0 0x7f5bb8069ac0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4009e60 tx=0x7f5bb4009780 comp rx=0 tx=0).stop 2023-10-09T06:38:05.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.854+0000 7f5bbea7d700 1 -- 172.21.15.163:0/988611938 shutdown_connections 2023-10-09T06:38:05.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.854+0000 7f5bbea7d700 1 --2- 172.21.15.163:0/988611938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb806a2b0 0x7f5bb806a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:05.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.854+0000 7f5bbea7d700 1 --2- 172.21.15.163:0/988611938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bb81599e0 0x7f5bb8069ac0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:05.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.854+0000 7f5bbea7d700 1 -- 172.21.15.163:0/988611938 >> 172.21.15.163:0/988611938 conn(0x7f5bb80c9860 msgr2=0x7f5bb80c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:05.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.854+0000 7f5bbea7d700 1 -- 172.21.15.163:0/988611938 shutdown_connections 2023-10-09T06:38:05.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.854+0000 7f5bbea7d700 1 -- 172.21.15.163:0/988611938 wait complete. 2023-10-09T06:38:05.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.855+0000 7f5bbea7d700 1 Processor -- start 2023-10-09T06:38:05.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.855+0000 7f5bbea7d700 1 -- start start 2023-10-09T06:38:05.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.856+0000 7f5bbea7d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bb806a2b0 0x7f5bb8144990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:05.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.856+0000 7f5bbea7d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb81599e0 0x7f5bb815a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:05.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.856+0000 7f5bbea7d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb8159db0 0x7f5bb815ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:05.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.856+0000 7f5bbea7d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb80c4d50 con 0x7f5bb806a2b0 2023-10-09T06:38:05.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.856+0000 7f5bbea7d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb80c3ea0 con 0x7f5bb81599e0 2023-10-09T06:38:05.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.856+0000 7f5bbea7d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb815a760 con 0x7f5bb8159db0 2023-10-09T06:38:05.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.856+0000 7f5bbc819700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bb806a2b0 0x7f5bb8144990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:05.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.856+0000 7f5bbc819700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bb806a2b0 0x7f5bb8144990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42266/0 (socket says 172.21.15.163:42266) 2023-10-09T06:38:05.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.856+0000 7f5bbc819700 1 -- 172.21.15.163:0/3043230284 learned_addr learned my addr 172.21.15.163:0/3043230284 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:05.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.856+0000 7f5bbd01a700 1 --2- 172.21.15.163:0/3043230284 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb8159db0 0x7f5bb815ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:05.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.856+0000 7f5baffff700 1 --2- 172.21.15.163:0/3043230284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb81599e0 0x7f5bb815a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:05.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.856+0000 7f5bbc819700 1 -- 172.21.15.163:0/3043230284 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb8159db0 msgr2=0x7f5bb815ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:05.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.856+0000 7f5bbc819700 1 --2- 172.21.15.163:0/3043230284 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb8159db0 0x7f5bb815ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:05.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.856+0000 7f5bbc819700 1 -- 172.21.15.163:0/3043230284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb81599e0 msgr2=0x7f5bb815a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:05.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.856+0000 7f5bbc819700 1 --2- 172.21.15.163:0/3043230284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb81599e0 0x7f5bb815a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:05.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.856+0000 7f5bbc819700 1 -- 172.21.15.163:0/3043230284 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb4009a20 con 0x7f5bb806a2b0 2023-10-09T06:38:05.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.856+0000 7f5baffff700 1 --2- 172.21.15.163:0/3043230284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb81599e0 0x7f5bb815a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:05.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.856+0000 7f5bbd01a700 1 --2- 172.21.15.163:0/3043230284 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb8159db0 0x7f5bb815ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:05.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.856+0000 7f5bbc819700 1 --2- 172.21.15.163:0/3043230284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bb806a2b0 0x7f5bb8144990 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80afa80 tx=0x7f5bb4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:05.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.856+0000 7f5bacff9700 1 -- 172.21.15.163:0/3043230284 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4012e50 con 0x7f5bb806a2b0 2023-10-09T06:38:05.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.856+0000 7f5bacff9700 1 -- 172.21.15.163:0/3043230284 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb401d630 con 0x7f5bb806a2b0 2023-10-09T06:38:05.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.856+0000 7f5bbea7d700 1 -- 172.21.15.163:0/3043230284 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bb80c4140 con 0x7f5bb806a2b0 2023-10-09T06:38:05.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.856+0000 7f5bacff9700 1 -- 172.21.15.163:0/3043230284 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb401d9f0 con 0x7f5bb806a2b0 2023-10-09T06:38:05.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.856+0000 7f5bbea7d700 1 -- 172.21.15.163:0/3043230284 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb80c4700 con 0x7f5bb806a2b0 2023-10-09T06:38:05.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.857+0000 7f5bbea7d700 1 -- 172.21.15.163:0/3043230284 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bb804c3c0 con 0x7f5bb806a2b0 2023-10-09T06:38:05.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.858+0000 7f5bacff9700 1 -- 172.21.15.163:0/3043230284 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5bb4024070 con 0x7f5bb806a2b0 2023-10-09T06:38:05.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.858+0000 7f5bacff9700 1 --2- 172.21.15.163:0/3043230284 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5ba4067320 0x7f5ba40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:05.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.858+0000 7f5bacff9700 1 -- 172.21.15.163:0/3043230284 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f5bb4087440 con 0x7f5bb806a2b0 2023-10-09T06:38:05.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.858+0000 7f5baffff700 1 --2- 172.21.15.163:0/3043230284 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5ba4067320 0x7f5ba40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:05.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.859+0000 7f5baffff700 1 --2- 172.21.15.163:0/3043230284 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5ba4067320 0x7f5ba40697d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5bb815ab10 tx=0x7f5ba8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:05.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:05.862+0000 7f5bacff9700 1 -- 172.21.15.163:0/3043230284 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5bb4054080 con 0x7f5bb806a2b0 2023-10-09T06:38:06.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.086+0000 7f5bbea7d700 1 -- 172.21.15.163:0/3043230284 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5bb80c7ae0 con 0x7f5bb806a2b0 2023-10-09T06:38:06.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.087+0000 7f5bacff9700 1 -- 172.21.15.163:0/3043230284 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4630 (secure 0 0 0) 0x7f5bb4057660 con 0x7f5bb806a2b0 2023-10-09T06:38:06.088 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:06.088 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":95,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1554520486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1554520486},{"type":"v1","addr":"172.21.15.181:6837","nonce":1554520486}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1695339871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1695339871},{"type":"v1","addr":"172.21.15.181:6836","nonce":1695339871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":7515,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3668692283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3668692283},{"type":"v1","addr":"172.21.15.163:6834","nonce":3668692283}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:37:46.529939+0000","modified":"2023-10-09T06:38:02.561321+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7518":{"gid":7518,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/3043729326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3043729326},{"type":"v1","addr":"172.21.15.163:6835","nonce":3043729326}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"volume_0000000000063288","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-09T06:38:06.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.089+0000 7f5bbea7d700 1 -- 172.21.15.163:0/3043230284 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5ba4067320 msgr2=0x7f5ba40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:06.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.089+0000 7f5bbea7d700 1 --2- 172.21.15.163:0/3043230284 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5ba4067320 0x7f5ba40697d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5bb815ab10 tx=0x7f5ba8009040 comp rx=0 tx=0).stop 2023-10-09T06:38:06.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.089+0000 7f5bbea7d700 1 -- 172.21.15.163:0/3043230284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bb806a2b0 msgr2=0x7f5bb8144990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:06.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.089+0000 7f5bbea7d700 1 --2- 172.21.15.163:0/3043230284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bb806a2b0 0x7f5bb8144990 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80afa80 tx=0x7f5bb4019000 comp rx=0 tx=0).stop 2023-10-09T06:38:06.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.089+0000 7f5bbea7d700 1 -- 172.21.15.163:0/3043230284 shutdown_connections 2023-10-09T06:38:06.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.089+0000 7f5bbea7d700 1 --2- 172.21.15.163:0/3043230284 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5ba4067320 0x7f5ba40697d0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:06.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.089+0000 7f5bbea7d700 1 --2- 172.21.15.163:0/3043230284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bb806a2b0 0x7f5bb8144990 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:06.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.089+0000 7f5bbea7d700 1 --2- 172.21.15.163:0/3043230284 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bb8159db0 0x7f5bb815ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:06.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.089+0000 7f5bbea7d700 1 --2- 172.21.15.163:0/3043230284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb81599e0 0x7f5bb815a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:06.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.089+0000 7f5bbea7d700 1 -- 172.21.15.163:0/3043230284 >> 172.21.15.163:0/3043230284 conn(0x7f5bb80c9860 msgr2=0x7f5bb8068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:06.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.089+0000 7f5bbea7d700 1 -- 172.21.15.163:0/3043230284 shutdown_connections 2023-10-09T06:38:06.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.089+0000 7f5bbea7d700 1 -- 172.21.15.163:0/3043230284 wait complete. 2023-10-09T06:38:06.092 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 95 2023-10-09T06:38:06.102 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:38:06.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.216+0000 7f5a094c5700 1 Processor -- start 2023-10-09T06:38:06.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.216+0000 7f5a094c5700 1 -- start start 2023-10-09T06:38:06.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.217+0000 7f5a094c5700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5a04157940 0x7f5a0406f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:06.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.217+0000 7f5a094c5700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a0406c750 con 0x7f5a0406c1e0 2023-10-09T06:38:06.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.217+0000 7f5a094c5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a0406ca30 con 0x7f5a0415baa0 2023-10-09T06:38:06.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.217+0000 7f5a094c5700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a0406cd10 con 0x7f5a04157940 2023-10-09T06:38:06.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.217+0000 7f5a02ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5a04157940 0x7f5a0406f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:06.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.217+0000 7f59fa7fc700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f5a0406c1e0 0x7f5a040cb6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:46964/0 (socket says 172.21.15.163:46964) 2023-10-09T06:38:06.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.217+0000 7f59fa7fc700 1 -- 172.21.15.163:0/1723024065 learned_addr learned my addr 172.21.15.163:0/1723024065 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:06.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.218+0000 7f5a02ffd700 1 -- 172.21.15.163:0/1723024065 >> v1:172.21.15.181:6789/0 conn(0x7f5a0415baa0 legacy=0x7f5a0415bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:06.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.218+0000 7f5a02ffd700 1 -- 172.21.15.163:0/1723024065 >> v1:172.21.15.163:6789/0 conn(0x7f5a0406c1e0 legacy=0x7f5a040cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:06.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.218+0000 7f5a02ffd700 1 -- 172.21.15.163:0/1723024065 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0406cff0 con 0x7f5a04157940 2023-10-09T06:38:06.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.218+0000 7f5a027fc700 1 -- 172.21.15.163:0/1723024065 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3680164213 0 0) 0x7f5a0406c750 con 0x7f5a0406c1e0 2023-10-09T06:38:06.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.218+0000 7f5a02ffd700 1 --2- 172.21.15.163:0/1723024065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a04157940 0x7f5a0406f5a0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f59f40098e0 tx=0x7f59f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e566dcc1f38e7e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:06.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.218+0000 7f5a027fc700 1 -- 172.21.15.163:0/1723024065 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f4009e80 con 0x7f5a04157940 2023-10-09T06:38:06.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.218+0000 7f5a027fc700 1 -- 172.21.15.163:0/1723024065 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f400b820 con 0x7f5a04157940 2023-10-09T06:38:06.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.218+0000 7f5a027fc700 1 -- 172.21.15.163:0/1723024065 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f400bba0 con 0x7f5a04157940 2023-10-09T06:38:06.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.218+0000 7f5a094c5700 1 -- 172.21.15.163:0/1723024065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a04157940 msgr2=0x7f5a0406f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:06.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.218+0000 7f5a094c5700 1 --2- 172.21.15.163:0/1723024065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a04157940 0x7f5a0406f5a0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f59f40098e0 tx=0x7f59f40091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:06.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.218+0000 7f5a094c5700 1 -- 172.21.15.163:0/1723024065 shutdown_connections 2023-10-09T06:38:06.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.218+0000 7f5a094c5700 1 --2- 172.21.15.163:0/1723024065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a04157940 0x7f5a0406f5a0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:06.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.218+0000 7f5a094c5700 1 -- 172.21.15.163:0/1723024065 >> 172.21.15.163:0/1723024065 conn(0x7f5a040cd480 msgr2=0x7f5a040cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:06.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.218+0000 7f5a094c5700 1 -- 172.21.15.163:0/1723024065 shutdown_connections 2023-10-09T06:38:06.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.218+0000 7f5a094c5700 1 -- 172.21.15.163:0/1723024065 wait complete. 2023-10-09T06:38:06.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.220+0000 7f5a094c5700 1 Processor -- start 2023-10-09T06:38:06.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.220+0000 7f5a094c5700 1 -- start start 2023-10-09T06:38:06.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.220+0000 7f5a094c5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a0406c1e0 0x7f5a04147f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:06.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.220+0000 7f5a094c5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a04157940 0x7f5a040a7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:06.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.220+0000 7f5a094c5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a0415baa0 0x7f5a040a8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:06.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.220+0000 7f5a094c5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a040c7d60 con 0x7f5a0415baa0 2023-10-09T06:38:06.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.220+0000 7f5a094c5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a040c7150 con 0x7f5a04157940 2023-10-09T06:38:06.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.220+0000 7f5a094c5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a040c7430 con 0x7f5a0406c1e0 2023-10-09T06:38:06.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.220+0000 7f5a037fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a0415baa0 0x7f5a040a8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:06.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.221+0000 7f5a037fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a0415baa0 0x7f5a040a8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42278/0 (socket says 172.21.15.163:42278) 2023-10-09T06:38:06.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.221+0000 7f5a037fe700 1 -- 172.21.15.163:0/537920909 learned_addr learned my addr 172.21.15.163:0/537920909 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:06.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.221+0000 7f59fa7fc700 1 --2- 172.21.15.163:0/537920909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a04157940 0x7f5a040a7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:06.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.221+0000 7f5a02ffd700 1 --2- 172.21.15.163:0/537920909 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a0406c1e0 0x7f5a04147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:06.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.221+0000 7f5a037fe700 1 -- 172.21.15.163:0/537920909 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a0406c1e0 msgr2=0x7f5a04147f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:06.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.221+0000 7f5a037fe700 1 --2- 172.21.15.163:0/537920909 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a0406c1e0 0x7f5a04147f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:06.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.221+0000 7f5a037fe700 1 -- 172.21.15.163:0/537920909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a04157940 msgr2=0x7f5a040a7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:06.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.221+0000 7f5a037fe700 1 --2- 172.21.15.163:0/537920909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a04157940 0x7f5a040a7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:06.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.221+0000 7f5a037fe700 1 -- 172.21.15.163:0/537920909 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f4009420 con 0x7f5a0415baa0 2023-10-09T06:38:06.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.221+0000 7f59fa7fc700 1 --2- 172.21.15.163:0/537920909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a04157940 0x7f5a040a7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:06.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.221+0000 7f5a037fe700 1 --2- 172.21.15.163:0/537920909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a0415baa0 0x7f5a040a8020 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00bc30 tx=0x7f59fc009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:06.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.221+0000 7f59fb7fe700 1 -- 172.21.15.163:0/537920909 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc00c9b0 con 0x7f5a0415baa0 2023-10-09T06:38:06.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.221+0000 7f59fb7fe700 1 -- 172.21.15.163:0/537920909 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59fc00cb50 con 0x7f5a0415baa0 2023-10-09T06:38:06.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.221+0000 7f5a094c5700 1 -- 172.21.15.163:0/537920909 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a040c7710 con 0x7f5a0415baa0 2023-10-09T06:38:06.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.221+0000 7f59fb7fe700 1 -- 172.21.15.163:0/537920909 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc014410 con 0x7f5a0415baa0 2023-10-09T06:38:06.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.221+0000 7f5a094c5700 1 -- 172.21.15.163:0/537920909 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a04148490 con 0x7f5a0415baa0 2023-10-09T06:38:06.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.222+0000 7f5a094c5700 1 -- 172.21.15.163:0/537920909 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a040c7f10 con 0x7f5a0415baa0 2023-10-09T06:38:06.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.223+0000 7f59fb7fe700 1 -- 172.21.15.163:0/537920909 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f59fc014800 con 0x7f5a0415baa0 2023-10-09T06:38:06.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.223+0000 7f59fb7fe700 1 --2- 172.21.15.163:0/537920909 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f59e4067320 0x7f59e40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:06.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.223+0000 7f59fb7fe700 1 -- 172.21.15.163:0/537920909 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f59fc084d60 con 0x7f5a0415baa0 2023-10-09T06:38:06.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.223+0000 7f5a02ffd700 1 --2- 172.21.15.163:0/537920909 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f59e4067320 0x7f59e40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:06.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.224+0000 7f5a02ffd700 1 --2- 172.21.15.163:0/537920909 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f59e4067320 0x7f59e40697d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009fd0 tx=0x7f59f4016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:06.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.227+0000 7f59fb7fe700 1 -- 172.21.15.163:0/537920909 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59fc04c5a0 con 0x7f5a0415baa0 2023-10-09T06:38:06.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.414+0000 7f5a094c5700 1 -- 172.21.15.163:0/537920909 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a0404c3c0 con 0x7f5a0415baa0 2023-10-09T06:38:06.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.414+0000 7f59fb7fe700 1 -- 172.21.15.163:0/537920909 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+15490 (secure 0 0 0) 0x7f59fc0519a0 con 0x7f5a0415baa0 2023-10-09T06:38:06.415 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:06.415 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":140,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:38:04.576334+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs.volume_0000000000063288.meta","create_time":"2023-10-09T06:37:44.861841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000063288"}}},{"pool":25,"pool_name":"cephfs.volume_0000000000063288.data","create_time":"2023-10-09T06:37:45.496241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000063288"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:38:06.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.416+0000 7f5a094c5700 1 -- 172.21.15.163:0/537920909 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f59e4067320 msgr2=0x7f59e40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:06.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.416+0000 7f5a094c5700 1 --2- 172.21.15.163:0/537920909 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f59e4067320 0x7f59e40697d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009fd0 tx=0x7f59f4016000 comp rx=0 tx=0).stop 2023-10-09T06:38:06.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.416+0000 7f5a094c5700 1 -- 172.21.15.163:0/537920909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a0415baa0 msgr2=0x7f5a040a8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:06.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.416+0000 7f5a094c5700 1 --2- 172.21.15.163:0/537920909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a0415baa0 0x7f5a040a8020 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00bc30 tx=0x7f59fc009820 comp rx=0 tx=0).stop 2023-10-09T06:38:06.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.417+0000 7f5a094c5700 1 -- 172.21.15.163:0/537920909 shutdown_connections 2023-10-09T06:38:06.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.417+0000 7f5a094c5700 1 --2- 172.21.15.163:0/537920909 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f59e4067320 0x7f59e40697d0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:06.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.417+0000 7f5a094c5700 1 --2- 172.21.15.163:0/537920909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a0415baa0 0x7f5a040a8020 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:06.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.417+0000 7f5a094c5700 1 --2- 172.21.15.163:0/537920909 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a0406c1e0 0x7f5a04147f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:06.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.417+0000 7f5a094c5700 1 --2- 172.21.15.163:0/537920909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a04157940 0x7f5a040a7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:06.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.417+0000 7f5a094c5700 1 -- 172.21.15.163:0/537920909 >> 172.21.15.163:0/537920909 conn(0x7f5a040cd480 msgr2=0x7f5a040b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:06.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.417+0000 7f5a094c5700 1 -- 172.21.15.163:0/537920909 shutdown_connections 2023-10-09T06:38:06.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.417+0000 7f5a094c5700 1 -- 172.21.15.163:0/537920909 wait complete. 2023-10-09T06:38:06.430 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 24, 'pool_name': 'cephfs.volume_0000000000063288.meta', 'create_time': '2023-10-09T06:37:44.861841+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '138', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'volume_0000000000063288'}}}, {'pool': 25, 'pool_name': 'cephfs.volume_0000000000063288.data', 'create_time': '2023-10-09T06:37:45.496241+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '138', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'volume_0000000000063288'}}}] 2023-10-09T06:38:06.430 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:38:06.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.547+0000 7f4ea8dae700 1 Processor -- start 2023-10-09T06:38:06.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.547+0000 7f4ea8dae700 1 -- start start 2023-10-09T06:38:06.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.548+0000 7f4ea8dae700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ea4141500 0x7f4ea40c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:06.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.548+0000 7f4ea8dae700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea415c2a0 con 0x7f4ea406bcd0 2023-10-09T06:38:06.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.548+0000 7f4ea8dae700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea415b690 con 0x7f4ea406c0a0 2023-10-09T06:38:06.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.548+0000 7f4ea8dae700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea415b970 con 0x7f4ea4141500 2023-10-09T06:38:06.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.548+0000 7f4ea259c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f4ea406bcd0 0x7f4ea4140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:46974/0 (socket says 172.21.15.163:46974) 2023-10-09T06:38:06.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.548+0000 7f4ea259c700 1 -- 172.21.15.163:0/28939006 learned_addr learned my addr 172.21.15.163:0/28939006 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:06.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.548+0000 7f4ea1d9b700 1 -- 172.21.15.163:0/28939006 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1457763677 0 0) 0x7f4ea415c2a0 con 0x7f4ea406bcd0 2023-10-09T06:38:06.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.548+0000 7f4ea1d9b700 1 -- 172.21.15.163:0/28939006 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e90003570 con 0x7f4ea406bcd0 2023-10-09T06:38:06.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.548+0000 7f4e9bfff700 1 --2- 172.21.15.163:0/28939006 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ea4141500 0x7f4ea40c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:06.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.549+0000 7f4ea1d9b700 1 -- 172.21.15.163:0/28939006 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1591617633 0 0) 0x7f4e90003570 con 0x7f4ea406bcd0 2023-10-09T06:38:06.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.549+0000 7f4ea1d9b700 1 -- 172.21.15.163:0/28939006 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ea4141500 msgr2=0x7f4ea40c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:06.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.549+0000 7f4ea1d9b700 1 --2- 172.21.15.163:0/28939006 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ea4141500 0x7f4ea40c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:06.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.549+0000 7f4ea1d9b700 1 -- 172.21.15.163:0/28939006 >> v1:172.21.15.181:6789/0 conn(0x7f4ea406c0a0 legacy=0x7f4ea40c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:06.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.549+0000 7f4ea1d9b700 1 -- 172.21.15.163:0/28939006 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea415bc50 con 0x7f4ea406bcd0 2023-10-09T06:38:06.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.549+0000 7f4ea1d9b700 1 -- 172.21.15.163:0/28939006 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f4e94001470 con 0x7f4ea406bcd0 2023-10-09T06:38:06.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.549+0000 7f4ea1d9b700 1 -- 172.21.15.163:0/28939006 >> v1:172.21.15.163:6789/0 conn(0x7f4ea406bcd0 legacy=0x7f4ea4140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:06.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.549+0000 7f4ea1d9b700 1 --2- 172.21.15.163:0/28939006 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ea4141500 0x7f4e90003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:06.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.549+0000 7f4ea1d9b700 1 --2- 172.21.15.163:0/28939006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e900040c0 0x7f4e90004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:06.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.549+0000 7f4ea1d9b700 1 --2- 172.21.15.163:0/28939006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e90004da0 0x7f4e900051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:06.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.549+0000 7f4ea1d9b700 1 -- 172.21.15.163:0/28939006 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea415b970 con 0x7f4e900040c0 2023-10-09T06:38:06.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.549+0000 7f4ea1d9b700 1 -- 172.21.15.163:0/28939006 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9000bc70 con 0x7f4e90004da0 2023-10-09T06:38:06.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.549+0000 7f4ea1d9b700 1 -- 172.21.15.163:0/28939006 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9000be30 con 0x7f4ea4141500 2023-10-09T06:38:06.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.549+0000 7f4e9bfff700 1 --2- 172.21.15.163:0/28939006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e900040c0 0x7f4e90004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:06.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.549+0000 7f4ea259c700 1 --2- 172.21.15.163:0/28939006 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ea4141500 0x7f4e90003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:06.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.549+0000 7f4ea2d9d700 1 --2- 172.21.15.163:0/28939006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e90004da0 0x7f4e900051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:06.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.549+0000 7f4e9bfff700 1 -- 172.21.15.163:0/28939006 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ea4141500 msgr2=0x7f4e90003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:06.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.549+0000 7f4e9bfff700 1 --2- 172.21.15.163:0/28939006 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ea4141500 0x7f4e90003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:06.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.549+0000 7f4e9bfff700 1 -- 172.21.15.163:0/28939006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e90004da0 msgr2=0x7f4e900051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:06.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.549+0000 7f4e9bfff700 1 --2- 172.21.15.163:0/28939006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e90004da0 0x7f4e900051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:06.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.549+0000 7f4e9bfff700 1 -- 172.21.15.163:0/28939006 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e90003570 con 0x7f4e900040c0 2023-10-09T06:38:06.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.549+0000 7f4e9bfff700 1 --2- 172.21.15.163:0/28939006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e900040c0 0x7f4e90004810 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00bb90 tx=0x7f4e8c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3539b24268e6489e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:06.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.549+0000 7f4ea1d9b700 1 -- 172.21.15.163:0/28939006 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c00d5d0 con 0x7f4e900040c0 2023-10-09T06:38:06.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.550+0000 7f4ea1d9b700 1 -- 172.21.15.163:0/28939006 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8c00d730 con 0x7f4e900040c0 2023-10-09T06:38:06.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.550+0000 7f4ea1d9b700 1 -- 172.21.15.163:0/28939006 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c00daf0 con 0x7f4e900040c0 2023-10-09T06:38:06.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.550+0000 7f4ea8dae700 1 -- 172.21.15.163:0/28939006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e900040c0 msgr2=0x7f4e90004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:06.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.550+0000 7f4ea8dae700 1 --2- 172.21.15.163:0/28939006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e900040c0 0x7f4e90004810 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00bb90 tx=0x7f4e8c008040 comp rx=0 tx=0).stop 2023-10-09T06:38:06.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.550+0000 7f4ea2d9d700 1 -- 172.21.15.163:0/28939006 reap_dead start 2023-10-09T06:38:06.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.550+0000 7f4ea8dae700 1 -- 172.21.15.163:0/28939006 shutdown_connections 2023-10-09T06:38:06.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.550+0000 7f4ea8dae700 1 -- 172.21.15.163:0/28939006 >> 172.21.15.163:0/28939006 conn(0x7f4ea40ce600 msgr2=0x7f4ea40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:06.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.550+0000 7f4ea8dae700 1 -- 172.21.15.163:0/28939006 shutdown_connections 2023-10-09T06:38:06.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.550+0000 7f4ea8dae700 1 -- 172.21.15.163:0/28939006 wait complete. 2023-10-09T06:38:06.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.551+0000 7f4ea8dae700 1 Processor -- start 2023-10-09T06:38:06.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.551+0000 7f4ea8dae700 1 -- start start 2023-10-09T06:38:06.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.552+0000 7f4ea8dae700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e900040c0 0x7f4ea413dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:06.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.552+0000 7f4ea8dae700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ea4156650 0x7f4ea4159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:06.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.552+0000 7f4ea8dae700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ea4156ac0 0x7f4ea415a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:06.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.552+0000 7f4ea8dae700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea413e200 con 0x7f4e900040c0 2023-10-09T06:38:06.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.552+0000 7f4ea8dae700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea416e1c0 con 0x7f4ea4156650 2023-10-09T06:38:06.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.552+0000 7f4ea8dae700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea416d5b0 con 0x7f4ea4156ac0 2023-10-09T06:38:06.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.552+0000 7f4ea259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e900040c0 0x7f4ea413dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:06.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.552+0000 7f4ea259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e900040c0 0x7f4ea413dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42300/0 (socket says 172.21.15.163:42300) 2023-10-09T06:38:06.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.552+0000 7f4ea259c700 1 -- 172.21.15.163:0/2474300819 learned_addr learned my addr 172.21.15.163:0/2474300819 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:06.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.552+0000 7f4ea2d9d700 1 --2- 172.21.15.163:0/2474300819 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ea4156ac0 0x7f4ea415a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:06.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.552+0000 7f4e9bfff700 1 --2- 172.21.15.163:0/2474300819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ea4156650 0x7f4ea4159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:06.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.552+0000 7f4ea259c700 1 -- 172.21.15.163:0/2474300819 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ea4156ac0 msgr2=0x7f4ea415a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:06.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.552+0000 7f4ea259c700 1 --2- 172.21.15.163:0/2474300819 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ea4156ac0 0x7f4ea415a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:06.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.552+0000 7f4ea259c700 1 -- 172.21.15.163:0/2474300819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ea4156650 msgr2=0x7f4ea4159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:06.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.552+0000 7f4ea259c700 1 --2- 172.21.15.163:0/2474300819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ea4156650 0x7f4ea4159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:06.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.552+0000 7f4ea259c700 1 -- 172.21.15.163:0/2474300819 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea416d890 con 0x7f4e900040c0 2023-10-09T06:38:06.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.552+0000 7f4e9bfff700 1 --2- 172.21.15.163:0/2474300819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ea4156650 0x7f4ea4159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:06.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.552+0000 7f4ea2d9d700 1 --2- 172.21.15.163:0/2474300819 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ea4156ac0 0x7f4ea415a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:06.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.552+0000 7f4ea259c700 1 --2- 172.21.15.163:0/2474300819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e900040c0 0x7f4ea413dcc0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f4e94007390 tx=0x7f4ea406e470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:06.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.553+0000 7f4e9a7fc700 1 -- 172.21.15.163:0/2474300819 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9400f8d0 con 0x7f4e900040c0 2023-10-09T06:38:06.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.553+0000 7f4e9a7fc700 1 -- 172.21.15.163:0/2474300819 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9400fa50 con 0x7f4e900040c0 2023-10-09T06:38:06.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.553+0000 7f4ea8dae700 1 -- 172.21.15.163:0/2474300819 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea416db70 con 0x7f4e900040c0 2023-10-09T06:38:06.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.553+0000 7f4e9a7fc700 1 -- 172.21.15.163:0/2474300819 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9400fe10 con 0x7f4e900040c0 2023-10-09T06:38:06.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.553+0000 7f4ea8dae700 1 -- 172.21.15.163:0/2474300819 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea40c74f0 con 0x7f4e900040c0 2023-10-09T06:38:06.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.554+0000 7f4ea8dae700 1 -- 172.21.15.163:0/2474300819 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea404c3c0 con 0x7f4e900040c0 2023-10-09T06:38:06.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.554+0000 7f4e9a7fc700 1 -- 172.21.15.163:0/2474300819 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4e94013600 con 0x7f4e900040c0 2023-10-09T06:38:06.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.554+0000 7f4e9a7fc700 1 --2- 172.21.15.163:0/2474300819 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4e84067320 0x7f4e840697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:06.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.555+0000 7f4e9a7fc700 1 -- 172.21.15.163:0/2474300819 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f4e94084e20 con 0x7f4e900040c0 2023-10-09T06:38:06.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.555+0000 7f4e9bfff700 1 --2- 172.21.15.163:0/2474300819 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4e84067320 0x7f4e840697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:06.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.555+0000 7f4e9bfff700 1 --2- 172.21.15.163:0/2474300819 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4e84067320 0x7f4e840697d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00bb90 tx=0x7f4e8c01b000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:06.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.559+0000 7f4e9a7fc700 1 -- 172.21.15.163:0/2474300819 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4e94051b70 con 0x7f4e900040c0 2023-10-09T06:38:06.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.740+0000 7f4ea8dae700 1 -- 172.21.15.163:0/2474300819 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4ea415bf30 con 0x7f4e900040c0 2023-10-09T06:38:06.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.740+0000 7f4e9a7fc700 1 -- 172.21.15.163:0/2474300819 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v140) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4e94055190 con 0x7f4e900040c0 2023-10-09T06:38:06.742 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:38:06.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.742+0000 7f4ea8dae700 1 -- 172.21.15.163:0/2474300819 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4e84067320 msgr2=0x7f4e840697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:06.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.742+0000 7f4ea8dae700 1 --2- 172.21.15.163:0/2474300819 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4e84067320 0x7f4e840697d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00bb90 tx=0x7f4e8c01b000 comp rx=0 tx=0).stop 2023-10-09T06:38:06.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.742+0000 7f4ea8dae700 1 -- 172.21.15.163:0/2474300819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e900040c0 msgr2=0x7f4ea413dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:06.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.742+0000 7f4ea8dae700 1 --2- 172.21.15.163:0/2474300819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e900040c0 0x7f4ea413dcc0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f4e94007390 tx=0x7f4ea406e470 comp rx=0 tx=0).stop 2023-10-09T06:38:06.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.742+0000 7f4ea8dae700 1 -- 172.21.15.163:0/2474300819 shutdown_connections 2023-10-09T06:38:06.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.742+0000 7f4ea8dae700 1 --2- 172.21.15.163:0/2474300819 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4e84067320 0x7f4e840697d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:06.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.742+0000 7f4ea8dae700 1 --2- 172.21.15.163:0/2474300819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e900040c0 0x7f4ea413dcc0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:06.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.742+0000 7f4ea8dae700 1 --2- 172.21.15.163:0/2474300819 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ea4156ac0 0x7f4ea415a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:06.746 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.742+0000 7f4ea8dae700 1 --2- 172.21.15.163:0/2474300819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ea4156650 0x7f4ea4159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:06.746 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.742+0000 7f4ea8dae700 1 -- 172.21.15.163:0/2474300819 >> 172.21.15.163:0/2474300819 conn(0x7f4ea40ce600 msgr2=0x7f4ea41575a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:06.746 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.743+0000 7f4ea8dae700 1 -- 172.21.15.163:0/2474300819 shutdown_connections 2023-10-09T06:38:06.746 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.743+0000 7f4ea8dae700 1 -- 172.21.15.163:0/2474300819 wait complete. 2023-10-09T06:38:06.757 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000063288.meta pg_num 2023-10-09T06:38:06.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.875+0000 7fc002521700 1 Processor -- start 2023-10-09T06:38:06.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.875+0000 7fc002521700 1 -- start start 2023-10-09T06:38:06.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.876+0000 7fc002521700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbffc159c60 0x7fbffc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:06.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.876+0000 7fc002521700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbffc15a160 con 0x7fbffc159890 2023-10-09T06:38:06.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.876+0000 7fc002521700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbffc0d13a0 con 0x7fbffc15db10 2023-10-09T06:38:06.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.876+0000 7fc002521700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbffc15df50 con 0x7fbffc159c60 2023-10-09T06:38:06.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.876+0000 7fbffbfff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fbffc159890 0x7fbffc15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:46980/0 (socket says 172.21.15.163:46980) 2023-10-09T06:38:06.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.876+0000 7fbffbfff700 1 -- 172.21.15.163:0/2203439658 learned_addr learned my addr 172.21.15.163:0/2203439658 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:06.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.876+0000 7fbffb7fe700 1 --2- 172.21.15.163:0/2203439658 >> v2:172.21.15.181:3300/0 conn(0x7fbffc159c60 0x7fbffc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:06.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.876+0000 7fbffaffd700 1 -- 172.21.15.163:0/2203439658 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3140136516 0 0) 0x7fbffc15a160 con 0x7fbffc159890 2023-10-09T06:38:06.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.876+0000 7fbffaffd700 1 -- 172.21.15.163:0/2203439658 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfe4003570 con 0x7fbffc159890 2023-10-09T06:38:06.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.876+0000 7fbffaffd700 1 -- 172.21.15.163:0/2203439658 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1200179225 0 0) 0x7fbfe4003570 con 0x7fbffc159890 2023-10-09T06:38:06.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.876+0000 7fbffaffd700 1 -- 172.21.15.163:0/2203439658 >> v2:172.21.15.181:3300/0 conn(0x7fbffc159c60 msgr2=0x7fbffc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:06.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.876+0000 7fbffaffd700 1 --2- 172.21.15.163:0/2203439658 >> v2:172.21.15.181:3300/0 conn(0x7fbffc159c60 0x7fbffc15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:06.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.876+0000 7fbffaffd700 1 -- 172.21.15.163:0/2203439658 >> v1:172.21.15.181:6789/0 conn(0x7fbffc15db10 legacy=0x7fbffc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:06.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.876+0000 7fbffaffd700 1 -- 172.21.15.163:0/2203439658 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc15e140 con 0x7fbffc159890 2023-10-09T06:38:06.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.876+0000 7fbffaffd700 1 -- 172.21.15.163:0/2203439658 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fbfec001470 con 0x7fbffc159890 2023-10-09T06:38:06.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.876+0000 7fbffaffd700 1 -- 172.21.15.163:0/2203439658 >> v1:172.21.15.163:6789/0 conn(0x7fbffc159890 legacy=0x7fbffc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:06.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.876+0000 7fbffaffd700 1 --2- 172.21.15.163:0/2203439658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfe4003a60 0x7fbfe4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:06.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.876+0000 7fbffaffd700 1 --2- 172.21.15.163:0/2203439658 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfe4004810 0x7fbfe4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:06.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.876+0000 7fbffaffd700 1 --2- 172.21.15.163:0/2203439658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfe40051c0 0x7fbfe400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:06.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.876+0000 7fbffaffd700 1 -- 172.21.15.163:0/2203439658 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe400c0a0 con 0x7fbfe4003a60 2023-10-09T06:38:06.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.876+0000 7fbffaffd700 1 -- 172.21.15.163:0/2203439658 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe400c210 con 0x7fbfe40051c0 2023-10-09T06:38:06.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.876+0000 7fbffaffd700 1 -- 172.21.15.163:0/2203439658 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe400c3d0 con 0x7fbfe4004810 2023-10-09T06:38:06.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.876+0000 7fbffb7fe700 1 --2- 172.21.15.163:0/2203439658 >> v2:172.21.15.181:3300/0 conn(0x7fbffc159c60 0x7fbffc15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:38:06.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.876+0000 7fbffbfff700 1 --2- 172.21.15.163:0/2203439658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfe4003a60 0x7fbfe4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:06.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.877+0000 7fc000abe700 1 --2- 172.21.15.163:0/2203439658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfe40051c0 0x7fbfe400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:06.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.877+0000 7fbffb7fe700 1 --2- 172.21.15.163:0/2203439658 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfe4004810 0x7fbfe4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:06.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.877+0000 7fbffbfff700 1 -- 172.21.15.163:0/2203439658 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfe4004810 msgr2=0x7fbfe4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:06.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.877+0000 7fbffbfff700 1 --2- 172.21.15.163:0/2203439658 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfe4004810 0x7fbfe4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:06.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.877+0000 7fbffbfff700 1 -- 172.21.15.163:0/2203439658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfe40051c0 msgr2=0x7fbfe400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:06.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.877+0000 7fbffbfff700 1 --2- 172.21.15.163:0/2203439658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfe40051c0 0x7fbfe400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:06.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.877+0000 7fc000abe700 1 -- 172.21.15.163:0/2203439658 reap_dead start 2023-10-09T06:38:06.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.877+0000 7fbffbfff700 1 -- 172.21.15.163:0/2203439658 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe4003570 con 0x7fbfe4003a60 2023-10-09T06:38:06.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.877+0000 7fbffbfff700 1 --2- 172.21.15.163:0/2203439658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfe4003a60 0x7fbfe4003e70 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fbfec00c3c0 tx=0x7fbfec001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6b8bab759293a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:06.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.877+0000 7fbffaffd700 1 -- 172.21.15.163:0/2203439658 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec00cc90 con 0x7fbfe4003a60 2023-10-09T06:38:06.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.877+0000 7fbffaffd700 1 -- 172.21.15.163:0/2203439658 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfec00cdf0 con 0x7fbfe4003a60 2023-10-09T06:38:06.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.877+0000 7fbffaffd700 1 -- 172.21.15.163:0/2203439658 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec005b70 con 0x7fbfe4003a60 2023-10-09T06:38:06.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.877+0000 7fc002521700 1 -- 172.21.15.163:0/2203439658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfe4003a60 msgr2=0x7fbfe4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:06.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.877+0000 7fc002521700 1 --2- 172.21.15.163:0/2203439658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfe4003a60 0x7fbfe4003e70 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fbfec00c3c0 tx=0x7fbfec001e50 comp rx=0 tx=0).stop 2023-10-09T06:38:06.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.877+0000 7fc002521700 1 -- 172.21.15.163:0/2203439658 shutdown_connections 2023-10-09T06:38:06.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.877+0000 7fc002521700 1 --2- 172.21.15.163:0/2203439658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfe4003a60 0x7fbfe4003e70 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:06.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.877+0000 7fc002521700 1 -- 172.21.15.163:0/2203439658 >> 172.21.15.163:0/2203439658 conn(0x7fbffc068e20 msgr2=0x7fbffc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:06.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.878+0000 7fc002521700 1 -- 172.21.15.163:0/2203439658 shutdown_connections 2023-10-09T06:38:06.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.878+0000 7fc002521700 1 -- 172.21.15.163:0/2203439658 wait complete. 2023-10-09T06:38:06.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.879+0000 7fc002521700 1 Processor -- start 2023-10-09T06:38:06.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.879+0000 7fc002521700 1 -- start start 2023-10-09T06:38:06.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.879+0000 7fc002521700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfe4003a60 0x7fbffc0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:06.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.879+0000 7fc002521700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbffc0ce8c0 0x7fbffc150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:06.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.879+0000 7fc002521700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbffc16d2b0 0x7fbffc16d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:06.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.880+0000 7fc002521700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc0c9460 con 0x7fbfe4003a60 2023-10-09T06:38:06.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.880+0000 7fc002521700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc16dc50 con 0x7fbffc0ce8c0 2023-10-09T06:38:06.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.880+0000 7fc002521700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc0ce430 con 0x7fbffc16d2b0 2023-10-09T06:38:06.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.880+0000 7fbffbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfe4003a60 0x7fbffc0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:06.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.880+0000 7fbffbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfe4003a60 0x7fbffc0b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42324/0 (socket says 172.21.15.163:42324) 2023-10-09T06:38:06.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.880+0000 7fbffbfff700 1 -- 172.21.15.163:0/2500423882 learned_addr learned my addr 172.21.15.163:0/2500423882 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:06.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.880+0000 7fbffb7fe700 1 --2- 172.21.15.163:0/2500423882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbffc0ce8c0 0x7fbffc150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:06.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.880+0000 7fc000abe700 1 --2- 172.21.15.163:0/2500423882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbffc16d2b0 0x7fbffc16d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:06.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.880+0000 7fbffbfff700 1 -- 172.21.15.163:0/2500423882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbffc16d2b0 msgr2=0x7fbffc16d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:06.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.880+0000 7fbffbfff700 1 --2- 172.21.15.163:0/2500423882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbffc16d2b0 0x7fbffc16d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:06.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.880+0000 7fbffbfff700 1 -- 172.21.15.163:0/2500423882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbffc0ce8c0 msgr2=0x7fbffc150c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:06.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.880+0000 7fbffbfff700 1 --2- 172.21.15.163:0/2500423882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbffc0ce8c0 0x7fbffc150c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:06.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.880+0000 7fbffbfff700 1 -- 172.21.15.163:0/2500423882 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc0cf870 con 0x7fbfe4003a60 2023-10-09T06:38:06.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.880+0000 7fc000abe700 1 --2- 172.21.15.163:0/2500423882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbffc16d2b0 0x7fbffc16d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:06.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.880+0000 7fbffbfff700 1 --2- 172.21.15.163:0/2500423882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfe4003a60 0x7fbffc0b3590 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400c1e0 tx=0x7fbfec015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:06.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.880+0000 7fbfe3fff700 1 -- 172.21.15.163:0/2500423882 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec000bc0 con 0x7fbfe4003a60 2023-10-09T06:38:06.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.880+0000 7fbfe3fff700 1 -- 172.21.15.163:0/2500423882 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfec0070d0 con 0x7fbfe4003a60 2023-10-09T06:38:06.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.880+0000 7fbfe3fff700 1 -- 172.21.15.163:0/2500423882 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec007490 con 0x7fbfe4003a60 2023-10-09T06:38:06.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.880+0000 7fc002521700 1 -- 172.21.15.163:0/2500423882 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbffc0cfb10 con 0x7fbfe4003a60 2023-10-09T06:38:06.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.880+0000 7fc002521700 1 -- 172.21.15.163:0/2500423882 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbffc0ce0c0 con 0x7fbfe4003a60 2023-10-09T06:38:06.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.881+0000 7fc002521700 1 -- 172.21.15.163:0/2500423882 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbffc04c3c0 con 0x7fbfe4003a60 2023-10-09T06:38:06.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.882+0000 7fbfe3fff700 1 -- 172.21.15.163:0/2500423882 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbfec01e430 con 0x7fbfe4003a60 2023-10-09T06:38:06.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.882+0000 7fbfe3fff700 1 --2- 172.21.15.163:0/2500423882 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbfe8067320 0x7fbfe80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:06.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.882+0000 7fbfe3fff700 1 -- 172.21.15.163:0/2500423882 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5783+0+0 (secure 0 0 0) 0x7fbfec085fd0 con 0x7fbfe4003a60 2023-10-09T06:38:06.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.882+0000 7fbffb7fe700 1 --2- 172.21.15.163:0/2500423882 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbfe8067320 0x7fbfe80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:06.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.883+0000 7fbffb7fe700 1 --2- 172.21.15.163:0/2500423882 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbfe8067320 0x7fbfe80697d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fbff0006d80 tx=0x7fbff0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:06.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:06.886+0000 7fbfe3fff700 1 -- 172.21.15.163:0/2500423882 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbfec052c10 con 0x7fbfe4003a60 2023-10-09T06:38:07.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.070+0000 7fc002521700 1 -- 172.21.15.163:0/2500423882 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000063288.meta", "var": "pg_num"} v 0) v1 -- 0x7fbffc06a5b0 con 0x7fbfe4003a60 2023-10-09T06:38:07.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.071+0000 7fbfe3fff700 1 -- 172.21.15.163:0/2500423882 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000063288.meta", "var": "pg_num"}]=0 v140) v1 ==== 124+0+11 (secure 0 0 0) 0x7fbfec017070 con 0x7fbfe4003a60 2023-10-09T06:38:07.072 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 32 2023-10-09T06:38:07.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.073+0000 7fc002521700 1 -- 172.21.15.163:0/2500423882 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbfe8067320 msgr2=0x7fbfe80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:07.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.073+0000 7fc002521700 1 --2- 172.21.15.163:0/2500423882 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbfe8067320 0x7fbfe80697d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fbff0006d80 tx=0x7fbff0008040 comp rx=0 tx=0).stop 2023-10-09T06:38:07.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.073+0000 7fc002521700 1 -- 172.21.15.163:0/2500423882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfe4003a60 msgr2=0x7fbffc0b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:07.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.073+0000 7fc002521700 1 --2- 172.21.15.163:0/2500423882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfe4003a60 0x7fbffc0b3590 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400c1e0 tx=0x7fbfec015000 comp rx=0 tx=0).stop 2023-10-09T06:38:07.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.073+0000 7fc002521700 1 -- 172.21.15.163:0/2500423882 shutdown_connections 2023-10-09T06:38:07.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.073+0000 7fc002521700 1 --2- 172.21.15.163:0/2500423882 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbfe8067320 0x7fbfe80697d0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.073+0000 7fc002521700 1 --2- 172.21.15.163:0/2500423882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfe4003a60 0x7fbffc0b3590 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.073+0000 7fc002521700 1 --2- 172.21.15.163:0/2500423882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbffc16d2b0 0x7fbffc16d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.073+0000 7fc002521700 1 --2- 172.21.15.163:0/2500423882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbffc0ce8c0 0x7fbffc150c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.073+0000 7fc002521700 1 -- 172.21.15.163:0/2500423882 >> 172.21.15.163:0/2500423882 conn(0x7fbffc068e20 msgr2=0x7fbffc152c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:07.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.073+0000 7fc002521700 1 -- 172.21.15.163:0/2500423882 shutdown_connections 2023-10-09T06:38:07.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.073+0000 7fc002521700 1 -- 172.21.15.163:0/2500423882 wait complete. 2023-10-09T06:38:07.086 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000063288.data pg_num 2023-10-09T06:38:07.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.204+0000 7f720f59b700 1 Processor -- start 2023-10-09T06:38:07.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.204+0000 7f720f59b700 1 -- start start 2023-10-09T06:38:07.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.205+0000 7f720f59b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7208159890 0x7f720815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:07.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.205+0000 7f720f59b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7208069560 0x7f72080699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:07.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.205+0000 7f720f59b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7208069ee0 con 0x7f7208159c60 2023-10-09T06:38:07.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.205+0000 7f720f59b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f720815a160 con 0x7f7208159890 2023-10-09T06:38:07.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.205+0000 7f720f59b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72080d13a0 con 0x7f7208069560 2023-10-09T06:38:07.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.205+0000 7f720db38700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7208069560 0x7f72080699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:07.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.205+0000 7f720d337700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7208159890 0x7f720815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:07.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.205+0000 7f720cb36700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7208159c60 0x7f720815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:42648/0 (socket says 172.21.15.163:42648) 2023-10-09T06:38:07.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.205+0000 7f720d337700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7208159890 0x7f720815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:45334/0 (socket says 172.21.15.163:45334) 2023-10-09T06:38:07.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.205+0000 7f720cb36700 1 -- 172.21.15.163:0/3360452851 learned_addr learned my addr 172.21.15.163:0/3360452851 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:07.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.206+0000 7f720d337700 1 -- 172.21.15.163:0/3360452851 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7208069560 msgr2=0x7f72080699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:07.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.206+0000 7f720d337700 1 --2- 172.21.15.163:0/3360452851 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7208069560 0x7f72080699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.206+0000 7f720d337700 1 -- 172.21.15.163:0/3360452851 >> v1:172.21.15.181:6789/0 conn(0x7f7208159c60 legacy=0x7f720815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:07.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.206+0000 7f720d337700 1 -- 172.21.15.163:0/3360452851 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720815e140 con 0x7f7208159890 2023-10-09T06:38:07.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.206+0000 7f720d337700 1 --2- 172.21.15.163:0/3360452851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7208159890 0x7f720815abd0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7204009e60 tx=0x7f7204009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=add01d223d13914e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:07.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.206+0000 7f71fffff700 1 -- 172.21.15.163:0/3360452851 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720400e070 con 0x7f7208159890 2023-10-09T06:38:07.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.206+0000 7f71fffff700 1 -- 172.21.15.163:0/3360452851 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720400bb60 con 0x7f7208159890 2023-10-09T06:38:07.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.206+0000 7f71fffff700 1 -- 172.21.15.163:0/3360452851 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7204012430 con 0x7f7208159890 2023-10-09T06:38:07.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.206+0000 7f720f59b700 1 -- 172.21.15.163:0/3360452851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7208159890 msgr2=0x7f720815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:07.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.206+0000 7f720f59b700 1 --2- 172.21.15.163:0/3360452851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7208159890 0x7f720815abd0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7204009e60 tx=0x7f7204009780 comp rx=0 tx=0).stop 2023-10-09T06:38:07.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.206+0000 7f720f59b700 1 -- 172.21.15.163:0/3360452851 shutdown_connections 2023-10-09T06:38:07.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.206+0000 7f720f59b700 1 --2- 172.21.15.163:0/3360452851 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7208159890 0x7f720815abd0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.206+0000 7f720f59b700 1 --2- 172.21.15.163:0/3360452851 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7208069560 0x7f72080699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.206+0000 7f720f59b700 1 -- 172.21.15.163:0/3360452851 >> 172.21.15.163:0/3360452851 conn(0x7f7208068e20 msgr2=0x7f72080d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:07.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.207+0000 7f720f59b700 1 -- 172.21.15.163:0/3360452851 shutdown_connections 2023-10-09T06:38:07.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.207+0000 7f720f59b700 1 -- 172.21.15.163:0/3360452851 wait complete. 2023-10-09T06:38:07.212 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.208+0000 7f720f59b700 1 Processor -- start 2023-10-09T06:38:07.212 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.208+0000 7f720f59b700 1 -- start start 2023-10-09T06:38:07.212 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.208+0000 7f720f59b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7208069560 0x7f720814d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:07.212 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.208+0000 7f720f59b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7208159890 0x7f720816a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:07.212 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.208+0000 7f720f59b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7208159c60 0x7f72080b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:07.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.208+0000 7f720f59b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72080ce120 con 0x7f7208159c60 2023-10-09T06:38:07.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.208+0000 7f720f59b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72080b4e30 con 0x7f7208159890 2023-10-09T06:38:07.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.208+0000 7f720f59b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f720816ad10 con 0x7f7208069560 2023-10-09T06:38:07.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.209+0000 7f720db38700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7208159c60 0x7f72080b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:07.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.209+0000 7f720db38700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7208159c60 0x7f72080b1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42328/0 (socket says 172.21.15.163:42328) 2023-10-09T06:38:07.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.209+0000 7f720db38700 1 -- 172.21.15.163:0/2754826839 learned_addr learned my addr 172.21.15.163:0/2754826839 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:07.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.209+0000 7f720cb36700 1 --2- 172.21.15.163:0/2754826839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7208159890 0x7f720816a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:07.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.209+0000 7f720d337700 1 --2- 172.21.15.163:0/2754826839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7208069560 0x7f720814d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:07.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.209+0000 7f720db38700 1 -- 172.21.15.163:0/2754826839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7208069560 msgr2=0x7f720814d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:07.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.209+0000 7f720db38700 1 --2- 172.21.15.163:0/2754826839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7208069560 0x7f720814d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.209+0000 7f720db38700 1 -- 172.21.15.163:0/2754826839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7208159890 msgr2=0x7f720816a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:07.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.209+0000 7f720db38700 1 --2- 172.21.15.163:0/2754826839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7208159890 0x7f720816a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.209+0000 7f720db38700 1 -- 172.21.15.163:0/2754826839 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7204009a20 con 0x7f7208159c60 2023-10-09T06:38:07.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.209+0000 7f720db38700 1 --2- 172.21.15.163:0/2754826839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7208159c60 0x7f72080b1bd0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f720000e8d0 tx=0x7f72000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:07.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.209+0000 7f71fd7fa700 1 -- 172.21.15.163:0/2754826839 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7200013680 con 0x7f7208159c60 2023-10-09T06:38:07.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.209+0000 7f71fd7fa700 1 -- 172.21.15.163:0/2754826839 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72000137e0 con 0x7f7208159c60 2023-10-09T06:38:07.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.209+0000 7f720f59b700 1 -- 172.21.15.163:0/2754826839 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72080cd5a0 con 0x7f7208159c60 2023-10-09T06:38:07.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.209+0000 7f71fd7fa700 1 -- 172.21.15.163:0/2754826839 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7200013ba0 con 0x7f7208159c60 2023-10-09T06:38:07.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.209+0000 7f720f59b700 1 -- 172.21.15.163:0/2754826839 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72080cdb60 con 0x7f7208159c60 2023-10-09T06:38:07.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.210+0000 7f720f59b700 1 -- 172.21.15.163:0/2754826839 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f720804c3c0 con 0x7f7208159c60 2023-10-09T06:38:07.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.211+0000 7f71fd7fa700 1 -- 172.21.15.163:0/2754826839 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f720001c450 con 0x7f7208159c60 2023-10-09T06:38:07.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.211+0000 7f71fd7fa700 1 --2- 172.21.15.163:0/2754826839 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f71ec067320 0x7f71ec0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:07.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.211+0000 7f71fd7fa700 1 -- 172.21.15.163:0/2754826839 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f7200084d40 con 0x7f7208159c60 2023-10-09T06:38:07.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.211+0000 7f720d337700 1 --2- 172.21.15.163:0/2754826839 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f71ec067320 0x7f71ec0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:07.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.212+0000 7f720d337700 1 --2- 172.21.15.163:0/2754826839 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f71ec067320 0x7f71ec0697d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f72040097c0 tx=0x7f7204019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:07.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.215+0000 7f71fd7fa700 1 -- 172.21.15.163:0/2754826839 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7200051980 con 0x7f7208159c60 2023-10-09T06:38:07.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.395+0000 7f720f59b700 1 -- 172.21.15.163:0/2754826839 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000063288.data", "var": "pg_num"} v 0) v1 -- 0x7f720806a5b0 con 0x7f7208159c60 2023-10-09T06:38:07.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.396+0000 7f71fd7fa700 1 -- 172.21.15.163:0/2754826839 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000063288.data", "var": "pg_num"}]=0 v140) v1 ==== 124+0+11 (secure 0 0 0) 0x7f72000170c0 con 0x7f7208159c60 2023-10-09T06:38:07.397 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 64 2023-10-09T06:38:07.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.398+0000 7f720f59b700 1 -- 172.21.15.163:0/2754826839 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f71ec067320 msgr2=0x7f71ec0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:07.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.398+0000 7f720f59b700 1 --2- 172.21.15.163:0/2754826839 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f71ec067320 0x7f71ec0697d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f72040097c0 tx=0x7f7204019000 comp rx=0 tx=0).stop 2023-10-09T06:38:07.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.398+0000 7f720f59b700 1 -- 172.21.15.163:0/2754826839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7208159c60 msgr2=0x7f72080b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:07.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.398+0000 7f720f59b700 1 --2- 172.21.15.163:0/2754826839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7208159c60 0x7f72080b1bd0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f720000e8d0 tx=0x7f72000091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.398+0000 7f720f59b700 1 -- 172.21.15.163:0/2754826839 shutdown_connections 2023-10-09T06:38:07.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.398+0000 7f720f59b700 1 --2- 172.21.15.163:0/2754826839 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f71ec067320 0x7f71ec0697d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.398+0000 7f720f59b700 1 --2- 172.21.15.163:0/2754826839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7208159c60 0x7f72080b1bd0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.398+0000 7f720f59b700 1 --2- 172.21.15.163:0/2754826839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7208069560 0x7f720814d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.400 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.398+0000 7f720f59b700 1 --2- 172.21.15.163:0/2754826839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7208159890 0x7f720816a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.400 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.398+0000 7f720f59b700 1 -- 172.21.15.163:0/2754826839 >> 172.21.15.163:0/2754826839 conn(0x7f7208068e20 msgr2=0x7f7208158790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:07.400 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.398+0000 7f720f59b700 1 -- 172.21.15.163:0/2754826839 shutdown_connections 2023-10-09T06:38:07.400 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.398+0000 7f720f59b700 1 -- 172.21.15.163:0/2754826839 wait complete. 2023-10-09T06:38:07.411 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:38:07.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.525+0000 7fc90f1f5700 1 Processor -- start 2023-10-09T06:38:07.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.525+0000 7fc90f1f5700 1 -- start start 2023-10-09T06:38:07.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.526+0000 7fc90f1f5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc908159c60 0x7fc90815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:07.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.526+0000 7fc90f1f5700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc90815a160 con 0x7fc908159890 2023-10-09T06:38:07.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.526+0000 7fc90f1f5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9080d13a0 con 0x7fc90815db10 2023-10-09T06:38:07.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.526+0000 7fc90f1f5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc90815df50 con 0x7fc908159c60 2023-10-09T06:38:07.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.526+0000 7fc90cf91700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fc908159890 0x7fc90815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:46992/0 (socket says 172.21.15.163:46992) 2023-10-09T06:38:07.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.526+0000 7fc90cf91700 1 -- 172.21.15.163:0/485133044 learned_addr learned my addr 172.21.15.163:0/485133044 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:07.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.526+0000 7fc9077fe700 1 -- 172.21.15.163:0/485133044 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2624407062 0 0) 0x7fc90815a160 con 0x7fc908159890 2023-10-09T06:38:07.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.527+0000 7fc9077fe700 1 -- 172.21.15.163:0/485133044 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8f0003570 con 0x7fc908159890 2023-10-09T06:38:07.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.527+0000 7fc907fff700 1 --2- 172.21.15.163:0/485133044 >> v2:172.21.15.181:3300/0 conn(0x7fc908159c60 0x7fc90815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:07.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.527+0000 7fc9077fe700 1 -- 172.21.15.163:0/485133044 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 303698487 0 0) 0x7fc8f0003570 con 0x7fc908159890 2023-10-09T06:38:07.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.527+0000 7fc9077fe700 1 -- 172.21.15.163:0/485133044 >> v2:172.21.15.181:3300/0 conn(0x7fc908159c60 msgr2=0x7fc90815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:07.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.527+0000 7fc9077fe700 1 --2- 172.21.15.163:0/485133044 >> v2:172.21.15.181:3300/0 conn(0x7fc908159c60 0x7fc90815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.527+0000 7fc9077fe700 1 -- 172.21.15.163:0/485133044 >> v1:172.21.15.181:6789/0 conn(0x7fc90815db10 legacy=0x7fc908069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:07.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.527+0000 7fc9077fe700 1 -- 172.21.15.163:0/485133044 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90815e140 con 0x7fc908159890 2023-10-09T06:38:07.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.527+0000 7fc9077fe700 1 -- 172.21.15.163:0/485133044 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fc8f80017f0 con 0x7fc908159890 2023-10-09T06:38:07.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.527+0000 7fc9077fe700 1 -- 172.21.15.163:0/485133044 >> v1:172.21.15.163:6789/0 conn(0x7fc908159890 legacy=0x7fc90815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:07.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.527+0000 7fc9077fe700 1 --2- 172.21.15.163:0/485133044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8f0003a60 0x7fc8f0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:07.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.527+0000 7fc9077fe700 1 --2- 172.21.15.163:0/485133044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f0004810 0x7fc8f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:07.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.527+0000 7fc9077fe700 1 --2- 172.21.15.163:0/485133044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8f00051c0 0x7fc8f000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:07.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.527+0000 7fc9077fe700 1 -- 172.21.15.163:0/485133044 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f000c0a0 con 0x7fc8f0004810 2023-10-09T06:38:07.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.527+0000 7fc9077fe700 1 -- 172.21.15.163:0/485133044 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f000c210 con 0x7fc8f00051c0 2023-10-09T06:38:07.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.527+0000 7fc9077fe700 1 -- 172.21.15.163:0/485133044 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f000c3d0 con 0x7fc8f0003a60 2023-10-09T06:38:07.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.527+0000 7fc907fff700 1 --2- 172.21.15.163:0/485133044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f0004810 0x7fc8f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:07.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.527+0000 7fc90d792700 1 --2- 172.21.15.163:0/485133044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8f00051c0 0x7fc8f000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:07.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.527+0000 7fc90cf91700 1 --2- 172.21.15.163:0/485133044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8f0003a60 0x7fc8f0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:07.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.527+0000 7fc907fff700 1 -- 172.21.15.163:0/485133044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8f0003a60 msgr2=0x7fc8f0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:07.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.527+0000 7fc907fff700 1 --2- 172.21.15.163:0/485133044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8f0003a60 0x7fc8f0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.527+0000 7fc907fff700 1 -- 172.21.15.163:0/485133044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8f00051c0 msgr2=0x7fc8f000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:07.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.527+0000 7fc907fff700 1 --2- 172.21.15.163:0/485133044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8f00051c0 0x7fc8f000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.527+0000 7fc907fff700 1 -- 172.21.15.163:0/485133044 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f0003570 con 0x7fc8f0004810 2023-10-09T06:38:07.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.527+0000 7fc90d792700 1 -- 172.21.15.163:0/485133044 reap_dead start 2023-10-09T06:38:07.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.527+0000 7fc907fff700 1 --2- 172.21.15.163:0/485133044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f0004810 0x7fc8f0004c80 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009c00 tx=0x7fc8fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10401a8b801d6125 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:07.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.528+0000 7fc9077fe700 1 -- 172.21.15.163:0/485133044 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc00e710 con 0x7fc8f0004810 2023-10-09T06:38:07.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.528+0000 7fc9077fe700 1 -- 172.21.15.163:0/485133044 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8fc00e870 con 0x7fc8f0004810 2023-10-09T06:38:07.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.528+0000 7fc9077fe700 1 -- 172.21.15.163:0/485133044 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc00ec30 con 0x7fc8f0004810 2023-10-09T06:38:07.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.528+0000 7fc90f1f5700 1 -- 172.21.15.163:0/485133044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f0004810 msgr2=0x7fc8f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:07.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.528+0000 7fc90f1f5700 1 --2- 172.21.15.163:0/485133044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f0004810 0x7fc8f0004c80 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009c00 tx=0x7fc8fc0091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.528+0000 7fc90f1f5700 1 -- 172.21.15.163:0/485133044 shutdown_connections 2023-10-09T06:38:07.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.528+0000 7fc90f1f5700 1 --2- 172.21.15.163:0/485133044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f0004810 0x7fc8f0004c80 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.528+0000 7fc90f1f5700 1 -- 172.21.15.163:0/485133044 >> 172.21.15.163:0/485133044 conn(0x7fc908068e20 msgr2=0x7fc9080d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:07.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.528+0000 7fc90f1f5700 1 -- 172.21.15.163:0/485133044 shutdown_connections 2023-10-09T06:38:07.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.528+0000 7fc90f1f5700 1 -- 172.21.15.163:0/485133044 wait complete. 2023-10-09T06:38:07.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.529+0000 7fc90f1f5700 1 Processor -- start 2023-10-09T06:38:07.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.529+0000 7fc90f1f5700 1 -- start start 2023-10-09T06:38:07.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.530+0000 7fc90f1f5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f0004810 0x7fc90815ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:07.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.530+0000 7fc90f1f5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc908156f20 0x7fc908157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:07.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.530+0000 7fc90f1f5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9080cf2e0 0x7fc9080cf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:07.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.530+0000 7fc90f1f5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90816a830 con 0x7fc8f0004810 2023-10-09T06:38:07.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.530+0000 7fc90f1f5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc908169ca0 con 0x7fc9080cf2e0 2023-10-09T06:38:07.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.530+0000 7fc90f1f5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9080cffa0 con 0x7fc908156f20 2023-10-09T06:38:07.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.530+0000 7fc90cf91700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f0004810 0x7fc90815ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:07.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.530+0000 7fc90cf91700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f0004810 0x7fc90815ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42358/0 (socket says 172.21.15.163:42358) 2023-10-09T06:38:07.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.530+0000 7fc90cf91700 1 -- 172.21.15.163:0/2616986567 learned_addr learned my addr 172.21.15.163:0/2616986567 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:07.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.530+0000 7fc90d792700 1 --2- 172.21.15.163:0/2616986567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9080cf2e0 0x7fc9080cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:07.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.530+0000 7fc907fff700 1 --2- 172.21.15.163:0/2616986567 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc908156f20 0x7fc908157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:07.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.530+0000 7fc90cf91700 1 -- 172.21.15.163:0/2616986567 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc908156f20 msgr2=0x7fc908157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:07.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.530+0000 7fc90cf91700 1 --2- 172.21.15.163:0/2616986567 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc908156f20 0x7fc908157390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.530+0000 7fc90cf91700 1 -- 172.21.15.163:0/2616986567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9080cf2e0 msgr2=0x7fc9080cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:07.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.530+0000 7fc90cf91700 1 --2- 172.21.15.163:0/2616986567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9080cf2e0 0x7fc9080cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.530+0000 7fc90cf91700 1 -- 172.21.15.163:0/2616986567 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9081531a0 con 0x7fc8f0004810 2023-10-09T06:38:07.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.531+0000 7fc90cf91700 1 --2- 172.21.15.163:0/2616986567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f0004810 0x7fc90815ce50 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8007c10 tx=0x7fc8f8001060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:07.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.531+0000 7fc904ff9700 1 -- 172.21.15.163:0/2616986567 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f800eca0 con 0x7fc8f0004810 2023-10-09T06:38:07.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.531+0000 7fc904ff9700 1 -- 172.21.15.163:0/2616986567 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f800ee00 con 0x7fc8f0004810 2023-10-09T06:38:07.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.531+0000 7fc90f1f5700 1 -- 172.21.15.163:0/2616986567 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9081578d0 con 0x7fc8f0004810 2023-10-09T06:38:07.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.531+0000 7fc904ff9700 1 -- 172.21.15.163:0/2616986567 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8013670 con 0x7fc8f0004810 2023-10-09T06:38:07.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.531+0000 7fc90f1f5700 1 -- 172.21.15.163:0/2616986567 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90815b4d0 con 0x7fc8f0004810 2023-10-09T06:38:07.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.532+0000 7fc90f1f5700 1 -- 172.21.15.163:0/2616986567 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc90804c3c0 con 0x7fc8f0004810 2023-10-09T06:38:07.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.532+0000 7fc904ff9700 1 -- 172.21.15.163:0/2616986567 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc8f80139e0 con 0x7fc8f0004810 2023-10-09T06:38:07.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.533+0000 7fc904ff9700 1 --2- 172.21.15.163:0/2616986567 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc8f4067320 0x7fc8f40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:07.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.533+0000 7fc904ff9700 1 -- 172.21.15.163:0/2616986567 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5783+0+0 (secure 0 0 0) 0x7fc8f80853c0 con 0x7fc8f0004810 2023-10-09T06:38:07.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.533+0000 7fc907fff700 1 --2- 172.21.15.163:0/2616986567 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc8f4067320 0x7fc8f40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:07.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.533+0000 7fc907fff700 1 --2- 172.21.15.163:0/2616986567 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc8f4067320 0x7fc8f40697d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc0091e0 tx=0x7fc8fc01a000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:07.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.537+0000 7fc904ff9700 1 -- 172.21.15.163:0/2616986567 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc8f80520c0 con 0x7fc8f0004810 2023-10-09T06:38:07.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.762+0000 7fc90f1f5700 1 -- 172.21.15.163:0/2616986567 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc90806a5b0 con 0x7fc8f0004810 2023-10-09T06:38:07.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.763+0000 7fc904ff9700 1 -- 172.21.15.163:0/2616986567 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4630 (secure 0 0 0) 0x7fc8f80556e0 con 0x7fc8f0004810 2023-10-09T06:38:07.764 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:07.765 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":95,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1554520486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1554520486},{"type":"v1","addr":"172.21.15.181:6837","nonce":1554520486}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1695339871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1695339871},{"type":"v1","addr":"172.21.15.181:6836","nonce":1695339871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":7515,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3668692283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3668692283},{"type":"v1","addr":"172.21.15.163:6834","nonce":3668692283}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:37:46.529939+0000","modified":"2023-10-09T06:38:02.561321+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7518":{"gid":7518,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/3043729326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3043729326},{"type":"v1","addr":"172.21.15.163:6835","nonce":3043729326}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"volume_0000000000063288","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-09T06:38:07.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.765+0000 7fc90f1f5700 1 -- 172.21.15.163:0/2616986567 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc8f4067320 msgr2=0x7fc8f40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:07.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.765+0000 7fc90f1f5700 1 --2- 172.21.15.163:0/2616986567 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc8f4067320 0x7fc8f40697d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc0091e0 tx=0x7fc8fc01a000 comp rx=0 tx=0).stop 2023-10-09T06:38:07.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.765+0000 7fc90f1f5700 1 -- 172.21.15.163:0/2616986567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f0004810 msgr2=0x7fc90815ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:07.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.765+0000 7fc90f1f5700 1 --2- 172.21.15.163:0/2616986567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f0004810 0x7fc90815ce50 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8007c10 tx=0x7fc8f8001060 comp rx=0 tx=0).stop 2023-10-09T06:38:07.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.765+0000 7fc90f1f5700 1 -- 172.21.15.163:0/2616986567 shutdown_connections 2023-10-09T06:38:07.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.765+0000 7fc90f1f5700 1 --2- 172.21.15.163:0/2616986567 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc8f4067320 0x7fc8f40697d0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.765+0000 7fc90f1f5700 1 --2- 172.21.15.163:0/2616986567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f0004810 0x7fc90815ce50 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.765+0000 7fc90f1f5700 1 --2- 172.21.15.163:0/2616986567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9080cf2e0 0x7fc9080cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.765+0000 7fc90f1f5700 1 --2- 172.21.15.163:0/2616986567 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc908156f20 0x7fc908157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.765+0000 7fc90f1f5700 1 -- 172.21.15.163:0/2616986567 >> 172.21.15.163:0/2616986567 conn(0x7fc908068e20 msgr2=0x7fc9081589d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:07.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.765+0000 7fc90f1f5700 1 -- 172.21.15.163:0/2616986567 shutdown_connections 2023-10-09T06:38:07.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.765+0000 7fc90f1f5700 1 -- 172.21.15.163:0/2616986567 wait complete. 2023-10-09T06:38:07.768 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 95 2023-10-09T06:38:07.779 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:38:07.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.896+0000 7f75fb447700 1 Processor -- start 2023-10-09T06:38:07.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.897+0000 7f75fb447700 1 -- start start 2023-10-09T06:38:07.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.897+0000 7f75fb447700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f75f406bcd0 0x7f75f40caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:07.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.897+0000 7f75fb447700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75f415df50 0x7f75f415e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:07.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.897+0000 7f75fb447700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75f415d7e0 con 0x7f75f406c0a0 2023-10-09T06:38:07.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.897+0000 7f75fb447700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75f415dac0 con 0x7f75f406bcd0 2023-10-09T06:38:07.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.897+0000 7f75fb447700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f415aa30 con 0x7f75f415df50 2023-10-09T06:38:07.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.898+0000 7f75f89e2700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f75f406c0a0 0x7f75f415a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:47008/0 (socket says 172.21.15.163:47008) 2023-10-09T06:38:07.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.898+0000 7f75f89e2700 1 -- 172.21.15.163:0/742769872 learned_addr learned my addr 172.21.15.163:0/742769872 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:07.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.898+0000 7f75f99e4700 1 --2- 172.21.15.163:0/742769872 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75f415df50 0x7f75f415e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:07.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.898+0000 7f75f91e3700 1 --2- 172.21.15.163:0/742769872 >> v2:172.21.15.181:3300/0 conn(0x7f75f406bcd0 0x7f75f40caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:07.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.898+0000 7f75ebfff700 1 -- 172.21.15.163:0/742769872 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3448398278 0 0) 0x7f75f415d7e0 con 0x7f75f406c0a0 2023-10-09T06:38:07.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.898+0000 7f75ebfff700 1 -- 172.21.15.163:0/742769872 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75e0003570 con 0x7f75f406c0a0 2023-10-09T06:38:07.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.898+0000 7f75f91e3700 1 -- 172.21.15.163:0/742769872 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75f415df50 msgr2=0x7f75f415e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:07.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.898+0000 7f75f91e3700 1 --2- 172.21.15.163:0/742769872 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75f415df50 0x7f75f415e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.898+0000 7f75f91e3700 1 -- 172.21.15.163:0/742769872 >> v1:172.21.15.163:6789/0 conn(0x7f75f406c0a0 legacy=0x7f75f415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:07.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.898+0000 7f75f91e3700 1 -- 172.21.15.163:0/742769872 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f415ad10 con 0x7f75f406bcd0 2023-10-09T06:38:07.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.898+0000 7f75f99e4700 1 --2- 172.21.15.163:0/742769872 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75f415df50 0x7f75f415e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:38:07.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.898+0000 7f75f91e3700 1 --2- 172.21.15.163:0/742769872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75f406bcd0 0x7f75f40caa90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f75f0009e60 tx=0x7f75f0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ece688eaf2b79a11 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:07.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.898+0000 7f75ebfff700 1 -- 172.21.15.163:0/742769872 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f000e070 con 0x7f75f406bcd0 2023-10-09T06:38:07.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.899+0000 7f75ebfff700 1 -- 172.21.15.163:0/742769872 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f000bab0 con 0x7f75f406bcd0 2023-10-09T06:38:07.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.899+0000 7f75ebfff700 1 -- 172.21.15.163:0/742769872 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f000be50 con 0x7f75f406bcd0 2023-10-09T06:38:07.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.899+0000 7f75fb447700 1 -- 172.21.15.163:0/742769872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75f406bcd0 msgr2=0x7f75f40caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:07.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.899+0000 7f75fb447700 1 --2- 172.21.15.163:0/742769872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75f406bcd0 0x7f75f40caa90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f75f0009e60 tx=0x7f75f0009780 comp rx=0 tx=0).stop 2023-10-09T06:38:07.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.899+0000 7f75fb447700 1 -- 172.21.15.163:0/742769872 shutdown_connections 2023-10-09T06:38:07.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.899+0000 7f75fb447700 1 --2- 172.21.15.163:0/742769872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75f406bcd0 0x7f75f40caa90 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.899+0000 7f75fb447700 1 --2- 172.21.15.163:0/742769872 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75f415df50 0x7f75f415e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.899+0000 7f75fb447700 1 -- 172.21.15.163:0/742769872 >> 172.21.15.163:0/742769872 conn(0x7f75f40ce600 msgr2=0x7f75f40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:07.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.899+0000 7f75fb447700 1 -- 172.21.15.163:0/742769872 shutdown_connections 2023-10-09T06:38:07.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.899+0000 7f75fb447700 1 -- 172.21.15.163:0/742769872 wait complete. 2023-10-09T06:38:07.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.900+0000 7f75fb447700 1 Processor -- start 2023-10-09T06:38:07.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.900+0000 7f75fb447700 1 -- start start 2023-10-09T06:38:07.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.901+0000 7f75fb447700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75f406bcd0 0x7f75f414c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:07.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.901+0000 7f75fb447700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75f406c0a0 0x7f75f40b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:07.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.901+0000 7f75fb447700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75f415df50 0x7f75f40c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:07.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.901+0000 7f75fb447700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f40c56b0 con 0x7f75f406c0a0 2023-10-09T06:38:07.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.901+0000 7f75fb447700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f40c6510 con 0x7f75f415df50 2023-10-09T06:38:07.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.901+0000 7f75fb447700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f40c69a0 con 0x7f75f406bcd0 2023-10-09T06:38:07.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.901+0000 7f75f89e2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75f406c0a0 0x7f75f40b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:07.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.901+0000 7f75f89e2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75f406c0a0 0x7f75f40b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42374/0 (socket says 172.21.15.163:42374) 2023-10-09T06:38:07.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.901+0000 7f75f89e2700 1 -- 172.21.15.163:0/3637232285 learned_addr learned my addr 172.21.15.163:0/3637232285 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:07.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.901+0000 7f75f89e2700 1 -- 172.21.15.163:0/3637232285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75f406bcd0 msgr2=0x7f75f414c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:07.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.901+0000 7f75f89e2700 1 --2- 172.21.15.163:0/3637232285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75f406bcd0 0x7f75f414c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.901+0000 7f75f89e2700 1 -- 172.21.15.163:0/3637232285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75f415df50 msgr2=0x7f75f40c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:07.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.901+0000 7f75f99e4700 1 --2- 172.21.15.163:0/3637232285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75f415df50 0x7f75f40c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:07.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.901+0000 7f75f89e2700 1 --2- 172.21.15.163:0/3637232285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75f415df50 0x7f75f40c7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:07.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.901+0000 7f75f89e2700 1 -- 172.21.15.163:0/3637232285 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ec009420 con 0x7f75f406c0a0 2023-10-09T06:38:07.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.902+0000 7f75f89e2700 1 --2- 172.21.15.163:0/3637232285 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75f406c0a0 0x7f75f40b3c30 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f75e400c5c0 tx=0x7f75e4007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:07.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.902+0000 7f75e97fa700 1 -- 172.21.15.163:0/3637232285 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e4005b30 con 0x7f75f406c0a0 2023-10-09T06:38:07.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.902+0000 7f75e97fa700 1 -- 172.21.15.163:0/3637232285 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e4005cb0 con 0x7f75f406c0a0 2023-10-09T06:38:07.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.902+0000 7f75fb447700 1 -- 172.21.15.163:0/3637232285 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f0009a20 con 0x7f75f406c0a0 2023-10-09T06:38:07.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.902+0000 7f75e97fa700 1 -- 172.21.15.163:0/3637232285 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e4007320 con 0x7f75f406c0a0 2023-10-09T06:38:07.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.902+0000 7f75fb447700 1 -- 172.21.15.163:0/3637232285 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f40c5900 con 0x7f75f406c0a0 2023-10-09T06:38:07.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.903+0000 7f75fb447700 1 -- 172.21.15.163:0/3637232285 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75f404c3c0 con 0x7f75f406c0a0 2023-10-09T06:38:07.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.903+0000 7f75e97fa700 1 -- 172.21.15.163:0/3637232285 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f75e401b3b0 con 0x7f75f406c0a0 2023-10-09T06:38:07.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.903+0000 7f75e97fa700 1 --2- 172.21.15.163:0/3637232285 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f75d8067320 0x7f75d80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:07.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.904+0000 7f75e97fa700 1 -- 172.21.15.163:0/3637232285 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f75e4084230 con 0x7f75f406c0a0 2023-10-09T06:38:07.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.904+0000 7f75f91e3700 1 --2- 172.21.15.163:0/3637232285 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f75d8067320 0x7f75d80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:07.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.904+0000 7f75f91e3700 1 --2- 172.21.15.163:0/3637232285 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f75d8067320 0x7f75d80697d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f75f0009e60 tx=0x7f75f0019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:07.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:07.908+0000 7f75e97fa700 1 -- 172.21.15.163:0/3637232285 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f75e4050e70 con 0x7f75f406c0a0 2023-10-09T06:38:08.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.092+0000 7f75fb447700 1 -- 172.21.15.163:0/3637232285 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75f40c6650 con 0x7f75f406c0a0 2023-10-09T06:38:08.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.093+0000 7f75e97fa700 1 -- 172.21.15.163:0/3637232285 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+15490 (secure 0 0 0) 0x7f75e4054490 con 0x7f75f406c0a0 2023-10-09T06:38:08.094 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:08.094 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":140,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:38:04.576334+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs.volume_0000000000063288.meta","create_time":"2023-10-09T06:37:44.861841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000063288"}}},{"pool":25,"pool_name":"cephfs.volume_0000000000063288.data","create_time":"2023-10-09T06:37:45.496241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000063288"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:38:08.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.096+0000 7f75fb447700 1 -- 172.21.15.163:0/3637232285 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f75d8067320 msgr2=0x7f75d80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:08.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.096+0000 7f75fb447700 1 --2- 172.21.15.163:0/3637232285 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f75d8067320 0x7f75d80697d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f75f0009e60 tx=0x7f75f0019000 comp rx=0 tx=0).stop 2023-10-09T06:38:08.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.096+0000 7f75fb447700 1 -- 172.21.15.163:0/3637232285 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75f406c0a0 msgr2=0x7f75f40b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:08.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.096+0000 7f75fb447700 1 --2- 172.21.15.163:0/3637232285 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75f406c0a0 0x7f75f40b3c30 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f75e400c5c0 tx=0x7f75e4007e00 comp rx=0 tx=0).stop 2023-10-09T06:38:08.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.096+0000 7f75fb447700 1 -- 172.21.15.163:0/3637232285 shutdown_connections 2023-10-09T06:38:08.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.096+0000 7f75fb447700 1 --2- 172.21.15.163:0/3637232285 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f75d8067320 0x7f75d80697d0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:08.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.096+0000 7f75fb447700 1 --2- 172.21.15.163:0/3637232285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75f415df50 0x7f75f40c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:08.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.096+0000 7f75fb447700 1 --2- 172.21.15.163:0/3637232285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75f406bcd0 0x7f75f414c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:08.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.096+0000 7f75fb447700 1 --2- 172.21.15.163:0/3637232285 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75f406c0a0 0x7f75f40b3c30 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:08.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.096+0000 7f75fb447700 1 -- 172.21.15.163:0/3637232285 >> 172.21.15.163:0/3637232285 conn(0x7f75f40ce600 msgr2=0x7f75f415c780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:08.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.096+0000 7f75fb447700 1 -- 172.21.15.163:0/3637232285 shutdown_connections 2023-10-09T06:38:08.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.096+0000 7f75fb447700 1 -- 172.21.15.163:0/3637232285 wait complete. 2023-10-09T06:38:08.109 INFO:tasks.cephfs.filesystem:Destroying file system volume_0000000000063288 and related pools 2023-10-09T06:38:08.109 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:38:08.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.227+0000 7fd61919f700 1 Processor -- start 2023-10-09T06:38:08.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.227+0000 7fd61919f700 1 -- start start 2023-10-09T06:38:08.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.227+0000 7fd61919f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6141597f0 0x7fd6140b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:08.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.228+0000 7fd61919f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd61415d230 0x7fd61415d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:08.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.228+0000 7fd61919f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd61415dbd0 con 0x7fd614159420 2023-10-09T06:38:08.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.228+0000 7fd61919f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6140b11d0 con 0x7fd61415d230 2023-10-09T06:38:08.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.228+0000 7fd61919f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6140b14b0 con 0x7fd6141597f0 2023-10-09T06:38:08.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.228+0000 7fd613fff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fd614159420 0x7fd6140b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:47014/0 (socket says 172.21.15.163:47014) 2023-10-09T06:38:08.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.228+0000 7fd613fff700 1 -- 172.21.15.163:0/3268482311 learned_addr learned my addr 172.21.15.163:0/3268482311 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:08.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.228+0000 7fd6137fe700 1 --2- 172.21.15.163:0/3268482311 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6141597f0 0x7fd6140b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:08.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.228+0000 7fd61899e700 1 --2- 172.21.15.163:0/3268482311 >> v2:172.21.15.181:3300/0 conn(0x7fd61415d230 0x7fd61415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:08.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.228+0000 7fd612ffd700 1 -- 172.21.15.163:0/3268482311 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2661923793 0 0) 0x7fd604001110 con 0x7fd614159420 2023-10-09T06:38:08.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.228+0000 7fd612ffd700 1 -- 172.21.15.163:0/3268482311 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5fc003570 con 0x7fd614159420 2023-10-09T06:38:08.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.228+0000 7fd61899e700 1 -- 172.21.15.163:0/3268482311 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6141597f0 msgr2=0x7fd6140b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:08.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.228+0000 7fd612ffd700 1 -- 172.21.15.163:0/3268482311 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2289595486 0 0) 0x7fd5fc003570 con 0x7fd614159420 2023-10-09T06:38:08.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.228+0000 7fd61899e700 1 --2- 172.21.15.163:0/3268482311 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6141597f0 0x7fd6140b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:08.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.228+0000 7fd61899e700 1 -- 172.21.15.163:0/3268482311 >> v1:172.21.15.163:6789/0 conn(0x7fd614159420 legacy=0x7fd6140b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:08.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.228+0000 7fd61899e700 1 -- 172.21.15.163:0/3268482311 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6140b1790 con 0x7fd61415d230 2023-10-09T06:38:08.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.228+0000 7fd6137fe700 1 --2- 172.21.15.163:0/3268482311 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6141597f0 0x7fd6140b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:38:08.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.228+0000 7fd61899e700 1 --2- 172.21.15.163:0/3268482311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd61415d230 0x7fd61415d690 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd60c009e60 tx=0x7fd60c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f866e413f477fab7 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:08.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.229+0000 7fd612ffd700 1 -- 172.21.15.163:0/3268482311 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c00e070 con 0x7fd61415d230 2023-10-09T06:38:08.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.229+0000 7fd612ffd700 1 -- 172.21.15.163:0/3268482311 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60c00b7a0 con 0x7fd61415d230 2023-10-09T06:38:08.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.229+0000 7fd612ffd700 1 -- 172.21.15.163:0/3268482311 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd604001600 con 0x7fd61415d230 2023-10-09T06:38:08.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.229+0000 7fd61919f700 1 -- 172.21.15.163:0/3268482311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd61415d230 msgr2=0x7fd61415d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:08.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.229+0000 7fd61919f700 1 --2- 172.21.15.163:0/3268482311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd61415d230 0x7fd61415d690 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd60c009e60 tx=0x7fd60c009780 comp rx=0 tx=0).stop 2023-10-09T06:38:08.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.229+0000 7fd61919f700 1 -- 172.21.15.163:0/3268482311 shutdown_connections 2023-10-09T06:38:08.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.229+0000 7fd61919f700 1 --2- 172.21.15.163:0/3268482311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd61415d230 0x7fd61415d690 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:08.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.229+0000 7fd61919f700 1 --2- 172.21.15.163:0/3268482311 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6141597f0 0x7fd6140b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:08.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.229+0000 7fd61919f700 1 -- 172.21.15.163:0/3268482311 >> 172.21.15.163:0/3268482311 conn(0x7fd6140d0660 msgr2=0x7fd6140d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:08.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.229+0000 7fd61919f700 1 -- 172.21.15.163:0/3268482311 shutdown_connections 2023-10-09T06:38:08.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.229+0000 7fd61919f700 1 -- 172.21.15.163:0/3268482311 wait complete. 2023-10-09T06:38:08.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.230+0000 7fd61919f700 1 Processor -- start 2023-10-09T06:38:08.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.230+0000 7fd61919f700 1 -- start start 2023-10-09T06:38:08.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.231+0000 7fd61919f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd614159420 0x7fd614154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:08.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.231+0000 7fd61919f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6141597f0 0x7fd614154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:08.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.231+0000 7fd61919f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd61415d230 0x7fd6140b5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:08.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.231+0000 7fd61919f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61415ad90 con 0x7fd6141597f0 2023-10-09T06:38:08.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.231+0000 7fd61919f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61415a160 con 0x7fd61415d230 2023-10-09T06:38:08.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.231+0000 7fd61919f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61415a440 con 0x7fd614159420 2023-10-09T06:38:08.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.231+0000 7fd6137fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6141597f0 0x7fd614154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:08.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.231+0000 7fd6137fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6141597f0 0x7fd614154560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42378/0 (socket says 172.21.15.163:42378) 2023-10-09T06:38:08.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.231+0000 7fd6137fe700 1 -- 172.21.15.163:0/864758475 learned_addr learned my addr 172.21.15.163:0/864758475 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:08.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.231+0000 7fd61899e700 1 --2- 172.21.15.163:0/864758475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd61415d230 0x7fd6140b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:08.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.231+0000 7fd6137fe700 1 -- 172.21.15.163:0/864758475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd614159420 msgr2=0x7fd614154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:08.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.231+0000 7fd613fff700 1 --2- 172.21.15.163:0/864758475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd614159420 0x7fd614154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:08.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.231+0000 7fd6137fe700 1 --2- 172.21.15.163:0/864758475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd614159420 0x7fd614154bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:08.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.231+0000 7fd6137fe700 1 -- 172.21.15.163:0/864758475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd61415d230 msgr2=0x7fd6140b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:08.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.231+0000 7fd6137fe700 1 --2- 172.21.15.163:0/864758475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd61415d230 0x7fd6140b5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:08.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.232+0000 7fd6137fe700 1 -- 172.21.15.163:0/864758475 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd608009420 con 0x7fd6141597f0 2023-10-09T06:38:08.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.232+0000 7fd61899e700 1 --2- 172.21.15.163:0/864758475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd61415d230 0x7fd6140b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:08.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.232+0000 7fd613fff700 1 --2- 172.21.15.163:0/864758475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd614159420 0x7fd614154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:08.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.232+0000 7fd6137fe700 1 --2- 172.21.15.163:0/864758475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6141597f0 0x7fd614154560 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fd60800ec80 tx=0x7fd6080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:08.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.232+0000 7fd5fbfff700 1 -- 172.21.15.163:0/864758475 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60800c9d0 con 0x7fd6141597f0 2023-10-09T06:38:08.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.232+0000 7fd5fbfff700 1 -- 172.21.15.163:0/864758475 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60800cb30 con 0x7fd6141597f0 2023-10-09T06:38:08.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.232+0000 7fd61919f700 1 -- 172.21.15.163:0/864758475 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd60c009a20 con 0x7fd6141597f0 2023-10-09T06:38:08.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.232+0000 7fd5fbfff700 1 -- 172.21.15.163:0/864758475 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60800ceb0 con 0x7fd6141597f0 2023-10-09T06:38:08.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.232+0000 7fd61919f700 1 -- 172.21.15.163:0/864758475 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd61415aa00 con 0x7fd6141597f0 2023-10-09T06:38:08.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.233+0000 7fd61919f700 1 -- 172.21.15.163:0/864758475 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd61404c3c0 con 0x7fd6141597f0 2023-10-09T06:38:08.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.233+0000 7fd5fbfff700 1 -- 172.21.15.163:0/864758475 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd608013680 con 0x7fd6141597f0 2023-10-09T06:38:08.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.234+0000 7fd5fbfff700 1 --2- 172.21.15.163:0/864758475 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd600067320 0x7fd6000697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:08.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.234+0000 7fd5fbfff700 1 -- 172.21.15.163:0/864758475 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5783+0+0 (secure 0 0 0) 0x7fd608085140 con 0x7fd6141597f0 2023-10-09T06:38:08.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.234+0000 7fd613fff700 1 --2- 172.21.15.163:0/864758475 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd600067320 0x7fd6000697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:08.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.234+0000 7fd613fff700 1 --2- 172.21.15.163:0/864758475 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd600067320 0x7fd6000697d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd604007ca0 tx=0x7fd604009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:08.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.238+0000 7fd5fbfff700 1 -- 172.21.15.163:0/864758475 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd608051d80 con 0x7fd6141597f0 2023-10-09T06:38:08.463 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.462+0000 7fd61919f700 1 -- 172.21.15.163:0/864758475 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6140b1a70 con 0x7fd6141597f0 2023-10-09T06:38:08.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.462+0000 7fd5fbfff700 1 -- 172.21.15.163:0/864758475 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4630 (secure 0 0 0) 0x7fd6080553a0 con 0x7fd6141597f0 2023-10-09T06:38:08.464 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:08.464 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":95,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1554520486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1554520486},{"type":"v1","addr":"172.21.15.181:6837","nonce":1554520486}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1695339871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1695339871},{"type":"v1","addr":"172.21.15.181:6836","nonce":1695339871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":7515,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3668692283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3668692283},{"type":"v1","addr":"172.21.15.163:6834","nonce":3668692283}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:37:46.529939+0000","modified":"2023-10-09T06:38:02.561321+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7518":{"gid":7518,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/3043729326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3043729326},{"type":"v1","addr":"172.21.15.163:6835","nonce":3043729326}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"volume_0000000000063288","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-09T06:38:08.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.464+0000 7fd61919f700 1 -- 172.21.15.163:0/864758475 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd600067320 msgr2=0x7fd6000697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:08.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.464+0000 7fd61919f700 1 --2- 172.21.15.163:0/864758475 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd600067320 0x7fd6000697d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd604007ca0 tx=0x7fd604009040 comp rx=0 tx=0).stop 2023-10-09T06:38:08.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.464+0000 7fd61919f700 1 -- 172.21.15.163:0/864758475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6141597f0 msgr2=0x7fd614154560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:08.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.464+0000 7fd61919f700 1 --2- 172.21.15.163:0/864758475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6141597f0 0x7fd614154560 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fd60800ec80 tx=0x7fd6080091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:08.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.464+0000 7fd61919f700 1 -- 172.21.15.163:0/864758475 shutdown_connections 2023-10-09T06:38:08.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.464+0000 7fd61919f700 1 --2- 172.21.15.163:0/864758475 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd600067320 0x7fd6000697d0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:08.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.464+0000 7fd61919f700 1 --2- 172.21.15.163:0/864758475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd61415d230 0x7fd6140b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:08.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.464+0000 7fd61919f700 1 --2- 172.21.15.163:0/864758475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd614159420 0x7fd614154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:08.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.464+0000 7fd61919f700 1 --2- 172.21.15.163:0/864758475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6141597f0 0x7fd614154560 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:08.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.464+0000 7fd61919f700 1 -- 172.21.15.163:0/864758475 >> 172.21.15.163:0/864758475 conn(0x7fd6140d0660 msgr2=0x7fd614069770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:08.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.465+0000 7fd61919f700 1 -- 172.21.15.163:0/864758475 shutdown_connections 2023-10-09T06:38:08.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.465+0000 7fd61919f700 1 -- 172.21.15.163:0/864758475 wait complete. 2023-10-09T06:38:08.467 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 95 2023-10-09T06:38:08.479 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:38:08.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.596+0000 7f2997f73700 1 Processor -- start 2023-10-09T06:38:08.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.597+0000 7f2997f73700 1 -- start start 2023-10-09T06:38:08.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.597+0000 7f2997f73700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f29900aca80 0x7f299006aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:08.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.598+0000 7f2997f73700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29900c83c0 0x7f29900cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:08.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.598+0000 7f2997f73700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f299006bdb0 con 0x7f2990069d60 2023-10-09T06:38:08.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.598+0000 7f2997f73700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29900cb370 con 0x7f29900aca80 2023-10-09T06:38:08.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.598+0000 7f2997f73700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2990159f60 con 0x7f29900c83c0 2023-10-09T06:38:08.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.598+0000 7f2995d0f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f29900aca80 0x7f299006aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:08.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.598+0000 7f2996510700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29900c83c0 0x7f29900cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:08.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.598+0000 7f299550e700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f2990069d60 0x7f29900c7cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:48666/0 (socket says 172.21.15.163:48666) 2023-10-09T06:38:08.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.598+0000 7f299550e700 1 -- 172.21.15.163:0/638651141 learned_addr learned my addr 172.21.15.163:0/638651141 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:08.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.598+0000 7f2996510700 1 -- 172.21.15.163:0/638651141 >> v2:172.21.15.181:3300/0 conn(0x7f29900aca80 msgr2=0x7f299006aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:08.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.598+0000 7f2996510700 1 --2- 172.21.15.163:0/638651141 >> v2:172.21.15.181:3300/0 conn(0x7f29900aca80 0x7f299006aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:08.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.598+0000 7f2996510700 1 -- 172.21.15.163:0/638651141 >> v1:172.21.15.181:6789/0 conn(0x7f2990069d60 legacy=0x7f29900c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:08.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.598+0000 7f2996510700 1 -- 172.21.15.163:0/638651141 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299006b510 con 0x7f29900c83c0 2023-10-09T06:38:08.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.598+0000 7f2995d0f700 1 --2- 172.21.15.163:0/638651141 >> v2:172.21.15.181:3300/0 conn(0x7f29900aca80 0x7f299006aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:38:08.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.598+0000 7f2996510700 1 --2- 172.21.15.163:0/638651141 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29900c83c0 0x7f29900cb800 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2988009e60 tx=0x7f2988009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cc6dd2f34acb468 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:08.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.599+0000 7f2994d0d700 1 -- 172.21.15.163:0/638651141 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298800e070 con 0x7f29900c83c0 2023-10-09T06:38:08.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.599+0000 7f2994d0d700 1 -- 172.21.15.163:0/638651141 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f298800bb60 con 0x7f29900c83c0 2023-10-09T06:38:08.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.599+0000 7f2994d0d700 1 -- 172.21.15.163:0/638651141 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2988012430 con 0x7f29900c83c0 2023-10-09T06:38:08.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.599+0000 7f2997f73700 1 -- 172.21.15.163:0/638651141 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29900c83c0 msgr2=0x7f29900cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:08.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.599+0000 7f2997f73700 1 --2- 172.21.15.163:0/638651141 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29900c83c0 0x7f29900cb800 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2988009e60 tx=0x7f2988009780 comp rx=0 tx=0).stop 2023-10-09T06:38:08.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.599+0000 7f2997f73700 1 -- 172.21.15.163:0/638651141 shutdown_connections 2023-10-09T06:38:08.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.599+0000 7f2997f73700 1 --2- 172.21.15.163:0/638651141 >> v2:172.21.15.181:3300/0 conn(0x7f29900aca80 0x7f299006aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:08.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.599+0000 7f2997f73700 1 --2- 172.21.15.163:0/638651141 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29900c83c0 0x7f29900cb800 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:08.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.599+0000 7f2997f73700 1 -- 172.21.15.163:0/638651141 >> 172.21.15.163:0/638651141 conn(0x7f29900cd670 msgr2=0x7f29900cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:08.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.599+0000 7f2997f73700 1 -- 172.21.15.163:0/638651141 shutdown_connections 2023-10-09T06:38:08.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.599+0000 7f2997f73700 1 -- 172.21.15.163:0/638651141 wait complete. 2023-10-09T06:38:08.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.600+0000 7f2997f73700 1 Processor -- start 2023-10-09T06:38:08.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.600+0000 7f2997f73700 1 -- start start 2023-10-09T06:38:08.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.601+0000 7f2997f73700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2990069d60 0x7f29900c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:08.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.601+0000 7f2997f73700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29900aca80 0x7f29900b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:08.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.601+0000 7f2997f73700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29900c83c0 0x7f2990157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:08.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.601+0000 7f2997f73700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29900ace50 con 0x7f2990069d60 2023-10-09T06:38:08.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.601+0000 7f2997f73700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29900b0950 con 0x7f29900c83c0 2023-10-09T06:38:08.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.601+0000 7f2997f73700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29900b0bd0 con 0x7f29900aca80 2023-10-09T06:38:08.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.601+0000 7f2995d0f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2990069d60 0x7f29900c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:08.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.601+0000 7f2995d0f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2990069d60 0x7f29900c7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59976/0 (socket says 172.21.15.163:59976) 2023-10-09T06:38:08.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.601+0000 7f2995d0f700 1 -- 172.21.15.163:0/3552625044 learned_addr learned my addr 172.21.15.163:0/3552625044 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:08.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.601+0000 7f299550e700 1 --2- 172.21.15.163:0/3552625044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29900aca80 0x7f29900b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:08.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.601+0000 7f2996510700 1 --2- 172.21.15.163:0/3552625044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29900c83c0 0x7f2990157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:08.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.602+0000 7f2995d0f700 1 -- 172.21.15.163:0/3552625044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29900aca80 msgr2=0x7f29900b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:08.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.602+0000 7f2995d0f700 1 --2- 172.21.15.163:0/3552625044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29900aca80 0x7f29900b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:08.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.602+0000 7f2995d0f700 1 -- 172.21.15.163:0/3552625044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29900c83c0 msgr2=0x7f2990157f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:08.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.602+0000 7f2995d0f700 1 --2- 172.21.15.163:0/3552625044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29900c83c0 0x7f2990157f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:08.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.602+0000 7f2995d0f700 1 -- 172.21.15.163:0/3552625044 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298c0093c0 con 0x7f2990069d60 2023-10-09T06:38:08.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.602+0000 7f2995d0f700 1 --2- 172.21.15.163:0/3552625044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2990069d60 0x7f29900c7dc0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f298c00ec80 tx=0x7f298c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:08.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.602+0000 7f2985ffb700 1 -- 172.21.15.163:0/3552625044 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298c005a00 con 0x7f2990069d60 2023-10-09T06:38:08.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.602+0000 7f2985ffb700 1 -- 172.21.15.163:0/3552625044 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f298c005b60 con 0x7f2990069d60 2023-10-09T06:38:08.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.602+0000 7f2997f73700 1 -- 172.21.15.163:0/3552625044 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2988009a20 con 0x7f2990069d60 2023-10-09T06:38:08.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.602+0000 7f2985ffb700 1 -- 172.21.15.163:0/3552625044 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298c00c850 con 0x7f2990069d60 2023-10-09T06:38:08.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.602+0000 7f2997f73700 1 -- 172.21.15.163:0/3552625044 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29900b1190 con 0x7f2990069d60 2023-10-09T06:38:08.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.603+0000 7f2997f73700 1 -- 172.21.15.163:0/3552625044 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f299004c3c0 con 0x7f2990069d60 2023-10-09T06:38:08.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.603+0000 7f2985ffb700 1 -- 172.21.15.163:0/3552625044 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f298c00cbc0 con 0x7f2990069d60 2023-10-09T06:38:08.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.604+0000 7f2985ffb700 1 --2- 172.21.15.163:0/3552625044 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2974067320 0x7f29740697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:08.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.604+0000 7f2985ffb700 1 -- 172.21.15.163:0/3552625044 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f298c085130 con 0x7f2990069d60 2023-10-09T06:38:08.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.604+0000 7f299550e700 1 --2- 172.21.15.163:0/3552625044 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2974067320 0x7f29740697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:08.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.604+0000 7f299550e700 1 --2- 172.21.15.163:0/3552625044 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2974067320 0x7f29740697d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f29900c8aa0 tx=0x7f2980007eb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:08.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.608+0000 7f2985ffb700 1 -- 172.21.15.163:0/3552625044 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f298c055800 con 0x7f2990069d60 2023-10-09T06:38:08.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.830+0000 7f2997f73700 1 -- 172.21.15.163:0/3552625044 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f299006b7f0 con 0x7f2990069d60 2023-10-09T06:38:08.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.831+0000 7f2985ffb700 1 -- 172.21.15.163:0/3552625044 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4630 (secure 0 0 0) 0x7f298c055390 con 0x7f2990069d60 2023-10-09T06:38:08.832 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:08.832 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":95,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1554520486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1554520486},{"type":"v1","addr":"172.21.15.181:6837","nonce":1554520486}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1695339871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1695339871},{"type":"v1","addr":"172.21.15.181:6836","nonce":1695339871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":7515,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3668692283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3668692283},{"type":"v1","addr":"172.21.15.163:6834","nonce":3668692283}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:37:46.529939+0000","modified":"2023-10-09T06:38:02.561321+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7518":{"gid":7518,"name":"c","rank":0,"incarnation":89,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/3043729326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3043729326},{"type":"v1","addr":"172.21.15.163:6835","nonce":3043729326}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"volume_0000000000063288","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-09T06:38:08.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.833+0000 7f2997f73700 1 -- 172.21.15.163:0/3552625044 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2974067320 msgr2=0x7f29740697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:08.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.833+0000 7f2997f73700 1 --2- 172.21.15.163:0/3552625044 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2974067320 0x7f29740697d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f29900c8aa0 tx=0x7f2980007eb0 comp rx=0 tx=0).stop 2023-10-09T06:38:08.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.833+0000 7f2997f73700 1 -- 172.21.15.163:0/3552625044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2990069d60 msgr2=0x7f29900c7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:08.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.833+0000 7f2997f73700 1 --2- 172.21.15.163:0/3552625044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2990069d60 0x7f29900c7dc0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f298c00ec80 tx=0x7f298c009120 comp rx=0 tx=0).stop 2023-10-09T06:38:08.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.833+0000 7f2997f73700 1 -- 172.21.15.163:0/3552625044 shutdown_connections 2023-10-09T06:38:08.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.833+0000 7f2997f73700 1 --2- 172.21.15.163:0/3552625044 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2974067320 0x7f29740697d0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:08.835 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.833+0000 7f2997f73700 1 --2- 172.21.15.163:0/3552625044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2990069d60 0x7f29900c7dc0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:08.835 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.833+0000 7f2997f73700 1 --2- 172.21.15.163:0/3552625044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29900c83c0 0x7f2990157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:08.835 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.833+0000 7f2997f73700 1 --2- 172.21.15.163:0/3552625044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29900aca80 0x7f29900b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:08.835 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.833+0000 7f2997f73700 1 -- 172.21.15.163:0/3552625044 >> 172.21.15.163:0/3552625044 conn(0x7f29900cd670 msgr2=0x7f2990158c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:08.835 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.833+0000 7f2997f73700 1 -- 172.21.15.163:0/3552625044 shutdown_connections 2023-10-09T06:38:08.835 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.833+0000 7f2997f73700 1 -- 172.21.15.163:0/3552625044 wait complete. 2023-10-09T06:38:08.835 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 95 2023-10-09T06:38:08.846 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:38:08.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.963+0000 7fe4a2cac700 1 Processor -- start 2023-10-09T06:38:08.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.963+0000 7fe4a2cac700 1 -- start start 2023-10-09T06:38:08.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.964+0000 7fe4a2cac700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe49c06bcd0 0x7fe49c140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:08.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.964+0000 7fe4a2cac700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe49c06c0a0 0x7fe49c141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:08.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.964+0000 7fe4a2cac700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe49c15c2a0 con 0x7fe49c0c8e00 2023-10-09T06:38:08.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.964+0000 7fe4a2cac700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe49c15b690 con 0x7fe49c06c0a0 2023-10-09T06:38:08.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.964+0000 7fe4a2cac700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c15b970 con 0x7fe49c06bcd0 2023-10-09T06:38:08.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.965+0000 7fe49bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe49c06c0a0 0x7fe49c141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:08.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.965+0000 7fe4a1249700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe49c0c8e00 0x7fe49c0c9260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:48672/0 (socket says 172.21.15.163:48672) 2023-10-09T06:38:08.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.965+0000 7fe4a1249700 1 -- 172.21.15.163:0/3873803428 learned_addr learned my addr 172.21.15.163:0/3873803428 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:08.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.965+0000 7fe4a0a48700 1 --2- 172.21.15.163:0/3873803428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe49c06bcd0 0x7fe49c140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:08.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.965+0000 7fe4a0a48700 1 -- 172.21.15.163:0/3873803428 >> v2:172.21.15.181:3300/0 conn(0x7fe49c06c0a0 msgr2=0x7fe49c141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:08.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.965+0000 7fe4a0a48700 1 --2- 172.21.15.163:0/3873803428 >> v2:172.21.15.181:3300/0 conn(0x7fe49c06c0a0 0x7fe49c141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:08.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.965+0000 7fe4a0a48700 1 -- 172.21.15.163:0/3873803428 >> v1:172.21.15.181:6789/0 conn(0x7fe49c0c8e00 legacy=0x7fe49c0c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:08.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.965+0000 7fe4a0a48700 1 -- 172.21.15.163:0/3873803428 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49c15bc50 con 0x7fe49c06bcd0 2023-10-09T06:38:08.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.965+0000 7fe49bfff700 1 --2- 172.21.15.163:0/3873803428 >> v2:172.21.15.181:3300/0 conn(0x7fe49c06c0a0 0x7fe49c141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:38:08.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.965+0000 7fe4a0a48700 1 --2- 172.21.15.163:0/3873803428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe49c06bcd0 0x7fe49c140df0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009e60 tx=0x7fe48c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87d368b9f1b5caa2 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:08.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.965+0000 7fe49b7fe700 1 -- 172.21.15.163:0/3873803428 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c00e070 con 0x7fe49c06bcd0 2023-10-09T06:38:08.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.965+0000 7fe49b7fe700 1 -- 172.21.15.163:0/3873803428 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe48c00bb60 con 0x7fe49c06bcd0 2023-10-09T06:38:08.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.965+0000 7fe49b7fe700 1 -- 172.21.15.163:0/3873803428 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c012430 con 0x7fe49c06bcd0 2023-10-09T06:38:08.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.966+0000 7fe4a2cac700 1 -- 172.21.15.163:0/3873803428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe49c06bcd0 msgr2=0x7fe49c140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:08.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.966+0000 7fe4a2cac700 1 --2- 172.21.15.163:0/3873803428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe49c06bcd0 0x7fe49c140df0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009e60 tx=0x7fe48c009780 comp rx=0 tx=0).stop 2023-10-09T06:38:08.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.966+0000 7fe4a2cac700 1 -- 172.21.15.163:0/3873803428 shutdown_connections 2023-10-09T06:38:08.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.966+0000 7fe4a2cac700 1 --2- 172.21.15.163:0/3873803428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe49c06bcd0 0x7fe49c140df0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:08.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.966+0000 7fe4a2cac700 1 --2- 172.21.15.163:0/3873803428 >> v2:172.21.15.181:3300/0 conn(0x7fe49c06c0a0 0x7fe49c141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:08.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.966+0000 7fe4a2cac700 1 -- 172.21.15.163:0/3873803428 >> 172.21.15.163:0/3873803428 conn(0x7fe49c0ce600 msgr2=0x7fe49c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:08.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.966+0000 7fe4a2cac700 1 -- 172.21.15.163:0/3873803428 shutdown_connections 2023-10-09T06:38:08.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.966+0000 7fe4a2cac700 1 -- 172.21.15.163:0/3873803428 wait complete. 2023-10-09T06:38:08.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.967+0000 7fe4a2cac700 1 Processor -- start 2023-10-09T06:38:08.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.967+0000 7fe4a2cac700 1 -- start start 2023-10-09T06:38:08.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.968+0000 7fe4a2cac700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe49c06bcd0 0x7fe49c0a7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:08.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.968+0000 7fe4a2cac700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe49c06c0a0 0x7fe49c0cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:08.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.968+0000 7fe4a2cac700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe49c0c8e00 0x7fe49c0cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:08.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.968+0000 7fe4a2cac700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c167ff0 con 0x7fe49c06c0a0 2023-10-09T06:38:08.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.968+0000 7fe4a2cac700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c168200 con 0x7fe49c06bcd0 2023-10-09T06:38:08.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.968+0000 7fe4a2cac700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c1684e0 con 0x7fe49c0c8e00 2023-10-09T06:38:08.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.968+0000 7fe49bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe49c06c0a0 0x7fe49c0cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:08.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.968+0000 7fe49bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe49c06c0a0 0x7fe49c0cd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59980/0 (socket says 172.21.15.163:59980) 2023-10-09T06:38:08.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.968+0000 7fe49bfff700 1 -- 172.21.15.163:0/2195814177 learned_addr learned my addr 172.21.15.163:0/2195814177 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:08.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.968+0000 7fe4a0a48700 1 --2- 172.21.15.163:0/2195814177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe49c06bcd0 0x7fe49c0a7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:08.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.968+0000 7fe4a1249700 1 --2- 172.21.15.163:0/2195814177 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe49c0c8e00 0x7fe49c0cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:08.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.968+0000 7fe49bfff700 1 -- 172.21.15.163:0/2195814177 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe49c0c8e00 msgr2=0x7fe49c0cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:08.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.968+0000 7fe49bfff700 1 --2- 172.21.15.163:0/2195814177 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe49c0c8e00 0x7fe49c0cd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:08.977 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.968+0000 7fe49bfff700 1 -- 172.21.15.163:0/2195814177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe49c06bcd0 msgr2=0x7fe49c0a7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:08.977 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.968+0000 7fe49bfff700 1 --2- 172.21.15.163:0/2195814177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe49c06bcd0 0x7fe49c0a7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:08.977 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.968+0000 7fe49bfff700 1 -- 172.21.15.163:0/2195814177 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe490009420 con 0x7fe49c06c0a0 2023-10-09T06:38:08.977 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.969+0000 7fe49bfff700 1 --2- 172.21.15.163:0/2195814177 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe49c06c0a0 0x7fe49c0cd290 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fe49000ec80 tx=0x7fe4900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:08.977 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.969+0000 7fe498ff9700 1 -- 172.21.15.163:0/2195814177 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49000c920 con 0x7fe49c06c0a0 2023-10-09T06:38:08.977 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.969+0000 7fe498ff9700 1 -- 172.21.15.163:0/2195814177 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe49000ca80 con 0x7fe49c06c0a0 2023-10-09T06:38:08.978 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.969+0000 7fe4a2cac700 1 -- 172.21.15.163:0/2195814177 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe48c009a20 con 0x7fe49c06c0a0 2023-10-09T06:38:08.978 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.969+0000 7fe498ff9700 1 -- 172.21.15.163:0/2195814177 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49000ce00 con 0x7fe49c06c0a0 2023-10-09T06:38:08.978 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.969+0000 7fe4a2cac700 1 -- 172.21.15.163:0/2195814177 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe49c168aa0 con 0x7fe49c06c0a0 2023-10-09T06:38:08.978 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.970+0000 7fe4a2cac700 1 -- 172.21.15.163:0/2195814177 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe49c168e10 con 0x7fe49c06c0a0 2023-10-09T06:38:08.978 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.970+0000 7fe498ff9700 1 -- 172.21.15.163:0/2195814177 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe490013680 con 0x7fe49c06c0a0 2023-10-09T06:38:08.978 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.970+0000 7fe498ff9700 1 --2- 172.21.15.163:0/2195814177 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe488067320 0x7fe4880697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:08.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.970+0000 7fe498ff9700 1 -- 172.21.15.163:0/2195814177 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5783+0+0 (secure 0 0 0) 0x7fe4900850a0 con 0x7fe49c06c0a0 2023-10-09T06:38:08.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.971+0000 7fe4a0a48700 1 --2- 172.21.15.163:0/2195814177 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe488067320 0x7fe4880697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:08.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.971+0000 7fe4a0a48700 1 --2- 172.21.15.163:0/2195814177 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe488067320 0x7fe4880697d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0097c0 tx=0x7fe48c019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:08.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:08.975+0000 7fe498ff9700 1 -- 172.21.15.163:0/2195814177 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe490051ce0 con 0x7fe49c06c0a0 2023-10-09T06:38:09.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.158+0000 7fe4a2cac700 1 -- 172.21.15.163:0/2195814177 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe49c04c3c0 con 0x7fe49c06c0a0 2023-10-09T06:38:09.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.159+0000 7fe498ff9700 1 -- 172.21.15.163:0/2195814177 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+15490 (secure 0 0 0) 0x7fe490055300 con 0x7fe49c06c0a0 2023-10-09T06:38:09.161 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:09.161 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":140,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:38:04.576334+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs.volume_0000000000063288.meta","create_time":"2023-10-09T06:37:44.861841+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000063288"}}},{"pool":25,"pool_name":"cephfs.volume_0000000000063288.data","create_time":"2023-10-09T06:37:45.496241+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000063288"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:38:09.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.161+0000 7fe4a2cac700 1 -- 172.21.15.163:0/2195814177 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe488067320 msgr2=0x7fe4880697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:09.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.161+0000 7fe4a2cac700 1 --2- 172.21.15.163:0/2195814177 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe488067320 0x7fe4880697d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0097c0 tx=0x7fe48c019000 comp rx=0 tx=0).stop 2023-10-09T06:38:09.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.161+0000 7fe4a2cac700 1 -- 172.21.15.163:0/2195814177 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe49c06c0a0 msgr2=0x7fe49c0cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:09.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.161+0000 7fe4a2cac700 1 --2- 172.21.15.163:0/2195814177 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe49c06c0a0 0x7fe49c0cd290 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fe49000ec80 tx=0x7fe4900091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:09.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.161+0000 7fe4a2cac700 1 -- 172.21.15.163:0/2195814177 shutdown_connections 2023-10-09T06:38:09.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.161+0000 7fe4a2cac700 1 --2- 172.21.15.163:0/2195814177 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe488067320 0x7fe4880697d0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:09.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.161+0000 7fe4a2cac700 1 --2- 172.21.15.163:0/2195814177 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe49c0c8e00 0x7fe49c0cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:09.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.161+0000 7fe4a2cac700 1 --2- 172.21.15.163:0/2195814177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe49c06bcd0 0x7fe49c0a7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:09.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.161+0000 7fe4a2cac700 1 --2- 172.21.15.163:0/2195814177 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe49c06c0a0 0x7fe49c0cd290 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:09.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.161+0000 7fe4a2cac700 1 -- 172.21.15.163:0/2195814177 >> 172.21.15.163:0/2195814177 conn(0x7fe49c0ce600 msgr2=0x7fe49c0c9260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:09.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.161+0000 7fe4a2cac700 1 -- 172.21.15.163:0/2195814177 shutdown_connections 2023-10-09T06:38:09.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.161+0000 7fe4a2cac700 1 -- 172.21.15.163:0/2195814177 wait complete. 2023-10-09T06:38:09.175 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail volume_0000000000063288 2023-10-09T06:38:09.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.294+0000 7f0f92c57700 1 Processor -- start 2023-10-09T06:38:09.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.294+0000 7f0f92c57700 1 -- start start 2023-10-09T06:38:09.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.294+0000 7f0f92c57700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0f8c069560 0x7f0f8c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:09.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.294+0000 7f0f92c57700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f8c069ee0 con 0x7f0f8c159890 2023-10-09T06:38:09.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.294+0000 7f0f92c57700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f8c15a160 con 0x7f0f8c159c60 2023-10-09T06:38:09.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.294+0000 7f0f92c57700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f8c0d13a0 con 0x7f0f8c069560 2023-10-09T06:38:09.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.295+0000 7f0f909f3700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f0f8c159890 0x7f0f8c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:51888/0 (socket says 172.21.15.163:51888) 2023-10-09T06:38:09.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.295+0000 7f0f909f3700 1 -- 172.21.15.163:0/722034621 learned_addr learned my addr 172.21.15.163:0/722034621 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:09.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.295+0000 7f0f837fe700 1 -- 172.21.15.163:0/722034621 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2430509561 0 0) 0x7f0f8c069ee0 con 0x7f0f8c159890 2023-10-09T06:38:09.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.295+0000 7f0f837fe700 1 -- 172.21.15.163:0/722034621 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f74003570 con 0x7f0f8c159890 2023-10-09T06:38:09.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.295+0000 7f0f911f4700 1 --2- 172.21.15.163:0/722034621 >> v2:172.21.15.181:3300/0 conn(0x7f0f8c069560 0x7f0f8c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:09.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.295+0000 7f0f837fe700 1 -- 172.21.15.163:0/722034621 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3658865260 0 0) 0x7f0f74003570 con 0x7f0f8c159890 2023-10-09T06:38:09.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.295+0000 7f0f837fe700 1 -- 172.21.15.163:0/722034621 >> v2:172.21.15.181:3300/0 conn(0x7f0f8c069560 msgr2=0x7f0f8c0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:09.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.295+0000 7f0f837fe700 1 --2- 172.21.15.163:0/722034621 >> v2:172.21.15.181:3300/0 conn(0x7f0f8c069560 0x7f0f8c0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:09.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.295+0000 7f0f837fe700 1 -- 172.21.15.163:0/722034621 >> v1:172.21.15.181:6789/0 conn(0x7f0f8c159c60 legacy=0x7f0f8c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:09.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.295+0000 7f0f837fe700 1 -- 172.21.15.163:0/722034621 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f8c15e140 con 0x7f0f8c159890 2023-10-09T06:38:09.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.295+0000 7f0f837fe700 1 -- 172.21.15.163:0/722034621 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f0f88001470 con 0x7f0f8c159890 2023-10-09T06:38:09.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.295+0000 7f0f837fe700 1 -- 172.21.15.163:0/722034621 >> v1:172.21.15.163:6789/0 conn(0x7f0f8c159890 legacy=0x7f0f8c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:09.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.295+0000 7f0f837fe700 1 --2- 172.21.15.163:0/722034621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f74003a60 0x7f0f74003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:09.310 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.295+0000 7f0f837fe700 1 --2- 172.21.15.163:0/722034621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f74004810 0x7f0f74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:09.310 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.295+0000 7f0f837fe700 1 --2- 172.21.15.163:0/722034621 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f740051c0 0x7f0f7400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:09.310 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.295+0000 7f0f837fe700 1 -- 172.21.15.163:0/722034621 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c0d13a0 con 0x7f0f74003a60 2023-10-09T06:38:09.310 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.295+0000 7f0f837fe700 1 -- 172.21.15.163:0/722034621 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7400c0d0 con 0x7f0f74004810 2023-10-09T06:38:09.310 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.295+0000 7f0f837fe700 1 -- 172.21.15.163:0/722034621 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7400c290 con 0x7f0f740051c0 2023-10-09T06:38:09.310 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.295+0000 7f0f909f3700 1 --2- 172.21.15.163:0/722034621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f74003a60 0x7f0f74003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:09.310 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.295+0000 7f0f83fff700 1 --2- 172.21.15.163:0/722034621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f74004810 0x7f0f74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:09.310 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.295+0000 7f0f911f4700 1 --2- 172.21.15.163:0/722034621 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f740051c0 0x7f0f7400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:09.310 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.296+0000 7f0f909f3700 1 -- 172.21.15.163:0/722034621 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f740051c0 msgr2=0x7f0f7400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:09.310 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.296+0000 7f0f909f3700 1 --2- 172.21.15.163:0/722034621 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f740051c0 0x7f0f7400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:09.310 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.296+0000 7f0f909f3700 1 -- 172.21.15.163:0/722034621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f74004810 msgr2=0x7f0f74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:09.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.296+0000 7f0f909f3700 1 --2- 172.21.15.163:0/722034621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f74004810 0x7f0f74004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:09.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.296+0000 7f0f909f3700 1 -- 172.21.15.163:0/722034621 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f74003570 con 0x7f0f74003a60 2023-10-09T06:38:09.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.296+0000 7f0f911f4700 1 -- 172.21.15.163:0/722034621 reap_dead start 2023-10-09T06:38:09.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.296+0000 7f0f83fff700 1 --2- 172.21.15.163:0/722034621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f74004810 0x7f0f74004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:09.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.296+0000 7f0f909f3700 1 --2- 172.21.15.163:0/722034621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f74003a60 0x7f0f74003e70 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800c3c0 tx=0x7f0f88001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97eb0bddb1066f8c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:09.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.296+0000 7f0f837fe700 1 -- 172.21.15.163:0/722034621 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8800cc90 con 0x7f0f74003a60 2023-10-09T06:38:09.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.296+0000 7f0f837fe700 1 -- 172.21.15.163:0/722034621 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8800cdf0 con 0x7f0f74003a60 2023-10-09T06:38:09.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.296+0000 7f0f837fe700 1 -- 172.21.15.163:0/722034621 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f88005b50 con 0x7f0f74003a60 2023-10-09T06:38:09.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.296+0000 7f0f92c57700 1 -- 172.21.15.163:0/722034621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f74003a60 msgr2=0x7f0f74003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:09.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.296+0000 7f0f92c57700 1 --2- 172.21.15.163:0/722034621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f74003a60 0x7f0f74003e70 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800c3c0 tx=0x7f0f88001e50 comp rx=0 tx=0).stop 2023-10-09T06:38:09.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.296+0000 7f0f92c57700 1 -- 172.21.15.163:0/722034621 shutdown_connections 2023-10-09T06:38:09.311 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.296+0000 7f0f92c57700 1 --2- 172.21.15.163:0/722034621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f74003a60 0x7f0f74003e70 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:09.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.296+0000 7f0f92c57700 1 -- 172.21.15.163:0/722034621 >> 172.21.15.163:0/722034621 conn(0x7f0f8c068e20 msgr2=0x7f0f8c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:09.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.296+0000 7f0f92c57700 1 -- 172.21.15.163:0/722034621 shutdown_connections 2023-10-09T06:38:09.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.296+0000 7f0f92c57700 1 -- 172.21.15.163:0/722034621 wait complete. 2023-10-09T06:38:09.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.298+0000 7f0f92c57700 1 Processor -- start 2023-10-09T06:38:09.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.298+0000 7f0f92c57700 1 -- start start 2023-10-09T06:38:09.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.298+0000 7f0f92c57700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f74003a60 0x7f0f8c152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:09.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.298+0000 7f0f92c57700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f8c0cdc00 0x7f0f8c0ce070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:09.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.298+0000 7f0f92c57700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f8c15cdb0 0x7f0f8c15d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:09.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.298+0000 7f0f92c57700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c15d760 con 0x7f0f74003a60 2023-10-09T06:38:09.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.298+0000 7f0f92c57700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c152fa0 con 0x7f0f8c0cdc00 2023-10-09T06:38:09.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.298+0000 7f0f92c57700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c0ce6a0 con 0x7f0f8c15cdb0 2023-10-09T06:38:09.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.299+0000 7f0f909f3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f74003a60 0x7f0f8c152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:09.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.299+0000 7f0f909f3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f74003a60 0x7f0f8c152a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:59998/0 (socket says 172.21.15.163:59998) 2023-10-09T06:38:09.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.299+0000 7f0f909f3700 1 -- 172.21.15.163:0/3990452389 learned_addr learned my addr 172.21.15.163:0/3990452389 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:09.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.299+0000 7f0f83fff700 1 --2- 172.21.15.163:0/3990452389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f8c0cdc00 0x7f0f8c0ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:09.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.299+0000 7f0f911f4700 1 --2- 172.21.15.163:0/3990452389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f8c15cdb0 0x7f0f8c15d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:09.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.299+0000 7f0f909f3700 1 -- 172.21.15.163:0/3990452389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f8c15cdb0 msgr2=0x7f0f8c15d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:09.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.299+0000 7f0f909f3700 1 --2- 172.21.15.163:0/3990452389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f8c15cdb0 0x7f0f8c15d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:09.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.299+0000 7f0f909f3700 1 -- 172.21.15.163:0/3990452389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f8c0cdc00 msgr2=0x7f0f8c0ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:09.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.299+0000 7f0f909f3700 1 --2- 172.21.15.163:0/3990452389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f8c0cdc00 0x7f0f8c0ce070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:09.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.299+0000 7f0f909f3700 1 -- 172.21.15.163:0/3990452389 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f8c0c7940 con 0x7f0f74003a60 2023-10-09T06:38:09.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.299+0000 7f0f911f4700 1 --2- 172.21.15.163:0/3990452389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f8c15cdb0 0x7f0f8c15d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:09.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.299+0000 7f0f909f3700 1 --2- 172.21.15.163:0/3990452389 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f74003a60 0x7f0f8c152a60 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f0f7400c0a0 tx=0x7f0f88015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:09.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.299+0000 7f0f80ff9700 1 -- 172.21.15.163:0/3990452389 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f88001430 con 0x7f0f74003a60 2023-10-09T06:38:09.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.299+0000 7f0f80ff9700 1 -- 172.21.15.163:0/3990452389 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f88001650 con 0x7f0f74003a60 2023-10-09T06:38:09.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.299+0000 7f0f92c57700 1 -- 172.21.15.163:0/3990452389 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f8c0c7c20 con 0x7f0f74003a60 2023-10-09T06:38:09.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.299+0000 7f0f80ff9700 1 -- 172.21.15.163:0/3990452389 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f88007320 con 0x7f0f74003a60 2023-10-09T06:38:09.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.299+0000 7f0f92c57700 1 -- 172.21.15.163:0/3990452389 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8c0c81e0 con 0x7f0f74003a60 2023-10-09T06:38:09.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.300+0000 7f0f92c57700 1 -- 172.21.15.163:0/3990452389 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f8c04c3c0 con 0x7f0f74003a60 2023-10-09T06:38:09.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.300+0000 7f0f80ff9700 1 -- 172.21.15.163:0/3990452389 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0f88007690 con 0x7f0f74003a60 2023-10-09T06:38:09.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.301+0000 7f0f80ff9700 1 --2- 172.21.15.163:0/3990452389 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0f78067320 0x7f0f780697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:09.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.301+0000 7f0f80ff9700 1 -- 172.21.15.163:0/3990452389 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f0f88085b60 con 0x7f0f74003a60 2023-10-09T06:38:09.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.301+0000 7f0f83fff700 1 --2- 172.21.15.163:0/3990452389 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0f78067320 0x7f0f780697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:09.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.301+0000 7f0f83fff700 1 --2- 172.21.15.163:0/3990452389 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0f78067320 0x7f0f780697d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c007bc0 tx=0x7f0f7c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:09.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.305+0000 7f0f80ff9700 1 -- 172.21.15.163:0/3990452389 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0f880527a0 con 0x7f0f74003a60 2023-10-09T06:38:09.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.527+0000 7f0f92c57700 1 -- 172.21.15.163:0/3990452389 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "volume_0000000000063288"} v 0) v1 -- 0x7f0f8c06a5b0 con 0x7f0f74003a60 2023-10-09T06:38:09.529 INFO:tasks.ceph.mon.a.smithi163.stderr:2023-10-09T06:38:09.528+0000 7ff251c04700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-09T06:38:09.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.532+0000 7f0f80ff9700 1 -- 172.21.15.163:0/3990452389 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "volume_0000000000063288"}]=0 volume_0000000000063288 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v96) v1 ==== 195+0+0 (secure 0 0 0) 0x7f0f88055dc0 con 0x7f0f74003a60 2023-10-09T06:38:09.533 INFO:tasks.ceph.mds.c.smithi163.stderr: -847> 2023-10-09T06:37:52.436+0000 7ff249cc3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-09T06:38:09.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.534+0000 7f0f92c57700 1 -- 172.21.15.163:0/3990452389 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0f78067320 msgr2=0x7f0f780697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:09.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.535+0000 7f0f92c57700 1 --2- 172.21.15.163:0/3990452389 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0f78067320 0x7f0f780697d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c007bc0 tx=0x7f0f7c009040 comp rx=0 tx=0).stop 2023-10-09T06:38:09.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.535+0000 7f0f92c57700 1 -- 172.21.15.163:0/3990452389 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f74003a60 msgr2=0x7f0f8c152a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:09.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.535+0000 7f0f92c57700 1 --2- 172.21.15.163:0/3990452389 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f74003a60 0x7f0f8c152a60 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f0f7400c0a0 tx=0x7f0f88015000 comp rx=0 tx=0).stop 2023-10-09T06:38:09.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.535+0000 7f0f92c57700 1 -- 172.21.15.163:0/3990452389 shutdown_connections 2023-10-09T06:38:09.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.535+0000 7f0f92c57700 1 --2- 172.21.15.163:0/3990452389 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0f78067320 0x7f0f780697d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:09.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.535+0000 7f0f92c57700 1 --2- 172.21.15.163:0/3990452389 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f74003a60 0x7f0f8c152a60 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:09.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.535+0000 7f0f92c57700 1 --2- 172.21.15.163:0/3990452389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f8c15cdb0 0x7f0f8c15d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:09.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.535+0000 7f0f92c57700 1 --2- 172.21.15.163:0/3990452389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f8c0cdc00 0x7f0f8c0ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:09.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.535+0000 7f0f92c57700 1 -- 172.21.15.163:0/3990452389 >> 172.21.15.163:0/3990452389 conn(0x7f0f8c068e20 msgr2=0x7f0f8c0b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:09.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.535+0000 7f0f92c57700 1 -- 172.21.15.163:0/3990452389 shutdown_connections 2023-10-09T06:38:09.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.535+0000 7f0f92c57700 1 -- 172.21.15.163:0/3990452389 wait complete. 2023-10-09T06:38:09.537 INFO:teuthology.orchestra.run.smithi163.stderr:volume_0000000000063288 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-09T06:38:09.548 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm volume_0000000000063288 --yes-i-really-mean-it 2023-10-09T06:38:09.553 INFO:tasks.ceph.mds.c.smithi163.stdout:starting mds.c at 2023-10-09T06:38:09.669 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.669+0000 7f75b6965700 1 Processor -- start 2023-10-09T06:38:09.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.669+0000 7f75b6965700 1 -- start start 2023-10-09T06:38:09.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.669+0000 7f75b6965700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75b00af480 0x7f75b00ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:09.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.669+0000 7f75b6965700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f75b0156a60 0x7f75b00694a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:09.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.669+0000 7f75b6965700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b0157e30 con 0x7f75b0156690 2023-10-09T06:38:09.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.669+0000 7f75b6965700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b006aa00 con 0x7f75b0156a60 2023-10-09T06:38:09.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.669+0000 7f75b6965700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b00699e0 con 0x7f75b00af480 2023-10-09T06:38:09.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.669+0000 7f75b4f02700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f75b0156a60 0x7f75b00694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:09.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.670+0000 7f75b4f02700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f75b0156a60 0x7f75b00694a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:60018/0 (socket says 172.21.15.163:60018) 2023-10-09T06:38:09.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.670+0000 7f75b4f02700 1 -- 172.21.15.163:0/2748141793 learned_addr learned my addr 172.21.15.163:0/2748141793 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:09.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.670+0000 7f75af7fe700 1 --2- 172.21.15.163:0/2748141793 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75b00af480 0x7f75b00ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:09.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.670+0000 7f75b4f02700 1 -- 172.21.15.163:0/2748141793 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75b00af480 msgr2=0x7f75b00ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:09.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.670+0000 7f75b4f02700 1 --2- 172.21.15.163:0/2748141793 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75b00af480 0x7f75b00ca9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:09.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.670+0000 7f75b4f02700 1 -- 172.21.15.163:0/2748141793 >> v1:172.21.15.181:6789/0 conn(0x7f75b0156690 legacy=0x7f75b00c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:09.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.670+0000 7f75b4f02700 1 -- 172.21.15.163:0/2748141793 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b01578b0 con 0x7f75b0156a60 2023-10-09T06:38:09.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.670+0000 7f75b4f02700 1 --2- 172.21.15.163:0/2748141793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75b0156a60 0x7f75b00694a0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f75a800d1a0 tx=0x7f75a800ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967208485735e452 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:09.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.670+0000 7f75aeffd700 1 -- 172.21.15.163:0/2748141793 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a8010070 con 0x7f75b0156a60 2023-10-09T06:38:09.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.670+0000 7f75aeffd700 1 -- 172.21.15.163:0/2748141793 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a800de80 con 0x7f75b0156a60 2023-10-09T06:38:09.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.670+0000 7f75aeffd700 1 -- 172.21.15.163:0/2748141793 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a800b320 con 0x7f75b0156a60 2023-10-09T06:38:09.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.671+0000 7f75b6965700 1 -- 172.21.15.163:0/2748141793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75b0156a60 msgr2=0x7f75b00694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:09.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.671+0000 7f75b6965700 1 --2- 172.21.15.163:0/2748141793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75b0156a60 0x7f75b00694a0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f75a800d1a0 tx=0x7f75a800ba50 comp rx=0 tx=0).stop 2023-10-09T06:38:09.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.671+0000 7f75b6965700 1 -- 172.21.15.163:0/2748141793 shutdown_connections 2023-10-09T06:38:09.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.671+0000 7f75b6965700 1 --2- 172.21.15.163:0/2748141793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75b0156a60 0x7f75b00694a0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:09.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.671+0000 7f75b6965700 1 --2- 172.21.15.163:0/2748141793 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75b00af480 0x7f75b00ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:09.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.671+0000 7f75b6965700 1 -- 172.21.15.163:0/2748141793 >> 172.21.15.163:0/2748141793 conn(0x7f75b00c9870 msgr2=0x7f75b00c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:09.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.671+0000 7f75b6965700 1 -- 172.21.15.163:0/2748141793 shutdown_connections 2023-10-09T06:38:09.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.671+0000 7f75b6965700 1 -- 172.21.15.163:0/2748141793 wait complete. 2023-10-09T06:38:09.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.672+0000 7f75b6965700 1 Processor -- start 2023-10-09T06:38:09.677 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.672+0000 7f75b6965700 1 -- start start 2023-10-09T06:38:09.677 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.673+0000 7f75b6965700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75b00af480 0x7f75b00c3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:09.677 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.673+0000 7f75b6965700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75b0156690 0x7f75b01443a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:09.677 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.673+0000 7f75b6965700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75b0156a60 0x7f75b01448e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:09.678 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.673+0000 7f75b6965700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b0169510 con 0x7f75b0156690 2023-10-09T06:38:09.678 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.673+0000 7f75b6965700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b0144e20 con 0x7f75b0156a60 2023-10-09T06:38:09.678 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.673+0000 7f75b6965700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b0169760 con 0x7f75b00af480 2023-10-09T06:38:09.678 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.673+0000 7f75af7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75b0156690 0x7f75b01443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:09.678 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.673+0000 7f75af7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75b0156690 0x7f75b01443a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:60032/0 (socket says 172.21.15.163:60032) 2023-10-09T06:38:09.678 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.673+0000 7f75af7fe700 1 -- 172.21.15.163:0/3587554717 learned_addr learned my addr 172.21.15.163:0/3587554717 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:09.678 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.673+0000 7f75affff700 1 --2- 172.21.15.163:0/3587554717 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75b00af480 0x7f75b00c3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:09.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.673+0000 7f75b4f02700 1 --2- 172.21.15.163:0/3587554717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75b0156a60 0x7f75b01448e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:09.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.673+0000 7f75af7fe700 1 -- 172.21.15.163:0/3587554717 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75b00af480 msgr2=0x7f75b00c3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:09.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.673+0000 7f75af7fe700 1 --2- 172.21.15.163:0/3587554717 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75b00af480 0x7f75b00c3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:09.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.673+0000 7f75af7fe700 1 -- 172.21.15.163:0/3587554717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75b0156a60 msgr2=0x7f75b01448e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:09.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.673+0000 7f75af7fe700 1 --2- 172.21.15.163:0/3587554717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75b0156a60 0x7f75b01448e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:09.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.673+0000 7f75af7fe700 1 -- 172.21.15.163:0/3587554717 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a800bcf0 con 0x7f75b0156690 2023-10-09T06:38:09.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.673+0000 7f75af7fe700 1 --2- 172.21.15.163:0/3587554717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75b0156690 0x7f75b01443a0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f75a400c8b0 tx=0x7f75a4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:09.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.673+0000 7f7597fff700 1 -- 172.21.15.163:0/3587554717 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a4011600 con 0x7f75b0156690 2023-10-09T06:38:09.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.673+0000 7f7597fff700 1 -- 172.21.15.163:0/3587554717 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a4011780 con 0x7f75b0156690 2023-10-09T06:38:09.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.673+0000 7f75b6965700 1 -- 172.21.15.163:0/3587554717 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b0169a40 con 0x7f75b0156690 2023-10-09T06:38:09.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.673+0000 7f7597fff700 1 -- 172.21.15.163:0/3587554717 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a4011b40 con 0x7f75b0156690 2023-10-09T06:38:09.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.673+0000 7f75b6965700 1 -- 172.21.15.163:0/3587554717 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b016a000 con 0x7f75b0156690 2023-10-09T06:38:09.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.674+0000 7f75b6965700 1 -- 172.21.15.163:0/3587554717 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b0156fb0 con 0x7f75b0156690 2023-10-09T06:38:09.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.675+0000 7f7597fff700 1 -- 172.21.15.163:0/3587554717 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f75a4019400 con 0x7f75b0156690 2023-10-09T06:38:09.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.675+0000 7f7597fff700 1 --2- 172.21.15.163:0/3587554717 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f759c067320 0x7f759c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:09.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.675+0000 7f7597fff700 1 -- 172.21.15.163:0/3587554717 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5869+0+0 (secure 0 0 0) 0x7f75a40849a0 con 0x7f75b0156690 2023-10-09T06:38:09.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.675+0000 7f75affff700 1 --2- 172.21.15.163:0/3587554717 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f759c067320 0x7f759c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:09.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.676+0000 7f75affff700 1 --2- 172.21.15.163:0/3587554717 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f759c067320 0x7f759c0697d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f75a0003a30 tx=0x7f75a0007ea0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:09.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.679+0000 7f7597fff700 1 -- 172.21.15.163:0/3587554717 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f75a4051480 con 0x7f75b0156690 2023-10-09T06:38:09.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:09.900+0000 7f75b6965700 1 -- 172.21.15.163:0/3587554717 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "volume_0000000000063288", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f75b016a220 con 0x7f75b0156690 2023-10-09T06:38:10.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.532+0000 7f7597fff700 1 -- 172.21.15.163:0/3587554717 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "volume_0000000000063288", "yes_i_really_mean_it": true}]=0 v97) v1 ==== 121+0+0 (secure 0 0 0) 0x7f75a40150c0 con 0x7f75b0156690 2023-10-09T06:38:10.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.535+0000 7f75b6965700 1 -- 172.21.15.163:0/3587554717 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f759c067320 msgr2=0x7f759c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:10.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.535+0000 7f75b6965700 1 --2- 172.21.15.163:0/3587554717 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f759c067320 0x7f759c0697d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f75a0003a30 tx=0x7f75a0007ea0 comp rx=0 tx=0).stop 2023-10-09T06:38:10.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.535+0000 7f75b6965700 1 -- 172.21.15.163:0/3587554717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75b0156690 msgr2=0x7f75b01443a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:10.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.535+0000 7f75b6965700 1 --2- 172.21.15.163:0/3587554717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75b0156690 0x7f75b01443a0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f75a400c8b0 tx=0x7f75a4008040 comp rx=0 tx=0).stop 2023-10-09T06:38:10.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.535+0000 7f75b6965700 1 -- 172.21.15.163:0/3587554717 shutdown_connections 2023-10-09T06:38:10.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.535+0000 7f75b6965700 1 --2- 172.21.15.163:0/3587554717 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f759c067320 0x7f759c0697d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:10.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.535+0000 7f75b6965700 1 --2- 172.21.15.163:0/3587554717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75b0156a60 0x7f75b01448e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:10.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.535+0000 7f75b6965700 1 --2- 172.21.15.163:0/3587554717 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75b00af480 0x7f75b00c3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:10.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.535+0000 7f75b6965700 1 --2- 172.21.15.163:0/3587554717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75b0156690 0x7f75b01443a0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:10.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.535+0000 7f75b6965700 1 -- 172.21.15.163:0/3587554717 >> 172.21.15.163:0/3587554717 conn(0x7f75b00c9870 msgr2=0x7f75b0155860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:10.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.535+0000 7f75b6965700 1 -- 172.21.15.163:0/3587554717 shutdown_connections 2023-10-09T06:38:10.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.535+0000 7f75b6965700 1 -- 172.21.15.163:0/3587554717 wait complete. 2023-10-09T06:38:10.549 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000063288.meta cephfs.volume_0000000000063288.meta --yes-i-really-really-mean-it 2023-10-09T06:38:10.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.671+0000 7fe7118b6700 1 Processor -- start 2023-10-09T06:38:10.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.671+0000 7fe7118b6700 1 -- start start 2023-10-09T06:38:10.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.672+0000 7fe7118b6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe70c155360 0x7fe70c0b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:10.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.672+0000 7fe7118b6700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe70c0ced30 0x7fe70c0cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:10.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.672+0000 7fe7118b6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe70c0b4560 con 0x7fe70c0b40f0 2023-10-09T06:38:10.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.672+0000 7fe7118b6700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe70c15c320 con 0x7fe70c0ced30 2023-10-09T06:38:10.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.672+0000 7fe7118b6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c15b710 con 0x7fe70c155360 2023-10-09T06:38:10.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.672+0000 7fe70b7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe70c0ced30 0x7fe70c0cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:10.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.672+0000 7fe70b7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe70c0ced30 0x7fe70c0cc9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:60042/0 (socket says 172.21.15.163:60042) 2023-10-09T06:38:10.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.672+0000 7fe70b7fe700 1 -- 172.21.15.163:0/570456894 learned_addr learned my addr 172.21.15.163:0/570456894 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:10.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.672+0000 7fe70affd700 1 --2- 172.21.15.163:0/570456894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe70c155360 0x7fe70c0b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:10.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.673+0000 7fe70b7fe700 1 -- 172.21.15.163:0/570456894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe70c155360 msgr2=0x7fe70c0b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:10.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.673+0000 7fe70b7fe700 1 --2- 172.21.15.163:0/570456894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe70c155360 0x7fe70c0b3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:10.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.673+0000 7fe70b7fe700 1 -- 172.21.15.163:0/570456894 >> v1:172.21.15.181:6789/0 conn(0x7fe70c0b40f0 legacy=0x7fe70c0ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:10.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.673+0000 7fe70b7fe700 1 -- 172.21.15.163:0/570456894 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70c15b9f0 con 0x7fe70c0ced30 2023-10-09T06:38:10.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.673+0000 7fe70b7fe700 1 --2- 172.21.15.163:0/570456894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe70c0ced30 0x7fe70c0cc9c0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fe704009f20 tx=0x7fe704009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0590af50a7094c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:10.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.673+0000 7fe70a7fc700 1 -- 172.21.15.163:0/570456894 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70400e070 con 0x7fe70c0ced30 2023-10-09T06:38:10.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.673+0000 7fe70a7fc700 1 -- 172.21.15.163:0/570456894 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70400ba80 con 0x7fe70c0ced30 2023-10-09T06:38:10.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.673+0000 7fe70a7fc700 1 -- 172.21.15.163:0/570456894 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7040092e0 con 0x7fe70c0ced30 2023-10-09T06:38:10.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.673+0000 7fe7118b6700 1 -- 172.21.15.163:0/570456894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe70c0ced30 msgr2=0x7fe70c0cc9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:10.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.673+0000 7fe7118b6700 1 --2- 172.21.15.163:0/570456894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe70c0ced30 0x7fe70c0cc9c0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fe704009f20 tx=0x7fe704009840 comp rx=0 tx=0).stop 2023-10-09T06:38:10.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.673+0000 7fe7118b6700 1 -- 172.21.15.163:0/570456894 shutdown_connections 2023-10-09T06:38:10.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.673+0000 7fe7118b6700 1 --2- 172.21.15.163:0/570456894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe70c0ced30 0x7fe70c0cc9c0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:10.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.673+0000 7fe7118b6700 1 --2- 172.21.15.163:0/570456894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe70c155360 0x7fe70c0b3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:10.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.673+0000 7fe7118b6700 1 -- 172.21.15.163:0/570456894 >> 172.21.15.163:0/570456894 conn(0x7fe70c06baf0 msgr2=0x7fe70c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:10.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.673+0000 7fe7118b6700 1 -- 172.21.15.163:0/570456894 shutdown_connections 2023-10-09T06:38:10.677 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.673+0000 7fe7118b6700 1 -- 172.21.15.163:0/570456894 wait complete. 2023-10-09T06:38:10.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.675+0000 7fe7118b6700 1 Processor -- start 2023-10-09T06:38:10.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.675+0000 7fe7118b6700 1 -- start start 2023-10-09T06:38:10.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.675+0000 7fe7118b6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe70c0b40f0 0x7fe70c151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:10.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.675+0000 7fe7118b6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe70c0ced30 0x7fe70c16b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:10.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.675+0000 7fe7118b6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe70c155360 0x7fe70c16b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:10.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.675+0000 7fe7118b6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c0b0f30 con 0x7fe70c155360 2023-10-09T06:38:10.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.675+0000 7fe7118b6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c14de60 con 0x7fe70c0b40f0 2023-10-09T06:38:10.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.675+0000 7fe7118b6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c16c2e0 con 0x7fe70c0ced30 2023-10-09T06:38:10.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.676+0000 7fe70b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe70c155360 0x7fe70c16b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:10.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.676+0000 7fe70b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe70c155360 0x7fe70c16b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:60054/0 (socket says 172.21.15.163:60054) 2023-10-09T06:38:10.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.676+0000 7fe70b7fe700 1 -- 172.21.15.163:0/834941688 learned_addr learned my addr 172.21.15.163:0/834941688 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:10.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.676+0000 7fe70affd700 1 --2- 172.21.15.163:0/834941688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe70c0b40f0 0x7fe70c151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:10.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.676+0000 7fe7027fc700 1 --2- 172.21.15.163:0/834941688 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe70c0ced30 0x7fe70c16b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:10.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.676+0000 7fe70b7fe700 1 -- 172.21.15.163:0/834941688 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe70c0ced30 msgr2=0x7fe70c16b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:10.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.676+0000 7fe70b7fe700 1 --2- 172.21.15.163:0/834941688 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe70c0ced30 0x7fe70c16b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:10.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.676+0000 7fe70b7fe700 1 -- 172.21.15.163:0/834941688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe70c0b40f0 msgr2=0x7fe70c151ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:10.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.676+0000 7fe70b7fe700 1 --2- 172.21.15.163:0/834941688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe70c0b40f0 0x7fe70c151ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:10.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.676+0000 7fe70b7fe700 1 -- 172.21.15.163:0/834941688 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe704009ae0 con 0x7fe70c155360 2023-10-09T06:38:10.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.676+0000 7fe70b7fe700 1 --2- 172.21.15.163:0/834941688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe70c155360 0x7fe70c16b990 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fe70c06f610 tx=0x7fe704018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:10.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.676+0000 7fe7037fe700 1 -- 172.21.15.163:0/834941688 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7040032f0 con 0x7fe70c155360 2023-10-09T06:38:10.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.676+0000 7fe7037fe700 1 -- 172.21.15.163:0/834941688 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe704008d20 con 0x7fe70c155360 2023-10-09T06:38:10.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.676+0000 7fe7118b6700 1 -- 172.21.15.163:0/834941688 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe70c152020 con 0x7fe70c155360 2023-10-09T06:38:10.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.676+0000 7fe7037fe700 1 -- 172.21.15.163:0/834941688 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe704026690 con 0x7fe70c155360 2023-10-09T06:38:10.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.676+0000 7fe7118b6700 1 -- 172.21.15.163:0/834941688 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe70c14d530 con 0x7fe70c155360 2023-10-09T06:38:10.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.677+0000 7fe7118b6700 1 -- 172.21.15.163:0/834941688 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe70c14e010 con 0x7fe70c155360 2023-10-09T06:38:10.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.678+0000 7fe7037fe700 1 -- 172.21.15.163:0/834941688 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe704003040 con 0x7fe70c155360 2023-10-09T06:38:10.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.678+0000 7fe7037fe700 1 --2- 172.21.15.163:0/834941688 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe6ec067320 0x7fe6ec0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:10.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.678+0000 7fe7037fe700 1 -- 172.21.15.163:0/834941688 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5869+0+0 (secure 0 0 0) 0x7fe704086d70 con 0x7fe70c155360 2023-10-09T06:38:10.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.678+0000 7fe70affd700 1 --2- 172.21.15.163:0/834941688 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe6ec067320 0x7fe6ec0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:10.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.679+0000 7fe70affd700 1 --2- 172.21.15.163:0/834941688 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe6ec067320 0x7fe6ec0697d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc006fd0 tx=0x7fe6fc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:10.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.683+0000 7fe7037fe700 1 -- 172.21.15.163:0/834941688 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe70c14e010 con 0x7fe70c155360 2023-10-09T06:38:10.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:10.864+0000 7fe7118b6700 1 -- 172.21.15.163:0/834941688 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000063288.meta", "pool2": "cephfs.volume_0000000000063288.meta", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe70c14dfd0 con 0x7fe70c155360 2023-10-09T06:38:11.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.610+0000 7fe7037fe700 1 -- 172.21.15.163:0/834941688 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000063288.meta", "pool2": "cephfs.volume_0000000000063288.meta", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000063288.meta' removed v142) v1 ==== 241+0+0 (secure 0 0 0) 0x7fe704053850 con 0x7fe70c155360 2023-10-09T06:38:11.614 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.613+0000 7fe7118b6700 1 -- 172.21.15.163:0/834941688 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe6ec067320 msgr2=0x7fe6ec0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:11.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.613+0000 7fe7118b6700 1 --2- 172.21.15.163:0/834941688 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe6ec067320 0x7fe6ec0697d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc006fd0 tx=0x7fe6fc008040 comp rx=0 tx=0).stop 2023-10-09T06:38:11.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.613+0000 7fe7118b6700 1 -- 172.21.15.163:0/834941688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe70c155360 msgr2=0x7fe70c16b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:11.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.613+0000 7fe7118b6700 1 --2- 172.21.15.163:0/834941688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe70c155360 0x7fe70c16b990 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fe70c06f610 tx=0x7fe704018000 comp rx=0 tx=0).stop 2023-10-09T06:38:11.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.613+0000 7fe7118b6700 1 -- 172.21.15.163:0/834941688 shutdown_connections 2023-10-09T06:38:11.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.613+0000 7fe7118b6700 1 --2- 172.21.15.163:0/834941688 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe6ec067320 0x7fe6ec0697d0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:11.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.613+0000 7fe7118b6700 1 --2- 172.21.15.163:0/834941688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe70c155360 0x7fe70c16b990 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:11.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.613+0000 7fe7118b6700 1 --2- 172.21.15.163:0/834941688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe70c0b40f0 0x7fe70c151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:11.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.613+0000 7fe7118b6700 1 --2- 172.21.15.163:0/834941688 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe70c0ced30 0x7fe70c16b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:11.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.613+0000 7fe7118b6700 1 -- 172.21.15.163:0/834941688 >> 172.21.15.163:0/834941688 conn(0x7fe70c06baf0 msgr2=0x7fe70c0b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:11.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.613+0000 7fe7118b6700 1 -- 172.21.15.163:0/834941688 shutdown_connections 2023-10-09T06:38:11.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.613+0000 7fe7118b6700 1 -- 172.21.15.163:0/834941688 wait complete. 2023-10-09T06:38:11.616 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs.volume_0000000000063288.meta' removed 2023-10-09T06:38:11.629 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000063288.data cephfs.volume_0000000000063288.data --yes-i-really-really-mean-it 2023-10-09T06:38:11.748 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.747+0000 7fae75cb4700 1 Processor -- start 2023-10-09T06:38:11.748 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.747+0000 7fae75cb4700 1 -- start start 2023-10-09T06:38:11.750 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.748+0000 7fae75cb4700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fae700b03c0 0x7fae700cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:11.750 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.748+0000 7fae75cb4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fae700b0790 0x7fae700b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:11.750 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.748+0000 7fae75cb4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae701591e0 0x7fae70159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:11.751 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.748+0000 7fae75cb4700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae700cfa00 con 0x7fae700b03c0 2023-10-09T06:38:11.751 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.748+0000 7fae75cb4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae700cb580 con 0x7fae700b0790 2023-10-09T06:38:11.751 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.748+0000 7fae75cb4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae700b2de0 con 0x7fae701591e0 2023-10-09T06:38:11.751 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.748+0000 7fae74cb2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fae700b03c0 0x7fae700cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:11.751 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.748+0000 7fae74cb2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fae700b03c0 0x7fae700cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:60062/0 (socket says 172.21.15.163:60062) 2023-10-09T06:38:11.751 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.748+0000 7fae74cb2700 1 -- 172.21.15.163:0/3442246560 learned_addr learned my addr 172.21.15.163:0/3442246560 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:11.751 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.748+0000 7fae754b3700 1 --2- 172.21.15.163:0/3442246560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae701591e0 0x7fae70159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:11.751 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.748+0000 7fae6ffff700 1 --2- 172.21.15.163:0/3442246560 >> v2:172.21.15.181:3300/0 conn(0x7fae700b0790 0x7fae700b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:11.751 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.748+0000 7fae74cb2700 1 -- 172.21.15.163:0/3442246560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae701591e0 msgr2=0x7fae70159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:11.751 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.748+0000 7fae74cb2700 1 --2- 172.21.15.163:0/3442246560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae701591e0 0x7fae70159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:11.751 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.748+0000 7fae74cb2700 1 -- 172.21.15.163:0/3442246560 >> v2:172.21.15.181:3300/0 conn(0x7fae700b0790 msgr2=0x7fae700b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:11.752 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.748+0000 7fae74cb2700 1 --2- 172.21.15.163:0/3442246560 >> v2:172.21.15.181:3300/0 conn(0x7fae700b0790 0x7fae700b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:11.752 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.748+0000 7fae74cb2700 1 -- 172.21.15.163:0/3442246560 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae700cf160 con 0x7fae700b03c0 2023-10-09T06:38:11.752 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.748+0000 7fae754b3700 1 --2- 172.21.15.163:0/3442246560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae701591e0 0x7fae70159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:11.752 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.749+0000 7fae74cb2700 1 --2- 172.21.15.163:0/3442246560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae700b03c0 0x7fae700cea70 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fae60009f20 tx=0x7fae60009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f780ffb68e281e44 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:11.752 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.749+0000 7fae6f7fe700 1 -- 172.21.15.163:0/3442246560 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6000e070 con 0x7fae700b03c0 2023-10-09T06:38:11.752 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.749+0000 7fae6f7fe700 1 -- 172.21.15.163:0/3442246560 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae6000ba40 con 0x7fae700b03c0 2023-10-09T06:38:11.752 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.749+0000 7fae6f7fe700 1 -- 172.21.15.163:0/3442246560 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae600092e0 con 0x7fae700b03c0 2023-10-09T06:38:11.752 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.749+0000 7fae75cb4700 1 -- 172.21.15.163:0/3442246560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae700b03c0 msgr2=0x7fae700cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:11.752 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.749+0000 7fae75cb4700 1 --2- 172.21.15.163:0/3442246560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae700b03c0 0x7fae700cea70 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fae60009f20 tx=0x7fae60009840 comp rx=0 tx=0).stop 2023-10-09T06:38:11.752 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.749+0000 7fae75cb4700 1 -- 172.21.15.163:0/3442246560 shutdown_connections 2023-10-09T06:38:11.752 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.749+0000 7fae75cb4700 1 --2- 172.21.15.163:0/3442246560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae701591e0 0x7fae70159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:11.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.749+0000 7fae75cb4700 1 --2- 172.21.15.163:0/3442246560 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae700b03c0 0x7fae700cea70 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:11.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.749+0000 7fae75cb4700 1 --2- 172.21.15.163:0/3442246560 >> v2:172.21.15.181:3300/0 conn(0x7fae700b0790 0x7fae700b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:11.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.749+0000 7fae75cb4700 1 -- 172.21.15.163:0/3442246560 >> 172.21.15.163:0/3442246560 conn(0x7fae70063230 msgr2=0x7fae70063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:11.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.749+0000 7fae75cb4700 1 -- 172.21.15.163:0/3442246560 shutdown_connections 2023-10-09T06:38:11.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.749+0000 7fae75cb4700 1 -- 172.21.15.163:0/3442246560 wait complete. 2023-10-09T06:38:11.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.750+0000 7fae75cb4700 1 Processor -- start 2023-10-09T06:38:11.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.751+0000 7fae75cb4700 1 -- start start 2023-10-09T06:38:11.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.751+0000 7fae75cb4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae700b03c0 0x7fae70159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:11.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.751+0000 7fae75cb4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae700b0790 0x7fae700c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:11.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.751+0000 7fae75cb4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae701591e0 0x7fae7015d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:11.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.751+0000 7fae75cb4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae700c8ab0 con 0x7fae701591e0 2023-10-09T06:38:11.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.751+0000 7fae75cb4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae700c3b40 con 0x7fae700b0790 2023-10-09T06:38:11.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.751+0000 7fae75cb4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae700c98c0 con 0x7fae700b03c0 2023-10-09T06:38:11.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.751+0000 7fae754b3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae701591e0 0x7fae7015d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:11.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.751+0000 7fae754b3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae701591e0 0x7fae7015d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:60072/0 (socket says 172.21.15.163:60072) 2023-10-09T06:38:11.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.752+0000 7fae754b3700 1 -- 172.21.15.163:0/1668012425 learned_addr learned my addr 172.21.15.163:0/1668012425 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:11.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.752+0000 7fae74cb2700 1 --2- 172.21.15.163:0/1668012425 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae700b03c0 0x7fae70159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:11.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.752+0000 7fae6ffff700 1 --2- 172.21.15.163:0/1668012425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae700b0790 0x7fae700c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:11.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.752+0000 7fae754b3700 1 -- 172.21.15.163:0/1668012425 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae700b03c0 msgr2=0x7fae70159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:11.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.752+0000 7fae754b3700 1 --2- 172.21.15.163:0/1668012425 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae700b03c0 0x7fae70159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:11.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.752+0000 7fae754b3700 1 -- 172.21.15.163:0/1668012425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae700b0790 msgr2=0x7fae700c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:11.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.752+0000 7fae754b3700 1 --2- 172.21.15.163:0/1668012425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae700b0790 0x7fae700c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:11.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.752+0000 7fae754b3700 1 -- 172.21.15.163:0/1668012425 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae60009ae0 con 0x7fae701591e0 2023-10-09T06:38:11.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.752+0000 7fae754b3700 1 --2- 172.21.15.163:0/1668012425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae701591e0 0x7fae7015d4e0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fae6800c850 tx=0x7fae68008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:11.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.752+0000 7fae6cff9700 1 -- 172.21.15.163:0/1668012425 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae68011600 con 0x7fae701591e0 2023-10-09T06:38:11.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.752+0000 7fae6cff9700 1 -- 172.21.15.163:0/1668012425 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae68011760 con 0x7fae701591e0 2023-10-09T06:38:11.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.752+0000 7fae6cff9700 1 -- 172.21.15.163:0/1668012425 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae68011b20 con 0x7fae701591e0 2023-10-09T06:38:11.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.752+0000 7fae75cb4700 1 -- 172.21.15.163:0/1668012425 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae7015da20 con 0x7fae701591e0 2023-10-09T06:38:11.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.752+0000 7fae75cb4700 1 -- 172.21.15.163:0/1668012425 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae700c8f90 con 0x7fae701591e0 2023-10-09T06:38:11.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.753+0000 7fae75cb4700 1 -- 172.21.15.163:0/1668012425 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae700c9a70 con 0x7fae701591e0 2023-10-09T06:38:11.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.753+0000 7fae6cff9700 1 -- 172.21.15.163:0/1668012425 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fae68018350 con 0x7fae701591e0 2023-10-09T06:38:11.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.754+0000 7fae6cff9700 1 --2- 172.21.15.163:0/1668012425 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fae50067320 0x7fae500697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:11.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.754+0000 7fae6cff9700 1 -- 172.21.15.163:0/1668012425 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5394+0+0 (secure 0 0 0) 0x7fae680849b0 con 0x7fae701591e0 2023-10-09T06:38:11.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.754+0000 7fae74cb2700 1 --2- 172.21.15.163:0/1668012425 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fae50067320 0x7fae500697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:11.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.754+0000 7fae74cb2700 1 --2- 172.21.15.163:0/1668012425 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fae50067320 0x7fae500697d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fae60009880 tx=0x7fae60018000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:11.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.758+0000 7fae6cff9700 1 -- 172.21.15.163:0/1668012425 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae68051660 con 0x7fae701591e0 2023-10-09T06:38:11.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:11.940+0000 7fae75cb4700 1 -- 172.21.15.163:0/1668012425 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000063288.data", "pool2": "cephfs.volume_0000000000063288.data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fae7004c3c0 con 0x7fae701591e0 2023-10-09T06:38:12.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.625+0000 7fae6cff9700 1 -- 172.21.15.163:0/1668012425 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000063288.data", "pool2": "cephfs.volume_0000000000063288.data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000063288.data' removed v143) v1 ==== 241+0+0 (secure 0 0 0) 0x7fae6801e100 con 0x7fae701591e0 2023-10-09T06:38:12.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.629+0000 7fae75cb4700 1 -- 172.21.15.163:0/1668012425 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fae50067320 msgr2=0x7fae500697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:12.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.629+0000 7fae75cb4700 1 --2- 172.21.15.163:0/1668012425 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fae50067320 0x7fae500697d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fae60009880 tx=0x7fae60018000 comp rx=0 tx=0).stop 2023-10-09T06:38:12.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.629+0000 7fae75cb4700 1 -- 172.21.15.163:0/1668012425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae701591e0 msgr2=0x7fae7015d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:12.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.629+0000 7fae75cb4700 1 --2- 172.21.15.163:0/1668012425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae701591e0 0x7fae7015d4e0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fae6800c850 tx=0x7fae68008040 comp rx=0 tx=0).stop 2023-10-09T06:38:12.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.629+0000 7fae75cb4700 1 -- 172.21.15.163:0/1668012425 shutdown_connections 2023-10-09T06:38:12.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.629+0000 7fae75cb4700 1 --2- 172.21.15.163:0/1668012425 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fae50067320 0x7fae500697d0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:12.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.629+0000 7fae75cb4700 1 --2- 172.21.15.163:0/1668012425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae701591e0 0x7fae7015d4e0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:12.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.629+0000 7fae75cb4700 1 --2- 172.21.15.163:0/1668012425 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae700b03c0 0x7fae70159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:12.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.629+0000 7fae75cb4700 1 --2- 172.21.15.163:0/1668012425 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae700b0790 0x7fae700c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:12.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.630+0000 7fae75cb4700 1 -- 172.21.15.163:0/1668012425 >> 172.21.15.163:0/1668012425 conn(0x7fae70063230 msgr2=0x7fae700b51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:12.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.630+0000 7fae75cb4700 1 -- 172.21.15.163:0/1668012425 shutdown_connections 2023-10-09T06:38:12.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.630+0000 7fae75cb4700 1 -- 172.21.15.163:0/1668012425 wait complete. 2023-10-09T06:38:12.632 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs.volume_0000000000063288.data' removed 2023-10-09T06:38:12.645 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-09T06:38:12.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.761+0000 7f2a5eb34700 1 Processor -- start 2023-10-09T06:38:12.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.761+0000 7f2a5eb34700 1 -- start start 2023-10-09T06:38:12.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.762+0000 7f2a5eb34700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2a58159890 0x7f2a5815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:12.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.762+0000 7f2a5eb34700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2a58159c60 0x7f2a5815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:12.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.762+0000 7f2a5eb34700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a5815db10 0x7f2a58069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:12.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.762+0000 7f2a5eb34700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a5815df50 con 0x7f2a58159890 2023-10-09T06:38:12.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.762+0000 7f2a5eb34700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a5815a160 con 0x7f2a58159c60 2023-10-09T06:38:12.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.762+0000 7f2a5eb34700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a580d13a0 con 0x7f2a5815db10 2023-10-09T06:38:12.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.762+0000 7f2a5c8d0700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2a58159890 0x7f2a5815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:12.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.762+0000 7f2a5c8d0700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2a58159890 0x7f2a5815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:60078/0 (socket says 172.21.15.163:60078) 2023-10-09T06:38:12.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.762+0000 7f2a5c8d0700 1 -- 172.21.15.163:0/2474756154 learned_addr learned my addr 172.21.15.163:0/2474756154 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:12.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.763+0000 7f2a4ffff700 1 --2- 172.21.15.163:0/2474756154 >> v2:172.21.15.181:3300/0 conn(0x7f2a58159c60 0x7f2a5815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:12.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.763+0000 7f2a5d0d1700 1 --2- 172.21.15.163:0/2474756154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a5815db10 0x7f2a58069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:12.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.763+0000 7f2a5c8d0700 1 -- 172.21.15.163:0/2474756154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a5815db10 msgr2=0x7f2a58069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:12.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.763+0000 7f2a5c8d0700 1 --2- 172.21.15.163:0/2474756154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a5815db10 0x7f2a58069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:12.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.763+0000 7f2a5c8d0700 1 -- 172.21.15.163:0/2474756154 >> v2:172.21.15.181:3300/0 conn(0x7f2a58159c60 msgr2=0x7f2a5815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:12.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.763+0000 7f2a5c8d0700 1 --2- 172.21.15.163:0/2474756154 >> v2:172.21.15.181:3300/0 conn(0x7f2a58159c60 0x7f2a5815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:12.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.763+0000 7f2a5c8d0700 1 -- 172.21.15.163:0/2474756154 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5815e140 con 0x7f2a58159890 2023-10-09T06:38:12.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.763+0000 7f2a5c8d0700 1 --2- 172.21.15.163:0/2474756154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a58159890 0x7f2a5815abd0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f2a580b67f0 tx=0x7f2a54009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a600b1eed74331e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:12.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.763+0000 7f2a4f7fe700 1 -- 172.21.15.163:0/2474756154 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5400e070 con 0x7f2a58159890 2023-10-09T06:38:12.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.763+0000 7f2a4f7fe700 1 -- 172.21.15.163:0/2474756154 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5400ba50 con 0x7f2a58159890 2023-10-09T06:38:12.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.763+0000 7f2a4f7fe700 1 -- 172.21.15.163:0/2474756154 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a540092e0 con 0x7f2a58159890 2023-10-09T06:38:12.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.763+0000 7f2a5eb34700 1 -- 172.21.15.163:0/2474756154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a58159890 msgr2=0x7f2a5815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:12.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.763+0000 7f2a5eb34700 1 --2- 172.21.15.163:0/2474756154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a58159890 0x7f2a5815abd0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f2a580b67f0 tx=0x7f2a54009840 comp rx=0 tx=0).stop 2023-10-09T06:38:12.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.763+0000 7f2a5eb34700 1 -- 172.21.15.163:0/2474756154 shutdown_connections 2023-10-09T06:38:12.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.764+0000 7f2a5eb34700 1 --2- 172.21.15.163:0/2474756154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a5815db10 0x7f2a58069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:12.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.764+0000 7f2a5eb34700 1 --2- 172.21.15.163:0/2474756154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a58159890 0x7f2a5815abd0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:12.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.764+0000 7f2a5eb34700 1 --2- 172.21.15.163:0/2474756154 >> v2:172.21.15.181:3300/0 conn(0x7f2a58159c60 0x7f2a5815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:12.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.764+0000 7f2a5eb34700 1 -- 172.21.15.163:0/2474756154 >> 172.21.15.163:0/2474756154 conn(0x7f2a58068e20 msgr2=0x7f2a580d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:12.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.764+0000 7f2a5eb34700 1 -- 172.21.15.163:0/2474756154 shutdown_connections 2023-10-09T06:38:12.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.764+0000 7f2a5eb34700 1 -- 172.21.15.163:0/2474756154 wait complete. 2023-10-09T06:38:12.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.765+0000 7f2a5eb34700 1 Processor -- start 2023-10-09T06:38:12.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.765+0000 7f2a5eb34700 1 -- start start 2023-10-09T06:38:12.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.765+0000 7f2a5eb34700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a58159890 0x7f2a580b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:12.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.765+0000 7f2a5eb34700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a58159c60 0x7f2a580b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:12.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.765+0000 7f2a5eb34700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a5815db10 0x7f2a580b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:12.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.765+0000 7f2a5eb34700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a58151c80 con 0x7f2a58159890 2023-10-09T06:38:12.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.765+0000 7f2a5eb34700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a58151390 con 0x7f2a5815db10 2023-10-09T06:38:12.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.765+0000 7f2a5eb34700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a58151630 con 0x7f2a58159c60 2023-10-09T06:38:12.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.766+0000 7f2a5c8d0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a58159890 0x7f2a580b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:12.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.766+0000 7f2a5c8d0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a58159890 0x7f2a580b5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:60088/0 (socket says 172.21.15.163:60088) 2023-10-09T06:38:12.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.766+0000 7f2a5c8d0700 1 -- 172.21.15.163:0/1114193451 learned_addr learned my addr 172.21.15.163:0/1114193451 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:12.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.766+0000 7f2a4ffff700 1 --2- 172.21.15.163:0/1114193451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a58159c60 0x7f2a580b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:12.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.766+0000 7f2a5d0d1700 1 --2- 172.21.15.163:0/1114193451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a5815db10 0x7f2a580b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:12.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.766+0000 7f2a5c8d0700 1 -- 172.21.15.163:0/1114193451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a58159c60 msgr2=0x7f2a580b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:12.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.766+0000 7f2a5c8d0700 1 --2- 172.21.15.163:0/1114193451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a58159c60 0x7f2a580b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:12.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.766+0000 7f2a5c8d0700 1 -- 172.21.15.163:0/1114193451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a5815db10 msgr2=0x7f2a580b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:12.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.766+0000 7f2a5c8d0700 1 --2- 172.21.15.163:0/1114193451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a5815db10 0x7f2a580b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:12.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.766+0000 7f2a5c8d0700 1 -- 172.21.15.163:0/1114193451 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a54009ae0 con 0x7f2a58159890 2023-10-09T06:38:12.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.766+0000 7f2a5c8d0700 1 --2- 172.21.15.163:0/1114193451 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a58159890 0x7f2a580b5db0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f2a58070e20 tx=0x7f2a54018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:12.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.766+0000 7f2a4cff9700 1 -- 172.21.15.163:0/1114193451 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a540088b0 con 0x7f2a58159890 2023-10-09T06:38:12.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.766+0000 7f2a4cff9700 1 -- 172.21.15.163:0/1114193451 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a54008a30 con 0x7f2a58159890 2023-10-09T06:38:12.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.766+0000 7f2a4cff9700 1 -- 172.21.15.163:0/1114193451 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a54008df0 con 0x7f2a58159890 2023-10-09T06:38:12.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.766+0000 7f2a5eb34700 1 -- 172.21.15.163:0/1114193451 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a58151910 con 0x7f2a58159890 2023-10-09T06:38:12.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.766+0000 7f2a5eb34700 1 -- 172.21.15.163:0/1114193451 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a58152110 con 0x7f2a58159890 2023-10-09T06:38:12.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.767+0000 7f2a5eb34700 1 -- 172.21.15.163:0/1114193451 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a58151f60 con 0x7f2a58159890 2023-10-09T06:38:12.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.767+0000 7f2a4cff9700 1 -- 172.21.15.163:0/1114193451 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2a54003040 con 0x7f2a58159890 2023-10-09T06:38:12.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.768+0000 7f2a4cff9700 1 --2- 172.21.15.163:0/1114193451 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2a44067320 0x7f2a440697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:12.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.768+0000 7f2a4cff9700 1 -- 172.21.15.163:0/1114193451 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2a54086340 con 0x7f2a58159890 2023-10-09T06:38:12.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.768+0000 7f2a4ffff700 1 --2- 172.21.15.163:0/1114193451 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2a44067320 0x7f2a440697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:12.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.769+0000 7f2a4ffff700 1 --2- 172.21.15.163:0/1114193451 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2a44067320 0x7f2a440697d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2a5816d100 tx=0x7f2a48008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:12.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.772+0000 7f2a4cff9700 1 -- 172.21.15.163:0/1114193451 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a54053270 con 0x7f2a58159890 2023-10-09T06:38:12.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.955+0000 7f2a5eb34700 1 -- 172.21.15.163:0/1114193451 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2a5804c3c0 con 0x7f2a58159890 2023-10-09T06:38:12.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.959+0000 7f2a4cff9700 1 -- 172.21.15.163:0/1114193451 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2a54056890 con 0x7f2a58159890 2023-10-09T06:38:12.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.961+0000 7f2a5eb34700 1 -- 172.21.15.163:0/1114193451 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2a44067320 msgr2=0x7f2a440697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:12.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.961+0000 7f2a5eb34700 1 --2- 172.21.15.163:0/1114193451 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2a44067320 0x7f2a440697d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2a5816d100 tx=0x7f2a48008040 comp rx=0 tx=0).stop 2023-10-09T06:38:12.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.961+0000 7f2a5eb34700 1 -- 172.21.15.163:0/1114193451 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a58159890 msgr2=0x7f2a580b5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:12.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.961+0000 7f2a5eb34700 1 --2- 172.21.15.163:0/1114193451 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a58159890 0x7f2a580b5db0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f2a58070e20 tx=0x7f2a54018000 comp rx=0 tx=0).stop 2023-10-09T06:38:12.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.961+0000 7f2a5eb34700 1 -- 172.21.15.163:0/1114193451 shutdown_connections 2023-10-09T06:38:12.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.961+0000 7f2a5eb34700 1 --2- 172.21.15.163:0/1114193451 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2a44067320 0x7f2a440697d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:12.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.961+0000 7f2a5eb34700 1 --2- 172.21.15.163:0/1114193451 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a58159890 0x7f2a580b5db0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:12.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.961+0000 7f2a5eb34700 1 --2- 172.21.15.163:0/1114193451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a5815db10 0x7f2a580b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:12.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.961+0000 7f2a5eb34700 1 --2- 172.21.15.163:0/1114193451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a58159c60 0x7f2a580b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:12.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.961+0000 7f2a5eb34700 1 -- 172.21.15.163:0/1114193451 >> 172.21.15.163:0/1114193451 conn(0x7f2a58068e20 msgr2=0x7f2a581533c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:12.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.961+0000 7f2a5eb34700 1 -- 172.21.15.163:0/1114193451 shutdown_connections 2023-10-09T06:38:12.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:12.961+0000 7f2a5eb34700 1 -- 172.21.15.163:0/1114193451 wait complete. 2023-10-09T06:38:12.974 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename' 2023-10-09T06:38:13.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.087+0000 7f6d7bfff700 1 Processor -- start 2023-10-09T06:38:13.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.087+0000 7f6d7bfff700 1 -- start start 2023-10-09T06:38:13.090 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.088+0000 7f6d7bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6d7c157270 0x7f6d7c068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:13.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.088+0000 7f6d7bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6d7c0688d0 0x7f6d7c0b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:13.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.088+0000 7f6d7bfff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d7c15a980 con 0x7f6d7c0b0a00 2023-10-09T06:38:13.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.088+0000 7f6d7bfff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d7c15e310 con 0x7f6d7c0688d0 2023-10-09T06:38:13.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.088+0000 7f6d7bfff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d7c068d50 con 0x7f6d7c157270 2023-10-09T06:38:13.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.088+0000 7f6d73fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6d7c0688d0 0x7f6d7c0b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:13.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.088+0000 7f6d7b7fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f6d7c0b0a00 0x7f6d7c06a7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:51898/0 (socket says 172.21.15.163:51898) 2023-10-09T06:38:13.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.088+0000 7f6d7b7fe700 1 -- 172.21.15.163:0/417593127 learned_addr learned my addr 172.21.15.163:0/417593127 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:13.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.089+0000 7f6d7affd700 1 --2- 172.21.15.163:0/417593127 >> v2:172.21.15.181:3300/0 conn(0x7f6d7c157270 0x7f6d7c068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:13.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.089+0000 7f6d73fff700 1 -- 172.21.15.163:0/417593127 >> v2:172.21.15.181:3300/0 conn(0x7f6d7c157270 msgr2=0x7f6d7c068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:13.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.089+0000 7f6d73fff700 1 --2- 172.21.15.163:0/417593127 >> v2:172.21.15.181:3300/0 conn(0x7f6d7c157270 0x7f6d7c068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:13.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.089+0000 7f6d73fff700 1 -- 172.21.15.163:0/417593127 >> v1:172.21.15.163:6789/0 conn(0x7f6d7c0b0a00 legacy=0x7f6d7c06a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:13.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.089+0000 7f6d73fff700 1 -- 172.21.15.163:0/417593127 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7c15a050 con 0x7f6d7c0688d0 2023-10-09T06:38:13.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.089+0000 7f6d7affd700 1 --2- 172.21.15.163:0/417593127 >> v2:172.21.15.181:3300/0 conn(0x7f6d7c157270 0x7f6d7c068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:13.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.089+0000 7f6d7a7fc700 1 -- 172.21.15.163:0/417593127 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4199416653 0 0) 0x7f6d7c15a980 con 0x7f6d7c0b0a00 2023-10-09T06:38:13.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.089+0000 7f6d73fff700 1 --2- 172.21.15.163:0/417593127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d7c0688d0 0x7f6d7c0b04c0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0b3360 tx=0x7f6d640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba79995020a648f server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:13.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.089+0000 7f6d7a7fc700 1 -- 172.21.15.163:0/417593127 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64009e80 con 0x7f6d7c0688d0 2023-10-09T06:38:13.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.089+0000 7f6d7a7fc700 1 -- 172.21.15.163:0/417593127 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d6400b820 con 0x7f6d7c0688d0 2023-10-09T06:38:13.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.089+0000 7f6d7a7fc700 1 -- 172.21.15.163:0/417593127 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6400bba0 con 0x7f6d7c0688d0 2023-10-09T06:38:13.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.090+0000 7f6d7bfff700 1 -- 172.21.15.163:0/417593127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d7c0688d0 msgr2=0x7f6d7c0b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:13.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.090+0000 7f6d7bfff700 1 --2- 172.21.15.163:0/417593127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d7c0688d0 0x7f6d7c0b04c0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0b3360 tx=0x7f6d640091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:13.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.090+0000 7f6d7bfff700 1 -- 172.21.15.163:0/417593127 shutdown_connections 2023-10-09T06:38:13.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.090+0000 7f6d7bfff700 1 --2- 172.21.15.163:0/417593127 >> v2:172.21.15.181:3300/0 conn(0x7f6d7c157270 0x7f6d7c068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:13.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.090+0000 7f6d7bfff700 1 --2- 172.21.15.163:0/417593127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d7c0688d0 0x7f6d7c0b04c0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:13.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.090+0000 7f6d7bfff700 1 -- 172.21.15.163:0/417593127 >> 172.21.15.163:0/417593127 conn(0x7f6d7c0cd310 msgr2=0x7f6d7c0cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:13.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.090+0000 7f6d7bfff700 1 -- 172.21.15.163:0/417593127 shutdown_connections 2023-10-09T06:38:13.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.090+0000 7f6d7bfff700 1 -- 172.21.15.163:0/417593127 wait complete. 2023-10-09T06:38:13.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.091+0000 7f6d7bfff700 1 Processor -- start 2023-10-09T06:38:13.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.091+0000 7f6d7bfff700 1 -- start start 2023-10-09T06:38:13.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.091+0000 7f6d7bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d7c0688d0 0x7f6d7c14fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:13.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.091+0000 7f6d7bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d7c0b0a00 0x7f6d7c0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:13.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.091+0000 7f6d7bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d7c157270 0x7f6d7c1552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:13.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.091+0000 7f6d7bfff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c16a060 con 0x7f6d7c0688d0 2023-10-09T06:38:13.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.091+0000 7f6d7bfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c16db00 con 0x7f6d7c0b0a00 2023-10-09T06:38:13.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.091+0000 7f6d7bfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c16dde0 con 0x7f6d7c157270 2023-10-09T06:38:13.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.091+0000 7f6d7affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d7c0688d0 0x7f6d7c14fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:13.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.092+0000 7f6d7affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d7c0688d0 0x7f6d7c14fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:60100/0 (socket says 172.21.15.163:60100) 2023-10-09T06:38:13.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.092+0000 7f6d7affd700 1 -- 172.21.15.163:0/2524061689 learned_addr learned my addr 172.21.15.163:0/2524061689 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:13.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.092+0000 7f6d73fff700 1 --2- 172.21.15.163:0/2524061689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d7c0b0a00 0x7f6d7c0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:13.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.092+0000 7f6d7b7fe700 1 --2- 172.21.15.163:0/2524061689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d7c157270 0x7f6d7c1552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:13.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.092+0000 7f6d7affd700 1 -- 172.21.15.163:0/2524061689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d7c157270 msgr2=0x7f6d7c1552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:13.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.092+0000 7f6d7affd700 1 --2- 172.21.15.163:0/2524061689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d7c157270 0x7f6d7c1552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:13.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.092+0000 7f6d7affd700 1 -- 172.21.15.163:0/2524061689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d7c0b0a00 msgr2=0x7f6d7c0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:13.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.092+0000 7f6d7affd700 1 --2- 172.21.15.163:0/2524061689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d7c0b0a00 0x7f6d7c0cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:13.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.092+0000 7f6d7affd700 1 -- 172.21.15.163:0/2524061689 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d64009420 con 0x7f6d7c0688d0 2023-10-09T06:38:13.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.092+0000 7f6d7affd700 1 --2- 172.21.15.163:0/2524061689 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d7c0688d0 0x7f6d7c14fab0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00c850 tx=0x7f6d6c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:13.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.092+0000 7f6d72ffd700 1 -- 172.21.15.163:0/2524061689 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c011600 con 0x7f6d7c0688d0 2023-10-09T06:38:13.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.092+0000 7f6d72ffd700 1 -- 172.21.15.163:0/2524061689 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d6c011760 con 0x7f6d7c0688d0 2023-10-09T06:38:13.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.092+0000 7f6d72ffd700 1 -- 172.21.15.163:0/2524061689 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c011b20 con 0x7f6d7c0688d0 2023-10-09T06:38:13.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.092+0000 7f6d7bfff700 1 -- 172.21.15.163:0/2524061689 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d7c16e0c0 con 0x7f6d7c0688d0 2023-10-09T06:38:13.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.092+0000 7f6d7bfff700 1 -- 172.21.15.163:0/2524061689 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d7c14fff0 con 0x7f6d7c0688d0 2023-10-09T06:38:13.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.093+0000 7f6d7bfff700 1 -- 172.21.15.163:0/2524061689 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d7c04c3c0 con 0x7f6d7c0688d0 2023-10-09T06:38:13.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.093+0000 7f6d72ffd700 1 -- 172.21.15.163:0/2524061689 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6d6c018350 con 0x7f6d7c0688d0 2023-10-09T06:38:13.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.094+0000 7f6d72ffd700 1 --2- 172.21.15.163:0/2524061689 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6d5c067320 0x7f6d5c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:13.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.094+0000 7f6d72ffd700 1 -- 172.21.15.163:0/2524061689 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f6d6c0849f0 con 0x7f6d7c0688d0 2023-10-09T06:38:13.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.094+0000 7f6d73fff700 1 --2- 172.21.15.163:0/2524061689 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6d5c067320 0x7f6d5c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:13.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.094+0000 7f6d73fff700 1 --2- 172.21.15.163:0/2524061689 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6d5c067320 0x7f6d5c0697d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6d640093f0 tx=0x7f6d64017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:13.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.098+0000 7f6d72ffd700 1 -- 172.21.15.163:0/2524061689 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d6c051870 con 0x7f6d7c0688d0 2023-10-09T06:38:13.354 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.354+0000 7f6d7bfff700 1 -- 172.21.15.163:0/2524061689 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename"]} v 0) v1 -- 0x7f6d7c15a330 con 0x7f6d7c0688d0 2023-10-09T06:38:13.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.628+0000 7f6d72ffd700 1 -- 172.21.15.163:0/2524061689 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename"]}]=0 v382) v1 ==== 135+0+0 (secure 0 0 0) 0x7f6d6c01e0c0 con 0x7f6d7c0688d0 2023-10-09T06:38:13.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.630+0000 7f6d7bfff700 1 -- 172.21.15.163:0/2524061689 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6d5c067320 msgr2=0x7f6d5c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:13.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.630+0000 7f6d7bfff700 1 --2- 172.21.15.163:0/2524061689 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6d5c067320 0x7f6d5c0697d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6d640093f0 tx=0x7f6d64017000 comp rx=0 tx=0).stop 2023-10-09T06:38:13.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.630+0000 7f6d7bfff700 1 -- 172.21.15.163:0/2524061689 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d7c0688d0 msgr2=0x7f6d7c14fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:13.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.630+0000 7f6d7bfff700 1 --2- 172.21.15.163:0/2524061689 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d7c0688d0 0x7f6d7c14fab0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00c850 tx=0x7f6d6c008040 comp rx=0 tx=0).stop 2023-10-09T06:38:13.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.630+0000 7f6d7bfff700 1 -- 172.21.15.163:0/2524061689 shutdown_connections 2023-10-09T06:38:13.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.630+0000 7f6d7bfff700 1 --2- 172.21.15.163:0/2524061689 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6d5c067320 0x7f6d5c0697d0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:13.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.630+0000 7f6d7bfff700 1 --2- 172.21.15.163:0/2524061689 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6d7c0688d0 0x7f6d7c14fab0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:13.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.630+0000 7f6d7bfff700 1 --2- 172.21.15.163:0/2524061689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d7c157270 0x7f6d7c1552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:13.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.630+0000 7f6d7bfff700 1 --2- 172.21.15.163:0/2524061689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d7c0b0a00 0x7f6d7c0cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:13.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.630+0000 7f6d7bfff700 1 -- 172.21.15.163:0/2524061689 >> 172.21.15.163:0/2524061689 conn(0x7f6d7c0cd310 msgr2=0x7f6d7c1507c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:13.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.630+0000 7f6d7bfff700 1 -- 172.21.15.163:0/2524061689 shutdown_connections 2023-10-09T06:38:13.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.630+0000 7f6d7bfff700 1 -- 172.21.15.163:0/2524061689 wait complete. 2023-10-09T06:38:13.643 INFO:tasks.cephfs_test_runner:That volume, its file system and pools, can be renamed. ... ok 2023-10-09T06:38:13.643 INFO:tasks.cephfs_test_runner:Starting test: test_volume_rename_fails_without_confirmation_flag (tasks.cephfs.test_volumes.TestVolumes) That renaming volume fails without --yes-i-really-mean-it flag. 2023-10-09T06:38:13.643 INFO:tasks.cephfs_test_runner:test_volume_rename_fails_without_confirmation_flag (tasks.cephfs.test_volumes.TestVolumes) 2023-10-09T06:38:13.644 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag' 2023-10-09T06:38:13.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.758+0000 7f4b220a7700 1 Processor -- start 2023-10-09T06:38:13.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.758+0000 7f4b220a7700 1 -- start start 2023-10-09T06:38:13.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.759+0000 7f4b220a7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b1c159840 0x7f4b1c0cb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:13.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.759+0000 7f4b220a7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4b1c0cba70 0x7f4b1c0694a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:13.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.759+0000 7f4b220a7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b1c0cfb30 con 0x7f4b1c159c10 2023-10-09T06:38:13.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.759+0000 7f4b220a7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b1c0cbf10 con 0x7f4b1c0cba70 2023-10-09T06:38:13.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.759+0000 7f4b220a7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c0cf4e0 con 0x7f4b1c159840 2023-10-09T06:38:13.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.759+0000 7f4b1b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b1c159840 0x7f4b1c0cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:13.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.759+0000 7f4b1affd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4b1c0cba70 0x7f4b1c0694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:13.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.759+0000 7f4b1bfff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4b1c159c10 0x7f4b1c0ce330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:48712/0 (socket says 172.21.15.163:48712) 2023-10-09T06:38:13.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.759+0000 7f4b1b7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b1c159840 0x7f4b1c0cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:47460/0 (socket says 172.21.15.163:47460) 2023-10-09T06:38:13.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.759+0000 7f4b1bfff700 1 -- 172.21.15.163:0/4067186915 learned_addr learned my addr 172.21.15.163:0/4067186915 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:13.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.760+0000 7f4b1b7fe700 1 -- 172.21.15.163:0/4067186915 >> v2:172.21.15.181:3300/0 conn(0x7f4b1c0cba70 msgr2=0x7f4b1c0694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:13.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.760+0000 7f4b1b7fe700 1 --2- 172.21.15.163:0/4067186915 >> v2:172.21.15.181:3300/0 conn(0x7f4b1c0cba70 0x7f4b1c0694a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:13.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.760+0000 7f4b1b7fe700 1 -- 172.21.15.163:0/4067186915 >> v1:172.21.15.181:6789/0 conn(0x7f4b1c159c10 legacy=0x7f4b1c0ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:13.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.760+0000 7f4b1b7fe700 1 -- 172.21.15.163:0/4067186915 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1c0cf7c0 con 0x7f4b1c159840 2023-10-09T06:38:13.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.760+0000 7f4b1affd700 1 --2- 172.21.15.163:0/4067186915 >> v2:172.21.15.181:3300/0 conn(0x7f4b1c0cba70 0x7f4b1c0694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:38:13.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.760+0000 7f4b1b7fe700 1 --2- 172.21.15.163:0/4067186915 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b1c159840 0x7f4b1c0cb530 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c0098e0 tx=0x7f4b0c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=536ff7886a09c324 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:13.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.760+0000 7f4b1a7fc700 1 -- 172.21.15.163:0/4067186915 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c00b640 con 0x7f4b1c159840 2023-10-09T06:38:13.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.760+0000 7f4b1a7fc700 1 -- 172.21.15.163:0/4067186915 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0c00b7a0 con 0x7f4b1c159840 2023-10-09T06:38:13.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.760+0000 7f4b1a7fc700 1 -- 172.21.15.163:0/4067186915 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c00bb40 con 0x7f4b1c159840 2023-10-09T06:38:13.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.760+0000 7f4b220a7700 1 -- 172.21.15.163:0/4067186915 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b1c159840 msgr2=0x7f4b1c0cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:13.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.760+0000 7f4b220a7700 1 --2- 172.21.15.163:0/4067186915 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b1c159840 0x7f4b1c0cb530 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c0098e0 tx=0x7f4b0c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:13.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.760+0000 7f4b220a7700 1 -- 172.21.15.163:0/4067186915 shutdown_connections 2023-10-09T06:38:13.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.760+0000 7f4b220a7700 1 --2- 172.21.15.163:0/4067186915 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b1c159840 0x7f4b1c0cb530 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:13.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.760+0000 7f4b220a7700 1 --2- 172.21.15.163:0/4067186915 >> v2:172.21.15.181:3300/0 conn(0x7f4b1c0cba70 0x7f4b1c0694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:13.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.760+0000 7f4b220a7700 1 -- 172.21.15.163:0/4067186915 >> 172.21.15.163:0/4067186915 conn(0x7f4b1c068e10 msgr2=0x7f4b1c0d0760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:13.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.760+0000 7f4b220a7700 1 -- 172.21.15.163:0/4067186915 shutdown_connections 2023-10-09T06:38:13.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.760+0000 7f4b220a7700 1 -- 172.21.15.163:0/4067186915 wait complete. 2023-10-09T06:38:13.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.762+0000 7f4b220a7700 1 Processor -- start 2023-10-09T06:38:13.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.762+0000 7f4b220a7700 1 -- start start 2023-10-09T06:38:13.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.762+0000 7f4b220a7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b1c0cba70 0x7f4b1c1480e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:13.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.762+0000 7f4b220a7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b1c159840 0x7f4b1c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:13.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.762+0000 7f4b220a7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b1c159c10 0x7f4b1c069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:13.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.762+0000 7f4b220a7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c0b5ec0 con 0x7f4b1c159c10 2023-10-09T06:38:13.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.762+0000 7f4b220a7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c0b52b0 con 0x7f4b1c0cba70 2023-10-09T06:38:13.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.762+0000 7f4b220a7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c0b5590 con 0x7f4b1c159840 2023-10-09T06:38:13.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.763+0000 7f4b1bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b1c159c10 0x7f4b1c069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:13.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.763+0000 7f4b1bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b1c159c10 0x7f4b1c069bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:60112/0 (socket says 172.21.15.163:60112) 2023-10-09T06:38:13.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.763+0000 7f4b1bfff700 1 -- 172.21.15.163:0/2676436927 learned_addr learned my addr 172.21.15.163:0/2676436927 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:13.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.763+0000 7f4b1b7fe700 1 --2- 172.21.15.163:0/2676436927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b1c0cba70 0x7f4b1c1480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:13.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.763+0000 7f4b1affd700 1 --2- 172.21.15.163:0/2676436927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b1c159840 0x7f4b1c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:13.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.763+0000 7f4b1bfff700 1 -- 172.21.15.163:0/2676436927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b1c159840 msgr2=0x7f4b1c0696b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:13.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.763+0000 7f4b1bfff700 1 --2- 172.21.15.163:0/2676436927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b1c159840 0x7f4b1c0696b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:13.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.763+0000 7f4b1bfff700 1 -- 172.21.15.163:0/2676436927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b1c0cba70 msgr2=0x7f4b1c1480e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:13.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.763+0000 7f4b1bfff700 1 --2- 172.21.15.163:0/2676436927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b1c0cba70 0x7f4b1c1480e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:13.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.763+0000 7f4b1bfff700 1 -- 172.21.15.163:0/2676436927 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b10009420 con 0x7f4b1c159c10 2023-10-09T06:38:13.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.763+0000 7f4b1bfff700 1 --2- 172.21.15.163:0/2676436927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b1c159c10 0x7f4b1c069bf0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400cbc0 tx=0x7f4b140019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:13.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.763+0000 7f4b037fe700 1 -- 172.21.15.163:0/2676436927 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1400a7e0 con 0x7f4b1c159c10 2023-10-09T06:38:13.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.763+0000 7f4b037fe700 1 -- 172.21.15.163:0/2676436927 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1400a940 con 0x7f4b1c159c10 2023-10-09T06:38:13.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.763+0000 7f4b220a7700 1 -- 172.21.15.163:0/2676436927 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b0c009420 con 0x7f4b1c159c10 2023-10-09T06:38:13.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.763+0000 7f4b037fe700 1 -- 172.21.15.163:0/2676436927 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1400acc0 con 0x7f4b1c159c10 2023-10-09T06:38:13.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.763+0000 7f4b220a7700 1 -- 172.21.15.163:0/2676436927 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b1c0b5b50 con 0x7f4b1c159c10 2023-10-09T06:38:13.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.764+0000 7f4b220a7700 1 -- 172.21.15.163:0/2676436927 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b1c064550 con 0x7f4b1c159c10 2023-10-09T06:38:13.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.765+0000 7f4b037fe700 1 -- 172.21.15.163:0/2676436927 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4b14007300 con 0x7f4b1c159c10 2023-10-09T06:38:13.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.765+0000 7f4b037fe700 1 --2- 172.21.15.163:0/2676436927 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4b08067320 0x7f4b080697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:13.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.765+0000 7f4b037fe700 1 -- 172.21.15.163:0/2676436927 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f4b14084c50 con 0x7f4b1c159c10 2023-10-09T06:38:13.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.765+0000 7f4b1b7fe700 1 --2- 172.21.15.163:0/2676436927 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4b08067320 0x7f4b080697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:13.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.766+0000 7f4b1b7fe700 1 --2- 172.21.15.163:0/2676436927 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4b08067320 0x7f4b080697d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c00a010 tx=0x7f4b0c017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:13.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:13.769+0000 7f4b037fe700 1 -- 172.21.15.163:0/2676436927 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4b14051bd0 con 0x7f4b1c159c10 2023-10-09T06:38:14.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.026+0000 7f4b220a7700 1 -- 172.21.15.163:0/2676436927 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag"]} v 0) v1 -- 0x7f4b1c04c3c0 con 0x7f4b1c159c10 2023-10-09T06:38:14.056 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:38:14.058 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:38:14.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.632+0000 7f4b037fe700 1 -- 172.21.15.163:0/2676436927 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag"]}]=0 v383) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4b140551f0 con 0x7f4b1c159c10 2023-10-09T06:38:14.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.634+0000 7f4b220a7700 1 -- 172.21.15.163:0/2676436927 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4b08067320 msgr2=0x7f4b080697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:14.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.635+0000 7f4b220a7700 1 --2- 172.21.15.163:0/2676436927 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4b08067320 0x7f4b080697d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c00a010 tx=0x7f4b0c017000 comp rx=0 tx=0).stop 2023-10-09T06:38:14.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.635+0000 7f4b220a7700 1 -- 172.21.15.163:0/2676436927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b1c159c10 msgr2=0x7f4b1c069bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:14.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.635+0000 7f4b220a7700 1 --2- 172.21.15.163:0/2676436927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b1c159c10 0x7f4b1c069bf0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400cbc0 tx=0x7f4b140019c0 comp rx=0 tx=0).stop 2023-10-09T06:38:14.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.635+0000 7f4b220a7700 1 -- 172.21.15.163:0/2676436927 shutdown_connections 2023-10-09T06:38:14.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.635+0000 7f4b220a7700 1 --2- 172.21.15.163:0/2676436927 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4b08067320 0x7f4b080697d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:14.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.635+0000 7f4b220a7700 1 --2- 172.21.15.163:0/2676436927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b1c159c10 0x7f4b1c069bf0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:14.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.635+0000 7f4b220a7700 1 --2- 172.21.15.163:0/2676436927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b1c0cba70 0x7f4b1c1480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:14.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.635+0000 7f4b220a7700 1 --2- 172.21.15.163:0/2676436927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b1c159840 0x7f4b1c0696b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:14.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.635+0000 7f4b220a7700 1 -- 172.21.15.163:0/2676436927 >> 172.21.15.163:0/2676436927 conn(0x7f4b1c068e10 msgr2=0x7f4b1c0c5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:14.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.635+0000 7f4b220a7700 1 -- 172.21.15.163:0/2676436927 shutdown_connections 2023-10-09T06:38:14.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.635+0000 7f4b220a7700 1 -- 172.21.15.163:0/2676436927 wait complete. 2023-10-09T06:38:14.648 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-09T06:38:14.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.768+0000 7f9bd27a8700 1 Processor -- start 2023-10-09T06:38:14.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.768+0000 7f9bd27a8700 1 -- start start 2023-10-09T06:38:14.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.769+0000 7f9bd27a8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bcc0b4e70 0x7f9bcc0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:14.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.769+0000 7f9bd27a8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9bcc0b16d0 0x7f9bcc0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:14.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.769+0000 7f9bd27a8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bcc0b2080 con 0x7f9bcc15d240 2023-10-09T06:38:14.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.769+0000 7f9bd27a8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bcc0cfb00 con 0x7f9bcc0b16d0 2023-10-09T06:38:14.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.769+0000 7f9bd27a8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc0b5c40 con 0x7f9bcc0b4e70 2023-10-09T06:38:14.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.769+0000 7f9bd17a6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bcc0b4e70 0x7f9bcc0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:14.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.769+0000 7f9bd0fa5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9bcc0b16d0 0x7f9bcc0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:14.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.769+0000 7f9bd1fa7700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9bcc15d240 0x7f9bcc15d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:48720/0 (socket says 172.21.15.163:48720) 2023-10-09T06:38:14.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.769+0000 7f9bd0fa5700 1 --2- 172.21.15.163:0/685557442 >> v2:172.21.15.181:3300/0 conn(0x7f9bcc0b16d0 0x7f9bcc0b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:37652/0 (socket says 172.21.15.163:37652) 2023-10-09T06:38:14.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.769+0000 7f9bd1fa7700 1 -- 172.21.15.163:0/685557442 learned_addr learned my addr 172.21.15.163:0/685557442 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:14.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.770+0000 7f9bd17a6700 1 -- 172.21.15.163:0/685557442 >> v2:172.21.15.181:3300/0 conn(0x7f9bcc0b16d0 msgr2=0x7f9bcc0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:14.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.770+0000 7f9bd17a6700 1 --2- 172.21.15.163:0/685557442 >> v2:172.21.15.181:3300/0 conn(0x7f9bcc0b16d0 0x7f9bcc0b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:14.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.770+0000 7f9bd17a6700 1 -- 172.21.15.163:0/685557442 >> v1:172.21.15.181:6789/0 conn(0x7f9bcc15d240 legacy=0x7f9bcc15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:14.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.770+0000 7f9bd17a6700 1 -- 172.21.15.163:0/685557442 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bcc0b5f20 con 0x7f9bcc0b4e70 2023-10-09T06:38:14.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.770+0000 7f9bd0fa5700 1 --2- 172.21.15.163:0/685557442 >> v2:172.21.15.181:3300/0 conn(0x7f9bcc0b16d0 0x7f9bcc0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:38:14.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.770+0000 7f9bd17a6700 1 --2- 172.21.15.163:0/685557442 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bcc0b4e70 0x7f9bcc0ccff0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009e60 tx=0x7f9bc8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=742b44de38c18339 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:14.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.770+0000 7f9bc3fff700 1 -- 172.21.15.163:0/685557442 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc800e070 con 0x7f9bcc0b4e70 2023-10-09T06:38:14.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.770+0000 7f9bc3fff700 1 -- 172.21.15.163:0/685557442 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc800bb60 con 0x7f9bcc0b4e70 2023-10-09T06:38:14.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.770+0000 7f9bc3fff700 1 -- 172.21.15.163:0/685557442 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8012430 con 0x7f9bcc0b4e70 2023-10-09T06:38:14.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.770+0000 7f9bd27a8700 1 -- 172.21.15.163:0/685557442 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bcc0b4e70 msgr2=0x7f9bcc0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:14.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.770+0000 7f9bd27a8700 1 --2- 172.21.15.163:0/685557442 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bcc0b4e70 0x7f9bcc0ccff0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009e60 tx=0x7f9bc8009780 comp rx=0 tx=0).stop 2023-10-09T06:38:14.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.770+0000 7f9bd27a8700 1 -- 172.21.15.163:0/685557442 shutdown_connections 2023-10-09T06:38:14.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.770+0000 7f9bd27a8700 1 --2- 172.21.15.163:0/685557442 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bcc0b4e70 0x7f9bcc0ccff0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:14.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.770+0000 7f9bd27a8700 1 --2- 172.21.15.163:0/685557442 >> v2:172.21.15.181:3300/0 conn(0x7f9bcc0b16d0 0x7f9bcc0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:14.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.770+0000 7f9bd27a8700 1 -- 172.21.15.163:0/685557442 >> 172.21.15.163:0/685557442 conn(0x7f9bcc0d10c0 msgr2=0x7f9bcc0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:14.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.771+0000 7f9bd27a8700 1 -- 172.21.15.163:0/685557442 shutdown_connections 2023-10-09T06:38:14.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.771+0000 7f9bd27a8700 1 -- 172.21.15.163:0/685557442 wait complete. 2023-10-09T06:38:14.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.772+0000 7f9bd27a8700 1 Processor -- start 2023-10-09T06:38:14.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.772+0000 7f9bd27a8700 1 -- start start 2023-10-09T06:38:14.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.772+0000 7f9bd27a8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bcc0b16d0 0x7f9bcc153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:14.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.772+0000 7f9bd27a8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bcc0b4e70 0x7f9bcc1536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:14.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.772+0000 7f9bd27a8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bcc15d240 0x7f9bcc16d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:14.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.772+0000 7f9bd27a8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc155fb0 con 0x7f9bcc0b4e70 2023-10-09T06:38:14.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.772+0000 7f9bd27a8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc1512d0 con 0x7f9bcc0b16d0 2023-10-09T06:38:14.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.772+0000 7f9bd27a8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc155cd0 con 0x7f9bcc15d240 2023-10-09T06:38:14.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.773+0000 7f9bd0fa5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bcc0b4e70 0x7f9bcc1536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:14.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.773+0000 7f9bd0fa5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bcc0b4e70 0x7f9bcc1536b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:60126/0 (socket says 172.21.15.163:60126) 2023-10-09T06:38:14.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.773+0000 7f9bd17a6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bcc0b16d0 0x7f9bcc153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:14.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.773+0000 7f9bd0fa5700 1 -- 172.21.15.163:0/1146658056 learned_addr learned my addr 172.21.15.163:0/1146658056 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:14.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.773+0000 7f9bd1fa7700 1 --2- 172.21.15.163:0/1146658056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bcc15d240 0x7f9bcc16d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:14.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.773+0000 7f9bd0fa5700 1 -- 172.21.15.163:0/1146658056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bcc15d240 msgr2=0x7f9bcc16d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:14.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.773+0000 7f9bd0fa5700 1 --2- 172.21.15.163:0/1146658056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bcc15d240 0x7f9bcc16d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:14.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.773+0000 7f9bd0fa5700 1 -- 172.21.15.163:0/1146658056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bcc0b16d0 msgr2=0x7f9bcc153170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:14.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.773+0000 7f9bd0fa5700 1 --2- 172.21.15.163:0/1146658056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bcc0b16d0 0x7f9bcc153170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:14.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.773+0000 7f9bd0fa5700 1 -- 172.21.15.163:0/1146658056 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bbc009420 con 0x7f9bcc0b4e70 2023-10-09T06:38:14.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.773+0000 7f9bd17a6700 1 --2- 172.21.15.163:0/1146658056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bcc0b16d0 0x7f9bcc153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:38:14.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.773+0000 7f9bd0fa5700 1 --2- 172.21.15.163:0/1146658056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bcc0b4e70 0x7f9bcc1536b0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc00ec80 tx=0x7f9bbc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:14.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.773+0000 7f9bc17fa700 1 -- 172.21.15.163:0/1146658056 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc00c920 con 0x7f9bcc0b4e70 2023-10-09T06:38:14.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.773+0000 7f9bc17fa700 1 -- 172.21.15.163:0/1146658056 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bbc00ca80 con 0x7f9bcc0b4e70 2023-10-09T06:38:14.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.773+0000 7f9bd27a8700 1 -- 172.21.15.163:0/1146658056 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc8009a20 con 0x7f9bcc0b4e70 2023-10-09T06:38:14.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.773+0000 7f9bc17fa700 1 -- 172.21.15.163:0/1146658056 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc00ce00 con 0x7f9bcc0b4e70 2023-10-09T06:38:14.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.773+0000 7f9bd27a8700 1 -- 172.21.15.163:0/1146658056 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bcc150f60 con 0x7f9bcc0b4e70 2023-10-09T06:38:14.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.774+0000 7f9bd27a8700 1 -- 172.21.15.163:0/1146658056 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bcc150ae0 con 0x7f9bcc0b4e70 2023-10-09T06:38:14.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.775+0000 7f9bc17fa700 1 -- 172.21.15.163:0/1146658056 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9bbc013680 con 0x7f9bcc0b4e70 2023-10-09T06:38:14.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.775+0000 7f9bc17fa700 1 --2- 172.21.15.163:0/1146658056 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9bb0067320 0x7f9bb00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:14.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.775+0000 7f9bc17fa700 1 -- 172.21.15.163:0/1146658056 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f9bbc084d50 con 0x7f9bcc0b4e70 2023-10-09T06:38:14.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.775+0000 7f9bd17a6700 1 --2- 172.21.15.163:0/1146658056 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9bb0067320 0x7f9bb00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:14.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.776+0000 7f9bd17a6700 1 --2- 172.21.15.163:0/1146658056 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9bb0067320 0x7f9bb00697d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80097c0 tx=0x7f9bc8019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:14.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.779+0000 7f9bc17fa700 1 -- 172.21.15.163:0/1146658056 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9bcc150ae0 con 0x7f9bcc0b4e70 2023-10-09T06:38:14.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.960+0000 7f9bd27a8700 1 -- 172.21.15.163:0/1146658056 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9bcc150ae0 con 0x7f9bcc0b4e70 2023-10-09T06:38:14.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.964+0000 7f9bc17fa700 1 -- 172.21.15.163:0/1146658056 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9bcc150ae0 con 0x7f9bcc0b4e70 2023-10-09T06:38:14.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.966+0000 7f9bd27a8700 1 -- 172.21.15.163:0/1146658056 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9bb0067320 msgr2=0x7f9bb00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:14.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.966+0000 7f9bd27a8700 1 --2- 172.21.15.163:0/1146658056 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9bb0067320 0x7f9bb00697d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80097c0 tx=0x7f9bc8019000 comp rx=0 tx=0).stop 2023-10-09T06:38:14.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.966+0000 7f9bd27a8700 1 -- 172.21.15.163:0/1146658056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bcc0b4e70 msgr2=0x7f9bcc1536b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:14.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.966+0000 7f9bd27a8700 1 --2- 172.21.15.163:0/1146658056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bcc0b4e70 0x7f9bcc1536b0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc00ec80 tx=0x7f9bbc0091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:14.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.966+0000 7f9bd27a8700 1 -- 172.21.15.163:0/1146658056 shutdown_connections 2023-10-09T06:38:14.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.966+0000 7f9bd27a8700 1 --2- 172.21.15.163:0/1146658056 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9bb0067320 0x7f9bb00697d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:14.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.966+0000 7f9bd27a8700 1 --2- 172.21.15.163:0/1146658056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bcc15d240 0x7f9bcc16d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:14.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.966+0000 7f9bd27a8700 1 --2- 172.21.15.163:0/1146658056 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bcc0b16d0 0x7f9bcc153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:14.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.966+0000 7f9bd27a8700 1 --2- 172.21.15.163:0/1146658056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bcc0b4e70 0x7f9bcc1536b0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:14.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.966+0000 7f9bd27a8700 1 -- 172.21.15.163:0/1146658056 >> 172.21.15.163:0/1146658056 conn(0x7f9bcc0d10c0 msgr2=0x7f9bcc069ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:14.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.966+0000 7f9bd27a8700 1 -- 172.21.15.163:0/1146658056 shutdown_connections 2023-10-09T06:38:14.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:14.966+0000 7f9bd27a8700 1 -- 172.21.15.163:0/1146658056 wait complete. 2023-10-09T06:38:14.979 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-09T06:38:14.980 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:38:15.022 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:38:15.064 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-09T06:38:15.064 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-09T06:38:15.065 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:38:15.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.178+0000 7f2d5359e700 1 Processor -- start 2023-10-09T06:38:15.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.178+0000 7f2d5359e700 1 -- start start 2023-10-09T06:38:15.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.179+0000 7f2d5359e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2d541597f0 0x7f2d540b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:15.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.179+0000 7f2d5359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d5415d230 0x7f2d5415d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:15.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.179+0000 7f2d5359e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d5415dbb0 con 0x7f2d54159420 2023-10-09T06:38:15.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.179+0000 7f2d5359e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d540b11d0 con 0x7f2d541597f0 2023-10-09T06:38:15.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.179+0000 7f2d5359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d540b14b0 con 0x7f2d5415d230 2023-10-09T06:38:15.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.179+0000 7f2d49d9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2d541597f0 0x7f2d540b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:15.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.179+0000 7f2d49d9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2d541597f0 0x7f2d540b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:60134/0 (socket says 172.21.15.163:60134) 2023-10-09T06:38:15.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.179+0000 7f2d49d9b700 1 -- 172.21.15.163:0/2065210484 learned_addr learned my addr 172.21.15.163:0/2065210484 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:15.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.179+0000 7f2d49d9b700 1 -- 172.21.15.163:0/2065210484 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d5415d230 msgr2=0x7f2d5415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:15.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.179+0000 7f2d52d9d700 1 --2- 172.21.15.163:0/2065210484 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d5415d230 0x7f2d5415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:15.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.179+0000 7f2d49d9b700 1 --2- 172.21.15.163:0/2065210484 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d5415d230 0x7f2d5415d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:15.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.179+0000 7f2d49d9b700 1 -- 172.21.15.163:0/2065210484 >> v1:172.21.15.181:6789/0 conn(0x7f2d54159420 legacy=0x7f2d540b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:15.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.179+0000 7f2d49d9b700 1 -- 172.21.15.163:0/2065210484 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d540b1790 con 0x7f2d541597f0 2023-10-09T06:38:15.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.180+0000 7f2d49d9b700 1 --2- 172.21.15.163:0/2065210484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d541597f0 0x7f2d540b2270 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c0099b0 tx=0x7f2d3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6df25690ca8541f server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:15.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.180+0000 7f2d51d9b700 1 -- 172.21.15.163:0/2065210484 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c00b820 con 0x7f2d541597f0 2023-10-09T06:38:15.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.180+0000 7f2d51d9b700 1 -- 172.21.15.163:0/2065210484 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3c00b980 con 0x7f2d541597f0 2023-10-09T06:38:15.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.180+0000 7f2d51d9b700 1 -- 172.21.15.163:0/2065210484 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c00bd20 con 0x7f2d541597f0 2023-10-09T06:38:15.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.180+0000 7f2d5359e700 1 -- 172.21.15.163:0/2065210484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d541597f0 msgr2=0x7f2d540b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:15.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.180+0000 7f2d5359e700 1 --2- 172.21.15.163:0/2065210484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d541597f0 0x7f2d540b2270 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c0099b0 tx=0x7f2d3c009250 comp rx=0 tx=0).stop 2023-10-09T06:38:15.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.180+0000 7f2d5359e700 1 -- 172.21.15.163:0/2065210484 shutdown_connections 2023-10-09T06:38:15.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.180+0000 7f2d5359e700 1 --2- 172.21.15.163:0/2065210484 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d5415d230 0x7f2d5415d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:15.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.180+0000 7f2d5359e700 1 --2- 172.21.15.163:0/2065210484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d541597f0 0x7f2d540b2270 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:15.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.180+0000 7f2d5359e700 1 -- 172.21.15.163:0/2065210484 >> 172.21.15.163:0/2065210484 conn(0x7f2d540d0660 msgr2=0x7f2d540d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:15.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.180+0000 7f2d5359e700 1 -- 172.21.15.163:0/2065210484 shutdown_connections 2023-10-09T06:38:15.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.180+0000 7f2d5359e700 1 -- 172.21.15.163:0/2065210484 wait complete. 2023-10-09T06:38:15.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.182+0000 7f2d5359e700 1 Processor -- start 2023-10-09T06:38:15.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.182+0000 7f2d5359e700 1 -- start start 2023-10-09T06:38:15.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.182+0000 7f2d5359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d54159420 0x7f2d5415b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:15.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.182+0000 7f2d5359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d541597f0 0x7f2d540c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:15.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.182+0000 7f2d5359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d5415d230 0x7f2d540c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:15.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.182+0000 7f2d5359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5415c810 con 0x7f2d541597f0 2023-10-09T06:38:15.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.182+0000 7f2d5359e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5415baf0 con 0x7f2d54159420 2023-10-09T06:38:15.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.182+0000 7f2d5359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5416d6f0 con 0x7f2d5415d230 2023-10-09T06:38:15.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.182+0000 7f2d49d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d541597f0 0x7f2d540c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:15.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.182+0000 7f2d49d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d541597f0 0x7f2d540c6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:60144/0 (socket says 172.21.15.163:60144) 2023-10-09T06:38:15.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.182+0000 7f2d49d9b700 1 -- 172.21.15.163:0/1877208359 learned_addr learned my addr 172.21.15.163:0/1877208359 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:15.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.182+0000 7f2d5259c700 1 --2- 172.21.15.163:0/1877208359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d54159420 0x7f2d5415b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:15.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.183+0000 7f2d52d9d700 1 --2- 172.21.15.163:0/1877208359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d5415d230 0x7f2d540c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:15.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.183+0000 7f2d49d9b700 1 -- 172.21.15.163:0/1877208359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d5415d230 msgr2=0x7f2d540c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:15.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.183+0000 7f2d49d9b700 1 --2- 172.21.15.163:0/1877208359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d5415d230 0x7f2d540c7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:15.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.183+0000 7f2d49d9b700 1 -- 172.21.15.163:0/1877208359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d54159420 msgr2=0x7f2d5415b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:15.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.183+0000 7f2d49d9b700 1 --2- 172.21.15.163:0/1877208359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d54159420 0x7f2d5415b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:15.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.183+0000 7f2d49d9b700 1 -- 172.21.15.163:0/1877208359 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d3c0094f0 con 0x7f2d541597f0 2023-10-09T06:38:15.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.183+0000 7f2d49d9b700 1 --2- 172.21.15.163:0/1877208359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d541597f0 0x7f2d540c6b30 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c009290 tx=0x7f2d3c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:15.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.183+0000 7f2d4affd700 1 -- 172.21.15.163:0/1877208359 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c011cc0 con 0x7f2d541597f0 2023-10-09T06:38:15.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.183+0000 7f2d4affd700 1 -- 172.21.15.163:0/1877208359 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3c020070 con 0x7f2d541597f0 2023-10-09T06:38:15.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.183+0000 7f2d5359e700 1 -- 172.21.15.163:0/1877208359 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d5416d9d0 con 0x7f2d541597f0 2023-10-09T06:38:15.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.183+0000 7f2d4affd700 1 -- 172.21.15.163:0/1877208359 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c01b6a0 con 0x7f2d541597f0 2023-10-09T06:38:15.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.183+0000 7f2d5359e700 1 -- 172.21.15.163:0/1877208359 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d54169cb0 con 0x7f2d541597f0 2023-10-09T06:38:15.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.184+0000 7f2d5359e700 1 -- 172.21.15.163:0/1877208359 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d54169ea0 con 0x7f2d541597f0 2023-10-09T06:38:15.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.184+0000 7f2d4affd700 1 -- 172.21.15.163:0/1877208359 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2d3c022030 con 0x7f2d541597f0 2023-10-09T06:38:15.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.185+0000 7f2d4affd700 1 --2- 172.21.15.163:0/1877208359 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2d34067320 0x7f2d340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:15.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.185+0000 7f2d4affd700 1 -- 172.21.15.163:0/1877208359 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2d3c086810 con 0x7f2d541597f0 2023-10-09T06:38:15.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.185+0000 7f2d5259c700 1 --2- 172.21.15.163:0/1877208359 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2d34067320 0x7f2d340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:15.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.186+0000 7f2d5259c700 1 --2- 172.21.15.163:0/1877208359 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2d34067320 0x7f2d340697d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2d44003a30 tx=0x7f2d44007ea0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:15.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.189+0000 7f2d4affd700 1 -- 172.21.15.163:0/1877208359 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d3c0538b0 con 0x7f2d541597f0 2023-10-09T06:38:15.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.414+0000 7f2d5359e700 1 -- 172.21.15.163:0/1877208359 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d54169ea0 con 0x7f2d541597f0 2023-10-09T06:38:15.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.414+0000 7f2d4affd700 1 -- 172.21.15.163:0/1877208359 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+3541 (secure 0 0 0) 0x7f2d3c056ed0 con 0x7f2d541597f0 2023-10-09T06:38:15.418 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:15.418 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":97,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1554520486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1554520486},{"type":"v1","addr":"172.21.15.181:6837","nonce":1554520486}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":6931,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1695339871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1695339871},{"type":"v1","addr":"172.21.15.181:6836","nonce":1695339871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":7515,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3668692283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3668692283},{"type":"v1","addr":"172.21.15.163:6834","nonce":3668692283}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":7821,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3923579131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3923579131},{"type":"v1","addr":"172.21.15.163:6835","nonce":3923579131}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":97}],"filesystems":[]} 2023-10-09T06:38:15.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.416+0000 7f2d5359e700 1 -- 172.21.15.163:0/1877208359 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2d34067320 msgr2=0x7f2d340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:15.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.416+0000 7f2d5359e700 1 --2- 172.21.15.163:0/1877208359 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2d34067320 0x7f2d340697d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2d44003a30 tx=0x7f2d44007ea0 comp rx=0 tx=0).stop 2023-10-09T06:38:15.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.416+0000 7f2d5359e700 1 -- 172.21.15.163:0/1877208359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d541597f0 msgr2=0x7f2d540c6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:15.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.416+0000 7f2d5359e700 1 --2- 172.21.15.163:0/1877208359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d541597f0 0x7f2d540c6b30 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c009290 tx=0x7f2d3c016000 comp rx=0 tx=0).stop 2023-10-09T06:38:15.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.416+0000 7f2d5359e700 1 -- 172.21.15.163:0/1877208359 shutdown_connections 2023-10-09T06:38:15.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.416+0000 7f2d5359e700 1 --2- 172.21.15.163:0/1877208359 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2d34067320 0x7f2d340697d0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:15.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.416+0000 7f2d5359e700 1 --2- 172.21.15.163:0/1877208359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d5415d230 0x7f2d540c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:15.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.416+0000 7f2d5359e700 1 --2- 172.21.15.163:0/1877208359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d54159420 0x7f2d5415b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:15.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.416+0000 7f2d5359e700 1 --2- 172.21.15.163:0/1877208359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d541597f0 0x7f2d540c6b30 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:15.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.416+0000 7f2d5359e700 1 -- 172.21.15.163:0/1877208359 >> 172.21.15.163:0/1877208359 conn(0x7f2d540d0660 msgr2=0x7f2d54155220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:15.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.416+0000 7f2d5359e700 1 -- 172.21.15.163:0/1877208359 shutdown_connections 2023-10-09T06:38:15.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.416+0000 7f2d5359e700 1 -- 172.21.15.163:0/1877208359 wait complete. 2023-10-09T06:38:15.420 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 97 2023-10-09T06:38:15.429 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-09T06:38:15.429 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-09T06:38:15.430 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-09T06:38:15.430 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:38:15.430 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-09T06:38:15.430 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-09T06:38:15.430 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-09T06:38:15.430 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:38:15.431 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-09T06:38:15.431 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-09T06:38:15.431 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-09T06:38:15.431 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:38:15.431 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-09T06:38:15.431 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-09T06:38:15.431 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-09T06:38:15.432 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:38:15.438 INFO:tasks.ceph.mds.a:Stopped 2023-10-09T06:38:15.438 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-09T06:38:15.439 INFO:tasks.ceph.mds.c:Stopped 2023-10-09T06:38:15.439 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-09T06:38:15.442 INFO:tasks.ceph.mds.b:Stopped 2023-10-09T06:38:15.442 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-09T06:38:15.442 INFO:tasks.ceph.mds.d:Stopped 2023-10-09T06:38:15.443 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-09T06:38:15.480 INFO:tasks.ceph.mds.a:Started 2023-10-09T06:38:15.480 DEBUG:teuthology.parallel:result is None 2023-10-09T06:38:15.481 INFO:tasks.ceph.mds.c:Started 2023-10-09T06:38:15.481 DEBUG:teuthology.parallel:result is None 2023-10-09T06:38:15.483 INFO:tasks.ceph.mds.b:Started 2023-10-09T06:38:15.484 DEBUG:teuthology.parallel:result is None 2023-10-09T06:38:15.484 INFO:tasks.ceph.mds.d:Started 2023-10-09T06:38:15.484 DEBUG:teuthology.parallel:result is None 2023-10-09T06:38:15.485 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-09T06:38:15.553 INFO:tasks.ceph.mds.a.smithi163.stdout:starting mds.a at 2023-10-09T06:38:15.554 INFO:tasks.ceph.mds.c.smithi163.stdout:starting mds.c at 2023-10-09T06:38:15.564 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-09T06:38:15.564 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-09T06:38:15.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.605+0000 7f3fcb59e700 1 Processor -- start 2023-10-09T06:38:15.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.605+0000 7f3fcb59e700 1 -- start start 2023-10-09T06:38:15.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.606+0000 7f3fcb59e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3fcc157380 0x7f3fcc157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:15.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.606+0000 7f3fcb59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fcc0c6b30 0x7f3fcc15bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:15.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.606+0000 7f3fcb59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fcc15c110 con 0x7f3fcc0ae840 2023-10-09T06:38:15.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.606+0000 7f3fcb59e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fcc06ea60 con 0x7f3fcc157380 2023-10-09T06:38:15.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.606+0000 7f3fcb59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc06ed40 con 0x7f3fcc0c6b30 2023-10-09T06:38:15.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.606+0000 7f3fcad9d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f3fcc0ae840 0x7f3fcc0aec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:51932/0 (socket says 172.21.15.163:51932) 2023-10-09T06:38:15.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.606+0000 7f3fcad9d700 1 -- 172.21.15.163:0/496931576 learned_addr learned my addr 172.21.15.163:0/496931576 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:15.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.606+0000 7f3fca59c700 1 --2- 172.21.15.163:0/496931576 >> v2:172.21.15.181:3300/0 conn(0x7f3fcc157380 0x7f3fcc157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:15.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.606+0000 7f3fc9d9b700 1 --2- 172.21.15.163:0/496931576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fcc0c6b30 0x7f3fcc15bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:15.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.606+0000 7f3fc959a700 1 -- 172.21.15.163:0/496931576 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1005473249 0 0) 0x7f3fcc15c110 con 0x7f3fcc0ae840 2023-10-09T06:38:15.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.607+0000 7f3fc959a700 1 -- 172.21.15.163:0/496931576 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fb4003570 con 0x7f3fcc0ae840 2023-10-09T06:38:15.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.607+0000 7f3fc959a700 1 -- 172.21.15.163:0/496931576 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3318750737 0 0) 0x7f3fb4003570 con 0x7f3fcc0ae840 2023-10-09T06:38:15.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.607+0000 7f3fc9d9b700 1 -- 172.21.15.163:0/496931576 >> v2:172.21.15.181:3300/0 conn(0x7f3fcc157380 msgr2=0x7f3fcc157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:15.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.607+0000 7f3fc9d9b700 1 --2- 172.21.15.163:0/496931576 >> v2:172.21.15.181:3300/0 conn(0x7f3fcc157380 0x7f3fcc157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:15.610 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.607+0000 7f3fc9d9b700 1 -- 172.21.15.163:0/496931576 >> v1:172.21.15.163:6789/0 conn(0x7f3fcc0ae840 legacy=0x7f3fcc0aec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:15.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.607+0000 7f3fc9d9b700 1 -- 172.21.15.163:0/496931576 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fcc0afe60 con 0x7f3fcc0c6b30 2023-10-09T06:38:15.611 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.607+0000 7f3fca59c700 1 --2- 172.21.15.163:0/496931576 >> v2:172.21.15.181:3300/0 conn(0x7f3fcc157380 0x7f3fcc157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:38:15.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.607+0000 7f3fc9d9b700 1 --2- 172.21.15.163:0/496931576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fcc0c6b30 0x7f3fcc15bbd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00098e0 tx=0x7f3fc00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1e6251cb1881a00 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:15.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.607+0000 7f3fc959a700 1 -- 172.21.15.163:0/496931576 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4001470 con 0x7f3fcc0c6b30 2023-10-09T06:38:15.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.607+0000 7f3fc959a700 1 -- 172.21.15.163:0/496931576 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc0011420 con 0x7f3fcc0c6b30 2023-10-09T06:38:15.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.607+0000 7f3fc959a700 1 -- 172.21.15.163:0/496931576 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc00117c0 con 0x7f3fcc0c6b30 2023-10-09T06:38:15.612 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.608+0000 7f3fcb59e700 1 -- 172.21.15.163:0/496931576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fcc0c6b30 msgr2=0x7f3fcc15bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:15.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.608+0000 7f3fcb59e700 1 --2- 172.21.15.163:0/496931576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fcc0c6b30 0x7f3fcc15bbd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00098e0 tx=0x7f3fc00091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:15.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.608+0000 7f3fcb59e700 1 -- 172.21.15.163:0/496931576 shutdown_connections 2023-10-09T06:38:15.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.608+0000 7f3fcb59e700 1 --2- 172.21.15.163:0/496931576 >> v2:172.21.15.181:3300/0 conn(0x7f3fcc157380 0x7f3fcc157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:15.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.608+0000 7f3fcb59e700 1 --2- 172.21.15.163:0/496931576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fcc0c6b30 0x7f3fcc15bbd0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:15.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.608+0000 7f3fcb59e700 1 -- 172.21.15.163:0/496931576 >> 172.21.15.163:0/496931576 conn(0x7f3fcc06ddb0 msgr2=0x7f3fcc06e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:15.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.608+0000 7f3fcb59e700 1 -- 172.21.15.163:0/496931576 shutdown_connections 2023-10-09T06:38:15.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.608+0000 7f3fcb59e700 1 -- 172.21.15.163:0/496931576 wait complete. 2023-10-09T06:38:15.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.609+0000 7f3fcb59e700 1 Processor -- start 2023-10-09T06:38:15.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.609+0000 7f3fcb59e700 1 -- start start 2023-10-09T06:38:15.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.610+0000 7f3fcb59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fcc0ae840 0x7f3fcc145580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:15.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.610+0000 7f3fcb59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fcc0c6b30 0x7f3fcc1682b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:15.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.610+0000 7f3fcb59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fcc157380 0x7f3fcc06a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:15.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.610+0000 7f3fcb59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc145ac0 con 0x7f3fcc0ae840 2023-10-09T06:38:15.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.610+0000 7f3fcb59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc1687f0 con 0x7f3fcc157380 2023-10-09T06:38:15.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.610+0000 7f3fcb59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc068d30 con 0x7f3fcc0c6b30 2023-10-09T06:38:15.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.610+0000 7f3fca59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fcc0ae840 0x7f3fcc145580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:15.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.610+0000 7f3fca59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fcc0ae840 0x7f3fcc145580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:60170/0 (socket says 172.21.15.163:60170) 2023-10-09T06:38:15.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.610+0000 7f3fca59c700 1 -- 172.21.15.163:0/2225463980 learned_addr learned my addr 172.21.15.163:0/2225463980 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:15.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.610+0000 7f3fcad9d700 1 --2- 172.21.15.163:0/2225463980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fcc157380 0x7f3fcc06a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:15.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.610+0000 7f3fc9d9b700 1 --2- 172.21.15.163:0/2225463980 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fcc0c6b30 0x7f3fcc1682b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:15.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.610+0000 7f3fca59c700 1 -- 172.21.15.163:0/2225463980 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fcc0c6b30 msgr2=0x7f3fcc1682b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:15.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.610+0000 7f3fca59c700 1 --2- 172.21.15.163:0/2225463980 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fcc0c6b30 0x7f3fcc1682b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:15.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.610+0000 7f3fca59c700 1 -- 172.21.15.163:0/2225463980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fcc157380 msgr2=0x7f3fcc06a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:15.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.610+0000 7f3fca59c700 1 --2- 172.21.15.163:0/2225463980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fcc157380 0x7f3fcc06a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:15.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.610+0000 7f3fca59c700 1 -- 172.21.15.163:0/2225463980 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fbc009a20 con 0x7f3fcc0ae840 2023-10-09T06:38:15.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.610+0000 7f3fca59c700 1 --2- 172.21.15.163:0/2225463980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fcc0ae840 0x7f3fcc145580 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc00c0c0 tx=0x7f3fbc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:15.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.610+0000 7f3fba7fc700 1 -- 172.21.15.163:0/2225463980 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc00cc00 con 0x7f3fcc0ae840 2023-10-09T06:38:15.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.610+0000 7f3fba7fc700 1 -- 172.21.15.163:0/2225463980 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fbc00f040 con 0x7f3fcc0ae840 2023-10-09T06:38:15.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.610+0000 7f3fcb59e700 1 -- 172.21.15.163:0/2225463980 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc0009420 con 0x7f3fcc0ae840 2023-10-09T06:38:15.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.610+0000 7f3fba7fc700 1 -- 172.21.15.163:0/2225463980 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc014480 con 0x7f3fcc0ae840 2023-10-09T06:38:15.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.610+0000 7f3fcb59e700 1 -- 172.21.15.163:0/2225463980 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fcc0692f0 con 0x7f3fcc0ae840 2023-10-09T06:38:15.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.611+0000 7f3fcb59e700 1 -- 172.21.15.163:0/2225463980 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fcc069c20 con 0x7f3fcc0ae840 2023-10-09T06:38:15.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.612+0000 7f3fba7fc700 1 -- 172.21.15.163:0/2225463980 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3fbc0147f0 con 0x7f3fcc0ae840 2023-10-09T06:38:15.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.612+0000 7f3fba7fc700 1 --2- 172.21.15.163:0/2225463980 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3fac067320 0x7f3fac0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:15.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.612+0000 7f3fba7fc700 1 -- 172.21.15.163:0/2225463980 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f3fbc085780 con 0x7f3fcc0ae840 2023-10-09T06:38:15.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.612+0000 7f3fc9d9b700 1 --2- 172.21.15.163:0/2225463980 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3fac067320 0x7f3fac0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:15.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.612+0000 7f3fc9d9b700 1 --2- 172.21.15.163:0/2225463980 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3fac067320 0x7f3fac0697d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc0c6f70 tx=0x7f3fc0016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:15.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.616+0000 7f3fba7fc700 1 -- 172.21.15.163:0/2225463980 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3fbc052600 con 0x7f3fcc0ae840 2023-10-09T06:38:15.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:15.817+0000 7f3fcb59e700 1 -- 172.21.15.163:0/2225463980 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3fcc04c3c0 con 0x7f3fcc0ae840 2023-10-09T06:38:16.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.655+0000 7f3fba7fc700 1 -- 172.21.15.163:0/2225463980 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v148) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3fbc055c20 con 0x7f3fcc0ae840 2023-10-09T06:38:16.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.658+0000 7f3fcb59e700 1 -- 172.21.15.163:0/2225463980 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3fac067320 msgr2=0x7f3fac0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:16.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.658+0000 7f3fcb59e700 1 --2- 172.21.15.163:0/2225463980 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3fac067320 0x7f3fac0697d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc0c6f70 tx=0x7f3fc0016000 comp rx=0 tx=0).stop 2023-10-09T06:38:16.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.658+0000 7f3fcb59e700 1 -- 172.21.15.163:0/2225463980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fcc0ae840 msgr2=0x7f3fcc145580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:16.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.658+0000 7f3fcb59e700 1 --2- 172.21.15.163:0/2225463980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fcc0ae840 0x7f3fcc145580 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc00c0c0 tx=0x7f3fbc00b040 comp rx=0 tx=0).stop 2023-10-09T06:38:16.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.658+0000 7f3fcb59e700 1 -- 172.21.15.163:0/2225463980 shutdown_connections 2023-10-09T06:38:16.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.658+0000 7f3fcb59e700 1 --2- 172.21.15.163:0/2225463980 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3fac067320 0x7f3fac0697d0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:16.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.658+0000 7f3fcb59e700 1 --2- 172.21.15.163:0/2225463980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fcc0ae840 0x7f3fcc145580 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:16.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.658+0000 7f3fcb59e700 1 --2- 172.21.15.163:0/2225463980 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fcc157380 0x7f3fcc06a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:16.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.658+0000 7f3fcb59e700 1 --2- 172.21.15.163:0/2225463980 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fcc0c6b30 0x7f3fcc1682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:16.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.658+0000 7f3fcb59e700 1 -- 172.21.15.163:0/2225463980 >> 172.21.15.163:0/2225463980 conn(0x7f3fcc06ddb0 msgr2=0x7f3fcc156390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:16.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.658+0000 7f3fcb59e700 1 -- 172.21.15.163:0/2225463980 shutdown_connections 2023-10-09T06:38:16.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.658+0000 7f3fcb59e700 1 -- 172.21.15.163:0/2225463980 wait complete. 2023-10-09T06:38:16.661 INFO:teuthology.orchestra.run.smithi163.stderr: removed all blocklist entries 2023-10-09T06:38:16.673 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-09T06:38:16.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.790+0000 7f9199dca700 1 Processor -- start 2023-10-09T06:38:16.791 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.790+0000 7f9199dca700 1 -- start start 2023-10-09T06:38:16.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.791+0000 7f9199dca700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9194159420 0x7f91940b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:16.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.791+0000 7f9199dca700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91941597f0 0x7f91940b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:16.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.791+0000 7f9199dca700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91940b11d0 con 0x7f919415d230 2023-10-09T06:38:16.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.791+0000 7f9199dca700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91940b14b0 con 0x7f9194159420 2023-10-09T06:38:16.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.791+0000 7f9199dca700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91940b1790 con 0x7f91941597f0 2023-10-09T06:38:16.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.791+0000 7f9198dc8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9194159420 0x7f91940b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:16.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.791+0000 7f91995c9700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f919415d230 0x7f91940b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:48748/0 (socket says 172.21.15.163:48748) 2023-10-09T06:38:16.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.791+0000 7f9193fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91941597f0 0x7f91940b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:16.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.791+0000 7f91995c9700 1 -- 172.21.15.163:0/3252775878 learned_addr learned my addr 172.21.15.163:0/3252775878 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:16.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.791+0000 7f9193fff700 1 -- 172.21.15.163:0/3252775878 >> v2:172.21.15.181:3300/0 conn(0x7f9194159420 msgr2=0x7f91940b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:16.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.791+0000 7f9193fff700 1 --2- 172.21.15.163:0/3252775878 >> v2:172.21.15.181:3300/0 conn(0x7f9194159420 0x7f91940b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:16.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.791+0000 7f9193fff700 1 -- 172.21.15.163:0/3252775878 >> v1:172.21.15.181:6789/0 conn(0x7f919415d230 legacy=0x7f91940b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:16.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.791+0000 7f9193fff700 1 -- 172.21.15.163:0/3252775878 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91940b1a70 con 0x7f91941597f0 2023-10-09T06:38:16.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.791+0000 7f9198dc8700 1 --2- 172.21.15.163:0/3252775878 >> v2:172.21.15.181:3300/0 conn(0x7f9194159420 0x7f91940b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:38:16.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.792+0000 7f9193fff700 1 --2- 172.21.15.163:0/3252775878 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91941597f0 0x7f91940b2270 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f91800098e0 tx=0x7f91800091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4965d308526d6a32 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:16.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.792+0000 7f91937fe700 1 -- 172.21.15.163:0/3252775878 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918000b640 con 0x7f91941597f0 2023-10-09T06:38:16.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.792+0000 7f91937fe700 1 -- 172.21.15.163:0/3252775878 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f918000b7a0 con 0x7f91941597f0 2023-10-09T06:38:16.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.792+0000 7f91937fe700 1 -- 172.21.15.163:0/3252775878 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918000bb40 con 0x7f91941597f0 2023-10-09T06:38:16.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.792+0000 7f9199dca700 1 -- 172.21.15.163:0/3252775878 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91941597f0 msgr2=0x7f91940b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:16.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.792+0000 7f9199dca700 1 --2- 172.21.15.163:0/3252775878 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91941597f0 0x7f91940b2270 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f91800098e0 tx=0x7f91800091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:16.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.792+0000 7f9199dca700 1 -- 172.21.15.163:0/3252775878 shutdown_connections 2023-10-09T06:38:16.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.792+0000 7f9199dca700 1 --2- 172.21.15.163:0/3252775878 >> v2:172.21.15.181:3300/0 conn(0x7f9194159420 0x7f91940b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:16.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.792+0000 7f9199dca700 1 --2- 172.21.15.163:0/3252775878 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91941597f0 0x7f91940b2270 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:16.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.792+0000 7f9199dca700 1 -- 172.21.15.163:0/3252775878 >> 172.21.15.163:0/3252775878 conn(0x7f91940d0660 msgr2=0x7f91940d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:16.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.792+0000 7f9199dca700 1 -- 172.21.15.163:0/3252775878 shutdown_connections 2023-10-09T06:38:16.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.792+0000 7f9199dca700 1 -- 172.21.15.163:0/3252775878 wait complete. 2023-10-09T06:38:16.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.793+0000 7f9199dca700 1 Processor -- start 2023-10-09T06:38:16.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.794+0000 7f9199dca700 1 -- start start 2023-10-09T06:38:16.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.794+0000 7f9199dca700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9194159420 0x7f919416a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:16.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.794+0000 7f9199dca700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91941597f0 0x7f91940ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:16.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.794+0000 7f9199dca700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f919415d230 0x7f91940c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:16.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.794+0000 7f9199dca700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91940cb640 con 0x7f91941597f0 2023-10-09T06:38:16.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.794+0000 7f9199dca700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91940caac0 con 0x7f919415d230 2023-10-09T06:38:16.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.794+0000 7f9199dca700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91940cada0 con 0x7f9194159420 2023-10-09T06:38:16.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.794+0000 7f9193fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91941597f0 0x7f91940ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:16.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.794+0000 7f9193fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91941597f0 0x7f91940ca070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:60180/0 (socket says 172.21.15.163:60180) 2023-10-09T06:38:16.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.794+0000 7f9193fff700 1 -- 172.21.15.163:0/1650059586 learned_addr learned my addr 172.21.15.163:0/1650059586 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:16.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.795+0000 7f91995c9700 1 --2- 172.21.15.163:0/1650059586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f919415d230 0x7f91940c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:16.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.795+0000 7f9198dc8700 1 --2- 172.21.15.163:0/1650059586 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9194159420 0x7f919416a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:16.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.795+0000 7f9193fff700 1 -- 172.21.15.163:0/1650059586 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9194159420 msgr2=0x7f919416a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:16.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.795+0000 7f9193fff700 1 --2- 172.21.15.163:0/1650059586 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9194159420 0x7f919416a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:16.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.795+0000 7f9193fff700 1 -- 172.21.15.163:0/1650059586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f919415d230 msgr2=0x7f91940c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:16.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.795+0000 7f9193fff700 1 --2- 172.21.15.163:0/1650059586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f919415d230 0x7f91940c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:16.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.795+0000 7f9193fff700 1 -- 172.21.15.163:0/1650059586 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9188009420 con 0x7f91941597f0 2023-10-09T06:38:16.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.795+0000 7f9193fff700 1 --2- 172.21.15.163:0/1650059586 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91941597f0 0x7f91940ca070 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f91940b18d0 tx=0x7f9180017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:16.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.795+0000 7f9190ff9700 1 -- 172.21.15.163:0/1650059586 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918001bc40 con 0x7f91941597f0 2023-10-09T06:38:16.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.795+0000 7f9190ff9700 1 -- 172.21.15.163:0/1650059586 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f918001bde0 con 0x7f91941597f0 2023-10-09T06:38:16.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.795+0000 7f9199dca700 1 -- 172.21.15.163:0/1650059586 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9180009420 con 0x7f91941597f0 2023-10-09T06:38:16.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.795+0000 7f9190ff9700 1 -- 172.21.15.163:0/1650059586 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180011670 con 0x7f91941597f0 2023-10-09T06:38:16.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.795+0000 7f9199dca700 1 -- 172.21.15.163:0/1650059586 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91940cb360 con 0x7f91941597f0 2023-10-09T06:38:16.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.796+0000 7f9199dca700 1 -- 172.21.15.163:0/1650059586 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f919404c3c0 con 0x7f91941597f0 2023-10-09T06:38:16.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.796+0000 7f9190ff9700 1 -- 172.21.15.163:0/1650059586 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f91800119e0 con 0x7f91941597f0 2023-10-09T06:38:16.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.797+0000 7f9190ff9700 1 --2- 172.21.15.163:0/1650059586 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9178067320 0x7f91780697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:16.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.797+0000 7f9190ff9700 1 -- 172.21.15.163:0/1650059586 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f9180085fe0 con 0x7f91941597f0 2023-10-09T06:38:16.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.797+0000 7f9198dc8700 1 --2- 172.21.15.163:0/1650059586 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9178067320 0x7f91780697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:16.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.798+0000 7f9198dc8700 1 --2- 172.21.15.163:0/1650059586 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9178067320 0x7f91780697d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f91940cc5b0 tx=0x7f91880091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:16.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:16.801+0000 7f9190ff9700 1 -- 172.21.15.163:0/1650059586 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9180052fc0 con 0x7f91941597f0 2023-10-09T06:38:17.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.058+0000 7f9199dca700 1 -- 172.21.15.163:0/1650059586 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f91940b1d50 con 0x7f91941597f0 2023-10-09T06:38:17.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.059+0000 7f9190ff9700 1 -- 172.21.15.163:0/1650059586 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f91800565e0 con 0x7f91941597f0 2023-10-09T06:38:17.060 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:17.060 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:38:17.060 INFO:teuthology.orchestra.run.smithi163.stdout: "auth_dump": [ 2023-10-09T06:38:17.060 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:17.060 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.a", 2023-10-09T06:38:17.060 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNltXtwDxAAlnrWRsfnZDLEmsJVowh3Mg==", 2023-10-09T06:38:17.060 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:17.060 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:38:17.060 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:38:17.060 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:38:17.061 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:17.061 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:17.061 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:17.061 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:17.061 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.b", 2023-10-09T06:38:17.061 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlTdGhGhAAXuldy0HrFd+44sSu/EV3qA==", 2023-10-09T06:38:17.061 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:17.061 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:38:17.061 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:38:17.061 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:38:17.061 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:17.061 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:17.061 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:17.062 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:17.062 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.c", 2023-10-09T06:38:17.062 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlGis7FhAAYG76TiOFX8QkTwjNX2+Ncw==", 2023-10-09T06:38:17.062 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:17.062 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:38:17.062 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:38:17.062 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:38:17.062 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:17.062 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:17.062 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:17.062 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:17.063 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.d", 2023-10-09T06:38:17.063 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlKhRuIRAAfhG5Gpc7klXJloYXepOhsg==", 2023-10-09T06:38:17.063 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:17.063 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:38:17.063 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:38:17.063 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:38:17.063 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:17.063 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:17.063 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:17.063 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:17.063 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.0", 2023-10-09T06:38:17.063 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDcnSNlrQeFOxAA+s6UitrTn5TK1N7kgySoXw==", 2023-10-09T06:38:17.064 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:17.064 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:38:17.064 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:38:17.064 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:17.064 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:17.064 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:17.064 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:17.064 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.1", 2023-10-09T06:38:17.064 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDfnSNlleccORAAnB+lNUFblivdoY44JfELxA==", 2023-10-09T06:38:17.064 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:17.064 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:38:17.064 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:38:17.064 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:17.065 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:17.065 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:17.065 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:17.065 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.2", 2023-10-09T06:38:17.065 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDinSNlFoB4NxAAT0+trRBij5rqgc+F0eWUfQ==", 2023-10-09T06:38:17.065 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:17.065 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:38:17.065 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:38:17.065 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:17.065 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:17.065 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:17.065 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:17.066 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.3", 2023-10-09T06:38:17.066 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDlnSNlRRQ6NhAABoBtrFjSvQujolq0Pxrd1Q==", 2023-10-09T06:38:17.066 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:17.066 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:38:17.066 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:38:17.066 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:17.066 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:17.066 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:17.067 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:17.067 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.4", 2023-10-09T06:38:17.067 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDrnSNl308uOxAA5OJyPLGQmVSJqha5d+nKKQ==", 2023-10-09T06:38:17.067 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:17.067 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:38:17.067 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:38:17.067 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:17.067 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:17.067 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:17.067 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:17.068 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.5", 2023-10-09T06:38:17.068 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDunSNlHnK+OBAAU7l4dCJ82SbGSzBEm8Axzw==", 2023-10-09T06:38:17.068 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:17.068 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:38:17.068 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:38:17.068 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:17.068 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:17.068 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:17.068 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:17.068 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.6", 2023-10-09T06:38:17.068 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDxnSNlwUSLOBAAN5N+F5ho+Co8pBh4Rz+lXQ==", 2023-10-09T06:38:17.068 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:17.068 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:38:17.069 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:38:17.069 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:17.069 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:17.069 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:17.069 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:17.069 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.7", 2023-10-09T06:38:17.069 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQD0nSNl2gxGORAAoE8OLGEifcEx/km8rfSV9g==", 2023-10-09T06:38:17.069 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:17.069 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:38:17.069 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:38:17.069 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:17.069 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:17.070 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:17.070 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:17.070 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.0", 2023-10-09T06:38:17.070 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlnxY5JRAApr6czcpMdQYYtXQ9yg2q6A==", 2023-10-09T06:38:17.070 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:17.070 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:38:17.070 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow r", 2023-10-09T06:38:17.070 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-09T06:38:17.070 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:17.070 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:17.070 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:17.070 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.1", 2023-10-09T06:38:17.070 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlk6ybLRAAuShqTDAtymGdxN8KBdZDDA==", 2023-10-09T06:38:17.071 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:17.071 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:38:17.071 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:38:17.071 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:38:17.071 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:38:17.071 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:17.071 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:17.071 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:17.071 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.2", 2023-10-09T06:38:17.071 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlYyOmKhAAQmpr+mGfJH4GUa28dtlJwA==", 2023-10-09T06:38:17.071 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:17.071 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:38:17.072 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:38:17.072 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:38:17.072 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:38:17.072 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:17.072 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:17.072 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:17.072 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.3", 2023-10-09T06:38:17.072 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlD2H1MhAA4zvaXY/Gd0kndzi+QSwp7w==", 2023-10-09T06:38:17.072 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:17.072 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:38:17.072 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:38:17.072 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:38:17.072 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:38:17.073 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:17.073 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:17.073 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:17.073 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.admin", 2023-10-09T06:38:17.073 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDYnSNlsHH4LxAA85xeFEP1CUwoblaePYk5zA==", 2023-10-09T06:38:17.073 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:17.073 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:38:17.073 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:38:17.073 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:38:17.073 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:17.073 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:17.073 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:17.073 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:17.074 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mgr.x", 2023-10-09T06:38:17.074 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNl2sloDBAA0FoWN9iovCl6783o3/btiA==", 2023-10-09T06:38:17.074 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:17.074 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:38:17.074 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile mgr", 2023-10-09T06:38:17.074 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:17.074 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:17.074 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:17.074 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:17.074 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mgr.y", 2023-10-09T06:38:17.074 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlcFWVCBAAR2dKPg01ciTiNpSX0JIKFg==", 2023-10-09T06:38:17.074 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:17.075 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:38:17.075 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile mgr", 2023-10-09T06:38:17.075 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:17.075 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:17.075 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:17.075 INFO:teuthology.orchestra.run.smithi163.stdout: ] 2023-10-09T06:38:17.075 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:38:17.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.061+0000 7f9199dca700 1 -- 172.21.15.163:0/1650059586 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9178067320 msgr2=0x7f91780697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:17.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.061+0000 7f9199dca700 1 --2- 172.21.15.163:0/1650059586 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9178067320 0x7f91780697d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f91940cc5b0 tx=0x7f91880091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.061+0000 7f9199dca700 1 -- 172.21.15.163:0/1650059586 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91941597f0 msgr2=0x7f91940ca070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:17.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.061+0000 7f9199dca700 1 --2- 172.21.15.163:0/1650059586 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91941597f0 0x7f91940ca070 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f91940b18d0 tx=0x7f9180017000 comp rx=0 tx=0).stop 2023-10-09T06:38:17.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.061+0000 7f9199dca700 1 -- 172.21.15.163:0/1650059586 shutdown_connections 2023-10-09T06:38:17.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.061+0000 7f9199dca700 1 --2- 172.21.15.163:0/1650059586 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9178067320 0x7f91780697d0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.061+0000 7f9199dca700 1 --2- 172.21.15.163:0/1650059586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f919415d230 0x7f91940c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.061+0000 7f9199dca700 1 --2- 172.21.15.163:0/1650059586 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9194159420 0x7f919416a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.061+0000 7f9199dca700 1 --2- 172.21.15.163:0/1650059586 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91941597f0 0x7f91940ca070 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.061+0000 7f9199dca700 1 -- 172.21.15.163:0/1650059586 >> 172.21.15.163:0/1650059586 conn(0x7f91940d0660 msgr2=0x7f91940698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:17.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.061+0000 7f9199dca700 1 -- 172.21.15.163:0/1650059586 shutdown_connections 2023-10-09T06:38:17.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.061+0000 7f9199dca700 1 -- 172.21.15.163:0/1650059586 wait complete. 2023-10-09T06:38:17.078 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:38:17.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.192+0000 7fa3d6b46700 1 Processor -- start 2023-10-09T06:38:17.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.192+0000 7fa3d6b46700 1 -- start start 2023-10-09T06:38:17.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.193+0000 7fa3d6b46700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa3d0159420 0x7fa3d00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:17.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.193+0000 7fa3d6b46700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa3d015d230 0x7fa3d015d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:17.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.193+0000 7fa3d6b46700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3d00b2980 con 0x7fa3d01597f0 2023-10-09T06:38:17.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.193+0000 7fa3d6b46700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3d015db70 con 0x7fa3d0159420 2023-10-09T06:38:17.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.193+0000 7fa3d6b46700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3d00b11d0 con 0x7fa3d015d230 2023-10-09T06:38:17.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.193+0000 7fa3d5b44700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa3d0159420 0x7fa3d00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:17.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.193+0000 7fa3d5343700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fa3d01597f0 0x7fa3d00b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:51946/0 (socket says 172.21.15.163:51946) 2023-10-09T06:38:17.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.193+0000 7fa3d5343700 1 -- 172.21.15.163:0/1988315276 learned_addr learned my addr 172.21.15.163:0/1988315276 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:17.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.193+0000 7fa3d6345700 1 --2- 172.21.15.163:0/1988315276 >> v2:172.21.15.181:3300/0 conn(0x7fa3d015d230 0x7fa3d015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:17.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.193+0000 7fa3d5b44700 1 -- 172.21.15.163:0/1988315276 >> v2:172.21.15.181:3300/0 conn(0x7fa3d015d230 msgr2=0x7fa3d015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:17.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.193+0000 7fa3d5b44700 1 --2- 172.21.15.163:0/1988315276 >> v2:172.21.15.181:3300/0 conn(0x7fa3d015d230 0x7fa3d015d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.193+0000 7fa3d5b44700 1 -- 172.21.15.163:0/1988315276 >> v1:172.21.15.163:6789/0 conn(0x7fa3d01597f0 legacy=0x7fa3d00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:17.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.193+0000 7fa3d5b44700 1 -- 172.21.15.163:0/1988315276 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d00b14b0 con 0x7fa3d0159420 2023-10-09T06:38:17.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.193+0000 7fa3d4b42700 1 -- 172.21.15.163:0/1988315276 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1095772319 0 0) 0x7fa3d00b2980 con 0x7fa3d01597f0 2023-10-09T06:38:17.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.194+0000 7fa3d5b44700 1 --2- 172.21.15.163:0/1988315276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3d0159420 0x7fa3d00b5f40 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fa3d015d170 tx=0x7fa3cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0bfe569ea12a4b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:17.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.194+0000 7fa3d4b42700 1 -- 172.21.15.163:0/1988315276 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc00b6b0 con 0x7fa3d0159420 2023-10-09T06:38:17.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.194+0000 7fa3d4b42700 1 -- 172.21.15.163:0/1988315276 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3cc00b810 con 0x7fa3d0159420 2023-10-09T06:38:17.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.194+0000 7fa3d4b42700 1 -- 172.21.15.163:0/1988315276 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc00bbb0 con 0x7fa3d0159420 2023-10-09T06:38:17.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.194+0000 7fa3d6b46700 1 -- 172.21.15.163:0/1988315276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3d0159420 msgr2=0x7fa3d00b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:17.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.194+0000 7fa3d6b46700 1 --2- 172.21.15.163:0/1988315276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3d0159420 0x7fa3d00b5f40 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fa3d015d170 tx=0x7fa3cc0091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.194+0000 7fa3d6b46700 1 -- 172.21.15.163:0/1988315276 shutdown_connections 2023-10-09T06:38:17.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.194+0000 7fa3d6b46700 1 --2- 172.21.15.163:0/1988315276 >> v2:172.21.15.181:3300/0 conn(0x7fa3d015d230 0x7fa3d015d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.194+0000 7fa3d6b46700 1 --2- 172.21.15.163:0/1988315276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3d0159420 0x7fa3d00b5f40 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.194+0000 7fa3d6b46700 1 -- 172.21.15.163:0/1988315276 >> 172.21.15.163:0/1988315276 conn(0x7fa3d00d0660 msgr2=0x7fa3d00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:17.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.194+0000 7fa3d6b46700 1 -- 172.21.15.163:0/1988315276 shutdown_connections 2023-10-09T06:38:17.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.194+0000 7fa3d6b46700 1 -- 172.21.15.163:0/1988315276 wait complete. 2023-10-09T06:38:17.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.195+0000 7fa3d6b46700 1 Processor -- start 2023-10-09T06:38:17.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.195+0000 7fa3d6b46700 1 -- start start 2023-10-09T06:38:17.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.196+0000 7fa3d6b46700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3d0159420 0x7fa3d016a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:17.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.196+0000 7fa3d6b46700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3d01597f0 0x7fa3d016c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:17.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.196+0000 7fa3d6b46700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3d015d230 0x7fa3d01566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:17.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.196+0000 7fa3d6b46700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d0152eb0 con 0x7fa3d01597f0 2023-10-09T06:38:17.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.196+0000 7fa3d6b46700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d016a810 con 0x7fa3d0159420 2023-10-09T06:38:17.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.196+0000 7fa3d6b46700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d016d1b0 con 0x7fa3d015d230 2023-10-09T06:38:17.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.196+0000 7fa3d5343700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3d01597f0 0x7fa3d016c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:17.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.196+0000 7fa3d5343700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3d01597f0 0x7fa3d016c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:60194/0 (socket says 172.21.15.163:60194) 2023-10-09T06:38:17.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.196+0000 7fa3d5343700 1 -- 172.21.15.163:0/410670161 learned_addr learned my addr 172.21.15.163:0/410670161 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:17.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.196+0000 7fa3d5b44700 1 --2- 172.21.15.163:0/410670161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3d0159420 0x7fa3d016a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:17.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.196+0000 7fa3d6345700 1 --2- 172.21.15.163:0/410670161 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3d015d230 0x7fa3d01566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:17.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.197+0000 7fa3d5343700 1 -- 172.21.15.163:0/410670161 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3d015d230 msgr2=0x7fa3d01566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:17.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.197+0000 7fa3d5343700 1 --2- 172.21.15.163:0/410670161 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3d015d230 0x7fa3d01566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.197+0000 7fa3d5343700 1 -- 172.21.15.163:0/410670161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3d0159420 msgr2=0x7fa3d016a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:17.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.197+0000 7fa3d5343700 1 --2- 172.21.15.163:0/410670161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3d0159420 0x7fa3d016a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.197+0000 7fa3d5343700 1 -- 172.21.15.163:0/410670161 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc009420 con 0x7fa3d01597f0 2023-10-09T06:38:17.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.197+0000 7fa3d5343700 1 --2- 172.21.15.163:0/410670161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3d01597f0 0x7fa3d016c3a0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000c810 tx=0x7fa3c0007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:17.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.197+0000 7fa3c5ffb700 1 -- 172.21.15.163:0/410670161 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0005c00 con 0x7fa3d01597f0 2023-10-09T06:38:17.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.197+0000 7fa3c5ffb700 1 -- 172.21.15.163:0/410670161 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c0005d60 con 0x7fa3d01597f0 2023-10-09T06:38:17.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.197+0000 7fa3d6b46700 1 -- 172.21.15.163:0/410670161 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d016d740 con 0x7fa3d01597f0 2023-10-09T06:38:17.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.197+0000 7fa3c5ffb700 1 -- 172.21.15.163:0/410670161 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c00072e0 con 0x7fa3d01597f0 2023-10-09T06:38:17.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.197+0000 7fa3d6b46700 1 -- 172.21.15.163:0/410670161 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d016a5b0 con 0x7fa3d01597f0 2023-10-09T06:38:17.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.198+0000 7fa3d6b46700 1 -- 172.21.15.163:0/410670161 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d004c3c0 con 0x7fa3d01597f0 2023-10-09T06:38:17.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.198+0000 7fa3c5ffb700 1 -- 172.21.15.163:0/410670161 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa3c0007690 con 0x7fa3d01597f0 2023-10-09T06:38:17.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.199+0000 7fa3c5ffb700 1 --2- 172.21.15.163:0/410670161 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa3b4067320 0x7fa3b40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:17.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.199+0000 7fa3c5ffb700 1 -- 172.21.15.163:0/410670161 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fa3c0083d40 con 0x7fa3d01597f0 2023-10-09T06:38:17.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.199+0000 7fa3d5b44700 1 --2- 172.21.15.163:0/410670161 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa3b4067320 0x7fa3b40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:17.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.199+0000 7fa3d5b44700 1 --2- 172.21.15.163:0/410670161 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa3b4067320 0x7fa3b40697d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0093f0 tx=0x7fa3cc017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:17.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.203+0000 7fa3c5ffb700 1 -- 172.21.15.163:0/410670161 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa3c004b5a0 con 0x7fa3d01597f0 2023-10-09T06:38:17.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.384+0000 7fa3d6b46700 1 -- 172.21.15.163:0/410670161 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3d016a950 con 0x7fa3d01597f0 2023-10-09T06:38:17.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.384+0000 7fa3c5ffb700 1 -- 172.21.15.163:0/410670161 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+12192 (secure 0 0 0) 0x7fa3c0017020 con 0x7fa3d01597f0 2023-10-09T06:38:17.385 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:17.385 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":148,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:38:16.654438+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:38:17.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.386+0000 7fa3d6b46700 1 -- 172.21.15.163:0/410670161 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa3b4067320 msgr2=0x7fa3b40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:17.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.387+0000 7fa3d6b46700 1 --2- 172.21.15.163:0/410670161 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa3b4067320 0x7fa3b40697d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0093f0 tx=0x7fa3cc017000 comp rx=0 tx=0).stop 2023-10-09T06:38:17.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.387+0000 7fa3d6b46700 1 -- 172.21.15.163:0/410670161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3d01597f0 msgr2=0x7fa3d016c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:17.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.387+0000 7fa3d6b46700 1 --2- 172.21.15.163:0/410670161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3d01597f0 0x7fa3d016c3a0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000c810 tx=0x7fa3c0007db0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.387+0000 7fa3d6b46700 1 -- 172.21.15.163:0/410670161 shutdown_connections 2023-10-09T06:38:17.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.387+0000 7fa3d6b46700 1 --2- 172.21.15.163:0/410670161 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa3b4067320 0x7fa3b40697d0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.387+0000 7fa3d6b46700 1 --2- 172.21.15.163:0/410670161 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3d015d230 0x7fa3d01566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.387+0000 7fa3d6b46700 1 --2- 172.21.15.163:0/410670161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3d0159420 0x7fa3d016a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.387+0000 7fa3d6b46700 1 --2- 172.21.15.163:0/410670161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa3d01597f0 0x7fa3d016c3a0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.387+0000 7fa3d6b46700 1 -- 172.21.15.163:0/410670161 >> 172.21.15.163:0/410670161 conn(0x7fa3d00d0660 msgr2=0x7fa3d00ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:17.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.387+0000 7fa3d6b46700 1 -- 172.21.15.163:0/410670161 shutdown_connections 2023-10-09T06:38:17.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.387+0000 7fa3d6b46700 1 -- 172.21.15.163:0/410670161 wait complete. 2023-10-09T06:38:17.400 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-09T06:38:17.400 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:38:17.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.517+0000 7f76668f3700 1 Processor -- start 2023-10-09T06:38:17.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.517+0000 7f76668f3700 1 -- start start 2023-10-09T06:38:17.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.518+0000 7f76668f3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7660159c60 0x7f766015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:17.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.518+0000 7f76668f3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f766015db10 0x7f7660069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:17.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.518+0000 7f76668f3700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f766015a160 con 0x7f7660159890 2023-10-09T06:38:17.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.518+0000 7f76668f3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f766015df50 con 0x7f7660159c60 2023-10-09T06:38:17.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.518+0000 7f76668f3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76600d13a0 con 0x7f766015db10 2023-10-09T06:38:17.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.519+0000 7f765f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7660159c60 0x7f766015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:17.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.519+0000 7f7664e90700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f766015db10 0x7f7660069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:17.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.519+0000 7f765f7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7660159c60 0x7f766015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:37776/0 (socket says 172.21.15.163:37776) 2023-10-09T06:38:17.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.519+0000 7f7664e90700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f766015db10 0x7f7660069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:47574/0 (socket says 172.21.15.163:47574) 2023-10-09T06:38:17.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.519+0000 7f765f7fe700 1 -- 172.21.15.163:0/1370224662 learned_addr learned my addr 172.21.15.163:0/1370224662 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:17.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.519+0000 7f7664e90700 1 -- 172.21.15.163:0/1370224662 >> v2:172.21.15.181:3300/0 conn(0x7f7660159c60 msgr2=0x7f766015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:17.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.519+0000 7f7664e90700 1 --2- 172.21.15.163:0/1370224662 >> v2:172.21.15.181:3300/0 conn(0x7f7660159c60 0x7f766015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.519+0000 7f7664e90700 1 -- 172.21.15.163:0/1370224662 >> v1:172.21.15.181:6789/0 conn(0x7f7660159890 legacy=0x7f766015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:17.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.519+0000 7f7664e90700 1 -- 172.21.15.163:0/1370224662 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766015e140 con 0x7f766015db10 2023-10-09T06:38:17.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.519+0000 7f7664e90700 1 --2- 172.21.15.163:0/1370224662 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f766015db10 0x7f7660069560 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7658009e60 tx=0x7f7658009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24629118f4c178b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:17.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.519+0000 7f765effd700 1 -- 172.21.15.163:0/1370224662 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765800e070 con 0x7f766015db10 2023-10-09T06:38:17.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.519+0000 7f765effd700 1 -- 172.21.15.163:0/1370224662 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765800bb60 con 0x7f766015db10 2023-10-09T06:38:17.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.519+0000 7f765effd700 1 -- 172.21.15.163:0/1370224662 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7658012430 con 0x7f766015db10 2023-10-09T06:38:17.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.520+0000 7f76668f3700 1 -- 172.21.15.163:0/1370224662 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f766015db10 msgr2=0x7f7660069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:17.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.520+0000 7f76668f3700 1 --2- 172.21.15.163:0/1370224662 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f766015db10 0x7f7660069560 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7658009e60 tx=0x7f7658009780 comp rx=0 tx=0).stop 2023-10-09T06:38:17.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.520+0000 7f76668f3700 1 -- 172.21.15.163:0/1370224662 shutdown_connections 2023-10-09T06:38:17.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.520+0000 7f76668f3700 1 --2- 172.21.15.163:0/1370224662 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f766015db10 0x7f7660069560 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.520+0000 7f76668f3700 1 --2- 172.21.15.163:0/1370224662 >> v2:172.21.15.181:3300/0 conn(0x7f7660159c60 0x7f766015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.520+0000 7f76668f3700 1 -- 172.21.15.163:0/1370224662 >> 172.21.15.163:0/1370224662 conn(0x7f7660068e20 msgr2=0x7f76600d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:17.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.520+0000 7f76668f3700 1 -- 172.21.15.163:0/1370224662 shutdown_connections 2023-10-09T06:38:17.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.520+0000 7f76668f3700 1 -- 172.21.15.163:0/1370224662 wait complete. 2023-10-09T06:38:17.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.521+0000 7f76668f3700 1 Processor -- start 2023-10-09T06:38:17.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.521+0000 7f76668f3700 1 -- start start 2023-10-09T06:38:17.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.522+0000 7f76668f3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7660159890 0x7f76600c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:17.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.522+0000 7f76668f3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7660159c60 0x7f76600cf200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:17.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.522+0000 7f76668f3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f766015db10 0x7f7660149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:17.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.522+0000 7f76668f3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76600c98b0 con 0x7f7660159c60 2023-10-09T06:38:17.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.522+0000 7f76668f3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76600cffa0 con 0x7f7660159890 2023-10-09T06:38:17.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.522+0000 7f76668f3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766015bb00 con 0x7f766015db10 2023-10-09T06:38:17.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.522+0000 7f765f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7660159c60 0x7f76600cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:17.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.522+0000 7f765f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7660159c60 0x7f76600cf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:60210/0 (socket says 172.21.15.163:60210) 2023-10-09T06:38:17.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.522+0000 7f765f7fe700 1 -- 172.21.15.163:0/3600167368 learned_addr learned my addr 172.21.15.163:0/3600167368 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:17.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.522+0000 7f765ffff700 1 --2- 172.21.15.163:0/3600167368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7660159890 0x7f76600c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:17.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.522+0000 7f7664e90700 1 --2- 172.21.15.163:0/3600167368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f766015db10 0x7f7660149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:17.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.522+0000 7f765f7fe700 1 -- 172.21.15.163:0/3600167368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f766015db10 msgr2=0x7f7660149210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:17.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.522+0000 7f765f7fe700 1 --2- 172.21.15.163:0/3600167368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f766015db10 0x7f7660149210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.522+0000 7f765f7fe700 1 -- 172.21.15.163:0/3600167368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7660159890 msgr2=0x7f76600c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:17.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.522+0000 7f765f7fe700 1 --2- 172.21.15.163:0/3600167368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7660159890 0x7f76600c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.522+0000 7f765f7fe700 1 -- 172.21.15.163:0/3600167368 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7658009a20 con 0x7f7660159c60 2023-10-09T06:38:17.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.523+0000 7f765f7fe700 1 --2- 172.21.15.163:0/3600167368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7660159c60 0x7f76600cf200 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f765400ea50 tx=0x7f7654009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:17.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.523+0000 7f7647fff700 1 -- 172.21.15.163:0/3600167368 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765400c7f0 con 0x7f7660159c60 2023-10-09T06:38:17.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.523+0000 7f76668f3700 1 -- 172.21.15.163:0/3600167368 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76600cf740 con 0x7f7660159c60 2023-10-09T06:38:17.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.523+0000 7f7647fff700 1 -- 172.21.15.163:0/3600167368 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765400c950 con 0x7f7660159c60 2023-10-09T06:38:17.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.523+0000 7f7647fff700 1 -- 172.21.15.163:0/3600167368 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765400ccd0 con 0x7f7660159c60 2023-10-09T06:38:17.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.523+0000 7f76668f3700 1 -- 172.21.15.163:0/3600167368 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76600cfcc0 con 0x7f7660159c60 2023-10-09T06:38:17.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.524+0000 7f76668f3700 1 -- 172.21.15.163:0/3600167368 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f766004c3c0 con 0x7f7660159c60 2023-10-09T06:38:17.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.524+0000 7f7647fff700 1 -- 172.21.15.163:0/3600167368 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7654013680 con 0x7f7660159c60 2023-10-09T06:38:17.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.525+0000 7f7647fff700 1 --2- 172.21.15.163:0/3600167368 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f764c067320 0x7f764c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:17.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.525+0000 7f7647fff700 1 -- 172.21.15.163:0/3600167368 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f7654084c40 con 0x7f7660159c60 2023-10-09T06:38:17.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.525+0000 7f765ffff700 1 --2- 172.21.15.163:0/3600167368 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f764c067320 0x7f764c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:17.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.525+0000 7f765ffff700 1 --2- 172.21.15.163:0/3600167368 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f764c067320 0x7f764c0697d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7650003860 tx=0x7f7650009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:17.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.529+0000 7f7647fff700 1 -- 172.21.15.163:0/3600167368 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7654051c20 con 0x7f7660159c60 2023-10-09T06:38:17.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.712+0000 7f76668f3700 1 -- 172.21.15.163:0/3600167368 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f766006a5b0 con 0x7f7660159c60 2023-10-09T06:38:17.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.712+0000 7f7647fff700 1 -- 172.21.15.163:0/3600167368 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v148) v1 ==== 93+0+10 (secure 0 0 0) 0x7f76540170a0 con 0x7f7660159c60 2023-10-09T06:38:17.713 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:38:17.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.714+0000 7f76668f3700 1 -- 172.21.15.163:0/3600167368 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f764c067320 msgr2=0x7f764c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:17.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.714+0000 7f76668f3700 1 --2- 172.21.15.163:0/3600167368 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f764c067320 0x7f764c0697d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7650003860 tx=0x7f7650009040 comp rx=0 tx=0).stop 2023-10-09T06:38:17.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.714+0000 7f76668f3700 1 -- 172.21.15.163:0/3600167368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7660159c60 msgr2=0x7f76600cf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:17.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.714+0000 7f76668f3700 1 --2- 172.21.15.163:0/3600167368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7660159c60 0x7f76600cf200 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f765400ea50 tx=0x7f7654009250 comp rx=0 tx=0).stop 2023-10-09T06:38:17.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.714+0000 7f76668f3700 1 -- 172.21.15.163:0/3600167368 shutdown_connections 2023-10-09T06:38:17.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.714+0000 7f76668f3700 1 --2- 172.21.15.163:0/3600167368 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f764c067320 0x7f764c0697d0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.714+0000 7f76668f3700 1 --2- 172.21.15.163:0/3600167368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f766015db10 0x7f7660149210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.714+0000 7f76668f3700 1 --2- 172.21.15.163:0/3600167368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7660159890 0x7f76600c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.714+0000 7f76668f3700 1 --2- 172.21.15.163:0/3600167368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7660159c60 0x7f76600cf200 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.714+0000 7f76668f3700 1 -- 172.21.15.163:0/3600167368 >> 172.21.15.163:0/3600167368 conn(0x7f7660068e20 msgr2=0x7f76600b1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:17.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.714+0000 7f76668f3700 1 -- 172.21.15.163:0/3600167368 shutdown_connections 2023-10-09T06:38:17.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.714+0000 7f76668f3700 1 -- 172.21.15.163:0/3600167368 wait complete. 2023-10-09T06:38:17.727 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-09T06:38:17.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.841+0000 7f43503cf700 1 Processor -- start 2023-10-09T06:38:17.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.841+0000 7f43503cf700 1 -- start start 2023-10-09T06:38:17.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.842+0000 7f43503cf700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f43480caa90 0x7f43480c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:17.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.842+0000 7f43503cf700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f434815d7e0 con 0x7f434806c0a0 2023-10-09T06:38:17.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.842+0000 7f43503cf700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f434815dac0 con 0x7f434806bcd0 2023-10-09T06:38:17.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.842+0000 7f43503cf700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43480caef0 con 0x7f43480caa90 2023-10-09T06:38:17.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.843+0000 7f434d96a700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f434806c0a0 0x7f434815df50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:51956/0 (socket says 172.21.15.163:51956) 2023-10-09T06:38:17.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.843+0000 7f434d96a700 1 -- 172.21.15.163:0/1770863842 learned_addr learned my addr 172.21.15.163:0/1770863842 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:17.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.843+0000 7f434e96c700 1 --2- 172.21.15.163:0/1770863842 >> v2:172.21.15.181:3300/0 conn(0x7f43480caa90 0x7f43480c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:17.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.843+0000 7f434d169700 1 -- 172.21.15.163:0/1770863842 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1890783328 0 0) 0x7f434815d7e0 con 0x7f434806c0a0 2023-10-09T06:38:17.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.843+0000 7f434d169700 1 -- 172.21.15.163:0/1770863842 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4334003570 con 0x7f434806c0a0 2023-10-09T06:38:17.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.843+0000 7f434d169700 1 -- 172.21.15.163:0/1770863842 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 311931981 0 0) 0x7f434815dac0 con 0x7f434806bcd0 2023-10-09T06:38:17.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.843+0000 7f434d169700 1 -- 172.21.15.163:0/1770863842 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f434815d7e0 con 0x7f434806bcd0 2023-10-09T06:38:17.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.843+0000 7f434d169700 1 -- 172.21.15.163:0/1770863842 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1128354077 0 0) 0x7f4334003570 con 0x7f434806c0a0 2023-10-09T06:38:17.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.843+0000 7f434d169700 1 -- 172.21.15.163:0/1770863842 >> v2:172.21.15.181:3300/0 conn(0x7f43480caa90 msgr2=0x7f43480c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:17.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.843+0000 7f434d169700 1 --2- 172.21.15.163:0/1770863842 >> v2:172.21.15.181:3300/0 conn(0x7f43480caa90 0x7f43480c8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.843+0000 7f434d169700 1 -- 172.21.15.163:0/1770863842 >> v1:172.21.15.181:6789/0 conn(0x7f434806bcd0 legacy=0x7f434815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:17.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.843+0000 7f434d169700 1 -- 172.21.15.163:0/1770863842 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434815aa30 con 0x7f434806c0a0 2023-10-09T06:38:17.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.843+0000 7f434d169700 1 -- 172.21.15.163:0/1770863842 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f4338001470 con 0x7f434806c0a0 2023-10-09T06:38:17.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.843+0000 7f434e96c700 1 --2- 172.21.15.163:0/1770863842 >> v2:172.21.15.181:3300/0 conn(0x7f43480caa90 0x7f43480c8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:38:17.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.843+0000 7f434d169700 1 -- 172.21.15.163:0/1770863842 >> v1:172.21.15.163:6789/0 conn(0x7f434806c0a0 legacy=0x7f434815df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:17.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.843+0000 7f434d169700 1 --2- 172.21.15.163:0/1770863842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4334003bf0 0x7f4334003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:17.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.843+0000 7f434d169700 1 --2- 172.21.15.163:0/1770863842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4334004810 0x7f4334004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:17.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.843+0000 7f434d169700 1 --2- 172.21.15.163:0/1770863842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4334005230 0x7f433400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:17.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.843+0000 7f434d169700 1 -- 172.21.15.163:0/1770863842 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43480caef0 con 0x7f4334003bf0 2023-10-09T06:38:17.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.843+0000 7f434d169700 1 -- 172.21.15.163:0/1770863842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433400c200 con 0x7f4334004810 2023-10-09T06:38:17.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.843+0000 7f434d169700 1 -- 172.21.15.163:0/1770863842 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f433400c3c0 con 0x7f4334005230 2023-10-09T06:38:17.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.843+0000 7f434e16b700 1 --2- 172.21.15.163:0/1770863842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4334003bf0 0x7f4334003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:17.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.843+0000 7f434d96a700 1 --2- 172.21.15.163:0/1770863842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4334004810 0x7f4334004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:17.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.843+0000 7f434e96c700 1 --2- 172.21.15.163:0/1770863842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4334005230 0x7f433400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:17.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.844+0000 7f434e16b700 1 -- 172.21.15.163:0/1770863842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4334005230 msgr2=0x7f433400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:17.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.844+0000 7f434e16b700 1 --2- 172.21.15.163:0/1770863842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4334005230 0x7f433400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.844+0000 7f434e16b700 1 -- 172.21.15.163:0/1770863842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4334004810 msgr2=0x7f4334004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:17.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.844+0000 7f434e16b700 1 --2- 172.21.15.163:0/1770863842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4334004810 0x7f4334004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.844+0000 7f434e16b700 1 -- 172.21.15.163:0/1770863842 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4334003570 con 0x7f4334003bf0 2023-10-09T06:38:17.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.844+0000 7f434e96c700 1 -- 172.21.15.163:0/1770863842 reap_dead start 2023-10-09T06:38:17.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.844+0000 7f434e16b700 1 --2- 172.21.15.163:0/1770863842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4334003bf0 0x7f4334003fe0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f4344002120 tx=0x7f43440017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c56d3525f7c241 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:17.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.844+0000 7f434d169700 1 -- 172.21.15.163:0/1770863842 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4344001ab0 con 0x7f4334003bf0 2023-10-09T06:38:17.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.844+0000 7f434d169700 1 -- 172.21.15.163:0/1770863842 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434400c870 con 0x7f4334003bf0 2023-10-09T06:38:17.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.844+0000 7f434d169700 1 -- 172.21.15.163:0/1770863842 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434400cc10 con 0x7f4334003bf0 2023-10-09T06:38:17.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.844+0000 7f43503cf700 1 -- 172.21.15.163:0/1770863842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4334003bf0 msgr2=0x7f4334003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:17.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.844+0000 7f43503cf700 1 --2- 172.21.15.163:0/1770863842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4334003bf0 0x7f4334003fe0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f4344002120 tx=0x7f43440017a0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.844+0000 7f43503cf700 1 -- 172.21.15.163:0/1770863842 shutdown_connections 2023-10-09T06:38:17.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.844+0000 7f43503cf700 1 --2- 172.21.15.163:0/1770863842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4334003bf0 0x7f4334003fe0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.844+0000 7f43503cf700 1 -- 172.21.15.163:0/1770863842 >> 172.21.15.163:0/1770863842 conn(0x7f43480ce600 msgr2=0x7f43480cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:17.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.844+0000 7f43503cf700 1 -- 172.21.15.163:0/1770863842 shutdown_connections 2023-10-09T06:38:17.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.845+0000 7f43503cf700 1 -- 172.21.15.163:0/1770863842 wait complete. 2023-10-09T06:38:17.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.846+0000 7f43503cf700 1 Processor -- start 2023-10-09T06:38:17.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.846+0000 7f43503cf700 1 -- start start 2023-10-09T06:38:17.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.846+0000 7f43503cf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4334003bf0 0x7f43480c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:17.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.846+0000 7f43503cf700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43480c63c0 0x7f4348169b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:17.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.846+0000 7f43503cf700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f434816a070 0x7f434816a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:17.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.846+0000 7f43503cf700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43480b22b0 con 0x7f434816a070 2023-10-09T06:38:17.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.846+0000 7f43503cf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43480b4200 con 0x7f4334003bf0 2023-10-09T06:38:17.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.846+0000 7f43503cf700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43480c8f60 con 0x7f43480c63c0 2023-10-09T06:38:17.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.846+0000 7f434e96c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f434816a070 0x7f434816a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:17.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.846+0000 7f434e96c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f434816a070 0x7f434816a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:60226/0 (socket says 172.21.15.163:60226) 2023-10-09T06:38:17.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.846+0000 7f434e96c700 1 -- 172.21.15.163:0/2881689397 learned_addr learned my addr 172.21.15.163:0/2881689397 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:17.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.846+0000 7f434e16b700 1 --2- 172.21.15.163:0/2881689397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4334003bf0 0x7f43480c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:17.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.846+0000 7f434d96a700 1 --2- 172.21.15.163:0/2881689397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43480c63c0 0x7f4348169b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:17.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.847+0000 7f434e96c700 1 -- 172.21.15.163:0/2881689397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43480c63c0 msgr2=0x7f4348169b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:17.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.847+0000 7f434e96c700 1 --2- 172.21.15.163:0/2881689397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43480c63c0 0x7f4348169b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.847+0000 7f434e96c700 1 -- 172.21.15.163:0/2881689397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4334003bf0 msgr2=0x7f43480c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:17.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.847+0000 7f434e96c700 1 --2- 172.21.15.163:0/2881689397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4334003bf0 0x7f43480c8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:17.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.847+0000 7f434e96c700 1 -- 172.21.15.163:0/2881689397 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43480c9240 con 0x7f434816a070 2023-10-09T06:38:17.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.847+0000 7f434e96c700 1 --2- 172.21.15.163:0/2881689397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f434816a070 0x7f434816a4e0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f434806c9a0 tx=0x7f4340008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:17.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.847+0000 7f433e7fc700 1 -- 172.21.15.163:0/2881689397 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4340015670 con 0x7f434816a070 2023-10-09T06:38:17.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.847+0000 7f433e7fc700 1 -- 172.21.15.163:0/2881689397 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43400157d0 con 0x7f434816a070 2023-10-09T06:38:17.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.847+0000 7f43503cf700 1 -- 172.21.15.163:0/2881689397 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43480c9520 con 0x7f434816a070 2023-10-09T06:38:17.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.847+0000 7f433e7fc700 1 -- 172.21.15.163:0/2881689397 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4340015b70 con 0x7f434816a070 2023-10-09T06:38:17.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.847+0000 7f43503cf700 1 -- 172.21.15.163:0/2881689397 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43481590b0 con 0x7f434816a070 2023-10-09T06:38:17.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.848+0000 7f43503cf700 1 -- 172.21.15.163:0/2881689397 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43481592d0 con 0x7f434816a070 2023-10-09T06:38:17.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.848+0000 7f433e7fc700 1 -- 172.21.15.163:0/2881689397 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4340020460 con 0x7f434816a070 2023-10-09T06:38:17.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.849+0000 7f433e7fc700 1 --2- 172.21.15.163:0/2881689397 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f432c067320 0x7f432c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:17.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.849+0000 7f433e7fc700 1 -- 172.21.15.163:0/2881689397 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4340084d90 con 0x7f434816a070 2023-10-09T06:38:17.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.849+0000 7f434e16b700 1 --2- 172.21.15.163:0/2881689397 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f432c067320 0x7f432c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:17.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.849+0000 7f434e16b700 1 --2- 172.21.15.163:0/2881689397 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f432c067320 0x7f432c0697d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4344000f80 tx=0x7f4344016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:17.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:17.853+0000 7f433e7fc700 1 -- 172.21.15.163:0/2881689397 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4340051d70 con 0x7f434816a070 2023-10-09T06:38:18.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.038+0000 7f43503cf700 1 -- 172.21.15.163:0/2881689397 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f434804c3c0 con 0x7f434816a070 2023-10-09T06:38:18.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.038+0000 7f433e7fc700 1 -- 172.21.15.163:0/2881689397 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v148) v1 ==== 84+0+65 (secure 0 0 0) 0x7f4340055390 con 0x7f434816a070 2023-10-09T06:38:18.039 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:18.039 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:38:18.040 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:18.040 INFO:teuthology.orchestra.run.smithi163.stdout: "poolnum": 1, 2023-10-09T06:38:18.040 INFO:teuthology.orchestra.run.smithi163.stdout: "poolname": ".mgr" 2023-10-09T06:38:18.040 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:18.040 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:38:18.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.040+0000 7f43503cf700 1 -- 172.21.15.163:0/2881689397 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f432c067320 msgr2=0x7f432c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:18.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.040+0000 7f43503cf700 1 --2- 172.21.15.163:0/2881689397 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f432c067320 0x7f432c0697d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4344000f80 tx=0x7f4344016000 comp rx=0 tx=0).stop 2023-10-09T06:38:18.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.040+0000 7f43503cf700 1 -- 172.21.15.163:0/2881689397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f434816a070 msgr2=0x7f434816a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:18.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.040+0000 7f43503cf700 1 --2- 172.21.15.163:0/2881689397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f434816a070 0x7f434816a4e0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f434806c9a0 tx=0x7f4340008040 comp rx=0 tx=0).stop 2023-10-09T06:38:18.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.040+0000 7f43503cf700 1 -- 172.21.15.163:0/2881689397 shutdown_connections 2023-10-09T06:38:18.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.040+0000 7f43503cf700 1 --2- 172.21.15.163:0/2881689397 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f432c067320 0x7f432c0697d0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:18.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.040+0000 7f43503cf700 1 --2- 172.21.15.163:0/2881689397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f434816a070 0x7f434816a4e0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:18.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.040+0000 7f43503cf700 1 --2- 172.21.15.163:0/2881689397 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4334003bf0 0x7f43480c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:18.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.040+0000 7f43503cf700 1 --2- 172.21.15.163:0/2881689397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43480c63c0 0x7f4348169b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:18.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.040+0000 7f43503cf700 1 -- 172.21.15.163:0/2881689397 >> 172.21.15.163:0/2881689397 conn(0x7f43480ce600 msgr2=0x7f43480cc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:18.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.040+0000 7f43503cf700 1 -- 172.21.15.163:0/2881689397 shutdown_connections 2023-10-09T06:38:18.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.040+0000 7f43503cf700 1 -- 172.21.15.163:0/2881689397 wait complete. 2023-10-09T06:38:18.053 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-09T06:38:18.053 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-09T06:38:18.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.171+0000 7f59ef3f6700 1 Processor -- start 2023-10-09T06:38:18.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.171+0000 7f59ef3f6700 1 -- start start 2023-10-09T06:38:18.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.172+0000 7f59ef3f6700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f59e80b03c0 0x7f59e80cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:18.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.172+0000 7f59ef3f6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f59e81591e0 0x7f59e8159600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:18.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.172+0000 7f59ef3f6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59e80cfa00 con 0x7f59e80b0790 2023-10-09T06:38:18.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.172+0000 7f59ef3f6700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e80cb580 con 0x7f59e80b03c0 2023-10-09T06:38:18.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.172+0000 7f59ef3f6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e80cf160 con 0x7f59e81591e0 2023-10-09T06:38:18.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.172+0000 7f59ee3f4700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f59e80b03c0 0x7f59e80cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:18.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.172+0000 7f59ee3f4700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f59e80b03c0 0x7f59e80cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:60240/0 (socket says 172.21.15.163:60240) 2023-10-09T06:38:18.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.172+0000 7f59ee3f4700 1 -- 172.21.15.163:0/1073299062 learned_addr learned my addr 172.21.15.163:0/1073299062 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:18.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.172+0000 7f59eebf5700 1 --2- 172.21.15.163:0/1073299062 >> v2:172.21.15.181:3300/0 conn(0x7f59e81591e0 0x7f59e8159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:18.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.172+0000 7f59ee3f4700 1 -- 172.21.15.163:0/1073299062 >> v2:172.21.15.181:3300/0 conn(0x7f59e81591e0 msgr2=0x7f59e8159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:18.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.172+0000 7f59ee3f4700 1 --2- 172.21.15.163:0/1073299062 >> v2:172.21.15.181:3300/0 conn(0x7f59e81591e0 0x7f59e8159600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:18.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.172+0000 7f59ee3f4700 1 -- 172.21.15.163:0/1073299062 >> v1:172.21.15.181:6789/0 conn(0x7f59e80b0790 legacy=0x7f59e80b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:18.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.172+0000 7f59ee3f4700 1 -- 172.21.15.163:0/1073299062 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e80cf440 con 0x7f59e80b03c0 2023-10-09T06:38:18.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.173+0000 7f59ee3f4700 1 --2- 172.21.15.163:0/1073299062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59e80b03c0 0x7f59e80cea70 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f59e40099b0 tx=0x7f59e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b21ae31d3e9aa2 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:18.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.173+0000 7f59ed3f2700 1 -- 172.21.15.163:0/1073299062 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e400b7e0 con 0x7f59e80b03c0 2023-10-09T06:38:18.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.173+0000 7f59ed3f2700 1 -- 172.21.15.163:0/1073299062 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e400b940 con 0x7f59e80b03c0 2023-10-09T06:38:18.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.173+0000 7f59ed3f2700 1 -- 172.21.15.163:0/1073299062 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e400bd00 con 0x7f59e80b03c0 2023-10-09T06:38:18.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.173+0000 7f59ef3f6700 1 -- 172.21.15.163:0/1073299062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59e80b03c0 msgr2=0x7f59e80cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:18.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.173+0000 7f59ef3f6700 1 --2- 172.21.15.163:0/1073299062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59e80b03c0 0x7f59e80cea70 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f59e40099b0 tx=0x7f59e4009250 comp rx=0 tx=0).stop 2023-10-09T06:38:18.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.173+0000 7f59ef3f6700 1 -- 172.21.15.163:0/1073299062 shutdown_connections 2023-10-09T06:38:18.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.173+0000 7f59ef3f6700 1 --2- 172.21.15.163:0/1073299062 >> v2:172.21.15.181:3300/0 conn(0x7f59e81591e0 0x7f59e8159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:18.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.173+0000 7f59ef3f6700 1 --2- 172.21.15.163:0/1073299062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59e80b03c0 0x7f59e80cea70 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:18.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.173+0000 7f59ef3f6700 1 -- 172.21.15.163:0/1073299062 >> 172.21.15.163:0/1073299062 conn(0x7f59e8063230 msgr2=0x7f59e8063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:18.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.173+0000 7f59ef3f6700 1 -- 172.21.15.163:0/1073299062 shutdown_connections 2023-10-09T06:38:18.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.173+0000 7f59ef3f6700 1 -- 172.21.15.163:0/1073299062 wait complete. 2023-10-09T06:38:18.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.174+0000 7f59ef3f6700 1 Processor -- start 2023-10-09T06:38:18.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.174+0000 7f59ef3f6700 1 -- start start 2023-10-09T06:38:18.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.175+0000 7f59ef3f6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59e80b03c0 0x7f59e80c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:18.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.175+0000 7f59ef3f6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59e80b0790 0x7f59e815d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:18.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.175+0000 7f59ef3f6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59e81591e0 0x7f59e815daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:18.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.175+0000 7f59ef3f6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e81585a0 con 0x7f59e80b0790 2023-10-09T06:38:18.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.175+0000 7f59ef3f6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e815dfe0 con 0x7f59e80b03c0 2023-10-09T06:38:18.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.175+0000 7f59ef3f6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e816ade0 con 0x7f59e81591e0 2023-10-09T06:38:18.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.175+0000 7f59edbf3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59e80b0790 0x7f59e815d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:18.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.175+0000 7f59edbf3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59e80b0790 0x7f59e815d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:60244/0 (socket says 172.21.15.163:60244) 2023-10-09T06:38:18.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.175+0000 7f59edbf3700 1 -- 172.21.15.163:0/92183535 learned_addr learned my addr 172.21.15.163:0/92183535 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:18.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.175+0000 7f59ee3f4700 1 --2- 172.21.15.163:0/92183535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59e80b03c0 0x7f59e80c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:18.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.175+0000 7f59eebf5700 1 --2- 172.21.15.163:0/92183535 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59e81591e0 0x7f59e815daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:18.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.176+0000 7f59edbf3700 1 -- 172.21.15.163:0/92183535 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59e81591e0 msgr2=0x7f59e815daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:18.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.176+0000 7f59edbf3700 1 --2- 172.21.15.163:0/92183535 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59e81591e0 0x7f59e815daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:18.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.176+0000 7f59edbf3700 1 -- 172.21.15.163:0/92183535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59e80b03c0 msgr2=0x7f59e80c8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:18.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.176+0000 7f59edbf3700 1 --2- 172.21.15.163:0/92183535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59e80b03c0 0x7f59e80c8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:18.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.176+0000 7f59edbf3700 1 -- 172.21.15.163:0/92183535 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e40094f0 con 0x7f59e80b0790 2023-10-09T06:38:18.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.176+0000 7f59eebf5700 1 --2- 172.21.15.163:0/92183535 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59e81591e0 0x7f59e815daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:18.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.176+0000 7f59ee3f4700 1 --2- 172.21.15.163:0/92183535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59e80b03c0 0x7f59e80c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:18.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.176+0000 7f59edbf3700 1 --2- 172.21.15.163:0/92183535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59e80b0790 0x7f59e815d560 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f59d800b600 tx=0x7f59d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:18.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.176+0000 7f59de7fc700 1 -- 172.21.15.163:0/92183535 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d800f510 con 0x7f59e80b0790 2023-10-09T06:38:18.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.176+0000 7f59de7fc700 1 -- 172.21.15.163:0/92183535 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d800f670 con 0x7f59e80b0790 2023-10-09T06:38:18.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.176+0000 7f59ef3f6700 1 -- 172.21.15.163:0/92183535 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e80ca8f0 con 0x7f59e80b0790 2023-10-09T06:38:18.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.176+0000 7f59de7fc700 1 -- 172.21.15.163:0/92183535 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d800fa30 con 0x7f59e80b0790 2023-10-09T06:38:18.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.176+0000 7f59ef3f6700 1 -- 172.21.15.163:0/92183535 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e816a300 con 0x7f59e80b0790 2023-10-09T06:38:18.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.177+0000 7f59ef3f6700 1 -- 172.21.15.163:0/92183535 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e80c92a0 con 0x7f59e80b0790 2023-10-09T06:38:18.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.177+0000 7f59de7fc700 1 -- 172.21.15.163:0/92183535 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f59d801a400 con 0x7f59e80b0790 2023-10-09T06:38:18.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.177+0000 7f59de7fc700 1 --2- 172.21.15.163:0/92183535 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f59cc067350 0x7f59cc069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:18.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.178+0000 7f59de7fc700 1 -- 172.21.15.163:0/92183535 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f59d8084360 con 0x7f59e80b0790 2023-10-09T06:38:18.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.178+0000 7f59ee3f4700 1 --2- 172.21.15.163:0/92183535 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f59cc067350 0x7f59cc069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:18.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.178+0000 7f59ee3f4700 1 --2- 172.21.15.163:0/92183535 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f59cc067350 0x7f59cc069800 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f59e40099b0 tx=0x7f59e4016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:18.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.182+0000 7f59de7fc700 1 -- 172.21.15.163:0/92183535 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59e80c92a0 con 0x7f59e80b0790 2023-10-09T06:38:18.364 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.363+0000 7f59ef3f6700 1 -- 172.21.15.163:0/92183535 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f59e80c92a0 con 0x7f59e80b0790 2023-10-09T06:38:18.668 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.668+0000 7f59de7fc700 1 -- 172.21.15.163:0/92183535 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v149) v1 ==== 138+0+0 (secure 0 0 0) 0x7f59e80c92a0 con 0x7f59e80b0790 2023-10-09T06:38:18.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.671+0000 7f59ef3f6700 1 -- 172.21.15.163:0/92183535 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f59cc067350 msgr2=0x7f59cc069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:18.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.671+0000 7f59ef3f6700 1 --2- 172.21.15.163:0/92183535 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f59cc067350 0x7f59cc069800 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f59e40099b0 tx=0x7f59e4016000 comp rx=0 tx=0).stop 2023-10-09T06:38:18.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.671+0000 7f59ef3f6700 1 -- 172.21.15.163:0/92183535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59e80b0790 msgr2=0x7f59e815d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:18.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.671+0000 7f59ef3f6700 1 --2- 172.21.15.163:0/92183535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59e80b0790 0x7f59e815d560 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f59d800b600 tx=0x7f59d80091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:18.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.671+0000 7f59ef3f6700 1 -- 172.21.15.163:0/92183535 shutdown_connections 2023-10-09T06:38:18.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.671+0000 7f59ef3f6700 1 --2- 172.21.15.163:0/92183535 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f59cc067350 0x7f59cc069800 secure :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f59e40099b0 tx=0x7f59e4016000 comp rx=0 tx=0).stop 2023-10-09T06:38:18.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.671+0000 7f59ef3f6700 1 --2- 172.21.15.163:0/92183535 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59e81591e0 0x7f59e815daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:18.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.671+0000 7f59ef3f6700 1 --2- 172.21.15.163:0/92183535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59e80b03c0 0x7f59e80c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:18.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.671+0000 7f59ef3f6700 1 --2- 172.21.15.163:0/92183535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59e80b0790 0x7f59e815d560 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:18.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.671+0000 7f59ef3f6700 1 -- 172.21.15.163:0/92183535 >> 172.21.15.163:0/92183535 conn(0x7f59e8063230 msgr2=0x7f59e8158a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:18.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.672+0000 7f59ef3f6700 1 -- 172.21.15.163:0/92183535 shutdown_connections 2023-10-09T06:38:18.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.672+0000 7f59ef3f6700 1 -- 172.21.15.163:0/92183535 wait complete. 2023-10-09T06:38:18.674 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_metadata' created 2023-10-09T06:38:18.687 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-09T06:38:18.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.808+0000 7fe522641700 1 Processor -- start 2023-10-09T06:38:18.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.809+0000 7fe522641700 1 -- start start 2023-10-09T06:38:18.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.809+0000 7fe522641700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe51c159950 0x7fe51c0cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:18.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.809+0000 7fe522641700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe51c069c70 0x7fe51c0cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:18.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.809+0000 7fe522641700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe51c0ce380 con 0x7fe51c0af320 2023-10-09T06:38:18.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.809+0000 7fe522641700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe51c0cec20 con 0x7fe51c159950 2023-10-09T06:38:18.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.809+0000 7fe522641700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe51c0afa90 con 0x7fe51c069c70 2023-10-09T06:38:18.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.809+0000 7fe51bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe51c159950 0x7fe51c0cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:18.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.810+0000 7fe51bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe51c159950 0x7fe51c0cbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42044/0 (socket says 172.21.15.163:42044) 2023-10-09T06:38:18.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.810+0000 7fe51bfff700 1 -- 172.21.15.163:0/4010822979 learned_addr learned my addr 172.21.15.163:0/4010822979 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:18.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.810+0000 7fe520bde700 1 --2- 172.21.15.163:0/4010822979 >> v2:172.21.15.181:3300/0 conn(0x7fe51c069c70 0x7fe51c0cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:18.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.810+0000 7fe51bfff700 1 -- 172.21.15.163:0/4010822979 >> v2:172.21.15.181:3300/0 conn(0x7fe51c069c70 msgr2=0x7fe51c0cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:18.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.810+0000 7fe51bfff700 1 --2- 172.21.15.163:0/4010822979 >> v2:172.21.15.181:3300/0 conn(0x7fe51c069c70 0x7fe51c0cf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:18.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.810+0000 7fe51bfff700 1 -- 172.21.15.163:0/4010822979 >> v1:172.21.15.181:6789/0 conn(0x7fe51c0af320 legacy=0x7fe51c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:18.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.810+0000 7fe51bfff700 1 -- 172.21.15.163:0/4010822979 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe51c0ce5d0 con 0x7fe51c159950 2023-10-09T06:38:18.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.810+0000 7fe520bde700 1 --2- 172.21.15.163:0/4010822979 >> v2:172.21.15.181:3300/0 conn(0x7fe51c069c70 0x7fe51c0cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:18.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.810+0000 7fe51bfff700 1 --2- 172.21.15.163:0/4010822979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe51c159950 0x7fe51c0cbb90 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fe51c063d90 tx=0x7fe50c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5facf6cc2a4b655a server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:18.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.810+0000 7fe51affd700 1 -- 172.21.15.163:0/4010822979 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c00b820 con 0x7fe51c159950 2023-10-09T06:38:18.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.810+0000 7fe51affd700 1 -- 172.21.15.163:0/4010822979 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50c00b980 con 0x7fe51c159950 2023-10-09T06:38:18.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.810+0000 7fe51affd700 1 -- 172.21.15.163:0/4010822979 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c00bd20 con 0x7fe51c159950 2023-10-09T06:38:18.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.811+0000 7fe522641700 1 -- 172.21.15.163:0/4010822979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe51c159950 msgr2=0x7fe51c0cbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:18.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.811+0000 7fe522641700 1 --2- 172.21.15.163:0/4010822979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe51c159950 0x7fe51c0cbb90 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fe51c063d90 tx=0x7fe50c009250 comp rx=0 tx=0).stop 2023-10-09T06:38:18.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.811+0000 7fe522641700 1 -- 172.21.15.163:0/4010822979 shutdown_connections 2023-10-09T06:38:18.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.811+0000 7fe522641700 1 --2- 172.21.15.163:0/4010822979 >> v2:172.21.15.181:3300/0 conn(0x7fe51c069c70 0x7fe51c0cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:18.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.811+0000 7fe522641700 1 --2- 172.21.15.163:0/4010822979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe51c159950 0x7fe51c0cbb90 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:18.814 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.811+0000 7fe522641700 1 -- 172.21.15.163:0/4010822979 >> 172.21.15.163:0/4010822979 conn(0x7fe51c068e00 msgr2=0x7fe51c0d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:18.814 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.811+0000 7fe522641700 1 -- 172.21.15.163:0/4010822979 shutdown_connections 2023-10-09T06:38:18.814 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.811+0000 7fe522641700 1 -- 172.21.15.163:0/4010822979 wait complete. 2023-10-09T06:38:18.816 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.812+0000 7fe522641700 1 Processor -- start 2023-10-09T06:38:18.816 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.812+0000 7fe522641700 1 -- start start 2023-10-09T06:38:18.816 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.813+0000 7fe522641700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe51c069c70 0x7fe51c1527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:18.816 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.813+0000 7fe522641700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe51c0af320 0x7fe51c0c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:18.816 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.813+0000 7fe522641700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe51c159950 0x7fe51c15d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:18.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.813+0000 7fe522641700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c0c9170 con 0x7fe51c0af320 2023-10-09T06:38:18.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.813+0000 7fe522641700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c0c7c60 con 0x7fe51c159950 2023-10-09T06:38:18.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.813+0000 7fe522641700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c0ca670 con 0x7fe51c069c70 2023-10-09T06:38:18.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.813+0000 7fe51b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe51c0af320 0x7fe51c0c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:18.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.813+0000 7fe51b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe51c0af320 0x7fe51c0c8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42048/0 (socket says 172.21.15.163:42048) 2023-10-09T06:38:18.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.813+0000 7fe51b7fe700 1 -- 172.21.15.163:0/2774799486 learned_addr learned my addr 172.21.15.163:0/2774799486 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:18.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.813+0000 7fe520bde700 1 --2- 172.21.15.163:0/2774799486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe51c159950 0x7fe51c15d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:18.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.813+0000 7fe51bfff700 1 --2- 172.21.15.163:0/2774799486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe51c069c70 0x7fe51c1527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:18.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.813+0000 7fe51b7fe700 1 -- 172.21.15.163:0/2774799486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe51c069c70 msgr2=0x7fe51c1527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:18.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.813+0000 7fe51b7fe700 1 --2- 172.21.15.163:0/2774799486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe51c069c70 0x7fe51c1527e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:18.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.813+0000 7fe51b7fe700 1 -- 172.21.15.163:0/2774799486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe51c159950 msgr2=0x7fe51c15d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:18.817 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.813+0000 7fe51b7fe700 1 --2- 172.21.15.163:0/2774799486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe51c159950 0x7fe51c15d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:18.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.813+0000 7fe51b7fe700 1 -- 172.21.15.163:0/2774799486 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe50c0094f0 con 0x7fe51c0af320 2023-10-09T06:38:18.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.813+0000 7fe520bde700 1 --2- 172.21.15.163:0/2774799486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe51c159950 0x7fe51c15d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:18.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.813+0000 7fe51b7fe700 1 --2- 172.21.15.163:0/2774799486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe51c0af320 0x7fe51c0c8c30 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fe51000c5d0 tx=0x7fe510007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:18.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.813+0000 7fe503fff700 1 -- 172.21.15.163:0/2774799486 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe510011590 con 0x7fe51c0af320 2023-10-09T06:38:18.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.814+0000 7fe503fff700 1 -- 172.21.15.163:0/2774799486 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe510011710 con 0x7fe51c0af320 2023-10-09T06:38:18.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.814+0000 7fe522641700 1 -- 172.21.15.163:0/2774799486 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe51c0cab00 con 0x7fe51c0af320 2023-10-09T06:38:18.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.814+0000 7fe503fff700 1 -- 172.21.15.163:0/2774799486 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe510011ad0 con 0x7fe51c0af320 2023-10-09T06:38:18.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.814+0000 7fe522641700 1 -- 172.21.15.163:0/2774799486 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe51c0c9d40 con 0x7fe51c0af320 2023-10-09T06:38:18.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.814+0000 7fe522641700 1 -- 172.21.15.163:0/2774799486 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe51c0718b0 con 0x7fe51c0af320 2023-10-09T06:38:18.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.815+0000 7fe503fff700 1 -- 172.21.15.163:0/2774799486 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe51001c460 con 0x7fe51c0af320 2023-10-09T06:38:18.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.815+0000 7fe503fff700 1 --2- 172.21.15.163:0/2774799486 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe508067320 0x7fe5080697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:18.818 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.815+0000 7fe503fff700 1 -- 172.21.15.163:0/2774799486 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fe510084950 con 0x7fe51c0af320 2023-10-09T06:38:18.819 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.816+0000 7fe51bfff700 1 --2- 172.21.15.163:0/2774799486 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe508067320 0x7fe5080697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:18.819 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.816+0000 7fe51bfff700 1 --2- 172.21.15.163:0/2774799486 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe508067320 0x7fe5080697d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe51c063d90 tx=0x7fe50c016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:18.820 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:18.820+0000 7fe503fff700 1 -- 172.21.15.163:0/2774799486 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe510051780 con 0x7fe51c0af320 2023-10-09T06:38:19.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.003+0000 7fe522641700 1 -- 172.21.15.163:0/2774799486 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe51c0ce8b0 con 0x7fe51c0af320 2023-10-09T06:38:19.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.673+0000 7fe503fff700 1 -- 172.21.15.163:0/2774799486 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v150) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe510054da0 con 0x7fe51c0af320 2023-10-09T06:38:19.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.688+0000 7fe522641700 1 -- 172.21.15.163:0/2774799486 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe508067320 msgr2=0x7fe5080697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:19.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.688+0000 7fe522641700 1 --2- 172.21.15.163:0/2774799486 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe508067320 0x7fe5080697d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe51c063d90 tx=0x7fe50c016000 comp rx=0 tx=0).stop 2023-10-09T06:38:19.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.688+0000 7fe522641700 1 -- 172.21.15.163:0/2774799486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe51c0af320 msgr2=0x7fe51c0c8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:19.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.688+0000 7fe522641700 1 --2- 172.21.15.163:0/2774799486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe51c0af320 0x7fe51c0c8c30 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fe51000c5d0 tx=0x7fe510007f60 comp rx=0 tx=0).stop 2023-10-09T06:38:19.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.688+0000 7fe522641700 1 -- 172.21.15.163:0/2774799486 shutdown_connections 2023-10-09T06:38:19.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.688+0000 7fe522641700 1 --2- 172.21.15.163:0/2774799486 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe508067320 0x7fe5080697d0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:19.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.688+0000 7fe522641700 1 --2- 172.21.15.163:0/2774799486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe51c159950 0x7fe51c15d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:19.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.688+0000 7fe522641700 1 --2- 172.21.15.163:0/2774799486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe51c069c70 0x7fe51c1527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:19.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.688+0000 7fe522641700 1 --2- 172.21.15.163:0/2774799486 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe51c0af320 0x7fe51c0c8c30 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:19.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.688+0000 7fe522641700 1 -- 172.21.15.163:0/2774799486 >> 172.21.15.163:0/2774799486 conn(0x7fe51c068e00 msgr2=0x7fe51c0d05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:19.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.689+0000 7fe522641700 1 -- 172.21.15.163:0/2774799486 shutdown_connections 2023-10-09T06:38:19.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.689+0000 7fe522641700 1 -- 172.21.15.163:0/2774799486 wait complete. 2023-10-09T06:38:19.691 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_data' created 2023-10-09T06:38:19.704 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-09T06:38:19.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.822+0000 7ffa63b46700 1 Processor -- start 2023-10-09T06:38:19.823 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.822+0000 7ffa63b46700 1 -- start start 2023-10-09T06:38:19.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.823+0000 7ffa63b46700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ffa5c06aa40 0x7ffa5c0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:19.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.823+0000 7ffa63b46700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffa5c06bac0 0x7ffa5c06ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:19.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.823+0000 7ffa63b46700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa5c0cc210 con 0x7ffa5c06b350 2023-10-09T06:38:19.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.823+0000 7ffa63b46700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa5c0cc4f0 con 0x7ffa5c06aa40 2023-10-09T06:38:19.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.823+0000 7ffa63b46700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa5c0cc7d0 con 0x7ffa5c06bac0 2023-10-09T06:38:19.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.823+0000 7ffa618e2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ffa5c06aa40 0x7ffa5c0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:19.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.823+0000 7ffa618e2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ffa5c06aa40 0x7ffa5c0cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42054/0 (socket says 172.21.15.163:42054) 2023-10-09T06:38:19.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.823+0000 7ffa618e2700 1 -- 172.21.15.163:0/2201060162 learned_addr learned my addr 172.21.15.163:0/2201060162 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:19.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.823+0000 7ffa618e2700 1 -- 172.21.15.163:0/2201060162 >> v2:172.21.15.181:3300/0 conn(0x7ffa5c06bac0 msgr2=0x7ffa5c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:19.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.823+0000 7ffa618e2700 1 --2- 172.21.15.163:0/2201060162 >> v2:172.21.15.181:3300/0 conn(0x7ffa5c06bac0 0x7ffa5c06ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:19.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.824+0000 7ffa618e2700 1 -- 172.21.15.163:0/2201060162 >> v1:172.21.15.181:6789/0 conn(0x7ffa5c06b350 legacy=0x7ffa5c0c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:19.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.824+0000 7ffa618e2700 1 -- 172.21.15.163:0/2201060162 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa5c0ccab0 con 0x7ffa5c06aa40 2023-10-09T06:38:19.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.824+0000 7ffa618e2700 1 --2- 172.21.15.163:0/2201060162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa5c06aa40 0x7ffa5c0cbc00 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c0cc140 tx=0x7ffa5800ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5a009f14880f server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:19.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.824+0000 7ffa608e0700 1 -- 172.21.15.163:0/2201060162 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa58010070 con 0x7ffa5c06aa40 2023-10-09T06:38:19.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.824+0000 7ffa608e0700 1 -- 172.21.15.163:0/2201060162 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5800b0d0 con 0x7ffa5c06aa40 2023-10-09T06:38:19.826 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.824+0000 7ffa608e0700 1 -- 172.21.15.163:0/2201060162 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5800dea0 con 0x7ffa5c06aa40 2023-10-09T06:38:19.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.824+0000 7ffa63b46700 1 -- 172.21.15.163:0/2201060162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa5c06aa40 msgr2=0x7ffa5c0cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:19.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.824+0000 7ffa63b46700 1 --2- 172.21.15.163:0/2201060162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa5c06aa40 0x7ffa5c0cbc00 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c0cc140 tx=0x7ffa5800ba50 comp rx=0 tx=0).stop 2023-10-09T06:38:19.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.824+0000 7ffa63b46700 1 -- 172.21.15.163:0/2201060162 shutdown_connections 2023-10-09T06:38:19.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.824+0000 7ffa63b46700 1 --2- 172.21.15.163:0/2201060162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa5c06aa40 0x7ffa5c0cbc00 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:19.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.824+0000 7ffa63b46700 1 --2- 172.21.15.163:0/2201060162 >> v2:172.21.15.181:3300/0 conn(0x7ffa5c06bac0 0x7ffa5c06ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:19.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.824+0000 7ffa63b46700 1 -- 172.21.15.163:0/2201060162 >> 172.21.15.163:0/2201060162 conn(0x7ffa5c0cd670 msgr2=0x7ffa5c0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:19.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.824+0000 7ffa63b46700 1 -- 172.21.15.163:0/2201060162 shutdown_connections 2023-10-09T06:38:19.827 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.824+0000 7ffa63b46700 1 -- 172.21.15.163:0/2201060162 wait complete. 2023-10-09T06:38:19.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.825+0000 7ffa63b46700 1 Processor -- start 2023-10-09T06:38:19.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.826+0000 7ffa63b46700 1 -- start start 2023-10-09T06:38:19.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.826+0000 7ffa63b46700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa5c06aa40 0x7ffa5c0b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:19.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.826+0000 7ffa63b46700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa5c06b350 0x7ffa5c144080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:19.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.826+0000 7ffa63b46700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa5c06bac0 0x7ffa5c0c5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:19.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.826+0000 7ffa63b46700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c0b1420 con 0x7ffa5c06aa40 2023-10-09T06:38:19.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.826+0000 7ffa63b46700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c0b0410 con 0x7ffa5c06bac0 2023-10-09T06:38:19.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.826+0000 7ffa63b46700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c0b0f90 con 0x7ffa5c06b350 2023-10-09T06:38:19.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.826+0000 7ffa618e2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa5c06aa40 0x7ffa5c0b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:19.830 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.826+0000 7ffa618e2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa5c06aa40 0x7ffa5c0b1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42058/0 (socket says 172.21.15.163:42058) 2023-10-09T06:38:19.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.826+0000 7ffa618e2700 1 -- 172.21.15.163:0/1169481366 learned_addr learned my addr 172.21.15.163:0/1169481366 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:19.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.827+0000 7ffa610e1700 1 --2- 172.21.15.163:0/1169481366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa5c06b350 0x7ffa5c144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:19.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.827+0000 7ffa620e3700 1 --2- 172.21.15.163:0/1169481366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa5c06bac0 0x7ffa5c0c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:19.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.827+0000 7ffa618e2700 1 -- 172.21.15.163:0/1169481366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa5c06b350 msgr2=0x7ffa5c144080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:19.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.827+0000 7ffa618e2700 1 --2- 172.21.15.163:0/1169481366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa5c06b350 0x7ffa5c144080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:19.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.827+0000 7ffa618e2700 1 -- 172.21.15.163:0/1169481366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa5c06bac0 msgr2=0x7ffa5c0c5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:19.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.827+0000 7ffa618e2700 1 --2- 172.21.15.163:0/1169481366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa5c06bac0 0x7ffa5c0c5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:19.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.827+0000 7ffa618e2700 1 -- 172.21.15.163:0/1169481366 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa5800bcf0 con 0x7ffa5c06aa40 2023-10-09T06:38:19.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.827+0000 7ffa618e2700 1 --2- 172.21.15.163:0/1169481366 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa5c06aa40 0x7ffa5c0b1a60 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7ffa5800d910 tx=0x7ffa58019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:19.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.827+0000 7ffa51ffb700 1 -- 172.21.15.163:0/1169481366 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa58009cd0 con 0x7ffa5c06aa40 2023-10-09T06:38:19.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.827+0000 7ffa51ffb700 1 -- 172.21.15.163:0/1169481366 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa58009e30 con 0x7ffa5c06aa40 2023-10-09T06:38:19.831 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.827+0000 7ffa51ffb700 1 -- 172.21.15.163:0/1169481366 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa58014680 con 0x7ffa5c06aa40 2023-10-09T06:38:19.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.827+0000 7ffa63b46700 1 -- 172.21.15.163:0/1169481366 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa5c0b1fa0 con 0x7ffa5c06aa40 2023-10-09T06:38:19.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.827+0000 7ffa63b46700 1 -- 172.21.15.163:0/1169481366 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5c0b0940 con 0x7ffa5c06aa40 2023-10-09T06:38:19.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.828+0000 7ffa63b46700 1 -- 172.21.15.163:0/1169481366 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa5c04c3c0 con 0x7ffa5c06aa40 2023-10-09T06:38:19.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.829+0000 7ffa51ffb700 1 -- 172.21.15.163:0/1169481366 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ffa58014a70 con 0x7ffa5c06aa40 2023-10-09T06:38:19.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.829+0000 7ffa51ffb700 1 --2- 172.21.15.163:0/1169481366 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffa40067320 0x7ffa400697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:19.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.829+0000 7ffa51ffb700 1 -- 172.21.15.163:0/1169481366 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5605+0+0 (secure 0 0 0) 0x7ffa58088790 con 0x7ffa5c06aa40 2023-10-09T06:38:19.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.829+0000 7ffa610e1700 1 --2- 172.21.15.163:0/1169481366 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffa40067320 0x7ffa400697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:19.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.830+0000 7ffa610e1700 1 --2- 172.21.15.163:0/1169481366 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffa40067320 0x7ffa400697d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c0c3ea0 tx=0x7ffa4c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:19.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:19.833+0000 7ffa51ffb700 1 -- 172.21.15.163:0/1169481366 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa58055370 con 0x7ffa5c06aa40 2023-10-09T06:38:20.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.058+0000 7ffa63b46700 1 -- 172.21.15.163:0/1169481366 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ffa5c0c7cb0 con 0x7ffa5c06aa40 2023-10-09T06:38:20.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.687+0000 7ffa51ffb700 1 -- 172.21.15.163:0/1169481366 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 26 and data pool 27 v99) v1 ==== 174+0+0 (secure 0 0 0) 0x7ffa58058990 con 0x7ffa5c06aa40 2023-10-09T06:38:20.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.695+0000 7ffa63b46700 1 -- 172.21.15.163:0/1169481366 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffa40067320 msgr2=0x7ffa400697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:20.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.695+0000 7ffa63b46700 1 --2- 172.21.15.163:0/1169481366 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffa40067320 0x7ffa400697d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c0c3ea0 tx=0x7ffa4c008040 comp rx=0 tx=0).stop 2023-10-09T06:38:20.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.695+0000 7ffa63b46700 1 -- 172.21.15.163:0/1169481366 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa5c06aa40 msgr2=0x7ffa5c0b1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:20.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.695+0000 7ffa63b46700 1 --2- 172.21.15.163:0/1169481366 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa5c06aa40 0x7ffa5c0b1a60 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7ffa5800d910 tx=0x7ffa58019000 comp rx=0 tx=0).stop 2023-10-09T06:38:20.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.696+0000 7ffa63b46700 1 -- 172.21.15.163:0/1169481366 shutdown_connections 2023-10-09T06:38:20.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.696+0000 7ffa63b46700 1 --2- 172.21.15.163:0/1169481366 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffa40067320 0x7ffa400697d0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:20.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.696+0000 7ffa63b46700 1 --2- 172.21.15.163:0/1169481366 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa5c06aa40 0x7ffa5c0b1a60 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:20.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.696+0000 7ffa63b46700 1 --2- 172.21.15.163:0/1169481366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa5c06bac0 0x7ffa5c0c5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:20.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.696+0000 7ffa63b46700 1 --2- 172.21.15.163:0/1169481366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa5c06b350 0x7ffa5c144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:20.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.696+0000 7ffa63b46700 1 -- 172.21.15.163:0/1169481366 >> 172.21.15.163:0/1169481366 conn(0x7ffa5c0cd670 msgr2=0x7ffa5c0c86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:20.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.697+0000 7ffa63b46700 1 -- 172.21.15.163:0/1169481366 shutdown_connections 2023-10-09T06:38:20.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.697+0000 7ffa63b46700 1 -- 172.21.15.163:0/1169481366 wait complete. 2023-10-09T06:38:20.701 INFO:teuthology.orchestra.run.smithi163.stderr:new fs with metadata pool 26 and data pool 27 2023-10-09T06:38:20.716 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:38:20.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.832+0000 7f0b8c81e700 1 Processor -- start 2023-10-09T06:38:20.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.832+0000 7f0b8c81e700 1 -- start start 2023-10-09T06:38:20.836 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.833+0000 7f0b8c81e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b8806c0f0 0x7f0b88159c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:20.836 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.833+0000 7f0b8c81e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0b880b3810 0x7f0b880b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:20.836 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.833+0000 7f0b8c81e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b88143420 con 0x7f0b88143050 2023-10-09T06:38:20.836 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.833+0000 7f0b8c81e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b8806a1e0 con 0x7f0b880b3810 2023-10-09T06:38:20.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.833+0000 7f0b8c81e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b880695d0 con 0x7f0b8806c0f0 2023-10-09T06:38:20.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.833+0000 7f0b87fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0b880b3810 0x7f0b880b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:20.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.833+0000 7f0b87fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0b880b3810 0x7f0b880b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42068/0 (socket says 172.21.15.163:42068) 2023-10-09T06:38:20.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.833+0000 7f0b87fff700 1 -- 172.21.15.163:0/3270827899 learned_addr learned my addr 172.21.15.163:0/3270827899 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:20.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.833+0000 7f0b87fff700 1 -- 172.21.15.163:0/3270827899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b8806c0f0 msgr2=0x7f0b88159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:20.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.833+0000 7f0b86ffd700 1 --2- 172.21.15.163:0/3270827899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b8806c0f0 0x7f0b88159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:20.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.833+0000 7f0b87fff700 1 --2- 172.21.15.163:0/3270827899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b8806c0f0 0x7f0b88159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:20.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.833+0000 7f0b87fff700 1 -- 172.21.15.163:0/3270827899 >> v1:172.21.15.181:6789/0 conn(0x7f0b88143050 legacy=0x7f0b8806b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:20.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.833+0000 7f0b87fff700 1 -- 172.21.15.163:0/3270827899 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b880698b0 con 0x7f0b880b3810 2023-10-09T06:38:20.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.833+0000 7f0b86ffd700 1 --2- 172.21.15.163:0/3270827899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b8806c0f0 0x7f0b88159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:20.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.834+0000 7f0b87fff700 1 --2- 172.21.15.163:0/3270827899 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b880b3810 0x7f0b880b3c90 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f0b80009f20 tx=0x7f0b80009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527f3ef47600ab65 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:20.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.834+0000 7f0b867fc700 1 -- 172.21.15.163:0/3270827899 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8000e070 con 0x7f0b880b3810 2023-10-09T06:38:20.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.834+0000 7f0b867fc700 1 -- 172.21.15.163:0/3270827899 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b8000ba40 con 0x7f0b880b3810 2023-10-09T06:38:20.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.834+0000 7f0b867fc700 1 -- 172.21.15.163:0/3270827899 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b800092e0 con 0x7f0b880b3810 2023-10-09T06:38:20.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.834+0000 7f0b8c81e700 1 -- 172.21.15.163:0/3270827899 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b880b3810 msgr2=0x7f0b880b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:20.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.834+0000 7f0b8c81e700 1 --2- 172.21.15.163:0/3270827899 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b880b3810 0x7f0b880b3c90 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f0b80009f20 tx=0x7f0b80009840 comp rx=0 tx=0).stop 2023-10-09T06:38:20.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.834+0000 7f0b8c81e700 1 -- 172.21.15.163:0/3270827899 shutdown_connections 2023-10-09T06:38:20.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.834+0000 7f0b8c81e700 1 --2- 172.21.15.163:0/3270827899 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b880b3810 0x7f0b880b3c90 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:20.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.834+0000 7f0b8c81e700 1 --2- 172.21.15.163:0/3270827899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b8806c0f0 0x7f0b88159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:20.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.834+0000 7f0b8c81e700 1 -- 172.21.15.163:0/3270827899 >> 172.21.15.163:0/3270827899 conn(0x7f0b880ce430 msgr2=0x7f0b880ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:20.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.834+0000 7f0b8c81e700 1 -- 172.21.15.163:0/3270827899 shutdown_connections 2023-10-09T06:38:20.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.834+0000 7f0b8c81e700 1 -- 172.21.15.163:0/3270827899 wait complete. 2023-10-09T06:38:20.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.835+0000 7f0b8c81e700 1 Processor -- start 2023-10-09T06:38:20.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.836+0000 7f0b8c81e700 1 -- start start 2023-10-09T06:38:20.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.836+0000 7f0b8c81e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b8806c0f0 0x7f0b8813b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:20.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.836+0000 7f0b8c81e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b880b3810 0x7f0b8815c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:20.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.836+0000 7f0b8c81e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b88143050 0x7f0b881395f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:20.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.836+0000 7f0b8c81e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b88138750 con 0x7f0b880b3810 2023-10-09T06:38:20.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.836+0000 7f0b8c81e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8813acc0 con 0x7f0b88143050 2023-10-09T06:38:20.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.836+0000 7f0b8c81e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8813a0b0 con 0x7f0b8806c0f0 2023-10-09T06:38:20.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.836+0000 7f0b86ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b880b3810 0x7f0b8815c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:20.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.836+0000 7f0b86ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b880b3810 0x7f0b8815c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42074/0 (socket says 172.21.15.163:42074) 2023-10-09T06:38:20.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.836+0000 7f0b86ffd700 1 -- 172.21.15.163:0/336596717 learned_addr learned my addr 172.21.15.163:0/336596717 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:20.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.836+0000 7f0b877fe700 1 --2- 172.21.15.163:0/336596717 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b8806c0f0 0x7f0b8813b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:20.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.836+0000 7f0b87fff700 1 --2- 172.21.15.163:0/336596717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b88143050 0x7f0b881395f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:20.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.837+0000 7f0b86ffd700 1 -- 172.21.15.163:0/336596717 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b8806c0f0 msgr2=0x7f0b8813b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:20.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.837+0000 7f0b86ffd700 1 --2- 172.21.15.163:0/336596717 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b8806c0f0 0x7f0b8813b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:20.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.837+0000 7f0b86ffd700 1 -- 172.21.15.163:0/336596717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b88143050 msgr2=0x7f0b881395f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:20.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.837+0000 7f0b86ffd700 1 --2- 172.21.15.163:0/336596717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b88143050 0x7f0b881395f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:20.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.837+0000 7f0b86ffd700 1 -- 172.21.15.163:0/336596717 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b80009ae0 con 0x7f0b880b3810 2023-10-09T06:38:20.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.837+0000 7f0b87fff700 1 --2- 172.21.15.163:0/336596717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b88143050 0x7f0b881395f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:20.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.837+0000 7f0b86ffd700 1 --2- 172.21.15.163:0/336596717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b880b3810 0x7f0b8815c440 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00c830 tx=0x7f0b7c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:20.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.837+0000 7f0b6f7fe700 1 -- 172.21.15.163:0/336596717 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c011600 con 0x7f0b880b3810 2023-10-09T06:38:20.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.837+0000 7f0b6f7fe700 1 -- 172.21.15.163:0/336596717 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b7c011760 con 0x7f0b880b3810 2023-10-09T06:38:20.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.837+0000 7f0b6f7fe700 1 -- 172.21.15.163:0/336596717 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c011b20 con 0x7f0b880b3810 2023-10-09T06:38:20.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.837+0000 7f0b8c81e700 1 -- 172.21.15.163:0/336596717 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b8813a390 con 0x7f0b880b3810 2023-10-09T06:38:20.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.837+0000 7f0b8c81e700 1 -- 172.21.15.163:0/336596717 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b8813a950 con 0x7f0b880b3810 2023-10-09T06:38:20.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.838+0000 7f0b8c81e700 1 -- 172.21.15.163:0/336596717 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b8813ae30 con 0x7f0b880b3810 2023-10-09T06:38:20.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.838+0000 7f0b6f7fe700 1 -- 172.21.15.163:0/336596717 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0b7c0193b0 con 0x7f0b880b3810 2023-10-09T06:38:20.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.839+0000 7f0b6f7fe700 1 --2- 172.21.15.163:0/336596717 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0b74067320 0x7f0b740697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:20.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.839+0000 7f0b6f7fe700 1 -- 172.21.15.163:0/336596717 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0b7c084d60 con 0x7f0b880b3810 2023-10-09T06:38:20.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.839+0000 7f0b877fe700 1 --2- 172.21.15.163:0/336596717 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0b74067320 0x7f0b740697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:20.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.839+0000 7f0b877fe700 1 --2- 172.21.15.163:0/336596717 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0b74067320 0x7f0b740697d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0b78006fd0 tx=0x7f0b78009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:20.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:20.843+0000 7f0b6f7fe700 1 -- 172.21.15.163:0/336596717 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0b7c0518e0 con 0x7f0b880b3810 2023-10-09T06:38:21.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.025+0000 7f0b8c81e700 1 -- 172.21.15.163:0/336596717 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b8804c3c0 con 0x7f0b880b3810 2023-10-09T06:38:21.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.026+0000 7f0b6f7fe700 1 -- 172.21.15.163:0/336596717 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+15425 (secure 0 0 0) 0x7f0b7c015090 con 0x7f0b880b3810 2023-10-09T06:38:21.027 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:21.027 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":151,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:38:20.059860+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:38:18.381850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-09T06:38:19.022637+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:38:21.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.028+0000 7f0b8c81e700 1 -- 172.21.15.163:0/336596717 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0b74067320 msgr2=0x7f0b740697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:21.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.028+0000 7f0b8c81e700 1 --2- 172.21.15.163:0/336596717 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0b74067320 0x7f0b740697d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0b78006fd0 tx=0x7f0b78009060 comp rx=0 tx=0).stop 2023-10-09T06:38:21.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.028+0000 7f0b8c81e700 1 -- 172.21.15.163:0/336596717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b880b3810 msgr2=0x7f0b8815c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:21.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.028+0000 7f0b8c81e700 1 --2- 172.21.15.163:0/336596717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b880b3810 0x7f0b8815c440 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00c830 tx=0x7f0b7c008040 comp rx=0 tx=0).stop 2023-10-09T06:38:21.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.028+0000 7f0b8c81e700 1 -- 172.21.15.163:0/336596717 shutdown_connections 2023-10-09T06:38:21.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.028+0000 7f0b8c81e700 1 --2- 172.21.15.163:0/336596717 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0b74067320 0x7f0b740697d0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:21.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.028+0000 7f0b8c81e700 1 --2- 172.21.15.163:0/336596717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b88143050 0x7f0b881395f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:21.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.028+0000 7f0b8c81e700 1 --2- 172.21.15.163:0/336596717 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b8806c0f0 0x7f0b8813b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:21.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.028+0000 7f0b8c81e700 1 --2- 172.21.15.163:0/336596717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b880b3810 0x7f0b8815c440 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:21.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.028+0000 7f0b8c81e700 1 -- 172.21.15.163:0/336596717 >> 172.21.15.163:0/336596717 conn(0x7f0b880ce430 msgr2=0x7f0b880cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:21.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.028+0000 7f0b8c81e700 1 -- 172.21.15.163:0/336596717 shutdown_connections 2023-10-09T06:38:21.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.028+0000 7f0b8c81e700 1 -- 172.21.15.163:0/336596717 wait complete. 2023-10-09T06:38:21.041 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:38:21.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.156+0000 7fddc7422700 1 Processor -- start 2023-10-09T06:38:21.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.156+0000 7fddc7422700 1 -- start start 2023-10-09T06:38:21.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.157+0000 7fddc7422700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fddc0159c60 0x7fddc015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:21.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.157+0000 7fddc7422700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc015a160 con 0x7fddc015db10 2023-10-09T06:38:21.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.157+0000 7fddc7422700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc00d13a0 con 0x7fddc0159890 2023-10-09T06:38:21.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.157+0000 7fddc7422700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc015df50 con 0x7fddc0159c60 2023-10-09T06:38:21.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.157+0000 7fddc49bd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fddc0159c60 0x7fddc015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:21.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.157+0000 7fddc59bf700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fddc015db10 0x7fddc0069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33602/0 (socket says 172.21.15.163:33602) 2023-10-09T06:38:21.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.157+0000 7fddc59bf700 1 -- 172.21.15.163:0/184888817 learned_addr learned my addr 172.21.15.163:0/184888817 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:21.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.157+0000 7fddc49bd700 1 -- 172.21.15.163:0/184888817 >> v1:172.21.15.181:6789/0 conn(0x7fddc0159890 legacy=0x7fddc015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:21.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.157+0000 7fddc49bd700 1 -- 172.21.15.163:0/184888817 >> v1:172.21.15.163:6789/0 conn(0x7fddc015db10 legacy=0x7fddc0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:21.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.157+0000 7fddb7fff700 1 -- 172.21.15.163:0/184888817 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2505160810 0 0) 0x7fddc015a160 con 0x7fddc015db10 2023-10-09T06:38:21.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.157+0000 7fddc49bd700 1 -- 172.21.15.163:0/184888817 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc015e140 con 0x7fddc0159c60 2023-10-09T06:38:21.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.158+0000 7fddc49bd700 1 --2- 172.21.15.163:0/184888817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddc0159c60 0x7fddc015d5d0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fddb00098e0 tx=0x7fddb00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07a6550853c2c92 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:21.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.158+0000 7fddb7fff700 1 -- 172.21.15.163:0/184888817 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb0009e80 con 0x7fddc0159c60 2023-10-09T06:38:21.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.158+0000 7fddb7fff700 1 -- 172.21.15.163:0/184888817 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb000b860 con 0x7fddc0159c60 2023-10-09T06:38:21.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.158+0000 7fddb7fff700 1 -- 172.21.15.163:0/184888817 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb000bbe0 con 0x7fddc0159c60 2023-10-09T06:38:21.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.158+0000 7fddc7422700 1 -- 172.21.15.163:0/184888817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddc0159c60 msgr2=0x7fddc015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:21.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.158+0000 7fddc7422700 1 --2- 172.21.15.163:0/184888817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddc0159c60 0x7fddc015d5d0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fddb00098e0 tx=0x7fddb00091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:21.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.158+0000 7fddc7422700 1 -- 172.21.15.163:0/184888817 shutdown_connections 2023-10-09T06:38:21.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.158+0000 7fddc7422700 1 --2- 172.21.15.163:0/184888817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddc0159c60 0x7fddc015d5d0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:21.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.158+0000 7fddc7422700 1 -- 172.21.15.163:0/184888817 >> 172.21.15.163:0/184888817 conn(0x7fddc0068e20 msgr2=0x7fddc00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:21.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.158+0000 7fddc7422700 1 -- 172.21.15.163:0/184888817 shutdown_connections 2023-10-09T06:38:21.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.158+0000 7fddc7422700 1 -- 172.21.15.163:0/184888817 wait complete. 2023-10-09T06:38:21.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.159+0000 7fddc7422700 1 Processor -- start 2023-10-09T06:38:21.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.159+0000 7fddc7422700 1 -- start start 2023-10-09T06:38:21.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.160+0000 7fddc7422700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fddc0159890 0x7fddc0155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:21.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.160+0000 7fddc7422700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddc0159c60 0x7fddc0148d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:21.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.160+0000 7fddc7422700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddc015db10 0x7fddc01492b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:21.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.160+0000 7fddc7422700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc01497f0 con 0x7fddc0159c60 2023-10-09T06:38:21.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.160+0000 7fddc7422700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc00cfcf0 con 0x7fddc015db10 2023-10-09T06:38:21.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.160+0000 7fddc7422700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc016d280 con 0x7fddc0159890 2023-10-09T06:38:21.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.160+0000 7fddc49bd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddc0159c60 0x7fddc0148d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:21.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.160+0000 7fddc49bd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddc0159c60 0x7fddc0148d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42090/0 (socket says 172.21.15.163:42090) 2023-10-09T06:38:21.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.160+0000 7fddc49bd700 1 -- 172.21.15.163:0/2923835343 learned_addr learned my addr 172.21.15.163:0/2923835343 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:21.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.160+0000 7fddc59bf700 1 --2- 172.21.15.163:0/2923835343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddc015db10 0x7fddc01492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:21.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.160+0000 7fddc51be700 1 --2- 172.21.15.163:0/2923835343 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fddc0159890 0x7fddc0155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:21.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.161+0000 7fddc49bd700 1 -- 172.21.15.163:0/2923835343 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fddc0159890 msgr2=0x7fddc0155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:21.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.161+0000 7fddc49bd700 1 --2- 172.21.15.163:0/2923835343 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fddc0159890 0x7fddc0155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:21.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.161+0000 7fddc49bd700 1 -- 172.21.15.163:0/2923835343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddc015db10 msgr2=0x7fddc01492b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:21.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.161+0000 7fddc49bd700 1 --2- 172.21.15.163:0/2923835343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddc015db10 0x7fddc01492b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:21.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.161+0000 7fddc49bd700 1 -- 172.21.15.163:0/2923835343 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb0009420 con 0x7fddc0159c60 2023-10-09T06:38:21.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.161+0000 7fddc59bf700 1 --2- 172.21.15.163:0/2923835343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddc015db10 0x7fddc01492b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:21.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.161+0000 7fddc51be700 1 --2- 172.21.15.163:0/2923835343 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fddc0159890 0x7fddc0155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:21.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.161+0000 7fddc49bd700 1 --2- 172.21.15.163:0/2923835343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddc0159c60 0x7fddc0148d70 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fddc0070e20 tx=0x7fddb0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:21.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.161+0000 7fddb57fa700 1 -- 172.21.15.163:0/2923835343 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb0011420 con 0x7fddc0159c60 2023-10-09T06:38:21.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.161+0000 7fddb57fa700 1 -- 172.21.15.163:0/2923835343 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb0011580 con 0x7fddc0159c60 2023-10-09T06:38:21.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.161+0000 7fddb57fa700 1 -- 172.21.15.163:0/2923835343 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb0011920 con 0x7fddc0159c60 2023-10-09T06:38:21.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.161+0000 7fddc7422700 1 -- 172.21.15.163:0/2923835343 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc016d560 con 0x7fddc0159c60 2023-10-09T06:38:21.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.161+0000 7fddc7422700 1 -- 172.21.15.163:0/2923835343 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc016db20 con 0x7fddc0159c60 2023-10-09T06:38:21.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.162+0000 7fddc7422700 1 -- 172.21.15.163:0/2923835343 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc004c3c0 con 0x7fddc0159c60 2023-10-09T06:38:21.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.162+0000 7fddb57fa700 1 -- 172.21.15.163:0/2923835343 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fddb0011c90 con 0x7fddc0159c60 2023-10-09T06:38:21.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.163+0000 7fddb57fa700 1 --2- 172.21.15.163:0/2923835343 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdda4067320 0x7fdda40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:21.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.163+0000 7fddb57fa700 1 -- 172.21.15.163:0/2923835343 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fddb0086640 con 0x7fddc0159c60 2023-10-09T06:38:21.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.163+0000 7fddc51be700 1 --2- 172.21.15.163:0/2923835343 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdda4067320 0x7fdda40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:21.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.164+0000 7fddc51be700 1 --2- 172.21.15.163:0/2923835343 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdda4067320 0x7fdda40697d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fddb8007fd0 tx=0x7fddb8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:21.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.167+0000 7fddb57fa700 1 -- 172.21.15.163:0/2923835343 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fddb0053450 con 0x7fddc0159c60 2023-10-09T06:38:21.352 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.351+0000 7fddc7422700 1 -- 172.21.15.163:0/2923835343 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fddc00890e0 con 0x7fddc0159c60 2023-10-09T06:38:21.353 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.352+0000 7fddb57fa700 1 -- 172.21.15.163:0/2923835343 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+15425 (secure 0 0 0) 0x7fddb0056a70 con 0x7fddc0159c60 2023-10-09T06:38:21.353 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:21.353 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":151,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:38:20.059860+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:38:18.381850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-09T06:38:19.022637+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:38:21.355 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.354+0000 7fddc7422700 1 -- 172.21.15.163:0/2923835343 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdda4067320 msgr2=0x7fdda40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:21.355 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.354+0000 7fddc7422700 1 --2- 172.21.15.163:0/2923835343 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdda4067320 0x7fdda40697d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fddb8007fd0 tx=0x7fddb8009040 comp rx=0 tx=0).stop 2023-10-09T06:38:21.355 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.354+0000 7fddc7422700 1 -- 172.21.15.163:0/2923835343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddc0159c60 msgr2=0x7fddc0148d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:21.355 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.354+0000 7fddc7422700 1 --2- 172.21.15.163:0/2923835343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddc0159c60 0x7fddc0148d70 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fddc0070e20 tx=0x7fddb0017000 comp rx=0 tx=0).stop 2023-10-09T06:38:21.355 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.354+0000 7fddc7422700 1 -- 172.21.15.163:0/2923835343 shutdown_connections 2023-10-09T06:38:21.355 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.354+0000 7fddc7422700 1 --2- 172.21.15.163:0/2923835343 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdda4067320 0x7fdda40697d0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:21.356 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.354+0000 7fddc7422700 1 --2- 172.21.15.163:0/2923835343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fddc015db10 0x7fddc01492b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:21.356 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.354+0000 7fddc7422700 1 --2- 172.21.15.163:0/2923835343 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fddc0159890 0x7fddc0155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:21.356 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.354+0000 7fddc7422700 1 --2- 172.21.15.163:0/2923835343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fddc0159c60 0x7fddc0148d70 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:21.356 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.354+0000 7fddc7422700 1 -- 172.21.15.163:0/2923835343 >> 172.21.15.163:0/2923835343 conn(0x7fddc0068e20 msgr2=0x7fddc015cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:21.356 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.354+0000 7fddc7422700 1 -- 172.21.15.163:0/2923835343 shutdown_connections 2023-10-09T06:38:21.356 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.354+0000 7fddc7422700 1 -- 172.21.15.163:0/2923835343 wait complete. 2023-10-09T06:38:21.367 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-09T06:38:21.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.484+0000 7f95882d4700 1 Processor -- start 2023-10-09T06:38:21.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.484+0000 7f95882d4700 1 -- start start 2023-10-09T06:38:21.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.485+0000 7f95882d4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f958006bcd0 0x7f95800caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:21.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.485+0000 7f95882d4700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f958015d7e0 con 0x7f958015df50 2023-10-09T06:38:21.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.485+0000 7f95882d4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f958015dac0 con 0x7f958006c0a0 2023-10-09T06:38:21.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.485+0000 7f95882d4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958015aa30 con 0x7f958006bcd0 2023-10-09T06:38:21.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.486+0000 7f9586871700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f958015df50 0x7f95800c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33608/0 (socket says 172.21.15.163:33608) 2023-10-09T06:38:21.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.486+0000 7f9586871700 1 -- 172.21.15.163:0/593895627 learned_addr learned my addr 172.21.15.163:0/593895627 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:21.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.486+0000 7f9586070700 1 --2- 172.21.15.163:0/593895627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f958006bcd0 0x7f95800caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:21.489 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.486+0000 7f958506e700 1 -- 172.21.15.163:0/593895627 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3225121884 0 0) 0x7f958015d7e0 con 0x7f958015df50 2023-10-09T06:38:21.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.486+0000 7f958506e700 1 -- 172.21.15.163:0/593895627 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f956c003570 con 0x7f958015df50 2023-10-09T06:38:21.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.486+0000 7f958506e700 1 -- 172.21.15.163:0/593895627 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 706997340 0 0) 0x7f956c003570 con 0x7f958015df50 2023-10-09T06:38:21.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.486+0000 7f958506e700 1 -- 172.21.15.163:0/593895627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f958006bcd0 msgr2=0x7f95800caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:21.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.486+0000 7f958506e700 1 --2- 172.21.15.163:0/593895627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f958006bcd0 0x7f95800caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:21.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.486+0000 7f958506e700 1 -- 172.21.15.163:0/593895627 >> v1:172.21.15.181:6789/0 conn(0x7f958006c0a0 legacy=0x7f958015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:21.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.486+0000 7f958506e700 1 -- 172.21.15.163:0/593895627 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958015ad10 con 0x7f958015df50 2023-10-09T06:38:21.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.486+0000 7f958506e700 1 -- 172.21.15.163:0/593895627 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f9578001470 con 0x7f958015df50 2023-10-09T06:38:21.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.486+0000 7f958506e700 1 -- 172.21.15.163:0/593895627 >> v1:172.21.15.163:6789/0 conn(0x7f958015df50 legacy=0x7f95800c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:21.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.486+0000 7f958506e700 1 --2- 172.21.15.163:0/593895627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f956c003a90 0x7f956c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:21.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.486+0000 7f958506e700 1 --2- 172.21.15.163:0/593895627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f956c004810 0x7f956c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:21.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.486+0000 7f958506e700 1 --2- 172.21.15.163:0/593895627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f958006bcd0 0x7f956c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:21.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.486+0000 7f958506e700 1 -- 172.21.15.163:0/593895627 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958015aa30 con 0x7f956c004810 2023-10-09T06:38:21.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.486+0000 7f958506e700 1 -- 172.21.15.163:0/593895627 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956c00bb90 con 0x7f956c003a90 2023-10-09T06:38:21.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.486+0000 7f958506e700 1 -- 172.21.15.163:0/593895627 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f956c00bd50 con 0x7f958006bcd0 2023-10-09T06:38:21.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.486+0000 7f958586f700 1 --2- 172.21.15.163:0/593895627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f956c004810 0x7f956c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:21.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.486+0000 7f9586070700 1 --2- 172.21.15.163:0/593895627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f956c003a90 0x7f956c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:21.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.486+0000 7f9586871700 1 --2- 172.21.15.163:0/593895627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f958006bcd0 0x7f956c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:21.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.487+0000 7f958586f700 1 -- 172.21.15.163:0/593895627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f958006bcd0 msgr2=0x7f956c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:21.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.487+0000 7f958586f700 1 --2- 172.21.15.163:0/593895627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f958006bcd0 0x7f956c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:21.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.487+0000 7f958586f700 1 -- 172.21.15.163:0/593895627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f956c003a90 msgr2=0x7f956c003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:21.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.487+0000 7f958586f700 1 --2- 172.21.15.163:0/593895627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f956c003a90 0x7f956c003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:21.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.487+0000 7f958586f700 1 -- 172.21.15.163:0/593895627 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f956c003570 con 0x7f956c004810 2023-10-09T06:38:21.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.487+0000 7f958586f700 1 --2- 172.21.15.163:0/593895627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f956c004810 0x7f956c004c80 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f9570001a60 tx=0x7f9570007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9612d7db0646f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:21.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.487+0000 7f958506e700 1 -- 172.21.15.163:0/593895627 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570001ea0 con 0x7f956c004810 2023-10-09T06:38:21.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.487+0000 7f958506e700 1 -- 172.21.15.163:0/593895627 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f957000c590 con 0x7f956c004810 2023-10-09T06:38:21.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.487+0000 7f958506e700 1 -- 172.21.15.163:0/593895627 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957000c930 con 0x7f956c004810 2023-10-09T06:38:21.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.487+0000 7f95882d4700 1 -- 172.21.15.163:0/593895627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f956c004810 msgr2=0x7f956c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:21.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.487+0000 7f95882d4700 1 --2- 172.21.15.163:0/593895627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f956c004810 0x7f956c004c80 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f9570001a60 tx=0x7f9570007d30 comp rx=0 tx=0).stop 2023-10-09T06:38:21.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.487+0000 7f9586871700 1 -- 172.21.15.163:0/593895627 reap_dead start 2023-10-09T06:38:21.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.487+0000 7f95882d4700 1 -- 172.21.15.163:0/593895627 shutdown_connections 2023-10-09T06:38:21.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.487+0000 7f95882d4700 1 -- 172.21.15.163:0/593895627 >> 172.21.15.163:0/593895627 conn(0x7f95800ce600 msgr2=0x7f95800cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:21.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.487+0000 7f95882d4700 1 -- 172.21.15.163:0/593895627 shutdown_connections 2023-10-09T06:38:21.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.488+0000 7f95882d4700 1 -- 172.21.15.163:0/593895627 wait complete. 2023-10-09T06:38:21.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.489+0000 7f95882d4700 1 Processor -- start 2023-10-09T06:38:21.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.489+0000 7f95882d4700 1 -- start start 2023-10-09T06:38:21.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.489+0000 7f95882d4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f956c004810 0x7f9580143e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:21.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.489+0000 7f95882d4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95800cb250 0x7f958006cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:21.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.489+0000 7f95882d4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95800cb6c0 0x7f958006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:21.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.489+0000 7f95882d4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9580169d70 con 0x7f956c004810 2023-10-09T06:38:21.495 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.489+0000 7f95882d4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95800c5a00 con 0x7f95800cb6c0 2023-10-09T06:38:21.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.489+0000 7f95882d4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9580144370 con 0x7f95800cb250 2023-10-09T06:38:21.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.489+0000 7f9586070700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f956c004810 0x7f9580143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:21.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.489+0000 7f9586070700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f956c004810 0x7f9580143e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42112/0 (socket says 172.21.15.163:42112) 2023-10-09T06:38:21.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.489+0000 7f9586070700 1 -- 172.21.15.163:0/1790896043 learned_addr learned my addr 172.21.15.163:0/1790896043 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:21.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.490+0000 7f9586070700 1 -- 172.21.15.163:0/1790896043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95800cb250 msgr2=0x7f958006cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:21.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.490+0000 7f9586871700 1 --2- 172.21.15.163:0/1790896043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95800cb6c0 0x7f958006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:21.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.490+0000 7f958586f700 1 --2- 172.21.15.163:0/1790896043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95800cb250 0x7f958006cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:21.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.490+0000 7f9586070700 1 --2- 172.21.15.163:0/1790896043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95800cb250 0x7f958006cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:21.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.490+0000 7f9586070700 1 -- 172.21.15.163:0/1790896043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95800cb6c0 msgr2=0x7f958006d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:21.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.490+0000 7f9586070700 1 --2- 172.21.15.163:0/1790896043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95800cb6c0 0x7f958006d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:21.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.490+0000 7f9586070700 1 -- 172.21.15.163:0/1790896043 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95800c89a0 con 0x7f956c004810 2023-10-09T06:38:21.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.490+0000 7f958586f700 1 --2- 172.21.15.163:0/1790896043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95800cb250 0x7f958006cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:21.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.490+0000 7f9586871700 1 --2- 172.21.15.163:0/1790896043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95800cb6c0 0x7f958006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:21.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.490+0000 7f9586070700 1 --2- 172.21.15.163:0/1790896043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f956c004810 0x7f9580143e30 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f957c011f80 tx=0x7f957c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:21.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.490+0000 7f95767fc700 1 -- 172.21.15.163:0/1790896043 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c00e040 con 0x7f956c004810 2023-10-09T06:38:21.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.490+0000 7f95767fc700 1 -- 172.21.15.163:0/1790896043 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f957c00f040 con 0x7f956c004810 2023-10-09T06:38:21.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.490+0000 7f95882d4700 1 -- 172.21.15.163:0/1790896043 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95800c7900 con 0x7f956c004810 2023-10-09T06:38:21.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.490+0000 7f95767fc700 1 -- 172.21.15.163:0/1790896043 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c015430 con 0x7f956c004810 2023-10-09T06:38:21.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.490+0000 7f95882d4700 1 -- 172.21.15.163:0/1790896043 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95800c7ec0 con 0x7f956c004810 2023-10-09T06:38:21.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.491+0000 7f95882d4700 1 -- 172.21.15.163:0/1790896043 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f958004c3c0 con 0x7f956c004810 2023-10-09T06:38:21.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.491+0000 7f95767fc700 1 -- 172.21.15.163:0/1790896043 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f957c0157e0 con 0x7f956c004810 2023-10-09T06:38:21.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.492+0000 7f95767fc700 1 --2- 172.21.15.163:0/1790896043 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9564067320 0x7f95640697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:21.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.492+0000 7f95767fc700 1 -- 172.21.15.163:0/1790896043 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f957c085b30 con 0x7f956c004810 2023-10-09T06:38:21.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.492+0000 7f958586f700 1 --2- 172.21.15.163:0/1790896043 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9564067320 0x7f95640697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:21.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.492+0000 7f958586f700 1 --2- 172.21.15.163:0/1790896043 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9564067320 0x7f95640697d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f95800b1730 tx=0x7f9570015000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:21.498 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.496+0000 7f95767fc700 1 -- 172.21.15.163:0/1790896043 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f957c0526b0 con 0x7f956c004810 2023-10-09T06:38:21.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:21.724+0000 7f95882d4700 1 -- 172.21.15.163:0/1790896043 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f958015aff0 con 0x7f956c004810 2023-10-09T06:38:22.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.696+0000 7f95767fc700 1 -- 172.21.15.163:0/1790896043 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v102) v1 ==== 118+0+0 (secure 0 0 0) 0x7f957c055cd0 con 0x7f956c004810 2023-10-09T06:38:22.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.699+0000 7f95882d4700 1 -- 172.21.15.163:0/1790896043 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9564067320 msgr2=0x7f95640697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:22.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.699+0000 7f95882d4700 1 --2- 172.21.15.163:0/1790896043 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9564067320 0x7f95640697d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f95800b1730 tx=0x7f9570015000 comp rx=0 tx=0).stop 2023-10-09T06:38:22.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.699+0000 7f95882d4700 1 -- 172.21.15.163:0/1790896043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f956c004810 msgr2=0x7f9580143e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:22.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.699+0000 7f95882d4700 1 --2- 172.21.15.163:0/1790896043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f956c004810 0x7f9580143e30 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f957c011f80 tx=0x7f957c00b040 comp rx=0 tx=0).stop 2023-10-09T06:38:22.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.699+0000 7f95882d4700 1 -- 172.21.15.163:0/1790896043 shutdown_connections 2023-10-09T06:38:22.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.699+0000 7f95882d4700 1 --2- 172.21.15.163:0/1790896043 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9564067320 0x7f95640697d0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:22.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.699+0000 7f95882d4700 1 --2- 172.21.15.163:0/1790896043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f956c004810 0x7f9580143e30 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:22.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.699+0000 7f95882d4700 1 --2- 172.21.15.163:0/1790896043 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95800cb6c0 0x7f958006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:22.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.699+0000 7f95882d4700 1 --2- 172.21.15.163:0/1790896043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95800cb250 0x7f958006cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:22.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.699+0000 7f95882d4700 1 -- 172.21.15.163:0/1790896043 >> 172.21.15.163:0/1790896043 conn(0x7f95800ce600 msgr2=0x7f958015b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:22.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.699+0000 7f95882d4700 1 -- 172.21.15.163:0/1790896043 shutdown_connections 2023-10-09T06:38:22.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.699+0000 7f95882d4700 1 -- 172.21.15.163:0/1790896043 wait complete. 2023-10-09T06:38:22.715 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-09T06:38:22.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.837+0000 7fbd78ade700 1 Processor -- start 2023-10-09T06:38:22.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.837+0000 7fbd78ade700 1 -- start start 2023-10-09T06:38:22.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.838+0000 7fbd78ade700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbd74159890 0x7fbd7415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:22.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.838+0000 7fbd78ade700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd74159c60 0x7fbd7415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:22.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.838+0000 7fbd78ade700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd7415a160 con 0x7fbd7415db10 2023-10-09T06:38:22.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.838+0000 7fbd78ade700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd740d13a0 con 0x7fbd74159890 2023-10-09T06:38:22.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.838+0000 7fbd78ade700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7415e140 con 0x7fbd74159c60 2023-10-09T06:38:22.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.838+0000 7fbd7259c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbd74159890 0x7fbd7415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:22.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.838+0000 7fbd72d9d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fbd7415db10 0x7fbd74069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33622/0 (socket says 172.21.15.163:33622) 2023-10-09T06:38:22.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.838+0000 7fbd72d9d700 1 -- 172.21.15.163:0/3478364094 learned_addr learned my addr 172.21.15.163:0/3478364094 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:22.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.838+0000 7fbd7259c700 1 -- 172.21.15.163:0/3478364094 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd74159c60 msgr2=0x7fbd7415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:22.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.838+0000 7fbd7259c700 1 --2- 172.21.15.163:0/3478364094 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd74159c60 0x7fbd7415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:22.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.838+0000 7fbd7259c700 1 -- 172.21.15.163:0/3478364094 >> v1:172.21.15.163:6789/0 conn(0x7fbd7415db10 legacy=0x7fbd74069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:22.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.838+0000 7fbd71d9b700 1 -- 172.21.15.163:0/3478364094 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3126870912 0 0) 0x7fbd7415a160 con 0x7fbd7415db10 2023-10-09T06:38:22.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.838+0000 7fbd7259c700 1 -- 172.21.15.163:0/3478364094 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7415a3d0 con 0x7fbd74159890 2023-10-09T06:38:22.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.839+0000 7fbd7259c700 1 --2- 172.21.15.163:0/3478364094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd74159890 0x7fbd7415abd0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fbd640098e0 tx=0x7fbd640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a030bd2140d448 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:22.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.839+0000 7fbd71d9b700 1 -- 172.21.15.163:0/3478364094 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd64009e80 con 0x7fbd74159890 2023-10-09T06:38:22.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.839+0000 7fbd71d9b700 1 -- 172.21.15.163:0/3478364094 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6400b860 con 0x7fbd74159890 2023-10-09T06:38:22.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.839+0000 7fbd71d9b700 1 -- 172.21.15.163:0/3478364094 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6400bbe0 con 0x7fbd74159890 2023-10-09T06:38:22.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.839+0000 7fbd78ade700 1 -- 172.21.15.163:0/3478364094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd74159890 msgr2=0x7fbd7415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:22.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.839+0000 7fbd78ade700 1 --2- 172.21.15.163:0/3478364094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd74159890 0x7fbd7415abd0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fbd640098e0 tx=0x7fbd640091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:22.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.839+0000 7fbd78ade700 1 -- 172.21.15.163:0/3478364094 shutdown_connections 2023-10-09T06:38:22.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.839+0000 7fbd78ade700 1 --2- 172.21.15.163:0/3478364094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd74159890 0x7fbd7415abd0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:22.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.839+0000 7fbd78ade700 1 --2- 172.21.15.163:0/3478364094 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd74159c60 0x7fbd7415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:22.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.839+0000 7fbd78ade700 1 -- 172.21.15.163:0/3478364094 >> 172.21.15.163:0/3478364094 conn(0x7fbd74068e20 msgr2=0x7fbd740d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:22.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.839+0000 7fbd78ade700 1 -- 172.21.15.163:0/3478364094 shutdown_connections 2023-10-09T06:38:22.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.839+0000 7fbd78ade700 1 -- 172.21.15.163:0/3478364094 wait complete. 2023-10-09T06:38:22.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.841+0000 7fbd78ade700 1 Processor -- start 2023-10-09T06:38:22.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.841+0000 7fbd78ade700 1 -- start start 2023-10-09T06:38:22.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.841+0000 7fbd78ade700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd74159890 0x7fbd740ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:22.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.841+0000 7fbd78ade700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd74159c60 0x7fbd741522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:22.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.841+0000 7fbd78ade700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd7415db10 0x7fbd740b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:22.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.841+0000 7fbd78ade700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd740c99f0 con 0x7fbd74159c60 2023-10-09T06:38:22.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.841+0000 7fbd78ade700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd740b4740 con 0x7fbd74159890 2023-10-09T06:38:22.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.841+0000 7fbd78ade700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd740b3b30 con 0x7fbd7415db10 2023-10-09T06:38:22.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.841+0000 7fbd69d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd74159c60 0x7fbd741522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:22.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.841+0000 7fbd69d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd74159c60 0x7fbd741522a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42124/0 (socket says 172.21.15.163:42124) 2023-10-09T06:38:22.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.841+0000 7fbd69d9b700 1 -- 172.21.15.163:0/912407640 learned_addr learned my addr 172.21.15.163:0/912407640 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:22.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.842+0000 7fbd72d9d700 1 --2- 172.21.15.163:0/912407640 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd7415db10 0x7fbd740b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:22.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.842+0000 7fbd7259c700 1 --2- 172.21.15.163:0/912407640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd74159890 0x7fbd740ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:22.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.842+0000 7fbd69d9b700 1 -- 172.21.15.163:0/912407640 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd7415db10 msgr2=0x7fbd740b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:22.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.842+0000 7fbd69d9b700 1 --2- 172.21.15.163:0/912407640 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd7415db10 0x7fbd740b07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:22.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.842+0000 7fbd69d9b700 1 -- 172.21.15.163:0/912407640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd74159890 msgr2=0x7fbd740ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:22.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.842+0000 7fbd69d9b700 1 --2- 172.21.15.163:0/912407640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd74159890 0x7fbd740ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:22.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.842+0000 7fbd69d9b700 1 -- 172.21.15.163:0/912407640 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd64009420 con 0x7fbd74159c60 2023-10-09T06:38:22.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.842+0000 7fbd69d9b700 1 --2- 172.21.15.163:0/912407640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd74159c60 0x7fbd741522a0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c00b810 tx=0x7fbd5c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:22.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.842+0000 7fbd6affd700 1 -- 172.21.15.163:0/912407640 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c00f5f0 con 0x7fbd74159c60 2023-10-09T06:38:22.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.842+0000 7fbd6affd700 1 -- 172.21.15.163:0/912407640 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5c00f750 con 0x7fbd74159c60 2023-10-09T06:38:22.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.842+0000 7fbd78ade700 1 -- 172.21.15.163:0/912407640 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd740b3e10 con 0x7fbd74159c60 2023-10-09T06:38:22.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.842+0000 7fbd6affd700 1 -- 172.21.15.163:0/912407640 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c00fb10 con 0x7fbd74159c60 2023-10-09T06:38:22.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.842+0000 7fbd78ade700 1 -- 172.21.15.163:0/912407640 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd740b41a0 con 0x7fbd74159c60 2023-10-09T06:38:22.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.843+0000 7fbd78ade700 1 -- 172.21.15.163:0/912407640 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd740b48d0 con 0x7fbd74159c60 2023-10-09T06:38:22.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.843+0000 7fbd6affd700 1 -- 172.21.15.163:0/912407640 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbd5c010480 con 0x7fbd74159c60 2023-10-09T06:38:22.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.844+0000 7fbd6affd700 1 --2- 172.21.15.163:0/912407640 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbd54067320 0x7fbd540697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:22.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.844+0000 7fbd6affd700 1 -- 172.21.15.163:0/912407640 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbd5c084b60 con 0x7fbd74159c60 2023-10-09T06:38:22.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.844+0000 7fbd7259c700 1 --2- 172.21.15.163:0/912407640 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbd54067320 0x7fbd540697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:22.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.844+0000 7fbd7259c700 1 --2- 172.21.15.163:0/912407640 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbd54067320 0x7fbd540697d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbd640091e0 tx=0x7fbd64017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:22.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:22.848+0000 7fbd6affd700 1 -- 172.21.15.163:0/912407640 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd5c051870 con 0x7fbd74159c60 2023-10-09T06:38:23.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.076+0000 7fbd78ade700 1 -- 172.21.15.163:0/912407640 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fbd7404c3c0 con 0x7fbd74159c60 2023-10-09T06:38:23.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.700+0000 7fbd6affd700 1 -- 172.21.15.163:0/912407640 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v103) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbd5c054e90 con 0x7fbd74159c60 2023-10-09T06:38:23.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.702+0000 7fbd78ade700 1 -- 172.21.15.163:0/912407640 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbd54067320 msgr2=0x7fbd540697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:23.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.702+0000 7fbd78ade700 1 --2- 172.21.15.163:0/912407640 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbd54067320 0x7fbd540697d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbd640091e0 tx=0x7fbd64017000 comp rx=0 tx=0).stop 2023-10-09T06:38:23.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.702+0000 7fbd78ade700 1 -- 172.21.15.163:0/912407640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd74159c60 msgr2=0x7fbd741522a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:23.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.702+0000 7fbd78ade700 1 --2- 172.21.15.163:0/912407640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd74159c60 0x7fbd741522a0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c00b810 tx=0x7fbd5c009220 comp rx=0 tx=0).stop 2023-10-09T06:38:23.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.702+0000 7fbd78ade700 1 -- 172.21.15.163:0/912407640 shutdown_connections 2023-10-09T06:38:23.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.702+0000 7fbd78ade700 1 --2- 172.21.15.163:0/912407640 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbd54067320 0x7fbd540697d0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:23.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.702+0000 7fbd78ade700 1 --2- 172.21.15.163:0/912407640 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd7415db10 0x7fbd740b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:23.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.702+0000 7fbd78ade700 1 --2- 172.21.15.163:0/912407640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd74159890 0x7fbd740ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:23.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.702+0000 7fbd78ade700 1 --2- 172.21.15.163:0/912407640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd74159c60 0x7fbd741522a0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:23.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.702+0000 7fbd78ade700 1 -- 172.21.15.163:0/912407640 >> 172.21.15.163:0/912407640 conn(0x7fbd74068e20 msgr2=0x7fbd740b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:23.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.702+0000 7fbd78ade700 1 -- 172.21.15.163:0/912407640 shutdown_connections 2023-10-09T06:38:23.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.702+0000 7fbd78ade700 1 -- 172.21.15.163:0/912407640 wait complete. 2023-10-09T06:38:23.717 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:38:23.834 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.833+0000 7f6ae1fc2700 1 Processor -- start 2023-10-09T06:38:23.835 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.833+0000 7f6ae1fc2700 1 -- start start 2023-10-09T06:38:23.836 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.834+0000 7f6ae1fc2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6adc069560 0x7f6adc0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:23.836 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.834+0000 7f6ae1fc2700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6adc069ee0 con 0x7f6adc159890 2023-10-09T06:38:23.836 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.834+0000 7f6ae1fc2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6adc15a160 con 0x7f6adc159c60 2023-10-09T06:38:23.836 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.834+0000 7f6ae1fc2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc0d13a0 con 0x7f6adc069560 2023-10-09T06:38:23.836 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.834+0000 7f6adb7fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f6adc159890 0x7f6adc15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33624/0 (socket says 172.21.15.163:33624) 2023-10-09T06:38:23.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.834+0000 7f6adb7fe700 1 -- 172.21.15.163:0/2157600588 learned_addr learned my addr 172.21.15.163:0/2157600588 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:23.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.835+0000 7f6adbfff700 1 --2- 172.21.15.163:0/2157600588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6adc069560 0x7f6adc0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:23.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.835+0000 7f6ada7fc700 1 -- 172.21.15.163:0/2157600588 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3460843338 0 0) 0x7f6adc069ee0 con 0x7f6adc159890 2023-10-09T06:38:23.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.835+0000 7f6ada7fc700 1 -- 172.21.15.163:0/2157600588 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ac4003570 con 0x7f6adc159890 2023-10-09T06:38:23.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.835+0000 7f6ada7fc700 1 -- 172.21.15.163:0/2157600588 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4278919757 0 0) 0x7f6adc15a160 con 0x7f6adc159c60 2023-10-09T06:38:23.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.835+0000 7f6ada7fc700 1 -- 172.21.15.163:0/2157600588 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6adc069ee0 con 0x7f6adc159c60 2023-10-09T06:38:23.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.835+0000 7f6adbfff700 1 -- 172.21.15.163:0/2157600588 >> v1:172.21.15.181:6789/0 conn(0x7f6adc159c60 legacy=0x7f6adc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:23.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.835+0000 7f6ada7fc700 1 -- 172.21.15.163:0/2157600588 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3756362641 0 0) 0x7f6ac4003570 con 0x7f6adc159890 2023-10-09T06:38:23.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.835+0000 7f6adbfff700 1 -- 172.21.15.163:0/2157600588 >> v1:172.21.15.163:6789/0 conn(0x7f6adc159890 legacy=0x7f6adc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:23.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.835+0000 7f6adbfff700 1 -- 172.21.15.163:0/2157600588 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6adc15e140 con 0x7f6adc069560 2023-10-09T06:38:23.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.835+0000 7f6adbfff700 1 --2- 172.21.15.163:0/2157600588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6adc069560 0x7f6adc0699a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009910 tx=0x7f6ad40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7572b3f1a568676 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:23.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.835+0000 7f6ada7fc700 1 -- 172.21.15.163:0/2157600588 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc0017f0 con 0x7f6adc069560 2023-10-09T06:38:23.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.835+0000 7f6ada7fc700 1 -- 172.21.15.163:0/2157600588 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad4011420 con 0x7f6adc069560 2023-10-09T06:38:23.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.835+0000 7f6ada7fc700 1 -- 172.21.15.163:0/2157600588 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad40117a0 con 0x7f6adc069560 2023-10-09T06:38:23.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.836+0000 7f6ae1fc2700 1 -- 172.21.15.163:0/2157600588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6adc069560 msgr2=0x7f6adc0699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:23.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.836+0000 7f6ae1fc2700 1 --2- 172.21.15.163:0/2157600588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6adc069560 0x7f6adc0699a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009910 tx=0x7f6ad40091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:23.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.836+0000 7f6ae1fc2700 1 -- 172.21.15.163:0/2157600588 shutdown_connections 2023-10-09T06:38:23.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.836+0000 7f6ae1fc2700 1 --2- 172.21.15.163:0/2157600588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6adc069560 0x7f6adc0699a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:23.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.836+0000 7f6ae1fc2700 1 -- 172.21.15.163:0/2157600588 >> 172.21.15.163:0/2157600588 conn(0x7f6adc068e20 msgr2=0x7f6adc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:23.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.836+0000 7f6ae1fc2700 1 -- 172.21.15.163:0/2157600588 shutdown_connections 2023-10-09T06:38:23.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.836+0000 7f6ae1fc2700 1 -- 172.21.15.163:0/2157600588 wait complete. 2023-10-09T06:38:23.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.837+0000 7f6ae1fc2700 1 Processor -- start 2023-10-09T06:38:23.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.837+0000 7f6ae1fc2700 1 -- start start 2023-10-09T06:38:23.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.838+0000 7f6ae1fc2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6adc069560 0x7f6adc0b5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:23.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.838+0000 7f6ae1fc2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6adc159890 0x7f6adc0c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:23.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.838+0000 7f6ae1fc2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6adc159c60 0x7f6adc154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:23.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.838+0000 7f6ae1fc2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc0b4db0 con 0x7f6adc069560 2023-10-09T06:38:23.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.838+0000 7f6ae1fc2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc0c78a0 con 0x7f6adc159890 2023-10-09T06:38:23.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.838+0000 7f6ae1fc2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc0c7b80 con 0x7f6adc159c60 2023-10-09T06:38:23.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.838+0000 7f6adb7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6adc069560 0x7f6adc0b5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:23.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.838+0000 7f6adb7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6adc069560 0x7f6adc0b5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42136/0 (socket says 172.21.15.163:42136) 2023-10-09T06:38:23.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.838+0000 7f6adb7fe700 1 -- 172.21.15.163:0/2522640209 learned_addr learned my addr 172.21.15.163:0/2522640209 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:23.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.838+0000 7f6adaffd700 1 --2- 172.21.15.163:0/2522640209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6adc159890 0x7f6adc0c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:23.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.838+0000 7f6adbfff700 1 --2- 172.21.15.163:0/2522640209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6adc159c60 0x7f6adc154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:23.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.838+0000 7f6adb7fe700 1 -- 172.21.15.163:0/2522640209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6adc159c60 msgr2=0x7f6adc154170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:23.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.838+0000 7f6adb7fe700 1 --2- 172.21.15.163:0/2522640209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6adc159c60 0x7f6adc154170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:23.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.838+0000 7f6adb7fe700 1 -- 172.21.15.163:0/2522640209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6adc159890 msgr2=0x7f6adc0c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:23.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.838+0000 7f6adb7fe700 1 --2- 172.21.15.163:0/2522640209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6adc159890 0x7f6adc0c97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:23.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.838+0000 7f6adb7fe700 1 -- 172.21.15.163:0/2522640209 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad4009420 con 0x7f6adc069560 2023-10-09T06:38:23.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.838+0000 7f6adbfff700 1 --2- 172.21.15.163:0/2522640209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6adc159c60 0x7f6adc154170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:23.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.838+0000 7f6adb7fe700 1 --2- 172.21.15.163:0/2522640209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6adc069560 0x7f6adc0b5ec0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f6acc00cc70 tx=0x7f6acc007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:23.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.839+0000 7f6ac37fe700 1 -- 172.21.15.163:0/2522640209 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc00ce20 con 0x7f6adc069560 2023-10-09T06:38:23.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.839+0000 7f6ac37fe700 1 -- 172.21.15.163:0/2522640209 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6acc004dc0 con 0x7f6adc069560 2023-10-09T06:38:23.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.839+0000 7f6ae1fc2700 1 -- 172.21.15.163:0/2522640209 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6adc0b6400 con 0x7f6adc069560 2023-10-09T06:38:23.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.839+0000 7f6ac37fe700 1 -- 172.21.15.163:0/2522640209 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc00abb0 con 0x7f6adc069560 2023-10-09T06:38:23.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.839+0000 7f6ae1fc2700 1 -- 172.21.15.163:0/2522640209 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6adc0b5280 con 0x7f6adc069560 2023-10-09T06:38:23.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.840+0000 7f6ae1fc2700 1 -- 172.21.15.163:0/2522640209 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6adc04c3c0 con 0x7f6adc069560 2023-10-09T06:38:23.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.840+0000 7f6ac37fe700 1 -- 172.21.15.163:0/2522640209 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6acc007150 con 0x7f6adc069560 2023-10-09T06:38:23.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.840+0000 7f6ac37fe700 1 --2- 172.21.15.163:0/2522640209 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6ac8067320 0x7f6ac80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:23.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.840+0000 7f6ac37fe700 1 -- 172.21.15.163:0/2522640209 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6acc084580 con 0x7f6adc069560 2023-10-09T06:38:23.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.841+0000 7f6adaffd700 1 --2- 172.21.15.163:0/2522640209 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6ac8067320 0x7f6ac80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:23.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.841+0000 7f6adaffd700 1 --2- 172.21.15.163:0/2522640209 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6ac8067320 0x7f6ac80697d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6adc0b5be0 tx=0x7f6ad0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:23.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:23.845+0000 7f6ac37fe700 1 -- 172.21.15.163:0/2522640209 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6acc051210 con 0x7f6adc069560 2023-10-09T06:38:24.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.069+0000 7f6ae1fc2700 1 -- 172.21.15.163:0/2522640209 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6adc069930 con 0x7f6adc069560 2023-10-09T06:38:24.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.070+0000 7f6ac37fe700 1 -- 172.21.15.163:0/2522640209 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4615 (secure 0 0 0) 0x7f6acc058e60 con 0x7f6adc069560 2023-10-09T06:38:24.071 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:24.071 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":103,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6371,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3797578839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3797578839},{"type":"v1","addr":"172.21.15.163:6834","nonce":3797578839}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":6383,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2371721802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2371721802},{"type":"v1","addr":"172.21.15.181:6837","nonce":2371721802}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":7228,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2068214884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2068214884},{"type":"v1","addr":"172.21.15.181:6836","nonce":2068214884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:38:20.060175+0000","modified":"2023-10-09T06:38:23.693400+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7875},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7875":{"gid":7875,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/3794912550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3794912550},{"type":"v1","addr":"172.21.15.163:6835","nonce":3794912550}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-09T06:38:24.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.072+0000 7f6ae1fc2700 1 -- 172.21.15.163:0/2522640209 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6ac8067320 msgr2=0x7f6ac80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:24.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.072+0000 7f6ae1fc2700 1 --2- 172.21.15.163:0/2522640209 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6ac8067320 0x7f6ac80697d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6adc0b5be0 tx=0x7f6ad0009040 comp rx=0 tx=0).stop 2023-10-09T06:38:24.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.072+0000 7f6ae1fc2700 1 -- 172.21.15.163:0/2522640209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6adc069560 msgr2=0x7f6adc0b5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:24.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.072+0000 7f6ae1fc2700 1 --2- 172.21.15.163:0/2522640209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6adc069560 0x7f6adc0b5ec0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f6acc00cc70 tx=0x7f6acc007d20 comp rx=0 tx=0).stop 2023-10-09T06:38:24.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.072+0000 7f6ae1fc2700 1 -- 172.21.15.163:0/2522640209 shutdown_connections 2023-10-09T06:38:24.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.072+0000 7f6ae1fc2700 1 --2- 172.21.15.163:0/2522640209 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6ac8067320 0x7f6ac80697d0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:24.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.072+0000 7f6ae1fc2700 1 --2- 172.21.15.163:0/2522640209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6adc069560 0x7f6adc0b5ec0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:24.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.072+0000 7f6ae1fc2700 1 --2- 172.21.15.163:0/2522640209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6adc159c60 0x7f6adc154170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:24.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.072+0000 7f6ae1fc2700 1 --2- 172.21.15.163:0/2522640209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6adc159890 0x7f6adc0c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:24.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.072+0000 7f6ae1fc2700 1 -- 172.21.15.163:0/2522640209 >> 172.21.15.163:0/2522640209 conn(0x7f6adc068e20 msgr2=0x7f6adc1553b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:24.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.072+0000 7f6ae1fc2700 1 -- 172.21.15.163:0/2522640209 shutdown_connections 2023-10-09T06:38:24.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.072+0000 7f6ae1fc2700 1 -- 172.21.15.163:0/2522640209 wait complete. 2023-10-09T06:38:24.074 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 103 2023-10-09T06:38:24.085 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:38:24.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.198+0000 7f1adaccb700 1 Processor -- start 2023-10-09T06:38:24.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.198+0000 7f1adaccb700 1 -- start start 2023-10-09T06:38:24.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.199+0000 7f1adaccb700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1ad4159420 0x7f1ad40b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:24.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.199+0000 7f1adaccb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1ad41597f0 0x7f1ad40b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:24.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.199+0000 7f1adaccb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ad415d230 0x7f1ad415d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:24.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.199+0000 7f1adaccb700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad415dbb0 con 0x7f1ad4159420 2023-10-09T06:38:24.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.199+0000 7f1adaccb700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad40b11d0 con 0x7f1ad41597f0 2023-10-09T06:38:24.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.199+0000 7f1adaccb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad40b14b0 con 0x7f1ad415d230 2023-10-09T06:38:24.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.199+0000 7f1ad9cc9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1ad4159420 0x7f1ad40b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:24.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.199+0000 7f1ad9cc9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1ad4159420 0x7f1ad40b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42138/0 (socket says 172.21.15.163:42138) 2023-10-09T06:38:24.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.199+0000 7f1ad9cc9700 1 -- 172.21.15.163:0/4054378473 learned_addr learned my addr 172.21.15.163:0/4054378473 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:24.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.199+0000 7f1ad94c8700 1 --2- 172.21.15.163:0/4054378473 >> v2:172.21.15.181:3300/0 conn(0x7f1ad41597f0 0x7f1ad40b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:24.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.199+0000 7f1ad9cc9700 1 -- 172.21.15.163:0/4054378473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ad415d230 msgr2=0x7f1ad415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:24.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.199+0000 7f1ada4ca700 1 --2- 172.21.15.163:0/4054378473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ad415d230 0x7f1ad415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:24.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.200+0000 7f1ad9cc9700 1 --2- 172.21.15.163:0/4054378473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ad415d230 0x7f1ad415d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:24.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.200+0000 7f1ad9cc9700 1 -- 172.21.15.163:0/4054378473 >> v2:172.21.15.181:3300/0 conn(0x7f1ad41597f0 msgr2=0x7f1ad40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:24.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.200+0000 7f1ad9cc9700 1 --2- 172.21.15.163:0/4054378473 >> v2:172.21.15.181:3300/0 conn(0x7f1ad41597f0 0x7f1ad40b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:24.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.200+0000 7f1ad9cc9700 1 -- 172.21.15.163:0/4054378473 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad40b1790 con 0x7f1ad4159420 2023-10-09T06:38:24.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.200+0000 7f1ad94c8700 1 --2- 172.21.15.163:0/4054378473 >> v2:172.21.15.181:3300/0 conn(0x7f1ad41597f0 0x7f1ad40b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:24.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.200+0000 7f1ad9cc9700 1 --2- 172.21.15.163:0/4054378473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ad4159420 0x7f1ad40b5f40 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4089750 tx=0x7f1ad0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8558cc5b7c403946 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:24.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.200+0000 7f1ad8cc7700 1 -- 172.21.15.163:0/4054378473 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad000b860 con 0x7f1ad4159420 2023-10-09T06:38:24.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.200+0000 7f1ad8cc7700 1 -- 172.21.15.163:0/4054378473 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad000b9c0 con 0x7f1ad4159420 2023-10-09T06:38:24.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.200+0000 7f1ad8cc7700 1 -- 172.21.15.163:0/4054378473 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad000bd60 con 0x7f1ad4159420 2023-10-09T06:38:24.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.200+0000 7f1adaccb700 1 -- 172.21.15.163:0/4054378473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ad4159420 msgr2=0x7f1ad40b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:24.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.200+0000 7f1adaccb700 1 --2- 172.21.15.163:0/4054378473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ad4159420 0x7f1ad40b5f40 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4089750 tx=0x7f1ad0009250 comp rx=0 tx=0).stop 2023-10-09T06:38:24.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.200+0000 7f1adaccb700 1 -- 172.21.15.163:0/4054378473 shutdown_connections 2023-10-09T06:38:24.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.200+0000 7f1adaccb700 1 --2- 172.21.15.163:0/4054378473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ad415d230 0x7f1ad415d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:24.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.200+0000 7f1adaccb700 1 --2- 172.21.15.163:0/4054378473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ad4159420 0x7f1ad40b5f40 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:24.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.200+0000 7f1adaccb700 1 --2- 172.21.15.163:0/4054378473 >> v2:172.21.15.181:3300/0 conn(0x7f1ad41597f0 0x7f1ad40b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:24.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.200+0000 7f1adaccb700 1 -- 172.21.15.163:0/4054378473 >> 172.21.15.163:0/4054378473 conn(0x7f1ad40d0660 msgr2=0x7f1ad40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:24.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.200+0000 7f1adaccb700 1 -- 172.21.15.163:0/4054378473 shutdown_connections 2023-10-09T06:38:24.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.200+0000 7f1adaccb700 1 -- 172.21.15.163:0/4054378473 wait complete. 2023-10-09T06:38:24.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.202+0000 7f1adaccb700 1 Processor -- start 2023-10-09T06:38:24.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.202+0000 7f1adaccb700 1 -- start start 2023-10-09T06:38:24.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.202+0000 7f1adaccb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ad4159420 0x7f1ad4169f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:24.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.202+0000 7f1adaccb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ad41597f0 0x7f1ad4157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:24.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.202+0000 7f1adaccb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ad415d230 0x7f1ad40c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:24.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.202+0000 7f1adaccb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad416a450 con 0x7f1ad4159420 2023-10-09T06:38:24.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.202+0000 7f1adaccb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad41579d0 con 0x7f1ad41597f0 2023-10-09T06:38:24.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.202+0000 7f1adaccb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad40c7100 con 0x7f1ad415d230 2023-10-09T06:38:24.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.202+0000 7f1ad9cc9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ad4159420 0x7f1ad4169f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:24.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.202+0000 7f1ad9cc9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ad4159420 0x7f1ad4169f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42142/0 (socket says 172.21.15.163:42142) 2023-10-09T06:38:24.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.202+0000 7f1ad9cc9700 1 -- 172.21.15.163:0/2940657891 learned_addr learned my addr 172.21.15.163:0/2940657891 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:24.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.202+0000 7f1ada4ca700 1 --2- 172.21.15.163:0/2940657891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ad415d230 0x7f1ad40c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:24.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.203+0000 7f1ad94c8700 1 --2- 172.21.15.163:0/2940657891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ad41597f0 0x7f1ad4157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:24.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.203+0000 7f1ad9cc9700 1 -- 172.21.15.163:0/2940657891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ad415d230 msgr2=0x7f1ad40c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:24.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.203+0000 7f1ad9cc9700 1 --2- 172.21.15.163:0/2940657891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ad415d230 0x7f1ad40c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:24.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.203+0000 7f1ad9cc9700 1 -- 172.21.15.163:0/2940657891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ad41597f0 msgr2=0x7f1ad4157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:24.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.203+0000 7f1ad9cc9700 1 --2- 172.21.15.163:0/2940657891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ad41597f0 0x7f1ad4157490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:24.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.203+0000 7f1ad9cc9700 1 -- 172.21.15.163:0/2940657891 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad00094f0 con 0x7f1ad4159420 2023-10-09T06:38:24.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.203+0000 7f1ada4ca700 1 --2- 172.21.15.163:0/2940657891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ad415d230 0x7f1ad40c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:24.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.203+0000 7f1ad94c8700 1 --2- 172.21.15.163:0/2940657891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ad41597f0 0x7f1ad4157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:24.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.203+0000 7f1ad9cc9700 1 --2- 172.21.15.163:0/2940657891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ad4159420 0x7f1ad4169f10 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4089340 tx=0x7f1ad0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:24.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.203+0000 7f1ac9ffb700 1 -- 172.21.15.163:0/2940657891 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0024440 con 0x7f1ad4159420 2023-10-09T06:38:24.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.203+0000 7f1ac9ffb700 1 -- 172.21.15.163:0/2940657891 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad00245a0 con 0x7f1ad4159420 2023-10-09T06:38:24.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.203+0000 7f1adaccb700 1 -- 172.21.15.163:0/2940657891 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad40c73e0 con 0x7f1ad4159420 2023-10-09T06:38:24.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.203+0000 7f1ac9ffb700 1 -- 172.21.15.163:0/2940657891 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0024960 con 0x7f1ad4159420 2023-10-09T06:38:24.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.203+0000 7f1adaccb700 1 -- 172.21.15.163:0/2940657891 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad40c79a0 con 0x7f1ad4159420 2023-10-09T06:38:24.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.204+0000 7f1adaccb700 1 -- 172.21.15.163:0/2940657891 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad404c3c0 con 0x7f1ad4159420 2023-10-09T06:38:24.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.204+0000 7f1ac9ffb700 1 -- 172.21.15.163:0/2940657891 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1ad001a420 con 0x7f1ad4159420 2023-10-09T06:38:24.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.205+0000 7f1ac9ffb700 1 --2- 172.21.15.163:0/2940657891 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1ab8067320 0x7f1ab80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:24.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.205+0000 7f1ac9ffb700 1 -- 172.21.15.163:0/2940657891 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1ad0086a90 con 0x7f1ad4159420 2023-10-09T06:38:24.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.205+0000 7f1ad94c8700 1 --2- 172.21.15.163:0/2940657891 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1ab8067320 0x7f1ab80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:24.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.205+0000 7f1ad94c8700 1 --2- 172.21.15.163:0/2940657891 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1ab8067320 0x7f1ab80697d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1ad415d200 tx=0x7f1ac4006f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:24.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.209+0000 7f1ac9ffb700 1 -- 172.21.15.163:0/2940657891 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ad0053720 con 0x7f1ad4159420 2023-10-09T06:38:24.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.389+0000 7f1adaccb700 1 -- 172.21.15.163:0/2940657891 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ad40b1a70 con 0x7f1ad4159420 2023-10-09T06:38:24.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.390+0000 7f1ac9ffb700 1 -- 172.21.15.163:0/2940657891 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) v1 ==== 74+0+15412 (secure 0 0 0) 0x7f1ad0056d40 con 0x7f1ad4159420 2023-10-09T06:38:24.392 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:24.392 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":152,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:38:21.683060+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:38:18.381850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-09T06:38:19.022637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:38:24.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.393+0000 7f1adaccb700 1 -- 172.21.15.163:0/2940657891 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1ab8067320 msgr2=0x7f1ab80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:24.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.393+0000 7f1adaccb700 1 --2- 172.21.15.163:0/2940657891 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1ab8067320 0x7f1ab80697d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1ad415d200 tx=0x7f1ac4006f90 comp rx=0 tx=0).stop 2023-10-09T06:38:24.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.393+0000 7f1adaccb700 1 -- 172.21.15.163:0/2940657891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ad4159420 msgr2=0x7f1ad4169f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:24.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.393+0000 7f1adaccb700 1 --2- 172.21.15.163:0/2940657891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ad4159420 0x7f1ad4169f10 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4089340 tx=0x7f1ad0017000 comp rx=0 tx=0).stop 2023-10-09T06:38:24.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.393+0000 7f1adaccb700 1 -- 172.21.15.163:0/2940657891 shutdown_connections 2023-10-09T06:38:24.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.393+0000 7f1adaccb700 1 --2- 172.21.15.163:0/2940657891 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1ab8067320 0x7f1ab80697d0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:24.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.393+0000 7f1adaccb700 1 --2- 172.21.15.163:0/2940657891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ad4159420 0x7f1ad4169f10 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:24.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.393+0000 7f1adaccb700 1 --2- 172.21.15.163:0/2940657891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ad415d230 0x7f1ad40c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:24.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.393+0000 7f1adaccb700 1 --2- 172.21.15.163:0/2940657891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ad41597f0 0x7f1ad4157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:24.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.393+0000 7f1adaccb700 1 -- 172.21.15.163:0/2940657891 >> 172.21.15.163:0/2940657891 conn(0x7f1ad40d0660 msgr2=0x7f1ad40ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:24.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.393+0000 7f1adaccb700 1 -- 172.21.15.163:0/2940657891 shutdown_connections 2023-10-09T06:38:24.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.393+0000 7f1adaccb700 1 -- 172.21.15.163:0/2940657891 wait complete. 2023-10-09T06:38:24.406 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-09T06:38:24.406 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:38:24.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.519+0000 7f50ad776700 1 Processor -- start 2023-10-09T06:38:24.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.520+0000 7f50ad776700 1 -- start start 2023-10-09T06:38:24.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.520+0000 7f50ad776700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f50a8159890 0x7f50a815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:24.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.520+0000 7f50ad776700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a815a160 con 0x7f50a8069560 2023-10-09T06:38:24.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.520+0000 7f50ad776700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a80d13a0 con 0x7f50a8159c60 2023-10-09T06:38:24.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.520+0000 7f50ad776700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50a815e140 con 0x7f50a8159890 2023-10-09T06:38:24.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.520+0000 7f50a6ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f50a8159890 0x7f50a815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:24.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.521+0000 7f50a77fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f50a8069560 0x7f50a80ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33632/0 (socket says 172.21.15.163:33632) 2023-10-09T06:38:24.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.521+0000 7f50a77fe700 1 -- 172.21.15.163:0/4134001973 learned_addr learned my addr 172.21.15.163:0/4134001973 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:24.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.521+0000 7f50a6ffd700 1 -- 172.21.15.163:0/4134001973 >> v1:172.21.15.181:6789/0 conn(0x7f50a8159c60 legacy=0x7f50a815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:24.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.521+0000 7f50a6ffd700 1 -- 172.21.15.163:0/4134001973 >> v1:172.21.15.163:6789/0 conn(0x7f50a8069560 legacy=0x7f50a80ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:24.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.521+0000 7f50a67fc700 1 -- 172.21.15.163:0/4134001973 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2742660399 0 0) 0x7f50a815a160 con 0x7f50a8069560 2023-10-09T06:38:24.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.521+0000 7f50a6ffd700 1 -- 172.21.15.163:0/4134001973 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a815a3d0 con 0x7f50a8159890 2023-10-09T06:38:24.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.521+0000 7f50a6ffd700 1 --2- 172.21.15.163:0/4134001973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50a8159890 0x7f50a815abd0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f50980098e0 tx=0x7f50980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4255d5511ac98f server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:24.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.521+0000 7f50a67fc700 1 -- 172.21.15.163:0/4134001973 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5098009e80 con 0x7f50a8159890 2023-10-09T06:38:24.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.521+0000 7f50a67fc700 1 -- 172.21.15.163:0/4134001973 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509800b820 con 0x7f50a8159890 2023-10-09T06:38:24.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.521+0000 7f50a67fc700 1 -- 172.21.15.163:0/4134001973 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509800bba0 con 0x7f50a8159890 2023-10-09T06:38:24.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.522+0000 7f50ad776700 1 -- 172.21.15.163:0/4134001973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50a8159890 msgr2=0x7f50a815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:24.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.522+0000 7f50ad776700 1 --2- 172.21.15.163:0/4134001973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50a8159890 0x7f50a815abd0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f50980098e0 tx=0x7f50980091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:24.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.522+0000 7f50ad776700 1 -- 172.21.15.163:0/4134001973 shutdown_connections 2023-10-09T06:38:24.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.522+0000 7f50ad776700 1 --2- 172.21.15.163:0/4134001973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50a8159890 0x7f50a815abd0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:24.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.522+0000 7f50ad776700 1 -- 172.21.15.163:0/4134001973 >> 172.21.15.163:0/4134001973 conn(0x7f50a8068e20 msgr2=0x7f50a80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:24.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.522+0000 7f50ad776700 1 -- 172.21.15.163:0/4134001973 shutdown_connections 2023-10-09T06:38:24.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.522+0000 7f50ad776700 1 -- 172.21.15.163:0/4134001973 wait complete. 2023-10-09T06:38:24.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.523+0000 7f50ad776700 1 Processor -- start 2023-10-09T06:38:24.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.523+0000 7f50ad776700 1 -- start start 2023-10-09T06:38:24.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.523+0000 7f50ad776700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50a8069560 0x7f50a815ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:24.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.524+0000 7f50ad776700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50a8159890 0x7f50a80ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:24.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.524+0000 7f50ad776700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50a8159c60 0x7f50a816dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:24.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.524+0000 7f50ad776700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a80ca750 con 0x7f50a8069560 2023-10-09T06:38:24.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.524+0000 7f50ad776700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a816e2b0 con 0x7f50a8159890 2023-10-09T06:38:24.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.524+0000 7f50ad776700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a815c030 con 0x7f50a8159c60 2023-10-09T06:38:24.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.524+0000 7f50a6ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50a8069560 0x7f50a815ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:24.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.524+0000 7f50a6ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50a8069560 0x7f50a815ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42154/0 (socket says 172.21.15.163:42154) 2023-10-09T06:38:24.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.524+0000 7f50a6ffd700 1 -- 172.21.15.163:0/397485354 learned_addr learned my addr 172.21.15.163:0/397485354 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:24.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.524+0000 7f509e7fc700 1 --2- 172.21.15.163:0/397485354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50a8159890 0x7f50a80ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:24.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.524+0000 7f50a77fe700 1 --2- 172.21.15.163:0/397485354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50a8159c60 0x7f50a816dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:24.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.524+0000 7f50a6ffd700 1 -- 172.21.15.163:0/397485354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50a8159c60 msgr2=0x7f50a816dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:24.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.524+0000 7f50a6ffd700 1 --2- 172.21.15.163:0/397485354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50a8159c60 0x7f50a816dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:24.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.524+0000 7f50a6ffd700 1 -- 172.21.15.163:0/397485354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50a8159890 msgr2=0x7f50a80ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:24.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.524+0000 7f50a6ffd700 1 --2- 172.21.15.163:0/397485354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50a8159890 0x7f50a80ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:24.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.524+0000 7f50a6ffd700 1 -- 172.21.15.163:0/397485354 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5098009420 con 0x7f50a8069560 2023-10-09T06:38:24.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.524+0000 7f509e7fc700 1 --2- 172.21.15.163:0/397485354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50a8159890 0x7f50a80ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:24.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.524+0000 7f50a6ffd700 1 --2- 172.21.15.163:0/397485354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50a8069560 0x7f50a815ba50 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f50980030c0 tx=0x7f5098016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:24.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.524+0000 7f509f7fe700 1 -- 172.21.15.163:0/397485354 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509800bae0 con 0x7f50a8069560 2023-10-09T06:38:24.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.524+0000 7f509f7fe700 1 -- 172.21.15.163:0/397485354 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5098020070 con 0x7f50a8069560 2023-10-09T06:38:24.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.524+0000 7f50ad776700 1 -- 172.21.15.163:0/397485354 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a815c310 con 0x7f50a8069560 2023-10-09T06:38:24.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.524+0000 7f509f7fe700 1 -- 172.21.15.163:0/397485354 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509801b6a0 con 0x7f50a8069560 2023-10-09T06:38:24.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.524+0000 7f50ad776700 1 -- 172.21.15.163:0/397485354 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a815c8d0 con 0x7f50a8069560 2023-10-09T06:38:24.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.525+0000 7f50ad776700 1 -- 172.21.15.163:0/397485354 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a804c3c0 con 0x7f50a8069560 2023-10-09T06:38:24.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.525+0000 7f509f7fe700 1 -- 172.21.15.163:0/397485354 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5098022030 con 0x7f50a8069560 2023-10-09T06:38:24.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.526+0000 7f509f7fe700 1 --2- 172.21.15.163:0/397485354 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5088067320 0x7f50880697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:24.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.526+0000 7f509f7fe700 1 -- 172.21.15.163:0/397485354 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5098086bd0 con 0x7f50a8069560 2023-10-09T06:38:24.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.526+0000 7f509e7fc700 1 --2- 172.21.15.163:0/397485354 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5088067320 0x7f50880697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:24.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.526+0000 7f509e7fc700 1 --2- 172.21.15.163:0/397485354 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5088067320 0x7f50880697d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5090007fd0 tx=0x7f5090009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:24.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.530+0000 7f509f7fe700 1 -- 172.21.15.163:0/397485354 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5098053970 con 0x7f50a8069560 2023-10-09T06:38:24.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.710+0000 7f50ad776700 1 -- 172.21.15.163:0/397485354 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f50a804c3c0 con 0x7f5088067320 2023-10-09T06:38:24.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.719+0000 7f509f7fe700 1 -- 172.21.15.163:0/397485354 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287181 (secure 0 0 0) 0x7f50a804c3c0 con 0x7f5088067320 2023-10-09T06:38:24.723 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:24.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.722+0000 7f50ad776700 1 -- 172.21.15.163:0/397485354 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5088067320 msgr2=0x7f50880697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:24.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.722+0000 7f50ad776700 1 --2- 172.21.15.163:0/397485354 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5088067320 0x7f50880697d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5090007fd0 tx=0x7f5090009040 comp rx=0 tx=0).stop 2023-10-09T06:38:24.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.722+0000 7f50ad776700 1 -- 172.21.15.163:0/397485354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50a8069560 msgr2=0x7f50a815ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:24.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.722+0000 7f50ad776700 1 --2- 172.21.15.163:0/397485354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50a8069560 0x7f50a815ba50 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f50980030c0 tx=0x7f5098016000 comp rx=0 tx=0).stop 2023-10-09T06:38:24.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.722+0000 7f50ad776700 1 -- 172.21.15.163:0/397485354 shutdown_connections 2023-10-09T06:38:24.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.722+0000 7f50ad776700 1 --2- 172.21.15.163:0/397485354 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5088067320 0x7f50880697d0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:24.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.722+0000 7f50ad776700 1 --2- 172.21.15.163:0/397485354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50a8069560 0x7f50a815ba50 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:24.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.722+0000 7f50ad776700 1 --2- 172.21.15.163:0/397485354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50a8159c60 0x7f50a816dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:24.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.722+0000 7f50ad776700 1 --2- 172.21.15.163:0/397485354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50a8159890 0x7f50a80ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:24.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.722+0000 7f50ad776700 1 -- 172.21.15.163:0/397485354 >> 172.21.15.163:0/397485354 conn(0x7f50a8068e20 msgr2=0x7f50a80b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:24.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.722+0000 7f50ad776700 1 -- 172.21.15.163:0/397485354 shutdown_connections 2023-10-09T06:38:24.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.722+0000 7f50ad776700 1 -- 172.21.15.163:0/397485354 wait complete. 2023-10-09T06:38:24.727 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:38:24.735 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2023-10-09T06:38:24.410910+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2427056,"kb_used_data":15984,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752547664,"statfs":{"total":773094113280,"available":770608807936,"internally_reserved":0,"allocated":16367616,"data_stored":7948834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.742617"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.687243+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:15:57.542772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.686956+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:10:38.081403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.681576+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:11:37.888420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.685055+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:58:03.322193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678873+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:41:30.382056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678481+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:36:50.516609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682433+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:46:35.635953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.676958+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:58:03.322193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.688589+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:20:02.974752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678660+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:46:35.635953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.680844+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:25:57.096853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678704+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:48:51.565773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.681087+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:25:53.616204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.683792+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:34:55.917445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678799+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:06:23.000411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677797+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:06:23.000411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682655+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:44:35.145850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678258+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:44:51.812594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.679357+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:44:35.145850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.676535+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:16:26.649802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.683129+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:38:53.437170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678996+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:19:50.331458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.681279+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:21:46.727517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682901+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:26:55.538336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.687816+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:08:34.030754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677830+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:01:29.075717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.679264+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:29:07.591719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.681628+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:49:58.537930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.680535+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:46:34.855757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.687098+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:46:34.855757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.676565+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:58:03.322193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677453+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:08:26.202100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.688987+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:42:48.250267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.691611+0000","last_change":"2023-10-09T06:38:19.691611+0000","last_active":"2023-10-09T06:38:19.691611+0000","last_peered":"2023-10-09T06:38:19.691611+0000","last_clean":"2023-10-09T06:38:19.691611+0000","last_became_active":"2023-10-09T06:38:19.691367+0000","last_became_peered":"2023-10-09T06:38:19.691367+0000","last_unstale":"2023-10-09T06:38:19.691611+0000","last_undegraded":"2023-10-09T06:38:19.691611+0000","last_fullsized":"2023-10-09T06:38:19.691611+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:13:34.033630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.679700+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:15:52.880306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.694250+0000","last_change":"2023-10-09T06:38:19.694250+0000","last_active":"2023-10-09T06:38:19.694250+0000","last_peered":"2023-10-09T06:38:19.694250+0000","last_clean":"2023-10-09T06:38:19.694250+0000","last_became_active":"2023-10-09T06:38:19.693478+0000","last_became_peered":"2023-10-09T06:38:19.693478+0000","last_unstale":"2023-10-09T06:38:19.694250+0000","last_undegraded":"2023-10-09T06:38:19.694250+0000","last_fullsized":"2023-10-09T06:38:19.694250+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:50:27.283148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677343+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:52:50.901568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.688925+0000","last_change":"2023-10-09T06:38:19.688925+0000","last_active":"2023-10-09T06:38:19.688925+0000","last_peered":"2023-10-09T06:38:19.688925+0000","last_clean":"2023-10-09T06:38:19.688925+0000","last_became_active":"2023-10-09T06:38:19.688479+0000","last_became_peered":"2023-10-09T06:38:19.688479+0000","last_unstale":"2023-10-09T06:38:19.688925+0000","last_undegraded":"2023-10-09T06:38:19.688925+0000","last_fullsized":"2023-10-09T06:38:19.688925+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:32:15.314337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677169+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:36:50.516609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.692752+0000","last_change":"2023-10-09T06:38:19.692752+0000","last_active":"2023-10-09T06:38:19.692752+0000","last_peered":"2023-10-09T06:38:19.692752+0000","last_clean":"2023-10-09T06:38:19.692752+0000","last_became_active":"2023-10-09T06:38:19.692249+0000","last_became_peered":"2023-10-09T06:38:19.692249+0000","last_unstale":"2023-10-09T06:38:19.692752+0000","last_undegraded":"2023-10-09T06:38:19.692752+0000","last_fullsized":"2023-10-09T06:38:19.692752+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:52:11.581942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682687+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:40:22.810227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.690895+0000","last_change":"2023-10-09T06:38:19.690895+0000","last_active":"2023-10-09T06:38:19.690895+0000","last_peered":"2023-10-09T06:38:19.690895+0000","last_clean":"2023-10-09T06:38:19.690895+0000","last_became_active":"2023-10-09T06:38:19.690633+0000","last_became_peered":"2023-10-09T06:38:19.690633+0000","last_unstale":"2023-10-09T06:38:19.690895+0000","last_undegraded":"2023-10-09T06:38:19.690895+0000","last_fullsized":"2023-10-09T06:38:19.690895+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:26:20.360953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.676460+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:08:26.202100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.687201+0000","last_change":"2023-10-09T06:38:19.687201+0000","last_active":"2023-10-09T06:38:19.687201+0000","last_peered":"2023-10-09T06:38:19.687201+0000","last_clean":"2023-10-09T06:38:19.687201+0000","last_became_active":"2023-10-09T06:38:19.686979+0000","last_became_peered":"2023-10-09T06:38:19.686979+0000","last_unstale":"2023-10-09T06:38:19.687201+0000","last_undegraded":"2023-10-09T06:38:19.687201+0000","last_fullsized":"2023-10-09T06:38:19.687201+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:04:37.633413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682248+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:22:26.955724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.690010+0000","last_change":"2023-10-09T06:38:19.690010+0000","last_active":"2023-10-09T06:38:19.690010+0000","last_peered":"2023-10-09T06:38:19.690010+0000","last_clean":"2023-10-09T06:38:19.690010+0000","last_became_active":"2023-10-09T06:38:19.689749+0000","last_became_peered":"2023-10-09T06:38:19.689749+0000","last_unstale":"2023-10-09T06:38:19.690010+0000","last_undegraded":"2023-10-09T06:38:19.690010+0000","last_fullsized":"2023-10-09T06:38:19.690010+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:08:33.020051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682331+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:26:21.371656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.693531+0000","last_change":"2023-10-09T06:38:19.693531+0000","last_active":"2023-10-09T06:38:19.693531+0000","last_peered":"2023-10-09T06:38:19.693531+0000","last_clean":"2023-10-09T06:38:19.693531+0000","last_became_active":"2023-10-09T06:38:19.693246+0000","last_became_peered":"2023-10-09T06:38:19.693246+0000","last_unstale":"2023-10-09T06:38:19.693531+0000","last_undegraded":"2023-10-09T06:38:19.693531+0000","last_fullsized":"2023-10-09T06:38:19.693531+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:26:54.527634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.680775+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:50:10.718730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.694122+0000","last_change":"2023-10-09T06:38:19.694122+0000","last_active":"2023-10-09T06:38:19.694122+0000","last_peered":"2023-10-09T06:38:19.694122+0000","last_clean":"2023-10-09T06:38:19.694122+0000","last_became_active":"2023-10-09T06:38:19.693847+0000","last_became_peered":"2023-10-09T06:38:19.693847+0000","last_unstale":"2023-10-09T06:38:19.694122+0000","last_undegraded":"2023-10-09T06:38:19.694122+0000","last_fullsized":"2023-10-09T06:38:19.694122+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:50:50.722486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.683619+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:52:50.901568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.688209+0000","last_change":"2023-10-09T06:38:19.688209+0000","last_active":"2023-10-09T06:38:19.688209+0000","last_peered":"2023-10-09T06:38:19.688209+0000","last_clean":"2023-10-09T06:38:19.688209+0000","last_became_active":"2023-10-09T06:38:19.687961+0000","last_became_peered":"2023-10-09T06:38:19.687961+0000","last_unstale":"2023-10-09T06:38:19.688209+0000","last_undegraded":"2023-10-09T06:38:19.688209+0000","last_fullsized":"2023-10-09T06:38:19.688209+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:52:49.890865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678258+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:36:50.516609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.693838+0000","last_change":"2023-10-09T06:38:19.693838+0000","last_active":"2023-10-09T06:38:19.693838+0000","last_peered":"2023-10-09T06:38:19.693838+0000","last_clean":"2023-10-09T06:38:19.693838+0000","last_became_active":"2023-10-09T06:38:19.693584+0000","last_became_peered":"2023-10-09T06:38:19.693584+0000","last_unstale":"2023-10-09T06:38:19.693838+0000","last_undegraded":"2023-10-09T06:38:19.693838+0000","last_fullsized":"2023-10-09T06:38:19.693838+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:42:55.756537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677099+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:01:29.075717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.684840+0000","last_change":"2023-10-09T06:38:19.684840+0000","last_active":"2023-10-09T06:38:19.684840+0000","last_peered":"2023-10-09T06:38:19.684840+0000","last_clean":"2023-10-09T06:38:19.684840+0000","last_became_active":"2023-10-09T06:38:19.684531+0000","last_became_peered":"2023-10-09T06:38:19.684531+0000","last_unstale":"2023-10-09T06:38:19.684840+0000","last_undegraded":"2023-10-09T06:38:19.684840+0000","last_fullsized":"2023-10-09T06:38:19.684840+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:42:47.239564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677625+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:41:30.382056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.692513+0000","last_change":"2023-10-09T06:38:19.692513+0000","last_active":"2023-10-09T06:38:19.692513+0000","last_peered":"2023-10-09T06:38:19.692513+0000","last_clean":"2023-10-09T06:38:19.692513+0000","last_became_active":"2023-10-09T06:38:19.692237+0000","last_became_peered":"2023-10-09T06:38:19.692237+0000","last_unstale":"2023-10-09T06:38:19.692513+0000","last_undegraded":"2023-10-09T06:38:19.692513+0000","last_fullsized":"2023-10-09T06:38:19.692513+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:56:50.090947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677087+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:21:10.321869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.681581+0000","last_change":"2023-10-09T06:38:19.681581+0000","last_active":"2023-10-09T06:38:19.681581+0000","last_peered":"2023-10-09T06:38:19.681581+0000","last_clean":"2023-10-09T06:38:19.681581+0000","last_became_active":"2023-10-09T06:38:19.681316+0000","last_became_peered":"2023-10-09T06:38:19.681316+0000","last_unstale":"2023-10-09T06:38:19.681581+0000","last_undegraded":"2023-10-09T06:38:19.681581+0000","last_fullsized":"2023-10-09T06:38:19.681581+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:38:52.426468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677747+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:16:26.649802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.688731+0000","last_change":"2023-10-09T06:38:19.688731+0000","last_active":"2023-10-09T06:38:19.688731+0000","last_peered":"2023-10-09T06:38:19.688731+0000","last_clean":"2023-10-09T06:38:19.688731+0000","last_became_active":"2023-10-09T06:38:19.688486+0000","last_became_peered":"2023-10-09T06:38:19.688486+0000","last_unstale":"2023-10-09T06:38:19.688731+0000","last_undegraded":"2023-10-09T06:38:19.688731+0000","last_fullsized":"2023-10-09T06:38:19.688731+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:12:32.876485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.687784+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:57:12.115891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.683035+0000","last_change":"2023-10-09T06:38:19.683035+0000","last_active":"2023-10-09T06:38:19.683035+0000","last_peered":"2023-10-09T06:38:19.683035+0000","last_clean":"2023-10-09T06:38:19.683035+0000","last_became_active":"2023-10-09T06:38:19.682754+0000","last_became_peered":"2023-10-09T06:38:19.682754+0000","last_unstale":"2023-10-09T06:38:19.683035+0000","last_undegraded":"2023-10-09T06:38:19.683035+0000","last_fullsized":"2023-10-09T06:38:19.683035+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:40:21.799524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.687350+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:40:00.965033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.694515+0000","last_change":"2023-10-09T06:38:19.694515+0000","last_active":"2023-10-09T06:38:19.694515+0000","last_peered":"2023-10-09T06:38:19.694515+0000","last_clean":"2023-10-09T06:38:19.694515+0000","last_became_active":"2023-10-09T06:38:19.694041+0000","last_became_peered":"2023-10-09T06:38:19.694041+0000","last_unstale":"2023-10-09T06:38:19.694515+0000","last_undegraded":"2023-10-09T06:38:19.694515+0000","last_fullsized":"2023-10-09T06:38:19.694515+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:40:48.522136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.681184+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:59:57.861748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.690795+0000","last_change":"2023-10-09T06:38:19.690795+0000","last_active":"2023-10-09T06:38:19.690795+0000","last_peered":"2023-10-09T06:38:19.690795+0000","last_clean":"2023-10-09T06:38:19.690795+0000","last_became_active":"2023-10-09T06:38:19.690512+0000","last_became_peered":"2023-10-09T06:38:19.690512+0000","last_unstale":"2023-10-09T06:38:19.690795+0000","last_undegraded":"2023-10-09T06:38:19.690795+0000","last_fullsized":"2023-10-09T06:38:19.690795+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:40:21.799524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.676742+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:01:29.075717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.688034+0000","last_change":"2023-10-09T06:38:19.688034+0000","last_active":"2023-10-09T06:38:19.688034+0000","last_peered":"2023-10-09T06:38:19.688034+0000","last_clean":"2023-10-09T06:38:19.688034+0000","last_became_active":"2023-10-09T06:38:19.687764+0000","last_became_peered":"2023-10-09T06:38:19.687764+0000","last_unstale":"2023-10-09T06:38:19.688034+0000","last_undegraded":"2023-10-09T06:38:19.688034+0000","last_fullsized":"2023-10-09T06:38:19.688034+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:37:57.642710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.676045+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:16:26.649802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.686556+0000","last_change":"2023-10-09T06:38:19.686556+0000","last_active":"2023-10-09T06:38:19.686556+0000","last_peered":"2023-10-09T06:38:19.686556+0000","last_clean":"2023-10-09T06:38:19.686556+0000","last_became_active":"2023-10-09T06:38:19.686253+0000","last_became_peered":"2023-10-09T06:38:19.686253+0000","last_unstale":"2023-10-09T06:38:19.686556+0000","last_undegraded":"2023-10-09T06:38:19.686556+0000","last_fullsized":"2023-10-09T06:38:19.686556+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:26:54.527634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677131+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:16:34.306892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.693762+0000","last_change":"2023-10-09T06:38:19.693762+0000","last_active":"2023-10-09T06:38:19.693762+0000","last_peered":"2023-10-09T06:38:19.693762+0000","last_clean":"2023-10-09T06:38:19.693762+0000","last_became_active":"2023-10-09T06:38:19.693501+0000","last_became_peered":"2023-10-09T06:38:19.693501+0000","last_unstale":"2023-10-09T06:38:19.693762+0000","last_undegraded":"2023-10-09T06:38:19.693762+0000","last_fullsized":"2023-10-09T06:38:19.693762+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:26:54.527634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.683097+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:48:51.565773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.691160+0000","last_change":"2023-10-09T06:38:19.691160+0000","last_active":"2023-10-09T06:38:19.691160+0000","last_peered":"2023-10-09T06:38:19.691160+0000","last_clean":"2023-10-09T06:38:19.691160+0000","last_became_active":"2023-10-09T06:38:19.690880+0000","last_became_peered":"2023-10-09T06:38:19.690880+0000","last_unstale":"2023-10-09T06:38:19.691160+0000","last_undegraded":"2023-10-09T06:38:19.691160+0000","last_fullsized":"2023-10-09T06:38:19.691160+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:40:21.799524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.679938+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:43:32.088320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.684011+0000","last_change":"2023-10-09T06:38:19.684011+0000","last_active":"2023-10-09T06:38:19.684011+0000","last_peered":"2023-10-09T06:38:19.684011+0000","last_clean":"2023-10-09T06:38:19.684011+0000","last_became_active":"2023-10-09T06:38:19.683541+0000","last_became_peered":"2023-10-09T06:38:19.683541+0000","last_unstale":"2023-10-09T06:38:19.684011+0000","last_undegraded":"2023-10-09T06:38:19.684011+0000","last_fullsized":"2023-10-09T06:38:19.684011+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:54:42.729290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.679867+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:32:06.438731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.691307+0000","last_change":"2023-10-09T06:38:19.691307+0000","last_active":"2023-10-09T06:38:19.691307+0000","last_peered":"2023-10-09T06:38:19.691307+0000","last_clean":"2023-10-09T06:38:19.691307+0000","last_became_active":"2023-10-09T06:38:19.691050+0000","last_became_peered":"2023-10-09T06:38:19.691050+0000","last_unstale":"2023-10-09T06:38:19.691307+0000","last_undegraded":"2023-10-09T06:38:19.691307+0000","last_fullsized":"2023-10-09T06:38:19.691307+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:15:20.738142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677792+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:58:03.322193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.691686+0000","last_change":"2023-10-09T06:38:19.691686+0000","last_active":"2023-10-09T06:38:19.691686+0000","last_peered":"2023-10-09T06:38:19.691686+0000","last_clean":"2023-10-09T06:38:19.691686+0000","last_became_active":"2023-10-09T06:38:19.690753+0000","last_became_peered":"2023-10-09T06:38:19.690753+0000","last_unstale":"2023-10-09T06:38:19.691686+0000","last_undegraded":"2023-10-09T06:38:19.691686+0000","last_fullsized":"2023-10-09T06:38:19.691686+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:04:37.633413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.683688+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:29:07.591719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.687181+0000","last_change":"2023-10-09T06:38:19.687181+0000","last_active":"2023-10-09T06:38:19.687181+0000","last_peered":"2023-10-09T06:38:19.687181+0000","last_clean":"2023-10-09T06:38:19.687181+0000","last_became_active":"2023-10-09T06:38:19.686940+0000","last_became_peered":"2023-10-09T06:38:19.686940+0000","last_unstale":"2023-10-09T06:38:19.687181+0000","last_undegraded":"2023-10-09T06:38:19.687181+0000","last_fullsized":"2023-10-09T06:38:19.687181+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:52:11.581942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.680141+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:31:22.541312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.693786+0000","last_change":"2023-10-09T06:38:19.693786+0000","last_active":"2023-10-09T06:38:19.693786+0000","last_peered":"2023-10-09T06:38:19.693786+0000","last_clean":"2023-10-09T06:38:19.693786+0000","last_became_active":"2023-10-09T06:38:19.693529+0000","last_became_peered":"2023-10-09T06:38:19.693529+0000","last_unstale":"2023-10-09T06:38:19.693786+0000","last_undegraded":"2023-10-09T06:38:19.693786+0000","last_fullsized":"2023-10-09T06:38:19.693786+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:09:07.415429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678527+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:41:30.382056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.690991+0000","last_change":"2023-10-09T06:38:19.690991+0000","last_active":"2023-10-09T06:38:19.690991+0000","last_peered":"2023-10-09T06:38:19.690991+0000","last_clean":"2023-10-09T06:38:19.690991+0000","last_became_active":"2023-10-09T06:38:19.690659+0000","last_became_peered":"2023-10-09T06:38:19.690659+0000","last_unstale":"2023-10-09T06:38:19.690991+0000","last_undegraded":"2023-10-09T06:38:19.690991+0000","last_fullsized":"2023-10-09T06:38:19.690991+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:02:57.917372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.680109+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:10:38.081403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.684393+0000","last_change":"2023-10-09T06:38:19.684393+0000","last_active":"2023-10-09T06:38:19.684393+0000","last_peered":"2023-10-09T06:38:19.684393+0000","last_clean":"2023-10-09T06:38:19.684393+0000","last_became_active":"2023-10-09T06:38:19.683050+0000","last_became_peered":"2023-10-09T06:38:19.683050+0000","last_unstale":"2023-10-09T06:38:19.684393+0000","last_undegraded":"2023-10-09T06:38:19.684393+0000","last_fullsized":"2023-10-09T06:38:19.684393+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:22:25.945021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.676616+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:49:51.877549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.685457+0000","last_change":"2023-10-09T06:38:19.685457+0000","last_active":"2023-10-09T06:38:19.685457+0000","last_peered":"2023-10-09T06:38:19.685457+0000","last_clean":"2023-10-09T06:38:19.685457+0000","last_became_active":"2023-10-09T06:38:19.685181+0000","last_became_peered":"2023-10-09T06:38:19.685181+0000","last_unstale":"2023-10-09T06:38:19.685457+0000","last_undegraded":"2023-10-09T06:38:19.685457+0000","last_fullsized":"2023-10-09T06:38:19.685457+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:39:59.954330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682501+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:05:32.721283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.692581+0000","last_change":"2023-10-09T06:38:19.692581+0000","last_active":"2023-10-09T06:38:19.692581+0000","last_peered":"2023-10-09T06:38:19.692581+0000","last_clean":"2023-10-09T06:38:19.692581+0000","last_became_active":"2023-10-09T06:38:19.692312+0000","last_became_peered":"2023-10-09T06:38:19.692312+0000","last_unstale":"2023-10-09T06:38:19.692581+0000","last_undegraded":"2023-10-09T06:38:19.692581+0000","last_fullsized":"2023-10-09T06:38:19.692581+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:52:30.325627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682069+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:44:51.812594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.683671+0000","last_change":"2023-10-09T06:38:19.683671+0000","last_active":"2023-10-09T06:38:19.683671+0000","last_peered":"2023-10-09T06:38:19.683671+0000","last_clean":"2023-10-09T06:38:19.683671+0000","last_became_active":"2023-10-09T06:38:19.683355+0000","last_became_peered":"2023-10-09T06:38:19.683355+0000","last_unstale":"2023-10-09T06:38:19.683671+0000","last_undegraded":"2023-10-09T06:38:19.683671+0000","last_fullsized":"2023-10-09T06:38:19.683671+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:23:57.642507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":270,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-09T06:38:18.676452+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:38:18.676452+0000","last_peered":"2023-10-09T06:38:18.676452+0000","last_clean":"2023-10-09T06:38:18.676452+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:38:18.676452+0000","last_undegraded":"2023-10-09T06:38:18.676452+0000","last_fullsized":"2023-10-09T06:38:18.676452+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574946,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303820,"kb_used_data":2436,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068020,"statfs":{"total":96636764160,"available":96325652480,"internally_reserved":0,"allocated":2494464,"data_stored":1437946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52000000000000002,"15min":0.50700000000000001},"min":{"1min":0.34799999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.37,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67600000000000005,"5min":0.753,"15min":0.753},"last":0.52900000000000003}]},{"osd":1,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.503,"15min":0.51900000000000002},"min":{"1min":0.33600000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.74299999999999999,"5min":0.77400000000000002,"15min":1.2030000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.378,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.57899999999999996}]},{"osd":2,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53200000000000003,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75800000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.44,"5min":0.36399999999999999,"15min":0.315},"max":{"1min":0.77900000000000003,"5min":0.80700000000000005,"15min":0.83099999999999996},"last":0.56499999999999995}]},{"osd":3,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53800000000000003,"15min":0.55000000000000004},"min":{"1min":0.41199999999999998,"5min":0.35899999999999999,"15min":0.33500000000000002},"max":{"1min":0.68600000000000005,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55600000000000005,"15min":0.59199999999999997},"min":{"1min":0.45300000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.754,"5min":0.82099999999999995,"15min":1.1479999999999999},"last":0.50900000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.48199999999999998,"15min":0.51900000000000002},"min":{"1min":0.35599999999999998,"5min":0.30299999999999999,"15min":0.24399999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.93999999999999995},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56599999999999995,"15min":0.60599999999999998},"min":{"1min":0.36799999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76700000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.59199999999999997}]},{"osd":5,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52500000000000002,"15min":0.56999999999999995},"min":{"1min":0.36499999999999999,"5min":0.32500000000000001,"15min":0.307},"max":{"1min":0.81999999999999995,"5min":0.85799999999999998,"15min":0.91100000000000003},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59599999999999997,"15min":0.62},"min":{"1min":0.52900000000000003,"5min":0.41699999999999998,"15min":0.34200000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.621}]},{"osd":6,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55700000000000005,"15min":0.59299999999999997},"min":{"1min":0.38600000000000001,"5min":0.33500000000000002,"15min":0.28699999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.88600000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.56100000000000005,"15min":0.61399999999999999},"min":{"1min":0.45100000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.84599999999999997},"last":0.64300000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574945,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303228,"kb_used_data":1844,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068612,"statfs":{"total":96636764160,"available":96326258688,"internally_reserved":0,"allocated":1888256,"data_stored":841315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.50800000000000001,"15min":0.502},"min":{"1min":0.35499999999999998,"5min":0.32000000000000001,"15min":0.30399999999999999},"max":{"1min":0.66500000000000004,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.50700000000000001,"15min":0.51000000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70399999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.51300000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.502,"15min":0.51200000000000001},"min":{"1min":0.33600000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68300000000000005,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.5},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52900000000000003,"15min":0.53200000000000003},"min":{"1min":0.38700000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.77100000000000002,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.60299999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.34499999999999997,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.72399999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51100000000000001,"15min":0.52900000000000003},"min":{"1min":0.39000000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.66600000000000004,"5min":0.67500000000000004,"15min":0.752},"last":0.44500000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.38900000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66600000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.54300000000000004,"15min":0.68799999999999994},"min":{"1min":0.40400000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.752,"5min":0.752,"15min":4.7510000000000003},"last":0.53600000000000003}]},{"osd":4,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46400000000000002,"15min":0.47699999999999998},"min":{"1min":0.29899999999999999,"5min":0.28199999999999997,"15min":0.27900000000000003},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.76400000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56100000000000005,"15min":0.58099999999999996},"min":{"1min":0.41799999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.77200000000000002,"5min":0.81299999999999994,"15min":0.83699999999999997},"last":0.66500000000000004}]},{"osd":5,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.52100000000000002,"15min":0.53000000000000003},"min":{"1min":0.31900000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.66100000000000003,"5min":0.79200000000000004,"15min":0.81200000000000006},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.59499999999999997},"min":{"1min":0.46899999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76000000000000001,"5min":0.78000000000000003,"15min":0.86599999999999999},"last":0.58999999999999997}]},{"osd":7,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55200000000000005,"15min":0.58199999999999996},"min":{"1min":0.41399999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80500000000000005,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58799999999999997,"15min":0.58899999999999997},"min":{"1min":0.46000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78400000000000003,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.57399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":847578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.316,"15min":0.311},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.503,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.34899999999999998,"15min":0.35799999999999998},"min":{"1min":0.28399999999999997,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.51900000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.29599999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.34399999999999997,"15min":0.35099999999999998},"min":{"1min":0.215,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.53700000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.38300000000000001,"15min":0.38500000000000001},"min":{"1min":0.30299999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.55300000000000005,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.34399999999999997}]},{"osd":3,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.36499999999999999,"15min":0.36399999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.55800000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.34599999999999997,"15min":0.53500000000000003},"min":{"1min":0.23400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.40400000000000003,"5min":0.56000000000000005,"15min":5.8529999999999998},"last":0.39200000000000002}]},{"osd":4,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58299999999999996,"15min":0.57499999999999996},"min":{"1min":0.36199999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78300000000000003,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.40000000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.81100000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.64200000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57699999999999996,"15min":0.74099999999999999},"min":{"1min":0.36499999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88,"5min":0.88,"15min":5.9459999999999997},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61199999999999999,"15min":0.61099999999999999},"min":{"1min":0.38900000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82599999999999996,"5min":0.92700000000000005,"15min":0.93999999999999995},"last":0.66400000000000003}]},{"osd":6,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54900000000000004,"15min":0.71199999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79100000000000004,"5min":0.79600000000000004,"15min":5.9160000000000004},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.379,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84799999999999998,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.61299999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57199999999999995,"15min":0.58599999999999997},"min":{"1min":0.39000000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.80900000000000005,"5min":0.89500000000000002,"15min":0.97099999999999997},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57099999999999995,"15min":0.749},"min":{"1min":0.41399999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":5.9729999999999999},"last":0.60499999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574947,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303808,"kb_used_data":2424,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068032,"statfs":{"total":96636764160,"available":96325664768,"internally_reserved":0,"allocated":2482176,"data_stored":1431683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.30199999999999999,"15min":0.315},"min":{"1min":0.216,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.433,"5min":0.498,"15min":0.498},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.32100000000000001,"15min":0.34399999999999997},"min":{"1min":0.222,"5min":0.214,"15min":0.214},"max":{"1min":0.497,"5min":0.53600000000000003,"15min":0.94499999999999995},"last":0.34899999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35199999999999998,"15min":0.36199999999999999},"min":{"1min":0.25900000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.50900000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.371,"15min":0.38800000000000001},"min":{"1min":0.30499999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.56499999999999995,"5min":0.69399999999999995,"15min":0.91700000000000004},"last":0.47099999999999997}]},{"osd":3,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.32800000000000001,"15min":0.38400000000000001},"min":{"1min":0.23699999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":1.554},"last":0.499},{"interface":"front","average":{"1min":0.371,"5min":0.35999999999999999,"15min":0.39700000000000002},"min":{"1min":0.27700000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54400000000000004,"5min":0.58099999999999996,"15min":0.79000000000000004},"last":0.311}]},{"osd":4,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53600000000000003,"15min":0.55100000000000005},"min":{"1min":0.41199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":1.212},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.72099999999999997},"min":{"1min":0.42199999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":5.484},"last":0.77800000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.72499999999999998},"min":{"1min":0.41999999999999998,"5min":0.35799999999999998,"15min":0.35499999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":5.5449999999999999},"last":0.627},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.83299999999999996,"5min":0.84499999999999997,"15min":1.113},"last":0.52400000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":1.0529999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59299999999999997,"15min":0.60699999999999998},"min":{"1min":0.45800000000000002,"5min":0.42799999999999999,"15min":0.41599999999999998},"max":{"1min":0.878,"5min":0.878,"15min":1.169},"last":0.61099999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57099999999999995,"15min":0.58899999999999997},"min":{"1min":0.40600000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.78700000000000003,"5min":0.85499999999999998,"15min":1.2390000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56399999999999995,"15min":0.749},"min":{"1min":0.38,"5min":0.375,"15min":0.375},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":5.5800000000000001},"last":0.55800000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574948,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":847578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.316,"15min":0.35299999999999998},"min":{"1min":0.23200000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":1.4199999999999999},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.55900000000000005},"min":{"1min":0.27500000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.51600000000000001,"5min":0.53800000000000003,"15min":7.2699999999999996},"last":0.27800000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.317,"15min":0.34399999999999997},"min":{"1min":0.23999999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.65400000000000003},"last":0.25},{"interface":"front","average":{"1min":0.372,"5min":0.37,"15min":0.61299999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":7.327},"last":0.313}]},{"osd":3,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.36799999999999999,"15min":0.39100000000000001},"min":{"1min":0.29399999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.68600000000000005},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.36799999999999999,"15min":0.39200000000000002},"min":{"1min":0.27900000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.72699999999999998},"last":0.30199999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54500000000000004,"15min":0.54400000000000004},"min":{"1min":0.40799999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.78400000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56200000000000006,"15min":0.78600000000000003},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82699999999999996,"5min":0.84699999999999998,"15min":7.2389999999999999},"last":0.35999999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.58099999999999996,"15min":0.91500000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82699999999999996,"5min":0.83799999999999997,"15min":10.890000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":0.84299999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.51000000000000001}]},{"osd":6,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56399999999999995,"15min":0.79100000000000004},"min":{"1min":0.42999999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.748,"5min":0.77300000000000002,"15min":7.2999999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.45800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.433}]},{"osd":7,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59299999999999997,"15min":0.82199999999999995},"min":{"1min":0.47499999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":7.1879999999999997},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60399999999999998,"15min":0.624},"min":{"1min":0.49399999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.90500000000000003},"last":0.5}]}]},{"osd":3,"up_from":13,"seq":55834574947,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":847578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.32200000000000001,"15min":0.33200000000000002},"min":{"1min":0.23100000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":5.8700000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.34699999999999998,"15min":0.36699999999999999},"min":{"1min":0.23999999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":5.6769999999999996},"last":0.25600000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.35799999999999998,"15min":0.36099999999999999},"min":{"1min":0.26800000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":4.0469999999999997},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.372,"15min":0.38800000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":4.1509999999999998},"last":0.34799999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.35199999999999998,"15min":0.374},"min":{"1min":0.29999999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.50800000000000001,"5min":0.56799999999999995,"15min":3.1139999999999999},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.373,"15min":0.38600000000000001},"min":{"1min":0.28100000000000003,"5min":0.26100000000000001,"15min":0.25600000000000001},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":3.3180000000000001},"last":0.309}]},{"osd":4,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55800000000000005,"15min":0.57099999999999995},"min":{"1min":0.38100000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":3.0899999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56499999999999995,"15min":0.56799999999999995},"min":{"1min":0.40300000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76400000000000001,"5min":0.91100000000000003,"15min":3.0710000000000002},"last":0.443}]},{"osd":5,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57499999999999996,"15min":0.58899999999999997},"min":{"1min":0.39100000000000001,"5min":0.34599999999999997,"15min":0.32700000000000001},"max":{"1min":0.877,"5min":0.877,"15min":6.2300000000000004},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.441,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79800000000000004,"5min":0.89500000000000002,"15min":3.0219999999999998},"last":0.505}]},{"osd":6,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.44500000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":2.6850000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.65200000000000002},"min":{"1min":0.43099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":6.5970000000000004},"last":0.55000000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60699999999999998,"15min":0.61399999999999999},"min":{"1min":0.44700000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86099999999999999,"5min":0.93300000000000005,"15min":4.2629999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59999999999999998,"15min":0.61299999999999999},"min":{"1min":0.40500000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.82899999999999996,"5min":0.97199999999999998,"15min":7.3650000000000002},"last":0.56299999999999994}]}]},{"osd":4,"up_from":13,"seq":55834574946,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":847578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.51400000000000001,"15min":0.55300000000000005},"min":{"1min":0.318,"5min":0.28100000000000003,"15min":0.27000000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":1.083},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.35699999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.79900000000000004,"5min":0.97799999999999998,"15min":1.054},"last":0.52400000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.39900000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.501,"5min":0.50700000000000001,"15min":0.49399999999999999},"min":{"1min":0.33800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.76300000000000001,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.505}]},{"osd":2,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52700000000000002,"15min":0.53900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.32900000000000001},"max":{"1min":0.76600000000000001,"5min":0.82099999999999995,"15min":1.0249999999999999},"last":0.443},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55700000000000005,"15min":0.59099999999999997},"min":{"1min":0.39400000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.93799999999999994},"last":0.41999999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52900000000000003,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.32200000000000001,"15min":0.29199999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.98499999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57699999999999996,"15min":0.58799999999999997},"min":{"1min":0.38700000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.1319999999999999},"last":0.48399999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51600000000000001,"15min":0.55100000000000005},"min":{"1min":0.31900000000000001,"5min":0.30399999999999999,"15min":0.27300000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58199999999999996,"15min":0.60599999999999998},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.35099999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.97999999999999998},"last":0.54300000000000004}]},{"osd":6,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.56799999999999995},"min":{"1min":0.41599999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.76800000000000002,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.59799999999999998},"min":{"1min":0.38700000000000001,"5min":0.34999999999999998,"15min":0.32700000000000001},"max":{"1min":0.85599999999999998,"5min":1.01,"15min":1.01},"last":0.64200000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54300000000000004,"15min":0.56999999999999995},"min":{"1min":0.40200000000000002,"5min":0.33600000000000002,"15min":0.28299999999999997},"max":{"1min":0.78900000000000003,"5min":0.85599999999999998,"15min":0.90000000000000002},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.60699999999999998,"15min":0.64500000000000002},"min":{"1min":0.36699999999999999,"5min":0.34999999999999998,"15min":0.32200000000000001},"max":{"1min":0.89000000000000001,"5min":0.98999999999999999,"15min":1.175},"last":0.627}]}]},{"osd":5,"up_from":13,"seq":55834574946,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":847578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.503,"15min":0.50800000000000001},"min":{"1min":0.38800000000000001,"5min":0.29299999999999998,"15min":0.28599999999999998},"max":{"1min":0.67900000000000005,"5min":0.71399999999999997,"15min":0.86599999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51800000000000002,"15min":0.51400000000000001},"min":{"1min":0.42499999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.60599999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.496}]},{"osd":1,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.38400000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.63900000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52300000000000002,"15min":0.53100000000000003},"min":{"1min":0.43099999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72799999999999998,"5min":0.73899999999999999,"15min":0.81599999999999995},"last":0.47999999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54300000000000004,"15min":0.54400000000000004},"min":{"1min":0.41099999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.63900000000000001,"5min":0.72999999999999998,"15min":0.83999999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.47199999999999998,"5min":0.40999999999999998,"15min":0.39500000000000002},"max":{"1min":0.72699999999999998,"5min":0.74099999999999999,"15min":0.79900000000000004},"last":0.53600000000000003}]},{"osd":3,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.42399999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.66900000000000004,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.54900000000000004,"15min":0.56000000000000005},"min":{"1min":0.42499999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.63500000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.55800000000000005}]},{"osd":4,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.50600000000000001,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.36199999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.80700000000000005},"last":0.433}]},{"osd":6,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55300000000000005,"15min":0.53600000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.83199999999999996},"last":0.378},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.36199999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77900000000000003,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.40200000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.57899999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71499999999999997,"5min":0.79500000000000004,"15min":0.85599999999999998},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.374,"5min":0.35199999999999998,"15min":0.314},"max":{"1min":0.79500000000000004,"5min":0.84499999999999997,"15min":0.90600000000000003},"last":0.44700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:38:24.743 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:38:24.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.890+0000 7f0c4cb9f700 1 Processor -- start 2023-10-09T06:38:24.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.890+0000 7f0c4cb9f700 1 -- start start 2023-10-09T06:38:24.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.891+0000 7f0c4cb9f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c48159420 0x7f0c480b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:24.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.891+0000 7f0c4cb9f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0c481597f0 0x7f0c480b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:24.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.891+0000 7f0c4cb9f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c480b11d0 con 0x7f0c4815d230 2023-10-09T06:38:24.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.891+0000 7f0c4cb9f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c480b14b0 con 0x7f0c481597f0 2023-10-09T06:38:24.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.891+0000 7f0c4cb9f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c480b1790 con 0x7f0c48159420 2023-10-09T06:38:24.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.891+0000 7f0c477fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c48159420 0x7f0c480b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:24.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.891+0000 7f0c46ffd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0c481597f0 0x7f0c480b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:24.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.891+0000 7f0c477fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c48159420 0x7f0c480b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:41328/0 (socket says 172.21.15.163:41328) 2023-10-09T06:38:24.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.891+0000 7f0c477fe700 1 -- 172.21.15.163:0/243132320 learned_addr learned my addr 172.21.15.163:0/243132320 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:24.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.891+0000 7f0c47fff700 1 --1- 172.21.15.163:0/243132320 >> v1:172.21.15.181:6789/0 conn(0x7f0c4815d230 0x7f0c480b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:34964/0 (socket says 172.21.15.163:34964) 2023-10-09T06:38:24.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.891+0000 7f0c46ffd700 1 -- 172.21.15.163:0/243132320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c48159420 msgr2=0x7f0c480b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:24.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.891+0000 7f0c46ffd700 1 --2- 172.21.15.163:0/243132320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c48159420 0x7f0c480b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:24.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.891+0000 7f0c46ffd700 1 -- 172.21.15.163:0/243132320 >> v1:172.21.15.181:6789/0 conn(0x7f0c4815d230 legacy=0x7f0c480b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:24.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.891+0000 7f0c46ffd700 1 -- 172.21.15.163:0/243132320 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c480b1a70 con 0x7f0c481597f0 2023-10-09T06:38:24.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.891+0000 7f0c477fe700 1 --2- 172.21.15.163:0/243132320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c48159420 0x7f0c480b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:38:24.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.892+0000 7f0c46ffd700 1 --2- 172.21.15.163:0/243132320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c481597f0 0x7f0c480b2270 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0c380098e0 tx=0x7f0c380091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c02f163da61ba5b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:24.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.892+0000 7f0c467fc700 1 -- 172.21.15.163:0/243132320 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3800b640 con 0x7f0c481597f0 2023-10-09T06:38:24.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.892+0000 7f0c467fc700 1 -- 172.21.15.163:0/243132320 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3800b7a0 con 0x7f0c481597f0 2023-10-09T06:38:24.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.892+0000 7f0c467fc700 1 -- 172.21.15.163:0/243132320 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3800bb40 con 0x7f0c481597f0 2023-10-09T06:38:24.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.892+0000 7f0c4cb9f700 1 -- 172.21.15.163:0/243132320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c481597f0 msgr2=0x7f0c480b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:24.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.892+0000 7f0c4cb9f700 1 --2- 172.21.15.163:0/243132320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c481597f0 0x7f0c480b2270 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0c380098e0 tx=0x7f0c380091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:24.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.892+0000 7f0c4cb9f700 1 -- 172.21.15.163:0/243132320 shutdown_connections 2023-10-09T06:38:24.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.892+0000 7f0c4cb9f700 1 --2- 172.21.15.163:0/243132320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c48159420 0x7f0c480b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:24.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.892+0000 7f0c4cb9f700 1 --2- 172.21.15.163:0/243132320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c481597f0 0x7f0c480b2270 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:24.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.892+0000 7f0c4cb9f700 1 -- 172.21.15.163:0/243132320 >> 172.21.15.163:0/243132320 conn(0x7f0c480d0660 msgr2=0x7f0c480d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:24.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.892+0000 7f0c4cb9f700 1 -- 172.21.15.163:0/243132320 shutdown_connections 2023-10-09T06:38:24.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.892+0000 7f0c4cb9f700 1 -- 172.21.15.163:0/243132320 wait complete. 2023-10-09T06:38:24.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.893+0000 7f0c4cb9f700 1 Processor -- start 2023-10-09T06:38:24.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.894+0000 7f0c4cb9f700 1 -- start start 2023-10-09T06:38:24.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.894+0000 7f0c4cb9f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c48159420 0x7f0c4815e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:24.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.894+0000 7f0c4cb9f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c481597f0 0x7f0c4816a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:24.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.894+0000 7f0c4cb9f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c4815d230 0x7f0c481695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:24.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.894+0000 7f0c4cb9f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4816aca0 con 0x7f0c4815d230 2023-10-09T06:38:24.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.894+0000 7f0c4cb9f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4815e800 con 0x7f0c48159420 2023-10-09T06:38:24.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.894+0000 7f0c4cb9f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4815eae0 con 0x7f0c481597f0 2023-10-09T06:38:24.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.894+0000 7f0c47fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c4815d230 0x7f0c481695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:24.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.894+0000 7f0c47fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c4815d230 0x7f0c481695c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42160/0 (socket says 172.21.15.163:42160) 2023-10-09T06:38:24.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.894+0000 7f0c47fff700 1 -- 172.21.15.163:0/406256721 learned_addr learned my addr 172.21.15.163:0/406256721 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:24.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.895+0000 7f0c46ffd700 1 --2- 172.21.15.163:0/406256721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c481597f0 0x7f0c4816a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:24.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.895+0000 7f0c477fe700 1 --2- 172.21.15.163:0/406256721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c48159420 0x7f0c4815e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:24.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.895+0000 7f0c47fff700 1 -- 172.21.15.163:0/406256721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c481597f0 msgr2=0x7f0c4816a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:24.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.895+0000 7f0c47fff700 1 --2- 172.21.15.163:0/406256721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c481597f0 0x7f0c4816a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:24.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.895+0000 7f0c47fff700 1 -- 172.21.15.163:0/406256721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c48159420 msgr2=0x7f0c4815e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:24.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.895+0000 7f0c47fff700 1 --2- 172.21.15.163:0/406256721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c48159420 0x7f0c4815e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:24.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.895+0000 7f0c47fff700 1 -- 172.21.15.163:0/406256721 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3c0094f0 con 0x7f0c4815d230 2023-10-09T06:38:24.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.895+0000 7f0c477fe700 1 --2- 172.21.15.163:0/406256721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c48159420 0x7f0c4815e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:24.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.895+0000 7f0c47fff700 1 --2- 172.21.15.163:0/406256721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c4815d230 0x7f0c481695c0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000cc70 tx=0x7f0c40001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:24.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.895+0000 7f0c2f7fe700 1 -- 172.21.15.163:0/406256721 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c40005df0 con 0x7f0c4815d230 2023-10-09T06:38:24.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.895+0000 7f0c2f7fe700 1 -- 172.21.15.163:0/406256721 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c4000a660 con 0x7f0c4815d230 2023-10-09T06:38:24.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.895+0000 7f0c4cb9f700 1 -- 172.21.15.163:0/406256721 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c38009420 con 0x7f0c4815d230 2023-10-09T06:38:24.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.895+0000 7f0c2f7fe700 1 -- 172.21.15.163:0/406256721 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4000a9e0 con 0x7f0c4815d230 2023-10-09T06:38:24.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.895+0000 7f0c4cb9f700 1 -- 172.21.15.163:0/406256721 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c4815f0a0 con 0x7f0c4815d230 2023-10-09T06:38:24.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.896+0000 7f0c4cb9f700 1 -- 172.21.15.163:0/406256721 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c4804c3c0 con 0x7f0c4815d230 2023-10-09T06:38:24.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.896+0000 7f0c2f7fe700 1 -- 172.21.15.163:0/406256721 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0c400070f0 con 0x7f0c4815d230 2023-10-09T06:38:24.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.897+0000 7f0c2f7fe700 1 --2- 172.21.15.163:0/406256721 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0c34067320 0x7f0c340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:24.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.897+0000 7f0c2f7fe700 1 -- 172.21.15.163:0/406256721 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0c40084c90 con 0x7f0c4815d230 2023-10-09T06:38:24.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.897+0000 7f0c477fe700 1 --2- 172.21.15.163:0/406256721 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0c34067320 0x7f0c340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:24.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.897+0000 7f0c477fe700 1 --2- 172.21.15.163:0/406256721 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0c34067320 0x7f0c340697d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c006d40 tx=0x7f0c3c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:24.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:24.901+0000 7f0c2f7fe700 1 -- 172.21.15.163:0/406256721 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c40051a50 con 0x7f0c4815d230 2023-10-09T06:38:25.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.079+0000 7f0c4cb9f700 1 -- 172.21.15.163:0/406256721 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0c480b1d50 con 0x7f0c34067320 2023-10-09T06:38:25.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.088+0000 7f0c2f7fe700 1 -- 172.21.15.163:0/406256721 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287181 (secure 0 0 0) 0x7f0c480b1d50 con 0x7f0c34067320 2023-10-09T06:38:25.092 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:25.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.091+0000 7f0c4cb9f700 1 -- 172.21.15.163:0/406256721 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0c34067320 msgr2=0x7f0c340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:25.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.091+0000 7f0c4cb9f700 1 --2- 172.21.15.163:0/406256721 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0c34067320 0x7f0c340697d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c006d40 tx=0x7f0c3c008040 comp rx=0 tx=0).stop 2023-10-09T06:38:25.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.091+0000 7f0c4cb9f700 1 -- 172.21.15.163:0/406256721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c4815d230 msgr2=0x7f0c481695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:25.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.091+0000 7f0c4cb9f700 1 --2- 172.21.15.163:0/406256721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c4815d230 0x7f0c481695c0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000cc70 tx=0x7f0c40001a70 comp rx=0 tx=0).stop 2023-10-09T06:38:25.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.091+0000 7f0c4cb9f700 1 -- 172.21.15.163:0/406256721 shutdown_connections 2023-10-09T06:38:25.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.091+0000 7f0c4cb9f700 1 --2- 172.21.15.163:0/406256721 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0c34067320 0x7f0c340697d0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:25.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.091+0000 7f0c4cb9f700 1 --2- 172.21.15.163:0/406256721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c4815d230 0x7f0c481695c0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:25.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.091+0000 7f0c4cb9f700 1 --2- 172.21.15.163:0/406256721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c48159420 0x7f0c4815e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:25.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.091+0000 7f0c4cb9f700 1 --2- 172.21.15.163:0/406256721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c481597f0 0x7f0c4816a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:25.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.091+0000 7f0c4cb9f700 1 -- 172.21.15.163:0/406256721 >> 172.21.15.163:0/406256721 conn(0x7f0c480d0660 msgr2=0x7f0c480b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:25.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.091+0000 7f0c4cb9f700 1 -- 172.21.15.163:0/406256721 shutdown_connections 2023-10-09T06:38:25.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.091+0000 7f0c4cb9f700 1 -- 172.21.15.163:0/406256721 wait complete. 2023-10-09T06:38:25.095 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:38:25.104 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2023-10-09T06:38:24.410910+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2427056,"kb_used_data":15984,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752547664,"statfs":{"total":773094113280,"available":770608807936,"internally_reserved":0,"allocated":16367616,"data_stored":7948834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.742617"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.687243+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:15:57.542772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.686956+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:10:38.081403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.681576+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:11:37.888420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.685055+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:58:03.322193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678873+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:41:30.382056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678481+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:36:50.516609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682433+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:46:35.635953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.676958+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:58:03.322193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.688589+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:20:02.974752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678660+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:46:35.635953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.680844+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:25:57.096853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678704+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:48:51.565773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.681087+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:25:53.616204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.683792+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:34:55.917445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678799+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:06:23.000411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677797+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:06:23.000411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682655+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:44:35.145850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678258+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:44:51.812594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.679357+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:44:35.145850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.676535+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:16:26.649802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.683129+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:38:53.437170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678996+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:19:50.331458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.681279+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:21:46.727517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682901+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:26:55.538336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.687816+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:08:34.030754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677830+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:01:29.075717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.679264+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:29:07.591719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.681628+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:49:58.537930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.680535+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:46:34.855757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.687098+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:46:34.855757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.676565+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:58:03.322193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677453+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:08:26.202100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.688987+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:42:48.250267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.691611+0000","last_change":"2023-10-09T06:38:19.691611+0000","last_active":"2023-10-09T06:38:19.691611+0000","last_peered":"2023-10-09T06:38:19.691611+0000","last_clean":"2023-10-09T06:38:19.691611+0000","last_became_active":"2023-10-09T06:38:19.691367+0000","last_became_peered":"2023-10-09T06:38:19.691367+0000","last_unstale":"2023-10-09T06:38:19.691611+0000","last_undegraded":"2023-10-09T06:38:19.691611+0000","last_fullsized":"2023-10-09T06:38:19.691611+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:13:34.033630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.679700+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:15:52.880306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.694250+0000","last_change":"2023-10-09T06:38:19.694250+0000","last_active":"2023-10-09T06:38:19.694250+0000","last_peered":"2023-10-09T06:38:19.694250+0000","last_clean":"2023-10-09T06:38:19.694250+0000","last_became_active":"2023-10-09T06:38:19.693478+0000","last_became_peered":"2023-10-09T06:38:19.693478+0000","last_unstale":"2023-10-09T06:38:19.694250+0000","last_undegraded":"2023-10-09T06:38:19.694250+0000","last_fullsized":"2023-10-09T06:38:19.694250+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:50:27.283148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677343+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:52:50.901568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.688925+0000","last_change":"2023-10-09T06:38:19.688925+0000","last_active":"2023-10-09T06:38:19.688925+0000","last_peered":"2023-10-09T06:38:19.688925+0000","last_clean":"2023-10-09T06:38:19.688925+0000","last_became_active":"2023-10-09T06:38:19.688479+0000","last_became_peered":"2023-10-09T06:38:19.688479+0000","last_unstale":"2023-10-09T06:38:19.688925+0000","last_undegraded":"2023-10-09T06:38:19.688925+0000","last_fullsized":"2023-10-09T06:38:19.688925+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:32:15.314337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677169+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:36:50.516609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.692752+0000","last_change":"2023-10-09T06:38:19.692752+0000","last_active":"2023-10-09T06:38:19.692752+0000","last_peered":"2023-10-09T06:38:19.692752+0000","last_clean":"2023-10-09T06:38:19.692752+0000","last_became_active":"2023-10-09T06:38:19.692249+0000","last_became_peered":"2023-10-09T06:38:19.692249+0000","last_unstale":"2023-10-09T06:38:19.692752+0000","last_undegraded":"2023-10-09T06:38:19.692752+0000","last_fullsized":"2023-10-09T06:38:19.692752+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:52:11.581942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682687+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:40:22.810227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.690895+0000","last_change":"2023-10-09T06:38:19.690895+0000","last_active":"2023-10-09T06:38:19.690895+0000","last_peered":"2023-10-09T06:38:19.690895+0000","last_clean":"2023-10-09T06:38:19.690895+0000","last_became_active":"2023-10-09T06:38:19.690633+0000","last_became_peered":"2023-10-09T06:38:19.690633+0000","last_unstale":"2023-10-09T06:38:19.690895+0000","last_undegraded":"2023-10-09T06:38:19.690895+0000","last_fullsized":"2023-10-09T06:38:19.690895+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:26:20.360953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.676460+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:08:26.202100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.687201+0000","last_change":"2023-10-09T06:38:19.687201+0000","last_active":"2023-10-09T06:38:19.687201+0000","last_peered":"2023-10-09T06:38:19.687201+0000","last_clean":"2023-10-09T06:38:19.687201+0000","last_became_active":"2023-10-09T06:38:19.686979+0000","last_became_peered":"2023-10-09T06:38:19.686979+0000","last_unstale":"2023-10-09T06:38:19.687201+0000","last_undegraded":"2023-10-09T06:38:19.687201+0000","last_fullsized":"2023-10-09T06:38:19.687201+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:04:37.633413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682248+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:22:26.955724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.690010+0000","last_change":"2023-10-09T06:38:19.690010+0000","last_active":"2023-10-09T06:38:19.690010+0000","last_peered":"2023-10-09T06:38:19.690010+0000","last_clean":"2023-10-09T06:38:19.690010+0000","last_became_active":"2023-10-09T06:38:19.689749+0000","last_became_peered":"2023-10-09T06:38:19.689749+0000","last_unstale":"2023-10-09T06:38:19.690010+0000","last_undegraded":"2023-10-09T06:38:19.690010+0000","last_fullsized":"2023-10-09T06:38:19.690010+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:08:33.020051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682331+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:26:21.371656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.693531+0000","last_change":"2023-10-09T06:38:19.693531+0000","last_active":"2023-10-09T06:38:19.693531+0000","last_peered":"2023-10-09T06:38:19.693531+0000","last_clean":"2023-10-09T06:38:19.693531+0000","last_became_active":"2023-10-09T06:38:19.693246+0000","last_became_peered":"2023-10-09T06:38:19.693246+0000","last_unstale":"2023-10-09T06:38:19.693531+0000","last_undegraded":"2023-10-09T06:38:19.693531+0000","last_fullsized":"2023-10-09T06:38:19.693531+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:26:54.527634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.680775+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:50:10.718730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.694122+0000","last_change":"2023-10-09T06:38:19.694122+0000","last_active":"2023-10-09T06:38:19.694122+0000","last_peered":"2023-10-09T06:38:19.694122+0000","last_clean":"2023-10-09T06:38:19.694122+0000","last_became_active":"2023-10-09T06:38:19.693847+0000","last_became_peered":"2023-10-09T06:38:19.693847+0000","last_unstale":"2023-10-09T06:38:19.694122+0000","last_undegraded":"2023-10-09T06:38:19.694122+0000","last_fullsized":"2023-10-09T06:38:19.694122+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:50:50.722486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.683619+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:52:50.901568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.688209+0000","last_change":"2023-10-09T06:38:19.688209+0000","last_active":"2023-10-09T06:38:19.688209+0000","last_peered":"2023-10-09T06:38:19.688209+0000","last_clean":"2023-10-09T06:38:19.688209+0000","last_became_active":"2023-10-09T06:38:19.687961+0000","last_became_peered":"2023-10-09T06:38:19.687961+0000","last_unstale":"2023-10-09T06:38:19.688209+0000","last_undegraded":"2023-10-09T06:38:19.688209+0000","last_fullsized":"2023-10-09T06:38:19.688209+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:52:49.890865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678258+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:36:50.516609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.693838+0000","last_change":"2023-10-09T06:38:19.693838+0000","last_active":"2023-10-09T06:38:19.693838+0000","last_peered":"2023-10-09T06:38:19.693838+0000","last_clean":"2023-10-09T06:38:19.693838+0000","last_became_active":"2023-10-09T06:38:19.693584+0000","last_became_peered":"2023-10-09T06:38:19.693584+0000","last_unstale":"2023-10-09T06:38:19.693838+0000","last_undegraded":"2023-10-09T06:38:19.693838+0000","last_fullsized":"2023-10-09T06:38:19.693838+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:42:55.756537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677099+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:01:29.075717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.684840+0000","last_change":"2023-10-09T06:38:19.684840+0000","last_active":"2023-10-09T06:38:19.684840+0000","last_peered":"2023-10-09T06:38:19.684840+0000","last_clean":"2023-10-09T06:38:19.684840+0000","last_became_active":"2023-10-09T06:38:19.684531+0000","last_became_peered":"2023-10-09T06:38:19.684531+0000","last_unstale":"2023-10-09T06:38:19.684840+0000","last_undegraded":"2023-10-09T06:38:19.684840+0000","last_fullsized":"2023-10-09T06:38:19.684840+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:42:47.239564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677625+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:41:30.382056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.692513+0000","last_change":"2023-10-09T06:38:19.692513+0000","last_active":"2023-10-09T06:38:19.692513+0000","last_peered":"2023-10-09T06:38:19.692513+0000","last_clean":"2023-10-09T06:38:19.692513+0000","last_became_active":"2023-10-09T06:38:19.692237+0000","last_became_peered":"2023-10-09T06:38:19.692237+0000","last_unstale":"2023-10-09T06:38:19.692513+0000","last_undegraded":"2023-10-09T06:38:19.692513+0000","last_fullsized":"2023-10-09T06:38:19.692513+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:56:50.090947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677087+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:21:10.321869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.681581+0000","last_change":"2023-10-09T06:38:19.681581+0000","last_active":"2023-10-09T06:38:19.681581+0000","last_peered":"2023-10-09T06:38:19.681581+0000","last_clean":"2023-10-09T06:38:19.681581+0000","last_became_active":"2023-10-09T06:38:19.681316+0000","last_became_peered":"2023-10-09T06:38:19.681316+0000","last_unstale":"2023-10-09T06:38:19.681581+0000","last_undegraded":"2023-10-09T06:38:19.681581+0000","last_fullsized":"2023-10-09T06:38:19.681581+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:38:52.426468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677747+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:16:26.649802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.688731+0000","last_change":"2023-10-09T06:38:19.688731+0000","last_active":"2023-10-09T06:38:19.688731+0000","last_peered":"2023-10-09T06:38:19.688731+0000","last_clean":"2023-10-09T06:38:19.688731+0000","last_became_active":"2023-10-09T06:38:19.688486+0000","last_became_peered":"2023-10-09T06:38:19.688486+0000","last_unstale":"2023-10-09T06:38:19.688731+0000","last_undegraded":"2023-10-09T06:38:19.688731+0000","last_fullsized":"2023-10-09T06:38:19.688731+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:12:32.876485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.687784+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:57:12.115891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.683035+0000","last_change":"2023-10-09T06:38:19.683035+0000","last_active":"2023-10-09T06:38:19.683035+0000","last_peered":"2023-10-09T06:38:19.683035+0000","last_clean":"2023-10-09T06:38:19.683035+0000","last_became_active":"2023-10-09T06:38:19.682754+0000","last_became_peered":"2023-10-09T06:38:19.682754+0000","last_unstale":"2023-10-09T06:38:19.683035+0000","last_undegraded":"2023-10-09T06:38:19.683035+0000","last_fullsized":"2023-10-09T06:38:19.683035+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:40:21.799524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.687350+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:40:00.965033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.694515+0000","last_change":"2023-10-09T06:38:19.694515+0000","last_active":"2023-10-09T06:38:19.694515+0000","last_peered":"2023-10-09T06:38:19.694515+0000","last_clean":"2023-10-09T06:38:19.694515+0000","last_became_active":"2023-10-09T06:38:19.694041+0000","last_became_peered":"2023-10-09T06:38:19.694041+0000","last_unstale":"2023-10-09T06:38:19.694515+0000","last_undegraded":"2023-10-09T06:38:19.694515+0000","last_fullsized":"2023-10-09T06:38:19.694515+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:40:48.522136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.681184+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:59:57.861748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.690795+0000","last_change":"2023-10-09T06:38:19.690795+0000","last_active":"2023-10-09T06:38:19.690795+0000","last_peered":"2023-10-09T06:38:19.690795+0000","last_clean":"2023-10-09T06:38:19.690795+0000","last_became_active":"2023-10-09T06:38:19.690512+0000","last_became_peered":"2023-10-09T06:38:19.690512+0000","last_unstale":"2023-10-09T06:38:19.690795+0000","last_undegraded":"2023-10-09T06:38:19.690795+0000","last_fullsized":"2023-10-09T06:38:19.690795+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:40:21.799524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.676742+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:01:29.075717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.688034+0000","last_change":"2023-10-09T06:38:19.688034+0000","last_active":"2023-10-09T06:38:19.688034+0000","last_peered":"2023-10-09T06:38:19.688034+0000","last_clean":"2023-10-09T06:38:19.688034+0000","last_became_active":"2023-10-09T06:38:19.687764+0000","last_became_peered":"2023-10-09T06:38:19.687764+0000","last_unstale":"2023-10-09T06:38:19.688034+0000","last_undegraded":"2023-10-09T06:38:19.688034+0000","last_fullsized":"2023-10-09T06:38:19.688034+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:37:57.642710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.676045+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:16:26.649802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.686556+0000","last_change":"2023-10-09T06:38:19.686556+0000","last_active":"2023-10-09T06:38:19.686556+0000","last_peered":"2023-10-09T06:38:19.686556+0000","last_clean":"2023-10-09T06:38:19.686556+0000","last_became_active":"2023-10-09T06:38:19.686253+0000","last_became_peered":"2023-10-09T06:38:19.686253+0000","last_unstale":"2023-10-09T06:38:19.686556+0000","last_undegraded":"2023-10-09T06:38:19.686556+0000","last_fullsized":"2023-10-09T06:38:19.686556+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:26:54.527634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677131+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:16:34.306892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.693762+0000","last_change":"2023-10-09T06:38:19.693762+0000","last_active":"2023-10-09T06:38:19.693762+0000","last_peered":"2023-10-09T06:38:19.693762+0000","last_clean":"2023-10-09T06:38:19.693762+0000","last_became_active":"2023-10-09T06:38:19.693501+0000","last_became_peered":"2023-10-09T06:38:19.693501+0000","last_unstale":"2023-10-09T06:38:19.693762+0000","last_undegraded":"2023-10-09T06:38:19.693762+0000","last_fullsized":"2023-10-09T06:38:19.693762+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:26:54.527634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.683097+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:48:51.565773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.691160+0000","last_change":"2023-10-09T06:38:19.691160+0000","last_active":"2023-10-09T06:38:19.691160+0000","last_peered":"2023-10-09T06:38:19.691160+0000","last_clean":"2023-10-09T06:38:19.691160+0000","last_became_active":"2023-10-09T06:38:19.690880+0000","last_became_peered":"2023-10-09T06:38:19.690880+0000","last_unstale":"2023-10-09T06:38:19.691160+0000","last_undegraded":"2023-10-09T06:38:19.691160+0000","last_fullsized":"2023-10-09T06:38:19.691160+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:40:21.799524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.679938+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:43:32.088320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.684011+0000","last_change":"2023-10-09T06:38:19.684011+0000","last_active":"2023-10-09T06:38:19.684011+0000","last_peered":"2023-10-09T06:38:19.684011+0000","last_clean":"2023-10-09T06:38:19.684011+0000","last_became_active":"2023-10-09T06:38:19.683541+0000","last_became_peered":"2023-10-09T06:38:19.683541+0000","last_unstale":"2023-10-09T06:38:19.684011+0000","last_undegraded":"2023-10-09T06:38:19.684011+0000","last_fullsized":"2023-10-09T06:38:19.684011+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:54:42.729290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.679867+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:32:06.438731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.691307+0000","last_change":"2023-10-09T06:38:19.691307+0000","last_active":"2023-10-09T06:38:19.691307+0000","last_peered":"2023-10-09T06:38:19.691307+0000","last_clean":"2023-10-09T06:38:19.691307+0000","last_became_active":"2023-10-09T06:38:19.691050+0000","last_became_peered":"2023-10-09T06:38:19.691050+0000","last_unstale":"2023-10-09T06:38:19.691307+0000","last_undegraded":"2023-10-09T06:38:19.691307+0000","last_fullsized":"2023-10-09T06:38:19.691307+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:15:20.738142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677792+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:58:03.322193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.691686+0000","last_change":"2023-10-09T06:38:19.691686+0000","last_active":"2023-10-09T06:38:19.691686+0000","last_peered":"2023-10-09T06:38:19.691686+0000","last_clean":"2023-10-09T06:38:19.691686+0000","last_became_active":"2023-10-09T06:38:19.690753+0000","last_became_peered":"2023-10-09T06:38:19.690753+0000","last_unstale":"2023-10-09T06:38:19.691686+0000","last_undegraded":"2023-10-09T06:38:19.691686+0000","last_fullsized":"2023-10-09T06:38:19.691686+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:04:37.633413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.683688+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:29:07.591719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.687181+0000","last_change":"2023-10-09T06:38:19.687181+0000","last_active":"2023-10-09T06:38:19.687181+0000","last_peered":"2023-10-09T06:38:19.687181+0000","last_clean":"2023-10-09T06:38:19.687181+0000","last_became_active":"2023-10-09T06:38:19.686940+0000","last_became_peered":"2023-10-09T06:38:19.686940+0000","last_unstale":"2023-10-09T06:38:19.687181+0000","last_undegraded":"2023-10-09T06:38:19.687181+0000","last_fullsized":"2023-10-09T06:38:19.687181+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:52:11.581942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.680141+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:31:22.541312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.693786+0000","last_change":"2023-10-09T06:38:19.693786+0000","last_active":"2023-10-09T06:38:19.693786+0000","last_peered":"2023-10-09T06:38:19.693786+0000","last_clean":"2023-10-09T06:38:19.693786+0000","last_became_active":"2023-10-09T06:38:19.693529+0000","last_became_peered":"2023-10-09T06:38:19.693529+0000","last_unstale":"2023-10-09T06:38:19.693786+0000","last_undegraded":"2023-10-09T06:38:19.693786+0000","last_fullsized":"2023-10-09T06:38:19.693786+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:09:07.415429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678527+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:41:30.382056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.690991+0000","last_change":"2023-10-09T06:38:19.690991+0000","last_active":"2023-10-09T06:38:19.690991+0000","last_peered":"2023-10-09T06:38:19.690991+0000","last_clean":"2023-10-09T06:38:19.690991+0000","last_became_active":"2023-10-09T06:38:19.690659+0000","last_became_peered":"2023-10-09T06:38:19.690659+0000","last_unstale":"2023-10-09T06:38:19.690991+0000","last_undegraded":"2023-10-09T06:38:19.690991+0000","last_fullsized":"2023-10-09T06:38:19.690991+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:02:57.917372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.680109+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:10:38.081403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.684393+0000","last_change":"2023-10-09T06:38:19.684393+0000","last_active":"2023-10-09T06:38:19.684393+0000","last_peered":"2023-10-09T06:38:19.684393+0000","last_clean":"2023-10-09T06:38:19.684393+0000","last_became_active":"2023-10-09T06:38:19.683050+0000","last_became_peered":"2023-10-09T06:38:19.683050+0000","last_unstale":"2023-10-09T06:38:19.684393+0000","last_undegraded":"2023-10-09T06:38:19.684393+0000","last_fullsized":"2023-10-09T06:38:19.684393+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:22:25.945021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.676616+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:49:51.877549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.685457+0000","last_change":"2023-10-09T06:38:19.685457+0000","last_active":"2023-10-09T06:38:19.685457+0000","last_peered":"2023-10-09T06:38:19.685457+0000","last_clean":"2023-10-09T06:38:19.685457+0000","last_became_active":"2023-10-09T06:38:19.685181+0000","last_became_peered":"2023-10-09T06:38:19.685181+0000","last_unstale":"2023-10-09T06:38:19.685457+0000","last_undegraded":"2023-10-09T06:38:19.685457+0000","last_fullsized":"2023-10-09T06:38:19.685457+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:39:59.954330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682501+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:05:32.721283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.692581+0000","last_change":"2023-10-09T06:38:19.692581+0000","last_active":"2023-10-09T06:38:19.692581+0000","last_peered":"2023-10-09T06:38:19.692581+0000","last_clean":"2023-10-09T06:38:19.692581+0000","last_became_active":"2023-10-09T06:38:19.692312+0000","last_became_peered":"2023-10-09T06:38:19.692312+0000","last_unstale":"2023-10-09T06:38:19.692581+0000","last_undegraded":"2023-10-09T06:38:19.692581+0000","last_fullsized":"2023-10-09T06:38:19.692581+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:52:30.325627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682069+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:44:51.812594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.683671+0000","last_change":"2023-10-09T06:38:19.683671+0000","last_active":"2023-10-09T06:38:19.683671+0000","last_peered":"2023-10-09T06:38:19.683671+0000","last_clean":"2023-10-09T06:38:19.683671+0000","last_became_active":"2023-10-09T06:38:19.683355+0000","last_became_peered":"2023-10-09T06:38:19.683355+0000","last_unstale":"2023-10-09T06:38:19.683671+0000","last_undegraded":"2023-10-09T06:38:19.683671+0000","last_fullsized":"2023-10-09T06:38:19.683671+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:23:57.642507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":270,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-09T06:38:18.676452+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:38:18.676452+0000","last_peered":"2023-10-09T06:38:18.676452+0000","last_clean":"2023-10-09T06:38:18.676452+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:38:18.676452+0000","last_undegraded":"2023-10-09T06:38:18.676452+0000","last_fullsized":"2023-10-09T06:38:18.676452+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574946,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303820,"kb_used_data":2436,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068020,"statfs":{"total":96636764160,"available":96325652480,"internally_reserved":0,"allocated":2494464,"data_stored":1437946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52000000000000002,"15min":0.50700000000000001},"min":{"1min":0.34799999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.37,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67600000000000005,"5min":0.753,"15min":0.753},"last":0.52900000000000003}]},{"osd":1,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.503,"15min":0.51900000000000002},"min":{"1min":0.33600000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.74299999999999999,"5min":0.77400000000000002,"15min":1.2030000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.378,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.57899999999999996}]},{"osd":2,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53200000000000003,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75800000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.44,"5min":0.36399999999999999,"15min":0.315},"max":{"1min":0.77900000000000003,"5min":0.80700000000000005,"15min":0.83099999999999996},"last":0.56499999999999995}]},{"osd":3,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53800000000000003,"15min":0.55000000000000004},"min":{"1min":0.41199999999999998,"5min":0.35899999999999999,"15min":0.33500000000000002},"max":{"1min":0.68600000000000005,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55600000000000005,"15min":0.59199999999999997},"min":{"1min":0.45300000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.754,"5min":0.82099999999999995,"15min":1.1479999999999999},"last":0.50900000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.48199999999999998,"15min":0.51900000000000002},"min":{"1min":0.35599999999999998,"5min":0.30299999999999999,"15min":0.24399999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.93999999999999995},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56599999999999995,"15min":0.60599999999999998},"min":{"1min":0.36799999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76700000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.59199999999999997}]},{"osd":5,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52500000000000002,"15min":0.56999999999999995},"min":{"1min":0.36499999999999999,"5min":0.32500000000000001,"15min":0.307},"max":{"1min":0.81999999999999995,"5min":0.85799999999999998,"15min":0.91100000000000003},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59599999999999997,"15min":0.62},"min":{"1min":0.52900000000000003,"5min":0.41699999999999998,"15min":0.34200000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.621}]},{"osd":6,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55700000000000005,"15min":0.59299999999999997},"min":{"1min":0.38600000000000001,"5min":0.33500000000000002,"15min":0.28699999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.88600000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.56100000000000005,"15min":0.61399999999999999},"min":{"1min":0.45100000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.84599999999999997},"last":0.64300000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574945,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303228,"kb_used_data":1844,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068612,"statfs":{"total":96636764160,"available":96326258688,"internally_reserved":0,"allocated":1888256,"data_stored":841315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.50800000000000001,"15min":0.502},"min":{"1min":0.35499999999999998,"5min":0.32000000000000001,"15min":0.30399999999999999},"max":{"1min":0.66500000000000004,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.50700000000000001,"15min":0.51000000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70399999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.51300000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.502,"15min":0.51200000000000001},"min":{"1min":0.33600000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68300000000000005,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.5},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52900000000000003,"15min":0.53200000000000003},"min":{"1min":0.38700000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.77100000000000002,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.60299999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.34499999999999997,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.72399999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51100000000000001,"15min":0.52900000000000003},"min":{"1min":0.39000000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.66600000000000004,"5min":0.67500000000000004,"15min":0.752},"last":0.44500000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.38900000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66600000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.54300000000000004,"15min":0.68799999999999994},"min":{"1min":0.40400000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.752,"5min":0.752,"15min":4.7510000000000003},"last":0.53600000000000003}]},{"osd":4,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46400000000000002,"15min":0.47699999999999998},"min":{"1min":0.29899999999999999,"5min":0.28199999999999997,"15min":0.27900000000000003},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.76400000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56100000000000005,"15min":0.58099999999999996},"min":{"1min":0.41799999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.77200000000000002,"5min":0.81299999999999994,"15min":0.83699999999999997},"last":0.66500000000000004}]},{"osd":5,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.52100000000000002,"15min":0.53000000000000003},"min":{"1min":0.31900000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.66100000000000003,"5min":0.79200000000000004,"15min":0.81200000000000006},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.59499999999999997},"min":{"1min":0.46899999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76000000000000001,"5min":0.78000000000000003,"15min":0.86599999999999999},"last":0.58999999999999997}]},{"osd":7,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55200000000000005,"15min":0.58199999999999996},"min":{"1min":0.41399999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80500000000000005,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58799999999999997,"15min":0.58899999999999997},"min":{"1min":0.46000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78400000000000003,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.57399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":847578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.316,"15min":0.311},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.503,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.34899999999999998,"15min":0.35799999999999998},"min":{"1min":0.28399999999999997,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.51900000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.29599999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.34399999999999997,"15min":0.35099999999999998},"min":{"1min":0.215,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.53700000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.38300000000000001,"15min":0.38500000000000001},"min":{"1min":0.30299999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.55300000000000005,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.34399999999999997}]},{"osd":3,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.36499999999999999,"15min":0.36399999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.55800000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.34599999999999997,"15min":0.53500000000000003},"min":{"1min":0.23400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.40400000000000003,"5min":0.56000000000000005,"15min":5.8529999999999998},"last":0.39200000000000002}]},{"osd":4,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58299999999999996,"15min":0.57499999999999996},"min":{"1min":0.36199999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78300000000000003,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.40000000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.81100000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.64200000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57699999999999996,"15min":0.74099999999999999},"min":{"1min":0.36499999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88,"5min":0.88,"15min":5.9459999999999997},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61199999999999999,"15min":0.61099999999999999},"min":{"1min":0.38900000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82599999999999996,"5min":0.92700000000000005,"15min":0.93999999999999995},"last":0.66400000000000003}]},{"osd":6,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54900000000000004,"15min":0.71199999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79100000000000004,"5min":0.79600000000000004,"15min":5.9160000000000004},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.379,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84799999999999998,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.61299999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57199999999999995,"15min":0.58599999999999997},"min":{"1min":0.39000000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.80900000000000005,"5min":0.89500000000000002,"15min":0.97099999999999997},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57099999999999995,"15min":0.749},"min":{"1min":0.41399999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":5.9729999999999999},"last":0.60499999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574947,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303808,"kb_used_data":2424,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068032,"statfs":{"total":96636764160,"available":96325664768,"internally_reserved":0,"allocated":2482176,"data_stored":1431683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.30199999999999999,"15min":0.315},"min":{"1min":0.216,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.433,"5min":0.498,"15min":0.498},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.32100000000000001,"15min":0.34399999999999997},"min":{"1min":0.222,"5min":0.214,"15min":0.214},"max":{"1min":0.497,"5min":0.53600000000000003,"15min":0.94499999999999995},"last":0.34899999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35199999999999998,"15min":0.36199999999999999},"min":{"1min":0.25900000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.50900000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.371,"15min":0.38800000000000001},"min":{"1min":0.30499999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.56499999999999995,"5min":0.69399999999999995,"15min":0.91700000000000004},"last":0.47099999999999997}]},{"osd":3,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.32800000000000001,"15min":0.38400000000000001},"min":{"1min":0.23699999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":1.554},"last":0.499},{"interface":"front","average":{"1min":0.371,"5min":0.35999999999999999,"15min":0.39700000000000002},"min":{"1min":0.27700000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54400000000000004,"5min":0.58099999999999996,"15min":0.79000000000000004},"last":0.311}]},{"osd":4,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53600000000000003,"15min":0.55100000000000005},"min":{"1min":0.41199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":1.212},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.72099999999999997},"min":{"1min":0.42199999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":5.484},"last":0.77800000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.72499999999999998},"min":{"1min":0.41999999999999998,"5min":0.35799999999999998,"15min":0.35499999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":5.5449999999999999},"last":0.627},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.83299999999999996,"5min":0.84499999999999997,"15min":1.113},"last":0.52400000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":1.0529999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59299999999999997,"15min":0.60699999999999998},"min":{"1min":0.45800000000000002,"5min":0.42799999999999999,"15min":0.41599999999999998},"max":{"1min":0.878,"5min":0.878,"15min":1.169},"last":0.61099999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57099999999999995,"15min":0.58899999999999997},"min":{"1min":0.40600000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.78700000000000003,"5min":0.85499999999999998,"15min":1.2390000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56399999999999995,"15min":0.749},"min":{"1min":0.38,"5min":0.375,"15min":0.375},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":5.5800000000000001},"last":0.55800000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574948,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":847578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.316,"15min":0.35299999999999998},"min":{"1min":0.23200000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":1.4199999999999999},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.55900000000000005},"min":{"1min":0.27500000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.51600000000000001,"5min":0.53800000000000003,"15min":7.2699999999999996},"last":0.27800000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.317,"15min":0.34399999999999997},"min":{"1min":0.23999999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.65400000000000003},"last":0.25},{"interface":"front","average":{"1min":0.372,"5min":0.37,"15min":0.61299999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":7.327},"last":0.313}]},{"osd":3,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.36799999999999999,"15min":0.39100000000000001},"min":{"1min":0.29399999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.68600000000000005},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.36799999999999999,"15min":0.39200000000000002},"min":{"1min":0.27900000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.72699999999999998},"last":0.30199999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54500000000000004,"15min":0.54400000000000004},"min":{"1min":0.40799999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.78400000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56200000000000006,"15min":0.78600000000000003},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82699999999999996,"5min":0.84699999999999998,"15min":7.2389999999999999},"last":0.35999999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.58099999999999996,"15min":0.91500000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82699999999999996,"5min":0.83799999999999997,"15min":10.890000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":0.84299999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.51000000000000001}]},{"osd":6,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56399999999999995,"15min":0.79100000000000004},"min":{"1min":0.42999999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.748,"5min":0.77300000000000002,"15min":7.2999999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.45800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.433}]},{"osd":7,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59299999999999997,"15min":0.82199999999999995},"min":{"1min":0.47499999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":7.1879999999999997},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60399999999999998,"15min":0.624},"min":{"1min":0.49399999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.90500000000000003},"last":0.5}]}]},{"osd":3,"up_from":13,"seq":55834574947,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":847578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.32200000000000001,"15min":0.33200000000000002},"min":{"1min":0.23100000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":5.8700000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.34699999999999998,"15min":0.36699999999999999},"min":{"1min":0.23999999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":5.6769999999999996},"last":0.25600000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.35799999999999998,"15min":0.36099999999999999},"min":{"1min":0.26800000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":4.0469999999999997},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.372,"15min":0.38800000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":4.1509999999999998},"last":0.34799999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.35199999999999998,"15min":0.374},"min":{"1min":0.29999999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.50800000000000001,"5min":0.56799999999999995,"15min":3.1139999999999999},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.373,"15min":0.38600000000000001},"min":{"1min":0.28100000000000003,"5min":0.26100000000000001,"15min":0.25600000000000001},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":3.3180000000000001},"last":0.309}]},{"osd":4,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55800000000000005,"15min":0.57099999999999995},"min":{"1min":0.38100000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":3.0899999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56499999999999995,"15min":0.56799999999999995},"min":{"1min":0.40300000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76400000000000001,"5min":0.91100000000000003,"15min":3.0710000000000002},"last":0.443}]},{"osd":5,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57499999999999996,"15min":0.58899999999999997},"min":{"1min":0.39100000000000001,"5min":0.34599999999999997,"15min":0.32700000000000001},"max":{"1min":0.877,"5min":0.877,"15min":6.2300000000000004},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.441,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79800000000000004,"5min":0.89500000000000002,"15min":3.0219999999999998},"last":0.505}]},{"osd":6,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.44500000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":2.6850000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.65200000000000002},"min":{"1min":0.43099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":6.5970000000000004},"last":0.55000000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60699999999999998,"15min":0.61399999999999999},"min":{"1min":0.44700000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86099999999999999,"5min":0.93300000000000005,"15min":4.2629999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59999999999999998,"15min":0.61299999999999999},"min":{"1min":0.40500000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.82899999999999996,"5min":0.97199999999999998,"15min":7.3650000000000002},"last":0.56299999999999994}]}]},{"osd":4,"up_from":13,"seq":55834574946,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":847578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.51400000000000001,"15min":0.55300000000000005},"min":{"1min":0.318,"5min":0.28100000000000003,"15min":0.27000000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":1.083},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.35699999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.79900000000000004,"5min":0.97799999999999998,"15min":1.054},"last":0.52400000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.39900000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.501,"5min":0.50700000000000001,"15min":0.49399999999999999},"min":{"1min":0.33800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.76300000000000001,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.505}]},{"osd":2,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52700000000000002,"15min":0.53900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.32900000000000001},"max":{"1min":0.76600000000000001,"5min":0.82099999999999995,"15min":1.0249999999999999},"last":0.443},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55700000000000005,"15min":0.59099999999999997},"min":{"1min":0.39400000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.93799999999999994},"last":0.41999999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52900000000000003,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.32200000000000001,"15min":0.29199999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.98499999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57699999999999996,"15min":0.58799999999999997},"min":{"1min":0.38700000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.1319999999999999},"last":0.48399999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51600000000000001,"15min":0.55100000000000005},"min":{"1min":0.31900000000000001,"5min":0.30399999999999999,"15min":0.27300000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58199999999999996,"15min":0.60599999999999998},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.35099999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.97999999999999998},"last":0.54300000000000004}]},{"osd":6,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.56799999999999995},"min":{"1min":0.41599999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.76800000000000002,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.59799999999999998},"min":{"1min":0.38700000000000001,"5min":0.34999999999999998,"15min":0.32700000000000001},"max":{"1min":0.85599999999999998,"5min":1.01,"15min":1.01},"last":0.64200000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54300000000000004,"15min":0.56999999999999995},"min":{"1min":0.40200000000000002,"5min":0.33600000000000002,"15min":0.28299999999999997},"max":{"1min":0.78900000000000003,"5min":0.85599999999999998,"15min":0.90000000000000002},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.60699999999999998,"15min":0.64500000000000002},"min":{"1min":0.36699999999999999,"5min":0.34999999999999998,"15min":0.32200000000000001},"max":{"1min":0.89000000000000001,"5min":0.98999999999999999,"15min":1.175},"last":0.627}]}]},{"osd":5,"up_from":13,"seq":55834574946,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":847578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.503,"15min":0.50800000000000001},"min":{"1min":0.38800000000000001,"5min":0.29299999999999998,"15min":0.28599999999999998},"max":{"1min":0.67900000000000005,"5min":0.71399999999999997,"15min":0.86599999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51800000000000002,"15min":0.51400000000000001},"min":{"1min":0.42499999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.60599999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.496}]},{"osd":1,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.38400000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.63900000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52300000000000002,"15min":0.53100000000000003},"min":{"1min":0.43099999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72799999999999998,"5min":0.73899999999999999,"15min":0.81599999999999995},"last":0.47999999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54300000000000004,"15min":0.54400000000000004},"min":{"1min":0.41099999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.63900000000000001,"5min":0.72999999999999998,"15min":0.83999999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.47199999999999998,"5min":0.40999999999999998,"15min":0.39500000000000002},"max":{"1min":0.72699999999999998,"5min":0.74099999999999999,"15min":0.79900000000000004},"last":0.53600000000000003}]},{"osd":3,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.42399999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.66900000000000004,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.54900000000000004,"15min":0.56000000000000005},"min":{"1min":0.42499999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.63500000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.55800000000000005}]},{"osd":4,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.50600000000000001,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.36199999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.80700000000000005},"last":0.433}]},{"osd":6,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55300000000000005,"15min":0.53600000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.83199999999999996},"last":0.378},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.36199999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77900000000000003,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.40200000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.57899999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71499999999999997,"5min":0.79500000000000004,"15min":0.85599999999999998},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.374,"5min":0.35199999999999998,"15min":0.314},"max":{"1min":0.79500000000000004,"5min":0.84499999999999997,"15min":0.90600000000000003},"last":0.44700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:38:25.113 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:38:25.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.258+0000 7f2b3e766700 1 Processor -- start 2023-10-09T06:38:25.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.258+0000 7f2b3e766700 1 -- start start 2023-10-09T06:38:25.262 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.259+0000 7f2b3e766700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b38159c60 0x7f2b3815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:25.262 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.259+0000 7f2b3e766700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2b3815db10 0x7f2b38069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:25.262 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.259+0000 7f2b3e766700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b3815a160 con 0x7f2b38159890 2023-10-09T06:38:25.262 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.259+0000 7f2b3e766700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b380d13a0 con 0x7f2b3815db10 2023-10-09T06:38:25.262 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.259+0000 7f2b3e766700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3815e140 con 0x7f2b38159c60 2023-10-09T06:38:25.262 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.259+0000 7f2b3cd03700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2b3815db10 0x7f2b38069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:25.262 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.259+0000 7f2b3cd03700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2b3815db10 0x7f2b38069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42174/0 (socket says 172.21.15.163:42174) 2023-10-09T06:38:25.262 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.259+0000 7f2b3cd03700 1 -- 172.21.15.163:0/4134125588 learned_addr learned my addr 172.21.15.163:0/4134125588 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:25.262 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.259+0000 7f2b37fff700 1 --1- 172.21.15.163:0/4134125588 >> v1:172.21.15.163:6789/0 conn(0x7f2b38159890 0x7f2b3815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33648/0 (socket says 172.21.15.163:33648) 2023-10-09T06:38:25.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.259+0000 7f2b377fe700 1 --2- 172.21.15.163:0/4134125588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b38159c60 0x7f2b3815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:25.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.260+0000 7f2b3cd03700 1 -- 172.21.15.163:0/4134125588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b38159c60 msgr2=0x7f2b3815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:25.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.260+0000 7f2b3cd03700 1 --2- 172.21.15.163:0/4134125588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b38159c60 0x7f2b3815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:25.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.260+0000 7f2b3cd03700 1 -- 172.21.15.163:0/4134125588 >> v1:172.21.15.163:6789/0 conn(0x7f2b38159890 legacy=0x7f2b3815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:25.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.260+0000 7f2b3cd03700 1 -- 172.21.15.163:0/4134125588 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b3815a3d0 con 0x7f2b3815db10 2023-10-09T06:38:25.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.260+0000 7f2b36ffd700 1 -- 172.21.15.163:0/4134125588 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1813271522 0 0) 0x7f2b3815a160 con 0x7f2b38159890 2023-10-09T06:38:25.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.260+0000 7f2b3cd03700 1 --2- 172.21.15.163:0/4134125588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b3815db10 0x7f2b38069560 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f2b30009f20 tx=0x7f2b30009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31ca97ef108c738 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:25.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.260+0000 7f2b36ffd700 1 -- 172.21.15.163:0/4134125588 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3000e070 con 0x7f2b3815db10 2023-10-09T06:38:25.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.260+0000 7f2b36ffd700 1 -- 172.21.15.163:0/4134125588 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b3000ba80 con 0x7f2b3815db10 2023-10-09T06:38:25.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.260+0000 7f2b36ffd700 1 -- 172.21.15.163:0/4134125588 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b300092e0 con 0x7f2b3815db10 2023-10-09T06:38:25.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.260+0000 7f2b3e766700 1 -- 172.21.15.163:0/4134125588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b3815db10 msgr2=0x7f2b38069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:25.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.260+0000 7f2b3e766700 1 --2- 172.21.15.163:0/4134125588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b3815db10 0x7f2b38069560 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f2b30009f20 tx=0x7f2b30009840 comp rx=0 tx=0).stop 2023-10-09T06:38:25.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.260+0000 7f2b3e766700 1 -- 172.21.15.163:0/4134125588 shutdown_connections 2023-10-09T06:38:25.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.260+0000 7f2b3e766700 1 --2- 172.21.15.163:0/4134125588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b3815db10 0x7f2b38069560 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:25.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.260+0000 7f2b3e766700 1 --2- 172.21.15.163:0/4134125588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b38159c60 0x7f2b3815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:25.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.260+0000 7f2b3e766700 1 -- 172.21.15.163:0/4134125588 >> 172.21.15.163:0/4134125588 conn(0x7f2b38068e20 msgr2=0x7f2b380d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:25.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.260+0000 7f2b3e766700 1 -- 172.21.15.163:0/4134125588 shutdown_connections 2023-10-09T06:38:25.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.260+0000 7f2b3e766700 1 -- 172.21.15.163:0/4134125588 wait complete. 2023-10-09T06:38:25.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.261+0000 7f2b3e766700 1 Processor -- start 2023-10-09T06:38:25.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.262+0000 7f2b3e766700 1 -- start start 2023-10-09T06:38:25.267 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.262+0000 7f2b3e766700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b38159890 0x7f2b380b4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:25.267 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.262+0000 7f2b3e766700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b38159c60 0x7f2b380b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:25.267 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.262+0000 7f2b3e766700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b3815db10 0x7f2b380b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:25.267 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.262+0000 7f2b3e766700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b380cd0d0 con 0x7f2b38159890 2023-10-09T06:38:25.267 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.262+0000 7f2b3e766700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b380cd690 con 0x7f2b38159c60 2023-10-09T06:38:25.267 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.262+0000 7f2b3e766700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3816a5e0 con 0x7f2b3815db10 2023-10-09T06:38:25.267 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.262+0000 7f2b37fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b38159890 0x7f2b380b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:25.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.263+0000 7f2b37fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b38159890 0x7f2b380b4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42178/0 (socket says 172.21.15.163:42178) 2023-10-09T06:38:25.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.263+0000 7f2b37fff700 1 -- 172.21.15.163:0/1344827139 learned_addr learned my addr 172.21.15.163:0/1344827139 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:25.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.263+0000 7f2b37fff700 1 -- 172.21.15.163:0/1344827139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b3815db10 msgr2=0x7f2b380b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:25.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.263+0000 7f2b37fff700 1 --2- 172.21.15.163:0/1344827139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b3815db10 0x7f2b380b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:25.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.263+0000 7f2b37fff700 1 -- 172.21.15.163:0/1344827139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b38159c60 msgr2=0x7f2b380b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:25.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.263+0000 7f2b37fff700 1 --2- 172.21.15.163:0/1344827139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b38159c60 0x7f2b380b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:25.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.263+0000 7f2b37fff700 1 -- 172.21.15.163:0/1344827139 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b30009ae0 con 0x7f2b38159890 2023-10-09T06:38:25.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.263+0000 7f2b37fff700 1 --2- 172.21.15.163:0/1344827139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b38159890 0x7f2b380b4940 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f2b280011e0 tx=0x7f2b28007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:25.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.263+0000 7f2b1ffff700 1 -- 172.21.15.163:0/1344827139 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b28007300 con 0x7f2b38159890 2023-10-09T06:38:25.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.263+0000 7f2b1ffff700 1 -- 172.21.15.163:0/1344827139 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b28007460 con 0x7f2b38159890 2023-10-09T06:38:25.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.263+0000 7f2b3e766700 1 -- 172.21.15.163:0/1344827139 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b3816a8c0 con 0x7f2b38159890 2023-10-09T06:38:25.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.263+0000 7f2b1ffff700 1 -- 172.21.15.163:0/1344827139 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b28004830 con 0x7f2b38159890 2023-10-09T06:38:25.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.263+0000 7f2b3e766700 1 -- 172.21.15.163:0/1344827139 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b380b4e80 con 0x7f2b38159890 2023-10-09T06:38:25.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.264+0000 7f2b3e766700 1 -- 172.21.15.163:0/1344827139 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b3804c3c0 con 0x7f2b38159890 2023-10-09T06:38:25.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.264+0000 7f2b1ffff700 1 -- 172.21.15.163:0/1344827139 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2b28004be0 con 0x7f2b38159890 2023-10-09T06:38:25.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.265+0000 7f2b1ffff700 1 --2- 172.21.15.163:0/1344827139 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2b24067320 0x7f2b240697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:25.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.265+0000 7f2b1ffff700 1 -- 172.21.15.163:0/1344827139 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2b28084180 con 0x7f2b38159890 2023-10-09T06:38:25.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.265+0000 7f2b377fe700 1 --2- 172.21.15.163:0/1344827139 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2b24067320 0x7f2b240697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:25.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.266+0000 7f2b377fe700 1 --2- 172.21.15.163:0/1344827139 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2b24067320 0x7f2b240697d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2b380c6be0 tx=0x7f2b2c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:25.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.269+0000 7f2b1ffff700 1 -- 172.21.15.163:0/1344827139 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2b28050e90 con 0x7f2b38159890 2023-10-09T06:38:25.448 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.447+0000 7f2b3e766700 1 -- 172.21.15.163:0/1344827139 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2b380d1730 con 0x7f2b24067320 2023-10-09T06:38:25.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.456+0000 7f2b1ffff700 1 -- 172.21.15.163:0/1344827139 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287181 (secure 0 0 0) 0x7f2b380d1730 con 0x7f2b24067320 2023-10-09T06:38:25.459 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:25.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.459+0000 7f2b3e766700 1 -- 172.21.15.163:0/1344827139 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2b24067320 msgr2=0x7f2b240697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:25.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.459+0000 7f2b3e766700 1 --2- 172.21.15.163:0/1344827139 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2b24067320 0x7f2b240697d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2b380c6be0 tx=0x7f2b2c008040 comp rx=0 tx=0).stop 2023-10-09T06:38:25.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.459+0000 7f2b3e766700 1 -- 172.21.15.163:0/1344827139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b38159890 msgr2=0x7f2b380b4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:25.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.459+0000 7f2b3e766700 1 --2- 172.21.15.163:0/1344827139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b38159890 0x7f2b380b4940 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f2b280011e0 tx=0x7f2b28007e10 comp rx=0 tx=0).stop 2023-10-09T06:38:25.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.459+0000 7f2b3e766700 1 -- 172.21.15.163:0/1344827139 shutdown_connections 2023-10-09T06:38:25.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.459+0000 7f2b3e766700 1 --2- 172.21.15.163:0/1344827139 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2b24067320 0x7f2b240697d0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:25.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.459+0000 7f2b3e766700 1 --2- 172.21.15.163:0/1344827139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b38159890 0x7f2b380b4940 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:25.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.459+0000 7f2b3e766700 1 --2- 172.21.15.163:0/1344827139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b3815db10 0x7f2b380b20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:25.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.459+0000 7f2b3e766700 1 --2- 172.21.15.163:0/1344827139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b38159c60 0x7f2b380b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:25.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.459+0000 7f2b3e766700 1 -- 172.21.15.163:0/1344827139 >> 172.21.15.163:0/1344827139 conn(0x7f2b38068e20 msgr2=0x7f2b380c7130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:25.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.459+0000 7f2b3e766700 1 -- 172.21.15.163:0/1344827139 shutdown_connections 2023-10-09T06:38:25.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.459+0000 7f2b3e766700 1 -- 172.21.15.163:0/1344827139 wait complete. 2023-10-09T06:38:25.463 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:38:25.472 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2023-10-09T06:38:24.410910+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2427056,"kb_used_data":15984,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752547664,"statfs":{"total":773094113280,"available":770608807936,"internally_reserved":0,"allocated":16367616,"data_stored":7948834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.742617"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.687243+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:15:57.542772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.686956+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:10:38.081403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.681576+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:11:37.888420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.685055+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:58:03.322193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678873+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:41:30.382056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678481+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:36:50.516609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682433+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:46:35.635953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.676958+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:58:03.322193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.688589+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:20:02.974752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678660+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:46:35.635953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.680844+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:25:57.096853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678704+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:48:51.565773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.681087+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:25:53.616204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.683792+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:34:55.917445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678799+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:06:23.000411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677797+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:06:23.000411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682655+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:44:35.145850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678258+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:44:51.812594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.679357+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:44:35.145850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.676535+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:16:26.649802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.683129+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:38:53.437170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678996+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:19:50.331458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.681279+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:21:46.727517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682901+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:26:55.538336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.687816+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:08:34.030754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677830+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:01:29.075717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.679264+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:29:07.591719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.681628+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:49:58.537930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.680535+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:46:34.855757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.687098+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:46:34.855757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.676565+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:58:03.322193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677453+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:08:26.202100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.688987+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:42:48.250267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.691611+0000","last_change":"2023-10-09T06:38:19.691611+0000","last_active":"2023-10-09T06:38:19.691611+0000","last_peered":"2023-10-09T06:38:19.691611+0000","last_clean":"2023-10-09T06:38:19.691611+0000","last_became_active":"2023-10-09T06:38:19.691367+0000","last_became_peered":"2023-10-09T06:38:19.691367+0000","last_unstale":"2023-10-09T06:38:19.691611+0000","last_undegraded":"2023-10-09T06:38:19.691611+0000","last_fullsized":"2023-10-09T06:38:19.691611+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:13:34.033630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.679700+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:15:52.880306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.694250+0000","last_change":"2023-10-09T06:38:19.694250+0000","last_active":"2023-10-09T06:38:19.694250+0000","last_peered":"2023-10-09T06:38:19.694250+0000","last_clean":"2023-10-09T06:38:19.694250+0000","last_became_active":"2023-10-09T06:38:19.693478+0000","last_became_peered":"2023-10-09T06:38:19.693478+0000","last_unstale":"2023-10-09T06:38:19.694250+0000","last_undegraded":"2023-10-09T06:38:19.694250+0000","last_fullsized":"2023-10-09T06:38:19.694250+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:50:27.283148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677343+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:52:50.901568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.688925+0000","last_change":"2023-10-09T06:38:19.688925+0000","last_active":"2023-10-09T06:38:19.688925+0000","last_peered":"2023-10-09T06:38:19.688925+0000","last_clean":"2023-10-09T06:38:19.688925+0000","last_became_active":"2023-10-09T06:38:19.688479+0000","last_became_peered":"2023-10-09T06:38:19.688479+0000","last_unstale":"2023-10-09T06:38:19.688925+0000","last_undegraded":"2023-10-09T06:38:19.688925+0000","last_fullsized":"2023-10-09T06:38:19.688925+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:32:15.314337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677169+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:36:50.516609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.692752+0000","last_change":"2023-10-09T06:38:19.692752+0000","last_active":"2023-10-09T06:38:19.692752+0000","last_peered":"2023-10-09T06:38:19.692752+0000","last_clean":"2023-10-09T06:38:19.692752+0000","last_became_active":"2023-10-09T06:38:19.692249+0000","last_became_peered":"2023-10-09T06:38:19.692249+0000","last_unstale":"2023-10-09T06:38:19.692752+0000","last_undegraded":"2023-10-09T06:38:19.692752+0000","last_fullsized":"2023-10-09T06:38:19.692752+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:52:11.581942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682687+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:40:22.810227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.690895+0000","last_change":"2023-10-09T06:38:19.690895+0000","last_active":"2023-10-09T06:38:19.690895+0000","last_peered":"2023-10-09T06:38:19.690895+0000","last_clean":"2023-10-09T06:38:19.690895+0000","last_became_active":"2023-10-09T06:38:19.690633+0000","last_became_peered":"2023-10-09T06:38:19.690633+0000","last_unstale":"2023-10-09T06:38:19.690895+0000","last_undegraded":"2023-10-09T06:38:19.690895+0000","last_fullsized":"2023-10-09T06:38:19.690895+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:26:20.360953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.676460+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:08:26.202100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.687201+0000","last_change":"2023-10-09T06:38:19.687201+0000","last_active":"2023-10-09T06:38:19.687201+0000","last_peered":"2023-10-09T06:38:19.687201+0000","last_clean":"2023-10-09T06:38:19.687201+0000","last_became_active":"2023-10-09T06:38:19.686979+0000","last_became_peered":"2023-10-09T06:38:19.686979+0000","last_unstale":"2023-10-09T06:38:19.687201+0000","last_undegraded":"2023-10-09T06:38:19.687201+0000","last_fullsized":"2023-10-09T06:38:19.687201+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:04:37.633413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682248+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:22:26.955724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.690010+0000","last_change":"2023-10-09T06:38:19.690010+0000","last_active":"2023-10-09T06:38:19.690010+0000","last_peered":"2023-10-09T06:38:19.690010+0000","last_clean":"2023-10-09T06:38:19.690010+0000","last_became_active":"2023-10-09T06:38:19.689749+0000","last_became_peered":"2023-10-09T06:38:19.689749+0000","last_unstale":"2023-10-09T06:38:19.690010+0000","last_undegraded":"2023-10-09T06:38:19.690010+0000","last_fullsized":"2023-10-09T06:38:19.690010+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:08:33.020051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682331+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:26:21.371656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.693531+0000","last_change":"2023-10-09T06:38:19.693531+0000","last_active":"2023-10-09T06:38:19.693531+0000","last_peered":"2023-10-09T06:38:19.693531+0000","last_clean":"2023-10-09T06:38:19.693531+0000","last_became_active":"2023-10-09T06:38:19.693246+0000","last_became_peered":"2023-10-09T06:38:19.693246+0000","last_unstale":"2023-10-09T06:38:19.693531+0000","last_undegraded":"2023-10-09T06:38:19.693531+0000","last_fullsized":"2023-10-09T06:38:19.693531+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:26:54.527634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.680775+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:50:10.718730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.694122+0000","last_change":"2023-10-09T06:38:19.694122+0000","last_active":"2023-10-09T06:38:19.694122+0000","last_peered":"2023-10-09T06:38:19.694122+0000","last_clean":"2023-10-09T06:38:19.694122+0000","last_became_active":"2023-10-09T06:38:19.693847+0000","last_became_peered":"2023-10-09T06:38:19.693847+0000","last_unstale":"2023-10-09T06:38:19.694122+0000","last_undegraded":"2023-10-09T06:38:19.694122+0000","last_fullsized":"2023-10-09T06:38:19.694122+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:50:50.722486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.683619+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:52:50.901568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.688209+0000","last_change":"2023-10-09T06:38:19.688209+0000","last_active":"2023-10-09T06:38:19.688209+0000","last_peered":"2023-10-09T06:38:19.688209+0000","last_clean":"2023-10-09T06:38:19.688209+0000","last_became_active":"2023-10-09T06:38:19.687961+0000","last_became_peered":"2023-10-09T06:38:19.687961+0000","last_unstale":"2023-10-09T06:38:19.688209+0000","last_undegraded":"2023-10-09T06:38:19.688209+0000","last_fullsized":"2023-10-09T06:38:19.688209+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:52:49.890865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678258+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:36:50.516609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.693838+0000","last_change":"2023-10-09T06:38:19.693838+0000","last_active":"2023-10-09T06:38:19.693838+0000","last_peered":"2023-10-09T06:38:19.693838+0000","last_clean":"2023-10-09T06:38:19.693838+0000","last_became_active":"2023-10-09T06:38:19.693584+0000","last_became_peered":"2023-10-09T06:38:19.693584+0000","last_unstale":"2023-10-09T06:38:19.693838+0000","last_undegraded":"2023-10-09T06:38:19.693838+0000","last_fullsized":"2023-10-09T06:38:19.693838+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:42:55.756537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677099+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:01:29.075717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.684840+0000","last_change":"2023-10-09T06:38:19.684840+0000","last_active":"2023-10-09T06:38:19.684840+0000","last_peered":"2023-10-09T06:38:19.684840+0000","last_clean":"2023-10-09T06:38:19.684840+0000","last_became_active":"2023-10-09T06:38:19.684531+0000","last_became_peered":"2023-10-09T06:38:19.684531+0000","last_unstale":"2023-10-09T06:38:19.684840+0000","last_undegraded":"2023-10-09T06:38:19.684840+0000","last_fullsized":"2023-10-09T06:38:19.684840+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:42:47.239564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677625+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:41:30.382056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.692513+0000","last_change":"2023-10-09T06:38:19.692513+0000","last_active":"2023-10-09T06:38:19.692513+0000","last_peered":"2023-10-09T06:38:19.692513+0000","last_clean":"2023-10-09T06:38:19.692513+0000","last_became_active":"2023-10-09T06:38:19.692237+0000","last_became_peered":"2023-10-09T06:38:19.692237+0000","last_unstale":"2023-10-09T06:38:19.692513+0000","last_undegraded":"2023-10-09T06:38:19.692513+0000","last_fullsized":"2023-10-09T06:38:19.692513+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:56:50.090947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677087+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:21:10.321869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.681581+0000","last_change":"2023-10-09T06:38:19.681581+0000","last_active":"2023-10-09T06:38:19.681581+0000","last_peered":"2023-10-09T06:38:19.681581+0000","last_clean":"2023-10-09T06:38:19.681581+0000","last_became_active":"2023-10-09T06:38:19.681316+0000","last_became_peered":"2023-10-09T06:38:19.681316+0000","last_unstale":"2023-10-09T06:38:19.681581+0000","last_undegraded":"2023-10-09T06:38:19.681581+0000","last_fullsized":"2023-10-09T06:38:19.681581+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:38:52.426468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677747+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:16:26.649802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.688731+0000","last_change":"2023-10-09T06:38:19.688731+0000","last_active":"2023-10-09T06:38:19.688731+0000","last_peered":"2023-10-09T06:38:19.688731+0000","last_clean":"2023-10-09T06:38:19.688731+0000","last_became_active":"2023-10-09T06:38:19.688486+0000","last_became_peered":"2023-10-09T06:38:19.688486+0000","last_unstale":"2023-10-09T06:38:19.688731+0000","last_undegraded":"2023-10-09T06:38:19.688731+0000","last_fullsized":"2023-10-09T06:38:19.688731+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:12:32.876485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.687784+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:57:12.115891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.683035+0000","last_change":"2023-10-09T06:38:19.683035+0000","last_active":"2023-10-09T06:38:19.683035+0000","last_peered":"2023-10-09T06:38:19.683035+0000","last_clean":"2023-10-09T06:38:19.683035+0000","last_became_active":"2023-10-09T06:38:19.682754+0000","last_became_peered":"2023-10-09T06:38:19.682754+0000","last_unstale":"2023-10-09T06:38:19.683035+0000","last_undegraded":"2023-10-09T06:38:19.683035+0000","last_fullsized":"2023-10-09T06:38:19.683035+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:40:21.799524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.687350+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:40:00.965033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.694515+0000","last_change":"2023-10-09T06:38:19.694515+0000","last_active":"2023-10-09T06:38:19.694515+0000","last_peered":"2023-10-09T06:38:19.694515+0000","last_clean":"2023-10-09T06:38:19.694515+0000","last_became_active":"2023-10-09T06:38:19.694041+0000","last_became_peered":"2023-10-09T06:38:19.694041+0000","last_unstale":"2023-10-09T06:38:19.694515+0000","last_undegraded":"2023-10-09T06:38:19.694515+0000","last_fullsized":"2023-10-09T06:38:19.694515+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:40:48.522136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.681184+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:59:57.861748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.690795+0000","last_change":"2023-10-09T06:38:19.690795+0000","last_active":"2023-10-09T06:38:19.690795+0000","last_peered":"2023-10-09T06:38:19.690795+0000","last_clean":"2023-10-09T06:38:19.690795+0000","last_became_active":"2023-10-09T06:38:19.690512+0000","last_became_peered":"2023-10-09T06:38:19.690512+0000","last_unstale":"2023-10-09T06:38:19.690795+0000","last_undegraded":"2023-10-09T06:38:19.690795+0000","last_fullsized":"2023-10-09T06:38:19.690795+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:40:21.799524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.676742+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:01:29.075717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.688034+0000","last_change":"2023-10-09T06:38:19.688034+0000","last_active":"2023-10-09T06:38:19.688034+0000","last_peered":"2023-10-09T06:38:19.688034+0000","last_clean":"2023-10-09T06:38:19.688034+0000","last_became_active":"2023-10-09T06:38:19.687764+0000","last_became_peered":"2023-10-09T06:38:19.687764+0000","last_unstale":"2023-10-09T06:38:19.688034+0000","last_undegraded":"2023-10-09T06:38:19.688034+0000","last_fullsized":"2023-10-09T06:38:19.688034+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:37:57.642710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.676045+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:16:26.649802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.686556+0000","last_change":"2023-10-09T06:38:19.686556+0000","last_active":"2023-10-09T06:38:19.686556+0000","last_peered":"2023-10-09T06:38:19.686556+0000","last_clean":"2023-10-09T06:38:19.686556+0000","last_became_active":"2023-10-09T06:38:19.686253+0000","last_became_peered":"2023-10-09T06:38:19.686253+0000","last_unstale":"2023-10-09T06:38:19.686556+0000","last_undegraded":"2023-10-09T06:38:19.686556+0000","last_fullsized":"2023-10-09T06:38:19.686556+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:26:54.527634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677131+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:16:34.306892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.693762+0000","last_change":"2023-10-09T06:38:19.693762+0000","last_active":"2023-10-09T06:38:19.693762+0000","last_peered":"2023-10-09T06:38:19.693762+0000","last_clean":"2023-10-09T06:38:19.693762+0000","last_became_active":"2023-10-09T06:38:19.693501+0000","last_became_peered":"2023-10-09T06:38:19.693501+0000","last_unstale":"2023-10-09T06:38:19.693762+0000","last_undegraded":"2023-10-09T06:38:19.693762+0000","last_fullsized":"2023-10-09T06:38:19.693762+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:26:54.527634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.683097+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:48:51.565773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.691160+0000","last_change":"2023-10-09T06:38:19.691160+0000","last_active":"2023-10-09T06:38:19.691160+0000","last_peered":"2023-10-09T06:38:19.691160+0000","last_clean":"2023-10-09T06:38:19.691160+0000","last_became_active":"2023-10-09T06:38:19.690880+0000","last_became_peered":"2023-10-09T06:38:19.690880+0000","last_unstale":"2023-10-09T06:38:19.691160+0000","last_undegraded":"2023-10-09T06:38:19.691160+0000","last_fullsized":"2023-10-09T06:38:19.691160+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:40:21.799524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.679938+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:43:32.088320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.684011+0000","last_change":"2023-10-09T06:38:19.684011+0000","last_active":"2023-10-09T06:38:19.684011+0000","last_peered":"2023-10-09T06:38:19.684011+0000","last_clean":"2023-10-09T06:38:19.684011+0000","last_became_active":"2023-10-09T06:38:19.683541+0000","last_became_peered":"2023-10-09T06:38:19.683541+0000","last_unstale":"2023-10-09T06:38:19.684011+0000","last_undegraded":"2023-10-09T06:38:19.684011+0000","last_fullsized":"2023-10-09T06:38:19.684011+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:54:42.729290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.679867+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:32:06.438731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.691307+0000","last_change":"2023-10-09T06:38:19.691307+0000","last_active":"2023-10-09T06:38:19.691307+0000","last_peered":"2023-10-09T06:38:19.691307+0000","last_clean":"2023-10-09T06:38:19.691307+0000","last_became_active":"2023-10-09T06:38:19.691050+0000","last_became_peered":"2023-10-09T06:38:19.691050+0000","last_unstale":"2023-10-09T06:38:19.691307+0000","last_undegraded":"2023-10-09T06:38:19.691307+0000","last_fullsized":"2023-10-09T06:38:19.691307+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:15:20.738142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677792+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:58:03.322193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.691686+0000","last_change":"2023-10-09T06:38:19.691686+0000","last_active":"2023-10-09T06:38:19.691686+0000","last_peered":"2023-10-09T06:38:19.691686+0000","last_clean":"2023-10-09T06:38:19.691686+0000","last_became_active":"2023-10-09T06:38:19.690753+0000","last_became_peered":"2023-10-09T06:38:19.690753+0000","last_unstale":"2023-10-09T06:38:19.691686+0000","last_undegraded":"2023-10-09T06:38:19.691686+0000","last_fullsized":"2023-10-09T06:38:19.691686+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:04:37.633413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.683688+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:29:07.591719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.687181+0000","last_change":"2023-10-09T06:38:19.687181+0000","last_active":"2023-10-09T06:38:19.687181+0000","last_peered":"2023-10-09T06:38:19.687181+0000","last_clean":"2023-10-09T06:38:19.687181+0000","last_became_active":"2023-10-09T06:38:19.686940+0000","last_became_peered":"2023-10-09T06:38:19.686940+0000","last_unstale":"2023-10-09T06:38:19.687181+0000","last_undegraded":"2023-10-09T06:38:19.687181+0000","last_fullsized":"2023-10-09T06:38:19.687181+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:52:11.581942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.680141+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:31:22.541312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.693786+0000","last_change":"2023-10-09T06:38:19.693786+0000","last_active":"2023-10-09T06:38:19.693786+0000","last_peered":"2023-10-09T06:38:19.693786+0000","last_clean":"2023-10-09T06:38:19.693786+0000","last_became_active":"2023-10-09T06:38:19.693529+0000","last_became_peered":"2023-10-09T06:38:19.693529+0000","last_unstale":"2023-10-09T06:38:19.693786+0000","last_undegraded":"2023-10-09T06:38:19.693786+0000","last_fullsized":"2023-10-09T06:38:19.693786+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:09:07.415429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678527+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:41:30.382056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.690991+0000","last_change":"2023-10-09T06:38:19.690991+0000","last_active":"2023-10-09T06:38:19.690991+0000","last_peered":"2023-10-09T06:38:19.690991+0000","last_clean":"2023-10-09T06:38:19.690991+0000","last_became_active":"2023-10-09T06:38:19.690659+0000","last_became_peered":"2023-10-09T06:38:19.690659+0000","last_unstale":"2023-10-09T06:38:19.690991+0000","last_undegraded":"2023-10-09T06:38:19.690991+0000","last_fullsized":"2023-10-09T06:38:19.690991+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:02:57.917372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.680109+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:10:38.081403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.684393+0000","last_change":"2023-10-09T06:38:19.684393+0000","last_active":"2023-10-09T06:38:19.684393+0000","last_peered":"2023-10-09T06:38:19.684393+0000","last_clean":"2023-10-09T06:38:19.684393+0000","last_became_active":"2023-10-09T06:38:19.683050+0000","last_became_peered":"2023-10-09T06:38:19.683050+0000","last_unstale":"2023-10-09T06:38:19.684393+0000","last_undegraded":"2023-10-09T06:38:19.684393+0000","last_fullsized":"2023-10-09T06:38:19.684393+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:22:25.945021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.676616+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:49:51.877549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.685457+0000","last_change":"2023-10-09T06:38:19.685457+0000","last_active":"2023-10-09T06:38:19.685457+0000","last_peered":"2023-10-09T06:38:19.685457+0000","last_clean":"2023-10-09T06:38:19.685457+0000","last_became_active":"2023-10-09T06:38:19.685181+0000","last_became_peered":"2023-10-09T06:38:19.685181+0000","last_unstale":"2023-10-09T06:38:19.685457+0000","last_undegraded":"2023-10-09T06:38:19.685457+0000","last_fullsized":"2023-10-09T06:38:19.685457+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:39:59.954330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682501+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:05:32.721283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.692581+0000","last_change":"2023-10-09T06:38:19.692581+0000","last_active":"2023-10-09T06:38:19.692581+0000","last_peered":"2023-10-09T06:38:19.692581+0000","last_clean":"2023-10-09T06:38:19.692581+0000","last_became_active":"2023-10-09T06:38:19.692312+0000","last_became_peered":"2023-10-09T06:38:19.692312+0000","last_unstale":"2023-10-09T06:38:19.692581+0000","last_undegraded":"2023-10-09T06:38:19.692581+0000","last_fullsized":"2023-10-09T06:38:19.692581+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:52:30.325627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682069+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:44:51.812594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.683671+0000","last_change":"2023-10-09T06:38:19.683671+0000","last_active":"2023-10-09T06:38:19.683671+0000","last_peered":"2023-10-09T06:38:19.683671+0000","last_clean":"2023-10-09T06:38:19.683671+0000","last_became_active":"2023-10-09T06:38:19.683355+0000","last_became_peered":"2023-10-09T06:38:19.683355+0000","last_unstale":"2023-10-09T06:38:19.683671+0000","last_undegraded":"2023-10-09T06:38:19.683671+0000","last_fullsized":"2023-10-09T06:38:19.683671+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:23:57.642507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":270,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-09T06:38:18.676452+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:38:18.676452+0000","last_peered":"2023-10-09T06:38:18.676452+0000","last_clean":"2023-10-09T06:38:18.676452+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:38:18.676452+0000","last_undegraded":"2023-10-09T06:38:18.676452+0000","last_fullsized":"2023-10-09T06:38:18.676452+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574946,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303820,"kb_used_data":2436,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068020,"statfs":{"total":96636764160,"available":96325652480,"internally_reserved":0,"allocated":2494464,"data_stored":1437946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52000000000000002,"15min":0.50700000000000001},"min":{"1min":0.34799999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.37,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67600000000000005,"5min":0.753,"15min":0.753},"last":0.52900000000000003}]},{"osd":1,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.503,"15min":0.51900000000000002},"min":{"1min":0.33600000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.74299999999999999,"5min":0.77400000000000002,"15min":1.2030000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.378,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.57899999999999996}]},{"osd":2,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53200000000000003,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75800000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.44,"5min":0.36399999999999999,"15min":0.315},"max":{"1min":0.77900000000000003,"5min":0.80700000000000005,"15min":0.83099999999999996},"last":0.56499999999999995}]},{"osd":3,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53800000000000003,"15min":0.55000000000000004},"min":{"1min":0.41199999999999998,"5min":0.35899999999999999,"15min":0.33500000000000002},"max":{"1min":0.68600000000000005,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55600000000000005,"15min":0.59199999999999997},"min":{"1min":0.45300000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.754,"5min":0.82099999999999995,"15min":1.1479999999999999},"last":0.50900000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.48199999999999998,"15min":0.51900000000000002},"min":{"1min":0.35599999999999998,"5min":0.30299999999999999,"15min":0.24399999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.93999999999999995},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56599999999999995,"15min":0.60599999999999998},"min":{"1min":0.36799999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76700000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.59199999999999997}]},{"osd":5,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52500000000000002,"15min":0.56999999999999995},"min":{"1min":0.36499999999999999,"5min":0.32500000000000001,"15min":0.307},"max":{"1min":0.81999999999999995,"5min":0.85799999999999998,"15min":0.91100000000000003},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59599999999999997,"15min":0.62},"min":{"1min":0.52900000000000003,"5min":0.41699999999999998,"15min":0.34200000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.621}]},{"osd":6,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55700000000000005,"15min":0.59299999999999997},"min":{"1min":0.38600000000000001,"5min":0.33500000000000002,"15min":0.28699999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.88600000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.56100000000000005,"15min":0.61399999999999999},"min":{"1min":0.45100000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.84599999999999997},"last":0.64300000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574945,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303228,"kb_used_data":1844,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068612,"statfs":{"total":96636764160,"available":96326258688,"internally_reserved":0,"allocated":1888256,"data_stored":841315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.50800000000000001,"15min":0.502},"min":{"1min":0.35499999999999998,"5min":0.32000000000000001,"15min":0.30399999999999999},"max":{"1min":0.66500000000000004,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.50700000000000001,"15min":0.51000000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70399999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.51300000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.502,"15min":0.51200000000000001},"min":{"1min":0.33600000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68300000000000005,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.5},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52900000000000003,"15min":0.53200000000000003},"min":{"1min":0.38700000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.77100000000000002,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.60299999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.34499999999999997,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.72399999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51100000000000001,"15min":0.52900000000000003},"min":{"1min":0.39000000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.66600000000000004,"5min":0.67500000000000004,"15min":0.752},"last":0.44500000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.38900000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66600000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.54300000000000004,"15min":0.68799999999999994},"min":{"1min":0.40400000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.752,"5min":0.752,"15min":4.7510000000000003},"last":0.53600000000000003}]},{"osd":4,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46400000000000002,"15min":0.47699999999999998},"min":{"1min":0.29899999999999999,"5min":0.28199999999999997,"15min":0.27900000000000003},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.76400000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56100000000000005,"15min":0.58099999999999996},"min":{"1min":0.41799999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.77200000000000002,"5min":0.81299999999999994,"15min":0.83699999999999997},"last":0.66500000000000004}]},{"osd":5,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.52100000000000002,"15min":0.53000000000000003},"min":{"1min":0.31900000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.66100000000000003,"5min":0.79200000000000004,"15min":0.81200000000000006},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.59499999999999997},"min":{"1min":0.46899999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76000000000000001,"5min":0.78000000000000003,"15min":0.86599999999999999},"last":0.58999999999999997}]},{"osd":7,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55200000000000005,"15min":0.58199999999999996},"min":{"1min":0.41399999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80500000000000005,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58799999999999997,"15min":0.58899999999999997},"min":{"1min":0.46000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78400000000000003,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.57399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":847578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.316,"15min":0.311},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.503,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.34899999999999998,"15min":0.35799999999999998},"min":{"1min":0.28399999999999997,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.51900000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.29599999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.34399999999999997,"15min":0.35099999999999998},"min":{"1min":0.215,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.53700000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.38300000000000001,"15min":0.38500000000000001},"min":{"1min":0.30299999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.55300000000000005,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.34399999999999997}]},{"osd":3,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.36499999999999999,"15min":0.36399999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.55800000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.34599999999999997,"15min":0.53500000000000003},"min":{"1min":0.23400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.40400000000000003,"5min":0.56000000000000005,"15min":5.8529999999999998},"last":0.39200000000000002}]},{"osd":4,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58299999999999996,"15min":0.57499999999999996},"min":{"1min":0.36199999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78300000000000003,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.40000000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.81100000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.64200000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57699999999999996,"15min":0.74099999999999999},"min":{"1min":0.36499999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88,"5min":0.88,"15min":5.9459999999999997},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61199999999999999,"15min":0.61099999999999999},"min":{"1min":0.38900000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82599999999999996,"5min":0.92700000000000005,"15min":0.93999999999999995},"last":0.66400000000000003}]},{"osd":6,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54900000000000004,"15min":0.71199999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79100000000000004,"5min":0.79600000000000004,"15min":5.9160000000000004},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.379,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84799999999999998,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.61299999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57199999999999995,"15min":0.58599999999999997},"min":{"1min":0.39000000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.80900000000000005,"5min":0.89500000000000002,"15min":0.97099999999999997},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57099999999999995,"15min":0.749},"min":{"1min":0.41399999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":5.9729999999999999},"last":0.60499999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574947,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303808,"kb_used_data":2424,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068032,"statfs":{"total":96636764160,"available":96325664768,"internally_reserved":0,"allocated":2482176,"data_stored":1431683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.30199999999999999,"15min":0.315},"min":{"1min":0.216,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.433,"5min":0.498,"15min":0.498},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.32100000000000001,"15min":0.34399999999999997},"min":{"1min":0.222,"5min":0.214,"15min":0.214},"max":{"1min":0.497,"5min":0.53600000000000003,"15min":0.94499999999999995},"last":0.34899999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35199999999999998,"15min":0.36199999999999999},"min":{"1min":0.25900000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.50900000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.371,"15min":0.38800000000000001},"min":{"1min":0.30499999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.56499999999999995,"5min":0.69399999999999995,"15min":0.91700000000000004},"last":0.47099999999999997}]},{"osd":3,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.32800000000000001,"15min":0.38400000000000001},"min":{"1min":0.23699999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":1.554},"last":0.499},{"interface":"front","average":{"1min":0.371,"5min":0.35999999999999999,"15min":0.39700000000000002},"min":{"1min":0.27700000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54400000000000004,"5min":0.58099999999999996,"15min":0.79000000000000004},"last":0.311}]},{"osd":4,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53600000000000003,"15min":0.55100000000000005},"min":{"1min":0.41199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":1.212},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.72099999999999997},"min":{"1min":0.42199999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":5.484},"last":0.77800000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.72499999999999998},"min":{"1min":0.41999999999999998,"5min":0.35799999999999998,"15min":0.35499999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":5.5449999999999999},"last":0.627},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.83299999999999996,"5min":0.84499999999999997,"15min":1.113},"last":0.52400000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":1.0529999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59299999999999997,"15min":0.60699999999999998},"min":{"1min":0.45800000000000002,"5min":0.42799999999999999,"15min":0.41599999999999998},"max":{"1min":0.878,"5min":0.878,"15min":1.169},"last":0.61099999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57099999999999995,"15min":0.58899999999999997},"min":{"1min":0.40600000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.78700000000000003,"5min":0.85499999999999998,"15min":1.2390000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56399999999999995,"15min":0.749},"min":{"1min":0.38,"5min":0.375,"15min":0.375},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":5.5800000000000001},"last":0.55800000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574948,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":847578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.316,"15min":0.35299999999999998},"min":{"1min":0.23200000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":1.4199999999999999},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.55900000000000005},"min":{"1min":0.27500000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.51600000000000001,"5min":0.53800000000000003,"15min":7.2699999999999996},"last":0.27800000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.317,"15min":0.34399999999999997},"min":{"1min":0.23999999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.65400000000000003},"last":0.25},{"interface":"front","average":{"1min":0.372,"5min":0.37,"15min":0.61299999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":7.327},"last":0.313}]},{"osd":3,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.36799999999999999,"15min":0.39100000000000001},"min":{"1min":0.29399999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.68600000000000005},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.36799999999999999,"15min":0.39200000000000002},"min":{"1min":0.27900000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.72699999999999998},"last":0.30199999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54500000000000004,"15min":0.54400000000000004},"min":{"1min":0.40799999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.78400000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56200000000000006,"15min":0.78600000000000003},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82699999999999996,"5min":0.84699999999999998,"15min":7.2389999999999999},"last":0.35999999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.58099999999999996,"15min":0.91500000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82699999999999996,"5min":0.83799999999999997,"15min":10.890000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":0.84299999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.51000000000000001}]},{"osd":6,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56399999999999995,"15min":0.79100000000000004},"min":{"1min":0.42999999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.748,"5min":0.77300000000000002,"15min":7.2999999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.45800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.433}]},{"osd":7,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59299999999999997,"15min":0.82199999999999995},"min":{"1min":0.47499999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":7.1879999999999997},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60399999999999998,"15min":0.624},"min":{"1min":0.49399999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.90500000000000003},"last":0.5}]}]},{"osd":3,"up_from":13,"seq":55834574947,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":847578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.32200000000000001,"15min":0.33200000000000002},"min":{"1min":0.23100000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":5.8700000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.34699999999999998,"15min":0.36699999999999999},"min":{"1min":0.23999999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":5.6769999999999996},"last":0.25600000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.35799999999999998,"15min":0.36099999999999999},"min":{"1min":0.26800000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":4.0469999999999997},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.372,"15min":0.38800000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":4.1509999999999998},"last":0.34799999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.35199999999999998,"15min":0.374},"min":{"1min":0.29999999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.50800000000000001,"5min":0.56799999999999995,"15min":3.1139999999999999},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.373,"15min":0.38600000000000001},"min":{"1min":0.28100000000000003,"5min":0.26100000000000001,"15min":0.25600000000000001},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":3.3180000000000001},"last":0.309}]},{"osd":4,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55800000000000005,"15min":0.57099999999999995},"min":{"1min":0.38100000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":3.0899999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56499999999999995,"15min":0.56799999999999995},"min":{"1min":0.40300000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76400000000000001,"5min":0.91100000000000003,"15min":3.0710000000000002},"last":0.443}]},{"osd":5,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57499999999999996,"15min":0.58899999999999997},"min":{"1min":0.39100000000000001,"5min":0.34599999999999997,"15min":0.32700000000000001},"max":{"1min":0.877,"5min":0.877,"15min":6.2300000000000004},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.441,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79800000000000004,"5min":0.89500000000000002,"15min":3.0219999999999998},"last":0.505}]},{"osd":6,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.44500000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":2.6850000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.65200000000000002},"min":{"1min":0.43099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":6.5970000000000004},"last":0.55000000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60699999999999998,"15min":0.61399999999999999},"min":{"1min":0.44700000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86099999999999999,"5min":0.93300000000000005,"15min":4.2629999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59999999999999998,"15min":0.61299999999999999},"min":{"1min":0.40500000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.82899999999999996,"5min":0.97199999999999998,"15min":7.3650000000000002},"last":0.56299999999999994}]}]},{"osd":4,"up_from":13,"seq":55834574946,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":847578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.51400000000000001,"15min":0.55300000000000005},"min":{"1min":0.318,"5min":0.28100000000000003,"15min":0.27000000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":1.083},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.35699999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.79900000000000004,"5min":0.97799999999999998,"15min":1.054},"last":0.52400000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.39900000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.501,"5min":0.50700000000000001,"15min":0.49399999999999999},"min":{"1min":0.33800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.76300000000000001,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.505}]},{"osd":2,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52700000000000002,"15min":0.53900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.32900000000000001},"max":{"1min":0.76600000000000001,"5min":0.82099999999999995,"15min":1.0249999999999999},"last":0.443},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55700000000000005,"15min":0.59099999999999997},"min":{"1min":0.39400000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.93799999999999994},"last":0.41999999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52900000000000003,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.32200000000000001,"15min":0.29199999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.98499999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57699999999999996,"15min":0.58799999999999997},"min":{"1min":0.38700000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.1319999999999999},"last":0.48399999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51600000000000001,"15min":0.55100000000000005},"min":{"1min":0.31900000000000001,"5min":0.30399999999999999,"15min":0.27300000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58199999999999996,"15min":0.60599999999999998},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.35099999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.97999999999999998},"last":0.54300000000000004}]},{"osd":6,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.56799999999999995},"min":{"1min":0.41599999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.76800000000000002,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.59799999999999998},"min":{"1min":0.38700000000000001,"5min":0.34999999999999998,"15min":0.32700000000000001},"max":{"1min":0.85599999999999998,"5min":1.01,"15min":1.01},"last":0.64200000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54300000000000004,"15min":0.56999999999999995},"min":{"1min":0.40200000000000002,"5min":0.33600000000000002,"15min":0.28299999999999997},"max":{"1min":0.78900000000000003,"5min":0.85599999999999998,"15min":0.90000000000000002},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.60699999999999998,"15min":0.64500000000000002},"min":{"1min":0.36699999999999999,"5min":0.34999999999999998,"15min":0.32200000000000001},"max":{"1min":0.89000000000000001,"5min":0.98999999999999999,"15min":1.175},"last":0.627}]}]},{"osd":5,"up_from":13,"seq":55834574946,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":847578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.503,"15min":0.50800000000000001},"min":{"1min":0.38800000000000001,"5min":0.29299999999999998,"15min":0.28599999999999998},"max":{"1min":0.67900000000000005,"5min":0.71399999999999997,"15min":0.86599999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51800000000000002,"15min":0.51400000000000001},"min":{"1min":0.42499999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.60599999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.496}]},{"osd":1,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.38400000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.63900000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52300000000000002,"15min":0.53100000000000003},"min":{"1min":0.43099999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72799999999999998,"5min":0.73899999999999999,"15min":0.81599999999999995},"last":0.47999999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54300000000000004,"15min":0.54400000000000004},"min":{"1min":0.41099999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.63900000000000001,"5min":0.72999999999999998,"15min":0.83999999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.47199999999999998,"5min":0.40999999999999998,"15min":0.39500000000000002},"max":{"1min":0.72699999999999998,"5min":0.74099999999999999,"15min":0.79900000000000004},"last":0.53600000000000003}]},{"osd":3,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.42399999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.66900000000000004,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.54900000000000004,"15min":0.56000000000000005},"min":{"1min":0.42499999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.63500000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.55800000000000005}]},{"osd":4,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.50600000000000001,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.36199999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.80700000000000005},"last":0.433}]},{"osd":6,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55300000000000005,"15min":0.53600000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.83199999999999996},"last":0.378},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.36199999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77900000000000003,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.40200000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.57899999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71499999999999997,"5min":0.79500000000000004,"15min":0.85599999999999998},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.374,"5min":0.35199999999999998,"15min":0.314},"max":{"1min":0.79500000000000004,"5min":0.84499999999999997,"15min":0.90600000000000003},"last":0.44700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:38:25.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.3f is not active+clean 2023-10-09T06:38:25.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.687243+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T09:15:57.542772+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-09T06:38:25.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.3e is not active+clean 2023-10-09T06:38:25.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.686956+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T12:10:38.081403+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-09T06:38:25.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.3d is not active+clean 2023-10-09T06:38:25.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.681576+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T07:11:37.888420+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-09T06:38:25.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.3c is not active+clean 2023-10-09T06:38:25.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.685055+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T09:58:03.322193+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-09T06:38:25.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.3b is not active+clean 2023-10-09T06:38:25.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.678873+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T14:41:30.382056+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-09T06:38:25.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.3a is not active+clean 2023-10-09T06:38:25.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.678481+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T12:36:50.516609+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-09T06:38:25.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.39 is not active+clean 2023-10-09T06:38:25.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.682433+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T08:46:35.635953+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-09T06:38:25.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.38 is not active+clean 2023-10-09T06:38:25.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.676958+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T09:58:03.322193+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-09T06:38:25.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.37 is not active+clean 2023-10-09T06:38:25.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.688589+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T16:20:02.974752+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-09T06:38:25.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.36 is not active+clean 2023-10-09T06:38:25.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.678660+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T08:46:35.635953+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-09T06:38:25.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.35 is not active+clean 2023-10-09T06:38:25.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.680844+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T09:25:57.096853+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-09T06:38:25.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.34 is not active+clean 2023-10-09T06:38:25.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.678704+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T08:48:51.565773+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-09T06:38:25.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.33 is not active+clean 2023-10-09T06:38:25.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.681087+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T09:25:53.616204+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-09T06:38:25.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.32 is not active+clean 2023-10-09T06:38:25.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.683792+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T16:34:55.917445+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-09T06:38:25.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.31 is not active+clean 2023-10-09T06:38:25.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.678799+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T17:06:23.000411+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-09T06:38:25.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.30 is not active+clean 2023-10-09T06:38:25.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.677797+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T17:06:23.000411+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-09T06:38:25.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.2f is not active+clean 2023-10-09T06:38:25.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.682655+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T14:44:35.145850+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-09T06:38:25.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.2e is not active+clean 2023-10-09T06:38:25.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.678258+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T10:44:51.812594+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-09T06:38:25.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.2d is not active+clean 2023-10-09T06:38:25.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.679357+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T14:44:35.145850+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-09T06:38:25.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.2c is not active+clean 2023-10-09T06:38:25.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.676535+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T17:16:26.649802+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-09T06:38:25.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.2b is not active+clean 2023-10-09T06:38:25.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.683129+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T15:38:53.437170+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-09T06:38:25.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.2a is not active+clean 2023-10-09T06:38:25.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.678996+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T17:19:50.331458+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-09T06:38:25.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.29 is not active+clean 2023-10-09T06:38:25.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.681279+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T11:21:46.727517+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-09T06:38:25.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.28 is not active+clean 2023-10-09T06:38:25.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.682901+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T16:26:55.538336+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-09T06:38:25.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.27 is not active+clean 2023-10-09T06:38:25.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.687816+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T09:08:34.030754+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-09T06:38:25.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.26 is not active+clean 2023-10-09T06:38:25.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.677830+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T08:01:29.075717+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-09T06:38:25.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.25 is not active+clean 2023-10-09T06:38:25.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.679264+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T07:29:07.591719+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-09T06:38:25.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.24 is not active+clean 2023-10-09T06:38:25.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.681628+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T08:49:58.537930+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-09T06:38:25.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.23 is not active+clean 2023-10-09T06:38:25.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.680535+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T11:46:34.855757+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-09T06:38:25.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.22 is not active+clean 2023-10-09T06:38:25.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.687098+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T11:46:34.855757+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-09T06:38:25.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.21 is not active+clean 2023-10-09T06:38:25.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.676565+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T09:58:03.322193+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-09T06:38:25.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.20 is not active+clean 2023-10-09T06:38:25.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.677453+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T15:08:26.202100+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-09T06:38:25.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.1e is not active+clean 2023-10-09T06:38:25.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.688987+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T11:42:48.250267+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-09T06:38:25.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.1f is not active+clean 2023-10-09T06:38:25.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.679700+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T08:15:52.880306+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-09T06:38:25.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.1c is not active+clean 2023-10-09T06:38:25.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.677343+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T15:52:50.901568+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-09T06:38:25.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.1d is not active+clean 2023-10-09T06:38:25.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.677169+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T12:36:50.516609+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-09T06:38:25.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.1b is not active+clean 2023-10-09T06:38:25.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.682687+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T12:40:22.810227+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-09T06:38:25.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.18 is not active+clean 2023-10-09T06:38:25.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.676460+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T15:08:26.202100+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-09T06:38:25.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.19 is not active+clean 2023-10-09T06:38:25.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.682248+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T11:22:26.955724+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-09T06:38:25.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.16 is not active+clean 2023-10-09T06:38:25.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.682331+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T09:26:21.371656+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-09T06:38:25.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.17 is not active+clean 2023-10-09T06:38:25.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.680775+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T10:50:10.718730+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-09T06:38:25.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.14 is not active+clean 2023-10-09T06:38:25.488 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.683619+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T15:52:50.901568+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-09T06:38:25.488 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.15 is not active+clean 2023-10-09T06:38:25.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.678258+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T12:36:50.516609+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-09T06:38:25.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.12 is not active+clean 2023-10-09T06:38:25.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.677099+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T08:01:29.075717+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-09T06:38:25.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.13 is not active+clean 2023-10-09T06:38:25.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.677625+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T14:41:30.382056+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-09T06:38:25.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.10 is not active+clean 2023-10-09T06:38:25.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.677087+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T12:21:10.321869+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-09T06:38:25.489 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.11 is not active+clean 2023-10-09T06:38:25.489 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.677747+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T17:16:26.649802+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-09T06:38:25.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.e is not active+clean 2023-10-09T06:38:25.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.687784+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T16:57:12.115891+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-09T06:38:25.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.f is not active+clean 2023-10-09T06:38:25.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.687350+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T06:40:00.965033+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-09T06:38:25.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.c is not active+clean 2023-10-09T06:38:25.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.681184+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T07:59:57.861748+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-09T06:38:25.490 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.d is not active+clean 2023-10-09T06:38:25.490 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.676742+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T08:01:29.075717+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-09T06:38:25.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.a is not active+clean 2023-10-09T06:38:25.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.676045+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T17:16:26.649802+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-09T06:38:25.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.b is not active+clean 2023-10-09T06:38:25.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.677131+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T17:16:34.306892+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-09T06:38:25.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.8 is not active+clean 2023-10-09T06:38:25.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.683097+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T08:48:51.565773+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-09T06:38:25.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.9 is not active+clean 2023-10-09T06:38:25.491 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.679938+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T12:43:32.088320+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-09T06:38:25.491 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.6 is not active+clean 2023-10-09T06:38:25.492 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.679867+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T17:32:06.438731+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-09T06:38:25.492 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.7 is not active+clean 2023-10-09T06:38:25.492 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.677792+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T09:58:03.322193+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-09T06:38:25.492 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.4 is not active+clean 2023-10-09T06:38:25.492 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.683688+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T07:29:07.591719+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-09T06:38:25.492 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.5 is not active+clean 2023-10-09T06:38:25.492 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.680141+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T11:31:22.541312+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-09T06:38:25.492 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.2 is not active+clean 2023-10-09T06:38:25.492 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.678527+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T14:41:30.382056+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-09T06:38:25.493 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.3 is not active+clean 2023-10-09T06:38:25.493 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.680109+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T12:10:38.081403+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-09T06:38:25.493 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.0 is not active+clean 2023-10-09T06:38:25.493 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.676616+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T12:49:51.877549+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-09T06:38:25.493 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.1 is not active+clean 2023-10-09T06:38:25.493 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.682501+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T17:05:32.721283+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-09T06:38:25.493 INFO:tasks.cephfs.filesystem.ceph_manager:PG 27.1a is not active+clean 2023-10-09T06:38:25.493 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '27.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-09T06:38:19.667674+0000', 'last_change': '2023-10-09T06:38:19.682069+0000', 'last_active': '2023-10-09T06:38:19.667674+0000', 'last_peered': '2023-10-09T06:38:19.667674+0000', 'last_clean': '2023-10-09T06:38:19.667674+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-09T06:38:19.667674+0000', 'last_undegraded': '2023-10-09T06:38:19.667674+0000', 'last_fullsized': '2023-10-09T06:38:19.667674+0000', 'mapping_epoch': 150, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 150, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'last_clean_scrub_stamp': '2023-10-09T06:38:19.667674+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T10:44:51.812594+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-09T06:38:25.494 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-09T06:38:25.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.625+0000 7f1ea2af2700 1 Processor -- start 2023-10-09T06:38:25.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.625+0000 7f1ea2af2700 1 -- start start 2023-10-09T06:38:25.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.625+0000 7f1ea2af2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e9c159ec0 0x7f1e9c0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:25.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.626+0000 7f1ea2af2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1e9c0b53d0 0x7f1e9c0cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:25.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.626+0000 7f1ea2af2700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e9c0b5b00 con 0x7f1e9c0cce40 2023-10-09T06:38:25.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.626+0000 7f1ea2af2700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e9c0b5da0 con 0x7f1e9c0b53d0 2023-10-09T06:38:25.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.626+0000 7f1ea2af2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c0b6080 con 0x7f1e9c159ec0 2023-10-09T06:38:25.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.626+0000 7f1e9bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1e9c0b53d0 0x7f1e9c0cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:25.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.626+0000 7f1ea108f700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f1e9c0cce40 0x7f1e9c0cd280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:33660/0 (socket says 172.21.15.163:33660) 2023-10-09T06:38:25.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.626+0000 7f1ea108f700 1 -- 172.21.15.163:0/2398582976 learned_addr learned my addr 172.21.15.163:0/2398582976 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:25.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.626+0000 7f1e9bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1e9c0b53d0 0x7f1e9c0cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42192/0 (socket says 172.21.15.163:42192) 2023-10-09T06:38:25.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.626+0000 7f1ea088e700 1 --2- 172.21.15.163:0/2398582976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e9c159ec0 0x7f1e9c0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:25.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.626+0000 7f1e9bfff700 1 -- 172.21.15.163:0/2398582976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e9c159ec0 msgr2=0x7f1e9c0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:25.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.626+0000 7f1e9bfff700 1 --2- 172.21.15.163:0/2398582976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e9c159ec0 0x7f1e9c0b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:25.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.626+0000 7f1e9bfff700 1 -- 172.21.15.163:0/2398582976 >> v1:172.21.15.163:6789/0 conn(0x7f1e9c0cce40 legacy=0x7f1e9c0cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:25.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.626+0000 7f1e9bfff700 1 -- 172.21.15.163:0/2398582976 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e9c0cc100 con 0x7f1e9c0b53d0 2023-10-09T06:38:25.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.626+0000 7f1e9b7fe700 1 -- 172.21.15.163:0/2398582976 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 322419198 0 0) 0x7f1e9c0b5b00 con 0x7f1e9c0cce40 2023-10-09T06:38:25.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.626+0000 7f1e9bfff700 1 --2- 172.21.15.163:0/2398582976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e9c0b53d0 0x7f1e9c0cc900 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f1e900098e0 tx=0x7f1e900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9147c5afb8ea53cb server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:25.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.626+0000 7f1e9b7fe700 1 -- 172.21.15.163:0/2398582976 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9000b640 con 0x7f1e9c0b53d0 2023-10-09T06:38:25.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.626+0000 7f1e9b7fe700 1 -- 172.21.15.163:0/2398582976 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9000b7a0 con 0x7f1e9c0b53d0 2023-10-09T06:38:25.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.626+0000 7f1e9b7fe700 1 -- 172.21.15.163:0/2398582976 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9000bb40 con 0x7f1e9c0b53d0 2023-10-09T06:38:25.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.627+0000 7f1ea2af2700 1 -- 172.21.15.163:0/2398582976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e9c0b53d0 msgr2=0x7f1e9c0cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:25.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.627+0000 7f1ea2af2700 1 --2- 172.21.15.163:0/2398582976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e9c0b53d0 0x7f1e9c0cc900 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f1e900098e0 tx=0x7f1e900091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:25.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.627+0000 7f1ea2af2700 1 -- 172.21.15.163:0/2398582976 shutdown_connections 2023-10-09T06:38:25.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.627+0000 7f1ea2af2700 1 --2- 172.21.15.163:0/2398582976 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e9c159ec0 0x7f1e9c0b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:25.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.627+0000 7f1ea2af2700 1 --2- 172.21.15.163:0/2398582976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e9c0b53d0 0x7f1e9c0cc900 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:25.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.627+0000 7f1ea2af2700 1 -- 172.21.15.163:0/2398582976 >> 172.21.15.163:0/2398582976 conn(0x7f1e9c0999f0 msgr2=0x7f1e9c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:25.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.627+0000 7f1ea2af2700 1 -- 172.21.15.163:0/2398582976 shutdown_connections 2023-10-09T06:38:25.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.627+0000 7f1ea2af2700 1 -- 172.21.15.163:0/2398582976 wait complete. 2023-10-09T06:38:25.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.628+0000 7f1ea2af2700 1 Processor -- start 2023-10-09T06:38:25.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.628+0000 7f1ea2af2700 1 -- start start 2023-10-09T06:38:25.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.630+0000 7f1ea2af2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e9c0b53d0 0x7f1e9c15e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:25.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.630+0000 7f1ea2af2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e9c0cce40 0x7f1e9c15cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:25.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.630+0000 7f1ea2af2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e9c159ec0 0x7f1e9c169da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:25.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.630+0000 7f1ea2af2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c15d300 con 0x7f1e9c159ec0 2023-10-09T06:38:25.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.630+0000 7f1ea2af2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c15d5a0 con 0x7f1e9c0b53d0 2023-10-09T06:38:25.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.630+0000 7f1ea2af2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c068430 con 0x7f1e9c0cce40 2023-10-09T06:38:25.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.630+0000 7f1ea108f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e9c159ec0 0x7f1e9c169da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:25.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.630+0000 7f1ea108f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e9c159ec0 0x7f1e9c169da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42208/0 (socket says 172.21.15.163:42208) 2023-10-09T06:38:25.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.630+0000 7f1ea108f700 1 -- 172.21.15.163:0/2844852734 learned_addr learned my addr 172.21.15.163:0/2844852734 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:25.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.630+0000 7f1e9bfff700 1 --2- 172.21.15.163:0/2844852734 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e9c0cce40 0x7f1e9c15cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:25.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.630+0000 7f1ea108f700 1 -- 172.21.15.163:0/2844852734 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e9c0cce40 msgr2=0x7f1e9c15cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:25.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.630+0000 7f1ea108f700 1 --2- 172.21.15.163:0/2844852734 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e9c0cce40 0x7f1e9c15cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:25.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.630+0000 7f1ea088e700 1 --2- 172.21.15.163:0/2844852734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e9c0b53d0 0x7f1e9c15e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:25.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.630+0000 7f1ea108f700 1 -- 172.21.15.163:0/2844852734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e9c0b53d0 msgr2=0x7f1e9c15e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:25.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.630+0000 7f1ea108f700 1 --2- 172.21.15.163:0/2844852734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e9c0b53d0 0x7f1e9c15e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:25.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.630+0000 7f1ea108f700 1 -- 172.21.15.163:0/2844852734 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e90009420 con 0x7f1e9c159ec0 2023-10-09T06:38:25.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.630+0000 7f1ea108f700 1 --2- 172.21.15.163:0/2844852734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e9c159ec0 0x7f1e9c169da0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f1e9400c4f0 tx=0x7f1e94007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:25.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.630+0000 7f1e98ff9700 1 -- 172.21.15.163:0/2844852734 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e94005b30 con 0x7f1e9c159ec0 2023-10-09T06:38:25.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.630+0000 7f1e98ff9700 1 -- 172.21.15.163:0/2844852734 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e94005cb0 con 0x7f1e9c159ec0 2023-10-09T06:38:25.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.630+0000 7f1ea2af2700 1 -- 172.21.15.163:0/2844852734 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e9c068710 con 0x7f1e9c159ec0 2023-10-09T06:38:25.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.630+0000 7f1e98ff9700 1 -- 172.21.15.163:0/2844852734 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e94007320 con 0x7f1e9c159ec0 2023-10-09T06:38:25.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.630+0000 7f1ea2af2700 1 -- 172.21.15.163:0/2844852734 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e9c068cd0 con 0x7f1e9c159ec0 2023-10-09T06:38:25.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.630+0000 7f1ea2af2700 1 -- 172.21.15.163:0/2844852734 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e9c063c90 con 0x7f1e9c159ec0 2023-10-09T06:38:25.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.631+0000 7f1e98ff9700 1 -- 172.21.15.163:0/2844852734 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1e9401b3b0 con 0x7f1e9c159ec0 2023-10-09T06:38:25.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.631+0000 7f1e98ff9700 1 --2- 172.21.15.163:0/2844852734 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1e88067320 0x7f1e880697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:25.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.631+0000 7f1e98ff9700 1 -- 172.21.15.163:0/2844852734 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1e94084140 con 0x7f1e9c159ec0 2023-10-09T06:38:25.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.631+0000 7f1ea088e700 1 --2- 172.21.15.163:0/2844852734 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1e88067320 0x7f1e880697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:25.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.632+0000 7f1ea088e700 1 --2- 172.21.15.163:0/2844852734 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1e88067320 0x7f1e880697d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c009fd0 tx=0x7f1e8c00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:25.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.636+0000 7f1e98ff9700 1 -- 172.21.15.163:0/2844852734 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1e94050dd0 con 0x7f1e9c159ec0 2023-10-09T06:38:25.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.895+0000 7f1ea2af2700 1 -- 172.21.15.163:0/2844852734 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1e9c04c3c0 con 0x7f1e9c159ec0 2023-10-09T06:38:25.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.896+0000 7f1e98ff9700 1 -- 172.21.15.163:0/2844852734 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1460 (secure 0 0 0) 0x7f1e940543f0 con 0x7f1e9c159ec0 2023-10-09T06:38:25.897 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:25.897 INFO:teuthology.orchestra.run.smithi163.stdout:{"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":487,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":152,"num_osds":8,"num_up_osds":8,"osd_up_since":1696833029,"num_in_osds":8,"osd_in_since":1696833021,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":2485305344,"bytes_avail":770608807936,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":103,"id":13,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":13,"rank":0,"name":"a","status":"up:active","gid":7875}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2023-10-09T06:38:10.407813+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-09T06:38:25.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.898+0000 7f1ea2af2700 1 -- 172.21.15.163:0/2844852734 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1e88067320 msgr2=0x7f1e880697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:25.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.898+0000 7f1ea2af2700 1 --2- 172.21.15.163:0/2844852734 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1e88067320 0x7f1e880697d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c009fd0 tx=0x7f1e8c00b040 comp rx=0 tx=0).stop 2023-10-09T06:38:25.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.898+0000 7f1ea2af2700 1 -- 172.21.15.163:0/2844852734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e9c159ec0 msgr2=0x7f1e9c169da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:25.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.898+0000 7f1ea2af2700 1 --2- 172.21.15.163:0/2844852734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e9c159ec0 0x7f1e9c169da0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f1e9400c4f0 tx=0x7f1e94007f50 comp rx=0 tx=0).stop 2023-10-09T06:38:25.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.898+0000 7f1ea2af2700 1 -- 172.21.15.163:0/2844852734 shutdown_connections 2023-10-09T06:38:25.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.898+0000 7f1ea2af2700 1 --2- 172.21.15.163:0/2844852734 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1e88067320 0x7f1e880697d0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:25.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.898+0000 7f1ea2af2700 1 --2- 172.21.15.163:0/2844852734 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e9c159ec0 0x7f1e9c169da0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:25.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.898+0000 7f1ea2af2700 1 --2- 172.21.15.163:0/2844852734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e9c0b53d0 0x7f1e9c15e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:25.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.898+0000 7f1ea2af2700 1 --2- 172.21.15.163:0/2844852734 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e9c0cce40 0x7f1e9c15cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:25.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.898+0000 7f1ea2af2700 1 -- 172.21.15.163:0/2844852734 >> 172.21.15.163:0/2844852734 conn(0x7f1e9c0999f0 msgr2=0x7f1e9c0c98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:25.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.898+0000 7f1ea2af2700 1 -- 172.21.15.163:0/2844852734 shutdown_connections 2023-10-09T06:38:25.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:25.898+0000 7f1ea2af2700 1 -- 172.21.15.163:0/2844852734 wait complete. 2023-10-09T06:38:25.911 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-09T06:38:25.911 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:38:25.970 INFO:tasks.ceph.mds.a.smithi163.stderr:2023-10-09T06:38:25.969+0000 7fd90cfde700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-09T06:38:26.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.026+0000 7f120312e700 1 Processor -- start 2023-10-09T06:38:26.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.026+0000 7f120312e700 1 -- start start 2023-10-09T06:38:26.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.026+0000 7f120312e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f11fc159890 0x7f11fc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:26.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.026+0000 7f120312e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11fc069560 0x7f11fc0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:26.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.026+0000 7f120312e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11fc069ee0 con 0x7f11fc159c60 2023-10-09T06:38:26.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.026+0000 7f120312e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11fc15a160 con 0x7f11fc159890 2023-10-09T06:38:26.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.026+0000 7f120312e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc0d13a0 con 0x7f11fc069560 2023-10-09T06:38:26.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.026+0000 7f1200eca700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f11fc159890 0x7f11fc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:26.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.026+0000 7f1200eca700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f11fc159890 0x7f11fc15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42216/0 (socket says 172.21.15.163:42216) 2023-10-09T06:38:26.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.026+0000 7f1200eca700 1 -- 172.21.15.163:0/3939737641 learned_addr learned my addr 172.21.15.163:0/3939737641 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:26.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.026+0000 7f12016cb700 1 --2- 172.21.15.163:0/3939737641 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11fc069560 0x7f11fc0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:26.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.027+0000 7f1200eca700 1 -- 172.21.15.163:0/3939737641 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11fc069560 msgr2=0x7f11fc0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:26.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.027+0000 7f1200eca700 1 --2- 172.21.15.163:0/3939737641 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11fc069560 0x7f11fc0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:26.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.027+0000 7f1200eca700 1 -- 172.21.15.163:0/3939737641 >> v1:172.21.15.181:6789/0 conn(0x7f11fc159c60 legacy=0x7f11fc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:26.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.027+0000 7f1200eca700 1 -- 172.21.15.163:0/3939737641 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc15e140 con 0x7f11fc159890 2023-10-09T06:38:26.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.027+0000 7f12016cb700 1 --2- 172.21.15.163:0/3939737641 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11fc069560 0x7f11fc0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:26.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.027+0000 7f1200eca700 1 --2- 172.21.15.163:0/3939737641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11fc159890 0x7f11fc15abd0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f11ec009f20 tx=0x7f11ec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eca218ab599ef8b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:26.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.027+0000 7f11fb7fe700 1 -- 172.21.15.163:0/3939737641 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec00e070 con 0x7f11fc159890 2023-10-09T06:38:26.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.027+0000 7f11fb7fe700 1 -- 172.21.15.163:0/3939737641 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ec00ba80 con 0x7f11fc159890 2023-10-09T06:38:26.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.027+0000 7f11fb7fe700 1 -- 172.21.15.163:0/3939737641 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec0092e0 con 0x7f11fc159890 2023-10-09T06:38:26.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.027+0000 7f120312e700 1 -- 172.21.15.163:0/3939737641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11fc159890 msgr2=0x7f11fc15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:26.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.027+0000 7f120312e700 1 --2- 172.21.15.163:0/3939737641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11fc159890 0x7f11fc15abd0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f11ec009f20 tx=0x7f11ec009840 comp rx=0 tx=0).stop 2023-10-09T06:38:26.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.027+0000 7f120312e700 1 -- 172.21.15.163:0/3939737641 shutdown_connections 2023-10-09T06:38:26.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.028+0000 7f120312e700 1 --2- 172.21.15.163:0/3939737641 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11fc069560 0x7f11fc0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:26.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.028+0000 7f120312e700 1 --2- 172.21.15.163:0/3939737641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11fc159890 0x7f11fc15abd0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:26.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.028+0000 7f120312e700 1 -- 172.21.15.163:0/3939737641 >> 172.21.15.163:0/3939737641 conn(0x7f11fc068e20 msgr2=0x7f11fc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:26.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.028+0000 7f120312e700 1 -- 172.21.15.163:0/3939737641 shutdown_connections 2023-10-09T06:38:26.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.028+0000 7f120312e700 1 -- 172.21.15.163:0/3939737641 wait complete. 2023-10-09T06:38:26.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.029+0000 7f120312e700 1 Processor -- start 2023-10-09T06:38:26.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.029+0000 7f120312e700 1 -- start start 2023-10-09T06:38:26.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.030+0000 7f120312e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11fc069560 0x7f11fc1524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:26.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.030+0000 7f120312e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11fc159890 0x7f11fc0cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:26.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.030+0000 7f120312e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11fc159c60 0x7f11fc0b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:26.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.030+0000 7f120312e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc0b3a20 con 0x7f11fc159c60 2023-10-09T06:38:26.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.030+0000 7f120312e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc0b3d00 con 0x7f11fc159890 2023-10-09T06:38:26.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.030+0000 7f120312e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc0b3fe0 con 0x7f11fc069560 2023-10-09T06:38:26.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.030+0000 7f12016cb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11fc159c60 0x7f11fc0b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:26.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.030+0000 7f12016cb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11fc159c60 0x7f11fc0b14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42230/0 (socket says 172.21.15.163:42230) 2023-10-09T06:38:26.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.030+0000 7f12016cb700 1 -- 172.21.15.163:0/2748879156 learned_addr learned my addr 172.21.15.163:0/2748879156 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:26.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.030+0000 7f1200eca700 1 --2- 172.21.15.163:0/2748879156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11fc069560 0x7f11fc1524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:26.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.030+0000 7f11fbfff700 1 --2- 172.21.15.163:0/2748879156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11fc159890 0x7f11fc0cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:26.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.030+0000 7f12016cb700 1 -- 172.21.15.163:0/2748879156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11fc069560 msgr2=0x7f11fc1524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:26.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.030+0000 7f12016cb700 1 --2- 172.21.15.163:0/2748879156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11fc069560 0x7f11fc1524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:26.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.030+0000 7f12016cb700 1 -- 172.21.15.163:0/2748879156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11fc159890 msgr2=0x7f11fc0cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:26.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.030+0000 7f12016cb700 1 --2- 172.21.15.163:0/2748879156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11fc159890 0x7f11fc0cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:26.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.030+0000 7f12016cb700 1 -- 172.21.15.163:0/2748879156 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ec009ae0 con 0x7f11fc159c60 2023-10-09T06:38:26.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.030+0000 7f12016cb700 1 --2- 172.21.15.163:0/2748879156 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11fc159c60 0x7f11fc0b14e0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f11f400d860 tx=0x7f11f4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:26.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.030+0000 7f11f8ff9700 1 -- 172.21.15.163:0/2748879156 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f40125f0 con 0x7f11fc159c60 2023-10-09T06:38:26.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.030+0000 7f11f8ff9700 1 -- 172.21.15.163:0/2748879156 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f4012750 con 0x7f11fc159c60 2023-10-09T06:38:26.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.030+0000 7f120312e700 1 -- 172.21.15.163:0/2748879156 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11fc0b42c0 con 0x7f11fc159c60 2023-10-09T06:38:26.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.030+0000 7f11f8ff9700 1 -- 172.21.15.163:0/2748879156 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f4012ad0 con 0x7f11fc159c60 2023-10-09T06:38:26.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.030+0000 7f120312e700 1 -- 172.21.15.163:0/2748879156 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11fc1529f0 con 0x7f11fc159c60 2023-10-09T06:38:26.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.031+0000 7f120312e700 1 -- 172.21.15.163:0/2748879156 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11fc169b90 con 0x7f11fc159c60 2023-10-09T06:38:26.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.032+0000 7f11f8ff9700 1 -- 172.21.15.163:0/2748879156 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f11f4013480 con 0x7f11fc159c60 2023-10-09T06:38:26.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.032+0000 7f11f8ff9700 1 --2- 172.21.15.163:0/2748879156 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f11e8067320 0x7f11e80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:26.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.032+0000 7f11f8ff9700 1 -- 172.21.15.163:0/2748879156 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f11f4084d60 con 0x7f11fc159c60 2023-10-09T06:38:26.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.032+0000 7f1200eca700 1 --2- 172.21.15.163:0/2748879156 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f11e8067320 0x7f11e80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:26.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.033+0000 7f1200eca700 1 --2- 172.21.15.163:0/2748879156 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f11e8067320 0x7f11e80697d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f11ec009880 tx=0x7f11ec018000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:26.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.037+0000 7f11f8ff9700 1 -- 172.21.15.163:0/2748879156 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f11f4055480 con 0x7f11fc159c60 2023-10-09T06:38:26.212 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.211+0000 7f120312e700 1 -- 172.21.15.163:0/2748879156 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f11fc0b4150 con 0x7f11e8067320 2023-10-09T06:38:26.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.220+0000 7f11f8ff9700 1 -- 172.21.15.163:0/2748879156 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287181 (secure 0 0 0) 0x7f11fc0b4150 con 0x7f11e8067320 2023-10-09T06:38:26.224 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:26.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.223+0000 7f120312e700 1 -- 172.21.15.163:0/2748879156 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f11e8067320 msgr2=0x7f11e80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:26.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.223+0000 7f120312e700 1 --2- 172.21.15.163:0/2748879156 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f11e8067320 0x7f11e80697d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f11ec009880 tx=0x7f11ec018000 comp rx=0 tx=0).stop 2023-10-09T06:38:26.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.223+0000 7f120312e700 1 -- 172.21.15.163:0/2748879156 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11fc159c60 msgr2=0x7f11fc0b14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:26.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.223+0000 7f120312e700 1 --2- 172.21.15.163:0/2748879156 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11fc159c60 0x7f11fc0b14e0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f11f400d860 tx=0x7f11f4008040 comp rx=0 tx=0).stop 2023-10-09T06:38:26.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.223+0000 7f120312e700 1 -- 172.21.15.163:0/2748879156 shutdown_connections 2023-10-09T06:38:26.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.223+0000 7f120312e700 1 --2- 172.21.15.163:0/2748879156 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f11e8067320 0x7f11e80697d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:26.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.223+0000 7f120312e700 1 --2- 172.21.15.163:0/2748879156 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11fc159c60 0x7f11fc0b14e0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:26.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.223+0000 7f120312e700 1 --2- 172.21.15.163:0/2748879156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11fc069560 0x7f11fc1524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:26.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.223+0000 7f120312e700 1 --2- 172.21.15.163:0/2748879156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11fc159890 0x7f11fc0cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:26.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.223+0000 7f120312e700 1 -- 172.21.15.163:0/2748879156 >> 172.21.15.163:0/2748879156 conn(0x7f11fc068e20 msgr2=0x7f11fc0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:26.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.223+0000 7f120312e700 1 -- 172.21.15.163:0/2748879156 shutdown_connections 2023-10-09T06:38:26.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:26.223+0000 7f120312e700 1 -- 172.21.15.163:0/2748879156 wait complete. 2023-10-09T06:38:26.228 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:38:26.236 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2023-10-09T06:38:24.410910+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2427056,"kb_used_data":15984,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752547664,"statfs":{"total":773094113280,"available":770608807936,"internally_reserved":0,"allocated":16367616,"data_stored":7948834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.742617"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.687243+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:15:57.542772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.686956+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:10:38.081403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.681576+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:11:37.888420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.685055+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:58:03.322193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678873+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:41:30.382056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678481+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:36:50.516609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682433+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:46:35.635953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.676958+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:58:03.322193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.688589+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:20:02.974752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678660+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:46:35.635953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.680844+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:25:57.096853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678704+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:48:51.565773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.681087+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:25:53.616204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.683792+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:34:55.917445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678799+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:06:23.000411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677797+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:06:23.000411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682655+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:44:35.145850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678258+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:44:51.812594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.679357+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:44:35.145850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.676535+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:16:26.649802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.683129+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:38:53.437170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678996+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:19:50.331458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.681279+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:21:46.727517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682901+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:26:55.538336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.687816+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:08:34.030754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677830+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:01:29.075717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.679264+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:29:07.591719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.681628+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:49:58.537930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.680535+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:46:34.855757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.687098+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:46:34.855757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.676565+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:58:03.322193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677453+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:08:26.202100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.688987+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:42:48.250267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.691611+0000","last_change":"2023-10-09T06:38:19.691611+0000","last_active":"2023-10-09T06:38:19.691611+0000","last_peered":"2023-10-09T06:38:19.691611+0000","last_clean":"2023-10-09T06:38:19.691611+0000","last_became_active":"2023-10-09T06:38:19.691367+0000","last_became_peered":"2023-10-09T06:38:19.691367+0000","last_unstale":"2023-10-09T06:38:19.691611+0000","last_undegraded":"2023-10-09T06:38:19.691611+0000","last_fullsized":"2023-10-09T06:38:19.691611+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:13:34.033630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.679700+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:15:52.880306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.694250+0000","last_change":"2023-10-09T06:38:19.694250+0000","last_active":"2023-10-09T06:38:19.694250+0000","last_peered":"2023-10-09T06:38:19.694250+0000","last_clean":"2023-10-09T06:38:19.694250+0000","last_became_active":"2023-10-09T06:38:19.693478+0000","last_became_peered":"2023-10-09T06:38:19.693478+0000","last_unstale":"2023-10-09T06:38:19.694250+0000","last_undegraded":"2023-10-09T06:38:19.694250+0000","last_fullsized":"2023-10-09T06:38:19.694250+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:50:27.283148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677343+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:52:50.901568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.688925+0000","last_change":"2023-10-09T06:38:19.688925+0000","last_active":"2023-10-09T06:38:19.688925+0000","last_peered":"2023-10-09T06:38:19.688925+0000","last_clean":"2023-10-09T06:38:19.688925+0000","last_became_active":"2023-10-09T06:38:19.688479+0000","last_became_peered":"2023-10-09T06:38:19.688479+0000","last_unstale":"2023-10-09T06:38:19.688925+0000","last_undegraded":"2023-10-09T06:38:19.688925+0000","last_fullsized":"2023-10-09T06:38:19.688925+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:32:15.314337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677169+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:36:50.516609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.692752+0000","last_change":"2023-10-09T06:38:19.692752+0000","last_active":"2023-10-09T06:38:19.692752+0000","last_peered":"2023-10-09T06:38:19.692752+0000","last_clean":"2023-10-09T06:38:19.692752+0000","last_became_active":"2023-10-09T06:38:19.692249+0000","last_became_peered":"2023-10-09T06:38:19.692249+0000","last_unstale":"2023-10-09T06:38:19.692752+0000","last_undegraded":"2023-10-09T06:38:19.692752+0000","last_fullsized":"2023-10-09T06:38:19.692752+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:52:11.581942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682687+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:40:22.810227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.690895+0000","last_change":"2023-10-09T06:38:19.690895+0000","last_active":"2023-10-09T06:38:19.690895+0000","last_peered":"2023-10-09T06:38:19.690895+0000","last_clean":"2023-10-09T06:38:19.690895+0000","last_became_active":"2023-10-09T06:38:19.690633+0000","last_became_peered":"2023-10-09T06:38:19.690633+0000","last_unstale":"2023-10-09T06:38:19.690895+0000","last_undegraded":"2023-10-09T06:38:19.690895+0000","last_fullsized":"2023-10-09T06:38:19.690895+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:26:20.360953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.676460+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:08:26.202100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.687201+0000","last_change":"2023-10-09T06:38:19.687201+0000","last_active":"2023-10-09T06:38:19.687201+0000","last_peered":"2023-10-09T06:38:19.687201+0000","last_clean":"2023-10-09T06:38:19.687201+0000","last_became_active":"2023-10-09T06:38:19.686979+0000","last_became_peered":"2023-10-09T06:38:19.686979+0000","last_unstale":"2023-10-09T06:38:19.687201+0000","last_undegraded":"2023-10-09T06:38:19.687201+0000","last_fullsized":"2023-10-09T06:38:19.687201+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:04:37.633413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682248+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:22:26.955724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.690010+0000","last_change":"2023-10-09T06:38:19.690010+0000","last_active":"2023-10-09T06:38:19.690010+0000","last_peered":"2023-10-09T06:38:19.690010+0000","last_clean":"2023-10-09T06:38:19.690010+0000","last_became_active":"2023-10-09T06:38:19.689749+0000","last_became_peered":"2023-10-09T06:38:19.689749+0000","last_unstale":"2023-10-09T06:38:19.690010+0000","last_undegraded":"2023-10-09T06:38:19.690010+0000","last_fullsized":"2023-10-09T06:38:19.690010+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:08:33.020051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682331+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:26:21.371656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.693531+0000","last_change":"2023-10-09T06:38:19.693531+0000","last_active":"2023-10-09T06:38:19.693531+0000","last_peered":"2023-10-09T06:38:19.693531+0000","last_clean":"2023-10-09T06:38:19.693531+0000","last_became_active":"2023-10-09T06:38:19.693246+0000","last_became_peered":"2023-10-09T06:38:19.693246+0000","last_unstale":"2023-10-09T06:38:19.693531+0000","last_undegraded":"2023-10-09T06:38:19.693531+0000","last_fullsized":"2023-10-09T06:38:19.693531+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:26:54.527634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.680775+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:50:10.718730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.694122+0000","last_change":"2023-10-09T06:38:19.694122+0000","last_active":"2023-10-09T06:38:19.694122+0000","last_peered":"2023-10-09T06:38:19.694122+0000","last_clean":"2023-10-09T06:38:19.694122+0000","last_became_active":"2023-10-09T06:38:19.693847+0000","last_became_peered":"2023-10-09T06:38:19.693847+0000","last_unstale":"2023-10-09T06:38:19.694122+0000","last_undegraded":"2023-10-09T06:38:19.694122+0000","last_fullsized":"2023-10-09T06:38:19.694122+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:50:50.722486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.683619+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:52:50.901568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.688209+0000","last_change":"2023-10-09T06:38:19.688209+0000","last_active":"2023-10-09T06:38:19.688209+0000","last_peered":"2023-10-09T06:38:19.688209+0000","last_clean":"2023-10-09T06:38:19.688209+0000","last_became_active":"2023-10-09T06:38:19.687961+0000","last_became_peered":"2023-10-09T06:38:19.687961+0000","last_unstale":"2023-10-09T06:38:19.688209+0000","last_undegraded":"2023-10-09T06:38:19.688209+0000","last_fullsized":"2023-10-09T06:38:19.688209+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:52:49.890865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678258+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:36:50.516609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.693838+0000","last_change":"2023-10-09T06:38:19.693838+0000","last_active":"2023-10-09T06:38:19.693838+0000","last_peered":"2023-10-09T06:38:19.693838+0000","last_clean":"2023-10-09T06:38:19.693838+0000","last_became_active":"2023-10-09T06:38:19.693584+0000","last_became_peered":"2023-10-09T06:38:19.693584+0000","last_unstale":"2023-10-09T06:38:19.693838+0000","last_undegraded":"2023-10-09T06:38:19.693838+0000","last_fullsized":"2023-10-09T06:38:19.693838+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:42:55.756537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677099+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:01:29.075717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.684840+0000","last_change":"2023-10-09T06:38:19.684840+0000","last_active":"2023-10-09T06:38:19.684840+0000","last_peered":"2023-10-09T06:38:19.684840+0000","last_clean":"2023-10-09T06:38:19.684840+0000","last_became_active":"2023-10-09T06:38:19.684531+0000","last_became_peered":"2023-10-09T06:38:19.684531+0000","last_unstale":"2023-10-09T06:38:19.684840+0000","last_undegraded":"2023-10-09T06:38:19.684840+0000","last_fullsized":"2023-10-09T06:38:19.684840+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:42:47.239564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677625+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:41:30.382056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.692513+0000","last_change":"2023-10-09T06:38:19.692513+0000","last_active":"2023-10-09T06:38:19.692513+0000","last_peered":"2023-10-09T06:38:19.692513+0000","last_clean":"2023-10-09T06:38:19.692513+0000","last_became_active":"2023-10-09T06:38:19.692237+0000","last_became_peered":"2023-10-09T06:38:19.692237+0000","last_unstale":"2023-10-09T06:38:19.692513+0000","last_undegraded":"2023-10-09T06:38:19.692513+0000","last_fullsized":"2023-10-09T06:38:19.692513+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:56:50.090947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677087+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:21:10.321869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.681581+0000","last_change":"2023-10-09T06:38:19.681581+0000","last_active":"2023-10-09T06:38:19.681581+0000","last_peered":"2023-10-09T06:38:19.681581+0000","last_clean":"2023-10-09T06:38:19.681581+0000","last_became_active":"2023-10-09T06:38:19.681316+0000","last_became_peered":"2023-10-09T06:38:19.681316+0000","last_unstale":"2023-10-09T06:38:19.681581+0000","last_undegraded":"2023-10-09T06:38:19.681581+0000","last_fullsized":"2023-10-09T06:38:19.681581+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:38:52.426468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677747+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:16:26.649802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.688731+0000","last_change":"2023-10-09T06:38:19.688731+0000","last_active":"2023-10-09T06:38:19.688731+0000","last_peered":"2023-10-09T06:38:19.688731+0000","last_clean":"2023-10-09T06:38:19.688731+0000","last_became_active":"2023-10-09T06:38:19.688486+0000","last_became_peered":"2023-10-09T06:38:19.688486+0000","last_unstale":"2023-10-09T06:38:19.688731+0000","last_undegraded":"2023-10-09T06:38:19.688731+0000","last_fullsized":"2023-10-09T06:38:19.688731+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:12:32.876485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.687784+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:57:12.115891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.683035+0000","last_change":"2023-10-09T06:38:19.683035+0000","last_active":"2023-10-09T06:38:19.683035+0000","last_peered":"2023-10-09T06:38:19.683035+0000","last_clean":"2023-10-09T06:38:19.683035+0000","last_became_active":"2023-10-09T06:38:19.682754+0000","last_became_peered":"2023-10-09T06:38:19.682754+0000","last_unstale":"2023-10-09T06:38:19.683035+0000","last_undegraded":"2023-10-09T06:38:19.683035+0000","last_fullsized":"2023-10-09T06:38:19.683035+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:40:21.799524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.687350+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:40:00.965033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.694515+0000","last_change":"2023-10-09T06:38:19.694515+0000","last_active":"2023-10-09T06:38:19.694515+0000","last_peered":"2023-10-09T06:38:19.694515+0000","last_clean":"2023-10-09T06:38:19.694515+0000","last_became_active":"2023-10-09T06:38:19.694041+0000","last_became_peered":"2023-10-09T06:38:19.694041+0000","last_unstale":"2023-10-09T06:38:19.694515+0000","last_undegraded":"2023-10-09T06:38:19.694515+0000","last_fullsized":"2023-10-09T06:38:19.694515+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:40:48.522136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.681184+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:59:57.861748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.690795+0000","last_change":"2023-10-09T06:38:19.690795+0000","last_active":"2023-10-09T06:38:19.690795+0000","last_peered":"2023-10-09T06:38:19.690795+0000","last_clean":"2023-10-09T06:38:19.690795+0000","last_became_active":"2023-10-09T06:38:19.690512+0000","last_became_peered":"2023-10-09T06:38:19.690512+0000","last_unstale":"2023-10-09T06:38:19.690795+0000","last_undegraded":"2023-10-09T06:38:19.690795+0000","last_fullsized":"2023-10-09T06:38:19.690795+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:40:21.799524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.676742+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:01:29.075717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.688034+0000","last_change":"2023-10-09T06:38:19.688034+0000","last_active":"2023-10-09T06:38:19.688034+0000","last_peered":"2023-10-09T06:38:19.688034+0000","last_clean":"2023-10-09T06:38:19.688034+0000","last_became_active":"2023-10-09T06:38:19.687764+0000","last_became_peered":"2023-10-09T06:38:19.687764+0000","last_unstale":"2023-10-09T06:38:19.688034+0000","last_undegraded":"2023-10-09T06:38:19.688034+0000","last_fullsized":"2023-10-09T06:38:19.688034+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:37:57.642710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.676045+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:16:26.649802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.686556+0000","last_change":"2023-10-09T06:38:19.686556+0000","last_active":"2023-10-09T06:38:19.686556+0000","last_peered":"2023-10-09T06:38:19.686556+0000","last_clean":"2023-10-09T06:38:19.686556+0000","last_became_active":"2023-10-09T06:38:19.686253+0000","last_became_peered":"2023-10-09T06:38:19.686253+0000","last_unstale":"2023-10-09T06:38:19.686556+0000","last_undegraded":"2023-10-09T06:38:19.686556+0000","last_fullsized":"2023-10-09T06:38:19.686556+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:26:54.527634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677131+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:16:34.306892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.693762+0000","last_change":"2023-10-09T06:38:19.693762+0000","last_active":"2023-10-09T06:38:19.693762+0000","last_peered":"2023-10-09T06:38:19.693762+0000","last_clean":"2023-10-09T06:38:19.693762+0000","last_became_active":"2023-10-09T06:38:19.693501+0000","last_became_peered":"2023-10-09T06:38:19.693501+0000","last_unstale":"2023-10-09T06:38:19.693762+0000","last_undegraded":"2023-10-09T06:38:19.693762+0000","last_fullsized":"2023-10-09T06:38:19.693762+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:26:54.527634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.683097+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:48:51.565773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.691160+0000","last_change":"2023-10-09T06:38:19.691160+0000","last_active":"2023-10-09T06:38:19.691160+0000","last_peered":"2023-10-09T06:38:19.691160+0000","last_clean":"2023-10-09T06:38:19.691160+0000","last_became_active":"2023-10-09T06:38:19.690880+0000","last_became_peered":"2023-10-09T06:38:19.690880+0000","last_unstale":"2023-10-09T06:38:19.691160+0000","last_undegraded":"2023-10-09T06:38:19.691160+0000","last_fullsized":"2023-10-09T06:38:19.691160+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:40:21.799524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.679938+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:43:32.088320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.684011+0000","last_change":"2023-10-09T06:38:19.684011+0000","last_active":"2023-10-09T06:38:19.684011+0000","last_peered":"2023-10-09T06:38:19.684011+0000","last_clean":"2023-10-09T06:38:19.684011+0000","last_became_active":"2023-10-09T06:38:19.683541+0000","last_became_peered":"2023-10-09T06:38:19.683541+0000","last_unstale":"2023-10-09T06:38:19.684011+0000","last_undegraded":"2023-10-09T06:38:19.684011+0000","last_fullsized":"2023-10-09T06:38:19.684011+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:54:42.729290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.679867+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:32:06.438731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.691307+0000","last_change":"2023-10-09T06:38:19.691307+0000","last_active":"2023-10-09T06:38:19.691307+0000","last_peered":"2023-10-09T06:38:19.691307+0000","last_clean":"2023-10-09T06:38:19.691307+0000","last_became_active":"2023-10-09T06:38:19.691050+0000","last_became_peered":"2023-10-09T06:38:19.691050+0000","last_unstale":"2023-10-09T06:38:19.691307+0000","last_undegraded":"2023-10-09T06:38:19.691307+0000","last_fullsized":"2023-10-09T06:38:19.691307+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:15:20.738142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.677792+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:58:03.322193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.691686+0000","last_change":"2023-10-09T06:38:19.691686+0000","last_active":"2023-10-09T06:38:19.691686+0000","last_peered":"2023-10-09T06:38:19.691686+0000","last_clean":"2023-10-09T06:38:19.691686+0000","last_became_active":"2023-10-09T06:38:19.690753+0000","last_became_peered":"2023-10-09T06:38:19.690753+0000","last_unstale":"2023-10-09T06:38:19.691686+0000","last_undegraded":"2023-10-09T06:38:19.691686+0000","last_fullsized":"2023-10-09T06:38:19.691686+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:04:37.633413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.683688+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:29:07.591719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.687181+0000","last_change":"2023-10-09T06:38:19.687181+0000","last_active":"2023-10-09T06:38:19.687181+0000","last_peered":"2023-10-09T06:38:19.687181+0000","last_clean":"2023-10-09T06:38:19.687181+0000","last_became_active":"2023-10-09T06:38:19.686940+0000","last_became_peered":"2023-10-09T06:38:19.686940+0000","last_unstale":"2023-10-09T06:38:19.687181+0000","last_undegraded":"2023-10-09T06:38:19.687181+0000","last_fullsized":"2023-10-09T06:38:19.687181+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:52:11.581942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.680141+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:31:22.541312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.693786+0000","last_change":"2023-10-09T06:38:19.693786+0000","last_active":"2023-10-09T06:38:19.693786+0000","last_peered":"2023-10-09T06:38:19.693786+0000","last_clean":"2023-10-09T06:38:19.693786+0000","last_became_active":"2023-10-09T06:38:19.693529+0000","last_became_peered":"2023-10-09T06:38:19.693529+0000","last_unstale":"2023-10-09T06:38:19.693786+0000","last_undegraded":"2023-10-09T06:38:19.693786+0000","last_fullsized":"2023-10-09T06:38:19.693786+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:09:07.415429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.678527+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:41:30.382056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.690991+0000","last_change":"2023-10-09T06:38:19.690991+0000","last_active":"2023-10-09T06:38:19.690991+0000","last_peered":"2023-10-09T06:38:19.690991+0000","last_clean":"2023-10-09T06:38:19.690991+0000","last_became_active":"2023-10-09T06:38:19.690659+0000","last_became_peered":"2023-10-09T06:38:19.690659+0000","last_unstale":"2023-10-09T06:38:19.690991+0000","last_undegraded":"2023-10-09T06:38:19.690991+0000","last_fullsized":"2023-10-09T06:38:19.690991+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:02:57.917372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.680109+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:10:38.081403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.684393+0000","last_change":"2023-10-09T06:38:19.684393+0000","last_active":"2023-10-09T06:38:19.684393+0000","last_peered":"2023-10-09T06:38:19.684393+0000","last_clean":"2023-10-09T06:38:19.684393+0000","last_became_active":"2023-10-09T06:38:19.683050+0000","last_became_peered":"2023-10-09T06:38:19.683050+0000","last_unstale":"2023-10-09T06:38:19.684393+0000","last_undegraded":"2023-10-09T06:38:19.684393+0000","last_fullsized":"2023-10-09T06:38:19.684393+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:22:25.945021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.676616+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:49:51.877549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.685457+0000","last_change":"2023-10-09T06:38:19.685457+0000","last_active":"2023-10-09T06:38:19.685457+0000","last_peered":"2023-10-09T06:38:19.685457+0000","last_clean":"2023-10-09T06:38:19.685457+0000","last_became_active":"2023-10-09T06:38:19.685181+0000","last_became_peered":"2023-10-09T06:38:19.685181+0000","last_unstale":"2023-10-09T06:38:19.685457+0000","last_undegraded":"2023-10-09T06:38:19.685457+0000","last_fullsized":"2023-10-09T06:38:19.685457+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:39:59.954330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682501+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:05:32.721283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.692581+0000","last_change":"2023-10-09T06:38:19.692581+0000","last_active":"2023-10-09T06:38:19.692581+0000","last_peered":"2023-10-09T06:38:19.692581+0000","last_clean":"2023-10-09T06:38:19.692581+0000","last_became_active":"2023-10-09T06:38:19.692312+0000","last_became_peered":"2023-10-09T06:38:19.692312+0000","last_unstale":"2023-10-09T06:38:19.692581+0000","last_undegraded":"2023-10-09T06:38:19.692581+0000","last_fullsized":"2023-10-09T06:38:19.692581+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:52:30.325627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-09T06:38:19.667674+0000","last_change":"2023-10-09T06:38:19.682069+0000","last_active":"2023-10-09T06:38:19.667674+0000","last_peered":"2023-10-09T06:38:19.667674+0000","last_clean":"2023-10-09T06:38:19.667674+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-09T06:38:19.667674+0000","last_undegraded":"2023-10-09T06:38:19.667674+0000","last_fullsized":"2023-10-09T06:38:19.667674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:44:51.812594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":8,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-10-09T06:38:19.683671+0000","last_change":"2023-10-09T06:38:19.683671+0000","last_active":"2023-10-09T06:38:19.683671+0000","last_peered":"2023-10-09T06:38:19.683671+0000","last_clean":"2023-10-09T06:38:19.683671+0000","last_became_active":"2023-10-09T06:38:19.683355+0000","last_became_peered":"2023-10-09T06:38:19.683355+0000","last_unstale":"2023-10-09T06:38:19.683671+0000","last_undegraded":"2023-10-09T06:38:19.683671+0000","last_fullsized":"2023-10-09T06:38:19.683671+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:23:57.642507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":270,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-09T06:38:18.676452+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:38:18.676452+0000","last_peered":"2023-10-09T06:38:18.676452+0000","last_clean":"2023-10-09T06:38:18.676452+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:38:18.676452+0000","last_undegraded":"2023-10-09T06:38:18.676452+0000","last_fullsized":"2023-10-09T06:38:18.676452+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574946,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303820,"kb_used_data":2436,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068020,"statfs":{"total":96636764160,"available":96325652480,"internally_reserved":0,"allocated":2494464,"data_stored":1437946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52000000000000002,"15min":0.50700000000000001},"min":{"1min":0.34799999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.37,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67600000000000005,"5min":0.753,"15min":0.753},"last":0.52900000000000003}]},{"osd":1,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.503,"15min":0.51900000000000002},"min":{"1min":0.33600000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.74299999999999999,"5min":0.77400000000000002,"15min":1.2030000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.378,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.57899999999999996}]},{"osd":2,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53200000000000003,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75800000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.44,"5min":0.36399999999999999,"15min":0.315},"max":{"1min":0.77900000000000003,"5min":0.80700000000000005,"15min":0.83099999999999996},"last":0.56499999999999995}]},{"osd":3,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53800000000000003,"15min":0.55000000000000004},"min":{"1min":0.41199999999999998,"5min":0.35899999999999999,"15min":0.33500000000000002},"max":{"1min":0.68600000000000005,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55600000000000005,"15min":0.59199999999999997},"min":{"1min":0.45300000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.754,"5min":0.82099999999999995,"15min":1.1479999999999999},"last":0.50900000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.48199999999999998,"15min":0.51900000000000002},"min":{"1min":0.35599999999999998,"5min":0.30299999999999999,"15min":0.24399999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.93999999999999995},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56599999999999995,"15min":0.60599999999999998},"min":{"1min":0.36799999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76700000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.59199999999999997}]},{"osd":5,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52500000000000002,"15min":0.56999999999999995},"min":{"1min":0.36499999999999999,"5min":0.32500000000000001,"15min":0.307},"max":{"1min":0.81999999999999995,"5min":0.85799999999999998,"15min":0.91100000000000003},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59599999999999997,"15min":0.62},"min":{"1min":0.52900000000000003,"5min":0.41699999999999998,"15min":0.34200000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.621}]},{"osd":6,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55700000000000005,"15min":0.59299999999999997},"min":{"1min":0.38600000000000001,"5min":0.33500000000000002,"15min":0.28699999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.88600000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.56100000000000005,"15min":0.61399999999999999},"min":{"1min":0.45100000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.84599999999999997},"last":0.64300000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574945,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303228,"kb_used_data":1844,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068612,"statfs":{"total":96636764160,"available":96326258688,"internally_reserved":0,"allocated":1888256,"data_stored":841315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.50800000000000001,"15min":0.502},"min":{"1min":0.35499999999999998,"5min":0.32000000000000001,"15min":0.30399999999999999},"max":{"1min":0.66500000000000004,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.50700000000000001,"15min":0.51000000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70399999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.51300000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.502,"15min":0.51200000000000001},"min":{"1min":0.33600000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68300000000000005,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.5},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52900000000000003,"15min":0.53200000000000003},"min":{"1min":0.38700000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.77100000000000002,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.60299999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.34499999999999997,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.72399999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51100000000000001,"15min":0.52900000000000003},"min":{"1min":0.39000000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.66600000000000004,"5min":0.67500000000000004,"15min":0.752},"last":0.44500000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.38900000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66600000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.54300000000000004,"15min":0.68799999999999994},"min":{"1min":0.40400000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.752,"5min":0.752,"15min":4.7510000000000003},"last":0.53600000000000003}]},{"osd":4,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46400000000000002,"15min":0.47699999999999998},"min":{"1min":0.29899999999999999,"5min":0.28199999999999997,"15min":0.27900000000000003},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.76400000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56100000000000005,"15min":0.58099999999999996},"min":{"1min":0.41799999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.77200000000000002,"5min":0.81299999999999994,"15min":0.83699999999999997},"last":0.66500000000000004}]},{"osd":5,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.52100000000000002,"15min":0.53000000000000003},"min":{"1min":0.31900000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.66100000000000003,"5min":0.79200000000000004,"15min":0.81200000000000006},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.59499999999999997},"min":{"1min":0.46899999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76000000000000001,"5min":0.78000000000000003,"15min":0.86599999999999999},"last":0.58999999999999997}]},{"osd":7,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55200000000000005,"15min":0.58199999999999996},"min":{"1min":0.41399999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80500000000000005,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58799999999999997,"15min":0.58899999999999997},"min":{"1min":0.46000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78400000000000003,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.57399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":847578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.316,"15min":0.311},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.503,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.34899999999999998,"15min":0.35799999999999998},"min":{"1min":0.28399999999999997,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.51900000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.29599999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.34399999999999997,"15min":0.35099999999999998},"min":{"1min":0.215,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.53700000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.38300000000000001,"15min":0.38500000000000001},"min":{"1min":0.30299999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.55300000000000005,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.34399999999999997}]},{"osd":3,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.36499999999999999,"15min":0.36399999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.55800000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.34599999999999997,"15min":0.53500000000000003},"min":{"1min":0.23400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.40400000000000003,"5min":0.56000000000000005,"15min":5.8529999999999998},"last":0.39200000000000002}]},{"osd":4,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58299999999999996,"15min":0.57499999999999996},"min":{"1min":0.36199999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78300000000000003,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.40000000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.81100000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.64200000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57699999999999996,"15min":0.74099999999999999},"min":{"1min":0.36499999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88,"5min":0.88,"15min":5.9459999999999997},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61199999999999999,"15min":0.61099999999999999},"min":{"1min":0.38900000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82599999999999996,"5min":0.92700000000000005,"15min":0.93999999999999995},"last":0.66400000000000003}]},{"osd":6,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54900000000000004,"15min":0.71199999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79100000000000004,"5min":0.79600000000000004,"15min":5.9160000000000004},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.379,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84799999999999998,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.61299999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57199999999999995,"15min":0.58599999999999997},"min":{"1min":0.39000000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.80900000000000005,"5min":0.89500000000000002,"15min":0.97099999999999997},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57099999999999995,"15min":0.749},"min":{"1min":0.41399999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":5.9729999999999999},"last":0.60499999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574947,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303808,"kb_used_data":2424,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068032,"statfs":{"total":96636764160,"available":96325664768,"internally_reserved":0,"allocated":2482176,"data_stored":1431683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.30199999999999999,"15min":0.315},"min":{"1min":0.216,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.433,"5min":0.498,"15min":0.498},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.32100000000000001,"15min":0.34399999999999997},"min":{"1min":0.222,"5min":0.214,"15min":0.214},"max":{"1min":0.497,"5min":0.53600000000000003,"15min":0.94499999999999995},"last":0.34899999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35199999999999998,"15min":0.36199999999999999},"min":{"1min":0.25900000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.50900000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.371,"15min":0.38800000000000001},"min":{"1min":0.30499999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.56499999999999995,"5min":0.69399999999999995,"15min":0.91700000000000004},"last":0.47099999999999997}]},{"osd":3,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.32800000000000001,"15min":0.38400000000000001},"min":{"1min":0.23699999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":1.554},"last":0.499},{"interface":"front","average":{"1min":0.371,"5min":0.35999999999999999,"15min":0.39700000000000002},"min":{"1min":0.27700000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54400000000000004,"5min":0.58099999999999996,"15min":0.79000000000000004},"last":0.311}]},{"osd":4,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53600000000000003,"15min":0.55100000000000005},"min":{"1min":0.41199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":1.212},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.72099999999999997},"min":{"1min":0.42199999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":5.484},"last":0.77800000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.72499999999999998},"min":{"1min":0.41999999999999998,"5min":0.35799999999999998,"15min":0.35499999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":5.5449999999999999},"last":0.627},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.83299999999999996,"5min":0.84499999999999997,"15min":1.113},"last":0.52400000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":1.0529999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59299999999999997,"15min":0.60699999999999998},"min":{"1min":0.45800000000000002,"5min":0.42799999999999999,"15min":0.41599999999999998},"max":{"1min":0.878,"5min":0.878,"15min":1.169},"last":0.61099999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57099999999999995,"15min":0.58899999999999997},"min":{"1min":0.40600000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.78700000000000003,"5min":0.85499999999999998,"15min":1.2390000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56399999999999995,"15min":0.749},"min":{"1min":0.38,"5min":0.375,"15min":0.375},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":5.5800000000000001},"last":0.55800000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574948,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":847578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.316,"15min":0.35299999999999998},"min":{"1min":0.23200000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":1.4199999999999999},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.55900000000000005},"min":{"1min":0.27500000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.51600000000000001,"5min":0.53800000000000003,"15min":7.2699999999999996},"last":0.27800000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.317,"15min":0.34399999999999997},"min":{"1min":0.23999999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.65400000000000003},"last":0.25},{"interface":"front","average":{"1min":0.372,"5min":0.37,"15min":0.61299999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":7.327},"last":0.313}]},{"osd":3,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.36799999999999999,"15min":0.39100000000000001},"min":{"1min":0.29399999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.68600000000000005},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.36799999999999999,"15min":0.39200000000000002},"min":{"1min":0.27900000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.72699999999999998},"last":0.30199999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54500000000000004,"15min":0.54400000000000004},"min":{"1min":0.40799999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.78400000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56200000000000006,"15min":0.78600000000000003},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82699999999999996,"5min":0.84699999999999998,"15min":7.2389999999999999},"last":0.35999999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.58099999999999996,"15min":0.91500000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82699999999999996,"5min":0.83799999999999997,"15min":10.890000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":0.84299999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.51000000000000001}]},{"osd":6,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56399999999999995,"15min":0.79100000000000004},"min":{"1min":0.42999999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.748,"5min":0.77300000000000002,"15min":7.2999999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.45800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.433}]},{"osd":7,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59299999999999997,"15min":0.82199999999999995},"min":{"1min":0.47499999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":7.1879999999999997},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60399999999999998,"15min":0.624},"min":{"1min":0.49399999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.90500000000000003},"last":0.5}]}]},{"osd":3,"up_from":13,"seq":55834574947,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":847578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.32200000000000001,"15min":0.33200000000000002},"min":{"1min":0.23100000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":5.8700000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.34699999999999998,"15min":0.36699999999999999},"min":{"1min":0.23999999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":5.6769999999999996},"last":0.25600000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.35799999999999998,"15min":0.36099999999999999},"min":{"1min":0.26800000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":4.0469999999999997},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.372,"15min":0.38800000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":4.1509999999999998},"last":0.34799999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.35199999999999998,"15min":0.374},"min":{"1min":0.29999999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.50800000000000001,"5min":0.56799999999999995,"15min":3.1139999999999999},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.373,"15min":0.38600000000000001},"min":{"1min":0.28100000000000003,"5min":0.26100000000000001,"15min":0.25600000000000001},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":3.3180000000000001},"last":0.309}]},{"osd":4,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55800000000000005,"15min":0.57099999999999995},"min":{"1min":0.38100000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":3.0899999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56499999999999995,"15min":0.56799999999999995},"min":{"1min":0.40300000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76400000000000001,"5min":0.91100000000000003,"15min":3.0710000000000002},"last":0.443}]},{"osd":5,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57499999999999996,"15min":0.58899999999999997},"min":{"1min":0.39100000000000001,"5min":0.34599999999999997,"15min":0.32700000000000001},"max":{"1min":0.877,"5min":0.877,"15min":6.2300000000000004},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.441,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79800000000000004,"5min":0.89500000000000002,"15min":3.0219999999999998},"last":0.505}]},{"osd":6,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.44500000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":2.6850000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.65200000000000002},"min":{"1min":0.43099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":6.5970000000000004},"last":0.55000000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60699999999999998,"15min":0.61399999999999999},"min":{"1min":0.44700000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86099999999999999,"5min":0.93300000000000005,"15min":4.2629999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59999999999999998,"15min":0.61299999999999999},"min":{"1min":0.40500000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.82899999999999996,"5min":0.97199999999999998,"15min":7.3650000000000002},"last":0.56299999999999994}]}]},{"osd":4,"up_from":13,"seq":55834574946,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":847578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.51400000000000001,"15min":0.55300000000000005},"min":{"1min":0.318,"5min":0.28100000000000003,"15min":0.27000000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":1.083},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.35699999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.79900000000000004,"5min":0.97799999999999998,"15min":1.054},"last":0.52400000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.39900000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.501,"5min":0.50700000000000001,"15min":0.49399999999999999},"min":{"1min":0.33800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.76300000000000001,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.505}]},{"osd":2,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52700000000000002,"15min":0.53900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.32900000000000001},"max":{"1min":0.76600000000000001,"5min":0.82099999999999995,"15min":1.0249999999999999},"last":0.443},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55700000000000005,"15min":0.59099999999999997},"min":{"1min":0.39400000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.93799999999999994},"last":0.41999999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52900000000000003,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.32200000000000001,"15min":0.29199999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.98499999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57699999999999996,"15min":0.58799999999999997},"min":{"1min":0.38700000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.1319999999999999},"last":0.48399999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51600000000000001,"15min":0.55100000000000005},"min":{"1min":0.31900000000000001,"5min":0.30399999999999999,"15min":0.27300000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58199999999999996,"15min":0.60599999999999998},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.35099999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.97999999999999998},"last":0.54300000000000004}]},{"osd":6,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.56799999999999995},"min":{"1min":0.41599999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.76800000000000002,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.59799999999999998},"min":{"1min":0.38700000000000001,"5min":0.34999999999999998,"15min":0.32700000000000001},"max":{"1min":0.85599999999999998,"5min":1.01,"15min":1.01},"last":0.64200000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54300000000000004,"15min":0.56999999999999995},"min":{"1min":0.40200000000000002,"5min":0.33600000000000002,"15min":0.28299999999999997},"max":{"1min":0.78900000000000003,"5min":0.85599999999999998,"15min":0.90000000000000002},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.60699999999999998,"15min":0.64500000000000002},"min":{"1min":0.36699999999999999,"5min":0.34999999999999998,"15min":0.32200000000000001},"max":{"1min":0.89000000000000001,"5min":0.98999999999999999,"15min":1.175},"last":0.627}]}]},{"osd":5,"up_from":13,"seq":55834574946,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303240,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068600,"statfs":{"total":96636764160,"available":96326246400,"internally_reserved":0,"allocated":1900544,"data_stored":847578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.503,"15min":0.50800000000000001},"min":{"1min":0.38800000000000001,"5min":0.29299999999999998,"15min":0.28599999999999998},"max":{"1min":0.67900000000000005,"5min":0.71399999999999997,"15min":0.86599999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51800000000000002,"15min":0.51400000000000001},"min":{"1min":0.42499999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.60599999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.496}]},{"osd":1,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.38400000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.63900000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52300000000000002,"15min":0.53100000000000003},"min":{"1min":0.43099999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72799999999999998,"5min":0.73899999999999999,"15min":0.81599999999999995},"last":0.47999999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54300000000000004,"15min":0.54400000000000004},"min":{"1min":0.41099999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.63900000000000001,"5min":0.72999999999999998,"15min":0.83999999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.47199999999999998,"5min":0.40999999999999998,"15min":0.39500000000000002},"max":{"1min":0.72699999999999998,"5min":0.74099999999999999,"15min":0.79900000000000004},"last":0.53600000000000003}]},{"osd":3,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.42399999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.66900000000000004,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.54900000000000004,"15min":0.56000000000000005},"min":{"1min":0.42499999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.63500000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.55800000000000005}]},{"osd":4,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.50600000000000001,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.36199999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.80700000000000005},"last":0.433}]},{"osd":6,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55300000000000005,"15min":0.53600000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.83199999999999996},"last":0.378},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.36199999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77900000000000003,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.40200000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.57899999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71499999999999997,"5min":0.79500000000000004,"15min":0.85599999999999998},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.374,"5min":0.35199999999999998,"15min":0.314},"max":{"1min":0.79500000000000004,"5min":0.84499999999999997,"15min":0.90600000000000003},"last":0.44700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:38:29.246 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:38:29.374 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.374+0000 7f56b9dae700 1 Processor -- start 2023-10-09T06:38:29.374 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.374+0000 7f56b9dae700 1 -- start start 2023-10-09T06:38:29.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.375+0000 7f56b9dae700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f56b406c0f0 0x7f56b40ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:29.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.375+0000 7f56b9dae700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56b41410a0 0x7f56b40cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:29.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.375+0000 7f56b9dae700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56b40b1500 con 0x7f56b4140cd0 2023-10-09T06:38:29.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.375+0000 7f56b9dae700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56b40cd790 con 0x7f56b406c0f0 2023-10-09T06:38:29.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.375+0000 7f56b9dae700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b40cda20 con 0x7f56b41410a0 2023-10-09T06:38:29.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.375+0000 7f56b95ad700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56b41410a0 0x7f56b40cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:29.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.375+0000 7f56abfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f56b406c0f0 0x7f56b40ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:29.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.375+0000 7f56b8dac700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f56b4140cd0 0x7f56b406b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:43932/0 (socket says 172.21.15.163:43932) 2023-10-09T06:38:29.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.375+0000 7f56b95ad700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56b41410a0 0x7f56b40cd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:43006/0 (socket says 172.21.15.163:43006) 2023-10-09T06:38:29.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.375+0000 7f56abfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f56b406c0f0 0x7f56b40ccd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:51724/0 (socket says 172.21.15.163:51724) 2023-10-09T06:38:29.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.375+0000 7f56b8dac700 1 -- 172.21.15.163:0/3952842372 learned_addr learned my addr 172.21.15.163:0/3952842372 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:29.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.376+0000 7f56ab7fe700 1 -- 172.21.15.163:0/3952842372 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1935685673 0 0) 0x7f56b40b1500 con 0x7f56b4140cd0 2023-10-09T06:38:29.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.376+0000 7f56ab7fe700 1 -- 172.21.15.163:0/3952842372 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f569c003570 con 0x7f56b4140cd0 2023-10-09T06:38:29.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.376+0000 7f56b95ad700 1 -- 172.21.15.163:0/3952842372 >> v2:172.21.15.181:3300/0 conn(0x7f56b406c0f0 msgr2=0x7f56b40ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:29.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.376+0000 7f56b95ad700 1 --2- 172.21.15.163:0/3952842372 >> v2:172.21.15.181:3300/0 conn(0x7f56b406c0f0 0x7f56b40ccd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:29.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.376+0000 7f56b95ad700 1 -- 172.21.15.163:0/3952842372 >> v1:172.21.15.181:6789/0 conn(0x7f56b4140cd0 legacy=0x7f56b406b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:29.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.376+0000 7f56b95ad700 1 -- 172.21.15.163:0/3952842372 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b40cdd00 con 0x7f56b41410a0 2023-10-09T06:38:29.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.376+0000 7f56abfff700 1 --2- 172.21.15.163:0/3952842372 >> v2:172.21.15.181:3300/0 conn(0x7f56b406c0f0 0x7f56b40ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:38:29.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.376+0000 7f56b95ad700 1 --2- 172.21.15.163:0/3952842372 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56b41410a0 0x7f56b40cd250 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009e90 tx=0x7f56ac009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e6f37e9f5c4b193 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:29.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.376+0000 7f56ab7fe700 1 -- 172.21.15.163:0/3952842372 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac00e070 con 0x7f56b41410a0 2023-10-09T06:38:29.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.376+0000 7f56ab7fe700 1 -- 172.21.15.163:0/3952842372 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56ac00bb40 con 0x7f56b41410a0 2023-10-09T06:38:29.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.376+0000 7f56ab7fe700 1 -- 172.21.15.163:0/3952842372 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac012430 con 0x7f56b41410a0 2023-10-09T06:38:29.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.377+0000 7f56b9dae700 1 -- 172.21.15.163:0/3952842372 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56b41410a0 msgr2=0x7f56b40cd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:29.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.377+0000 7f56b9dae700 1 --2- 172.21.15.163:0/3952842372 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56b41410a0 0x7f56b40cd250 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009e90 tx=0x7f56ac009780 comp rx=0 tx=0).stop 2023-10-09T06:38:29.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.377+0000 7f56b9dae700 1 -- 172.21.15.163:0/3952842372 shutdown_connections 2023-10-09T06:38:29.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.377+0000 7f56b9dae700 1 --2- 172.21.15.163:0/3952842372 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56b41410a0 0x7f56b40cd250 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:29.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.377+0000 7f56b9dae700 1 --2- 172.21.15.163:0/3952842372 >> v2:172.21.15.181:3300/0 conn(0x7f56b406c0f0 0x7f56b40ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:29.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.377+0000 7f56b9dae700 1 -- 172.21.15.163:0/3952842372 >> 172.21.15.163:0/3952842372 conn(0x7f56b40ce430 msgr2=0x7f56b40ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:29.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.377+0000 7f56b9dae700 1 -- 172.21.15.163:0/3952842372 shutdown_connections 2023-10-09T06:38:29.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.377+0000 7f56b9dae700 1 -- 172.21.15.163:0/3952842372 wait complete. 2023-10-09T06:38:29.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.378+0000 7f56b9dae700 1 Processor -- start 2023-10-09T06:38:29.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.378+0000 7f56b9dae700 1 -- start start 2023-10-09T06:38:29.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.379+0000 7f56b9dae700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56b406c0f0 0x7f56b40addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:29.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.379+0000 7f56b9dae700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56b4140cd0 0x7f56b40b0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:29.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.379+0000 7f56b9dae700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56b41410a0 0x7f56b415c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:29.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.379+0000 7f56b9dae700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b40ae3d0 con 0x7f56b41410a0 2023-10-09T06:38:29.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.379+0000 7f56b9dae700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b40afb40 con 0x7f56b4140cd0 2023-10-09T06:38:29.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.379+0000 7f56b9dae700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b40b0ab0 con 0x7f56b406c0f0 2023-10-09T06:38:29.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.379+0000 7f56b95ad700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56b41410a0 0x7f56b415c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:29.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.379+0000 7f56b95ad700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56b41410a0 0x7f56b415c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46936/0 (socket says 172.21.15.163:46936) 2023-10-09T06:38:29.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.379+0000 7f56b95ad700 1 -- 172.21.15.163:0/1127828711 learned_addr learned my addr 172.21.15.163:0/1127828711 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:29.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.379+0000 7f56b8dac700 1 --2- 172.21.15.163:0/1127828711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56b406c0f0 0x7f56b40addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:29.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.379+0000 7f56abfff700 1 --2- 172.21.15.163:0/1127828711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56b4140cd0 0x7f56b40b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:29.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.379+0000 7f56b95ad700 1 -- 172.21.15.163:0/1127828711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56b406c0f0 msgr2=0x7f56b40addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:29.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.379+0000 7f56b95ad700 1 --2- 172.21.15.163:0/1127828711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56b406c0f0 0x7f56b40addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:29.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.379+0000 7f56b95ad700 1 -- 172.21.15.163:0/1127828711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56b4140cd0 msgr2=0x7f56b40b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:29.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.379+0000 7f56b95ad700 1 --2- 172.21.15.163:0/1127828711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56b4140cd0 0x7f56b40b0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:29.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.379+0000 7f56b95ad700 1 -- 172.21.15.163:0/1127828711 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a4009420 con 0x7f56b41410a0 2023-10-09T06:38:29.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.379+0000 7f56abfff700 1 --2- 172.21.15.163:0/1127828711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56b4140cd0 0x7f56b40b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:29.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.379+0000 7f56b95ad700 1 --2- 172.21.15.163:0/1127828711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56b41410a0 0x7f56b415c130 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f56b40cdb60 tx=0x7f56ac019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:29.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.379+0000 7f56a8ff9700 1 -- 172.21.15.163:0/1127828711 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac009420 con 0x7f56b41410a0 2023-10-09T06:38:29.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.380+0000 7f56a8ff9700 1 -- 172.21.15.163:0/1127828711 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56ac027470 con 0x7f56b41410a0 2023-10-09T06:38:29.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.380+0000 7f56b9dae700 1 -- 172.21.15.163:0/1127828711 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56ac009a20 con 0x7f56b41410a0 2023-10-09T06:38:29.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.380+0000 7f56a8ff9700 1 -- 172.21.15.163:0/1127828711 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac027810 con 0x7f56b41410a0 2023-10-09T06:38:29.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.380+0000 7f56b9dae700 1 -- 172.21.15.163:0/1127828711 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b40af480 con 0x7f56b41410a0 2023-10-09T06:38:29.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.380+0000 7f56b9dae700 1 -- 172.21.15.163:0/1127828711 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b404c3c0 con 0x7f56b41410a0 2023-10-09T06:38:29.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.381+0000 7f56a8ff9700 1 -- 172.21.15.163:0/1127828711 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f56ac024070 con 0x7f56b41410a0 2023-10-09T06:38:29.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.381+0000 7f56a8ff9700 1 --2- 172.21.15.163:0/1127828711 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f56a0067320 0x7f56a00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:29.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.381+0000 7f56a8ff9700 1 -- 172.21.15.163:0/1127828711 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f56ac087370 con 0x7f56b41410a0 2023-10-09T06:38:29.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.382+0000 7f56b8dac700 1 --2- 172.21.15.163:0/1127828711 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f56a0067320 0x7f56a00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:29.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.382+0000 7f56b8dac700 1 --2- 172.21.15.163:0/1127828711 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f56a0067320 0x7f56a00697d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f56b0007fd0 tx=0x7f56b0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:29.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.386+0000 7f56a8ff9700 1 -- 172.21.15.163:0/1127828711 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f56ac054000 con 0x7f56b41410a0 2023-10-09T06:38:29.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.564+0000 7f56b9dae700 1 -- 172.21.15.163:0/1127828711 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f56b40af820 con 0x7f56a0067320 2023-10-09T06:38:29.572 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.573+0000 7f56a8ff9700 1 -- 172.21.15.163:0/1127828711 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290325 (secure 0 0 0) 0x7f56b40af820 con 0x7f56a0067320 2023-10-09T06:38:29.576 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:29.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.576+0000 7f56b9dae700 1 -- 172.21.15.163:0/1127828711 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f56a0067320 msgr2=0x7f56a00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:29.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.576+0000 7f56b9dae700 1 --2- 172.21.15.163:0/1127828711 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f56a0067320 0x7f56a00697d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f56b0007fd0 tx=0x7f56b0009040 comp rx=0 tx=0).stop 2023-10-09T06:38:29.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.576+0000 7f56b9dae700 1 -- 172.21.15.163:0/1127828711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56b41410a0 msgr2=0x7f56b415c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:29.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.576+0000 7f56b9dae700 1 --2- 172.21.15.163:0/1127828711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56b41410a0 0x7f56b415c130 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f56b40cdb60 tx=0x7f56ac019000 comp rx=0 tx=0).stop 2023-10-09T06:38:29.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.576+0000 7f56b9dae700 1 -- 172.21.15.163:0/1127828711 shutdown_connections 2023-10-09T06:38:29.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.576+0000 7f56b9dae700 1 --2- 172.21.15.163:0/1127828711 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f56a0067320 0x7f56a00697d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:29.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.576+0000 7f56b9dae700 1 --2- 172.21.15.163:0/1127828711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56b41410a0 0x7f56b415c130 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:29.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.576+0000 7f56b9dae700 1 --2- 172.21.15.163:0/1127828711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56b406c0f0 0x7f56b40addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:29.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.576+0000 7f56b9dae700 1 --2- 172.21.15.163:0/1127828711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56b4140cd0 0x7f56b40b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:29.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.576+0000 7f56b9dae700 1 -- 172.21.15.163:0/1127828711 >> 172.21.15.163:0/1127828711 conn(0x7f56b40ce430 msgr2=0x7f56b413d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:29.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.577+0000 7f56b9dae700 1 -- 172.21.15.163:0/1127828711 shutdown_connections 2023-10-09T06:38:29.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.577+0000 7f56b9dae700 1 -- 172.21.15.163:0/1127828711 wait complete. 2023-10-09T06:38:29.579 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:38:29.588 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2023-10-09T06:38:28.412187+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2427336,"kb_used_data":16264,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752547384,"statfs":{"total":773094113280,"available":770608521216,"internally_reserved":0,"allocated":16654336,"data_stored":8070380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002170"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.698539+0000","last_change":"2023-10-09T06:38:20.694283+0000","last_active":"2023-10-09T06:38:20.698539+0000","last_peered":"2023-10-09T06:38:20.698539+0000","last_clean":"2023-10-09T06:38:20.698539+0000","last_became_active":"2023-10-09T06:38:20.694023+0000","last_became_peered":"2023-10-09T06:38:20.694023+0000","last_unstale":"2023-10-09T06:38:20.698539+0000","last_undegraded":"2023-10-09T06:38:20.698539+0000","last_fullsized":"2023-10-09T06:38:20.698539+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:25:07.106994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.699834+0000","last_change":"2023-10-09T06:38:20.692248+0000","last_active":"2023-10-09T06:38:20.699834+0000","last_peered":"2023-10-09T06:38:20.699834+0000","last_clean":"2023-10-09T06:38:20.699834+0000","last_became_active":"2023-10-09T06:38:20.691962+0000","last_became_peered":"2023-10-09T06:38:20.691962+0000","last_unstale":"2023-10-09T06:38:20.699834+0000","last_undegraded":"2023-10-09T06:38:20.699834+0000","last_fullsized":"2023-10-09T06:38:20.699834+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:15:52.880306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.700849+0000","last_change":"2023-10-09T06:38:20.695692+0000","last_active":"2023-10-09T06:38:20.700849+0000","last_peered":"2023-10-09T06:38:20.700849+0000","last_clean":"2023-10-09T06:38:20.700849+0000","last_became_active":"2023-10-09T06:38:20.695456+0000","last_became_peered":"2023-10-09T06:38:20.695456+0000","last_unstale":"2023-10-09T06:38:20.700849+0000","last_undegraded":"2023-10-09T06:38:20.700849+0000","last_fullsized":"2023-10-09T06:38:20.700849+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:23:58.653210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.696438+0000","last_change":"2023-10-09T06:38:20.692753+0000","last_active":"2023-10-09T06:38:20.696438+0000","last_peered":"2023-10-09T06:38:20.696438+0000","last_clean":"2023-10-09T06:38:20.696438+0000","last_became_active":"2023-10-09T06:38:20.692403+0000","last_became_peered":"2023-10-09T06:38:20.692403+0000","last_unstale":"2023-10-09T06:38:20.696438+0000","last_undegraded":"2023-10-09T06:38:20.696438+0000","last_fullsized":"2023-10-09T06:38:20.696438+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:41:30.382056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.707162+0000","last_change":"2023-10-09T06:38:20.700807+0000","last_active":"2023-10-09T06:38:20.707162+0000","last_peered":"2023-10-09T06:38:20.707162+0000","last_clean":"2023-10-09T06:38:20.707162+0000","last_became_active":"2023-10-09T06:38:20.700551+0000","last_became_peered":"2023-10-09T06:38:20.700551+0000","last_unstale":"2023-10-09T06:38:20.707162+0000","last_undegraded":"2023-10-09T06:38:20.707162+0000","last_fullsized":"2023-10-09T06:38:20.707162+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:46:35.635953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.707488+0000","last_change":"2023-10-09T06:38:20.702019+0000","last_active":"2023-10-09T06:38:20.707488+0000","last_peered":"2023-10-09T06:38:20.707488+0000","last_clean":"2023-10-09T06:38:20.707488+0000","last_became_active":"2023-10-09T06:38:20.701741+0000","last_became_peered":"2023-10-09T06:38:20.701741+0000","last_unstale":"2023-10-09T06:38:20.707488+0000","last_undegraded":"2023-10-09T06:38:20.707488+0000","last_fullsized":"2023-10-09T06:38:20.707488+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:54:10.755034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.699438+0000","last_change":"2023-10-09T06:38:20.687232+0000","last_active":"2023-10-09T06:38:20.699438+0000","last_peered":"2023-10-09T06:38:20.699438+0000","last_clean":"2023-10-09T06:38:20.699438+0000","last_became_active":"2023-10-09T06:38:20.686971+0000","last_became_peered":"2023-10-09T06:38:20.686971+0000","last_unstale":"2023-10-09T06:38:20.699438+0000","last_undegraded":"2023-10-09T06:38:20.699438+0000","last_fullsized":"2023-10-09T06:38:20.699438+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:48:16.228608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.707491+0000","last_change":"2023-10-09T06:38:20.702146+0000","last_active":"2023-10-09T06:38:20.707491+0000","last_peered":"2023-10-09T06:38:20.707491+0000","last_clean":"2023-10-09T06:38:20.707491+0000","last_became_active":"2023-10-09T06:38:20.701910+0000","last_became_peered":"2023-10-09T06:38:20.701910+0000","last_unstale":"2023-10-09T06:38:20.707491+0000","last_undegraded":"2023-10-09T06:38:20.707491+0000","last_fullsized":"2023-10-09T06:38:20.707491+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:44:35.145850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.701855+0000","last_change":"2023-10-09T06:38:20.696972+0000","last_active":"2023-10-09T06:38:20.701855+0000","last_peered":"2023-10-09T06:38:20.701855+0000","last_clean":"2023-10-09T06:38:20.701855+0000","last_became_active":"2023-10-09T06:38:20.696736+0000","last_became_peered":"2023-10-09T06:38:20.696736+0000","last_unstale":"2023-10-09T06:38:20.701855+0000","last_undegraded":"2023-10-09T06:38:20.701855+0000","last_fullsized":"2023-10-09T06:38:20.701855+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:43:32.088320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.707194+0000","last_change":"2023-10-09T06:38:20.704300+0000","last_active":"2023-10-09T06:38:20.707194+0000","last_peered":"2023-10-09T06:38:20.707194+0000","last_clean":"2023-10-09T06:38:20.707194+0000","last_became_active":"2023-10-09T06:38:20.701018+0000","last_became_peered":"2023-10-09T06:38:20.701018+0000","last_unstale":"2023-10-09T06:38:20.707194+0000","last_undegraded":"2023-10-09T06:38:20.707194+0000","last_fullsized":"2023-10-09T06:38:20.707194+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:15:57.542772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.700828+0000","last_change":"2023-10-09T06:38:20.695106+0000","last_active":"2023-10-09T06:38:20.700828+0000","last_peered":"2023-10-09T06:38:20.700828+0000","last_clean":"2023-10-09T06:38:20.700828+0000","last_became_active":"2023-10-09T06:38:20.694858+0000","last_became_peered":"2023-10-09T06:38:20.694858+0000","last_unstale":"2023-10-09T06:38:20.700828+0000","last_undegraded":"2023-10-09T06:38:20.700828+0000","last_fullsized":"2023-10-09T06:38:20.700828+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:05:32.721283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.703499+0000","last_change":"2023-10-09T06:38:20.702368+0000","last_active":"2023-10-09T06:38:20.703499+0000","last_peered":"2023-10-09T06:38:20.703499+0000","last_clean":"2023-10-09T06:38:20.703499+0000","last_became_active":"2023-10-09T06:38:20.702045+0000","last_became_peered":"2023-10-09T06:38:20.702045+0000","last_unstale":"2023-10-09T06:38:20.703499+0000","last_undegraded":"2023-10-09T06:38:20.703499+0000","last_fullsized":"2023-10-09T06:38:20.703499+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:08:34.030754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.694410+0000","last_change":"2023-10-09T06:38:20.690722+0000","last_active":"2023-10-09T06:38:20.694410+0000","last_peered":"2023-10-09T06:38:20.694410+0000","last_clean":"2023-10-09T06:38:20.694410+0000","last_became_active":"2023-10-09T06:38:20.690444+0000","last_became_peered":"2023-10-09T06:38:20.690444+0000","last_unstale":"2023-10-09T06:38:20.694410+0000","last_undegraded":"2023-10-09T06:38:20.694410+0000","last_fullsized":"2023-10-09T06:38:20.694410+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:42:48.250267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.690218+0000","last_change":"2023-10-09T06:38:20.686127+0000","last_active":"2023-10-09T06:38:20.690218+0000","last_peered":"2023-10-09T06:38:20.690218+0000","last_clean":"2023-10-09T06:38:20.690218+0000","last_became_active":"2023-10-09T06:38:20.685876+0000","last_became_peered":"2023-10-09T06:38:20.685876+0000","last_unstale":"2023-10-09T06:38:20.690218+0000","last_undegraded":"2023-10-09T06:38:20.690218+0000","last_fullsized":"2023-10-09T06:38:20.690218+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:59:11.725076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.699458+0000","last_change":"2023-10-09T06:38:20.693050+0000","last_active":"2023-10-09T06:38:20.699458+0000","last_peered":"2023-10-09T06:38:20.699458+0000","last_clean":"2023-10-09T06:38:20.699458+0000","last_became_active":"2023-10-09T06:38:20.689963+0000","last_became_peered":"2023-10-09T06:38:20.689963+0000","last_unstale":"2023-10-09T06:38:20.699458+0000","last_undegraded":"2023-10-09T06:38:20.699458+0000","last_fullsized":"2023-10-09T06:38:20.699458+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:04:31.978834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.700543+0000","last_change":"2023-10-09T06:38:20.697956+0000","last_active":"2023-10-09T06:38:20.700543+0000","last_peered":"2023-10-09T06:38:20.700543+0000","last_clean":"2023-10-09T06:38:20.700543+0000","last_became_active":"2023-10-09T06:38:20.697679+0000","last_became_peered":"2023-10-09T06:38:20.697679+0000","last_unstale":"2023-10-09T06:38:20.700543+0000","last_undegraded":"2023-10-09T06:38:20.700543+0000","last_fullsized":"2023-10-09T06:38:20.700543+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:54:10.755034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.698522+0000","last_change":"2023-10-09T06:38:20.694243+0000","last_active":"2023-10-09T06:38:20.698522+0000","last_peered":"2023-10-09T06:38:20.698522+0000","last_clean":"2023-10-09T06:38:20.698522+0000","last_became_active":"2023-10-09T06:38:20.693960+0000","last_became_peered":"2023-10-09T06:38:20.693960+0000","last_unstale":"2023-10-09T06:38:20.698522+0000","last_undegraded":"2023-10-09T06:38:20.698522+0000","last_fullsized":"2023-10-09T06:38:20.698522+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:52:53.328173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.697824+0000","last_change":"2023-10-09T06:38:20.692830+0000","last_active":"2023-10-09T06:38:20.697824+0000","last_peered":"2023-10-09T06:38:20.697824+0000","last_clean":"2023-10-09T06:38:20.697824+0000","last_became_active":"2023-10-09T06:38:20.692561+0000","last_became_peered":"2023-10-09T06:38:20.692561+0000","last_unstale":"2023-10-09T06:38:20.697824+0000","last_undegraded":"2023-10-09T06:38:20.697824+0000","last_fullsized":"2023-10-09T06:38:20.697824+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:48:16.228608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.707231+0000","last_change":"2023-10-09T06:38:20.705065+0000","last_active":"2023-10-09T06:38:20.707231+0000","last_peered":"2023-10-09T06:38:20.707231+0000","last_clean":"2023-10-09T06:38:20.707231+0000","last_became_active":"2023-10-09T06:38:20.704775+0000","last_became_peered":"2023-10-09T06:38:20.704775+0000","last_unstale":"2023-10-09T06:38:20.707231+0000","last_undegraded":"2023-10-09T06:38:20.707231+0000","last_fullsized":"2023-10-09T06:38:20.707231+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:16:56.138017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.707134+0000","last_change":"2023-10-09T06:38:20.700196+0000","last_active":"2023-10-09T06:38:20.707134+0000","last_peered":"2023-10-09T06:38:20.707134+0000","last_clean":"2023-10-09T06:38:20.707134+0000","last_became_active":"2023-10-09T06:38:20.699954+0000","last_became_peered":"2023-10-09T06:38:20.699954+0000","last_unstale":"2023-10-09T06:38:20.707134+0000","last_undegraded":"2023-10-09T06:38:20.707134+0000","last_fullsized":"2023-10-09T06:38:20.707134+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:44:51.812594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.696127+0000","last_change":"2023-10-09T06:38:20.692189+0000","last_active":"2023-10-09T06:38:20.696127+0000","last_peered":"2023-10-09T06:38:20.696127+0000","last_clean":"2023-10-09T06:38:20.696127+0000","last_became_active":"2023-10-09T06:38:20.691860+0000","last_became_peered":"2023-10-09T06:38:20.691860+0000","last_unstale":"2023-10-09T06:38:20.696127+0000","last_undegraded":"2023-10-09T06:38:20.696127+0000","last_fullsized":"2023-10-09T06:38:20.696127+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:01:29.075717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.701516+0000","last_change":"2023-10-09T06:38:20.699259+0000","last_active":"2023-10-09T06:38:20.701516+0000","last_peered":"2023-10-09T06:38:20.701516+0000","last_clean":"2023-10-09T06:38:20.701516+0000","last_became_active":"2023-10-09T06:38:20.698992+0000","last_became_peered":"2023-10-09T06:38:20.698992+0000","last_unstale":"2023-10-09T06:38:20.701516+0000","last_undegraded":"2023-10-09T06:38:20.701516+0000","last_fullsized":"2023-10-09T06:38:20.701516+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:58:43.534501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.701048+0000","last_change":"2023-10-09T06:38:20.694984+0000","last_active":"2023-10-09T06:38:20.701048+0000","last_peered":"2023-10-09T06:38:20.701048+0000","last_clean":"2023-10-09T06:38:20.701048+0000","last_became_active":"2023-10-09T06:38:20.694621+0000","last_became_peered":"2023-10-09T06:38:20.694621+0000","last_unstale":"2023-10-09T06:38:20.701048+0000","last_undegraded":"2023-10-09T06:38:20.701048+0000","last_fullsized":"2023-10-09T06:38:20.701048+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:16:31.813113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.696175+0000","last_change":"2023-10-09T06:38:20.691950+0000","last_active":"2023-10-09T06:38:20.696175+0000","last_peered":"2023-10-09T06:38:20.696175+0000","last_clean":"2023-10-09T06:38:20.696175+0000","last_became_active":"2023-10-09T06:38:20.691549+0000","last_became_peered":"2023-10-09T06:38:20.691549+0000","last_unstale":"2023-10-09T06:38:20.696175+0000","last_undegraded":"2023-10-09T06:38:20.696175+0000","last_fullsized":"2023-10-09T06:38:20.696175+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:36:50.516609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.702888+0000","last_change":"2023-10-09T06:38:20.700638+0000","last_active":"2023-10-09T06:38:20.702888+0000","last_peered":"2023-10-09T06:38:20.702888+0000","last_clean":"2023-10-09T06:38:20.702888+0000","last_became_active":"2023-10-09T06:38:20.700356+0000","last_became_peered":"2023-10-09T06:38:20.700356+0000","last_unstale":"2023-10-09T06:38:20.702888+0000","last_undegraded":"2023-10-09T06:38:20.702888+0000","last_fullsized":"2023-10-09T06:38:20.702888+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:19:50.331458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.690074+0000","last_change":"2023-10-09T06:38:20.684561+0000","last_active":"2023-10-09T06:38:20.690074+0000","last_peered":"2023-10-09T06:38:20.690074+0000","last_clean":"2023-10-09T06:38:20.690074+0000","last_became_active":"2023-10-09T06:38:20.684322+0000","last_became_peered":"2023-10-09T06:38:20.684322+0000","last_unstale":"2023-10-09T06:38:20.690074+0000","last_undegraded":"2023-10-09T06:38:20.690074+0000","last_fullsized":"2023-10-09T06:38:20.690074+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:42:23.336035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.704680+0000","last_change":"2023-10-09T06:38:20.703586+0000","last_active":"2023-10-09T06:38:20.704680+0000","last_peered":"2023-10-09T06:38:20.704680+0000","last_clean":"2023-10-09T06:38:20.704680+0000","last_became_active":"2023-10-09T06:38:20.703291+0000","last_became_peered":"2023-10-09T06:38:20.703291+0000","last_unstale":"2023-10-09T06:38:20.704680+0000","last_undegraded":"2023-10-09T06:38:20.704680+0000","last_fullsized":"2023-10-09T06:38:20.704680+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:20:02.974752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.706633+0000","last_change":"2023-10-09T06:38:20.703800+0000","last_active":"2023-10-09T06:38:20.706633+0000","last_peered":"2023-10-09T06:38:20.706633+0000","last_clean":"2023-10-09T06:38:20.706633+0000","last_became_active":"2023-10-09T06:38:20.703558+0000","last_became_peered":"2023-10-09T06:38:20.703558+0000","last_unstale":"2023-10-09T06:38:20.706633+0000","last_undegraded":"2023-10-09T06:38:20.706633+0000","last_fullsized":"2023-10-09T06:38:20.706633+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:52:31.336330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.703741+0000","last_change":"2023-10-09T06:38:20.702904+0000","last_active":"2023-10-09T06:38:20.703741+0000","last_peered":"2023-10-09T06:38:20.703741+0000","last_clean":"2023-10-09T06:38:20.703741+0000","last_became_active":"2023-10-09T06:38:20.702646+0000","last_became_peered":"2023-10-09T06:38:20.702646+0000","last_unstale":"2023-10-09T06:38:20.703741+0000","last_undegraded":"2023-10-09T06:38:20.703741+0000","last_fullsized":"2023-10-09T06:38:20.703741+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:40:00.965033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.704744+0000","last_change":"2023-10-09T06:38:20.703414+0000","last_active":"2023-10-09T06:38:20.704744+0000","last_peered":"2023-10-09T06:38:20.704744+0000","last_clean":"2023-10-09T06:38:20.704744+0000","last_became_active":"2023-10-09T06:38:20.703091+0000","last_became_peered":"2023-10-09T06:38:20.703091+0000","last_unstale":"2023-10-09T06:38:20.704744+0000","last_undegraded":"2023-10-09T06:38:20.704744+0000","last_fullsized":"2023-10-09T06:38:20.704744+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:50:47.758995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.697977+0000","last_change":"2023-10-09T06:38:20.697187+0000","last_active":"2023-10-09T06:38:20.697977+0000","last_peered":"2023-10-09T06:38:20.697977+0000","last_clean":"2023-10-09T06:38:20.697977+0000","last_became_active":"2023-10-09T06:38:20.696941+0000","last_became_peered":"2023-10-09T06:38:20.696941+0000","last_unstale":"2023-10-09T06:38:20.697977+0000","last_undegraded":"2023-10-09T06:38:20.697977+0000","last_fullsized":"2023-10-09T06:38:20.697977+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:34:55.917445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.698678+0000","last_change":"2023-10-09T06:38:20.695497+0000","last_active":"2023-10-09T06:38:20.698678+0000","last_peered":"2023-10-09T06:38:20.698678+0000","last_clean":"2023-10-09T06:38:20.698678+0000","last_became_active":"2023-10-09T06:38:20.695238+0000","last_became_peered":"2023-10-09T06:38:20.695238+0000","last_unstale":"2023-10-09T06:38:20.698678+0000","last_undegraded":"2023-10-09T06:38:20.698678+0000","last_fullsized":"2023-10-09T06:38:20.698678+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:54:10.755034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.699854+0000","last_change":"2023-10-09T06:38:20.692799+0000","last_active":"2023-10-09T06:38:20.699854+0000","last_peered":"2023-10-09T06:38:20.699854+0000","last_clean":"2023-10-09T06:38:20.699854+0000","last_became_active":"2023-10-09T06:38:20.692495+0000","last_became_peered":"2023-10-09T06:38:20.692495+0000","last_unstale":"2023-10-09T06:38:20.699854+0000","last_undegraded":"2023-10-09T06:38:20.699854+0000","last_fullsized":"2023-10-09T06:38:20.699854+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:58:43.534501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"151'4","reported_seq":14,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.704876+0000","last_change":"2023-10-09T06:38:19.691611+0000","last_active":"2023-10-09T06:38:20.704876+0000","last_peered":"2023-10-09T06:38:20.704876+0000","last_clean":"2023-10-09T06:38:20.704876+0000","last_became_active":"2023-10-09T06:38:19.691367+0000","last_became_peered":"2023-10-09T06:38:19.691367+0000","last_unstale":"2023-10-09T06:38:20.704876+0000","last_undegraded":"2023-10-09T06:38:20.704876+0000","last_fullsized":"2023-10-09T06:38:20.704876+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:13:34.033630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.691956+0000","last_change":"2023-10-09T06:38:20.684098+0000","last_active":"2023-10-09T06:38:20.691956+0000","last_peered":"2023-10-09T06:38:20.691956+0000","last_clean":"2023-10-09T06:38:20.691956+0000","last_became_active":"2023-10-09T06:38:20.683792+0000","last_became_peered":"2023-10-09T06:38:20.683792+0000","last_unstale":"2023-10-09T06:38:20.691956+0000","last_undegraded":"2023-10-09T06:38:20.691956+0000","last_fullsized":"2023-10-09T06:38:20.691956+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:43:13.592368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"151'1","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.703745+0000","last_change":"2023-10-09T06:38:19.694250+0000","last_active":"2023-10-09T06:38:20.703745+0000","last_peered":"2023-10-09T06:38:20.703745+0000","last_clean":"2023-10-09T06:38:20.703745+0000","last_became_active":"2023-10-09T06:38:19.693478+0000","last_became_peered":"2023-10-09T06:38:19.693478+0000","last_unstale":"2023-10-09T06:38:20.703745+0000","last_undegraded":"2023-10-09T06:38:20.703745+0000","last_fullsized":"2023-10-09T06:38:20.703745+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:50:27.283148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.690045+0000","last_change":"2023-10-09T06:38:20.684080+0000","last_active":"2023-10-09T06:38:20.690045+0000","last_peered":"2023-10-09T06:38:20.690045+0000","last_clean":"2023-10-09T06:38:20.690045+0000","last_became_active":"2023-10-09T06:38:20.683822+0000","last_became_peered":"2023-10-09T06:38:20.683822+0000","last_unstale":"2023-10-09T06:38:20.690045+0000","last_undegraded":"2023-10-09T06:38:20.690045+0000","last_fullsized":"2023-10-09T06:38:20.690045+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:09:51.130631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"151'2","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.703265+0000","last_change":"2023-10-09T06:38:19.688925+0000","last_active":"2023-10-09T06:38:20.703265+0000","last_peered":"2023-10-09T06:38:20.703265+0000","last_clean":"2023-10-09T06:38:20.703265+0000","last_became_active":"2023-10-09T06:38:19.688479+0000","last_became_peered":"2023-10-09T06:38:19.688479+0000","last_unstale":"2023-10-09T06:38:20.703265+0000","last_undegraded":"2023-10-09T06:38:20.703265+0000","last_fullsized":"2023-10-09T06:38:20.703265+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:32:15.314337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.707204+0000","last_change":"2023-10-09T06:38:20.704865+0000","last_active":"2023-10-09T06:38:20.707204+0000","last_peered":"2023-10-09T06:38:20.707204+0000","last_clean":"2023-10-09T06:38:20.707204+0000","last_became_active":"2023-10-09T06:38:20.698131+0000","last_became_peered":"2023-10-09T06:38:20.698131+0000","last_unstale":"2023-10-09T06:38:20.707204+0000","last_undegraded":"2023-10-09T06:38:20.707204+0000","last_fullsized":"2023-10-09T06:38:20.707204+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:42:23.336035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"151'2","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.701918+0000","last_change":"2023-10-09T06:38:19.692752+0000","last_active":"2023-10-09T06:38:20.701918+0000","last_peered":"2023-10-09T06:38:20.701918+0000","last_clean":"2023-10-09T06:38:20.701918+0000","last_became_active":"2023-10-09T06:38:19.692249+0000","last_became_peered":"2023-10-09T06:38:19.692249+0000","last_unstale":"2023-10-09T06:38:20.701918+0000","last_undegraded":"2023-10-09T06:38:20.701918+0000","last_fullsized":"2023-10-09T06:38:20.701918+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:52:11.581942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.696147+0000","last_change":"2023-10-09T06:38:20.692607+0000","last_active":"2023-10-09T06:38:20.696147+0000","last_peered":"2023-10-09T06:38:20.696147+0000","last_clean":"2023-10-09T06:38:20.696147+0000","last_became_active":"2023-10-09T06:38:20.692339+0000","last_became_peered":"2023-10-09T06:38:20.692339+0000","last_unstale":"2023-10-09T06:38:20.696147+0000","last_undegraded":"2023-10-09T06:38:20.696147+0000","last_fullsized":"2023-10-09T06:38:20.696147+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:08:26.202100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"151'1","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.706883+0000","last_change":"2023-10-09T06:38:19.690895+0000","last_active":"2023-10-09T06:38:20.706883+0000","last_peered":"2023-10-09T06:38:20.706883+0000","last_clean":"2023-10-09T06:38:20.706883+0000","last_became_active":"2023-10-09T06:38:19.690633+0000","last_became_peered":"2023-10-09T06:38:19.690633+0000","last_unstale":"2023-10-09T06:38:20.706883+0000","last_undegraded":"2023-10-09T06:38:20.706883+0000","last_fullsized":"2023-10-09T06:38:20.706883+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:26:20.360953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.706855+0000","last_change":"2023-10-09T06:38:20.694662+0000","last_active":"2023-10-09T06:38:20.706855+0000","last_peered":"2023-10-09T06:38:20.706855+0000","last_clean":"2023-10-09T06:38:20.706855+0000","last_became_active":"2023-10-09T06:38:20.694236+0000","last_became_peered":"2023-10-09T06:38:20.694236+0000","last_unstale":"2023-10-09T06:38:20.706855+0000","last_undegraded":"2023-10-09T06:38:20.706855+0000","last_fullsized":"2023-10-09T06:38:20.706855+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:04:31.978834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.682817+0000","last_change":"2023-10-09T06:38:19.687201+0000","last_active":"2023-10-09T06:38:20.682817+0000","last_peered":"2023-10-09T06:38:20.682817+0000","last_clean":"2023-10-09T06:38:20.682817+0000","last_became_active":"2023-10-09T06:38:19.686979+0000","last_became_peered":"2023-10-09T06:38:19.686979+0000","last_unstale":"2023-10-09T06:38:20.682817+0000","last_undegraded":"2023-10-09T06:38:20.682817+0000","last_fullsized":"2023-10-09T06:38:20.682817+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:04:37.633413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.696255+0000","last_change":"2023-10-09T06:38:20.693739+0000","last_active":"2023-10-09T06:38:20.696255+0000","last_peered":"2023-10-09T06:38:20.696255+0000","last_clean":"2023-10-09T06:38:20.696255+0000","last_became_active":"2023-10-09T06:38:20.693499+0000","last_became_peered":"2023-10-09T06:38:20.693499+0000","last_unstale":"2023-10-09T06:38:20.696255+0000","last_undegraded":"2023-10-09T06:38:20.696255+0000","last_fullsized":"2023-10-09T06:38:20.696255+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:16:26.649802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.680550+0000","last_change":"2023-10-09T06:38:19.690010+0000","last_active":"2023-10-09T06:38:20.680550+0000","last_peered":"2023-10-09T06:38:20.680550+0000","last_clean":"2023-10-09T06:38:20.680550+0000","last_became_active":"2023-10-09T06:38:19.689749+0000","last_became_peered":"2023-10-09T06:38:19.689749+0000","last_unstale":"2023-10-09T06:38:20.680550+0000","last_undegraded":"2023-10-09T06:38:20.680550+0000","last_fullsized":"2023-10-09T06:38:20.680550+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:08:33.020051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.697019+0000","last_change":"2023-10-09T06:38:20.694606+0000","last_active":"2023-10-09T06:38:20.697019+0000","last_peered":"2023-10-09T06:38:20.697019+0000","last_clean":"2023-10-09T06:38:20.697019+0000","last_became_active":"2023-10-09T06:38:20.694268+0000","last_became_peered":"2023-10-09T06:38:20.694268+0000","last_unstale":"2023-10-09T06:38:20.697019+0000","last_undegraded":"2023-10-09T06:38:20.697019+0000","last_fullsized":"2023-10-09T06:38:20.697019+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:44:51.812594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.682916+0000","last_change":"2023-10-09T06:38:19.693531+0000","last_active":"2023-10-09T06:38:20.682916+0000","last_peered":"2023-10-09T06:38:20.682916+0000","last_clean":"2023-10-09T06:38:20.682916+0000","last_became_active":"2023-10-09T06:38:19.693246+0000","last_became_peered":"2023-10-09T06:38:19.693246+0000","last_unstale":"2023-10-09T06:38:20.682916+0000","last_undegraded":"2023-10-09T06:38:20.682916+0000","last_fullsized":"2023-10-09T06:38:20.682916+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:26:54.527634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.700871+0000","last_change":"2023-10-09T06:38:20.697037+0000","last_active":"2023-10-09T06:38:20.700871+0000","last_peered":"2023-10-09T06:38:20.700871+0000","last_clean":"2023-10-09T06:38:20.700871+0000","last_became_active":"2023-10-09T06:38:20.694894+0000","last_became_peered":"2023-10-09T06:38:20.694894+0000","last_unstale":"2023-10-09T06:38:20.700871+0000","last_undegraded":"2023-10-09T06:38:20.700871+0000","last_fullsized":"2023-10-09T06:38:20.700871+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:29:07.591719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.681167+0000","last_change":"2023-10-09T06:38:19.694122+0000","last_active":"2023-10-09T06:38:20.681167+0000","last_peered":"2023-10-09T06:38:20.681167+0000","last_clean":"2023-10-09T06:38:20.681167+0000","last_became_active":"2023-10-09T06:38:19.693847+0000","last_became_peered":"2023-10-09T06:38:19.693847+0000","last_unstale":"2023-10-09T06:38:20.681167+0000","last_undegraded":"2023-10-09T06:38:20.681167+0000","last_fullsized":"2023-10-09T06:38:20.681167+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:50:50.722486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.696457+0000","last_change":"2023-10-09T06:38:20.693263+0000","last_active":"2023-10-09T06:38:20.696457+0000","last_peered":"2023-10-09T06:38:20.696457+0000","last_clean":"2023-10-09T06:38:20.696457+0000","last_became_active":"2023-10-09T06:38:20.692971+0000","last_became_peered":"2023-10-09T06:38:20.692971+0000","last_unstale":"2023-10-09T06:38:20.696457+0000","last_undegraded":"2023-10-09T06:38:20.696457+0000","last_fullsized":"2023-10-09T06:38:20.696457+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:06:23.000411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"151'3","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.708537+0000","last_change":"2023-10-09T06:38:19.688209+0000","last_active":"2023-10-09T06:38:20.708537+0000","last_peered":"2023-10-09T06:38:20.708537+0000","last_clean":"2023-10-09T06:38:20.708537+0000","last_became_active":"2023-10-09T06:38:19.687961+0000","last_became_peered":"2023-10-09T06:38:19.687961+0000","last_unstale":"2023-10-09T06:38:20.708537+0000","last_undegraded":"2023-10-09T06:38:20.708537+0000","last_fullsized":"2023-10-09T06:38:20.708537+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:52:49.890865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.698340+0000","last_change":"2023-10-09T06:38:20.689957+0000","last_active":"2023-10-09T06:38:20.698340+0000","last_peered":"2023-10-09T06:38:20.698340+0000","last_clean":"2023-10-09T06:38:20.698340+0000","last_became_active":"2023-10-09T06:38:20.689676+0000","last_became_peered":"2023-10-09T06:38:20.689676+0000","last_unstale":"2023-10-09T06:38:20.698340+0000","last_undegraded":"2023-10-09T06:38:20.698340+0000","last_fullsized":"2023-10-09T06:38:20.698340+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:41:35.398397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"151'6","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.708077+0000","last_change":"2023-10-09T06:38:19.693838+0000","last_active":"2023-10-09T06:38:20.708077+0000","last_peered":"2023-10-09T06:38:20.708077+0000","last_clean":"2023-10-09T06:38:20.708077+0000","last_became_active":"2023-10-09T06:38:19.693584+0000","last_became_peered":"2023-10-09T06:38:19.693584+0000","last_unstale":"2023-10-09T06:38:20.708077+0000","last_undegraded":"2023-10-09T06:38:20.708077+0000","last_fullsized":"2023-10-09T06:38:20.708077+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:42:55.756537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.707554+0000","last_change":"2023-10-09T06:38:20.702889+0000","last_active":"2023-10-09T06:38:20.707554+0000","last_peered":"2023-10-09T06:38:20.707554+0000","last_clean":"2023-10-09T06:38:20.707554+0000","last_became_active":"2023-10-09T06:38:20.702079+0000","last_became_peered":"2023-10-09T06:38:20.702079+0000","last_unstale":"2023-10-09T06:38:20.707554+0000","last_undegraded":"2023-10-09T06:38:20.707554+0000","last_fullsized":"2023-10-09T06:38:20.707554+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:06:23.000411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"151'2","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.707604+0000","last_change":"2023-10-09T06:38:19.684840+0000","last_active":"2023-10-09T06:38:20.707604+0000","last_peered":"2023-10-09T06:38:20.707604+0000","last_clean":"2023-10-09T06:38:20.707604+0000","last_became_active":"2023-10-09T06:38:19.684531+0000","last_became_peered":"2023-10-09T06:38:19.684531+0000","last_unstale":"2023-10-09T06:38:20.707604+0000","last_undegraded":"2023-10-09T06:38:20.707604+0000","last_fullsized":"2023-10-09T06:38:20.707604+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:42:47.239564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.698790+0000","last_change":"2023-10-09T06:38:20.693502+0000","last_active":"2023-10-09T06:38:20.698790+0000","last_peered":"2023-10-09T06:38:20.698790+0000","last_clean":"2023-10-09T06:38:20.698790+0000","last_became_active":"2023-10-09T06:38:20.693254+0000","last_became_peered":"2023-10-09T06:38:20.693254+0000","last_unstale":"2023-10-09T06:38:20.698790+0000","last_undegraded":"2023-10-09T06:38:20.698790+0000","last_fullsized":"2023-10-09T06:38:20.698790+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:00:32.065698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.680828+0000","last_change":"2023-10-09T06:38:19.692513+0000","last_active":"2023-10-09T06:38:20.680828+0000","last_peered":"2023-10-09T06:38:20.680828+0000","last_clean":"2023-10-09T06:38:20.680828+0000","last_became_active":"2023-10-09T06:38:19.692237+0000","last_became_peered":"2023-10-09T06:38:19.692237+0000","last_unstale":"2023-10-09T06:38:20.680828+0000","last_undegraded":"2023-10-09T06:38:20.680828+0000","last_fullsized":"2023-10-09T06:38:20.680828+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:56:50.090947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.700874+0000","last_change":"2023-10-09T06:38:20.696588+0000","last_active":"2023-10-09T06:38:20.700874+0000","last_peered":"2023-10-09T06:38:20.700874+0000","last_clean":"2023-10-09T06:38:20.700874+0000","last_became_active":"2023-10-09T06:38:20.696172+0000","last_became_peered":"2023-10-09T06:38:20.696172+0000","last_unstale":"2023-10-09T06:38:20.700874+0000","last_undegraded":"2023-10-09T06:38:20.700874+0000","last_fullsized":"2023-10-09T06:38:20.700874+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:50:28.293851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.683240+0000","last_change":"2023-10-09T06:38:19.681581+0000","last_active":"2023-10-09T06:38:20.683240+0000","last_peered":"2023-10-09T06:38:20.683240+0000","last_clean":"2023-10-09T06:38:20.683240+0000","last_became_active":"2023-10-09T06:38:19.681316+0000","last_became_peered":"2023-10-09T06:38:19.681316+0000","last_unstale":"2023-10-09T06:38:20.683240+0000","last_undegraded":"2023-10-09T06:38:20.683240+0000","last_fullsized":"2023-10-09T06:38:20.683240+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:38:52.426468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.699480+0000","last_change":"2023-10-09T06:38:20.694875+0000","last_active":"2023-10-09T06:38:20.699480+0000","last_peered":"2023-10-09T06:38:20.699480+0000","last_clean":"2023-10-09T06:38:20.699480+0000","last_became_active":"2023-10-09T06:38:20.694616+0000","last_became_peered":"2023-10-09T06:38:20.694616+0000","last_unstale":"2023-10-09T06:38:20.699480+0000","last_undegraded":"2023-10-09T06:38:20.699480+0000","last_fullsized":"2023-10-09T06:38:20.699480+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:16:56.138017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"151'8","reported_seq":18,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.704628+0000","last_change":"2023-10-09T06:38:19.688731+0000","last_active":"2023-10-09T06:38:20.704628+0000","last_peered":"2023-10-09T06:38:20.704628+0000","last_clean":"2023-10-09T06:38:20.704628+0000","last_became_active":"2023-10-09T06:38:19.688486+0000","last_became_peered":"2023-10-09T06:38:19.688486+0000","last_unstale":"2023-10-09T06:38:20.704628+0000","last_undegraded":"2023-10-09T06:38:20.704628+0000","last_fullsized":"2023-10-09T06:38:20.704628+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:12:32.876485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.701888+0000","last_change":"2023-10-09T06:38:20.697097+0000","last_active":"2023-10-09T06:38:20.701888+0000","last_peered":"2023-10-09T06:38:20.701888+0000","last_clean":"2023-10-09T06:38:20.701888+0000","last_became_active":"2023-10-09T06:38:20.696733+0000","last_became_peered":"2023-10-09T06:38:20.696733+0000","last_unstale":"2023-10-09T06:38:20.701888+0000","last_undegraded":"2023-10-09T06:38:20.701888+0000","last_fullsized":"2023-10-09T06:38:20.701888+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:31:22.541312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"151'1","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.707067+0000","last_change":"2023-10-09T06:38:19.683035+0000","last_active":"2023-10-09T06:38:20.707067+0000","last_peered":"2023-10-09T06:38:20.707067+0000","last_clean":"2023-10-09T06:38:20.707067+0000","last_became_active":"2023-10-09T06:38:19.682754+0000","last_became_peered":"2023-10-09T06:38:19.682754+0000","last_unstale":"2023-10-09T06:38:20.707067+0000","last_undegraded":"2023-10-09T06:38:20.707067+0000","last_fullsized":"2023-10-09T06:38:20.707067+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:40:21.799524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.702790+0000","last_change":"2023-10-09T06:38:20.700584+0000","last_active":"2023-10-09T06:38:20.702790+0000","last_peered":"2023-10-09T06:38:20.702790+0000","last_clean":"2023-10-09T06:38:20.702790+0000","last_became_active":"2023-10-09T06:38:20.700274+0000","last_became_peered":"2023-10-09T06:38:20.700274+0000","last_unstale":"2023-10-09T06:38:20.702790+0000","last_undegraded":"2023-10-09T06:38:20.702790+0000","last_fullsized":"2023-10-09T06:38:20.702790+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:50:51.733189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"151'2","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.707397+0000","last_change":"2023-10-09T06:38:19.694515+0000","last_active":"2023-10-09T06:38:20.707397+0000","last_peered":"2023-10-09T06:38:20.707397+0000","last_clean":"2023-10-09T06:38:20.707397+0000","last_became_active":"2023-10-09T06:38:19.694041+0000","last_became_peered":"2023-10-09T06:38:19.694041+0000","last_unstale":"2023-10-09T06:38:20.707397+0000","last_undegraded":"2023-10-09T06:38:20.707397+0000","last_fullsized":"2023-10-09T06:38:20.707397+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:40:48.522136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.698184+0000","last_change":"2023-10-09T06:38:20.693401+0000","last_active":"2023-10-09T06:38:20.698184+0000","last_peered":"2023-10-09T06:38:20.698184+0000","last_clean":"2023-10-09T06:38:20.698184+0000","last_became_active":"2023-10-09T06:38:20.693156+0000","last_became_peered":"2023-10-09T06:38:20.693156+0000","last_unstale":"2023-10-09T06:38:20.698184+0000","last_undegraded":"2023-10-09T06:38:20.698184+0000","last_fullsized":"2023-10-09T06:38:20.698184+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:48:51.565773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"151'1","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.709124+0000","last_change":"2023-10-09T06:38:19.690795+0000","last_active":"2023-10-09T06:38:20.709124+0000","last_peered":"2023-10-09T06:38:20.709124+0000","last_clean":"2023-10-09T06:38:20.709124+0000","last_became_active":"2023-10-09T06:38:19.690512+0000","last_became_peered":"2023-10-09T06:38:19.690512+0000","last_unstale":"2023-10-09T06:38:20.709124+0000","last_undegraded":"2023-10-09T06:38:20.709124+0000","last_fullsized":"2023-10-09T06:38:20.709124+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:40:21.799524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.697584+0000","last_change":"2023-10-09T06:38:20.695194+0000","last_active":"2023-10-09T06:38:20.697584+0000","last_peered":"2023-10-09T06:38:20.697584+0000","last_clean":"2023-10-09T06:38:20.697584+0000","last_became_active":"2023-10-09T06:38:20.694944+0000","last_became_peered":"2023-10-09T06:38:20.694944+0000","last_unstale":"2023-10-09T06:38:20.697584+0000","last_undegraded":"2023-10-09T06:38:20.697584+0000","last_fullsized":"2023-10-09T06:38:20.697584+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:09:51.130631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.685730+0000","last_change":"2023-10-09T06:38:19.688034+0000","last_active":"2023-10-09T06:38:20.685730+0000","last_peered":"2023-10-09T06:38:20.685730+0000","last_clean":"2023-10-09T06:38:20.685730+0000","last_became_active":"2023-10-09T06:38:19.687764+0000","last_became_peered":"2023-10-09T06:38:19.687764+0000","last_unstale":"2023-10-09T06:38:20.685730+0000","last_undegraded":"2023-10-09T06:38:20.685730+0000","last_fullsized":"2023-10-09T06:38:20.685730+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:37:57.642710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.707344+0000","last_change":"2023-10-09T06:38:20.706044+0000","last_active":"2023-10-09T06:38:20.707344+0000","last_peered":"2023-10-09T06:38:20.707344+0000","last_clean":"2023-10-09T06:38:20.707344+0000","last_became_active":"2023-10-09T06:38:20.705698+0000","last_became_peered":"2023-10-09T06:38:20.705698+0000","last_unstale":"2023-10-09T06:38:20.707344+0000","last_undegraded":"2023-10-09T06:38:20.707344+0000","last_fullsized":"2023-10-09T06:38:20.707344+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:04:31.978834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"151'2","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.704890+0000","last_change":"2023-10-09T06:38:19.686556+0000","last_active":"2023-10-09T06:38:20.704890+0000","last_peered":"2023-10-09T06:38:20.704890+0000","last_clean":"2023-10-09T06:38:20.704890+0000","last_became_active":"2023-10-09T06:38:19.686253+0000","last_became_peered":"2023-10-09T06:38:19.686253+0000","last_unstale":"2023-10-09T06:38:20.704890+0000","last_undegraded":"2023-10-09T06:38:20.704890+0000","last_fullsized":"2023-10-09T06:38:20.704890+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:26:54.527634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.706192+0000","last_change":"2023-10-09T06:38:20.703031+0000","last_active":"2023-10-09T06:38:20.706192+0000","last_peered":"2023-10-09T06:38:20.706192+0000","last_clean":"2023-10-09T06:38:20.706192+0000","last_became_active":"2023-10-09T06:38:20.702760+0000","last_became_peered":"2023-10-09T06:38:20.702760+0000","last_unstale":"2023-10-09T06:38:20.706192+0000","last_undegraded":"2023-10-09T06:38:20.706192+0000","last_fullsized":"2023-10-09T06:38:20.706192+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:13:35.044333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.679070+0000","last_change":"2023-10-09T06:38:19.693762+0000","last_active":"2023-10-09T06:38:20.679070+0000","last_peered":"2023-10-09T06:38:20.679070+0000","last_clean":"2023-10-09T06:38:20.679070+0000","last_became_active":"2023-10-09T06:38:19.693501+0000","last_became_peered":"2023-10-09T06:38:19.693501+0000","last_unstale":"2023-10-09T06:38:20.679070+0000","last_undegraded":"2023-10-09T06:38:20.679070+0000","last_fullsized":"2023-10-09T06:38:20.679070+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:26:54.527634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.699257+0000","last_change":"2023-10-09T06:38:20.692600+0000","last_active":"2023-10-09T06:38:20.699257+0000","last_peered":"2023-10-09T06:38:20.699257+0000","last_clean":"2023-10-09T06:38:20.699257+0000","last_became_active":"2023-10-09T06:38:20.691957+0000","last_became_peered":"2023-10-09T06:38:20.691957+0000","last_unstale":"2023-10-09T06:38:20.699257+0000","last_undegraded":"2023-10-09T06:38:20.699257+0000","last_fullsized":"2023-10-09T06:38:20.699257+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:25:53.616204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.679020+0000","last_change":"2023-10-09T06:38:19.691160+0000","last_active":"2023-10-09T06:38:20.679020+0000","last_peered":"2023-10-09T06:38:20.679020+0000","last_clean":"2023-10-09T06:38:20.679020+0000","last_became_active":"2023-10-09T06:38:19.690880+0000","last_became_peered":"2023-10-09T06:38:19.690880+0000","last_unstale":"2023-10-09T06:38:20.679020+0000","last_undegraded":"2023-10-09T06:38:20.679020+0000","last_fullsized":"2023-10-09T06:38:20.679020+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:40:21.799524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.691605+0000","last_change":"2023-10-09T06:38:20.684157+0000","last_active":"2023-10-09T06:38:20.691605+0000","last_peered":"2023-10-09T06:38:20.691605+0000","last_clean":"2023-10-09T06:38:20.691605+0000","last_became_active":"2023-10-09T06:38:20.683867+0000","last_became_peered":"2023-10-09T06:38:20.683867+0000","last_unstale":"2023-10-09T06:38:20.691605+0000","last_undegraded":"2023-10-09T06:38:20.691605+0000","last_fullsized":"2023-10-09T06:38:20.691605+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:46:20.963679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.684631+0000","last_change":"2023-10-09T06:38:19.684011+0000","last_active":"2023-10-09T06:38:20.684631+0000","last_peered":"2023-10-09T06:38:20.684631+0000","last_clean":"2023-10-09T06:38:20.684631+0000","last_became_active":"2023-10-09T06:38:19.683541+0000","last_became_peered":"2023-10-09T06:38:19.683541+0000","last_unstale":"2023-10-09T06:38:20.684631+0000","last_undegraded":"2023-10-09T06:38:20.684631+0000","last_fullsized":"2023-10-09T06:38:20.684631+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:54:42.729290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.696578+0000","last_change":"2023-10-09T06:38:20.695825+0000","last_active":"2023-10-09T06:38:20.696578+0000","last_peered":"2023-10-09T06:38:20.696578+0000","last_clean":"2023-10-09T06:38:20.696578+0000","last_became_active":"2023-10-09T06:38:20.695569+0000","last_became_peered":"2023-10-09T06:38:20.695569+0000","last_unstale":"2023-10-09T06:38:20.696578+0000","last_undegraded":"2023-10-09T06:38:20.696578+0000","last_fullsized":"2023-10-09T06:38:20.696578+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:57:12.115891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"151'2","reported_seq":12,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.702260+0000","last_change":"2023-10-09T06:38:19.691307+0000","last_active":"2023-10-09T06:38:20.702260+0000","last_peered":"2023-10-09T06:38:20.702260+0000","last_clean":"2023-10-09T06:38:20.702260+0000","last_became_active":"2023-10-09T06:38:19.691050+0000","last_became_peered":"2023-10-09T06:38:19.691050+0000","last_unstale":"2023-10-09T06:38:20.702260+0000","last_undegraded":"2023-10-09T06:38:20.702260+0000","last_fullsized":"2023-10-09T06:38:20.702260+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:15:20.738142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.698443+0000","last_change":"2023-10-09T06:38:20.690712+0000","last_active":"2023-10-09T06:38:20.698443+0000","last_peered":"2023-10-09T06:38:20.698443+0000","last_clean":"2023-10-09T06:38:20.698443+0000","last_became_active":"2023-10-09T06:38:20.690473+0000","last_became_peered":"2023-10-09T06:38:20.690473+0000","last_unstale":"2023-10-09T06:38:20.698443+0000","last_undegraded":"2023-10-09T06:38:20.698443+0000","last_fullsized":"2023-10-09T06:38:20.698443+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:08:42.312456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.685612+0000","last_change":"2023-10-09T06:38:19.691686+0000","last_active":"2023-10-09T06:38:20.685612+0000","last_peered":"2023-10-09T06:38:20.685612+0000","last_clean":"2023-10-09T06:38:20.685612+0000","last_became_active":"2023-10-09T06:38:19.690753+0000","last_became_peered":"2023-10-09T06:38:19.690753+0000","last_unstale":"2023-10-09T06:38:20.685612+0000","last_undegraded":"2023-10-09T06:38:20.685612+0000","last_fullsized":"2023-10-09T06:38:20.685612+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:04:37.633413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.702751+0000","last_change":"2023-10-09T06:38:20.701369+0000","last_active":"2023-10-09T06:38:20.702751+0000","last_peered":"2023-10-09T06:38:20.702751+0000","last_clean":"2023-10-09T06:38:20.702751+0000","last_became_active":"2023-10-09T06:38:20.701098+0000","last_became_peered":"2023-10-09T06:38:20.701098+0000","last_unstale":"2023-10-09T06:38:20.702751+0000","last_undegraded":"2023-10-09T06:38:20.702751+0000","last_fullsized":"2023-10-09T06:38:20.702751+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:56:51.101650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.685231+0000","last_change":"2023-10-09T06:38:19.687181+0000","last_active":"2023-10-09T06:38:20.685231+0000","last_peered":"2023-10-09T06:38:20.685231+0000","last_clean":"2023-10-09T06:38:20.685231+0000","last_became_active":"2023-10-09T06:38:19.686940+0000","last_became_peered":"2023-10-09T06:38:19.686940+0000","last_unstale":"2023-10-09T06:38:20.685231+0000","last_undegraded":"2023-10-09T06:38:20.685231+0000","last_fullsized":"2023-10-09T06:38:20.685231+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:52:11.581942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.697534+0000","last_change":"2023-10-09T06:38:20.690999+0000","last_active":"2023-10-09T06:38:20.697534+0000","last_peered":"2023-10-09T06:38:20.697534+0000","last_clean":"2023-10-09T06:38:20.697534+0000","last_became_active":"2023-10-09T06:38:20.690722+0000","last_became_peered":"2023-10-09T06:38:20.690722+0000","last_unstale":"2023-10-09T06:38:20.697534+0000","last_undegraded":"2023-10-09T06:38:20.697534+0000","last_fullsized":"2023-10-09T06:38:20.697534+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:43:24.921591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.685823+0000","last_change":"2023-10-09T06:38:19.693786+0000","last_active":"2023-10-09T06:38:20.685823+0000","last_peered":"2023-10-09T06:38:20.685823+0000","last_clean":"2023-10-09T06:38:20.685823+0000","last_became_active":"2023-10-09T06:38:19.693529+0000","last_became_peered":"2023-10-09T06:38:19.693529+0000","last_unstale":"2023-10-09T06:38:20.685823+0000","last_undegraded":"2023-10-09T06:38:20.685823+0000","last_fullsized":"2023-10-09T06:38:20.685823+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:09:07.415429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.698134+0000","last_change":"2023-10-09T06:38:20.689373+0000","last_active":"2023-10-09T06:38:20.698134+0000","last_peered":"2023-10-09T06:38:20.698134+0000","last_clean":"2023-10-09T06:38:20.698134+0000","last_became_active":"2023-10-09T06:38:20.689128+0000","last_became_peered":"2023-10-09T06:38:20.689128+0000","last_unstale":"2023-10-09T06:38:20.698134+0000","last_undegraded":"2023-10-09T06:38:20.698134+0000","last_fullsized":"2023-10-09T06:38:20.698134+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:00:32.065698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.685161+0000","last_change":"2023-10-09T06:38:19.690991+0000","last_active":"2023-10-09T06:38:20.685161+0000","last_peered":"2023-10-09T06:38:20.685161+0000","last_clean":"2023-10-09T06:38:20.685161+0000","last_became_active":"2023-10-09T06:38:19.690659+0000","last_became_peered":"2023-10-09T06:38:19.690659+0000","last_unstale":"2023-10-09T06:38:20.685161+0000","last_undegraded":"2023-10-09T06:38:20.685161+0000","last_fullsized":"2023-10-09T06:38:20.685161+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:02:57.917372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.703000+0000","last_change":"2023-10-09T06:38:20.700432+0000","last_active":"2023-10-09T06:38:20.703000+0000","last_peered":"2023-10-09T06:38:20.703000+0000","last_clean":"2023-10-09T06:38:20.703000+0000","last_became_active":"2023-10-09T06:38:20.699733+0000","last_became_peered":"2023-10-09T06:38:20.699733+0000","last_unstale":"2023-10-09T06:38:20.703000+0000","last_undegraded":"2023-10-09T06:38:20.703000+0000","last_fullsized":"2023-10-09T06:38:20.703000+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:56:51.101650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.685615+0000","last_change":"2023-10-09T06:38:19.684393+0000","last_active":"2023-10-09T06:38:20.685615+0000","last_peered":"2023-10-09T06:38:20.685615+0000","last_clean":"2023-10-09T06:38:20.685615+0000","last_became_active":"2023-10-09T06:38:19.683050+0000","last_became_peered":"2023-10-09T06:38:19.683050+0000","last_unstale":"2023-10-09T06:38:20.685615+0000","last_undegraded":"2023-10-09T06:38:20.685615+0000","last_fullsized":"2023-10-09T06:38:20.685615+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:22:25.945021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.700856+0000","last_change":"2023-10-09T06:38:20.696414+0000","last_active":"2023-10-09T06:38:20.700856+0000","last_peered":"2023-10-09T06:38:20.700856+0000","last_clean":"2023-10-09T06:38:20.700856+0000","last_became_active":"2023-10-09T06:38:20.695865+0000","last_became_peered":"2023-10-09T06:38:20.695865+0000","last_unstale":"2023-10-09T06:38:20.700856+0000","last_undegraded":"2023-10-09T06:38:20.700856+0000","last_fullsized":"2023-10-09T06:38:20.700856+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:40:49.532839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.681336+0000","last_change":"2023-10-09T06:38:19.685457+0000","last_active":"2023-10-09T06:38:20.681336+0000","last_peered":"2023-10-09T06:38:20.681336+0000","last_clean":"2023-10-09T06:38:20.681336+0000","last_became_active":"2023-10-09T06:38:19.685181+0000","last_became_peered":"2023-10-09T06:38:19.685181+0000","last_unstale":"2023-10-09T06:38:20.681336+0000","last_undegraded":"2023-10-09T06:38:20.681336+0000","last_fullsized":"2023-10-09T06:38:20.681336+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:39:59.954330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.701697+0000","last_change":"2023-10-09T06:38:20.696229+0000","last_active":"2023-10-09T06:38:20.701697+0000","last_peered":"2023-10-09T06:38:20.701697+0000","last_clean":"2023-10-09T06:38:20.701697+0000","last_became_active":"2023-10-09T06:38:20.695976+0000","last_became_peered":"2023-10-09T06:38:20.695976+0000","last_unstale":"2023-10-09T06:38:20.701697+0000","last_undegraded":"2023-10-09T06:38:20.701697+0000","last_fullsized":"2023-10-09T06:38:20.701697+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:32:06.438731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":10,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.678094+0000","last_change":"2023-10-09T06:38:19.692581+0000","last_active":"2023-10-09T06:38:20.678094+0000","last_peered":"2023-10-09T06:38:20.678094+0000","last_clean":"2023-10-09T06:38:20.678094+0000","last_became_active":"2023-10-09T06:38:19.692312+0000","last_became_peered":"2023-10-09T06:38:19.692312+0000","last_unstale":"2023-10-09T06:38:20.678094+0000","last_undegraded":"2023-10-09T06:38:20.678094+0000","last_fullsized":"2023-10-09T06:38:20.678094+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:52:30.325627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":9,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.698355+0000","last_change":"2023-10-09T06:38:20.690283+0000","last_active":"2023-10-09T06:38:20.698355+0000","last_peered":"2023-10-09T06:38:20.698355+0000","last_clean":"2023-10-09T06:38:20.698355+0000","last_became_active":"2023-10-09T06:38:20.689933+0000","last_became_peered":"2023-10-09T06:38:20.689933+0000","last_unstale":"2023-10-09T06:38:20.698355+0000","last_undegraded":"2023-10-09T06:38:20.698355+0000","last_fullsized":"2023-10-09T06:38:20.698355+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:19.667674+0000","last_clean_scrub_stamp":"2023-10-09T06:38:19.667674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:22:35.237426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"151'1","reported_seq":11,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.705019+0000","last_change":"2023-10-09T06:38:19.683671+0000","last_active":"2023-10-09T06:38:20.705019+0000","last_peered":"2023-10-09T06:38:20.705019+0000","last_clean":"2023-10-09T06:38:20.705019+0000","last_became_active":"2023-10-09T06:38:19.683355+0000","last_became_peered":"2023-10-09T06:38:19.683355+0000","last_unstale":"2023-10-09T06:38:20.705019+0000","last_undegraded":"2023-10-09T06:38:20.705019+0000","last_fullsized":"2023-10-09T06:38:20.705019+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:18.656972+0000","last_clean_scrub_stamp":"2023-10-09T06:38:18.656972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:23:57.642507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":272,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-10-09T06:38:20.683915+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:38:20.683915+0000","last_peered":"2023-10-09T06:38:20.683915+0000","last_clean":"2023-10-09T06:38:20.683915+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:38:20.683915+0000","last_undegraded":"2023-10-09T06:38:20.683915+0000","last_fullsized":"2023-10-09T06:38:20.683915+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574947,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303856,"kb_used_data":2472,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067984,"statfs":{"total":96636764160,"available":96325615616,"internally_reserved":0,"allocated":2531328,"data_stored":1452533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52000000000000002,"15min":0.50700000000000001},"min":{"1min":0.34799999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.37,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67600000000000005,"5min":0.753,"15min":0.753},"last":0.45900000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.503,"15min":0.51900000000000002},"min":{"1min":0.33600000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.74299999999999999,"5min":0.77400000000000002,"15min":1.2030000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.378,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.60999999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53200000000000003,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75800000000000001,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.44,"5min":0.36399999999999999,"15min":0.315},"max":{"1min":0.77900000000000003,"5min":0.80700000000000005,"15min":0.83099999999999996},"last":0.52600000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53800000000000003,"15min":0.55000000000000004},"min":{"1min":0.41199999999999998,"5min":0.35899999999999999,"15min":0.33500000000000002},"max":{"1min":0.68600000000000005,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55600000000000005,"15min":0.59199999999999997},"min":{"1min":0.45300000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.754,"5min":0.82099999999999995,"15min":1.1479999999999999},"last":0.49399999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.48199999999999998,"15min":0.51900000000000002},"min":{"1min":0.35599999999999998,"5min":0.30299999999999999,"15min":0.24399999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.93999999999999995},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56599999999999995,"15min":0.60599999999999998},"min":{"1min":0.36799999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76700000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.53900000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52500000000000002,"15min":0.56999999999999995},"min":{"1min":0.36499999999999999,"5min":0.32500000000000001,"15min":0.307},"max":{"1min":0.81999999999999995,"5min":0.85799999999999998,"15min":0.91100000000000003},"last":0.371},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.59599999999999997,"15min":0.62},"min":{"1min":0.52900000000000003,"5min":0.41699999999999998,"15min":0.34200000000000003},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.58599999999999997}]},{"osd":6,"last update":"Mon Oct 9 06:37:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55700000000000005,"15min":0.59299999999999997},"min":{"1min":0.38600000000000001,"5min":0.33500000000000002,"15min":0.28699999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.88600000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.56100000000000005,"15min":0.61399999999999999},"min":{"1min":0.45100000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.84599999999999997},"last":0.41099999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574946,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303268,"kb_used_data":1884,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068572,"statfs":{"total":96636764160,"available":96326217728,"internally_reserved":0,"allocated":1929216,"data_stored":861183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.50800000000000001,"15min":0.502},"min":{"1min":0.35499999999999998,"5min":0.32000000000000001,"15min":0.30399999999999999},"max":{"1min":0.66500000000000004,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.50700000000000001,"15min":0.51000000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70399999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.64800000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.502,"15min":0.51200000000000001},"min":{"1min":0.33600000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.68300000000000005,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52900000000000003,"15min":0.53200000000000003},"min":{"1min":0.38700000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.77100000000000002,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.69099999999999995}]},{"osd":2,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.34499999999999997,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.72399999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51100000000000001,"15min":0.52900000000000003},"min":{"1min":0.39000000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.66600000000000004,"5min":0.67500000000000004,"15min":0.752},"last":0.63100000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.38900000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66600000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.54300000000000004,"15min":0.68799999999999994},"min":{"1min":0.40400000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.752,"5min":0.752,"15min":4.7510000000000003},"last":0.61699999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46400000000000002,"15min":0.47699999999999998},"min":{"1min":0.29899999999999999,"5min":0.28199999999999997,"15min":0.27900000000000003},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.76400000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56100000000000005,"15min":0.58099999999999996},"min":{"1min":0.41799999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.77200000000000002,"5min":0.81299999999999994,"15min":0.83699999999999997},"last":0.57399999999999995}]},{"osd":5,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.52100000000000002,"15min":0.53000000000000003},"min":{"1min":0.31900000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.66100000000000003,"5min":0.79200000000000004,"15min":0.81200000000000006},"last":0.44},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.59499999999999997},"min":{"1min":0.46899999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76000000000000001,"5min":0.78000000000000003,"15min":0.86599999999999999},"last":0.55900000000000005}]},{"osd":7,"last update":"Mon Oct 9 06:37:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55200000000000005,"15min":0.58199999999999996},"min":{"1min":0.41399999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80500000000000005,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58799999999999997,"15min":0.58899999999999997},"min":{"1min":0.46000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78400000000000003,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.54000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303272,"kb_used_data":1888,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068568,"statfs":{"total":96636764160,"available":96326213632,"internally_reserved":0,"allocated":1933312,"data_stored":860685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.316,"15min":0.311},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.503,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.34899999999999998,"15min":0.35799999999999998},"min":{"1min":0.28399999999999997,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.51900000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.32000000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.34399999999999997,"15min":0.35099999999999998},"min":{"1min":0.215,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.53700000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.38300000000000001,"15min":0.38500000000000001},"min":{"1min":0.30299999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.55300000000000005,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.35599999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.36499999999999999,"15min":0.36399999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.55800000000000005,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.34599999999999997,"15min":0.53500000000000003},"min":{"1min":0.23400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.40400000000000003,"5min":0.56000000000000005,"15min":5.8529999999999998},"last":0.36799999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.58299999999999996,"15min":0.57499999999999996},"min":{"1min":0.36199999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78300000000000003,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.40000000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.81100000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.52100000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57699999999999996,"15min":0.74099999999999999},"min":{"1min":0.36499999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88,"5min":0.88,"15min":5.9459999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61199999999999999,"15min":0.61099999999999999},"min":{"1min":0.38900000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82599999999999996,"5min":0.92700000000000005,"15min":0.93999999999999995},"last":0.58599999999999997}]},{"osd":6,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54900000000000004,"15min":0.71199999999999997},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79100000000000004,"5min":0.79600000000000004,"15min":5.9160000000000004},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.379,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84799999999999998,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.45200000000000001}]},{"osd":7,"last update":"Mon Oct 9 06:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57199999999999995,"15min":0.58599999999999997},"min":{"1min":0.39000000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.80900000000000005,"5min":0.89500000000000002,"15min":0.97099999999999997},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57099999999999995,"15min":0.749},"min":{"1min":0.41399999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":5.9729999999999999},"last":0.59599999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574948,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303848,"kb_used_data":2464,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067992,"statfs":{"total":96636764160,"available":96325623808,"internally_reserved":0,"allocated":2523136,"data_stored":1451087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.30199999999999999,"15min":0.315},"min":{"1min":0.216,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.433,"5min":0.498,"15min":0.498},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.32100000000000001,"15min":0.34399999999999997},"min":{"1min":0.222,"5min":0.214,"15min":0.214},"max":{"1min":0.497,"5min":0.53600000000000003,"15min":0.94499999999999995},"last":0.28199999999999997}]},{"osd":2,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35199999999999998,"15min":0.36199999999999999},"min":{"1min":0.25900000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.50900000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.371,"15min":0.38800000000000001},"min":{"1min":0.30499999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.56499999999999995,"5min":0.69399999999999995,"15min":0.91700000000000004},"last":0.40000000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.32800000000000001,"15min":0.38400000000000001},"min":{"1min":0.23699999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":1.554},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.371,"5min":0.35999999999999999,"15min":0.39700000000000002},"min":{"1min":0.27700000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54400000000000004,"5min":0.58099999999999996,"15min":0.79000000000000004},"last":0.48799999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53600000000000003,"15min":0.55100000000000005},"min":{"1min":0.41199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":1.212},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.72099999999999997},"min":{"1min":0.42199999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":5.484},"last":0.505}]},{"osd":5,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.72499999999999998},"min":{"1min":0.41999999999999998,"5min":0.35799999999999998,"15min":0.35499999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":5.5449999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.83299999999999996,"5min":0.84499999999999997,"15min":1.113},"last":0.56999999999999995}]},{"osd":6,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":1.0529999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59299999999999997,"15min":0.60699999999999998},"min":{"1min":0.45800000000000002,"5min":0.42799999999999999,"15min":0.41599999999999998},"max":{"1min":0.878,"5min":0.878,"15min":1.169},"last":0.63700000000000001}]},{"osd":7,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57099999999999995,"15min":0.58899999999999997},"min":{"1min":0.40600000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.78700000000000003,"5min":0.85499999999999998,"15min":1.2390000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56399999999999995,"15min":0.749},"min":{"1min":0.38,"5min":0.375,"15min":0.375},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":5.5800000000000001},"last":0.624}]}]},{"osd":2,"up_from":13,"seq":55834574949,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303264,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068576,"statfs":{"total":96636764160,"available":96326221824,"internally_reserved":0,"allocated":1925120,"data_stored":860629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.316,"15min":0.35299999999999998},"min":{"1min":0.23200000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":1.4199999999999999},"last":0.25},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.55900000000000005},"min":{"1min":0.27500000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.51600000000000001,"5min":0.53800000000000003,"15min":7.2699999999999996},"last":0.315}]},{"osd":1,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.317,"15min":0.34399999999999997},"min":{"1min":0.23999999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.65400000000000003},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.37,"15min":0.61299999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":7.327},"last":0.35099999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.36799999999999999,"15min":0.39100000000000001},"min":{"1min":0.29399999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.68600000000000005},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.36799999999999999,"15min":0.39200000000000002},"min":{"1min":0.27900000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.72699999999999998},"last":0.33700000000000002}]},{"osd":4,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54500000000000004,"15min":0.54400000000000004},"min":{"1min":0.40799999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.78400000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56200000000000006,"15min":0.78600000000000003},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82699999999999996,"5min":0.84699999999999998,"15min":7.2389999999999999},"last":0.45800000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.58099999999999996,"15min":0.91500000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82699999999999996,"5min":0.83799999999999997,"15min":10.890000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":0.84299999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.54500000000000004}]},{"osd":6,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56399999999999995,"15min":0.79100000000000004},"min":{"1min":0.42999999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.748,"5min":0.77300000000000002,"15min":7.2999999999999998},"last":0.433},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.45800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.52500000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59299999999999997,"15min":0.82199999999999995},"min":{"1min":0.47499999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":7.1879999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60399999999999998,"15min":0.624},"min":{"1min":0.49399999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.90500000000000003},"last":0.55300000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574948,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303276,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068564,"statfs":{"total":96636764160,"available":96326209536,"internally_reserved":0,"allocated":1937408,"data_stored":860811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.32200000000000001,"15min":0.33200000000000002},"min":{"1min":0.23100000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":5.8700000000000001},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.34699999999999998,"15min":0.36699999999999999},"min":{"1min":0.23999999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":5.6769999999999996},"last":0.32800000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.35799999999999998,"15min":0.36099999999999999},"min":{"1min":0.26800000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":4.0469999999999997},"last":0.312},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.372,"15min":0.38800000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":4.1509999999999998},"last":0.35499999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.35199999999999998,"15min":0.374},"min":{"1min":0.29999999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.50800000000000001,"5min":0.56799999999999995,"15min":3.1139999999999999},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.373,"15min":0.38600000000000001},"min":{"1min":0.28100000000000003,"5min":0.26100000000000001,"15min":0.25600000000000001},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":3.3180000000000001},"last":0.34799999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55800000000000005,"15min":0.57099999999999995},"min":{"1min":0.38100000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":3.0899999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56499999999999995,"15min":0.56799999999999995},"min":{"1min":0.40300000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76400000000000001,"5min":0.91100000000000003,"15min":3.0710000000000002},"last":0.60499999999999998}]},{"osd":5,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57499999999999996,"15min":0.58899999999999997},"min":{"1min":0.39100000000000001,"5min":0.34599999999999997,"15min":0.32700000000000001},"max":{"1min":0.877,"5min":0.877,"15min":6.2300000000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.441,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79800000000000004,"5min":0.89500000000000002,"15min":3.0219999999999998},"last":0.621}]},{"osd":6,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.44500000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":2.6850000000000001},"last":0.627},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.65200000000000002},"min":{"1min":0.43099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":6.5970000000000004},"last":0.61599999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:37:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60699999999999998,"15min":0.61399999999999999},"min":{"1min":0.44700000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86099999999999999,"5min":0.93300000000000005,"15min":4.2629999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59999999999999998,"15min":0.61299999999999999},"min":{"1min":0.40500000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.82899999999999996,"5min":0.97199999999999998,"15min":7.3650000000000002},"last":0.54700000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574947,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303284,"kb_used_data":1900,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068556,"statfs":{"total":96636764160,"available":96326201344,"internally_reserved":0,"allocated":1945600,"data_stored":862269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.51400000000000001,"15min":0.55300000000000005},"min":{"1min":0.318,"5min":0.28100000000000003,"15min":0.27000000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":1.083},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.35699999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.79900000000000004,"5min":0.97799999999999998,"15min":1.054},"last":0.56200000000000006}]},{"osd":1,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.39900000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.50700000000000001,"15min":0.49399999999999999},"min":{"1min":0.33800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.76300000000000001,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.51100000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52700000000000002,"15min":0.53900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.32900000000000001},"max":{"1min":0.76600000000000001,"5min":0.82099999999999995,"15min":1.0249999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55700000000000005,"15min":0.59099999999999997},"min":{"1min":0.39400000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.93799999999999994},"last":0.47399999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52900000000000003,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.32200000000000001,"15min":0.29199999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.98499999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57699999999999996,"15min":0.58799999999999997},"min":{"1min":0.38700000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.1319999999999999},"last":0.59199999999999997}]},{"osd":5,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51600000000000001,"15min":0.55100000000000005},"min":{"1min":0.31900000000000001,"5min":0.30399999999999999,"15min":0.27300000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.998},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58199999999999996,"15min":0.60599999999999998},"min":{"1min":0.40500000000000003,"5min":0.36299999999999999,"15min":0.35099999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.97999999999999998},"last":0.79600000000000004}]},{"osd":6,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.56799999999999995},"min":{"1min":0.41599999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.76800000000000002,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.59799999999999998},"min":{"1min":0.38700000000000001,"5min":0.34999999999999998,"15min":0.32700000000000001},"max":{"1min":0.85599999999999998,"5min":1.01,"15min":1.01},"last":0.78300000000000003}]},{"osd":7,"last update":"Mon Oct 9 06:37:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54300000000000004,"15min":0.56999999999999995},"min":{"1min":0.40200000000000002,"5min":0.33600000000000002,"15min":0.28299999999999997},"max":{"1min":0.78900000000000003,"5min":0.85599999999999998,"15min":0.90000000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.60699999999999998,"15min":0.64500000000000002},"min":{"1min":0.36699999999999999,"5min":0.34999999999999998,"15min":0.32200000000000001},"max":{"1min":0.89000000000000001,"5min":0.98999999999999999,"15min":1.175},"last":0.81999999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574947,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303268,"kb_used_data":1884,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068572,"statfs":{"total":96636764160,"available":96326217728,"internally_reserved":0,"allocated":1929216,"data_stored":861183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.503,"15min":0.50800000000000001},"min":{"1min":0.38800000000000001,"5min":0.29299999999999998,"15min":0.28599999999999998},"max":{"1min":0.67900000000000005,"5min":0.71399999999999997,"15min":0.86599999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51800000000000002,"15min":0.51400000000000001},"min":{"1min":0.42499999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.60599999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.67600000000000005}]},{"osd":1,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.505,"15min":0.50900000000000001},"min":{"1min":0.38400000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.63900000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52300000000000002,"15min":0.53100000000000003},"min":{"1min":0.43099999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72799999999999998,"5min":0.73899999999999999,"15min":0.81599999999999995},"last":0.74399999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54300000000000004,"15min":0.54400000000000004},"min":{"1min":0.41099999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.63900000000000001,"5min":0.72999999999999998,"15min":0.83999999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.47199999999999998,"5min":0.40999999999999998,"15min":0.39500000000000002},"max":{"1min":0.72699999999999998,"5min":0.74099999999999999,"15min":0.79900000000000004},"last":0.66100000000000003}]},{"osd":3,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.42399999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.66900000000000004,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.54900000000000004,"15min":0.56000000000000005},"min":{"1min":0.42499999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.63500000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.69199999999999995}]},{"osd":4,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.50600000000000001,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.36199999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.80700000000000005},"last":0.78300000000000003}]},{"osd":6,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55300000000000005,"15min":0.53600000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.83199999999999996},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.36199999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77900000000000003,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.64400000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:37:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.57899999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71499999999999997,"5min":0.79500000000000004,"15min":0.85599999999999998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.374,"5min":0.35199999999999998,"15min":0.314},"max":{"1min":0.79500000000000004,"5min":0.84499999999999997,"15min":0.90600000000000003},"last":0.53800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:38:29.594 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-09T06:38:29.595 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-09T06:38:29.751 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.751+0000 7f32b418f700 1 Processor -- start 2023-10-09T06:38:29.751 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.752+0000 7f32b418f700 1 -- start start 2023-10-09T06:38:29.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.752+0000 7f32b418f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f32ac06bcd0 0x7f32ac0c9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:29.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.752+0000 7f32b418f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32ac1414c0 0x7f32ac0cc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:29.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.752+0000 7f32b418f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32ac15b790 con 0x7f32ac158d40 2023-10-09T06:38:29.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.752+0000 7f32b418f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32ac15ba70 con 0x7f32ac06bcd0 2023-10-09T06:38:29.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.752+0000 7f32b418f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac15bd50 con 0x7f32ac1414c0 2023-10-09T06:38:29.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.752+0000 7f32b172a700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f32ac158d40 0x7f32ac140db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:45982/0 (socket says 172.21.15.163:45982) 2023-10-09T06:38:29.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.752+0000 7f32b172a700 1 -- 172.21.15.163:0/2604615987 learned_addr learned my addr 172.21.15.163:0/2604615987 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:29.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.753+0000 7f32b272c700 1 --2- 172.21.15.163:0/2604615987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32ac1414c0 0x7f32ac0cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:29.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.753+0000 7f32b1f2b700 1 --2- 172.21.15.163:0/2604615987 >> v2:172.21.15.181:3300/0 conn(0x7f32ac06bcd0 0x7f32ac0c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:29.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.753+0000 7f32b0f29700 1 -- 172.21.15.163:0/2604615987 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2746968050 0 0) 0x7f32ac15b790 con 0x7f32ac158d40 2023-10-09T06:38:29.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.753+0000 7f32b0f29700 1 -- 172.21.15.163:0/2604615987 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3298003570 con 0x7f32ac158d40 2023-10-09T06:38:29.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.753+0000 7f32b0f29700 1 -- 172.21.15.163:0/2604615987 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3028041545 0 0) 0x7f3298003570 con 0x7f32ac158d40 2023-10-09T06:38:29.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.753+0000 7f32b0f29700 1 -- 172.21.15.163:0/2604615987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32ac1414c0 msgr2=0x7f32ac0cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:29.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.753+0000 7f32b0f29700 1 --2- 172.21.15.163:0/2604615987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32ac1414c0 0x7f32ac0cc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:29.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.753+0000 7f32b0f29700 1 -- 172.21.15.163:0/2604615987 >> v2:172.21.15.181:3300/0 conn(0x7f32ac06bcd0 msgr2=0x7f32ac0c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:29.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.753+0000 7f32b0f29700 1 --2- 172.21.15.163:0/2604615987 >> v2:172.21.15.181:3300/0 conn(0x7f32ac06bcd0 0x7f32ac0c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:29.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.753+0000 7f32b0f29700 1 -- 172.21.15.163:0/2604615987 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ac15c030 con 0x7f32ac158d40 2023-10-09T06:38:29.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.753+0000 7f32b0f29700 1 -- 172.21.15.163:0/2604615987 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f329c001470 con 0x7f32ac158d40 2023-10-09T06:38:29.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.753+0000 7f32b1f2b700 1 --2- 172.21.15.163:0/2604615987 >> v2:172.21.15.181:3300/0 conn(0x7f32ac06bcd0 0x7f32ac0c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:38:29.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.753+0000 7f32b0f29700 1 -- 172.21.15.163:0/2604615987 >> v1:172.21.15.163:6789/0 conn(0x7f32ac158d40 legacy=0x7f32ac140db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:29.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.753+0000 7f32b0f29700 1 --2- 172.21.15.163:0/2604615987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32ac1414c0 0x7f3298003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:29.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.753+0000 7f32b0f29700 1 --2- 172.21.15.163:0/2604615987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3298003fe0 0x7f3298004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:29.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.753+0000 7f32b0f29700 1 --2- 172.21.15.163:0/2604615987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3298004d50 0x7f32980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:29.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.753+0000 7f32b0f29700 1 -- 172.21.15.163:0/2604615987 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac15ba70 con 0x7f3298003fe0 2023-10-09T06:38:29.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.753+0000 7f32b0f29700 1 -- 172.21.15.163:0/2604615987 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac15bd50 con 0x7f3298004d50 2023-10-09T06:38:29.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.753+0000 7f32b0f29700 1 -- 172.21.15.163:0/2604615987 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f329800ba90 con 0x7f32ac1414c0 2023-10-09T06:38:29.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.753+0000 7f32b172a700 1 --2- 172.21.15.163:0/2604615987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3298003fe0 0x7f3298004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:29.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.753+0000 7f32b1f2b700 1 --2- 172.21.15.163:0/2604615987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32ac1414c0 0x7f3298003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:29.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.753+0000 7f32b272c700 1 --2- 172.21.15.163:0/2604615987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3298004d50 0x7f32980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:29.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.753+0000 7f32b172a700 1 -- 172.21.15.163:0/2604615987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32ac1414c0 msgr2=0x7f3298003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:29.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.753+0000 7f32b172a700 1 --2- 172.21.15.163:0/2604615987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32ac1414c0 0x7f3298003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:29.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.753+0000 7f32b172a700 1 -- 172.21.15.163:0/2604615987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3298004d50 msgr2=0x7f32980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:29.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.753+0000 7f32b172a700 1 --2- 172.21.15.163:0/2604615987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3298004d50 0x7f32980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:29.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.753+0000 7f32b172a700 1 -- 172.21.15.163:0/2604615987 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3298003570 con 0x7f3298003fe0 2023-10-09T06:38:29.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.754+0000 7f32b172a700 1 --2- 172.21.15.163:0/2604615987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3298003fe0 0x7f3298004810 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f329c00c3c0 tx=0x7f329c001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3980aa71766a1797 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:29.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.754+0000 7f32b0f29700 1 -- 172.21.15.163:0/2604615987 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c00cc30 con 0x7f3298003fe0 2023-10-09T06:38:29.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.754+0000 7f32b0f29700 1 -- 172.21.15.163:0/2604615987 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329c00cdd0 con 0x7f3298003fe0 2023-10-09T06:38:29.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.754+0000 7f32b0f29700 1 -- 172.21.15.163:0/2604615987 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c005b70 con 0x7f3298003fe0 2023-10-09T06:38:29.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.754+0000 7f32b418f700 1 -- 172.21.15.163:0/2604615987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3298003fe0 msgr2=0x7f3298004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:29.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.754+0000 7f32b418f700 1 --2- 172.21.15.163:0/2604615987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3298003fe0 0x7f3298004810 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f329c00c3c0 tx=0x7f329c001e90 comp rx=0 tx=0).stop 2023-10-09T06:38:29.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.754+0000 7f32b272c700 1 -- 172.21.15.163:0/2604615987 reap_dead start 2023-10-09T06:38:29.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.754+0000 7f32b418f700 1 -- 172.21.15.163:0/2604615987 shutdown_connections 2023-10-09T06:38:29.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.754+0000 7f32b418f700 1 -- 172.21.15.163:0/2604615987 >> 172.21.15.163:0/2604615987 conn(0x7f32ac0ce5c0 msgr2=0x7f32ac0ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:29.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.754+0000 7f32b418f700 1 -- 172.21.15.163:0/2604615987 shutdown_connections 2023-10-09T06:38:29.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.754+0000 7f32b418f700 1 -- 172.21.15.163:0/2604615987 wait complete. 2023-10-09T06:38:29.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.755+0000 7f32b418f700 1 Processor -- start 2023-10-09T06:38:29.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.756+0000 7f32b418f700 1 -- start start 2023-10-09T06:38:29.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.756+0000 7f32b418f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3298003fe0 0x7f32ac15a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:29.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.756+0000 7f32b418f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32ac1680f0 0x7f32ac13bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:29.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.756+0000 7f32b418f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32ac15a810 0x7f32ac13c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:29.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.756+0000 7f32b418f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac0c71d0 con 0x7f32ac15a810 2023-10-09T06:38:29.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.756+0000 7f32b418f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac0b2e00 con 0x7f3298003fe0 2023-10-09T06:38:29.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.756+0000 7f32b418f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac0c6c10 con 0x7f32ac1680f0 2023-10-09T06:38:29.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.756+0000 7f32b272c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32ac15a810 0x7f32ac13c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:29.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.756+0000 7f32b272c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32ac15a810 0x7f32ac13c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46952/0 (socket says 172.21.15.163:46952) 2023-10-09T06:38:29.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.756+0000 7f32b272c700 1 -- 172.21.15.163:0/3546402952 learned_addr learned my addr 172.21.15.163:0/3546402952 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:29.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.757+0000 7f32b272c700 1 -- 172.21.15.163:0/3546402952 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32ac1680f0 msgr2=0x7f32ac13bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:29.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.757+0000 7f32b272c700 1 --2- 172.21.15.163:0/3546402952 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32ac1680f0 0x7f32ac13bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:29.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.757+0000 7f32b272c700 1 -- 172.21.15.163:0/3546402952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3298003fe0 msgr2=0x7f32ac15a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:29.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.757+0000 7f32b272c700 1 --2- 172.21.15.163:0/3546402952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3298003fe0 0x7f32ac15a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:29.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.757+0000 7f32b272c700 1 -- 172.21.15.163:0/3546402952 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ac0c6ef0 con 0x7f32ac15a810 2023-10-09T06:38:29.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.757+0000 7f32b272c700 1 --2- 172.21.15.163:0/3546402952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32ac15a810 0x7f32ac13c310 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f32ac15b550 tx=0x7f32a400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:29.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.757+0000 7f32a1ffb700 1 -- 172.21.15.163:0/3546402952 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a400e040 con 0x7f32ac15a810 2023-10-09T06:38:29.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.757+0000 7f32a1ffb700 1 -- 172.21.15.163:0/3546402952 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a400f040 con 0x7f32ac15a810 2023-10-09T06:38:29.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.757+0000 7f32b418f700 1 -- 172.21.15.163:0/3546402952 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32ac0b2a90 con 0x7f32ac15a810 2023-10-09T06:38:29.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.757+0000 7f32a1ffb700 1 -- 172.21.15.163:0/3546402952 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a400fe20 con 0x7f32ac15a810 2023-10-09T06:38:29.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.757+0000 7f32b418f700 1 -- 172.21.15.163:0/3546402952 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32ac0c6080 con 0x7f32ac15a810 2023-10-09T06:38:29.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.758+0000 7f32b418f700 1 -- 172.21.15.163:0/3546402952 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32ac0b28f0 con 0x7f32ac15a810 2023-10-09T06:38:29.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.758+0000 7f32a1ffb700 1 -- 172.21.15.163:0/3546402952 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f32a4015640 con 0x7f32ac15a810 2023-10-09T06:38:29.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.759+0000 7f32a1ffb700 1 --2- 172.21.15.163:0/3546402952 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3290067320 0x7f32900697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:29.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.759+0000 7f32a1ffb700 1 -- 172.21.15.163:0/3546402952 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f32a40856b0 con 0x7f32ac15a810 2023-10-09T06:38:29.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.759+0000 7f32b1f2b700 1 --2- 172.21.15.163:0/3546402952 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3290067320 0x7f32900697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:29.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.760+0000 7f32b1f2b700 1 --2- 172.21.15.163:0/3546402952 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3290067320 0x7f32900697d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f32a8006f00 tx=0x7f32a80091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:29.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:29.763+0000 7f32a1ffb700 1 -- 172.21.15.163:0/3546402952 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32ac0b28f0 con 0x7f32ac15a810 2023-10-09T06:38:30.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.021+0000 7f32b418f700 1 -- 172.21.15.163:0/3546402952 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f32ac0b28f0 con 0x7f32ac15a810 2023-10-09T06:38:30.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.025+0000 7f32a1ffb700 1 -- 172.21.15.163:0/3546402952 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+0 (secure 0 0 0) 0x7f32ac0b28f0 con 0x7f32ac15a810 2023-10-09T06:38:30.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.027+0000 7f32b418f700 1 -- 172.21.15.163:0/3546402952 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3290067320 msgr2=0x7f32900697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:30.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.027+0000 7f32b418f700 1 --2- 172.21.15.163:0/3546402952 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3290067320 0x7f32900697d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f32a8006f00 tx=0x7f32a80091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.027+0000 7f32b418f700 1 -- 172.21.15.163:0/3546402952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32ac15a810 msgr2=0x7f32ac13c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:30.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.027+0000 7f32b418f700 1 --2- 172.21.15.163:0/3546402952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32ac15a810 0x7f32ac13c310 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f32ac15b550 tx=0x7f32a400b040 comp rx=0 tx=0).stop 2023-10-09T06:38:30.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.027+0000 7f32b418f700 1 -- 172.21.15.163:0/3546402952 shutdown_connections 2023-10-09T06:38:30.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.027+0000 7f32b418f700 1 --2- 172.21.15.163:0/3546402952 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3290067320 0x7f32900697d0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.027+0000 7f32b418f700 1 --2- 172.21.15.163:0/3546402952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32ac15a810 0x7f32ac13c310 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.027+0000 7f32b418f700 1 --2- 172.21.15.163:0/3546402952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3298003fe0 0x7f32ac15a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.027+0000 7f32b418f700 1 --2- 172.21.15.163:0/3546402952 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32ac1680f0 0x7f32ac13bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.027+0000 7f32b418f700 1 -- 172.21.15.163:0/3546402952 >> 172.21.15.163:0/3546402952 conn(0x7f32ac0ce5c0 msgr2=0x7f32ac0c47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:30.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.027+0000 7f32b418f700 1 -- 172.21.15.163:0/3546402952 shutdown_connections 2023-10-09T06:38:30.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.027+0000 7f32b418f700 1 -- 172.21.15.163:0/3546402952 wait complete. 2023-10-09T06:38:30.029 INFO:teuthology.orchestra.run.smithi163.stderr:updated caps for client.0 2023-10-09T06:38:30.039 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:38:30.154 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.154+0000 7fe97759e700 1 Processor -- start 2023-10-09T06:38:30.154 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.154+0000 7fe97759e700 1 -- start start 2023-10-09T06:38:30.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.155+0000 7fe97759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe97806c0f0 0x7fe978159c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:30.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.155+0000 7fe97759e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe9780b3810 0x7fe9780b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:30.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.155+0000 7fe97759e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe978143420 con 0x7fe978143050 2023-10-09T06:38:30.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.155+0000 7fe97759e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe97806a1e0 con 0x7fe9780b3810 2023-10-09T06:38:30.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.155+0000 7fe97759e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9780695d0 con 0x7fe97806c0f0 2023-10-09T06:38:30.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.155+0000 7fe976d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe9780b3810 0x7fe9780b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:30.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.155+0000 7fe96dd9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe97806c0f0 0x7fe978159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:30.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.155+0000 7fe976d9d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe9780b3810 0x7fe9780b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:51776/0 (socket says 172.21.15.163:51776) 2023-10-09T06:38:30.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.155+0000 7fe96dd9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe97806c0f0 0x7fe978159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:43054/0 (socket says 172.21.15.163:43054) 2023-10-09T06:38:30.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.155+0000 7fe97659c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe978143050 0x7fe97806b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:43938/0 (socket says 172.21.15.163:43938) 2023-10-09T06:38:30.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.155+0000 7fe976d9d700 1 -- 172.21.15.163:0/3250301009 learned_addr learned my addr 172.21.15.163:0/3250301009 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:30.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.156+0000 7fe976d9d700 1 -- 172.21.15.163:0/3250301009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe97806c0f0 msgr2=0x7fe978159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:30.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.156+0000 7fe976d9d700 1 --2- 172.21.15.163:0/3250301009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe97806c0f0 0x7fe978159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.156+0000 7fe976d9d700 1 -- 172.21.15.163:0/3250301009 >> v1:172.21.15.181:6789/0 conn(0x7fe978143050 legacy=0x7fe97806b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:30.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.156+0000 7fe976d9d700 1 -- 172.21.15.163:0/3250301009 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9780698b0 con 0x7fe9780b3810 2023-10-09T06:38:30.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.156+0000 7fe96dd9b700 1 --2- 172.21.15.163:0/3250301009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe97806c0f0 0x7fe978159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:38:30.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.156+0000 7fe976d9d700 1 --2- 172.21.15.163:0/3250301009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9780b3810 0x7fe9780b3c90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe970009f20 tx=0x7fe970009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38e2ca1b7efb3c19 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:30.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.156+0000 7fe975d9b700 1 -- 172.21.15.163:0/3250301009 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97000e070 con 0x7fe9780b3810 2023-10-09T06:38:30.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.156+0000 7fe975d9b700 1 -- 172.21.15.163:0/3250301009 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe97000b9d0 con 0x7fe9780b3810 2023-10-09T06:38:30.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.156+0000 7fe975d9b700 1 -- 172.21.15.163:0/3250301009 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9700092e0 con 0x7fe9780b3810 2023-10-09T06:38:30.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.156+0000 7fe97759e700 1 -- 172.21.15.163:0/3250301009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9780b3810 msgr2=0x7fe9780b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:30.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.156+0000 7fe97759e700 1 --2- 172.21.15.163:0/3250301009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9780b3810 0x7fe9780b3c90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe970009f20 tx=0x7fe970009840 comp rx=0 tx=0).stop 2023-10-09T06:38:30.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.156+0000 7fe97759e700 1 -- 172.21.15.163:0/3250301009 shutdown_connections 2023-10-09T06:38:30.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.156+0000 7fe97759e700 1 --2- 172.21.15.163:0/3250301009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9780b3810 0x7fe9780b3c90 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.156+0000 7fe97759e700 1 --2- 172.21.15.163:0/3250301009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe97806c0f0 0x7fe978159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.157+0000 7fe97759e700 1 -- 172.21.15.163:0/3250301009 >> 172.21.15.163:0/3250301009 conn(0x7fe9780ce430 msgr2=0x7fe9780ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:30.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.157+0000 7fe97759e700 1 -- 172.21.15.163:0/3250301009 shutdown_connections 2023-10-09T06:38:30.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.157+0000 7fe97759e700 1 -- 172.21.15.163:0/3250301009 wait complete. 2023-10-09T06:38:30.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.158+0000 7fe97759e700 1 Processor -- start 2023-10-09T06:38:30.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.158+0000 7fe97759e700 1 -- start start 2023-10-09T06:38:30.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.159+0000 7fe97759e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe97806c0f0 0x7fe97815d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:30.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.159+0000 7fe97759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9780b3810 0x7fe97815e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:30.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.159+0000 7fe97759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe978143050 0x7fe97816d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:30.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.159+0000 7fe97759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97815dd70 con 0x7fe9780b3810 2023-10-09T06:38:30.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.159+0000 7fe97759e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97815cdf0 con 0x7fe97806c0f0 2023-10-09T06:38:30.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.159+0000 7fe97759e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe97815c1e0 con 0x7fe978143050 2023-10-09T06:38:30.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.159+0000 7fe96dd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9780b3810 0x7fe97815e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:30.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.159+0000 7fe96dd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9780b3810 0x7fe97815e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46966/0 (socket says 172.21.15.163:46966) 2023-10-09T06:38:30.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.159+0000 7fe96dd9b700 1 -- 172.21.15.163:0/196374538 learned_addr learned my addr 172.21.15.163:0/196374538 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:30.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.159+0000 7fe97659c700 1 --2- 172.21.15.163:0/196374538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe97806c0f0 0x7fe97815d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:30.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.159+0000 7fe976d9d700 1 --2- 172.21.15.163:0/196374538 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe978143050 0x7fe97816d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:30.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.159+0000 7fe96dd9b700 1 -- 172.21.15.163:0/196374538 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe978143050 msgr2=0x7fe97816d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:30.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.159+0000 7fe96dd9b700 1 --2- 172.21.15.163:0/196374538 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe978143050 0x7fe97816d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.159+0000 7fe96dd9b700 1 -- 172.21.15.163:0/196374538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe97806c0f0 msgr2=0x7fe97815d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:30.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.159+0000 7fe96dd9b700 1 --2- 172.21.15.163:0/196374538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe97806c0f0 0x7fe97815d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.159+0000 7fe96dd9b700 1 -- 172.21.15.163:0/196374538 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe960009420 con 0x7fe9780b3810 2023-10-09T06:38:30.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.159+0000 7fe96dd9b700 1 --2- 172.21.15.163:0/196374538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9780b3810 0x7fe97815e8c0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fe96000ec80 tx=0x7fe9600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:30.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.159+0000 7fe96effd700 1 -- 172.21.15.163:0/196374538 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96000c920 con 0x7fe9780b3810 2023-10-09T06:38:30.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.159+0000 7fe97759e700 1 -- 172.21.15.163:0/196374538 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe970009ae0 con 0x7fe9780b3810 2023-10-09T06:38:30.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.159+0000 7fe96effd700 1 -- 172.21.15.163:0/196374538 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96000ca80 con 0x7fe9780b3810 2023-10-09T06:38:30.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.159+0000 7fe96effd700 1 -- 172.21.15.163:0/196374538 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96000ce00 con 0x7fe9780b3810 2023-10-09T06:38:30.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.159+0000 7fe97759e700 1 -- 172.21.15.163:0/196374538 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe97815c7a0 con 0x7fe9780b3810 2023-10-09T06:38:30.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.160+0000 7fe97759e700 1 -- 172.21.15.163:0/196374538 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe97815cf60 con 0x7fe9780b3810 2023-10-09T06:38:30.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.161+0000 7fe96effd700 1 -- 172.21.15.163:0/196374538 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe960013680 con 0x7fe9780b3810 2023-10-09T06:38:30.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.161+0000 7fe96effd700 1 --2- 172.21.15.163:0/196374538 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe958067320 0x7fe9580697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:30.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.161+0000 7fe96effd700 1 -- 172.21.15.163:0/196374538 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe960085020 con 0x7fe9780b3810 2023-10-09T06:38:30.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.162+0000 7fe97659c700 1 --2- 172.21.15.163:0/196374538 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe958067320 0x7fe9580697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:30.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.162+0000 7fe97659c700 1 --2- 172.21.15.163:0/196374538 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe958067320 0x7fe9580697d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe968007fd0 tx=0x7fe968007f00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:30.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.166+0000 7fe96effd700 1 -- 172.21.15.163:0/196374538 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe960051cb0 con 0x7fe9780b3810 2023-10-09T06:38:30.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.391+0000 7fe97759e700 1 -- 172.21.15.163:0/196374538 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe97804c3c0 con 0x7fe9780b3810 2023-10-09T06:38:30.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.391+0000 7fe96effd700 1 -- 172.21.15.163:0/196374538 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4615 (secure 0 0 0) 0x7fe960017090 con 0x7fe9780b3810 2023-10-09T06:38:30.391 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:30.392 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":103,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6371,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3797578839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3797578839},{"type":"v1","addr":"172.21.15.163:6834","nonce":3797578839}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":6383,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2371721802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2371721802},{"type":"v1","addr":"172.21.15.181:6837","nonce":2371721802}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":7228,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2068214884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2068214884},{"type":"v1","addr":"172.21.15.181:6836","nonce":2068214884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:38:20.060175+0000","modified":"2023-10-09T06:38:23.693400+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7875},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7875":{"gid":7875,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/3794912550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3794912550},{"type":"v1","addr":"172.21.15.163:6835","nonce":3794912550}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-09T06:38:30.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.393+0000 7fe97759e700 1 -- 172.21.15.163:0/196374538 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe958067320 msgr2=0x7fe9580697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:30.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.393+0000 7fe97759e700 1 --2- 172.21.15.163:0/196374538 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe958067320 0x7fe9580697d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe968007fd0 tx=0x7fe968007f00 comp rx=0 tx=0).stop 2023-10-09T06:38:30.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.393+0000 7fe97759e700 1 -- 172.21.15.163:0/196374538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9780b3810 msgr2=0x7fe97815e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:30.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.393+0000 7fe97759e700 1 --2- 172.21.15.163:0/196374538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9780b3810 0x7fe97815e8c0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fe96000ec80 tx=0x7fe9600091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.394+0000 7fe97759e700 1 -- 172.21.15.163:0/196374538 shutdown_connections 2023-10-09T06:38:30.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.394+0000 7fe97759e700 1 --2- 172.21.15.163:0/196374538 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe958067320 0x7fe9580697d0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.394+0000 7fe97759e700 1 --2- 172.21.15.163:0/196374538 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe978143050 0x7fe97816d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.394+0000 7fe97759e700 1 --2- 172.21.15.163:0/196374538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe97806c0f0 0x7fe97815d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.394+0000 7fe97759e700 1 --2- 172.21.15.163:0/196374538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9780b3810 0x7fe97815e8c0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.394+0000 7fe97759e700 1 -- 172.21.15.163:0/196374538 >> 172.21.15.163:0/196374538 conn(0x7fe9780ce430 msgr2=0x7fe9780ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:30.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.394+0000 7fe97759e700 1 -- 172.21.15.163:0/196374538 shutdown_connections 2023-10-09T06:38:30.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.394+0000 7fe97759e700 1 -- 172.21.15.163:0/196374538 wait complete. 2023-10-09T06:38:30.395 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 103 2023-10-09T06:38:30.808 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 103, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-09T06:38:20.060175+0000', 'modified': '2023-10-09T06:38:23.693400+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7875}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7875': {'gid': 7875, 'name': 'a', 'rank': 0, 'incarnation': 99, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.163:6835/3794912550', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6833', 'nonce': 3794912550}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 3794912550}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-09T06:38:30.808 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-09T06:38:30.808 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-09T06:38:30.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.924+0000 7fe87f7dd700 1 Processor -- start 2023-10-09T06:38:30.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.924+0000 7fe87f7dd700 1 -- start start 2023-10-09T06:38:30.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.925+0000 7fe87f7dd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe878159c60 0x7fe87815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:30.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.925+0000 7fe87f7dd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe87815db10 0x7fe878069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:30.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.925+0000 7fe87f7dd700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe87815a160 con 0x7fe878159890 2023-10-09T06:38:30.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.925+0000 7fe87f7dd700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8780d13a0 con 0x7fe87815db10 2023-10-09T06:38:30.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.925+0000 7fe87f7dd700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe87815e140 con 0x7fe878159c60 2023-10-09T06:38:30.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.925+0000 7fe87dd7a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe87815db10 0x7fe878069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:30.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.925+0000 7fe87dd7a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe87815db10 0x7fe878069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46982/0 (socket says 172.21.15.163:46982) 2023-10-09T06:38:30.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.925+0000 7fe87dd7a700 1 -- 172.21.15.163:0/4196294914 learned_addr learned my addr 172.21.15.163:0/4196294914 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:30.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.926+0000 7fe87cd78700 1 --2- 172.21.15.163:0/4196294914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe878159c60 0x7fe87815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:30.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.926+0000 7fe87dd7a700 1 -- 172.21.15.163:0/4196294914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe878159c60 msgr2=0x7fe87815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:30.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.926+0000 7fe87dd7a700 1 --2- 172.21.15.163:0/4196294914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe878159c60 0x7fe87815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.926+0000 7fe87dd7a700 1 -- 172.21.15.163:0/4196294914 >> v1:172.21.15.181:6789/0 conn(0x7fe878159890 legacy=0x7fe87815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:30.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.926+0000 7fe87dd7a700 1 -- 172.21.15.163:0/4196294914 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87815a3d0 con 0x7fe87815db10 2023-10-09T06:38:30.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.926+0000 7fe87dd7a700 1 --2- 172.21.15.163:0/4196294914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe87815db10 0x7fe878069560 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fe8700099b0 tx=0x7fe870009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fabe261fc87d3896 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:30.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.926+0000 7fe86ffff700 1 -- 172.21.15.163:0/4196294914 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87000b860 con 0x7fe87815db10 2023-10-09T06:38:30.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.926+0000 7fe86ffff700 1 -- 172.21.15.163:0/4196294914 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87000b9c0 con 0x7fe87815db10 2023-10-09T06:38:30.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.926+0000 7fe86ffff700 1 -- 172.21.15.163:0/4196294914 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87000bd60 con 0x7fe87815db10 2023-10-09T06:38:30.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.926+0000 7fe87f7dd700 1 -- 172.21.15.163:0/4196294914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe87815db10 msgr2=0x7fe878069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:30.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.926+0000 7fe87f7dd700 1 --2- 172.21.15.163:0/4196294914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe87815db10 0x7fe878069560 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fe8700099b0 tx=0x7fe870009250 comp rx=0 tx=0).stop 2023-10-09T06:38:30.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.926+0000 7fe87f7dd700 1 -- 172.21.15.163:0/4196294914 shutdown_connections 2023-10-09T06:38:30.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.927+0000 7fe87f7dd700 1 --2- 172.21.15.163:0/4196294914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe87815db10 0x7fe878069560 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.927+0000 7fe87f7dd700 1 --2- 172.21.15.163:0/4196294914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe878159c60 0x7fe87815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.927+0000 7fe87f7dd700 1 -- 172.21.15.163:0/4196294914 >> 172.21.15.163:0/4196294914 conn(0x7fe878068e20 msgr2=0x7fe8780d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:30.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.927+0000 7fe87f7dd700 1 -- 172.21.15.163:0/4196294914 shutdown_connections 2023-10-09T06:38:30.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.927+0000 7fe87f7dd700 1 -- 172.21.15.163:0/4196294914 wait complete. 2023-10-09T06:38:30.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.928+0000 7fe87f7dd700 1 Processor -- start 2023-10-09T06:38:30.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.928+0000 7fe87f7dd700 1 -- start start 2023-10-09T06:38:30.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.928+0000 7fe87f7dd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe878159890 0x7fe8780ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:30.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.928+0000 7fe87f7dd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe878159c60 0x7fe87814be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:30.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.928+0000 7fe87f7dd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe87815db10 0x7fe87815cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:30.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.929+0000 7fe87f7dd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8780ce550 con 0x7fe878159c60 2023-10-09T06:38:30.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.929+0000 7fe87f7dd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87815ba40 con 0x7fe87815db10 2023-10-09T06:38:30.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.929+0000 7fe87f7dd700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe87815bd20 con 0x7fe878159890 2023-10-09T06:38:30.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.929+0000 7fe87cd78700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe878159c60 0x7fe87814be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:30.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.929+0000 7fe87cd78700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe878159c60 0x7fe87814be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:46998/0 (socket says 172.21.15.163:46998) 2023-10-09T06:38:30.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.929+0000 7fe87cd78700 1 -- 172.21.15.163:0/2687647266 learned_addr learned my addr 172.21.15.163:0/2687647266 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:30.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.929+0000 7fe87d579700 1 --2- 172.21.15.163:0/2687647266 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe878159890 0x7fe8780ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:30.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.929+0000 7fe87dd7a700 1 --2- 172.21.15.163:0/2687647266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe87815db10 0x7fe87815cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:30.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.929+0000 7fe87cd78700 1 -- 172.21.15.163:0/2687647266 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe878159890 msgr2=0x7fe8780ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:30.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.929+0000 7fe87cd78700 1 --2- 172.21.15.163:0/2687647266 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe878159890 0x7fe8780ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.929+0000 7fe87cd78700 1 -- 172.21.15.163:0/2687647266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe87815db10 msgr2=0x7fe87815cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:30.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.929+0000 7fe87cd78700 1 --2- 172.21.15.163:0/2687647266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe87815db10 0x7fe87815cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.929+0000 7fe87cd78700 1 -- 172.21.15.163:0/2687647266 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8700094f0 con 0x7fe878159c60 2023-10-09T06:38:30.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.929+0000 7fe87dd7a700 1 --2- 172.21.15.163:0/2687647266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe87815db10 0x7fe87815cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:30.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.929+0000 7fe87cd78700 1 --2- 172.21.15.163:0/2687647266 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe878159c60 0x7fe87814be70 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fe86800d860 tx=0x7fe868008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:30.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.929+0000 7fe86d7fa700 1 -- 172.21.15.163:0/2687647266 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8680125f0 con 0x7fe878159c60 2023-10-09T06:38:30.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.929+0000 7fe86d7fa700 1 -- 172.21.15.163:0/2687647266 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe868012750 con 0x7fe878159c60 2023-10-09T06:38:30.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.929+0000 7fe87f7dd700 1 -- 172.21.15.163:0/2687647266 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe87815c000 con 0x7fe878159c60 2023-10-09T06:38:30.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.929+0000 7fe86d7fa700 1 -- 172.21.15.163:0/2687647266 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe868012af0 con 0x7fe878159c60 2023-10-09T06:38:30.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.929+0000 7fe87f7dd700 1 -- 172.21.15.163:0/2687647266 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe87815c5c0 con 0x7fe878159c60 2023-10-09T06:38:30.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.931+0000 7fe86d7fa700 1 -- 172.21.15.163:0/2687647266 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe86801e460 con 0x7fe878159c60 2023-10-09T06:38:30.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.931+0000 7fe86d7fa700 1 --2- 172.21.15.163:0/2687647266 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe85c067320 0x7fe85c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:30.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.931+0000 7fe86d7fa700 1 -- 172.21.15.163:0/2687647266 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe868085260 con 0x7fe878159c60 2023-10-09T06:38:30.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.931+0000 7fe87d579700 1 --2- 172.21.15.163:0/2687647266 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe85c067320 0x7fe85c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:30.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.932+0000 7fe87d579700 1 --2- 172.21.15.163:0/2687647266 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe85c067320 0x7fe85c0697d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe874007fd0 tx=0x7fe8740016f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:30.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.933+0000 7fe888ad5b80 1 Processor -- start 2023-10-09T06:38:30.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.933+0000 7fe888ad5b80 1 -- start start 2023-10-09T06:38:30.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.934+0000 7fe888ad5b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d33fc1a50 0x564d33fc1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:30.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.934+0000 7fe888ad5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d3400bd00 0x564d33feaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:30.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.934+0000 7fe888ad5b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564d33feaf60 0x564d33fbdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:30.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.934+0000 7fe888ad5b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d33f18a70 con 0x564d33fc1a50 2023-10-09T06:38:30.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.934+0000 7fe888ad5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d33f16270 con 0x564d3400bd00 2023-10-09T06:38:30.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.934+0000 7fe888ad5b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d33f12060 con 0x564d33feaf60 2023-10-09T06:38:30.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.934+0000 7fe87d579700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d33fc1a50 0x564d33fc1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:30.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.934+0000 7fe87d579700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d33fc1a50 0x564d33fc1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47010/0 (socket says 172.21.15.163:47010) 2023-10-09T06:38:30.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.934+0000 7fe87d579700 1 -- 172.21.15.163:0/1888171911 learned_addr learned my addr 172.21.15.163:0/1888171911 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:30.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.934+0000 7fe87dd7a700 1 --2- 172.21.15.163:0/1888171911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564d33feaf60 0x564d33fbdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:30.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.934+0000 7fe87cd78700 1 --2- 172.21.15.163:0/1888171911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d3400bd00 0x564d33feaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:30.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.934+0000 7fe87d579700 1 -- 172.21.15.163:0/1888171911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564d33feaf60 msgr2=0x564d33fbdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:30.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.934+0000 7fe87d579700 1 --2- 172.21.15.163:0/1888171911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564d33feaf60 0x564d33fbdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.934+0000 7fe87d579700 1 -- 172.21.15.163:0/1888171911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d3400bd00 msgr2=0x564d33feaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:30.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.934+0000 7fe87d579700 1 --2- 172.21.15.163:0/1888171911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d3400bd00 0x564d33feaa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.934+0000 7fe87d579700 1 -- 172.21.15.163:0/1888171911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d33dac270 con 0x564d33fc1a50 2023-10-09T06:38:30.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.934+0000 7fe87d579700 1 --2- 172.21.15.163:0/1888171911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d33fc1a50 0x564d33fc1e20 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fe87400ed60 tx=0x7fe874014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4062d33baea9ca57 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:30.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.934+0000 7fe8627fc700 1 -- 172.21.15.163:0/1888171911 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe874002c60 con 0x564d33fc1a50 2023-10-09T06:38:30.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.934+0000 7fe8627fc700 1 -- 172.21.15.163:0/1888171911 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe874015c00 con 0x564d33fc1a50 2023-10-09T06:38:30.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.934+0000 7fe8627fc700 1 -- 172.21.15.163:0/1888171911 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe874020400 con 0x564d33fc1a50 2023-10-09T06:38:30.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.935+0000 7fe888ad5b80 1 -- 172.21.15.163:0/1888171911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d33fc1a50 msgr2=0x564d33fc1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:30.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.935+0000 7fe888ad5b80 1 --2- 172.21.15.163:0/1888171911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d33fc1a50 0x564d33fc1e20 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fe87400ed60 tx=0x7fe874014000 comp rx=0 tx=0).stop 2023-10-09T06:38:30.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.935+0000 7fe888ad5b80 1 -- 172.21.15.163:0/1888171911 shutdown_connections 2023-10-09T06:38:30.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.935+0000 7fe888ad5b80 1 --2- 172.21.15.163:0/1888171911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d33fc1a50 0x564d33fc1e20 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.935+0000 7fe888ad5b80 1 --2- 172.21.15.163:0/1888171911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564d33feaf60 0x564d33fbdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.935+0000 7fe888ad5b80 1 --2- 172.21.15.163:0/1888171911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d3400bd00 0x564d33feaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.935+0000 7fe888ad5b80 1 -- 172.21.15.163:0/1888171911 >> 172.21.15.163:0/1888171911 conn(0x564d33fc3370 msgr2=0x564d33fc3740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:30.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.935+0000 7fe888ad5b80 1 -- 172.21.15.163:0/1888171911 shutdown_connections 2023-10-09T06:38:30.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.935+0000 7fe888ad5b80 1 -- 172.21.15.163:0/1888171911 wait complete. 2023-10-09T06:38:30.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.935+0000 7fe888ad5b80 1 -- start start 2023-10-09T06:38:30.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.936+0000 7fe888ad5b80 1 Processor -- start 2023-10-09T06:38:30.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.936+0000 7fe888ad5b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d33fc1a50 0x564d3400b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:30.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.936+0000 7fe888ad5b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564d33feaf60 0x564d34003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:30.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.937+0000 7fe888ad5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d3400bd00 0x564d33fbe780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:30.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.937+0000 7fe888ad5b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d33ebd890 con 0x564d33fc1a50 2023-10-09T06:38:30.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.937+0000 7fe888ad5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d33ed8ea0 con 0x564d3400bd00 2023-10-09T06:38:30.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.937+0000 7fe888ad5b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d33d94e30 con 0x564d33feaf60 2023-10-09T06:38:30.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.937+0000 7fe888ad5b80 10 client.0 _ll_register_callbacks cb 0x564d33f85110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-09T06:38:30.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.937+0000 7fe87d579700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d33fc1a50 0x564d3400b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:30.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.937+0000 7fe87d579700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d33fc1a50 0x564d3400b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47014/0 (socket says 172.21.15.163:47014) 2023-10-09T06:38:30.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.937+0000 7fe87d579700 1 -- 172.21.15.163:0/1368236292 learned_addr learned my addr 172.21.15.163:0/1368236292 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:30.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.937+0000 7fe87cd78700 1 --2- 172.21.15.163:0/1368236292 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564d33feaf60 0x564d34003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:30.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.937+0000 7fe87dd7a700 1 --2- 172.21.15.163:0/1368236292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d3400bd00 0x564d33fbe780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:30.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.937+0000 7fe87d579700 1 -- 172.21.15.163:0/1368236292 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564d33feaf60 msgr2=0x564d34003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:30.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.937+0000 7fe87d579700 1 --2- 172.21.15.163:0/1368236292 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564d33feaf60 0x564d34003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.937+0000 7fe87d579700 1 -- 172.21.15.163:0/1368236292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d3400bd00 msgr2=0x564d33fbe780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:30.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.937+0000 7fe87d579700 1 --2- 172.21.15.163:0/1368236292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d3400bd00 0x564d33fbe780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.937+0000 7fe87d579700 1 -- 172.21.15.163:0/1368236292 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe87400f040 con 0x564d33fc1a50 2023-10-09T06:38:30.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.937+0000 7fe87d579700 1 --2- 172.21.15.163:0/1368236292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d33fc1a50 0x564d3400b7b0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fe87400ec50 tx=0x7fe874017030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1ae0a91ebcc397 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:30.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.937+0000 7fe8577fe700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-10-09T06:38:30.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.937+0000 7fe8577fe700 1 -- 172.21.15.163:0/1368236292 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe874006ba0 con 0x564d33fc1a50 2023-10-09T06:38:30.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.937+0000 7fe8577fe700 1 -- 172.21.15.163:0/1368236292 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe874006d00 con 0x564d33fc1a50 2023-10-09T06:38:30.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.937+0000 7fe888ad5b80 1 -- 172.21.15.163:0/1368236292 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564d33fb6b70 con 0x564d33fc1a50 2023-10-09T06:38:30.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.937+0000 7fe8577fe700 1 -- 172.21.15.163:0/1368236292 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe874020a40 con 0x564d33fc1a50 2023-10-09T06:38:30.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.937+0000 7fe888ad5b80 10 client.8037 fetch_fsmap learned FSMap version 103 2023-10-09T06:38:30.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.937+0000 7fe888ad5b80 1 -- 172.21.15.163:0/1368236292 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x564d33d4efe0 con 0x564d33fc1a50 2023-10-09T06:38:30.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.938+0000 7fe8577fe700 1 -- 172.21.15.163:0/1368236292 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564d33fb6b70 con 0x564d33fc1a50 2023-10-09T06:38:30.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.938+0000 7fe8577fe700 1 -- 172.21.15.163:0/1368236292 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 103) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe874020ba0 con 0x564d33fc1a50 2023-10-09T06:38:30.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.938+0000 7fe888ad5b80 10 client.8037 fetch_fsmap finished waiting for FSMap version 103 2023-10-09T06:38:30.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.938+0000 7fe888ad5b80 10 client.8037 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-09T06:38:30.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.938+0000 7fe888ad5b80 20 client.8037 populate_metadata read hostname 'smithi163' 2023-10-09T06:38:30.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.938+0000 7fe888ad5b80 1 --2- 172.21.15.163:0/1368236292 >> [v2:172.21.15.163:6833/3794912550,v1:172.21.15.163:6835/3794912550] conn(0x564d3401bb00 0x564d3401bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:30.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.938+0000 7fe888ad5b80 4 client.8037 mds_command: new command op to 7875 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-09T06:38:30.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.938+0000 7fe888ad5b80 1 -- 172.21.15.163:0/1368236292 --> [v2:172.21.15.163:6833/3794912550,v1:172.21.15.163:6835/3794912550] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d33f93aa0 con 0x564d3401bb00 2023-10-09T06:38:30.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.938+0000 7fe87cd78700 1 --2- 172.21.15.163:0/1368236292 >> [v2:172.21.15.163:6833/3794912550,v1:172.21.15.163:6835/3794912550] conn(0x564d3401bb00 0x564d3401bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:30.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.938+0000 7fe8577fe700 10 client.8037 ms_handle_connect on v2:172.21.15.163:6833/3794912550 2023-10-09T06:38:30.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.938+0000 7fe87cd78700 1 --2- 172.21.15.163:0/1368236292 >> [v2:172.21.15.163:6833/3794912550,v1:172.21.15.163:6835/3794912550] conn(0x564d3401bb00 0x564d3401bed0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=408f9033204b1090 server_cookie=8f13640cc7cff110 in_seq=0 out_seq=0 2023-10-09T06:38:30.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.939+0000 7fe8577fe700 1 -- 172.21.15.163:0/1368236292 <== mds.0 v2:172.21.15.163:6833/3794912550 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x564d33f93aa0 con 0x564d3401bb00 2023-10-09T06:38:30.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.939+0000 7fe8577fe700 10 client.8037 handle_command_reply: tid=0 2023-10-09T06:38:30.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.939+0000 7fe888ad5b80 1 client.8037 shutdown 2023-10-09T06:38:30.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.939+0000 7fe888ad5b80 1 -- 172.21.15.163:0/1368236292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d33fc1a50 msgr2=0x564d3400b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:30.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.939+0000 7fe888ad5b80 1 --2- 172.21.15.163:0/1368236292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d33fc1a50 0x564d3400b7b0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fe87400ec50 tx=0x7fe874017030 comp rx=0 tx=0).stop 2023-10-09T06:38:30.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.939+0000 7fe888ad5b80 1 -- 172.21.15.163:0/1368236292 shutdown_connections 2023-10-09T06:38:30.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.939+0000 7fe888ad5b80 1 --2- 172.21.15.163:0/1368236292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d33fc1a50 0x564d3400b7b0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.939+0000 7fe888ad5b80 1 --2- 172.21.15.163:0/1368236292 >> [v2:172.21.15.163:6833/3794912550,v1:172.21.15.163:6835/3794912550] conn(0x564d3401bb00 0x564d3401bed0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.939+0000 7fe888ad5b80 1 --2- 172.21.15.163:0/1368236292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d3400bd00 0x564d33fbe780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.939+0000 7fe888ad5b80 1 --2- 172.21.15.163:0/1368236292 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564d33feaf60 0x564d34003f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.939+0000 7fe8577fe700 0 client.8037 ms_handle_reset on v2:172.21.15.163:6833/3794912550 2023-10-09T06:38:30.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.939+0000 7fe888ad5b80 1 -- 172.21.15.163:0/1368236292 >> 172.21.15.163:0/1368236292 conn(0x564d33fc3370 msgr2=0x564d33fc2350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:30.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.939+0000 7fe888ad5b80 1 -- 172.21.15.163:0/1368236292 shutdown_connections 2023-10-09T06:38:30.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.939+0000 7fe888ad5b80 1 -- 172.21.15.163:0/1368236292 wait complete. 2023-10-09T06:38:30.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.939+0000 7fe888ad5b80 20 client.8037 trim_cache size 0 max 16384 2023-10-09T06:38:30.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.956+0000 7fe888ad5b80 1 Processor -- start 2023-10-09T06:38:30.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.956+0000 7fe888ad5b80 1 -- start start 2023-10-09T06:38:30.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.956+0000 7fe888ad5b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564d33fc1a50 0x564d34003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:30.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.956+0000 7fe888ad5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d33feaf60 0x564d340042c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:30.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.956+0000 7fe888ad5b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d3400bd00 0x564d34007130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:30.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.956+0000 7fe888ad5b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d33fc4500 con 0x564d3400bd00 2023-10-09T06:38:30.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.956+0000 7fe888ad5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d33ffff90 con 0x564d33feaf60 2023-10-09T06:38:30.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.956+0000 7fe888ad5b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d33fbc830 con 0x564d33fc1a50 2023-10-09T06:38:30.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.956+0000 7fe87dd7a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d3400bd00 0x564d34007130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:30.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.956+0000 7fe87dd7a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d3400bd00 0x564d34007130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47026/0 (socket says 172.21.15.163:47026) 2023-10-09T06:38:30.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.956+0000 7fe87dd7a700 1 -- 172.21.15.163:0/4222726618 learned_addr learned my addr 172.21.15.163:0/4222726618 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:30.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.957+0000 7fe87dd7a700 1 -- 172.21.15.163:0/4222726618 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564d33fc1a50 msgr2=0x564d34003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:30.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.957+0000 7fe87dd7a700 1 --2- 172.21.15.163:0/4222726618 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564d33fc1a50 0x564d34003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.957+0000 7fe87dd7a700 1 -- 172.21.15.163:0/4222726618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d33feaf60 msgr2=0x564d340042c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:30.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.957+0000 7fe87dd7a700 1 --2- 172.21.15.163:0/4222726618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d33feaf60 0x564d340042c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.957+0000 7fe87dd7a700 1 -- 172.21.15.163:0/4222726618 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d33ee0780 con 0x564d3400bd00 2023-10-09T06:38:30.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.957+0000 7fe87dd7a700 1 --2- 172.21.15.163:0/4222726618 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d3400bd00 0x564d34007130 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fe870006fd0 tx=0x7fe870017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d4d2c28d6129fd server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:30.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.957+0000 7fe862ffd700 1 -- 172.21.15.163:0/4222726618 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe870020040 con 0x564d3400bd00 2023-10-09T06:38:30.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.957+0000 7fe862ffd700 1 -- 172.21.15.163:0/4222726618 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe870021040 con 0x564d3400bd00 2023-10-09T06:38:30.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.957+0000 7fe862ffd700 1 -- 172.21.15.163:0/4222726618 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe870011630 con 0x564d3400bd00 2023-10-09T06:38:30.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.957+0000 7fe888ad5b80 1 -- 172.21.15.163:0/4222726618 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d3400bd00 msgr2=0x564d34007130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:30.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.957+0000 7fe888ad5b80 1 --2- 172.21.15.163:0/4222726618 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d3400bd00 0x564d34007130 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fe870006fd0 tx=0x7fe870017000 comp rx=0 tx=0).stop 2023-10-09T06:38:30.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.957+0000 7fe888ad5b80 1 -- 172.21.15.163:0/4222726618 shutdown_connections 2023-10-09T06:38:30.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.957+0000 7fe888ad5b80 1 --2- 172.21.15.163:0/4222726618 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d3400bd00 0x564d34007130 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.957+0000 7fe888ad5b80 1 --2- 172.21.15.163:0/4222726618 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564d33fc1a50 0x564d34003d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.957+0000 7fe888ad5b80 1 --2- 172.21.15.163:0/4222726618 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d33feaf60 0x564d340042c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.957+0000 7fe888ad5b80 1 -- 172.21.15.163:0/4222726618 >> 172.21.15.163:0/4222726618 conn(0x564d33fc3370 msgr2=0x564d3401afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:30.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.958+0000 7fe888ad5b80 1 -- 172.21.15.163:0/4222726618 shutdown_connections 2023-10-09T06:38:30.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.958+0000 7fe888ad5b80 1 -- 172.21.15.163:0/4222726618 wait complete. 2023-10-09T06:38:30.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.958+0000 7fe888ad5b80 1 -- start start 2023-10-09T06:38:30.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.958+0000 7fe888ad5b80 1 Processor -- start 2023-10-09T06:38:30.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.959+0000 7fe888ad5b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d33fc1a50 0x564d34002e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:30.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.959+0000 7fe888ad5b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d33feaf60 0x564d340033d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:30.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.959+0000 7fe888ad5b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564d3400bd00 0x564d34071430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:30.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.959+0000 7fe888ad5b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d33fba800 con 0x564d33feaf60 2023-10-09T06:38:30.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.959+0000 7fe888ad5b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3406dd30 con 0x564d33fc1a50 2023-10-09T06:38:30.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.959+0000 7fe888ad5b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d3406e190 con 0x564d3400bd00 2023-10-09T06:38:30.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.959+0000 7fe888ad5b80 10 client.0 _ll_register_callbacks cb 0x564d33f85110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-09T06:38:30.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.959+0000 7fe87cd78700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d33feaf60 0x564d340033d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:30.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.959+0000 7fe87cd78700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d33feaf60 0x564d340033d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47040/0 (socket says 172.21.15.163:47040) 2023-10-09T06:38:30.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.959+0000 7fe87cd78700 1 -- 172.21.15.163:0/1434882292 learned_addr learned my addr 172.21.15.163:0/1434882292 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:30.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.959+0000 7fe87dd7a700 1 --2- 172.21.15.163:0/1434882292 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564d3400bd00 0x564d34071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:30.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.959+0000 7fe87d579700 1 --2- 172.21.15.163:0/1434882292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d33fc1a50 0x564d34002e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:30.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.959+0000 7fe87cd78700 1 -- 172.21.15.163:0/1434882292 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564d3400bd00 msgr2=0x564d34071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:30.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.959+0000 7fe87cd78700 1 --2- 172.21.15.163:0/1434882292 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564d3400bd00 0x564d34071430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.959+0000 7fe87cd78700 1 -- 172.21.15.163:0/1434882292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d33fc1a50 msgr2=0x564d34002e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:30.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.959+0000 7fe87cd78700 1 --2- 172.21.15.163:0/1434882292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d33fc1a50 0x564d34002e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.959+0000 7fe87cd78700 1 -- 172.21.15.163:0/1434882292 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe87815a3d0 con 0x564d33feaf60 2023-10-09T06:38:30.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.960+0000 7fe87cd78700 1 --2- 172.21.15.163:0/1434882292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d33feaf60 0x564d340033d0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fe86800df00 tx=0x7fe868016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3bfec556b6d7ae8 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:30.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.960+0000 7fe8577fe700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-10-09T06:38:30.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.960+0000 7fe8577fe700 1 -- 172.21.15.163:0/1434882292 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe868053fd0 con 0x564d33feaf60 2023-10-09T06:38:30.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.960+0000 7fe8577fe700 1 -- 172.21.15.163:0/1434882292 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe868056020 con 0x564d33feaf60 2023-10-09T06:38:30.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.960+0000 7fe888ad5b80 1 -- 172.21.15.163:0/1434882292 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe874015c00 con 0x564d33feaf60 2023-10-09T06:38:30.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.960+0000 7fe8577fe700 1 -- 172.21.15.163:0/1434882292 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8680552b0 con 0x564d33feaf60 2023-10-09T06:38:30.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.960+0000 7fe888ad5b80 10 client.8043 fetch_fsmap learned FSMap version 103 2023-10-09T06:38:30.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.960+0000 7fe888ad5b80 1 -- 172.21.15.163:0/1434882292 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x564d33ec7470 con 0x564d33feaf60 2023-10-09T06:38:30.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.960+0000 7fe8577fe700 1 -- 172.21.15.163:0/1434882292 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe868060080 con 0x564d33feaf60 2023-10-09T06:38:30.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.960+0000 7fe8577fe700 1 -- 172.21.15.163:0/1434882292 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 103) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe86805c6c0 con 0x564d33feaf60 2023-10-09T06:38:30.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.960+0000 7fe888ad5b80 10 client.8043 fetch_fsmap finished waiting for FSMap version 103 2023-10-09T06:38:30.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.960+0000 7fe888ad5b80 10 client.8043 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-09T06:38:30.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.960+0000 7fe888ad5b80 20 client.8043 populate_metadata read hostname 'smithi163' 2023-10-09T06:38:30.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.960+0000 7fe888ad5b80 1 --2- 172.21.15.163:0/1434882292 >> [v2:172.21.15.163:6833/3794912550,v1:172.21.15.163:6835/3794912550] conn(0x564d34071970 0x564d34071d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:30.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.960+0000 7fe888ad5b80 4 client.8043 mds_command: new command op to 7875 tid=0[{"prefix": "status"}] 2023-10-09T06:38:30.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.960+0000 7fe888ad5b80 1 -- 172.21.15.163:0/1434882292 --> [v2:172.21.15.163:6833/3794912550,v1:172.21.15.163:6835/3794912550] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564d33eb4e00 con 0x564d34071970 2023-10-09T06:38:30.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.960+0000 7fe87d579700 1 --2- 172.21.15.163:0/1434882292 >> [v2:172.21.15.163:6833/3794912550,v1:172.21.15.163:6835/3794912550] conn(0x564d34071970 0x564d34071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:30.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.960+0000 7fe8577fe700 10 client.8043 ms_handle_connect on v2:172.21.15.163:6833/3794912550 2023-10-09T06:38:30.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.961+0000 7fe87d579700 1 --2- 172.21.15.163:0/1434882292 >> [v2:172.21.15.163:6833/3794912550,v1:172.21.15.163:6835/3794912550] conn(0x564d34071970 0x564d34071d40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52be00b0a547dffd server_cookie=729241f3f608aa4c in_seq=0 out_seq=0 2023-10-09T06:38:30.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.961+0000 7fe8577fe700 1 -- 172.21.15.163:0/1434882292 <== mds.0 v2:172.21.15.163:6833/3794912550 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x564d33eb4e00 con 0x564d34071970 2023-10-09T06:38:30.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.961+0000 7fe8577fe700 10 client.8043 handle_command_reply: tid=0 2023-10-09T06:38:30.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.961+0000 7fe888ad5b80 1 client.8043 shutdown 2023-10-09T06:38:30.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.961+0000 7fe888ad5b80 1 -- 172.21.15.163:0/1434882292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d33feaf60 msgr2=0x564d340033d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:30.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.961+0000 7fe888ad5b80 1 --2- 172.21.15.163:0/1434882292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d33feaf60 0x564d340033d0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fe86800df00 tx=0x7fe868016040 comp rx=0 tx=0).stop 2023-10-09T06:38:30.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.961+0000 7fe888ad5b80 1 -- 172.21.15.163:0/1434882292 shutdown_connections 2023-10-09T06:38:30.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.961+0000 7fe888ad5b80 1 --2- 172.21.15.163:0/1434882292 >> [v2:172.21.15.163:6833/3794912550,v1:172.21.15.163:6835/3794912550] conn(0x564d34071970 0x564d34071d40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.961+0000 7fe888ad5b80 1 --2- 172.21.15.163:0/1434882292 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564d3400bd00 0x564d34071430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.961+0000 7fe8577fe700 0 client.8043 ms_handle_reset on v2:172.21.15.163:6833/3794912550 2023-10-09T06:38:30.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.961+0000 7fe888ad5b80 1 --2- 172.21.15.163:0/1434882292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564d33fc1a50 0x564d34002e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.961+0000 7fe888ad5b80 1 --2- 172.21.15.163:0/1434882292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d33feaf60 0x564d340033d0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.961+0000 7fe888ad5b80 1 -- 172.21.15.163:0/1434882292 >> 172.21.15.163:0/1434882292 conn(0x564d33fc3370 msgr2=0x564d3401afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:30.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.961+0000 7fe888ad5b80 1 -- 172.21.15.163:0/1434882292 shutdown_connections 2023-10-09T06:38:30.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.961+0000 7fe888ad5b80 1 -- 172.21.15.163:0/1434882292 wait complete. 2023-10-09T06:38:30.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.961+0000 7fe888ad5b80 20 client.8043 trim_cache size 0 max 16384 2023-10-09T06:38:30.969 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:38:30.969 INFO:teuthology.orchestra.run.smithi163.stdout: "cluster_fsid": "2520b8f7-539f-4bae-9f17-1fce66f66ddd", 2023-10-09T06:38:30.969 INFO:teuthology.orchestra.run.smithi163.stdout: "whoami": 0, 2023-10-09T06:38:30.969 INFO:teuthology.orchestra.run.smithi163.stdout: "id": 7875, 2023-10-09T06:38:30.969 INFO:teuthology.orchestra.run.smithi163.stdout: "want_state": "up:active", 2023-10-09T06:38:30.969 INFO:teuthology.orchestra.run.smithi163.stdout: "state": "up:active", 2023-10-09T06:38:30.969 INFO:teuthology.orchestra.run.smithi163.stdout: "fs_name": "cephfs", 2023-10-09T06:38:30.969 INFO:teuthology.orchestra.run.smithi163.stdout: "rank_uptime": 10.273005697, 2023-10-09T06:38:30.969 INFO:teuthology.orchestra.run.smithi163.stdout: "mdsmap_epoch": 103, 2023-10-09T06:38:30.969 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch": 152, 2023-10-09T06:38:30.969 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch_barrier": 152, 2023-10-09T06:38:30.970 INFO:teuthology.orchestra.run.smithi163.stdout: "uptime": 15.405652586 2023-10-09T06:38:30.970 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:38:30.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.961+0000 7fe87f7dd700 1 -- 172.21.15.163:0/2687647266 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe85c067320 msgr2=0x7fe85c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:30.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.961+0000 7fe87f7dd700 1 --2- 172.21.15.163:0/2687647266 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe85c067320 0x7fe85c0697d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe874007fd0 tx=0x7fe8740016f0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.961+0000 7fe87f7dd700 1 -- 172.21.15.163:0/2687647266 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe878159c60 msgr2=0x7fe87814be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:30.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.961+0000 7fe87f7dd700 1 --2- 172.21.15.163:0/2687647266 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe878159c60 0x7fe87814be70 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fe86800d860 tx=0x7fe868008040 comp rx=0 tx=0).stop 2023-10-09T06:38:30.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.962+0000 7fe87f7dd700 1 -- 172.21.15.163:0/2687647266 shutdown_connections 2023-10-09T06:38:30.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.962+0000 7fe87f7dd700 1 --2- 172.21.15.163:0/2687647266 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe85c067320 0x7fe85c0697d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.962+0000 7fe87f7dd700 1 --2- 172.21.15.163:0/2687647266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe87815db10 0x7fe87815cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.962+0000 7fe87f7dd700 1 --2- 172.21.15.163:0/2687647266 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe878159890 0x7fe8780ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.962+0000 7fe87f7dd700 1 --2- 172.21.15.163:0/2687647266 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe878159c60 0x7fe87814be70 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:30.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.962+0000 7fe87f7dd700 1 -- 172.21.15.163:0/2687647266 >> 172.21.15.163:0/2687647266 conn(0x7fe878068e20 msgr2=0x7fe87814d780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:30.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.962+0000 7fe87f7dd700 1 -- 172.21.15.163:0/2687647266 shutdown_connections 2023-10-09T06:38:30.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:30.962+0000 7fe87f7dd700 1 -- 172.21.15.163:0/2687647266 wait complete. 2023-10-09T06:38:30.973 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-09T06:38:31.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.089+0000 7f6aa51a0700 1 Processor -- start 2023-10-09T06:38:31.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.089+0000 7f6aa51a0700 1 -- start start 2023-10-09T06:38:31.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.089+0000 7f6aa51a0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6aa01597f0 0x7f6aa00b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:31.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.089+0000 7f6aa51a0700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa00b11d0 con 0x7f6aa015d230 2023-10-09T06:38:31.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.089+0000 7f6aa51a0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa00b14b0 con 0x7f6aa0159420 2023-10-09T06:38:31.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.089+0000 7f6aa51a0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa00b1790 con 0x7f6aa01597f0 2023-10-09T06:38:31.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.090+0000 7f6aa499f700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f6aa015d230 0x7f6aa00b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:45988/0 (socket says 172.21.15.163:45988) 2023-10-09T06:38:31.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.090+0000 7f6aa499f700 1 -- 172.21.15.163:0/806010680 learned_addr learned my addr 172.21.15.163:0/806010680 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:31.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.090+0000 7f6a9f7fe700 1 --2- 172.21.15.163:0/806010680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6aa01597f0 0x7f6aa00b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:31.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.090+0000 7f6a9effd700 1 -- 172.21.15.163:0/806010680 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 933046060 0 0) 0x7f6aa00b11d0 con 0x7f6aa015d230 2023-10-09T06:38:31.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.090+0000 7f6a9effd700 1 -- 172.21.15.163:0/806010680 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a88003570 con 0x7f6aa015d230 2023-10-09T06:38:31.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.090+0000 7f6a9effd700 1 -- 172.21.15.163:0/806010680 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3582430082 0 0) 0x7f6aa00b14b0 con 0x7f6aa0159420 2023-10-09T06:38:31.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.090+0000 7f6a9effd700 1 -- 172.21.15.163:0/806010680 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6aa00b11d0 con 0x7f6aa0159420 2023-10-09T06:38:31.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.090+0000 7f6a9f7fe700 1 -- 172.21.15.163:0/806010680 >> v1:172.21.15.181:6789/0 conn(0x7f6aa0159420 legacy=0x7f6aa00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:31.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.090+0000 7f6a9f7fe700 1 -- 172.21.15.163:0/806010680 >> v1:172.21.15.163:6789/0 conn(0x7f6aa015d230 legacy=0x7f6aa00b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:31.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.090+0000 7f6a9f7fe700 1 -- 172.21.15.163:0/806010680 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa00b1a70 con 0x7f6aa01597f0 2023-10-09T06:38:31.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.090+0000 7f6a9f7fe700 1 --2- 172.21.15.163:0/806010680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6aa01597f0 0x7f6aa00b2270 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6a94009910 tx=0x7f6a940091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fa86c5bd622c8b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:31.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.090+0000 7f6a9effd700 1 -- 172.21.15.163:0/806010680 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a94011630 con 0x7f6aa01597f0 2023-10-09T06:38:31.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.090+0000 7f6a9effd700 1 -- 172.21.15.163:0/806010680 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a94011790 con 0x7f6aa01597f0 2023-10-09T06:38:31.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.090+0000 7f6a9effd700 1 -- 172.21.15.163:0/806010680 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a94011b30 con 0x7f6aa01597f0 2023-10-09T06:38:31.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.091+0000 7f6aa51a0700 1 -- 172.21.15.163:0/806010680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6aa01597f0 msgr2=0x7f6aa00b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:31.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.091+0000 7f6aa51a0700 1 --2- 172.21.15.163:0/806010680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6aa01597f0 0x7f6aa00b2270 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6a94009910 tx=0x7f6a940091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:31.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.091+0000 7f6aa51a0700 1 -- 172.21.15.163:0/806010680 shutdown_connections 2023-10-09T06:38:31.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.091+0000 7f6aa51a0700 1 --2- 172.21.15.163:0/806010680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6aa01597f0 0x7f6aa00b2270 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:31.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.091+0000 7f6aa51a0700 1 -- 172.21.15.163:0/806010680 >> 172.21.15.163:0/806010680 conn(0x7f6aa00d0660 msgr2=0x7f6aa00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:31.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.091+0000 7f6aa51a0700 1 -- 172.21.15.163:0/806010680 shutdown_connections 2023-10-09T06:38:31.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.091+0000 7f6aa51a0700 1 -- 172.21.15.163:0/806010680 wait complete. 2023-10-09T06:38:31.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.092+0000 7f6aa51a0700 1 Processor -- start 2023-10-09T06:38:31.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.092+0000 7f6aa51a0700 1 -- start start 2023-10-09T06:38:31.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.093+0000 7f6aa51a0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6aa0159420 0x7f6aa0157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:31.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.093+0000 7f6aa51a0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6aa01597f0 0x7f6aa016aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:31.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.093+0000 7f6aa51a0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6aa015d230 0x7f6aa00c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:31.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.093+0000 7f6aa51a0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa015a810 con 0x7f6aa015d230 2023-10-09T06:38:31.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.093+0000 7f6aa51a0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa0159c00 con 0x7f6aa0159420 2023-10-09T06:38:31.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.093+0000 7f6aa51a0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa0159ee0 con 0x7f6aa01597f0 2023-10-09T06:38:31.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.093+0000 7f6aa499f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6aa015d230 0x7f6aa00c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:31.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.093+0000 7f6aa499f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6aa015d230 0x7f6aa00c5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47044/0 (socket says 172.21.15.163:47044) 2023-10-09T06:38:31.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.093+0000 7f6aa499f700 1 -- 172.21.15.163:0/520505553 learned_addr learned my addr 172.21.15.163:0/520505553 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:31.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.093+0000 7f6a9ffff700 1 --2- 172.21.15.163:0/520505553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6aa0159420 0x7f6aa0157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:31.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.093+0000 7f6a9f7fe700 1 --2- 172.21.15.163:0/520505553 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6aa01597f0 0x7f6aa016aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:31.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.093+0000 7f6aa499f700 1 -- 172.21.15.163:0/520505553 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6aa01597f0 msgr2=0x7f6aa016aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:31.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.093+0000 7f6aa499f700 1 --2- 172.21.15.163:0/520505553 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6aa01597f0 0x7f6aa016aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:31.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.093+0000 7f6aa499f700 1 -- 172.21.15.163:0/520505553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6aa0159420 msgr2=0x7f6aa0157410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:31.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.093+0000 7f6aa499f700 1 --2- 172.21.15.163:0/520505553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6aa0159420 0x7f6aa0157410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:31.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.093+0000 7f6aa499f700 1 -- 172.21.15.163:0/520505553 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a94009420 con 0x7f6aa015d230 2023-10-09T06:38:31.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.094+0000 7f6aa499f700 1 --2- 172.21.15.163:0/520505553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6aa015d230 0x7f6aa00c5de0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800cc20 tx=0x7f6a98001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:31.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.094+0000 7f6a87fff700 1 -- 172.21.15.163:0/520505553 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9800a850 con 0x7f6aa015d230 2023-10-09T06:38:31.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.094+0000 7f6a87fff700 1 -- 172.21.15.163:0/520505553 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a9800a9d0 con 0x7f6aa015d230 2023-10-09T06:38:31.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.094+0000 7f6a87fff700 1 -- 172.21.15.163:0/520505553 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9800ad90 con 0x7f6aa015d230 2023-10-09T06:38:31.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.094+0000 7f6aa51a0700 1 -- 172.21.15.163:0/520505553 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa015a1c0 con 0x7f6aa015d230 2023-10-09T06:38:31.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.094+0000 7f6aa51a0700 1 -- 172.21.15.163:0/520505553 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa015c930 con 0x7f6aa015d230 2023-10-09T06:38:31.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.095+0000 7f6aa51a0700 1 -- 172.21.15.163:0/520505553 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa004c3c0 con 0x7f6aa015d230 2023-10-09T06:38:31.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.095+0000 7f6a87fff700 1 -- 172.21.15.163:0/520505553 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6a98007300 con 0x7f6aa015d230 2023-10-09T06:38:31.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.095+0000 7f6a87fff700 1 --2- 172.21.15.163:0/520505553 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6a8c067320 0x7f6a8c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:31.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.096+0000 7f6a87fff700 1 -- 172.21.15.163:0/520505553 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6a98084f40 con 0x7f6aa015d230 2023-10-09T06:38:31.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.096+0000 7f6a9ffff700 1 --2- 172.21.15.163:0/520505553 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6a8c067320 0x7f6a8c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:31.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.096+0000 7f6a9ffff700 1 --2- 172.21.15.163:0/520505553 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6a8c067320 0x7f6a8c0697d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6aa015ced0 tx=0x7f6a90007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:31.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.100+0000 7f6a87fff700 1 -- 172.21.15.163:0/520505553 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6a98051c50 con 0x7f6aa015d230 2023-10-09T06:38:31.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.325+0000 7f6aa51a0700 1 -- 172.21.15.163:0/520505553 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6aa00b1d50 con 0x7f6aa015d230 2023-10-09T06:38:31.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.326+0000 7f6a87fff700 1 -- 172.21.15.163:0/520505553 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v103) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6a98055270 con 0x7f6aa015d230 2023-10-09T06:38:31.326 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-09T06:38:31.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.328+0000 7f6aa51a0700 1 -- 172.21.15.163:0/520505553 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6a8c067320 msgr2=0x7f6a8c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:31.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.328+0000 7f6aa51a0700 1 --2- 172.21.15.163:0/520505553 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6a8c067320 0x7f6a8c0697d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6aa015ced0 tx=0x7f6a90007d50 comp rx=0 tx=0).stop 2023-10-09T06:38:31.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.328+0000 7f6aa51a0700 1 -- 172.21.15.163:0/520505553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6aa015d230 msgr2=0x7f6aa00c5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:31.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.328+0000 7f6aa51a0700 1 --2- 172.21.15.163:0/520505553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6aa015d230 0x7f6aa00c5de0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800cc20 tx=0x7f6a98001a40 comp rx=0 tx=0).stop 2023-10-09T06:38:31.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.328+0000 7f6aa51a0700 1 -- 172.21.15.163:0/520505553 shutdown_connections 2023-10-09T06:38:31.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.328+0000 7f6aa51a0700 1 --2- 172.21.15.163:0/520505553 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6a8c067320 0x7f6a8c0697d0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:31.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.328+0000 7f6aa51a0700 1 --2- 172.21.15.163:0/520505553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6aa015d230 0x7f6aa00c5de0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:31.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.328+0000 7f6aa51a0700 1 --2- 172.21.15.163:0/520505553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6aa0159420 0x7f6aa0157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:31.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.328+0000 7f6aa51a0700 1 --2- 172.21.15.163:0/520505553 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6aa01597f0 0x7f6aa016aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:31.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.328+0000 7f6aa51a0700 1 -- 172.21.15.163:0/520505553 >> 172.21.15.163:0/520505553 conn(0x7f6aa00d0660 msgr2=0x7f6aa00af2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:31.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.328+0000 7f6aa51a0700 1 -- 172.21.15.163:0/520505553 shutdown_connections 2023-10-09T06:38:31.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:31.328+0000 7f6aa51a0700 1 -- 172.21.15.163:0/520505553 wait complete. 2023-10-09T06:38:31.340 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-09T06:38:31.340 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-09T06:38:31.340 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi163.front.sepia.ceph.com 2023-10-09T06:38:31.340 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-09T06:38:31.340 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-09T06:38:31.340 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-09T06:38:31.340 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-09T06:38:31.340 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-09T06:38:31.340 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:38:31.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:31.341 DEBUG:teuthology.orchestra.run.smithi163:> ip addr 2023-10-09T06:38:31.356 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:38:31.357 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:38:31.357 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:38:31.357 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:38:31.357 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2023-10-09T06:38:31.357 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:38:31.357 INFO:teuthology.orchestra.run.smithi163.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:38:31.357 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:06 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:38:31.357 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f0 2023-10-09T06:38:31.357 INFO:teuthology.orchestra.run.smithi163.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:38:31.357 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:07 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:38:31.357 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f1 2023-10-09T06:38:31.358 INFO:teuthology.orchestra.run.smithi163.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:38:31.358 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5a brd ff:ff:ff:ff:ff:ff 2023-10-09T06:38:31.358 INFO:teuthology.orchestra.run.smithi163.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-09T06:38:31.358 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5b brd ff:ff:ff:ff:ff:ff 2023-10-09T06:38:31.358 INFO:teuthology.orchestra.run.smithi163.stdout: inet 172.21.15.163/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-09T06:38:31.358 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft 42012sec preferred_lft 42012sec 2023-10-09T06:38:31.358 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::ec4:7aff:fe88:7d5b/64 scope link 2023-10-09T06:38:31.358 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:38:31.358 INFO:teuthology.orchestra.run.smithi163.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-09T06:38:31.358 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 2e:01:84:b9:f4:8e brd ff:ff:ff:ff:ff:ff 2023-10-09T06:38:31.358 INFO:teuthology.orchestra.run.smithi163.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-09T06:38:31.359 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:38:31.359 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::d03c:93ff:fe9e:c8cb/64 scope link 2023-10-09T06:38:31.359 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:38:31.359 INFO:teuthology.orchestra.run.smithi163.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:38:31.359 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 2e:01:84:b9:f4:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-09T06:38:31.359 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::2c01:84ff:feb9:f48e/64 scope link 2023-10-09T06:38:31.359 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:38:31.359 INFO:teuthology.orchestra.run.smithi163.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:38:31.359 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether c6:88:3e:d7:c1:56 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-09T06:38:31.359 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::c488:3eff:fed7:c156/64 scope link 2023-10-09T06:38:31.359 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:38:31.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:31.360 DEBUG:teuthology.orchestra.run.smithi163:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-09T06:38:31.420 INFO:teuthology.orchestra.run.smithi163.stdout:1 2023-10-09T06:38:31.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:31.422 DEBUG:teuthology.orchestra.run.smithi163:> route 2023-10-09T06:38:31.478 INFO:teuthology.orchestra.run.smithi163.stdout:Kernel IP routing table 2023-10-09T06:38:31.478 INFO:teuthology.orchestra.run.smithi163.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-09T06:38:31.478 INFO:teuthology.orchestra.run.smithi163.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-09T06:38:31.478 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-09T06:38:31.478 INFO:teuthology.orchestra.run.smithi163.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-09T06:38:31.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:31.478 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-09T06:38:31.479 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-09T06:38:31.479 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-09T06:38:31.479 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-09T06:38:31.479 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-09T06:38:31.479 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-09T06:38:31.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:31.579 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2023-10-09T06:38:31.594 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-09T06:38:31.594 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-09T06:38:31.594 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:38:31.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:31.595 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set brx.0 up 2023-10-09T06:38:31.658 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:38:31.658 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-09T06:38:31.712 INFO:teuthology.orchestra.run.smithi163.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:38:31.713 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:38:31.713 DEBUG:teuthology.orchestra.run.smithi163:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-09T06:38:31.767 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-09T06:38:31.767 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-09T06:38:31.768 DEBUG:teuthology.orchestra.run.smithi163:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:38:31.848 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:38:31.848 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:38:31.865 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:38:31.866 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:38:31.866 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:38:31.866 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with new device syntax: 0@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:38:31.866 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:38:31.866 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:38:31.866 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:38:31.866 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:38:31.889 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:38:31.889 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:38:31.889 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:38:31.889 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with new device syntax: 0@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:38:31.889 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:38:31.889 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:38:31.889 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:38:31.889 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:38:31.890 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790" 2023-10-09T06:38:31.890 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790:/ 2023-10-09T06:38:31.890 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=2520b8f7-539f-4bae-9f17-1fce66f66ddd" will pass to kernel 2023-10-09T06:38:31.891 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-09T06:38:31.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:31.891 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-09T06:38:31.956 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-09T06:38:32.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.110+0000 7f87af59e700 1 Processor -- start 2023-10-09T06:38:32.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.110+0000 7f87af59e700 1 -- start start 2023-10-09T06:38:32.112 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.111+0000 7f87af59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87b015d230 0x7f87b015d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:32.112 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.111+0000 7f87af59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b00b2980 con 0x7f87b01597f0 2023-10-09T06:38:32.112 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.111+0000 7f87af59e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b015dbb0 con 0x7f87b0159420 2023-10-09T06:38:32.112 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.111+0000 7f87af59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b00b11d0 con 0x7f87b015d230 2023-10-09T06:38:32.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.111+0000 7f87add9b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f87b01597f0 0x7f87b00b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:46000/0 (socket says 172.21.15.163:46000) 2023-10-09T06:38:32.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.111+0000 7f87add9b700 1 -- 172.21.15.163:0/4135982156 learned_addr learned my addr 172.21.15.163:0/4135982156 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:32.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.111+0000 7f87aed9d700 1 --2- 172.21.15.163:0/4135982156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87b015d230 0x7f87b015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:32.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.111+0000 7f87ad59a700 1 -- 172.21.15.163:0/4135982156 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 264704344 0 0) 0x7f87b00b2980 con 0x7f87b01597f0 2023-10-09T06:38:32.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.111+0000 7f87ad59a700 1 -- 172.21.15.163:0/4135982156 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8798003570 con 0x7f87b01597f0 2023-10-09T06:38:32.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.111+0000 7f87ad59a700 1 -- 172.21.15.163:0/4135982156 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 790762961 0 0) 0x7f87b015dbb0 con 0x7f87b0159420 2023-10-09T06:38:32.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.111+0000 7f87ad59a700 1 -- 172.21.15.163:0/4135982156 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87b00b2980 con 0x7f87b0159420 2023-10-09T06:38:32.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.111+0000 7f87aed9d700 1 -- 172.21.15.163:0/4135982156 >> v1:172.21.15.181:6789/0 conn(0x7f87b0159420 legacy=0x7f87b00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:32.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.111+0000 7f87aed9d700 1 -- 172.21.15.163:0/4135982156 >> v1:172.21.15.163:6789/0 conn(0x7f87b01597f0 legacy=0x7f87b00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:32.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.111+0000 7f87aed9d700 1 -- 172.21.15.163:0/4135982156 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b00b14b0 con 0x7f87b015d230 2023-10-09T06:38:32.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.112+0000 7f87aed9d700 1 --2- 172.21.15.163:0/4135982156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87b015d230 0x7f87b015d670 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009e90 tx=0x7f87a8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89ee00daa93e67af server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:32.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.112+0000 7f87ad59a700 1 -- 172.21.15.163:0/4135982156 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a800e070 con 0x7f87b015d230 2023-10-09T06:38:32.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.112+0000 7f87ad59a700 1 -- 172.21.15.163:0/4135982156 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a800bb80 con 0x7f87b015d230 2023-10-09T06:38:32.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.112+0000 7f87ad59a700 1 -- 172.21.15.163:0/4135982156 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a8012430 con 0x7f87b015d230 2023-10-09T06:38:32.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.112+0000 7f87af59e700 1 -- 172.21.15.163:0/4135982156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87b015d230 msgr2=0x7f87b015d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:32.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.112+0000 7f87af59e700 1 --2- 172.21.15.163:0/4135982156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87b015d230 0x7f87b015d670 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009e90 tx=0x7f87a8009780 comp rx=0 tx=0).stop 2023-10-09T06:38:32.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.112+0000 7f87af59e700 1 -- 172.21.15.163:0/4135982156 shutdown_connections 2023-10-09T06:38:32.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.112+0000 7f87af59e700 1 --2- 172.21.15.163:0/4135982156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87b015d230 0x7f87b015d670 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:32.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.112+0000 7f87af59e700 1 -- 172.21.15.163:0/4135982156 >> 172.21.15.163:0/4135982156 conn(0x7f87b00d0660 msgr2=0x7f87b00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:32.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.112+0000 7f87af59e700 1 -- 172.21.15.163:0/4135982156 shutdown_connections 2023-10-09T06:38:32.114 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.112+0000 7f87af59e700 1 -- 172.21.15.163:0/4135982156 wait complete. 2023-10-09T06:38:32.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.113+0000 7f87af59e700 1 Processor -- start 2023-10-09T06:38:32.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.114+0000 7f87af59e700 1 -- start start 2023-10-09T06:38:32.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.114+0000 7f87af59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87b0159420 0x7f87b016d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:32.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.114+0000 7f87af59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87b01597f0 0x7f87b016d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:32.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.114+0000 7f87af59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87b015d230 0x7f87b00b4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:32.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.114+0000 7f87af59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b016ae60 con 0x7f87b01597f0 2023-10-09T06:38:32.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.114+0000 7f87af59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b016b700 con 0x7f87b015d230 2023-10-09T06:38:32.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.114+0000 7f87af59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b00b56b0 con 0x7f87b0159420 2023-10-09T06:38:32.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.114+0000 7f87add9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87b01597f0 0x7f87b016d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:32.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.114+0000 7f87add9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87b01597f0 0x7f87b016d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47054/0 (socket says 172.21.15.163:47054) 2023-10-09T06:38:32.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.114+0000 7f87add9b700 1 -- 172.21.15.163:0/1143418233 learned_addr learned my addr 172.21.15.163:0/1143418233 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:32.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.115+0000 7f87aed9d700 1 --2- 172.21.15.163:0/1143418233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87b015d230 0x7f87b00b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:32.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.115+0000 7f87ae59c700 1 --2- 172.21.15.163:0/1143418233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87b0159420 0x7f87b016d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:32.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.115+0000 7f87add9b700 1 -- 172.21.15.163:0/1143418233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87b0159420 msgr2=0x7f87b016d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:32.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.115+0000 7f87add9b700 1 --2- 172.21.15.163:0/1143418233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87b0159420 0x7f87b016d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:32.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.115+0000 7f87add9b700 1 -- 172.21.15.163:0/1143418233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87b015d230 msgr2=0x7f87b00b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:32.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.115+0000 7f87add9b700 1 --2- 172.21.15.163:0/1143418233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87b015d230 0x7f87b00b4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:32.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.115+0000 7f87add9b700 1 -- 172.21.15.163:0/1143418233 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a8009a20 con 0x7f87b01597f0 2023-10-09T06:38:32.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.115+0000 7f87add9b700 1 --2- 172.21.15.163:0/1143418233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87b01597f0 0x7f87b016d7d0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f87a400c790 tx=0x7f87a4007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:32.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.115+0000 7f879e7fc700 1 -- 172.21.15.163:0/1143418233 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a4005b30 con 0x7f87b01597f0 2023-10-09T06:38:32.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.115+0000 7f879e7fc700 1 -- 172.21.15.163:0/1143418233 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a4005c90 con 0x7f87b01597f0 2023-10-09T06:38:32.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.115+0000 7f87af59e700 1 -- 172.21.15.163:0/1143418233 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b016b0b0 con 0x7f87b01597f0 2023-10-09T06:38:32.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.115+0000 7f879e7fc700 1 -- 172.21.15.163:0/1143418233 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a4007300 con 0x7f87b01597f0 2023-10-09T06:38:32.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.115+0000 7f87af59e700 1 -- 172.21.15.163:0/1143418233 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b0154d90 con 0x7f87b01597f0 2023-10-09T06:38:32.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.116+0000 7f87af59e700 1 -- 172.21.15.163:0/1143418233 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b016b870 con 0x7f87b01597f0 2023-10-09T06:38:32.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.116+0000 7f879e7fc700 1 -- 172.21.15.163:0/1143418233 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f87a401b400 con 0x7f87b01597f0 2023-10-09T06:38:32.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.117+0000 7f879e7fc700 1 --2- 172.21.15.163:0/1143418233 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8790067320 0x7f87900697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:32.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.117+0000 7f879e7fc700 1 -- 172.21.15.163:0/1143418233 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f87a4084620 con 0x7f87b01597f0 2023-10-09T06:38:32.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.117+0000 7f87ae59c700 1 --2- 172.21.15.163:0/1143418233 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8790067320 0x7f87900697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:32.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.117+0000 7f87ae59c700 1 --2- 172.21.15.163:0/1143418233 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8790067320 0x7f87900697d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f87b01586b0 tx=0x7f87a0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:32.120 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.121+0000 7f879e7fc700 1 -- 172.21.15.163:0/1143418233 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f87a4054f70 con 0x7f87b01597f0 2023-10-09T06:38:32.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.346+0000 7f87af59e700 1 -- 172.21.15.163:0/1143418233 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f87b004c3c0 con 0x7f87b01597f0 2023-10-09T06:38:32.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.346+0000 7f879e7fc700 1 -- 172.21.15.163:0/1143418233 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v103) v1 ==== 53+0+73 (secure 0 0 0) 0x7f87a40076f0 con 0x7f87b01597f0 2023-10-09T06:38:32.346 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-09T06:38:32.348 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.348+0000 7f87af59e700 1 -- 172.21.15.163:0/1143418233 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8790067320 msgr2=0x7f87900697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:32.348 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.348+0000 7f87af59e700 1 --2- 172.21.15.163:0/1143418233 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8790067320 0x7f87900697d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f87b01586b0 tx=0x7f87a0009040 comp rx=0 tx=0).stop 2023-10-09T06:38:32.349 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.348+0000 7f87af59e700 1 -- 172.21.15.163:0/1143418233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87b01597f0 msgr2=0x7f87b016d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:32.349 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.348+0000 7f87af59e700 1 --2- 172.21.15.163:0/1143418233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87b01597f0 0x7f87b016d7d0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f87a400c790 tx=0x7f87a4007e40 comp rx=0 tx=0).stop 2023-10-09T06:38:32.349 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.348+0000 7f87af59e700 1 -- 172.21.15.163:0/1143418233 shutdown_connections 2023-10-09T06:38:32.349 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.348+0000 7f87af59e700 1 --2- 172.21.15.163:0/1143418233 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8790067320 0x7f87900697d0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:32.349 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.348+0000 7f87af59e700 1 --2- 172.21.15.163:0/1143418233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87b015d230 0x7f87b00b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:32.349 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.348+0000 7f87af59e700 1 --2- 172.21.15.163:0/1143418233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87b0159420 0x7f87b016d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:32.349 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.348+0000 7f87af59e700 1 --2- 172.21.15.163:0/1143418233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87b01597f0 0x7f87b016d7d0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:32.349 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.348+0000 7f87af59e700 1 -- 172.21.15.163:0/1143418233 >> 172.21.15.163:0/1143418233 conn(0x7f87b00d0660 msgr2=0x7f87b00d0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:32.349 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.348+0000 7f87af59e700 1 -- 172.21.15.163:0/1143418233 shutdown_connections 2023-10-09T06:38:32.349 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:32.348+0000 7f87af59e700 1 -- 172.21.15.163:0/1143418233 wait complete. 2023-10-09T06:38:32.360 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-09T06:38:32.361 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-09T06:38:32.361 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-09T06:38:32.361 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-09T06:38:32.361 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-09T06:38:32.361 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-09T06:38:32.361 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-09T06:38:32.361 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-09T06:38:32.361 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:38:32.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:32.362 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-09T06:38:32.380 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:38:32.380 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:38:32.381 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:38:32.381 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:38:32.381 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-09T06:38:32.381 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:38:32.381 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:38:32.381 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:38:32.381 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2023-10-09T06:38:32.381 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:38:32.381 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:38:32.381 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2023-10-09T06:38:32.381 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:38:32.382 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-09T06:38:32.382 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-09T06:38:32.382 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-09T06:38:32.382 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-09T06:38:32.382 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 41988sec preferred_lft 41988sec 2023-10-09T06:38:32.382 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2023-10-09T06:38:32.382 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:38:32.382 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-09T06:38:32.383 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 72:62:e6:72:7c:2e brd ff:ff:ff:ff:ff:ff 2023-10-09T06:38:32.383 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-09T06:38:32.383 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:38:32.383 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::e89a:4cff:fefb:7198/64 scope link 2023-10-09T06:38:32.383 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:38:32.384 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:38:32.384 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 72:62:e6:72:7c:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-09T06:38:32.384 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::7062:e6ff:fe72:7c2e/64 scope link 2023-10-09T06:38:32.384 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:38:32.384 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:38:32.384 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether fa:4e:a3:9d:5a:5f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-09T06:38:32.384 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::f84e:a3ff:fe9d:5a5f/64 scope link 2023-10-09T06:38:32.384 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:38:32.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:32.385 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-09T06:38:32.449 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-09T06:38:32.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:32.451 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-09T06:38:32.509 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-09T06:38:32.509 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-09T06:38:32.509 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-09T06:38:32.509 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-09T06:38:32.509 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-09T06:38:32.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:32.510 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-09T06:38:32.510 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-09T06:38:32.510 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-09T06:38:32.510 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-09T06:38:32.510 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-09T06:38:32.510 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-09T06:38:32.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:32.615 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-09T06:38:32.630 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-09T06:38:32.630 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-09T06:38:32.630 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:38:32.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:32.630 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-09T06:38:32.695 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:38:32.695 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-09T06:38:32.749 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:38:32.750 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:38:32.750 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-09T06:38:32.804 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-09T06:38:32.804 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-09T06:38:32.804 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:38:32.883 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:38:32.883 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:38:32.901 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:38:32.902 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:38:32.902 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:38:32.902 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 1@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:38:32.902 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:38:32.902 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:38:32.902 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:38:32.902 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:38:32.925 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:38:32.925 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:38:32.925 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:38:32.925 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 1@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:38:32.925 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:38:32.925 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:38:32.926 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:38:32.926 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:38:32.926 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790" 2023-10-09T06:38:32.926 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790:/ 2023-10-09T06:38:32.926 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=2520b8f7-539f-4bae-9f17-1fce66f66ddd" will pass to kernel 2023-10-09T06:38:32.927 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-09T06:38:32.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:32.927 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-09T06:38:32.992 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-09T06:38:33.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.113+0000 7fe77f11a700 1 Processor -- start 2023-10-09T06:38:33.113 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.113+0000 7fe77f11a700 1 -- start start 2023-10-09T06:38:33.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.114+0000 7fe77f11a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe778159ec0 0x7fe7780b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:33.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.114+0000 7fe77f11a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe7780b53d0 0x7fe7780cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:33.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.114+0000 7fe77f11a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7780b5b00 con 0x7fe7780cce40 2023-10-09T06:38:33.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.114+0000 7fe77f11a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7780b5da0 con 0x7fe7780b53d0 2023-10-09T06:38:33.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.114+0000 7fe77f11a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7780b6080 con 0x7fe778159ec0 2023-10-09T06:38:33.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.114+0000 7fe777fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe7780b53d0 0x7fe7780cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:33.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.114+0000 7fe777fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe7780b53d0 0x7fe7780cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47066/0 (socket says 172.21.15.163:47066) 2023-10-09T06:38:33.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.114+0000 7fe777fff700 1 -- 172.21.15.163:0/1011747127 learned_addr learned my addr 172.21.15.163:0/1011747127 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:33.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.114+0000 7fe77ceb6700 1 --2- 172.21.15.163:0/1011747127 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe778159ec0 0x7fe7780b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:33.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.114+0000 7fe777fff700 1 -- 172.21.15.163:0/1011747127 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe778159ec0 msgr2=0x7fe7780b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:33.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.114+0000 7fe777fff700 1 --2- 172.21.15.163:0/1011747127 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe778159ec0 0x7fe7780b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:33.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.114+0000 7fe777fff700 1 -- 172.21.15.163:0/1011747127 >> v1:172.21.15.181:6789/0 conn(0x7fe7780cce40 legacy=0x7fe7780cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:33.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.114+0000 7fe777fff700 1 -- 172.21.15.163:0/1011747127 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7780cc100 con 0x7fe7780b53d0 2023-10-09T06:38:33.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.114+0000 7fe777fff700 1 --2- 172.21.15.163:0/1011747127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7780b53d0 0x7fe7780cc900 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fe7600099b0 tx=0x7fe760009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d63cda9d1532e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:33.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.114+0000 7fe7777fe700 1 -- 172.21.15.163:0/1011747127 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76000b860 con 0x7fe7780b53d0 2023-10-09T06:38:33.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.114+0000 7fe7777fe700 1 -- 172.21.15.163:0/1011747127 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76000b9c0 con 0x7fe7780b53d0 2023-10-09T06:38:33.128 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.114+0000 7fe7777fe700 1 -- 172.21.15.163:0/1011747127 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76000bd60 con 0x7fe7780b53d0 2023-10-09T06:38:33.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.115+0000 7fe77f11a700 1 -- 172.21.15.163:0/1011747127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7780b53d0 msgr2=0x7fe7780cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:33.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.115+0000 7fe77f11a700 1 --2- 172.21.15.163:0/1011747127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7780b53d0 0x7fe7780cc900 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fe7600099b0 tx=0x7fe760009250 comp rx=0 tx=0).stop 2023-10-09T06:38:33.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.115+0000 7fe77f11a700 1 -- 172.21.15.163:0/1011747127 shutdown_connections 2023-10-09T06:38:33.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.115+0000 7fe77f11a700 1 --2- 172.21.15.163:0/1011747127 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe778159ec0 0x7fe7780b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:33.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.115+0000 7fe77f11a700 1 --2- 172.21.15.163:0/1011747127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7780b53d0 0x7fe7780cc900 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:33.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.115+0000 7fe77f11a700 1 -- 172.21.15.163:0/1011747127 >> 172.21.15.163:0/1011747127 conn(0x7fe7780999f0 msgr2=0x7fe7780b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:33.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.115+0000 7fe77f11a700 1 -- 172.21.15.163:0/1011747127 shutdown_connections 2023-10-09T06:38:33.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.115+0000 7fe77f11a700 1 -- 172.21.15.163:0/1011747127 wait complete. 2023-10-09T06:38:33.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.116+0000 7fe77f11a700 1 Processor -- start 2023-10-09T06:38:33.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.116+0000 7fe77f11a700 1 -- start start 2023-10-09T06:38:33.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.117+0000 7fe77f11a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7780b53d0 0x7fe778069b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:33.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.117+0000 7fe77f11a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7780cce40 0x7fe778155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:33.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.117+0000 7fe77f11a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe778159ec0 0x7fe778156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:33.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.117+0000 7fe77f11a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7780696d0 con 0x7fe7780cce40 2023-10-09T06:38:33.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.117+0000 7fe77f11a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe778159250 con 0x7fe7780b53d0 2023-10-09T06:38:33.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.117+0000 7fe77f11a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe778069080 con 0x7fe778159ec0 2023-10-09T06:38:33.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.117+0000 7fe777fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7780cce40 0x7fe778155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:33.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.117+0000 7fe777fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7780cce40 0x7fe778155fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47074/0 (socket says 172.21.15.163:47074) 2023-10-09T06:38:33.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.117+0000 7fe777fff700 1 -- 172.21.15.163:0/2836140813 learned_addr learned my addr 172.21.15.163:0/2836140813 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:33.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.117+0000 7fe77ceb6700 1 --2- 172.21.15.163:0/2836140813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7780b53d0 0x7fe778069b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:33.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.117+0000 7fe77d6b7700 1 --2- 172.21.15.163:0/2836140813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe778159ec0 0x7fe778156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:33.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.117+0000 7fe777fff700 1 -- 172.21.15.163:0/2836140813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe778159ec0 msgr2=0x7fe778156da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:33.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.117+0000 7fe777fff700 1 --2- 172.21.15.163:0/2836140813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe778159ec0 0x7fe778156da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:33.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.117+0000 7fe777fff700 1 -- 172.21.15.163:0/2836140813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7780b53d0 msgr2=0x7fe778069b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:33.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.117+0000 7fe777fff700 1 --2- 172.21.15.163:0/2836140813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7780b53d0 0x7fe778069b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:33.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.117+0000 7fe777fff700 1 -- 172.21.15.163:0/2836140813 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7600094f0 con 0x7fe7780cce40 2023-10-09T06:38:33.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.118+0000 7fe777fff700 1 --2- 172.21.15.163:0/2836140813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7780cce40 0x7fe778155fd0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fe778089390 tx=0x7fe760017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:33.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.118+0000 7fe774ff9700 1 -- 172.21.15.163:0/2836140813 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe760011e40 con 0x7fe7780cce40 2023-10-09T06:38:33.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.118+0000 7fe774ff9700 1 -- 172.21.15.163:0/2836140813 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe760024440 con 0x7fe7780cce40 2023-10-09T06:38:33.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.118+0000 7fe77f11a700 1 -- 172.21.15.163:0/2836140813 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe778069360 con 0x7fe7780cce40 2023-10-09T06:38:33.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.118+0000 7fe774ff9700 1 -- 172.21.15.163:0/2836140813 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe760024800 con 0x7fe7780cce40 2023-10-09T06:38:33.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.118+0000 7fe77f11a700 1 -- 172.21.15.163:0/2836140813 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe778158e00 con 0x7fe7780cce40 2023-10-09T06:38:33.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.119+0000 7fe77f11a700 1 -- 172.21.15.163:0/2836140813 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe778063c90 con 0x7fe7780cce40 2023-10-09T06:38:33.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.119+0000 7fe774ff9700 1 -- 172.21.15.163:0/2836140813 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe76001a420 con 0x7fe7780cce40 2023-10-09T06:38:33.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.119+0000 7fe774ff9700 1 --2- 172.21.15.163:0/2836140813 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe758067320 0x7fe7580697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:33.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.120+0000 7fe774ff9700 1 -- 172.21.15.163:0/2836140813 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe760086ae0 con 0x7fe7780cce40 2023-10-09T06:38:33.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.120+0000 7fe77ceb6700 1 --2- 172.21.15.163:0/2836140813 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe758067320 0x7fe7580697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:33.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.120+0000 7fe77ceb6700 1 --2- 172.21.15.163:0/2836140813 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe758067320 0x7fe7580697d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe7700075e0 tx=0x7fe770009830 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:33.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.124+0000 7fe774ff9700 1 -- 172.21.15.163:0/2836140813 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe760053730 con 0x7fe7780cce40 2023-10-09T06:38:33.351 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.351+0000 7fe77f11a700 1 -- 172.21.15.163:0/2836140813 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe7780718b0 con 0x7fe7780cce40 2023-10-09T06:38:33.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.735+0000 7fe774ff9700 1 -- 172.21.15.163:0/2836140813 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v104) v1 ==== 136+0+0 (secure 0 0 0) 0x7fe7600150c0 con 0x7fe7780cce40 2023-10-09T06:38:33.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.738+0000 7fe77f11a700 1 -- 172.21.15.163:0/2836140813 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe758067320 msgr2=0x7fe7580697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:33.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.738+0000 7fe77f11a700 1 --2- 172.21.15.163:0/2836140813 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe758067320 0x7fe7580697d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe7700075e0 tx=0x7fe770009830 comp rx=0 tx=0).stop 2023-10-09T06:38:33.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.738+0000 7fe77f11a700 1 -- 172.21.15.163:0/2836140813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7780cce40 msgr2=0x7fe778155fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:33.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.738+0000 7fe77f11a700 1 --2- 172.21.15.163:0/2836140813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7780cce40 0x7fe778155fd0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fe778089390 tx=0x7fe760017000 comp rx=0 tx=0).stop 2023-10-09T06:38:33.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.738+0000 7fe77f11a700 1 -- 172.21.15.163:0/2836140813 shutdown_connections 2023-10-09T06:38:33.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.738+0000 7fe77f11a700 1 --2- 172.21.15.163:0/2836140813 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe758067320 0x7fe7580697d0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:33.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.738+0000 7fe77f11a700 1 --2- 172.21.15.163:0/2836140813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe778159ec0 0x7fe778156da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:33.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.738+0000 7fe77f11a700 1 --2- 172.21.15.163:0/2836140813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7780b53d0 0x7fe778069b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:33.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.738+0000 7fe77f11a700 1 --2- 172.21.15.163:0/2836140813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7780cce40 0x7fe778155fd0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:33.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.738+0000 7fe77f11a700 1 -- 172.21.15.163:0/2836140813 >> 172.21.15.163:0/2836140813 conn(0x7fe7780999f0 msgr2=0x7fe778155650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:33.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.738+0000 7fe77f11a700 1 -- 172.21.15.163:0/2836140813 shutdown_connections 2023-10-09T06:38:33.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.738+0000 7fe77f11a700 1 -- 172.21.15.163:0/2836140813 wait complete. 2023-10-09T06:38:33.750 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-09T06:38:33.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.866+0000 7f402a9ac700 1 Processor -- start 2023-10-09T06:38:33.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.866+0000 7f402a9ac700 1 -- start start 2023-10-09T06:38:33.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.867+0000 7f402a9ac700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4024159890 0x7f402415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:33.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.867+0000 7f402a9ac700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4024159c60 0x7f402415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:33.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.867+0000 7f402a9ac700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f402415a160 con 0x7f402415db10 2023-10-09T06:38:33.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.867+0000 7f402a9ac700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40240d13a0 con 0x7f4024159c60 2023-10-09T06:38:33.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.867+0000 7f402a9ac700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f402415e140 con 0x7f4024159890 2023-10-09T06:38:33.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.867+0000 7f4028f49700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f402415db10 0x7f4024069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:46008/0 (socket says 172.21.15.163:46008) 2023-10-09T06:38:33.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.867+0000 7f4028f49700 1 -- 172.21.15.163:0/3916230005 learned_addr learned my addr 172.21.15.163:0/3916230005 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:33.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.868+0000 7f40237fe700 1 --2- 172.21.15.163:0/3916230005 >> v2:172.21.15.181:3300/0 conn(0x7f4024159c60 0x7f402415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:33.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.868+0000 7f4023fff700 1 --2- 172.21.15.163:0/3916230005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4024159890 0x7f402415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:33.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.868+0000 7f4022ffd700 1 -- 172.21.15.163:0/3916230005 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3310802857 0 0) 0x7f402415a160 con 0x7f402415db10 2023-10-09T06:38:33.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.868+0000 7f4022ffd700 1 -- 172.21.15.163:0/3916230005 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f400c003570 con 0x7f402415db10 2023-10-09T06:38:33.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.868+0000 7f4022ffd700 1 -- 172.21.15.163:0/3916230005 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3588433213 0 0) 0x7f400c003570 con 0x7f402415db10 2023-10-09T06:38:33.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.868+0000 7f40237fe700 1 -- 172.21.15.163:0/3916230005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4024159890 msgr2=0x7f402415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:33.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.868+0000 7f40237fe700 1 --2- 172.21.15.163:0/3916230005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4024159890 0x7f402415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:33.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.868+0000 7f40237fe700 1 -- 172.21.15.163:0/3916230005 >> v1:172.21.15.163:6789/0 conn(0x7f402415db10 legacy=0x7f4024069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:33.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.868+0000 7f40237fe700 1 -- 172.21.15.163:0/3916230005 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f402415a3d0 con 0x7f4024159c60 2023-10-09T06:38:33.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.868+0000 7f4023fff700 1 --2- 172.21.15.163:0/3916230005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4024159890 0x7f402415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:38:33.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.868+0000 7f40237fe700 1 --2- 172.21.15.163:0/3916230005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4024159c60 0x7f402415d5d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f40180098e0 tx=0x7f40180091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd221f6941f33241 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:33.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.868+0000 7f4022ffd700 1 -- 172.21.15.163:0/3916230005 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c0017f0 con 0x7f4024159c60 2023-10-09T06:38:33.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.868+0000 7f4022ffd700 1 -- 172.21.15.163:0/3916230005 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4018011420 con 0x7f4024159c60 2023-10-09T06:38:33.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.868+0000 7f4022ffd700 1 -- 172.21.15.163:0/3916230005 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40180117c0 con 0x7f4024159c60 2023-10-09T06:38:33.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.869+0000 7f402a9ac700 1 -- 172.21.15.163:0/3916230005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4024159c60 msgr2=0x7f402415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:33.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.869+0000 7f402a9ac700 1 --2- 172.21.15.163:0/3916230005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4024159c60 0x7f402415d5d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f40180098e0 tx=0x7f40180091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:33.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.869+0000 7f402a9ac700 1 -- 172.21.15.163:0/3916230005 shutdown_connections 2023-10-09T06:38:33.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.869+0000 7f402a9ac700 1 --2- 172.21.15.163:0/3916230005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4024159890 0x7f402415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:33.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.869+0000 7f402a9ac700 1 --2- 172.21.15.163:0/3916230005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4024159c60 0x7f402415d5d0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:33.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.869+0000 7f402a9ac700 1 -- 172.21.15.163:0/3916230005 >> 172.21.15.163:0/3916230005 conn(0x7f4024068e20 msgr2=0x7f40240d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:33.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.869+0000 7f402a9ac700 1 -- 172.21.15.163:0/3916230005 shutdown_connections 2023-10-09T06:38:33.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.869+0000 7f402a9ac700 1 -- 172.21.15.163:0/3916230005 wait complete. 2023-10-09T06:38:33.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.870+0000 7f402a9ac700 1 Processor -- start 2023-10-09T06:38:33.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.870+0000 7f402a9ac700 1 -- start start 2023-10-09T06:38:33.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.871+0000 7f402a9ac700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4024159890 0x7f40241441e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:33.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.871+0000 7f402a9ac700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4024159c60 0x7f402415b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:33.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.871+0000 7f402a9ac700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f402415db10 0x7f40240b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:33.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.871+0000 7f402a9ac700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402415cb60 con 0x7f402415db10 2023-10-09T06:38:33.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.871+0000 7f402a9ac700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402415bf50 con 0x7f4024159c60 2023-10-09T06:38:33.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.871+0000 7f402a9ac700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f402415c230 con 0x7f4024159890 2023-10-09T06:38:33.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.871+0000 7f4028f49700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f402415db10 0x7f40240b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:33.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.871+0000 7f4028f49700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f402415db10 0x7f40240b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47076/0 (socket says 172.21.15.163:47076) 2023-10-09T06:38:33.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.871+0000 7f4028f49700 1 -- 172.21.15.163:0/2971691643 learned_addr learned my addr 172.21.15.163:0/2971691643 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:33.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.871+0000 7f40237fe700 1 --2- 172.21.15.163:0/2971691643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4024159c60 0x7f402415b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:33.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.871+0000 7f4023fff700 1 --2- 172.21.15.163:0/2971691643 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4024159890 0x7f40241441e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:33.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.871+0000 7f4028f49700 1 -- 172.21.15.163:0/2971691643 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4024159890 msgr2=0x7f40241441e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:33.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.871+0000 7f4028f49700 1 --2- 172.21.15.163:0/2971691643 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4024159890 0x7f40241441e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:33.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.871+0000 7f4028f49700 1 -- 172.21.15.163:0/2971691643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4024159c60 msgr2=0x7f402415b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:33.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.871+0000 7f4028f49700 1 --2- 172.21.15.163:0/2971691643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4024159c60 0x7f402415b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:33.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.871+0000 7f4028f49700 1 -- 172.21.15.163:0/2971691643 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4014009420 con 0x7f402415db10 2023-10-09T06:38:33.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.871+0000 7f4023fff700 1 --2- 172.21.15.163:0/2971691643 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4024159890 0x7f40241441e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:33.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.871+0000 7f4028f49700 1 --2- 172.21.15.163:0/2971691643 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f402415db10 0x7f40240b3590 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f401c00cae0 tx=0x7f401c007ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:33.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.872+0000 7f400bfff700 1 -- 172.21.15.163:0/2971691643 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c004dc0 con 0x7f402415db10 2023-10-09T06:38:33.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.872+0000 7f400bfff700 1 -- 172.21.15.163:0/2971691643 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f401c00a5d0 con 0x7f402415db10 2023-10-09T06:38:33.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.872+0000 7f402a9ac700 1 -- 172.21.15.163:0/2971691643 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4018009420 con 0x7f402415db10 2023-10-09T06:38:33.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.872+0000 7f400bfff700 1 -- 172.21.15.163:0/2971691643 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c00a990 con 0x7f402415db10 2023-10-09T06:38:33.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.872+0000 7f402a9ac700 1 -- 172.21.15.163:0/2971691643 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f402415c7f0 con 0x7f402415db10 2023-10-09T06:38:33.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.872+0000 7f402a9ac700 1 -- 172.21.15.163:0/2971691643 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f402415ccd0 con 0x7f402415db10 2023-10-09T06:38:33.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.873+0000 7f400bfff700 1 -- 172.21.15.163:0/2971691643 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f401c007150 con 0x7f402415db10 2023-10-09T06:38:33.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.873+0000 7f400bfff700 1 --2- 172.21.15.163:0/2971691643 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4010067320 0x7f40100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:33.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.873+0000 7f400bfff700 1 -- 172.21.15.163:0/2971691643 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f401c0845f0 con 0x7f402415db10 2023-10-09T06:38:33.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.873+0000 7f4023fff700 1 --2- 172.21.15.163:0/2971691643 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4010067320 0x7f40100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:33.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.874+0000 7f4023fff700 1 --2- 172.21.15.163:0/2971691643 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4010067320 0x7f40100697d0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f40140093f0 tx=0x7f4014008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:33.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:33.877+0000 7f400bfff700 1 -- 172.21.15.163:0/2971691643 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f401c04c5a0 con 0x7f402415db10 2023-10-09T06:38:34.088 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.088+0000 7f402a9ac700 1 -- 172.21.15.163:0/2971691643 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f402404c3c0 con 0x7f4010067320 2023-10-09T06:38:34.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.090+0000 7f400bfff700 1 -- 172.21.15.163:0/2971691643 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f402404c3c0 con 0x7f4010067320 2023-10-09T06:38:34.089 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:38:34.090 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:34.090 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "cephfs" 2023-10-09T06:38:34.090 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:34.090 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:38:34.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.092+0000 7f402a9ac700 1 -- 172.21.15.163:0/2971691643 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4010067320 msgr2=0x7f40100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:34.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.092+0000 7f402a9ac700 1 --2- 172.21.15.163:0/2971691643 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4010067320 0x7f40100697d0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f40140093f0 tx=0x7f4014008040 comp rx=0 tx=0).stop 2023-10-09T06:38:34.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.092+0000 7f402a9ac700 1 -- 172.21.15.163:0/2971691643 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f402415db10 msgr2=0x7f40240b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:34.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.092+0000 7f402a9ac700 1 --2- 172.21.15.163:0/2971691643 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f402415db10 0x7f40240b3590 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f401c00cae0 tx=0x7f401c007ca0 comp rx=0 tx=0).stop 2023-10-09T06:38:34.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.092+0000 7f402a9ac700 1 -- 172.21.15.163:0/2971691643 shutdown_connections 2023-10-09T06:38:34.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.092+0000 7f402a9ac700 1 --2- 172.21.15.163:0/2971691643 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4010067320 0x7f40100697d0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:34.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.092+0000 7f402a9ac700 1 --2- 172.21.15.163:0/2971691643 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f402415db10 0x7f40240b3590 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:34.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.092+0000 7f402a9ac700 1 --2- 172.21.15.163:0/2971691643 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4024159890 0x7f40241441e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:34.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.092+0000 7f402a9ac700 1 --2- 172.21.15.163:0/2971691643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4024159c60 0x7f402415b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:34.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.092+0000 7f402a9ac700 1 -- 172.21.15.163:0/2971691643 >> 172.21.15.163:0/2971691643 conn(0x7f4024068e20 msgr2=0x7f4024152fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:34.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.092+0000 7f402a9ac700 1 -- 172.21.15.163:0/2971691643 shutdown_connections 2023-10-09T06:38:34.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.092+0000 7f402a9ac700 1 -- 172.21.15.163:0/2971691643 wait complete. 2023-10-09T06:38:34.104 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-09T06:38:34.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.221+0000 7fa8e913b700 1 Processor -- start 2023-10-09T06:38:34.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.221+0000 7fa8e913b700 1 -- start start 2023-10-09T06:38:34.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.222+0000 7fa8e913b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa8e406bcd0 0x7fa8e40caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:34.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.222+0000 7fa8e913b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa8e406c0a0 0x7fa8e415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:34.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.222+0000 7fa8e913b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8e415df50 0x7fa8e415e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:34.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.222+0000 7fa8e913b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e415d7e0 con 0x7fa8e406c0a0 2023-10-09T06:38:34.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.222+0000 7fa8e913b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e415dac0 con 0x7fa8e406bcd0 2023-10-09T06:38:34.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.222+0000 7fa8e913b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e415aa30 con 0x7fa8e415df50 2023-10-09T06:38:34.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.222+0000 7fa8e259c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa8e406c0a0 0x7fa8e415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:34.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.222+0000 7fa8e259c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa8e406c0a0 0x7fa8e415a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47090/0 (socket says 172.21.15.163:47090) 2023-10-09T06:38:34.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.222+0000 7fa8e259c700 1 -- 172.21.15.163:0/2867898623 learned_addr learned my addr 172.21.15.163:0/2867898623 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:34.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.222+0000 7fa8e359e700 1 --2- 172.21.15.163:0/2867898623 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8e415df50 0x7fa8e415e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:34.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.222+0000 7fa8e2d9d700 1 --2- 172.21.15.163:0/2867898623 >> v2:172.21.15.181:3300/0 conn(0x7fa8e406bcd0 0x7fa8e40caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:34.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.222+0000 7fa8e259c700 1 -- 172.21.15.163:0/2867898623 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8e415df50 msgr2=0x7fa8e415e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:34.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.222+0000 7fa8e259c700 1 --2- 172.21.15.163:0/2867898623 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8e415df50 0x7fa8e415e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:34.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.222+0000 7fa8e259c700 1 -- 172.21.15.163:0/2867898623 >> v2:172.21.15.181:3300/0 conn(0x7fa8e406bcd0 msgr2=0x7fa8e40caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:34.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.222+0000 7fa8e259c700 1 --2- 172.21.15.163:0/2867898623 >> v2:172.21.15.181:3300/0 conn(0x7fa8e406bcd0 0x7fa8e40caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:34.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.222+0000 7fa8e259c700 1 -- 172.21.15.163:0/2867898623 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e415ad10 con 0x7fa8e406c0a0 2023-10-09T06:38:34.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.222+0000 7fa8e2d9d700 1 --2- 172.21.15.163:0/2867898623 >> v2:172.21.15.181:3300/0 conn(0x7fa8e406bcd0 0x7fa8e40caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:34.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.223+0000 7fa8e259c700 1 --2- 172.21.15.163:0/2867898623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8e406c0a0 0x7fa8e415a1e0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fa8e40691a0 tx=0x7fa8d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385d061ae50451eb server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:34.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.223+0000 7fa8e1d9b700 1 -- 172.21.15.163:0/2867898623 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d800b860 con 0x7fa8e406c0a0 2023-10-09T06:38:34.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.223+0000 7fa8e1d9b700 1 -- 172.21.15.163:0/2867898623 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d800b9c0 con 0x7fa8e406c0a0 2023-10-09T06:38:34.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.223+0000 7fa8e1d9b700 1 -- 172.21.15.163:0/2867898623 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d800bd80 con 0x7fa8e406c0a0 2023-10-09T06:38:34.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.223+0000 7fa8e913b700 1 -- 172.21.15.163:0/2867898623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8e406c0a0 msgr2=0x7fa8e415a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:34.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.223+0000 7fa8e913b700 1 --2- 172.21.15.163:0/2867898623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8e406c0a0 0x7fa8e415a1e0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fa8e40691a0 tx=0x7fa8d8009250 comp rx=0 tx=0).stop 2023-10-09T06:38:34.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.223+0000 7fa8e913b700 1 -- 172.21.15.163:0/2867898623 shutdown_connections 2023-10-09T06:38:34.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.223+0000 7fa8e913b700 1 --2- 172.21.15.163:0/2867898623 >> v2:172.21.15.181:3300/0 conn(0x7fa8e406bcd0 0x7fa8e40caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:34.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.223+0000 7fa8e913b700 1 --2- 172.21.15.163:0/2867898623 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8e415df50 0x7fa8e415e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:34.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.223+0000 7fa8e913b700 1 --2- 172.21.15.163:0/2867898623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8e406c0a0 0x7fa8e415a1e0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:34.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.223+0000 7fa8e913b700 1 -- 172.21.15.163:0/2867898623 >> 172.21.15.163:0/2867898623 conn(0x7fa8e40ce600 msgr2=0x7fa8e40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:34.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.223+0000 7fa8e913b700 1 -- 172.21.15.163:0/2867898623 shutdown_connections 2023-10-09T06:38:34.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.223+0000 7fa8e913b700 1 -- 172.21.15.163:0/2867898623 wait complete. 2023-10-09T06:38:34.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.225+0000 7fa8e913b700 1 Processor -- start 2023-10-09T06:38:34.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.225+0000 7fa8e913b700 1 -- start start 2023-10-09T06:38:34.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.225+0000 7fa8e913b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8e406bcd0 0x7fa8e414a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:34.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.225+0000 7fa8e913b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8e406c0a0 0x7fa8e40c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:34.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.225+0000 7fa8e913b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8e415df50 0x7fa8e416b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:34.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.225+0000 7fa8e913b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e416b080 con 0x7fa8e406bcd0 2023-10-09T06:38:34.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.225+0000 7fa8e913b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e40c83c0 con 0x7fa8e406c0a0 2023-10-09T06:38:34.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.225+0000 7fa8e913b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e40c86a0 con 0x7fa8e415df50 2023-10-09T06:38:34.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.225+0000 7fa8e2d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8e406bcd0 0x7fa8e414a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:34.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.225+0000 7fa8e2d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8e406bcd0 0x7fa8e414a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47098/0 (socket says 172.21.15.163:47098) 2023-10-09T06:38:34.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.225+0000 7fa8e2d9d700 1 -- 172.21.15.163:0/3791102940 learned_addr learned my addr 172.21.15.163:0/3791102940 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:34.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.226+0000 7fa8e359e700 1 --2- 172.21.15.163:0/3791102940 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8e415df50 0x7fa8e416b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:34.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.226+0000 7fa8e259c700 1 --2- 172.21.15.163:0/3791102940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8e406c0a0 0x7fa8e40c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:34.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.226+0000 7fa8e2d9d700 1 -- 172.21.15.163:0/3791102940 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8e415df50 msgr2=0x7fa8e416b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:34.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.226+0000 7fa8e2d9d700 1 --2- 172.21.15.163:0/3791102940 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8e415df50 0x7fa8e416b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:34.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.226+0000 7fa8e2d9d700 1 -- 172.21.15.163:0/3791102940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8e406c0a0 msgr2=0x7fa8e40c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:34.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.226+0000 7fa8e2d9d700 1 --2- 172.21.15.163:0/3791102940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8e406c0a0 0x7fa8e40c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:34.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.226+0000 7fa8e2d9d700 1 -- 172.21.15.163:0/3791102940 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d80094f0 con 0x7fa8e406bcd0 2023-10-09T06:38:34.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.226+0000 7fa8e2d9d700 1 --2- 172.21.15.163:0/3791102940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8e406bcd0 0x7fa8e414a150 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400d860 tx=0x7fa8d4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:34.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.226+0000 7fa8caffd700 1 -- 172.21.15.163:0/3791102940 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d40125f0 con 0x7fa8e406bcd0 2023-10-09T06:38:34.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.226+0000 7fa8caffd700 1 -- 172.21.15.163:0/3791102940 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d4012750 con 0x7fa8e406bcd0 2023-10-09T06:38:34.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.226+0000 7fa8e913b700 1 -- 172.21.15.163:0/3791102940 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8e40c8980 con 0x7fa8e406bcd0 2023-10-09T06:38:34.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.226+0000 7fa8caffd700 1 -- 172.21.15.163:0/3791102940 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4012ad0 con 0x7fa8e406bcd0 2023-10-09T06:38:34.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.226+0000 7fa8e913b700 1 -- 172.21.15.163:0/3791102940 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e40c8070 con 0x7fa8e406bcd0 2023-10-09T06:38:34.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.227+0000 7fa8e913b700 1 -- 172.21.15.163:0/3791102940 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8e404c3c0 con 0x7fa8e406bcd0 2023-10-09T06:38:34.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.227+0000 7fa8caffd700 1 -- 172.21.15.163:0/3791102940 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa8d4014430 con 0x7fa8e406bcd0 2023-10-09T06:38:34.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.228+0000 7fa8caffd700 1 --2- 172.21.15.163:0/3791102940 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa8d0067320 0x7fa8d00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:34.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.228+0000 7fa8caffd700 1 -- 172.21.15.163:0/3791102940 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa8d4085100 con 0x7fa8e406bcd0 2023-10-09T06:38:34.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.228+0000 7fa8e259c700 1 --2- 172.21.15.163:0/3791102940 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa8d0067320 0x7fa8d00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:34.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.228+0000 7fa8e259c700 1 --2- 172.21.15.163:0/3791102940 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa8d0067320 0x7fa8d00697d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa8e40691a0 tx=0x7fa8d8017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:34.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.232+0000 7fa8caffd700 1 -- 172.21.15.163:0/3791102940 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa8d4055820 con 0x7fa8e406bcd0 2023-10-09T06:38:34.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.414+0000 7fa8e913b700 1 -- 172.21.15.163:0/3791102940 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa8e415aff0 con 0x7fa8e406bcd0 2023-10-09T06:38:34.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.414+0000 7fa8caffd700 1 -- 172.21.15.163:0/3791102940 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa8d40553b0 con 0x7fa8e406bcd0 2023-10-09T06:38:34.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.416+0000 7fa8e913b700 1 -- 172.21.15.163:0/3791102940 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa8d0067320 msgr2=0x7fa8d00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:34.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.416+0000 7fa8e913b700 1 --2- 172.21.15.163:0/3791102940 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa8d0067320 0x7fa8d00697d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa8e40691a0 tx=0x7fa8d8017000 comp rx=0 tx=0).stop 2023-10-09T06:38:34.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.416+0000 7fa8e913b700 1 -- 172.21.15.163:0/3791102940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8e406bcd0 msgr2=0x7fa8e414a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:34.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.416+0000 7fa8e913b700 1 --2- 172.21.15.163:0/3791102940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8e406bcd0 0x7fa8e414a150 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400d860 tx=0x7fa8d4008040 comp rx=0 tx=0).stop 2023-10-09T06:38:34.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.417+0000 7fa8e913b700 1 -- 172.21.15.163:0/3791102940 shutdown_connections 2023-10-09T06:38:34.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.417+0000 7fa8e913b700 1 --2- 172.21.15.163:0/3791102940 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa8d0067320 0x7fa8d00697d0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:34.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.417+0000 7fa8e913b700 1 --2- 172.21.15.163:0/3791102940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8e406bcd0 0x7fa8e414a150 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:34.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.417+0000 7fa8e913b700 1 --2- 172.21.15.163:0/3791102940 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8e415df50 0x7fa8e416b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:34.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.417+0000 7fa8e913b700 1 --2- 172.21.15.163:0/3791102940 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8e406c0a0 0x7fa8e40c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:34.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.417+0000 7fa8e913b700 1 -- 172.21.15.163:0/3791102940 >> 172.21.15.163:0/3791102940 conn(0x7fa8e40ce600 msgr2=0x7fa8e4155e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:34.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.417+0000 7fa8e913b700 1 -- 172.21.15.163:0/3791102940 shutdown_connections 2023-10-09T06:38:34.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.417+0000 7fa8e913b700 1 -- 172.21.15.163:0/3791102940 wait complete. 2023-10-09T06:38:34.429 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rename cephfs volume_0000000000256160 2023-10-09T06:38:34.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.546+0000 7f170691d700 1 Processor -- start 2023-10-09T06:38:34.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.546+0000 7f170691d700 1 -- start start 2023-10-09T06:38:34.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.547+0000 7f170691d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1700069d60 0x7f17000c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:34.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.547+0000 7f170691d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f17000c81f0 0x7f17000cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:34.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.547+0000 7f170691d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f170006bdb0 con 0x7f17000aca80 2023-10-09T06:38:34.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.547+0000 7f170691d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17000c8670 con 0x7f17000c81f0 2023-10-09T06:38:34.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.547+0000 7f170691d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17000cb370 con 0x7f1700069d60 2023-10-09T06:38:34.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.547+0000 7f16ff7fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1700069d60 0x7f17000c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:34.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.547+0000 7f1704eba700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f17000c81f0 0x7f17000cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:34.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.548+0000 7f16fffff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f17000aca80 0x7f170006aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:43978/0 (socket says 172.21.15.163:43978) 2023-10-09T06:38:34.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.548+0000 7f16fffff700 1 -- 172.21.15.163:0/1487794133 learned_addr learned my addr 172.21.15.163:0/1487794133 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:34.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.548+0000 7f16ff7fe700 1 -- 172.21.15.163:0/1487794133 >> v2:172.21.15.181:3300/0 conn(0x7f17000c81f0 msgr2=0x7f17000cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:34.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.548+0000 7f16ff7fe700 1 --2- 172.21.15.163:0/1487794133 >> v2:172.21.15.181:3300/0 conn(0x7f17000c81f0 0x7f17000cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:34.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.548+0000 7f16ff7fe700 1 -- 172.21.15.163:0/1487794133 >> v1:172.21.15.181:6789/0 conn(0x7f17000aca80 legacy=0x7f170006aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:34.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.548+0000 7f16ff7fe700 1 -- 172.21.15.163:0/1487794133 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1700159f60 con 0x7f1700069d60 2023-10-09T06:38:34.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.548+0000 7f1704eba700 1 --2- 172.21.15.163:0/1487794133 >> v2:172.21.15.181:3300/0 conn(0x7f17000c81f0 0x7f17000cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:38:34.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.548+0000 7f16ff7fe700 1 --2- 172.21.15.163:0/1487794133 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1700069d60 0x7f17000c7cb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f16f40098e0 tx=0x7f16f40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f73097e4d1c9e6e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:34.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.548+0000 7f16feffd700 1 -- 172.21.15.163:0/1487794133 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f400b640 con 0x7f1700069d60 2023-10-09T06:38:34.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.548+0000 7f16feffd700 1 -- 172.21.15.163:0/1487794133 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f400b7a0 con 0x7f1700069d60 2023-10-09T06:38:34.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.548+0000 7f16feffd700 1 -- 172.21.15.163:0/1487794133 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f400bb40 con 0x7f1700069d60 2023-10-09T06:38:34.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.549+0000 7f170691d700 1 -- 172.21.15.163:0/1487794133 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1700069d60 msgr2=0x7f17000c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:34.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.549+0000 7f170691d700 1 --2- 172.21.15.163:0/1487794133 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1700069d60 0x7f17000c7cb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f16f40098e0 tx=0x7f16f40091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:34.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.549+0000 7f170691d700 1 -- 172.21.15.163:0/1487794133 shutdown_connections 2023-10-09T06:38:34.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.549+0000 7f170691d700 1 --2- 172.21.15.163:0/1487794133 >> v2:172.21.15.181:3300/0 conn(0x7f17000c81f0 0x7f17000cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:34.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.549+0000 7f170691d700 1 --2- 172.21.15.163:0/1487794133 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1700069d60 0x7f17000c7cb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:34.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.549+0000 7f170691d700 1 -- 172.21.15.163:0/1487794133 >> 172.21.15.163:0/1487794133 conn(0x7f17000cd670 msgr2=0x7f17000cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:34.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.549+0000 7f170691d700 1 -- 172.21.15.163:0/1487794133 shutdown_connections 2023-10-09T06:38:34.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.549+0000 7f170691d700 1 -- 172.21.15.163:0/1487794133 wait complete. 2023-10-09T06:38:34.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.550+0000 7f170691d700 1 Processor -- start 2023-10-09T06:38:34.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.550+0000 7f170691d700 1 -- start start 2023-10-09T06:38:34.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.551+0000 7f170691d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1700069d60 0x7f17001579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:34.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.551+0000 7f170691d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17000aca80 0x7f1700156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:34.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.551+0000 7f170691d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17000c81f0 0x7f1700156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:34.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.551+0000 7f170691d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17000adc30 con 0x7f17000aca80 2023-10-09T06:38:34.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.551+0000 7f170691d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17000b3400 con 0x7f1700069d60 2023-10-09T06:38:34.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.551+0000 7f170691d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17000ad290 con 0x7f17000c81f0 2023-10-09T06:38:34.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.551+0000 7f16ff7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17000aca80 0x7f1700156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:34.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.551+0000 7f16ff7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17000aca80 0x7f1700156420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47110/0 (socket says 172.21.15.163:47110) 2023-10-09T06:38:34.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.551+0000 7f16ff7fe700 1 -- 172.21.15.163:0/1389144051 learned_addr learned my addr 172.21.15.163:0/1389144051 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:34.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.551+0000 7f16fffff700 1 --2- 172.21.15.163:0/1389144051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1700069d60 0x7f17001579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:34.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.551+0000 7f1704eba700 1 --2- 172.21.15.163:0/1389144051 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17000c81f0 0x7f1700156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:34.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.551+0000 7f16ff7fe700 1 -- 172.21.15.163:0/1389144051 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17000c81f0 msgr2=0x7f1700156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:34.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.551+0000 7f16ff7fe700 1 --2- 172.21.15.163:0/1389144051 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17000c81f0 0x7f1700156960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:34.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.551+0000 7f16ff7fe700 1 -- 172.21.15.163:0/1389144051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1700069d60 msgr2=0x7f17001579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:34.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.551+0000 7f16ff7fe700 1 --2- 172.21.15.163:0/1389144051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1700069d60 0x7f17001579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:34.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.551+0000 7f16ff7fe700 1 -- 172.21.15.163:0/1389144051 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f8009a20 con 0x7f17000aca80 2023-10-09T06:38:34.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.551+0000 7f16ff7fe700 1 --2- 172.21.15.163:0/1389144051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17000aca80 0x7f1700156420 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f16f4013000 tx=0x7f16f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:34.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.551+0000 7f16e7fff700 1 -- 172.21.15.163:0/1389144051 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f401bcf0 con 0x7f17000aca80 2023-10-09T06:38:34.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.551+0000 7f16e7fff700 1 -- 172.21.15.163:0/1389144051 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f401be70 con 0x7f17000aca80 2023-10-09T06:38:34.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.551+0000 7f170691d700 1 -- 172.21.15.163:0/1389144051 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f4009420 con 0x7f17000aca80 2023-10-09T06:38:34.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.551+0000 7f16e7fff700 1 -- 172.21.15.163:0/1389144051 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f4011650 con 0x7f17000aca80 2023-10-09T06:38:34.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.552+0000 7f170691d700 1 -- 172.21.15.163:0/1389144051 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17000ad850 con 0x7f17000aca80 2023-10-09T06:38:34.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.552+0000 7f170691d700 1 -- 172.21.15.163:0/1389144051 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f170004c3c0 con 0x7f17000aca80 2023-10-09T06:38:34.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.553+0000 7f16e7fff700 1 -- 172.21.15.163:0/1389144051 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f16f4011a00 con 0x7f17000aca80 2023-10-09T06:38:34.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.553+0000 7f16e7fff700 1 --2- 172.21.15.163:0/1389144051 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f16ec067320 0x7f16ec0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:34.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.553+0000 7f16e7fff700 1 -- 172.21.15.163:0/1389144051 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f16f4086180 con 0x7f17000aca80 2023-10-09T06:38:34.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.553+0000 7f16fffff700 1 --2- 172.21.15.163:0/1389144051 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f16ec067320 0x7f16ec0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:34.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.554+0000 7f16fffff700 1 --2- 172.21.15.163:0/1389144051 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f16ec067320 0x7f16ec0697d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f16f0005920 tx=0x7f16f0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:34.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.557+0000 7f16e7fff700 1 -- 172.21.15.163:0/1389144051 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f16f4052ed0 con 0x7f17000aca80 2023-10-09T06:38:34.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.767+0000 7f170691d700 1 -- 172.21.15.163:0/1389144051 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume rename", "vol_name": "cephfs", "new_vol_name": "volume_0000000000256160", "target": ["mon-mgr", ""]}) v1 -- 0x7f17000636a0 con 0x7f16ec067320 2023-10-09T06:38:34.768 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:38:34.768+0000 7ff9594e4700 -1 mgr.server reply reply (1) Operation not permitted WARNING: This will rename the filesystem and possibly its pools. It is a potentially disruptive operation, clients' cephx credentials need reauthorized to access the file system and its pools with the new name. Add --yes-i-really-mean-it if you are sure you wish to continue. 2023-10-09T06:38:34.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:34.769+0000 7f16e7fff700 1 -- 172.21.15.163:0/1389144051 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: -1 WARNING: This will rename the filesystem and possibly its pools. It is a potentially disruptive operation, clients' cephx credentials need reauthorized to access the file system and its pools with the new name. Add --yes-i-really-mean-it if you are sure you wish to continue.) v1 ==== 283+0+0 (secure 0 0 0) 0x7f17000636a0 con 0x7f16ec067320 2023-10-09T06:38:34.770 INFO:teuthology.orchestra.run.smithi163.stderr:Error EPERM: WARNING: This will rename the filesystem and possibly its pools. It is a potentially disruptive operation, clients' cephx credentials need reauthorized to access the file system and its pools with the new name. Add --yes-i-really-mean-it if you are sure you wish to continue. 2023-10-09T06:38:34.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-09T06:38:34.774 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-09T06:38:34.774 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:38:34.812 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:38:34.852 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-09T06:38:34.853 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-09T06:38:34.853 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:38:34.853 DEBUG:teuthology.orchestra.run.smithi163:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-09T06:38:34.892 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi163.front.sepia.ceph.com 2023-10-09T06:38:34.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:34.892 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-09T06:38:34.946 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-09T06:38:34.946 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:38:34.947 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-09T06:38:34.988 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-09T06:38:34.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:34.989 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-09T06:38:35.046 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:38:35.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.161+0000 7f5c167df700 1 Processor -- start 2023-10-09T06:38:35.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.161+0000 7f5c167df700 1 -- start start 2023-10-09T06:38:35.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.162+0000 7f5c167df700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5c10159890 0x7f5c1015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:35.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.162+0000 7f5c167df700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5c10069560 0x7f5c100699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:35.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.162+0000 7f5c167df700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c10069ee0 con 0x7f5c10159c60 2023-10-09T06:38:35.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.162+0000 7f5c167df700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c1015a160 con 0x7f5c10069560 2023-10-09T06:38:35.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.162+0000 7f5c167df700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c100d13a0 con 0x7f5c10159890 2023-10-09T06:38:35.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.162+0000 7f5c14d7c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5c10069560 0x7f5c100699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:35.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.162+0000 7f5c0f7fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f5c10159c60 0x7f5c1015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:46016/0 (socket says 172.21.15.163:46016) 2023-10-09T06:38:35.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.162+0000 7f5c0f7fe700 1 -- 172.21.15.163:0/3657060520 learned_addr learned my addr 172.21.15.163:0/3657060520 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:35.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.162+0000 7f5c0ffff700 1 --2- 172.21.15.163:0/3657060520 >> v2:172.21.15.181:3300/0 conn(0x7f5c10159890 0x7f5c1015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:35.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.162+0000 7f5c14d7c700 1 -- 172.21.15.163:0/3657060520 >> v2:172.21.15.181:3300/0 conn(0x7f5c10159890 msgr2=0x7f5c1015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:35.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.162+0000 7f5c14d7c700 1 --2- 172.21.15.163:0/3657060520 >> v2:172.21.15.181:3300/0 conn(0x7f5c10159890 0x7f5c1015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:35.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.162+0000 7f5c14d7c700 1 -- 172.21.15.163:0/3657060520 >> v1:172.21.15.163:6789/0 conn(0x7f5c10159c60 legacy=0x7f5c1015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:35.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.162+0000 7f5c0effd700 1 -- 172.21.15.163:0/3657060520 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4075188206 0 0) 0x7f5c10069ee0 con 0x7f5c10159c60 2023-10-09T06:38:35.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.162+0000 7f5c14d7c700 1 -- 172.21.15.163:0/3657060520 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1015e140 con 0x7f5c10069560 2023-10-09T06:38:35.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.162+0000 7f5c14d7c700 1 --2- 172.21.15.163:0/3657060520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c10069560 0x7f5c100699a0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f5c100d1090 tx=0x7f5c08009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=993b807b3df4beec server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:35.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.163+0000 7f5c0effd700 1 -- 172.21.15.163:0/3657060520 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0800e070 con 0x7f5c10069560 2023-10-09T06:38:35.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.163+0000 7f5c0effd700 1 -- 172.21.15.163:0/3657060520 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0800bc30 con 0x7f5c10069560 2023-10-09T06:38:35.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.163+0000 7f5c0effd700 1 -- 172.21.15.163:0/3657060520 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c08012430 con 0x7f5c10069560 2023-10-09T06:38:35.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.163+0000 7f5c167df700 1 -- 172.21.15.163:0/3657060520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c10069560 msgr2=0x7f5c100699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:35.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.163+0000 7f5c167df700 1 --2- 172.21.15.163:0/3657060520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c10069560 0x7f5c100699a0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f5c100d1090 tx=0x7f5c08009780 comp rx=0 tx=0).stop 2023-10-09T06:38:35.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.163+0000 7f5c167df700 1 -- 172.21.15.163:0/3657060520 shutdown_connections 2023-10-09T06:38:35.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.163+0000 7f5c167df700 1 --2- 172.21.15.163:0/3657060520 >> v2:172.21.15.181:3300/0 conn(0x7f5c10159890 0x7f5c1015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:35.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.163+0000 7f5c167df700 1 --2- 172.21.15.163:0/3657060520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c10069560 0x7f5c100699a0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:35.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.163+0000 7f5c167df700 1 -- 172.21.15.163:0/3657060520 >> 172.21.15.163:0/3657060520 conn(0x7f5c10068e20 msgr2=0x7f5c100d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:35.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.163+0000 7f5c167df700 1 -- 172.21.15.163:0/3657060520 shutdown_connections 2023-10-09T06:38:35.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.163+0000 7f5c167df700 1 -- 172.21.15.163:0/3657060520 wait complete. 2023-10-09T06:38:35.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.164+0000 7f5c167df700 1 Processor -- start 2023-10-09T06:38:35.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.164+0000 7f5c167df700 1 -- start start 2023-10-09T06:38:35.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.165+0000 7f5c167df700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c10069560 0x7f5c100b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:35.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.165+0000 7f5c167df700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c10159890 0x7f5c1016d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:35.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.165+0000 7f5c167df700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c10159c60 0x7f5c100b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:35.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.165+0000 7f5c167df700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1015b600 con 0x7f5c10159c60 2023-10-09T06:38:35.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.165+0000 7f5c167df700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1016df30 con 0x7f5c10159890 2023-10-09T06:38:35.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.165+0000 7f5c167df700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c100b2030 con 0x7f5c10069560 2023-10-09T06:38:35.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.165+0000 7f5c14d7c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c10159c60 0x7f5c100b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:35.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.165+0000 7f5c14d7c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c10159c60 0x7f5c100b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47128/0 (socket says 172.21.15.163:47128) 2023-10-09T06:38:35.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.165+0000 7f5c14d7c700 1 -- 172.21.15.163:0/4273761133 learned_addr learned my addr 172.21.15.163:0/4273761133 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:35.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.165+0000 7f5c0f7fe700 1 --2- 172.21.15.163:0/4273761133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c10159890 0x7f5c1016d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:35.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.165+0000 7f5c0ffff700 1 --2- 172.21.15.163:0/4273761133 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c10069560 0x7f5c100b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:35.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.166+0000 7f5c14d7c700 1 -- 172.21.15.163:0/4273761133 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c10069560 msgr2=0x7f5c100b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:35.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.166+0000 7f5c14d7c700 1 --2- 172.21.15.163:0/4273761133 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c10069560 0x7f5c100b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:35.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.166+0000 7f5c14d7c700 1 -- 172.21.15.163:0/4273761133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c10159890 msgr2=0x7f5c1016d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:35.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.166+0000 7f5c14d7c700 1 --2- 172.21.15.163:0/4273761133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c10159890 0x7f5c1016d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:35.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.166+0000 7f5c14d7c700 1 -- 172.21.15.163:0/4273761133 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c08009a20 con 0x7f5c10159c60 2023-10-09T06:38:35.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.166+0000 7f5c14d7c700 1 --2- 172.21.15.163:0/4273761133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c10159c60 0x7f5c100b1af0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f5c080097c0 tx=0x7f5c08019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:35.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.166+0000 7f5bf7fff700 1 -- 172.21.15.163:0/4273761133 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0801e630 con 0x7f5c10159c60 2023-10-09T06:38:35.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.166+0000 7f5bf7fff700 1 -- 172.21.15.163:0/4273761133 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0801e7b0 con 0x7f5c10159c60 2023-10-09T06:38:35.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.166+0000 7f5c167df700 1 -- 172.21.15.163:0/4273761133 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1016d690 con 0x7f5c10159c60 2023-10-09T06:38:35.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.166+0000 7f5bf7fff700 1 -- 172.21.15.163:0/4273761133 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0801eb70 con 0x7f5c10159c60 2023-10-09T06:38:35.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.166+0000 7f5c167df700 1 -- 172.21.15.163:0/4273761133 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c1016dc50 con 0x7f5c10159c60 2023-10-09T06:38:35.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.167+0000 7f5c167df700 1 -- 172.21.15.163:0/4273761133 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c1015b770 con 0x7f5c10159c60 2023-10-09T06:38:35.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.167+0000 7f5bf7fff700 1 -- 172.21.15.163:0/4273761133 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5c08024070 con 0x7f5c10159c60 2023-10-09T06:38:35.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.168+0000 7f5bf7fff700 1 --2- 172.21.15.163:0/4273761133 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5bfc067320 0x7f5bfc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:35.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.168+0000 7f5bf7fff700 1 -- 172.21.15.163:0/4273761133 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5c08087510 con 0x7f5c10159c60 2023-10-09T06:38:35.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.168+0000 7f5c0ffff700 1 --2- 172.21.15.163:0/4273761133 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5bfc067320 0x7f5bfc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:35.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.168+0000 7f5c0ffff700 1 --2- 172.21.15.163:0/4273761133 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5bfc067320 0x7f5bfc0697d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5c00006fd0 tx=0x7f5c00008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:35.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.172+0000 7f5bf7fff700 1 -- 172.21.15.163:0/4273761133 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c08054250 con 0x7f5c10159c60 2023-10-09T06:38:35.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.394+0000 7f5c167df700 1 -- 172.21.15.163:0/4273761133 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c1004c3c0 con 0x7f5c10159c60 2023-10-09T06:38:35.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.395+0000 7f5bf7fff700 1 -- 172.21.15.163:0/4273761133 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4615 (secure 0 0 0) 0x7f5c08057870 con 0x7f5c10159c60 2023-10-09T06:38:35.395 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:35.395 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":104,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6371,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3797578839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3797578839},{"type":"v1","addr":"172.21.15.163:6834","nonce":3797578839}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":6383,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2371721802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2371721802},{"type":"v1","addr":"172.21.15.181:6837","nonce":2371721802}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":7228,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2068214884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2068214884},{"type":"v1","addr":"172.21.15.181:6836","nonce":2068214884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:38:20.060175+0000","modified":"2023-10-09T06:38:23.693400+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7875},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7875":{"gid":7875,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/3794912550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3794912550},{"type":"v1","addr":"172.21.15.163:6835","nonce":3794912550}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-09T06:38:35.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.397+0000 7f5c167df700 1 -- 172.21.15.163:0/4273761133 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5bfc067320 msgr2=0x7f5bfc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:35.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.397+0000 7f5c167df700 1 --2- 172.21.15.163:0/4273761133 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5bfc067320 0x7f5bfc0697d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5c00006fd0 tx=0x7f5c00008040 comp rx=0 tx=0).stop 2023-10-09T06:38:35.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.397+0000 7f5c167df700 1 -- 172.21.15.163:0/4273761133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c10159c60 msgr2=0x7f5c100b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:35.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.397+0000 7f5c167df700 1 --2- 172.21.15.163:0/4273761133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c10159c60 0x7f5c100b1af0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f5c080097c0 tx=0x7f5c08019000 comp rx=0 tx=0).stop 2023-10-09T06:38:35.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.397+0000 7f5c167df700 1 -- 172.21.15.163:0/4273761133 shutdown_connections 2023-10-09T06:38:35.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.397+0000 7f5c167df700 1 --2- 172.21.15.163:0/4273761133 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5bfc067320 0x7f5bfc0697d0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:35.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.397+0000 7f5c167df700 1 --2- 172.21.15.163:0/4273761133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c10159c60 0x7f5c100b1af0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:35.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.397+0000 7f5c167df700 1 --2- 172.21.15.163:0/4273761133 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c10069560 0x7f5c100b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:35.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.397+0000 7f5c167df700 1 --2- 172.21.15.163:0/4273761133 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c10159890 0x7f5c1016d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:35.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.397+0000 7f5c167df700 1 -- 172.21.15.163:0/4273761133 >> 172.21.15.163:0/4273761133 conn(0x7f5c10068e20 msgr2=0x7f5c10070530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:35.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.397+0000 7f5c167df700 1 -- 172.21.15.163:0/4273761133 shutdown_connections 2023-10-09T06:38:35.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.397+0000 7f5c167df700 1 -- 172.21.15.163:0/4273761133 wait complete. 2023-10-09T06:38:35.399 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 104 2023-10-09T06:38:35.409 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:38:35.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.524+0000 7f7f20ba0700 1 Processor -- start 2023-10-09T06:38:35.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.524+0000 7f7f20ba0700 1 -- start start 2023-10-09T06:38:35.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.525+0000 7f7f20ba0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7f1c06bcd0 0x7f7f1c140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:35.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.525+0000 7f7f20ba0700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7f1c0c9510 0x7f7f1c141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:35.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.525+0000 7f7f20ba0700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f1c15c2a0 con 0x7f7f1c06c0a0 2023-10-09T06:38:35.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.525+0000 7f7f20ba0700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f1c0c9900 con 0x7f7f1c0c9510 2023-10-09T06:38:35.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.525+0000 7f7f20ba0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f1c15b690 con 0x7f7f1c06bcd0 2023-10-09T06:38:35.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.525+0000 7f7f1ad9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7f1c0c9510 0x7f7f1c141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:35.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.525+0000 7f7f11d9b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f7f1c06c0a0 0x7f7f1c0c8e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:46020/0 (socket says 172.21.15.163:46020) 2023-10-09T06:38:35.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.525+0000 7f7f11d9b700 1 -- 172.21.15.163:0/4100608717 learned_addr learned my addr 172.21.15.163:0/4100608717 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:35.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.526+0000 7f7f1a59c700 1 --2- 172.21.15.163:0/4100608717 >> v2:172.21.15.181:3300/0 conn(0x7f7f1c06bcd0 0x7f7f1c140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:35.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.526+0000 7f7f1ad9d700 1 -- 172.21.15.163:0/4100608717 >> v2:172.21.15.181:3300/0 conn(0x7f7f1c06bcd0 msgr2=0x7f7f1c140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:35.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.526+0000 7f7f1ad9d700 1 --2- 172.21.15.163:0/4100608717 >> v2:172.21.15.181:3300/0 conn(0x7f7f1c06bcd0 0x7f7f1c140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:35.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.526+0000 7f7f1ad9d700 1 -- 172.21.15.163:0/4100608717 >> v1:172.21.15.163:6789/0 conn(0x7f7f1c06c0a0 legacy=0x7f7f1c0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:35.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.526+0000 7f7f19d9b700 1 -- 172.21.15.163:0/4100608717 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 11028301 0 0) 0x7f7f1c15c2a0 con 0x7f7f1c06c0a0 2023-10-09T06:38:35.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.526+0000 7f7f1ad9d700 1 -- 172.21.15.163:0/4100608717 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1c15b970 con 0x7f7f1c0c9510 2023-10-09T06:38:35.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.526+0000 7f7f1ad9d700 1 --2- 172.21.15.163:0/4100608717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f1c0c9510 0x7f7f1c141330 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c0cafd0 tx=0x7f7f14009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3229a2c54e54d458 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:35.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.526+0000 7f7f19d9b700 1 -- 172.21.15.163:0/4100608717 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1400e070 con 0x7f7f1c0c9510 2023-10-09T06:38:35.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.526+0000 7f7f19d9b700 1 -- 172.21.15.163:0/4100608717 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1400bc30 con 0x7f7f1c0c9510 2023-10-09T06:38:35.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.526+0000 7f7f19d9b700 1 -- 172.21.15.163:0/4100608717 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f14012430 con 0x7f7f1c0c9510 2023-10-09T06:38:35.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.526+0000 7f7f20ba0700 1 -- 172.21.15.163:0/4100608717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f1c0c9510 msgr2=0x7f7f1c141330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:35.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.526+0000 7f7f20ba0700 1 --2- 172.21.15.163:0/4100608717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f1c0c9510 0x7f7f1c141330 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c0cafd0 tx=0x7f7f14009780 comp rx=0 tx=0).stop 2023-10-09T06:38:35.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.526+0000 7f7f20ba0700 1 -- 172.21.15.163:0/4100608717 shutdown_connections 2023-10-09T06:38:35.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.526+0000 7f7f20ba0700 1 --2- 172.21.15.163:0/4100608717 >> v2:172.21.15.181:3300/0 conn(0x7f7f1c06bcd0 0x7f7f1c140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:35.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.526+0000 7f7f20ba0700 1 --2- 172.21.15.163:0/4100608717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f1c0c9510 0x7f7f1c141330 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:35.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.526+0000 7f7f20ba0700 1 -- 172.21.15.163:0/4100608717 >> 172.21.15.163:0/4100608717 conn(0x7f7f1c0ce600 msgr2=0x7f7f1c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:35.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.527+0000 7f7f20ba0700 1 -- 172.21.15.163:0/4100608717 shutdown_connections 2023-10-09T06:38:35.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.527+0000 7f7f20ba0700 1 -- 172.21.15.163:0/4100608717 wait complete. 2023-10-09T06:38:35.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.528+0000 7f7f20ba0700 1 Processor -- start 2023-10-09T06:38:35.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.528+0000 7f7f20ba0700 1 -- start start 2023-10-09T06:38:35.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.528+0000 7f7f20ba0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f1c06bcd0 0x7f7f1c13f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:35.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.529+0000 7f7f20ba0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f1c06c0a0 0x7f7f1c0b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:35.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.529+0000 7f7f20ba0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f1c0c9510 0x7f7f1c0c47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:35.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.529+0000 7f7f20ba0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c0b3380 con 0x7f7f1c06bcd0 2023-10-09T06:38:35.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.529+0000 7f7f20ba0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c0b3660 con 0x7f7f1c0c9510 2023-10-09T06:38:35.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.529+0000 7f7f20ba0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c0b3940 con 0x7f7f1c06c0a0 2023-10-09T06:38:35.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.529+0000 7f7f1a59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f1c06bcd0 0x7f7f1c13f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:35.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.529+0000 7f7f1a59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f1c06bcd0 0x7f7f1c13f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47138/0 (socket says 172.21.15.163:47138) 2023-10-09T06:38:35.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.529+0000 7f7f1a59c700 1 -- 172.21.15.163:0/1631452460 learned_addr learned my addr 172.21.15.163:0/1631452460 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:35.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.529+0000 7f7f1ad9d700 1 --2- 172.21.15.163:0/1631452460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f1c0c9510 0x7f7f1c0c47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:35.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.529+0000 7f7f11d9b700 1 --2- 172.21.15.163:0/1631452460 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f1c06c0a0 0x7f7f1c0b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:35.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.529+0000 7f7f1a59c700 1 -- 172.21.15.163:0/1631452460 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f1c06c0a0 msgr2=0x7f7f1c0b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:35.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.529+0000 7f7f1a59c700 1 --2- 172.21.15.163:0/1631452460 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f1c06c0a0 0x7f7f1c0b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:35.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.529+0000 7f7f1a59c700 1 -- 172.21.15.163:0/1631452460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f1c0c9510 msgr2=0x7f7f1c0c47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:35.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.529+0000 7f7f1a59c700 1 --2- 172.21.15.163:0/1631452460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f1c0c9510 0x7f7f1c0c47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:35.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.529+0000 7f7f1a59c700 1 -- 172.21.15.163:0/1631452460 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f14009a20 con 0x7f7f1c06bcd0 2023-10-09T06:38:35.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.529+0000 7f7f1ad9d700 1 --2- 172.21.15.163:0/1631452460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f1c0c9510 0x7f7f1c0c47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:35.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.529+0000 7f7f1a59c700 1 --2- 172.21.15.163:0/1631452460 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f1c06bcd0 0x7f7f1c13f3c0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00d890 tx=0x7f7f0c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:35.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.529+0000 7f7f12ffd700 1 -- 172.21.15.163:0/1631452460 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c0125f0 con 0x7f7f1c06bcd0 2023-10-09T06:38:35.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.529+0000 7f7f12ffd700 1 -- 172.21.15.163:0/1631452460 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0c012750 con 0x7f7f1c06bcd0 2023-10-09T06:38:35.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.529+0000 7f7f20ba0700 1 -- 172.21.15.163:0/1631452460 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f1c0b3c20 con 0x7f7f1c06bcd0 2023-10-09T06:38:35.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.529+0000 7f7f12ffd700 1 -- 172.21.15.163:0/1631452460 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c012ad0 con 0x7f7f1c06bcd0 2023-10-09T06:38:35.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.529+0000 7f7f20ba0700 1 -- 172.21.15.163:0/1631452460 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f1c167cf0 con 0x7f7f1c06bcd0 2023-10-09T06:38:35.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.530+0000 7f7f20ba0700 1 -- 172.21.15.163:0/1631452460 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f1c0b40c0 con 0x7f7f1c06bcd0 2023-10-09T06:38:35.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.531+0000 7f7f12ffd700 1 -- 172.21.15.163:0/1631452460 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7f0c01e460 con 0x7f7f1c06bcd0 2023-10-09T06:38:35.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.531+0000 7f7f12ffd700 1 --2- 172.21.15.163:0/1631452460 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7efc067320 0x7f7efc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:35.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.531+0000 7f7f12ffd700 1 -- 172.21.15.163:0/1631452460 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7f0c085290 con 0x7f7f1c06bcd0 2023-10-09T06:38:35.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.531+0000 7f7f11d9b700 1 --2- 172.21.15.163:0/1631452460 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7efc067320 0x7f7efc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:35.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.532+0000 7f7f11d9b700 1 --2- 172.21.15.163:0/1631452460 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7efc067320 0x7f7efc0697d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c15ae80 tx=0x7f7f04007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:35.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.535+0000 7f7f12ffd700 1 -- 172.21.15.163:0/1631452460 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f0c051f20 con 0x7f7f1c06bcd0 2023-10-09T06:38:35.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.720+0000 7f7f20ba0700 1 -- 172.21.15.163:0/1631452460 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f1c04c3c0 con 0x7f7f1c06bcd0 2023-10-09T06:38:35.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.721+0000 7f7f12ffd700 1 -- 172.21.15.163:0/1631452460 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) v1 ==== 74+0+15412 (secure 0 0 0) 0x7f7f0c055540 con 0x7f7f1c06bcd0 2023-10-09T06:38:35.722 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:35.723 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":152,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:38:21.683060+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:38:18.381850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-09T06:38:19.022637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:38:35.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.723+0000 7f7f20ba0700 1 -- 172.21.15.163:0/1631452460 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7efc067320 msgr2=0x7f7efc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:35.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.723+0000 7f7f20ba0700 1 --2- 172.21.15.163:0/1631452460 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7efc067320 0x7f7efc0697d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c15ae80 tx=0x7f7f04007d30 comp rx=0 tx=0).stop 2023-10-09T06:38:35.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.723+0000 7f7f20ba0700 1 -- 172.21.15.163:0/1631452460 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f1c06bcd0 msgr2=0x7f7f1c13f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:35.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.723+0000 7f7f20ba0700 1 --2- 172.21.15.163:0/1631452460 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f1c06bcd0 0x7f7f1c13f3c0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00d890 tx=0x7f7f0c008040 comp rx=0 tx=0).stop 2023-10-09T06:38:35.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.723+0000 7f7f20ba0700 1 -- 172.21.15.163:0/1631452460 shutdown_connections 2023-10-09T06:38:35.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.723+0000 7f7f20ba0700 1 --2- 172.21.15.163:0/1631452460 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7efc067320 0x7f7efc0697d0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:35.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.723+0000 7f7f20ba0700 1 --2- 172.21.15.163:0/1631452460 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f1c06bcd0 0x7f7f1c13f3c0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:35.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.723+0000 7f7f20ba0700 1 --2- 172.21.15.163:0/1631452460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f1c0c9510 0x7f7f1c0c47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:35.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.723+0000 7f7f20ba0700 1 --2- 172.21.15.163:0/1631452460 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f1c06c0a0 0x7f7f1c0b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:35.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.723+0000 7f7f20ba0700 1 -- 172.21.15.163:0/1631452460 >> 172.21.15.163:0/1631452460 conn(0x7f7f1c0ce600 msgr2=0x7f7f1c0c3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:35.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.723+0000 7f7f20ba0700 1 -- 172.21.15.163:0/1631452460 shutdown_connections 2023-10-09T06:38:35.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.723+0000 7f7f20ba0700 1 -- 172.21.15.163:0/1631452460 wait complete. 2023-10-09T06:38:35.736 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-09T06:38:18.381850+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '151', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2023-10-09T06:38:19.022637+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '152', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-09T06:38:35.737 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:38:35.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.853+0000 7f45bc849700 1 Processor -- start 2023-10-09T06:38:35.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.853+0000 7f45bc849700 1 -- start start 2023-10-09T06:38:35.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.854+0000 7f45bc849700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f45b8159c60 0x7f45b815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:35.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.854+0000 7f45bc849700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45b815db10 0x7f45b8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:35.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.854+0000 7f45bc849700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45b815a160 con 0x7f45b8159890 2023-10-09T06:38:35.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.854+0000 7f45bc849700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b815df50 con 0x7f45b8159c60 2023-10-09T06:38:35.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.854+0000 7f45bc849700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b80d13a0 con 0x7f45b815db10 2023-10-09T06:38:35.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.854+0000 7f45add9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f45b8159c60 0x7f45b815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:35.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.854+0000 7f45b6d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45b815db10 0x7f45b8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:35.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.854+0000 7f45b659c700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f45b8159890 0x7f45b815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:43980/0 (socket says 172.21.15.163:43980) 2023-10-09T06:38:35.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.855+0000 7f45b659c700 1 -- 172.21.15.163:0/1477519348 learned_addr learned my addr 172.21.15.163:0/1477519348 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:35.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.855+0000 7f45add9b700 1 -- 172.21.15.163:0/1477519348 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45b815db10 msgr2=0x7f45b8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:35.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.855+0000 7f45add9b700 1 --2- 172.21.15.163:0/1477519348 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45b815db10 0x7f45b8069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:35.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.855+0000 7f45add9b700 1 -- 172.21.15.163:0/1477519348 >> v1:172.21.15.181:6789/0 conn(0x7f45b8159890 legacy=0x7f45b815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:35.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.855+0000 7f45b5d9b700 1 -- 172.21.15.163:0/1477519348 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1649456146 0 0) 0x7f45b815a160 con 0x7f45b8159890 2023-10-09T06:38:35.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.855+0000 7f45add9b700 1 -- 172.21.15.163:0/1477519348 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b815e140 con 0x7f45b8159c60 2023-10-09T06:38:35.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.855+0000 7f45b6d9d700 1 --2- 172.21.15.163:0/1477519348 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45b815db10 0x7f45b8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:38:35.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.855+0000 7f45add9b700 1 --2- 172.21.15.163:0/1477519348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45b8159c60 0x7f45b815d5d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f45a00098e0 tx=0x7f45a00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34f37952657702f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:35.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.855+0000 7f45b5d9b700 1 -- 172.21.15.163:0/1477519348 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a000b640 con 0x7f45b8159c60 2023-10-09T06:38:35.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.855+0000 7f45b5d9b700 1 -- 172.21.15.163:0/1477519348 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a000b7a0 con 0x7f45b8159c60 2023-10-09T06:38:35.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.855+0000 7f45b5d9b700 1 -- 172.21.15.163:0/1477519348 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a000bb40 con 0x7f45b8159c60 2023-10-09T06:38:35.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.856+0000 7f45bc849700 1 -- 172.21.15.163:0/1477519348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45b8159c60 msgr2=0x7f45b815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:35.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.856+0000 7f45bc849700 1 --2- 172.21.15.163:0/1477519348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45b8159c60 0x7f45b815d5d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f45a00098e0 tx=0x7f45a00091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:35.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.856+0000 7f45bc849700 1 -- 172.21.15.163:0/1477519348 shutdown_connections 2023-10-09T06:38:35.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.856+0000 7f45bc849700 1 --2- 172.21.15.163:0/1477519348 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45b815db10 0x7f45b8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:35.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.856+0000 7f45bc849700 1 --2- 172.21.15.163:0/1477519348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45b8159c60 0x7f45b815d5d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:35.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.856+0000 7f45bc849700 1 -- 172.21.15.163:0/1477519348 >> 172.21.15.163:0/1477519348 conn(0x7f45b8068e20 msgr2=0x7f45b80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:35.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.856+0000 7f45bc849700 1 -- 172.21.15.163:0/1477519348 shutdown_connections 2023-10-09T06:38:35.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.856+0000 7f45bc849700 1 -- 172.21.15.163:0/1477519348 wait complete. 2023-10-09T06:38:35.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.857+0000 7f45bc849700 1 Processor -- start 2023-10-09T06:38:35.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.857+0000 7f45bc849700 1 -- start start 2023-10-09T06:38:35.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.858+0000 7f45bc849700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45b8159890 0x7f45b816da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:35.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.858+0000 7f45bc849700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45b8159c60 0x7f45b8151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:35.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.858+0000 7f45bc849700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45b815db10 0x7f45b814a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:35.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.858+0000 7f45bc849700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b80b1150 con 0x7f45b815db10 2023-10-09T06:38:35.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.858+0000 7f45bc849700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b81521e0 con 0x7f45b8159c60 2023-10-09T06:38:35.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.858+0000 7f45bc849700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b80b0540 con 0x7f45b8159890 2023-10-09T06:38:35.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.858+0000 7f45b6d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45b815db10 0x7f45b814a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:35.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.858+0000 7f45b6d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45b815db10 0x7f45b814a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47150/0 (socket says 172.21.15.163:47150) 2023-10-09T06:38:35.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.858+0000 7f45b6d9d700 1 -- 172.21.15.163:0/1038791062 learned_addr learned my addr 172.21.15.163:0/1038791062 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:35.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.858+0000 7f45b659c700 1 --2- 172.21.15.163:0/1038791062 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45b8159890 0x7f45b816da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:35.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.858+0000 7f45add9b700 1 --2- 172.21.15.163:0/1038791062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45b8159c60 0x7f45b8151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:35.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.858+0000 7f45b6d9d700 1 -- 172.21.15.163:0/1038791062 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45b8159890 msgr2=0x7f45b816da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:35.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.858+0000 7f45b6d9d700 1 --2- 172.21.15.163:0/1038791062 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45b8159890 0x7f45b816da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:35.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.858+0000 7f45b6d9d700 1 -- 172.21.15.163:0/1038791062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45b8159c60 msgr2=0x7f45b8151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:35.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.858+0000 7f45b6d9d700 1 --2- 172.21.15.163:0/1038791062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45b8159c60 0x7f45b8151ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:35.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.858+0000 7f45b6d9d700 1 -- 172.21.15.163:0/1038791062 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b0009a20 con 0x7f45b815db10 2023-10-09T06:38:35.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.859+0000 7f45b6d9d700 1 --2- 172.21.15.163:0/1038791062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45b815db10 0x7f45b814a5e0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009f20 tx=0x7f45b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:35.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.859+0000 7f45aeffd700 1 -- 172.21.15.163:0/1038791062 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b000cd10 con 0x7f45b815db10 2023-10-09T06:38:35.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.859+0000 7f45aeffd700 1 -- 172.21.15.163:0/1038791062 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b000f040 con 0x7f45b815db10 2023-10-09T06:38:35.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.859+0000 7f45bc849700 1 -- 172.21.15.163:0/1038791062 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a0009420 con 0x7f45b815db10 2023-10-09T06:38:35.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.859+0000 7f45aeffd700 1 -- 172.21.15.163:0/1038791062 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b00146d0 con 0x7f45b815db10 2023-10-09T06:38:35.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.859+0000 7f45bc849700 1 -- 172.21.15.163:0/1038791062 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b80b0b00 con 0x7f45b815db10 2023-10-09T06:38:35.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.860+0000 7f45bc849700 1 -- 172.21.15.163:0/1038791062 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b80b12c0 con 0x7f45b815db10 2023-10-09T06:38:35.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.860+0000 7f45aeffd700 1 -- 172.21.15.163:0/1038791062 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f45b0003020 con 0x7f45b815db10 2023-10-09T06:38:35.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.860+0000 7f45aeffd700 1 --2- 172.21.15.163:0/1038791062 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4598067320 0x7f45980697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:35.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.861+0000 7f45aeffd700 1 -- 172.21.15.163:0/1038791062 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f45b00857f0 con 0x7f45b815db10 2023-10-09T06:38:35.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.861+0000 7f45b659c700 1 --2- 172.21.15.163:0/1038791062 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4598067320 0x7f45980697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:35.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.861+0000 7f45b659c700 1 --2- 172.21.15.163:0/1038791062 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4598067320 0x7f45980697d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f45a80011e0 tx=0x7f45a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:35.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:35.865+0000 7f45aeffd700 1 -- 172.21.15.163:0/1038791062 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f45b0052370 con 0x7f45b815db10 2023-10-09T06:38:36.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.045+0000 7f45bc849700 1 -- 172.21.15.163:0/1038791062 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f45b804c3c0 con 0x7f45b815db10 2023-10-09T06:38:36.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.046+0000 7f45aeffd700 1 -- 172.21.15.163:0/1038791062 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v152) v1 ==== 93+0+10 (secure 0 0 0) 0x7f45b0055990 con 0x7f45b815db10 2023-10-09T06:38:36.263 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:38:36.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.048+0000 7f45bc849700 1 -- 172.21.15.163:0/1038791062 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4598067320 msgr2=0x7f45980697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:36.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.048+0000 7f45bc849700 1 --2- 172.21.15.163:0/1038791062 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4598067320 0x7f45980697d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f45a80011e0 tx=0x7f45a8009040 comp rx=0 tx=0).stop 2023-10-09T06:38:36.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.048+0000 7f45bc849700 1 -- 172.21.15.163:0/1038791062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45b815db10 msgr2=0x7f45b814a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:36.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.048+0000 7f45bc849700 1 --2- 172.21.15.163:0/1038791062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45b815db10 0x7f45b814a5e0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009f20 tx=0x7f45b000b040 comp rx=0 tx=0).stop 2023-10-09T06:38:36.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.048+0000 7f45bc849700 1 -- 172.21.15.163:0/1038791062 shutdown_connections 2023-10-09T06:38:36.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.048+0000 7f45bc849700 1 --2- 172.21.15.163:0/1038791062 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4598067320 0x7f45980697d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:36.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.048+0000 7f45bc849700 1 --2- 172.21.15.163:0/1038791062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45b815db10 0x7f45b814a5e0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:36.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.048+0000 7f45bc849700 1 --2- 172.21.15.163:0/1038791062 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45b8159890 0x7f45b816da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:36.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.048+0000 7f45bc849700 1 --2- 172.21.15.163:0/1038791062 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45b8159c60 0x7f45b8151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:36.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.048+0000 7f45bc849700 1 -- 172.21.15.163:0/1038791062 >> 172.21.15.163:0/1038791062 conn(0x7f45b8068e20 msgr2=0x7f45b80cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:36.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.048+0000 7f45bc849700 1 -- 172.21.15.163:0/1038791062 shutdown_connections 2023-10-09T06:38:36.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.048+0000 7f45bc849700 1 -- 172.21.15.163:0/1038791062 wait complete. 2023-10-09T06:38:36.266 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-09T06:38:36.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.386+0000 7fb521fcc700 1 Processor -- start 2023-10-09T06:38:36.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.386+0000 7fb521fcc700 1 -- start start 2023-10-09T06:38:36.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.387+0000 7fb521fcc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb51c0b4e70 0x7fb51c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:36.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.387+0000 7fb521fcc700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb51c0cfb00 con 0x7fb51c0b1b40 2023-10-09T06:38:36.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.387+0000 7fb521fcc700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb51c15e060 con 0x7fb51c0b16d0 2023-10-09T06:38:36.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.387+0000 7fb521fcc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c0b5c40 con 0x7fb51c0b4e70 2023-10-09T06:38:36.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.387+0000 7fb5217cb700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb51c0b1b40 0x7fb51c15d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:46036/0 (socket says 172.21.15.163:46036) 2023-10-09T06:38:36.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.387+0000 7fb5217cb700 1 -- 172.21.15.163:0/3961376712 learned_addr learned my addr 172.21.15.163:0/3961376712 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:36.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.387+0000 7fb51b7fe700 1 -- 172.21.15.163:0/3961376712 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1464430981 0 0) 0x7fb51c0cfb00 con 0x7fb51c0b1b40 2023-10-09T06:38:36.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.387+0000 7fb51b7fe700 1 -- 172.21.15.163:0/3961376712 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb504003570 con 0x7fb51c0b1b40 2023-10-09T06:38:36.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.387+0000 7fb520fca700 1 --2- 172.21.15.163:0/3961376712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb51c0b4e70 0x7fb51c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:36.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.387+0000 7fb51b7fe700 1 -- 172.21.15.163:0/3961376712 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2901503894 0 0) 0x7fb504003570 con 0x7fb51c0b1b40 2023-10-09T06:38:36.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.387+0000 7fb51b7fe700 1 -- 172.21.15.163:0/3961376712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb51c0b4e70 msgr2=0x7fb51c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:36.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.387+0000 7fb51b7fe700 1 --2- 172.21.15.163:0/3961376712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb51c0b4e70 0x7fb51c0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:36.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.387+0000 7fb51b7fe700 1 -- 172.21.15.163:0/3961376712 >> v1:172.21.15.181:6789/0 conn(0x7fb51c0b16d0 legacy=0x7fb51c15d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:36.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.387+0000 7fb51b7fe700 1 -- 172.21.15.163:0/3961376712 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51c0b5f20 con 0x7fb51c0b1b40 2023-10-09T06:38:36.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.387+0000 7fb51b7fe700 1 -- 172.21.15.163:0/3961376712 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fb514001470 con 0x7fb51c0b1b40 2023-10-09T06:38:36.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.387+0000 7fb51b7fe700 1 -- 172.21.15.163:0/3961376712 >> v1:172.21.15.163:6789/0 conn(0x7fb51c0b1b40 legacy=0x7fb51c15d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:36.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.387+0000 7fb51b7fe700 1 --2- 172.21.15.163:0/3961376712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb51c0b4e70 0x7fb504003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:36.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.388+0000 7fb51b7fe700 1 --2- 172.21.15.163:0/3961376712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb504004010 0x7fb504004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:36.400 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.388+0000 7fb51b7fe700 1 --2- 172.21.15.163:0/3961376712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb504004d50 0x7fb5040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:36.400 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.388+0000 7fb51b7fe700 1 -- 172.21.15.163:0/3961376712 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c0b5c40 con 0x7fb504004d50 2023-10-09T06:38:36.400 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.388+0000 7fb51b7fe700 1 -- 172.21.15.163:0/3961376712 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50400bbf0 con 0x7fb504004010 2023-10-09T06:38:36.400 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.388+0000 7fb51b7fe700 1 -- 172.21.15.163:0/3961376712 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50400bdb0 con 0x7fb51c0b4e70 2023-10-09T06:38:36.400 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.388+0000 7fb5217cb700 1 --2- 172.21.15.163:0/3961376712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb504004d50 0x7fb5040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:36.400 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.388+0000 7fb520fca700 1 --2- 172.21.15.163:0/3961376712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb51c0b4e70 0x7fb504003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:36.400 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.388+0000 7fb51bfff700 1 --2- 172.21.15.163:0/3961376712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb504004010 0x7fb504004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:36.400 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.388+0000 7fb5217cb700 1 -- 172.21.15.163:0/3961376712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb51c0b4e70 msgr2=0x7fb504003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:36.400 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.388+0000 7fb5217cb700 1 --2- 172.21.15.163:0/3961376712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb51c0b4e70 0x7fb504003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:36.400 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.388+0000 7fb5217cb700 1 -- 172.21.15.163:0/3961376712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb504004010 msgr2=0x7fb504004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:36.401 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.388+0000 7fb5217cb700 1 --2- 172.21.15.163:0/3961376712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb504004010 0x7fb504004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:36.401 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.388+0000 7fb5217cb700 1 -- 172.21.15.163:0/3961376712 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb504003570 con 0x7fb504004d50 2023-10-09T06:38:36.401 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.388+0000 7fb5217cb700 1 --2- 172.21.15.163:0/3961376712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb504004d50 0x7fb5040051c0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fb51400c390 tx=0x7fb514001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec91e1218862fcfb server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:36.401 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.388+0000 7fb51b7fe700 1 -- 172.21.15.163:0/3961376712 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51400cc60 con 0x7fb504004d50 2023-10-09T06:38:36.401 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.388+0000 7fb51b7fe700 1 -- 172.21.15.163:0/3961376712 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51400cdc0 con 0x7fb504004d50 2023-10-09T06:38:36.401 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.388+0000 7fb51b7fe700 1 -- 172.21.15.163:0/3961376712 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb514005b70 con 0x7fb504004d50 2023-10-09T06:38:36.401 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.389+0000 7fb521fcc700 1 -- 172.21.15.163:0/3961376712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb504004d50 msgr2=0x7fb5040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:36.401 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.389+0000 7fb521fcc700 1 --2- 172.21.15.163:0/3961376712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb504004d50 0x7fb5040051c0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fb51400c390 tx=0x7fb514001e50 comp rx=0 tx=0).stop 2023-10-09T06:38:36.401 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.389+0000 7fb5217cb700 1 -- 172.21.15.163:0/3961376712 reap_dead start 2023-10-09T06:38:36.401 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.389+0000 7fb521fcc700 1 -- 172.21.15.163:0/3961376712 shutdown_connections 2023-10-09T06:38:36.401 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.389+0000 7fb521fcc700 1 -- 172.21.15.163:0/3961376712 >> 172.21.15.163:0/3961376712 conn(0x7fb51c0d10c0 msgr2=0x7fb51c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:36.401 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.389+0000 7fb521fcc700 1 -- 172.21.15.163:0/3961376712 shutdown_connections 2023-10-09T06:38:36.402 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.389+0000 7fb521fcc700 1 -- 172.21.15.163:0/3961376712 wait complete. 2023-10-09T06:38:36.402 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.390+0000 7fb521fcc700 1 Processor -- start 2023-10-09T06:38:36.402 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.390+0000 7fb521fcc700 1 -- start start 2023-10-09T06:38:36.402 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.391+0000 7fb521fcc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb51c15b620 0x7fb51c15cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:36.402 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.391+0000 7fb521fcc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb51c0cad10 0x7fb51c15b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:36.402 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.391+0000 7fb521fcc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb51c0cb180 0x7fb51c15bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:36.402 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.391+0000 7fb521fcc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c1479c0 con 0x7fb51c15b620 2023-10-09T06:38:36.402 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.391+0000 7fb521fcc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c15c470 con 0x7fb51c0cad10 2023-10-09T06:38:36.402 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.391+0000 7fb521fcc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c0cb6e0 con 0x7fb51c0cb180 2023-10-09T06:38:36.402 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.391+0000 7fb520fca700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb51c15b620 0x7fb51c15cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:36.403 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.391+0000 7fb520fca700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb51c15b620 0x7fb51c15cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47168/0 (socket says 172.21.15.163:47168) 2023-10-09T06:38:36.403 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.391+0000 7fb520fca700 1 -- 172.21.15.163:0/3306792792 learned_addr learned my addr 172.21.15.163:0/3306792792 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:36.403 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.391+0000 7fb51bfff700 1 --2- 172.21.15.163:0/3306792792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb51c0cad10 0x7fb51c15b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:36.403 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.391+0000 7fb5217cb700 1 --2- 172.21.15.163:0/3306792792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb51c0cb180 0x7fb51c15bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:36.403 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.391+0000 7fb520fca700 1 -- 172.21.15.163:0/3306792792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb51c0cb180 msgr2=0x7fb51c15bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:36.403 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.391+0000 7fb520fca700 1 --2- 172.21.15.163:0/3306792792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb51c0cb180 0x7fb51c15bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:36.403 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.391+0000 7fb520fca700 1 -- 172.21.15.163:0/3306792792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb51c0cad10 msgr2=0x7fb51c15b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:36.403 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.391+0000 7fb520fca700 1 --2- 172.21.15.163:0/3306792792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb51c0cad10 0x7fb51c15b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:36.403 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.391+0000 7fb520fca700 1 -- 172.21.15.163:0/3306792792 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51c0cb9c0 con 0x7fb51c15b620 2023-10-09T06:38:36.403 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.391+0000 7fb5217cb700 1 --2- 172.21.15.163:0/3306792792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb51c0cb180 0x7fb51c15bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:36.403 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.391+0000 7fb520fca700 1 --2- 172.21.15.163:0/3306792792 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb51c15b620 0x7fb51c15cb00 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fb50c006ac0 tx=0x7fb50c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:36.404 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.391+0000 7fb518ff9700 1 -- 172.21.15.163:0/3306792792 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c00f940 con 0x7fb51c15b620 2023-10-09T06:38:36.404 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.391+0000 7fb518ff9700 1 -- 172.21.15.163:0/3306792792 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50c00faa0 con 0x7fb51c15b620 2023-10-09T06:38:36.404 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.391+0000 7fb521fcc700 1 -- 172.21.15.163:0/3306792792 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51c0cbca0 con 0x7fb51c15b620 2023-10-09T06:38:36.404 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.391+0000 7fb518ff9700 1 -- 172.21.15.163:0/3306792792 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c00fe20 con 0x7fb51c15b620 2023-10-09T06:38:36.404 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.391+0000 7fb521fcc700 1 -- 172.21.15.163:0/3306792792 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb51c147e50 con 0x7fb51c15b620 2023-10-09T06:38:36.404 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.392+0000 7fb521fcc700 1 -- 172.21.15.163:0/3306792792 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb51c147ca0 con 0x7fb51c15b620 2023-10-09T06:38:36.404 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.393+0000 7fb518ff9700 1 -- 172.21.15.163:0/3306792792 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb50c027480 con 0x7fb51c15b620 2023-10-09T06:38:36.404 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.393+0000 7fb518ff9700 1 --2- 172.21.15.163:0/3306792792 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb5080673a0 0x7fb508069850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:36.404 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.394+0000 7fb518ff9700 1 -- 172.21.15.163:0/3306792792 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb50c018030 con 0x7fb51c15b620 2023-10-09T06:38:36.404 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.394+0000 7fb51bfff700 1 --2- 172.21.15.163:0/3306792792 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb5080673a0 0x7fb508069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:36.404 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.394+0000 7fb51bfff700 1 --2- 172.21.15.163:0/3306792792 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb5080673a0 0x7fb508069850 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fb51c16d040 tx=0x7fb510001640 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:36.405 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.397+0000 7fb518ff9700 1 -- 172.21.15.163:0/3306792792 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb50c052810 con 0x7fb51c15b620 2023-10-09T06:38:36.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.579+0000 7fb521fcc700 1 -- 172.21.15.163:0/3306792792 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb51c04bb80 con 0x7fb51c15b620 2023-10-09T06:38:36.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.580+0000 7fb518ff9700 1 -- 172.21.15.163:0/3306792792 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v152) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb50c0220a0 con 0x7fb51c15b620 2023-10-09T06:38:36.581 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 32 2023-10-09T06:38:36.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.582+0000 7fb521fcc700 1 -- 172.21.15.163:0/3306792792 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb5080673a0 msgr2=0x7fb508069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:36.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.582+0000 7fb521fcc700 1 --2- 172.21.15.163:0/3306792792 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb5080673a0 0x7fb508069850 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fb51c16d040 tx=0x7fb510001640 comp rx=0 tx=0).stop 2023-10-09T06:38:36.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.582+0000 7fb521fcc700 1 -- 172.21.15.163:0/3306792792 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb51c15b620 msgr2=0x7fb51c15cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:36.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.582+0000 7fb521fcc700 1 --2- 172.21.15.163:0/3306792792 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb51c15b620 0x7fb51c15cb00 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fb50c006ac0 tx=0x7fb50c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:36.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.582+0000 7fb521fcc700 1 -- 172.21.15.163:0/3306792792 shutdown_connections 2023-10-09T06:38:36.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.582+0000 7fb521fcc700 1 --2- 172.21.15.163:0/3306792792 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb5080673a0 0x7fb508069850 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:36.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.582+0000 7fb521fcc700 1 --2- 172.21.15.163:0/3306792792 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb51c15b620 0x7fb51c15cb00 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:36.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.582+0000 7fb521fcc700 1 --2- 172.21.15.163:0/3306792792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb51c0cb180 0x7fb51c15bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:36.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.582+0000 7fb521fcc700 1 --2- 172.21.15.163:0/3306792792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb51c0cad10 0x7fb51c15b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:36.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.582+0000 7fb521fcc700 1 -- 172.21.15.163:0/3306792792 >> 172.21.15.163:0/3306792792 conn(0x7fb51c0d10c0 msgr2=0x7fb51c0b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:36.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.582+0000 7fb521fcc700 1 -- 172.21.15.163:0/3306792792 shutdown_connections 2023-10-09T06:38:36.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.582+0000 7fb521fcc700 1 -- 172.21.15.163:0/3306792792 wait complete. 2023-10-09T06:38:36.595 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-09T06:38:36.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.714+0000 7ff2ccabd700 1 Processor -- start 2023-10-09T06:38:36.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.714+0000 7ff2ccabd700 1 -- start start 2023-10-09T06:38:36.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.715+0000 7ff2ccabd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff2c8157680 0x7ff2c80cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:36.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.715+0000 7ff2ccabd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff2c8157a50 0x7ff2c80cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:36.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.715+0000 7ff2ccabd700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2c80cb900 con 0x7ff2c80b1450 2023-10-09T06:38:36.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.715+0000 7ff2ccabd700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2c80cc700 con 0x7ff2c8157680 2023-10-09T06:38:36.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.715+0000 7ff2ccabd700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2c80cd240 con 0x7ff2c8157a50 2023-10-09T06:38:36.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.715+0000 7ff2c77fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff2c8157680 0x7ff2c80cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:36.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.715+0000 7ff2c7fff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff2c80b1450 0x7ff2c80c8cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:46048/0 (socket says 172.21.15.163:46048) 2023-10-09T06:38:36.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.715+0000 7ff2c7fff700 1 -- 172.21.15.163:0/1427421866 learned_addr learned my addr 172.21.15.163:0/1427421866 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:36.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.716+0000 7ff2c6ffd700 1 --2- 172.21.15.163:0/1427421866 >> v2:172.21.15.181:3300/0 conn(0x7ff2c8157a50 0x7ff2c80cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:36.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.716+0000 7ff2c77fe700 1 -- 172.21.15.163:0/1427421866 >> v2:172.21.15.181:3300/0 conn(0x7ff2c8157a50 msgr2=0x7ff2c80cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:36.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.716+0000 7ff2c77fe700 1 --2- 172.21.15.163:0/1427421866 >> v2:172.21.15.181:3300/0 conn(0x7ff2c8157a50 0x7ff2c80cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:36.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.716+0000 7ff2c77fe700 1 -- 172.21.15.163:0/1427421866 >> v1:172.21.15.163:6789/0 conn(0x7ff2c80b1450 legacy=0x7ff2c80c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:36.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.716+0000 7ff2c77fe700 1 -- 172.21.15.163:0/1427421866 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c80c9a20 con 0x7ff2c8157680 2023-10-09T06:38:36.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.716+0000 7ff2c77fe700 1 --2- 172.21.15.163:0/1427421866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2c8157680 0x7ff2c80cae80 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7ff2c80cbc70 tx=0x7ff2b8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5d11e15d1165b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:36.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.716+0000 7ff2c67fc700 1 -- 172.21.15.163:0/1427421866 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b800e070 con 0x7ff2c8157680 2023-10-09T06:38:36.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.716+0000 7ff2c67fc700 1 -- 172.21.15.163:0/1427421866 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b800bc30 con 0x7ff2c8157680 2023-10-09T06:38:36.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.716+0000 7ff2c67fc700 1 -- 172.21.15.163:0/1427421866 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8012430 con 0x7ff2c8157680 2023-10-09T06:38:36.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.716+0000 7ff2ccabd700 1 -- 172.21.15.163:0/1427421866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2c8157680 msgr2=0x7ff2c80cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:36.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.716+0000 7ff2ccabd700 1 --2- 172.21.15.163:0/1427421866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2c8157680 0x7ff2c80cae80 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7ff2c80cbc70 tx=0x7ff2b8009780 comp rx=0 tx=0).stop 2023-10-09T06:38:36.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.716+0000 7ff2ccabd700 1 -- 172.21.15.163:0/1427421866 shutdown_connections 2023-10-09T06:38:36.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.716+0000 7ff2ccabd700 1 --2- 172.21.15.163:0/1427421866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2c8157680 0x7ff2c80cae80 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:36.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.716+0000 7ff2ccabd700 1 --2- 172.21.15.163:0/1427421866 >> v2:172.21.15.181:3300/0 conn(0x7ff2c8157a50 0x7ff2c80cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:36.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.716+0000 7ff2ccabd700 1 -- 172.21.15.163:0/1427421866 >> 172.21.15.163:0/1427421866 conn(0x7ff2c806baa0 msgr2=0x7ff2c806bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:36.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.717+0000 7ff2ccabd700 1 -- 172.21.15.163:0/1427421866 shutdown_connections 2023-10-09T06:38:36.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.717+0000 7ff2ccabd700 1 -- 172.21.15.163:0/1427421866 wait complete. 2023-10-09T06:38:36.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.718+0000 7ff2ccabd700 1 Processor -- start 2023-10-09T06:38:36.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.718+0000 7ff2ccabd700 1 -- start start 2023-10-09T06:38:36.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.718+0000 7ff2ccabd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2c80b1450 0x7ff2c816a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:36.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.719+0000 7ff2ccabd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2c8157680 0x7ff2c80b0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:36.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.719+0000 7ff2ccabd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2c8157a50 0x7ff2c81595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:36.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.719+0000 7ff2ccabd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c8069530 con 0x7ff2c80b1450 2023-10-09T06:38:36.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.719+0000 7ff2ccabd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c816a650 con 0x7ff2c8157680 2023-10-09T06:38:36.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.719+0000 7ff2ccabd700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c816a930 con 0x7ff2c8157a50 2023-10-09T06:38:36.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.719+0000 7ff2c77fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2c80b1450 0x7ff2c816a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:36.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.719+0000 7ff2c77fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2c80b1450 0x7ff2c816a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47186/0 (socket says 172.21.15.163:47186) 2023-10-09T06:38:36.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.719+0000 7ff2c77fe700 1 -- 172.21.15.163:0/931006344 learned_addr learned my addr 172.21.15.163:0/931006344 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:36.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.719+0000 7ff2c6ffd700 1 --2- 172.21.15.163:0/931006344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2c8157680 0x7ff2c80b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:36.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.719+0000 7ff2c7fff700 1 --2- 172.21.15.163:0/931006344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2c8157a50 0x7ff2c81595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:36.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.719+0000 7ff2c77fe700 1 -- 172.21.15.163:0/931006344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2c8157a50 msgr2=0x7ff2c81595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:36.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.719+0000 7ff2c77fe700 1 --2- 172.21.15.163:0/931006344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2c8157a50 0x7ff2c81595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:36.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.719+0000 7ff2c77fe700 1 -- 172.21.15.163:0/931006344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2c8157680 msgr2=0x7ff2c80b0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:36.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.719+0000 7ff2c77fe700 1 --2- 172.21.15.163:0/931006344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2c8157680 0x7ff2c80b0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:36.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.719+0000 7ff2c77fe700 1 -- 172.21.15.163:0/931006344 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b8009a20 con 0x7ff2c80b1450 2023-10-09T06:38:36.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.719+0000 7ff2c6ffd700 1 --2- 172.21.15.163:0/931006344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2c8157680 0x7ff2c80b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:36.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.719+0000 7ff2c77fe700 1 --2- 172.21.15.163:0/931006344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2c80b1450 0x7ff2c816a020 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80097c0 tx=0x7ff2b8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:36.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.719+0000 7ff2af7fe700 1 -- 172.21.15.163:0/931006344 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8001090 con 0x7ff2c80b1450 2023-10-09T06:38:36.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.719+0000 7ff2af7fe700 1 -- 172.21.15.163:0/931006344 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b8012e20 con 0x7ff2c80b1450 2023-10-09T06:38:36.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.719+0000 7ff2ccabd700 1 -- 172.21.15.163:0/931006344 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2c816ac10 con 0x7ff2c80b1450 2023-10-09T06:38:36.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.719+0000 7ff2af7fe700 1 -- 172.21.15.163:0/931006344 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b801d860 con 0x7ff2c80b1450 2023-10-09T06:38:36.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.719+0000 7ff2ccabd700 1 -- 172.21.15.163:0/931006344 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2c80b0550 con 0x7ff2c80b1450 2023-10-09T06:38:36.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.720+0000 7ff2ccabd700 1 -- 172.21.15.163:0/931006344 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2c806eba0 con 0x7ff2c80b1450 2023-10-09T06:38:36.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.721+0000 7ff2af7fe700 1 -- 172.21.15.163:0/931006344 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff2b8024070 con 0x7ff2c80b1450 2023-10-09T06:38:36.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.721+0000 7ff2af7fe700 1 --2- 172.21.15.163:0/931006344 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff2b4067320 0x7ff2b40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:36.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.721+0000 7ff2af7fe700 1 -- 172.21.15.163:0/931006344 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff2b8087370 con 0x7ff2c80b1450 2023-10-09T06:38:36.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.721+0000 7ff2c6ffd700 1 --2- 172.21.15.163:0/931006344 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff2b4067320 0x7ff2b40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:36.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.722+0000 7ff2c6ffd700 1 --2- 172.21.15.163:0/931006344 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff2b4067320 0x7ff2b40697d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8151ae0 tx=0x7ff2bc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:36.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.725+0000 7ff2af7fe700 1 -- 172.21.15.163:0/931006344 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff2b8054000 con 0x7ff2c80b1450 2023-10-09T06:38:36.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.909+0000 7ff2ccabd700 1 -- 172.21.15.163:0/931006344 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff2c80cc910 con 0x7ff2c80b1450 2023-10-09T06:38:36.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.909+0000 7ff2af7fe700 1 -- 172.21.15.163:0/931006344 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v152) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff2b8057620 con 0x7ff2c80b1450 2023-10-09T06:38:36.915 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 64 2023-10-09T06:38:36.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.911+0000 7ff2ccabd700 1 -- 172.21.15.163:0/931006344 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff2b4067320 msgr2=0x7ff2b40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:36.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.911+0000 7ff2ccabd700 1 --2- 172.21.15.163:0/931006344 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff2b4067320 0x7ff2b40697d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8151ae0 tx=0x7ff2bc008040 comp rx=0 tx=0).stop 2023-10-09T06:38:36.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.911+0000 7ff2ccabd700 1 -- 172.21.15.163:0/931006344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2c80b1450 msgr2=0x7ff2c816a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:36.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.911+0000 7ff2ccabd700 1 --2- 172.21.15.163:0/931006344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2c80b1450 0x7ff2c816a020 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80097c0 tx=0x7ff2b8019000 comp rx=0 tx=0).stop 2023-10-09T06:38:36.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.911+0000 7ff2ccabd700 1 -- 172.21.15.163:0/931006344 shutdown_connections 2023-10-09T06:38:36.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.911+0000 7ff2ccabd700 1 --2- 172.21.15.163:0/931006344 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff2b4067320 0x7ff2b40697d0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:36.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.911+0000 7ff2ccabd700 1 --2- 172.21.15.163:0/931006344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2c80b1450 0x7ff2c816a020 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:36.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.911+0000 7ff2ccabd700 1 --2- 172.21.15.163:0/931006344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2c8157a50 0x7ff2c81595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:36.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.911+0000 7ff2ccabd700 1 --2- 172.21.15.163:0/931006344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2c8157680 0x7ff2c80b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:36.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.911+0000 7ff2ccabd700 1 -- 172.21.15.163:0/931006344 >> 172.21.15.163:0/931006344 conn(0x7ff2c806baa0 msgr2=0x7ff2c81531b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:36.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.911+0000 7ff2ccabd700 1 -- 172.21.15.163:0/931006344 shutdown_connections 2023-10-09T06:38:36.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:36.912+0000 7ff2ccabd700 1 -- 172.21.15.163:0/931006344 wait complete. 2023-10-09T06:38:36.925 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:38:37.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.038+0000 7fc8ffaa7700 1 Processor -- start 2023-10-09T06:38:37.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.039+0000 7fc8ffaa7700 1 -- start start 2023-10-09T06:38:37.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.039+0000 7fc8ffaa7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc8f8068920 0x7fc8f806f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:37.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.039+0000 7fc8ffaa7700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f806ec70 con 0x7fc8f806f640 2023-10-09T06:38:37.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.039+0000 7fc8ffaa7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f80b2e70 con 0x7fc8f8068550 2023-10-09T06:38:37.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.039+0000 7fc8ffaa7700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f8068d90 con 0x7fc8f8068920 2023-10-09T06:38:37.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.039+0000 7fc8fd042700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc8f8068920 0x7fc8f806f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:37.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.039+0000 7fc8fd042700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc8f8068920 0x7fc8f806f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47200/0 (socket says 172.21.15.163:47200) 2023-10-09T06:38:37.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.039+0000 7fc8fd042700 1 -- 172.21.15.163:0/3008808239 learned_addr learned my addr 172.21.15.163:0/3008808239 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:37.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.039+0000 7fc8fe044700 1 --1- 172.21.15.163:0/3008808239 >> v1:172.21.15.163:6789/0 conn(0x7fc8f806f640 0x7fc8f8159510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:46052/0 (socket says 172.21.15.163:46052) 2023-10-09T06:38:37.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.040+0000 7fc8fd042700 1 -- 172.21.15.163:0/3008808239 >> v1:172.21.15.181:6789/0 conn(0x7fc8f8068550 legacy=0x7fc8f815e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:37.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.040+0000 7fc8fd042700 1 -- 172.21.15.163:0/3008808239 >> v1:172.21.15.163:6789/0 conn(0x7fc8f806f640 legacy=0x7fc8f8159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:37.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.040+0000 7fc8fd042700 1 -- 172.21.15.163:0/3008808239 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f80b3150 con 0x7fc8f8068920 2023-10-09T06:38:37.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.040+0000 7fc8fc841700 1 -- 172.21.15.163:0/3008808239 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 542497093 0 0) 0x7fc8f806ec70 con 0x7fc8f806f640 2023-10-09T06:38:37.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.040+0000 7fc8fd042700 1 --2- 172.21.15.163:0/3008808239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f8068920 0x7fc8f806f100 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fc8e80099b0 tx=0x7fc8e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f70d487335eddb70 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:37.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.040+0000 7fc8fc841700 1 -- 172.21.15.163:0/3008808239 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e800b860 con 0x7fc8f8068920 2023-10-09T06:38:37.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.040+0000 7fc8fc841700 1 -- 172.21.15.163:0/3008808239 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e800b9c0 con 0x7fc8f8068920 2023-10-09T06:38:37.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.040+0000 7fc8fc841700 1 -- 172.21.15.163:0/3008808239 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e800bd60 con 0x7fc8f8068920 2023-10-09T06:38:37.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.040+0000 7fc8ffaa7700 1 -- 172.21.15.163:0/3008808239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f8068920 msgr2=0x7fc8f806f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:37.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.040+0000 7fc8ffaa7700 1 --2- 172.21.15.163:0/3008808239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f8068920 0x7fc8f806f100 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fc8e80099b0 tx=0x7fc8e8009250 comp rx=0 tx=0).stop 2023-10-09T06:38:37.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.041+0000 7fc8ffaa7700 1 -- 172.21.15.163:0/3008808239 shutdown_connections 2023-10-09T06:38:37.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.041+0000 7fc8ffaa7700 1 --2- 172.21.15.163:0/3008808239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f8068920 0x7fc8f806f100 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.041+0000 7fc8ffaa7700 1 -- 172.21.15.163:0/3008808239 >> 172.21.15.163:0/3008808239 conn(0x7fc8f80cd480 msgr2=0x7fc8f80cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:37.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.041+0000 7fc8ffaa7700 1 -- 172.21.15.163:0/3008808239 shutdown_connections 2023-10-09T06:38:37.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.041+0000 7fc8ffaa7700 1 -- 172.21.15.163:0/3008808239 wait complete. 2023-10-09T06:38:37.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.042+0000 7fc8ffaa7700 1 Processor -- start 2023-10-09T06:38:37.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.042+0000 7fc8ffaa7700 1 -- start start 2023-10-09T06:38:37.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.042+0000 7fc8ffaa7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f8068550 0x7fc8f815b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:37.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.042+0000 7fc8ffaa7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8f8068920 0x7fc8f815b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:37.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.042+0000 7fc8ffaa7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8f806f640 0x7fc8f8156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:37.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.042+0000 7fc8ffaa7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f815cb90 con 0x7fc8f8068550 2023-10-09T06:38:37.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.042+0000 7fc8ffaa7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f815bcd0 con 0x7fc8f806f640 2023-10-09T06:38:37.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.042+0000 7fc8ffaa7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f815bf80 con 0x7fc8f8068920 2023-10-09T06:38:37.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.043+0000 7fc8fd843700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f8068550 0x7fc8f815b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:37.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.043+0000 7fc8fd843700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f8068550 0x7fc8f815b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47206/0 (socket says 172.21.15.163:47206) 2023-10-09T06:38:37.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.043+0000 7fc8fd843700 1 -- 172.21.15.163:0/707529967 learned_addr learned my addr 172.21.15.163:0/707529967 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:37.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.043+0000 7fc8fd042700 1 --2- 172.21.15.163:0/707529967 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8f8068920 0x7fc8f815b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:37.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.043+0000 7fc8fe044700 1 --2- 172.21.15.163:0/707529967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8f806f640 0x7fc8f8156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:37.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.043+0000 7fc8fd843700 1 -- 172.21.15.163:0/707529967 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8f8068920 msgr2=0x7fc8f815b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:37.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.043+0000 7fc8fd843700 1 --2- 172.21.15.163:0/707529967 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8f8068920 0x7fc8f815b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.043+0000 7fc8fd843700 1 -- 172.21.15.163:0/707529967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8f806f640 msgr2=0x7fc8f8156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:37.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.043+0000 7fc8fd843700 1 --2- 172.21.15.163:0/707529967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8f806f640 0x7fc8f8156ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.043+0000 7fc8fd843700 1 -- 172.21.15.163:0/707529967 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e80094f0 con 0x7fc8f8068550 2023-10-09T06:38:37.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.043+0000 7fc8fe044700 1 --2- 172.21.15.163:0/707529967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8f806f640 0x7fc8f8156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:37.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.043+0000 7fc8fd042700 1 --2- 172.21.15.163:0/707529967 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8f8068920 0x7fc8f815b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:37.049 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.043+0000 7fc8fd843700 1 --2- 172.21.15.163:0/707529967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f8068550 0x7fc8f815b250 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400c4e0 tx=0x7fc8f400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:37.049 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.043+0000 7fc8edffb700 1 -- 172.21.15.163:0/707529967 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4011590 con 0x7fc8f8068550 2023-10-09T06:38:37.049 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.043+0000 7fc8edffb700 1 -- 172.21.15.163:0/707529967 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f4011710 con 0x7fc8f8068550 2023-10-09T06:38:37.049 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.043+0000 7fc8edffb700 1 -- 172.21.15.163:0/707529967 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4011ad0 con 0x7fc8f8068550 2023-10-09T06:38:37.049 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.043+0000 7fc8ffaa7700 1 -- 172.21.15.163:0/707529967 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f815c260 con 0x7fc8f8068550 2023-10-09T06:38:37.049 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.043+0000 7fc8ffaa7700 1 -- 172.21.15.163:0/707529967 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f815c820 con 0x7fc8f8068550 2023-10-09T06:38:37.049 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.044+0000 7fc8ffaa7700 1 -- 172.21.15.163:0/707529967 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f804c3c0 con 0x7fc8f8068550 2023-10-09T06:38:37.049 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.044+0000 7fc8edffb700 1 -- 172.21.15.163:0/707529967 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc8f401b3b0 con 0x7fc8f8068550 2023-10-09T06:38:37.049 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.045+0000 7fc8edffb700 1 --2- 172.21.15.163:0/707529967 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc8dc067320 0x7fc8dc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:37.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.045+0000 7fc8edffb700 1 -- 172.21.15.163:0/707529967 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc8f40841a0 con 0x7fc8f8068550 2023-10-09T06:38:37.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.045+0000 7fc8fd042700 1 --2- 172.21.15.163:0/707529967 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc8dc067320 0x7fc8dc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:37.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.046+0000 7fc8fd042700 1 --2- 172.21.15.163:0/707529967 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc8dc067320 0x7fc8dc0697d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc8e80099b0 tx=0x7fc8e8017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:37.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.049+0000 7fc8edffb700 1 -- 172.21.15.163:0/707529967 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc8f4050e30 con 0x7fc8f8068550 2023-10-09T06:38:37.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.277+0000 7fc8ffaa7700 1 -- 172.21.15.163:0/707529967 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8f806c990 con 0x7fc8f8068550 2023-10-09T06:38:37.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.278+0000 7fc8edffb700 1 -- 172.21.15.163:0/707529967 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4615 (secure 0 0 0) 0x7fc8f4017090 con 0x7fc8f8068550 2023-10-09T06:38:37.279 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:37.279 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":104,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6371,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3797578839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3797578839},{"type":"v1","addr":"172.21.15.163:6834","nonce":3797578839}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":6383,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2371721802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2371721802},{"type":"v1","addr":"172.21.15.181:6837","nonce":2371721802}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":7228,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2068214884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2068214884},{"type":"v1","addr":"172.21.15.181:6836","nonce":2068214884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:38:20.060175+0000","modified":"2023-10-09T06:38:23.693400+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7875},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7875":{"gid":7875,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/3794912550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3794912550},{"type":"v1","addr":"172.21.15.163:6835","nonce":3794912550}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-09T06:38:37.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.280+0000 7fc8ffaa7700 1 -- 172.21.15.163:0/707529967 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc8dc067320 msgr2=0x7fc8dc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:37.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.280+0000 7fc8ffaa7700 1 --2- 172.21.15.163:0/707529967 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc8dc067320 0x7fc8dc0697d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc8e80099b0 tx=0x7fc8e8017000 comp rx=0 tx=0).stop 2023-10-09T06:38:37.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.280+0000 7fc8ffaa7700 1 -- 172.21.15.163:0/707529967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f8068550 msgr2=0x7fc8f815b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:37.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.280+0000 7fc8ffaa7700 1 --2- 172.21.15.163:0/707529967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f8068550 0x7fc8f815b250 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400c4e0 tx=0x7fc8f400c8a0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.280+0000 7fc8ffaa7700 1 -- 172.21.15.163:0/707529967 shutdown_connections 2023-10-09T06:38:37.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.280+0000 7fc8ffaa7700 1 --2- 172.21.15.163:0/707529967 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc8dc067320 0x7fc8dc0697d0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.280+0000 7fc8ffaa7700 1 --2- 172.21.15.163:0/707529967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f8068550 0x7fc8f815b250 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.280+0000 7fc8ffaa7700 1 --2- 172.21.15.163:0/707529967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8f806f640 0x7fc8f8156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.280+0000 7fc8ffaa7700 1 --2- 172.21.15.163:0/707529967 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8f8068920 0x7fc8f815b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.280+0000 7fc8ffaa7700 1 -- 172.21.15.163:0/707529967 >> 172.21.15.163:0/707529967 conn(0x7fc8f80cd480 msgr2=0x7fc8f8155e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:37.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.280+0000 7fc8ffaa7700 1 -- 172.21.15.163:0/707529967 shutdown_connections 2023-10-09T06:38:37.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.280+0000 7fc8ffaa7700 1 -- 172.21.15.163:0/707529967 wait complete. 2023-10-09T06:38:37.282 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 104 2023-10-09T06:38:37.293 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:38:37.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.407+0000 7f7fac9d9700 1 Processor -- start 2023-10-09T06:38:37.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.407+0000 7f7fac9d9700 1 -- start start 2023-10-09T06:38:37.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.408+0000 7f7fac9d9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fa8159420 0x7f7fa80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:37.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.408+0000 7f7fac9d9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7fa81597f0 0x7f7fa80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:37.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.408+0000 7f7fac9d9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa80b11d0 con 0x7f7fa815d230 2023-10-09T06:38:37.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.408+0000 7f7fac9d9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa80b14b0 con 0x7f7fa81597f0 2023-10-09T06:38:37.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.408+0000 7f7fac9d9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa80b1790 con 0x7f7fa8159420 2023-10-09T06:38:37.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.408+0000 7f7f9e59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7fa81597f0 0x7f7fa80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:37.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.408+0000 7f7fa6d9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fa8159420 0x7f7fa80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:37.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.408+0000 7f7fa759e700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7fa815d230 0x7f7fa80b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:44004/0 (socket says 172.21.15.163:44004) 2023-10-09T06:38:37.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.408+0000 7f7fa759e700 1 -- 172.21.15.163:0/2780343215 learned_addr learned my addr 172.21.15.163:0/2780343215 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:37.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.409+0000 7f7f9e59c700 1 -- 172.21.15.163:0/2780343215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fa8159420 msgr2=0x7f7fa80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:37.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.409+0000 7f7f9e59c700 1 --2- 172.21.15.163:0/2780343215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fa8159420 0x7f7fa80b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.409+0000 7f7f9e59c700 1 -- 172.21.15.163:0/2780343215 >> v1:172.21.15.181:6789/0 conn(0x7f7fa815d230 legacy=0x7f7fa80b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:37.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.409+0000 7f7fa659c700 1 -- 172.21.15.163:0/2780343215 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3910053253 0 0) 0x7f7fa80b11d0 con 0x7f7fa815d230 2023-10-09T06:38:37.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.409+0000 7f7f9e59c700 1 -- 172.21.15.163:0/2780343215 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa80b1a70 con 0x7f7fa81597f0 2023-10-09T06:38:37.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.409+0000 7f7fa6d9d700 1 --2- 172.21.15.163:0/2780343215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fa8159420 0x7f7fa80b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:38:37.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.409+0000 7f7f9e59c700 1 --2- 172.21.15.163:0/2780343215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fa81597f0 0x7f7fa80b2270 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7f900098e0 tx=0x7f7f900091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3846e2cc8abf767 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:37.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.409+0000 7f7fa659c700 1 -- 172.21.15.163:0/2780343215 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9000b640 con 0x7f7fa81597f0 2023-10-09T06:38:37.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.409+0000 7f7fa659c700 1 -- 172.21.15.163:0/2780343215 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f9000b7a0 con 0x7f7fa81597f0 2023-10-09T06:38:37.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.409+0000 7f7fa659c700 1 -- 172.21.15.163:0/2780343215 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9000bb40 con 0x7f7fa81597f0 2023-10-09T06:38:37.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.410+0000 7f7fac9d9700 1 -- 172.21.15.163:0/2780343215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fa81597f0 msgr2=0x7f7fa80b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:37.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.410+0000 7f7fac9d9700 1 --2- 172.21.15.163:0/2780343215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fa81597f0 0x7f7fa80b2270 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7f900098e0 tx=0x7f7f900091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.410+0000 7f7fac9d9700 1 -- 172.21.15.163:0/2780343215 shutdown_connections 2023-10-09T06:38:37.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.410+0000 7f7fac9d9700 1 --2- 172.21.15.163:0/2780343215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fa8159420 0x7f7fa80b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.410+0000 7f7fac9d9700 1 --2- 172.21.15.163:0/2780343215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fa81597f0 0x7f7fa80b2270 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.410+0000 7f7fac9d9700 1 -- 172.21.15.163:0/2780343215 >> 172.21.15.163:0/2780343215 conn(0x7f7fa80d0660 msgr2=0x7f7fa80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:37.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.410+0000 7f7fac9d9700 1 -- 172.21.15.163:0/2780343215 shutdown_connections 2023-10-09T06:38:37.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.410+0000 7f7fac9d9700 1 -- 172.21.15.163:0/2780343215 wait complete. 2023-10-09T06:38:37.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.411+0000 7f7fac9d9700 1 Processor -- start 2023-10-09T06:38:37.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.411+0000 7f7fac9d9700 1 -- start start 2023-10-09T06:38:37.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.412+0000 7f7fac9d9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fa8159420 0x7f7fa81633f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:37.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.412+0000 7f7fac9d9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fa81597f0 0x7f7fa815d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:37.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.412+0000 7f7fac9d9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fa815d230 0x7f7fa815dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:37.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.412+0000 7f7fac9d9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa81698e0 con 0x7f7fa815d230 2023-10-09T06:38:37.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.412+0000 7f7fac9d9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa815e4d0 con 0x7f7fa8159420 2023-10-09T06:38:37.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.412+0000 7f7fac9d9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa815e7b0 con 0x7f7fa81597f0 2023-10-09T06:38:37.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.412+0000 7f7fa759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fa815d230 0x7f7fa815dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:37.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.412+0000 7f7fa759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fa815d230 0x7f7fa815dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47208/0 (socket says 172.21.15.163:47208) 2023-10-09T06:38:37.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.412+0000 7f7fa759e700 1 -- 172.21.15.163:0/1011874972 learned_addr learned my addr 172.21.15.163:0/1011874972 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:37.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.412+0000 7f7f9e59c700 1 --2- 172.21.15.163:0/1011874972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fa81597f0 0x7f7fa815d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:37.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.412+0000 7f7fa6d9d700 1 --2- 172.21.15.163:0/1011874972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fa8159420 0x7f7fa81633f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:37.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.412+0000 7f7fa759e700 1 -- 172.21.15.163:0/1011874972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fa81597f0 msgr2=0x7f7fa815d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:37.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.412+0000 7f7fa759e700 1 --2- 172.21.15.163:0/1011874972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fa81597f0 0x7f7fa815d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.412+0000 7f7fa759e700 1 -- 172.21.15.163:0/1011874972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fa8159420 msgr2=0x7f7fa81633f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:37.418 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.412+0000 7f7fa759e700 1 --2- 172.21.15.163:0/1011874972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fa8159420 0x7f7fa81633f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.412+0000 7f7fa759e700 1 -- 172.21.15.163:0/1011874972 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f98009420 con 0x7f7fa815d230 2023-10-09T06:38:37.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.412+0000 7f7fa759e700 1 --2- 172.21.15.163:0/1011874972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fa815d230 0x7f7fa815dea0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f7fa000cbf0 tx=0x7f7fa00019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:37.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.412+0000 7f7f9f7fe700 1 -- 172.21.15.163:0/1011874972 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa000a8b0 con 0x7f7fa815d230 2023-10-09T06:38:37.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.412+0000 7f7f9f7fe700 1 -- 172.21.15.163:0/1011874972 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa000aa10 con 0x7f7fa815d230 2023-10-09T06:38:37.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.412+0000 7f7fac9d9700 1 -- 172.21.15.163:0/1011874972 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f90009420 con 0x7f7fa815d230 2023-10-09T06:38:37.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.413+0000 7f7f9f7fe700 1 -- 172.21.15.163:0/1011874972 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa000add0 con 0x7f7fa815d230 2023-10-09T06:38:37.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.413+0000 7f7fac9d9700 1 -- 172.21.15.163:0/1011874972 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa8159ee0 con 0x7f7fa815d230 2023-10-09T06:38:37.419 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.413+0000 7f7fac9d9700 1 -- 172.21.15.163:0/1011874972 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa804c3c0 con 0x7f7fa815d230 2023-10-09T06:38:37.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.414+0000 7f7f9f7fe700 1 -- 172.21.15.163:0/1011874972 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7fa0007300 con 0x7f7fa815d230 2023-10-09T06:38:37.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.414+0000 7f7f9f7fe700 1 --2- 172.21.15.163:0/1011874972 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7f88067320 0x7f7f880697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:37.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.414+0000 7f7f9f7fe700 1 -- 172.21.15.163:0/1011874972 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7fa0084f10 con 0x7f7fa815d230 2023-10-09T06:38:37.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.414+0000 7f7fa6d9d700 1 --2- 172.21.15.163:0/1011874972 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7f88067320 0x7f7f880697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:37.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.415+0000 7f7fa6d9d700 1 --2- 172.21.15.163:0/1011874972 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7f88067320 0x7f7f880697d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7fa80af7f0 tx=0x7f7f980091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:37.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.418+0000 7f7f9f7fe700 1 -- 172.21.15.163:0/1011874972 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7fa0051c20 con 0x7f7fa815d230 2023-10-09T06:38:37.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.601+0000 7f7fac9d9700 1 -- 172.21.15.163:0/1011874972 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7fa80b1d50 con 0x7f7fa815d230 2023-10-09T06:38:37.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.601+0000 7f7f9f7fe700 1 -- 172.21.15.163:0/1011874972 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) v1 ==== 74+0+15412 (secure 0 0 0) 0x7f7fa0055240 con 0x7f7fa815d230 2023-10-09T06:38:37.602 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:37.603 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":152,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:38:21.683060+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:38:18.381850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-09T06:38:19.022637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:38:37.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.603+0000 7f7fac9d9700 1 -- 172.21.15.163:0/1011874972 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7f88067320 msgr2=0x7f7f880697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:37.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.603+0000 7f7fac9d9700 1 --2- 172.21.15.163:0/1011874972 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7f88067320 0x7f7f880697d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7fa80af7f0 tx=0x7f7f980091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.603+0000 7f7fac9d9700 1 -- 172.21.15.163:0/1011874972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fa815d230 msgr2=0x7f7fa815dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:37.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.603+0000 7f7fac9d9700 1 --2- 172.21.15.163:0/1011874972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fa815d230 0x7f7fa815dea0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f7fa000cbf0 tx=0x7f7fa00019b0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.603+0000 7f7fac9d9700 1 -- 172.21.15.163:0/1011874972 shutdown_connections 2023-10-09T06:38:37.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.603+0000 7f7fac9d9700 1 --2- 172.21.15.163:0/1011874972 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7f88067320 0x7f7f880697d0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.603+0000 7f7fac9d9700 1 --2- 172.21.15.163:0/1011874972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fa815d230 0x7f7fa815dea0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.603+0000 7f7fac9d9700 1 --2- 172.21.15.163:0/1011874972 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fa8159420 0x7f7fa81633f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.603+0000 7f7fac9d9700 1 --2- 172.21.15.163:0/1011874972 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fa81597f0 0x7f7fa815d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.603+0000 7f7fac9d9700 1 -- 172.21.15.163:0/1011874972 >> 172.21.15.163:0/1011874972 conn(0x7f7fa80d0660 msgr2=0x7f7fa8155300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:37.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.604+0000 7f7fac9d9700 1 -- 172.21.15.163:0/1011874972 shutdown_connections 2023-10-09T06:38:37.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.604+0000 7f7fac9d9700 1 -- 172.21.15.163:0/1011874972 wait complete. 2023-10-09T06:38:37.617 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-09T06:38:37.617 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:38:37.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.731+0000 7f9fddee1700 1 Processor -- start 2023-10-09T06:38:37.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.731+0000 7f9fddee1700 1 -- start start 2023-10-09T06:38:37.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.732+0000 7f9fddee1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9fd81597f0 0x7f9fd80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:37.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.732+0000 7f9fddee1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fd815d230 0x7f9fd815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:37.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.732+0000 7f9fddee1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd815dbb0 con 0x7f9fd8159420 2023-10-09T06:38:37.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.732+0000 7f9fddee1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fd80b11d0 con 0x7f9fd81597f0 2023-10-09T06:38:37.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.732+0000 7f9fddee1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd80b14b0 con 0x7f9fd815d230 2023-10-09T06:38:37.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.732+0000 7f9fd7fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9fd81597f0 0x7f9fd80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:37.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.732+0000 7f9fdd6e0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fd815d230 0x7f9fd815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:37.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.732+0000 7f9fdcedf700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9fd8159420 0x7f9fd80b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:44012/0 (socket says 172.21.15.163:44012) 2023-10-09T06:38:37.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.732+0000 7f9fdcedf700 1 -- 172.21.15.163:0/2804921486 learned_addr learned my addr 172.21.15.163:0/2804921486 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:37.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.732+0000 7f9fd77fe700 1 -- 172.21.15.163:0/2804921486 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4016814281 0 0) 0x7f9fd815dbb0 con 0x7f9fd8159420 2023-10-09T06:38:37.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.732+0000 7f9fd77fe700 1 -- 172.21.15.163:0/2804921486 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fc8003570 con 0x7f9fd8159420 2023-10-09T06:38:37.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.732+0000 7f9fdd6e0700 1 -- 172.21.15.163:0/2804921486 >> v2:172.21.15.181:3300/0 conn(0x7f9fd81597f0 msgr2=0x7f9fd80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:37.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.732+0000 7f9fdd6e0700 1 --2- 172.21.15.163:0/2804921486 >> v2:172.21.15.181:3300/0 conn(0x7f9fd81597f0 0x7f9fd80b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.732+0000 7f9fdd6e0700 1 -- 172.21.15.163:0/2804921486 >> v1:172.21.15.181:6789/0 conn(0x7f9fd8159420 legacy=0x7f9fd80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:37.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.733+0000 7f9fdd6e0700 1 -- 172.21.15.163:0/2804921486 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd80b1790 con 0x7f9fd815d230 2023-10-09T06:38:37.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.733+0000 7f9fd7fff700 1 --2- 172.21.15.163:0/2804921486 >> v2:172.21.15.181:3300/0 conn(0x7f9fd81597f0 0x7f9fd80b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:38:37.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.733+0000 7f9fdd6e0700 1 --2- 172.21.15.163:0/2804921486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fd815d230 0x7f9fd815d670 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0009e60 tx=0x7f9fd0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce3b5cd1fdabaebc server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:37.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.733+0000 7f9fd77fe700 1 -- 172.21.15.163:0/2804921486 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd000e070 con 0x7f9fd815d230 2023-10-09T06:38:37.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.733+0000 7f9fd77fe700 1 -- 172.21.15.163:0/2804921486 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd000bab0 con 0x7f9fd815d230 2023-10-09T06:38:37.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.733+0000 7f9fd77fe700 1 -- 172.21.15.163:0/2804921486 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd000be50 con 0x7f9fd815d230 2023-10-09T06:38:37.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.733+0000 7f9fddee1700 1 -- 172.21.15.163:0/2804921486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fd815d230 msgr2=0x7f9fd815d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:37.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.733+0000 7f9fddee1700 1 --2- 172.21.15.163:0/2804921486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fd815d230 0x7f9fd815d670 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0009e60 tx=0x7f9fd0009780 comp rx=0 tx=0).stop 2023-10-09T06:38:37.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.733+0000 7f9fddee1700 1 -- 172.21.15.163:0/2804921486 shutdown_connections 2023-10-09T06:38:37.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.733+0000 7f9fddee1700 1 --2- 172.21.15.163:0/2804921486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fd815d230 0x7f9fd815d670 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.733+0000 7f9fddee1700 1 --2- 172.21.15.163:0/2804921486 >> v2:172.21.15.181:3300/0 conn(0x7f9fd81597f0 0x7f9fd80b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.733+0000 7f9fddee1700 1 -- 172.21.15.163:0/2804921486 >> 172.21.15.163:0/2804921486 conn(0x7f9fd80d0660 msgr2=0x7f9fd80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:37.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.733+0000 7f9fddee1700 1 -- 172.21.15.163:0/2804921486 shutdown_connections 2023-10-09T06:38:37.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.733+0000 7f9fddee1700 1 -- 172.21.15.163:0/2804921486 wait complete. 2023-10-09T06:38:37.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.734+0000 7f9fddee1700 1 Processor -- start 2023-10-09T06:38:37.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.735+0000 7f9fddee1700 1 -- start start 2023-10-09T06:38:37.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.735+0000 7f9fddee1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fd8159420 0x7f9fd81693c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:37.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.735+0000 7f9fddee1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fd81597f0 0x7f9fd814e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:37.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.735+0000 7f9fddee1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fd815d230 0x7f9fd815bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:37.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.735+0000 7f9fddee1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd814e6a0 con 0x7f9fd8159420 2023-10-09T06:38:37.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.735+0000 7f9fddee1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd815ae70 con 0x7f9fd81597f0 2023-10-09T06:38:37.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.735+0000 7f9fddee1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd815b300 con 0x7f9fd815d230 2023-10-09T06:38:37.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.735+0000 7f9fdcedf700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fd8159420 0x7f9fd81693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:37.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.735+0000 7f9fdcedf700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fd8159420 0x7f9fd81693c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47222/0 (socket says 172.21.15.163:47222) 2023-10-09T06:38:37.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.735+0000 7f9fdcedf700 1 -- 172.21.15.163:0/1919819073 learned_addr learned my addr 172.21.15.163:0/1919819073 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:37.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.736+0000 7f9fdd6e0700 1 --2- 172.21.15.163:0/1919819073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fd815d230 0x7f9fd815bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:37.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.736+0000 7f9fd7fff700 1 --2- 172.21.15.163:0/1919819073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fd81597f0 0x7f9fd814e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:37.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.736+0000 7f9fdcedf700 1 -- 172.21.15.163:0/1919819073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fd815d230 msgr2=0x7f9fd815bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:37.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.736+0000 7f9fdcedf700 1 --2- 172.21.15.163:0/1919819073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fd815d230 0x7f9fd815bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.736+0000 7f9fdcedf700 1 -- 172.21.15.163:0/1919819073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fd81597f0 msgr2=0x7f9fd814e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:37.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.736+0000 7f9fdcedf700 1 --2- 172.21.15.163:0/1919819073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fd81597f0 0x7f9fd814e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.736+0000 7f9fdcedf700 1 -- 172.21.15.163:0/1919819073 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc4009420 con 0x7f9fd8159420 2023-10-09T06:38:37.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.736+0000 7f9fdcedf700 1 --2- 172.21.15.163:0/1919819073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fd8159420 0x7f9fd81693c0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0011e0 tx=0x7f9fcc007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:37.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.736+0000 7f9fd4ff9700 1 -- 172.21.15.163:0/1919819073 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc005b30 con 0x7f9fd8159420 2023-10-09T06:38:37.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.736+0000 7f9fd4ff9700 1 -- 172.21.15.163:0/1919819073 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fcc005c90 con 0x7f9fd8159420 2023-10-09T06:38:37.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.736+0000 7f9fddee1700 1 -- 172.21.15.163:0/1919819073 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd0009a20 con 0x7f9fd8159420 2023-10-09T06:38:37.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.736+0000 7f9fd4ff9700 1 -- 172.21.15.163:0/1919819073 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc0072e0 con 0x7f9fd8159420 2023-10-09T06:38:37.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.736+0000 7f9fddee1700 1 -- 172.21.15.163:0/1919819073 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd815a820 con 0x7f9fd8159420 2023-10-09T06:38:37.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.737+0000 7f9fddee1700 1 -- 172.21.15.163:0/1919819073 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd815b150 con 0x7f9fd8159420 2023-10-09T06:38:37.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.737+0000 7f9fd4ff9700 1 -- 172.21.15.163:0/1919819073 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9fcc007650 con 0x7f9fd8159420 2023-10-09T06:38:37.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.738+0000 7f9fd4ff9700 1 --2- 172.21.15.163:0/1919819073 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9fbc067350 0x7f9fbc069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:37.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.738+0000 7f9fd4ff9700 1 -- 172.21.15.163:0/1919819073 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9fcc084080 con 0x7f9fd8159420 2023-10-09T06:38:37.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.738+0000 7f9fd7fff700 1 --2- 172.21.15.163:0/1919819073 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9fbc067350 0x7f9fbc069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:37.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.738+0000 7f9fd7fff700 1 --2- 172.21.15.163:0/1919819073 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9fbc067350 0x7f9fbc069800 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9fc40093f0 tx=0x7f9fc4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:37.743 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.742+0000 7f9fd4ff9700 1 -- 172.21.15.163:0/1919819073 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9fcc04b5a0 con 0x7f9fd8159420 2023-10-09T06:38:37.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.966+0000 7f9fddee1700 1 -- 172.21.15.163:0/1919819073 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fd804c3f0 con 0x7f9fd8159420 2023-10-09T06:38:37.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.967+0000 7f9fd4ff9700 1 -- 172.21.15.163:0/1919819073 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4615 (secure 0 0 0) 0x7f9fcc017020 con 0x7f9fd8159420 2023-10-09T06:38:37.968 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:37.968 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":104,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6371,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3797578839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3797578839},{"type":"v1","addr":"172.21.15.163:6834","nonce":3797578839}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":6383,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2371721802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2371721802},{"type":"v1","addr":"172.21.15.181:6837","nonce":2371721802}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":7228,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2068214884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2068214884},{"type":"v1","addr":"172.21.15.181:6836","nonce":2068214884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:38:20.060175+0000","modified":"2023-10-09T06:38:23.693400+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7875},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7875":{"gid":7875,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/3794912550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3794912550},{"type":"v1","addr":"172.21.15.163:6835","nonce":3794912550}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-09T06:38:37.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.969+0000 7f9fddee1700 1 -- 172.21.15.163:0/1919819073 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9fbc067350 msgr2=0x7f9fbc069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:37.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.969+0000 7f9fddee1700 1 --2- 172.21.15.163:0/1919819073 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9fbc067350 0x7f9fbc069800 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9fc40093f0 tx=0x7f9fc4008040 comp rx=0 tx=0).stop 2023-10-09T06:38:37.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.969+0000 7f9fddee1700 1 -- 172.21.15.163:0/1919819073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fd8159420 msgr2=0x7f9fd81693c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:37.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.969+0000 7f9fddee1700 1 --2- 172.21.15.163:0/1919819073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fd8159420 0x7f9fd81693c0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0011e0 tx=0x7f9fcc007df0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.969+0000 7f9fddee1700 1 -- 172.21.15.163:0/1919819073 shutdown_connections 2023-10-09T06:38:37.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.969+0000 7f9fddee1700 1 --2- 172.21.15.163:0/1919819073 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9fbc067350 0x7f9fbc069800 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.969+0000 7f9fddee1700 1 --2- 172.21.15.163:0/1919819073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fd8159420 0x7f9fd81693c0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.969+0000 7f9fddee1700 1 --2- 172.21.15.163:0/1919819073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fd815d230 0x7f9fd815bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.969+0000 7f9fddee1700 1 --2- 172.21.15.163:0/1919819073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fd81597f0 0x7f9fd814e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:37.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.969+0000 7f9fddee1700 1 -- 172.21.15.163:0/1919819073 >> 172.21.15.163:0/1919819073 conn(0x7f9fd80d0660 msgr2=0x7f9fd8068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:37.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.969+0000 7f9fddee1700 1 -- 172.21.15.163:0/1919819073 shutdown_connections 2023-10-09T06:38:37.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:37.969+0000 7f9fddee1700 1 -- 172.21.15.163:0/1919819073 wait complete. 2023-10-09T06:38:37.971 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 104 2023-10-09T06:38:37.982 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:38:38.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.096+0000 7f5e52ed7700 1 Processor -- start 2023-10-09T06:38:38.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.096+0000 7f5e52ed7700 1 -- start start 2023-10-09T06:38:38.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.096+0000 7f5e52ed7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5e4c0c91e0 0x7f5e4c0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:38.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.096+0000 7f5e52ed7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e4c06ae10 0x7f5e4c06b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:38.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.096+0000 7f5e52ed7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e4c0cc210 con 0x7f5e4c06aa40 2023-10-09T06:38:38.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.096+0000 7f5e52ed7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e4c0cc4f0 con 0x7f5e4c0c91e0 2023-10-09T06:38:38.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.096+0000 7f5e52ed7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c0cc7d0 con 0x7f5e4c06ae10 2023-10-09T06:38:38.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.097+0000 7f5e4bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5e4c0c91e0 0x7f5e4c0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:38.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.097+0000 7f5e51474700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e4c06ae10 0x7f5e4c06b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:38.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.097+0000 7f5e4bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5e4c0c91e0 0x7f5e4c0cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:52052/0 (socket says 172.21.15.163:52052) 2023-10-09T06:38:38.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.097+0000 7f5e50c73700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f5e4c06aa40 0x7f5e4c06bac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:44024/0 (socket says 172.21.15.163:44024) 2023-10-09T06:38:38.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.097+0000 7f5e4bfff700 1 -- 172.21.15.163:0/3692357655 learned_addr learned my addr 172.21.15.163:0/3692357655 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:38.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.097+0000 7f5e4b7fe700 1 -- 172.21.15.163:0/3692357655 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 610686981 0 0) 0x7f5e4c0cc210 con 0x7f5e4c06aa40 2023-10-09T06:38:38.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.097+0000 7f5e4b7fe700 1 -- 172.21.15.163:0/3692357655 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e38003570 con 0x7f5e4c06aa40 2023-10-09T06:38:38.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.097+0000 7f5e51474700 1 -- 172.21.15.163:0/3692357655 >> v2:172.21.15.181:3300/0 conn(0x7f5e4c0c91e0 msgr2=0x7f5e4c0cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:38.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.097+0000 7f5e51474700 1 --2- 172.21.15.163:0/3692357655 >> v2:172.21.15.181:3300/0 conn(0x7f5e4c0c91e0 0x7f5e4c0cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:38.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.097+0000 7f5e51474700 1 -- 172.21.15.163:0/3692357655 >> v1:172.21.15.181:6789/0 conn(0x7f5e4c06aa40 legacy=0x7f5e4c06bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:38.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.097+0000 7f5e51474700 1 -- 172.21.15.163:0/3692357655 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4c0ccab0 con 0x7f5e4c06ae10 2023-10-09T06:38:38.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.097+0000 7f5e4bfff700 1 --2- 172.21.15.163:0/3692357655 >> v2:172.21.15.181:3300/0 conn(0x7f5e4c0c91e0 0x7f5e4c0cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:38:38.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.097+0000 7f5e51474700 1 --2- 172.21.15.163:0/3692357655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e4c06ae10 0x7f5e4c06b2b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5e44009910 tx=0x7f5e440091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecd70ec402bc55bb server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:38.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.098+0000 7f5e4b7fe700 1 -- 172.21.15.163:0/3692357655 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e44011630 con 0x7f5e4c06ae10 2023-10-09T06:38:38.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.098+0000 7f5e4b7fe700 1 -- 172.21.15.163:0/3692357655 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e44011790 con 0x7f5e4c06ae10 2023-10-09T06:38:38.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.098+0000 7f5e4b7fe700 1 -- 172.21.15.163:0/3692357655 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e44011b10 con 0x7f5e4c06ae10 2023-10-09T06:38:38.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.098+0000 7f5e52ed7700 1 -- 172.21.15.163:0/3692357655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e4c06ae10 msgr2=0x7f5e4c06b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:38.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.098+0000 7f5e52ed7700 1 --2- 172.21.15.163:0/3692357655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e4c06ae10 0x7f5e4c06b2b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5e44009910 tx=0x7f5e440091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:38.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.098+0000 7f5e52ed7700 1 -- 172.21.15.163:0/3692357655 shutdown_connections 2023-10-09T06:38:38.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.098+0000 7f5e52ed7700 1 --2- 172.21.15.163:0/3692357655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e4c06ae10 0x7f5e4c06b2b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:38.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.098+0000 7f5e52ed7700 1 --2- 172.21.15.163:0/3692357655 >> v2:172.21.15.181:3300/0 conn(0x7f5e4c0c91e0 0x7f5e4c0cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:38.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.098+0000 7f5e52ed7700 1 -- 172.21.15.163:0/3692357655 >> 172.21.15.163:0/3692357655 conn(0x7f5e4c0cd670 msgr2=0x7f5e4c0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:38.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.098+0000 7f5e52ed7700 1 -- 172.21.15.163:0/3692357655 shutdown_connections 2023-10-09T06:38:38.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.098+0000 7f5e52ed7700 1 -- 172.21.15.163:0/3692357655 wait complete. 2023-10-09T06:38:38.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.099+0000 7f5e52ed7700 1 Processor -- start 2023-10-09T06:38:38.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.099+0000 7f5e52ed7700 1 -- start start 2023-10-09T06:38:38.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.100+0000 7f5e52ed7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e4c06aa40 0x7f5e4c14ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:38.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.100+0000 7f5e52ed7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e4c06ae10 0x7f5e4c0b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:38.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.100+0000 7f5e52ed7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e4c0c91e0 0x7f5e4c16db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:38.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.100+0000 7f5e52ed7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c0b0e10 con 0x7f5e4c06aa40 2023-10-09T06:38:38.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.100+0000 7f5e52ed7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c0ae650 con 0x7f5e4c0c91e0 2023-10-09T06:38:38.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.100+0000 7f5e52ed7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c16e070 con 0x7f5e4c06ae10 2023-10-09T06:38:38.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.100+0000 7f5e50c73700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e4c06aa40 0x7f5e4c14ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:38.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.100+0000 7f5e50c73700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e4c06aa40 0x7f5e4c14ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47226/0 (socket says 172.21.15.163:47226) 2023-10-09T06:38:38.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.100+0000 7f5e50c73700 1 -- 172.21.15.163:0/116740861 learned_addr learned my addr 172.21.15.163:0/116740861 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:38.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.100+0000 7f5e4bfff700 1 --2- 172.21.15.163:0/116740861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e4c06ae10 0x7f5e4c0b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:38.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.100+0000 7f5e51474700 1 --2- 172.21.15.163:0/116740861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e4c0c91e0 0x7f5e4c16db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:38.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.101+0000 7f5e50c73700 1 -- 172.21.15.163:0/116740861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e4c06ae10 msgr2=0x7f5e4c0b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:38.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.101+0000 7f5e50c73700 1 --2- 172.21.15.163:0/116740861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e4c06ae10 0x7f5e4c0b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:38.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.101+0000 7f5e50c73700 1 -- 172.21.15.163:0/116740861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e4c0c91e0 msgr2=0x7f5e4c16db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:38.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.101+0000 7f5e50c73700 1 --2- 172.21.15.163:0/116740861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e4c0c91e0 0x7f5e4c16db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:38.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.101+0000 7f5e50c73700 1 -- 172.21.15.163:0/116740861 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e340094f0 con 0x7f5e4c06aa40 2023-10-09T06:38:38.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.101+0000 7f5e50c73700 1 --2- 172.21.15.163:0/116740861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e4c06aa40 0x7f5e4c14ee00 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00cc70 tx=0x7f5e3c001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:38.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.101+0000 7f5e48ff9700 1 -- 172.21.15.163:0/116740861 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c00a960 con 0x7f5e4c06aa40 2023-10-09T06:38:38.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.101+0000 7f5e48ff9700 1 -- 172.21.15.163:0/116740861 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3c00aac0 con 0x7f5e4c06aa40 2023-10-09T06:38:38.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.101+0000 7f5e52ed7700 1 -- 172.21.15.163:0/116740861 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e44009420 con 0x7f5e4c06aa40 2023-10-09T06:38:38.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.101+0000 7f5e48ff9700 1 -- 172.21.15.163:0/116740861 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c00ae80 con 0x7f5e4c06aa40 2023-10-09T06:38:38.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.101+0000 7f5e52ed7700 1 -- 172.21.15.163:0/116740861 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e4c14fa30 con 0x7f5e4c06aa40 2023-10-09T06:38:38.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.102+0000 7f5e52ed7700 1 -- 172.21.15.163:0/116740861 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e4c0ccd50 con 0x7f5e4c06aa40 2023-10-09T06:38:38.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.102+0000 7f5e48ff9700 1 -- 172.21.15.163:0/116740861 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5e3c007300 con 0x7f5e4c06aa40 2023-10-09T06:38:38.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.103+0000 7f5e48ff9700 1 --2- 172.21.15.163:0/116740861 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5e2c067320 0x7f5e2c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:38.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.103+0000 7f5e48ff9700 1 -- 172.21.15.163:0/116740861 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5e3c085120 con 0x7f5e4c06aa40 2023-10-09T06:38:38.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.103+0000 7f5e4bfff700 1 --2- 172.21.15.163:0/116740861 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5e2c067320 0x7f5e2c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:38.106 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.103+0000 7f5e4bfff700 1 --2- 172.21.15.163:0/116740861 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5e2c067320 0x7f5e2c0697d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c150e20 tx=0x7f5e34008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:38.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.107+0000 7f5e48ff9700 1 -- 172.21.15.163:0/116740861 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5e3c051e30 con 0x7f5e4c06aa40 2023-10-09T06:38:38.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.330+0000 7f5e52ed7700 1 -- 172.21.15.163:0/116740861 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e4c06de90 con 0x7f5e4c06aa40 2023-10-09T06:38:38.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.330+0000 7f5e48ff9700 1 -- 172.21.15.163:0/116740861 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4615 (secure 0 0 0) 0x7f5e3c055450 con 0x7f5e4c06aa40 2023-10-09T06:38:38.331 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:38.331 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":104,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6371,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3797578839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3797578839},{"type":"v1","addr":"172.21.15.163:6834","nonce":3797578839}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":6383,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2371721802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2371721802},{"type":"v1","addr":"172.21.15.181:6837","nonce":2371721802}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":7228,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2068214884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2068214884},{"type":"v1","addr":"172.21.15.181:6836","nonce":2068214884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:38:20.060175+0000","modified":"2023-10-09T06:38:23.693400+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7875},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7875":{"gid":7875,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/3794912550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3794912550},{"type":"v1","addr":"172.21.15.163:6835","nonce":3794912550}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-09T06:38:38.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.332+0000 7f5e52ed7700 1 -- 172.21.15.163:0/116740861 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5e2c067320 msgr2=0x7f5e2c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:38.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.332+0000 7f5e52ed7700 1 --2- 172.21.15.163:0/116740861 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5e2c067320 0x7f5e2c0697d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c150e20 tx=0x7f5e34008040 comp rx=0 tx=0).stop 2023-10-09T06:38:38.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.332+0000 7f5e52ed7700 1 -- 172.21.15.163:0/116740861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e4c06aa40 msgr2=0x7f5e4c14ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:38.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.332+0000 7f5e52ed7700 1 --2- 172.21.15.163:0/116740861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e4c06aa40 0x7f5e4c14ee00 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00cc70 tx=0x7f5e3c001ae0 comp rx=0 tx=0).stop 2023-10-09T06:38:38.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.332+0000 7f5e52ed7700 1 -- 172.21.15.163:0/116740861 shutdown_connections 2023-10-09T06:38:38.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.332+0000 7f5e52ed7700 1 --2- 172.21.15.163:0/116740861 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5e2c067320 0x7f5e2c0697d0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:38.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.332+0000 7f5e52ed7700 1 --2- 172.21.15.163:0/116740861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e4c06aa40 0x7f5e4c14ee00 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:38.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.332+0000 7f5e52ed7700 1 --2- 172.21.15.163:0/116740861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e4c0c91e0 0x7f5e4c16db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:38.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.332+0000 7f5e52ed7700 1 --2- 172.21.15.163:0/116740861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e4c06ae10 0x7f5e4c0b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:38.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.332+0000 7f5e52ed7700 1 -- 172.21.15.163:0/116740861 >> 172.21.15.163:0/116740861 conn(0x7f5e4c0cd670 msgr2=0x7f5e4c15ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:38.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.332+0000 7f5e52ed7700 1 -- 172.21.15.163:0/116740861 shutdown_connections 2023-10-09T06:38:38.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.332+0000 7f5e52ed7700 1 -- 172.21.15.163:0/116740861 wait complete. 2023-10-09T06:38:38.335 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 104 2023-10-09T06:38:38.346 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:38:38.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.460+0000 7f02d88bf700 1 Processor -- start 2023-10-09T06:38:38.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.460+0000 7f02d88bf700 1 -- start start 2023-10-09T06:38:38.463 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.461+0000 7f02d88bf700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f02d406aa40 0x7f02d40cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:38.463 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.461+0000 7f02d88bf700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d406bac0 0x7f02d406ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:38.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.461+0000 7f02d88bf700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02d40cc210 con 0x7f02d406b350 2023-10-09T06:38:38.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.461+0000 7f02d88bf700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02d40cc4f0 con 0x7f02d406aa40 2023-10-09T06:38:38.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.461+0000 7f02d88bf700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d40cc7d0 con 0x7f02d406bac0 2023-10-09T06:38:38.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.461+0000 7f02d259c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f02d406aa40 0x7f02d40cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:38.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.461+0000 7f02c9d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d406bac0 0x7f02d406ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:38.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.461+0000 7f02d259c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f02d406aa40 0x7f02d40cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:52070/0 (socket says 172.21.15.163:52070) 2023-10-09T06:38:38.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.461+0000 7f02d259c700 1 -- 172.21.15.163:0/220942104 learned_addr learned my addr 172.21.15.163:0/220942104 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:38.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.461+0000 7f02d2d9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f02d406b350 0x7f02d40c81d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:44030/0 (socket says 172.21.15.163:44030) 2023-10-09T06:38:38.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.462+0000 7f02d259c700 1 -- 172.21.15.163:0/220942104 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d406bac0 msgr2=0x7f02d406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:38.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.462+0000 7f02d259c700 1 --2- 172.21.15.163:0/220942104 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d406bac0 0x7f02d406ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:38.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.462+0000 7f02d259c700 1 -- 172.21.15.163:0/220942104 >> v1:172.21.15.181:6789/0 conn(0x7f02d406b350 legacy=0x7f02d40c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:38.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.462+0000 7f02d1d9b700 1 -- 172.21.15.163:0/220942104 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1153405074 0 0) 0x7f02d40cc210 con 0x7f02d406b350 2023-10-09T06:38:38.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.462+0000 7f02d259c700 1 -- 172.21.15.163:0/220942104 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d40ccab0 con 0x7f02d406aa40 2023-10-09T06:38:38.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.462+0000 7f02c9d9b700 1 --2- 172.21.15.163:0/220942104 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d406bac0 0x7f02d406ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:38:38.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.462+0000 7f02d259c700 1 --2- 172.21.15.163:0/220942104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02d406aa40 0x7f02d40cbc00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f02c4009990 tx=0x7f02c4009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c85b8e48c45b4a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:38.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.462+0000 7f02d1d9b700 1 -- 172.21.15.163:0/220942104 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c4009e80 con 0x7f02d406aa40 2023-10-09T06:38:38.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.462+0000 7f02d1d9b700 1 -- 172.21.15.163:0/220942104 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c4005c10 con 0x7f02d406aa40 2023-10-09T06:38:38.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.462+0000 7f02d1d9b700 1 -- 172.21.15.163:0/220942104 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c400b650 con 0x7f02d406aa40 2023-10-09T06:38:38.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.463+0000 7f02d88bf700 1 -- 172.21.15.163:0/220942104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02d406aa40 msgr2=0x7f02d40cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:38.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.463+0000 7f02d88bf700 1 --2- 172.21.15.163:0/220942104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02d406aa40 0x7f02d40cbc00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f02c4009990 tx=0x7f02c4009220 comp rx=0 tx=0).stop 2023-10-09T06:38:38.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.463+0000 7f02d88bf700 1 -- 172.21.15.163:0/220942104 shutdown_connections 2023-10-09T06:38:38.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.463+0000 7f02d88bf700 1 --2- 172.21.15.163:0/220942104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02d406aa40 0x7f02d40cbc00 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:38.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.463+0000 7f02d88bf700 1 --2- 172.21.15.163:0/220942104 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d406bac0 0x7f02d406ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:38.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.463+0000 7f02d88bf700 1 -- 172.21.15.163:0/220942104 >> 172.21.15.163:0/220942104 conn(0x7f02d40cd670 msgr2=0x7f02d40cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:38.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.463+0000 7f02d88bf700 1 -- 172.21.15.163:0/220942104 shutdown_connections 2023-10-09T06:38:38.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.463+0000 7f02d88bf700 1 -- 172.21.15.163:0/220942104 wait complete. 2023-10-09T06:38:38.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.464+0000 7f02d88bf700 1 Processor -- start 2023-10-09T06:38:38.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.464+0000 7f02d88bf700 1 -- start start 2023-10-09T06:38:38.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.465+0000 7f02d88bf700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02d406aa40 0x7f02d4142fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:38.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.465+0000 7f02d88bf700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d406b350 0x7f02d416ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:38.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.465+0000 7f02d88bf700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02d406bac0 0x7f02d415b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:38.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.465+0000 7f02d88bf700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d415ade0 con 0x7f02d406aa40 2023-10-09T06:38:38.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.465+0000 7f02d88bf700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d416b860 con 0x7f02d406bac0 2023-10-09T06:38:38.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.465+0000 7f02d88bf700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d416bb40 con 0x7f02d406b350 2023-10-09T06:38:38.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.465+0000 7f02d259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02d406aa40 0x7f02d4142fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:38.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.465+0000 7f02d259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02d406aa40 0x7f02d4142fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:47234/0 (socket says 172.21.15.163:47234) 2023-10-09T06:38:38.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.465+0000 7f02d259c700 1 -- 172.21.15.163:0/789651209 learned_addr learned my addr 172.21.15.163:0/789651209 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:38.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.465+0000 7f02d2d9d700 1 --2- 172.21.15.163:0/789651209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02d406bac0 0x7f02d415b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:38.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.465+0000 7f02c9d9b700 1 --2- 172.21.15.163:0/789651209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d406b350 0x7f02d416ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:38.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.465+0000 7f02d259c700 1 -- 172.21.15.163:0/789651209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d406b350 msgr2=0x7f02d416ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:38.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.465+0000 7f02d259c700 1 --2- 172.21.15.163:0/789651209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d406b350 0x7f02d416ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:38.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.465+0000 7f02d259c700 1 -- 172.21.15.163:0/789651209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02d406bac0 msgr2=0x7f02d415b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:38.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.465+0000 7f02d259c700 1 --2- 172.21.15.163:0/789651209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02d406bac0 0x7f02d415b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:38.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.465+0000 7f02d259c700 1 -- 172.21.15.163:0/789651209 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02bc009420 con 0x7f02d406aa40 2023-10-09T06:38:38.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.465+0000 7f02d259c700 1 --2- 172.21.15.163:0/789651209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02d406aa40 0x7f02d4142fb0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f02d40ce660 tx=0x7f02c4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:38.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.465+0000 7f02caffd700 1 -- 172.21.15.163:0/789651209 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c401db60 con 0x7f02d406aa40 2023-10-09T06:38:38.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.465+0000 7f02caffd700 1 -- 172.21.15.163:0/789651209 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c401dce0 con 0x7f02d406aa40 2023-10-09T06:38:38.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.465+0000 7f02d88bf700 1 -- 172.21.15.163:0/789651209 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c4009480 con 0x7f02d406aa40 2023-10-09T06:38:38.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.465+0000 7f02caffd700 1 -- 172.21.15.163:0/789651209 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c401b6d0 con 0x7f02d406aa40 2023-10-09T06:38:38.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.465+0000 7f02d88bf700 1 -- 172.21.15.163:0/789651209 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d415b320 con 0x7f02d406aa40 2023-10-09T06:38:38.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.466+0000 7f02d88bf700 1 -- 172.21.15.163:0/789651209 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02d404c3c0 con 0x7f02d406aa40 2023-10-09T06:38:38.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.467+0000 7f02caffd700 1 -- 172.21.15.163:0/789651209 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f02c401ba40 con 0x7f02d406aa40 2023-10-09T06:38:38.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.467+0000 7f02caffd700 1 --2- 172.21.15.163:0/789651209 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f02b4067320 0x7f02b40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:38.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.467+0000 7f02caffd700 1 -- 172.21.15.163:0/789651209 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f02c4086b40 con 0x7f02d406aa40 2023-10-09T06:38:38.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.467+0000 7f02c9d9b700 1 --2- 172.21.15.163:0/789651209 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f02b4067320 0x7f02b40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:38.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.468+0000 7f02c9d9b700 1 --2- 172.21.15.163:0/789651209 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f02b4067320 0x7f02b40697d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f02d40cc140 tx=0x7f02bc0091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:38.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.471+0000 7f02caffd700 1 -- 172.21.15.163:0/789651209 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02c4053850 con 0x7f02d406aa40 2023-10-09T06:38:38.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.655+0000 7f02d88bf700 1 -- 172.21.15.163:0/789651209 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02d40c7cb0 con 0x7f02d406aa40 2023-10-09T06:38:38.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.655+0000 7f02caffd700 1 -- 172.21.15.163:0/789651209 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) v1 ==== 74+0+15412 (secure 0 0 0) 0x7f02c4056e70 con 0x7f02d406aa40 2023-10-09T06:38:38.656 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:38.657 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":152,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:38:21.683060+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:38:18.381850+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-09T06:38:19.022637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:38:38.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.657+0000 7f02d88bf700 1 -- 172.21.15.163:0/789651209 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f02b4067320 msgr2=0x7f02b40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:38.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.657+0000 7f02d88bf700 1 --2- 172.21.15.163:0/789651209 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f02b4067320 0x7f02b40697d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f02d40cc140 tx=0x7f02bc0091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:38.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.657+0000 7f02d88bf700 1 -- 172.21.15.163:0/789651209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02d406aa40 msgr2=0x7f02d4142fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:38.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.657+0000 7f02d88bf700 1 --2- 172.21.15.163:0/789651209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02d406aa40 0x7f02d4142fb0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f02d40ce660 tx=0x7f02c4016000 comp rx=0 tx=0).stop 2023-10-09T06:38:38.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.657+0000 7f02d88bf700 1 -- 172.21.15.163:0/789651209 shutdown_connections 2023-10-09T06:38:38.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.657+0000 7f02d88bf700 1 --2- 172.21.15.163:0/789651209 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f02b4067320 0x7f02b40697d0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:38.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.657+0000 7f02d88bf700 1 --2- 172.21.15.163:0/789651209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02d406aa40 0x7f02d4142fb0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:38.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.657+0000 7f02d88bf700 1 --2- 172.21.15.163:0/789651209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02d406bac0 0x7f02d415b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:38.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.657+0000 7f02d88bf700 1 --2- 172.21.15.163:0/789651209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d406b350 0x7f02d416ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:38.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.657+0000 7f02d88bf700 1 -- 172.21.15.163:0/789651209 >> 172.21.15.163:0/789651209 conn(0x7f02d40cd670 msgr2=0x7f02d41522c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:38.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.658+0000 7f02d88bf700 1 -- 172.21.15.163:0/789651209 shutdown_connections 2023-10-09T06:38:38.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.658+0000 7f02d88bf700 1 -- 172.21.15.163:0/789651209 wait complete. 2023-10-09T06:38:38.671 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-09T06:38:38.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.787+0000 7f80a41df700 1 Processor -- start 2023-10-09T06:38:38.788 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.787+0000 7f80a41df700 1 -- start start 2023-10-09T06:38:38.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.788+0000 7f80a41df700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f809c159890 0x7f809c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:38.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.788+0000 7f80a41df700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f809c069560 0x7f809c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:38.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.788+0000 7f80a41df700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f809c069ee0 con 0x7f809c159c60 2023-10-09T06:38:38.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.788+0000 7f80a41df700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f809c15a160 con 0x7f809c159890 2023-10-09T06:38:38.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.788+0000 7f80a41df700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f809c0d13a0 con 0x7f809c069560 2023-10-09T06:38:38.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.788+0000 7f80a177a700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f809c159c60 0x7f809c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:56290/0 (socket says 172.21.15.163:56290) 2023-10-09T06:38:38.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.788+0000 7f80a177a700 1 -- 172.21.15.163:0/3055328953 learned_addr learned my addr 172.21.15.163:0/3055328953 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:38.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.788+0000 7f80a1f7b700 1 --2- 172.21.15.163:0/3055328953 >> v2:172.21.15.181:3300/0 conn(0x7f809c159890 0x7f809c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:38.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.788+0000 7f80a277c700 1 --2- 172.21.15.163:0/3055328953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f809c069560 0x7f809c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:38.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.788+0000 7f80a0f79700 1 -- 172.21.15.163:0/3055328953 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1205860683 0 0) 0x7f809c069ee0 con 0x7f809c159c60 2023-10-09T06:38:38.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.788+0000 7f80a0f79700 1 -- 172.21.15.163:0/3055328953 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8088003570 con 0x7f809c159c60 2023-10-09T06:38:38.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.789+0000 7f80a0f79700 1 -- 172.21.15.163:0/3055328953 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 449971595 0 0) 0x7f8088003570 con 0x7f809c159c60 2023-10-09T06:38:38.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.789+0000 7f80a0f79700 1 -- 172.21.15.163:0/3055328953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f809c069560 msgr2=0x7f809c0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:38.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.789+0000 7f80a0f79700 1 --2- 172.21.15.163:0/3055328953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f809c069560 0x7f809c0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:38.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.789+0000 7f80a0f79700 1 -- 172.21.15.163:0/3055328953 >> v2:172.21.15.181:3300/0 conn(0x7f809c159890 msgr2=0x7f809c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:38.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.789+0000 7f80a0f79700 1 --2- 172.21.15.163:0/3055328953 >> v2:172.21.15.181:3300/0 conn(0x7f809c159890 0x7f809c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:38.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.789+0000 7f80a0f79700 1 -- 172.21.15.163:0/3055328953 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809c15e140 con 0x7f809c159c60 2023-10-09T06:38:38.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.789+0000 7f80a0f79700 1 -- 172.21.15.163:0/3055328953 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f808c001470 con 0x7f809c159c60 2023-10-09T06:38:38.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.789+0000 7f80a0f79700 1 -- 172.21.15.163:0/3055328953 >> v1:172.21.15.163:6789/0 conn(0x7f809c159c60 legacy=0x7f809c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:38.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.789+0000 7f80a0f79700 1 --2- 172.21.15.163:0/3055328953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8088003a60 0x7f8088003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:38.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.789+0000 7f80a0f79700 1 --2- 172.21.15.163:0/3055328953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f809c069560 0x7f8088004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:38.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.789+0000 7f80a0f79700 1 --2- 172.21.15.163:0/3055328953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8088004d50 0x7f80880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:38.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.789+0000 7f80a0f79700 1 -- 172.21.15.163:0/3055328953 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c15a160 con 0x7f8088003a60 2023-10-09T06:38:38.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.789+0000 7f80a0f79700 1 -- 172.21.15.163:0/3055328953 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c0d13a0 con 0x7f8088004d50 2023-10-09T06:38:38.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.789+0000 7f80a0f79700 1 -- 172.21.15.163:0/3055328953 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808800ba60 con 0x7f809c069560 2023-10-09T06:38:38.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.789+0000 7f80a1f7b700 1 --2- 172.21.15.163:0/3055328953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8088003a60 0x7f8088003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:38.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.789+0000 7f80a177a700 1 --2- 172.21.15.163:0/3055328953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f809c069560 0x7f8088004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:38.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.789+0000 7f80a277c700 1 --2- 172.21.15.163:0/3055328953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8088004d50 0x7f80880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:38.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.789+0000 7f80a1f7b700 1 -- 172.21.15.163:0/3055328953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f809c069560 msgr2=0x7f8088004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:38.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.789+0000 7f80a1f7b700 1 --2- 172.21.15.163:0/3055328953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f809c069560 0x7f8088004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:38.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.789+0000 7f80a1f7b700 1 -- 172.21.15.163:0/3055328953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8088004d50 msgr2=0x7f80880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:38.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.789+0000 7f80a1f7b700 1 --2- 172.21.15.163:0/3055328953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8088004d50 0x7f80880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:38.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.789+0000 7f80a1f7b700 1 -- 172.21.15.163:0/3055328953 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8088003570 con 0x7f8088003a60 2023-10-09T06:38:38.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.789+0000 7f80a1f7b700 1 --2- 172.21.15.163:0/3055328953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8088003a60 0x7f8088003e70 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f8098009c00 tx=0x7f80980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0fca6aa7291798 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:38.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.789+0000 7f80a0f79700 1 -- 172.21.15.163:0/3055328953 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809800e6b0 con 0x7f8088003a60 2023-10-09T06:38:38.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.790+0000 7f80a0f79700 1 -- 172.21.15.163:0/3055328953 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809800e810 con 0x7f8088003a60 2023-10-09T06:38:38.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.790+0000 7f80a0f79700 1 -- 172.21.15.163:0/3055328953 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809800ebd0 con 0x7f8088003a60 2023-10-09T06:38:38.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.790+0000 7f80a41df700 1 -- 172.21.15.163:0/3055328953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8088003a60 msgr2=0x7f8088003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:38.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.790+0000 7f80a41df700 1 --2- 172.21.15.163:0/3055328953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8088003a60 0x7f8088003e70 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f8098009c00 tx=0x7f80980091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:38.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.790+0000 7f80a277c700 1 -- 172.21.15.163:0/3055328953 reap_dead start 2023-10-09T06:38:38.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.790+0000 7f80a41df700 1 -- 172.21.15.163:0/3055328953 shutdown_connections 2023-10-09T06:38:38.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.790+0000 7f80a41df700 1 -- 172.21.15.163:0/3055328953 >> 172.21.15.163:0/3055328953 conn(0x7f809c068e20 msgr2=0x7f809c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:38.798 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.790+0000 7f80a41df700 1 -- 172.21.15.163:0/3055328953 shutdown_connections 2023-10-09T06:38:38.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.790+0000 7f80a41df700 1 -- 172.21.15.163:0/3055328953 wait complete. 2023-10-09T06:38:38.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.791+0000 7f80a41df700 1 Processor -- start 2023-10-09T06:38:38.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.791+0000 7f80a41df700 1 -- start start 2023-10-09T06:38:38.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.792+0000 7f80a41df700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8088003a60 0x7f809c151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:38.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.792+0000 7f80a41df700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f809c16e1e0 0x7f809c14b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:38.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.792+0000 7f80a41df700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f809c15f920 0x7f809c15d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:38.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.792+0000 7f80a41df700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c151b40 con 0x7f8088003a60 2023-10-09T06:38:38.799 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.792+0000 7f80a41df700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c160220 con 0x7f809c16e1e0 2023-10-09T06:38:38.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.792+0000 7f80a41df700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f809c151210 con 0x7f809c15f920 2023-10-09T06:38:38.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.792+0000 7f80a1f7b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8088003a60 0x7f809c151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:38.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.792+0000 7f80a1f7b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8088003a60 0x7f809c151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38472/0 (socket says 172.21.15.163:38472) 2023-10-09T06:38:38.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.792+0000 7f80a1f7b700 1 -- 172.21.15.163:0/165192402 learned_addr learned my addr 172.21.15.163:0/165192402 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:38.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.792+0000 7f80a1f7b700 1 -- 172.21.15.163:0/165192402 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f809c15f920 msgr2=0x7f809c15d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:38.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.792+0000 7f80a1f7b700 1 --2- 172.21.15.163:0/165192402 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f809c15f920 0x7f809c15d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:38.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.792+0000 7f80a1f7b700 1 -- 172.21.15.163:0/165192402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f809c16e1e0 msgr2=0x7f809c14b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:38.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.792+0000 7f80a1f7b700 1 --2- 172.21.15.163:0/165192402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f809c16e1e0 0x7f809c14b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:38.800 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.792+0000 7f80a1f7b700 1 -- 172.21.15.163:0/165192402 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809c1514f0 con 0x7f8088003a60 2023-10-09T06:38:38.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.793+0000 7f80a1f7b700 1 --2- 172.21.15.163:0/165192402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8088003a60 0x7f809c151fd0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f8088004760 tx=0x7f809801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:38.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.793+0000 7f8091ffb700 1 -- 172.21.15.163:0/165192402 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8098022070 con 0x7f8088003a60 2023-10-09T06:38:38.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.793+0000 7f8091ffb700 1 -- 172.21.15.163:0/165192402 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809800e980 con 0x7f8088003a60 2023-10-09T06:38:38.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.793+0000 7f80a41df700 1 -- 172.21.15.163:0/165192402 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f809c151b40 con 0x7f8088003a60 2023-10-09T06:38:38.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.793+0000 7f8091ffb700 1 -- 172.21.15.163:0/165192402 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8098027470 con 0x7f8088003a60 2023-10-09T06:38:38.801 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.793+0000 7f80a41df700 1 -- 172.21.15.163:0/165192402 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f809c14b920 con 0x7f8088003a60 2023-10-09T06:38:38.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.794+0000 7f80a41df700 1 -- 172.21.15.163:0/165192402 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f809c151e20 con 0x7f8088003a60 2023-10-09T06:38:38.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.794+0000 7f8091ffb700 1 -- 172.21.15.163:0/165192402 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f809801c450 con 0x7f8088003a60 2023-10-09T06:38:38.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.794+0000 7f8091ffb700 1 --2- 172.21.15.163:0/165192402 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8080067320 0x7f80800697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:38.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.795+0000 7f8091ffb700 1 -- 172.21.15.163:0/165192402 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f80980867e0 con 0x7f8088003a60 2023-10-09T06:38:38.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.795+0000 7f80a177a700 1 --2- 172.21.15.163:0/165192402 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8080067320 0x7f80800697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:38.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.795+0000 7f80a177a700 1 --2- 172.21.15.163:0/165192402 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8080067320 0x7f80800697d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f808c001120 tx=0x7f808c001e90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:38.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:38.799+0000 7f8091ffb700 1 -- 172.21.15.163:0/165192402 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8098053530 con 0x7f8088003a60 2023-10-09T06:38:39.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.024+0000 7f80a41df700 1 -- 172.21.15.163:0/165192402 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f809c04c3c0 con 0x7f8088003a60 2023-10-09T06:38:39.025 INFO:tasks.ceph.mon.a.smithi163.stderr:2023-10-09T06:38:39.024+0000 7ff251c04700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-09T06:38:39.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.028+0000 7f8091ffb700 1 -- 172.21.15.163:0/165192402 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v105) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8098017090 con 0x7f8088003a60 2023-10-09T06:38:39.029 INFO:tasks.ceph.mds.a.smithi163.stderr: -706> 2023-10-09T06:38:25.969+0000 7fd90cfde700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-09T06:38:39.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.031+0000 7f80a41df700 1 -- 172.21.15.163:0/165192402 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8080067320 msgr2=0x7f80800697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:39.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.031+0000 7f80a41df700 1 --2- 172.21.15.163:0/165192402 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8080067320 0x7f80800697d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f808c001120 tx=0x7f808c001e90 comp rx=0 tx=0).stop 2023-10-09T06:38:39.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.031+0000 7f80a41df700 1 -- 172.21.15.163:0/165192402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8088003a60 msgr2=0x7f809c151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:39.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.031+0000 7f80a41df700 1 --2- 172.21.15.163:0/165192402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8088003a60 0x7f809c151fd0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f8088004760 tx=0x7f809801a000 comp rx=0 tx=0).stop 2023-10-09T06:38:39.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.031+0000 7f80a41df700 1 -- 172.21.15.163:0/165192402 shutdown_connections 2023-10-09T06:38:39.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.031+0000 7f80a41df700 1 --2- 172.21.15.163:0/165192402 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8080067320 0x7f80800697d0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:39.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.031+0000 7f80a41df700 1 --2- 172.21.15.163:0/165192402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8088003a60 0x7f809c151fd0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:39.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.031+0000 7f80a41df700 1 --2- 172.21.15.163:0/165192402 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f809c15f920 0x7f809c15d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:39.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.031+0000 7f80a41df700 1 --2- 172.21.15.163:0/165192402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f809c16e1e0 0x7f809c14b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:39.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.031+0000 7f80a41df700 1 -- 172.21.15.163:0/165192402 >> 172.21.15.163:0/165192402 conn(0x7f809c068e20 msgr2=0x7f809c0c68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:39.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.031+0000 7f80a41df700 1 -- 172.21.15.163:0/165192402 shutdown_connections 2023-10-09T06:38:39.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.031+0000 7f80a41df700 1 -- 172.21.15.163:0/165192402 wait complete. 2023-10-09T06:38:39.033 INFO:teuthology.orchestra.run.smithi163.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-09T06:38:39.044 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-09T06:38:39.049 INFO:tasks.ceph.mds.a.smithi163.stdout:starting mds.a at 2023-10-09T06:38:39.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.159+0000 7f7893bc5700 1 Processor -- start 2023-10-09T06:38:39.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.159+0000 7f7893bc5700 1 -- start start 2023-10-09T06:38:39.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.160+0000 7f7893bc5700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f788c0aca80 0x7f788c06aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:39.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.160+0000 7f7893bc5700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f788c069d60 0x7f788c0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:39.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.160+0000 7f7893bc5700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f788c06bdb0 con 0x7f788c0c81f0 2023-10-09T06:38:39.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.160+0000 7f7893bc5700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f788c0cb370 con 0x7f788c0aca80 2023-10-09T06:38:39.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.160+0000 7f7893bc5700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f788c159f60 con 0x7f788c069d60 2023-10-09T06:38:39.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.160+0000 7f7891961700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f788c0aca80 0x7f788c06aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:39.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.160+0000 7f7892162700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f788c0c81f0 0x7f788c068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:56294/0 (socket says 172.21.15.163:56294) 2023-10-09T06:38:39.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.160+0000 7f7892162700 1 -- 172.21.15.163:0/2594157249 learned_addr learned my addr 172.21.15.163:0/2594157249 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:39.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.160+0000 7f7891961700 1 -- 172.21.15.163:0/2594157249 >> v2:172.21.15.181:3300/0 conn(0x7f788c069d60 msgr2=0x7f788c0c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:39.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.160+0000 7f7891160700 1 --2- 172.21.15.163:0/2594157249 >> v2:172.21.15.181:3300/0 conn(0x7f788c069d60 0x7f788c0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:39.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.161+0000 7f7891961700 1 --2- 172.21.15.163:0/2594157249 >> v2:172.21.15.181:3300/0 conn(0x7f788c069d60 0x7f788c0c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:39.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.160+0000 7f789095f700 1 -- 172.21.15.163:0/2594157249 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2168255631 0 0) 0x7f788c06bdb0 con 0x7f788c0c81f0 2023-10-09T06:38:39.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.161+0000 7f7891961700 1 -- 172.21.15.163:0/2594157249 >> v1:172.21.15.163:6789/0 conn(0x7f788c0c81f0 legacy=0x7f788c068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:39.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.161+0000 7f7891961700 1 -- 172.21.15.163:0/2594157249 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788c0c8610 con 0x7f788c0aca80 2023-10-09T06:38:39.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.161+0000 7f7891961700 1 --2- 172.21.15.163:0/2594157249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f788c0aca80 0x7f788c06aba0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f788c06e540 tx=0x7f7888009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306c0f7c4b29e475 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:39.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.161+0000 7f789095f700 1 -- 172.21.15.163:0/2594157249 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7888005c10 con 0x7f788c0aca80 2023-10-09T06:38:39.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.161+0000 7f789095f700 1 -- 172.21.15.163:0/2594157249 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7888005d70 con 0x7f788c0aca80 2023-10-09T06:38:39.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.161+0000 7f789095f700 1 -- 172.21.15.163:0/2594157249 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7888006260 con 0x7f788c0aca80 2023-10-09T06:38:39.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.161+0000 7f7893bc5700 1 -- 172.21.15.163:0/2594157249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f788c0aca80 msgr2=0x7f788c06aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:39.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.161+0000 7f7893bc5700 1 --2- 172.21.15.163:0/2594157249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f788c0aca80 0x7f788c06aba0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f788c06e540 tx=0x7f7888009120 comp rx=0 tx=0).stop 2023-10-09T06:38:39.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.161+0000 7f7893bc5700 1 -- 172.21.15.163:0/2594157249 shutdown_connections 2023-10-09T06:38:39.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.161+0000 7f7893bc5700 1 --2- 172.21.15.163:0/2594157249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f788c0aca80 0x7f788c06aba0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:39.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.161+0000 7f7893bc5700 1 --2- 172.21.15.163:0/2594157249 >> v2:172.21.15.181:3300/0 conn(0x7f788c069d60 0x7f788c0c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:39.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.161+0000 7f7893bc5700 1 -- 172.21.15.163:0/2594157249 >> 172.21.15.163:0/2594157249 conn(0x7f788c0cd670 msgr2=0x7f788c0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:39.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.161+0000 7f7893bc5700 1 -- 172.21.15.163:0/2594157249 shutdown_connections 2023-10-09T06:38:39.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.161+0000 7f7893bc5700 1 -- 172.21.15.163:0/2594157249 wait complete. 2023-10-09T06:38:39.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.163+0000 7f7893bc5700 1 Processor -- start 2023-10-09T06:38:39.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.163+0000 7f7893bc5700 1 -- start start 2023-10-09T06:38:39.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.163+0000 7f7893bc5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f788c069d60 0x7f788c0b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:39.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.163+0000 7f7893bc5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f788c0aca80 0x7f788c1526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:39.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.163+0000 7f7893bc5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f788c0c81f0 0x7f788c0b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:39.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.163+0000 7f7893bc5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c152be0 con 0x7f788c0aca80 2023-10-09T06:38:39.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.163+0000 7f7893bc5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c0acf10 con 0x7f788c069d60 2023-10-09T06:38:39.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.163+0000 7f7893bc5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f788c0ad4d0 con 0x7f788c0c81f0 2023-10-09T06:38:39.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.163+0000 7f7891160700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f788c0aca80 0x7f788c1526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:39.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.163+0000 7f7891160700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f788c0aca80 0x7f788c1526a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38504/0 (socket says 172.21.15.163:38504) 2023-10-09T06:38:39.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.163+0000 7f7891160700 1 -- 172.21.15.163:0/3004700753 learned_addr learned my addr 172.21.15.163:0/3004700753 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:39.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.164+0000 7f7891961700 1 --2- 172.21.15.163:0/3004700753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f788c069d60 0x7f788c0b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:39.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.164+0000 7f7891160700 1 -- 172.21.15.163:0/3004700753 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f788c0c81f0 msgr2=0x7f788c0b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:39.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.164+0000 7f7892162700 1 --2- 172.21.15.163:0/3004700753 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f788c0c81f0 0x7f788c0b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:39.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.164+0000 7f7891160700 1 --2- 172.21.15.163:0/3004700753 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f788c0c81f0 0x7f788c0b1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:39.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.164+0000 7f7891160700 1 -- 172.21.15.163:0/3004700753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f788c069d60 msgr2=0x7f788c0b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:39.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.164+0000 7f7891160700 1 --2- 172.21.15.163:0/3004700753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f788c069d60 0x7f788c0b0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:39.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.164+0000 7f7891160700 1 -- 172.21.15.163:0/3004700753 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78880093c0 con 0x7f788c0aca80 2023-10-09T06:38:39.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.164+0000 7f7891961700 1 --2- 172.21.15.163:0/3004700753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f788c069d60 0x7f788c0b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:39.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.164+0000 7f7891160700 1 --2- 172.21.15.163:0/3004700753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f788c0aca80 0x7f788c1526a0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f787c00c8b0 tx=0x7f787c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:39.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.164+0000 7f7881ffb700 1 -- 172.21.15.163:0/3004700753 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c011600 con 0x7f788c0aca80 2023-10-09T06:38:39.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.164+0000 7f7881ffb700 1 -- 172.21.15.163:0/3004700753 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787c011760 con 0x7f788c0aca80 2023-10-09T06:38:39.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.164+0000 7f7893bc5700 1 -- 172.21.15.163:0/3004700753 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f788c15a6c0 con 0x7f788c0aca80 2023-10-09T06:38:39.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.164+0000 7f7881ffb700 1 -- 172.21.15.163:0/3004700753 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c011b00 con 0x7f788c0aca80 2023-10-09T06:38:39.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.164+0000 7f7893bc5700 1 -- 172.21.15.163:0/3004700753 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f788c0b16c0 con 0x7f788c0aca80 2023-10-09T06:38:39.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.165+0000 7f7893bc5700 1 -- 172.21.15.163:0/3004700753 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f788c04c3c0 con 0x7f788c0aca80 2023-10-09T06:38:39.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.166+0000 7f7881ffb700 1 -- 172.21.15.163:0/3004700753 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f787c010350 con 0x7f788c0aca80 2023-10-09T06:38:39.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.166+0000 7f7881ffb700 1 --2- 172.21.15.163:0/3004700753 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7870067320 0x7f78700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:39.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.166+0000 7f7881ffb700 1 -- 172.21.15.163:0/3004700753 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f787c084b50 con 0x7f788c0aca80 2023-10-09T06:38:39.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.166+0000 7f7891961700 1 --2- 172.21.15.163:0/3004700753 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7870067320 0x7f78700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:39.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.167+0000 7f7891961700 1 --2- 172.21.15.163:0/3004700753 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7870067320 0x7f78700697d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f788c0ac490 tx=0x7f7888016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:39.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.170+0000 7f7881ffb700 1 -- 172.21.15.163:0/3004700753 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f787c051680 con 0x7f788c0aca80 2023-10-09T06:38:39.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:39.395+0000 7f7893bc5700 1 -- 172.21.15.163:0/3004700753 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f788c06b510 con 0x7f788c0aca80 2023-10-09T06:38:40.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.028+0000 7f7881ffb700 1 -- 172.21.15.163:0/3004700753 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v106) v1 ==== 104+0+0 (secure 0 0 0) 0x7f787c0150a0 con 0x7f788c0aca80 2023-10-09T06:38:40.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.031+0000 7f7893bc5700 1 -- 172.21.15.163:0/3004700753 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7870067320 msgr2=0x7f78700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:40.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.031+0000 7f7893bc5700 1 --2- 172.21.15.163:0/3004700753 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7870067320 0x7f78700697d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f788c0ac490 tx=0x7f7888016000 comp rx=0 tx=0).stop 2023-10-09T06:38:40.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.031+0000 7f7893bc5700 1 -- 172.21.15.163:0/3004700753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f788c0aca80 msgr2=0x7f788c1526a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:40.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.031+0000 7f7893bc5700 1 --2- 172.21.15.163:0/3004700753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f788c0aca80 0x7f788c1526a0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f787c00c8b0 tx=0x7f787c008040 comp rx=0 tx=0).stop 2023-10-09T06:38:40.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.031+0000 7f7893bc5700 1 -- 172.21.15.163:0/3004700753 shutdown_connections 2023-10-09T06:38:40.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.031+0000 7f7893bc5700 1 --2- 172.21.15.163:0/3004700753 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7870067320 0x7f78700697d0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:40.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.031+0000 7f7893bc5700 1 --2- 172.21.15.163:0/3004700753 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f788c0c81f0 0x7f788c0b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:40.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.031+0000 7f7893bc5700 1 --2- 172.21.15.163:0/3004700753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f788c069d60 0x7f788c0b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:40.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.031+0000 7f7893bc5700 1 --2- 172.21.15.163:0/3004700753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f788c0aca80 0x7f788c1526a0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:40.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.031+0000 7f7893bc5700 1 -- 172.21.15.163:0/3004700753 >> 172.21.15.163:0/3004700753 conn(0x7f788c0cd670 msgr2=0x7f788c0c2620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:40.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.031+0000 7f7893bc5700 1 -- 172.21.15.163:0/3004700753 shutdown_connections 2023-10-09T06:38:40.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.031+0000 7f7893bc5700 1 -- 172.21.15.163:0/3004700753 wait complete. 2023-10-09T06:38:40.045 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-09T06:38:40.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.203+0000 7f4a0bfff700 1 Processor -- start 2023-10-09T06:38:40.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.203+0000 7f4a0bfff700 1 -- start start 2023-10-09T06:38:40.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.204+0000 7f4a0bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4a0c0b03c0 0x7f4a0c0cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:40.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.204+0000 7f4a0bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a0c1591e0 0x7f4a0c159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:40.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.204+0000 7f4a0bfff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a0c0cfa00 con 0x7f4a0c0b0790 2023-10-09T06:38:40.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.204+0000 7f4a0bfff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a0c0cb580 con 0x7f4a0c0b03c0 2023-10-09T06:38:40.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.204+0000 7f4a0bfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c0cf160 con 0x7f4a0c1591e0 2023-10-09T06:38:40.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.204+0000 7f4a0affd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4a0c0b03c0 0x7f4a0c0cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:40.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.204+0000 7f4a0a7fc700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f4a0c0b0790 0x7f4a0c0b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:59372/0 (socket says 172.21.15.163:59372) 2023-10-09T06:38:40.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.204+0000 7f4a0a7fc700 1 -- 172.21.15.163:0/1267597923 learned_addr learned my addr 172.21.15.163:0/1267597923 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:40.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.204+0000 7f4a0b7fe700 1 --2- 172.21.15.163:0/1267597923 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a0c1591e0 0x7f4a0c159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:40.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.205+0000 7f4a0affd700 1 -- 172.21.15.163:0/1267597923 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a0c1591e0 msgr2=0x7f4a0c159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:40.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.205+0000 7f4a0affd700 1 --2- 172.21.15.163:0/1267597923 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a0c1591e0 0x7f4a0c159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:40.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.205+0000 7f4a0affd700 1 -- 172.21.15.163:0/1267597923 >> v1:172.21.15.181:6789/0 conn(0x7f4a0c0b0790 legacy=0x7f4a0c0b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:40.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.205+0000 7f4a0affd700 1 -- 172.21.15.163:0/1267597923 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a0c0cf440 con 0x7f4a0c0b03c0 2023-10-09T06:38:40.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.205+0000 7f4a0affd700 1 --2- 172.21.15.163:0/1267597923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a0c0b03c0 0x7f4a0c0cea70 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009e90 tx=0x7f49fc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff7dbc11319929c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:40.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.205+0000 7f4a09ffb700 1 -- 172.21.15.163:0/1267597923 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc00e070 con 0x7f4a0c0b03c0 2023-10-09T06:38:40.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.205+0000 7f4a09ffb700 1 -- 172.21.15.163:0/1267597923 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49fc00bbf0 con 0x7f4a0c0b03c0 2023-10-09T06:38:40.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.205+0000 7f4a09ffb700 1 -- 172.21.15.163:0/1267597923 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc012430 con 0x7f4a0c0b03c0 2023-10-09T06:38:40.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.206+0000 7f4a0bfff700 1 -- 172.21.15.163:0/1267597923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a0c0b03c0 msgr2=0x7f4a0c0cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:40.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.206+0000 7f4a0bfff700 1 --2- 172.21.15.163:0/1267597923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a0c0b03c0 0x7f4a0c0cea70 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009e90 tx=0x7f49fc009780 comp rx=0 tx=0).stop 2023-10-09T06:38:40.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.206+0000 7f4a0bfff700 1 -- 172.21.15.163:0/1267597923 shutdown_connections 2023-10-09T06:38:40.211 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.206+0000 7f4a0bfff700 1 --2- 172.21.15.163:0/1267597923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a0c0b03c0 0x7f4a0c0cea70 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:40.211 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.206+0000 7f4a0bfff700 1 --2- 172.21.15.163:0/1267597923 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a0c1591e0 0x7f4a0c159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:40.211 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.206+0000 7f4a0bfff700 1 -- 172.21.15.163:0/1267597923 >> 172.21.15.163:0/1267597923 conn(0x7f4a0c063230 msgr2=0x7f4a0c063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:40.211 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.206+0000 7f4a0bfff700 1 -- 172.21.15.163:0/1267597923 shutdown_connections 2023-10-09T06:38:40.211 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.206+0000 7f4a0bfff700 1 -- 172.21.15.163:0/1267597923 wait complete. 2023-10-09T06:38:40.211 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.207+0000 7f4a0bfff700 1 Processor -- start 2023-10-09T06:38:40.211 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.207+0000 7f4a0bfff700 1 -- start start 2023-10-09T06:38:40.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.208+0000 7f4a0bfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a0c0b03c0 0x7f4a0c158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:40.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.208+0000 7f4a0bfff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a0c0b0790 0x7f4a0c0c59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:40.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.208+0000 7f4a0bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a0c1591e0 0x7f4a0c0c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:40.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.208+0000 7f4a0bfff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c15d1e0 con 0x7f4a0c1591e0 2023-10-09T06:38:40.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.208+0000 7f4a0bfff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c15d4c0 con 0x7f4a0c0b03c0 2023-10-09T06:38:40.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.208+0000 7f4a0bfff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c15d7a0 con 0x7f4a0c0b0790 2023-10-09T06:38:40.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.208+0000 7f4a0b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a0c1591e0 0x7f4a0c0c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:40.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.208+0000 7f4a0b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a0c1591e0 0x7f4a0c0c5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38510/0 (socket says 172.21.15.163:38510) 2023-10-09T06:38:40.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.208+0000 7f4a0b7fe700 1 -- 172.21.15.163:0/3196610994 learned_addr learned my addr 172.21.15.163:0/3196610994 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:40.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.208+0000 7f4a0affd700 1 --2- 172.21.15.163:0/3196610994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a0c0b03c0 0x7f4a0c158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:40.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.208+0000 7f4a0a7fc700 1 --2- 172.21.15.163:0/3196610994 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a0c0b0790 0x7f4a0c0c59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:40.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.208+0000 7f4a0b7fe700 1 -- 172.21.15.163:0/3196610994 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a0c0b0790 msgr2=0x7f4a0c0c59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:40.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.208+0000 7f4a0b7fe700 1 --2- 172.21.15.163:0/3196610994 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a0c0b0790 0x7f4a0c0c59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:40.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.208+0000 7f4a0b7fe700 1 -- 172.21.15.163:0/3196610994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a0c0b03c0 msgr2=0x7f4a0c158910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:40.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.208+0000 7f4a0b7fe700 1 --2- 172.21.15.163:0/3196610994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a0c0b03c0 0x7f4a0c158910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:40.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.208+0000 7f4a0b7fe700 1 -- 172.21.15.163:0/3196610994 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49fc009a20 con 0x7f4a0c1591e0 2023-10-09T06:38:40.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.209+0000 7f4a0b7fe700 1 --2- 172.21.15.163:0/3196610994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a0c1591e0 0x7f4a0c0c5ee0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400ea50 tx=0x7f4a040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:40.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.209+0000 7f49f2ffd700 1 -- 172.21.15.163:0/3196610994 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0400ee90 con 0x7f4a0c1591e0 2023-10-09T06:38:40.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.209+0000 7f49f2ffd700 1 -- 172.21.15.163:0/3196610994 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0400c850 con 0x7f4a0c1591e0 2023-10-09T06:38:40.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.209+0000 7f4a0bfff700 1 -- 172.21.15.163:0/3196610994 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a0c15da80 con 0x7f4a0c1591e0 2023-10-09T06:38:40.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.209+0000 7f49f2ffd700 1 -- 172.21.15.163:0/3196610994 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0400cc10 con 0x7f4a0c1591e0 2023-10-09T06:38:40.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.209+0000 7f4a0bfff700 1 -- 172.21.15.163:0/3196610994 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a0c158e50 con 0x7f4a0c1591e0 2023-10-09T06:38:40.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.210+0000 7f4a0bfff700 1 -- 172.21.15.163:0/3196610994 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a0c0c8c30 con 0x7f4a0c1591e0 2023-10-09T06:38:40.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.210+0000 7f49f2ffd700 1 -- 172.21.15.163:0/3196610994 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4a04013470 con 0x7f4a0c1591e0 2023-10-09T06:38:40.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.210+0000 7f49f2ffd700 1 --2- 172.21.15.163:0/3196610994 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f49f8067320 0x7f49f80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:40.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.211+0000 7f49f2ffd700 1 -- 172.21.15.163:0/3196610994 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f4a04084f40 con 0x7f4a0c1591e0 2023-10-09T06:38:40.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.211+0000 7f4a0affd700 1 --2- 172.21.15.163:0/3196610994 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f49f8067320 0x7f49f80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:40.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.211+0000 7f4a0affd700 1 --2- 172.21.15.163:0/3196610994 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f49f8067320 0x7f49f80697d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0097c0 tx=0x7f49fc019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:40.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.215+0000 7f49f2ffd700 1 -- 172.21.15.163:0/3196610994 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a0c0c8c30 con 0x7f4a0c1591e0 2023-10-09T06:38:40.400 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:40.399+0000 7f4a0bfff700 1 -- 172.21.15.163:0/3196610994 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4a0c0c8c30 con 0x7f4a0c1591e0 2023-10-09T06:38:41.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.034+0000 7f49f2ffd700 1 -- 172.21.15.163:0/3196610994 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v154) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4a0c0c8c30 con 0x7f4a0c1591e0 2023-10-09T06:38:41.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.037+0000 7f4a0bfff700 1 -- 172.21.15.163:0/3196610994 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f49f8067320 msgr2=0x7f49f80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:41.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.037+0000 7f4a0bfff700 1 --2- 172.21.15.163:0/3196610994 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f49f8067320 0x7f49f80697d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0097c0 tx=0x7f49fc019000 comp rx=0 tx=0).stop 2023-10-09T06:38:41.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.037+0000 7f4a0bfff700 1 -- 172.21.15.163:0/3196610994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a0c1591e0 msgr2=0x7f4a0c0c5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:41.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.037+0000 7f4a0bfff700 1 --2- 172.21.15.163:0/3196610994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a0c1591e0 0x7f4a0c0c5ee0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400ea50 tx=0x7f4a040091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:41.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.037+0000 7f4a0bfff700 1 -- 172.21.15.163:0/3196610994 shutdown_connections 2023-10-09T06:38:41.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.037+0000 7f4a0bfff700 1 --2- 172.21.15.163:0/3196610994 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f49f8067320 0x7f49f80697d0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:41.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.037+0000 7f4a0bfff700 1 --2- 172.21.15.163:0/3196610994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a0c1591e0 0x7f4a0c0c5ee0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:41.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.037+0000 7f4a0bfff700 1 --2- 172.21.15.163:0/3196610994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a0c0b03c0 0x7f4a0c158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:41.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.037+0000 7f4a0bfff700 1 --2- 172.21.15.163:0/3196610994 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a0c0b0790 0x7f4a0c0c59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:41.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.037+0000 7f4a0bfff700 1 -- 172.21.15.163:0/3196610994 >> 172.21.15.163:0/3196610994 conn(0x7f4a0c063230 msgr2=0x7f4a0c0ce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:41.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.037+0000 7f4a0bfff700 1 -- 172.21.15.163:0/3196610994 shutdown_connections 2023-10-09T06:38:41.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.037+0000 7f4a0bfff700 1 -- 172.21.15.163:0/3196610994 wait complete. 2023-10-09T06:38:41.039 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_metadata' removed 2023-10-09T06:38:41.051 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-09T06:38:41.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.174+0000 7f79c86be700 1 Processor -- start 2023-10-09T06:38:41.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.174+0000 7f79c86be700 1 -- start start 2023-10-09T06:38:41.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.175+0000 7f79c86be700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f79c0159b80 0x7f79c006a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:41.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.175+0000 7f79c86be700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f79c006a7f0 0x7f79c0163650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:41.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.176+0000 7f79c86be700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79c00af900 con 0x7f79c01597b0 2023-10-09T06:38:41.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.176+0000 7f79c86be700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79c0068390 con 0x7f79c006a7f0 2023-10-09T06:38:41.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.176+0000 7f79c86be700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79c00ac530 con 0x7f79c0159b80 2023-10-09T06:38:41.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.176+0000 7f79c6c5b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f79c006a7f0 0x7f79c0163650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:41.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.176+0000 7f79c645a700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f79c01597b0 0x7f79c00af1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:56306/0 (socket says 172.21.15.163:56306) 2023-10-09T06:38:41.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.176+0000 7f79c645a700 1 -- 172.21.15.163:0/2795241072 learned_addr learned my addr 172.21.15.163:0/2795241072 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:41.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.176+0000 7f79c5c59700 1 --2- 172.21.15.163:0/2795241072 >> v2:172.21.15.181:3300/0 conn(0x7f79c0159b80 0x7f79c006a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:41.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.176+0000 7f79c6c5b700 1 -- 172.21.15.163:0/2795241072 >> v2:172.21.15.181:3300/0 conn(0x7f79c0159b80 msgr2=0x7f79c006a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:41.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.176+0000 7f79c6c5b700 1 --2- 172.21.15.163:0/2795241072 >> v2:172.21.15.181:3300/0 conn(0x7f79c0159b80 0x7f79c006a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:41.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.176+0000 7f79c6c5b700 1 -- 172.21.15.163:0/2795241072 >> v1:172.21.15.163:6789/0 conn(0x7f79c01597b0 legacy=0x7f79c00af1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:41.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.176+0000 7f79c6c5b700 1 -- 172.21.15.163:0/2795241072 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c006ac30 con 0x7f79c006a7f0 2023-10-09T06:38:41.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.176+0000 7f79c5458700 1 -- 172.21.15.163:0/2795241072 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3523078616 0 0) 0x7f79c00af900 con 0x7f79c01597b0 2023-10-09T06:38:41.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.176+0000 7f79c6c5b700 1 --2- 172.21.15.163:0/2795241072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79c006a7f0 0x7f79c0163650 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f79b8009e90 tx=0x7f79b8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dff86d5eddcdc48 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:41.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.176+0000 7f79c5458700 1 -- 172.21.15.163:0/2795241072 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b800e070 con 0x7f79c006a7f0 2023-10-09T06:38:41.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.176+0000 7f79c5458700 1 -- 172.21.15.163:0/2795241072 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b800bbb0 con 0x7f79c006a7f0 2023-10-09T06:38:41.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.176+0000 7f79c5458700 1 -- 172.21.15.163:0/2795241072 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b8012430 con 0x7f79c006a7f0 2023-10-09T06:38:41.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.177+0000 7f79c86be700 1 -- 172.21.15.163:0/2795241072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79c006a7f0 msgr2=0x7f79c0163650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:41.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.177+0000 7f79c86be700 1 --2- 172.21.15.163:0/2795241072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79c006a7f0 0x7f79c0163650 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f79b8009e90 tx=0x7f79b8009780 comp rx=0 tx=0).stop 2023-10-09T06:38:41.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.178+0000 7f79c86be700 1 -- 172.21.15.163:0/2795241072 shutdown_connections 2023-10-09T06:38:41.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.178+0000 7f79c86be700 1 --2- 172.21.15.163:0/2795241072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79c006a7f0 0x7f79c0163650 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:41.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.178+0000 7f79c86be700 1 --2- 172.21.15.163:0/2795241072 >> v2:172.21.15.181:3300/0 conn(0x7f79c0159b80 0x7f79c006a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:41.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.178+0000 7f79c86be700 1 -- 172.21.15.163:0/2795241072 >> 172.21.15.163:0/2795241072 conn(0x7f79c00cd880 msgr2=0x7f79c00ae930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:41.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.178+0000 7f79c86be700 1 -- 172.21.15.163:0/2795241072 shutdown_connections 2023-10-09T06:38:41.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.178+0000 7f79c86be700 1 -- 172.21.15.163:0/2795241072 wait complete. 2023-10-09T06:38:41.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.179+0000 7f79c86be700 1 Processor -- start 2023-10-09T06:38:41.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.179+0000 7f79c86be700 1 -- start start 2023-10-09T06:38:41.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.180+0000 7f79c86be700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79c006a7f0 0x7f79c015e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:41.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.180+0000 7f79c86be700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79c01597b0 0x7f79c0156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:41.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.180+0000 7f79c86be700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79c0159b80 0x7f79c0169510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:41.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.180+0000 7f79c86be700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c015e540 con 0x7f79c006a7f0 2023-10-09T06:38:41.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.180+0000 7f79c86be700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c015d5b0 con 0x7f79c0159b80 2023-10-09T06:38:41.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.180+0000 7f79c86be700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c015db70 con 0x7f79c01597b0 2023-10-09T06:38:41.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.180+0000 7f79c645a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79c006a7f0 0x7f79c015e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:41.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.180+0000 7f79c645a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79c006a7f0 0x7f79c015e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38526/0 (socket says 172.21.15.163:38526) 2023-10-09T06:38:41.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.180+0000 7f79c645a700 1 -- 172.21.15.163:0/3290187219 learned_addr learned my addr 172.21.15.163:0/3290187219 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:41.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.180+0000 7f79c5c59700 1 --2- 172.21.15.163:0/3290187219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79c01597b0 0x7f79c0156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:41.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.180+0000 7f79c6c5b700 1 --2- 172.21.15.163:0/3290187219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79c0159b80 0x7f79c0169510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:41.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.180+0000 7f79c645a700 1 -- 172.21.15.163:0/3290187219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79c01597b0 msgr2=0x7f79c0156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:41.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.180+0000 7f79c645a700 1 --2- 172.21.15.163:0/3290187219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79c01597b0 0x7f79c0156f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:41.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.180+0000 7f79c645a700 1 -- 172.21.15.163:0/3290187219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79c0159b80 msgr2=0x7f79c0169510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:41.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.180+0000 7f79c645a700 1 --2- 172.21.15.163:0/3290187219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79c0159b80 0x7f79c0169510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:41.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.180+0000 7f79c645a700 1 -- 172.21.15.163:0/3290187219 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b8009a20 con 0x7f79c006a7f0 2023-10-09T06:38:41.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.180+0000 7f79c645a700 1 --2- 172.21.15.163:0/3290187219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79c006a7f0 0x7f79c015e000 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f79bc00c810 tx=0x7f79bc007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:41.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.181+0000 7f79b67fc700 1 -- 172.21.15.163:0/3290187219 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79bc005b30 con 0x7f79c006a7f0 2023-10-09T06:38:41.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.181+0000 7f79b67fc700 1 -- 172.21.15.163:0/3290187219 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79bc005c90 con 0x7f79c006a7f0 2023-10-09T06:38:41.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.181+0000 7f79c86be700 1 -- 172.21.15.163:0/3290187219 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c016a020 con 0x7f79c006a7f0 2023-10-09T06:38:41.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.181+0000 7f79b67fc700 1 -- 172.21.15.163:0/3290187219 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79bc007300 con 0x7f79c006a7f0 2023-10-09T06:38:41.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.181+0000 7f79c86be700 1 -- 172.21.15.163:0/3290187219 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c015d800 con 0x7f79c006a7f0 2023-10-09T06:38:41.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.182+0000 7f79c86be700 1 -- 172.21.15.163:0/3290187219 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79c015de50 con 0x7f79c006a7f0 2023-10-09T06:38:41.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.182+0000 7f79b67fc700 1 -- 172.21.15.163:0/3290187219 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f79bc01b3b0 con 0x7f79c006a7f0 2023-10-09T06:38:41.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.182+0000 7f79b67fc700 1 --2- 172.21.15.163:0/3290187219 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f79a4067320 0x7f79a40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:41.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.183+0000 7f79b67fc700 1 -- 172.21.15.163:0/3290187219 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f79bc084010 con 0x7f79c006a7f0 2023-10-09T06:38:41.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.183+0000 7f79c5c59700 1 --2- 172.21.15.163:0/3290187219 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f79a4067320 0x7f79a40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:41.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.183+0000 7f79c5c59700 1 --2- 172.21.15.163:0/3290187219 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f79a4067320 0x7f79a40697d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f79c015d540 tx=0x7f79b00091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:41.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.187+0000 7f79b67fc700 1 -- 172.21.15.163:0/3290187219 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f79bc050e00 con 0x7f79c006a7f0 2023-10-09T06:38:41.371 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:41.370+0000 7f79c86be700 1 -- 172.21.15.163:0/3290187219 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f79c00718b0 con 0x7f79c006a7f0 2023-10-09T06:38:42.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.038+0000 7f79b67fc700 1 -- 172.21.15.163:0/3290187219 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v155) v1 ==== 169+0+0 (secure 0 0 0) 0x7f79bc017020 con 0x7f79c006a7f0 2023-10-09T06:38:42.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.041+0000 7f79c86be700 1 -- 172.21.15.163:0/3290187219 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f79a4067320 msgr2=0x7f79a40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:42.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.041+0000 7f79c86be700 1 --2- 172.21.15.163:0/3290187219 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f79a4067320 0x7f79a40697d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f79c015d540 tx=0x7f79b00091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:42.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.041+0000 7f79c86be700 1 -- 172.21.15.163:0/3290187219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79c006a7f0 msgr2=0x7f79c015e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:42.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.041+0000 7f79c86be700 1 --2- 172.21.15.163:0/3290187219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79c006a7f0 0x7f79c015e000 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f79bc00c810 tx=0x7f79bc007df0 comp rx=0 tx=0).stop 2023-10-09T06:38:42.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.041+0000 7f79c86be700 1 -- 172.21.15.163:0/3290187219 shutdown_connections 2023-10-09T06:38:42.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.041+0000 7f79c86be700 1 --2- 172.21.15.163:0/3290187219 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f79a4067320 0x7f79a40697d0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:42.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.041+0000 7f79c86be700 1 --2- 172.21.15.163:0/3290187219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79c006a7f0 0x7f79c015e000 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:42.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.041+0000 7f79c86be700 1 --2- 172.21.15.163:0/3290187219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79c0159b80 0x7f79c0169510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:42.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.041+0000 7f79c86be700 1 --2- 172.21.15.163:0/3290187219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79c01597b0 0x7f79c0156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:42.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.042+0000 7f79c86be700 1 -- 172.21.15.163:0/3290187219 >> 172.21.15.163:0/3290187219 conn(0x7f79c00cd880 msgr2=0x7f79c00afcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:42.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.042+0000 7f79c86be700 1 -- 172.21.15.163:0/3290187219 shutdown_connections 2023-10-09T06:38:42.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.042+0000 7f79c86be700 1 -- 172.21.15.163:0/3290187219 wait complete. 2023-10-09T06:38:42.044 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_data' removed 2023-10-09T06:38:42.056 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-09T06:38:42.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.172+0000 7f0ad4ba9700 1 Processor -- start 2023-10-09T06:38:42.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.172+0000 7f0ad4ba9700 1 -- start start 2023-10-09T06:38:42.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.173+0000 7f0ad4ba9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0ad0140720 0x7f0ad0068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:42.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.173+0000 7f0ad4ba9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0ad00688d0 0x7f0ad00c8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:42.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.173+0000 7f0ad4ba9700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ad0068d30 con 0x7f0ad0140350 2023-10-09T06:38:42.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.173+0000 7f0ad4ba9700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ad00c7de0 con 0x7f0ad00688d0 2023-10-09T06:38:42.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.173+0000 7f0ad4ba9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ad0140c10 con 0x7f0ad0140720 2023-10-09T06:38:42.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.173+0000 7f0ace59c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f0ad0140350 0x7f0ad0069c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:56322/0 (socket says 172.21.15.163:56322) 2023-10-09T06:38:42.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.173+0000 7f0ace59c700 1 -- 172.21.15.163:0/2482893990 learned_addr learned my addr 172.21.15.163:0/2482893990 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:42.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.173+0000 7f0aced9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0ad00688d0 0x7f0ad00c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:42.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.173+0000 7f0ac7fff700 1 --2- 172.21.15.163:0/2482893990 >> v2:172.21.15.181:3300/0 conn(0x7f0ad0140720 0x7f0ad0068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:42.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.173+0000 7f0aced9d700 1 -- 172.21.15.163:0/2482893990 >> v2:172.21.15.181:3300/0 conn(0x7f0ad0140720 msgr2=0x7f0ad0068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:42.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.173+0000 7f0aced9d700 1 --2- 172.21.15.163:0/2482893990 >> v2:172.21.15.181:3300/0 conn(0x7f0ad0140720 0x7f0ad0068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:42.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.173+0000 7f0aced9d700 1 -- 172.21.15.163:0/2482893990 >> v1:172.21.15.163:6789/0 conn(0x7f0ad0140350 legacy=0x7f0ad0069c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:42.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.173+0000 7f0aced9d700 1 -- 172.21.15.163:0/2482893990 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad0141490 con 0x7f0ad00688d0 2023-10-09T06:38:42.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.173+0000 7f0acdd9b700 1 -- 172.21.15.163:0/2482893990 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2852196592 0 0) 0x7f0ac0001290 con 0x7f0ad0140350 2023-10-09T06:38:42.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.173+0000 7f0ac7fff700 1 --2- 172.21.15.163:0/2482893990 >> v2:172.21.15.181:3300/0 conn(0x7f0ad0140720 0x7f0ad0068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:42.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.173+0000 7f0aced9d700 1 --2- 172.21.15.163:0/2482893990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ad00688d0 0x7f0ad00c8270 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f0ad006f770 tx=0x7f0ac8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1f247a10aca001 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:42.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.173+0000 7f0acdd9b700 1 -- 172.21.15.163:0/2482893990 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac800e070 con 0x7f0ad00688d0 2023-10-09T06:38:42.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.173+0000 7f0acdd9b700 1 -- 172.21.15.163:0/2482893990 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac800bc30 con 0x7f0ad00688d0 2023-10-09T06:38:42.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.173+0000 7f0acdd9b700 1 -- 172.21.15.163:0/2482893990 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac8012430 con 0x7f0ad00688d0 2023-10-09T06:38:42.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.174+0000 7f0ad4ba9700 1 -- 172.21.15.163:0/2482893990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ad00688d0 msgr2=0x7f0ad00c8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:42.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.174+0000 7f0ad4ba9700 1 --2- 172.21.15.163:0/2482893990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ad00688d0 0x7f0ad00c8270 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f0ad006f770 tx=0x7f0ac8009780 comp rx=0 tx=0).stop 2023-10-09T06:38:42.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.174+0000 7f0ad4ba9700 1 -- 172.21.15.163:0/2482893990 shutdown_connections 2023-10-09T06:38:42.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.174+0000 7f0ad4ba9700 1 --2- 172.21.15.163:0/2482893990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ad00688d0 0x7f0ad00c8270 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:42.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.174+0000 7f0ad4ba9700 1 --2- 172.21.15.163:0/2482893990 >> v2:172.21.15.181:3300/0 conn(0x7f0ad0140720 0x7f0ad0068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:42.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.174+0000 7f0ad4ba9700 1 -- 172.21.15.163:0/2482893990 >> 172.21.15.163:0/2482893990 conn(0x7f0ad00cd670 msgr2=0x7f0ad00cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:42.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.174+0000 7f0ad4ba9700 1 -- 172.21.15.163:0/2482893990 shutdown_connections 2023-10-09T06:38:42.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.174+0000 7f0ad4ba9700 1 -- 172.21.15.163:0/2482893990 wait complete. 2023-10-09T06:38:42.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.175+0000 7f0ad4ba9700 1 Processor -- start 2023-10-09T06:38:42.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.175+0000 7f0ad4ba9700 1 -- start start 2023-10-09T06:38:42.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.176+0000 7f0ad4ba9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ad00688d0 0x7f0ad00b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:42.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.176+0000 7f0ad4ba9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ad0140350 0x7f0ad00b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:42.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.176+0000 7f0ad4ba9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ad0140720 0x7f0ad00cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:42.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.176+0000 7f0ad4ba9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad00cb9f0 con 0x7f0ad0140350 2023-10-09T06:38:42.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.176+0000 7f0ad4ba9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad00b0950 con 0x7f0ad0140720 2023-10-09T06:38:42.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.176+0000 7f0ad4ba9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad00cb3e0 con 0x7f0ad00688d0 2023-10-09T06:38:42.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.176+0000 7f0ac7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ad0140350 0x7f0ad00b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:42.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.176+0000 7f0ac7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ad0140350 0x7f0ad00b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38538/0 (socket says 172.21.15.163:38538) 2023-10-09T06:38:42.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.176+0000 7f0ac7fff700 1 -- 172.21.15.163:0/2197403296 learned_addr learned my addr 172.21.15.163:0/2197403296 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:42.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.176+0000 7f0ace59c700 1 --2- 172.21.15.163:0/2197403296 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ad00688d0 0x7f0ad00b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:42.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.176+0000 7f0aced9d700 1 --2- 172.21.15.163:0/2197403296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ad0140720 0x7f0ad00cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:42.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.176+0000 7f0ac7fff700 1 -- 172.21.15.163:0/2197403296 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ad00688d0 msgr2=0x7f0ad00b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:42.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.176+0000 7f0ac7fff700 1 --2- 172.21.15.163:0/2197403296 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ad00688d0 0x7f0ad00b2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:42.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.176+0000 7f0ac7fff700 1 -- 172.21.15.163:0/2197403296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ad0140720 msgr2=0x7f0ad00cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:42.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.176+0000 7f0ac7fff700 1 --2- 172.21.15.163:0/2197403296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ad0140720 0x7f0ad00cdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:42.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.176+0000 7f0ac7fff700 1 -- 172.21.15.163:0/2197403296 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac8009a20 con 0x7f0ad0140350 2023-10-09T06:38:42.194 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.176+0000 7f0ac7fff700 1 --2- 172.21.15.163:0/2197403296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ad0140350 0x7f0ad00b0410 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800c850 tx=0x7f0ab8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:42.194 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.177+0000 7f0ac67fc700 1 -- 172.21.15.163:0/2197403296 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8011600 con 0x7f0ad0140350 2023-10-09T06:38:42.194 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.177+0000 7f0ac67fc700 1 -- 172.21.15.163:0/2197403296 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab8011760 con 0x7f0ad0140350 2023-10-09T06:38:42.194 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.177+0000 7f0ad4ba9700 1 -- 172.21.15.163:0/2197403296 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ad00b0eb0 con 0x7f0ad0140350 2023-10-09T06:38:42.194 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.177+0000 7f0ac67fc700 1 -- 172.21.15.163:0/2197403296 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8011b00 con 0x7f0ad0140350 2023-10-09T06:38:42.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.177+0000 7f0ad4ba9700 1 -- 172.21.15.163:0/2197403296 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ad00cb680 con 0x7f0ad0140350 2023-10-09T06:38:42.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.177+0000 7f0ad4ba9700 1 -- 172.21.15.163:0/2197403296 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ad004c3c0 con 0x7f0ad0140350 2023-10-09T06:38:42.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.178+0000 7f0ac67fc700 1 -- 172.21.15.163:0/2197403296 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0ab80073b0 con 0x7f0ad0140350 2023-10-09T06:38:42.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.178+0000 7f0ac67fc700 1 --2- 172.21.15.163:0/2197403296 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0ab0067350 0x7f0ab0069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:42.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.178+0000 7f0ac67fc700 1 -- 172.21.15.163:0/2197403296 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0ab80849f0 con 0x7f0ad0140350 2023-10-09T06:38:42.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.179+0000 7f0ace59c700 1 --2- 172.21.15.163:0/2197403296 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0ab0067350 0x7f0ab0069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:42.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.179+0000 7f0ace59c700 1 --2- 172.21.15.163:0/2197403296 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0ab0067350 0x7f0ab0069800 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0002890 tx=0x7f0ac0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:42.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.183+0000 7f0ac67fc700 1 -- 172.21.15.163:0/2197403296 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0ab8051870 con 0x7f0ad0140350 2023-10-09T06:38:42.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.368+0000 7f0ad4ba9700 1 -- 172.21.15.163:0/2197403296 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0ad0140ed0 con 0x7f0ad0140350 2023-10-09T06:38:42.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.371+0000 7f0ac67fc700 1 -- 172.21.15.163:0/2197403296 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0ab8054e90 con 0x7f0ad0140350 2023-10-09T06:38:42.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.373+0000 7f0ad4ba9700 1 -- 172.21.15.163:0/2197403296 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0ab0067350 msgr2=0x7f0ab0069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:42.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.373+0000 7f0ad4ba9700 1 --2- 172.21.15.163:0/2197403296 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0ab0067350 0x7f0ab0069800 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0002890 tx=0x7f0ac0009040 comp rx=0 tx=0).stop 2023-10-09T06:38:42.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.373+0000 7f0ad4ba9700 1 -- 172.21.15.163:0/2197403296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ad0140350 msgr2=0x7f0ad00b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:42.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.373+0000 7f0ad4ba9700 1 --2- 172.21.15.163:0/2197403296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ad0140350 0x7f0ad00b0410 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800c850 tx=0x7f0ab8008040 comp rx=0 tx=0).stop 2023-10-09T06:38:42.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.374+0000 7f0ad4ba9700 1 -- 172.21.15.163:0/2197403296 shutdown_connections 2023-10-09T06:38:42.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.374+0000 7f0ad4ba9700 1 --2- 172.21.15.163:0/2197403296 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0ab0067350 0x7f0ab0069800 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:42.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.374+0000 7f0ad4ba9700 1 --2- 172.21.15.163:0/2197403296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ad0140720 0x7f0ad00cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:42.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.374+0000 7f0ad4ba9700 1 --2- 172.21.15.163:0/2197403296 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ad00688d0 0x7f0ad00b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:42.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.374+0000 7f0ad4ba9700 1 --2- 172.21.15.163:0/2197403296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ad0140350 0x7f0ad00b0410 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:42.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.374+0000 7f0ad4ba9700 1 -- 172.21.15.163:0/2197403296 >> 172.21.15.163:0/2197403296 conn(0x7f0ad00cd670 msgr2=0x7f0ad00c2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:42.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.374+0000 7f0ad4ba9700 1 -- 172.21.15.163:0/2197403296 shutdown_connections 2023-10-09T06:38:42.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.374+0000 7f0ad4ba9700 1 -- 172.21.15.163:0/2197403296 wait complete. 2023-10-09T06:38:42.390 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag' 2023-10-09T06:38:42.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.504+0000 7fb89e271700 1 Processor -- start 2023-10-09T06:38:42.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.504+0000 7fb89e271700 1 -- start start 2023-10-09T06:38:42.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.505+0000 7fb89e271700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb898159890 0x7fb89815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:42.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.505+0000 7fb89e271700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb898159c60 0x7fb89815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:42.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.505+0000 7fb89e271700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb89815a160 con 0x7fb89815db10 2023-10-09T06:38:42.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.505+0000 7fb89e271700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8980d13a0 con 0x7fb898159890 2023-10-09T06:38:42.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.505+0000 7fb89e271700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb89815e140 con 0x7fb898159c60 2023-10-09T06:38:42.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.505+0000 7fb8977fe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb898159c60 0x7fb89815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:42.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.505+0000 7fb897fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb898159890 0x7fb89815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:42.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.505+0000 7fb89c80e700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fb89815db10 0x7fb898069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:59378/0 (socket says 172.21.15.163:59378) 2023-10-09T06:38:42.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.505+0000 7fb89c80e700 1 -- 172.21.15.163:0/2190488499 learned_addr learned my addr 172.21.15.163:0/2190488499 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:42.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.505+0000 7fb896ffd700 1 -- 172.21.15.163:0/2190488499 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1928239581 0 0) 0x7fb89815a160 con 0x7fb89815db10 2023-10-09T06:38:42.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.505+0000 7fb896ffd700 1 -- 172.21.15.163:0/2190488499 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb880003570 con 0x7fb89815db10 2023-10-09T06:38:42.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.506+0000 7fb897fff700 1 -- 172.21.15.163:0/2190488499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb898159c60 msgr2=0x7fb89815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:42.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.506+0000 7fb897fff700 1 --2- 172.21.15.163:0/2190488499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb898159c60 0x7fb89815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:42.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.506+0000 7fb897fff700 1 -- 172.21.15.163:0/2190488499 >> v1:172.21.15.181:6789/0 conn(0x7fb89815db10 legacy=0x7fb898069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:42.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.506+0000 7fb897fff700 1 -- 172.21.15.163:0/2190488499 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb89815a3d0 con 0x7fb898159890 2023-10-09T06:38:42.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.506+0000 7fb8977fe700 1 --2- 172.21.15.163:0/2190488499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb898159c60 0x7fb89815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:38:42.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.506+0000 7fb897fff700 1 --2- 172.21.15.163:0/2190488499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb898159890 0x7fb89815abd0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb8880098e0 tx=0x7fb8880091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f2fc83a774738a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:42.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.506+0000 7fb896ffd700 1 -- 172.21.15.163:0/2190488499 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb888011630 con 0x7fb898159890 2023-10-09T06:38:42.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.506+0000 7fb896ffd700 1 -- 172.21.15.163:0/2190488499 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb888011790 con 0x7fb898159890 2023-10-09T06:38:42.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.506+0000 7fb896ffd700 1 -- 172.21.15.163:0/2190488499 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb888011b30 con 0x7fb898159890 2023-10-09T06:38:42.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.506+0000 7fb89e271700 1 -- 172.21.15.163:0/2190488499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb898159890 msgr2=0x7fb89815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:42.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.506+0000 7fb89e271700 1 --2- 172.21.15.163:0/2190488499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb898159890 0x7fb89815abd0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb8880098e0 tx=0x7fb8880091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:42.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.506+0000 7fb89e271700 1 -- 172.21.15.163:0/2190488499 shutdown_connections 2023-10-09T06:38:42.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.506+0000 7fb89e271700 1 --2- 172.21.15.163:0/2190488499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb898159890 0x7fb89815abd0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:42.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.506+0000 7fb89e271700 1 --2- 172.21.15.163:0/2190488499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb898159c60 0x7fb89815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:42.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.506+0000 7fb89e271700 1 -- 172.21.15.163:0/2190488499 >> 172.21.15.163:0/2190488499 conn(0x7fb898068e20 msgr2=0x7fb8980d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:42.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.506+0000 7fb89e271700 1 -- 172.21.15.163:0/2190488499 shutdown_connections 2023-10-09T06:38:42.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.506+0000 7fb89e271700 1 -- 172.21.15.163:0/2190488499 wait complete. 2023-10-09T06:38:42.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.507+0000 7fb89e271700 1 Processor -- start 2023-10-09T06:38:42.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.508+0000 7fb89e271700 1 -- start start 2023-10-09T06:38:42.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.508+0000 7fb89e271700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb898159890 0x7fb898153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:42.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.508+0000 7fb89e271700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb898159c60 0x7fb898153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:42.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.508+0000 7fb89e271700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb89815db10 0x7fb8980cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:42.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.508+0000 7fb89e271700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8980cee50 con 0x7fb898159890 2023-10-09T06:38:42.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.508+0000 7fb89e271700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8980ce9c0 con 0x7fb89815db10 2023-10-09T06:38:42.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.508+0000 7fb89e271700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8980cddb0 con 0x7fb898159c60 2023-10-09T06:38:42.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.508+0000 7fb897fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb898159890 0x7fb898153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:42.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.508+0000 7fb897fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb898159890 0x7fb898153a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38552/0 (socket says 172.21.15.163:38552) 2023-10-09T06:38:42.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.508+0000 7fb897fff700 1 -- 172.21.15.163:0/2866718002 learned_addr learned my addr 172.21.15.163:0/2866718002 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:42.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.509+0000 7fb89c80e700 1 --2- 172.21.15.163:0/2866718002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb89815db10 0x7fb8980cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:42.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.509+0000 7fb8977fe700 1 --2- 172.21.15.163:0/2866718002 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb898159c60 0x7fb898153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:42.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.509+0000 7fb897fff700 1 -- 172.21.15.163:0/2866718002 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb898159c60 msgr2=0x7fb898153f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:42.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.509+0000 7fb897fff700 1 --2- 172.21.15.163:0/2866718002 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb898159c60 0x7fb898153f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:42.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.509+0000 7fb897fff700 1 -- 172.21.15.163:0/2866718002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb89815db10 msgr2=0x7fb8980cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:42.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.509+0000 7fb897fff700 1 --2- 172.21.15.163:0/2866718002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb89815db10 0x7fb8980cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:42.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.509+0000 7fb897fff700 1 -- 172.21.15.163:0/2866718002 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88c009420 con 0x7fb898159890 2023-10-09T06:38:42.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.509+0000 7fb8977fe700 1 --2- 172.21.15.163:0/2866718002 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb898159c60 0x7fb898153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:42.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.509+0000 7fb897fff700 1 --2- 172.21.15.163:0/2866718002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb898159890 0x7fb898153a00 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fb888009ed0 tx=0x7fb888017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:42.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.509+0000 7fb87ffff700 1 -- 172.21.15.163:0/2866718002 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb888000d10 con 0x7fb898159890 2023-10-09T06:38:42.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.509+0000 7fb87ffff700 1 -- 172.21.15.163:0/2866718002 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb888011b10 con 0x7fb898159890 2023-10-09T06:38:42.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.509+0000 7fb89e271700 1 -- 172.21.15.163:0/2866718002 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb888009420 con 0x7fb898159890 2023-10-09T06:38:42.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.509+0000 7fb87ffff700 1 -- 172.21.15.163:0/2866718002 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb888025640 con 0x7fb898159890 2023-10-09T06:38:42.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.509+0000 7fb89e271700 1 -- 172.21.15.163:0/2866718002 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8980ce370 con 0x7fb898159890 2023-10-09T06:38:42.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.510+0000 7fb89e271700 1 -- 172.21.15.163:0/2866718002 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb89804c3c0 con 0x7fb898159890 2023-10-09T06:38:42.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.511+0000 7fb87ffff700 1 -- 172.21.15.163:0/2866718002 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb888025a30 con 0x7fb898159890 2023-10-09T06:38:42.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.511+0000 7fb87ffff700 1 --2- 172.21.15.163:0/2866718002 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb884067320 0x7fb8840697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:42.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.511+0000 7fb87ffff700 1 -- 172.21.15.163:0/2866718002 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fb8880861e0 con 0x7fb898159890 2023-10-09T06:38:42.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.511+0000 7fb8977fe700 1 --2- 172.21.15.163:0/2866718002 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb884067320 0x7fb8840697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:42.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.512+0000 7fb8977fe700 1 --2- 172.21.15.163:0/2866718002 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb884067320 0x7fb8840697d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb898150040 tx=0x7fb88c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:42.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.515+0000 7fb87ffff700 1 -- 172.21.15.163:0/2866718002 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb888053170 con 0x7fb898159890 2023-10-09T06:38:42.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:42.770+0000 7fb89e271700 1 -- 172.21.15.163:0/2866718002 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag"]} v 0) v1 -- 0x7fb8980ceca0 con 0x7fb898159890 2023-10-09T06:38:43.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.039+0000 7fb87ffff700 1 -- 172.21.15.163:0/2866718002 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag"]}]=0 v410) v1 ==== 167+0+0 (secure 0 0 0) 0x7fb888056790 con 0x7fb898159890 2023-10-09T06:38:43.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.041+0000 7fb89e271700 1 -- 172.21.15.163:0/2866718002 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb884067320 msgr2=0x7fb8840697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:43.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.041+0000 7fb89e271700 1 --2- 172.21.15.163:0/2866718002 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb884067320 0x7fb8840697d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb898150040 tx=0x7fb88c008040 comp rx=0 tx=0).stop 2023-10-09T06:38:43.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.041+0000 7fb89e271700 1 -- 172.21.15.163:0/2866718002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb898159890 msgr2=0x7fb898153a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:43.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.041+0000 7fb89e271700 1 --2- 172.21.15.163:0/2866718002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb898159890 0x7fb898153a00 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fb888009ed0 tx=0x7fb888017000 comp rx=0 tx=0).stop 2023-10-09T06:38:43.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.041+0000 7fb89e271700 1 -- 172.21.15.163:0/2866718002 shutdown_connections 2023-10-09T06:38:43.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.041+0000 7fb89e271700 1 --2- 172.21.15.163:0/2866718002 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb884067320 0x7fb8840697d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:43.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.041+0000 7fb89e271700 1 --2- 172.21.15.163:0/2866718002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb898159890 0x7fb898153a00 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:43.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.041+0000 7fb89e271700 1 --2- 172.21.15.163:0/2866718002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb89815db10 0x7fb8980cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:43.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.041+0000 7fb89e271700 1 --2- 172.21.15.163:0/2866718002 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb898159c60 0x7fb898153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:43.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.041+0000 7fb89e271700 1 -- 172.21.15.163:0/2866718002 >> 172.21.15.163:0/2866718002 conn(0x7fb898068e20 msgr2=0x7fb89815b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:43.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.041+0000 7fb89e271700 1 -- 172.21.15.163:0/2866718002 shutdown_connections 2023-10-09T06:38:43.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.041+0000 7fb89e271700 1 -- 172.21.15.163:0/2866718002 wait complete. 2023-10-09T06:38:43.055 INFO:tasks.cephfs_test_runner:That renaming volume fails without --yes-i-really-mean-it flag. ... ok 2023-10-09T06:38:43.055 INFO:tasks.cephfs_test_runner:Starting test: test_volume_rename_for_more_than_one_data_pool (tasks.cephfs.test_volumes.TestVolumes) That renaming a volume with more than one data pool does not change 2023-10-09T06:38:43.055 INFO:tasks.cephfs_test_runner:test_volume_rename_for_more_than_one_data_pool (tasks.cephfs.test_volumes.TestVolumes) 2023-10-09T06:38:43.055 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool' 2023-10-09T06:38:43.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.173+0000 7f138cd03700 1 Processor -- start 2023-10-09T06:38:43.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.173+0000 7f138cd03700 1 -- start start 2023-10-09T06:38:43.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.174+0000 7f138cd03700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1388159c60 0x7f138815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:43.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.174+0000 7f138cd03700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f138815a160 con 0x7f1388159890 2023-10-09T06:38:43.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.174+0000 7f138cd03700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13880d13a0 con 0x7f138815db10 2023-10-09T06:38:43.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.174+0000 7f138cd03700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f138815df50 con 0x7f1388159c60 2023-10-09T06:38:43.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.174+0000 7f138659c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f1388159890 0x7f138815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:56330/0 (socket says 172.21.15.163:56330) 2023-10-09T06:38:43.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.174+0000 7f138659c700 1 -- 172.21.15.163:0/1265354096 learned_addr learned my addr 172.21.15.163:0/1265354096 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:43.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.174+0000 7f137dd9b700 1 --2- 172.21.15.163:0/1265354096 >> v2:172.21.15.181:3300/0 conn(0x7f1388159c60 0x7f138815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:43.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.174+0000 7f1385d9b700 1 -- 172.21.15.163:0/1265354096 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 6895738 0 0) 0x7f138815a160 con 0x7f1388159890 2023-10-09T06:38:43.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.174+0000 7f1385d9b700 1 -- 172.21.15.163:0/1265354096 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1374003570 con 0x7f1388159890 2023-10-09T06:38:43.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.174+0000 7f1385d9b700 1 -- 172.21.15.163:0/1265354096 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1603749908 0 0) 0x7f1374003570 con 0x7f1388159890 2023-10-09T06:38:43.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.174+0000 7f1385d9b700 1 -- 172.21.15.163:0/1265354096 >> v2:172.21.15.181:3300/0 conn(0x7f1388159c60 msgr2=0x7f138815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:43.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.174+0000 7f1385d9b700 1 --2- 172.21.15.163:0/1265354096 >> v2:172.21.15.181:3300/0 conn(0x7f1388159c60 0x7f138815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:43.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.174+0000 7f1385d9b700 1 -- 172.21.15.163:0/1265354096 >> v1:172.21.15.181:6789/0 conn(0x7f138815db10 legacy=0x7f1388069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:43.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.174+0000 7f1385d9b700 1 -- 172.21.15.163:0/1265354096 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f138815e140 con 0x7f1388159890 2023-10-09T06:38:43.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.174+0000 7f1385d9b700 1 -- 172.21.15.163:0/1265354096 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f1378001470 con 0x7f1388159890 2023-10-09T06:38:43.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.174+0000 7f1385d9b700 1 -- 172.21.15.163:0/1265354096 >> v1:172.21.15.163:6789/0 conn(0x7f1388159890 legacy=0x7f138815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:43.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.174+0000 7f1385d9b700 1 --2- 172.21.15.163:0/1265354096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1374003a60 0x7f1374003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:43.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.174+0000 7f1385d9b700 1 --2- 172.21.15.163:0/1265354096 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1374004810 0x7f1374004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:43.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.174+0000 7f1385d9b700 1 --2- 172.21.15.163:0/1265354096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13740051c0 0x7f137400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:43.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.174+0000 7f1385d9b700 1 -- 172.21.15.163:0/1265354096 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137400c0a0 con 0x7f1374003a60 2023-10-09T06:38:43.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.174+0000 7f1385d9b700 1 -- 172.21.15.163:0/1265354096 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137400c210 con 0x7f13740051c0 2023-10-09T06:38:43.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.174+0000 7f1385d9b700 1 -- 172.21.15.163:0/1265354096 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137400c3d0 con 0x7f1374004810 2023-10-09T06:38:43.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.175+0000 7f138659c700 1 --2- 172.21.15.163:0/1265354096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1374003a60 0x7f1374003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:43.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.175+0000 7f137dd9b700 1 --2- 172.21.15.163:0/1265354096 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1374004810 0x7f1374004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:43.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.175+0000 7f1386d9d700 1 --2- 172.21.15.163:0/1265354096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13740051c0 0x7f137400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:43.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.175+0000 7f138659c700 1 -- 172.21.15.163:0/1265354096 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1374004810 msgr2=0x7f1374004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:43.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.175+0000 7f138659c700 1 --2- 172.21.15.163:0/1265354096 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1374004810 0x7f1374004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:43.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.175+0000 7f138659c700 1 -- 172.21.15.163:0/1265354096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13740051c0 msgr2=0x7f137400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:43.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.175+0000 7f138659c700 1 --2- 172.21.15.163:0/1265354096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13740051c0 0x7f137400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:43.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.175+0000 7f138659c700 1 -- 172.21.15.163:0/1265354096 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1374003570 con 0x7f1374003a60 2023-10-09T06:38:43.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.175+0000 7f1386d9d700 1 --2- 172.21.15.163:0/1265354096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13740051c0 0x7f137400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:43.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.175+0000 7f1386d9d700 1 -- 172.21.15.163:0/1265354096 reap_dead start 2023-10-09T06:38:43.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.175+0000 7f138659c700 1 --2- 172.21.15.163:0/1265354096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1374003a60 0x7f1374003e70 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f137800c3c0 tx=0x7f1378001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df5b8634664d62 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:43.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.175+0000 7f1385d9b700 1 -- 172.21.15.163:0/1265354096 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137800cc90 con 0x7f1374003a60 2023-10-09T06:38:43.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.175+0000 7f1385d9b700 1 -- 172.21.15.163:0/1265354096 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137800cdf0 con 0x7f1374003a60 2023-10-09T06:38:43.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.175+0000 7f1385d9b700 1 -- 172.21.15.163:0/1265354096 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1378005b50 con 0x7f1374003a60 2023-10-09T06:38:43.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.176+0000 7f138cd03700 1 -- 172.21.15.163:0/1265354096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1374003a60 msgr2=0x7f1374003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:43.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.176+0000 7f138cd03700 1 --2- 172.21.15.163:0/1265354096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1374003a60 0x7f1374003e70 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f137800c3c0 tx=0x7f1378001e50 comp rx=0 tx=0).stop 2023-10-09T06:38:43.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.176+0000 7f138cd03700 1 -- 172.21.15.163:0/1265354096 shutdown_connections 2023-10-09T06:38:43.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.176+0000 7f138cd03700 1 --2- 172.21.15.163:0/1265354096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1374003a60 0x7f1374003e70 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:43.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.176+0000 7f138cd03700 1 -- 172.21.15.163:0/1265354096 >> 172.21.15.163:0/1265354096 conn(0x7f1388068e20 msgr2=0x7f13880d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:43.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.176+0000 7f138cd03700 1 -- 172.21.15.163:0/1265354096 shutdown_connections 2023-10-09T06:38:43.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.176+0000 7f138cd03700 1 -- 172.21.15.163:0/1265354096 wait complete. 2023-10-09T06:38:43.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.177+0000 7f138cd03700 1 Processor -- start 2023-10-09T06:38:43.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.177+0000 7f138cd03700 1 -- start start 2023-10-09T06:38:43.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.178+0000 7f138cd03700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1374003a60 0x7f13880b3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:43.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.178+0000 7f138cd03700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1388155ca0 0x7f1388152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:43.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.178+0000 7f138cd03700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f138816d1c0 0x7f138816d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:43.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.178+0000 7f138cd03700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138816dba0 con 0x7f1388155ca0 2023-10-09T06:38:43.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.178+0000 7f138cd03700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13880b4aa0 con 0x7f1374003a60 2023-10-09T06:38:43.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.178+0000 7f138cd03700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13880b5340 con 0x7f138816d1c0 2023-10-09T06:38:43.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.178+0000 7f137dd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1388155ca0 0x7f1388152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:43.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.178+0000 7f137dd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1388155ca0 0x7f1388152000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38572/0 (socket says 172.21.15.163:38572) 2023-10-09T06:38:43.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.178+0000 7f137dd9b700 1 -- 172.21.15.163:0/4005600477 learned_addr learned my addr 172.21.15.163:0/4005600477 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:43.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.178+0000 7f138659c700 1 --2- 172.21.15.163:0/4005600477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1374003a60 0x7f13880b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:43.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.178+0000 7f1386d9d700 1 --2- 172.21.15.163:0/4005600477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f138816d1c0 0x7f138816d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:43.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.178+0000 7f137dd9b700 1 -- 172.21.15.163:0/4005600477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f138816d1c0 msgr2=0x7f138816d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:43.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.178+0000 7f137dd9b700 1 --2- 172.21.15.163:0/4005600477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f138816d1c0 0x7f138816d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:43.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.178+0000 7f137dd9b700 1 -- 172.21.15.163:0/4005600477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1374003a60 msgr2=0x7f13880b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:43.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.178+0000 7f137dd9b700 1 --2- 172.21.15.163:0/4005600477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1374003a60 0x7f13880b3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:43.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.178+0000 7f137dd9b700 1 -- 172.21.15.163:0/4005600477 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1388152540 con 0x7f1388155ca0 2023-10-09T06:38:43.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.178+0000 7f137dd9b700 1 --2- 172.21.15.163:0/4005600477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1388155ca0 0x7f1388152000 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f1370011d70 tx=0x7f13700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:43.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.178+0000 7f137effd700 1 -- 172.21.15.163:0/4005600477 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137000fac0 con 0x7f1388155ca0 2023-10-09T06:38:43.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.178+0000 7f137effd700 1 -- 172.21.15.163:0/4005600477 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137000fc20 con 0x7f1388155ca0 2023-10-09T06:38:43.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.178+0000 7f137effd700 1 -- 172.21.15.163:0/4005600477 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1370014440 con 0x7f1388155ca0 2023-10-09T06:38:43.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.178+0000 7f138cd03700 1 -- 172.21.15.163:0/4005600477 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13880b3d00 con 0x7f1388155ca0 2023-10-09T06:38:43.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.178+0000 7f138cd03700 1 -- 172.21.15.163:0/4005600477 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13880b4fd0 con 0x7f1388155ca0 2023-10-09T06:38:43.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.179+0000 7f138cd03700 1 -- 172.21.15.163:0/4005600477 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f138804c3c0 con 0x7f1388155ca0 2023-10-09T06:38:43.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.180+0000 7f137effd700 1 -- 172.21.15.163:0/4005600477 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1370014870 con 0x7f1388155ca0 2023-10-09T06:38:43.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.180+0000 7f137effd700 1 --2- 172.21.15.163:0/4005600477 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1368067320 0x7f13680697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:43.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.180+0000 7f137effd700 1 -- 172.21.15.163:0/4005600477 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f1370018030 con 0x7f1388155ca0 2023-10-09T06:38:43.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.180+0000 7f138659c700 1 --2- 172.21.15.163:0/4005600477 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1368067320 0x7f13680697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:43.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.181+0000 7f138659c700 1 --2- 172.21.15.163:0/4005600477 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1368067320 0x7f13680697d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1378007cd0 tx=0x7f1378015000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:43.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.184+0000 7f137effd700 1 -- 172.21.15.163:0/4005600477 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1370052270 con 0x7f1388155ca0 2023-10-09T06:38:43.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:43.445+0000 7f138cd03700 1 -- 172.21.15.163:0/4005600477 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool"]} v 0) v1 -- 0x7f138806a5b0 con 0x7f1388155ca0 2023-10-09T06:38:44.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.040+0000 7f137effd700 1 -- 172.21.15.163:0/4005600477 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool"]}]=0 v411) v1 ==== 166+0+0 (secure 0 0 0) 0x7f1370055890 con 0x7f1388155ca0 2023-10-09T06:38:44.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.042+0000 7f138cd03700 1 -- 172.21.15.163:0/4005600477 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1368067320 msgr2=0x7f13680697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:44.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.042+0000 7f138cd03700 1 --2- 172.21.15.163:0/4005600477 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1368067320 0x7f13680697d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1378007cd0 tx=0x7f1378015000 comp rx=0 tx=0).stop 2023-10-09T06:38:44.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.042+0000 7f138cd03700 1 -- 172.21.15.163:0/4005600477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1388155ca0 msgr2=0x7f1388152000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:44.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.042+0000 7f138cd03700 1 --2- 172.21.15.163:0/4005600477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1388155ca0 0x7f1388152000 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f1370011d70 tx=0x7f13700091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:44.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.042+0000 7f138cd03700 1 -- 172.21.15.163:0/4005600477 shutdown_connections 2023-10-09T06:38:44.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.042+0000 7f138cd03700 1 --2- 172.21.15.163:0/4005600477 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1368067320 0x7f13680697d0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:44.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.042+0000 7f138cd03700 1 --2- 172.21.15.163:0/4005600477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f138816d1c0 0x7f138816d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:44.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.042+0000 7f138cd03700 1 --2- 172.21.15.163:0/4005600477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1374003a60 0x7f13880b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:44.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.042+0000 7f138cd03700 1 --2- 172.21.15.163:0/4005600477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1388155ca0 0x7f1388152000 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:44.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.042+0000 7f138cd03700 1 -- 172.21.15.163:0/4005600477 >> 172.21.15.163:0/4005600477 conn(0x7f1388068e20 msgr2=0x7f13880b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:44.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.042+0000 7f138cd03700 1 -- 172.21.15.163:0/4005600477 shutdown_connections 2023-10-09T06:38:44.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.042+0000 7f138cd03700 1 -- 172.21.15.163:0/4005600477 wait complete. 2023-10-09T06:38:44.055 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-09T06:38:44.129 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:38:44.133 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:38:44.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.177+0000 7f2fd7363700 1 Processor -- start 2023-10-09T06:38:44.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.177+0000 7f2fd7363700 1 -- start start 2023-10-09T06:38:44.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.178+0000 7f2fd7363700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2fd00b4e70 0x7f2fd00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:44.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.178+0000 7f2fd7363700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fd00b16d0 0x7f2fd00b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:44.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.178+0000 7f2fd7363700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2fd015d240 0x7f2fd015d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:44.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.178+0000 7f2fd7363700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd00b2080 con 0x7f2fd00b4e70 2023-10-09T06:38:44.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.178+0000 7f2fd7363700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd00cfb00 con 0x7f2fd015d240 2023-10-09T06:38:44.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.178+0000 7f2fd7363700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd00b5c40 con 0x7f2fd00b16d0 2023-10-09T06:38:44.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.178+0000 7f2fd6361700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2fd00b4e70 0x7f2fd00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:44.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.178+0000 7f2fd6361700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2fd00b4e70 0x7f2fd00ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38588/0 (socket says 172.21.15.163:38588) 2023-10-09T06:38:44.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.178+0000 7f2fd6361700 1 -- 172.21.15.163:0/71207255 learned_addr learned my addr 172.21.15.163:0/71207255 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:44.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.178+0000 7f2fd6b62700 1 --2- 172.21.15.163:0/71207255 >> v2:172.21.15.181:3300/0 conn(0x7f2fd015d240 0x7f2fd015d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:44.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.178+0000 7f2fd5b60700 1 --2- 172.21.15.163:0/71207255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fd00b16d0 0x7f2fd00b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:44.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.178+0000 7f2fd6361700 1 -- 172.21.15.163:0/71207255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fd00b16d0 msgr2=0x7f2fd00b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:44.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.178+0000 7f2fd6361700 1 --2- 172.21.15.163:0/71207255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fd00b16d0 0x7f2fd00b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:44.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.178+0000 7f2fd6361700 1 -- 172.21.15.163:0/71207255 >> v2:172.21.15.181:3300/0 conn(0x7f2fd015d240 msgr2=0x7f2fd015d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:44.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.178+0000 7f2fd6361700 1 --2- 172.21.15.163:0/71207255 >> v2:172.21.15.181:3300/0 conn(0x7f2fd015d240 0x7f2fd015d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:44.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.178+0000 7f2fd6361700 1 -- 172.21.15.163:0/71207255 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd00b5f20 con 0x7f2fd00b4e70 2023-10-09T06:38:44.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.178+0000 7f2fd5b60700 1 --2- 172.21.15.163:0/71207255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fd00b16d0 0x7f2fd00b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:44.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.178+0000 7f2fd6b62700 1 --2- 172.21.15.163:0/71207255 >> v2:172.21.15.181:3300/0 conn(0x7f2fd015d240 0x7f2fd015d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:44.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.178+0000 7f2fd6361700 1 --2- 172.21.15.163:0/71207255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fd00b4e70 0x7f2fd00ccff0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f2fd015d180 tx=0x7f2fcc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b4eea34e6a9820b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:44.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.178+0000 7f2fd535f700 1 -- 172.21.15.163:0/71207255 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc00b860 con 0x7f2fd00b4e70 2023-10-09T06:38:44.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.179+0000 7f2fd535f700 1 -- 172.21.15.163:0/71207255 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fcc00b9c0 con 0x7f2fd00b4e70 2023-10-09T06:38:44.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.179+0000 7f2fd535f700 1 -- 172.21.15.163:0/71207255 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc00bd60 con 0x7f2fd00b4e70 2023-10-09T06:38:44.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.179+0000 7f2fd7363700 1 -- 172.21.15.163:0/71207255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fd00b4e70 msgr2=0x7f2fd00ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:44.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.179+0000 7f2fd7363700 1 --2- 172.21.15.163:0/71207255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fd00b4e70 0x7f2fd00ccff0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f2fd015d180 tx=0x7f2fcc009250 comp rx=0 tx=0).stop 2023-10-09T06:38:44.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.179+0000 7f2fd7363700 1 -- 172.21.15.163:0/71207255 shutdown_connections 2023-10-09T06:38:44.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.179+0000 7f2fd7363700 1 --2- 172.21.15.163:0/71207255 >> v2:172.21.15.181:3300/0 conn(0x7f2fd015d240 0x7f2fd015d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:44.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.179+0000 7f2fd7363700 1 --2- 172.21.15.163:0/71207255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fd00b4e70 0x7f2fd00ccff0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:44.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.179+0000 7f2fd7363700 1 --2- 172.21.15.163:0/71207255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fd00b16d0 0x7f2fd00b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:44.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.179+0000 7f2fd7363700 1 -- 172.21.15.163:0/71207255 >> 172.21.15.163:0/71207255 conn(0x7f2fd00d10c0 msgr2=0x7f2fd00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:44.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.179+0000 7f2fd7363700 1 -- 172.21.15.163:0/71207255 shutdown_connections 2023-10-09T06:38:44.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.179+0000 7f2fd7363700 1 -- 172.21.15.163:0/71207255 wait complete. 2023-10-09T06:38:44.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.180+0000 7f2fd7363700 1 Processor -- start 2023-10-09T06:38:44.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.180+0000 7f2fd7363700 1 -- start start 2023-10-09T06:38:44.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.181+0000 7f2fd7363700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fd00b16d0 0x7f2fd00cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:44.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.181+0000 7f2fd7363700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fd00b4e70 0x7f2fd00c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:44.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.181+0000 7f2fd7363700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fd015d240 0x7f2fd00c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:44.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.181+0000 7f2fd7363700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd016af20 con 0x7f2fd00b4e70 2023-10-09T06:38:44.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.181+0000 7f2fd7363700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd00c85c0 con 0x7f2fd015d240 2023-10-09T06:38:44.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.181+0000 7f2fd7363700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd00c8000 con 0x7f2fd00b16d0 2023-10-09T06:38:44.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.181+0000 7f2fd5b60700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fd00b4e70 0x7f2fd00c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:44.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.181+0000 7f2fd5b60700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fd00b4e70 0x7f2fd00c74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38602/0 (socket says 172.21.15.163:38602) 2023-10-09T06:38:44.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.181+0000 7f2fd5b60700 1 -- 172.21.15.163:0/2220561883 learned_addr learned my addr 172.21.15.163:0/2220561883 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:44.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.181+0000 7f2fd6361700 1 --2- 172.21.15.163:0/2220561883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fd00b16d0 0x7f2fd00cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:44.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.181+0000 7f2fd5b60700 1 -- 172.21.15.163:0/2220561883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fd00b16d0 msgr2=0x7f2fd00cbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:44.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.181+0000 7f2fd6b62700 1 --2- 172.21.15.163:0/2220561883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fd015d240 0x7f2fd00c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:44.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.181+0000 7f2fd5b60700 1 --2- 172.21.15.163:0/2220561883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fd00b16d0 0x7f2fd00cbc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:44.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.181+0000 7f2fd5b60700 1 -- 172.21.15.163:0/2220561883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fd015d240 msgr2=0x7f2fd00c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:44.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.181+0000 7f2fd5b60700 1 --2- 172.21.15.163:0/2220561883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fd015d240 0x7f2fd00c7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:44.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.181+0000 7f2fd5b60700 1 -- 172.21.15.163:0/2220561883 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fcc0094f0 con 0x7f2fd00b4e70 2023-10-09T06:38:44.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.182+0000 7f2fd5b60700 1 --2- 172.21.15.163:0/2220561883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fd00b4e70 0x7f2fd00c74e0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000e8d0 tx=0x7f2fc0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:44.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.182+0000 7f2fc67fc700 1 -- 172.21.15.163:0/2220561883 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc000ee70 con 0x7f2fd00b4e70 2023-10-09T06:38:44.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.182+0000 7f2fc67fc700 1 -- 172.21.15.163:0/2220561883 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc000c5e0 con 0x7f2fd00b4e70 2023-10-09T06:38:44.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.182+0000 7f2fd7363700 1 -- 172.21.15.163:0/2220561883 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd00c82e0 con 0x7f2fd00b4e70 2023-10-09T06:38:44.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.182+0000 7f2fc67fc700 1 -- 172.21.15.163:0/2220561883 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc000c980 con 0x7f2fd00b4e70 2023-10-09T06:38:44.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.182+0000 7f2fd7363700 1 -- 172.21.15.163:0/2220561883 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd016dce0 con 0x7f2fd00b4e70 2023-10-09T06:38:44.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.183+0000 7f2fd7363700 1 -- 172.21.15.163:0/2220561883 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd016b090 con 0x7f2fd00b4e70 2023-10-09T06:38:44.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.183+0000 7f2fc67fc700 1 -- 172.21.15.163:0/2220561883 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2fc0013410 con 0x7f2fd00b4e70 2023-10-09T06:38:44.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.183+0000 7f2fc67fc700 1 --2- 172.21.15.163:0/2220561883 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2fb4067320 0x7f2fb40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:44.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.184+0000 7f2fc67fc700 1 -- 172.21.15.163:0/2220561883 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2fc0084870 con 0x7f2fd00b4e70 2023-10-09T06:38:44.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.184+0000 7f2fd6361700 1 --2- 172.21.15.163:0/2220561883 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2fb4067320 0x7f2fb40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:44.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.184+0000 7f2fd6361700 1 --2- 172.21.15.163:0/2220561883 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2fb4067320 0x7f2fb40697d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2fd015d180 tx=0x7f2fcc017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:44.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.188+0000 7f2fc67fc700 1 -- 172.21.15.163:0/2220561883 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2fc00516f0 con 0x7f2fd00b4e70 2023-10-09T06:38:44.372 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.371+0000 7f2fd7363700 1 -- 172.21.15.163:0/2220561883 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2fd004bb80 con 0x7f2fd00b4e70 2023-10-09T06:38:44.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.374+0000 7f2fc67fc700 1 -- 172.21.15.163:0/2220561883 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2fc0054d10 con 0x7f2fd00b4e70 2023-10-09T06:38:44.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.377+0000 7f2fd7363700 1 -- 172.21.15.163:0/2220561883 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2fb4067320 msgr2=0x7f2fb40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:44.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.377+0000 7f2fd7363700 1 --2- 172.21.15.163:0/2220561883 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2fb4067320 0x7f2fb40697d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2fd015d180 tx=0x7f2fcc017000 comp rx=0 tx=0).stop 2023-10-09T06:38:44.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.377+0000 7f2fd7363700 1 -- 172.21.15.163:0/2220561883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fd00b4e70 msgr2=0x7f2fd00c74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:44.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.377+0000 7f2fd7363700 1 --2- 172.21.15.163:0/2220561883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fd00b4e70 0x7f2fd00c74e0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000e8d0 tx=0x7f2fc0008040 comp rx=0 tx=0).stop 2023-10-09T06:38:44.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.377+0000 7f2fd7363700 1 -- 172.21.15.163:0/2220561883 shutdown_connections 2023-10-09T06:38:44.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.377+0000 7f2fd7363700 1 --2- 172.21.15.163:0/2220561883 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2fb4067320 0x7f2fb40697d0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:44.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.377+0000 7f2fd7363700 1 --2- 172.21.15.163:0/2220561883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fd015d240 0x7f2fd00c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:44.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.377+0000 7f2fd7363700 1 --2- 172.21.15.163:0/2220561883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fd00b16d0 0x7f2fd00cbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:44.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.377+0000 7f2fd7363700 1 --2- 172.21.15.163:0/2220561883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fd00b4e70 0x7f2fd00c74e0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:44.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.377+0000 7f2fd7363700 1 -- 172.21.15.163:0/2220561883 >> 172.21.15.163:0/2220561883 conn(0x7f2fd00d10c0 msgr2=0x7f2fd015c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:44.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.377+0000 7f2fd7363700 1 -- 172.21.15.163:0/2220561883 shutdown_connections 2023-10-09T06:38:44.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.377+0000 7f2fd7363700 1 -- 172.21.15.163:0/2220561883 wait complete. 2023-10-09T06:38:44.390 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-09T06:38:44.390 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:38:44.478 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:38:44.524 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-09T06:38:44.524 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-09T06:38:44.525 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:38:44.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.638+0000 7f48188e4700 1 Processor -- start 2023-10-09T06:38:44.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.638+0000 7f48188e4700 1 -- start start 2023-10-09T06:38:44.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.639+0000 7f48188e4700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4814159c60 0x7f481415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:44.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.639+0000 7f48188e4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f481415db10 0x7f4814069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:44.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.639+0000 7f48188e4700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f481415a160 con 0x7f4814159890 2023-10-09T06:38:44.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.639+0000 7f48188e4700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f481415df50 con 0x7f4814159c60 2023-10-09T06:38:44.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.639+0000 7f48188e4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48140d13a0 con 0x7f481415db10 2023-10-09T06:38:44.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.639+0000 7f481259c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f4814159890 0x7f481415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:56342/0 (socket says 172.21.15.163:56342) 2023-10-09T06:38:44.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.639+0000 7f481259c700 1 -- 172.21.15.163:0/3637276708 learned_addr learned my addr 172.21.15.163:0/3637276708 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:44.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.639+0000 7f4811d9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4814159c60 0x7f481415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:44.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.639+0000 7f4812d9d700 1 --2- 172.21.15.163:0/3637276708 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f481415db10 0x7f4814069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:44.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.639+0000 7f4811d9b700 1 -- 172.21.15.163:0/3637276708 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f481415db10 msgr2=0x7f4814069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:44.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.639+0000 7f4811d9b700 1 --2- 172.21.15.163:0/3637276708 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f481415db10 0x7f4814069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:44.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.639+0000 7f4811d9b700 1 -- 172.21.15.163:0/3637276708 >> v1:172.21.15.163:6789/0 conn(0x7f4814159890 legacy=0x7f481415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:44.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.639+0000 7f4811d9b700 1 -- 172.21.15.163:0/3637276708 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481415e140 con 0x7f4814159c60 2023-10-09T06:38:44.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.639+0000 7f481159a700 1 -- 172.21.15.163:0/3637276708 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 998305388 0 0) 0x7f481415a160 con 0x7f4814159890 2023-10-09T06:38:44.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.639+0000 7f4811d9b700 1 --2- 172.21.15.163:0/3637276708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4814159c60 0x7f481415d5d0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f47fc0098e0 tx=0x7f47fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104061c9a4a6b311 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:44.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.640+0000 7f481159a700 1 -- 172.21.15.163:0/3637276708 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc009e80 con 0x7f4814159c60 2023-10-09T06:38:44.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.640+0000 7f481159a700 1 -- 172.21.15.163:0/3637276708 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47fc00b860 con 0x7f4814159c60 2023-10-09T06:38:44.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.640+0000 7f481159a700 1 -- 172.21.15.163:0/3637276708 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc00bbe0 con 0x7f4814159c60 2023-10-09T06:38:44.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.640+0000 7f48188e4700 1 -- 172.21.15.163:0/3637276708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4814159c60 msgr2=0x7f481415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:44.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.640+0000 7f48188e4700 1 --2- 172.21.15.163:0/3637276708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4814159c60 0x7f481415d5d0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f47fc0098e0 tx=0x7f47fc0091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:44.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.640+0000 7f48188e4700 1 -- 172.21.15.163:0/3637276708 shutdown_connections 2023-10-09T06:38:44.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.640+0000 7f48188e4700 1 --2- 172.21.15.163:0/3637276708 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f481415db10 0x7f4814069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:44.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.640+0000 7f48188e4700 1 --2- 172.21.15.163:0/3637276708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4814159c60 0x7f481415d5d0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:44.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.640+0000 7f48188e4700 1 -- 172.21.15.163:0/3637276708 >> 172.21.15.163:0/3637276708 conn(0x7f4814068e20 msgr2=0x7f48140d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:44.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.640+0000 7f48188e4700 1 -- 172.21.15.163:0/3637276708 shutdown_connections 2023-10-09T06:38:44.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.640+0000 7f48188e4700 1 -- 172.21.15.163:0/3637276708 wait complete. 2023-10-09T06:38:44.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.641+0000 7f48188e4700 1 Processor -- start 2023-10-09T06:38:44.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.641+0000 7f48188e4700 1 -- start start 2023-10-09T06:38:44.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.642+0000 7f48188e4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4814159890 0x7f4814144120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:44.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.642+0000 7f48188e4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4814159c60 0x7f48140b3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:44.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.642+0000 7f48188e4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f481415db10 0x7f4814158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:44.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.642+0000 7f48188e4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48140b2770 con 0x7f4814159890 2023-10-09T06:38:44.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.642+0000 7f48188e4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48140b2490 con 0x7f4814159c60 2023-10-09T06:38:44.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.642+0000 7f48188e4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48140b5dd0 con 0x7f481415db10 2023-10-09T06:38:44.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.642+0000 7f481259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4814159890 0x7f4814144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:44.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.642+0000 7f481259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4814159890 0x7f4814144120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38620/0 (socket says 172.21.15.163:38620) 2023-10-09T06:38:44.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.642+0000 7f481259c700 1 -- 172.21.15.163:0/1774868594 learned_addr learned my addr 172.21.15.163:0/1774868594 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:44.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.642+0000 7f4812d9d700 1 --2- 172.21.15.163:0/1774868594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f481415db10 0x7f4814158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:44.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.642+0000 7f4811d9b700 1 --2- 172.21.15.163:0/1774868594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4814159c60 0x7f48140b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:44.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.642+0000 7f481259c700 1 -- 172.21.15.163:0/1774868594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f481415db10 msgr2=0x7f4814158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:44.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.642+0000 7f481259c700 1 --2- 172.21.15.163:0/1774868594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f481415db10 0x7f4814158460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:44.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.642+0000 7f481259c700 1 -- 172.21.15.163:0/1774868594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4814159c60 msgr2=0x7f48140b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:44.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.642+0000 7f481259c700 1 --2- 172.21.15.163:0/1774868594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4814159c60 0x7f48140b3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:44.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.642+0000 7f481259c700 1 -- 172.21.15.163:0/1774868594 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47fc009420 con 0x7f4814159890 2023-10-09T06:38:44.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.643+0000 7f481259c700 1 --2- 172.21.15.163:0/1774868594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4814159890 0x7f4814144120 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f480c00cd80 tx=0x7f480c001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:44.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.643+0000 7f480a7fc700 1 -- 172.21.15.163:0/1774868594 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c00ab10 con 0x7f4814159890 2023-10-09T06:38:44.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.643+0000 7f480a7fc700 1 -- 172.21.15.163:0/1774868594 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480c00ac70 con 0x7f4814159890 2023-10-09T06:38:44.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.643+0000 7f480a7fc700 1 -- 172.21.15.163:0/1774868594 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c0070f0 con 0x7f4814159890 2023-10-09T06:38:44.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.643+0000 7f48188e4700 1 -- 172.21.15.163:0/1774868594 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48140b60b0 con 0x7f4814159890 2023-10-09T06:38:44.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.643+0000 7f48188e4700 1 -- 172.21.15.163:0/1774868594 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4814144660 con 0x7f4814159890 2023-10-09T06:38:44.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.644+0000 7f48188e4700 1 -- 172.21.15.163:0/1774868594 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48140cd100 con 0x7f4814159890 2023-10-09T06:38:44.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.644+0000 7f480a7fc700 1 -- 172.21.15.163:0/1774868594 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f480c0074e0 con 0x7f4814159890 2023-10-09T06:38:44.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.644+0000 7f480a7fc700 1 --2- 172.21.15.163:0/1774868594 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f47f4067320 0x7f47f40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:44.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.644+0000 7f480a7fc700 1 -- 172.21.15.163:0/1774868594 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f480c084ea0 con 0x7f4814159890 2023-10-09T06:38:44.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.645+0000 7f4811d9b700 1 --2- 172.21.15.163:0/1774868594 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f47f4067320 0x7f47f40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:44.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.645+0000 7f4811d9b700 1 --2- 172.21.15.163:0/1774868594 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f47f4067320 0x7f47f40697d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f47fc0091e0 tx=0x7f47fc017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:44.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.649+0000 7f480a7fc700 1 -- 172.21.15.163:0/1774868594 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f480c051eb0 con 0x7f4814159890 2023-10-09T06:38:44.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.873+0000 7f48188e4700 1 -- 172.21.15.163:0/1774868594 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f481404c3c0 con 0x7f4814159890 2023-10-09T06:38:44.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.873+0000 7f480a7fc700 1 -- 172.21.15.163:0/1774868594 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+3543 (secure 0 0 0) 0x7f480c0554d0 con 0x7f4814159890 2023-10-09T06:38:44.874 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:44.875 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":106,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6371,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3797578839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3797578839},{"type":"v1","addr":"172.21.15.163:6834","nonce":3797578839}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":6383,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2371721802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2371721802},{"type":"v1","addr":"172.21.15.181:6837","nonce":2371721802}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":7228,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2068214884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2068214884},{"type":"v1","addr":"172.21.15.181:6836","nonce":2068214884}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":8148,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3619131074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3619131074},{"type":"v1","addr":"172.21.15.163:6835","nonce":3619131074}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106}],"filesystems":[]} 2023-10-09T06:38:44.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.875+0000 7f48188e4700 1 -- 172.21.15.163:0/1774868594 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f47f4067320 msgr2=0x7f47f40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:44.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.875+0000 7f48188e4700 1 --2- 172.21.15.163:0/1774868594 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f47f4067320 0x7f47f40697d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f47fc0091e0 tx=0x7f47fc017000 comp rx=0 tx=0).stop 2023-10-09T06:38:44.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.875+0000 7f48188e4700 1 -- 172.21.15.163:0/1774868594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4814159890 msgr2=0x7f4814144120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:44.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.875+0000 7f48188e4700 1 --2- 172.21.15.163:0/1774868594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4814159890 0x7f4814144120 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f480c00cd80 tx=0x7f480c001ae0 comp rx=0 tx=0).stop 2023-10-09T06:38:44.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.875+0000 7f48188e4700 1 -- 172.21.15.163:0/1774868594 shutdown_connections 2023-10-09T06:38:44.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.875+0000 7f48188e4700 1 --2- 172.21.15.163:0/1774868594 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f47f4067320 0x7f47f40697d0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:44.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.875+0000 7f48188e4700 1 --2- 172.21.15.163:0/1774868594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4814159890 0x7f4814144120 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:44.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.875+0000 7f48188e4700 1 --2- 172.21.15.163:0/1774868594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f481415db10 0x7f4814158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:44.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.875+0000 7f48188e4700 1 --2- 172.21.15.163:0/1774868594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4814159c60 0x7f48140b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:44.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.875+0000 7f48188e4700 1 -- 172.21.15.163:0/1774868594 >> 172.21.15.163:0/1774868594 conn(0x7f4814068e20 msgr2=0x7f4814153d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:44.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.875+0000 7f48188e4700 1 -- 172.21.15.163:0/1774868594 shutdown_connections 2023-10-09T06:38:44.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:44.875+0000 7f48188e4700 1 -- 172.21.15.163:0/1774868594 wait complete. 2023-10-09T06:38:44.878 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 106 2023-10-09T06:38:44.889 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-09T06:38:44.889 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-09T06:38:44.889 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-09T06:38:44.889 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:38:44.889 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-09T06:38:44.890 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-09T06:38:44.890 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-09T06:38:44.890 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:38:44.890 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-09T06:38:44.890 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-09T06:38:44.891 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-09T06:38:44.891 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:38:44.891 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-09T06:38:44.891 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-09T06:38:44.891 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-09T06:38:44.892 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:38:44.898 INFO:tasks.ceph.mds.a:Stopped 2023-10-09T06:38:44.899 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-09T06:38:44.900 INFO:tasks.ceph.mds.c:Stopped 2023-10-09T06:38:44.900 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-09T06:38:44.906 INFO:tasks.ceph.mds.d:Stopped 2023-10-09T06:38:44.906 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-09T06:38:44.907 INFO:tasks.ceph.mds.b:Stopped 2023-10-09T06:38:44.907 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-09T06:38:44.941 INFO:tasks.ceph.mds.a:Started 2023-10-09T06:38:44.941 DEBUG:teuthology.parallel:result is None 2023-10-09T06:38:44.942 INFO:tasks.ceph.mds.c:Started 2023-10-09T06:38:44.942 DEBUG:teuthology.parallel:result is None 2023-10-09T06:38:44.949 INFO:tasks.ceph.mds.d:Started 2023-10-09T06:38:44.949 DEBUG:teuthology.parallel:result is None 2023-10-09T06:38:44.950 INFO:tasks.ceph.mds.b:Started 2023-10-09T06:38:44.950 DEBUG:teuthology.parallel:result is None 2023-10-09T06:38:44.950 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-09T06:38:45.015 INFO:tasks.ceph.mds.c.smithi163.stdout:starting mds.c at 2023-10-09T06:38:45.015 INFO:tasks.ceph.mds.a.smithi163.stdout:starting mds.a at 2023-10-09T06:38:45.030 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-09T06:38:45.030 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-09T06:38:45.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.072+0000 7ffb71500700 1 Processor -- start 2023-10-09T06:38:45.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.072+0000 7ffb71500700 1 -- start start 2023-10-09T06:38:45.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.073+0000 7ffb71500700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ffb6c159c60 0x7ffb6c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:45.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.073+0000 7ffb71500700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb6c15db10 0x7ffb6c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:45.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.073+0000 7ffb71500700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb6c15a160 con 0x7ffb6c159890 2023-10-09T06:38:45.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.073+0000 7ffb71500700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb6c15df50 con 0x7ffb6c159c60 2023-10-09T06:38:45.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.073+0000 7ffb71500700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c0d13a0 con 0x7ffb6c15db10 2023-10-09T06:38:45.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.073+0000 7ffb627fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ffb6c159c60 0x7ffb6c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:45.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.073+0000 7ffb627fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ffb6c159c60 0x7ffb6c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38656/0 (socket says 172.21.15.163:38656) 2023-10-09T06:38:45.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.073+0000 7ffb627fc700 1 -- 172.21.15.163:0/1092541011 learned_addr learned my addr 172.21.15.163:0/1092541011 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:45.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.074+0000 7ffb6b7fe700 1 --2- 172.21.15.163:0/1092541011 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb6c15db10 0x7ffb6c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:45.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.074+0000 7ffb627fc700 1 -- 172.21.15.163:0/1092541011 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb6c15db10 msgr2=0x7ffb6c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:45.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.074+0000 7ffb627fc700 1 --2- 172.21.15.163:0/1092541011 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb6c15db10 0x7ffb6c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:45.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.074+0000 7ffb627fc700 1 -- 172.21.15.163:0/1092541011 >> v1:172.21.15.181:6789/0 conn(0x7ffb6c159890 legacy=0x7ffb6c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:45.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.074+0000 7ffb627fc700 1 -- 172.21.15.163:0/1092541011 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6c15e140 con 0x7ffb6c159c60 2023-10-09T06:38:45.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.074+0000 7ffb627fc700 1 --2- 172.21.15.163:0/1092541011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb6c159c60 0x7ffb6c15d5d0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7ffb540099b0 tx=0x7ffb54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd41929bc333a39 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:45.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.074+0000 7ffb6a7fc700 1 -- 172.21.15.163:0/1092541011 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5400b820 con 0x7ffb6c159c60 2023-10-09T06:38:45.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.074+0000 7ffb6a7fc700 1 -- 172.21.15.163:0/1092541011 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5400b980 con 0x7ffb6c159c60 2023-10-09T06:38:45.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.074+0000 7ffb6a7fc700 1 -- 172.21.15.163:0/1092541011 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5400bd20 con 0x7ffb6c159c60 2023-10-09T06:38:45.079 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.075+0000 7ffb71500700 1 -- 172.21.15.163:0/1092541011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb6c159c60 msgr2=0x7ffb6c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:45.079 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.075+0000 7ffb71500700 1 --2- 172.21.15.163:0/1092541011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb6c159c60 0x7ffb6c15d5d0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7ffb540099b0 tx=0x7ffb54009250 comp rx=0 tx=0).stop 2023-10-09T06:38:45.079 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.075+0000 7ffb71500700 1 -- 172.21.15.163:0/1092541011 shutdown_connections 2023-10-09T06:38:45.079 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.075+0000 7ffb71500700 1 --2- 172.21.15.163:0/1092541011 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb6c15db10 0x7ffb6c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:45.079 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.075+0000 7ffb71500700 1 --2- 172.21.15.163:0/1092541011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb6c159c60 0x7ffb6c15d5d0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:45.079 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.075+0000 7ffb71500700 1 -- 172.21.15.163:0/1092541011 >> 172.21.15.163:0/1092541011 conn(0x7ffb6c068e20 msgr2=0x7ffb6c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:45.079 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.075+0000 7ffb71500700 1 -- 172.21.15.163:0/1092541011 shutdown_connections 2023-10-09T06:38:45.079 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.075+0000 7ffb71500700 1 -- 172.21.15.163:0/1092541011 wait complete. 2023-10-09T06:38:45.079 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.076+0000 7ffb71500700 1 Processor -- start 2023-10-09T06:38:45.079 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.076+0000 7ffb71500700 1 -- start start 2023-10-09T06:38:45.079 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.077+0000 7ffb71500700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb6c159890 0x7ffb6c0b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:45.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.077+0000 7ffb71500700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb6c159c60 0x7ffb6c0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:45.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.077+0000 7ffb71500700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb6c15db10 0x7ffb6c153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:45.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.077+0000 7ffb71500700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c0ce140 con 0x7ffb6c159c60 2023-10-09T06:38:45.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.077+0000 7ffb71500700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c0cd770 con 0x7ffb6c15db10 2023-10-09T06:38:45.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.077+0000 7ffb71500700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c0cd160 con 0x7ffb6c159890 2023-10-09T06:38:45.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.077+0000 7ffb627fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb6c159c60 0x7ffb6c0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:45.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.077+0000 7ffb627fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb6c159c60 0x7ffb6c0cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38660/0 (socket says 172.21.15.163:38660) 2023-10-09T06:38:45.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.077+0000 7ffb627fc700 1 -- 172.21.15.163:0/1570298169 learned_addr learned my addr 172.21.15.163:0/1570298169 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:45.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.077+0000 7ffb6affd700 1 --2- 172.21.15.163:0/1570298169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb6c159890 0x7ffb6c0b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:45.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.077+0000 7ffb6b7fe700 1 --2- 172.21.15.163:0/1570298169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb6c15db10 0x7ffb6c153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:45.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.077+0000 7ffb627fc700 1 -- 172.21.15.163:0/1570298169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb6c159890 msgr2=0x7ffb6c0b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:45.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.077+0000 7ffb627fc700 1 --2- 172.21.15.163:0/1570298169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb6c159890 0x7ffb6c0b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:45.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.077+0000 7ffb627fc700 1 -- 172.21.15.163:0/1570298169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb6c15db10 msgr2=0x7ffb6c153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:45.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.077+0000 7ffb627fc700 1 --2- 172.21.15.163:0/1570298169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb6c15db10 0x7ffb6c153980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:45.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.077+0000 7ffb627fc700 1 -- 172.21.15.163:0/1570298169 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb540094f0 con 0x7ffb6c159c60 2023-10-09T06:38:45.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.077+0000 7ffb627fc700 1 --2- 172.21.15.163:0/1570298169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb6c159c60 0x7ffb6c0cdc00 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7ffb54009290 tx=0x7ffb54016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:45.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.077+0000 7ffb637fe700 1 -- 172.21.15.163:0/1570298169 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb54011c30 con 0x7ffb6c159c60 2023-10-09T06:38:45.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.077+0000 7ffb637fe700 1 -- 172.21.15.163:0/1570298169 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb54020070 con 0x7ffb6c159c60 2023-10-09T06:38:45.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.077+0000 7ffb71500700 1 -- 172.21.15.163:0/1570298169 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb6c0cfbc0 con 0x7ffb6c159c60 2023-10-09T06:38:45.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.077+0000 7ffb637fe700 1 -- 172.21.15.163:0/1570298169 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5401b450 con 0x7ffb6c159c60 2023-10-09T06:38:45.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.077+0000 7ffb71500700 1 -- 172.21.15.163:0/1570298169 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb6c0cfea0 con 0x7ffb6c159c60 2023-10-09T06:38:45.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.078+0000 7ffb71500700 1 -- 172.21.15.163:0/1570298169 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb6c0cd8e0 con 0x7ffb6c159c60 2023-10-09T06:38:45.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.079+0000 7ffb637fe700 1 -- 172.21.15.163:0/1570298169 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ffb54022030 con 0x7ffb6c159c60 2023-10-09T06:38:45.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.079+0000 7ffb637fe700 1 --2- 172.21.15.163:0/1570298169 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffb4c067320 0x7ffb4c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:45.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.079+0000 7ffb637fe700 1 -- 172.21.15.163:0/1570298169 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ffb540866a0 con 0x7ffb6c159c60 2023-10-09T06:38:45.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.079+0000 7ffb6affd700 1 --2- 172.21.15.163:0/1570298169 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffb4c067320 0x7ffb4c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:45.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.080+0000 7ffb6affd700 1 --2- 172.21.15.163:0/1570298169 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffb4c067320 0x7ffb4c0697d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c003a30 tx=0x7ffb5c007ea0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:45.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.084+0000 7ffb637fe700 1 -- 172.21.15.163:0/1570298169 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffb54053740 con 0x7ffb6c159c60 2023-10-09T06:38:45.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:45.282+0000 7ffb71500700 1 -- 172.21.15.163:0/1570298169 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ffb6c0cd8e0 con 0x7ffb6c159c60 2023-10-09T06:38:46.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.079+0000 7ffb637fe700 1 -- 172.21.15.163:0/1570298169 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v160) v1 ==== 97+0+0 (secure 0 0 0) 0x7ffb54056d60 con 0x7ffb6c159c60 2023-10-09T06:38:46.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.081+0000 7ffb71500700 1 -- 172.21.15.163:0/1570298169 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffb4c067320 msgr2=0x7ffb4c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:46.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.081+0000 7ffb71500700 1 --2- 172.21.15.163:0/1570298169 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffb4c067320 0x7ffb4c0697d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c003a30 tx=0x7ffb5c007ea0 comp rx=0 tx=0).stop 2023-10-09T06:38:46.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.081+0000 7ffb71500700 1 -- 172.21.15.163:0/1570298169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb6c159c60 msgr2=0x7ffb6c0cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:46.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.081+0000 7ffb71500700 1 --2- 172.21.15.163:0/1570298169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb6c159c60 0x7ffb6c0cdc00 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7ffb54009290 tx=0x7ffb54016000 comp rx=0 tx=0).stop 2023-10-09T06:38:46.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.081+0000 7ffb71500700 1 -- 172.21.15.163:0/1570298169 shutdown_connections 2023-10-09T06:38:46.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.081+0000 7ffb71500700 1 --2- 172.21.15.163:0/1570298169 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffb4c067320 0x7ffb4c0697d0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:46.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.081+0000 7ffb71500700 1 --2- 172.21.15.163:0/1570298169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb6c15db10 0x7ffb6c153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:46.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.081+0000 7ffb71500700 1 --2- 172.21.15.163:0/1570298169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb6c159890 0x7ffb6c0b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:46.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.081+0000 7ffb71500700 1 --2- 172.21.15.163:0/1570298169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb6c159c60 0x7ffb6c0cdc00 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:46.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.081+0000 7ffb71500700 1 -- 172.21.15.163:0/1570298169 >> 172.21.15.163:0/1570298169 conn(0x7ffb6c068e20 msgr2=0x7ffb6c158630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:46.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.082+0000 7ffb71500700 1 -- 172.21.15.163:0/1570298169 shutdown_connections 2023-10-09T06:38:46.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.082+0000 7ffb71500700 1 -- 172.21.15.163:0/1570298169 wait complete. 2023-10-09T06:38:46.084 INFO:teuthology.orchestra.run.smithi163.stderr: removed all blocklist entries 2023-10-09T06:38:46.095 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-09T06:38:46.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.209+0000 7f6b92393700 1 Processor -- start 2023-10-09T06:38:46.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.209+0000 7f6b92393700 1 -- start start 2023-10-09T06:38:46.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.210+0000 7f6b92393700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6b8c159890 0x7f6b8c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:46.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.210+0000 7f6b92393700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b8c15a160 con 0x7f6b8c159c60 2023-10-09T06:38:46.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.210+0000 7f6b92393700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b8c0d13a0 con 0x7f6b8c069560 2023-10-09T06:38:46.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.210+0000 7f6b92393700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b8c15e140 con 0x7f6b8c159890 2023-10-09T06:38:46.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.210+0000 7f6b8b7fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f6b8c159c60 0x7f6b8c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:56370/0 (socket says 172.21.15.163:56370) 2023-10-09T06:38:46.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.210+0000 7f6b8b7fe700 1 -- 172.21.15.163:0/1954091846 learned_addr learned my addr 172.21.15.163:0/1954091846 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:46.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.210+0000 7f6b8affd700 1 -- 172.21.15.163:0/1954091846 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 453611811 0 0) 0x7f6b8c15a160 con 0x7f6b8c159c60 2023-10-09T06:38:46.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.210+0000 7f6b8bfff700 1 --2- 172.21.15.163:0/1954091846 >> v2:172.21.15.181:3300/0 conn(0x7f6b8c159890 0x7f6b8c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:46.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.210+0000 7f6b8affd700 1 -- 172.21.15.163:0/1954091846 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b74003570 con 0x7f6b8c159c60 2023-10-09T06:38:46.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.210+0000 7f6b8affd700 1 -- 172.21.15.163:0/1954091846 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3044827735 0 0) 0x7f6b74003570 con 0x7f6b8c159c60 2023-10-09T06:38:46.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.210+0000 7f6b8affd700 1 -- 172.21.15.163:0/1954091846 >> v2:172.21.15.181:3300/0 conn(0x7f6b8c159890 msgr2=0x7f6b8c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:46.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.210+0000 7f6b8affd700 1 --2- 172.21.15.163:0/1954091846 >> v2:172.21.15.181:3300/0 conn(0x7f6b8c159890 0x7f6b8c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:46.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.210+0000 7f6b8affd700 1 -- 172.21.15.163:0/1954091846 >> v1:172.21.15.181:6789/0 conn(0x7f6b8c069560 legacy=0x7f6b8c0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:46.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.210+0000 7f6b8affd700 1 -- 172.21.15.163:0/1954091846 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b8c15a3d0 con 0x7f6b8c159c60 2023-10-09T06:38:46.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.210+0000 7f6b8affd700 1 -- 172.21.15.163:0/1954091846 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f6b7c001470 con 0x7f6b8c159c60 2023-10-09T06:38:46.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.210+0000 7f6b8affd700 1 -- 172.21.15.163:0/1954091846 >> v1:172.21.15.163:6789/0 conn(0x7f6b8c159c60 legacy=0x7f6b8c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:46.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.210+0000 7f6b8affd700 1 --2- 172.21.15.163:0/1954091846 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b74003a60 0x7f6b74003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:46.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.210+0000 7f6b8affd700 1 --2- 172.21.15.163:0/1954091846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b74004810 0x7f6b74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:46.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.210+0000 7f6b8affd700 1 --2- 172.21.15.163:0/1954091846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b740051c0 0x7f6b7400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:46.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.210+0000 7f6b8affd700 1 -- 172.21.15.163:0/1954091846 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8c15e140 con 0x7f6b740051c0 2023-10-09T06:38:46.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.210+0000 7f6b8affd700 1 -- 172.21.15.163:0/1954091846 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7400c0d0 con 0x7f6b74004810 2023-10-09T06:38:46.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.210+0000 7f6b8affd700 1 -- 172.21.15.163:0/1954091846 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7400c290 con 0x7f6b74003a60 2023-10-09T06:38:46.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.211+0000 7f6b90930700 1 --2- 172.21.15.163:0/1954091846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b740051c0 0x7f6b7400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:46.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.211+0000 7f6b8b7fe700 1 --2- 172.21.15.163:0/1954091846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b74004810 0x7f6b74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:46.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.211+0000 7f6b8bfff700 1 --2- 172.21.15.163:0/1954091846 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b74003a60 0x7f6b74003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:46.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.211+0000 7f6b90930700 1 -- 172.21.15.163:0/1954091846 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b74003a60 msgr2=0x7f6b74003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:46.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.211+0000 7f6b90930700 1 --2- 172.21.15.163:0/1954091846 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b74003a60 0x7f6b74003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:46.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.211+0000 7f6b90930700 1 -- 172.21.15.163:0/1954091846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b74004810 msgr2=0x7f6b74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:46.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.211+0000 7f6b90930700 1 --2- 172.21.15.163:0/1954091846 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b74004810 0x7f6b74004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:46.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.211+0000 7f6b90930700 1 -- 172.21.15.163:0/1954091846 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b74003570 con 0x7f6b740051c0 2023-10-09T06:38:46.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.211+0000 7f6b90930700 1 -- 172.21.15.163:0/1954091846 reap_dead start 2023-10-09T06:38:46.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.211+0000 7f6b90930700 1 --2- 172.21.15.163:0/1954091846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b740051c0 0x7f6b7400ba60 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f6b84001fc0 tx=0x7f6b84001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875c89a9fb341f88 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:46.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.211+0000 7f6b8affd700 1 -- 172.21.15.163:0/1954091846 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b84001950 con 0x7f6b740051c0 2023-10-09T06:38:46.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.211+0000 7f6b8affd700 1 -- 172.21.15.163:0/1954091846 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8400c8b0 con 0x7f6b740051c0 2023-10-09T06:38:46.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.211+0000 7f6b8affd700 1 -- 172.21.15.163:0/1954091846 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8400cc90 con 0x7f6b740051c0 2023-10-09T06:38:46.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.212+0000 7f6b92393700 1 -- 172.21.15.163:0/1954091846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b740051c0 msgr2=0x7f6b7400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:46.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.212+0000 7f6b92393700 1 --2- 172.21.15.163:0/1954091846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b740051c0 0x7f6b7400ba60 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f6b84001fc0 tx=0x7f6b84001640 comp rx=0 tx=0).stop 2023-10-09T06:38:46.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.212+0000 7f6b92393700 1 -- 172.21.15.163:0/1954091846 shutdown_connections 2023-10-09T06:38:46.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.212+0000 7f6b92393700 1 --2- 172.21.15.163:0/1954091846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b740051c0 0x7f6b7400ba60 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:46.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.212+0000 7f6b92393700 1 -- 172.21.15.163:0/1954091846 >> 172.21.15.163:0/1954091846 conn(0x7f6b8c068e20 msgr2=0x7f6b8c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:46.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.212+0000 7f6b92393700 1 -- 172.21.15.163:0/1954091846 shutdown_connections 2023-10-09T06:38:46.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.212+0000 7f6b92393700 1 -- 172.21.15.163:0/1954091846 wait complete. 2023-10-09T06:38:46.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.213+0000 7f6b92393700 1 Processor -- start 2023-10-09T06:38:46.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.213+0000 7f6b92393700 1 -- start start 2023-10-09T06:38:46.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.214+0000 7f6b92393700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b740051c0 0x7f6b8c0cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:46.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.214+0000 7f6b92393700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b8c0b5f80 0x7f6b8c151d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:46.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.214+0000 7f6b92393700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b8c0cf3f0 0x7f6b8c0cf870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:46.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.214+0000 7f6b92393700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8c0cfdb0 con 0x7f6b740051c0 2023-10-09T06:38:46.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.214+0000 7f6b92393700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8c0c7cb0 con 0x7f6b8c0cf3f0 2023-10-09T06:38:46.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.214+0000 7f6b92393700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8c1522b0 con 0x7f6b8c0b5f80 2023-10-09T06:38:46.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.214+0000 7f6b8bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b740051c0 0x7f6b8c0cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:46.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.214+0000 7f6b8bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b740051c0 0x7f6b8c0cdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38686/0 (socket says 172.21.15.163:38686) 2023-10-09T06:38:46.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.214+0000 7f6b8bfff700 1 -- 172.21.15.163:0/3353161332 learned_addr learned my addr 172.21.15.163:0/3353161332 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:46.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.214+0000 7f6b8b7fe700 1 --2- 172.21.15.163:0/3353161332 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b8c0b5f80 0x7f6b8c151d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:46.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.214+0000 7f6b90930700 1 --2- 172.21.15.163:0/3353161332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b8c0cf3f0 0x7f6b8c0cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:46.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.214+0000 7f6b8bfff700 1 -- 172.21.15.163:0/3353161332 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b8c0b5f80 msgr2=0x7f6b8c151d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:46.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.214+0000 7f6b8bfff700 1 --2- 172.21.15.163:0/3353161332 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b8c0b5f80 0x7f6b8c151d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:46.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.214+0000 7f6b8bfff700 1 -- 172.21.15.163:0/3353161332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b8c0cf3f0 msgr2=0x7f6b8c0cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:46.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.214+0000 7f6b8bfff700 1 --2- 172.21.15.163:0/3353161332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b8c0cf3f0 0x7f6b8c0cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:46.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.214+0000 7f6b8bfff700 1 -- 172.21.15.163:0/3353161332 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b8c0b63f0 con 0x7f6b740051c0 2023-10-09T06:38:46.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.214+0000 7f6b8bfff700 1 --2- 172.21.15.163:0/3353161332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b740051c0 0x7f6b8c0cdad0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f6b80011b10 tx=0x7f6b800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:46.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.214+0000 7f6b73fff700 1 -- 172.21.15.163:0/3353161332 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8000f8e0 con 0x7f6b740051c0 2023-10-09T06:38:46.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.214+0000 7f6b73fff700 1 -- 172.21.15.163:0/3353161332 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8000fa40 con 0x7f6b740051c0 2023-10-09T06:38:46.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.214+0000 7f6b92393700 1 -- 172.21.15.163:0/3353161332 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b8c0ce790 con 0x7f6b740051c0 2023-10-09T06:38:46.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.214+0000 7f6b73fff700 1 -- 172.21.15.163:0/3353161332 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8000fdc0 con 0x7f6b740051c0 2023-10-09T06:38:46.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.214+0000 7f6b92393700 1 -- 172.21.15.163:0/3353161332 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b8c0ce4b0 con 0x7f6b740051c0 2023-10-09T06:38:46.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.215+0000 7f6b92393700 1 -- 172.21.15.163:0/3353161332 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b8c04c3c0 con 0x7f6b740051c0 2023-10-09T06:38:46.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.216+0000 7f6b73fff700 1 -- 172.21.15.163:0/3353161332 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6b80014650 con 0x7f6b740051c0 2023-10-09T06:38:46.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.216+0000 7f6b73fff700 1 --2- 172.21.15.163:0/3353161332 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6b78067320 0x7f6b780697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:46.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.216+0000 7f6b73fff700 1 -- 172.21.15.163:0/3353161332 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6b80018030 con 0x7f6b740051c0 2023-10-09T06:38:46.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.216+0000 7f6b8b7fe700 1 --2- 172.21.15.163:0/3353161332 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6b78067320 0x7f6b780697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:46.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.217+0000 7f6b8b7fe700 1 --2- 172.21.15.163:0/3353161332 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6b78067320 0x7f6b780697d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c007f70 tx=0x7f6b7c001e50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:46.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.220+0000 7f6b73fff700 1 -- 172.21.15.163:0/3353161332 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6b800521b0 con 0x7f6b740051c0 2023-10-09T06:38:46.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.477+0000 7f6b92393700 1 -- 172.21.15.163:0/3353161332 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6b8c06a5b0 con 0x7f6b740051c0 2023-10-09T06:38:46.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.477+0000 7f6b73fff700 1 -- 172.21.15.163:0/3353161332 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6b800557d0 con 0x7f6b740051c0 2023-10-09T06:38:46.479 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:46.479 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:38:46.479 INFO:teuthology.orchestra.run.smithi163.stdout: "auth_dump": [ 2023-10-09T06:38:46.479 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:46.479 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.a", 2023-10-09T06:38:46.479 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNltXtwDxAAlnrWRsfnZDLEmsJVowh3Mg==", 2023-10-09T06:38:46.479 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:46.479 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:38:46.480 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:38:46.480 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:38:46.480 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:46.480 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:46.480 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:46.480 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:46.480 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.b", 2023-10-09T06:38:46.480 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlTdGhGhAAXuldy0HrFd+44sSu/EV3qA==", 2023-10-09T06:38:46.480 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:46.481 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:38:46.481 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:38:46.481 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:38:46.481 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:46.481 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:46.481 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:46.481 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:46.481 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.c", 2023-10-09T06:38:46.481 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlGis7FhAAYG76TiOFX8QkTwjNX2+Ncw==", 2023-10-09T06:38:46.481 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:46.482 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:38:46.482 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:38:46.482 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:38:46.482 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:46.482 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:46.482 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:46.482 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:46.482 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.d", 2023-10-09T06:38:46.482 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlKhRuIRAAfhG5Gpc7klXJloYXepOhsg==", 2023-10-09T06:38:46.482 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:46.482 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:38:46.483 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:38:46.483 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:38:46.483 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:46.483 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:46.483 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:46.483 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:46.483 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.0", 2023-10-09T06:38:46.483 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDcnSNlrQeFOxAA+s6UitrTn5TK1N7kgySoXw==", 2023-10-09T06:38:46.483 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:46.483 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:38:46.483 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:38:46.483 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:46.483 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:46.484 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:46.484 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:46.484 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.1", 2023-10-09T06:38:46.484 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDfnSNlleccORAAnB+lNUFblivdoY44JfELxA==", 2023-10-09T06:38:46.484 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:46.484 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:38:46.484 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:38:46.484 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:46.484 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:46.484 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:46.484 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:46.484 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.2", 2023-10-09T06:38:46.485 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDinSNlFoB4NxAAT0+trRBij5rqgc+F0eWUfQ==", 2023-10-09T06:38:46.485 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:46.485 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:38:46.485 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:38:46.485 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:46.485 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:46.485 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:46.485 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:46.485 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.3", 2023-10-09T06:38:46.485 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDlnSNlRRQ6NhAABoBtrFjSvQujolq0Pxrd1Q==", 2023-10-09T06:38:46.485 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:46.485 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:38:46.486 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:38:46.486 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:46.486 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:46.486 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:46.486 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:46.486 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.4", 2023-10-09T06:38:46.486 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDrnSNl308uOxAA5OJyPLGQmVSJqha5d+nKKQ==", 2023-10-09T06:38:46.486 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:46.486 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:38:46.486 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:38:46.486 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:46.486 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:46.487 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:46.487 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:46.487 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.5", 2023-10-09T06:38:46.487 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDunSNlHnK+OBAAU7l4dCJ82SbGSzBEm8Axzw==", 2023-10-09T06:38:46.487 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:46.487 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:38:46.487 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:38:46.487 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:46.487 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:46.487 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:46.488 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:46.488 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.6", 2023-10-09T06:38:46.488 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDxnSNlwUSLOBAAN5N+F5ho+Co8pBh4Rz+lXQ==", 2023-10-09T06:38:46.488 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:46.488 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:38:46.488 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:38:46.488 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:46.488 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:46.488 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:46.488 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:46.488 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.7", 2023-10-09T06:38:46.488 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQD0nSNl2gxGORAAoE8OLGEifcEx/km8rfSV9g==", 2023-10-09T06:38:46.488 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:46.489 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:38:46.489 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:38:46.489 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:46.489 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:46.489 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:46.489 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:46.489 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.0", 2023-10-09T06:38:46.489 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlnxY5JRAApr6czcpMdQYYtXQ9yg2q6A==", 2023-10-09T06:38:46.489 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:46.489 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:38:46.489 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow r", 2023-10-09T06:38:46.489 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-09T06:38:46.490 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:46.490 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:46.490 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:46.490 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.1", 2023-10-09T06:38:46.490 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlk6ybLRAAuShqTDAtymGdxN8KBdZDDA==", 2023-10-09T06:38:46.490 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:46.490 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:38:46.490 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:38:46.490 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:38:46.490 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:38:46.490 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:46.490 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:46.491 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:46.491 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.2", 2023-10-09T06:38:46.491 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlYyOmKhAAQmpr+mGfJH4GUa28dtlJwA==", 2023-10-09T06:38:46.491 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:46.491 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:38:46.491 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:38:46.491 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:38:46.491 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:38:46.491 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:46.491 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:46.491 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:46.491 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.3", 2023-10-09T06:38:46.491 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlD2H1MhAA4zvaXY/Gd0kndzi+QSwp7w==", 2023-10-09T06:38:46.492 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:46.492 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:38:46.492 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:38:46.492 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:38:46.492 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:38:46.492 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:46.492 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:46.492 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:46.492 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.admin", 2023-10-09T06:38:46.492 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDYnSNlsHH4LxAA85xeFEP1CUwoblaePYk5zA==", 2023-10-09T06:38:46.492 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:46.492 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:38:46.493 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:38:46.493 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:38:46.493 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:46.493 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:46.493 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:46.493 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:46.493 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mgr.x", 2023-10-09T06:38:46.493 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNl2sloDBAA0FoWN9iovCl6783o3/btiA==", 2023-10-09T06:38:46.493 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:46.493 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:38:46.493 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile mgr", 2023-10-09T06:38:46.494 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:46.494 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:46.494 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:38:46.494 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:46.494 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mgr.y", 2023-10-09T06:38:46.494 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlcFWVCBAAR2dKPg01ciTiNpSX0JIKFg==", 2023-10-09T06:38:46.494 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:38:46.494 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:38:46.494 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile mgr", 2023-10-09T06:38:46.494 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:38:46.494 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:46.494 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:46.495 INFO:teuthology.orchestra.run.smithi163.stdout: ] 2023-10-09T06:38:46.495 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:38:46.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.479+0000 7f6b92393700 1 -- 172.21.15.163:0/3353161332 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6b78067320 msgr2=0x7f6b780697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:46.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.479+0000 7f6b92393700 1 --2- 172.21.15.163:0/3353161332 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6b78067320 0x7f6b780697d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c007f70 tx=0x7f6b7c001e50 comp rx=0 tx=0).stop 2023-10-09T06:38:46.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.479+0000 7f6b92393700 1 -- 172.21.15.163:0/3353161332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b740051c0 msgr2=0x7f6b8c0cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:46.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.479+0000 7f6b92393700 1 --2- 172.21.15.163:0/3353161332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b740051c0 0x7f6b8c0cdad0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f6b80011b10 tx=0x7f6b800091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:46.496 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.479+0000 7f6b92393700 1 -- 172.21.15.163:0/3353161332 shutdown_connections 2023-10-09T06:38:46.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.479+0000 7f6b92393700 1 --2- 172.21.15.163:0/3353161332 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6b78067320 0x7f6b780697d0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:46.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.479+0000 7f6b92393700 1 --2- 172.21.15.163:0/3353161332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b740051c0 0x7f6b8c0cdad0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:46.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.479+0000 7f6b92393700 1 --2- 172.21.15.163:0/3353161332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b8c0cf3f0 0x7f6b8c0cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:46.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.479+0000 7f6b92393700 1 --2- 172.21.15.163:0/3353161332 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b8c0b5f80 0x7f6b8c151d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:46.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.479+0000 7f6b92393700 1 -- 172.21.15.163:0/3353161332 >> 172.21.15.163:0/3353161332 conn(0x7f6b8c068e20 msgr2=0x7f6b8c152ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:46.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.480+0000 7f6b92393700 1 -- 172.21.15.163:0/3353161332 shutdown_connections 2023-10-09T06:38:46.497 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.480+0000 7f6b92393700 1 -- 172.21.15.163:0/3353161332 wait complete. 2023-10-09T06:38:46.498 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:38:46.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.612+0000 7fbb6759e700 1 Processor -- start 2023-10-09T06:38:46.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.612+0000 7fbb6759e700 1 -- start start 2023-10-09T06:38:46.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.613+0000 7fbb6759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb681597f0 0x7fbb680b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:46.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.613+0000 7fbb6759e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbb6815d230 0x7fbb6815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:46.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.613+0000 7fbb6759e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb6815dbd0 con 0x7fbb68159420 2023-10-09T06:38:46.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.613+0000 7fbb6759e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb680b11d0 con 0x7fbb6815d230 2023-10-09T06:38:46.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.613+0000 7fbb6759e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb680b14b0 con 0x7fbb681597f0 2023-10-09T06:38:46.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.613+0000 7fbb66d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbb6815d230 0x7fbb6815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:46.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.613+0000 7fbb6659c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fbb68159420 0x7fbb680b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:56374/0 (socket says 172.21.15.163:56374) 2023-10-09T06:38:46.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.613+0000 7fbb6659c700 1 -- 172.21.15.163:0/450463981 learned_addr learned my addr 172.21.15.163:0/450463981 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:46.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.613+0000 7fbb65d9b700 1 --2- 172.21.15.163:0/450463981 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb681597f0 0x7fbb680b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:46.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.613+0000 7fbb66d9d700 1 -- 172.21.15.163:0/450463981 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb681597f0 msgr2=0x7fbb680b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:46.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.613+0000 7fbb66d9d700 1 --2- 172.21.15.163:0/450463981 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb681597f0 0x7fbb680b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:46.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.613+0000 7fbb66d9d700 1 -- 172.21.15.163:0/450463981 >> v1:172.21.15.163:6789/0 conn(0x7fbb68159420 legacy=0x7fbb680b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:46.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.613+0000 7fbb6559a700 1 -- 172.21.15.163:0/450463981 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 875732773 0 0) 0x7fbb58001110 con 0x7fbb68159420 2023-10-09T06:38:46.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.613+0000 7fbb66d9d700 1 -- 172.21.15.163:0/450463981 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb680b1790 con 0x7fbb6815d230 2023-10-09T06:38:46.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.613+0000 7fbb65d9b700 1 --2- 172.21.15.163:0/450463981 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb681597f0 0x7fbb680b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:46.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.613+0000 7fbb66d9d700 1 --2- 172.21.15.163:0/450463981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb6815d230 0x7fbb6815d690 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fbb60009e60 tx=0x7fbb60009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=384c29993f4e4feb server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:46.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.614+0000 7fbb6559a700 1 -- 172.21.15.163:0/450463981 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6000e070 con 0x7fbb6815d230 2023-10-09T06:38:46.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.614+0000 7fbb6559a700 1 -- 172.21.15.163:0/450463981 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6000bc30 con 0x7fbb6815d230 2023-10-09T06:38:46.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.614+0000 7fbb6559a700 1 -- 172.21.15.163:0/450463981 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb60012430 con 0x7fbb6815d230 2023-10-09T06:38:46.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.614+0000 7fbb6759e700 1 -- 172.21.15.163:0/450463981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb6815d230 msgr2=0x7fbb6815d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:46.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.614+0000 7fbb6759e700 1 --2- 172.21.15.163:0/450463981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb6815d230 0x7fbb6815d690 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fbb60009e60 tx=0x7fbb60009780 comp rx=0 tx=0).stop 2023-10-09T06:38:46.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.614+0000 7fbb6759e700 1 -- 172.21.15.163:0/450463981 shutdown_connections 2023-10-09T06:38:46.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.614+0000 7fbb6759e700 1 --2- 172.21.15.163:0/450463981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb6815d230 0x7fbb6815d690 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:46.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.614+0000 7fbb6759e700 1 --2- 172.21.15.163:0/450463981 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb681597f0 0x7fbb680b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:46.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.614+0000 7fbb6759e700 1 -- 172.21.15.163:0/450463981 >> 172.21.15.163:0/450463981 conn(0x7fbb680d0660 msgr2=0x7fbb680d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:46.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.614+0000 7fbb6759e700 1 -- 172.21.15.163:0/450463981 shutdown_connections 2023-10-09T06:38:46.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.614+0000 7fbb6759e700 1 -- 172.21.15.163:0/450463981 wait complete. 2023-10-09T06:38:46.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.615+0000 7fbb6759e700 1 Processor -- start 2023-10-09T06:38:46.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.615+0000 7fbb6759e700 1 -- start start 2023-10-09T06:38:46.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.616+0000 7fbb6759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb68159420 0x7fbb6814f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:46.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.616+0000 7fbb6759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb681597f0 0x7fbb6815a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:46.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.616+0000 7fbb6759e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb6815d230 0x7fbb681500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:46.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.616+0000 7fbb6759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6814fae0 con 0x7fbb68159420 2023-10-09T06:38:46.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.616+0000 7fbb6759e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6815bcd0 con 0x7fbb6815d230 2023-10-09T06:38:46.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.616+0000 7fbb6759e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6815b0c0 con 0x7fbb681597f0 2023-10-09T06:38:46.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.616+0000 7fbb6659c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb68159420 0x7fbb6814f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:46.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.616+0000 7fbb6659c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb68159420 0x7fbb6814f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38694/0 (socket says 172.21.15.163:38694) 2023-10-09T06:38:46.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.616+0000 7fbb6659c700 1 -- 172.21.15.163:0/3651397429 learned_addr learned my addr 172.21.15.163:0/3651397429 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:46.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.616+0000 7fbb65d9b700 1 --2- 172.21.15.163:0/3651397429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb681597f0 0x7fbb6815a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:46.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.616+0000 7fbb66d9d700 1 --2- 172.21.15.163:0/3651397429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb6815d230 0x7fbb681500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:46.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.616+0000 7fbb6659c700 1 -- 172.21.15.163:0/3651397429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb681597f0 msgr2=0x7fbb6815a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:46.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.616+0000 7fbb6659c700 1 --2- 172.21.15.163:0/3651397429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb681597f0 0x7fbb6815a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:46.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.616+0000 7fbb6659c700 1 -- 172.21.15.163:0/3651397429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb6815d230 msgr2=0x7fbb681500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:46.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.616+0000 7fbb6659c700 1 --2- 172.21.15.163:0/3651397429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb6815d230 0x7fbb681500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:46.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.616+0000 7fbb6659c700 1 -- 172.21.15.163:0/3651397429 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb60009a20 con 0x7fbb68159420 2023-10-09T06:38:46.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.616+0000 7fbb65d9b700 1 --2- 172.21.15.163:0/3651397429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb681597f0 0x7fbb6815a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:46.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.616+0000 7fbb66d9d700 1 --2- 172.21.15.163:0/3651397429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb6815d230 0x7fbb681500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:46.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.617+0000 7fbb6659c700 1 --2- 172.21.15.163:0/3651397429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb68159420 0x7fbb6814f5a0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800c810 tx=0x7fbb58007c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:46.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.617+0000 7fbb567fc700 1 -- 172.21.15.163:0/3651397429 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb58005b30 con 0x7fbb68159420 2023-10-09T06:38:46.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.617+0000 7fbb567fc700 1 -- 172.21.15.163:0/3651397429 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb58005c90 con 0x7fbb68159420 2023-10-09T06:38:46.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.617+0000 7fbb6759e700 1 -- 172.21.15.163:0/3651397429 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb6815b3a0 con 0x7fbb68159420 2023-10-09T06:38:46.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.617+0000 7fbb567fc700 1 -- 172.21.15.163:0/3651397429 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb58007300 con 0x7fbb68159420 2023-10-09T06:38:46.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.617+0000 7fbb6759e700 1 -- 172.21.15.163:0/3651397429 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb6815b960 con 0x7fbb68159420 2023-10-09T06:38:46.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.618+0000 7fbb6759e700 1 -- 172.21.15.163:0/3651397429 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb6814ff00 con 0x7fbb68159420 2023-10-09T06:38:46.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.618+0000 7fbb567fc700 1 -- 172.21.15.163:0/3651397429 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbb5801b400 con 0x7fbb68159420 2023-10-09T06:38:46.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.618+0000 7fbb567fc700 1 --2- 172.21.15.163:0/3651397429 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb48067320 0x7fbb480697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:46.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.618+0000 7fbb567fc700 1 -- 172.21.15.163:0/3651397429 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fbb580842a0 con 0x7fbb68159420 2023-10-09T06:38:46.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.619+0000 7fbb65d9b700 1 --2- 172.21.15.163:0/3651397429 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb48067320 0x7fbb480697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:46.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.619+0000 7fbb65d9b700 1 --2- 172.21.15.163:0/3651397429 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb48067320 0x7fbb480697d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fbb680b3ee0 tx=0x7fbb5c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:46.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.623+0000 7fbb567fc700 1 -- 172.21.15.163:0/3651397429 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb58051430 con 0x7fbb68159420 2023-10-09T06:38:46.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.809+0000 7fbb6759e700 1 -- 172.21.15.163:0/3651397429 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb6815dbd0 con 0x7fbb68159420 2023-10-09T06:38:46.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.810+0000 7fbb567fc700 1 -- 172.21.15.163:0/3651397429 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+12192 (secure 0 0 0) 0x7fbb6815dbd0 con 0x7fbb68159420 2023-10-09T06:38:46.810 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:46.811 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":160,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:38:46.062194+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:38:46.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.812+0000 7fbb6759e700 1 -- 172.21.15.163:0/3651397429 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb48067320 msgr2=0x7fbb480697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:46.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.812+0000 7fbb6759e700 1 --2- 172.21.15.163:0/3651397429 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb48067320 0x7fbb480697d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fbb680b3ee0 tx=0x7fbb5c008040 comp rx=0 tx=0).stop 2023-10-09T06:38:46.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.812+0000 7fbb6759e700 1 -- 172.21.15.163:0/3651397429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb68159420 msgr2=0x7fbb6814f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:46.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.812+0000 7fbb6759e700 1 --2- 172.21.15.163:0/3651397429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb68159420 0x7fbb6814f5a0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800c810 tx=0x7fbb58007c50 comp rx=0 tx=0).stop 2023-10-09T06:38:46.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.812+0000 7fbb6759e700 1 -- 172.21.15.163:0/3651397429 shutdown_connections 2023-10-09T06:38:46.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.812+0000 7fbb6759e700 1 --2- 172.21.15.163:0/3651397429 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb48067320 0x7fbb480697d0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:46.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.812+0000 7fbb6759e700 1 --2- 172.21.15.163:0/3651397429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb68159420 0x7fbb6814f5a0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:46.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.812+0000 7fbb6759e700 1 --2- 172.21.15.163:0/3651397429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb6815d230 0x7fbb681500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:46.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.812+0000 7fbb6759e700 1 --2- 172.21.15.163:0/3651397429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb681597f0 0x7fbb6815a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:46.814 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.812+0000 7fbb6759e700 1 -- 172.21.15.163:0/3651397429 >> 172.21.15.163:0/3651397429 conn(0x7fbb680d0660 msgr2=0x7fbb681570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:46.814 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.812+0000 7fbb6759e700 1 -- 172.21.15.163:0/3651397429 shutdown_connections 2023-10-09T06:38:46.814 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.812+0000 7fbb6759e700 1 -- 172.21.15.163:0/3651397429 wait complete. 2023-10-09T06:38:46.825 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-09T06:38:46.825 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:38:46.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.942+0000 7f4858bc4700 1 Processor -- start 2023-10-09T06:38:46.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.942+0000 7f4858bc4700 1 -- start start 2023-10-09T06:38:46.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.943+0000 7f4858bc4700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f48540b4e70 0x7f48540ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:46.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.943+0000 7f4858bc4700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48540cfb00 con 0x7f48540b1b40 2023-10-09T06:38:46.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.943+0000 7f4858bc4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f485415e060 con 0x7f48540b16d0 2023-10-09T06:38:46.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.943+0000 7f4858bc4700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48540b5c40 con 0x7f48540b4e70 2023-10-09T06:38:46.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.943+0000 7f48537fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f48540b4e70 0x7f48540ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:46.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.943+0000 7f4853fff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f48540b1b40 0x7f485415d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:56386/0 (socket says 172.21.15.163:56386) 2023-10-09T06:38:46.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.943+0000 7f48537fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f48540b4e70 0x7f48540ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38704/0 (socket says 172.21.15.163:38704) 2023-10-09T06:38:46.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.943+0000 7f4853fff700 1 -- 172.21.15.163:0/1160266911 learned_addr learned my addr 172.21.15.163:0/1160266911 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:46.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.943+0000 7f48537fe700 1 -- 172.21.15.163:0/1160266911 >> v1:172.21.15.181:6789/0 conn(0x7f48540b16d0 legacy=0x7f485415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:46.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.943+0000 7f48537fe700 1 -- 172.21.15.163:0/1160266911 >> v1:172.21.15.163:6789/0 conn(0x7f48540b1b40 legacy=0x7f485415d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:46.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.943+0000 7f48537fe700 1 -- 172.21.15.163:0/1160266911 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48540b5f20 con 0x7f48540b4e70 2023-10-09T06:38:46.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.943+0000 7f48527fc700 1 -- 172.21.15.163:0/1160266911 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3358591613 0 0) 0x7f48540cfb00 con 0x7f48540b1b40 2023-10-09T06:38:46.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.943+0000 7f48537fe700 1 --2- 172.21.15.163:0/1160266911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48540b4e70 0x7f48540ccff0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f48440098e0 tx=0x7f48440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0dc0fedfbc6cb86 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:46.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.944+0000 7f48527fc700 1 -- 172.21.15.163:0/1160266911 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4844009e80 con 0x7f48540b4e70 2023-10-09T06:38:46.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.944+0000 7f48527fc700 1 -- 172.21.15.163:0/1160266911 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f484400b820 con 0x7f48540b4e70 2023-10-09T06:38:46.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.944+0000 7f48527fc700 1 -- 172.21.15.163:0/1160266911 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484400bba0 con 0x7f48540b4e70 2023-10-09T06:38:46.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.944+0000 7f4858bc4700 1 -- 172.21.15.163:0/1160266911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48540b4e70 msgr2=0x7f48540ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:46.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.944+0000 7f4858bc4700 1 --2- 172.21.15.163:0/1160266911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48540b4e70 0x7f48540ccff0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f48440098e0 tx=0x7f48440091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:46.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.944+0000 7f4858bc4700 1 -- 172.21.15.163:0/1160266911 shutdown_connections 2023-10-09T06:38:46.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.944+0000 7f4858bc4700 1 --2- 172.21.15.163:0/1160266911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48540b4e70 0x7f48540ccff0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:46.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.944+0000 7f4858bc4700 1 -- 172.21.15.163:0/1160266911 >> 172.21.15.163:0/1160266911 conn(0x7f48540d10c0 msgr2=0x7f48540affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:46.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.944+0000 7f4858bc4700 1 -- 172.21.15.163:0/1160266911 shutdown_connections 2023-10-09T06:38:46.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.944+0000 7f4858bc4700 1 -- 172.21.15.163:0/1160266911 wait complete. 2023-10-09T06:38:46.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.945+0000 7f4858bc4700 1 Processor -- start 2023-10-09T06:38:46.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.945+0000 7f4858bc4700 1 -- start start 2023-10-09T06:38:46.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.946+0000 7f4858bc4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48540b16d0 0x7f485414c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:46.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.946+0000 7f4858bc4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48540b1b40 0x7f485416a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:46.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.946+0000 7f4858bc4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48540b4e70 0x7f48540c6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:46.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.946+0000 7f4858bc4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48540c94c0 con 0x7f48540b1b40 2023-10-09T06:38:46.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.946+0000 7f4858bc4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48540c88b0 con 0x7f48540b16d0 2023-10-09T06:38:46.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.946+0000 7f4858bc4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48540c8b90 con 0x7f48540b4e70 2023-10-09T06:38:46.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.946+0000 7f4852ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48540b1b40 0x7f485416a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:46.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.946+0000 7f4852ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48540b1b40 0x7f485416a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38718/0 (socket says 172.21.15.163:38718) 2023-10-09T06:38:46.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.946+0000 7f4852ffd700 1 -- 172.21.15.163:0/1255760423 learned_addr learned my addr 172.21.15.163:0/1255760423 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:46.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.946+0000 7f48537fe700 1 --2- 172.21.15.163:0/1255760423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48540b16d0 0x7f485414c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:46.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.946+0000 7f4853fff700 1 --2- 172.21.15.163:0/1255760423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48540b4e70 0x7f48540c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:46.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.946+0000 7f4852ffd700 1 -- 172.21.15.163:0/1255760423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48540b4e70 msgr2=0x7f48540c6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:46.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.946+0000 7f4852ffd700 1 --2- 172.21.15.163:0/1255760423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48540b4e70 0x7f48540c6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:46.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.946+0000 7f4852ffd700 1 -- 172.21.15.163:0/1255760423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48540b16d0 msgr2=0x7f485414c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:46.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.946+0000 7f4852ffd700 1 --2- 172.21.15.163:0/1255760423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48540b16d0 0x7f485414c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:46.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.946+0000 7f4852ffd700 1 -- 172.21.15.163:0/1255760423 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4844009420 con 0x7f48540b1b40 2023-10-09T06:38:46.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.946+0000 7f4853fff700 1 --2- 172.21.15.163:0/1255760423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48540b4e70 0x7f48540c6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:46.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.947+0000 7f4852ffd700 1 --2- 172.21.15.163:0/1255760423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48540b1b40 0x7f485416a310 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f484800c5d0 tx=0x7f4848007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:46.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.947+0000 7f483b7fe700 1 -- 172.21.15.163:0/1255760423 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4848011590 con 0x7f48540b1b40 2023-10-09T06:38:46.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.947+0000 7f483b7fe700 1 -- 172.21.15.163:0/1255760423 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4848011710 con 0x7f48540b1b40 2023-10-09T06:38:46.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.947+0000 7f483b7fe700 1 -- 172.21.15.163:0/1255760423 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4848011ad0 con 0x7f48540b1b40 2023-10-09T06:38:46.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.947+0000 7f4858bc4700 1 -- 172.21.15.163:0/1255760423 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48540c8e70 con 0x7f48540b1b40 2023-10-09T06:38:46.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.947+0000 7f4858bc4700 1 -- 172.21.15.163:0/1255760423 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f485414c580 con 0x7f48540b1b40 2023-10-09T06:38:46.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.948+0000 7f4858bc4700 1 -- 172.21.15.163:0/1255760423 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48540c9630 con 0x7f48540b1b40 2023-10-09T06:38:46.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.948+0000 7f483b7fe700 1 -- 172.21.15.163:0/1255760423 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f484801c460 con 0x7f48540b1b40 2023-10-09T06:38:46.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.949+0000 7f483b7fe700 1 --2- 172.21.15.163:0/1255760423 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4840067320 0x7f48400697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:46.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.949+0000 7f483b7fe700 1 -- 172.21.15.163:0/1255760423 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4848084720 con 0x7f48540b1b40 2023-10-09T06:38:46.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.949+0000 7f48537fe700 1 --2- 172.21.15.163:0/1255760423 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4840067320 0x7f48400697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:46.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.949+0000 7f48537fe700 1 --2- 172.21.15.163:0/1255760423 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4840067320 0x7f48400697d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f48440030c0 tx=0x7f4844016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:46.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:46.953+0000 7f483b7fe700 1 -- 172.21.15.163:0/1255760423 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4848051700 con 0x7f48540b1b40 2023-10-09T06:38:47.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.136+0000 7f4858bc4700 1 -- 172.21.15.163:0/1255760423 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4854157690 con 0x7f48540b1b40 2023-10-09T06:38:47.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.136+0000 7f483b7fe700 1 -- 172.21.15.163:0/1255760423 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v160) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4848054d20 con 0x7f48540b1b40 2023-10-09T06:38:47.137 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:38:47.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.138+0000 7f4858bc4700 1 -- 172.21.15.163:0/1255760423 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4840067320 msgr2=0x7f48400697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:47.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.138+0000 7f4858bc4700 1 --2- 172.21.15.163:0/1255760423 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4840067320 0x7f48400697d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f48440030c0 tx=0x7f4844016000 comp rx=0 tx=0).stop 2023-10-09T06:38:47.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.138+0000 7f4858bc4700 1 -- 172.21.15.163:0/1255760423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48540b1b40 msgr2=0x7f485416a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:47.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.138+0000 7f4858bc4700 1 --2- 172.21.15.163:0/1255760423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48540b1b40 0x7f485416a310 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f484800c5d0 tx=0x7f4848007f60 comp rx=0 tx=0).stop 2023-10-09T06:38:47.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.138+0000 7f4858bc4700 1 -- 172.21.15.163:0/1255760423 shutdown_connections 2023-10-09T06:38:47.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.138+0000 7f4858bc4700 1 --2- 172.21.15.163:0/1255760423 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4840067320 0x7f48400697d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:47.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.138+0000 7f4858bc4700 1 --2- 172.21.15.163:0/1255760423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48540b4e70 0x7f48540c6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:47.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.138+0000 7f4858bc4700 1 --2- 172.21.15.163:0/1255760423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48540b16d0 0x7f485414c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:47.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.138+0000 7f4858bc4700 1 --2- 172.21.15.163:0/1255760423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48540b1b40 0x7f485416a310 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:47.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.138+0000 7f4858bc4700 1 -- 172.21.15.163:0/1255760423 >> 172.21.15.163:0/1255760423 conn(0x7f48540d10c0 msgr2=0x7f48540ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:47.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.138+0000 7f4858bc4700 1 -- 172.21.15.163:0/1255760423 shutdown_connections 2023-10-09T06:38:47.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.138+0000 7f4858bc4700 1 -- 172.21.15.163:0/1255760423 wait complete. 2023-10-09T06:38:47.151 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-09T06:38:47.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.265+0000 7fdd5a143700 1 Processor -- start 2023-10-09T06:38:47.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.265+0000 7fdd5a143700 1 -- start start 2023-10-09T06:38:47.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.266+0000 7fdd5a143700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdd54159890 0x7fdd5415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:47.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.266+0000 7fdd5a143700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdd54069560 0x7fdd540699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:47.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.266+0000 7fdd5a143700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd54069ee0 con 0x7fdd54159c60 2023-10-09T06:38:47.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.266+0000 7fdd5a143700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd5415a160 con 0x7fdd54159890 2023-10-09T06:38:47.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.266+0000 7fdd5a143700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd540d13a0 con 0x7fdd54069560 2023-10-09T06:38:47.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.266+0000 7fdd537fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdd54159890 0x7fdd5415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:47.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.266+0000 7fdd52ffd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fdd54159c60 0x7fdd5415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:56392/0 (socket says 172.21.15.163:56392) 2023-10-09T06:38:47.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.266+0000 7fdd52ffd700 1 -- 172.21.15.163:0/3740955529 learned_addr learned my addr 172.21.15.163:0/3740955529 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:47.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.266+0000 7fdd537fe700 1 -- 172.21.15.163:0/3740955529 >> v2:172.21.15.181:3300/0 conn(0x7fdd54069560 msgr2=0x7fdd540699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:47.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.266+0000 7fdd537fe700 1 --2- 172.21.15.163:0/3740955529 >> v2:172.21.15.181:3300/0 conn(0x7fdd54069560 0x7fdd540699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:47.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.266+0000 7fdd537fe700 1 -- 172.21.15.163:0/3740955529 >> v1:172.21.15.163:6789/0 conn(0x7fdd54159c60 legacy=0x7fdd5415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:47.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.266+0000 7fdd537fe700 1 -- 172.21.15.163:0/3740955529 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5415e140 con 0x7fdd54159890 2023-10-09T06:38:47.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.266+0000 7fdd527fc700 1 -- 172.21.15.163:0/3740955529 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1015144572 0 0) 0x7fdd54069ee0 con 0x7fdd54159c60 2023-10-09T06:38:47.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.267+0000 7fdd537fe700 1 --2- 172.21.15.163:0/3740955529 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd54159890 0x7fdd5415abd0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fdd5406a890 tx=0x7fdd480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5693905489b135d server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:47.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.267+0000 7fdd527fc700 1 -- 172.21.15.163:0/3740955529 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4800b6b0 con 0x7fdd54159890 2023-10-09T06:38:47.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.267+0000 7fdd527fc700 1 -- 172.21.15.163:0/3740955529 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4800b810 con 0x7fdd54159890 2023-10-09T06:38:47.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.267+0000 7fdd527fc700 1 -- 172.21.15.163:0/3740955529 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4800bbb0 con 0x7fdd54159890 2023-10-09T06:38:47.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.267+0000 7fdd5a143700 1 -- 172.21.15.163:0/3740955529 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd54159890 msgr2=0x7fdd5415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:47.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.267+0000 7fdd5a143700 1 --2- 172.21.15.163:0/3740955529 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd54159890 0x7fdd5415abd0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fdd5406a890 tx=0x7fdd480091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:47.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.267+0000 7fdd5a143700 1 -- 172.21.15.163:0/3740955529 shutdown_connections 2023-10-09T06:38:47.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.267+0000 7fdd5a143700 1 --2- 172.21.15.163:0/3740955529 >> v2:172.21.15.181:3300/0 conn(0x7fdd54069560 0x7fdd540699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:47.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.267+0000 7fdd5a143700 1 --2- 172.21.15.163:0/3740955529 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd54159890 0x7fdd5415abd0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:47.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.267+0000 7fdd5a143700 1 -- 172.21.15.163:0/3740955529 >> 172.21.15.163:0/3740955529 conn(0x7fdd54068e20 msgr2=0x7fdd540d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:47.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.267+0000 7fdd5a143700 1 -- 172.21.15.163:0/3740955529 shutdown_connections 2023-10-09T06:38:47.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.267+0000 7fdd5a143700 1 -- 172.21.15.163:0/3740955529 wait complete. 2023-10-09T06:38:47.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.268+0000 7fdd5a143700 1 Processor -- start 2023-10-09T06:38:47.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.268+0000 7fdd5a143700 1 -- start start 2023-10-09T06:38:47.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.269+0000 7fdd5a143700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd54069560 0x7fdd540b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:47.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.269+0000 7fdd5a143700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd54159890 0x7fdd5414c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:47.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.269+0000 7fdd5a143700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd54159c60 0x7fdd540cf200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:47.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.269+0000 7fdd5a143700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd540ce440 con 0x7fdd54159c60 2023-10-09T06:38:47.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.269+0000 7fdd5a143700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd540ce720 con 0x7fdd54159890 2023-10-09T06:38:47.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.269+0000 7fdd5a143700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd540cea00 con 0x7fdd54069560 2023-10-09T06:38:47.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.269+0000 7fdd53fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd54159c60 0x7fdd540cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:47.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.269+0000 7fdd53fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd54159c60 0x7fdd540cf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38740/0 (socket says 172.21.15.163:38740) 2023-10-09T06:38:47.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.269+0000 7fdd53fff700 1 -- 172.21.15.163:0/1160757873 learned_addr learned my addr 172.21.15.163:0/1160757873 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:47.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.270+0000 7fdd537fe700 1 --2- 172.21.15.163:0/1160757873 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd54069560 0x7fdd540b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:47.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.270+0000 7fdd53fff700 1 -- 172.21.15.163:0/1160757873 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd54069560 msgr2=0x7fdd540b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:47.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.270+0000 7fdd53fff700 1 --2- 172.21.15.163:0/1160757873 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd54069560 0x7fdd540b4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:47.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.270+0000 7fdd52ffd700 1 --2- 172.21.15.163:0/1160757873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd54159890 0x7fdd5414c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:47.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.270+0000 7fdd53fff700 1 -- 172.21.15.163:0/1160757873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd54159890 msgr2=0x7fdd5414c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:47.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.270+0000 7fdd53fff700 1 --2- 172.21.15.163:0/1160757873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd54159890 0x7fdd5414c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:47.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.270+0000 7fdd53fff700 1 -- 172.21.15.163:0/1160757873 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd48009420 con 0x7fdd54159c60 2023-10-09T06:38:47.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.270+0000 7fdd53fff700 1 --2- 172.21.15.163:0/1160757873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd54159c60 0x7fdd540cf200 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c00bc30 tx=0x7fdd4c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:47.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.270+0000 7fdd3b7fe700 1 -- 172.21.15.163:0/1160757873 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c0092e0 con 0x7fdd54159c60 2023-10-09T06:38:47.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.270+0000 7fdd3b7fe700 1 -- 172.21.15.163:0/1160757873 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c00c7a0 con 0x7fdd54159c60 2023-10-09T06:38:47.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.270+0000 7fdd5a143700 1 -- 172.21.15.163:0/1160757873 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd540cece0 con 0x7fdd54159c60 2023-10-09T06:38:47.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.270+0000 7fdd3b7fe700 1 -- 172.21.15.163:0/1160757873 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c00cb80 con 0x7fdd54159c60 2023-10-09T06:38:47.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.270+0000 7fdd5a143700 1 -- 172.21.15.163:0/1160757873 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5414c5e0 con 0x7fdd54159c60 2023-10-09T06:38:47.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.271+0000 7fdd5a143700 1 -- 172.21.15.163:0/1160757873 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd540cef20 con 0x7fdd54159c60 2023-10-09T06:38:47.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.271+0000 7fdd3b7fe700 1 -- 172.21.15.163:0/1160757873 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fdd4c012070 con 0x7fdd54159c60 2023-10-09T06:38:47.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.272+0000 7fdd3b7fe700 1 --2- 172.21.15.163:0/1160757873 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdd40067320 0x7fdd400697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:47.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.272+0000 7fdd3b7fe700 1 -- 172.21.15.163:0/1160757873 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fdd4c084610 con 0x7fdd54159c60 2023-10-09T06:38:47.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.272+0000 7fdd537fe700 1 --2- 172.21.15.163:0/1160757873 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdd40067320 0x7fdd400697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:47.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.272+0000 7fdd537fe700 1 --2- 172.21.15.163:0/1160757873 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdd40067320 0x7fdd400697d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800c010 tx=0x7fdd48017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:47.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.276+0000 7fdd3b7fe700 1 -- 172.21.15.163:0/1160757873 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd540cef20 con 0x7fdd54159c60 2023-10-09T06:38:47.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.460+0000 7fdd5a143700 1 -- 172.21.15.163:0/1160757873 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdd540cef20 con 0x7fdd54159c60 2023-10-09T06:38:47.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.460+0000 7fdd3b7fe700 1 -- 172.21.15.163:0/1160757873 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v160) v1 ==== 84+0+65 (secure 0 0 0) 0x7fdd540cef20 con 0x7fdd54159c60 2023-10-09T06:38:47.461 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:47.461 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:38:47.462 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:47.462 INFO:teuthology.orchestra.run.smithi163.stdout: "poolnum": 1, 2023-10-09T06:38:47.462 INFO:teuthology.orchestra.run.smithi163.stdout: "poolname": ".mgr" 2023-10-09T06:38:47.462 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:47.462 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:38:47.463 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.462+0000 7fdd5a143700 1 -- 172.21.15.163:0/1160757873 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdd40067320 msgr2=0x7fdd400697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:47.463 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.462+0000 7fdd5a143700 1 --2- 172.21.15.163:0/1160757873 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdd40067320 0x7fdd400697d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800c010 tx=0x7fdd48017000 comp rx=0 tx=0).stop 2023-10-09T06:38:47.463 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.462+0000 7fdd5a143700 1 -- 172.21.15.163:0/1160757873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd54159c60 msgr2=0x7fdd540cf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:47.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.462+0000 7fdd5a143700 1 --2- 172.21.15.163:0/1160757873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd54159c60 0x7fdd540cf200 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c00bc30 tx=0x7fdd4c009840 comp rx=0 tx=0).stop 2023-10-09T06:38:47.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.462+0000 7fdd5a143700 1 -- 172.21.15.163:0/1160757873 shutdown_connections 2023-10-09T06:38:47.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.462+0000 7fdd5a143700 1 --2- 172.21.15.163:0/1160757873 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdd40067320 0x7fdd400697d0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:47.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.462+0000 7fdd5a143700 1 --2- 172.21.15.163:0/1160757873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd54159c60 0x7fdd540cf200 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:47.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.462+0000 7fdd5a143700 1 --2- 172.21.15.163:0/1160757873 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd54069560 0x7fdd540b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:47.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.462+0000 7fdd5a143700 1 --2- 172.21.15.163:0/1160757873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd54159890 0x7fdd5414c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:47.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.462+0000 7fdd5a143700 1 -- 172.21.15.163:0/1160757873 >> 172.21.15.163:0/1160757873 conn(0x7fdd54068e20 msgr2=0x7fdd540b07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:47.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.462+0000 7fdd5a143700 1 -- 172.21.15.163:0/1160757873 shutdown_connections 2023-10-09T06:38:47.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.462+0000 7fdd5a143700 1 -- 172.21.15.163:0/1160757873 wait complete. 2023-10-09T06:38:47.475 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-09T06:38:47.475 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-09T06:38:47.594 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.593+0000 7fa791fb5700 1 Processor -- start 2023-10-09T06:38:47.594 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.593+0000 7fa791fb5700 1 -- start start 2023-10-09T06:38:47.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.594+0000 7fa791fb5700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa78c0b03c0 0x7fa78c0cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:47.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.594+0000 7fa791fb5700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa78c0cfa00 con 0x7fa78c1591e0 2023-10-09T06:38:47.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.594+0000 7fa791fb5700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa78c0cb580 con 0x7fa78c0b0790 2023-10-09T06:38:47.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.594+0000 7fa791fb5700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa78c0cf160 con 0x7fa78c0b03c0 2023-10-09T06:38:47.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.594+0000 7fa790fb3700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa78c0b03c0 0x7fa78c0cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:47.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.594+0000 7fa7917b4700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fa78c1591e0 0x7fa78c159600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:56398/0 (socket says 172.21.15.163:56398) 2023-10-09T06:38:47.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.594+0000 7fa7917b4700 1 -- 172.21.15.163:0/3788572831 learned_addr learned my addr 172.21.15.163:0/3788572831 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:47.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.594+0000 7fa790fb3700 1 -- 172.21.15.163:0/3788572831 >> v1:172.21.15.181:6789/0 conn(0x7fa78c0b0790 legacy=0x7fa78c0b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:47.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.594+0000 7fa790fb3700 1 -- 172.21.15.163:0/3788572831 >> v1:172.21.15.163:6789/0 conn(0x7fa78c1591e0 legacy=0x7fa78c159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:47.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.594+0000 7fa790fb3700 1 -- 172.21.15.163:0/3788572831 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa78c0cf440 con 0x7fa78c0b03c0 2023-10-09T06:38:47.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.594+0000 7fa78b7fe700 1 -- 172.21.15.163:0/3788572831 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 692247811 0 0) 0x7fa78c0cfa00 con 0x7fa78c1591e0 2023-10-09T06:38:47.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.595+0000 7fa790fb3700 1 --2- 172.21.15.163:0/3788572831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa78c0b03c0 0x7fa78c0cea70 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fa77c009910 tx=0x7fa77c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b9949a8cb3cd38 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:47.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.595+0000 7fa78b7fe700 1 -- 172.21.15.163:0/3788572831 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c00b820 con 0x7fa78c0b03c0 2023-10-09T06:38:47.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.595+0000 7fa78b7fe700 1 -- 172.21.15.163:0/3788572831 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77c00b980 con 0x7fa78c0b03c0 2023-10-09T06:38:47.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.595+0000 7fa78b7fe700 1 -- 172.21.15.163:0/3788572831 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c00bd20 con 0x7fa78c0b03c0 2023-10-09T06:38:47.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.595+0000 7fa791fb5700 1 -- 172.21.15.163:0/3788572831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa78c0b03c0 msgr2=0x7fa78c0cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:47.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.595+0000 7fa791fb5700 1 --2- 172.21.15.163:0/3788572831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa78c0b03c0 0x7fa78c0cea70 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fa77c009910 tx=0x7fa77c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:47.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.595+0000 7fa791fb5700 1 -- 172.21.15.163:0/3788572831 shutdown_connections 2023-10-09T06:38:47.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.595+0000 7fa791fb5700 1 --2- 172.21.15.163:0/3788572831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa78c0b03c0 0x7fa78c0cea70 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:47.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.595+0000 7fa791fb5700 1 -- 172.21.15.163:0/3788572831 >> 172.21.15.163:0/3788572831 conn(0x7fa78c063230 msgr2=0x7fa78c063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:47.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.595+0000 7fa791fb5700 1 -- 172.21.15.163:0/3788572831 shutdown_connections 2023-10-09T06:38:47.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.595+0000 7fa791fb5700 1 -- 172.21.15.163:0/3788572831 wait complete. 2023-10-09T06:38:47.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.596+0000 7fa791fb5700 1 Processor -- start 2023-10-09T06:38:47.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.597+0000 7fa791fb5700 1 -- start start 2023-10-09T06:38:47.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.597+0000 7fa791fb5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa78c0b03c0 0x7fa78c158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:47.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.597+0000 7fa791fb5700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa78c0b0790 0x7fa78c15ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:47.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.597+0000 7fa791fb5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa78c1591e0 0x7fa78c15c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:47.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.597+0000 7fa791fb5700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c0c5390 con 0x7fa78c1591e0 2023-10-09T06:38:47.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.597+0000 7fa791fb5700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c0c94b0 con 0x7fa78c0b0790 2023-10-09T06:38:47.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.597+0000 7fa791fb5700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c0c9790 con 0x7fa78c0b03c0 2023-10-09T06:38:47.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.597+0000 7fa7917b4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa78c1591e0 0x7fa78c15c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:47.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.597+0000 7fa7917b4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa78c1591e0 0x7fa78c15c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38766/0 (socket says 172.21.15.163:38766) 2023-10-09T06:38:47.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.597+0000 7fa7917b4700 1 -- 172.21.15.163:0/380749715 learned_addr learned my addr 172.21.15.163:0/380749715 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:47.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.598+0000 7fa7917b4700 1 -- 172.21.15.163:0/380749715 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa78c0b03c0 msgr2=0x7fa78c158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:47.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.598+0000 7fa7917b4700 1 --2- 172.21.15.163:0/380749715 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa78c0b03c0 0x7fa78c158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:47.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.598+0000 7fa7917b4700 1 -- 172.21.15.163:0/380749715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa78c0b0790 msgr2=0x7fa78c15ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:47.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.598+0000 7fa7917b4700 1 --2- 172.21.15.163:0/380749715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa78c0b0790 0x7fa78c15ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:47.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.598+0000 7fa7917b4700 1 -- 172.21.15.163:0/380749715 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77c009420 con 0x7fa78c1591e0 2023-10-09T06:38:47.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.598+0000 7fa7917b4700 1 --2- 172.21.15.163:0/380749715 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa78c1591e0 0x7fa78c15c280 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fa78400cbf0 tx=0x7fa7840019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:47.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.598+0000 7fa788ff9700 1 -- 172.21.15.163:0/380749715 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78400a9c0 con 0x7fa78c1591e0 2023-10-09T06:38:47.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.598+0000 7fa788ff9700 1 -- 172.21.15.163:0/380749715 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78400ab20 con 0x7fa78c1591e0 2023-10-09T06:38:47.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.598+0000 7fa788ff9700 1 -- 172.21.15.163:0/380749715 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7840070f0 con 0x7fa78c1591e0 2023-10-09T06:38:47.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.598+0000 7fa791fb5700 1 -- 172.21.15.163:0/380749715 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa78c0c9a70 con 0x7fa78c1591e0 2023-10-09T06:38:47.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.598+0000 7fa791fb5700 1 -- 172.21.15.163:0/380749715 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa78c15bf60 con 0x7fa78c1591e0 2023-10-09T06:38:47.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.599+0000 7fa791fb5700 1 -- 172.21.15.163:0/380749715 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa78c04c3c0 con 0x7fa78c1591e0 2023-10-09T06:38:47.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.599+0000 7fa788ff9700 1 -- 172.21.15.163:0/380749715 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa784007520 con 0x7fa78c1591e0 2023-10-09T06:38:47.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.600+0000 7fa788ff9700 1 --2- 172.21.15.163:0/380749715 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa778067320 0x7fa7780697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:47.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.600+0000 7fa788ff9700 1 -- 172.21.15.163:0/380749715 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fa784084d00 con 0x7fa78c1591e0 2023-10-09T06:38:47.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.600+0000 7fa790fb3700 1 --2- 172.21.15.163:0/380749715 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa778067320 0x7fa7780697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:47.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.600+0000 7fa790fb3700 1 --2- 172.21.15.163:0/380749715 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa778067320 0x7fa7780697d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa78c0b5ae0 tx=0x7fa77c017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:47.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.604+0000 7fa788ff9700 1 -- 172.21.15.163:0/380749715 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa784051d60 con 0x7fa78c1591e0 2023-10-09T06:38:47.787 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:47.786+0000 7fa791fb5700 1 -- 172.21.15.163:0/380749715 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa78c0c54d0 con 0x7fa78c1591e0 2023-10-09T06:38:48.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.091+0000 7fa788ff9700 1 -- 172.21.15.163:0/380749715 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v161) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa784055380 con 0x7fa78c1591e0 2023-10-09T06:38:48.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.094+0000 7fa791fb5700 1 -- 172.21.15.163:0/380749715 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa778067320 msgr2=0x7fa7780697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:48.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.094+0000 7fa791fb5700 1 --2- 172.21.15.163:0/380749715 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa778067320 0x7fa7780697d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa78c0b5ae0 tx=0x7fa77c017000 comp rx=0 tx=0).stop 2023-10-09T06:38:48.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.094+0000 7fa791fb5700 1 -- 172.21.15.163:0/380749715 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa78c1591e0 msgr2=0x7fa78c15c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:48.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.094+0000 7fa791fb5700 1 --2- 172.21.15.163:0/380749715 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa78c1591e0 0x7fa78c15c280 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fa78400cbf0 tx=0x7fa7840019b0 comp rx=0 tx=0).stop 2023-10-09T06:38:48.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.094+0000 7fa791fb5700 1 -- 172.21.15.163:0/380749715 shutdown_connections 2023-10-09T06:38:48.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.094+0000 7fa791fb5700 1 --2- 172.21.15.163:0/380749715 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa778067320 0x7fa7780697d0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:48.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.094+0000 7fa791fb5700 1 --2- 172.21.15.163:0/380749715 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa78c1591e0 0x7fa78c15c280 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:48.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.094+0000 7fa791fb5700 1 --2- 172.21.15.163:0/380749715 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa78c0b03c0 0x7fa78c158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:48.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.094+0000 7fa791fb5700 1 --2- 172.21.15.163:0/380749715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa78c0b0790 0x7fa78c15ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:48.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.094+0000 7fa791fb5700 1 -- 172.21.15.163:0/380749715 >> 172.21.15.163:0/380749715 conn(0x7fa78c063230 msgr2=0x7fa78c068bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:48.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.095+0000 7fa791fb5700 1 -- 172.21.15.163:0/380749715 shutdown_connections 2023-10-09T06:38:48.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.095+0000 7fa791fb5700 1 -- 172.21.15.163:0/380749715 wait complete. 2023-10-09T06:38:48.097 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_metadata' created 2023-10-09T06:38:48.110 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-09T06:38:48.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.231+0000 7fa5eaccc700 1 Processor -- start 2023-10-09T06:38:48.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.231+0000 7fa5eaccc700 1 -- start start 2023-10-09T06:38:48.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.232+0000 7fa5eaccc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa5e4159950 0x7fa5e40cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:48.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.232+0000 7fa5eaccc700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5e40ce380 con 0x7fa5e40af320 2023-10-09T06:38:48.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.232+0000 7fa5eaccc700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5e40cec20 con 0x7fa5e4069c70 2023-10-09T06:38:48.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.232+0000 7fa5eaccc700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5e40afa90 con 0x7fa5e4159950 2023-10-09T06:38:48.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.232+0000 7fa5e8a68700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa5e4159950 0x7fa5e40cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:48.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.232+0000 7fa5dbfff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fa5e40af320 0x7fa5e4069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:56412/0 (socket says 172.21.15.163:56412) 2023-10-09T06:38:48.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.232+0000 7fa5dbfff700 1 -- 172.21.15.163:0/2073544984 learned_addr learned my addr 172.21.15.163:0/2073544984 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:48.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.232+0000 7fa5e8a68700 1 -- 172.21.15.163:0/2073544984 >> v1:172.21.15.181:6789/0 conn(0x7fa5e4069c70 legacy=0x7fa5e40cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:48.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.232+0000 7fa5db7fe700 1 -- 172.21.15.163:0/2073544984 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 748765355 0 0) 0x7fa5e40ce380 con 0x7fa5e40af320 2023-10-09T06:38:48.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.232+0000 7fa5e8a68700 1 -- 172.21.15.163:0/2073544984 >> v1:172.21.15.163:6789/0 conn(0x7fa5e40af320 legacy=0x7fa5e4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:48.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.232+0000 7fa5e8a68700 1 -- 172.21.15.163:0/2073544984 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e40ce5d0 con 0x7fa5e4159950 2023-10-09T06:38:48.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.232+0000 7fa5e8a68700 1 --2- 172.21.15.163:0/2073544984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5e4159950 0x7fa5e40cbb90 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009e60 tx=0x7fa5e0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0b4c6743602c8f server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:48.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.232+0000 7fa5db7fe700 1 -- 172.21.15.163:0/2073544984 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e000e070 con 0x7fa5e4159950 2023-10-09T06:38:48.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.232+0000 7fa5db7fe700 1 -- 172.21.15.163:0/2073544984 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e000bbf0 con 0x7fa5e4159950 2023-10-09T06:38:48.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.232+0000 7fa5db7fe700 1 -- 172.21.15.163:0/2073544984 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e0012430 con 0x7fa5e4159950 2023-10-09T06:38:48.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.233+0000 7fa5eaccc700 1 -- 172.21.15.163:0/2073544984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5e4159950 msgr2=0x7fa5e40cbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:48.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.233+0000 7fa5eaccc700 1 --2- 172.21.15.163:0/2073544984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5e4159950 0x7fa5e40cbb90 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009e60 tx=0x7fa5e0009780 comp rx=0 tx=0).stop 2023-10-09T06:38:48.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.233+0000 7fa5eaccc700 1 -- 172.21.15.163:0/2073544984 shutdown_connections 2023-10-09T06:38:48.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.233+0000 7fa5eaccc700 1 --2- 172.21.15.163:0/2073544984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5e4159950 0x7fa5e40cbb90 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:48.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.233+0000 7fa5eaccc700 1 -- 172.21.15.163:0/2073544984 >> 172.21.15.163:0/2073544984 conn(0x7fa5e4068e00 msgr2=0x7fa5e40d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:48.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.233+0000 7fa5eaccc700 1 -- 172.21.15.163:0/2073544984 shutdown_connections 2023-10-09T06:38:48.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.233+0000 7fa5eaccc700 1 -- 172.21.15.163:0/2073544984 wait complete. 2023-10-09T06:38:48.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.234+0000 7fa5eaccc700 1 Processor -- start 2023-10-09T06:38:48.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.234+0000 7fa5eaccc700 1 -- start start 2023-10-09T06:38:48.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.235+0000 7fa5eaccc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5e4069c70 0x7fa5e40ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:48.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.235+0000 7fa5eaccc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5e40af320 0x7fa5e40c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:48.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.235+0000 7fa5eaccc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5e4159950 0x7fa5e415ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:48.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.235+0000 7fa5eaccc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e40b4200 con 0x7fa5e40af320 2023-10-09T06:38:48.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.235+0000 7fa5eaccc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e40b35f0 con 0x7fa5e4159950 2023-10-09T06:38:48.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.235+0000 7fa5eaccc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e40b38d0 con 0x7fa5e4069c70 2023-10-09T06:38:48.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.235+0000 7fa5dbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5e40af320 0x7fa5e40c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:48.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.235+0000 7fa5dbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5e40af320 0x7fa5e40c8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:38780/0 (socket says 172.21.15.163:38780) 2023-10-09T06:38:48.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.235+0000 7fa5dbfff700 1 -- 172.21.15.163:0/831366485 learned_addr learned my addr 172.21.15.163:0/831366485 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:48.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.235+0000 7fa5e8a68700 1 --2- 172.21.15.163:0/831366485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5e4069c70 0x7fa5e40ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:48.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.235+0000 7fa5e9269700 1 --2- 172.21.15.163:0/831366485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5e4159950 0x7fa5e415ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:48.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.235+0000 7fa5dbfff700 1 -- 172.21.15.163:0/831366485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5e4069c70 msgr2=0x7fa5e40ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:48.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.235+0000 7fa5dbfff700 1 --2- 172.21.15.163:0/831366485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5e4069c70 0x7fa5e40ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:48.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.235+0000 7fa5dbfff700 1 -- 172.21.15.163:0/831366485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5e4159950 msgr2=0x7fa5e415ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:48.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.235+0000 7fa5dbfff700 1 --2- 172.21.15.163:0/831366485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5e4159950 0x7fa5e415ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:48.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.235+0000 7fa5dbfff700 1 -- 172.21.15.163:0/831366485 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e0009a20 con 0x7fa5e40af320 2023-10-09T06:38:48.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.235+0000 7fa5e8a68700 1 --2- 172.21.15.163:0/831366485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5e4069c70 0x7fa5e40ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:48.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.235+0000 7fa5e9269700 1 --2- 172.21.15.163:0/831366485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5e4159950 0x7fa5e415ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:48.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.236+0000 7fa5dbfff700 1 --2- 172.21.15.163:0/831366485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5e40af320 0x7fa5e40c8520 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fa5d400c7e0 tx=0x7fa5d4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:48.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.236+0000 7fa5d8ff9700 1 -- 172.21.15.163:0/831366485 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d4005b30 con 0x7fa5e40af320 2023-10-09T06:38:48.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.236+0000 7fa5d8ff9700 1 -- 172.21.15.163:0/831366485 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d4005c90 con 0x7fa5e40af320 2023-10-09T06:38:48.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.236+0000 7fa5eaccc700 1 -- 172.21.15.163:0/831366485 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5e40b3bb0 con 0x7fa5e40af320 2023-10-09T06:38:48.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.236+0000 7fa5d8ff9700 1 -- 172.21.15.163:0/831366485 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d4007300 con 0x7fa5e40af320 2023-10-09T06:38:48.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.236+0000 7fa5eaccc700 1 -- 172.21.15.163:0/831366485 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e40b6160 con 0x7fa5e40af320 2023-10-09T06:38:48.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.237+0000 7fa5eaccc700 1 -- 172.21.15.163:0/831366485 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5e40b43b0 con 0x7fa5e40af320 2023-10-09T06:38:48.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.237+0000 7fa5d8ff9700 1 -- 172.21.15.163:0/831366485 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa5d401b400 con 0x7fa5e40af320 2023-10-09T06:38:48.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.238+0000 7fa5d8ff9700 1 --2- 172.21.15.163:0/831366485 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa5d0067320 0x7fa5d00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:48.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.238+0000 7fa5d8ff9700 1 -- 172.21.15.163:0/831366485 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fa5d4084470 con 0x7fa5e40af320 2023-10-09T06:38:48.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.238+0000 7fa5e8a68700 1 --2- 172.21.15.163:0/831366485 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa5d0067320 0x7fa5d00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:48.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.238+0000 7fa5e8a68700 1 --2- 172.21.15.163:0/831366485 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa5d0067320 0x7fa5d00697d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00099f0 tx=0x7fa5e0018000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:48.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.242+0000 7fa5d8ff9700 1 -- 172.21.15.163:0/831366485 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5e40b43b0 con 0x7fa5e40af320 2023-10-09T06:38:48.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:48.427+0000 7fa5eaccc700 1 -- 172.21.15.163:0/831366485 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa5e40718b0 con 0x7fa5e40af320 2023-10-09T06:38:49.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.094+0000 7fa5d8ff9700 1 -- 172.21.15.163:0/831366485 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v162) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa5d4054ef0 con 0x7fa5e40af320 2023-10-09T06:38:49.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.101+0000 7fa5eaccc700 1 -- 172.21.15.163:0/831366485 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa5d0067320 msgr2=0x7fa5d00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:49.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.101+0000 7fa5eaccc700 1 --2- 172.21.15.163:0/831366485 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa5d0067320 0x7fa5d00697d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00099f0 tx=0x7fa5e0018000 comp rx=0 tx=0).stop 2023-10-09T06:38:49.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.101+0000 7fa5eaccc700 1 -- 172.21.15.163:0/831366485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5e40af320 msgr2=0x7fa5e40c8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:49.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.101+0000 7fa5eaccc700 1 --2- 172.21.15.163:0/831366485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5e40af320 0x7fa5e40c8520 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fa5d400c7e0 tx=0x7fa5d4007df0 comp rx=0 tx=0).stop 2023-10-09T06:38:49.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.102+0000 7fa5eaccc700 1 -- 172.21.15.163:0/831366485 shutdown_connections 2023-10-09T06:38:49.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.102+0000 7fa5eaccc700 1 --2- 172.21.15.163:0/831366485 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa5d0067320 0x7fa5d00697d0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:49.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.102+0000 7fa5eaccc700 1 --2- 172.21.15.163:0/831366485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5e4159950 0x7fa5e415ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:49.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.102+0000 7fa5eaccc700 1 --2- 172.21.15.163:0/831366485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5e4069c70 0x7fa5e40ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:49.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.102+0000 7fa5eaccc700 1 --2- 172.21.15.163:0/831366485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5e40af320 0x7fa5e40c8520 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:49.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.102+0000 7fa5eaccc700 1 -- 172.21.15.163:0/831366485 >> 172.21.15.163:0/831366485 conn(0x7fa5e4068e00 msgr2=0x7fa5e40c61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:49.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.102+0000 7fa5eaccc700 1 -- 172.21.15.163:0/831366485 shutdown_connections 2023-10-09T06:38:49.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.102+0000 7fa5eaccc700 1 -- 172.21.15.163:0/831366485 wait complete. 2023-10-09T06:38:49.104 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_data' created 2023-10-09T06:38:49.122 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-09T06:38:49.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.240+0000 7f2554458700 1 Processor -- start 2023-10-09T06:38:49.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.240+0000 7f2554458700 1 -- start start 2023-10-09T06:38:49.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.241+0000 7f2554458700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f254c159c60 0x7f254c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:49.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.241+0000 7f2554458700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f254c15db10 0x7f254c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:49.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.241+0000 7f2554458700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f254c15a160 con 0x7f254c159890 2023-10-09T06:38:49.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.241+0000 7f2554458700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f254c15df50 con 0x7f254c159c60 2023-10-09T06:38:49.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.241+0000 7f2554458700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254c0d13a0 con 0x7f254c15db10 2023-10-09T06:38:49.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.241+0000 7f25519f3700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f254c159c60 0x7f254c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:49.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.241+0000 7f25519f3700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f254c159c60 0x7f254c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44540/0 (socket says 172.21.15.163:44540) 2023-10-09T06:38:49.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.241+0000 7f25519f3700 1 -- 172.21.15.163:0/2576862356 learned_addr learned my addr 172.21.15.163:0/2576862356 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:49.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.242+0000 7f25529f5700 1 --2- 172.21.15.163:0/2576862356 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f254c15db10 0x7f254c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:49.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.242+0000 7f25519f3700 1 -- 172.21.15.163:0/2576862356 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f254c15db10 msgr2=0x7f254c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:49.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.242+0000 7f25519f3700 1 --2- 172.21.15.163:0/2576862356 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f254c15db10 0x7f254c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:49.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.242+0000 7f25519f3700 1 -- 172.21.15.163:0/2576862356 >> v1:172.21.15.181:6789/0 conn(0x7f254c159890 legacy=0x7f254c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:49.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.242+0000 7f25519f3700 1 -- 172.21.15.163:0/2576862356 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254c15e140 con 0x7f254c159c60 2023-10-09T06:38:49.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.242+0000 7f25529f5700 1 --2- 172.21.15.163:0/2576862356 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f254c15db10 0x7f254c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:49.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.242+0000 7f25519f3700 1 --2- 172.21.15.163:0/2576862356 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f254c159c60 0x7f254c15d5d0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f25400099b0 tx=0x7f2540009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4c4560e990b95 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:49.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.242+0000 7f25511f2700 1 -- 172.21.15.163:0/2576862356 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254000b860 con 0x7f254c159c60 2023-10-09T06:38:49.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.242+0000 7f25511f2700 1 -- 172.21.15.163:0/2576862356 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254000b9c0 con 0x7f254c159c60 2023-10-09T06:38:49.256 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.242+0000 7f25511f2700 1 -- 172.21.15.163:0/2576862356 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254000bd60 con 0x7f254c159c60 2023-10-09T06:38:49.256 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.243+0000 7f2554458700 1 -- 172.21.15.163:0/2576862356 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f254c159c60 msgr2=0x7f254c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:49.256 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.243+0000 7f2554458700 1 --2- 172.21.15.163:0/2576862356 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f254c159c60 0x7f254c15d5d0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f25400099b0 tx=0x7f2540009250 comp rx=0 tx=0).stop 2023-10-09T06:38:49.256 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.243+0000 7f2554458700 1 -- 172.21.15.163:0/2576862356 shutdown_connections 2023-10-09T06:38:49.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.243+0000 7f2554458700 1 --2- 172.21.15.163:0/2576862356 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f254c15db10 0x7f254c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:49.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.243+0000 7f2554458700 1 --2- 172.21.15.163:0/2576862356 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f254c159c60 0x7f254c15d5d0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:49.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.243+0000 7f2554458700 1 -- 172.21.15.163:0/2576862356 >> 172.21.15.163:0/2576862356 conn(0x7f254c068e20 msgr2=0x7f254c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:49.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.243+0000 7f2554458700 1 -- 172.21.15.163:0/2576862356 shutdown_connections 2023-10-09T06:38:49.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.243+0000 7f2554458700 1 -- 172.21.15.163:0/2576862356 wait complete. 2023-10-09T06:38:49.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.244+0000 7f2554458700 1 Processor -- start 2023-10-09T06:38:49.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.244+0000 7f2554458700 1 -- start start 2023-10-09T06:38:49.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.245+0000 7f2554458700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f254c159890 0x7f254c15ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:49.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.245+0000 7f2554458700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f254c159c60 0x7f254c15d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:49.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.245+0000 7f2554458700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f254c15db10 0x7f254c0b54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:49.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.245+0000 7f2554458700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c0cd770 con 0x7f254c15db10 2023-10-09T06:38:49.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.245+0000 7f2554458700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c0cd160 con 0x7f254c159c60 2023-10-09T06:38:49.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.245+0000 7f2554458700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254c0b6120 con 0x7f254c159890 2023-10-09T06:38:49.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.245+0000 7f25529f5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f254c15db10 0x7f254c0b54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:49.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.245+0000 7f25529f5700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f254c15db10 0x7f254c0b54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44556/0 (socket says 172.21.15.163:44556) 2023-10-09T06:38:49.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.245+0000 7f25529f5700 1 -- 172.21.15.163:0/3105205253 learned_addr learned my addr 172.21.15.163:0/3105205253 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:49.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.245+0000 7f25521f4700 1 --2- 172.21.15.163:0/3105205253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f254c159890 0x7f254c15ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:49.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.245+0000 7f25519f3700 1 --2- 172.21.15.163:0/3105205253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f254c159c60 0x7f254c15d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:49.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.245+0000 7f25529f5700 1 -- 172.21.15.163:0/3105205253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f254c159890 msgr2=0x7f254c15ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:49.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.245+0000 7f25529f5700 1 --2- 172.21.15.163:0/3105205253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f254c159890 0x7f254c15ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:49.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.245+0000 7f25529f5700 1 -- 172.21.15.163:0/3105205253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f254c159c60 msgr2=0x7f254c15d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:49.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.245+0000 7f25529f5700 1 --2- 172.21.15.163:0/3105205253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f254c159c60 0x7f254c15d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:49.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.245+0000 7f25529f5700 1 -- 172.21.15.163:0/3105205253 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25400094f0 con 0x7f254c15db10 2023-10-09T06:38:49.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.245+0000 7f25529f5700 1 --2- 172.21.15.163:0/3105205253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f254c15db10 0x7f254c0b54d0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f254400d860 tx=0x7f2544008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:49.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.245+0000 7f254a7fc700 1 -- 172.21.15.163:0/3105205253 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25440125f0 con 0x7f254c15db10 2023-10-09T06:38:49.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.245+0000 7f254a7fc700 1 -- 172.21.15.163:0/3105205253 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2544012750 con 0x7f254c15db10 2023-10-09T06:38:49.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.245+0000 7f254a7fc700 1 -- 172.21.15.163:0/3105205253 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2544012ad0 con 0x7f254c15db10 2023-10-09T06:38:49.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.245+0000 7f2554458700 1 -- 172.21.15.163:0/3105205253 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f254c0cd400 con 0x7f254c15db10 2023-10-09T06:38:49.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.245+0000 7f2554458700 1 -- 172.21.15.163:0/3105205253 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f254c0b6400 con 0x7f254c15db10 2023-10-09T06:38:49.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.246+0000 7f2554458700 1 -- 172.21.15.163:0/3105205253 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f254c04c3c0 con 0x7f254c15db10 2023-10-09T06:38:49.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.247+0000 7f254a7fc700 1 -- 172.21.15.163:0/3105205253 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2544013480 con 0x7f254c15db10 2023-10-09T06:38:49.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.247+0000 7f254a7fc700 1 --2- 172.21.15.163:0/3105205253 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2538067320 0x7f25380697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:49.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.247+0000 7f254a7fc700 1 -- 172.21.15.163:0/3105205253 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f2544084d00 con 0x7f254c15db10 2023-10-09T06:38:49.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.247+0000 7f25521f4700 1 --2- 172.21.15.163:0/3105205253 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2538067320 0x7f25380697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:49.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.248+0000 7f25521f4700 1 --2- 172.21.15.163:0/3105205253 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2538067320 0x7f25380697d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f253c007fd0 tx=0x7f253c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:49.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.252+0000 7f254a7fc700 1 -- 172.21.15.163:0/3105205253 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2544055480 con 0x7f254c15db10 2023-10-09T06:38:49.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:49.478+0000 7f2554458700 1 -- 172.21.15.163:0/3105205253 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f254c0cd8b0 con 0x7f254c15db10 2023-10-09T06:38:50.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.128+0000 7f254a7fc700 1 -- 172.21.15.163:0/3105205253 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 28 and data pool 29 v108) v1 ==== 174+0+0 (secure 0 0 0) 0x7f2544055010 con 0x7f254c15db10 2023-10-09T06:38:50.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.133+0000 7f2554458700 1 -- 172.21.15.163:0/3105205253 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2538067320 msgr2=0x7f25380697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:50.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.133+0000 7f2554458700 1 --2- 172.21.15.163:0/3105205253 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2538067320 0x7f25380697d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f253c007fd0 tx=0x7f253c009040 comp rx=0 tx=0).stop 2023-10-09T06:38:50.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.133+0000 7f2554458700 1 -- 172.21.15.163:0/3105205253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f254c15db10 msgr2=0x7f254c0b54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:50.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.133+0000 7f2554458700 1 --2- 172.21.15.163:0/3105205253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f254c15db10 0x7f254c0b54d0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f254400d860 tx=0x7f2544008040 comp rx=0 tx=0).stop 2023-10-09T06:38:50.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.133+0000 7f2554458700 1 -- 172.21.15.163:0/3105205253 shutdown_connections 2023-10-09T06:38:50.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.133+0000 7f2554458700 1 --2- 172.21.15.163:0/3105205253 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2538067320 0x7f25380697d0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:50.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.133+0000 7f2554458700 1 --2- 172.21.15.163:0/3105205253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f254c15db10 0x7f254c0b54d0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:50.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.133+0000 7f2554458700 1 --2- 172.21.15.163:0/3105205253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f254c159890 0x7f254c15ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:50.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.133+0000 7f2554458700 1 --2- 172.21.15.163:0/3105205253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f254c159c60 0x7f254c15d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:50.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.133+0000 7f2554458700 1 -- 172.21.15.163:0/3105205253 >> 172.21.15.163:0/3105205253 conn(0x7f254c068e20 msgr2=0x7f254c153160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:50.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.133+0000 7f2554458700 1 -- 172.21.15.163:0/3105205253 shutdown_connections 2023-10-09T06:38:50.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.133+0000 7f2554458700 1 -- 172.21.15.163:0/3105205253 wait complete. 2023-10-09T06:38:50.136 INFO:teuthology.orchestra.run.smithi163.stderr:new fs with metadata pool 28 and data pool 29 2023-10-09T06:38:50.154 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:38:50.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.269+0000 7fe06c756700 1 Processor -- start 2023-10-09T06:38:50.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.269+0000 7fe06c756700 1 -- start start 2023-10-09T06:38:50.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.270+0000 7fe06c756700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe0640cd1e0 0x7fe0640c9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:50.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.270+0000 7fe06c756700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe06415c2a0 con 0x7fe06406bcd0 2023-10-09T06:38:50.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.270+0000 7fe06c756700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe06415b690 con 0x7fe064141500 2023-10-09T06:38:50.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.270+0000 7fe06c756700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe06415b970 con 0x7fe0640cd1e0 2023-10-09T06:38:50.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.270+0000 7fe06a4f2700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fe06406bcd0 0x7fe064140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:36238/0 (socket says 172.21.15.163:36238) 2023-10-09T06:38:50.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.270+0000 7fe06a4f2700 1 -- 172.21.15.163:0/3747983718 learned_addr learned my addr 172.21.15.163:0/3747983718 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:50.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.270+0000 7fe06acf3700 1 --2- 172.21.15.163:0/3747983718 >> v2:172.21.15.163:3300/0 conn(0x7fe0640cd1e0 0x7fe0640c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:50.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.271+0000 7fe0694f0700 1 -- 172.21.15.163:0/3747983718 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1368168401 0 0) 0x7fe06415c2a0 con 0x7fe06406bcd0 2023-10-09T06:38:50.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.271+0000 7fe0694f0700 1 -- 172.21.15.163:0/3747983718 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe050003570 con 0x7fe06406bcd0 2023-10-09T06:38:50.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.271+0000 7fe06acf3700 1 -- 172.21.15.163:0/3747983718 >> v1:172.21.15.181:6789/0 conn(0x7fe064141500 legacy=0x7fe0640c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:50.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.271+0000 7fe06acf3700 1 -- 172.21.15.163:0/3747983718 >> v1:172.21.15.163:6789/0 conn(0x7fe06406bcd0 legacy=0x7fe064140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:50.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.271+0000 7fe06acf3700 1 -- 172.21.15.163:0/3747983718 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe06415bc50 con 0x7fe0640cd1e0 2023-10-09T06:38:50.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.271+0000 7fe06acf3700 1 --2- 172.21.15.163:0/3747983718 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0640cd1e0 0x7fe0640c9510 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fe05c0098e0 tx=0x7fe05c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9975c4549f84a887 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:50.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.271+0000 7fe0694f0700 1 -- 172.21.15.163:0/3747983718 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c00b640 con 0x7fe0640cd1e0 2023-10-09T06:38:50.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.271+0000 7fe0694f0700 1 -- 172.21.15.163:0/3747983718 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe05c00b7a0 con 0x7fe0640cd1e0 2023-10-09T06:38:50.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.271+0000 7fe0694f0700 1 -- 172.21.15.163:0/3747983718 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c00bb40 con 0x7fe0640cd1e0 2023-10-09T06:38:50.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.271+0000 7fe06c756700 1 -- 172.21.15.163:0/3747983718 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0640cd1e0 msgr2=0x7fe0640c9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:50.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.271+0000 7fe06c756700 1 --2- 172.21.15.163:0/3747983718 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0640cd1e0 0x7fe0640c9510 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fe05c0098e0 tx=0x7fe05c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:50.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.272+0000 7fe06c756700 1 -- 172.21.15.163:0/3747983718 shutdown_connections 2023-10-09T06:38:50.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.272+0000 7fe06c756700 1 --2- 172.21.15.163:0/3747983718 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0640cd1e0 0x7fe0640c9510 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:50.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.272+0000 7fe06c756700 1 -- 172.21.15.163:0/3747983718 >> 172.21.15.163:0/3747983718 conn(0x7fe0640ce600 msgr2=0x7fe0640cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:50.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.272+0000 7fe06c756700 1 -- 172.21.15.163:0/3747983718 shutdown_connections 2023-10-09T06:38:50.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.272+0000 7fe06c756700 1 -- 172.21.15.163:0/3747983718 wait complete. 2023-10-09T06:38:50.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.273+0000 7fe06c756700 1 Processor -- start 2023-10-09T06:38:50.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.273+0000 7fe06c756700 1 -- start start 2023-10-09T06:38:50.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.273+0000 7fe06c756700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe06406bcd0 0x7fe0640c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:50.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.273+0000 7fe06c756700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0640cd1e0 0x7fe06413bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:50.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.273+0000 7fe06c756700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe064141500 0x7fe064156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:50.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.273+0000 7fe06c756700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0640b13a0 con 0x7fe064141500 2023-10-09T06:38:50.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.273+0000 7fe06c756700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0640c7bc0 con 0x7fe06406bcd0 2023-10-09T06:38:50.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.273+0000 7fe06c756700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0640b15f0 con 0x7fe0640cd1e0 2023-10-09T06:38:50.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.274+0000 7fe06acf3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe064141500 0x7fe064156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:50.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.274+0000 7fe06acf3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe064141500 0x7fe064156780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44576/0 (socket says 172.21.15.163:44576) 2023-10-09T06:38:50.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.274+0000 7fe06acf3700 1 -- 172.21.15.163:0/3964060197 learned_addr learned my addr 172.21.15.163:0/3964060197 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:50.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.274+0000 7fe06acf3700 1 -- 172.21.15.163:0/3964060197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0640cd1e0 msgr2=0x7fe06413bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:50.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.274+0000 7fe06acf3700 1 --2- 172.21.15.163:0/3964060197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0640cd1e0 0x7fe06413bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:50.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.274+0000 7fe06acf3700 1 -- 172.21.15.163:0/3964060197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe06406bcd0 msgr2=0x7fe0640c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:50.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.274+0000 7fe06a4f2700 1 --2- 172.21.15.163:0/3964060197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe06406bcd0 0x7fe0640c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:50.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.274+0000 7fe06acf3700 1 --2- 172.21.15.163:0/3964060197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe06406bcd0 0x7fe0640c7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:50.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.274+0000 7fe06acf3700 1 -- 172.21.15.163:0/3964060197 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05c009420 con 0x7fe064141500 2023-10-09T06:38:50.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.274+0000 7fe06acf3700 1 --2- 172.21.15.163:0/3964060197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe064141500 0x7fe064156780 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fe0640cd630 tx=0x7fe05c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:50.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.274+0000 7fe05a7fc700 1 -- 172.21.15.163:0/3964060197 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c00bae0 con 0x7fe064141500 2023-10-09T06:38:50.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.274+0000 7fe05a7fc700 1 -- 172.21.15.163:0/3964060197 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe05c011630 con 0x7fe064141500 2023-10-09T06:38:50.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.274+0000 7fe06c756700 1 -- 172.21.15.163:0/3964060197 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0640b18d0 con 0x7fe064141500 2023-10-09T06:38:50.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.274+0000 7fe05a7fc700 1 -- 172.21.15.163:0/3964060197 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c0119d0 con 0x7fe064141500 2023-10-09T06:38:50.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.274+0000 7fe06c756700 1 -- 172.21.15.163:0/3964060197 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe06415afa0 con 0x7fe064141500 2023-10-09T06:38:50.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.275+0000 7fe06c756700 1 -- 172.21.15.163:0/3964060197 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0640b1df0 con 0x7fe064141500 2023-10-09T06:38:50.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.275+0000 7fe05a7fc700 1 -- 172.21.15.163:0/3964060197 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe05c011d40 con 0x7fe064141500 2023-10-09T06:38:50.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.276+0000 7fe05a7fc700 1 --2- 172.21.15.163:0/3964060197 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe048067320 0x7fe0480697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:50.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.276+0000 7fe05a7fc700 1 -- 172.21.15.163:0/3964060197 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe05c086840 con 0x7fe064141500 2023-10-09T06:38:50.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.276+0000 7fe06a4f2700 1 --2- 172.21.15.163:0/3964060197 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe048067320 0x7fe0480697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:50.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.276+0000 7fe06a4f2700 1 --2- 172.21.15.163:0/3964060197 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe048067320 0x7fe0480697d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe060005c50 tx=0x7fe060009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:50.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.280+0000 7fe05a7fc700 1 -- 172.21.15.163:0/3964060197 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe0640b1df0 con 0x7fe064141500 2023-10-09T06:38:50.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.465+0000 7fe06c756700 1 -- 172.21.15.163:0/3964060197 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0640b1df0 con 0x7fe064141500 2023-10-09T06:38:50.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.466+0000 7fe05a7fc700 1 -- 172.21.15.163:0/3964060197 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) v1 ==== 74+0+15425 (secure 0 0 0) 0x7fe0640b1df0 con 0x7fe064141500 2023-10-09T06:38:50.466 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:50.467 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":163,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:38:49.480394+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:38:47.806008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-10-09T06:38:48.446181+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:38:50.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.468+0000 7fe06c756700 1 -- 172.21.15.163:0/3964060197 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe048067320 msgr2=0x7fe0480697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:50.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.468+0000 7fe06c756700 1 --2- 172.21.15.163:0/3964060197 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe048067320 0x7fe0480697d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe060005c50 tx=0x7fe060009040 comp rx=0 tx=0).stop 2023-10-09T06:38:50.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.468+0000 7fe06c756700 1 -- 172.21.15.163:0/3964060197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe064141500 msgr2=0x7fe064156780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:50.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.468+0000 7fe06c756700 1 --2- 172.21.15.163:0/3964060197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe064141500 0x7fe064156780 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fe0640cd630 tx=0x7fe05c017000 comp rx=0 tx=0).stop 2023-10-09T06:38:50.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.468+0000 7fe06c756700 1 -- 172.21.15.163:0/3964060197 shutdown_connections 2023-10-09T06:38:50.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.468+0000 7fe06c756700 1 --2- 172.21.15.163:0/3964060197 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe048067320 0x7fe0480697d0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:50.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.468+0000 7fe06c756700 1 --2- 172.21.15.163:0/3964060197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe064141500 0x7fe064156780 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:50.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.468+0000 7fe06c756700 1 --2- 172.21.15.163:0/3964060197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe06406bcd0 0x7fe0640c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:50.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.468+0000 7fe06c756700 1 --2- 172.21.15.163:0/3964060197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0640cd1e0 0x7fe06413bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:50.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.468+0000 7fe06c756700 1 -- 172.21.15.163:0/3964060197 >> 172.21.15.163:0/3964060197 conn(0x7fe0640ce600 msgr2=0x7fe0640c9210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:50.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.468+0000 7fe06c756700 1 -- 172.21.15.163:0/3964060197 shutdown_connections 2023-10-09T06:38:50.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.468+0000 7fe06c756700 1 -- 172.21.15.163:0/3964060197 wait complete. 2023-10-09T06:38:50.481 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:38:50.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.595+0000 7f8158932700 1 Processor -- start 2023-10-09T06:38:50.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.595+0000 7f8158932700 1 -- start start 2023-10-09T06:38:50.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.596+0000 7f8158932700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8154069560 0x7f81540699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:50.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.596+0000 7f8158932700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8154069ee0 con 0x7f8154159c60 2023-10-09T06:38:50.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.596+0000 7f8158932700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f815415a160 con 0x7f8154159890 2023-10-09T06:38:50.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.596+0000 7f8158932700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81540d13a0 con 0x7f8154069560 2023-10-09T06:38:50.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.596+0000 7f8151d9b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f8154159c60 0x7f815415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:36250/0 (socket says 172.21.15.163:36250) 2023-10-09T06:38:50.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.596+0000 7f8151d9b700 1 -- 172.21.15.163:0/968758311 learned_addr learned my addr 172.21.15.163:0/968758311 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:50.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.596+0000 7f8152d9d700 1 --2- 172.21.15.163:0/968758311 >> v2:172.21.15.181:3300/0 conn(0x7f8154069560 0x7f81540699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:50.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.597+0000 7f815159a700 1 -- 172.21.15.163:0/968758311 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2788147136 0 0) 0x7f8154069ee0 con 0x7f8154159c60 2023-10-09T06:38:50.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.597+0000 7f815159a700 1 -- 172.21.15.163:0/968758311 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8140003570 con 0x7f8154159c60 2023-10-09T06:38:50.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.597+0000 7f8152d9d700 1 -- 172.21.15.163:0/968758311 >> v1:172.21.15.181:6789/0 conn(0x7f8154159890 legacy=0x7f815415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:50.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.597+0000 7f815159a700 1 -- 172.21.15.163:0/968758311 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 999635517 0 0) 0x7f8140003570 con 0x7f8154159c60 2023-10-09T06:38:50.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.597+0000 7f8152d9d700 1 -- 172.21.15.163:0/968758311 >> v1:172.21.15.163:6789/0 conn(0x7f8154159c60 legacy=0x7f815415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:50.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.597+0000 7f8152d9d700 1 -- 172.21.15.163:0/968758311 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f815415e140 con 0x7f8154069560 2023-10-09T06:38:50.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.597+0000 7f8152d9d700 1 --2- 172.21.15.163:0/968758311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8154069560 0x7f81540699a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f814c0098e0 tx=0x7f814c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee6eded12815221f server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:50.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.597+0000 7f815159a700 1 -- 172.21.15.163:0/968758311 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c001470 con 0x7f8154069560 2023-10-09T06:38:50.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.597+0000 7f815159a700 1 -- 172.21.15.163:0/968758311 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814c011420 con 0x7f8154069560 2023-10-09T06:38:50.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.597+0000 7f815159a700 1 -- 172.21.15.163:0/968758311 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c0117a0 con 0x7f8154069560 2023-10-09T06:38:50.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.598+0000 7f8158932700 1 -- 172.21.15.163:0/968758311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8154069560 msgr2=0x7f81540699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:50.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.598+0000 7f8158932700 1 --2- 172.21.15.163:0/968758311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8154069560 0x7f81540699a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f814c0098e0 tx=0x7f814c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:50.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.598+0000 7f8158932700 1 -- 172.21.15.163:0/968758311 shutdown_connections 2023-10-09T06:38:50.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.598+0000 7f8158932700 1 --2- 172.21.15.163:0/968758311 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8154069560 0x7f81540699a0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:50.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.598+0000 7f8158932700 1 -- 172.21.15.163:0/968758311 >> 172.21.15.163:0/968758311 conn(0x7f8154068e20 msgr2=0x7f81540d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:50.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.598+0000 7f8158932700 1 -- 172.21.15.163:0/968758311 shutdown_connections 2023-10-09T06:38:50.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.598+0000 7f8158932700 1 -- 172.21.15.163:0/968758311 wait complete. 2023-10-09T06:38:50.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.599+0000 7f8158932700 1 Processor -- start 2023-10-09T06:38:50.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.599+0000 7f8158932700 1 -- start start 2023-10-09T06:38:50.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.600+0000 7f8158932700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8154069560 0x7f81540cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:50.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.600+0000 7f8158932700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8154159890 0x7f8154155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:50.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.600+0000 7f8158932700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8154159c60 0x7f81540c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:50.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.600+0000 7f8158932700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81540c8690 con 0x7f8154159c60 2023-10-09T06:38:50.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.600+0000 7f8158932700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81540ce440 con 0x7f8154159890 2023-10-09T06:38:50.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.600+0000 7f8158932700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81540cf2d0 con 0x7f8154069560 2023-10-09T06:38:50.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.600+0000 7f8152d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8154159c60 0x7f81540c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:50.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.600+0000 7f8152d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8154159c60 0x7f81540c9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44590/0 (socket says 172.21.15.163:44590) 2023-10-09T06:38:50.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.600+0000 7f8152d9d700 1 -- 172.21.15.163:0/2006274995 learned_addr learned my addr 172.21.15.163:0/2006274995 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:50.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.600+0000 7f8151d9b700 1 --2- 172.21.15.163:0/2006274995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8154159890 0x7f8154155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:50.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.600+0000 7f815259c700 1 --2- 172.21.15.163:0/2006274995 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8154069560 0x7f81540cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:50.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.600+0000 7f8152d9d700 1 -- 172.21.15.163:0/2006274995 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8154069560 msgr2=0x7f81540cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:50.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.600+0000 7f8152d9d700 1 --2- 172.21.15.163:0/2006274995 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8154069560 0x7f81540cdf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:50.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.600+0000 7f8152d9d700 1 -- 172.21.15.163:0/2006274995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8154159890 msgr2=0x7f8154155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:50.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.600+0000 7f8152d9d700 1 --2- 172.21.15.163:0/2006274995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8154159890 0x7f8154155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:50.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.600+0000 7f8152d9d700 1 -- 172.21.15.163:0/2006274995 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814c009420 con 0x7f8154159c60 2023-10-09T06:38:50.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.600+0000 7f8152d9d700 1 --2- 172.21.15.163:0/2006274995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8154159c60 0x7f81540c9ac0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f815406a070 tx=0x7f814c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:50.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.600+0000 7f814a7fc700 1 -- 172.21.15.163:0/2006274995 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c01ce30 con 0x7f8154159c60 2023-10-09T06:38:50.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.600+0000 7f814a7fc700 1 -- 172.21.15.163:0/2006274995 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814c01a450 con 0x7f8154159c60 2023-10-09T06:38:50.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.600+0000 7f8158932700 1 -- 172.21.15.163:0/2006274995 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81540ce6c0 con 0x7f8154159c60 2023-10-09T06:38:50.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.600+0000 7f814a7fc700 1 -- 172.21.15.163:0/2006274995 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c01a7d0 con 0x7f8154159c60 2023-10-09T06:38:50.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.600+0000 7f8158932700 1 -- 172.21.15.163:0/2006274995 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81540cec80 con 0x7f8154159c60 2023-10-09T06:38:50.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.601+0000 7f8158932700 1 -- 172.21.15.163:0/2006274995 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f815404c3c0 con 0x7f8154159c60 2023-10-09T06:38:50.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.602+0000 7f814a7fc700 1 -- 172.21.15.163:0/2006274995 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f814c01abc0 con 0x7f8154159c60 2023-10-09T06:38:50.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.602+0000 7f814a7fc700 1 --2- 172.21.15.163:0/2006274995 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8134067320 0x7f81340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:50.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.602+0000 7f814a7fc700 1 -- 172.21.15.163:0/2006274995 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f814c086660 con 0x7f8154159c60 2023-10-09T06:38:50.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.602+0000 7f815259c700 1 --2- 172.21.15.163:0/2006274995 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8134067320 0x7f81340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:50.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.603+0000 7f815259c700 1 --2- 172.21.15.163:0/2006274995 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8134067320 0x7f81340697d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f8144007fd0 tx=0x7f81440016f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:50.609 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.606+0000 7f814a7fc700 1 -- 172.21.15.163:0/2006274995 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f814c0532f0 con 0x7f8154159c60 2023-10-09T06:38:50.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.788+0000 7f8158932700 1 -- 172.21.15.163:0/2006274995 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f815406a5b0 con 0x7f8154159c60 2023-10-09T06:38:50.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.789+0000 7f814a7fc700 1 -- 172.21.15.163:0/2006274995 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) v1 ==== 74+0+15425 (secure 0 0 0) 0x7f814c056910 con 0x7f8154159c60 2023-10-09T06:38:50.790 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:50.790 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":163,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:38:49.480394+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:38:47.806008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-10-09T06:38:48.446181+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:38:50.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.791+0000 7f8158932700 1 -- 172.21.15.163:0/2006274995 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8134067320 msgr2=0x7f81340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:50.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.791+0000 7f8158932700 1 --2- 172.21.15.163:0/2006274995 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8134067320 0x7f81340697d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f8144007fd0 tx=0x7f81440016f0 comp rx=0 tx=0).stop 2023-10-09T06:38:50.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.791+0000 7f8158932700 1 -- 172.21.15.163:0/2006274995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8154159c60 msgr2=0x7f81540c9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:50.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.791+0000 7f8158932700 1 --2- 172.21.15.163:0/2006274995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8154159c60 0x7f81540c9ac0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f815406a070 tx=0x7f814c016000 comp rx=0 tx=0).stop 2023-10-09T06:38:50.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.791+0000 7f8158932700 1 -- 172.21.15.163:0/2006274995 shutdown_connections 2023-10-09T06:38:50.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.791+0000 7f8158932700 1 --2- 172.21.15.163:0/2006274995 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8134067320 0x7f81340697d0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:50.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.791+0000 7f8158932700 1 --2- 172.21.15.163:0/2006274995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8154159c60 0x7f81540c9ac0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:50.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.791+0000 7f8158932700 1 --2- 172.21.15.163:0/2006274995 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8154069560 0x7f81540cdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:50.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.791+0000 7f8158932700 1 --2- 172.21.15.163:0/2006274995 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8154159890 0x7f8154155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:50.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.791+0000 7f8158932700 1 -- 172.21.15.163:0/2006274995 >> 172.21.15.163:0/2006274995 conn(0x7f8154068e20 msgr2=0x7f8154157a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:50.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.791+0000 7f8158932700 1 -- 172.21.15.163:0/2006274995 shutdown_connections 2023-10-09T06:38:50.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.791+0000 7f8158932700 1 -- 172.21.15.163:0/2006274995 wait complete. 2023-10-09T06:38:50.804 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-09T06:38:50.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.921+0000 7f0dd3323700 1 Processor -- start 2023-10-09T06:38:50.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.921+0000 7f0dd3323700 1 -- start start 2023-10-09T06:38:50.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.922+0000 7f0dd3323700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0dcc0b4e70 0x7f0dcc0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:50.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.922+0000 7f0dd3323700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dcc0cfb00 con 0x7f0dcc0b16d0 2023-10-09T06:38:50.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.922+0000 7f0dd3323700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dcc15e060 con 0x7f0dcc0b1b40 2023-10-09T06:38:50.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.922+0000 7f0dd3323700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dcc0b5c40 con 0x7f0dcc0b4e70 2023-10-09T06:38:50.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.922+0000 7f0dd1b20700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f0dcc0b16d0 0x7f0dcc15d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:36254/0 (socket says 172.21.15.163:36254) 2023-10-09T06:38:50.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.922+0000 7f0dd1b20700 1 -- 172.21.15.163:0/1786338934 learned_addr learned my addr 172.21.15.163:0/1786338934 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:50.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.922+0000 7f0dd2321700 1 --2- 172.21.15.163:0/1786338934 >> v2:172.21.15.181:3300/0 conn(0x7f0dcc0b4e70 0x7f0dcc0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:50.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.922+0000 7f0dd131f700 1 -- 172.21.15.163:0/1786338934 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1842023047 0 0) 0x7f0dcc0cfb00 con 0x7f0dcc0b16d0 2023-10-09T06:38:50.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.922+0000 7f0dd131f700 1 -- 172.21.15.163:0/1786338934 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0db8003570 con 0x7f0dcc0b16d0 2023-10-09T06:38:50.925 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.923+0000 7f0dd2321700 1 -- 172.21.15.163:0/1786338934 >> v1:172.21.15.181:6789/0 conn(0x7f0dcc0b1b40 legacy=0x7f0dcc15d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:50.925 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.923+0000 7f0dd2321700 1 -- 172.21.15.163:0/1786338934 >> v1:172.21.15.163:6789/0 conn(0x7f0dcc0b16d0 legacy=0x7f0dcc15d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:50.925 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.923+0000 7f0dd131f700 1 -- 172.21.15.163:0/1786338934 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3677330533 0 0) 0x7f0db8003570 con 0x7f0dcc0b16d0 2023-10-09T06:38:50.925 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.923+0000 7f0dd2321700 1 -- 172.21.15.163:0/1786338934 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dcc0b5f20 con 0x7f0dcc0b4e70 2023-10-09T06:38:50.925 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.923+0000 7f0dd2321700 1 --2- 172.21.15.163:0/1786338934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dcc0b4e70 0x7f0dcc0ccff0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009e60 tx=0x7f0dc8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a7f6c947d660f51 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:50.925 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.923+0000 7f0dd131f700 1 -- 172.21.15.163:0/1786338934 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc800e070 con 0x7f0dcc0b4e70 2023-10-09T06:38:50.925 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.923+0000 7f0dd131f700 1 -- 172.21.15.163:0/1786338934 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc800b850 con 0x7f0dcc0b4e70 2023-10-09T06:38:50.925 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.923+0000 7f0dd131f700 1 -- 172.21.15.163:0/1786338934 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc001470 con 0x7f0dcc0b4e70 2023-10-09T06:38:50.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.924+0000 7f0dd3323700 1 -- 172.21.15.163:0/1786338934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dcc0b4e70 msgr2=0x7f0dcc0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:50.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.924+0000 7f0dd3323700 1 --2- 172.21.15.163:0/1786338934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dcc0b4e70 0x7f0dcc0ccff0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009e60 tx=0x7f0dc8009780 comp rx=0 tx=0).stop 2023-10-09T06:38:50.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.924+0000 7f0dd3323700 1 -- 172.21.15.163:0/1786338934 shutdown_connections 2023-10-09T06:38:50.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.924+0000 7f0dd3323700 1 --2- 172.21.15.163:0/1786338934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dcc0b4e70 0x7f0dcc0ccff0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:50.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.924+0000 7f0dd3323700 1 -- 172.21.15.163:0/1786338934 >> 172.21.15.163:0/1786338934 conn(0x7f0dcc0d10c0 msgr2=0x7f0dcc0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:50.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.924+0000 7f0dd3323700 1 -- 172.21.15.163:0/1786338934 shutdown_connections 2023-10-09T06:38:50.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.924+0000 7f0dd3323700 1 -- 172.21.15.163:0/1786338934 wait complete. 2023-10-09T06:38:50.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.925+0000 7f0dd3323700 1 Processor -- start 2023-10-09T06:38:50.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.925+0000 7f0dd3323700 1 -- start start 2023-10-09T06:38:50.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.925+0000 7f0dd3323700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0dcc0b16d0 0x7f0dcc151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:50.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.925+0000 7f0dd3323700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dcc0b1b40 0x7f0dcc15f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:50.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.925+0000 7f0dd3323700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0dcc0b4e70 0x7f0dcc1519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:50.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.926+0000 7f0dd3323700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc160bc0 con 0x7f0dcc0b16d0 2023-10-09T06:38:50.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.926+0000 7f0dd3323700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc160730 con 0x7f0dcc0b1b40 2023-10-09T06:38:50.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.926+0000 7f0dd3323700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc15fb20 con 0x7f0dcc0b4e70 2023-10-09T06:38:50.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.926+0000 7f0dd2321700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0dcc0b16d0 0x7f0dcc151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:50.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.926+0000 7f0dd2321700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0dcc0b16d0 0x7f0dcc151190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44606/0 (socket says 172.21.15.163:44606) 2023-10-09T06:38:50.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.926+0000 7f0dd2321700 1 -- 172.21.15.163:0/1042313273 learned_addr learned my addr 172.21.15.163:0/1042313273 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:50.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.926+0000 7f0dd1b20700 1 --2- 172.21.15.163:0/1042313273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dcc0b1b40 0x7f0dcc15f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:50.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.926+0000 7f0dd2b22700 1 --2- 172.21.15.163:0/1042313273 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0dcc0b4e70 0x7f0dcc1519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:50.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.926+0000 7f0dd2321700 1 -- 172.21.15.163:0/1042313273 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0dcc0b4e70 msgr2=0x7f0dcc1519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:50.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.926+0000 7f0dd2321700 1 --2- 172.21.15.163:0/1042313273 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0dcc0b4e70 0x7f0dcc1519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:50.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.926+0000 7f0dd2321700 1 -- 172.21.15.163:0/1042313273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dcc0b1b40 msgr2=0x7f0dcc15f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:50.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.926+0000 7f0dd2321700 1 --2- 172.21.15.163:0/1042313273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dcc0b1b40 0x7f0dcc15f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:50.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.926+0000 7f0dd2321700 1 -- 172.21.15.163:0/1042313273 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc8009a20 con 0x7f0dcc0b16d0 2023-10-09T06:38:50.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.926+0000 7f0dd2321700 1 --2- 172.21.15.163:0/1042313273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0dcc0b16d0 0x7f0dcc151190 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc070dd0 tx=0x7f0dc8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:50.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.926+0000 7f0dc27fc700 1 -- 172.21.15.163:0/1042313273 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8001090 con 0x7f0dcc0b16d0 2023-10-09T06:38:50.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.926+0000 7f0dc27fc700 1 -- 172.21.15.163:0/1042313273 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc8012b70 con 0x7f0dcc0b16d0 2023-10-09T06:38:50.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.926+0000 7f0dc27fc700 1 -- 172.21.15.163:0/1042313273 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc801e450 con 0x7f0dcc0b16d0 2023-10-09T06:38:50.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.926+0000 7f0dd3323700 1 -- 172.21.15.163:0/1042313273 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dcc15fe00 con 0x7f0dcc0b16d0 2023-10-09T06:38:50.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.926+0000 7f0dd3323700 1 -- 172.21.15.163:0/1042313273 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dcc1603c0 con 0x7f0dcc0b16d0 2023-10-09T06:38:50.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.927+0000 7f0dd3323700 1 -- 172.21.15.163:0/1042313273 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dcc0b20b0 con 0x7f0dcc0b16d0 2023-10-09T06:38:50.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.928+0000 7f0dc27fc700 1 -- 172.21.15.163:0/1042313273 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0dc801e800 con 0x7f0dcc0b16d0 2023-10-09T06:38:50.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.928+0000 7f0dc27fc700 1 --2- 172.21.15.163:0/1042313273 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0db0067320 0x7f0db00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:50.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.928+0000 7f0dc27fc700 1 -- 172.21.15.163:0/1042313273 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0dc8086f70 con 0x7f0dcc0b16d0 2023-10-09T06:38:50.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.928+0000 7f0dd1b20700 1 --2- 172.21.15.163:0/1042313273 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0db0067320 0x7f0db00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:50.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.929+0000 7f0dd1b20700 1 --2- 172.21.15.163:0/1042313273 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0db0067320 0x7f0db00697d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc159290 tx=0x7f0dbc000bc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:50.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:50.932+0000 7f0dc27fc700 1 -- 172.21.15.163:0/1042313273 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0dc8053af0 con 0x7f0dcc0b16d0 2023-10-09T06:38:51.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:51.162+0000 7f0dd3323700 1 -- 172.21.15.163:0/1042313273 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0dcc160a10 con 0x7f0dcc0b16d0 2023-10-09T06:38:52.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.141+0000 7f0dc27fc700 1 -- 172.21.15.163:0/1042313273 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v111) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0dc8057110 con 0x7f0dcc0b16d0 2023-10-09T06:38:52.144 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.143+0000 7f0dd3323700 1 -- 172.21.15.163:0/1042313273 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0db0067320 msgr2=0x7f0db00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:52.144 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.143+0000 7f0dd3323700 1 --2- 172.21.15.163:0/1042313273 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0db0067320 0x7f0db00697d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc159290 tx=0x7f0dbc000bc0 comp rx=0 tx=0).stop 2023-10-09T06:38:52.145 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.143+0000 7f0dd3323700 1 -- 172.21.15.163:0/1042313273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0dcc0b16d0 msgr2=0x7f0dcc151190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:52.145 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.143+0000 7f0dd3323700 1 --2- 172.21.15.163:0/1042313273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0dcc0b16d0 0x7f0dcc151190 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc070dd0 tx=0x7f0dc8017000 comp rx=0 tx=0).stop 2023-10-09T06:38:52.145 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.143+0000 7f0dd3323700 1 -- 172.21.15.163:0/1042313273 shutdown_connections 2023-10-09T06:38:52.145 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.143+0000 7f0dd3323700 1 --2- 172.21.15.163:0/1042313273 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0db0067320 0x7f0db00697d0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:52.145 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.143+0000 7f0dd3323700 1 --2- 172.21.15.163:0/1042313273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0dcc0b16d0 0x7f0dcc151190 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:52.145 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.143+0000 7f0dd3323700 1 --2- 172.21.15.163:0/1042313273 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0dcc0b4e70 0x7f0dcc1519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:52.145 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.143+0000 7f0dd3323700 1 --2- 172.21.15.163:0/1042313273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dcc0b1b40 0x7f0dcc15f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:52.145 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.143+0000 7f0dd3323700 1 -- 172.21.15.163:0/1042313273 >> 172.21.15.163:0/1042313273 conn(0x7f0dcc0d10c0 msgr2=0x7f0dcc068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:52.145 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.143+0000 7f0dd3323700 1 -- 172.21.15.163:0/1042313273 shutdown_connections 2023-10-09T06:38:52.145 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.143+0000 7f0dd3323700 1 -- 172.21.15.163:0/1042313273 wait complete. 2023-10-09T06:38:52.158 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-09T06:38:52.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.276+0000 7f002f94f700 1 Processor -- start 2023-10-09T06:38:52.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.276+0000 7f002f94f700 1 -- start start 2023-10-09T06:38:52.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.277+0000 7f002f94f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f002806c0a0 0x7f00280caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:52.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.277+0000 7f002f94f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f002815d7e0 con 0x7f002806bcd0 2023-10-09T06:38:52.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.277+0000 7f002f94f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f002815dac0 con 0x7f002815df50 2023-10-09T06:38:52.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.277+0000 7f002f94f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f002815aa30 con 0x7f002806c0a0 2023-10-09T06:38:52.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.277+0000 7f002d6eb700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f002806bcd0 0x7f002815a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:36270/0 (socket says 172.21.15.163:36270) 2023-10-09T06:38:52.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.277+0000 7f002d6eb700 1 -- 172.21.15.163:0/333359265 learned_addr learned my addr 172.21.15.163:0/333359265 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:52.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.278+0000 7f002ceea700 1 --2- 172.21.15.163:0/333359265 >> v2:172.21.15.181:3300/0 conn(0x7f002806c0a0 0x7f00280caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:52.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.278+0000 7f0027fff700 1 -- 172.21.15.163:0/333359265 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2415675160 0 0) 0x7f002815d7e0 con 0x7f002806bcd0 2023-10-09T06:38:52.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.278+0000 7f0027fff700 1 -- 172.21.15.163:0/333359265 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0010003570 con 0x7f002806bcd0 2023-10-09T06:38:52.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.278+0000 7f0027fff700 1 -- 172.21.15.163:0/333359265 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1990842395 0 0) 0x7f0010003570 con 0x7f002806bcd0 2023-10-09T06:38:52.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.278+0000 7f002ceea700 1 -- 172.21.15.163:0/333359265 >> v1:172.21.15.181:6789/0 conn(0x7f002815df50 legacy=0x7f00280c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:52.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.278+0000 7f002ceea700 1 -- 172.21.15.163:0/333359265 >> v1:172.21.15.163:6789/0 conn(0x7f002806bcd0 legacy=0x7f002815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:52.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.278+0000 7f002ceea700 1 -- 172.21.15.163:0/333359265 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002815ad10 con 0x7f002806c0a0 2023-10-09T06:38:52.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.278+0000 7f002ceea700 1 --2- 172.21.15.163:0/333359265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f002806c0a0 0x7f00280caa90 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f001c0098e0 tx=0x7f001c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14bba43a8347bf54 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:52.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.278+0000 7f0027fff700 1 -- 172.21.15.163:0/333359265 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00180017f0 con 0x7f002806c0a0 2023-10-09T06:38:52.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.278+0000 7f0027fff700 1 -- 172.21.15.163:0/333359265 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001c011420 con 0x7f002806c0a0 2023-10-09T06:38:52.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.278+0000 7f0027fff700 1 -- 172.21.15.163:0/333359265 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c0117a0 con 0x7f002806c0a0 2023-10-09T06:38:52.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.279+0000 7f002f94f700 1 -- 172.21.15.163:0/333359265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f002806c0a0 msgr2=0x7f00280caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:52.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.279+0000 7f002f94f700 1 --2- 172.21.15.163:0/333359265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f002806c0a0 0x7f00280caa90 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f001c0098e0 tx=0x7f001c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:52.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.279+0000 7f002f94f700 1 -- 172.21.15.163:0/333359265 shutdown_connections 2023-10-09T06:38:52.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.279+0000 7f002f94f700 1 --2- 172.21.15.163:0/333359265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f002806c0a0 0x7f00280caa90 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:52.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.279+0000 7f002f94f700 1 -- 172.21.15.163:0/333359265 >> 172.21.15.163:0/333359265 conn(0x7f00280ce600 msgr2=0x7f00280cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:52.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.279+0000 7f002f94f700 1 -- 172.21.15.163:0/333359265 shutdown_connections 2023-10-09T06:38:52.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.279+0000 7f002f94f700 1 -- 172.21.15.163:0/333359265 wait complete. 2023-10-09T06:38:52.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.280+0000 7f002f94f700 1 Processor -- start 2023-10-09T06:38:52.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.280+0000 7f002f94f700 1 -- start start 2023-10-09T06:38:52.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.281+0000 7f002f94f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f002806bcd0 0x7f00280cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:52.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.281+0000 7f002f94f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f002806c0a0 0x7f002814c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:52.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.281+0000 7f002f94f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f002815df50 0x7f002814ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:52.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.281+0000 7f002f94f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002814cf60 con 0x7f002806c0a0 2023-10-09T06:38:52.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.281+0000 7f002f94f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00281499e0 con 0x7f002806bcd0 2023-10-09T06:38:52.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.281+0000 7f002f94f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0028148960 con 0x7f002815df50 2023-10-09T06:38:52.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.281+0000 7f002ceea700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f002806c0a0 0x7f002814c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:52.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.281+0000 7f002ceea700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f002806c0a0 0x7f002814c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44608/0 (socket says 172.21.15.163:44608) 2023-10-09T06:38:52.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.281+0000 7f002ceea700 1 -- 172.21.15.163:0/603381152 learned_addr learned my addr 172.21.15.163:0/603381152 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:52.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.281+0000 7f002d6eb700 1 --2- 172.21.15.163:0/603381152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f002806bcd0 0x7f00280cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:52.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.281+0000 7f002deec700 1 --2- 172.21.15.163:0/603381152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f002815df50 0x7f002814ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:52.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.281+0000 7f002ceea700 1 -- 172.21.15.163:0/603381152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f002815df50 msgr2=0x7f002814ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:52.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.281+0000 7f002ceea700 1 --2- 172.21.15.163:0/603381152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f002815df50 0x7f002814ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:52.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.281+0000 7f002ceea700 1 -- 172.21.15.163:0/603381152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f002806bcd0 msgr2=0x7f00280cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:52.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.281+0000 7f002ceea700 1 --2- 172.21.15.163:0/603381152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f002806bcd0 0x7f00280cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:52.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.281+0000 7f002ceea700 1 -- 172.21.15.163:0/603381152 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001c009420 con 0x7f002806c0a0 2023-10-09T06:38:52.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.281+0000 7f002d6eb700 1 --2- 172.21.15.163:0/603381152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f002806bcd0 0x7f00280cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:52.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.282+0000 7f002ceea700 1 --2- 172.21.15.163:0/603381152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f002806c0a0 0x7f002814c4e0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f001c009fd0 tx=0x7f001c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:52.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.282+0000 7f00257fa700 1 -- 172.21.15.163:0/603381152 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c01cb60 con 0x7f002806c0a0 2023-10-09T06:38:52.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.282+0000 7f00257fa700 1 -- 172.21.15.163:0/603381152 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001c01cce0 con 0x7f002806c0a0 2023-10-09T06:38:52.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.282+0000 7f002f94f700 1 -- 172.21.15.163:0/603381152 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0028148c40 con 0x7f002806c0a0 2023-10-09T06:38:52.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.282+0000 7f00257fa700 1 -- 172.21.15.163:0/603381152 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c01a6a0 con 0x7f002806c0a0 2023-10-09T06:38:52.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.282+0000 7f002f94f700 1 -- 172.21.15.163:0/603381152 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0028148fd0 con 0x7f002806c0a0 2023-10-09T06:38:52.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.283+0000 7f002f94f700 1 -- 172.21.15.163:0/603381152 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0028148dd0 con 0x7f002806c0a0 2023-10-09T06:38:52.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.283+0000 7f00257fa700 1 -- 172.21.15.163:0/603381152 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f001c01aad0 con 0x7f002806c0a0 2023-10-09T06:38:52.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.283+0000 7f00257fa700 1 --2- 172.21.15.163:0/603381152 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0014067320 0x7f00140697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:52.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.284+0000 7f00257fa700 1 -- 172.21.15.163:0/603381152 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f001c086690 con 0x7f002806c0a0 2023-10-09T06:38:52.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.284+0000 7f002d6eb700 1 --2- 172.21.15.163:0/603381152 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0014067320 0x7f00140697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:52.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.284+0000 7f002d6eb700 1 --2- 172.21.15.163:0/603381152 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0014067320 0x7f00140697d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0018001e20 tx=0x7f0018000f40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:52.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.288+0000 7f00257fa700 1 -- 172.21.15.163:0/603381152 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f001c053320 con 0x7f002806c0a0 2023-10-09T06:38:52.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:52.515+0000 7f002f94f700 1 -- 172.21.15.163:0/603381152 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f002804c3c0 con 0x7f002806c0a0 2023-10-09T06:38:53.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.143+0000 7f00257fa700 1 -- 172.21.15.163:0/603381152 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v112) v1 ==== 122+0+0 (secure 0 0 0) 0x7f001c056940 con 0x7f002806c0a0 2023-10-09T06:38:53.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.145+0000 7f002f94f700 1 -- 172.21.15.163:0/603381152 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0014067320 msgr2=0x7f00140697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:53.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.145+0000 7f002f94f700 1 --2- 172.21.15.163:0/603381152 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0014067320 0x7f00140697d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0018001e20 tx=0x7f0018000f40 comp rx=0 tx=0).stop 2023-10-09T06:38:53.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.145+0000 7f002f94f700 1 -- 172.21.15.163:0/603381152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f002806c0a0 msgr2=0x7f002814c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:53.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.145+0000 7f002f94f700 1 --2- 172.21.15.163:0/603381152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f002806c0a0 0x7f002814c4e0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f001c009fd0 tx=0x7f001c016000 comp rx=0 tx=0).stop 2023-10-09T06:38:53.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.145+0000 7f002f94f700 1 -- 172.21.15.163:0/603381152 shutdown_connections 2023-10-09T06:38:53.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.145+0000 7f002f94f700 1 --2- 172.21.15.163:0/603381152 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0014067320 0x7f00140697d0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.145+0000 7f002f94f700 1 --2- 172.21.15.163:0/603381152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f002815df50 0x7f002814ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.145+0000 7f002f94f700 1 --2- 172.21.15.163:0/603381152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f002806bcd0 0x7f00280cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.145+0000 7f002f94f700 1 --2- 172.21.15.163:0/603381152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f002806c0a0 0x7f002814c4e0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.145+0000 7f002f94f700 1 -- 172.21.15.163:0/603381152 >> 172.21.15.163:0/603381152 conn(0x7f00280ce600 msgr2=0x7f0028069f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:53.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.145+0000 7f002f94f700 1 -- 172.21.15.163:0/603381152 shutdown_connections 2023-10-09T06:38:53.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.145+0000 7f002f94f700 1 -- 172.21.15.163:0/603381152 wait complete. 2023-10-09T06:38:53.159 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:38:53.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.273+0000 7fc37759e700 1 Processor -- start 2023-10-09T06:38:53.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.273+0000 7fc37759e700 1 -- start start 2023-10-09T06:38:53.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.274+0000 7fc37759e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc3781597f0 0x7fc3780b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:53.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.274+0000 7fc37759e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc37815d230 0x7fc37815d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:53.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.274+0000 7fc37759e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc37815db70 con 0x7fc378159420 2023-10-09T06:38:53.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.274+0000 7fc37759e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3780b11d0 con 0x7fc3781597f0 2023-10-09T06:38:53.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.274+0000 7fc37759e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3780b14b0 con 0x7fc37815d230 2023-10-09T06:38:53.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.274+0000 7fc36dd9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc3781597f0 0x7fc3780b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:53.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.274+0000 7fc36dd9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc3781597f0 0x7fc3780b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44614/0 (socket says 172.21.15.163:44614) 2023-10-09T06:38:53.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.274+0000 7fc36dd9b700 1 -- 172.21.15.163:0/222260849 learned_addr learned my addr 172.21.15.163:0/222260849 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:53.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.274+0000 7fc36dd9b700 1 -- 172.21.15.163:0/222260849 >> v2:172.21.15.181:3300/0 conn(0x7fc37815d230 msgr2=0x7fc37815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:53.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.274+0000 7fc36dd9b700 1 --2- 172.21.15.163:0/222260849 >> v2:172.21.15.181:3300/0 conn(0x7fc37815d230 0x7fc37815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.274+0000 7fc36dd9b700 1 -- 172.21.15.163:0/222260849 >> v1:172.21.15.181:6789/0 conn(0x7fc378159420 legacy=0x7fc3780b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:53.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.274+0000 7fc36dd9b700 1 -- 172.21.15.163:0/222260849 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3780b1790 con 0x7fc3781597f0 2023-10-09T06:38:53.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.274+0000 7fc36dd9b700 1 --2- 172.21.15.163:0/222260849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3781597f0 0x7fc3780b2270 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fc37815d170 tx=0x7fc360009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1fafe7ea2cbb3f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:53.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.275+0000 7fc375d9b700 1 -- 172.21.15.163:0/222260849 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36000b820 con 0x7fc3781597f0 2023-10-09T06:38:53.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.275+0000 7fc375d9b700 1 -- 172.21.15.163:0/222260849 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36000b980 con 0x7fc3781597f0 2023-10-09T06:38:53.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.275+0000 7fc375d9b700 1 -- 172.21.15.163:0/222260849 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36000bd20 con 0x7fc3781597f0 2023-10-09T06:38:53.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.275+0000 7fc37759e700 1 -- 172.21.15.163:0/222260849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3781597f0 msgr2=0x7fc3780b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:53.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.275+0000 7fc37759e700 1 --2- 172.21.15.163:0/222260849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3781597f0 0x7fc3780b2270 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fc37815d170 tx=0x7fc360009250 comp rx=0 tx=0).stop 2023-10-09T06:38:53.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.275+0000 7fc37759e700 1 -- 172.21.15.163:0/222260849 shutdown_connections 2023-10-09T06:38:53.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.275+0000 7fc37759e700 1 --2- 172.21.15.163:0/222260849 >> v2:172.21.15.181:3300/0 conn(0x7fc37815d230 0x7fc37815d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.275+0000 7fc37759e700 1 --2- 172.21.15.163:0/222260849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3781597f0 0x7fc3780b2270 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.275+0000 7fc37759e700 1 -- 172.21.15.163:0/222260849 >> 172.21.15.163:0/222260849 conn(0x7fc3780d0660 msgr2=0x7fc3780d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:53.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.275+0000 7fc37759e700 1 -- 172.21.15.163:0/222260849 shutdown_connections 2023-10-09T06:38:53.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.275+0000 7fc37759e700 1 -- 172.21.15.163:0/222260849 wait complete. 2023-10-09T06:38:53.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.276+0000 7fc37759e700 1 Processor -- start 2023-10-09T06:38:53.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.276+0000 7fc37759e700 1 -- start start 2023-10-09T06:38:53.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.277+0000 7fc37759e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc378159420 0x7fc378154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:53.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.277+0000 7fc37759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3781597f0 0x7fc37815c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:53.281 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.277+0000 7fc37759e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc37815d230 0x7fc378155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:53.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.277+0000 7fc37759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37815baf0 con 0x7fc3781597f0 2023-10-09T06:38:53.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.277+0000 7fc37759e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3781548a0 con 0x7fc37815d230 2023-10-09T06:38:53.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.277+0000 7fc37759e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc37815ac00 con 0x7fc378159420 2023-10-09T06:38:53.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.277+0000 7fc36dd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3781597f0 0x7fc37815c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:53.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.277+0000 7fc36dd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3781597f0 0x7fc37815c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44618/0 (socket says 172.21.15.163:44618) 2023-10-09T06:38:53.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.277+0000 7fc36dd9b700 1 -- 172.21.15.163:0/3429783589 learned_addr learned my addr 172.21.15.163:0/3429783589 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:53.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.277+0000 7fc36dd9b700 1 -- 172.21.15.163:0/3429783589 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc378159420 msgr2=0x7fc378154360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:53.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.278+0000 7fc36dd9b700 1 --2- 172.21.15.163:0/3429783589 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc378159420 0x7fc378154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.278+0000 7fc36dd9b700 1 -- 172.21.15.163:0/3429783589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc37815d230 msgr2=0x7fc378155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:53.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.278+0000 7fc376d9d700 1 --2- 172.21.15.163:0/3429783589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc37815d230 0x7fc378155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:53.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.278+0000 7fc36dd9b700 1 --2- 172.21.15.163:0/3429783589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc37815d230 0x7fc378155bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.282 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.278+0000 7fc36dd9b700 1 -- 172.21.15.163:0/3429783589 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3600094f0 con 0x7fc3781597f0 2023-10-09T06:38:53.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.278+0000 7fc36dd9b700 1 --2- 172.21.15.163:0/3429783589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3781597f0 0x7fc37815c930 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fc360009290 tx=0x7fc360016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:53.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.278+0000 7fc36effd700 1 -- 172.21.15.163:0/3429783589 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36001a460 con 0x7fc3781597f0 2023-10-09T06:38:53.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.278+0000 7fc36effd700 1 -- 172.21.15.163:0/3429783589 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36001a5c0 con 0x7fc3781597f0 2023-10-09T06:38:53.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.278+0000 7fc37759e700 1 -- 172.21.15.163:0/3429783589 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc37815aee0 con 0x7fc3781597f0 2023-10-09T06:38:53.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.278+0000 7fc36effd700 1 -- 172.21.15.163:0/3429783589 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36001a980 con 0x7fc3781597f0 2023-10-09T06:38:53.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.278+0000 7fc37759e700 1 -- 172.21.15.163:0/3429783589 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc37815b4a0 con 0x7fc3781597f0 2023-10-09T06:38:53.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.279+0000 7fc37759e700 1 -- 172.21.15.163:0/3429783589 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc37804c3c0 con 0x7fc3781597f0 2023-10-09T06:38:53.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.279+0000 7fc36effd700 1 -- 172.21.15.163:0/3429783589 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc36001b440 con 0x7fc3781597f0 2023-10-09T06:38:53.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.280+0000 7fc36effd700 1 --2- 172.21.15.163:0/3429783589 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc358067320 0x7fc3580697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:53.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.280+0000 7fc36effd700 1 -- 172.21.15.163:0/3429783589 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc360086680 con 0x7fc3781597f0 2023-10-09T06:38:53.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.280+0000 7fc37659c700 1 --2- 172.21.15.163:0/3429783589 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc358067320 0x7fc3580697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:53.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.280+0000 7fc37659c700 1 --2- 172.21.15.163:0/3429783589 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc358067320 0x7fc3580697d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc3680097c0 tx=0x7fc368006ce0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:53.284 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.284+0000 7fc36effd700 1 -- 172.21.15.163:0/3429783589 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc360053310 con 0x7fc3781597f0 2023-10-09T06:38:53.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.506+0000 7fc37759e700 1 -- 172.21.15.163:0/3429783589 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3780b1a70 con 0x7fc3781597f0 2023-10-09T06:38:53.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.507+0000 7fc36effd700 1 -- 172.21.15.163:0/3429783589 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4619 (secure 0 0 0) 0x7fc360056930 con 0x7fc3781597f0 2023-10-09T06:38:53.508 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:53.508 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":112,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8211,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3056573272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3056573272},{"type":"v1","addr":"172.21.15.163:6834","nonce":3056573272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":8214,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3482861499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3482861499},{"type":"v1","addr":"172.21.15.163:6835","nonce":3482861499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":8223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/4239124825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4239124825},{"type":"v1","addr":"172.21.15.181:6836","nonce":4239124825}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:38:49.480715+0000","modified":"2023-10-09T06:38:53.141039+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8226":{"gid":8226,"name":"b","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/3494162891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3494162891},{"type":"v1","addr":"172.21.15.181:6837","nonce":3494162891}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-09T06:38:53.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.509+0000 7fc37759e700 1 -- 172.21.15.163:0/3429783589 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc358067320 msgr2=0x7fc3580697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:53.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.509+0000 7fc37759e700 1 --2- 172.21.15.163:0/3429783589 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc358067320 0x7fc3580697d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc3680097c0 tx=0x7fc368006ce0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.509+0000 7fc37759e700 1 -- 172.21.15.163:0/3429783589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3781597f0 msgr2=0x7fc37815c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:53.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.509+0000 7fc37759e700 1 --2- 172.21.15.163:0/3429783589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3781597f0 0x7fc37815c930 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fc360009290 tx=0x7fc360016000 comp rx=0 tx=0).stop 2023-10-09T06:38:53.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.509+0000 7fc37759e700 1 -- 172.21.15.163:0/3429783589 shutdown_connections 2023-10-09T06:38:53.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.509+0000 7fc37759e700 1 --2- 172.21.15.163:0/3429783589 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc358067320 0x7fc3580697d0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.509+0000 7fc37759e700 1 --2- 172.21.15.163:0/3429783589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc37815d230 0x7fc378155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.509+0000 7fc37759e700 1 --2- 172.21.15.163:0/3429783589 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc378159420 0x7fc378154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.509+0000 7fc37759e700 1 --2- 172.21.15.163:0/3429783589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3781597f0 0x7fc37815c930 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.509+0000 7fc37759e700 1 -- 172.21.15.163:0/3429783589 >> 172.21.15.163:0/3429783589 conn(0x7fc3780d0660 msgr2=0x7fc3780b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:53.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.509+0000 7fc37759e700 1 -- 172.21.15.163:0/3429783589 shutdown_connections 2023-10-09T06:38:53.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.509+0000 7fc37759e700 1 -- 172.21.15.163:0/3429783589 wait complete. 2023-10-09T06:38:53.512 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 112 2023-10-09T06:38:53.523 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:38:53.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.636+0000 7f4c24fa7700 1 Processor -- start 2023-10-09T06:38:53.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.636+0000 7f4c24fa7700 1 -- start start 2023-10-09T06:38:53.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.637+0000 7f4c24fa7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c20159890 0x7f4c2015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:53.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.637+0000 7f4c24fa7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4c20159c60 0x7f4c2015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:53.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.637+0000 7f4c24fa7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c2015a160 con 0x7f4c2015db10 2023-10-09T06:38:53.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.637+0000 7f4c24fa7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c200d13a0 con 0x7f4c20159c60 2023-10-09T06:38:53.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.637+0000 7f4c24fa7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2015e140 con 0x7f4c20159890 2023-10-09T06:38:53.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.637+0000 7f4c15d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4c20159c60 0x7f4c2015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:53.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.637+0000 7f4c1e59c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c20159890 0x7f4c2015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:53.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.637+0000 7f4c15d9b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4c20159c60 0x7f4c2015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:60346/0 (socket says 172.21.15.163:60346) 2023-10-09T06:38:53.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.637+0000 7f4c15d9b700 1 -- 172.21.15.163:0/405622070 learned_addr learned my addr 172.21.15.163:0/405622070 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:53.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.637+0000 7f4c1ed9d700 1 --1- 172.21.15.163:0/405622070 >> v1:172.21.15.181:6789/0 conn(0x7f4c2015db10 0x7f4c20069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:51332/0 (socket says 172.21.15.163:51332) 2023-10-09T06:38:53.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.638+0000 7f4c15d9b700 1 -- 172.21.15.163:0/405622070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c20159890 msgr2=0x7f4c2015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:53.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.638+0000 7f4c15d9b700 1 --2- 172.21.15.163:0/405622070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c20159890 0x7f4c2015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.638+0000 7f4c15d9b700 1 -- 172.21.15.163:0/405622070 >> v1:172.21.15.181:6789/0 conn(0x7f4c2015db10 legacy=0x7f4c20069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:53.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.638+0000 7f4c15d9b700 1 -- 172.21.15.163:0/405622070 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2015a3d0 con 0x7f4c20159c60 2023-10-09T06:38:53.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.638+0000 7f4c1dd9b700 1 -- 172.21.15.163:0/405622070 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926001392 0 0) 0x7f4c2015a160 con 0x7f4c2015db10 2023-10-09T06:38:53.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.638+0000 7f4c1e59c700 1 --2- 172.21.15.163:0/405622070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c20159890 0x7f4c2015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:38:53.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.638+0000 7f4c15d9b700 1 --2- 172.21.15.163:0/405622070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c20159c60 0x7f4c2015d5d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4c080099b0 tx=0x7f4c08009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c75833f7c221fc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:53.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.638+0000 7f4c1dd9b700 1 -- 172.21.15.163:0/405622070 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c08009ea0 con 0x7f4c20159c60 2023-10-09T06:38:53.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.638+0000 7f4c1dd9b700 1 -- 172.21.15.163:0/405622070 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0800b860 con 0x7f4c20159c60 2023-10-09T06:38:53.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.638+0000 7f4c1dd9b700 1 -- 172.21.15.163:0/405622070 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0800bc00 con 0x7f4c20159c60 2023-10-09T06:38:53.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.639+0000 7f4c24fa7700 1 -- 172.21.15.163:0/405622070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c20159c60 msgr2=0x7f4c2015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:53.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.639+0000 7f4c24fa7700 1 --2- 172.21.15.163:0/405622070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c20159c60 0x7f4c2015d5d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4c080099b0 tx=0x7f4c08009250 comp rx=0 tx=0).stop 2023-10-09T06:38:53.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.639+0000 7f4c24fa7700 1 -- 172.21.15.163:0/405622070 shutdown_connections 2023-10-09T06:38:53.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.639+0000 7f4c24fa7700 1 --2- 172.21.15.163:0/405622070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c20159890 0x7f4c2015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.639+0000 7f4c24fa7700 1 --2- 172.21.15.163:0/405622070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c20159c60 0x7f4c2015d5d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.639+0000 7f4c24fa7700 1 -- 172.21.15.163:0/405622070 >> 172.21.15.163:0/405622070 conn(0x7f4c20068e20 msgr2=0x7f4c200d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:53.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.639+0000 7f4c24fa7700 1 -- 172.21.15.163:0/405622070 shutdown_connections 2023-10-09T06:38:53.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.639+0000 7f4c24fa7700 1 -- 172.21.15.163:0/405622070 wait complete. 2023-10-09T06:38:53.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.640+0000 7f4c24fa7700 1 Processor -- start 2023-10-09T06:38:53.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.640+0000 7f4c24fa7700 1 -- start start 2023-10-09T06:38:53.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.641+0000 7f4c24fa7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c20159890 0x7f4c20151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:53.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.641+0000 7f4c24fa7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c20159c60 0x7f4c200cf880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:53.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.641+0000 7f4c24fa7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c2015db10 0x7f4c200cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:53.646 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.641+0000 7f4c24fa7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c20152e30 con 0x7f4c20159c60 2023-10-09T06:38:53.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.641+0000 7f4c24fa7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c201529a0 con 0x7f4c20159890 2023-10-09T06:38:53.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.641+0000 7f4c24fa7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c20151d90 con 0x7f4c2015db10 2023-10-09T06:38:53.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.641+0000 7f4c15d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c20159c60 0x7f4c200cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:53.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.641+0000 7f4c15d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c20159c60 0x7f4c200cf880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44624/0 (socket says 172.21.15.163:44624) 2023-10-09T06:38:53.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.641+0000 7f4c15d9b700 1 -- 172.21.15.163:0/720938393 learned_addr learned my addr 172.21.15.163:0/720938393 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:53.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.641+0000 7f4c1e59c700 1 --2- 172.21.15.163:0/720938393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c20159890 0x7f4c20151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:53.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.641+0000 7f4c1ed9d700 1 --2- 172.21.15.163:0/720938393 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c2015db10 0x7f4c200cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:53.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.641+0000 7f4c15d9b700 1 -- 172.21.15.163:0/720938393 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c2015db10 msgr2=0x7f4c200cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:53.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.641+0000 7f4c15d9b700 1 --2- 172.21.15.163:0/720938393 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c2015db10 0x7f4c200cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.641+0000 7f4c15d9b700 1 -- 172.21.15.163:0/720938393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c20159890 msgr2=0x7f4c20151490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:53.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.641+0000 7f4c15d9b700 1 --2- 172.21.15.163:0/720938393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c20159890 0x7f4c20151490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.647 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.641+0000 7f4c15d9b700 1 -- 172.21.15.163:0/720938393 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c10009420 con 0x7f4c20159c60 2023-10-09T06:38:53.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.641+0000 7f4c1e59c700 1 --2- 172.21.15.163:0/720938393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c20159890 0x7f4c20151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:38:53.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.641+0000 7f4c15d9b700 1 --2- 172.21.15.163:0/720938393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c20159c60 0x7f4c200cf880 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f4c08006fd0 tx=0x7f4c08017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:53.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.641+0000 7f4c16ffd700 1 -- 172.21.15.163:0/720938393 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c08000d10 con 0x7f4c20159c60 2023-10-09T06:38:53.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.641+0000 7f4c16ffd700 1 -- 172.21.15.163:0/720938393 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0800bbe0 con 0x7f4c20159c60 2023-10-09T06:38:53.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.641+0000 7f4c24fa7700 1 -- 172.21.15.163:0/720938393 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c080094f0 con 0x7f4c20159c60 2023-10-09T06:38:53.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.641+0000 7f4c16ffd700 1 -- 172.21.15.163:0/720938393 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c08011670 con 0x7f4c20159c60 2023-10-09T06:38:53.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.641+0000 7f4c24fa7700 1 -- 172.21.15.163:0/720938393 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c20152350 con 0x7f4c20159c60 2023-10-09T06:38:53.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.642+0000 7f4c24fa7700 1 -- 172.21.15.163:0/720938393 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c20152b30 con 0x7f4c20159c60 2023-10-09T06:38:53.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.643+0000 7f4c16ffd700 1 -- 172.21.15.163:0/720938393 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4c08011a60 con 0x7f4c20159c60 2023-10-09T06:38:53.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.643+0000 7f4c16ffd700 1 --2- 172.21.15.163:0/720938393 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4c00067320 0x7f4c000697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:53.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.643+0000 7f4c16ffd700 1 -- 172.21.15.163:0/720938393 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4c08086460 con 0x7f4c20159c60 2023-10-09T06:38:53.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.643+0000 7f4c1e59c700 1 --2- 172.21.15.163:0/720938393 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4c00067320 0x7f4c000697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:53.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.644+0000 7f4c1e59c700 1 --2- 172.21.15.163:0/720938393 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4c00067320 0x7f4c000697d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4c10011600 tx=0x7f4c100091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:53.649 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.647+0000 7f4c16ffd700 1 -- 172.21.15.163:0/720938393 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c080530f0 con 0x7f4c20159c60 2023-10-09T06:38:53.832 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.831+0000 7f4c24fa7700 1 -- 172.21.15.163:0/720938393 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c2004c3c0 con 0x7f4c20159c60 2023-10-09T06:38:53.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.832+0000 7f4c16ffd700 1 -- 172.21.15.163:0/720938393 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v164) v1 ==== 74+0+15412 (secure 0 0 0) 0x7f4c08056710 con 0x7f4c20159c60 2023-10-09T06:38:53.833 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:53.833 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":164,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:38:51.114957+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:38:47.806008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-10-09T06:38:48.446181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:38:53.836 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.835+0000 7f4c24fa7700 1 -- 172.21.15.163:0/720938393 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4c00067320 msgr2=0x7f4c000697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:53.836 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.835+0000 7f4c24fa7700 1 --2- 172.21.15.163:0/720938393 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4c00067320 0x7f4c000697d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4c10011600 tx=0x7f4c100091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.836 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.835+0000 7f4c24fa7700 1 -- 172.21.15.163:0/720938393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c20159c60 msgr2=0x7f4c200cf880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:53.836 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.835+0000 7f4c24fa7700 1 --2- 172.21.15.163:0/720938393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c20159c60 0x7f4c200cf880 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f4c08006fd0 tx=0x7f4c08017000 comp rx=0 tx=0).stop 2023-10-09T06:38:53.836 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.835+0000 7f4c24fa7700 1 -- 172.21.15.163:0/720938393 shutdown_connections 2023-10-09T06:38:53.836 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.835+0000 7f4c24fa7700 1 --2- 172.21.15.163:0/720938393 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4c00067320 0x7f4c000697d0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.836 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.835+0000 7f4c24fa7700 1 --2- 172.21.15.163:0/720938393 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c2015db10 0x7f4c200cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.835+0000 7f4c24fa7700 1 --2- 172.21.15.163:0/720938393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c20159890 0x7f4c20151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.835+0000 7f4c24fa7700 1 --2- 172.21.15.163:0/720938393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c20159c60 0x7f4c200cf880 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.835+0000 7f4c24fa7700 1 -- 172.21.15.163:0/720938393 >> 172.21.15.163:0/720938393 conn(0x7f4c20068e20 msgr2=0x7f4c20154280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:53.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.835+0000 7f4c24fa7700 1 -- 172.21.15.163:0/720938393 shutdown_connections 2023-10-09T06:38:53.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.835+0000 7f4c24fa7700 1 -- 172.21.15.163:0/720938393 wait complete. 2023-10-09T06:38:53.848 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-09T06:38:53.848 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:38:53.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.962+0000 7f382df64700 1 Processor -- start 2023-10-09T06:38:53.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.962+0000 7f382df64700 1 -- start start 2023-10-09T06:38:53.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.963+0000 7f382df64700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f382815d230 0x7f382815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:53.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.963+0000 7f382df64700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38280b2980 con 0x7f3828159420 2023-10-09T06:38:53.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.963+0000 7f382df64700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f382815dbb0 con 0x7f38281597f0 2023-10-09T06:38:53.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.963+0000 7f382df64700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38280b11d0 con 0x7f382815d230 2023-10-09T06:38:53.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.963+0000 7f382cf62700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f3828159420 0x7f38280b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:36286/0 (socket says 172.21.15.163:36286) 2023-10-09T06:38:53.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.963+0000 7f382cf62700 1 -- 172.21.15.163:0/220192384 learned_addr learned my addr 172.21.15.163:0/220192384 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:53.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.963+0000 7f382d763700 1 --2- 172.21.15.163:0/220192384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f382815d230 0x7f382815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:53.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.963+0000 7f38277fe700 1 -- 172.21.15.163:0/220192384 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1618318143 0 0) 0x7f38280b2980 con 0x7f3828159420 2023-10-09T06:38:53.967 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.963+0000 7f38277fe700 1 -- 172.21.15.163:0/220192384 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3814003570 con 0x7f3828159420 2023-10-09T06:38:53.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.963+0000 7f38277fe700 1 -- 172.21.15.163:0/220192384 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3328937574 0 0) 0x7f382815dbb0 con 0x7f38281597f0 2023-10-09T06:38:53.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.963+0000 7f38277fe700 1 -- 172.21.15.163:0/220192384 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38280b2980 con 0x7f38281597f0 2023-10-09T06:38:53.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.963+0000 7f38277fe700 1 -- 172.21.15.163:0/220192384 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1726682454 0 0) 0x7f3814003570 con 0x7f3828159420 2023-10-09T06:38:53.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.963+0000 7f38277fe700 1 -- 172.21.15.163:0/220192384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f382815d230 msgr2=0x7f382815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:53.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.963+0000 7f38277fe700 1 --2- 172.21.15.163:0/220192384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f382815d230 0x7f382815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.963+0000 7f38277fe700 1 -- 172.21.15.163:0/220192384 >> v1:172.21.15.181:6789/0 conn(0x7f38281597f0 legacy=0x7f38280b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:53.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.963+0000 7f38277fe700 1 -- 172.21.15.163:0/220192384 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38280b14b0 con 0x7f3828159420 2023-10-09T06:38:53.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.963+0000 7f38277fe700 1 -- 172.21.15.163:0/220192384 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f38200017f0 con 0x7f3828159420 2023-10-09T06:38:53.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.964+0000 7f38277fe700 1 -- 172.21.15.163:0/220192384 >> v1:172.21.15.163:6789/0 conn(0x7f3828159420 legacy=0x7f38280b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:53.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.964+0000 7f38277fe700 1 --2- 172.21.15.163:0/220192384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3814003c20 0x7f3814004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:53.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.964+0000 7f38277fe700 1 --2- 172.21.15.163:0/220192384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3814004810 0x7f3814004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:53.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.964+0000 7f38277fe700 1 --2- 172.21.15.163:0/220192384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38140051c0 0x7f381400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:53.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.964+0000 7f38277fe700 1 -- 172.21.15.163:0/220192384 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38280b11d0 con 0x7f3814003c20 2023-10-09T06:38:53.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.964+0000 7f38277fe700 1 -- 172.21.15.163:0/220192384 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381400c080 con 0x7f38140051c0 2023-10-09T06:38:53.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.964+0000 7f38277fe700 1 -- 172.21.15.163:0/220192384 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381400c240 con 0x7f3814004810 2023-10-09T06:38:53.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.964+0000 7f382d763700 1 --2- 172.21.15.163:0/220192384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f382815d230 0x7f382815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:38:53.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.964+0000 7f382cf62700 1 --2- 172.21.15.163:0/220192384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3814003c20 0x7f3814004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:53.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.964+0000 7f3827fff700 1 --2- 172.21.15.163:0/220192384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3814004810 0x7f3814004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:53.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.964+0000 7f382d763700 1 --2- 172.21.15.163:0/220192384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38140051c0 0x7f381400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:53.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.964+0000 7f382cf62700 1 -- 172.21.15.163:0/220192384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3814004810 msgr2=0x7f3814004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:53.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.964+0000 7f382cf62700 1 --2- 172.21.15.163:0/220192384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3814004810 0x7f3814004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.964+0000 7f382cf62700 1 -- 172.21.15.163:0/220192384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38140051c0 msgr2=0x7f381400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:53.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.964+0000 7f382cf62700 1 --2- 172.21.15.163:0/220192384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38140051c0 0x7f381400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.964+0000 7f382cf62700 1 -- 172.21.15.163:0/220192384 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3814003570 con 0x7f3814003c20 2023-10-09T06:38:53.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.964+0000 7f382d763700 1 --2- 172.21.15.163:0/220192384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38140051c0 0x7f381400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:53.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.964+0000 7f382cf62700 1 --2- 172.21.15.163:0/220192384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3814003c20 0x7f3814004010 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f382000a820 tx=0x7f38200020c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd0cf6c6880a5c1c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:53.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.964+0000 7f38277fe700 1 -- 172.21.15.163:0/220192384 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38200070f0 con 0x7f3814003c20 2023-10-09T06:38:53.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.964+0000 7f38277fe700 1 -- 172.21.15.163:0/220192384 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3820007250 con 0x7f3814003c20 2023-10-09T06:38:53.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.964+0000 7f38277fe700 1 -- 172.21.15.163:0/220192384 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3820007610 con 0x7f3814003c20 2023-10-09T06:38:53.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.965+0000 7f382df64700 1 -- 172.21.15.163:0/220192384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3814003c20 msgr2=0x7f3814004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:53.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.965+0000 7f382df64700 1 --2- 172.21.15.163:0/220192384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3814003c20 0x7f3814004010 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f382000a820 tx=0x7f38200020c0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.965+0000 7f382d763700 1 -- 172.21.15.163:0/220192384 reap_dead start 2023-10-09T06:38:53.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.965+0000 7f382df64700 1 -- 172.21.15.163:0/220192384 shutdown_connections 2023-10-09T06:38:53.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.965+0000 7f382df64700 1 -- 172.21.15.163:0/220192384 >> 172.21.15.163:0/220192384 conn(0x7f38280d0660 msgr2=0x7f38280d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:53.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.965+0000 7f382df64700 1 -- 172.21.15.163:0/220192384 shutdown_connections 2023-10-09T06:38:53.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.965+0000 7f382df64700 1 -- 172.21.15.163:0/220192384 wait complete. 2023-10-09T06:38:53.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.966+0000 7f382df64700 1 Processor -- start 2023-10-09T06:38:53.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.966+0000 7f382df64700 1 -- start start 2023-10-09T06:38:53.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.967+0000 7f382df64700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3814003c20 0x7f38281552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:53.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.967+0000 7f382df64700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38280c6fe0 0x7f3828068400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:53.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.967+0000 7f382df64700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38280c7450 0x7f3828068940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:53.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.967+0000 7f382df64700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38280c8b30 con 0x7f3814003c20 2023-10-09T06:38:53.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.967+0000 7f382df64700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38280c7f20 con 0x7f38280c7450 2023-10-09T06:38:53.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.967+0000 7f382df64700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38280c8200 con 0x7f38280c6fe0 2023-10-09T06:38:53.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.967+0000 7f382cf62700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3814003c20 0x7f38281552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:53.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.967+0000 7f382cf62700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3814003c20 0x7f38281552d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44636/0 (socket says 172.21.15.163:44636) 2023-10-09T06:38:53.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.967+0000 7f382cf62700 1 -- 172.21.15.163:0/1505728042 learned_addr learned my addr 172.21.15.163:0/1505728042 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:53.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.967+0000 7f3827fff700 1 --2- 172.21.15.163:0/1505728042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38280c6fe0 0x7f3828068400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:53.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.967+0000 7f382cf62700 1 -- 172.21.15.163:0/1505728042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38280c6fe0 msgr2=0x7f3828068400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:53.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.967+0000 7f382cf62700 1 --2- 172.21.15.163:0/1505728042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38280c6fe0 0x7f3828068400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.967+0000 7f382d763700 1 --2- 172.21.15.163:0/1505728042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38280c7450 0x7f3828068940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:53.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.967+0000 7f382cf62700 1 -- 172.21.15.163:0/1505728042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38280c7450 msgr2=0x7f3828068940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:53.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.967+0000 7f382cf62700 1 --2- 172.21.15.163:0/1505728042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38280c7450 0x7f3828068940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:53.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.967+0000 7f382cf62700 1 -- 172.21.15.163:0/1505728042 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38280c84e0 con 0x7f3814003c20 2023-10-09T06:38:53.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.967+0000 7f382cf62700 1 --2- 172.21.15.163:0/1505728042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3814003c20 0x7f38281552d0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f381400c050 tx=0x7f3820016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:53.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.967+0000 7f3824ff9700 1 -- 172.21.15.163:0/1505728042 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3820007070 con 0x7f3814003c20 2023-10-09T06:38:53.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.967+0000 7f3824ff9700 1 -- 172.21.15.163:0/1505728042 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38200016f0 con 0x7f3814003c20 2023-10-09T06:38:53.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.967+0000 7f382df64700 1 -- 172.21.15.163:0/1505728042 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38280c87c0 con 0x7f3814003c20 2023-10-09T06:38:53.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.967+0000 7f3824ff9700 1 -- 172.21.15.163:0/1505728042 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3820007980 con 0x7f3814003c20 2023-10-09T06:38:53.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.967+0000 7f382df64700 1 -- 172.21.15.163:0/1505728042 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38280c7b40 con 0x7f3814003c20 2023-10-09T06:38:53.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.968+0000 7f382df64700 1 -- 172.21.15.163:0/1505728042 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38280c8e10 con 0x7f3814003c20 2023-10-09T06:38:53.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.969+0000 7f3824ff9700 1 -- 172.21.15.163:0/1505728042 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3820004bb0 con 0x7f3814003c20 2023-10-09T06:38:53.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.969+0000 7f3824ff9700 1 --2- 172.21.15.163:0/1505728042 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3808067320 0x7f38080697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:53.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.969+0000 7f3824ff9700 1 -- 172.21.15.163:0/1505728042 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f38200878c0 con 0x7f3814003c20 2023-10-09T06:38:53.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.969+0000 7f3827fff700 1 --2- 172.21.15.163:0/1505728042 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3808067320 0x7f38080697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:53.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.970+0000 7f3827fff700 1 --2- 172.21.15.163:0/1505728042 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3808067320 0x7f38080697d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f3818000c00 tx=0x7f3818009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:53.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:53.973+0000 7f3824ff9700 1 -- 172.21.15.163:0/1505728042 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3820054660 con 0x7f3814003c20 2023-10-09T06:38:54.153 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.152+0000 7f382df64700 1 -- 172.21.15.163:0/1505728042 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f38280c8c70 con 0x7f3808067320 2023-10-09T06:38:54.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.161+0000 7f3824ff9700 1 -- 172.21.15.163:0/1505728042 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290201 (secure 0 0 0) 0x7f38180106c0 con 0x7f3808067320 2023-10-09T06:38:54.165 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:54.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.164+0000 7f382df64700 1 -- 172.21.15.163:0/1505728042 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3808067320 msgr2=0x7f38080697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:54.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.164+0000 7f382df64700 1 --2- 172.21.15.163:0/1505728042 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3808067320 0x7f38080697d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f3818000c00 tx=0x7f3818009040 comp rx=0 tx=0).stop 2023-10-09T06:38:54.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.164+0000 7f382df64700 1 -- 172.21.15.163:0/1505728042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3814003c20 msgr2=0x7f38281552d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:54.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.164+0000 7f382df64700 1 --2- 172.21.15.163:0/1505728042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3814003c20 0x7f38281552d0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f381400c050 tx=0x7f3820016000 comp rx=0 tx=0).stop 2023-10-09T06:38:54.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.164+0000 7f382df64700 1 -- 172.21.15.163:0/1505728042 shutdown_connections 2023-10-09T06:38:54.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.164+0000 7f382df64700 1 --2- 172.21.15.163:0/1505728042 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3808067320 0x7f38080697d0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:54.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.164+0000 7f382df64700 1 --2- 172.21.15.163:0/1505728042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3814003c20 0x7f38281552d0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:54.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.164+0000 7f382df64700 1 --2- 172.21.15.163:0/1505728042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38280c7450 0x7f3828068940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:54.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.164+0000 7f382df64700 1 --2- 172.21.15.163:0/1505728042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38280c6fe0 0x7f3828068400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:54.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.164+0000 7f382df64700 1 -- 172.21.15.163:0/1505728042 >> 172.21.15.163:0/1505728042 conn(0x7f38280d0660 msgr2=0x7f3828156a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:54.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.164+0000 7f382df64700 1 -- 172.21.15.163:0/1505728042 shutdown_connections 2023-10-09T06:38:54.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.164+0000 7f382df64700 1 -- 172.21.15.163:0/1505728042 wait complete. 2023-10-09T06:38:54.168 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:38:54.177 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2023-10-09T06:38:52.417682+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":97,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2428360,"kb_used_data":17288,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752546360,"statfs":{"total":773094113280,"available":770607472640,"internally_reserved":0,"allocated":17702912,"data_stored":8530853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.323997"},"pg_stats":[{"pgid":"29.3f","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.142209+0000","last_change":"2023-10-09T06:38:50.142209+0000","last_active":"2023-10-09T06:38:50.142209+0000","last_peered":"2023-10-09T06:38:50.142209+0000","last_clean":"2023-10-09T06:38:50.142209+0000","last_became_active":"2023-10-09T06:38:50.137271+0000","last_became_peered":"2023-10-09T06:38:50.137271+0000","last_unstale":"2023-10-09T06:38:50.142209+0000","last_undegraded":"2023-10-09T06:38:50.142209+0000","last_fullsized":"2023-10-09T06:38:50.142209+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:53:25.259901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.142647+0000","last_change":"2023-10-09T06:38:50.142647+0000","last_active":"2023-10-09T06:38:50.142647+0000","last_peered":"2023-10-09T06:38:50.142647+0000","last_clean":"2023-10-09T06:38:50.142647+0000","last_became_active":"2023-10-09T06:38:50.142421+0000","last_became_peered":"2023-10-09T06:38:50.142421+0000","last_unstale":"2023-10-09T06:38:50.142647+0000","last_undegraded":"2023-10-09T06:38:50.142647+0000","last_fullsized":"2023-10-09T06:38:50.142647+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:17:07.094300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.128345+0000","last_change":"2023-10-09T06:38:50.128345+0000","last_active":"2023-10-09T06:38:50.128345+0000","last_peered":"2023-10-09T06:38:50.128345+0000","last_clean":"2023-10-09T06:38:50.128345+0000","last_became_active":"2023-10-09T06:38:50.128047+0000","last_became_peered":"2023-10-09T06:38:50.128047+0000","last_unstale":"2023-10-09T06:38:50.128345+0000","last_undegraded":"2023-10-09T06:38:50.128345+0000","last_fullsized":"2023-10-09T06:38:50.128345+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:07:11.948474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.139495+0000","last_change":"2023-10-09T06:38:50.139495+0000","last_active":"2023-10-09T06:38:50.139495+0000","last_peered":"2023-10-09T06:38:50.139495+0000","last_clean":"2023-10-09T06:38:50.139495+0000","last_became_active":"2023-10-09T06:38:50.139119+0000","last_became_peered":"2023-10-09T06:38:50.139119+0000","last_unstale":"2023-10-09T06:38:50.139495+0000","last_undegraded":"2023-10-09T06:38:50.139495+0000","last_fullsized":"2023-10-09T06:38:50.139495+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:23:19.098076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.129120+0000","last_change":"2023-10-09T06:38:50.129120+0000","last_active":"2023-10-09T06:38:50.129120+0000","last_peered":"2023-10-09T06:38:50.129120+0000","last_clean":"2023-10-09T06:38:50.129120+0000","last_became_active":"2023-10-09T06:38:50.128850+0000","last_became_peered":"2023-10-09T06:38:50.128850+0000","last_unstale":"2023-10-09T06:38:50.129120+0000","last_undegraded":"2023-10-09T06:38:50.129120+0000","last_fullsized":"2023-10-09T06:38:50.129120+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:55:40.976759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.136957+0000","last_change":"2023-10-09T06:38:50.136957+0000","last_active":"2023-10-09T06:38:50.136957+0000","last_peered":"2023-10-09T06:38:50.136957+0000","last_clean":"2023-10-09T06:38:50.136957+0000","last_became_active":"2023-10-09T06:38:50.136676+0000","last_became_peered":"2023-10-09T06:38:50.136676+0000","last_unstale":"2023-10-09T06:38:50.136957+0000","last_undegraded":"2023-10-09T06:38:50.136957+0000","last_fullsized":"2023-10-09T06:38:50.136957+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:34:34.991435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.126270+0000","last_change":"2023-10-09T06:38:50.126270+0000","last_active":"2023-10-09T06:38:50.126270+0000","last_peered":"2023-10-09T06:38:50.126270+0000","last_clean":"2023-10-09T06:38:50.126270+0000","last_became_active":"2023-10-09T06:38:50.125929+0000","last_became_peered":"2023-10-09T06:38:50.125929+0000","last_unstale":"2023-10-09T06:38:50.126270+0000","last_undegraded":"2023-10-09T06:38:50.126270+0000","last_fullsized":"2023-10-09T06:38:50.126270+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:26:23.603659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.143893+0000","last_change":"2023-10-09T06:38:50.143893+0000","last_active":"2023-10-09T06:38:50.143893+0000","last_peered":"2023-10-09T06:38:50.143893+0000","last_clean":"2023-10-09T06:38:50.143893+0000","last_became_active":"2023-10-09T06:38:50.143698+0000","last_became_peered":"2023-10-09T06:38:50.143698+0000","last_unstale":"2023-10-09T06:38:50.143893+0000","last_undegraded":"2023-10-09T06:38:50.143893+0000","last_fullsized":"2023-10-09T06:38:50.143893+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:41:59.805054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.136862+0000","last_change":"2023-10-09T06:38:50.136862+0000","last_active":"2023-10-09T06:38:50.136862+0000","last_peered":"2023-10-09T06:38:50.136862+0000","last_clean":"2023-10-09T06:38:50.136862+0000","last_became_active":"2023-10-09T06:38:50.136547+0000","last_became_peered":"2023-10-09T06:38:50.136547+0000","last_unstale":"2023-10-09T06:38:50.136862+0000","last_undegraded":"2023-10-09T06:38:50.136862+0000","last_fullsized":"2023-10-09T06:38:50.136862+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:23:22.572456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.129908+0000","last_change":"2023-10-09T06:38:50.129908+0000","last_active":"2023-10-09T06:38:50.129908+0000","last_peered":"2023-10-09T06:38:50.129908+0000","last_clean":"2023-10-09T06:38:50.129908+0000","last_became_active":"2023-10-09T06:38:50.129558+0000","last_became_peered":"2023-10-09T06:38:50.129558+0000","last_unstale":"2023-10-09T06:38:50.129908+0000","last_undegraded":"2023-10-09T06:38:50.129908+0000","last_fullsized":"2023-10-09T06:38:50.129908+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:17:20.306377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.148995+0000","last_change":"2023-10-09T06:38:50.148995+0000","last_active":"2023-10-09T06:38:50.148995+0000","last_peered":"2023-10-09T06:38:50.148995+0000","last_clean":"2023-10-09T06:38:50.148995+0000","last_became_active":"2023-10-09T06:38:50.148046+0000","last_became_peered":"2023-10-09T06:38:50.148046+0000","last_unstale":"2023-10-09T06:38:50.148995+0000","last_undegraded":"2023-10-09T06:38:50.148995+0000","last_fullsized":"2023-10-09T06:38:50.148995+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:08:51.778097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.139409+0000","last_change":"2023-10-09T06:38:50.139409+0000","last_active":"2023-10-09T06:38:50.139409+0000","last_peered":"2023-10-09T06:38:50.139409+0000","last_clean":"2023-10-09T06:38:50.139409+0000","last_became_active":"2023-10-09T06:38:50.138790+0000","last_became_peered":"2023-10-09T06:38:50.138790+0000","last_unstale":"2023-10-09T06:38:50.139409+0000","last_undegraded":"2023-10-09T06:38:50.139409+0000","last_fullsized":"2023-10-09T06:38:50.139409+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:26:23.603659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.140569+0000","last_change":"2023-10-09T06:38:50.140569+0000","last_active":"2023-10-09T06:38:50.140569+0000","last_peered":"2023-10-09T06:38:50.140569+0000","last_clean":"2023-10-09T06:38:50.140569+0000","last_became_active":"2023-10-09T06:38:50.140323+0000","last_became_peered":"2023-10-09T06:38:50.140323+0000","last_unstale":"2023-10-09T06:38:50.140569+0000","last_undegraded":"2023-10-09T06:38:50.140569+0000","last_fullsized":"2023-10-09T06:38:50.140569+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:49:03.761584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.137128+0000","last_change":"2023-10-09T06:38:50.137128+0000","last_active":"2023-10-09T06:38:50.137128+0000","last_peered":"2023-10-09T06:38:50.137128+0000","last_clean":"2023-10-09T06:38:50.137128+0000","last_became_active":"2023-10-09T06:38:50.136724+0000","last_became_peered":"2023-10-09T06:38:50.136724+0000","last_unstale":"2023-10-09T06:38:50.137128+0000","last_undegraded":"2023-10-09T06:38:50.137128+0000","last_fullsized":"2023-10-09T06:38:50.137128+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:42:10.906439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.125984+0000","last_change":"2023-10-09T06:38:50.125984+0000","last_active":"2023-10-09T06:38:50.125984+0000","last_peered":"2023-10-09T06:38:50.125984+0000","last_clean":"2023-10-09T06:38:50.125984+0000","last_became_active":"2023-10-09T06:38:50.125740+0000","last_became_peered":"2023-10-09T06:38:50.125740+0000","last_unstale":"2023-10-09T06:38:50.125984+0000","last_undegraded":"2023-10-09T06:38:50.125984+0000","last_fullsized":"2023-10-09T06:38:50.125984+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:19:55.720934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.140091+0000","last_change":"2023-10-09T06:38:50.140091+0000","last_active":"2023-10-09T06:38:50.140091+0000","last_peered":"2023-10-09T06:38:50.140091+0000","last_clean":"2023-10-09T06:38:50.140091+0000","last_became_active":"2023-10-09T06:38:50.139725+0000","last_became_peered":"2023-10-09T06:38:50.139725+0000","last_unstale":"2023-10-09T06:38:50.140091+0000","last_undegraded":"2023-10-09T06:38:50.140091+0000","last_fullsized":"2023-10-09T06:38:50.140091+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:01:54.560937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.130575+0000","last_change":"2023-10-09T06:38:50.130575+0000","last_active":"2023-10-09T06:38:50.130575+0000","last_peered":"2023-10-09T06:38:50.130575+0000","last_clean":"2023-10-09T06:38:50.130575+0000","last_became_active":"2023-10-09T06:38:50.130366+0000","last_became_peered":"2023-10-09T06:38:50.130366+0000","last_unstale":"2023-10-09T06:38:50.130575+0000","last_undegraded":"2023-10-09T06:38:50.130575+0000","last_fullsized":"2023-10-09T06:38:50.130575+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:48:16.621531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.128746+0000","last_change":"2023-10-09T06:38:50.128746+0000","last_active":"2023-10-09T06:38:50.128746+0000","last_peered":"2023-10-09T06:38:50.128746+0000","last_clean":"2023-10-09T06:38:50.128746+0000","last_became_active":"2023-10-09T06:38:50.128567+0000","last_became_peered":"2023-10-09T06:38:50.128567+0000","last_unstale":"2023-10-09T06:38:50.128746+0000","last_undegraded":"2023-10-09T06:38:50.128746+0000","last_fullsized":"2023-10-09T06:38:50.128746+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:46:02.257675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.138027+0000","last_change":"2023-10-09T06:38:50.138027+0000","last_active":"2023-10-09T06:38:50.138027+0000","last_peered":"2023-10-09T06:38:50.138027+0000","last_clean":"2023-10-09T06:38:50.138027+0000","last_became_active":"2023-10-09T06:38:50.137781+0000","last_became_peered":"2023-10-09T06:38:50.137781+0000","last_unstale":"2023-10-09T06:38:50.138027+0000","last_undegraded":"2023-10-09T06:38:50.138027+0000","last_fullsized":"2023-10-09T06:38:50.138027+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:34:34.991435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.127547+0000","last_change":"2023-10-09T06:38:50.127547+0000","last_active":"2023-10-09T06:38:50.127547+0000","last_peered":"2023-10-09T06:38:50.127547+0000","last_clean":"2023-10-09T06:38:50.127547+0000","last_became_active":"2023-10-09T06:38:50.127141+0000","last_became_peered":"2023-10-09T06:38:50.127141+0000","last_unstale":"2023-10-09T06:38:50.127547+0000","last_undegraded":"2023-10-09T06:38:50.127547+0000","last_fullsized":"2023-10-09T06:38:50.127547+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:26:57.066092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.136089+0000","last_change":"2023-10-09T06:38:50.136089+0000","last_active":"2023-10-09T06:38:50.136089+0000","last_peered":"2023-10-09T06:38:50.136089+0000","last_clean":"2023-10-09T06:38:50.136089+0000","last_became_active":"2023-10-09T06:38:50.135839+0000","last_became_peered":"2023-10-09T06:38:50.135839+0000","last_unstale":"2023-10-09T06:38:50.136089+0000","last_undegraded":"2023-10-09T06:38:50.136089+0000","last_fullsized":"2023-10-09T06:38:50.136089+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:17:20.306377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.144887+0000","last_change":"2023-10-09T06:38:50.144887+0000","last_active":"2023-10-09T06:38:50.144887+0000","last_peered":"2023-10-09T06:38:50.144887+0000","last_clean":"2023-10-09T06:38:50.144887+0000","last_became_active":"2023-10-09T06:38:50.144587+0000","last_became_peered":"2023-10-09T06:38:50.144587+0000","last_unstale":"2023-10-09T06:38:50.144887+0000","last_undegraded":"2023-10-09T06:38:50.144887+0000","last_fullsized":"2023-10-09T06:38:50.144887+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:37:57.121013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.143123+0000","last_change":"2023-10-09T06:38:50.143123+0000","last_active":"2023-10-09T06:38:50.143123+0000","last_peered":"2023-10-09T06:38:50.143123+0000","last_clean":"2023-10-09T06:38:50.143123+0000","last_became_active":"2023-10-09T06:38:50.142843+0000","last_became_peered":"2023-10-09T06:38:50.142843+0000","last_unstale":"2023-10-09T06:38:50.143123+0000","last_undegraded":"2023-10-09T06:38:50.143123+0000","last_fullsized":"2023-10-09T06:38:50.143123+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:26:57.066092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.129008+0000","last_change":"2023-10-09T06:38:50.129008+0000","last_active":"2023-10-09T06:38:50.129008+0000","last_peered":"2023-10-09T06:38:50.129008+0000","last_clean":"2023-10-09T06:38:50.129008+0000","last_became_active":"2023-10-09T06:38:50.128764+0000","last_became_peered":"2023-10-09T06:38:50.128764+0000","last_unstale":"2023-10-09T06:38:50.129008+0000","last_undegraded":"2023-10-09T06:38:50.129008+0000","last_fullsized":"2023-10-09T06:38:50.129008+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:20:19.103577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.142413+0000","last_change":"2023-10-09T06:38:50.142413+0000","last_active":"2023-10-09T06:38:50.142413+0000","last_peered":"2023-10-09T06:38:50.142413+0000","last_clean":"2023-10-09T06:38:50.142413+0000","last_became_active":"2023-10-09T06:38:50.141917+0000","last_became_peered":"2023-10-09T06:38:50.141917+0000","last_unstale":"2023-10-09T06:38:50.142413+0000","last_undegraded":"2023-10-09T06:38:50.142413+0000","last_fullsized":"2023-10-09T06:38:50.142413+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:52:24.913167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.137057+0000","last_change":"2023-10-09T06:38:50.137057+0000","last_active":"2023-10-09T06:38:50.137057+0000","last_peered":"2023-10-09T06:38:50.137057+0000","last_clean":"2023-10-09T06:38:50.137057+0000","last_became_active":"2023-10-09T06:38:50.136800+0000","last_became_peered":"2023-10-09T06:38:50.136800+0000","last_unstale":"2023-10-09T06:38:50.137057+0000","last_undegraded":"2023-10-09T06:38:50.137057+0000","last_fullsized":"2023-10-09T06:38:50.137057+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:06:52.423409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.149219+0000","last_change":"2023-10-09T06:38:50.149219+0000","last_active":"2023-10-09T06:38:50.149219+0000","last_peered":"2023-10-09T06:38:50.149219+0000","last_clean":"2023-10-09T06:38:50.149219+0000","last_became_active":"2023-10-09T06:38:50.148371+0000","last_became_peered":"2023-10-09T06:38:50.148371+0000","last_unstale":"2023-10-09T06:38:50.149219+0000","last_undegraded":"2023-10-09T06:38:50.149219+0000","last_fullsized":"2023-10-09T06:38:50.149219+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:26:08.728520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.127386+0000","last_change":"2023-10-09T06:38:50.127386+0000","last_active":"2023-10-09T06:38:50.127386+0000","last_peered":"2023-10-09T06:38:50.127386+0000","last_clean":"2023-10-09T06:38:50.127386+0000","last_became_active":"2023-10-09T06:38:50.126871+0000","last_became_peered":"2023-10-09T06:38:50.126871+0000","last_unstale":"2023-10-09T06:38:50.127386+0000","last_undegraded":"2023-10-09T06:38:50.127386+0000","last_fullsized":"2023-10-09T06:38:50.127386+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:55:40.976759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.131108+0000","last_change":"2023-10-09T06:38:50.131108+0000","last_active":"2023-10-09T06:38:50.131108+0000","last_peered":"2023-10-09T06:38:50.131108+0000","last_clean":"2023-10-09T06:38:50.131108+0000","last_became_active":"2023-10-09T06:38:50.130807+0000","last_became_peered":"2023-10-09T06:38:50.130807+0000","last_unstale":"2023-10-09T06:38:50.131108+0000","last_undegraded":"2023-10-09T06:38:50.131108+0000","last_fullsized":"2023-10-09T06:38:50.131108+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:26:23.603659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.143917+0000","last_change":"2023-10-09T06:38:50.143917+0000","last_active":"2023-10-09T06:38:50.143917+0000","last_peered":"2023-10-09T06:38:50.143917+0000","last_clean":"2023-10-09T06:38:50.143917+0000","last_became_active":"2023-10-09T06:38:50.143335+0000","last_became_peered":"2023-10-09T06:38:50.143335+0000","last_unstale":"2023-10-09T06:38:50.143917+0000","last_undegraded":"2023-10-09T06:38:50.143917+0000","last_fullsized":"2023-10-09T06:38:50.143917+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:45:21.421760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.129565+0000","last_change":"2023-10-09T06:38:50.129565+0000","last_active":"2023-10-09T06:38:50.129565+0000","last_peered":"2023-10-09T06:38:50.129565+0000","last_clean":"2023-10-09T06:38:50.129565+0000","last_became_active":"2023-10-09T06:38:50.129264+0000","last_became_peered":"2023-10-09T06:38:50.129264+0000","last_unstale":"2023-10-09T06:38:50.129565+0000","last_undegraded":"2023-10-09T06:38:50.129565+0000","last_fullsized":"2023-10-09T06:38:50.129565+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:45:21.421760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.132697+0000","last_change":"2023-10-09T06:38:50.132697+0000","last_active":"2023-10-09T06:38:50.132697+0000","last_peered":"2023-10-09T06:38:50.132697+0000","last_clean":"2023-10-09T06:38:50.132697+0000","last_became_active":"2023-10-09T06:38:50.132414+0000","last_became_peered":"2023-10-09T06:38:50.132414+0000","last_unstale":"2023-10-09T06:38:50.132697+0000","last_undegraded":"2023-10-09T06:38:50.132697+0000","last_fullsized":"2023-10-09T06:38:50.132697+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:23:19.098076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.143724+0000","last_change":"2023-10-09T06:38:50.143724+0000","last_active":"2023-10-09T06:38:50.143724+0000","last_peered":"2023-10-09T06:38:50.143724+0000","last_clean":"2023-10-09T06:38:50.143724+0000","last_became_active":"2023-10-09T06:38:50.143463+0000","last_became_peered":"2023-10-09T06:38:50.143463+0000","last_unstale":"2023-10-09T06:38:50.143724+0000","last_undegraded":"2023-10-09T06:38:50.143724+0000","last_fullsized":"2023-10-09T06:38:50.143724+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:01:54.560937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1f","version":"163'4","reported_seq":13,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.151562+0000","last_change":"2023-10-09T06:38:49.112506+0000","last_active":"2023-10-09T06:38:50.151562+0000","last_peered":"2023-10-09T06:38:50.151562+0000","last_clean":"2023-10-09T06:38:50.151562+0000","last_became_active":"2023-10-09T06:38:49.112176+0000","last_became_peered":"2023-10-09T06:38:49.112176+0000","last_unstale":"2023-10-09T06:38:50.151562+0000","last_undegraded":"2023-10-09T06:38:50.151562+0000","last_fullsized":"2023-10-09T06:38:50.151562+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:53:24.253235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.136317+0000","last_change":"2023-10-09T06:38:50.136317+0000","last_active":"2023-10-09T06:38:50.136317+0000","last_peered":"2023-10-09T06:38:50.136317+0000","last_clean":"2023-10-09T06:38:50.136317+0000","last_became_active":"2023-10-09T06:38:50.136038+0000","last_became_peered":"2023-10-09T06:38:50.136038+0000","last_unstale":"2023-10-09T06:38:50.136317+0000","last_undegraded":"2023-10-09T06:38:50.136317+0000","last_fullsized":"2023-10-09T06:38:50.136317+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:45:21.235591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1e","version":"163'1","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.130050+0000","last_change":"2023-10-09T06:38:49.116156+0000","last_active":"2023-10-09T06:38:50.130050+0000","last_peered":"2023-10-09T06:38:50.130050+0000","last_clean":"2023-10-09T06:38:50.130050+0000","last_became_active":"2023-10-09T06:38:49.115853+0000","last_became_peered":"2023-10-09T06:38:49.115853+0000","last_unstale":"2023-10-09T06:38:50.130050+0000","last_undegraded":"2023-10-09T06:38:50.130050+0000","last_fullsized":"2023-10-09T06:38:50.130050+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:42:03.814729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.139399+0000","last_change":"2023-10-09T06:38:50.139399+0000","last_active":"2023-10-09T06:38:50.139399+0000","last_peered":"2023-10-09T06:38:50.139399+0000","last_clean":"2023-10-09T06:38:50.139399+0000","last_became_active":"2023-10-09T06:38:50.137147+0000","last_became_peered":"2023-10-09T06:38:50.137147+0000","last_unstale":"2023-10-09T06:38:50.139399+0000","last_undegraded":"2023-10-09T06:38:50.139399+0000","last_fullsized":"2023-10-09T06:38:50.139399+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:25:54.353541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1c","version":"163'2","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.147380+0000","last_change":"2023-10-09T06:38:49.112364+0000","last_active":"2023-10-09T06:38:50.147380+0000","last_peered":"2023-10-09T06:38:50.147380+0000","last_clean":"2023-10-09T06:38:50.147380+0000","last_became_active":"2023-10-09T06:38:49.112087+0000","last_became_peered":"2023-10-09T06:38:49.112087+0000","last_unstale":"2023-10-09T06:38:50.147380+0000","last_undegraded":"2023-10-09T06:38:50.147380+0000","last_fullsized":"2023-10-09T06:38:50.147380+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:09:10.728788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.144124+0000","last_change":"2023-10-09T06:38:50.144124+0000","last_active":"2023-10-09T06:38:50.144124+0000","last_peered":"2023-10-09T06:38:50.144124+0000","last_clean":"2023-10-09T06:38:50.144124+0000","last_became_active":"2023-10-09T06:38:50.143662+0000","last_became_peered":"2023-10-09T06:38:50.143662+0000","last_unstale":"2023-10-09T06:38:50.144124+0000","last_undegraded":"2023-10-09T06:38:50.144124+0000","last_fullsized":"2023-10-09T06:38:50.144124+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:46:02.257675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1b","version":"163'1","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.121667+0000","last_change":"2023-10-09T06:38:49.109096+0000","last_active":"2023-10-09T06:38:50.121667+0000","last_peered":"2023-10-09T06:38:50.121667+0000","last_clean":"2023-10-09T06:38:50.121667+0000","last_became_active":"2023-10-09T06:38:49.108830+0000","last_became_peered":"2023-10-09T06:38:49.108830+0000","last_unstale":"2023-10-09T06:38:50.121667+0000","last_undegraded":"2023-10-09T06:38:50.121667+0000","last_fullsized":"2023-10-09T06:38:50.121667+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:26:49.787987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.127011+0000","last_change":"2023-10-09T06:38:50.127011+0000","last_active":"2023-10-09T06:38:50.127011+0000","last_peered":"2023-10-09T06:38:50.127011+0000","last_clean":"2023-10-09T06:38:50.127011+0000","last_became_active":"2023-10-09T06:38:50.126728+0000","last_became_peered":"2023-10-09T06:38:50.126728+0000","last_unstale":"2023-10-09T06:38:50.127011+0000","last_undegraded":"2023-10-09T06:38:50.127011+0000","last_fullsized":"2023-10-09T06:38:50.127011+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:19:55.720934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1a","version":"163'1","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.123484+0000","last_change":"2023-10-09T06:38:49.117890+0000","last_active":"2023-10-09T06:38:50.123484+0000","last_peered":"2023-10-09T06:38:50.123484+0000","last_clean":"2023-10-09T06:38:50.123484+0000","last_became_active":"2023-10-09T06:38:49.117627+0000","last_became_peered":"2023-10-09T06:38:49.117627+0000","last_unstale":"2023-10-09T06:38:50.123484+0000","last_undegraded":"2023-10-09T06:38:50.123484+0000","last_fullsized":"2023-10-09T06:38:50.123484+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:54:39.171366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.128204+0000","last_change":"2023-10-09T06:38:50.128204+0000","last_active":"2023-10-09T06:38:50.128204+0000","last_peered":"2023-10-09T06:38:50.128204+0000","last_clean":"2023-10-09T06:38:50.128204+0000","last_became_active":"2023-10-09T06:38:50.127968+0000","last_became_peered":"2023-10-09T06:38:50.127968+0000","last_unstale":"2023-10-09T06:38:50.128204+0000","last_undegraded":"2023-10-09T06:38:50.128204+0000","last_fullsized":"2023-10-09T06:38:50.128204+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:53:25.259901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.119760+0000","last_change":"2023-10-09T06:38:49.114469+0000","last_active":"2023-10-09T06:38:49.119760+0000","last_peered":"2023-10-09T06:38:49.119760+0000","last_clean":"2023-10-09T06:38:49.119760+0000","last_became_active":"2023-10-09T06:38:49.114197+0000","last_became_peered":"2023-10-09T06:38:49.114197+0000","last_unstale":"2023-10-09T06:38:49.119760+0000","last_undegraded":"2023-10-09T06:38:49.119760+0000","last_fullsized":"2023-10-09T06:38:49.119760+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:48:44.644939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.139595+0000","last_change":"2023-10-09T06:38:50.139595+0000","last_active":"2023-10-09T06:38:50.139595+0000","last_peered":"2023-10-09T06:38:50.139595+0000","last_clean":"2023-10-09T06:38:50.139595+0000","last_became_active":"2023-10-09T06:38:50.139255+0000","last_became_peered":"2023-10-09T06:38:50.139255+0000","last_unstale":"2023-10-09T06:38:50.139595+0000","last_undegraded":"2023-10-09T06:38:50.139595+0000","last_fullsized":"2023-10-09T06:38:50.139595+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:24:54.452573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.116097+0000","last_change":"2023-10-09T06:38:49.108941+0000","last_active":"2023-10-09T06:38:49.116097+0000","last_peered":"2023-10-09T06:38:49.116097+0000","last_clean":"2023-10-09T06:38:49.116097+0000","last_became_active":"2023-10-09T06:38:49.108640+0000","last_became_peered":"2023-10-09T06:38:49.108640+0000","last_unstale":"2023-10-09T06:38:49.116097+0000","last_undegraded":"2023-10-09T06:38:49.116097+0000","last_fullsized":"2023-10-09T06:38:49.116097+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:10:32.770444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.142718+0000","last_change":"2023-10-09T06:38:50.142718+0000","last_active":"2023-10-09T06:38:50.142718+0000","last_peered":"2023-10-09T06:38:50.142718+0000","last_clean":"2023-10-09T06:38:50.142718+0000","last_became_active":"2023-10-09T06:38:50.142441+0000","last_became_peered":"2023-10-09T06:38:50.142441+0000","last_unstale":"2023-10-09T06:38:50.142718+0000","last_undegraded":"2023-10-09T06:38:50.142718+0000","last_fullsized":"2023-10-09T06:38:50.142718+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:05:05.275319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.123196+0000","last_change":"2023-10-09T06:38:49.117192+0000","last_active":"2023-10-09T06:38:49.123196+0000","last_peered":"2023-10-09T06:38:49.123196+0000","last_clean":"2023-10-09T06:38:49.123196+0000","last_became_active":"2023-10-09T06:38:49.116891+0000","last_became_peered":"2023-10-09T06:38:49.116891+0000","last_unstale":"2023-10-09T06:38:49.123196+0000","last_undegraded":"2023-10-09T06:38:49.123196+0000","last_fullsized":"2023-10-09T06:38:49.123196+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:24:02.414649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.145141+0000","last_change":"2023-10-09T06:38:50.145141+0000","last_active":"2023-10-09T06:38:50.145141+0000","last_peered":"2023-10-09T06:38:50.145141+0000","last_clean":"2023-10-09T06:38:50.145141+0000","last_became_active":"2023-10-09T06:38:50.144885+0000","last_became_peered":"2023-10-09T06:38:50.144885+0000","last_unstale":"2023-10-09T06:38:50.145141+0000","last_undegraded":"2023-10-09T06:38:50.145141+0000","last_fullsized":"2023-10-09T06:38:50.145141+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:55:40.976759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.111103+0000","last_change":"2023-10-09T06:38:49.107265+0000","last_active":"2023-10-09T06:38:49.111103+0000","last_peered":"2023-10-09T06:38:49.111103+0000","last_clean":"2023-10-09T06:38:49.111103+0000","last_became_active":"2023-10-09T06:38:49.106980+0000","last_became_peered":"2023-10-09T06:38:49.106980+0000","last_unstale":"2023-10-09T06:38:49.111103+0000","last_undegraded":"2023-10-09T06:38:49.111103+0000","last_fullsized":"2023-10-09T06:38:49.111103+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:08:50.771431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.137365+0000","last_change":"2023-10-09T06:38:50.137365+0000","last_active":"2023-10-09T06:38:50.137365+0000","last_peered":"2023-10-09T06:38:50.137365+0000","last_clean":"2023-10-09T06:38:50.137365+0000","last_became_active":"2023-10-09T06:38:50.137109+0000","last_became_peered":"2023-10-09T06:38:50.137109+0000","last_unstale":"2023-10-09T06:38:50.137365+0000","last_undegraded":"2023-10-09T06:38:50.137365+0000","last_fullsized":"2023-10-09T06:38:50.137365+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:01:54.560937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.15","version":"163'3","reported_seq":12,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.150752+0000","last_change":"2023-10-09T06:38:49.110046+0000","last_active":"2023-10-09T06:38:50.150752+0000","last_peered":"2023-10-09T06:38:50.150752+0000","last_clean":"2023-10-09T06:38:50.150752+0000","last_became_active":"2023-10-09T06:38:49.109773+0000","last_became_peered":"2023-10-09T06:38:49.109773+0000","last_unstale":"2023-10-09T06:38:50.150752+0000","last_undegraded":"2023-10-09T06:38:50.150752+0000","last_fullsized":"2023-10-09T06:38:50.150752+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:10:49.853949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.149400+0000","last_change":"2023-10-09T06:38:50.149400+0000","last_active":"2023-10-09T06:38:50.149400+0000","last_peered":"2023-10-09T06:38:50.149400+0000","last_clean":"2023-10-09T06:38:50.149400+0000","last_became_active":"2023-10-09T06:38:50.148660+0000","last_became_peered":"2023-10-09T06:38:50.148660+0000","last_unstale":"2023-10-09T06:38:50.149400+0000","last_undegraded":"2023-10-09T06:38:50.149400+0000","last_fullsized":"2023-10-09T06:38:50.149400+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:34:34.991435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"163'6","reported_seq":15,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.145871+0000","last_change":"2023-10-09T06:38:49.106500+0000","last_active":"2023-10-09T06:38:50.145871+0000","last_peered":"2023-10-09T06:38:50.145871+0000","last_clean":"2023-10-09T06:38:50.145871+0000","last_became_active":"2023-10-09T06:38:49.106246+0000","last_became_peered":"2023-10-09T06:38:49.106246+0000","last_unstale":"2023-10-09T06:38:50.145871+0000","last_undegraded":"2023-10-09T06:38:50.145871+0000","last_fullsized":"2023-10-09T06:38:50.145871+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:33:12.834062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.125489+0000","last_change":"2023-10-09T06:38:50.125489+0000","last_active":"2023-10-09T06:38:50.125489+0000","last_peered":"2023-10-09T06:38:50.125489+0000","last_clean":"2023-10-09T06:38:50.125489+0000","last_became_active":"2023-10-09T06:38:50.125221+0000","last_became_peered":"2023-10-09T06:38:50.125221+0000","last_unstale":"2023-10-09T06:38:50.125489+0000","last_undegraded":"2023-10-09T06:38:50.125489+0000","last_fullsized":"2023-10-09T06:38:50.125489+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:40:27.206952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"163'2","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.148014+0000","last_change":"2023-10-09T06:38:49.116838+0000","last_active":"2023-10-09T06:38:50.148014+0000","last_peered":"2023-10-09T06:38:50.148014+0000","last_clean":"2023-10-09T06:38:50.148014+0000","last_became_active":"2023-10-09T06:38:49.116575+0000","last_became_peered":"2023-10-09T06:38:49.116575+0000","last_unstale":"2023-10-09T06:38:50.148014+0000","last_undegraded":"2023-10-09T06:38:50.148014+0000","last_fullsized":"2023-10-09T06:38:50.148014+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:40:51.226559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.131804+0000","last_change":"2023-10-09T06:38:50.131804+0000","last_active":"2023-10-09T06:38:50.131804+0000","last_peered":"2023-10-09T06:38:50.131804+0000","last_clean":"2023-10-09T06:38:50.131804+0000","last_became_active":"2023-10-09T06:38:50.131529+0000","last_became_peered":"2023-10-09T06:38:50.131529+0000","last_unstale":"2023-10-09T06:38:50.131804+0000","last_undegraded":"2023-10-09T06:38:50.131804+0000","last_fullsized":"2023-10-09T06:38:50.131804+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:05:05.275319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.119198+0000","last_change":"2023-10-09T06:38:49.114630+0000","last_active":"2023-10-09T06:38:49.119198+0000","last_peered":"2023-10-09T06:38:49.119198+0000","last_clean":"2023-10-09T06:38:49.119198+0000","last_became_active":"2023-10-09T06:38:49.114381+0000","last_became_peered":"2023-10-09T06:38:49.114381+0000","last_unstale":"2023-10-09T06:38:49.119198+0000","last_undegraded":"2023-10-09T06:38:49.119198+0000","last_fullsized":"2023-10-09T06:38:49.119198+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:26:56.059426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.144250+0000","last_change":"2023-10-09T06:38:50.144250+0000","last_active":"2023-10-09T06:38:50.144250+0000","last_peered":"2023-10-09T06:38:50.144250+0000","last_clean":"2023-10-09T06:38:50.144250+0000","last_became_active":"2023-10-09T06:38:50.143986+0000","last_became_peered":"2023-10-09T06:38:50.143986+0000","last_unstale":"2023-10-09T06:38:50.144250+0000","last_undegraded":"2023-10-09T06:38:50.144250+0000","last_fullsized":"2023-10-09T06:38:50.144250+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:57:38.908632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.111133+0000","last_change":"2023-10-09T06:38:49.107041+0000","last_active":"2023-10-09T06:38:49.111133+0000","last_peered":"2023-10-09T06:38:49.111133+0000","last_clean":"2023-10-09T06:38:49.111133+0000","last_became_active":"2023-10-09T06:38:49.106781+0000","last_became_peered":"2023-10-09T06:38:49.106781+0000","last_unstale":"2023-10-09T06:38:49.111133+0000","last_undegraded":"2023-10-09T06:38:49.111133+0000","last_fullsized":"2023-10-09T06:38:49.111133+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:52:11.478202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.136830+0000","last_change":"2023-10-09T06:38:50.136830+0000","last_active":"2023-10-09T06:38:50.136830+0000","last_peered":"2023-10-09T06:38:50.136830+0000","last_clean":"2023-10-09T06:38:50.136830+0000","last_became_active":"2023-10-09T06:38:50.136592+0000","last_became_peered":"2023-10-09T06:38:50.136592+0000","last_unstale":"2023-10-09T06:38:50.136830+0000","last_undegraded":"2023-10-09T06:38:50.136830+0000","last_fullsized":"2023-10-09T06:38:50.136830+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:55:47.819136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.10","version":"163'8","reported_seq":17,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.151452+0000","last_change":"2023-10-09T06:38:49.108752+0000","last_active":"2023-10-09T06:38:50.151452+0000","last_peered":"2023-10-09T06:38:50.151452+0000","last_clean":"2023-10-09T06:38:50.151452+0000","last_became_active":"2023-10-09T06:38:49.108496+0000","last_became_peered":"2023-10-09T06:38:49.108496+0000","last_unstale":"2023-10-09T06:38:50.151452+0000","last_undegraded":"2023-10-09T06:38:50.151452+0000","last_fullsized":"2023-10-09T06:38:50.151452+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:23:03.653758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.131441+0000","last_change":"2023-10-09T06:38:50.131441+0000","last_active":"2023-10-09T06:38:50.131441+0000","last_peered":"2023-10-09T06:38:50.131441+0000","last_clean":"2023-10-09T06:38:50.131441+0000","last_became_active":"2023-10-09T06:38:50.131185+0000","last_became_peered":"2023-10-09T06:38:50.131185+0000","last_unstale":"2023-10-09T06:38:50.131441+0000","last_undegraded":"2023-10-09T06:38:50.131441+0000","last_fullsized":"2023-10-09T06:38:50.131441+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:37:19.939607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.f","version":"163'1","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.126131+0000","last_change":"2023-10-09T06:38:49.118475+0000","last_active":"2023-10-09T06:38:50.126131+0000","last_peered":"2023-10-09T06:38:50.126131+0000","last_clean":"2023-10-09T06:38:50.126131+0000","last_became_active":"2023-10-09T06:38:49.118204+0000","last_became_peered":"2023-10-09T06:38:49.118204+0000","last_unstale":"2023-10-09T06:38:50.126131+0000","last_undegraded":"2023-10-09T06:38:50.126131+0000","last_fullsized":"2023-10-09T06:38:50.126131+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:05:00.395166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.129036+0000","last_change":"2023-10-09T06:38:50.129036+0000","last_active":"2023-10-09T06:38:50.129036+0000","last_peered":"2023-10-09T06:38:50.129036+0000","last_clean":"2023-10-09T06:38:50.129036+0000","last_became_active":"2023-10-09T06:38:50.128847+0000","last_became_peered":"2023-10-09T06:38:50.128847+0000","last_unstale":"2023-10-09T06:38:50.129036+0000","last_undegraded":"2023-10-09T06:38:50.129036+0000","last_fullsized":"2023-10-09T06:38:50.129036+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:45:21.421760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"163'2","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.148853+0000","last_change":"2023-10-09T06:38:49.113368+0000","last_active":"2023-10-09T06:38:50.148853+0000","last_peered":"2023-10-09T06:38:50.148853+0000","last_clean":"2023-10-09T06:38:50.148853+0000","last_became_active":"2023-10-09T06:38:49.113114+0000","last_became_peered":"2023-10-09T06:38:49.113114+0000","last_unstale":"2023-10-09T06:38:50.148853+0000","last_undegraded":"2023-10-09T06:38:50.148853+0000","last_fullsized":"2023-10-09T06:38:50.148853+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:36:50.152112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.129641+0000","last_change":"2023-10-09T06:38:50.129641+0000","last_active":"2023-10-09T06:38:50.129641+0000","last_peered":"2023-10-09T06:38:50.129641+0000","last_clean":"2023-10-09T06:38:50.129641+0000","last_became_active":"2023-10-09T06:38:50.129011+0000","last_became_peered":"2023-10-09T06:38:50.129011+0000","last_unstale":"2023-10-09T06:38:50.129641+0000","last_undegraded":"2023-10-09T06:38:50.129641+0000","last_fullsized":"2023-10-09T06:38:50.129641+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:24:54.452573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.d","version":"163'1","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.135881+0000","last_change":"2023-10-09T06:38:49.113614+0000","last_active":"2023-10-09T06:38:50.135881+0000","last_peered":"2023-10-09T06:38:50.135881+0000","last_clean":"2023-10-09T06:38:50.135881+0000","last_became_active":"2023-10-09T06:38:49.113308+0000","last_became_peered":"2023-10-09T06:38:49.113308+0000","last_unstale":"2023-10-09T06:38:50.135881+0000","last_undegraded":"2023-10-09T06:38:50.135881+0000","last_fullsized":"2023-10-09T06:38:50.135881+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:16:45.760322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.128577+0000","last_change":"2023-10-09T06:38:50.128577+0000","last_active":"2023-10-09T06:38:50.128577+0000","last_peered":"2023-10-09T06:38:50.128577+0000","last_clean":"2023-10-09T06:38:50.128577+0000","last_became_active":"2023-10-09T06:38:50.128280+0000","last_became_peered":"2023-10-09T06:38:50.128280+0000","last_unstale":"2023-10-09T06:38:50.128577+0000","last_undegraded":"2023-10-09T06:38:50.128577+0000","last_fullsized":"2023-10-09T06:38:50.128577+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:59:41.148074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.116078+0000","last_change":"2023-10-09T06:38:49.108672+0000","last_active":"2023-10-09T06:38:49.116078+0000","last_peered":"2023-10-09T06:38:49.116078+0000","last_clean":"2023-10-09T06:38:49.116078+0000","last_became_active":"2023-10-09T06:38:49.108244+0000","last_became_peered":"2023-10-09T06:38:49.108244+0000","last_unstale":"2023-10-09T06:38:49.116078+0000","last_undegraded":"2023-10-09T06:38:49.116078+0000","last_fullsized":"2023-10-09T06:38:49.116078+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:03:02.251589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.130147+0000","last_change":"2023-10-09T06:38:50.130147+0000","last_active":"2023-10-09T06:38:50.130147+0000","last_peered":"2023-10-09T06:38:50.130147+0000","last_clean":"2023-10-09T06:38:50.130147+0000","last_became_active":"2023-10-09T06:38:50.129885+0000","last_became_peered":"2023-10-09T06:38:50.129885+0000","last_unstale":"2023-10-09T06:38:50.130147+0000","last_undegraded":"2023-10-09T06:38:50.130147+0000","last_fullsized":"2023-10-09T06:38:50.130147+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:37:57.121013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.b","version":"163'2","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.153319+0000","last_change":"2023-10-09T06:38:49.117518+0000","last_active":"2023-10-09T06:38:50.153319+0000","last_peered":"2023-10-09T06:38:50.153319+0000","last_clean":"2023-10-09T06:38:50.153319+0000","last_became_active":"2023-10-09T06:38:49.117227+0000","last_became_peered":"2023-10-09T06:38:49.117227+0000","last_unstale":"2023-10-09T06:38:50.153319+0000","last_undegraded":"2023-10-09T06:38:50.153319+0000","last_fullsized":"2023-10-09T06:38:50.153319+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:10:19.546962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.136974+0000","last_change":"2023-10-09T06:38:50.136974+0000","last_active":"2023-10-09T06:38:50.136974+0000","last_peered":"2023-10-09T06:38:50.136974+0000","last_clean":"2023-10-09T06:38:50.136974+0000","last_became_active":"2023-10-09T06:38:50.136647+0000","last_became_peered":"2023-10-09T06:38:50.136647+0000","last_unstale":"2023-10-09T06:38:50.136974+0000","last_undegraded":"2023-10-09T06:38:50.136974+0000","last_fullsized":"2023-10-09T06:38:50.136974+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:49:03.761584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.119795+0000","last_change":"2023-10-09T06:38:49.114686+0000","last_active":"2023-10-09T06:38:49.119795+0000","last_peered":"2023-10-09T06:38:49.119795+0000","last_clean":"2023-10-09T06:38:49.119795+0000","last_became_active":"2023-10-09T06:38:49.114457+0000","last_became_peered":"2023-10-09T06:38:49.114457+0000","last_unstale":"2023-10-09T06:38:49.119795+0000","last_undegraded":"2023-10-09T06:38:49.119795+0000","last_fullsized":"2023-10-09T06:38:49.119795+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:09:10.728788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.130937+0000","last_change":"2023-10-09T06:38:50.130937+0000","last_active":"2023-10-09T06:38:50.130937+0000","last_peered":"2023-10-09T06:38:50.130937+0000","last_clean":"2023-10-09T06:38:50.130937+0000","last_became_active":"2023-10-09T06:38:50.130663+0000","last_became_peered":"2023-10-09T06:38:50.130663+0000","last_unstale":"2023-10-09T06:38:50.130937+0000","last_undegraded":"2023-10-09T06:38:50.130937+0000","last_fullsized":"2023-10-09T06:38:50.130937+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:58:32.745191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.123565+0000","last_change":"2023-10-09T06:38:49.117011+0000","last_active":"2023-10-09T06:38:49.123565+0000","last_peered":"2023-10-09T06:38:49.123565+0000","last_clean":"2023-10-09T06:38:49.123565+0000","last_became_active":"2023-10-09T06:38:49.116664+0000","last_became_peered":"2023-10-09T06:38:49.116664+0000","last_unstale":"2023-10-09T06:38:49.123565+0000","last_undegraded":"2023-10-09T06:38:49.123565+0000","last_fullsized":"2023-10-09T06:38:49.123565+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:42:51.752367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.139241+0000","last_change":"2023-10-09T06:38:50.139241+0000","last_active":"2023-10-09T06:38:50.139241+0000","last_peered":"2023-10-09T06:38:50.139241+0000","last_clean":"2023-10-09T06:38:50.139241+0000","last_became_active":"2023-10-09T06:38:50.138974+0000","last_became_peered":"2023-10-09T06:38:50.138974+0000","last_unstale":"2023-10-09T06:38:50.139241+0000","last_undegraded":"2023-10-09T06:38:50.139241+0000","last_fullsized":"2023-10-09T06:38:50.139241+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:40:27.206952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.127085+0000","last_change":"2023-10-09T06:38:49.121745+0000","last_active":"2023-10-09T06:38:49.127085+0000","last_peered":"2023-10-09T06:38:49.127085+0000","last_clean":"2023-10-09T06:38:49.127085+0000","last_became_active":"2023-10-09T06:38:49.121512+0000","last_became_peered":"2023-10-09T06:38:49.121512+0000","last_unstale":"2023-10-09T06:38:49.127085+0000","last_undegraded":"2023-10-09T06:38:49.127085+0000","last_fullsized":"2023-10-09T06:38:49.127085+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:35:24.333777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.144785+0000","last_change":"2023-10-09T06:38:50.144785+0000","last_active":"2023-10-09T06:38:50.144785+0000","last_peered":"2023-10-09T06:38:50.144785+0000","last_clean":"2023-10-09T06:38:50.144785+0000","last_became_active":"2023-10-09T06:38:50.144524+0000","last_became_peered":"2023-10-09T06:38:50.144524+0000","last_unstale":"2023-10-09T06:38:50.144785+0000","last_undegraded":"2023-10-09T06:38:50.144785+0000","last_fullsized":"2023-10-09T06:38:50.144785+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:46:02.257675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.7","version":"163'2","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.148968+0000","last_change":"2023-10-09T06:38:49.120567+0000","last_active":"2023-10-09T06:38:50.148968+0000","last_peered":"2023-10-09T06:38:50.148968+0000","last_clean":"2023-10-09T06:38:50.148968+0000","last_became_active":"2023-10-09T06:38:49.120358+0000","last_became_peered":"2023-10-09T06:38:49.120358+0000","last_unstale":"2023-10-09T06:38:50.148968+0000","last_undegraded":"2023-10-09T06:38:50.148968+0000","last_fullsized":"2023-10-09T06:38:50.148968+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:16:14.105619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.140455+0000","last_change":"2023-10-09T06:38:50.140455+0000","last_active":"2023-10-09T06:38:50.140455+0000","last_peered":"2023-10-09T06:38:50.140455+0000","last_clean":"2023-10-09T06:38:50.140455+0000","last_became_active":"2023-10-09T06:38:50.140189+0000","last_became_peered":"2023-10-09T06:38:50.140189+0000","last_unstale":"2023-10-09T06:38:50.140455+0000","last_undegraded":"2023-10-09T06:38:50.140455+0000","last_fullsized":"2023-10-09T06:38:50.140455+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:31:26.597493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.119558+0000","last_change":"2023-10-09T06:38:49.113566+0000","last_active":"2023-10-09T06:38:49.119558+0000","last_peered":"2023-10-09T06:38:49.119558+0000","last_clean":"2023-10-09T06:38:49.119558+0000","last_became_active":"2023-10-09T06:38:49.113318+0000","last_became_peered":"2023-10-09T06:38:49.113318+0000","last_unstale":"2023-10-09T06:38:49.119558+0000","last_undegraded":"2023-10-09T06:38:49.119558+0000","last_fullsized":"2023-10-09T06:38:49.119558+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:20:46.683927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.137309+0000","last_change":"2023-10-09T06:38:50.137309+0000","last_active":"2023-10-09T06:38:50.137309+0000","last_peered":"2023-10-09T06:38:50.137309+0000","last_clean":"2023-10-09T06:38:50.137309+0000","last_became_active":"2023-10-09T06:38:50.137058+0000","last_became_peered":"2023-10-09T06:38:50.137058+0000","last_unstale":"2023-10-09T06:38:50.137309+0000","last_undegraded":"2023-10-09T06:38:50.137309+0000","last_fullsized":"2023-10-09T06:38:50.137309+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:13:21.243618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.122345+0000","last_change":"2023-10-09T06:38:49.115125+0000","last_active":"2023-10-09T06:38:49.122345+0000","last_peered":"2023-10-09T06:38:49.122345+0000","last_clean":"2023-10-09T06:38:49.122345+0000","last_became_active":"2023-10-09T06:38:49.114810+0000","last_became_peered":"2023-10-09T06:38:49.114810+0000","last_unstale":"2023-10-09T06:38:49.122345+0000","last_undegraded":"2023-10-09T06:38:49.122345+0000","last_fullsized":"2023-10-09T06:38:49.122345+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:03:35.678065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.125809+0000","last_change":"2023-10-09T06:38:50.125809+0000","last_active":"2023-10-09T06:38:50.125809+0000","last_peered":"2023-10-09T06:38:50.125809+0000","last_clean":"2023-10-09T06:38:50.125809+0000","last_became_active":"2023-10-09T06:38:50.125550+0000","last_became_peered":"2023-10-09T06:38:50.125550+0000","last_unstale":"2023-10-09T06:38:50.125809+0000","last_undegraded":"2023-10-09T06:38:50.125809+0000","last_fullsized":"2023-10-09T06:38:50.125809+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:53:25.259901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.123454+0000","last_change":"2023-10-09T06:38:49.117628+0000","last_active":"2023-10-09T06:38:49.123454+0000","last_peered":"2023-10-09T06:38:49.123454+0000","last_clean":"2023-10-09T06:38:49.123454+0000","last_became_active":"2023-10-09T06:38:49.117348+0000","last_became_peered":"2023-10-09T06:38:49.117348+0000","last_unstale":"2023-10-09T06:38:49.123454+0000","last_undegraded":"2023-10-09T06:38:49.123454+0000","last_fullsized":"2023-10-09T06:38:49.123454+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:16:25.959103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.139266+0000","last_change":"2023-10-09T06:38:50.139266+0000","last_active":"2023-10-09T06:38:50.139266+0000","last_peered":"2023-10-09T06:38:50.139266+0000","last_clean":"2023-10-09T06:38:50.139266+0000","last_became_active":"2023-10-09T06:38:50.138981+0000","last_became_peered":"2023-10-09T06:38:50.138981+0000","last_unstale":"2023-10-09T06:38:50.139266+0000","last_undegraded":"2023-10-09T06:38:50.139266+0000","last_fullsized":"2023-10-09T06:38:50.139266+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:25:54.353541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.123612+0000","last_change":"2023-10-09T06:38:49.117365+0000","last_active":"2023-10-09T06:38:49.123612+0000","last_peered":"2023-10-09T06:38:49.123612+0000","last_clean":"2023-10-09T06:38:49.123612+0000","last_became_active":"2023-10-09T06:38:49.117111+0000","last_became_peered":"2023-10-09T06:38:49.117111+0000","last_unstale":"2023-10-09T06:38:49.123612+0000","last_undegraded":"2023-10-09T06:38:49.123612+0000","last_fullsized":"2023-10-09T06:38:49.123612+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:01:00.482030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.126011+0000","last_change":"2023-10-09T06:38:50.126011+0000","last_active":"2023-10-09T06:38:50.126011+0000","last_peered":"2023-10-09T06:38:50.126011+0000","last_clean":"2023-10-09T06:38:50.126011+0000","last_became_active":"2023-10-09T06:38:50.125620+0000","last_became_peered":"2023-10-09T06:38:50.125620+0000","last_unstale":"2023-10-09T06:38:50.126011+0000","last_undegraded":"2023-10-09T06:38:50.126011+0000","last_fullsized":"2023-10-09T06:38:50.126011+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:25:54.353541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.111132+0000","last_change":"2023-10-09T06:38:49.107253+0000","last_active":"2023-10-09T06:38:49.111132+0000","last_peered":"2023-10-09T06:38:49.111132+0000","last_clean":"2023-10-09T06:38:49.111132+0000","last_became_active":"2023-10-09T06:38:49.106952+0000","last_became_peered":"2023-10-09T06:38:49.106952+0000","last_unstale":"2023-10-09T06:38:49.111132+0000","last_undegraded":"2023-10-09T06:38:49.111132+0000","last_fullsized":"2023-10-09T06:38:49.111132+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:52:23.906501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.140824+0000","last_change":"2023-10-09T06:38:50.140824+0000","last_active":"2023-10-09T06:38:50.140824+0000","last_peered":"2023-10-09T06:38:50.140824+0000","last_clean":"2023-10-09T06:38:50.140824+0000","last_became_active":"2023-10-09T06:38:50.140557+0000","last_became_peered":"2023-10-09T06:38:50.140557+0000","last_unstale":"2023-10-09T06:38:50.140824+0000","last_undegraded":"2023-10-09T06:38:50.140824+0000","last_fullsized":"2023-10-09T06:38:50.140824+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:17:07.094300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.122444+0000","last_change":"2023-10-09T06:38:49.119885+0000","last_active":"2023-10-09T06:38:49.122444+0000","last_peered":"2023-10-09T06:38:49.122444+0000","last_clean":"2023-10-09T06:38:49.122444+0000","last_became_active":"2023-10-09T06:38:49.119692+0000","last_became_peered":"2023-10-09T06:38:49.119692+0000","last_unstale":"2023-10-09T06:38:49.122444+0000","last_undegraded":"2023-10-09T06:38:49.122444+0000","last_fullsized":"2023-10-09T06:38:49.122444+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:01:05.239585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.144748+0000","last_change":"2023-10-09T06:38:50.144748+0000","last_active":"2023-10-09T06:38:50.144748+0000","last_peered":"2023-10-09T06:38:50.144748+0000","last_clean":"2023-10-09T06:38:50.144748+0000","last_became_active":"2023-10-09T06:38:50.144387+0000","last_became_peered":"2023-10-09T06:38:50.144387+0000","last_unstale":"2023-10-09T06:38:50.144748+0000","last_undegraded":"2023-10-09T06:38:50.144748+0000","last_fullsized":"2023-10-09T06:38:50.144748+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:17:20.306377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.123266+0000","last_change":"2023-10-09T06:38:49.116981+0000","last_active":"2023-10-09T06:38:49.123266+0000","last_peered":"2023-10-09T06:38:49.123266+0000","last_clean":"2023-10-09T06:38:49.123266+0000","last_became_active":"2023-10-09T06:38:49.116703+0000","last_became_peered":"2023-10-09T06:38:49.116703+0000","last_unstale":"2023-10-09T06:38:49.123266+0000","last_undegraded":"2023-10-09T06:38:49.123266+0000","last_fullsized":"2023-10-09T06:38:49.123266+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:45:03.562181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.141195+0000","last_change":"2023-10-09T06:38:50.141195+0000","last_active":"2023-10-09T06:38:50.141195+0000","last_peered":"2023-10-09T06:38:50.141195+0000","last_clean":"2023-10-09T06:38:50.141195+0000","last_became_active":"2023-10-09T06:38:50.137321+0000","last_became_peered":"2023-10-09T06:38:50.137321+0000","last_unstale":"2023-10-09T06:38:50.141195+0000","last_undegraded":"2023-10-09T06:38:50.141195+0000","last_fullsized":"2023-10-09T06:38:50.141195+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:36:51.158778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1d","version":"163'2","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.147066+0000","last_change":"2023-10-09T06:38:49.120114+0000","last_active":"2023-10-09T06:38:50.147066+0000","last_peered":"2023-10-09T06:38:50.147066+0000","last_clean":"2023-10-09T06:38:50.147066+0000","last_became_active":"2023-10-09T06:38:49.119849+0000","last_became_peered":"2023-10-09T06:38:49.119849+0000","last_unstale":"2023-10-09T06:38:50.147066+0000","last_undegraded":"2023-10-09T06:38:50.147066+0000","last_fullsized":"2023-10-09T06:38:50.147066+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:17:24.554348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":282,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.103482+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:38:49.103482+0000","last_peered":"2023-10-09T06:38:49.103482+0000","last_clean":"2023-10-09T06:38:49.103482+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:38:49.103482+0000","last_undegraded":"2023-10-09T06:38:49.103482+0000","last_fullsized":"2023-10-09T06:38:49.103482+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574952,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303972,"kb_used_data":2588,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067868,"statfs":{"total":96636764160,"available":96325496832,"internally_reserved":0,"allocated":2650112,"data_stored":1506317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:38:42 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51300000000000001,"15min":0.50800000000000001},"min":{"1min":0.36499999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.629,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":3.778},{"interface":"front","average":{"1min":0.498,"5min":0.502,"15min":0.51000000000000001},"min":{"1min":0.33400000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.64700000000000002,"5min":0.70199999999999996,"15min":0.753},"last":1.768}]},{"osd":1,"last update":"Mon Oct 9 06:38:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.50700000000000001,"15min":0.51800000000000002},"min":{"1min":0.36899999999999999,"5min":0.33600000000000002,"15min":0.27100000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":1.2030000000000001},"last":1.5840000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53000000000000003,"15min":0.52600000000000002},"min":{"1min":0.39900000000000002,"5min":0.32900000000000001,"15min":0.29699999999999999},"max":{"1min":0.67000000000000004,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":1.806}]},{"osd":2,"last update":"Mon Oct 9 06:38:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53100000000000003,"15min":0.51700000000000002},"min":{"1min":0.37,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.746,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":3.7210000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53200000000000003,"15min":0.54300000000000004},"min":{"1min":0.36799999999999999,"5min":0.36399999999999999,"15min":0.315},"max":{"1min":0.68100000000000005,"5min":0.80700000000000005,"15min":0.83099999999999996},"last":1.51}]},{"osd":3,"last update":"Mon Oct 9 06:38:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54200000000000004,"15min":0.55000000000000004},"min":{"1min":0.434,"5min":0.35899999999999999,"15min":0.33500000000000002},"max":{"1min":0.76200000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":3.8090000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55100000000000005,"15min":0.58599999999999997},"min":{"1min":0.38400000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69899999999999995,"5min":0.82099999999999995,"15min":1.1479999999999999},"last":1.6870000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:38:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.49299999999999999,"15min":0.51600000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.24399999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.93999999999999995},"last":1.9019999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58099999999999996,"15min":0.60399999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.30299999999999999},"max":{"1min":0.78600000000000003,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":4.8239999999999998}]},{"osd":5,"last update":"Mon Oct 9 06:38:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53700000000000003,"15min":0.56699999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.307},"max":{"1min":0.751,"5min":0.85799999999999998,"15min":0.91100000000000003},"last":1.7230000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60799999999999998,"15min":0.61799999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34200000000000003},"max":{"1min":0.79400000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":3.6909999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:38:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56999999999999995,"15min":0.59099999999999997},"min":{"1min":0.318,"5min":0.318,"15min":0.28699999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.88600000000000001},"last":3.5529999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58199999999999996,"15min":0.61099999999999999},"min":{"1min":0.34399999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.81999999999999995,"5min":0.82499999999999996,"15min":0.84599999999999997},"last":4.7789999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574951,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303392,"kb_used_data":2008,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068448,"statfs":{"total":96636764160,"available":96326090752,"internally_reserved":0,"allocated":2056192,"data_stored":915949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.35899999999999999,"5min":0.35399999999999998,"15min":0.30399999999999999},"max":{"1min":0.70699999999999996,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.51100000000000001,"15min":0.51200000000000001},"min":{"1min":0.36299999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69699999999999995,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.61199999999999999}]},{"osd":1,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50700000000000001,"15min":0.51200000000000001},"min":{"1min":0.43099999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.72699999999999998,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53500000000000003,"15min":0.53300000000000003},"min":{"1min":0.437,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.69099999999999995,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.48099999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.34300000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67800000000000005,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.53000000000000003},"min":{"1min":0.39300000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.499}]},{"osd":3,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.39800000000000002,"5min":0.38900000000000001,"15min":0.33500000000000002},"max":{"1min":0.79000000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54400000000000004,"15min":0.67100000000000004},"min":{"1min":0.40300000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":4.7510000000000003},"last":0.51100000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.46000000000000002,"15min":0.47499999999999998},"min":{"1min":0.28999999999999998,"5min":0.28199999999999997,"15min":0.27900000000000003},"max":{"1min":0.63700000000000001,"5min":0.74299999999999999,"15min":0.76400000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56699999999999995,"15min":0.58299999999999996},"min":{"1min":0.51500000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.75,"5min":0.80800000000000005,"15min":0.83699999999999997},"last":0.52300000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.52100000000000002,"15min":0.53100000000000003},"min":{"1min":0.37,"5min":0.246,"15min":0.246},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.375},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57199999999999995,"15min":0.59299999999999997},"min":{"1min":0.47299999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70899999999999996,"5min":0.78000000000000003,"15min":0.86599999999999999},"last":0.53600000000000003}]},{"osd":7,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55600000000000005,"15min":0.57999999999999996},"min":{"1min":0.33400000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82399999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.438,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80500000000000005,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.59599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303392,"kb_used_data":2008,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068448,"statfs":{"total":96636764160,"available":96326090752,"internally_reserved":0,"allocated":2056192,"data_stored":915949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.317,"15min":0.311},"min":{"1min":0.22500000000000001,"5min":0.192,"15min":0.192},"max":{"1min":0.47699999999999998,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.318},{"interface":"front","average":{"1min":0.37,"5min":0.35399999999999998,"15min":0.35899999999999999},"min":{"1min":0.27700000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.376}]},{"osd":2,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34699999999999998,"15min":0.34999999999999998},"min":{"1min":0.23899999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.51600000000000001,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38400000000000001,"15min":0.38500000000000001},"min":{"1min":0.25800000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.60399999999999998,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.40999999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.36599999999999999,"15min":0.36299999999999999},"min":{"1min":0.24299999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.503,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.374,"5min":0.35099999999999998,"15min":0.51600000000000001},"min":{"1min":0.255,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":5.8529999999999998},"last":0.39600000000000002}]},{"osd":4,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.36499999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59799999999999998,"15min":0.58799999999999997},"min":{"1min":0.46700000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.64300000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57899999999999996,"15min":0.72299999999999998},"min":{"1min":0.40699999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":5.9459999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.42499999999999999,"5min":0.38900000000000001,"15min":0.34300000000000003},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.62}]},{"osd":6,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54800000000000004,"15min":0.69399999999999995},"min":{"1min":0.38700000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":5.9160000000000004},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.40999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88600000000000001,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.54900000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.58399999999999996},"min":{"1min":0.42899999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.97099999999999997},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56899999999999995,"15min":0.72899999999999998},"min":{"1min":0.41799999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":5.9729999999999999},"last":0.53900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574953,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303992,"kb_used_data":2608,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067848,"statfs":{"total":96636764160,"available":96325476352,"internally_reserved":0,"allocated":2670592,"data_stored":1513724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.316,"15min":0.318},"min":{"1min":0.255,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.33000000000000002,"15min":0.34399999999999997},"min":{"1min":0.24199999999999999,"5min":0.222,"15min":0.214},"max":{"1min":0.48699999999999999,"5min":0.53600000000000003,"15min":0.94499999999999995},"last":0.24199999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.36299999999999999,"15min":0.36399999999999999},"min":{"1min":0.223,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.53200000000000003,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.223},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.379,"15min":0.38800000000000001},"min":{"1min":0.29099999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.52900000000000003,"5min":0.69399999999999995,"15min":0.91700000000000004},"last":0.29099999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.33300000000000002,"15min":0.38},"min":{"1min":0.26000000000000001,"5min":0.23599999999999999,"15min":0.23300000000000001},"max":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":1.554},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.37,"15min":0.39600000000000002},"min":{"1min":0.30599999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54500000000000004,"5min":0.58099999999999996,"15min":0.79000000000000004},"last":0.30599999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.75900000000000001,"5min":0.80200000000000005,"15min":1.212},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56200000000000006,"15min":0.70499999999999996},"min":{"1min":0.39000000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77800000000000002,"5min":0.89900000000000002,"15min":5.484},"last":0.39000000000000001}]},{"osd":5,"last update":"Mon Oct 9 06:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56100000000000005,"15min":0.70899999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80100000000000005,"5min":0.81399999999999995,"15min":5.5449999999999999},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56899999999999995,"15min":0.57499999999999996},"min":{"1min":0.41399999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.80100000000000005,"5min":0.84499999999999997,"15min":1.113},"last":0.41399999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59399999999999997,"15min":0.60499999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.78700000000000003,"5min":0.85599999999999998,"15min":1.0529999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.42899999999999999,"5min":0.42799999999999999,"15min":0.41599999999999998},"max":{"1min":0.78700000000000003,"5min":0.878,"15min":1.169},"last":0.42899999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58199999999999996,"15min":0.59199999999999997},"min":{"1min":0.40200000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.80300000000000005,"5min":0.85499999999999998,"15min":1.2390000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56599999999999995,"15min":0.73099999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77000000000000002,"5min":0.86299999999999999,"15min":5.5800000000000001},"last":0.35899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303424,"kb_used_data":2040,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068416,"statfs":{"total":96636764160,"available":96326057984,"internally_reserved":0,"allocated":2088960,"data_stored":924304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.316,"15min":0.35299999999999998},"min":{"1min":0.23200000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":1.4199999999999999},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.55900000000000005},"min":{"1min":0.27500000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.51600000000000001,"5min":0.53800000000000003,"15min":7.2699999999999996},"last":0.23300000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.317,"15min":0.34399999999999997},"min":{"1min":0.23999999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.65400000000000003},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.37,"15min":0.61299999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":7.327},"last":0.313}]},{"osd":3,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.36799999999999999,"15min":0.39100000000000001},"min":{"1min":0.29399999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.68600000000000005},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.36799999999999999,"15min":0.39200000000000002},"min":{"1min":0.27900000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.72699999999999998},"last":0.307}]},{"osd":4,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54500000000000004,"15min":0.54400000000000004},"min":{"1min":0.40799999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.78400000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56200000000000006,"15min":0.78600000000000003},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82699999999999996,"5min":0.84699999999999998,"15min":7.2389999999999999},"last":0.51000000000000001}]},{"osd":5,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.58099999999999996,"15min":0.91500000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82699999999999996,"5min":0.83799999999999997,"15min":10.890000000000001},"last":0.442},{"interface":"front","average":{"1min":0.627,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":0.84299999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.52000000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56399999999999995,"15min":0.79100000000000004},"min":{"1min":0.42999999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.748,"5min":0.77300000000000002,"15min":7.2999999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.45800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.53200000000000003}]},{"osd":7,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59299999999999997,"15min":0.82199999999999995},"min":{"1min":0.47499999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":7.1879999999999997},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60399999999999998,"15min":0.624},"min":{"1min":0.49399999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.90500000000000003},"last":0.55600000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574953,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303392,"kb_used_data":2008,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068448,"statfs":{"total":96636764160,"available":96326090752,"internally_reserved":0,"allocated":2056192,"data_stored":915949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.32800000000000001,"15min":0.33400000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.20599999999999999},"max":{"1min":0.51300000000000001,"5min":0.61699999999999999,"15min":5.8700000000000001},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34699999999999998,"15min":0.36599999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":5.6769999999999996},"last":0.36199999999999999}]},{"osd":1,"last update":"Mon Oct 9 06:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.35099999999999998,"15min":0.35899999999999999},"min":{"1min":0.21299999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.44,"5min":0.59499999999999997,"15min":4.0469999999999997},"last":0.372},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.377,"15min":0.38900000000000001},"min":{"1min":0.30099999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.624,"5min":0.68600000000000005,"15min":4.1509999999999998},"last":0.34999999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.35199999999999998,"15min":0.372},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.222},"max":{"1min":0.51900000000000002,"5min":0.56799999999999995,"15min":3.1139999999999999},"last":0.311},{"interface":"front","average":{"1min":0.37,"5min":0.374,"15min":0.38500000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.54500000000000004,"5min":0.66900000000000004,"15min":3.3180000000000001},"last":0.41299999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55400000000000005,"15min":0.56999999999999995},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.68799999999999994,"5min":0.81699999999999995,"15min":3.0899999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.749,"5min":0.79200000000000004,"15min":3.0710000000000002},"last":0.496}]},{"osd":5,"last update":"Mon Oct 9 06:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56799999999999995,"15min":0.58799999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.32700000000000001},"max":{"1min":0.745,"5min":0.877,"15min":6.2300000000000004},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58799999999999997,"15min":0.59499999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.755,"5min":0.83099999999999996,"15min":3.0219999999999998},"last":0.42499999999999999}]},{"osd":6,"last update":"Mon Oct 9 06:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80800000000000005,"5min":0.89200000000000002,"15min":2.6850000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63800000000000001,"15min":0.65100000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.76800000000000002,"5min":0.94299999999999995,"15min":6.5970000000000004},"last":0.46200000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.41699999999999998,"5min":0.40200000000000002,"15min":0.38200000000000001},"max":{"1min":0.81999999999999995,"5min":0.88500000000000001,"15min":4.2629999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58899999999999997,"15min":0.61199999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.79100000000000004,"5min":0.82899999999999996,"15min":7.3650000000000002},"last":0.63300000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303392,"kb_used_data":2008,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068448,"statfs":{"total":96636764160,"available":96326090752,"internally_reserved":0,"allocated":2056192,"data_stored":915949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.51600000000000001,"15min":0.55100000000000005},"min":{"1min":0.373,"5min":0.28100000000000003,"15min":0.27000000000000002},"max":{"1min":0.82599999999999996,"5min":0.88900000000000001,"15min":1.083},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52500000000000002,"15min":0.52000000000000002},"min":{"1min":0.36499999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.76100000000000001,"5min":0.97799999999999998,"15min":1.054},"last":0.65600000000000003}]},{"osd":1,"last update":"Mon Oct 9 06:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53500000000000003,"15min":0.54700000000000004},"min":{"1min":0.28399999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.71099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.499},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.501,"15min":0.496},"min":{"1min":0.34000000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.75700000000000001,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.46100000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52000000000000002,"15min":0.53800000000000003},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":1.0249999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54600000000000004,"15min":0.58599999999999997},"min":{"1min":0.38100000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.66800000000000004,"5min":0.90500000000000003,"15min":0.93799999999999994},"last":0.60799999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52100000000000002,"15min":0.54800000000000004},"min":{"1min":0.33300000000000002,"5min":0.32200000000000001,"15min":0.29199999999999998},"max":{"1min":0.71599999999999997,"5min":0.86899999999999999,"15min":0.98499999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56699999999999995,"15min":0.58699999999999997},"min":{"1min":0.40699999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77500000000000002,"5min":0.92900000000000005,"15min":1.1319999999999999},"last":0.53300000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.51300000000000001,"15min":0.54800000000000004},"min":{"1min":0.34399999999999997,"5min":0.30399999999999999,"15min":0.27300000000000002},"max":{"1min":0.73999999999999999,"5min":0.81100000000000005,"15min":0.998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57599999999999996,"15min":0.60399999999999998},"min":{"1min":0.46000000000000002,"5min":0.36299999999999999,"15min":0.35099999999999998},"max":{"1min":0.82399999999999995,"5min":0.95599999999999996,"15min":0.97999999999999998},"last":0.59199999999999997}]},{"osd":6,"last update":"Mon Oct 9 06:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54800000000000004,"15min":0.56599999999999995},"min":{"1min":0.39400000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70299999999999996,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56899999999999995,"15min":0.59599999999999997},"min":{"1min":0.41399999999999998,"5min":0.34999999999999998,"15min":0.32700000000000001},"max":{"1min":0.78300000000000003,"5min":1.01,"15min":1.01},"last":0.63600000000000001}]},{"osd":7,"last update":"Mon Oct 9 06:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53800000000000003,"15min":0.56699999999999995},"min":{"1min":0.35299999999999998,"5min":0.33600000000000002,"15min":0.28299999999999997},"max":{"1min":0.70299999999999996,"5min":0.83199999999999996,"15min":0.90000000000000002},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60499999999999998,"15min":0.64100000000000001},"min":{"1min":0.45000000000000001,"5min":0.34999999999999998,"15min":0.32200000000000001},"max":{"1min":0.81999999999999995,"5min":0.98999999999999999,"15min":1.175},"last":0.622}]}]},{"osd":5,"up_from":13,"seq":55834574952,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303404,"kb_used_data":2020,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068436,"statfs":{"total":96636764160,"available":96326078464,"internally_reserved":0,"allocated":2068480,"data_stored":922712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.502,"15min":0.50900000000000001},"min":{"1min":0.40100000000000002,"5min":0.29299999999999998,"15min":0.28599999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.86599999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51200000000000001,"15min":0.51500000000000001},"min":{"1min":0.38200000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72899999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.72899999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.504,"15min":0.51000000000000001},"min":{"1min":0.38,"5min":0.311,"15min":0.311},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.78200000000000003},"last":0.63},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.52200000000000002,"15min":0.53100000000000003},"min":{"1min":0.41199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.81599999999999995},"last":0.76600000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.47199999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.83999999999999997},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57199999999999995,"15min":0.58499999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.39500000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79900000000000004},"last":0.79300000000000004}]},{"osd":3,"last update":"Mon Oct 9 06:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55200000000000005,"15min":0.54600000000000004},"min":{"1min":0.39100000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.55900000000000005},"min":{"1min":0.40699999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.69199999999999995,"5min":0.71099999999999997,"15min":0.81100000000000005},"last":0.67400000000000004}]},{"osd":4,"last update":"Mon Oct 9 06:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.50700000000000001,"15min":0.5},"min":{"1min":0.34200000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68400000000000005,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.40300000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.80700000000000005},"last":0.54200000000000004}]},{"osd":6,"last update":"Mon Oct 9 06:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55500000000000005,"15min":0.53900000000000003},"min":{"1min":0.378,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.83199999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56499999999999995,"15min":0.56200000000000006},"min":{"1min":0.40200000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71799999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.64600000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.35999999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73199999999999998,"5min":0.79500000000000004,"15min":0.85599999999999998},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.44700000000000001,"5min":0.35199999999999998,"15min":0.314},"max":{"1min":0.77400000000000002,"5min":0.84499999999999997,"15min":0.90600000000000003},"last":0.745}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:38:54.185 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:38:54.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.332+0000 7fdd4b096700 1 Processor -- start 2023-10-09T06:38:54.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.332+0000 7fdd4b096700 1 -- start start 2023-10-09T06:38:54.336 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.333+0000 7fdd4b096700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd4406c0a0 0x7fdd440caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:54.336 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.333+0000 7fdd4b096700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdd4415df50 0x7fdd4415e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:54.336 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.333+0000 7fdd4b096700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd4415d7e0 con 0x7fdd4406bcd0 2023-10-09T06:38:54.336 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.333+0000 7fdd4b096700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd4415dac0 con 0x7fdd4415df50 2023-10-09T06:38:54.336 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.333+0000 7fdd4b096700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4415aa30 con 0x7fdd4406c0a0 2023-10-09T06:38:54.337 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.333+0000 7fdd49633700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdd4415df50 0x7fdd4415e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:54.337 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.333+0000 7fdd43fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd4406c0a0 0x7fdd440caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:54.337 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.333+0000 7fdd48e32700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fdd4406bcd0 0x7fdd4415a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:51348/0 (socket says 172.21.15.163:51348) 2023-10-09T06:38:54.337 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.333+0000 7fdd48e32700 1 -- 172.21.15.163:0/1236263716 learned_addr learned my addr 172.21.15.163:0/1236263716 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:54.337 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.333+0000 7fdd49633700 1 -- 172.21.15.163:0/1236263716 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd4406c0a0 msgr2=0x7fdd440caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:54.337 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.333+0000 7fdd49633700 1 --2- 172.21.15.163:0/1236263716 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd4406c0a0 0x7fdd440caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:54.337 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.333+0000 7fdd49633700 1 -- 172.21.15.163:0/1236263716 >> v1:172.21.15.181:6789/0 conn(0x7fdd4406bcd0 legacy=0x7fdd4415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:54.337 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.333+0000 7fdd437fe700 1 -- 172.21.15.163:0/1236263716 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3918194972 0 0) 0x7fdd4415d7e0 con 0x7fdd4406bcd0 2023-10-09T06:38:54.338 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.333+0000 7fdd49633700 1 -- 172.21.15.163:0/1236263716 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4415ad10 con 0x7fdd4415df50 2023-10-09T06:38:54.338 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.333+0000 7fdd43fff700 1 --2- 172.21.15.163:0/1236263716 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd4406c0a0 0x7fdd440caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:38:54.338 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.334+0000 7fdd49633700 1 --2- 172.21.15.163:0/1236263716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd4415df50 0x7fdd4415e3c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0098e0 tx=0x7fdd2c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4c5b1b0eb15b6c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:54.338 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.334+0000 7fdd437fe700 1 -- 172.21.15.163:0/1236263716 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c00b640 con 0x7fdd4415df50 2023-10-09T06:38:54.338 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.334+0000 7fdd437fe700 1 -- 172.21.15.163:0/1236263716 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd2c00b7a0 con 0x7fdd4415df50 2023-10-09T06:38:54.338 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.334+0000 7fdd437fe700 1 -- 172.21.15.163:0/1236263716 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c00bb40 con 0x7fdd4415df50 2023-10-09T06:38:54.339 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.334+0000 7fdd4b096700 1 -- 172.21.15.163:0/1236263716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd4415df50 msgr2=0x7fdd4415e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:54.339 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.334+0000 7fdd4b096700 1 --2- 172.21.15.163:0/1236263716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd4415df50 0x7fdd4415e3c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0098e0 tx=0x7fdd2c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:54.339 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.334+0000 7fdd4b096700 1 -- 172.21.15.163:0/1236263716 shutdown_connections 2023-10-09T06:38:54.339 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.334+0000 7fdd4b096700 1 --2- 172.21.15.163:0/1236263716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd4415df50 0x7fdd4415e3c0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:54.339 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.334+0000 7fdd4b096700 1 --2- 172.21.15.163:0/1236263716 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd4406c0a0 0x7fdd440caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:54.339 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.334+0000 7fdd4b096700 1 -- 172.21.15.163:0/1236263716 >> 172.21.15.163:0/1236263716 conn(0x7fdd440ce600 msgr2=0x7fdd440cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:54.339 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.334+0000 7fdd4b096700 1 -- 172.21.15.163:0/1236263716 shutdown_connections 2023-10-09T06:38:54.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.334+0000 7fdd4b096700 1 -- 172.21.15.163:0/1236263716 wait complete. 2023-10-09T06:38:54.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.335+0000 7fdd4b096700 1 Processor -- start 2023-10-09T06:38:54.344 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.336+0000 7fdd4b096700 1 -- start start 2023-10-09T06:38:54.344 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.336+0000 7fdd4b096700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd4406bcd0 0x7fdd44146450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:54.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.336+0000 7fdd4b096700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd4406c0a0 0x7fdd4416acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:54.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.336+0000 7fdd4b096700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd4415df50 0x7fdd441587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:54.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.336+0000 7fdd4b096700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd44159020 con 0x7fdd4406c0a0 2023-10-09T06:38:54.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.336+0000 7fdd4b096700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4415ee70 con 0x7fdd4415df50 2023-10-09T06:38:54.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.336+0000 7fdd4b096700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4415f150 con 0x7fdd4406bcd0 2023-10-09T06:38:54.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.336+0000 7fdd43fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd4406c0a0 0x7fdd4416acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:54.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.336+0000 7fdd43fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd4406c0a0 0x7fdd4416acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44652/0 (socket says 172.21.15.163:44652) 2023-10-09T06:38:54.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.336+0000 7fdd43fff700 1 -- 172.21.15.163:0/2484578032 learned_addr learned my addr 172.21.15.163:0/2484578032 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:54.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.337+0000 7fdd48e32700 1 --2- 172.21.15.163:0/2484578032 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd4406bcd0 0x7fdd44146450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:54.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.337+0000 7fdd49633700 1 --2- 172.21.15.163:0/2484578032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd4415df50 0x7fdd441587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:54.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.337+0000 7fdd43fff700 1 -- 172.21.15.163:0/2484578032 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd4406bcd0 msgr2=0x7fdd44146450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:54.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.337+0000 7fdd43fff700 1 --2- 172.21.15.163:0/2484578032 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd4406bcd0 0x7fdd44146450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:54.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.337+0000 7fdd43fff700 1 -- 172.21.15.163:0/2484578032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd4415df50 msgr2=0x7fdd441587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:54.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.337+0000 7fdd43fff700 1 --2- 172.21.15.163:0/2484578032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd4415df50 0x7fdd441587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:54.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.337+0000 7fdd43fff700 1 -- 172.21.15.163:0/2484578032 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd34009420 con 0x7fdd4406c0a0 2023-10-09T06:38:54.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.337+0000 7fdd43fff700 1 --2- 172.21.15.163:0/2484578032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd4406c0a0 0x7fdd4416acf0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400ec80 tx=0x7fdd340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:54.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.337+0000 7fdd40ff9700 1 -- 172.21.15.163:0/2484578032 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3400c9d0 con 0x7fdd4406c0a0 2023-10-09T06:38:54.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.337+0000 7fdd40ff9700 1 -- 172.21.15.163:0/2484578032 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3400cb30 con 0x7fdd4406c0a0 2023-10-09T06:38:54.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.337+0000 7fdd4b096700 1 -- 172.21.15.163:0/2484578032 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd2c009420 con 0x7fdd4406c0a0 2023-10-09T06:38:54.347 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.337+0000 7fdd40ff9700 1 -- 172.21.15.163:0/2484578032 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3400ceb0 con 0x7fdd4406c0a0 2023-10-09T06:38:54.347 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.337+0000 7fdd4b096700 1 -- 172.21.15.163:0/2484578032 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd4415f710 con 0x7fdd4406c0a0 2023-10-09T06:38:54.347 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.338+0000 7fdd4b096700 1 -- 172.21.15.163:0/2484578032 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd4415fa80 con 0x7fdd4406c0a0 2023-10-09T06:38:54.348 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.338+0000 7fdd40ff9700 1 -- 172.21.15.163:0/2484578032 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fdd34013680 con 0x7fdd4406c0a0 2023-10-09T06:38:54.348 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.339+0000 7fdd40ff9700 1 --2- 172.21.15.163:0/2484578032 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdd24067320 0x7fdd240697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:54.348 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.339+0000 7fdd40ff9700 1 -- 172.21.15.163:0/2484578032 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdd34085080 con 0x7fdd4406c0a0 2023-10-09T06:38:54.348 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.339+0000 7fdd48e32700 1 --2- 172.21.15.163:0/2484578032 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdd24067320 0x7fdd240697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:54.348 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.339+0000 7fdd48e32700 1 --2- 172.21.15.163:0/2484578032 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdd24067320 0x7fdd240697d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c002230 tx=0x7fdd3c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:54.349 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.343+0000 7fdd40ff9700 1 -- 172.21.15.163:0/2484578032 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd340b65a0 con 0x7fdd4406c0a0 2023-10-09T06:38:54.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.520+0000 7fdd4b096700 1 -- 172.21.15.163:0/2484578032 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdd4415efe0 con 0x7fdd24067320 2023-10-09T06:38:54.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.528+0000 7fdd40ff9700 1 -- 172.21.15.163:0/2484578032 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290201 (secure 0 0 0) 0x7fdd4415efe0 con 0x7fdd24067320 2023-10-09T06:38:54.533 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:54.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.532+0000 7fdd4b096700 1 -- 172.21.15.163:0/2484578032 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdd24067320 msgr2=0x7fdd240697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:54.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.532+0000 7fdd4b096700 1 --2- 172.21.15.163:0/2484578032 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdd24067320 0x7fdd240697d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c002230 tx=0x7fdd3c009040 comp rx=0 tx=0).stop 2023-10-09T06:38:54.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.532+0000 7fdd4b096700 1 -- 172.21.15.163:0/2484578032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd4406c0a0 msgr2=0x7fdd4416acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:54.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.532+0000 7fdd4b096700 1 --2- 172.21.15.163:0/2484578032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd4406c0a0 0x7fdd4416acf0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400ec80 tx=0x7fdd340091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:54.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.532+0000 7fdd4b096700 1 -- 172.21.15.163:0/2484578032 shutdown_connections 2023-10-09T06:38:54.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.532+0000 7fdd4b096700 1 --2- 172.21.15.163:0/2484578032 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fdd24067320 0x7fdd240697d0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:54.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.532+0000 7fdd4b096700 1 --2- 172.21.15.163:0/2484578032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd4415df50 0x7fdd441587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:54.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.532+0000 7fdd4b096700 1 --2- 172.21.15.163:0/2484578032 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd4406bcd0 0x7fdd44146450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:54.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.532+0000 7fdd4b096700 1 --2- 172.21.15.163:0/2484578032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd4406c0a0 0x7fdd4416acf0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:54.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.532+0000 7fdd4b096700 1 -- 172.21.15.163:0/2484578032 >> 172.21.15.163:0/2484578032 conn(0x7fdd440ce600 msgr2=0x7fdd4406cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:54.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.532+0000 7fdd4b096700 1 -- 172.21.15.163:0/2484578032 shutdown_connections 2023-10-09T06:38:54.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.532+0000 7fdd4b096700 1 -- 172.21.15.163:0/2484578032 wait complete. 2023-10-09T06:38:54.537 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:38:54.546 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2023-10-09T06:38:54.418074+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":97,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2428360,"kb_used_data":17288,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752546360,"statfs":{"total":773094113280,"available":770607472640,"internally_reserved":0,"allocated":17702912,"data_stored":8530853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001204"},"pg_stats":[{"pgid":"29.3f","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.142209+0000","last_change":"2023-10-09T06:38:50.142209+0000","last_active":"2023-10-09T06:38:50.142209+0000","last_peered":"2023-10-09T06:38:50.142209+0000","last_clean":"2023-10-09T06:38:50.142209+0000","last_became_active":"2023-10-09T06:38:50.137271+0000","last_became_peered":"2023-10-09T06:38:50.137271+0000","last_unstale":"2023-10-09T06:38:50.142209+0000","last_undegraded":"2023-10-09T06:38:50.142209+0000","last_fullsized":"2023-10-09T06:38:50.142209+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:53:25.259901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.142647+0000","last_change":"2023-10-09T06:38:50.142647+0000","last_active":"2023-10-09T06:38:50.142647+0000","last_peered":"2023-10-09T06:38:50.142647+0000","last_clean":"2023-10-09T06:38:50.142647+0000","last_became_active":"2023-10-09T06:38:50.142421+0000","last_became_peered":"2023-10-09T06:38:50.142421+0000","last_unstale":"2023-10-09T06:38:50.142647+0000","last_undegraded":"2023-10-09T06:38:50.142647+0000","last_fullsized":"2023-10-09T06:38:50.142647+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:17:07.094300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.128345+0000","last_change":"2023-10-09T06:38:50.128345+0000","last_active":"2023-10-09T06:38:50.128345+0000","last_peered":"2023-10-09T06:38:50.128345+0000","last_clean":"2023-10-09T06:38:50.128345+0000","last_became_active":"2023-10-09T06:38:50.128047+0000","last_became_peered":"2023-10-09T06:38:50.128047+0000","last_unstale":"2023-10-09T06:38:50.128345+0000","last_undegraded":"2023-10-09T06:38:50.128345+0000","last_fullsized":"2023-10-09T06:38:50.128345+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:07:11.948474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.139495+0000","last_change":"2023-10-09T06:38:50.139495+0000","last_active":"2023-10-09T06:38:50.139495+0000","last_peered":"2023-10-09T06:38:50.139495+0000","last_clean":"2023-10-09T06:38:50.139495+0000","last_became_active":"2023-10-09T06:38:50.139119+0000","last_became_peered":"2023-10-09T06:38:50.139119+0000","last_unstale":"2023-10-09T06:38:50.139495+0000","last_undegraded":"2023-10-09T06:38:50.139495+0000","last_fullsized":"2023-10-09T06:38:50.139495+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:23:19.098076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.129120+0000","last_change":"2023-10-09T06:38:50.129120+0000","last_active":"2023-10-09T06:38:50.129120+0000","last_peered":"2023-10-09T06:38:50.129120+0000","last_clean":"2023-10-09T06:38:50.129120+0000","last_became_active":"2023-10-09T06:38:50.128850+0000","last_became_peered":"2023-10-09T06:38:50.128850+0000","last_unstale":"2023-10-09T06:38:50.129120+0000","last_undegraded":"2023-10-09T06:38:50.129120+0000","last_fullsized":"2023-10-09T06:38:50.129120+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:55:40.976759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.136957+0000","last_change":"2023-10-09T06:38:50.136957+0000","last_active":"2023-10-09T06:38:50.136957+0000","last_peered":"2023-10-09T06:38:50.136957+0000","last_clean":"2023-10-09T06:38:50.136957+0000","last_became_active":"2023-10-09T06:38:50.136676+0000","last_became_peered":"2023-10-09T06:38:50.136676+0000","last_unstale":"2023-10-09T06:38:50.136957+0000","last_undegraded":"2023-10-09T06:38:50.136957+0000","last_fullsized":"2023-10-09T06:38:50.136957+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:34:34.991435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.126270+0000","last_change":"2023-10-09T06:38:50.126270+0000","last_active":"2023-10-09T06:38:50.126270+0000","last_peered":"2023-10-09T06:38:50.126270+0000","last_clean":"2023-10-09T06:38:50.126270+0000","last_became_active":"2023-10-09T06:38:50.125929+0000","last_became_peered":"2023-10-09T06:38:50.125929+0000","last_unstale":"2023-10-09T06:38:50.126270+0000","last_undegraded":"2023-10-09T06:38:50.126270+0000","last_fullsized":"2023-10-09T06:38:50.126270+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:26:23.603659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.143893+0000","last_change":"2023-10-09T06:38:50.143893+0000","last_active":"2023-10-09T06:38:50.143893+0000","last_peered":"2023-10-09T06:38:50.143893+0000","last_clean":"2023-10-09T06:38:50.143893+0000","last_became_active":"2023-10-09T06:38:50.143698+0000","last_became_peered":"2023-10-09T06:38:50.143698+0000","last_unstale":"2023-10-09T06:38:50.143893+0000","last_undegraded":"2023-10-09T06:38:50.143893+0000","last_fullsized":"2023-10-09T06:38:50.143893+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:41:59.805054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.136862+0000","last_change":"2023-10-09T06:38:50.136862+0000","last_active":"2023-10-09T06:38:50.136862+0000","last_peered":"2023-10-09T06:38:50.136862+0000","last_clean":"2023-10-09T06:38:50.136862+0000","last_became_active":"2023-10-09T06:38:50.136547+0000","last_became_peered":"2023-10-09T06:38:50.136547+0000","last_unstale":"2023-10-09T06:38:50.136862+0000","last_undegraded":"2023-10-09T06:38:50.136862+0000","last_fullsized":"2023-10-09T06:38:50.136862+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:23:22.572456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.129908+0000","last_change":"2023-10-09T06:38:50.129908+0000","last_active":"2023-10-09T06:38:50.129908+0000","last_peered":"2023-10-09T06:38:50.129908+0000","last_clean":"2023-10-09T06:38:50.129908+0000","last_became_active":"2023-10-09T06:38:50.129558+0000","last_became_peered":"2023-10-09T06:38:50.129558+0000","last_unstale":"2023-10-09T06:38:50.129908+0000","last_undegraded":"2023-10-09T06:38:50.129908+0000","last_fullsized":"2023-10-09T06:38:50.129908+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:17:20.306377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.148995+0000","last_change":"2023-10-09T06:38:50.148995+0000","last_active":"2023-10-09T06:38:50.148995+0000","last_peered":"2023-10-09T06:38:50.148995+0000","last_clean":"2023-10-09T06:38:50.148995+0000","last_became_active":"2023-10-09T06:38:50.148046+0000","last_became_peered":"2023-10-09T06:38:50.148046+0000","last_unstale":"2023-10-09T06:38:50.148995+0000","last_undegraded":"2023-10-09T06:38:50.148995+0000","last_fullsized":"2023-10-09T06:38:50.148995+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:08:51.778097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.139409+0000","last_change":"2023-10-09T06:38:50.139409+0000","last_active":"2023-10-09T06:38:50.139409+0000","last_peered":"2023-10-09T06:38:50.139409+0000","last_clean":"2023-10-09T06:38:50.139409+0000","last_became_active":"2023-10-09T06:38:50.138790+0000","last_became_peered":"2023-10-09T06:38:50.138790+0000","last_unstale":"2023-10-09T06:38:50.139409+0000","last_undegraded":"2023-10-09T06:38:50.139409+0000","last_fullsized":"2023-10-09T06:38:50.139409+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:26:23.603659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.140569+0000","last_change":"2023-10-09T06:38:50.140569+0000","last_active":"2023-10-09T06:38:50.140569+0000","last_peered":"2023-10-09T06:38:50.140569+0000","last_clean":"2023-10-09T06:38:50.140569+0000","last_became_active":"2023-10-09T06:38:50.140323+0000","last_became_peered":"2023-10-09T06:38:50.140323+0000","last_unstale":"2023-10-09T06:38:50.140569+0000","last_undegraded":"2023-10-09T06:38:50.140569+0000","last_fullsized":"2023-10-09T06:38:50.140569+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:49:03.761584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.137128+0000","last_change":"2023-10-09T06:38:50.137128+0000","last_active":"2023-10-09T06:38:50.137128+0000","last_peered":"2023-10-09T06:38:50.137128+0000","last_clean":"2023-10-09T06:38:50.137128+0000","last_became_active":"2023-10-09T06:38:50.136724+0000","last_became_peered":"2023-10-09T06:38:50.136724+0000","last_unstale":"2023-10-09T06:38:50.137128+0000","last_undegraded":"2023-10-09T06:38:50.137128+0000","last_fullsized":"2023-10-09T06:38:50.137128+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:42:10.906439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.125984+0000","last_change":"2023-10-09T06:38:50.125984+0000","last_active":"2023-10-09T06:38:50.125984+0000","last_peered":"2023-10-09T06:38:50.125984+0000","last_clean":"2023-10-09T06:38:50.125984+0000","last_became_active":"2023-10-09T06:38:50.125740+0000","last_became_peered":"2023-10-09T06:38:50.125740+0000","last_unstale":"2023-10-09T06:38:50.125984+0000","last_undegraded":"2023-10-09T06:38:50.125984+0000","last_fullsized":"2023-10-09T06:38:50.125984+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:19:55.720934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.140091+0000","last_change":"2023-10-09T06:38:50.140091+0000","last_active":"2023-10-09T06:38:50.140091+0000","last_peered":"2023-10-09T06:38:50.140091+0000","last_clean":"2023-10-09T06:38:50.140091+0000","last_became_active":"2023-10-09T06:38:50.139725+0000","last_became_peered":"2023-10-09T06:38:50.139725+0000","last_unstale":"2023-10-09T06:38:50.140091+0000","last_undegraded":"2023-10-09T06:38:50.140091+0000","last_fullsized":"2023-10-09T06:38:50.140091+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:01:54.560937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.130575+0000","last_change":"2023-10-09T06:38:50.130575+0000","last_active":"2023-10-09T06:38:50.130575+0000","last_peered":"2023-10-09T06:38:50.130575+0000","last_clean":"2023-10-09T06:38:50.130575+0000","last_became_active":"2023-10-09T06:38:50.130366+0000","last_became_peered":"2023-10-09T06:38:50.130366+0000","last_unstale":"2023-10-09T06:38:50.130575+0000","last_undegraded":"2023-10-09T06:38:50.130575+0000","last_fullsized":"2023-10-09T06:38:50.130575+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:48:16.621531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.128746+0000","last_change":"2023-10-09T06:38:50.128746+0000","last_active":"2023-10-09T06:38:50.128746+0000","last_peered":"2023-10-09T06:38:50.128746+0000","last_clean":"2023-10-09T06:38:50.128746+0000","last_became_active":"2023-10-09T06:38:50.128567+0000","last_became_peered":"2023-10-09T06:38:50.128567+0000","last_unstale":"2023-10-09T06:38:50.128746+0000","last_undegraded":"2023-10-09T06:38:50.128746+0000","last_fullsized":"2023-10-09T06:38:50.128746+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:46:02.257675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.138027+0000","last_change":"2023-10-09T06:38:50.138027+0000","last_active":"2023-10-09T06:38:50.138027+0000","last_peered":"2023-10-09T06:38:50.138027+0000","last_clean":"2023-10-09T06:38:50.138027+0000","last_became_active":"2023-10-09T06:38:50.137781+0000","last_became_peered":"2023-10-09T06:38:50.137781+0000","last_unstale":"2023-10-09T06:38:50.138027+0000","last_undegraded":"2023-10-09T06:38:50.138027+0000","last_fullsized":"2023-10-09T06:38:50.138027+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:34:34.991435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.127547+0000","last_change":"2023-10-09T06:38:50.127547+0000","last_active":"2023-10-09T06:38:50.127547+0000","last_peered":"2023-10-09T06:38:50.127547+0000","last_clean":"2023-10-09T06:38:50.127547+0000","last_became_active":"2023-10-09T06:38:50.127141+0000","last_became_peered":"2023-10-09T06:38:50.127141+0000","last_unstale":"2023-10-09T06:38:50.127547+0000","last_undegraded":"2023-10-09T06:38:50.127547+0000","last_fullsized":"2023-10-09T06:38:50.127547+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:26:57.066092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.136089+0000","last_change":"2023-10-09T06:38:50.136089+0000","last_active":"2023-10-09T06:38:50.136089+0000","last_peered":"2023-10-09T06:38:50.136089+0000","last_clean":"2023-10-09T06:38:50.136089+0000","last_became_active":"2023-10-09T06:38:50.135839+0000","last_became_peered":"2023-10-09T06:38:50.135839+0000","last_unstale":"2023-10-09T06:38:50.136089+0000","last_undegraded":"2023-10-09T06:38:50.136089+0000","last_fullsized":"2023-10-09T06:38:50.136089+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:17:20.306377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.144887+0000","last_change":"2023-10-09T06:38:50.144887+0000","last_active":"2023-10-09T06:38:50.144887+0000","last_peered":"2023-10-09T06:38:50.144887+0000","last_clean":"2023-10-09T06:38:50.144887+0000","last_became_active":"2023-10-09T06:38:50.144587+0000","last_became_peered":"2023-10-09T06:38:50.144587+0000","last_unstale":"2023-10-09T06:38:50.144887+0000","last_undegraded":"2023-10-09T06:38:50.144887+0000","last_fullsized":"2023-10-09T06:38:50.144887+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:37:57.121013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.143123+0000","last_change":"2023-10-09T06:38:50.143123+0000","last_active":"2023-10-09T06:38:50.143123+0000","last_peered":"2023-10-09T06:38:50.143123+0000","last_clean":"2023-10-09T06:38:50.143123+0000","last_became_active":"2023-10-09T06:38:50.142843+0000","last_became_peered":"2023-10-09T06:38:50.142843+0000","last_unstale":"2023-10-09T06:38:50.143123+0000","last_undegraded":"2023-10-09T06:38:50.143123+0000","last_fullsized":"2023-10-09T06:38:50.143123+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:26:57.066092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.129008+0000","last_change":"2023-10-09T06:38:50.129008+0000","last_active":"2023-10-09T06:38:50.129008+0000","last_peered":"2023-10-09T06:38:50.129008+0000","last_clean":"2023-10-09T06:38:50.129008+0000","last_became_active":"2023-10-09T06:38:50.128764+0000","last_became_peered":"2023-10-09T06:38:50.128764+0000","last_unstale":"2023-10-09T06:38:50.129008+0000","last_undegraded":"2023-10-09T06:38:50.129008+0000","last_fullsized":"2023-10-09T06:38:50.129008+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:20:19.103577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.142413+0000","last_change":"2023-10-09T06:38:50.142413+0000","last_active":"2023-10-09T06:38:50.142413+0000","last_peered":"2023-10-09T06:38:50.142413+0000","last_clean":"2023-10-09T06:38:50.142413+0000","last_became_active":"2023-10-09T06:38:50.141917+0000","last_became_peered":"2023-10-09T06:38:50.141917+0000","last_unstale":"2023-10-09T06:38:50.142413+0000","last_undegraded":"2023-10-09T06:38:50.142413+0000","last_fullsized":"2023-10-09T06:38:50.142413+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:52:24.913167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.137057+0000","last_change":"2023-10-09T06:38:50.137057+0000","last_active":"2023-10-09T06:38:50.137057+0000","last_peered":"2023-10-09T06:38:50.137057+0000","last_clean":"2023-10-09T06:38:50.137057+0000","last_became_active":"2023-10-09T06:38:50.136800+0000","last_became_peered":"2023-10-09T06:38:50.136800+0000","last_unstale":"2023-10-09T06:38:50.137057+0000","last_undegraded":"2023-10-09T06:38:50.137057+0000","last_fullsized":"2023-10-09T06:38:50.137057+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:06:52.423409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.149219+0000","last_change":"2023-10-09T06:38:50.149219+0000","last_active":"2023-10-09T06:38:50.149219+0000","last_peered":"2023-10-09T06:38:50.149219+0000","last_clean":"2023-10-09T06:38:50.149219+0000","last_became_active":"2023-10-09T06:38:50.148371+0000","last_became_peered":"2023-10-09T06:38:50.148371+0000","last_unstale":"2023-10-09T06:38:50.149219+0000","last_undegraded":"2023-10-09T06:38:50.149219+0000","last_fullsized":"2023-10-09T06:38:50.149219+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:26:08.728520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.127386+0000","last_change":"2023-10-09T06:38:50.127386+0000","last_active":"2023-10-09T06:38:50.127386+0000","last_peered":"2023-10-09T06:38:50.127386+0000","last_clean":"2023-10-09T06:38:50.127386+0000","last_became_active":"2023-10-09T06:38:50.126871+0000","last_became_peered":"2023-10-09T06:38:50.126871+0000","last_unstale":"2023-10-09T06:38:50.127386+0000","last_undegraded":"2023-10-09T06:38:50.127386+0000","last_fullsized":"2023-10-09T06:38:50.127386+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:55:40.976759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.131108+0000","last_change":"2023-10-09T06:38:50.131108+0000","last_active":"2023-10-09T06:38:50.131108+0000","last_peered":"2023-10-09T06:38:50.131108+0000","last_clean":"2023-10-09T06:38:50.131108+0000","last_became_active":"2023-10-09T06:38:50.130807+0000","last_became_peered":"2023-10-09T06:38:50.130807+0000","last_unstale":"2023-10-09T06:38:50.131108+0000","last_undegraded":"2023-10-09T06:38:50.131108+0000","last_fullsized":"2023-10-09T06:38:50.131108+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:26:23.603659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.143917+0000","last_change":"2023-10-09T06:38:50.143917+0000","last_active":"2023-10-09T06:38:50.143917+0000","last_peered":"2023-10-09T06:38:50.143917+0000","last_clean":"2023-10-09T06:38:50.143917+0000","last_became_active":"2023-10-09T06:38:50.143335+0000","last_became_peered":"2023-10-09T06:38:50.143335+0000","last_unstale":"2023-10-09T06:38:50.143917+0000","last_undegraded":"2023-10-09T06:38:50.143917+0000","last_fullsized":"2023-10-09T06:38:50.143917+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:45:21.421760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.129565+0000","last_change":"2023-10-09T06:38:50.129565+0000","last_active":"2023-10-09T06:38:50.129565+0000","last_peered":"2023-10-09T06:38:50.129565+0000","last_clean":"2023-10-09T06:38:50.129565+0000","last_became_active":"2023-10-09T06:38:50.129264+0000","last_became_peered":"2023-10-09T06:38:50.129264+0000","last_unstale":"2023-10-09T06:38:50.129565+0000","last_undegraded":"2023-10-09T06:38:50.129565+0000","last_fullsized":"2023-10-09T06:38:50.129565+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:45:21.421760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.132697+0000","last_change":"2023-10-09T06:38:50.132697+0000","last_active":"2023-10-09T06:38:50.132697+0000","last_peered":"2023-10-09T06:38:50.132697+0000","last_clean":"2023-10-09T06:38:50.132697+0000","last_became_active":"2023-10-09T06:38:50.132414+0000","last_became_peered":"2023-10-09T06:38:50.132414+0000","last_unstale":"2023-10-09T06:38:50.132697+0000","last_undegraded":"2023-10-09T06:38:50.132697+0000","last_fullsized":"2023-10-09T06:38:50.132697+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:23:19.098076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.143724+0000","last_change":"2023-10-09T06:38:50.143724+0000","last_active":"2023-10-09T06:38:50.143724+0000","last_peered":"2023-10-09T06:38:50.143724+0000","last_clean":"2023-10-09T06:38:50.143724+0000","last_became_active":"2023-10-09T06:38:50.143463+0000","last_became_peered":"2023-10-09T06:38:50.143463+0000","last_unstale":"2023-10-09T06:38:50.143724+0000","last_undegraded":"2023-10-09T06:38:50.143724+0000","last_fullsized":"2023-10-09T06:38:50.143724+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:01:54.560937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1f","version":"163'4","reported_seq":13,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.151562+0000","last_change":"2023-10-09T06:38:49.112506+0000","last_active":"2023-10-09T06:38:50.151562+0000","last_peered":"2023-10-09T06:38:50.151562+0000","last_clean":"2023-10-09T06:38:50.151562+0000","last_became_active":"2023-10-09T06:38:49.112176+0000","last_became_peered":"2023-10-09T06:38:49.112176+0000","last_unstale":"2023-10-09T06:38:50.151562+0000","last_undegraded":"2023-10-09T06:38:50.151562+0000","last_fullsized":"2023-10-09T06:38:50.151562+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:53:24.253235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.136317+0000","last_change":"2023-10-09T06:38:50.136317+0000","last_active":"2023-10-09T06:38:50.136317+0000","last_peered":"2023-10-09T06:38:50.136317+0000","last_clean":"2023-10-09T06:38:50.136317+0000","last_became_active":"2023-10-09T06:38:50.136038+0000","last_became_peered":"2023-10-09T06:38:50.136038+0000","last_unstale":"2023-10-09T06:38:50.136317+0000","last_undegraded":"2023-10-09T06:38:50.136317+0000","last_fullsized":"2023-10-09T06:38:50.136317+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:45:21.235591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1e","version":"163'1","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.130050+0000","last_change":"2023-10-09T06:38:49.116156+0000","last_active":"2023-10-09T06:38:50.130050+0000","last_peered":"2023-10-09T06:38:50.130050+0000","last_clean":"2023-10-09T06:38:50.130050+0000","last_became_active":"2023-10-09T06:38:49.115853+0000","last_became_peered":"2023-10-09T06:38:49.115853+0000","last_unstale":"2023-10-09T06:38:50.130050+0000","last_undegraded":"2023-10-09T06:38:50.130050+0000","last_fullsized":"2023-10-09T06:38:50.130050+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:42:03.814729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.139399+0000","last_change":"2023-10-09T06:38:50.139399+0000","last_active":"2023-10-09T06:38:50.139399+0000","last_peered":"2023-10-09T06:38:50.139399+0000","last_clean":"2023-10-09T06:38:50.139399+0000","last_became_active":"2023-10-09T06:38:50.137147+0000","last_became_peered":"2023-10-09T06:38:50.137147+0000","last_unstale":"2023-10-09T06:38:50.139399+0000","last_undegraded":"2023-10-09T06:38:50.139399+0000","last_fullsized":"2023-10-09T06:38:50.139399+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:25:54.353541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1c","version":"163'2","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.147380+0000","last_change":"2023-10-09T06:38:49.112364+0000","last_active":"2023-10-09T06:38:50.147380+0000","last_peered":"2023-10-09T06:38:50.147380+0000","last_clean":"2023-10-09T06:38:50.147380+0000","last_became_active":"2023-10-09T06:38:49.112087+0000","last_became_peered":"2023-10-09T06:38:49.112087+0000","last_unstale":"2023-10-09T06:38:50.147380+0000","last_undegraded":"2023-10-09T06:38:50.147380+0000","last_fullsized":"2023-10-09T06:38:50.147380+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:09:10.728788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.144124+0000","last_change":"2023-10-09T06:38:50.144124+0000","last_active":"2023-10-09T06:38:50.144124+0000","last_peered":"2023-10-09T06:38:50.144124+0000","last_clean":"2023-10-09T06:38:50.144124+0000","last_became_active":"2023-10-09T06:38:50.143662+0000","last_became_peered":"2023-10-09T06:38:50.143662+0000","last_unstale":"2023-10-09T06:38:50.144124+0000","last_undegraded":"2023-10-09T06:38:50.144124+0000","last_fullsized":"2023-10-09T06:38:50.144124+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:46:02.257675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1b","version":"163'1","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.121667+0000","last_change":"2023-10-09T06:38:49.109096+0000","last_active":"2023-10-09T06:38:50.121667+0000","last_peered":"2023-10-09T06:38:50.121667+0000","last_clean":"2023-10-09T06:38:50.121667+0000","last_became_active":"2023-10-09T06:38:49.108830+0000","last_became_peered":"2023-10-09T06:38:49.108830+0000","last_unstale":"2023-10-09T06:38:50.121667+0000","last_undegraded":"2023-10-09T06:38:50.121667+0000","last_fullsized":"2023-10-09T06:38:50.121667+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:26:49.787987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.127011+0000","last_change":"2023-10-09T06:38:50.127011+0000","last_active":"2023-10-09T06:38:50.127011+0000","last_peered":"2023-10-09T06:38:50.127011+0000","last_clean":"2023-10-09T06:38:50.127011+0000","last_became_active":"2023-10-09T06:38:50.126728+0000","last_became_peered":"2023-10-09T06:38:50.126728+0000","last_unstale":"2023-10-09T06:38:50.127011+0000","last_undegraded":"2023-10-09T06:38:50.127011+0000","last_fullsized":"2023-10-09T06:38:50.127011+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:19:55.720934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1a","version":"163'1","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.123484+0000","last_change":"2023-10-09T06:38:49.117890+0000","last_active":"2023-10-09T06:38:50.123484+0000","last_peered":"2023-10-09T06:38:50.123484+0000","last_clean":"2023-10-09T06:38:50.123484+0000","last_became_active":"2023-10-09T06:38:49.117627+0000","last_became_peered":"2023-10-09T06:38:49.117627+0000","last_unstale":"2023-10-09T06:38:50.123484+0000","last_undegraded":"2023-10-09T06:38:50.123484+0000","last_fullsized":"2023-10-09T06:38:50.123484+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:54:39.171366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.128204+0000","last_change":"2023-10-09T06:38:50.128204+0000","last_active":"2023-10-09T06:38:50.128204+0000","last_peered":"2023-10-09T06:38:50.128204+0000","last_clean":"2023-10-09T06:38:50.128204+0000","last_became_active":"2023-10-09T06:38:50.127968+0000","last_became_peered":"2023-10-09T06:38:50.127968+0000","last_unstale":"2023-10-09T06:38:50.128204+0000","last_undegraded":"2023-10-09T06:38:50.128204+0000","last_fullsized":"2023-10-09T06:38:50.128204+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:53:25.259901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.119760+0000","last_change":"2023-10-09T06:38:49.114469+0000","last_active":"2023-10-09T06:38:49.119760+0000","last_peered":"2023-10-09T06:38:49.119760+0000","last_clean":"2023-10-09T06:38:49.119760+0000","last_became_active":"2023-10-09T06:38:49.114197+0000","last_became_peered":"2023-10-09T06:38:49.114197+0000","last_unstale":"2023-10-09T06:38:49.119760+0000","last_undegraded":"2023-10-09T06:38:49.119760+0000","last_fullsized":"2023-10-09T06:38:49.119760+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:48:44.644939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.139595+0000","last_change":"2023-10-09T06:38:50.139595+0000","last_active":"2023-10-09T06:38:50.139595+0000","last_peered":"2023-10-09T06:38:50.139595+0000","last_clean":"2023-10-09T06:38:50.139595+0000","last_became_active":"2023-10-09T06:38:50.139255+0000","last_became_peered":"2023-10-09T06:38:50.139255+0000","last_unstale":"2023-10-09T06:38:50.139595+0000","last_undegraded":"2023-10-09T06:38:50.139595+0000","last_fullsized":"2023-10-09T06:38:50.139595+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:24:54.452573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.116097+0000","last_change":"2023-10-09T06:38:49.108941+0000","last_active":"2023-10-09T06:38:49.116097+0000","last_peered":"2023-10-09T06:38:49.116097+0000","last_clean":"2023-10-09T06:38:49.116097+0000","last_became_active":"2023-10-09T06:38:49.108640+0000","last_became_peered":"2023-10-09T06:38:49.108640+0000","last_unstale":"2023-10-09T06:38:49.116097+0000","last_undegraded":"2023-10-09T06:38:49.116097+0000","last_fullsized":"2023-10-09T06:38:49.116097+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:10:32.770444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.142718+0000","last_change":"2023-10-09T06:38:50.142718+0000","last_active":"2023-10-09T06:38:50.142718+0000","last_peered":"2023-10-09T06:38:50.142718+0000","last_clean":"2023-10-09T06:38:50.142718+0000","last_became_active":"2023-10-09T06:38:50.142441+0000","last_became_peered":"2023-10-09T06:38:50.142441+0000","last_unstale":"2023-10-09T06:38:50.142718+0000","last_undegraded":"2023-10-09T06:38:50.142718+0000","last_fullsized":"2023-10-09T06:38:50.142718+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:05:05.275319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.123196+0000","last_change":"2023-10-09T06:38:49.117192+0000","last_active":"2023-10-09T06:38:49.123196+0000","last_peered":"2023-10-09T06:38:49.123196+0000","last_clean":"2023-10-09T06:38:49.123196+0000","last_became_active":"2023-10-09T06:38:49.116891+0000","last_became_peered":"2023-10-09T06:38:49.116891+0000","last_unstale":"2023-10-09T06:38:49.123196+0000","last_undegraded":"2023-10-09T06:38:49.123196+0000","last_fullsized":"2023-10-09T06:38:49.123196+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:24:02.414649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.145141+0000","last_change":"2023-10-09T06:38:50.145141+0000","last_active":"2023-10-09T06:38:50.145141+0000","last_peered":"2023-10-09T06:38:50.145141+0000","last_clean":"2023-10-09T06:38:50.145141+0000","last_became_active":"2023-10-09T06:38:50.144885+0000","last_became_peered":"2023-10-09T06:38:50.144885+0000","last_unstale":"2023-10-09T06:38:50.145141+0000","last_undegraded":"2023-10-09T06:38:50.145141+0000","last_fullsized":"2023-10-09T06:38:50.145141+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:55:40.976759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.111103+0000","last_change":"2023-10-09T06:38:49.107265+0000","last_active":"2023-10-09T06:38:49.111103+0000","last_peered":"2023-10-09T06:38:49.111103+0000","last_clean":"2023-10-09T06:38:49.111103+0000","last_became_active":"2023-10-09T06:38:49.106980+0000","last_became_peered":"2023-10-09T06:38:49.106980+0000","last_unstale":"2023-10-09T06:38:49.111103+0000","last_undegraded":"2023-10-09T06:38:49.111103+0000","last_fullsized":"2023-10-09T06:38:49.111103+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:08:50.771431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.137365+0000","last_change":"2023-10-09T06:38:50.137365+0000","last_active":"2023-10-09T06:38:50.137365+0000","last_peered":"2023-10-09T06:38:50.137365+0000","last_clean":"2023-10-09T06:38:50.137365+0000","last_became_active":"2023-10-09T06:38:50.137109+0000","last_became_peered":"2023-10-09T06:38:50.137109+0000","last_unstale":"2023-10-09T06:38:50.137365+0000","last_undegraded":"2023-10-09T06:38:50.137365+0000","last_fullsized":"2023-10-09T06:38:50.137365+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:01:54.560937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.15","version":"163'3","reported_seq":12,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.150752+0000","last_change":"2023-10-09T06:38:49.110046+0000","last_active":"2023-10-09T06:38:50.150752+0000","last_peered":"2023-10-09T06:38:50.150752+0000","last_clean":"2023-10-09T06:38:50.150752+0000","last_became_active":"2023-10-09T06:38:49.109773+0000","last_became_peered":"2023-10-09T06:38:49.109773+0000","last_unstale":"2023-10-09T06:38:50.150752+0000","last_undegraded":"2023-10-09T06:38:50.150752+0000","last_fullsized":"2023-10-09T06:38:50.150752+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:10:49.853949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.149400+0000","last_change":"2023-10-09T06:38:50.149400+0000","last_active":"2023-10-09T06:38:50.149400+0000","last_peered":"2023-10-09T06:38:50.149400+0000","last_clean":"2023-10-09T06:38:50.149400+0000","last_became_active":"2023-10-09T06:38:50.148660+0000","last_became_peered":"2023-10-09T06:38:50.148660+0000","last_unstale":"2023-10-09T06:38:50.149400+0000","last_undegraded":"2023-10-09T06:38:50.149400+0000","last_fullsized":"2023-10-09T06:38:50.149400+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:34:34.991435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"163'6","reported_seq":15,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.145871+0000","last_change":"2023-10-09T06:38:49.106500+0000","last_active":"2023-10-09T06:38:50.145871+0000","last_peered":"2023-10-09T06:38:50.145871+0000","last_clean":"2023-10-09T06:38:50.145871+0000","last_became_active":"2023-10-09T06:38:49.106246+0000","last_became_peered":"2023-10-09T06:38:49.106246+0000","last_unstale":"2023-10-09T06:38:50.145871+0000","last_undegraded":"2023-10-09T06:38:50.145871+0000","last_fullsized":"2023-10-09T06:38:50.145871+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:33:12.834062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.125489+0000","last_change":"2023-10-09T06:38:50.125489+0000","last_active":"2023-10-09T06:38:50.125489+0000","last_peered":"2023-10-09T06:38:50.125489+0000","last_clean":"2023-10-09T06:38:50.125489+0000","last_became_active":"2023-10-09T06:38:50.125221+0000","last_became_peered":"2023-10-09T06:38:50.125221+0000","last_unstale":"2023-10-09T06:38:50.125489+0000","last_undegraded":"2023-10-09T06:38:50.125489+0000","last_fullsized":"2023-10-09T06:38:50.125489+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:40:27.206952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"163'2","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.148014+0000","last_change":"2023-10-09T06:38:49.116838+0000","last_active":"2023-10-09T06:38:50.148014+0000","last_peered":"2023-10-09T06:38:50.148014+0000","last_clean":"2023-10-09T06:38:50.148014+0000","last_became_active":"2023-10-09T06:38:49.116575+0000","last_became_peered":"2023-10-09T06:38:49.116575+0000","last_unstale":"2023-10-09T06:38:50.148014+0000","last_undegraded":"2023-10-09T06:38:50.148014+0000","last_fullsized":"2023-10-09T06:38:50.148014+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:40:51.226559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.131804+0000","last_change":"2023-10-09T06:38:50.131804+0000","last_active":"2023-10-09T06:38:50.131804+0000","last_peered":"2023-10-09T06:38:50.131804+0000","last_clean":"2023-10-09T06:38:50.131804+0000","last_became_active":"2023-10-09T06:38:50.131529+0000","last_became_peered":"2023-10-09T06:38:50.131529+0000","last_unstale":"2023-10-09T06:38:50.131804+0000","last_undegraded":"2023-10-09T06:38:50.131804+0000","last_fullsized":"2023-10-09T06:38:50.131804+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:05:05.275319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.119198+0000","last_change":"2023-10-09T06:38:49.114630+0000","last_active":"2023-10-09T06:38:49.119198+0000","last_peered":"2023-10-09T06:38:49.119198+0000","last_clean":"2023-10-09T06:38:49.119198+0000","last_became_active":"2023-10-09T06:38:49.114381+0000","last_became_peered":"2023-10-09T06:38:49.114381+0000","last_unstale":"2023-10-09T06:38:49.119198+0000","last_undegraded":"2023-10-09T06:38:49.119198+0000","last_fullsized":"2023-10-09T06:38:49.119198+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:26:56.059426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.144250+0000","last_change":"2023-10-09T06:38:50.144250+0000","last_active":"2023-10-09T06:38:50.144250+0000","last_peered":"2023-10-09T06:38:50.144250+0000","last_clean":"2023-10-09T06:38:50.144250+0000","last_became_active":"2023-10-09T06:38:50.143986+0000","last_became_peered":"2023-10-09T06:38:50.143986+0000","last_unstale":"2023-10-09T06:38:50.144250+0000","last_undegraded":"2023-10-09T06:38:50.144250+0000","last_fullsized":"2023-10-09T06:38:50.144250+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:57:38.908632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.111133+0000","last_change":"2023-10-09T06:38:49.107041+0000","last_active":"2023-10-09T06:38:49.111133+0000","last_peered":"2023-10-09T06:38:49.111133+0000","last_clean":"2023-10-09T06:38:49.111133+0000","last_became_active":"2023-10-09T06:38:49.106781+0000","last_became_peered":"2023-10-09T06:38:49.106781+0000","last_unstale":"2023-10-09T06:38:49.111133+0000","last_undegraded":"2023-10-09T06:38:49.111133+0000","last_fullsized":"2023-10-09T06:38:49.111133+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:52:11.478202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.136830+0000","last_change":"2023-10-09T06:38:50.136830+0000","last_active":"2023-10-09T06:38:50.136830+0000","last_peered":"2023-10-09T06:38:50.136830+0000","last_clean":"2023-10-09T06:38:50.136830+0000","last_became_active":"2023-10-09T06:38:50.136592+0000","last_became_peered":"2023-10-09T06:38:50.136592+0000","last_unstale":"2023-10-09T06:38:50.136830+0000","last_undegraded":"2023-10-09T06:38:50.136830+0000","last_fullsized":"2023-10-09T06:38:50.136830+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:55:47.819136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.10","version":"163'8","reported_seq":17,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.151452+0000","last_change":"2023-10-09T06:38:49.108752+0000","last_active":"2023-10-09T06:38:50.151452+0000","last_peered":"2023-10-09T06:38:50.151452+0000","last_clean":"2023-10-09T06:38:50.151452+0000","last_became_active":"2023-10-09T06:38:49.108496+0000","last_became_peered":"2023-10-09T06:38:49.108496+0000","last_unstale":"2023-10-09T06:38:50.151452+0000","last_undegraded":"2023-10-09T06:38:50.151452+0000","last_fullsized":"2023-10-09T06:38:50.151452+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:23:03.653758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.131441+0000","last_change":"2023-10-09T06:38:50.131441+0000","last_active":"2023-10-09T06:38:50.131441+0000","last_peered":"2023-10-09T06:38:50.131441+0000","last_clean":"2023-10-09T06:38:50.131441+0000","last_became_active":"2023-10-09T06:38:50.131185+0000","last_became_peered":"2023-10-09T06:38:50.131185+0000","last_unstale":"2023-10-09T06:38:50.131441+0000","last_undegraded":"2023-10-09T06:38:50.131441+0000","last_fullsized":"2023-10-09T06:38:50.131441+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:37:19.939607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.f","version":"163'1","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.126131+0000","last_change":"2023-10-09T06:38:49.118475+0000","last_active":"2023-10-09T06:38:50.126131+0000","last_peered":"2023-10-09T06:38:50.126131+0000","last_clean":"2023-10-09T06:38:50.126131+0000","last_became_active":"2023-10-09T06:38:49.118204+0000","last_became_peered":"2023-10-09T06:38:49.118204+0000","last_unstale":"2023-10-09T06:38:50.126131+0000","last_undegraded":"2023-10-09T06:38:50.126131+0000","last_fullsized":"2023-10-09T06:38:50.126131+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:05:00.395166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.129036+0000","last_change":"2023-10-09T06:38:50.129036+0000","last_active":"2023-10-09T06:38:50.129036+0000","last_peered":"2023-10-09T06:38:50.129036+0000","last_clean":"2023-10-09T06:38:50.129036+0000","last_became_active":"2023-10-09T06:38:50.128847+0000","last_became_peered":"2023-10-09T06:38:50.128847+0000","last_unstale":"2023-10-09T06:38:50.129036+0000","last_undegraded":"2023-10-09T06:38:50.129036+0000","last_fullsized":"2023-10-09T06:38:50.129036+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:45:21.421760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"163'2","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.148853+0000","last_change":"2023-10-09T06:38:49.113368+0000","last_active":"2023-10-09T06:38:50.148853+0000","last_peered":"2023-10-09T06:38:50.148853+0000","last_clean":"2023-10-09T06:38:50.148853+0000","last_became_active":"2023-10-09T06:38:49.113114+0000","last_became_peered":"2023-10-09T06:38:49.113114+0000","last_unstale":"2023-10-09T06:38:50.148853+0000","last_undegraded":"2023-10-09T06:38:50.148853+0000","last_fullsized":"2023-10-09T06:38:50.148853+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:36:50.152112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.129641+0000","last_change":"2023-10-09T06:38:50.129641+0000","last_active":"2023-10-09T06:38:50.129641+0000","last_peered":"2023-10-09T06:38:50.129641+0000","last_clean":"2023-10-09T06:38:50.129641+0000","last_became_active":"2023-10-09T06:38:50.129011+0000","last_became_peered":"2023-10-09T06:38:50.129011+0000","last_unstale":"2023-10-09T06:38:50.129641+0000","last_undegraded":"2023-10-09T06:38:50.129641+0000","last_fullsized":"2023-10-09T06:38:50.129641+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:24:54.452573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.d","version":"163'1","reported_seq":10,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.135881+0000","last_change":"2023-10-09T06:38:49.113614+0000","last_active":"2023-10-09T06:38:50.135881+0000","last_peered":"2023-10-09T06:38:50.135881+0000","last_clean":"2023-10-09T06:38:50.135881+0000","last_became_active":"2023-10-09T06:38:49.113308+0000","last_became_peered":"2023-10-09T06:38:49.113308+0000","last_unstale":"2023-10-09T06:38:50.135881+0000","last_undegraded":"2023-10-09T06:38:50.135881+0000","last_fullsized":"2023-10-09T06:38:50.135881+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:16:45.760322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.128577+0000","last_change":"2023-10-09T06:38:50.128577+0000","last_active":"2023-10-09T06:38:50.128577+0000","last_peered":"2023-10-09T06:38:50.128577+0000","last_clean":"2023-10-09T06:38:50.128577+0000","last_became_active":"2023-10-09T06:38:50.128280+0000","last_became_peered":"2023-10-09T06:38:50.128280+0000","last_unstale":"2023-10-09T06:38:50.128577+0000","last_undegraded":"2023-10-09T06:38:50.128577+0000","last_fullsized":"2023-10-09T06:38:50.128577+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:59:41.148074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.116078+0000","last_change":"2023-10-09T06:38:49.108672+0000","last_active":"2023-10-09T06:38:49.116078+0000","last_peered":"2023-10-09T06:38:49.116078+0000","last_clean":"2023-10-09T06:38:49.116078+0000","last_became_active":"2023-10-09T06:38:49.108244+0000","last_became_peered":"2023-10-09T06:38:49.108244+0000","last_unstale":"2023-10-09T06:38:49.116078+0000","last_undegraded":"2023-10-09T06:38:49.116078+0000","last_fullsized":"2023-10-09T06:38:49.116078+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:03:02.251589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.130147+0000","last_change":"2023-10-09T06:38:50.130147+0000","last_active":"2023-10-09T06:38:50.130147+0000","last_peered":"2023-10-09T06:38:50.130147+0000","last_clean":"2023-10-09T06:38:50.130147+0000","last_became_active":"2023-10-09T06:38:50.129885+0000","last_became_peered":"2023-10-09T06:38:50.129885+0000","last_unstale":"2023-10-09T06:38:50.130147+0000","last_undegraded":"2023-10-09T06:38:50.130147+0000","last_fullsized":"2023-10-09T06:38:50.130147+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:37:57.121013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.b","version":"163'2","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.153319+0000","last_change":"2023-10-09T06:38:49.117518+0000","last_active":"2023-10-09T06:38:50.153319+0000","last_peered":"2023-10-09T06:38:50.153319+0000","last_clean":"2023-10-09T06:38:50.153319+0000","last_became_active":"2023-10-09T06:38:49.117227+0000","last_became_peered":"2023-10-09T06:38:49.117227+0000","last_unstale":"2023-10-09T06:38:50.153319+0000","last_undegraded":"2023-10-09T06:38:50.153319+0000","last_fullsized":"2023-10-09T06:38:50.153319+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:10:19.546962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.136974+0000","last_change":"2023-10-09T06:38:50.136974+0000","last_active":"2023-10-09T06:38:50.136974+0000","last_peered":"2023-10-09T06:38:50.136974+0000","last_clean":"2023-10-09T06:38:50.136974+0000","last_became_active":"2023-10-09T06:38:50.136647+0000","last_became_peered":"2023-10-09T06:38:50.136647+0000","last_unstale":"2023-10-09T06:38:50.136974+0000","last_undegraded":"2023-10-09T06:38:50.136974+0000","last_fullsized":"2023-10-09T06:38:50.136974+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:49:03.761584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.119795+0000","last_change":"2023-10-09T06:38:49.114686+0000","last_active":"2023-10-09T06:38:49.119795+0000","last_peered":"2023-10-09T06:38:49.119795+0000","last_clean":"2023-10-09T06:38:49.119795+0000","last_became_active":"2023-10-09T06:38:49.114457+0000","last_became_peered":"2023-10-09T06:38:49.114457+0000","last_unstale":"2023-10-09T06:38:49.119795+0000","last_undegraded":"2023-10-09T06:38:49.119795+0000","last_fullsized":"2023-10-09T06:38:49.119795+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:09:10.728788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.130937+0000","last_change":"2023-10-09T06:38:50.130937+0000","last_active":"2023-10-09T06:38:50.130937+0000","last_peered":"2023-10-09T06:38:50.130937+0000","last_clean":"2023-10-09T06:38:50.130937+0000","last_became_active":"2023-10-09T06:38:50.130663+0000","last_became_peered":"2023-10-09T06:38:50.130663+0000","last_unstale":"2023-10-09T06:38:50.130937+0000","last_undegraded":"2023-10-09T06:38:50.130937+0000","last_fullsized":"2023-10-09T06:38:50.130937+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:58:32.745191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.123565+0000","last_change":"2023-10-09T06:38:49.117011+0000","last_active":"2023-10-09T06:38:49.123565+0000","last_peered":"2023-10-09T06:38:49.123565+0000","last_clean":"2023-10-09T06:38:49.123565+0000","last_became_active":"2023-10-09T06:38:49.116664+0000","last_became_peered":"2023-10-09T06:38:49.116664+0000","last_unstale":"2023-10-09T06:38:49.123565+0000","last_undegraded":"2023-10-09T06:38:49.123565+0000","last_fullsized":"2023-10-09T06:38:49.123565+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:42:51.752367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.139241+0000","last_change":"2023-10-09T06:38:50.139241+0000","last_active":"2023-10-09T06:38:50.139241+0000","last_peered":"2023-10-09T06:38:50.139241+0000","last_clean":"2023-10-09T06:38:50.139241+0000","last_became_active":"2023-10-09T06:38:50.138974+0000","last_became_peered":"2023-10-09T06:38:50.138974+0000","last_unstale":"2023-10-09T06:38:50.139241+0000","last_undegraded":"2023-10-09T06:38:50.139241+0000","last_fullsized":"2023-10-09T06:38:50.139241+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:40:27.206952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.127085+0000","last_change":"2023-10-09T06:38:49.121745+0000","last_active":"2023-10-09T06:38:49.127085+0000","last_peered":"2023-10-09T06:38:49.127085+0000","last_clean":"2023-10-09T06:38:49.127085+0000","last_became_active":"2023-10-09T06:38:49.121512+0000","last_became_peered":"2023-10-09T06:38:49.121512+0000","last_unstale":"2023-10-09T06:38:49.127085+0000","last_undegraded":"2023-10-09T06:38:49.127085+0000","last_fullsized":"2023-10-09T06:38:49.127085+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:35:24.333777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.144785+0000","last_change":"2023-10-09T06:38:50.144785+0000","last_active":"2023-10-09T06:38:50.144785+0000","last_peered":"2023-10-09T06:38:50.144785+0000","last_clean":"2023-10-09T06:38:50.144785+0000","last_became_active":"2023-10-09T06:38:50.144524+0000","last_became_peered":"2023-10-09T06:38:50.144524+0000","last_unstale":"2023-10-09T06:38:50.144785+0000","last_undegraded":"2023-10-09T06:38:50.144785+0000","last_fullsized":"2023-10-09T06:38:50.144785+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:46:02.257675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.7","version":"163'2","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.148968+0000","last_change":"2023-10-09T06:38:49.120567+0000","last_active":"2023-10-09T06:38:50.148968+0000","last_peered":"2023-10-09T06:38:50.148968+0000","last_clean":"2023-10-09T06:38:50.148968+0000","last_became_active":"2023-10-09T06:38:49.120358+0000","last_became_peered":"2023-10-09T06:38:49.120358+0000","last_unstale":"2023-10-09T06:38:50.148968+0000","last_undegraded":"2023-10-09T06:38:50.148968+0000","last_fullsized":"2023-10-09T06:38:50.148968+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:16:14.105619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.140455+0000","last_change":"2023-10-09T06:38:50.140455+0000","last_active":"2023-10-09T06:38:50.140455+0000","last_peered":"2023-10-09T06:38:50.140455+0000","last_clean":"2023-10-09T06:38:50.140455+0000","last_became_active":"2023-10-09T06:38:50.140189+0000","last_became_peered":"2023-10-09T06:38:50.140189+0000","last_unstale":"2023-10-09T06:38:50.140455+0000","last_undegraded":"2023-10-09T06:38:50.140455+0000","last_fullsized":"2023-10-09T06:38:50.140455+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:31:26.597493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.119558+0000","last_change":"2023-10-09T06:38:49.113566+0000","last_active":"2023-10-09T06:38:49.119558+0000","last_peered":"2023-10-09T06:38:49.119558+0000","last_clean":"2023-10-09T06:38:49.119558+0000","last_became_active":"2023-10-09T06:38:49.113318+0000","last_became_peered":"2023-10-09T06:38:49.113318+0000","last_unstale":"2023-10-09T06:38:49.119558+0000","last_undegraded":"2023-10-09T06:38:49.119558+0000","last_fullsized":"2023-10-09T06:38:49.119558+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:20:46.683927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.137309+0000","last_change":"2023-10-09T06:38:50.137309+0000","last_active":"2023-10-09T06:38:50.137309+0000","last_peered":"2023-10-09T06:38:50.137309+0000","last_clean":"2023-10-09T06:38:50.137309+0000","last_became_active":"2023-10-09T06:38:50.137058+0000","last_became_peered":"2023-10-09T06:38:50.137058+0000","last_unstale":"2023-10-09T06:38:50.137309+0000","last_undegraded":"2023-10-09T06:38:50.137309+0000","last_fullsized":"2023-10-09T06:38:50.137309+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:13:21.243618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.122345+0000","last_change":"2023-10-09T06:38:49.115125+0000","last_active":"2023-10-09T06:38:49.122345+0000","last_peered":"2023-10-09T06:38:49.122345+0000","last_clean":"2023-10-09T06:38:49.122345+0000","last_became_active":"2023-10-09T06:38:49.114810+0000","last_became_peered":"2023-10-09T06:38:49.114810+0000","last_unstale":"2023-10-09T06:38:49.122345+0000","last_undegraded":"2023-10-09T06:38:49.122345+0000","last_fullsized":"2023-10-09T06:38:49.122345+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:03:35.678065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.125809+0000","last_change":"2023-10-09T06:38:50.125809+0000","last_active":"2023-10-09T06:38:50.125809+0000","last_peered":"2023-10-09T06:38:50.125809+0000","last_clean":"2023-10-09T06:38:50.125809+0000","last_became_active":"2023-10-09T06:38:50.125550+0000","last_became_peered":"2023-10-09T06:38:50.125550+0000","last_unstale":"2023-10-09T06:38:50.125809+0000","last_undegraded":"2023-10-09T06:38:50.125809+0000","last_fullsized":"2023-10-09T06:38:50.125809+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:53:25.259901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.123454+0000","last_change":"2023-10-09T06:38:49.117628+0000","last_active":"2023-10-09T06:38:49.123454+0000","last_peered":"2023-10-09T06:38:49.123454+0000","last_clean":"2023-10-09T06:38:49.123454+0000","last_became_active":"2023-10-09T06:38:49.117348+0000","last_became_peered":"2023-10-09T06:38:49.117348+0000","last_unstale":"2023-10-09T06:38:49.123454+0000","last_undegraded":"2023-10-09T06:38:49.123454+0000","last_fullsized":"2023-10-09T06:38:49.123454+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:16:25.959103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.139266+0000","last_change":"2023-10-09T06:38:50.139266+0000","last_active":"2023-10-09T06:38:50.139266+0000","last_peered":"2023-10-09T06:38:50.139266+0000","last_clean":"2023-10-09T06:38:50.139266+0000","last_became_active":"2023-10-09T06:38:50.138981+0000","last_became_peered":"2023-10-09T06:38:50.138981+0000","last_unstale":"2023-10-09T06:38:50.139266+0000","last_undegraded":"2023-10-09T06:38:50.139266+0000","last_fullsized":"2023-10-09T06:38:50.139266+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:25:54.353541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.123612+0000","last_change":"2023-10-09T06:38:49.117365+0000","last_active":"2023-10-09T06:38:49.123612+0000","last_peered":"2023-10-09T06:38:49.123612+0000","last_clean":"2023-10-09T06:38:49.123612+0000","last_became_active":"2023-10-09T06:38:49.117111+0000","last_became_peered":"2023-10-09T06:38:49.117111+0000","last_unstale":"2023-10-09T06:38:49.123612+0000","last_undegraded":"2023-10-09T06:38:49.123612+0000","last_fullsized":"2023-10-09T06:38:49.123612+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:01:00.482030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.126011+0000","last_change":"2023-10-09T06:38:50.126011+0000","last_active":"2023-10-09T06:38:50.126011+0000","last_peered":"2023-10-09T06:38:50.126011+0000","last_clean":"2023-10-09T06:38:50.126011+0000","last_became_active":"2023-10-09T06:38:50.125620+0000","last_became_peered":"2023-10-09T06:38:50.125620+0000","last_unstale":"2023-10-09T06:38:50.126011+0000","last_undegraded":"2023-10-09T06:38:50.126011+0000","last_fullsized":"2023-10-09T06:38:50.126011+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:25:54.353541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.111132+0000","last_change":"2023-10-09T06:38:49.107253+0000","last_active":"2023-10-09T06:38:49.111132+0000","last_peered":"2023-10-09T06:38:49.111132+0000","last_clean":"2023-10-09T06:38:49.111132+0000","last_became_active":"2023-10-09T06:38:49.106952+0000","last_became_peered":"2023-10-09T06:38:49.106952+0000","last_unstale":"2023-10-09T06:38:49.111132+0000","last_undegraded":"2023-10-09T06:38:49.111132+0000","last_fullsized":"2023-10-09T06:38:49.111132+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:52:23.906501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.140824+0000","last_change":"2023-10-09T06:38:50.140824+0000","last_active":"2023-10-09T06:38:50.140824+0000","last_peered":"2023-10-09T06:38:50.140824+0000","last_clean":"2023-10-09T06:38:50.140824+0000","last_became_active":"2023-10-09T06:38:50.140557+0000","last_became_peered":"2023-10-09T06:38:50.140557+0000","last_unstale":"2023-10-09T06:38:50.140824+0000","last_undegraded":"2023-10-09T06:38:50.140824+0000","last_fullsized":"2023-10-09T06:38:50.140824+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:17:07.094300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.122444+0000","last_change":"2023-10-09T06:38:49.119885+0000","last_active":"2023-10-09T06:38:49.122444+0000","last_peered":"2023-10-09T06:38:49.122444+0000","last_clean":"2023-10-09T06:38:49.122444+0000","last_became_active":"2023-10-09T06:38:49.119692+0000","last_became_peered":"2023-10-09T06:38:49.119692+0000","last_unstale":"2023-10-09T06:38:49.122444+0000","last_undegraded":"2023-10-09T06:38:49.122444+0000","last_fullsized":"2023-10-09T06:38:49.122444+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:01:05.239585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.144748+0000","last_change":"2023-10-09T06:38:50.144748+0000","last_active":"2023-10-09T06:38:50.144748+0000","last_peered":"2023-10-09T06:38:50.144748+0000","last_clean":"2023-10-09T06:38:50.144748+0000","last_became_active":"2023-10-09T06:38:50.144387+0000","last_became_peered":"2023-10-09T06:38:50.144387+0000","last_unstale":"2023-10-09T06:38:50.144748+0000","last_undegraded":"2023-10-09T06:38:50.144748+0000","last_fullsized":"2023-10-09T06:38:50.144748+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:17:20.306377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.123266+0000","last_change":"2023-10-09T06:38:49.116981+0000","last_active":"2023-10-09T06:38:49.123266+0000","last_peered":"2023-10-09T06:38:49.123266+0000","last_clean":"2023-10-09T06:38:49.123266+0000","last_became_active":"2023-10-09T06:38:49.116703+0000","last_became_peered":"2023-10-09T06:38:49.116703+0000","last_unstale":"2023-10-09T06:38:49.123266+0000","last_undegraded":"2023-10-09T06:38:49.123266+0000","last_fullsized":"2023-10-09T06:38:49.123266+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:45:03.562181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":8,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.141195+0000","last_change":"2023-10-09T06:38:50.141195+0000","last_active":"2023-10-09T06:38:50.141195+0000","last_peered":"2023-10-09T06:38:50.141195+0000","last_clean":"2023-10-09T06:38:50.141195+0000","last_became_active":"2023-10-09T06:38:50.137321+0000","last_became_peered":"2023-10-09T06:38:50.137321+0000","last_unstale":"2023-10-09T06:38:50.141195+0000","last_undegraded":"2023-10-09T06:38:50.141195+0000","last_fullsized":"2023-10-09T06:38:50.141195+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:49.090672+0000","last_clean_scrub_stamp":"2023-10-09T06:38:49.090672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:36:51.158778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1d","version":"163'2","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-09T06:38:50.147066+0000","last_change":"2023-10-09T06:38:49.120114+0000","last_active":"2023-10-09T06:38:50.147066+0000","last_peered":"2023-10-09T06:38:50.147066+0000","last_clean":"2023-10-09T06:38:50.147066+0000","last_became_active":"2023-10-09T06:38:49.119849+0000","last_became_peered":"2023-10-09T06:38:49.119849+0000","last_unstale":"2023-10-09T06:38:50.147066+0000","last_undegraded":"2023-10-09T06:38:50.147066+0000","last_fullsized":"2023-10-09T06:38:50.147066+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:38:48.084006+0000","last_clean_scrub_stamp":"2023-10-09T06:38:48.084006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:17:24.554348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":282,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-09T06:38:49.103482+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:38:49.103482+0000","last_peered":"2023-10-09T06:38:49.103482+0000","last_clean":"2023-10-09T06:38:49.103482+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:38:49.103482+0000","last_undegraded":"2023-10-09T06:38:49.103482+0000","last_fullsized":"2023-10-09T06:38:49.103482+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574952,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303972,"kb_used_data":2588,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067868,"statfs":{"total":96636764160,"available":96325496832,"internally_reserved":0,"allocated":2650112,"data_stored":1506317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:38:42 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51300000000000001,"15min":0.50800000000000001},"min":{"1min":0.36499999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.629,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":3.778},{"interface":"front","average":{"1min":0.498,"5min":0.502,"15min":0.51000000000000001},"min":{"1min":0.33400000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.64700000000000002,"5min":0.70199999999999996,"15min":0.753},"last":1.768}]},{"osd":1,"last update":"Mon Oct 9 06:38:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.50700000000000001,"15min":0.51800000000000002},"min":{"1min":0.36899999999999999,"5min":0.33600000000000002,"15min":0.27100000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":1.2030000000000001},"last":1.5840000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53000000000000003,"15min":0.52600000000000002},"min":{"1min":0.39900000000000002,"5min":0.32900000000000001,"15min":0.29699999999999999},"max":{"1min":0.67000000000000004,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":1.806}]},{"osd":2,"last update":"Mon Oct 9 06:38:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53100000000000003,"15min":0.51700000000000002},"min":{"1min":0.37,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.746,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":3.7210000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53200000000000003,"15min":0.54300000000000004},"min":{"1min":0.36799999999999999,"5min":0.36399999999999999,"15min":0.315},"max":{"1min":0.68100000000000005,"5min":0.80700000000000005,"15min":0.83099999999999996},"last":1.51}]},{"osd":3,"last update":"Mon Oct 9 06:38:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54200000000000004,"15min":0.55000000000000004},"min":{"1min":0.434,"5min":0.35899999999999999,"15min":0.33500000000000002},"max":{"1min":0.76200000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":3.8090000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55100000000000005,"15min":0.58599999999999997},"min":{"1min":0.38400000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69899999999999995,"5min":0.82099999999999995,"15min":1.1479999999999999},"last":1.6870000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:38:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.49299999999999999,"15min":0.51600000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.24399999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.93999999999999995},"last":1.9019999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58099999999999996,"15min":0.60399999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.30299999999999999},"max":{"1min":0.78600000000000003,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":4.8239999999999998}]},{"osd":5,"last update":"Mon Oct 9 06:38:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53700000000000003,"15min":0.56699999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.307},"max":{"1min":0.751,"5min":0.85799999999999998,"15min":0.91100000000000003},"last":1.7230000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60799999999999998,"15min":0.61799999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34200000000000003},"max":{"1min":0.79400000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":3.6909999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:38:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56999999999999995,"15min":0.59099999999999997},"min":{"1min":0.318,"5min":0.318,"15min":0.28699999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.88600000000000001},"last":3.5529999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58199999999999996,"15min":0.61099999999999999},"min":{"1min":0.34399999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.81999999999999995,"5min":0.82499999999999996,"15min":0.84599999999999997},"last":4.7789999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574951,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303392,"kb_used_data":2008,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068448,"statfs":{"total":96636764160,"available":96326090752,"internally_reserved":0,"allocated":2056192,"data_stored":915949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.35899999999999999,"5min":0.35399999999999998,"15min":0.30399999999999999},"max":{"1min":0.70699999999999996,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.51100000000000001,"15min":0.51200000000000001},"min":{"1min":0.36299999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69699999999999995,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.61199999999999999}]},{"osd":1,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50700000000000001,"15min":0.51200000000000001},"min":{"1min":0.43099999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.72699999999999998,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53500000000000003,"15min":0.53300000000000003},"min":{"1min":0.437,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.69099999999999995,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.48099999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.34300000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67800000000000005,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.53000000000000003},"min":{"1min":0.39300000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.499}]},{"osd":3,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.39800000000000002,"5min":0.38900000000000001,"15min":0.33500000000000002},"max":{"1min":0.79000000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54400000000000004,"15min":0.67100000000000004},"min":{"1min":0.40300000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":4.7510000000000003},"last":0.51100000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.46000000000000002,"15min":0.47499999999999998},"min":{"1min":0.28999999999999998,"5min":0.28199999999999997,"15min":0.27900000000000003},"max":{"1min":0.63700000000000001,"5min":0.74299999999999999,"15min":0.76400000000000001},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56699999999999995,"15min":0.58299999999999996},"min":{"1min":0.51500000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.75,"5min":0.80800000000000005,"15min":0.83699999999999997},"last":0.52300000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.52100000000000002,"15min":0.53100000000000003},"min":{"1min":0.37,"5min":0.246,"15min":0.246},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.375},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57199999999999995,"15min":0.59299999999999997},"min":{"1min":0.47299999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70899999999999996,"5min":0.78000000000000003,"15min":0.86599999999999999},"last":0.53600000000000003}]},{"osd":7,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55600000000000005,"15min":0.57999999999999996},"min":{"1min":0.33400000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82399999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.438,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80500000000000005,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.59599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303392,"kb_used_data":2008,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068448,"statfs":{"total":96636764160,"available":96326090752,"internally_reserved":0,"allocated":2056192,"data_stored":915949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.317,"15min":0.311},"min":{"1min":0.22500000000000001,"5min":0.192,"15min":0.192},"max":{"1min":0.47699999999999998,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.318},{"interface":"front","average":{"1min":0.37,"5min":0.35399999999999998,"15min":0.35899999999999999},"min":{"1min":0.27700000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.376}]},{"osd":2,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34699999999999998,"15min":0.34999999999999998},"min":{"1min":0.23899999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.51600000000000001,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38400000000000001,"15min":0.38500000000000001},"min":{"1min":0.25800000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.60399999999999998,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.40999999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.36599999999999999,"15min":0.36299999999999999},"min":{"1min":0.24299999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.503,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.374,"5min":0.35099999999999998,"15min":0.51600000000000001},"min":{"1min":0.255,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":5.8529999999999998},"last":0.39600000000000002}]},{"osd":4,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.36499999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59799999999999998,"15min":0.58799999999999997},"min":{"1min":0.46700000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.64300000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57899999999999996,"15min":0.72299999999999998},"min":{"1min":0.40699999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":5.9459999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.42499999999999999,"5min":0.38900000000000001,"15min":0.34300000000000003},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.62}]},{"osd":6,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54800000000000004,"15min":0.69399999999999995},"min":{"1min":0.38700000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":5.9160000000000004},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.40999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88600000000000001,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.54900000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.58399999999999996},"min":{"1min":0.42899999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.97099999999999997},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56899999999999995,"15min":0.72899999999999998},"min":{"1min":0.41799999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":5.9729999999999999},"last":0.53900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574953,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303992,"kb_used_data":2608,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067848,"statfs":{"total":96636764160,"available":96325476352,"internally_reserved":0,"allocated":2670592,"data_stored":1513724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.316,"15min":0.318},"min":{"1min":0.255,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.33000000000000002,"15min":0.34399999999999997},"min":{"1min":0.24199999999999999,"5min":0.222,"15min":0.214},"max":{"1min":0.48699999999999999,"5min":0.53600000000000003,"15min":0.94499999999999995},"last":0.24199999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.36299999999999999,"15min":0.36399999999999999},"min":{"1min":0.223,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.53200000000000003,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.223},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.379,"15min":0.38800000000000001},"min":{"1min":0.29099999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.52900000000000003,"5min":0.69399999999999995,"15min":0.91700000000000004},"last":0.29099999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.33300000000000002,"15min":0.38},"min":{"1min":0.26000000000000001,"5min":0.23599999999999999,"15min":0.23300000000000001},"max":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":1.554},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.37,"15min":0.39600000000000002},"min":{"1min":0.30599999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54500000000000004,"5min":0.58099999999999996,"15min":0.79000000000000004},"last":0.30599999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.75900000000000001,"5min":0.80200000000000005,"15min":1.212},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56200000000000006,"15min":0.70499999999999996},"min":{"1min":0.39000000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77800000000000002,"5min":0.89900000000000002,"15min":5.484},"last":0.39000000000000001}]},{"osd":5,"last update":"Mon Oct 9 06:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56100000000000005,"15min":0.70899999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80100000000000005,"5min":0.81399999999999995,"15min":5.5449999999999999},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56899999999999995,"15min":0.57499999999999996},"min":{"1min":0.41399999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.80100000000000005,"5min":0.84499999999999997,"15min":1.113},"last":0.41399999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59399999999999997,"15min":0.60499999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.78700000000000003,"5min":0.85599999999999998,"15min":1.0529999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.42899999999999999,"5min":0.42799999999999999,"15min":0.41599999999999998},"max":{"1min":0.78700000000000003,"5min":0.878,"15min":1.169},"last":0.42899999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58199999999999996,"15min":0.59199999999999997},"min":{"1min":0.40200000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.80300000000000005,"5min":0.85499999999999998,"15min":1.2390000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56599999999999995,"15min":0.73099999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77000000000000002,"5min":0.86299999999999999,"15min":5.5800000000000001},"last":0.35899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303424,"kb_used_data":2040,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068416,"statfs":{"total":96636764160,"available":96326057984,"internally_reserved":0,"allocated":2088960,"data_stored":924304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.316,"15min":0.35299999999999998},"min":{"1min":0.23200000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":1.4199999999999999},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.55900000000000005},"min":{"1min":0.27500000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.51600000000000001,"5min":0.53800000000000003,"15min":7.2699999999999996},"last":0.23300000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.317,"15min":0.34399999999999997},"min":{"1min":0.23999999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.65400000000000003},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.37,"15min":0.61299999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":7.327},"last":0.313}]},{"osd":3,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.36799999999999999,"15min":0.39100000000000001},"min":{"1min":0.29399999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.68600000000000005},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.36799999999999999,"15min":0.39200000000000002},"min":{"1min":0.27900000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.72699999999999998},"last":0.307}]},{"osd":4,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54500000000000004,"15min":0.54400000000000004},"min":{"1min":0.40799999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.78400000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56200000000000006,"15min":0.78600000000000003},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.82699999999999996,"5min":0.84699999999999998,"15min":7.2389999999999999},"last":0.51000000000000001}]},{"osd":5,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.58099999999999996,"15min":0.91500000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82699999999999996,"5min":0.83799999999999997,"15min":10.890000000000001},"last":0.442},{"interface":"front","average":{"1min":0.627,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":0.84299999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.52000000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56399999999999995,"15min":0.79100000000000004},"min":{"1min":0.42999999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.748,"5min":0.77300000000000002,"15min":7.2999999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.45800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.53200000000000003}]},{"osd":7,"last update":"Mon Oct 9 06:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59299999999999997,"15min":0.82199999999999995},"min":{"1min":0.47499999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":7.1879999999999997},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60399999999999998,"15min":0.624},"min":{"1min":0.49399999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.90500000000000003},"last":0.55600000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574953,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303392,"kb_used_data":2008,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068448,"statfs":{"total":96636764160,"available":96326090752,"internally_reserved":0,"allocated":2056192,"data_stored":915949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.32800000000000001,"15min":0.33400000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.20599999999999999},"max":{"1min":0.51300000000000001,"5min":0.61699999999999999,"15min":5.8700000000000001},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34699999999999998,"15min":0.36599999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":5.6769999999999996},"last":0.36199999999999999}]},{"osd":1,"last update":"Mon Oct 9 06:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.35099999999999998,"15min":0.35899999999999999},"min":{"1min":0.21299999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.44,"5min":0.59499999999999997,"15min":4.0469999999999997},"last":0.372},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.377,"15min":0.38900000000000001},"min":{"1min":0.30099999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.624,"5min":0.68600000000000005,"15min":4.1509999999999998},"last":0.34999999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.35199999999999998,"15min":0.372},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.222},"max":{"1min":0.51900000000000002,"5min":0.56799999999999995,"15min":3.1139999999999999},"last":0.311},{"interface":"front","average":{"1min":0.37,"5min":0.374,"15min":0.38500000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.54500000000000004,"5min":0.66900000000000004,"15min":3.3180000000000001},"last":0.41299999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55400000000000005,"15min":0.56999999999999995},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.68799999999999994,"5min":0.81699999999999995,"15min":3.0899999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.749,"5min":0.79200000000000004,"15min":3.0710000000000002},"last":0.496}]},{"osd":5,"last update":"Mon Oct 9 06:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56799999999999995,"15min":0.58799999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.32700000000000001},"max":{"1min":0.745,"5min":0.877,"15min":6.2300000000000004},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58799999999999997,"15min":0.59499999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.755,"5min":0.83099999999999996,"15min":3.0219999999999998},"last":0.42499999999999999}]},{"osd":6,"last update":"Mon Oct 9 06:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80800000000000005,"5min":0.89200000000000002,"15min":2.6850000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63800000000000001,"15min":0.65100000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.76800000000000002,"5min":0.94299999999999995,"15min":6.5970000000000004},"last":0.46200000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.41699999999999998,"5min":0.40200000000000002,"15min":0.38200000000000001},"max":{"1min":0.81999999999999995,"5min":0.88500000000000001,"15min":4.2629999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58899999999999997,"15min":0.61199999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.79100000000000004,"5min":0.82899999999999996,"15min":7.3650000000000002},"last":0.63300000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303392,"kb_used_data":2008,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068448,"statfs":{"total":96636764160,"available":96326090752,"internally_reserved":0,"allocated":2056192,"data_stored":915949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.51600000000000001,"15min":0.55100000000000005},"min":{"1min":0.373,"5min":0.28100000000000003,"15min":0.27000000000000002},"max":{"1min":0.82599999999999996,"5min":0.88900000000000001,"15min":1.083},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52500000000000002,"15min":0.52000000000000002},"min":{"1min":0.36499999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.76100000000000001,"5min":0.97799999999999998,"15min":1.054},"last":0.65600000000000003}]},{"osd":1,"last update":"Mon Oct 9 06:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53500000000000003,"15min":0.54700000000000004},"min":{"1min":0.28399999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.71099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.499},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.501,"15min":0.496},"min":{"1min":0.34000000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.75700000000000001,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.46100000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52000000000000002,"15min":0.53800000000000003},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":1.0249999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54600000000000004,"15min":0.58599999999999997},"min":{"1min":0.38100000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.66800000000000004,"5min":0.90500000000000003,"15min":0.93799999999999994},"last":0.60799999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52100000000000002,"15min":0.54800000000000004},"min":{"1min":0.33300000000000002,"5min":0.32200000000000001,"15min":0.29199999999999998},"max":{"1min":0.71599999999999997,"5min":0.86899999999999999,"15min":0.98499999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56699999999999995,"15min":0.58699999999999997},"min":{"1min":0.40699999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77500000000000002,"5min":0.92900000000000005,"15min":1.1319999999999999},"last":0.53300000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.51300000000000001,"15min":0.54800000000000004},"min":{"1min":0.34399999999999997,"5min":0.30399999999999999,"15min":0.27300000000000002},"max":{"1min":0.73999999999999999,"5min":0.81100000000000005,"15min":0.998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57599999999999996,"15min":0.60399999999999998},"min":{"1min":0.46000000000000002,"5min":0.36299999999999999,"15min":0.35099999999999998},"max":{"1min":0.82399999999999995,"5min":0.95599999999999996,"15min":0.97999999999999998},"last":0.59199999999999997}]},{"osd":6,"last update":"Mon Oct 9 06:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54800000000000004,"15min":0.56599999999999995},"min":{"1min":0.39400000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70299999999999996,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56899999999999995,"15min":0.59599999999999997},"min":{"1min":0.41399999999999998,"5min":0.34999999999999998,"15min":0.32700000000000001},"max":{"1min":0.78300000000000003,"5min":1.01,"15min":1.01},"last":0.63600000000000001}]},{"osd":7,"last update":"Mon Oct 9 06:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53800000000000003,"15min":0.56699999999999995},"min":{"1min":0.35299999999999998,"5min":0.33600000000000002,"15min":0.28299999999999997},"max":{"1min":0.70299999999999996,"5min":0.83199999999999996,"15min":0.90000000000000002},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60499999999999998,"15min":0.64100000000000001},"min":{"1min":0.45000000000000001,"5min":0.34999999999999998,"15min":0.32200000000000001},"max":{"1min":0.81999999999999995,"5min":0.98999999999999999,"15min":1.175},"last":0.622}]}]},{"osd":5,"up_from":13,"seq":55834574952,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303404,"kb_used_data":2020,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068436,"statfs":{"total":96636764160,"available":96326078464,"internally_reserved":0,"allocated":2068480,"data_stored":922712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.502,"15min":0.50900000000000001},"min":{"1min":0.40100000000000002,"5min":0.29299999999999998,"15min":0.28599999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.86599999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51200000000000001,"15min":0.51500000000000001},"min":{"1min":0.38200000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72899999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.72899999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.504,"15min":0.51000000000000001},"min":{"1min":0.38,"5min":0.311,"15min":0.311},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.78200000000000003},"last":0.63},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.52200000000000002,"15min":0.53100000000000003},"min":{"1min":0.41199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.81599999999999995},"last":0.76600000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.47199999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.83999999999999997},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57199999999999995,"15min":0.58499999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.39500000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79900000000000004},"last":0.79300000000000004}]},{"osd":3,"last update":"Mon Oct 9 06:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55200000000000005,"15min":0.54600000000000004},"min":{"1min":0.39100000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.55900000000000005},"min":{"1min":0.40699999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.69199999999999995,"5min":0.71099999999999997,"15min":0.81100000000000005},"last":0.67400000000000004}]},{"osd":4,"last update":"Mon Oct 9 06:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.50700000000000001,"15min":0.5},"min":{"1min":0.34200000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68400000000000005,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.40300000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.80700000000000005},"last":0.54200000000000004}]},{"osd":6,"last update":"Mon Oct 9 06:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55500000000000005,"15min":0.53900000000000003},"min":{"1min":0.378,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.83199999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56499999999999995,"15min":0.56200000000000006},"min":{"1min":0.40200000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71799999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.64600000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.35999999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73199999999999998,"5min":0.79500000000000004,"15min":0.85599999999999998},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.44700000000000001,"5min":0.35199999999999998,"15min":0.314},"max":{"1min":0.77400000000000002,"5min":0.84499999999999997,"15min":0.90600000000000003},"last":0.745}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:38:54.727 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-09T06:38:54.727 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-09T06:38:54.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.849+0000 7f0e25f82700 1 Processor -- start 2023-10-09T06:38:54.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.849+0000 7f0e25f82700 1 -- start start 2023-10-09T06:38:54.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.850+0000 7f0e25f82700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e20159950 0x7f0e200cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:54.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.850+0000 7f0e25f82700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0e200af320 0x7f0e20069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:54.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.850+0000 7f0e25f82700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e200ce380 con 0x7f0e20069aa0 2023-10-09T06:38:54.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.850+0000 7f0e25f82700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e20069f20 con 0x7f0e200af320 2023-10-09T06:38:54.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.850+0000 7f0e25f82700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e200cec20 con 0x7f0e20159950 2023-10-09T06:38:54.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.850+0000 7f0e1ffff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f0e20069aa0 0x7f0e2015d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:36294/0 (socket says 172.21.15.163:36294) 2023-10-09T06:38:54.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.850+0000 7f0e1ffff700 1 -- 172.21.15.163:0/2313410388 learned_addr learned my addr 172.21.15.163:0/2313410388 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:54.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.850+0000 7f0e1f7fe700 1 --2- 172.21.15.163:0/2313410388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e20159950 0x7f0e200cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:54.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.850+0000 7f0e1effd700 1 --2- 172.21.15.163:0/2313410388 >> v2:172.21.15.181:3300/0 conn(0x7f0e200af320 0x7f0e20069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:54.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.850+0000 7f0e1e7fc700 1 -- 172.21.15.163:0/2313410388 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1989007612 0 0) 0x7f0e200ce380 con 0x7f0e20069aa0 2023-10-09T06:38:54.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.850+0000 7f0e1e7fc700 1 -- 172.21.15.163:0/2313410388 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e08003570 con 0x7f0e20069aa0 2023-10-09T06:38:54.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.850+0000 7f0e1e7fc700 1 -- 172.21.15.163:0/2313410388 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2084343066 0 0) 0x7f0e08003570 con 0x7f0e20069aa0 2023-10-09T06:38:54.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.850+0000 7f0e1e7fc700 1 -- 172.21.15.163:0/2313410388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e20159950 msgr2=0x7f0e200cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:54.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.850+0000 7f0e1e7fc700 1 --2- 172.21.15.163:0/2313410388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e20159950 0x7f0e200cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:54.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.850+0000 7f0e1e7fc700 1 -- 172.21.15.163:0/2313410388 >> v2:172.21.15.181:3300/0 conn(0x7f0e200af320 msgr2=0x7f0e20069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:54.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.850+0000 7f0e1e7fc700 1 --2- 172.21.15.163:0/2313410388 >> v2:172.21.15.181:3300/0 conn(0x7f0e200af320 0x7f0e20069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:54.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.850+0000 7f0e1e7fc700 1 -- 172.21.15.163:0/2313410388 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e200afa90 con 0x7f0e20069aa0 2023-10-09T06:38:54.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.850+0000 7f0e1e7fc700 1 -- 172.21.15.163:0/2313410388 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f0e18001470 con 0x7f0e20069aa0 2023-10-09T06:38:54.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.850+0000 7f0e1e7fc700 1 -- 172.21.15.163:0/2313410388 >> v1:172.21.15.163:6789/0 conn(0x7f0e20069aa0 legacy=0x7f0e2015d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:54.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.850+0000 7f0e1e7fc700 1 --2- 172.21.15.163:0/2313410388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e08003a60 0x7f0e08003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:54.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.850+0000 7f0e1e7fc700 1 --2- 172.21.15.163:0/2313410388 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e08004810 0x7f0e08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:54.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.850+0000 7f0e1e7fc700 1 --2- 172.21.15.163:0/2313410388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e20159950 0x7f0e080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:54.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.850+0000 7f0e1e7fc700 1 -- 172.21.15.163:0/2313410388 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e20069f20 con 0x7f0e08004810 2023-10-09T06:38:54.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.851+0000 7f0e1e7fc700 1 -- 172.21.15.163:0/2313410388 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e200cec20 con 0x7f0e08003a60 2023-10-09T06:38:54.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.851+0000 7f0e1e7fc700 1 -- 172.21.15.163:0/2313410388 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0800ba60 con 0x7f0e20159950 2023-10-09T06:38:54.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.851+0000 7f0e1effd700 1 --2- 172.21.15.163:0/2313410388 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e08004810 0x7f0e08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:54.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.851+0000 7f0e1f7fe700 1 --2- 172.21.15.163:0/2313410388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e08003a60 0x7f0e08003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:54.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.851+0000 7f0e1ffff700 1 --2- 172.21.15.163:0/2313410388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e20159950 0x7f0e080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:54.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.851+0000 7f0e1effd700 1 -- 172.21.15.163:0/2313410388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e20159950 msgr2=0x7f0e080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:54.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.851+0000 7f0e1effd700 1 --2- 172.21.15.163:0/2313410388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e20159950 0x7f0e080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:54.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.851+0000 7f0e1effd700 1 -- 172.21.15.163:0/2313410388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e08003a60 msgr2=0x7f0e08003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:54.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.851+0000 7f0e1effd700 1 --2- 172.21.15.163:0/2313410388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e08003a60 0x7f0e08003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:54.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.851+0000 7f0e1effd700 1 -- 172.21.15.163:0/2313410388 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e08003570 con 0x7f0e08004810 2023-10-09T06:38:54.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.851+0000 7f0e1ffff700 1 --2- 172.21.15.163:0/2313410388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e20159950 0x7f0e080051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:54.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.851+0000 7f0e1effd700 1 --2- 172.21.15.163:0/2313410388 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e08004810 0x7f0e08004c80 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009c30 tx=0x7f0e140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad33ba018942421 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:54.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.851+0000 7f0e1e7fc700 1 -- 172.21.15.163:0/2313410388 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1400e710 con 0x7f0e08004810 2023-10-09T06:38:54.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.851+0000 7f0e1e7fc700 1 -- 172.21.15.163:0/2313410388 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1400e870 con 0x7f0e08004810 2023-10-09T06:38:54.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.851+0000 7f0e1e7fc700 1 -- 172.21.15.163:0/2313410388 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1400ec30 con 0x7f0e08004810 2023-10-09T06:38:54.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.852+0000 7f0e25f82700 1 -- 172.21.15.163:0/2313410388 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e08004810 msgr2=0x7f0e08004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:54.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.852+0000 7f0e25f82700 1 --2- 172.21.15.163:0/2313410388 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e08004810 0x7f0e08004c80 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009c30 tx=0x7f0e140091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:54.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.852+0000 7f0e1ffff700 1 -- 172.21.15.163:0/2313410388 reap_dead start 2023-10-09T06:38:54.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.852+0000 7f0e25f82700 1 -- 172.21.15.163:0/2313410388 shutdown_connections 2023-10-09T06:38:54.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.852+0000 7f0e25f82700 1 -- 172.21.15.163:0/2313410388 >> 172.21.15.163:0/2313410388 conn(0x7f0e20068e00 msgr2=0x7f0e200d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:54.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.852+0000 7f0e25f82700 1 -- 172.21.15.163:0/2313410388 shutdown_connections 2023-10-09T06:38:54.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.852+0000 7f0e25f82700 1 -- 172.21.15.163:0/2313410388 wait complete. 2023-10-09T06:38:54.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.853+0000 7f0e25f82700 1 Processor -- start 2023-10-09T06:38:54.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.853+0000 7f0e25f82700 1 -- start start 2023-10-09T06:38:54.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.854+0000 7f0e25f82700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e08004810 0x7f0e2014cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:54.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.854+0000 7f0e25f82700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e200d07b0 0x7f0e200d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:54.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.854+0000 7f0e25f82700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e200c99f0 0x7f0e200c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:54.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.854+0000 7f0e25f82700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e200d1160 con 0x7f0e200d07b0 2023-10-09T06:38:54.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.854+0000 7f0e25f82700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e200b3f20 con 0x7f0e08004810 2023-10-09T06:38:54.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.854+0000 7f0e25f82700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e200b5660 con 0x7f0e200c99f0 2023-10-09T06:38:54.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.854+0000 7f0e1effd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e200d07b0 0x7f0e200d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:54.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.854+0000 7f0e1effd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e200d07b0 0x7f0e200d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44672/0 (socket says 172.21.15.163:44672) 2023-10-09T06:38:54.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.854+0000 7f0e1effd700 1 -- 172.21.15.163:0/2705822961 learned_addr learned my addr 172.21.15.163:0/2705822961 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:54.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.854+0000 7f0e1f7fe700 1 --2- 172.21.15.163:0/2705822961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e08004810 0x7f0e2014cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:54.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.854+0000 7f0e1ffff700 1 --2- 172.21.15.163:0/2705822961 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e200c99f0 0x7f0e200c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:54.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.854+0000 7f0e1effd700 1 -- 172.21.15.163:0/2705822961 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e200c99f0 msgr2=0x7f0e200c9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:54.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.854+0000 7f0e1effd700 1 --2- 172.21.15.163:0/2705822961 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e200c99f0 0x7f0e200c9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:54.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.854+0000 7f0e1effd700 1 -- 172.21.15.163:0/2705822961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e08004810 msgr2=0x7f0e2014cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:54.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.854+0000 7f0e1effd700 1 --2- 172.21.15.163:0/2705822961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e08004810 0x7f0e2014cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:54.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.854+0000 7f0e1effd700 1 -- 172.21.15.163:0/2705822961 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2014d130 con 0x7f0e200d07b0 2023-10-09T06:38:54.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.854+0000 7f0e1ffff700 1 --2- 172.21.15.163:0/2705822961 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e200c99f0 0x7f0e200c9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:54.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.854+0000 7f0e1effd700 1 --2- 172.21.15.163:0/2705822961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e200d07b0 0x7f0e200d0c20 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f0e080046d0 tx=0x7f0e1401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:54.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.854+0000 7f0e077fe700 1 -- 172.21.15.163:0/2705822961 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14022070 con 0x7f0e200d07b0 2023-10-09T06:38:54.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.854+0000 7f0e077fe700 1 -- 172.21.15.163:0/2705822961 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1400e9e0 con 0x7f0e200d07b0 2023-10-09T06:38:54.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.854+0000 7f0e25f82700 1 -- 172.21.15.163:0/2705822961 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e200ca760 con 0x7f0e200d07b0 2023-10-09T06:38:54.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.854+0000 7f0e077fe700 1 -- 172.21.15.163:0/2705822961 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14027470 con 0x7f0e200d07b0 2023-10-09T06:38:54.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.854+0000 7f0e25f82700 1 -- 172.21.15.163:0/2705822961 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e200b38d0 con 0x7f0e200d07b0 2023-10-09T06:38:54.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.855+0000 7f0e25f82700 1 -- 172.21.15.163:0/2705822961 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e200718b0 con 0x7f0e200d07b0 2023-10-09T06:38:54.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.856+0000 7f0e077fe700 1 -- 172.21.15.163:0/2705822961 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0e1401c450 con 0x7f0e200d07b0 2023-10-09T06:38:54.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.857+0000 7f0e077fe700 1 --2- 172.21.15.163:0/2705822961 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0e0c067320 0x7f0e0c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:54.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.857+0000 7f0e077fe700 1 -- 172.21.15.163:0/2705822961 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0e14086da0 con 0x7f0e200d07b0 2023-10-09T06:38:54.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.857+0000 7f0e1f7fe700 1 --2- 172.21.15.163:0/2705822961 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0e0c067320 0x7f0e0c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:54.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.858+0000 7f0e1f7fe700 1 --2- 172.21.15.163:0/2705822961 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0e0c067320 0x7f0e0c0697d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0e200cf360 tx=0x7f0e1000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:54.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:54.860+0000 7f0e077fe700 1 -- 172.21.15.163:0/2705822961 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e14053a70 con 0x7f0e200d07b0 2023-10-09T06:38:55.116 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.115+0000 7f0e25f82700 1 -- 172.21.15.163:0/2705822961 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f0e200d1160 con 0x7f0e200d07b0 2023-10-09T06:38:55.119 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.119+0000 7f0e077fe700 1 -- 172.21.15.163:0/2705822961 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) v1 ==== 190+0+0 (secure 0 0 0) 0x7f0e14057090 con 0x7f0e200d07b0 2023-10-09T06:38:55.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.121+0000 7f0e25f82700 1 -- 172.21.15.163:0/2705822961 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0e0c067320 msgr2=0x7f0e0c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:55.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.121+0000 7f0e25f82700 1 --2- 172.21.15.163:0/2705822961 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0e0c067320 0x7f0e0c0697d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0e200cf360 tx=0x7f0e1000b040 comp rx=0 tx=0).stop 2023-10-09T06:38:55.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.121+0000 7f0e25f82700 1 -- 172.21.15.163:0/2705822961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e200d07b0 msgr2=0x7f0e200d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:55.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.121+0000 7f0e25f82700 1 --2- 172.21.15.163:0/2705822961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e200d07b0 0x7f0e200d0c20 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f0e080046d0 tx=0x7f0e1401a000 comp rx=0 tx=0).stop 2023-10-09T06:38:55.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.121+0000 7f0e25f82700 1 -- 172.21.15.163:0/2705822961 shutdown_connections 2023-10-09T06:38:55.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.121+0000 7f0e25f82700 1 --2- 172.21.15.163:0/2705822961 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0e0c067320 0x7f0e0c0697d0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:55.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.121+0000 7f0e25f82700 1 --2- 172.21.15.163:0/2705822961 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e200c99f0 0x7f0e200c9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:55.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.121+0000 7f0e25f82700 1 --2- 172.21.15.163:0/2705822961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e08004810 0x7f0e2014cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:55.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.121+0000 7f0e25f82700 1 --2- 172.21.15.163:0/2705822961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e200d07b0 0x7f0e200d0c20 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:55.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.121+0000 7f0e25f82700 1 -- 172.21.15.163:0/2705822961 >> 172.21.15.163:0/2705822961 conn(0x7f0e20068e00 msgr2=0x7f0e20154710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:55.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.121+0000 7f0e25f82700 1 -- 172.21.15.163:0/2705822961 shutdown_connections 2023-10-09T06:38:55.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.121+0000 7f0e25f82700 1 -- 172.21.15.163:0/2705822961 wait complete. 2023-10-09T06:38:55.124 INFO:teuthology.orchestra.run.smithi163.stderr:updated caps for client.0 2023-10-09T06:38:55.135 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:38:55.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.249+0000 7f25b9808700 1 Processor -- start 2023-10-09T06:38:55.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.249+0000 7f25b9808700 1 -- start start 2023-10-09T06:38:55.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.250+0000 7f25b9808700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f25b4159420 0x7f25b40b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:55.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.250+0000 7f25b9808700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b40b2980 con 0x7f25b415d230 2023-10-09T06:38:55.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.250+0000 7f25b9808700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b40b11d0 con 0x7f25b41597f0 2023-10-09T06:38:55.252 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.250+0000 7f25b9808700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b40b14b0 con 0x7f25b4159420 2023-10-09T06:38:55.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.250+0000 7f25b8806700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f25b4159420 0x7f25b40b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:55.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.250+0000 7f25b9007700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f25b415d230 0x7f25b415d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:36304/0 (socket says 172.21.15.163:36304) 2023-10-09T06:38:55.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.250+0000 7f25b9007700 1 -- 172.21.15.163:0/75894581 learned_addr learned my addr 172.21.15.163:0/75894581 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:55.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.250+0000 7f25b8806700 1 -- 172.21.15.163:0/75894581 >> v1:172.21.15.181:6789/0 conn(0x7f25b41597f0 legacy=0x7f25b40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:55.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.250+0000 7f25b8806700 1 -- 172.21.15.163:0/75894581 >> v1:172.21.15.163:6789/0 conn(0x7f25b415d230 legacy=0x7f25b415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:55.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.250+0000 7f25b8806700 1 -- 172.21.15.163:0/75894581 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b40b1790 con 0x7f25b4159420 2023-10-09T06:38:55.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.250+0000 7f25b37fe700 1 -- 172.21.15.163:0/75894581 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2746704389 0 0) 0x7f25b40b2980 con 0x7f25b415d230 2023-10-09T06:38:55.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.250+0000 7f25b8806700 1 --2- 172.21.15.163:0/75894581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25b4159420 0x7f25b40b5f40 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f25a80098e0 tx=0x7f25a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f7184a12db063af server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:55.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.251+0000 7f25b37fe700 1 -- 172.21.15.163:0/75894581 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a8009e80 con 0x7f25b4159420 2023-10-09T06:38:55.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.251+0000 7f25b37fe700 1 -- 172.21.15.163:0/75894581 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a800b820 con 0x7f25b4159420 2023-10-09T06:38:55.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.251+0000 7f25b37fe700 1 -- 172.21.15.163:0/75894581 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a800bba0 con 0x7f25b4159420 2023-10-09T06:38:55.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.251+0000 7f25b9808700 1 -- 172.21.15.163:0/75894581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25b4159420 msgr2=0x7f25b40b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:55.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.251+0000 7f25b9808700 1 --2- 172.21.15.163:0/75894581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25b4159420 0x7f25b40b5f40 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f25a80098e0 tx=0x7f25a80091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:55.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.251+0000 7f25b9808700 1 -- 172.21.15.163:0/75894581 shutdown_connections 2023-10-09T06:38:55.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.251+0000 7f25b9808700 1 --2- 172.21.15.163:0/75894581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25b4159420 0x7f25b40b5f40 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:55.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.251+0000 7f25b9808700 1 -- 172.21.15.163:0/75894581 >> 172.21.15.163:0/75894581 conn(0x7f25b40d0660 msgr2=0x7f25b40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:55.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.251+0000 7f25b9808700 1 -- 172.21.15.163:0/75894581 shutdown_connections 2023-10-09T06:38:55.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.251+0000 7f25b9808700 1 -- 172.21.15.163:0/75894581 wait complete. 2023-10-09T06:38:55.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.252+0000 7f25b9808700 1 Processor -- start 2023-10-09T06:38:55.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.252+0000 7f25b9808700 1 -- start start 2023-10-09T06:38:55.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.253+0000 7f25b9808700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25b4159420 0x7f25b40c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:55.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.253+0000 7f25b9808700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25b41597f0 0x7f25b4155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:55.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.253+0000 7f25b9808700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25b415d230 0x7f25b41581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:55.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.253+0000 7f25b9808700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b40c5bd0 con 0x7f25b41597f0 2023-10-09T06:38:55.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.253+0000 7f25b9808700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b416a520 con 0x7f25b415d230 2023-10-09T06:38:55.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.253+0000 7f25b9808700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b40ca430 con 0x7f25b4159420 2023-10-09T06:38:55.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.253+0000 7f25b3fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25b41597f0 0x7f25b4155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:55.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.253+0000 7f25b3fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25b41597f0 0x7f25b4155350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44692/0 (socket says 172.21.15.163:44692) 2023-10-09T06:38:55.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.253+0000 7f25b3fff700 1 -- 172.21.15.163:0/1756966760 learned_addr learned my addr 172.21.15.163:0/1756966760 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:55.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.254+0000 7f25b8806700 1 --2- 172.21.15.163:0/1756966760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25b4159420 0x7f25b40c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:55.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.254+0000 7f25b3fff700 1 -- 172.21.15.163:0/1756966760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25b4159420 msgr2=0x7f25b40c60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:55.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.254+0000 7f25b9007700 1 --2- 172.21.15.163:0/1756966760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25b415d230 0x7f25b41581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:55.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.254+0000 7f25b3fff700 1 --2- 172.21.15.163:0/1756966760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25b4159420 0x7f25b40c60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:55.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.254+0000 7f25b3fff700 1 -- 172.21.15.163:0/1756966760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25b415d230 msgr2=0x7f25b41581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:55.259 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.254+0000 7f25b3fff700 1 --2- 172.21.15.163:0/1756966760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25b415d230 0x7f25b41581b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:55.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.254+0000 7f25b3fff700 1 -- 172.21.15.163:0/1756966760 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a8009420 con 0x7f25b41597f0 2023-10-09T06:38:55.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.254+0000 7f25b3fff700 1 --2- 172.21.15.163:0/1756966760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25b41597f0 0x7f25b4155350 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f25a000c590 tx=0x7f25a0007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:55.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.254+0000 7f25b0ff9700 1 -- 172.21.15.163:0/1756966760 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a0011590 con 0x7f25b41597f0 2023-10-09T06:38:55.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.254+0000 7f25b0ff9700 1 -- 172.21.15.163:0/1756966760 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a0011730 con 0x7f25b41597f0 2023-10-09T06:38:55.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.254+0000 7f25b9808700 1 -- 172.21.15.163:0/1756966760 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b40ca710 con 0x7f25b41597f0 2023-10-09T06:38:55.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.254+0000 7f25b0ff9700 1 -- 172.21.15.163:0/1756966760 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a0011af0 con 0x7f25b41597f0 2023-10-09T06:38:55.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.254+0000 7f25b9808700 1 -- 172.21.15.163:0/1756966760 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b40c6600 con 0x7f25b41597f0 2023-10-09T06:38:55.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.255+0000 7f25b9808700 1 -- 172.21.15.163:0/1756966760 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b40cac30 con 0x7f25b41597f0 2023-10-09T06:38:55.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.255+0000 7f25b0ff9700 1 -- 172.21.15.163:0/1756966760 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f25a001c460 con 0x7f25b41597f0 2023-10-09T06:38:55.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.256+0000 7f25b0ff9700 1 --2- 172.21.15.163:0/1756966760 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2598067320 0x7f25980697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:55.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.256+0000 7f25b0ff9700 1 -- 172.21.15.163:0/1756966760 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f25a0084a50 con 0x7f25b41597f0 2023-10-09T06:38:55.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.256+0000 7f25b8806700 1 --2- 172.21.15.163:0/1756966760 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2598067320 0x7f25980697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:55.261 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.256+0000 7f25b8806700 1 --2- 172.21.15.163:0/1756966760 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2598067320 0x7f25980697d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f25a80091e0 tx=0x7f25a8016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:55.261 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.260+0000 7f25b0ff9700 1 -- 172.21.15.163:0/1756966760 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25b40cac30 con 0x7f25b41597f0 2023-10-09T06:38:55.386 INFO:tasks.ceph.mds.b.smithi181.stderr:2023-10-09T06:38:55.385+0000 7fa3ae362700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-09T06:38:55.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.487+0000 7f25b9808700 1 -- 172.21.15.163:0/1756966760 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25b40cac30 con 0x7f25b41597f0 2023-10-09T06:38:55.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.487+0000 7f25b0ff9700 1 -- 172.21.15.163:0/1756966760 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4619 (secure 0 0 0) 0x7f25b40cac30 con 0x7f25b41597f0 2023-10-09T06:38:55.488 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:38:55.489 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":112,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8211,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3056573272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3056573272},{"type":"v1","addr":"172.21.15.163:6834","nonce":3056573272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":8214,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3482861499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3482861499},{"type":"v1","addr":"172.21.15.163:6835","nonce":3482861499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":8223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/4239124825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4239124825},{"type":"v1","addr":"172.21.15.181:6836","nonce":4239124825}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:38:49.480715+0000","modified":"2023-10-09T06:38:53.141039+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8226":{"gid":8226,"name":"b","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/3494162891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3494162891},{"type":"v1","addr":"172.21.15.181:6837","nonce":3494162891}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-09T06:38:55.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.489+0000 7f25b9808700 1 -- 172.21.15.163:0/1756966760 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2598067320 msgr2=0x7f25980697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:55.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.489+0000 7f25b9808700 1 --2- 172.21.15.163:0/1756966760 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2598067320 0x7f25980697d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f25a80091e0 tx=0x7f25a8016000 comp rx=0 tx=0).stop 2023-10-09T06:38:55.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.489+0000 7f25b9808700 1 -- 172.21.15.163:0/1756966760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25b41597f0 msgr2=0x7f25b4155350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:55.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.489+0000 7f25b9808700 1 --2- 172.21.15.163:0/1756966760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25b41597f0 0x7f25b4155350 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f25a000c590 tx=0x7f25a0007f30 comp rx=0 tx=0).stop 2023-10-09T06:38:55.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.490+0000 7f25b9808700 1 -- 172.21.15.163:0/1756966760 shutdown_connections 2023-10-09T06:38:55.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.490+0000 7f25b9808700 1 --2- 172.21.15.163:0/1756966760 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2598067320 0x7f25980697d0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:55.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.490+0000 7f25b9808700 1 --2- 172.21.15.163:0/1756966760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25b415d230 0x7f25b41581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:55.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.490+0000 7f25b9808700 1 --2- 172.21.15.163:0/1756966760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25b4159420 0x7f25b40c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:55.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.490+0000 7f25b9808700 1 --2- 172.21.15.163:0/1756966760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25b41597f0 0x7f25b4155350 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:55.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.490+0000 7f25b9808700 1 -- 172.21.15.163:0/1756966760 >> 172.21.15.163:0/1756966760 conn(0x7f25b40d0660 msgr2=0x7f25b4156190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:55.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.490+0000 7f25b9808700 1 -- 172.21.15.163:0/1756966760 shutdown_connections 2023-10-09T06:38:55.492 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:55.490+0000 7f25b9808700 1 -- 172.21.15.163:0/1756966760 wait complete. 2023-10-09T06:38:55.492 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 112 2023-10-09T06:38:55.905 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 112, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-09T06:38:49.480715+0000', 'modified': '2023-10-09T06:38:53.141039+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8226}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8226': {'gid': 8226, 'name': 'b', 'rank': 0, 'incarnation': 108, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6837/3494162891', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6835', 'nonce': 3494162891}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 3494162891}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [29], 'metadata_pool': 28, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-09T06:38:55.905 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-09T06:38:55.906 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-09T06:38:56.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.022+0000 7fbf4b59e700 1 Processor -- start 2023-10-09T06:38:56.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.022+0000 7fbf4b59e700 1 -- start start 2023-10-09T06:38:56.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.023+0000 7fbf4b59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf4c15d230 0x7fbf4c15d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:56.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.023+0000 7fbf4b59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf4c0b2980 con 0x7fbf4c1597f0 2023-10-09T06:38:56.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.023+0000 7fbf4b59e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf4c15dbb0 con 0x7fbf4c159420 2023-10-09T06:38:56.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.023+0000 7fbf4b59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c0b11d0 con 0x7fbf4c15d230 2023-10-09T06:38:56.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.023+0000 7fbf41d9b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fbf4c1597f0 0x7fbf4c0b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:36314/0 (socket says 172.21.15.163:36314) 2023-10-09T06:38:56.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.023+0000 7fbf41d9b700 1 -- 172.21.15.163:0/2582403190 learned_addr learned my addr 172.21.15.163:0/2582403190 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:56.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.023+0000 7fbf4ad9d700 1 --2- 172.21.15.163:0/2582403190 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf4c15d230 0x7fbf4c15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:56.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.023+0000 7fbf49d9b700 1 -- 172.21.15.163:0/2582403190 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 615981951 0 0) 0x7fbf4c0b2980 con 0x7fbf4c1597f0 2023-10-09T06:38:56.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.023+0000 7fbf49d9b700 1 -- 172.21.15.163:0/2582403190 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf38003570 con 0x7fbf4c1597f0 2023-10-09T06:38:56.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.023+0000 7fbf49d9b700 1 -- 172.21.15.163:0/2582403190 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1390065584 0 0) 0x7fbf4c15dbb0 con 0x7fbf4c159420 2023-10-09T06:38:56.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.023+0000 7fbf49d9b700 1 -- 172.21.15.163:0/2582403190 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf4c0b2980 con 0x7fbf4c159420 2023-10-09T06:38:56.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.023+0000 7fbf49d9b700 1 -- 172.21.15.163:0/2582403190 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3884235674 0 0) 0x7fbf38003570 con 0x7fbf4c1597f0 2023-10-09T06:38:56.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.023+0000 7fbf4ad9d700 1 -- 172.21.15.163:0/2582403190 >> v1:172.21.15.181:6789/0 conn(0x7fbf4c159420 legacy=0x7fbf4c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:56.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.023+0000 7fbf4ad9d700 1 -- 172.21.15.163:0/2582403190 >> v1:172.21.15.163:6789/0 conn(0x7fbf4c1597f0 legacy=0x7fbf4c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:56.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.023+0000 7fbf4ad9d700 1 -- 172.21.15.163:0/2582403190 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4c0b14b0 con 0x7fbf4c15d230 2023-10-09T06:38:56.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.024+0000 7fbf4ad9d700 1 --2- 172.21.15.163:0/2582403190 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf4c15d230 0x7fbf4c15d670 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009e90 tx=0x7fbf44009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de8c8d6240fdbabe server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:56.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.024+0000 7fbf49d9b700 1 -- 172.21.15.163:0/2582403190 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4400e070 con 0x7fbf4c15d230 2023-10-09T06:38:56.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.024+0000 7fbf49d9b700 1 -- 172.21.15.163:0/2582403190 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4400b870 con 0x7fbf4c15d230 2023-10-09T06:38:56.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.024+0000 7fbf49d9b700 1 -- 172.21.15.163:0/2582403190 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf34001470 con 0x7fbf4c15d230 2023-10-09T06:38:56.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.024+0000 7fbf4b59e700 1 -- 172.21.15.163:0/2582403190 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf4c15d230 msgr2=0x7fbf4c15d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:56.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.024+0000 7fbf4b59e700 1 --2- 172.21.15.163:0/2582403190 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf4c15d230 0x7fbf4c15d670 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009e90 tx=0x7fbf44009780 comp rx=0 tx=0).stop 2023-10-09T06:38:56.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.024+0000 7fbf4b59e700 1 -- 172.21.15.163:0/2582403190 shutdown_connections 2023-10-09T06:38:56.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.024+0000 7fbf4b59e700 1 --2- 172.21.15.163:0/2582403190 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf4c15d230 0x7fbf4c15d670 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.024+0000 7fbf4b59e700 1 -- 172.21.15.163:0/2582403190 >> 172.21.15.163:0/2582403190 conn(0x7fbf4c0d0660 msgr2=0x7fbf4c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:56.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.024+0000 7fbf4b59e700 1 -- 172.21.15.163:0/2582403190 shutdown_connections 2023-10-09T06:38:56.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.024+0000 7fbf4b59e700 1 -- 172.21.15.163:0/2582403190 wait complete. 2023-10-09T06:38:56.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.025+0000 7fbf4b59e700 1 Processor -- start 2023-10-09T06:38:56.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.026+0000 7fbf4b59e700 1 -- start start 2023-10-09T06:38:56.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.026+0000 7fbf4b59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf4c159420 0x7fbf4c0c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:56.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.026+0000 7fbf4b59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf4c1597f0 0x7fbf4c0c5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:56.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.026+0000 7fbf4b59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf4c15d230 0x7fbf4c0c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:56.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.026+0000 7fbf4b59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c0c6fc0 con 0x7fbf4c15d230 2023-10-09T06:38:56.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.026+0000 7fbf4b59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c0c63b0 con 0x7fbf4c1597f0 2023-10-09T06:38:56.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.026+0000 7fbf4b59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c0c6690 con 0x7fbf4c159420 2023-10-09T06:38:56.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.026+0000 7fbf4ad9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf4c15d230 0x7fbf4c0c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:56.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.026+0000 7fbf4ad9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf4c15d230 0x7fbf4c0c7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44708/0 (socket says 172.21.15.163:44708) 2023-10-09T06:38:56.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.026+0000 7fbf4ad9d700 1 -- 172.21.15.163:0/2326480232 learned_addr learned my addr 172.21.15.163:0/2326480232 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:56.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.027+0000 7fbf41d9b700 1 --2- 172.21.15.163:0/2326480232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf4c1597f0 0x7fbf4c0c5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:56.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.027+0000 7fbf4ad9d700 1 -- 172.21.15.163:0/2326480232 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf4c159420 msgr2=0x7fbf4c0c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:56.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.027+0000 7fbf4a59c700 1 --2- 172.21.15.163:0/2326480232 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf4c159420 0x7fbf4c0c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:56.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.027+0000 7fbf4ad9d700 1 --2- 172.21.15.163:0/2326480232 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf4c159420 0x7fbf4c0c84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.027+0000 7fbf4ad9d700 1 -- 172.21.15.163:0/2326480232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf4c1597f0 msgr2=0x7fbf4c0c5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:56.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.027+0000 7fbf4ad9d700 1 --2- 172.21.15.163:0/2326480232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf4c1597f0 0x7fbf4c0c5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.027+0000 7fbf4ad9d700 1 -- 172.21.15.163:0/2326480232 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf44009a20 con 0x7fbf4c15d230 2023-10-09T06:38:56.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.027+0000 7fbf4ad9d700 1 --2- 172.21.15.163:0/2326480232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf4c15d230 0x7fbf4c0c7450 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009de0 tx=0x7fbf44017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:56.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.027+0000 7fbf42ffd700 1 -- 172.21.15.163:0/2326480232 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44012d80 con 0x7fbf4c15d230 2023-10-09T06:38:56.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.027+0000 7fbf42ffd700 1 -- 172.21.15.163:0/2326480232 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4401e450 con 0x7fbf4c15d230 2023-10-09T06:38:56.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.027+0000 7fbf4b59e700 1 -- 172.21.15.163:0/2326480232 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf4c0c6970 con 0x7fbf4c15d230 2023-10-09T06:38:56.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.027+0000 7fbf42ffd700 1 -- 172.21.15.163:0/2326480232 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4401e810 con 0x7fbf4c15d230 2023-10-09T06:38:56.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.027+0000 7fbf4b59e700 1 -- 172.21.15.163:0/2326480232 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf4c0c89e0 con 0x7fbf4c15d230 2023-10-09T06:38:56.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.028+0000 7fbf42ffd700 1 -- 172.21.15.163:0/2326480232 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbf4401ec40 con 0x7fbf4c15d230 2023-10-09T06:38:56.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.029+0000 7fbf42ffd700 1 --2- 172.21.15.163:0/2326480232 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbf2c067320 0x7fbf2c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:56.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.029+0000 7fbf42ffd700 1 -- 172.21.15.163:0/2326480232 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbf44086cd0 con 0x7fbf4c15d230 2023-10-09T06:38:56.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.029+0000 7fbf4a59c700 1 --2- 172.21.15.163:0/2326480232 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbf2c067320 0x7fbf2c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:56.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.029+0000 7fbf4a59c700 1 --2- 172.21.15.163:0/2326480232 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbf2c067320 0x7fbf2c0697d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c0027d0 tx=0x7fbf3c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:56.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.031+0000 7fbf59c3ab80 1 Processor -- start 2023-10-09T06:38:56.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.031+0000 7fbf59c3ab80 1 -- start start 2023-10-09T06:38:56.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.032+0000 7fbf59c3ab80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x557931631480 0x5579316107d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:56.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.032+0000 7fbf59c3ab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579315e45f0 0x557931631850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:56.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.032+0000 7fbf59c3ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579315e6b80 0x5579315e6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:56.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.032+0000 7fbf59c3ab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793153da70 con 0x5579315e45f0 2023-10-09T06:38:56.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.032+0000 7fbf59c3ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793153b270 con 0x5579315e6b80 2023-10-09T06:38:56.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.032+0000 7fbf59c3ab80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x557931537060 con 0x557931631480 2023-10-09T06:38:56.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.032+0000 7fbf41d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579315e45f0 0x557931631850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:56.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.032+0000 7fbf41d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579315e45f0 0x557931631850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44720/0 (socket says 172.21.15.163:44720) 2023-10-09T06:38:56.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.032+0000 7fbf41d9b700 1 -- 172.21.15.163:0/623658864 learned_addr learned my addr 172.21.15.163:0/623658864 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:56.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.032+0000 7fbf4a59c700 1 --2- 172.21.15.163:0/623658864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x557931631480 0x5579316107d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:56.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.032+0000 7fbf4ad9d700 1 --2- 172.21.15.163:0/623658864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579315e6b80 0x5579315e6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:56.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.032+0000 7fbf41d9b700 1 -- 172.21.15.163:0/623658864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x557931631480 msgr2=0x5579316107d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:56.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.032+0000 7fbf41d9b700 1 --2- 172.21.15.163:0/623658864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x557931631480 0x5579316107d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.032+0000 7fbf41d9b700 1 -- 172.21.15.163:0/623658864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579315e6b80 msgr2=0x5579315e6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:56.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.032+0000 7fbf41d9b700 1 --2- 172.21.15.163:0/623658864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579315e6b80 0x5579315e6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.032+0000 7fbf41d9b700 1 -- 172.21.15.163:0/623658864 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579313d1270 con 0x5579315e45f0 2023-10-09T06:38:56.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.032+0000 7fbf41d9b700 1 --2- 172.21.15.163:0/623658864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579315e45f0 0x557931631850 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fbf34007ae0 tx=0x7fbf34009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0705d9d2065bc44 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:56.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.032+0000 7fbf2b7fe700 1 -- 172.21.15.163:0/623658864 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3400b070 con 0x5579315e45f0 2023-10-09T06:38:56.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.032+0000 7fbf2b7fe700 1 -- 172.21.15.163:0/623658864 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf34005b10 con 0x5579315e45f0 2023-10-09T06:38:56.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.032+0000 7fbf2b7fe700 1 -- 172.21.15.163:0/623658864 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf340070d0 con 0x5579315e45f0 2023-10-09T06:38:56.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.033+0000 7fbf59c3ab80 1 -- 172.21.15.163:0/623658864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579315e45f0 msgr2=0x557931631850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:56.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.033+0000 7fbf59c3ab80 1 --2- 172.21.15.163:0/623658864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579315e45f0 0x557931631850 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fbf34007ae0 tx=0x7fbf34009040 comp rx=0 tx=0).stop 2023-10-09T06:38:56.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.033+0000 7fbf59c3ab80 1 -- 172.21.15.163:0/623658864 shutdown_connections 2023-10-09T06:38:56.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.033+0000 7fbf59c3ab80 1 --2- 172.21.15.163:0/623658864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579315e6b80 0x5579315e6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.033+0000 7fbf59c3ab80 1 --2- 172.21.15.163:0/623658864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x557931631480 0x5579316107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.033+0000 7fbf59c3ab80 1 --2- 172.21.15.163:0/623658864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579315e45f0 0x557931631850 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.033+0000 7fbf59c3ab80 1 -- 172.21.15.163:0/623658864 >> 172.21.15.163:0/623658864 conn(0x5579315e80c0 msgr2=0x5579315e88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:56.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.033+0000 7fbf59c3ab80 1 -- 172.21.15.163:0/623658864 shutdown_connections 2023-10-09T06:38:56.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.033+0000 7fbf59c3ab80 1 -- 172.21.15.163:0/623658864 wait complete. 2023-10-09T06:38:56.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.034+0000 7fbf59c3ab80 1 -- start start 2023-10-09T06:38:56.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.034+0000 7fbf59c3ab80 1 Processor -- start 2023-10-09T06:38:56.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.035+0000 7fbf59c3ab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579315e45f0 0x557931611c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:56.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.035+0000 7fbf59c3ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579315e6b80 0x5579315e73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:56.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.035+0000 7fbf59c3ab80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x557931631480 0x55793160a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:56.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.035+0000 7fbf59c3ab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579314e2890 con 0x5579315e45f0 2023-10-09T06:38:56.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.035+0000 7fbf59c3ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579314fdea0 con 0x5579315e6b80 2023-10-09T06:38:56.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.035+0000 7fbf59c3ab80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5579313b9e30 con 0x557931631480 2023-10-09T06:38:56.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.035+0000 7fbf59c3ab80 10 client.0 _ll_register_callbacks cb 0x5579315aa2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-09T06:38:56.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.035+0000 7fbf4a59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579315e45f0 0x557931611c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:56.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.035+0000 7fbf4a59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579315e45f0 0x557931611c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44722/0 (socket says 172.21.15.163:44722) 2023-10-09T06:38:56.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.035+0000 7fbf4a59c700 1 -- 172.21.15.163:0/679582341 learned_addr learned my addr 172.21.15.163:0/679582341 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:56.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.035+0000 7fbf41d9b700 1 --2- 172.21.15.163:0/679582341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579315e6b80 0x5579315e73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:56.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.035+0000 7fbf4ad9d700 1 --2- 172.21.15.163:0/679582341 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x557931631480 0x55793160a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:56.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.035+0000 7fbf4a59c700 1 -- 172.21.15.163:0/679582341 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x557931631480 msgr2=0x55793160a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:56.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.035+0000 7fbf4a59c700 1 --2- 172.21.15.163:0/679582341 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x557931631480 0x55793160a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.035+0000 7fbf4a59c700 1 -- 172.21.15.163:0/679582341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579315e6b80 msgr2=0x5579315e73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:56.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.035+0000 7fbf4a59c700 1 --2- 172.21.15.163:0/679582341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579315e6b80 0x5579315e73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.035+0000 7fbf4a59c700 1 -- 172.21.15.163:0/679582341 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf34001c70 con 0x5579315e45f0 2023-10-09T06:38:56.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.035+0000 7fbf4a59c700 1 --2- 172.21.15.163:0/679582341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579315e45f0 0x557931611c40 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c014fd0 tx=0x7fbf3c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6a239352e3d46b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:56.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.035+0000 7fbf28ff9700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-10-09T06:38:56.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.035+0000 7fbf28ff9700 1 -- 172.21.15.163:0/679582341 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c00dd90 con 0x5579315e45f0 2023-10-09T06:38:56.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.035+0000 7fbf28ff9700 1 -- 172.21.15.163:0/679582341 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbf3c0203f0 con 0x5579315e45f0 2023-10-09T06:38:56.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.035+0000 7fbf59c3ab80 1 -- 172.21.15.163:0/679582341 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5579315dbcd0 con 0x5579315e45f0 2023-10-09T06:38:56.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.036+0000 7fbf28ff9700 1 -- 172.21.15.163:0/679582341 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf3c0207c0 con 0x5579315e45f0 2023-10-09T06:38:56.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.036+0000 7fbf59c3ab80 10 client.8361 fetch_fsmap learned FSMap version 112 2023-10-09T06:38:56.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.036+0000 7fbf59c3ab80 1 -- 172.21.15.163:0/679582341 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x557931373fe0 con 0x5579315e45f0 2023-10-09T06:38:56.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.036+0000 7fbf28ff9700 1 -- 172.21.15.163:0/679582341 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5579315dbcd0 con 0x5579315e45f0 2023-10-09T06:38:56.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.036+0000 7fbf28ff9700 1 -- 172.21.15.163:0/679582341 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 112) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fbf3c01f070 con 0x5579315e45f0 2023-10-09T06:38:56.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.036+0000 7fbf59c3ab80 10 client.8361 fetch_fsmap finished waiting for FSMap version 112 2023-10-09T06:38:56.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.036+0000 7fbf59c3ab80 10 client.8361 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-09T06:38:56.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.036+0000 7fbf59c3ab80 20 client.8361 populate_metadata read hostname 'smithi163' 2023-10-09T06:38:56.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.036+0000 7fbf59c3ab80 1 --2- 172.21.15.163:0/679582341 >> [v2:172.21.15.181:6835/3494162891,v1:172.21.15.181:6837/3494162891] conn(0x557931640ba0 0x557931640f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:56.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.036+0000 7fbf59c3ab80 4 client.8361 mds_command: new command op to 8226 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-09T06:38:56.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.036+0000 7fbf59c3ab80 1 -- 172.21.15.163:0/679582341 --> [v2:172.21.15.181:6835/3494162891,v1:172.21.15.181:6837/3494162891] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5579315b8aa0 con 0x557931640ba0 2023-10-09T06:38:56.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.036+0000 7fbf41d9b700 1 --2- 172.21.15.163:0/679582341 >> [v2:172.21.15.181:6835/3494162891,v1:172.21.15.181:6837/3494162891] conn(0x557931640ba0 0x557931640f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:56.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.037+0000 7fbf28ff9700 10 client.8361 ms_handle_connect on v2:172.21.15.181:6835/3494162891 2023-10-09T06:38:56.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.037+0000 7fbf41d9b700 1 --2- 172.21.15.163:0/679582341 >> [v2:172.21.15.181:6835/3494162891,v1:172.21.15.181:6837/3494162891] conn(0x557931640ba0 0x557931640f70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db73fd354a1d038f server_cookie=6c2829ccd101b372 in_seq=0 out_seq=0 2023-10-09T06:38:56.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.037+0000 7fbf28ff9700 1 -- 172.21.15.163:0/679582341 <== mds.0 v2:172.21.15.181:6835/3494162891 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5579315b8aa0 con 0x557931640ba0 2023-10-09T06:38:56.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.037+0000 7fbf28ff9700 10 client.8361 handle_command_reply: tid=0 2023-10-09T06:38:56.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.037+0000 7fbf59c3ab80 1 client.8361 shutdown 2023-10-09T06:38:56.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.038+0000 7fbf59c3ab80 1 -- 172.21.15.163:0/679582341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579315e45f0 msgr2=0x557931611c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:56.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.038+0000 7fbf59c3ab80 1 --2- 172.21.15.163:0/679582341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579315e45f0 0x557931611c40 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c014fd0 tx=0x7fbf3c014000 comp rx=0 tx=0).stop 2023-10-09T06:38:56.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.038+0000 7fbf59c3ab80 1 -- 172.21.15.163:0/679582341 shutdown_connections 2023-10-09T06:38:56.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.038+0000 7fbf59c3ab80 1 --2- 172.21.15.163:0/679582341 >> [v2:172.21.15.181:6835/3494162891,v1:172.21.15.181:6837/3494162891] conn(0x557931640ba0 0x557931640f70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.038+0000 7fbf59c3ab80 1 --2- 172.21.15.163:0/679582341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579315e45f0 0x557931611c40 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.038+0000 7fbf28ff9700 0 client.8361 ms_handle_reset on v2:172.21.15.181:6835/3494162891 2023-10-09T06:38:56.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.038+0000 7fbf59c3ab80 1 --2- 172.21.15.163:0/679582341 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x557931631480 0x55793160a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.038+0000 7fbf59c3ab80 1 --2- 172.21.15.163:0/679582341 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5579315e6b80 0x5579315e73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.038+0000 7fbf59c3ab80 1 -- 172.21.15.163:0/679582341 >> 172.21.15.163:0/679582341 conn(0x5579315e80c0 msgr2=0x5579315ddcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:56.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.038+0000 7fbf59c3ab80 1 -- 172.21.15.163:0/679582341 shutdown_connections 2023-10-09T06:38:56.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.038+0000 7fbf59c3ab80 1 -- 172.21.15.163:0/679582341 wait complete. 2023-10-09T06:38:56.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.038+0000 7fbf59c3ab80 20 client.8361 trim_cache size 0 max 16384 2023-10-09T06:38:56.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.054+0000 7fbf59c3ab80 1 Processor -- start 2023-10-09T06:38:56.055 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.054+0000 7fbf59c3ab80 1 -- start start 2023-10-09T06:38:56.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.055+0000 7fbf59c3ab80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5579315e45f0 0x557931611c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:56.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.055+0000 7fbf59c3ab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579315e6b80 0x5579315e3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:56.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.055+0000 7fbf59c3ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557931631480 0x5579315e3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:56.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.055+0000 7fbf59c3ab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x557931612530 con 0x5579315e6b80 2023-10-09T06:38:56.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.055+0000 7fbf59c3ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x557931630f20 con 0x557931631480 2023-10-09T06:38:56.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.055+0000 7fbf59c3ab80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x557931631b20 con 0x5579315e45f0 2023-10-09T06:38:56.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.055+0000 7fbf41d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579315e6b80 0x5579315e3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:56.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.055+0000 7fbf41d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579315e6b80 0x5579315e3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44724/0 (socket says 172.21.15.163:44724) 2023-10-09T06:38:56.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.055+0000 7fbf41d9b700 1 -- 172.21.15.163:0/2884350288 learned_addr learned my addr 172.21.15.163:0/2884350288 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:56.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.055+0000 7fbf4a59c700 1 --2- 172.21.15.163:0/2884350288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5579315e45f0 0x557931611c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:56.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.055+0000 7fbf4ad9d700 1 --2- 172.21.15.163:0/2884350288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557931631480 0x5579315e3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:56.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.055+0000 7fbf41d9b700 1 -- 172.21.15.163:0/2884350288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5579315e45f0 msgr2=0x557931611c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:56.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.055+0000 7fbf41d9b700 1 --2- 172.21.15.163:0/2884350288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5579315e45f0 0x557931611c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.055+0000 7fbf41d9b700 1 -- 172.21.15.163:0/2884350288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557931631480 msgr2=0x5579315e3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:56.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.055+0000 7fbf41d9b700 1 --2- 172.21.15.163:0/2884350288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557931631480 0x5579315e3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.055+0000 7fbf41d9b700 1 -- 172.21.15.163:0/2884350288 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557931505780 con 0x5579315e6b80 2023-10-09T06:38:56.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.055+0000 7fbf41d9b700 1 --2- 172.21.15.163:0/2884350288 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579315e6b80 0x5579315e3a40 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fbf34006000 tx=0x7fbf34026250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dadc69dc0e00cd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:56.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.056+0000 7fbf2bfff700 1 -- 172.21.15.163:0/2884350288 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf34024070 con 0x5579315e6b80 2023-10-09T06:38:56.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.056+0000 7fbf2bfff700 1 -- 172.21.15.163:0/2884350288 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf34005c80 con 0x5579315e6b80 2023-10-09T06:38:56.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.056+0000 7fbf2bfff700 1 -- 172.21.15.163:0/2884350288 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf34005e10 con 0x5579315e6b80 2023-10-09T06:38:56.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.056+0000 7fbf59c3ab80 1 -- 172.21.15.163:0/2884350288 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579315e6b80 msgr2=0x5579315e3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:56.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.056+0000 7fbf59c3ab80 1 --2- 172.21.15.163:0/2884350288 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579315e6b80 0x5579315e3a40 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fbf34006000 tx=0x7fbf34026250 comp rx=0 tx=0).stop 2023-10-09T06:38:56.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.056+0000 7fbf59c3ab80 1 -- 172.21.15.163:0/2884350288 shutdown_connections 2023-10-09T06:38:56.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.056+0000 7fbf59c3ab80 1 --2- 172.21.15.163:0/2884350288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557931631480 0x5579315e3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.056+0000 7fbf59c3ab80 1 --2- 172.21.15.163:0/2884350288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5579315e45f0 0x557931611c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.056+0000 7fbf59c3ab80 1 --2- 172.21.15.163:0/2884350288 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579315e6b80 0x5579315e3a40 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.056+0000 7fbf59c3ab80 1 -- 172.21.15.163:0/2884350288 >> 172.21.15.163:0/2884350288 conn(0x5579315e80c0 msgr2=0x55793160a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:56.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.056+0000 7fbf59c3ab80 1 -- 172.21.15.163:0/2884350288 shutdown_connections 2023-10-09T06:38:56.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.056+0000 7fbf59c3ab80 1 -- 172.21.15.163:0/2884350288 wait complete. 2023-10-09T06:38:56.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.057+0000 7fbf59c3ab80 1 -- start start 2023-10-09T06:38:56.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.057+0000 7fbf59c3ab80 1 Processor -- start 2023-10-09T06:38:56.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.058+0000 7fbf59c3ab80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5579315e45f0 0x557931615cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:56.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.058+0000 7fbf59c3ab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579315e6b80 0x5579315e3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:56.066 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.058+0000 7fbf59c3ab80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557931631480 0x557931612a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:56.066 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.058+0000 7fbf59c3ab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x557931616200 con 0x5579315e6b80 2023-10-09T06:38:56.066 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.058+0000 7fbf59c3ab80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579316136f0 con 0x557931631480 2023-10-09T06:38:56.066 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.058+0000 7fbf59c3ab80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55793162f9c0 con 0x5579315e45f0 2023-10-09T06:38:56.066 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.058+0000 7fbf59c3ab80 10 client.0 _ll_register_callbacks cb 0x5579315aa2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-09T06:38:56.066 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.058+0000 7fbf41d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579315e6b80 0x5579315e3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:56.066 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.058+0000 7fbf41d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579315e6b80 0x5579315e3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44734/0 (socket says 172.21.15.163:44734) 2023-10-09T06:38:56.066 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.058+0000 7fbf41d9b700 1 -- 172.21.15.163:0/2478077913 learned_addr learned my addr 172.21.15.163:0/2478077913 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:56.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.058+0000 7fbf4a59c700 1 --2- 172.21.15.163:0/2478077913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5579315e45f0 0x557931615cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:56.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.058+0000 7fbf4ad9d700 1 --2- 172.21.15.163:0/2478077913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557931631480 0x557931612a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:56.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.058+0000 7fbf41d9b700 1 -- 172.21.15.163:0/2478077913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5579315e45f0 msgr2=0x557931615cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:56.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.058+0000 7fbf41d9b700 1 --2- 172.21.15.163:0/2478077913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5579315e45f0 0x557931615cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.058+0000 7fbf41d9b700 1 -- 172.21.15.163:0/2478077913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557931631480 msgr2=0x557931612a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:56.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.058+0000 7fbf41d9b700 1 --2- 172.21.15.163:0/2478077913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557931631480 0x557931612a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.058+0000 7fbf41d9b700 1 -- 172.21.15.163:0/2478077913 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5579313d1270 con 0x5579315e6b80 2023-10-09T06:38:56.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.058+0000 7fbf41d9b700 1 --2- 172.21.15.163:0/2478077913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579315e6b80 0x5579315e3c90 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fbf340019d0 tx=0x7fbf34024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4adbe4bc2b86ef server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:56.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.058+0000 7fbf28ff9700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-10-09T06:38:56.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.058+0000 7fbf28ff9700 1 -- 172.21.15.163:0/2478077913 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf34004830 con 0x5579315e6b80 2023-10-09T06:38:56.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.058+0000 7fbf28ff9700 1 -- 172.21.15.163:0/2478077913 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbf34004990 con 0x5579315e6b80 2023-10-09T06:38:56.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.058+0000 7fbf59c3ab80 1 -- 172.21.15.163:0/2478077913 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf34005b10 con 0x5579315e6b80 2023-10-09T06:38:56.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.059+0000 7fbf28ff9700 1 -- 172.21.15.163:0/2478077913 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf34004d60 con 0x5579315e6b80 2023-10-09T06:38:56.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.059+0000 7fbf59c3ab80 10 client.8367 fetch_fsmap learned FSMap version 112 2023-10-09T06:38:56.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.059+0000 7fbf59c3ab80 1 -- 172.21.15.163:0/2478077913 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x5579314ec470 con 0x5579315e6b80 2023-10-09T06:38:56.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.059+0000 7fbf28ff9700 1 -- 172.21.15.163:0/2478077913 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf34005b10 con 0x5579315e6b80 2023-10-09T06:38:56.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.059+0000 7fbf28ff9700 1 -- 172.21.15.163:0/2478077913 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 112) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fbf3400d640 con 0x5579315e6b80 2023-10-09T06:38:56.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.059+0000 7fbf59c3ab80 10 client.8367 fetch_fsmap finished waiting for FSMap version 112 2023-10-09T06:38:56.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.059+0000 7fbf59c3ab80 10 client.8367 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-09T06:38:56.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.059+0000 7fbf59c3ab80 20 client.8367 populate_metadata read hostname 'smithi163' 2023-10-09T06:38:56.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.059+0000 7fbf59c3ab80 1 --2- 172.21.15.163:0/2478077913 >> [v2:172.21.15.181:6835/3494162891,v1:172.21.15.181:6837/3494162891] conn(0x5579315e4db0 0x557931612fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:56.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.059+0000 7fbf59c3ab80 4 client.8367 mds_command: new command op to 8226 tid=0[{"prefix": "status"}] 2023-10-09T06:38:56.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.059+0000 7fbf59c3ab80 1 -- 172.21.15.163:0/2478077913 --> [v2:172.21.15.181:6835/3494162891,v1:172.21.15.181:6837/3494162891] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5579314d9e00 con 0x5579315e4db0 2023-10-09T06:38:56.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.059+0000 7fbf4a59c700 1 --2- 172.21.15.163:0/2478077913 >> [v2:172.21.15.181:6835/3494162891,v1:172.21.15.181:6837/3494162891] conn(0x5579315e4db0 0x557931612fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:56.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.059+0000 7fbf28ff9700 10 client.8367 ms_handle_connect on v2:172.21.15.181:6835/3494162891 2023-10-09T06:38:56.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.059+0000 7fbf4a59c700 1 --2- 172.21.15.163:0/2478077913 >> [v2:172.21.15.181:6835/3494162891,v1:172.21.15.181:6837/3494162891] conn(0x5579315e4db0 0x557931612fd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a55d026c6522ae07 server_cookie=af30adb38cf7d147 in_seq=0 out_seq=0 2023-10-09T06:38:56.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.060+0000 7fbf28ff9700 1 -- 172.21.15.163:0/2478077913 <== mds.0 v2:172.21.15.181:6835/3494162891 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x5579314d9e00 con 0x5579315e4db0 2023-10-09T06:38:56.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.060+0000 7fbf28ff9700 10 client.8367 handle_command_reply: tid=0 2023-10-09T06:38:56.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.060+0000 7fbf59c3ab80 1 client.8367 shutdown 2023-10-09T06:38:56.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.060+0000 7fbf59c3ab80 1 -- 172.21.15.163:0/2478077913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579315e6b80 msgr2=0x5579315e3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:56.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.060+0000 7fbf59c3ab80 1 --2- 172.21.15.163:0/2478077913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579315e6b80 0x5579315e3c90 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fbf340019d0 tx=0x7fbf34024000 comp rx=0 tx=0).stop 2023-10-09T06:38:56.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.060+0000 7fbf59c3ab80 1 -- 172.21.15.163:0/2478077913 shutdown_connections 2023-10-09T06:38:56.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.060+0000 7fbf59c3ab80 1 --2- 172.21.15.163:0/2478077913 >> [v2:172.21.15.181:6835/3494162891,v1:172.21.15.181:6837/3494162891] conn(0x5579315e4db0 0x557931612fd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.060+0000 7fbf59c3ab80 1 --2- 172.21.15.163:0/2478077913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x557931631480 0x557931612a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.060+0000 7fbf59c3ab80 1 --2- 172.21.15.163:0/2478077913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5579315e45f0 0x557931615cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.060+0000 7fbf28ff9700 0 client.8367 ms_handle_reset on v2:172.21.15.181:6835/3494162891 2023-10-09T06:38:56.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.060+0000 7fbf59c3ab80 1 --2- 172.21.15.163:0/2478077913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5579315e6b80 0x5579315e3c90 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.060+0000 7fbf59c3ab80 1 -- 172.21.15.163:0/2478077913 >> 172.21.15.163:0/2478077913 conn(0x5579315e80c0 msgr2=0x55793162c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:56.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.060+0000 7fbf59c3ab80 1 -- 172.21.15.163:0/2478077913 shutdown_connections 2023-10-09T06:38:56.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.060+0000 7fbf59c3ab80 1 -- 172.21.15.163:0/2478077913 wait complete. 2023-10-09T06:38:56.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.060+0000 7fbf59c3ab80 20 client.8367 trim_cache size 0 max 16384 2023-10-09T06:38:56.071 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:38:56.071 INFO:teuthology.orchestra.run.smithi163.stdout: "cluster_fsid": "2520b8f7-539f-4bae-9f17-1fce66f66ddd", 2023-10-09T06:38:56.071 INFO:teuthology.orchestra.run.smithi163.stdout: "whoami": 0, 2023-10-09T06:38:56.071 INFO:teuthology.orchestra.run.smithi163.stdout: "id": 8226, 2023-10-09T06:38:56.072 INFO:teuthology.orchestra.run.smithi163.stdout: "want_state": "up:active", 2023-10-09T06:38:56.072 INFO:teuthology.orchestra.run.smithi163.stdout: "state": "up:active", 2023-10-09T06:38:56.072 INFO:teuthology.orchestra.run.smithi163.stdout: "fs_name": "cephfs", 2023-10-09T06:38:56.072 INFO:teuthology.orchestra.run.smithi163.stdout: "rank_uptime": 5.9309524649999998, 2023-10-09T06:38:56.072 INFO:teuthology.orchestra.run.smithi163.stdout: "mdsmap_epoch": 112, 2023-10-09T06:38:56.072 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch": 164, 2023-10-09T06:38:56.072 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch_barrier": 164, 2023-10-09T06:38:56.072 INFO:teuthology.orchestra.run.smithi163.stdout: "uptime": 11.028977725000001 2023-10-09T06:38:56.072 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:38:56.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.060+0000 7fbf4b59e700 1 -- 172.21.15.163:0/2326480232 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbf2c067320 msgr2=0x7fbf2c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:56.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.060+0000 7fbf4b59e700 1 --2- 172.21.15.163:0/2326480232 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbf2c067320 0x7fbf2c0697d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c0027d0 tx=0x7fbf3c009040 comp rx=0 tx=0).stop 2023-10-09T06:38:56.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.060+0000 7fbf4b59e700 1 -- 172.21.15.163:0/2326480232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf4c15d230 msgr2=0x7fbf4c0c7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:56.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.060+0000 7fbf4b59e700 1 --2- 172.21.15.163:0/2326480232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf4c15d230 0x7fbf4c0c7450 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009de0 tx=0x7fbf44017000 comp rx=0 tx=0).stop 2023-10-09T06:38:56.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.060+0000 7fbf4b59e700 1 -- 172.21.15.163:0/2326480232 shutdown_connections 2023-10-09T06:38:56.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.060+0000 7fbf4b59e700 1 --2- 172.21.15.163:0/2326480232 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbf2c067320 0x7fbf2c0697d0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.060+0000 7fbf4b59e700 1 --2- 172.21.15.163:0/2326480232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf4c15d230 0x7fbf4c0c7450 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.060+0000 7fbf4b59e700 1 --2- 172.21.15.163:0/2326480232 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf4c159420 0x7fbf4c0c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.060+0000 7fbf4b59e700 1 --2- 172.21.15.163:0/2326480232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf4c1597f0 0x7fbf4c0c5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.061+0000 7fbf4b59e700 1 -- 172.21.15.163:0/2326480232 >> 172.21.15.163:0/2326480232 conn(0x7fbf4c0d0660 msgr2=0x7fbf4c0b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:56.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.061+0000 7fbf4b59e700 1 -- 172.21.15.163:0/2326480232 shutdown_connections 2023-10-09T06:38:56.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.061+0000 7fbf4b59e700 1 -- 172.21.15.163:0/2326480232 wait complete. 2023-10-09T06:38:56.075 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-09T06:38:56.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.189+0000 7ff160647700 1 Processor -- start 2023-10-09T06:38:56.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.189+0000 7ff160647700 1 -- start start 2023-10-09T06:38:56.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.190+0000 7ff160647700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff158159890 0x7ff15815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:56.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.190+0000 7ff160647700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff158069560 0x7ff1580699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:56.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.190+0000 7ff160647700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff15815a160 con 0x7ff158159c60 2023-10-09T06:38:56.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.190+0000 7ff160647700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff158069f20 con 0x7ff158069560 2023-10-09T06:38:56.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.190+0000 7ff160647700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1580d13a0 con 0x7ff158159890 2023-10-09T06:38:56.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.190+0000 7ff15ebe4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff158069560 0x7ff1580699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:56.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.190+0000 7ff15e3e3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff158159890 0x7ff15815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:56.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.190+0000 7ff15ebe4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff158069560 0x7ff1580699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:60468/0 (socket says 172.21.15.163:60468) 2023-10-09T06:38:56.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.190+0000 7ff15e3e3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff158159890 0x7ff15815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:46448/0 (socket says 172.21.15.163:46448) 2023-10-09T06:38:56.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.190+0000 7ff15ebe4700 1 -- 172.21.15.163:0/1154807360 learned_addr learned my addr 172.21.15.163:0/1154807360 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:56.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.190+0000 7ff15dbe2700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff158159c60 0x7ff15815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:51366/0 (socket says 172.21.15.163:51366) 2023-10-09T06:38:56.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.190+0000 7ff15d3e1700 1 -- 172.21.15.163:0/1154807360 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1949892636 0 0) 0x7ff15815a160 con 0x7ff158159c60 2023-10-09T06:38:56.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.190+0000 7ff15d3e1700 1 -- 172.21.15.163:0/1154807360 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff144003570 con 0x7ff158159c60 2023-10-09T06:38:56.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.191+0000 7ff15e3e3700 1 -- 172.21.15.163:0/1154807360 >> v2:172.21.15.181:3300/0 conn(0x7ff158069560 msgr2=0x7ff1580699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:56.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.191+0000 7ff15e3e3700 1 --2- 172.21.15.163:0/1154807360 >> v2:172.21.15.181:3300/0 conn(0x7ff158069560 0x7ff1580699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.191+0000 7ff15e3e3700 1 -- 172.21.15.163:0/1154807360 >> v1:172.21.15.181:6789/0 conn(0x7ff158159c60 legacy=0x7ff15815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:56.194 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.191+0000 7ff15e3e3700 1 -- 172.21.15.163:0/1154807360 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15815e140 con 0x7ff158159890 2023-10-09T06:38:56.194 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.191+0000 7ff15ebe4700 1 --2- 172.21.15.163:0/1154807360 >> v2:172.21.15.181:3300/0 conn(0x7ff158069560 0x7ff1580699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:38:56.194 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.191+0000 7ff15e3e3700 1 --2- 172.21.15.163:0/1154807360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff158159890 0x7ff15815abd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff1540098e0 tx=0x7ff1540091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=603e9246d6f206d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:56.194 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.191+0000 7ff15d3e1700 1 -- 172.21.15.163:0/1154807360 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff154011630 con 0x7ff158159890 2023-10-09T06:38:56.194 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.191+0000 7ff15d3e1700 1 -- 172.21.15.163:0/1154807360 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff154011790 con 0x7ff158159890 2023-10-09T06:38:56.194 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.191+0000 7ff15d3e1700 1 -- 172.21.15.163:0/1154807360 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff154011b30 con 0x7ff158159890 2023-10-09T06:38:56.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.191+0000 7ff160647700 1 -- 172.21.15.163:0/1154807360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff158159890 msgr2=0x7ff15815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:56.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.191+0000 7ff160647700 1 --2- 172.21.15.163:0/1154807360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff158159890 0x7ff15815abd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff1540098e0 tx=0x7ff1540091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.192+0000 7ff160647700 1 -- 172.21.15.163:0/1154807360 shutdown_connections 2023-10-09T06:38:56.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.192+0000 7ff160647700 1 --2- 172.21.15.163:0/1154807360 >> v2:172.21.15.181:3300/0 conn(0x7ff158069560 0x7ff1580699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.192+0000 7ff160647700 1 --2- 172.21.15.163:0/1154807360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff158159890 0x7ff15815abd0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.192+0000 7ff160647700 1 -- 172.21.15.163:0/1154807360 >> 172.21.15.163:0/1154807360 conn(0x7ff158068e20 msgr2=0x7ff1580d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:56.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.192+0000 7ff160647700 1 -- 172.21.15.163:0/1154807360 shutdown_connections 2023-10-09T06:38:56.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.192+0000 7ff160647700 1 -- 172.21.15.163:0/1154807360 wait complete. 2023-10-09T06:38:56.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.193+0000 7ff160647700 1 Processor -- start 2023-10-09T06:38:56.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.193+0000 7ff160647700 1 -- start start 2023-10-09T06:38:56.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.193+0000 7ff160647700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff158069560 0x7ff1580ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:56.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.193+0000 7ff160647700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff158159890 0x7ff1580c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:56.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.193+0000 7ff160647700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff158159c60 0x7ff1580b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:56.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.193+0000 7ff160647700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1580ce280 con 0x7ff158159890 2023-10-09T06:38:56.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.193+0000 7ff160647700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1580cd670 con 0x7ff158069560 2023-10-09T06:38:56.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.193+0000 7ff160647700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1580cd950 con 0x7ff158159c60 2023-10-09T06:38:56.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.194+0000 7ff15dbe2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff158159890 0x7ff1580c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:56.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.194+0000 7ff15e3e3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff158069560 0x7ff1580ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:56.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.194+0000 7ff15dbe2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff158159890 0x7ff1580c7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44748/0 (socket says 172.21.15.163:44748) 2023-10-09T06:38:56.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.194+0000 7ff15dbe2700 1 -- 172.21.15.163:0/3752773248 learned_addr learned my addr 172.21.15.163:0/3752773248 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:56.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.194+0000 7ff15ebe4700 1 --2- 172.21.15.163:0/3752773248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff158159c60 0x7ff1580b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:56.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.194+0000 7ff15dbe2700 1 -- 172.21.15.163:0/3752773248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff158159c60 msgr2=0x7ff1580b5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:56.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.194+0000 7ff15dbe2700 1 --2- 172.21.15.163:0/3752773248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff158159c60 0x7ff1580b5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.194+0000 7ff15dbe2700 1 -- 172.21.15.163:0/3752773248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff158069560 msgr2=0x7ff1580ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:56.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.194+0000 7ff15dbe2700 1 --2- 172.21.15.163:0/3752773248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff158069560 0x7ff1580ca530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.194+0000 7ff15dbe2700 1 -- 172.21.15.163:0/3752773248 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff150009ae0 con 0x7ff158159890 2023-10-09T06:38:56.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.194+0000 7ff15dbe2700 1 --2- 172.21.15.163:0/3752773248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff158159890 0x7ff1580c7af0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7ff14800c7c0 tx=0x7ff148007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:56.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.194+0000 7ff14e7fc700 1 -- 172.21.15.163:0/3752773248 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff148005b40 con 0x7ff158159890 2023-10-09T06:38:56.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.194+0000 7ff14e7fc700 1 -- 172.21.15.163:0/3752773248 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff148005ca0 con 0x7ff158159890 2023-10-09T06:38:56.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.194+0000 7ff14e7fc700 1 -- 172.21.15.163:0/3752773248 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff148007320 con 0x7ff158159890 2023-10-09T06:38:56.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.194+0000 7ff160647700 1 -- 172.21.15.163:0/3752773248 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff154009420 con 0x7ff158159890 2023-10-09T06:38:56.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.194+0000 7ff160647700 1 -- 172.21.15.163:0/3752773248 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1580cdf10 con 0x7ff158159890 2023-10-09T06:38:56.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.195+0000 7ff160647700 1 -- 172.21.15.163:0/3752773248 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1580ce3f0 con 0x7ff158159890 2023-10-09T06:38:56.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.196+0000 7ff14e7fc700 1 -- 172.21.15.163:0/3752773248 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff14801b3b0 con 0x7ff158159890 2023-10-09T06:38:56.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.196+0000 7ff14e7fc700 1 --2- 172.21.15.163:0/3752773248 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff13c067320 0x7ff13c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:56.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.196+0000 7ff14e7fc700 1 -- 172.21.15.163:0/3752773248 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff1480840f0 con 0x7ff158159890 2023-10-09T06:38:56.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.196+0000 7ff15e3e3700 1 --2- 172.21.15.163:0/3752773248 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff13c067320 0x7ff13c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:56.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.197+0000 7ff15e3e3700 1 --2- 172.21.15.163:0/3752773248 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff13c067320 0x7ff13c0697d0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff1540091e0 tx=0x7ff154017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:56.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.200+0000 7ff14e7fc700 1 -- 172.21.15.163:0/3752773248 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff148050d80 con 0x7ff158159890 2023-10-09T06:38:56.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.427+0000 7ff160647700 1 -- 172.21.15.163:0/3752773248 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff15804c3c0 con 0x7ff158159890 2023-10-09T06:38:56.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.427+0000 7ff14e7fc700 1 -- 172.21.15.163:0/3752773248 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v112) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff148017060 con 0x7ff158159890 2023-10-09T06:38:56.428 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-09T06:38:56.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.429+0000 7ff160647700 1 -- 172.21.15.163:0/3752773248 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff13c067320 msgr2=0x7ff13c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:56.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.429+0000 7ff160647700 1 --2- 172.21.15.163:0/3752773248 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff13c067320 0x7ff13c0697d0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff1540091e0 tx=0x7ff154017000 comp rx=0 tx=0).stop 2023-10-09T06:38:56.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.429+0000 7ff160647700 1 -- 172.21.15.163:0/3752773248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff158159890 msgr2=0x7ff1580c7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:56.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.429+0000 7ff160647700 1 --2- 172.21.15.163:0/3752773248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff158159890 0x7ff1580c7af0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7ff14800c7c0 tx=0x7ff148007dd0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.429+0000 7ff160647700 1 -- 172.21.15.163:0/3752773248 shutdown_connections 2023-10-09T06:38:56.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.429+0000 7ff160647700 1 --2- 172.21.15.163:0/3752773248 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff13c067320 0x7ff13c0697d0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.429+0000 7ff160647700 1 --2- 172.21.15.163:0/3752773248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff158159c60 0x7ff1580b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.429+0000 7ff160647700 1 --2- 172.21.15.163:0/3752773248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff158069560 0x7ff1580ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.429+0000 7ff160647700 1 --2- 172.21.15.163:0/3752773248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff158159890 0x7ff1580c7af0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:56.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.429+0000 7ff160647700 1 -- 172.21.15.163:0/3752773248 >> 172.21.15.163:0/3752773248 conn(0x7ff158068e20 msgr2=0x7ff1580b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:56.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.429+0000 7ff160647700 1 -- 172.21.15.163:0/3752773248 shutdown_connections 2023-10-09T06:38:56.431 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:56.429+0000 7ff160647700 1 -- 172.21.15.163:0/3752773248 wait complete. 2023-10-09T06:38:56.442 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-09T06:38:56.444 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-09T06:38:56.444 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi163.front.sepia.ceph.com 2023-10-09T06:38:56.444 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-09T06:38:56.444 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-09T06:38:56.444 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-09T06:38:56.444 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-09T06:38:56.444 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-09T06:38:56.444 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:38:56.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:56.445 DEBUG:teuthology.orchestra.run.smithi163:> ip addr 2023-10-09T06:38:56.460 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:38:56.460 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:38:56.460 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:38:56.460 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:38:56.460 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2023-10-09T06:38:56.460 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:38:56.460 INFO:teuthology.orchestra.run.smithi163.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:38:56.461 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:06 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:38:56.461 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f0 2023-10-09T06:38:56.461 INFO:teuthology.orchestra.run.smithi163.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:38:56.461 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:07 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:38:56.461 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f1 2023-10-09T06:38:56.461 INFO:teuthology.orchestra.run.smithi163.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:38:56.461 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5a brd ff:ff:ff:ff:ff:ff 2023-10-09T06:38:56.461 INFO:teuthology.orchestra.run.smithi163.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-09T06:38:56.461 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5b brd ff:ff:ff:ff:ff:ff 2023-10-09T06:38:56.461 INFO:teuthology.orchestra.run.smithi163.stdout: inet 172.21.15.163/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-09T06:38:56.461 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft 41987sec preferred_lft 41987sec 2023-10-09T06:38:56.461 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::ec4:7aff:fe88:7d5b/64 scope link 2023-10-09T06:38:56.461 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:38:56.462 INFO:teuthology.orchestra.run.smithi163.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-09T06:38:56.462 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 2e:01:84:b9:f4:8e brd ff:ff:ff:ff:ff:ff 2023-10-09T06:38:56.462 INFO:teuthology.orchestra.run.smithi163.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-09T06:38:56.462 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:38:56.462 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::d03c:93ff:fe9e:c8cb/64 scope link 2023-10-09T06:38:56.462 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:38:56.462 INFO:teuthology.orchestra.run.smithi163.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:38:56.462 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 2e:01:84:b9:f4:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-09T06:38:56.462 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::2c01:84ff:feb9:f48e/64 scope link 2023-10-09T06:38:56.462 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:38:56.462 INFO:teuthology.orchestra.run.smithi163.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:38:56.462 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether c6:88:3e:d7:c1:56 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-09T06:38:56.463 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::c488:3eff:fed7:c156/64 scope link 2023-10-09T06:38:56.463 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:38:56.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:56.463 DEBUG:teuthology.orchestra.run.smithi163:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-09T06:38:56.525 INFO:teuthology.orchestra.run.smithi163.stdout:1 2023-10-09T06:38:56.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:56.526 DEBUG:teuthology.orchestra.run.smithi163:> route 2023-10-09T06:38:56.583 INFO:teuthology.orchestra.run.smithi163.stdout:Kernel IP routing table 2023-10-09T06:38:56.583 INFO:teuthology.orchestra.run.smithi163.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-09T06:38:56.583 INFO:teuthology.orchestra.run.smithi163.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-09T06:38:56.583 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-09T06:38:56.583 INFO:teuthology.orchestra.run.smithi163.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-09T06:38:56.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:56.584 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-09T06:38:56.584 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-09T06:38:56.585 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-09T06:38:56.585 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-09T06:38:56.585 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-09T06:38:56.585 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-09T06:38:56.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:56.683 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2023-10-09T06:38:56.697 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-09T06:38:56.697 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-09T06:38:56.697 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:38:56.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:56.698 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set brx.0 up 2023-10-09T06:38:56.762 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:38:56.762 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-09T06:38:56.817 INFO:teuthology.orchestra.run.smithi163.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:38:56.818 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:38:56.818 DEBUG:teuthology.orchestra.run.smithi163:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-09T06:38:56.872 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-09T06:38:56.872 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-09T06:38:56.872 DEBUG:teuthology.orchestra.run.smithi163:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:38:56.952 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:38:56.953 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:38:56.971 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:38:56.971 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:38:56.972 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:38:56.972 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with new device syntax: 0@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:38:56.972 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:38:56.972 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:38:56.972 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:38:56.972 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:38:56.994 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:38:56.995 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:38:56.995 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:38:56.995 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with new device syntax: 0@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:38:56.995 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:38:56.995 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:38:56.995 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:38:56.996 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:38:56.996 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790" 2023-10-09T06:38:56.996 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790:/ 2023-10-09T06:38:56.996 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=2520b8f7-539f-4bae-9f17-1fce66f66ddd" will pass to kernel 2023-10-09T06:38:56.997 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-09T06:38:56.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:56.998 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-09T06:38:57.063 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-09T06:38:57.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.219+0000 7f2f3f207700 1 Processor -- start 2023-10-09T06:38:57.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.219+0000 7f2f3f207700 1 -- start start 2023-10-09T06:38:57.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.220+0000 7f2f3f207700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f381566e0 0x7f2f380ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:57.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.220+0000 7f2f3f207700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f380ca860 con 0x7f2f380cb900 2023-10-09T06:38:57.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.220+0000 7f2f3f207700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f380cab40 con 0x7f2f38156310 2023-10-09T06:38:57.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.220+0000 7f2f3f207700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f380cae20 con 0x7f2f381566e0 2023-10-09T06:38:57.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.220+0000 7f2f3ea06700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f2f380cb900 0x7f2f380c6830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:36330/0 (socket says 172.21.15.163:36330) 2023-10-09T06:38:57.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.220+0000 7f2f3ea06700 1 -- 172.21.15.163:0/4150121958 learned_addr learned my addr 172.21.15.163:0/4150121958 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:57.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.220+0000 7f2f3da04700 1 --2- 172.21.15.163:0/4150121958 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f381566e0 0x7f2f380ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:57.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.220+0000 7f2f3d203700 1 -- 172.21.15.163:0/4150121958 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 46618425 0 0) 0x7f2f380ca860 con 0x7f2f380cb900 2023-10-09T06:38:57.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.220+0000 7f2f3d203700 1 -- 172.21.15.163:0/4150121958 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f24003570 con 0x7f2f380cb900 2023-10-09T06:38:57.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.220+0000 7f2f3d203700 1 -- 172.21.15.163:0/4150121958 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4171748102 0 0) 0x7f2f380cab40 con 0x7f2f38156310 2023-10-09T06:38:57.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.220+0000 7f2f3d203700 1 -- 172.21.15.163:0/4150121958 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f380ca860 con 0x7f2f38156310 2023-10-09T06:38:57.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.220+0000 7f2f3d203700 1 -- 172.21.15.163:0/4150121958 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1667589376 0 0) 0x7f2f24003570 con 0x7f2f380cb900 2023-10-09T06:38:57.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.220+0000 7f2f3d203700 1 -- 172.21.15.163:0/4150121958 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f381566e0 msgr2=0x7f2f380ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:57.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.220+0000 7f2f3d203700 1 --2- 172.21.15.163:0/4150121958 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f381566e0 0x7f2f380ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:57.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.220+0000 7f2f3d203700 1 -- 172.21.15.163:0/4150121958 >> v1:172.21.15.181:6789/0 conn(0x7f2f38156310 legacy=0x7f2f380c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:57.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.220+0000 7f2f3d203700 1 -- 172.21.15.163:0/4150121958 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f380cb100 con 0x7f2f380cb900 2023-10-09T06:38:57.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.220+0000 7f2f3d203700 1 -- 172.21.15.163:0/4150121958 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f2f300017f0 con 0x7f2f380cb900 2023-10-09T06:38:57.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.220+0000 7f2f3d203700 1 -- 172.21.15.163:0/4150121958 >> v1:172.21.15.163:6789/0 conn(0x7f2f380cb900 legacy=0x7f2f380c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:57.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.221+0000 7f2f3d203700 1 --2- 172.21.15.163:0/4150121958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f24003c20 0x7f2f24004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:57.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.221+0000 7f2f3d203700 1 --2- 172.21.15.163:0/4150121958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f24004810 0x7f2f24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:57.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.221+0000 7f2f3d203700 1 --2- 172.21.15.163:0/4150121958 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f381566e0 0x7f2f240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:57.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.221+0000 7f2f3d203700 1 -- 172.21.15.163:0/4150121958 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f380cae20 con 0x7f2f24004810 2023-10-09T06:38:57.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.221+0000 7f2f3d203700 1 -- 172.21.15.163:0/4150121958 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2400bb40 con 0x7f2f24003c20 2023-10-09T06:38:57.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.221+0000 7f2f3d203700 1 -- 172.21.15.163:0/4150121958 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2400bd00 con 0x7f2f381566e0 2023-10-09T06:38:57.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.221+0000 7f2f3da04700 1 --2- 172.21.15.163:0/4150121958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f24004810 0x7f2f24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:57.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.221+0000 7f2f3e205700 1 --2- 172.21.15.163:0/4150121958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f24003c20 0x7f2f24004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:57.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.221+0000 7f2f3ea06700 1 --2- 172.21.15.163:0/4150121958 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f381566e0 0x7f2f240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:57.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.221+0000 7f2f3da04700 1 -- 172.21.15.163:0/4150121958 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f381566e0 msgr2=0x7f2f240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:57.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.221+0000 7f2f3da04700 1 --2- 172.21.15.163:0/4150121958 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f381566e0 0x7f2f240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:57.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.221+0000 7f2f3da04700 1 -- 172.21.15.163:0/4150121958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f24003c20 msgr2=0x7f2f24004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:57.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.221+0000 7f2f3da04700 1 --2- 172.21.15.163:0/4150121958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f24003c20 0x7f2f24004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:57.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.221+0000 7f2f3da04700 1 -- 172.21.15.163:0/4150121958 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f24003570 con 0x7f2f24004810 2023-10-09T06:38:57.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.221+0000 7f2f3ea06700 1 --2- 172.21.15.163:0/4150121958 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f381566e0 0x7f2f240051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:57.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.221+0000 7f2f3da04700 1 --2- 172.21.15.163:0/4150121958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f24004810 0x7f2f24004c80 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f2f38156b50 tx=0x7f2f280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97df830646aa9ab5 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:57.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.221+0000 7f2f3d203700 1 -- 172.21.15.163:0/4150121958 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2800e710 con 0x7f2f24004810 2023-10-09T06:38:57.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.221+0000 7f2f3d203700 1 -- 172.21.15.163:0/4150121958 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2800e870 con 0x7f2f24004810 2023-10-09T06:38:57.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.221+0000 7f2f3d203700 1 -- 172.21.15.163:0/4150121958 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2800ec30 con 0x7f2f24004810 2023-10-09T06:38:57.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.222+0000 7f2f3f207700 1 -- 172.21.15.163:0/4150121958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f24004810 msgr2=0x7f2f24004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:57.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.222+0000 7f2f3f207700 1 --2- 172.21.15.163:0/4150121958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f24004810 0x7f2f24004c80 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f2f38156b50 tx=0x7f2f280091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:57.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.222+0000 7f2f3ea06700 1 -- 172.21.15.163:0/4150121958 reap_dead start 2023-10-09T06:38:57.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.222+0000 7f2f3f207700 1 -- 172.21.15.163:0/4150121958 shutdown_connections 2023-10-09T06:38:57.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.222+0000 7f2f3f207700 1 -- 172.21.15.163:0/4150121958 >> 172.21.15.163:0/4150121958 conn(0x7f2f380cc1f0 msgr2=0x7f2f380cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:57.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.222+0000 7f2f3f207700 1 -- 172.21.15.163:0/4150121958 shutdown_connections 2023-10-09T06:38:57.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.222+0000 7f2f3f207700 1 -- 172.21.15.163:0/4150121958 wait complete. 2023-10-09T06:38:57.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.223+0000 7f2f3f207700 1 Processor -- start 2023-10-09T06:38:57.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.223+0000 7f2f3f207700 1 -- start start 2023-10-09T06:38:57.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.224+0000 7f2f3f207700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f24004810 0x7f2f38153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:57.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.224+0000 7f2f3f207700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f381548e0 0x7f2f380c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:57.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.224+0000 7f2f3f207700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f38167f00 0x7f2f380c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:57.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.224+0000 7f2f3f207700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f38151650 con 0x7f2f24004810 2023-10-09T06:38:57.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.224+0000 7f2f3f207700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f380c5290 con 0x7f2f38167f00 2023-10-09T06:38:57.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.224+0000 7f2f3f207700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f381683c0 con 0x7f2f381548e0 2023-10-09T06:38:57.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.224+0000 7f2f3e205700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f24004810 0x7f2f38153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:57.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.224+0000 7f2f3e205700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f24004810 0x7f2f38153ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44776/0 (socket says 172.21.15.163:44776) 2023-10-09T06:38:57.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.224+0000 7f2f3e205700 1 -- 172.21.15.163:0/383174612 learned_addr learned my addr 172.21.15.163:0/383174612 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:57.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.224+0000 7f2f3da04700 1 --2- 172.21.15.163:0/383174612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f381548e0 0x7f2f380c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:57.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.224+0000 7f2f3ea06700 1 --2- 172.21.15.163:0/383174612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f38167f00 0x7f2f380c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:57.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.224+0000 7f2f3e205700 1 -- 172.21.15.163:0/383174612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f381548e0 msgr2=0x7f2f380c4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:57.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.224+0000 7f2f3e205700 1 --2- 172.21.15.163:0/383174612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f381548e0 0x7f2f380c4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:57.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.224+0000 7f2f3e205700 1 -- 172.21.15.163:0/383174612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f38167f00 msgr2=0x7f2f380c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:57.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.224+0000 7f2f3e205700 1 --2- 172.21.15.163:0/383174612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f38167f00 0x7f2f380c32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:57.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.224+0000 7f2f3e205700 1 -- 172.21.15.163:0/383174612 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f381686a0 con 0x7f2f24004810 2023-10-09T06:38:57.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.224+0000 7f2f3da04700 1 --2- 172.21.15.163:0/383174612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f381548e0 0x7f2f380c4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:57.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.224+0000 7f2f3ea06700 1 --2- 172.21.15.163:0/383174612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f38167f00 0x7f2f380c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:38:57.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.225+0000 7f2f3e205700 1 --2- 172.21.15.163:0/383174612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f24004810 0x7f2f38153ba0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f2f34002c10 tx=0x7f2f34007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:57.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.225+0000 7f2f2e7fc700 1 -- 172.21.15.163:0/383174612 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3400fa80 con 0x7f2f24004810 2023-10-09T06:38:57.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.225+0000 7f2f2e7fc700 1 -- 172.21.15.163:0/383174612 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3400fc00 con 0x7f2f24004810 2023-10-09T06:38:57.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.225+0000 7f2f3f207700 1 -- 172.21.15.163:0/383174612 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f38168980 con 0x7f2f24004810 2023-10-09T06:38:57.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.225+0000 7f2f2e7fc700 1 -- 172.21.15.163:0/383174612 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f340133f0 con 0x7f2f24004810 2023-10-09T06:38:57.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.225+0000 7f2f3f207700 1 -- 172.21.15.163:0/383174612 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f380c37e0 con 0x7f2f24004810 2023-10-09T06:38:57.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.226+0000 7f2f3f207700 1 -- 172.21.15.163:0/383174612 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f3804c3c0 con 0x7f2f24004810 2023-10-09T06:38:57.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.226+0000 7f2f2e7fc700 1 -- 172.21.15.163:0/383174612 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2f340137e0 con 0x7f2f24004810 2023-10-09T06:38:57.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.226+0000 7f2f2e7fc700 1 --2- 172.21.15.163:0/383174612 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2f1c067320 0x7f2f1c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:57.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.227+0000 7f2f2e7fc700 1 -- 172.21.15.163:0/383174612 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2f340854c0 con 0x7f2f24004810 2023-10-09T06:38:57.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.227+0000 7f2f3da04700 1 --2- 172.21.15.163:0/383174612 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2f1c067320 0x7f2f1c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:57.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.227+0000 7f2f3da04700 1 --2- 172.21.15.163:0/383174612 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2f1c067320 0x7f2f1c0697d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2f38165b40 tx=0x7f2f2801a000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:57.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.231+0000 7f2f2e7fc700 1 -- 172.21.15.163:0/383174612 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2f34052150 con 0x7f2f24004810 2023-10-09T06:38:57.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.455+0000 7f2f3f207700 1 -- 172.21.15.163:0/383174612 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2f3816b7a0 con 0x7f2f24004810 2023-10-09T06:38:57.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.455+0000 7f2f2e7fc700 1 -- 172.21.15.163:0/383174612 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v112) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2f34055770 con 0x7f2f24004810 2023-10-09T06:38:57.456 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-09T06:38:57.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.457+0000 7f2f3f207700 1 -- 172.21.15.163:0/383174612 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2f1c067320 msgr2=0x7f2f1c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:57.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.457+0000 7f2f3f207700 1 --2- 172.21.15.163:0/383174612 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2f1c067320 0x7f2f1c0697d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2f38165b40 tx=0x7f2f2801a000 comp rx=0 tx=0).stop 2023-10-09T06:38:57.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.457+0000 7f2f3f207700 1 -- 172.21.15.163:0/383174612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f24004810 msgr2=0x7f2f38153ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:57.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.457+0000 7f2f3f207700 1 --2- 172.21.15.163:0/383174612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f24004810 0x7f2f38153ba0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f2f34002c10 tx=0x7f2f34007d30 comp rx=0 tx=0).stop 2023-10-09T06:38:57.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.457+0000 7f2f3f207700 1 -- 172.21.15.163:0/383174612 shutdown_connections 2023-10-09T06:38:57.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.457+0000 7f2f3f207700 1 --2- 172.21.15.163:0/383174612 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2f1c067320 0x7f2f1c0697d0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:57.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.457+0000 7f2f3f207700 1 --2- 172.21.15.163:0/383174612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f24004810 0x7f2f38153ba0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:57.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.457+0000 7f2f3f207700 1 --2- 172.21.15.163:0/383174612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f38167f00 0x7f2f380c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:57.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.457+0000 7f2f3f207700 1 --2- 172.21.15.163:0/383174612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f381548e0 0x7f2f380c4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:57.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.457+0000 7f2f3f207700 1 -- 172.21.15.163:0/383174612 >> 172.21.15.163:0/383174612 conn(0x7f2f380cc1f0 msgr2=0x7f2f380af1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:57.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.457+0000 7f2f3f207700 1 -- 172.21.15.163:0/383174612 shutdown_connections 2023-10-09T06:38:57.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:57.457+0000 7f2f3f207700 1 -- 172.21.15.163:0/383174612 wait complete. 2023-10-09T06:38:57.470 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-09T06:38:57.470 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-09T06:38:57.470 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-09T06:38:57.471 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-09T06:38:57.471 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-09T06:38:57.471 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-09T06:38:57.471 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-09T06:38:57.471 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-09T06:38:57.471 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:38:57.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:57.471 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-09T06:38:57.490 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:38:57.490 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:38:57.491 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:38:57.491 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:38:57.491 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-09T06:38:57.491 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:38:57.491 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:38:57.491 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:38:57.492 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2023-10-09T06:38:57.492 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:38:57.492 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:38:57.492 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2023-10-09T06:38:57.492 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:38:57.492 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-09T06:38:57.493 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-09T06:38:57.493 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-09T06:38:57.493 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-09T06:38:57.493 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 41963sec preferred_lft 41963sec 2023-10-09T06:38:57.493 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2023-10-09T06:38:57.494 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:38:57.494 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-09T06:38:57.494 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 72:62:e6:72:7c:2e brd ff:ff:ff:ff:ff:ff 2023-10-09T06:38:57.494 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-09T06:38:57.494 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:38:57.494 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::e89a:4cff:fefb:7198/64 scope link 2023-10-09T06:38:57.495 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:38:57.495 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:38:57.495 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 72:62:e6:72:7c:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-09T06:38:57.495 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::7062:e6ff:fe72:7c2e/64 scope link 2023-10-09T06:38:57.495 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:38:57.495 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:38:57.496 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether fa:4e:a3:9d:5a:5f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-09T06:38:57.496 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::f84e:a3ff:fe9d:5a5f/64 scope link 2023-10-09T06:38:57.496 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:38:57.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:57.497 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-09T06:38:57.557 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-09T06:38:57.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:57.559 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-09T06:38:57.616 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-09T06:38:57.616 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-09T06:38:57.617 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-09T06:38:57.617 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-09T06:38:57.617 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-09T06:38:57.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:57.619 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-09T06:38:57.619 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-09T06:38:57.619 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-09T06:38:57.619 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-09T06:38:57.620 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-09T06:38:57.620 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-09T06:38:57.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:57.720 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-09T06:38:57.735 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-09T06:38:57.736 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-09T06:38:57.736 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:38:57.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:57.736 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-09T06:38:57.800 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:38:57.800 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-09T06:38:57.855 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:38:57.855 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:38:57.856 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-09T06:38:57.910 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-09T06:38:57.910 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-09T06:38:57.910 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:38:57.990 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:38:57.990 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:38:58.008 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:38:58.009 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:38:58.009 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:38:58.009 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 1@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:38:58.009 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:38:58.009 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:38:58.009 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:38:58.009 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:38:58.034 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:38:58.034 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:38:58.034 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:38:58.034 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 1@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:38:58.034 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:38:58.034 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:38:58.035 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:38:58.035 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:38:58.035 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790" 2023-10-09T06:38:58.035 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790:/ 2023-10-09T06:38:58.035 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=2520b8f7-539f-4bae-9f17-1fce66f66ddd" will pass to kernel 2023-10-09T06:38:58.035 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-09T06:38:58.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:58.035 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-09T06:38:58.099 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-09T06:38:58.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.221+0000 7f05b3dd9700 1 Processor -- start 2023-10-09T06:38:58.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.222+0000 7f05b3dd9700 1 -- start start 2023-10-09T06:38:58.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.223+0000 7f05b3dd9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f05ac0af8c0 0x7f05ac159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:58.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.223+0000 7f05b3dd9700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ac06a0b0 con 0x7f05ac068390 2023-10-09T06:38:58.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.223+0000 7f05b3dd9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ac068900 con 0x7f05ac06ab40 2023-10-09T06:38:58.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.223+0000 7f05b3dd9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05ac068be0 con 0x7f05ac0af8c0 2023-10-09T06:38:58.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.223+0000 7f05b1374700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f05ac068390 0x7f05ac0c10c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:36332/0 (socket says 172.21.15.163:36332) 2023-10-09T06:38:58.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.223+0000 7f05b1374700 1 -- 172.21.15.163:0/833987437 learned_addr learned my addr 172.21.15.163:0/833987437 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:58.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.223+0000 7f05b2376700 1 --2- 172.21.15.163:0/833987437 >> v2:172.21.15.181:3300/0 conn(0x7f05ac0af8c0 0x7f05ac159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:58.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.223+0000 7f05b0b73700 1 -- 172.21.15.163:0/833987437 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2875027151 0 0) 0x7f05ac06a0b0 con 0x7f05ac068390 2023-10-09T06:38:58.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.223+0000 7f05b0b73700 1 -- 172.21.15.163:0/833987437 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0598003570 con 0x7f05ac068390 2023-10-09T06:38:58.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.223+0000 7f05b0b73700 1 -- 172.21.15.163:0/833987437 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2000102713 0 0) 0x7f05ac068900 con 0x7f05ac06ab40 2023-10-09T06:38:58.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.223+0000 7f05b0b73700 1 -- 172.21.15.163:0/833987437 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05ac06a0b0 con 0x7f05ac06ab40 2023-10-09T06:38:58.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.223+0000 7f05b2376700 1 -- 172.21.15.163:0/833987437 >> v1:172.21.15.181:6789/0 conn(0x7f05ac06ab40 legacy=0x7f05ac0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:58.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.223+0000 7f05b2376700 1 -- 172.21.15.163:0/833987437 >> v1:172.21.15.163:6789/0 conn(0x7f05ac068390 legacy=0x7f05ac0c10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:58.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.223+0000 7f05b2376700 1 -- 172.21.15.163:0/833987437 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ac068ec0 con 0x7f05ac0af8c0 2023-10-09T06:38:58.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.223+0000 7f05b2376700 1 --2- 172.21.15.163:0/833987437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05ac0af8c0 0x7f05ac159620 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f05a40098e0 tx=0x7f05a40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b4bdee176fd9866 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:58.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.223+0000 7f05b0b73700 1 -- 172.21.15.163:0/833987437 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a4011630 con 0x7f05ac0af8c0 2023-10-09T06:38:58.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.223+0000 7f05b0b73700 1 -- 172.21.15.163:0/833987437 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a4011790 con 0x7f05ac0af8c0 2023-10-09T06:38:58.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.223+0000 7f05b0b73700 1 -- 172.21.15.163:0/833987437 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a4011b30 con 0x7f05ac0af8c0 2023-10-09T06:38:58.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.224+0000 7f05b3dd9700 1 -- 172.21.15.163:0/833987437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05ac0af8c0 msgr2=0x7f05ac159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:58.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.224+0000 7f05b3dd9700 1 --2- 172.21.15.163:0/833987437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05ac0af8c0 0x7f05ac159620 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f05a40098e0 tx=0x7f05a40091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:58.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.224+0000 7f05b3dd9700 1 -- 172.21.15.163:0/833987437 shutdown_connections 2023-10-09T06:38:58.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.224+0000 7f05b3dd9700 1 --2- 172.21.15.163:0/833987437 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05ac0af8c0 0x7f05ac159620 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:58.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.224+0000 7f05b3dd9700 1 -- 172.21.15.163:0/833987437 >> 172.21.15.163:0/833987437 conn(0x7f05ac0cd640 msgr2=0x7f05ac0cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:58.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.224+0000 7f05b3dd9700 1 -- 172.21.15.163:0/833987437 shutdown_connections 2023-10-09T06:38:58.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.224+0000 7f05b3dd9700 1 -- 172.21.15.163:0/833987437 wait complete. 2023-10-09T06:38:58.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.225+0000 7f05b3dd9700 1 Processor -- start 2023-10-09T06:38:58.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.225+0000 7f05b3dd9700 1 -- start start 2023-10-09T06:38:58.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.226+0000 7f05b3dd9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05ac068390 0x7f05ac15cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:58.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.226+0000 7f05b3dd9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f05ac06ab40 0x7f05ac0c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:58.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.226+0000 7f05b3dd9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05ac0af8c0 0x7f05ac158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:58.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.226+0000 7f05b3dd9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac15d470 con 0x7f05ac0af8c0 2023-10-09T06:38:58.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.226+0000 7f05b3dd9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac159ce0 con 0x7f05ac068390 2023-10-09T06:38:58.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.226+0000 7f05b3dd9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac1593f0 con 0x7f05ac06ab40 2023-10-09T06:38:58.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.226+0000 7f05b2376700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05ac0af8c0 0x7f05ac158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:58.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.226+0000 7f05b2376700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05ac0af8c0 0x7f05ac158460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44790/0 (socket says 172.21.15.163:44790) 2023-10-09T06:38:58.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.226+0000 7f05b2376700 1 -- 172.21.15.163:0/1662536776 learned_addr learned my addr 172.21.15.163:0/1662536776 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:58.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.226+0000 7f05b1374700 1 --2- 172.21.15.163:0/1662536776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f05ac06ab40 0x7f05ac0c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:58.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.226+0000 7f05b1b75700 1 --2- 172.21.15.163:0/1662536776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05ac068390 0x7f05ac15cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:58.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.226+0000 7f05b2376700 1 -- 172.21.15.163:0/1662536776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f05ac06ab40 msgr2=0x7f05ac0c68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:58.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.226+0000 7f05b2376700 1 --2- 172.21.15.163:0/1662536776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f05ac06ab40 0x7f05ac0c68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:58.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.226+0000 7f05b2376700 1 -- 172.21.15.163:0/1662536776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05ac068390 msgr2=0x7f05ac15cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:58.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.226+0000 7f05b2376700 1 --2- 172.21.15.163:0/1662536776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05ac068390 0x7f05ac15cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:58.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.226+0000 7f05b2376700 1 -- 172.21.15.163:0/1662536776 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a4009420 con 0x7f05ac0af8c0 2023-10-09T06:38:58.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.226+0000 7f05b1b75700 1 --2- 172.21.15.163:0/1662536776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05ac068390 0x7f05ac15cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:38:58.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.227+0000 7f05b2376700 1 --2- 172.21.15.163:0/1662536776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05ac0af8c0 0x7f05ac158460 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f05ac071430 tx=0x7f05a4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:58.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.227+0000 7f05a1ffb700 1 -- 172.21.15.163:0/1662536776 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a401f070 con 0x7f05ac0af8c0 2023-10-09T06:38:58.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.227+0000 7f05a1ffb700 1 -- 172.21.15.163:0/1662536776 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a4011900 con 0x7f05ac0af8c0 2023-10-09T06:38:58.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.227+0000 7f05b3dd9700 1 -- 172.21.15.163:0/1662536776 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ac1607e0 con 0x7f05ac0af8c0 2023-10-09T06:38:58.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.227+0000 7f05a1ffb700 1 -- 172.21.15.163:0/1662536776 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a4025420 con 0x7f05ac0af8c0 2023-10-09T06:38:58.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.227+0000 7f05b3dd9700 1 -- 172.21.15.163:0/1662536776 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ac159970 con 0x7f05ac0af8c0 2023-10-09T06:38:58.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.228+0000 7f05b3dd9700 1 -- 172.21.15.163:0/1662536776 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ac0718b0 con 0x7f05ac0af8c0 2023-10-09T06:38:58.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.228+0000 7f05a1ffb700 1 -- 172.21.15.163:0/1662536776 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f05a4023070 con 0x7f05ac0af8c0 2023-10-09T06:38:58.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.229+0000 7f05a1ffb700 1 --2- 172.21.15.163:0/1662536776 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0590067320 0x7f05900697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:58.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.229+0000 7f05a1ffb700 1 -- 172.21.15.163:0/1662536776 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f05a40861c0 con 0x7f05ac0af8c0 2023-10-09T06:38:58.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.229+0000 7f05b1b75700 1 --2- 172.21.15.163:0/1662536776 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0590067320 0x7f05900697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:58.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.229+0000 7f05b1b75700 1 --2- 172.21.15.163:0/1662536776 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0590067320 0x7f05900697d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f05a8004040 tx=0x7f05a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:58.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.233+0000 7f05a1ffb700 1 -- 172.21.15.163:0/1662536776 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05a4052e50 con 0x7f05ac0af8c0 2023-10-09T06:38:58.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:58.460+0000 7f05b3dd9700 1 -- 172.21.15.163:0/1662536776 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f05ac159e20 con 0x7f05ac0af8c0 2023-10-09T06:38:59.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.166+0000 7f05a1ffb700 1 -- 172.21.15.163:0/1662536776 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v113) v1 ==== 136+0+0 (secure 0 0 0) 0x7f05a4056470 con 0x7f05ac0af8c0 2023-10-09T06:38:59.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.168+0000 7f05b3dd9700 1 -- 172.21.15.163:0/1662536776 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0590067320 msgr2=0x7f05900697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:59.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.168+0000 7f05b3dd9700 1 --2- 172.21.15.163:0/1662536776 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0590067320 0x7f05900697d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f05a8004040 tx=0x7f05a8009040 comp rx=0 tx=0).stop 2023-10-09T06:38:59.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.168+0000 7f05b3dd9700 1 -- 172.21.15.163:0/1662536776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05ac0af8c0 msgr2=0x7f05ac158460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:59.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.168+0000 7f05b3dd9700 1 --2- 172.21.15.163:0/1662536776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05ac0af8c0 0x7f05ac158460 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f05ac071430 tx=0x7f05a4017000 comp rx=0 tx=0).stop 2023-10-09T06:38:59.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.168+0000 7f05b3dd9700 1 -- 172.21.15.163:0/1662536776 shutdown_connections 2023-10-09T06:38:59.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.168+0000 7f05b3dd9700 1 --2- 172.21.15.163:0/1662536776 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0590067320 0x7f05900697d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:59.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.168+0000 7f05b3dd9700 1 --2- 172.21.15.163:0/1662536776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05ac0af8c0 0x7f05ac158460 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:59.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.168+0000 7f05b3dd9700 1 --2- 172.21.15.163:0/1662536776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05ac068390 0x7f05ac15cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:59.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.168+0000 7f05b3dd9700 1 --2- 172.21.15.163:0/1662536776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f05ac06ab40 0x7f05ac0c68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:59.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.168+0000 7f05b3dd9700 1 -- 172.21.15.163:0/1662536776 >> 172.21.15.163:0/1662536776 conn(0x7f05ac0cd640 msgr2=0x7f05ac0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:59.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.168+0000 7f05b3dd9700 1 -- 172.21.15.163:0/1662536776 shutdown_connections 2023-10-09T06:38:59.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.168+0000 7f05b3dd9700 1 -- 172.21.15.163:0/1662536776 wait complete. 2023-10-09T06:38:59.182 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-09T06:38:59.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.298+0000 7f8362a17700 1 Processor -- start 2023-10-09T06:38:59.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.298+0000 7f8362a17700 1 -- start start 2023-10-09T06:38:59.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.298+0000 7f8362a17700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f835c159890 0x7f835c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:59.303 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.298+0000 7f8362a17700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f835c069560 0x7f835c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:59.303 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.298+0000 7f8362a17700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f835c069ee0 con 0x7f835c159c60 2023-10-09T06:38:59.303 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.298+0000 7f8362a17700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f835c15a160 con 0x7f835c159890 2023-10-09T06:38:59.303 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.298+0000 7f8362a17700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f835c0d13a0 con 0x7f835c069560 2023-10-09T06:38:59.303 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.299+0000 7f8360fb4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f835c069560 0x7f835c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:59.303 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.299+0000 7f835bfff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f835c159890 0x7f835c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:59.303 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.299+0000 7f8360fb4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f835c069560 0x7f835c0699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:33182/0 (socket says 172.21.15.163:33182) 2023-10-09T06:38:59.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.299+0000 7f8360fb4700 1 -- 172.21.15.163:0/3255657007 learned_addr learned my addr 172.21.15.163:0/3255657007 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:59.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.299+0000 7f835affd700 1 -- 172.21.15.163:0/3255657007 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2070005038 0 0) 0x7f835c069ee0 con 0x7f835c159c60 2023-10-09T06:38:59.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.299+0000 7f8360fb4700 1 -- 172.21.15.163:0/3255657007 >> v2:172.21.15.181:3300/0 conn(0x7f835c159890 msgr2=0x7f835c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:59.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.299+0000 7f8360fb4700 1 --2- 172.21.15.163:0/3255657007 >> v2:172.21.15.181:3300/0 conn(0x7f835c159890 0x7f835c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:59.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.299+0000 7f8360fb4700 1 -- 172.21.15.163:0/3255657007 >> v1:172.21.15.181:6789/0 conn(0x7f835c159c60 legacy=0x7f835c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:59.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.299+0000 7f8360fb4700 1 -- 172.21.15.163:0/3255657007 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f835c15e140 con 0x7f835c069560 2023-10-09T06:38:59.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.299+0000 7f8360fb4700 1 --2- 172.21.15.163:0/3255657007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f835c069560 0x7f835c0699a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8354009f20 tx=0x7f8354009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6ef0b6b13f3c65a server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:59.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.299+0000 7f835affd700 1 -- 172.21.15.163:0/3255657007 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835400e070 con 0x7f835c069560 2023-10-09T06:38:59.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.300+0000 7f835affd700 1 -- 172.21.15.163:0/3255657007 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f835400b9d0 con 0x7f835c069560 2023-10-09T06:38:59.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.300+0000 7f835affd700 1 -- 172.21.15.163:0/3255657007 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83540092e0 con 0x7f835c069560 2023-10-09T06:38:59.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.300+0000 7f8362a17700 1 -- 172.21.15.163:0/3255657007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f835c069560 msgr2=0x7f835c0699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:59.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.300+0000 7f8362a17700 1 --2- 172.21.15.163:0/3255657007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f835c069560 0x7f835c0699a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8354009f20 tx=0x7f8354009840 comp rx=0 tx=0).stop 2023-10-09T06:38:59.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.300+0000 7f8362a17700 1 -- 172.21.15.163:0/3255657007 shutdown_connections 2023-10-09T06:38:59.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.300+0000 7f8362a17700 1 --2- 172.21.15.163:0/3255657007 >> v2:172.21.15.181:3300/0 conn(0x7f835c159890 0x7f835c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:59.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.300+0000 7f8362a17700 1 --2- 172.21.15.163:0/3255657007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f835c069560 0x7f835c0699a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:59.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.300+0000 7f8362a17700 1 -- 172.21.15.163:0/3255657007 >> 172.21.15.163:0/3255657007 conn(0x7f835c068e20 msgr2=0x7f835c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:59.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.300+0000 7f8362a17700 1 -- 172.21.15.163:0/3255657007 shutdown_connections 2023-10-09T06:38:59.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.300+0000 7f8362a17700 1 -- 172.21.15.163:0/3255657007 wait complete. 2023-10-09T06:38:59.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.301+0000 7f8362a17700 1 Processor -- start 2023-10-09T06:38:59.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.301+0000 7f8362a17700 1 -- start start 2023-10-09T06:38:59.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.302+0000 7f8362a17700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f835c069560 0x7f835c155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:59.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.302+0000 7f8362a17700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f835c159890 0x7f835c0b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:59.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.302+0000 7f8362a17700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f835c159c60 0x7f835c0b42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:59.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.302+0000 7f8362a17700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835c0b4820 con 0x7f835c159c60 2023-10-09T06:38:59.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.302+0000 7f8362a17700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835c0ce2c0 con 0x7f835c159890 2023-10-09T06:38:59.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.302+0000 7f8362a17700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f835c15d280 con 0x7f835c069560 2023-10-09T06:38:59.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.302+0000 7f8360fb4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f835c159c60 0x7f835c0b42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:59.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.302+0000 7f8360fb4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f835c159c60 0x7f835c0b42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42326/0 (socket says 172.21.15.163:42326) 2023-10-09T06:38:59.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.302+0000 7f8360fb4700 1 -- 172.21.15.163:0/3620464105 learned_addr learned my addr 172.21.15.163:0/3620464105 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:59.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.302+0000 7f835b7fe700 1 --2- 172.21.15.163:0/3620464105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f835c159890 0x7f835c0b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:59.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.302+0000 7f835bfff700 1 --2- 172.21.15.163:0/3620464105 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f835c069560 0x7f835c155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:59.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.303+0000 7f8360fb4700 1 -- 172.21.15.163:0/3620464105 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f835c069560 msgr2=0x7f835c155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:59.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.303+0000 7f8360fb4700 1 --2- 172.21.15.163:0/3620464105 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f835c069560 0x7f835c155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:59.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.303+0000 7f8360fb4700 1 -- 172.21.15.163:0/3620464105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f835c159890 msgr2=0x7f835c0b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:59.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.303+0000 7f8360fb4700 1 --2- 172.21.15.163:0/3620464105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f835c159890 0x7f835c0b3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:59.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.303+0000 7f8360fb4700 1 -- 172.21.15.163:0/3620464105 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8354009ae0 con 0x7f835c159c60 2023-10-09T06:38:59.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.303+0000 7f8360fb4700 1 --2- 172.21.15.163:0/3620464105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f835c159c60 0x7f835c0b42e0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f835400b610 tx=0x7f8354018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:59.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.303+0000 7f8343fff700 1 -- 172.21.15.163:0/3620464105 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8354008ad0 con 0x7f835c159c60 2023-10-09T06:38:59.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.303+0000 7f8343fff700 1 -- 172.21.15.163:0/3620464105 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8354008c30 con 0x7f835c159c60 2023-10-09T06:38:59.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.303+0000 7f8362a17700 1 -- 172.21.15.163:0/3620464105 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f835c0cd990 con 0x7f835c159c60 2023-10-09T06:38:59.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.303+0000 7f8343fff700 1 -- 172.21.15.163:0/3620464105 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835401c410 con 0x7f835c159c60 2023-10-09T06:38:59.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.303+0000 7f8362a17700 1 -- 172.21.15.163:0/3620464105 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f835c0cdf50 con 0x7f835c159c60 2023-10-09T06:38:59.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.304+0000 7f8362a17700 1 -- 172.21.15.163:0/3620464105 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f835c0ce450 con 0x7f835c159c60 2023-10-09T06:38:59.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.305+0000 7f8343fff700 1 -- 172.21.15.163:0/3620464105 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8354003040 con 0x7f835c159c60 2023-10-09T06:38:59.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.305+0000 7f8343fff700 1 --2- 172.21.15.163:0/3620464105 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8348067320 0x7f83480697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:59.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.305+0000 7f8343fff700 1 -- 172.21.15.163:0/3620464105 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8354086840 con 0x7f835c159c60 2023-10-09T06:38:59.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.305+0000 7f835bfff700 1 --2- 172.21.15.163:0/3620464105 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8348067320 0x7f83480697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:59.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.306+0000 7f835bfff700 1 --2- 172.21.15.163:0/3620464105 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8348067320 0x7f83480697d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f834c009db0 tx=0x7f834c0091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:59.316 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.309+0000 7f8343fff700 1 -- 172.21.15.163:0/3620464105 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f83540533c0 con 0x7f835c159c60 2023-10-09T06:38:59.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.516+0000 7f8362a17700 1 -- 172.21.15.163:0/3620464105 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f835c04c3c0 con 0x7f8348067320 2023-10-09T06:38:59.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.517+0000 7f8343fff700 1 -- 172.21.15.163:0/3620464105 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f835c04c3c0 con 0x7f8348067320 2023-10-09T06:38:59.519 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:38:59.519 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:38:59.519 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "cephfs" 2023-10-09T06:38:59.519 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:38:59.519 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:38:59.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.519+0000 7f8362a17700 1 -- 172.21.15.163:0/3620464105 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8348067320 msgr2=0x7f83480697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:59.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.519+0000 7f8362a17700 1 --2- 172.21.15.163:0/3620464105 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8348067320 0x7f83480697d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f834c009db0 tx=0x7f834c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:38:59.520 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.519+0000 7f8362a17700 1 -- 172.21.15.163:0/3620464105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f835c159c60 msgr2=0x7f835c0b42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:59.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.519+0000 7f8362a17700 1 --2- 172.21.15.163:0/3620464105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f835c159c60 0x7f835c0b42e0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f835400b610 tx=0x7f8354018000 comp rx=0 tx=0).stop 2023-10-09T06:38:59.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.519+0000 7f8362a17700 1 -- 172.21.15.163:0/3620464105 shutdown_connections 2023-10-09T06:38:59.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.519+0000 7f8362a17700 1 --2- 172.21.15.163:0/3620464105 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8348067320 0x7f83480697d0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:59.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.519+0000 7f8362a17700 1 --2- 172.21.15.163:0/3620464105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f835c159c60 0x7f835c0b42e0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:59.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.519+0000 7f8362a17700 1 --2- 172.21.15.163:0/3620464105 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f835c069560 0x7f835c155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:59.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.519+0000 7f8362a17700 1 --2- 172.21.15.163:0/3620464105 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f835c159890 0x7f835c0b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:59.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.519+0000 7f8362a17700 1 -- 172.21.15.163:0/3620464105 >> 172.21.15.163:0/3620464105 conn(0x7f835c068e20 msgr2=0x7f835c156e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:59.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.519+0000 7f8362a17700 1 -- 172.21.15.163:0/3620464105 shutdown_connections 2023-10-09T06:38:59.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.519+0000 7f8362a17700 1 -- 172.21.15.163:0/3620464105 wait complete. 2023-10-09T06:38:59.532 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-09T06:38:59.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.649+0000 7fb62ffb3700 1 Processor -- start 2023-10-09T06:38:59.650 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.649+0000 7fb62ffb3700 1 -- start start 2023-10-09T06:38:59.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.650+0000 7fb62ffb3700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb62806aa40 0x7fb6280cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:59.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.650+0000 7fb62ffb3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb62806bac0 0x7fb62806ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:59.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.650+0000 7fb62ffb3700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6280cc210 con 0x7fb62806b350 2023-10-09T06:38:59.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.650+0000 7fb62ffb3700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6280cc4f0 con 0x7fb62806aa40 2023-10-09T06:38:59.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.650+0000 7fb62ffb3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6280cc7d0 con 0x7fb62806bac0 2023-10-09T06:38:59.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.650+0000 7fb62dd4f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb62806aa40 0x7fb6280cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:59.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.650+0000 7fb62e550700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb62806b350 0x7fb6280c81d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:50544/0 (socket says 172.21.15.163:50544) 2023-10-09T06:38:59.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.650+0000 7fb62e550700 1 -- 172.21.15.163:0/1038177183 learned_addr learned my addr 172.21.15.163:0/1038177183 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:38:59.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.650+0000 7fb62d54e700 1 --2- 172.21.15.163:0/1038177183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb62806bac0 0x7fb62806ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:59.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.650+0000 7fb62dd4f700 1 -- 172.21.15.163:0/1038177183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb62806bac0 msgr2=0x7fb62806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:38:59.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.650+0000 7fb62dd4f700 1 --2- 172.21.15.163:0/1038177183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb62806bac0 0x7fb62806ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:59.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.651+0000 7fb62dd4f700 1 -- 172.21.15.163:0/1038177183 >> v1:172.21.15.163:6789/0 conn(0x7fb62806b350 legacy=0x7fb6280c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:59.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.650+0000 7fb62cd4d700 1 -- 172.21.15.163:0/1038177183 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2676662477 0 0) 0x7fb6280cc210 con 0x7fb62806b350 2023-10-09T06:38:59.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.651+0000 7fb62dd4f700 1 -- 172.21.15.163:0/1038177183 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6280ccab0 con 0x7fb62806aa40 2023-10-09T06:38:59.653 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.651+0000 7fb62dd4f700 1 --2- 172.21.15.163:0/1038177183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb62806aa40 0x7fb6280cbc00 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fb6240098d0 tx=0x7fb624009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d640ab90867ff587 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:59.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.651+0000 7fb62cd4d700 1 -- 172.21.15.163:0/1038177183 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb624009e70 con 0x7fb62806aa40 2023-10-09T06:38:59.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.651+0000 7fb62cd4d700 1 -- 172.21.15.163:0/1038177183 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb624005c10 con 0x7fb62806aa40 2023-10-09T06:38:59.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.651+0000 7fb62cd4d700 1 -- 172.21.15.163:0/1038177183 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62400b650 con 0x7fb62806aa40 2023-10-09T06:38:59.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.651+0000 7fb62ffb3700 1 -- 172.21.15.163:0/1038177183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb62806aa40 msgr2=0x7fb6280cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:59.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.651+0000 7fb62ffb3700 1 --2- 172.21.15.163:0/1038177183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb62806aa40 0x7fb6280cbc00 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fb6240098d0 tx=0x7fb624009120 comp rx=0 tx=0).stop 2023-10-09T06:38:59.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.651+0000 7fb62ffb3700 1 -- 172.21.15.163:0/1038177183 shutdown_connections 2023-10-09T06:38:59.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.651+0000 7fb62ffb3700 1 --2- 172.21.15.163:0/1038177183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb62806aa40 0x7fb6280cbc00 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:59.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.651+0000 7fb62ffb3700 1 --2- 172.21.15.163:0/1038177183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb62806bac0 0x7fb62806ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:59.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.651+0000 7fb62ffb3700 1 -- 172.21.15.163:0/1038177183 >> 172.21.15.163:0/1038177183 conn(0x7fb6280cd670 msgr2=0x7fb6280cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:59.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.651+0000 7fb62ffb3700 1 -- 172.21.15.163:0/1038177183 shutdown_connections 2023-10-09T06:38:59.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.651+0000 7fb62ffb3700 1 -- 172.21.15.163:0/1038177183 wait complete. 2023-10-09T06:38:59.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.652+0000 7fb62ffb3700 1 Processor -- start 2023-10-09T06:38:59.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.653+0000 7fb62ffb3700 1 -- start start 2023-10-09T06:38:59.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.653+0000 7fb62ffb3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb62806aa40 0x7fb62814a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:59.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.653+0000 7fb62ffb3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb62806b350 0x7fb628142fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:59.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.653+0000 7fb62ffb3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb62806bac0 0x7fb6280b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:59.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.653+0000 7fb62ffb3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62814a960 con 0x7fb62806b350 2023-10-09T06:38:59.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.653+0000 7fb62ffb3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6280c5fd0 con 0x7fb62806bac0 2023-10-09T06:38:59.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.653+0000 7fb62ffb3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6281434f0 con 0x7fb62806aa40 2023-10-09T06:38:59.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.653+0000 7fb62d54e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb62806b350 0x7fb628142fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:59.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.653+0000 7fb62d54e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb62806b350 0x7fb628142fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42344/0 (socket says 172.21.15.163:42344) 2023-10-09T06:38:59.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.653+0000 7fb62d54e700 1 -- 172.21.15.163:0/2315460597 learned_addr learned my addr 172.21.15.163:0/2315460597 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:38:59.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.653+0000 7fb62dd4f700 1 --2- 172.21.15.163:0/2315460597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb62806aa40 0x7fb62814a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:59.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.653+0000 7fb62e550700 1 --2- 172.21.15.163:0/2315460597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb62806bac0 0x7fb6280b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:59.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.654+0000 7fb62d54e700 1 -- 172.21.15.163:0/2315460597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb62806aa40 msgr2=0x7fb62814a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:59.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.654+0000 7fb62d54e700 1 --2- 172.21.15.163:0/2315460597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb62806aa40 0x7fb62814a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:59.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.654+0000 7fb62d54e700 1 -- 172.21.15.163:0/2315460597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb62806bac0 msgr2=0x7fb6280b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:59.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.654+0000 7fb62d54e700 1 --2- 172.21.15.163:0/2315460597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb62806bac0 0x7fb6280b1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:59.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.654+0000 7fb62d54e700 1 -- 172.21.15.163:0/2315460597 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6240093c0 con 0x7fb62806b350 2023-10-09T06:38:59.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.654+0000 7fb62d54e700 1 --2- 172.21.15.163:0/2315460597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb62806b350 0x7fb628142fb0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fb61800d860 tx=0x7fb618008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:59.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.654+0000 7fb61dffb700 1 -- 172.21.15.163:0/2315460597 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6180125f0 con 0x7fb62806b350 2023-10-09T06:38:59.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.654+0000 7fb61dffb700 1 -- 172.21.15.163:0/2315460597 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb618012770 con 0x7fb62806b350 2023-10-09T06:38:59.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.654+0000 7fb62ffb3700 1 -- 172.21.15.163:0/2315460597 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6280c53c0 con 0x7fb62806b350 2023-10-09T06:38:59.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.654+0000 7fb61dffb700 1 -- 172.21.15.163:0/2315460597 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb618012b30 con 0x7fb62806b350 2023-10-09T06:38:59.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.654+0000 7fb62ffb3700 1 -- 172.21.15.163:0/2315460597 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6280c5980 con 0x7fb62806b350 2023-10-09T06:38:59.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.655+0000 7fb62ffb3700 1 -- 172.21.15.163:0/2315460597 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6280c6160 con 0x7fb62806b350 2023-10-09T06:38:59.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.655+0000 7fb61dffb700 1 -- 172.21.15.163:0/2315460597 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb61801e460 con 0x7fb62806b350 2023-10-09T06:38:59.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.656+0000 7fb61dffb700 1 --2- 172.21.15.163:0/2315460597 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb60c067320 0x7fb60c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:38:59.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.656+0000 7fb61dffb700 1 -- 172.21.15.163:0/2315460597 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb6180852c0 con 0x7fb62806b350 2023-10-09T06:38:59.659 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.656+0000 7fb62dd4f700 1 --2- 172.21.15.163:0/2315460597 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb60c067320 0x7fb60c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:38:59.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.656+0000 7fb62dd4f700 1 --2- 172.21.15.163:0/2315460597 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb60c067320 0x7fb60c0697d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb624009160 tx=0x7fb624016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:38:59.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.660+0000 7fb61dffb700 1 -- 172.21.15.163:0/2315460597 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6180559e0 con 0x7fb62806b350 2023-10-09T06:38:59.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.841+0000 7fb62ffb3700 1 -- 172.21.15.163:0/2315460597 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb62804c3c0 con 0x7fb62806b350 2023-10-09T06:38:59.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.842+0000 7fb61dffb700 1 -- 172.21.15.163:0/2315460597 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb618055570 con 0x7fb62806b350 2023-10-09T06:38:59.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.844+0000 7fb62ffb3700 1 -- 172.21.15.163:0/2315460597 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb60c067320 msgr2=0x7fb60c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:59.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.844+0000 7fb62ffb3700 1 --2- 172.21.15.163:0/2315460597 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb60c067320 0x7fb60c0697d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb624009160 tx=0x7fb624016000 comp rx=0 tx=0).stop 2023-10-09T06:38:59.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.844+0000 7fb62ffb3700 1 -- 172.21.15.163:0/2315460597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb62806b350 msgr2=0x7fb628142fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:38:59.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.844+0000 7fb62ffb3700 1 --2- 172.21.15.163:0/2315460597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb62806b350 0x7fb628142fb0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fb61800d860 tx=0x7fb618008040 comp rx=0 tx=0).stop 2023-10-09T06:38:59.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.844+0000 7fb62ffb3700 1 -- 172.21.15.163:0/2315460597 shutdown_connections 2023-10-09T06:38:59.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.844+0000 7fb62ffb3700 1 --2- 172.21.15.163:0/2315460597 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb60c067320 0x7fb60c0697d0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:59.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.844+0000 7fb62ffb3700 1 --2- 172.21.15.163:0/2315460597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb62806bac0 0x7fb6280b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:59.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.844+0000 7fb62ffb3700 1 --2- 172.21.15.163:0/2315460597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb62806aa40 0x7fb62814a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:59.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.844+0000 7fb62ffb3700 1 --2- 172.21.15.163:0/2315460597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb62806b350 0x7fb628142fb0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:38:59.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.844+0000 7fb62ffb3700 1 -- 172.21.15.163:0/2315460597 >> 172.21.15.163:0/2315460597 conn(0x7fb6280cd670 msgr2=0x7fb62814c640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:38:59.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.844+0000 7fb62ffb3700 1 -- 172.21.15.163:0/2315460597 shutdown_connections 2023-10-09T06:38:59.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:38:59.844+0000 7fb62ffb3700 1 -- 172.21.15.163:0/2315460597 wait complete. 2023-10-09T06:38:59.856 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-09T06:38:59.856 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:38:59.857 DEBUG:teuthology.orchestra.run.smithi163:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-09T06:38:59.902 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi163.front.sepia.ceph.com 2023-10-09T06:38:59.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:59.902 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-09T06:38:59.917 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-09T06:38:59.917 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:38:59.917 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-09T06:38:59.963 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-09T06:38:59.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:59.964 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-09T06:38:59.980 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi163.front.sepia.ceph.com 2023-10-09T06:38:59.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:59.980 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-09T06:38:59.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-09T06:38:59.995 INFO:teuthology.orchestra.run.smithi163.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-09T06:38:59.995 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-09T06:38:59.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:38:59.995 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-09T06:39:00.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-09T06:39:00.037 INFO:teuthology.orchestra.run.smithi181.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-09T06:39:00.037 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create another-data-pool --pg_num_min 64 2023-10-09T06:39:00.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.155+0000 7ff6eb191700 1 Processor -- start 2023-10-09T06:39:00.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.155+0000 7ff6eb191700 1 -- start start 2023-10-09T06:39:00.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.156+0000 7ff6eb191700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6e4159ec0 0x7ff6e40b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:00.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.156+0000 7ff6eb191700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e40b5b00 con 0x7ff6e40cf350 2023-10-09T06:39:00.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.156+0000 7ff6eb191700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e40b5da0 con 0x7ff6e40b53d0 2023-10-09T06:39:00.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.156+0000 7ff6eb191700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e40b6080 con 0x7ff6e4159ec0 2023-10-09T06:39:00.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.156+0000 7ff6e972e700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff6e40cf350 0x7ff6e40cd010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:50552/0 (socket says 172.21.15.163:50552) 2023-10-09T06:39:00.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.156+0000 7ff6e972e700 1 -- 172.21.15.163:0/2623445502 learned_addr learned my addr 172.21.15.163:0/2623445502 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:00.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.156+0000 7ff6e37fe700 1 -- 172.21.15.163:0/2623445502 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1909730242 0 0) 0x7ff6e40b5b00 con 0x7ff6e40cf350 2023-10-09T06:39:00.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.156+0000 7ff6e37fe700 1 -- 172.21.15.163:0/2623445502 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6d4003570 con 0x7ff6e40cf350 2023-10-09T06:39:00.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.156+0000 7ff6e8f2d700 1 --2- 172.21.15.163:0/2623445502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6e4159ec0 0x7ff6e40b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:00.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.156+0000 7ff6e37fe700 1 -- 172.21.15.163:0/2623445502 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2952124172 0 0) 0x7ff6d4003570 con 0x7ff6e40cf350 2023-10-09T06:39:00.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.156+0000 7ff6e37fe700 1 -- 172.21.15.163:0/2623445502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6e4159ec0 msgr2=0x7ff6e40b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:00.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.156+0000 7ff6e37fe700 1 --2- 172.21.15.163:0/2623445502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6e4159ec0 0x7ff6e40b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:00.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.156+0000 7ff6e37fe700 1 -- 172.21.15.163:0/2623445502 >> v1:172.21.15.181:6789/0 conn(0x7ff6e40b53d0 legacy=0x7ff6e40cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:00.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.156+0000 7ff6e37fe700 1 -- 172.21.15.163:0/2623445502 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e40cc100 con 0x7ff6e40cf350 2023-10-09T06:39:00.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.156+0000 7ff6e37fe700 1 -- 172.21.15.163:0/2623445502 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7ff6dc0017f0 con 0x7ff6e40cf350 2023-10-09T06:39:00.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.156+0000 7ff6e37fe700 1 -- 172.21.15.163:0/2623445502 >> v1:172.21.15.163:6789/0 conn(0x7ff6e40cf350 legacy=0x7ff6e40cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:00.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.157+0000 7ff6e37fe700 1 --2- 172.21.15.163:0/2623445502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6d4003a90 0x7ff6d4003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:00.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.157+0000 7ff6e37fe700 1 --2- 172.21.15.163:0/2623445502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6e4159ec0 0x7ff6d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:00.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.157+0000 7ff6e37fe700 1 --2- 172.21.15.163:0/2623445502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d4004d50 0x7ff6d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:00.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.157+0000 7ff6e37fe700 1 -- 172.21.15.163:0/2623445502 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e40b6080 con 0x7ff6d4003a90 2023-10-09T06:39:00.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.157+0000 7ff6e37fe700 1 -- 172.21.15.163:0/2623445502 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d400bb70 con 0x7ff6d4004d50 2023-10-09T06:39:00.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.157+0000 7ff6e37fe700 1 -- 172.21.15.163:0/2623445502 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d400bd30 con 0x7ff6e4159ec0 2023-10-09T06:39:00.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.157+0000 7ff6e8f2d700 1 --2- 172.21.15.163:0/2623445502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6d4003a90 0x7ff6d4003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:00.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.157+0000 7ff6e3fff700 1 --2- 172.21.15.163:0/2623445502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6e4159ec0 0x7ff6d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:00.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.157+0000 7ff6e972e700 1 --2- 172.21.15.163:0/2623445502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d4004d50 0x7ff6d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:00.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.157+0000 7ff6e8f2d700 1 -- 172.21.15.163:0/2623445502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6e4159ec0 msgr2=0x7ff6d4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:00.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.157+0000 7ff6e8f2d700 1 --2- 172.21.15.163:0/2623445502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6e4159ec0 0x7ff6d4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:00.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.157+0000 7ff6e8f2d700 1 -- 172.21.15.163:0/2623445502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d4004d50 msgr2=0x7ff6d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:00.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.157+0000 7ff6e8f2d700 1 --2- 172.21.15.163:0/2623445502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d4004d50 0x7ff6d40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:00.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.157+0000 7ff6e8f2d700 1 -- 172.21.15.163:0/2623445502 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d4003570 con 0x7ff6d4003a90 2023-10-09T06:39:00.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.157+0000 7ff6e3fff700 1 --2- 172.21.15.163:0/2623445502 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6e4159ec0 0x7ff6d4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:00.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.157+0000 7ff6e972e700 1 --2- 172.21.15.163:0/2623445502 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d4004d50 0x7ff6d40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:00.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.157+0000 7ff6e8f2d700 1 --2- 172.21.15.163:0/2623445502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6d4003a90 0x7ff6d4003ea0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8009bf0 tx=0x7ff6d8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf21848ca6b3021 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:00.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.157+0000 7ff6e37fe700 1 -- 172.21.15.163:0/2623445502 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8005a00 con 0x7ff6d4003a90 2023-10-09T06:39:00.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.157+0000 7ff6e37fe700 1 -- 172.21.15.163:0/2623445502 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d8005b60 con 0x7ff6d4003a90 2023-10-09T06:39:00.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.157+0000 7ff6e37fe700 1 -- 172.21.15.163:0/2623445502 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d800e710 con 0x7ff6d4003a90 2023-10-09T06:39:00.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.158+0000 7ff6eb191700 1 -- 172.21.15.163:0/2623445502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6d4003a90 msgr2=0x7ff6d4003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:00.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.158+0000 7ff6eb191700 1 --2- 172.21.15.163:0/2623445502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6d4003a90 0x7ff6d4003ea0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8009bf0 tx=0x7ff6d8009120 comp rx=0 tx=0).stop 2023-10-09T06:39:00.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.158+0000 7ff6e972e700 1 -- 172.21.15.163:0/2623445502 reap_dead start 2023-10-09T06:39:00.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.158+0000 7ff6eb191700 1 -- 172.21.15.163:0/2623445502 shutdown_connections 2023-10-09T06:39:00.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.158+0000 7ff6eb191700 1 -- 172.21.15.163:0/2623445502 >> 172.21.15.163:0/2623445502 conn(0x7ff6e40999f0 msgr2=0x7ff6e40b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:00.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.158+0000 7ff6eb191700 1 -- 172.21.15.163:0/2623445502 shutdown_connections 2023-10-09T06:39:00.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.158+0000 7ff6eb191700 1 -- 172.21.15.163:0/2623445502 wait complete. 2023-10-09T06:39:00.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.159+0000 7ff6eb191700 1 Processor -- start 2023-10-09T06:39:00.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.159+0000 7ff6eb191700 1 -- start start 2023-10-09T06:39:00.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.160+0000 7ff6eb191700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d4003a90 0x7ff6e4152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:00.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.160+0000 7ff6eb191700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6e40c9910 0x7ff6e41516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:00.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.160+0000 7ff6eb191700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6e4069400 0x7ff6e4069870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:00.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.160+0000 7ff6eb191700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e4152d90 con 0x7ff6e4069400 2023-10-09T06:39:00.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.160+0000 7ff6eb191700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e40c9480 con 0x7ff6d4003a90 2023-10-09T06:39:00.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.160+0000 7ff6eb191700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e409df50 con 0x7ff6e40c9910 2023-10-09T06:39:00.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.160+0000 7ff6e972e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6e4069400 0x7ff6e4069870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:00.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.160+0000 7ff6e972e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6e4069400 0x7ff6e4069870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42356/0 (socket says 172.21.15.163:42356) 2023-10-09T06:39:00.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.160+0000 7ff6e972e700 1 -- 172.21.15.163:0/1144618934 learned_addr learned my addr 172.21.15.163:0/1144618934 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:00.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.160+0000 7ff6e8f2d700 1 --2- 172.21.15.163:0/1144618934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d4003a90 0x7ff6e4152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:00.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.160+0000 7ff6e3fff700 1 --2- 172.21.15.163:0/1144618934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6e40c9910 0x7ff6e41516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:00.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.160+0000 7ff6e972e700 1 -- 172.21.15.163:0/1144618934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6e40c9910 msgr2=0x7ff6e41516c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:00.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.160+0000 7ff6e972e700 1 --2- 172.21.15.163:0/1144618934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6e40c9910 0x7ff6e41516c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:00.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.160+0000 7ff6e972e700 1 -- 172.21.15.163:0/1144618934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d4003a90 msgr2=0x7ff6e4152850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:00.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.160+0000 7ff6e972e700 1 --2- 172.21.15.163:0/1144618934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d4003a90 0x7ff6e4152850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:00.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.160+0000 7ff6e972e700 1 -- 172.21.15.163:0/1144618934 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e409e230 con 0x7ff6e4069400 2023-10-09T06:39:00.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.160+0000 7ff6e972e700 1 --2- 172.21.15.163:0/1144618934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6e4069400 0x7ff6e4069870 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc001370 tx=0x7ff6dc002160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:00.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.160+0000 7ff6e0ff9700 1 -- 172.21.15.163:0/1144618934 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc00eb10 con 0x7ff6e4069400 2023-10-09T06:39:00.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.160+0000 7ff6e0ff9700 1 -- 172.21.15.163:0/1144618934 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6dc00ec90 con 0x7ff6e4069400 2023-10-09T06:39:00.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.160+0000 7ff6eb191700 1 -- 172.21.15.163:0/1144618934 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e40c9110 con 0x7ff6e4069400 2023-10-09T06:39:00.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.160+0000 7ff6e0ff9700 1 -- 172.21.15.163:0/1144618934 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc013420 con 0x7ff6e4069400 2023-10-09T06:39:00.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.160+0000 7ff6eb191700 1 -- 172.21.15.163:0/1144618934 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e40c8de0 con 0x7ff6e4069400 2023-10-09T06:39:00.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.161+0000 7ff6eb191700 1 -- 172.21.15.163:0/1144618934 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e40c5cc0 con 0x7ff6e4069400 2023-10-09T06:39:00.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.162+0000 7ff6e0ff9700 1 -- 172.21.15.163:0/1144618934 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff6dc013790 con 0x7ff6e4069400 2023-10-09T06:39:00.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.162+0000 7ff6e0ff9700 1 --2- 172.21.15.163:0/1144618934 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff6c8067320 0x7ff6c80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:00.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.162+0000 7ff6e0ff9700 1 -- 172.21.15.163:0/1144618934 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff6dc0851b0 con 0x7ff6e4069400 2023-10-09T06:39:00.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.162+0000 7ff6e8f2d700 1 --2- 172.21.15.163:0/1144618934 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff6c8067320 0x7ff6c80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:00.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.163+0000 7ff6e8f2d700 1 --2- 172.21.15.163:0/1144618934 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff6c8067320 0x7ff6c80697d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8009160 tx=0x7ff6d8019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:00.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.166+0000 7ff6e0ff9700 1 -- 172.21.15.163:0/1144618934 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6dc051e40 con 0x7ff6e4069400 2023-10-09T06:39:00.349 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:00.348+0000 7ff6eb191700 1 -- 172.21.15.163:0/1144618934 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "another-data-pool", "pg_num_min": 64} v 0) v1 -- 0x7ff6e4063c90 con 0x7ff6e4069400 2023-10-09T06:39:01.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.178+0000 7ff6e0ff9700 1 -- 172.21.15.163:0/1144618934 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "another-data-pool", "pg_num_min": 64}]=0 pool 'another-data-pool' created v165) v1 ==== 142+0+0 (secure 0 0 0) 0x7ff6dc055460 con 0x7ff6e4069400 2023-10-09T06:39:01.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.183+0000 7ff6eb191700 1 -- 172.21.15.163:0/1144618934 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff6c8067320 msgr2=0x7ff6c80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:01.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.183+0000 7ff6eb191700 1 --2- 172.21.15.163:0/1144618934 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff6c8067320 0x7ff6c80697d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8009160 tx=0x7ff6d8019000 comp rx=0 tx=0).stop 2023-10-09T06:39:01.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.183+0000 7ff6eb191700 1 -- 172.21.15.163:0/1144618934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6e4069400 msgr2=0x7ff6e4069870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:01.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.183+0000 7ff6eb191700 1 --2- 172.21.15.163:0/1144618934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6e4069400 0x7ff6e4069870 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc001370 tx=0x7ff6dc002160 comp rx=0 tx=0).stop 2023-10-09T06:39:01.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.183+0000 7ff6eb191700 1 -- 172.21.15.163:0/1144618934 shutdown_connections 2023-10-09T06:39:01.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.183+0000 7ff6eb191700 1 --2- 172.21.15.163:0/1144618934 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff6c8067320 0x7ff6c80697d0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:01.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.183+0000 7ff6eb191700 1 --2- 172.21.15.163:0/1144618934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6e4069400 0x7ff6e4069870 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:01.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.183+0000 7ff6eb191700 1 --2- 172.21.15.163:0/1144618934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d4003a90 0x7ff6e4152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:01.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.183+0000 7ff6eb191700 1 --2- 172.21.15.163:0/1144618934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6e40c9910 0x7ff6e41516c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:01.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.184+0000 7ff6eb191700 1 -- 172.21.15.163:0/1144618934 >> 172.21.15.163:0/1144618934 conn(0x7ff6e40999f0 msgr2=0x7ff6e40d1100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:01.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.184+0000 7ff6eb191700 1 -- 172.21.15.163:0/1144618934 shutdown_connections 2023-10-09T06:39:01.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.185+0000 7ff6eb191700 1 -- 172.21.15.163:0/1144618934 wait complete. 2023-10-09T06:39:01.187 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'another-data-pool' created 2023-10-09T06:39:01.199 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs another-data-pool 2023-10-09T06:39:01.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.316+0000 7fefd166d700 1 Processor -- start 2023-10-09T06:39:01.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.316+0000 7fefd166d700 1 -- start start 2023-10-09T06:39:01.319 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.317+0000 7fefd166d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fefcc06e3f0 0x7fefcc0af660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:01.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.317+0000 7fefd166d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefcc06ad70 con 0x7fefcc0c7780 2023-10-09T06:39:01.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.317+0000 7fefd166d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefcc06e990 con 0x7fefcc0c93d0 2023-10-09T06:39:01.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.317+0000 7fefd166d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefcc069b00 con 0x7fefcc06e3f0 2023-10-09T06:39:01.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.317+0000 7fefca7fc700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fefcc0c7780 0x7fefcc06a660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:50568/0 (socket says 172.21.15.163:50568) 2023-10-09T06:39:01.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.317+0000 7fefca7fc700 1 -- 172.21.15.163:0/3475510705 learned_addr learned my addr 172.21.15.163:0/3475510705 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:01.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.317+0000 7fefcb7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fefcc06e3f0 0x7fefcc0af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:01.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.318+0000 7fefc9ffb700 1 -- 172.21.15.163:0/3475510705 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2587099415 0 0) 0x7fefcc06ad70 con 0x7fefcc0c7780 2023-10-09T06:39:01.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.318+0000 7fefc9ffb700 1 -- 172.21.15.163:0/3475510705 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefb4003570 con 0x7fefcc0c7780 2023-10-09T06:39:01.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.318+0000 7fefcb7fe700 1 -- 172.21.15.163:0/3475510705 >> v1:172.21.15.181:6789/0 conn(0x7fefcc0c93d0 legacy=0x7fefcc069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:01.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.318+0000 7fefcb7fe700 1 -- 172.21.15.163:0/3475510705 >> v1:172.21.15.163:6789/0 conn(0x7fefcc0c7780 legacy=0x7fefcc06a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:01.320 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.318+0000 7fefcb7fe700 1 -- 172.21.15.163:0/3475510705 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefcc0c6e60 con 0x7fefcc06e3f0 2023-10-09T06:39:01.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.318+0000 7fefcb7fe700 1 --2- 172.21.15.163:0/3475510705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefcc06e3f0 0x7fefcc0af660 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fefc4009e60 tx=0x7fefc4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77dc9fac8b7d308e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:01.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.318+0000 7fefc9ffb700 1 -- 172.21.15.163:0/3475510705 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc400e070 con 0x7fefcc06e3f0 2023-10-09T06:39:01.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.318+0000 7fefc9ffb700 1 -- 172.21.15.163:0/3475510705 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc400bb60 con 0x7fefcc06e3f0 2023-10-09T06:39:01.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.318+0000 7fefc9ffb700 1 -- 172.21.15.163:0/3475510705 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc4012430 con 0x7fefcc06e3f0 2023-10-09T06:39:01.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.319+0000 7fefd166d700 1 -- 172.21.15.163:0/3475510705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefcc06e3f0 msgr2=0x7fefcc0af660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:01.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.319+0000 7fefd166d700 1 --2- 172.21.15.163:0/3475510705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefcc06e3f0 0x7fefcc0af660 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fefc4009e60 tx=0x7fefc4009780 comp rx=0 tx=0).stop 2023-10-09T06:39:01.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.319+0000 7fefd166d700 1 -- 172.21.15.163:0/3475510705 shutdown_connections 2023-10-09T06:39:01.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.319+0000 7fefd166d700 1 --2- 172.21.15.163:0/3475510705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefcc06e3f0 0x7fefcc0af660 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:01.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.319+0000 7fefd166d700 1 -- 172.21.15.163:0/3475510705 >> 172.21.15.163:0/3475510705 conn(0x7fefcc068390 msgr2=0x7fefcc068790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:01.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.319+0000 7fefd166d700 1 -- 172.21.15.163:0/3475510705 shutdown_connections 2023-10-09T06:39:01.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.319+0000 7fefd166d700 1 -- 172.21.15.163:0/3475510705 wait complete. 2023-10-09T06:39:01.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.320+0000 7fefd166d700 1 Processor -- start 2023-10-09T06:39:01.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.320+0000 7fefd166d700 1 -- start start 2023-10-09T06:39:01.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.321+0000 7fefd166d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefcc06e3f0 0x7fefcc157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:01.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.321+0000 7fefd166d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fefcc0c7780 0x7fefcc145100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:01.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.321+0000 7fefd166d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fefcc0c93d0 0x7fefcc0c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:01.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.321+0000 7fefd166d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc0c4800 con 0x7fefcc06e3f0 2023-10-09T06:39:01.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.321+0000 7fefd166d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc0c5660 con 0x7fefcc0c7780 2023-10-09T06:39:01.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.321+0000 7fefd166d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc157eb0 con 0x7fefcc0c93d0 2023-10-09T06:39:01.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.321+0000 7fefcaffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefcc06e3f0 0x7fefcc157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:01.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.321+0000 7fefcaffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefcc06e3f0 0x7fefcc157970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42366/0 (socket says 172.21.15.163:42366) 2023-10-09T06:39:01.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.321+0000 7fefcaffd700 1 -- 172.21.15.163:0/2314739649 learned_addr learned my addr 172.21.15.163:0/2314739649 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:01.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.321+0000 7fefca7fc700 1 --2- 172.21.15.163:0/2314739649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fefcc0c7780 0x7fefcc145100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:01.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.321+0000 7fefcb7fe700 1 --2- 172.21.15.163:0/2314739649 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fefcc0c93d0 0x7fefcc0c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:01.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.321+0000 7fefcaffd700 1 -- 172.21.15.163:0/2314739649 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fefcc0c93d0 msgr2=0x7fefcc0c5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:01.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.321+0000 7fefcaffd700 1 --2- 172.21.15.163:0/2314739649 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fefcc0c93d0 0x7fefcc0c5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:01.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.321+0000 7fefcaffd700 1 -- 172.21.15.163:0/2314739649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fefcc0c7780 msgr2=0x7fefcc145100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:01.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.321+0000 7fefcaffd700 1 --2- 172.21.15.163:0/2314739649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fefcc0c7780 0x7fefcc145100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:01.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.321+0000 7fefcaffd700 1 -- 172.21.15.163:0/2314739649 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc4009a20 con 0x7fefcc06e3f0 2023-10-09T06:39:01.325 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.321+0000 7fefcb7fe700 1 --2- 172.21.15.163:0/2314739649 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fefcc0c93d0 0x7fefcc0c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:01.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.321+0000 7fefcaffd700 1 --2- 172.21.15.163:0/2314739649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefcc06e3f0 0x7fefcc157970 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fefbc00c7f0 tx=0x7fefbc007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:01.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.321+0000 7fefb2ffd700 1 -- 172.21.15.163:0/2314739649 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc011590 con 0x7fefcc06e3f0 2023-10-09T06:39:01.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.321+0000 7fefb2ffd700 1 -- 172.21.15.163:0/2314739649 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefbc011710 con 0x7fefcc06e3f0 2023-10-09T06:39:01.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.321+0000 7fefb2ffd700 1 -- 172.21.15.163:0/2314739649 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc011ad0 con 0x7fefcc06e3f0 2023-10-09T06:39:01.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.321+0000 7fefd166d700 1 -- 172.21.15.163:0/2314739649 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefcc0c4a50 con 0x7fefcc06e3f0 2023-10-09T06:39:01.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.321+0000 7fefd166d700 1 -- 172.21.15.163:0/2314739649 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefcc0c5010 con 0x7fefcc06e3f0 2023-10-09T06:39:01.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.322+0000 7fefd166d700 1 -- 172.21.15.163:0/2314739649 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefcc04c3c0 con 0x7fefcc06e3f0 2023-10-09T06:39:01.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.323+0000 7fefb2ffd700 1 -- 172.21.15.163:0/2314739649 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fefbc01b3b0 con 0x7fefcc06e3f0 2023-10-09T06:39:01.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.323+0000 7fefb2ffd700 1 --2- 172.21.15.163:0/2314739649 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fefb8067320 0x7fefb80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:01.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.323+0000 7fefb2ffd700 1 -- 172.21.15.163:0/2314739649 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6077+0+0 (secure 0 0 0) 0x7fefbc0841b0 con 0x7fefcc06e3f0 2023-10-09T06:39:01.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.323+0000 7fefca7fc700 1 --2- 172.21.15.163:0/2314739649 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fefb8067320 0x7fefb80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:01.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.324+0000 7fefca7fc700 1 --2- 172.21.15.163:0/2314739649 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fefb8067320 0x7fefb80697d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fefcc153e60 tx=0x7fefc0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:01.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.328+0000 7fefb2ffd700 1 -- 172.21.15.163:0/2314739649 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fefbc050d80 con 0x7fefcc06e3f0 2023-10-09T06:39:01.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:01.555+0000 7fefd166d700 1 -- 172.21.15.163:0/2314739649 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "another-data-pool"} v 0) v1 -- 0x7fefcc0c5940 con 0x7fefcc06e3f0 2023-10-09T06:39:02.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.178+0000 7fefb2ffd700 1 -- 172.21.15.163:0/2314739649 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "another-data-pool"}]=0 added data pool 30 to fsmap v114) v1 ==== 141+0+0 (secure 0 0 0) 0x7fefbc0170b0 con 0x7fefcc06e3f0 2023-10-09T06:39:02.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.184+0000 7fefd166d700 1 -- 172.21.15.163:0/2314739649 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fefb8067320 msgr2=0x7fefb80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:02.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.184+0000 7fefd166d700 1 --2- 172.21.15.163:0/2314739649 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fefb8067320 0x7fefb80697d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fefcc153e60 tx=0x7fefc0009040 comp rx=0 tx=0).stop 2023-10-09T06:39:02.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.184+0000 7fefd166d700 1 -- 172.21.15.163:0/2314739649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefcc06e3f0 msgr2=0x7fefcc157970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:02.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.184+0000 7fefd166d700 1 --2- 172.21.15.163:0/2314739649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefcc06e3f0 0x7fefcc157970 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fefbc00c7f0 tx=0x7fefbc007f70 comp rx=0 tx=0).stop 2023-10-09T06:39:02.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.185+0000 7fefd166d700 1 -- 172.21.15.163:0/2314739649 shutdown_connections 2023-10-09T06:39:02.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.185+0000 7fefd166d700 1 --2- 172.21.15.163:0/2314739649 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fefb8067320 0x7fefb80697d0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:02.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.185+0000 7fefd166d700 1 --2- 172.21.15.163:0/2314739649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefcc06e3f0 0x7fefcc157970 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:02.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.185+0000 7fefd166d700 1 --2- 172.21.15.163:0/2314739649 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fefcc0c93d0 0x7fefcc0c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:02.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.185+0000 7fefd166d700 1 --2- 172.21.15.163:0/2314739649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fefcc0c7780 0x7fefcc145100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:02.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.185+0000 7fefd166d700 1 -- 172.21.15.163:0/2314739649 >> 172.21.15.163:0/2314739649 conn(0x7fefcc068390 msgr2=0x7fefcc0b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:02.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.186+0000 7fefd166d700 1 -- 172.21.15.163:0/2314739649 shutdown_connections 2023-10-09T06:39:02.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.186+0000 7fefd166d700 1 -- 172.21.15.163:0/2314739649 wait complete. 2023-10-09T06:39:02.190 INFO:teuthology.orchestra.run.smithi163.stderr:added data pool 30 to fsmap 2023-10-09T06:39:02.204 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:39:02.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.318+0000 7f9723bfe700 1 Processor -- start 2023-10-09T06:39:02.319 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.318+0000 7f9723bfe700 1 -- start start 2023-10-09T06:39:02.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.319+0000 7f9723bfe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f971c159c60 0x7f971c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:02.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.319+0000 7f9723bfe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f971c15db10 0x7f971c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:02.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.319+0000 7f9723bfe700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f971c15a160 con 0x7f971c159890 2023-10-09T06:39:02.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.319+0000 7f9723bfe700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f971c15df50 con 0x7f971c159c60 2023-10-09T06:39:02.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.319+0000 7f9723bfe700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f971c0d13a0 con 0x7f971c15db10 2023-10-09T06:39:02.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.319+0000 7f9721199700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f971c159c60 0x7f971c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:02.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.319+0000 7f972199a700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f971c159890 0x7f971c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:50578/0 (socket says 172.21.15.163:50578) 2023-10-09T06:39:02.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.319+0000 7f972199a700 1 -- 172.21.15.163:0/3245858967 learned_addr learned my addr 172.21.15.163:0/3245858967 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:02.321 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.319+0000 7f972219b700 1 --2- 172.21.15.163:0/3245858967 >> v2:172.21.15.181:3300/0 conn(0x7f971c15db10 0x7f971c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:02.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.319+0000 7f9721199700 1 -- 172.21.15.163:0/3245858967 >> v2:172.21.15.181:3300/0 conn(0x7f971c15db10 msgr2=0x7f971c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:02.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.319+0000 7f9721199700 1 --2- 172.21.15.163:0/3245858967 >> v2:172.21.15.181:3300/0 conn(0x7f971c15db10 0x7f971c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:02.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.319+0000 7f9721199700 1 -- 172.21.15.163:0/3245858967 >> v1:172.21.15.163:6789/0 conn(0x7f971c159890 legacy=0x7f971c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:02.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.319+0000 7f9720998700 1 -- 172.21.15.163:0/3245858967 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 71573226 0 0) 0x7f971c15a160 con 0x7f971c159890 2023-10-09T06:39:02.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.319+0000 7f9721199700 1 -- 172.21.15.163:0/3245858967 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f971c15e140 con 0x7f971c159c60 2023-10-09T06:39:02.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.319+0000 7f972219b700 1 --2- 172.21.15.163:0/3245858967 >> v2:172.21.15.181:3300/0 conn(0x7f971c15db10 0x7f971c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:02.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.319+0000 7f9721199700 1 --2- 172.21.15.163:0/3245858967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f971c159c60 0x7f971c15d5d0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f971c06a890 tx=0x7f970c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525a3b067a4abe73 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:02.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.319+0000 7f9720998700 1 -- 172.21.15.163:0/3245858967 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c00b6b0 con 0x7f971c159c60 2023-10-09T06:39:02.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.319+0000 7f9720998700 1 -- 172.21.15.163:0/3245858967 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f970c00b810 con 0x7f971c159c60 2023-10-09T06:39:02.322 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.319+0000 7f9720998700 1 -- 172.21.15.163:0/3245858967 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c00bbb0 con 0x7f971c159c60 2023-10-09T06:39:02.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.320+0000 7f9723bfe700 1 -- 172.21.15.163:0/3245858967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f971c159c60 msgr2=0x7f971c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:02.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.320+0000 7f9723bfe700 1 --2- 172.21.15.163:0/3245858967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f971c159c60 0x7f971c15d5d0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f971c06a890 tx=0x7f970c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:02.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.320+0000 7f9723bfe700 1 -- 172.21.15.163:0/3245858967 shutdown_connections 2023-10-09T06:39:02.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.320+0000 7f9723bfe700 1 --2- 172.21.15.163:0/3245858967 >> v2:172.21.15.181:3300/0 conn(0x7f971c15db10 0x7f971c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:02.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.320+0000 7f9723bfe700 1 --2- 172.21.15.163:0/3245858967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f971c159c60 0x7f971c15d5d0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:02.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.320+0000 7f9723bfe700 1 -- 172.21.15.163:0/3245858967 >> 172.21.15.163:0/3245858967 conn(0x7f971c068e20 msgr2=0x7f971c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:02.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.320+0000 7f9723bfe700 1 -- 172.21.15.163:0/3245858967 shutdown_connections 2023-10-09T06:39:02.323 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.320+0000 7f9723bfe700 1 -- 172.21.15.163:0/3245858967 wait complete. 2023-10-09T06:39:02.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.321+0000 7f9723bfe700 1 Processor -- start 2023-10-09T06:39:02.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.321+0000 7f9723bfe700 1 -- start start 2023-10-09T06:39:02.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.322+0000 7f9723bfe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f971c159890 0x7f971c14a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:02.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.322+0000 7f9723bfe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f971c159c60 0x7f971c151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:02.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.322+0000 7f9723bfe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f971c15db10 0x7f971c14ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:02.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.322+0000 7f9723bfe700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c0b12b0 con 0x7f971c159890 2023-10-09T06:39:02.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.322+0000 7f9723bfe700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c0b06c0 con 0x7f971c159c60 2023-10-09T06:39:02.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.322+0000 7f9723bfe700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f971c0b0980 con 0x7f971c15db10 2023-10-09T06:39:02.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.322+0000 7f972199a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f971c159890 0x7f971c14a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:02.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.322+0000 7f972199a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f971c159890 0x7f971c14a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42380/0 (socket says 172.21.15.163:42380) 2023-10-09T06:39:02.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.322+0000 7f972199a700 1 -- 172.21.15.163:0/2155256481 learned_addr learned my addr 172.21.15.163:0/2155256481 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:02.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.322+0000 7f972219b700 1 --2- 172.21.15.163:0/2155256481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f971c15db10 0x7f971c14ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:02.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.322+0000 7f9721199700 1 --2- 172.21.15.163:0/2155256481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f971c159c60 0x7f971c151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:02.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.322+0000 7f972199a700 1 -- 172.21.15.163:0/2155256481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f971c15db10 msgr2=0x7f971c14ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:02.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.322+0000 7f972199a700 1 --2- 172.21.15.163:0/2155256481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f971c15db10 0x7f971c14ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:02.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.322+0000 7f972199a700 1 -- 172.21.15.163:0/2155256481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f971c159c60 msgr2=0x7f971c151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:02.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.322+0000 7f972199a700 1 --2- 172.21.15.163:0/2155256481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f971c159c60 0x7f971c151d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:02.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.322+0000 7f972199a700 1 -- 172.21.15.163:0/2155256481 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970c009420 con 0x7f971c159890 2023-10-09T06:39:02.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.322+0000 7f9721199700 1 --2- 172.21.15.163:0/2155256481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f971c159c60 0x7f971c151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:02.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.322+0000 7f972219b700 1 --2- 172.21.15.163:0/2155256481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f971c15db10 0x7f971c14ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:02.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.322+0000 7f972199a700 1 --2- 172.21.15.163:0/2155256481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f971c159890 0x7f971c14a460 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f971800c7e0 tx=0x7f9718007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:02.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.322+0000 7f9711ffb700 1 -- 172.21.15.163:0/2155256481 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9718005b30 con 0x7f971c159890 2023-10-09T06:39:02.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.322+0000 7f9711ffb700 1 -- 172.21.15.163:0/2155256481 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9718005c90 con 0x7f971c159890 2023-10-09T06:39:02.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.323+0000 7f9711ffb700 1 -- 172.21.15.163:0/2155256481 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9718007300 con 0x7f971c159890 2023-10-09T06:39:02.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.323+0000 7f9723bfe700 1 -- 172.21.15.163:0/2155256481 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f971c0b0c60 con 0x7f971c159890 2023-10-09T06:39:02.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.323+0000 7f9723bfe700 1 -- 172.21.15.163:0/2155256481 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f971c14a9a0 con 0x7f971c159890 2023-10-09T06:39:02.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.324+0000 7f9723bfe700 1 -- 172.21.15.163:0/2155256481 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f971c04c3c0 con 0x7f971c159890 2023-10-09T06:39:02.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.324+0000 7f9711ffb700 1 -- 172.21.15.163:0/2155256481 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f971801b400 con 0x7f971c159890 2023-10-09T06:39:02.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.324+0000 7f9711ffb700 1 --2- 172.21.15.163:0/2155256481 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9700067320 0x7f97000697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:02.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.324+0000 7f9711ffb700 1 -- 172.21.15.163:0/2155256481 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6109+0+0 (secure 0 0 0) 0x7f9718084970 con 0x7f971c159890 2023-10-09T06:39:02.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.325+0000 7f9721199700 1 --2- 172.21.15.163:0/2155256481 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9700067320 0x7f97000697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:02.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.325+0000 7f9721199700 1 --2- 172.21.15.163:0/2155256481 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9700067320 0x7f97000697d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f970c0093f0 tx=0x7f970c017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:02.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.328+0000 7f9711ffb700 1 -- 172.21.15.163:0/2155256481 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97180514f0 con 0x7f971c159890 2023-10-09T06:39:02.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.552+0000 7f9723bfe700 1 -- 172.21.15.163:0/2155256481 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f971c06a5b0 con 0x7f971c159890 2023-10-09T06:39:02.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.552+0000 7f9711ffb700 1 -- 172.21.15.163:0/2155256481 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+4622 (secure 0 0 0) 0x7f9718054b10 con 0x7f971c159890 2023-10-09T06:39:02.554 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:02.554 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":114,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8211,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3056573272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3056573272},{"type":"v1","addr":"172.21.15.163:6834","nonce":3056573272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":8214,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3482861499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3482861499},{"type":"v1","addr":"172.21.15.163:6835","nonce":3482861499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":8223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/4239124825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4239124825},{"type":"v1","addr":"172.21.15.181:6836","nonce":4239124825}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:38:49.480715+0000","modified":"2023-10-09T06:39:02.175673+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8226":{"gid":8226,"name":"b","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/3494162891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3494162891},{"type":"v1","addr":"172.21.15.181:6837","nonce":3494162891}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29,30],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-09T06:39:02.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.554+0000 7f9723bfe700 1 -- 172.21.15.163:0/2155256481 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9700067320 msgr2=0x7f97000697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:02.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.554+0000 7f9723bfe700 1 --2- 172.21.15.163:0/2155256481 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9700067320 0x7f97000697d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f970c0093f0 tx=0x7f970c017000 comp rx=0 tx=0).stop 2023-10-09T06:39:02.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.554+0000 7f9723bfe700 1 -- 172.21.15.163:0/2155256481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f971c159890 msgr2=0x7f971c14a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:02.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.554+0000 7f9723bfe700 1 --2- 172.21.15.163:0/2155256481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f971c159890 0x7f971c14a460 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f971800c7e0 tx=0x7f9718007df0 comp rx=0 tx=0).stop 2023-10-09T06:39:02.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.554+0000 7f9723bfe700 1 -- 172.21.15.163:0/2155256481 shutdown_connections 2023-10-09T06:39:02.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.554+0000 7f9723bfe700 1 --2- 172.21.15.163:0/2155256481 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9700067320 0x7f97000697d0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:02.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.554+0000 7f9723bfe700 1 --2- 172.21.15.163:0/2155256481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f971c159890 0x7f971c14a460 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:02.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.554+0000 7f9723bfe700 1 --2- 172.21.15.163:0/2155256481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f971c15db10 0x7f971c14ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:02.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.554+0000 7f9723bfe700 1 --2- 172.21.15.163:0/2155256481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f971c159c60 0x7f971c151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:02.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.554+0000 7f9723bfe700 1 -- 172.21.15.163:0/2155256481 >> 172.21.15.163:0/2155256481 conn(0x7f971c068e20 msgr2=0x7f971c1579b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:02.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.555+0000 7f9723bfe700 1 -- 172.21.15.163:0/2155256481 shutdown_connections 2023-10-09T06:39:02.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.555+0000 7f9723bfe700 1 -- 172.21.15.163:0/2155256481 wait complete. 2023-10-09T06:39:02.558 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 114 2023-10-09T06:39:02.568 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:39:02.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.681+0000 7f905fe24700 1 Processor -- start 2023-10-09T06:39:02.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.681+0000 7f905fe24700 1 -- start start 2023-10-09T06:39:02.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.682+0000 7f905fe24700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f905806c0a0 0x7f90580caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:02.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.682+0000 7f905fe24700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f905815df50 0x7f905815e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:02.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.682+0000 7f905fe24700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f905815d7e0 con 0x7f905806bcd0 2023-10-09T06:39:02.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.682+0000 7f905fe24700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f905815dac0 con 0x7f905806c0a0 2023-10-09T06:39:02.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.682+0000 7f905fe24700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f905815aa30 con 0x7f905815df50 2023-10-09T06:39:02.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.682+0000 7f905d3bf700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f905806c0a0 0x7f90580caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:02.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.682+0000 7f905d3bf700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f905806c0a0 0x7f90580caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42382/0 (socket says 172.21.15.163:42382) 2023-10-09T06:39:02.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.682+0000 7f905d3bf700 1 -- 172.21.15.163:0/3021654978 learned_addr learned my addr 172.21.15.163:0/3021654978 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:02.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.683+0000 7f905e3c1700 1 --2- 172.21.15.163:0/3021654978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f905815df50 0x7f905815e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:02.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.683+0000 7f905d3bf700 1 -- 172.21.15.163:0/3021654978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f905815df50 msgr2=0x7f905815e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:02.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.683+0000 7f905d3bf700 1 --2- 172.21.15.163:0/3021654978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f905815df50 0x7f905815e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:02.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.683+0000 7f905d3bf700 1 -- 172.21.15.163:0/3021654978 >> v1:172.21.15.181:6789/0 conn(0x7f905806bcd0 legacy=0x7f905815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:02.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.683+0000 7f905d3bf700 1 -- 172.21.15.163:0/3021654978 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905815ad10 con 0x7f905806c0a0 2023-10-09T06:39:02.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.683+0000 7f905d3bf700 1 --2- 172.21.15.163:0/3021654978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f905806c0a0 0x7f90580caa90 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f90480099b0 tx=0x7f9048009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad287cc9c5ba306 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:02.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.683+0000 7f905cbbe700 1 -- 172.21.15.163:0/3021654978 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9048009ea0 con 0x7f905806c0a0 2023-10-09T06:39:02.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.683+0000 7f905cbbe700 1 -- 172.21.15.163:0/3021654978 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904800b860 con 0x7f905806c0a0 2023-10-09T06:39:02.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.683+0000 7f905cbbe700 1 -- 172.21.15.163:0/3021654978 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904800bc20 con 0x7f905806c0a0 2023-10-09T06:39:02.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.684+0000 7f905fe24700 1 -- 172.21.15.163:0/3021654978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f905806c0a0 msgr2=0x7f90580caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:02.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.684+0000 7f905fe24700 1 --2- 172.21.15.163:0/3021654978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f905806c0a0 0x7f90580caa90 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f90480099b0 tx=0x7f9048009250 comp rx=0 tx=0).stop 2023-10-09T06:39:02.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.684+0000 7f905fe24700 1 -- 172.21.15.163:0/3021654978 shutdown_connections 2023-10-09T06:39:02.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.684+0000 7f905fe24700 1 --2- 172.21.15.163:0/3021654978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f905815df50 0x7f905815e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:02.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.684+0000 7f905fe24700 1 --2- 172.21.15.163:0/3021654978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f905806c0a0 0x7f90580caa90 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:02.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.684+0000 7f905fe24700 1 -- 172.21.15.163:0/3021654978 >> 172.21.15.163:0/3021654978 conn(0x7f90580ce600 msgr2=0x7f90580cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:02.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.684+0000 7f905fe24700 1 -- 172.21.15.163:0/3021654978 shutdown_connections 2023-10-09T06:39:02.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.684+0000 7f905fe24700 1 -- 172.21.15.163:0/3021654978 wait complete. 2023-10-09T06:39:02.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.685+0000 7f905fe24700 1 Processor -- start 2023-10-09T06:39:02.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.685+0000 7f905fe24700 1 -- start start 2023-10-09T06:39:02.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.685+0000 7f905fe24700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f905806bcd0 0x7f90581440f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:02.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.685+0000 7f905fe24700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f905806c0a0 0x7f9058069480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:02.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.685+0000 7f905fe24700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f905815df50 0x7f90580699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:02.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.685+0000 7f905fe24700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905816b1a0 con 0x7f905815df50 2023-10-09T06:39:02.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.685+0000 7f905fe24700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90580c7730 con 0x7f905806c0a0 2023-10-09T06:39:02.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.685+0000 7f905fe24700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90580c7a10 con 0x7f905806bcd0 2023-10-09T06:39:02.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.686+0000 7f905e3c1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f905815df50 0x7f90580699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:02.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.686+0000 7f905e3c1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f905815df50 0x7f90580699c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42396/0 (socket says 172.21.15.163:42396) 2023-10-09T06:39:02.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.686+0000 7f905e3c1700 1 -- 172.21.15.163:0/1985732579 learned_addr learned my addr 172.21.15.163:0/1985732579 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:02.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.686+0000 7f905d3bf700 1 --2- 172.21.15.163:0/1985732579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f905806c0a0 0x7f9058069480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:02.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.686+0000 7f905dbc0700 1 --2- 172.21.15.163:0/1985732579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f905806bcd0 0x7f90581440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:02.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.686+0000 7f905e3c1700 1 -- 172.21.15.163:0/1985732579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f905806bcd0 msgr2=0x7f90581440f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:02.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.686+0000 7f905e3c1700 1 --2- 172.21.15.163:0/1985732579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f905806bcd0 0x7f90581440f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:02.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.686+0000 7f905e3c1700 1 -- 172.21.15.163:0/1985732579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f905806c0a0 msgr2=0x7f9058069480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:02.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.686+0000 7f905e3c1700 1 --2- 172.21.15.163:0/1985732579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f905806c0a0 0x7f9058069480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:02.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.686+0000 7f905e3c1700 1 -- 172.21.15.163:0/1985732579 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90480094f0 con 0x7f905815df50 2023-10-09T06:39:02.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.686+0000 7f905dbc0700 1 --2- 172.21.15.163:0/1985732579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f905806bcd0 0x7f90581440f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:02.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.686+0000 7f905e3c1700 1 --2- 172.21.15.163:0/1985732579 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f905815df50 0x7f90580699c0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f905000ea50 tx=0x7f90500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:02.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.686+0000 7f904dffb700 1 -- 172.21.15.163:0/1985732579 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905000c850 con 0x7f905815df50 2023-10-09T06:39:02.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.686+0000 7f904dffb700 1 -- 172.21.15.163:0/1985732579 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905000c9d0 con 0x7f905815df50 2023-10-09T06:39:02.696 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.686+0000 7f904dffb700 1 -- 172.21.15.163:0/1985732579 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905000cd90 con 0x7f905815df50 2023-10-09T06:39:02.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.686+0000 7f905fe24700 1 -- 172.21.15.163:0/1985732579 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90580c7cf0 con 0x7f905815df50 2023-10-09T06:39:02.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.686+0000 7f905fe24700 1 -- 172.21.15.163:0/1985732579 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9058144630 con 0x7f905815df50 2023-10-09T06:39:02.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.687+0000 7f905fe24700 1 -- 172.21.15.163:0/1985732579 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90580c7590 con 0x7f905815df50 2023-10-09T06:39:02.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.688+0000 7f904dffb700 1 -- 172.21.15.163:0/1985732579 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9050013680 con 0x7f905815df50 2023-10-09T06:39:02.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.688+0000 7f904dffb700 1 --2- 172.21.15.163:0/1985732579 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f903c067320 0x7f903c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:02.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.688+0000 7f904dffb700 1 -- 172.21.15.163:0/1985732579 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6109+0+0 (secure 0 0 0) 0x7f9050085080 con 0x7f905815df50 2023-10-09T06:39:02.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.688+0000 7f905dbc0700 1 --2- 172.21.15.163:0/1985732579 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f903c067320 0x7f903c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:02.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.689+0000 7f905dbc0700 1 --2- 172.21.15.163:0/1985732579 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f903c067320 0x7f903c0697d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f9054007fd0 tx=0x7f9054009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:02.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.692+0000 7f904dffb700 1 -- 172.21.15.163:0/1985732579 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90580c7590 con 0x7f905815df50 2023-10-09T06:39:02.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.876+0000 7f905fe24700 1 -- 172.21.15.163:0/1985732579 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90580c7590 con 0x7f905815df50 2023-10-09T06:39:02.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.877+0000 7f904dffb700 1 -- 172.21.15.163:0/1985732579 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) v1 ==== 74+0+16995 (secure 0 0 0) 0x7f905006be60 con 0x7f905815df50 2023-10-09T06:39:02.878 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:02.878 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":166,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:39:01.556835+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:38:47.806008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-10-09T06:38:48.446181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":30,"pool_name":"another-data-pool","create_time":"2023-10-09T06:39:00.367621+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:39:02.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.879+0000 7f905fe24700 1 -- 172.21.15.163:0/1985732579 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f903c067320 msgr2=0x7f903c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:02.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.879+0000 7f905fe24700 1 --2- 172.21.15.163:0/1985732579 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f903c067320 0x7f903c0697d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f9054007fd0 tx=0x7f9054009040 comp rx=0 tx=0).stop 2023-10-09T06:39:02.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.879+0000 7f905fe24700 1 -- 172.21.15.163:0/1985732579 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f905815df50 msgr2=0x7f90580699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:02.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.879+0000 7f905fe24700 1 --2- 172.21.15.163:0/1985732579 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f905815df50 0x7f90580699c0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f905000ea50 tx=0x7f90500091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:02.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.879+0000 7f905fe24700 1 -- 172.21.15.163:0/1985732579 shutdown_connections 2023-10-09T06:39:02.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.879+0000 7f905fe24700 1 --2- 172.21.15.163:0/1985732579 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f903c067320 0x7f903c0697d0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:02.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.879+0000 7f905fe24700 1 --2- 172.21.15.163:0/1985732579 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f905815df50 0x7f90580699c0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:02.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.879+0000 7f905fe24700 1 --2- 172.21.15.163:0/1985732579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f905806bcd0 0x7f90581440f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:02.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.879+0000 7f905fe24700 1 --2- 172.21.15.163:0/1985732579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f905806c0a0 0x7f9058069480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:02.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.879+0000 7f905fe24700 1 -- 172.21.15.163:0/1985732579 >> 172.21.15.163:0/1985732579 conn(0x7f90580ce600 msgr2=0x7f90580c4570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:02.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.879+0000 7f905fe24700 1 -- 172.21.15.163:0/1985732579 shutdown_connections 2023-10-09T06:39:02.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:02.879+0000 7f905fe24700 1 -- 172.21.15.163:0/1985732579 wait complete. 2023-10-09T06:39:02.892 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:39:03.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.006+0000 7f98c3a82700 1 Processor -- start 2023-10-09T06:39:03.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.007+0000 7f98c3a82700 1 -- start start 2023-10-09T06:39:03.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.007+0000 7f98c3a82700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f98bc069560 0x7f98bc0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:03.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.007+0000 7f98c3a82700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98bc069ee0 con 0x7f98bc159890 2023-10-09T06:39:03.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.007+0000 7f98c3a82700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98bc15a160 con 0x7f98bc159c60 2023-10-09T06:39:03.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.007+0000 7f98c3a82700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98bc0d13a0 con 0x7f98bc069560 2023-10-09T06:39:03.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.008+0000 7f98c181e700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f98bc159890 0x7f98bc15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:50594/0 (socket says 172.21.15.163:50594) 2023-10-09T06:39:03.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.008+0000 7f98c181e700 1 -- 172.21.15.163:0/3905182504 learned_addr learned my addr 172.21.15.163:0/3905182504 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:03.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.008+0000 7f98c201f700 1 --2- 172.21.15.163:0/3905182504 >> v2:172.21.15.181:3300/0 conn(0x7f98bc069560 0x7f98bc0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:03.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.008+0000 7f98c081c700 1 -- 172.21.15.163:0/3905182504 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 599316055 0 0) 0x7f98bc069ee0 con 0x7f98bc159890 2023-10-09T06:39:03.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.008+0000 7f98c081c700 1 -- 172.21.15.163:0/3905182504 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98a8003570 con 0x7f98bc159890 2023-10-09T06:39:03.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.008+0000 7f98c081c700 1 -- 172.21.15.163:0/3905182504 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2327455114 0 0) 0x7f98a8003570 con 0x7f98bc159890 2023-10-09T06:39:03.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.008+0000 7f98c081c700 1 -- 172.21.15.163:0/3905182504 >> v2:172.21.15.181:3300/0 conn(0x7f98bc069560 msgr2=0x7f98bc0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:03.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.008+0000 7f98c081c700 1 --2- 172.21.15.163:0/3905182504 >> v2:172.21.15.181:3300/0 conn(0x7f98bc069560 0x7f98bc0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:03.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.008+0000 7f98c081c700 1 -- 172.21.15.163:0/3905182504 >> v1:172.21.15.181:6789/0 conn(0x7f98bc159c60 legacy=0x7f98bc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:03.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.008+0000 7f98c081c700 1 -- 172.21.15.163:0/3905182504 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98bc15e140 con 0x7f98bc159890 2023-10-09T06:39:03.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.008+0000 7f98c081c700 1 -- 172.21.15.163:0/3905182504 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f98b80017f0 con 0x7f98bc159890 2023-10-09T06:39:03.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.008+0000 7f98c081c700 1 -- 172.21.15.163:0/3905182504 >> v1:172.21.15.163:6789/0 conn(0x7f98bc159890 legacy=0x7f98bc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:03.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.008+0000 7f98c081c700 1 --2- 172.21.15.163:0/3905182504 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98a8003a60 0x7f98a8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:03.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.008+0000 7f98c081c700 1 --2- 172.21.15.163:0/3905182504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98a8004810 0x7f98a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:03.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.008+0000 7f98c081c700 1 --2- 172.21.15.163:0/3905182504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98a80051c0 0x7f98a800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:03.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.008+0000 7f98c081c700 1 -- 172.21.15.163:0/3905182504 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc0d13a0 con 0x7f98a8004810 2023-10-09T06:39:03.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.008+0000 7f98c081c700 1 -- 172.21.15.163:0/3905182504 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a800c0d0 con 0x7f98a80051c0 2023-10-09T06:39:03.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.008+0000 7f98c081c700 1 -- 172.21.15.163:0/3905182504 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a800c290 con 0x7f98a8003a60 2023-10-09T06:39:03.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.008+0000 7f98c101d700 1 --2- 172.21.15.163:0/3905182504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98a8004810 0x7f98a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:03.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.008+0000 7f98c181e700 1 --2- 172.21.15.163:0/3905182504 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98a8003a60 0x7f98a8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:03.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.009+0000 7f98c201f700 1 --2- 172.21.15.163:0/3905182504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98a80051c0 0x7f98a800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:03.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.009+0000 7f98c101d700 1 -- 172.21.15.163:0/3905182504 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98a8003a60 msgr2=0x7f98a8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:03.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.009+0000 7f98c101d700 1 --2- 172.21.15.163:0/3905182504 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98a8003a60 0x7f98a8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:03.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.009+0000 7f98c101d700 1 -- 172.21.15.163:0/3905182504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98a80051c0 msgr2=0x7f98a800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:03.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.009+0000 7f98c101d700 1 --2- 172.21.15.163:0/3905182504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98a80051c0 0x7f98a800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:03.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.009+0000 7f98c101d700 1 -- 172.21.15.163:0/3905182504 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a8003570 con 0x7f98a8004810 2023-10-09T06:39:03.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.009+0000 7f98c201f700 1 --2- 172.21.15.163:0/3905182504 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98a80051c0 0x7f98a800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:03.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.009+0000 7f98c181e700 1 --2- 172.21.15.163:0/3905182504 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98a8003a60 0x7f98a8003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:03.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.009+0000 7f98c201f700 1 -- 172.21.15.163:0/3905182504 reap_dead start 2023-10-09T06:39:03.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.009+0000 7f98c101d700 1 --2- 172.21.15.163:0/3905182504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98a8004810 0x7f98a8004c80 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0018b0 tx=0x7f98ac007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9108aa355b9cf447 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:03.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.009+0000 7f98c081c700 1 -- 172.21.15.163:0/3905182504 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac012420 con 0x7f98a8004810 2023-10-09T06:39:03.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.009+0000 7f98c081c700 1 -- 172.21.15.163:0/3905182504 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ac0125a0 con 0x7f98a8004810 2023-10-09T06:39:03.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.009+0000 7f98c081c700 1 -- 172.21.15.163:0/3905182504 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac012960 con 0x7f98a8004810 2023-10-09T06:39:03.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.009+0000 7f98c3a82700 1 -- 172.21.15.163:0/3905182504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98a8004810 msgr2=0x7f98a8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:03.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.009+0000 7f98c3a82700 1 --2- 172.21.15.163:0/3905182504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98a8004810 0x7f98a8004c80 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0018b0 tx=0x7f98ac007d30 comp rx=0 tx=0).stop 2023-10-09T06:39:03.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.009+0000 7f98c3a82700 1 -- 172.21.15.163:0/3905182504 shutdown_connections 2023-10-09T06:39:03.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.009+0000 7f98c3a82700 1 --2- 172.21.15.163:0/3905182504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98a8004810 0x7f98a8004c80 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:03.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.009+0000 7f98c3a82700 1 -- 172.21.15.163:0/3905182504 >> 172.21.15.163:0/3905182504 conn(0x7f98bc068e20 msgr2=0x7f98bc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:03.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.010+0000 7f98c3a82700 1 -- 172.21.15.163:0/3905182504 shutdown_connections 2023-10-09T06:39:03.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.010+0000 7f98c3a82700 1 -- 172.21.15.163:0/3905182504 wait complete. 2023-10-09T06:39:03.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.011+0000 7f98c3a82700 1 Processor -- start 2023-10-09T06:39:03.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.011+0000 7f98c3a82700 1 -- start start 2023-10-09T06:39:03.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.011+0000 7f98c3a82700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98a8004810 0x7f98bc0cd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:03.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.011+0000 7f98c3a82700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98bc0c5590 0x7f98bc0c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:03.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.011+0000 7f98c3a82700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98bc152130 0x7f98bc1525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:03.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.011+0000 7f98c3a82700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc0cd7a0 con 0x7f98bc152130 2023-10-09T06:39:03.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.011+0000 7f98c3a82700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc151ca0 con 0x7f98a8004810 2023-10-09T06:39:03.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.011+0000 7f98c3a82700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc0c6d00 con 0x7f98bc0c5590 2023-10-09T06:39:03.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.011+0000 7f98c201f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98bc152130 0x7f98bc1525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:03.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.011+0000 7f98c201f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98bc152130 0x7f98bc1525a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42414/0 (socket says 172.21.15.163:42414) 2023-10-09T06:39:03.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.011+0000 7f98c201f700 1 -- 172.21.15.163:0/3980721922 learned_addr learned my addr 172.21.15.163:0/3980721922 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:03.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.012+0000 7f98c181e700 1 --2- 172.21.15.163:0/3980721922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98a8004810 0x7f98bc0cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:03.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.012+0000 7f98c201f700 1 -- 172.21.15.163:0/3980721922 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98bc0c5590 msgr2=0x7f98bc0c6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:03.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.012+0000 7f98c201f700 1 --2- 172.21.15.163:0/3980721922 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98bc0c5590 0x7f98bc0c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:03.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.012+0000 7f98c201f700 1 -- 172.21.15.163:0/3980721922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98a8004810 msgr2=0x7f98bc0cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:03.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.012+0000 7f98c201f700 1 --2- 172.21.15.163:0/3980721922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98a8004810 0x7f98bc0cd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:03.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.012+0000 7f98c201f700 1 -- 172.21.15.163:0/3980721922 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98bc151650 con 0x7f98bc152130 2023-10-09T06:39:03.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.012+0000 7f98c201f700 1 --2- 172.21.15.163:0/3980721922 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98bc152130 0x7f98bc1525a0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f98bc0d03b0 tx=0x7f98b4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:03.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.012+0000 7f98b1ffb700 1 -- 172.21.15.163:0/3980721922 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b4015740 con 0x7f98bc152130 2023-10-09T06:39:03.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.012+0000 7f98b1ffb700 1 -- 172.21.15.163:0/3980721922 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b40158c0 con 0x7f98bc152130 2023-10-09T06:39:03.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.012+0000 7f98c3a82700 1 -- 172.21.15.163:0/3980721922 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98bc151930 con 0x7f98bc152130 2023-10-09T06:39:03.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.012+0000 7f98b1ffb700 1 -- 172.21.15.163:0/3980721922 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b4015c80 con 0x7f98bc152130 2023-10-09T06:39:03.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.012+0000 7f98c3a82700 1 -- 172.21.15.163:0/3980721922 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98bc0cf5c0 con 0x7f98bc152130 2023-10-09T06:39:03.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.013+0000 7f98c3a82700 1 -- 172.21.15.163:0/3980721922 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98bc1514b0 con 0x7f98bc152130 2023-10-09T06:39:03.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.014+0000 7f98b1ffb700 1 -- 172.21.15.163:0/3980721922 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f98b40145f0 con 0x7f98bc152130 2023-10-09T06:39:03.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.014+0000 7f98b1ffb700 1 --2- 172.21.15.163:0/3980721922 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f98a0067320 0x7f98a00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:03.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.014+0000 7f98b1ffb700 1 -- 172.21.15.163:0/3980721922 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6109+0+0 (secure 0 0 0) 0x7f98b4084d40 con 0x7f98bc152130 2023-10-09T06:39:03.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.014+0000 7f98c181e700 1 --2- 172.21.15.163:0/3980721922 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f98a0067320 0x7f98a00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:03.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.015+0000 7f98c181e700 1 --2- 172.21.15.163:0/3980721922 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f98a0067320 0x7f98a00697d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f98b8001940 tx=0x7f98b8001040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:03.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.018+0000 7f98b1ffb700 1 -- 172.21.15.163:0/3980721922 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f98bc1514b0 con 0x7f98bc152130 2023-10-09T06:39:03.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.247+0000 7f98c3a82700 1 -- 172.21.15.163:0/3980721922 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98bc1514b0 con 0x7f98bc152130 2023-10-09T06:39:03.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.247+0000 7f98b1ffb700 1 -- 172.21.15.163:0/3980721922 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+4622 (secure 0 0 0) 0x7f98bc1514b0 con 0x7f98bc152130 2023-10-09T06:39:03.248 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:03.248 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":114,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8211,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3056573272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3056573272},{"type":"v1","addr":"172.21.15.163:6834","nonce":3056573272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":8214,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3482861499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3482861499},{"type":"v1","addr":"172.21.15.163:6835","nonce":3482861499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":8223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/4239124825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4239124825},{"type":"v1","addr":"172.21.15.181:6836","nonce":4239124825}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:38:49.480715+0000","modified":"2023-10-09T06:39:02.175673+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8226":{"gid":8226,"name":"b","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/3494162891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3494162891},{"type":"v1","addr":"172.21.15.181:6837","nonce":3494162891}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29,30],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-09T06:39:03.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.249+0000 7f98c3a82700 1 -- 172.21.15.163:0/3980721922 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f98a0067320 msgr2=0x7f98a00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:03.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.249+0000 7f98c3a82700 1 --2- 172.21.15.163:0/3980721922 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f98a0067320 0x7f98a00697d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f98b8001940 tx=0x7f98b8001040 comp rx=0 tx=0).stop 2023-10-09T06:39:03.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.249+0000 7f98c3a82700 1 -- 172.21.15.163:0/3980721922 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98bc152130 msgr2=0x7f98bc1525a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:03.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.249+0000 7f98c3a82700 1 --2- 172.21.15.163:0/3980721922 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98bc152130 0x7f98bc1525a0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f98bc0d03b0 tx=0x7f98b4008040 comp rx=0 tx=0).stop 2023-10-09T06:39:03.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.249+0000 7f98c3a82700 1 -- 172.21.15.163:0/3980721922 shutdown_connections 2023-10-09T06:39:03.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.249+0000 7f98c3a82700 1 --2- 172.21.15.163:0/3980721922 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f98a0067320 0x7f98a00697d0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:03.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.249+0000 7f98c3a82700 1 --2- 172.21.15.163:0/3980721922 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98bc152130 0x7f98bc1525a0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:03.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.249+0000 7f98c3a82700 1 --2- 172.21.15.163:0/3980721922 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98a8004810 0x7f98bc0cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:03.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.249+0000 7f98c3a82700 1 --2- 172.21.15.163:0/3980721922 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98bc0c5590 0x7f98bc0c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:03.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.249+0000 7f98c3a82700 1 -- 172.21.15.163:0/3980721922 >> 172.21.15.163:0/3980721922 conn(0x7f98bc068e20 msgr2=0x7f98bc158b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:03.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.249+0000 7f98c3a82700 1 -- 172.21.15.163:0/3980721922 shutdown_connections 2023-10-09T06:39:03.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.249+0000 7f98c3a82700 1 -- 172.21.15.163:0/3980721922 wait complete. 2023-10-09T06:39:03.251 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 114 2023-10-09T06:39:03.262 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:39:03.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.379+0000 7f483c460700 1 Processor -- start 2023-10-09T06:39:03.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.379+0000 7f483c460700 1 -- start start 2023-10-09T06:39:03.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.380+0000 7f483c460700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4834159c60 0x7f483415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:03.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.380+0000 7f483c460700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f483415db10 0x7f4834069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:03.382 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.380+0000 7f483c460700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f483415a160 con 0x7f4834159890 2023-10-09T06:39:03.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.380+0000 7f483c460700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f483415df50 con 0x7f4834159c60 2023-10-09T06:39:03.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.380+0000 7f483c460700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48340d13a0 con 0x7f483415db10 2023-10-09T06:39:03.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.380+0000 7f48399fb700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4834159c60 0x7f483415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:03.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.380+0000 7f483a1fc700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f4834159890 0x7f483415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:50608/0 (socket says 172.21.15.163:50608) 2023-10-09T06:39:03.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.380+0000 7f483a1fc700 1 -- 172.21.15.163:0/3827673270 learned_addr learned my addr 172.21.15.163:0/3827673270 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:03.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.380+0000 7f48399fb700 1 --2- 172.21.15.163:0/3827673270 >> v2:172.21.15.163:3300/0 conn(0x7f4834159c60 0x7f483415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42418/0 (socket says 172.21.15.163:42418) 2023-10-09T06:39:03.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.380+0000 7f483a9fd700 1 --2- 172.21.15.163:0/3827673270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f483415db10 0x7f4834069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:03.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.380+0000 7f48399fb700 1 -- 172.21.15.163:0/3827673270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f483415db10 msgr2=0x7f4834069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:03.383 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.380+0000 7f48399fb700 1 --2- 172.21.15.163:0/3827673270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f483415db10 0x7f4834069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:03.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.380+0000 7f48399fb700 1 -- 172.21.15.163:0/3827673270 >> v1:172.21.15.163:6789/0 conn(0x7f4834159890 legacy=0x7f483415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:03.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.380+0000 7f48399fb700 1 -- 172.21.15.163:0/3827673270 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483415e140 con 0x7f4834159c60 2023-10-09T06:39:03.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.380+0000 7f48391fa700 1 -- 172.21.15.163:0/3827673270 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2242921384 0 0) 0x7f483415a160 con 0x7f4834159890 2023-10-09T06:39:03.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.380+0000 7f48399fb700 1 --2- 172.21.15.163:0/3827673270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4834159c60 0x7f483415d5d0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f48240098e0 tx=0x7f48240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc89f48358036b38 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:03.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.381+0000 7f48391fa700 1 -- 172.21.15.163:0/3827673270 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4824009e80 con 0x7f4834159c60 2023-10-09T06:39:03.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.381+0000 7f48391fa700 1 -- 172.21.15.163:0/3827673270 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482400b860 con 0x7f4834159c60 2023-10-09T06:39:03.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.381+0000 7f48391fa700 1 -- 172.21.15.163:0/3827673270 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482400bbe0 con 0x7f4834159c60 2023-10-09T06:39:03.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.381+0000 7f483c460700 1 -- 172.21.15.163:0/3827673270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4834159c60 msgr2=0x7f483415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:03.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.381+0000 7f483c460700 1 --2- 172.21.15.163:0/3827673270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4834159c60 0x7f483415d5d0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f48240098e0 tx=0x7f48240091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:03.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.381+0000 7f483c460700 1 -- 172.21.15.163:0/3827673270 shutdown_connections 2023-10-09T06:39:03.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.381+0000 7f483c460700 1 --2- 172.21.15.163:0/3827673270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f483415db10 0x7f4834069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:03.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.381+0000 7f483c460700 1 --2- 172.21.15.163:0/3827673270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4834159c60 0x7f483415d5d0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:03.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.381+0000 7f483c460700 1 -- 172.21.15.163:0/3827673270 >> 172.21.15.163:0/3827673270 conn(0x7f4834068e20 msgr2=0x7f48340d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:03.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.381+0000 7f483c460700 1 -- 172.21.15.163:0/3827673270 shutdown_connections 2023-10-09T06:39:03.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.381+0000 7f483c460700 1 -- 172.21.15.163:0/3827673270 wait complete. 2023-10-09T06:39:03.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.382+0000 7f483c460700 1 Processor -- start 2023-10-09T06:39:03.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.382+0000 7f483c460700 1 -- start start 2023-10-09T06:39:03.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.383+0000 7f483c460700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4834159890 0x7f4834145bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:03.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.383+0000 7f483c460700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4834159c60 0x7f48340c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:03.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.383+0000 7f483c460700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f483415db10 0x7f4834146350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:03.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.383+0000 7f483c460700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48340c6770 con 0x7f4834159890 2023-10-09T06:39:03.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.383+0000 7f483c460700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4834152600 con 0x7f4834159c60 2023-10-09T06:39:03.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.383+0000 7f483c460700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48340c5f10 con 0x7f483415db10 2023-10-09T06:39:03.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.383+0000 7f483a1fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4834159890 0x7f4834145bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:03.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.383+0000 7f483a1fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4834159890 0x7f4834145bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42428/0 (socket says 172.21.15.163:42428) 2023-10-09T06:39:03.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.383+0000 7f483a1fc700 1 -- 172.21.15.163:0/266304872 learned_addr learned my addr 172.21.15.163:0/266304872 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:03.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.383+0000 7f483a1fc700 1 -- 172.21.15.163:0/266304872 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f483415db10 msgr2=0x7f4834146350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:03.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.383+0000 7f483a1fc700 1 --2- 172.21.15.163:0/266304872 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f483415db10 0x7f4834146350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:03.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.383+0000 7f483a1fc700 1 -- 172.21.15.163:0/266304872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4834159c60 msgr2=0x7f48340c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:03.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.383+0000 7f483a1fc700 1 --2- 172.21.15.163:0/266304872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4834159c60 0x7f48340c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:03.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.383+0000 7f483a1fc700 1 -- 172.21.15.163:0/266304872 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4824009420 con 0x7f4834159890 2023-10-09T06:39:03.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.384+0000 7f483a1fc700 1 --2- 172.21.15.163:0/266304872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4834159890 0x7f4834145bf0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f483000c7e0 tx=0x7f4830007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:03.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.384+0000 7f482a7fc700 1 -- 172.21.15.163:0/266304872 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4830005b30 con 0x7f4834159890 2023-10-09T06:39:03.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.384+0000 7f482a7fc700 1 -- 172.21.15.163:0/266304872 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4830005c90 con 0x7f4834159890 2023-10-09T06:39:03.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.384+0000 7f482a7fc700 1 -- 172.21.15.163:0/266304872 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4830007300 con 0x7f4834159890 2023-10-09T06:39:03.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.384+0000 7f483c460700 1 -- 172.21.15.163:0/266304872 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4834151ff0 con 0x7f4834159890 2023-10-09T06:39:03.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.384+0000 7f483c460700 1 -- 172.21.15.163:0/266304872 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48340c61b0 con 0x7f4834159890 2023-10-09T06:39:03.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.385+0000 7f483c460700 1 -- 172.21.15.163:0/266304872 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f483404c3c0 con 0x7f4834159890 2023-10-09T06:39:03.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.385+0000 7f482a7fc700 1 -- 172.21.15.163:0/266304872 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f483001b400 con 0x7f4834159890 2023-10-09T06:39:03.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.385+0000 7f482a7fc700 1 --2- 172.21.15.163:0/266304872 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4818067320 0x7f48180697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:03.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.386+0000 7f482a7fc700 1 -- 172.21.15.163:0/266304872 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6109+0+0 (secure 0 0 0) 0x7f4830084890 con 0x7f4834159890 2023-10-09T06:39:03.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.386+0000 7f48399fb700 1 --2- 172.21.15.163:0/266304872 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4818067320 0x7f48180697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:03.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.386+0000 7f48399fb700 1 --2- 172.21.15.163:0/266304872 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4818067320 0x7f48180697d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f48341559a0 tx=0x7f4824017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:03.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.390+0000 7f482a7fc700 1 -- 172.21.15.163:0/266304872 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4830051490 con 0x7f4834159890 2023-10-09T06:39:03.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.577+0000 7f483c460700 1 -- 172.21.15.163:0/266304872 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48340c6550 con 0x7f4834159890 2023-10-09T06:39:03.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.577+0000 7f482a7fc700 1 -- 172.21.15.163:0/266304872 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) v1 ==== 74+0+16982 (secure 0 0 0) 0x7f4830054ab0 con 0x7f4834159890 2023-10-09T06:39:03.579 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:03.579 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":167,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:39:03.187675+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:38:47.806008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-10-09T06:38:48.446181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":30,"pool_name":"another-data-pool","create_time":"2023-10-09T06:39:00.367621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:39:03.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.580+0000 7f483c460700 1 -- 172.21.15.163:0/266304872 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4818067320 msgr2=0x7f48180697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:03.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.580+0000 7f483c460700 1 --2- 172.21.15.163:0/266304872 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4818067320 0x7f48180697d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f48341559a0 tx=0x7f4824017000 comp rx=0 tx=0).stop 2023-10-09T06:39:03.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.580+0000 7f483c460700 1 -- 172.21.15.163:0/266304872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4834159890 msgr2=0x7f4834145bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:03.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.580+0000 7f483c460700 1 --2- 172.21.15.163:0/266304872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4834159890 0x7f4834145bf0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f483000c7e0 tx=0x7f4830007df0 comp rx=0 tx=0).stop 2023-10-09T06:39:03.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.580+0000 7f483c460700 1 -- 172.21.15.163:0/266304872 shutdown_connections 2023-10-09T06:39:03.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.580+0000 7f483c460700 1 --2- 172.21.15.163:0/266304872 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4818067320 0x7f48180697d0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:03.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.580+0000 7f483c460700 1 --2- 172.21.15.163:0/266304872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4834159890 0x7f4834145bf0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:03.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.580+0000 7f483c460700 1 --2- 172.21.15.163:0/266304872 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f483415db10 0x7f4834146350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:03.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.580+0000 7f483c460700 1 --2- 172.21.15.163:0/266304872 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4834159c60 0x7f48340c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:03.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.580+0000 7f483c460700 1 -- 172.21.15.163:0/266304872 >> 172.21.15.163:0/266304872 conn(0x7f4834068e20 msgr2=0x7f4834156e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:03.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.580+0000 7f483c460700 1 -- 172.21.15.163:0/266304872 shutdown_connections 2023-10-09T06:39:03.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.580+0000 7f483c460700 1 -- 172.21.15.163:0/266304872 wait complete. 2023-10-09T06:39:03.593 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rename cephfs volume_0000000000836565 --yes-i-really-mean-it 2023-10-09T06:39:03.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.711+0000 7f2a3971c700 1 Processor -- start 2023-10-09T06:39:03.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.711+0000 7f2a3971c700 1 -- start start 2023-10-09T06:39:03.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.712+0000 7f2a3971c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2a340b03c0 0x7f2a340cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:03.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.712+0000 7f2a3971c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a341591e0 0x7f2a34159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:03.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.712+0000 7f2a3971c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a340cfa00 con 0x7f2a340b0790 2023-10-09T06:39:03.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.712+0000 7f2a3971c700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a340cb580 con 0x7f2a340b03c0 2023-10-09T06:39:03.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.712+0000 7f2a3971c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a340cf160 con 0x7f2a341591e0 2023-10-09T06:39:03.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.712+0000 7f2a33fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2a340b03c0 0x7f2a340cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:03.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.712+0000 7f2a33fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2a340b03c0 0x7f2a340cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42444/0 (socket says 172.21.15.163:42444) 2023-10-09T06:39:03.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.712+0000 7f2a33fff700 1 -- 172.21.15.163:0/620395581 learned_addr learned my addr 172.21.15.163:0/620395581 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:03.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.712+0000 7f2a38f1b700 1 --2- 172.21.15.163:0/620395581 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a341591e0 0x7f2a34159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:03.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.712+0000 7f2a33fff700 1 -- 172.21.15.163:0/620395581 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a341591e0 msgr2=0x7f2a34159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:03.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.712+0000 7f2a33fff700 1 --2- 172.21.15.163:0/620395581 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a341591e0 0x7f2a34159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:03.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.712+0000 7f2a33fff700 1 -- 172.21.15.163:0/620395581 >> v1:172.21.15.181:6789/0 conn(0x7f2a340b0790 legacy=0x7f2a340b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:03.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.712+0000 7f2a33fff700 1 -- 172.21.15.163:0/620395581 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a340cf440 con 0x7f2a340b03c0 2023-10-09T06:39:03.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.712+0000 7f2a33fff700 1 --2- 172.21.15.163:0/620395581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a340b03c0 0x7f2a340cea70 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f2a240099e0 tx=0x7f2a24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a71bb3c77df2f server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:03.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.712+0000 7f2a32ffd700 1 -- 172.21.15.163:0/620395581 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2400b820 con 0x7f2a340b03c0 2023-10-09T06:39:03.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.712+0000 7f2a32ffd700 1 -- 172.21.15.163:0/620395581 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2400b980 con 0x7f2a340b03c0 2023-10-09T06:39:03.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.712+0000 7f2a32ffd700 1 -- 172.21.15.163:0/620395581 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2400bd20 con 0x7f2a340b03c0 2023-10-09T06:39:03.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.713+0000 7f2a3971c700 1 -- 172.21.15.163:0/620395581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a340b03c0 msgr2=0x7f2a340cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:03.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.713+0000 7f2a3971c700 1 --2- 172.21.15.163:0/620395581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a340b03c0 0x7f2a340cea70 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f2a240099e0 tx=0x7f2a24009250 comp rx=0 tx=0).stop 2023-10-09T06:39:03.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.713+0000 7f2a3971c700 1 -- 172.21.15.163:0/620395581 shutdown_connections 2023-10-09T06:39:03.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.713+0000 7f2a3971c700 1 --2- 172.21.15.163:0/620395581 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a341591e0 0x7f2a34159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:03.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.713+0000 7f2a3971c700 1 --2- 172.21.15.163:0/620395581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a340b03c0 0x7f2a340cea70 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:03.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.713+0000 7f2a3971c700 1 -- 172.21.15.163:0/620395581 >> 172.21.15.163:0/620395581 conn(0x7f2a34063230 msgr2=0x7f2a34063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:03.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.713+0000 7f2a3971c700 1 -- 172.21.15.163:0/620395581 shutdown_connections 2023-10-09T06:39:03.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.713+0000 7f2a3971c700 1 -- 172.21.15.163:0/620395581 wait complete. 2023-10-09T06:39:03.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.714+0000 7f2a3971c700 1 Processor -- start 2023-10-09T06:39:03.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.714+0000 7f2a3971c700 1 -- start start 2023-10-09T06:39:03.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.715+0000 7f2a3971c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a340b03c0 0x7f2a340af5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:03.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.715+0000 7f2a3971c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a340b0790 0x7f2a3416a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:03.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.715+0000 7f2a3971c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a341591e0 0x7f2a34069360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:03.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.715+0000 7f2a3971c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3415b630 con 0x7f2a341591e0 2023-10-09T06:39:03.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.715+0000 7f2a3971c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3416ab80 con 0x7f2a340b0790 2023-10-09T06:39:03.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.715+0000 7f2a3971c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a340c6f30 con 0x7f2a340b03c0 2023-10-09T06:39:03.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.715+0000 7f2a38f1b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a341591e0 0x7f2a34069360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:03.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.715+0000 7f2a38f1b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a341591e0 0x7f2a34069360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42450/0 (socket says 172.21.15.163:42450) 2023-10-09T06:39:03.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.715+0000 7f2a38f1b700 1 -- 172.21.15.163:0/2869570527 learned_addr learned my addr 172.21.15.163:0/2869570527 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:03.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.715+0000 7f2a33fff700 1 --2- 172.21.15.163:0/2869570527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a340b03c0 0x7f2a340af5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:03.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.715+0000 7f2a337fe700 1 --2- 172.21.15.163:0/2869570527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a340b0790 0x7f2a3416a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:03.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.715+0000 7f2a38f1b700 1 -- 172.21.15.163:0/2869570527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a340b03c0 msgr2=0x7f2a340af5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:03.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.715+0000 7f2a38f1b700 1 --2- 172.21.15.163:0/2869570527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a340b03c0 0x7f2a340af5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:03.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.715+0000 7f2a38f1b700 1 -- 172.21.15.163:0/2869570527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a340b0790 msgr2=0x7f2a3416a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:03.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.715+0000 7f2a38f1b700 1 --2- 172.21.15.163:0/2869570527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a340b0790 0x7f2a3416a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:03.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.715+0000 7f2a38f1b700 1 -- 172.21.15.163:0/2869570527 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a240094f0 con 0x7f2a341591e0 2023-10-09T06:39:03.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.715+0000 7f2a337fe700 1 --2- 172.21.15.163:0/2869570527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a340b0790 0x7f2a3416a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:03.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.716+0000 7f2a38f1b700 1 --2- 172.21.15.163:0/2869570527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a341591e0 0x7f2a34069360 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c009df0 tx=0x7f2a2c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:03.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.716+0000 7f2a1bfff700 1 -- 172.21.15.163:0/2869570527 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c00caf0 con 0x7f2a341591e0 2023-10-09T06:39:03.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.716+0000 7f2a1bfff700 1 -- 172.21.15.163:0/2869570527 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2c00f040 con 0x7f2a341591e0 2023-10-09T06:39:03.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.716+0000 7f2a3971c700 1 -- 172.21.15.163:0/2869570527 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a340c73c0 con 0x7f2a341591e0 2023-10-09T06:39:03.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.716+0000 7f2a1bfff700 1 -- 172.21.15.163:0/2869570527 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c00cec0 con 0x7f2a341591e0 2023-10-09T06:39:03.721 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.716+0000 7f2a3971c700 1 -- 172.21.15.163:0/2869570527 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a340c6600 con 0x7f2a341591e0 2023-10-09T06:39:03.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.717+0000 7f2a3971c700 1 -- 172.21.15.163:0/2869570527 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a340c70a0 con 0x7f2a341591e0 2023-10-09T06:39:03.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.717+0000 7f2a1bfff700 1 -- 172.21.15.163:0/2869570527 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2a2c014690 con 0x7f2a341591e0 2023-10-09T06:39:03.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.718+0000 7f2a1bfff700 1 --2- 172.21.15.163:0/2869570527 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2a20067320 0x7f2a200697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:03.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.718+0000 7f2a1bfff700 1 -- 172.21.15.163:0/2869570527 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6109+0+0 (secure 0 0 0) 0x7f2a2c085810 con 0x7f2a341591e0 2023-10-09T06:39:03.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.718+0000 7f2a33fff700 1 --2- 172.21.15.163:0/2869570527 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2a20067320 0x7f2a200697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:03.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.718+0000 7f2a33fff700 1 --2- 172.21.15.163:0/2869570527 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2a20067320 0x7f2a200697d0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2a240099e0 tx=0x7f2a24017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:03.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.722+0000 7f2a1bfff700 1 -- 172.21.15.163:0/2869570527 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a2c055c90 con 0x7f2a341591e0 2023-10-09T06:39:03.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:03.931+0000 7f2a3971c700 1 -- 172.21.15.163:0/2869570527 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume rename", "vol_name": "cephfs", "new_vol_name": "volume_0000000000836565", "yes_i_really_mean_it": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f2a3404c3c0 con 0x7f2a20067320 2023-10-09T06:39:05.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.566+0000 7f2a1bfff700 1 -- 172.21.15.163:0/2869570527 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+127 (secure 0 0 0) 0x7f2a3404c3c0 con 0x7f2a20067320 2023-10-09T06:39:05.567 INFO:teuthology.orchestra.run.smithi163.stdout:FS volume 'cephfs' renamed to 'volume_0000000000836565'. But failed to rename data pools as more than one data pool was found. 2023-10-09T06:39:05.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.569+0000 7f2a3971c700 1 -- 172.21.15.163:0/2869570527 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2a20067320 msgr2=0x7f2a200697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:05.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.569+0000 7f2a3971c700 1 --2- 172.21.15.163:0/2869570527 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2a20067320 0x7f2a200697d0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2a240099e0 tx=0x7f2a24017000 comp rx=0 tx=0).stop 2023-10-09T06:39:05.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.569+0000 7f2a3971c700 1 -- 172.21.15.163:0/2869570527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a341591e0 msgr2=0x7f2a34069360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:05.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.569+0000 7f2a3971c700 1 --2- 172.21.15.163:0/2869570527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a341591e0 0x7f2a34069360 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c009df0 tx=0x7f2a2c00b040 comp rx=0 tx=0).stop 2023-10-09T06:39:05.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.569+0000 7f2a3971c700 1 -- 172.21.15.163:0/2869570527 shutdown_connections 2023-10-09T06:39:05.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.569+0000 7f2a3971c700 1 --2- 172.21.15.163:0/2869570527 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2a20067320 0x7f2a200697d0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:05.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.569+0000 7f2a3971c700 1 --2- 172.21.15.163:0/2869570527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a341591e0 0x7f2a34069360 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:05.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.569+0000 7f2a3971c700 1 --2- 172.21.15.163:0/2869570527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a340b03c0 0x7f2a340af5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:05.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.569+0000 7f2a3971c700 1 --2- 172.21.15.163:0/2869570527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a340b0790 0x7f2a3416a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:05.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.569+0000 7f2a3971c700 1 -- 172.21.15.163:0/2869570527 >> 172.21.15.163:0/2869570527 conn(0x7f2a34063230 msgr2=0x7f2a340cba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:05.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.569+0000 7f2a3971c700 1 -- 172.21.15.163:0/2869570527 shutdown_connections 2023-10-09T06:39:05.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.569+0000 7f2a3971c700 1 -- 172.21.15.163:0/2869570527 wait complete. 2023-10-09T06:39:05.584 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-09T06:39:05.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.705+0000 7f0d79f14700 1 Processor -- start 2023-10-09T06:39:05.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.705+0000 7f0d79f14700 1 -- start start 2023-10-09T06:39:05.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.706+0000 7f0d79f14700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0d740cc3c0 0x7f0d7406b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:05.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.706+0000 7f0d79f14700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d740b16a0 con 0x7f0d7406bf20 2023-10-09T06:39:05.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.706+0000 7f0d79f14700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d740b1c60 con 0x7f0d741574d0 2023-10-09T06:39:05.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.706+0000 7f0d79f14700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d740cd980 con 0x7f0d740cc3c0 2023-10-09T06:39:05.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.706+0000 7f0d73fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0d740cc3c0 0x7f0d7406b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:05.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.706+0000 7f0d79713700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f0d7406bf20 0x7f0d740b20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:50616/0 (socket says 172.21.15.163:50616) 2023-10-09T06:39:05.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.706+0000 7f0d79713700 1 -- 172.21.15.163:0/2558919123 learned_addr learned my addr 172.21.15.163:0/2558919123 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:05.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.706+0000 7f0d73fff700 1 -- 172.21.15.163:0/2558919123 >> v1:172.21.15.181:6789/0 conn(0x7f0d741574d0 legacy=0x7f0d740cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:05.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.706+0000 7f0d73fff700 1 -- 172.21.15.163:0/2558919123 >> v1:172.21.15.163:6789/0 conn(0x7f0d7406bf20 legacy=0x7f0d740b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:05.708 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.706+0000 7f0d73fff700 1 -- 172.21.15.163:0/2558919123 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7406c360 con 0x7f0d740cc3c0 2023-10-09T06:39:05.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.706+0000 7f0d73fff700 1 --2- 172.21.15.163:0/2558919123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d740cc3c0 0x7f0d7406b9e0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f0d680098e0 tx=0x7f0d680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf7bb78da8d8781 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:05.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.706+0000 7f0d737fe700 1 -- 172.21.15.163:0/2558919123 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d68009e80 con 0x7f0d740cc3c0 2023-10-09T06:39:05.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.706+0000 7f0d737fe700 1 -- 172.21.15.163:0/2558919123 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6800b860 con 0x7f0d740cc3c0 2023-10-09T06:39:05.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.706+0000 7f0d737fe700 1 -- 172.21.15.163:0/2558919123 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6800bbe0 con 0x7f0d740cc3c0 2023-10-09T06:39:05.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.707+0000 7f0d79f14700 1 -- 172.21.15.163:0/2558919123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d740cc3c0 msgr2=0x7f0d7406b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:05.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.707+0000 7f0d79f14700 1 --2- 172.21.15.163:0/2558919123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d740cc3c0 0x7f0d7406b9e0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f0d680098e0 tx=0x7f0d680091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:05.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.707+0000 7f0d79f14700 1 -- 172.21.15.163:0/2558919123 shutdown_connections 2023-10-09T06:39:05.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.707+0000 7f0d79f14700 1 --2- 172.21.15.163:0/2558919123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d740cc3c0 0x7f0d7406b9e0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:05.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.707+0000 7f0d79f14700 1 -- 172.21.15.163:0/2558919123 >> 172.21.15.163:0/2558919123 conn(0x7f0d740ce430 msgr2=0x7f0d740ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:05.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.707+0000 7f0d79f14700 1 -- 172.21.15.163:0/2558919123 shutdown_connections 2023-10-09T06:39:05.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.707+0000 7f0d79f14700 1 -- 172.21.15.163:0/2558919123 wait complete. 2023-10-09T06:39:05.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.708+0000 7f0d79f14700 1 Processor -- start 2023-10-09T06:39:05.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.708+0000 7f0d79f14700 1 -- start start 2023-10-09T06:39:05.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.709+0000 7f0d79f14700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d7406bf20 0x7f0d74153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:05.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.709+0000 7f0d79f14700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d740cc3c0 0x7f0d740a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:05.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.709+0000 7f0d79f14700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d741574d0 0x7f0d740a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:05.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.709+0000 7f0d79f14700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d740aa100 con 0x7f0d741574d0 2023-10-09T06:39:05.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.709+0000 7f0d79f14700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d740afc00 con 0x7f0d740cc3c0 2023-10-09T06:39:05.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.709+0000 7f0d79f14700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d740afee0 con 0x7f0d7406bf20 2023-10-09T06:39:05.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.709+0000 7f0d79713700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d741574d0 0x7f0d740a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:05.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.709+0000 7f0d79713700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d741574d0 0x7f0d740a60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42468/0 (socket says 172.21.15.163:42468) 2023-10-09T06:39:05.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.709+0000 7f0d79713700 1 -- 172.21.15.163:0/201408882 learned_addr learned my addr 172.21.15.163:0/201408882 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:05.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.709+0000 7f0d73fff700 1 --2- 172.21.15.163:0/201408882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d740cc3c0 0x7f0d740a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:05.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.709+0000 7f0d78f12700 1 --2- 172.21.15.163:0/201408882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d7406bf20 0x7f0d74153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:05.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.709+0000 7f0d79713700 1 -- 172.21.15.163:0/201408882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d7406bf20 msgr2=0x7f0d74153d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:05.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.709+0000 7f0d79713700 1 --2- 172.21.15.163:0/201408882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d7406bf20 0x7f0d74153d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:05.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.709+0000 7f0d79713700 1 -- 172.21.15.163:0/201408882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d740cc3c0 msgr2=0x7f0d740a58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:05.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.709+0000 7f0d79713700 1 --2- 172.21.15.163:0/201408882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d740cc3c0 0x7f0d740a58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:05.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.709+0000 7f0d79713700 1 -- 172.21.15.163:0/201408882 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d68009420 con 0x7f0d741574d0 2023-10-09T06:39:05.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.709+0000 7f0d73fff700 1 --2- 172.21.15.163:0/201408882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d740cc3c0 0x7f0d740a58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:05.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.710+0000 7f0d79713700 1 --2- 172.21.15.163:0/201408882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d741574d0 0x7f0d740a60f0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f0d6400c590 tx=0x7f0d64007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:05.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.710+0000 7f0d70ff9700 1 -- 172.21.15.163:0/201408882 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d64005b30 con 0x7f0d741574d0 2023-10-09T06:39:05.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.710+0000 7f0d70ff9700 1 -- 172.21.15.163:0/201408882 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d64005cb0 con 0x7f0d741574d0 2023-10-09T06:39:05.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.710+0000 7f0d79f14700 1 -- 172.21.15.163:0/201408882 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d740b01c0 con 0x7f0d741574d0 2023-10-09T06:39:05.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.710+0000 7f0d70ff9700 1 -- 172.21.15.163:0/201408882 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d64007320 con 0x7f0d741574d0 2023-10-09T06:39:05.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.710+0000 7f0d79f14700 1 -- 172.21.15.163:0/201408882 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d74154240 con 0x7f0d741574d0 2023-10-09T06:39:05.713 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.711+0000 7f0d79f14700 1 -- 172.21.15.163:0/201408882 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d7404c3c0 con 0x7f0d741574d0 2023-10-09T06:39:05.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.711+0000 7f0d70ff9700 1 -- 172.21.15.163:0/201408882 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0d6401b3b0 con 0x7f0d741574d0 2023-10-09T06:39:05.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.711+0000 7f0d70ff9700 1 --2- 172.21.15.163:0/201408882 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0d60067320 0x7f0d600697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:05.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.712+0000 7f0d70ff9700 1 -- 172.21.15.163:0/201408882 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6180+0+0 (secure 0 0 0) 0x7f0d640843f0 con 0x7f0d741574d0 2023-10-09T06:39:05.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.712+0000 7f0d78f12700 1 --2- 172.21.15.163:0/201408882 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0d60067320 0x7f0d600697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:05.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.712+0000 7f0d78f12700 1 --2- 172.21.15.163:0/201408882 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0d60067320 0x7f0d600697d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c00e160 tx=0x7f0d6c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:05.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.716+0000 7f0d70ff9700 1 -- 172.21.15.163:0/201408882 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d64050e60 con 0x7f0d741574d0 2023-10-09T06:39:05.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.925+0000 7f0d79f14700 1 -- 172.21.15.163:0/201408882 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d740cf1b0 con 0x7f0d60067320 2023-10-09T06:39:05.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.928+0000 7f0d70ff9700 1 -- 172.21.15.163:0/201408882 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+58 (secure 0 0 0) 0x7f0d740cf1b0 con 0x7f0d60067320 2023-10-09T06:39:05.928 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:39:05.928 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:05.928 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "volume_0000000000836565" 2023-10-09T06:39:05.928 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:05.928 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:39:05.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.929+0000 7f0d79f14700 1 -- 172.21.15.163:0/201408882 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0d60067320 msgr2=0x7f0d600697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:05.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.929+0000 7f0d79f14700 1 --2- 172.21.15.163:0/201408882 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0d60067320 0x7f0d600697d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c00e160 tx=0x7f0d6c009040 comp rx=0 tx=0).stop 2023-10-09T06:39:05.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.929+0000 7f0d79f14700 1 -- 172.21.15.163:0/201408882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d741574d0 msgr2=0x7f0d740a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:05.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.929+0000 7f0d79f14700 1 --2- 172.21.15.163:0/201408882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d741574d0 0x7f0d740a60f0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f0d6400c590 tx=0x7f0d64007ef0 comp rx=0 tx=0).stop 2023-10-09T06:39:05.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.929+0000 7f0d79f14700 1 -- 172.21.15.163:0/201408882 shutdown_connections 2023-10-09T06:39:05.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.929+0000 7f0d79f14700 1 --2- 172.21.15.163:0/201408882 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0d60067320 0x7f0d600697d0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:05.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.929+0000 7f0d79f14700 1 --2- 172.21.15.163:0/201408882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d741574d0 0x7f0d740a60f0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:05.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.929+0000 7f0d79f14700 1 --2- 172.21.15.163:0/201408882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d7406bf20 0x7f0d74153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:05.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.929+0000 7f0d79f14700 1 --2- 172.21.15.163:0/201408882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d740cc3c0 0x7f0d740a58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:05.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.929+0000 7f0d79f14700 1 -- 172.21.15.163:0/201408882 >> 172.21.15.163:0/201408882 conn(0x7f0d740ce430 msgr2=0x7f0d740c95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:05.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.929+0000 7f0d79f14700 1 -- 172.21.15.163:0/201408882 shutdown_connections 2023-10-09T06:39:05.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:05.929+0000 7f0d79f14700 1 -- 172.21.15.163:0/201408882 wait complete. 2023-10-09T06:39:05.942 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:39:06.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.056+0000 7f9129a05700 1 Processor -- start 2023-10-09T06:39:06.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.056+0000 7f9129a05700 1 -- start start 2023-10-09T06:39:06.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.057+0000 7f9129a05700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9124069560 0x7f91240699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:06.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.057+0000 7f9129a05700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9124069ee0 con 0x7f9124159c60 2023-10-09T06:39:06.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.057+0000 7f9129a05700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f912415a160 con 0x7f9124159890 2023-10-09T06:39:06.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.057+0000 7f9129a05700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91240d13a0 con 0x7f9124069560 2023-10-09T06:39:06.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.057+0000 7f911a7fc700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f9124159c60 0x7f912415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:50622/0 (socket says 172.21.15.163:50622) 2023-10-09T06:39:06.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.057+0000 7f911a7fc700 1 -- 172.21.15.163:0/2048064315 learned_addr learned my addr 172.21.15.163:0/2048064315 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:06.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.057+0000 7f91237fe700 1 --2- 172.21.15.163:0/2048064315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9124069560 0x7f91240699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:06.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.057+0000 7f91227fc700 1 -- 172.21.15.163:0/2048064315 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3071616974 0 0) 0x7f9124069ee0 con 0x7f9124159c60 2023-10-09T06:39:06.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.058+0000 7f91227fc700 1 -- 172.21.15.163:0/2048064315 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9110003570 con 0x7f9124159c60 2023-10-09T06:39:06.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.058+0000 7f91227fc700 1 -- 172.21.15.163:0/2048064315 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3585883798 0 0) 0x7f9110003570 con 0x7f9124159c60 2023-10-09T06:39:06.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.058+0000 7f91227fc700 1 -- 172.21.15.163:0/2048064315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9124069560 msgr2=0x7f91240699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:06.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.058+0000 7f91227fc700 1 --2- 172.21.15.163:0/2048064315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9124069560 0x7f91240699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:06.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.058+0000 7f91227fc700 1 -- 172.21.15.163:0/2048064315 >> v1:172.21.15.181:6789/0 conn(0x7f9124159890 legacy=0x7f912415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:06.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.058+0000 7f91227fc700 1 -- 172.21.15.163:0/2048064315 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912415e140 con 0x7f9124159c60 2023-10-09T06:39:06.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.058+0000 7f91227fc700 1 -- 172.21.15.163:0/2048064315 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f910c001470 con 0x7f9124159c60 2023-10-09T06:39:06.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.058+0000 7f91227fc700 1 -- 172.21.15.163:0/2048064315 >> v1:172.21.15.163:6789/0 conn(0x7f9124159c60 legacy=0x7f912415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:06.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.058+0000 7f91227fc700 1 --2- 172.21.15.163:0/2048064315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9124069560 0x7f9110003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:06.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.058+0000 7f91227fc700 1 --2- 172.21.15.163:0/2048064315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9110004010 0x7f9110004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:06.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.058+0000 7f91227fc700 1 --2- 172.21.15.163:0/2048064315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9110004d50 0x7f91100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:06.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.058+0000 7f91227fc700 1 -- 172.21.15.163:0/2048064315 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91240d13a0 con 0x7f9110004010 2023-10-09T06:39:06.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.058+0000 7f91227fc700 1 -- 172.21.15.163:0/2048064315 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911000bbf0 con 0x7f9110004d50 2023-10-09T06:39:06.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.058+0000 7f91227fc700 1 -- 172.21.15.163:0/2048064315 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911000bdb0 con 0x7f9124069560 2023-10-09T06:39:06.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.058+0000 7f911a7fc700 1 --2- 172.21.15.163:0/2048064315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9110004010 0x7f9110004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:06.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.058+0000 7f9122ffd700 1 --2- 172.21.15.163:0/2048064315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9124069560 0x7f9110003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:06.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.058+0000 7f91237fe700 1 --2- 172.21.15.163:0/2048064315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9110004d50 0x7f91100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:06.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.058+0000 7f911a7fc700 1 -- 172.21.15.163:0/2048064315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9124069560 msgr2=0x7f9110003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:06.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.058+0000 7f911a7fc700 1 --2- 172.21.15.163:0/2048064315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9124069560 0x7f9110003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:06.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.058+0000 7f911a7fc700 1 -- 172.21.15.163:0/2048064315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9110004d50 msgr2=0x7f91100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:06.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.058+0000 7f911a7fc700 1 --2- 172.21.15.163:0/2048064315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9110004d50 0x7f91100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:06.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.058+0000 7f911a7fc700 1 -- 172.21.15.163:0/2048064315 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9110003570 con 0x7f9110004010 2023-10-09T06:39:06.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.058+0000 7f91237fe700 1 --2- 172.21.15.163:0/2048064315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9110004d50 0x7f91100051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:06.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.058+0000 7f911a7fc700 1 --2- 172.21.15.163:0/2048064315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9110004010 0x7f9110004810 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f910c00c390 tx=0x7f910c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec12c2c3d71a5cf2 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:06.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.058+0000 7f91227fc700 1 -- 172.21.15.163:0/2048064315 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c00cc60 con 0x7f9110004010 2023-10-09T06:39:06.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.059+0000 7f91227fc700 1 -- 172.21.15.163:0/2048064315 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f910c00cdc0 con 0x7f9110004010 2023-10-09T06:39:06.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.059+0000 7f91227fc700 1 -- 172.21.15.163:0/2048064315 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c005b50 con 0x7f9110004010 2023-10-09T06:39:06.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.059+0000 7f9129a05700 1 -- 172.21.15.163:0/2048064315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9110004010 msgr2=0x7f9110004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:06.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.059+0000 7f9129a05700 1 --2- 172.21.15.163:0/2048064315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9110004010 0x7f9110004810 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f910c00c390 tx=0x7f910c001e50 comp rx=0 tx=0).stop 2023-10-09T06:39:06.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.059+0000 7f91237fe700 1 -- 172.21.15.163:0/2048064315 reap_dead start 2023-10-09T06:39:06.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.059+0000 7f9129a05700 1 -- 172.21.15.163:0/2048064315 shutdown_connections 2023-10-09T06:39:06.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.059+0000 7f9129a05700 1 -- 172.21.15.163:0/2048064315 >> 172.21.15.163:0/2048064315 conn(0x7f9124068e20 msgr2=0x7f91240d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:06.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.059+0000 7f9129a05700 1 -- 172.21.15.163:0/2048064315 shutdown_connections 2023-10-09T06:39:06.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.059+0000 7f9129a05700 1 -- 172.21.15.163:0/2048064315 wait complete. 2023-10-09T06:39:06.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.060+0000 7f9129a05700 1 Processor -- start 2023-10-09T06:39:06.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.060+0000 7f9129a05700 1 -- start start 2023-10-09T06:39:06.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.061+0000 7f9129a05700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9110004010 0x7f9124158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:06.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.061+0000 7f9129a05700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91240b3f50 0x7f91240b5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:06.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.061+0000 7f9129a05700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9124152170 0x7f91241525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:06.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.061+0000 7f9129a05700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91240b50b0 con 0x7f9124152170 2023-10-09T06:39:06.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.061+0000 7f9129a05700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9124152ae0 con 0x7f9110004010 2023-10-09T06:39:06.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.061+0000 7f9129a05700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91240b44a0 con 0x7f91240b3f50 2023-10-09T06:39:06.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.061+0000 7f91237fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9124152170 0x7f91241525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:06.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.061+0000 7f91237fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9124152170 0x7f91241525a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42482/0 (socket says 172.21.15.163:42482) 2023-10-09T06:39:06.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.061+0000 7f91237fe700 1 -- 172.21.15.163:0/2267037935 learned_addr learned my addr 172.21.15.163:0/2267037935 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:06.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.061+0000 7f91237fe700 1 -- 172.21.15.163:0/2267037935 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91240b3f50 msgr2=0x7f91240b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:06.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.061+0000 7f91237fe700 1 --2- 172.21.15.163:0/2267037935 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91240b3f50 0x7f91240b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:06.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.061+0000 7f91237fe700 1 -- 172.21.15.163:0/2267037935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9110004010 msgr2=0x7f9124158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:06.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.061+0000 7f9122ffd700 1 --2- 172.21.15.163:0/2267037935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9110004010 0x7f9124158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:06.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.061+0000 7f91237fe700 1 --2- 172.21.15.163:0/2267037935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9110004010 0x7f9124158a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:06.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.061+0000 7f91237fe700 1 -- 172.21.15.163:0/2267037935 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91240b4780 con 0x7f9124152170 2023-10-09T06:39:06.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.061+0000 7f9122ffd700 1 --2- 172.21.15.163:0/2267037935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9110004010 0x7f9124158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:06.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.062+0000 7f91237fe700 1 --2- 172.21.15.163:0/2267037935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9124152170 0x7f91241525a0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f91240cadc0 tx=0x7f911c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:06.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.062+0000 7f911b7fe700 1 -- 172.21.15.163:0/2267037935 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c015670 con 0x7f9124152170 2023-10-09T06:39:06.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.062+0000 7f911b7fe700 1 -- 172.21.15.163:0/2267037935 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911c0157d0 con 0x7f9124152170 2023-10-09T06:39:06.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.062+0000 7f9129a05700 1 -- 172.21.15.163:0/2267037935 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91240b4a60 con 0x7f9124152170 2023-10-09T06:39:06.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.062+0000 7f911b7fe700 1 -- 172.21.15.163:0/2267037935 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c015b90 con 0x7f9124152170 2023-10-09T06:39:06.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.062+0000 7f9129a05700 1 -- 172.21.15.163:0/2267037935 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9124158f60 con 0x7f9124152170 2023-10-09T06:39:06.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.063+0000 7f9129a05700 1 -- 172.21.15.163:0/2267037935 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91240c78a0 con 0x7f9124152170 2023-10-09T06:39:06.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.063+0000 7f911b7fe700 1 -- 172.21.15.163:0/2267037935 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f911c020460 con 0x7f9124152170 2023-10-09T06:39:06.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.063+0000 7f911b7fe700 1 --2- 172.21.15.163:0/2267037935 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9104067320 0x7f91040697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:06.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.063+0000 7f911b7fe700 1 -- 172.21.15.163:0/2267037935 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6180+0+0 (secure 0 0 0) 0x7f911c085040 con 0x7f9124152170 2023-10-09T06:39:06.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.064+0000 7f9122ffd700 1 --2- 172.21.15.163:0/2267037935 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9104067320 0x7f91040697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:06.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.064+0000 7f9122ffd700 1 --2- 172.21.15.163:0/2267037935 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9104067320 0x7f91040697d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f912406a070 tx=0x7f9114009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:06.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.067+0000 7f911b7fe700 1 -- 172.21.15.163:0/2267037935 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f91240c78a0 con 0x7f9124152170 2023-10-09T06:39:06.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.300+0000 7f9129a05700 1 -- 172.21.15.163:0/2267037935 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f912404c3c0 con 0x7f9124152170 2023-10-09T06:39:06.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.301+0000 7f911b7fe700 1 -- 172.21.15.163:0/2267037935 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+4639 (secure 0 0 0) 0x7f911c055580 con 0x7f9124152170 2023-10-09T06:39:06.302 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:06.302 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":115,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8211,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3056573272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3056573272},{"type":"v1","addr":"172.21.15.163:6834","nonce":3056573272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":8214,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3482861499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3482861499},{"type":"v1","addr":"172.21.15.163:6835","nonce":3482861499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":8223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/4239124825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4239124825},{"type":"v1","addr":"172.21.15.181:6836","nonce":4239124825}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:38:49.480715+0000","modified":"2023-10-09T06:39:04.559594+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8226":{"gid":8226,"name":"b","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/3494162891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3494162891},{"type":"v1","addr":"172.21.15.181:6837","nonce":3494162891}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29,30],"metadata_pool":28,"enabled":true,"fs_name":"volume_0000000000836565","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-09T06:39:06.303 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.303+0000 7f9129a05700 1 -- 172.21.15.163:0/2267037935 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9104067320 msgr2=0x7f91040697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:06.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.303+0000 7f9129a05700 1 --2- 172.21.15.163:0/2267037935 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9104067320 0x7f91040697d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f912406a070 tx=0x7f9114009040 comp rx=0 tx=0).stop 2023-10-09T06:39:06.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.303+0000 7f9129a05700 1 -- 172.21.15.163:0/2267037935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9124152170 msgr2=0x7f91241525a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:06.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.303+0000 7f9129a05700 1 --2- 172.21.15.163:0/2267037935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9124152170 0x7f91241525a0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f91240cadc0 tx=0x7f911c008040 comp rx=0 tx=0).stop 2023-10-09T06:39:06.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.303+0000 7f9129a05700 1 -- 172.21.15.163:0/2267037935 shutdown_connections 2023-10-09T06:39:06.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.303+0000 7f9129a05700 1 --2- 172.21.15.163:0/2267037935 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9104067320 0x7f91040697d0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:06.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.303+0000 7f9129a05700 1 --2- 172.21.15.163:0/2267037935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9124152170 0x7f91241525a0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:06.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.303+0000 7f9129a05700 1 --2- 172.21.15.163:0/2267037935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9110004010 0x7f9124158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:06.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.303+0000 7f9129a05700 1 --2- 172.21.15.163:0/2267037935 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91240b3f50 0x7f91240b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:06.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.303+0000 7f9129a05700 1 -- 172.21.15.163:0/2267037935 >> 172.21.15.163:0/2267037935 conn(0x7f9124068e20 msgr2=0x7f91240cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:06.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.303+0000 7f9129a05700 1 -- 172.21.15.163:0/2267037935 shutdown_connections 2023-10-09T06:39:06.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.303+0000 7f9129a05700 1 -- 172.21.15.163:0/2267037935 wait complete. 2023-10-09T06:39:06.305 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 115 2023-10-09T06:39:06.316 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:39:06.432 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.431+0000 7eff5632d700 1 Processor -- start 2023-10-09T06:39:06.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.431+0000 7eff5632d700 1 -- start start 2023-10-09T06:39:06.437 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.432+0000 7eff5632d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7eff50159890 0x7eff5015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:06.437 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.432+0000 7eff5632d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7eff50159c60 0x7eff5015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:06.438 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.432+0000 7eff5632d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff5015a160 con 0x7eff5015db10 2023-10-09T06:39:06.438 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.432+0000 7eff5632d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff5015df50 con 0x7eff50159c60 2023-10-09T06:39:06.438 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.432+0000 7eff5632d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff500d13a0 con 0x7eff50159890 2023-10-09T06:39:06.438 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.432+0000 7eff4f7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7eff50159c60 0x7eff5015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:06.438 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.432+0000 7eff4f7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7eff50159c60 0x7eff5015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42488/0 (socket says 172.21.15.163:42488) 2023-10-09T06:39:06.438 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.432+0000 7eff4f7fe700 1 -- 172.21.15.163:0/1345671400 learned_addr learned my addr 172.21.15.163:0/1345671400 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:06.438 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.432+0000 7eff4f7fe700 1 -- 172.21.15.163:0/1345671400 >> v2:172.21.15.181:3300/0 conn(0x7eff50159890 msgr2=0x7eff5015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:06.438 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.432+0000 7eff4ffff700 1 --2- 172.21.15.163:0/1345671400 >> v2:172.21.15.181:3300/0 conn(0x7eff50159890 0x7eff5015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:06.438 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.432+0000 7eff4f7fe700 1 --2- 172.21.15.163:0/1345671400 >> v2:172.21.15.181:3300/0 conn(0x7eff50159890 0x7eff5015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:06.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.432+0000 7eff4f7fe700 1 -- 172.21.15.163:0/1345671400 >> v1:172.21.15.181:6789/0 conn(0x7eff5015db10 legacy=0x7eff50069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:06.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.432+0000 7eff4f7fe700 1 -- 172.21.15.163:0/1345671400 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff5015e140 con 0x7eff50159c60 2023-10-09T06:39:06.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.432+0000 7eff4f7fe700 1 --2- 172.21.15.163:0/1345671400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff50159c60 0x7eff5015d5d0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7eff500d1090 tx=0x7eff44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3ec384923506e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:06.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.432+0000 7eff4effd700 1 -- 172.21.15.163:0/1345671400 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4400b820 con 0x7eff50159c60 2023-10-09T06:39:06.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.432+0000 7eff4effd700 1 -- 172.21.15.163:0/1345671400 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff4400b980 con 0x7eff50159c60 2023-10-09T06:39:06.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.432+0000 7eff4effd700 1 -- 172.21.15.163:0/1345671400 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4400bd20 con 0x7eff50159c60 2023-10-09T06:39:06.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.433+0000 7eff5632d700 1 -- 172.21.15.163:0/1345671400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff50159c60 msgr2=0x7eff5015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:06.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.433+0000 7eff5632d700 1 --2- 172.21.15.163:0/1345671400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff50159c60 0x7eff5015d5d0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7eff500d1090 tx=0x7eff44009250 comp rx=0 tx=0).stop 2023-10-09T06:39:06.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.433+0000 7eff5632d700 1 -- 172.21.15.163:0/1345671400 shutdown_connections 2023-10-09T06:39:06.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.433+0000 7eff5632d700 1 --2- 172.21.15.163:0/1345671400 >> v2:172.21.15.181:3300/0 conn(0x7eff50159890 0x7eff5015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:06.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.433+0000 7eff5632d700 1 --2- 172.21.15.163:0/1345671400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff50159c60 0x7eff5015d5d0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:06.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.433+0000 7eff5632d700 1 -- 172.21.15.163:0/1345671400 >> 172.21.15.163:0/1345671400 conn(0x7eff50068e20 msgr2=0x7eff500d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:06.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.433+0000 7eff5632d700 1 -- 172.21.15.163:0/1345671400 shutdown_connections 2023-10-09T06:39:06.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.433+0000 7eff5632d700 1 -- 172.21.15.163:0/1345671400 wait complete. 2023-10-09T06:39:06.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.434+0000 7eff5632d700 1 Processor -- start 2023-10-09T06:39:06.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.434+0000 7eff5632d700 1 -- start start 2023-10-09T06:39:06.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.435+0000 7eff5632d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff50159890 0x7eff500c5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:06.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.435+0000 7eff5632d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff50159c60 0x7eff5016dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:06.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.435+0000 7eff5632d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff5015db10 0x7eff5015c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:06.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.435+0000 7eff5632d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5016e250 con 0x7eff50159890 2023-10-09T06:39:06.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.435+0000 7eff5632d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5016a0a0 con 0x7eff50159c60 2023-10-09T06:39:06.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.435+0000 7eff5632d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff5016a380 con 0x7eff5015db10 2023-10-09T06:39:06.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.435+0000 7eff4ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff50159890 0x7eff500c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:06.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.435+0000 7eff4ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff50159890 0x7eff500c5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42500/0 (socket says 172.21.15.163:42500) 2023-10-09T06:39:06.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.435+0000 7eff4ffff700 1 -- 172.21.15.163:0/2527656251 learned_addr learned my addr 172.21.15.163:0/2527656251 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:06.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.435+0000 7eff4f7fe700 1 --2- 172.21.15.163:0/2527656251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff50159c60 0x7eff5016dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:06.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.435+0000 7eff4ffff700 1 -- 172.21.15.163:0/2527656251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff5015db10 msgr2=0x7eff5015c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:06.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.435+0000 7eff4ffff700 1 --2- 172.21.15.163:0/2527656251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff5015db10 0x7eff5015c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:06.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.435+0000 7eff4ffff700 1 -- 172.21.15.163:0/2527656251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff50159c60 msgr2=0x7eff5016dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:06.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.435+0000 7eff4ffff700 1 --2- 172.21.15.163:0/2527656251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff50159c60 0x7eff5016dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:06.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.435+0000 7eff4ffff700 1 -- 172.21.15.163:0/2527656251 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff440094f0 con 0x7eff50159890 2023-10-09T06:39:06.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.436+0000 7eff4ffff700 1 --2- 172.21.15.163:0/2527656251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff50159890 0x7eff500c5640 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7eff4000c7d0 tx=0x7eff40008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:06.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.436+0000 7eff37fff700 1 -- 172.21.15.163:0/2527656251 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff40011600 con 0x7eff50159890 2023-10-09T06:39:06.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.436+0000 7eff37fff700 1 -- 172.21.15.163:0/2527656251 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff40011760 con 0x7eff50159890 2023-10-09T06:39:06.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.436+0000 7eff5632d700 1 -- 172.21.15.163:0/2527656251 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff5016a660 con 0x7eff50159890 2023-10-09T06:39:06.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.436+0000 7eff37fff700 1 -- 172.21.15.163:0/2527656251 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff40011b20 con 0x7eff50159890 2023-10-09T06:39:06.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.436+0000 7eff5632d700 1 -- 172.21.15.163:0/2527656251 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff50169d10 con 0x7eff50159890 2023-10-09T06:39:06.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.437+0000 7eff5632d700 1 -- 172.21.15.163:0/2527656251 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff5004c3c0 con 0x7eff50159890 2023-10-09T06:39:06.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.437+0000 7eff37fff700 1 -- 172.21.15.163:0/2527656251 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7eff400193f0 con 0x7eff50159890 2023-10-09T06:39:06.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.437+0000 7eff37fff700 1 --2- 172.21.15.163:0/2527656251 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7eff3c067320 0x7eff3c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:06.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.438+0000 7eff37fff700 1 -- 172.21.15.163:0/2527656251 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6180+0+0 (secure 0 0 0) 0x7eff40085370 con 0x7eff50159890 2023-10-09T06:39:06.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.438+0000 7eff4f7fe700 1 --2- 172.21.15.163:0/2527656251 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7eff3c067320 0x7eff3c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:06.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.438+0000 7eff4f7fe700 1 --2- 172.21.15.163:0/2527656251 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7eff3c067320 0x7eff3c0697d0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7eff500d1090 tx=0x7eff44016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:06.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.442+0000 7eff37fff700 1 -- 172.21.15.163:0/2527656251 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff40051d10 con 0x7eff50159890 2023-10-09T06:39:06.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.627+0000 7eff5632d700 1 -- 172.21.15.163:0/2527656251 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff5006a5b0 con 0x7eff50159890 2023-10-09T06:39:06.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.627+0000 7eff37fff700 1 -- 172.21.15.163:0/2527656251 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) v1 ==== 74+0+17053 (secure 0 0 0) 0x7eff40055330 con 0x7eff50159890 2023-10-09T06:39:06.628 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:06.629 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":169,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:39:05.560867+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":28,"pool_name":"cephfs.volume_0000000000836565.meta","create_time":"2023-10-09T06:38:47.806008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000836565"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-10-09T06:38:48.446181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000836565"}}},{"pool":30,"pool_name":"another-data-pool","create_time":"2023-10-09T06:39:00.367621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"volume_0000000000836565"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:39:06.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.630+0000 7eff5632d700 1 -- 172.21.15.163:0/2527656251 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7eff3c067320 msgr2=0x7eff3c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:06.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.630+0000 7eff5632d700 1 --2- 172.21.15.163:0/2527656251 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7eff3c067320 0x7eff3c0697d0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7eff500d1090 tx=0x7eff44016000 comp rx=0 tx=0).stop 2023-10-09T06:39:06.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.630+0000 7eff5632d700 1 -- 172.21.15.163:0/2527656251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff50159890 msgr2=0x7eff500c5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:06.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.630+0000 7eff5632d700 1 --2- 172.21.15.163:0/2527656251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff50159890 0x7eff500c5640 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7eff4000c7d0 tx=0x7eff40008040 comp rx=0 tx=0).stop 2023-10-09T06:39:06.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.630+0000 7eff5632d700 1 -- 172.21.15.163:0/2527656251 shutdown_connections 2023-10-09T06:39:06.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.630+0000 7eff5632d700 1 --2- 172.21.15.163:0/2527656251 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7eff3c067320 0x7eff3c0697d0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:06.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.630+0000 7eff5632d700 1 --2- 172.21.15.163:0/2527656251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff50159890 0x7eff500c5640 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:06.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.630+0000 7eff5632d700 1 --2- 172.21.15.163:0/2527656251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff5015db10 0x7eff5015c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:06.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.630+0000 7eff5632d700 1 --2- 172.21.15.163:0/2527656251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff50159c60 0x7eff5016dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:06.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.630+0000 7eff5632d700 1 -- 172.21.15.163:0/2527656251 >> 172.21.15.163:0/2527656251 conn(0x7eff50068e20 msgr2=0x7eff5014a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:06.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.630+0000 7eff5632d700 1 -- 172.21.15.163:0/2527656251 shutdown_connections 2023-10-09T06:39:06.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.630+0000 7eff5632d700 1 -- 172.21.15.163:0/2527656251 wait complete. 2023-10-09T06:39:06.643 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-09T06:39:06.644 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:39:06.685 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:39:06.728 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-09T06:39:06.728 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:39:06.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.845+0000 7fcad20d4700 1 Processor -- start 2023-10-09T06:39:06.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.845+0000 7fcad20d4700 1 -- start start 2023-10-09T06:39:06.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.846+0000 7fcad20d4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcacc159c60 0x7fcacc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:06.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.846+0000 7fcad20d4700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcacc15db10 0x7fcacc069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:06.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.846+0000 7fcad20d4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcacc15a160 con 0x7fcacc159890 2023-10-09T06:39:06.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.846+0000 7fcad20d4700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcacc15df50 con 0x7fcacc15db10 2023-10-09T06:39:06.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.846+0000 7fcad20d4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcacc0d13a0 con 0x7fcacc159c60 2023-10-09T06:39:06.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.846+0000 7fcacbfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcacc15db10 0x7fcacc069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:06.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.846+0000 7fcacbfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcacc15db10 0x7fcacc069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42506/0 (socket says 172.21.15.163:42506) 2023-10-09T06:39:06.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.846+0000 7fcacbfff700 1 -- 172.21.15.163:0/585000985 learned_addr learned my addr 172.21.15.163:0/585000985 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:06.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.847+0000 7fcacaffd700 1 --2- 172.21.15.163:0/585000985 >> v2:172.21.15.181:3300/0 conn(0x7fcacc159c60 0x7fcacc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:06.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.847+0000 7fcacbfff700 1 -- 172.21.15.163:0/585000985 >> v2:172.21.15.181:3300/0 conn(0x7fcacc159c60 msgr2=0x7fcacc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:06.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.847+0000 7fcacbfff700 1 --2- 172.21.15.163:0/585000985 >> v2:172.21.15.181:3300/0 conn(0x7fcacc159c60 0x7fcacc15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:06.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.847+0000 7fcacbfff700 1 -- 172.21.15.163:0/585000985 >> v1:172.21.15.181:6789/0 conn(0x7fcacc159890 legacy=0x7fcacc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:06.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.847+0000 7fcacbfff700 1 -- 172.21.15.163:0/585000985 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcacc15e140 con 0x7fcacc15db10 2023-10-09T06:39:06.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.847+0000 7fcacbfff700 1 --2- 172.21.15.163:0/585000985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcacc15db10 0x7fcacc069560 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0b67f0 tx=0x7fcac4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea6f2316a3209f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:06.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.847+0000 7fcaca7fc700 1 -- 172.21.15.163:0/585000985 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac400e070 con 0x7fcacc15db10 2023-10-09T06:39:06.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.847+0000 7fcaca7fc700 1 -- 172.21.15.163:0/585000985 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac400ba20 con 0x7fcacc15db10 2023-10-09T06:39:06.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.847+0000 7fcaca7fc700 1 -- 172.21.15.163:0/585000985 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac40092e0 con 0x7fcacc15db10 2023-10-09T06:39:06.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.848+0000 7fcad20d4700 1 -- 172.21.15.163:0/585000985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcacc15db10 msgr2=0x7fcacc069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:06.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.848+0000 7fcad20d4700 1 --2- 172.21.15.163:0/585000985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcacc15db10 0x7fcacc069560 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0b67f0 tx=0x7fcac4009840 comp rx=0 tx=0).stop 2023-10-09T06:39:06.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.848+0000 7fcad20d4700 1 -- 172.21.15.163:0/585000985 shutdown_connections 2023-10-09T06:39:06.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.848+0000 7fcad20d4700 1 --2- 172.21.15.163:0/585000985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcacc15db10 0x7fcacc069560 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:06.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.848+0000 7fcad20d4700 1 --2- 172.21.15.163:0/585000985 >> v2:172.21.15.181:3300/0 conn(0x7fcacc159c60 0x7fcacc15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:06.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.848+0000 7fcad20d4700 1 -- 172.21.15.163:0/585000985 >> 172.21.15.163:0/585000985 conn(0x7fcacc068e20 msgr2=0x7fcacc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:06.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.848+0000 7fcad20d4700 1 -- 172.21.15.163:0/585000985 shutdown_connections 2023-10-09T06:39:06.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.848+0000 7fcad20d4700 1 -- 172.21.15.163:0/585000985 wait complete. 2023-10-09T06:39:06.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.849+0000 7fcad20d4700 1 Processor -- start 2023-10-09T06:39:06.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.849+0000 7fcad20d4700 1 -- start start 2023-10-09T06:39:06.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.850+0000 7fcad20d4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcacc159890 0x7fcacc152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:06.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.850+0000 7fcad20d4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcacc159c60 0x7fcacc0ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:06.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.850+0000 7fcad20d4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcacc15db10 0x7fcacc15c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:06.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.850+0000 7fcad20d4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc0cf730 con 0x7fcacc15db10 2023-10-09T06:39:06.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.850+0000 7fcad20d4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc0cd560 con 0x7fcacc159c60 2023-10-09T06:39:06.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.850+0000 7fcad20d4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc0cf170 con 0x7fcacc159890 2023-10-09T06:39:06.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.850+0000 7fcacbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcacc15db10 0x7fcacc15c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:06.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.850+0000 7fcacbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcacc15db10 0x7fcacc15c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42514/0 (socket says 172.21.15.163:42514) 2023-10-09T06:39:06.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.850+0000 7fcacbfff700 1 -- 172.21.15.163:0/3428056869 learned_addr learned my addr 172.21.15.163:0/3428056869 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:06.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.850+0000 7fcacaffd700 1 --2- 172.21.15.163:0/3428056869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcacc159c60 0x7fcacc0ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:06.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.850+0000 7fcacb7fe700 1 --2- 172.21.15.163:0/3428056869 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcacc159890 0x7fcacc152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:06.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.850+0000 7fcacbfff700 1 -- 172.21.15.163:0/3428056869 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcacc159890 msgr2=0x7fcacc152560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:06.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.850+0000 7fcacbfff700 1 --2- 172.21.15.163:0/3428056869 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcacc159890 0x7fcacc152560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:06.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.850+0000 7fcacbfff700 1 -- 172.21.15.163:0/3428056869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcacc159c60 msgr2=0x7fcacc0ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:06.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.850+0000 7fcacbfff700 1 --2- 172.21.15.163:0/3428056869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcacc159c60 0x7fcacc0ceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:06.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.850+0000 7fcacbfff700 1 -- 172.21.15.163:0/3428056869 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac4009ae0 con 0x7fcacc15db10 2023-10-09T06:39:06.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.850+0000 7fcacbfff700 1 --2- 172.21.15.163:0/3428056869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcacc15db10 0x7fcacc15c0a0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009880 tx=0x7fcac4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:06.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.850+0000 7fcab37fe700 1 -- 172.21.15.163:0/3428056869 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac4008a80 con 0x7fcacc15db10 2023-10-09T06:39:06.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.850+0000 7fcab37fe700 1 -- 172.21.15.163:0/3428056869 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac4008be0 con 0x7fcacc15db10 2023-10-09T06:39:06.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.850+0000 7fcad20d4700 1 -- 172.21.15.163:0/3428056869 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcacc0cf450 con 0x7fcacc15db10 2023-10-09T06:39:06.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.850+0000 7fcab37fe700 1 -- 172.21.15.163:0/3428056869 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac4025460 con 0x7fcacc15db10 2023-10-09T06:39:06.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.851+0000 7fcad20d4700 1 -- 172.21.15.163:0/3428056869 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcacc0cd9f0 con 0x7fcacc15db10 2023-10-09T06:39:06.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.851+0000 7fcad20d4700 1 -- 172.21.15.163:0/3428056869 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcacc0cd710 con 0x7fcacc15db10 2023-10-09T06:39:06.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.852+0000 7fcab37fe700 1 -- 172.21.15.163:0/3428056869 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fcac4003040 con 0x7fcacc15db10 2023-10-09T06:39:06.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.852+0000 7fcab37fe700 1 --2- 172.21.15.163:0/3428056869 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcab8067320 0x7fcab80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:06.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.852+0000 7fcab37fe700 1 -- 172.21.15.163:0/3428056869 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6180+0+0 (secure 0 0 0) 0x7fcac4086480 con 0x7fcacc15db10 2023-10-09T06:39:06.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.853+0000 7fcacb7fe700 1 --2- 172.21.15.163:0/3428056869 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcab8067320 0x7fcab80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:06.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.853+0000 7fcacb7fe700 1 --2- 172.21.15.163:0/3428056869 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcab8067320 0x7fcab80697d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fcabc003a80 tx=0x7fcabc007ec0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:06.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:06.857+0000 7fcab37fe700 1 -- 172.21.15.163:0/3428056869 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcacc0cd710 con 0x7fcacc15db10 2023-10-09T06:39:07.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.081+0000 7fcad20d4700 1 -- 172.21.15.163:0/3428056869 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcacc0cd3b0 con 0x7fcacc15db10 2023-10-09T06:39:07.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.081+0000 7fcab37fe700 1 -- 172.21.15.163:0/3428056869 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+4639 (secure 0 0 0) 0x7fcac4052e90 con 0x7fcacc15db10 2023-10-09T06:39:07.082 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:07.083 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":115,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8211,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3056573272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3056573272},{"type":"v1","addr":"172.21.15.163:6834","nonce":3056573272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":8214,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3482861499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3482861499},{"type":"v1","addr":"172.21.15.163:6835","nonce":3482861499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":8223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/4239124825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4239124825},{"type":"v1","addr":"172.21.15.181:6836","nonce":4239124825}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:38:49.480715+0000","modified":"2023-10-09T06:39:04.559594+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8226":{"gid":8226,"name":"b","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/3494162891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3494162891},{"type":"v1","addr":"172.21.15.181:6837","nonce":3494162891}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29,30],"metadata_pool":28,"enabled":true,"fs_name":"volume_0000000000836565","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-09T06:39:07.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.084+0000 7fcad20d4700 1 -- 172.21.15.163:0/3428056869 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcab8067320 msgr2=0x7fcab80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:07.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.084+0000 7fcad20d4700 1 --2- 172.21.15.163:0/3428056869 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcab8067320 0x7fcab80697d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fcabc003a80 tx=0x7fcabc007ec0 comp rx=0 tx=0).stop 2023-10-09T06:39:07.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.084+0000 7fcad20d4700 1 -- 172.21.15.163:0/3428056869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcacc15db10 msgr2=0x7fcacc15c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:07.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.084+0000 7fcad20d4700 1 --2- 172.21.15.163:0/3428056869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcacc15db10 0x7fcacc15c0a0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009880 tx=0x7fcac4018000 comp rx=0 tx=0).stop 2023-10-09T06:39:07.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.084+0000 7fcad20d4700 1 -- 172.21.15.163:0/3428056869 shutdown_connections 2023-10-09T06:39:07.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.084+0000 7fcad20d4700 1 --2- 172.21.15.163:0/3428056869 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fcab8067320 0x7fcab80697d0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:07.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.084+0000 7fcad20d4700 1 --2- 172.21.15.163:0/3428056869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcacc15db10 0x7fcacc15c0a0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:07.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.084+0000 7fcad20d4700 1 --2- 172.21.15.163:0/3428056869 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcacc159890 0x7fcacc152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:07.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.084+0000 7fcad20d4700 1 --2- 172.21.15.163:0/3428056869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcacc159c60 0x7fcacc0ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:07.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.084+0000 7fcad20d4700 1 -- 172.21.15.163:0/3428056869 >> 172.21.15.163:0/3428056869 conn(0x7fcacc068e20 msgr2=0x7fcacc1583b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:07.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.084+0000 7fcad20d4700 1 -- 172.21.15.163:0/3428056869 shutdown_connections 2023-10-09T06:39:07.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.084+0000 7fcad20d4700 1 -- 172.21.15.163:0/3428056869 wait complete. 2023-10-09T06:39:07.086 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 115 2023-10-09T06:39:07.097 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:39:07.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.212+0000 7f20ba1c0700 1 Processor -- start 2023-10-09T06:39:07.213 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.212+0000 7f20ba1c0700 1 -- start start 2023-10-09T06:39:07.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.213+0000 7f20ba1c0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b4159420 0x7f20b40b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:07.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.213+0000 7f20ba1c0700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20b40b2980 con 0x7f20b415d230 2023-10-09T06:39:07.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.213+0000 7f20ba1c0700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20b40b11d0 con 0x7f20b41597f0 2023-10-09T06:39:07.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.213+0000 7f20ba1c0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b40b14b0 con 0x7f20b4159420 2023-10-09T06:39:07.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.213+0000 7f20b99bf700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f20b415d230 0x7f20b40b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:50628/0 (socket says 172.21.15.163:50628) 2023-10-09T06:39:07.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.213+0000 7f20b99bf700 1 -- 172.21.15.163:0/1071512328 learned_addr learned my addr 172.21.15.163:0/1071512328 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:07.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.213+0000 7f20b3fff700 1 -- 172.21.15.163:0/1071512328 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2365895309 0 0) 0x7f20b40b2980 con 0x7f20b415d230 2023-10-09T06:39:07.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.213+0000 7f20b3fff700 1 -- 172.21.15.163:0/1071512328 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f209c003570 con 0x7f20b415d230 2023-10-09T06:39:07.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.213+0000 7f20b91be700 1 --2- 172.21.15.163:0/1071512328 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b4159420 0x7f20b40b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:07.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.213+0000 7f20b3fff700 1 -- 172.21.15.163:0/1071512328 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 891325759 0 0) 0x7f209c003570 con 0x7f20b415d230 2023-10-09T06:39:07.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.213+0000 7f20b3fff700 1 -- 172.21.15.163:0/1071512328 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b4159420 msgr2=0x7f20b40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:07.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.213+0000 7f20b3fff700 1 --2- 172.21.15.163:0/1071512328 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b4159420 0x7f20b40b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:07.217 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.213+0000 7f20b3fff700 1 -- 172.21.15.163:0/1071512328 >> v1:172.21.15.181:6789/0 conn(0x7f20b41597f0 legacy=0x7f20b40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:07.218 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.213+0000 7f20b3fff700 1 -- 172.21.15.163:0/1071512328 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b40b1790 con 0x7f20b415d230 2023-10-09T06:39:07.218 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.213+0000 7f20b3fff700 1 -- 172.21.15.163:0/1071512328 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f20a40017f0 con 0x7f20b415d230 2023-10-09T06:39:07.218 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.214+0000 7f20b3fff700 1 -- 172.21.15.163:0/1071512328 >> v1:172.21.15.163:6789/0 conn(0x7f20b415d230 legacy=0x7f20b40b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:07.218 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.214+0000 7f20b3fff700 1 --2- 172.21.15.163:0/1071512328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f209c003a90 0x7f209c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:07.218 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.214+0000 7f20b3fff700 1 --2- 172.21.15.163:0/1071512328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f209c004810 0x7f209c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:07.218 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.214+0000 7f20b3fff700 1 --2- 172.21.15.163:0/1071512328 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b4159420 0x7f209c0051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:07.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.214+0000 7f20b3fff700 1 -- 172.21.15.163:0/1071512328 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b40b14b0 con 0x7f209c004810 2023-10-09T06:39:07.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.214+0000 7f20b3fff700 1 -- 172.21.15.163:0/1071512328 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209c00bb60 con 0x7f209c003a90 2023-10-09T06:39:07.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.214+0000 7f20b3fff700 1 -- 172.21.15.163:0/1071512328 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f209c00bd20 con 0x7f20b4159420 2023-10-09T06:39:07.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.214+0000 7f20b89bd700 1 --2- 172.21.15.163:0/1071512328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f209c004810 0x7f209c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:07.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.214+0000 7f20b91be700 1 --2- 172.21.15.163:0/1071512328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f209c003a90 0x7f209c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:07.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.214+0000 7f20b99bf700 1 --2- 172.21.15.163:0/1071512328 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b4159420 0x7f209c0051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:07.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.214+0000 7f20b89bd700 1 -- 172.21.15.163:0/1071512328 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b4159420 msgr2=0x7f209c0051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:07.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.214+0000 7f20b89bd700 1 --2- 172.21.15.163:0/1071512328 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b4159420 0x7f209c0051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:07.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.214+0000 7f20b89bd700 1 -- 172.21.15.163:0/1071512328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f209c003a90 msgr2=0x7f209c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:07.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.214+0000 7f20b89bd700 1 --2- 172.21.15.163:0/1071512328 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f209c003a90 0x7f209c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:07.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.214+0000 7f20b89bd700 1 -- 172.21.15.163:0/1071512328 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209c003570 con 0x7f209c004810 2023-10-09T06:39:07.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.214+0000 7f20b89bd700 1 --2- 172.21.15.163:0/1071512328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f209c004810 0x7f209c004c80 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f20a80018b0 tx=0x7f20a8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb77399eb329e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:07.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.214+0000 7f20b3fff700 1 -- 172.21.15.163:0/1071512328 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a8012420 con 0x7f209c004810 2023-10-09T06:39:07.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.214+0000 7f20b3fff700 1 -- 172.21.15.163:0/1071512328 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a8012580 con 0x7f209c004810 2023-10-09T06:39:07.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.214+0000 7f20b3fff700 1 -- 172.21.15.163:0/1071512328 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a8012920 con 0x7f209c004810 2023-10-09T06:39:07.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.215+0000 7f20ba1c0700 1 -- 172.21.15.163:0/1071512328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f209c004810 msgr2=0x7f209c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:07.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.215+0000 7f20ba1c0700 1 --2- 172.21.15.163:0/1071512328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f209c004810 0x7f209c004c80 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f20a80018b0 tx=0x7f20a8007d30 comp rx=0 tx=0).stop 2023-10-09T06:39:07.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.215+0000 7f20b99bf700 1 -- 172.21.15.163:0/1071512328 reap_dead start 2023-10-09T06:39:07.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.215+0000 7f20ba1c0700 1 -- 172.21.15.163:0/1071512328 shutdown_connections 2023-10-09T06:39:07.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.215+0000 7f20ba1c0700 1 -- 172.21.15.163:0/1071512328 >> 172.21.15.163:0/1071512328 conn(0x7f20b40d0660 msgr2=0x7f20b40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:07.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.215+0000 7f20ba1c0700 1 -- 172.21.15.163:0/1071512328 shutdown_connections 2023-10-09T06:39:07.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.215+0000 7f20ba1c0700 1 -- 172.21.15.163:0/1071512328 wait complete. 2023-10-09T06:39:07.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.216+0000 7f20ba1c0700 1 Processor -- start 2023-10-09T06:39:07.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.216+0000 7f20ba1c0700 1 -- start start 2023-10-09T06:39:07.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.217+0000 7f20ba1c0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f209c004810 0x7f20b41676f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:07.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.217+0000 7f20ba1c0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b416d0b0 0x7f20b416d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:07.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.217+0000 7f20ba1c0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20b415baa0 0x7f20b415bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:07.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.217+0000 7f20ba1c0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b416da60 con 0x7f209c004810 2023-10-09T06:39:07.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.217+0000 7f20ba1c0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b416bc50 con 0x7f20b415baa0 2023-10-09T06:39:07.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.217+0000 7f20ba1c0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b416b040 con 0x7f20b416d0b0 2023-10-09T06:39:07.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.217+0000 7f20b91be700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f209c004810 0x7f20b41676f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:07.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.217+0000 7f20b91be700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f209c004810 0x7f20b41676f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42540/0 (socket says 172.21.15.163:42540) 2023-10-09T06:39:07.226 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.217+0000 7f20b91be700 1 -- 172.21.15.163:0/3221891825 learned_addr learned my addr 172.21.15.163:0/3221891825 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:07.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.217+0000 7f20b99bf700 1 --2- 172.21.15.163:0/3221891825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20b415baa0 0x7f20b415bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:07.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.217+0000 7f20b91be700 1 -- 172.21.15.163:0/3221891825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b416d0b0 msgr2=0x7f20b416d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:07.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.217+0000 7f20b89bd700 1 --2- 172.21.15.163:0/3221891825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b416d0b0 0x7f20b416d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:07.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.217+0000 7f20b91be700 1 --2- 172.21.15.163:0/3221891825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b416d0b0 0x7f20b416d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:07.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.217+0000 7f20b91be700 1 -- 172.21.15.163:0/3221891825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20b415baa0 msgr2=0x7f20b415bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:07.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.217+0000 7f20b91be700 1 --2- 172.21.15.163:0/3221891825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20b415baa0 0x7f20b415bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:07.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.217+0000 7f20b91be700 1 -- 172.21.15.163:0/3221891825 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b416b320 con 0x7f209c004810 2023-10-09T06:39:07.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.217+0000 7f20b99bf700 1 --2- 172.21.15.163:0/3221891825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20b415baa0 0x7f20b415bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:07.227 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.217+0000 7f20b89bd700 1 --2- 172.21.15.163:0/3221891825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b416d0b0 0x7f20b416d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:07.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.218+0000 7f20b91be700 1 --2- 172.21.15.163:0/3221891825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f209c004810 0x7f20b41676f0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f20ac011d40 tx=0x7f20ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:07.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.218+0000 7f20b17fa700 1 -- 172.21.15.163:0/3221891825 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac00fac0 con 0x7f209c004810 2023-10-09T06:39:07.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.218+0000 7f20b17fa700 1 -- 172.21.15.163:0/3221891825 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ac00fc20 con 0x7f209c004810 2023-10-09T06:39:07.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.218+0000 7f20ba1c0700 1 -- 172.21.15.163:0/3221891825 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b416b600 con 0x7f209c004810 2023-10-09T06:39:07.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.218+0000 7f20b17fa700 1 -- 172.21.15.163:0/3221891825 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac014440 con 0x7f209c004810 2023-10-09T06:39:07.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.218+0000 7f20ba1c0700 1 -- 172.21.15.163:0/3221891825 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b416ac80 con 0x7f209c004810 2023-10-09T06:39:07.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.219+0000 7f20ba1c0700 1 -- 172.21.15.163:0/3221891825 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b416baa0 con 0x7f209c004810 2023-10-09T06:39:07.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.219+0000 7f20b17fa700 1 -- 172.21.15.163:0/3221891825 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f20ac014a80 con 0x7f209c004810 2023-10-09T06:39:07.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.219+0000 7f20b17fa700 1 --2- 172.21.15.163:0/3221891825 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f20a0067320 0x7f20a00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:07.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.219+0000 7f20b17fa700 1 -- 172.21.15.163:0/3221891825 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6180+0+0 (secure 0 0 0) 0x7f20ac018030 con 0x7f209c004810 2023-10-09T06:39:07.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.220+0000 7f20b89bd700 1 --2- 172.21.15.163:0/3221891825 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f20a0067320 0x7f20a00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:07.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.220+0000 7f20b89bd700 1 --2- 172.21.15.163:0/3221891825 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f20a0067320 0x7f20a00697d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f20a80018b0 tx=0x7f20a8015000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:07.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.223+0000 7f20b17fa700 1 -- 172.21.15.163:0/3221891825 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f20ac052630 con 0x7f209c004810 2023-10-09T06:39:07.407 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.407+0000 7f20ba1c0700 1 -- 172.21.15.163:0/3221891825 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20b404c3c0 con 0x7f209c004810 2023-10-09T06:39:07.420 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.407+0000 7f20b17fa700 1 -- 172.21.15.163:0/3221891825 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) v1 ==== 74+0+17053 (secure 0 0 0) 0x7f20ac022020 con 0x7f209c004810 2023-10-09T06:39:07.420 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:07.420 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":169,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:39:05.560867+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":28,"pool_name":"cephfs.volume_0000000000836565.meta","create_time":"2023-10-09T06:38:47.806008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000836565"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-10-09T06:38:48.446181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000836565"}}},{"pool":30,"pool_name":"another-data-pool","create_time":"2023-10-09T06:39:00.367621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"volume_0000000000836565"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:39:07.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.410+0000 7f20ba1c0700 1 -- 172.21.15.163:0/3221891825 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f20a0067320 msgr2=0x7f20a00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:07.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.410+0000 7f20ba1c0700 1 --2- 172.21.15.163:0/3221891825 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f20a0067320 0x7f20a00697d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f20a80018b0 tx=0x7f20a8015000 comp rx=0 tx=0).stop 2023-10-09T06:39:07.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.410+0000 7f20ba1c0700 1 -- 172.21.15.163:0/3221891825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f209c004810 msgr2=0x7f20b41676f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:07.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.410+0000 7f20ba1c0700 1 --2- 172.21.15.163:0/3221891825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f209c004810 0x7f20b41676f0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f20ac011d40 tx=0x7f20ac0091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:07.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.410+0000 7f20ba1c0700 1 -- 172.21.15.163:0/3221891825 shutdown_connections 2023-10-09T06:39:07.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.410+0000 7f20ba1c0700 1 --2- 172.21.15.163:0/3221891825 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f20a0067320 0x7f20a00697d0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:07.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.410+0000 7f20ba1c0700 1 --2- 172.21.15.163:0/3221891825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f209c004810 0x7f20b41676f0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:07.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.410+0000 7f20ba1c0700 1 --2- 172.21.15.163:0/3221891825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20b415baa0 0x7f20b415bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:07.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.410+0000 7f20ba1c0700 1 --2- 172.21.15.163:0/3221891825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20b416d0b0 0x7f20b416d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:07.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.410+0000 7f20ba1c0700 1 -- 172.21.15.163:0/3221891825 >> 172.21.15.163:0/3221891825 conn(0x7f20b40d0660 msgr2=0x7f20b4155df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:07.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.410+0000 7f20ba1c0700 1 -- 172.21.15.163:0/3221891825 shutdown_connections 2023-10-09T06:39:07.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.410+0000 7f20ba1c0700 1 -- 172.21.15.163:0/3221891825 wait complete. 2023-10-09T06:39:07.423 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 28, 'pool_name': 'cephfs.volume_0000000000836565.meta', 'create_time': '2023-10-09T06:38:47.806008+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '168', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'volume_0000000000836565'}}}, {'pool': 29, 'pool_name': 'cephfs_data', 'create_time': '2023-10-09T06:38:48.446181+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '168', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'volume_0000000000836565'}}}, {'pool': 30, 'pool_name': 'another-data-pool', 'create_time': '2023-10-09T06:39:00.367621+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '168', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'volume_0000000000836565'}}}] 2023-10-09T06:39:07.424 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:39:07.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.540+0000 7f86fce44700 1 Processor -- start 2023-10-09T06:39:07.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.540+0000 7f86fce44700 1 -- start start 2023-10-09T06:39:07.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.541+0000 7f86fce44700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f86f806bcd0 0x7f86f80caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:07.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.541+0000 7f86fce44700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f86f815df50 0x7f86f815e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:07.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.541+0000 7f86fce44700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f815d7e0 con 0x7f86f806c0a0 2023-10-09T06:39:07.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.541+0000 7f86fce44700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f815dac0 con 0x7f86f806bcd0 2023-10-09T06:39:07.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.541+0000 7f86fce44700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f815aa30 con 0x7f86f815df50 2023-10-09T06:39:07.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.541+0000 7f86f659c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f86f806bcd0 0x7f86f80caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:07.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.541+0000 7f86f659c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f86f806bcd0 0x7f86f80caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42556/0 (socket says 172.21.15.163:42556) 2023-10-09T06:39:07.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.541+0000 7f86edd9b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f86f806c0a0 0x7f86f815a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:50630/0 (socket says 172.21.15.163:50630) 2023-10-09T06:39:07.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.541+0000 7f86f659c700 1 -- 172.21.15.163:0/484264514 learned_addr learned my addr 172.21.15.163:0/484264514 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:07.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.541+0000 7f86f6d9d700 1 --2- 172.21.15.163:0/484264514 >> v2:172.21.15.181:3300/0 conn(0x7f86f815df50 0x7f86f815e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:07.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.542+0000 7f86f659c700 1 -- 172.21.15.163:0/484264514 >> v2:172.21.15.181:3300/0 conn(0x7f86f815df50 msgr2=0x7f86f815e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:07.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.542+0000 7f86f659c700 1 --2- 172.21.15.163:0/484264514 >> v2:172.21.15.181:3300/0 conn(0x7f86f815df50 0x7f86f815e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:07.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.542+0000 7f86f659c700 1 -- 172.21.15.163:0/484264514 >> v1:172.21.15.163:6789/0 conn(0x7f86f806c0a0 legacy=0x7f86f815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:07.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.542+0000 7f86f659c700 1 -- 172.21.15.163:0/484264514 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f815ad10 con 0x7f86f806bcd0 2023-10-09T06:39:07.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.542+0000 7f86f5d9b700 1 -- 172.21.15.163:0/484264514 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4237937214 0 0) 0x7f86f815d7e0 con 0x7f86f806c0a0 2023-10-09T06:39:07.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.542+0000 7f86f659c700 1 --2- 172.21.15.163:0/484264514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86f806bcd0 0x7f86f80caa90 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f86f815d720 tx=0x7f86e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd779a613895f858 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:07.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.542+0000 7f86f5d9b700 1 -- 172.21.15.163:0/484264514 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e8009e70 con 0x7f86f806bcd0 2023-10-09T06:39:07.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.542+0000 7f86f5d9b700 1 -- 172.21.15.163:0/484264514 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e800b860 con 0x7f86f806bcd0 2023-10-09T06:39:07.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.542+0000 7f86f5d9b700 1 -- 172.21.15.163:0/484264514 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e800bc20 con 0x7f86f806bcd0 2023-10-09T06:39:07.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.542+0000 7f86fce44700 1 -- 172.21.15.163:0/484264514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86f806bcd0 msgr2=0x7f86f80caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:07.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.542+0000 7f86fce44700 1 --2- 172.21.15.163:0/484264514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86f806bcd0 0x7f86f80caa90 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f86f815d720 tx=0x7f86e8009250 comp rx=0 tx=0).stop 2023-10-09T06:39:07.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.542+0000 7f86fce44700 1 -- 172.21.15.163:0/484264514 shutdown_connections 2023-10-09T06:39:07.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.542+0000 7f86fce44700 1 --2- 172.21.15.163:0/484264514 >> v2:172.21.15.181:3300/0 conn(0x7f86f815df50 0x7f86f815e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:07.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.542+0000 7f86fce44700 1 --2- 172.21.15.163:0/484264514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86f806bcd0 0x7f86f80caa90 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:07.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.542+0000 7f86fce44700 1 -- 172.21.15.163:0/484264514 >> 172.21.15.163:0/484264514 conn(0x7f86f80ce600 msgr2=0x7f86f80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:07.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.542+0000 7f86fce44700 1 -- 172.21.15.163:0/484264514 shutdown_connections 2023-10-09T06:39:07.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.542+0000 7f86fce44700 1 -- 172.21.15.163:0/484264514 wait complete. 2023-10-09T06:39:07.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.544+0000 7f86fce44700 1 Processor -- start 2023-10-09T06:39:07.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.544+0000 7f86fce44700 1 -- start start 2023-10-09T06:39:07.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.544+0000 7f86fce44700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86f806bcd0 0x7f86f8169b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:07.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.544+0000 7f86fce44700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86f806c0a0 0x7f86f80c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:07.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.544+0000 7f86fce44700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86f815df50 0x7f86f816ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:07.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.544+0000 7f86fce44700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f816b6e0 con 0x7f86f806bcd0 2023-10-09T06:39:07.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.544+0000 7f86fce44700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f816c540 con 0x7f86f806c0a0 2023-10-09T06:39:07.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.544+0000 7f86fce44700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f80aa4a0 con 0x7f86f815df50 2023-10-09T06:39:07.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.544+0000 7f86f659c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86f806bcd0 0x7f86f8169b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:07.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.545+0000 7f86f659c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86f806bcd0 0x7f86f8169b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42564/0 (socket says 172.21.15.163:42564) 2023-10-09T06:39:07.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.545+0000 7f86f659c700 1 -- 172.21.15.163:0/3328850199 learned_addr learned my addr 172.21.15.163:0/3328850199 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:07.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.545+0000 7f86f6d9d700 1 --2- 172.21.15.163:0/3328850199 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86f815df50 0x7f86f816ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:07.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.545+0000 7f86edd9b700 1 --2- 172.21.15.163:0/3328850199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86f806c0a0 0x7f86f80c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:07.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.545+0000 7f86f659c700 1 -- 172.21.15.163:0/3328850199 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86f815df50 msgr2=0x7f86f816ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:07.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.545+0000 7f86f659c700 1 --2- 172.21.15.163:0/3328850199 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86f815df50 0x7f86f816ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:07.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.545+0000 7f86f659c700 1 -- 172.21.15.163:0/3328850199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86f806c0a0 msgr2=0x7f86f80c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:07.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.545+0000 7f86f659c700 1 --2- 172.21.15.163:0/3328850199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86f806c0a0 0x7f86f80c56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:07.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.545+0000 7f86f659c700 1 -- 172.21.15.163:0/3328850199 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e80094f0 con 0x7f86f806bcd0 2023-10-09T06:39:07.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.545+0000 7f86f6d9d700 1 --2- 172.21.15.163:0/3328850199 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86f815df50 0x7f86f816ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:07.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.545+0000 7f86edd9b700 1 --2- 172.21.15.163:0/3328850199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86f806c0a0 0x7f86f80c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:07.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.545+0000 7f86f659c700 1 --2- 172.21.15.163:0/3328850199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86f806bcd0 0x7f86f8169b10 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f86e800b3c0 tx=0x7f86e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:07.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.545+0000 7f86eeffd700 1 -- 172.21.15.163:0/3328850199 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e801f070 con 0x7f86f806bcd0 2023-10-09T06:39:07.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.545+0000 7f86eeffd700 1 -- 172.21.15.163:0/3328850199 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e8000bc0 con 0x7f86f806bcd0 2023-10-09T06:39:07.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.545+0000 7f86eeffd700 1 -- 172.21.15.163:0/3328850199 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e800bc00 con 0x7f86f806bcd0 2023-10-09T06:39:07.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.545+0000 7f86fce44700 1 -- 172.21.15.163:0/3328850199 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f816a050 con 0x7f86f806bcd0 2023-10-09T06:39:07.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.545+0000 7f86fce44700 1 -- 172.21.15.163:0/3328850199 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f816bc10 con 0x7f86f806bcd0 2023-10-09T06:39:07.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.546+0000 7f86fce44700 1 -- 172.21.15.163:0/3328850199 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f804c3c0 con 0x7f86f806bcd0 2023-10-09T06:39:07.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.546+0000 7f86eeffd700 1 -- 172.21.15.163:0/3328850199 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f86e8023070 con 0x7f86f806bcd0 2023-10-09T06:39:07.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.547+0000 7f86eeffd700 1 --2- 172.21.15.163:0/3328850199 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f86d8067320 0x7f86d80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:07.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.547+0000 7f86eeffd700 1 -- 172.21.15.163:0/3328850199 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6180+0+0 (secure 0 0 0) 0x7f86e8086100 con 0x7f86f806bcd0 2023-10-09T06:39:07.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.547+0000 7f86edd9b700 1 --2- 172.21.15.163:0/3328850199 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f86d8067320 0x7f86d80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:07.554 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.547+0000 7f86edd9b700 1 --2- 172.21.15.163:0/3328850199 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f86d8067320 0x7f86d80697d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f86e0007fd0 tx=0x7f86e0007e30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:07.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.551+0000 7f86eeffd700 1 -- 172.21.15.163:0/3328850199 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f86e8052bb0 con 0x7f86f806bcd0 2023-10-09T06:39:07.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.735+0000 7f86fce44700 1 -- 172.21.15.163:0/3328850199 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f86f815aff0 con 0x7f86f806bcd0 2023-10-09T06:39:07.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.736+0000 7f86eeffd700 1 -- 172.21.15.163:0/3328850199 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v169) v1 ==== 93+0+10 (secure 0 0 0) 0x7f86e8009300 con 0x7f86f806bcd0 2023-10-09T06:39:07.737 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:39:07.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.738+0000 7f86fce44700 1 -- 172.21.15.163:0/3328850199 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f86d8067320 msgr2=0x7f86d80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:07.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.738+0000 7f86fce44700 1 --2- 172.21.15.163:0/3328850199 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f86d8067320 0x7f86d80697d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f86e0007fd0 tx=0x7f86e0007e30 comp rx=0 tx=0).stop 2023-10-09T06:39:07.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.738+0000 7f86fce44700 1 -- 172.21.15.163:0/3328850199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86f806bcd0 msgr2=0x7f86f8169b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:07.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.738+0000 7f86fce44700 1 --2- 172.21.15.163:0/3328850199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86f806bcd0 0x7f86f8169b10 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f86e800b3c0 tx=0x7f86e8017000 comp rx=0 tx=0).stop 2023-10-09T06:39:07.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.738+0000 7f86fce44700 1 -- 172.21.15.163:0/3328850199 shutdown_connections 2023-10-09T06:39:07.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.738+0000 7f86fce44700 1 --2- 172.21.15.163:0/3328850199 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f86d8067320 0x7f86d80697d0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:07.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.738+0000 7f86fce44700 1 --2- 172.21.15.163:0/3328850199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86f806bcd0 0x7f86f8169b10 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:07.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.738+0000 7f86fce44700 1 --2- 172.21.15.163:0/3328850199 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86f815df50 0x7f86f816ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:07.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.738+0000 7f86fce44700 1 --2- 172.21.15.163:0/3328850199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86f806c0a0 0x7f86f80c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:07.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.738+0000 7f86fce44700 1 -- 172.21.15.163:0/3328850199 >> 172.21.15.163:0/3328850199 conn(0x7f86f80ce600 msgr2=0x7f86f80c8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:07.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.738+0000 7f86fce44700 1 -- 172.21.15.163:0/3328850199 shutdown_connections 2023-10-09T06:39:07.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.738+0000 7f86fce44700 1 -- 172.21.15.163:0/3328850199 wait complete. 2023-10-09T06:39:07.751 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000836565.meta pg_num 2023-10-09T06:39:07.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.868+0000 7fc2ed176700 1 Processor -- start 2023-10-09T06:39:07.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.868+0000 7fc2ed176700 1 -- start start 2023-10-09T06:39:07.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.869+0000 7fc2ed176700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc2e8159c60 0x7fc2e815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:07.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.869+0000 7fc2ed176700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc2e815db10 0x7fc2e8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:07.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.869+0000 7fc2ed176700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2e815a160 con 0x7fc2e8159890 2023-10-09T06:39:07.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.869+0000 7fc2ed176700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e815df50 con 0x7fc2e8159c60 2023-10-09T06:39:07.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.869+0000 7fc2ed176700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e80d13a0 con 0x7fc2e815db10 2023-10-09T06:39:07.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.869+0000 7fc2e659c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc2e8159c60 0x7fc2e815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:07.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.869+0000 7fc2e659c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc2e8159c60 0x7fc2e815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42570/0 (socket says 172.21.15.163:42570) 2023-10-09T06:39:07.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.869+0000 7fc2e659c700 1 -- 172.21.15.163:0/3236705421 learned_addr learned my addr 172.21.15.163:0/3236705421 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:07.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.869+0000 7fc2e759e700 1 --2- 172.21.15.163:0/3236705421 >> v2:172.21.15.181:3300/0 conn(0x7fc2e815db10 0x7fc2e8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:07.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.869+0000 7fc2e659c700 1 -- 172.21.15.163:0/3236705421 >> v2:172.21.15.181:3300/0 conn(0x7fc2e815db10 msgr2=0x7fc2e8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:07.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.869+0000 7fc2e659c700 1 --2- 172.21.15.163:0/3236705421 >> v2:172.21.15.181:3300/0 conn(0x7fc2e815db10 0x7fc2e8069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:07.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.869+0000 7fc2e659c700 1 -- 172.21.15.163:0/3236705421 >> v1:172.21.15.181:6789/0 conn(0x7fc2e8159890 legacy=0x7fc2e815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:07.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.869+0000 7fc2e659c700 1 -- 172.21.15.163:0/3236705421 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e815e140 con 0x7fc2e8159c60 2023-10-09T06:39:07.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.869+0000 7fc2e659c700 1 --2- 172.21.15.163:0/3236705421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2e8159c60 0x7fc2e815d5d0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fc2e806a890 tx=0x7fc2dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e849bb20e0167ff server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:07.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.869+0000 7fc2e5d9b700 1 -- 172.21.15.163:0/3236705421 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc00b820 con 0x7fc2e8159c60 2023-10-09T06:39:07.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.869+0000 7fc2e5d9b700 1 -- 172.21.15.163:0/3236705421 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2dc00b980 con 0x7fc2e8159c60 2023-10-09T06:39:07.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.869+0000 7fc2e5d9b700 1 -- 172.21.15.163:0/3236705421 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc00bd20 con 0x7fc2e8159c60 2023-10-09T06:39:07.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.870+0000 7fc2ed176700 1 -- 172.21.15.163:0/3236705421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2e8159c60 msgr2=0x7fc2e815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:07.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.870+0000 7fc2ed176700 1 --2- 172.21.15.163:0/3236705421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2e8159c60 0x7fc2e815d5d0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fc2e806a890 tx=0x7fc2dc009250 comp rx=0 tx=0).stop 2023-10-09T06:39:07.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.870+0000 7fc2ed176700 1 -- 172.21.15.163:0/3236705421 shutdown_connections 2023-10-09T06:39:07.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.870+0000 7fc2ed176700 1 --2- 172.21.15.163:0/3236705421 >> v2:172.21.15.181:3300/0 conn(0x7fc2e815db10 0x7fc2e8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:07.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.870+0000 7fc2ed176700 1 --2- 172.21.15.163:0/3236705421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2e8159c60 0x7fc2e815d5d0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:07.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.870+0000 7fc2ed176700 1 -- 172.21.15.163:0/3236705421 >> 172.21.15.163:0/3236705421 conn(0x7fc2e8068e20 msgr2=0x7fc2e80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:07.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.870+0000 7fc2ed176700 1 -- 172.21.15.163:0/3236705421 shutdown_connections 2023-10-09T06:39:07.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.870+0000 7fc2ed176700 1 -- 172.21.15.163:0/3236705421 wait complete. 2023-10-09T06:39:07.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.871+0000 7fc2ed176700 1 Processor -- start 2023-10-09T06:39:07.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.871+0000 7fc2ed176700 1 -- start start 2023-10-09T06:39:07.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.872+0000 7fc2ed176700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2e8159890 0x7fc2e80cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:07.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.872+0000 7fc2ed176700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2e8159c60 0x7fc2e80b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:07.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.872+0000 7fc2ed176700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2e815db10 0x7fc2e80c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:07.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.872+0000 7fc2ed176700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e816ad90 con 0x7fc2e8159c60 2023-10-09T06:39:07.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.872+0000 7fc2ed176700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e816a180 con 0x7fc2e815db10 2023-10-09T06:39:07.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.872+0000 7fc2ed176700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e816a460 con 0x7fc2e8159890 2023-10-09T06:39:07.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.872+0000 7fc2e659c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2e8159c60 0x7fc2e80b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:07.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.872+0000 7fc2e659c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2e8159c60 0x7fc2e80b37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42584/0 (socket says 172.21.15.163:42584) 2023-10-09T06:39:07.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.872+0000 7fc2e659c700 1 -- 172.21.15.163:0/2471830643 learned_addr learned my addr 172.21.15.163:0/2471830643 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:07.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.872+0000 7fc2e659c700 1 -- 172.21.15.163:0/2471830643 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2e8159890 msgr2=0x7fc2e80cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:07.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.872+0000 7fc2e759e700 1 --2- 172.21.15.163:0/2471830643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2e815db10 0x7fc2e80c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:07.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.872+0000 7fc2e6d9d700 1 --2- 172.21.15.163:0/2471830643 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2e8159890 0x7fc2e80cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:07.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.872+0000 7fc2e659c700 1 --2- 172.21.15.163:0/2471830643 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2e8159890 0x7fc2e80cd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:07.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.872+0000 7fc2e659c700 1 -- 172.21.15.163:0/2471830643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2e815db10 msgr2=0x7fc2e80c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:07.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.872+0000 7fc2e659c700 1 --2- 172.21.15.163:0/2471830643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2e815db10 0x7fc2e80c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:07.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.872+0000 7fc2e659c700 1 -- 172.21.15.163:0/2471830643 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2dc0094f0 con 0x7fc2e8159c60 2023-10-09T06:39:07.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.872+0000 7fc2e759e700 1 --2- 172.21.15.163:0/2471830643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2e815db10 0x7fc2e80c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:07.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.873+0000 7fc2e659c700 1 --2- 172.21.15.163:0/2471830643 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2e8159c60 0x7fc2e80b37e0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc0030c0 tx=0x7fc2dc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:07.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.873+0000 7fc2ceffd700 1 -- 172.21.15.163:0/2471830643 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc011e60 con 0x7fc2e8159c60 2023-10-09T06:39:07.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.873+0000 7fc2ceffd700 1 -- 172.21.15.163:0/2471830643 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2dc020070 con 0x7fc2e8159c60 2023-10-09T06:39:07.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.873+0000 7fc2ceffd700 1 -- 172.21.15.163:0/2471830643 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc01b6a0 con 0x7fc2e8159c60 2023-10-09T06:39:07.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.873+0000 7fc2ed176700 1 -- 172.21.15.163:0/2471830643 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e816a740 con 0x7fc2e8159c60 2023-10-09T06:39:07.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.873+0000 7fc2ed176700 1 -- 172.21.15.163:0/2471830643 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e8169e90 con 0x7fc2e8159c60 2023-10-09T06:39:07.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.874+0000 7fc2ed176700 1 -- 172.21.15.163:0/2471830643 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2e80b0540 con 0x7fc2e8159c60 2023-10-09T06:39:07.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.874+0000 7fc2ceffd700 1 -- 172.21.15.163:0/2471830643 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc2dc022030 con 0x7fc2e8159c60 2023-10-09T06:39:07.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.874+0000 7fc2ceffd700 1 --2- 172.21.15.163:0/2471830643 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc2d4067350 0x7fc2d4069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:07.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.874+0000 7fc2ceffd700 1 -- 172.21.15.163:0/2471830643 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6180+0+0 (secure 0 0 0) 0x7fc2dc086c70 con 0x7fc2e8159c60 2023-10-09T06:39:07.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.875+0000 7fc2e6d9d700 1 --2- 172.21.15.163:0/2471830643 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc2d4067350 0x7fc2d4069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:07.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.875+0000 7fc2e6d9d700 1 --2- 172.21.15.163:0/2471830643 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc2d4067350 0x7fc2d4069800 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fc2d80027e0 tx=0x7fc2d8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:07.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:07.879+0000 7fc2ceffd700 1 -- 172.21.15.163:0/2471830643 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc2e80b0540 con 0x7fc2e8159c60 2023-10-09T06:39:08.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.064+0000 7fc2ed176700 1 -- 172.21.15.163:0/2471830643 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000836565.meta", "var": "pg_num"} v 0) v1 -- 0x7fc2e80b0540 con 0x7fc2e8159c60 2023-10-09T06:39:08.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.064+0000 7fc2ceffd700 1 -- 172.21.15.163:0/2471830643 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000836565.meta", "var": "pg_num"}]=0 v169) v1 ==== 124+0+11 (secure 0 0 0) 0x7fc2e80b0540 con 0x7fc2e8159c60 2023-10-09T06:39:08.065 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 32 2023-10-09T06:39:08.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.066+0000 7fc2ed176700 1 -- 172.21.15.163:0/2471830643 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc2d4067350 msgr2=0x7fc2d4069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:08.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.066+0000 7fc2ed176700 1 --2- 172.21.15.163:0/2471830643 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc2d4067350 0x7fc2d4069800 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fc2d80027e0 tx=0x7fc2d8009040 comp rx=0 tx=0).stop 2023-10-09T06:39:08.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.066+0000 7fc2ed176700 1 -- 172.21.15.163:0/2471830643 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2e8159c60 msgr2=0x7fc2e80b37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:08.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.066+0000 7fc2ed176700 1 --2- 172.21.15.163:0/2471830643 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2e8159c60 0x7fc2e80b37e0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc0030c0 tx=0x7fc2dc016000 comp rx=0 tx=0).stop 2023-10-09T06:39:08.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.066+0000 7fc2ed176700 1 -- 172.21.15.163:0/2471830643 shutdown_connections 2023-10-09T06:39:08.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.066+0000 7fc2ed176700 1 --2- 172.21.15.163:0/2471830643 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc2d4067350 0x7fc2d4069800 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.066+0000 7fc2ed176700 1 --2- 172.21.15.163:0/2471830643 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2e815db10 0x7fc2e80c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.066+0000 7fc2ed176700 1 --2- 172.21.15.163:0/2471830643 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2e8159890 0x7fc2e80cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.066+0000 7fc2ed176700 1 --2- 172.21.15.163:0/2471830643 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2e8159c60 0x7fc2e80b37e0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.066+0000 7fc2ed176700 1 -- 172.21.15.163:0/2471830643 >> 172.21.15.163:0/2471830643 conn(0x7fc2e8068e20 msgr2=0x7fc2e8153200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:08.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.066+0000 7fc2ed176700 1 -- 172.21.15.163:0/2471830643 shutdown_connections 2023-10-09T06:39:08.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.066+0000 7fc2ed176700 1 -- 172.21.15.163:0/2471830643 wait complete. 2023-10-09T06:39:08.082 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-09T06:39:08.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.199+0000 7f85f04f4700 1 Processor -- start 2023-10-09T06:39:08.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.199+0000 7f85f04f4700 1 -- start start 2023-10-09T06:39:08.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.200+0000 7f85f04f4700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f85e8159990 0x7f85e80b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:08.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.200+0000 7f85f04f4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85e8159d60 0x7f85e815ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:08.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.200+0000 7f85f04f4700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f85e80b2020 0x7f85e806d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:08.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.200+0000 7f85f04f4700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e80b4520 con 0x7f85e8159990 2023-10-09T06:39:08.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.200+0000 7f85f04f4700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e815b1c0 con 0x7f85e80b2020 2023-10-09T06:39:08.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.200+0000 7f85f04f4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e80b4240 con 0x7f85e8159d60 2023-10-09T06:39:08.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.200+0000 7f85ee290700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f85e8159990 0x7f85e80b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:08.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.200+0000 7f85ee290700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f85e8159990 0x7f85e80b3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42596/0 (socket says 172.21.15.163:42596) 2023-10-09T06:39:08.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.200+0000 7f85ee290700 1 -- 172.21.15.163:0/3037858760 learned_addr learned my addr 172.21.15.163:0/3037858760 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:08.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.200+0000 7f85eda8f700 1 --2- 172.21.15.163:0/3037858760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85e8159d60 0x7f85e815ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:08.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.200+0000 7f85eea91700 1 --2- 172.21.15.163:0/3037858760 >> v2:172.21.15.181:3300/0 conn(0x7f85e80b2020 0x7f85e806d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:08.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.200+0000 7f85ee290700 1 -- 172.21.15.163:0/3037858760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85e8159d60 msgr2=0x7f85e815ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:08.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.200+0000 7f85ee290700 1 --2- 172.21.15.163:0/3037858760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85e8159d60 0x7f85e815ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.200+0000 7f85ee290700 1 -- 172.21.15.163:0/3037858760 >> v2:172.21.15.181:3300/0 conn(0x7f85e80b2020 msgr2=0x7f85e806d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:08.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.200+0000 7f85ee290700 1 --2- 172.21.15.163:0/3037858760 >> v2:172.21.15.181:3300/0 conn(0x7f85e80b2020 0x7f85e806d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.200+0000 7f85ee290700 1 -- 172.21.15.163:0/3037858760 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e80b1540 con 0x7f85e8159990 2023-10-09T06:39:08.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.200+0000 7f85eda8f700 1 --2- 172.21.15.163:0/3037858760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85e8159d60 0x7f85e815ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:08.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.200+0000 7f85eea91700 1 --2- 172.21.15.163:0/3037858760 >> v2:172.21.15.181:3300/0 conn(0x7f85e80b2020 0x7f85e806d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:08.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.201+0000 7f85ee290700 1 --2- 172.21.15.163:0/3037858760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85e8159990 0x7f85e80b3c40 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f85e80697d0 tx=0x7f85e4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9034e6b8415d7a4c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:08.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.201+0000 7f85ed28e700 1 -- 172.21.15.163:0/3037858760 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e400e070 con 0x7f85e8159990 2023-10-09T06:39:08.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.201+0000 7f85ed28e700 1 -- 172.21.15.163:0/3037858760 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e400ba50 con 0x7f85e8159990 2023-10-09T06:39:08.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.201+0000 7f85ed28e700 1 -- 172.21.15.163:0/3037858760 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e40092e0 con 0x7f85e8159990 2023-10-09T06:39:08.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.201+0000 7f85f04f4700 1 -- 172.21.15.163:0/3037858760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85e8159990 msgr2=0x7f85e80b3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:08.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.201+0000 7f85f04f4700 1 --2- 172.21.15.163:0/3037858760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85e8159990 0x7f85e80b3c40 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f85e80697d0 tx=0x7f85e4009840 comp rx=0 tx=0).stop 2023-10-09T06:39:08.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.201+0000 7f85f04f4700 1 -- 172.21.15.163:0/3037858760 shutdown_connections 2023-10-09T06:39:08.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.201+0000 7f85f04f4700 1 --2- 172.21.15.163:0/3037858760 >> v2:172.21.15.181:3300/0 conn(0x7f85e80b2020 0x7f85e806d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.201+0000 7f85f04f4700 1 --2- 172.21.15.163:0/3037858760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85e8159990 0x7f85e80b3c40 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.201+0000 7f85f04f4700 1 --2- 172.21.15.163:0/3037858760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85e8159d60 0x7f85e815ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.201+0000 7f85f04f4700 1 -- 172.21.15.163:0/3037858760 >> 172.21.15.163:0/3037858760 conn(0x7f85e806bb50 msgr2=0x7f85e806bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:08.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.201+0000 7f85f04f4700 1 -- 172.21.15.163:0/3037858760 shutdown_connections 2023-10-09T06:39:08.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.201+0000 7f85f04f4700 1 -- 172.21.15.163:0/3037858760 wait complete. 2023-10-09T06:39:08.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.202+0000 7f85f04f4700 1 Processor -- start 2023-10-09T06:39:08.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.202+0000 7f85f04f4700 1 -- start start 2023-10-09T06:39:08.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.203+0000 7f85f04f4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85e80b2020 0x7f85e816e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:08.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.203+0000 7f85f04f4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85e8159990 0x7f85e80adf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:08.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.203+0000 7f85f04f4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85e8159d60 0x7f85e814e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:08.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.203+0000 7f85f04f4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e80ae4c0 con 0x7f85e8159d60 2023-10-09T06:39:08.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.203+0000 7f85f04f4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e80af2e0 con 0x7f85e8159990 2023-10-09T06:39:08.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.203+0000 7f85f04f4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e80af6b0 con 0x7f85e80b2020 2023-10-09T06:39:08.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.203+0000 7f85eea91700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85e8159d60 0x7f85e814e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:08.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.203+0000 7f85eea91700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85e8159d60 0x7f85e814e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:42600/0 (socket says 172.21.15.163:42600) 2023-10-09T06:39:08.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.203+0000 7f85eea91700 1 -- 172.21.15.163:0/2624331188 learned_addr learned my addr 172.21.15.163:0/2624331188 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:08.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.203+0000 7f85eda8f700 1 --2- 172.21.15.163:0/2624331188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85e8159990 0x7f85e80adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:08.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.203+0000 7f85ee290700 1 --2- 172.21.15.163:0/2624331188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85e80b2020 0x7f85e816e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:08.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.204+0000 7f85eea91700 1 -- 172.21.15.163:0/2624331188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85e80b2020 msgr2=0x7f85e816e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:08.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.204+0000 7f85eea91700 1 --2- 172.21.15.163:0/2624331188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85e80b2020 0x7f85e816e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.204+0000 7f85eea91700 1 -- 172.21.15.163:0/2624331188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85e8159990 msgr2=0x7f85e80adf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:08.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.204+0000 7f85eea91700 1 --2- 172.21.15.163:0/2624331188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85e8159990 0x7f85e80adf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.204+0000 7f85eea91700 1 -- 172.21.15.163:0/2624331188 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e4009ae0 con 0x7f85e8159d60 2023-10-09T06:39:08.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.204+0000 7f85eea91700 1 --2- 172.21.15.163:0/2624331188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85e8159d60 0x7f85e814e3a0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f85e000c880 tx=0x7f85e0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:08.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.204+0000 7f85de7fc700 1 -- 172.21.15.163:0/2624331188 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e0011660 con 0x7f85e8159d60 2023-10-09T06:39:08.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.204+0000 7f85de7fc700 1 -- 172.21.15.163:0/2624331188 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e00117c0 con 0x7f85e8159d60 2023-10-09T06:39:08.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.204+0000 7f85f04f4700 1 -- 172.21.15.163:0/2624331188 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e80ae6d0 con 0x7f85e8159d60 2023-10-09T06:39:08.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.204+0000 7f85de7fc700 1 -- 172.21.15.163:0/2624331188 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e0011b80 con 0x7f85e8159d60 2023-10-09T06:39:08.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.204+0000 7f85f04f4700 1 -- 172.21.15.163:0/2624331188 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e80aec90 con 0x7f85e8159d60 2023-10-09T06:39:08.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.205+0000 7f85f04f4700 1 -- 172.21.15.163:0/2624331188 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e804c3c0 con 0x7f85e8159d60 2023-10-09T06:39:08.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.205+0000 7f85de7fc700 1 -- 172.21.15.163:0/2624331188 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f85e0021350 con 0x7f85e8159d60 2023-10-09T06:39:08.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.205+0000 7f85de7fc700 1 --2- 172.21.15.163:0/2624331188 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f85cc067320 0x7f85cc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:08.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.206+0000 7f85de7fc700 1 -- 172.21.15.163:0/2624331188 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6180+0+0 (secure 0 0 0) 0x7f85e0084cc0 con 0x7f85e8159d60 2023-10-09T06:39:08.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.206+0000 7f85ee290700 1 --2- 172.21.15.163:0/2624331188 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f85cc067320 0x7f85cc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:08.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.206+0000 7f85ee290700 1 --2- 172.21.15.163:0/2624331188 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f85cc067320 0x7f85cc0697d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009880 tx=0x7f85e4018000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:08.211 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.210+0000 7f85de7fc700 1 -- 172.21.15.163:0/2624331188 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f85e008b080 con 0x7f85e8159d60 2023-10-09T06:39:08.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.392+0000 7f85f04f4700 1 -- 172.21.15.163:0/2624331188 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f85e8069820 con 0x7f85e8159d60 2023-10-09T06:39:08.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.392+0000 7f85de7fc700 1 -- 172.21.15.163:0/2624331188 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v169) v1 ==== 100+0+11 (secure 0 0 0) 0x7f85e001e0a0 con 0x7f85e8159d60 2023-10-09T06:39:08.394 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 64 2023-10-09T06:39:08.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.394+0000 7f85f04f4700 1 -- 172.21.15.163:0/2624331188 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f85cc067320 msgr2=0x7f85cc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:08.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.394+0000 7f85f04f4700 1 --2- 172.21.15.163:0/2624331188 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f85cc067320 0x7f85cc0697d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009880 tx=0x7f85e4018000 comp rx=0 tx=0).stop 2023-10-09T06:39:08.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.394+0000 7f85f04f4700 1 -- 172.21.15.163:0/2624331188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85e8159d60 msgr2=0x7f85e814e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:08.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.394+0000 7f85f04f4700 1 --2- 172.21.15.163:0/2624331188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85e8159d60 0x7f85e814e3a0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f85e000c880 tx=0x7f85e0008040 comp rx=0 tx=0).stop 2023-10-09T06:39:08.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.394+0000 7f85f04f4700 1 -- 172.21.15.163:0/2624331188 shutdown_connections 2023-10-09T06:39:08.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.394+0000 7f85f04f4700 1 --2- 172.21.15.163:0/2624331188 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f85cc067320 0x7f85cc0697d0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.394+0000 7f85f04f4700 1 --2- 172.21.15.163:0/2624331188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85e8159d60 0x7f85e814e3a0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.394+0000 7f85f04f4700 1 --2- 172.21.15.163:0/2624331188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85e80b2020 0x7f85e816e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.394+0000 7f85f04f4700 1 --2- 172.21.15.163:0/2624331188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85e8159990 0x7f85e80adf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.394+0000 7f85f04f4700 1 -- 172.21.15.163:0/2624331188 >> 172.21.15.163:0/2624331188 conn(0x7f85e806bb50 msgr2=0x7f85e806d170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:08.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.395+0000 7f85f04f4700 1 -- 172.21.15.163:0/2624331188 shutdown_connections 2023-10-09T06:39:08.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.395+0000 7f85f04f4700 1 -- 172.21.15.163:0/2624331188 wait complete. 2023-10-09T06:39:08.407 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get another-data-pool pg_num 2023-10-09T06:39:08.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.524+0000 7f7be55a7700 1 Processor -- start 2023-10-09T06:39:08.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.524+0000 7f7be55a7700 1 -- start start 2023-10-09T06:39:08.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.525+0000 7f7be55a7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7be0141500 0x7f7be00c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:08.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.525+0000 7f7be55a7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7be006c0a0 0x7f7be00c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:08.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.525+0000 7f7be55a7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7be015c2a0 con 0x7f7be006bcd0 2023-10-09T06:39:08.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.525+0000 7f7be55a7700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7be00c9880 con 0x7f7be0141500 2023-10-09T06:39:08.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.525+0000 7f7be55a7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7be015b690 con 0x7f7be006c0a0 2023-10-09T06:39:08.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.525+0000 7f7bd67fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7be0141500 0x7f7be00c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:08.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.525+0000 7f7bdeffd700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7be006bcd0 0x7f7be0140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:43520/0 (socket says 172.21.15.163:43520) 2023-10-09T06:39:08.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.525+0000 7f7bd67fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7be0141500 0x7f7be00c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44620/0 (socket says 172.21.15.163:44620) 2023-10-09T06:39:08.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.525+0000 7f7bdeffd700 1 -- 172.21.15.163:0/1672227213 learned_addr learned my addr 172.21.15.163:0/1672227213 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:08.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.525+0000 7f7bdf7fe700 1 --2- 172.21.15.163:0/1672227213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7be006c0a0 0x7f7be00c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:08.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.526+0000 7f7bde7fc700 1 -- 172.21.15.163:0/1672227213 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2871543240 0 0) 0x7f7be015c2a0 con 0x7f7be006bcd0 2023-10-09T06:39:08.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.526+0000 7f7bd67fc700 1 -- 172.21.15.163:0/1672227213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7be006c0a0 msgr2=0x7f7be00c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:08.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.526+0000 7f7bd67fc700 1 --2- 172.21.15.163:0/1672227213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7be006c0a0 0x7f7be00c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.526+0000 7f7bd67fc700 1 -- 172.21.15.163:0/1672227213 >> v1:172.21.15.181:6789/0 conn(0x7f7be006bcd0 legacy=0x7f7be0140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:08.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.526+0000 7f7bd67fc700 1 -- 172.21.15.163:0/1672227213 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be015b970 con 0x7f7be0141500 2023-10-09T06:39:08.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.526+0000 7f7bd67fc700 1 --2- 172.21.15.163:0/1672227213 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7be0141500 0x7f7be00c8e00 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80098e0 tx=0x7f7bc80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dfaef8df321463a server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:08.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.526+0000 7f7bde7fc700 1 -- 172.21.15.163:0/1672227213 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc800b640 con 0x7f7be0141500 2023-10-09T06:39:08.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.526+0000 7f7bde7fc700 1 -- 172.21.15.163:0/1672227213 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc800b7a0 con 0x7f7be0141500 2023-10-09T06:39:08.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.526+0000 7f7bde7fc700 1 -- 172.21.15.163:0/1672227213 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc800bb40 con 0x7f7be0141500 2023-10-09T06:39:08.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.526+0000 7f7be55a7700 1 -- 172.21.15.163:0/1672227213 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7be0141500 msgr2=0x7f7be00c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:08.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.526+0000 7f7be55a7700 1 --2- 172.21.15.163:0/1672227213 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7be0141500 0x7f7be00c8e00 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80098e0 tx=0x7f7bc80091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.526+0000 7f7be55a7700 1 -- 172.21.15.163:0/1672227213 shutdown_connections 2023-10-09T06:39:08.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.526+0000 7f7be55a7700 1 --2- 172.21.15.163:0/1672227213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7be006c0a0 0x7f7be00c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.526+0000 7f7be55a7700 1 --2- 172.21.15.163:0/1672227213 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7be0141500 0x7f7be00c8e00 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.526+0000 7f7be55a7700 1 -- 172.21.15.163:0/1672227213 >> 172.21.15.163:0/1672227213 conn(0x7f7be00ce600 msgr2=0x7f7be00cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:08.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.526+0000 7f7be55a7700 1 -- 172.21.15.163:0/1672227213 shutdown_connections 2023-10-09T06:39:08.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.527+0000 7f7be55a7700 1 -- 172.21.15.163:0/1672227213 wait complete. 2023-10-09T06:39:08.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.528+0000 7f7be55a7700 1 Processor -- start 2023-10-09T06:39:08.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.528+0000 7f7be55a7700 1 -- start start 2023-10-09T06:39:08.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.528+0000 7f7be55a7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7be006bcd0 0x7f7be0156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:08.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.528+0000 7f7be55a7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7be006c0a0 0x7f7be0156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:08.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.528+0000 7f7be55a7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7be0141500 0x7f7be0156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:08.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.528+0000 7f7be55a7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be0069e60 con 0x7f7be006c0a0 2023-10-09T06:39:08.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.528+0000 7f7be55a7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be006ae10 con 0x7f7be006bcd0 2023-10-09T06:39:08.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.528+0000 7f7be55a7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7be006a980 con 0x7f7be0141500 2023-10-09T06:39:08.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.529+0000 7f7bd67fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7be006c0a0 0x7f7be0156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:08.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.529+0000 7f7bd67fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7be006c0a0 0x7f7be0156900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44622/0 (socket says 172.21.15.163:44622) 2023-10-09T06:39:08.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.529+0000 7f7bd67fc700 1 -- 172.21.15.163:0/1563556278 learned_addr learned my addr 172.21.15.163:0/1563556278 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:08.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.529+0000 7f7bdf7fe700 1 --2- 172.21.15.163:0/1563556278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7be0141500 0x7f7be0156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:08.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.529+0000 7f7bdeffd700 1 --2- 172.21.15.163:0/1563556278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7be006bcd0 0x7f7be0156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:08.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.529+0000 7f7bd67fc700 1 -- 172.21.15.163:0/1563556278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7be0141500 msgr2=0x7f7be0156e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:08.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.529+0000 7f7bd67fc700 1 --2- 172.21.15.163:0/1563556278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7be0141500 0x7f7be0156e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.529+0000 7f7bd67fc700 1 -- 172.21.15.163:0/1563556278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7be006bcd0 msgr2=0x7f7be0156150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:08.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.529+0000 7f7bd67fc700 1 --2- 172.21.15.163:0/1563556278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7be006bcd0 0x7f7be0156150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.529+0000 7f7bd67fc700 1 -- 172.21.15.163:0/1563556278 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc8009420 con 0x7f7be006c0a0 2023-10-09T06:39:08.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.529+0000 7f7bd67fc700 1 --2- 172.21.15.163:0/1563556278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7be006c0a0 0x7f7be0156900 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f7be0069220 tx=0x7f7bc8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:08.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.529+0000 7f7bd77fe700 1 -- 172.21.15.163:0/1563556278 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc800bb20 con 0x7f7be006c0a0 2023-10-09T06:39:08.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.529+0000 7f7bd77fe700 1 -- 172.21.15.163:0/1563556278 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc8000c90 con 0x7f7be006c0a0 2023-10-09T06:39:08.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.529+0000 7f7be55a7700 1 -- 172.21.15.163:0/1563556278 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7be006a050 con 0x7f7be006c0a0 2023-10-09T06:39:08.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.529+0000 7f7bd77fe700 1 -- 172.21.15.163:0/1563556278 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8011670 con 0x7f7be006c0a0 2023-10-09T06:39:08.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.529+0000 7f7be55a7700 1 -- 172.21.15.163:0/1563556278 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be006a610 con 0x7f7be006c0a0 2023-10-09T06:39:08.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.530+0000 7f7be55a7700 1 -- 172.21.15.163:0/1563556278 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7be006ab10 con 0x7f7be006c0a0 2023-10-09T06:39:08.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.531+0000 7f7bd77fe700 1 -- 172.21.15.163:0/1563556278 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7bc8011a20 con 0x7f7be006c0a0 2023-10-09T06:39:08.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.532+0000 7f7bd77fe700 1 --2- 172.21.15.163:0/1563556278 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7bc0067420 0x7f7bc00698d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:08.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.532+0000 7f7bd77fe700 1 -- 172.21.15.163:0/1563556278 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6180+0+0 (secure 0 0 0) 0x7f7bc80869e0 con 0x7f7be006c0a0 2023-10-09T06:39:08.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.532+0000 7f7bdeffd700 1 --2- 172.21.15.163:0/1563556278 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7bc0067420 0x7f7bc00698d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:08.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.532+0000 7f7bdeffd700 1 --2- 172.21.15.163:0/1563556278 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7bc0067420 0x7f7bc00698d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0007fd0 tx=0x7f7bd0007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:08.536 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.535+0000 7f7bd77fe700 1 -- 172.21.15.163:0/1563556278 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7bc8053450 con 0x7f7be006c0a0 2023-10-09T06:39:08.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.719+0000 7f7be55a7700 1 -- 172.21.15.163:0/1563556278 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "another-data-pool", "var": "pg_num"} v 0) v1 -- 0x7f7be015bc50 con 0x7f7be006c0a0 2023-10-09T06:39:08.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.719+0000 7f7bd77fe700 1 -- 172.21.15.163:0/1563556278 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "another-data-pool", "var": "pg_num"}]=0 v169) v1 ==== 106+0+11 (secure 0 0 0) 0x7f7bc8015020 con 0x7f7be006c0a0 2023-10-09T06:39:08.720 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 32 2023-10-09T06:39:08.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.721+0000 7f7be55a7700 1 -- 172.21.15.163:0/1563556278 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7bc0067420 msgr2=0x7f7bc00698d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:08.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.721+0000 7f7be55a7700 1 --2- 172.21.15.163:0/1563556278 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7bc0067420 0x7f7bc00698d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0007fd0 tx=0x7f7bd0007d30 comp rx=0 tx=0).stop 2023-10-09T06:39:08.722 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.721+0000 7f7be55a7700 1 -- 172.21.15.163:0/1563556278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7be006c0a0 msgr2=0x7f7be0156900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:08.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.721+0000 7f7be55a7700 1 --2- 172.21.15.163:0/1563556278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7be006c0a0 0x7f7be0156900 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f7be0069220 tx=0x7f7bc8017000 comp rx=0 tx=0).stop 2023-10-09T06:39:08.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.721+0000 7f7be55a7700 1 -- 172.21.15.163:0/1563556278 shutdown_connections 2023-10-09T06:39:08.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.721+0000 7f7be55a7700 1 --2- 172.21.15.163:0/1563556278 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7bc0067420 0x7f7bc00698d0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.721+0000 7f7be55a7700 1 --2- 172.21.15.163:0/1563556278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7be0141500 0x7f7be0156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.721+0000 7f7be55a7700 1 --2- 172.21.15.163:0/1563556278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7be006bcd0 0x7f7be0156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.721+0000 7f7be55a7700 1 --2- 172.21.15.163:0/1563556278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7be006c0a0 0x7f7be0156900 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.721+0000 7f7be55a7700 1 -- 172.21.15.163:0/1563556278 >> 172.21.15.163:0/1563556278 conn(0x7f7be00ce600 msgr2=0x7f7be00c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:08.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.721+0000 7f7be55a7700 1 -- 172.21.15.163:0/1563556278 shutdown_connections 2023-10-09T06:39:08.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.721+0000 7f7be55a7700 1 -- 172.21.15.163:0/1563556278 wait complete. 2023-10-09T06:39:08.735 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:39:08.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.848+0000 7f3e86e43700 1 Processor -- start 2023-10-09T06:39:08.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.848+0000 7f3e86e43700 1 -- start start 2023-10-09T06:39:08.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.849+0000 7f3e86e43700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3e80159420 0x7f3e800b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:08.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.849+0000 7f3e86e43700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e801597f0 0x7f3e800b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:08.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.849+0000 7f3e86e43700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3e8015d230 0x7f3e8015d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:08.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.849+0000 7f3e86e43700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e8015dbd0 con 0x7f3e8015d230 2023-10-09T06:39:08.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.849+0000 7f3e86e43700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e800b11d0 con 0x7f3e80159420 2023-10-09T06:39:08.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.849+0000 7f3e86e43700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e800b14b0 con 0x7f3e801597f0 2023-10-09T06:39:08.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.849+0000 7f3e86642700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3e8015d230 0x7f3e8015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:08.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.849+0000 7f3e86642700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3e8015d230 0x7f3e8015d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44638/0 (socket says 172.21.15.163:44638) 2023-10-09T06:39:08.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.849+0000 7f3e86642700 1 -- 172.21.15.163:0/3447773322 learned_addr learned my addr 172.21.15.163:0/3447773322 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:08.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.849+0000 7f3e85640700 1 --2- 172.21.15.163:0/3447773322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e801597f0 0x7f3e800b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:08.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.849+0000 7f3e85e41700 1 --2- 172.21.15.163:0/3447773322 >> v2:172.21.15.181:3300/0 conn(0x7f3e80159420 0x7f3e800b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:08.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.849+0000 7f3e86642700 1 -- 172.21.15.163:0/3447773322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e801597f0 msgr2=0x7f3e800b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:08.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.850+0000 7f3e86642700 1 --2- 172.21.15.163:0/3447773322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e801597f0 0x7f3e800b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.850+0000 7f3e86642700 1 -- 172.21.15.163:0/3447773322 >> v2:172.21.15.181:3300/0 conn(0x7f3e80159420 msgr2=0x7f3e800b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:08.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.850+0000 7f3e86642700 1 --2- 172.21.15.163:0/3447773322 >> v2:172.21.15.181:3300/0 conn(0x7f3e80159420 0x7f3e800b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.850+0000 7f3e86642700 1 -- 172.21.15.163:0/3447773322 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e800b1790 con 0x7f3e8015d230 2023-10-09T06:39:08.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.850+0000 7f3e85640700 1 --2- 172.21.15.163:0/3447773322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e801597f0 0x7f3e800b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:08.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.850+0000 7f3e85e41700 1 --2- 172.21.15.163:0/3447773322 >> v2:172.21.15.181:3300/0 conn(0x7f3e80159420 0x7f3e800b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:08.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.850+0000 7f3e86642700 1 --2- 172.21.15.163:0/3447773322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e8015d230 0x7f3e8015d690 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f3e800b2f20 tx=0x7f3e78009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b332f2de5f04d4d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:08.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.850+0000 7f3e84e3f700 1 -- 172.21.15.163:0/3447773322 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7800e070 con 0x7f3e8015d230 2023-10-09T06:39:08.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.850+0000 7f3e84e3f700 1 -- 172.21.15.163:0/3447773322 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7800ba50 con 0x7f3e8015d230 2023-10-09T06:39:08.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.850+0000 7f3e84e3f700 1 -- 172.21.15.163:0/3447773322 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e780092e0 con 0x7f3e8015d230 2023-10-09T06:39:08.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.850+0000 7f3e86e43700 1 -- 172.21.15.163:0/3447773322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e8015d230 msgr2=0x7f3e8015d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:08.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.850+0000 7f3e86e43700 1 --2- 172.21.15.163:0/3447773322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e8015d230 0x7f3e8015d690 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f3e800b2f20 tx=0x7f3e78009840 comp rx=0 tx=0).stop 2023-10-09T06:39:08.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.850+0000 7f3e86e43700 1 -- 172.21.15.163:0/3447773322 shutdown_connections 2023-10-09T06:39:08.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.850+0000 7f3e86e43700 1 --2- 172.21.15.163:0/3447773322 >> v2:172.21.15.181:3300/0 conn(0x7f3e80159420 0x7f3e800b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.850+0000 7f3e86e43700 1 --2- 172.21.15.163:0/3447773322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e8015d230 0x7f3e8015d690 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.850+0000 7f3e86e43700 1 --2- 172.21.15.163:0/3447773322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e801597f0 0x7f3e800b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.850+0000 7f3e86e43700 1 -- 172.21.15.163:0/3447773322 >> 172.21.15.163:0/3447773322 conn(0x7f3e800d0660 msgr2=0x7f3e800d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:08.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.850+0000 7f3e86e43700 1 -- 172.21.15.163:0/3447773322 shutdown_connections 2023-10-09T06:39:08.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.850+0000 7f3e86e43700 1 -- 172.21.15.163:0/3447773322 wait complete. 2023-10-09T06:39:08.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.852+0000 7f3e86e43700 1 Processor -- start 2023-10-09T06:39:08.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.852+0000 7f3e86e43700 1 -- start start 2023-10-09T06:39:08.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.852+0000 7f3e86e43700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e80159420 0x7f3e80154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:08.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.852+0000 7f3e86e43700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e801597f0 0x7f3e8015b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:08.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.852+0000 7f3e86e43700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e8015d230 0x7f3e800b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:08.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.852+0000 7f3e86e43700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8015bae0 con 0x7f3e80159420 2023-10-09T06:39:08.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.852+0000 7f3e86e43700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8015bdc0 con 0x7f3e8015d230 2023-10-09T06:39:08.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.852+0000 7f3e86e43700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8015c0a0 con 0x7f3e801597f0 2023-10-09T06:39:08.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.852+0000 7f3e85e41700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e80159420 0x7f3e80154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:08.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.852+0000 7f3e85e41700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e80159420 0x7f3e80154fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44654/0 (socket says 172.21.15.163:44654) 2023-10-09T06:39:08.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.852+0000 7f3e85e41700 1 -- 172.21.15.163:0/1627762930 learned_addr learned my addr 172.21.15.163:0/1627762930 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:08.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.853+0000 7f3e85640700 1 --2- 172.21.15.163:0/1627762930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e801597f0 0x7f3e8015b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:08.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.853+0000 7f3e86642700 1 --2- 172.21.15.163:0/1627762930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e8015d230 0x7f3e800b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:08.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.853+0000 7f3e85e41700 1 -- 172.21.15.163:0/1627762930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e801597f0 msgr2=0x7f3e8015b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:08.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.853+0000 7f3e85e41700 1 --2- 172.21.15.163:0/1627762930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e801597f0 0x7f3e8015b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.853+0000 7f3e85e41700 1 -- 172.21.15.163:0/1627762930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e8015d230 msgr2=0x7f3e800b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:08.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.853+0000 7f3e85e41700 1 --2- 172.21.15.163:0/1627762930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e8015d230 0x7f3e800b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:08.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.853+0000 7f3e85e41700 1 -- 172.21.15.163:0/1627762930 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e78009ae0 con 0x7f3e80159420 2023-10-09T06:39:08.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.853+0000 7f3e85e41700 1 --2- 172.21.15.163:0/1627762930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e80159420 0x7f3e80154fb0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c00e8d0 tx=0x7f3e7c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:08.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.853+0000 7f3e75ffb700 1 -- 172.21.15.163:0/1627762930 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c013410 con 0x7f3e80159420 2023-10-09T06:39:08.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.853+0000 7f3e75ffb700 1 -- 172.21.15.163:0/1627762930 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7c013570 con 0x7f3e80159420 2023-10-09T06:39:08.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.853+0000 7f3e86e43700 1 -- 172.21.15.163:0/1627762930 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e8015c380 con 0x7f3e80159420 2023-10-09T06:39:08.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.853+0000 7f3e75ffb700 1 -- 172.21.15.163:0/1627762930 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c013910 con 0x7f3e80159420 2023-10-09T06:39:08.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.853+0000 7f3e86e43700 1 -- 172.21.15.163:0/1627762930 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e800c58d0 con 0x7f3e80159420 2023-10-09T06:39:08.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.854+0000 7f3e86e43700 1 -- 172.21.15.163:0/1627762930 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e8004c3c0 con 0x7f3e80159420 2023-10-09T06:39:08.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.855+0000 7f3e75ffb700 1 -- 172.21.15.163:0/1627762930 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3e7c013d00 con 0x7f3e80159420 2023-10-09T06:39:08.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.855+0000 7f3e75ffb700 1 --2- 172.21.15.163:0/1627762930 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3e64067320 0x7f3e640697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:08.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.855+0000 7f3e75ffb700 1 -- 172.21.15.163:0/1627762930 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6180+0+0 (secure 0 0 0) 0x7f3e7c084c80 con 0x7f3e80159420 2023-10-09T06:39:08.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.855+0000 7f3e85640700 1 --2- 172.21.15.163:0/1627762930 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3e64067320 0x7f3e640697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:08.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.856+0000 7f3e85640700 1 --2- 172.21.15.163:0/1627762930 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3e64067320 0x7f3e640697d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3e800af1f0 tx=0x7f3e70008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:08.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:08.859+0000 7f3e75ffb700 1 -- 172.21.15.163:0/1627762930 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3e7c051620 con 0x7f3e80159420 2023-10-09T06:39:09.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.081+0000 7f3e86e43700 1 -- 172.21.15.163:0/1627762930 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e800b1a70 con 0x7f3e80159420 2023-10-09T06:39:09.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.082+0000 7f3e75ffb700 1 -- 172.21.15.163:0/1627762930 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+4639 (secure 0 0 0) 0x7f3e7c020090 con 0x7f3e80159420 2023-10-09T06:39:09.083 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:09.083 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":115,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8211,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3056573272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3056573272},{"type":"v1","addr":"172.21.15.163:6834","nonce":3056573272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":8214,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3482861499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3482861499},{"type":"v1","addr":"172.21.15.163:6835","nonce":3482861499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":8223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/4239124825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4239124825},{"type":"v1","addr":"172.21.15.181:6836","nonce":4239124825}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:38:49.480715+0000","modified":"2023-10-09T06:39:04.559594+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8226":{"gid":8226,"name":"b","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/3494162891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3494162891},{"type":"v1","addr":"172.21.15.181:6837","nonce":3494162891}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29,30],"metadata_pool":28,"enabled":true,"fs_name":"volume_0000000000836565","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-09T06:39:09.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.084+0000 7f3e86e43700 1 -- 172.21.15.163:0/1627762930 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3e64067320 msgr2=0x7f3e640697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:09.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.084+0000 7f3e86e43700 1 --2- 172.21.15.163:0/1627762930 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3e64067320 0x7f3e640697d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3e800af1f0 tx=0x7f3e70008040 comp rx=0 tx=0).stop 2023-10-09T06:39:09.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.084+0000 7f3e86e43700 1 -- 172.21.15.163:0/1627762930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e80159420 msgr2=0x7f3e80154fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:09.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.084+0000 7f3e86e43700 1 --2- 172.21.15.163:0/1627762930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e80159420 0x7f3e80154fb0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c00e8d0 tx=0x7f3e7c008040 comp rx=0 tx=0).stop 2023-10-09T06:39:09.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.084+0000 7f3e86e43700 1 -- 172.21.15.163:0/1627762930 shutdown_connections 2023-10-09T06:39:09.085 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.084+0000 7f3e86e43700 1 --2- 172.21.15.163:0/1627762930 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3e64067320 0x7f3e640697d0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:09.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.084+0000 7f3e86e43700 1 --2- 172.21.15.163:0/1627762930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e80159420 0x7f3e80154fb0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:09.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.084+0000 7f3e86e43700 1 --2- 172.21.15.163:0/1627762930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e8015d230 0x7f3e800b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:09.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.084+0000 7f3e86e43700 1 --2- 172.21.15.163:0/1627762930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e801597f0 0x7f3e8015b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:09.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.084+0000 7f3e86e43700 1 -- 172.21.15.163:0/1627762930 >> 172.21.15.163:0/1627762930 conn(0x7f3e800d0660 msgr2=0x7f3e80069ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:09.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.084+0000 7f3e86e43700 1 -- 172.21.15.163:0/1627762930 shutdown_connections 2023-10-09T06:39:09.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.084+0000 7f3e86e43700 1 -- 172.21.15.163:0/1627762930 wait complete. 2023-10-09T06:39:09.086 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 115 2023-10-09T06:39:09.097 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:39:09.212 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.211+0000 7f054511d700 1 Processor -- start 2023-10-09T06:39:09.212 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.211+0000 7f054511d700 1 -- start start 2023-10-09T06:39:09.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.212+0000 7f054511d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f054006c0a0 0x7f05400caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:09.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.212+0000 7f054511d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f054015d7e0 con 0x7f054006bcd0 2023-10-09T06:39:09.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.212+0000 7f054511d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f054015dac0 con 0x7f054015df50 2023-10-09T06:39:09.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.212+0000 7f054511d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f054015aa30 con 0x7f054006c0a0 2023-10-09T06:39:09.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.212+0000 7f053659c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f054006c0a0 0x7f05400caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:09.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.212+0000 7f053ed9d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f054006bcd0 0x7f054015a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:51894/0 (socket says 172.21.15.163:51894) 2023-10-09T06:39:09.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.212+0000 7f053ed9d700 1 -- 172.21.15.163:0/1779839045 learned_addr learned my addr 172.21.15.163:0/1779839045 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:09.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.213+0000 7f053659c700 1 -- 172.21.15.163:0/1779839045 >> v1:172.21.15.181:6789/0 conn(0x7f054015df50 legacy=0x7f05400c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:09.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.213+0000 7f053659c700 1 -- 172.21.15.163:0/1779839045 >> v1:172.21.15.163:6789/0 conn(0x7f054006bcd0 legacy=0x7f054015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:09.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.213+0000 7f053e59c700 1 -- 172.21.15.163:0/1779839045 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1891894565 0 0) 0x7f054015d7e0 con 0x7f054006bcd0 2023-10-09T06:39:09.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.213+0000 7f053659c700 1 -- 172.21.15.163:0/1779839045 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054015ad10 con 0x7f054006c0a0 2023-10-09T06:39:09.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.213+0000 7f053659c700 1 --2- 172.21.15.163:0/1779839045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f054006c0a0 0x7f05400caa90 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f05280098e0 tx=0x7f05280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e468309f59a7e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:09.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.213+0000 7f053e59c700 1 -- 172.21.15.163:0/1779839045 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0528009e80 con 0x7f054006c0a0 2023-10-09T06:39:09.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.213+0000 7f053e59c700 1 -- 172.21.15.163:0/1779839045 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052800b820 con 0x7f054006c0a0 2023-10-09T06:39:09.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.213+0000 7f053e59c700 1 -- 172.21.15.163:0/1779839045 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052800bba0 con 0x7f054006c0a0 2023-10-09T06:39:09.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.213+0000 7f054511d700 1 -- 172.21.15.163:0/1779839045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f054006c0a0 msgr2=0x7f05400caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:09.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.213+0000 7f054511d700 1 --2- 172.21.15.163:0/1779839045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f054006c0a0 0x7f05400caa90 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f05280098e0 tx=0x7f05280091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:09.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.213+0000 7f054511d700 1 -- 172.21.15.163:0/1779839045 shutdown_connections 2023-10-09T06:39:09.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.213+0000 7f054511d700 1 --2- 172.21.15.163:0/1779839045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f054006c0a0 0x7f05400caa90 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:09.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.213+0000 7f054511d700 1 -- 172.21.15.163:0/1779839045 >> 172.21.15.163:0/1779839045 conn(0x7f05400ce600 msgr2=0x7f05400cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:09.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.213+0000 7f054511d700 1 -- 172.21.15.163:0/1779839045 shutdown_connections 2023-10-09T06:39:09.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.213+0000 7f054511d700 1 -- 172.21.15.163:0/1779839045 wait complete. 2023-10-09T06:39:09.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.215+0000 7f054511d700 1 Processor -- start 2023-10-09T06:39:09.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.215+0000 7f054511d700 1 -- start start 2023-10-09T06:39:09.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.215+0000 7f054511d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f054006bcd0 0x7f0540154980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:09.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.215+0000 7f054511d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f054006c0a0 0x7f05400b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:09.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.215+0000 7f054511d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f054015df50 0x7f0540159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:09.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.215+0000 7f054511d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05400c7ae0 con 0x7f054015df50 2023-10-09T06:39:09.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.215+0000 7f054511d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05400c8c30 con 0x7f054006bcd0 2023-10-09T06:39:09.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.215+0000 7f054511d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0540159050 con 0x7f054006c0a0 2023-10-09T06:39:09.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.216+0000 7f053f59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f054015df50 0x7f0540159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:09.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.216+0000 7f053f59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f054015df50 0x7f0540159cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44668/0 (socket says 172.21.15.163:44668) 2023-10-09T06:39:09.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.216+0000 7f053f59e700 1 -- 172.21.15.163:0/3786632126 learned_addr learned my addr 172.21.15.163:0/3786632126 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:09.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.216+0000 7f053ed9d700 1 --2- 172.21.15.163:0/3786632126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f054006bcd0 0x7f0540154980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:09.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.216+0000 7f053659c700 1 --2- 172.21.15.163:0/3786632126 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f054006c0a0 0x7f05400b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:09.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.216+0000 7f053f59e700 1 -- 172.21.15.163:0/3786632126 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f054006c0a0 msgr2=0x7f05400b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:09.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.216+0000 7f053f59e700 1 --2- 172.21.15.163:0/3786632126 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f054006c0a0 0x7f05400b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:09.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.216+0000 7f053f59e700 1 -- 172.21.15.163:0/3786632126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f054006bcd0 msgr2=0x7f0540154980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:09.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.216+0000 7f053f59e700 1 --2- 172.21.15.163:0/3786632126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f054006bcd0 0x7f0540154980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:09.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.216+0000 7f053f59e700 1 -- 172.21.15.163:0/3786632126 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0528009420 con 0x7f054015df50 2023-10-09T06:39:09.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.216+0000 7f053ed9d700 1 --2- 172.21.15.163:0/3786632126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f054006bcd0 0x7f0540154980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:09.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.216+0000 7f053f59e700 1 --2- 172.21.15.163:0/3786632126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f054015df50 0x7f0540159cc0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f053800bc30 tx=0x7f0538009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:09.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.216+0000 7f05377fe700 1 -- 172.21.15.163:0/3786632126 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053800c9b0 con 0x7f054015df50 2023-10-09T06:39:09.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.216+0000 7f05377fe700 1 -- 172.21.15.163:0/3786632126 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053800cb50 con 0x7f054015df50 2023-10-09T06:39:09.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.216+0000 7f054511d700 1 -- 172.21.15.163:0/3786632126 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05400c8300 con 0x7f054015df50 2023-10-09T06:39:09.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.216+0000 7f05377fe700 1 -- 172.21.15.163:0/3786632126 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0538014410 con 0x7f054015df50 2023-10-09T06:39:09.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.216+0000 7f054511d700 1 -- 172.21.15.163:0/3786632126 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05400c8940 con 0x7f054015df50 2023-10-09T06:39:09.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.217+0000 7f054511d700 1 -- 172.21.15.163:0/3786632126 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05400c8de0 con 0x7f054015df50 2023-10-09T06:39:09.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.217+0000 7f05377fe700 1 -- 172.21.15.163:0/3786632126 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0538014800 con 0x7f054015df50 2023-10-09T06:39:09.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.218+0000 7f05377fe700 1 --2- 172.21.15.163:0/3786632126 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0520067320 0x7f05200697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:09.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.218+0000 7f05377fe700 1 -- 172.21.15.163:0/3786632126 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6180+0+0 (secure 0 0 0) 0x7f0538084f40 con 0x7f054015df50 2023-10-09T06:39:09.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.218+0000 7f053ed9d700 1 --2- 172.21.15.163:0/3786632126 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0520067320 0x7f05200697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:09.222 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.219+0000 7f053ed9d700 1 --2- 172.21.15.163:0/3786632126 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0520067320 0x7f05200697d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0530001890 tx=0x7f0530009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:09.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.223+0000 7f05377fe700 1 -- 172.21.15.163:0/3786632126 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05400c8de0 con 0x7f054015df50 2023-10-09T06:39:09.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.407+0000 7f054511d700 1 -- 172.21.15.163:0/3786632126 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05400c8090 con 0x7f054015df50 2023-10-09T06:39:09.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.408+0000 7f05377fe700 1 -- 172.21.15.163:0/3786632126 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) v1 ==== 74+0+17053 (secure 0 0 0) 0x7f05380519f0 con 0x7f054015df50 2023-10-09T06:39:09.409 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:09.409 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":169,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:39:05.560867+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":28,"pool_name":"cephfs.volume_0000000000836565.meta","create_time":"2023-10-09T06:38:47.806008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000836565"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-10-09T06:38:48.446181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000836565"}}},{"pool":30,"pool_name":"another-data-pool","create_time":"2023-10-09T06:39:00.367621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"volume_0000000000836565"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:39:09.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.411+0000 7f054511d700 1 -- 172.21.15.163:0/3786632126 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0520067320 msgr2=0x7f05200697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:09.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.411+0000 7f054511d700 1 --2- 172.21.15.163:0/3786632126 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0520067320 0x7f05200697d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0530001890 tx=0x7f0530009040 comp rx=0 tx=0).stop 2023-10-09T06:39:09.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.411+0000 7f054511d700 1 -- 172.21.15.163:0/3786632126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f054015df50 msgr2=0x7f0540159cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:09.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.411+0000 7f054511d700 1 --2- 172.21.15.163:0/3786632126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f054015df50 0x7f0540159cc0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f053800bc30 tx=0x7f0538009820 comp rx=0 tx=0).stop 2023-10-09T06:39:09.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.411+0000 7f054511d700 1 -- 172.21.15.163:0/3786632126 shutdown_connections 2023-10-09T06:39:09.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.411+0000 7f054511d700 1 --2- 172.21.15.163:0/3786632126 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0520067320 0x7f05200697d0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:09.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.411+0000 7f054511d700 1 --2- 172.21.15.163:0/3786632126 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f054015df50 0x7f0540159cc0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:09.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.411+0000 7f054511d700 1 --2- 172.21.15.163:0/3786632126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f054006bcd0 0x7f0540154980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:09.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.411+0000 7f054511d700 1 --2- 172.21.15.163:0/3786632126 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f054006c0a0 0x7f05400b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:09.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.411+0000 7f054511d700 1 -- 172.21.15.163:0/3786632126 >> 172.21.15.163:0/3786632126 conn(0x7f05400ce600 msgr2=0x7f05400cc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:09.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.411+0000 7f054511d700 1 -- 172.21.15.163:0/3786632126 shutdown_connections 2023-10-09T06:39:09.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.411+0000 7f054511d700 1 -- 172.21.15.163:0/3786632126 wait complete. 2023-10-09T06:39:09.423 INFO:tasks.cephfs.filesystem:Destroying file system volume_0000000000836565 and related pools 2023-10-09T06:39:09.424 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:39:09.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.540+0000 7fe92e51b700 1 Processor -- start 2023-10-09T06:39:09.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.540+0000 7fe92e51b700 1 -- start start 2023-10-09T06:39:09.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.540+0000 7fe92e51b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9280c91e0 0x7fe9280cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:09.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.540+0000 7fe92e51b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9280cc210 con 0x7fe92806aa40 2023-10-09T06:39:09.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.540+0000 7fe92e51b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9280cc4f0 con 0x7fe92806ae10 2023-10-09T06:39:09.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.540+0000 7fe92e51b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9280cc7d0 con 0x7fe9280c91e0 2023-10-09T06:39:09.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.540+0000 7fe927fff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fe92806aa40 0x7fe92806bac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:51900/0 (socket says 172.21.15.163:51900) 2023-10-09T06:39:09.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.540+0000 7fe927fff700 1 -- 172.21.15.163:0/441854248 learned_addr learned my addr 172.21.15.163:0/441854248 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:09.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.540+0000 7fe9277fe700 1 --2- 172.21.15.163:0/441854248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9280c91e0 0x7fe9280cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:09.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.540+0000 7fe926ffd700 1 -- 172.21.15.163:0/441854248 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2085039727 0 0) 0x7fe9280cc210 con 0x7fe92806aa40 2023-10-09T06:39:09.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.540+0000 7fe926ffd700 1 -- 172.21.15.163:0/441854248 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe910003570 con 0x7fe92806aa40 2023-10-09T06:39:09.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.540+0000 7fe9277fe700 1 -- 172.21.15.163:0/441854248 >> v1:172.21.15.181:6789/0 conn(0x7fe92806ae10 legacy=0x7fe92806b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:09.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.540+0000 7fe9277fe700 1 -- 172.21.15.163:0/441854248 >> v1:172.21.15.163:6789/0 conn(0x7fe92806aa40 legacy=0x7fe92806bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:09.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.540+0000 7fe9277fe700 1 -- 172.21.15.163:0/441854248 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9280ccab0 con 0x7fe9280c91e0 2023-10-09T06:39:09.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.540+0000 7fe9277fe700 1 --2- 172.21.15.163:0/441854248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9280c91e0 0x7fe9280cbc00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe918009890 tx=0x7fe9180091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4f765bca89648ed server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:09.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.541+0000 7fe926ffd700 1 -- 172.21.15.163:0/441854248 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe918011630 con 0x7fe9280c91e0 2023-10-09T06:39:09.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.541+0000 7fe926ffd700 1 -- 172.21.15.163:0/441854248 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe918011790 con 0x7fe9280c91e0 2023-10-09T06:39:09.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.541+0000 7fe926ffd700 1 -- 172.21.15.163:0/441854248 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe918011b50 con 0x7fe9280c91e0 2023-10-09T06:39:09.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.541+0000 7fe92e51b700 1 -- 172.21.15.163:0/441854248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9280c91e0 msgr2=0x7fe9280cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:09.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.541+0000 7fe92e51b700 1 --2- 172.21.15.163:0/441854248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9280c91e0 0x7fe9280cbc00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe918009890 tx=0x7fe9180091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:09.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.541+0000 7fe92e51b700 1 -- 172.21.15.163:0/441854248 shutdown_connections 2023-10-09T06:39:09.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.541+0000 7fe92e51b700 1 --2- 172.21.15.163:0/441854248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9280c91e0 0x7fe9280cbc00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:09.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.541+0000 7fe92e51b700 1 -- 172.21.15.163:0/441854248 >> 172.21.15.163:0/441854248 conn(0x7fe9280cd670 msgr2=0x7fe9280cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:09.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.541+0000 7fe92e51b700 1 -- 172.21.15.163:0/441854248 shutdown_connections 2023-10-09T06:39:09.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.541+0000 7fe92e51b700 1 -- 172.21.15.163:0/441854248 wait complete. 2023-10-09T06:39:09.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.542+0000 7fe92e51b700 1 Processor -- start 2023-10-09T06:39:09.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.543+0000 7fe92e51b700 1 -- start start 2023-10-09T06:39:09.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.543+0000 7fe92e51b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe92806aa40 0x7fe9280b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:09.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.543+0000 7fe92e51b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe92806ae10 0x7fe9280b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:09.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.543+0000 7fe92e51b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9280c91e0 0x7fe9280c34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:09.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.543+0000 7fe92e51b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9280b0f10 con 0x7fe92806ae10 2023-10-09T06:39:09.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.543+0000 7fe92e51b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9280c4720 con 0x7fe9280c91e0 2023-10-09T06:39:09.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.543+0000 7fe92e51b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe928150f20 con 0x7fe92806aa40 2023-10-09T06:39:09.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.543+0000 7fe9277fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe92806ae10 0x7fe9280b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:09.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.543+0000 7fe9277fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe92806ae10 0x7fe9280b2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44674/0 (socket says 172.21.15.163:44674) 2023-10-09T06:39:09.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.543+0000 7fe9277fe700 1 -- 172.21.15.163:0/4106146523 learned_addr learned my addr 172.21.15.163:0/4106146523 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:09.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.544+0000 7fe9277fe700 1 -- 172.21.15.163:0/4106146523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe92806aa40 msgr2=0x7fe9280b09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:09.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.544+0000 7fe927fff700 1 --2- 172.21.15.163:0/4106146523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe92806aa40 0x7fe9280b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:09.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.544+0000 7fe9277fe700 1 --2- 172.21.15.163:0/4106146523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe92806aa40 0x7fe9280b09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:09.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.544+0000 7fe92cab8700 1 --2- 172.21.15.163:0/4106146523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9280c91e0 0x7fe9280c34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:09.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.544+0000 7fe9277fe700 1 -- 172.21.15.163:0/4106146523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9280c91e0 msgr2=0x7fe9280c34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:09.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.544+0000 7fe9277fe700 1 --2- 172.21.15.163:0/4106146523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9280c91e0 0x7fe9280c34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:09.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.544+0000 7fe9277fe700 1 -- 172.21.15.163:0/4106146523 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe918009420 con 0x7fe92806ae10 2023-10-09T06:39:09.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.544+0000 7fe9277fe700 1 --2- 172.21.15.163:0/4106146523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe92806ae10 0x7fe9280b2c00 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fe92806c6a0 tx=0x7fe918016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:09.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.544+0000 7fe90ffff700 1 -- 172.21.15.163:0/4106146523 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe918011b10 con 0x7fe92806ae10 2023-10-09T06:39:09.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.544+0000 7fe90ffff700 1 -- 172.21.15.163:0/4106146523 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91801a450 con 0x7fe92806ae10 2023-10-09T06:39:09.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.544+0000 7fe92e51b700 1 -- 172.21.15.163:0/4106146523 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe928151200 con 0x7fe92806ae10 2023-10-09T06:39:09.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.544+0000 7fe90ffff700 1 -- 172.21.15.163:0/4106146523 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91801a810 con 0x7fe92806ae10 2023-10-09T06:39:09.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.544+0000 7fe92e51b700 1 -- 172.21.15.163:0/4106146523 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9281517c0 con 0x7fe92806ae10 2023-10-09T06:39:09.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.545+0000 7fe92e51b700 1 -- 172.21.15.163:0/4106146523 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe92804c3c0 con 0x7fe92806ae10 2023-10-09T06:39:09.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.545+0000 7fe90ffff700 1 -- 172.21.15.163:0/4106146523 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe91801ac00 con 0x7fe92806ae10 2023-10-09T06:39:09.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.546+0000 7fe90ffff700 1 --2- 172.21.15.163:0/4106146523 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe914067320 0x7fe9140697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:09.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.546+0000 7fe90ffff700 1 -- 172.21.15.163:0/4106146523 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6180+0+0 (secure 0 0 0) 0x7fe918086ac0 con 0x7fe92806ae10 2023-10-09T06:39:09.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.546+0000 7fe927fff700 1 --2- 172.21.15.163:0/4106146523 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe914067320 0x7fe9140697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:09.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.546+0000 7fe927fff700 1 --2- 172.21.15.163:0/4106146523 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe914067320 0x7fe9140697d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe91c007fd0 tx=0x7fe91c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:09.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.550+0000 7fe90ffff700 1 -- 172.21.15.163:0/4106146523 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe918053530 con 0x7fe92806ae10 2023-10-09T06:39:09.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.773+0000 7fe92e51b700 1 -- 172.21.15.163:0/4106146523 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9280c7cb0 con 0x7fe92806ae10 2023-10-09T06:39:09.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.774+0000 7fe90ffff700 1 -- 172.21.15.163:0/4106146523 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+4639 (secure 0 0 0) 0x7fe918056b50 con 0x7fe92806ae10 2023-10-09T06:39:09.775 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:09.775 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":115,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8211,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3056573272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3056573272},{"type":"v1","addr":"172.21.15.163:6834","nonce":3056573272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":8214,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3482861499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3482861499},{"type":"v1","addr":"172.21.15.163:6835","nonce":3482861499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":8223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/4239124825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4239124825},{"type":"v1","addr":"172.21.15.181:6836","nonce":4239124825}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:38:49.480715+0000","modified":"2023-10-09T06:39:04.559594+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8226":{"gid":8226,"name":"b","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/3494162891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3494162891},{"type":"v1","addr":"172.21.15.181:6837","nonce":3494162891}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29,30],"metadata_pool":28,"enabled":true,"fs_name":"volume_0000000000836565","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-09T06:39:09.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.776+0000 7fe92e51b700 1 -- 172.21.15.163:0/4106146523 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe914067320 msgr2=0x7fe9140697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:09.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.776+0000 7fe92e51b700 1 --2- 172.21.15.163:0/4106146523 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe914067320 0x7fe9140697d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe91c007fd0 tx=0x7fe91c009040 comp rx=0 tx=0).stop 2023-10-09T06:39:09.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.776+0000 7fe92e51b700 1 -- 172.21.15.163:0/4106146523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe92806ae10 msgr2=0x7fe9280b2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:09.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.776+0000 7fe92e51b700 1 --2- 172.21.15.163:0/4106146523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe92806ae10 0x7fe9280b2c00 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fe92806c6a0 tx=0x7fe918016000 comp rx=0 tx=0).stop 2023-10-09T06:39:09.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.776+0000 7fe92e51b700 1 -- 172.21.15.163:0/4106146523 shutdown_connections 2023-10-09T06:39:09.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.776+0000 7fe92e51b700 1 --2- 172.21.15.163:0/4106146523 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe914067320 0x7fe9140697d0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:09.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.776+0000 7fe92e51b700 1 --2- 172.21.15.163:0/4106146523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9280c91e0 0x7fe9280c34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:09.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.776+0000 7fe92e51b700 1 --2- 172.21.15.163:0/4106146523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe92806aa40 0x7fe9280b09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:09.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.776+0000 7fe92e51b700 1 --2- 172.21.15.163:0/4106146523 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe92806ae10 0x7fe9280b2c00 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:09.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.776+0000 7fe92e51b700 1 -- 172.21.15.163:0/4106146523 >> 172.21.15.163:0/4106146523 conn(0x7fe9280cd670 msgr2=0x7fe928150300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:09.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.776+0000 7fe92e51b700 1 -- 172.21.15.163:0/4106146523 shutdown_connections 2023-10-09T06:39:09.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.776+0000 7fe92e51b700 1 -- 172.21.15.163:0/4106146523 wait complete. 2023-10-09T06:39:09.779 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 115 2023-10-09T06:39:09.789 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:39:09.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.902+0000 7fb8f6b5d700 1 Processor -- start 2023-10-09T06:39:09.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.902+0000 7fb8f6b5d700 1 -- start start 2023-10-09T06:39:09.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.904+0000 7fb8f6b5d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb8f0159890 0x7fb8f015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:09.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.904+0000 7fb8f6b5d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb8f0159c60 0x7fb8f015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:09.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.904+0000 7fb8f6b5d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8f015a160 con 0x7fb8f015db10 2023-10-09T06:39:09.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.904+0000 7fb8f6b5d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8f015df50 con 0x7fb8f0159890 2023-10-09T06:39:09.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.904+0000 7fb8f6b5d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8f00d13a0 con 0x7fb8f0159c60 2023-10-09T06:39:09.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.904+0000 7fb8f48f9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb8f0159890 0x7fb8f015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:09.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.904+0000 7fb8f50fa700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb8f015db10 0x7fb8f0069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:51912/0 (socket says 172.21.15.163:51912) 2023-10-09T06:39:09.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.904+0000 7fb8f50fa700 1 -- 172.21.15.163:0/4094621174 learned_addr learned my addr 172.21.15.163:0/4094621174 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:09.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.904+0000 7fb8e7fff700 1 --2- 172.21.15.163:0/4094621174 >> v2:172.21.15.181:3300/0 conn(0x7fb8f0159c60 0x7fb8f015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:09.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.904+0000 7fb8f48f9700 1 -- 172.21.15.163:0/4094621174 >> v2:172.21.15.181:3300/0 conn(0x7fb8f0159c60 msgr2=0x7fb8f015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:09.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.904+0000 7fb8f48f9700 1 --2- 172.21.15.163:0/4094621174 >> v2:172.21.15.181:3300/0 conn(0x7fb8f0159c60 0x7fb8f015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:09.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.904+0000 7fb8f48f9700 1 -- 172.21.15.163:0/4094621174 >> v1:172.21.15.163:6789/0 conn(0x7fb8f015db10 legacy=0x7fb8f0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:09.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.904+0000 7fb8f48f9700 1 -- 172.21.15.163:0/4094621174 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f015e140 con 0x7fb8f0159890 2023-10-09T06:39:09.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.904+0000 7fb8e77fe700 1 -- 172.21.15.163:0/4094621174 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3070553484 0 0) 0x7fb8f015a160 con 0x7fb8f015db10 2023-10-09T06:39:09.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.904+0000 7fb8f48f9700 1 --2- 172.21.15.163:0/4094621174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8f0159890 0x7fb8f015abd0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fb8f00b67f0 tx=0x7fb8ec009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a081512e11eeb320 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:09.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.904+0000 7fb8e77fe700 1 -- 172.21.15.163:0/4094621174 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec00e070 con 0x7fb8f0159890 2023-10-09T06:39:09.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.904+0000 7fb8e77fe700 1 -- 172.21.15.163:0/4094621174 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ec00bb60 con 0x7fb8f0159890 2023-10-09T06:39:09.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.904+0000 7fb8e77fe700 1 -- 172.21.15.163:0/4094621174 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec012430 con 0x7fb8f0159890 2023-10-09T06:39:09.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.904+0000 7fb8f6b5d700 1 -- 172.21.15.163:0/4094621174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8f0159890 msgr2=0x7fb8f015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:09.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.905+0000 7fb8f6b5d700 1 --2- 172.21.15.163:0/4094621174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8f0159890 0x7fb8f015abd0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fb8f00b67f0 tx=0x7fb8ec009780 comp rx=0 tx=0).stop 2023-10-09T06:39:09.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.905+0000 7fb8f6b5d700 1 -- 172.21.15.163:0/4094621174 shutdown_connections 2023-10-09T06:39:09.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.905+0000 7fb8f6b5d700 1 --2- 172.21.15.163:0/4094621174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8f0159890 0x7fb8f015abd0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:09.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.905+0000 7fb8f6b5d700 1 --2- 172.21.15.163:0/4094621174 >> v2:172.21.15.181:3300/0 conn(0x7fb8f0159c60 0x7fb8f015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:09.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.905+0000 7fb8f6b5d700 1 -- 172.21.15.163:0/4094621174 >> 172.21.15.163:0/4094621174 conn(0x7fb8f0068e20 msgr2=0x7fb8f00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:09.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.905+0000 7fb8f6b5d700 1 -- 172.21.15.163:0/4094621174 shutdown_connections 2023-10-09T06:39:09.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.905+0000 7fb8f6b5d700 1 -- 172.21.15.163:0/4094621174 wait complete. 2023-10-09T06:39:09.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.906+0000 7fb8f6b5d700 1 Processor -- start 2023-10-09T06:39:09.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.906+0000 7fb8f6b5d700 1 -- start start 2023-10-09T06:39:09.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.907+0000 7fb8f6b5d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8f0159890 0x7fb8f00b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:09.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.907+0000 7fb8f6b5d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8f0159c60 0x7fb8f00cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:09.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.907+0000 7fb8f6b5d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8f015db10 0x7fb8f014d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:09.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.907+0000 7fb8f6b5d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f00b4780 con 0x7fb8f0159c60 2023-10-09T06:39:09.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.907+0000 7fb8f6b5d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f00ce0f0 con 0x7fb8f0159890 2023-10-09T06:39:09.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.907+0000 7fb8f6b5d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f00b3b70 con 0x7fb8f015db10 2023-10-09T06:39:09.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.907+0000 7fb8e7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8f0159c60 0x7fb8f00cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:09.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.907+0000 7fb8e7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8f0159c60 0x7fb8f00cdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44680/0 (socket says 172.21.15.163:44680) 2023-10-09T06:39:09.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.907+0000 7fb8e7fff700 1 -- 172.21.15.163:0/2486008591 learned_addr learned my addr 172.21.15.163:0/2486008591 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:09.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.907+0000 7fb8f48f9700 1 --2- 172.21.15.163:0/2486008591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8f0159890 0x7fb8f00b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:09.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.907+0000 7fb8f50fa700 1 --2- 172.21.15.163:0/2486008591 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8f015db10 0x7fb8f014d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:09.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.907+0000 7fb8e7fff700 1 -- 172.21.15.163:0/2486008591 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8f015db10 msgr2=0x7fb8f014d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:09.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.907+0000 7fb8e7fff700 1 --2- 172.21.15.163:0/2486008591 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8f015db10 0x7fb8f014d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:09.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.907+0000 7fb8e7fff700 1 -- 172.21.15.163:0/2486008591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8f0159890 msgr2=0x7fb8f00b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:09.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.907+0000 7fb8e7fff700 1 --2- 172.21.15.163:0/2486008591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8f0159890 0x7fb8f00b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:09.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.907+0000 7fb8e7fff700 1 -- 172.21.15.163:0/2486008591 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ec009a20 con 0x7fb8f0159c60 2023-10-09T06:39:09.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.907+0000 7fb8f48f9700 1 --2- 172.21.15.163:0/2486008591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8f0159890 0x7fb8f00b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:09.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.907+0000 7fb8e7fff700 1 --2- 172.21.15.163:0/2486008591 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8f0159c60 0x7fb8f00cdbb0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000ec80 tx=0x7fb8e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:09.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.907+0000 7fb8e4ff9700 1 -- 172.21.15.163:0/2486008591 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e000c850 con 0x7fb8f0159c60 2023-10-09T06:39:09.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.907+0000 7fb8e4ff9700 1 -- 172.21.15.163:0/2486008591 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e000c9b0 con 0x7fb8f0159c60 2023-10-09T06:39:09.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.907+0000 7fb8e4ff9700 1 -- 172.21.15.163:0/2486008591 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e000cd50 con 0x7fb8f0159c60 2023-10-09T06:39:09.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.907+0000 7fb8f6b5d700 1 -- 172.21.15.163:0/2486008591 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f00b3e50 con 0x7fb8f0159c60 2023-10-09T06:39:09.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.907+0000 7fb8f6b5d700 1 -- 172.21.15.163:0/2486008591 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f00b4410 con 0x7fb8f0159c60 2023-10-09T06:39:09.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.908+0000 7fb8f6b5d700 1 -- 172.21.15.163:0/2486008591 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f004c3c0 con 0x7fb8f0159c60 2023-10-09T06:39:09.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.909+0000 7fb8e4ff9700 1 -- 172.21.15.163:0/2486008591 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb8e0013680 con 0x7fb8f0159c60 2023-10-09T06:39:09.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.909+0000 7fb8e4ff9700 1 --2- 172.21.15.163:0/2486008591 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb8dc067320 0x7fb8dc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:09.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.909+0000 7fb8e4ff9700 1 -- 172.21.15.163:0/2486008591 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6180+0+0 (secure 0 0 0) 0x7fb8e0085430 con 0x7fb8f0159c60 2023-10-09T06:39:09.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.909+0000 7fb8f48f9700 1 --2- 172.21.15.163:0/2486008591 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb8dc067320 0x7fb8dc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:09.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.910+0000 7fb8f48f9700 1 --2- 172.21.15.163:0/2486008591 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb8dc067320 0x7fb8dc0697d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec009fd0 tx=0x7fb8ec019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:09.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:09.914+0000 7fb8e4ff9700 1 -- 172.21.15.163:0/2486008591 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb8e0051ee0 con 0x7fb8f0159c60 2023-10-09T06:39:10.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.137+0000 7fb8f6b5d700 1 -- 172.21.15.163:0/2486008591 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8f00b2e50 con 0x7fb8f0159c60 2023-10-09T06:39:10.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.138+0000 7fb8e4ff9700 1 -- 172.21.15.163:0/2486008591 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+4639 (secure 0 0 0) 0x7fb8e0055500 con 0x7fb8f0159c60 2023-10-09T06:39:10.138 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:10.138 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":115,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8211,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3056573272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3056573272},{"type":"v1","addr":"172.21.15.163:6834","nonce":3056573272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":8214,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3482861499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3482861499},{"type":"v1","addr":"172.21.15.163:6835","nonce":3482861499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":8223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/4239124825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4239124825},{"type":"v1","addr":"172.21.15.181:6836","nonce":4239124825}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:38:49.480715+0000","modified":"2023-10-09T06:39:04.559594+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8226":{"gid":8226,"name":"b","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/3494162891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3494162891},{"type":"v1","addr":"172.21.15.181:6837","nonce":3494162891}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29,30],"metadata_pool":28,"enabled":true,"fs_name":"volume_0000000000836565","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-09T06:39:10.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.140+0000 7fb8f6b5d700 1 -- 172.21.15.163:0/2486008591 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb8dc067320 msgr2=0x7fb8dc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:10.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.140+0000 7fb8f6b5d700 1 --2- 172.21.15.163:0/2486008591 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb8dc067320 0x7fb8dc0697d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec009fd0 tx=0x7fb8ec019000 comp rx=0 tx=0).stop 2023-10-09T06:39:10.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.140+0000 7fb8f6b5d700 1 -- 172.21.15.163:0/2486008591 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8f0159c60 msgr2=0x7fb8f00cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:10.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.140+0000 7fb8f6b5d700 1 --2- 172.21.15.163:0/2486008591 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8f0159c60 0x7fb8f00cdbb0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000ec80 tx=0x7fb8e00091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.140+0000 7fb8f6b5d700 1 -- 172.21.15.163:0/2486008591 shutdown_connections 2023-10-09T06:39:10.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.140+0000 7fb8f6b5d700 1 --2- 172.21.15.163:0/2486008591 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb8dc067320 0x7fb8dc0697d0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.140+0000 7fb8f6b5d700 1 --2- 172.21.15.163:0/2486008591 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8f015db10 0x7fb8f014d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.140+0000 7fb8f6b5d700 1 --2- 172.21.15.163:0/2486008591 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8f0159890 0x7fb8f00b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.140+0000 7fb8f6b5d700 1 --2- 172.21.15.163:0/2486008591 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8f0159c60 0x7fb8f00cdbb0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.140+0000 7fb8f6b5d700 1 -- 172.21.15.163:0/2486008591 >> 172.21.15.163:0/2486008591 conn(0x7fb8f0068e20 msgr2=0x7fb8f00c6e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:10.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.140+0000 7fb8f6b5d700 1 -- 172.21.15.163:0/2486008591 shutdown_connections 2023-10-09T06:39:10.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.140+0000 7fb8f6b5d700 1 -- 172.21.15.163:0/2486008591 wait complete. 2023-10-09T06:39:10.143 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 115 2023-10-09T06:39:10.152 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:39:10.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.268+0000 7f03cb0d8700 1 Processor -- start 2023-10-09T06:39:10.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.268+0000 7f03cb0d8700 1 -- start start 2023-10-09T06:39:10.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.269+0000 7f03cb0d8700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f03c406bcd0 0x7f03c40caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:10.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.269+0000 7f03cb0d8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f03c415df50 0x7f03c415e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:10.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.269+0000 7f03cb0d8700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03c415d7e0 con 0x7f03c406c0a0 2023-10-09T06:39:10.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.269+0000 7f03cb0d8700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03c415dac0 con 0x7f03c406bcd0 2023-10-09T06:39:10.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.269+0000 7f03cb0d8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03c415aa30 con 0x7f03c415df50 2023-10-09T06:39:10.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.269+0000 7f03c8e74700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f03c406bcd0 0x7f03c40caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:10.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.269+0000 7f03bbfff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f03c406c0a0 0x7f03c415a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:51922/0 (socket says 172.21.15.163:51922) 2023-10-09T06:39:10.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.269+0000 7f03bbfff700 1 -- 172.21.15.163:0/376113692 learned_addr learned my addr 172.21.15.163:0/376113692 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:10.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.270+0000 7f03c9675700 1 --2- 172.21.15.163:0/376113692 >> v2:172.21.15.181:3300/0 conn(0x7f03c415df50 0x7f03c415e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:10.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.270+0000 7f03c8e74700 1 -- 172.21.15.163:0/376113692 >> v2:172.21.15.181:3300/0 conn(0x7f03c415df50 msgr2=0x7f03c415e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:10.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.270+0000 7f03c8e74700 1 --2- 172.21.15.163:0/376113692 >> v2:172.21.15.181:3300/0 conn(0x7f03c415df50 0x7f03c415e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.270+0000 7f03c8e74700 1 -- 172.21.15.163:0/376113692 >> v1:172.21.15.163:6789/0 conn(0x7f03c406c0a0 legacy=0x7f03c415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:10.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.270+0000 7f03c8e74700 1 -- 172.21.15.163:0/376113692 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c415ad10 con 0x7f03c406bcd0 2023-10-09T06:39:10.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.270+0000 7f03bb7fe700 1 -- 172.21.15.163:0/376113692 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2357393270 0 0) 0x7f03c415d7e0 con 0x7f03c406c0a0 2023-10-09T06:39:10.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.270+0000 7f03c9675700 1 --2- 172.21.15.163:0/376113692 >> v2:172.21.15.181:3300/0 conn(0x7f03c415df50 0x7f03c415e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:10.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.270+0000 7f03c8e74700 1 --2- 172.21.15.163:0/376113692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03c406bcd0 0x7f03c40caa90 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f03c415d720 tx=0x7f03c0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be7ced30fda1d0b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:10.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.270+0000 7f03bb7fe700 1 -- 172.21.15.163:0/376113692 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c000e070 con 0x7f03c406bcd0 2023-10-09T06:39:10.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.270+0000 7f03bb7fe700 1 -- 172.21.15.163:0/376113692 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c000bc30 con 0x7f03c406bcd0 2023-10-09T06:39:10.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.270+0000 7f03bb7fe700 1 -- 172.21.15.163:0/376113692 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c0012430 con 0x7f03c406bcd0 2023-10-09T06:39:10.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.271+0000 7f03cb0d8700 1 -- 172.21.15.163:0/376113692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03c406bcd0 msgr2=0x7f03c40caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:10.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.271+0000 7f03cb0d8700 1 --2- 172.21.15.163:0/376113692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03c406bcd0 0x7f03c40caa90 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f03c415d720 tx=0x7f03c0009780 comp rx=0 tx=0).stop 2023-10-09T06:39:10.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.271+0000 7f03cb0d8700 1 -- 172.21.15.163:0/376113692 shutdown_connections 2023-10-09T06:39:10.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.271+0000 7f03cb0d8700 1 --2- 172.21.15.163:0/376113692 >> v2:172.21.15.181:3300/0 conn(0x7f03c415df50 0x7f03c415e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.271+0000 7f03cb0d8700 1 --2- 172.21.15.163:0/376113692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03c406bcd0 0x7f03c40caa90 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.271+0000 7f03cb0d8700 1 -- 172.21.15.163:0/376113692 >> 172.21.15.163:0/376113692 conn(0x7f03c40ce600 msgr2=0x7f03c40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:10.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.271+0000 7f03cb0d8700 1 -- 172.21.15.163:0/376113692 shutdown_connections 2023-10-09T06:39:10.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.271+0000 7f03cb0d8700 1 -- 172.21.15.163:0/376113692 wait complete. 2023-10-09T06:39:10.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.272+0000 7f03cb0d8700 1 Processor -- start 2023-10-09T06:39:10.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.272+0000 7f03cb0d8700 1 -- start start 2023-10-09T06:39:10.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.273+0000 7f03cb0d8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03c406bcd0 0x7f03c416a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:10.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.273+0000 7f03cb0d8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03c406c0a0 0x7f03c40c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:10.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.273+0000 7f03cb0d8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03c415df50 0x7f03c40b2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:10.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.273+0000 7f03cb0d8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c40b2710 con 0x7f03c406c0a0 2023-10-09T06:39:10.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.273+0000 7f03cb0d8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c40b2410 con 0x7f03c406bcd0 2023-10-09T06:39:10.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.273+0000 7f03cb0d8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03c416ac10 con 0x7f03c415df50 2023-10-09T06:39:10.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.273+0000 7f03bbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03c406c0a0 0x7f03c40c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:10.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.273+0000 7f03bbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03c406c0a0 0x7f03c40c7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44702/0 (socket says 172.21.15.163:44702) 2023-10-09T06:39:10.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.273+0000 7f03bbfff700 1 -- 172.21.15.163:0/3441888666 learned_addr learned my addr 172.21.15.163:0/3441888666 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:10.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.273+0000 7f03c8e74700 1 --2- 172.21.15.163:0/3441888666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03c406bcd0 0x7f03c416a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:10.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.273+0000 7f03c9675700 1 --2- 172.21.15.163:0/3441888666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03c415df50 0x7f03c40b2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:10.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.273+0000 7f03bbfff700 1 -- 172.21.15.163:0/3441888666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03c415df50 msgr2=0x7f03c40b2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:10.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.273+0000 7f03bbfff700 1 --2- 172.21.15.163:0/3441888666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03c415df50 0x7f03c40b2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.273+0000 7f03bbfff700 1 -- 172.21.15.163:0/3441888666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03c406bcd0 msgr2=0x7f03c416a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:10.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.273+0000 7f03bbfff700 1 --2- 172.21.15.163:0/3441888666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03c406bcd0 0x7f03c416a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.273+0000 7f03bbfff700 1 -- 172.21.15.163:0/3441888666 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c0009a20 con 0x7f03c406c0a0 2023-10-09T06:39:10.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.273+0000 7f03bbfff700 1 --2- 172.21.15.163:0/3441888666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03c406c0a0 0x7f03c40c7770 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f03b400c5e0 tx=0x7f03b4001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:10.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.273+0000 7f03b8ff9700 1 -- 172.21.15.163:0/3441888666 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b4005b30 con 0x7f03c406c0a0 2023-10-09T06:39:10.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.273+0000 7f03b8ff9700 1 -- 172.21.15.163:0/3441888666 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b4005c90 con 0x7f03c406c0a0 2023-10-09T06:39:10.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.273+0000 7f03cb0d8700 1 -- 172.21.15.163:0/3441888666 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c416aef0 con 0x7f03c406c0a0 2023-10-09T06:39:10.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.273+0000 7f03b8ff9700 1 -- 172.21.15.163:0/3441888666 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b40072e0 con 0x7f03c406c0a0 2023-10-09T06:39:10.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.273+0000 7f03cb0d8700 1 -- 172.21.15.163:0/3441888666 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c416a880 con 0x7f03c406c0a0 2023-10-09T06:39:10.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.274+0000 7f03cb0d8700 1 -- 172.21.15.163:0/3441888666 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03c40b28c0 con 0x7f03c406c0a0 2023-10-09T06:39:10.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.275+0000 7f03b8ff9700 1 -- 172.21.15.163:0/3441888666 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f03b401b3b0 con 0x7f03c406c0a0 2023-10-09T06:39:10.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.275+0000 7f03b8ff9700 1 --2- 172.21.15.163:0/3441888666 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f03b0067320 0x7f03b00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:10.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.275+0000 7f03b8ff9700 1 -- 172.21.15.163:0/3441888666 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6180+0+0 (secure 0 0 0) 0x7f03b40842a0 con 0x7f03c406c0a0 2023-10-09T06:39:10.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.276+0000 7f03c8e74700 1 --2- 172.21.15.163:0/3441888666 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f03b0067320 0x7f03b00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:10.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.276+0000 7f03c8e74700 1 --2- 172.21.15.163:0/3441888666 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f03b0067320 0x7f03b00697d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f03c406dc70 tx=0x7f03c0019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:10.280 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.280+0000 7f03b8ff9700 1 -- 172.21.15.163:0/3441888666 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f03b404b5a0 con 0x7f03c406c0a0 2023-10-09T06:39:10.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.464+0000 7f03cb0d8700 1 -- 172.21.15.163:0/3441888666 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03c404c3c0 con 0x7f03c406c0a0 2023-10-09T06:39:10.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.465+0000 7f03b8ff9700 1 -- 172.21.15.163:0/3441888666 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) v1 ==== 74+0+17053 (secure 0 0 0) 0x7f03c40b28c0 con 0x7f03c406c0a0 2023-10-09T06:39:10.465 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:10.465 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":169,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:39:05.560867+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":28,"pool_name":"cephfs.volume_0000000000836565.meta","create_time":"2023-10-09T06:38:47.806008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000836565"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-10-09T06:38:48.446181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000836565"}}},{"pool":30,"pool_name":"another-data-pool","create_time":"2023-10-09T06:39:00.367621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"volume_0000000000836565"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:39:10.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.466+0000 7f03cb0d8700 1 -- 172.21.15.163:0/3441888666 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f03b0067320 msgr2=0x7f03b00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:10.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.466+0000 7f03cb0d8700 1 --2- 172.21.15.163:0/3441888666 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f03b0067320 0x7f03b00697d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f03c406dc70 tx=0x7f03c0019000 comp rx=0 tx=0).stop 2023-10-09T06:39:10.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.466+0000 7f03cb0d8700 1 -- 172.21.15.163:0/3441888666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03c406c0a0 msgr2=0x7f03c40c7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:10.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.466+0000 7f03cb0d8700 1 --2- 172.21.15.163:0/3441888666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03c406c0a0 0x7f03c40c7770 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f03b400c5e0 tx=0x7f03b4001070 comp rx=0 tx=0).stop 2023-10-09T06:39:10.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.467+0000 7f03cb0d8700 1 -- 172.21.15.163:0/3441888666 shutdown_connections 2023-10-09T06:39:10.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.467+0000 7f03cb0d8700 1 --2- 172.21.15.163:0/3441888666 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f03b0067320 0x7f03b00697d0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.467+0000 7f03cb0d8700 1 --2- 172.21.15.163:0/3441888666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03c415df50 0x7f03c40b2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.467+0000 7f03cb0d8700 1 --2- 172.21.15.163:0/3441888666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03c406bcd0 0x7f03c416a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.467+0000 7f03cb0d8700 1 --2- 172.21.15.163:0/3441888666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03c406c0a0 0x7f03c40c7770 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.467+0000 7f03cb0d8700 1 -- 172.21.15.163:0/3441888666 >> 172.21.15.163:0/3441888666 conn(0x7f03c40ce600 msgr2=0x7f03c4159de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:10.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.467+0000 7f03cb0d8700 1 -- 172.21.15.163:0/3441888666 shutdown_connections 2023-10-09T06:39:10.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.467+0000 7f03cb0d8700 1 -- 172.21.15.163:0/3441888666 wait complete. 2023-10-09T06:39:10.479 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail volume_0000000000836565 2023-10-09T06:39:10.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.596+0000 7fd7c99f8700 1 Processor -- start 2023-10-09T06:39:10.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.596+0000 7fd7c99f8700 1 -- start start 2023-10-09T06:39:10.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.597+0000 7fd7c99f8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7c4159420 0x7fd7c40b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:10.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.597+0000 7fd7c99f8700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd7c41597f0 0x7fd7c40b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:10.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.597+0000 7fd7c99f8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd7c415d230 0x7fd7c415d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:10.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.597+0000 7fd7c99f8700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7c415dbd0 con 0x7fd7c41597f0 2023-10-09T06:39:10.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.597+0000 7fd7c99f8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7c40b11d0 con 0x7fd7c415d230 2023-10-09T06:39:10.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.597+0000 7fd7c99f8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c40b14b0 con 0x7fd7c4159420 2023-10-09T06:39:10.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.597+0000 7fd7bbfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd7c41597f0 0x7fd7c40b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:10.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.597+0000 7fd7bbfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd7c41597f0 0x7fd7c40b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44716/0 (socket says 172.21.15.163:44716) 2023-10-09T06:39:10.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.597+0000 7fd7bbfff700 1 -- 172.21.15.163:0/573671074 learned_addr learned my addr 172.21.15.163:0/573671074 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:10.599 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.597+0000 7fd7c91f7700 1 --2- 172.21.15.163:0/573671074 >> v2:172.21.15.181:3300/0 conn(0x7fd7c415d230 0x7fd7c415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:10.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.597+0000 7fd7c89f6700 1 --2- 172.21.15.163:0/573671074 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7c4159420 0x7fd7c40b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:10.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.597+0000 7fd7bbfff700 1 -- 172.21.15.163:0/573671074 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7c4159420 msgr2=0x7fd7c40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:10.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.597+0000 7fd7bbfff700 1 --2- 172.21.15.163:0/573671074 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7c4159420 0x7fd7c40b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.597+0000 7fd7bbfff700 1 -- 172.21.15.163:0/573671074 >> v2:172.21.15.181:3300/0 conn(0x7fd7c415d230 msgr2=0x7fd7c415d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:10.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.597+0000 7fd7bbfff700 1 --2- 172.21.15.163:0/573671074 >> v2:172.21.15.181:3300/0 conn(0x7fd7c415d230 0x7fd7c415d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.597+0000 7fd7bbfff700 1 -- 172.21.15.163:0/573671074 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c40b1790 con 0x7fd7c41597f0 2023-10-09T06:39:10.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.597+0000 7fd7c91f7700 1 --2- 172.21.15.163:0/573671074 >> v2:172.21.15.181:3300/0 conn(0x7fd7c415d230 0x7fd7c415d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:10.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.598+0000 7fd7bbfff700 1 --2- 172.21.15.163:0/573671074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7c41597f0 0x7fd7c40b2270 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40d1720 tx=0x7fd7b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7eb1ad918b62b48 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:10.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.598+0000 7fd7bb7fe700 1 -- 172.21.15.163:0/573671074 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b400b860 con 0x7fd7c41597f0 2023-10-09T06:39:10.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.598+0000 7fd7bb7fe700 1 -- 172.21.15.163:0/573671074 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b400b9c0 con 0x7fd7c41597f0 2023-10-09T06:39:10.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.598+0000 7fd7bb7fe700 1 -- 172.21.15.163:0/573671074 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b400bd60 con 0x7fd7c41597f0 2023-10-09T06:39:10.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.598+0000 7fd7c99f8700 1 -- 172.21.15.163:0/573671074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7c41597f0 msgr2=0x7fd7c40b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:10.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.598+0000 7fd7c99f8700 1 --2- 172.21.15.163:0/573671074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7c41597f0 0x7fd7c40b2270 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40d1720 tx=0x7fd7b4009250 comp rx=0 tx=0).stop 2023-10-09T06:39:10.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.598+0000 7fd7c99f8700 1 -- 172.21.15.163:0/573671074 shutdown_connections 2023-10-09T06:39:10.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.598+0000 7fd7c99f8700 1 --2- 172.21.15.163:0/573671074 >> v2:172.21.15.181:3300/0 conn(0x7fd7c415d230 0x7fd7c415d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.598+0000 7fd7c99f8700 1 --2- 172.21.15.163:0/573671074 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7c4159420 0x7fd7c40b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.598+0000 7fd7c99f8700 1 --2- 172.21.15.163:0/573671074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7c41597f0 0x7fd7c40b2270 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.598+0000 7fd7c99f8700 1 -- 172.21.15.163:0/573671074 >> 172.21.15.163:0/573671074 conn(0x7fd7c40d0660 msgr2=0x7fd7c40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:10.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.598+0000 7fd7c99f8700 1 -- 172.21.15.163:0/573671074 shutdown_connections 2023-10-09T06:39:10.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.598+0000 7fd7c99f8700 1 -- 172.21.15.163:0/573671074 wait complete. 2023-10-09T06:39:10.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.599+0000 7fd7c99f8700 1 Processor -- start 2023-10-09T06:39:10.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.600+0000 7fd7c99f8700 1 -- start start 2023-10-09T06:39:10.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.600+0000 7fd7c99f8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7c4159420 0x7fd7c415b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:10.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.600+0000 7fd7c99f8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7c41597f0 0x7fd7c414edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:10.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.600+0000 7fd7c99f8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7c415d230 0x7fd7c4152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:10.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.600+0000 7fd7c99f8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c415a450 con 0x7fd7c415d230 2023-10-09T06:39:10.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.600+0000 7fd7c99f8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c415a730 con 0x7fd7c4159420 2023-10-09T06:39:10.604 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.600+0000 7fd7c99f8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c415aa10 con 0x7fd7c41597f0 2023-10-09T06:39:10.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.600+0000 7fd7c91f7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7c415d230 0x7fd7c4152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:10.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.600+0000 7fd7c91f7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7c415d230 0x7fd7c4152710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44726/0 (socket says 172.21.15.163:44726) 2023-10-09T06:39:10.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.600+0000 7fd7c91f7700 1 -- 172.21.15.163:0/4183588987 learned_addr learned my addr 172.21.15.163:0/4183588987 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:10.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.601+0000 7fd7c89f6700 1 --2- 172.21.15.163:0/4183588987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7c4159420 0x7fd7c415b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:10.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.601+0000 7fd7bbfff700 1 --2- 172.21.15.163:0/4183588987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7c41597f0 0x7fd7c414edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:10.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.601+0000 7fd7c91f7700 1 -- 172.21.15.163:0/4183588987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7c41597f0 msgr2=0x7fd7c414edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:10.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.601+0000 7fd7c91f7700 1 --2- 172.21.15.163:0/4183588987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7c41597f0 0x7fd7c414edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.601+0000 7fd7c91f7700 1 -- 172.21.15.163:0/4183588987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7c4159420 msgr2=0x7fd7c415b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:10.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.601+0000 7fd7c91f7700 1 --2- 172.21.15.163:0/4183588987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7c4159420 0x7fd7c415b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.601+0000 7fd7c91f7700 1 -- 172.21.15.163:0/4183588987 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b40094f0 con 0x7fd7c415d230 2023-10-09T06:39:10.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.601+0000 7fd7c89f6700 1 --2- 172.21.15.163:0/4183588987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7c4159420 0x7fd7c415b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:10.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.601+0000 7fd7c91f7700 1 --2- 172.21.15.163:0/4183588987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7c415d230 0x7fd7c4152710 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00cef0 tx=0x7fd7bc007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:10.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.601+0000 7fd7b8ff9700 1 -- 172.21.15.163:0/4183588987 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc00ae00 con 0x7fd7c415d230 2023-10-09T06:39:10.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.601+0000 7fd7b8ff9700 1 -- 172.21.15.163:0/4183588987 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7bc016420 con 0x7fd7c415d230 2023-10-09T06:39:10.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.601+0000 7fd7c99f8700 1 -- 172.21.15.163:0/4183588987 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c415acf0 con 0x7fd7c415d230 2023-10-09T06:39:10.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.601+0000 7fd7b8ff9700 1 -- 172.21.15.163:0/4183588987 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc0167e0 con 0x7fd7c415d230 2023-10-09T06:39:10.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.601+0000 7fd7c99f8700 1 -- 172.21.15.163:0/4183588987 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c415ba30 con 0x7fd7c415d230 2023-10-09T06:39:10.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.602+0000 7fd7c99f8700 1 -- 172.21.15.163:0/4183588987 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c40afa40 con 0x7fd7c415d230 2023-10-09T06:39:10.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.602+0000 7fd7b8ff9700 1 -- 172.21.15.163:0/4183588987 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd7bc016bd0 con 0x7fd7c415d230 2023-10-09T06:39:10.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.603+0000 7fd7b8ff9700 1 --2- 172.21.15.163:0/4183588987 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd7b0067320 0x7fd7b00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:10.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.603+0000 7fd7b8ff9700 1 -- 172.21.15.163:0/4183588987 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6180+0+0 (secure 0 0 0) 0x7fd7bc012070 con 0x7fd7c415d230 2023-10-09T06:39:10.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.603+0000 7fd7c89f6700 1 --2- 172.21.15.163:0/4183588987 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd7b0067320 0x7fd7b00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:10.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.604+0000 7fd7c89f6700 1 --2- 172.21.15.163:0/4183588987 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd7b0067320 0x7fd7b00697d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0008a80 tx=0x7fd7c0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:10.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.607+0000 7fd7b8ff9700 1 -- 172.21.15.163:0/4183588987 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd7bc04e5a0 con 0x7fd7c415d230 2023-10-09T06:39:10.833 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.833+0000 7fd7c99f8700 1 -- 172.21.15.163:0/4183588987 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "volume_0000000000836565"} v 0) v1 -- 0x7fd7c404c3c0 con 0x7fd7c415d230 2023-10-09T06:39:10.833 INFO:tasks.ceph.mon.a.smithi163.stderr:2023-10-09T06:39:10.834+0000 7ff251c04700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-09T06:39:10.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.837+0000 7fd7b8ff9700 1 -- 172.21.15.163:0/4183588987 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "volume_0000000000836565"}]=0 volume_0000000000836565 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v116) v1 ==== 195+0+0 (secure 0 0 0) 0x7fd7bc0522a0 con 0x7fd7c415d230 2023-10-09T06:39:10.838 INFO:tasks.ceph.mds.b.smithi181.stderr: -825> 2023-10-09T06:38:55.385+0000 7fa3ae362700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-09T06:39:10.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.840+0000 7fd7c99f8700 1 -- 172.21.15.163:0/4183588987 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd7b0067320 msgr2=0x7fd7b00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:10.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.840+0000 7fd7c99f8700 1 --2- 172.21.15.163:0/4183588987 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd7b0067320 0x7fd7b00697d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0008a80 tx=0x7fd7c0008040 comp rx=0 tx=0).stop 2023-10-09T06:39:10.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.840+0000 7fd7c99f8700 1 -- 172.21.15.163:0/4183588987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7c415d230 msgr2=0x7fd7c4152710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:10.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.840+0000 7fd7c99f8700 1 --2- 172.21.15.163:0/4183588987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7c415d230 0x7fd7c4152710 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00cef0 tx=0x7fd7bc007860 comp rx=0 tx=0).stop 2023-10-09T06:39:10.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.840+0000 7fd7c99f8700 1 -- 172.21.15.163:0/4183588987 shutdown_connections 2023-10-09T06:39:10.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.840+0000 7fd7c99f8700 1 --2- 172.21.15.163:0/4183588987 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd7b0067320 0x7fd7b00697d0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.840+0000 7fd7c99f8700 1 --2- 172.21.15.163:0/4183588987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7c415d230 0x7fd7c4152710 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.840+0000 7fd7c99f8700 1 --2- 172.21.15.163:0/4183588987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd7c4159420 0x7fd7c415b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.840+0000 7fd7c99f8700 1 --2- 172.21.15.163:0/4183588987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd7c41597f0 0x7fd7c414edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.840+0000 7fd7c99f8700 1 -- 172.21.15.163:0/4183588987 >> 172.21.15.163:0/4183588987 conn(0x7fd7c40d0660 msgr2=0x7fd7c4157390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:10.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.840+0000 7fd7c99f8700 1 -- 172.21.15.163:0/4183588987 shutdown_connections 2023-10-09T06:39:10.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.840+0000 7fd7c99f8700 1 -- 172.21.15.163:0/4183588987 wait complete. 2023-10-09T06:39:10.841 INFO:teuthology.orchestra.run.smithi163.stderr:volume_0000000000836565 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-09T06:39:10.852 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm volume_0000000000836565 --yes-i-really-mean-it 2023-10-09T06:39:10.861 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-09T06:39:10.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.970+0000 7fad05f37700 1 Processor -- start 2023-10-09T06:39:10.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.970+0000 7fad05f37700 1 -- start start 2023-10-09T06:39:10.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.971+0000 7fad05f37700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad00157680 0x7fad000cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:10.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.971+0000 7fad05f37700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fad00157a50 0x7fad000cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:10.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.971+0000 7fad05f37700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad000cb900 con 0x7fad000b1450 2023-10-09T06:39:10.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.971+0000 7fad05f37700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad000cc700 con 0x7fad00157a50 2023-10-09T06:39:10.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.971+0000 7fad05f37700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad000cd240 con 0x7fad00157680 2023-10-09T06:39:10.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.971+0000 7facfffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fad00157a50 0x7fad000cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:10.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.971+0000 7facfffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fad00157a50 0x7fad000cb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44740/0 (socket says 172.21.15.163:44740) 2023-10-09T06:39:10.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.971+0000 7facfffff700 1 -- 172.21.15.163:0/2883296984 learned_addr learned my addr 172.21.15.163:0/2883296984 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:10.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.971+0000 7fad04f35700 1 --2- 172.21.15.163:0/2883296984 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad00157680 0x7fad000cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:10.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.971+0000 7facfffff700 1 -- 172.21.15.163:0/2883296984 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad00157680 msgr2=0x7fad000cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:10.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.971+0000 7facfffff700 1 --2- 172.21.15.163:0/2883296984 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad00157680 0x7fad000cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.971+0000 7facfffff700 1 -- 172.21.15.163:0/2883296984 >> v1:172.21.15.181:6789/0 conn(0x7fad000b1450 legacy=0x7fad000c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:10.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.971+0000 7facfffff700 1 -- 172.21.15.163:0/2883296984 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad000c9a20 con 0x7fad00157a50 2023-10-09T06:39:10.973 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.972+0000 7facfffff700 1 --2- 172.21.15.163:0/2883296984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad00157a50 0x7fad000cb3c0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7facec0099b0 tx=0x7facec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35de0ea8b3b17268 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:10.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.972+0000 7facff7fe700 1 -- 172.21.15.163:0/2883296984 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec009ea0 con 0x7fad00157a50 2023-10-09T06:39:10.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.972+0000 7facff7fe700 1 -- 172.21.15.163:0/2883296984 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facec00b860 con 0x7fad00157a50 2023-10-09T06:39:10.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.972+0000 7facff7fe700 1 -- 172.21.15.163:0/2883296984 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec00bc00 con 0x7fad00157a50 2023-10-09T06:39:10.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.972+0000 7fad05f37700 1 -- 172.21.15.163:0/2883296984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad00157a50 msgr2=0x7fad000cb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:10.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.972+0000 7fad05f37700 1 --2- 172.21.15.163:0/2883296984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad00157a50 0x7fad000cb3c0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7facec0099b0 tx=0x7facec009250 comp rx=0 tx=0).stop 2023-10-09T06:39:10.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.972+0000 7fad05f37700 1 -- 172.21.15.163:0/2883296984 shutdown_connections 2023-10-09T06:39:10.974 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.972+0000 7fad05f37700 1 --2- 172.21.15.163:0/2883296984 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad00157680 0x7fad000cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.972+0000 7fad05f37700 1 --2- 172.21.15.163:0/2883296984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad00157a50 0x7fad000cb3c0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.972+0000 7fad05f37700 1 -- 172.21.15.163:0/2883296984 >> 172.21.15.163:0/2883296984 conn(0x7fad0006baa0 msgr2=0x7fad0006bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:10.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.972+0000 7fad05f37700 1 -- 172.21.15.163:0/2883296984 shutdown_connections 2023-10-09T06:39:10.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.972+0000 7fad05f37700 1 -- 172.21.15.163:0/2883296984 wait complete. 2023-10-09T06:39:10.977 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.973+0000 7fad05f37700 1 Processor -- start 2023-10-09T06:39:10.977 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.974+0000 7fad05f37700 1 -- start start 2023-10-09T06:39:10.978 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.974+0000 7fad05f37700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad000b1450 0x7fad00151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:10.978 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.974+0000 7fad05f37700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad00157680 0x7fad001684c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:10.978 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.974+0000 7fad05f37700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad00157a50 0x7fad000c4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:10.978 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.974+0000 7fad05f37700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad000b0240 con 0x7fad00157680 2023-10-09T06:39:10.978 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.974+0000 7fad05f37700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad00151b50 con 0x7fad000b1450 2023-10-09T06:39:10.978 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.974+0000 7fad05f37700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad00159bd0 con 0x7fad00157a50 2023-10-09T06:39:10.978 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.974+0000 7facfffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad00157680 0x7fad001684c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:10.978 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.974+0000 7facfffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad00157680 0x7fad001684c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44752/0 (socket says 172.21.15.163:44752) 2023-10-09T06:39:10.978 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.974+0000 7facfffff700 1 -- 172.21.15.163:0/2674063246 learned_addr learned my addr 172.21.15.163:0/2674063246 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:10.978 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.974+0000 7fad04f35700 1 --2- 172.21.15.163:0/2674063246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad000b1450 0x7fad00151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:10.978 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.975+0000 7fad05736700 1 --2- 172.21.15.163:0/2674063246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad00157a50 0x7fad000c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:10.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.975+0000 7facfffff700 1 -- 172.21.15.163:0/2674063246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad00157a50 msgr2=0x7fad000c4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:10.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.975+0000 7facfffff700 1 --2- 172.21.15.163:0/2674063246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad00157a50 0x7fad000c4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.975+0000 7facfffff700 1 -- 172.21.15.163:0/2674063246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad000b1450 msgr2=0x7fad00151610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:10.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.975+0000 7facfffff700 1 --2- 172.21.15.163:0/2674063246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad000b1450 0x7fad00151610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:10.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.975+0000 7facfffff700 1 -- 172.21.15.163:0/2674063246 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facec0094f0 con 0x7fad00157680 2023-10-09T06:39:10.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.975+0000 7fad04f35700 1 --2- 172.21.15.163:0/2674063246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad000b1450 0x7fad00151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:39:10.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.975+0000 7facfffff700 1 --2- 172.21.15.163:0/2674063246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad00157680 0x7fad001684c0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7facec009290 tx=0x7facec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:10.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.975+0000 7facfcff9700 1 -- 172.21.15.163:0/2674063246 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec00b9d0 con 0x7fad00157680 2023-10-09T06:39:10.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.975+0000 7facfcff9700 1 -- 172.21.15.163:0/2674063246 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facec00bb50 con 0x7fad00157680 2023-10-09T06:39:10.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.975+0000 7fad05f37700 1 -- 172.21.15.163:0/2674063246 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad00159eb0 con 0x7fad00157680 2023-10-09T06:39:10.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.975+0000 7facfcff9700 1 -- 172.21.15.163:0/2674063246 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec025670 con 0x7fad00157680 2023-10-09T06:39:10.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.975+0000 7fad05f37700 1 -- 172.21.15.163:0/2674063246 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad0015a470 con 0x7fad00157680 2023-10-09T06:39:10.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.976+0000 7fad05f37700 1 -- 172.21.15.163:0/2674063246 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad0006eba0 con 0x7fad00157680 2023-10-09T06:39:10.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.976+0000 7facfcff9700 1 -- 172.21.15.163:0/2674063246 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7facec023070 con 0x7fad00157680 2023-10-09T06:39:10.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.977+0000 7facfcff9700 1 --2- 172.21.15.163:0/2674063246 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7face4067320 0x7face40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:10.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.977+0000 7facfcff9700 1 -- 172.21.15.163:0/2674063246 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6266+0+0 (secure 0 0 0) 0x7facec0862c0 con 0x7fad00157680 2023-10-09T06:39:10.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.977+0000 7fad04f35700 1 --2- 172.21.15.163:0/2674063246 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7face4067320 0x7face40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:10.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.977+0000 7fad04f35700 1 --2- 172.21.15.163:0/2674063246 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7face4067320 0x7face40697d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fad00165680 tx=0x7facf40091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:10.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:10.981+0000 7facfcff9700 1 -- 172.21.15.163:0/2674063246 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7facec052d20 con 0x7fad00157680 2023-10-09T06:39:11.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.203+0000 7fad05f37700 1 -- 172.21.15.163:0/2674063246 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "volume_0000000000836565", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fad000cc940 con 0x7fad00157680 2023-10-09T06:39:11.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.838+0000 7facfcff9700 1 -- 172.21.15.163:0/2674063246 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "volume_0000000000836565", "yes_i_really_mean_it": true}]=0 v117) v1 ==== 121+0+0 (secure 0 0 0) 0x7facec056340 con 0x7fad00157680 2023-10-09T06:39:11.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.840+0000 7fad05f37700 1 -- 172.21.15.163:0/2674063246 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7face4067320 msgr2=0x7face40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:11.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.840+0000 7fad05f37700 1 --2- 172.21.15.163:0/2674063246 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7face4067320 0x7face40697d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fad00165680 tx=0x7facf40091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:11.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.840+0000 7fad05f37700 1 -- 172.21.15.163:0/2674063246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad00157680 msgr2=0x7fad001684c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:11.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.840+0000 7fad05f37700 1 --2- 172.21.15.163:0/2674063246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad00157680 0x7fad001684c0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7facec009290 tx=0x7facec017000 comp rx=0 tx=0).stop 2023-10-09T06:39:11.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.840+0000 7fad05f37700 1 -- 172.21.15.163:0/2674063246 shutdown_connections 2023-10-09T06:39:11.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.840+0000 7fad05f37700 1 --2- 172.21.15.163:0/2674063246 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7face4067320 0x7face40697d0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:11.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.840+0000 7fad05f37700 1 --2- 172.21.15.163:0/2674063246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad00157a50 0x7fad000c4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:11.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.840+0000 7fad05f37700 1 --2- 172.21.15.163:0/2674063246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad000b1450 0x7fad00151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:11.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.840+0000 7fad05f37700 1 --2- 172.21.15.163:0/2674063246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad00157680 0x7fad001684c0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:11.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.840+0000 7fad05f37700 1 -- 172.21.15.163:0/2674063246 >> 172.21.15.163:0/2674063246 conn(0x7fad0006baa0 msgr2=0x7fad000cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:11.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.840+0000 7fad05f37700 1 -- 172.21.15.163:0/2674063246 shutdown_connections 2023-10-09T06:39:11.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.840+0000 7fad05f37700 1 -- 172.21.15.163:0/2674063246 wait complete. 2023-10-09T06:39:11.854 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000836565.meta cephfs.volume_0000000000836565.meta --yes-i-really-really-mean-it 2023-10-09T06:39:11.975 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.975+0000 7fbac7715700 1 Processor -- start 2023-10-09T06:39:11.976 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.976+0000 7fbac7715700 1 -- start start 2023-10-09T06:39:11.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.976+0000 7fbac7715700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbac0153f50 0x7fbac00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:11.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.976+0000 7fbac7715700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac00b4570 con 0x7fbac0154320 2023-10-09T06:39:11.979 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.976+0000 7fbac7715700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac015a730 con 0x7fbac006d840 2023-10-09T06:39:11.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.976+0000 7fbac7715700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbac015aa10 con 0x7fbac0153f50 2023-10-09T06:39:11.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.977+0000 7fbac4cb0700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fbac0154320 0x7fbac006d130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:51938/0 (socket says 172.21.15.163:51938) 2023-10-09T06:39:11.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.977+0000 7fbac4cb0700 1 -- 172.21.15.163:0/3941413606 learned_addr learned my addr 172.21.15.163:0/3941413606 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:11.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.977+0000 7fbac54b1700 1 --2- 172.21.15.163:0/3941413606 >> v2:172.21.15.181:3300/0 conn(0x7fbac0153f50 0x7fbac00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:11.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.977+0000 7fbab7fff700 1 -- 172.21.15.163:0/3941413606 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3664679859 0 0) 0x7fbac00b4570 con 0x7fbac0154320 2023-10-09T06:39:11.980 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.977+0000 7fbab7fff700 1 -- 172.21.15.163:0/3941413606 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbaac003570 con 0x7fbac0154320 2023-10-09T06:39:11.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.977+0000 7fbab7fff700 1 -- 172.21.15.163:0/3941413606 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2300763360 0 0) 0x7fbaac003570 con 0x7fbac0154320 2023-10-09T06:39:11.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.977+0000 7fbab7fff700 1 -- 172.21.15.163:0/3941413606 >> v2:172.21.15.181:3300/0 conn(0x7fbac0153f50 msgr2=0x7fbac00b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:11.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.977+0000 7fbab7fff700 1 --2- 172.21.15.163:0/3941413606 >> v2:172.21.15.181:3300/0 conn(0x7fbac0153f50 0x7fbac00b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:11.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.977+0000 7fbab7fff700 1 -- 172.21.15.163:0/3941413606 >> v1:172.21.15.181:6789/0 conn(0x7fbac006d840 legacy=0x7fbac0069e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:11.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.977+0000 7fbab7fff700 1 -- 172.21.15.163:0/3941413606 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac015acf0 con 0x7fbac0154320 2023-10-09T06:39:11.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.977+0000 7fbab7fff700 1 -- 172.21.15.163:0/3941413606 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fbab0001470 con 0x7fbac0154320 2023-10-09T06:39:11.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.977+0000 7fbab7fff700 1 -- 172.21.15.163:0/3941413606 >> v1:172.21.15.163:6789/0 conn(0x7fbac0154320 legacy=0x7fbac006d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:11.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.977+0000 7fbab7fff700 1 --2- 172.21.15.163:0/3941413606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaac003a60 0x7fbaac003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:11.981 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.977+0000 7fbab7fff700 1 --2- 172.21.15.163:0/3941413606 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbaac004810 0x7fbaac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:11.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.977+0000 7fbab7fff700 1 --2- 172.21.15.163:0/3941413606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaac0051c0 0x7fbaac00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:11.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.977+0000 7fbab7fff700 1 -- 172.21.15.163:0/3941413606 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac015aa10 con 0x7fbaac003a60 2023-10-09T06:39:11.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.977+0000 7fbab7fff700 1 -- 172.21.15.163:0/3941413606 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaac00c0d0 con 0x7fbaac0051c0 2023-10-09T06:39:11.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.977+0000 7fbab7fff700 1 -- 172.21.15.163:0/3941413606 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaac00c290 con 0x7fbaac004810 2023-10-09T06:39:11.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.977+0000 7fbac54b1700 1 --2- 172.21.15.163:0/3941413606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaac003a60 0x7fbaac003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:11.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.977+0000 7fbac4cb0700 1 --2- 172.21.15.163:0/3941413606 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbaac004810 0x7fbaac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:11.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.977+0000 7fbac5cb2700 1 --2- 172.21.15.163:0/3941413606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaac0051c0 0x7fbaac00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:11.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.978+0000 7fbac54b1700 1 -- 172.21.15.163:0/3941413606 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbaac004810 msgr2=0x7fbaac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:11.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.978+0000 7fbac54b1700 1 --2- 172.21.15.163:0/3941413606 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbaac004810 0x7fbaac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:11.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.978+0000 7fbac54b1700 1 -- 172.21.15.163:0/3941413606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaac0051c0 msgr2=0x7fbaac00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:11.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.978+0000 7fbac54b1700 1 --2- 172.21.15.163:0/3941413606 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaac0051c0 0x7fbaac00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:11.982 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.978+0000 7fbac54b1700 1 -- 172.21.15.163:0/3941413606 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaac003570 con 0x7fbaac003a60 2023-10-09T06:39:11.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.978+0000 7fbac5cb2700 1 -- 172.21.15.163:0/3941413606 reap_dead start 2023-10-09T06:39:11.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.978+0000 7fbac54b1700 1 --2- 172.21.15.163:0/3941413606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaac003a60 0x7fbaac003e70 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fbabc00e210 tx=0x7fbabc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=176688a83b95ab0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:11.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.978+0000 7fbab7fff700 1 -- 172.21.15.163:0/3941413606 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc00eac0 con 0x7fbaac003a60 2023-10-09T06:39:11.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.978+0000 7fbab7fff700 1 -- 172.21.15.163:0/3941413606 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbabc00f040 con 0x7fbaac003a60 2023-10-09T06:39:11.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.978+0000 7fbab7fff700 1 -- 172.21.15.163:0/3941413606 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc00ee70 con 0x7fbaac003a60 2023-10-09T06:39:11.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.978+0000 7fbac7715700 1 -- 172.21.15.163:0/3941413606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaac003a60 msgr2=0x7fbaac003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:11.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.978+0000 7fbac7715700 1 --2- 172.21.15.163:0/3941413606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaac003a60 0x7fbaac003e70 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fbabc00e210 tx=0x7fbabc00b040 comp rx=0 tx=0).stop 2023-10-09T06:39:11.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.978+0000 7fbac7715700 1 -- 172.21.15.163:0/3941413606 shutdown_connections 2023-10-09T06:39:11.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.978+0000 7fbac7715700 1 --2- 172.21.15.163:0/3941413606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaac003a60 0x7fbaac003e70 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:11.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.978+0000 7fbac7715700 1 -- 172.21.15.163:0/3941413606 >> 172.21.15.163:0/3941413606 conn(0x7fbac006baf0 msgr2=0x7fbac006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:11.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.979+0000 7fbac7715700 1 -- 172.21.15.163:0/3941413606 shutdown_connections 2023-10-09T06:39:11.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.979+0000 7fbac7715700 1 -- 172.21.15.163:0/3941413606 wait complete. 2023-10-09T06:39:11.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.980+0000 7fbac7715700 1 Processor -- start 2023-10-09T06:39:11.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.980+0000 7fbac7715700 1 -- start start 2023-10-09T06:39:11.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.980+0000 7fbac7715700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaac003a60 0x7fbac00ae510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:11.986 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.980+0000 7fbac7715700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac00aeef0 0x7fbac014d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:11.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.980+0000 7fbac7715700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbac014fb50 0x7fbac015c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:11.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.980+0000 7fbac7715700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac00ae080 con 0x7fbaac003a60 2023-10-09T06:39:11.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.980+0000 7fbac7715700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac014da40 con 0x7fbac00aeef0 2023-10-09T06:39:11.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.980+0000 7fbac7715700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbac014dd20 con 0x7fbac014fb50 2023-10-09T06:39:11.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.981+0000 7fbac54b1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaac003a60 0x7fbac00ae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:11.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.981+0000 7fbac54b1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaac003a60 0x7fbac00ae510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44774/0 (socket says 172.21.15.163:44774) 2023-10-09T06:39:11.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.981+0000 7fbac54b1700 1 -- 172.21.15.163:0/759906862 learned_addr learned my addr 172.21.15.163:0/759906862 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:11.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.981+0000 7fbac4cb0700 1 --2- 172.21.15.163:0/759906862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac00aeef0 0x7fbac014d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:11.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.981+0000 7fbac5cb2700 1 --2- 172.21.15.163:0/759906862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbac014fb50 0x7fbac015c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:11.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.981+0000 7fbac54b1700 1 -- 172.21.15.163:0/759906862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbac014fb50 msgr2=0x7fbac015c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:11.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.981+0000 7fbac54b1700 1 --2- 172.21.15.163:0/759906862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbac014fb50 0x7fbac015c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:11.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.981+0000 7fbac54b1700 1 -- 172.21.15.163:0/759906862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac00aeef0 msgr2=0x7fbac014d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:11.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.981+0000 7fbac54b1700 1 --2- 172.21.15.163:0/759906862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac00aeef0 0x7fbac014d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:11.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.981+0000 7fbac54b1700 1 -- 172.21.15.163:0/759906862 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac00add10 con 0x7fbaac003a60 2023-10-09T06:39:11.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.981+0000 7fbac54b1700 1 --2- 172.21.15.163:0/759906862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaac003a60 0x7fbac00ae510 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fbabc00afd0 tx=0x7fbabc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:11.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.981+0000 7fbab57fa700 1 -- 172.21.15.163:0/759906862 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc01d420 con 0x7fbaac003a60 2023-10-09T06:39:11.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.981+0000 7fbab57fa700 1 -- 172.21.15.163:0/759906862 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbabc01d5a0 con 0x7fbaac003a60 2023-10-09T06:39:11.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.981+0000 7fbab57fa700 1 -- 172.21.15.163:0/759906862 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc01d9a0 con 0x7fbaac003a60 2023-10-09T06:39:11.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.981+0000 7fbac7715700 1 -- 172.21.15.163:0/759906862 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbac014e000 con 0x7fbaac003a60 2023-10-09T06:39:11.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.981+0000 7fbac7715700 1 -- 172.21.15.163:0/759906862 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac014d610 con 0x7fbaac003a60 2023-10-09T06:39:11.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.982+0000 7fbac7715700 1 -- 172.21.15.163:0/759906862 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbac00718b0 con 0x7fbaac003a60 2023-10-09T06:39:11.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.982+0000 7fbab57fa700 1 -- 172.21.15.163:0/759906862 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbabc003040 con 0x7fbaac003a60 2023-10-09T06:39:11.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.983+0000 7fbab57fa700 1 --2- 172.21.15.163:0/759906862 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbaa4067320 0x7fbaa40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:11.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.983+0000 7fbab57fa700 1 -- 172.21.15.163:0/759906862 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6266+0+0 (secure 0 0 0) 0x7fbabc087460 con 0x7fbaac003a60 2023-10-09T06:39:11.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.983+0000 7fbac4cb0700 1 --2- 172.21.15.163:0/759906862 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbaa4067320 0x7fbaa40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:11.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.983+0000 7fbac4cb0700 1 --2- 172.21.15.163:0/759906862 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbaa4067320 0x7fbaa40697d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fbab0001050 tx=0x7fbab0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:11.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:11.987+0000 7fbab57fa700 1 -- 172.21.15.163:0/759906862 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbabc053db0 con 0x7fbaac003a60 2023-10-09T06:39:12.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.172+0000 7fbac7715700 1 -- 172.21.15.163:0/759906862 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000836565.meta", "pool2": "cephfs.volume_0000000000836565.meta", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbac015afd0 con 0x7fbaac003a60 2023-10-09T06:39:12.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.850+0000 7fbab57fa700 1 -- 172.21.15.163:0/759906862 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000836565.meta", "pool2": "cephfs.volume_0000000000836565.meta", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000836565.meta' removed v171) v1 ==== 241+0+0 (secure 0 0 0) 0x7fbabc012340 con 0x7fbaac003a60 2023-10-09T06:39:12.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.853+0000 7fbac7715700 1 -- 172.21.15.163:0/759906862 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbaa4067320 msgr2=0x7fbaa40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:12.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.853+0000 7fbac7715700 1 --2- 172.21.15.163:0/759906862 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbaa4067320 0x7fbaa40697d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fbab0001050 tx=0x7fbab0009040 comp rx=0 tx=0).stop 2023-10-09T06:39:12.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.853+0000 7fbac7715700 1 -- 172.21.15.163:0/759906862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaac003a60 msgr2=0x7fbac00ae510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:12.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.853+0000 7fbac7715700 1 --2- 172.21.15.163:0/759906862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaac003a60 0x7fbac00ae510 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fbabc00afd0 tx=0x7fbabc01a000 comp rx=0 tx=0).stop 2023-10-09T06:39:12.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.854+0000 7fbac7715700 1 -- 172.21.15.163:0/759906862 shutdown_connections 2023-10-09T06:39:12.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.854+0000 7fbac7715700 1 --2- 172.21.15.163:0/759906862 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbaa4067320 0x7fbaa40697d0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:12.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.854+0000 7fbac7715700 1 --2- 172.21.15.163:0/759906862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaac003a60 0x7fbac00ae510 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:12.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.854+0000 7fbac7715700 1 --2- 172.21.15.163:0/759906862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbac014fb50 0x7fbac015c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:12.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.854+0000 7fbac7715700 1 --2- 172.21.15.163:0/759906862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac00aeef0 0x7fbac014d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:12.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.854+0000 7fbac7715700 1 -- 172.21.15.163:0/759906862 >> 172.21.15.163:0/759906862 conn(0x7fbac006baf0 msgr2=0x7fbac006ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:12.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.854+0000 7fbac7715700 1 -- 172.21.15.163:0/759906862 shutdown_connections 2023-10-09T06:39:12.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.854+0000 7fbac7715700 1 -- 172.21.15.163:0/759906862 wait complete. 2023-10-09T06:39:12.856 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs.volume_0000000000836565.meta' removed 2023-10-09T06:39:12.868 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-09T06:39:12.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.992+0000 7fb232446700 1 Processor -- start 2023-10-09T06:39:12.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.992+0000 7fb232446700 1 -- start start 2023-10-09T06:39:13.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.993+0000 7fb232446700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb22c0b0790 0x7fb22c0cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:13.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.993+0000 7fb232446700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb22c0cfa00 con 0x7fb22c0b03c0 2023-10-09T06:39:13.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.993+0000 7fb232446700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb22c0cb580 con 0x7fb22c1591e0 2023-10-09T06:39:13.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.993+0000 7fb232446700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c0cf160 con 0x7fb22c0b0790 2023-10-09T06:39:13.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.993+0000 7fb231444700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb22c0b03c0 0x7fb22c0b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:51948/0 (socket says 172.21.15.163:51948) 2023-10-09T06:39:13.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.993+0000 7fb231444700 1 -- 172.21.15.163:0/1186500199 learned_addr learned my addr 172.21.15.163:0/1186500199 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:13.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.993+0000 7fb223fff700 1 -- 172.21.15.163:0/1186500199 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2628875069 0 0) 0x7fb22c0cfa00 con 0x7fb22c0b03c0 2023-10-09T06:39:13.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.993+0000 7fb230c43700 1 --2- 172.21.15.163:0/1186500199 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb22c0b0790 0x7fb22c0cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:13.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.993+0000 7fb223fff700 1 -- 172.21.15.163:0/1186500199 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb218003570 con 0x7fb22c0b03c0 2023-10-09T06:39:13.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.994+0000 7fb223fff700 1 -- 172.21.15.163:0/1186500199 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 699007214 0 0) 0x7fb218003570 con 0x7fb22c0b03c0 2023-10-09T06:39:13.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.994+0000 7fb223fff700 1 -- 172.21.15.163:0/1186500199 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb22c0b0790 msgr2=0x7fb22c0cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:13.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.994+0000 7fb223fff700 1 --2- 172.21.15.163:0/1186500199 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb22c0b0790 0x7fb22c0cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:13.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.994+0000 7fb223fff700 1 -- 172.21.15.163:0/1186500199 >> v1:172.21.15.181:6789/0 conn(0x7fb22c1591e0 legacy=0x7fb22c159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:13.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.994+0000 7fb223fff700 1 -- 172.21.15.163:0/1186500199 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb22c0cf440 con 0x7fb22c0b03c0 2023-10-09T06:39:13.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.994+0000 7fb223fff700 1 -- 172.21.15.163:0/1186500199 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fb228001470 con 0x7fb22c0b03c0 2023-10-09T06:39:13.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.994+0000 7fb223fff700 1 -- 172.21.15.163:0/1186500199 >> v1:172.21.15.163:6789/0 conn(0x7fb22c0b03c0 legacy=0x7fb22c0b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:13.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.994+0000 7fb223fff700 1 --2- 172.21.15.163:0/1186500199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb218003a90 0x7fb218003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:13.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.994+0000 7fb223fff700 1 --2- 172.21.15.163:0/1186500199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb218004810 0x7fb218004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:13.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.994+0000 7fb223fff700 1 --2- 172.21.15.163:0/1186500199 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb22c0b0790 0x7fb2180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:13.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.994+0000 7fb223fff700 1 -- 172.21.15.163:0/1186500199 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c0cf160 con 0x7fb218004810 2023-10-09T06:39:13.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.994+0000 7fb223fff700 1 -- 172.21.15.163:0/1186500199 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21800bb40 con 0x7fb218003a90 2023-10-09T06:39:13.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.994+0000 7fb223fff700 1 -- 172.21.15.163:0/1186500199 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb21800bd00 con 0x7fb22c0b0790 2023-10-09T06:39:13.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.994+0000 7fb230c43700 1 --2- 172.21.15.163:0/1186500199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb218004810 0x7fb218004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:13.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.994+0000 7fb231444700 1 --2- 172.21.15.163:0/1186500199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb218003a90 0x7fb218003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:13.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.994+0000 7fb231c45700 1 --2- 172.21.15.163:0/1186500199 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb22c0b0790 0x7fb2180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:13.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.994+0000 7fb230c43700 1 -- 172.21.15.163:0/1186500199 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb22c0b0790 msgr2=0x7fb2180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:13.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.994+0000 7fb230c43700 1 --2- 172.21.15.163:0/1186500199 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb22c0b0790 0x7fb2180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:13.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.994+0000 7fb230c43700 1 -- 172.21.15.163:0/1186500199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb218003a90 msgr2=0x7fb218003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:13.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.994+0000 7fb230c43700 1 --2- 172.21.15.163:0/1186500199 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb218003a90 0x7fb218003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:13.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.994+0000 7fb230c43700 1 -- 172.21.15.163:0/1186500199 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb218003570 con 0x7fb218004810 2023-10-09T06:39:13.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.994+0000 7fb231c45700 1 --2- 172.21.15.163:0/1186500199 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb22c0b0790 0x7fb2180051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:13.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.994+0000 7fb230c43700 1 --2- 172.21.15.163:0/1186500199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb218004810 0x7fb218004c80 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fb21c009c00 tx=0x7fb21c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d705bdd417da8d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:13.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.994+0000 7fb223fff700 1 -- 172.21.15.163:0/1186500199 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c00e710 con 0x7fb218004810 2023-10-09T06:39:13.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.995+0000 7fb223fff700 1 -- 172.21.15.163:0/1186500199 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21c00e870 con 0x7fb218004810 2023-10-09T06:39:13.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.995+0000 7fb223fff700 1 -- 172.21.15.163:0/1186500199 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c00ec30 con 0x7fb218004810 2023-10-09T06:39:13.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.995+0000 7fb232446700 1 -- 172.21.15.163:0/1186500199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb218004810 msgr2=0x7fb218004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:13.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.995+0000 7fb232446700 1 --2- 172.21.15.163:0/1186500199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb218004810 0x7fb218004c80 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fb21c009c00 tx=0x7fb21c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:13.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.995+0000 7fb231c45700 1 -- 172.21.15.163:0/1186500199 reap_dead start 2023-10-09T06:39:13.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.995+0000 7fb232446700 1 -- 172.21.15.163:0/1186500199 shutdown_connections 2023-10-09T06:39:13.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.995+0000 7fb232446700 1 -- 172.21.15.163:0/1186500199 >> 172.21.15.163:0/1186500199 conn(0x7fb22c063230 msgr2=0x7fb22c063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:13.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.995+0000 7fb232446700 1 -- 172.21.15.163:0/1186500199 shutdown_connections 2023-10-09T06:39:13.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.995+0000 7fb232446700 1 -- 172.21.15.163:0/1186500199 wait complete. 2023-10-09T06:39:13.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.996+0000 7fb232446700 1 Processor -- start 2023-10-09T06:39:13.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.996+0000 7fb232446700 1 -- start start 2023-10-09T06:39:13.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.997+0000 7fb232446700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb218004810 0x7fb22c1568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:13.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.997+0000 7fb232446700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb22c0c5700 0x7fb22c0c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:13.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.997+0000 7fb232446700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb22c0c5c50 0x7fb22c0c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:13.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.997+0000 7fb232446700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c0c94a0 con 0x7fb218004810 2023-10-09T06:39:13.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.997+0000 7fb232446700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c0c65e0 con 0x7fb22c0c5700 2023-10-09T06:39:13.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.997+0000 7fb232446700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c0c8890 con 0x7fb22c0c5c50 2023-10-09T06:39:13.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.997+0000 7fb231444700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb218004810 0x7fb22c1568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:13.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.997+0000 7fb231444700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb218004810 0x7fb22c1568e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44802/0 (socket says 172.21.15.163:44802) 2023-10-09T06:39:13.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.997+0000 7fb231444700 1 -- 172.21.15.163:0/3374537624 learned_addr learned my addr 172.21.15.163:0/3374537624 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:13.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.997+0000 7fb230c43700 1 --2- 172.21.15.163:0/3374537624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb22c0c5700 0x7fb22c0c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:13.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.997+0000 7fb231444700 1 -- 172.21.15.163:0/3374537624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb22c0c5c50 msgr2=0x7fb22c0c60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:13.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.997+0000 7fb231444700 1 --2- 172.21.15.163:0/3374537624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb22c0c5c50 0x7fb22c0c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:13.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.997+0000 7fb231444700 1 -- 172.21.15.163:0/3374537624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb22c0c5700 msgr2=0x7fb22c0c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:13.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.997+0000 7fb231444700 1 --2- 172.21.15.163:0/3374537624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb22c0c5700 0x7fb22c0c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:13.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.998+0000 7fb231444700 1 -- 172.21.15.163:0/3374537624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb22c0c8b70 con 0x7fb218004810 2023-10-09T06:39:13.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.998+0000 7fb230c43700 1 --2- 172.21.15.163:0/3374537624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb22c0c5700 0x7fb22c0c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:13.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.998+0000 7fb231444700 1 --2- 172.21.15.163:0/3374537624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb218004810 0x7fb22c1568e0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fb228001c20 tx=0x7fb22c0b0c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:13.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.998+0000 7fb2217fa700 1 -- 172.21.15.163:0/3374537624 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb228018030 con 0x7fb218004810 2023-10-09T06:39:13.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.998+0000 7fb2217fa700 1 -- 172.21.15.163:0/3374537624 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22800f8d0 con 0x7fb218004810 2023-10-09T06:39:13.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.998+0000 7fb232446700 1 -- 172.21.15.163:0/3374537624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb22c0c8e50 con 0x7fb218004810 2023-10-09T06:39:13.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.998+0000 7fb2217fa700 1 -- 172.21.15.163:0/3374537624 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22800fc70 con 0x7fb218004810 2023-10-09T06:39:13.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.998+0000 7fb232446700 1 -- 172.21.15.163:0/3374537624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb22c156e20 con 0x7fb218004810 2023-10-09T06:39:13.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.999+0000 7fb232446700 1 -- 172.21.15.163:0/3374537624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb22c0c9830 con 0x7fb218004810 2023-10-09T06:39:13.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.999+0000 7fb2217fa700 1 -- 172.21.15.163:0/3374537624 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb22801b070 con 0x7fb218004810 2023-10-09T06:39:13.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:12.999+0000 7fb2217fa700 1 --2- 172.21.15.163:0/3374537624 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb210067320 0x7fb2100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:13.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.000+0000 7fb2217fa700 1 -- 172.21.15.163:0/3374537624 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fb228019070 con 0x7fb218004810 2023-10-09T06:39:13.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.000+0000 7fb230c43700 1 --2- 172.21.15.163:0/3374537624 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb210067320 0x7fb2100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:13.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.000+0000 7fb230c43700 1 --2- 172.21.15.163:0/3374537624 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb210067320 0x7fb2100697d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb22c063a10 tx=0x7fb21c01a000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:13.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.004+0000 7fb2217fa700 1 -- 172.21.15.163:0/3374537624 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2280518e0 con 0x7fb218004810 2023-10-09T06:39:13.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.188+0000 7fb232446700 1 -- 172.21.15.163:0/3374537624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb22c04c3c0 con 0x7fb218004810 2023-10-09T06:39:13.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.853+0000 7fb2217fa700 1 -- 172.21.15.163:0/3374537624 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v172) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb228054f00 con 0x7fb218004810 2023-10-09T06:39:13.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.856+0000 7fb232446700 1 -- 172.21.15.163:0/3374537624 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb210067320 msgr2=0x7fb2100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:13.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.856+0000 7fb232446700 1 --2- 172.21.15.163:0/3374537624 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb210067320 0x7fb2100697d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb22c063a10 tx=0x7fb21c01a000 comp rx=0 tx=0).stop 2023-10-09T06:39:13.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.856+0000 7fb232446700 1 -- 172.21.15.163:0/3374537624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb218004810 msgr2=0x7fb22c1568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:13.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.856+0000 7fb232446700 1 --2- 172.21.15.163:0/3374537624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb218004810 0x7fb22c1568e0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fb228001c20 tx=0x7fb22c0b0c40 comp rx=0 tx=0).stop 2023-10-09T06:39:13.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.856+0000 7fb232446700 1 -- 172.21.15.163:0/3374537624 shutdown_connections 2023-10-09T06:39:13.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.856+0000 7fb232446700 1 --2- 172.21.15.163:0/3374537624 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb210067320 0x7fb2100697d0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:13.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.856+0000 7fb232446700 1 --2- 172.21.15.163:0/3374537624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb218004810 0x7fb22c1568e0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:13.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.856+0000 7fb232446700 1 --2- 172.21.15.163:0/3374537624 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb22c0c5c50 0x7fb22c0c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:13.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.856+0000 7fb232446700 1 --2- 172.21.15.163:0/3374537624 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb22c0c5700 0x7fb22c0c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:13.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.856+0000 7fb232446700 1 -- 172.21.15.163:0/3374537624 >> 172.21.15.163:0/3374537624 conn(0x7fb22c063230 msgr2=0x7fb22c0b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:13.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.857+0000 7fb232446700 1 -- 172.21.15.163:0/3374537624 shutdown_connections 2023-10-09T06:39:13.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.857+0000 7fb232446700 1 -- 172.21.15.163:0/3374537624 wait complete. 2023-10-09T06:39:13.858 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_data' removed 2023-10-09T06:39:13.870 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm another-data-pool another-data-pool --yes-i-really-really-mean-it 2023-10-09T06:39:13.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.989+0000 7fc0fb557700 1 Processor -- start 2023-10-09T06:39:13.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.989+0000 7fc0fb557700 1 -- start start 2023-10-09T06:39:13.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.990+0000 7fc0fb557700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc0f40b03c0 0x7fc0f40cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:13.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.990+0000 7fc0fb557700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc0f41591e0 0x7fc0f4159600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:13.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.990+0000 7fc0fb557700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0f40cfa00 con 0x7fc0f40b0790 2023-10-09T06:39:13.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.990+0000 7fc0fb557700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0f40cb580 con 0x7fc0f41591e0 2023-10-09T06:39:13.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.990+0000 7fc0fb557700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0f40cf160 con 0x7fc0f40b03c0 2023-10-09T06:39:13.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.990+0000 7fc0fad56700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc0f41591e0 0x7fc0f4159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:13.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.990+0000 7fc0fad56700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc0f41591e0 0x7fc0f4159600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44804/0 (socket says 172.21.15.163:44804) 2023-10-09T06:39:13.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.990+0000 7fc0f9d54700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fc0f40b0790 0x7fc0f40b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:51950/0 (socket says 172.21.15.163:51950) 2023-10-09T06:39:13.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.990+0000 7fc0fad56700 1 -- 172.21.15.163:0/3016148354 learned_addr learned my addr 172.21.15.163:0/3016148354 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:13.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.990+0000 7fc0fa555700 1 --2- 172.21.15.163:0/3016148354 >> v2:172.21.15.181:3300/0 conn(0x7fc0f40b03c0 0x7fc0f40cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:13.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.990+0000 7fc0fad56700 1 -- 172.21.15.163:0/3016148354 >> v2:172.21.15.181:3300/0 conn(0x7fc0f40b03c0 msgr2=0x7fc0f40cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:13.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.990+0000 7fc0fad56700 1 --2- 172.21.15.163:0/3016148354 >> v2:172.21.15.181:3300/0 conn(0x7fc0f40b03c0 0x7fc0f40cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:13.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.990+0000 7fc0fad56700 1 -- 172.21.15.163:0/3016148354 >> v1:172.21.15.163:6789/0 conn(0x7fc0f40b0790 legacy=0x7fc0f40b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:13.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.990+0000 7fc0f9553700 1 -- 172.21.15.163:0/3016148354 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 706143512 0 0) 0x7fc0f40cfa00 con 0x7fc0f40b0790 2023-10-09T06:39:13.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.990+0000 7fc0fad56700 1 -- 172.21.15.163:0/3016148354 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f40cf440 con 0x7fc0f41591e0 2023-10-09T06:39:13.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.990+0000 7fc0fad56700 1 --2- 172.21.15.163:0/3016148354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0f41591e0 0x7fc0f4159600 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fc0e40099b0 tx=0x7fc0e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a94f57348d368c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:13.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.991+0000 7fc0f9553700 1 -- 172.21.15.163:0/3016148354 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e400b820 con 0x7fc0f41591e0 2023-10-09T06:39:13.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.991+0000 7fc0f9553700 1 -- 172.21.15.163:0/3016148354 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e400b980 con 0x7fc0f41591e0 2023-10-09T06:39:13.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.991+0000 7fc0f9553700 1 -- 172.21.15.163:0/3016148354 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e400bd40 con 0x7fc0f41591e0 2023-10-09T06:39:13.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.991+0000 7fc0fb557700 1 -- 172.21.15.163:0/3016148354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0f41591e0 msgr2=0x7fc0f4159600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:13.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.991+0000 7fc0fb557700 1 --2- 172.21.15.163:0/3016148354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0f41591e0 0x7fc0f4159600 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fc0e40099b0 tx=0x7fc0e4009250 comp rx=0 tx=0).stop 2023-10-09T06:39:13.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.991+0000 7fc0fb557700 1 -- 172.21.15.163:0/3016148354 shutdown_connections 2023-10-09T06:39:13.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.991+0000 7fc0fb557700 1 --2- 172.21.15.163:0/3016148354 >> v2:172.21.15.181:3300/0 conn(0x7fc0f40b03c0 0x7fc0f40cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:13.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.991+0000 7fc0fb557700 1 --2- 172.21.15.163:0/3016148354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0f41591e0 0x7fc0f4159600 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:13.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.991+0000 7fc0fb557700 1 -- 172.21.15.163:0/3016148354 >> 172.21.15.163:0/3016148354 conn(0x7fc0f4063230 msgr2=0x7fc0f4063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:13.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.991+0000 7fc0fb557700 1 -- 172.21.15.163:0/3016148354 shutdown_connections 2023-10-09T06:39:13.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.991+0000 7fc0fb557700 1 -- 172.21.15.163:0/3016148354 wait complete. 2023-10-09T06:39:13.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.992+0000 7fc0fb557700 1 Processor -- start 2023-10-09T06:39:13.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.992+0000 7fc0fb557700 1 -- start start 2023-10-09T06:39:13.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.993+0000 7fc0fb557700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0f40b03c0 0x7fc0f415a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:13.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.993+0000 7fc0fb557700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0f40b0790 0x7fc0f40c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:13.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.993+0000 7fc0fb557700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0f41591e0 0x7fc0f4157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:13.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.993+0000 7fc0fb557700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f416d9e0 con 0x7fc0f41591e0 2023-10-09T06:39:13.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.993+0000 7fc0fb557700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f40c5700 con 0x7fc0f40b03c0 2023-10-09T06:39:13.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.993+0000 7fc0fb557700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f415c780 con 0x7fc0f40b0790 2023-10-09T06:39:13.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.993+0000 7fc0fad56700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0f41591e0 0x7fc0f4157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:13.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.993+0000 7fc0fad56700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0f41591e0 0x7fc0f4157c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44816/0 (socket says 172.21.15.163:44816) 2023-10-09T06:39:13.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.993+0000 7fc0fad56700 1 -- 172.21.15.163:0/3222364913 learned_addr learned my addr 172.21.15.163:0/3222364913 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:13.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.993+0000 7fc0fa555700 1 --2- 172.21.15.163:0/3222364913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0f40b03c0 0x7fc0f415a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:13.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.993+0000 7fc0f9d54700 1 --2- 172.21.15.163:0/3222364913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0f40b0790 0x7fc0f40c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:13.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.993+0000 7fc0fad56700 1 -- 172.21.15.163:0/3222364913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0f40b0790 msgr2=0x7fc0f40c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:13.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.993+0000 7fc0fad56700 1 --2- 172.21.15.163:0/3222364913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0f40b0790 0x7fc0f40c8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:13.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.993+0000 7fc0fad56700 1 -- 172.21.15.163:0/3222364913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0f40b03c0 msgr2=0x7fc0f415a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:13.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.993+0000 7fc0fad56700 1 --2- 172.21.15.163:0/3222364913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0f40b03c0 0x7fc0f415a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:13.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.994+0000 7fc0fad56700 1 -- 172.21.15.163:0/3222364913 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e40094f0 con 0x7fc0f41591e0 2023-10-09T06:39:13.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.994+0000 7fc0f9d54700 1 --2- 172.21.15.163:0/3222364913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0f40b0790 0x7fc0f40c8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:13.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.994+0000 7fc0fa555700 1 --2- 172.21.15.163:0/3222364913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0f40b03c0 0x7fc0f415a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:13.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.994+0000 7fc0fad56700 1 --2- 172.21.15.163:0/3222364913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0f41591e0 0x7fc0f4157c10 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40cfe40 tx=0x7fc0e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:13.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.994+0000 7fc0f27fc700 1 -- 172.21.15.163:0/3222364913 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e401f070 con 0x7fc0f41591e0 2023-10-09T06:39:13.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.994+0000 7fc0f27fc700 1 -- 172.21.15.163:0/3222364913 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e4002f30 con 0x7fc0f41591e0 2023-10-09T06:39:13.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.994+0000 7fc0f27fc700 1 -- 172.21.15.163:0/3222364913 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4025690 con 0x7fc0f41591e0 2023-10-09T06:39:13.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.994+0000 7fc0fb557700 1 -- 172.21.15.163:0/3222364913 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f416ce60 con 0x7fc0f41591e0 2023-10-09T06:39:13.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.994+0000 7fc0fb557700 1 -- 172.21.15.163:0/3222364913 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f416d1f0 con 0x7fc0f41591e0 2023-10-09T06:39:13.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.995+0000 7fc0fb557700 1 -- 172.21.15.163:0/3222364913 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0f404c3c0 con 0x7fc0f41591e0 2023-10-09T06:39:13.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.995+0000 7fc0f27fc700 1 -- 172.21.15.163:0/3222364913 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc0e4023070 con 0x7fc0f41591e0 2023-10-09T06:39:13.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.996+0000 7fc0f27fc700 1 --2- 172.21.15.163:0/3222364913 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc0e0067320 0x7fc0e00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:13.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.996+0000 7fc0f27fc700 1 -- 172.21.15.163:0/3222364913 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5360+0+0 (secure 0 0 0) 0x7fc0e4086000 con 0x7fc0f41591e0 2023-10-09T06:39:13.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.996+0000 7fc0fa555700 1 --2- 172.21.15.163:0/3222364913 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc0e0067320 0x7fc0e00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:13.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:13.996+0000 7fc0fa555700 1 --2- 172.21.15.163:0/3222364913 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc0e0067320 0x7fc0e00697d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40cde20 tx=0x7fc0ec008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:13.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.000+0000 7fc0f27fc700 1 -- 172.21.15.163:0/3222364913 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc0e4052ea0 con 0x7fc0f41591e0 2023-10-09T06:39:14.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.183+0000 7fc0fb557700 1 -- 172.21.15.163:0/3222364913 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "another-data-pool", "pool2": "another-data-pool", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc0f416d7c0 con 0x7fc0f41591e0 2023-10-09T06:39:14.235 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:39:14.238 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:39:14.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.860+0000 7fc0f27fc700 1 -- 172.21.15.163:0/3222364913 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "another-data-pool", "pool2": "another-data-pool", "yes_i_really_really_mean_it": true}]=0 pool 'another-data-pool' removed v173) v1 ==== 187+0+0 (secure 0 0 0) 0x7fc0e402a0e0 con 0x7fc0f41591e0 2023-10-09T06:39:14.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.863+0000 7fc0fb557700 1 -- 172.21.15.163:0/3222364913 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc0e0067320 msgr2=0x7fc0e00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:14.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.863+0000 7fc0fb557700 1 --2- 172.21.15.163:0/3222364913 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc0e0067320 0x7fc0e00697d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40cde20 tx=0x7fc0ec008040 comp rx=0 tx=0).stop 2023-10-09T06:39:14.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.863+0000 7fc0fb557700 1 -- 172.21.15.163:0/3222364913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0f41591e0 msgr2=0x7fc0f4157c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:14.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.863+0000 7fc0fb557700 1 --2- 172.21.15.163:0/3222364913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0f41591e0 0x7fc0f4157c10 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40cfe40 tx=0x7fc0e4017000 comp rx=0 tx=0).stop 2023-10-09T06:39:14.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.863+0000 7fc0fb557700 1 -- 172.21.15.163:0/3222364913 shutdown_connections 2023-10-09T06:39:14.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.863+0000 7fc0fb557700 1 --2- 172.21.15.163:0/3222364913 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc0e0067320 0x7fc0e00697d0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:14.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.863+0000 7fc0fb557700 1 --2- 172.21.15.163:0/3222364913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0f41591e0 0x7fc0f4157c10 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:14.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.863+0000 7fc0fb557700 1 --2- 172.21.15.163:0/3222364913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0f40b03c0 0x7fc0f415a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:14.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.863+0000 7fc0fb557700 1 --2- 172.21.15.163:0/3222364913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0f40b0790 0x7fc0f40c8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:14.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.863+0000 7fc0fb557700 1 -- 172.21.15.163:0/3222364913 >> 172.21.15.163:0/3222364913 conn(0x7fc0f4063230 msgr2=0x7fc0f40ca0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:14.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.863+0000 7fc0fb557700 1 -- 172.21.15.163:0/3222364913 shutdown_connections 2023-10-09T06:39:14.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.863+0000 7fc0fb557700 1 -- 172.21.15.163:0/3222364913 wait complete. 2023-10-09T06:39:14.867 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'another-data-pool' removed 2023-10-09T06:39:14.876 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-09T06:39:14.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.993+0000 7fd9ce687700 1 Processor -- start 2023-10-09T06:39:14.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.993+0000 7fd9ce687700 1 -- start start 2023-10-09T06:39:14.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.994+0000 7fd9ce687700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd9c80b4e70 0x7fd9c80ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:14.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.994+0000 7fd9ce687700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9c80b1b40 0x7fd9c815d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:14.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.994+0000 7fd9ce687700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c80cfb00 con 0x7fd9c80b16d0 2023-10-09T06:39:14.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.994+0000 7fd9ce687700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9c80b5c40 con 0x7fd9c80b4e70 2023-10-09T06:39:14.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.994+0000 7fd9ce687700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c80b5f20 con 0x7fd9c80b1b40 2023-10-09T06:39:14.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.994+0000 7fd9cde86700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9c80b1b40 0x7fd9c815d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:14.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.994+0000 7fd9cde86700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9c80b1b40 0x7fd9c815d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:45106/0 (socket says 172.21.15.163:45106) 2023-10-09T06:39:14.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.994+0000 7fd9cde86700 1 -- 172.21.15.163:0/4206482830 learned_addr learned my addr 172.21.15.163:0/4206482830 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:14.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.995+0000 7fd9cd685700 1 --2- 172.21.15.163:0/4206482830 >> v2:172.21.15.181:3300/0 conn(0x7fd9c80b4e70 0x7fd9c80ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:14.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.995+0000 7fd9bffff700 1 -- 172.21.15.163:0/4206482830 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2809861974 0 0) 0x7fd9c80cfb00 con 0x7fd9c80b16d0 2023-10-09T06:39:14.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.995+0000 7fd9bffff700 1 -- 172.21.15.163:0/4206482830 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9b4003570 con 0x7fd9c80b16d0 2023-10-09T06:39:14.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.995+0000 7fd9cde86700 1 -- 172.21.15.163:0/4206482830 >> v2:172.21.15.181:3300/0 conn(0x7fd9c80b4e70 msgr2=0x7fd9c80ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:14.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.995+0000 7fd9cde86700 1 --2- 172.21.15.163:0/4206482830 >> v2:172.21.15.181:3300/0 conn(0x7fd9c80b4e70 0x7fd9c80ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:14.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.995+0000 7fd9cde86700 1 -- 172.21.15.163:0/4206482830 >> v1:172.21.15.181:6789/0 conn(0x7fd9c80b16d0 legacy=0x7fd9c815d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:14.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.995+0000 7fd9cde86700 1 -- 172.21.15.163:0/4206482830 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c80b6200 con 0x7fd9c80b1b40 2023-10-09T06:39:14.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.995+0000 7fd9cd685700 1 --2- 172.21.15.163:0/4206482830 >> v2:172.21.15.181:3300/0 conn(0x7fd9c80b4e70 0x7fd9c80ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:39:14.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.995+0000 7fd9cde86700 1 --2- 172.21.15.163:0/4206482830 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9c80b1b40 0x7fd9c815d950 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd9c00099b0 tx=0x7fd9c0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a04f9a1c12e191b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:14.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.995+0000 7fd9bffff700 1 -- 172.21.15.163:0/4206482830 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c000b680 con 0x7fd9c80b1b40 2023-10-09T06:39:14.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.996+0000 7fd9bffff700 1 -- 172.21.15.163:0/4206482830 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c000b7e0 con 0x7fd9c80b1b40 2023-10-09T06:39:14.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.996+0000 7fd9bffff700 1 -- 172.21.15.163:0/4206482830 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c000bb80 con 0x7fd9c80b1b40 2023-10-09T06:39:14.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.996+0000 7fd9ce687700 1 -- 172.21.15.163:0/4206482830 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9c80b1b40 msgr2=0x7fd9c815d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:14.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.996+0000 7fd9ce687700 1 --2- 172.21.15.163:0/4206482830 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9c80b1b40 0x7fd9c815d950 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd9c00099b0 tx=0x7fd9c0009250 comp rx=0 tx=0).stop 2023-10-09T06:39:14.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.996+0000 7fd9ce687700 1 -- 172.21.15.163:0/4206482830 shutdown_connections 2023-10-09T06:39:14.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.996+0000 7fd9ce687700 1 --2- 172.21.15.163:0/4206482830 >> v2:172.21.15.181:3300/0 conn(0x7fd9c80b4e70 0x7fd9c80ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:14.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.996+0000 7fd9ce687700 1 --2- 172.21.15.163:0/4206482830 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9c80b1b40 0x7fd9c815d950 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:14.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.996+0000 7fd9ce687700 1 -- 172.21.15.163:0/4206482830 >> 172.21.15.163:0/4206482830 conn(0x7fd9c80d10c0 msgr2=0x7fd9c80affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:14.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.997+0000 7fd9ce687700 1 -- 172.21.15.163:0/4206482830 shutdown_connections 2023-10-09T06:39:14.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.997+0000 7fd9ce687700 1 -- 172.21.15.163:0/4206482830 wait complete. 2023-10-09T06:39:14.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.999+0000 7fd9ce687700 1 Processor -- start 2023-10-09T06:39:14.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:14.999+0000 7fd9ce687700 1 -- start start 2023-10-09T06:39:15.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.000+0000 7fd9ce687700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9c80b16d0 0x7fd9c8068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:15.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.000+0000 7fd9ce687700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9c80b1b40 0x7fd9c80688d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:15.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.000+0000 7fd9ce687700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9c80b4e70 0x7fd9c816de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:15.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.000+0000 7fd9ce687700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c816ee20 con 0x7fd9c80b4e70 2023-10-09T06:39:15.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.000+0000 7fd9ce687700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c816f7a0 con 0x7fd9c80b16d0 2023-10-09T06:39:15.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.000+0000 7fd9ce687700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c816eb40 con 0x7fd9c80b1b40 2023-10-09T06:39:15.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.001+0000 7fd9cde86700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9c80b4e70 0x7fd9c816de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:15.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.001+0000 7fd9cce84700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9c80b1b40 0x7fd9c80688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:15.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.001+0000 7fd9cde86700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9c80b4e70 0x7fd9c816de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44832/0 (socket says 172.21.15.163:44832) 2023-10-09T06:39:15.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.001+0000 7fd9cde86700 1 -- 172.21.15.163:0/3824212966 learned_addr learned my addr 172.21.15.163:0/3824212966 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:15.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.001+0000 7fd9cd685700 1 --2- 172.21.15.163:0/3824212966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9c80b16d0 0x7fd9c8068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:15.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.001+0000 7fd9cd685700 1 -- 172.21.15.163:0/3824212966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9c80b1b40 msgr2=0x7fd9c80688d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:15.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.001+0000 7fd9cd685700 1 --2- 172.21.15.163:0/3824212966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9c80b1b40 0x7fd9c80688d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:15.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.001+0000 7fd9cd685700 1 -- 172.21.15.163:0/3824212966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9c80b4e70 msgr2=0x7fd9c816de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:15.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.001+0000 7fd9cd685700 1 --2- 172.21.15.163:0/3824212966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9c80b4e70 0x7fd9c816de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:15.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.001+0000 7fd9cd685700 1 -- 172.21.15.163:0/3824212966 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c4009a20 con 0x7fd9c80b16d0 2023-10-09T06:39:15.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.002+0000 7fd9cd685700 1 --2- 172.21.15.163:0/3824212966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9c80b16d0 0x7fd9c8068390 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009fa0 tx=0x7fd9c400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:15.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.002+0000 7fd9bd7fa700 1 -- 172.21.15.163:0/3824212966 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c400cba0 con 0x7fd9c80b16d0 2023-10-09T06:39:15.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.002+0000 7fd9bd7fa700 1 -- 172.21.15.163:0/3824212966 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c400f040 con 0x7fd9c80b16d0 2023-10-09T06:39:15.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.002+0000 7fd9bd7fa700 1 -- 172.21.15.163:0/3824212966 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4014480 con 0x7fd9c80b16d0 2023-10-09T06:39:15.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.002+0000 7fd9ce687700 1 -- 172.21.15.163:0/3824212966 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c00094f0 con 0x7fd9c80b16d0 2023-10-09T06:39:15.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.002+0000 7fd9ce687700 1 -- 172.21.15.163:0/3824212966 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c816f430 con 0x7fd9c80b16d0 2023-10-09T06:39:15.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.003+0000 7fd9ce687700 1 -- 172.21.15.163:0/3824212966 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c80ccbe0 con 0x7fd9c80b16d0 2023-10-09T06:39:15.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.004+0000 7fd9bd7fa700 1 -- 172.21.15.163:0/3824212966 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd9c4003020 con 0x7fd9c80b16d0 2023-10-09T06:39:15.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.004+0000 7fd9bd7fa700 1 --2- 172.21.15.163:0/3824212966 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd9ac067320 0x7fd9ac0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:15.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.004+0000 7fd9bd7fa700 1 -- 172.21.15.163:0/3824212966 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fd9c4085420 con 0x7fd9c80b16d0 2023-10-09T06:39:15.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.004+0000 7fd9cce84700 1 --2- 172.21.15.163:0/3824212966 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd9ac067320 0x7fd9ac0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:15.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.005+0000 7fd9cce84700 1 --2- 172.21.15.163:0/3824212966 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd9ac067320 0x7fd9ac0697d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd9c80b4e10 tx=0x7fd9b8007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:15.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.009+0000 7fd9bd7fa700 1 -- 172.21.15.163:0/3824212966 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9c4052320 con 0x7fd9c80b16d0 2023-10-09T06:39:15.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.192+0000 7fd9ce687700 1 -- 172.21.15.163:0/3824212966 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd9c804bb80 con 0x7fd9c80b16d0 2023-10-09T06:39:15.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.197+0000 7fd9bd7fa700 1 -- 172.21.15.163:0/3824212966 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd9c4009880 con 0x7fd9c80b16d0 2023-10-09T06:39:15.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.200+0000 7fd9ce687700 1 -- 172.21.15.163:0/3824212966 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd9ac067320 msgr2=0x7fd9ac0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:15.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.200+0000 7fd9ce687700 1 --2- 172.21.15.163:0/3824212966 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd9ac067320 0x7fd9ac0697d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd9c80b4e10 tx=0x7fd9b8007d50 comp rx=0 tx=0).stop 2023-10-09T06:39:15.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.200+0000 7fd9ce687700 1 -- 172.21.15.163:0/3824212966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9c80b16d0 msgr2=0x7fd9c8068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:15.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.200+0000 7fd9ce687700 1 --2- 172.21.15.163:0/3824212966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9c80b16d0 0x7fd9c8068390 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009fa0 tx=0x7fd9c400b040 comp rx=0 tx=0).stop 2023-10-09T06:39:15.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.200+0000 7fd9ce687700 1 -- 172.21.15.163:0/3824212966 shutdown_connections 2023-10-09T06:39:15.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.200+0000 7fd9ce687700 1 --2- 172.21.15.163:0/3824212966 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd9ac067320 0x7fd9ac0697d0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:15.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.200+0000 7fd9ce687700 1 --2- 172.21.15.163:0/3824212966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9c80b4e70 0x7fd9c816de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:15.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.200+0000 7fd9ce687700 1 --2- 172.21.15.163:0/3824212966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9c80b16d0 0x7fd9c8068390 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:15.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.200+0000 7fd9ce687700 1 --2- 172.21.15.163:0/3824212966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9c80b1b40 0x7fd9c80688d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:15.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.200+0000 7fd9ce687700 1 -- 172.21.15.163:0/3824212966 >> 172.21.15.163:0/3824212966 conn(0x7fd9c80d10c0 msgr2=0x7fd9c80b54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:15.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.200+0000 7fd9ce687700 1 -- 172.21.15.163:0/3824212966 shutdown_connections 2023-10-09T06:39:15.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.200+0000 7fd9ce687700 1 -- 172.21.15.163:0/3824212966 wait complete. 2023-10-09T06:39:15.212 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool' 2023-10-09T06:39:15.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.327+0000 7f5a0d26c700 1 Processor -- start 2023-10-09T06:39:15.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.327+0000 7f5a0d26c700 1 -- start start 2023-10-09T06:39:15.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.328+0000 7f5a0d26c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5a08159c60 0x7f5a0815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:15.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.328+0000 7f5a0d26c700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a0815a160 con 0x7f5a08159890 2023-10-09T06:39:15.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.328+0000 7f5a0d26c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a080d13a0 con 0x7f5a0815db10 2023-10-09T06:39:15.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.328+0000 7f5a0d26c700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a0815df50 con 0x7f5a08159c60 2023-10-09T06:39:15.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.328+0000 7f59fe7fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5a08159c60 0x7f5a0815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:15.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.328+0000 7f59fe7fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5a08159c60 0x7f5a0815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44842/0 (socket says 172.21.15.163:44842) 2023-10-09T06:39:15.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.328+0000 7f5a06ffd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f5a08159890 0x7f5a0815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:51958/0 (socket says 172.21.15.163:51958) 2023-10-09T06:39:15.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.328+0000 7f59fe7fc700 1 -- 172.21.15.163:0/1162282256 learned_addr learned my addr 172.21.15.163:0/1162282256 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:15.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.328+0000 7f59fe7fc700 1 -- 172.21.15.163:0/1162282256 >> v1:172.21.15.181:6789/0 conn(0x7f5a0815db10 legacy=0x7f5a08069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:15.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.328+0000 7f59fe7fc700 1 -- 172.21.15.163:0/1162282256 >> v1:172.21.15.163:6789/0 conn(0x7f5a08159890 legacy=0x7f5a0815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:15.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.328+0000 7f59fe7fc700 1 -- 172.21.15.163:0/1162282256 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0815e140 con 0x7f5a08159c60 2023-10-09T06:39:15.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.328+0000 7f5a067fc700 1 -- 172.21.15.163:0/1162282256 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2424410878 0 0) 0x7f5a0815a160 con 0x7f5a08159890 2023-10-09T06:39:15.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.328+0000 7f59fe7fc700 1 --2- 172.21.15.163:0/1162282256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a08159c60 0x7f5a0815d5d0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f59f00099b0 tx=0x7f59f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea043e4bd11de34 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:15.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.328+0000 7f5a067fc700 1 -- 172.21.15.163:0/1162282256 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f000b820 con 0x7f5a08159c60 2023-10-09T06:39:15.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.328+0000 7f5a067fc700 1 -- 172.21.15.163:0/1162282256 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f000b980 con 0x7f5a08159c60 2023-10-09T06:39:15.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.328+0000 7f5a067fc700 1 -- 172.21.15.163:0/1162282256 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f000bd20 con 0x7f5a08159c60 2023-10-09T06:39:15.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.329+0000 7f5a0d26c700 1 -- 172.21.15.163:0/1162282256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a08159c60 msgr2=0x7f5a0815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:15.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.329+0000 7f5a0d26c700 1 --2- 172.21.15.163:0/1162282256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a08159c60 0x7f5a0815d5d0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f59f00099b0 tx=0x7f59f0009250 comp rx=0 tx=0).stop 2023-10-09T06:39:15.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.329+0000 7f5a0d26c700 1 -- 172.21.15.163:0/1162282256 shutdown_connections 2023-10-09T06:39:15.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.329+0000 7f5a0d26c700 1 --2- 172.21.15.163:0/1162282256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a08159c60 0x7f5a0815d5d0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:15.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.329+0000 7f5a0d26c700 1 -- 172.21.15.163:0/1162282256 >> 172.21.15.163:0/1162282256 conn(0x7f5a08068e20 msgr2=0x7f5a080d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:15.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.329+0000 7f5a0d26c700 1 -- 172.21.15.163:0/1162282256 shutdown_connections 2023-10-09T06:39:15.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.329+0000 7f5a0d26c700 1 -- 172.21.15.163:0/1162282256 wait complete. 2023-10-09T06:39:15.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.330+0000 7f5a0d26c700 1 Processor -- start 2023-10-09T06:39:15.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.330+0000 7f5a0d26c700 1 -- start start 2023-10-09T06:39:15.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.331+0000 7f5a0d26c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a08159890 0x7f5a081501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:15.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.331+0000 7f5a0d26c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a08159c60 0x7f5a080cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:15.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.331+0000 7f5a0d26c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a0815db10 0x7f5a080c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:15.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.331+0000 7f5a0d26c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0816a6a0 con 0x7f5a08159c60 2023-10-09T06:39:15.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.331+0000 7f5a0d26c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a08159060 con 0x7f5a08159890 2023-10-09T06:39:15.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.331+0000 7f5a0d26c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a080cdba0 con 0x7f5a0815db10 2023-10-09T06:39:15.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.331+0000 7f59fe7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a08159c60 0x7f5a080cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:15.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.331+0000 7f59fe7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a08159c60 0x7f5a080cd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44856/0 (socket says 172.21.15.163:44856) 2023-10-09T06:39:15.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.331+0000 7f59fe7fc700 1 -- 172.21.15.163:0/1559077848 learned_addr learned my addr 172.21.15.163:0/1559077848 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:15.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.331+0000 7f59fe7fc700 1 -- 172.21.15.163:0/1559077848 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a0815db10 msgr2=0x7f5a080c67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:15.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.331+0000 7f59fe7fc700 1 --2- 172.21.15.163:0/1559077848 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a0815db10 0x7f5a080c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:15.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.331+0000 7f59fe7fc700 1 -- 172.21.15.163:0/1559077848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a08159890 msgr2=0x7f5a081501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:15.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.331+0000 7f59fe7fc700 1 --2- 172.21.15.163:0/1559077848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a08159890 0x7f5a081501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:15.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.331+0000 7f59fe7fc700 1 -- 172.21.15.163:0/1559077848 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f00094f0 con 0x7f5a08159c60 2023-10-09T06:39:15.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.331+0000 7f59fe7fc700 1 --2- 172.21.15.163:0/1559077848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a08159c60 0x7f5a080cd100 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f59f0009290 tx=0x7f59f0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:15.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.331+0000 7f59ff7fe700 1 -- 172.21.15.163:0/1559077848 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f000bc40 con 0x7f5a08159c60 2023-10-09T06:39:15.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.331+0000 7f59ff7fe700 1 -- 172.21.15.163:0/1559077848 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f0020070 con 0x7f5a08159c60 2023-10-09T06:39:15.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.331+0000 7f5a0d26c700 1 -- 172.21.15.163:0/1559077848 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a08169d70 con 0x7f5a08159c60 2023-10-09T06:39:15.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.331+0000 7f59ff7fe700 1 -- 172.21.15.163:0/1559077848 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f001b6a0 con 0x7f5a08159c60 2023-10-09T06:39:15.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.332+0000 7f5a0d26c700 1 -- 172.21.15.163:0/1559077848 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a0816a330 con 0x7f5a08159c60 2023-10-09T06:39:15.334 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.333+0000 7f5a0d26c700 1 -- 172.21.15.163:0/1559077848 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a0804c3c0 con 0x7f5a08159c60 2023-10-09T06:39:15.335 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.333+0000 7f59ff7fe700 1 -- 172.21.15.163:0/1559077848 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f59f0022030 con 0x7f5a08159c60 2023-10-09T06:39:15.335 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.333+0000 7f59ff7fe700 1 --2- 172.21.15.163:0/1559077848 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f59e8067320 0x7f59e80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:15.335 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.334+0000 7f59ff7fe700 1 -- 172.21.15.163:0/1559077848 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f59f0086830 con 0x7f5a08159c60 2023-10-09T06:39:15.335 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.334+0000 7f5a06ffd700 1 --2- 172.21.15.163:0/1559077848 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f59e8067320 0x7f59e80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:15.335 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.334+0000 7f5a06ffd700 1 --2- 172.21.15.163:0/1559077848 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f59e8067320 0x7f59e80697d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f5a08157090 tx=0x7f59f8007b70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:15.337 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.338+0000 7f59ff7fe700 1 -- 172.21.15.163:0/1559077848 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59f00538d0 con 0x7f5a08159c60 2023-10-09T06:39:15.594 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.594+0000 7f5a0d26c700 1 -- 172.21.15.163:0/1559077848 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool"]} v 0) v1 -- 0x7f5a0804c3c0 con 0x7f5a08159c60 2023-10-09T06:39:15.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.859+0000 7f59ff7fe700 1 -- 172.21.15.163:0/1559077848 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool"]}]=0 v442) v1 ==== 163+0+0 (secure 0 0 0) 0x7f59f0056ef0 con 0x7f5a08159c60 2023-10-09T06:39:15.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.861+0000 7f5a0d26c700 1 -- 172.21.15.163:0/1559077848 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f59e8067320 msgr2=0x7f59e80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:15.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.861+0000 7f5a0d26c700 1 --2- 172.21.15.163:0/1559077848 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f59e8067320 0x7f59e80697d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f5a08157090 tx=0x7f59f8007b70 comp rx=0 tx=0).stop 2023-10-09T06:39:15.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.861+0000 7f5a0d26c700 1 -- 172.21.15.163:0/1559077848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a08159c60 msgr2=0x7f5a080cd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:15.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.861+0000 7f5a0d26c700 1 --2- 172.21.15.163:0/1559077848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a08159c60 0x7f5a080cd100 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f59f0009290 tx=0x7f59f0016000 comp rx=0 tx=0).stop 2023-10-09T06:39:15.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.861+0000 7f5a0d26c700 1 -- 172.21.15.163:0/1559077848 shutdown_connections 2023-10-09T06:39:15.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.861+0000 7f5a0d26c700 1 --2- 172.21.15.163:0/1559077848 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f59e8067320 0x7f59e80697d0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:15.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.861+0000 7f5a0d26c700 1 --2- 172.21.15.163:0/1559077848 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a0815db10 0x7f5a080c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:15.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.861+0000 7f5a0d26c700 1 --2- 172.21.15.163:0/1559077848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a08159890 0x7f5a081501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:15.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.861+0000 7f5a0d26c700 1 --2- 172.21.15.163:0/1559077848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a08159c60 0x7f5a080cd100 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:15.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.861+0000 7f5a0d26c700 1 -- 172.21.15.163:0/1559077848 >> 172.21.15.163:0/1559077848 conn(0x7f5a08068e20 msgr2=0x7f5a081570c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:15.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.861+0000 7f5a0d26c700 1 -- 172.21.15.163:0/1559077848 shutdown_connections 2023-10-09T06:39:15.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.861+0000 7f5a0d26c700 1 -- 172.21.15.163:0/1559077848 wait complete. 2023-10-09T06:39:15.874 INFO:tasks.cephfs_test_runner:That renaming a volume with more than one data pool does not change ... ok 2023-10-09T06:39:15.874 INFO:tasks.cephfs_test_runner:Starting test: test_volume_rename_idempotency (tasks.cephfs.test_volumes.TestVolumes) That volume rename is idempotent. 2023-10-09T06:39:15.874 INFO:tasks.cephfs_test_runner:test_volume_rename_idempotency (tasks.cephfs.test_volumes.TestVolumes) 2023-10-09T06:39:15.874 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_idempotency' 2023-10-09T06:39:15.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.991+0000 7f05e8ea1700 1 Processor -- start 2023-10-09T06:39:15.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.991+0000 7f05e8ea1700 1 -- start start 2023-10-09T06:39:15.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.992+0000 7f05e8ea1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f05e4159420 0x7f05e40b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:15.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.992+0000 7f05e8ea1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f05e41597f0 0x7f05e40b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:15.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.992+0000 7f05e8ea1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f05e415d230 0x7f05e415d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:15.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.992+0000 7f05e8ea1700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e415dbb0 con 0x7f05e41597f0 2023-10-09T06:39:15.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.992+0000 7f05e8ea1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e40b11d0 con 0x7f05e4159420 2023-10-09T06:39:15.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.992+0000 7f05e8ea1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e40b14b0 con 0x7f05e415d230 2023-10-09T06:39:15.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.992+0000 7f05e2ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f05e41597f0 0x7f05e40b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:15.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.992+0000 7f05e2ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f05e41597f0 0x7f05e40b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44870/0 (socket says 172.21.15.163:44870) 2023-10-09T06:39:15.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.992+0000 7f05e2ffd700 1 -- 172.21.15.163:0/668411778 learned_addr learned my addr 172.21.15.163:0/668411778 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:15.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.992+0000 7f05e37fe700 1 --2- 172.21.15.163:0/668411778 >> v2:172.21.15.181:3300/0 conn(0x7f05e4159420 0x7f05e40b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.992+0000 7f05e2ffd700 1 -- 172.21.15.163:0/668411778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f05e415d230 msgr2=0x7f05e415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.992+0000 7f05e3fff700 1 --2- 172.21.15.163:0/668411778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f05e415d230 0x7f05e415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.993+0000 7f05e2ffd700 1 --2- 172.21.15.163:0/668411778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f05e415d230 0x7f05e415d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.993+0000 7f05e2ffd700 1 -- 172.21.15.163:0/668411778 >> v2:172.21.15.181:3300/0 conn(0x7f05e4159420 msgr2=0x7f05e40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.993+0000 7f05e2ffd700 1 --2- 172.21.15.163:0/668411778 >> v2:172.21.15.181:3300/0 conn(0x7f05e4159420 0x7f05e40b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.993+0000 7f05e2ffd700 1 -- 172.21.15.163:0/668411778 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e40b1790 con 0x7f05e41597f0 2023-10-09T06:39:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.993+0000 7f05e2ffd700 1 --2- 172.21.15.163:0/668411778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05e41597f0 0x7f05e40b2270 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f05e40b2f20 tx=0x7f05d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ac1b80add5159c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.993+0000 7f05e27fc700 1 -- 172.21.15.163:0/668411778 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d800b860 con 0x7f05e41597f0 2023-10-09T06:39:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.993+0000 7f05e27fc700 1 -- 172.21.15.163:0/668411778 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d800b9c0 con 0x7f05e41597f0 2023-10-09T06:39:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.993+0000 7f05e27fc700 1 -- 172.21.15.163:0/668411778 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d800bd60 con 0x7f05e41597f0 2023-10-09T06:39:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.993+0000 7f05e8ea1700 1 -- 172.21.15.163:0/668411778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05e41597f0 msgr2=0x7f05e40b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:15.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.993+0000 7f05e8ea1700 1 --2- 172.21.15.163:0/668411778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05e41597f0 0x7f05e40b2270 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f05e40b2f20 tx=0x7f05d8009250 comp rx=0 tx=0).stop 2023-10-09T06:39:15.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.993+0000 7f05e8ea1700 1 -- 172.21.15.163:0/668411778 shutdown_connections 2023-10-09T06:39:15.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.993+0000 7f05e8ea1700 1 --2- 172.21.15.163:0/668411778 >> v2:172.21.15.181:3300/0 conn(0x7f05e4159420 0x7f05e40b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:15.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.993+0000 7f05e8ea1700 1 --2- 172.21.15.163:0/668411778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f05e415d230 0x7f05e415d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:15.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.993+0000 7f05e8ea1700 1 --2- 172.21.15.163:0/668411778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05e41597f0 0x7f05e40b2270 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:15.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.993+0000 7f05e8ea1700 1 -- 172.21.15.163:0/668411778 >> 172.21.15.163:0/668411778 conn(0x7f05e40d0660 msgr2=0x7f05e40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:15.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.993+0000 7f05e8ea1700 1 -- 172.21.15.163:0/668411778 shutdown_connections 2023-10-09T06:39:15.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.993+0000 7f05e8ea1700 1 -- 172.21.15.163:0/668411778 wait complete. 2023-10-09T06:39:15.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.995+0000 7f05e8ea1700 1 Processor -- start 2023-10-09T06:39:15.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.995+0000 7f05e8ea1700 1 -- start start 2023-10-09T06:39:15.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.995+0000 7f05e8ea1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f05e4159420 0x7f05e414cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:16.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.995+0000 7f05e8ea1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05e41597f0 0x7f05e4150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:16.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.995+0000 7f05e8ea1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05e415d230 0x7f05e414f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:16.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.995+0000 7f05e8ea1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e414ca10 con 0x7f05e415d230 2023-10-09T06:39:16.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.995+0000 7f05e8ea1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e4169940 con 0x7f05e41597f0 2023-10-09T06:39:16.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.995+0000 7f05e8ea1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e416d320 con 0x7f05e4159420 2023-10-09T06:39:16.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.996+0000 7f05e3fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05e415d230 0x7f05e414f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:16.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.996+0000 7f05e3fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05e415d230 0x7f05e414f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44874/0 (socket says 172.21.15.163:44874) 2023-10-09T06:39:16.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.996+0000 7f05e3fff700 1 -- 172.21.15.163:0/2927953044 learned_addr learned my addr 172.21.15.163:0/2927953044 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:16.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.996+0000 7f05e37fe700 1 --2- 172.21.15.163:0/2927953044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f05e4159420 0x7f05e414cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:16.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.996+0000 7f05e2ffd700 1 --2- 172.21.15.163:0/2927953044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05e41597f0 0x7f05e4150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:16.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.996+0000 7f05e3fff700 1 -- 172.21.15.163:0/2927953044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f05e4159420 msgr2=0x7f05e414cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:16.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.996+0000 7f05e3fff700 1 --2- 172.21.15.163:0/2927953044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f05e4159420 0x7f05e414cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:16.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.996+0000 7f05e3fff700 1 -- 172.21.15.163:0/2927953044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05e41597f0 msgr2=0x7f05e4150840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:16.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.996+0000 7f05e3fff700 1 --2- 172.21.15.163:0/2927953044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05e41597f0 0x7f05e4150840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:16.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.996+0000 7f05e3fff700 1 -- 172.21.15.163:0/2927953044 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d80094f0 con 0x7f05e415d230 2023-10-09T06:39:16.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.996+0000 7f05e3fff700 1 --2- 172.21.15.163:0/2927953044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05e415d230 0x7f05e414f830 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00cdb0 tx=0x7f05dc0077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:16.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.996+0000 7f05cb7fe700 1 -- 172.21.15.163:0/2927953044 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc00a9d0 con 0x7f05e415d230 2023-10-09T06:39:16.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.996+0000 7f05cb7fe700 1 -- 172.21.15.163:0/2927953044 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05dc00ab30 con 0x7f05e415d230 2023-10-09T06:39:16.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.996+0000 7f05e8ea1700 1 -- 172.21.15.163:0/2927953044 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e416d600 con 0x7f05e415d230 2023-10-09T06:39:16.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.996+0000 7f05cb7fe700 1 -- 172.21.15.163:0/2927953044 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc017430 con 0x7f05e415d230 2023-10-09T06:39:16.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.996+0000 7f05e8ea1700 1 -- 172.21.15.163:0/2927953044 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e416dbc0 con 0x7f05e415d230 2023-10-09T06:39:16.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.997+0000 7f05e8ea1700 1 -- 172.21.15.163:0/2927953044 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05e404c3c0 con 0x7f05e415d230 2023-10-09T06:39:16.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.998+0000 7f05cb7fe700 1 -- 172.21.15.163:0/2927953044 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f05dc003020 con 0x7f05e415d230 2023-10-09T06:39:16.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.998+0000 7f05cb7fe700 1 --2- 172.21.15.163:0/2927953044 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f05d0067320 0x7f05d00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:16.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.998+0000 7f05cb7fe700 1 -- 172.21.15.163:0/2927953044 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f05dc013070 con 0x7f05e415d230 2023-10-09T06:39:16.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.998+0000 7f05e37fe700 1 --2- 172.21.15.163:0/2927953044 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f05d0067320 0x7f05d00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:16.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:15.999+0000 7f05e37fe700 1 --2- 172.21.15.163:0/2927953044 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f05d0067320 0x7f05d00697d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f05d400acb0 tx=0x7f05d4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:16.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.002+0000 7f05cb7fe700 1 -- 172.21.15.163:0/2927953044 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05dc051d20 con 0x7f05e415d230 2023-10-09T06:39:16.260 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.260+0000 7f05e8ea1700 1 -- 172.21.15.163:0/2927953044 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_idempotency"]} v 0) v1 -- 0x7f05e40b1a70 con 0x7f05e415d230 2023-10-09T06:39:16.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.865+0000 7f05cb7fe700 1 -- 172.21.15.163:0/2927953044 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_idempotency"]}]=0 v443) v1 ==== 150+0+0 (secure 0 0 0) 0x7f05dc0078f0 con 0x7f05e415d230 2023-10-09T06:39:16.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.867+0000 7f05e8ea1700 1 -- 172.21.15.163:0/2927953044 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f05d0067320 msgr2=0x7f05d00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:16.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.867+0000 7f05e8ea1700 1 --2- 172.21.15.163:0/2927953044 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f05d0067320 0x7f05d00697d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f05d400acb0 tx=0x7f05d4008040 comp rx=0 tx=0).stop 2023-10-09T06:39:16.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.867+0000 7f05e8ea1700 1 -- 172.21.15.163:0/2927953044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05e415d230 msgr2=0x7f05e414f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:16.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.867+0000 7f05e8ea1700 1 --2- 172.21.15.163:0/2927953044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05e415d230 0x7f05e414f830 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00cdb0 tx=0x7f05dc0077d0 comp rx=0 tx=0).stop 2023-10-09T06:39:16.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.867+0000 7f05e8ea1700 1 -- 172.21.15.163:0/2927953044 shutdown_connections 2023-10-09T06:39:16.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.867+0000 7f05e8ea1700 1 --2- 172.21.15.163:0/2927953044 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f05d0067320 0x7f05d00697d0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:16.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.867+0000 7f05e8ea1700 1 --2- 172.21.15.163:0/2927953044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05e415d230 0x7f05e414f830 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:16.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.867+0000 7f05e8ea1700 1 --2- 172.21.15.163:0/2927953044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f05e4159420 0x7f05e414cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:16.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.867+0000 7f05e8ea1700 1 --2- 172.21.15.163:0/2927953044 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05e41597f0 0x7f05e4150840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:16.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.867+0000 7f05e8ea1700 1 -- 172.21.15.163:0/2927953044 >> 172.21.15.163:0/2927953044 conn(0x7f05e40d0660 msgr2=0x7f05e40cf130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:16.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.867+0000 7f05e8ea1700 1 -- 172.21.15.163:0/2927953044 shutdown_connections 2023-10-09T06:39:16.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.867+0000 7f05e8ea1700 1 -- 172.21.15.163:0/2927953044 wait complete. 2023-10-09T06:39:16.880 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-09T06:39:16.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.997+0000 7fe989cd4700 1 Processor -- start 2023-10-09T06:39:16.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.997+0000 7fe989cd4700 1 -- start start 2023-10-09T06:39:16.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.998+0000 7fe989cd4700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe984159c60 0x7fe98415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:16.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.998+0000 7fe989cd4700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe98415a160 con 0x7fe984159890 2023-10-09T06:39:16.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.998+0000 7fe989cd4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9840d13a0 con 0x7fe98415db10 2023-10-09T06:39:17.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.998+0000 7fe989cd4700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe98415df50 con 0x7fe984159c60 2023-10-09T06:39:17.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.998+0000 7fe982ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe984159c60 0x7fe98415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:17.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.998+0000 7fe9837fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fe984159890 0x7fe98415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:51964/0 (socket says 172.21.15.163:51964) 2023-10-09T06:39:17.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.998+0000 7fe9837fe700 1 -- 172.21.15.163:0/289184306 learned_addr learned my addr 172.21.15.163:0/289184306 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:17.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.999+0000 7fe982ffd700 1 -- 172.21.15.163:0/289184306 >> v1:172.21.15.181:6789/0 conn(0x7fe98415db10 legacy=0x7fe984069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:17.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.999+0000 7fe9827fc700 1 -- 172.21.15.163:0/289184306 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2325587699 0 0) 0x7fe98415a160 con 0x7fe984159890 2023-10-09T06:39:17.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.999+0000 7fe982ffd700 1 -- 172.21.15.163:0/289184306 >> v1:172.21.15.163:6789/0 conn(0x7fe984159890 legacy=0x7fe98415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:17.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.999+0000 7fe982ffd700 1 -- 172.21.15.163:0/289184306 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98415e140 con 0x7fe984159c60 2023-10-09T06:39:17.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.999+0000 7fe982ffd700 1 --2- 172.21.15.163:0/289184306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe984159c60 0x7fe98415d5d0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fe9780098e0 tx=0x7fe9780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f10d43bd779d7ed server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:17.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.999+0000 7fe9827fc700 1 -- 172.21.15.163:0/289184306 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978009e80 con 0x7fe984159c60 2023-10-09T06:39:17.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.999+0000 7fe9827fc700 1 -- 172.21.15.163:0/289184306 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe97800b820 con 0x7fe984159c60 2023-10-09T06:39:17.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.999+0000 7fe9827fc700 1 -- 172.21.15.163:0/289184306 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97800bba0 con 0x7fe984159c60 2023-10-09T06:39:17.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.999+0000 7fe989cd4700 1 -- 172.21.15.163:0/289184306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe984159c60 msgr2=0x7fe98415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:17.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.999+0000 7fe989cd4700 1 --2- 172.21.15.163:0/289184306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe984159c60 0x7fe98415d5d0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fe9780098e0 tx=0x7fe9780091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:17.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.999+0000 7fe989cd4700 1 -- 172.21.15.163:0/289184306 shutdown_connections 2023-10-09T06:39:17.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.999+0000 7fe989cd4700 1 --2- 172.21.15.163:0/289184306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe984159c60 0x7fe98415d5d0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:17.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.999+0000 7fe989cd4700 1 -- 172.21.15.163:0/289184306 >> 172.21.15.163:0/289184306 conn(0x7fe984068e20 msgr2=0x7fe9840d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:17.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.999+0000 7fe989cd4700 1 -- 172.21.15.163:0/289184306 shutdown_connections 2023-10-09T06:39:17.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:16.999+0000 7fe989cd4700 1 -- 172.21.15.163:0/289184306 wait complete. 2023-10-09T06:39:17.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.001+0000 7fe989cd4700 1 Processor -- start 2023-10-09T06:39:17.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.001+0000 7fe989cd4700 1 -- start start 2023-10-09T06:39:17.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.001+0000 7fe989cd4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe984159890 0x7fe9841504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:17.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.001+0000 7fe989cd4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe984159c60 0x7fe9840cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:17.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.001+0000 7fe989cd4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe98415db10 0x7fe9841584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:17.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.001+0000 7fe989cd4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9840ce610 con 0x7fe98415db10 2023-10-09T06:39:17.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.001+0000 7fe989cd4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9840cda00 con 0x7fe984159c60 2023-10-09T06:39:17.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.001+0000 7fe989cd4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9840cdce0 con 0x7fe984159890 2023-10-09T06:39:17.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.001+0000 7fe983fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe98415db10 0x7fe9841584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:17.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.002+0000 7fe983fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe98415db10 0x7fe9841584b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44890/0 (socket says 172.21.15.163:44890) 2023-10-09T06:39:17.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.002+0000 7fe983fff700 1 -- 172.21.15.163:0/2217057590 learned_addr learned my addr 172.21.15.163:0/2217057590 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:17.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.002+0000 7fe982ffd700 1 --2- 172.21.15.163:0/2217057590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe984159c60 0x7fe9840cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:17.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.002+0000 7fe9837fe700 1 --2- 172.21.15.163:0/2217057590 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe984159890 0x7fe9841504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:17.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.002+0000 7fe983fff700 1 -- 172.21.15.163:0/2217057590 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe984159890 msgr2=0x7fe9841504c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:17.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.002+0000 7fe983fff700 1 --2- 172.21.15.163:0/2217057590 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe984159890 0x7fe9841504c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:17.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.002+0000 7fe983fff700 1 -- 172.21.15.163:0/2217057590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe984159c60 msgr2=0x7fe9840cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:17.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.002+0000 7fe983fff700 1 --2- 172.21.15.163:0/2217057590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe984159c60 0x7fe9840cd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:17.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.002+0000 7fe983fff700 1 -- 172.21.15.163:0/2217057590 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe978009420 con 0x7fe98415db10 2023-10-09T06:39:17.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.002+0000 7fe983fff700 1 --2- 172.21.15.163:0/2217057590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe98415db10 0x7fe9841584b0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fe97c00c5b0 tx=0x7fe97c007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:17.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.002+0000 7fe96b7fe700 1 -- 172.21.15.163:0/2217057590 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97c011590 con 0x7fe98415db10 2023-10-09T06:39:17.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.002+0000 7fe96b7fe700 1 -- 172.21.15.163:0/2217057590 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe97c0116f0 con 0x7fe98415db10 2023-10-09T06:39:17.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.002+0000 7fe96b7fe700 1 -- 172.21.15.163:0/2217057590 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97c011ab0 con 0x7fe98415db10 2023-10-09T06:39:17.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.002+0000 7fe989cd4700 1 -- 172.21.15.163:0/2217057590 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9840cdfc0 con 0x7fe98415db10 2023-10-09T06:39:17.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.002+0000 7fe989cd4700 1 -- 172.21.15.163:0/2217057590 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe984150a00 con 0x7fe98415db10 2023-10-09T06:39:17.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.003+0000 7fe989cd4700 1 -- 172.21.15.163:0/2217057590 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe98404c3c0 con 0x7fe98415db10 2023-10-09T06:39:17.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.003+0000 7fe96b7fe700 1 -- 172.21.15.163:0/2217057590 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe97c01c460 con 0x7fe98415db10 2023-10-09T06:39:17.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.004+0000 7fe96b7fe700 1 --2- 172.21.15.163:0/2217057590 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe970067320 0x7fe9700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:17.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.004+0000 7fe96b7fe700 1 -- 172.21.15.163:0/2217057590 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fe97c0845a0 con 0x7fe98415db10 2023-10-09T06:39:17.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.004+0000 7fe9837fe700 1 --2- 172.21.15.163:0/2217057590 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe970067320 0x7fe9700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:17.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.004+0000 7fe9837fe700 1 --2- 172.21.15.163:0/2217057590 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe970067320 0x7fe9700697d0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe974002180 tx=0x7fe974009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:17.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.008+0000 7fe96b7fe700 1 -- 172.21.15.163:0/2217057590 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe97c051530 con 0x7fe98415db10 2023-10-09T06:39:17.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.191+0000 7fe989cd4700 1 -- 172.21.15.163:0/2217057590 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe98406a5b0 con 0x7fe98415db10 2023-10-09T06:39:17.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.195+0000 7fe96b7fe700 1 -- 172.21.15.163:0/2217057590 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe97c054b50 con 0x7fe98415db10 2023-10-09T06:39:17.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.197+0000 7fe989cd4700 1 -- 172.21.15.163:0/2217057590 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe970067320 msgr2=0x7fe9700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:17.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.197+0000 7fe989cd4700 1 --2- 172.21.15.163:0/2217057590 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe970067320 0x7fe9700697d0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe974002180 tx=0x7fe974009040 comp rx=0 tx=0).stop 2023-10-09T06:39:17.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.197+0000 7fe989cd4700 1 -- 172.21.15.163:0/2217057590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe98415db10 msgr2=0x7fe9841584b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:17.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.197+0000 7fe989cd4700 1 --2- 172.21.15.163:0/2217057590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe98415db10 0x7fe9841584b0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fe97c00c5b0 tx=0x7fe97c007f60 comp rx=0 tx=0).stop 2023-10-09T06:39:17.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.197+0000 7fe989cd4700 1 -- 172.21.15.163:0/2217057590 shutdown_connections 2023-10-09T06:39:17.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.197+0000 7fe989cd4700 1 --2- 172.21.15.163:0/2217057590 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe970067320 0x7fe9700697d0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:17.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.197+0000 7fe989cd4700 1 --2- 172.21.15.163:0/2217057590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe98415db10 0x7fe9841584b0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:17.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.197+0000 7fe989cd4700 1 --2- 172.21.15.163:0/2217057590 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe984159890 0x7fe9841504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:17.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.197+0000 7fe989cd4700 1 --2- 172.21.15.163:0/2217057590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe984159c60 0x7fe9840cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:17.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.197+0000 7fe989cd4700 1 -- 172.21.15.163:0/2217057590 >> 172.21.15.163:0/2217057590 conn(0x7fe984068e20 msgr2=0x7fe984153d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:17.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.197+0000 7fe989cd4700 1 -- 172.21.15.163:0/2217057590 shutdown_connections 2023-10-09T06:39:17.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.197+0000 7fe989cd4700 1 -- 172.21.15.163:0/2217057590 wait complete. 2023-10-09T06:39:17.209 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-09T06:39:17.210 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:39:17.246 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:39:17.289 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-09T06:39:17.290 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-09T06:39:17.290 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:39:17.404 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.404+0000 7f01060d6700 1 Processor -- start 2023-10-09T06:39:17.405 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.404+0000 7f01060d6700 1 -- start start 2023-10-09T06:39:17.407 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.405+0000 7f01060d6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f010006bcd0 0x7f0100140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:17.407 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.405+0000 7f01060d6700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f010006c0a0 0x7f0100141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:17.407 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.405+0000 7f01060d6700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f010015c2a0 con 0x7f01000c8e00 2023-10-09T06:39:17.407 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.405+0000 7f01060d6700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01000c9900 con 0x7f010006c0a0 2023-10-09T06:39:17.407 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.405+0000 7f01060d6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f010015b690 con 0x7f010006bcd0 2023-10-09T06:39:17.407 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.405+0000 7f00feffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f010006c0a0 0x7f0100141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:17.407 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.405+0000 7f00feffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f010006c0a0 0x7f0100141330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44906/0 (socket says 172.21.15.163:44906) 2023-10-09T06:39:17.407 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.405+0000 7f00fffff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f01000c8e00 0x7f01000c91f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:51970/0 (socket says 172.21.15.163:51970) 2023-10-09T06:39:17.407 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.405+0000 7f00feffd700 1 -- 172.21.15.163:0/892090108 learned_addr learned my addr 172.21.15.163:0/892090108 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:17.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.406+0000 7f00ff7fe700 1 --2- 172.21.15.163:0/892090108 >> v2:172.21.15.181:3300/0 conn(0x7f010006bcd0 0x7f0100140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:17.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.406+0000 7f00feffd700 1 -- 172.21.15.163:0/892090108 >> v2:172.21.15.181:3300/0 conn(0x7f010006bcd0 msgr2=0x7f0100140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:17.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.406+0000 7f00feffd700 1 --2- 172.21.15.163:0/892090108 >> v2:172.21.15.181:3300/0 conn(0x7f010006bcd0 0x7f0100140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:17.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.406+0000 7f00feffd700 1 -- 172.21.15.163:0/892090108 >> v1:172.21.15.163:6789/0 conn(0x7f01000c8e00 legacy=0x7f01000c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:17.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.406+0000 7f00feffd700 1 -- 172.21.15.163:0/892090108 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010015b970 con 0x7f010006c0a0 2023-10-09T06:39:17.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.406+0000 7f00fe7fc700 1 -- 172.21.15.163:0/892090108 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1283812682 0 0) 0x7f010015c2a0 con 0x7f01000c8e00 2023-10-09T06:39:17.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.406+0000 7f00feffd700 1 --2- 172.21.15.163:0/892090108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f010006c0a0 0x7f0100141330 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f01000cafd0 tx=0x7f00f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753b3d05335b3496 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:17.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.406+0000 7f00fe7fc700 1 -- 172.21.15.163:0/892090108 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f400b860 con 0x7f010006c0a0 2023-10-09T06:39:17.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.406+0000 7f00fe7fc700 1 -- 172.21.15.163:0/892090108 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f400b9c0 con 0x7f010006c0a0 2023-10-09T06:39:17.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.406+0000 7f00fe7fc700 1 -- 172.21.15.163:0/892090108 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f400bd60 con 0x7f010006c0a0 2023-10-09T06:39:17.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.407+0000 7f01060d6700 1 -- 172.21.15.163:0/892090108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f010006c0a0 msgr2=0x7f0100141330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:17.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.407+0000 7f01060d6700 1 --2- 172.21.15.163:0/892090108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f010006c0a0 0x7f0100141330 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f01000cafd0 tx=0x7f00f4009250 comp rx=0 tx=0).stop 2023-10-09T06:39:17.409 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.407+0000 7f01060d6700 1 -- 172.21.15.163:0/892090108 shutdown_connections 2023-10-09T06:39:17.409 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.407+0000 7f01060d6700 1 --2- 172.21.15.163:0/892090108 >> v2:172.21.15.181:3300/0 conn(0x7f010006bcd0 0x7f0100140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:17.409 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.407+0000 7f01060d6700 1 --2- 172.21.15.163:0/892090108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f010006c0a0 0x7f0100141330 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:17.409 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.407+0000 7f01060d6700 1 -- 172.21.15.163:0/892090108 >> 172.21.15.163:0/892090108 conn(0x7f01000ce600 msgr2=0x7f01000cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:17.409 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.407+0000 7f01060d6700 1 -- 172.21.15.163:0/892090108 shutdown_connections 2023-10-09T06:39:17.409 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.407+0000 7f01060d6700 1 -- 172.21.15.163:0/892090108 wait complete. 2023-10-09T06:39:17.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.408+0000 7f01060d6700 1 Processor -- start 2023-10-09T06:39:17.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.408+0000 7f01060d6700 1 -- start start 2023-10-09T06:39:17.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.409+0000 7f01060d6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f010006bcd0 0x7f010013cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:17.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.409+0000 7f01060d6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f010006c0a0 0x7f010014fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:17.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.409+0000 7f01060d6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01000c8e00 0x7f0100154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:17.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.409+0000 7f01060d6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01001540a0 con 0x7f010006c0a0 2023-10-09T06:39:17.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.409+0000 7f01060d6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0100153500 con 0x7f01000c8e00 2023-10-09T06:39:17.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.409+0000 7f01060d6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0100153770 con 0x7f010006bcd0 2023-10-09T06:39:17.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.409+0000 7f00feffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f010006c0a0 0x7f010014fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:17.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.409+0000 7f00feffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f010006c0a0 0x7f010014fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44910/0 (socket says 172.21.15.163:44910) 2023-10-09T06:39:17.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.409+0000 7f00feffd700 1 -- 172.21.15.163:0/1989189984 learned_addr learned my addr 172.21.15.163:0/1989189984 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:17.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.409+0000 7f00ff7fe700 1 --2- 172.21.15.163:0/1989189984 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f010006bcd0 0x7f010013cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:17.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.409+0000 7f00fffff700 1 --2- 172.21.15.163:0/1989189984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01000c8e00 0x7f0100154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:17.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.409+0000 7f00feffd700 1 -- 172.21.15.163:0/1989189984 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f010006bcd0 msgr2=0x7f010013cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:17.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.409+0000 7f00feffd700 1 --2- 172.21.15.163:0/1989189984 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f010006bcd0 0x7f010013cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:17.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.409+0000 7f00feffd700 1 -- 172.21.15.163:0/1989189984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01000c8e00 msgr2=0x7f0100154530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:17.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.409+0000 7f00feffd700 1 --2- 172.21.15.163:0/1989189984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01000c8e00 0x7f0100154530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:17.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.409+0000 7f00feffd700 1 -- 172.21.15.163:0/1989189984 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f40094f0 con 0x7f010006c0a0 2023-10-09T06:39:17.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.409+0000 7f00ff7fe700 1 --2- 172.21.15.163:0/1989189984 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f010006bcd0 0x7f010013cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:17.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.409+0000 7f00fffff700 1 --2- 172.21.15.163:0/1989189984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01000c8e00 0x7f0100154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:17.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.409+0000 7f00feffd700 1 --2- 172.21.15.163:0/1989189984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f010006c0a0 0x7f010014fed0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f0100069220 tx=0x7f00f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:17.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.409+0000 7f00e77fe700 1 -- 172.21.15.163:0/1989189984 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4026450 con 0x7f010006c0a0 2023-10-09T06:39:17.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.409+0000 7f00e77fe700 1 -- 172.21.15.163:0/1989189984 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f40265b0 con 0x7f010006c0a0 2023-10-09T06:39:17.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.409+0000 7f00e77fe700 1 -- 172.21.15.163:0/1989189984 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4026970 con 0x7f010006c0a0 2023-10-09T06:39:17.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.409+0000 7f01060d6700 1 -- 172.21.15.163:0/1989189984 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0100153a50 con 0x7f010006c0a0 2023-10-09T06:39:17.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.409+0000 7f01060d6700 1 -- 172.21.15.163:0/1989189984 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010013d0d0 con 0x7f010006c0a0 2023-10-09T06:39:17.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.410+0000 7f01060d6700 1 -- 172.21.15.163:0/1989189984 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0100154250 con 0x7f010006c0a0 2023-10-09T06:39:17.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.411+0000 7f00e77fe700 1 -- 172.21.15.163:0/1989189984 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f00f401a420 con 0x7f010006c0a0 2023-10-09T06:39:17.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.411+0000 7f00e77fe700 1 --2- 172.21.15.163:0/1989189984 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f00ec067320 0x7f00ec0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:17.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.411+0000 7f00e77fe700 1 -- 172.21.15.163:0/1989189984 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f00f40866a0 con 0x7f010006c0a0 2023-10-09T06:39:17.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.411+0000 7f00ff7fe700 1 --2- 172.21.15.163:0/1989189984 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f00ec067320 0x7f00ec0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:17.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.412+0000 7f00ff7fe700 1 --2- 172.21.15.163:0/1989189984 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f00ec067320 0x7f00ec0697d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f00f00075e0 tx=0x7f00f00097e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:17.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.415+0000 7f00e77fe700 1 -- 172.21.15.163:0/1989189984 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0100154250 con 0x7f010006c0a0 2023-10-09T06:39:17.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.638+0000 7f01060d6700 1 -- 172.21.15.163:0/1989189984 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0100154250 con 0x7f010006c0a0 2023-10-09T06:39:17.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.639+0000 7f00e77fe700 1 -- 172.21.15.163:0/1989189984 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) v1 ==== 95+0+3540 (secure 0 0 0) 0x7f00f4015df0 con 0x7f010006c0a0 2023-10-09T06:39:17.639 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:17.640 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":117,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7756,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/78355781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":78355781},{"type":"v1","addr":"172.21.15.181:6837","nonce":78355781}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":117},{"gid":8211,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3056573272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3056573272},{"type":"v1","addr":"172.21.15.163:6834","nonce":3056573272}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":8214,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3482861499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3482861499},{"type":"v1","addr":"172.21.15.163:6835","nonce":3482861499}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":8223,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/4239124825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":4239124825},{"type":"v1","addr":"172.21.15.181:6836","nonce":4239124825}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[]} 2023-10-09T06:39:17.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.640+0000 7f01060d6700 1 -- 172.21.15.163:0/1989189984 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f00ec067320 msgr2=0x7f00ec0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:17.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.640+0000 7f01060d6700 1 --2- 172.21.15.163:0/1989189984 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f00ec067320 0x7f00ec0697d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f00f00075e0 tx=0x7f00f00097e0 comp rx=0 tx=0).stop 2023-10-09T06:39:17.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.640+0000 7f01060d6700 1 -- 172.21.15.163:0/1989189984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f010006c0a0 msgr2=0x7f010014fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:17.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.640+0000 7f01060d6700 1 --2- 172.21.15.163:0/1989189984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f010006c0a0 0x7f010014fed0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f0100069220 tx=0x7f00f4017000 comp rx=0 tx=0).stop 2023-10-09T06:39:17.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.640+0000 7f01060d6700 1 -- 172.21.15.163:0/1989189984 shutdown_connections 2023-10-09T06:39:17.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.640+0000 7f01060d6700 1 --2- 172.21.15.163:0/1989189984 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f00ec067320 0x7f00ec0697d0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:17.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.640+0000 7f01060d6700 1 --2- 172.21.15.163:0/1989189984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01000c8e00 0x7f0100154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:17.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.640+0000 7f01060d6700 1 --2- 172.21.15.163:0/1989189984 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f010006bcd0 0x7f010013cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:17.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.640+0000 7f01060d6700 1 --2- 172.21.15.163:0/1989189984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f010006c0a0 0x7f010014fed0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:17.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.640+0000 7f01060d6700 1 -- 172.21.15.163:0/1989189984 >> 172.21.15.163:0/1989189984 conn(0x7f01000ce600 msgr2=0x7f01000c44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:17.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.641+0000 7f01060d6700 1 -- 172.21.15.163:0/1989189984 shutdown_connections 2023-10-09T06:39:17.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.641+0000 7f01060d6700 1 -- 172.21.15.163:0/1989189984 wait complete. 2023-10-09T06:39:17.643 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 117 2023-10-09T06:39:17.653 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-09T06:39:17.653 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-09T06:39:17.653 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-09T06:39:17.654 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:39:17.654 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-09T06:39:17.654 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-09T06:39:17.654 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-09T06:39:17.654 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:39:17.655 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-09T06:39:17.655 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-09T06:39:17.655 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-09T06:39:17.655 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:39:17.655 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-09T06:39:17.655 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-09T06:39:17.656 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-09T06:39:17.656 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:39:17.662 INFO:tasks.ceph.mds.a:Stopped 2023-10-09T06:39:17.663 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-09T06:39:17.663 INFO:tasks.ceph.mds.c:Stopped 2023-10-09T06:39:17.664 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-09T06:39:17.668 INFO:tasks.ceph.mds.b:Stopped 2023-10-09T06:39:17.668 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-09T06:39:17.669 INFO:tasks.ceph.mds.d:Stopped 2023-10-09T06:39:17.669 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-09T06:39:17.705 INFO:tasks.ceph.mds.a:Started 2023-10-09T06:39:17.705 DEBUG:teuthology.parallel:result is None 2023-10-09T06:39:17.706 INFO:tasks.ceph.mds.c:Started 2023-10-09T06:39:17.706 DEBUG:teuthology.parallel:result is None 2023-10-09T06:39:17.712 INFO:tasks.ceph.mds.b:Started 2023-10-09T06:39:17.712 INFO:tasks.ceph.mds.d:Started 2023-10-09T06:39:17.713 DEBUG:teuthology.parallel:result is None 2023-10-09T06:39:17.713 DEBUG:teuthology.parallel:result is None 2023-10-09T06:39:17.713 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-09T06:39:17.779 INFO:tasks.ceph.mds.a.smithi163.stdout:starting mds.a at 2023-10-09T06:39:17.779 INFO:tasks.ceph.mds.c.smithi163.stdout:starting mds.c at 2023-10-09T06:39:17.792 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-09T06:39:17.792 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-09T06:39:17.835 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.836+0000 7fe91754c700 1 Processor -- start 2023-10-09T06:39:17.836 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.836+0000 7fe91754c700 1 -- start start 2023-10-09T06:39:17.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.836+0000 7fe91754c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe910159890 0x7fe91015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:17.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.837+0000 7fe91754c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe910159c60 0x7fe91015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:17.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.837+0000 7fe91754c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe91015db10 0x7fe910069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:17.837 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.837+0000 7fe91754c700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe91015df50 con 0x7fe910159890 2023-10-09T06:39:17.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.837+0000 7fe91754c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe91015a160 con 0x7fe910159c60 2023-10-09T06:39:17.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.837+0000 7fe91754c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9100d13a0 con 0x7fe91015db10 2023-10-09T06:39:17.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.837+0000 7fe9152e8700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe910159890 0x7fe91015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:17.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.837+0000 7fe9152e8700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe910159890 0x7fe91015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44956/0 (socket says 172.21.15.163:44956) 2023-10-09T06:39:17.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.837+0000 7fe9152e8700 1 -- 172.21.15.163:0/2363055833 learned_addr learned my addr 172.21.15.163:0/2363055833 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:17.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.837+0000 7fe915ae9700 1 --2- 172.21.15.163:0/2363055833 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe91015db10 0x7fe910069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:17.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.837+0000 7fe914ae7700 1 --2- 172.21.15.163:0/2363055833 >> v2:172.21.15.181:3300/0 conn(0x7fe910159c60 0x7fe91015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:17.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.837+0000 7fe9152e8700 1 -- 172.21.15.163:0/2363055833 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe91015db10 msgr2=0x7fe910069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:17.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.837+0000 7fe9152e8700 1 --2- 172.21.15.163:0/2363055833 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe91015db10 0x7fe910069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:17.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.837+0000 7fe9152e8700 1 -- 172.21.15.163:0/2363055833 >> v2:172.21.15.181:3300/0 conn(0x7fe910159c60 msgr2=0x7fe91015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:17.838 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.837+0000 7fe9152e8700 1 --2- 172.21.15.163:0/2363055833 >> v2:172.21.15.181:3300/0 conn(0x7fe910159c60 0x7fe91015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:17.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.837+0000 7fe9152e8700 1 -- 172.21.15.163:0/2363055833 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe91015e140 con 0x7fe910159890 2023-10-09T06:39:17.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.837+0000 7fe9152e8700 1 --2- 172.21.15.163:0/2363055833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe910159890 0x7fe91015abd0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fe9100b67f0 tx=0x7fe904009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8cad473d591fe3 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:17.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.837+0000 7fe903fff700 1 -- 172.21.15.163:0/2363055833 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90400b640 con 0x7fe910159890 2023-10-09T06:39:17.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.837+0000 7fe903fff700 1 -- 172.21.15.163:0/2363055833 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe90400b7a0 con 0x7fe910159890 2023-10-09T06:39:17.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.837+0000 7fe903fff700 1 -- 172.21.15.163:0/2363055833 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90400bb60 con 0x7fe910159890 2023-10-09T06:39:17.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.838+0000 7fe91754c700 1 -- 172.21.15.163:0/2363055833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe910159890 msgr2=0x7fe91015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:17.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.838+0000 7fe91754c700 1 --2- 172.21.15.163:0/2363055833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe910159890 0x7fe91015abd0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fe9100b67f0 tx=0x7fe904009250 comp rx=0 tx=0).stop 2023-10-09T06:39:17.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.838+0000 7fe91754c700 1 -- 172.21.15.163:0/2363055833 shutdown_connections 2023-10-09T06:39:17.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.838+0000 7fe91754c700 1 --2- 172.21.15.163:0/2363055833 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe91015db10 0x7fe910069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:17.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.838+0000 7fe91754c700 1 --2- 172.21.15.163:0/2363055833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe910159890 0x7fe91015abd0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:17.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.838+0000 7fe91754c700 1 --2- 172.21.15.163:0/2363055833 >> v2:172.21.15.181:3300/0 conn(0x7fe910159c60 0x7fe91015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:17.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.838+0000 7fe91754c700 1 -- 172.21.15.163:0/2363055833 >> 172.21.15.163:0/2363055833 conn(0x7fe910068e20 msgr2=0x7fe9100d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:17.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.838+0000 7fe91754c700 1 -- 172.21.15.163:0/2363055833 shutdown_connections 2023-10-09T06:39:17.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.838+0000 7fe91754c700 1 -- 172.21.15.163:0/2363055833 wait complete. 2023-10-09T06:39:17.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.839+0000 7fe91754c700 1 Processor -- start 2023-10-09T06:39:17.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.839+0000 7fe91754c700 1 -- start start 2023-10-09T06:39:17.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.840+0000 7fe91754c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe910159890 0x7fe9100b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:17.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.840+0000 7fe91754c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe910159c60 0x7fe9100c9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:17.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.840+0000 7fe91754c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe91015db10 0x7fe9100c9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:17.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.840+0000 7fe91754c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe910153650 con 0x7fe910159890 2023-10-09T06:39:17.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.840+0000 7fe91754c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9100c9dd0 con 0x7fe910159c60 2023-10-09T06:39:17.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.840+0000 7fe91754c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9100b3630 con 0x7fe91015db10 2023-10-09T06:39:17.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.840+0000 7fe9152e8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe910159890 0x7fe9100b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:17.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.840+0000 7fe9152e8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe910159890 0x7fe9100b2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:44970/0 (socket says 172.21.15.163:44970) 2023-10-09T06:39:17.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.840+0000 7fe9152e8700 1 -- 172.21.15.163:0/1721248693 learned_addr learned my addr 172.21.15.163:0/1721248693 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:17.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.840+0000 7fe914ae7700 1 --2- 172.21.15.163:0/1721248693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe910159c60 0x7fe9100c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:17.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.840+0000 7fe915ae9700 1 --2- 172.21.15.163:0/1721248693 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe91015db10 0x7fe9100c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:17.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.840+0000 7fe9152e8700 1 -- 172.21.15.163:0/1721248693 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe91015db10 msgr2=0x7fe9100c9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:17.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.840+0000 7fe9152e8700 1 --2- 172.21.15.163:0/1721248693 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe91015db10 0x7fe9100c9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:17.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.840+0000 7fe9152e8700 1 -- 172.21.15.163:0/1721248693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe910159c60 msgr2=0x7fe9100c9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:17.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.840+0000 7fe9152e8700 1 --2- 172.21.15.163:0/1721248693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe910159c60 0x7fe9100c9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:17.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.840+0000 7fe9152e8700 1 -- 172.21.15.163:0/1721248693 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9040094f0 con 0x7fe910159890 2023-10-09T06:39:17.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.840+0000 7fe914ae7700 1 --2- 172.21.15.163:0/1721248693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe910159c60 0x7fe9100c9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:17.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.841+0000 7fe9152e8700 1 --2- 172.21.15.163:0/1721248693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe910159890 0x7fe9100b2170 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fe910070e20 tx=0x7fe904017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:17.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.841+0000 7fe9017fa700 1 -- 172.21.15.163:0/1721248693 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe904000d10 con 0x7fe910159890 2023-10-09T06:39:17.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.841+0000 7fe9017fa700 1 -- 172.21.15.163:0/1721248693 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe90400bb20 con 0x7fe910159890 2023-10-09T06:39:17.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.841+0000 7fe91754c700 1 -- 172.21.15.163:0/1721248693 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9100b3910 con 0x7fe910159890 2023-10-09T06:39:17.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.841+0000 7fe9017fa700 1 -- 172.21.15.163:0/1721248693 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe904011670 con 0x7fe910159890 2023-10-09T06:39:17.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.841+0000 7fe91754c700 1 -- 172.21.15.163:0/1721248693 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9100b3ed0 con 0x7fe910159890 2023-10-09T06:39:17.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.842+0000 7fe91754c700 1 -- 172.21.15.163:0/1721248693 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe91004c3c0 con 0x7fe910159890 2023-10-09T06:39:17.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.842+0000 7fe9017fa700 1 -- 172.21.15.163:0/1721248693 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe9040119e0 con 0x7fe910159890 2023-10-09T06:39:17.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.842+0000 7fe9017fa700 1 --2- 172.21.15.163:0/1721248693 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe8f4067320 0x7fe8f40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:17.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.843+0000 7fe9017fa700 1 -- 172.21.15.163:0/1721248693 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fe904086170 con 0x7fe910159890 2023-10-09T06:39:17.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.843+0000 7fe914ae7700 1 --2- 172.21.15.163:0/1721248693 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe8f4067320 0x7fe8f40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:17.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.843+0000 7fe914ae7700 1 --2- 172.21.15.163:0/1721248693 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe8f4067320 0x7fe8f40697d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe90c006f00 tx=0x7fe90c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:17.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:17.847+0000 7fe9017fa700 1 -- 172.21.15.163:0/1721248693 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe904053100 con 0x7fe910159890 2023-10-09T06:39:18.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:18.046+0000 7fe91754c700 1 -- 172.21.15.163:0/1721248693 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe91006a5b0 con 0x7fe910159890 2023-10-09T06:39:18.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:18.885+0000 7fe9017fa700 1 -- 172.21.15.163:0/1721248693 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v178) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe904056720 con 0x7fe910159890 2023-10-09T06:39:18.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:18.888+0000 7fe91754c700 1 -- 172.21.15.163:0/1721248693 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe8f4067320 msgr2=0x7fe8f40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:18.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:18.888+0000 7fe91754c700 1 --2- 172.21.15.163:0/1721248693 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe8f4067320 0x7fe8f40697d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe90c006f00 tx=0x7fe90c008040 comp rx=0 tx=0).stop 2023-10-09T06:39:18.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:18.888+0000 7fe91754c700 1 -- 172.21.15.163:0/1721248693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe910159890 msgr2=0x7fe9100b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:18.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:18.888+0000 7fe91754c700 1 --2- 172.21.15.163:0/1721248693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe910159890 0x7fe9100b2170 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fe910070e20 tx=0x7fe904017000 comp rx=0 tx=0).stop 2023-10-09T06:39:18.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:18.888+0000 7fe91754c700 1 -- 172.21.15.163:0/1721248693 shutdown_connections 2023-10-09T06:39:18.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:18.888+0000 7fe91754c700 1 --2- 172.21.15.163:0/1721248693 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe8f4067320 0x7fe8f40697d0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:18.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:18.888+0000 7fe91754c700 1 --2- 172.21.15.163:0/1721248693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe910159890 0x7fe9100b2170 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:18.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:18.888+0000 7fe91754c700 1 --2- 172.21.15.163:0/1721248693 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe91015db10 0x7fe9100c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:18.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:18.888+0000 7fe91754c700 1 --2- 172.21.15.163:0/1721248693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe910159c60 0x7fe9100c9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:18.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:18.888+0000 7fe91754c700 1 -- 172.21.15.163:0/1721248693 >> 172.21.15.163:0/1721248693 conn(0x7fe910068e20 msgr2=0x7fe9100c7070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:18.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:18.888+0000 7fe91754c700 1 -- 172.21.15.163:0/1721248693 shutdown_connections 2023-10-09T06:39:18.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:18.888+0000 7fe91754c700 1 -- 172.21.15.163:0/1721248693 wait complete. 2023-10-09T06:39:18.889 INFO:teuthology.orchestra.run.smithi163.stderr: removed all blocklist entries 2023-10-09T06:39:18.901 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-09T06:39:19.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.019+0000 7f4651a08700 1 Processor -- start 2023-10-09T06:39:19.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.019+0000 7f4651a08700 1 -- start start 2023-10-09T06:39:19.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.019+0000 7f4651a08700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f464c159c60 0x7f464c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:19.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.019+0000 7f4651a08700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f464c15db10 0x7f464c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:19.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.020+0000 7f4651a08700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f464c15a160 con 0x7f464c159890 2023-10-09T06:39:19.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.020+0000 7f4651a08700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f464c15df50 con 0x7f464c159c60 2023-10-09T06:39:19.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.020+0000 7f4651a08700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f464c0d13a0 con 0x7f464c15db10 2023-10-09T06:39:19.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.020+0000 7f464a7fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f464c159c60 0x7f464c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:19.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.020+0000 7f464affd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f464c159890 0x7f464c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53094/0 (socket says 172.21.15.163:53094) 2023-10-09T06:39:19.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.020+0000 7f464affd700 1 -- 172.21.15.163:0/517917479 learned_addr learned my addr 172.21.15.163:0/517917479 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:19.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.020+0000 7f464a7fc700 1 -- 172.21.15.163:0/517917479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f464c15db10 msgr2=0x7f464c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:19.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.020+0000 7f464b7fe700 1 --2- 172.21.15.163:0/517917479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f464c15db10 0x7f464c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:19.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.020+0000 7f464a7fc700 1 --2- 172.21.15.163:0/517917479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f464c15db10 0x7f464c069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.020+0000 7f4649ffb700 1 -- 172.21.15.163:0/517917479 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1058216217 0 0) 0x7f464c15a160 con 0x7f464c159890 2023-10-09T06:39:19.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.020+0000 7f464a7fc700 1 -- 172.21.15.163:0/517917479 >> v1:172.21.15.163:6789/0 conn(0x7f464c159890 legacy=0x7f464c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:19.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.020+0000 7f464a7fc700 1 -- 172.21.15.163:0/517917479 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f464c15e140 con 0x7f464c159c60 2023-10-09T06:39:19.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.020+0000 7f464a7fc700 1 --2- 172.21.15.163:0/517917479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f464c159c60 0x7f464c15d5d0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f46340098e0 tx=0x7f46340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac0c15b1ead8a4c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:19.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.020+0000 7f4649ffb700 1 -- 172.21.15.163:0/517917479 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4634009e80 con 0x7f464c159c60 2023-10-09T06:39:19.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.020+0000 7f4649ffb700 1 -- 172.21.15.163:0/517917479 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463400b860 con 0x7f464c159c60 2023-10-09T06:39:19.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.020+0000 7f4649ffb700 1 -- 172.21.15.163:0/517917479 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463400bbe0 con 0x7f464c159c60 2023-10-09T06:39:19.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.021+0000 7f4651a08700 1 -- 172.21.15.163:0/517917479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f464c159c60 msgr2=0x7f464c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:19.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.021+0000 7f4651a08700 1 --2- 172.21.15.163:0/517917479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f464c159c60 0x7f464c15d5d0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f46340098e0 tx=0x7f46340091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.021+0000 7f4651a08700 1 -- 172.21.15.163:0/517917479 shutdown_connections 2023-10-09T06:39:19.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.021+0000 7f4651a08700 1 --2- 172.21.15.163:0/517917479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f464c15db10 0x7f464c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.021+0000 7f4651a08700 1 --2- 172.21.15.163:0/517917479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f464c159c60 0x7f464c15d5d0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.021+0000 7f4651a08700 1 -- 172.21.15.163:0/517917479 >> 172.21.15.163:0/517917479 conn(0x7f464c068e20 msgr2=0x7f464c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:19.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.021+0000 7f4651a08700 1 -- 172.21.15.163:0/517917479 shutdown_connections 2023-10-09T06:39:19.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.021+0000 7f4651a08700 1 -- 172.21.15.163:0/517917479 wait complete. 2023-10-09T06:39:19.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.022+0000 7f4651a08700 1 Processor -- start 2023-10-09T06:39:19.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.022+0000 7f4651a08700 1 -- start start 2023-10-09T06:39:19.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.023+0000 7f4651a08700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f464c159890 0x7f464c16dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:19.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.023+0000 7f4651a08700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f464c159c60 0x7f464c1537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:19.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.023+0000 7f4651a08700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f464c15db10 0x7f464c153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:19.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.023+0000 7f4651a08700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c0c7690 con 0x7f464c159890 2023-10-09T06:39:19.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.023+0000 7f4651a08700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c1514f0 con 0x7f464c15db10 2023-10-09T06:39:19.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.023+0000 7f4651a08700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f464c1517d0 con 0x7f464c159c60 2023-10-09T06:39:19.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.023+0000 7f464affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f464c159890 0x7f464c16dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:19.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.023+0000 7f464affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f464c159890 0x7f464c16dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36282/0 (socket says 172.21.15.163:36282) 2023-10-09T06:39:19.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.023+0000 7f464affd700 1 -- 172.21.15.163:0/3649433789 learned_addr learned my addr 172.21.15.163:0/3649433789 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:19.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.023+0000 7f464affd700 1 -- 172.21.15.163:0/3649433789 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f464c159c60 msgr2=0x7f464c1537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:19.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.023+0000 7f464affd700 1 --2- 172.21.15.163:0/3649433789 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f464c159c60 0x7f464c1537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.023+0000 7f464affd700 1 -- 172.21.15.163:0/3649433789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f464c15db10 msgr2=0x7f464c153d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:19.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.023+0000 7f464affd700 1 --2- 172.21.15.163:0/3649433789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f464c15db10 0x7f464c153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.023+0000 7f464affd700 1 -- 172.21.15.163:0/3649433789 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4634009420 con 0x7f464c159890 2023-10-09T06:39:19.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.023+0000 7f464affd700 1 --2- 172.21.15.163:0/3649433789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f464c159890 0x7f464c16dd80 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f463c00cd80 tx=0x7f463c001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:19.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.023+0000 7f4642ffd700 1 -- 172.21.15.163:0/3649433789 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c00ab10 con 0x7f464c159890 2023-10-09T06:39:19.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.023+0000 7f4642ffd700 1 -- 172.21.15.163:0/3649433789 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463c00ac70 con 0x7f464c159890 2023-10-09T06:39:19.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.023+0000 7f4651a08700 1 -- 172.21.15.163:0/3649433789 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f464c151ab0 con 0x7f464c159890 2023-10-09T06:39:19.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.024+0000 7f4642ffd700 1 -- 172.21.15.163:0/3649433789 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c0070f0 con 0x7f464c159890 2023-10-09T06:39:19.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.024+0000 7f4651a08700 1 -- 172.21.15.163:0/3649433789 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f464c152070 con 0x7f464c159890 2023-10-09T06:39:19.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.024+0000 7f4651a08700 1 -- 172.21.15.163:0/3649433789 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f464c04c3c0 con 0x7f464c159890 2023-10-09T06:39:19.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.025+0000 7f4642ffd700 1 -- 172.21.15.163:0/3649433789 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f463c0074e0 con 0x7f464c159890 2023-10-09T06:39:19.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.025+0000 7f4642ffd700 1 --2- 172.21.15.163:0/3649433789 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f462c067320 0x7f462c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:19.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.025+0000 7f4642ffd700 1 -- 172.21.15.163:0/3649433789 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f463c084e50 con 0x7f464c159890 2023-10-09T06:39:19.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.026+0000 7f464a7fc700 1 --2- 172.21.15.163:0/3649433789 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f462c067320 0x7f462c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:19.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.026+0000 7f464a7fc700 1 --2- 172.21.15.163:0/3649433789 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f462c067320 0x7f462c0697d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f464c0b6630 tx=0x7f4634017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:19.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.029+0000 7f4642ffd700 1 -- 172.21.15.163:0/3649433789 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f463c051eb0 con 0x7f464c159890 2023-10-09T06:39:19.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.285+0000 7f4651a08700 1 -- 172.21.15.163:0/3649433789 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f464c0d1560 con 0x7f464c159890 2023-10-09T06:39:19.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.286+0000 7f4642ffd700 1 -- 172.21.15.163:0/3649433789 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f463c0554d0 con 0x7f464c159890 2023-10-09T06:39:19.286 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:19.286 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:39:19.286 INFO:teuthology.orchestra.run.smithi163.stdout: "auth_dump": [ 2023-10-09T06:39:19.286 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:19.286 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.a", 2023-10-09T06:39:19.286 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNltXtwDxAAlnrWRsfnZDLEmsJVowh3Mg==", 2023-10-09T06:39:19.286 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:19.286 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:39:19.287 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:39:19.287 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:39:19.287 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:19.287 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:19.287 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:19.287 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:19.287 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.b", 2023-10-09T06:39:19.287 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlTdGhGhAAXuldy0HrFd+44sSu/EV3qA==", 2023-10-09T06:39:19.287 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:19.287 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:39:19.288 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:39:19.288 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:39:19.288 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:19.288 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:19.288 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:19.288 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:19.288 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.c", 2023-10-09T06:39:19.288 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlGis7FhAAYG76TiOFX8QkTwjNX2+Ncw==", 2023-10-09T06:39:19.288 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:19.288 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:39:19.288 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:39:19.289 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:39:19.289 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:19.289 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:19.289 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:19.289 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:19.289 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.d", 2023-10-09T06:39:19.289 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlKhRuIRAAfhG5Gpc7klXJloYXepOhsg==", 2023-10-09T06:39:19.289 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:19.289 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:39:19.290 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:39:19.290 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:39:19.290 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:19.290 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:19.290 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:19.290 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:19.290 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.0", 2023-10-09T06:39:19.290 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDcnSNlrQeFOxAA+s6UitrTn5TK1N7kgySoXw==", 2023-10-09T06:39:19.290 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:19.290 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:39:19.291 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:39:19.291 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:19.291 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:19.291 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:19.291 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:19.291 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.1", 2023-10-09T06:39:19.291 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDfnSNlleccORAAnB+lNUFblivdoY44JfELxA==", 2023-10-09T06:39:19.291 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:19.291 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:39:19.291 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:39:19.291 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:19.291 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:19.292 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:19.292 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:19.292 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.2", 2023-10-09T06:39:19.292 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDinSNlFoB4NxAAT0+trRBij5rqgc+F0eWUfQ==", 2023-10-09T06:39:19.292 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:19.292 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:39:19.292 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:39:19.292 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:19.292 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:19.292 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:19.292 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:19.292 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.3", 2023-10-09T06:39:19.293 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDlnSNlRRQ6NhAABoBtrFjSvQujolq0Pxrd1Q==", 2023-10-09T06:39:19.293 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:19.293 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:39:19.293 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:39:19.293 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:19.293 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:19.293 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:19.293 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:19.293 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.4", 2023-10-09T06:39:19.293 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDrnSNl308uOxAA5OJyPLGQmVSJqha5d+nKKQ==", 2023-10-09T06:39:19.293 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:19.293 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:39:19.294 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:39:19.294 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:19.294 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:19.294 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:19.294 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:19.294 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.5", 2023-10-09T06:39:19.294 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDunSNlHnK+OBAAU7l4dCJ82SbGSzBEm8Axzw==", 2023-10-09T06:39:19.294 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:19.294 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:39:19.294 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:39:19.294 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:19.294 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:19.295 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:19.295 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:19.295 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.6", 2023-10-09T06:39:19.295 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDxnSNlwUSLOBAAN5N+F5ho+Co8pBh4Rz+lXQ==", 2023-10-09T06:39:19.295 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:19.295 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:39:19.295 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:39:19.295 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:19.295 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:19.295 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:19.295 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:19.296 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.7", 2023-10-09T06:39:19.296 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQD0nSNl2gxGORAAoE8OLGEifcEx/km8rfSV9g==", 2023-10-09T06:39:19.296 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:19.296 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:39:19.296 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:39:19.296 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:19.296 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:19.296 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:19.296 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:19.296 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.0", 2023-10-09T06:39:19.297 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlnxY5JRAApr6czcpMdQYYtXQ9yg2q6A==", 2023-10-09T06:39:19.297 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:19.297 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:39:19.297 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow r", 2023-10-09T06:39:19.297 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-09T06:39:19.297 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:19.297 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:19.297 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:19.297 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.1", 2023-10-09T06:39:19.297 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlk6ybLRAAuShqTDAtymGdxN8KBdZDDA==", 2023-10-09T06:39:19.297 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:19.298 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:39:19.298 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:39:19.298 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:39:19.298 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:39:19.298 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:19.298 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:19.298 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:19.298 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.2", 2023-10-09T06:39:19.298 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlYyOmKhAAQmpr+mGfJH4GUa28dtlJwA==", 2023-10-09T06:39:19.298 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:19.298 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:39:19.299 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:39:19.299 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:39:19.299 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:39:19.299 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:19.299 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:19.299 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:19.299 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.3", 2023-10-09T06:39:19.299 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlD2H1MhAA4zvaXY/Gd0kndzi+QSwp7w==", 2023-10-09T06:39:19.299 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:19.299 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:39:19.300 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:39:19.300 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:39:19.300 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:39:19.300 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:19.300 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:19.300 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:19.300 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.admin", 2023-10-09T06:39:19.300 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDYnSNlsHH4LxAA85xeFEP1CUwoblaePYk5zA==", 2023-10-09T06:39:19.300 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:19.300 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:39:19.300 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:39:19.301 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:39:19.301 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:19.301 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:19.301 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:19.301 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:19.301 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mgr.x", 2023-10-09T06:39:19.301 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNl2sloDBAA0FoWN9iovCl6783o3/btiA==", 2023-10-09T06:39:19.301 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:19.301 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:39:19.301 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile mgr", 2023-10-09T06:39:19.301 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:19.302 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:19.302 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:19.302 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:19.302 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mgr.y", 2023-10-09T06:39:19.302 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlcFWVCBAAR2dKPg01ciTiNpSX0JIKFg==", 2023-10-09T06:39:19.302 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:19.302 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:39:19.302 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile mgr", 2023-10-09T06:39:19.302 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:19.302 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:19.302 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:19.303 INFO:teuthology.orchestra.run.smithi163.stdout: ] 2023-10-09T06:39:19.303 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:39:19.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.288+0000 7f4651a08700 1 -- 172.21.15.163:0/3649433789 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f462c067320 msgr2=0x7f462c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:19.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.288+0000 7f4651a08700 1 --2- 172.21.15.163:0/3649433789 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f462c067320 0x7f462c0697d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f464c0b6630 tx=0x7f4634017000 comp rx=0 tx=0).stop 2023-10-09T06:39:19.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.288+0000 7f4651a08700 1 -- 172.21.15.163:0/3649433789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f464c159890 msgr2=0x7f464c16dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:19.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.288+0000 7f4651a08700 1 --2- 172.21.15.163:0/3649433789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f464c159890 0x7f464c16dd80 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f463c00cd80 tx=0x7f463c001ae0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.288+0000 7f4651a08700 1 -- 172.21.15.163:0/3649433789 shutdown_connections 2023-10-09T06:39:19.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.288+0000 7f4651a08700 1 --2- 172.21.15.163:0/3649433789 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f462c067320 0x7f462c0697d0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.288+0000 7f4651a08700 1 --2- 172.21.15.163:0/3649433789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f464c159890 0x7f464c16dd80 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.288+0000 7f4651a08700 1 --2- 172.21.15.163:0/3649433789 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f464c15db10 0x7f464c153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.288+0000 7f4651a08700 1 --2- 172.21.15.163:0/3649433789 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f464c159c60 0x7f464c1537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.288+0000 7f4651a08700 1 -- 172.21.15.163:0/3649433789 >> 172.21.15.163:0/3649433789 conn(0x7f464c068e20 msgr2=0x7f464c15fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:19.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.288+0000 7f4651a08700 1 -- 172.21.15.163:0/3649433789 shutdown_connections 2023-10-09T06:39:19.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.288+0000 7f4651a08700 1 -- 172.21.15.163:0/3649433789 wait complete. 2023-10-09T06:39:19.306 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:39:19.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.420+0000 7f91cb46a700 1 Processor -- start 2023-10-09T06:39:19.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.420+0000 7f91cb46a700 1 -- start start 2023-10-09T06:39:19.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.421+0000 7f91cb46a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91c4069560 0x7f91c40699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:19.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.421+0000 7f91cb46a700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91c4069ee0 con 0x7f91c4159890 2023-10-09T06:39:19.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.421+0000 7f91cb46a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91c415a160 con 0x7f91c4159c60 2023-10-09T06:39:19.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.421+0000 7f91cb46a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c40d13a0 con 0x7f91c4069560 2023-10-09T06:39:19.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.421+0000 7f91c9206700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f91c4159890 0x7f91c415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53096/0 (socket says 172.21.15.163:53096) 2023-10-09T06:39:19.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.421+0000 7f91c9206700 1 -- 172.21.15.163:0/2516228604 learned_addr learned my addr 172.21.15.163:0/2516228604 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:19.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.422+0000 7f91c9a07700 1 --2- 172.21.15.163:0/2516228604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91c4069560 0x7f91c40699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:19.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.422+0000 7f91c3fff700 1 -- 172.21.15.163:0/2516228604 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2422630307 0 0) 0x7f91c4069ee0 con 0x7f91c4159890 2023-10-09T06:39:19.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.422+0000 7f91c3fff700 1 -- 172.21.15.163:0/2516228604 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91ac003570 con 0x7f91c4159890 2023-10-09T06:39:19.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.422+0000 7f91c3fff700 1 -- 172.21.15.163:0/2516228604 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 922098514 0 0) 0x7f91ac003570 con 0x7f91c4159890 2023-10-09T06:39:19.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.422+0000 7f91c3fff700 1 -- 172.21.15.163:0/2516228604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91c4069560 msgr2=0x7f91c40699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:19.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.422+0000 7f91c3fff700 1 --2- 172.21.15.163:0/2516228604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91c4069560 0x7f91c40699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.422+0000 7f91c3fff700 1 -- 172.21.15.163:0/2516228604 >> v1:172.21.15.181:6789/0 conn(0x7f91c4159c60 legacy=0x7f91c415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:19.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.422+0000 7f91c3fff700 1 -- 172.21.15.163:0/2516228604 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c415e140 con 0x7f91c4159890 2023-10-09T06:39:19.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.422+0000 7f91c3fff700 1 -- 172.21.15.163:0/2516228604 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f91b40017f0 con 0x7f91c4159890 2023-10-09T06:39:19.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.422+0000 7f91c3fff700 1 -- 172.21.15.163:0/2516228604 >> v1:172.21.15.163:6789/0 conn(0x7f91c4159890 legacy=0x7f91c415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:19.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.422+0000 7f91c3fff700 1 --2- 172.21.15.163:0/2516228604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91ac003a90 0x7f91ac003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:19.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.422+0000 7f91c3fff700 1 --2- 172.21.15.163:0/2516228604 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91ac004810 0x7f91ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:19.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.422+0000 7f91c3fff700 1 --2- 172.21.15.163:0/2516228604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91c4069560 0x7f91ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:19.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.422+0000 7f91c3fff700 1 -- 172.21.15.163:0/2516228604 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c40d13a0 con 0x7f91ac004810 2023-10-09T06:39:19.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.422+0000 7f91c3fff700 1 -- 172.21.15.163:0/2516228604 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac00bb40 con 0x7f91ac003a90 2023-10-09T06:39:19.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.422+0000 7f91c3fff700 1 -- 172.21.15.163:0/2516228604 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac00bd00 con 0x7f91c4069560 2023-10-09T06:39:19.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.422+0000 7f91c8a05700 1 --2- 172.21.15.163:0/2516228604 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91ac004810 0x7f91ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:19.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.422+0000 7f91c9206700 1 --2- 172.21.15.163:0/2516228604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91ac003a90 0x7f91ac003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:19.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.422+0000 7f91c9a07700 1 --2- 172.21.15.163:0/2516228604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91c4069560 0x7f91ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:19.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.422+0000 7f91c8a05700 1 -- 172.21.15.163:0/2516228604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91c4069560 msgr2=0x7f91ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:19.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.422+0000 7f91c8a05700 1 --2- 172.21.15.163:0/2516228604 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91c4069560 0x7f91ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.422+0000 7f91c8a05700 1 -- 172.21.15.163:0/2516228604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91ac003a90 msgr2=0x7f91ac003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:19.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.422+0000 7f91c8a05700 1 --2- 172.21.15.163:0/2516228604 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91ac003a90 0x7f91ac003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.422+0000 7f91c8a05700 1 -- 172.21.15.163:0/2516228604 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ac003570 con 0x7f91ac004810 2023-10-09T06:39:19.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.422+0000 7f91c8a05700 1 --2- 172.21.15.163:0/2516228604 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91ac004810 0x7f91ac004c80 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f91b8001970 tx=0x7f91b8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667ca69d694c7366 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:19.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.423+0000 7f91c3fff700 1 -- 172.21.15.163:0/2516228604 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b800c5d0 con 0x7f91ac004810 2023-10-09T06:39:19.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.423+0000 7f91c3fff700 1 -- 172.21.15.163:0/2516228604 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b800c730 con 0x7f91ac004810 2023-10-09T06:39:19.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.423+0000 7f91c3fff700 1 -- 172.21.15.163:0/2516228604 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b800cad0 con 0x7f91ac004810 2023-10-09T06:39:19.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.423+0000 7f91cb46a700 1 -- 172.21.15.163:0/2516228604 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91ac004810 msgr2=0x7f91ac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:19.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.423+0000 7f91cb46a700 1 --2- 172.21.15.163:0/2516228604 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91ac004810 0x7f91ac004c80 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f91b8001970 tx=0x7f91b8007d30 comp rx=0 tx=0).stop 2023-10-09T06:39:19.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.423+0000 7f91c9a07700 1 -- 172.21.15.163:0/2516228604 reap_dead start 2023-10-09T06:39:19.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.423+0000 7f91cb46a700 1 -- 172.21.15.163:0/2516228604 shutdown_connections 2023-10-09T06:39:19.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.423+0000 7f91cb46a700 1 -- 172.21.15.163:0/2516228604 >> 172.21.15.163:0/2516228604 conn(0x7f91c4068e20 msgr2=0x7f91c40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:19.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.423+0000 7f91cb46a700 1 -- 172.21.15.163:0/2516228604 shutdown_connections 2023-10-09T06:39:19.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.423+0000 7f91cb46a700 1 -- 172.21.15.163:0/2516228604 wait complete. 2023-10-09T06:39:19.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.424+0000 7f91cb46a700 1 Processor -- start 2023-10-09T06:39:19.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.424+0000 7f91cb46a700 1 -- start start 2023-10-09T06:39:19.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.425+0000 7f91cb46a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91ac004810 0x7f91c4150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:19.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.425+0000 7f91cb46a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91c40cfa10 0x7f91c416c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:19.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.425+0000 7f91cb46a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91c40b14d0 0x7f91c40b1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:19.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.425+0000 7f91cb46a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c40b1e80 con 0x7f91ac004810 2023-10-09T06:39:19.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.425+0000 7f91cb46a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c40cfe80 con 0x7f91c40b14d0 2023-10-09T06:39:19.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.425+0000 7f91cb46a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c4169c40 con 0x7f91c40cfa10 2023-10-09T06:39:19.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.425+0000 7f91c9206700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91ac004810 0x7f91c4150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:19.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.425+0000 7f91c9206700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91ac004810 0x7f91c4150030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36302/0 (socket says 172.21.15.163:36302) 2023-10-09T06:39:19.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.425+0000 7f91c9206700 1 -- 172.21.15.163:0/3763554038 learned_addr learned my addr 172.21.15.163:0/3763554038 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:19.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.426+0000 7f91c8a05700 1 --2- 172.21.15.163:0/3763554038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91c40cfa10 0x7f91c416c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:19.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.426+0000 7f91c9a07700 1 --2- 172.21.15.163:0/3763554038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91c40b14d0 0x7f91c40b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:19.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.426+0000 7f91c9206700 1 -- 172.21.15.163:0/3763554038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91c40cfa10 msgr2=0x7f91c416c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:19.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.426+0000 7f91c9206700 1 --2- 172.21.15.163:0/3763554038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91c40cfa10 0x7f91c416c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.426+0000 7f91c9206700 1 -- 172.21.15.163:0/3763554038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91c40b14d0 msgr2=0x7f91c40b1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:19.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.426+0000 7f91c9206700 1 --2- 172.21.15.163:0/3763554038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91c40b14d0 0x7f91c40b1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.426+0000 7f91c9206700 1 -- 172.21.15.163:0/3763554038 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c4169f20 con 0x7f91ac004810 2023-10-09T06:39:19.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.426+0000 7f91c8a05700 1 --2- 172.21.15.163:0/3763554038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91c40cfa10 0x7f91c416c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:19.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.426+0000 7f91c9206700 1 --2- 172.21.15.163:0/3763554038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91ac004810 0x7f91c4150030 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f91b4007ac0 tx=0x7f91b40021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:19.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.426+0000 7f91c17fa700 1 -- 172.21.15.163:0/3763554038 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b400ad90 con 0x7f91ac004810 2023-10-09T06:39:19.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.426+0000 7f91c17fa700 1 -- 172.21.15.163:0/3763554038 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b40133f0 con 0x7f91ac004810 2023-10-09T06:39:19.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.426+0000 7f91cb46a700 1 -- 172.21.15.163:0/3763554038 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91c416a200 con 0x7f91ac004810 2023-10-09T06:39:19.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.426+0000 7f91c17fa700 1 -- 172.21.15.163:0/3763554038 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b40137b0 con 0x7f91ac004810 2023-10-09T06:39:19.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.426+0000 7f91cb46a700 1 -- 172.21.15.163:0/3763554038 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c41507b0 con 0x7f91ac004810 2023-10-09T06:39:19.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.427+0000 7f91cb46a700 1 -- 172.21.15.163:0/3763554038 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91c416a720 con 0x7f91ac004810 2023-10-09T06:39:19.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.427+0000 7f91c17fa700 1 -- 172.21.15.163:0/3763554038 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f91b4013b20 con 0x7f91ac004810 2023-10-09T06:39:19.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.428+0000 7f91c17fa700 1 --2- 172.21.15.163:0/3763554038 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f91b0067320 0x7f91b00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:19.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.428+0000 7f91c17fa700 1 -- 172.21.15.163:0/3763554038 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f91b40856a0 con 0x7f91ac004810 2023-10-09T06:39:19.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.428+0000 7f91c8a05700 1 --2- 172.21.15.163:0/3763554038 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f91b0067320 0x7f91b00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:19.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.428+0000 7f91c8a05700 1 --2- 172.21.15.163:0/3763554038 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f91b0067320 0x7f91b00697d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f91c40b2150 tx=0x7f91b8015000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:19.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.432+0000 7f91c17fa700 1 -- 172.21.15.163:0/3763554038 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f91c416a720 con 0x7f91ac004810 2023-10-09T06:39:19.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.615+0000 7f91cb46a700 1 -- 172.21.15.163:0/3763554038 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91c416a720 con 0x7f91ac004810 2023-10-09T06:39:19.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.616+0000 7f91c17fa700 1 -- 172.21.15.163:0/3763554038 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v178) v1 ==== 74+0+12192 (secure 0 0 0) 0x7f91c416a720 con 0x7f91ac004810 2023-10-09T06:39:19.617 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:19.617 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":178,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:39:18.878848+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:39:19.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.618+0000 7f91cb46a700 1 -- 172.21.15.163:0/3763554038 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f91b0067320 msgr2=0x7f91b00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:19.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.618+0000 7f91cb46a700 1 --2- 172.21.15.163:0/3763554038 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f91b0067320 0x7f91b00697d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f91c40b2150 tx=0x7f91b8015000 comp rx=0 tx=0).stop 2023-10-09T06:39:19.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.618+0000 7f91cb46a700 1 -- 172.21.15.163:0/3763554038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91ac004810 msgr2=0x7f91c4150030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:19.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.618+0000 7f91cb46a700 1 --2- 172.21.15.163:0/3763554038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91ac004810 0x7f91c4150030 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f91b4007ac0 tx=0x7f91b40021f0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.619+0000 7f91cb46a700 1 -- 172.21.15.163:0/3763554038 shutdown_connections 2023-10-09T06:39:19.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.619+0000 7f91cb46a700 1 --2- 172.21.15.163:0/3763554038 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f91b0067320 0x7f91b00697d0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.619+0000 7f91cb46a700 1 --2- 172.21.15.163:0/3763554038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91ac004810 0x7f91c4150030 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.619+0000 7f91cb46a700 1 --2- 172.21.15.163:0/3763554038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91c40b14d0 0x7f91c40b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.619+0000 7f91cb46a700 1 --2- 172.21.15.163:0/3763554038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91c40cfa10 0x7f91c416c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.619+0000 7f91cb46a700 1 -- 172.21.15.163:0/3763554038 >> 172.21.15.163:0/3763554038 conn(0x7f91c4068e20 msgr2=0x7f91c4153550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:19.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.619+0000 7f91cb46a700 1 -- 172.21.15.163:0/3763554038 shutdown_connections 2023-10-09T06:39:19.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.619+0000 7f91cb46a700 1 -- 172.21.15.163:0/3763554038 wait complete. 2023-10-09T06:39:19.632 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-09T06:39:19.632 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:39:19.750 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.749+0000 7fe294a72700 1 Processor -- start 2023-10-09T06:39:19.750 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.749+0000 7fe294a72700 1 -- start start 2023-10-09T06:39:19.752 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.750+0000 7fe294a72700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe290159890 0x7fe29015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:19.752 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.750+0000 7fe294a72700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe290159c60 0x7fe29015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:19.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.750+0000 7fe294a72700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe29015a160 con 0x7fe29015db10 2023-10-09T06:39:19.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.750+0000 7fe294a72700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe29015df50 con 0x7fe290159c60 2023-10-09T06:39:19.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.750+0000 7fe294a72700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2900d13a0 con 0x7fe290159890 2023-10-09T06:39:19.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.750+0000 7fe28dd9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe290159c60 0x7fe29015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:19.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.750+0000 7fe28ed9d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fe29015db10 0x7fe290069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53104/0 (socket says 172.21.15.163:53104) 2023-10-09T06:39:19.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.750+0000 7fe28ed9d700 1 -- 172.21.15.163:0/1358061341 learned_addr learned my addr 172.21.15.163:0/1358061341 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:19.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.750+0000 7fe28e59c700 1 --2- 172.21.15.163:0/1358061341 >> v2:172.21.15.181:3300/0 conn(0x7fe290159890 0x7fe29015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:19.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.750+0000 7fe28dd9b700 1 -- 172.21.15.163:0/1358061341 >> v2:172.21.15.181:3300/0 conn(0x7fe290159890 msgr2=0x7fe29015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:19.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.750+0000 7fe28dd9b700 1 --2- 172.21.15.163:0/1358061341 >> v2:172.21.15.181:3300/0 conn(0x7fe290159890 0x7fe29015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.750+0000 7fe28dd9b700 1 -- 172.21.15.163:0/1358061341 >> v1:172.21.15.163:6789/0 conn(0x7fe29015db10 legacy=0x7fe290069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:19.754 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.750+0000 7fe28dd9b700 1 -- 172.21.15.163:0/1358061341 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29015e140 con 0x7fe290159c60 2023-10-09T06:39:19.754 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.750+0000 7fe28d59a700 1 -- 172.21.15.163:0/1358061341 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4110368464 0 0) 0x7fe29015a160 con 0x7fe29015db10 2023-10-09T06:39:19.754 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.750+0000 7fe28e59c700 1 --2- 172.21.15.163:0/1358061341 >> v2:172.21.15.181:3300/0 conn(0x7fe290159890 0x7fe29015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:19.754 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.751+0000 7fe28dd9b700 1 --2- 172.21.15.163:0/1358061341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe290159c60 0x7fe29015d5d0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fe2900d1090 tx=0x7fe2840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511e8bb79454a359 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:19.754 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.751+0000 7fe28d59a700 1 -- 172.21.15.163:0/1358061341 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28400b6b0 con 0x7fe290159c60 2023-10-09T06:39:19.754 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.751+0000 7fe28d59a700 1 -- 172.21.15.163:0/1358061341 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28400b810 con 0x7fe290159c60 2023-10-09T06:39:19.754 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.751+0000 7fe28d59a700 1 -- 172.21.15.163:0/1358061341 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28400bbb0 con 0x7fe290159c60 2023-10-09T06:39:19.754 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.751+0000 7fe294a72700 1 -- 172.21.15.163:0/1358061341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe290159c60 msgr2=0x7fe29015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:19.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.751+0000 7fe294a72700 1 --2- 172.21.15.163:0/1358061341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe290159c60 0x7fe29015d5d0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fe2900d1090 tx=0x7fe2840091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.751+0000 7fe294a72700 1 -- 172.21.15.163:0/1358061341 shutdown_connections 2023-10-09T06:39:19.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.751+0000 7fe294a72700 1 --2- 172.21.15.163:0/1358061341 >> v2:172.21.15.181:3300/0 conn(0x7fe290159890 0x7fe29015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.751+0000 7fe294a72700 1 --2- 172.21.15.163:0/1358061341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe290159c60 0x7fe29015d5d0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.751+0000 7fe294a72700 1 -- 172.21.15.163:0/1358061341 >> 172.21.15.163:0/1358061341 conn(0x7fe290068e20 msgr2=0x7fe2900d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:19.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.751+0000 7fe294a72700 1 -- 172.21.15.163:0/1358061341 shutdown_connections 2023-10-09T06:39:19.755 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.751+0000 7fe294a72700 1 -- 172.21.15.163:0/1358061341 wait complete. 2023-10-09T06:39:19.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.752+0000 7fe294a72700 1 Processor -- start 2023-10-09T06:39:19.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.752+0000 7fe294a72700 1 -- start start 2023-10-09T06:39:19.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.753+0000 7fe294a72700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe290159890 0x7fe290143240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:19.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.753+0000 7fe294a72700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe290159c60 0x7fe29015cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:19.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.753+0000 7fe294a72700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe29015db10 0x7fe2900c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:19.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.753+0000 7fe294a72700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe290143780 con 0x7fe29015db10 2023-10-09T06:39:19.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.753+0000 7fe294a72700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2900c78a0 con 0x7fe290159890 2023-10-09T06:39:19.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.753+0000 7fe294a72700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2900c8c00 con 0x7fe290159c60 2023-10-09T06:39:19.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.753+0000 7fe28ed9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe29015db10 0x7fe2900c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:19.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.753+0000 7fe28ed9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe29015db10 0x7fe2900c91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36314/0 (socket says 172.21.15.163:36314) 2023-10-09T06:39:19.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.753+0000 7fe28ed9d700 1 -- 172.21.15.163:0/1177786078 learned_addr learned my addr 172.21.15.163:0/1177786078 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:19.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.753+0000 7fe28e59c700 1 --2- 172.21.15.163:0/1177786078 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe290159890 0x7fe290143240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:19.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.753+0000 7fe28dd9b700 1 --2- 172.21.15.163:0/1177786078 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe290159c60 0x7fe29015cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:19.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.753+0000 7fe28ed9d700 1 -- 172.21.15.163:0/1177786078 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe290159c60 msgr2=0x7fe29015cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:19.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.753+0000 7fe28ed9d700 1 --2- 172.21.15.163:0/1177786078 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe290159c60 0x7fe29015cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.754+0000 7fe28ed9d700 1 -- 172.21.15.163:0/1177786078 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe290159890 msgr2=0x7fe290143240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:19.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.754+0000 7fe28ed9d700 1 --2- 172.21.15.163:0/1177786078 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe290159890 0x7fe290143240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.754+0000 7fe28ed9d700 1 -- 172.21.15.163:0/1177786078 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe284009420 con 0x7fe29015db10 2023-10-09T06:39:19.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.754+0000 7fe28dd9b700 1 --2- 172.21.15.163:0/1177786078 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe290159c60 0x7fe29015cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:19.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.754+0000 7fe28ed9d700 1 --2- 172.21.15.163:0/1177786078 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe29015db10 0x7fe2900c91d0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fe28800c530 tx=0x7fe288007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:19.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.754+0000 7fe27e7fc700 1 -- 172.21.15.163:0/1177786078 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe288005b30 con 0x7fe29015db10 2023-10-09T06:39:19.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.754+0000 7fe27e7fc700 1 -- 172.21.15.163:0/1177786078 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe288005c90 con 0x7fe29015db10 2023-10-09T06:39:19.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.754+0000 7fe27e7fc700 1 -- 172.21.15.163:0/1177786078 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe288007320 con 0x7fe29015db10 2023-10-09T06:39:19.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.754+0000 7fe294a72700 1 -- 172.21.15.163:0/1177786078 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2900c7b60 con 0x7fe29015db10 2023-10-09T06:39:19.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.754+0000 7fe294a72700 1 -- 172.21.15.163:0/1177786078 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2900c8120 con 0x7fe29015db10 2023-10-09T06:39:19.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.755+0000 7fe294a72700 1 -- 172.21.15.163:0/1177786078 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe29004c3c0 con 0x7fe29015db10 2023-10-09T06:39:19.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.755+0000 7fe27e7fc700 1 -- 172.21.15.163:0/1177786078 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe28801b3b0 con 0x7fe29015db10 2023-10-09T06:39:19.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.755+0000 7fe27e7fc700 1 --2- 172.21.15.163:0/1177786078 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe270067320 0x7fe2700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:19.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.756+0000 7fe27e7fc700 1 -- 172.21.15.163:0/1177786078 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fe288083dd0 con 0x7fe29015db10 2023-10-09T06:39:19.762 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.756+0000 7fe28e59c700 1 --2- 172.21.15.163:0/1177786078 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe270067320 0x7fe2700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:19.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.756+0000 7fe28e59c700 1 --2- 172.21.15.163:0/1177786078 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe270067320 0x7fe2700697d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe28000d6e0 tx=0x7fe280007850 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:19.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.760+0000 7fe27e7fc700 1 -- 172.21.15.163:0/1177786078 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe288050d70 con 0x7fe29015db10 2023-10-09T06:39:19.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.942+0000 7fe294a72700 1 -- 172.21.15.163:0/1177786078 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe29006a5b0 con 0x7fe29015db10 2023-10-09T06:39:19.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.943+0000 7fe27e7fc700 1 -- 172.21.15.163:0/1177786078 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v178) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe2880170a0 con 0x7fe29015db10 2023-10-09T06:39:19.945 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:39:19.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.945+0000 7fe294a72700 1 -- 172.21.15.163:0/1177786078 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe270067320 msgr2=0x7fe2700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:19.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.945+0000 7fe294a72700 1 --2- 172.21.15.163:0/1177786078 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe270067320 0x7fe2700697d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe28000d6e0 tx=0x7fe280007850 comp rx=0 tx=0).stop 2023-10-09T06:39:19.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.945+0000 7fe294a72700 1 -- 172.21.15.163:0/1177786078 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe29015db10 msgr2=0x7fe2900c91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:19.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.945+0000 7fe294a72700 1 --2- 172.21.15.163:0/1177786078 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe29015db10 0x7fe2900c91d0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fe28800c530 tx=0x7fe288007f90 comp rx=0 tx=0).stop 2023-10-09T06:39:19.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.945+0000 7fe294a72700 1 -- 172.21.15.163:0/1177786078 shutdown_connections 2023-10-09T06:39:19.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.945+0000 7fe294a72700 1 --2- 172.21.15.163:0/1177786078 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe270067320 0x7fe2700697d0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.945+0000 7fe294a72700 1 --2- 172.21.15.163:0/1177786078 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe29015db10 0x7fe2900c91d0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.945+0000 7fe294a72700 1 --2- 172.21.15.163:0/1177786078 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe290159890 0x7fe290143240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.945+0000 7fe294a72700 1 --2- 172.21.15.163:0/1177786078 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe290159c60 0x7fe29015cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:19.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.945+0000 7fe294a72700 1 -- 172.21.15.163:0/1177786078 >> 172.21.15.163:0/1177786078 conn(0x7fe290068e20 msgr2=0x7fe290157230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:19.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.945+0000 7fe294a72700 1 -- 172.21.15.163:0/1177786078 shutdown_connections 2023-10-09T06:39:19.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:19.945+0000 7fe294a72700 1 -- 172.21.15.163:0/1177786078 wait complete. 2023-10-09T06:39:19.958 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-09T06:39:20.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.071+0000 7f3e5f55f700 1 Processor -- start 2023-10-09T06:39:20.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.071+0000 7f3e5f55f700 1 -- start start 2023-10-09T06:39:20.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.072+0000 7f3e5f55f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e58159890 0x7f3e5815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:20.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.072+0000 7f3e5f55f700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3e58159c60 0x7f3e5815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:20.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.072+0000 7f3e5f55f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e5815a160 con 0x7f3e5815db10 2023-10-09T06:39:20.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.072+0000 7f3e5f55f700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e580d13a0 con 0x7f3e58159c60 2023-10-09T06:39:20.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.072+0000 7f3e5f55f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5815e140 con 0x7f3e58159890 2023-10-09T06:39:20.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.072+0000 7f3e5d2fb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e58159890 0x7f3e5815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:20.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.072+0000 7f3e5cafa700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3e58159c60 0x7f3e5815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:20.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.072+0000 7f3e5d2fb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e58159890 0x7f3e5815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:48444/0 (socket says 172.21.15.163:48444) 2023-10-09T06:39:20.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.072+0000 7f3e5d2fb700 1 -- 172.21.15.163:0/954464560 learned_addr learned my addr 172.21.15.163:0/954464560 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:20.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.072+0000 7f3e5cafa700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3e58159c60 0x7f3e5815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:50518/0 (socket says 172.21.15.163:50518) 2023-10-09T06:39:20.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.072+0000 7f3e5dafc700 1 --1- 172.21.15.163:0/954464560 >> v1:172.21.15.181:6789/0 conn(0x7f3e5815db10 0x7f3e58069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:49840/0 (socket says 172.21.15.163:49840) 2023-10-09T06:39:20.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.073+0000 7f3e5d2fb700 1 -- 172.21.15.163:0/954464560 >> v2:172.21.15.181:3300/0 conn(0x7f3e58159c60 msgr2=0x7f3e5815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:20.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.073+0000 7f3e5d2fb700 1 --2- 172.21.15.163:0/954464560 >> v2:172.21.15.181:3300/0 conn(0x7f3e58159c60 0x7f3e5815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:20.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.073+0000 7f3e5d2fb700 1 -- 172.21.15.163:0/954464560 >> v1:172.21.15.181:6789/0 conn(0x7f3e5815db10 legacy=0x7f3e58069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:20.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.073+0000 7f3e5d2fb700 1 -- 172.21.15.163:0/954464560 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5815a3d0 con 0x7f3e58159890 2023-10-09T06:39:20.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.073+0000 7f3e5d2fb700 1 --2- 172.21.15.163:0/954464560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e58159890 0x7f3e5815abd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3e54009f20 tx=0x7f3e54009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad4543898797a12b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:20.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.073+0000 7f3e4ffff700 1 -- 172.21.15.163:0/954464560 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5400e070 con 0x7f3e58159890 2023-10-09T06:39:20.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.073+0000 7f3e4ffff700 1 -- 172.21.15.163:0/954464560 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e5400b9d0 con 0x7f3e58159890 2023-10-09T06:39:20.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.073+0000 7f3e4ffff700 1 -- 172.21.15.163:0/954464560 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e540092e0 con 0x7f3e58159890 2023-10-09T06:39:20.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.074+0000 7f3e5f55f700 1 -- 172.21.15.163:0/954464560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e58159890 msgr2=0x7f3e5815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:20.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.074+0000 7f3e5f55f700 1 --2- 172.21.15.163:0/954464560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e58159890 0x7f3e5815abd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3e54009f20 tx=0x7f3e54009840 comp rx=0 tx=0).stop 2023-10-09T06:39:20.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.074+0000 7f3e5f55f700 1 -- 172.21.15.163:0/954464560 shutdown_connections 2023-10-09T06:39:20.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.074+0000 7f3e5f55f700 1 --2- 172.21.15.163:0/954464560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e58159890 0x7f3e5815abd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:20.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.074+0000 7f3e5f55f700 1 --2- 172.21.15.163:0/954464560 >> v2:172.21.15.181:3300/0 conn(0x7f3e58159c60 0x7f3e5815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:20.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.074+0000 7f3e5f55f700 1 -- 172.21.15.163:0/954464560 >> 172.21.15.163:0/954464560 conn(0x7f3e58068e20 msgr2=0x7f3e580d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:20.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.074+0000 7f3e5f55f700 1 -- 172.21.15.163:0/954464560 shutdown_connections 2023-10-09T06:39:20.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.074+0000 7f3e5f55f700 1 -- 172.21.15.163:0/954464560 wait complete. 2023-10-09T06:39:20.079 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.075+0000 7f3e5f55f700 1 Processor -- start 2023-10-09T06:39:20.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.075+0000 7f3e5f55f700 1 -- start start 2023-10-09T06:39:20.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.076+0000 7f3e5f55f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e58159890 0x7f3e580c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:20.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.076+0000 7f3e5f55f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e58159c60 0x7f3e580c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:20.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.076+0000 7f3e5f55f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e5815db10 0x7f3e5815cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:20.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.076+0000 7f3e5f55f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e580b3e30 con 0x7f3e58159890 2023-10-09T06:39:20.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.076+0000 7f3e5f55f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e580c8210 con 0x7f3e5815db10 2023-10-09T06:39:20.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.076+0000 7f3e5f55f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e580c84f0 con 0x7f3e58159c60 2023-10-09T06:39:20.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.076+0000 7f3e5d2fb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e58159890 0x7f3e580c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:20.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.076+0000 7f3e5d2fb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e58159890 0x7f3e580c78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36324/0 (socket says 172.21.15.163:36324) 2023-10-09T06:39:20.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.076+0000 7f3e5d2fb700 1 -- 172.21.15.163:0/1726718842 learned_addr learned my addr 172.21.15.163:0/1726718842 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:20.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.076+0000 7f3e5cafa700 1 --2- 172.21.15.163:0/1726718842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e58159c60 0x7f3e580c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:20.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.076+0000 7f3e5dafc700 1 --2- 172.21.15.163:0/1726718842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e5815db10 0x7f3e5815cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:20.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.076+0000 7f3e5d2fb700 1 -- 172.21.15.163:0/1726718842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e58159c60 msgr2=0x7f3e580c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:20.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.076+0000 7f3e5d2fb700 1 --2- 172.21.15.163:0/1726718842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e58159c60 0x7f3e580c9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:20.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.076+0000 7f3e5d2fb700 1 -- 172.21.15.163:0/1726718842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e5815db10 msgr2=0x7f3e5815cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:20.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.076+0000 7f3e5d2fb700 1 --2- 172.21.15.163:0/1726718842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e5815db10 0x7f3e5815cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:20.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.076+0000 7f3e5d2fb700 1 -- 172.21.15.163:0/1726718842 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e54009ae0 con 0x7f3e58159890 2023-10-09T06:39:20.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.076+0000 7f3e5d2fb700 1 --2- 172.21.15.163:0/1726718842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e58159890 0x7f3e580c78a0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400b610 tx=0x7f3e54018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:20.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.077+0000 7f3e4d7fa700 1 -- 172.21.15.163:0/1726718842 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e54008a50 con 0x7f3e58159890 2023-10-09T06:39:20.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.077+0000 7f3e4d7fa700 1 -- 172.21.15.163:0/1726718842 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e54008bb0 con 0x7f3e58159890 2023-10-09T06:39:20.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.077+0000 7f3e5f55f700 1 -- 172.21.15.163:0/1726718842 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e580c87d0 con 0x7f3e58159890 2023-10-09T06:39:20.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.077+0000 7f3e4d7fa700 1 -- 172.21.15.163:0/1726718842 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e54026460 con 0x7f3e58159890 2023-10-09T06:39:20.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.077+0000 7f3e5f55f700 1 -- 172.21.15.163:0/1726718842 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e580c7de0 con 0x7f3e58159890 2023-10-09T06:39:20.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.078+0000 7f3e5f55f700 1 -- 172.21.15.163:0/1726718842 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e580b4310 con 0x7f3e58159890 2023-10-09T06:39:20.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.078+0000 7f3e4d7fa700 1 -- 172.21.15.163:0/1726718842 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3e54003040 con 0x7f3e58159890 2023-10-09T06:39:20.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.078+0000 7f3e4d7fa700 1 --2- 172.21.15.163:0/1726718842 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3e3c067320 0x7f3e3c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:20.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.079+0000 7f3e4d7fa700 1 -- 172.21.15.163:0/1726718842 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f3e54086870 con 0x7f3e58159890 2023-10-09T06:39:20.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.079+0000 7f3e5cafa700 1 --2- 172.21.15.163:0/1726718842 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3e3c067320 0x7f3e3c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:20.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.079+0000 7f3e5cafa700 1 --2- 172.21.15.163:0/1726718842 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3e3c067320 0x7f3e3c0697d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f3e580b3640 tx=0x7f3e48008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:20.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.083+0000 7f3e4d7fa700 1 -- 172.21.15.163:0/1726718842 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3e580b4310 con 0x7f3e58159890 2023-10-09T06:39:20.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.268+0000 7f3e5f55f700 1 -- 172.21.15.163:0/1726718842 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3e580b4310 con 0x7f3e58159890 2023-10-09T06:39:20.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.268+0000 7f3e4d7fa700 1 -- 172.21.15.163:0/1726718842 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v178) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3e540537f0 con 0x7f3e58159890 2023-10-09T06:39:20.269 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:20.269 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:39:20.269 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:20.270 INFO:teuthology.orchestra.run.smithi163.stdout: "poolnum": 1, 2023-10-09T06:39:20.270 INFO:teuthology.orchestra.run.smithi163.stdout: "poolname": ".mgr" 2023-10-09T06:39:20.270 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:20.270 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:39:20.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.270+0000 7f3e5f55f700 1 -- 172.21.15.163:0/1726718842 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3e3c067320 msgr2=0x7f3e3c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:20.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.270+0000 7f3e5f55f700 1 --2- 172.21.15.163:0/1726718842 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3e3c067320 0x7f3e3c0697d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f3e580b3640 tx=0x7f3e48008040 comp rx=0 tx=0).stop 2023-10-09T06:39:20.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.270+0000 7f3e5f55f700 1 -- 172.21.15.163:0/1726718842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e58159890 msgr2=0x7f3e580c78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:20.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.270+0000 7f3e5f55f700 1 --2- 172.21.15.163:0/1726718842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e58159890 0x7f3e580c78a0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400b610 tx=0x7f3e54018000 comp rx=0 tx=0).stop 2023-10-09T06:39:20.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.270+0000 7f3e5f55f700 1 -- 172.21.15.163:0/1726718842 shutdown_connections 2023-10-09T06:39:20.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.270+0000 7f3e5f55f700 1 --2- 172.21.15.163:0/1726718842 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3e3c067320 0x7f3e3c0697d0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:20.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.270+0000 7f3e5f55f700 1 --2- 172.21.15.163:0/1726718842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e58159890 0x7f3e580c78a0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:20.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.270+0000 7f3e5f55f700 1 --2- 172.21.15.163:0/1726718842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e5815db10 0x7f3e5815cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:20.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.270+0000 7f3e5f55f700 1 --2- 172.21.15.163:0/1726718842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e58159c60 0x7f3e580c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:20.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.270+0000 7f3e5f55f700 1 -- 172.21.15.163:0/1726718842 >> 172.21.15.163:0/1726718842 conn(0x7f3e58068e20 msgr2=0x7f3e580c6da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:20.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.270+0000 7f3e5f55f700 1 -- 172.21.15.163:0/1726718842 shutdown_connections 2023-10-09T06:39:20.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.270+0000 7f3e5f55f700 1 -- 172.21.15.163:0/1726718842 wait complete. 2023-10-09T06:39:20.283 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-09T06:39:20.284 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-09T06:39:20.403 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.402+0000 7f0153f55700 1 Processor -- start 2023-10-09T06:39:20.403 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.402+0000 7f0153f55700 1 -- start start 2023-10-09T06:39:20.405 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.403+0000 7f0153f55700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f014c0cd910 0x7f014c0ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:20.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.403+0000 7f0153f55700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f014c0b5b00 con 0x7f014c159ec0 2023-10-09T06:39:20.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.403+0000 7f0153f55700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f014c0b5da0 con 0x7f014c0b55a0 2023-10-09T06:39:20.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.403+0000 7f0153f55700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014c0b6080 con 0x7f014c0cd910 2023-10-09T06:39:20.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.403+0000 7f0151cf1700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f014c159ec0 0x7f014c0b4e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53120/0 (socket says 172.21.15.163:53120) 2023-10-09T06:39:20.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.403+0000 7f0151cf1700 1 -- 172.21.15.163:0/46540177 learned_addr learned my addr 172.21.15.163:0/46540177 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:20.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.403+0000 7f01514f0700 1 --2- 172.21.15.163:0/46540177 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f014c0cd910 0x7f014c0ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:20.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.403+0000 7f0150cef700 1 -- 172.21.15.163:0/46540177 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1779389667 0 0) 0x7f014c0b5b00 con 0x7f014c159ec0 2023-10-09T06:39:20.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.404+0000 7f0150cef700 1 -- 172.21.15.163:0/46540177 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0138003570 con 0x7f014c159ec0 2023-10-09T06:39:20.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.404+0000 7f0150cef700 1 -- 172.21.15.163:0/46540177 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2919058438 0 0) 0x7f014c0b5da0 con 0x7f014c0b55a0 2023-10-09T06:39:20.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.404+0000 7f0150cef700 1 -- 172.21.15.163:0/46540177 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f014c0b5b00 con 0x7f014c0b55a0 2023-10-09T06:39:20.407 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.404+0000 7f0150cef700 1 -- 172.21.15.163:0/46540177 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1441877024 0 0) 0x7f0138003570 con 0x7f014c159ec0 2023-10-09T06:39:20.407 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.404+0000 7f01514f0700 1 -- 172.21.15.163:0/46540177 >> v1:172.21.15.181:6789/0 conn(0x7f014c0b55a0 legacy=0x7f014c0ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:20.407 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.404+0000 7f01514f0700 1 -- 172.21.15.163:0/46540177 >> v1:172.21.15.163:6789/0 conn(0x7f014c159ec0 legacy=0x7f014c0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:20.407 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.404+0000 7f01514f0700 1 -- 172.21.15.163:0/46540177 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014c0cc100 con 0x7f014c0cd910 2023-10-09T06:39:20.407 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.404+0000 7f01514f0700 1 --2- 172.21.15.163:0/46540177 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f014c0cd910 0x7f014c0ce340 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f013c009910 tx=0x7f013c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24f9e3780fa2d1ea server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:20.407 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.404+0000 7f0150cef700 1 -- 172.21.15.163:0/46540177 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01480015f0 con 0x7f014c0cd910 2023-10-09T06:39:20.407 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.404+0000 7f0150cef700 1 -- 172.21.15.163:0/46540177 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013c011420 con 0x7f014c0cd910 2023-10-09T06:39:20.407 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.404+0000 7f0150cef700 1 -- 172.21.15.163:0/46540177 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c0117c0 con 0x7f014c0cd910 2023-10-09T06:39:20.407 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.405+0000 7f0153f55700 1 -- 172.21.15.163:0/46540177 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f014c0cd910 msgr2=0x7f014c0ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:20.407 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.405+0000 7f0153f55700 1 --2- 172.21.15.163:0/46540177 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f014c0cd910 0x7f014c0ce340 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f013c009910 tx=0x7f013c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:20.407 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.405+0000 7f0153f55700 1 -- 172.21.15.163:0/46540177 shutdown_connections 2023-10-09T06:39:20.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.405+0000 7f0153f55700 1 --2- 172.21.15.163:0/46540177 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f014c0cd910 0x7f014c0ce340 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:20.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.405+0000 7f0153f55700 1 -- 172.21.15.163:0/46540177 >> 172.21.15.163:0/46540177 conn(0x7f014c0999f0 msgr2=0x7f014c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:20.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.405+0000 7f0153f55700 1 -- 172.21.15.163:0/46540177 shutdown_connections 2023-10-09T06:39:20.408 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.405+0000 7f0153f55700 1 -- 172.21.15.163:0/46540177 wait complete. 2023-10-09T06:39:20.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.406+0000 7f0153f55700 1 Processor -- start 2023-10-09T06:39:20.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.406+0000 7f0153f55700 1 -- start start 2023-10-09T06:39:20.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.406+0000 7f0153f55700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f014c0b55a0 0x7f014c155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:20.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.406+0000 7f0153f55700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f014c159ec0 0x7f014c1450a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:20.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.406+0000 7f0153f55700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f014c0c8a80 0x7f014c0c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:20.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.406+0000 7f0153f55700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014c0c9460 con 0x7f014c159ec0 2023-10-09T06:39:20.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.406+0000 7f0153f55700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014c068390 con 0x7f014c0b55a0 2023-10-09T06:39:20.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.406+0000 7f0153f55700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014c068670 con 0x7f014c0c8a80 2023-10-09T06:39:20.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.407+0000 7f01514f0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f014c159ec0 0x7f014c1450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:20.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.407+0000 7f01514f0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f014c159ec0 0x7f014c1450a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36330/0 (socket says 172.21.15.163:36330) 2023-10-09T06:39:20.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.407+0000 7f01514f0700 1 -- 172.21.15.163:0/995205993 learned_addr learned my addr 172.21.15.163:0/995205993 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:20.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.407+0000 7f0151cf1700 1 --2- 172.21.15.163:0/995205993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f014c0b55a0 0x7f014c155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:20.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.407+0000 7f01524f2700 1 --2- 172.21.15.163:0/995205993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f014c0c8a80 0x7f014c0c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:20.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.407+0000 7f01514f0700 1 -- 172.21.15.163:0/995205993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f014c0c8a80 msgr2=0x7f014c0c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:20.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.407+0000 7f01514f0700 1 --2- 172.21.15.163:0/995205993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f014c0c8a80 0x7f014c0c8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:20.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.407+0000 7f01514f0700 1 -- 172.21.15.163:0/995205993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f014c0b55a0 msgr2=0x7f014c155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:20.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.407+0000 7f01514f0700 1 --2- 172.21.15.163:0/995205993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f014c0b55a0 0x7f014c155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:20.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.407+0000 7f01514f0700 1 -- 172.21.15.163:0/995205993 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f013c009420 con 0x7f014c159ec0 2023-10-09T06:39:20.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.407+0000 7f01514f0700 1 --2- 172.21.15.163:0/995205993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f014c159ec0 0x7f014c1450a0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f013c0030c0 tx=0x7f013c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:20.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.407+0000 7f0141ffb700 1 -- 172.21.15.163:0/995205993 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c01caf0 con 0x7f014c159ec0 2023-10-09T06:39:20.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.407+0000 7f0141ffb700 1 -- 172.21.15.163:0/995205993 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013c01cc50 con 0x7f014c159ec0 2023-10-09T06:39:20.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.407+0000 7f0153f55700 1 -- 172.21.15.163:0/995205993 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f014c09d6f0 con 0x7f014c159ec0 2023-10-09T06:39:20.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.407+0000 7f0141ffb700 1 -- 172.21.15.163:0/995205993 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c01a450 con 0x7f014c159ec0 2023-10-09T06:39:20.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.408+0000 7f0153f55700 1 -- 172.21.15.163:0/995205993 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f014c09dcb0 con 0x7f014c159ec0 2023-10-09T06:39:20.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.408+0000 7f0153f55700 1 -- 172.21.15.163:0/995205993 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f014c063c90 con 0x7f014c159ec0 2023-10-09T06:39:20.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.409+0000 7f0141ffb700 1 -- 172.21.15.163:0/995205993 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f013c01a840 con 0x7f014c159ec0 2023-10-09T06:39:20.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.409+0000 7f0141ffb700 1 --2- 172.21.15.163:0/995205993 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0130067320 0x7f01300697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:20.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.409+0000 7f0141ffb700 1 -- 172.21.15.163:0/995205993 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f013c086520 con 0x7f014c159ec0 2023-10-09T06:39:20.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.409+0000 7f0151cf1700 1 --2- 172.21.15.163:0/995205993 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0130067320 0x7f01300697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:20.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.410+0000 7f0151cf1700 1 --2- 172.21.15.163:0/995205993 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0130067320 0x7f01300697d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f01480017e0 tx=0x7f0148001770 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:20.415 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.414+0000 7f0141ffb700 1 -- 172.21.15.163:0/995205993 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f013c0535d0 con 0x7f014c159ec0 2023-10-09T06:39:20.594 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.593+0000 7f0153f55700 1 -- 172.21.15.163:0/995205993 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f014c0718b0 con 0x7f014c159ec0 2023-10-09T06:39:20.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.895+0000 7f0141ffb700 1 -- 172.21.15.163:0/995205993 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v179) v1 ==== 138+0+0 (secure 0 0 0) 0x7f013c086030 con 0x7f014c159ec0 2023-10-09T06:39:20.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.898+0000 7f0153f55700 1 -- 172.21.15.163:0/995205993 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0130067320 msgr2=0x7f01300697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:20.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.898+0000 7f0153f55700 1 --2- 172.21.15.163:0/995205993 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0130067320 0x7f01300697d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f01480017e0 tx=0x7f0148001770 comp rx=0 tx=0).stop 2023-10-09T06:39:20.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.898+0000 7f0153f55700 1 -- 172.21.15.163:0/995205993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f014c159ec0 msgr2=0x7f014c1450a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:20.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.898+0000 7f0153f55700 1 --2- 172.21.15.163:0/995205993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f014c159ec0 0x7f014c1450a0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f013c0030c0 tx=0x7f013c016000 comp rx=0 tx=0).stop 2023-10-09T06:39:20.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.899+0000 7f0153f55700 1 -- 172.21.15.163:0/995205993 shutdown_connections 2023-10-09T06:39:20.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.899+0000 7f0153f55700 1 --2- 172.21.15.163:0/995205993 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0130067320 0x7f01300697d0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:20.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.899+0000 7f0153f55700 1 --2- 172.21.15.163:0/995205993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f014c0c8a80 0x7f014c0c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:20.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.899+0000 7f0153f55700 1 --2- 172.21.15.163:0/995205993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f014c0b55a0 0x7f014c155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:20.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.899+0000 7f0153f55700 1 --2- 172.21.15.163:0/995205993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f014c159ec0 0x7f014c1450a0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:20.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.899+0000 7f0153f55700 1 -- 172.21.15.163:0/995205993 >> 172.21.15.163:0/995205993 conn(0x7f014c0999f0 msgr2=0x7f014c158be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:20.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.900+0000 7f0153f55700 1 -- 172.21.15.163:0/995205993 shutdown_connections 2023-10-09T06:39:20.902 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:20.900+0000 7f0153f55700 1 -- 172.21.15.163:0/995205993 wait complete. 2023-10-09T06:39:20.902 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_metadata' created 2023-10-09T06:39:20.913 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-09T06:39:21.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.035+0000 7f1529460700 1 Processor -- start 2023-10-09T06:39:21.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.035+0000 7f1529460700 1 -- start start 2023-10-09T06:39:21.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.035+0000 7f1529460700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f15240cbb90 0x7f15240af320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:21.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.035+0000 7f1529460700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15240af860 con 0x7f1524069c70 2023-10-09T06:39:21.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.035+0000 7f1529460700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15240ce380 con 0x7f1524159950 2023-10-09T06:39:21.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.035+0000 7f1529460700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15240cec20 con 0x7f15240cbb90 2023-10-09T06:39:21.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.036+0000 7f151a7fc700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f1524069c70 0x7f15240cf0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53130/0 (socket says 172.21.15.163:53130) 2023-10-09T06:39:21.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.036+0000 7f151a7fc700 1 -- 172.21.15.163:0/1444813970 learned_addr learned my addr 172.21.15.163:0/1444813970 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:21.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.036+0000 7f15237fe700 1 --2- 172.21.15.163:0/1444813970 >> v2:172.21.15.181:3300/0 conn(0x7f15240cbb90 0x7f15240af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:21.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.036+0000 7f15227fc700 1 -- 172.21.15.163:0/1444813970 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1605790491 0 0) 0x7f15240af860 con 0x7f1524069c70 2023-10-09T06:39:21.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.036+0000 7f15227fc700 1 -- 172.21.15.163:0/1444813970 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1510003570 con 0x7f1524069c70 2023-10-09T06:39:21.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.036+0000 7f15227fc700 1 -- 172.21.15.163:0/1444813970 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 473392264 0 0) 0x7f1510003570 con 0x7f1524069c70 2023-10-09T06:39:21.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.036+0000 7f15227fc700 1 -- 172.21.15.163:0/1444813970 >> v2:172.21.15.181:3300/0 conn(0x7f15240cbb90 msgr2=0x7f15240af320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:21.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.036+0000 7f15227fc700 1 --2- 172.21.15.163:0/1444813970 >> v2:172.21.15.181:3300/0 conn(0x7f15240cbb90 0x7f15240af320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:21.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.036+0000 7f15227fc700 1 -- 172.21.15.163:0/1444813970 >> v1:172.21.15.181:6789/0 conn(0x7f1524159950 legacy=0x7f1524069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:21.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.036+0000 7f15227fc700 1 -- 172.21.15.163:0/1444813970 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15240cbfb0 con 0x7f1524069c70 2023-10-09T06:39:21.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.036+0000 7f15227fc700 1 -- 172.21.15.163:0/1444813970 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f150c001470 con 0x7f1524069c70 2023-10-09T06:39:21.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.036+0000 7f15227fc700 1 -- 172.21.15.163:0/1444813970 >> v1:172.21.15.163:6789/0 conn(0x7f1524069c70 legacy=0x7f15240cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:21.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.036+0000 7f15227fc700 1 --2- 172.21.15.163:0/1444813970 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1510003a60 0x7f1510003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:21.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.036+0000 7f15227fc700 1 --2- 172.21.15.163:0/1444813970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1510004810 0x7f1510004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:21.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.036+0000 7f15227fc700 1 --2- 172.21.15.163:0/1444813970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15100051c0 0x7f151000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:21.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.036+0000 7f15227fc700 1 -- 172.21.15.163:0/1444813970 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15240cec20 con 0x7f1510004810 2023-10-09T06:39:21.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.036+0000 7f15227fc700 1 -- 172.21.15.163:0/1444813970 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151000c160 con 0x7f15100051c0 2023-10-09T06:39:21.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.036+0000 7f15227fc700 1 -- 172.21.15.163:0/1444813970 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151000c320 con 0x7f1510003a60 2023-10-09T06:39:21.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.036+0000 7f151a7fc700 1 --2- 172.21.15.163:0/1444813970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1510004810 0x7f1510004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:21.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.036+0000 7f15237fe700 1 --2- 172.21.15.163:0/1444813970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15100051c0 0x7f151000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:21.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.036+0000 7f1522ffd700 1 --2- 172.21.15.163:0/1444813970 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1510003a60 0x7f1510003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:21.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.037+0000 7f151a7fc700 1 -- 172.21.15.163:0/1444813970 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1510003a60 msgr2=0x7f1510003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:21.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.037+0000 7f151a7fc700 1 --2- 172.21.15.163:0/1444813970 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1510003a60 0x7f1510003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:21.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.037+0000 7f151a7fc700 1 -- 172.21.15.163:0/1444813970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15100051c0 msgr2=0x7f151000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:21.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.037+0000 7f151a7fc700 1 --2- 172.21.15.163:0/1444813970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15100051c0 0x7f151000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:21.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.037+0000 7f151a7fc700 1 -- 172.21.15.163:0/1444813970 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1510003570 con 0x7f1510004810 2023-10-09T06:39:21.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.037+0000 7f15237fe700 1 -- 172.21.15.163:0/1444813970 reap_dead start 2023-10-09T06:39:21.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.037+0000 7f151a7fc700 1 --2- 172.21.15.163:0/1444813970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1510004810 0x7f1510004c80 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f150c00c340 tx=0x7f15240ced60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83733a9f7650e2fc server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:21.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.037+0000 7f15227fc700 1 -- 172.21.15.163:0/1444813970 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c00cbf0 con 0x7f1510004810 2023-10-09T06:39:21.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.037+0000 7f15227fc700 1 -- 172.21.15.163:0/1444813970 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150c00cd70 con 0x7f1510004810 2023-10-09T06:39:21.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.037+0000 7f15227fc700 1 -- 172.21.15.163:0/1444813970 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c005b70 con 0x7f1510004810 2023-10-09T06:39:21.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.037+0000 7f1529460700 1 -- 172.21.15.163:0/1444813970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1510004810 msgr2=0x7f1510004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:21.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.037+0000 7f1529460700 1 --2- 172.21.15.163:0/1444813970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1510004810 0x7f1510004c80 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f150c00c340 tx=0x7f15240ced60 comp rx=0 tx=0).stop 2023-10-09T06:39:21.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.038+0000 7f1529460700 1 -- 172.21.15.163:0/1444813970 shutdown_connections 2023-10-09T06:39:21.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.038+0000 7f1529460700 1 --2- 172.21.15.163:0/1444813970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1510004810 0x7f1510004c80 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:21.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.038+0000 7f1529460700 1 -- 172.21.15.163:0/1444813970 >> 172.21.15.163:0/1444813970 conn(0x7f1524068e00 msgr2=0x7f15240d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:21.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.038+0000 7f1529460700 1 -- 172.21.15.163:0/1444813970 shutdown_connections 2023-10-09T06:39:21.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.038+0000 7f1529460700 1 -- 172.21.15.163:0/1444813970 wait complete. 2023-10-09T06:39:21.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.039+0000 7f1529460700 1 Processor -- start 2023-10-09T06:39:21.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.039+0000 7f1529460700 1 -- start start 2023-10-09T06:39:21.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.040+0000 7f1529460700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1510004810 0x7f15240ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:21.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.040+0000 7f1529460700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1524157580 0x7f152414c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:21.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.040+0000 7f1529460700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f152415c120 0x7f152415c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:21.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.040+0000 7f1529460700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152415cae0 con 0x7f1524157580 2023-10-09T06:39:21.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.040+0000 7f1529460700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152415b500 con 0x7f1510004810 2023-10-09T06:39:21.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.040+0000 7f1529460700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f152415a6d0 con 0x7f152415c120 2023-10-09T06:39:21.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.040+0000 7f151a7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1524157580 0x7f152414c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:21.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.040+0000 7f151a7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1524157580 0x7f152414c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36350/0 (socket says 172.21.15.163:36350) 2023-10-09T06:39:21.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.040+0000 7f151a7fc700 1 -- 172.21.15.163:0/1102275813 learned_addr learned my addr 172.21.15.163:0/1102275813 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:21.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.040+0000 7f1522ffd700 1 --2- 172.21.15.163:0/1102275813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1510004810 0x7f15240ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:21.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.040+0000 7f15237fe700 1 --2- 172.21.15.163:0/1102275813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f152415c120 0x7f152415c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:21.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.040+0000 7f151a7fc700 1 -- 172.21.15.163:0/1102275813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f152415c120 msgr2=0x7f152415c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:21.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.040+0000 7f151a7fc700 1 --2- 172.21.15.163:0/1102275813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f152415c120 0x7f152415c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:21.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.040+0000 7f151a7fc700 1 -- 172.21.15.163:0/1102275813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1510004810 msgr2=0x7f15240ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:21.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.040+0000 7f151a7fc700 1 --2- 172.21.15.163:0/1102275813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1510004810 0x7f15240ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:21.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.040+0000 7f151a7fc700 1 -- 172.21.15.163:0/1102275813 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f152415a980 con 0x7f1524157580 2023-10-09T06:39:21.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.040+0000 7f151a7fc700 1 --2- 172.21.15.163:0/1102275813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1524157580 0x7f152414c160 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f150c00c930 tx=0x7f150c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:21.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.040+0000 7f151b7fe700 1 -- 172.21.15.163:0/1102275813 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c00cee0 con 0x7f1524157580 2023-10-09T06:39:21.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.040+0000 7f151b7fe700 1 -- 172.21.15.163:0/1102275813 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150c00cbf0 con 0x7f1524157580 2023-10-09T06:39:21.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.040+0000 7f1529460700 1 -- 172.21.15.163:0/1102275813 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f152415ac60 con 0x7f1524157580 2023-10-09T06:39:21.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.040+0000 7f151b7fe700 1 -- 172.21.15.163:0/1102275813 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c0072e0 con 0x7f1524157580 2023-10-09T06:39:21.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.040+0000 7f1529460700 1 -- 172.21.15.163:0/1102275813 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f152415b220 con 0x7f1524157580 2023-10-09T06:39:21.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.041+0000 7f1529460700 1 -- 172.21.15.163:0/1102275813 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15240718b0 con 0x7f1524157580 2023-10-09T06:39:21.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.042+0000 7f151b7fe700 1 -- 172.21.15.163:0/1102275813 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f150c007690 con 0x7f1524157580 2023-10-09T06:39:21.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.042+0000 7f151b7fe700 1 --2- 172.21.15.163:0/1102275813 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1504067320 0x7f15040697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:21.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.042+0000 7f151b7fe700 1 -- 172.21.15.163:0/1102275813 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f150c085100 con 0x7f1524157580 2023-10-09T06:39:21.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.043+0000 7f1522ffd700 1 --2- 172.21.15.163:0/1102275813 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1504067320 0x7f15040697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:21.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.043+0000 7f1522ffd700 1 --2- 172.21.15.163:0/1102275813 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1504067320 0x7f15040697d0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1524158ee0 tx=0x7f1514007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:21.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.047+0000 7f151b7fe700 1 -- 172.21.15.163:0/1102275813 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f150c051f70 con 0x7f1524157580 2023-10-09T06:39:21.229 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.228+0000 7f1529460700 1 -- 172.21.15.163:0/1102275813 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f15240afa90 con 0x7f1524157580 2023-10-09T06:39:21.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.893+0000 7f151b7fe700 1 -- 172.21.15.163:0/1102275813 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v180) v1 ==== 170+0+0 (secure 0 0 0) 0x7f150c055590 con 0x7f1524157580 2023-10-09T06:39:21.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.897+0000 7f1529460700 1 -- 172.21.15.163:0/1102275813 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1504067320 msgr2=0x7f15040697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:21.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.897+0000 7f1529460700 1 --2- 172.21.15.163:0/1102275813 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1504067320 0x7f15040697d0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1524158ee0 tx=0x7f1514007d30 comp rx=0 tx=0).stop 2023-10-09T06:39:21.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.897+0000 7f1529460700 1 -- 172.21.15.163:0/1102275813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1524157580 msgr2=0x7f152414c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:21.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.897+0000 7f1529460700 1 --2- 172.21.15.163:0/1102275813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1524157580 0x7f152414c160 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f150c00c930 tx=0x7f150c012000 comp rx=0 tx=0).stop 2023-10-09T06:39:21.899 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.897+0000 7f1529460700 1 -- 172.21.15.163:0/1102275813 shutdown_connections 2023-10-09T06:39:21.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.897+0000 7f1529460700 1 --2- 172.21.15.163:0/1102275813 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1504067320 0x7f15040697d0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:21.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.897+0000 7f1529460700 1 --2- 172.21.15.163:0/1102275813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f152415c120 0x7f152415c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:21.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.897+0000 7f1529460700 1 --2- 172.21.15.163:0/1102275813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1510004810 0x7f15240ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:21.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.897+0000 7f1529460700 1 --2- 172.21.15.163:0/1102275813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1524157580 0x7f152414c160 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:21.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.897+0000 7f1529460700 1 -- 172.21.15.163:0/1102275813 >> 172.21.15.163:0/1102275813 conn(0x7f1524068e00 msgr2=0x7f15240cadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:21.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.898+0000 7f1529460700 1 -- 172.21.15.163:0/1102275813 shutdown_connections 2023-10-09T06:39:21.900 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:21.898+0000 7f1529460700 1 -- 172.21.15.163:0/1102275813 wait complete. 2023-10-09T06:39:21.903 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_data' created 2023-10-09T06:39:21.920 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-09T06:39:22.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.037+0000 7f1364621700 1 Processor -- start 2023-10-09T06:39:22.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.037+0000 7f1364621700 1 -- start start 2023-10-09T06:39:22.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.038+0000 7f1364621700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f135c159890 0x7f135c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:22.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.038+0000 7f1364621700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f135c159c60 0x7f135c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:22.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.038+0000 7f1364621700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f135c15a160 con 0x7f135c15db10 2023-10-09T06:39:22.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.038+0000 7f1364621700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f135c15df50 con 0x7f135c159890 2023-10-09T06:39:22.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.038+0000 7f1364621700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f135c0d13a0 con 0x7f135c159c60 2023-10-09T06:39:22.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.038+0000 7f13623bd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f135c159890 0x7f135c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:22.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.038+0000 7f1362bbe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f135c15db10 0x7f135c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53136/0 (socket says 172.21.15.163:53136) 2023-10-09T06:39:22.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.038+0000 7f13623bd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f135c159890 0x7f135c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36360/0 (socket says 172.21.15.163:36360) 2023-10-09T06:39:22.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.038+0000 7f1362bbe700 1 -- 172.21.15.163:0/2712707676 learned_addr learned my addr 172.21.15.163:0/2712707676 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:22.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.038+0000 7f1361bbc700 1 --2- 172.21.15.163:0/2712707676 >> v2:172.21.15.181:3300/0 conn(0x7f135c159c60 0x7f135c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:22.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.038+0000 7f13623bd700 1 -- 172.21.15.163:0/2712707676 >> v2:172.21.15.181:3300/0 conn(0x7f135c159c60 msgr2=0x7f135c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:22.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.038+0000 7f13623bd700 1 --2- 172.21.15.163:0/2712707676 >> v2:172.21.15.181:3300/0 conn(0x7f135c159c60 0x7f135c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:22.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.038+0000 7f13623bd700 1 -- 172.21.15.163:0/2712707676 >> v1:172.21.15.163:6789/0 conn(0x7f135c15db10 legacy=0x7f135c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:22.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.038+0000 7f13623bd700 1 -- 172.21.15.163:0/2712707676 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f135c15e140 con 0x7f135c159890 2023-10-09T06:39:22.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.038+0000 7f13613bb700 1 -- 172.21.15.163:0/2712707676 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3730336389 0 0) 0x7f135c15a160 con 0x7f135c15db10 2023-10-09T06:39:22.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.039+0000 7f13623bd700 1 --2- 172.21.15.163:0/2712707676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f135c159890 0x7f135c15abd0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f135c0b67f0 tx=0x7f1358009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5582d913ace95a18 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:22.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.039+0000 7f13613bb700 1 -- 172.21.15.163:0/2712707676 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135800e070 con 0x7f135c159890 2023-10-09T06:39:22.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.039+0000 7f13613bb700 1 -- 172.21.15.163:0/2712707676 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f135800bc30 con 0x7f135c159890 2023-10-09T06:39:22.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.039+0000 7f13613bb700 1 -- 172.21.15.163:0/2712707676 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1358012430 con 0x7f135c159890 2023-10-09T06:39:22.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.039+0000 7f1364621700 1 -- 172.21.15.163:0/2712707676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f135c159890 msgr2=0x7f135c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:22.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.039+0000 7f1364621700 1 --2- 172.21.15.163:0/2712707676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f135c159890 0x7f135c15abd0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f135c0b67f0 tx=0x7f1358009780 comp rx=0 tx=0).stop 2023-10-09T06:39:22.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.039+0000 7f1364621700 1 -- 172.21.15.163:0/2712707676 shutdown_connections 2023-10-09T06:39:22.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.039+0000 7f1364621700 1 --2- 172.21.15.163:0/2712707676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f135c159890 0x7f135c15abd0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:22.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.039+0000 7f1364621700 1 --2- 172.21.15.163:0/2712707676 >> v2:172.21.15.181:3300/0 conn(0x7f135c159c60 0x7f135c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:22.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.039+0000 7f1364621700 1 -- 172.21.15.163:0/2712707676 >> 172.21.15.163:0/2712707676 conn(0x7f135c068e20 msgr2=0x7f135c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:22.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.039+0000 7f1364621700 1 -- 172.21.15.163:0/2712707676 shutdown_connections 2023-10-09T06:39:22.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.039+0000 7f1364621700 1 -- 172.21.15.163:0/2712707676 wait complete. 2023-10-09T06:39:22.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.040+0000 7f1364621700 1 Processor -- start 2023-10-09T06:39:22.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.040+0000 7f1364621700 1 -- start start 2023-10-09T06:39:22.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.041+0000 7f1364621700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f135c159890 0x7f135c14c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:22.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.041+0000 7f1364621700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f135c159c60 0x7f135c0cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:22.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.041+0000 7f1364621700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f135c15db10 0x7f135c1522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:22.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.041+0000 7f1364621700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135c0cdfb0 con 0x7f135c15db10 2023-10-09T06:39:22.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.041+0000 7f1364621700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135c14cf10 con 0x7f135c159c60 2023-10-09T06:39:22.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.041+0000 7f1364621700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f135c0cd0c0 con 0x7f135c159890 2023-10-09T06:39:22.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.041+0000 7f1362bbe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f135c15db10 0x7f135c1522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:22.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.041+0000 7f1362bbe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f135c15db10 0x7f135c1522a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36366/0 (socket says 172.21.15.163:36366) 2023-10-09T06:39:22.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.041+0000 7f1362bbe700 1 -- 172.21.15.163:0/549074103 learned_addr learned my addr 172.21.15.163:0/549074103 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:22.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.041+0000 7f1361bbc700 1 --2- 172.21.15.163:0/549074103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f135c159c60 0x7f135c0cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:22.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.041+0000 7f13623bd700 1 --2- 172.21.15.163:0/549074103 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f135c159890 0x7f135c14c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:22.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.041+0000 7f1362bbe700 1 -- 172.21.15.163:0/549074103 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f135c159890 msgr2=0x7f135c14c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:22.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.041+0000 7f1362bbe700 1 --2- 172.21.15.163:0/549074103 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f135c159890 0x7f135c14c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:22.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.041+0000 7f1362bbe700 1 -- 172.21.15.163:0/549074103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f135c159c60 msgr2=0x7f135c0cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:22.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.041+0000 7f1362bbe700 1 --2- 172.21.15.163:0/549074103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f135c159c60 0x7f135c0cf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:22.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.041+0000 7f1362bbe700 1 -- 172.21.15.163:0/549074103 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1358009a20 con 0x7f135c15db10 2023-10-09T06:39:22.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.041+0000 7f1361bbc700 1 --2- 172.21.15.163:0/549074103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f135c159c60 0x7f135c0cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:22.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.042+0000 7f1362bbe700 1 --2- 172.21.15.163:0/549074103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f135c15db10 0x7f135c1522a0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f135400c530 tx=0x7f1354007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:22.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.042+0000 7f13527fc700 1 -- 172.21.15.163:0/549074103 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1354005b30 con 0x7f135c15db10 2023-10-09T06:39:22.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.042+0000 7f13527fc700 1 -- 172.21.15.163:0/549074103 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1354005c90 con 0x7f135c15db10 2023-10-09T06:39:22.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.042+0000 7f13527fc700 1 -- 172.21.15.163:0/549074103 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1354007320 con 0x7f135c15db10 2023-10-09T06:39:22.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.042+0000 7f1364621700 1 -- 172.21.15.163:0/549074103 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f135c0cd3a0 con 0x7f135c15db10 2023-10-09T06:39:22.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.042+0000 7f1364621700 1 -- 172.21.15.163:0/549074103 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f135c0cd960 con 0x7f135c15db10 2023-10-09T06:39:22.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.043+0000 7f1364621700 1 -- 172.21.15.163:0/549074103 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f135c04c3c0 con 0x7f135c15db10 2023-10-09T06:39:22.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.043+0000 7f13527fc700 1 -- 172.21.15.163:0/549074103 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f135401b3b0 con 0x7f135c15db10 2023-10-09T06:39:22.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.044+0000 7f13527fc700 1 --2- 172.21.15.163:0/549074103 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1340067320 0x7f13400697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:22.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.044+0000 7f13527fc700 1 -- 172.21.15.163:0/549074103 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f1354084130 con 0x7f135c15db10 2023-10-09T06:39:22.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.044+0000 7f13623bd700 1 --2- 172.21.15.163:0/549074103 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1340067320 0x7f13400697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:22.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.044+0000 7f13623bd700 1 --2- 172.21.15.163:0/549074103 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1340067320 0x7f13400697d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f135c0b67f0 tx=0x7f1358019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:22.049 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.048+0000 7f13527fc700 1 -- 172.21.15.163:0/549074103 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1354050e20 con 0x7f135c15db10 2023-10-09T06:39:22.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.271+0000 7f1364621700 1 -- 172.21.15.163:0/549074103 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f135c0ce0f0 con 0x7f135c15db10 2023-10-09T06:39:22.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.926+0000 7f13527fc700 1 -- 172.21.15.163:0/549074103 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 31 and data pool 32 v119) v1 ==== 174+0+0 (secure 0 0 0) 0x7f1354054440 con 0x7f135c15db10 2023-10-09T06:39:22.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.930+0000 7f1364621700 1 -- 172.21.15.163:0/549074103 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1340067320 msgr2=0x7f13400697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:22.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.930+0000 7f1364621700 1 --2- 172.21.15.163:0/549074103 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1340067320 0x7f13400697d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f135c0b67f0 tx=0x7f1358019000 comp rx=0 tx=0).stop 2023-10-09T06:39:22.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.930+0000 7f1364621700 1 -- 172.21.15.163:0/549074103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f135c15db10 msgr2=0x7f135c1522a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:22.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.930+0000 7f1364621700 1 --2- 172.21.15.163:0/549074103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f135c15db10 0x7f135c1522a0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f135400c530 tx=0x7f1354007f90 comp rx=0 tx=0).stop 2023-10-09T06:39:22.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.930+0000 7f1364621700 1 -- 172.21.15.163:0/549074103 shutdown_connections 2023-10-09T06:39:22.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.930+0000 7f1364621700 1 --2- 172.21.15.163:0/549074103 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1340067320 0x7f13400697d0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:22.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.930+0000 7f1364621700 1 --2- 172.21.15.163:0/549074103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f135c15db10 0x7f135c1522a0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:22.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.930+0000 7f1364621700 1 --2- 172.21.15.163:0/549074103 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f135c159890 0x7f135c14c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:22.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.930+0000 7f1364621700 1 --2- 172.21.15.163:0/549074103 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f135c159c60 0x7f135c0cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:22.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.930+0000 7f1364621700 1 -- 172.21.15.163:0/549074103 >> 172.21.15.163:0/549074103 conn(0x7f135c068e20 msgr2=0x7f135c153720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:22.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.930+0000 7f1364621700 1 -- 172.21.15.163:0/549074103 shutdown_connections 2023-10-09T06:39:22.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:22.930+0000 7f1364621700 1 -- 172.21.15.163:0/549074103 wait complete. 2023-10-09T06:39:22.935 INFO:teuthology.orchestra.run.smithi163.stderr:new fs with metadata pool 31 and data pool 32 2023-10-09T06:39:22.947 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:39:23.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.066+0000 7fbb6a142700 1 Processor -- start 2023-10-09T06:39:23.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.066+0000 7fbb6a142700 1 -- start start 2023-10-09T06:39:23.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.067+0000 7fbb6a142700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbb64159c60 0x7fbb6415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:23.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.067+0000 7fbb6a142700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb6415a160 con 0x7fbb6415db10 2023-10-09T06:39:23.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.067+0000 7fbb6a142700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb640d13a0 con 0x7fbb64159890 2023-10-09T06:39:23.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.067+0000 7fbb6a142700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb6415df50 con 0x7fbb64159c60 2023-10-09T06:39:23.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.067+0000 7fbb62ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbb64159c60 0x7fbb6415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:23.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.067+0000 7fbb62ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbb64159c60 0x7fbb6415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36370/0 (socket says 172.21.15.163:36370) 2023-10-09T06:39:23.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.067+0000 7fbb62ffd700 1 -- 172.21.15.163:0/3811018108 learned_addr learned my addr 172.21.15.163:0/3811018108 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:23.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.067+0000 7fbb63fff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fbb6415db10 0x7fbb64069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53138/0 (socket says 172.21.15.163:53138) 2023-10-09T06:39:23.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.068+0000 7fbb62ffd700 1 -- 172.21.15.163:0/3811018108 >> v1:172.21.15.181:6789/0 conn(0x7fbb64159890 legacy=0x7fbb6415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:23.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.068+0000 7fbb62ffd700 1 -- 172.21.15.163:0/3811018108 >> v1:172.21.15.163:6789/0 conn(0x7fbb6415db10 legacy=0x7fbb64069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:23.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.068+0000 7fbb62ffd700 1 -- 172.21.15.163:0/3811018108 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6415e140 con 0x7fbb64159c60 2023-10-09T06:39:23.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.068+0000 7fbb62ffd700 1 --2- 172.21.15.163:0/3811018108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb64159c60 0x7fbb6415d5d0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fbb580099b0 tx=0x7fbb58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12860db7f3c94ca server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:23.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.068+0000 7fbb627fc700 1 -- 172.21.15.163:0/3811018108 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5800b860 con 0x7fbb64159c60 2023-10-09T06:39:23.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.068+0000 7fbb627fc700 1 -- 172.21.15.163:0/3811018108 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5800b9c0 con 0x7fbb64159c60 2023-10-09T06:39:23.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.068+0000 7fbb627fc700 1 -- 172.21.15.163:0/3811018108 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5800bd60 con 0x7fbb64159c60 2023-10-09T06:39:23.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.068+0000 7fbb6a142700 1 -- 172.21.15.163:0/3811018108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb64159c60 msgr2=0x7fbb6415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:23.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.068+0000 7fbb6a142700 1 --2- 172.21.15.163:0/3811018108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb64159c60 0x7fbb6415d5d0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fbb580099b0 tx=0x7fbb58009250 comp rx=0 tx=0).stop 2023-10-09T06:39:23.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.068+0000 7fbb6a142700 1 -- 172.21.15.163:0/3811018108 shutdown_connections 2023-10-09T06:39:23.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.068+0000 7fbb6a142700 1 --2- 172.21.15.163:0/3811018108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb64159c60 0x7fbb6415d5d0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:23.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.068+0000 7fbb6a142700 1 -- 172.21.15.163:0/3811018108 >> 172.21.15.163:0/3811018108 conn(0x7fbb64068e20 msgr2=0x7fbb640d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:23.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.068+0000 7fbb6a142700 1 -- 172.21.15.163:0/3811018108 shutdown_connections 2023-10-09T06:39:23.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.069+0000 7fbb6a142700 1 -- 172.21.15.163:0/3811018108 wait complete. 2023-10-09T06:39:23.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.070+0000 7fbb6a142700 1 Processor -- start 2023-10-09T06:39:23.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.070+0000 7fbb6a142700 1 -- start start 2023-10-09T06:39:23.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.070+0000 7fbb6a142700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb64159890 0x7fbb640c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:23.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.070+0000 7fbb6a142700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb64159c60 0x7fbb64155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:23.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.070+0000 7fbb6a142700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb6415db10 0x7fbb640c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:23.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.070+0000 7fbb6a142700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb640b3630 con 0x7fbb64159890 2023-10-09T06:39:23.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.070+0000 7fbb6a142700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb640b3910 con 0x7fbb6415db10 2023-10-09T06:39:23.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.070+0000 7fbb6a142700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb640b3bf0 con 0x7fbb64159c60 2023-10-09T06:39:23.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.071+0000 7fbb637fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb64159890 0x7fbb640c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:23.078 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.071+0000 7fbb637fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb64159890 0x7fbb640c7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36386/0 (socket says 172.21.15.163:36386) 2023-10-09T06:39:23.078 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.071+0000 7fbb637fe700 1 -- 172.21.15.163:0/2832921937 learned_addr learned my addr 172.21.15.163:0/2832921937 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:23.079 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.071+0000 7fbb63fff700 1 --2- 172.21.15.163:0/2832921937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb6415db10 0x7fbb640c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:23.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.071+0000 7fbb62ffd700 1 --2- 172.21.15.163:0/2832921937 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb64159c60 0x7fbb64155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:23.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.071+0000 7fbb637fe700 1 -- 172.21.15.163:0/2832921937 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb64159c60 msgr2=0x7fbb64155c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:23.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.071+0000 7fbb637fe700 1 --2- 172.21.15.163:0/2832921937 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb64159c60 0x7fbb64155c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:23.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.071+0000 7fbb637fe700 1 -- 172.21.15.163:0/2832921937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb6415db10 msgr2=0x7fbb640c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:23.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.071+0000 7fbb637fe700 1 --2- 172.21.15.163:0/2832921937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb6415db10 0x7fbb640c9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:23.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.071+0000 7fbb637fe700 1 -- 172.21.15.163:0/2832921937 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb580094f0 con 0x7fbb64159890 2023-10-09T06:39:23.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.071+0000 7fbb62ffd700 1 --2- 172.21.15.163:0/2832921937 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb64159c60 0x7fbb64155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:23.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.071+0000 7fbb637fe700 1 --2- 172.21.15.163:0/2832921937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb64159890 0x7fbb640c7af0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fbb5400c590 tx=0x7fbb5400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:23.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.071+0000 7fbb4b7fe700 1 -- 172.21.15.163:0/2832921937 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb54011590 con 0x7fbb64159890 2023-10-09T06:39:23.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.071+0000 7fbb4b7fe700 1 -- 172.21.15.163:0/2832921937 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb540116f0 con 0x7fbb64159890 2023-10-09T06:39:23.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.071+0000 7fbb6a142700 1 -- 172.21.15.163:0/2832921937 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb640b3ed0 con 0x7fbb64159890 2023-10-09T06:39:23.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.071+0000 7fbb4b7fe700 1 -- 172.21.15.163:0/2832921937 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb54011ab0 con 0x7fbb64159890 2023-10-09T06:39:23.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.071+0000 7fbb6a142700 1 -- 172.21.15.163:0/2832921937 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb640c8610 con 0x7fbb64159890 2023-10-09T06:39:23.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.072+0000 7fbb6a142700 1 -- 172.21.15.163:0/2832921937 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb6415aad0 con 0x7fbb64159890 2023-10-09T06:39:23.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.072+0000 7fbb4b7fe700 1 -- 172.21.15.163:0/2832921937 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbb5401b3b0 con 0x7fbb64159890 2023-10-09T06:39:23.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.073+0000 7fbb4b7fe700 1 --2- 172.21.15.163:0/2832921937 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb50067320 0x7fbb500697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:23.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.073+0000 7fbb4b7fe700 1 -- 172.21.15.163:0/2832921937 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbb540842b0 con 0x7fbb64159890 2023-10-09T06:39:23.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.073+0000 7fbb62ffd700 1 --2- 172.21.15.163:0/2832921937 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb50067320 0x7fbb500697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:23.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.073+0000 7fbb62ffd700 1 --2- 172.21.15.163:0/2832921937 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb50067320 0x7fbb500697d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fbb580099b0 tx=0x7fbb58017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:23.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.077+0000 7fbb4b7fe700 1 -- 172.21.15.163:0/2832921937 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb54050f40 con 0x7fbb64159890 2023-10-09T06:39:23.261 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.260+0000 7fbb6a142700 1 -- 172.21.15.163:0/2832921937 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb6404c3c0 con 0x7fbb64159890 2023-10-09T06:39:23.262 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.261+0000 7fbb4b7fe700 1 -- 172.21.15.163:0/2832921937 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v181) v1 ==== 74+0+15425 (secure 0 0 0) 0x7fbb54054560 con 0x7fbb64159890 2023-10-09T06:39:23.262 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:23.262 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":181,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:39:22.273307+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:39:20.612285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-09T06:39:21.247510+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:39:23.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.263+0000 7fbb6a142700 1 -- 172.21.15.163:0/2832921937 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb50067320 msgr2=0x7fbb500697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:23.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.263+0000 7fbb6a142700 1 --2- 172.21.15.163:0/2832921937 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb50067320 0x7fbb500697d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fbb580099b0 tx=0x7fbb58017000 comp rx=0 tx=0).stop 2023-10-09T06:39:23.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.263+0000 7fbb6a142700 1 -- 172.21.15.163:0/2832921937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb64159890 msgr2=0x7fbb640c7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:23.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.263+0000 7fbb6a142700 1 --2- 172.21.15.163:0/2832921937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb64159890 0x7fbb640c7af0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fbb5400c590 tx=0x7fbb5400c950 comp rx=0 tx=0).stop 2023-10-09T06:39:23.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.263+0000 7fbb6a142700 1 -- 172.21.15.163:0/2832921937 shutdown_connections 2023-10-09T06:39:23.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.263+0000 7fbb6a142700 1 --2- 172.21.15.163:0/2832921937 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbb50067320 0x7fbb500697d0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:23.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.263+0000 7fbb6a142700 1 --2- 172.21.15.163:0/2832921937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb64159890 0x7fbb640c7af0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:23.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.263+0000 7fbb6a142700 1 --2- 172.21.15.163:0/2832921937 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb6415db10 0x7fbb640c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:23.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.263+0000 7fbb6a142700 1 --2- 172.21.15.163:0/2832921937 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb64159c60 0x7fbb64155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:23.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.263+0000 7fbb6a142700 1 -- 172.21.15.163:0/2832921937 >> 172.21.15.163:0/2832921937 conn(0x7fbb64068e20 msgr2=0x7fbb640b1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:23.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.263+0000 7fbb6a142700 1 -- 172.21.15.163:0/2832921937 shutdown_connections 2023-10-09T06:39:23.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.263+0000 7fbb6a142700 1 -- 172.21.15.163:0/2832921937 wait complete. 2023-10-09T06:39:23.277 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:39:23.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.390+0000 7f0ec0d5c700 1 Processor -- start 2023-10-09T06:39:23.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.390+0000 7f0ec0d5c700 1 -- start start 2023-10-09T06:39:23.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.391+0000 7f0ec0d5c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0ebc06bcd0 0x7f0ebc0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:23.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.391+0000 7f0ec0d5c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ebc06c0a0 0x7f0ebc15a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:23.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.391+0000 7f0ec0d5c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ebc15d7e0 con 0x7f0ebc15df50 2023-10-09T06:39:23.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.391+0000 7f0ec0d5c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ebc15dac0 con 0x7f0ebc06bcd0 2023-10-09T06:39:23.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.391+0000 7f0ec0d5c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc15aa30 con 0x7f0ebc06c0a0 2023-10-09T06:39:23.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.392+0000 7f0eba59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0ebc06bcd0 0x7f0ebc0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:23.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.392+0000 7f0eb1d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ebc06c0a0 0x7f0ebc15a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:23.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.392+0000 7f0ebad9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f0ebc15df50 0x7f0ebc0c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:49882/0 (socket says 172.21.15.163:49882) 2023-10-09T06:39:23.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.392+0000 7f0eb1d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ebc06c0a0 0x7f0ebc15a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:48528/0 (socket says 172.21.15.163:48528) 2023-10-09T06:39:23.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.392+0000 7f0eba59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0ebc06bcd0 0x7f0ebc0caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:50614/0 (socket says 172.21.15.163:50614) 2023-10-09T06:39:23.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.392+0000 7f0ebad9d700 1 -- 172.21.15.163:0/1316199626 learned_addr learned my addr 172.21.15.163:0/1316199626 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:23.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.392+0000 7f0eba59c700 1 -- 172.21.15.163:0/1316199626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ebc06c0a0 msgr2=0x7f0ebc15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:23.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.392+0000 7f0eba59c700 1 --2- 172.21.15.163:0/1316199626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ebc06c0a0 0x7f0ebc15a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:23.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.392+0000 7f0eba59c700 1 -- 172.21.15.163:0/1316199626 >> v1:172.21.15.181:6789/0 conn(0x7f0ebc15df50 legacy=0x7f0ebc0c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:23.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.392+0000 7f0eb9d9b700 1 -- 172.21.15.163:0/1316199626 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2769037135 0 0) 0x7f0ebc15d7e0 con 0x7f0ebc15df50 2023-10-09T06:39:23.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.392+0000 7f0eba59c700 1 -- 172.21.15.163:0/1316199626 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ebc15ad10 con 0x7f0ebc06bcd0 2023-10-09T06:39:23.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.392+0000 7f0eba59c700 1 --2- 172.21.15.163:0/1316199626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ebc06bcd0 0x7f0ebc0caa90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0eac0098e0 tx=0x7f0eac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa17da7b5fb515dd server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:23.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.392+0000 7f0eb9d9b700 1 -- 172.21.15.163:0/1316199626 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac00b640 con 0x7f0ebc06bcd0 2023-10-09T06:39:23.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.392+0000 7f0eb9d9b700 1 -- 172.21.15.163:0/1316199626 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eac00b7a0 con 0x7f0ebc06bcd0 2023-10-09T06:39:23.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.392+0000 7f0eb9d9b700 1 -- 172.21.15.163:0/1316199626 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac00bb40 con 0x7f0ebc06bcd0 2023-10-09T06:39:23.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.393+0000 7f0ec0d5c700 1 -- 172.21.15.163:0/1316199626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ebc06bcd0 msgr2=0x7f0ebc0caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:23.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.393+0000 7f0ec0d5c700 1 --2- 172.21.15.163:0/1316199626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ebc06bcd0 0x7f0ebc0caa90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0eac0098e0 tx=0x7f0eac0091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:23.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.393+0000 7f0ec0d5c700 1 -- 172.21.15.163:0/1316199626 shutdown_connections 2023-10-09T06:39:23.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.393+0000 7f0ec0d5c700 1 --2- 172.21.15.163:0/1316199626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ebc06bcd0 0x7f0ebc0caa90 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:23.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.393+0000 7f0ec0d5c700 1 --2- 172.21.15.163:0/1316199626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ebc06c0a0 0x7f0ebc15a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:23.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.393+0000 7f0ec0d5c700 1 -- 172.21.15.163:0/1316199626 >> 172.21.15.163:0/1316199626 conn(0x7f0ebc0ce600 msgr2=0x7f0ebc0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:23.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.393+0000 7f0ec0d5c700 1 -- 172.21.15.163:0/1316199626 shutdown_connections 2023-10-09T06:39:23.397 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.393+0000 7f0ec0d5c700 1 -- 172.21.15.163:0/1316199626 wait complete. 2023-10-09T06:39:23.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.394+0000 7f0ec0d5c700 1 Processor -- start 2023-10-09T06:39:23.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.394+0000 7f0ec0d5c700 1 -- start start 2023-10-09T06:39:23.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.395+0000 7f0ec0d5c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ebc06bcd0 0x7f0ebc0c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:23.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.395+0000 7f0ec0d5c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ebc06c0a0 0x7f0ebc153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:23.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.395+0000 7f0ec0d5c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ebc15df50 0x7f0ebc0b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:23.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.395+0000 7f0ec0d5c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc0b2150 con 0x7f0ebc06bcd0 2023-10-09T06:39:23.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.395+0000 7f0ec0d5c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc0b1540 con 0x7f0ebc15df50 2023-10-09T06:39:23.400 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.395+0000 7f0ec0d5c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc0b1820 con 0x7f0ebc06c0a0 2023-10-09T06:39:23.400 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.395+0000 7f0eba59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ebc06bcd0 0x7f0ebc0c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:23.400 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.395+0000 7f0eba59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ebc06bcd0 0x7f0ebc0c59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36390/0 (socket says 172.21.15.163:36390) 2023-10-09T06:39:23.400 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.395+0000 7f0eba59c700 1 -- 172.21.15.163:0/450748805 learned_addr learned my addr 172.21.15.163:0/450748805 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:23.400 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.395+0000 7f0ebad9d700 1 --2- 172.21.15.163:0/450748805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ebc15df50 0x7f0ebc0b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:23.400 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.395+0000 7f0eb1d9b700 1 --2- 172.21.15.163:0/450748805 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ebc06c0a0 0x7f0ebc153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:23.400 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.395+0000 7f0eba59c700 1 -- 172.21.15.163:0/450748805 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ebc06c0a0 msgr2=0x7f0ebc153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:23.400 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.395+0000 7f0eba59c700 1 --2- 172.21.15.163:0/450748805 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ebc06c0a0 0x7f0ebc153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:23.400 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.395+0000 7f0eba59c700 1 -- 172.21.15.163:0/450748805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ebc15df50 msgr2=0x7f0ebc0b25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:23.400 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.395+0000 7f0eba59c700 1 --2- 172.21.15.163:0/450748805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ebc15df50 0x7f0ebc0b25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:23.400 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.395+0000 7f0eba59c700 1 -- 172.21.15.163:0/450748805 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eac009420 con 0x7f0ebc06bcd0 2023-10-09T06:39:23.400 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.396+0000 7f0eba59c700 1 --2- 172.21.15.163:0/450748805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ebc06bcd0 0x7f0ebc0c59d0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f0eac009f80 tx=0x7f0eac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:23.401 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.396+0000 7f0eb2ffd700 1 -- 172.21.15.163:0/450748805 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac00b910 con 0x7f0ebc06bcd0 2023-10-09T06:39:23.401 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.396+0000 7f0eb2ffd700 1 -- 172.21.15.163:0/450748805 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eac00bab0 con 0x7f0ebc06bcd0 2023-10-09T06:39:23.401 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.396+0000 7f0eb2ffd700 1 -- 172.21.15.163:0/450748805 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac011670 con 0x7f0ebc06bcd0 2023-10-09T06:39:23.401 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.396+0000 7f0ec0d5c700 1 -- 172.21.15.163:0/450748805 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ebc0b1b00 con 0x7f0ebc06bcd0 2023-10-09T06:39:23.401 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.396+0000 7f0ec0d5c700 1 -- 172.21.15.163:0/450748805 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ebc154260 con 0x7f0ebc06bcd0 2023-10-09T06:39:23.401 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.397+0000 7f0ec0d5c700 1 -- 172.21.15.163:0/450748805 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ebc0b2430 con 0x7f0ebc06bcd0 2023-10-09T06:39:23.401 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.397+0000 7f0eb2ffd700 1 -- 172.21.15.163:0/450748805 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0eac011a60 con 0x7f0ebc06bcd0 2023-10-09T06:39:23.401 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.398+0000 7f0eb2ffd700 1 --2- 172.21.15.163:0/450748805 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0e9c067320 0x7f0e9c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:23.401 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.398+0000 7f0eb2ffd700 1 -- 172.21.15.163:0/450748805 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0eac086550 con 0x7f0ebc06bcd0 2023-10-09T06:39:23.401 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.398+0000 7f0eb1d9b700 1 --2- 172.21.15.163:0/450748805 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0e9c067320 0x7f0e9c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:23.402 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.398+0000 7f0eb1d9b700 1 --2- 172.21.15.163:0/450748805 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0e9c067320 0x7f0e9c0697d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc16d890 tx=0x7f0ea40091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:23.403 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.402+0000 7f0eb2ffd700 1 -- 172.21.15.163:0/450748805 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0eac0531e0 con 0x7f0ebc06bcd0 2023-10-09T06:39:23.590 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.589+0000 7f0ec0d5c700 1 -- 172.21.15.163:0/450748805 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ebc04c3c0 con 0x7f0ebc06bcd0 2023-10-09T06:39:23.591 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.589+0000 7f0eb2ffd700 1 -- 172.21.15.163:0/450748805 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v181) v1 ==== 74+0+15425 (secure 0 0 0) 0x7f0eac056800 con 0x7f0ebc06bcd0 2023-10-09T06:39:23.591 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:23.591 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":181,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:39:22.273307+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:39:20.612285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-09T06:39:21.247510+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:39:23.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.591+0000 7f0ec0d5c700 1 -- 172.21.15.163:0/450748805 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0e9c067320 msgr2=0x7f0e9c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:23.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.591+0000 7f0ec0d5c700 1 --2- 172.21.15.163:0/450748805 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0e9c067320 0x7f0e9c0697d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc16d890 tx=0x7f0ea40091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:23.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.591+0000 7f0ec0d5c700 1 -- 172.21.15.163:0/450748805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ebc06bcd0 msgr2=0x7f0ebc0c59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:23.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.591+0000 7f0ec0d5c700 1 --2- 172.21.15.163:0/450748805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ebc06bcd0 0x7f0ebc0c59d0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f0eac009f80 tx=0x7f0eac017000 comp rx=0 tx=0).stop 2023-10-09T06:39:23.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.592+0000 7f0ec0d5c700 1 -- 172.21.15.163:0/450748805 shutdown_connections 2023-10-09T06:39:23.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.592+0000 7f0ec0d5c700 1 --2- 172.21.15.163:0/450748805 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0e9c067320 0x7f0e9c0697d0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:23.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.592+0000 7f0ec0d5c700 1 --2- 172.21.15.163:0/450748805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ebc06bcd0 0x7f0ebc0c59d0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:23.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.592+0000 7f0ec0d5c700 1 --2- 172.21.15.163:0/450748805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ebc15df50 0x7f0ebc0b25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:23.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.592+0000 7f0ec0d5c700 1 --2- 172.21.15.163:0/450748805 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ebc06c0a0 0x7f0ebc153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:23.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.592+0000 7f0ec0d5c700 1 -- 172.21.15.163:0/450748805 >> 172.21.15.163:0/450748805 conn(0x7f0ebc0ce600 msgr2=0x7f0ebc156b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:23.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.592+0000 7f0ec0d5c700 1 -- 172.21.15.163:0/450748805 shutdown_connections 2023-10-09T06:39:23.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.592+0000 7f0ec0d5c700 1 -- 172.21.15.163:0/450748805 wait complete. 2023-10-09T06:39:23.605 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-09T06:39:23.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.723+0000 7f7294c75700 1 Processor -- start 2023-10-09T06:39:23.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.723+0000 7f7294c75700 1 -- start start 2023-10-09T06:39:23.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.724+0000 7f7294c75700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7290159c60 0x7f729015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:23.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.724+0000 7f7294c75700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f729015db10 0x7f7290069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:23.733 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.724+0000 7f7294c75700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f729015a160 con 0x7f7290159890 2023-10-09T06:39:23.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.724+0000 7f7294c75700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f729015df50 con 0x7f7290159c60 2023-10-09T06:39:23.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.724+0000 7f7294c75700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72900d13a0 con 0x7f729015db10 2023-10-09T06:39:23.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.724+0000 7f728e59c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f7290159890 0x7f729015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53142/0 (socket says 172.21.15.163:53142) 2023-10-09T06:39:23.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.724+0000 7f728e59c700 1 -- 172.21.15.163:0/2805044763 learned_addr learned my addr 172.21.15.163:0/2805044763 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:23.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.724+0000 7f728dd9b700 1 --2- 172.21.15.163:0/2805044763 >> v2:172.21.15.163:3300/0 conn(0x7f7290159c60 0x7f729015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:23.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.724+0000 7f728ed9d700 1 --2- 172.21.15.163:0/2805044763 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f729015db10 0x7f7290069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:23.734 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.724+0000 7f728d59a700 1 -- 172.21.15.163:0/2805044763 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3298617786 0 0) 0x7f729015a160 con 0x7f7290159890 2023-10-09T06:39:23.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.724+0000 7f728d59a700 1 -- 172.21.15.163:0/2805044763 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f727c003570 con 0x7f7290159890 2023-10-09T06:39:23.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.724+0000 7f728dd9b700 1 -- 172.21.15.163:0/2805044763 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f729015db10 msgr2=0x7f7290069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:23.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.724+0000 7f728dd9b700 1 --2- 172.21.15.163:0/2805044763 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f729015db10 0x7f7290069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:23.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.724+0000 7f728dd9b700 1 -- 172.21.15.163:0/2805044763 >> v1:172.21.15.163:6789/0 conn(0x7f7290159890 legacy=0x7f729015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:23.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.724+0000 7f728dd9b700 1 -- 172.21.15.163:0/2805044763 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729015e140 con 0x7f7290159c60 2023-10-09T06:39:23.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.725+0000 7f728dd9b700 1 --2- 172.21.15.163:0/2805044763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7290159c60 0x7f729015d5d0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f72780098e0 tx=0x7f72780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4008c063fcfbcf8a server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:23.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.725+0000 7f728d59a700 1 -- 172.21.15.163:0/2805044763 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727800b640 con 0x7f7290159c60 2023-10-09T06:39:23.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.725+0000 7f728d59a700 1 -- 172.21.15.163:0/2805044763 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727800b7a0 con 0x7f7290159c60 2023-10-09T06:39:23.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.725+0000 7f728d59a700 1 -- 172.21.15.163:0/2805044763 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727800bb40 con 0x7f7290159c60 2023-10-09T06:39:23.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.725+0000 7f7294c75700 1 -- 172.21.15.163:0/2805044763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7290159c60 msgr2=0x7f729015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:23.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.725+0000 7f7294c75700 1 --2- 172.21.15.163:0/2805044763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7290159c60 0x7f729015d5d0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f72780098e0 tx=0x7f72780091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:23.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.725+0000 7f7294c75700 1 -- 172.21.15.163:0/2805044763 shutdown_connections 2023-10-09T06:39:23.735 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.725+0000 7f7294c75700 1 --2- 172.21.15.163:0/2805044763 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f729015db10 0x7f7290069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:23.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.725+0000 7f7294c75700 1 --2- 172.21.15.163:0/2805044763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7290159c60 0x7f729015d5d0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:23.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.725+0000 7f7294c75700 1 -- 172.21.15.163:0/2805044763 >> 172.21.15.163:0/2805044763 conn(0x7f7290068e20 msgr2=0x7f72900d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:23.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.725+0000 7f7294c75700 1 -- 172.21.15.163:0/2805044763 shutdown_connections 2023-10-09T06:39:23.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.725+0000 7f7294c75700 1 -- 172.21.15.163:0/2805044763 wait complete. 2023-10-09T06:39:23.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.726+0000 7f7294c75700 1 Processor -- start 2023-10-09T06:39:23.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.726+0000 7f7294c75700 1 -- start start 2023-10-09T06:39:23.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.727+0000 7f7294c75700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7290159890 0x7f72900c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:23.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.727+0000 7f7294c75700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7290159c60 0x7f72900b2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:23.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.727+0000 7f7294c75700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f729015db10 0x7f729015ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:23.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.727+0000 7f7294c75700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729014c990 con 0x7f7290159c60 2023-10-09T06:39:23.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.727+0000 7f7294c75700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729015ea70 con 0x7f7290159890 2023-10-09T06:39:23.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.727+0000 7f7294c75700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72900b0920 con 0x7f729015db10 2023-10-09T06:39:23.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.727+0000 7f728dd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7290159c60 0x7f72900b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:23.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.727+0000 7f728dd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7290159c60 0x7f72900b2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36408/0 (socket says 172.21.15.163:36408) 2023-10-09T06:39:23.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.727+0000 7f728dd9b700 1 -- 172.21.15.163:0/4070951106 learned_addr learned my addr 172.21.15.163:0/4070951106 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:23.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.727+0000 7f728e59c700 1 --2- 172.21.15.163:0/4070951106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7290159890 0x7f72900c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:23.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.727+0000 7f728ed9d700 1 --2- 172.21.15.163:0/4070951106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f729015db10 0x7f729015ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:23.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.728+0000 7f728dd9b700 1 -- 172.21.15.163:0/4070951106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f729015db10 msgr2=0x7f729015ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:23.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.728+0000 7f728dd9b700 1 --2- 172.21.15.163:0/4070951106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f729015db10 0x7f729015ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:23.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.728+0000 7f728dd9b700 1 -- 172.21.15.163:0/4070951106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7290159890 msgr2=0x7f72900c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:23.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.728+0000 7f728dd9b700 1 --2- 172.21.15.163:0/4070951106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7290159890 0x7f72900c6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:23.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.728+0000 7f728dd9b700 1 -- 172.21.15.163:0/4070951106 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7278009420 con 0x7f7290159c60 2023-10-09T06:39:23.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.728+0000 7f728e59c700 1 --2- 172.21.15.163:0/4070951106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7290159890 0x7f72900c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:23.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.728+0000 7f728dd9b700 1 --2- 172.21.15.163:0/4070951106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7290159c60 0x7f72900b2140 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f72900899c0 tx=0x7f7278017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:23.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.728+0000 7f72867fc700 1 -- 172.21.15.163:0/4070951106 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727801bc20 con 0x7f7290159c60 2023-10-09T06:39:23.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.728+0000 7f72867fc700 1 -- 172.21.15.163:0/4070951106 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727801bd80 con 0x7f7290159c60 2023-10-09T06:39:23.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.728+0000 7f72867fc700 1 -- 172.21.15.163:0/4070951106 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7278011670 con 0x7f7290159c60 2023-10-09T06:39:23.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.728+0000 7f7294c75700 1 -- 172.21.15.163:0/4070951106 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f729015e420 con 0x7f7290159c60 2023-10-09T06:39:23.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.728+0000 7f7294c75700 1 -- 172.21.15.163:0/4070951106 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72900b0c00 con 0x7f7290159c60 2023-10-09T06:39:23.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.729+0000 7f7294c75700 1 -- 172.21.15.163:0/4070951106 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f729015ebe0 con 0x7f7290159c60 2023-10-09T06:39:23.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.729+0000 7f72867fc700 1 -- 172.21.15.163:0/4070951106 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f72780119e0 con 0x7f7290159c60 2023-10-09T06:39:23.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.730+0000 7f72867fc700 1 --2- 172.21.15.163:0/4070951106 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7270067320 0x7f72700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:23.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.730+0000 7f72867fc700 1 -- 172.21.15.163:0/4070951106 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7278086100 con 0x7f7290159c60 2023-10-09T06:39:23.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.730+0000 7f728e59c700 1 --2- 172.21.15.163:0/4070951106 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7270067320 0x7f72700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:23.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.730+0000 7f728e59c700 1 --2- 172.21.15.163:0/4070951106 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7270067320 0x7f72700697d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f7280005c50 tx=0x7f7280009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:23.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.734+0000 7f72867fc700 1 -- 172.21.15.163:0/4070951106 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7278052d90 con 0x7f7290159c60 2023-10-09T06:39:23.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:23.961+0000 7f7294c75700 1 -- 172.21.15.163:0/4070951106 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f729004c3c0 con 0x7f7290159c60 2023-10-09T06:39:24.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:24.934+0000 7f72867fc700 1 -- 172.21.15.163:0/4070951106 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v121) v1 ==== 118+0+0 (secure 0 0 0) 0x7f72780563b0 con 0x7f7290159c60 2023-10-09T06:39:24.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:24.937+0000 7f7294c75700 1 -- 172.21.15.163:0/4070951106 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7270067320 msgr2=0x7f72700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:24.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:24.937+0000 7f7294c75700 1 --2- 172.21.15.163:0/4070951106 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7270067320 0x7f72700697d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f7280005c50 tx=0x7f7280009040 comp rx=0 tx=0).stop 2023-10-09T06:39:24.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:24.937+0000 7f7294c75700 1 -- 172.21.15.163:0/4070951106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7290159c60 msgr2=0x7f72900b2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:24.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:24.937+0000 7f7294c75700 1 --2- 172.21.15.163:0/4070951106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7290159c60 0x7f72900b2140 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f72900899c0 tx=0x7f7278017000 comp rx=0 tx=0).stop 2023-10-09T06:39:24.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:24.937+0000 7f7294c75700 1 -- 172.21.15.163:0/4070951106 shutdown_connections 2023-10-09T06:39:24.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:24.937+0000 7f7294c75700 1 --2- 172.21.15.163:0/4070951106 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7270067320 0x7f72700697d0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:24.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:24.937+0000 7f7294c75700 1 --2- 172.21.15.163:0/4070951106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f729015db10 0x7f729015ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:24.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:24.937+0000 7f7294c75700 1 --2- 172.21.15.163:0/4070951106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7290159890 0x7f72900c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:24.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:24.937+0000 7f7294c75700 1 --2- 172.21.15.163:0/4070951106 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7290159c60 0x7f72900b2140 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:24.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:24.937+0000 7f7294c75700 1 -- 172.21.15.163:0/4070951106 >> 172.21.15.163:0/4070951106 conn(0x7f7290068e20 msgr2=0x7f7290158bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:24.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:24.937+0000 7f7294c75700 1 -- 172.21.15.163:0/4070951106 shutdown_connections 2023-10-09T06:39:24.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:24.937+0000 7f7294c75700 1 -- 172.21.15.163:0/4070951106 wait complete. 2023-10-09T06:39:24.951 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-09T06:39:25.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.070+0000 7f2738751700 1 Processor -- start 2023-10-09T06:39:25.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.070+0000 7f2738751700 1 -- start start 2023-10-09T06:39:25.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.071+0000 7f2738751700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f273006bcd0 0x7f27300caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:25.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.071+0000 7f2738751700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f273006c0a0 0x7f273015a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:25.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.071+0000 7f2738751700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f273015d7e0 con 0x7f273015df50 2023-10-09T06:39:25.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.071+0000 7f2738751700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f273015dac0 con 0x7f273006bcd0 2023-10-09T06:39:25.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.071+0000 7f2738751700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f273015aa30 con 0x7f273006c0a0 2023-10-09T06:39:25.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.071+0000 7f27364ed700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f273006bcd0 0x7f27300caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:25.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.071+0000 7f2736cee700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f273015df50 0x7f27300c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53156/0 (socket says 172.21.15.163:53156) 2023-10-09T06:39:25.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.071+0000 7f2736cee700 1 -- 172.21.15.163:0/410206787 learned_addr learned my addr 172.21.15.163:0/410206787 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:25.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.072+0000 7f2735cec700 1 --2- 172.21.15.163:0/410206787 >> v2:172.21.15.181:3300/0 conn(0x7f273006c0a0 0x7f273015a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:25.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.072+0000 7f27364ed700 1 -- 172.21.15.163:0/410206787 >> v2:172.21.15.181:3300/0 conn(0x7f273006c0a0 msgr2=0x7f273015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:25.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.072+0000 7f27364ed700 1 --2- 172.21.15.163:0/410206787 >> v2:172.21.15.181:3300/0 conn(0x7f273006c0a0 0x7f273015a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:25.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.072+0000 7f27364ed700 1 -- 172.21.15.163:0/410206787 >> v1:172.21.15.163:6789/0 conn(0x7f273015df50 legacy=0x7f27300c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:25.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.072+0000 7f27364ed700 1 -- 172.21.15.163:0/410206787 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273015ad10 con 0x7f273006bcd0 2023-10-09T06:39:25.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.072+0000 7f2735cec700 1 --2- 172.21.15.163:0/410206787 >> v2:172.21.15.181:3300/0 conn(0x7f273006c0a0 0x7f273015a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:25.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.072+0000 7f27364ed700 1 --2- 172.21.15.163:0/410206787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f273006bcd0 0x7f27300caa90 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f273006b4e0 tx=0x7f272c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39fbe800aac4c9f server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:25.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.072+0000 7f27354eb700 1 -- 172.21.15.163:0/410206787 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272c00e070 con 0x7f273006bcd0 2023-10-09T06:39:25.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.072+0000 7f27354eb700 1 -- 172.21.15.163:0/410206787 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f272c00bc30 con 0x7f273006bcd0 2023-10-09T06:39:25.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.072+0000 7f27354eb700 1 -- 172.21.15.163:0/410206787 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272c012430 con 0x7f273006bcd0 2023-10-09T06:39:25.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.072+0000 7f2738751700 1 -- 172.21.15.163:0/410206787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f273006bcd0 msgr2=0x7f27300caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:25.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.072+0000 7f2738751700 1 --2- 172.21.15.163:0/410206787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f273006bcd0 0x7f27300caa90 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f273006b4e0 tx=0x7f272c009780 comp rx=0 tx=0).stop 2023-10-09T06:39:25.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.072+0000 7f2738751700 1 -- 172.21.15.163:0/410206787 shutdown_connections 2023-10-09T06:39:25.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.073+0000 7f2738751700 1 --2- 172.21.15.163:0/410206787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f273006bcd0 0x7f27300caa90 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:25.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.073+0000 7f2738751700 1 --2- 172.21.15.163:0/410206787 >> v2:172.21.15.181:3300/0 conn(0x7f273006c0a0 0x7f273015a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:25.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.073+0000 7f2738751700 1 -- 172.21.15.163:0/410206787 >> 172.21.15.163:0/410206787 conn(0x7f27300ce600 msgr2=0x7f27300cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:25.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.073+0000 7f2738751700 1 -- 172.21.15.163:0/410206787 shutdown_connections 2023-10-09T06:39:25.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.073+0000 7f2738751700 1 -- 172.21.15.163:0/410206787 wait complete. 2023-10-09T06:39:25.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.074+0000 7f2738751700 1 Processor -- start 2023-10-09T06:39:25.078 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.074+0000 7f2738751700 1 -- start start 2023-10-09T06:39:25.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.074+0000 7f2738751700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f273006bcd0 0x7f2730147330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:25.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.074+0000 7f2738751700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f273006c0a0 0x7f2730154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:25.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.074+0000 7f2738751700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f273015df50 0x7f27300cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:25.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.074+0000 7f2738751700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27300b2320 con 0x7f273006c0a0 2023-10-09T06:39:25.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.074+0000 7f2738751700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27300b1710 con 0x7f273015df50 2023-10-09T06:39:25.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.074+0000 7f2738751700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27300b19f0 con 0x7f273006bcd0 2023-10-09T06:39:25.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.074+0000 7f2735cec700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f273006c0a0 0x7f2730154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:25.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.074+0000 7f2735cec700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f273006c0a0 0x7f2730154080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36424/0 (socket says 172.21.15.163:36424) 2023-10-09T06:39:25.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.074+0000 7f2735cec700 1 -- 172.21.15.163:0/4180106404 learned_addr learned my addr 172.21.15.163:0/4180106404 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:25.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.075+0000 7f2736cee700 1 --2- 172.21.15.163:0/4180106404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f273015df50 0x7f27300cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:25.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.075+0000 7f27364ed700 1 --2- 172.21.15.163:0/4180106404 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f273006bcd0 0x7f2730147330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:25.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.075+0000 7f2735cec700 1 -- 172.21.15.163:0/4180106404 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f273006bcd0 msgr2=0x7f2730147330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:25.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.075+0000 7f2735cec700 1 --2- 172.21.15.163:0/4180106404 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f273006bcd0 0x7f2730147330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:25.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.075+0000 7f2735cec700 1 -- 172.21.15.163:0/4180106404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f273015df50 msgr2=0x7f27300cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:25.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.075+0000 7f2735cec700 1 --2- 172.21.15.163:0/4180106404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f273015df50 0x7f27300cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:25.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.075+0000 7f2735cec700 1 -- 172.21.15.163:0/4180106404 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f272c009a20 con 0x7f273006c0a0 2023-10-09T06:39:25.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.075+0000 7f2736cee700 1 --2- 172.21.15.163:0/4180106404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f273015df50 0x7f27300cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:25.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.075+0000 7f27364ed700 1 --2- 172.21.15.163:0/4180106404 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f273006bcd0 0x7f2730147330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:25.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.075+0000 7f2735cec700 1 --2- 172.21.15.163:0/4180106404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f273006c0a0 0x7f2730154080 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f272000c850 tx=0x7f2720008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:25.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.075+0000 7f27267fc700 1 -- 172.21.15.163:0/4180106404 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2720011600 con 0x7f273006c0a0 2023-10-09T06:39:25.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.075+0000 7f27267fc700 1 -- 172.21.15.163:0/4180106404 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2720011760 con 0x7f273006c0a0 2023-10-09T06:39:25.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.075+0000 7f2738751700 1 -- 172.21.15.163:0/4180106404 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27300b1cd0 con 0x7f273006c0a0 2023-10-09T06:39:25.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.075+0000 7f27267fc700 1 -- 172.21.15.163:0/4180106404 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2720011b00 con 0x7f273006c0a0 2023-10-09T06:39:25.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.075+0000 7f2738751700 1 -- 172.21.15.163:0/4180106404 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27300b13a0 con 0x7f273006c0a0 2023-10-09T06:39:25.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.076+0000 7f2738751700 1 -- 172.21.15.163:0/4180106404 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27300b2980 con 0x7f273006c0a0 2023-10-09T06:39:25.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.076+0000 7f27267fc700 1 -- 172.21.15.163:0/4180106404 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f27200073b0 con 0x7f273006c0a0 2023-10-09T06:39:25.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.077+0000 7f27267fc700 1 --2- 172.21.15.163:0/4180106404 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2714067320 0x7f27140697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:25.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.077+0000 7f27267fc700 1 -- 172.21.15.163:0/4180106404 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2720084d00 con 0x7f273006c0a0 2023-10-09T06:39:25.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.077+0000 7f27364ed700 1 --2- 172.21.15.163:0/4180106404 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2714067320 0x7f27140697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:25.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.077+0000 7f27364ed700 1 --2- 172.21.15.163:0/4180106404 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2714067320 0x7f27140697d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f272c009fd0 tx=0x7f272c019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:25.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.081+0000 7f27267fc700 1 -- 172.21.15.163:0/4180106404 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f27300b2980 con 0x7f273006c0a0 2023-10-09T06:39:25.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.304+0000 7f2738751700 1 -- 172.21.15.163:0/4180106404 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f27300b2980 con 0x7f273006c0a0 2023-10-09T06:39:25.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.937+0000 7f27267fc700 1 -- 172.21.15.163:0/4180106404 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v122) v1 ==== 122+0+0 (secure 0 0 0) 0x7f27300b2980 con 0x7f273006c0a0 2023-10-09T06:39:25.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.939+0000 7f2738751700 1 -- 172.21.15.163:0/4180106404 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2714067320 msgr2=0x7f27140697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:25.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.939+0000 7f2738751700 1 --2- 172.21.15.163:0/4180106404 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2714067320 0x7f27140697d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f272c009fd0 tx=0x7f272c019000 comp rx=0 tx=0).stop 2023-10-09T06:39:25.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.939+0000 7f2738751700 1 -- 172.21.15.163:0/4180106404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f273006c0a0 msgr2=0x7f2730154080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:25.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.939+0000 7f2738751700 1 --2- 172.21.15.163:0/4180106404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f273006c0a0 0x7f2730154080 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f272000c850 tx=0x7f2720008040 comp rx=0 tx=0).stop 2023-10-09T06:39:25.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.940+0000 7f2738751700 1 -- 172.21.15.163:0/4180106404 shutdown_connections 2023-10-09T06:39:25.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.940+0000 7f2738751700 1 --2- 172.21.15.163:0/4180106404 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2714067320 0x7f27140697d0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:25.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.940+0000 7f2738751700 1 --2- 172.21.15.163:0/4180106404 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f273015df50 0x7f27300cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:25.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.940+0000 7f2738751700 1 --2- 172.21.15.163:0/4180106404 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f273006bcd0 0x7f2730147330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:25.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.940+0000 7f2738751700 1 --2- 172.21.15.163:0/4180106404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f273006c0a0 0x7f2730154080 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:25.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.940+0000 7f2738751700 1 -- 172.21.15.163:0/4180106404 >> 172.21.15.163:0/4180106404 conn(0x7f27300ce600 msgr2=0x7f27301587e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:25.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.940+0000 7f2738751700 1 -- 172.21.15.163:0/4180106404 shutdown_connections 2023-10-09T06:39:25.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:25.940+0000 7f2738751700 1 -- 172.21.15.163:0/4180106404 wait complete. 2023-10-09T06:39:25.953 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:39:26.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.068+0000 7fe1cb10e700 1 Processor -- start 2023-10-09T06:39:26.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.068+0000 7fe1cb10e700 1 -- start start 2023-10-09T06:39:26.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.069+0000 7fe1cb10e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1c4159890 0x7fe1c415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:26.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.069+0000 7fe1cb10e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe1c4069560 0x7fe1c40699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:26.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.069+0000 7fe1cb10e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c415a160 con 0x7fe1c4159c60 2023-10-09T06:39:26.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.069+0000 7fe1cb10e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1c4069f20 con 0x7fe1c4069560 2023-10-09T06:39:26.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.069+0000 7fe1cb10e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c40d13a0 con 0x7fe1c4159890 2023-10-09T06:39:26.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.069+0000 7fe1c8eaa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1c4159890 0x7fe1c415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:26.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.069+0000 7fe1c96ab700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe1c4069560 0x7fe1c40699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:26.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.069+0000 7fe1c3fff700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe1c4159c60 0x7fe1c415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:49886/0 (socket says 172.21.15.163:49886) 2023-10-09T06:39:26.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.069+0000 7fe1c96ab700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe1c4069560 0x7fe1c40699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:50654/0 (socket says 172.21.15.163:50654) 2023-10-09T06:39:26.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.069+0000 7fe1c3fff700 1 -- 172.21.15.163:0/3617218506 learned_addr learned my addr 172.21.15.163:0/3617218506 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:26.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.069+0000 7fe1c8eaa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1c4159890 0x7fe1c415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:48576/0 (socket says 172.21.15.163:48576) 2023-10-09T06:39:26.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.069+0000 7fe1c8eaa700 1 -- 172.21.15.163:0/3617218506 >> v2:172.21.15.181:3300/0 conn(0x7fe1c4069560 msgr2=0x7fe1c40699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:26.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.069+0000 7fe1c8eaa700 1 --2- 172.21.15.163:0/3617218506 >> v2:172.21.15.181:3300/0 conn(0x7fe1c4069560 0x7fe1c40699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:26.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.069+0000 7fe1c8eaa700 1 -- 172.21.15.163:0/3617218506 >> v1:172.21.15.181:6789/0 conn(0x7fe1c4159c60 legacy=0x7fe1c415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:26.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.069+0000 7fe1c37fe700 1 -- 172.21.15.163:0/3617218506 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 266718019 0 0) 0x7fe1c415a160 con 0x7fe1c4159c60 2023-10-09T06:39:26.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.069+0000 7fe1c8eaa700 1 -- 172.21.15.163:0/3617218506 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c415e140 con 0x7fe1c4159890 2023-10-09T06:39:26.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.069+0000 7fe1c96ab700 1 --2- 172.21.15.163:0/3617218506 >> v2:172.21.15.181:3300/0 conn(0x7fe1c4069560 0x7fe1c40699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:39:26.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.070+0000 7fe1c8eaa700 1 --2- 172.21.15.163:0/3617218506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1c4159890 0x7fe1c415abd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4009e60 tx=0x7fe1b4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc3cfc003e9839fc server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:26.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.070+0000 7fe1c37fe700 1 -- 172.21.15.163:0/3617218506 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b400e070 con 0x7fe1c4159890 2023-10-09T06:39:26.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.070+0000 7fe1c37fe700 1 -- 172.21.15.163:0/3617218506 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b400bb60 con 0x7fe1c4159890 2023-10-09T06:39:26.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.070+0000 7fe1c37fe700 1 -- 172.21.15.163:0/3617218506 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4012430 con 0x7fe1c4159890 2023-10-09T06:39:26.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.070+0000 7fe1cb10e700 1 -- 172.21.15.163:0/3617218506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1c4159890 msgr2=0x7fe1c415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:26.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.070+0000 7fe1cb10e700 1 --2- 172.21.15.163:0/3617218506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1c4159890 0x7fe1c415abd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4009e60 tx=0x7fe1b4009780 comp rx=0 tx=0).stop 2023-10-09T06:39:26.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.070+0000 7fe1cb10e700 1 -- 172.21.15.163:0/3617218506 shutdown_connections 2023-10-09T06:39:26.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.070+0000 7fe1cb10e700 1 --2- 172.21.15.163:0/3617218506 >> v2:172.21.15.181:3300/0 conn(0x7fe1c4069560 0x7fe1c40699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:26.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.070+0000 7fe1cb10e700 1 --2- 172.21.15.163:0/3617218506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1c4159890 0x7fe1c415abd0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:26.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.070+0000 7fe1cb10e700 1 -- 172.21.15.163:0/3617218506 >> 172.21.15.163:0/3617218506 conn(0x7fe1c4068e20 msgr2=0x7fe1c40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:26.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.070+0000 7fe1cb10e700 1 -- 172.21.15.163:0/3617218506 shutdown_connections 2023-10-09T06:39:26.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.070+0000 7fe1cb10e700 1 -- 172.21.15.163:0/3617218506 wait complete. 2023-10-09T06:39:26.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.071+0000 7fe1cb10e700 1 Processor -- start 2023-10-09T06:39:26.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.072+0000 7fe1cb10e700 1 -- start start 2023-10-09T06:39:26.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.072+0000 7fe1cb10e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1c4069560 0x7fe1c4153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:26.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.072+0000 7fe1cb10e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1c4159890 0x7fe1c41564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:26.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.072+0000 7fe1cb10e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1c4159c60 0x7fe1c40b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:26.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.072+0000 7fe1cb10e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c4157a40 con 0x7fe1c4159890 2023-10-09T06:39:26.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.072+0000 7fe1cb10e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c4154460 con 0x7fe1c4159c60 2023-10-09T06:39:26.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.072+0000 7fe1cb10e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c4156e30 con 0x7fe1c4069560 2023-10-09T06:39:26.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.072+0000 7fe1c3fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1c4159890 0x7fe1c41564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:26.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.072+0000 7fe1c3fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1c4159890 0x7fe1c41564c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36434/0 (socket says 172.21.15.163:36434) 2023-10-09T06:39:26.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.072+0000 7fe1c3fff700 1 -- 172.21.15.163:0/2404995954 learned_addr learned my addr 172.21.15.163:0/2404995954 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:26.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.072+0000 7fe1c8eaa700 1 --2- 172.21.15.163:0/2404995954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1c4069560 0x7fe1c4153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:26.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.072+0000 7fe1c96ab700 1 --2- 172.21.15.163:0/2404995954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1c4159c60 0x7fe1c40b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:26.078 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.073+0000 7fe1c3fff700 1 -- 172.21.15.163:0/2404995954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1c4069560 msgr2=0x7fe1c4153f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:26.078 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.073+0000 7fe1c3fff700 1 --2- 172.21.15.163:0/2404995954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1c4069560 0x7fe1c4153f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:26.078 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.073+0000 7fe1c3fff700 1 -- 172.21.15.163:0/2404995954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1c4159c60 msgr2=0x7fe1c40b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:26.078 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.073+0000 7fe1c3fff700 1 --2- 172.21.15.163:0/2404995954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1c4159c60 0x7fe1c40b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:26.078 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.073+0000 7fe1c3fff700 1 -- 172.21.15.163:0/2404995954 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1bc009420 con 0x7fe1c4159890 2023-10-09T06:39:26.078 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.073+0000 7fe1c3fff700 1 --2- 172.21.15.163:0/2404995954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1c4159890 0x7fe1c41564c0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac00c810 tx=0x7fe1ac007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:26.078 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.073+0000 7fe1c0ff9700 1 -- 172.21.15.163:0/2404995954 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac005c00 con 0x7fe1c4159890 2023-10-09T06:39:26.078 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.073+0000 7fe1cb10e700 1 -- 172.21.15.163:0/2404995954 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b4009a20 con 0x7fe1c4159890 2023-10-09T06:39:26.078 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.073+0000 7fe1c0ff9700 1 -- 172.21.15.163:0/2404995954 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1ac005d60 con 0x7fe1c4159890 2023-10-09T06:39:26.078 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.073+0000 7fe1c0ff9700 1 -- 172.21.15.163:0/2404995954 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac0072e0 con 0x7fe1c4159890 2023-10-09T06:39:26.078 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.073+0000 7fe1cb10e700 1 -- 172.21.15.163:0/2404995954 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c41573f0 con 0x7fe1c4159890 2023-10-09T06:39:26.079 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.074+0000 7fe1cb10e700 1 -- 172.21.15.163:0/2404995954 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c404c3c0 con 0x7fe1c4159890 2023-10-09T06:39:26.079 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.074+0000 7fe1c0ff9700 1 -- 172.21.15.163:0/2404995954 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe1ac007690 con 0x7fe1c4159890 2023-10-09T06:39:26.079 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.075+0000 7fe1c0ff9700 1 --2- 172.21.15.163:0/2404995954 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe1a4067320 0x7fe1a40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:26.079 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.075+0000 7fe1c0ff9700 1 -- 172.21.15.163:0/2404995954 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe1ac084030 con 0x7fe1c4159890 2023-10-09T06:39:26.079 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.075+0000 7fe1c8eaa700 1 --2- 172.21.15.163:0/2404995954 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe1a4067320 0x7fe1a40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:26.079 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.075+0000 7fe1c8eaa700 1 --2- 172.21.15.163:0/2404995954 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe1a4067320 0x7fe1a40697d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40097c0 tx=0x7fe1b4019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:26.079 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.079+0000 7fe1c0ff9700 1 -- 172.21.15.163:0/2404995954 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe1ac0b55a0 con 0x7fe1c4159890 2023-10-09T06:39:26.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.302+0000 7fe1cb10e700 1 -- 172.21.15.163:0/2404995954 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1c406a5b0 con 0x7fe1c4159890 2023-10-09T06:39:26.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.303+0000 7fe1c0ff9700 1 -- 172.21.15.163:0/2404995954 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) v1 ==== 95+0+4613 (secure 0 0 0) 0x7fe1ac017070 con 0x7fe1c4159890 2023-10-09T06:39:26.304 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:26.304 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":122,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6815,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3960800872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3960800872},{"type":"v1","addr":"172.21.15.181:6836","nonce":3960800872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":8631,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/380361821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":380361821},{"type":"v1","addr":"172.21.15.163:6834","nonce":380361821}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":8634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2847943599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2847943599},{"type":"v1","addr":"172.21.15.163:6835","nonce":2847943599}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:39:22.273633+0000","modified":"2023-10-09T06:39:25.934461+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8649":{"gid":8649,"name":"d","rank":0,"incarnation":119,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/119178917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":119178917},{"type":"v1","addr":"172.21.15.181:6837","nonce":119178917}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-09T06:39:26.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.305+0000 7fe1cb10e700 1 -- 172.21.15.163:0/2404995954 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe1a4067320 msgr2=0x7fe1a40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:26.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.305+0000 7fe1cb10e700 1 --2- 172.21.15.163:0/2404995954 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe1a4067320 0x7fe1a40697d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40097c0 tx=0x7fe1b4019000 comp rx=0 tx=0).stop 2023-10-09T06:39:26.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.305+0000 7fe1cb10e700 1 -- 172.21.15.163:0/2404995954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1c4159890 msgr2=0x7fe1c41564c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:26.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.305+0000 7fe1cb10e700 1 --2- 172.21.15.163:0/2404995954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1c4159890 0x7fe1c41564c0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac00c810 tx=0x7fe1ac007db0 comp rx=0 tx=0).stop 2023-10-09T06:39:26.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.305+0000 7fe1cb10e700 1 -- 172.21.15.163:0/2404995954 shutdown_connections 2023-10-09T06:39:26.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.305+0000 7fe1cb10e700 1 --2- 172.21.15.163:0/2404995954 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe1a4067320 0x7fe1a40697d0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:26.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.305+0000 7fe1cb10e700 1 --2- 172.21.15.163:0/2404995954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1c4159c60 0x7fe1c40b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:26.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.305+0000 7fe1cb10e700 1 --2- 172.21.15.163:0/2404995954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1c4069560 0x7fe1c4153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:26.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.305+0000 7fe1cb10e700 1 --2- 172.21.15.163:0/2404995954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1c4159890 0x7fe1c41564c0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:26.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.305+0000 7fe1cb10e700 1 -- 172.21.15.163:0/2404995954 >> 172.21.15.163:0/2404995954 conn(0x7fe1c4068e20 msgr2=0x7fe1c4155090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:26.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.305+0000 7fe1cb10e700 1 -- 172.21.15.163:0/2404995954 shutdown_connections 2023-10-09T06:39:26.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.305+0000 7fe1cb10e700 1 -- 172.21.15.163:0/2404995954 wait complete. 2023-10-09T06:39:26.307 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 122 2023-10-09T06:39:26.318 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:39:26.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.432+0000 7f2cd8fbe700 1 Processor -- start 2023-10-09T06:39:26.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.432+0000 7f2cd8fbe700 1 -- start start 2023-10-09T06:39:26.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.433+0000 7f2cd8fbe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2cd4068550 0x7f2cd4068920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:26.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.433+0000 7f2cd8fbe700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd406ec70 con 0x7f2cd406f810 2023-10-09T06:39:26.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.433+0000 7f2cd8fbe700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd40b2e70 con 0x7f2cd415e010 2023-10-09T06:39:26.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.433+0000 7f2cd8fbe700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cd40b3430 con 0x7f2cd4068550 2023-10-09T06:39:26.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.433+0000 7f2cd259c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2cd4068550 0x7f2cd4068920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:26.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.433+0000 7f2cd259c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2cd4068550 0x7f2cd4068920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36438/0 (socket says 172.21.15.163:36438) 2023-10-09T06:39:26.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.433+0000 7f2cd259c700 1 -- 172.21.15.163:0/278442193 learned_addr learned my addr 172.21.15.163:0/278442193 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:26.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.433+0000 7f2cd2d9d700 1 --1- 172.21.15.163:0/278442193 >> v1:172.21.15.163:6789/0 conn(0x7f2cd406f810 0x7f2cd4159510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53172/0 (socket says 172.21.15.163:53172) 2023-10-09T06:39:26.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.434+0000 7f2cd259c700 1 -- 172.21.15.163:0/278442193 >> v1:172.21.15.181:6789/0 conn(0x7f2cd415e010 legacy=0x7f2cd406f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:26.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.434+0000 7f2cd259c700 1 -- 172.21.15.163:0/278442193 >> v1:172.21.15.163:6789/0 conn(0x7f2cd406f810 legacy=0x7f2cd4159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:26.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.434+0000 7f2cd259c700 1 -- 172.21.15.163:0/278442193 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd40b3150 con 0x7f2cd4068550 2023-10-09T06:39:26.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.434+0000 7f2cd159a700 1 -- 172.21.15.163:0/278442193 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1241531562 0 0) 0x7f2cd406ec70 con 0x7f2cd406f810 2023-10-09T06:39:26.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.434+0000 7f2cd259c700 1 --2- 172.21.15.163:0/278442193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cd4068550 0x7f2cd4068920 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4009f20 tx=0x7f2cc4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a796a1b286d3fa server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:26.437 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.434+0000 7f2cd159a700 1 -- 172.21.15.163:0/278442193 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc400e070 con 0x7f2cd4068550 2023-10-09T06:39:26.437 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.434+0000 7f2cd159a700 1 -- 172.21.15.163:0/278442193 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc400ba40 con 0x7f2cd4068550 2023-10-09T06:39:26.437 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.434+0000 7f2cd159a700 1 -- 172.21.15.163:0/278442193 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40092e0 con 0x7f2cd4068550 2023-10-09T06:39:26.437 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.435+0000 7f2cd8fbe700 1 -- 172.21.15.163:0/278442193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cd4068550 msgr2=0x7f2cd4068920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:26.437 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.435+0000 7f2cd8fbe700 1 --2- 172.21.15.163:0/278442193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cd4068550 0x7f2cd4068920 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4009f20 tx=0x7f2cc4009840 comp rx=0 tx=0).stop 2023-10-09T06:39:26.437 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.435+0000 7f2cd8fbe700 1 -- 172.21.15.163:0/278442193 shutdown_connections 2023-10-09T06:39:26.437 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.435+0000 7f2cd8fbe700 1 --2- 172.21.15.163:0/278442193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cd4068550 0x7f2cd4068920 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:26.437 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.435+0000 7f2cd8fbe700 1 -- 172.21.15.163:0/278442193 >> 172.21.15.163:0/278442193 conn(0x7f2cd40cd480 msgr2=0x7f2cd40cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:26.437 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.435+0000 7f2cd8fbe700 1 -- 172.21.15.163:0/278442193 shutdown_connections 2023-10-09T06:39:26.437 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.435+0000 7f2cd8fbe700 1 -- 172.21.15.163:0/278442193 wait complete. 2023-10-09T06:39:26.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.436+0000 7f2cd8fbe700 1 Processor -- start 2023-10-09T06:39:26.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.436+0000 7f2cd8fbe700 1 -- start start 2023-10-09T06:39:26.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.436+0000 7f2cd8fbe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2cd4068550 0x7f2cd416e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:26.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.436+0000 7f2cd8fbe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cd406f810 0x7f2cd4147630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:26.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.436+0000 7f2cd8fbe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cd415e010 0x7f2cd40aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:26.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.436+0000 7f2cd8fbe700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd4144cd0 con 0x7f2cd406f810 2023-10-09T06:39:26.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.436+0000 7f2cd8fbe700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd416e850 con 0x7f2cd415e010 2023-10-09T06:39:26.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.436+0000 7f2cd8fbe700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd4159d90 con 0x7f2cd4068550 2023-10-09T06:39:26.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.437+0000 7f2cd1d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cd406f810 0x7f2cd4147630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:26.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.437+0000 7f2cd1d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cd406f810 0x7f2cd4147630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36440/0 (socket says 172.21.15.163:36440) 2023-10-09T06:39:26.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.437+0000 7f2cd1d9b700 1 -- 172.21.15.163:0/951863497 learned_addr learned my addr 172.21.15.163:0/951863497 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:26.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.437+0000 7f2cd2d9d700 1 --2- 172.21.15.163:0/951863497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cd415e010 0x7f2cd40aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:26.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.437+0000 7f2cd259c700 1 --2- 172.21.15.163:0/951863497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2cd4068550 0x7f2cd416e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:26.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.437+0000 7f2cd1d9b700 1 -- 172.21.15.163:0/951863497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2cd4068550 msgr2=0x7f2cd416e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:26.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.437+0000 7f2cd1d9b700 1 --2- 172.21.15.163:0/951863497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2cd4068550 0x7f2cd416e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:26.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.437+0000 7f2cd1d9b700 1 -- 172.21.15.163:0/951863497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cd415e010 msgr2=0x7f2cd40aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:26.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.437+0000 7f2cd1d9b700 1 --2- 172.21.15.163:0/951863497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cd415e010 0x7f2cd40aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:26.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.437+0000 7f2cd1d9b700 1 -- 172.21.15.163:0/951863497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc4009ae0 con 0x7f2cd406f810 2023-10-09T06:39:26.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.437+0000 7f2cd1d9b700 1 --2- 172.21.15.163:0/951863497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cd406f810 0x7f2cd4147630 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800b5e0 tx=0x7f2cc8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:26.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.437+0000 7f2cc27fc700 1 -- 172.21.15.163:0/951863497 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc800f510 con 0x7f2cd406f810 2023-10-09T06:39:26.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.437+0000 7f2cc27fc700 1 -- 172.21.15.163:0/951863497 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc800f670 con 0x7f2cd406f810 2023-10-09T06:39:26.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.437+0000 7f2cd8fbe700 1 -- 172.21.15.163:0/951863497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd415a070 con 0x7f2cd406f810 2023-10-09T06:39:26.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.437+0000 7f2cc27fc700 1 -- 172.21.15.163:0/951863497 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc800fa30 con 0x7f2cd406f810 2023-10-09T06:39:26.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.437+0000 7f2cd8fbe700 1 -- 172.21.15.163:0/951863497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd415a630 con 0x7f2cd406f810 2023-10-09T06:39:26.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.438+0000 7f2cd8fbe700 1 -- 172.21.15.163:0/951863497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cd404c3c0 con 0x7f2cd406f810 2023-10-09T06:39:26.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.438+0000 7f2cc27fc700 1 -- 172.21.15.163:0/951863497 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2cc801a400 con 0x7f2cd406f810 2023-10-09T06:39:26.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.439+0000 7f2cc27fc700 1 --2- 172.21.15.163:0/951863497 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2cb4067320 0x7f2cb40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:26.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.439+0000 7f2cc27fc700 1 -- 172.21.15.163:0/951863497 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2cc8084840 con 0x7f2cd406f810 2023-10-09T06:39:26.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.439+0000 7f2cd259c700 1 --2- 172.21.15.163:0/951863497 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2cb4067320 0x7f2cb40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:26.442 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.439+0000 7f2cd259c700 1 --2- 172.21.15.163:0/951863497 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2cb4067320 0x7f2cb40697d0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4009ab0 tx=0x7f2cc4017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:26.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.443+0000 7f2cc27fc700 1 -- 172.21.15.163:0/951863497 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2cc8051550 con 0x7f2cd406f810 2023-10-09T06:39:26.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.626+0000 7f2cd8fbe700 1 -- 172.21.15.163:0/951863497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2cd4154730 con 0x7f2cd406f810 2023-10-09T06:39:26.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.627+0000 7f2cc27fc700 1 -- 172.21.15.163:0/951863497 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v182) v1 ==== 74+0+15412 (secure 0 0 0) 0x7f2cc8054b70 con 0x7f2cd406f810 2023-10-09T06:39:26.628 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:26.628 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":182,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:39:23.906977+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:39:20.612285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-09T06:39:21.247510+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:39:26.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.629+0000 7f2cd8fbe700 1 -- 172.21.15.163:0/951863497 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2cb4067320 msgr2=0x7f2cb40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:26.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.629+0000 7f2cd8fbe700 1 --2- 172.21.15.163:0/951863497 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2cb4067320 0x7f2cb40697d0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4009ab0 tx=0x7f2cc4017000 comp rx=0 tx=0).stop 2023-10-09T06:39:26.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.629+0000 7f2cd8fbe700 1 -- 172.21.15.163:0/951863497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cd406f810 msgr2=0x7f2cd4147630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:26.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.629+0000 7f2cd8fbe700 1 --2- 172.21.15.163:0/951863497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cd406f810 0x7f2cd4147630 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800b5e0 tx=0x7f2cc8009120 comp rx=0 tx=0).stop 2023-10-09T06:39:26.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.629+0000 7f2cd8fbe700 1 -- 172.21.15.163:0/951863497 shutdown_connections 2023-10-09T06:39:26.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.629+0000 7f2cd8fbe700 1 --2- 172.21.15.163:0/951863497 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2cb4067320 0x7f2cb40697d0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:26.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.629+0000 7f2cd8fbe700 1 --2- 172.21.15.163:0/951863497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2cd415e010 0x7f2cd40aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:26.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.629+0000 7f2cd8fbe700 1 --2- 172.21.15.163:0/951863497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2cd4068550 0x7f2cd416e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:26.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.629+0000 7f2cd8fbe700 1 --2- 172.21.15.163:0/951863497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cd406f810 0x7f2cd4147630 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:26.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.629+0000 7f2cd8fbe700 1 -- 172.21.15.163:0/951863497 >> 172.21.15.163:0/951863497 conn(0x7f2cd40cd480 msgr2=0x7f2cd41589b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:26.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.629+0000 7f2cd8fbe700 1 -- 172.21.15.163:0/951863497 shutdown_connections 2023-10-09T06:39:26.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.629+0000 7f2cd8fbe700 1 -- 172.21.15.163:0/951863497 wait complete. 2023-10-09T06:39:26.643 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-09T06:39:26.643 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:39:26.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.755+0000 7fab5c7fa700 1 Processor -- start 2023-10-09T06:39:26.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.755+0000 7fab5c7fa700 1 -- start start 2023-10-09T06:39:26.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.756+0000 7fab5c7fa700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fab54159c60 0x7fab5415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:26.758 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.756+0000 7fab5c7fa700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab5415a160 con 0x7fab5415db10 2023-10-09T06:39:26.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.756+0000 7fab5c7fa700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab540d13a0 con 0x7fab54159890 2023-10-09T06:39:26.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.756+0000 7fab5c7fa700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab5415df50 con 0x7fab54159c60 2023-10-09T06:39:26.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.756+0000 7fab59d95700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fab54159c60 0x7fab5415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:26.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.756+0000 7fab59d95700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fab54159c60 0x7fab5415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36442/0 (socket says 172.21.15.163:36442) 2023-10-09T06:39:26.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.756+0000 7fab59d95700 1 -- 172.21.15.163:0/1819689481 learned_addr learned my addr 172.21.15.163:0/1819689481 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:26.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.757+0000 7fab59d95700 1 -- 172.21.15.163:0/1819689481 >> v1:172.21.15.181:6789/0 conn(0x7fab54159890 legacy=0x7fab5415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:26.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.757+0000 7fab59d95700 1 -- 172.21.15.163:0/1819689481 >> v1:172.21.15.163:6789/0 conn(0x7fab5415db10 legacy=0x7fab54069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:26.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.757+0000 7fab59d95700 1 -- 172.21.15.163:0/1819689481 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5415e140 con 0x7fab54159c60 2023-10-09T06:39:26.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.757+0000 7fab59594700 1 -- 172.21.15.163:0/1819689481 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3279697930 0 0) 0x7fab5415a160 con 0x7fab5415db10 2023-10-09T06:39:26.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.757+0000 7fab59d95700 1 --2- 172.21.15.163:0/1819689481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab54159c60 0x7fab5415d5d0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fab480099b0 tx=0x7fab48009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb379bc692cc681b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:26.759 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.757+0000 7fab59594700 1 -- 172.21.15.163:0/1819689481 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4800b820 con 0x7fab54159c60 2023-10-09T06:39:26.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.757+0000 7fab59594700 1 -- 172.21.15.163:0/1819689481 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4800b980 con 0x7fab54159c60 2023-10-09T06:39:26.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.757+0000 7fab59594700 1 -- 172.21.15.163:0/1819689481 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4800bd20 con 0x7fab54159c60 2023-10-09T06:39:26.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.757+0000 7fab5c7fa700 1 -- 172.21.15.163:0/1819689481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab54159c60 msgr2=0x7fab5415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:26.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.757+0000 7fab5c7fa700 1 --2- 172.21.15.163:0/1819689481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab54159c60 0x7fab5415d5d0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fab480099b0 tx=0x7fab48009250 comp rx=0 tx=0).stop 2023-10-09T06:39:26.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.758+0000 7fab5c7fa700 1 -- 172.21.15.163:0/1819689481 shutdown_connections 2023-10-09T06:39:26.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.758+0000 7fab5c7fa700 1 --2- 172.21.15.163:0/1819689481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab54159c60 0x7fab5415d5d0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:26.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.758+0000 7fab5c7fa700 1 -- 172.21.15.163:0/1819689481 >> 172.21.15.163:0/1819689481 conn(0x7fab54068e20 msgr2=0x7fab540d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:26.760 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.758+0000 7fab5c7fa700 1 -- 172.21.15.163:0/1819689481 shutdown_connections 2023-10-09T06:39:26.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.758+0000 7fab5c7fa700 1 -- 172.21.15.163:0/1819689481 wait complete. 2023-10-09T06:39:26.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.759+0000 7fab5c7fa700 1 Processor -- start 2023-10-09T06:39:26.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.759+0000 7fab5c7fa700 1 -- start start 2023-10-09T06:39:26.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.759+0000 7fab5c7fa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab54159890 0x7fab54154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:26.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.759+0000 7fab5c7fa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab54159c60 0x7fab540c9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:26.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.759+0000 7fab5c7fa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab5415db10 0x7fab540b1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:26.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.759+0000 7fab5c7fa700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab540c78a0 con 0x7fab5415db10 2023-10-09T06:39:26.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.759+0000 7fab5c7fa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab540cdff0 con 0x7fab54159c60 2023-10-09T06:39:26.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.759+0000 7fab5c7fa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab540cd100 con 0x7fab54159890 2023-10-09T06:39:26.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.759+0000 7fab5ad97700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab5415db10 0x7fab540b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:26.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.759+0000 7fab5ad97700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab5415db10 0x7fab540b1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36448/0 (socket says 172.21.15.163:36448) 2023-10-09T06:39:26.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.760+0000 7fab5ad97700 1 -- 172.21.15.163:0/1889601750 learned_addr learned my addr 172.21.15.163:0/1889601750 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:26.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.760+0000 7fab59d95700 1 --2- 172.21.15.163:0/1889601750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab54159c60 0x7fab540c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:26.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.760+0000 7fab5a596700 1 --2- 172.21.15.163:0/1889601750 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab54159890 0x7fab54154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:26.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.760+0000 7fab5ad97700 1 -- 172.21.15.163:0/1889601750 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab54159890 msgr2=0x7fab54154220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:26.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.760+0000 7fab5ad97700 1 --2- 172.21.15.163:0/1889601750 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab54159890 0x7fab54154220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:26.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.760+0000 7fab5ad97700 1 -- 172.21.15.163:0/1889601750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab54159c60 msgr2=0x7fab540c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:26.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.760+0000 7fab5ad97700 1 --2- 172.21.15.163:0/1889601750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab54159c60 0x7fab540c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:26.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.760+0000 7fab5ad97700 1 -- 172.21.15.163:0/1889601750 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab480094f0 con 0x7fab5415db10 2023-10-09T06:39:26.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.760+0000 7fab59d95700 1 --2- 172.21.15.163:0/1889601750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab54159c60 0x7fab540c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:26.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.760+0000 7fab5ad97700 1 --2- 172.21.15.163:0/1889601750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab5415db10 0x7fab540b1740 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fab4c001560 tx=0x7fab4c001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:26.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.760+0000 7fab527fc700 1 -- 172.21.15.163:0/1889601750 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c00a780 con 0x7fab5415db10 2023-10-09T06:39:26.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.760+0000 7fab527fc700 1 -- 172.21.15.163:0/1889601750 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4c00a8e0 con 0x7fab5415db10 2023-10-09T06:39:26.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.760+0000 7fab5c7fa700 1 -- 172.21.15.163:0/1889601750 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab540cd3e0 con 0x7fab5415db10 2023-10-09T06:39:26.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.760+0000 7fab527fc700 1 -- 172.21.15.163:0/1889601750 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c00aca0 con 0x7fab5415db10 2023-10-09T06:39:26.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.760+0000 7fab5c7fa700 1 -- 172.21.15.163:0/1889601750 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab540cd9a0 con 0x7fab5415db10 2023-10-09T06:39:26.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.761+0000 7fab5c7fa700 1 -- 172.21.15.163:0/1889601750 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab5404c3c0 con 0x7fab5415db10 2023-10-09T06:39:26.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.761+0000 7fab527fc700 1 -- 172.21.15.163:0/1889601750 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fab4c012630 con 0x7fab5415db10 2023-10-09T06:39:26.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.762+0000 7fab527fc700 1 --2- 172.21.15.163:0/1889601750 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fab40067320 0x7fab400697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:26.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.762+0000 7fab527fc700 1 -- 172.21.15.163:0/1889601750 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fab4c084fc0 con 0x7fab5415db10 2023-10-09T06:39:26.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.762+0000 7fab5a596700 1 --2- 172.21.15.163:0/1889601750 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fab40067320 0x7fab400697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:26.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.763+0000 7fab5a596700 1 --2- 172.21.15.163:0/1889601750 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fab40067320 0x7fab400697d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fab44006fd0 tx=0x7fab44008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:26.767 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.766+0000 7fab527fc700 1 -- 172.21.15.163:0/1889601750 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab4c051c50 con 0x7fab5415db10 2023-10-09T06:39:26.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.947+0000 7fab5c7fa700 1 -- 172.21.15.163:0/1889601750 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fab540b42f0 con 0x7fab40067320 2023-10-09T06:39:26.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.955+0000 7fab527fc700 1 -- 172.21.15.163:0/1889601750 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290266 (secure 0 0 0) 0x7fab540b42f0 con 0x7fab40067320 2023-10-09T06:39:26.959 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:26.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.958+0000 7fab5c7fa700 1 -- 172.21.15.163:0/1889601750 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fab40067320 msgr2=0x7fab400697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:26.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.958+0000 7fab5c7fa700 1 --2- 172.21.15.163:0/1889601750 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fab40067320 0x7fab400697d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fab44006fd0 tx=0x7fab44008040 comp rx=0 tx=0).stop 2023-10-09T06:39:26.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.958+0000 7fab5c7fa700 1 -- 172.21.15.163:0/1889601750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab5415db10 msgr2=0x7fab540b1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:26.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.958+0000 7fab5c7fa700 1 --2- 172.21.15.163:0/1889601750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab5415db10 0x7fab540b1740 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fab4c001560 tx=0x7fab4c001770 comp rx=0 tx=0).stop 2023-10-09T06:39:26.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.959+0000 7fab5c7fa700 1 -- 172.21.15.163:0/1889601750 shutdown_connections 2023-10-09T06:39:26.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.959+0000 7fab5c7fa700 1 --2- 172.21.15.163:0/1889601750 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fab40067320 0x7fab400697d0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:26.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.959+0000 7fab5c7fa700 1 --2- 172.21.15.163:0/1889601750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab5415db10 0x7fab540b1740 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:26.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.959+0000 7fab5c7fa700 1 --2- 172.21.15.163:0/1889601750 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab54159890 0x7fab54154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:26.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.959+0000 7fab5c7fa700 1 --2- 172.21.15.163:0/1889601750 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab54159c60 0x7fab540c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:26.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.959+0000 7fab5c7fa700 1 -- 172.21.15.163:0/1889601750 >> 172.21.15.163:0/1889601750 conn(0x7fab54068e20 msgr2=0x7fab541574d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:26.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.959+0000 7fab5c7fa700 1 -- 172.21.15.163:0/1889601750 shutdown_connections 2023-10-09T06:39:26.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:26.959+0000 7fab5c7fa700 1 -- 172.21.15.163:0/1889601750 wait complete. 2023-10-09T06:39:26.963 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:39:26.971 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2023-10-09T06:39:26.425493+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":158,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2430352,"kb_used_data":19280,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752544368,"statfs":{"total":773094113280,"available":770605432832,"internally_reserved":0,"allocated":19742720,"data_stored":9479932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.534801"},"pg_stats":[{"pgid":"32.1f","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924019+0000","last_change":"2023-10-09T06:39:22.918243+0000","last_active":"2023-10-09T06:39:22.924019+0000","last_peered":"2023-10-09T06:39:22.924019+0000","last_clean":"2023-10-09T06:39:22.924019+0000","last_became_active":"2023-10-09T06:39:22.917999+0000","last_became_peered":"2023-10-09T06:39:22.917999+0000","last_unstale":"2023-10-09T06:39:22.924019+0000","last_undegraded":"2023-10-09T06:39:22.924019+0000","last_fullsized":"2023-10-09T06:39:22.924019+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:51:57.698976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.925424+0000","last_change":"2023-10-09T06:39:22.924721+0000","last_active":"2023-10-09T06:39:22.925424+0000","last_peered":"2023-10-09T06:39:22.925424+0000","last_clean":"2023-10-09T06:39:22.925424+0000","last_became_active":"2023-10-09T06:39:22.924325+0000","last_became_peered":"2023-10-09T06:39:22.924325+0000","last_unstale":"2023-10-09T06:39:22.925424+0000","last_undegraded":"2023-10-09T06:39:22.925424+0000","last_fullsized":"2023-10-09T06:39:22.925424+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:37:23.957584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.928431+0000","last_change":"2023-10-09T06:39:22.928026+0000","last_active":"2023-10-09T06:39:22.928431+0000","last_peered":"2023-10-09T06:39:22.928431+0000","last_clean":"2023-10-09T06:39:22.928431+0000","last_became_active":"2023-10-09T06:39:22.927754+0000","last_became_peered":"2023-10-09T06:39:22.927754+0000","last_unstale":"2023-10-09T06:39:22.928431+0000","last_undegraded":"2023-10-09T06:39:22.928431+0000","last_fullsized":"2023-10-09T06:39:22.928431+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:28:57.980961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924443+0000","last_change":"2023-10-09T06:39:22.921410+0000","last_active":"2023-10-09T06:39:22.924443+0000","last_peered":"2023-10-09T06:39:22.924443+0000","last_clean":"2023-10-09T06:39:22.924443+0000","last_became_active":"2023-10-09T06:39:22.921166+0000","last_became_peered":"2023-10-09T06:39:22.921166+0000","last_unstale":"2023-10-09T06:39:22.924443+0000","last_undegraded":"2023-10-09T06:39:22.924443+0000","last_fullsized":"2023-10-09T06:39:22.924443+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:55:17.506264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.925794+0000","last_change":"2023-10-09T06:39:22.921075+0000","last_active":"2023-10-09T06:39:22.925794+0000","last_peered":"2023-10-09T06:39:22.925794+0000","last_clean":"2023-10-09T06:39:22.925794+0000","last_became_active":"2023-10-09T06:39:22.920582+0000","last_became_peered":"2023-10-09T06:39:22.920582+0000","last_unstale":"2023-10-09T06:39:22.925794+0000","last_undegraded":"2023-10-09T06:39:22.925794+0000","last_fullsized":"2023-10-09T06:39:22.925794+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:06:02.686563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.925152+0000","last_change":"2023-10-09T06:39:22.923203+0000","last_active":"2023-10-09T06:39:22.925152+0000","last_peered":"2023-10-09T06:39:22.925152+0000","last_clean":"2023-10-09T06:39:22.925152+0000","last_became_active":"2023-10-09T06:39:22.922819+0000","last_became_peered":"2023-10-09T06:39:22.922819+0000","last_unstale":"2023-10-09T06:39:22.925152+0000","last_undegraded":"2023-10-09T06:39:22.925152+0000","last_fullsized":"2023-10-09T06:39:22.925152+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:29:21.434311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.927606+0000","last_change":"2023-10-09T06:39:22.920741+0000","last_active":"2023-10-09T06:39:22.927606+0000","last_peered":"2023-10-09T06:39:22.927606+0000","last_clean":"2023-10-09T06:39:22.927606+0000","last_became_active":"2023-10-09T06:39:22.920476+0000","last_became_peered":"2023-10-09T06:39:22.920476+0000","last_unstale":"2023-10-09T06:39:22.927606+0000","last_undegraded":"2023-10-09T06:39:22.927606+0000","last_fullsized":"2023-10-09T06:39:22.927606+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:52:03.783431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.927871+0000","last_change":"2023-10-09T06:39:22.922168+0000","last_active":"2023-10-09T06:39:22.927871+0000","last_peered":"2023-10-09T06:39:22.927871+0000","last_clean":"2023-10-09T06:39:22.927871+0000","last_became_active":"2023-10-09T06:39:22.921880+0000","last_became_peered":"2023-10-09T06:39:22.921880+0000","last_unstale":"2023-10-09T06:39:22.927871+0000","last_undegraded":"2023-10-09T06:39:22.927871+0000","last_fullsized":"2023-10-09T06:39:22.927871+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:23:16.252593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.925222+0000","last_change":"2023-10-09T06:39:22.923941+0000","last_active":"2023-10-09T06:39:22.925222+0000","last_peered":"2023-10-09T06:39:22.925222+0000","last_clean":"2023-10-09T06:39:22.925222+0000","last_became_active":"2023-10-09T06:39:22.923648+0000","last_became_peered":"2023-10-09T06:39:22.923648+0000","last_unstale":"2023-10-09T06:39:22.925222+0000","last_undegraded":"2023-10-09T06:39:22.925222+0000","last_fullsized":"2023-10-09T06:39:22.925222+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:55:17.506264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924734+0000","last_change":"2023-10-09T06:39:22.923005+0000","last_active":"2023-10-09T06:39:22.924734+0000","last_peered":"2023-10-09T06:39:22.924734+0000","last_clean":"2023-10-09T06:39:22.924734+0000","last_became_active":"2023-10-09T06:39:22.922737+0000","last_became_peered":"2023-10-09T06:39:22.922737+0000","last_unstale":"2023-10-09T06:39:22.924734+0000","last_undegraded":"2023-10-09T06:39:22.924734+0000","last_fullsized":"2023-10-09T06:39:22.924734+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:41:04.076555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924747+0000","last_change":"2023-10-09T06:39:22.923140+0000","last_active":"2023-10-09T06:39:22.924747+0000","last_peered":"2023-10-09T06:39:22.924747+0000","last_clean":"2023-10-09T06:39:22.924747+0000","last_became_active":"2023-10-09T06:39:22.921689+0000","last_became_peered":"2023-10-09T06:39:22.921689+0000","last_unstale":"2023-10-09T06:39:22.924747+0000","last_undegraded":"2023-10-09T06:39:22.924747+0000","last_fullsized":"2023-10-09T06:39:22.924747+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:55:24.130687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.925820+0000","last_change":"2023-10-09T06:39:22.924888+0000","last_active":"2023-10-09T06:39:22.925820+0000","last_peered":"2023-10-09T06:39:22.925820+0000","last_clean":"2023-10-09T06:39:22.925820+0000","last_became_active":"2023-10-09T06:39:22.924650+0000","last_became_peered":"2023-10-09T06:39:22.924650+0000","last_unstale":"2023-10-09T06:39:22.925820+0000","last_undegraded":"2023-10-09T06:39:22.925820+0000","last_fullsized":"2023-10-09T06:39:22.925820+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:16:35.540661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.923598+0000","last_change":"2023-10-09T06:39:22.917960+0000","last_active":"2023-10-09T06:39:22.923598+0000","last_peered":"2023-10-09T06:39:22.923598+0000","last_clean":"2023-10-09T06:39:22.923598+0000","last_became_active":"2023-10-09T06:39:22.917670+0000","last_became_peered":"2023-10-09T06:39:22.917670+0000","last_unstale":"2023-10-09T06:39:22.923598+0000","last_undegraded":"2023-10-09T06:39:22.923598+0000","last_fullsized":"2023-10-09T06:39:22.923598+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:56:58.122241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924305+0000","last_change":"2023-10-09T06:39:22.920914+0000","last_active":"2023-10-09T06:39:22.924305+0000","last_peered":"2023-10-09T06:39:22.924305+0000","last_clean":"2023-10-09T06:39:22.924305+0000","last_became_active":"2023-10-09T06:39:22.920671+0000","last_became_peered":"2023-10-09T06:39:22.920671+0000","last_unstale":"2023-10-09T06:39:22.924305+0000","last_undegraded":"2023-10-09T06:39:22.924305+0000","last_fullsized":"2023-10-09T06:39:22.924305+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:00:45.522795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.927510+0000","last_change":"2023-10-09T06:39:22.925948+0000","last_active":"2023-10-09T06:39:22.927510+0000","last_peered":"2023-10-09T06:39:22.927510+0000","last_clean":"2023-10-09T06:39:22.927510+0000","last_became_active":"2023-10-09T06:39:22.925134+0000","last_became_peered":"2023-10-09T06:39:22.925134+0000","last_unstale":"2023-10-09T06:39:22.927510+0000","last_undegraded":"2023-10-09T06:39:22.927510+0000","last_fullsized":"2023-10-09T06:39:22.927510+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:26:36.333743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.923371+0000","last_change":"2023-10-09T06:39:22.918671+0000","last_active":"2023-10-09T06:39:22.923371+0000","last_peered":"2023-10-09T06:39:22.923371+0000","last_clean":"2023-10-09T06:39:22.923371+0000","last_became_active":"2023-10-09T06:39:22.918419+0000","last_became_peered":"2023-10-09T06:39:22.918419+0000","last_unstale":"2023-10-09T06:39:22.923371+0000","last_undegraded":"2023-10-09T06:39:22.923371+0000","last_fullsized":"2023-10-09T06:39:22.923371+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:19:52.836263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.928967+0000","last_change":"2023-10-09T06:39:22.922412+0000","last_active":"2023-10-09T06:39:22.928967+0000","last_peered":"2023-10-09T06:39:22.928967+0000","last_clean":"2023-10-09T06:39:22.928967+0000","last_became_active":"2023-10-09T06:39:22.922142+0000","last_became_peered":"2023-10-09T06:39:22.922142+0000","last_unstale":"2023-10-09T06:39:22.928967+0000","last_undegraded":"2023-10-09T06:39:22.928967+0000","last_fullsized":"2023-10-09T06:39:22.928967+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:59:29.019677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924311+0000","last_change":"2023-10-09T06:39:22.919688+0000","last_active":"2023-10-09T06:39:22.924311+0000","last_peered":"2023-10-09T06:39:22.924311+0000","last_clean":"2023-10-09T06:39:22.924311+0000","last_became_active":"2023-10-09T06:39:22.919449+0000","last_became_peered":"2023-10-09T06:39:22.919449+0000","last_unstale":"2023-10-09T06:39:22.924311+0000","last_undegraded":"2023-10-09T06:39:22.924311+0000","last_fullsized":"2023-10-09T06:39:22.924311+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:54:14.749388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.923507+0000","last_change":"2023-10-09T06:39:22.920391+0000","last_active":"2023-10-09T06:39:22.923507+0000","last_peered":"2023-10-09T06:39:22.923507+0000","last_clean":"2023-10-09T06:39:22.923507+0000","last_became_active":"2023-10-09T06:39:22.918360+0000","last_became_peered":"2023-10-09T06:39:22.918360+0000","last_unstale":"2023-10-09T06:39:22.923507+0000","last_undegraded":"2023-10-09T06:39:22.923507+0000","last_fullsized":"2023-10-09T06:39:22.923507+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:20:47.054074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.925056+0000","last_change":"2023-10-09T06:39:22.922113+0000","last_active":"2023-10-09T06:39:22.925056+0000","last_peered":"2023-10-09T06:39:22.925056+0000","last_clean":"2023-10-09T06:39:22.925056+0000","last_became_active":"2023-10-09T06:39:22.921870+0000","last_became_peered":"2023-10-09T06:39:22.921870+0000","last_unstale":"2023-10-09T06:39:22.925056+0000","last_undegraded":"2023-10-09T06:39:22.925056+0000","last_fullsized":"2023-10-09T06:39:22.925056+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:56:13.775564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.930217+0000","last_change":"2023-10-09T06:39:22.925514+0000","last_active":"2023-10-09T06:39:22.930217+0000","last_peered":"2023-10-09T06:39:22.930217+0000","last_clean":"2023-10-09T06:39:22.930217+0000","last_became_active":"2023-10-09T06:39:22.925240+0000","last_became_peered":"2023-10-09T06:39:22.925240+0000","last_unstale":"2023-10-09T06:39:22.930217+0000","last_undegraded":"2023-10-09T06:39:22.930217+0000","last_fullsized":"2023-10-09T06:39:22.930217+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:23:48.469889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.926126+0000","last_change":"2023-10-09T06:39:22.921875+0000","last_active":"2023-10-09T06:39:22.926126+0000","last_peered":"2023-10-09T06:39:22.926126+0000","last_clean":"2023-10-09T06:39:22.926126+0000","last_became_active":"2023-10-09T06:39:22.921587+0000","last_became_peered":"2023-10-09T06:39:22.921587+0000","last_unstale":"2023-10-09T06:39:22.926126+0000","last_undegraded":"2023-10-09T06:39:22.926126+0000","last_fullsized":"2023-10-09T06:39:22.926126+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:40:35.649919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.923939+0000","last_change":"2023-10-09T06:39:22.921914+0000","last_active":"2023-10-09T06:39:22.923939+0000","last_peered":"2023-10-09T06:39:22.923939+0000","last_clean":"2023-10-09T06:39:22.923939+0000","last_became_active":"2023-10-09T06:39:22.921651+0000","last_became_peered":"2023-10-09T06:39:22.921651+0000","last_unstale":"2023-10-09T06:39:22.923939+0000","last_undegraded":"2023-10-09T06:39:22.923939+0000","last_fullsized":"2023-10-09T06:39:22.923939+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:24:30.093575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.933391+0000","last_change":"2023-10-09T06:39:22.927247+0000","last_active":"2023-10-09T06:39:22.933391+0000","last_peered":"2023-10-09T06:39:22.933391+0000","last_clean":"2023-10-09T06:39:22.933391+0000","last_became_active":"2023-10-09T06:39:22.926991+0000","last_became_peered":"2023-10-09T06:39:22.926991+0000","last_unstale":"2023-10-09T06:39:22.933391+0000","last_undegraded":"2023-10-09T06:39:22.933391+0000","last_fullsized":"2023-10-09T06:39:22.933391+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:00:20.989336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.919641+0000","last_change":"2023-10-09T06:39:22.915060+0000","last_active":"2023-10-09T06:39:22.919641+0000","last_peered":"2023-10-09T06:39:22.919641+0000","last_clean":"2023-10-09T06:39:22.919641+0000","last_became_active":"2023-10-09T06:39:22.914812+0000","last_became_peered":"2023-10-09T06:39:22.914812+0000","last_unstale":"2023-10-09T06:39:22.919641+0000","last_undegraded":"2023-10-09T06:39:22.919641+0000","last_fullsized":"2023-10-09T06:39:22.919641+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:46:35.056481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.927696+0000","last_change":"2023-10-09T06:39:22.921428+0000","last_active":"2023-10-09T06:39:22.927696+0000","last_peered":"2023-10-09T06:39:22.927696+0000","last_clean":"2023-10-09T06:39:22.927696+0000","last_became_active":"2023-10-09T06:39:22.921131+0000","last_became_peered":"2023-10-09T06:39:22.921131+0000","last_unstale":"2023-10-09T06:39:22.927696+0000","last_undegraded":"2023-10-09T06:39:22.927696+0000","last_fullsized":"2023-10-09T06:39:22.927696+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:22:51.139705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.934313+0000","last_change":"2023-10-09T06:39:22.927263+0000","last_active":"2023-10-09T06:39:22.934313+0000","last_peered":"2023-10-09T06:39:22.934313+0000","last_clean":"2023-10-09T06:39:22.934313+0000","last_became_active":"2023-10-09T06:39:22.927000+0000","last_became_peered":"2023-10-09T06:39:22.927000+0000","last_unstale":"2023-10-09T06:39:22.934313+0000","last_undegraded":"2023-10-09T06:39:22.934313+0000","last_fullsized":"2023-10-09T06:39:22.934313+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:27:06.948168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.925124+0000","last_change":"2023-10-09T06:39:22.923055+0000","last_active":"2023-10-09T06:39:22.925124+0000","last_peered":"2023-10-09T06:39:22.925124+0000","last_clean":"2023-10-09T06:39:22.925124+0000","last_became_active":"2023-10-09T06:39:22.922789+0000","last_became_peered":"2023-10-09T06:39:22.922789+0000","last_unstale":"2023-10-09T06:39:22.925124+0000","last_undegraded":"2023-10-09T06:39:22.925124+0000","last_fullsized":"2023-10-09T06:39:22.925124+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:56:53.405665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.914096+0000","last_change":"2023-10-09T06:39:22.913361+0000","last_active":"2023-10-09T06:39:22.914096+0000","last_peered":"2023-10-09T06:39:22.914096+0000","last_clean":"2023-10-09T06:39:22.914096+0000","last_became_active":"2023-10-09T06:39:22.913123+0000","last_became_peered":"2023-10-09T06:39:22.913123+0000","last_unstale":"2023-10-09T06:39:22.914096+0000","last_undegraded":"2023-10-09T06:39:22.914096+0000","last_fullsized":"2023-10-09T06:39:22.914096+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:17:30.457482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924321+0000","last_change":"2023-10-09T06:39:22.922363+0000","last_active":"2023-10-09T06:39:22.924321+0000","last_peered":"2023-10-09T06:39:22.924321+0000","last_clean":"2023-10-09T06:39:22.924321+0000","last_became_active":"2023-10-09T06:39:22.922096+0000","last_became_peered":"2023-10-09T06:39:22.922096+0000","last_unstale":"2023-10-09T06:39:22.924321+0000","last_undegraded":"2023-10-09T06:39:22.924321+0000","last_fullsized":"2023-10-09T06:39:22.924321+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:16:25.063143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924975+0000","last_change":"2023-10-09T06:39:22.921159+0000","last_active":"2023-10-09T06:39:22.924975+0000","last_peered":"2023-10-09T06:39:22.924975+0000","last_clean":"2023-10-09T06:39:22.924975+0000","last_became_active":"2023-10-09T06:39:22.920910+0000","last_became_peered":"2023-10-09T06:39:22.920910+0000","last_unstale":"2023-10-09T06:39:22.924975+0000","last_undegraded":"2023-10-09T06:39:22.924975+0000","last_fullsized":"2023-10-09T06:39:22.924975+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:45:54.220566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.926031+0000","last_change":"2023-10-09T06:39:22.925186+0000","last_active":"2023-10-09T06:39:22.926031+0000","last_peered":"2023-10-09T06:39:22.926031+0000","last_clean":"2023-10-09T06:39:22.926031+0000","last_became_active":"2023-10-09T06:39:22.924944+0000","last_became_peered":"2023-10-09T06:39:22.924944+0000","last_unstale":"2023-10-09T06:39:22.926031+0000","last_undegraded":"2023-10-09T06:39:22.926031+0000","last_fullsized":"2023-10-09T06:39:22.926031+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:41:04.076555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924988+0000","last_change":"2023-10-09T06:39:22.922371+0000","last_active":"2023-10-09T06:39:22.924988+0000","last_peered":"2023-10-09T06:39:22.924988+0000","last_clean":"2023-10-09T06:39:22.924988+0000","last_became_active":"2023-10-09T06:39:22.920746+0000","last_became_peered":"2023-10-09T06:39:22.920746+0000","last_unstale":"2023-10-09T06:39:22.924988+0000","last_undegraded":"2023-10-09T06:39:22.924988+0000","last_fullsized":"2023-10-09T06:39:22.924988+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:38:29.919819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"181'4","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.937045+0000","last_change":"2023-10-09T06:39:21.914681+0000","last_active":"2023-10-09T06:39:22.937045+0000","last_peered":"2023-10-09T06:39:22.937045+0000","last_clean":"2023-10-09T06:39:22.937045+0000","last_became_active":"2023-10-09T06:39:21.914359+0000","last_became_peered":"2023-10-09T06:39:21.914359+0000","last_unstale":"2023-10-09T06:39:22.937045+0000","last_undegraded":"2023-10-09T06:39:22.937045+0000","last_fullsized":"2023-10-09T06:39:22.937045+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:28:38.998280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.923964+0000","last_change":"2023-10-09T06:39:22.923224+0000","last_active":"2023-10-09T06:39:22.923964+0000","last_peered":"2023-10-09T06:39:22.923964+0000","last_clean":"2023-10-09T06:39:22.923964+0000","last_became_active":"2023-10-09T06:39:22.922967+0000","last_became_peered":"2023-10-09T06:39:22.922967+0000","last_unstale":"2023-10-09T06:39:22.923964+0000","last_undegraded":"2023-10-09T06:39:22.923964+0000","last_fullsized":"2023-10-09T06:39:22.923964+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:50:15.511899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"181'2","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.934816+0000","last_change":"2023-10-09T06:39:21.908996+0000","last_active":"2023-10-09T06:39:22.934816+0000","last_peered":"2023-10-09T06:39:22.934816+0000","last_clean":"2023-10-09T06:39:22.934816+0000","last_became_active":"2023-10-09T06:39:21.908602+0000","last_became_peered":"2023-10-09T06:39:21.908602+0000","last_unstale":"2023-10-09T06:39:22.934816+0000","last_undegraded":"2023-10-09T06:39:22.934816+0000","last_fullsized":"2023-10-09T06:39:22.934816+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:43:06.663080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.925196+0000","last_change":"2023-10-09T06:39:22.923563+0000","last_active":"2023-10-09T06:39:22.925196+0000","last_peered":"2023-10-09T06:39:22.925196+0000","last_clean":"2023-10-09T06:39:22.925196+0000","last_became_active":"2023-10-09T06:39:22.923268+0000","last_became_peered":"2023-10-09T06:39:22.923268+0000","last_unstale":"2023-10-09T06:39:22.925196+0000","last_undegraded":"2023-10-09T06:39:22.925196+0000","last_fullsized":"2023-10-09T06:39:22.925196+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:34:08.783653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"181'2","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.940492+0000","last_change":"2023-10-09T06:39:21.910385+0000","last_active":"2023-10-09T06:39:22.940492+0000","last_peered":"2023-10-09T06:39:22.940492+0000","last_clean":"2023-10-09T06:39:22.940492+0000","last_became_active":"2023-10-09T06:39:21.910135+0000","last_became_peered":"2023-10-09T06:39:21.910135+0000","last_unstale":"2023-10-09T06:39:22.940492+0000","last_undegraded":"2023-10-09T06:39:22.940492+0000","last_fullsized":"2023-10-09T06:39:22.940492+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:20:46.047881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.916219+0000","last_change":"2023-10-09T06:39:22.914972+0000","last_active":"2023-10-09T06:39:22.916219+0000","last_peered":"2023-10-09T06:39:22.916219+0000","last_clean":"2023-10-09T06:39:22.916219+0000","last_became_active":"2023-10-09T06:39:22.914730+0000","last_became_peered":"2023-10-09T06:39:22.914730+0000","last_unstale":"2023-10-09T06:39:22.916219+0000","last_undegraded":"2023-10-09T06:39:22.916219+0000","last_fullsized":"2023-10-09T06:39:22.916219+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:11:29.767572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"181'1","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.934237+0000","last_change":"2023-10-09T06:39:21.909550+0000","last_active":"2023-10-09T06:39:22.934237+0000","last_peered":"2023-10-09T06:39:22.934237+0000","last_clean":"2023-10-09T06:39:22.934237+0000","last_became_active":"2023-10-09T06:39:21.909172+0000","last_became_peered":"2023-10-09T06:39:21.909172+0000","last_unstale":"2023-10-09T06:39:22.934237+0000","last_undegraded":"2023-10-09T06:39:22.934237+0000","last_fullsized":"2023-10-09T06:39:22.934237+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:14:26.618748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.933648+0000","last_change":"2023-10-09T06:39:22.928252+0000","last_active":"2023-10-09T06:39:22.933648+0000","last_peered":"2023-10-09T06:39:22.933648+0000","last_clean":"2023-10-09T06:39:22.933648+0000","last_became_active":"2023-10-09T06:39:22.925747+0000","last_became_peered":"2023-10-09T06:39:22.925747+0000","last_unstale":"2023-10-09T06:39:22.933648+0000","last_undegraded":"2023-10-09T06:39:22.933648+0000","last_fullsized":"2023-10-09T06:39:22.933648+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:46:41.535238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"181'1","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.940363+0000","last_change":"2023-10-09T06:39:21.907441+0000","last_active":"2023-10-09T06:39:22.940363+0000","last_peered":"2023-10-09T06:39:22.940363+0000","last_clean":"2023-10-09T06:39:22.940363+0000","last_became_active":"2023-10-09T06:39:21.907190+0000","last_became_peered":"2023-10-09T06:39:21.907190+0000","last_unstale":"2023-10-09T06:39:22.940363+0000","last_undegraded":"2023-10-09T06:39:22.940363+0000","last_fullsized":"2023-10-09T06:39:22.940363+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:54:27.546478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924422+0000","last_change":"2023-10-09T06:39:22.921304+0000","last_active":"2023-10-09T06:39:22.924422+0000","last_peered":"2023-10-09T06:39:22.924422+0000","last_clean":"2023-10-09T06:39:22.924422+0000","last_became_active":"2023-10-09T06:39:22.921090+0000","last_became_peered":"2023-10-09T06:39:22.921090+0000","last_unstale":"2023-10-09T06:39:22.924422+0000","last_undegraded":"2023-10-09T06:39:22.924422+0000","last_fullsized":"2023-10-09T06:39:22.924422+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:12:53.364651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.913770+0000","last_change":"2023-10-09T06:39:21.912198+0000","last_active":"2023-10-09T06:39:22.913770+0000","last_peered":"2023-10-09T06:39:22.913770+0000","last_clean":"2023-10-09T06:39:22.913770+0000","last_became_active":"2023-10-09T06:39:21.911792+0000","last_became_peered":"2023-10-09T06:39:21.911792+0000","last_unstale":"2023-10-09T06:39:22.913770+0000","last_undegraded":"2023-10-09T06:39:22.913770+0000","last_fullsized":"2023-10-09T06:39:22.913770+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:26:35.327550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.933816+0000","last_change":"2023-10-09T06:39:22.928755+0000","last_active":"2023-10-09T06:39:22.933816+0000","last_peered":"2023-10-09T06:39:22.933816+0000","last_clean":"2023-10-09T06:39:22.933816+0000","last_became_active":"2023-10-09T06:39:22.928456+0000","last_became_peered":"2023-10-09T06:39:22.928456+0000","last_unstale":"2023-10-09T06:39:22.933816+0000","last_undegraded":"2023-10-09T06:39:22.933816+0000","last_fullsized":"2023-10-09T06:39:22.933816+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:05:19.341633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.913641+0000","last_change":"2023-10-09T06:39:21.910184+0000","last_active":"2023-10-09T06:39:22.913641+0000","last_peered":"2023-10-09T06:39:22.913641+0000","last_clean":"2023-10-09T06:39:22.913641+0000","last_became_active":"2023-10-09T06:39:21.909867+0000","last_became_peered":"2023-10-09T06:39:21.909867+0000","last_unstale":"2023-10-09T06:39:22.913641+0000","last_undegraded":"2023-10-09T06:39:22.913641+0000","last_fullsized":"2023-10-09T06:39:22.913641+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:15:31.981803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.933646+0000","last_change":"2023-10-09T06:39:22.928119+0000","last_active":"2023-10-09T06:39:22.933646+0000","last_peered":"2023-10-09T06:39:22.933646+0000","last_clean":"2023-10-09T06:39:22.933646+0000","last_became_active":"2023-10-09T06:39:22.927810+0000","last_became_peered":"2023-10-09T06:39:22.927810+0000","last_unstale":"2023-10-09T06:39:22.933646+0000","last_undegraded":"2023-10-09T06:39:22.933646+0000","last_fullsized":"2023-10-09T06:39:22.933646+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:28:11.548143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.908986+0000","last_change":"2023-10-09T06:39:21.911407+0000","last_active":"2023-10-09T06:39:22.908986+0000","last_peered":"2023-10-09T06:39:22.908986+0000","last_clean":"2023-10-09T06:39:22.908986+0000","last_became_active":"2023-10-09T06:39:21.911115+0000","last_became_peered":"2023-10-09T06:39:21.911115+0000","last_unstale":"2023-10-09T06:39:22.908986+0000","last_undegraded":"2023-10-09T06:39:22.908986+0000","last_fullsized":"2023-10-09T06:39:22.908986+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:21:31.967962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.926054+0000","last_change":"2023-10-09T06:39:22.921859+0000","last_active":"2023-10-09T06:39:22.926054+0000","last_peered":"2023-10-09T06:39:22.926054+0000","last_clean":"2023-10-09T06:39:22.926054+0000","last_became_active":"2023-10-09T06:39:22.921542+0000","last_became_peered":"2023-10-09T06:39:22.921542+0000","last_unstale":"2023-10-09T06:39:22.926054+0000","last_undegraded":"2023-10-09T06:39:22.926054+0000","last_fullsized":"2023-10-09T06:39:22.926054+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:41:28.129645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.914596+0000","last_change":"2023-10-09T06:39:21.905424+0000","last_active":"2023-10-09T06:39:22.914596+0000","last_peered":"2023-10-09T06:39:22.914596+0000","last_clean":"2023-10-09T06:39:22.914596+0000","last_became_active":"2023-10-09T06:39:21.905164+0000","last_became_peered":"2023-10-09T06:39:21.905164+0000","last_unstale":"2023-10-09T06:39:22.914596+0000","last_undegraded":"2023-10-09T06:39:22.914596+0000","last_fullsized":"2023-10-09T06:39:22.914596+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:21:31.967962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924825+0000","last_change":"2023-10-09T06:39:22.921960+0000","last_active":"2023-10-09T06:39:22.924825+0000","last_peered":"2023-10-09T06:39:22.924825+0000","last_clean":"2023-10-09T06:39:22.924825+0000","last_became_active":"2023-10-09T06:39:22.921703+0000","last_became_peered":"2023-10-09T06:39:22.921703+0000","last_unstale":"2023-10-09T06:39:22.924825+0000","last_undegraded":"2023-10-09T06:39:22.924825+0000","last_fullsized":"2023-10-09T06:39:22.924825+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:54:58.571706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"181'3","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.938498+0000","last_change":"2023-10-09T06:39:21.906036+0000","last_active":"2023-10-09T06:39:22.938498+0000","last_peered":"2023-10-09T06:39:22.938498+0000","last_clean":"2023-10-09T06:39:22.938498+0000","last_became_active":"2023-10-09T06:39:21.905785+0000","last_became_peered":"2023-10-09T06:39:21.905785+0000","last_unstale":"2023-10-09T06:39:22.938498+0000","last_undegraded":"2023-10-09T06:39:22.938498+0000","last_fullsized":"2023-10-09T06:39:22.938498+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:00:19.983143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.930239+0000","last_change":"2023-10-09T06:39:22.925546+0000","last_active":"2023-10-09T06:39:22.930239+0000","last_peered":"2023-10-09T06:39:22.930239+0000","last_clean":"2023-10-09T06:39:22.930239+0000","last_became_active":"2023-10-09T06:39:22.925250+0000","last_became_peered":"2023-10-09T06:39:22.925250+0000","last_unstale":"2023-10-09T06:39:22.930239+0000","last_undegraded":"2023-10-09T06:39:22.930239+0000","last_fullsized":"2023-10-09T06:39:22.930239+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:43:07.669273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"181'6","reported_seq":16,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.941108+0000","last_change":"2023-10-09T06:39:21.906268+0000","last_active":"2023-10-09T06:39:22.941108+0000","last_peered":"2023-10-09T06:39:22.941108+0000","last_clean":"2023-10-09T06:39:22.941108+0000","last_became_active":"2023-10-09T06:39:21.905999+0000","last_became_peered":"2023-10-09T06:39:21.905999+0000","last_unstale":"2023-10-09T06:39:22.941108+0000","last_undegraded":"2023-10-09T06:39:22.941108+0000","last_fullsized":"2023-10-09T06:39:22.941108+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:35:15.234596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.923496+0000","last_change":"2023-10-09T06:39:22.918236+0000","last_active":"2023-10-09T06:39:22.923496+0000","last_peered":"2023-10-09T06:39:22.923496+0000","last_clean":"2023-10-09T06:39:22.923496+0000","last_became_active":"2023-10-09T06:39:22.917863+0000","last_became_peered":"2023-10-09T06:39:22.917863+0000","last_unstale":"2023-10-09T06:39:22.923496+0000","last_undegraded":"2023-10-09T06:39:22.923496+0000","last_fullsized":"2023-10-09T06:39:22.923496+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:17:53.105183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"181'2","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.937388+0000","last_change":"2023-10-09T06:39:21.912929+0000","last_active":"2023-10-09T06:39:22.937388+0000","last_peered":"2023-10-09T06:39:22.937388+0000","last_clean":"2023-10-09T06:39:22.937388+0000","last_became_active":"2023-10-09T06:39:21.912692+0000","last_became_peered":"2023-10-09T06:39:21.912692+0000","last_unstale":"2023-10-09T06:39:22.937388+0000","last_undegraded":"2023-10-09T06:39:22.937388+0000","last_fullsized":"2023-10-09T06:39:22.937388+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:23:16.266975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.933624+0000","last_change":"2023-10-09T06:39:22.927379+0000","last_active":"2023-10-09T06:39:22.933624+0000","last_peered":"2023-10-09T06:39:22.933624+0000","last_clean":"2023-10-09T06:39:22.933624+0000","last_became_active":"2023-10-09T06:39:22.926026+0000","last_became_peered":"2023-10-09T06:39:22.926026+0000","last_unstale":"2023-10-09T06:39:22.933624+0000","last_undegraded":"2023-10-09T06:39:22.933624+0000","last_fullsized":"2023-10-09T06:39:22.933624+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:52:16.785202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.916166+0000","last_change":"2023-10-09T06:39:21.908941+0000","last_active":"2023-10-09T06:39:22.916166+0000","last_peered":"2023-10-09T06:39:22.916166+0000","last_clean":"2023-10-09T06:39:22.916166+0000","last_became_active":"2023-10-09T06:39:21.908631+0000","last_became_peered":"2023-10-09T06:39:21.908631+0000","last_unstale":"2023-10-09T06:39:22.916166+0000","last_undegraded":"2023-10-09T06:39:22.916166+0000","last_fullsized":"2023-10-09T06:39:22.916166+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:50:24.784960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.927345+0000","last_change":"2023-10-09T06:39:22.925564+0000","last_active":"2023-10-09T06:39:22.927345+0000","last_peered":"2023-10-09T06:39:22.927345+0000","last_clean":"2023-10-09T06:39:22.927345+0000","last_became_active":"2023-10-09T06:39:22.925329+0000","last_became_peered":"2023-10-09T06:39:22.925329+0000","last_unstale":"2023-10-09T06:39:22.927345+0000","last_undegraded":"2023-10-09T06:39:22.927345+0000","last_fullsized":"2023-10-09T06:39:22.927345+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:54:28.552671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.911903+0000","last_change":"2023-10-09T06:39:21.907716+0000","last_active":"2023-10-09T06:39:22.911903+0000","last_peered":"2023-10-09T06:39:22.911903+0000","last_clean":"2023-10-09T06:39:22.911903+0000","last_became_active":"2023-10-09T06:39:21.907467+0000","last_became_peered":"2023-10-09T06:39:21.907467+0000","last_unstale":"2023-10-09T06:39:22.911903+0000","last_undegraded":"2023-10-09T06:39:22.911903+0000","last_fullsized":"2023-10-09T06:39:22.911903+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:03:25.178538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.933798+0000","last_change":"2023-10-09T06:39:22.925676+0000","last_active":"2023-10-09T06:39:22.933798+0000","last_peered":"2023-10-09T06:39:22.933798+0000","last_clean":"2023-10-09T06:39:22.933798+0000","last_became_active":"2023-10-09T06:39:22.925439+0000","last_became_peered":"2023-10-09T06:39:22.925439+0000","last_unstale":"2023-10-09T06:39:22.933798+0000","last_undegraded":"2023-10-09T06:39:22.933798+0000","last_fullsized":"2023-10-09T06:39:22.933798+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:24:34.424297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"181'8","reported_seq":18,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.939258+0000","last_change":"2023-10-09T06:39:21.903661+0000","last_active":"2023-10-09T06:39:22.939258+0000","last_peered":"2023-10-09T06:39:22.939258+0000","last_clean":"2023-10-09T06:39:22.939258+0000","last_became_active":"2023-10-09T06:39:21.903407+0000","last_became_peered":"2023-10-09T06:39:21.903407+0000","last_unstale":"2023-10-09T06:39:22.939258+0000","last_undegraded":"2023-10-09T06:39:22.939258+0000","last_fullsized":"2023-10-09T06:39:22.939258+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:00:13.235101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.921855+0000","last_change":"2023-10-09T06:39:22.919313+0000","last_active":"2023-10-09T06:39:22.921855+0000","last_peered":"2023-10-09T06:39:22.921855+0000","last_clean":"2023-10-09T06:39:22.921855+0000","last_became_active":"2023-10-09T06:39:22.919054+0000","last_became_peered":"2023-10-09T06:39:22.919054+0000","last_unstale":"2023-10-09T06:39:22.921855+0000","last_undegraded":"2023-10-09T06:39:22.921855+0000","last_fullsized":"2023-10-09T06:39:22.921855+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:34:26.593928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"181'1","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.938177+0000","last_change":"2023-10-09T06:39:21.912159+0000","last_active":"2023-10-09T06:39:22.938177+0000","last_peered":"2023-10-09T06:39:22.938177+0000","last_clean":"2023-10-09T06:39:22.938177+0000","last_became_active":"2023-10-09T06:39:21.911920+0000","last_became_peered":"2023-10-09T06:39:21.911920+0000","last_unstale":"2023-10-09T06:39:22.938177+0000","last_undegraded":"2023-10-09T06:39:22.938177+0000","last_fullsized":"2023-10-09T06:39:22.938177+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:16:24.056950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.921865+0000","last_change":"2023-10-09T06:39:22.918953+0000","last_active":"2023-10-09T06:39:22.921865+0000","last_peered":"2023-10-09T06:39:22.921865+0000","last_clean":"2023-10-09T06:39:22.921865+0000","last_became_active":"2023-10-09T06:39:22.918688+0000","last_became_peered":"2023-10-09T06:39:22.918688+0000","last_unstale":"2023-10-09T06:39:22.921865+0000","last_undegraded":"2023-10-09T06:39:22.921865+0000","last_fullsized":"2023-10-09T06:39:22.921865+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:08:03.674524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"181'2","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.941300+0000","last_change":"2023-10-09T06:39:21.907852+0000","last_active":"2023-10-09T06:39:22.941300+0000","last_peered":"2023-10-09T06:39:22.941300+0000","last_clean":"2023-10-09T06:39:22.941300+0000","last_became_active":"2023-10-09T06:39:21.907554+0000","last_became_peered":"2023-10-09T06:39:21.907554+0000","last_unstale":"2023-10-09T06:39:22.941300+0000","last_undegraded":"2023-10-09T06:39:22.941300+0000","last_fullsized":"2023-10-09T06:39:22.941300+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:01:38.038864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924935+0000","last_change":"2023-10-09T06:39:22.922275+0000","last_active":"2023-10-09T06:39:22.924935+0000","last_peered":"2023-10-09T06:39:22.924935+0000","last_clean":"2023-10-09T06:39:22.924935+0000","last_became_active":"2023-10-09T06:39:22.922012+0000","last_became_peered":"2023-10-09T06:39:22.922012+0000","last_unstale":"2023-10-09T06:39:22.924935+0000","last_undegraded":"2023-10-09T06:39:22.924935+0000","last_fullsized":"2023-10-09T06:39:22.924935+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:35:07.790241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"181'1","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.934760+0000","last_change":"2023-10-09T06:39:21.908834+0000","last_active":"2023-10-09T06:39:22.934760+0000","last_peered":"2023-10-09T06:39:22.934760+0000","last_clean":"2023-10-09T06:39:22.934760+0000","last_became_active":"2023-10-09T06:39:21.908235+0000","last_became_peered":"2023-10-09T06:39:21.908235+0000","last_unstale":"2023-10-09T06:39:22.934760+0000","last_undegraded":"2023-10-09T06:39:22.934760+0000","last_fullsized":"2023-10-09T06:39:22.934760+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:28:10.541950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.926965+0000","last_change":"2023-10-09T06:39:22.926361+0000","last_active":"2023-10-09T06:39:22.926965+0000","last_peered":"2023-10-09T06:39:22.926965+0000","last_clean":"2023-10-09T06:39:22.926965+0000","last_became_active":"2023-10-09T06:39:22.926100+0000","last_became_peered":"2023-10-09T06:39:22.926100+0000","last_unstale":"2023-10-09T06:39:22.926965+0000","last_undegraded":"2023-10-09T06:39:22.926965+0000","last_fullsized":"2023-10-09T06:39:22.926965+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:29:27.307841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.916782+0000","last_change":"2023-10-09T06:39:21.906627+0000","last_active":"2023-10-09T06:39:22.916782+0000","last_peered":"2023-10-09T06:39:22.916782+0000","last_clean":"2023-10-09T06:39:22.916782+0000","last_became_active":"2023-10-09T06:39:21.906358+0000","last_became_peered":"2023-10-09T06:39:21.906358+0000","last_unstale":"2023-10-09T06:39:22.916782+0000","last_undegraded":"2023-10-09T06:39:22.916782+0000","last_fullsized":"2023-10-09T06:39:22.916782+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:03:49.588861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.923554+0000","last_change":"2023-10-09T06:39:22.919897+0000","last_active":"2023-10-09T06:39:22.923554+0000","last_peered":"2023-10-09T06:39:22.923554+0000","last_clean":"2023-10-09T06:39:22.923554+0000","last_became_active":"2023-10-09T06:39:22.919672+0000","last_became_peered":"2023-10-09T06:39:22.919672+0000","last_unstale":"2023-10-09T06:39:22.923554+0000","last_undegraded":"2023-10-09T06:39:22.923554+0000","last_fullsized":"2023-10-09T06:39:22.923554+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:14:37.831447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"181'2","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.938088+0000","last_change":"2023-10-09T06:39:21.905819+0000","last_active":"2023-10-09T06:39:22.938088+0000","last_peered":"2023-10-09T06:39:22.938088+0000","last_clean":"2023-10-09T06:39:22.938088+0000","last_became_active":"2023-10-09T06:39:21.905562+0000","last_became_peered":"2023-10-09T06:39:21.905562+0000","last_unstale":"2023-10-09T06:39:22.938088+0000","last_undegraded":"2023-10-09T06:39:22.938088+0000","last_fullsized":"2023-10-09T06:39:22.938088+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:00:38.917682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924624+0000","last_change":"2023-10-09T06:39:22.921748+0000","last_active":"2023-10-09T06:39:22.924624+0000","last_peered":"2023-10-09T06:39:22.924624+0000","last_clean":"2023-10-09T06:39:22.924624+0000","last_became_active":"2023-10-09T06:39:22.921450+0000","last_became_peered":"2023-10-09T06:39:22.921450+0000","last_unstale":"2023-10-09T06:39:22.924624+0000","last_undegraded":"2023-10-09T06:39:22.924624+0000","last_fullsized":"2023-10-09T06:39:22.924624+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:53:58.058707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.913239+0000","last_change":"2023-10-09T06:39:21.909520+0000","last_active":"2023-10-09T06:39:22.913239+0000","last_peered":"2023-10-09T06:39:22.913239+0000","last_clean":"2023-10-09T06:39:22.913239+0000","last_became_active":"2023-10-09T06:39:21.909250+0000","last_became_peered":"2023-10-09T06:39:21.909250+0000","last_unstale":"2023-10-09T06:39:22.913239+0000","last_undegraded":"2023-10-09T06:39:22.913239+0000","last_fullsized":"2023-10-09T06:39:22.913239+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:22:50.133512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.927713+0000","last_change":"2023-10-09T06:39:22.920313+0000","last_active":"2023-10-09T06:39:22.927713+0000","last_peered":"2023-10-09T06:39:22.927713+0000","last_clean":"2023-10-09T06:39:22.927713+0000","last_became_active":"2023-10-09T06:39:22.920017+0000","last_became_peered":"2023-10-09T06:39:22.920017+0000","last_unstale":"2023-10-09T06:39:22.927713+0000","last_undegraded":"2023-10-09T06:39:22.927713+0000","last_fullsized":"2023-10-09T06:39:22.927713+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:04:35.088573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.911778+0000","last_change":"2023-10-09T06:39:21.909639+0000","last_active":"2023-10-09T06:39:22.911778+0000","last_peered":"2023-10-09T06:39:22.911778+0000","last_clean":"2023-10-09T06:39:22.911778+0000","last_became_active":"2023-10-09T06:39:21.909361+0000","last_became_peered":"2023-10-09T06:39:21.909361+0000","last_unstale":"2023-10-09T06:39:22.911778+0000","last_undegraded":"2023-10-09T06:39:22.911778+0000","last_fullsized":"2023-10-09T06:39:22.911778+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:23:47.463696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.928944+0000","last_change":"2023-10-09T06:39:22.919880+0000","last_active":"2023-10-09T06:39:22.928944+0000","last_peered":"2023-10-09T06:39:22.928944+0000","last_clean":"2023-10-09T06:39:22.928944+0000","last_became_active":"2023-10-09T06:39:22.919611+0000","last_became_peered":"2023-10-09T06:39:22.919611+0000","last_unstale":"2023-10-09T06:39:22.928944+0000","last_undegraded":"2023-10-09T06:39:22.928944+0000","last_fullsized":"2023-10-09T06:39:22.928944+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:12:17.467994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.908522+0000","last_change":"2023-10-09T06:39:21.911934+0000","last_active":"2023-10-09T06:39:22.908522+0000","last_peered":"2023-10-09T06:39:22.908522+0000","last_clean":"2023-10-09T06:39:22.908522+0000","last_became_active":"2023-10-09T06:39:21.911682+0000","last_became_peered":"2023-10-09T06:39:21.911682+0000","last_unstale":"2023-10-09T06:39:22.908522+0000","last_undegraded":"2023-10-09T06:39:22.908522+0000","last_fullsized":"2023-10-09T06:39:22.908522+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:26:35.327550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.934248+0000","last_change":"2023-10-09T06:39:22.928672+0000","last_active":"2023-10-09T06:39:22.934248+0000","last_peered":"2023-10-09T06:39:22.934248+0000","last_clean":"2023-10-09T06:39:22.934248+0000","last_became_active":"2023-10-09T06:39:22.928355+0000","last_became_peered":"2023-10-09T06:39:22.928355+0000","last_unstale":"2023-10-09T06:39:22.934248+0000","last_undegraded":"2023-10-09T06:39:22.934248+0000","last_fullsized":"2023-10-09T06:39:22.934248+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:58:36.972142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"181'2","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.934569+0000","last_change":"2023-10-09T06:39:21.905701+0000","last_active":"2023-10-09T06:39:22.934569+0000","last_peered":"2023-10-09T06:39:22.934569+0000","last_clean":"2023-10-09T06:39:22.934569+0000","last_became_active":"2023-10-09T06:39:21.905461+0000","last_became_peered":"2023-10-09T06:39:21.905461+0000","last_unstale":"2023-10-09T06:39:22.934569+0000","last_undegraded":"2023-10-09T06:39:22.934569+0000","last_fullsized":"2023-10-09T06:39:22.934569+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:24:29.087382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.927492+0000","last_change":"2023-10-09T06:39:22.925393+0000","last_active":"2023-10-09T06:39:22.927492+0000","last_peered":"2023-10-09T06:39:22.927492+0000","last_clean":"2023-10-09T06:39:22.927492+0000","last_became_active":"2023-10-09T06:39:22.924748+0000","last_became_peered":"2023-10-09T06:39:22.924748+0000","last_unstale":"2023-10-09T06:39:22.927492+0000","last_undegraded":"2023-10-09T06:39:22.927492+0000","last_fullsized":"2023-10-09T06:39:22.927492+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:22:00.814515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.910321+0000","last_change":"2023-10-09T06:39:21.906395+0000","last_active":"2023-10-09T06:39:22.910321+0000","last_peered":"2023-10-09T06:39:22.910321+0000","last_clean":"2023-10-09T06:39:22.910321+0000","last_became_active":"2023-10-09T06:39:21.906073+0000","last_became_peered":"2023-10-09T06:39:21.906073+0000","last_unstale":"2023-10-09T06:39:22.910321+0000","last_undegraded":"2023-10-09T06:39:22.910321+0000","last_fullsized":"2023-10-09T06:39:22.910321+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:10:38.345541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924409+0000","last_change":"2023-10-09T06:39:22.920901+0000","last_active":"2023-10-09T06:39:22.924409+0000","last_peered":"2023-10-09T06:39:22.924409+0000","last_clean":"2023-10-09T06:39:22.924409+0000","last_became_active":"2023-10-09T06:39:22.920668+0000","last_became_peered":"2023-10-09T06:39:22.920668+0000","last_unstale":"2023-10-09T06:39:22.924409+0000","last_undegraded":"2023-10-09T06:39:22.924409+0000","last_fullsized":"2023-10-09T06:39:22.924409+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:29:21.434311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.908625+0000","last_change":"2023-10-09T06:39:21.908920+0000","last_active":"2023-10-09T06:39:22.908625+0000","last_peered":"2023-10-09T06:39:22.908625+0000","last_clean":"2023-10-09T06:39:22.908625+0000","last_became_active":"2023-10-09T06:39:21.908640+0000","last_became_peered":"2023-10-09T06:39:21.908640+0000","last_unstale":"2023-10-09T06:39:22.908625+0000","last_undegraded":"2023-10-09T06:39:22.908625+0000","last_fullsized":"2023-10-09T06:39:22.908625+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:24:29.087382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.925812+0000","last_change":"2023-10-09T06:39:22.921370+0000","last_active":"2023-10-09T06:39:22.925812+0000","last_peered":"2023-10-09T06:39:22.925812+0000","last_clean":"2023-10-09T06:39:22.925812+0000","last_became_active":"2023-10-09T06:39:22.921099+0000","last_became_peered":"2023-10-09T06:39:22.921099+0000","last_unstale":"2023-10-09T06:39:22.925812+0000","last_undegraded":"2023-10-09T06:39:22.925812+0000","last_fullsized":"2023-10-09T06:39:22.925812+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:21:36.106172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.909067+0000","last_change":"2023-10-09T06:39:21.911822+0000","last_active":"2023-10-09T06:39:22.909067+0000","last_peered":"2023-10-09T06:39:22.909067+0000","last_clean":"2023-10-09T06:39:22.909067+0000","last_became_active":"2023-10-09T06:39:21.911564+0000","last_became_peered":"2023-10-09T06:39:21.911564+0000","last_unstale":"2023-10-09T06:39:22.909067+0000","last_undegraded":"2023-10-09T06:39:22.909067+0000","last_fullsized":"2023-10-09T06:39:22.909067+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:21:59.808322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.928237+0000","last_change":"2023-10-09T06:39:22.927632+0000","last_active":"2023-10-09T06:39:22.928237+0000","last_peered":"2023-10-09T06:39:22.928237+0000","last_clean":"2023-10-09T06:39:22.928237+0000","last_became_active":"2023-10-09T06:39:22.927399+0000","last_became_peered":"2023-10-09T06:39:22.927399+0000","last_unstale":"2023-10-09T06:39:22.928237+0000","last_undegraded":"2023-10-09T06:39:22.928237+0000","last_fullsized":"2023-10-09T06:39:22.928237+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:44:11.254567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.912315+0000","last_change":"2023-10-09T06:39:21.908656+0000","last_active":"2023-10-09T06:39:22.912315+0000","last_peered":"2023-10-09T06:39:22.912315+0000","last_clean":"2023-10-09T06:39:22.912315+0000","last_became_active":"2023-10-09T06:39:21.908397+0000","last_became_peered":"2023-10-09T06:39:21.908397+0000","last_unstale":"2023-10-09T06:39:22.912315+0000","last_undegraded":"2023-10-09T06:39:22.912315+0000","last_fullsized":"2023-10-09T06:39:22.912315+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:46:40.529045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.921985+0000","last_change":"2023-10-09T06:39:22.918045+0000","last_active":"2023-10-09T06:39:22.921985+0000","last_peered":"2023-10-09T06:39:22.921985+0000","last_clean":"2023-10-09T06:39:22.921985+0000","last_became_active":"2023-10-09T06:39:22.916972+0000","last_became_peered":"2023-10-09T06:39:22.916972+0000","last_unstale":"2023-10-09T06:39:22.921985+0000","last_undegraded":"2023-10-09T06:39:22.921985+0000","last_fullsized":"2023-10-09T06:39:22.921985+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:12:53.364651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.916264+0000","last_change":"2023-10-09T06:39:21.904709+0000","last_active":"2023-10-09T06:39:22.916264+0000","last_peered":"2023-10-09T06:39:22.916264+0000","last_clean":"2023-10-09T06:39:22.916264+0000","last_became_active":"2023-10-09T06:39:21.904436+0000","last_became_peered":"2023-10-09T06:39:21.904436+0000","last_unstale":"2023-10-09T06:39:22.916264+0000","last_undegraded":"2023-10-09T06:39:22.916264+0000","last_fullsized":"2023-10-09T06:39:22.916264+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:04:34.082380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924799+0000","last_change":"2023-10-09T06:39:22.921026+0000","last_active":"2023-10-09T06:39:22.924799+0000","last_peered":"2023-10-09T06:39:22.924799+0000","last_clean":"2023-10-09T06:39:22.924799+0000","last_became_active":"2023-10-09T06:39:22.920777+0000","last_became_peered":"2023-10-09T06:39:22.920777+0000","last_unstale":"2023-10-09T06:39:22.924799+0000","last_undegraded":"2023-10-09T06:39:22.924799+0000","last_fullsized":"2023-10-09T06:39:22.924799+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:41:00.005758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.913864+0000","last_change":"2023-10-09T06:39:21.912036+0000","last_active":"2023-10-09T06:39:22.913864+0000","last_peered":"2023-10-09T06:39:22.913864+0000","last_clean":"2023-10-09T06:39:22.913864+0000","last_became_active":"2023-10-09T06:39:21.911464+0000","last_became_peered":"2023-10-09T06:39:21.911464+0000","last_unstale":"2023-10-09T06:39:22.913864+0000","last_undegraded":"2023-10-09T06:39:22.913864+0000","last_fullsized":"2023-10-09T06:39:22.913864+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:54:27.546478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.929923+0000","last_change":"2023-10-09T06:39:22.925943+0000","last_active":"2023-10-09T06:39:22.929923+0000","last_peered":"2023-10-09T06:39:22.929923+0000","last_clean":"2023-10-09T06:39:22.929923+0000","last_became_active":"2023-10-09T06:39:22.925662+0000","last_became_peered":"2023-10-09T06:39:22.925662+0000","last_unstale":"2023-10-09T06:39:22.929923+0000","last_undegraded":"2023-10-09T06:39:22.929923+0000","last_fullsized":"2023-10-09T06:39:22.929923+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:38:43.178334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.910327+0000","last_change":"2023-10-09T06:39:21.906363+0000","last_active":"2023-10-09T06:39:22.910327+0000","last_peered":"2023-10-09T06:39:22.910327+0000","last_clean":"2023-10-09T06:39:22.910327+0000","last_became_active":"2023-10-09T06:39:21.906105+0000","last_became_peered":"2023-10-09T06:39:21.906105+0000","last_unstale":"2023-10-09T06:39:22.910327+0000","last_undegraded":"2023-10-09T06:39:22.910327+0000","last_fullsized":"2023-10-09T06:39:22.910327+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:10:09.771360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.923364+0000","last_change":"2023-10-09T06:39:22.919673+0000","last_active":"2023-10-09T06:39:22.923364+0000","last_peered":"2023-10-09T06:39:22.923364+0000","last_clean":"2023-10-09T06:39:22.923364+0000","last_became_active":"2023-10-09T06:39:22.919440+0000","last_became_peered":"2023-10-09T06:39:22.919440+0000","last_unstale":"2023-10-09T06:39:22.923364+0000","last_undegraded":"2023-10-09T06:39:22.923364+0000","last_fullsized":"2023-10-09T06:39:22.923364+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:28:57.980961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"181'1","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.936549+0000","last_change":"2023-10-09T06:39:21.907398+0000","last_active":"2023-10-09T06:39:22.936549+0000","last_peered":"2023-10-09T06:39:22.936549+0000","last_clean":"2023-10-09T06:39:22.936549+0000","last_became_active":"2023-10-09T06:39:21.905514+0000","last_became_peered":"2023-10-09T06:39:21.905514+0000","last_unstale":"2023-10-09T06:39:22.936549+0000","last_undegraded":"2023-10-09T06:39:22.936549+0000","last_fullsized":"2023-10-09T06:39:22.936549+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:38:28.913626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":296,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.917173+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:39:22.917173+0000","last_peered":"2023-10-09T06:39:22.917173+0000","last_clean":"2023-10-09T06:39:22.917173+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:39:22.917173+0000","last_undegraded":"2023-10-09T06:39:22.917173+0000","last_fullsized":"2023-10-09T06:39:22.917173+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574959,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304232,"kb_used_data":2848,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067608,"statfs":{"total":96636764160,"available":96325230592,"internally_reserved":0,"allocated":2916352,"data_stored":1628887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:38:42 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51300000000000001,"15min":0.50800000000000001},"min":{"1min":0.36499999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.629,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.498,"5min":0.502,"15min":0.51000000000000001},"min":{"1min":0.33400000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.64700000000000002,"5min":0.70199999999999996,"15min":0.753},"last":0.82899999999999996}]},{"osd":1,"last update":"Mon Oct 9 06:38:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.50700000000000001,"15min":0.51800000000000002},"min":{"1min":0.36899999999999999,"5min":0.33600000000000002,"15min":0.27100000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":1.2030000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53000000000000003,"15min":0.52600000000000002},"min":{"1min":0.39900000000000002,"5min":0.32900000000000001,"15min":0.29699999999999999},"max":{"1min":0.67000000000000004,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.77400000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:38:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53100000000000003,"15min":0.51700000000000002},"min":{"1min":0.37,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.746,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53200000000000003,"15min":0.54300000000000004},"min":{"1min":0.36799999999999999,"5min":0.36399999999999999,"15min":0.315},"max":{"1min":0.68100000000000005,"5min":0.80700000000000005,"15min":0.83099999999999996},"last":0.754}]},{"osd":3,"last update":"Mon Oct 9 06:38:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54200000000000004,"15min":0.55000000000000004},"min":{"1min":0.434,"5min":0.35899999999999999,"15min":0.33500000000000002},"max":{"1min":0.76200000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55100000000000005,"15min":0.58599999999999997},"min":{"1min":0.38400000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69899999999999995,"5min":0.82099999999999995,"15min":1.1479999999999999},"last":0.81100000000000005}]},{"osd":4,"last update":"Mon Oct 9 06:38:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.49299999999999999,"15min":0.51600000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.24399999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.93999999999999995},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58099999999999996,"15min":0.60399999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.30299999999999999},"max":{"1min":0.78600000000000003,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.622}]},{"osd":5,"last update":"Mon Oct 9 06:38:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53700000000000003,"15min":0.56699999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.307},"max":{"1min":0.751,"5min":0.85799999999999998,"15min":0.91100000000000003},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60799999999999998,"15min":0.61799999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34200000000000003},"max":{"1min":0.79400000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.52800000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:38:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56999999999999995,"15min":0.59099999999999997},"min":{"1min":0.318,"5min":0.318,"15min":0.28699999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.88600000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58199999999999996,"15min":0.61099999999999999},"min":{"1min":0.34399999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.81999999999999995,"5min":0.82499999999999996,"15min":0.84599999999999997},"last":0.58199999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574958,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303652,"kb_used_data":2268,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068188,"statfs":{"total":96636764160,"available":96325824512,"internally_reserved":0,"allocated":2322432,"data_stored":1038507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.35899999999999999,"5min":0.35399999999999998,"15min":0.30399999999999999},"max":{"1min":0.70699999999999996,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.505},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.51100000000000001,"15min":0.51200000000000001},"min":{"1min":0.36299999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69699999999999995,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.52700000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50700000000000001,"15min":0.51200000000000001},"min":{"1min":0.43099999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.72699999999999998,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53500000000000003,"15min":0.53300000000000003},"min":{"1min":0.437,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.69099999999999995,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.60199999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.34300000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67800000000000005,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.53000000000000003},"min":{"1min":0.39300000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.58199999999999996}]},{"osd":3,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.39800000000000002,"5min":0.38900000000000001,"15min":0.33500000000000002},"max":{"1min":0.79000000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54400000000000004,"15min":0.67100000000000004},"min":{"1min":0.40300000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":4.7510000000000003},"last":0.66900000000000004}]},{"osd":4,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.46000000000000002,"15min":0.47499999999999998},"min":{"1min":0.28999999999999998,"5min":0.28199999999999997,"15min":0.27900000000000003},"max":{"1min":0.63700000000000001,"5min":0.74299999999999999,"15min":0.76400000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56699999999999995,"15min":0.58299999999999996},"min":{"1min":0.51500000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.75,"5min":0.80800000000000005,"15min":0.83699999999999997},"last":0.77200000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.52100000000000002,"15min":0.53100000000000003},"min":{"1min":0.37,"5min":0.246,"15min":0.246},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57199999999999995,"15min":0.59299999999999997},"min":{"1min":0.47299999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70899999999999996,"5min":0.78000000000000003,"15min":0.86599999999999999},"last":0.71899999999999997}]},{"osd":7,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55600000000000005,"15min":0.57999999999999996},"min":{"1min":0.33400000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82399999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.438,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80500000000000005,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.78500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574961,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303632,"kb_used_data":2248,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068208,"statfs":{"total":96636764160,"available":96325844992,"internally_reserved":0,"allocated":2301952,"data_stored":1032107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.317,"15min":0.311},"min":{"1min":0.22500000000000001,"5min":0.192,"15min":0.192},"max":{"1min":0.47699999999999998,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.35399999999999998,"15min":0.35899999999999999},"min":{"1min":0.27700000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.30399999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34699999999999998,"15min":0.34999999999999998},"min":{"1min":0.23899999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.51600000000000001,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38400000000000001,"15min":0.38500000000000001},"min":{"1min":0.25800000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.60399999999999998,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.27100000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.36599999999999999,"15min":0.36299999999999999},"min":{"1min":0.24299999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.503,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.374,"5min":0.35099999999999998,"15min":0.51600000000000001},"min":{"1min":0.255,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":5.8529999999999998},"last":0.23999999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.36499999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59799999999999998,"15min":0.58799999999999997},"min":{"1min":0.46700000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.52700000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57899999999999996,"15min":0.72299999999999998},"min":{"1min":0.40699999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":5.9459999999999997},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.42499999999999999,"5min":0.38900000000000001,"15min":0.34300000000000003},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.5}]},{"osd":6,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54800000000000004,"15min":0.69399999999999995},"min":{"1min":0.38700000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":5.9160000000000004},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.40999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88600000000000001,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.46999999999999997}]},{"osd":7,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.58399999999999996},"min":{"1min":0.42899999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.97099999999999997},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56899999999999995,"15min":0.72899999999999998},"min":{"1min":0.41799999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":5.9729999999999999},"last":0.51300000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574960,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304236,"kb_used_data":2852,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067604,"statfs":{"total":96636764160,"available":96325226496,"internally_reserved":0,"allocated":2920448,"data_stored":1630299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.316,"15min":0.318},"min":{"1min":0.255,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.311},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.33000000000000002,"15min":0.34399999999999997},"min":{"1min":0.24199999999999999,"5min":0.222,"15min":0.214},"max":{"1min":0.48699999999999999,"5min":0.53600000000000003,"15min":0.94499999999999995},"last":0.32600000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.36299999999999999,"15min":0.36399999999999999},"min":{"1min":0.223,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.53200000000000003,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.379,"15min":0.38800000000000001},"min":{"1min":0.29099999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.52900000000000003,"5min":0.69399999999999995,"15min":0.91700000000000004},"last":0.38900000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.33300000000000002,"15min":0.38},"min":{"1min":0.26000000000000001,"5min":0.23599999999999999,"15min":0.23300000000000001},"max":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":1.554},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.37,"15min":0.39600000000000002},"min":{"1min":0.30599999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54500000000000004,"5min":0.58099999999999996,"15min":0.79000000000000004},"last":0.34300000000000003}]},{"osd":4,"last update":"Mon Oct 9 06:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.75900000000000001,"5min":0.80200000000000005,"15min":1.212},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56200000000000006,"15min":0.70499999999999996},"min":{"1min":0.39000000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77800000000000002,"5min":0.89900000000000002,"15min":5.484},"last":0.46899999999999997}]},{"osd":5,"last update":"Mon Oct 9 06:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56100000000000005,"15min":0.70899999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80100000000000005,"5min":0.81399999999999995,"15min":5.5449999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56899999999999995,"15min":0.57499999999999996},"min":{"1min":0.41399999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.80100000000000005,"5min":0.84499999999999997,"15min":1.113},"last":0.48399999999999999}]},{"osd":6,"last update":"Mon Oct 9 06:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59399999999999997,"15min":0.60499999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.78700000000000003,"5min":0.85599999999999998,"15min":1.0529999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.42899999999999999,"5min":0.42799999999999999,"15min":0.41599999999999998},"max":{"1min":0.78700000000000003,"5min":0.878,"15min":1.169},"last":0.52000000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58199999999999996,"15min":0.59199999999999997},"min":{"1min":0.40200000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.80300000000000005,"5min":0.85499999999999998,"15min":1.2390000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56599999999999995,"15min":0.73099999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77000000000000002,"5min":0.86299999999999999,"15min":5.5800000000000001},"last":0.442}]}]},{"osd":2,"up_from":13,"seq":55834574961,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303652,"kb_used_data":2268,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068188,"statfs":{"total":96636764160,"available":96325824512,"internally_reserved":0,"allocated":2322432,"data_stored":1039019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.318,"15min":0.34999999999999998},"min":{"1min":0.21099999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":1.4199999999999999},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.32700000000000001,"15min":0.53500000000000003},"min":{"1min":0.23200000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":7.2699999999999996},"last":0.25900000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.317,"15min":0.34200000000000003},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.48699999999999999,"5min":0.55200000000000005,"15min":0.65400000000000003},"last":0.246},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.374,"15min":0.58699999999999997},"min":{"1min":0.28199999999999997,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.56999999999999995,"5min":0.57999999999999996,"15min":7.327},"last":0.32300000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.373,"15min":0.39000000000000001},"min":{"1min":0.29499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.53400000000000003,"5min":0.64600000000000002,"15min":0.68600000000000005},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.375,"5min":0.371,"15min":0.39000000000000001},"min":{"1min":0.252,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.72699999999999998},"last":0.33300000000000002}]},{"osd":4,"last update":"Mon Oct 9 06:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54600000000000004,"15min":0.54300000000000004},"min":{"1min":0.33600000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.748,"5min":0.79700000000000004,"15min":0.80300000000000005},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.55600000000000005,"15min":0.75900000000000001},"min":{"1min":0.35999999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68899999999999995,"5min":0.84699999999999998,"15min":7.2389999999999999},"last":0.53500000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57999999999999996,"15min":0.877},"min":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77200000000000002,"5min":0.83799999999999997,"15min":10.890000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59999999999999998,"15min":0.60199999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.58099999999999996}]},{"osd":6,"last update":"Mon Oct 9 06:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56499999999999995,"15min":0.76600000000000001},"min":{"1min":0.40100000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71599999999999997,"5min":0.77300000000000002,"15min":7.2999999999999998},"last":0.504},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.40300000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.59899999999999998}]},{"osd":7,"last update":"Mon Oct 9 06:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58999999999999997,"15min":0.79500000000000004},"min":{"1min":0.40200000000000002,"5min":0.38600000000000001,"15min":0.37},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":7.1879999999999997},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60099999999999998,"15min":0.622},"min":{"1min":0.437,"5min":0.38,"15min":0.38},"max":{"1min":0.80000000000000004,"5min":0.89800000000000002,"15min":0.90500000000000003},"last":0.55300000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574960,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303668,"kb_used_data":2284,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068172,"statfs":{"total":96636764160,"available":96325808128,"internally_reserved":0,"allocated":2338816,"data_stored":1040541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.32800000000000001,"15min":0.33400000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.20599999999999999},"max":{"1min":0.51300000000000001,"5min":0.61699999999999999,"15min":5.8700000000000001},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34699999999999998,"15min":0.36599999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":5.6769999999999996},"last":0.34399999999999997}]},{"osd":1,"last update":"Mon Oct 9 06:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.35099999999999998,"15min":0.35899999999999999},"min":{"1min":0.21299999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.44,"5min":0.59499999999999997,"15min":4.0469999999999997},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.377,"15min":0.38900000000000001},"min":{"1min":0.30099999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.624,"5min":0.68600000000000005,"15min":4.1509999999999998},"last":0.38700000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.35199999999999998,"15min":0.372},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.222},"max":{"1min":0.51900000000000002,"5min":0.56799999999999995,"15min":3.1139999999999999},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.374,"15min":0.38500000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.54500000000000004,"5min":0.66900000000000004,"15min":3.3180000000000001},"last":0.35899999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55400000000000005,"15min":0.56999999999999995},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.68799999999999994,"5min":0.81699999999999995,"15min":3.0899999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.749,"5min":0.79200000000000004,"15min":3.0710000000000002},"last":0.54300000000000004}]},{"osd":5,"last update":"Mon Oct 9 06:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56799999999999995,"15min":0.58799999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.32700000000000001},"max":{"1min":0.745,"5min":0.877,"15min":6.2300000000000004},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58799999999999997,"15min":0.59499999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.755,"5min":0.83099999999999996,"15min":3.0219999999999998},"last":0.60799999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80800000000000005,"5min":0.89200000000000002,"15min":2.6850000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63800000000000001,"15min":0.65100000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.76800000000000002,"5min":0.94299999999999995,"15min":6.5970000000000004},"last":0.68300000000000005}]},{"osd":7,"last update":"Mon Oct 9 06:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.41699999999999998,"5min":0.40200000000000002,"15min":0.38200000000000001},"max":{"1min":0.81999999999999995,"5min":0.88500000000000001,"15min":4.2629999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58899999999999997,"15min":0.61199999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.79100000000000004,"5min":0.82899999999999996,"15min":7.3650000000000002},"last":0.67300000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574959,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303648,"kb_used_data":2264,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068192,"statfs":{"total":96636764160,"available":96325828608,"internally_reserved":0,"allocated":2318336,"data_stored":1038465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.51600000000000001,"15min":0.55100000000000005},"min":{"1min":0.373,"5min":0.28100000000000003,"15min":0.27000000000000002},"max":{"1min":0.82599999999999996,"5min":0.88900000000000001,"15min":1.083},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52500000000000002,"15min":0.52000000000000002},"min":{"1min":0.36499999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.76100000000000001,"5min":0.97799999999999998,"15min":1.054},"last":0.53700000000000003}]},{"osd":1,"last update":"Mon Oct 9 06:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53500000000000003,"15min":0.54700000000000004},"min":{"1min":0.28399999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.71099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.501,"15min":0.496},"min":{"1min":0.34000000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.75700000000000001,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.621}]},{"osd":2,"last update":"Mon Oct 9 06:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52000000000000002,"15min":0.53800000000000003},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":1.0249999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54600000000000004,"15min":0.58599999999999997},"min":{"1min":0.38100000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.66800000000000004,"5min":0.90500000000000003,"15min":0.93799999999999994},"last":0.64300000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52100000000000002,"15min":0.54800000000000004},"min":{"1min":0.33300000000000002,"5min":0.32200000000000001,"15min":0.29199999999999998},"max":{"1min":0.71599999999999997,"5min":0.86899999999999999,"15min":0.98499999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56699999999999995,"15min":0.58699999999999997},"min":{"1min":0.40699999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77500000000000002,"5min":0.92900000000000005,"15min":1.1319999999999999},"last":0.754}]},{"osd":5,"last update":"Mon Oct 9 06:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.51300000000000001,"15min":0.54800000000000004},"min":{"1min":0.34399999999999997,"5min":0.30399999999999999,"15min":0.27300000000000002},"max":{"1min":0.73999999999999999,"5min":0.81100000000000005,"15min":0.998},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57599999999999996,"15min":0.60399999999999998},"min":{"1min":0.46000000000000002,"5min":0.36299999999999999,"15min":0.35099999999999998},"max":{"1min":0.82399999999999995,"5min":0.95599999999999996,"15min":0.97999999999999998},"last":0.78000000000000003}]},{"osd":6,"last update":"Mon Oct 9 06:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54800000000000004,"15min":0.56599999999999995},"min":{"1min":0.39400000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70299999999999996,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56899999999999995,"15min":0.59599999999999997},"min":{"1min":0.41399999999999998,"5min":0.34999999999999998,"15min":0.32700000000000001},"max":{"1min":0.78300000000000003,"5min":1.01,"15min":1.01},"last":0.76900000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53800000000000003,"15min":0.56699999999999995},"min":{"1min":0.35299999999999998,"5min":0.33600000000000002,"15min":0.28299999999999997},"max":{"1min":0.70299999999999996,"5min":0.83199999999999996,"15min":0.90000000000000002},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60499999999999998,"15min":0.64100000000000001},"min":{"1min":0.45000000000000001,"5min":0.34999999999999998,"15min":0.32200000000000001},"max":{"1min":0.81999999999999995,"5min":0.98999999999999999,"15min":1.175},"last":0.83999999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574959,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303632,"kb_used_data":2248,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068208,"statfs":{"total":96636764160,"available":96325844992,"internally_reserved":0,"allocated":2301952,"data_stored":1032107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.502,"15min":0.50900000000000001},"min":{"1min":0.40100000000000002,"5min":0.29299999999999998,"15min":0.28599999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.86599999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51200000000000001,"15min":0.51500000000000001},"min":{"1min":0.38200000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72899999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.54400000000000004}]},{"osd":1,"last update":"Mon Oct 9 06:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.504,"15min":0.51000000000000001},"min":{"1min":0.38,"5min":0.311,"15min":0.311},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.78200000000000003},"last":0.496},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.52200000000000002,"15min":0.53100000000000003},"min":{"1min":0.41199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.81599999999999995},"last":0.51900000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.47199999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.83999999999999997},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57199999999999995,"15min":0.58499999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.39500000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79900000000000004},"last":0.57199999999999995}]},{"osd":3,"last update":"Mon Oct 9 06:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55200000000000005,"15min":0.54600000000000004},"min":{"1min":0.39100000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.55900000000000005},"min":{"1min":0.40699999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.69199999999999995,"5min":0.71099999999999997,"15min":0.81100000000000005},"last":0.61699999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.50700000000000001,"15min":0.5},"min":{"1min":0.34200000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68400000000000005,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.40300000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.80700000000000005},"last":0.64800000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55500000000000005,"15min":0.53900000000000003},"min":{"1min":0.378,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.83199999999999996},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56499999999999995,"15min":0.56200000000000006},"min":{"1min":0.40200000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71799999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.66300000000000003}]},{"osd":7,"last update":"Mon Oct 9 06:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.35999999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73199999999999998,"5min":0.79500000000000004,"15min":0.85599999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.44700000000000001,"5min":0.35199999999999998,"15min":0.314},"max":{"1min":0.77400000000000002,"5min":0.84499999999999997,"15min":0.90600000000000003},"last":0.68100000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:39:26.979 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:39:27.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.125+0000 7f29034b1700 1 Processor -- start 2023-10-09T06:39:27.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.126+0000 7f29034b1700 1 -- start start 2023-10-09T06:39:27.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.126+0000 7f29034b1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f28fc069560 0x7f28fc0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:27.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.126+0000 7f29034b1700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28fc069ee0 con 0x7f28fc159890 2023-10-09T06:39:27.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.126+0000 7f29034b1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28fc15a160 con 0x7f28fc159c60 2023-10-09T06:39:27.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.126+0000 7f29034b1700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28fc0d13a0 con 0x7f28fc069560 2023-10-09T06:39:27.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.127+0000 7f2901a4e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f28fc069560 0x7f28fc0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:27.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.127+0000 7f2901a4e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f28fc069560 0x7f28fc0699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36464/0 (socket says 172.21.15.163:36464) 2023-10-09T06:39:27.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.127+0000 7f290124d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f28fc159890 0x7f28fc15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:53202/0 (socket says 172.21.15.163:53202) 2023-10-09T06:39:27.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.127+0000 7f2901a4e700 1 -- 172.21.15.163:0/180607729 learned_addr learned my addr 172.21.15.163:0/180607729 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:27.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.127+0000 7f2901a4e700 1 -- 172.21.15.163:0/180607729 >> v1:172.21.15.181:6789/0 conn(0x7f28fc159c60 legacy=0x7f28fc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:27.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.127+0000 7f2901a4e700 1 -- 172.21.15.163:0/180607729 >> v1:172.21.15.163:6789/0 conn(0x7f28fc159890 legacy=0x7f28fc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:27.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.127+0000 7f28f3fff700 1 -- 172.21.15.163:0/180607729 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1193789094 0 0) 0x7f28fc069ee0 con 0x7f28fc159890 2023-10-09T06:39:27.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.127+0000 7f2901a4e700 1 -- 172.21.15.163:0/180607729 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28fc15e140 con 0x7f28fc069560 2023-10-09T06:39:27.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.127+0000 7f2901a4e700 1 --2- 172.21.15.163:0/180607729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28fc069560 0x7f28fc0699a0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009f20 tx=0x7f28f4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30f69fe85b49384 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:27.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.127+0000 7f28f3fff700 1 -- 172.21.15.163:0/180607729 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f400e070 con 0x7f28fc069560 2023-10-09T06:39:27.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.127+0000 7f28f3fff700 1 -- 172.21.15.163:0/180607729 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f400ba40 con 0x7f28fc069560 2023-10-09T06:39:27.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.127+0000 7f28f3fff700 1 -- 172.21.15.163:0/180607729 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f40092e0 con 0x7f28fc069560 2023-10-09T06:39:27.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.128+0000 7f29034b1700 1 -- 172.21.15.163:0/180607729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28fc069560 msgr2=0x7f28fc0699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:27.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.128+0000 7f29034b1700 1 --2- 172.21.15.163:0/180607729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28fc069560 0x7f28fc0699a0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009f20 tx=0x7f28f4009840 comp rx=0 tx=0).stop 2023-10-09T06:39:27.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.128+0000 7f29034b1700 1 -- 172.21.15.163:0/180607729 shutdown_connections 2023-10-09T06:39:27.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.128+0000 7f29034b1700 1 --2- 172.21.15.163:0/180607729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28fc069560 0x7f28fc0699a0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:27.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.128+0000 7f29034b1700 1 -- 172.21.15.163:0/180607729 >> 172.21.15.163:0/180607729 conn(0x7f28fc068e20 msgr2=0x7f28fc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:27.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.128+0000 7f29034b1700 1 -- 172.21.15.163:0/180607729 shutdown_connections 2023-10-09T06:39:27.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.128+0000 7f29034b1700 1 -- 172.21.15.163:0/180607729 wait complete. 2023-10-09T06:39:27.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.129+0000 7f29034b1700 1 Processor -- start 2023-10-09T06:39:27.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.129+0000 7f29034b1700 1 -- start start 2023-10-09T06:39:27.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.130+0000 7f29034b1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28fc069560 0x7f28fc154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:27.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.130+0000 7f29034b1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28fc159890 0x7f28fc0cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:27.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.130+0000 7f29034b1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28fc159c60 0x7f28fc169af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:27.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.130+0000 7f29034b1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc15c8b0 con 0x7f28fc159c60 2023-10-09T06:39:27.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.130+0000 7f29034b1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc0c8d10 con 0x7f28fc159890 2023-10-09T06:39:27.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.130+0000 7f29034b1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc0c84f0 con 0x7f28fc069560 2023-10-09T06:39:27.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.130+0000 7f2901a4e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28fc159c60 0x7f28fc169af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:27.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.130+0000 7f2901a4e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28fc159c60 0x7f28fc169af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36468/0 (socket says 172.21.15.163:36468) 2023-10-09T06:39:27.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.130+0000 7f2901a4e700 1 -- 172.21.15.163:0/3759701860 learned_addr learned my addr 172.21.15.163:0/3759701860 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:27.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.130+0000 7f290124d700 1 --2- 172.21.15.163:0/3759701860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28fc069560 0x7f28fc154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:27.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.130+0000 7f2900a4c700 1 --2- 172.21.15.163:0/3759701860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28fc159890 0x7f28fc0cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:27.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.130+0000 7f2901a4e700 1 -- 172.21.15.163:0/3759701860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28fc069560 msgr2=0x7f28fc154940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:27.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.130+0000 7f2901a4e700 1 --2- 172.21.15.163:0/3759701860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28fc069560 0x7f28fc154940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:27.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.130+0000 7f2901a4e700 1 -- 172.21.15.163:0/3759701860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28fc159890 msgr2=0x7f28fc0cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:27.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.130+0000 7f2901a4e700 1 --2- 172.21.15.163:0/3759701860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28fc159890 0x7f28fc0cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:27.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.130+0000 7f2901a4e700 1 -- 172.21.15.163:0/3759701860 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f4009ae0 con 0x7f28fc159c60 2023-10-09T06:39:27.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.130+0000 7f290124d700 1 --2- 172.21.15.163:0/3759701860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28fc069560 0x7f28fc154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:27.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.130+0000 7f2901a4e700 1 --2- 172.21.15.163:0/3759701860 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28fc159c60 0x7f28fc169af0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f28fc06a070 tx=0x7f28f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:27.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.130+0000 7f28f17fa700 1 -- 172.21.15.163:0/3759701860 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4012ea0 con 0x7f28fc159c60 2023-10-09T06:39:27.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.130+0000 7f28f17fa700 1 -- 172.21.15.163:0/3759701860 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f40094e0 con 0x7f28fc159c60 2023-10-09T06:39:27.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.130+0000 7f28f17fa700 1 -- 172.21.15.163:0/3759701860 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f401e660 con 0x7f28fc159c60 2023-10-09T06:39:27.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.130+0000 7f29034b1700 1 -- 172.21.15.163:0/3759701860 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28fc0c87d0 con 0x7f28fc159c60 2023-10-09T06:39:27.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.131+0000 7f29034b1700 1 -- 172.21.15.163:0/3759701860 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28fc15c540 con 0x7f28fc159c60 2023-10-09T06:39:27.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.131+0000 7f29034b1700 1 -- 172.21.15.163:0/3759701860 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28fc15ca20 con 0x7f28fc159c60 2023-10-09T06:39:27.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.133+0000 7f28f17fa700 1 -- 172.21.15.163:0/3759701860 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f28f401ea90 con 0x7f28fc159c60 2023-10-09T06:39:27.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.133+0000 7f28f17fa700 1 --2- 172.21.15.163:0/3759701860 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f28e0067320 0x7f28e00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:27.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.133+0000 7f28f17fa700 1 -- 172.21.15.163:0/3759701860 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f28f4087500 con 0x7f28fc159c60 2023-10-09T06:39:27.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.134+0000 7f290124d700 1 --2- 172.21.15.163:0/3759701860 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f28e0067320 0x7f28e00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:27.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.134+0000 7f290124d700 1 --2- 172.21.15.163:0/3759701860 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f28e0067320 0x7f28e00697d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f28f80011e0 tx=0x7f28f8007c90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:27.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.137+0000 7f28f17fa700 1 -- 172.21.15.163:0/3759701860 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28f40540c0 con 0x7f28fc159c60 2023-10-09T06:39:27.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.314+0000 7f29034b1700 1 -- 172.21.15.163:0/3759701860 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f28fc04c3c0 con 0x7f28e0067320 2023-10-09T06:39:27.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.323+0000 7f28f17fa700 1 -- 172.21.15.163:0/3759701860 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290266 (secure 0 0 0) 0x7f28fc04c3c0 con 0x7f28e0067320 2023-10-09T06:39:27.327 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:27.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.326+0000 7f29034b1700 1 -- 172.21.15.163:0/3759701860 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f28e0067320 msgr2=0x7f28e00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:27.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.326+0000 7f29034b1700 1 --2- 172.21.15.163:0/3759701860 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f28e0067320 0x7f28e00697d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f28f80011e0 tx=0x7f28f8007c90 comp rx=0 tx=0).stop 2023-10-09T06:39:27.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.326+0000 7f29034b1700 1 -- 172.21.15.163:0/3759701860 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28fc159c60 msgr2=0x7f28fc169af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:27.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.326+0000 7f29034b1700 1 --2- 172.21.15.163:0/3759701860 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28fc159c60 0x7f28fc169af0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f28fc06a070 tx=0x7f28f4017000 comp rx=0 tx=0).stop 2023-10-09T06:39:27.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.326+0000 7f29034b1700 1 -- 172.21.15.163:0/3759701860 shutdown_connections 2023-10-09T06:39:27.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.326+0000 7f29034b1700 1 --2- 172.21.15.163:0/3759701860 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f28e0067320 0x7f28e00697d0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:27.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.326+0000 7f29034b1700 1 --2- 172.21.15.163:0/3759701860 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28fc159c60 0x7f28fc169af0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:27.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.326+0000 7f29034b1700 1 --2- 172.21.15.163:0/3759701860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28fc069560 0x7f28fc154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:27.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.326+0000 7f29034b1700 1 --2- 172.21.15.163:0/3759701860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f28fc159890 0x7f28fc0cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:27.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.326+0000 7f29034b1700 1 -- 172.21.15.163:0/3759701860 >> 172.21.15.163:0/3759701860 conn(0x7f28fc068e20 msgr2=0x7f28fc157930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:27.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.326+0000 7f29034b1700 1 -- 172.21.15.163:0/3759701860 shutdown_connections 2023-10-09T06:39:27.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.326+0000 7f29034b1700 1 -- 172.21.15.163:0/3759701860 wait complete. 2023-10-09T06:39:27.332 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:39:27.339 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2023-10-09T06:39:26.425493+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":158,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2430352,"kb_used_data":19280,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752544368,"statfs":{"total":773094113280,"available":770605432832,"internally_reserved":0,"allocated":19742720,"data_stored":9479932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.534801"},"pg_stats":[{"pgid":"32.1f","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924019+0000","last_change":"2023-10-09T06:39:22.918243+0000","last_active":"2023-10-09T06:39:22.924019+0000","last_peered":"2023-10-09T06:39:22.924019+0000","last_clean":"2023-10-09T06:39:22.924019+0000","last_became_active":"2023-10-09T06:39:22.917999+0000","last_became_peered":"2023-10-09T06:39:22.917999+0000","last_unstale":"2023-10-09T06:39:22.924019+0000","last_undegraded":"2023-10-09T06:39:22.924019+0000","last_fullsized":"2023-10-09T06:39:22.924019+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:51:57.698976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.925424+0000","last_change":"2023-10-09T06:39:22.924721+0000","last_active":"2023-10-09T06:39:22.925424+0000","last_peered":"2023-10-09T06:39:22.925424+0000","last_clean":"2023-10-09T06:39:22.925424+0000","last_became_active":"2023-10-09T06:39:22.924325+0000","last_became_peered":"2023-10-09T06:39:22.924325+0000","last_unstale":"2023-10-09T06:39:22.925424+0000","last_undegraded":"2023-10-09T06:39:22.925424+0000","last_fullsized":"2023-10-09T06:39:22.925424+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:37:23.957584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.928431+0000","last_change":"2023-10-09T06:39:22.928026+0000","last_active":"2023-10-09T06:39:22.928431+0000","last_peered":"2023-10-09T06:39:22.928431+0000","last_clean":"2023-10-09T06:39:22.928431+0000","last_became_active":"2023-10-09T06:39:22.927754+0000","last_became_peered":"2023-10-09T06:39:22.927754+0000","last_unstale":"2023-10-09T06:39:22.928431+0000","last_undegraded":"2023-10-09T06:39:22.928431+0000","last_fullsized":"2023-10-09T06:39:22.928431+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:28:57.980961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924443+0000","last_change":"2023-10-09T06:39:22.921410+0000","last_active":"2023-10-09T06:39:22.924443+0000","last_peered":"2023-10-09T06:39:22.924443+0000","last_clean":"2023-10-09T06:39:22.924443+0000","last_became_active":"2023-10-09T06:39:22.921166+0000","last_became_peered":"2023-10-09T06:39:22.921166+0000","last_unstale":"2023-10-09T06:39:22.924443+0000","last_undegraded":"2023-10-09T06:39:22.924443+0000","last_fullsized":"2023-10-09T06:39:22.924443+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:55:17.506264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.925794+0000","last_change":"2023-10-09T06:39:22.921075+0000","last_active":"2023-10-09T06:39:22.925794+0000","last_peered":"2023-10-09T06:39:22.925794+0000","last_clean":"2023-10-09T06:39:22.925794+0000","last_became_active":"2023-10-09T06:39:22.920582+0000","last_became_peered":"2023-10-09T06:39:22.920582+0000","last_unstale":"2023-10-09T06:39:22.925794+0000","last_undegraded":"2023-10-09T06:39:22.925794+0000","last_fullsized":"2023-10-09T06:39:22.925794+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:06:02.686563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.925152+0000","last_change":"2023-10-09T06:39:22.923203+0000","last_active":"2023-10-09T06:39:22.925152+0000","last_peered":"2023-10-09T06:39:22.925152+0000","last_clean":"2023-10-09T06:39:22.925152+0000","last_became_active":"2023-10-09T06:39:22.922819+0000","last_became_peered":"2023-10-09T06:39:22.922819+0000","last_unstale":"2023-10-09T06:39:22.925152+0000","last_undegraded":"2023-10-09T06:39:22.925152+0000","last_fullsized":"2023-10-09T06:39:22.925152+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:29:21.434311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.927606+0000","last_change":"2023-10-09T06:39:22.920741+0000","last_active":"2023-10-09T06:39:22.927606+0000","last_peered":"2023-10-09T06:39:22.927606+0000","last_clean":"2023-10-09T06:39:22.927606+0000","last_became_active":"2023-10-09T06:39:22.920476+0000","last_became_peered":"2023-10-09T06:39:22.920476+0000","last_unstale":"2023-10-09T06:39:22.927606+0000","last_undegraded":"2023-10-09T06:39:22.927606+0000","last_fullsized":"2023-10-09T06:39:22.927606+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:52:03.783431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.927871+0000","last_change":"2023-10-09T06:39:22.922168+0000","last_active":"2023-10-09T06:39:22.927871+0000","last_peered":"2023-10-09T06:39:22.927871+0000","last_clean":"2023-10-09T06:39:22.927871+0000","last_became_active":"2023-10-09T06:39:22.921880+0000","last_became_peered":"2023-10-09T06:39:22.921880+0000","last_unstale":"2023-10-09T06:39:22.927871+0000","last_undegraded":"2023-10-09T06:39:22.927871+0000","last_fullsized":"2023-10-09T06:39:22.927871+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:23:16.252593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.925222+0000","last_change":"2023-10-09T06:39:22.923941+0000","last_active":"2023-10-09T06:39:22.925222+0000","last_peered":"2023-10-09T06:39:22.925222+0000","last_clean":"2023-10-09T06:39:22.925222+0000","last_became_active":"2023-10-09T06:39:22.923648+0000","last_became_peered":"2023-10-09T06:39:22.923648+0000","last_unstale":"2023-10-09T06:39:22.925222+0000","last_undegraded":"2023-10-09T06:39:22.925222+0000","last_fullsized":"2023-10-09T06:39:22.925222+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:55:17.506264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924734+0000","last_change":"2023-10-09T06:39:22.923005+0000","last_active":"2023-10-09T06:39:22.924734+0000","last_peered":"2023-10-09T06:39:22.924734+0000","last_clean":"2023-10-09T06:39:22.924734+0000","last_became_active":"2023-10-09T06:39:22.922737+0000","last_became_peered":"2023-10-09T06:39:22.922737+0000","last_unstale":"2023-10-09T06:39:22.924734+0000","last_undegraded":"2023-10-09T06:39:22.924734+0000","last_fullsized":"2023-10-09T06:39:22.924734+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:41:04.076555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924747+0000","last_change":"2023-10-09T06:39:22.923140+0000","last_active":"2023-10-09T06:39:22.924747+0000","last_peered":"2023-10-09T06:39:22.924747+0000","last_clean":"2023-10-09T06:39:22.924747+0000","last_became_active":"2023-10-09T06:39:22.921689+0000","last_became_peered":"2023-10-09T06:39:22.921689+0000","last_unstale":"2023-10-09T06:39:22.924747+0000","last_undegraded":"2023-10-09T06:39:22.924747+0000","last_fullsized":"2023-10-09T06:39:22.924747+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:55:24.130687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.925820+0000","last_change":"2023-10-09T06:39:22.924888+0000","last_active":"2023-10-09T06:39:22.925820+0000","last_peered":"2023-10-09T06:39:22.925820+0000","last_clean":"2023-10-09T06:39:22.925820+0000","last_became_active":"2023-10-09T06:39:22.924650+0000","last_became_peered":"2023-10-09T06:39:22.924650+0000","last_unstale":"2023-10-09T06:39:22.925820+0000","last_undegraded":"2023-10-09T06:39:22.925820+0000","last_fullsized":"2023-10-09T06:39:22.925820+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:16:35.540661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.923598+0000","last_change":"2023-10-09T06:39:22.917960+0000","last_active":"2023-10-09T06:39:22.923598+0000","last_peered":"2023-10-09T06:39:22.923598+0000","last_clean":"2023-10-09T06:39:22.923598+0000","last_became_active":"2023-10-09T06:39:22.917670+0000","last_became_peered":"2023-10-09T06:39:22.917670+0000","last_unstale":"2023-10-09T06:39:22.923598+0000","last_undegraded":"2023-10-09T06:39:22.923598+0000","last_fullsized":"2023-10-09T06:39:22.923598+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:56:58.122241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924305+0000","last_change":"2023-10-09T06:39:22.920914+0000","last_active":"2023-10-09T06:39:22.924305+0000","last_peered":"2023-10-09T06:39:22.924305+0000","last_clean":"2023-10-09T06:39:22.924305+0000","last_became_active":"2023-10-09T06:39:22.920671+0000","last_became_peered":"2023-10-09T06:39:22.920671+0000","last_unstale":"2023-10-09T06:39:22.924305+0000","last_undegraded":"2023-10-09T06:39:22.924305+0000","last_fullsized":"2023-10-09T06:39:22.924305+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:00:45.522795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.927510+0000","last_change":"2023-10-09T06:39:22.925948+0000","last_active":"2023-10-09T06:39:22.927510+0000","last_peered":"2023-10-09T06:39:22.927510+0000","last_clean":"2023-10-09T06:39:22.927510+0000","last_became_active":"2023-10-09T06:39:22.925134+0000","last_became_peered":"2023-10-09T06:39:22.925134+0000","last_unstale":"2023-10-09T06:39:22.927510+0000","last_undegraded":"2023-10-09T06:39:22.927510+0000","last_fullsized":"2023-10-09T06:39:22.927510+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:26:36.333743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.923371+0000","last_change":"2023-10-09T06:39:22.918671+0000","last_active":"2023-10-09T06:39:22.923371+0000","last_peered":"2023-10-09T06:39:22.923371+0000","last_clean":"2023-10-09T06:39:22.923371+0000","last_became_active":"2023-10-09T06:39:22.918419+0000","last_became_peered":"2023-10-09T06:39:22.918419+0000","last_unstale":"2023-10-09T06:39:22.923371+0000","last_undegraded":"2023-10-09T06:39:22.923371+0000","last_fullsized":"2023-10-09T06:39:22.923371+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:19:52.836263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.928967+0000","last_change":"2023-10-09T06:39:22.922412+0000","last_active":"2023-10-09T06:39:22.928967+0000","last_peered":"2023-10-09T06:39:22.928967+0000","last_clean":"2023-10-09T06:39:22.928967+0000","last_became_active":"2023-10-09T06:39:22.922142+0000","last_became_peered":"2023-10-09T06:39:22.922142+0000","last_unstale":"2023-10-09T06:39:22.928967+0000","last_undegraded":"2023-10-09T06:39:22.928967+0000","last_fullsized":"2023-10-09T06:39:22.928967+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:59:29.019677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924311+0000","last_change":"2023-10-09T06:39:22.919688+0000","last_active":"2023-10-09T06:39:22.924311+0000","last_peered":"2023-10-09T06:39:22.924311+0000","last_clean":"2023-10-09T06:39:22.924311+0000","last_became_active":"2023-10-09T06:39:22.919449+0000","last_became_peered":"2023-10-09T06:39:22.919449+0000","last_unstale":"2023-10-09T06:39:22.924311+0000","last_undegraded":"2023-10-09T06:39:22.924311+0000","last_fullsized":"2023-10-09T06:39:22.924311+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:54:14.749388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.923507+0000","last_change":"2023-10-09T06:39:22.920391+0000","last_active":"2023-10-09T06:39:22.923507+0000","last_peered":"2023-10-09T06:39:22.923507+0000","last_clean":"2023-10-09T06:39:22.923507+0000","last_became_active":"2023-10-09T06:39:22.918360+0000","last_became_peered":"2023-10-09T06:39:22.918360+0000","last_unstale":"2023-10-09T06:39:22.923507+0000","last_undegraded":"2023-10-09T06:39:22.923507+0000","last_fullsized":"2023-10-09T06:39:22.923507+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:20:47.054074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.925056+0000","last_change":"2023-10-09T06:39:22.922113+0000","last_active":"2023-10-09T06:39:22.925056+0000","last_peered":"2023-10-09T06:39:22.925056+0000","last_clean":"2023-10-09T06:39:22.925056+0000","last_became_active":"2023-10-09T06:39:22.921870+0000","last_became_peered":"2023-10-09T06:39:22.921870+0000","last_unstale":"2023-10-09T06:39:22.925056+0000","last_undegraded":"2023-10-09T06:39:22.925056+0000","last_fullsized":"2023-10-09T06:39:22.925056+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:56:13.775564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.930217+0000","last_change":"2023-10-09T06:39:22.925514+0000","last_active":"2023-10-09T06:39:22.930217+0000","last_peered":"2023-10-09T06:39:22.930217+0000","last_clean":"2023-10-09T06:39:22.930217+0000","last_became_active":"2023-10-09T06:39:22.925240+0000","last_became_peered":"2023-10-09T06:39:22.925240+0000","last_unstale":"2023-10-09T06:39:22.930217+0000","last_undegraded":"2023-10-09T06:39:22.930217+0000","last_fullsized":"2023-10-09T06:39:22.930217+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:23:48.469889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.926126+0000","last_change":"2023-10-09T06:39:22.921875+0000","last_active":"2023-10-09T06:39:22.926126+0000","last_peered":"2023-10-09T06:39:22.926126+0000","last_clean":"2023-10-09T06:39:22.926126+0000","last_became_active":"2023-10-09T06:39:22.921587+0000","last_became_peered":"2023-10-09T06:39:22.921587+0000","last_unstale":"2023-10-09T06:39:22.926126+0000","last_undegraded":"2023-10-09T06:39:22.926126+0000","last_fullsized":"2023-10-09T06:39:22.926126+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:40:35.649919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.923939+0000","last_change":"2023-10-09T06:39:22.921914+0000","last_active":"2023-10-09T06:39:22.923939+0000","last_peered":"2023-10-09T06:39:22.923939+0000","last_clean":"2023-10-09T06:39:22.923939+0000","last_became_active":"2023-10-09T06:39:22.921651+0000","last_became_peered":"2023-10-09T06:39:22.921651+0000","last_unstale":"2023-10-09T06:39:22.923939+0000","last_undegraded":"2023-10-09T06:39:22.923939+0000","last_fullsized":"2023-10-09T06:39:22.923939+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:24:30.093575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.933391+0000","last_change":"2023-10-09T06:39:22.927247+0000","last_active":"2023-10-09T06:39:22.933391+0000","last_peered":"2023-10-09T06:39:22.933391+0000","last_clean":"2023-10-09T06:39:22.933391+0000","last_became_active":"2023-10-09T06:39:22.926991+0000","last_became_peered":"2023-10-09T06:39:22.926991+0000","last_unstale":"2023-10-09T06:39:22.933391+0000","last_undegraded":"2023-10-09T06:39:22.933391+0000","last_fullsized":"2023-10-09T06:39:22.933391+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:00:20.989336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.919641+0000","last_change":"2023-10-09T06:39:22.915060+0000","last_active":"2023-10-09T06:39:22.919641+0000","last_peered":"2023-10-09T06:39:22.919641+0000","last_clean":"2023-10-09T06:39:22.919641+0000","last_became_active":"2023-10-09T06:39:22.914812+0000","last_became_peered":"2023-10-09T06:39:22.914812+0000","last_unstale":"2023-10-09T06:39:22.919641+0000","last_undegraded":"2023-10-09T06:39:22.919641+0000","last_fullsized":"2023-10-09T06:39:22.919641+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:46:35.056481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.927696+0000","last_change":"2023-10-09T06:39:22.921428+0000","last_active":"2023-10-09T06:39:22.927696+0000","last_peered":"2023-10-09T06:39:22.927696+0000","last_clean":"2023-10-09T06:39:22.927696+0000","last_became_active":"2023-10-09T06:39:22.921131+0000","last_became_peered":"2023-10-09T06:39:22.921131+0000","last_unstale":"2023-10-09T06:39:22.927696+0000","last_undegraded":"2023-10-09T06:39:22.927696+0000","last_fullsized":"2023-10-09T06:39:22.927696+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:22:51.139705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.934313+0000","last_change":"2023-10-09T06:39:22.927263+0000","last_active":"2023-10-09T06:39:22.934313+0000","last_peered":"2023-10-09T06:39:22.934313+0000","last_clean":"2023-10-09T06:39:22.934313+0000","last_became_active":"2023-10-09T06:39:22.927000+0000","last_became_peered":"2023-10-09T06:39:22.927000+0000","last_unstale":"2023-10-09T06:39:22.934313+0000","last_undegraded":"2023-10-09T06:39:22.934313+0000","last_fullsized":"2023-10-09T06:39:22.934313+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:27:06.948168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.925124+0000","last_change":"2023-10-09T06:39:22.923055+0000","last_active":"2023-10-09T06:39:22.925124+0000","last_peered":"2023-10-09T06:39:22.925124+0000","last_clean":"2023-10-09T06:39:22.925124+0000","last_became_active":"2023-10-09T06:39:22.922789+0000","last_became_peered":"2023-10-09T06:39:22.922789+0000","last_unstale":"2023-10-09T06:39:22.925124+0000","last_undegraded":"2023-10-09T06:39:22.925124+0000","last_fullsized":"2023-10-09T06:39:22.925124+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:56:53.405665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.914096+0000","last_change":"2023-10-09T06:39:22.913361+0000","last_active":"2023-10-09T06:39:22.914096+0000","last_peered":"2023-10-09T06:39:22.914096+0000","last_clean":"2023-10-09T06:39:22.914096+0000","last_became_active":"2023-10-09T06:39:22.913123+0000","last_became_peered":"2023-10-09T06:39:22.913123+0000","last_unstale":"2023-10-09T06:39:22.914096+0000","last_undegraded":"2023-10-09T06:39:22.914096+0000","last_fullsized":"2023-10-09T06:39:22.914096+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:17:30.457482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924321+0000","last_change":"2023-10-09T06:39:22.922363+0000","last_active":"2023-10-09T06:39:22.924321+0000","last_peered":"2023-10-09T06:39:22.924321+0000","last_clean":"2023-10-09T06:39:22.924321+0000","last_became_active":"2023-10-09T06:39:22.922096+0000","last_became_peered":"2023-10-09T06:39:22.922096+0000","last_unstale":"2023-10-09T06:39:22.924321+0000","last_undegraded":"2023-10-09T06:39:22.924321+0000","last_fullsized":"2023-10-09T06:39:22.924321+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:16:25.063143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924975+0000","last_change":"2023-10-09T06:39:22.921159+0000","last_active":"2023-10-09T06:39:22.924975+0000","last_peered":"2023-10-09T06:39:22.924975+0000","last_clean":"2023-10-09T06:39:22.924975+0000","last_became_active":"2023-10-09T06:39:22.920910+0000","last_became_peered":"2023-10-09T06:39:22.920910+0000","last_unstale":"2023-10-09T06:39:22.924975+0000","last_undegraded":"2023-10-09T06:39:22.924975+0000","last_fullsized":"2023-10-09T06:39:22.924975+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:45:54.220566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.926031+0000","last_change":"2023-10-09T06:39:22.925186+0000","last_active":"2023-10-09T06:39:22.926031+0000","last_peered":"2023-10-09T06:39:22.926031+0000","last_clean":"2023-10-09T06:39:22.926031+0000","last_became_active":"2023-10-09T06:39:22.924944+0000","last_became_peered":"2023-10-09T06:39:22.924944+0000","last_unstale":"2023-10-09T06:39:22.926031+0000","last_undegraded":"2023-10-09T06:39:22.926031+0000","last_fullsized":"2023-10-09T06:39:22.926031+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:41:04.076555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924988+0000","last_change":"2023-10-09T06:39:22.922371+0000","last_active":"2023-10-09T06:39:22.924988+0000","last_peered":"2023-10-09T06:39:22.924988+0000","last_clean":"2023-10-09T06:39:22.924988+0000","last_became_active":"2023-10-09T06:39:22.920746+0000","last_became_peered":"2023-10-09T06:39:22.920746+0000","last_unstale":"2023-10-09T06:39:22.924988+0000","last_undegraded":"2023-10-09T06:39:22.924988+0000","last_fullsized":"2023-10-09T06:39:22.924988+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:38:29.919819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"181'4","reported_seq":14,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.937045+0000","last_change":"2023-10-09T06:39:21.914681+0000","last_active":"2023-10-09T06:39:22.937045+0000","last_peered":"2023-10-09T06:39:22.937045+0000","last_clean":"2023-10-09T06:39:22.937045+0000","last_became_active":"2023-10-09T06:39:21.914359+0000","last_became_peered":"2023-10-09T06:39:21.914359+0000","last_unstale":"2023-10-09T06:39:22.937045+0000","last_undegraded":"2023-10-09T06:39:22.937045+0000","last_fullsized":"2023-10-09T06:39:22.937045+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:28:38.998280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.923964+0000","last_change":"2023-10-09T06:39:22.923224+0000","last_active":"2023-10-09T06:39:22.923964+0000","last_peered":"2023-10-09T06:39:22.923964+0000","last_clean":"2023-10-09T06:39:22.923964+0000","last_became_active":"2023-10-09T06:39:22.922967+0000","last_became_peered":"2023-10-09T06:39:22.922967+0000","last_unstale":"2023-10-09T06:39:22.923964+0000","last_undegraded":"2023-10-09T06:39:22.923964+0000","last_fullsized":"2023-10-09T06:39:22.923964+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:50:15.511899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"181'2","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.934816+0000","last_change":"2023-10-09T06:39:21.908996+0000","last_active":"2023-10-09T06:39:22.934816+0000","last_peered":"2023-10-09T06:39:22.934816+0000","last_clean":"2023-10-09T06:39:22.934816+0000","last_became_active":"2023-10-09T06:39:21.908602+0000","last_became_peered":"2023-10-09T06:39:21.908602+0000","last_unstale":"2023-10-09T06:39:22.934816+0000","last_undegraded":"2023-10-09T06:39:22.934816+0000","last_fullsized":"2023-10-09T06:39:22.934816+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:43:06.663080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.925196+0000","last_change":"2023-10-09T06:39:22.923563+0000","last_active":"2023-10-09T06:39:22.925196+0000","last_peered":"2023-10-09T06:39:22.925196+0000","last_clean":"2023-10-09T06:39:22.925196+0000","last_became_active":"2023-10-09T06:39:22.923268+0000","last_became_peered":"2023-10-09T06:39:22.923268+0000","last_unstale":"2023-10-09T06:39:22.925196+0000","last_undegraded":"2023-10-09T06:39:22.925196+0000","last_fullsized":"2023-10-09T06:39:22.925196+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:34:08.783653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"181'2","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.940492+0000","last_change":"2023-10-09T06:39:21.910385+0000","last_active":"2023-10-09T06:39:22.940492+0000","last_peered":"2023-10-09T06:39:22.940492+0000","last_clean":"2023-10-09T06:39:22.940492+0000","last_became_active":"2023-10-09T06:39:21.910135+0000","last_became_peered":"2023-10-09T06:39:21.910135+0000","last_unstale":"2023-10-09T06:39:22.940492+0000","last_undegraded":"2023-10-09T06:39:22.940492+0000","last_fullsized":"2023-10-09T06:39:22.940492+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:20:46.047881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.916219+0000","last_change":"2023-10-09T06:39:22.914972+0000","last_active":"2023-10-09T06:39:22.916219+0000","last_peered":"2023-10-09T06:39:22.916219+0000","last_clean":"2023-10-09T06:39:22.916219+0000","last_became_active":"2023-10-09T06:39:22.914730+0000","last_became_peered":"2023-10-09T06:39:22.914730+0000","last_unstale":"2023-10-09T06:39:22.916219+0000","last_undegraded":"2023-10-09T06:39:22.916219+0000","last_fullsized":"2023-10-09T06:39:22.916219+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:11:29.767572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"181'1","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.934237+0000","last_change":"2023-10-09T06:39:21.909550+0000","last_active":"2023-10-09T06:39:22.934237+0000","last_peered":"2023-10-09T06:39:22.934237+0000","last_clean":"2023-10-09T06:39:22.934237+0000","last_became_active":"2023-10-09T06:39:21.909172+0000","last_became_peered":"2023-10-09T06:39:21.909172+0000","last_unstale":"2023-10-09T06:39:22.934237+0000","last_undegraded":"2023-10-09T06:39:22.934237+0000","last_fullsized":"2023-10-09T06:39:22.934237+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:14:26.618748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.933648+0000","last_change":"2023-10-09T06:39:22.928252+0000","last_active":"2023-10-09T06:39:22.933648+0000","last_peered":"2023-10-09T06:39:22.933648+0000","last_clean":"2023-10-09T06:39:22.933648+0000","last_became_active":"2023-10-09T06:39:22.925747+0000","last_became_peered":"2023-10-09T06:39:22.925747+0000","last_unstale":"2023-10-09T06:39:22.933648+0000","last_undegraded":"2023-10-09T06:39:22.933648+0000","last_fullsized":"2023-10-09T06:39:22.933648+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:46:41.535238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"181'1","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.940363+0000","last_change":"2023-10-09T06:39:21.907441+0000","last_active":"2023-10-09T06:39:22.940363+0000","last_peered":"2023-10-09T06:39:22.940363+0000","last_clean":"2023-10-09T06:39:22.940363+0000","last_became_active":"2023-10-09T06:39:21.907190+0000","last_became_peered":"2023-10-09T06:39:21.907190+0000","last_unstale":"2023-10-09T06:39:22.940363+0000","last_undegraded":"2023-10-09T06:39:22.940363+0000","last_fullsized":"2023-10-09T06:39:22.940363+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:54:27.546478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924422+0000","last_change":"2023-10-09T06:39:22.921304+0000","last_active":"2023-10-09T06:39:22.924422+0000","last_peered":"2023-10-09T06:39:22.924422+0000","last_clean":"2023-10-09T06:39:22.924422+0000","last_became_active":"2023-10-09T06:39:22.921090+0000","last_became_peered":"2023-10-09T06:39:22.921090+0000","last_unstale":"2023-10-09T06:39:22.924422+0000","last_undegraded":"2023-10-09T06:39:22.924422+0000","last_fullsized":"2023-10-09T06:39:22.924422+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:12:53.364651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.913770+0000","last_change":"2023-10-09T06:39:21.912198+0000","last_active":"2023-10-09T06:39:22.913770+0000","last_peered":"2023-10-09T06:39:22.913770+0000","last_clean":"2023-10-09T06:39:22.913770+0000","last_became_active":"2023-10-09T06:39:21.911792+0000","last_became_peered":"2023-10-09T06:39:21.911792+0000","last_unstale":"2023-10-09T06:39:22.913770+0000","last_undegraded":"2023-10-09T06:39:22.913770+0000","last_fullsized":"2023-10-09T06:39:22.913770+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:26:35.327550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.933816+0000","last_change":"2023-10-09T06:39:22.928755+0000","last_active":"2023-10-09T06:39:22.933816+0000","last_peered":"2023-10-09T06:39:22.933816+0000","last_clean":"2023-10-09T06:39:22.933816+0000","last_became_active":"2023-10-09T06:39:22.928456+0000","last_became_peered":"2023-10-09T06:39:22.928456+0000","last_unstale":"2023-10-09T06:39:22.933816+0000","last_undegraded":"2023-10-09T06:39:22.933816+0000","last_fullsized":"2023-10-09T06:39:22.933816+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:05:19.341633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.913641+0000","last_change":"2023-10-09T06:39:21.910184+0000","last_active":"2023-10-09T06:39:22.913641+0000","last_peered":"2023-10-09T06:39:22.913641+0000","last_clean":"2023-10-09T06:39:22.913641+0000","last_became_active":"2023-10-09T06:39:21.909867+0000","last_became_peered":"2023-10-09T06:39:21.909867+0000","last_unstale":"2023-10-09T06:39:22.913641+0000","last_undegraded":"2023-10-09T06:39:22.913641+0000","last_fullsized":"2023-10-09T06:39:22.913641+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:15:31.981803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.933646+0000","last_change":"2023-10-09T06:39:22.928119+0000","last_active":"2023-10-09T06:39:22.933646+0000","last_peered":"2023-10-09T06:39:22.933646+0000","last_clean":"2023-10-09T06:39:22.933646+0000","last_became_active":"2023-10-09T06:39:22.927810+0000","last_became_peered":"2023-10-09T06:39:22.927810+0000","last_unstale":"2023-10-09T06:39:22.933646+0000","last_undegraded":"2023-10-09T06:39:22.933646+0000","last_fullsized":"2023-10-09T06:39:22.933646+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:28:11.548143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.908986+0000","last_change":"2023-10-09T06:39:21.911407+0000","last_active":"2023-10-09T06:39:22.908986+0000","last_peered":"2023-10-09T06:39:22.908986+0000","last_clean":"2023-10-09T06:39:22.908986+0000","last_became_active":"2023-10-09T06:39:21.911115+0000","last_became_peered":"2023-10-09T06:39:21.911115+0000","last_unstale":"2023-10-09T06:39:22.908986+0000","last_undegraded":"2023-10-09T06:39:22.908986+0000","last_fullsized":"2023-10-09T06:39:22.908986+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:21:31.967962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.926054+0000","last_change":"2023-10-09T06:39:22.921859+0000","last_active":"2023-10-09T06:39:22.926054+0000","last_peered":"2023-10-09T06:39:22.926054+0000","last_clean":"2023-10-09T06:39:22.926054+0000","last_became_active":"2023-10-09T06:39:22.921542+0000","last_became_peered":"2023-10-09T06:39:22.921542+0000","last_unstale":"2023-10-09T06:39:22.926054+0000","last_undegraded":"2023-10-09T06:39:22.926054+0000","last_fullsized":"2023-10-09T06:39:22.926054+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:41:28.129645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.914596+0000","last_change":"2023-10-09T06:39:21.905424+0000","last_active":"2023-10-09T06:39:22.914596+0000","last_peered":"2023-10-09T06:39:22.914596+0000","last_clean":"2023-10-09T06:39:22.914596+0000","last_became_active":"2023-10-09T06:39:21.905164+0000","last_became_peered":"2023-10-09T06:39:21.905164+0000","last_unstale":"2023-10-09T06:39:22.914596+0000","last_undegraded":"2023-10-09T06:39:22.914596+0000","last_fullsized":"2023-10-09T06:39:22.914596+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:21:31.967962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924825+0000","last_change":"2023-10-09T06:39:22.921960+0000","last_active":"2023-10-09T06:39:22.924825+0000","last_peered":"2023-10-09T06:39:22.924825+0000","last_clean":"2023-10-09T06:39:22.924825+0000","last_became_active":"2023-10-09T06:39:22.921703+0000","last_became_peered":"2023-10-09T06:39:22.921703+0000","last_unstale":"2023-10-09T06:39:22.924825+0000","last_undegraded":"2023-10-09T06:39:22.924825+0000","last_fullsized":"2023-10-09T06:39:22.924825+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:54:58.571706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"181'3","reported_seq":13,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.938498+0000","last_change":"2023-10-09T06:39:21.906036+0000","last_active":"2023-10-09T06:39:22.938498+0000","last_peered":"2023-10-09T06:39:22.938498+0000","last_clean":"2023-10-09T06:39:22.938498+0000","last_became_active":"2023-10-09T06:39:21.905785+0000","last_became_peered":"2023-10-09T06:39:21.905785+0000","last_unstale":"2023-10-09T06:39:22.938498+0000","last_undegraded":"2023-10-09T06:39:22.938498+0000","last_fullsized":"2023-10-09T06:39:22.938498+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:00:19.983143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.930239+0000","last_change":"2023-10-09T06:39:22.925546+0000","last_active":"2023-10-09T06:39:22.930239+0000","last_peered":"2023-10-09T06:39:22.930239+0000","last_clean":"2023-10-09T06:39:22.930239+0000","last_became_active":"2023-10-09T06:39:22.925250+0000","last_became_peered":"2023-10-09T06:39:22.925250+0000","last_unstale":"2023-10-09T06:39:22.930239+0000","last_undegraded":"2023-10-09T06:39:22.930239+0000","last_fullsized":"2023-10-09T06:39:22.930239+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:43:07.669273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"181'6","reported_seq":16,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.941108+0000","last_change":"2023-10-09T06:39:21.906268+0000","last_active":"2023-10-09T06:39:22.941108+0000","last_peered":"2023-10-09T06:39:22.941108+0000","last_clean":"2023-10-09T06:39:22.941108+0000","last_became_active":"2023-10-09T06:39:21.905999+0000","last_became_peered":"2023-10-09T06:39:21.905999+0000","last_unstale":"2023-10-09T06:39:22.941108+0000","last_undegraded":"2023-10-09T06:39:22.941108+0000","last_fullsized":"2023-10-09T06:39:22.941108+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:35:15.234596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.923496+0000","last_change":"2023-10-09T06:39:22.918236+0000","last_active":"2023-10-09T06:39:22.923496+0000","last_peered":"2023-10-09T06:39:22.923496+0000","last_clean":"2023-10-09T06:39:22.923496+0000","last_became_active":"2023-10-09T06:39:22.917863+0000","last_became_peered":"2023-10-09T06:39:22.917863+0000","last_unstale":"2023-10-09T06:39:22.923496+0000","last_undegraded":"2023-10-09T06:39:22.923496+0000","last_fullsized":"2023-10-09T06:39:22.923496+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:17:53.105183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"181'2","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.937388+0000","last_change":"2023-10-09T06:39:21.912929+0000","last_active":"2023-10-09T06:39:22.937388+0000","last_peered":"2023-10-09T06:39:22.937388+0000","last_clean":"2023-10-09T06:39:22.937388+0000","last_became_active":"2023-10-09T06:39:21.912692+0000","last_became_peered":"2023-10-09T06:39:21.912692+0000","last_unstale":"2023-10-09T06:39:22.937388+0000","last_undegraded":"2023-10-09T06:39:22.937388+0000","last_fullsized":"2023-10-09T06:39:22.937388+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:23:16.266975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.933624+0000","last_change":"2023-10-09T06:39:22.927379+0000","last_active":"2023-10-09T06:39:22.933624+0000","last_peered":"2023-10-09T06:39:22.933624+0000","last_clean":"2023-10-09T06:39:22.933624+0000","last_became_active":"2023-10-09T06:39:22.926026+0000","last_became_peered":"2023-10-09T06:39:22.926026+0000","last_unstale":"2023-10-09T06:39:22.933624+0000","last_undegraded":"2023-10-09T06:39:22.933624+0000","last_fullsized":"2023-10-09T06:39:22.933624+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:52:16.785202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.916166+0000","last_change":"2023-10-09T06:39:21.908941+0000","last_active":"2023-10-09T06:39:22.916166+0000","last_peered":"2023-10-09T06:39:22.916166+0000","last_clean":"2023-10-09T06:39:22.916166+0000","last_became_active":"2023-10-09T06:39:21.908631+0000","last_became_peered":"2023-10-09T06:39:21.908631+0000","last_unstale":"2023-10-09T06:39:22.916166+0000","last_undegraded":"2023-10-09T06:39:22.916166+0000","last_fullsized":"2023-10-09T06:39:22.916166+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:50:24.784960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.927345+0000","last_change":"2023-10-09T06:39:22.925564+0000","last_active":"2023-10-09T06:39:22.927345+0000","last_peered":"2023-10-09T06:39:22.927345+0000","last_clean":"2023-10-09T06:39:22.927345+0000","last_became_active":"2023-10-09T06:39:22.925329+0000","last_became_peered":"2023-10-09T06:39:22.925329+0000","last_unstale":"2023-10-09T06:39:22.927345+0000","last_undegraded":"2023-10-09T06:39:22.927345+0000","last_fullsized":"2023-10-09T06:39:22.927345+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:54:28.552671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.911903+0000","last_change":"2023-10-09T06:39:21.907716+0000","last_active":"2023-10-09T06:39:22.911903+0000","last_peered":"2023-10-09T06:39:22.911903+0000","last_clean":"2023-10-09T06:39:22.911903+0000","last_became_active":"2023-10-09T06:39:21.907467+0000","last_became_peered":"2023-10-09T06:39:21.907467+0000","last_unstale":"2023-10-09T06:39:22.911903+0000","last_undegraded":"2023-10-09T06:39:22.911903+0000","last_fullsized":"2023-10-09T06:39:22.911903+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:03:25.178538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.933798+0000","last_change":"2023-10-09T06:39:22.925676+0000","last_active":"2023-10-09T06:39:22.933798+0000","last_peered":"2023-10-09T06:39:22.933798+0000","last_clean":"2023-10-09T06:39:22.933798+0000","last_became_active":"2023-10-09T06:39:22.925439+0000","last_became_peered":"2023-10-09T06:39:22.925439+0000","last_unstale":"2023-10-09T06:39:22.933798+0000","last_undegraded":"2023-10-09T06:39:22.933798+0000","last_fullsized":"2023-10-09T06:39:22.933798+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:24:34.424297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"181'8","reported_seq":18,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.939258+0000","last_change":"2023-10-09T06:39:21.903661+0000","last_active":"2023-10-09T06:39:22.939258+0000","last_peered":"2023-10-09T06:39:22.939258+0000","last_clean":"2023-10-09T06:39:22.939258+0000","last_became_active":"2023-10-09T06:39:21.903407+0000","last_became_peered":"2023-10-09T06:39:21.903407+0000","last_unstale":"2023-10-09T06:39:22.939258+0000","last_undegraded":"2023-10-09T06:39:22.939258+0000","last_fullsized":"2023-10-09T06:39:22.939258+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:00:13.235101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.921855+0000","last_change":"2023-10-09T06:39:22.919313+0000","last_active":"2023-10-09T06:39:22.921855+0000","last_peered":"2023-10-09T06:39:22.921855+0000","last_clean":"2023-10-09T06:39:22.921855+0000","last_became_active":"2023-10-09T06:39:22.919054+0000","last_became_peered":"2023-10-09T06:39:22.919054+0000","last_unstale":"2023-10-09T06:39:22.921855+0000","last_undegraded":"2023-10-09T06:39:22.921855+0000","last_fullsized":"2023-10-09T06:39:22.921855+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:34:26.593928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"181'1","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.938177+0000","last_change":"2023-10-09T06:39:21.912159+0000","last_active":"2023-10-09T06:39:22.938177+0000","last_peered":"2023-10-09T06:39:22.938177+0000","last_clean":"2023-10-09T06:39:22.938177+0000","last_became_active":"2023-10-09T06:39:21.911920+0000","last_became_peered":"2023-10-09T06:39:21.911920+0000","last_unstale":"2023-10-09T06:39:22.938177+0000","last_undegraded":"2023-10-09T06:39:22.938177+0000","last_fullsized":"2023-10-09T06:39:22.938177+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:16:24.056950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.921865+0000","last_change":"2023-10-09T06:39:22.918953+0000","last_active":"2023-10-09T06:39:22.921865+0000","last_peered":"2023-10-09T06:39:22.921865+0000","last_clean":"2023-10-09T06:39:22.921865+0000","last_became_active":"2023-10-09T06:39:22.918688+0000","last_became_peered":"2023-10-09T06:39:22.918688+0000","last_unstale":"2023-10-09T06:39:22.921865+0000","last_undegraded":"2023-10-09T06:39:22.921865+0000","last_fullsized":"2023-10-09T06:39:22.921865+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:08:03.674524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"181'2","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.941300+0000","last_change":"2023-10-09T06:39:21.907852+0000","last_active":"2023-10-09T06:39:22.941300+0000","last_peered":"2023-10-09T06:39:22.941300+0000","last_clean":"2023-10-09T06:39:22.941300+0000","last_became_active":"2023-10-09T06:39:21.907554+0000","last_became_peered":"2023-10-09T06:39:21.907554+0000","last_unstale":"2023-10-09T06:39:22.941300+0000","last_undegraded":"2023-10-09T06:39:22.941300+0000","last_fullsized":"2023-10-09T06:39:22.941300+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:01:38.038864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924935+0000","last_change":"2023-10-09T06:39:22.922275+0000","last_active":"2023-10-09T06:39:22.924935+0000","last_peered":"2023-10-09T06:39:22.924935+0000","last_clean":"2023-10-09T06:39:22.924935+0000","last_became_active":"2023-10-09T06:39:22.922012+0000","last_became_peered":"2023-10-09T06:39:22.922012+0000","last_unstale":"2023-10-09T06:39:22.924935+0000","last_undegraded":"2023-10-09T06:39:22.924935+0000","last_fullsized":"2023-10-09T06:39:22.924935+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:35:07.790241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"181'1","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.934760+0000","last_change":"2023-10-09T06:39:21.908834+0000","last_active":"2023-10-09T06:39:22.934760+0000","last_peered":"2023-10-09T06:39:22.934760+0000","last_clean":"2023-10-09T06:39:22.934760+0000","last_became_active":"2023-10-09T06:39:21.908235+0000","last_became_peered":"2023-10-09T06:39:21.908235+0000","last_unstale":"2023-10-09T06:39:22.934760+0000","last_undegraded":"2023-10-09T06:39:22.934760+0000","last_fullsized":"2023-10-09T06:39:22.934760+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:28:10.541950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.926965+0000","last_change":"2023-10-09T06:39:22.926361+0000","last_active":"2023-10-09T06:39:22.926965+0000","last_peered":"2023-10-09T06:39:22.926965+0000","last_clean":"2023-10-09T06:39:22.926965+0000","last_became_active":"2023-10-09T06:39:22.926100+0000","last_became_peered":"2023-10-09T06:39:22.926100+0000","last_unstale":"2023-10-09T06:39:22.926965+0000","last_undegraded":"2023-10-09T06:39:22.926965+0000","last_fullsized":"2023-10-09T06:39:22.926965+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:29:27.307841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.916782+0000","last_change":"2023-10-09T06:39:21.906627+0000","last_active":"2023-10-09T06:39:22.916782+0000","last_peered":"2023-10-09T06:39:22.916782+0000","last_clean":"2023-10-09T06:39:22.916782+0000","last_became_active":"2023-10-09T06:39:21.906358+0000","last_became_peered":"2023-10-09T06:39:21.906358+0000","last_unstale":"2023-10-09T06:39:22.916782+0000","last_undegraded":"2023-10-09T06:39:22.916782+0000","last_fullsized":"2023-10-09T06:39:22.916782+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:03:49.588861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.923554+0000","last_change":"2023-10-09T06:39:22.919897+0000","last_active":"2023-10-09T06:39:22.923554+0000","last_peered":"2023-10-09T06:39:22.923554+0000","last_clean":"2023-10-09T06:39:22.923554+0000","last_became_active":"2023-10-09T06:39:22.919672+0000","last_became_peered":"2023-10-09T06:39:22.919672+0000","last_unstale":"2023-10-09T06:39:22.923554+0000","last_undegraded":"2023-10-09T06:39:22.923554+0000","last_fullsized":"2023-10-09T06:39:22.923554+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:14:37.831447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"181'2","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.938088+0000","last_change":"2023-10-09T06:39:21.905819+0000","last_active":"2023-10-09T06:39:22.938088+0000","last_peered":"2023-10-09T06:39:22.938088+0000","last_clean":"2023-10-09T06:39:22.938088+0000","last_became_active":"2023-10-09T06:39:21.905562+0000","last_became_peered":"2023-10-09T06:39:21.905562+0000","last_unstale":"2023-10-09T06:39:22.938088+0000","last_undegraded":"2023-10-09T06:39:22.938088+0000","last_fullsized":"2023-10-09T06:39:22.938088+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:00:38.917682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924624+0000","last_change":"2023-10-09T06:39:22.921748+0000","last_active":"2023-10-09T06:39:22.924624+0000","last_peered":"2023-10-09T06:39:22.924624+0000","last_clean":"2023-10-09T06:39:22.924624+0000","last_became_active":"2023-10-09T06:39:22.921450+0000","last_became_peered":"2023-10-09T06:39:22.921450+0000","last_unstale":"2023-10-09T06:39:22.924624+0000","last_undegraded":"2023-10-09T06:39:22.924624+0000","last_fullsized":"2023-10-09T06:39:22.924624+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:53:58.058707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.913239+0000","last_change":"2023-10-09T06:39:21.909520+0000","last_active":"2023-10-09T06:39:22.913239+0000","last_peered":"2023-10-09T06:39:22.913239+0000","last_clean":"2023-10-09T06:39:22.913239+0000","last_became_active":"2023-10-09T06:39:21.909250+0000","last_became_peered":"2023-10-09T06:39:21.909250+0000","last_unstale":"2023-10-09T06:39:22.913239+0000","last_undegraded":"2023-10-09T06:39:22.913239+0000","last_fullsized":"2023-10-09T06:39:22.913239+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:22:50.133512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.927713+0000","last_change":"2023-10-09T06:39:22.920313+0000","last_active":"2023-10-09T06:39:22.927713+0000","last_peered":"2023-10-09T06:39:22.927713+0000","last_clean":"2023-10-09T06:39:22.927713+0000","last_became_active":"2023-10-09T06:39:22.920017+0000","last_became_peered":"2023-10-09T06:39:22.920017+0000","last_unstale":"2023-10-09T06:39:22.927713+0000","last_undegraded":"2023-10-09T06:39:22.927713+0000","last_fullsized":"2023-10-09T06:39:22.927713+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:04:35.088573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.911778+0000","last_change":"2023-10-09T06:39:21.909639+0000","last_active":"2023-10-09T06:39:22.911778+0000","last_peered":"2023-10-09T06:39:22.911778+0000","last_clean":"2023-10-09T06:39:22.911778+0000","last_became_active":"2023-10-09T06:39:21.909361+0000","last_became_peered":"2023-10-09T06:39:21.909361+0000","last_unstale":"2023-10-09T06:39:22.911778+0000","last_undegraded":"2023-10-09T06:39:22.911778+0000","last_fullsized":"2023-10-09T06:39:22.911778+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:23:47.463696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.928944+0000","last_change":"2023-10-09T06:39:22.919880+0000","last_active":"2023-10-09T06:39:22.928944+0000","last_peered":"2023-10-09T06:39:22.928944+0000","last_clean":"2023-10-09T06:39:22.928944+0000","last_became_active":"2023-10-09T06:39:22.919611+0000","last_became_peered":"2023-10-09T06:39:22.919611+0000","last_unstale":"2023-10-09T06:39:22.928944+0000","last_undegraded":"2023-10-09T06:39:22.928944+0000","last_fullsized":"2023-10-09T06:39:22.928944+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:12:17.467994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.908522+0000","last_change":"2023-10-09T06:39:21.911934+0000","last_active":"2023-10-09T06:39:22.908522+0000","last_peered":"2023-10-09T06:39:22.908522+0000","last_clean":"2023-10-09T06:39:22.908522+0000","last_became_active":"2023-10-09T06:39:21.911682+0000","last_became_peered":"2023-10-09T06:39:21.911682+0000","last_unstale":"2023-10-09T06:39:22.908522+0000","last_undegraded":"2023-10-09T06:39:22.908522+0000","last_fullsized":"2023-10-09T06:39:22.908522+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:26:35.327550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.934248+0000","last_change":"2023-10-09T06:39:22.928672+0000","last_active":"2023-10-09T06:39:22.934248+0000","last_peered":"2023-10-09T06:39:22.934248+0000","last_clean":"2023-10-09T06:39:22.934248+0000","last_became_active":"2023-10-09T06:39:22.928355+0000","last_became_peered":"2023-10-09T06:39:22.928355+0000","last_unstale":"2023-10-09T06:39:22.934248+0000","last_undegraded":"2023-10-09T06:39:22.934248+0000","last_fullsized":"2023-10-09T06:39:22.934248+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:58:36.972142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"181'2","reported_seq":12,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.934569+0000","last_change":"2023-10-09T06:39:21.905701+0000","last_active":"2023-10-09T06:39:22.934569+0000","last_peered":"2023-10-09T06:39:22.934569+0000","last_clean":"2023-10-09T06:39:22.934569+0000","last_became_active":"2023-10-09T06:39:21.905461+0000","last_became_peered":"2023-10-09T06:39:21.905461+0000","last_unstale":"2023-10-09T06:39:22.934569+0000","last_undegraded":"2023-10-09T06:39:22.934569+0000","last_fullsized":"2023-10-09T06:39:22.934569+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:24:29.087382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.927492+0000","last_change":"2023-10-09T06:39:22.925393+0000","last_active":"2023-10-09T06:39:22.927492+0000","last_peered":"2023-10-09T06:39:22.927492+0000","last_clean":"2023-10-09T06:39:22.927492+0000","last_became_active":"2023-10-09T06:39:22.924748+0000","last_became_peered":"2023-10-09T06:39:22.924748+0000","last_unstale":"2023-10-09T06:39:22.927492+0000","last_undegraded":"2023-10-09T06:39:22.927492+0000","last_fullsized":"2023-10-09T06:39:22.927492+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:22:00.814515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.910321+0000","last_change":"2023-10-09T06:39:21.906395+0000","last_active":"2023-10-09T06:39:22.910321+0000","last_peered":"2023-10-09T06:39:22.910321+0000","last_clean":"2023-10-09T06:39:22.910321+0000","last_became_active":"2023-10-09T06:39:21.906073+0000","last_became_peered":"2023-10-09T06:39:21.906073+0000","last_unstale":"2023-10-09T06:39:22.910321+0000","last_undegraded":"2023-10-09T06:39:22.910321+0000","last_fullsized":"2023-10-09T06:39:22.910321+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:10:38.345541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924409+0000","last_change":"2023-10-09T06:39:22.920901+0000","last_active":"2023-10-09T06:39:22.924409+0000","last_peered":"2023-10-09T06:39:22.924409+0000","last_clean":"2023-10-09T06:39:22.924409+0000","last_became_active":"2023-10-09T06:39:22.920668+0000","last_became_peered":"2023-10-09T06:39:22.920668+0000","last_unstale":"2023-10-09T06:39:22.924409+0000","last_undegraded":"2023-10-09T06:39:22.924409+0000","last_fullsized":"2023-10-09T06:39:22.924409+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:29:21.434311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.908625+0000","last_change":"2023-10-09T06:39:21.908920+0000","last_active":"2023-10-09T06:39:22.908625+0000","last_peered":"2023-10-09T06:39:22.908625+0000","last_clean":"2023-10-09T06:39:22.908625+0000","last_became_active":"2023-10-09T06:39:21.908640+0000","last_became_peered":"2023-10-09T06:39:21.908640+0000","last_unstale":"2023-10-09T06:39:22.908625+0000","last_undegraded":"2023-10-09T06:39:22.908625+0000","last_fullsized":"2023-10-09T06:39:22.908625+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:24:29.087382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.925812+0000","last_change":"2023-10-09T06:39:22.921370+0000","last_active":"2023-10-09T06:39:22.925812+0000","last_peered":"2023-10-09T06:39:22.925812+0000","last_clean":"2023-10-09T06:39:22.925812+0000","last_became_active":"2023-10-09T06:39:22.921099+0000","last_became_peered":"2023-10-09T06:39:22.921099+0000","last_unstale":"2023-10-09T06:39:22.925812+0000","last_undegraded":"2023-10-09T06:39:22.925812+0000","last_fullsized":"2023-10-09T06:39:22.925812+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:21:36.106172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.909067+0000","last_change":"2023-10-09T06:39:21.911822+0000","last_active":"2023-10-09T06:39:22.909067+0000","last_peered":"2023-10-09T06:39:22.909067+0000","last_clean":"2023-10-09T06:39:22.909067+0000","last_became_active":"2023-10-09T06:39:21.911564+0000","last_became_peered":"2023-10-09T06:39:21.911564+0000","last_unstale":"2023-10-09T06:39:22.909067+0000","last_undegraded":"2023-10-09T06:39:22.909067+0000","last_fullsized":"2023-10-09T06:39:22.909067+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:21:59.808322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.928237+0000","last_change":"2023-10-09T06:39:22.927632+0000","last_active":"2023-10-09T06:39:22.928237+0000","last_peered":"2023-10-09T06:39:22.928237+0000","last_clean":"2023-10-09T06:39:22.928237+0000","last_became_active":"2023-10-09T06:39:22.927399+0000","last_became_peered":"2023-10-09T06:39:22.927399+0000","last_unstale":"2023-10-09T06:39:22.928237+0000","last_undegraded":"2023-10-09T06:39:22.928237+0000","last_fullsized":"2023-10-09T06:39:22.928237+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:44:11.254567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.912315+0000","last_change":"2023-10-09T06:39:21.908656+0000","last_active":"2023-10-09T06:39:22.912315+0000","last_peered":"2023-10-09T06:39:22.912315+0000","last_clean":"2023-10-09T06:39:22.912315+0000","last_became_active":"2023-10-09T06:39:21.908397+0000","last_became_peered":"2023-10-09T06:39:21.908397+0000","last_unstale":"2023-10-09T06:39:22.912315+0000","last_undegraded":"2023-10-09T06:39:22.912315+0000","last_fullsized":"2023-10-09T06:39:22.912315+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:46:40.529045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.921985+0000","last_change":"2023-10-09T06:39:22.918045+0000","last_active":"2023-10-09T06:39:22.921985+0000","last_peered":"2023-10-09T06:39:22.921985+0000","last_clean":"2023-10-09T06:39:22.921985+0000","last_became_active":"2023-10-09T06:39:22.916972+0000","last_became_peered":"2023-10-09T06:39:22.916972+0000","last_unstale":"2023-10-09T06:39:22.921985+0000","last_undegraded":"2023-10-09T06:39:22.921985+0000","last_fullsized":"2023-10-09T06:39:22.921985+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:12:53.364651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.916264+0000","last_change":"2023-10-09T06:39:21.904709+0000","last_active":"2023-10-09T06:39:22.916264+0000","last_peered":"2023-10-09T06:39:22.916264+0000","last_clean":"2023-10-09T06:39:22.916264+0000","last_became_active":"2023-10-09T06:39:21.904436+0000","last_became_peered":"2023-10-09T06:39:21.904436+0000","last_unstale":"2023-10-09T06:39:22.916264+0000","last_undegraded":"2023-10-09T06:39:22.916264+0000","last_fullsized":"2023-10-09T06:39:22.916264+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:04:34.082380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.924799+0000","last_change":"2023-10-09T06:39:22.921026+0000","last_active":"2023-10-09T06:39:22.924799+0000","last_peered":"2023-10-09T06:39:22.924799+0000","last_clean":"2023-10-09T06:39:22.924799+0000","last_became_active":"2023-10-09T06:39:22.920777+0000","last_became_peered":"2023-10-09T06:39:22.920777+0000","last_unstale":"2023-10-09T06:39:22.924799+0000","last_undegraded":"2023-10-09T06:39:22.924799+0000","last_fullsized":"2023-10-09T06:39:22.924799+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:41:00.005758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.913864+0000","last_change":"2023-10-09T06:39:21.912036+0000","last_active":"2023-10-09T06:39:22.913864+0000","last_peered":"2023-10-09T06:39:22.913864+0000","last_clean":"2023-10-09T06:39:22.913864+0000","last_became_active":"2023-10-09T06:39:21.911464+0000","last_became_peered":"2023-10-09T06:39:21.911464+0000","last_unstale":"2023-10-09T06:39:22.913864+0000","last_undegraded":"2023-10-09T06:39:22.913864+0000","last_fullsized":"2023-10-09T06:39:22.913864+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:54:27.546478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.929923+0000","last_change":"2023-10-09T06:39:22.925943+0000","last_active":"2023-10-09T06:39:22.929923+0000","last_peered":"2023-10-09T06:39:22.929923+0000","last_clean":"2023-10-09T06:39:22.929923+0000","last_became_active":"2023-10-09T06:39:22.925662+0000","last_became_peered":"2023-10-09T06:39:22.925662+0000","last_unstale":"2023-10-09T06:39:22.929923+0000","last_undegraded":"2023-10-09T06:39:22.929923+0000","last_fullsized":"2023-10-09T06:39:22.929923+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:38:43.178334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":10,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.910327+0000","last_change":"2023-10-09T06:39:21.906363+0000","last_active":"2023-10-09T06:39:22.910327+0000","last_peered":"2023-10-09T06:39:22.910327+0000","last_clean":"2023-10-09T06:39:22.910327+0000","last_became_active":"2023-10-09T06:39:21.906105+0000","last_became_peered":"2023-10-09T06:39:21.906105+0000","last_unstale":"2023-10-09T06:39:22.910327+0000","last_undegraded":"2023-10-09T06:39:22.910327+0000","last_fullsized":"2023-10-09T06:39:22.910327+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:10:09.771360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":9,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.923364+0000","last_change":"2023-10-09T06:39:22.919673+0000","last_active":"2023-10-09T06:39:22.923364+0000","last_peered":"2023-10-09T06:39:22.923364+0000","last_clean":"2023-10-09T06:39:22.923364+0000","last_became_active":"2023-10-09T06:39:22.919440+0000","last_became_peered":"2023-10-09T06:39:22.919440+0000","last_unstale":"2023-10-09T06:39:22.923364+0000","last_undegraded":"2023-10-09T06:39:22.923364+0000","last_fullsized":"2023-10-09T06:39:22.923364+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:21.889478+0000","last_clean_scrub_stamp":"2023-10-09T06:39:21.889478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:28:57.980961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"181'1","reported_seq":11,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.936549+0000","last_change":"2023-10-09T06:39:21.907398+0000","last_active":"2023-10-09T06:39:22.936549+0000","last_peered":"2023-10-09T06:39:22.936549+0000","last_clean":"2023-10-09T06:39:22.936549+0000","last_became_active":"2023-10-09T06:39:21.905514+0000","last_became_peered":"2023-10-09T06:39:21.905514+0000","last_unstale":"2023-10-09T06:39:22.936549+0000","last_undegraded":"2023-10-09T06:39:22.936549+0000","last_fullsized":"2023-10-09T06:39:22.936549+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:20.883285+0000","last_clean_scrub_stamp":"2023-10-09T06:39:20.883285+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:38:28.913626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":296,"reported_epoch":181,"state":"active+clean","last_fresh":"2023-10-09T06:39:22.917173+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:39:22.917173+0000","last_peered":"2023-10-09T06:39:22.917173+0000","last_clean":"2023-10-09T06:39:22.917173+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:39:22.917173+0000","last_undegraded":"2023-10-09T06:39:22.917173+0000","last_fullsized":"2023-10-09T06:39:22.917173+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574959,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304232,"kb_used_data":2848,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067608,"statfs":{"total":96636764160,"available":96325230592,"internally_reserved":0,"allocated":2916352,"data_stored":1628887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:38:42 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51300000000000001,"15min":0.50800000000000001},"min":{"1min":0.36499999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.629,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.498,"5min":0.502,"15min":0.51000000000000001},"min":{"1min":0.33400000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.64700000000000002,"5min":0.70199999999999996,"15min":0.753},"last":0.82899999999999996}]},{"osd":1,"last update":"Mon Oct 9 06:38:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.50700000000000001,"15min":0.51800000000000002},"min":{"1min":0.36899999999999999,"5min":0.33600000000000002,"15min":0.27100000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":1.2030000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53000000000000003,"15min":0.52600000000000002},"min":{"1min":0.39900000000000002,"5min":0.32900000000000001,"15min":0.29699999999999999},"max":{"1min":0.67000000000000004,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.77400000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:38:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53100000000000003,"15min":0.51700000000000002},"min":{"1min":0.37,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.746,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53200000000000003,"15min":0.54300000000000004},"min":{"1min":0.36799999999999999,"5min":0.36399999999999999,"15min":0.315},"max":{"1min":0.68100000000000005,"5min":0.80700000000000005,"15min":0.83099999999999996},"last":0.754}]},{"osd":3,"last update":"Mon Oct 9 06:38:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54200000000000004,"15min":0.55000000000000004},"min":{"1min":0.434,"5min":0.35899999999999999,"15min":0.33500000000000002},"max":{"1min":0.76200000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55100000000000005,"15min":0.58599999999999997},"min":{"1min":0.38400000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69899999999999995,"5min":0.82099999999999995,"15min":1.1479999999999999},"last":0.81100000000000005}]},{"osd":4,"last update":"Mon Oct 9 06:38:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.49299999999999999,"15min":0.51600000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.24399999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.93999999999999995},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58099999999999996,"15min":0.60399999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.30299999999999999},"max":{"1min":0.78600000000000003,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.622}]},{"osd":5,"last update":"Mon Oct 9 06:38:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53700000000000003,"15min":0.56699999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.307},"max":{"1min":0.751,"5min":0.85799999999999998,"15min":0.91100000000000003},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60799999999999998,"15min":0.61799999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34200000000000003},"max":{"1min":0.79400000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.52800000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:38:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56999999999999995,"15min":0.59099999999999997},"min":{"1min":0.318,"5min":0.318,"15min":0.28699999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.88600000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58199999999999996,"15min":0.61099999999999999},"min":{"1min":0.34399999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.81999999999999995,"5min":0.82499999999999996,"15min":0.84599999999999997},"last":0.58199999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574958,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303652,"kb_used_data":2268,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068188,"statfs":{"total":96636764160,"available":96325824512,"internally_reserved":0,"allocated":2322432,"data_stored":1038507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51600000000000001,"15min":0.504},"min":{"1min":0.35899999999999999,"5min":0.35399999999999998,"15min":0.30399999999999999},"max":{"1min":0.70699999999999996,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.505},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.51100000000000001,"15min":0.51200000000000001},"min":{"1min":0.36299999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69699999999999995,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.52700000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50700000000000001,"15min":0.51200000000000001},"min":{"1min":0.43099999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.72699999999999998,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.53500000000000003,"15min":0.53300000000000003},"min":{"1min":0.437,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.69099999999999995,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.60199999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.34300000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67800000000000005,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.53000000000000003},"min":{"1min":0.39300000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.58199999999999996}]},{"osd":3,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.39800000000000002,"5min":0.38900000000000001,"15min":0.33500000000000002},"max":{"1min":0.79000000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54400000000000004,"15min":0.67100000000000004},"min":{"1min":0.40300000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":4.7510000000000003},"last":0.66900000000000004}]},{"osd":4,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.46000000000000002,"15min":0.47499999999999998},"min":{"1min":0.28999999999999998,"5min":0.28199999999999997,"15min":0.27900000000000003},"max":{"1min":0.63700000000000001,"5min":0.74299999999999999,"15min":0.76400000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56699999999999995,"15min":0.58299999999999996},"min":{"1min":0.51500000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.75,"5min":0.80800000000000005,"15min":0.83699999999999997},"last":0.77200000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.52100000000000002,"15min":0.53100000000000003},"min":{"1min":0.37,"5min":0.246,"15min":0.246},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57199999999999995,"15min":0.59299999999999997},"min":{"1min":0.47299999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.70899999999999996,"5min":0.78000000000000003,"15min":0.86599999999999999},"last":0.71899999999999997}]},{"osd":7,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55600000000000005,"15min":0.57999999999999996},"min":{"1min":0.33400000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82399999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.438,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80500000000000005,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.78500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574961,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303632,"kb_used_data":2248,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068208,"statfs":{"total":96636764160,"available":96325844992,"internally_reserved":0,"allocated":2301952,"data_stored":1032107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.317,"15min":0.311},"min":{"1min":0.22500000000000001,"5min":0.192,"15min":0.192},"max":{"1min":0.47699999999999998,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.35399999999999998,"15min":0.35899999999999999},"min":{"1min":0.27700000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.30399999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34699999999999998,"15min":0.34999999999999998},"min":{"1min":0.23899999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.51600000000000001,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38400000000000001,"15min":0.38500000000000001},"min":{"1min":0.25800000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.60399999999999998,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.27100000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.36599999999999999,"15min":0.36299999999999999},"min":{"1min":0.24299999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.503,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.374,"5min":0.35099999999999998,"15min":0.51600000000000001},"min":{"1min":0.255,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":5.8529999999999998},"last":0.23999999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.36499999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59799999999999998,"15min":0.58799999999999997},"min":{"1min":0.46700000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.52700000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57899999999999996,"15min":0.72299999999999998},"min":{"1min":0.40699999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":5.9459999999999997},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.42499999999999999,"5min":0.38900000000000001,"15min":0.34300000000000003},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.5}]},{"osd":6,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54800000000000004,"15min":0.69399999999999995},"min":{"1min":0.38700000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":5.9160000000000004},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.40999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88600000000000001,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.46999999999999997}]},{"osd":7,"last update":"Mon Oct 9 06:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.58399999999999996},"min":{"1min":0.42899999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.97099999999999997},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56899999999999995,"15min":0.72899999999999998},"min":{"1min":0.41799999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":5.9729999999999999},"last":0.51300000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574960,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304236,"kb_used_data":2852,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067604,"statfs":{"total":96636764160,"available":96325226496,"internally_reserved":0,"allocated":2920448,"data_stored":1630299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.316,"15min":0.318},"min":{"1min":0.255,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.498,"5min":0.498,"15min":0.498},"last":0.311},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.33000000000000002,"15min":0.34399999999999997},"min":{"1min":0.24199999999999999,"5min":0.222,"15min":0.214},"max":{"1min":0.48699999999999999,"5min":0.53600000000000003,"15min":0.94499999999999995},"last":0.32600000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.36299999999999999,"15min":0.36399999999999999},"min":{"1min":0.223,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.53200000000000003,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.379,"15min":0.38800000000000001},"min":{"1min":0.29099999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.52900000000000003,"5min":0.69399999999999995,"15min":0.91700000000000004},"last":0.38900000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.33300000000000002,"15min":0.38},"min":{"1min":0.26000000000000001,"5min":0.23599999999999999,"15min":0.23300000000000001},"max":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":1.554},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.37,"15min":0.39600000000000002},"min":{"1min":0.30599999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54500000000000004,"5min":0.58099999999999996,"15min":0.79000000000000004},"last":0.34300000000000003}]},{"osd":4,"last update":"Mon Oct 9 06:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.75900000000000001,"5min":0.80200000000000005,"15min":1.212},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56200000000000006,"15min":0.70499999999999996},"min":{"1min":0.39000000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77800000000000002,"5min":0.89900000000000002,"15min":5.484},"last":0.46899999999999997}]},{"osd":5,"last update":"Mon Oct 9 06:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56100000000000005,"15min":0.70899999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80100000000000005,"5min":0.81399999999999995,"15min":5.5449999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56899999999999995,"15min":0.57499999999999996},"min":{"1min":0.41399999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.80100000000000005,"5min":0.84499999999999997,"15min":1.113},"last":0.48399999999999999}]},{"osd":6,"last update":"Mon Oct 9 06:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59399999999999997,"15min":0.60499999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.78700000000000003,"5min":0.85599999999999998,"15min":1.0529999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.42899999999999999,"5min":0.42799999999999999,"15min":0.41599999999999998},"max":{"1min":0.78700000000000003,"5min":0.878,"15min":1.169},"last":0.52000000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58199999999999996,"15min":0.59199999999999997},"min":{"1min":0.40200000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.80300000000000005,"5min":0.85499999999999998,"15min":1.2390000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56599999999999995,"15min":0.73099999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77000000000000002,"5min":0.86299999999999999,"15min":5.5800000000000001},"last":0.442}]}]},{"osd":2,"up_from":13,"seq":55834574961,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303652,"kb_used_data":2268,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068188,"statfs":{"total":96636764160,"available":96325824512,"internally_reserved":0,"allocated":2322432,"data_stored":1039019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.318,"15min":0.34999999999999998},"min":{"1min":0.21099999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":1.4199999999999999},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.32700000000000001,"15min":0.53500000000000003},"min":{"1min":0.23200000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":7.2699999999999996},"last":0.25900000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.317,"15min":0.34200000000000003},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.48699999999999999,"5min":0.55200000000000005,"15min":0.65400000000000003},"last":0.246},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.374,"15min":0.58699999999999997},"min":{"1min":0.28199999999999997,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.56999999999999995,"5min":0.57999999999999996,"15min":7.327},"last":0.32300000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.373,"15min":0.39000000000000001},"min":{"1min":0.29499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.53400000000000003,"5min":0.64600000000000002,"15min":0.68600000000000005},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.375,"5min":0.371,"15min":0.39000000000000001},"min":{"1min":0.252,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.72699999999999998},"last":0.33300000000000002}]},{"osd":4,"last update":"Mon Oct 9 06:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54600000000000004,"15min":0.54300000000000004},"min":{"1min":0.33600000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.748,"5min":0.79700000000000004,"15min":0.80300000000000005},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.55600000000000005,"15min":0.75900000000000001},"min":{"1min":0.35999999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68899999999999995,"5min":0.84699999999999998,"15min":7.2389999999999999},"last":0.53500000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57999999999999996,"15min":0.877},"min":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77200000000000002,"5min":0.83799999999999997,"15min":10.890000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59999999999999998,"15min":0.60199999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.58099999999999996}]},{"osd":6,"last update":"Mon Oct 9 06:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56499999999999995,"15min":0.76600000000000001},"min":{"1min":0.40100000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71599999999999997,"5min":0.77300000000000002,"15min":7.2999999999999998},"last":0.504},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.40300000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.59899999999999998}]},{"osd":7,"last update":"Mon Oct 9 06:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.58999999999999997,"15min":0.79500000000000004},"min":{"1min":0.40200000000000002,"5min":0.38600000000000001,"15min":0.37},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":7.1879999999999997},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60099999999999998,"15min":0.622},"min":{"1min":0.437,"5min":0.38,"15min":0.38},"max":{"1min":0.80000000000000004,"5min":0.89800000000000002,"15min":0.90500000000000003},"last":0.55300000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574960,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303668,"kb_used_data":2284,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068172,"statfs":{"total":96636764160,"available":96325808128,"internally_reserved":0,"allocated":2338816,"data_stored":1040541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.32800000000000001,"15min":0.33400000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.20599999999999999},"max":{"1min":0.51300000000000001,"5min":0.61699999999999999,"15min":5.8700000000000001},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34699999999999998,"15min":0.36599999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":5.6769999999999996},"last":0.34399999999999997}]},{"osd":1,"last update":"Mon Oct 9 06:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.35099999999999998,"15min":0.35899999999999999},"min":{"1min":0.21299999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.44,"5min":0.59499999999999997,"15min":4.0469999999999997},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.377,"15min":0.38900000000000001},"min":{"1min":0.30099999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.624,"5min":0.68600000000000005,"15min":4.1509999999999998},"last":0.38700000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.35199999999999998,"15min":0.372},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.222},"max":{"1min":0.51900000000000002,"5min":0.56799999999999995,"15min":3.1139999999999999},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.374,"15min":0.38500000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.54500000000000004,"5min":0.66900000000000004,"15min":3.3180000000000001},"last":0.35899999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55400000000000005,"15min":0.56999999999999995},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.68799999999999994,"5min":0.81699999999999995,"15min":3.0899999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.749,"5min":0.79200000000000004,"15min":3.0710000000000002},"last":0.54300000000000004}]},{"osd":5,"last update":"Mon Oct 9 06:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56799999999999995,"15min":0.58799999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.32700000000000001},"max":{"1min":0.745,"5min":0.877,"15min":6.2300000000000004},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58799999999999997,"15min":0.59499999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.755,"5min":0.83099999999999996,"15min":3.0219999999999998},"last":0.60799999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80800000000000005,"5min":0.89200000000000002,"15min":2.6850000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63800000000000001,"15min":0.65100000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.76800000000000002,"5min":0.94299999999999995,"15min":6.5970000000000004},"last":0.68300000000000005}]},{"osd":7,"last update":"Mon Oct 9 06:38:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.60599999999999998,"15min":0.61299999999999999},"min":{"1min":0.41699999999999998,"5min":0.40200000000000002,"15min":0.38200000000000001},"max":{"1min":0.81999999999999995,"5min":0.88500000000000001,"15min":4.2629999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58899999999999997,"15min":0.61199999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.79100000000000004,"5min":0.82899999999999996,"15min":7.3650000000000002},"last":0.67300000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574959,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303648,"kb_used_data":2264,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068192,"statfs":{"total":96636764160,"available":96325828608,"internally_reserved":0,"allocated":2318336,"data_stored":1038465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.51600000000000001,"15min":0.55100000000000005},"min":{"1min":0.373,"5min":0.28100000000000003,"15min":0.27000000000000002},"max":{"1min":0.82599999999999996,"5min":0.88900000000000001,"15min":1.083},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52500000000000002,"15min":0.52000000000000002},"min":{"1min":0.36499999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.76100000000000001,"5min":0.97799999999999998,"15min":1.054},"last":0.53700000000000003}]},{"osd":1,"last update":"Mon Oct 9 06:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53500000000000003,"15min":0.54700000000000004},"min":{"1min":0.28399999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.71099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.501,"15min":0.496},"min":{"1min":0.34000000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.75700000000000001,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.621}]},{"osd":2,"last update":"Mon Oct 9 06:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52000000000000002,"15min":0.53800000000000003},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":1.0249999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54600000000000004,"15min":0.58599999999999997},"min":{"1min":0.38100000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.66800000000000004,"5min":0.90500000000000003,"15min":0.93799999999999994},"last":0.64300000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52100000000000002,"15min":0.54800000000000004},"min":{"1min":0.33300000000000002,"5min":0.32200000000000001,"15min":0.29199999999999998},"max":{"1min":0.71599999999999997,"5min":0.86899999999999999,"15min":0.98499999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56699999999999995,"15min":0.58699999999999997},"min":{"1min":0.40699999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77500000000000002,"5min":0.92900000000000005,"15min":1.1319999999999999},"last":0.754}]},{"osd":5,"last update":"Mon Oct 9 06:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.51300000000000001,"15min":0.54800000000000004},"min":{"1min":0.34399999999999997,"5min":0.30399999999999999,"15min":0.27300000000000002},"max":{"1min":0.73999999999999999,"5min":0.81100000000000005,"15min":0.998},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57599999999999996,"15min":0.60399999999999998},"min":{"1min":0.46000000000000002,"5min":0.36299999999999999,"15min":0.35099999999999998},"max":{"1min":0.82399999999999995,"5min":0.95599999999999996,"15min":0.97999999999999998},"last":0.78000000000000003}]},{"osd":6,"last update":"Mon Oct 9 06:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54800000000000004,"15min":0.56599999999999995},"min":{"1min":0.39400000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70299999999999996,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.56899999999999995,"15min":0.59599999999999997},"min":{"1min":0.41399999999999998,"5min":0.34999999999999998,"15min":0.32700000000000001},"max":{"1min":0.78300000000000003,"5min":1.01,"15min":1.01},"last":0.76900000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:38:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.53800000000000003,"15min":0.56699999999999995},"min":{"1min":0.35299999999999998,"5min":0.33600000000000002,"15min":0.28299999999999997},"max":{"1min":0.70299999999999996,"5min":0.83199999999999996,"15min":0.90000000000000002},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60499999999999998,"15min":0.64100000000000001},"min":{"1min":0.45000000000000001,"5min":0.34999999999999998,"15min":0.32200000000000001},"max":{"1min":0.81999999999999995,"5min":0.98999999999999999,"15min":1.175},"last":0.83999999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574959,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303632,"kb_used_data":2248,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068208,"statfs":{"total":96636764160,"available":96325844992,"internally_reserved":0,"allocated":2301952,"data_stored":1032107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.502,"15min":0.50900000000000001},"min":{"1min":0.40100000000000002,"5min":0.29299999999999998,"15min":0.28599999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.86599999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51200000000000001,"15min":0.51500000000000001},"min":{"1min":0.38200000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72899999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.54400000000000004}]},{"osd":1,"last update":"Mon Oct 9 06:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.504,"15min":0.51000000000000001},"min":{"1min":0.38,"5min":0.311,"15min":0.311},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.78200000000000003},"last":0.496},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.52200000000000002,"15min":0.53100000000000003},"min":{"1min":0.41199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.81599999999999995},"last":0.51900000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.47199999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.83999999999999997},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57199999999999995,"15min":0.58499999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.39500000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79900000000000004},"last":0.57199999999999995}]},{"osd":3,"last update":"Mon Oct 9 06:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55200000000000005,"15min":0.54600000000000004},"min":{"1min":0.39100000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.55900000000000005},"min":{"1min":0.40699999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.69199999999999995,"5min":0.71099999999999997,"15min":0.81100000000000005},"last":0.61699999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.50700000000000001,"15min":0.5},"min":{"1min":0.34200000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68400000000000005,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.40300000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.80700000000000005},"last":0.64800000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55500000000000005,"15min":0.53900000000000003},"min":{"1min":0.378,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.83199999999999996},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56499999999999995,"15min":0.56200000000000006},"min":{"1min":0.40200000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71799999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.66300000000000003}]},{"osd":7,"last update":"Mon Oct 9 06:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.35999999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73199999999999998,"5min":0.79500000000000004,"15min":0.85599999999999998},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.44700000000000001,"5min":0.35199999999999998,"15min":0.314},"max":{"1min":0.77400000000000002,"5min":0.84499999999999997,"15min":0.90600000000000003},"last":0.68100000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:39:27.386 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-09T06:39:27.386 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-09T06:39:27.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.507+0000 7f2fb9126700 1 Processor -- start 2023-10-09T06:39:27.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.507+0000 7f2fb9126700 1 -- start start 2023-10-09T06:39:27.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.508+0000 7f2fb9126700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fb4159950 0x7f2fb40cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:27.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.508+0000 7f2fb9126700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2fb40af320 0x7f2fb4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:27.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.508+0000 7f2fb9126700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2fb4069aa0 0x7f2fb415d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:27.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.508+0000 7f2fb9126700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fb4069f20 con 0x7f2fb4069aa0 2023-10-09T06:39:27.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.508+0000 7f2fb9126700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fb40ce380 con 0x7f2fb40af320 2023-10-09T06:39:27.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.508+0000 7f2fb9126700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb40cec20 con 0x7f2fb4159950 2023-10-09T06:39:27.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.508+0000 7f2fb359e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2fb4069aa0 0x7f2fb415d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:27.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.508+0000 7f2fb359e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2fb4069aa0 0x7f2fb415d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36482/0 (socket says 172.21.15.163:36482) 2023-10-09T06:39:27.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.508+0000 7f2fb359e700 1 -- 172.21.15.163:0/4235996071 learned_addr learned my addr 172.21.15.163:0/4235996071 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:27.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.508+0000 7f2faa59c700 1 --2- 172.21.15.163:0/4235996071 >> v2:172.21.15.181:3300/0 conn(0x7f2fb40af320 0x7f2fb4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:27.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.508+0000 7f2fb2d9d700 1 --2- 172.21.15.163:0/4235996071 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fb4159950 0x7f2fb40cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:27.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.508+0000 7f2fb359e700 1 -- 172.21.15.163:0/4235996071 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fb4159950 msgr2=0x7f2fb40cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:27.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.508+0000 7f2fb359e700 1 --2- 172.21.15.163:0/4235996071 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fb4159950 0x7f2fb40cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:27.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.508+0000 7f2fb359e700 1 -- 172.21.15.163:0/4235996071 >> v2:172.21.15.181:3300/0 conn(0x7f2fb40af320 msgr2=0x7f2fb4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:27.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.508+0000 7f2fb359e700 1 --2- 172.21.15.163:0/4235996071 >> v2:172.21.15.181:3300/0 conn(0x7f2fb40af320 0x7f2fb4069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:27.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.508+0000 7f2fb359e700 1 -- 172.21.15.163:0/4235996071 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb40afa90 con 0x7f2fb4069aa0 2023-10-09T06:39:27.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.508+0000 7f2fb359e700 1 --2- 172.21.15.163:0/4235996071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fb4069aa0 0x7f2fb415d4f0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f2fb40680d0 tx=0x7f2fac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e94e09485f5289e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:27.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.509+0000 7f2fb259c700 1 -- 172.21.15.163:0/4235996071 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac00e070 con 0x7f2fb4069aa0 2023-10-09T06:39:27.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.509+0000 7f2fb259c700 1 -- 172.21.15.163:0/4235996071 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fac00b970 con 0x7f2fb4069aa0 2023-10-09T06:39:27.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.509+0000 7f2fb259c700 1 -- 172.21.15.163:0/4235996071 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac0092e0 con 0x7f2fb4069aa0 2023-10-09T06:39:27.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.509+0000 7f2fb9126700 1 -- 172.21.15.163:0/4235996071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fb4069aa0 msgr2=0x7f2fb415d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:27.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.509+0000 7f2fb9126700 1 --2- 172.21.15.163:0/4235996071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fb4069aa0 0x7f2fb415d4f0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f2fb40680d0 tx=0x7f2fac009840 comp rx=0 tx=0).stop 2023-10-09T06:39:27.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.509+0000 7f2fb9126700 1 -- 172.21.15.163:0/4235996071 shutdown_connections 2023-10-09T06:39:27.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.509+0000 7f2fb9126700 1 --2- 172.21.15.163:0/4235996071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fb4069aa0 0x7f2fb415d4f0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:27.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.509+0000 7f2fb9126700 1 --2- 172.21.15.163:0/4235996071 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fb4159950 0x7f2fb40cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:27.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.509+0000 7f2fb9126700 1 --2- 172.21.15.163:0/4235996071 >> v2:172.21.15.181:3300/0 conn(0x7f2fb40af320 0x7f2fb4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:27.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.509+0000 7f2fb9126700 1 -- 172.21.15.163:0/4235996071 >> 172.21.15.163:0/4235996071 conn(0x7f2fb4068e00 msgr2=0x7f2fb40d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:27.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.509+0000 7f2fb9126700 1 -- 172.21.15.163:0/4235996071 shutdown_connections 2023-10-09T06:39:27.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.509+0000 7f2fb9126700 1 -- 172.21.15.163:0/4235996071 wait complete. 2023-10-09T06:39:27.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.510+0000 7f2fb9126700 1 Processor -- start 2023-10-09T06:39:27.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.510+0000 7f2fb9126700 1 -- start start 2023-10-09T06:39:27.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.511+0000 7f2fb9126700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fb4069aa0 0x7f2fb416dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:27.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.511+0000 7f2fb9126700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fb40af320 0x7f2fb4155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:27.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.511+0000 7f2fb9126700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fb4159950 0x7f2fb415e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:27.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.511+0000 7f2fb9126700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb415eea0 con 0x7f2fb4069aa0 2023-10-09T06:39:27.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.511+0000 7f2fb9126700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb415f180 con 0x7f2fb40af320 2023-10-09T06:39:27.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.511+0000 7f2fb9126700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb416d500 con 0x7f2fb4159950 2023-10-09T06:39:27.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.511+0000 7f2fb2d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fb4069aa0 0x7f2fb416dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:27.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.511+0000 7f2fb2d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fb4069aa0 0x7f2fb416dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36492/0 (socket says 172.21.15.163:36492) 2023-10-09T06:39:27.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.511+0000 7f2fb2d9d700 1 -- 172.21.15.163:0/48164962 learned_addr learned my addr 172.21.15.163:0/48164962 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:27.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.511+0000 7f2faa59c700 1 --2- 172.21.15.163:0/48164962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fb40af320 0x7f2fb4155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:27.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.511+0000 7f2fb359e700 1 --2- 172.21.15.163:0/48164962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fb4159950 0x7f2fb415e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:27.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.511+0000 7f2fb2d9d700 1 -- 172.21.15.163:0/48164962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fb4159950 msgr2=0x7f2fb415e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:27.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.511+0000 7f2fb2d9d700 1 --2- 172.21.15.163:0/48164962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fb4159950 0x7f2fb415e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:27.517 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.511+0000 7f2fb2d9d700 1 -- 172.21.15.163:0/48164962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fb40af320 msgr2=0x7f2fb4155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:27.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.511+0000 7f2fb2d9d700 1 --2- 172.21.15.163:0/48164962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fb40af320 0x7f2fb4155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:27.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.511+0000 7f2fb2d9d700 1 -- 172.21.15.163:0/48164962 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fac009ae0 con 0x7f2fb4069aa0 2023-10-09T06:39:27.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.511+0000 7f2fb359e700 1 --2- 172.21.15.163:0/48164962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fb4159950 0x7f2fb415e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:27.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.512+0000 7f2fb2d9d700 1 --2- 172.21.15.163:0/48164962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fb4069aa0 0x7f2fb416dfe0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400e930 tx=0x7f2fa40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:27.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.512+0000 7f2fab7fe700 1 -- 172.21.15.163:0/48164962 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa400eed0 con 0x7f2fb4069aa0 2023-10-09T06:39:27.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.512+0000 7f2fab7fe700 1 -- 172.21.15.163:0/48164962 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa400c850 con 0x7f2fb4069aa0 2023-10-09T06:39:27.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.512+0000 7f2fb9126700 1 -- 172.21.15.163:0/48164962 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb416d7e0 con 0x7f2fb4069aa0 2023-10-09T06:39:27.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.512+0000 7f2fab7fe700 1 -- 172.21.15.163:0/48164962 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa400cbf0 con 0x7f2fb4069aa0 2023-10-09T06:39:27.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.512+0000 7f2fb9126700 1 -- 172.21.15.163:0/48164962 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb415c150 con 0x7f2fb4069aa0 2023-10-09T06:39:27.518 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.513+0000 7f2fb9126700 1 -- 172.21.15.163:0/48164962 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fb416dc80 con 0x7f2fb4069aa0 2023-10-09T06:39:27.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.513+0000 7f2fab7fe700 1 -- 172.21.15.163:0/48164962 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2fa4013470 con 0x7f2fb4069aa0 2023-10-09T06:39:27.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.513+0000 7f2fab7fe700 1 --2- 172.21.15.163:0/48164962 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2f94067320 0x7f2f940697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:27.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.514+0000 7f2fab7fe700 1 -- 172.21.15.163:0/48164962 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2fa4084ee0 con 0x7f2fb4069aa0 2023-10-09T06:39:27.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.514+0000 7f2faa59c700 1 --2- 172.21.15.163:0/48164962 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2f94067320 0x7f2f940697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:27.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.514+0000 7f2faa59c700 1 --2- 172.21.15.163:0/48164962 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2f94067320 0x7f2f940697d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4158240 tx=0x7f2f9c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:27.519 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.518+0000 7f2fab7fe700 1 -- 172.21.15.163:0/48164962 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2fa4051b30 con 0x7f2fb4069aa0 2023-10-09T06:39:27.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.775+0000 7f2fb9126700 1 -- 172.21.15.163:0/48164962 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f2fb416de30 con 0x7f2fb4069aa0 2023-10-09T06:39:27.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.779+0000 7f2fab7fe700 1 -- 172.21.15.163:0/48164962 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) v1 ==== 190+0+0 (secure 0 0 0) 0x7f2fa4055150 con 0x7f2fb4069aa0 2023-10-09T06:39:27.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.781+0000 7f2fb9126700 1 -- 172.21.15.163:0/48164962 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2f94067320 msgr2=0x7f2f940697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:27.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.781+0000 7f2fb9126700 1 --2- 172.21.15.163:0/48164962 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2f94067320 0x7f2f940697d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4158240 tx=0x7f2f9c008040 comp rx=0 tx=0).stop 2023-10-09T06:39:27.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.781+0000 7f2fb9126700 1 -- 172.21.15.163:0/48164962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fb4069aa0 msgr2=0x7f2fb416dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:27.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.781+0000 7f2fb9126700 1 --2- 172.21.15.163:0/48164962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fb4069aa0 0x7f2fb416dfe0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400e930 tx=0x7f2fa40091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:27.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.781+0000 7f2fb9126700 1 -- 172.21.15.163:0/48164962 shutdown_connections 2023-10-09T06:39:27.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.781+0000 7f2fb9126700 1 --2- 172.21.15.163:0/48164962 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2f94067320 0x7f2f940697d0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:27.782 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.781+0000 7f2fb9126700 1 --2- 172.21.15.163:0/48164962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fb4069aa0 0x7f2fb416dfe0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:27.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.781+0000 7f2fb9126700 1 --2- 172.21.15.163:0/48164962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fb4159950 0x7f2fb415e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:27.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.781+0000 7f2fb9126700 1 --2- 172.21.15.163:0/48164962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fb40af320 0x7f2fb4155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:27.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.781+0000 7f2fb9126700 1 -- 172.21.15.163:0/48164962 >> 172.21.15.163:0/48164962 conn(0x7f2fb4068e00 msgr2=0x7f2fb40c6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:27.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.781+0000 7f2fb9126700 1 -- 172.21.15.163:0/48164962 shutdown_connections 2023-10-09T06:39:27.783 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.781+0000 7f2fb9126700 1 -- 172.21.15.163:0/48164962 wait complete. 2023-10-09T06:39:27.783 INFO:teuthology.orchestra.run.smithi163.stderr:updated caps for client.0 2023-10-09T06:39:27.794 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:39:27.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.908+0000 7ff58f59e700 1 Processor -- start 2023-10-09T06:39:27.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.908+0000 7ff58f59e700 1 -- start start 2023-10-09T06:39:27.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.909+0000 7ff58f59e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff590159420 0x7ff5900b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:27.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.909+0000 7ff58f59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5901597f0 0x7ff5900b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:27.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.909+0000 7ff58f59e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5900b11d0 con 0x7ff59015d230 2023-10-09T06:39:27.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.909+0000 7ff58f59e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5900b14b0 con 0x7ff590159420 2023-10-09T06:39:27.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.909+0000 7ff58f59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5900b1790 con 0x7ff5901597f0 2023-10-09T06:39:27.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.909+0000 7ff585d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5901597f0 0x7ff5900b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:27.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.909+0000 7ff58e59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff590159420 0x7ff5900b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:27.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.909+0000 7ff58e59c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff590159420 0x7ff5900b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:50714/0 (socket says 172.21.15.163:50714) 2023-10-09T06:39:27.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.909+0000 7ff58e59c700 1 -- 172.21.15.163:0/4247776746 learned_addr learned my addr 172.21.15.163:0/4247776746 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:27.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.909+0000 7ff58ed9d700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff59015d230 0x7ff5900b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:49918/0 (socket says 172.21.15.163:49918) 2023-10-09T06:39:27.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.909+0000 7ff585d9b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5901597f0 0x7ff5900b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:48636/0 (socket says 172.21.15.163:48636) 2023-10-09T06:39:27.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.909+0000 7ff58e59c700 1 -- 172.21.15.163:0/4247776746 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5901597f0 msgr2=0x7ff5900b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:27.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.909+0000 7ff58e59c700 1 --2- 172.21.15.163:0/4247776746 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5901597f0 0x7ff5900b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:27.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.909+0000 7ff58e59c700 1 -- 172.21.15.163:0/4247776746 >> v1:172.21.15.181:6789/0 conn(0x7ff59015d230 legacy=0x7ff5900b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:27.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.909+0000 7ff58e59c700 1 -- 172.21.15.163:0/4247776746 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5900b1a70 con 0x7ff590159420 2023-10-09T06:39:27.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.909+0000 7ff585d9b700 1 --2- 172.21.15.163:0/4247776746 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5901597f0 0x7ff5900b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:39:27.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.910+0000 7ff58e59c700 1 --2- 172.21.15.163:0/4247776746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff590159420 0x7ff5900b5f40 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff5800099b0 tx=0x7ff580009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cea5a7c280df7ee server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:27.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.910+0000 7ff58dd9b700 1 -- 172.21.15.163:0/4247776746 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff580009ea0 con 0x7ff590159420 2023-10-09T06:39:27.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.910+0000 7ff58dd9b700 1 -- 172.21.15.163:0/4247776746 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58000b860 con 0x7ff590159420 2023-10-09T06:39:27.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.910+0000 7ff58dd9b700 1 -- 172.21.15.163:0/4247776746 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58000bc00 con 0x7ff590159420 2023-10-09T06:39:27.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.910+0000 7ff58f59e700 1 -- 172.21.15.163:0/4247776746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff590159420 msgr2=0x7ff5900b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:27.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.910+0000 7ff58f59e700 1 --2- 172.21.15.163:0/4247776746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff590159420 0x7ff5900b5f40 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff5800099b0 tx=0x7ff580009250 comp rx=0 tx=0).stop 2023-10-09T06:39:27.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.910+0000 7ff58f59e700 1 -- 172.21.15.163:0/4247776746 shutdown_connections 2023-10-09T06:39:27.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.910+0000 7ff58f59e700 1 --2- 172.21.15.163:0/4247776746 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff590159420 0x7ff5900b5f40 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:27.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.910+0000 7ff58f59e700 1 --2- 172.21.15.163:0/4247776746 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5901597f0 0x7ff5900b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:27.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.910+0000 7ff58f59e700 1 -- 172.21.15.163:0/4247776746 >> 172.21.15.163:0/4247776746 conn(0x7ff5900d0660 msgr2=0x7ff5900d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:27.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.910+0000 7ff58f59e700 1 -- 172.21.15.163:0/4247776746 shutdown_connections 2023-10-09T06:39:27.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.910+0000 7ff58f59e700 1 -- 172.21.15.163:0/4247776746 wait complete. 2023-10-09T06:39:27.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.912+0000 7ff58f59e700 1 Processor -- start 2023-10-09T06:39:27.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.912+0000 7ff58f59e700 1 -- start start 2023-10-09T06:39:27.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.912+0000 7ff58f59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff590159420 0x7ff590149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:27.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.912+0000 7ff58f59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5901597f0 0x7ff5900c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:27.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.912+0000 7ff58f59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff59015d230 0x7ff590156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:27.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.912+0000 7ff58f59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5900c7270 con 0x7ff59015d230 2023-10-09T06:39:27.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.912+0000 7ff58f59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5900c6660 con 0x7ff590159420 2023-10-09T06:39:27.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.912+0000 7ff58f59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5900c6940 con 0x7ff5901597f0 2023-10-09T06:39:27.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.912+0000 7ff58ed9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff59015d230 0x7ff590156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:27.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.912+0000 7ff58e59c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff590159420 0x7ff590149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:27.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.912+0000 7ff58ed9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff59015d230 0x7ff590156e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:36504/0 (socket says 172.21.15.163:36504) 2023-10-09T06:39:27.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.913+0000 7ff58ed9d700 1 -- 172.21.15.163:0/2683568637 learned_addr learned my addr 172.21.15.163:0/2683568637 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:27.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.913+0000 7ff585d9b700 1 --2- 172.21.15.163:0/2683568637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5901597f0 0x7ff5900c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:27.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.913+0000 7ff58ed9d700 1 -- 172.21.15.163:0/2683568637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5901597f0 msgr2=0x7ff5900c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:27.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.913+0000 7ff58ed9d700 1 --2- 172.21.15.163:0/2683568637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5901597f0 0x7ff5900c5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:27.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.913+0000 7ff58ed9d700 1 -- 172.21.15.163:0/2683568637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff590159420 msgr2=0x7ff590149210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:27.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.913+0000 7ff58ed9d700 1 --2- 172.21.15.163:0/2683568637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff590159420 0x7ff590149210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:27.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.913+0000 7ff58ed9d700 1 -- 172.21.15.163:0/2683568637 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff578009420 con 0x7ff59015d230 2023-10-09T06:39:27.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.913+0000 7ff58e59c700 1 --2- 172.21.15.163:0/2683568637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff590159420 0x7ff590149210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:39:27.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.913+0000 7ff58ed9d700 1 --2- 172.21.15.163:0/2683568637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff59015d230 0x7ff590156e70 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7ff58800cc70 tx=0x7ff588001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:27.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.913+0000 7ff586ffd700 1 -- 172.21.15.163:0/2683568637 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff588005df0 con 0x7ff59015d230 2023-10-09T06:39:27.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.913+0000 7ff586ffd700 1 -- 172.21.15.163:0/2683568637 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58800a5b0 con 0x7ff59015d230 2023-10-09T06:39:27.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.913+0000 7ff58f59e700 1 -- 172.21.15.163:0/2683568637 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5800094f0 con 0x7ff59015d230 2023-10-09T06:39:27.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.913+0000 7ff586ffd700 1 -- 172.21.15.163:0/2683568637 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58800a930 con 0x7ff59015d230 2023-10-09T06:39:27.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.913+0000 7ff58f59e700 1 -- 172.21.15.163:0/2683568637 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5900c6f00 con 0x7ff59015d230 2023-10-09T06:39:27.918 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.914+0000 7ff58f59e700 1 -- 172.21.15.163:0/2683568637 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5900c7600 con 0x7ff59015d230 2023-10-09T06:39:27.919 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.915+0000 7ff586ffd700 1 -- 172.21.15.163:0/2683568637 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff58800ace0 con 0x7ff59015d230 2023-10-09T06:39:27.919 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.915+0000 7ff586ffd700 1 --2- 172.21.15.163:0/2683568637 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff570067320 0x7ff5700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:27.919 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.915+0000 7ff586ffd700 1 -- 172.21.15.163:0/2683568637 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff588084c00 con 0x7ff59015d230 2023-10-09T06:39:27.919 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.915+0000 7ff58e59c700 1 --2- 172.21.15.163:0/2683568637 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff570067320 0x7ff5700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:27.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.916+0000 7ff58e59c700 1 --2- 172.21.15.163:0/2683568637 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff570067320 0x7ff5700697d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ff5800099b0 tx=0x7ff580017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:27.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:27.919+0000 7ff586ffd700 1 -- 172.21.15.163:0/2683568637 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff5900c7600 con 0x7ff59015d230 2023-10-09T06:39:28.146 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.146+0000 7ff58f59e700 1 -- 172.21.15.163:0/2683568637 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5900c7600 con 0x7ff59015d230 2023-10-09T06:39:28.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.146+0000 7ff586ffd700 1 -- 172.21.15.163:0/2683568637 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) v1 ==== 95+0+4613 (secure 0 0 0) 0x7ff5900c7600 con 0x7ff59015d230 2023-10-09T06:39:28.147 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:28.147 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":122,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6815,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3960800872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3960800872},{"type":"v1","addr":"172.21.15.181:6836","nonce":3960800872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":8631,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/380361821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":380361821},{"type":"v1","addr":"172.21.15.163:6834","nonce":380361821}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":8634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2847943599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2847943599},{"type":"v1","addr":"172.21.15.163:6835","nonce":2847943599}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:39:22.273633+0000","modified":"2023-10-09T06:39:25.934461+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8649":{"gid":8649,"name":"d","rank":0,"incarnation":119,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/119178917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":119178917},{"type":"v1","addr":"172.21.15.181:6837","nonce":119178917}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-09T06:39:28.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.148+0000 7ff58f59e700 1 -- 172.21.15.163:0/2683568637 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff570067320 msgr2=0x7ff5700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:28.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.148+0000 7ff58f59e700 1 --2- 172.21.15.163:0/2683568637 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff570067320 0x7ff5700697d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ff5800099b0 tx=0x7ff580017000 comp rx=0 tx=0).stop 2023-10-09T06:39:28.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.148+0000 7ff58f59e700 1 -- 172.21.15.163:0/2683568637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff59015d230 msgr2=0x7ff590156e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:28.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.148+0000 7ff58f59e700 1 --2- 172.21.15.163:0/2683568637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff59015d230 0x7ff590156e70 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7ff58800cc70 tx=0x7ff588001a70 comp rx=0 tx=0).stop 2023-10-09T06:39:28.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.148+0000 7ff58f59e700 1 -- 172.21.15.163:0/2683568637 shutdown_connections 2023-10-09T06:39:28.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.148+0000 7ff58f59e700 1 --2- 172.21.15.163:0/2683568637 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff570067320 0x7ff5700697d0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.148+0000 7ff58f59e700 1 --2- 172.21.15.163:0/2683568637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff59015d230 0x7ff590156e70 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.148+0000 7ff58f59e700 1 --2- 172.21.15.163:0/2683568637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff590159420 0x7ff590149210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.148+0000 7ff58f59e700 1 --2- 172.21.15.163:0/2683568637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5901597f0 0x7ff5900c5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.148+0000 7ff58f59e700 1 -- 172.21.15.163:0/2683568637 >> 172.21.15.163:0/2683568637 conn(0x7ff5900d0660 msgr2=0x7ff5900b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:28.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.148+0000 7ff58f59e700 1 -- 172.21.15.163:0/2683568637 shutdown_connections 2023-10-09T06:39:28.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.148+0000 7ff58f59e700 1 -- 172.21.15.163:0/2683568637 wait complete. 2023-10-09T06:39:28.151 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 122 2023-10-09T06:39:28.200 INFO:tasks.ceph.mds.d.smithi181.stderr:2023-10-09T06:39:28.199+0000 7fd4a76fb700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-09T06:39:28.564 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 122, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-09T06:39:22.273633+0000', 'modified': '2023-10-09T06:39:25.934461+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8649}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8649': {'gid': 8649, 'name': 'd', 'rank': 0, 'incarnation': 119, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6837/119178917', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6835', 'nonce': 119178917}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 119178917}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [32], 'metadata_pool': 31, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-09T06:39:28.564 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-09T06:39:28.565 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-09T06:39:28.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.682+0000 7f6abc2c6700 1 Processor -- start 2023-10-09T06:39:28.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.682+0000 7f6abc2c6700 1 -- start start 2023-10-09T06:39:28.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.683+0000 7f6abc2c6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ab406bcd0 0x7f6ab40caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:28.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.683+0000 7f6abc2c6700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6ab406c0a0 0x7f6ab415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:28.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.683+0000 7f6abc2c6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6ab415df50 0x7f6ab415e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:28.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.683+0000 7f6abc2c6700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ab415d7e0 con 0x7f6ab406c0a0 2023-10-09T06:39:28.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.683+0000 7f6abc2c6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ab415dac0 con 0x7f6ab415df50 2023-10-09T06:39:28.685 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.683+0000 7f6abc2c6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab415aa30 con 0x7f6ab406bcd0 2023-10-09T06:39:28.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.683+0000 7f6ab9861700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6ab406c0a0 0x7f6ab415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:28.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.683+0000 7f6ab9861700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6ab406c0a0 0x7f6ab415a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41726/0 (socket says 172.21.15.163:41726) 2023-10-09T06:39:28.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.683+0000 7f6ab9861700 1 -- 172.21.15.163:0/1137767181 learned_addr learned my addr 172.21.15.163:0/1137767181 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:28.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.683+0000 7f6aba062700 1 --2- 172.21.15.163:0/1137767181 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ab406bcd0 0x7f6ab40caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:28.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.683+0000 7f6aba863700 1 --2- 172.21.15.163:0/1137767181 >> v2:172.21.15.181:3300/0 conn(0x7f6ab415df50 0x7f6ab415e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:28.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.683+0000 7f6ab9861700 1 -- 172.21.15.163:0/1137767181 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ab406bcd0 msgr2=0x7f6ab40caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:28.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.683+0000 7f6ab9861700 1 --2- 172.21.15.163:0/1137767181 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ab406bcd0 0x7f6ab40caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.683+0000 7f6ab9861700 1 -- 172.21.15.163:0/1137767181 >> v2:172.21.15.181:3300/0 conn(0x7f6ab415df50 msgr2=0x7f6ab415e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:28.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.683+0000 7f6ab9861700 1 --2- 172.21.15.163:0/1137767181 >> v2:172.21.15.181:3300/0 conn(0x7f6ab415df50 0x7f6ab415e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.683+0000 7f6ab9861700 1 -- 172.21.15.163:0/1137767181 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab415ad10 con 0x7f6ab406c0a0 2023-10-09T06:39:28.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.683+0000 7f6aba863700 1 --2- 172.21.15.163:0/1137767181 >> v2:172.21.15.181:3300/0 conn(0x7f6ab415df50 0x7f6ab415e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:28.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.683+0000 7f6ab9861700 1 --2- 172.21.15.163:0/1137767181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ab406c0a0 0x7f6ab415a1e0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f6ab406b620 tx=0x7f6aa4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9508705065ec0d2f server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:28.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.683+0000 7f6ab9060700 1 -- 172.21.15.163:0/1137767181 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4009e70 con 0x7f6ab406c0a0 2023-10-09T06:39:28.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.683+0000 7f6ab9060700 1 -- 172.21.15.163:0/1137767181 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa400b860 con 0x7f6ab406c0a0 2023-10-09T06:39:28.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.683+0000 7f6ab9060700 1 -- 172.21.15.163:0/1137767181 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa400bc20 con 0x7f6ab406c0a0 2023-10-09T06:39:28.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.684+0000 7f6abc2c6700 1 -- 172.21.15.163:0/1137767181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ab406c0a0 msgr2=0x7f6ab415a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:28.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.684+0000 7f6abc2c6700 1 --2- 172.21.15.163:0/1137767181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ab406c0a0 0x7f6ab415a1e0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f6ab406b620 tx=0x7f6aa4009250 comp rx=0 tx=0).stop 2023-10-09T06:39:28.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.684+0000 7f6abc2c6700 1 -- 172.21.15.163:0/1137767181 shutdown_connections 2023-10-09T06:39:28.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.684+0000 7f6abc2c6700 1 --2- 172.21.15.163:0/1137767181 >> v2:172.21.15.181:3300/0 conn(0x7f6ab415df50 0x7f6ab415e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.684+0000 7f6abc2c6700 1 --2- 172.21.15.163:0/1137767181 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ab406bcd0 0x7f6ab40caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.684+0000 7f6abc2c6700 1 --2- 172.21.15.163:0/1137767181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ab406c0a0 0x7f6ab415a1e0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.684+0000 7f6abc2c6700 1 -- 172.21.15.163:0/1137767181 >> 172.21.15.163:0/1137767181 conn(0x7f6ab40ce600 msgr2=0x7f6ab40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:28.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.684+0000 7f6abc2c6700 1 -- 172.21.15.163:0/1137767181 shutdown_connections 2023-10-09T06:39:28.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.684+0000 7f6abc2c6700 1 -- 172.21.15.163:0/1137767181 wait complete. 2023-10-09T06:39:28.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.685+0000 7f6abc2c6700 1 Processor -- start 2023-10-09T06:39:28.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.685+0000 7f6abc2c6700 1 -- start start 2023-10-09T06:39:28.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.686+0000 7f6abc2c6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ab406bcd0 0x7f6ab40cc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:28.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.686+0000 7f6abc2c6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ab406c0a0 0x7f6ab4154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:28.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.686+0000 7f6abc2c6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ab415df50 0x7f6ab41553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:28.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.686+0000 7f6abc2c6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab4155910 con 0x7f6ab415df50 2023-10-09T06:39:28.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.686+0000 7f6abc2c6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab40c9640 con 0x7f6ab406bcd0 2023-10-09T06:39:28.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.686+0000 7f6abc2c6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab40c8880 con 0x7f6ab406c0a0 2023-10-09T06:39:28.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.686+0000 7f6aba863700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ab415df50 0x7f6ab41553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:28.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.686+0000 7f6aba863700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ab415df50 0x7f6ab41553d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41736/0 (socket says 172.21.15.163:41736) 2023-10-09T06:39:28.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.686+0000 7f6aba863700 1 -- 172.21.15.163:0/3493600210 learned_addr learned my addr 172.21.15.163:0/3493600210 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:28.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.686+0000 7f6ab9861700 1 --2- 172.21.15.163:0/3493600210 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ab406c0a0 0x7f6ab4154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:28.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.686+0000 7f6aba062700 1 --2- 172.21.15.163:0/3493600210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ab406bcd0 0x7f6ab40cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:28.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.686+0000 7f6aba863700 1 -- 172.21.15.163:0/3493600210 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ab406c0a0 msgr2=0x7f6ab4154e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:28.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.686+0000 7f6aba863700 1 --2- 172.21.15.163:0/3493600210 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ab406c0a0 0x7f6ab4154e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.686+0000 7f6aba863700 1 -- 172.21.15.163:0/3493600210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ab406bcd0 msgr2=0x7f6ab40cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:28.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.686+0000 7f6aba863700 1 --2- 172.21.15.163:0/3493600210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ab406bcd0 0x7f6ab40cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.686+0000 7f6aba863700 1 -- 172.21.15.163:0/3493600210 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa40094f0 con 0x7f6ab415df50 2023-10-09T06:39:28.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.686+0000 7f6aba062700 1 --2- 172.21.15.163:0/3493600210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ab406bcd0 0x7f6ab40cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:28.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.686+0000 7f6aba863700 1 --2- 172.21.15.163:0/3493600210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ab415df50 0x7f6ab41553d0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f6aac00d180 tx=0x7f6aac009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:28.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.686+0000 7f6aaa7fc700 1 -- 172.21.15.163:0/3493600210 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aac00de20 con 0x7f6ab415df50 2023-10-09T06:39:28.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.687+0000 7f6aaa7fc700 1 -- 172.21.15.163:0/3493600210 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aac00f040 con 0x7f6ab415df50 2023-10-09T06:39:28.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.687+0000 7f6abc2c6700 1 -- 172.21.15.163:0/3493600210 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ab40c8b60 con 0x7f6ab415df50 2023-10-09T06:39:28.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.687+0000 7f6aaa7fc700 1 -- 172.21.15.163:0/3493600210 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aac015690 con 0x7f6ab415df50 2023-10-09T06:39:28.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.687+0000 7f6abc2c6700 1 -- 172.21.15.163:0/3493600210 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ab40c91a0 con 0x7f6ab415df50 2023-10-09T06:39:28.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.688+0000 7f6aaa7fc700 1 -- 172.21.15.163:0/3493600210 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6aac01e3b0 con 0x7f6ab415df50 2023-10-09T06:39:28.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.688+0000 7f6aaa7fc700 1 --2- 172.21.15.163:0/3493600210 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6a98067320 0x7f6a980697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:28.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.689+0000 7f6aaa7fc700 1 -- 172.21.15.163:0/3493600210 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6aac085350 con 0x7f6ab415df50 2023-10-09T06:39:28.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.689+0000 7f6aba062700 1 --2- 172.21.15.163:0/3493600210 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6a98067320 0x7f6a980697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:28.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.689+0000 7f6aba062700 1 --2- 172.21.15.163:0/3493600210 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6a98067320 0x7f6a980697d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0006f00 tx=0x7f6ab0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:28.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.690+0000 7f6ac55beb80 1 Processor -- start 2023-10-09T06:39:28.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.690+0000 7f6ac55beb80 1 -- start start 2023-10-09T06:39:28.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.691+0000 7f6ac55beb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564643b30a50 0x564643b30e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:28.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.691+0000 7f6ac55beb80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564643b7ad00 0x564643b59a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:28.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.691+0000 7f6ac55beb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564643b59f60 0x564643b2ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:28.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.691+0000 7f6ac55beb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x564643a87a70 con 0x564643b59f60 2023-10-09T06:39:28.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.691+0000 7f6ac55beb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564643a85270 con 0x564643b30a50 2023-10-09T06:39:28.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.691+0000 7f6ac55beb80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x564643a81060 con 0x564643b7ad00 2023-10-09T06:39:28.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.691+0000 7f6aba863700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564643b59f60 0x564643b2ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:28.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.691+0000 7f6aba863700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564643b59f60 0x564643b2ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41744/0 (socket says 172.21.15.163:41744) 2023-10-09T06:39:28.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.691+0000 7f6aba863700 1 -- 172.21.15.163:0/1045979998 learned_addr learned my addr 172.21.15.163:0/1045979998 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:28.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.691+0000 7f6aba062700 1 --2- 172.21.15.163:0/1045979998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564643b30a50 0x564643b30e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:28.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.691+0000 7f6ab9861700 1 --2- 172.21.15.163:0/1045979998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564643b7ad00 0x564643b59a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:28.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.692+0000 7f6aba863700 1 -- 172.21.15.163:0/1045979998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564643b7ad00 msgr2=0x564643b59a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:28.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.692+0000 7f6aba863700 1 --2- 172.21.15.163:0/1045979998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564643b7ad00 0x564643b59a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.692+0000 7f6aba863700 1 -- 172.21.15.163:0/1045979998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564643b30a50 msgr2=0x564643b30e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:28.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.692+0000 7f6aba863700 1 --2- 172.21.15.163:0/1045979998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564643b30a50 0x564643b30e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.692+0000 7f6aba863700 1 -- 172.21.15.163:0/1045979998 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56464391b270 con 0x564643b59f60 2023-10-09T06:39:28.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.692+0000 7f6aba062700 1 --2- 172.21.15.163:0/1045979998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564643b30a50 0x564643b30e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:28.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.692+0000 7f6ab9861700 1 --2- 172.21.15.163:0/1045979998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564643b7ad00 0x564643b59a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:28.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.692+0000 7f6aba863700 1 --2- 172.21.15.163:0/1045979998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564643b59f60 0x564643b2ccf0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f6aac083250 tx=0x7f6aac0559f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5a90cf404b7bf9 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:28.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.692+0000 7f6a977fe700 1 -- 172.21.15.163:0/1045979998 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aac058e60 con 0x564643b59f60 2023-10-09T06:39:28.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.692+0000 7f6a977fe700 1 -- 172.21.15.163:0/1045979998 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aac05cb30 con 0x564643b59f60 2023-10-09T06:39:28.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.692+0000 7f6a977fe700 1 -- 172.21.15.163:0/1045979998 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aac0850d0 con 0x564643b59f60 2023-10-09T06:39:28.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.692+0000 7f6ac55beb80 1 -- 172.21.15.163:0/1045979998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564643b59f60 msgr2=0x564643b2ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:28.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.692+0000 7f6ac55beb80 1 --2- 172.21.15.163:0/1045979998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564643b59f60 0x564643b2ccf0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f6aac083250 tx=0x7f6aac0559f0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.692+0000 7f6ac55beb80 1 -- 172.21.15.163:0/1045979998 shutdown_connections 2023-10-09T06:39:28.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.692+0000 7f6ac55beb80 1 --2- 172.21.15.163:0/1045979998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564643b59f60 0x564643b2ccf0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.692+0000 7f6ac55beb80 1 --2- 172.21.15.163:0/1045979998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564643b30a50 0x564643b30e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.701 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.692+0000 7f6ac55beb80 1 --2- 172.21.15.163:0/1045979998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564643b7ad00 0x564643b59a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.692+0000 7f6ac55beb80 1 -- 172.21.15.163:0/1045979998 >> 172.21.15.163:0/1045979998 conn(0x564643b32370 msgr2=0x564643b32740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:28.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.693+0000 7f6ac55beb80 1 -- 172.21.15.163:0/1045979998 shutdown_connections 2023-10-09T06:39:28.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.693+0000 7f6ac55beb80 1 -- 172.21.15.163:0/1045979998 wait complete. 2023-10-09T06:39:28.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.693+0000 7f6ac55beb80 1 -- start start 2023-10-09T06:39:28.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.693+0000 7f6ac55beb80 1 Processor -- start 2023-10-09T06:39:28.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.694+0000 7f6ac55beb80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564643b30a50 0x564643b72be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:28.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.694+0000 7f6ac55beb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564643b59f60 0x564643b5ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:28.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.694+0000 7f6ac55beb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564643b7ad00 0x564643b60480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:28.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.694+0000 7f6ac55beb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x564643a2c890 con 0x564643b59f60 2023-10-09T06:39:28.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.694+0000 7f6ac55beb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564643a47ea0 con 0x564643b7ad00 2023-10-09T06:39:28.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.694+0000 7f6ac55beb80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x564643903e30 con 0x564643b30a50 2023-10-09T06:39:28.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.694+0000 7f6ac55beb80 10 client.0 _ll_register_callbacks cb 0x564643af4110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-09T06:39:28.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.694+0000 7f6ab9861700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564643b59f60 0x564643b5ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:28.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.694+0000 7f6ab9861700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564643b59f60 0x564643b5ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41746/0 (socket says 172.21.15.163:41746) 2023-10-09T06:39:28.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.694+0000 7f6ab9861700 1 -- 172.21.15.163:0/4147516198 learned_addr learned my addr 172.21.15.163:0/4147516198 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:28.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.694+0000 7f6aba863700 1 --2- 172.21.15.163:0/4147516198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564643b7ad00 0x564643b60480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:28.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.694+0000 7f6aba062700 1 --2- 172.21.15.163:0/4147516198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564643b30a50 0x564643b72be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:28.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.694+0000 7f6ab9861700 1 -- 172.21.15.163:0/4147516198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564643b30a50 msgr2=0x564643b72be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:28.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.694+0000 7f6ab9861700 1 --2- 172.21.15.163:0/4147516198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564643b30a50 0x564643b72be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.694+0000 7f6ab9861700 1 -- 172.21.15.163:0/4147516198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564643b7ad00 msgr2=0x564643b60480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:28.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.694+0000 7f6ab9861700 1 --2- 172.21.15.163:0/4147516198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564643b7ad00 0x564643b60480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.694+0000 7f6ab9861700 1 -- 172.21.15.163:0/4147516198 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ab40c91a0 con 0x564643b59f60 2023-10-09T06:39:28.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.695+0000 7f6ab9861700 1 --2- 172.21.15.163:0/4147516198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564643b59f60 0x564643b5ff40 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f6aa40090b0 tx=0x7f6aa4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d91d5078c4044c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:28.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.695+0000 7f6a94ff9700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-10-09T06:39:28.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.695+0000 7f6a94ff9700 1 -- 172.21.15.163:0/4147516198 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4021070 con 0x564643b59f60 2023-10-09T06:39:28.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.695+0000 7f6a94ff9700 1 -- 172.21.15.163:0/4147516198 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(182..182 src has 1..182) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6aa4026ac0 con 0x564643b59f60 2023-10-09T06:39:28.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.695+0000 7f6ac55beb80 1 -- 172.21.15.163:0/4147516198 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564643b25b70 con 0x564643b59f60 2023-10-09T06:39:28.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.695+0000 7f6a94ff9700 1 -- 172.21.15.163:0/4147516198 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6aa4026e90 con 0x564643b59f60 2023-10-09T06:39:28.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.695+0000 7f6ac55beb80 10 client.8784 fetch_fsmap learned FSMap version 122 2023-10-09T06:39:28.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.695+0000 7f6ac55beb80 1 -- 172.21.15.163:0/4147516198 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=2+}) v3 -- 0x5646438bdfe0 con 0x564643b59f60 2023-10-09T06:39:28.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.695+0000 7f6a94ff9700 1 -- 172.21.15.163:0/4147516198 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564643b25b70 con 0x564643b59f60 2023-10-09T06:39:28.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.695+0000 7f6a94ff9700 1 -- 172.21.15.163:0/4147516198 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 122) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f6aa4026020 con 0x564643b59f60 2023-10-09T06:39:28.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.695+0000 7f6ac55beb80 10 client.8784 fetch_fsmap finished waiting for FSMap version 122 2023-10-09T06:39:28.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.695+0000 7f6ac55beb80 10 client.8784 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-09T06:39:28.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.695+0000 7f6ac55beb80 20 client.8784 populate_metadata read hostname 'smithi163' 2023-10-09T06:39:28.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.695+0000 7f6ac55beb80 1 --2- 172.21.15.163:0/4147516198 >> [v2:172.21.15.181:6835/119178917,v1:172.21.15.181:6837/119178917] conn(0x564643b55ea0 0x564643b56270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:28.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.695+0000 7f6ac55beb80 4 client.8784 mds_command: new command op to 8649 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-09T06:39:28.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.695+0000 7f6ac55beb80 1 -- 172.21.15.163:0/4147516198 --> [v2:172.21.15.181:6835/119178917,v1:172.21.15.181:6837/119178917] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564643b02aa0 con 0x564643b55ea0 2023-10-09T06:39:28.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.695+0000 7f6aba062700 1 --2- 172.21.15.163:0/4147516198 >> [v2:172.21.15.181:6835/119178917,v1:172.21.15.181:6837/119178917] conn(0x564643b55ea0 0x564643b56270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:28.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.696+0000 7f6a94ff9700 10 client.8784 ms_handle_connect on v2:172.21.15.181:6835/119178917 2023-10-09T06:39:28.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.696+0000 7f6aba062700 1 --2- 172.21.15.163:0/4147516198 >> [v2:172.21.15.181:6835/119178917,v1:172.21.15.181:6837/119178917] conn(0x564643b55ea0 0x564643b56270 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd6804723a5b5432 server_cookie=939ecd88ef16e791 in_seq=0 out_seq=0 2023-10-09T06:39:28.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.697+0000 7f6a94ff9700 1 -- 172.21.15.163:0/4147516198 <== mds.0 v2:172.21.15.181:6835/119178917 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x564643b02aa0 con 0x564643b55ea0 2023-10-09T06:39:28.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.697+0000 7f6a94ff9700 10 client.8784 handle_command_reply: tid=0 2023-10-09T06:39:28.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.697+0000 7f6ac55beb80 1 client.8784 shutdown 2023-10-09T06:39:28.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.697+0000 7f6ac55beb80 1 -- 172.21.15.163:0/4147516198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564643b59f60 msgr2=0x564643b5ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:28.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.697+0000 7f6ac55beb80 1 --2- 172.21.15.163:0/4147516198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564643b59f60 0x564643b5ff40 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f6aa40090b0 tx=0x7f6aa4017000 comp rx=0 tx=0).stop 2023-10-09T06:39:28.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.697+0000 7f6ac55beb80 1 -- 172.21.15.163:0/4147516198 shutdown_connections 2023-10-09T06:39:28.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.697+0000 7f6ac55beb80 1 --2- 172.21.15.163:0/4147516198 >> [v2:172.21.15.181:6835/119178917,v1:172.21.15.181:6837/119178917] conn(0x564643b55ea0 0x564643b56270 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.697+0000 7f6ac55beb80 1 --2- 172.21.15.163:0/4147516198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564643b7ad00 0x564643b60480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.697+0000 7f6ac55beb80 1 --2- 172.21.15.163:0/4147516198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564643b30a50 0x564643b72be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.697+0000 7f6ac55beb80 1 --2- 172.21.15.163:0/4147516198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564643b59f60 0x564643b5ff40 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.697+0000 7f6a94ff9700 0 client.8784 ms_handle_reset on v2:172.21.15.181:6835/119178917 2023-10-09T06:39:28.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.697+0000 7f6ac55beb80 1 -- 172.21.15.163:0/4147516198 >> 172.21.15.163:0/4147516198 conn(0x564643b32370 msgr2=0x564643b33bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:28.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.697+0000 7f6ac55beb80 1 -- 172.21.15.163:0/4147516198 shutdown_connections 2023-10-09T06:39:28.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.697+0000 7f6ac55beb80 1 -- 172.21.15.163:0/4147516198 wait complete. 2023-10-09T06:39:28.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.697+0000 7f6ac55beb80 20 client.8784 trim_cache size 0 max 16384 2023-10-09T06:39:28.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.714+0000 7f6ac55beb80 1 Processor -- start 2023-10-09T06:39:28.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.714+0000 7f6ac55beb80 1 -- start start 2023-10-09T06:39:28.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.714+0000 7f6ac55beb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564643b30a50 0x564643b86370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:28.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.714+0000 7f6ac55beb80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564643b59f60 0x564643b728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:28.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.714+0000 7f6ac55beb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564643b7ad00 0x564643b2d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:28.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.714+0000 7f6ac55beb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x564643b9a810 con 0x564643b30a50 2023-10-09T06:39:28.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.714+0000 7f6ac55beb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564643b29990 con 0x564643b7ad00 2023-10-09T06:39:28.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.714+0000 7f6ac55beb80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x564643b2ebf0 con 0x564643b59f60 2023-10-09T06:39:28.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.714+0000 7f6aba062700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564643b30a50 0x564643b86370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:28.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.714+0000 7f6aba062700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564643b30a50 0x564643b86370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41748/0 (socket says 172.21.15.163:41748) 2023-10-09T06:39:28.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.714+0000 7f6aba062700 1 -- 172.21.15.163:0/19808540 learned_addr learned my addr 172.21.15.163:0/19808540 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:28.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.715+0000 7f6aba062700 1 -- 172.21.15.163:0/19808540 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564643b59f60 msgr2=0x564643b728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:28.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.715+0000 7f6ab9861700 1 --2- 172.21.15.163:0/19808540 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564643b59f60 0x564643b728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:28.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.715+0000 7f6aba863700 1 --2- 172.21.15.163:0/19808540 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564643b7ad00 0x564643b2d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:28.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.715+0000 7f6aba062700 1 --2- 172.21.15.163:0/19808540 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564643b59f60 0x564643b728b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.715+0000 7f6aba062700 1 -- 172.21.15.163:0/19808540 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564643b7ad00 msgr2=0x564643b2d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:28.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.715+0000 7f6aba062700 1 --2- 172.21.15.163:0/19808540 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564643b7ad00 0x564643b2d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.715+0000 7f6aba062700 1 -- 172.21.15.163:0/19808540 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564643a4f780 con 0x564643b30a50 2023-10-09T06:39:28.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.715+0000 7f6ab9861700 1 --2- 172.21.15.163:0/19808540 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564643b59f60 0x564643b728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:28.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.715+0000 7f6aba863700 1 --2- 172.21.15.163:0/19808540 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564643b7ad00 0x564643b2d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:28.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.715+0000 7f6aba062700 1 --2- 172.21.15.163:0/19808540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564643b30a50 0x564643b86370 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009600 tx=0x7f6ab0026040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a58dca2b698a3cd server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:28.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.715+0000 7f6a97fff700 1 -- 172.21.15.163:0/19808540 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0030070 con 0x564643b30a50 2023-10-09T06:39:28.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.715+0000 7f6a97fff700 1 -- 172.21.15.163:0/19808540 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab0015c60 con 0x564643b30a50 2023-10-09T06:39:28.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.715+0000 7f6a97fff700 1 -- 172.21.15.163:0/19808540 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0029040 con 0x564643b30a50 2023-10-09T06:39:28.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.715+0000 7f6ac55beb80 1 -- 172.21.15.163:0/19808540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564643b30a50 msgr2=0x564643b86370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:28.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.715+0000 7f6ac55beb80 1 --2- 172.21.15.163:0/19808540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564643b30a50 0x564643b86370 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009600 tx=0x7f6ab0026040 comp rx=0 tx=0).stop 2023-10-09T06:39:28.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.716+0000 7f6ac55beb80 1 -- 172.21.15.163:0/19808540 shutdown_connections 2023-10-09T06:39:28.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.716+0000 7f6ac55beb80 1 --2- 172.21.15.163:0/19808540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564643b30a50 0x564643b86370 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.716+0000 7f6ac55beb80 1 --2- 172.21.15.163:0/19808540 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564643b7ad00 0x564643b2d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.716+0000 7f6ac55beb80 1 --2- 172.21.15.163:0/19808540 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564643b59f60 0x564643b728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.719 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.716+0000 7f6ac55beb80 1 -- 172.21.15.163:0/19808540 >> 172.21.15.163:0/19808540 conn(0x564643b32370 msgr2=0x564643b2c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:28.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.716+0000 7f6ac55beb80 1 -- 172.21.15.163:0/19808540 shutdown_connections 2023-10-09T06:39:28.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.716+0000 7f6ac55beb80 1 -- 172.21.15.163:0/19808540 wait complete. 2023-10-09T06:39:28.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.716+0000 7f6ac55beb80 1 -- start start 2023-10-09T06:39:28.720 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.716+0000 7f6ac55beb80 1 Processor -- start 2023-10-09T06:39:28.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.717+0000 7f6ac55beb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564643b30a50 0x564643b71ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:28.723 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.717+0000 7f6ac55beb80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564643b59f60 0x564643b58a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:28.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.717+0000 7f6ac55beb80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564643b7ad00 0x564643b58f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:28.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.717+0000 7f6ac55beb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x564643b71ff0 con 0x564643b30a50 2023-10-09T06:39:28.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.717+0000 7f6ac55beb80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x564643b7a340 con 0x564643b7ad00 2023-10-09T06:39:28.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.717+0000 7f6ac55beb80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x564643b72220 con 0x564643b59f60 2023-10-09T06:39:28.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.717+0000 7f6ac55beb80 10 client.0 _ll_register_callbacks cb 0x564643af4110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-09T06:39:28.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.717+0000 7f6aba062700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564643b30a50 0x564643b71ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:28.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.717+0000 7f6aba062700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564643b30a50 0x564643b71ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41762/0 (socket says 172.21.15.163:41762) 2023-10-09T06:39:28.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.717+0000 7f6aba062700 1 -- 172.21.15.163:0/1059370610 learned_addr learned my addr 172.21.15.163:0/1059370610 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:28.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.717+0000 7f6ab9861700 1 --2- 172.21.15.163:0/1059370610 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564643b59f60 0x564643b58a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:28.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.717+0000 7f6aba863700 1 --2- 172.21.15.163:0/1059370610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564643b7ad00 0x564643b58f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:28.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.717+0000 7f6aba062700 1 -- 172.21.15.163:0/1059370610 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564643b59f60 msgr2=0x564643b58a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:28.724 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.717+0000 7f6aba062700 1 --2- 172.21.15.163:0/1059370610 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564643b59f60 0x564643b58a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.717+0000 7f6aba062700 1 -- 172.21.15.163:0/1059370610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564643b7ad00 msgr2=0x564643b58f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:28.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.717+0000 7f6aba062700 1 --2- 172.21.15.163:0/1059370610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564643b7ad00 0x564643b58f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.717+0000 7f6aba062700 1 -- 172.21.15.163:0/1059370610 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ab000e050 con 0x564643b30a50 2023-10-09T06:39:28.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.717+0000 7f6aba863700 1 --2- 172.21.15.163:0/1059370610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564643b7ad00 0x564643b58f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:28.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.718+0000 7f6aba062700 1 --2- 172.21.15.163:0/1059370610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564643b30a50 0x564643b71ab0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f6ab002e040 tx=0x7f6ab002a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50780519ad4edf02 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:28.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.718+0000 7f6a94ff9700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-10-09T06:39:28.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.718+0000 7f6a94ff9700 1 -- 172.21.15.163:0/1059370610 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0030050 con 0x564643b30a50 2023-10-09T06:39:28.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.718+0000 7f6a94ff9700 1 -- 172.21.15.163:0/1059370610 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(182..182 src has 1..182) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6ab0024b40 con 0x564643b30a50 2023-10-09T06:39:28.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.718+0000 7f6ac55beb80 1 -- 172.21.15.163:0/1059370610 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6aac05cb30 con 0x564643b30a50 2023-10-09T06:39:28.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.718+0000 7f6a94ff9700 1 -- 172.21.15.163:0/1059370610 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=122) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ab0029040 con 0x564643b30a50 2023-10-09T06:39:28.725 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.718+0000 7f6ac55beb80 10 client.8790 fetch_fsmap learned FSMap version 122 2023-10-09T06:39:28.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.718+0000 7f6ac55beb80 1 -- 172.21.15.163:0/1059370610 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=122,monmap=2+}) v3 -- 0x564643a36470 con 0x564643b30a50 2023-10-09T06:39:28.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.718+0000 7f6a94ff9700 1 -- 172.21.15.163:0/1059370610 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aac05cb30 con 0x564643b30a50 2023-10-09T06:39:28.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.718+0000 7f6a94ff9700 1 -- 172.21.15.163:0/1059370610 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 122) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f6ab0016420 con 0x564643b30a50 2023-10-09T06:39:28.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.718+0000 7f6ac55beb80 10 client.8790 fetch_fsmap finished waiting for FSMap version 122 2023-10-09T06:39:28.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.718+0000 7f6ac55beb80 10 client.8790 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-09T06:39:28.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.718+0000 7f6ac55beb80 20 client.8790 populate_metadata read hostname 'smithi163' 2023-10-09T06:39:28.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.718+0000 7f6ac55beb80 1 --2- 172.21.15.163:0/1059370610 >> [v2:172.21.15.181:6835/119178917,v1:172.21.15.181:6837/119178917] conn(0x564643b6d810 0x564643b6dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:28.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.718+0000 7f6ac55beb80 4 client.8790 mds_command: new command op to 8649 tid=0[{"prefix": "status"}] 2023-10-09T06:39:28.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.718+0000 7f6ac55beb80 1 -- 172.21.15.163:0/1059370610 --> [v2:172.21.15.181:6835/119178917,v1:172.21.15.181:6837/119178917] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564643a23e00 con 0x564643b6d810 2023-10-09T06:39:28.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.718+0000 7f6ab9861700 1 --2- 172.21.15.163:0/1059370610 >> [v2:172.21.15.181:6835/119178917,v1:172.21.15.181:6837/119178917] conn(0x564643b6d810 0x564643b6dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:28.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.719+0000 7f6a94ff9700 10 client.8790 ms_handle_connect on v2:172.21.15.181:6835/119178917 2023-10-09T06:39:28.726 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.719+0000 7f6ab9861700 1 --2- 172.21.15.163:0/1059370610 >> [v2:172.21.15.181:6835/119178917,v1:172.21.15.181:6837/119178917] conn(0x564643b6d810 0x564643b6dbe0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acda280d9994906 server_cookie=a2b1109f22b2df08 in_seq=0 out_seq=0 2023-10-09T06:39:28.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.719+0000 7f6a94ff9700 1 -- 172.21.15.163:0/1059370610 <== mds.0 v2:172.21.15.181:6835/119178917 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x564643a23e00 con 0x564643b6d810 2023-10-09T06:39:28.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.719+0000 7f6a94ff9700 10 client.8790 handle_command_reply: tid=0 2023-10-09T06:39:28.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.719+0000 7f6ac55beb80 1 client.8790 shutdown 2023-10-09T06:39:28.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.719+0000 7f6ac55beb80 1 -- 172.21.15.163:0/1059370610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564643b30a50 msgr2=0x564643b71ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:28.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.719+0000 7f6ac55beb80 1 --2- 172.21.15.163:0/1059370610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564643b30a50 0x564643b71ab0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f6ab002e040 tx=0x7f6ab002a000 comp rx=0 tx=0).stop 2023-10-09T06:39:28.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.719+0000 7f6ac55beb80 1 -- 172.21.15.163:0/1059370610 shutdown_connections 2023-10-09T06:39:28.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.719+0000 7f6ac55beb80 1 --2- 172.21.15.163:0/1059370610 >> [v2:172.21.15.181:6835/119178917,v1:172.21.15.181:6837/119178917] conn(0x564643b6d810 0x564643b6dbe0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.719+0000 7f6ac55beb80 1 --2- 172.21.15.163:0/1059370610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564643b30a50 0x564643b71ab0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.719+0000 7f6a94ff9700 0 client.8790 ms_handle_reset on v2:172.21.15.181:6835/119178917 2023-10-09T06:39:28.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.719+0000 7f6ac55beb80 1 --2- 172.21.15.163:0/1059370610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x564643b7ad00 0x564643b58f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.719+0000 7f6ac55beb80 1 --2- 172.21.15.163:0/1059370610 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x564643b59f60 0x564643b58a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.719+0000 7f6ac55beb80 1 -- 172.21.15.163:0/1059370610 >> 172.21.15.163:0/1059370610 conn(0x564643b32370 msgr2=0x564643b33b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:28.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.719+0000 7f6ac55beb80 1 -- 172.21.15.163:0/1059370610 shutdown_connections 2023-10-09T06:39:28.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.719+0000 7f6ac55beb80 1 -- 172.21.15.163:0/1059370610 wait complete. 2023-10-09T06:39:28.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.719+0000 7f6ac55beb80 20 client.8790 trim_cache size 0 max 16384 2023-10-09T06:39:28.728 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:39:28.728 INFO:teuthology.orchestra.run.smithi163.stdout: "cluster_fsid": "2520b8f7-539f-4bae-9f17-1fce66f66ddd", 2023-10-09T06:39:28.728 INFO:teuthology.orchestra.run.smithi163.stdout: "whoami": 0, 2023-10-09T06:39:28.728 INFO:teuthology.orchestra.run.smithi163.stdout: "id": 8649, 2023-10-09T06:39:28.728 INFO:teuthology.orchestra.run.smithi163.stdout: "want_state": "up:active", 2023-10-09T06:39:28.729 INFO:teuthology.orchestra.run.smithi163.stdout: "state": "up:active", 2023-10-09T06:39:28.729 INFO:teuthology.orchestra.run.smithi163.stdout: "fs_name": "cephfs", 2023-10-09T06:39:28.729 INFO:teuthology.orchestra.run.smithi163.stdout: "rank_uptime": 5.7971586669999997, 2023-10-09T06:39:28.729 INFO:teuthology.orchestra.run.smithi163.stdout: "mdsmap_epoch": 122, 2023-10-09T06:39:28.729 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch": 182, 2023-10-09T06:39:28.729 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch_barrier": 182, 2023-10-09T06:39:28.729 INFO:teuthology.orchestra.run.smithi163.stdout: "uptime": 10.926465769 2023-10-09T06:39:28.729 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:39:28.730 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.720+0000 7f6abc2c6700 1 -- 172.21.15.163:0/3493600210 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6a98067320 msgr2=0x7f6a980697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:28.730 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.720+0000 7f6abc2c6700 1 --2- 172.21.15.163:0/3493600210 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6a98067320 0x7f6a980697d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0006f00 tx=0x7f6ab0008040 comp rx=0 tx=0).stop 2023-10-09T06:39:28.730 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.720+0000 7f6abc2c6700 1 -- 172.21.15.163:0/3493600210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ab415df50 msgr2=0x7f6ab41553d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:28.730 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.720+0000 7f6abc2c6700 1 --2- 172.21.15.163:0/3493600210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ab415df50 0x7f6ab41553d0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f6aac00d180 tx=0x7f6aac009a50 comp rx=0 tx=0).stop 2023-10-09T06:39:28.731 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.720+0000 7f6abc2c6700 1 -- 172.21.15.163:0/3493600210 shutdown_connections 2023-10-09T06:39:28.731 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.720+0000 7f6abc2c6700 1 --2- 172.21.15.163:0/3493600210 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6a98067320 0x7f6a980697d0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.731 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.720+0000 7f6abc2c6700 1 --2- 172.21.15.163:0/3493600210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ab415df50 0x7f6ab41553d0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.731 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.720+0000 7f6abc2c6700 1 --2- 172.21.15.163:0/3493600210 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ab406bcd0 0x7f6ab40cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.731 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.720+0000 7f6abc2c6700 1 --2- 172.21.15.163:0/3493600210 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ab406c0a0 0x7f6ab4154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.731 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.720+0000 7f6abc2c6700 1 -- 172.21.15.163:0/3493600210 >> 172.21.15.163:0/3493600210 conn(0x7f6ab40ce600 msgr2=0x7f6ab4069480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:28.731 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.720+0000 7f6abc2c6700 1 -- 172.21.15.163:0/3493600210 shutdown_connections 2023-10-09T06:39:28.731 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.720+0000 7f6abc2c6700 1 -- 172.21.15.163:0/3493600210 wait complete. 2023-10-09T06:39:28.732 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-09T06:39:28.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.846+0000 7f131817c700 1 Processor -- start 2023-10-09T06:39:28.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.846+0000 7f131817c700 1 -- start start 2023-10-09T06:39:28.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.847+0000 7f131817c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1310159890 0x7f131015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:28.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.847+0000 7f131817c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1310069560 0x7f13100699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:28.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.847+0000 7f131817c700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1310069ee0 con 0x7f1310159c60 2023-10-09T06:39:28.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.847+0000 7f131817c700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f131015a160 con 0x7f1310069560 2023-10-09T06:39:28.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.847+0000 7f131817c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13100d13a0 con 0x7f1310159890 2023-10-09T06:39:28.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.847+0000 7f1316719700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1310069560 0x7f13100699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:28.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.847+0000 7f1315717700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f1310159c60 0x7f131015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60992/0 (socket says 172.21.15.163:60992) 2023-10-09T06:39:28.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.847+0000 7f1315717700 1 -- 172.21.15.163:0/1136808714 learned_addr learned my addr 172.21.15.163:0/1136808714 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:28.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.847+0000 7f1315f18700 1 --2- 172.21.15.163:0/1136808714 >> v2:172.21.15.181:3300/0 conn(0x7f1310159890 0x7f131015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:28.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.847+0000 7f1316719700 1 -- 172.21.15.163:0/1136808714 >> v2:172.21.15.181:3300/0 conn(0x7f1310159890 msgr2=0x7f131015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:28.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.847+0000 7f1316719700 1 --2- 172.21.15.163:0/1136808714 >> v2:172.21.15.181:3300/0 conn(0x7f1310159890 0x7f131015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.847+0000 7f1316719700 1 -- 172.21.15.163:0/1136808714 >> v1:172.21.15.163:6789/0 conn(0x7f1310159c60 legacy=0x7f131015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:28.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.847+0000 7f1316719700 1 -- 172.21.15.163:0/1136808714 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f131015e140 con 0x7f1310069560 2023-10-09T06:39:28.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.847+0000 7f1314f16700 1 -- 172.21.15.163:0/1136808714 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 396250742 0 0) 0x7f1310069ee0 con 0x7f1310159c60 2023-10-09T06:39:28.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.847+0000 7f1316719700 1 --2- 172.21.15.163:0/1136808714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1310069560 0x7f13100699a0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f13100d1090 tx=0x7f13040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8d03f97fcf33e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:28.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.847+0000 7f1314f16700 1 -- 172.21.15.163:0/1136808714 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130400b6b0 con 0x7f1310069560 2023-10-09T06:39:28.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.847+0000 7f1314f16700 1 -- 172.21.15.163:0/1136808714 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f130400b810 con 0x7f1310069560 2023-10-09T06:39:28.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.847+0000 7f1314f16700 1 -- 172.21.15.163:0/1136808714 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130400bbb0 con 0x7f1310069560 2023-10-09T06:39:28.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.848+0000 7f131817c700 1 -- 172.21.15.163:0/1136808714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1310069560 msgr2=0x7f13100699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:28.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.848+0000 7f131817c700 1 --2- 172.21.15.163:0/1136808714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1310069560 0x7f13100699a0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f13100d1090 tx=0x7f13040091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.848+0000 7f131817c700 1 -- 172.21.15.163:0/1136808714 shutdown_connections 2023-10-09T06:39:28.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.848+0000 7f131817c700 1 --2- 172.21.15.163:0/1136808714 >> v2:172.21.15.181:3300/0 conn(0x7f1310159890 0x7f131015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.848+0000 7f131817c700 1 --2- 172.21.15.163:0/1136808714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1310069560 0x7f13100699a0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.848+0000 7f131817c700 1 -- 172.21.15.163:0/1136808714 >> 172.21.15.163:0/1136808714 conn(0x7f1310068e20 msgr2=0x7f13100d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:28.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.848+0000 7f131817c700 1 -- 172.21.15.163:0/1136808714 shutdown_connections 2023-10-09T06:39:28.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.848+0000 7f131817c700 1 -- 172.21.15.163:0/1136808714 wait complete. 2023-10-09T06:39:28.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.849+0000 7f131817c700 1 Processor -- start 2023-10-09T06:39:28.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.849+0000 7f131817c700 1 -- start start 2023-10-09T06:39:28.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.850+0000 7f131817c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1310069560 0x7f13100b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:28.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.850+0000 7f131817c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1310159890 0x7f13100cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:28.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.850+0000 7f131817c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1310159c60 0x7f13100b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:28.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.850+0000 7f131817c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131016dfd0 con 0x7f1310159c60 2023-10-09T06:39:28.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.850+0000 7f131817c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131016d6a0 con 0x7f1310159890 2023-10-09T06:39:28.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.850+0000 7f131817c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f131016d980 con 0x7f1310069560 2023-10-09T06:39:28.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.850+0000 7f1316719700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1310159c60 0x7f13100b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:28.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.850+0000 7f1316719700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1310159c60 0x7f13100b1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41778/0 (socket says 172.21.15.163:41778) 2023-10-09T06:39:28.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.850+0000 7f1316719700 1 -- 172.21.15.163:0/177080315 learned_addr learned my addr 172.21.15.163:0/177080315 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:28.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.850+0000 7f1315f18700 1 --2- 172.21.15.163:0/177080315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1310069560 0x7f13100b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:28.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.850+0000 7f1315717700 1 --2- 172.21.15.163:0/177080315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1310159890 0x7f13100cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:28.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.850+0000 7f1316719700 1 -- 172.21.15.163:0/177080315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1310069560 msgr2=0x7f13100b5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:28.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.850+0000 7f1316719700 1 --2- 172.21.15.163:0/177080315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1310069560 0x7f13100b5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.850+0000 7f1316719700 1 -- 172.21.15.163:0/177080315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1310159890 msgr2=0x7f13100cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:28.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.850+0000 7f1316719700 1 --2- 172.21.15.163:0/177080315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1310159890 0x7f13100cf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:28.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.850+0000 7f1316719700 1 -- 172.21.15.163:0/177080315 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1304009420 con 0x7f1310159c60 2023-10-09T06:39:28.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.850+0000 7f1315717700 1 --2- 172.21.15.163:0/177080315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1310159890 0x7f13100cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:28.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.850+0000 7f1316719700 1 --2- 172.21.15.163:0/177080315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1310159c60 0x7f13100b1590 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f1310070e20 tx=0x7f1304017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:28.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.851+0000 7f1301ffb700 1 -- 172.21.15.163:0/177080315 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1304011630 con 0x7f1310159c60 2023-10-09T06:39:28.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.851+0000 7f1301ffb700 1 -- 172.21.15.163:0/177080315 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1304011790 con 0x7f1310159c60 2023-10-09T06:39:28.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.851+0000 7f131817c700 1 -- 172.21.15.163:0/177080315 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f131016dc60 con 0x7f1310159c60 2023-10-09T06:39:28.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.851+0000 7f1301ffb700 1 -- 172.21.15.163:0/177080315 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1304011b30 con 0x7f1310159c60 2023-10-09T06:39:28.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.851+0000 7f131817c700 1 -- 172.21.15.163:0/177080315 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13100cfc10 con 0x7f1310159c60 2023-10-09T06:39:28.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.852+0000 7f131817c700 1 -- 172.21.15.163:0/177080315 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13100cc600 con 0x7f1310159c60 2023-10-09T06:39:28.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.852+0000 7f1301ffb700 1 -- 172.21.15.163:0/177080315 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f130402e440 con 0x7f1310159c60 2023-10-09T06:39:28.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.852+0000 7f1301ffb700 1 --2- 172.21.15.163:0/177080315 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f12f4067320 0x7f12f40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:28.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.853+0000 7f1301ffb700 1 -- 172.21.15.163:0/177080315 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1304086930 con 0x7f1310159c60 2023-10-09T06:39:28.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.853+0000 7f1315f18700 1 --2- 172.21.15.163:0/177080315 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f12f4067320 0x7f12f40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:28.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.853+0000 7f1315f18700 1 --2- 172.21.15.163:0/177080315 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f12f4067320 0x7f12f40697d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f13080075a0 tx=0x7f13080097a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:28.859 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:28.857+0000 7f1301ffb700 1 -- 172.21.15.163:0/177080315 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1304053580 con 0x7f1310159c60 2023-10-09T06:39:29.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.082+0000 7f131817c700 1 -- 172.21.15.163:0/177080315 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f131004c3c0 con 0x7f1310159c60 2023-10-09T06:39:29.084 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.083+0000 7f1301ffb700 1 -- 172.21.15.163:0/177080315 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v122) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1304056ba0 con 0x7f1310159c60 2023-10-09T06:39:29.084 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-09T06:39:29.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.085+0000 7f131817c700 1 -- 172.21.15.163:0/177080315 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f12f4067320 msgr2=0x7f12f40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:29.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.085+0000 7f131817c700 1 --2- 172.21.15.163:0/177080315 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f12f4067320 0x7f12f40697d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f13080075a0 tx=0x7f13080097a0 comp rx=0 tx=0).stop 2023-10-09T06:39:29.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.085+0000 7f131817c700 1 -- 172.21.15.163:0/177080315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1310159c60 msgr2=0x7f13100b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:29.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.085+0000 7f131817c700 1 --2- 172.21.15.163:0/177080315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1310159c60 0x7f13100b1590 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f1310070e20 tx=0x7f1304017000 comp rx=0 tx=0).stop 2023-10-09T06:39:29.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.085+0000 7f131817c700 1 -- 172.21.15.163:0/177080315 shutdown_connections 2023-10-09T06:39:29.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.085+0000 7f131817c700 1 --2- 172.21.15.163:0/177080315 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f12f4067320 0x7f12f40697d0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:29.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.085+0000 7f131817c700 1 --2- 172.21.15.163:0/177080315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1310159c60 0x7f13100b1590 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:29.086 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.085+0000 7f131817c700 1 --2- 172.21.15.163:0/177080315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1310069560 0x7f13100b5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:29.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.085+0000 7f131817c700 1 --2- 172.21.15.163:0/177080315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1310159890 0x7f13100cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:29.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.085+0000 7f131817c700 1 -- 172.21.15.163:0/177080315 >> 172.21.15.163:0/177080315 conn(0x7f1310068e20 msgr2=0x7f1310157e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:29.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.085+0000 7f131817c700 1 -- 172.21.15.163:0/177080315 shutdown_connections 2023-10-09T06:39:29.087 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.085+0000 7f131817c700 1 -- 172.21.15.163:0/177080315 wait complete. 2023-10-09T06:39:29.097 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-09T06:39:29.098 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-09T06:39:29.098 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi163.front.sepia.ceph.com 2023-10-09T06:39:29.098 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-09T06:39:29.098 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-09T06:39:29.098 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-09T06:39:29.098 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-09T06:39:29.098 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-09T06:39:29.098 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:39:29.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:39:29.099 DEBUG:teuthology.orchestra.run.smithi163:> ip addr 2023-10-09T06:39:29.114 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:39:29.115 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:39:29.115 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:39:29.115 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:39:29.115 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2023-10-09T06:39:29.115 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:39:29.115 INFO:teuthology.orchestra.run.smithi163.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:39:29.115 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:06 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:39:29.115 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f0 2023-10-09T06:39:29.115 INFO:teuthology.orchestra.run.smithi163.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:39:29.115 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:07 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:39:29.116 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f1 2023-10-09T06:39:29.116 INFO:teuthology.orchestra.run.smithi163.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:39:29.116 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5a brd ff:ff:ff:ff:ff:ff 2023-10-09T06:39:29.116 INFO:teuthology.orchestra.run.smithi163.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-09T06:39:29.116 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5b brd ff:ff:ff:ff:ff:ff 2023-10-09T06:39:29.116 INFO:teuthology.orchestra.run.smithi163.stdout: inet 172.21.15.163/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-09T06:39:29.116 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft 41954sec preferred_lft 41954sec 2023-10-09T06:39:29.116 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::ec4:7aff:fe88:7d5b/64 scope link 2023-10-09T06:39:29.116 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:39:29.116 INFO:teuthology.orchestra.run.smithi163.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-09T06:39:29.116 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 2e:01:84:b9:f4:8e brd ff:ff:ff:ff:ff:ff 2023-10-09T06:39:29.116 INFO:teuthology.orchestra.run.smithi163.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-09T06:39:29.116 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:39:29.117 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::d03c:93ff:fe9e:c8cb/64 scope link 2023-10-09T06:39:29.117 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:39:29.117 INFO:teuthology.orchestra.run.smithi163.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:39:29.117 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 2e:01:84:b9:f4:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-09T06:39:29.117 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::2c01:84ff:feb9:f48e/64 scope link 2023-10-09T06:39:29.117 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:39:29.117 INFO:teuthology.orchestra.run.smithi163.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:39:29.117 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether c6:88:3e:d7:c1:56 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-09T06:39:29.117 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::c488:3eff:fed7:c156/64 scope link 2023-10-09T06:39:29.117 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:39:29.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:39:29.118 DEBUG:teuthology.orchestra.run.smithi163:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-09T06:39:29.177 INFO:teuthology.orchestra.run.smithi163.stdout:1 2023-10-09T06:39:29.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:39:29.179 DEBUG:teuthology.orchestra.run.smithi163:> route 2023-10-09T06:39:29.234 INFO:teuthology.orchestra.run.smithi163.stdout:Kernel IP routing table 2023-10-09T06:39:29.235 INFO:teuthology.orchestra.run.smithi163.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-09T06:39:29.235 INFO:teuthology.orchestra.run.smithi163.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-09T06:39:29.235 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-09T06:39:29.235 INFO:teuthology.orchestra.run.smithi163.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-09T06:39:29.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:39:29.235 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-09T06:39:29.235 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-09T06:39:29.236 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-09T06:39:29.236 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-09T06:39:29.236 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-09T06:39:29.236 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-09T06:39:29.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:39:29.335 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2023-10-09T06:39:29.349 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-09T06:39:29.350 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-09T06:39:29.350 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:39:29.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:39:29.350 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set brx.0 up 2023-10-09T06:39:29.414 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:39:29.415 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-09T06:39:29.469 INFO:teuthology.orchestra.run.smithi163.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:39:29.470 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:39:29.470 DEBUG:teuthology.orchestra.run.smithi163:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-09T06:39:29.524 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-09T06:39:29.524 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-09T06:39:29.524 DEBUG:teuthology.orchestra.run.smithi163:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:39:29.603 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:39:29.603 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:39:29.621 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:39:29.622 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:39:29.622 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:39:29.622 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with new device syntax: 0@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:39:29.622 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:39:29.622 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:39:29.622 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:39:29.622 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:39:29.645 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:39:29.645 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:39:29.645 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:39:29.646 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with new device syntax: 0@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:39:29.646 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:39:29.646 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:39:29.646 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:39:29.646 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:39:29.646 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790" 2023-10-09T06:39:29.646 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790:/ 2023-10-09T06:39:29.646 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=2520b8f7-539f-4bae-9f17-1fce66f66ddd" will pass to kernel 2023-10-09T06:39:29.647 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-09T06:39:29.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:39:29.648 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-09T06:39:29.713 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-09T06:39:29.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.868+0000 7f0433861700 1 Processor -- start 2023-10-09T06:39:29.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.868+0000 7f0433861700 1 -- start start 2023-10-09T06:39:29.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.869+0000 7f0433861700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f042c06bcd0 0x7f042c0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:29.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.869+0000 7f0433861700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f042c06c0a0 0x7f042c15a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:29.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.869+0000 7f0433861700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f042c15df50 0x7f042c15e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:29.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.869+0000 7f0433861700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f042c15d7e0 con 0x7f042c06bcd0 2023-10-09T06:39:29.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.869+0000 7f0433861700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f042c15dac0 con 0x7f042c15df50 2023-10-09T06:39:29.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.869+0000 7f0433861700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f042c15aa30 con 0x7f042c06c0a0 2023-10-09T06:39:29.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.869+0000 7f04315fd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f042c06bcd0 0x7f042c0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:29.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.869+0000 7f04315fd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f042c06bcd0 0x7f042c0caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41794/0 (socket says 172.21.15.163:41794) 2023-10-09T06:39:29.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.869+0000 7f04315fd700 1 -- 172.21.15.163:0/2105414833 learned_addr learned my addr 172.21.15.163:0/2105414833 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:29.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.869+0000 7f0430dfc700 1 --2- 172.21.15.163:0/2105414833 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f042c06c0a0 0x7f042c15a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:29.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.869+0000 7f0431dfe700 1 --2- 172.21.15.163:0/2105414833 >> v2:172.21.15.181:3300/0 conn(0x7f042c15df50 0x7f042c15e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:29.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.869+0000 7f04315fd700 1 -- 172.21.15.163:0/2105414833 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f042c06c0a0 msgr2=0x7f042c15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:29.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.869+0000 7f04315fd700 1 --2- 172.21.15.163:0/2105414833 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f042c06c0a0 0x7f042c15a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:29.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.869+0000 7f04315fd700 1 -- 172.21.15.163:0/2105414833 >> v2:172.21.15.181:3300/0 conn(0x7f042c15df50 msgr2=0x7f042c15e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:29.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.869+0000 7f04315fd700 1 --2- 172.21.15.163:0/2105414833 >> v2:172.21.15.181:3300/0 conn(0x7f042c15df50 0x7f042c15e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:29.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.869+0000 7f04315fd700 1 -- 172.21.15.163:0/2105414833 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f042c15ad10 con 0x7f042c06bcd0 2023-10-09T06:39:29.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.869+0000 7f04315fd700 1 --2- 172.21.15.163:0/2105414833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f042c06bcd0 0x7f042c0caa90 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f042c0b4600 tx=0x7f0428009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be95af11afdda1e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:29.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.869+0000 7f0423fff700 1 -- 172.21.15.163:0/2105414833 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0428009e70 con 0x7f042c06bcd0 2023-10-09T06:39:29.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.869+0000 7f0423fff700 1 -- 172.21.15.163:0/2105414833 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042800b860 con 0x7f042c06bcd0 2023-10-09T06:39:29.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.869+0000 7f0423fff700 1 -- 172.21.15.163:0/2105414833 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042800bc20 con 0x7f042c06bcd0 2023-10-09T06:39:29.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.870+0000 7f0433861700 1 -- 172.21.15.163:0/2105414833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f042c06bcd0 msgr2=0x7f042c0caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:29.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.870+0000 7f0433861700 1 --2- 172.21.15.163:0/2105414833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f042c06bcd0 0x7f042c0caa90 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f042c0b4600 tx=0x7f0428009250 comp rx=0 tx=0).stop 2023-10-09T06:39:29.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.870+0000 7f0433861700 1 -- 172.21.15.163:0/2105414833 shutdown_connections 2023-10-09T06:39:29.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.870+0000 7f0433861700 1 --2- 172.21.15.163:0/2105414833 >> v2:172.21.15.181:3300/0 conn(0x7f042c15df50 0x7f042c15e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:29.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.870+0000 7f0433861700 1 --2- 172.21.15.163:0/2105414833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f042c06bcd0 0x7f042c0caa90 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:29.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.870+0000 7f0433861700 1 --2- 172.21.15.163:0/2105414833 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f042c06c0a0 0x7f042c15a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:29.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.870+0000 7f0433861700 1 -- 172.21.15.163:0/2105414833 >> 172.21.15.163:0/2105414833 conn(0x7f042c0ce600 msgr2=0x7f042c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:29.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.870+0000 7f0433861700 1 -- 172.21.15.163:0/2105414833 shutdown_connections 2023-10-09T06:39:29.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.870+0000 7f0433861700 1 -- 172.21.15.163:0/2105414833 wait complete. 2023-10-09T06:39:29.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.871+0000 7f0433861700 1 Processor -- start 2023-10-09T06:39:29.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.871+0000 7f0433861700 1 -- start start 2023-10-09T06:39:29.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.872+0000 7f0433861700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f042c06bcd0 0x7f042c1410c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:29.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.872+0000 7f0433861700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f042c06c0a0 0x7f042c0c4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:29.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.872+0000 7f0433861700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f042c15df50 0x7f042c0c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:29.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.872+0000 7f0433861700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042c1542d0 con 0x7f042c15df50 2023-10-09T06:39:29.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.872+0000 7f0433861700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042c0b31f0 con 0x7f042c06bcd0 2023-10-09T06:39:29.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.872+0000 7f0433861700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f042c1539a0 con 0x7f042c06c0a0 2023-10-09T06:39:29.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.872+0000 7f0431dfe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f042c15df50 0x7f042c0c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:29.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.872+0000 7f0431dfe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f042c15df50 0x7f042c0c48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41806/0 (socket says 172.21.15.163:41806) 2023-10-09T06:39:29.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.872+0000 7f0431dfe700 1 -- 172.21.15.163:0/1427416426 learned_addr learned my addr 172.21.15.163:0/1427416426 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:29.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.872+0000 7f0430dfc700 1 --2- 172.21.15.163:0/1427416426 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f042c06c0a0 0x7f042c0c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:29.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.872+0000 7f0431dfe700 1 -- 172.21.15.163:0/1427416426 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f042c06c0a0 msgr2=0x7f042c0c4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:29.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.872+0000 7f04315fd700 1 --2- 172.21.15.163:0/1427416426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f042c06bcd0 0x7f042c1410c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:29.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.872+0000 7f0431dfe700 1 --2- 172.21.15.163:0/1427416426 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f042c06c0a0 0x7f042c0c4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:29.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.873+0000 7f0431dfe700 1 -- 172.21.15.163:0/1427416426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f042c06bcd0 msgr2=0x7f042c1410c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:29.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.873+0000 7f0431dfe700 1 --2- 172.21.15.163:0/1427416426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f042c06bcd0 0x7f042c1410c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:29.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.873+0000 7f0431dfe700 1 -- 172.21.15.163:0/1427416426 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04280094f0 con 0x7f042c15df50 2023-10-09T06:39:29.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.873+0000 7f0430dfc700 1 --2- 172.21.15.163:0/1427416426 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f042c06c0a0 0x7f042c0c4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:29.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.873+0000 7f04315fd700 1 --2- 172.21.15.163:0/1427416426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f042c06bcd0 0x7f042c1410c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:29.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.873+0000 7f0431dfe700 1 --2- 172.21.15.163:0/1427416426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f042c15df50 0x7f042c0c48b0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f0424009fa0 tx=0x7f0424009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:29.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.873+0000 7f04217fa700 1 -- 172.21.15.163:0/1427416426 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042400ed20 con 0x7f042c15df50 2023-10-09T06:39:29.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.873+0000 7f04217fa700 1 -- 172.21.15.163:0/1427416426 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042400ee80 con 0x7f042c15df50 2023-10-09T06:39:29.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.873+0000 7f04217fa700 1 -- 172.21.15.163:0/1427416426 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0424014670 con 0x7f042c15df50 2023-10-09T06:39:29.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.873+0000 7f0433861700 1 -- 172.21.15.163:0/1427416426 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f042c153c80 con 0x7f042c15df50 2023-10-09T06:39:29.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.873+0000 7f0433861700 1 -- 172.21.15.163:0/1427416426 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f042c141600 con 0x7f042c15df50 2023-10-09T06:39:29.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.874+0000 7f0433861700 1 -- 172.21.15.163:0/1427416426 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f042c154440 con 0x7f042c15df50 2023-10-09T06:39:29.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.874+0000 7f04217fa700 1 -- 172.21.15.163:0/1427416426 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f04240149e0 con 0x7f042c15df50 2023-10-09T06:39:29.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.875+0000 7f04217fa700 1 --2- 172.21.15.163:0/1427416426 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0410067320 0x7f04100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:29.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.875+0000 7f04217fa700 1 -- 172.21.15.163:0/1427416426 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f04240853f0 con 0x7f042c15df50 2023-10-09T06:39:29.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.875+0000 7f04315fd700 1 --2- 172.21.15.163:0/1427416426 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0410067320 0x7f04100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:29.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.875+0000 7f04315fd700 1 --2- 172.21.15.163:0/1427416426 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0410067320 0x7f04100697d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f042c0b4600 tx=0x7f0428017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:29.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:29.879+0000 7f04217fa700 1 -- 172.21.15.163:0/1427416426 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0424052080 con 0x7f042c15df50 2023-10-09T06:39:30.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.106+0000 7f0433861700 1 -- 172.21.15.163:0/1427416426 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f042c04c3c0 con 0x7f042c15df50 2023-10-09T06:39:30.107 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.106+0000 7f04217fa700 1 -- 172.21.15.163:0/1427416426 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v122) v1 ==== 53+0+73 (secure 0 0 0) 0x7f04240556a0 con 0x7f042c15df50 2023-10-09T06:39:30.107 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-09T06:39:30.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.108+0000 7f0433861700 1 -- 172.21.15.163:0/1427416426 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0410067320 msgr2=0x7f04100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:30.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.108+0000 7f0433861700 1 --2- 172.21.15.163:0/1427416426 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0410067320 0x7f04100697d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f042c0b4600 tx=0x7f0428017000 comp rx=0 tx=0).stop 2023-10-09T06:39:30.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.108+0000 7f0433861700 1 -- 172.21.15.163:0/1427416426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f042c15df50 msgr2=0x7f042c0c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:30.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.108+0000 7f0433861700 1 --2- 172.21.15.163:0/1427416426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f042c15df50 0x7f042c0c48b0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f0424009fa0 tx=0x7f0424009900 comp rx=0 tx=0).stop 2023-10-09T06:39:30.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.108+0000 7f0433861700 1 -- 172.21.15.163:0/1427416426 shutdown_connections 2023-10-09T06:39:30.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.108+0000 7f0433861700 1 --2- 172.21.15.163:0/1427416426 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0410067320 0x7f04100697d0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:30.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.108+0000 7f0433861700 1 --2- 172.21.15.163:0/1427416426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f042c15df50 0x7f042c0c48b0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:30.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.108+0000 7f0433861700 1 --2- 172.21.15.163:0/1427416426 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f042c06bcd0 0x7f042c1410c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:30.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.108+0000 7f0433861700 1 --2- 172.21.15.163:0/1427416426 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f042c06c0a0 0x7f042c0c4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:30.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.108+0000 7f0433861700 1 -- 172.21.15.163:0/1427416426 >> 172.21.15.163:0/1427416426 conn(0x7f042c0ce600 msgr2=0x7f042c0cc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:30.110 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.109+0000 7f0433861700 1 -- 172.21.15.163:0/1427416426 shutdown_connections 2023-10-09T06:39:30.111 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.109+0000 7f0433861700 1 -- 172.21.15.163:0/1427416426 wait complete. 2023-10-09T06:39:30.121 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-09T06:39:30.122 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-09T06:39:30.122 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-09T06:39:30.122 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-09T06:39:30.122 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-09T06:39:30.122 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-09T06:39:30.122 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-09T06:39:30.122 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-09T06:39:30.122 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:39:30.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:39:30.123 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-09T06:39:30.140 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:39:30.141 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:39:30.141 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:39:30.141 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:39:30.141 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-09T06:39:30.141 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:39:30.141 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:39:30.141 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:39:30.141 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2023-10-09T06:39:30.141 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:39:30.141 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:39:30.141 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2023-10-09T06:39:30.142 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:39:30.142 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-09T06:39:30.142 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-09T06:39:30.142 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-09T06:39:30.142 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-09T06:39:30.142 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 41930sec preferred_lft 41930sec 2023-10-09T06:39:30.142 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2023-10-09T06:39:30.142 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:39:30.142 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-09T06:39:30.142 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 72:62:e6:72:7c:2e brd ff:ff:ff:ff:ff:ff 2023-10-09T06:39:30.142 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-09T06:39:30.142 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:39:30.143 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::e89a:4cff:fefb:7198/64 scope link 2023-10-09T06:39:30.143 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:39:30.143 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:39:30.143 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 72:62:e6:72:7c:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-09T06:39:30.143 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::7062:e6ff:fe72:7c2e/64 scope link 2023-10-09T06:39:30.143 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:39:30.143 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:39:30.143 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether fa:4e:a3:9d:5a:5f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-09T06:39:30.143 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::f84e:a3ff:fe9d:5a5f/64 scope link 2023-10-09T06:39:30.143 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:39:30.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:39:30.144 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-09T06:39:30.207 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-09T06:39:30.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:39:30.209 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-09T06:39:30.268 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-09T06:39:30.268 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-09T06:39:30.268 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-09T06:39:30.268 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-09T06:39:30.269 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-09T06:39:30.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:39:30.269 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-09T06:39:30.269 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-09T06:39:30.269 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-09T06:39:30.269 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-09T06:39:30.269 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-09T06:39:30.269 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-09T06:39:30.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:39:30.377 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-09T06:39:30.392 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-09T06:39:30.392 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-09T06:39:30.393 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:39:30.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:39:30.393 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-09T06:39:30.458 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:39:30.458 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-09T06:39:30.514 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:39:30.514 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:39:30.514 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-09T06:39:30.568 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-09T06:39:30.569 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-09T06:39:30.569 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:39:30.648 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:39:30.649 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:39:30.667 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:39:30.667 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:39:30.667 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:39:30.667 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 1@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:39:30.668 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:39:30.668 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:39:30.668 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:39:30.668 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:39:30.689 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:39:30.689 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:39:30.689 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:39:30.689 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 1@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:39:30.690 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:39:30.690 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:39:30.690 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:39:30.690 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:39:30.690 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790" 2023-10-09T06:39:30.690 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790:/ 2023-10-09T06:39:30.690 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=2520b8f7-539f-4bae-9f17-1fce66f66ddd" will pass to kernel 2023-10-09T06:39:30.691 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-09T06:39:30.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:39:30.691 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-09T06:39:30.756 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-09T06:39:30.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.878+0000 7f671ffff700 1 Processor -- start 2023-10-09T06:39:30.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.878+0000 7f671ffff700 1 -- start start 2023-10-09T06:39:30.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.879+0000 7f671ffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f67200b0790 0x7f67200cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:30.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.879+0000 7f671ffff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f67201591e0 0x7f6720159600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:30.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.879+0000 7f671ffff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67200cfa00 con 0x7f67200b03c0 2023-10-09T06:39:30.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.879+0000 7f671ffff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67200cb580 con 0x7f67200b0790 2023-10-09T06:39:30.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.879+0000 7f671ffff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67200cf160 con 0x7f67201591e0 2023-10-09T06:39:30.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.879+0000 7f671e7fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f67200b0790 0x7f67200cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:30.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.879+0000 7f671e7fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f67200b0790 0x7f67200cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41812/0 (socket says 172.21.15.163:41812) 2023-10-09T06:39:30.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.879+0000 7f671e7fc700 1 -- 172.21.15.163:0/1633638005 learned_addr learned my addr 172.21.15.163:0/1633638005 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:30.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.879+0000 7f671f7fe700 1 --2- 172.21.15.163:0/1633638005 >> v2:172.21.15.181:3300/0 conn(0x7f67201591e0 0x7f6720159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:30.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.879+0000 7f671e7fc700 1 -- 172.21.15.163:0/1633638005 >> v2:172.21.15.181:3300/0 conn(0x7f67201591e0 msgr2=0x7f6720159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:30.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.879+0000 7f671e7fc700 1 --2- 172.21.15.163:0/1633638005 >> v2:172.21.15.181:3300/0 conn(0x7f67201591e0 0x7f6720159600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:30.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.879+0000 7f671e7fc700 1 -- 172.21.15.163:0/1633638005 >> v1:172.21.15.181:6789/0 conn(0x7f67200b03c0 legacy=0x7f67200b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:30.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.879+0000 7f671e7fc700 1 -- 172.21.15.163:0/1633638005 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67200cf440 con 0x7f67200b0790 2023-10-09T06:39:30.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.880+0000 7f671e7fc700 1 --2- 172.21.15.163:0/1633638005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67200b0790 0x7f67200cea70 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f67140099b0 tx=0x7f6714009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50d5d92ccc25e76 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:30.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.880+0000 7f671dffb700 1 -- 172.21.15.163:0/1633638005 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671400b7e0 con 0x7f67200b0790 2023-10-09T06:39:30.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.880+0000 7f671dffb700 1 -- 172.21.15.163:0/1633638005 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671400b940 con 0x7f67200b0790 2023-10-09T06:39:30.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.880+0000 7f671dffb700 1 -- 172.21.15.163:0/1633638005 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671400bd00 con 0x7f67200b0790 2023-10-09T06:39:30.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.880+0000 7f671ffff700 1 -- 172.21.15.163:0/1633638005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67200b0790 msgr2=0x7f67200cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:30.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.880+0000 7f671ffff700 1 --2- 172.21.15.163:0/1633638005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67200b0790 0x7f67200cea70 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f67140099b0 tx=0x7f6714009250 comp rx=0 tx=0).stop 2023-10-09T06:39:30.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.880+0000 7f671ffff700 1 -- 172.21.15.163:0/1633638005 shutdown_connections 2023-10-09T06:39:30.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.880+0000 7f671ffff700 1 --2- 172.21.15.163:0/1633638005 >> v2:172.21.15.181:3300/0 conn(0x7f67201591e0 0x7f6720159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:30.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.880+0000 7f671ffff700 1 --2- 172.21.15.163:0/1633638005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67200b0790 0x7f67200cea70 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:30.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.880+0000 7f671ffff700 1 -- 172.21.15.163:0/1633638005 >> 172.21.15.163:0/1633638005 conn(0x7f6720063230 msgr2=0x7f6720063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:30.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.880+0000 7f671ffff700 1 -- 172.21.15.163:0/1633638005 shutdown_connections 2023-10-09T06:39:30.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.880+0000 7f671ffff700 1 -- 172.21.15.163:0/1633638005 wait complete. 2023-10-09T06:39:30.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.881+0000 7f671ffff700 1 Processor -- start 2023-10-09T06:39:30.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.882+0000 7f671ffff700 1 -- start start 2023-10-09T06:39:30.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.882+0000 7f671ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67200b03c0 0x7f672014da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:30.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.882+0000 7f671ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67200b0790 0x7f67200c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:30.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.882+0000 7f671ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67201591e0 0x7f67200cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:30.886 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.882+0000 7f671ffff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67200c8500 con 0x7f67200b0790 2023-10-09T06:39:30.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.882+0000 7f671ffff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67200c78f0 con 0x7f67200b03c0 2023-10-09T06:39:30.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.882+0000 7f671ffff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67200c7bd0 con 0x7f67201591e0 2023-10-09T06:39:30.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.882+0000 7f671e7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67200b0790 0x7f67200c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:30.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.882+0000 7f671e7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67200b0790 0x7f67200c5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41826/0 (socket says 172.21.15.163:41826) 2023-10-09T06:39:30.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.882+0000 7f671e7fc700 1 -- 172.21.15.163:0/3784581325 learned_addr learned my addr 172.21.15.163:0/3784581325 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:30.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.882+0000 7f671f7fe700 1 --2- 172.21.15.163:0/3784581325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67201591e0 0x7f67200cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:30.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.882+0000 7f671effd700 1 --2- 172.21.15.163:0/3784581325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67200b03c0 0x7f672014da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:30.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.883+0000 7f671e7fc700 1 -- 172.21.15.163:0/3784581325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67201591e0 msgr2=0x7f67200cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:30.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.883+0000 7f671e7fc700 1 --2- 172.21.15.163:0/3784581325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67201591e0 0x7f67200cc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:30.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.883+0000 7f671e7fc700 1 -- 172.21.15.163:0/3784581325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67200b03c0 msgr2=0x7f672014da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:30.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.883+0000 7f671e7fc700 1 --2- 172.21.15.163:0/3784581325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67200b03c0 0x7f672014da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:30.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.883+0000 7f671e7fc700 1 -- 172.21.15.163:0/3784581325 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67140094f0 con 0x7f67200b0790 2023-10-09T06:39:30.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.883+0000 7f671e7fc700 1 --2- 172.21.15.163:0/3784581325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67200b0790 0x7f67200c5db0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f67140030a0 tx=0x7f6714016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:30.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.883+0000 7f6706ffd700 1 -- 172.21.15.163:0/3784581325 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6714006a90 con 0x7f67200b0790 2023-10-09T06:39:30.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.883+0000 7f6706ffd700 1 -- 172.21.15.163:0/3784581325 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6714020070 con 0x7f67200b0790 2023-10-09T06:39:30.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.883+0000 7f6706ffd700 1 -- 172.21.15.163:0/3784581325 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6714011db0 con 0x7f67200b0790 2023-10-09T06:39:30.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.883+0000 7f671ffff700 1 -- 172.21.15.163:0/3784581325 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67200c7eb0 con 0x7f67200b0790 2023-10-09T06:39:30.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.883+0000 7f671ffff700 1 -- 172.21.15.163:0/3784581325 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67200c75a0 con 0x7f67200b0790 2023-10-09T06:39:30.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.884+0000 7f671ffff700 1 -- 172.21.15.163:0/3784581325 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f672004c3c0 con 0x7f67200b0790 2023-10-09T06:39:30.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.884+0000 7f6706ffd700 1 -- 172.21.15.163:0/3784581325 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6714022030 con 0x7f67200b0790 2023-10-09T06:39:30.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.885+0000 7f6706ffd700 1 --2- 172.21.15.163:0/3784581325 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f670c067320 0x7f670c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:30.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.885+0000 7f6706ffd700 1 -- 172.21.15.163:0/3784581325 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6714086860 con 0x7f67200b0790 2023-10-09T06:39:30.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.885+0000 7f671effd700 1 --2- 172.21.15.163:0/3784581325 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f670c067320 0x7f670c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:30.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.885+0000 7f671effd700 1 --2- 172.21.15.163:0/3784581325 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f670c067320 0x7f670c0697d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f67100075c0 tx=0x7f6710009780 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:30.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:30.889+0000 7f6706ffd700 1 -- 172.21.15.163:0/3784581325 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6714053600 con 0x7f67200b0790 2023-10-09T06:39:31.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:31.122+0000 7f671ffff700 1 -- 172.21.15.163:0/3784581325 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f672004c3c0 con 0x7f67200b0790 2023-10-09T06:39:31.960 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:31.959+0000 7f6706ffd700 1 -- 172.21.15.163:0/3784581325 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v123) v1 ==== 136+0+0 (secure 0 0 0) 0x7f6714023e60 con 0x7f67200b0790 2023-10-09T06:39:31.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:31.962+0000 7f671ffff700 1 -- 172.21.15.163:0/3784581325 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f670c067320 msgr2=0x7f670c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:31.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:31.962+0000 7f671ffff700 1 --2- 172.21.15.163:0/3784581325 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f670c067320 0x7f670c0697d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f67100075c0 tx=0x7f6710009780 comp rx=0 tx=0).stop 2023-10-09T06:39:31.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:31.962+0000 7f671ffff700 1 -- 172.21.15.163:0/3784581325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67200b0790 msgr2=0x7f67200c5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:31.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:31.962+0000 7f671ffff700 1 --2- 172.21.15.163:0/3784581325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67200b0790 0x7f67200c5db0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f67140030a0 tx=0x7f6714016000 comp rx=0 tx=0).stop 2023-10-09T06:39:31.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:31.962+0000 7f671ffff700 1 -- 172.21.15.163:0/3784581325 shutdown_connections 2023-10-09T06:39:31.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:31.962+0000 7f671ffff700 1 --2- 172.21.15.163:0/3784581325 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f670c067320 0x7f670c0697d0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:31.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:31.962+0000 7f671ffff700 1 --2- 172.21.15.163:0/3784581325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67201591e0 0x7f67200cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:31.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:31.962+0000 7f671ffff700 1 --2- 172.21.15.163:0/3784581325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67200b03c0 0x7f672014da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:31.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:31.962+0000 7f671ffff700 1 --2- 172.21.15.163:0/3784581325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67200b0790 0x7f67200c5db0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:31.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:31.962+0000 7f671ffff700 1 -- 172.21.15.163:0/3784581325 >> 172.21.15.163:0/3784581325 conn(0x7f6720063230 msgr2=0x7f6720068c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:31.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:31.962+0000 7f671ffff700 1 -- 172.21.15.163:0/3784581325 shutdown_connections 2023-10-09T06:39:31.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:31.962+0000 7f671ffff700 1 -- 172.21.15.163:0/3784581325 wait complete. 2023-10-09T06:39:31.976 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-09T06:39:32.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.093+0000 7f775afc9700 1 Processor -- start 2023-10-09T06:39:32.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.093+0000 7f775afc9700 1 -- start start 2023-10-09T06:39:32.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.094+0000 7f775afc9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7754155c10 0x7f7754155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:32.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.094+0000 7f775afc9700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77540acac0 con 0x7f77540ac230 2023-10-09T06:39:32.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.094+0000 7f775afc9700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77540ad930 con 0x7f77540ac6d0 2023-10-09T06:39:32.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.094+0000 7f775afc9700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77540acd20 con 0x7f7754155c10 2023-10-09T06:39:32.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.094+0000 7f7758d65700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7754155c10 0x7f7754155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:32.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.094+0000 7f7753fff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f77540ac230 0x7f77541568a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60996/0 (socket says 172.21.15.163:60996) 2023-10-09T06:39:32.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.094+0000 7f7753fff700 1 -- 172.21.15.163:0/1215855152 learned_addr learned my addr 172.21.15.163:0/1215855152 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:32.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.094+0000 7f7758d65700 1 -- 172.21.15.163:0/1215855152 >> v1:172.21.15.181:6789/0 conn(0x7f77540ac6d0 legacy=0x7f77540b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:32.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.094+0000 7f7758d65700 1 -- 172.21.15.163:0/1215855152 >> v1:172.21.15.163:6789/0 conn(0x7f77540ac230 legacy=0x7f77541568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:32.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.094+0000 7f7758d65700 1 -- 172.21.15.163:0/1215855152 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77540ad000 con 0x7f7754155c10 2023-10-09T06:39:32.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.094+0000 7f7758d65700 1 --2- 172.21.15.163:0/1215855152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7754155c10 0x7f7754155fe0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f77480098e0 tx=0x7f77480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b06fd70e3bb10c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:32.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.094+0000 7f77537fe700 1 -- 172.21.15.163:0/1215855152 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7748009e80 con 0x7f7754155c10 2023-10-09T06:39:32.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.095+0000 7f77537fe700 1 -- 172.21.15.163:0/1215855152 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f774800b820 con 0x7f7754155c10 2023-10-09T06:39:32.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.095+0000 7f77537fe700 1 -- 172.21.15.163:0/1215855152 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774800bba0 con 0x7f7754155c10 2023-10-09T06:39:32.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.095+0000 7f775afc9700 1 -- 172.21.15.163:0/1215855152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7754155c10 msgr2=0x7f7754155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:32.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.095+0000 7f775afc9700 1 --2- 172.21.15.163:0/1215855152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7754155c10 0x7f7754155fe0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f77480098e0 tx=0x7f77480091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:32.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.095+0000 7f775afc9700 1 -- 172.21.15.163:0/1215855152 shutdown_connections 2023-10-09T06:39:32.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.095+0000 7f775afc9700 1 --2- 172.21.15.163:0/1215855152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7754155c10 0x7f7754155fe0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:32.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.095+0000 7f775afc9700 1 -- 172.21.15.163:0/1215855152 >> 172.21.15.163:0/1215855152 conn(0x7f775406a320 msgr2=0x7f77540cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:32.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.095+0000 7f775afc9700 1 -- 172.21.15.163:0/1215855152 shutdown_connections 2023-10-09T06:39:32.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.095+0000 7f775afc9700 1 -- 172.21.15.163:0/1215855152 wait complete. 2023-10-09T06:39:32.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.096+0000 7f775afc9700 1 Processor -- start 2023-10-09T06:39:32.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.096+0000 7f775afc9700 1 -- start start 2023-10-09T06:39:32.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.097+0000 7f775afc9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77540ac230 0x7f775413c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:32.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.097+0000 7f775afc9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77540ac6d0 0x7f7754151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:32.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.097+0000 7f775afc9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7754155c10 0x7f77540a8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:32.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.097+0000 7f775afc9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775414bd90 con 0x7f7754155c10 2023-10-09T06:39:32.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.097+0000 7f775afc9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77540a5f00 con 0x7f77540ac6d0 2023-10-09T06:39:32.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.097+0000 7f775afc9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77540a61e0 con 0x7f77540ac230 2023-10-09T06:39:32.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.097+0000 7f7759566700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7754155c10 0x7f77540a8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:32.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.097+0000 7f7759566700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7754155c10 0x7f77540a8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41850/0 (socket says 172.21.15.163:41850) 2023-10-09T06:39:32.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.097+0000 7f7759566700 1 -- 172.21.15.163:0/2868649823 learned_addr learned my addr 172.21.15.163:0/2868649823 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:32.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.097+0000 7f7759566700 1 -- 172.21.15.163:0/2868649823 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77540ac230 msgr2=0x7f775413c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:32.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.097+0000 7f7759566700 1 --2- 172.21.15.163:0/2868649823 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77540ac230 0x7f775413c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:32.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.097+0000 7f7759566700 1 -- 172.21.15.163:0/2868649823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77540ac6d0 msgr2=0x7f7754151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:32.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.097+0000 7f7759566700 1 --2- 172.21.15.163:0/2868649823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77540ac6d0 0x7f7754151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:32.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.097+0000 7f7759566700 1 -- 172.21.15.163:0/2868649823 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7748009420 con 0x7f7754155c10 2023-10-09T06:39:32.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.098+0000 7f7759566700 1 --2- 172.21.15.163:0/2868649823 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7754155c10 0x7f77540a8580 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f774c00c930 tx=0x7f774c0026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:32.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.098+0000 7f7750ff9700 1 -- 172.21.15.163:0/2868649823 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c00ce70 con 0x7f7754155c10 2023-10-09T06:39:32.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.098+0000 7f7750ff9700 1 -- 172.21.15.163:0/2868649823 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f774c00a7e0 con 0x7f7754155c10 2023-10-09T06:39:32.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.098+0000 7f775afc9700 1 -- 172.21.15.163:0/2868649823 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77540a64c0 con 0x7f7754155c10 2023-10-09T06:39:32.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.098+0000 7f7750ff9700 1 -- 172.21.15.163:0/2868649823 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c00aba0 con 0x7f7754155c10 2023-10-09T06:39:32.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.098+0000 7f775afc9700 1 -- 172.21.15.163:0/2868649823 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77540a6a80 con 0x7f7754155c10 2023-10-09T06:39:32.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.099+0000 7f775afc9700 1 -- 172.21.15.163:0/2868649823 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f775404c3c0 con 0x7f7754155c10 2023-10-09T06:39:32.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.099+0000 7f7750ff9700 1 -- 172.21.15.163:0/2868649823 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f774c012420 con 0x7f7754155c10 2023-10-09T06:39:32.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.099+0000 7f7750ff9700 1 --2- 172.21.15.163:0/2868649823 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7738067320 0x7f77380697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:32.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.099+0000 7f7750ff9700 1 -- 172.21.15.163:0/2868649823 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f774c0850a0 con 0x7f7754155c10 2023-10-09T06:39:32.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.100+0000 7f7758d65700 1 --2- 172.21.15.163:0/2868649823 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7738067320 0x7f77380697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:32.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.100+0000 7f7758d65700 1 --2- 172.21.15.163:0/2868649823 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7738067320 0x7f77380697d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f77480098e0 tx=0x7f7748016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:32.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.103+0000 7f7750ff9700 1 -- 172.21.15.163:0/2868649823 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f774c055830 con 0x7f7754155c10 2023-10-09T06:39:32.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.313+0000 7f775afc9700 1 -- 172.21.15.163:0/2868649823 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f77540ad2e0 con 0x7f7738067320 2023-10-09T06:39:32.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.315+0000 7f7750ff9700 1 -- 172.21.15.163:0/2868649823 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f77540ad2e0 con 0x7f7738067320 2023-10-09T06:39:32.315 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:39:32.315 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:32.316 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "cephfs" 2023-10-09T06:39:32.316 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:32.316 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:39:32.317 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.316+0000 7f775afc9700 1 -- 172.21.15.163:0/2868649823 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7738067320 msgr2=0x7f77380697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:32.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.316+0000 7f775afc9700 1 --2- 172.21.15.163:0/2868649823 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7738067320 0x7f77380697d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f77480098e0 tx=0x7f7748016000 comp rx=0 tx=0).stop 2023-10-09T06:39:32.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.316+0000 7f775afc9700 1 -- 172.21.15.163:0/2868649823 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7754155c10 msgr2=0x7f77540a8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:32.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.316+0000 7f775afc9700 1 --2- 172.21.15.163:0/2868649823 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7754155c10 0x7f77540a8580 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f774c00c930 tx=0x7f774c0026e0 comp rx=0 tx=0).stop 2023-10-09T06:39:32.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.316+0000 7f775afc9700 1 -- 172.21.15.163:0/2868649823 shutdown_connections 2023-10-09T06:39:32.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.316+0000 7f775afc9700 1 --2- 172.21.15.163:0/2868649823 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7738067320 0x7f77380697d0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:32.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.316+0000 7f775afc9700 1 --2- 172.21.15.163:0/2868649823 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7754155c10 0x7f77540a8580 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:32.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.316+0000 7f775afc9700 1 --2- 172.21.15.163:0/2868649823 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77540ac230 0x7f775413c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:32.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.316+0000 7f775afc9700 1 --2- 172.21.15.163:0/2868649823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77540ac6d0 0x7f7754151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:32.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.316+0000 7f775afc9700 1 -- 172.21.15.163:0/2868649823 >> 172.21.15.163:0/2868649823 conn(0x7f775406a320 msgr2=0x7f77540af7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:32.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.316+0000 7f775afc9700 1 -- 172.21.15.163:0/2868649823 shutdown_connections 2023-10-09T06:39:32.319 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.316+0000 7f775afc9700 1 -- 172.21.15.163:0/2868649823 wait complete. 2023-10-09T06:39:32.330 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-09T06:39:32.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.446+0000 7f66fae7c700 1 Processor -- start 2023-10-09T06:39:32.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.446+0000 7f66fae7c700 1 -- start start 2023-10-09T06:39:32.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.447+0000 7f66fae7c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66f4159890 0x7f66f415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:32.449 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.447+0000 7f66fae7c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f66f4159c60 0x7f66f415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:32.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.447+0000 7f66fae7c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f66f415db10 0x7f66f4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:32.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.447+0000 7f66fae7c700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f415a160 con 0x7f66f4159c60 2023-10-09T06:39:32.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.447+0000 7f66fae7c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f40d13a0 con 0x7f66f415db10 2023-10-09T06:39:32.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.447+0000 7f66fae7c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f415e140 con 0x7f66f4159890 2023-10-09T06:39:32.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.447+0000 7f66f3fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f66f4159c60 0x7f66f415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:32.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.447+0000 7f66f3fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f66f4159c60 0x7f66f415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41866/0 (socket says 172.21.15.163:41866) 2023-10-09T06:39:32.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.447+0000 7f66f3fff700 1 -- 172.21.15.163:0/3733213180 learned_addr learned my addr 172.21.15.163:0/3733213180 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:32.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.447+0000 7f66f8c18700 1 --2- 172.21.15.163:0/3733213180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66f4159890 0x7f66f415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:32.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.448+0000 7f66f3fff700 1 -- 172.21.15.163:0/3733213180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66f4159890 msgr2=0x7f66f415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:32.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.448+0000 7f66f3fff700 1 --2- 172.21.15.163:0/3733213180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66f4159890 0x7f66f415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:32.450 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.448+0000 7f66f3fff700 1 -- 172.21.15.163:0/3733213180 >> v2:172.21.15.181:3300/0 conn(0x7f66f415db10 msgr2=0x7f66f4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:32.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.448+0000 7f66f3fff700 1 --2- 172.21.15.163:0/3733213180 >> v2:172.21.15.181:3300/0 conn(0x7f66f415db10 0x7f66f4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:32.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.448+0000 7f66f3fff700 1 -- 172.21.15.163:0/3733213180 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f415a3d0 con 0x7f66f4159c60 2023-10-09T06:39:32.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.448+0000 7f66f3fff700 1 --2- 172.21.15.163:0/3733213180 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f66f4159c60 0x7f66f415d5d0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f66f4089b50 tx=0x7f66e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10fd49b905f14d8e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:32.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.448+0000 7f66f37fe700 1 -- 172.21.15.163:0/3733213180 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e800b860 con 0x7f66f4159c60 2023-10-09T06:39:32.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.448+0000 7f66f37fe700 1 -- 172.21.15.163:0/3733213180 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e800b9c0 con 0x7f66f4159c60 2023-10-09T06:39:32.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.448+0000 7f66f37fe700 1 -- 172.21.15.163:0/3733213180 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e800bd60 con 0x7f66f4159c60 2023-10-09T06:39:32.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.448+0000 7f66fae7c700 1 -- 172.21.15.163:0/3733213180 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f66f4159c60 msgr2=0x7f66f415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:32.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.448+0000 7f66fae7c700 1 --2- 172.21.15.163:0/3733213180 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f66f4159c60 0x7f66f415d5d0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f66f4089b50 tx=0x7f66e8009250 comp rx=0 tx=0).stop 2023-10-09T06:39:32.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.448+0000 7f66fae7c700 1 -- 172.21.15.163:0/3733213180 shutdown_connections 2023-10-09T06:39:32.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.448+0000 7f66fae7c700 1 --2- 172.21.15.163:0/3733213180 >> v2:172.21.15.181:3300/0 conn(0x7f66f415db10 0x7f66f4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:32.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.448+0000 7f66fae7c700 1 --2- 172.21.15.163:0/3733213180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66f4159890 0x7f66f415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:32.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.448+0000 7f66fae7c700 1 --2- 172.21.15.163:0/3733213180 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f66f4159c60 0x7f66f415d5d0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:32.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.448+0000 7f66fae7c700 1 -- 172.21.15.163:0/3733213180 >> 172.21.15.163:0/3733213180 conn(0x7f66f4068e20 msgr2=0x7f66f40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:32.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.448+0000 7f66fae7c700 1 -- 172.21.15.163:0/3733213180 shutdown_connections 2023-10-09T06:39:32.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.448+0000 7f66fae7c700 1 -- 172.21.15.163:0/3733213180 wait complete. 2023-10-09T06:39:32.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.449+0000 7f66fae7c700 1 Processor -- start 2023-10-09T06:39:32.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.450+0000 7f66fae7c700 1 -- start start 2023-10-09T06:39:32.454 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.450+0000 7f66fae7c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66f4159890 0x7f66f40c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:32.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.450+0000 7f66fae7c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f66f4159c60 0x7f66f40c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:32.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.450+0000 7f66fae7c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66f415db10 0x7f66f4140e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:32.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.450+0000 7f66fae7c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f40cd7f0 con 0x7f66f4159c60 2023-10-09T06:39:32.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.450+0000 7f66fae7c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f41413a0 con 0x7f66f4159890 2023-10-09T06:39:32.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.450+0000 7f66fae7c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f40ca3f0 con 0x7f66f415db10 2023-10-09T06:39:32.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.450+0000 7f66f3fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f66f4159c60 0x7f66f40c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:32.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.450+0000 7f66f3fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f66f4159c60 0x7f66f40c78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41880/0 (socket says 172.21.15.163:41880) 2023-10-09T06:39:32.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.450+0000 7f66f3fff700 1 -- 172.21.15.163:0/2926085516 learned_addr learned my addr 172.21.15.163:0/2926085516 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:32.456 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.451+0000 7f66f9419700 1 --2- 172.21.15.163:0/2926085516 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66f415db10 0x7f66f4140e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:32.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.451+0000 7f66f8c18700 1 --2- 172.21.15.163:0/2926085516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66f4159890 0x7f66f40c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:32.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.451+0000 7f66f3fff700 1 -- 172.21.15.163:0/2926085516 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66f415db10 msgr2=0x7f66f4140e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:32.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.451+0000 7f66f3fff700 1 --2- 172.21.15.163:0/2926085516 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66f415db10 0x7f66f4140e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:32.457 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.451+0000 7f66f3fff700 1 -- 172.21.15.163:0/2926085516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66f4159890 msgr2=0x7f66f40c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:32.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.451+0000 7f66f3fff700 1 --2- 172.21.15.163:0/2926085516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66f4159890 0x7f66f40c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:32.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.451+0000 7f66f3fff700 1 -- 172.21.15.163:0/2926085516 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e80094f0 con 0x7f66f4159c60 2023-10-09T06:39:32.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.451+0000 7f66f9419700 1 --2- 172.21.15.163:0/2926085516 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66f415db10 0x7f66f4140e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:32.458 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.451+0000 7f66f3fff700 1 --2- 172.21.15.163:0/2926085516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f66f4159c60 0x7f66f40c78a0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009290 tx=0x7f66e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:32.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.451+0000 7f66f0ff9700 1 -- 172.21.15.163:0/2926085516 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e8026480 con 0x7f66f4159c60 2023-10-09T06:39:32.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.451+0000 7f66f0ff9700 1 -- 172.21.15.163:0/2926085516 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e801f070 con 0x7f66f4159c60 2023-10-09T06:39:32.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.451+0000 7f66fae7c700 1 -- 172.21.15.163:0/2926085516 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f40ca6d0 con 0x7f66f4159c60 2023-10-09T06:39:32.459 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.451+0000 7f66f0ff9700 1 -- 172.21.15.163:0/2926085516 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e8026830 con 0x7f66f4159c60 2023-10-09T06:39:32.460 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.451+0000 7f66fae7c700 1 -- 172.21.15.163:0/2926085516 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f40cd480 con 0x7f66f4159c60 2023-10-09T06:39:32.460 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.452+0000 7f66fae7c700 1 -- 172.21.15.163:0/2926085516 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f404c3c0 con 0x7f66f4159c60 2023-10-09T06:39:32.460 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.452+0000 7f66f0ff9700 1 -- 172.21.15.163:0/2926085516 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f66e8026ba0 con 0x7f66f4159c60 2023-10-09T06:39:32.460 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.453+0000 7f66f0ff9700 1 --2- 172.21.15.163:0/2926085516 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f66e0067320 0x7f66e00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:32.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.453+0000 7f66f0ff9700 1 -- 172.21.15.163:0/2926085516 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f66e8021070 con 0x7f66f4159c60 2023-10-09T06:39:32.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.453+0000 7f66f8c18700 1 --2- 172.21.15.163:0/2926085516 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f66e0067320 0x7f66e00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:32.461 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.453+0000 7f66f8c18700 1 --2- 172.21.15.163:0/2926085516 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f66e0067320 0x7f66e00697d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f66e400e4d0 tx=0x7f66e4007680 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:32.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.457+0000 7f66f0ff9700 1 -- 172.21.15.163:0/2926085516 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f66e8053850 con 0x7f66f4159c60 2023-10-09T06:39:32.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.639+0000 7f66fae7c700 1 -- 172.21.15.163:0/2926085516 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f66f406a5b0 con 0x7f66f4159c60 2023-10-09T06:39:32.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.639+0000 7f66f0ff9700 1 -- 172.21.15.163:0/2926085516 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7f66e80150c0 con 0x7f66f4159c60 2023-10-09T06:39:32.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.641+0000 7f66fae7c700 1 -- 172.21.15.163:0/2926085516 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f66e0067320 msgr2=0x7f66e00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:32.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.641+0000 7f66fae7c700 1 --2- 172.21.15.163:0/2926085516 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f66e0067320 0x7f66e00697d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f66e400e4d0 tx=0x7f66e4007680 comp rx=0 tx=0).stop 2023-10-09T06:39:32.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.641+0000 7f66fae7c700 1 -- 172.21.15.163:0/2926085516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f66f4159c60 msgr2=0x7f66f40c78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:32.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.641+0000 7f66fae7c700 1 --2- 172.21.15.163:0/2926085516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f66f4159c60 0x7f66f40c78a0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009290 tx=0x7f66e8017000 comp rx=0 tx=0).stop 2023-10-09T06:39:32.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.641+0000 7f66fae7c700 1 -- 172.21.15.163:0/2926085516 shutdown_connections 2023-10-09T06:39:32.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.641+0000 7f66fae7c700 1 --2- 172.21.15.163:0/2926085516 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f66e0067320 0x7f66e00697d0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:32.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.641+0000 7f66fae7c700 1 --2- 172.21.15.163:0/2926085516 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66f415db10 0x7f66f4140e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:32.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.641+0000 7f66fae7c700 1 --2- 172.21.15.163:0/2926085516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66f4159890 0x7f66f40c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:32.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.641+0000 7f66fae7c700 1 --2- 172.21.15.163:0/2926085516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f66f4159c60 0x7f66f40c78a0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:32.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.641+0000 7f66fae7c700 1 -- 172.21.15.163:0/2926085516 >> 172.21.15.163:0/2926085516 conn(0x7f66f4068e20 msgr2=0x7f66f40d0910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:32.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.641+0000 7f66fae7c700 1 -- 172.21.15.163:0/2926085516 shutdown_connections 2023-10-09T06:39:32.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:32.641+0000 7f66fae7c700 1 -- 172.21.15.163:0/2926085516 wait complete. 2023-10-09T06:39:32.654 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-09T06:39:32.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:39:32.655 DEBUG:teuthology.orchestra.run.smithi163:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-09T06:39:32.692 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi163.front.sepia.ceph.com 2023-10-09T06:39:32.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:39:32.693 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-09T06:39:32.747 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-09T06:39:32.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:39:32.748 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-09T06:39:32.790 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-09T06:39:32.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:39:32.791 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-09T06:39:32.847 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi163.front.sepia.ceph.com 2023-10-09T06:39:32.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:39:32.848 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-09T06:39:32.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-09T06:39:32.862 INFO:teuthology.orchestra.run.smithi163.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-09T06:39:32.863 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-09T06:39:32.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:39:32.863 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-09T06:39:32.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-09T06:39:32.904 INFO:teuthology.orchestra.run.smithi181.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-09T06:39:32.905 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rename cephfs volume_0000000000719180 --yes-i-really-mean-it 2023-10-09T06:39:33.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.024+0000 7f42fa445700 1 Processor -- start 2023-10-09T06:39:33.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.024+0000 7f42fa445700 1 -- start start 2023-10-09T06:39:33.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.025+0000 7f42fa445700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42f40b1950 0x7f42f40af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:33.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.025+0000 7f42fa445700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f42f40b1d20 0x7f42f4156890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:33.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.025+0000 7f42fa445700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42f40b06d0 con 0x7f42f40af610 2023-10-09T06:39:33.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.025+0000 7f42fa445700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42f40c9430 con 0x7f42f40b1d20 2023-10-09T06:39:33.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.025+0000 7f42fa445700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f40687a0 con 0x7f42f40b1950 2023-10-09T06:39:33.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.025+0000 7f42f89e2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f42f40b1d20 0x7f42f4156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:33.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.025+0000 7f42f89e2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f42f40b1d20 0x7f42f4156890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41892/0 (socket says 172.21.15.163:41892) 2023-10-09T06:39:33.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.025+0000 7f42f89e2700 1 -- 172.21.15.163:0/1488599917 learned_addr learned my addr 172.21.15.163:0/1488599917 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:33.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.025+0000 7f42f3fff700 1 --2- 172.21.15.163:0/1488599917 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42f40b1950 0x7f42f40af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:33.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.026+0000 7f42f89e2700 1 -- 172.21.15.163:0/1488599917 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42f40b1950 msgr2=0x7f42f40af0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:33.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.026+0000 7f42f89e2700 1 --2- 172.21.15.163:0/1488599917 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42f40b1950 0x7f42f40af0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:33.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.026+0000 7f42f89e2700 1 -- 172.21.15.163:0/1488599917 >> v1:172.21.15.181:6789/0 conn(0x7f42f40af610 legacy=0x7f42f4156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:33.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.026+0000 7f42f2ffd700 1 -- 172.21.15.163:0/1488599917 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3575373875 0 0) 0x7f42f40b06d0 con 0x7f42f40af610 2023-10-09T06:39:33.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.026+0000 7f42f89e2700 1 -- 172.21.15.163:0/1488599917 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f40b0ee0 con 0x7f42f40b1d20 2023-10-09T06:39:33.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.026+0000 7f42f89e2700 1 --2- 172.21.15.163:0/1488599917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42f40b1d20 0x7f42f4156890 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009f20 tx=0x7f42ec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8799a4c551bcc694 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:33.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.026+0000 7f42f2ffd700 1 -- 172.21.15.163:0/1488599917 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec00e070 con 0x7f42f40b1d20 2023-10-09T06:39:33.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.026+0000 7f42f2ffd700 1 -- 172.21.15.163:0/1488599917 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ec00b9d0 con 0x7f42f40b1d20 2023-10-09T06:39:33.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.026+0000 7f42f2ffd700 1 -- 172.21.15.163:0/1488599917 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec0092e0 con 0x7f42f40b1d20 2023-10-09T06:39:33.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.027+0000 7f42fa445700 1 -- 172.21.15.163:0/1488599917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42f40b1d20 msgr2=0x7f42f4156890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:33.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.027+0000 7f42fa445700 1 --2- 172.21.15.163:0/1488599917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42f40b1d20 0x7f42f4156890 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009f20 tx=0x7f42ec009840 comp rx=0 tx=0).stop 2023-10-09T06:39:33.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.027+0000 7f42fa445700 1 -- 172.21.15.163:0/1488599917 shutdown_connections 2023-10-09T06:39:33.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.027+0000 7f42fa445700 1 --2- 172.21.15.163:0/1488599917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42f40b1d20 0x7f42f4156890 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:33.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.027+0000 7f42fa445700 1 --2- 172.21.15.163:0/1488599917 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42f40b1950 0x7f42f40af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:33.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.027+0000 7f42fa445700 1 -- 172.21.15.163:0/1488599917 >> 172.21.15.163:0/1488599917 conn(0x7f42f40cd880 msgr2=0x7f42f40ae930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:33.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.027+0000 7f42fa445700 1 -- 172.21.15.163:0/1488599917 shutdown_connections 2023-10-09T06:39:33.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.027+0000 7f42fa445700 1 -- 172.21.15.163:0/1488599917 wait complete. 2023-10-09T06:39:33.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.028+0000 7f42fa445700 1 Processor -- start 2023-10-09T06:39:33.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.028+0000 7f42fa445700 1 -- start start 2023-10-09T06:39:33.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.029+0000 7f42fa445700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42f40af610 0x7f42f4154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:33.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.029+0000 7f42fa445700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42f40b1950 0x7f42f415adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:33.034 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.029+0000 7f42fa445700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42f40b1d20 0x7f42f415b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:33.034 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.029+0000 7f42fa445700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f415a360 con 0x7f42f40b1d20 2023-10-09T06:39:33.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.029+0000 7f42fa445700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f4155770 con 0x7f42f40af610 2023-10-09T06:39:33.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.029+0000 7f42fa445700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f4155a50 con 0x7f42f40b1950 2023-10-09T06:39:33.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.029+0000 7f42f89e2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42f40b1d20 0x7f42f415b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:33.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.029+0000 7f42f89e2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42f40b1d20 0x7f42f415b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41908/0 (socket says 172.21.15.163:41908) 2023-10-09T06:39:33.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.029+0000 7f42f89e2700 1 -- 172.21.15.163:0/3420138675 learned_addr learned my addr 172.21.15.163:0/3420138675 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:33.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.029+0000 7f42f3fff700 1 --2- 172.21.15.163:0/3420138675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42f40af610 0x7f42f4154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:33.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.029+0000 7f42f37fe700 1 --2- 172.21.15.163:0/3420138675 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42f40b1950 0x7f42f415adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:33.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.029+0000 7f42f89e2700 1 -- 172.21.15.163:0/3420138675 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42f40b1950 msgr2=0x7f42f415adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:33.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.029+0000 7f42f89e2700 1 --2- 172.21.15.163:0/3420138675 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42f40b1950 0x7f42f415adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:33.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.029+0000 7f42f89e2700 1 -- 172.21.15.163:0/3420138675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42f40af610 msgr2=0x7f42f4154df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:33.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.029+0000 7f42f89e2700 1 --2- 172.21.15.163:0/3420138675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42f40af610 0x7f42f4154df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:33.035 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.029+0000 7f42f89e2700 1 -- 172.21.15.163:0/3420138675 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ec009ae0 con 0x7f42f40b1d20 2023-10-09T06:39:33.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.029+0000 7f42f89e2700 1 --2- 172.21.15.163:0/3420138675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42f40b1d20 0x7f42f415b2f0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f42f4069fd0 tx=0x7f42ec018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:33.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.029+0000 7f42dbfff700 1 -- 172.21.15.163:0/3420138675 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec007b00 con 0x7f42f40b1d20 2023-10-09T06:39:33.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.029+0000 7f42dbfff700 1 -- 172.21.15.163:0/3420138675 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ec007c80 con 0x7f42f40b1d20 2023-10-09T06:39:33.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.029+0000 7f42dbfff700 1 -- 172.21.15.163:0/3420138675 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec01d420 con 0x7f42f40b1d20 2023-10-09T06:39:33.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.029+0000 7f42fa445700 1 -- 172.21.15.163:0/3420138675 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f415a5b0 con 0x7f42f40b1d20 2023-10-09T06:39:33.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.029+0000 7f42fa445700 1 -- 172.21.15.163:0/3420138675 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f414ef70 con 0x7f42f40b1d20 2023-10-09T06:39:33.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.030+0000 7f42fa445700 1 -- 172.21.15.163:0/3420138675 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f414f130 con 0x7f42f40b1d20 2023-10-09T06:39:33.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.031+0000 7f42dbfff700 1 -- 172.21.15.163:0/3420138675 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f42ec003040 con 0x7f42f40b1d20 2023-10-09T06:39:33.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.031+0000 7f42dbfff700 1 --2- 172.21.15.163:0/3420138675 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f42e0067320 0x7f42e00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:33.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.031+0000 7f42dbfff700 1 -- 172.21.15.163:0/3420138675 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f42ec086470 con 0x7f42f40b1d20 2023-10-09T06:39:33.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.031+0000 7f42f3fff700 1 --2- 172.21.15.163:0/3420138675 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f42e0067320 0x7f42e00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:33.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.032+0000 7f42f3fff700 1 --2- 172.21.15.163:0/3420138675 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f42e0067320 0x7f42e00697d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009c90 tx=0x7f42e4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:33.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.035+0000 7f42dbfff700 1 -- 172.21.15.163:0/3420138675 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f42ec052ff0 con 0x7f42f40b1d20 2023-10-09T06:39:33.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:33.241+0000 7f42fa445700 1 -- 172.21.15.163:0/3420138675 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume rename", "vol_name": "cephfs", "new_vol_name": "volume_0000000000719180", "yes_i_really_mean_it": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f42f40718b0 con 0x7f42e0067320 2023-10-09T06:39:35.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:35.986+0000 7f42dbfff700 1 -- 172.21.15.163:0/3420138675 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+56 (secure 0 0 0) 0x7f42f40718b0 con 0x7f42e0067320 2023-10-09T06:39:35.988 INFO:teuthology.orchestra.run.smithi163.stdout:FS volume 'cephfs' renamed to 'volume_0000000000719180' 2023-10-09T06:39:35.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:35.989+0000 7f42fa445700 1 -- 172.21.15.163:0/3420138675 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f42e0067320 msgr2=0x7f42e00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:35.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:35.989+0000 7f42fa445700 1 --2- 172.21.15.163:0/3420138675 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f42e0067320 0x7f42e00697d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009c90 tx=0x7f42e4008040 comp rx=0 tx=0).stop 2023-10-09T06:39:35.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:35.989+0000 7f42fa445700 1 -- 172.21.15.163:0/3420138675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42f40b1d20 msgr2=0x7f42f415b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:35.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:35.989+0000 7f42fa445700 1 --2- 172.21.15.163:0/3420138675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42f40b1d20 0x7f42f415b2f0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f42f4069fd0 tx=0x7f42ec018000 comp rx=0 tx=0).stop 2023-10-09T06:39:35.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:35.990+0000 7f42fa445700 1 -- 172.21.15.163:0/3420138675 shutdown_connections 2023-10-09T06:39:35.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:35.990+0000 7f42fa445700 1 --2- 172.21.15.163:0/3420138675 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f42e0067320 0x7f42e00697d0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:35.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:35.990+0000 7f42fa445700 1 --2- 172.21.15.163:0/3420138675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42f40b1d20 0x7f42f415b2f0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:35.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:35.990+0000 7f42fa445700 1 --2- 172.21.15.163:0/3420138675 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42f40af610 0x7f42f4154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:35.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:35.990+0000 7f42fa445700 1 --2- 172.21.15.163:0/3420138675 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42f40b1950 0x7f42f415adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:35.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:35.990+0000 7f42fa445700 1 -- 172.21.15.163:0/3420138675 >> 172.21.15.163:0/3420138675 conn(0x7f42f40cd880 msgr2=0x7f42f40aea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:35.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:35.990+0000 7f42fa445700 1 -- 172.21.15.163:0/3420138675 shutdown_connections 2023-10-09T06:39:35.992 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:35.990+0000 7f42fa445700 1 -- 172.21.15.163:0/3420138675 wait complete. 2023-10-09T06:39:36.005 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rename cephfs volume_0000000000719180 --yes-i-really-mean-it 2023-10-09T06:39:36.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.128+0000 7f72270c1700 1 Processor -- start 2023-10-09T06:39:36.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.128+0000 7f72270c1700 1 -- start start 2023-10-09T06:39:36.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.129+0000 7f72270c1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f72200cca20 0x7f72200cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:36.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.129+0000 7f72270c1700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72200cfa00 con 0x7f72200b0790 2023-10-09T06:39:36.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.129+0000 7f72270c1700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72200cb580 con 0x7f72200b03c0 2023-10-09T06:39:36.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.129+0000 7f72270c1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72200cf160 con 0x7f72200cca20 2023-10-09T06:39:36.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.129+0000 7f72258be700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f72200b0790 0x7f72201591e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:32770/0 (socket says 172.21.15.163:32770) 2023-10-09T06:39:36.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.129+0000 7f72258be700 1 -- 172.21.15.163:0/753457639 learned_addr learned my addr 172.21.15.163:0/753457639 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:36.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.129+0000 7f72268c0700 1 --2- 172.21.15.163:0/753457639 >> v2:172.21.15.181:3300/0 conn(0x7f72200cca20 0x7f72200cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:36.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.129+0000 7f72250bd700 1 -- 172.21.15.163:0/753457639 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2355576690 0 0) 0x7f72200cfa00 con 0x7f72200b0790 2023-10-09T06:39:36.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.129+0000 7f72250bd700 1 -- 172.21.15.163:0/753457639 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f720c003570 con 0x7f72200b0790 2023-10-09T06:39:36.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.129+0000 7f72268c0700 1 -- 172.21.15.163:0/753457639 >> v1:172.21.15.181:6789/0 conn(0x7f72200b03c0 legacy=0x7f72200b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:36.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.129+0000 7f72250bd700 1 -- 172.21.15.163:0/753457639 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 179305954 0 0) 0x7f72200cb580 con 0x7f72200b03c0 2023-10-09T06:39:36.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.129+0000 7f72268c0700 1 -- 172.21.15.163:0/753457639 >> v1:172.21.15.163:6789/0 conn(0x7f72200b0790 legacy=0x7f72201591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:36.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.129+0000 7f72268c0700 1 -- 172.21.15.163:0/753457639 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72200cf440 con 0x7f72200cca20 2023-10-09T06:39:36.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.130+0000 7f72268c0700 1 --2- 172.21.15.163:0/753457639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72200cca20 0x7f72200cea70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7218009e90 tx=0x7f7218009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e71e0b952ebac11 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:36.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.130+0000 7f72250bd700 1 -- 172.21.15.163:0/753457639 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721800e070 con 0x7f72200cca20 2023-10-09T06:39:36.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.130+0000 7f72250bd700 1 -- 172.21.15.163:0/753457639 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721800b920 con 0x7f72200cca20 2023-10-09T06:39:36.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.130+0000 7f72250bd700 1 -- 172.21.15.163:0/753457639 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7210001470 con 0x7f72200cca20 2023-10-09T06:39:36.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.130+0000 7f72270c1700 1 -- 172.21.15.163:0/753457639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72200cca20 msgr2=0x7f72200cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:36.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.130+0000 7f72270c1700 1 --2- 172.21.15.163:0/753457639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72200cca20 0x7f72200cea70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7218009e90 tx=0x7f7218009780 comp rx=0 tx=0).stop 2023-10-09T06:39:36.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.130+0000 7f72270c1700 1 -- 172.21.15.163:0/753457639 shutdown_connections 2023-10-09T06:39:36.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.130+0000 7f72270c1700 1 --2- 172.21.15.163:0/753457639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72200cca20 0x7f72200cea70 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:36.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.130+0000 7f72270c1700 1 -- 172.21.15.163:0/753457639 >> 172.21.15.163:0/753457639 conn(0x7f7220063230 msgr2=0x7f7220063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:36.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.130+0000 7f72270c1700 1 -- 172.21.15.163:0/753457639 shutdown_connections 2023-10-09T06:39:36.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.130+0000 7f72270c1700 1 -- 172.21.15.163:0/753457639 wait complete. 2023-10-09T06:39:36.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.131+0000 7f72270c1700 1 Processor -- start 2023-10-09T06:39:36.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.131+0000 7f72270c1700 1 -- start start 2023-10-09T06:39:36.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.132+0000 7f72270c1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72200b03c0 0x7f72201699a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:36.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.132+0000 7f72270c1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72200b0790 0x7f722015b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:36.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.132+0000 7f72270c1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72200cca20 0x7f722015bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:36.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.132+0000 7f72270c1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72200c40a0 con 0x7f72200b0790 2023-10-09T06:39:36.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.132+0000 7f72270c1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72200c5850 con 0x7f72200cca20 2023-10-09T06:39:36.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.132+0000 7f72270c1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72200ca520 con 0x7f72200b03c0 2023-10-09T06:39:36.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.132+0000 7f72258be700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72200b0790 0x7f722015b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:36.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.132+0000 7f72258be700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72200b0790 0x7f722015b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41910/0 (socket says 172.21.15.163:41910) 2023-10-09T06:39:36.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.132+0000 7f72258be700 1 -- 172.21.15.163:0/1166667960 learned_addr learned my addr 172.21.15.163:0/1166667960 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:36.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.133+0000 7f72268c0700 1 --2- 172.21.15.163:0/1166667960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72200cca20 0x7f722015bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:36.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.133+0000 7f72258be700 1 -- 172.21.15.163:0/1166667960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72200b03c0 msgr2=0x7f72201699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:36.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.133+0000 7f72260bf700 1 --2- 172.21.15.163:0/1166667960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72200b03c0 0x7f72201699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:36.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.133+0000 7f72258be700 1 --2- 172.21.15.163:0/1166667960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72200b03c0 0x7f72201699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:36.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.133+0000 7f72258be700 1 -- 172.21.15.163:0/1166667960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72200cca20 msgr2=0x7f722015bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:36.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.133+0000 7f72258be700 1 --2- 172.21.15.163:0/1166667960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72200cca20 0x7f722015bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:36.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.133+0000 7f72258be700 1 -- 172.21.15.163:0/1166667960 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7218009a20 con 0x7f72200b0790 2023-10-09T06:39:36.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.133+0000 7f72258be700 1 --2- 172.21.15.163:0/1166667960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72200b0790 0x7f722015b860 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f721000cb60 tx=0x7f7210001bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:36.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.133+0000 7f72167fc700 1 -- 172.21.15.163:0/1166667960 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7210007360 con 0x7f72200b0790 2023-10-09T06:39:36.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.133+0000 7f72167fc700 1 -- 172.21.15.163:0/1166667960 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72100074e0 con 0x7f72200b0790 2023-10-09T06:39:36.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.133+0000 7f72270c1700 1 -- 172.21.15.163:0/1166667960 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72200ca800 con 0x7f72200b0790 2023-10-09T06:39:36.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.133+0000 7f72167fc700 1 -- 172.21.15.163:0/1166667960 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7210004830 con 0x7f72200b0790 2023-10-09T06:39:36.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.133+0000 7f72270c1700 1 -- 172.21.15.163:0/1166667960 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72200ca160 con 0x7f72200b0790 2023-10-09T06:39:36.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.134+0000 7f72270c1700 1 -- 172.21.15.163:0/1166667960 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72200c4210 con 0x7f72200b0790 2023-10-09T06:39:36.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.134+0000 7f72167fc700 1 -- 172.21.15.163:0/1166667960 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7210004bc0 con 0x7f72200b0790 2023-10-09T06:39:36.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.135+0000 7f72167fc700 1 --2- 172.21.15.163:0/1166667960 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7204067320 0x7f72040697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:36.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.135+0000 7f72167fc700 1 -- 172.21.15.163:0/1166667960 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f72100846f0 con 0x7f72200b0790 2023-10-09T06:39:36.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.135+0000 7f72260bf700 1 --2- 172.21.15.163:0/1166667960 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7204067320 0x7f72040697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:36.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.135+0000 7f72260bf700 1 --2- 172.21.15.163:0/1166667960 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7204067320 0x7f72040697d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f721c001720 tx=0x7f721c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:36.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.139+0000 7f72167fc700 1 -- 172.21.15.163:0/1166667960 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7210051370 con 0x7f72200b0790 2023-10-09T06:39:36.347 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.346+0000 7f72270c1700 1 -- 172.21.15.163:0/1166667960 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume rename", "vol_name": "cephfs", "new_vol_name": "volume_0000000000719180", "yes_i_really_mean_it": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f722004c3c0 con 0x7f7204067320 2023-10-09T06:39:36.993 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.992+0000 7f72167fc700 1 -- 172.21.15.163:0/1166667960 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+56 (secure 0 0 0) 0x7f722004c3c0 con 0x7f7204067320 2023-10-09T06:39:36.993 INFO:teuthology.orchestra.run.smithi163.stdout:FS volume 'cephfs' renamed to 'volume_0000000000719180' 2023-10-09T06:39:36.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.995+0000 7f72270c1700 1 -- 172.21.15.163:0/1166667960 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7204067320 msgr2=0x7f72040697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:36.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.995+0000 7f72270c1700 1 --2- 172.21.15.163:0/1166667960 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7204067320 0x7f72040697d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f721c001720 tx=0x7f721c009040 comp rx=0 tx=0).stop 2023-10-09T06:39:36.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.995+0000 7f72270c1700 1 -- 172.21.15.163:0/1166667960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72200b0790 msgr2=0x7f722015b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:36.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.995+0000 7f72270c1700 1 --2- 172.21.15.163:0/1166667960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72200b0790 0x7f722015b860 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f721000cb60 tx=0x7f7210001bf0 comp rx=0 tx=0).stop 2023-10-09T06:39:36.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.995+0000 7f72270c1700 1 -- 172.21.15.163:0/1166667960 shutdown_connections 2023-10-09T06:39:36.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.995+0000 7f72270c1700 1 --2- 172.21.15.163:0/1166667960 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7204067320 0x7f72040697d0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:36.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.995+0000 7f72270c1700 1 --2- 172.21.15.163:0/1166667960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72200cca20 0x7f722015bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:36.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.995+0000 7f72270c1700 1 --2- 172.21.15.163:0/1166667960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72200b03c0 0x7f72201699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:36.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.995+0000 7f72270c1700 1 --2- 172.21.15.163:0/1166667960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72200b0790 0x7f722015b860 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:36.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.995+0000 7f72270c1700 1 -- 172.21.15.163:0/1166667960 >> 172.21.15.163:0/1166667960 conn(0x7f7220063230 msgr2=0x7f72200cba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:36.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.995+0000 7f72270c1700 1 -- 172.21.15.163:0/1166667960 shutdown_connections 2023-10-09T06:39:36.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:36.995+0000 7f72270c1700 1 -- 172.21.15.163:0/1166667960 wait complete. 2023-10-09T06:39:37.009 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-09T06:39:37.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.125+0000 7fa49f59e700 1 Processor -- start 2023-10-09T06:39:37.126 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.125+0000 7fa49f59e700 1 -- start start 2023-10-09T06:39:37.129 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.126+0000 7fa49f59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4a01597f0 0x7fa4a00b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:37.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.126+0000 7fa49f59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4a00b11d0 con 0x7fa4a0159420 2023-10-09T06:39:37.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.126+0000 7fa49f59e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4a00b14b0 con 0x7fa4a015d230 2023-10-09T06:39:37.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.126+0000 7fa49f59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a00b1790 con 0x7fa4a01597f0 2023-10-09T06:39:37.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.126+0000 7fa49e59c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fa4a0159420 0x7fa4a00b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:32776/0 (socket says 172.21.15.163:32776) 2023-10-09T06:39:37.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.126+0000 7fa49e59c700 1 -- 172.21.15.163:0/3296279457 learned_addr learned my addr 172.21.15.163:0/3296279457 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:37.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.126+0000 7fa49dd9b700 1 -- 172.21.15.163:0/3296279457 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3748105934 0 0) 0x7fa4a00b11d0 con 0x7fa4a0159420 2023-10-09T06:39:37.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.126+0000 7fa495d9b700 1 --2- 172.21.15.163:0/3296279457 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4a01597f0 0x7fa4a00b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:37.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.126+0000 7fa49dd9b700 1 -- 172.21.15.163:0/3296279457 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa48c003570 con 0x7fa4a0159420 2023-10-09T06:39:37.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.126+0000 7fa49dd9b700 1 -- 172.21.15.163:0/3296279457 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3688052223 0 0) 0x7fa48c003570 con 0x7fa4a0159420 2023-10-09T06:39:37.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.126+0000 7fa49dd9b700 1 -- 172.21.15.163:0/3296279457 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4a01597f0 msgr2=0x7fa4a00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:37.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.126+0000 7fa49dd9b700 1 --2- 172.21.15.163:0/3296279457 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4a01597f0 0x7fa4a00b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:37.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.126+0000 7fa49dd9b700 1 -- 172.21.15.163:0/3296279457 >> v1:172.21.15.181:6789/0 conn(0x7fa4a015d230 legacy=0x7fa4a00b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:37.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.126+0000 7fa49dd9b700 1 -- 172.21.15.163:0/3296279457 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a00b1a70 con 0x7fa4a0159420 2023-10-09T06:39:37.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.126+0000 7fa49dd9b700 1 -- 172.21.15.163:0/3296279457 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fa490001470 con 0x7fa4a0159420 2023-10-09T06:39:37.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.127+0000 7fa49dd9b700 1 -- 172.21.15.163:0/3296279457 >> v1:172.21.15.163:6789/0 conn(0x7fa4a0159420 legacy=0x7fa4a00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:37.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.127+0000 7fa49dd9b700 1 --2- 172.21.15.163:0/3296279457 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4a01597f0 0x7fa48c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:37.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.127+0000 7fa49dd9b700 1 --2- 172.21.15.163:0/3296279457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa48c004010 0x7fa48c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:37.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.127+0000 7fa49dd9b700 1 --2- 172.21.15.163:0/3296279457 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa48c004d50 0x7fa48c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:37.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.127+0000 7fa49dd9b700 1 -- 172.21.15.163:0/3296279457 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a00b1790 con 0x7fa48c004d50 2023-10-09T06:39:37.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.127+0000 7fa49dd9b700 1 -- 172.21.15.163:0/3296279457 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c00bbf0 con 0x7fa48c004010 2023-10-09T06:39:37.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.127+0000 7fa49dd9b700 1 -- 172.21.15.163:0/3296279457 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c00bdb0 con 0x7fa4a01597f0 2023-10-09T06:39:37.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.127+0000 7fa49e59c700 1 --2- 172.21.15.163:0/3296279457 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4a01597f0 0x7fa48c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:37.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.127+0000 7fa49ed9d700 1 --2- 172.21.15.163:0/3296279457 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa48c004d50 0x7fa48c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:37.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.127+0000 7fa495d9b700 1 --2- 172.21.15.163:0/3296279457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa48c004010 0x7fa48c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:37.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.127+0000 7fa49ed9d700 1 -- 172.21.15.163:0/3296279457 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4a01597f0 msgr2=0x7fa48c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:37.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.127+0000 7fa49ed9d700 1 --2- 172.21.15.163:0/3296279457 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4a01597f0 0x7fa48c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:37.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.127+0000 7fa49ed9d700 1 -- 172.21.15.163:0/3296279457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa48c004010 msgr2=0x7fa48c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:37.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.127+0000 7fa49ed9d700 1 --2- 172.21.15.163:0/3296279457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa48c004010 0x7fa48c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:37.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.127+0000 7fa49ed9d700 1 -- 172.21.15.163:0/3296279457 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa48c003570 con 0x7fa48c004d50 2023-10-09T06:39:37.132 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.127+0000 7fa49ed9d700 1 --2- 172.21.15.163:0/3296279457 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa48c004d50 0x7fa48c0051c0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fa498001f90 tx=0x7fa498001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d3577852dc5aa3 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:37.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.127+0000 7fa49dd9b700 1 -- 172.21.15.163:0/3296279457 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498001950 con 0x7fa48c004d50 2023-10-09T06:39:37.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.127+0000 7fa49dd9b700 1 -- 172.21.15.163:0/3296279457 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49800c8b0 con 0x7fa48c004d50 2023-10-09T06:39:37.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.127+0000 7fa49dd9b700 1 -- 172.21.15.163:0/3296279457 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49800cc70 con 0x7fa48c004d50 2023-10-09T06:39:37.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.128+0000 7fa49f59e700 1 -- 172.21.15.163:0/3296279457 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa48c004d50 msgr2=0x7fa48c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:37.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.128+0000 7fa49f59e700 1 --2- 172.21.15.163:0/3296279457 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa48c004d50 0x7fa48c0051c0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fa498001f90 tx=0x7fa498001640 comp rx=0 tx=0).stop 2023-10-09T06:39:37.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.128+0000 7fa49ed9d700 1 -- 172.21.15.163:0/3296279457 reap_dead start 2023-10-09T06:39:37.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.128+0000 7fa49f59e700 1 -- 172.21.15.163:0/3296279457 shutdown_connections 2023-10-09T06:39:37.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.128+0000 7fa49f59e700 1 -- 172.21.15.163:0/3296279457 >> 172.21.15.163:0/3296279457 conn(0x7fa4a00d0660 msgr2=0x7fa4a00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:37.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.128+0000 7fa49f59e700 1 -- 172.21.15.163:0/3296279457 shutdown_connections 2023-10-09T06:39:37.133 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.128+0000 7fa49f59e700 1 -- 172.21.15.163:0/3296279457 wait complete. 2023-10-09T06:39:37.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.129+0000 7fa49f59e700 1 Processor -- start 2023-10-09T06:39:37.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.129+0000 7fa49f59e700 1 -- start start 2023-10-09T06:39:37.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.130+0000 7fa49f59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4a0154ce0 0x7fa4a0169940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:37.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.130+0000 7fa49f59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4a01550b0 0x7fa4a015b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:37.136 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.130+0000 7fa49f59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4a015be90 0x7fa4a00c5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:37.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.130+0000 7fa49f59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a0169e80 con 0x7fa4a0154ce0 2023-10-09T06:39:37.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.130+0000 7fa49f59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a015e370 con 0x7fa4a015be90 2023-10-09T06:39:37.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.130+0000 7fa49f59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a015ebf0 con 0x7fa4a01550b0 2023-10-09T06:39:37.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.130+0000 7fa49e59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4a0154ce0 0x7fa4a0169940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:37.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.130+0000 7fa49e59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4a0154ce0 0x7fa4a0169940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41922/0 (socket says 172.21.15.163:41922) 2023-10-09T06:39:37.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.130+0000 7fa49e59c700 1 -- 172.21.15.163:0/2823950839 learned_addr learned my addr 172.21.15.163:0/2823950839 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:37.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.130+0000 7fa49ed9d700 1 --2- 172.21.15.163:0/2823950839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4a015be90 0x7fa4a00c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:37.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.130+0000 7fa495d9b700 1 --2- 172.21.15.163:0/2823950839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4a01550b0 0x7fa4a015b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:37.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.130+0000 7fa49e59c700 1 -- 172.21.15.163:0/2823950839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4a01550b0 msgr2=0x7fa4a015b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:37.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.130+0000 7fa49e59c700 1 --2- 172.21.15.163:0/2823950839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4a01550b0 0x7fa4a015b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:37.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.130+0000 7fa49e59c700 1 -- 172.21.15.163:0/2823950839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4a015be90 msgr2=0x7fa4a00c5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:37.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.130+0000 7fa49e59c700 1 --2- 172.21.15.163:0/2823950839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4a015be90 0x7fa4a00c5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:37.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.130+0000 7fa49e59c700 1 -- 172.21.15.163:0/2823950839 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a015c3d0 con 0x7fa4a0154ce0 2023-10-09T06:39:37.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.130+0000 7fa49e59c700 1 --2- 172.21.15.163:0/2823950839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4a0154ce0 0x7fa4a0169940 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fa490000f60 tx=0x7fa490001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:37.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.130+0000 7fa496ffd700 1 -- 172.21.15.163:0/2823950839 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa490018030 con 0x7fa4a0154ce0 2023-10-09T06:39:37.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.130+0000 7fa496ffd700 1 -- 172.21.15.163:0/2823950839 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49000f840 con 0x7fa4a0154ce0 2023-10-09T06:39:37.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.130+0000 7fa49f59e700 1 -- 172.21.15.163:0/2823950839 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a015e5a0 con 0x7fa4a0154ce0 2023-10-09T06:39:37.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.130+0000 7fa496ffd700 1 -- 172.21.15.163:0/2823950839 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49000fbe0 con 0x7fa4a0154ce0 2023-10-09T06:39:37.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.131+0000 7fa49f59e700 1 -- 172.21.15.163:0/2823950839 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a015aa00 con 0x7fa4a0154ce0 2023-10-09T06:39:37.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.131+0000 7fa49f59e700 1 -- 172.21.15.163:0/2823950839 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a004c3c0 con 0x7fa4a0154ce0 2023-10-09T06:39:37.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.132+0000 7fa496ffd700 1 -- 172.21.15.163:0/2823950839 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa49001b070 con 0x7fa4a0154ce0 2023-10-09T06:39:37.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.132+0000 7fa496ffd700 1 --2- 172.21.15.163:0/2823950839 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa4800673a0 0x7fa480069850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:37.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.132+0000 7fa496ffd700 1 -- 172.21.15.163:0/2823950839 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5783+0+0 (secure 0 0 0) 0x7fa490019070 con 0x7fa4a0154ce0 2023-10-09T06:39:37.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.133+0000 7fa495d9b700 1 --2- 172.21.15.163:0/2823950839 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa4800673a0 0x7fa480069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:37.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.133+0000 7fa495d9b700 1 --2- 172.21.15.163:0/2823950839 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa4800673a0 0x7fa480069850 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa4a01554c0 tx=0x7fa488008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:37.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.137+0000 7fa496ffd700 1 -- 172.21.15.163:0/2823950839 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa490051870 con 0x7fa4a0154ce0 2023-10-09T06:39:37.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.344+0000 7fa49f59e700 1 -- 172.21.15.163:0/2823950839 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4a00b1d50 con 0x7fa4800673a0 2023-10-09T06:39:37.347 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.347+0000 7fa496ffd700 1 -- 172.21.15.163:0/2823950839 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+58 (secure 0 0 0) 0x7fa4a00b1d50 con 0x7fa4800673a0 2023-10-09T06:39:37.347 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:39:37.347 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:37.347 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "volume_0000000000719180" 2023-10-09T06:39:37.347 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:37.348 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:39:37.349 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.348+0000 7fa49f59e700 1 -- 172.21.15.163:0/2823950839 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa4800673a0 msgr2=0x7fa480069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:37.349 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.348+0000 7fa49f59e700 1 --2- 172.21.15.163:0/2823950839 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa4800673a0 0x7fa480069850 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa4a01554c0 tx=0x7fa488008040 comp rx=0 tx=0).stop 2023-10-09T06:39:37.349 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.348+0000 7fa49f59e700 1 -- 172.21.15.163:0/2823950839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4a0154ce0 msgr2=0x7fa4a0169940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:37.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.348+0000 7fa49f59e700 1 --2- 172.21.15.163:0/2823950839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4a0154ce0 0x7fa4a0169940 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fa490000f60 tx=0x7fa490001e50 comp rx=0 tx=0).stop 2023-10-09T06:39:37.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.348+0000 7fa49f59e700 1 -- 172.21.15.163:0/2823950839 shutdown_connections 2023-10-09T06:39:37.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.348+0000 7fa49f59e700 1 --2- 172.21.15.163:0/2823950839 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fa4800673a0 0x7fa480069850 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:37.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.348+0000 7fa49f59e700 1 --2- 172.21.15.163:0/2823950839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4a0154ce0 0x7fa4a0169940 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:37.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.348+0000 7fa49f59e700 1 --2- 172.21.15.163:0/2823950839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4a015be90 0x7fa4a00c5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:37.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.348+0000 7fa49f59e700 1 --2- 172.21.15.163:0/2823950839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4a01550b0 0x7fa4a015b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:37.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.348+0000 7fa49f59e700 1 -- 172.21.15.163:0/2823950839 >> 172.21.15.163:0/2823950839 conn(0x7fa4a00d0660 msgr2=0x7fa4a0069640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:37.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.348+0000 7fa49f59e700 1 -- 172.21.15.163:0/2823950839 shutdown_connections 2023-10-09T06:39:37.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.348+0000 7fa49f59e700 1 -- 172.21.15.163:0/2823950839 wait complete. 2023-10-09T06:39:37.361 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:39:37.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.475+0000 7ff346111700 1 Processor -- start 2023-10-09T06:39:37.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.475+0000 7ff346111700 1 -- start start 2023-10-09T06:39:37.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.476+0000 7ff346111700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff340159c60 0x7ff34015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:37.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.476+0000 7ff346111700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff34015db10 0x7ff340069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:37.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.476+0000 7ff346111700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff34015a160 con 0x7ff340159890 2023-10-09T06:39:37.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.476+0000 7ff346111700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3400d13a0 con 0x7ff34015db10 2023-10-09T06:39:37.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.476+0000 7ff346111700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff34015e140 con 0x7ff340159c60 2023-10-09T06:39:37.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.476+0000 7ff33f7fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff340159890 0x7ff34015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:32778/0 (socket says 172.21.15.163:32778) 2023-10-09T06:39:37.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.476+0000 7ff33f7fe700 1 -- 172.21.15.163:0/1034849543 learned_addr learned my addr 172.21.15.163:0/1034849543 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:37.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.476+0000 7ff33effd700 1 --2- 172.21.15.163:0/1034849543 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff340159c60 0x7ff34015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:37.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.476+0000 7ff33ffff700 1 --2- 172.21.15.163:0/1034849543 >> v2:172.21.15.181:3300/0 conn(0x7ff34015db10 0x7ff340069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:37.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.477+0000 7ff33e7fc700 1 -- 172.21.15.163:0/1034849543 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3680210516 0 0) 0x7ff34015a160 con 0x7ff340159890 2023-10-09T06:39:37.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.477+0000 7ff33e7fc700 1 -- 172.21.15.163:0/1034849543 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff328003570 con 0x7ff340159890 2023-10-09T06:39:37.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.477+0000 7ff33effd700 1 -- 172.21.15.163:0/1034849543 >> v2:172.21.15.181:3300/0 conn(0x7ff34015db10 msgr2=0x7ff340069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:37.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.477+0000 7ff33effd700 1 --2- 172.21.15.163:0/1034849543 >> v2:172.21.15.181:3300/0 conn(0x7ff34015db10 0x7ff340069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:37.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.477+0000 7ff33effd700 1 -- 172.21.15.163:0/1034849543 >> v1:172.21.15.163:6789/0 conn(0x7ff340159890 legacy=0x7ff34015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:37.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.477+0000 7ff33effd700 1 -- 172.21.15.163:0/1034849543 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34015a3d0 con 0x7ff340159c60 2023-10-09T06:39:37.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.477+0000 7ff33effd700 1 --2- 172.21.15.163:0/1034849543 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff340159c60 0x7ff34015d5d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff3340098e0 tx=0x7ff3340091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b7919ad69279fac server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:37.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.477+0000 7ff33e7fc700 1 -- 172.21.15.163:0/1034849543 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff334011630 con 0x7ff340159c60 2023-10-09T06:39:37.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.477+0000 7ff33e7fc700 1 -- 172.21.15.163:0/1034849543 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff334011790 con 0x7ff340159c60 2023-10-09T06:39:37.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.477+0000 7ff33e7fc700 1 -- 172.21.15.163:0/1034849543 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff334011b30 con 0x7ff340159c60 2023-10-09T06:39:37.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.478+0000 7ff346111700 1 -- 172.21.15.163:0/1034849543 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff340159c60 msgr2=0x7ff34015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:37.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.478+0000 7ff346111700 1 --2- 172.21.15.163:0/1034849543 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff340159c60 0x7ff34015d5d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff3340098e0 tx=0x7ff3340091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:37.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.478+0000 7ff346111700 1 -- 172.21.15.163:0/1034849543 shutdown_connections 2023-10-09T06:39:37.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.478+0000 7ff346111700 1 --2- 172.21.15.163:0/1034849543 >> v2:172.21.15.181:3300/0 conn(0x7ff34015db10 0x7ff340069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:37.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.478+0000 7ff346111700 1 --2- 172.21.15.163:0/1034849543 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff340159c60 0x7ff34015d5d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:37.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.478+0000 7ff346111700 1 -- 172.21.15.163:0/1034849543 >> 172.21.15.163:0/1034849543 conn(0x7ff340068e20 msgr2=0x7ff3400d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:37.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.478+0000 7ff346111700 1 -- 172.21.15.163:0/1034849543 shutdown_connections 2023-10-09T06:39:37.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.478+0000 7ff346111700 1 -- 172.21.15.163:0/1034849543 wait complete. 2023-10-09T06:39:37.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.479+0000 7ff346111700 1 Processor -- start 2023-10-09T06:39:37.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.479+0000 7ff346111700 1 -- start start 2023-10-09T06:39:37.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.480+0000 7ff346111700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff340159890 0x7ff3400b5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:37.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.480+0000 7ff346111700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff340159c60 0x7ff340144250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:37.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.480+0000 7ff346111700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff34015db10 0x7ff3401587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:37.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.480+0000 7ff346111700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff340152960 con 0x7ff340159890 2023-10-09T06:39:37.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.480+0000 7ff346111700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff340144790 con 0x7ff34015db10 2023-10-09T06:39:37.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.480+0000 7ff346111700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3400b63e0 con 0x7ff340159c60 2023-10-09T06:39:37.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.480+0000 7ff33f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff340159890 0x7ff3400b5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:37.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.480+0000 7ff33f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff340159890 0x7ff3400b5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41930/0 (socket says 172.21.15.163:41930) 2023-10-09T06:39:37.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.480+0000 7ff33f7fe700 1 -- 172.21.15.163:0/1302003382 learned_addr learned my addr 172.21.15.163:0/1302003382 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:37.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.480+0000 7ff33effd700 1 --2- 172.21.15.163:0/1302003382 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff340159c60 0x7ff340144250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:37.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.480+0000 7ff33ffff700 1 --2- 172.21.15.163:0/1302003382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff34015db10 0x7ff3401587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:37.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.480+0000 7ff33f7fe700 1 -- 172.21.15.163:0/1302003382 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff340159c60 msgr2=0x7ff340144250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:37.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.480+0000 7ff33f7fe700 1 --2- 172.21.15.163:0/1302003382 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff340159c60 0x7ff340144250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:37.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.480+0000 7ff33f7fe700 1 -- 172.21.15.163:0/1302003382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff34015db10 msgr2=0x7ff3401587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:37.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.480+0000 7ff33f7fe700 1 --2- 172.21.15.163:0/1302003382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff34015db10 0x7ff3401587b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:37.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.480+0000 7ff33f7fe700 1 -- 172.21.15.163:0/1302003382 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff334009420 con 0x7ff340159890 2023-10-09T06:39:37.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.480+0000 7ff33ffff700 1 --2- 172.21.15.163:0/1302003382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff34015db10 0x7ff3401587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:37.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.480+0000 7ff33effd700 1 --2- 172.21.15.163:0/1302003382 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff340159c60 0x7ff340144250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:37.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.480+0000 7ff33f7fe700 1 --2- 172.21.15.163:0/1302003382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff340159890 0x7ff3400b5e00 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7ff33000c7c0 tx=0x7ff330007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:37.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.480+0000 7ff3277fe700 1 -- 172.21.15.163:0/1302003382 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff330005b30 con 0x7ff340159890 2023-10-09T06:39:37.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.480+0000 7ff3277fe700 1 -- 172.21.15.163:0/1302003382 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff330005c90 con 0x7ff340159890 2023-10-09T06:39:37.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.480+0000 7ff3277fe700 1 -- 172.21.15.163:0/1302003382 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff330007340 con 0x7ff340159890 2023-10-09T06:39:37.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.480+0000 7ff346111700 1 -- 172.21.15.163:0/1302003382 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3400b5ad0 con 0x7ff340159890 2023-10-09T06:39:37.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.480+0000 7ff346111700 1 -- 172.21.15.163:0/1302003382 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff34016ced0 con 0x7ff340159890 2023-10-09T06:39:37.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.481+0000 7ff346111700 1 -- 172.21.15.163:0/1302003382 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3400b5630 con 0x7ff340159890 2023-10-09T06:39:37.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.482+0000 7ff3277fe700 1 -- 172.21.15.163:0/1302003382 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff33001b400 con 0x7ff340159890 2023-10-09T06:39:37.486 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.482+0000 7ff3277fe700 1 --2- 172.21.15.163:0/1302003382 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff32c067320 0x7ff32c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:37.486 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.482+0000 7ff3277fe700 1 -- 172.21.15.163:0/1302003382 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5783+0+0 (secure 0 0 0) 0x7ff330084750 con 0x7ff340159890 2023-10-09T06:39:37.486 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.482+0000 7ff33effd700 1 --2- 172.21.15.163:0/1302003382 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff32c067320 0x7ff32c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:37.486 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.483+0000 7ff33effd700 1 --2- 172.21.15.163:0/1302003382 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff32c067320 0x7ff32c0697d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff34015a090 tx=0x7ff334017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:37.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.486+0000 7ff3277fe700 1 -- 172.21.15.163:0/1302003382 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff330051490 con 0x7ff340159890 2023-10-09T06:39:37.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.711+0000 7ff346111700 1 -- 172.21.15.163:0/1302003382 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff34004c3c0 con 0x7ff340159890 2023-10-09T06:39:37.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.711+0000 7ff3277fe700 1 -- 172.21.15.163:0/1302003382 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4630 (secure 0 0 0) 0x7ff330054ab0 con 0x7ff340159890 2023-10-09T06:39:37.713 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:37.713 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":125,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6815,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3960800872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3960800872},{"type":"v1","addr":"172.21.15.181:6836","nonce":3960800872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":8631,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/380361821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":380361821},{"type":"v1","addr":"172.21.15.163:6834","nonce":380361821}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":8634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2847943599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2847943599},{"type":"v1","addr":"172.21.15.163:6835","nonce":2847943599}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:39:22.273633+0000","modified":"2023-10-09T06:39:33.963575+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8649":{"gid":8649,"name":"d","rank":0,"incarnation":119,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/119178917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":119178917},{"type":"v1","addr":"172.21.15.181:6837","nonce":119178917}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"volume_0000000000719180","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-09T06:39:37.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.714+0000 7ff346111700 1 -- 172.21.15.163:0/1302003382 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff32c067320 msgr2=0x7ff32c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:37.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.714+0000 7ff346111700 1 --2- 172.21.15.163:0/1302003382 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff32c067320 0x7ff32c0697d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff34015a090 tx=0x7ff334017000 comp rx=0 tx=0).stop 2023-10-09T06:39:37.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.714+0000 7ff346111700 1 -- 172.21.15.163:0/1302003382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff340159890 msgr2=0x7ff3400b5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:37.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.714+0000 7ff346111700 1 --2- 172.21.15.163:0/1302003382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff340159890 0x7ff3400b5e00 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7ff33000c7c0 tx=0x7ff330007e00 comp rx=0 tx=0).stop 2023-10-09T06:39:37.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.714+0000 7ff346111700 1 -- 172.21.15.163:0/1302003382 shutdown_connections 2023-10-09T06:39:37.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.714+0000 7ff346111700 1 --2- 172.21.15.163:0/1302003382 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff32c067320 0x7ff32c0697d0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:37.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.714+0000 7ff346111700 1 --2- 172.21.15.163:0/1302003382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff340159890 0x7ff3400b5e00 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:37.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.714+0000 7ff346111700 1 --2- 172.21.15.163:0/1302003382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff34015db10 0x7ff3401587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:37.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.714+0000 7ff346111700 1 --2- 172.21.15.163:0/1302003382 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff340159c60 0x7ff340144250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:37.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.714+0000 7ff346111700 1 -- 172.21.15.163:0/1302003382 >> 172.21.15.163:0/1302003382 conn(0x7ff340068e20 msgr2=0x7ff3401530f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:37.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.714+0000 7ff346111700 1 -- 172.21.15.163:0/1302003382 shutdown_connections 2023-10-09T06:39:37.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.714+0000 7ff346111700 1 -- 172.21.15.163:0/1302003382 wait complete. 2023-10-09T06:39:37.716 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 125 2023-10-09T06:39:37.727 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:39:37.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.840+0000 7f16a352e700 1 Processor -- start 2023-10-09T06:39:37.841 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.841+0000 7f16a352e700 1 -- start start 2023-10-09T06:39:37.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.841+0000 7f16a352e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f169c159890 0x7f169c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:37.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.841+0000 7f16a352e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f169c069560 0x7f169c0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:37.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.841+0000 7f16a352e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f169c15a160 con 0x7f169c159c60 2023-10-09T06:39:37.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.841+0000 7f16a352e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f169c069f20 con 0x7f169c069560 2023-10-09T06:39:37.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.841+0000 7f16a352e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169c0d13a0 con 0x7f169c159890 2023-10-09T06:39:37.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.842+0000 7f16a1acb700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f169c069560 0x7f169c0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:37.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.842+0000 7f16a1acb700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f169c069560 0x7f169c0699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41944/0 (socket says 172.21.15.163:41944) 2023-10-09T06:39:37.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.842+0000 7f16a1acb700 1 -- 172.21.15.163:0/3695096505 learned_addr learned my addr 172.21.15.163:0/3695096505 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:37.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.842+0000 7f16a12ca700 1 --2- 172.21.15.163:0/3695096505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f169c159890 0x7f169c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:37.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.842+0000 7f16a1acb700 1 -- 172.21.15.163:0/3695096505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f169c159890 msgr2=0x7f169c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:37.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.842+0000 7f16a1acb700 1 --2- 172.21.15.163:0/3695096505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f169c159890 0x7f169c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:37.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.842+0000 7f16a1acb700 1 -- 172.21.15.163:0/3695096505 >> v1:172.21.15.181:6789/0 conn(0x7f169c159c60 legacy=0x7f169c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:37.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.842+0000 7f16a1acb700 1 -- 172.21.15.163:0/3695096505 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169c15e140 con 0x7f169c069560 2023-10-09T06:39:37.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.842+0000 7f16a1acb700 1 --2- 172.21.15.163:0/3695096505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f169c069560 0x7f169c0699e0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f1694009f20 tx=0x7f1694009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b5dab6157e1b98 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:37.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.842+0000 7f1693fff700 1 -- 172.21.15.163:0/3695096505 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169400e070 con 0x7f169c069560 2023-10-09T06:39:37.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.842+0000 7f1693fff700 1 -- 172.21.15.163:0/3695096505 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169400ba40 con 0x7f169c069560 2023-10-09T06:39:37.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.842+0000 7f1693fff700 1 -- 172.21.15.163:0/3695096505 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16940092e0 con 0x7f169c069560 2023-10-09T06:39:37.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.843+0000 7f16a352e700 1 -- 172.21.15.163:0/3695096505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f169c069560 msgr2=0x7f169c0699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:37.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.843+0000 7f16a352e700 1 --2- 172.21.15.163:0/3695096505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f169c069560 0x7f169c0699e0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f1694009f20 tx=0x7f1694009840 comp rx=0 tx=0).stop 2023-10-09T06:39:37.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.843+0000 7f16a352e700 1 -- 172.21.15.163:0/3695096505 shutdown_connections 2023-10-09T06:39:37.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.843+0000 7f16a352e700 1 --2- 172.21.15.163:0/3695096505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f169c069560 0x7f169c0699e0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:37.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.843+0000 7f16a352e700 1 --2- 172.21.15.163:0/3695096505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f169c159890 0x7f169c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:37.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.843+0000 7f16a352e700 1 -- 172.21.15.163:0/3695096505 >> 172.21.15.163:0/3695096505 conn(0x7f169c068e20 msgr2=0x7f169c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:37.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.843+0000 7f16a352e700 1 -- 172.21.15.163:0/3695096505 shutdown_connections 2023-10-09T06:39:37.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.843+0000 7f16a352e700 1 -- 172.21.15.163:0/3695096505 wait complete. 2023-10-09T06:39:37.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.844+0000 7f16a352e700 1 Processor -- start 2023-10-09T06:39:37.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.844+0000 7f16a352e700 1 -- start start 2023-10-09T06:39:37.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.845+0000 7f16a352e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f169c069560 0x7f169c153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:37.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.845+0000 7f16a352e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f169c159890 0x7f169c16d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:37.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.845+0000 7f16a352e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f169c159c60 0x7f169c16d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:37.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.845+0000 7f16a352e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169c154030 con 0x7f169c159c60 2023-10-09T06:39:37.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.845+0000 7f16a352e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169c154310 con 0x7f169c159890 2023-10-09T06:39:37.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.845+0000 7f16a352e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169c1545f0 con 0x7f169c069560 2023-10-09T06:39:37.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.845+0000 7f16a1acb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f169c159c60 0x7f169c16d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:37.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.845+0000 7f16a1acb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f169c159c60 0x7f169c16d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41952/0 (socket says 172.21.15.163:41952) 2023-10-09T06:39:37.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.845+0000 7f16a1acb700 1 -- 172.21.15.163:0/1960714756 learned_addr learned my addr 172.21.15.163:0/1960714756 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:37.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.845+0000 7f16a12ca700 1 --2- 172.21.15.163:0/1960714756 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f169c069560 0x7f169c153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:37.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.845+0000 7f16a0ac9700 1 --2- 172.21.15.163:0/1960714756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f169c159890 0x7f169c16d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:37.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.845+0000 7f16a1acb700 1 -- 172.21.15.163:0/1960714756 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f169c069560 msgr2=0x7f169c153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:37.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.845+0000 7f16a1acb700 1 --2- 172.21.15.163:0/1960714756 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f169c069560 0x7f169c153730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:37.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.845+0000 7f16a1acb700 1 -- 172.21.15.163:0/1960714756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f169c159890 msgr2=0x7f169c16d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:37.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.845+0000 7f16a1acb700 1 --2- 172.21.15.163:0/1960714756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f169c159890 0x7f169c16d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:37.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.845+0000 7f16a1acb700 1 -- 172.21.15.163:0/1960714756 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1694009ae0 con 0x7f169c159c60 2023-10-09T06:39:37.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.845+0000 7f16a1acb700 1 --2- 172.21.15.163:0/1960714756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f169c159c60 0x7f169c16d930 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f169c0899c0 tx=0x7f1694017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:37.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.845+0000 7f16917fa700 1 -- 172.21.15.163:0/1960714756 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16940092e0 con 0x7f169c159c60 2023-10-09T06:39:37.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.845+0000 7f16917fa700 1 -- 172.21.15.163:0/1960714756 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1694012d00 con 0x7f169c159c60 2023-10-09T06:39:37.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.846+0000 7f16a352e700 1 -- 172.21.15.163:0/1960714756 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f169c1548d0 con 0x7f169c159c60 2023-10-09T06:39:37.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.846+0000 7f16917fa700 1 -- 172.21.15.163:0/1960714756 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169401d670 con 0x7f169c159c60 2023-10-09T06:39:37.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.846+0000 7f16a352e700 1 -- 172.21.15.163:0/1960714756 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f169c153c70 con 0x7f169c159c60 2023-10-09T06:39:37.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.846+0000 7f16a352e700 1 -- 172.21.15.163:0/1960714756 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f169c04c3c0 con 0x7f169c159c60 2023-10-09T06:39:37.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.847+0000 7f16917fa700 1 -- 172.21.15.163:0/1960714756 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f169401d9e0 con 0x7f169c159c60 2023-10-09T06:39:37.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.847+0000 7f16917fa700 1 --2- 172.21.15.163:0/1960714756 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1680067320 0x7f16800697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:37.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.847+0000 7f16917fa700 1 -- 172.21.15.163:0/1960714756 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f1694086560 con 0x7f169c159c60 2023-10-09T06:39:37.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.847+0000 7f16a12ca700 1 --2- 172.21.15.163:0/1960714756 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1680067320 0x7f16800697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:37.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.848+0000 7f16a12ca700 1 --2- 172.21.15.163:0/1960714756 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1680067320 0x7f16800697d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1698006fd0 tx=0x7f1698008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:37.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:37.851+0000 7f16917fa700 1 -- 172.21.15.163:0/1960714756 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1694053090 con 0x7f169c159c60 2023-10-09T06:39:38.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.039+0000 7f16a352e700 1 -- 172.21.15.163:0/1960714756 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f169c069d00 con 0x7f169c159c60 2023-10-09T06:39:38.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.040+0000 7f16917fa700 1 -- 172.21.15.163:0/1960714756 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) v1 ==== 74+0+15490 (secure 0 0 0) 0x7f16940566b0 con 0x7f169c159c60 2023-10-09T06:39:38.042 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:38.042 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":185,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:39:35.978425+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":31,"pool_name":"cephfs.volume_0000000000719180.meta","create_time":"2023-10-09T06:39:20.612285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000719180"}}},{"pool":32,"pool_name":"cephfs.volume_0000000000719180.data","create_time":"2023-10-09T06:39:21.247510+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000719180"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:39:38.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.042+0000 7f16a352e700 1 -- 172.21.15.163:0/1960714756 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1680067320 msgr2=0x7f16800697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:38.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.042+0000 7f16a352e700 1 --2- 172.21.15.163:0/1960714756 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1680067320 0x7f16800697d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1698006fd0 tx=0x7f1698008040 comp rx=0 tx=0).stop 2023-10-09T06:39:38.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.042+0000 7f16a352e700 1 -- 172.21.15.163:0/1960714756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f169c159c60 msgr2=0x7f169c16d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:38.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.042+0000 7f16a352e700 1 --2- 172.21.15.163:0/1960714756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f169c159c60 0x7f169c16d930 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f169c0899c0 tx=0x7f1694017000 comp rx=0 tx=0).stop 2023-10-09T06:39:38.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.042+0000 7f16a352e700 1 -- 172.21.15.163:0/1960714756 shutdown_connections 2023-10-09T06:39:38.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.042+0000 7f16a352e700 1 --2- 172.21.15.163:0/1960714756 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1680067320 0x7f16800697d0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:38.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.042+0000 7f16a352e700 1 --2- 172.21.15.163:0/1960714756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f169c159c60 0x7f169c16d930 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:38.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.042+0000 7f16a352e700 1 --2- 172.21.15.163:0/1960714756 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f169c069560 0x7f169c153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:38.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.042+0000 7f16a352e700 1 --2- 172.21.15.163:0/1960714756 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f169c159890 0x7f169c16d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:38.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.042+0000 7f16a352e700 1 -- 172.21.15.163:0/1960714756 >> 172.21.15.163:0/1960714756 conn(0x7f169c068e20 msgr2=0x7f169c156e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:38.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.042+0000 7f16a352e700 1 -- 172.21.15.163:0/1960714756 shutdown_connections 2023-10-09T06:39:38.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.042+0000 7f16a352e700 1 -- 172.21.15.163:0/1960714756 wait complete. 2023-10-09T06:39:38.055 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-09T06:39:38.056 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:39:38.104 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:39:38.148 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-09T06:39:38.149 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:39:38.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.262+0000 7fe6edf7a700 1 Processor -- start 2023-10-09T06:39:38.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.262+0000 7fe6edf7a700 1 -- start start 2023-10-09T06:39:38.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.263+0000 7fe6edf7a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6e8159c60 0x7fe6e815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:38.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.263+0000 7fe6edf7a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe6e815db10 0x7fe6e8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:38.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.263+0000 7fe6edf7a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6e815a160 con 0x7fe6e8159890 2023-10-09T06:39:38.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.263+0000 7fe6edf7a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6e80d13a0 con 0x7fe6e815db10 2023-10-09T06:39:38.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.263+0000 7fe6edf7a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e815e140 con 0x7fe6e8159c60 2023-10-09T06:39:38.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.263+0000 7fe6e6ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6e8159c60 0x7fe6e815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:38.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.264+0000 7fe6e7fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe6e815db10 0x7fe6e8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:38.266 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.264+0000 7fe6e6ffd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6e8159c60 0x7fe6e815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:57278/0 (socket says 172.21.15.163:57278) 2023-10-09T06:39:38.267 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.264+0000 7fe6e77fe700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe6e8159890 0x7fe6e815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:58798/0 (socket says 172.21.15.163:58798) 2023-10-09T06:39:38.267 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.264+0000 7fe6e6ffd700 1 -- 172.21.15.163:0/1286564253 learned_addr learned my addr 172.21.15.163:0/1286564253 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:38.267 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.264+0000 7fe6e6ffd700 1 -- 172.21.15.163:0/1286564253 >> v2:172.21.15.181:3300/0 conn(0x7fe6e815db10 msgr2=0x7fe6e8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:38.267 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.264+0000 7fe6e6ffd700 1 --2- 172.21.15.163:0/1286564253 >> v2:172.21.15.181:3300/0 conn(0x7fe6e815db10 0x7fe6e8069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:38.267 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.264+0000 7fe6e6ffd700 1 -- 172.21.15.163:0/1286564253 >> v1:172.21.15.181:6789/0 conn(0x7fe6e8159890 legacy=0x7fe6e815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:38.267 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.264+0000 7fe6e6ffd700 1 -- 172.21.15.163:0/1286564253 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e815a3d0 con 0x7fe6e8159c60 2023-10-09T06:39:38.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.264+0000 7fe6e67fc700 1 -- 172.21.15.163:0/1286564253 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1591211478 0 0) 0x7fe6e815a160 con 0x7fe6e8159890 2023-10-09T06:39:38.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.264+0000 7fe6e7fff700 1 --2- 172.21.15.163:0/1286564253 >> v2:172.21.15.181:3300/0 conn(0x7fe6e815db10 0x7fe6e8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:39:38.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.264+0000 7fe6e6ffd700 1 --2- 172.21.15.163:0/1286564253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6e8159c60 0x7fe6e815d5d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc0099b0 tx=0x7fe6dc009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a617bb15279a8aa1 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:38.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.264+0000 7fe6e67fc700 1 -- 172.21.15.163:0/1286564253 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc009ea0 con 0x7fe6e8159c60 2023-10-09T06:39:38.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.264+0000 7fe6e67fc700 1 -- 172.21.15.163:0/1286564253 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6dc00b860 con 0x7fe6e8159c60 2023-10-09T06:39:38.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.264+0000 7fe6e67fc700 1 -- 172.21.15.163:0/1286564253 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc00bc00 con 0x7fe6e8159c60 2023-10-09T06:39:38.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.265+0000 7fe6edf7a700 1 -- 172.21.15.163:0/1286564253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6e8159c60 msgr2=0x7fe6e815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:38.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.265+0000 7fe6edf7a700 1 --2- 172.21.15.163:0/1286564253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6e8159c60 0x7fe6e815d5d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc0099b0 tx=0x7fe6dc009250 comp rx=0 tx=0).stop 2023-10-09T06:39:38.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.265+0000 7fe6edf7a700 1 -- 172.21.15.163:0/1286564253 shutdown_connections 2023-10-09T06:39:38.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.265+0000 7fe6edf7a700 1 --2- 172.21.15.163:0/1286564253 >> v2:172.21.15.181:3300/0 conn(0x7fe6e815db10 0x7fe6e8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:38.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.265+0000 7fe6edf7a700 1 --2- 172.21.15.163:0/1286564253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6e8159c60 0x7fe6e815d5d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:38.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.265+0000 7fe6edf7a700 1 -- 172.21.15.163:0/1286564253 >> 172.21.15.163:0/1286564253 conn(0x7fe6e8068e20 msgr2=0x7fe6e80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:38.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.265+0000 7fe6edf7a700 1 -- 172.21.15.163:0/1286564253 shutdown_connections 2023-10-09T06:39:38.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.265+0000 7fe6edf7a700 1 -- 172.21.15.163:0/1286564253 wait complete. 2023-10-09T06:39:38.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.266+0000 7fe6edf7a700 1 Processor -- start 2023-10-09T06:39:38.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.266+0000 7fe6edf7a700 1 -- start start 2023-10-09T06:39:38.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.267+0000 7fe6edf7a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6e8159890 0x7fe6e80b2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:38.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.267+0000 7fe6edf7a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6e8159c60 0x7fe6e80b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:38.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.267+0000 7fe6edf7a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6e815db10 0x7fe6e8152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:38.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.267+0000 7fe6edf7a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e814c260 con 0x7fe6e8159c60 2023-10-09T06:39:38.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.267+0000 7fe6edf7a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e80b16a0 con 0x7fe6e8159890 2023-10-09T06:39:38.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.267+0000 7fe6edf7a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e80b1980 con 0x7fe6e815db10 2023-10-09T06:39:38.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.267+0000 7fe6e6ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6e8159c60 0x7fe6e80b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:38.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.267+0000 7fe6e6ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6e8159c60 0x7fe6e80b0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:41956/0 (socket says 172.21.15.163:41956) 2023-10-09T06:39:38.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.267+0000 7fe6e6ffd700 1 -- 172.21.15.163:0/717694912 learned_addr learned my addr 172.21.15.163:0/717694912 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:38.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.267+0000 7fe6e77fe700 1 --2- 172.21.15.163:0/717694912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6e8159890 0x7fe6e80b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:38.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.267+0000 7fe6e7fff700 1 --2- 172.21.15.163:0/717694912 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6e815db10 0x7fe6e8152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:38.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.267+0000 7fe6e6ffd700 1 -- 172.21.15.163:0/717694912 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6e815db10 msgr2=0x7fe6e8152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:38.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.267+0000 7fe6e6ffd700 1 --2- 172.21.15.163:0/717694912 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6e815db10 0x7fe6e8152750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:38.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.267+0000 7fe6e6ffd700 1 -- 172.21.15.163:0/717694912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6e8159890 msgr2=0x7fe6e80b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:38.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.267+0000 7fe6e6ffd700 1 --2- 172.21.15.163:0/717694912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6e8159890 0x7fe6e80b2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:38.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.267+0000 7fe6e6ffd700 1 -- 172.21.15.163:0/717694912 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e0009420 con 0x7fe6e8159c60 2023-10-09T06:39:38.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.268+0000 7fe6e6ffd700 1 --2- 172.21.15.163:0/717694912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6e8159c60 0x7fe6e80b0bb0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8070e20 tx=0x7fe6dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:38.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.268+0000 7fe6cf7fe700 1 -- 172.21.15.163:0/717694912 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc00bbe0 con 0x7fe6e8159c60 2023-10-09T06:39:38.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.268+0000 7fe6cf7fe700 1 -- 172.21.15.163:0/717694912 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6dc011420 con 0x7fe6e8159c60 2023-10-09T06:39:38.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.268+0000 7fe6edf7a700 1 -- 172.21.15.163:0/717694912 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6dc0094f0 con 0x7fe6e8159c60 2023-10-09T06:39:38.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.268+0000 7fe6cf7fe700 1 -- 172.21.15.163:0/717694912 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc0117e0 con 0x7fe6e8159c60 2023-10-09T06:39:38.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.268+0000 7fe6edf7a700 1 -- 172.21.15.163:0/717694912 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e80b26c0 con 0x7fe6e8159c60 2023-10-09T06:39:38.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.269+0000 7fe6edf7a700 1 -- 172.21.15.163:0/717694912 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6e80b1e80 con 0x7fe6e8159c60 2023-10-09T06:39:38.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.269+0000 7fe6cf7fe700 1 -- 172.21.15.163:0/717694912 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe6dc011c10 con 0x7fe6e8159c60 2023-10-09T06:39:38.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.269+0000 7fe6cf7fe700 1 --2- 172.21.15.163:0/717694912 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe6d4067320 0x7fe6d40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:38.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.270+0000 7fe6cf7fe700 1 -- 172.21.15.163:0/717694912 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5783+0+0 (secure 0 0 0) 0x7fe6dc086610 con 0x7fe6e8159c60 2023-10-09T06:39:38.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.270+0000 7fe6e77fe700 1 --2- 172.21.15.163:0/717694912 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe6d4067320 0x7fe6d40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:38.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.270+0000 7fe6e77fe700 1 --2- 172.21.15.163:0/717694912 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe6d4067320 0x7fe6d40697d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8001560 tx=0x7fe6d8005d10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:38.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.274+0000 7fe6cf7fe700 1 -- 172.21.15.163:0/717694912 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe6dc053250 con 0x7fe6e8159c60 2023-10-09T06:39:38.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.500+0000 7fe6edf7a700 1 -- 172.21.15.163:0/717694912 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6e804c3c0 con 0x7fe6e8159c60 2023-10-09T06:39:38.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.500+0000 7fe6cf7fe700 1 -- 172.21.15.163:0/717694912 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4630 (secure 0 0 0) 0x7fe6dc056870 con 0x7fe6e8159c60 2023-10-09T06:39:38.501 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:38.502 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":125,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6815,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3960800872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3960800872},{"type":"v1","addr":"172.21.15.181:6836","nonce":3960800872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":8631,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/380361821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":380361821},{"type":"v1","addr":"172.21.15.163:6834","nonce":380361821}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":8634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2847943599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2847943599},{"type":"v1","addr":"172.21.15.163:6835","nonce":2847943599}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:39:22.273633+0000","modified":"2023-10-09T06:39:33.963575+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8649":{"gid":8649,"name":"d","rank":0,"incarnation":119,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/119178917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":119178917},{"type":"v1","addr":"172.21.15.181:6837","nonce":119178917}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"volume_0000000000719180","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-09T06:39:38.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.502+0000 7fe6edf7a700 1 -- 172.21.15.163:0/717694912 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe6d4067320 msgr2=0x7fe6d40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:38.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.502+0000 7fe6edf7a700 1 --2- 172.21.15.163:0/717694912 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe6d4067320 0x7fe6d40697d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8001560 tx=0x7fe6d8005d10 comp rx=0 tx=0).stop 2023-10-09T06:39:38.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.502+0000 7fe6edf7a700 1 -- 172.21.15.163:0/717694912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6e8159c60 msgr2=0x7fe6e80b0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:38.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.502+0000 7fe6edf7a700 1 --2- 172.21.15.163:0/717694912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6e8159c60 0x7fe6e80b0bb0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8070e20 tx=0x7fe6dc017000 comp rx=0 tx=0).stop 2023-10-09T06:39:38.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.502+0000 7fe6edf7a700 1 -- 172.21.15.163:0/717694912 shutdown_connections 2023-10-09T06:39:38.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.502+0000 7fe6edf7a700 1 --2- 172.21.15.163:0/717694912 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe6d4067320 0x7fe6d40697d0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:38.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.502+0000 7fe6edf7a700 1 --2- 172.21.15.163:0/717694912 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6e815db10 0x7fe6e8152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:38.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.502+0000 7fe6edf7a700 1 --2- 172.21.15.163:0/717694912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6e8159890 0x7fe6e80b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:38.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.502+0000 7fe6edf7a700 1 --2- 172.21.15.163:0/717694912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6e8159c60 0x7fe6e80b0bb0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:38.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.502+0000 7fe6edf7a700 1 -- 172.21.15.163:0/717694912 >> 172.21.15.163:0/717694912 conn(0x7fe6e8068e20 msgr2=0x7fe6e8156e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:38.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.503+0000 7fe6edf7a700 1 -- 172.21.15.163:0/717694912 shutdown_connections 2023-10-09T06:39:38.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.503+0000 7fe6edf7a700 1 -- 172.21.15.163:0/717694912 wait complete. 2023-10-09T06:39:38.505 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 125 2023-10-09T06:39:38.516 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:39:38.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.630+0000 7f661043d700 1 Processor -- start 2023-10-09T06:39:38.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.630+0000 7f661043d700 1 -- start start 2023-10-09T06:39:38.678 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.631+0000 7f661043d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6608141500 0x7f66080c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:38.678 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.631+0000 7f661043d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f660806c0a0 0x7f66080c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:38.678 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.631+0000 7f661043d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f660815c2a0 con 0x7f660806bcd0 2023-10-09T06:39:38.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.631+0000 7f661043d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66080c9880 con 0x7f6608141500 2023-10-09T06:39:38.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.631+0000 7f661043d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f660815b690 con 0x7f660806c0a0 2023-10-09T06:39:38.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.631+0000 7f660d9d8700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6608141500 0x7f66080c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:38.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.631+0000 7f660d9d8700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6608141500 0x7f66080c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39326/0 (socket says 172.21.15.163:39326) 2023-10-09T06:39:38.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.632+0000 7f660d9d8700 1 -- 172.21.15.163:0/3929692358 learned_addr learned my addr 172.21.15.163:0/3929692358 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:38.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.632+0000 7f660e9da700 1 --2- 172.21.15.163:0/3929692358 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f660806c0a0 0x7f66080c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:38.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.632+0000 7f660d9d8700 1 -- 172.21.15.163:0/3929692358 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f660806c0a0 msgr2=0x7f66080c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:38.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.632+0000 7f660d9d8700 1 --2- 172.21.15.163:0/3929692358 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f660806c0a0 0x7f66080c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:38.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.632+0000 7f660d9d8700 1 -- 172.21.15.163:0/3929692358 >> v1:172.21.15.181:6789/0 conn(0x7f660806bcd0 legacy=0x7f6608140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:38.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.632+0000 7f660d9d8700 1 -- 172.21.15.163:0/3929692358 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f660815b970 con 0x7f6608141500 2023-10-09T06:39:38.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.632+0000 7f660e9da700 1 --2- 172.21.15.163:0/3929692358 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f660806c0a0 0x7f66080c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:38.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.632+0000 7f660d9d8700 1 --2- 172.21.15.163:0/3929692358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6608141500 0x7f66080c8e00 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f65f80099b0 tx=0x7f65f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c7f4d38a1762c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:38.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.632+0000 7f660d1d7700 1 -- 172.21.15.163:0/3929692358 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f800b820 con 0x7f6608141500 2023-10-09T06:39:38.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.632+0000 7f660d1d7700 1 -- 172.21.15.163:0/3929692358 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f800b980 con 0x7f6608141500 2023-10-09T06:39:38.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.632+0000 7f660d1d7700 1 -- 172.21.15.163:0/3929692358 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f800bd20 con 0x7f6608141500 2023-10-09T06:39:38.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.633+0000 7f661043d700 1 -- 172.21.15.163:0/3929692358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6608141500 msgr2=0x7f66080c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:38.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.633+0000 7f661043d700 1 --2- 172.21.15.163:0/3929692358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6608141500 0x7f66080c8e00 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f65f80099b0 tx=0x7f65f8009250 comp rx=0 tx=0).stop 2023-10-09T06:39:38.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.633+0000 7f661043d700 1 -- 172.21.15.163:0/3929692358 shutdown_connections 2023-10-09T06:39:38.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.633+0000 7f661043d700 1 --2- 172.21.15.163:0/3929692358 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f660806c0a0 0x7f66080c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:38.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.633+0000 7f661043d700 1 --2- 172.21.15.163:0/3929692358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6608141500 0x7f66080c8e00 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:38.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.633+0000 7f661043d700 1 -- 172.21.15.163:0/3929692358 >> 172.21.15.163:0/3929692358 conn(0x7f66080ce600 msgr2=0x7f66080cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:38.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.633+0000 7f661043d700 1 -- 172.21.15.163:0/3929692358 shutdown_connections 2023-10-09T06:39:38.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.633+0000 7f661043d700 1 -- 172.21.15.163:0/3929692358 wait complete. 2023-10-09T06:39:38.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.634+0000 7f661043d700 1 Processor -- start 2023-10-09T06:39:38.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.634+0000 7f661043d700 1 -- start start 2023-10-09T06:39:38.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.635+0000 7f661043d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f660806bcd0 0x7f660813c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:38.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.635+0000 7f661043d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f660806c0a0 0x7f660813fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:38.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.635+0000 7f661043d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6608141500 0x7f6608156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:38.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.635+0000 7f661043d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66080b13a0 con 0x7f660806c0a0 2023-10-09T06:39:38.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.635+0000 7f661043d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660815ada0 con 0x7f6608141500 2023-10-09T06:39:38.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.635+0000 7f661043d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6608156850 con 0x7f660806bcd0 2023-10-09T06:39:38.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.635+0000 7f660d9d8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f660806c0a0 0x7f660813fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:38.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.635+0000 7f660d9d8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f660806c0a0 0x7f660813fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39334/0 (socket says 172.21.15.163:39334) 2023-10-09T06:39:38.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.635+0000 7f660d9d8700 1 -- 172.21.15.163:0/2205753239 learned_addr learned my addr 172.21.15.163:0/2205753239 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:38.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.635+0000 7f660e1d9700 1 --2- 172.21.15.163:0/2205753239 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f660806bcd0 0x7f660813c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:38.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.635+0000 7f660e9da700 1 --2- 172.21.15.163:0/2205753239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6608141500 0x7f6608156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:38.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.635+0000 7f660d9d8700 1 -- 172.21.15.163:0/2205753239 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f660806bcd0 msgr2=0x7f660813c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:38.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.635+0000 7f660d9d8700 1 --2- 172.21.15.163:0/2205753239 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f660806bcd0 0x7f660813c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:38.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.635+0000 7f660d9d8700 1 -- 172.21.15.163:0/2205753239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6608141500 msgr2=0x7f6608156d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:38.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.635+0000 7f660d9d8700 1 --2- 172.21.15.163:0/2205753239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6608141500 0x7f6608156d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:38.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.635+0000 7f660d9d8700 1 -- 172.21.15.163:0/2205753239 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f80094f0 con 0x7f660806c0a0 2023-10-09T06:39:38.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.635+0000 7f660e9da700 1 --2- 172.21.15.163:0/2205753239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6608141500 0x7f6608156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:38.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.635+0000 7f660d9d8700 1 --2- 172.21.15.163:0/2205753239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f660806c0a0 0x7f660813fce0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f65f8009290 tx=0x7f65f8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:38.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.635+0000 7f65fe7fc700 1 -- 172.21.15.163:0/2205753239 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f801c460 con 0x7f660806c0a0 2023-10-09T06:39:38.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.635+0000 7f65fe7fc700 1 -- 172.21.15.163:0/2205753239 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f801c5e0 con 0x7f660806c0a0 2023-10-09T06:39:38.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.635+0000 7f661043d700 1 -- 172.21.15.163:0/2205753239 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66080b15f0 con 0x7f660806c0a0 2023-10-09T06:39:38.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.635+0000 7f65fe7fc700 1 -- 172.21.15.163:0/2205753239 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f801c9a0 con 0x7f660806c0a0 2023-10-09T06:39:38.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.635+0000 7f661043d700 1 -- 172.21.15.163:0/2205753239 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66080b1bb0 con 0x7f660806c0a0 2023-10-09T06:39:38.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.636+0000 7f661043d700 1 -- 172.21.15.163:0/2205753239 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6608156a70 con 0x7f660806c0a0 2023-10-09T06:39:38.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.637+0000 7f65fe7fc700 1 -- 172.21.15.163:0/2205753239 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f65f801a450 con 0x7f660806c0a0 2023-10-09T06:39:38.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.637+0000 7f65fe7fc700 1 --2- 172.21.15.163:0/2205753239 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f65ec067320 0x7f65ec0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:38.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.637+0000 7f65fe7fc700 1 -- 172.21.15.163:0/2205753239 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f65f8087220 con 0x7f660806c0a0 2023-10-09T06:39:38.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.637+0000 7f660e1d9700 1 --2- 172.21.15.163:0/2205753239 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f65ec067320 0x7f65ec0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:38.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.638+0000 7f660e1d9700 1 --2- 172.21.15.163:0/2205753239 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f65ec067320 0x7f65ec0697d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6604003a30 tx=0x7f6604007ea0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:38.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.641+0000 7f65fe7fc700 1 -- 172.21.15.163:0/2205753239 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6608156a70 con 0x7f660806c0a0 2023-10-09T06:39:38.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.824+0000 7f661043d700 1 -- 172.21.15.163:0/2205753239 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6608156a70 con 0x7f660806c0a0 2023-10-09T06:39:38.825 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.825+0000 7f65fe7fc700 1 -- 172.21.15.163:0/2205753239 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) v1 ==== 74+0+15490 (secure 0 0 0) 0x7f6608156a70 con 0x7f660806c0a0 2023-10-09T06:39:38.826 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:38.826 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":185,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:39:35.978425+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":31,"pool_name":"cephfs.volume_0000000000719180.meta","create_time":"2023-10-09T06:39:20.612285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000719180"}}},{"pool":32,"pool_name":"cephfs.volume_0000000000719180.data","create_time":"2023-10-09T06:39:21.247510+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000719180"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:39:38.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.827+0000 7f661043d700 1 -- 172.21.15.163:0/2205753239 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f65ec067320 msgr2=0x7f65ec0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:38.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.827+0000 7f661043d700 1 --2- 172.21.15.163:0/2205753239 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f65ec067320 0x7f65ec0697d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6604003a30 tx=0x7f6604007ea0 comp rx=0 tx=0).stop 2023-10-09T06:39:38.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.827+0000 7f661043d700 1 -- 172.21.15.163:0/2205753239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f660806c0a0 msgr2=0x7f660813fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:38.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.827+0000 7f661043d700 1 --2- 172.21.15.163:0/2205753239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f660806c0a0 0x7f660813fce0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f65f8009290 tx=0x7f65f8016000 comp rx=0 tx=0).stop 2023-10-09T06:39:38.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.827+0000 7f661043d700 1 -- 172.21.15.163:0/2205753239 shutdown_connections 2023-10-09T06:39:38.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.827+0000 7f661043d700 1 --2- 172.21.15.163:0/2205753239 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f65ec067320 0x7f65ec0697d0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:38.828 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.827+0000 7f661043d700 1 --2- 172.21.15.163:0/2205753239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6608141500 0x7f6608156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:38.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.827+0000 7f661043d700 1 --2- 172.21.15.163:0/2205753239 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f660806bcd0 0x7f660813c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:38.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.827+0000 7f661043d700 1 --2- 172.21.15.163:0/2205753239 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f660806c0a0 0x7f660813fce0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:38.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.827+0000 7f661043d700 1 -- 172.21.15.163:0/2205753239 >> 172.21.15.163:0/2205753239 conn(0x7f66080ce600 msgr2=0x7f660806cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:38.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.827+0000 7f661043d700 1 -- 172.21.15.163:0/2205753239 shutdown_connections 2023-10-09T06:39:38.829 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.827+0000 7f661043d700 1 -- 172.21.15.163:0/2205753239 wait complete. 2023-10-09T06:39:38.841 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 31, 'pool_name': 'cephfs.volume_0000000000719180.meta', 'create_time': '2023-10-09T06:39:20.612285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '183', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'volume_0000000000719180'}}}, {'pool': 32, 'pool_name': 'cephfs.volume_0000000000719180.data', 'create_time': '2023-10-09T06:39:21.247510+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '183', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'volume_0000000000719180'}}}] 2023-10-09T06:39:38.841 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:39:38.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.960+0000 7f25935af700 1 Processor -- start 2023-10-09T06:39:38.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.960+0000 7f25935af700 1 -- start start 2023-10-09T06:39:38.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.961+0000 7f25935af700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f258c159890 0x7f258c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:38.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.961+0000 7f25935af700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f258c159c60 0x7f258c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:38.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.961+0000 7f25935af700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f258c15a160 con 0x7f258c15db10 2023-10-09T06:39:38.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.961+0000 7f25935af700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f258c15df50 con 0x7f258c159890 2023-10-09T06:39:38.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.961+0000 7f25935af700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f258c0d13a0 con 0x7f258c159c60 2023-10-09T06:39:38.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.961+0000 7f259134b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f258c159890 0x7f258c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:38.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.961+0000 7f259134b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f258c159890 0x7f258c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39350/0 (socket says 172.21.15.163:39350) 2023-10-09T06:39:38.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.961+0000 7f259134b700 1 -- 172.21.15.163:0/137857803 learned_addr learned my addr 172.21.15.163:0/137857803 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:38.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.961+0000 7f2590b4a700 1 --2- 172.21.15.163:0/137857803 >> v2:172.21.15.181:3300/0 conn(0x7f258c159c60 0x7f258c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:38.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.962+0000 7f259134b700 1 -- 172.21.15.163:0/137857803 >> v2:172.21.15.181:3300/0 conn(0x7f258c159c60 msgr2=0x7f258c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:38.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.962+0000 7f259134b700 1 --2- 172.21.15.163:0/137857803 >> v2:172.21.15.181:3300/0 conn(0x7f258c159c60 0x7f258c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:38.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.962+0000 7f259134b700 1 -- 172.21.15.163:0/137857803 >> v1:172.21.15.181:6789/0 conn(0x7f258c15db10 legacy=0x7f258c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:38.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.962+0000 7f259134b700 1 -- 172.21.15.163:0/137857803 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258c15e140 con 0x7f258c159890 2023-10-09T06:39:38.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.962+0000 7f259134b700 1 --2- 172.21.15.163:0/137857803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f258c159890 0x7f258c15abd0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f258c0b67f0 tx=0x7f257c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c004d40bf9ef62a server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:38.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.962+0000 7f258bfff700 1 -- 172.21.15.163:0/137857803 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c00e070 con 0x7f258c159890 2023-10-09T06:39:38.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.962+0000 7f258bfff700 1 -- 172.21.15.163:0/137857803 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257c00ba20 con 0x7f258c159890 2023-10-09T06:39:38.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.962+0000 7f258bfff700 1 -- 172.21.15.163:0/137857803 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c0092e0 con 0x7f258c159890 2023-10-09T06:39:38.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.962+0000 7f25935af700 1 -- 172.21.15.163:0/137857803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f258c159890 msgr2=0x7f258c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:38.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.962+0000 7f25935af700 1 --2- 172.21.15.163:0/137857803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f258c159890 0x7f258c15abd0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f258c0b67f0 tx=0x7f257c009840 comp rx=0 tx=0).stop 2023-10-09T06:39:38.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.962+0000 7f25935af700 1 -- 172.21.15.163:0/137857803 shutdown_connections 2023-10-09T06:39:38.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.962+0000 7f25935af700 1 --2- 172.21.15.163:0/137857803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f258c159890 0x7f258c15abd0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:38.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.962+0000 7f25935af700 1 --2- 172.21.15.163:0/137857803 >> v2:172.21.15.181:3300/0 conn(0x7f258c159c60 0x7f258c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:38.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.962+0000 7f25935af700 1 -- 172.21.15.163:0/137857803 >> 172.21.15.163:0/137857803 conn(0x7f258c068e20 msgr2=0x7f258c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:38.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.962+0000 7f25935af700 1 -- 172.21.15.163:0/137857803 shutdown_connections 2023-10-09T06:39:38.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.962+0000 7f25935af700 1 -- 172.21.15.163:0/137857803 wait complete. 2023-10-09T06:39:38.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.963+0000 7f25935af700 1 Processor -- start 2023-10-09T06:39:38.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.964+0000 7f25935af700 1 -- start start 2023-10-09T06:39:38.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.964+0000 7f25935af700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f258c159890 0x7f258c0b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:38.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.964+0000 7f25935af700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f258c159c60 0x7f258c0c9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:38.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.964+0000 7f25935af700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f258c15db10 0x7f258c16d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:38.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.964+0000 7f25935af700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c0cd7e0 con 0x7f258c159890 2023-10-09T06:39:38.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.964+0000 7f25935af700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c0c9ad0 con 0x7f258c159c60 2023-10-09T06:39:38.968 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.964+0000 7f25935af700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258c0c9db0 con 0x7f258c15db10 2023-10-09T06:39:38.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.964+0000 7f259134b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f258c159890 0x7f258c0b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:38.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.965+0000 7f259134b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f258c159890 0x7f258c0b3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39352/0 (socket says 172.21.15.163:39352) 2023-10-09T06:39:38.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.965+0000 7f259134b700 1 -- 172.21.15.163:0/2307342137 learned_addr learned my addr 172.21.15.163:0/2307342137 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:38.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.965+0000 7f2590b4a700 1 --2- 172.21.15.163:0/2307342137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f258c159c60 0x7f258c0c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:38.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.965+0000 7f2591b4c700 1 --2- 172.21.15.163:0/2307342137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f258c15db10 0x7f258c16d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:38.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.965+0000 7f259134b700 1 -- 172.21.15.163:0/2307342137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f258c15db10 msgr2=0x7f258c16d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:38.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.965+0000 7f259134b700 1 --2- 172.21.15.163:0/2307342137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f258c15db10 0x7f258c16d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:38.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.965+0000 7f259134b700 1 -- 172.21.15.163:0/2307342137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f258c159c60 msgr2=0x7f258c0c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:38.969 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.965+0000 7f259134b700 1 --2- 172.21.15.163:0/2307342137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f258c159c60 0x7f258c0c9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:38.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.965+0000 7f259134b700 1 -- 172.21.15.163:0/2307342137 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257c009ae0 con 0x7f258c159890 2023-10-09T06:39:38.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.965+0000 7f259134b700 1 --2- 172.21.15.163:0/2307342137 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f258c159890 0x7f258c0b3b10 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f258c070e20 tx=0x7f257c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:38.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.965+0000 7f25897fa700 1 -- 172.21.15.163:0/2307342137 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c0078f0 con 0x7f258c159890 2023-10-09T06:39:38.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.965+0000 7f25897fa700 1 -- 172.21.15.163:0/2307342137 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257c007a70 con 0x7f258c159890 2023-10-09T06:39:38.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.965+0000 7f25897fa700 1 -- 172.21.15.163:0/2307342137 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c007e30 con 0x7f258c159890 2023-10-09T06:39:38.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.965+0000 7f25935af700 1 -- 172.21.15.163:0/2307342137 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f258c0ca090 con 0x7f258c159890 2023-10-09T06:39:38.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.965+0000 7f25935af700 1 -- 172.21.15.163:0/2307342137 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f258c0c97c0 con 0x7f258c159890 2023-10-09T06:39:38.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.966+0000 7f25935af700 1 -- 172.21.15.163:0/2307342137 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f258c04c3c0 con 0x7f258c159890 2023-10-09T06:39:38.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.967+0000 7f25897fa700 1 -- 172.21.15.163:0/2307342137 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f257c003040 con 0x7f258c159890 2023-10-09T06:39:38.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.967+0000 7f25897fa700 1 --2- 172.21.15.163:0/2307342137 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2578067320 0x7f25780697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:38.970 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.967+0000 7f25897fa700 1 -- 172.21.15.163:0/2307342137 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f257c086910 con 0x7f258c159890 2023-10-09T06:39:38.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.967+0000 7f2590b4a700 1 --2- 172.21.15.163:0/2307342137 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2578067320 0x7f25780697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:38.971 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.968+0000 7f2590b4a700 1 --2- 172.21.15.163:0/2307342137 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2578067320 0x7f25780697d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f258c147b00 tx=0x7f25800091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:38.972 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:38.971+0000 7f25897fa700 1 -- 172.21.15.163:0/2307342137 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f257c0534f0 con 0x7f258c159890 2023-10-09T06:39:39.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.155+0000 7f25935af700 1 -- 172.21.15.163:0/2307342137 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f258c06a5b0 con 0x7f258c159890 2023-10-09T06:39:39.156 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.155+0000 7f25897fa700 1 -- 172.21.15.163:0/2307342137 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v185) v1 ==== 93+0+10 (secure 0 0 0) 0x7f257c056b10 con 0x7f258c159890 2023-10-09T06:39:39.156 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:39:39.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.158+0000 7f25935af700 1 -- 172.21.15.163:0/2307342137 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2578067320 msgr2=0x7f25780697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:39.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.158+0000 7f25935af700 1 --2- 172.21.15.163:0/2307342137 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2578067320 0x7f25780697d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f258c147b00 tx=0x7f25800091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.159 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.158+0000 7f25935af700 1 -- 172.21.15.163:0/2307342137 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f258c159890 msgr2=0x7f258c0b3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:39.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.158+0000 7f25935af700 1 --2- 172.21.15.163:0/2307342137 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f258c159890 0x7f258c0b3b10 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f258c070e20 tx=0x7f257c018000 comp rx=0 tx=0).stop 2023-10-09T06:39:39.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.158+0000 7f25935af700 1 -- 172.21.15.163:0/2307342137 shutdown_connections 2023-10-09T06:39:39.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.158+0000 7f25935af700 1 --2- 172.21.15.163:0/2307342137 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2578067320 0x7f25780697d0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.158+0000 7f25935af700 1 --2- 172.21.15.163:0/2307342137 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f258c159890 0x7f258c0b3b10 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.158+0000 7f25935af700 1 --2- 172.21.15.163:0/2307342137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f258c15db10 0x7f258c16d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.158+0000 7f25935af700 1 --2- 172.21.15.163:0/2307342137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f258c159c60 0x7f258c0c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.158+0000 7f25935af700 1 -- 172.21.15.163:0/2307342137 >> 172.21.15.163:0/2307342137 conn(0x7f258c068e20 msgr2=0x7f258c1531f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:39.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.158+0000 7f25935af700 1 -- 172.21.15.163:0/2307342137 shutdown_connections 2023-10-09T06:39:39.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.158+0000 7f25935af700 1 -- 172.21.15.163:0/2307342137 wait complete. 2023-10-09T06:39:39.171 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000719180.meta pg_num 2023-10-09T06:39:39.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.287+0000 7fabcb69e700 1 Processor -- start 2023-10-09T06:39:39.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.287+0000 7fabcb69e700 1 -- start start 2023-10-09T06:39:39.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.288+0000 7fabcb69e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabc406bcd0 0x7fabc40caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:39.292 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.288+0000 7fabcb69e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fabc406c0a0 0x7fabc415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:39.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.288+0000 7fabcb69e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc415d7e0 con 0x7fabc415df50 2023-10-09T06:39:39.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.288+0000 7fabcb69e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc415dac0 con 0x7fabc406c0a0 2023-10-09T06:39:39.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.288+0000 7fabcb69e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc415aa30 con 0x7fabc406bcd0 2023-10-09T06:39:39.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.288+0000 7fabc9c3b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fabc415df50 0x7fabc40c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:51996/0 (socket says 172.21.15.163:51996) 2023-10-09T06:39:39.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.288+0000 7fabc9c3b700 1 -- 172.21.15.163:0/1746360127 learned_addr learned my addr 172.21.15.163:0/1746360127 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:39.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.289+0000 7fabc943a700 1 --2- 172.21.15.163:0/1746360127 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabc406bcd0 0x7fabc40caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:39.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.289+0000 7fabc8c39700 1 --2- 172.21.15.163:0/1746360127 >> v2:172.21.15.181:3300/0 conn(0x7fabc406c0a0 0x7fabc415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:39.293 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.289+0000 7fabbbfff700 1 -- 172.21.15.163:0/1746360127 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2920259744 0 0) 0x7fabc415d7e0 con 0x7fabc415df50 2023-10-09T06:39:39.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.289+0000 7fabbbfff700 1 -- 172.21.15.163:0/1746360127 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabb0003570 con 0x7fabc415df50 2023-10-09T06:39:39.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.289+0000 7fabbbfff700 1 -- 172.21.15.163:0/1746360127 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 13746160 0 0) 0x7fabb0003570 con 0x7fabc415df50 2023-10-09T06:39:39.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.289+0000 7fabbbfff700 1 -- 172.21.15.163:0/1746360127 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabc406bcd0 msgr2=0x7fabc40caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:39.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.289+0000 7fabbbfff700 1 --2- 172.21.15.163:0/1746360127 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabc406bcd0 0x7fabc40caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.289+0000 7fabbbfff700 1 -- 172.21.15.163:0/1746360127 >> v2:172.21.15.181:3300/0 conn(0x7fabc406c0a0 msgr2=0x7fabc415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:39.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.289+0000 7fabbbfff700 1 --2- 172.21.15.163:0/1746360127 >> v2:172.21.15.181:3300/0 conn(0x7fabc406c0a0 0x7fabc415a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.289+0000 7fabbbfff700 1 -- 172.21.15.163:0/1746360127 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc415ad10 con 0x7fabc415df50 2023-10-09T06:39:39.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.289+0000 7fabbbfff700 1 -- 172.21.15.163:0/1746360127 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fabbc0017f0 con 0x7fabc415df50 2023-10-09T06:39:39.294 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.289+0000 7fabbbfff700 1 -- 172.21.15.163:0/1746360127 >> v1:172.21.15.163:6789/0 conn(0x7fabc415df50 legacy=0x7fabc40c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:39.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.289+0000 7fabbbfff700 1 --2- 172.21.15.163:0/1746360127 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabc406bcd0 0x7fabb0003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:39.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.289+0000 7fabbbfff700 1 --2- 172.21.15.163:0/1746360127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabb0003fe0 0x7fabb0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:39.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.289+0000 7fabbbfff700 1 --2- 172.21.15.163:0/1746360127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabb0004d50 0x7fabb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:39.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.289+0000 7fabbbfff700 1 -- 172.21.15.163:0/1746360127 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc415dac0 con 0x7fabb0004d50 2023-10-09T06:39:39.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.289+0000 7fabbbfff700 1 -- 172.21.15.163:0/1746360127 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc415aa30 con 0x7fabb0003fe0 2023-10-09T06:39:39.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.289+0000 7fabbbfff700 1 -- 172.21.15.163:0/1746360127 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb000bb40 con 0x7fabc406bcd0 2023-10-09T06:39:39.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.289+0000 7fabc9c3b700 1 --2- 172.21.15.163:0/1746360127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabb0004d50 0x7fabb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:39.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.289+0000 7fabc943a700 1 --2- 172.21.15.163:0/1746360127 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabc406bcd0 0x7fabb0003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:39.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.289+0000 7fabc8c39700 1 --2- 172.21.15.163:0/1746360127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabb0003fe0 0x7fabb0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:39.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.289+0000 7fabc9c3b700 1 -- 172.21.15.163:0/1746360127 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabc406bcd0 msgr2=0x7fabb0003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:39.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.289+0000 7fabc9c3b700 1 --2- 172.21.15.163:0/1746360127 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabc406bcd0 0x7fabb0003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.295 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.289+0000 7fabc9c3b700 1 -- 172.21.15.163:0/1746360127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabb0003fe0 msgr2=0x7fabb0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:39.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.289+0000 7fabc9c3b700 1 --2- 172.21.15.163:0/1746360127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabb0003fe0 0x7fabb0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.289+0000 7fabc9c3b700 1 -- 172.21.15.163:0/1746360127 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb0003570 con 0x7fabb0004d50 2023-10-09T06:39:39.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.290+0000 7fabc9c3b700 1 --2- 172.21.15.163:0/1746360127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabb0004d50 0x7fabb00051c0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fabbc00a940 tx=0x7fabbc002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dfb62b46e2e9ebc server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:39.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.290+0000 7fabbbfff700 1 -- 172.21.15.163:0/1746360127 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc007110 con 0x7fabb0004d50 2023-10-09T06:39:39.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.290+0000 7fabbbfff700 1 -- 172.21.15.163:0/1746360127 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabbc007290 con 0x7fabb0004d50 2023-10-09T06:39:39.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.290+0000 7fabbbfff700 1 -- 172.21.15.163:0/1746360127 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc007650 con 0x7fabb0004d50 2023-10-09T06:39:39.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.290+0000 7fabcb69e700 1 -- 172.21.15.163:0/1746360127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabb0004d50 msgr2=0x7fabb00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:39.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.290+0000 7fabcb69e700 1 --2- 172.21.15.163:0/1746360127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabb0004d50 0x7fabb00051c0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fabbc00a940 tx=0x7fabbc002210 comp rx=0 tx=0).stop 2023-10-09T06:39:39.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.290+0000 7fabc9c3b700 1 -- 172.21.15.163:0/1746360127 reap_dead start 2023-10-09T06:39:39.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.290+0000 7fabcb69e700 1 -- 172.21.15.163:0/1746360127 shutdown_connections 2023-10-09T06:39:39.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.290+0000 7fabcb69e700 1 -- 172.21.15.163:0/1746360127 >> 172.21.15.163:0/1746360127 conn(0x7fabc40ce600 msgr2=0x7fabc40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:39.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.290+0000 7fabcb69e700 1 -- 172.21.15.163:0/1746360127 shutdown_connections 2023-10-09T06:39:39.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.290+0000 7fabcb69e700 1 -- 172.21.15.163:0/1746360127 wait complete. 2023-10-09T06:39:39.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.291+0000 7fabcb69e700 1 Processor -- start 2023-10-09T06:39:39.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.291+0000 7fabcb69e700 1 -- start start 2023-10-09T06:39:39.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.292+0000 7fabcb69e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabc415ca20 0x7fabc40b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:39.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.292+0000 7fabcb69e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabc415cdf0 0x7fabc416e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:39.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.292+0000 7fabcb69e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabc40b21b0 0x7fabc40b2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:39.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.292+0000 7fabcb69e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc40b43b0 con 0x7fabc415ca20 2023-10-09T06:39:39.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.292+0000 7fabcb69e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc416e5e0 con 0x7fabc415cdf0 2023-10-09T06:39:39.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.292+0000 7fabcb69e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc40b3df0 con 0x7fabc40b21b0 2023-10-09T06:39:39.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.292+0000 7fabc943a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabc415ca20 0x7fabc40b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:39.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.292+0000 7fabc943a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabc415ca20 0x7fabc40b36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39356/0 (socket says 172.21.15.163:39356) 2023-10-09T06:39:39.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.292+0000 7fabc943a700 1 -- 172.21.15.163:0/3976967147 learned_addr learned my addr 172.21.15.163:0/3976967147 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:39.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.292+0000 7fabc9c3b700 1 --2- 172.21.15.163:0/3976967147 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabc40b21b0 0x7fabc40b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:39.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.293+0000 7fabc8c39700 1 --2- 172.21.15.163:0/3976967147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabc415cdf0 0x7fabc416e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:39.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.293+0000 7fabc943a700 1 -- 172.21.15.163:0/3976967147 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabc40b21b0 msgr2=0x7fabc40b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:39.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.293+0000 7fabc943a700 1 --2- 172.21.15.163:0/3976967147 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabc40b21b0 0x7fabc40b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.293+0000 7fabc943a700 1 -- 172.21.15.163:0/3976967147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabc415cdf0 msgr2=0x7fabc416e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:39.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.293+0000 7fabc943a700 1 --2- 172.21.15.163:0/3976967147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabc415cdf0 0x7fabc416e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.293+0000 7fabc943a700 1 -- 172.21.15.163:0/3976967147 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc40b40d0 con 0x7fabc415ca20 2023-10-09T06:39:39.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.293+0000 7fabc943a700 1 --2- 172.21.15.163:0/3976967147 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabc415ca20 0x7fabc40b36f0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fabc0011b10 tx=0x7fabc00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:39.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.293+0000 7fabb97fa700 1 -- 172.21.15.163:0/3976967147 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc000f940 con 0x7fabc415ca20 2023-10-09T06:39:39.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.293+0000 7fabb97fa700 1 -- 172.21.15.163:0/3976967147 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc000faa0 con 0x7fabc415ca20 2023-10-09T06:39:39.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.293+0000 7fabb97fa700 1 -- 172.21.15.163:0/3976967147 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc000fe20 con 0x7fabc415ca20 2023-10-09T06:39:39.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.293+0000 7fabcb69e700 1 -- 172.21.15.163:0/3976967147 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc40b19b0 con 0x7fabc415ca20 2023-10-09T06:39:39.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.293+0000 7fabcb69e700 1 -- 172.21.15.163:0/3976967147 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc40c6c90 con 0x7fabc415ca20 2023-10-09T06:39:39.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.294+0000 7fabcb69e700 1 -- 172.21.15.163:0/3976967147 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc40b1810 con 0x7fabc415ca20 2023-10-09T06:39:39.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.294+0000 7fabb97fa700 1 -- 172.21.15.163:0/3976967147 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fabc0014650 con 0x7fabc415ca20 2023-10-09T06:39:39.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.295+0000 7fabb97fa700 1 --2- 172.21.15.163:0/3976967147 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faba8067320 0x7faba80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:39.301 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.295+0000 7fabb97fa700 1 -- 172.21.15.163:0/3976967147 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5783+0+0 (secure 0 0 0) 0x7fabc0018030 con 0x7fabc415ca20 2023-10-09T06:39:39.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.295+0000 7fabc8c39700 1 --2- 172.21.15.163:0/3976967147 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faba8067320 0x7faba80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:39.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.295+0000 7fabc8c39700 1 --2- 172.21.15.163:0/3976967147 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faba8067320 0x7faba80697d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fabc40cbea0 tx=0x7fabb40091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:39.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.299+0000 7fabb97fa700 1 -- 172.21.15.163:0/3976967147 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fabc40b1810 con 0x7fabc415ca20 2023-10-09T06:39:39.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.478+0000 7fabcb69e700 1 -- 172.21.15.163:0/3976967147 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000719180.meta", "var": "pg_num"} v 0) v1 -- 0x7fabc40b1810 con 0x7fabc415ca20 2023-10-09T06:39:39.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.478+0000 7fabb97fa700 1 -- 172.21.15.163:0/3976967147 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000719180.meta", "var": "pg_num"}]=0 v185) v1 ==== 124+0+11 (secure 0 0 0) 0x7fabc40b1810 con 0x7fabc415ca20 2023-10-09T06:39:39.479 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 32 2023-10-09T06:39:39.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.480+0000 7fabcb69e700 1 -- 172.21.15.163:0/3976967147 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faba8067320 msgr2=0x7faba80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:39.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.480+0000 7fabcb69e700 1 --2- 172.21.15.163:0/3976967147 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faba8067320 0x7faba80697d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fabc40cbea0 tx=0x7fabb40091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.480+0000 7fabcb69e700 1 -- 172.21.15.163:0/3976967147 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabc415ca20 msgr2=0x7fabc40b36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:39.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.480+0000 7fabcb69e700 1 --2- 172.21.15.163:0/3976967147 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabc415ca20 0x7fabc40b36f0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fabc0011b10 tx=0x7fabc00091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.480+0000 7fabcb69e700 1 -- 172.21.15.163:0/3976967147 shutdown_connections 2023-10-09T06:39:39.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.480+0000 7fabcb69e700 1 --2- 172.21.15.163:0/3976967147 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faba8067320 0x7faba80697d0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.480+0000 7fabcb69e700 1 --2- 172.21.15.163:0/3976967147 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabc415ca20 0x7fabc40b36f0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.480+0000 7fabcb69e700 1 --2- 172.21.15.163:0/3976967147 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabc40b21b0 0x7fabc40b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.483 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.480+0000 7fabcb69e700 1 --2- 172.21.15.163:0/3976967147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabc415cdf0 0x7fabc416e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.480+0000 7fabcb69e700 1 -- 172.21.15.163:0/3976967147 >> 172.21.15.163:0/3976967147 conn(0x7fabc40ce600 msgr2=0x7fabc415b600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:39.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.480+0000 7fabcb69e700 1 -- 172.21.15.163:0/3976967147 shutdown_connections 2023-10-09T06:39:39.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.480+0000 7fabcb69e700 1 -- 172.21.15.163:0/3976967147 wait complete. 2023-10-09T06:39:39.494 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000719180.data pg_num 2023-10-09T06:39:39.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.611+0000 7f3536f9a700 1 Processor -- start 2023-10-09T06:39:39.615 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.611+0000 7f3536f9a700 1 -- start start 2023-10-09T06:39:39.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.612+0000 7f3536f9a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3530140350 0x7f3530069c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:39.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.612+0000 7f3536f9a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f353006a1c0 0x7f3530068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:39.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.612+0000 7f3536f9a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3530140720 0x7f35300688d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:39.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.612+0000 7f3536f9a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35300c7de0 con 0x7f3530140720 2023-10-09T06:39:39.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.612+0000 7f3536f9a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3530140c10 con 0x7f3530140350 2023-10-09T06:39:39.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.612+0000 7f3536f9a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3530141490 con 0x7f353006a1c0 2023-10-09T06:39:39.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.612+0000 7f3535537700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3530140720 0x7f35300688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:39.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.612+0000 7f3535537700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3530140720 0x7f35300688d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39370/0 (socket says 172.21.15.163:39370) 2023-10-09T06:39:39.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.613+0000 7f3535537700 1 -- 172.21.15.163:0/2688316488 learned_addr learned my addr 172.21.15.163:0/2688316488 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:39.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.613+0000 7f352ffff700 1 --2- 172.21.15.163:0/2688316488 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f353006a1c0 0x7f3530068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:39.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.613+0000 7f3534d36700 1 --2- 172.21.15.163:0/2688316488 >> v2:172.21.15.181:3300/0 conn(0x7f3530140350 0x7f3530069c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:39.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.613+0000 7f3535537700 1 -- 172.21.15.163:0/2688316488 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f353006a1c0 msgr2=0x7f3530068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:39.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.613+0000 7f3535537700 1 --2- 172.21.15.163:0/2688316488 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f353006a1c0 0x7f3530068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.613+0000 7f3535537700 1 -- 172.21.15.163:0/2688316488 >> v2:172.21.15.181:3300/0 conn(0x7f3530140350 msgr2=0x7f3530069c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:39.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.613+0000 7f3535537700 1 --2- 172.21.15.163:0/2688316488 >> v2:172.21.15.181:3300/0 conn(0x7f3530140350 0x7f3530069c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.613+0000 7f3535537700 1 -- 172.21.15.163:0/2688316488 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3530140ed0 con 0x7f3530140720 2023-10-09T06:39:39.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.613+0000 7f3535537700 1 --2- 172.21.15.163:0/2688316488 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3530140720 0x7f35300688d0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f353006d4b0 tx=0x7f3528009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf40e7947f46bfcb server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:39.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.613+0000 7f352f7fe700 1 -- 172.21.15.163:0/2688316488 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352800b860 con 0x7f3530140720 2023-10-09T06:39:39.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.613+0000 7f352f7fe700 1 -- 172.21.15.163:0/2688316488 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f352800b9c0 con 0x7f3530140720 2023-10-09T06:39:39.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.613+0000 7f352f7fe700 1 -- 172.21.15.163:0/2688316488 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352800bd60 con 0x7f3530140720 2023-10-09T06:39:39.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.614+0000 7f3536f9a700 1 -- 172.21.15.163:0/2688316488 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3530140720 msgr2=0x7f35300688d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:39.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.614+0000 7f3536f9a700 1 --2- 172.21.15.163:0/2688316488 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3530140720 0x7f35300688d0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f353006d4b0 tx=0x7f3528009250 comp rx=0 tx=0).stop 2023-10-09T06:39:39.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.614+0000 7f3536f9a700 1 -- 172.21.15.163:0/2688316488 shutdown_connections 2023-10-09T06:39:39.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.614+0000 7f3536f9a700 1 --2- 172.21.15.163:0/2688316488 >> v2:172.21.15.181:3300/0 conn(0x7f3530140350 0x7f3530069c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.614+0000 7f3536f9a700 1 --2- 172.21.15.163:0/2688316488 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3530140720 0x7f35300688d0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.614+0000 7f3536f9a700 1 --2- 172.21.15.163:0/2688316488 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f353006a1c0 0x7f3530068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.614+0000 7f3536f9a700 1 -- 172.21.15.163:0/2688316488 >> 172.21.15.163:0/2688316488 conn(0x7f35300cd670 msgr2=0x7f35300cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:39.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.614+0000 7f3536f9a700 1 -- 172.21.15.163:0/2688316488 shutdown_connections 2023-10-09T06:39:39.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.614+0000 7f3536f9a700 1 -- 172.21.15.163:0/2688316488 wait complete. 2023-10-09T06:39:39.618 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.615+0000 7f3536f9a700 1 Processor -- start 2023-10-09T06:39:39.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.615+0000 7f3536f9a700 1 -- start start 2023-10-09T06:39:39.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.616+0000 7f3536f9a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f353006a1c0 0x7f35300c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:39.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.616+0000 7f3536f9a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3530140350 0x7f353013b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:39.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.616+0000 7f3536f9a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3530140720 0x7f35300c34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:39.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.616+0000 7f3536f9a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35300c5670 con 0x7f3530140720 2023-10-09T06:39:39.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.616+0000 7f3536f9a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3530159180 con 0x7f3530140350 2023-10-09T06:39:39.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.616+0000 7f3536f9a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3530159460 con 0x7f353006a1c0 2023-10-09T06:39:39.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.616+0000 7f3535537700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3530140720 0x7f35300c34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:39.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.616+0000 7f3535537700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3530140720 0x7f35300c34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39384/0 (socket says 172.21.15.163:39384) 2023-10-09T06:39:39.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.616+0000 7f3535537700 1 -- 172.21.15.163:0/1009910064 learned_addr learned my addr 172.21.15.163:0/1009910064 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:39.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.616+0000 7f3534d36700 1 --2- 172.21.15.163:0/1009910064 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f353006a1c0 0x7f35300c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:39.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.616+0000 7f352ffff700 1 --2- 172.21.15.163:0/1009910064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3530140350 0x7f353013b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:39.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.616+0000 7f3535537700 1 -- 172.21.15.163:0/1009910064 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f353006a1c0 msgr2=0x7f35300c5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:39.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.616+0000 7f3535537700 1 --2- 172.21.15.163:0/1009910064 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f353006a1c0 0x7f35300c5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.616+0000 7f3535537700 1 -- 172.21.15.163:0/1009910064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3530140350 msgr2=0x7f353013b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:39.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.616+0000 7f3535537700 1 --2- 172.21.15.163:0/1009910064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3530140350 0x7f353013b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.616+0000 7f3535537700 1 -- 172.21.15.163:0/1009910064 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35280094f0 con 0x7f3530140720 2023-10-09T06:39:39.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.616+0000 7f352ffff700 1 --2- 172.21.15.163:0/1009910064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3530140350 0x7f353013b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:39.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.616+0000 7f3535537700 1 --2- 172.21.15.163:0/1009910064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3530140720 0x7f35300c34d0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f353006c6a0 tx=0x7f3528017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:39.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.616+0000 7f352cff9700 1 -- 172.21.15.163:0/1009910064 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352801a630 con 0x7f3530140720 2023-10-09T06:39:39.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.616+0000 7f352cff9700 1 -- 172.21.15.163:0/1009910064 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f352801a7b0 con 0x7f3530140720 2023-10-09T06:39:39.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.616+0000 7f3536f9a700 1 -- 172.21.15.163:0/1009910064 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3530159740 con 0x7f3530140720 2023-10-09T06:39:39.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.616+0000 7f352cff9700 1 -- 172.21.15.163:0/1009910064 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352801ab70 con 0x7f3530140720 2023-10-09T06:39:39.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.616+0000 7f3536f9a700 1 -- 172.21.15.163:0/1009910064 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3530159d00 con 0x7f3530140720 2023-10-09T06:39:39.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.617+0000 7f3536f9a700 1 -- 172.21.15.163:0/1009910064 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f353004c3c0 con 0x7f3530140720 2023-10-09T06:39:39.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.618+0000 7f352cff9700 1 -- 172.21.15.163:0/1009910064 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f352802d440 con 0x7f3530140720 2023-10-09T06:39:39.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.618+0000 7f352cff9700 1 --2- 172.21.15.163:0/1009910064 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f351c067320 0x7f351c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:39.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.618+0000 7f352cff9700 1 -- 172.21.15.163:0/1009910064 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f35280868a0 con 0x7f3530140720 2023-10-09T06:39:39.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.618+0000 7f3534d36700 1 --2- 172.21.15.163:0/1009910064 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f351c067320 0x7f351c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:39.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.619+0000 7f3534d36700 1 --2- 172.21.15.163:0/1009910064 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f351c067320 0x7f351c0697d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f3520007680 tx=0x7f35200098d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:39.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.622+0000 7f352cff9700 1 -- 172.21.15.163:0/1009910064 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f35280534e0 con 0x7f3530140720 2023-10-09T06:39:39.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.804+0000 7f3536f9a700 1 -- 172.21.15.163:0/1009910064 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000719180.data", "var": "pg_num"} v 0) v1 -- 0x7f35301411b0 con 0x7f3530140720 2023-10-09T06:39:39.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.805+0000 7f352cff9700 1 -- 172.21.15.163:0/1009910064 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000719180.data", "var": "pg_num"}]=0 v185) v1 ==== 124+0+11 (secure 0 0 0) 0x7f3528056b00 con 0x7f3530140720 2023-10-09T06:39:39.806 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 64 2023-10-09T06:39:39.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.807+0000 7f3536f9a700 1 -- 172.21.15.163:0/1009910064 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f351c067320 msgr2=0x7f351c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:39.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.807+0000 7f3536f9a700 1 --2- 172.21.15.163:0/1009910064 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f351c067320 0x7f351c0697d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f3520007680 tx=0x7f35200098d0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.807+0000 7f3536f9a700 1 -- 172.21.15.163:0/1009910064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3530140720 msgr2=0x7f35300c34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:39.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.807+0000 7f3536f9a700 1 --2- 172.21.15.163:0/1009910064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3530140720 0x7f35300c34d0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f353006c6a0 tx=0x7f3528017000 comp rx=0 tx=0).stop 2023-10-09T06:39:39.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.807+0000 7f3536f9a700 1 -- 172.21.15.163:0/1009910064 shutdown_connections 2023-10-09T06:39:39.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.807+0000 7f3536f9a700 1 --2- 172.21.15.163:0/1009910064 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f351c067320 0x7f351c0697d0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.807+0000 7f3536f9a700 1 --2- 172.21.15.163:0/1009910064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3530140720 0x7f35300c34d0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.807+0000 7f3536f9a700 1 --2- 172.21.15.163:0/1009910064 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f353006a1c0 0x7f35300c5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.807+0000 7f3536f9a700 1 --2- 172.21.15.163:0/1009910064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3530140350 0x7f353013b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.807+0000 7f3536f9a700 1 -- 172.21.15.163:0/1009910064 >> 172.21.15.163:0/1009910064 conn(0x7f35300cd670 msgr2=0x7f353006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:39.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.807+0000 7f3536f9a700 1 -- 172.21.15.163:0/1009910064 shutdown_connections 2023-10-09T06:39:39.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.807+0000 7f3536f9a700 1 -- 172.21.15.163:0/1009910064 wait complete. 2023-10-09T06:39:39.820 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:39:39.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.934+0000 7f54a136d700 1 Processor -- start 2023-10-09T06:39:39.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.934+0000 7f54a136d700 1 -- start start 2023-10-09T06:39:39.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.935+0000 7f54a136d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f549c159c60 0x7f549c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:39.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.935+0000 7f54a136d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549c15a160 con 0x7f549c159890 2023-10-09T06:39:39.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.935+0000 7f54a136d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549c0d13a0 con 0x7f549c15db10 2023-10-09T06:39:39.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.935+0000 7f54a136d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f549c15df50 con 0x7f549c159c60 2023-10-09T06:39:39.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.935+0000 7f549affd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f549c159890 0x7f549c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:52006/0 (socket says 172.21.15.163:52006) 2023-10-09T06:39:39.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.935+0000 7f549affd700 1 -- 172.21.15.163:0/3908098770 learned_addr learned my addr 172.21.15.163:0/3908098770 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:39.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.935+0000 7f549a7fc700 1 --2- 172.21.15.163:0/3908098770 >> v2:172.21.15.181:3300/0 conn(0x7f549c159c60 0x7f549c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:39.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.935+0000 7f5499ffb700 1 -- 172.21.15.163:0/3908098770 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3094882397 0 0) 0x7f549c15a160 con 0x7f549c159890 2023-10-09T06:39:39.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.935+0000 7f5499ffb700 1 -- 172.21.15.163:0/3908098770 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5484003570 con 0x7f549c159890 2023-10-09T06:39:39.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.936+0000 7f5499ffb700 1 -- 172.21.15.163:0/3908098770 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1717981902 0 0) 0x7f5484003570 con 0x7f549c159890 2023-10-09T06:39:39.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.936+0000 7f549a7fc700 1 -- 172.21.15.163:0/3908098770 >> v1:172.21.15.181:6789/0 conn(0x7f549c15db10 legacy=0x7f549c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:39.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.936+0000 7f549a7fc700 1 -- 172.21.15.163:0/3908098770 >> v1:172.21.15.163:6789/0 conn(0x7f549c159890 legacy=0x7f549c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:39.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.936+0000 7f549a7fc700 1 -- 172.21.15.163:0/3908098770 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549c15e140 con 0x7f549c159c60 2023-10-09T06:39:39.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.936+0000 7f549a7fc700 1 --2- 172.21.15.163:0/3908098770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f549c159c60 0x7f549c15d5d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f54900098e0 tx=0x7f54900091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87df62f9963a9f89 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:39.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.936+0000 7f5499ffb700 1 -- 172.21.15.163:0/3908098770 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c001470 con 0x7f549c159c60 2023-10-09T06:39:39.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.936+0000 7f5499ffb700 1 -- 172.21.15.163:0/3908098770 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5490011420 con 0x7f549c159c60 2023-10-09T06:39:39.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.936+0000 7f5499ffb700 1 -- 172.21.15.163:0/3908098770 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54900117a0 con 0x7f549c159c60 2023-10-09T06:39:39.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.936+0000 7f54a136d700 1 -- 172.21.15.163:0/3908098770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f549c159c60 msgr2=0x7f549c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:39.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.936+0000 7f54a136d700 1 --2- 172.21.15.163:0/3908098770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f549c159c60 0x7f549c15d5d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f54900098e0 tx=0x7f54900091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.937+0000 7f54a136d700 1 -- 172.21.15.163:0/3908098770 shutdown_connections 2023-10-09T06:39:39.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.937+0000 7f54a136d700 1 --2- 172.21.15.163:0/3908098770 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f549c159c60 0x7f549c15d5d0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.937+0000 7f54a136d700 1 -- 172.21.15.163:0/3908098770 >> 172.21.15.163:0/3908098770 conn(0x7f549c068e20 msgr2=0x7f549c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:39.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.937+0000 7f54a136d700 1 -- 172.21.15.163:0/3908098770 shutdown_connections 2023-10-09T06:39:39.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.937+0000 7f54a136d700 1 -- 172.21.15.163:0/3908098770 wait complete. 2023-10-09T06:39:39.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.938+0000 7f54a136d700 1 Processor -- start 2023-10-09T06:39:39.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.938+0000 7f54a136d700 1 -- start start 2023-10-09T06:39:39.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.939+0000 7f54a136d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f549c159890 0x7f549c0cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:39.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.939+0000 7f54a136d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f549c159c60 0x7f549c145020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:39.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.939+0000 7f54a136d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f549c15db10 0x7f549c154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:39.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.939+0000 7f54a136d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c0cf040 con 0x7f549c159c60 2023-10-09T06:39:39.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.939+0000 7f54a136d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c0c6560 con 0x7f549c159890 2023-10-09T06:39:39.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.939+0000 7f54a136d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549c0cd640 con 0x7f549c15db10 2023-10-09T06:39:39.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.939+0000 7f549a7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f549c159c60 0x7f549c145020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:39.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.939+0000 7f549a7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f549c159c60 0x7f549c145020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39396/0 (socket says 172.21.15.163:39396) 2023-10-09T06:39:39.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.939+0000 7f549a7fc700 1 -- 172.21.15.163:0/399813999 learned_addr learned my addr 172.21.15.163:0/399813999 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:39.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.939+0000 7f549affd700 1 --2- 172.21.15.163:0/399813999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f549c159890 0x7f549c0cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:39.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.939+0000 7f549b7fe700 1 --2- 172.21.15.163:0/399813999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f549c15db10 0x7f549c154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:39.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.939+0000 7f549a7fc700 1 -- 172.21.15.163:0/399813999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f549c15db10 msgr2=0x7f549c154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:39.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.939+0000 7f549a7fc700 1 --2- 172.21.15.163:0/399813999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f549c15db10 0x7f549c154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.939+0000 7f549a7fc700 1 -- 172.21.15.163:0/399813999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f549c159890 msgr2=0x7f549c0cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:39.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.939+0000 7f549a7fc700 1 --2- 172.21.15.163:0/399813999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f549c159890 0x7f549c0cd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:39.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.939+0000 7f549a7fc700 1 -- 172.21.15.163:0/399813999 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5490009420 con 0x7f549c159c60 2023-10-09T06:39:39.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.939+0000 7f549a7fc700 1 --2- 172.21.15.163:0/399813999 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f549c159c60 0x7f549c145020 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f549c070e20 tx=0x7f5490016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:39.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.939+0000 7f5482ffd700 1 -- 172.21.15.163:0/399813999 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549001cb80 con 0x7f549c159c60 2023-10-09T06:39:39.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.939+0000 7f5482ffd700 1 -- 172.21.15.163:0/399813999 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549001cce0 con 0x7f549c159c60 2023-10-09T06:39:39.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.939+0000 7f54a136d700 1 -- 172.21.15.163:0/399813999 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549c0c5950 con 0x7f549c159c60 2023-10-09T06:39:39.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.939+0000 7f5482ffd700 1 -- 172.21.15.163:0/399813999 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549001a6a0 con 0x7f549c159c60 2023-10-09T06:39:39.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.939+0000 7f54a136d700 1 -- 172.21.15.163:0/399813999 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549c0c5f10 con 0x7f549c159c60 2023-10-09T06:39:39.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.940+0000 7f54a136d700 1 -- 172.21.15.163:0/399813999 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f549c04c3c0 con 0x7f549c159c60 2023-10-09T06:39:39.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.941+0000 7f5482ffd700 1 -- 172.21.15.163:0/399813999 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f549001aa50 con 0x7f549c159c60 2023-10-09T06:39:39.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.941+0000 7f5482ffd700 1 --2- 172.21.15.163:0/399813999 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5488067320 0x7f54880697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:39.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.941+0000 7f5482ffd700 1 -- 172.21.15.163:0/399813999 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f54900865e0 con 0x7f549c159c60 2023-10-09T06:39:39.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.941+0000 7f549affd700 1 --2- 172.21.15.163:0/399813999 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5488067320 0x7f54880697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:39.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.942+0000 7f549affd700 1 --2- 172.21.15.163:0/399813999 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5488067320 0x7f54880697d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f548c001620 tx=0x7f548c000bc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:39.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:39.945+0000 7f5482ffd700 1 -- 172.21.15.163:0/399813999 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5490053220 con 0x7f549c159c60 2023-10-09T06:39:40.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.170+0000 7f54a136d700 1 -- 172.21.15.163:0/399813999 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f549c089680 con 0x7f549c159c60 2023-10-09T06:39:40.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.171+0000 7f5482ffd700 1 -- 172.21.15.163:0/399813999 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4630 (secure 0 0 0) 0x7f5490056840 con 0x7f549c159c60 2023-10-09T06:39:40.172 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:40.172 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":125,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6815,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3960800872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3960800872},{"type":"v1","addr":"172.21.15.181:6836","nonce":3960800872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":8631,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/380361821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":380361821},{"type":"v1","addr":"172.21.15.163:6834","nonce":380361821}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":8634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2847943599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2847943599},{"type":"v1","addr":"172.21.15.163:6835","nonce":2847943599}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:39:22.273633+0000","modified":"2023-10-09T06:39:33.963575+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8649":{"gid":8649,"name":"d","rank":0,"incarnation":119,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/119178917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":119178917},{"type":"v1","addr":"172.21.15.181:6837","nonce":119178917}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"volume_0000000000719180","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-09T06:39:40.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.173+0000 7f54a136d700 1 -- 172.21.15.163:0/399813999 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5488067320 msgr2=0x7f54880697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:40.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.173+0000 7f54a136d700 1 --2- 172.21.15.163:0/399813999 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5488067320 0x7f54880697d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f548c001620 tx=0x7f548c000bc0 comp rx=0 tx=0).stop 2023-10-09T06:39:40.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.173+0000 7f54a136d700 1 -- 172.21.15.163:0/399813999 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f549c159c60 msgr2=0x7f549c145020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:40.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.173+0000 7f54a136d700 1 --2- 172.21.15.163:0/399813999 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f549c159c60 0x7f549c145020 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f549c070e20 tx=0x7f5490016000 comp rx=0 tx=0).stop 2023-10-09T06:39:40.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.173+0000 7f54a136d700 1 -- 172.21.15.163:0/399813999 shutdown_connections 2023-10-09T06:39:40.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.173+0000 7f54a136d700 1 --2- 172.21.15.163:0/399813999 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5488067320 0x7f54880697d0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:40.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.173+0000 7f54a136d700 1 --2- 172.21.15.163:0/399813999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f549c15db10 0x7f549c154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:40.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.173+0000 7f54a136d700 1 --2- 172.21.15.163:0/399813999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f549c159890 0x7f549c0cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:40.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.173+0000 7f54a136d700 1 --2- 172.21.15.163:0/399813999 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f549c159c60 0x7f549c145020 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:40.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.173+0000 7f54a136d700 1 -- 172.21.15.163:0/399813999 >> 172.21.15.163:0/399813999 conn(0x7f549c068e20 msgr2=0x7f549c1554c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:40.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.173+0000 7f54a136d700 1 -- 172.21.15.163:0/399813999 shutdown_connections 2023-10-09T06:39:40.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.173+0000 7f54a136d700 1 -- 172.21.15.163:0/399813999 wait complete. 2023-10-09T06:39:40.176 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 125 2023-10-09T06:39:40.187 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:39:40.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.304+0000 7f92e0e9d700 1 Processor -- start 2023-10-09T06:39:40.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.304+0000 7f92e0e9d700 1 -- start start 2023-10-09T06:39:40.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.305+0000 7f92e0e9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f92dc159890 0x7f92dc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:40.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.305+0000 7f92e0e9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92dc069560 0x7f92dc0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:40.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.305+0000 7f92e0e9d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92dc069ee0 con 0x7f92dc159c60 2023-10-09T06:39:40.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.305+0000 7f92e0e9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92dc15a160 con 0x7f92dc159890 2023-10-09T06:39:40.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.305+0000 7f92e0e9d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92dc0d13a0 con 0x7f92dc069560 2023-10-09T06:39:40.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.305+0000 7f92da59c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f92dc159890 0x7f92dc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:40.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.305+0000 7f92da59c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f92dc159890 0x7f92dc15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39412/0 (socket says 172.21.15.163:39412) 2023-10-09T06:39:40.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.305+0000 7f92da59c700 1 -- 172.21.15.163:0/2485654418 learned_addr learned my addr 172.21.15.163:0/2485654418 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:40.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.306+0000 7f92da59c700 1 -- 172.21.15.163:0/2485654418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92dc069560 msgr2=0x7f92dc0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:40.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.306+0000 7f92da59c700 1 --2- 172.21.15.163:0/2485654418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92dc069560 0x7f92dc0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:40.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.306+0000 7f92da59c700 1 -- 172.21.15.163:0/2485654418 >> v1:172.21.15.181:6789/0 conn(0x7f92dc159c60 legacy=0x7f92dc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:40.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.306+0000 7f92da59c700 1 -- 172.21.15.163:0/2485654418 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92dc15e140 con 0x7f92dc159890 2023-10-09T06:39:40.308 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.306+0000 7f92da59c700 1 --2- 172.21.15.163:0/2485654418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92dc159890 0x7f92dc15abd0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f92cc009f20 tx=0x7f92cc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479e3f1bae0aa082 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:40.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.306+0000 7f92d959a700 1 -- 172.21.15.163:0/2485654418 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92cc00e070 con 0x7f92dc159890 2023-10-09T06:39:40.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.306+0000 7f92d959a700 1 -- 172.21.15.163:0/2485654418 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92cc00ba40 con 0x7f92dc159890 2023-10-09T06:39:40.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.306+0000 7f92d959a700 1 -- 172.21.15.163:0/2485654418 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92cc0092e0 con 0x7f92dc159890 2023-10-09T06:39:40.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.306+0000 7f92e0e9d700 1 -- 172.21.15.163:0/2485654418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92dc159890 msgr2=0x7f92dc15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:40.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.306+0000 7f92e0e9d700 1 --2- 172.21.15.163:0/2485654418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92dc159890 0x7f92dc15abd0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f92cc009f20 tx=0x7f92cc009840 comp rx=0 tx=0).stop 2023-10-09T06:39:40.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.306+0000 7f92e0e9d700 1 -- 172.21.15.163:0/2485654418 shutdown_connections 2023-10-09T06:39:40.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.306+0000 7f92e0e9d700 1 --2- 172.21.15.163:0/2485654418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92dc069560 0x7f92dc0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:40.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.306+0000 7f92e0e9d700 1 --2- 172.21.15.163:0/2485654418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92dc159890 0x7f92dc15abd0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:40.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.306+0000 7f92e0e9d700 1 -- 172.21.15.163:0/2485654418 >> 172.21.15.163:0/2485654418 conn(0x7f92dc068e20 msgr2=0x7f92dc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:40.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.306+0000 7f92e0e9d700 1 -- 172.21.15.163:0/2485654418 shutdown_connections 2023-10-09T06:39:40.309 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.306+0000 7f92e0e9d700 1 -- 172.21.15.163:0/2485654418 wait complete. 2023-10-09T06:39:40.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.307+0000 7f92e0e9d700 1 Processor -- start 2023-10-09T06:39:40.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.308+0000 7f92e0e9d700 1 -- start start 2023-10-09T06:39:40.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.308+0000 7f92e0e9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92dc069560 0x7f92dc1542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:40.312 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.308+0000 7f92e0e9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92dc159890 0x7f92dc157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:40.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.308+0000 7f92e0e9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92dc159c60 0x7f92dc0b1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:40.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.308+0000 7f92e0e9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92dc156db0 con 0x7f92dc159890 2023-10-09T06:39:40.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.308+0000 7f92e0e9d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92dc156480 con 0x7f92dc069560 2023-10-09T06:39:40.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.308+0000 7f92e0e9d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92dc156760 con 0x7f92dc159c60 2023-10-09T06:39:40.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.308+0000 7f92d9d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92dc159890 0x7f92dc157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:40.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.308+0000 7f92d9d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92dc159890 0x7f92dc157bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39420/0 (socket says 172.21.15.163:39420) 2023-10-09T06:39:40.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.308+0000 7f92d9d9b700 1 -- 172.21.15.163:0/1765971542 learned_addr learned my addr 172.21.15.163:0/1765971542 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:40.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.309+0000 7f92da59c700 1 --2- 172.21.15.163:0/1765971542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92dc069560 0x7f92dc1542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:40.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.309+0000 7f92d9d9b700 1 -- 172.21.15.163:0/1765971542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92dc159c60 msgr2=0x7f92dc0b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:40.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.309+0000 7f92dad9d700 1 --2- 172.21.15.163:0/1765971542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92dc159c60 0x7f92dc0b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:40.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.309+0000 7f92d9d9b700 1 --2- 172.21.15.163:0/1765971542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92dc159c60 0x7f92dc0b1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:40.313 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.309+0000 7f92d9d9b700 1 -- 172.21.15.163:0/1765971542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92dc069560 msgr2=0x7f92dc1542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:40.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.309+0000 7f92d9d9b700 1 --2- 172.21.15.163:0/1765971542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92dc069560 0x7f92dc1542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:40.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.309+0000 7f92d9d9b700 1 -- 172.21.15.163:0/1765971542 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92cc009ae0 con 0x7f92dc159890 2023-10-09T06:39:40.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.309+0000 7f92da59c700 1 --2- 172.21.15.163:0/1765971542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92dc069560 0x7f92dc1542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:40.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.309+0000 7f92dad9d700 1 --2- 172.21.15.163:0/1765971542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92dc159c60 0x7f92dc0b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:40.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.309+0000 7f92d9d9b700 1 --2- 172.21.15.163:0/1765971542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92dc159890 0x7f92dc157bc0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f92d000b600 tx=0x7f92d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:40.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.309+0000 7f92ca7fc700 1 -- 172.21.15.163:0/1765971542 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d000f510 con 0x7f92dc159890 2023-10-09T06:39:40.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.309+0000 7f92ca7fc700 1 -- 172.21.15.163:0/1765971542 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d000f670 con 0x7f92dc159890 2023-10-09T06:39:40.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.309+0000 7f92e0e9d700 1 -- 172.21.15.163:0/1765971542 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92dc156a40 con 0x7f92dc159890 2023-10-09T06:39:40.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.309+0000 7f92ca7fc700 1 -- 172.21.15.163:0/1765971542 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d000fa30 con 0x7f92dc159890 2023-10-09T06:39:40.314 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.309+0000 7f92e0e9d700 1 -- 172.21.15.163:0/1765971542 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92dc154820 con 0x7f92dc159890 2023-10-09T06:39:40.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.310+0000 7f92e0e9d700 1 -- 172.21.15.163:0/1765971542 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92dc156f60 con 0x7f92dc159890 2023-10-09T06:39:40.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.310+0000 7f92ca7fc700 1 -- 172.21.15.163:0/1765971542 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f92d001a400 con 0x7f92dc159890 2023-10-09T06:39:40.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.311+0000 7f92ca7fc700 1 --2- 172.21.15.163:0/1765971542 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f92bc067320 0x7f92bc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:40.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.311+0000 7f92ca7fc700 1 -- 172.21.15.163:0/1765971542 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f92d0084760 con 0x7f92dc159890 2023-10-09T06:39:40.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.311+0000 7f92da59c700 1 --2- 172.21.15.163:0/1765971542 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f92bc067320 0x7f92bc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:40.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.311+0000 7f92da59c700 1 --2- 172.21.15.163:0/1765971542 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f92bc067320 0x7f92bc0697d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f92cc009ab0 tx=0x7f92cc017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:40.315 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.315+0000 7f92ca7fc700 1 -- 172.21.15.163:0/1765971542 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f92dc156f60 con 0x7f92dc159890 2023-10-09T06:39:40.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.499+0000 7f92e0e9d700 1 -- 172.21.15.163:0/1765971542 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92dc04c3c0 con 0x7f92dc159890 2023-10-09T06:39:40.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.500+0000 7f92ca7fc700 1 -- 172.21.15.163:0/1765971542 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) v1 ==== 74+0+15490 (secure 0 0 0) 0x7f92d00514a0 con 0x7f92dc159890 2023-10-09T06:39:40.501 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:40.501 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":185,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:39:35.978425+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":31,"pool_name":"cephfs.volume_0000000000719180.meta","create_time":"2023-10-09T06:39:20.612285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000719180"}}},{"pool":32,"pool_name":"cephfs.volume_0000000000719180.data","create_time":"2023-10-09T06:39:21.247510+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000719180"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:39:40.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.502+0000 7f92e0e9d700 1 -- 172.21.15.163:0/1765971542 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f92bc067320 msgr2=0x7f92bc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:40.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.502+0000 7f92e0e9d700 1 --2- 172.21.15.163:0/1765971542 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f92bc067320 0x7f92bc0697d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f92cc009ab0 tx=0x7f92cc017000 comp rx=0 tx=0).stop 2023-10-09T06:39:40.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.502+0000 7f92e0e9d700 1 -- 172.21.15.163:0/1765971542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92dc159890 msgr2=0x7f92dc157bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:40.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.502+0000 7f92e0e9d700 1 --2- 172.21.15.163:0/1765971542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92dc159890 0x7f92dc157bc0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f92d000b600 tx=0x7f92d00091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:40.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.502+0000 7f92e0e9d700 1 -- 172.21.15.163:0/1765971542 shutdown_connections 2023-10-09T06:39:40.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.502+0000 7f92e0e9d700 1 --2- 172.21.15.163:0/1765971542 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f92bc067320 0x7f92bc0697d0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:40.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.502+0000 7f92e0e9d700 1 --2- 172.21.15.163:0/1765971542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92dc159c60 0x7f92dc0b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:40.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.502+0000 7f92e0e9d700 1 --2- 172.21.15.163:0/1765971542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92dc069560 0x7f92dc1542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:40.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.502+0000 7f92e0e9d700 1 --2- 172.21.15.163:0/1765971542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92dc159890 0x7f92dc157bc0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:40.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.502+0000 7f92e0e9d700 1 -- 172.21.15.163:0/1765971542 >> 172.21.15.163:0/1765971542 conn(0x7f92dc068e20 msgr2=0x7f92dc15b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:40.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.502+0000 7f92e0e9d700 1 -- 172.21.15.163:0/1765971542 shutdown_connections 2023-10-09T06:39:40.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.502+0000 7f92e0e9d700 1 -- 172.21.15.163:0/1765971542 wait complete. 2023-10-09T06:39:40.516 INFO:tasks.cephfs.filesystem:Destroying file system volume_0000000000719180 and related pools 2023-10-09T06:39:40.516 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:39:40.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.629+0000 7fec8359e700 1 Processor -- start 2023-10-09T06:39:40.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.629+0000 7fec8359e700 1 -- start start 2023-10-09T06:39:40.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.630+0000 7fec8359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec840cc3c0 0x7fec8406b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:40.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.630+0000 7fec8359e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fec8406bf20 0x7fec840b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:40.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.630+0000 7fec8359e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec8406c380 con 0x7fec841574d0 2023-10-09T06:39:40.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.630+0000 7fec8359e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec840cd980 con 0x7fec8406bf20 2023-10-09T06:39:40.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.630+0000 7fec8359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec840b16a0 con 0x7fec840cc3c0 2023-10-09T06:39:40.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.630+0000 7fec82d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fec8406bf20 0x7fec840b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:40.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.630+0000 7fec8259c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fec841574d0 0x7fec840cd270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:52020/0 (socket says 172.21.15.163:52020) 2023-10-09T06:39:40.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.630+0000 7fec8259c700 1 -- 172.21.15.163:0/3153817046 learned_addr learned my addr 172.21.15.163:0/3153817046 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:40.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.631+0000 7fec81d9b700 1 --2- 172.21.15.163:0/3153817046 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec840cc3c0 0x7fec8406b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:40.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.631+0000 7fec82d9d700 1 -- 172.21.15.163:0/3153817046 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec840cc3c0 msgr2=0x7fec8406b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:40.633 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.631+0000 7fec82d9d700 1 --2- 172.21.15.163:0/3153817046 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec840cc3c0 0x7fec8406b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:40.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.631+0000 7fec82d9d700 1 -- 172.21.15.163:0/3153817046 >> v1:172.21.15.163:6789/0 conn(0x7fec841574d0 legacy=0x7fec840cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:40.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.631+0000 7fec82d9d700 1 -- 172.21.15.163:0/3153817046 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec840b1c60 con 0x7fec8406bf20 2023-10-09T06:39:40.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.631+0000 7fec8159a700 1 -- 172.21.15.163:0/3153817046 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2737874695 0 0) 0x7fec74001610 con 0x7fec841574d0 2023-10-09T06:39:40.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.631+0000 7fec82d9d700 1 --2- 172.21.15.163:0/3153817046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec8406bf20 0x7fec840b20f0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0098e0 tx=0x7fec7c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ae42be848a344a server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:40.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.631+0000 7fec8159a700 1 -- 172.21.15.163:0/3153817046 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c009e80 con 0x7fec8406bf20 2023-10-09T06:39:40.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.631+0000 7fec8159a700 1 -- 172.21.15.163:0/3153817046 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7c00b860 con 0x7fec8406bf20 2023-10-09T06:39:40.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.631+0000 7fec8159a700 1 -- 172.21.15.163:0/3153817046 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c00bbe0 con 0x7fec8406bf20 2023-10-09T06:39:40.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.631+0000 7fec8359e700 1 -- 172.21.15.163:0/3153817046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec8406bf20 msgr2=0x7fec840b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:40.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.631+0000 7fec8359e700 1 --2- 172.21.15.163:0/3153817046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec8406bf20 0x7fec840b20f0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0098e0 tx=0x7fec7c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:40.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.632+0000 7fec8359e700 1 -- 172.21.15.163:0/3153817046 shutdown_connections 2023-10-09T06:39:40.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.632+0000 7fec8359e700 1 --2- 172.21.15.163:0/3153817046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec8406bf20 0x7fec840b20f0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:40.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.632+0000 7fec8359e700 1 --2- 172.21.15.163:0/3153817046 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec840cc3c0 0x7fec8406b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:40.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.632+0000 7fec8359e700 1 -- 172.21.15.163:0/3153817046 >> 172.21.15.163:0/3153817046 conn(0x7fec840ce430 msgr2=0x7fec840ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:40.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.632+0000 7fec8359e700 1 -- 172.21.15.163:0/3153817046 shutdown_connections 2023-10-09T06:39:40.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.632+0000 7fec8359e700 1 -- 172.21.15.163:0/3153817046 wait complete. 2023-10-09T06:39:40.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.633+0000 7fec8359e700 1 Processor -- start 2023-10-09T06:39:40.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.633+0000 7fec8359e700 1 -- start start 2023-10-09T06:39:40.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.633+0000 7fec8359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec8406bf20 0x7fec8416b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:40.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.633+0000 7fec8359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec840cc3c0 0x7fec84152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:40.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.633+0000 7fec8359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec841574d0 0x7fec84167660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:40.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.633+0000 7fec8359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec840c4380 con 0x7fec8406bf20 2023-10-09T06:39:40.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.633+0000 7fec8359e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8416bb10 con 0x7fec840cc3c0 2023-10-09T06:39:40.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.633+0000 7fec8359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec840c3770 con 0x7fec841574d0 2023-10-09T06:39:40.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.634+0000 7fec8259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec8406bf20 0x7fec8416b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:40.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.634+0000 7fec8259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec8406bf20 0x7fec8416b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39432/0 (socket says 172.21.15.163:39432) 2023-10-09T06:39:40.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.634+0000 7fec8259c700 1 -- 172.21.15.163:0/4027303625 learned_addr learned my addr 172.21.15.163:0/4027303625 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:40.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.634+0000 7fec82d9d700 1 --2- 172.21.15.163:0/4027303625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec841574d0 0x7fec84167660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:40.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.634+0000 7fec8259c700 1 -- 172.21.15.163:0/4027303625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec841574d0 msgr2=0x7fec84167660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:40.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.634+0000 7fec8259c700 1 --2- 172.21.15.163:0/4027303625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec841574d0 0x7fec84167660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:40.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.634+0000 7fec8259c700 1 -- 172.21.15.163:0/4027303625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec840cc3c0 msgr2=0x7fec84152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:40.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.634+0000 7fec8259c700 1 --2- 172.21.15.163:0/4027303625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec840cc3c0 0x7fec84152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:40.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.634+0000 7fec8259c700 1 -- 172.21.15.163:0/4027303625 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7c009420 con 0x7fec8406bf20 2023-10-09T06:39:40.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.634+0000 7fec8259c700 1 --2- 172.21.15.163:0/4027303625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec8406bf20 0x7fec8416b5d0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fec7400c930 tx=0x7fec740016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:40.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.634+0000 7fec7a7fc700 1 -- 172.21.15.163:0/4027303625 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec74005d80 con 0x7fec8406bf20 2023-10-09T06:39:40.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.634+0000 7fec7a7fc700 1 -- 172.21.15.163:0/4027303625 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7400a5d0 con 0x7fec8406bf20 2023-10-09T06:39:40.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.634+0000 7fec8359e700 1 -- 172.21.15.163:0/4027303625 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec840c3a50 con 0x7fec8406bf20 2023-10-09T06:39:40.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.634+0000 7fec7a7fc700 1 -- 172.21.15.163:0/4027303625 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7400a990 con 0x7fec8406bf20 2023-10-09T06:39:40.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.634+0000 7fec8359e700 1 -- 172.21.15.163:0/4027303625 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec840c4010 con 0x7fec8406bf20 2023-10-09T06:39:40.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.635+0000 7fec8359e700 1 -- 172.21.15.163:0/4027303625 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec8404c3c0 con 0x7fec8406bf20 2023-10-09T06:39:40.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.636+0000 7fec7a7fc700 1 -- 172.21.15.163:0/4027303625 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fec740070f0 con 0x7fec8406bf20 2023-10-09T06:39:40.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.636+0000 7fec7a7fc700 1 --2- 172.21.15.163:0/4027303625 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fec64067320 0x7fec640697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:40.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.636+0000 7fec7a7fc700 1 -- 172.21.15.163:0/4027303625 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5783+0+0 (secure 0 0 0) 0x7fec74084970 con 0x7fec8406bf20 2023-10-09T06:39:40.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.636+0000 7fec81d9b700 1 --2- 172.21.15.163:0/4027303625 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fec64067320 0x7fec640697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:40.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.637+0000 7fec81d9b700 1 --2- 172.21.15.163:0/4027303625 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fec64067320 0x7fec640697d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fec840ad920 tx=0x7fec6c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:40.641 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.640+0000 7fec7a7fc700 1 -- 172.21.15.163:0/4027303625 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec74051630 con 0x7fec8406bf20 2023-10-09T06:39:40.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.866+0000 7fec8359e700 1 -- 172.21.15.163:0/4027303625 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec8406c380 con 0x7fec8406bf20 2023-10-09T06:39:40.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.867+0000 7fec7a7fc700 1 -- 172.21.15.163:0/4027303625 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4630 (secure 0 0 0) 0x7fec8406c380 con 0x7fec8406bf20 2023-10-09T06:39:40.868 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:40.868 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":125,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6815,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3960800872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3960800872},{"type":"v1","addr":"172.21.15.181:6836","nonce":3960800872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":8631,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/380361821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":380361821},{"type":"v1","addr":"172.21.15.163:6834","nonce":380361821}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":8634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2847943599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2847943599},{"type":"v1","addr":"172.21.15.163:6835","nonce":2847943599}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:39:22.273633+0000","modified":"2023-10-09T06:39:33.963575+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8649":{"gid":8649,"name":"d","rank":0,"incarnation":119,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/119178917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":119178917},{"type":"v1","addr":"172.21.15.181:6837","nonce":119178917}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"volume_0000000000719180","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-09T06:39:40.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.869+0000 7fec8359e700 1 -- 172.21.15.163:0/4027303625 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fec64067320 msgr2=0x7fec640697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:40.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.869+0000 7fec8359e700 1 --2- 172.21.15.163:0/4027303625 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fec64067320 0x7fec640697d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fec840ad920 tx=0x7fec6c008040 comp rx=0 tx=0).stop 2023-10-09T06:39:40.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.869+0000 7fec8359e700 1 -- 172.21.15.163:0/4027303625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec8406bf20 msgr2=0x7fec8416b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:40.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.869+0000 7fec8359e700 1 --2- 172.21.15.163:0/4027303625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec8406bf20 0x7fec8416b5d0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fec7400c930 tx=0x7fec740016f0 comp rx=0 tx=0).stop 2023-10-09T06:39:40.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.869+0000 7fec8359e700 1 -- 172.21.15.163:0/4027303625 shutdown_connections 2023-10-09T06:39:40.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.869+0000 7fec8359e700 1 --2- 172.21.15.163:0/4027303625 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fec64067320 0x7fec640697d0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:40.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.869+0000 7fec8359e700 1 --2- 172.21.15.163:0/4027303625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec8406bf20 0x7fec8416b5d0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:40.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.869+0000 7fec8359e700 1 --2- 172.21.15.163:0/4027303625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec841574d0 0x7fec84167660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:40.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.869+0000 7fec8359e700 1 --2- 172.21.15.163:0/4027303625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec840cc3c0 0x7fec84152e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:40.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.869+0000 7fec8359e700 1 -- 172.21.15.163:0/4027303625 >> 172.21.15.163:0/4027303625 conn(0x7fec840ce430 msgr2=0x7fec840b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:40.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.869+0000 7fec8359e700 1 -- 172.21.15.163:0/4027303625 shutdown_connections 2023-10-09T06:39:40.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.869+0000 7fec8359e700 1 -- 172.21.15.163:0/4027303625 wait complete. 2023-10-09T06:39:40.871 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 125 2023-10-09T06:39:40.882 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:39:40.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.995+0000 7f991cafe700 1 Processor -- start 2023-10-09T06:39:40.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.995+0000 7f991cafe700 1 -- start start 2023-10-09T06:39:40.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.996+0000 7f991cafe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9918159420 0x7f99180b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:40.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.996+0000 7f991cafe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f99181597f0 0x7f99180b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:40.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.996+0000 7f991cafe700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99180b11d0 con 0x7f991815d230 2023-10-09T06:39:40.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.996+0000 7f991cafe700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99180b14b0 con 0x7f99181597f0 2023-10-09T06:39:40.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.996+0000 7f991cafe700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99180b1790 con 0x7f9918159420 2023-10-09T06:39:40.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.996+0000 7f9916ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f99181597f0 0x7f99180b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:40.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.996+0000 7f9916ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f99181597f0 0x7f99180b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39444/0 (socket says 172.21.15.163:39444) 2023-10-09T06:39:40.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.996+0000 7f9916ffd700 1 -- 172.21.15.163:0/311978103 learned_addr learned my addr 172.21.15.163:0/311978103 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:40.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.996+0000 7f99177fe700 1 --2- 172.21.15.163:0/311978103 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9918159420 0x7f99180b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:40.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.996+0000 7f9916ffd700 1 -- 172.21.15.163:0/311978103 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9918159420 msgr2=0x7f99180b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:40.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.996+0000 7f9916ffd700 1 --2- 172.21.15.163:0/311978103 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9918159420 0x7f99180b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:40.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.996+0000 7f9916ffd700 1 -- 172.21.15.163:0/311978103 >> v1:172.21.15.181:6789/0 conn(0x7f991815d230 legacy=0x7f99180b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:40.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.996+0000 7f9916ffd700 1 -- 172.21.15.163:0/311978103 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99180b1a70 con 0x7f99181597f0 2023-10-09T06:39:40.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.996+0000 7f99177fe700 1 --2- 172.21.15.163:0/311978103 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9918159420 0x7f99180b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:40.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.996+0000 7f9916ffd700 1 --2- 172.21.15.163:0/311978103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99181597f0 0x7f99180b2270 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f990c0099b0 tx=0x7f990c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a12e875b66bf082 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:41.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.997+0000 7f99167fc700 1 -- 172.21.15.163:0/311978103 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c00b860 con 0x7f99181597f0 2023-10-09T06:39:41.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.997+0000 7f99167fc700 1 -- 172.21.15.163:0/311978103 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990c00b9c0 con 0x7f99181597f0 2023-10-09T06:39:41.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.997+0000 7f99167fc700 1 -- 172.21.15.163:0/311978103 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c00bd60 con 0x7f99181597f0 2023-10-09T06:39:41.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.998+0000 7f991cafe700 1 -- 172.21.15.163:0/311978103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99181597f0 msgr2=0x7f99180b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:41.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.998+0000 7f991cafe700 1 --2- 172.21.15.163:0/311978103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99181597f0 0x7f99180b2270 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f990c0099b0 tx=0x7f990c009250 comp rx=0 tx=0).stop 2023-10-09T06:39:41.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.998+0000 7f991cafe700 1 -- 172.21.15.163:0/311978103 shutdown_connections 2023-10-09T06:39:41.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.998+0000 7f991cafe700 1 --2- 172.21.15.163:0/311978103 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9918159420 0x7f99180b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:41.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.998+0000 7f991cafe700 1 --2- 172.21.15.163:0/311978103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99181597f0 0x7f99180b2270 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:41.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.998+0000 7f991cafe700 1 -- 172.21.15.163:0/311978103 >> 172.21.15.163:0/311978103 conn(0x7f99180d0660 msgr2=0x7f99180d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:41.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.998+0000 7f991cafe700 1 -- 172.21.15.163:0/311978103 shutdown_connections 2023-10-09T06:39:41.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.998+0000 7f991cafe700 1 -- 172.21.15.163:0/311978103 wait complete. 2023-10-09T06:39:41.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.999+0000 7f991cafe700 1 Processor -- start 2023-10-09T06:39:41.001 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:40.999+0000 7f991cafe700 1 -- start start 2023-10-09T06:39:41.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.000+0000 7f991cafe700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9918159420 0x7f991815b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:41.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.000+0000 7f991cafe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99181597f0 0x7f991814f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:41.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.000+0000 7f991cafe700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f991815d230 0x7f99180b4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:41.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.000+0000 7f991cafe700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99180b5240 con 0x7f99181597f0 2023-10-09T06:39:41.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.000+0000 7f991cafe700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991815ae90 con 0x7f991815d230 2023-10-09T06:39:41.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.000+0000 7f991cafe700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991815a560 con 0x7f9918159420 2023-10-09T06:39:41.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.000+0000 7f9916ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99181597f0 0x7f991814f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:41.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.000+0000 7f9916ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99181597f0 0x7f991814f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39450/0 (socket says 172.21.15.163:39450) 2023-10-09T06:39:41.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.000+0000 7f9916ffd700 1 -- 172.21.15.163:0/1028767167 learned_addr learned my addr 172.21.15.163:0/1028767167 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:41.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.000+0000 7f99177fe700 1 --2- 172.21.15.163:0/1028767167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9918159420 0x7f991815b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:41.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.000+0000 7f9917fff700 1 --2- 172.21.15.163:0/1028767167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f991815d230 0x7f99180b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:41.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.000+0000 7f9916ffd700 1 -- 172.21.15.163:0/1028767167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9918159420 msgr2=0x7f991815b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:41.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.000+0000 7f9916ffd700 1 --2- 172.21.15.163:0/1028767167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9918159420 0x7f991815b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:41.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.000+0000 7f9916ffd700 1 -- 172.21.15.163:0/1028767167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f991815d230 msgr2=0x7f99180b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:41.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.000+0000 7f9916ffd700 1 --2- 172.21.15.163:0/1028767167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f991815d230 0x7f99180b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:41.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.000+0000 7f9916ffd700 1 -- 172.21.15.163:0/1028767167 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990c0094f0 con 0x7f99181597f0 2023-10-09T06:39:41.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.001+0000 7f9916ffd700 1 --2- 172.21.15.163:0/1028767167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99181597f0 0x7f991814f170 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f990c009290 tx=0x7f990c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:41.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.001+0000 7f98ff7fe700 1 -- 172.21.15.163:0/1028767167 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c011e00 con 0x7f99181597f0 2023-10-09T06:39:41.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.001+0000 7f98ff7fe700 1 -- 172.21.15.163:0/1028767167 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990c01f070 con 0x7f99181597f0 2023-10-09T06:39:41.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.001+0000 7f991cafe700 1 -- 172.21.15.163:0/1028767167 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991815a840 con 0x7f99181597f0 2023-10-09T06:39:41.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.001+0000 7f98ff7fe700 1 -- 172.21.15.163:0/1028767167 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c0266d0 con 0x7f99181597f0 2023-10-09T06:39:41.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.001+0000 7f991cafe700 1 -- 172.21.15.163:0/1028767167 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991814f6b0 con 0x7f99181597f0 2023-10-09T06:39:41.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.002+0000 7f991cafe700 1 -- 172.21.15.163:0/1028767167 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f991804c3c0 con 0x7f99181597f0 2023-10-09T06:39:41.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.002+0000 7f98ff7fe700 1 -- 172.21.15.163:0/1028767167 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f990c026a40 con 0x7f99181597f0 2023-10-09T06:39:41.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.002+0000 7f98ff7fe700 1 --2- 172.21.15.163:0/1028767167 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9904067320 0x7f99040697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:41.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.003+0000 7f98ff7fe700 1 -- 172.21.15.163:0/1028767167 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f990c021070 con 0x7f99181597f0 2023-10-09T06:39:41.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.003+0000 7f99177fe700 1 --2- 172.21.15.163:0/1028767167 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9904067320 0x7f99040697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:41.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.003+0000 7f99177fe700 1 --2- 172.21.15.163:0/1028767167 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9904067320 0x7f99040697d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f9918158da0 tx=0x7f9908008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:41.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.007+0000 7f98ff7fe700 1 -- 172.21.15.163:0/1028767167 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f990c053610 con 0x7f99181597f0 2023-10-09T06:39:41.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.229+0000 7f991cafe700 1 -- 172.21.15.163:0/1028767167 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99180b1d50 con 0x7f99181597f0 2023-10-09T06:39:41.230 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.230+0000 7f98ff7fe700 1 -- 172.21.15.163:0/1028767167 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4630 (secure 0 0 0) 0x7f990c02b090 con 0x7f99181597f0 2023-10-09T06:39:41.231 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:41.231 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":125,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6815,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3960800872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3960800872},{"type":"v1","addr":"172.21.15.181:6836","nonce":3960800872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":8631,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/380361821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":380361821},{"type":"v1","addr":"172.21.15.163:6834","nonce":380361821}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":8634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2847943599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2847943599},{"type":"v1","addr":"172.21.15.163:6835","nonce":2847943599}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:39:22.273633+0000","modified":"2023-10-09T06:39:33.963575+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8649":{"gid":8649,"name":"d","rank":0,"incarnation":119,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/119178917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":119178917},{"type":"v1","addr":"172.21.15.181:6837","nonce":119178917}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"volume_0000000000719180","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-09T06:39:41.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.231+0000 7f991cafe700 1 -- 172.21.15.163:0/1028767167 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9904067320 msgr2=0x7f99040697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:41.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.231+0000 7f991cafe700 1 --2- 172.21.15.163:0/1028767167 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9904067320 0x7f99040697d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f9918158da0 tx=0x7f9908008040 comp rx=0 tx=0).stop 2023-10-09T06:39:41.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.231+0000 7f991cafe700 1 -- 172.21.15.163:0/1028767167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99181597f0 msgr2=0x7f991814f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:41.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.231+0000 7f991cafe700 1 --2- 172.21.15.163:0/1028767167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99181597f0 0x7f991814f170 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f990c009290 tx=0x7f990c017000 comp rx=0 tx=0).stop 2023-10-09T06:39:41.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.232+0000 7f991cafe700 1 -- 172.21.15.163:0/1028767167 shutdown_connections 2023-10-09T06:39:41.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.232+0000 7f991cafe700 1 --2- 172.21.15.163:0/1028767167 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9904067320 0x7f99040697d0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:41.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.232+0000 7f991cafe700 1 --2- 172.21.15.163:0/1028767167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f991815d230 0x7f99180b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:41.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.232+0000 7f991cafe700 1 --2- 172.21.15.163:0/1028767167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9918159420 0x7f991815b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:41.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.232+0000 7f991cafe700 1 --2- 172.21.15.163:0/1028767167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99181597f0 0x7f991814f170 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:41.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.232+0000 7f991cafe700 1 -- 172.21.15.163:0/1028767167 >> 172.21.15.163:0/1028767167 conn(0x7f99180d0660 msgr2=0x7f99180b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:41.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.232+0000 7f991cafe700 1 -- 172.21.15.163:0/1028767167 shutdown_connections 2023-10-09T06:39:41.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.232+0000 7f991cafe700 1 -- 172.21.15.163:0/1028767167 wait complete. 2023-10-09T06:39:41.234 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 125 2023-10-09T06:39:41.245 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:39:41.359 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.358+0000 7f5181d19700 1 Processor -- start 2023-10-09T06:39:41.359 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.358+0000 7f5181d19700 1 -- start start 2023-10-09T06:39:41.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.359+0000 7f5181d19700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f517c06bcd0 0x7f517c0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:41.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.359+0000 7f5181d19700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f517c15df50 0x7f517c15e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:41.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.359+0000 7f5181d19700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f517c15d7e0 con 0x7f517c06c0a0 2023-10-09T06:39:41.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.359+0000 7f5181d19700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f517c15dac0 con 0x7f517c15df50 2023-10-09T06:39:41.361 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.359+0000 7f5181d19700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f517c15aa30 con 0x7f517c06bcd0 2023-10-09T06:39:41.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.359+0000 7f517bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f517c15df50 0x7f517c15e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:41.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.359+0000 7f517bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f517c15df50 0x7f517c15e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39458/0 (socket says 172.21.15.163:39458) 2023-10-09T06:39:41.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.359+0000 7f517bfff700 1 -- 172.21.15.163:0/2531595105 learned_addr learned my addr 172.21.15.163:0/2531595105 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:41.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.359+0000 7f517affd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f517c06c0a0 0x7f517c15a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:52034/0 (socket says 172.21.15.163:52034) 2023-10-09T06:39:41.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.359+0000 7f517bfff700 1 -- 172.21.15.163:0/2531595105 >> v2:172.21.15.181:3300/0 conn(0x7f517c06bcd0 msgr2=0x7f517c0caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:41.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.359+0000 7f517b7fe700 1 --2- 172.21.15.163:0/2531595105 >> v2:172.21.15.181:3300/0 conn(0x7f517c06bcd0 0x7f517c0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:41.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.359+0000 7f517bfff700 1 --2- 172.21.15.163:0/2531595105 >> v2:172.21.15.181:3300/0 conn(0x7f517c06bcd0 0x7f517c0caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:41.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.359+0000 7f517a7fc700 1 -- 172.21.15.163:0/2531595105 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2672361186 0 0) 0x7f517c15d7e0 con 0x7f517c06c0a0 2023-10-09T06:39:41.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.359+0000 7f517bfff700 1 -- 172.21.15.163:0/2531595105 >> v1:172.21.15.163:6789/0 conn(0x7f517c06c0a0 legacy=0x7f517c15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:41.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.359+0000 7f517bfff700 1 -- 172.21.15.163:0/2531595105 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f517c15ad10 con 0x7f517c15df50 2023-10-09T06:39:41.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.359+0000 7f517b7fe700 1 --2- 172.21.15.163:0/2531595105 >> v2:172.21.15.181:3300/0 conn(0x7f517c06bcd0 0x7f517c0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:41.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.359+0000 7f517bfff700 1 --2- 172.21.15.163:0/2531595105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f517c15df50 0x7f517c15e3b0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f517c0691a0 tx=0x7f5174009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1f614a45dd57a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:41.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.359+0000 7f517a7fc700 1 -- 172.21.15.163:0/2531595105 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517400e070 con 0x7f517c15df50 2023-10-09T06:39:41.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.359+0000 7f517a7fc700 1 -- 172.21.15.163:0/2531595105 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517400ba50 con 0x7f517c15df50 2023-10-09T06:39:41.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.359+0000 7f517a7fc700 1 -- 172.21.15.163:0/2531595105 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51740092e0 con 0x7f517c15df50 2023-10-09T06:39:41.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.360+0000 7f5181d19700 1 -- 172.21.15.163:0/2531595105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f517c15df50 msgr2=0x7f517c15e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:41.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.360+0000 7f5181d19700 1 --2- 172.21.15.163:0/2531595105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f517c15df50 0x7f517c15e3b0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f517c0691a0 tx=0x7f5174009840 comp rx=0 tx=0).stop 2023-10-09T06:39:41.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.360+0000 7f5181d19700 1 -- 172.21.15.163:0/2531595105 shutdown_connections 2023-10-09T06:39:41.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.360+0000 7f5181d19700 1 --2- 172.21.15.163:0/2531595105 >> v2:172.21.15.181:3300/0 conn(0x7f517c06bcd0 0x7f517c0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:41.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.360+0000 7f5181d19700 1 --2- 172.21.15.163:0/2531595105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f517c15df50 0x7f517c15e3b0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:41.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.360+0000 7f5181d19700 1 -- 172.21.15.163:0/2531595105 >> 172.21.15.163:0/2531595105 conn(0x7f517c0ce600 msgr2=0x7f517c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:41.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.360+0000 7f5181d19700 1 -- 172.21.15.163:0/2531595105 shutdown_connections 2023-10-09T06:39:41.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.360+0000 7f5181d19700 1 -- 172.21.15.163:0/2531595105 wait complete. 2023-10-09T06:39:41.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.361+0000 7f5181d19700 1 Processor -- start 2023-10-09T06:39:41.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.361+0000 7f5181d19700 1 -- start start 2023-10-09T06:39:41.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.362+0000 7f5181d19700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f517c06bcd0 0x7f517c1549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:41.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.362+0000 7f5181d19700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f517c06c0a0 0x7f517c0c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:41.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.362+0000 7f5181d19700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f517c15df50 0x7f517c156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:41.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.362+0000 7f5181d19700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517c0c59e0 con 0x7f517c06bcd0 2023-10-09T06:39:41.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.362+0000 7f5181d19700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517c0b1b20 con 0x7f517c06c0a0 2023-10-09T06:39:41.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.362+0000 7f5181d19700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f517c16ace0 con 0x7f517c15df50 2023-10-09T06:39:41.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.362+0000 7f517b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f517c06bcd0 0x7f517c1549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:41.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.362+0000 7f517b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f517c06bcd0 0x7f517c1549f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39462/0 (socket says 172.21.15.163:39462) 2023-10-09T06:39:41.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.362+0000 7f517b7fe700 1 -- 172.21.15.163:0/66802805 learned_addr learned my addr 172.21.15.163:0/66802805 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:41.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.362+0000 7f517affd700 1 --2- 172.21.15.163:0/66802805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f517c06c0a0 0x7f517c0c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:41.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.362+0000 7f517bfff700 1 --2- 172.21.15.163:0/66802805 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f517c15df50 0x7f517c156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:41.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.362+0000 7f517b7fe700 1 -- 172.21.15.163:0/66802805 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f517c15df50 msgr2=0x7f517c156530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:41.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.362+0000 7f517b7fe700 1 --2- 172.21.15.163:0/66802805 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f517c15df50 0x7f517c156530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:41.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.362+0000 7f517b7fe700 1 -- 172.21.15.163:0/66802805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f517c06c0a0 msgr2=0x7f517c0c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:41.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.362+0000 7f517b7fe700 1 --2- 172.21.15.163:0/66802805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f517c06c0a0 0x7f517c0c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:41.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.362+0000 7f517b7fe700 1 -- 172.21.15.163:0/66802805 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5174009ae0 con 0x7f517c06bcd0 2023-10-09T06:39:41.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.362+0000 7f517affd700 1 --2- 172.21.15.163:0/66802805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f517c06c0a0 0x7f517c0c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:41.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.362+0000 7f517b7fe700 1 --2- 172.21.15.163:0/66802805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f517c06bcd0 0x7f517c1549f0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f517000c850 tx=0x7f5170008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:41.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.363+0000 7f51637fe700 1 -- 172.21.15.163:0/66802805 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5170011600 con 0x7f517c06bcd0 2023-10-09T06:39:41.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.363+0000 7f51637fe700 1 -- 172.21.15.163:0/66802805 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5170011760 con 0x7f517c06bcd0 2023-10-09T06:39:41.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.363+0000 7f5181d19700 1 -- 172.21.15.163:0/66802805 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f517c16aa00 con 0x7f517c06bcd0 2023-10-09T06:39:41.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.363+0000 7f51637fe700 1 -- 172.21.15.163:0/66802805 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5170011b00 con 0x7f517c06bcd0 2023-10-09T06:39:41.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.363+0000 7f5181d19700 1 -- 172.21.15.163:0/66802805 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517c0b17b0 con 0x7f517c06bcd0 2023-10-09T06:39:41.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.364+0000 7f5181d19700 1 -- 172.21.15.163:0/66802805 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f517c04c3c0 con 0x7f517c06bcd0 2023-10-09T06:39:41.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.364+0000 7f51637fe700 1 -- 172.21.15.163:0/66802805 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f51700073b0 con 0x7f517c06bcd0 2023-10-09T06:39:41.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.364+0000 7f51637fe700 1 --2- 172.21.15.163:0/66802805 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5168067320 0x7f51680697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:41.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.364+0000 7f51637fe700 1 -- 172.21.15.163:0/66802805 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f5170084d60 con 0x7f517c06bcd0 2023-10-09T06:39:41.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.365+0000 7f517affd700 1 --2- 172.21.15.163:0/66802805 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5168067320 0x7f51680697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:41.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.365+0000 7f517affd700 1 --2- 172.21.15.163:0/66802805 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5168067320 0x7f51680697d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f516c007fd0 tx=0x7f516c007e30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:41.369 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.368+0000 7f51637fe700 1 -- 172.21.15.163:0/66802805 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5170055320 con 0x7f517c06bcd0 2023-10-09T06:39:41.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.551+0000 7f5181d19700 1 -- 172.21.15.163:0/66802805 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f517c15aff0 con 0x7f517c06bcd0 2023-10-09T06:39:41.552 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.552+0000 7f51637fe700 1 -- 172.21.15.163:0/66802805 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) v1 ==== 74+0+15490 (secure 0 0 0) 0x7f517001e090 con 0x7f517c06bcd0 2023-10-09T06:39:41.553 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:41.553 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":185,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:39:35.978425+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":31,"pool_name":"cephfs.volume_0000000000719180.meta","create_time":"2023-10-09T06:39:20.612285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000719180"}}},{"pool":32,"pool_name":"cephfs.volume_0000000000719180.data","create_time":"2023-10-09T06:39:21.247510+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000719180"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:39:41.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.554+0000 7f5181d19700 1 -- 172.21.15.163:0/66802805 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5168067320 msgr2=0x7f51680697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:41.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.554+0000 7f5181d19700 1 --2- 172.21.15.163:0/66802805 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5168067320 0x7f51680697d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f516c007fd0 tx=0x7f516c007e30 comp rx=0 tx=0).stop 2023-10-09T06:39:41.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.554+0000 7f5181d19700 1 -- 172.21.15.163:0/66802805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f517c06bcd0 msgr2=0x7f517c1549f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:41.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.554+0000 7f5181d19700 1 --2- 172.21.15.163:0/66802805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f517c06bcd0 0x7f517c1549f0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f517000c850 tx=0x7f5170008040 comp rx=0 tx=0).stop 2023-10-09T06:39:41.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.554+0000 7f5181d19700 1 -- 172.21.15.163:0/66802805 shutdown_connections 2023-10-09T06:39:41.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.554+0000 7f5181d19700 1 --2- 172.21.15.163:0/66802805 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5168067320 0x7f51680697d0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:41.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.554+0000 7f5181d19700 1 --2- 172.21.15.163:0/66802805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f517c06bcd0 0x7f517c1549f0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:41.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.554+0000 7f5181d19700 1 --2- 172.21.15.163:0/66802805 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f517c15df50 0x7f517c156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:41.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.554+0000 7f5181d19700 1 --2- 172.21.15.163:0/66802805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f517c06c0a0 0x7f517c0c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:41.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.554+0000 7f5181d19700 1 -- 172.21.15.163:0/66802805 >> 172.21.15.163:0/66802805 conn(0x7f517c0ce600 msgr2=0x7f517c0c8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:41.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.554+0000 7f5181d19700 1 -- 172.21.15.163:0/66802805 shutdown_connections 2023-10-09T06:39:41.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.554+0000 7f5181d19700 1 -- 172.21.15.163:0/66802805 wait complete. 2023-10-09T06:39:41.567 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail volume_0000000000719180 2023-10-09T06:39:41.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.683+0000 7f976bf75700 1 Processor -- start 2023-10-09T06:39:41.684 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.683+0000 7f976bf75700 1 -- start start 2023-10-09T06:39:41.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.684+0000 7f976bf75700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9764159890 0x7f976415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:41.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.684+0000 7f976bf75700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9764159c60 0x7f976415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:41.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.684+0000 7f976bf75700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f976415a160 con 0x7f976415db10 2023-10-09T06:39:41.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.684+0000 7f976bf75700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97640d13a0 con 0x7f9764159c60 2023-10-09T06:39:41.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.684+0000 7f976bf75700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f976415e140 con 0x7f9764159890 2023-10-09T06:39:41.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.684+0000 7f976a512700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f976415db10 0x7f9764069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:52050/0 (socket says 172.21.15.163:52050) 2023-10-09T06:39:41.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.684+0000 7f976a512700 1 -- 172.21.15.163:0/79951309 learned_addr learned my addr 172.21.15.163:0/79951309 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:41.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.684+0000 7f9769d11700 1 --2- 172.21.15.163:0/79951309 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9764159890 0x7f976415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:41.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.684+0000 7f9769510700 1 --2- 172.21.15.163:0/79951309 >> v2:172.21.15.181:3300/0 conn(0x7f9764159c60 0x7f976415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:41.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.684+0000 7f9768d0f700 1 -- 172.21.15.163:0/79951309 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4159420154 0 0) 0x7f976415a160 con 0x7f976415db10 2023-10-09T06:39:41.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.685+0000 7f9768d0f700 1 -- 172.21.15.163:0/79951309 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9750003570 con 0x7f976415db10 2023-10-09T06:39:41.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.685+0000 7f9769510700 1 -- 172.21.15.163:0/79951309 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9764159890 msgr2=0x7f976415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:41.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.685+0000 7f9769510700 1 --2- 172.21.15.163:0/79951309 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9764159890 0x7f976415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:41.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.685+0000 7f9769510700 1 -- 172.21.15.163:0/79951309 >> v1:172.21.15.163:6789/0 conn(0x7f976415db10 legacy=0x7f9764069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:41.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.685+0000 7f9769510700 1 -- 172.21.15.163:0/79951309 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976415a3d0 con 0x7f9764159c60 2023-10-09T06:39:41.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.685+0000 7f9769510700 1 --2- 172.21.15.163:0/79951309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9764159c60 0x7f976415d5d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f97540098e0 tx=0x7f97540091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0f21a5ba97bc75a server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:41.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.685+0000 7f9768d0f700 1 -- 172.21.15.163:0/79951309 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9754011630 con 0x7f9764159c60 2023-10-09T06:39:41.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.685+0000 7f9768d0f700 1 -- 172.21.15.163:0/79951309 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9754011790 con 0x7f9764159c60 2023-10-09T06:39:41.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.685+0000 7f9768d0f700 1 -- 172.21.15.163:0/79951309 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9754011b10 con 0x7f9764159c60 2023-10-09T06:39:41.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.685+0000 7f976bf75700 1 -- 172.21.15.163:0/79951309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9764159c60 msgr2=0x7f976415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:41.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.686+0000 7f976bf75700 1 --2- 172.21.15.163:0/79951309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9764159c60 0x7f976415d5d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f97540098e0 tx=0x7f97540091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:41.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.686+0000 7f976bf75700 1 -- 172.21.15.163:0/79951309 shutdown_connections 2023-10-09T06:39:41.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.686+0000 7f976bf75700 1 --2- 172.21.15.163:0/79951309 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9764159890 0x7f976415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:41.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.686+0000 7f976bf75700 1 --2- 172.21.15.163:0/79951309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9764159c60 0x7f976415d5d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:41.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.686+0000 7f976bf75700 1 -- 172.21.15.163:0/79951309 >> 172.21.15.163:0/79951309 conn(0x7f9764068e20 msgr2=0x7f97640d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:41.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.686+0000 7f976bf75700 1 -- 172.21.15.163:0/79951309 shutdown_connections 2023-10-09T06:39:41.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.686+0000 7f976bf75700 1 -- 172.21.15.163:0/79951309 wait complete. 2023-10-09T06:39:41.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.687+0000 7f976bf75700 1 Processor -- start 2023-10-09T06:39:41.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.687+0000 7f976bf75700 1 -- start start 2023-10-09T06:39:41.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.687+0000 7f976bf75700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9764159890 0x7f9764169b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:41.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.687+0000 7f976bf75700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9764159c60 0x7f97640b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:41.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.687+0000 7f976bf75700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f976415db10 0x7f976416ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:41.691 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.687+0000 7f976bf75700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97640b36c0 con 0x7f976415db10 2023-10-09T06:39:41.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.687+0000 7f976bf75700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976416a1a0 con 0x7f9764159890 2023-10-09T06:39:41.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.687+0000 7f976bf75700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f976416a480 con 0x7f9764159c60 2023-10-09T06:39:41.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.688+0000 7f976a512700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f976415db10 0x7f976416ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:41.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.688+0000 7f976a512700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f976415db10 0x7f976416ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39474/0 (socket says 172.21.15.163:39474) 2023-10-09T06:39:41.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.688+0000 7f976a512700 1 -- 172.21.15.163:0/3905914245 learned_addr learned my addr 172.21.15.163:0/3905914245 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:41.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.688+0000 7f9769d11700 1 --2- 172.21.15.163:0/3905914245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9764159890 0x7f9764169b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:41.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.688+0000 7f9769510700 1 --2- 172.21.15.163:0/3905914245 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9764159c60 0x7f97640b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:41.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.688+0000 7f976a512700 1 -- 172.21.15.163:0/3905914245 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9764159c60 msgr2=0x7f97640b56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:41.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.688+0000 7f976a512700 1 --2- 172.21.15.163:0/3905914245 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9764159c60 0x7f97640b56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:41.692 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.688+0000 7f976a512700 1 -- 172.21.15.163:0/3905914245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9764159890 msgr2=0x7f9764169b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:41.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.688+0000 7f976a512700 1 --2- 172.21.15.163:0/3905914245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9764159890 0x7f9764169b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:41.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.688+0000 7f976a512700 1 -- 172.21.15.163:0/3905914245 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9754009420 con 0x7f976415db10 2023-10-09T06:39:41.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.688+0000 7f976a512700 1 --2- 172.21.15.163:0/3905914245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f976415db10 0x7f976416ad20 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f975c00cb90 tx=0x7f975c0019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:41.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.688+0000 7f9759ffb700 1 -- 172.21.15.163:0/3905914245 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c00a7e0 con 0x7f976415db10 2023-10-09T06:39:41.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.688+0000 7f9759ffb700 1 -- 172.21.15.163:0/3905914245 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975c00a980 con 0x7f976415db10 2023-10-09T06:39:41.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.688+0000 7f976bf75700 1 -- 172.21.15.163:0/3905914245 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f976416a760 con 0x7f976415db10 2023-10-09T06:39:41.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.688+0000 7f9759ffb700 1 -- 172.21.15.163:0/3905914245 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c00ad40 con 0x7f976415db10 2023-10-09T06:39:41.693 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.688+0000 7f976bf75700 1 -- 172.21.15.163:0/3905914245 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97640b5260 con 0x7f976415db10 2023-10-09T06:39:41.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.689+0000 7f976bf75700 1 -- 172.21.15.163:0/3905914245 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97640b3830 con 0x7f976415db10 2023-10-09T06:39:41.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.690+0000 7f9759ffb700 1 -- 172.21.15.163:0/3905914245 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f975c007300 con 0x7f976415db10 2023-10-09T06:39:41.694 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.690+0000 7f9759ffb700 1 --2- 172.21.15.163:0/3905914245 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9748067320 0x7f97480697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:41.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.690+0000 7f9759ffb700 1 -- 172.21.15.163:0/3905914245 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f975c084fb0 con 0x7f976415db10 2023-10-09T06:39:41.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.690+0000 7f9769d11700 1 --2- 172.21.15.163:0/3905914245 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9748067320 0x7f97480697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:41.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.691+0000 7f9769d11700 1 --2- 172.21.15.163:0/3905914245 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9748067320 0x7f97480697d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f97640cf120 tx=0x7f97600091a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:41.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.694+0000 7f9759ffb700 1 -- 172.21.15.163:0/3905914245 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f975c051c70 con 0x7f976415db10 2023-10-09T06:39:41.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.921+0000 7f976bf75700 1 -- 172.21.15.163:0/3905914245 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "volume_0000000000719180"} v 0) v1 -- 0x7f976404c3c0 con 0x7f976415db10 2023-10-09T06:39:41.922 INFO:tasks.ceph.mon.a.smithi163.stderr:2023-10-09T06:39:41.922+0000 7ff251c04700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-09T06:39:41.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.925+0000 7f9759ffb700 1 -- 172.21.15.163:0/3905914245 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "volume_0000000000719180"}]=0 volume_0000000000719180 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v126) v1 ==== 195+0+0 (secure 0 0 0) 0x7f975c055290 con 0x7f976415db10 2023-10-09T06:39:41.928 INFO:tasks.ceph.mds.d.smithi181.stderr: -763> 2023-10-09T06:39:28.199+0000 7fd4a76fb700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-09T06:39:41.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.928+0000 7f976bf75700 1 -- 172.21.15.163:0/3905914245 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9748067320 msgr2=0x7f97480697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:41.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.928+0000 7f976bf75700 1 --2- 172.21.15.163:0/3905914245 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9748067320 0x7f97480697d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f97640cf120 tx=0x7f97600091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:41.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.928+0000 7f976bf75700 1 -- 172.21.15.163:0/3905914245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f976415db10 msgr2=0x7f976416ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:41.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.928+0000 7f976bf75700 1 --2- 172.21.15.163:0/3905914245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f976415db10 0x7f976416ad20 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f975c00cb90 tx=0x7f975c0019c0 comp rx=0 tx=0).stop 2023-10-09T06:39:41.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.928+0000 7f976bf75700 1 -- 172.21.15.163:0/3905914245 shutdown_connections 2023-10-09T06:39:41.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.928+0000 7f976bf75700 1 --2- 172.21.15.163:0/3905914245 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f9748067320 0x7f97480697d0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:41.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.928+0000 7f976bf75700 1 --2- 172.21.15.163:0/3905914245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f976415db10 0x7f976416ad20 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:41.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.928+0000 7f976bf75700 1 --2- 172.21.15.163:0/3905914245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9764159890 0x7f9764169b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:41.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.928+0000 7f976bf75700 1 --2- 172.21.15.163:0/3905914245 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9764159c60 0x7f97640b56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:41.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.928+0000 7f976bf75700 1 -- 172.21.15.163:0/3905914245 >> 172.21.15.163:0/3905914245 conn(0x7f9764068e20 msgr2=0x7f9764153300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:41.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.928+0000 7f976bf75700 1 -- 172.21.15.163:0/3905914245 shutdown_connections 2023-10-09T06:39:41.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:41.928+0000 7f976bf75700 1 -- 172.21.15.163:0/3905914245 wait complete. 2023-10-09T06:39:41.930 INFO:teuthology.orchestra.run.smithi163.stderr:volume_0000000000719180 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-09T06:39:41.941 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm volume_0000000000719180 --yes-i-really-mean-it 2023-10-09T06:39:41.950 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-09T06:39:42.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.059+0000 7f85c354d700 1 Processor -- start 2023-10-09T06:39:42.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.059+0000 7f85c354d700 1 -- start start 2023-10-09T06:39:42.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.060+0000 7f85c354d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc159c60 0x7f85bc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:42.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.060+0000 7f85c354d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f85bc15db10 0x7f85bc069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:42.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.060+0000 7f85c354d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85bc15a160 con 0x7f85bc159890 2023-10-09T06:39:42.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.060+0000 7f85c354d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85bc0d13a0 con 0x7f85bc15db10 2023-10-09T06:39:42.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.060+0000 7f85c354d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc15e140 con 0x7f85bc159c60 2023-10-09T06:39:42.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.060+0000 7f85c1aea700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f85bc15db10 0x7f85bc069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:42.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.060+0000 7f85c0ae8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc159c60 0x7f85bc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:42.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.060+0000 7f85c1aea700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f85bc15db10 0x7f85bc069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:57736/0 (socket says 172.21.15.163:57736) 2023-10-09T06:39:42.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.060+0000 7f85c0ae8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc159c60 0x7f85bc15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:45626/0 (socket says 172.21.15.163:45626) 2023-10-09T06:39:42.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.060+0000 7f85c12e9700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f85bc159890 0x7f85bc15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:57980/0 (socket says 172.21.15.163:57980) 2023-10-09T06:39:42.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.060+0000 7f85c1aea700 1 -- 172.21.15.163:0/899087412 learned_addr learned my addr 172.21.15.163:0/899087412 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:42.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.060+0000 7f85c0ae8700 1 -- 172.21.15.163:0/899087412 >> v2:172.21.15.181:3300/0 conn(0x7f85bc15db10 msgr2=0x7f85bc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:42.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.060+0000 7f85c0ae8700 1 --2- 172.21.15.163:0/899087412 >> v2:172.21.15.181:3300/0 conn(0x7f85bc15db10 0x7f85bc069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:42.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.060+0000 7f85c0ae8700 1 -- 172.21.15.163:0/899087412 >> v1:172.21.15.181:6789/0 conn(0x7f85bc159890 legacy=0x7f85bc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:42.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.060+0000 7f85c0ae8700 1 -- 172.21.15.163:0/899087412 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85bc15a3d0 con 0x7f85bc159c60 2023-10-09T06:39:42.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.060+0000 7f85b3fff700 1 -- 172.21.15.163:0/899087412 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 779948490 0 0) 0x7f85bc15a160 con 0x7f85bc159890 2023-10-09T06:39:42.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.060+0000 7f85c1aea700 1 --2- 172.21.15.163:0/899087412 >> v2:172.21.15.181:3300/0 conn(0x7f85bc15db10 0x7f85bc069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:39:42.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.061+0000 7f85c0ae8700 1 --2- 172.21.15.163:0/899087412 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc159c60 0x7f85bc15d5d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f85ac0098e0 tx=0x7f85ac0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e751d134bbb81616 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:42.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.061+0000 7f85b3fff700 1 -- 172.21.15.163:0/899087412 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac00b640 con 0x7f85bc159c60 2023-10-09T06:39:42.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.061+0000 7f85b3fff700 1 -- 172.21.15.163:0/899087412 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ac00b7a0 con 0x7f85bc159c60 2023-10-09T06:39:42.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.061+0000 7f85b3fff700 1 -- 172.21.15.163:0/899087412 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac00bb40 con 0x7f85bc159c60 2023-10-09T06:39:42.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.061+0000 7f85c354d700 1 -- 172.21.15.163:0/899087412 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc159c60 msgr2=0x7f85bc15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:42.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.061+0000 7f85c354d700 1 --2- 172.21.15.163:0/899087412 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc159c60 0x7f85bc15d5d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f85ac0098e0 tx=0x7f85ac0091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:42.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.061+0000 7f85c354d700 1 -- 172.21.15.163:0/899087412 shutdown_connections 2023-10-09T06:39:42.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.061+0000 7f85c354d700 1 --2- 172.21.15.163:0/899087412 >> v2:172.21.15.181:3300/0 conn(0x7f85bc15db10 0x7f85bc069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:42.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.061+0000 7f85c354d700 1 --2- 172.21.15.163:0/899087412 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc159c60 0x7f85bc15d5d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:42.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.061+0000 7f85c354d700 1 -- 172.21.15.163:0/899087412 >> 172.21.15.163:0/899087412 conn(0x7f85bc068e20 msgr2=0x7f85bc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:42.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.061+0000 7f85c354d700 1 -- 172.21.15.163:0/899087412 shutdown_connections 2023-10-09T06:39:42.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.061+0000 7f85c354d700 1 -- 172.21.15.163:0/899087412 wait complete. 2023-10-09T06:39:42.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.062+0000 7f85c354d700 1 Processor -- start 2023-10-09T06:39:42.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.062+0000 7f85c354d700 1 -- start start 2023-10-09T06:39:42.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.063+0000 7f85c354d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85bc159890 0x7f85bc0cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:42.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.063+0000 7f85c354d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc159c60 0x7f85bc169a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:42.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.063+0000 7f85c354d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85bc15db10 0x7f85bc151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:42.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.063+0000 7f85c354d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc0b1fc0 con 0x7f85bc15db10 2023-10-09T06:39:42.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.063+0000 7f85c354d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc0b0e40 con 0x7f85bc159890 2023-10-09T06:39:42.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.063+0000 7f85c354d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc0b2210 con 0x7f85bc159c60 2023-10-09T06:39:42.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.063+0000 7f85c1aea700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85bc15db10 0x7f85bc151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:42.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.063+0000 7f85c12e9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85bc159890 0x7f85bc0cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:42.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.063+0000 7f85c12e9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85bc159890 0x7f85bc0cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:57752/0 (socket says 172.21.15.163:57752) 2023-10-09T06:39:42.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.063+0000 7f85c1aea700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85bc15db10 0x7f85bc151330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39480/0 (socket says 172.21.15.163:39480) 2023-10-09T06:39:42.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.063+0000 7f85c12e9700 1 -- 172.21.15.163:0/2189275002 learned_addr learned my addr 172.21.15.163:0/2189275002 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:42.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.064+0000 7f85c0ae8700 1 --2- 172.21.15.163:0/2189275002 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc159c60 0x7f85bc169a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:42.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.064+0000 7f85c1aea700 1 -- 172.21.15.163:0/2189275002 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc159c60 msgr2=0x7f85bc169a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:42.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.064+0000 7f85c1aea700 1 --2- 172.21.15.163:0/2189275002 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc159c60 0x7f85bc169a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:42.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.064+0000 7f85c1aea700 1 -- 172.21.15.163:0/2189275002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85bc159890 msgr2=0x7f85bc0cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:42.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.064+0000 7f85c1aea700 1 --2- 172.21.15.163:0/2189275002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85bc159890 0x7f85bc0cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:42.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.064+0000 7f85c1aea700 1 -- 172.21.15.163:0/2189275002 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b4009ae0 con 0x7f85bc15db10 2023-10-09T06:39:42.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.064+0000 7f85c12e9700 1 --2- 172.21.15.163:0/2189275002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85bc159890 0x7f85bc0cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:39:42.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.064+0000 7f85c1aea700 1 --2- 172.21.15.163:0/2189275002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85bc15db10 0x7f85bc151330 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f85b4009e70 tx=0x7f85b400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:42.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.064+0000 7f85b17fa700 1 -- 172.21.15.163:0/2189275002 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b4018070 con 0x7f85bc15db10 2023-10-09T06:39:42.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.064+0000 7f85b17fa700 1 -- 172.21.15.163:0/2189275002 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b400f040 con 0x7f85bc15db10 2023-10-09T06:39:42.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.064+0000 7f85b17fa700 1 -- 172.21.15.163:0/2189275002 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b400cb00 con 0x7f85bc15db10 2023-10-09T06:39:42.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.064+0000 7f85c354d700 1 -- 172.21.15.163:0/2189275002 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85ac009420 con 0x7f85bc15db10 2023-10-09T06:39:42.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.064+0000 7f85c354d700 1 -- 172.21.15.163:0/2189275002 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85bc150850 con 0x7f85bc15db10 2023-10-09T06:39:42.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.065+0000 7f85c354d700 1 -- 172.21.15.163:0/2189275002 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85bc15aec0 con 0x7f85bc15db10 2023-10-09T06:39:42.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.066+0000 7f85b17fa700 1 -- 172.21.15.163:0/2189275002 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f85b4003020 con 0x7f85bc15db10 2023-10-09T06:39:42.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.066+0000 7f85b17fa700 1 --2- 172.21.15.163:0/2189275002 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f85a0067320 0x7f85a00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:42.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.066+0000 7f85b17fa700 1 -- 172.21.15.163:0/2189275002 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5869+0+0 (secure 0 0 0) 0x7f85b4085110 con 0x7f85bc15db10 2023-10-09T06:39:42.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.066+0000 7f85c12e9700 1 --2- 172.21.15.163:0/2189275002 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f85a0067320 0x7f85a00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:42.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.067+0000 7f85c12e9700 1 --2- 172.21.15.163:0/2189275002 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f85a0067320 0x7f85a00697d0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f85b8007fd0 tx=0x7f85b8007e10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:42.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.070+0000 7f85b17fa700 1 -- 172.21.15.163:0/2189275002 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f85b4051bf0 con 0x7f85bc15db10 2023-10-09T06:39:42.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.296+0000 7f85c354d700 1 -- 172.21.15.163:0/2189275002 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "volume_0000000000719180", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f85bc04c3c0 con 0x7f85bc15db10 2023-10-09T06:39:42.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.926+0000 7f85b17fa700 1 -- 172.21.15.163:0/2189275002 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "volume_0000000000719180", "yes_i_really_mean_it": true}]=0 v127) v1 ==== 121+0+0 (secure 0 0 0) 0x7f85b4055210 con 0x7f85bc15db10 2023-10-09T06:39:42.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.928+0000 7f85c354d700 1 -- 172.21.15.163:0/2189275002 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f85a0067320 msgr2=0x7f85a00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:42.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.928+0000 7f85c354d700 1 --2- 172.21.15.163:0/2189275002 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f85a0067320 0x7f85a00697d0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f85b8007fd0 tx=0x7f85b8007e10 comp rx=0 tx=0).stop 2023-10-09T06:39:42.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.928+0000 7f85c354d700 1 -- 172.21.15.163:0/2189275002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85bc15db10 msgr2=0x7f85bc151330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:42.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.928+0000 7f85c354d700 1 --2- 172.21.15.163:0/2189275002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85bc15db10 0x7f85bc151330 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f85b4009e70 tx=0x7f85b400b040 comp rx=0 tx=0).stop 2023-10-09T06:39:42.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.929+0000 7f85c354d700 1 -- 172.21.15.163:0/2189275002 shutdown_connections 2023-10-09T06:39:42.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.929+0000 7f85c354d700 1 --2- 172.21.15.163:0/2189275002 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f85a0067320 0x7f85a00697d0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:42.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.929+0000 7f85c354d700 1 --2- 172.21.15.163:0/2189275002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85bc15db10 0x7f85bc151330 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:42.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.929+0000 7f85c354d700 1 --2- 172.21.15.163:0/2189275002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85bc159890 0x7f85bc0cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:42.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.929+0000 7f85c354d700 1 --2- 172.21.15.163:0/2189275002 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85bc159c60 0x7f85bc169a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:42.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.929+0000 7f85c354d700 1 -- 172.21.15.163:0/2189275002 >> 172.21.15.163:0/2189275002 conn(0x7f85bc068e20 msgr2=0x7f85bc158610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:42.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.929+0000 7f85c354d700 1 -- 172.21.15.163:0/2189275002 shutdown_connections 2023-10-09T06:39:42.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:42.929+0000 7f85c354d700 1 -- 172.21.15.163:0/2189275002 wait complete. 2023-10-09T06:39:42.943 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000719180.meta cephfs.volume_0000000000719180.meta --yes-i-really-really-mean-it 2023-10-09T06:39:43.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.063+0000 7ff181ce8700 1 Processor -- start 2023-10-09T06:39:43.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.063+0000 7ff181ce8700 1 -- start start 2023-10-09T06:39:43.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.064+0000 7ff181ce8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff17c06d840 0x7ff17c164b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:43.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.064+0000 7ff181ce8700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff17c0b4570 con 0x7ff17c154320 2023-10-09T06:39:43.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.064+0000 7ff181ce8700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff17c0b2710 con 0x7ff17c153f50 2023-10-09T06:39:43.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.064+0000 7ff181ce8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c15a730 con 0x7ff17c06d840 2023-10-09T06:39:43.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.064+0000 7ff17affd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff17c154320 0x7ff17c06d130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:52052/0 (socket says 172.21.15.163:52052) 2023-10-09T06:39:43.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.064+0000 7ff17affd700 1 -- 172.21.15.163:0/2998876899 learned_addr learned my addr 172.21.15.163:0/2998876899 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:43.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.065+0000 7ff17bfff700 1 --2- 172.21.15.163:0/2998876899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff17c06d840 0x7ff17c164b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:43.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.065+0000 7ff17a7fc700 1 -- 172.21.15.163:0/2998876899 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1201113467 0 0) 0x7ff17c0b4570 con 0x7ff17c154320 2023-10-09T06:39:43.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.065+0000 7ff17a7fc700 1 -- 172.21.15.163:0/2998876899 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff164003570 con 0x7ff17c154320 2023-10-09T06:39:43.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.065+0000 7ff17a7fc700 1 -- 172.21.15.163:0/2998876899 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 78992541 0 0) 0x7ff17c0b2710 con 0x7ff17c153f50 2023-10-09T06:39:43.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.065+0000 7ff17a7fc700 1 -- 172.21.15.163:0/2998876899 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff17c0b4570 con 0x7ff17c153f50 2023-10-09T06:39:43.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.065+0000 7ff17a7fc700 1 -- 172.21.15.163:0/2998876899 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2810868060 0 0) 0x7ff164003570 con 0x7ff17c154320 2023-10-09T06:39:43.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.065+0000 7ff17a7fc700 1 -- 172.21.15.163:0/2998876899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff17c06d840 msgr2=0x7ff17c164b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:43.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.065+0000 7ff17a7fc700 1 --2- 172.21.15.163:0/2998876899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff17c06d840 0x7ff17c164b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:43.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.065+0000 7ff17a7fc700 1 -- 172.21.15.163:0/2998876899 >> v1:172.21.15.181:6789/0 conn(0x7ff17c153f50 legacy=0x7ff17c0b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:43.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.065+0000 7ff17a7fc700 1 -- 172.21.15.163:0/2998876899 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17c15aa10 con 0x7ff17c154320 2023-10-09T06:39:43.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.065+0000 7ff17a7fc700 1 -- 172.21.15.163:0/2998876899 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7ff170001470 con 0x7ff17c154320 2023-10-09T06:39:43.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.065+0000 7ff17a7fc700 1 -- 172.21.15.163:0/2998876899 >> v1:172.21.15.163:6789/0 conn(0x7ff17c154320 legacy=0x7ff17c06d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:43.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.065+0000 7ff17a7fc700 1 --2- 172.21.15.163:0/2998876899 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff164003c20 0x7ff164004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:43.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.065+0000 7ff17a7fc700 1 --2- 172.21.15.163:0/2998876899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff164004810 0x7ff164004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:43.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.065+0000 7ff17a7fc700 1 --2- 172.21.15.163:0/2998876899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1640051c0 0x7ff16400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:43.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.065+0000 7ff17a7fc700 1 -- 172.21.15.163:0/2998876899 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c15a730 con 0x7ff164003c20 2023-10-09T06:39:43.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.065+0000 7ff17a7fc700 1 -- 172.21.15.163:0/2998876899 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16400c080 con 0x7ff1640051c0 2023-10-09T06:39:43.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.065+0000 7ff17a7fc700 1 -- 172.21.15.163:0/2998876899 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff16400c240 con 0x7ff164004810 2023-10-09T06:39:43.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.065+0000 7ff17bfff700 1 --2- 172.21.15.163:0/2998876899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff17c06d840 0x7ff17c164b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:39:43.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.065+0000 7ff17b7fe700 1 --2- 172.21.15.163:0/2998876899 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff164003c20 0x7ff164004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:43.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.065+0000 7ff17affd700 1 --2- 172.21.15.163:0/2998876899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff164004810 0x7ff164004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:43.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.065+0000 7ff17bfff700 1 --2- 172.21.15.163:0/2998876899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1640051c0 0x7ff16400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:43.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.065+0000 7ff17b7fe700 1 -- 172.21.15.163:0/2998876899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff164004810 msgr2=0x7ff164004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:43.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.065+0000 7ff17b7fe700 1 --2- 172.21.15.163:0/2998876899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff164004810 0x7ff164004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:43.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.065+0000 7ff17b7fe700 1 -- 172.21.15.163:0/2998876899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1640051c0 msgr2=0x7ff16400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:43.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.065+0000 7ff17b7fe700 1 --2- 172.21.15.163:0/2998876899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1640051c0 0x7ff16400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:43.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.065+0000 7ff17b7fe700 1 -- 172.21.15.163:0/2998876899 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff164003570 con 0x7ff164003c20 2023-10-09T06:39:43.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.066+0000 7ff17b7fe700 1 --2- 172.21.15.163:0/2998876899 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff164003c20 0x7ff164004010 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7ff16c002110 tx=0x7ff16c0017c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc6f6b207c09675f server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:43.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.066+0000 7ff17a7fc700 1 -- 172.21.15.163:0/2998876899 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c001ad0 con 0x7ff164003c20 2023-10-09T06:39:43.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.066+0000 7ff17a7fc700 1 -- 172.21.15.163:0/2998876899 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16c00c980 con 0x7ff164003c20 2023-10-09T06:39:43.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.066+0000 7ff17a7fc700 1 -- 172.21.15.163:0/2998876899 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c00cd40 con 0x7ff164003c20 2023-10-09T06:39:43.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.066+0000 7ff181ce8700 1 -- 172.21.15.163:0/2998876899 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff164003c20 msgr2=0x7ff164004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:43.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.066+0000 7ff181ce8700 1 --2- 172.21.15.163:0/2998876899 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff164003c20 0x7ff164004010 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7ff16c002110 tx=0x7ff16c0017c0 comp rx=0 tx=0).stop 2023-10-09T06:39:43.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.066+0000 7ff17bfff700 1 -- 172.21.15.163:0/2998876899 reap_dead start 2023-10-09T06:39:43.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.066+0000 7ff181ce8700 1 -- 172.21.15.163:0/2998876899 shutdown_connections 2023-10-09T06:39:43.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.066+0000 7ff181ce8700 1 -- 172.21.15.163:0/2998876899 >> 172.21.15.163:0/2998876899 conn(0x7ff17c06baf0 msgr2=0x7ff17c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:43.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.066+0000 7ff181ce8700 1 -- 172.21.15.163:0/2998876899 shutdown_connections 2023-10-09T06:39:43.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.066+0000 7ff181ce8700 1 -- 172.21.15.163:0/2998876899 wait complete. 2023-10-09T06:39:43.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.068+0000 7ff181ce8700 1 Processor -- start 2023-10-09T06:39:43.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.068+0000 7ff181ce8700 1 -- start start 2023-10-09T06:39:43.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.068+0000 7ff181ce8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff164003c20 0x7ff17c151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:43.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.068+0000 7ff181ce8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff17c169fb0 0x7ff17c16a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:43.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.068+0000 7ff181ce8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff17c0c9960 0x7ff17c0c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:43.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.068+0000 7ff181ce8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c0ca320 con 0x7ff164003c20 2023-10-09T06:39:43.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.068+0000 7ff181ce8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c158160 con 0x7ff17c0c9960 2023-10-09T06:39:43.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.068+0000 7ff181ce8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c157b50 con 0x7ff17c169fb0 2023-10-09T06:39:43.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.068+0000 7ff17b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff164003c20 0x7ff17c151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:43.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.068+0000 7ff17b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff164003c20 0x7ff17c151870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39510/0 (socket says 172.21.15.163:39510) 2023-10-09T06:39:43.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.068+0000 7ff17b7fe700 1 -- 172.21.15.163:0/3625669387 learned_addr learned my addr 172.21.15.163:0/3625669387 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:43.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.069+0000 7ff17bfff700 1 --2- 172.21.15.163:0/3625669387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff17c0c9960 0x7ff17c0c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:43.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.069+0000 7ff17affd700 1 --2- 172.21.15.163:0/3625669387 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff17c169fb0 0x7ff17c16a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:43.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.069+0000 7ff17b7fe700 1 -- 172.21.15.163:0/3625669387 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff17c169fb0 msgr2=0x7ff17c16a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:43.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.069+0000 7ff17b7fe700 1 --2- 172.21.15.163:0/3625669387 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff17c169fb0 0x7ff17c16a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:43.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.069+0000 7ff17b7fe700 1 -- 172.21.15.163:0/3625669387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff17c0c9960 msgr2=0x7ff17c0c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:43.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.069+0000 7ff17b7fe700 1 --2- 172.21.15.163:0/3625669387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff17c0c9960 0x7ff17c0c9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:43.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.069+0000 7ff17b7fe700 1 -- 172.21.15.163:0/3625669387 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17c16aaa0 con 0x7ff164003c20 2023-10-09T06:39:43.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.069+0000 7ff17b7fe700 1 --2- 172.21.15.163:0/3625669387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff164003c20 0x7ff17c151870 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7ff16c012ce0 tx=0x7ff16c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:43.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.069+0000 7ff1637fe700 1 -- 172.21.15.163:0/3625669387 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c001ad0 con 0x7ff164003c20 2023-10-09T06:39:43.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.069+0000 7ff1637fe700 1 -- 172.21.15.163:0/3625669387 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16c01f650 con 0x7ff164003c20 2023-10-09T06:39:43.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.069+0000 7ff181ce8700 1 -- 172.21.15.163:0/3625669387 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff17c16ad80 con 0x7ff164003c20 2023-10-09T06:39:43.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.069+0000 7ff1637fe700 1 -- 172.21.15.163:0/3625669387 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c01f800 con 0x7ff164003c20 2023-10-09T06:39:43.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.069+0000 7ff181ce8700 1 -- 172.21.15.163:0/3625669387 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff17c0b0960 con 0x7ff164003c20 2023-10-09T06:39:43.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.070+0000 7ff181ce8700 1 -- 172.21.15.163:0/3625669387 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff17c157fb0 con 0x7ff164003c20 2023-10-09T06:39:43.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.070+0000 7ff1637fe700 1 -- 172.21.15.163:0/3625669387 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff16c025020 con 0x7ff164003c20 2023-10-09T06:39:43.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.071+0000 7ff1637fe700 1 --2- 172.21.15.163:0/3625669387 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff168067320 0x7ff1680697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:43.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.071+0000 7ff1637fe700 1 -- 172.21.15.163:0/3625669387 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5869+0+0 (secure 0 0 0) 0x7ff16c086d00 con 0x7ff164003c20 2023-10-09T06:39:43.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.071+0000 7ff17affd700 1 --2- 172.21.15.163:0/3625669387 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff168067320 0x7ff1680697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:43.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.072+0000 7ff17affd700 1 --2- 172.21.15.163:0/3625669387 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff168067320 0x7ff1680697d0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff170001200 tx=0x7ff170009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:43.078 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.075+0000 7ff1637fe700 1 -- 172.21.15.163:0/3625669387 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff16c053a00 con 0x7ff164003c20 2023-10-09T06:39:43.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:43.256+0000 7ff181ce8700 1 -- 172.21.15.163:0/3625669387 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000719180.meta", "pool2": "cephfs.volume_0000000000719180.meta", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff17c06d660 con 0x7ff164003c20 2023-10-09T06:39:44.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.021+0000 7ff1637fe700 1 -- 172.21.15.163:0/3625669387 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000719180.meta", "pool2": "cephfs.volume_0000000000719180.meta", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000719180.meta' removed v187) v1 ==== 241+0+0 (secure 0 0 0) 0x7ff16c02d020 con 0x7ff164003c20 2023-10-09T06:39:44.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.024+0000 7ff181ce8700 1 -- 172.21.15.163:0/3625669387 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff168067320 msgr2=0x7ff1680697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:44.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.024+0000 7ff181ce8700 1 --2- 172.21.15.163:0/3625669387 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff168067320 0x7ff1680697d0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff170001200 tx=0x7ff170009040 comp rx=0 tx=0).stop 2023-10-09T06:39:44.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.024+0000 7ff181ce8700 1 -- 172.21.15.163:0/3625669387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff164003c20 msgr2=0x7ff17c151870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:44.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.024+0000 7ff181ce8700 1 --2- 172.21.15.163:0/3625669387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff164003c20 0x7ff17c151870 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7ff16c012ce0 tx=0x7ff16c016000 comp rx=0 tx=0).stop 2023-10-09T06:39:44.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.025+0000 7ff181ce8700 1 -- 172.21.15.163:0/3625669387 shutdown_connections 2023-10-09T06:39:44.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.025+0000 7ff181ce8700 1 --2- 172.21.15.163:0/3625669387 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff168067320 0x7ff1680697d0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:44.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.025+0000 7ff181ce8700 1 --2- 172.21.15.163:0/3625669387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff164003c20 0x7ff17c151870 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:44.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.025+0000 7ff181ce8700 1 --2- 172.21.15.163:0/3625669387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff17c0c9960 0x7ff17c0c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:44.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.025+0000 7ff181ce8700 1 --2- 172.21.15.163:0/3625669387 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff17c169fb0 0x7ff17c16a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:44.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.025+0000 7ff181ce8700 1 -- 172.21.15.163:0/3625669387 >> 172.21.15.163:0/3625669387 conn(0x7ff17c06baf0 msgr2=0x7ff17c1534f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:44.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.025+0000 7ff181ce8700 1 -- 172.21.15.163:0/3625669387 shutdown_connections 2023-10-09T06:39:44.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.025+0000 7ff181ce8700 1 -- 172.21.15.163:0/3625669387 wait complete. 2023-10-09T06:39:44.027 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs.volume_0000000000719180.meta' removed 2023-10-09T06:39:44.040 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000719180.data cephfs.volume_0000000000719180.data --yes-i-really-really-mean-it 2023-10-09T06:39:44.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.173+0000 7f43df7aa700 1 Processor -- start 2023-10-09T06:39:44.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.173+0000 7f43df7aa700 1 -- start start 2023-10-09T06:39:44.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.174+0000 7f43df7aa700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f43d8159ec0 0x7f43d80b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:44.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.174+0000 7f43df7aa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43d80b53d0 0x7f43d80cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:44.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.174+0000 7f43df7aa700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d80b5b00 con 0x7f43d80cce40 2023-10-09T06:39:44.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.174+0000 7f43df7aa700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43d80b5850 con 0x7f43d8159ec0 2023-10-09T06:39:44.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.174+0000 7f43df7aa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d80b5da0 con 0x7f43d80b53d0 2023-10-09T06:39:44.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.174+0000 7f43ddd47700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f43d80cce40 0x7f43d80cd2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:52064/0 (socket says 172.21.15.163:52064) 2023-10-09T06:39:44.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.174+0000 7f43ddd47700 1 -- 172.21.15.163:0/3217222548 learned_addr learned my addr 172.21.15.163:0/3217222548 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:44.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.174+0000 7f43cffff700 1 -- 172.21.15.163:0/3217222548 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3668066459 0 0) 0x7f43d80b5b00 con 0x7f43d80cce40 2023-10-09T06:39:44.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.174+0000 7f43dcd45700 1 --2- 172.21.15.163:0/3217222548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43d80b53d0 0x7f43d80cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:44.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.174+0000 7f43cffff700 1 -- 172.21.15.163:0/3217222548 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43c4003570 con 0x7f43d80cce40 2023-10-09T06:39:44.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.174+0000 7f43dd546700 1 --2- 172.21.15.163:0/3217222548 >> v2:172.21.15.181:3300/0 conn(0x7f43d8159ec0 0x7f43d80b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:44.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.174+0000 7f43cffff700 1 -- 172.21.15.163:0/3217222548 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2930043823 0 0) 0x7f43c4003570 con 0x7f43d80cce40 2023-10-09T06:39:44.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.174+0000 7f43cffff700 1 -- 172.21.15.163:0/3217222548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43d80b53d0 msgr2=0x7f43d80cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:44.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.174+0000 7f43cffff700 1 --2- 172.21.15.163:0/3217222548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43d80b53d0 0x7f43d80cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:44.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.174+0000 7f43cffff700 1 -- 172.21.15.163:0/3217222548 >> v2:172.21.15.181:3300/0 conn(0x7f43d8159ec0 msgr2=0x7f43d80b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:44.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.174+0000 7f43cffff700 1 --2- 172.21.15.163:0/3217222548 >> v2:172.21.15.181:3300/0 conn(0x7f43d8159ec0 0x7f43d80b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:44.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.174+0000 7f43cffff700 1 -- 172.21.15.163:0/3217222548 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d80b6080 con 0x7f43d80cce40 2023-10-09T06:39:44.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.174+0000 7f43cffff700 1 -- 172.21.15.163:0/3217222548 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f43d40017f0 con 0x7f43d80cce40 2023-10-09T06:39:44.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.174+0000 7f43cffff700 1 -- 172.21.15.163:0/3217222548 >> v1:172.21.15.163:6789/0 conn(0x7f43d80cce40 legacy=0x7f43d80cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:44.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.174+0000 7f43cffff700 1 --2- 172.21.15.163:0/3217222548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43d80b53d0 0x7f43c4003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:44.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.175+0000 7f43cffff700 1 --2- 172.21.15.163:0/3217222548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43c4003fe0 0x7f43c4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:44.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.175+0000 7f43cffff700 1 --2- 172.21.15.163:0/3217222548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43c4004d50 0x7f43c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:44.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.175+0000 7f43cffff700 1 -- 172.21.15.163:0/3217222548 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d80b5850 con 0x7f43c4003fe0 2023-10-09T06:39:44.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.175+0000 7f43cffff700 1 -- 172.21.15.163:0/3217222548 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d80b5da0 con 0x7f43c4004d50 2023-10-09T06:39:44.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.175+0000 7f43cffff700 1 -- 172.21.15.163:0/3217222548 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c400bb40 con 0x7f43d80b53d0 2023-10-09T06:39:44.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.175+0000 7f43dcd45700 1 --2- 172.21.15.163:0/3217222548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43c4003fe0 0x7f43c4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:44.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.175+0000 7f43dd546700 1 --2- 172.21.15.163:0/3217222548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43d80b53d0 0x7f43c4003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:44.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.175+0000 7f43ddd47700 1 --2- 172.21.15.163:0/3217222548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43c4004d50 0x7f43c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:44.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.175+0000 7f43dcd45700 1 -- 172.21.15.163:0/3217222548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43d80b53d0 msgr2=0x7f43c4003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:44.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.175+0000 7f43dcd45700 1 --2- 172.21.15.163:0/3217222548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43d80b53d0 0x7f43c4003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:44.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.175+0000 7f43dcd45700 1 -- 172.21.15.163:0/3217222548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43c4004d50 msgr2=0x7f43c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:44.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.175+0000 7f43dcd45700 1 --2- 172.21.15.163:0/3217222548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43c4004d50 0x7f43c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:44.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.175+0000 7f43dcd45700 1 -- 172.21.15.163:0/3217222548 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c4003570 con 0x7f43c4003fe0 2023-10-09T06:39:44.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.175+0000 7f43ddd47700 1 --2- 172.21.15.163:0/3217222548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43c4004d50 0x7f43c40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:44.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.175+0000 7f43dcd45700 1 --2- 172.21.15.163:0/3217222548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43c4003fe0 0x7f43c4004810 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f43c8006f00 tx=0x7f43c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb167c2f160f0154 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:44.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.175+0000 7f43cffff700 1 -- 172.21.15.163:0/3217222548 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c800e710 con 0x7f43c4003fe0 2023-10-09T06:39:44.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.175+0000 7f43cffff700 1 -- 172.21.15.163:0/3217222548 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c800e870 con 0x7f43c4003fe0 2023-10-09T06:39:44.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.175+0000 7f43cffff700 1 -- 172.21.15.163:0/3217222548 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c800ec30 con 0x7f43c4003fe0 2023-10-09T06:39:44.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.176+0000 7f43df7aa700 1 -- 172.21.15.163:0/3217222548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43c4003fe0 msgr2=0x7f43c4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:44.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.176+0000 7f43df7aa700 1 --2- 172.21.15.163:0/3217222548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43c4003fe0 0x7f43c4004810 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f43c8006f00 tx=0x7f43c80091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:44.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.176+0000 7f43ddd47700 1 -- 172.21.15.163:0/3217222548 reap_dead start 2023-10-09T06:39:44.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.176+0000 7f43df7aa700 1 -- 172.21.15.163:0/3217222548 shutdown_connections 2023-10-09T06:39:44.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.176+0000 7f43df7aa700 1 -- 172.21.15.163:0/3217222548 >> 172.21.15.163:0/3217222548 conn(0x7f43d80999f0 msgr2=0x7f43d80b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:44.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.176+0000 7f43df7aa700 1 -- 172.21.15.163:0/3217222548 shutdown_connections 2023-10-09T06:39:44.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.176+0000 7f43df7aa700 1 -- 172.21.15.163:0/3217222548 wait complete. 2023-10-09T06:39:44.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.177+0000 7f43df7aa700 1 Processor -- start 2023-10-09T06:39:44.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.177+0000 7f43df7aa700 1 -- start start 2023-10-09T06:39:44.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.178+0000 7f43df7aa700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43c4003fe0 0x7f43d815ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:44.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.178+0000 7f43df7aa700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43d80c98c0 0x7f43d80c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:44.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.178+0000 7f43df7aa700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43d8069360 0x7f43d8069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:44.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.178+0000 7f43df7aa700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d809d360 con 0x7f43d8069360 2023-10-09T06:39:44.184 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.178+0000 7f43df7aa700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d80ca270 con 0x7f43c4003fe0 2023-10-09T06:39:44.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.178+0000 7f43df7aa700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d806a070 con 0x7f43d80c98c0 2023-10-09T06:39:44.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.178+0000 7f43ddd47700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43d8069360 0x7f43d8069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:44.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.178+0000 7f43ddd47700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43d8069360 0x7f43d8069790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39530/0 (socket says 172.21.15.163:39530) 2023-10-09T06:39:44.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.178+0000 7f43ddd47700 1 -- 172.21.15.163:0/904163242 learned_addr learned my addr 172.21.15.163:0/904163242 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:44.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.178+0000 7f43dcd45700 1 --2- 172.21.15.163:0/904163242 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43d80c98c0 0x7f43d80c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:44.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.178+0000 7f43dd546700 1 --2- 172.21.15.163:0/904163242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43c4003fe0 0x7f43d815ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:44.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.178+0000 7f43ddd47700 1 -- 172.21.15.163:0/904163242 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43d80c98c0 msgr2=0x7f43d80c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:44.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.178+0000 7f43ddd47700 1 --2- 172.21.15.163:0/904163242 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43d80c98c0 0x7f43d80c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:44.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.178+0000 7f43ddd47700 1 -- 172.21.15.163:0/904163242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43c4003fe0 msgr2=0x7f43d815ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:44.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.178+0000 7f43ddd47700 1 --2- 172.21.15.163:0/904163242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43c4003fe0 0x7f43d815ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:44.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.178+0000 7f43ddd47700 1 -- 172.21.15.163:0/904163242 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d806a350 con 0x7f43d8069360 2023-10-09T06:39:44.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.178+0000 7f43dd546700 1 --2- 172.21.15.163:0/904163242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43c4003fe0 0x7f43d815ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:44.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.178+0000 7f43ddd47700 1 --2- 172.21.15.163:0/904163242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43d8069360 0x7f43d8069790 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f43d4001350 tx=0x7f43d4001f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:44.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.179+0000 7f43cd7fa700 1 -- 172.21.15.163:0/904163242 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d4001850 con 0x7f43d8069360 2023-10-09T06:39:44.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.179+0000 7f43cd7fa700 1 -- 172.21.15.163:0/904163242 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d400ebc0 con 0x7f43d8069360 2023-10-09T06:39:44.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.179+0000 7f43df7aa700 1 -- 172.21.15.163:0/904163242 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d809a500 con 0x7f43d8069360 2023-10-09T06:39:44.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.179+0000 7f43cd7fa700 1 -- 172.21.15.163:0/904163242 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d4013420 con 0x7f43d8069360 2023-10-09T06:39:44.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.179+0000 7f43df7aa700 1 -- 172.21.15.163:0/904163242 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d809aac0 con 0x7f43d8069360 2023-10-09T06:39:44.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.179+0000 7f43df7aa700 1 -- 172.21.15.163:0/904163242 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d8063c90 con 0x7f43d8069360 2023-10-09T06:39:44.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.180+0000 7f43cd7fa700 1 -- 172.21.15.163:0/904163242 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f43d4013810 con 0x7f43d8069360 2023-10-09T06:39:44.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.180+0000 7f43cd7fa700 1 --2- 172.21.15.163:0/904163242 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f43bc067320 0x7f43bc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:44.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.180+0000 7f43cd7fa700 1 -- 172.21.15.163:0/904163242 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5394+0+0 (secure 0 0 0) 0x7f43d4085150 con 0x7f43d8069360 2023-10-09T06:39:44.186 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.181+0000 7f43dd546700 1 --2- 172.21.15.163:0/904163242 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f43bc067320 0x7f43bc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:44.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.181+0000 7f43dd546700 1 --2- 172.21.15.163:0/904163242 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f43bc067320 0x7f43bc0697d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f43d0006ea0 tx=0x7f43d0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:44.187 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.185+0000 7f43cd7fa700 1 -- 172.21.15.163:0/904163242 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43d4051f10 con 0x7f43d8069360 2023-10-09T06:39:44.318 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:39:44.320 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:39:44.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:44.379+0000 7f43df7aa700 1 -- 172.21.15.163:0/904163242 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000719180.data", "pool2": "cephfs.volume_0000000000719180.data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f43d809acb0 con 0x7f43d8069360 2023-10-09T06:39:45.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.025+0000 7f43cd7fa700 1 -- 172.21.15.163:0/904163242 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000719180.data", "pool2": "cephfs.volume_0000000000719180.data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000719180.data' removed v188) v1 ==== 241+0+0 (secure 0 0 0) 0x7f43d4055530 con 0x7f43d8069360 2023-10-09T06:39:45.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.029+0000 7f43df7aa700 1 -- 172.21.15.163:0/904163242 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f43bc067320 msgr2=0x7f43bc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:45.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.029+0000 7f43df7aa700 1 --2- 172.21.15.163:0/904163242 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f43bc067320 0x7f43bc0697d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f43d0006ea0 tx=0x7f43d0008040 comp rx=0 tx=0).stop 2023-10-09T06:39:45.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.029+0000 7f43df7aa700 1 -- 172.21.15.163:0/904163242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43d8069360 msgr2=0x7f43d8069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:45.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.029+0000 7f43df7aa700 1 --2- 172.21.15.163:0/904163242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43d8069360 0x7f43d8069790 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f43d4001350 tx=0x7f43d4001f50 comp rx=0 tx=0).stop 2023-10-09T06:39:45.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.029+0000 7f43df7aa700 1 -- 172.21.15.163:0/904163242 shutdown_connections 2023-10-09T06:39:45.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.029+0000 7f43df7aa700 1 --2- 172.21.15.163:0/904163242 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f43bc067320 0x7f43bc0697d0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:45.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.029+0000 7f43df7aa700 1 --2- 172.21.15.163:0/904163242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43d8069360 0x7f43d8069790 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:45.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.029+0000 7f43df7aa700 1 --2- 172.21.15.163:0/904163242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43c4003fe0 0x7f43d815ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:45.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.029+0000 7f43df7aa700 1 --2- 172.21.15.163:0/904163242 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43d80c98c0 0x7f43d80c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:45.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.029+0000 7f43df7aa700 1 -- 172.21.15.163:0/904163242 >> 172.21.15.163:0/904163242 conn(0x7f43d80999f0 msgr2=0x7f43d8158e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:45.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.029+0000 7f43df7aa700 1 -- 172.21.15.163:0/904163242 shutdown_connections 2023-10-09T06:39:45.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.029+0000 7f43df7aa700 1 -- 172.21.15.163:0/904163242 wait complete. 2023-10-09T06:39:45.031 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs.volume_0000000000719180.data' removed 2023-10-09T06:39:45.043 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-09T06:39:45.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.198+0000 7fe683fff700 1 Processor -- start 2023-10-09T06:39:45.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.198+0000 7fe683fff700 1 -- start start 2023-10-09T06:39:45.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.199+0000 7fe683fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe6840b16d0 0x7fe6840b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:45.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.199+0000 7fe683fff700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe6840b33b0 0x7fe6840b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:45.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.199+0000 7fe683fff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6840b2080 con 0x7fe6840b4e70 2023-10-09T06:39:45.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.199+0000 7fe683fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6840cfb00 con 0x7fe6840b16d0 2023-10-09T06:39:45.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.199+0000 7fe683fff700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6840b5c40 con 0x7fe6840b33b0 2023-10-09T06:39:45.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.199+0000 7fe6827fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe6840b16d0 0x7fe6840b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:45.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.199+0000 7fe6827fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe6840b16d0 0x7fe6840b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39542/0 (socket says 172.21.15.163:39542) 2023-10-09T06:39:45.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.199+0000 7fe6827fc700 1 -- 172.21.15.163:0/3131911522 learned_addr learned my addr 172.21.15.163:0/3131911522 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:45.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.199+0000 7fe6837fe700 1 --2- 172.21.15.163:0/3131911522 >> v2:172.21.15.181:3300/0 conn(0x7fe6840b33b0 0x7fe6840b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:45.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.199+0000 7fe6827fc700 1 -- 172.21.15.163:0/3131911522 >> v2:172.21.15.181:3300/0 conn(0x7fe6840b33b0 msgr2=0x7fe6840b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:45.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.199+0000 7fe6827fc700 1 --2- 172.21.15.163:0/3131911522 >> v2:172.21.15.181:3300/0 conn(0x7fe6840b33b0 0x7fe6840b37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:45.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.199+0000 7fe6827fc700 1 -- 172.21.15.163:0/3131911522 >> v1:172.21.15.181:6789/0 conn(0x7fe6840b4e70 legacy=0x7fe6840ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:45.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.199+0000 7fe6827fc700 1 -- 172.21.15.163:0/3131911522 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6840b5f20 con 0x7fe6840b16d0 2023-10-09T06:39:45.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.199+0000 7fe6827fc700 1 --2- 172.21.15.163:0/3131911522 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6840b16d0 0x7fe6840b1b40 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fe684071380 tx=0x7fe66c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4caeeabf2cb0ed77 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:45.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.200+0000 7fe681ffb700 1 -- 172.21.15.163:0/3131911522 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c00b860 con 0x7fe6840b16d0 2023-10-09T06:39:45.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.200+0000 7fe681ffb700 1 -- 172.21.15.163:0/3131911522 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66c00b9c0 con 0x7fe6840b16d0 2023-10-09T06:39:45.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.200+0000 7fe681ffb700 1 -- 172.21.15.163:0/3131911522 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c00bd80 con 0x7fe6840b16d0 2023-10-09T06:39:45.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.200+0000 7fe683fff700 1 -- 172.21.15.163:0/3131911522 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6840b16d0 msgr2=0x7fe6840b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:45.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.200+0000 7fe683fff700 1 --2- 172.21.15.163:0/3131911522 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6840b16d0 0x7fe6840b1b40 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fe684071380 tx=0x7fe66c009250 comp rx=0 tx=0).stop 2023-10-09T06:39:45.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.200+0000 7fe683fff700 1 -- 172.21.15.163:0/3131911522 shutdown_connections 2023-10-09T06:39:45.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.200+0000 7fe683fff700 1 --2- 172.21.15.163:0/3131911522 >> v2:172.21.15.181:3300/0 conn(0x7fe6840b33b0 0x7fe6840b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:45.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.200+0000 7fe683fff700 1 --2- 172.21.15.163:0/3131911522 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6840b16d0 0x7fe6840b1b40 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:45.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.200+0000 7fe683fff700 1 -- 172.21.15.163:0/3131911522 >> 172.21.15.163:0/3131911522 conn(0x7fe6840d10c0 msgr2=0x7fe6840affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:45.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.200+0000 7fe683fff700 1 -- 172.21.15.163:0/3131911522 shutdown_connections 2023-10-09T06:39:45.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.200+0000 7fe683fff700 1 -- 172.21.15.163:0/3131911522 wait complete. 2023-10-09T06:39:45.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.201+0000 7fe683fff700 1 Processor -- start 2023-10-09T06:39:45.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.201+0000 7fe683fff700 1 -- start start 2023-10-09T06:39:45.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.202+0000 7fe683fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6840b16d0 0x7fe6840cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:45.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.202+0000 7fe683fff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6840b33b0 0x7fe684158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:45.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.202+0000 7fe683fff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6840b4e70 0x7fe68415d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:45.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.202+0000 7fe683fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6840cc680 con 0x7fe6840b16d0 2023-10-09T06:39:45.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.202+0000 7fe683fff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe684169d70 con 0x7fe6840b33b0 2023-10-09T06:39:45.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.202+0000 7fe683fff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe684169fc0 con 0x7fe6840b4e70 2023-10-09T06:39:45.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.202+0000 7fe682ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6840b16d0 0x7fe6840cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:45.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.202+0000 7fe682ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6840b16d0 0x7fe6840cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39556/0 (socket says 172.21.15.163:39556) 2023-10-09T06:39:45.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.202+0000 7fe682ffd700 1 -- 172.21.15.163:0/1336388360 learned_addr learned my addr 172.21.15.163:0/1336388360 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:45.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.202+0000 7fe6837fe700 1 --2- 172.21.15.163:0/1336388360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6840b4e70 0x7fe68415d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:45.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.202+0000 7fe6827fc700 1 --2- 172.21.15.163:0/1336388360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6840b33b0 0x7fe684158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:45.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.203+0000 7fe682ffd700 1 -- 172.21.15.163:0/1336388360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6840b4e70 msgr2=0x7fe68415d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:45.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.203+0000 7fe682ffd700 1 --2- 172.21.15.163:0/1336388360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6840b4e70 0x7fe68415d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:45.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.203+0000 7fe682ffd700 1 -- 172.21.15.163:0/1336388360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6840b33b0 msgr2=0x7fe684158290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:45.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.203+0000 7fe682ffd700 1 --2- 172.21.15.163:0/1336388360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6840b33b0 0x7fe684158290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:45.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.203+0000 7fe682ffd700 1 -- 172.21.15.163:0/1336388360 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66c0094f0 con 0x7fe6840b16d0 2023-10-09T06:39:45.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.203+0000 7fe682ffd700 1 --2- 172.21.15.163:0/1336388360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6840b16d0 0x7fe6840cb5d0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fe67400cb10 tx=0x7fe6740017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:45.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.203+0000 7fe67affd700 1 -- 172.21.15.163:0/1336388360 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67400a760 con 0x7fe6840b16d0 2023-10-09T06:39:45.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.203+0000 7fe67affd700 1 -- 172.21.15.163:0/1336388360 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67400a8e0 con 0x7fe6840b16d0 2023-10-09T06:39:45.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.203+0000 7fe683fff700 1 -- 172.21.15.163:0/1336388360 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6840cbb10 con 0x7fe6840b16d0 2023-10-09T06:39:45.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.203+0000 7fe67affd700 1 -- 172.21.15.163:0/1336388360 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67400aca0 con 0x7fe6840b16d0 2023-10-09T06:39:45.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.203+0000 7fe683fff700 1 -- 172.21.15.163:0/1336388360 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6840cc3a0 con 0x7fe6840b16d0 2023-10-09T06:39:45.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.204+0000 7fe683fff700 1 -- 172.21.15.163:0/1336388360 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe684088990 con 0x7fe6840b16d0 2023-10-09T06:39:45.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.204+0000 7fe67affd700 1 -- 172.21.15.163:0/1336388360 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe674003030 con 0x7fe6840b16d0 2023-10-09T06:39:45.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.205+0000 7fe67affd700 1 --2- 172.21.15.163:0/1336388360 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe664067320 0x7fe6640697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:45.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.205+0000 7fe67affd700 1 -- 172.21.15.163:0/1336388360 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fe674084a30 con 0x7fe6840b16d0 2023-10-09T06:39:45.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.205+0000 7fe6827fc700 1 --2- 172.21.15.163:0/1336388360 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe664067320 0x7fe6640697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:45.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.205+0000 7fe6827fc700 1 --2- 172.21.15.163:0/1336388360 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe664067320 0x7fe6640697d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe684071380 tx=0x7fe66c017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:45.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.209+0000 7fe67affd700 1 -- 172.21.15.163:0/1336388360 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe6740519c0 con 0x7fe6840b16d0 2023-10-09T06:39:45.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.391+0000 7fe683fff700 1 -- 172.21.15.163:0/1336388360 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe68404bb80 con 0x7fe6840b16d0 2023-10-09T06:39:45.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.395+0000 7fe67affd700 1 -- 172.21.15.163:0/1336388360 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe674054fe0 con 0x7fe6840b16d0 2023-10-09T06:39:45.405 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.404+0000 7fe683fff700 1 -- 172.21.15.163:0/1336388360 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe664067320 msgr2=0x7fe6640697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:45.405 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.404+0000 7fe683fff700 1 --2- 172.21.15.163:0/1336388360 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe664067320 0x7fe6640697d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe684071380 tx=0x7fe66c017000 comp rx=0 tx=0).stop 2023-10-09T06:39:45.405 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.404+0000 7fe683fff700 1 -- 172.21.15.163:0/1336388360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6840b16d0 msgr2=0x7fe6840cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:45.405 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.404+0000 7fe683fff700 1 --2- 172.21.15.163:0/1336388360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6840b16d0 0x7fe6840cb5d0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fe67400cb10 tx=0x7fe6740017b0 comp rx=0 tx=0).stop 2023-10-09T06:39:45.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.404+0000 7fe683fff700 1 -- 172.21.15.163:0/1336388360 shutdown_connections 2023-10-09T06:39:45.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.404+0000 7fe683fff700 1 --2- 172.21.15.163:0/1336388360 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe664067320 0x7fe6640697d0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:45.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.404+0000 7fe683fff700 1 --2- 172.21.15.163:0/1336388360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6840b16d0 0x7fe6840cb5d0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:45.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.404+0000 7fe683fff700 1 --2- 172.21.15.163:0/1336388360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6840b4e70 0x7fe68415d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:45.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.404+0000 7fe683fff700 1 --2- 172.21.15.163:0/1336388360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6840b33b0 0x7fe684158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:45.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.404+0000 7fe683fff700 1 -- 172.21.15.163:0/1336388360 >> 172.21.15.163:0/1336388360 conn(0x7fe6840d10c0 msgr2=0x7fe6840c5420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:45.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.405+0000 7fe683fff700 1 -- 172.21.15.163:0/1336388360 shutdown_connections 2023-10-09T06:39:45.407 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.405+0000 7fe683fff700 1 -- 172.21.15.163:0/1336388360 wait complete. 2023-10-09T06:39:45.419 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_idempotency' 2023-10-09T06:39:45.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.533+0000 7f4137f65700 1 Processor -- start 2023-10-09T06:39:45.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.534+0000 7f4137f65700 1 -- start start 2023-10-09T06:39:45.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.534+0000 7f4137f65700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4130159890 0x7f413015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:45.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.534+0000 7f4137f65700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4130159c60 0x7f413015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:45.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.534+0000 7f4137f65700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f413015db10 0x7f4130069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:45.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.535+0000 7f4137f65700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f413015a160 con 0x7f4130159890 2023-10-09T06:39:45.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.535+0000 7f4137f65700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41300d13a0 con 0x7f413015db10 2023-10-09T06:39:45.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.535+0000 7f4137f65700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f413015e140 con 0x7f4130159c60 2023-10-09T06:39:45.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.535+0000 7f4135d01700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4130159890 0x7f413015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:45.537 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.535+0000 7f4135d01700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4130159890 0x7f413015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39572/0 (socket says 172.21.15.163:39572) 2023-10-09T06:39:45.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.535+0000 7f4135d01700 1 -- 172.21.15.163:0/2808406056 learned_addr learned my addr 172.21.15.163:0/2808406056 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:45.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.535+0000 7f4136502700 1 --2- 172.21.15.163:0/2808406056 >> v2:172.21.15.181:3300/0 conn(0x7f413015db10 0x7f4130069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:45.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.535+0000 7f4135500700 1 --2- 172.21.15.163:0/2808406056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4130159c60 0x7f413015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:45.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.535+0000 7f4135d01700 1 -- 172.21.15.163:0/2808406056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4130159c60 msgr2=0x7f413015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:45.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.535+0000 7f4135d01700 1 --2- 172.21.15.163:0/2808406056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4130159c60 0x7f413015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:45.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.535+0000 7f4135d01700 1 -- 172.21.15.163:0/2808406056 >> v2:172.21.15.181:3300/0 conn(0x7f413015db10 msgr2=0x7f4130069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:45.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.535+0000 7f4135d01700 1 --2- 172.21.15.163:0/2808406056 >> v2:172.21.15.181:3300/0 conn(0x7f413015db10 0x7f4130069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:45.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.535+0000 7f4135d01700 1 -- 172.21.15.163:0/2808406056 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413015a3d0 con 0x7f4130159890 2023-10-09T06:39:45.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.535+0000 7f4135500700 1 --2- 172.21.15.163:0/2808406056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4130159c60 0x7f413015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:45.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.535+0000 7f4135d01700 1 --2- 172.21.15.163:0/2808406056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4130159890 0x7f413015abd0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f4130089b50 tx=0x7f412c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625e9c51d9589134 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:45.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.535+0000 7f4134cff700 1 -- 172.21.15.163:0/2808406056 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412c00e070 con 0x7f4130159890 2023-10-09T06:39:45.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.535+0000 7f4134cff700 1 -- 172.21.15.163:0/2808406056 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412c00ba50 con 0x7f4130159890 2023-10-09T06:39:45.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.535+0000 7f4134cff700 1 -- 172.21.15.163:0/2808406056 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412c0092e0 con 0x7f4130159890 2023-10-09T06:39:45.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.536+0000 7f4137f65700 1 -- 172.21.15.163:0/2808406056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4130159890 msgr2=0x7f413015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:45.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.536+0000 7f4137f65700 1 --2- 172.21.15.163:0/2808406056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4130159890 0x7f413015abd0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f4130089b50 tx=0x7f412c009840 comp rx=0 tx=0).stop 2023-10-09T06:39:45.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.536+0000 7f4137f65700 1 -- 172.21.15.163:0/2808406056 shutdown_connections 2023-10-09T06:39:45.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.536+0000 7f4137f65700 1 --2- 172.21.15.163:0/2808406056 >> v2:172.21.15.181:3300/0 conn(0x7f413015db10 0x7f4130069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:45.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.536+0000 7f4137f65700 1 --2- 172.21.15.163:0/2808406056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4130159890 0x7f413015abd0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:45.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.536+0000 7f4137f65700 1 --2- 172.21.15.163:0/2808406056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4130159c60 0x7f413015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:45.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.536+0000 7f4137f65700 1 -- 172.21.15.163:0/2808406056 >> 172.21.15.163:0/2808406056 conn(0x7f4130068e20 msgr2=0x7f41300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:45.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.536+0000 7f4137f65700 1 -- 172.21.15.163:0/2808406056 shutdown_connections 2023-10-09T06:39:45.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.536+0000 7f4137f65700 1 -- 172.21.15.163:0/2808406056 wait complete. 2023-10-09T06:39:45.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.537+0000 7f4137f65700 1 Processor -- start 2023-10-09T06:39:45.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.537+0000 7f4137f65700 1 -- start start 2023-10-09T06:39:45.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.538+0000 7f4137f65700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4130159890 0x7f41300b5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:45.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.538+0000 7f4137f65700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4130159c60 0x7f41300cf460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:45.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.538+0000 7f4137f65700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f413015db10 0x7f4130146e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:45.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.538+0000 7f4137f65700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4130156770 con 0x7f4130159c60 2023-10-09T06:39:45.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.538+0000 7f4137f65700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41300b52e0 con 0x7f413015db10 2023-10-09T06:39:45.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.538+0000 7f4137f65700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4130156120 con 0x7f4130159890 2023-10-09T06:39:45.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.538+0000 7f4135500700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4130159c60 0x7f41300cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:45.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.538+0000 7f4135500700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4130159c60 0x7f41300cf460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39576/0 (socket says 172.21.15.163:39576) 2023-10-09T06:39:45.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.538+0000 7f4135500700 1 -- 172.21.15.163:0/568504633 learned_addr learned my addr 172.21.15.163:0/568504633 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:45.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.538+0000 7f4135d01700 1 --2- 172.21.15.163:0/568504633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4130159890 0x7f41300b5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:45.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.538+0000 7f4136502700 1 --2- 172.21.15.163:0/568504633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f413015db10 0x7f4130146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:45.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.538+0000 7f4135500700 1 -- 172.21.15.163:0/568504633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4130159890 msgr2=0x7f41300b5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:45.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.538+0000 7f4135500700 1 --2- 172.21.15.163:0/568504633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4130159890 0x7f41300b5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:45.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.538+0000 7f4135500700 1 -- 172.21.15.163:0/568504633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f413015db10 msgr2=0x7f4130146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:45.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.538+0000 7f4135500700 1 --2- 172.21.15.163:0/568504633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f413015db10 0x7f4130146e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:45.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.538+0000 7f4135500700 1 -- 172.21.15.163:0/568504633 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f412c009ae0 con 0x7f4130159c60 2023-10-09T06:39:45.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.538+0000 7f4135500700 1 --2- 172.21.15.163:0/568504633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4130159c60 0x7f41300cf460 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f412000c880 tx=0x7f4120008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:45.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.538+0000 7f4125ffb700 1 -- 172.21.15.163:0/568504633 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4120011600 con 0x7f4130159c60 2023-10-09T06:39:45.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.538+0000 7f4125ffb700 1 -- 172.21.15.163:0/568504633 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4120011760 con 0x7f4130159c60 2023-10-09T06:39:45.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.538+0000 7f4137f65700 1 -- 172.21.15.163:0/568504633 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4130156400 con 0x7f4130159c60 2023-10-09T06:39:45.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.538+0000 7f4125ffb700 1 -- 172.21.15.163:0/568504633 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4120011b00 con 0x7f4130159c60 2023-10-09T06:39:45.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.539+0000 7f4137f65700 1 -- 172.21.15.163:0/568504633 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f413016cd10 con 0x7f4130159c60 2023-10-09T06:39:45.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.540+0000 7f4137f65700 1 -- 172.21.15.163:0/568504633 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41301568e0 con 0x7f4130159c60 2023-10-09T06:39:45.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.540+0000 7f4125ffb700 1 -- 172.21.15.163:0/568504633 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4120018360 con 0x7f4130159c60 2023-10-09T06:39:45.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.540+0000 7f4125ffb700 1 --2- 172.21.15.163:0/568504633 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4114067320 0x7f41140697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:45.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.540+0000 7f4125ffb700 1 -- 172.21.15.163:0/568504633 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f4120084aa0 con 0x7f4130159c60 2023-10-09T06:39:45.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.540+0000 7f4135d01700 1 --2- 172.21.15.163:0/568504633 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4114067320 0x7f41140697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:45.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.541+0000 7f4135d01700 1 --2- 172.21.15.163:0/568504633 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4114067320 0x7f41140697d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f412c009880 tx=0x7f412c018000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:45.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.544+0000 7f4125ffb700 1 -- 172.21.15.163:0/568504633 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4120051920 con 0x7f4130159c60 2023-10-09T06:39:45.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:45.807+0000 7f4137f65700 1 -- 172.21.15.163:0/568504633 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_idempotency"]} v 0) v1 -- 0x7f413004c3c0 con 0x7f4130159c60 2023-10-09T06:39:46.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.025+0000 7f4125ffb700 1 -- 172.21.15.163:0/568504633 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_idempotency"]}]=0 v472) v1 ==== 147+0+0 (secure 0 0 0) 0x7f41200150d0 con 0x7f4130159c60 2023-10-09T06:39:46.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.027+0000 7f4137f65700 1 -- 172.21.15.163:0/568504633 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4114067320 msgr2=0x7f41140697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:46.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.027+0000 7f4137f65700 1 --2- 172.21.15.163:0/568504633 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4114067320 0x7f41140697d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f412c009880 tx=0x7f412c018000 comp rx=0 tx=0).stop 2023-10-09T06:39:46.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.027+0000 7f4137f65700 1 -- 172.21.15.163:0/568504633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4130159c60 msgr2=0x7f41300cf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:46.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.027+0000 7f4137f65700 1 --2- 172.21.15.163:0/568504633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4130159c60 0x7f41300cf460 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f412000c880 tx=0x7f4120008040 comp rx=0 tx=0).stop 2023-10-09T06:39:46.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.028+0000 7f4137f65700 1 -- 172.21.15.163:0/568504633 shutdown_connections 2023-10-09T06:39:46.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.028+0000 7f4137f65700 1 --2- 172.21.15.163:0/568504633 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4114067320 0x7f41140697d0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:46.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.028+0000 7f4137f65700 1 --2- 172.21.15.163:0/568504633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f413015db10 0x7f4130146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:46.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.028+0000 7f4137f65700 1 --2- 172.21.15.163:0/568504633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4130159890 0x7f41300b5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:46.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.028+0000 7f4137f65700 1 --2- 172.21.15.163:0/568504633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4130159c60 0x7f41300cf460 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:46.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.028+0000 7f4137f65700 1 -- 172.21.15.163:0/568504633 >> 172.21.15.163:0/568504633 conn(0x7f4130068e20 msgr2=0x7f413015bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:46.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.028+0000 7f4137f65700 1 -- 172.21.15.163:0/568504633 shutdown_connections 2023-10-09T06:39:46.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.028+0000 7f4137f65700 1 -- 172.21.15.163:0/568504633 wait complete. 2023-10-09T06:39:46.041 INFO:tasks.cephfs_test_runner:That volume rename is idempotent. ... ok 2023-10-09T06:39:46.042 INFO:tasks.cephfs_test_runner:Starting test: test_volume_rm (tasks.cephfs.test_volumes.TestVolumes) That the volume can only be removed when --yes-i-really-mean-it is used 2023-10-09T06:39:46.042 INFO:tasks.cephfs_test_runner:test_volume_rm (tasks.cephfs.test_volumes.TestVolumes) 2023-10-09T06:39:46.042 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rm' 2023-10-09T06:39:46.157 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.157+0000 7fb10a2c8700 1 Processor -- start 2023-10-09T06:39:46.158 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.157+0000 7fb10a2c8700 1 -- start start 2023-10-09T06:39:46.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.157+0000 7fb10a2c8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb10406a9a0 0x7fb10406ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:46.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.157+0000 7fb10a2c8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb1040b1bc0 0x7fb10415ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:46.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.157+0000 7fb10a2c8700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb1040ca690 0x7fb104156210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:46.160 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.157+0000 7fb10a2c8700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1040b05f0 con 0x7fb1040ca690 2023-10-09T06:39:46.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.157+0000 7fb10a2c8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb10415b280 con 0x7fb1040b1bc0 2023-10-09T06:39:46.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.157+0000 7fb10a2c8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10406f970 con 0x7fb10406a9a0 2023-10-09T06:39:46.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.158+0000 7fb108865700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb1040ca690 0x7fb104156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:46.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.158+0000 7fb108865700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb1040ca690 0x7fb104156210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39588/0 (socket says 172.21.15.163:39588) 2023-10-09T06:39:46.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.158+0000 7fb108865700 1 -- 172.21.15.163:0/4181019891 learned_addr learned my addr 172.21.15.163:0/4181019891 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:46.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.158+0000 7fb1037fe700 1 --2- 172.21.15.163:0/4181019891 >> v2:172.21.15.181:3300/0 conn(0x7fb1040b1bc0 0x7fb10415ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:46.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.158+0000 7fb103fff700 1 --2- 172.21.15.163:0/4181019891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb10406a9a0 0x7fb10406ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:46.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.158+0000 7fb108865700 1 -- 172.21.15.163:0/4181019891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb10406a9a0 msgr2=0x7fb10406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:46.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.158+0000 7fb108865700 1 --2- 172.21.15.163:0/4181019891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb10406a9a0 0x7fb10406ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:46.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.158+0000 7fb108865700 1 -- 172.21.15.163:0/4181019891 >> v2:172.21.15.181:3300/0 conn(0x7fb1040b1bc0 msgr2=0x7fb10415ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:46.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.158+0000 7fb108865700 1 --2- 172.21.15.163:0/4181019891 >> v2:172.21.15.181:3300/0 conn(0x7fb1040b1bc0 0x7fb10415ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:46.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.158+0000 7fb108865700 1 -- 172.21.15.163:0/4181019891 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1040b2030 con 0x7fb1040ca690 2023-10-09T06:39:46.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.158+0000 7fb1037fe700 1 --2- 172.21.15.163:0/4181019891 >> v2:172.21.15.181:3300/0 conn(0x7fb1040b1bc0 0x7fb10415ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:46.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.158+0000 7fb108865700 1 --2- 172.21.15.163:0/4181019891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1040ca690 0x7fb104156210 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fb10406d0a0 tx=0x7fb0fc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d6d216a59e520f server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:46.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.158+0000 7fb102ffd700 1 -- 172.21.15.163:0/4181019891 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc00e070 con 0x7fb1040ca690 2023-10-09T06:39:46.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.158+0000 7fb102ffd700 1 -- 172.21.15.163:0/4181019891 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0fc00ba50 con 0x7fb1040ca690 2023-10-09T06:39:46.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.158+0000 7fb102ffd700 1 -- 172.21.15.163:0/4181019891 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc0092e0 con 0x7fb1040ca690 2023-10-09T06:39:46.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.159+0000 7fb10a2c8700 1 -- 172.21.15.163:0/4181019891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1040ca690 msgr2=0x7fb104156210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:46.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.159+0000 7fb10a2c8700 1 --2- 172.21.15.163:0/4181019891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1040ca690 0x7fb104156210 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fb10406d0a0 tx=0x7fb0fc009840 comp rx=0 tx=0).stop 2023-10-09T06:39:46.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.159+0000 7fb10a2c8700 1 -- 172.21.15.163:0/4181019891 shutdown_connections 2023-10-09T06:39:46.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.159+0000 7fb10a2c8700 1 --2- 172.21.15.163:0/4181019891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1040ca690 0x7fb104156210 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:46.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.159+0000 7fb10a2c8700 1 --2- 172.21.15.163:0/4181019891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb10406a9a0 0x7fb10406ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:46.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.159+0000 7fb10a2c8700 1 --2- 172.21.15.163:0/4181019891 >> v2:172.21.15.181:3300/0 conn(0x7fb1040b1bc0 0x7fb10415ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:46.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.159+0000 7fb10a2c8700 1 -- 172.21.15.163:0/4181019891 >> 172.21.15.163:0/4181019891 conn(0x7fb1040cd480 msgr2=0x7fb1040cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:46.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.159+0000 7fb10a2c8700 1 -- 172.21.15.163:0/4181019891 shutdown_connections 2023-10-09T06:39:46.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.159+0000 7fb10a2c8700 1 -- 172.21.15.163:0/4181019891 wait complete. 2023-10-09T06:39:46.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.160+0000 7fb10a2c8700 1 Processor -- start 2023-10-09T06:39:46.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.160+0000 7fb10a2c8700 1 -- start start 2023-10-09T06:39:46.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.161+0000 7fb10a2c8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb10406a9a0 0x7fb104140e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:46.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.161+0000 7fb10a2c8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1040b1bc0 0x7fb104157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:46.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.161+0000 7fb10a2c8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1040ca690 0x7fb104157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:46.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.161+0000 7fb10a2c8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb104159780 con 0x7fb10406a9a0 2023-10-09T06:39:46.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.161+0000 7fb10a2c8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb104155580 con 0x7fb1040b1bc0 2023-10-09T06:39:46.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.161+0000 7fb10a2c8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb104155860 con 0x7fb1040ca690 2023-10-09T06:39:46.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.161+0000 7fb103fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb10406a9a0 0x7fb104140e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:46.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.161+0000 7fb103fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb10406a9a0 0x7fb104140e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39602/0 (socket says 172.21.15.163:39602) 2023-10-09T06:39:46.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.161+0000 7fb103fff700 1 -- 172.21.15.163:0/1237126113 learned_addr learned my addr 172.21.15.163:0/1237126113 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:46.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.161+0000 7fb108865700 1 --2- 172.21.15.163:0/1237126113 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1040ca690 0x7fb104157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:46.168 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.161+0000 7fb1037fe700 1 --2- 172.21.15.163:0/1237126113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1040b1bc0 0x7fb104157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:46.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.161+0000 7fb103fff700 1 -- 172.21.15.163:0/1237126113 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1040ca690 msgr2=0x7fb104157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:46.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.161+0000 7fb103fff700 1 --2- 172.21.15.163:0/1237126113 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1040ca690 0x7fb104157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:46.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.161+0000 7fb103fff700 1 -- 172.21.15.163:0/1237126113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1040b1bc0 msgr2=0x7fb104157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:46.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.161+0000 7fb103fff700 1 --2- 172.21.15.163:0/1237126113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1040b1bc0 0x7fb104157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:46.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.161+0000 7fb103fff700 1 -- 172.21.15.163:0/1237126113 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0fc009ae0 con 0x7fb10406a9a0 2023-10-09T06:39:46.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.162+0000 7fb103fff700 1 --2- 172.21.15.163:0/1237126113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb10406a9a0 0x7fb104140e50 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fb0f400fc10 tx=0x7fb0f4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:46.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.162+0000 7fb0ebfff700 1 -- 172.21.15.163:0/1237126113 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f40078f0 con 0x7fb10406a9a0 2023-10-09T06:39:46.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.162+0000 7fb0ebfff700 1 -- 172.21.15.163:0/1237126113 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f4007a50 con 0x7fb10406a9a0 2023-10-09T06:39:46.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.162+0000 7fb10a2c8700 1 -- 172.21.15.163:0/1237126113 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb104155b40 con 0x7fb10406a9a0 2023-10-09T06:39:46.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.162+0000 7fb0ebfff700 1 -- 172.21.15.163:0/1237126113 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4007df0 con 0x7fb10406a9a0 2023-10-09T06:39:46.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.162+0000 7fb10a2c8700 1 -- 172.21.15.163:0/1237126113 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1041593a0 con 0x7fb10406a9a0 2023-10-09T06:39:46.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.163+0000 7fb10a2c8700 1 -- 172.21.15.163:0/1237126113 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb10404c3c0 con 0x7fb10406a9a0 2023-10-09T06:39:46.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.163+0000 7fb0ebfff700 1 -- 172.21.15.163:0/1237126113 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb0f401a670 con 0x7fb10406a9a0 2023-10-09T06:39:46.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.163+0000 7fb0ebfff700 1 --2- 172.21.15.163:0/1237126113 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb0f0067320 0x7fb0f00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:46.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.163+0000 7fb0ebfff700 1 -- 172.21.15.163:0/1237126113 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fb0f4087030 con 0x7fb10406a9a0 2023-10-09T06:39:46.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.164+0000 7fb1037fe700 1 --2- 172.21.15.163:0/1237126113 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb0f0067320 0x7fb0f00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:46.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.164+0000 7fb1037fe700 1 --2- 172.21.15.163:0/1237126113 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb0f0067320 0x7fb0f00697d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb1040acb90 tx=0x7fb0f8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:46.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.168+0000 7fb0ebfff700 1 -- 172.21.15.163:0/1237126113 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0f4053eb0 con 0x7fb10406a9a0 2023-10-09T06:39:46.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:46.426+0000 7fb10a2c8700 1 -- 172.21.15.163:0/1237126113 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rm"]} v 0) v1 -- 0x7fb1040b2310 con 0x7fb10406a9a0 2023-10-09T06:39:47.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.028+0000 7fb0ebfff700 1 -- 172.21.15.163:0/1237126113 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rm"]}]=0 v473) v1 ==== 134+0+0 (secure 0 0 0) 0x7fb0f40150c0 con 0x7fb10406a9a0 2023-10-09T06:39:47.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.031+0000 7fb10a2c8700 1 -- 172.21.15.163:0/1237126113 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb0f0067320 msgr2=0x7fb0f00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:47.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.031+0000 7fb10a2c8700 1 --2- 172.21.15.163:0/1237126113 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb0f0067320 0x7fb0f00697d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb1040acb90 tx=0x7fb0f8008040 comp rx=0 tx=0).stop 2023-10-09T06:39:47.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.031+0000 7fb10a2c8700 1 -- 172.21.15.163:0/1237126113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb10406a9a0 msgr2=0x7fb104140e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:47.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.031+0000 7fb10a2c8700 1 --2- 172.21.15.163:0/1237126113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb10406a9a0 0x7fb104140e50 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fb0f400fc10 tx=0x7fb0f4008040 comp rx=0 tx=0).stop 2023-10-09T06:39:47.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.031+0000 7fb10a2c8700 1 -- 172.21.15.163:0/1237126113 shutdown_connections 2023-10-09T06:39:47.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.031+0000 7fb10a2c8700 1 --2- 172.21.15.163:0/1237126113 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb0f0067320 0x7fb0f00697d0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:47.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.031+0000 7fb10a2c8700 1 --2- 172.21.15.163:0/1237126113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb10406a9a0 0x7fb104140e50 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:47.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.031+0000 7fb10a2c8700 1 --2- 172.21.15.163:0/1237126113 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1040ca690 0x7fb104157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:47.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.031+0000 7fb10a2c8700 1 --2- 172.21.15.163:0/1237126113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1040b1bc0 0x7fb104157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:47.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.031+0000 7fb10a2c8700 1 -- 172.21.15.163:0/1237126113 >> 172.21.15.163:0/1237126113 conn(0x7fb1040cd480 msgr2=0x7fb1040aeaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:47.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.031+0000 7fb10a2c8700 1 -- 172.21.15.163:0/1237126113 shutdown_connections 2023-10-09T06:39:47.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.031+0000 7fb10a2c8700 1 -- 172.21.15.163:0/1237126113 wait complete. 2023-10-09T06:39:47.044 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-09T06:39:47.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.161+0000 7faaafeab700 1 Processor -- start 2023-10-09T06:39:47.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.161+0000 7faaafeab700 1 -- start start 2023-10-09T06:39:47.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.162+0000 7faaafeab700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faaa8159c60 0x7faaa815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:47.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.162+0000 7faaafeab700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faaa815db10 0x7faaa8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:47.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.162+0000 7faaafeab700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa815a160 con 0x7faaa8159890 2023-10-09T06:39:47.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.162+0000 7faaafeab700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaa815df50 con 0x7faaa8159c60 2023-10-09T06:39:47.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.162+0000 7faaafeab700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa80d13a0 con 0x7faaa815db10 2023-10-09T06:39:47.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.162+0000 7faaad446700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faaa8159c60 0x7faaa815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:47.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.162+0000 7faaae448700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faaa815db10 0x7faaa8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:47.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.162+0000 7faaad446700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faaa8159c60 0x7faaa815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:57876/0 (socket says 172.21.15.163:57876) 2023-10-09T06:39:47.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.162+0000 7faaadc47700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7faaa8159890 0x7faaa815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:58012/0 (socket says 172.21.15.163:58012) 2023-10-09T06:39:47.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.162+0000 7faaad446700 1 -- 172.21.15.163:0/281987483 learned_addr learned my addr 172.21.15.163:0/281987483 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:47.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.162+0000 7faaae448700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faaa815db10 0x7faaa8069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:45740/0 (socket says 172.21.15.163:45740) 2023-10-09T06:39:47.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.162+0000 7faaad446700 1 -- 172.21.15.163:0/281987483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faaa815db10 msgr2=0x7faaa8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:47.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.162+0000 7faaad446700 1 --2- 172.21.15.163:0/281987483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faaa815db10 0x7faaa8069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:47.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.162+0000 7faaad446700 1 -- 172.21.15.163:0/281987483 >> v1:172.21.15.181:6789/0 conn(0x7faaa8159890 legacy=0x7faaa815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:47.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.162+0000 7faaad446700 1 -- 172.21.15.163:0/281987483 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa815e140 con 0x7faaa8159c60 2023-10-09T06:39:47.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.162+0000 7faaae448700 1 --2- 172.21.15.163:0/281987483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faaa815db10 0x7faaa8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:39:47.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.162+0000 7faaacc45700 1 -- 172.21.15.163:0/281987483 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 430523631 0 0) 0x7faaa815a160 con 0x7faaa8159890 2023-10-09T06:39:47.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.163+0000 7faaad446700 1 --2- 172.21.15.163:0/281987483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faaa8159c60 0x7faaa815d5d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7faa980099b0 tx=0x7faa98009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad9a4e6624bb9abf server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:47.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.163+0000 7faaacc45700 1 -- 172.21.15.163:0/281987483 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa98009ea0 con 0x7faaa8159c60 2023-10-09T06:39:47.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.163+0000 7faaacc45700 1 -- 172.21.15.163:0/281987483 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa9800b860 con 0x7faaa8159c60 2023-10-09T06:39:47.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.163+0000 7faaacc45700 1 -- 172.21.15.163:0/281987483 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9800bc00 con 0x7faaa8159c60 2023-10-09T06:39:47.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.163+0000 7faaafeab700 1 -- 172.21.15.163:0/281987483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faaa8159c60 msgr2=0x7faaa815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:47.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.163+0000 7faaafeab700 1 --2- 172.21.15.163:0/281987483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faaa8159c60 0x7faaa815d5d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7faa980099b0 tx=0x7faa98009250 comp rx=0 tx=0).stop 2023-10-09T06:39:47.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.163+0000 7faaafeab700 1 -- 172.21.15.163:0/281987483 shutdown_connections 2023-10-09T06:39:47.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.163+0000 7faaafeab700 1 --2- 172.21.15.163:0/281987483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faaa815db10 0x7faaa8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:47.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.163+0000 7faaafeab700 1 --2- 172.21.15.163:0/281987483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faaa8159c60 0x7faaa815d5d0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:47.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.163+0000 7faaafeab700 1 -- 172.21.15.163:0/281987483 >> 172.21.15.163:0/281987483 conn(0x7faaa8068e20 msgr2=0x7faaa80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:47.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.163+0000 7faaafeab700 1 -- 172.21.15.163:0/281987483 shutdown_connections 2023-10-09T06:39:47.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.163+0000 7faaafeab700 1 -- 172.21.15.163:0/281987483 wait complete. 2023-10-09T06:39:47.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.164+0000 7faaafeab700 1 Processor -- start 2023-10-09T06:39:47.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.164+0000 7faaafeab700 1 -- start start 2023-10-09T06:39:47.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.165+0000 7faaafeab700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faaa8159890 0x7faaa815ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:47.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.165+0000 7faaafeab700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faaa8159c60 0x7faaa80b1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:47.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.165+0000 7faaafeab700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faaa815db10 0x7faaa8151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:47.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.165+0000 7faaafeab700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa814b500 con 0x7faaa8159c60 2023-10-09T06:39:47.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.165+0000 7faaafeab700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa80b1b50 con 0x7faaa815db10 2023-10-09T06:39:47.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.165+0000 7faaafeab700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa80b2690 con 0x7faaa8159890 2023-10-09T06:39:47.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.165+0000 7faaad446700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faaa8159c60 0x7faaa80b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:47.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.165+0000 7faaad446700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faaa8159c60 0x7faaa80b1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39606/0 (socket says 172.21.15.163:39606) 2023-10-09T06:39:47.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.165+0000 7faaad446700 1 -- 172.21.15.163:0/2510740915 learned_addr learned my addr 172.21.15.163:0/2510740915 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:47.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.166+0000 7faaae448700 1 --2- 172.21.15.163:0/2510740915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faaa815db10 0x7faaa8151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:47.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.166+0000 7faaadc47700 1 --2- 172.21.15.163:0/2510740915 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faaa8159890 0x7faaa815ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:47.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.166+0000 7faaad446700 1 -- 172.21.15.163:0/2510740915 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faaa8159890 msgr2=0x7faaa815ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:47.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.166+0000 7faaad446700 1 --2- 172.21.15.163:0/2510740915 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faaa8159890 0x7faaa815ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:47.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.166+0000 7faaad446700 1 -- 172.21.15.163:0/2510740915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faaa815db10 msgr2=0x7faaa8151f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:47.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.166+0000 7faaad446700 1 --2- 172.21.15.163:0/2510740915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faaa815db10 0x7faaa8151f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:47.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.166+0000 7faaad446700 1 -- 172.21.15.163:0/2510740915 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa0009a20 con 0x7faaa8159c60 2023-10-09T06:39:47.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.166+0000 7faaad446700 1 --2- 172.21.15.163:0/2510740915 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faaa8159c60 0x7faaa80b1610 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7faaa80899c0 tx=0x7faa98017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:47.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.166+0000 7faa9dffb700 1 -- 172.21.15.163:0/2510740915 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9800bbe0 con 0x7faaa8159c60 2023-10-09T06:39:47.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.166+0000 7faa9dffb700 1 -- 172.21.15.163:0/2510740915 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa98011420 con 0x7faaa8159c60 2023-10-09T06:39:47.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.166+0000 7faaafeab700 1 -- 172.21.15.163:0/2510740915 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa980094f0 con 0x7faaa8159c60 2023-10-09T06:39:47.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.166+0000 7faa9dffb700 1 -- 172.21.15.163:0/2510740915 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa980117e0 con 0x7faaa8159c60 2023-10-09T06:39:47.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.166+0000 7faaafeab700 1 -- 172.21.15.163:0/2510740915 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa80b20d0 con 0x7faaa8159c60 2023-10-09T06:39:47.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.167+0000 7faaafeab700 1 -- 172.21.15.163:0/2510740915 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa804c3c0 con 0x7faaa8159c60 2023-10-09T06:39:47.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.167+0000 7faa9dffb700 1 -- 172.21.15.163:0/2510740915 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7faa98011c10 con 0x7faaa8159c60 2023-10-09T06:39:47.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.168+0000 7faa9dffb700 1 --2- 172.21.15.163:0/2510740915 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faa8c067320 0x7faa8c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:47.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.168+0000 7faa9dffb700 1 -- 172.21.15.163:0/2510740915 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 4939+0+0 (secure 0 0 0) 0x7faa98086360 con 0x7faaa8159c60 2023-10-09T06:39:47.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.168+0000 7faaadc47700 1 --2- 172.21.15.163:0/2510740915 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faa8c067320 0x7faa8c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:47.177 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.169+0000 7faaadc47700 1 --2- 172.21.15.163:0/2510740915 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faa8c067320 0x7faa8c0697d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7faaa40011e0 tx=0x7faaa4007bf0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:47.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.172+0000 7faa9dffb700 1 -- 172.21.15.163:0/2510740915 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa980532f0 con 0x7faaa8159c60 2023-10-09T06:39:47.356 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.355+0000 7faaafeab700 1 -- 172.21.15.163:0/2510740915 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7faaa806a5b0 con 0x7faaa8159c60 2023-10-09T06:39:47.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.361+0000 7faa9dffb700 1 -- 172.21.15.163:0/2510740915 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7faa98056910 con 0x7faaa8159c60 2023-10-09T06:39:47.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.363+0000 7faaafeab700 1 -- 172.21.15.163:0/2510740915 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faa8c067320 msgr2=0x7faa8c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:47.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.363+0000 7faaafeab700 1 --2- 172.21.15.163:0/2510740915 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faa8c067320 0x7faa8c0697d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7faaa40011e0 tx=0x7faaa4007bf0 comp rx=0 tx=0).stop 2023-10-09T06:39:47.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.363+0000 7faaafeab700 1 -- 172.21.15.163:0/2510740915 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faaa8159c60 msgr2=0x7faaa80b1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:47.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.363+0000 7faaafeab700 1 --2- 172.21.15.163:0/2510740915 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faaa8159c60 0x7faaa80b1610 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7faaa80899c0 tx=0x7faa98017000 comp rx=0 tx=0).stop 2023-10-09T06:39:47.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.363+0000 7faaafeab700 1 -- 172.21.15.163:0/2510740915 shutdown_connections 2023-10-09T06:39:47.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.363+0000 7faaafeab700 1 --2- 172.21.15.163:0/2510740915 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faa8c067320 0x7faa8c0697d0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:47.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.363+0000 7faaafeab700 1 --2- 172.21.15.163:0/2510740915 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faaa815db10 0x7faaa8151f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:47.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.363+0000 7faaafeab700 1 --2- 172.21.15.163:0/2510740915 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faaa8159890 0x7faaa815ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:47.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.363+0000 7faaafeab700 1 --2- 172.21.15.163:0/2510740915 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faaa8159c60 0x7faaa80b1610 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:47.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.363+0000 7faaafeab700 1 -- 172.21.15.163:0/2510740915 >> 172.21.15.163:0/2510740915 conn(0x7faaa8068e20 msgr2=0x7faaa814d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:47.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.364+0000 7faaafeab700 1 -- 172.21.15.163:0/2510740915 shutdown_connections 2023-10-09T06:39:47.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.364+0000 7faaafeab700 1 -- 172.21.15.163:0/2510740915 wait complete. 2023-10-09T06:39:47.377 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-09T06:39:47.377 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:39:47.416 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:39:47.460 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-09T06:39:47.460 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-09T06:39:47.460 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:39:47.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.574+0000 7fc66551b700 1 Processor -- start 2023-10-09T06:39:47.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.574+0000 7fc66551b700 1 -- start start 2023-10-09T06:39:47.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.575+0000 7fc66551b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc660159420 0x7fc6600b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:47.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.575+0000 7fc66551b700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc6601597f0 0x7fc6600b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:47.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.575+0000 7fc66551b700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6600b11d0 con 0x7fc66015d230 2023-10-09T06:39:47.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.575+0000 7fc66551b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6600b14b0 con 0x7fc660159420 2023-10-09T06:39:47.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.575+0000 7fc66551b700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6600b1790 con 0x7fc6601597f0 2023-10-09T06:39:47.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.575+0000 7fc664d1a700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fc66015d230 0x7fc66015d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:52078/0 (socket says 172.21.15.163:52078) 2023-10-09T06:39:47.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.575+0000 7fc65ffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc660159420 0x7fc6600b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:47.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.575+0000 7fc664d1a700 1 -- 172.21.15.163:0/1832384192 learned_addr learned my addr 172.21.15.163:0/1832384192 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:47.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.575+0000 7fc65f7fe700 1 --2- 172.21.15.163:0/1832384192 >> v2:172.21.15.181:3300/0 conn(0x7fc6601597f0 0x7fc6600b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:47.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.575+0000 7fc65ffff700 1 -- 172.21.15.163:0/1832384192 >> v2:172.21.15.181:3300/0 conn(0x7fc6601597f0 msgr2=0x7fc6600b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:47.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.575+0000 7fc65ffff700 1 --2- 172.21.15.163:0/1832384192 >> v2:172.21.15.181:3300/0 conn(0x7fc6601597f0 0x7fc6600b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:47.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.576+0000 7fc65ffff700 1 -- 172.21.15.163:0/1832384192 >> v1:172.21.15.163:6789/0 conn(0x7fc66015d230 legacy=0x7fc66015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:47.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.576+0000 7fc65effd700 1 -- 172.21.15.163:0/1832384192 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1116845171 0 0) 0x7fc6600b11d0 con 0x7fc66015d230 2023-10-09T06:39:47.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.576+0000 7fc65ffff700 1 -- 172.21.15.163:0/1832384192 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6600b1a70 con 0x7fc660159420 2023-10-09T06:39:47.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.576+0000 7fc65ffff700 1 --2- 172.21.15.163:0/1832384192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc660159420 0x7fc6600b5f40 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fc660089750 tx=0x7fc6500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74cb98e3f2457e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:47.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.576+0000 7fc65effd700 1 -- 172.21.15.163:0/1832384192 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65000b6b0 con 0x7fc660159420 2023-10-09T06:39:47.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.576+0000 7fc65effd700 1 -- 172.21.15.163:0/1832384192 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65000b810 con 0x7fc660159420 2023-10-09T06:39:47.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.576+0000 7fc65effd700 1 -- 172.21.15.163:0/1832384192 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65000bbb0 con 0x7fc660159420 2023-10-09T06:39:47.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.576+0000 7fc66551b700 1 -- 172.21.15.163:0/1832384192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc660159420 msgr2=0x7fc6600b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:47.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.576+0000 7fc66551b700 1 --2- 172.21.15.163:0/1832384192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc660159420 0x7fc6600b5f40 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fc660089750 tx=0x7fc6500091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:47.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.576+0000 7fc66551b700 1 -- 172.21.15.163:0/1832384192 shutdown_connections 2023-10-09T06:39:47.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.576+0000 7fc66551b700 1 --2- 172.21.15.163:0/1832384192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc660159420 0x7fc6600b5f40 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:47.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.576+0000 7fc66551b700 1 --2- 172.21.15.163:0/1832384192 >> v2:172.21.15.181:3300/0 conn(0x7fc6601597f0 0x7fc6600b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:47.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.576+0000 7fc66551b700 1 -- 172.21.15.163:0/1832384192 >> 172.21.15.163:0/1832384192 conn(0x7fc6600d0660 msgr2=0x7fc6600d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:47.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.576+0000 7fc66551b700 1 -- 172.21.15.163:0/1832384192 shutdown_connections 2023-10-09T06:39:47.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.576+0000 7fc66551b700 1 -- 172.21.15.163:0/1832384192 wait complete. 2023-10-09T06:39:47.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.577+0000 7fc66551b700 1 Processor -- start 2023-10-09T06:39:47.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.578+0000 7fc66551b700 1 -- start start 2023-10-09T06:39:47.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.578+0000 7fc66551b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc660159420 0x7fc660155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:47.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.578+0000 7fc66551b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6601597f0 0x7fc6600c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:47.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.578+0000 7fc66551b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc66015d230 0x7fc6600c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:47.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.578+0000 7fc66551b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc660159ca0 con 0x7fc6601597f0 2023-10-09T06:39:47.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.578+0000 7fc66551b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66015a820 con 0x7fc66015d230 2023-10-09T06:39:47.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.578+0000 7fc66551b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc660159ef0 con 0x7fc660159420 2023-10-09T06:39:47.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.578+0000 7fc65f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6601597f0 0x7fc6600c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:47.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.578+0000 7fc65f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6601597f0 0x7fc6600c7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39626/0 (socket says 172.21.15.163:39626) 2023-10-09T06:39:47.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.579+0000 7fc65f7fe700 1 -- 172.21.15.163:0/1402234774 learned_addr learned my addr 172.21.15.163:0/1402234774 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:47.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.579+0000 7fc664d1a700 1 --2- 172.21.15.163:0/1402234774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc66015d230 0x7fc6600c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:47.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.579+0000 7fc65ffff700 1 --2- 172.21.15.163:0/1402234774 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc660159420 0x7fc660155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:47.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.579+0000 7fc65f7fe700 1 -- 172.21.15.163:0/1402234774 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc660159420 msgr2=0x7fc660155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:47.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.579+0000 7fc65f7fe700 1 --2- 172.21.15.163:0/1402234774 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc660159420 0x7fc660155570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:47.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.579+0000 7fc65f7fe700 1 -- 172.21.15.163:0/1402234774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc66015d230 msgr2=0x7fc6600c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:47.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.579+0000 7fc65f7fe700 1 --2- 172.21.15.163:0/1402234774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc66015d230 0x7fc6600c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:47.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.579+0000 7fc65f7fe700 1 -- 172.21.15.163:0/1402234774 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc650009420 con 0x7fc6601597f0 2023-10-09T06:39:47.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.579+0000 7fc664d1a700 1 --2- 172.21.15.163:0/1402234774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc66015d230 0x7fc6600c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:47.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.579+0000 7fc65f7fe700 1 --2- 172.21.15.163:0/1402234774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6601597f0 0x7fc6600c7c30 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fc65400c850 tx=0x7fc654008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:47.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.579+0000 7fc647fff700 1 -- 172.21.15.163:0/1402234774 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc654011600 con 0x7fc6601597f0 2023-10-09T06:39:47.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.579+0000 7fc647fff700 1 -- 172.21.15.163:0/1402234774 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc654011760 con 0x7fc6601597f0 2023-10-09T06:39:47.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.579+0000 7fc66551b700 1 -- 172.21.15.163:0/1402234774 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc66015a1d0 con 0x7fc6601597f0 2023-10-09T06:39:47.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.579+0000 7fc647fff700 1 -- 172.21.15.163:0/1402234774 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc654011b20 con 0x7fc6601597f0 2023-10-09T06:39:47.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.579+0000 7fc66551b700 1 -- 172.21.15.163:0/1402234774 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc660155ab0 con 0x7fc6601597f0 2023-10-09T06:39:47.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.580+0000 7fc66551b700 1 -- 172.21.15.163:0/1402234774 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6600b0d40 con 0x7fc6601597f0 2023-10-09T06:39:47.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.580+0000 7fc647fff700 1 -- 172.21.15.163:0/1402234774 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc6540193f0 con 0x7fc6601597f0 2023-10-09T06:39:47.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.581+0000 7fc647fff700 1 --2- 172.21.15.163:0/1402234774 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc64c067320 0x7fc64c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:47.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.581+0000 7fc647fff700 1 -- 172.21.15.163:0/1402234774 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fc654084ee0 con 0x7fc6601597f0 2023-10-09T06:39:47.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.581+0000 7fc65ffff700 1 --2- 172.21.15.163:0/1402234774 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc64c067320 0x7fc64c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:47.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.581+0000 7fc65ffff700 1 --2- 172.21.15.163:0/1402234774 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc64c067320 0x7fc64c0697d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc6500093f0 tx=0x7fc650017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:47.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.585+0000 7fc647fff700 1 -- 172.21.15.163:0/1402234774 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc654051d60 con 0x7fc6601597f0 2023-10-09T06:39:47.809 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.808+0000 7fc66551b700 1 -- 172.21.15.163:0/1402234774 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc66015a990 con 0x7fc6601597f0 2023-10-09T06:39:47.810 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.809+0000 7fc647fff700 1 -- 172.21.15.163:0/1402234774 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 127 v127) v1 ==== 95+0+3543 (secure 0 0 0) 0x7fc654055380 con 0x7fc6601597f0 2023-10-09T06:39:47.810 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:47.810 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":127,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6815,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3960800872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3960800872},{"type":"v1","addr":"172.21.15.181:6836","nonce":3960800872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":6989,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2592785085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":2592785085},{"type":"v1","addr":"172.21.15.181:6837","nonce":2592785085}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":127},{"gid":8631,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/380361821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":380361821},{"type":"v1","addr":"172.21.15.163:6834","nonce":380361821}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":8634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2847943599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2847943599},{"type":"v1","addr":"172.21.15.163:6835","nonce":2847943599}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118}],"filesystems":[]} 2023-10-09T06:39:47.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.811+0000 7fc66551b700 1 -- 172.21.15.163:0/1402234774 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc64c067320 msgr2=0x7fc64c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:47.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.811+0000 7fc66551b700 1 --2- 172.21.15.163:0/1402234774 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc64c067320 0x7fc64c0697d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc6500093f0 tx=0x7fc650017000 comp rx=0 tx=0).stop 2023-10-09T06:39:47.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.811+0000 7fc66551b700 1 -- 172.21.15.163:0/1402234774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6601597f0 msgr2=0x7fc6600c7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:47.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.811+0000 7fc66551b700 1 --2- 172.21.15.163:0/1402234774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6601597f0 0x7fc6600c7c30 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fc65400c850 tx=0x7fc654008040 comp rx=0 tx=0).stop 2023-10-09T06:39:47.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.811+0000 7fc66551b700 1 -- 172.21.15.163:0/1402234774 shutdown_connections 2023-10-09T06:39:47.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.811+0000 7fc66551b700 1 --2- 172.21.15.163:0/1402234774 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc64c067320 0x7fc64c0697d0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:47.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.811+0000 7fc66551b700 1 --2- 172.21.15.163:0/1402234774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc66015d230 0x7fc6600c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:47.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.811+0000 7fc66551b700 1 --2- 172.21.15.163:0/1402234774 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc660159420 0x7fc660155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:47.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.811+0000 7fc66551b700 1 --2- 172.21.15.163:0/1402234774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6601597f0 0x7fc6600c7c30 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:47.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.811+0000 7fc66551b700 1 -- 172.21.15.163:0/1402234774 >> 172.21.15.163:0/1402234774 conn(0x7fc6600d0660 msgr2=0x7fc6600b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:47.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.811+0000 7fc66551b700 1 -- 172.21.15.163:0/1402234774 shutdown_connections 2023-10-09T06:39:47.813 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:47.811+0000 7fc66551b700 1 -- 172.21.15.163:0/1402234774 wait complete. 2023-10-09T06:39:47.814 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 127 2023-10-09T06:39:47.824 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-09T06:39:47.824 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-09T06:39:47.825 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-09T06:39:47.825 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:39:47.825 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-09T06:39:47.825 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-09T06:39:47.825 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-09T06:39:47.825 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:39:47.826 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-09T06:39:47.826 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-09T06:39:47.826 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-09T06:39:47.826 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:39:47.826 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-09T06:39:47.826 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-09T06:39:47.826 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-09T06:39:47.827 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:39:47.834 INFO:tasks.ceph.mds.a:Stopped 2023-10-09T06:39:47.835 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-09T06:39:47.837 INFO:tasks.ceph.mds.c:Stopped 2023-10-09T06:39:47.837 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-09T06:39:47.838 INFO:tasks.ceph.mds.d:Stopped 2023-10-09T06:39:47.839 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-09T06:39:47.839 INFO:tasks.ceph.mds.b:Stopped 2023-10-09T06:39:47.839 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-09T06:39:47.877 INFO:tasks.ceph.mds.a:Started 2023-10-09T06:39:47.877 DEBUG:teuthology.parallel:result is None 2023-10-09T06:39:47.878 INFO:tasks.ceph.mds.c:Started 2023-10-09T06:39:47.878 DEBUG:teuthology.parallel:result is None 2023-10-09T06:39:47.881 INFO:tasks.ceph.mds.d:Started 2023-10-09T06:39:47.881 DEBUG:teuthology.parallel:result is None 2023-10-09T06:39:47.882 INFO:tasks.ceph.mds.b:Started 2023-10-09T06:39:47.882 DEBUG:teuthology.parallel:result is None 2023-10-09T06:39:47.882 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-09T06:39:47.950 INFO:tasks.ceph.mds.c.smithi163.stdout:starting mds.c at 2023-10-09T06:39:47.951 INFO:tasks.ceph.mds.a.smithi163.stdout:starting mds.a at 2023-10-09T06:39:47.965 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-09T06:39:47.966 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-09T06:39:48.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.003+0000 7ffb0615f700 1 Processor -- start 2023-10-09T06:39:48.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.003+0000 7ffb0615f700 1 -- start start 2023-10-09T06:39:48.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.004+0000 7ffb0615f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb00159c60 0x7ffb0015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:48.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.004+0000 7ffb0615f700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb0015a160 con 0x7ffb0015db10 2023-10-09T06:39:48.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.004+0000 7ffb0615f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb000d13a0 con 0x7ffb00159890 2023-10-09T06:39:48.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.004+0000 7ffb0615f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0015e140 con 0x7ffb00159c60 2023-10-09T06:39:48.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.004+0000 7ffafffff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ffb0015db10 0x7ffb00069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:52114/0 (socket says 172.21.15.163:52114) 2023-10-09T06:39:48.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.004+0000 7ffafffff700 1 -- 172.21.15.163:0/1426995407 learned_addr learned my addr 172.21.15.163:0/1426995407 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:48.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.004+0000 7ffafeffd700 1 --2- 172.21.15.163:0/1426995407 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb00159c60 0x7ffb0015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:48.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.004+0000 7ffafe7fc700 1 -- 172.21.15.163:0/1426995407 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1531437508 0 0) 0x7ffb0015a160 con 0x7ffb0015db10 2023-10-09T06:39:48.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.005+0000 7ffafe7fc700 1 -- 172.21.15.163:0/1426995407 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffae8003570 con 0x7ffb0015db10 2023-10-09T06:39:48.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.005+0000 7ffafe7fc700 1 -- 172.21.15.163:0/1426995407 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1811583440 0 0) 0x7ffb000d13a0 con 0x7ffb00159890 2023-10-09T06:39:48.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.005+0000 7ffafe7fc700 1 -- 172.21.15.163:0/1426995407 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb0015a160 con 0x7ffb00159890 2023-10-09T06:39:48.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.005+0000 7ffafeffd700 1 -- 172.21.15.163:0/1426995407 >> v1:172.21.15.181:6789/0 conn(0x7ffb00159890 legacy=0x7ffb0015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:48.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.005+0000 7ffafeffd700 1 -- 172.21.15.163:0/1426995407 >> v1:172.21.15.163:6789/0 conn(0x7ffb0015db10 legacy=0x7ffb00069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:48.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.005+0000 7ffafeffd700 1 -- 172.21.15.163:0/1426995407 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0015a3d0 con 0x7ffb00159c60 2023-10-09T06:39:48.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.005+0000 7ffafeffd700 1 --2- 172.21.15.163:0/1426995407 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb00159c60 0x7ffb0015d5d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009910 tx=0x7ffaf40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9a40c03fb94405b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:48.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.005+0000 7ffafe7fc700 1 -- 172.21.15.163:0/1426995407 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4011630 con 0x7ffb00159c60 2023-10-09T06:39:48.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.005+0000 7ffafe7fc700 1 -- 172.21.15.163:0/1426995407 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf4011790 con 0x7ffb00159c60 2023-10-09T06:39:48.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.005+0000 7ffafe7fc700 1 -- 172.21.15.163:0/1426995407 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4011b30 con 0x7ffb00159c60 2023-10-09T06:39:48.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.005+0000 7ffb0615f700 1 -- 172.21.15.163:0/1426995407 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb00159c60 msgr2=0x7ffb0015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:48.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.005+0000 7ffb0615f700 1 --2- 172.21.15.163:0/1426995407 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb00159c60 0x7ffb0015d5d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009910 tx=0x7ffaf40091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:48.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.006+0000 7ffb0615f700 1 -- 172.21.15.163:0/1426995407 shutdown_connections 2023-10-09T06:39:48.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.006+0000 7ffb0615f700 1 --2- 172.21.15.163:0/1426995407 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb00159c60 0x7ffb0015d5d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:48.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.006+0000 7ffb0615f700 1 -- 172.21.15.163:0/1426995407 >> 172.21.15.163:0/1426995407 conn(0x7ffb00068e20 msgr2=0x7ffb000d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:48.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.006+0000 7ffb0615f700 1 -- 172.21.15.163:0/1426995407 shutdown_connections 2023-10-09T06:39:48.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.006+0000 7ffb0615f700 1 -- 172.21.15.163:0/1426995407 wait complete. 2023-10-09T06:39:48.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.007+0000 7ffb0615f700 1 Processor -- start 2023-10-09T06:39:48.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.007+0000 7ffb0615f700 1 -- start start 2023-10-09T06:39:48.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.007+0000 7ffb0615f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb00159890 0x7ffb0013f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:48.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.008+0000 7ffb0615f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb00159c60 0x7ffb0016a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:48.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.008+0000 7ffb0615f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb0015db10 0x7ffb0016aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:48.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.008+0000 7ffb0615f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0013f7b0 con 0x7ffb00159890 2023-10-09T06:39:48.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.008+0000 7ffb0615f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0014c260 con 0x7ffb0015db10 2023-10-09T06:39:48.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.008+0000 7ffb0615f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0016d440 con 0x7ffb00159c60 2023-10-09T06:39:48.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.008+0000 7ffaff7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb00159890 0x7ffb0013f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:48.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.008+0000 7ffaff7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb00159890 0x7ffb0013f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39666/0 (socket says 172.21.15.163:39666) 2023-10-09T06:39:48.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.008+0000 7ffaff7fe700 1 -- 172.21.15.163:0/1596515886 learned_addr learned my addr 172.21.15.163:0/1596515886 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:48.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.008+0000 7ffafeffd700 1 --2- 172.21.15.163:0/1596515886 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb00159c60 0x7ffb0016a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:48.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.008+0000 7ffafffff700 1 --2- 172.21.15.163:0/1596515886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb0015db10 0x7ffb0016aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:48.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.008+0000 7ffaff7fe700 1 -- 172.21.15.163:0/1596515886 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb00159c60 msgr2=0x7ffb0016a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:48.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.008+0000 7ffaff7fe700 1 --2- 172.21.15.163:0/1596515886 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb00159c60 0x7ffb0016a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:48.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.008+0000 7ffaff7fe700 1 -- 172.21.15.163:0/1596515886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb0015db10 msgr2=0x7ffb0016aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:48.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.008+0000 7ffaff7fe700 1 --2- 172.21.15.163:0/1596515886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb0015db10 0x7ffb0016aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:48.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.008+0000 7ffaff7fe700 1 -- 172.21.15.163:0/1596515886 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf4009420 con 0x7ffb00159890 2023-10-09T06:39:48.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.008+0000 7ffaff7fe700 1 --2- 172.21.15.163:0/1596515886 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb00159890 0x7ffb0013f270 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7ffaf000cb10 tx=0x7ffaf0002820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:48.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.008+0000 7ffae77fe700 1 -- 172.21.15.163:0/1596515886 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf000ce20 con 0x7ffb00159890 2023-10-09T06:39:48.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.008+0000 7ffae77fe700 1 -- 172.21.15.163:0/1596515886 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf000a7e0 con 0x7ffb00159890 2023-10-09T06:39:48.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.008+0000 7ffb0615f700 1 -- 172.21.15.163:0/1596515886 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb0016d720 con 0x7ffb00159890 2023-10-09T06:39:48.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.008+0000 7ffae77fe700 1 -- 172.21.15.163:0/1596515886 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf000ab80 con 0x7ffb00159890 2023-10-09T06:39:48.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.008+0000 7ffb0615f700 1 -- 172.21.15.163:0/1596515886 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb0016dce0 con 0x7ffb00159890 2023-10-09T06:39:48.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.009+0000 7ffb0615f700 1 -- 172.21.15.163:0/1596515886 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb0004c3c0 con 0x7ffb00159890 2023-10-09T06:39:48.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.010+0000 7ffae77fe700 1 -- 172.21.15.163:0/1596515886 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ffaf0012420 con 0x7ffb00159890 2023-10-09T06:39:48.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.010+0000 7ffae77fe700 1 --2- 172.21.15.163:0/1596515886 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffaec067320 0x7ffaec0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:48.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.010+0000 7ffae77fe700 1 -- 172.21.15.163:0/1596515886 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ffaf0084c10 con 0x7ffb00159890 2023-10-09T06:39:48.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.010+0000 7ffafeffd700 1 --2- 172.21.15.163:0/1596515886 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffaec067320 0x7ffaec0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:48.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.011+0000 7ffafeffd700 1 --2- 172.21.15.163:0/1596515886 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffaec067320 0x7ffaec0697d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ffb0015a090 tx=0x7ffaf4017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:48.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.014+0000 7ffae77fe700 1 -- 172.21.15.163:0/1596515886 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffaf0051ba0 con 0x7ffb00159890 2023-10-09T06:39:48.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:48.213+0000 7ffb0615f700 1 -- 172.21.15.163:0/1596515886 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ffb0006a5b0 con 0x7ffb00159890 2023-10-09T06:39:49.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.052+0000 7ffae77fe700 1 -- 172.21.15.163:0/1596515886 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v193) v1 ==== 97+0+0 (secure 0 0 0) 0x7ffaf00551c0 con 0x7ffb00159890 2023-10-09T06:39:49.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.054+0000 7ffb0615f700 1 -- 172.21.15.163:0/1596515886 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffaec067320 msgr2=0x7ffaec0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:49.056 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.054+0000 7ffb0615f700 1 --2- 172.21.15.163:0/1596515886 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffaec067320 0x7ffaec0697d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ffb0015a090 tx=0x7ffaf4017000 comp rx=0 tx=0).stop 2023-10-09T06:39:49.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.054+0000 7ffb0615f700 1 -- 172.21.15.163:0/1596515886 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb00159890 msgr2=0x7ffb0013f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:49.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.054+0000 7ffb0615f700 1 --2- 172.21.15.163:0/1596515886 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb00159890 0x7ffb0013f270 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7ffaf000cb10 tx=0x7ffaf0002820 comp rx=0 tx=0).stop 2023-10-09T06:39:49.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.054+0000 7ffb0615f700 1 -- 172.21.15.163:0/1596515886 shutdown_connections 2023-10-09T06:39:49.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.054+0000 7ffb0615f700 1 --2- 172.21.15.163:0/1596515886 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ffaec067320 0x7ffaec0697d0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.054+0000 7ffb0615f700 1 --2- 172.21.15.163:0/1596515886 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb00159890 0x7ffb0013f270 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.057 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.054+0000 7ffb0615f700 1 --2- 172.21.15.163:0/1596515886 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb0015db10 0x7ffb0016aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.054+0000 7ffb0615f700 1 --2- 172.21.15.163:0/1596515886 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb00159c60 0x7ffb0016a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.054+0000 7ffb0615f700 1 -- 172.21.15.163:0/1596515886 >> 172.21.15.163:0/1596515886 conn(0x7ffb00068e20 msgr2=0x7ffb0015c110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:49.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.054+0000 7ffb0615f700 1 -- 172.21.15.163:0/1596515886 shutdown_connections 2023-10-09T06:39:49.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.054+0000 7ffb0615f700 1 -- 172.21.15.163:0/1596515886 wait complete. 2023-10-09T06:39:49.058 INFO:teuthology.orchestra.run.smithi163.stderr: removed all blocklist entries 2023-10-09T06:39:49.070 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-09T06:39:49.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.184+0000 7f5ce7431700 1 Processor -- start 2023-10-09T06:39:49.185 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.184+0000 7f5ce7431700 1 -- start start 2023-10-09T06:39:49.188 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.185+0000 7f5ce7431700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5ce0141500 0x7f5ce00c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:49.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.185+0000 7f5ce7431700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ce006c0a0 0x7f5ce00c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:49.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.185+0000 7f5ce7431700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce015c2a0 con 0x7f5ce006bcd0 2023-10-09T06:39:49.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.185+0000 7f5ce7431700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ce00c9880 con 0x7f5ce0141500 2023-10-09T06:39:49.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.185+0000 7f5ce7431700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce015b690 con 0x7f5ce006c0a0 2023-10-09T06:39:49.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.185+0000 7f5ce49cc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5ce0141500 0x7f5ce00c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:49.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.185+0000 7f5ce51cd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f5ce006bcd0 0x7f5ce0140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:36408/0 (socket says 172.21.15.163:36408) 2023-10-09T06:39:49.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.185+0000 7f5ce51cd700 1 -- 172.21.15.163:0/491528125 learned_addr learned my addr 172.21.15.163:0/491528125 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:49.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.186+0000 7f5ce59ce700 1 --2- 172.21.15.163:0/491528125 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ce006c0a0 0x7f5ce00c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:49.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.186+0000 7f5ce49cc700 1 -- 172.21.15.163:0/491528125 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ce006c0a0 msgr2=0x7f5ce00c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:49.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.186+0000 7f5ce49cc700 1 --2- 172.21.15.163:0/491528125 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ce006c0a0 0x7f5ce00c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.186+0000 7f5ce49cc700 1 -- 172.21.15.163:0/491528125 >> v1:172.21.15.163:6789/0 conn(0x7f5ce006bcd0 legacy=0x7f5ce0140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:49.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.186+0000 7f5cd7fff700 1 -- 172.21.15.163:0/491528125 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1901363647 0 0) 0x7f5ce015c2a0 con 0x7f5ce006bcd0 2023-10-09T06:39:49.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.186+0000 7f5ce49cc700 1 -- 172.21.15.163:0/491528125 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce015b970 con 0x7f5ce0141500 2023-10-09T06:39:49.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.186+0000 7f5ce59ce700 1 --2- 172.21.15.163:0/491528125 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ce006c0a0 0x7f5ce00c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:49.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.186+0000 7f5ce49cc700 1 --2- 172.21.15.163:0/491528125 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ce0141500 0x7f5ce00c8e00 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f5cd00098e0 tx=0x7f5cd00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d048da0f80693e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:49.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.186+0000 7f5cd7fff700 1 -- 172.21.15.163:0/491528125 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0009e80 con 0x7f5ce0141500 2023-10-09T06:39:49.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.186+0000 7f5cd7fff700 1 -- 172.21.15.163:0/491528125 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd000b860 con 0x7f5ce0141500 2023-10-09T06:39:49.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.186+0000 7f5cd7fff700 1 -- 172.21.15.163:0/491528125 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd000bbe0 con 0x7f5ce0141500 2023-10-09T06:39:49.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.187+0000 7f5ce7431700 1 -- 172.21.15.163:0/491528125 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ce0141500 msgr2=0x7f5ce00c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:49.190 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.187+0000 7f5ce7431700 1 --2- 172.21.15.163:0/491528125 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ce0141500 0x7f5ce00c8e00 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f5cd00098e0 tx=0x7f5cd00091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.187+0000 7f5ce7431700 1 -- 172.21.15.163:0/491528125 shutdown_connections 2023-10-09T06:39:49.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.187+0000 7f5ce7431700 1 --2- 172.21.15.163:0/491528125 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ce006c0a0 0x7f5ce00c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.187+0000 7f5ce7431700 1 --2- 172.21.15.163:0/491528125 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ce0141500 0x7f5ce00c8e00 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.187+0000 7f5ce7431700 1 -- 172.21.15.163:0/491528125 >> 172.21.15.163:0/491528125 conn(0x7f5ce00ce600 msgr2=0x7f5ce00cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:49.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.187+0000 7f5ce7431700 1 -- 172.21.15.163:0/491528125 shutdown_connections 2023-10-09T06:39:49.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.187+0000 7f5ce7431700 1 -- 172.21.15.163:0/491528125 wait complete. 2023-10-09T06:39:49.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.188+0000 7f5ce7431700 1 Processor -- start 2023-10-09T06:39:49.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.188+0000 7f5ce7431700 1 -- start start 2023-10-09T06:39:49.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.189+0000 7f5ce7431700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ce006bcd0 0x7f5ce00afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:49.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.189+0000 7f5ce7431700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ce006c0a0 0x7f5ce00c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:49.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.189+0000 7f5ce7431700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ce0141500 0x7f5ce00c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:49.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.189+0000 7f5ce7431700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce00b1b50 con 0x7f5ce006c0a0 2023-10-09T06:39:49.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.189+0000 7f5ce7431700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce006a400 con 0x7f5ce0141500 2023-10-09T06:39:49.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.189+0000 7f5ce7431700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce006a6e0 con 0x7f5ce006bcd0 2023-10-09T06:39:49.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.189+0000 7f5ce49cc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ce006c0a0 0x7f5ce00c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:49.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.189+0000 7f5ce49cc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ce006c0a0 0x7f5ce00c56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:48942/0 (socket says 172.21.15.163:48942) 2023-10-09T06:39:49.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.189+0000 7f5ce49cc700 1 -- 172.21.15.163:0/3607925121 learned_addr learned my addr 172.21.15.163:0/3607925121 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:49.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.189+0000 7f5ce59ce700 1 --2- 172.21.15.163:0/3607925121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ce0141500 0x7f5ce00c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:49.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.189+0000 7f5ce51cd700 1 --2- 172.21.15.163:0/3607925121 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ce006bcd0 0x7f5ce00afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:49.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.189+0000 7f5ce49cc700 1 -- 172.21.15.163:0/3607925121 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ce006bcd0 msgr2=0x7f5ce00afeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:49.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.189+0000 7f5ce49cc700 1 --2- 172.21.15.163:0/3607925121 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ce006bcd0 0x7f5ce00afeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.189+0000 7f5ce49cc700 1 -- 172.21.15.163:0/3607925121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ce0141500 msgr2=0x7f5ce00c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:49.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.189+0000 7f5ce49cc700 1 --2- 172.21.15.163:0/3607925121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ce0141500 0x7f5ce00c5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.189+0000 7f5ce49cc700 1 -- 172.21.15.163:0/3607925121 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd0009420 con 0x7f5ce006c0a0 2023-10-09T06:39:49.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.189+0000 7f5ce49cc700 1 --2- 172.21.15.163:0/3607925121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ce006c0a0 0x7f5ce00c56b0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0069220 tx=0x7f5cd0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:49.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.189+0000 7f5cd57fa700 1 -- 172.21.15.163:0/3607925121 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd000bba0 con 0x7f5ce006c0a0 2023-10-09T06:39:49.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.190+0000 7f5cd57fa700 1 -- 172.21.15.163:0/3607925121 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd0025420 con 0x7f5ce006c0a0 2023-10-09T06:39:49.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.190+0000 7f5cd57fa700 1 -- 172.21.15.163:0/3607925121 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd00257e0 con 0x7f5ce006c0a0 2023-10-09T06:39:49.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.190+0000 7f5ce7431700 1 -- 172.21.15.163:0/3607925121 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ce006a9c0 con 0x7f5ce006c0a0 2023-10-09T06:39:49.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.190+0000 7f5ce7431700 1 -- 172.21.15.163:0/3607925121 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ce00b03f0 con 0x7f5ce006c0a0 2023-10-09T06:39:49.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.190+0000 7f5ce7431700 1 -- 172.21.15.163:0/3607925121 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ce00b1d00 con 0x7f5ce006c0a0 2023-10-09T06:39:49.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.191+0000 7f5cd57fa700 1 -- 172.21.15.163:0/3607925121 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5cd0023070 con 0x7f5ce006c0a0 2023-10-09T06:39:49.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.191+0000 7f5cd57fa700 1 --2- 172.21.15.163:0/3607925121 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5cc4067320 0x7f5cc40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:49.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.191+0000 7f5cd57fa700 1 -- 172.21.15.163:0/3607925121 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f5cd00860a0 con 0x7f5ce006c0a0 2023-10-09T06:39:49.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.192+0000 7f5ce51cd700 1 --2- 172.21.15.163:0/3607925121 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5cc4067320 0x7f5cc40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:49.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.192+0000 7f5ce51cd700 1 --2- 172.21.15.163:0/3607925121 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5cc4067320 0x7f5cc40697d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5ce00c55e0 tx=0x7f5cdc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:49.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.196+0000 7f5cd57fa700 1 -- 172.21.15.163:0/3607925121 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ce00b1d00 con 0x7f5ce006c0a0 2023-10-09T06:39:49.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.452+0000 7f5ce7431700 1 -- 172.21.15.163:0/3607925121 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5ce00b1d00 con 0x7f5ce006c0a0 2023-10-09T06:39:49.453 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.452+0000 7f5cd57fa700 1 -- 172.21.15.163:0/3607925121 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v24) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5ce00b1d00 con 0x7f5ce006c0a0 2023-10-09T06:39:49.454 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:49.454 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:39:49.454 INFO:teuthology.orchestra.run.smithi163.stdout: "auth_dump": [ 2023-10-09T06:39:49.454 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:49.455 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.a", 2023-10-09T06:39:49.455 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNltXtwDxAAlnrWRsfnZDLEmsJVowh3Mg==", 2023-10-09T06:39:49.455 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:49.455 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:39:49.455 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:39:49.455 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:39:49.456 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:49.456 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:49.456 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:49.456 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:49.456 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.b", 2023-10-09T06:39:49.456 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlTdGhGhAAXuldy0HrFd+44sSu/EV3qA==", 2023-10-09T06:39:49.457 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:49.457 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:39:49.458 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:39:49.458 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:39:49.459 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:49.459 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:49.459 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:49.459 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:49.459 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.c", 2023-10-09T06:39:49.459 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlGis7FhAAYG76TiOFX8QkTwjNX2+Ncw==", 2023-10-09T06:39:49.459 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:49.459 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:39:49.459 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:39:49.459 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:39:49.460 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:49.460 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:49.460 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:49.460 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:49.460 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mds.d", 2023-10-09T06:39:49.460 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlKhRuIRAAfhG5Gpc7klXJloYXepOhsg==", 2023-10-09T06:39:49.460 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:49.460 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:39:49.460 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:39:49.460 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:39:49.461 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:49.461 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:49.461 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:49.461 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:49.461 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.0", 2023-10-09T06:39:49.461 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDcnSNlrQeFOxAA+s6UitrTn5TK1N7kgySoXw==", 2023-10-09T06:39:49.461 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:49.462 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:39:49.462 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:39:49.462 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:49.462 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:49.463 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:49.463 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:49.463 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.1", 2023-10-09T06:39:49.463 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDfnSNlleccORAAnB+lNUFblivdoY44JfELxA==", 2023-10-09T06:39:49.463 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:49.464 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:39:49.465 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:39:49.465 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:49.465 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:49.465 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:49.465 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:49.465 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.2", 2023-10-09T06:39:49.465 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDinSNlFoB4NxAAT0+trRBij5rqgc+F0eWUfQ==", 2023-10-09T06:39:49.465 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:49.466 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:39:49.466 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:39:49.466 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:49.466 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:49.466 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:49.466 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:49.466 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.3", 2023-10-09T06:39:49.466 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDlnSNlRRQ6NhAABoBtrFjSvQujolq0Pxrd1Q==", 2023-10-09T06:39:49.466 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:49.466 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:39:49.466 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:39:49.467 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:49.467 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:49.467 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:49.467 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:49.467 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.4", 2023-10-09T06:39:49.467 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDrnSNl308uOxAA5OJyPLGQmVSJqha5d+nKKQ==", 2023-10-09T06:39:49.467 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:49.467 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:39:49.467 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:39:49.467 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:49.468 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:49.468 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:49.468 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:49.468 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.5", 2023-10-09T06:39:49.468 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDunSNlHnK+OBAAU7l4dCJ82SbGSzBEm8Axzw==", 2023-10-09T06:39:49.468 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:49.468 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:39:49.468 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:39:49.468 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:49.468 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:49.468 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:49.468 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:49.469 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.6", 2023-10-09T06:39:49.469 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDxnSNlwUSLOBAAN5N+F5ho+Co8pBh4Rz+lXQ==", 2023-10-09T06:39:49.469 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:49.469 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:39:49.469 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:39:49.469 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:49.469 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:49.469 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:49.469 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:49.469 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "osd.7", 2023-10-09T06:39:49.470 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQD0nSNl2gxGORAAoE8OLGEifcEx/km8rfSV9g==", 2023-10-09T06:39:49.470 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:49.470 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow profile osd", 2023-10-09T06:39:49.470 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile osd", 2023-10-09T06:39:49.470 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:49.470 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:49.470 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:49.470 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:49.470 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.0", 2023-10-09T06:39:49.470 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlnxY5JRAApr6czcpMdQYYtXQ9yg2q6A==", 2023-10-09T06:39:49.470 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:49.471 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:39:49.471 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow r", 2023-10-09T06:39:49.471 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-09T06:39:49.471 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:49.471 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:49.471 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:49.471 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.1", 2023-10-09T06:39:49.471 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlk6ybLRAAuShqTDAtymGdxN8KBdZDDA==", 2023-10-09T06:39:49.471 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:49.471 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:39:49.471 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:39:49.471 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:39:49.471 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:39:49.472 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:49.472 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:49.472 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:49.472 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.2", 2023-10-09T06:39:49.472 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlYyOmKhAAQmpr+mGfJH4GUa28dtlJwA==", 2023-10-09T06:39:49.472 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:49.472 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:39:49.472 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:39:49.472 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:39:49.472 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:39:49.472 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:49.472 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:49.473 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:49.473 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.3", 2023-10-09T06:39:49.473 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlD2H1MhAA4zvaXY/Gd0kndzi+QSwp7w==", 2023-10-09T06:39:49.473 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:49.473 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow", 2023-10-09T06:39:49.473 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow r", 2023-10-09T06:39:49.473 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow rw", 2023-10-09T06:39:49.473 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow rwx" 2023-10-09T06:39:49.473 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:49.473 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:49.473 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:49.474 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "client.admin", 2023-10-09T06:39:49.474 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDYnSNlsHH4LxAA85xeFEP1CUwoblaePYk5zA==", 2023-10-09T06:39:49.474 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:49.474 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:39:49.474 INFO:teuthology.orchestra.run.smithi163.stdout: "mgr": "allow *", 2023-10-09T06:39:49.474 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow *", 2023-10-09T06:39:49.474 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:49.474 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:49.474 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:49.474 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:49.474 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mgr.x", 2023-10-09T06:39:49.474 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNl2sloDBAA0FoWN9iovCl6783o3/btiA==", 2023-10-09T06:39:49.475 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:49.475 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:39:49.475 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile mgr", 2023-10-09T06:39:49.475 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:49.475 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:49.475 INFO:teuthology.orchestra.run.smithi163.stdout: }, 2023-10-09T06:39:49.475 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:49.475 INFO:teuthology.orchestra.run.smithi163.stdout: "entity": "mgr.y", 2023-10-09T06:39:49.476 INFO:teuthology.orchestra.run.smithi163.stdout: "key": "AQDZnSNlcFWVCBAAR2dKPg01ciTiNpSX0JIKFg==", 2023-10-09T06:39:49.476 INFO:teuthology.orchestra.run.smithi163.stdout: "caps": { 2023-10-09T06:39:49.476 INFO:teuthology.orchestra.run.smithi163.stdout: "mds": "allow *", 2023-10-09T06:39:49.476 INFO:teuthology.orchestra.run.smithi163.stdout: "mon": "allow profile mgr", 2023-10-09T06:39:49.476 INFO:teuthology.orchestra.run.smithi163.stdout: "osd": "allow *" 2023-10-09T06:39:49.476 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:49.476 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:49.476 INFO:teuthology.orchestra.run.smithi163.stdout: ] 2023-10-09T06:39:49.476 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:39:49.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.454+0000 7f5ce7431700 1 -- 172.21.15.163:0/3607925121 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5cc4067320 msgr2=0x7f5cc40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:49.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.454+0000 7f5ce7431700 1 --2- 172.21.15.163:0/3607925121 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5cc4067320 0x7f5cc40697d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5ce00c55e0 tx=0x7f5cdc009040 comp rx=0 tx=0).stop 2023-10-09T06:39:49.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.454+0000 7f5ce7431700 1 -- 172.21.15.163:0/3607925121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ce006c0a0 msgr2=0x7f5ce00c56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:49.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.454+0000 7f5ce7431700 1 --2- 172.21.15.163:0/3607925121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ce006c0a0 0x7f5ce00c56b0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0069220 tx=0x7f5cd0017000 comp rx=0 tx=0).stop 2023-10-09T06:39:49.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.454+0000 7f5ce7431700 1 -- 172.21.15.163:0/3607925121 shutdown_connections 2023-10-09T06:39:49.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.454+0000 7f5ce7431700 1 --2- 172.21.15.163:0/3607925121 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5cc4067320 0x7f5cc40697d0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.454+0000 7f5ce7431700 1 --2- 172.21.15.163:0/3607925121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ce0141500 0x7f5ce00c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.454+0000 7f5ce7431700 1 --2- 172.21.15.163:0/3607925121 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ce006bcd0 0x7f5ce00afeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.454+0000 7f5ce7431700 1 --2- 172.21.15.163:0/3607925121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ce006c0a0 0x7f5ce00c56b0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.454+0000 7f5ce7431700 1 -- 172.21.15.163:0/3607925121 >> 172.21.15.163:0/3607925121 conn(0x7f5ce00ce600 msgr2=0x7f5ce00cd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:49.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.454+0000 7f5ce7431700 1 -- 172.21.15.163:0/3607925121 shutdown_connections 2023-10-09T06:39:49.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.454+0000 7f5ce7431700 1 -- 172.21.15.163:0/3607925121 wait complete. 2023-10-09T06:39:49.480 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:39:49.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.597+0000 7f8c7b10a700 1 Processor -- start 2023-10-09T06:39:49.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.597+0000 7f8c7b10a700 1 -- start start 2023-10-09T06:39:49.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.597+0000 7f8c7b10a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8c7406c0a0 0x7f8c740caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:49.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.597+0000 7f8c7b10a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8c7415df50 0x7f8c7415e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:49.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.597+0000 7f8c7b10a700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c7415d7e0 con 0x7f8c7406bcd0 2023-10-09T06:39:49.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.597+0000 7f8c7b10a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c7415dac0 con 0x7f8c7415df50 2023-10-09T06:39:49.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.597+0000 7f8c7b10a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c7415aa30 con 0x7f8c7406c0a0 2023-10-09T06:39:49.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.598+0000 7f8c796a7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8c7415df50 0x7f8c7415e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:49.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.598+0000 7f8c78ea6700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f8c7406bcd0 0x7f8c7415a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:36422/0 (socket says 172.21.15.163:36422) 2023-10-09T06:39:49.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.598+0000 7f8c78ea6700 1 -- 172.21.15.163:0/2233596695 learned_addr learned my addr 172.21.15.163:0/2233596695 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:49.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.598+0000 7f8c796a7700 1 -- 172.21.15.163:0/2233596695 >> v2:172.21.15.181:3300/0 conn(0x7f8c7406c0a0 msgr2=0x7f8c740caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:49.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.598+0000 7f8c73fff700 1 --2- 172.21.15.163:0/2233596695 >> v2:172.21.15.181:3300/0 conn(0x7f8c7406c0a0 0x7f8c740caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:49.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.598+0000 7f8c796a7700 1 --2- 172.21.15.163:0/2233596695 >> v2:172.21.15.181:3300/0 conn(0x7f8c7406c0a0 0x7f8c740caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.598+0000 7f8c796a7700 1 -- 172.21.15.163:0/2233596695 >> v1:172.21.15.163:6789/0 conn(0x7f8c7406bcd0 legacy=0x7f8c7415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:49.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.598+0000 7f8c737fe700 1 -- 172.21.15.163:0/2233596695 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2047798969 0 0) 0x7f8c7415d7e0 con 0x7f8c7406bcd0 2023-10-09T06:39:49.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.598+0000 7f8c796a7700 1 -- 172.21.15.163:0/2233596695 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7415ad10 con 0x7f8c7415df50 2023-10-09T06:39:49.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.598+0000 7f8c73fff700 1 --2- 172.21.15.163:0/2233596695 >> v2:172.21.15.181:3300/0 conn(0x7f8c7406c0a0 0x7f8c740caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:49.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.598+0000 7f8c796a7700 1 --2- 172.21.15.163:0/2233596695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c7415df50 0x7f8c7415e3b0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f8c7406b4e0 tx=0x7f8c6c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c6b3ecf183b466 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:49.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.598+0000 7f8c737fe700 1 -- 172.21.15.163:0/2233596695 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c00b6b0 con 0x7f8c7415df50 2023-10-09T06:39:49.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.598+0000 7f8c737fe700 1 -- 172.21.15.163:0/2233596695 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6c00b810 con 0x7f8c7415df50 2023-10-09T06:39:49.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.598+0000 7f8c737fe700 1 -- 172.21.15.163:0/2233596695 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c00bbb0 con 0x7f8c7415df50 2023-10-09T06:39:49.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.599+0000 7f8c7b10a700 1 -- 172.21.15.163:0/2233596695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c7415df50 msgr2=0x7f8c7415e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:49.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.599+0000 7f8c7b10a700 1 --2- 172.21.15.163:0/2233596695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c7415df50 0x7f8c7415e3b0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f8c7406b4e0 tx=0x7f8c6c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.599+0000 7f8c7b10a700 1 -- 172.21.15.163:0/2233596695 shutdown_connections 2023-10-09T06:39:49.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.599+0000 7f8c7b10a700 1 --2- 172.21.15.163:0/2233596695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c7415df50 0x7f8c7415e3b0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.599+0000 7f8c7b10a700 1 --2- 172.21.15.163:0/2233596695 >> v2:172.21.15.181:3300/0 conn(0x7f8c7406c0a0 0x7f8c740caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.599+0000 7f8c7b10a700 1 -- 172.21.15.163:0/2233596695 >> 172.21.15.163:0/2233596695 conn(0x7f8c740ce600 msgr2=0x7f8c740cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:49.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.599+0000 7f8c7b10a700 1 -- 172.21.15.163:0/2233596695 shutdown_connections 2023-10-09T06:39:49.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.599+0000 7f8c7b10a700 1 -- 172.21.15.163:0/2233596695 wait complete. 2023-10-09T06:39:49.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.600+0000 7f8c7b10a700 1 Processor -- start 2023-10-09T06:39:49.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.600+0000 7f8c7b10a700 1 -- start start 2023-10-09T06:39:49.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.601+0000 7f8c7b10a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c7406bcd0 0x7f8c740b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:49.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.601+0000 7f8c7b10a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c7406c0a0 0x7f8c740c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:49.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.601+0000 7f8c7b10a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c7415df50 0x7f8c740b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:49.605 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.601+0000 7f8c7b10a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c740c7c60 con 0x7f8c7406c0a0 2023-10-09T06:39:49.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.601+0000 7f8c7b10a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c740b2930 con 0x7f8c7406bcd0 2023-10-09T06:39:49.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.601+0000 7f8c7b10a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c740c7050 con 0x7f8c7415df50 2023-10-09T06:39:49.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.601+0000 7f8c73fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c7406c0a0 0x7f8c740c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:49.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.601+0000 7f8c73fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c7406c0a0 0x7f8c740c80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:48970/0 (socket says 172.21.15.163:48970) 2023-10-09T06:39:49.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.601+0000 7f8c73fff700 1 -- 172.21.15.163:0/2292605911 learned_addr learned my addr 172.21.15.163:0/2292605911 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:49.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.601+0000 7f8c78ea6700 1 --2- 172.21.15.163:0/2292605911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c7406bcd0 0x7f8c740b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:49.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.601+0000 7f8c796a7700 1 --2- 172.21.15.163:0/2292605911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c7415df50 0x7f8c740b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:49.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.601+0000 7f8c73fff700 1 -- 172.21.15.163:0/2292605911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c7415df50 msgr2=0x7f8c740b35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:49.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.601+0000 7f8c73fff700 1 --2- 172.21.15.163:0/2292605911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c7415df50 0x7f8c740b35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.601+0000 7f8c73fff700 1 -- 172.21.15.163:0/2292605911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c7406bcd0 msgr2=0x7f8c740b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:49.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.601+0000 7f8c73fff700 1 --2- 172.21.15.163:0/2292605911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c7406bcd0 0x7f8c740b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.606 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.601+0000 7f8c73fff700 1 -- 172.21.15.163:0/2292605911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6c009420 con 0x7f8c7406c0a0 2023-10-09T06:39:49.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.601+0000 7f8c73fff700 1 --2- 172.21.15.163:0/2292605911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c7406c0a0 0x7f8c740c80f0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c00c850 tx=0x7f8c5c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:49.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.601+0000 7f8c70ff9700 1 -- 172.21.15.163:0/2292605911 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c011600 con 0x7f8c7406c0a0 2023-10-09T06:39:49.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.601+0000 7f8c70ff9700 1 -- 172.21.15.163:0/2292605911 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c5c011760 con 0x7f8c7406c0a0 2023-10-09T06:39:49.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.602+0000 7f8c7b10a700 1 -- 172.21.15.163:0/2292605911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c740c7330 con 0x7f8c7406c0a0 2023-10-09T06:39:49.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.602+0000 7f8c70ff9700 1 -- 172.21.15.163:0/2292605911 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c011b00 con 0x7f8c7406c0a0 2023-10-09T06:39:49.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.602+0000 7f8c7b10a700 1 -- 172.21.15.163:0/2292605911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c740c78f0 con 0x7f8c7406c0a0 2023-10-09T06:39:49.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.602+0000 7f8c7b10a700 1 -- 172.21.15.163:0/2292605911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c740c7dd0 con 0x7f8c7406c0a0 2023-10-09T06:39:49.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.603+0000 7f8c70ff9700 1 -- 172.21.15.163:0/2292605911 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8c5c0073b0 con 0x7f8c7406c0a0 2023-10-09T06:39:49.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.603+0000 7f8c70ff9700 1 --2- 172.21.15.163:0/2292605911 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8c54067320 0x7f8c540697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:49.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.603+0000 7f8c70ff9700 1 -- 172.21.15.163:0/2292605911 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8c5c084a30 con 0x7f8c7406c0a0 2023-10-09T06:39:49.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.603+0000 7f8c78ea6700 1 --2- 172.21.15.163:0/2292605911 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8c54067320 0x7f8c540697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:49.607 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.604+0000 7f8c78ea6700 1 --2- 172.21.15.163:0/2292605911 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8c54067320 0x7f8c540697d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8c64002230 tx=0x7f8c64009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:49.608 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.607+0000 7f8c70ff9700 1 -- 172.21.15.163:0/2292605911 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c5c051900 con 0x7f8c7406c0a0 2023-10-09T06:39:49.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.790+0000 7f8c7b10a700 1 -- 172.21.15.163:0/2292605911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c7404c3c0 con 0x7f8c7406c0a0 2023-10-09T06:39:49.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.791+0000 7f8c70ff9700 1 -- 172.21.15.163:0/2292605911 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v193) v1 ==== 74+0+12192 (secure 0 0 0) 0x7f8c5c054f20 con 0x7f8c7406c0a0 2023-10-09T06:39:49.792 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:49.792 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":193,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:39:49.050695+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:39:49.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.793+0000 7f8c7b10a700 1 -- 172.21.15.163:0/2292605911 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8c54067320 msgr2=0x7f8c540697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:49.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.793+0000 7f8c7b10a700 1 --2- 172.21.15.163:0/2292605911 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8c54067320 0x7f8c540697d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8c64002230 tx=0x7f8c64009040 comp rx=0 tx=0).stop 2023-10-09T06:39:49.794 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.793+0000 7f8c7b10a700 1 -- 172.21.15.163:0/2292605911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c7406c0a0 msgr2=0x7f8c740c80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.793+0000 7f8c7b10a700 1 --2- 172.21.15.163:0/2292605911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c7406c0a0 0x7f8c740c80f0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c00c850 tx=0x7f8c5c008040 comp rx=0 tx=0).stop 2023-10-09T06:39:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.793+0000 7f8c7b10a700 1 -- 172.21.15.163:0/2292605911 shutdown_connections 2023-10-09T06:39:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.793+0000 7f8c7b10a700 1 --2- 172.21.15.163:0/2292605911 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8c54067320 0x7f8c540697d0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.795 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.793+0000 7f8c7b10a700 1 --2- 172.21.15.163:0/2292605911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c7415df50 0x7f8c740b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.793+0000 7f8c7b10a700 1 --2- 172.21.15.163:0/2292605911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c7406bcd0 0x7f8c740b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.793+0000 7f8c7b10a700 1 --2- 172.21.15.163:0/2292605911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c7406c0a0 0x7f8c740c80f0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.793+0000 7f8c7b10a700 1 -- 172.21.15.163:0/2292605911 >> 172.21.15.163:0/2292605911 conn(0x7f8c740ce600 msgr2=0x7f8c740c8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:49.796 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.793+0000 7f8c7b10a700 1 -- 172.21.15.163:0/2292605911 shutdown_connections 2023-10-09T06:39:49.797 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.793+0000 7f8c7b10a700 1 -- 172.21.15.163:0/2292605911 wait complete. 2023-10-09T06:39:49.806 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-09T06:39:49.806 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:39:49.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.923+0000 7fbf7423c700 1 Processor -- start 2023-10-09T06:39:49.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.923+0000 7fbf7423c700 1 -- start start 2023-10-09T06:39:49.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.924+0000 7fbf7423c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbf6c159890 0x7fbf6c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:49.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.924+0000 7fbf7423c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbf6c159c60 0x7fbf6c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:49.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.924+0000 7fbf7423c700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf6c15a160 con 0x7fbf6c15db10 2023-10-09T06:39:49.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.924+0000 7fbf7423c700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf6c15df50 con 0x7fbf6c159c60 2023-10-09T06:39:49.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.924+0000 7fbf7423c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf6c0d13a0 con 0x7fbf6c159890 2023-10-09T06:39:49.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.924+0000 7fbf717d7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbf6c159c60 0x7fbf6c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:49.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.924+0000 7fbf727d9700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fbf6c15db10 0x7fbf6c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:36426/0 (socket says 172.21.15.163:36426) 2023-10-09T06:39:49.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.924+0000 7fbf717d7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbf6c159c60 0x7fbf6c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:48974/0 (socket says 172.21.15.163:48974) 2023-10-09T06:39:49.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.924+0000 7fbf727d9700 1 -- 172.21.15.163:0/2700907977 learned_addr learned my addr 172.21.15.163:0/2700907977 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:49.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.924+0000 7fbf71fd8700 1 --2- 172.21.15.163:0/2700907977 >> v2:172.21.15.181:3300/0 conn(0x7fbf6c159890 0x7fbf6c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:49.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.925+0000 7fbf717d7700 1 -- 172.21.15.163:0/2700907977 >> v2:172.21.15.181:3300/0 conn(0x7fbf6c159890 msgr2=0x7fbf6c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:49.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.925+0000 7fbf717d7700 1 --2- 172.21.15.163:0/2700907977 >> v2:172.21.15.181:3300/0 conn(0x7fbf6c159890 0x7fbf6c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.925+0000 7fbf717d7700 1 -- 172.21.15.163:0/2700907977 >> v1:172.21.15.163:6789/0 conn(0x7fbf6c15db10 legacy=0x7fbf6c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:49.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.925+0000 7fbf717d7700 1 -- 172.21.15.163:0/2700907977 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf6c15e140 con 0x7fbf6c159c60 2023-10-09T06:39:49.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.925+0000 7fbf70fd6700 1 -- 172.21.15.163:0/2700907977 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4133627361 0 0) 0x7fbf6c15a160 con 0x7fbf6c15db10 2023-10-09T06:39:49.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.925+0000 7fbf717d7700 1 --2- 172.21.15.163:0/2700907977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf6c159c60 0x7fbf6c15d5d0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0d1090 tx=0x7fbf5c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f994f04bdc082c1d server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:49.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.925+0000 7fbf70fd6700 1 -- 172.21.15.163:0/2700907977 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c011630 con 0x7fbf6c159c60 2023-10-09T06:39:49.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.925+0000 7fbf70fd6700 1 -- 172.21.15.163:0/2700907977 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5c011790 con 0x7fbf6c159c60 2023-10-09T06:39:49.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.925+0000 7fbf70fd6700 1 -- 172.21.15.163:0/2700907977 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c011b30 con 0x7fbf6c159c60 2023-10-09T06:39:49.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.925+0000 7fbf7423c700 1 -- 172.21.15.163:0/2700907977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf6c159c60 msgr2=0x7fbf6c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:49.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.925+0000 7fbf7423c700 1 --2- 172.21.15.163:0/2700907977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf6c159c60 0x7fbf6c15d5d0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0d1090 tx=0x7fbf5c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.926+0000 7fbf7423c700 1 -- 172.21.15.163:0/2700907977 shutdown_connections 2023-10-09T06:39:49.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.926+0000 7fbf7423c700 1 --2- 172.21.15.163:0/2700907977 >> v2:172.21.15.181:3300/0 conn(0x7fbf6c159890 0x7fbf6c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.926+0000 7fbf7423c700 1 --2- 172.21.15.163:0/2700907977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf6c159c60 0x7fbf6c15d5d0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.926+0000 7fbf7423c700 1 -- 172.21.15.163:0/2700907977 >> 172.21.15.163:0/2700907977 conn(0x7fbf6c068e20 msgr2=0x7fbf6c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:49.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.926+0000 7fbf7423c700 1 -- 172.21.15.163:0/2700907977 shutdown_connections 2023-10-09T06:39:49.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.926+0000 7fbf7423c700 1 -- 172.21.15.163:0/2700907977 wait complete. 2023-10-09T06:39:49.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.927+0000 7fbf7423c700 1 Processor -- start 2023-10-09T06:39:49.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.927+0000 7fbf7423c700 1 -- start start 2023-10-09T06:39:49.931 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.928+0000 7fbf7423c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf6c159890 0x7fbf6c13f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:49.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.928+0000 7fbf7423c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf6c159c60 0x7fbf6c157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:49.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.928+0000 7fbf7423c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf6c15db10 0x7fbf6c1580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:49.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.928+0000 7fbf7423c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c0cd7b0 con 0x7fbf6c159c60 2023-10-09T06:39:49.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.928+0000 7fbf7423c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c0b2d50 con 0x7fbf6c159890 2023-10-09T06:39:49.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.928+0000 7fbf7423c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c0b2a70 con 0x7fbf6c15db10 2023-10-09T06:39:49.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.928+0000 7fbf717d7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf6c159c60 0x7fbf6c157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:49.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.928+0000 7fbf717d7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf6c159c60 0x7fbf6c157b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:48984/0 (socket says 172.21.15.163:48984) 2023-10-09T06:39:49.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.928+0000 7fbf717d7700 1 -- 172.21.15.163:0/1436687497 learned_addr learned my addr 172.21.15.163:0/1436687497 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:49.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.928+0000 7fbf71fd8700 1 --2- 172.21.15.163:0/1436687497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf6c159890 0x7fbf6c13f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:49.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.928+0000 7fbf727d9700 1 --2- 172.21.15.163:0/1436687497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf6c15db10 0x7fbf6c1580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:49.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.928+0000 7fbf717d7700 1 -- 172.21.15.163:0/1436687497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf6c15db10 msgr2=0x7fbf6c1580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:49.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.928+0000 7fbf717d7700 1 --2- 172.21.15.163:0/1436687497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf6c15db10 0x7fbf6c1580c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.928+0000 7fbf717d7700 1 -- 172.21.15.163:0/1436687497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf6c159890 msgr2=0x7fbf6c13f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:49.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.928+0000 7fbf717d7700 1 --2- 172.21.15.163:0/1436687497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf6c159890 0x7fbf6c13f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:49.934 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.928+0000 7fbf717d7700 1 -- 172.21.15.163:0/1436687497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5c009420 con 0x7fbf6c159c60 2023-10-09T06:39:49.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.928+0000 7fbf717d7700 1 --2- 172.21.15.163:0/1436687497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf6c159c60 0x7fbf6c157b80 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c070e20 tx=0x7fbf5c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:49.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.928+0000 7fbf61ffb700 1 -- 172.21.15.163:0/1436687497 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c000d10 con 0x7fbf6c159c60 2023-10-09T06:39:49.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.928+0000 7fbf61ffb700 1 -- 172.21.15.163:0/1436687497 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5c011b10 con 0x7fbf6c159c60 2023-10-09T06:39:49.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.928+0000 7fbf7423c700 1 -- 172.21.15.163:0/1436687497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf6c0cd160 con 0x7fbf6c159c60 2023-10-09T06:39:49.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.928+0000 7fbf61ffb700 1 -- 172.21.15.163:0/1436687497 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c025640 con 0x7fbf6c159c60 2023-10-09T06:39:49.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.929+0000 7fbf7423c700 1 -- 172.21.15.163:0/1436687497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf6c0b1d00 con 0x7fbf6c159c60 2023-10-09T06:39:49.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.929+0000 7fbf7423c700 1 -- 172.21.15.163:0/1436687497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf6c0cd920 con 0x7fbf6c159c60 2023-10-09T06:39:49.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.930+0000 7fbf61ffb700 1 -- 172.21.15.163:0/1436687497 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbf5c025a30 con 0x7fbf6c159c60 2023-10-09T06:39:49.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.931+0000 7fbf61ffb700 1 --2- 172.21.15.163:0/1436687497 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbf50067320 0x7fbf500697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:49.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.931+0000 7fbf61ffb700 1 -- 172.21.15.163:0/1436687497 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fbf5c086a00 con 0x7fbf6c159c60 2023-10-09T06:39:49.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.931+0000 7fbf71fd8700 1 --2- 172.21.15.163:0/1436687497 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbf50067320 0x7fbf500697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:49.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.931+0000 7fbf71fd8700 1 --2- 172.21.15.163:0/1436687497 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbf50067320 0x7fbf500697d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fbf6800f4d0 tx=0x7fbf6800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:49.937 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:49.934+0000 7fbf61ffb700 1 -- 172.21.15.163:0/1436687497 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbf5c053aa0 con 0x7fbf6c159c60 2023-10-09T06:39:50.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.117+0000 7fbf7423c700 1 -- 172.21.15.163:0/1436687497 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbf6c04c3c0 con 0x7fbf6c159c60 2023-10-09T06:39:50.118 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.117+0000 7fbf61ffb700 1 -- 172.21.15.163:0/1436687497 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v193) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbf5c0330a0 con 0x7fbf6c159c60 2023-10-09T06:39:50.118 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:39:50.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.120+0000 7fbf7423c700 1 -- 172.21.15.163:0/1436687497 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbf50067320 msgr2=0x7fbf500697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:50.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.120+0000 7fbf7423c700 1 --2- 172.21.15.163:0/1436687497 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbf50067320 0x7fbf500697d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fbf6800f4d0 tx=0x7fbf6800b040 comp rx=0 tx=0).stop 2023-10-09T06:39:50.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.120+0000 7fbf7423c700 1 -- 172.21.15.163:0/1436687497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf6c159c60 msgr2=0x7fbf6c157b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:50.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.120+0000 7fbf7423c700 1 --2- 172.21.15.163:0/1436687497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf6c159c60 0x7fbf6c157b80 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c070e20 tx=0x7fbf5c017000 comp rx=0 tx=0).stop 2023-10-09T06:39:50.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.120+0000 7fbf7423c700 1 -- 172.21.15.163:0/1436687497 shutdown_connections 2023-10-09T06:39:50.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.120+0000 7fbf7423c700 1 --2- 172.21.15.163:0/1436687497 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbf50067320 0x7fbf500697d0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:50.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.120+0000 7fbf7423c700 1 --2- 172.21.15.163:0/1436687497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf6c15db10 0x7fbf6c1580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:50.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.120+0000 7fbf7423c700 1 --2- 172.21.15.163:0/1436687497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf6c159890 0x7fbf6c13f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:50.121 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.120+0000 7fbf7423c700 1 --2- 172.21.15.163:0/1436687497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf6c159c60 0x7fbf6c157b80 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:50.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.120+0000 7fbf7423c700 1 -- 172.21.15.163:0/1436687497 >> 172.21.15.163:0/1436687497 conn(0x7fbf6c068e20 msgr2=0x7fbf6c1536e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:50.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.120+0000 7fbf7423c700 1 -- 172.21.15.163:0/1436687497 shutdown_connections 2023-10-09T06:39:50.122 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.120+0000 7fbf7423c700 1 -- 172.21.15.163:0/1436687497 wait complete. 2023-10-09T06:39:50.133 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-09T06:39:50.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.246+0000 7fda38139700 1 Processor -- start 2023-10-09T06:39:50.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.246+0000 7fda38139700 1 -- start start 2023-10-09T06:39:50.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.247+0000 7fda38139700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda30159890 0x7fda3015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:50.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.247+0000 7fda38139700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fda30069560 0x7fda300699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:50.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.247+0000 7fda38139700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda3015a160 con 0x7fda30159c60 2023-10-09T06:39:50.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.247+0000 7fda38139700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda30069f20 con 0x7fda30069560 2023-10-09T06:39:50.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.247+0000 7fda38139700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda300d13a0 con 0x7fda30159890 2023-10-09T06:39:50.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.247+0000 7fda366d6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fda30069560 0x7fda300699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:50.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.247+0000 7fda35ed5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda30159890 0x7fda3015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:50.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.248+0000 7fda356d4700 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fda30159c60 0x7fda3015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.163:54680/0 (socket says 172.21.15.163:54680) 2023-10-09T06:39:50.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.248+0000 7fda366d6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fda30069560 0x7fda300699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.163:46288/0 (socket says 172.21.15.163:46288) 2023-10-09T06:39:50.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.248+0000 7fda356d4700 1 -- 172.21.15.163:0/3515066627 learned_addr learned my addr 172.21.15.163:0/3515066627 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:50.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.248+0000 7fda35ed5700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda30159890 0x7fda3015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:47108/0 (socket says 172.21.15.163:47108) 2023-10-09T06:39:50.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.248+0000 7fda35ed5700 1 -- 172.21.15.163:0/3515066627 >> v2:172.21.15.181:3300/0 conn(0x7fda30069560 msgr2=0x7fda300699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:50.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.248+0000 7fda35ed5700 1 --2- 172.21.15.163:0/3515066627 >> v2:172.21.15.181:3300/0 conn(0x7fda30069560 0x7fda300699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:50.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.248+0000 7fda35ed5700 1 -- 172.21.15.163:0/3515066627 >> v1:172.21.15.181:6789/0 conn(0x7fda30159c60 legacy=0x7fda3015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:50.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.248+0000 7fda35ed5700 1 -- 172.21.15.163:0/3515066627 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3015e140 con 0x7fda30159890 2023-10-09T06:39:50.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.248+0000 7fda34ed3700 1 -- 172.21.15.163:0/3515066627 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 540911488 0 0) 0x7fda3015a160 con 0x7fda30159c60 2023-10-09T06:39:50.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.248+0000 7fda366d6700 1 --2- 172.21.15.163:0/3515066627 >> v2:172.21.15.181:3300/0 conn(0x7fda30069560 0x7fda300699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:39:50.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.248+0000 7fda35ed5700 1 --2- 172.21.15.163:0/3515066627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda30159890 0x7fda3015abd0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fda2c0098e0 tx=0x7fda2c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=107f210dd2aacfa server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:50.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.248+0000 7fda34ed3700 1 -- 172.21.15.163:0/3515066627 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c00b640 con 0x7fda30159890 2023-10-09T06:39:50.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.248+0000 7fda34ed3700 1 -- 172.21.15.163:0/3515066627 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2c00b7a0 con 0x7fda30159890 2023-10-09T06:39:50.251 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.248+0000 7fda34ed3700 1 -- 172.21.15.163:0/3515066627 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c00bb40 con 0x7fda30159890 2023-10-09T06:39:50.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.249+0000 7fda38139700 1 -- 172.21.15.163:0/3515066627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda30159890 msgr2=0x7fda3015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:50.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.249+0000 7fda38139700 1 --2- 172.21.15.163:0/3515066627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda30159890 0x7fda3015abd0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fda2c0098e0 tx=0x7fda2c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:50.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.249+0000 7fda38139700 1 -- 172.21.15.163:0/3515066627 shutdown_connections 2023-10-09T06:39:50.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.249+0000 7fda38139700 1 --2- 172.21.15.163:0/3515066627 >> v2:172.21.15.181:3300/0 conn(0x7fda30069560 0x7fda300699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:50.253 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.249+0000 7fda38139700 1 --2- 172.21.15.163:0/3515066627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda30159890 0x7fda3015abd0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:50.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.249+0000 7fda38139700 1 -- 172.21.15.163:0/3515066627 >> 172.21.15.163:0/3515066627 conn(0x7fda30068e20 msgr2=0x7fda300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:50.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.249+0000 7fda38139700 1 -- 172.21.15.163:0/3515066627 shutdown_connections 2023-10-09T06:39:50.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.249+0000 7fda38139700 1 -- 172.21.15.163:0/3515066627 wait complete. 2023-10-09T06:39:50.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.250+0000 7fda38139700 1 Processor -- start 2023-10-09T06:39:50.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.250+0000 7fda38139700 1 -- start start 2023-10-09T06:39:50.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.251+0000 7fda38139700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda30069560 0x7fda3014b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:50.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.251+0000 7fda38139700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda30159890 0x7fda300cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:50.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.251+0000 7fda38139700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda30159c60 0x7fda3016d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:50.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.251+0000 7fda38139700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda300b4f60 con 0x7fda30159c60 2023-10-09T06:39:50.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.251+0000 7fda38139700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3016dd50 con 0x7fda30069560 2023-10-09T06:39:50.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.251+0000 7fda38139700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda3016e030 con 0x7fda30159890 2023-10-09T06:39:50.254 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.251+0000 7fda366d6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda30159c60 0x7fda3016d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:50.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.251+0000 7fda366d6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda30159c60 0x7fda3016d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:48990/0 (socket says 172.21.15.163:48990) 2023-10-09T06:39:50.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.251+0000 7fda366d6700 1 -- 172.21.15.163:0/2071307507 learned_addr learned my addr 172.21.15.163:0/2071307507 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:50.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.251+0000 7fda356d4700 1 --2- 172.21.15.163:0/2071307507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda30159890 0x7fda300cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:50.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.251+0000 7fda35ed5700 1 --2- 172.21.15.163:0/2071307507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda30069560 0x7fda3014b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:50.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.251+0000 7fda366d6700 1 -- 172.21.15.163:0/2071307507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda30159890 msgr2=0x7fda300cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:50.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.251+0000 7fda366d6700 1 --2- 172.21.15.163:0/2071307507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda30159890 0x7fda300cd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:50.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.251+0000 7fda366d6700 1 -- 172.21.15.163:0/2071307507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda30069560 msgr2=0x7fda3014b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:50.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.251+0000 7fda366d6700 1 --2- 172.21.15.163:0/2071307507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda30069560 0x7fda3014b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:50.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.251+0000 7fda366d6700 1 -- 172.21.15.163:0/2071307507 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda28009a20 con 0x7fda30159c60 2023-10-09T06:39:50.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.251+0000 7fda35ed5700 1 --2- 172.21.15.163:0/2071307507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda30069560 0x7fda3014b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:50.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.251+0000 7fda366d6700 1 --2- 172.21.15.163:0/2071307507 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda30159c60 0x7fda3016d300 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fda28009fc0 tx=0x7fda2800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:50.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.251+0000 7fda25ffb700 1 -- 172.21.15.163:0/2071307507 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2800cd30 con 0x7fda30159c60 2023-10-09T06:39:50.256 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.252+0000 7fda25ffb700 1 -- 172.21.15.163:0/2071307507 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2800f040 con 0x7fda30159c60 2023-10-09T06:39:50.256 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.252+0000 7fda25ffb700 1 -- 172.21.15.163:0/2071307507 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda280146d0 con 0x7fda30159c60 2023-10-09T06:39:50.256 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.252+0000 7fda38139700 1 -- 172.21.15.163:0/2071307507 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda2c009420 con 0x7fda30159c60 2023-10-09T06:39:50.256 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.252+0000 7fda38139700 1 -- 172.21.15.163:0/2071307507 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda300b54e0 con 0x7fda30159c60 2023-10-09T06:39:50.256 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.253+0000 7fda38139700 1 -- 172.21.15.163:0/2071307507 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda3016e250 con 0x7fda30159c60 2023-10-09T06:39:50.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.253+0000 7fda25ffb700 1 -- 172.21.15.163:0/2071307507 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fda28003020 con 0x7fda30159c60 2023-10-09T06:39:50.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.253+0000 7fda25ffb700 1 --2- 172.21.15.163:0/2071307507 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fda14067320 0x7fda140697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:50.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.253+0000 7fda25ffb700 1 -- 172.21.15.163:0/2071307507 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fda280854c0 con 0x7fda30159c60 2023-10-09T06:39:50.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.253+0000 7fda35ed5700 1 --2- 172.21.15.163:0/2071307507 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fda14067320 0x7fda140697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:50.257 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.254+0000 7fda35ed5700 1 --2- 172.21.15.163:0/2071307507 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fda14067320 0x7fda140697d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fda2c003080 tx=0x7fda2c017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:50.258 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.257+0000 7fda25ffb700 1 -- 172.21.15.163:0/2071307507 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda28052390 con 0x7fda30159c60 2023-10-09T06:39:50.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.442+0000 7fda38139700 1 -- 172.21.15.163:0/2071307507 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fda300b5700 con 0x7fda30159c60 2023-10-09T06:39:50.443 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.442+0000 7fda25ffb700 1 -- 172.21.15.163:0/2071307507 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v193) v1 ==== 84+0+65 (secure 0 0 0) 0x7fda280559b0 con 0x7fda30159c60 2023-10-09T06:39:50.444 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:50.444 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:39:50.444 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:39:50.444 INFO:teuthology.orchestra.run.smithi163.stdout: "poolnum": 1, 2023-10-09T06:39:50.444 INFO:teuthology.orchestra.run.smithi163.stdout: "poolname": ".mgr" 2023-10-09T06:39:50.444 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:39:50.444 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:39:50.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.444+0000 7fda38139700 1 -- 172.21.15.163:0/2071307507 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fda14067320 msgr2=0x7fda140697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:50.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.444+0000 7fda38139700 1 --2- 172.21.15.163:0/2071307507 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fda14067320 0x7fda140697d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fda2c003080 tx=0x7fda2c017000 comp rx=0 tx=0).stop 2023-10-09T06:39:50.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.444+0000 7fda38139700 1 -- 172.21.15.163:0/2071307507 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda30159c60 msgr2=0x7fda3016d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:50.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.444+0000 7fda38139700 1 --2- 172.21.15.163:0/2071307507 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda30159c60 0x7fda3016d300 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fda28009fc0 tx=0x7fda2800b040 comp rx=0 tx=0).stop 2023-10-09T06:39:50.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.445+0000 7fda38139700 1 -- 172.21.15.163:0/2071307507 shutdown_connections 2023-10-09T06:39:50.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.445+0000 7fda38139700 1 --2- 172.21.15.163:0/2071307507 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fda14067320 0x7fda140697d0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:50.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.445+0000 7fda38139700 1 --2- 172.21.15.163:0/2071307507 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda30159c60 0x7fda3016d300 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:50.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.445+0000 7fda38139700 1 --2- 172.21.15.163:0/2071307507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda30069560 0x7fda3014b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:50.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.445+0000 7fda38139700 1 --2- 172.21.15.163:0/2071307507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda30159890 0x7fda300cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:50.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.445+0000 7fda38139700 1 -- 172.21.15.163:0/2071307507 >> 172.21.15.163:0/2071307507 conn(0x7fda30068e20 msgr2=0x7fda301537e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:50.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.445+0000 7fda38139700 1 -- 172.21.15.163:0/2071307507 shutdown_connections 2023-10-09T06:39:50.447 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.445+0000 7fda38139700 1 -- 172.21.15.163:0/2071307507 wait complete. 2023-10-09T06:39:50.458 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-09T06:39:50.458 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-09T06:39:50.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.576+0000 7f5a2f36f700 1 Processor -- start 2023-10-09T06:39:50.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.576+0000 7f5a2f36f700 1 -- start start 2023-10-09T06:39:50.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.577+0000 7f5a2f36f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a28159ec0 0x7f5a280b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:50.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.577+0000 7f5a2f36f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5a280b53d0 0x7f5a280cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:50.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.577+0000 7f5a2f36f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a280b5b00 con 0x7f5a280cce40 2023-10-09T06:39:50.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.577+0000 7f5a2f36f700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a280b5da0 con 0x7f5a280b53d0 2023-10-09T06:39:50.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.577+0000 7f5a2f36f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a280b6080 con 0x7f5a28159ec0 2023-10-09T06:39:50.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.577+0000 7f5a2c90a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5a280b53d0 0x7f5a280cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:50.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.577+0000 7f5a2c90a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5a280b53d0 0x7f5a280cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49002/0 (socket says 172.21.15.163:49002) 2023-10-09T06:39:50.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.577+0000 7f5a2c90a700 1 -- 172.21.15.163:0/1641093033 learned_addr learned my addr 172.21.15.163:0/1641093033 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:50.580 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.577+0000 7f5a2d10b700 1 --2- 172.21.15.163:0/1641093033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a28159ec0 0x7f5a280b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:50.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.578+0000 7f5a2c90a700 1 -- 172.21.15.163:0/1641093033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a28159ec0 msgr2=0x7f5a280b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:50.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.578+0000 7f5a2c90a700 1 --2- 172.21.15.163:0/1641093033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a28159ec0 0x7f5a280b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:50.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.578+0000 7f5a2c90a700 1 -- 172.21.15.163:0/1641093033 >> v1:172.21.15.181:6789/0 conn(0x7f5a280cce40 legacy=0x7f5a280cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:50.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.578+0000 7f5a2c90a700 1 -- 172.21.15.163:0/1641093033 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a280cc100 con 0x7f5a280b53d0 2023-10-09T06:39:50.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.578+0000 7f5a2c90a700 1 --2- 172.21.15.163:0/1641093033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a280b53d0 0x7f5a280cc900 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c0099b0 tx=0x7f5a1c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f813964bbb962692 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:50.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.578+0000 7f5a27fff700 1 -- 172.21.15.163:0/1641093033 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c00b860 con 0x7f5a280b53d0 2023-10-09T06:39:50.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.578+0000 7f5a27fff700 1 -- 172.21.15.163:0/1641093033 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a1c00b9c0 con 0x7f5a280b53d0 2023-10-09T06:39:50.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.578+0000 7f5a27fff700 1 -- 172.21.15.163:0/1641093033 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c00bd60 con 0x7f5a280b53d0 2023-10-09T06:39:50.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.578+0000 7f5a2f36f700 1 -- 172.21.15.163:0/1641093033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a280b53d0 msgr2=0x7f5a280cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:50.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.578+0000 7f5a2f36f700 1 --2- 172.21.15.163:0/1641093033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a280b53d0 0x7f5a280cc900 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c0099b0 tx=0x7f5a1c009250 comp rx=0 tx=0).stop 2023-10-09T06:39:50.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.578+0000 7f5a2f36f700 1 -- 172.21.15.163:0/1641093033 shutdown_connections 2023-10-09T06:39:50.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.578+0000 7f5a2f36f700 1 --2- 172.21.15.163:0/1641093033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a28159ec0 0x7f5a280b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:50.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.578+0000 7f5a2f36f700 1 --2- 172.21.15.163:0/1641093033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a280b53d0 0x7f5a280cc900 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:50.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.578+0000 7f5a2f36f700 1 -- 172.21.15.163:0/1641093033 >> 172.21.15.163:0/1641093033 conn(0x7f5a280999f0 msgr2=0x7f5a280b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:50.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.578+0000 7f5a2f36f700 1 -- 172.21.15.163:0/1641093033 shutdown_connections 2023-10-09T06:39:50.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.579+0000 7f5a2f36f700 1 -- 172.21.15.163:0/1641093033 wait complete. 2023-10-09T06:39:50.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.580+0000 7f5a2f36f700 1 Processor -- start 2023-10-09T06:39:50.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.580+0000 7f5a2f36f700 1 -- start start 2023-10-09T06:39:50.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.580+0000 7f5a2f36f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a280b53d0 0x7f5a2809b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:50.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.580+0000 7f5a2f36f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a280cce40 0x7f5a2809d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:50.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.580+0000 7f5a2f36f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a28159ec0 0x7f5a28155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:50.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.580+0000 7f5a2f36f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2809e320 con 0x7f5a280b53d0 2023-10-09T06:39:50.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.580+0000 7f5a2f36f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2809dac0 con 0x7f5a28159ec0 2023-10-09T06:39:50.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.580+0000 7f5a2f36f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2809dd60 con 0x7f5a280cce40 2023-10-09T06:39:50.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.581+0000 7f5a2d10b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a280b53d0 0x7f5a2809b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:50.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.581+0000 7f5a2d10b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a280b53d0 0x7f5a2809b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49010/0 (socket says 172.21.15.163:49010) 2023-10-09T06:39:50.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.581+0000 7f5a2d10b700 1 -- 172.21.15.163:0/2997749908 learned_addr learned my addr 172.21.15.163:0/2997749908 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:50.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.581+0000 7f5a2c90a700 1 --2- 172.21.15.163:0/2997749908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a280cce40 0x7f5a2809d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:50.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.581+0000 7f5a2d90c700 1 --2- 172.21.15.163:0/2997749908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a28159ec0 0x7f5a28155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:50.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.581+0000 7f5a2d10b700 1 -- 172.21.15.163:0/2997749908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a280cce40 msgr2=0x7f5a2809d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:50.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.581+0000 7f5a2d10b700 1 --2- 172.21.15.163:0/2997749908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a280cce40 0x7f5a2809d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:50.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.581+0000 7f5a2d10b700 1 -- 172.21.15.163:0/2997749908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a28159ec0 msgr2=0x7f5a28155e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:50.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.581+0000 7f5a2d10b700 1 --2- 172.21.15.163:0/2997749908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a28159ec0 0x7f5a28155e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:50.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.581+0000 7f5a2d10b700 1 -- 172.21.15.163:0/2997749908 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1c0094f0 con 0x7f5a280b53d0 2023-10-09T06:39:50.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.581+0000 7f5a2d10b700 1 --2- 172.21.15.163:0/2997749908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a280b53d0 0x7f5a2809b150 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800e0e0 tx=0x7f5a180098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:50.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.581+0000 7f5a257fa700 1 -- 172.21.15.163:0/2997749908 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1800ec20 con 0x7f5a280b53d0 2023-10-09T06:39:50.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.581+0000 7f5a257fa700 1 -- 172.21.15.163:0/2997749908 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a1800ed80 con 0x7f5a280b53d0 2023-10-09T06:39:50.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.581+0000 7f5a2f36f700 1 -- 172.21.15.163:0/2997749908 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a2809e040 con 0x7f5a280b53d0 2023-10-09T06:39:50.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.581+0000 7f5a257fa700 1 -- 172.21.15.163:0/2997749908 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a18014670 con 0x7f5a280b53d0 2023-10-09T06:39:50.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.581+0000 7f5a2f36f700 1 -- 172.21.15.163:0/2997749908 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a280687a0 con 0x7f5a280b53d0 2023-10-09T06:39:50.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.582+0000 7f5a2f36f700 1 -- 172.21.15.163:0/2997749908 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a28063c90 con 0x7f5a280b53d0 2023-10-09T06:39:50.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.583+0000 7f5a257fa700 1 -- 172.21.15.163:0/2997749908 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5a180149e0 con 0x7f5a280b53d0 2023-10-09T06:39:50.588 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.583+0000 7f5a257fa700 1 --2- 172.21.15.163:0/2997749908 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5a14067320 0x7f5a140697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:50.588 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.583+0000 7f5a257fa700 1 -- 172.21.15.163:0/2997749908 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f5a18084fc0 con 0x7f5a280b53d0 2023-10-09T06:39:50.588 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.583+0000 7f5a2c90a700 1 --2- 172.21.15.163:0/2997749908 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5a14067320 0x7f5a140697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:50.588 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.584+0000 7f5a2c90a700 1 --2- 172.21.15.163:0/2997749908 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5a14067320 0x7f5a140697d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c0099b0 tx=0x7f5a1c017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:50.588 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.587+0000 7f5a257fa700 1 -- 172.21.15.163:0/2997749908 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5a18051fa0 con 0x7f5a280b53d0 2023-10-09T06:39:50.768 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:50.767+0000 7f5a2f36f700 1 -- 172.21.15.163:0/2997749908 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5a280718b0 con 0x7f5a280b53d0 2023-10-09T06:39:51.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.058+0000 7f5a257fa700 1 -- 172.21.15.163:0/2997749908 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v194) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5a180555c0 con 0x7f5a280b53d0 2023-10-09T06:39:51.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.061+0000 7f5a2f36f700 1 -- 172.21.15.163:0/2997749908 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5a14067320 msgr2=0x7f5a140697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:51.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.061+0000 7f5a2f36f700 1 --2- 172.21.15.163:0/2997749908 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5a14067320 0x7f5a140697d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c0099b0 tx=0x7f5a1c017000 comp rx=0 tx=0).stop 2023-10-09T06:39:51.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.061+0000 7f5a2f36f700 1 -- 172.21.15.163:0/2997749908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a280b53d0 msgr2=0x7f5a2809b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:51.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.061+0000 7f5a2f36f700 1 --2- 172.21.15.163:0/2997749908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a280b53d0 0x7f5a2809b150 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800e0e0 tx=0x7f5a180098d0 comp rx=0 tx=0).stop 2023-10-09T06:39:51.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.061+0000 7f5a2f36f700 1 -- 172.21.15.163:0/2997749908 shutdown_connections 2023-10-09T06:39:51.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.061+0000 7f5a2f36f700 1 --2- 172.21.15.163:0/2997749908 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f5a14067320 0x7f5a140697d0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:51.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.061+0000 7f5a2f36f700 1 --2- 172.21.15.163:0/2997749908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a280b53d0 0x7f5a2809b150 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:51.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.061+0000 7f5a2f36f700 1 --2- 172.21.15.163:0/2997749908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a28159ec0 0x7f5a28155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:51.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.061+0000 7f5a2f36f700 1 --2- 172.21.15.163:0/2997749908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a280cce40 0x7f5a2809d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:51.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.061+0000 7f5a2f36f700 1 -- 172.21.15.163:0/2997749908 >> 172.21.15.163:0/2997749908 conn(0x7f5a280999f0 msgr2=0x7f5a280c58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:51.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.061+0000 7f5a2f36f700 1 -- 172.21.15.163:0/2997749908 shutdown_connections 2023-10-09T06:39:51.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.061+0000 7f5a2f36f700 1 -- 172.21.15.163:0/2997749908 wait complete. 2023-10-09T06:39:51.063 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_metadata' created 2023-10-09T06:39:51.076 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-09T06:39:51.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.197+0000 7fbe601ba700 1 Processor -- start 2023-10-09T06:39:51.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.198+0000 7fbe601ba700 1 -- start start 2023-10-09T06:39:51.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.198+0000 7fbe601ba700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbe580cbb90 0x7fbe580af320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:51.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.198+0000 7fbe601ba700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe580af860 con 0x7fbe58069c70 2023-10-09T06:39:51.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.198+0000 7fbe601ba700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe580ce380 con 0x7fbe58159950 2023-10-09T06:39:51.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.198+0000 7fbe601ba700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe580cec20 con 0x7fbe580cbb90 2023-10-09T06:39:51.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.199+0000 7fbe5d755700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fbe58069c70 0x7fbe580cf0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:36428/0 (socket says 172.21.15.163:36428) 2023-10-09T06:39:51.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.199+0000 7fbe5d755700 1 -- 172.21.15.163:0/4036082758 learned_addr learned my addr 172.21.15.163:0/4036082758 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:51.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.199+0000 7fbe5e757700 1 --2- 172.21.15.163:0/4036082758 >> v2:172.21.15.181:3300/0 conn(0x7fbe580cbb90 0x7fbe580af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:51.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.199+0000 7fbe5cf54700 1 -- 172.21.15.163:0/4036082758 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3881733406 0 0) 0x7fbe580af860 con 0x7fbe58069c70 2023-10-09T06:39:51.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.199+0000 7fbe5cf54700 1 -- 172.21.15.163:0/4036082758 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe44003570 con 0x7fbe58069c70 2023-10-09T06:39:51.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.199+0000 7fbe5cf54700 1 -- 172.21.15.163:0/4036082758 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2416087559 0 0) 0x7fbe580ce380 con 0x7fbe58159950 2023-10-09T06:39:51.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.199+0000 7fbe5cf54700 1 -- 172.21.15.163:0/4036082758 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe580af860 con 0x7fbe58159950 2023-10-09T06:39:51.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.199+0000 7fbe5e757700 1 -- 172.21.15.163:0/4036082758 >> v1:172.21.15.181:6789/0 conn(0x7fbe58159950 legacy=0x7fbe58069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:51.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.199+0000 7fbe5e757700 1 -- 172.21.15.163:0/4036082758 >> v1:172.21.15.163:6789/0 conn(0x7fbe58069c70 legacy=0x7fbe580cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:51.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.199+0000 7fbe5cf54700 1 -- 172.21.15.163:0/4036082758 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2574062179 0 0) 0x7fbe44003570 con 0x7fbe58069c70 2023-10-09T06:39:51.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.199+0000 7fbe5e757700 1 -- 172.21.15.163:0/4036082758 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe580cbfb0 con 0x7fbe580cbb90 2023-10-09T06:39:51.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.199+0000 7fbe5e757700 1 --2- 172.21.15.163:0/4036082758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe580cbb90 0x7fbe580af320 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fbe50009e60 tx=0x7fbe50009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3604e1ee9d071526 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:51.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.199+0000 7fbe5cf54700 1 -- 172.21.15.163:0/4036082758 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5000e070 con 0x7fbe580cbb90 2023-10-09T06:39:51.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.199+0000 7fbe5cf54700 1 -- 172.21.15.163:0/4036082758 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5000b7a0 con 0x7fbe580cbb90 2023-10-09T06:39:51.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.199+0000 7fbe5cf54700 1 -- 172.21.15.163:0/4036082758 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe48001470 con 0x7fbe580cbb90 2023-10-09T06:39:51.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.200+0000 7fbe601ba700 1 -- 172.21.15.163:0/4036082758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe580cbb90 msgr2=0x7fbe580af320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:51.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.200+0000 7fbe601ba700 1 --2- 172.21.15.163:0/4036082758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe580cbb90 0x7fbe580af320 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fbe50009e60 tx=0x7fbe50009780 comp rx=0 tx=0).stop 2023-10-09T06:39:51.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.200+0000 7fbe601ba700 1 -- 172.21.15.163:0/4036082758 shutdown_connections 2023-10-09T06:39:51.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.200+0000 7fbe601ba700 1 --2- 172.21.15.163:0/4036082758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe580cbb90 0x7fbe580af320 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:51.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.200+0000 7fbe601ba700 1 -- 172.21.15.163:0/4036082758 >> 172.21.15.163:0/4036082758 conn(0x7fbe58068e00 msgr2=0x7fbe580d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:51.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.200+0000 7fbe601ba700 1 -- 172.21.15.163:0/4036082758 shutdown_connections 2023-10-09T06:39:51.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.200+0000 7fbe601ba700 1 -- 172.21.15.163:0/4036082758 wait complete. 2023-10-09T06:39:51.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.201+0000 7fbe601ba700 1 Processor -- start 2023-10-09T06:39:51.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.201+0000 7fbe601ba700 1 -- start start 2023-10-09T06:39:51.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.202+0000 7fbe601ba700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe58069c70 0x7fbe580c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:51.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.202+0000 7fbe601ba700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe580cbb90 0x7fbe580b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:51.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.202+0000 7fbe601ba700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe58159950 0x7fbe5816d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:51.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.202+0000 7fbe601ba700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe580c8d70 con 0x7fbe58069c70 2023-10-09T06:39:51.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.202+0000 7fbe601ba700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe580c8f60 con 0x7fbe58159950 2023-10-09T06:39:51.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.202+0000 7fbe601ba700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe580c9240 con 0x7fbe580cbb90 2023-10-09T06:39:51.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.202+0000 7fbe5df56700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe58069c70 0x7fbe580c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:51.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.202+0000 7fbe5df56700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe58069c70 0x7fbe580c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49018/0 (socket says 172.21.15.163:49018) 2023-10-09T06:39:51.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.202+0000 7fbe5df56700 1 -- 172.21.15.163:0/1020422108 learned_addr learned my addr 172.21.15.163:0/1020422108 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:51.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.202+0000 7fbe5d755700 1 --2- 172.21.15.163:0/1020422108 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe580cbb90 0x7fbe580b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:51.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.202+0000 7fbe5e757700 1 --2- 172.21.15.163:0/1020422108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe58159950 0x7fbe5816d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:51.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.202+0000 7fbe5df56700 1 -- 172.21.15.163:0/1020422108 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe580cbb90 msgr2=0x7fbe580b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:51.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.202+0000 7fbe5df56700 1 --2- 172.21.15.163:0/1020422108 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe580cbb90 0x7fbe580b37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:51.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.202+0000 7fbe5df56700 1 -- 172.21.15.163:0/1020422108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe58159950 msgr2=0x7fbe5816d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:51.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.202+0000 7fbe5df56700 1 --2- 172.21.15.163:0/1020422108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe58159950 0x7fbe5816d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:51.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.202+0000 7fbe5df56700 1 -- 172.21.15.163:0/1020422108 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe50009a20 con 0x7fbe58069c70 2023-10-09T06:39:51.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.202+0000 7fbe5df56700 1 --2- 172.21.15.163:0/1020422108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe58069c70 0x7fbe580c8830 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400c580 tx=0x7fbe54007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:51.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.203+0000 7fbe4dffb700 1 -- 172.21.15.163:0/1020422108 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe54010510 con 0x7fbe58069c70 2023-10-09T06:39:51.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.203+0000 7fbe4dffb700 1 -- 172.21.15.163:0/1020422108 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe54010670 con 0x7fbe58069c70 2023-10-09T06:39:51.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.203+0000 7fbe601ba700 1 -- 172.21.15.163:0/1020422108 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe580c9520 con 0x7fbe58069c70 2023-10-09T06:39:51.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.203+0000 7fbe4dffb700 1 -- 172.21.15.163:0/1020422108 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe54010a30 con 0x7fbe58069c70 2023-10-09T06:39:51.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.203+0000 7fbe601ba700 1 -- 172.21.15.163:0/1020422108 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe580c9ae0 con 0x7fbe58069c70 2023-10-09T06:39:51.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.204+0000 7fbe601ba700 1 -- 172.21.15.163:0/1020422108 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe580718b0 con 0x7fbe58069c70 2023-10-09T06:39:51.207 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.204+0000 7fbe4dffb700 1 -- 172.21.15.163:0/1020422108 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbe5401b400 con 0x7fbe58069c70 2023-10-09T06:39:51.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.205+0000 7fbe4dffb700 1 --2- 172.21.15.163:0/1020422108 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbe3c067320 0x7fbe3c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:51.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.205+0000 7fbe4dffb700 1 -- 172.21.15.163:0/1020422108 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fbe54084600 con 0x7fbe58069c70 2023-10-09T06:39:51.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.205+0000 7fbe5d755700 1 --2- 172.21.15.163:0/1020422108 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbe3c067320 0x7fbe3c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:51.208 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.206+0000 7fbe5d755700 1 --2- 172.21.15.163:0/1020422108 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbe3c067320 0x7fbe3c0697d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fbe5815d420 tx=0x7fbe48000bc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:51.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.209+0000 7fbe4dffb700 1 -- 172.21.15.163:0/1020422108 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbe54051470 con 0x7fbe58069c70 2023-10-09T06:39:51.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:51.392+0000 7fbe601ba700 1 -- 172.21.15.163:0/1020422108 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbe580afa90 con 0x7fbe58069c70 2023-10-09T06:39:52.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.066+0000 7fbe4dffb700 1 -- 172.21.15.163:0/1020422108 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v195) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbe54054a90 con 0x7fbe58069c70 2023-10-09T06:39:52.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.072+0000 7fbe601ba700 1 -- 172.21.15.163:0/1020422108 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbe3c067320 msgr2=0x7fbe3c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:52.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.072+0000 7fbe601ba700 1 --2- 172.21.15.163:0/1020422108 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbe3c067320 0x7fbe3c0697d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fbe5815d420 tx=0x7fbe48000bc0 comp rx=0 tx=0).stop 2023-10-09T06:39:52.074 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.072+0000 7fbe601ba700 1 -- 172.21.15.163:0/1020422108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe58069c70 msgr2=0x7fbe580c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:52.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.072+0000 7fbe601ba700 1 --2- 172.21.15.163:0/1020422108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe58069c70 0x7fbe580c8830 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400c580 tx=0x7fbe54007e10 comp rx=0 tx=0).stop 2023-10-09T06:39:52.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.072+0000 7fbe601ba700 1 -- 172.21.15.163:0/1020422108 shutdown_connections 2023-10-09T06:39:52.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.072+0000 7fbe601ba700 1 --2- 172.21.15.163:0/1020422108 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fbe3c067320 0x7fbe3c0697d0 secure :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fbe5815d420 tx=0x7fbe48000bc0 comp rx=0 tx=0).stop 2023-10-09T06:39:52.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.073+0000 7fbe601ba700 1 --2- 172.21.15.163:0/1020422108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe58069c70 0x7fbe580c8830 secure :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400c580 tx=0x7fbe54007e10 comp rx=0 tx=0).stop 2023-10-09T06:39:52.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.073+0000 7fbe601ba700 1 --2- 172.21.15.163:0/1020422108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe58159950 0x7fbe5816d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:52.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.073+0000 7fbe601ba700 1 --2- 172.21.15.163:0/1020422108 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe580cbb90 0x7fbe580b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:52.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.073+0000 7fbe601ba700 1 -- 172.21.15.163:0/1020422108 >> 172.21.15.163:0/1020422108 conn(0x7fbe58068e00 msgr2=0x7fbe580b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:52.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.075+0000 7fbe601ba700 1 -- 172.21.15.163:0/1020422108 shutdown_connections 2023-10-09T06:39:52.076 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.075+0000 7fbe601ba700 1 -- 172.21.15.163:0/1020422108 wait complete. 2023-10-09T06:39:52.081 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_data' created 2023-10-09T06:39:52.097 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-09T06:39:52.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.215+0000 7f8dfd85c700 1 Processor -- start 2023-10-09T06:39:52.216 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.215+0000 7f8dfd85c700 1 -- start start 2023-10-09T06:39:52.218 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.216+0000 7f8dfd85c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8df8159890 0x7f8df815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:52.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.216+0000 7f8dfd85c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8df8159c60 0x7f8df815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:52.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.216+0000 7f8dfd85c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df815a160 con 0x7f8df815db10 2023-10-09T06:39:52.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.216+0000 7f8dfd85c700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df80d13a0 con 0x7f8df8159890 2023-10-09T06:39:52.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.216+0000 7f8dfd85c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df815e140 con 0x7f8df8159c60 2023-10-09T06:39:52.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.216+0000 7f8df6ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8df8159890 0x7f8df815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:52.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.216+0000 7f8df6ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8df8159890 0x7f8df815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49028/0 (socket says 172.21.15.163:49028) 2023-10-09T06:39:52.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.216+0000 7f8df6ffd700 1 -- 172.21.15.163:0/1367331836 learned_addr learned my addr 172.21.15.163:0/1367331836 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:52.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.217+0000 7f8dee7fc700 1 --2- 172.21.15.163:0/1367331836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8df8159c60 0x7f8df815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:52.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.217+0000 7f8df6ffd700 1 -- 172.21.15.163:0/1367331836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8df8159c60 msgr2=0x7f8df815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:52.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.217+0000 7f8df6ffd700 1 --2- 172.21.15.163:0/1367331836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8df8159c60 0x7f8df815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:52.219 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.217+0000 7f8df6ffd700 1 -- 172.21.15.163:0/1367331836 >> v1:172.21.15.181:6789/0 conn(0x7f8df815db10 legacy=0x7f8df8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:52.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.217+0000 7f8df6ffd700 1 -- 172.21.15.163:0/1367331836 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df815a3d0 con 0x7f8df8159890 2023-10-09T06:39:52.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.217+0000 7f8df6ffd700 1 --2- 172.21.15.163:0/1367331836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8df8159890 0x7f8df815abd0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f8de80099b0 tx=0x7f8de8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d963e8b8fa372a server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:52.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.217+0000 7f8df67fc700 1 -- 172.21.15.163:0/1367331836 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de800b860 con 0x7f8df8159890 2023-10-09T06:39:52.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.217+0000 7f8df67fc700 1 -- 172.21.15.163:0/1367331836 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de800b9c0 con 0x7f8df8159890 2023-10-09T06:39:52.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.217+0000 7f8df67fc700 1 -- 172.21.15.163:0/1367331836 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de800bd60 con 0x7f8df8159890 2023-10-09T06:39:52.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.217+0000 7f8dfd85c700 1 -- 172.21.15.163:0/1367331836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8df8159890 msgr2=0x7f8df815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:52.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.217+0000 7f8dfd85c700 1 --2- 172.21.15.163:0/1367331836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8df8159890 0x7f8df815abd0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f8de80099b0 tx=0x7f8de8009250 comp rx=0 tx=0).stop 2023-10-09T06:39:52.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.218+0000 7f8dfd85c700 1 -- 172.21.15.163:0/1367331836 shutdown_connections 2023-10-09T06:39:52.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.218+0000 7f8dfd85c700 1 --2- 172.21.15.163:0/1367331836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8df8159890 0x7f8df815abd0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:52.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.218+0000 7f8dfd85c700 1 --2- 172.21.15.163:0/1367331836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8df8159c60 0x7f8df815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:52.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.218+0000 7f8dfd85c700 1 -- 172.21.15.163:0/1367331836 >> 172.21.15.163:0/1367331836 conn(0x7f8df8068e20 msgr2=0x7f8df80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:52.220 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.218+0000 7f8dfd85c700 1 -- 172.21.15.163:0/1367331836 shutdown_connections 2023-10-09T06:39:52.221 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.218+0000 7f8dfd85c700 1 -- 172.21.15.163:0/1367331836 wait complete. 2023-10-09T06:39:52.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.219+0000 7f8dfd85c700 1 Processor -- start 2023-10-09T06:39:52.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.219+0000 7f8dfd85c700 1 -- start start 2023-10-09T06:39:52.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.220+0000 7f8dfd85c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8df8159890 0x7f8df80c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:52.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.220+0000 7f8dfd85c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8df8159c60 0x7f8df815cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:52.223 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.220+0000 7f8dfd85c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8df815db10 0x7f8df8156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:52.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.220+0000 7f8dfd85c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df8155920 con 0x7f8df8159c60 2023-10-09T06:39:52.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.220+0000 7f8dfd85c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df81575c0 con 0x7f8df815db10 2023-10-09T06:39:52.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.220+0000 7f8dfd85c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df8154ff0 con 0x7f8df8159890 2023-10-09T06:39:52.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.220+0000 7f8dee7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8df8159c60 0x7f8df815cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:52.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.220+0000 7f8dee7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8df8159c60 0x7f8df815cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49044/0 (socket says 172.21.15.163:49044) 2023-10-09T06:39:52.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.220+0000 7f8dee7fc700 1 -- 172.21.15.163:0/3576499036 learned_addr learned my addr 172.21.15.163:0/3576499036 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:52.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.220+0000 7f8df6ffd700 1 --2- 172.21.15.163:0/3576499036 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8df8159890 0x7f8df80c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:52.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.220+0000 7f8df77fe700 1 --2- 172.21.15.163:0/3576499036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8df815db10 0x7f8df8156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:52.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.220+0000 7f8dee7fc700 1 -- 172.21.15.163:0/3576499036 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8df8159890 msgr2=0x7f8df80c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:52.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.220+0000 7f8dee7fc700 1 --2- 172.21.15.163:0/3576499036 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8df8159890 0x7f8df80c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:52.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.220+0000 7f8dee7fc700 1 -- 172.21.15.163:0/3576499036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8df815db10 msgr2=0x7f8df8156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:52.224 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.220+0000 7f8dee7fc700 1 --2- 172.21.15.163:0/3576499036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8df815db10 0x7f8df8156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:52.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.220+0000 7f8dee7fc700 1 -- 172.21.15.163:0/3576499036 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de80094f0 con 0x7f8df8159c60 2023-10-09T06:39:52.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.220+0000 7f8dee7fc700 1 --2- 172.21.15.163:0/3576499036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8df8159c60 0x7f8df815cee0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f8de000d860 tx=0x7f8de0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:52.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.220+0000 7f8def7fe700 1 -- 172.21.15.163:0/3576499036 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de00125f0 con 0x7f8df8159c60 2023-10-09T06:39:52.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.220+0000 7f8def7fe700 1 -- 172.21.15.163:0/3576499036 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de0012750 con 0x7f8df8159c60 2023-10-09T06:39:52.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.220+0000 7f8dfd85c700 1 -- 172.21.15.163:0/3576499036 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df81552d0 con 0x7f8df8159c60 2023-10-09T06:39:52.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.220+0000 7f8def7fe700 1 -- 172.21.15.163:0/3576499036 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de0012af0 con 0x7f8df8159c60 2023-10-09T06:39:52.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.220+0000 7f8dfd85c700 1 -- 172.21.15.163:0/3576499036 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df80cb490 con 0x7f8df8159c60 2023-10-09T06:39:52.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.221+0000 7f8dfd85c700 1 -- 172.21.15.163:0/3576499036 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df8155a90 con 0x7f8df8159c60 2023-10-09T06:39:52.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.222+0000 7f8def7fe700 1 -- 172.21.15.163:0/3576499036 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8de0013480 con 0x7f8df8159c60 2023-10-09T06:39:52.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.222+0000 7f8def7fe700 1 --2- 172.21.15.163:0/3576499036 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8dd8067320 0x7f8dd80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:52.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.222+0000 7f8def7fe700 1 -- 172.21.15.163:0/3576499036 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f8de0084f70 con 0x7f8df8159c60 2023-10-09T06:39:52.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.222+0000 7f8df6ffd700 1 --2- 172.21.15.163:0/3576499036 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8dd8067320 0x7f8dd80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:52.225 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.223+0000 7f8df6ffd700 1 --2- 172.21.15.163:0/3576499036 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8dd8067320 0x7f8dd80697d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8de80099b0 tx=0x7f8de8017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:52.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.226+0000 7f8def7fe700 1 -- 172.21.15.163:0/3576499036 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8de00556f0 con 0x7f8df8159c60 2023-10-09T06:39:52.455 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:52.454+0000 7f8dfd85c700 1 -- 172.21.15.163:0/3576499036 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f8df804c3c0 con 0x7f8df8159c60 2023-10-09T06:39:53.090 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.089+0000 7f8def7fe700 1 -- 172.21.15.163:0/3576499036 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 33 and data pool 34 v129) v1 ==== 174+0+0 (secure 0 0 0) 0x7f8de0055280 con 0x7f8df8159c60 2023-10-09T06:39:53.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.097+0000 7f8dfd85c700 1 -- 172.21.15.163:0/3576499036 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8dd8067320 msgr2=0x7f8dd80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:53.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.097+0000 7f8dfd85c700 1 --2- 172.21.15.163:0/3576499036 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8dd8067320 0x7f8dd80697d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8de80099b0 tx=0x7f8de8017000 comp rx=0 tx=0).stop 2023-10-09T06:39:53.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.097+0000 7f8dfd85c700 1 -- 172.21.15.163:0/3576499036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8df8159c60 msgr2=0x7f8df815cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:53.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.097+0000 7f8dfd85c700 1 --2- 172.21.15.163:0/3576499036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8df8159c60 0x7f8df815cee0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f8de000d860 tx=0x7f8de0008040 comp rx=0 tx=0).stop 2023-10-09T06:39:53.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.098+0000 7f8dfd85c700 1 -- 172.21.15.163:0/3576499036 shutdown_connections 2023-10-09T06:39:53.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.098+0000 7f8dfd85c700 1 --2- 172.21.15.163:0/3576499036 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8dd8067320 0x7f8dd80697d0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:53.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.098+0000 7f8dfd85c700 1 --2- 172.21.15.163:0/3576499036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8df815db10 0x7f8df8156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:53.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.098+0000 7f8dfd85c700 1 --2- 172.21.15.163:0/3576499036 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8df8159890 0x7f8df80c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:53.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.098+0000 7f8dfd85c700 1 --2- 172.21.15.163:0/3576499036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8df8159c60 0x7f8df815cee0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:53.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.098+0000 7f8dfd85c700 1 -- 172.21.15.163:0/3576499036 >> 172.21.15.163:0/3576499036 conn(0x7f8df8068e20 msgr2=0x7f8df8154150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:53.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.098+0000 7f8dfd85c700 1 -- 172.21.15.163:0/3576499036 shutdown_connections 2023-10-09T06:39:53.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.098+0000 7f8dfd85c700 1 -- 172.21.15.163:0/3576499036 wait complete. 2023-10-09T06:39:53.100 INFO:teuthology.orchestra.run.smithi163.stderr:new fs with metadata pool 33 and data pool 34 2023-10-09T06:39:53.113 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:39:53.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.227+0000 7f0f78c68700 1 Processor -- start 2023-10-09T06:39:53.228 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.227+0000 7f0f78c68700 1 -- start start 2023-10-09T06:39:53.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.229+0000 7f0f78c68700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f74159c60 0x7f0f7415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:53.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.229+0000 7f0f78c68700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f7415a160 con 0x7f0f74159890 2023-10-09T06:39:53.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.229+0000 7f0f78c68700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f740d13a0 con 0x7f0f7415db10 2023-10-09T06:39:53.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.229+0000 7f0f78c68700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7415e140 con 0x7f0f74159c60 2023-10-09T06:39:53.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.229+0000 7f0f7259c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f0f74159890 0x7f0f7415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:36434/0 (socket says 172.21.15.163:36434) 2023-10-09T06:39:53.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.229+0000 7f0f7259c700 1 -- 172.21.15.163:0/64967542 learned_addr learned my addr 172.21.15.163:0/64967542 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:53.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.229+0000 7f0f69d9b700 1 --2- 172.21.15.163:0/64967542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f74159c60 0x7f0f7415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:53.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.229+0000 7f0f71d9b700 1 -- 172.21.15.163:0/64967542 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 141914861 0 0) 0x7f0f7415a160 con 0x7f0f74159890 2023-10-09T06:39:53.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.229+0000 7f0f71d9b700 1 -- 172.21.15.163:0/64967542 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f60003570 con 0x7f0f74159890 2023-10-09T06:39:53.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.229+0000 7f0f71d9b700 1 -- 172.21.15.163:0/64967542 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1362105925 0 0) 0x7f0f740d13a0 con 0x7f0f7415db10 2023-10-09T06:39:53.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.229+0000 7f0f71d9b700 1 -- 172.21.15.163:0/64967542 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f7415a160 con 0x7f0f7415db10 2023-10-09T06:39:53.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.229+0000 7f0f71d9b700 1 -- 172.21.15.163:0/64967542 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1175412232 0 0) 0x7f0f60003570 con 0x7f0f74159890 2023-10-09T06:39:53.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.229+0000 7f0f69d9b700 1 -- 172.21.15.163:0/64967542 >> v1:172.21.15.181:6789/0 conn(0x7f0f7415db10 legacy=0x7f0f74069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:53.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.229+0000 7f0f69d9b700 1 -- 172.21.15.163:0/64967542 >> v1:172.21.15.163:6789/0 conn(0x7f0f74159890 legacy=0x7f0f7415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:53.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.229+0000 7f0f69d9b700 1 -- 172.21.15.163:0/64967542 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7415a3d0 con 0x7f0f74159c60 2023-10-09T06:39:53.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.229+0000 7f0f69d9b700 1 --2- 172.21.15.163:0/64967542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f74159c60 0x7f0f7415d5d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c009910 tx=0x7f0f5c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45dbdfabb504b854 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:53.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.229+0000 7f0f71d9b700 1 -- 172.21.15.163:0/64967542 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f640013f0 con 0x7f0f74159c60 2023-10-09T06:39:53.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.229+0000 7f0f71d9b700 1 -- 172.21.15.163:0/64967542 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5c011420 con 0x7f0f74159c60 2023-10-09T06:39:53.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.229+0000 7f0f71d9b700 1 -- 172.21.15.163:0/64967542 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c0117c0 con 0x7f0f74159c60 2023-10-09T06:39:53.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.229+0000 7f0f78c68700 1 -- 172.21.15.163:0/64967542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f74159c60 msgr2=0x7f0f7415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:53.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.229+0000 7f0f78c68700 1 --2- 172.21.15.163:0/64967542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f74159c60 0x7f0f7415d5d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c009910 tx=0x7f0f5c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:53.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.230+0000 7f0f78c68700 1 -- 172.21.15.163:0/64967542 shutdown_connections 2023-10-09T06:39:53.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.230+0000 7f0f78c68700 1 --2- 172.21.15.163:0/64967542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f74159c60 0x7f0f7415d5d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:53.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.230+0000 7f0f78c68700 1 -- 172.21.15.163:0/64967542 >> 172.21.15.163:0/64967542 conn(0x7f0f74068e20 msgr2=0x7f0f740d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:53.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.230+0000 7f0f78c68700 1 -- 172.21.15.163:0/64967542 shutdown_connections 2023-10-09T06:39:53.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.230+0000 7f0f78c68700 1 -- 172.21.15.163:0/64967542 wait complete. 2023-10-09T06:39:53.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.231+0000 7f0f78c68700 1 Processor -- start 2023-10-09T06:39:53.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.231+0000 7f0f78c68700 1 -- start start 2023-10-09T06:39:53.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.231+0000 7f0f78c68700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f74159890 0x7f0f740cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:53.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.231+0000 7f0f78c68700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f74159c60 0x7f0f740b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:53.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.232+0000 7f0f78c68700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f7415db10 0x7f0f74155360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:53.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.232+0000 7f0f78c68700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f740ce970 con 0x7f0f7415db10 2023-10-09T06:39:53.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.232+0000 7f0f78c68700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f740cec50 con 0x7f0f74159c60 2023-10-09T06:39:53.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.232+0000 7f0f78c68700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f740cef30 con 0x7f0f74159890 2023-10-09T06:39:53.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.232+0000 7f0f72d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f7415db10 0x7f0f74155360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:53.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.232+0000 7f0f72d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f7415db10 0x7f0f74155360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49060/0 (socket says 172.21.15.163:49060) 2023-10-09T06:39:53.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.232+0000 7f0f72d9d700 1 -- 172.21.15.163:0/4233758485 learned_addr learned my addr 172.21.15.163:0/4233758485 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:53.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.232+0000 7f0f7259c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f74159890 0x7f0f740cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:53.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.232+0000 7f0f69d9b700 1 --2- 172.21.15.163:0/4233758485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f74159c60 0x7f0f740b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:53.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.232+0000 7f0f72d9d700 1 -- 172.21.15.163:0/4233758485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f74159890 msgr2=0x7f0f740cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:53.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.232+0000 7f0f72d9d700 1 --2- 172.21.15.163:0/4233758485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f74159890 0x7f0f740cdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:53.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.232+0000 7f0f72d9d700 1 -- 172.21.15.163:0/4233758485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f74159c60 msgr2=0x7f0f740b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:53.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.232+0000 7f0f72d9d700 1 --2- 172.21.15.163:0/4233758485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f74159c60 0x7f0f740b5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:53.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.232+0000 7f0f72d9d700 1 -- 172.21.15.163:0/4233758485 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5c009420 con 0x7f0f7415db10 2023-10-09T06:39:53.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.232+0000 7f0f72d9d700 1 --2- 172.21.15.163:0/4233758485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f7415db10 0x7f0f74155360 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00c900 tx=0x7f0f6c002780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:53.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.232+0000 7f0f6affd700 1 -- 172.21.15.163:0/4233758485 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c012630 con 0x7f0f7415db10 2023-10-09T06:39:53.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.232+0000 7f0f6affd700 1 -- 172.21.15.163:0/4233758485 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f6c012790 con 0x7f0f7415db10 2023-10-09T06:39:53.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.232+0000 7f0f6affd700 1 -- 172.21.15.163:0/4233758485 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c012b50 con 0x7f0f7415db10 2023-10-09T06:39:53.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.232+0000 7f0f78c68700 1 -- 172.21.15.163:0/4233758485 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f740cf210 con 0x7f0f7415db10 2023-10-09T06:39:53.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.232+0000 7f0f78c68700 1 -- 172.21.15.163:0/4233758485 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f741558a0 con 0x7f0f7415db10 2023-10-09T06:39:53.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.233+0000 7f0f78c68700 1 -- 172.21.15.163:0/4233758485 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f740cf400 con 0x7f0f7415db10 2023-10-09T06:39:53.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.234+0000 7f0f6affd700 1 -- 172.21.15.163:0/4233758485 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0f6c01b400 con 0x7f0f7415db10 2023-10-09T06:39:53.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.234+0000 7f0f6affd700 1 --2- 172.21.15.163:0/4233758485 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0f54067320 0x7f0f540697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:53.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.234+0000 7f0f6affd700 1 -- 172.21.15.163:0/4233758485 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0f6c084f50 con 0x7f0f7415db10 2023-10-09T06:39:53.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.234+0000 7f0f7259c700 1 --2- 172.21.15.163:0/4233758485 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0f54067320 0x7f0f540697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:53.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.234+0000 7f0f7259c700 1 --2- 172.21.15.163:0/4233758485 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0f54067320 0x7f0f540697d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f0f640011e0 tx=0x7f0f640015a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:53.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.239+0000 7f0f6affd700 1 -- 172.21.15.163:0/4233758485 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0f6c051be0 con 0x7f0f7415db10 2023-10-09T06:39:53.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.423+0000 7f0f78c68700 1 -- 172.21.15.163:0/4233758485 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f740ceab0 con 0x7f0f7415db10 2023-10-09T06:39:53.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.424+0000 7f0f6affd700 1 -- 172.21.15.163:0/4233758485 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v196) v1 ==== 74+0+15425 (secure 0 0 0) 0x7f0f740ceab0 con 0x7f0f7415db10 2023-10-09T06:39:53.425 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:53.425 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":196,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:39:52.456400+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:39:50.786303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":34,"pool_name":"cephfs_data","create_time":"2023-10-09T06:39:51.411021+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:39:53.427 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.426+0000 7f0f78c68700 1 -- 172.21.15.163:0/4233758485 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0f54067320 msgr2=0x7f0f540697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:53.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.426+0000 7f0f78c68700 1 --2- 172.21.15.163:0/4233758485 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0f54067320 0x7f0f540697d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f0f640011e0 tx=0x7f0f640015a0 comp rx=0 tx=0).stop 2023-10-09T06:39:53.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.426+0000 7f0f78c68700 1 -- 172.21.15.163:0/4233758485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f7415db10 msgr2=0x7f0f74155360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:53.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.426+0000 7f0f78c68700 1 --2- 172.21.15.163:0/4233758485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f7415db10 0x7f0f74155360 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00c900 tx=0x7f0f6c002780 comp rx=0 tx=0).stop 2023-10-09T06:39:53.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.426+0000 7f0f78c68700 1 -- 172.21.15.163:0/4233758485 shutdown_connections 2023-10-09T06:39:53.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.426+0000 7f0f78c68700 1 --2- 172.21.15.163:0/4233758485 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0f54067320 0x7f0f540697d0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:53.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.426+0000 7f0f78c68700 1 --2- 172.21.15.163:0/4233758485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f7415db10 0x7f0f74155360 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:53.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.426+0000 7f0f78c68700 1 --2- 172.21.15.163:0/4233758485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f74159890 0x7f0f740cdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:53.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.426+0000 7f0f78c68700 1 --2- 172.21.15.163:0/4233758485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f74159c60 0x7f0f740b5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:53.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.426+0000 7f0f78c68700 1 -- 172.21.15.163:0/4233758485 >> 172.21.15.163:0/4233758485 conn(0x7f0f74068e20 msgr2=0x7f0f74158aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:53.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.426+0000 7f0f78c68700 1 -- 172.21.15.163:0/4233758485 shutdown_connections 2023-10-09T06:39:53.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.426+0000 7f0f78c68700 1 -- 172.21.15.163:0/4233758485 wait complete. 2023-10-09T06:39:53.440 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:39:53.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.554+0000 7fe5ef47e700 1 Processor -- start 2023-10-09T06:39:53.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.555+0000 7fe5ef47e700 1 -- start start 2023-10-09T06:39:53.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.555+0000 7fe5ef47e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e806bcd0 0x7fe5e8140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:53.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.555+0000 7fe5ef47e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe5e806c0a0 0x7fe5e8141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:53.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.555+0000 7fe5ef47e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe5e80c8e00 0x7fe5e80c9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:53.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.555+0000 7fe5ef47e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e815c2a0 con 0x7fe5e806c0a0 2023-10-09T06:39:53.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.555+0000 7fe5ef47e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e815b690 con 0x7fe5e80c8e00 2023-10-09T06:39:53.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.555+0000 7fe5ef47e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e815b970 con 0x7fe5e806bcd0 2023-10-09T06:39:53.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.556+0000 7fe5eca19700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe5e806c0a0 0x7fe5e8141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:53.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.556+0000 7fe5eca19700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe5e806c0a0 0x7fe5e8141330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49066/0 (socket says 172.21.15.163:49066) 2023-10-09T06:39:53.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.556+0000 7fe5eca19700 1 -- 172.21.15.163:0/4135422965 learned_addr learned my addr 172.21.15.163:0/4135422965 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:53.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.556+0000 7fe5ed21a700 1 --2- 172.21.15.163:0/4135422965 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e806bcd0 0x7fe5e8140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:53.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.556+0000 7fe5eda1b700 1 --2- 172.21.15.163:0/4135422965 >> v2:172.21.15.181:3300/0 conn(0x7fe5e80c8e00 0x7fe5e80c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:53.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.556+0000 7fe5eca19700 1 -- 172.21.15.163:0/4135422965 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e806bcd0 msgr2=0x7fe5e8140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:53.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.556+0000 7fe5eca19700 1 --2- 172.21.15.163:0/4135422965 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e806bcd0 0x7fe5e8140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:53.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.556+0000 7fe5eca19700 1 -- 172.21.15.163:0/4135422965 >> v2:172.21.15.181:3300/0 conn(0x7fe5e80c8e00 msgr2=0x7fe5e80c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:53.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.556+0000 7fe5eca19700 1 --2- 172.21.15.163:0/4135422965 >> v2:172.21.15.181:3300/0 conn(0x7fe5e80c8e00 0x7fe5e80c9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:53.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.556+0000 7fe5eca19700 1 -- 172.21.15.163:0/4135422965 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e815bc50 con 0x7fe5e806c0a0 2023-10-09T06:39:53.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.556+0000 7fe5eda1b700 1 --2- 172.21.15.163:0/4135422965 >> v2:172.21.15.181:3300/0 conn(0x7fe5e80c8e00 0x7fe5e80c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:53.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.556+0000 7fe5eca19700 1 --2- 172.21.15.163:0/4135422965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e806c0a0 0x7fe5e8141330 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fe5e815b530 tx=0x7fe5d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8105186ee725f9b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:53.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.556+0000 7fe5dffff700 1 -- 172.21.15.163:0/4135422965 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8009e70 con 0x7fe5e806c0a0 2023-10-09T06:39:53.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.556+0000 7fe5dffff700 1 -- 172.21.15.163:0/4135422965 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d800b860 con 0x7fe5e806c0a0 2023-10-09T06:39:53.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.556+0000 7fe5dffff700 1 -- 172.21.15.163:0/4135422965 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d800bc00 con 0x7fe5e806c0a0 2023-10-09T06:39:53.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.557+0000 7fe5ef47e700 1 -- 172.21.15.163:0/4135422965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e806c0a0 msgr2=0x7fe5e8141330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:53.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.557+0000 7fe5ef47e700 1 --2- 172.21.15.163:0/4135422965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e806c0a0 0x7fe5e8141330 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fe5e815b530 tx=0x7fe5d8009250 comp rx=0 tx=0).stop 2023-10-09T06:39:53.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.557+0000 7fe5ef47e700 1 -- 172.21.15.163:0/4135422965 shutdown_connections 2023-10-09T06:39:53.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.557+0000 7fe5ef47e700 1 --2- 172.21.15.163:0/4135422965 >> v2:172.21.15.181:3300/0 conn(0x7fe5e80c8e00 0x7fe5e80c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:53.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.557+0000 7fe5ef47e700 1 --2- 172.21.15.163:0/4135422965 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e806bcd0 0x7fe5e8140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:53.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.557+0000 7fe5ef47e700 1 --2- 172.21.15.163:0/4135422965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e806c0a0 0x7fe5e8141330 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:53.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.557+0000 7fe5ef47e700 1 -- 172.21.15.163:0/4135422965 >> 172.21.15.163:0/4135422965 conn(0x7fe5e80ce600 msgr2=0x7fe5e80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:53.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.557+0000 7fe5ef47e700 1 -- 172.21.15.163:0/4135422965 shutdown_connections 2023-10-09T06:39:53.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.557+0000 7fe5ef47e700 1 -- 172.21.15.163:0/4135422965 wait complete. 2023-10-09T06:39:53.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.558+0000 7fe5ef47e700 1 Processor -- start 2023-10-09T06:39:53.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.558+0000 7fe5ef47e700 1 -- start start 2023-10-09T06:39:53.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.560+0000 7fe5ef47e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e806bcd0 0x7fe5e815a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:53.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.560+0000 7fe5ef47e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e806c0a0 0x7fe5e8157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:53.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.560+0000 7fe5ef47e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e80c8e00 0x7fe5e8167ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:53.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.560+0000 7fe5ef47e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e815b150 con 0x7fe5e806c0a0 2023-10-09T06:39:53.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.560+0000 7fe5ef47e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e815ced0 con 0x7fe5e806bcd0 2023-10-09T06:39:53.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.560+0000 7fe5ef47e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e815ae70 con 0x7fe5e80c8e00 2023-10-09T06:39:53.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.560+0000 7fe5eca19700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e806c0a0 0x7fe5e8157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:53.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.560+0000 7fe5eca19700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e806c0a0 0x7fe5e8157680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49072/0 (socket says 172.21.15.163:49072) 2023-10-09T06:39:53.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.560+0000 7fe5eca19700 1 -- 172.21.15.163:0/3944330545 learned_addr learned my addr 172.21.15.163:0/3944330545 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:53.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.560+0000 7fe5ed21a700 1 --2- 172.21.15.163:0/3944330545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e806bcd0 0x7fe5e815a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:53.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.560+0000 7fe5eda1b700 1 --2- 172.21.15.163:0/3944330545 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e80c8e00 0x7fe5e8167ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:53.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.560+0000 7fe5eca19700 1 -- 172.21.15.163:0/3944330545 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e80c8e00 msgr2=0x7fe5e8167ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:53.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.560+0000 7fe5eca19700 1 --2- 172.21.15.163:0/3944330545 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e80c8e00 0x7fe5e8167ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:53.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.560+0000 7fe5eca19700 1 -- 172.21.15.163:0/3944330545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e806bcd0 msgr2=0x7fe5e815a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:53.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.560+0000 7fe5eca19700 1 --2- 172.21.15.163:0/3944330545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e806bcd0 0x7fe5e815a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:53.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.560+0000 7fe5eca19700 1 -- 172.21.15.163:0/3944330545 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d80094f0 con 0x7fe5e806c0a0 2023-10-09T06:39:53.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.560+0000 7fe5ed21a700 1 --2- 172.21.15.163:0/3944330545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e806bcd0 0x7fe5e815a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:53.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.560+0000 7fe5eca19700 1 --2- 172.21.15.163:0/3944330545 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e806c0a0 0x7fe5e8157680 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80090b0 tx=0x7fe5d8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:53.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.560+0000 7fe5dd7fa700 1 -- 172.21.15.163:0/3944330545 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d800bbe0 con 0x7fe5e806c0a0 2023-10-09T06:39:53.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.560+0000 7fe5ef47e700 1 -- 172.21.15.163:0/3944330545 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e815c880 con 0x7fe5e806c0a0 2023-10-09T06:39:53.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.560+0000 7fe5dd7fa700 1 -- 172.21.15.163:0/3944330545 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d801f070 con 0x7fe5e806c0a0 2023-10-09T06:39:53.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.560+0000 7fe5dd7fa700 1 -- 172.21.15.163:0/3944330545 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d80266d0 con 0x7fe5e806c0a0 2023-10-09T06:39:53.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.560+0000 7fe5ef47e700 1 -- 172.21.15.163:0/3944330545 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e81595d0 con 0x7fe5e806c0a0 2023-10-09T06:39:53.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.561+0000 7fe5ef47e700 1 -- 172.21.15.163:0/3944330545 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e80c9570 con 0x7fe5e806c0a0 2023-10-09T06:39:53.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.561+0000 7fe5dd7fa700 1 -- 172.21.15.163:0/3944330545 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe5d801a430 con 0x7fe5e806c0a0 2023-10-09T06:39:53.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.562+0000 7fe5dd7fa700 1 --2- 172.21.15.163:0/3944330545 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe5cc067320 0x7fe5cc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:53.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.562+0000 7fe5ed21a700 1 --2- 172.21.15.163:0/3944330545 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe5cc067320 0x7fe5cc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:53.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.562+0000 7fe5ed21a700 1 --2- 172.21.15.163:0/3944330545 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe5cc067320 0x7fe5cc0697d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe5e40076c0 tx=0x7fe5e400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:53.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.563+0000 7fe5dd7fa700 1 -- 172.21.15.163:0/3944330545 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe5d8021070 con 0x7fe5e806c0a0 2023-10-09T06:39:53.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.568+0000 7fe5dd7fa700 1 -- 172.21.15.163:0/3944330545 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5d80537a0 con 0x7fe5e806c0a0 2023-10-09T06:39:53.753 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.752+0000 7fe5ef47e700 1 -- 172.21.15.163:0/3944330545 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5e804c3c0 con 0x7fe5e806c0a0 2023-10-09T06:39:53.754 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.753+0000 7fe5dd7fa700 1 -- 172.21.15.163:0/3944330545 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v196) v1 ==== 74+0+15425 (secure 0 0 0) 0x7fe5d8015090 con 0x7fe5e806c0a0 2023-10-09T06:39:53.754 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:53.755 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":196,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:39:52.456400+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:39:50.786303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":34,"pool_name":"cephfs_data","create_time":"2023-10-09T06:39:51.411021+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:39:53.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.755+0000 7fe5ef47e700 1 -- 172.21.15.163:0/3944330545 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe5cc067320 msgr2=0x7fe5cc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:53.756 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.755+0000 7fe5ef47e700 1 --2- 172.21.15.163:0/3944330545 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe5cc067320 0x7fe5cc0697d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe5e40076c0 tx=0x7fe5e400b040 comp rx=0 tx=0).stop 2023-10-09T06:39:53.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.755+0000 7fe5ef47e700 1 -- 172.21.15.163:0/3944330545 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e806c0a0 msgr2=0x7fe5e8157680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:53.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.755+0000 7fe5ef47e700 1 --2- 172.21.15.163:0/3944330545 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e806c0a0 0x7fe5e8157680 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80090b0 tx=0x7fe5d8017000 comp rx=0 tx=0).stop 2023-10-09T06:39:53.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.755+0000 7fe5ef47e700 1 -- 172.21.15.163:0/3944330545 shutdown_connections 2023-10-09T06:39:53.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.755+0000 7fe5ef47e700 1 --2- 172.21.15.163:0/3944330545 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe5cc067320 0x7fe5cc0697d0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:53.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.755+0000 7fe5ef47e700 1 --2- 172.21.15.163:0/3944330545 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e80c8e00 0x7fe5e8167ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:53.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.755+0000 7fe5ef47e700 1 --2- 172.21.15.163:0/3944330545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e806bcd0 0x7fe5e815a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:53.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.755+0000 7fe5ef47e700 1 --2- 172.21.15.163:0/3944330545 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e806c0a0 0x7fe5e8157680 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:53.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.755+0000 7fe5ef47e700 1 -- 172.21.15.163:0/3944330545 >> 172.21.15.163:0/3944330545 conn(0x7fe5e80ce600 msgr2=0x7fe5e80cc150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:53.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.755+0000 7fe5ef47e700 1 -- 172.21.15.163:0/3944330545 shutdown_connections 2023-10-09T06:39:53.757 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.755+0000 7fe5ef47e700 1 -- 172.21.15.163:0/3944330545 wait complete. 2023-10-09T06:39:53.769 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-09T06:39:53.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.887+0000 7f7982a11700 1 Processor -- start 2023-10-09T06:39:53.887 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.887+0000 7f7982a11700 1 -- start start 2023-10-09T06:39:53.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.888+0000 7f7982a11700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f797c159890 0x7f797c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:53.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.888+0000 7f7982a11700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f797c15a160 con 0x7f797c069560 2023-10-09T06:39:53.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.888+0000 7f7982a11700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f797c0d13a0 con 0x7f797c159c60 2023-10-09T06:39:53.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.888+0000 7f7982a11700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f797c15e140 con 0x7f797c159890 2023-10-09T06:39:53.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.888+0000 7f797bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f797c159890 0x7f797c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:53.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.888+0000 7f7980fae700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f797c069560 0x7f797c0ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:36448/0 (socket says 172.21.15.163:36448) 2023-10-09T06:39:53.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.888+0000 7f7980fae700 1 -- 172.21.15.163:0/417342746 learned_addr learned my addr 172.21.15.163:0/417342746 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:53.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.888+0000 7f797bfff700 1 -- 172.21.15.163:0/417342746 >> v1:172.21.15.181:6789/0 conn(0x7f797c159c60 legacy=0x7f797c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:53.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.888+0000 7f797bfff700 1 -- 172.21.15.163:0/417342746 >> v1:172.21.15.163:6789/0 conn(0x7f797c069560 legacy=0x7f797c0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:53.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.888+0000 7f797affd700 1 -- 172.21.15.163:0/417342746 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1982263604 0 0) 0x7f797c15a160 con 0x7f797c069560 2023-10-09T06:39:53.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.889+0000 7f797bfff700 1 -- 172.21.15.163:0/417342746 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797c15a3d0 con 0x7f797c159890 2023-10-09T06:39:53.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.889+0000 7f797bfff700 1 --2- 172.21.15.163:0/417342746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f797c159890 0x7f797c15abd0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f79700098e0 tx=0x7f79700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8baf39620b061727 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:53.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.889+0000 7f797affd700 1 -- 172.21.15.163:0/417342746 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7970009e80 con 0x7f797c159890 2023-10-09T06:39:53.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.889+0000 7f797affd700 1 -- 172.21.15.163:0/417342746 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797000b820 con 0x7f797c159890 2023-10-09T06:39:53.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.889+0000 7f797affd700 1 -- 172.21.15.163:0/417342746 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797000bba0 con 0x7f797c159890 2023-10-09T06:39:53.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.889+0000 7f7982a11700 1 -- 172.21.15.163:0/417342746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f797c159890 msgr2=0x7f797c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:53.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.889+0000 7f7982a11700 1 --2- 172.21.15.163:0/417342746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f797c159890 0x7f797c15abd0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f79700098e0 tx=0x7f79700091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:53.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.889+0000 7f7982a11700 1 -- 172.21.15.163:0/417342746 shutdown_connections 2023-10-09T06:39:53.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.889+0000 7f7982a11700 1 --2- 172.21.15.163:0/417342746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f797c159890 0x7f797c15abd0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:53.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.889+0000 7f7982a11700 1 -- 172.21.15.163:0/417342746 >> 172.21.15.163:0/417342746 conn(0x7f797c068e20 msgr2=0x7f797c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:53.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.889+0000 7f7982a11700 1 -- 172.21.15.163:0/417342746 shutdown_connections 2023-10-09T06:39:53.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.889+0000 7f7982a11700 1 -- 172.21.15.163:0/417342746 wait complete. 2023-10-09T06:39:53.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.890+0000 7f7982a11700 1 Processor -- start 2023-10-09T06:39:53.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.891+0000 7f7982a11700 1 -- start start 2023-10-09T06:39:53.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.891+0000 7f7982a11700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f797c069560 0x7f797c0c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:53.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.891+0000 7f7982a11700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f797c159890 0x7f797c0ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:53.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.891+0000 7f7982a11700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f797c159c60 0x7f797c169b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:53.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.891+0000 7f7982a11700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c16ab80 con 0x7f797c069560 2023-10-09T06:39:53.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.891+0000 7f7982a11700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c16a0a0 con 0x7f797c159c60 2023-10-09T06:39:53.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.891+0000 7f7982a11700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f797c0b4880 con 0x7f797c159890 2023-10-09T06:39:53.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.891+0000 7f797bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f797c069560 0x7f797c0c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:53.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.891+0000 7f797bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f797c069560 0x7f797c0c8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49090/0 (socket says 172.21.15.163:49090) 2023-10-09T06:39:53.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.891+0000 7f797bfff700 1 -- 172.21.15.163:0/3844584042 learned_addr learned my addr 172.21.15.163:0/3844584042 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:53.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.892+0000 7f797bfff700 1 -- 172.21.15.163:0/3844584042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f797c159890 msgr2=0x7f797c0ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:53.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.892+0000 7f797bfff700 1 --2- 172.21.15.163:0/3844584042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f797c159890 0x7f797c0ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:53.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.892+0000 7f797bfff700 1 -- 172.21.15.163:0/3844584042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f797c159c60 msgr2=0x7f797c169b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:53.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.892+0000 7f797bfff700 1 --2- 172.21.15.163:0/3844584042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f797c159c60 0x7f797c169b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:53.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.892+0000 7f797bfff700 1 -- 172.21.15.163:0/3844584042 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7970009420 con 0x7f797c069560 2023-10-09T06:39:53.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.892+0000 7f797bfff700 1 --2- 172.21.15.163:0/3844584042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f797c069560 0x7f797c0c8a40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f79700030c0 tx=0x7f7970016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:53.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.892+0000 7f7963fff700 1 -- 172.21.15.163:0/3844584042 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797000ba10 con 0x7f797c069560 2023-10-09T06:39:53.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.892+0000 7f7963fff700 1 -- 172.21.15.163:0/3844584042 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7970020070 con 0x7f797c069560 2023-10-09T06:39:53.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.892+0000 7f7982a11700 1 -- 172.21.15.163:0/3844584042 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f797c16a2e0 con 0x7f797c069560 2023-10-09T06:39:53.896 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.892+0000 7f7963fff700 1 -- 172.21.15.163:0/3844584042 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797001b680 con 0x7f797c069560 2023-10-09T06:39:53.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.892+0000 7f7982a11700 1 -- 172.21.15.163:0/3844584042 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f797c16a8a0 con 0x7f797c069560 2023-10-09T06:39:53.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.893+0000 7f7982a11700 1 -- 172.21.15.163:0/3844584042 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f797c0b4650 con 0x7f797c069560 2023-10-09T06:39:53.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.893+0000 7f7963fff700 1 -- 172.21.15.163:0/3844584042 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7970022030 con 0x7f797c069560 2023-10-09T06:39:53.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.894+0000 7f7963fff700 1 --2- 172.21.15.163:0/3844584042 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7968067320 0x7f79680697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:53.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.894+0000 7f7963fff700 1 -- 172.21.15.163:0/3844584042 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7970086b30 con 0x7f797c069560 2023-10-09T06:39:53.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.894+0000 7f797b7fe700 1 --2- 172.21.15.163:0/3844584042 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7968067320 0x7f79680697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:53.897 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.894+0000 7f797b7fe700 1 --2- 172.21.15.163:0/3844584042 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7968067320 0x7f79680697d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f797c0b08a0 tx=0x7f796c006ce0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:53.898 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:53.898+0000 7f7963fff700 1 -- 172.21.15.163:0/3844584042 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f797c0b4650 con 0x7f797c069560 2023-10-09T06:39:54.125 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:54.125+0000 7f7982a11700 1 -- 172.21.15.163:0/3844584042 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f797c0b4650 con 0x7f797c069560 2023-10-09T06:39:55.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.093+0000 7f7963fff700 1 -- 172.21.15.163:0/3844584042 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v131) v1 ==== 118+0+0 (secure 0 0 0) 0x7f797c0b4650 con 0x7f797c069560 2023-10-09T06:39:55.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.096+0000 7f7982a11700 1 -- 172.21.15.163:0/3844584042 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7968067320 msgr2=0x7f79680697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:55.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.096+0000 7f7982a11700 1 --2- 172.21.15.163:0/3844584042 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7968067320 0x7f79680697d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f797c0b08a0 tx=0x7f796c006ce0 comp rx=0 tx=0).stop 2023-10-09T06:39:55.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.096+0000 7f7982a11700 1 -- 172.21.15.163:0/3844584042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f797c069560 msgr2=0x7f797c0c8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:55.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.096+0000 7f7982a11700 1 --2- 172.21.15.163:0/3844584042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f797c069560 0x7f797c0c8a40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f79700030c0 tx=0x7f7970016000 comp rx=0 tx=0).stop 2023-10-09T06:39:55.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.096+0000 7f7982a11700 1 -- 172.21.15.163:0/3844584042 shutdown_connections 2023-10-09T06:39:55.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.096+0000 7f7982a11700 1 --2- 172.21.15.163:0/3844584042 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7968067320 0x7f79680697d0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:55.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.096+0000 7f7982a11700 1 --2- 172.21.15.163:0/3844584042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f797c069560 0x7f797c0c8a40 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:55.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.096+0000 7f7982a11700 1 --2- 172.21.15.163:0/3844584042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f797c159c60 0x7f797c169b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:55.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.096+0000 7f7982a11700 1 --2- 172.21.15.163:0/3844584042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f797c159890 0x7f797c0ca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:55.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.097+0000 7f7982a11700 1 -- 172.21.15.163:0/3844584042 >> 172.21.15.163:0/3844584042 conn(0x7f797c068e20 msgr2=0x7f797c153060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:55.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.097+0000 7f7982a11700 1 -- 172.21.15.163:0/3844584042 shutdown_connections 2023-10-09T06:39:55.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.097+0000 7f7982a11700 1 -- 172.21.15.163:0/3844584042 wait complete. 2023-10-09T06:39:55.112 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-09T06:39:55.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.236+0000 7f7c0906a700 1 Processor -- start 2023-10-09T06:39:55.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.236+0000 7f7c0906a700 1 -- start start 2023-10-09T06:39:55.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.237+0000 7f7c0906a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c0406bcd0 0x7f7c04140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:55.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.237+0000 7f7c0906a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7c040c9510 0x7f7c04141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:55.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.237+0000 7f7c0906a700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c0415c2a0 con 0x7f7c0406c0a0 2023-10-09T06:39:55.240 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.237+0000 7f7c0906a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c0415b690 con 0x7f7c040c9510 2023-10-09T06:39:55.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.237+0000 7f7c0906a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0415b970 con 0x7f7c0406bcd0 2023-10-09T06:39:55.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.237+0000 7f7c0259c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f7c0406c0a0 0x7f7c040c8e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:36450/0 (socket says 172.21.15.163:36450) 2023-10-09T06:39:55.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.237+0000 7f7c0259c700 1 -- 172.21.15.163:0/3206755669 learned_addr learned my addr 172.21.15.163:0/3206755669 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:55.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.237+0000 7f7c0359e700 1 --2- 172.21.15.163:0/3206755669 >> v2:172.21.15.181:3300/0 conn(0x7f7c040c9510 0x7f7c04141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:55.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.237+0000 7f7c02d9d700 1 --2- 172.21.15.163:0/3206755669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c0406bcd0 0x7f7c04140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:55.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.237+0000 7f7c01d9b700 1 -- 172.21.15.163:0/3206755669 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 147816819 0 0) 0x7f7c0415c2a0 con 0x7f7c0406c0a0 2023-10-09T06:39:55.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.237+0000 7f7c01d9b700 1 -- 172.21.15.163:0/3206755669 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bec003570 con 0x7f7c0406c0a0 2023-10-09T06:39:55.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.238+0000 7f7c01d9b700 1 -- 172.21.15.163:0/3206755669 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 214910416 0 0) 0x7f7bec003570 con 0x7f7c0406c0a0 2023-10-09T06:39:55.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.238+0000 7f7c01d9b700 1 -- 172.21.15.163:0/3206755669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c0406bcd0 msgr2=0x7f7c04140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:55.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.238+0000 7f7c01d9b700 1 --2- 172.21.15.163:0/3206755669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c0406bcd0 0x7f7c04140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:55.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.238+0000 7f7c01d9b700 1 -- 172.21.15.163:0/3206755669 >> v2:172.21.15.181:3300/0 conn(0x7f7c040c9510 msgr2=0x7f7c04141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:55.241 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.238+0000 7f7c01d9b700 1 --2- 172.21.15.163:0/3206755669 >> v2:172.21.15.181:3300/0 conn(0x7f7c040c9510 0x7f7c04141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:55.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.238+0000 7f7c01d9b700 1 -- 172.21.15.163:0/3206755669 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c0415bc50 con 0x7f7c0406c0a0 2023-10-09T06:39:55.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.238+0000 7f7c01d9b700 1 -- 172.21.15.163:0/3206755669 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f7bf8001470 con 0x7f7c0406c0a0 2023-10-09T06:39:55.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.238+0000 7f7c01d9b700 1 -- 172.21.15.163:0/3206755669 >> v1:172.21.15.163:6789/0 conn(0x7f7c0406c0a0 legacy=0x7f7c040c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:55.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.238+0000 7f7c01d9b700 1 --2- 172.21.15.163:0/3206755669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c0406bcd0 0x7f7bec003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:55.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.238+0000 7f7c01d9b700 1 --2- 172.21.15.163:0/3206755669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bec003fe0 0x7f7bec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:55.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.238+0000 7f7c01d9b700 1 --2- 172.21.15.163:0/3206755669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bec004d50 0x7f7bec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:55.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.238+0000 7f7c01d9b700 1 -- 172.21.15.163:0/3206755669 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0415b690 con 0x7f7bec004d50 2023-10-09T06:39:55.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.238+0000 7f7c01d9b700 1 -- 172.21.15.163:0/3206755669 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0415b970 con 0x7f7bec003fe0 2023-10-09T06:39:55.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.238+0000 7f7c01d9b700 1 -- 172.21.15.163:0/3206755669 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec00ba90 con 0x7f7c0406bcd0 2023-10-09T06:39:55.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.238+0000 7f7c0359e700 1 --2- 172.21.15.163:0/3206755669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bec004d50 0x7f7bec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:55.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.238+0000 7f7c02d9d700 1 --2- 172.21.15.163:0/3206755669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c0406bcd0 0x7f7bec003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:55.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.238+0000 7f7c0259c700 1 --2- 172.21.15.163:0/3206755669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bec003fe0 0x7f7bec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:55.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.238+0000 7f7c0359e700 1 -- 172.21.15.163:0/3206755669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c0406bcd0 msgr2=0x7f7bec003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:55.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.238+0000 7f7c0359e700 1 --2- 172.21.15.163:0/3206755669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c0406bcd0 0x7f7bec003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:55.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.238+0000 7f7c0359e700 1 -- 172.21.15.163:0/3206755669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bec003fe0 msgr2=0x7f7bec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:55.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.238+0000 7f7c0359e700 1 --2- 172.21.15.163:0/3206755669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bec003fe0 0x7f7bec004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:55.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.238+0000 7f7c0359e700 1 -- 172.21.15.163:0/3206755669 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bec003570 con 0x7f7bec004d50 2023-10-09T06:39:55.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.238+0000 7f7c0259c700 1 --2- 172.21.15.163:0/3206755669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bec003fe0 0x7f7bec004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:55.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.238+0000 7f7c0359e700 1 --2- 172.21.15.163:0/3206755669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bec004d50 0x7f7bec0051c0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc009c00 tx=0x7f7bfc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7fd024f87551994 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:55.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.238+0000 7f7c01d9b700 1 -- 172.21.15.163:0/3206755669 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc00e6b0 con 0x7f7bec004d50 2023-10-09T06:39:55.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.238+0000 7f7c01d9b700 1 -- 172.21.15.163:0/3206755669 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bfc00e810 con 0x7f7bec004d50 2023-10-09T06:39:55.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.238+0000 7f7c01d9b700 1 -- 172.21.15.163:0/3206755669 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc00ebd0 con 0x7f7bec004d50 2023-10-09T06:39:55.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.239+0000 7f7c0906a700 1 -- 172.21.15.163:0/3206755669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bec004d50 msgr2=0x7f7bec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:55.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.239+0000 7f7c0906a700 1 --2- 172.21.15.163:0/3206755669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bec004d50 0x7f7bec0051c0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc009c00 tx=0x7f7bfc0091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:55.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.239+0000 7f7c0359e700 1 -- 172.21.15.163:0/3206755669 reap_dead start 2023-10-09T06:39:55.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.239+0000 7f7c0906a700 1 -- 172.21.15.163:0/3206755669 shutdown_connections 2023-10-09T06:39:55.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.239+0000 7f7c0906a700 1 -- 172.21.15.163:0/3206755669 >> 172.21.15.163:0/3206755669 conn(0x7f7c040ce600 msgr2=0x7f7c040cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:55.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.239+0000 7f7c0906a700 1 -- 172.21.15.163:0/3206755669 shutdown_connections 2023-10-09T06:39:55.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.239+0000 7f7c0906a700 1 -- 172.21.15.163:0/3206755669 wait complete. 2023-10-09T06:39:55.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.240+0000 7f7c0906a700 1 Processor -- start 2023-10-09T06:39:55.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.240+0000 7f7c0906a700 1 -- start start 2023-10-09T06:39:55.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.241+0000 7f7c0906a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c0416d440 0x7f7c040c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:55.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.241+0000 7f7c0906a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c0413faa0 0x7f7c0416d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:55.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.241+0000 7f7c0906a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c0416dd50 0x7f7c0412f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:55.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.241+0000 7f7c0906a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c040c63b0 con 0x7f7c0413faa0 2023-10-09T06:39:55.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.241+0000 7f7c0906a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0415ca10 con 0x7f7c0416dd50 2023-10-09T06:39:55.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.241+0000 7f7c0906a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c040c7be0 con 0x7f7c0416d440 2023-10-09T06:39:55.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.241+0000 7f7c0259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c0413faa0 0x7f7c0416d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:55.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.241+0000 7f7c0259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c0413faa0 0x7f7c0416d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49106/0 (socket says 172.21.15.163:49106) 2023-10-09T06:39:55.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.241+0000 7f7c0259c700 1 -- 172.21.15.163:0/2547467672 learned_addr learned my addr 172.21.15.163:0/2547467672 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:55.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.241+0000 7f7c0359e700 1 --2- 172.21.15.163:0/2547467672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c0416dd50 0x7f7c0412f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:55.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.241+0000 7f7c0259c700 1 -- 172.21.15.163:0/2547467672 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c0416d440 msgr2=0x7f7c040c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:55.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.241+0000 7f7c02d9d700 1 --2- 172.21.15.163:0/2547467672 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c0416d440 0x7f7c040c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:55.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.241+0000 7f7c0259c700 1 --2- 172.21.15.163:0/2547467672 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c0416d440 0x7f7c040c76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:55.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.241+0000 7f7c0259c700 1 -- 172.21.15.163:0/2547467672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c0416dd50 msgr2=0x7f7c0412f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:55.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.241+0000 7f7c0259c700 1 --2- 172.21.15.163:0/2547467672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c0416dd50 0x7f7c0412f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:55.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.241+0000 7f7c0259c700 1 -- 172.21.15.163:0/2547467672 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c040c6600 con 0x7f7c0413faa0 2023-10-09T06:39:55.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.242+0000 7f7c0259c700 1 --2- 172.21.15.163:0/2547467672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c0413faa0 0x7f7c0416d810 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8000fd0 tx=0x7f7c0406e470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:55.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.242+0000 7f7beaffd700 1 -- 172.21.15.163:0/2547467672 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf800a7f0 con 0x7f7c0413faa0 2023-10-09T06:39:55.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.242+0000 7f7beaffd700 1 -- 172.21.15.163:0/2547467672 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf8010040 con 0x7f7c0413faa0 2023-10-09T06:39:55.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.242+0000 7f7c0906a700 1 -- 172.21.15.163:0/2547467672 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c040c68e0 con 0x7f7c0413faa0 2023-10-09T06:39:55.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.242+0000 7f7beaffd700 1 -- 172.21.15.163:0/2547467672 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf800ab60 con 0x7f7c0413faa0 2023-10-09T06:39:55.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.242+0000 7f7c0906a700 1 -- 172.21.15.163:0/2547467672 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c040c6ea0 con 0x7f7c0413faa0 2023-10-09T06:39:55.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.243+0000 7f7c0906a700 1 -- 172.21.15.163:0/2547467672 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c0404c3c0 con 0x7f7c0413faa0 2023-10-09T06:39:55.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.243+0000 7f7beaffd700 1 -- 172.21.15.163:0/2547467672 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7bf80143f0 con 0x7f7c0413faa0 2023-10-09T06:39:55.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.243+0000 7f7beaffd700 1 --2- 172.21.15.163:0/2547467672 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7bf0067350 0x7f7bf0069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:55.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.244+0000 7f7beaffd700 1 -- 172.21.15.163:0/2547467672 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7bf80840b0 con 0x7f7c0413faa0 2023-10-09T06:39:55.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.244+0000 7f7c02d9d700 1 --2- 172.21.15.163:0/2547467672 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7bf0067350 0x7f7bf0069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:55.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.244+0000 7f7c02d9d700 1 --2- 172.21.15.163:0/2547467672 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7bf0067350 0x7f7bf0069800 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40094d0 tx=0x7f7bf400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:55.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.248+0000 7f7beaffd700 1 -- 172.21.15.163:0/2547467672 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7bf804c5a0 con 0x7f7c0413faa0 2023-10-09T06:39:55.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:55.473+0000 7f7c0906a700 1 -- 172.21.15.163:0/2547467672 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7c0406c9d0 con 0x7f7c0413faa0 2023-10-09T06:39:56.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.094+0000 7f7beaffd700 1 -- 172.21.15.163:0/2547467672 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v132) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7bf80190c0 con 0x7f7c0413faa0 2023-10-09T06:39:56.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.097+0000 7f7c0906a700 1 -- 172.21.15.163:0/2547467672 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7bf0067350 msgr2=0x7f7bf0069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:56.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.097+0000 7f7c0906a700 1 --2- 172.21.15.163:0/2547467672 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7bf0067350 0x7f7bf0069800 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40094d0 tx=0x7f7bf400b040 comp rx=0 tx=0).stop 2023-10-09T06:39:56.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.097+0000 7f7c0906a700 1 -- 172.21.15.163:0/2547467672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c0413faa0 msgr2=0x7f7c0416d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:56.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.097+0000 7f7c0906a700 1 --2- 172.21.15.163:0/2547467672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c0413faa0 0x7f7c0416d810 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8000fd0 tx=0x7f7c0406e470 comp rx=0 tx=0).stop 2023-10-09T06:39:56.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.097+0000 7f7c0906a700 1 -- 172.21.15.163:0/2547467672 shutdown_connections 2023-10-09T06:39:56.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.097+0000 7f7c0906a700 1 --2- 172.21.15.163:0/2547467672 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7bf0067350 0x7f7bf0069800 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.097+0000 7f7c0906a700 1 --2- 172.21.15.163:0/2547467672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c0416dd50 0x7f7c0412f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.097+0000 7f7c0906a700 1 --2- 172.21.15.163:0/2547467672 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c0416d440 0x7f7c040c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.097+0000 7f7c0906a700 1 --2- 172.21.15.163:0/2547467672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c0413faa0 0x7f7c0416d810 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.097+0000 7f7c0906a700 1 -- 172.21.15.163:0/2547467672 >> 172.21.15.163:0/2547467672 conn(0x7f7c040ce600 msgr2=0x7f7c04069480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:56.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.097+0000 7f7c0906a700 1 -- 172.21.15.163:0/2547467672 shutdown_connections 2023-10-09T06:39:56.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.097+0000 7f7c0906a700 1 -- 172.21.15.163:0/2547467672 wait complete. 2023-10-09T06:39:56.110 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:39:56.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.227+0000 7fd66b59e700 1 Processor -- start 2023-10-09T06:39:56.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.227+0000 7fd66b59e700 1 -- start start 2023-10-09T06:39:56.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.228+0000 7fd66b59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd66c159420 0x7fd66c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:56.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.228+0000 7fd66b59e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd66c1597f0 0x7fd66c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:56.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.228+0000 7fd66b59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd66c15d230 0x7fd66c15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:56.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.228+0000 7fd66b59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd66c15dbd0 con 0x7fd66c15d230 2023-10-09T06:39:56.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.228+0000 7fd66b59e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd66c0b11d0 con 0x7fd66c1597f0 2023-10-09T06:39:56.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.228+0000 7fd66b59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd66c0b14b0 con 0x7fd66c159420 2023-10-09T06:39:56.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.228+0000 7fd66ad9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd66c15d230 0x7fd66c15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:56.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.228+0000 7fd66ad9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd66c15d230 0x7fd66c15d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49116/0 (socket says 172.21.15.163:49116) 2023-10-09T06:39:56.233 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.228+0000 7fd66ad9d700 1 -- 172.21.15.163:0/2850801326 learned_addr learned my addr 172.21.15.163:0/2850801326 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:56.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.228+0000 7fd66a59c700 1 --2- 172.21.15.163:0/2850801326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd66c159420 0x7fd66c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:56.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.228+0000 7fd663fff700 1 --2- 172.21.15.163:0/2850801326 >> v2:172.21.15.181:3300/0 conn(0x7fd66c1597f0 0x7fd66c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:56.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.228+0000 7fd66ad9d700 1 -- 172.21.15.163:0/2850801326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd66c159420 msgr2=0x7fd66c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:56.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.228+0000 7fd66ad9d700 1 --2- 172.21.15.163:0/2850801326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd66c159420 0x7fd66c0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.228+0000 7fd66ad9d700 1 -- 172.21.15.163:0/2850801326 >> v2:172.21.15.181:3300/0 conn(0x7fd66c1597f0 msgr2=0x7fd66c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:56.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.228+0000 7fd66ad9d700 1 --2- 172.21.15.163:0/2850801326 >> v2:172.21.15.181:3300/0 conn(0x7fd66c1597f0 0x7fd66c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.228+0000 7fd66ad9d700 1 -- 172.21.15.163:0/2850801326 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd66c0b1790 con 0x7fd66c15d230 2023-10-09T06:39:56.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.228+0000 7fd663fff700 1 --2- 172.21.15.163:0/2850801326 >> v2:172.21.15.181:3300/0 conn(0x7fd66c1597f0 0x7fd66c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:56.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.229+0000 7fd66ad9d700 1 --2- 172.21.15.163:0/2850801326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd66c15d230 0x7fd66c15d690 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fd66c0d1770 tx=0x7fd664009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2b2b5bf1ad074d server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:56.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.229+0000 7fd669d9b700 1 -- 172.21.15.163:0/2850801326 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66400e070 con 0x7fd66c15d230 2023-10-09T06:39:56.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.229+0000 7fd669d9b700 1 -- 172.21.15.163:0/2850801326 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66400ba20 con 0x7fd66c15d230 2023-10-09T06:39:56.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.229+0000 7fd669d9b700 1 -- 172.21.15.163:0/2850801326 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6640092e0 con 0x7fd66c15d230 2023-10-09T06:39:56.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.229+0000 7fd66b59e700 1 -- 172.21.15.163:0/2850801326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd66c15d230 msgr2=0x7fd66c15d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:56.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.229+0000 7fd66b59e700 1 --2- 172.21.15.163:0/2850801326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd66c15d230 0x7fd66c15d690 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fd66c0d1770 tx=0x7fd664009840 comp rx=0 tx=0).stop 2023-10-09T06:39:56.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.229+0000 7fd66b59e700 1 -- 172.21.15.163:0/2850801326 shutdown_connections 2023-10-09T06:39:56.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.229+0000 7fd66b59e700 1 --2- 172.21.15.163:0/2850801326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd66c15d230 0x7fd66c15d690 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.229+0000 7fd66b59e700 1 --2- 172.21.15.163:0/2850801326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd66c159420 0x7fd66c0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.229+0000 7fd66b59e700 1 --2- 172.21.15.163:0/2850801326 >> v2:172.21.15.181:3300/0 conn(0x7fd66c1597f0 0x7fd66c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.229+0000 7fd66b59e700 1 -- 172.21.15.163:0/2850801326 >> 172.21.15.163:0/2850801326 conn(0x7fd66c0d0660 msgr2=0x7fd66c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:56.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.229+0000 7fd66b59e700 1 -- 172.21.15.163:0/2850801326 shutdown_connections 2023-10-09T06:39:56.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.229+0000 7fd66b59e700 1 -- 172.21.15.163:0/2850801326 wait complete. 2023-10-09T06:39:56.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.230+0000 7fd66b59e700 1 Processor -- start 2023-10-09T06:39:56.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.230+0000 7fd66b59e700 1 -- start start 2023-10-09T06:39:56.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.231+0000 7fd66b59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd66c159420 0x7fd66c1441e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:56.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.231+0000 7fd66b59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd66c1597f0 0x7fd66c0ce980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:56.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.231+0000 7fd66b59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd66c15d230 0x7fd66c0ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:56.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.231+0000 7fd66b59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66c0aee50 con 0x7fd66c1597f0 2023-10-09T06:39:56.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.231+0000 7fd66b59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66c0ae9c0 con 0x7fd66c159420 2023-10-09T06:39:56.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.231+0000 7fd66b59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd66c0addb0 con 0x7fd66c15d230 2023-10-09T06:39:56.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.231+0000 7fd663fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd66c1597f0 0x7fd66c0ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:56.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.231+0000 7fd663fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd66c1597f0 0x7fd66c0ce980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49120/0 (socket says 172.21.15.163:49120) 2023-10-09T06:39:56.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.231+0000 7fd663fff700 1 -- 172.21.15.163:0/2053221659 learned_addr learned my addr 172.21.15.163:0/2053221659 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:56.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.231+0000 7fd66a59c700 1 --2- 172.21.15.163:0/2053221659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd66c159420 0x7fd66c1441e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:56.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.231+0000 7fd66ad9d700 1 --2- 172.21.15.163:0/2053221659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd66c15d230 0x7fd66c0ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:56.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.231+0000 7fd663fff700 1 -- 172.21.15.163:0/2053221659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd66c15d230 msgr2=0x7fd66c0ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:56.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.231+0000 7fd663fff700 1 --2- 172.21.15.163:0/2053221659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd66c15d230 0x7fd66c0ceec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.231+0000 7fd663fff700 1 -- 172.21.15.163:0/2053221659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd66c159420 msgr2=0x7fd66c1441e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:56.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.231+0000 7fd663fff700 1 --2- 172.21.15.163:0/2053221659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd66c159420 0x7fd66c1441e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.231+0000 7fd663fff700 1 -- 172.21.15.163:0/2053221659 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd664009ae0 con 0x7fd66c1597f0 2023-10-09T06:39:56.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.232+0000 7fd66a59c700 1 --2- 172.21.15.163:0/2053221659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd66c159420 0x7fd66c1441e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:56.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.232+0000 7fd663fff700 1 --2- 172.21.15.163:0/2053221659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd66c1597f0 0x7fd66c0ce980 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fd65400e8d0 tx=0x7fd654008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:56.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.232+0000 7fd6627fc700 1 -- 172.21.15.163:0/2053221659 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65400ee70 con 0x7fd66c1597f0 2023-10-09T06:39:56.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.232+0000 7fd6627fc700 1 -- 172.21.15.163:0/2053221659 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65400c5e0 con 0x7fd66c1597f0 2023-10-09T06:39:56.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.232+0000 7fd66b59e700 1 -- 172.21.15.163:0/2053221659 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd66c0ae090 con 0x7fd66c1597f0 2023-10-09T06:39:56.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.232+0000 7fd6627fc700 1 -- 172.21.15.163:0/2053221659 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65400c980 con 0x7fd66c1597f0 2023-10-09T06:39:56.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.232+0000 7fd66b59e700 1 -- 172.21.15.163:0/2053221659 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd66c0ae650 con 0x7fd66c1597f0 2023-10-09T06:39:56.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.233+0000 7fd66b59e700 1 -- 172.21.15.163:0/2053221659 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd66c0d1180 con 0x7fd66c1597f0 2023-10-09T06:39:56.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.233+0000 7fd6627fc700 1 -- 172.21.15.163:0/2053221659 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd654013410 con 0x7fd66c1597f0 2023-10-09T06:39:56.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.234+0000 7fd6627fc700 1 --2- 172.21.15.163:0/2053221659 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd64c067320 0x7fd64c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:56.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.234+0000 7fd6627fc700 1 -- 172.21.15.163:0/2053221659 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd654084b70 con 0x7fd66c1597f0 2023-10-09T06:39:56.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.234+0000 7fd66a59c700 1 --2- 172.21.15.163:0/2053221659 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd64c067320 0x7fd64c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:56.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.234+0000 7fd66a59c700 1 --2- 172.21.15.163:0/2053221659 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd64c067320 0x7fd64c0697d0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009d80 tx=0x7fd65c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:56.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.238+0000 7fd6627fc700 1 -- 172.21.15.163:0/2053221659 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd6540516f0 con 0x7fd66c1597f0 2023-10-09T06:39:56.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.464+0000 7fd66b59e700 1 -- 172.21.15.163:0/2053221659 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd66c04c3c0 con 0x7fd66c1597f0 2023-10-09T06:39:56.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.464+0000 7fd6627fc700 1 -- 172.21.15.163:0/2053221659 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) v1 ==== 95+0+4619 (secure 0 0 0) 0x7fd654054d10 con 0x7fd66c1597f0 2023-10-09T06:39:56.464 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:56.465 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":132,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7046,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3374277959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3374277959},{"type":"v1","addr":"172.21.15.181:6837","nonce":3374277959}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3834979450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3834979450},{"type":"v1","addr":"172.21.15.181:6836","nonce":3834979450}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":8991,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2681808199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2681808199},{"type":"v1","addr":"172.21.15.163:6834","nonce":2681808199}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128}],"filesystems":[{"mdsmap":{"epoch":132,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:39:52.456713+0000","modified":"2023-10-09T06:39:56.092448+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8994":{"gid":8994,"name":"a","rank":0,"incarnation":129,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/2564044136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2564044136},{"type":"v1","addr":"172.21.15.163:6835","nonce":2564044136}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-09T06:39:56.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.466+0000 7fd66b59e700 1 -- 172.21.15.163:0/2053221659 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd64c067320 msgr2=0x7fd64c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:56.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.466+0000 7fd66b59e700 1 --2- 172.21.15.163:0/2053221659 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd64c067320 0x7fd64c0697d0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009d80 tx=0x7fd65c008040 comp rx=0 tx=0).stop 2023-10-09T06:39:56.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.466+0000 7fd66b59e700 1 -- 172.21.15.163:0/2053221659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd66c1597f0 msgr2=0x7fd66c0ce980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:56.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.466+0000 7fd66b59e700 1 --2- 172.21.15.163:0/2053221659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd66c1597f0 0x7fd66c0ce980 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fd65400e8d0 tx=0x7fd654008040 comp rx=0 tx=0).stop 2023-10-09T06:39:56.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.466+0000 7fd66b59e700 1 -- 172.21.15.163:0/2053221659 shutdown_connections 2023-10-09T06:39:56.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.466+0000 7fd66b59e700 1 --2- 172.21.15.163:0/2053221659 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd64c067320 0x7fd64c0697d0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.466+0000 7fd66b59e700 1 --2- 172.21.15.163:0/2053221659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd66c15d230 0x7fd66c0ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.466+0000 7fd66b59e700 1 --2- 172.21.15.163:0/2053221659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd66c159420 0x7fd66c1441e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.466+0000 7fd66b59e700 1 --2- 172.21.15.163:0/2053221659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd66c1597f0 0x7fd66c0ce980 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.467+0000 7fd66b59e700 1 -- 172.21.15.163:0/2053221659 >> 172.21.15.163:0/2053221659 conn(0x7fd66c0d0660 msgr2=0x7fd66c0af200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:56.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.467+0000 7fd66b59e700 1 -- 172.21.15.163:0/2053221659 shutdown_connections 2023-10-09T06:39:56.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.467+0000 7fd66b59e700 1 -- 172.21.15.163:0/2053221659 wait complete. 2023-10-09T06:39:56.468 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 132 2023-10-09T06:39:56.479 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:39:56.593 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.593+0000 7ff2053d8700 1 Processor -- start 2023-10-09T06:39:56.593 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.593+0000 7ff2053d8700 1 -- start start 2023-10-09T06:39:56.595 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.594+0000 7ff2053d8700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff2001593b0 0x7ff200159780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:56.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.594+0000 7ff2053d8700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff2000ceaa0 0x7ff2000b6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:56.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.594+0000 7ff2053d8700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2000b6570 con 0x7ff2000b1740 2023-10-09T06:39:56.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.594+0000 7ff2053d8700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2000cfe20 con 0x7ff2000ceaa0 2023-10-09T06:39:56.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.594+0000 7ff2053d8700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2000cf210 con 0x7ff2001593b0 2023-10-09T06:39:56.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.594+0000 7ff1ff7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff2000ceaa0 0x7ff2000b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:56.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.594+0000 7ff204bd7700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff2000b1740 0x7ff2000b1b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:36464/0 (socket says 172.21.15.163:36464) 2023-10-09T06:39:56.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.594+0000 7ff1ff7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff2000ceaa0 0x7ff2000b6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49124/0 (socket says 172.21.15.163:49124) 2023-10-09T06:39:56.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.594+0000 7ff204bd7700 1 -- 172.21.15.163:0/2875212611 learned_addr learned my addr 172.21.15.163:0/2875212611 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:56.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.594+0000 7ff1fffff700 1 --2- 172.21.15.163:0/2875212611 >> v2:172.21.15.181:3300/0 conn(0x7ff2001593b0 0x7ff200159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:56.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.594+0000 7ff1ff7fe700 1 -- 172.21.15.163:0/2875212611 >> v2:172.21.15.181:3300/0 conn(0x7ff2001593b0 msgr2=0x7ff200159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:56.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.594+0000 7ff1ff7fe700 1 --2- 172.21.15.163:0/2875212611 >> v2:172.21.15.181:3300/0 conn(0x7ff2001593b0 0x7ff200159780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.594+0000 7ff1ff7fe700 1 -- 172.21.15.163:0/2875212611 >> v1:172.21.15.163:6789/0 conn(0x7ff2000b1740 legacy=0x7ff2000b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:56.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.594+0000 7ff1ff7fe700 1 -- 172.21.15.163:0/2875212611 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2000cf4f0 con 0x7ff2000ceaa0 2023-10-09T06:39:56.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.594+0000 7ff1feffd700 1 -- 172.21.15.163:0/2875212611 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3056189009 0 0) 0x7ff1f8001490 con 0x7ff2000b1740 2023-10-09T06:39:56.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.595+0000 7ff1ff7fe700 1 --2- 172.21.15.163:0/2875212611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2000ceaa0 0x7ff2000b6030 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7ff200070d60 tx=0x7ff1f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd0469146513cd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:56.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.595+0000 7ff1feffd700 1 -- 172.21.15.163:0/2875212611 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f400b6b0 con 0x7ff2000ceaa0 2023-10-09T06:39:56.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.595+0000 7ff1feffd700 1 -- 172.21.15.163:0/2875212611 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f400b810 con 0x7ff2000ceaa0 2023-10-09T06:39:56.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.595+0000 7ff1feffd700 1 -- 172.21.15.163:0/2875212611 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f400bbb0 con 0x7ff2000ceaa0 2023-10-09T06:39:56.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.595+0000 7ff2053d8700 1 -- 172.21.15.163:0/2875212611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2000ceaa0 msgr2=0x7ff2000b6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:56.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.595+0000 7ff2053d8700 1 --2- 172.21.15.163:0/2875212611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2000ceaa0 0x7ff2000b6030 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7ff200070d60 tx=0x7ff1f40091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.595+0000 7ff2053d8700 1 -- 172.21.15.163:0/2875212611 shutdown_connections 2023-10-09T06:39:56.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.595+0000 7ff2053d8700 1 --2- 172.21.15.163:0/2875212611 >> v2:172.21.15.181:3300/0 conn(0x7ff2001593b0 0x7ff200159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.595+0000 7ff2053d8700 1 --2- 172.21.15.163:0/2875212611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2000ceaa0 0x7ff2000b6030 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.597 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.595+0000 7ff2053d8700 1 -- 172.21.15.163:0/2875212611 >> 172.21.15.163:0/2875212611 conn(0x7ff2000d1130 msgr2=0x7ff2000b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:56.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.595+0000 7ff2053d8700 1 -- 172.21.15.163:0/2875212611 shutdown_connections 2023-10-09T06:39:56.598 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.595+0000 7ff2053d8700 1 -- 172.21.15.163:0/2875212611 wait complete. 2023-10-09T06:39:56.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.596+0000 7ff2053d8700 1 Processor -- start 2023-10-09T06:39:56.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.597+0000 7ff2053d8700 1 -- start start 2023-10-09T06:39:56.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.597+0000 7ff2053d8700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2000b1740 0x7ff20015a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:56.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.597+0000 7ff2053d8700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2000ceaa0 0x7ff2000cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:56.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.597+0000 7ff2053d8700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2001593b0 0x7ff2000cc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:56.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.597+0000 7ff2053d8700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20016b030 con 0x7ff2000ceaa0 2023-10-09T06:39:56.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.597+0000 7ff2053d8700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20015a700 con 0x7ff2001593b0 2023-10-09T06:39:56.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.597+0000 7ff2053d8700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff20015b220 con 0x7ff2000b1740 2023-10-09T06:39:56.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.597+0000 7ff1ff7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2000ceaa0 0x7ff2000cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:56.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.597+0000 7ff1ff7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2000ceaa0 0x7ff2000cbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49138/0 (socket says 172.21.15.163:49138) 2023-10-09T06:39:56.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.597+0000 7ff1ff7fe700 1 -- 172.21.15.163:0/2152249332 learned_addr learned my addr 172.21.15.163:0/2152249332 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:56.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.598+0000 7ff1ff7fe700 1 -- 172.21.15.163:0/2152249332 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2000b1740 msgr2=0x7ff20015a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:56.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.598+0000 7ff1ff7fe700 1 --2- 172.21.15.163:0/2152249332 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2000b1740 0x7ff20015a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.598+0000 7ff1ff7fe700 1 -- 172.21.15.163:0/2152249332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2001593b0 msgr2=0x7ff2000cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:56.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.598+0000 7ff1ff7fe700 1 --2- 172.21.15.163:0/2152249332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2001593b0 0x7ff2000cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.598+0000 7ff1ff7fe700 1 -- 172.21.15.163:0/2152249332 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f4009420 con 0x7ff2000ceaa0 2023-10-09T06:39:56.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.598+0000 7ff1ff7fe700 1 --2- 172.21.15.163:0/2152249332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2000ceaa0 0x7ff2000cbd40 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7ff1f40097b0 tx=0x7ff1f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:56.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.598+0000 7ff1e7fff700 1 -- 172.21.15.163:0/2152249332 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4011420 con 0x7ff2000ceaa0 2023-10-09T06:39:56.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.598+0000 7ff1e7fff700 1 -- 172.21.15.163:0/2152249332 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f4011580 con 0x7ff2000ceaa0 2023-10-09T06:39:56.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.598+0000 7ff2053d8700 1 -- 172.21.15.163:0/2152249332 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff20015a980 con 0x7ff2000ceaa0 2023-10-09T06:39:56.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.598+0000 7ff1e7fff700 1 -- 172.21.15.163:0/2152249332 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4011920 con 0x7ff2000ceaa0 2023-10-09T06:39:56.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.598+0000 7ff2053d8700 1 -- 172.21.15.163:0/2152249332 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff20015af40 con 0x7ff2000ceaa0 2023-10-09T06:39:56.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.599+0000 7ff2053d8700 1 -- 172.21.15.163:0/2152249332 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff20004c3c0 con 0x7ff2000ceaa0 2023-10-09T06:39:56.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.599+0000 7ff1e7fff700 1 -- 172.21.15.163:0/2152249332 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff1f4011d10 con 0x7ff2000ceaa0 2023-10-09T06:39:56.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.600+0000 7ff1e7fff700 1 --2- 172.21.15.163:0/2152249332 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff1ec067320 0x7ff1ec0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:56.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.600+0000 7ff1e7fff700 1 -- 172.21.15.163:0/2152249332 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff1f40864d0 con 0x7ff2000ceaa0 2023-10-09T06:39:56.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.600+0000 7ff1fffff700 1 --2- 172.21.15.163:0/2152249332 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff1ec067320 0x7ff1ec0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:56.602 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.600+0000 7ff1fffff700 1 --2- 172.21.15.163:0/2152249332 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff1ec067320 0x7ff1ec0697d0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000a900 tx=0x7ff1f0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:56.603 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.604+0000 7ff1e7fff700 1 -- 172.21.15.163:0/2152249332 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff1f4053120 con 0x7ff2000ceaa0 2023-10-09T06:39:56.789 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.789+0000 7ff2053d8700 1 -- 172.21.15.163:0/2152249332 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff20016b170 con 0x7ff2000ceaa0 2023-10-09T06:39:56.790 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.790+0000 7ff1e7fff700 1 -- 172.21.15.163:0/2152249332 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v197) v1 ==== 74+0+15412 (secure 0 0 0) 0x7ff1f4056740 con 0x7ff2000ceaa0 2023-10-09T06:39:56.790 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:56.790 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":197,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:39:54.066522+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:39:50.786303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":34,"pool_name":"cephfs_data","create_time":"2023-10-09T06:39:51.411021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:39:56.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.792+0000 7ff2053d8700 1 -- 172.21.15.163:0/2152249332 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff1ec067320 msgr2=0x7ff1ec0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:56.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.792+0000 7ff2053d8700 1 --2- 172.21.15.163:0/2152249332 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff1ec067320 0x7ff1ec0697d0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000a900 tx=0x7ff1f0008040 comp rx=0 tx=0).stop 2023-10-09T06:39:56.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.792+0000 7ff2053d8700 1 -- 172.21.15.163:0/2152249332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2000ceaa0 msgr2=0x7ff2000cbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:56.792 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.792+0000 7ff2053d8700 1 --2- 172.21.15.163:0/2152249332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2000ceaa0 0x7ff2000cbd40 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7ff1f40097b0 tx=0x7ff1f4017000 comp rx=0 tx=0).stop 2023-10-09T06:39:56.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.792+0000 7ff2053d8700 1 -- 172.21.15.163:0/2152249332 shutdown_connections 2023-10-09T06:39:56.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.792+0000 7ff2053d8700 1 --2- 172.21.15.163:0/2152249332 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff1ec067320 0x7ff1ec0697d0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.792+0000 7ff2053d8700 1 --2- 172.21.15.163:0/2152249332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2001593b0 0x7ff2000cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.792+0000 7ff2053d8700 1 --2- 172.21.15.163:0/2152249332 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2000b1740 0x7ff20015a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.792+0000 7ff2053d8700 1 --2- 172.21.15.163:0/2152249332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2000ceaa0 0x7ff2000cbd40 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.792+0000 7ff2053d8700 1 -- 172.21.15.163:0/2152249332 >> 172.21.15.163:0/2152249332 conn(0x7ff2000d1130 msgr2=0x7ff200157220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:56.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.792+0000 7ff2053d8700 1 -- 172.21.15.163:0/2152249332 shutdown_connections 2023-10-09T06:39:56.793 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.792+0000 7ff2053d8700 1 -- 172.21.15.163:0/2152249332 wait complete. 2023-10-09T06:39:56.805 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-09T06:39:56.805 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:39:56.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.920+0000 7fcce359e700 1 Processor -- start 2023-10-09T06:39:56.920 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.920+0000 7fcce359e700 1 -- start start 2023-10-09T06:39:56.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.921+0000 7fcce359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcce4159420 0x7fcce40b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:56.922 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.921+0000 7fcce359e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcce415d230 0x7fcce415d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:56.923 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.921+0000 7fcce359e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcce415dbd0 con 0x7fcce41597f0 2023-10-09T06:39:56.923 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.921+0000 7fcce359e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcce40b2980 con 0x7fcce415d230 2023-10-09T06:39:56.923 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.921+0000 7fcce359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce40b11d0 con 0x7fcce4159420 2023-10-09T06:39:56.923 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.921+0000 7fcce2d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcce415d230 0x7fcce415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:56.923 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.921+0000 7fcce1d9b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fcce41597f0 0x7fcce40b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:36476/0 (socket says 172.21.15.163:36476) 2023-10-09T06:39:56.923 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.921+0000 7fcce1d9b700 1 -- 172.21.15.163:0/1351700280 learned_addr learned my addr 172.21.15.163:0/1351700280 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:56.923 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.921+0000 7fcce2d9d700 1 -- 172.21.15.163:0/1351700280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcce4159420 msgr2=0x7fcce40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:56.923 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.921+0000 7fcce259c700 1 --2- 172.21.15.163:0/1351700280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcce4159420 0x7fcce40b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:56.923 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.921+0000 7fcce2d9d700 1 --2- 172.21.15.163:0/1351700280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcce4159420 0x7fcce40b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.923 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.921+0000 7fcce2d9d700 1 -- 172.21.15.163:0/1351700280 >> v1:172.21.15.163:6789/0 conn(0x7fcce41597f0 legacy=0x7fcce40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:56.923 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.921+0000 7fcce159a700 1 -- 172.21.15.163:0/1351700280 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1356814501 0 0) 0x7fccd8001110 con 0x7fcce41597f0 2023-10-09T06:39:56.923 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.921+0000 7fcce2d9d700 1 -- 172.21.15.163:0/1351700280 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce40b14b0 con 0x7fcce415d230 2023-10-09T06:39:56.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.922+0000 7fcce2d9d700 1 --2- 172.21.15.163:0/1351700280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcce415d230 0x7fcce415d690 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fccd40098e0 tx=0x7fccd40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1de308abd51059 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:56.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.922+0000 7fcce159a700 1 -- 172.21.15.163:0/1351700280 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd4009e80 con 0x7fcce415d230 2023-10-09T06:39:56.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.922+0000 7fcce159a700 1 -- 172.21.15.163:0/1351700280 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd400b860 con 0x7fcce415d230 2023-10-09T06:39:56.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.922+0000 7fcce159a700 1 -- 172.21.15.163:0/1351700280 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd400bbe0 con 0x7fcce415d230 2023-10-09T06:39:56.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.922+0000 7fcce359e700 1 -- 172.21.15.163:0/1351700280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcce415d230 msgr2=0x7fcce415d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:56.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.922+0000 7fcce359e700 1 --2- 172.21.15.163:0/1351700280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcce415d230 0x7fcce415d690 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fccd40098e0 tx=0x7fccd40091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.922+0000 7fcce359e700 1 -- 172.21.15.163:0/1351700280 shutdown_connections 2023-10-09T06:39:56.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.922+0000 7fcce359e700 1 --2- 172.21.15.163:0/1351700280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcce415d230 0x7fcce415d690 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.922+0000 7fcce359e700 1 --2- 172.21.15.163:0/1351700280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcce4159420 0x7fcce40b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.922+0000 7fcce359e700 1 -- 172.21.15.163:0/1351700280 >> 172.21.15.163:0/1351700280 conn(0x7fcce40d0660 msgr2=0x7fcce40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:56.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.922+0000 7fcce359e700 1 -- 172.21.15.163:0/1351700280 shutdown_connections 2023-10-09T06:39:56.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.922+0000 7fcce359e700 1 -- 172.21.15.163:0/1351700280 wait complete. 2023-10-09T06:39:56.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.923+0000 7fcce359e700 1 Processor -- start 2023-10-09T06:39:56.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.923+0000 7fcce359e700 1 -- start start 2023-10-09T06:39:56.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.924+0000 7fcce359e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcce4159420 0x7fcce4153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:56.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.924+0000 7fcce359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcce41597f0 0x7fcce415bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:56.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.924+0000 7fcce359e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcce415d230 0x7fcce4069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:56.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.924+0000 7fcce359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce40c8500 con 0x7fcce41597f0 2023-10-09T06:39:56.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.924+0000 7fcce359e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce40c60b0 con 0x7fcce4159420 2023-10-09T06:39:56.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.924+0000 7fcce359e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce40c54a0 con 0x7fcce415d230 2023-10-09T06:39:56.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.924+0000 7fcce1d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcce41597f0 0x7fcce415bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:56.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.924+0000 7fcce1d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcce41597f0 0x7fcce415bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49166/0 (socket says 172.21.15.163:49166) 2023-10-09T06:39:56.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.924+0000 7fcce1d9b700 1 -- 172.21.15.163:0/529218996 learned_addr learned my addr 172.21.15.163:0/529218996 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:56.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.924+0000 7fcce259c700 1 --2- 172.21.15.163:0/529218996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcce4159420 0x7fcce4153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:56.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.924+0000 7fcce2d9d700 1 --2- 172.21.15.163:0/529218996 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcce415d230 0x7fcce4069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:56.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.924+0000 7fcce1d9b700 1 -- 172.21.15.163:0/529218996 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcce415d230 msgr2=0x7fcce4069530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:56.928 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.925+0000 7fcce1d9b700 1 --2- 172.21.15.163:0/529218996 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcce415d230 0x7fcce4069530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.925+0000 7fcce1d9b700 1 -- 172.21.15.163:0/529218996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcce4159420 msgr2=0x7fcce4153560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:56.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.925+0000 7fcce1d9b700 1 --2- 172.21.15.163:0/529218996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcce4159420 0x7fcce4153560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:56.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.925+0000 7fcce1d9b700 1 -- 172.21.15.163:0/529218996 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd4009420 con 0x7fcce41597f0 2023-10-09T06:39:56.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.925+0000 7fcce1d9b700 1 --2- 172.21.15.163:0/529218996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcce41597f0 0x7fcce415bc30 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fccd800c810 tx=0x7fccd8007c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:56.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.925+0000 7fccd27fc700 1 -- 172.21.15.163:0/529218996 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd8005b30 con 0x7fcce41597f0 2023-10-09T06:39:56.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.925+0000 7fccd27fc700 1 -- 172.21.15.163:0/529218996 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd8005c90 con 0x7fcce41597f0 2023-10-09T06:39:56.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.925+0000 7fccd27fc700 1 -- 172.21.15.163:0/529218996 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd8007340 con 0x7fcce41597f0 2023-10-09T06:39:56.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.925+0000 7fcce359e700 1 -- 172.21.15.163:0/529218996 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce40c5780 con 0x7fcce41597f0 2023-10-09T06:39:56.929 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.925+0000 7fcce359e700 1 -- 172.21.15.163:0/529218996 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce40c5b10 con 0x7fcce41597f0 2023-10-09T06:39:56.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.926+0000 7fcce359e700 1 -- 172.21.15.163:0/529218996 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcce4153b60 con 0x7fcce41597f0 2023-10-09T06:39:56.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.926+0000 7fccd27fc700 1 -- 172.21.15.163:0/529218996 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fccd801b400 con 0x7fcce41597f0 2023-10-09T06:39:56.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.927+0000 7fccd27fc700 1 --2- 172.21.15.163:0/529218996 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fccc4067320 0x7fccc40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:56.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.927+0000 7fccd27fc700 1 -- 172.21.15.163:0/529218996 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fccd8084580 con 0x7fcce41597f0 2023-10-09T06:39:56.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.927+0000 7fcce259c700 1 --2- 172.21.15.163:0/529218996 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fccc4067320 0x7fccc40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:56.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.927+0000 7fcce259c700 1 --2- 172.21.15.163:0/529218996 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fccc4067320 0x7fccc40697d0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fcce40d1520 tx=0x7fccdc007710 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:56.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:56.931+0000 7fccd27fc700 1 -- 172.21.15.163:0/529218996 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fccd8051210 con 0x7fcce41597f0 2023-10-09T06:39:57.109 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.109+0000 7fcce359e700 1 -- 172.21.15.163:0/529218996 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcce415dbd0 con 0x7fccc4067320 2023-10-09T06:39:57.117 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.117+0000 7fccd27fc700 1 -- 172.21.15.163:0/529218996 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289885 (secure 0 0 0) 0x7fcce415dbd0 con 0x7fccc4067320 2023-10-09T06:39:57.121 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:57.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.120+0000 7fcce359e700 1 -- 172.21.15.163:0/529218996 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fccc4067320 msgr2=0x7fccc40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:57.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.120+0000 7fcce359e700 1 --2- 172.21.15.163:0/529218996 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fccc4067320 0x7fccc40697d0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fcce40d1520 tx=0x7fccdc007710 comp rx=0 tx=0).stop 2023-10-09T06:39:57.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.120+0000 7fcce359e700 1 -- 172.21.15.163:0/529218996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcce41597f0 msgr2=0x7fcce415bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:57.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.120+0000 7fcce359e700 1 --2- 172.21.15.163:0/529218996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcce41597f0 0x7fcce415bc30 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fccd800c810 tx=0x7fccd8007c50 comp rx=0 tx=0).stop 2023-10-09T06:39:57.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.120+0000 7fcce359e700 1 -- 172.21.15.163:0/529218996 shutdown_connections 2023-10-09T06:39:57.123 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.120+0000 7fcce359e700 1 --2- 172.21.15.163:0/529218996 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fccc4067320 0x7fccc40697d0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:57.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.120+0000 7fcce359e700 1 --2- 172.21.15.163:0/529218996 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcce415d230 0x7fcce4069530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:57.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.120+0000 7fcce359e700 1 --2- 172.21.15.163:0/529218996 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcce4159420 0x7fcce4153560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:57.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.121+0000 7fcce359e700 1 --2- 172.21.15.163:0/529218996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcce41597f0 0x7fcce415bc30 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:57.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.121+0000 7fcce359e700 1 -- 172.21.15.163:0/529218996 >> 172.21.15.163:0/529218996 conn(0x7fcce40d0660 msgr2=0x7fcce40b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:57.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.121+0000 7fcce359e700 1 -- 172.21.15.163:0/529218996 shutdown_connections 2023-10-09T06:39:57.124 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.121+0000 7fcce359e700 1 -- 172.21.15.163:0/529218996 wait complete. 2023-10-09T06:39:57.124 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:39:57.132 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":473,"stamp":"2023-10-09T06:39:56.433167+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2431880,"kb_used_data":20808,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752542840,"statfs":{"total":773094113280,"available":770603868160,"internally_reserved":0,"allocated":21307392,"data_stored":10191348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.374054"},"pg_stats":[{"pgid":"34.3f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.089234+0000","last_change":"2023-10-09T06:39:53.085861+0000","last_active":"2023-10-09T06:39:53.089234+0000","last_peered":"2023-10-09T06:39:53.089234+0000","last_clean":"2023-10-09T06:39:53.089234+0000","last_became_active":"2023-10-09T06:39:53.084847+0000","last_became_peered":"2023-10-09T06:39:53.084847+0000","last_unstale":"2023-10-09T06:39:53.089234+0000","last_undegraded":"2023-10-09T06:39:53.089234+0000","last_fullsized":"2023-10-09T06:39:53.089234+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:27:34.543589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.3e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.083972+0000","last_change":"2023-10-09T06:39:53.080593+0000","last_active":"2023-10-09T06:39:53.083972+0000","last_peered":"2023-10-09T06:39:53.083972+0000","last_clean":"2023-10-09T06:39:53.083972+0000","last_became_active":"2023-10-09T06:39:53.080325+0000","last_became_peered":"2023-10-09T06:39:53.080325+0000","last_unstale":"2023-10-09T06:39:53.083972+0000","last_undegraded":"2023-10-09T06:39:53.083972+0000","last_fullsized":"2023-10-09T06:39:53.083972+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:14:35.310477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.091828+0000","last_change":"2023-10-09T06:39:53.085906+0000","last_active":"2023-10-09T06:39:53.091828+0000","last_peered":"2023-10-09T06:39:53.091828+0000","last_clean":"2023-10-09T06:39:53.091828+0000","last_became_active":"2023-10-09T06:39:53.085659+0000","last_became_peered":"2023-10-09T06:39:53.085659+0000","last_unstale":"2023-10-09T06:39:53.091828+0000","last_undegraded":"2023-10-09T06:39:53.091828+0000","last_fullsized":"2023-10-09T06:39:53.091828+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:16:03.156409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.3c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.088003+0000","last_change":"2023-10-09T06:39:53.085913+0000","last_active":"2023-10-09T06:39:53.088003+0000","last_peered":"2023-10-09T06:39:53.088003+0000","last_clean":"2023-10-09T06:39:53.088003+0000","last_became_active":"2023-10-09T06:39:53.085631+0000","last_became_peered":"2023-10-09T06:39:53.085631+0000","last_unstale":"2023-10-09T06:39:53.088003+0000","last_undegraded":"2023-10-09T06:39:53.088003+0000","last_fullsized":"2023-10-09T06:39:53.088003+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:27:58.872044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.3b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.081904+0000","last_change":"2023-10-09T06:39:53.077555+0000","last_active":"2023-10-09T06:39:53.081904+0000","last_peered":"2023-10-09T06:39:53.081904+0000","last_clean":"2023-10-09T06:39:53.081904+0000","last_became_active":"2023-10-09T06:39:53.077294+0000","last_became_peered":"2023-10-09T06:39:53.077294+0000","last_unstale":"2023-10-09T06:39:53.081904+0000","last_undegraded":"2023-10-09T06:39:53.081904+0000","last_fullsized":"2023-10-09T06:39:53.081904+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:18:51.600579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3a","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.085929+0000","last_change":"2023-10-09T06:39:53.083376+0000","last_active":"2023-10-09T06:39:53.085929+0000","last_peered":"2023-10-09T06:39:53.085929+0000","last_clean":"2023-10-09T06:39:53.085929+0000","last_became_active":"2023-10-09T06:39:53.083112+0000","last_became_peered":"2023-10-09T06:39:53.083112+0000","last_unstale":"2023-10-09T06:39:53.085929+0000","last_undegraded":"2023-10-09T06:39:53.085929+0000","last_fullsized":"2023-10-09T06:39:53.085929+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:12:21.943169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.39","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.092240+0000","last_change":"2023-10-09T06:39:53.087854+0000","last_active":"2023-10-09T06:39:53.092240+0000","last_peered":"2023-10-09T06:39:53.092240+0000","last_clean":"2023-10-09T06:39:53.092240+0000","last_became_active":"2023-10-09T06:39:53.087576+0000","last_became_peered":"2023-10-09T06:39:53.087576+0000","last_unstale":"2023-10-09T06:39:53.092240+0000","last_undegraded":"2023-10-09T06:39:53.092240+0000","last_fullsized":"2023-10-09T06:39:53.092240+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:29:10.172886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.38","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.084701+0000","last_change":"2023-10-09T06:39:53.077764+0000","last_active":"2023-10-09T06:39:53.084701+0000","last_peered":"2023-10-09T06:39:53.084701+0000","last_clean":"2023-10-09T06:39:53.084701+0000","last_became_active":"2023-10-09T06:39:53.077514+0000","last_became_peered":"2023-10-09T06:39:53.077514+0000","last_unstale":"2023-10-09T06:39:53.084701+0000","last_undegraded":"2023-10-09T06:39:53.084701+0000","last_fullsized":"2023-10-09T06:39:53.084701+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:02:38.457232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.37","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.089697+0000","last_change":"2023-10-09T06:39:53.087859+0000","last_active":"2023-10-09T06:39:53.089697+0000","last_peered":"2023-10-09T06:39:53.089697+0000","last_clean":"2023-10-09T06:39:53.089697+0000","last_became_active":"2023-10-09T06:39:53.087272+0000","last_became_peered":"2023-10-09T06:39:53.087272+0000","last_unstale":"2023-10-09T06:39:53.089697+0000","last_undegraded":"2023-10-09T06:39:53.089697+0000","last_fullsized":"2023-10-09T06:39:53.089697+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:02:11.161904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.36","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.091854+0000","last_change":"2023-10-09T06:39:53.086702+0000","last_active":"2023-10-09T06:39:53.091854+0000","last_peered":"2023-10-09T06:39:53.091854+0000","last_clean":"2023-10-09T06:39:53.091854+0000","last_became_active":"2023-10-09T06:39:53.086381+0000","last_became_peered":"2023-10-09T06:39:53.086381+0000","last_unstale":"2023-10-09T06:39:53.091854+0000","last_undegraded":"2023-10-09T06:39:53.091854+0000","last_fullsized":"2023-10-09T06:39:53.091854+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:30:34.855906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.35","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.082750+0000","last_change":"2023-10-09T06:39:53.081502+0000","last_active":"2023-10-09T06:39:53.082750+0000","last_peered":"2023-10-09T06:39:53.082750+0000","last_clean":"2023-10-09T06:39:53.082750+0000","last_became_active":"2023-10-09T06:39:53.080997+0000","last_became_peered":"2023-10-09T06:39:53.080997+0000","last_unstale":"2023-10-09T06:39:53.082750+0000","last_undegraded":"2023-10-09T06:39:53.082750+0000","last_fullsized":"2023-10-09T06:39:53.082750+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:49:45.615239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.34","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.092819+0000","last_change":"2023-10-09T06:39:53.090829+0000","last_active":"2023-10-09T06:39:53.092819+0000","last_peered":"2023-10-09T06:39:53.092819+0000","last_clean":"2023-10-09T06:39:53.092819+0000","last_became_active":"2023-10-09T06:39:53.090294+0000","last_became_peered":"2023-10-09T06:39:53.090294+0000","last_unstale":"2023-10-09T06:39:53.092819+0000","last_undegraded":"2023-10-09T06:39:53.092819+0000","last_fullsized":"2023-10-09T06:39:53.092819+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:39:27.333849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.33","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.087111+0000","last_change":"2023-10-09T06:39:53.084305+0000","last_active":"2023-10-09T06:39:53.087111+0000","last_peered":"2023-10-09T06:39:53.087111+0000","last_clean":"2023-10-09T06:39:53.087111+0000","last_became_active":"2023-10-09T06:39:53.084022+0000","last_became_peered":"2023-10-09T06:39:53.084022+0000","last_unstale":"2023-10-09T06:39:53.087111+0000","last_undegraded":"2023-10-09T06:39:53.087111+0000","last_fullsized":"2023-10-09T06:39:53.087111+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:02:19.682732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.32","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.083947+0000","last_change":"2023-10-09T06:39:53.079249+0000","last_active":"2023-10-09T06:39:53.083947+0000","last_peered":"2023-10-09T06:39:53.083947+0000","last_clean":"2023-10-09T06:39:53.083947+0000","last_became_active":"2023-10-09T06:39:53.078043+0000","last_became_peered":"2023-10-09T06:39:53.078043+0000","last_unstale":"2023-10-09T06:39:53.083947+0000","last_undegraded":"2023-10-09T06:39:53.083947+0000","last_fullsized":"2023-10-09T06:39:53.083947+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:58:55.339669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.31","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.087170+0000","last_change":"2023-10-09T06:39:53.083004+0000","last_active":"2023-10-09T06:39:53.087170+0000","last_peered":"2023-10-09T06:39:53.087170+0000","last_clean":"2023-10-09T06:39:53.087170+0000","last_became_active":"2023-10-09T06:39:53.082727+0000","last_became_peered":"2023-10-09T06:39:53.082727+0000","last_unstale":"2023-10-09T06:39:53.087170+0000","last_undegraded":"2023-10-09T06:39:53.087170+0000","last_fullsized":"2023-10-09T06:39:53.087170+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:18:51.600579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.30","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.089491+0000","last_change":"2023-10-09T06:39:53.082325+0000","last_active":"2023-10-09T06:39:53.089491+0000","last_peered":"2023-10-09T06:39:53.089491+0000","last_clean":"2023-10-09T06:39:53.089491+0000","last_became_active":"2023-10-09T06:39:53.082098+0000","last_became_peered":"2023-10-09T06:39:53.082098+0000","last_unstale":"2023-10-09T06:39:53.089491+0000","last_undegraded":"2023-10-09T06:39:53.089491+0000","last_fullsized":"2023-10-09T06:39:53.089491+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:48:58.176885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.089434+0000","last_change":"2023-10-09T06:39:53.080816+0000","last_active":"2023-10-09T06:39:53.089434+0000","last_peered":"2023-10-09T06:39:53.089434+0000","last_clean":"2023-10-09T06:39:53.089434+0000","last_became_active":"2023-10-09T06:39:53.080465+0000","last_became_peered":"2023-10-09T06:39:53.080465+0000","last_unstale":"2023-10-09T06:39:53.089434+0000","last_undegraded":"2023-10-09T06:39:53.089434+0000","last_fullsized":"2023-10-09T06:39:53.089434+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:27:34.543589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.083613+0000","last_change":"2023-10-09T06:39:53.080867+0000","last_active":"2023-10-09T06:39:53.083613+0000","last_peered":"2023-10-09T06:39:53.083613+0000","last_clean":"2023-10-09T06:39:53.083613+0000","last_became_active":"2023-10-09T06:39:53.080615+0000","last_became_peered":"2023-10-09T06:39:53.080615+0000","last_unstale":"2023-10-09T06:39:53.083613+0000","last_undegraded":"2023-10-09T06:39:53.083613+0000","last_fullsized":"2023-10-09T06:39:53.083613+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:21:17.222488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.089450+0000","last_change":"2023-10-09T06:39:53.080919+0000","last_active":"2023-10-09T06:39:53.089450+0000","last_peered":"2023-10-09T06:39:53.089450+0000","last_clean":"2023-10-09T06:39:53.089450+0000","last_became_active":"2023-10-09T06:39:53.080650+0000","last_became_peered":"2023-10-09T06:39:53.080650+0000","last_unstale":"2023-10-09T06:39:53.089450+0000","last_undegraded":"2023-10-09T06:39:53.089450+0000","last_fullsized":"2023-10-09T06:39:53.089450+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:30:11.170346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.092797+0000","last_change":"2023-10-09T06:39:53.090488+0000","last_active":"2023-10-09T06:39:53.092797+0000","last_peered":"2023-10-09T06:39:53.092797+0000","last_clean":"2023-10-09T06:39:53.092797+0000","last_became_active":"2023-10-09T06:39:53.089700+0000","last_became_peered":"2023-10-09T06:39:53.089700+0000","last_unstale":"2023-10-09T06:39:53.092797+0000","last_undegraded":"2023-10-09T06:39:53.092797+0000","last_fullsized":"2023-10-09T06:39:53.092797+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:08:01.872171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.2b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.081871+0000","last_change":"2023-10-09T06:39:53.077357+0000","last_active":"2023-10-09T06:39:53.081871+0000","last_peered":"2023-10-09T06:39:53.081871+0000","last_clean":"2023-10-09T06:39:53.081871+0000","last_became_active":"2023-10-09T06:39:53.077094+0000","last_became_peered":"2023-10-09T06:39:53.077094+0000","last_unstale":"2023-10-09T06:39:53.081871+0000","last_undegraded":"2023-10-09T06:39:53.081871+0000","last_fullsized":"2023-10-09T06:39:53.081871+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:08:59.720890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.2a","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.089158+0000","last_change":"2023-10-09T06:39:53.085482+0000","last_active":"2023-10-09T06:39:53.089158+0000","last_peered":"2023-10-09T06:39:53.089158+0000","last_clean":"2023-10-09T06:39:53.089158+0000","last_became_active":"2023-10-09T06:39:53.084983+0000","last_became_peered":"2023-10-09T06:39:53.084983+0000","last_unstale":"2023-10-09T06:39:53.089158+0000","last_undegraded":"2023-10-09T06:39:53.089158+0000","last_fullsized":"2023-10-09T06:39:53.089158+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:51:23.445071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.29","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.083823+0000","last_change":"2023-10-09T06:39:53.080735+0000","last_active":"2023-10-09T06:39:53.083823+0000","last_peered":"2023-10-09T06:39:53.083823+0000","last_clean":"2023-10-09T06:39:53.083823+0000","last_became_active":"2023-10-09T06:39:53.080452+0000","last_became_peered":"2023-10-09T06:39:53.080452+0000","last_unstale":"2023-10-09T06:39:53.083823+0000","last_undegraded":"2023-10-09T06:39:53.083823+0000","last_fullsized":"2023-10-09T06:39:53.083823+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:35:42.393159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.28","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.082815+0000","last_change":"2023-10-09T06:39:53.081539+0000","last_active":"2023-10-09T06:39:53.082815+0000","last_peered":"2023-10-09T06:39:53.082815+0000","last_clean":"2023-10-09T06:39:53.082815+0000","last_became_active":"2023-10-09T06:39:53.081258+0000","last_became_peered":"2023-10-09T06:39:53.081258+0000","last_unstale":"2023-10-09T06:39:53.082815+0000","last_undegraded":"2023-10-09T06:39:53.082815+0000","last_fullsized":"2023-10-09T06:39:53.082815+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:56:08.251031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.27","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.089126+0000","last_change":"2023-10-09T06:39:53.084208+0000","last_active":"2023-10-09T06:39:53.089126+0000","last_peered":"2023-10-09T06:39:53.089126+0000","last_clean":"2023-10-09T06:39:53.089126+0000","last_became_active":"2023-10-09T06:39:53.083953+0000","last_became_peered":"2023-10-09T06:39:53.083953+0000","last_unstale":"2023-10-09T06:39:53.089126+0000","last_undegraded":"2023-10-09T06:39:53.089126+0000","last_fullsized":"2023-10-09T06:39:53.089126+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:50:05.704255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.26","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.087203+0000","last_change":"2023-10-09T06:39:53.083801+0000","last_active":"2023-10-09T06:39:53.087203+0000","last_peered":"2023-10-09T06:39:53.087203+0000","last_clean":"2023-10-09T06:39:53.087203+0000","last_became_active":"2023-10-09T06:39:53.083472+0000","last_became_peered":"2023-10-09T06:39:53.083472+0000","last_unstale":"2023-10-09T06:39:53.087203+0000","last_undegraded":"2023-10-09T06:39:53.087203+0000","last_fullsized":"2023-10-09T06:39:53.087203+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:21:01.957723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.25","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.091324+0000","last_change":"2023-10-09T06:39:53.085009+0000","last_active":"2023-10-09T06:39:53.091324+0000","last_peered":"2023-10-09T06:39:53.091324+0000","last_clean":"2023-10-09T06:39:53.091324+0000","last_became_active":"2023-10-09T06:39:53.084715+0000","last_became_peered":"2023-10-09T06:39:53.084715+0000","last_unstale":"2023-10-09T06:39:53.091324+0000","last_undegraded":"2023-10-09T06:39:53.091324+0000","last_fullsized":"2023-10-09T06:39:53.091324+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:30:34.855906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.24","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.089200+0000","last_change":"2023-10-09T06:39:53.085818+0000","last_active":"2023-10-09T06:39:53.089200+0000","last_peered":"2023-10-09T06:39:53.089200+0000","last_clean":"2023-10-09T06:39:53.089200+0000","last_became_active":"2023-10-09T06:39:53.084411+0000","last_became_peered":"2023-10-09T06:39:53.084411+0000","last_unstale":"2023-10-09T06:39:53.089200+0000","last_undegraded":"2023-10-09T06:39:53.089200+0000","last_fullsized":"2023-10-09T06:39:53.089200+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:30:11.170346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.22","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.087983+0000","last_change":"2023-10-09T06:39:53.084816+0000","last_active":"2023-10-09T06:39:53.087983+0000","last_peered":"2023-10-09T06:39:53.087983+0000","last_clean":"2023-10-09T06:39:53.087983+0000","last_became_active":"2023-10-09T06:39:53.084535+0000","last_became_peered":"2023-10-09T06:39:53.084535+0000","last_unstale":"2023-10-09T06:39:53.087983+0000","last_undegraded":"2023-10-09T06:39:53.087983+0000","last_fullsized":"2023-10-09T06:39:53.087983+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:30:48.220988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.21","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.083291+0000","last_change":"2023-10-09T06:39:53.078286+0000","last_active":"2023-10-09T06:39:53.083291+0000","last_peered":"2023-10-09T06:39:53.083291+0000","last_clean":"2023-10-09T06:39:53.083291+0000","last_became_active":"2023-10-09T06:39:53.077850+0000","last_became_peered":"2023-10-09T06:39:53.077850+0000","last_unstale":"2023-10-09T06:39:53.083291+0000","last_undegraded":"2023-10-09T06:39:53.083291+0000","last_fullsized":"2023-10-09T06:39:53.083291+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:31:52.134120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.20","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.083507+0000","last_change":"2023-10-09T06:39:53.080068+0000","last_active":"2023-10-09T06:39:53.083507+0000","last_peered":"2023-10-09T06:39:53.083507+0000","last_clean":"2023-10-09T06:39:53.083507+0000","last_became_active":"2023-10-09T06:39:53.079104+0000","last_became_peered":"2023-10-09T06:39:53.079104+0000","last_unstale":"2023-10-09T06:39:53.083507+0000","last_undegraded":"2023-10-09T06:39:53.083507+0000","last_fullsized":"2023-10-09T06:39:53.083507+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:16:10.435246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.1c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.091980+0000","last_change":"2023-10-09T06:39:53.087891+0000","last_active":"2023-10-09T06:39:53.091980+0000","last_peered":"2023-10-09T06:39:53.091980+0000","last_clean":"2023-10-09T06:39:53.091980+0000","last_became_active":"2023-10-09T06:39:53.087616+0000","last_became_peered":"2023-10-09T06:39:53.087616+0000","last_unstale":"2023-10-09T06:39:53.091980+0000","last_undegraded":"2023-10-09T06:39:53.091980+0000","last_fullsized":"2023-10-09T06:39:53.091980+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:02:42.734000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1f","version":"196'4","reported_seq":14,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.098093+0000","last_change":"2023-10-09T06:39:52.081149+0000","last_active":"2023-10-09T06:39:53.098093+0000","last_peered":"2023-10-09T06:39:53.098093+0000","last_clean":"2023-10-09T06:39:53.098093+0000","last_became_active":"2023-10-09T06:39:52.080868+0000","last_became_peered":"2023-10-09T06:39:52.080868+0000","last_unstale":"2023-10-09T06:39:53.098093+0000","last_undegraded":"2023-10-09T06:39:53.098093+0000","last_fullsized":"2023-10-09T06:39:53.098093+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:14:56.789141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.083080+0000","last_change":"2023-10-09T06:39:53.080497+0000","last_active":"2023-10-09T06:39:53.083080+0000","last_peered":"2023-10-09T06:39:53.083080+0000","last_clean":"2023-10-09T06:39:53.083080+0000","last_became_active":"2023-10-09T06:39:53.080249+0000","last_became_peered":"2023-10-09T06:39:53.080249+0000","last_unstale":"2023-10-09T06:39:53.083080+0000","last_undegraded":"2023-10-09T06:39:53.083080+0000","last_fullsized":"2023-10-09T06:39:53.083080+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:22:03.142568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"196'1","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.096566+0000","last_change":"2023-10-09T06:39:52.081470+0000","last_active":"2023-10-09T06:39:53.096566+0000","last_peered":"2023-10-09T06:39:53.096566+0000","last_clean":"2023-10-09T06:39:53.096566+0000","last_became_active":"2023-10-09T06:39:52.081170+0000","last_became_peered":"2023-10-09T06:39:52.081170+0000","last_unstale":"2023-10-09T06:39:53.096566+0000","last_undegraded":"2023-10-09T06:39:53.096566+0000","last_fullsized":"2023-10-09T06:39:53.096566+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:34:37.947853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.1e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.083585+0000","last_change":"2023-10-09T06:39:53.078610+0000","last_active":"2023-10-09T06:39:53.083585+0000","last_peered":"2023-10-09T06:39:53.083585+0000","last_clean":"2023-10-09T06:39:53.083585+0000","last_became_active":"2023-10-09T06:39:53.078292+0000","last_became_peered":"2023-10-09T06:39:53.078292+0000","last_unstale":"2023-10-09T06:39:53.083585+0000","last_undegraded":"2023-10-09T06:39:53.083585+0000","last_fullsized":"2023-10-09T06:39:53.083585+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:01:10.092288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1d","version":"196'2","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.098126+0000","last_change":"2023-10-09T06:39:52.083228+0000","last_active":"2023-10-09T06:39:53.098126+0000","last_peered":"2023-10-09T06:39:53.098126+0000","last_clean":"2023-10-09T06:39:53.098126+0000","last_became_active":"2023-10-09T06:39:52.082704+0000","last_became_peered":"2023-10-09T06:39:52.082704+0000","last_unstale":"2023-10-09T06:39:53.098126+0000","last_undegraded":"2023-10-09T06:39:53.098126+0000","last_fullsized":"2023-10-09T06:39:53.098126+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:55:53.294887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.1f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.088667+0000","last_change":"2023-10-09T06:39:53.086019+0000","last_active":"2023-10-09T06:39:53.088667+0000","last_peered":"2023-10-09T06:39:53.088667+0000","last_clean":"2023-10-09T06:39:53.088667+0000","last_became_active":"2023-10-09T06:39:53.085692+0000","last_became_peered":"2023-10-09T06:39:53.085692+0000","last_unstale":"2023-10-09T06:39:53.088667+0000","last_undegraded":"2023-10-09T06:39:53.088667+0000","last_fullsized":"2023-10-09T06:39:53.088667+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:46:36.690844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"196'2","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.099256+0000","last_change":"2023-10-09T06:39:52.078914+0000","last_active":"2023-10-09T06:39:53.099256+0000","last_peered":"2023-10-09T06:39:53.099256+0000","last_clean":"2023-10-09T06:39:53.099256+0000","last_became_active":"2023-10-09T06:39:52.078619+0000","last_became_peered":"2023-10-09T06:39:52.078619+0000","last_unstale":"2023-10-09T06:39:53.099256+0000","last_undegraded":"2023-10-09T06:39:53.099256+0000","last_fullsized":"2023-10-09T06:39:53.099256+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:17:04.704861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.086176+0000","last_change":"2023-10-09T06:39:53.082694+0000","last_active":"2023-10-09T06:39:53.086176+0000","last_peered":"2023-10-09T06:39:53.086176+0000","last_clean":"2023-10-09T06:39:53.086176+0000","last_became_active":"2023-10-09T06:39:53.082415+0000","last_became_peered":"2023-10-09T06:39:53.082415+0000","last_unstale":"2023-10-09T06:39:53.086176+0000","last_undegraded":"2023-10-09T06:39:53.086176+0000","last_fullsized":"2023-10-09T06:39:53.086176+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:36:18.420616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1b","version":"196'1","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.098688+0000","last_change":"2023-10-09T06:39:52.076258+0000","last_active":"2023-10-09T06:39:53.098688+0000","last_peered":"2023-10-09T06:39:53.098688+0000","last_clean":"2023-10-09T06:39:53.098688+0000","last_became_active":"2023-10-09T06:39:52.075552+0000","last_became_peered":"2023-10-09T06:39:52.075552+0000","last_unstale":"2023-10-09T06:39:53.098688+0000","last_undegraded":"2023-10-09T06:39:53.098688+0000","last_fullsized":"2023-10-09T06:39:53.098688+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:21:00.953510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.083321+0000","last_change":"2023-10-09T06:39:53.078827+0000","last_active":"2023-10-09T06:39:53.083321+0000","last_peered":"2023-10-09T06:39:53.083321+0000","last_clean":"2023-10-09T06:39:53.083321+0000","last_became_active":"2023-10-09T06:39:53.078562+0000","last_became_peered":"2023-10-09T06:39:53.078562+0000","last_unstale":"2023-10-09T06:39:53.083321+0000","last_undegraded":"2023-10-09T06:39:53.083321+0000","last_fullsized":"2023-10-09T06:39:53.083321+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:24:17.586832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"196'1","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.098216+0000","last_change":"2023-10-09T06:39:52.082928+0000","last_active":"2023-10-09T06:39:53.098216+0000","last_peered":"2023-10-09T06:39:53.098216+0000","last_clean":"2023-10-09T06:39:53.098216+0000","last_became_active":"2023-10-09T06:39:52.082707+0000","last_became_peered":"2023-10-09T06:39:52.082707+0000","last_unstale":"2023-10-09T06:39:53.098216+0000","last_undegraded":"2023-10-09T06:39:53.098216+0000","last_fullsized":"2023-10-09T06:39:53.098216+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:39:26.329636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.1a","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.092279+0000","last_change":"2023-10-09T06:39:53.088119+0000","last_active":"2023-10-09T06:39:53.092279+0000","last_peered":"2023-10-09T06:39:53.092279+0000","last_clean":"2023-10-09T06:39:53.092279+0000","last_became_active":"2023-10-09T06:39:53.087858+0000","last_became_peered":"2023-10-09T06:39:53.087858+0000","last_unstale":"2023-10-09T06:39:53.092279+0000","last_undegraded":"2023-10-09T06:39:53.092279+0000","last_fullsized":"2023-10-09T06:39:53.092279+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:16:55.231556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.075316+0000","last_change":"2023-10-09T06:39:52.075825+0000","last_active":"2023-10-09T06:39:53.075316+0000","last_peered":"2023-10-09T06:39:53.075316+0000","last_clean":"2023-10-09T06:39:53.075316+0000","last_became_active":"2023-10-09T06:39:52.075568+0000","last_became_peered":"2023-10-09T06:39:52.075568+0000","last_unstale":"2023-10-09T06:39:53.075316+0000","last_undegraded":"2023-10-09T06:39:53.075316+0000","last_fullsized":"2023-10-09T06:39:53.075316+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:42:30.773943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.1b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.084398+0000","last_change":"2023-10-09T06:39:53.077739+0000","last_active":"2023-10-09T06:39:53.084398+0000","last_peered":"2023-10-09T06:39:53.084398+0000","last_clean":"2023-10-09T06:39:53.084398+0000","last_became_active":"2023-10-09T06:39:53.077483+0000","last_became_peered":"2023-10-09T06:39:53.077483+0000","last_unstale":"2023-10-09T06:39:53.084398+0000","last_undegraded":"2023-10-09T06:39:53.084398+0000","last_fullsized":"2023-10-09T06:39:53.084398+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:17:04.774066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.072490+0000","last_change":"2023-10-09T06:39:52.079426+0000","last_active":"2023-10-09T06:39:53.072490+0000","last_peered":"2023-10-09T06:39:53.072490+0000","last_clean":"2023-10-09T06:39:53.072490+0000","last_became_active":"2023-10-09T06:39:52.079159+0000","last_became_peered":"2023-10-09T06:39:52.079159+0000","last_unstale":"2023-10-09T06:39:53.072490+0000","last_undegraded":"2023-10-09T06:39:53.072490+0000","last_fullsized":"2023-10-09T06:39:53.072490+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:27:11.297409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.14","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.088025+0000","last_change":"2023-10-09T06:39:53.086083+0000","last_active":"2023-10-09T06:39:53.088025+0000","last_peered":"2023-10-09T06:39:53.088025+0000","last_clean":"2023-10-09T06:39:53.088025+0000","last_became_active":"2023-10-09T06:39:53.085770+0000","last_became_peered":"2023-10-09T06:39:53.085770+0000","last_unstale":"2023-10-09T06:39:53.088025+0000","last_undegraded":"2023-10-09T06:39:53.088025+0000","last_fullsized":"2023-10-09T06:39:53.088025+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:53:10.470109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.068622+0000","last_change":"2023-10-09T06:39:52.081093+0000","last_active":"2023-10-09T06:39:53.068622+0000","last_peered":"2023-10-09T06:39:53.068622+0000","last_clean":"2023-10-09T06:39:53.068622+0000","last_became_active":"2023-10-09T06:39:52.080822+0000","last_became_peered":"2023-10-09T06:39:52.080822+0000","last_unstale":"2023-10-09T06:39:53.068622+0000","last_undegraded":"2023-10-09T06:39:53.068622+0000","last_fullsized":"2023-10-09T06:39:53.068622+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:46:33.850505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.15","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.086582+0000","last_change":"2023-10-09T06:39:53.082263+0000","last_active":"2023-10-09T06:39:53.086582+0000","last_peered":"2023-10-09T06:39:53.086582+0000","last_clean":"2023-10-09T06:39:53.086582+0000","last_became_active":"2023-10-09T06:39:53.081999+0000","last_became_peered":"2023-10-09T06:39:53.081999+0000","last_unstale":"2023-10-09T06:39:53.086582+0000","last_undegraded":"2023-10-09T06:39:53.086582+0000","last_fullsized":"2023-10-09T06:39:53.086582+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:25:00.261989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.075411+0000","last_change":"2023-10-09T06:39:52.076723+0000","last_active":"2023-10-09T06:39:53.075411+0000","last_peered":"2023-10-09T06:39:53.075411+0000","last_clean":"2023-10-09T06:39:53.075411+0000","last_became_active":"2023-10-09T06:39:52.076455+0000","last_became_peered":"2023-10-09T06:39:52.076455+0000","last_unstale":"2023-10-09T06:39:53.075411+0000","last_undegraded":"2023-10-09T06:39:53.075411+0000","last_fullsized":"2023-10-09T06:39:53.075411+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:58:29.869457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.088436+0000","last_change":"2023-10-09T06:39:53.084322+0000","last_active":"2023-10-09T06:39:53.088436+0000","last_peered":"2023-10-09T06:39:53.088436+0000","last_clean":"2023-10-09T06:39:53.088436+0000","last_became_active":"2023-10-09T06:39:53.084117+0000","last_became_peered":"2023-10-09T06:39:53.084117+0000","last_unstale":"2023-10-09T06:39:53.088436+0000","last_undegraded":"2023-10-09T06:39:53.088436+0000","last_fullsized":"2023-10-09T06:39:53.088436+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:52:10.706737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.15","version":"196'3","reported_seq":13,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.099243+0000","last_change":"2023-10-09T06:39:52.078205+0000","last_active":"2023-10-09T06:39:53.099243+0000","last_peered":"2023-10-09T06:39:53.099243+0000","last_clean":"2023-10-09T06:39:53.099243+0000","last_became_active":"2023-10-09T06:39:52.077945+0000","last_became_peered":"2023-10-09T06:39:52.077945+0000","last_unstale":"2023-10-09T06:39:53.099243+0000","last_undegraded":"2023-10-09T06:39:53.099243+0000","last_fullsized":"2023-10-09T06:39:53.099243+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:24:16.582618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.089064+0000","last_change":"2023-10-09T06:39:53.084391+0000","last_active":"2023-10-09T06:39:53.089064+0000","last_peered":"2023-10-09T06:39:53.089064+0000","last_clean":"2023-10-09T06:39:53.089064+0000","last_became_active":"2023-10-09T06:39:53.084044+0000","last_became_peered":"2023-10-09T06:39:53.084044+0000","last_unstale":"2023-10-09T06:39:53.089064+0000","last_undegraded":"2023-10-09T06:39:53.089064+0000","last_fullsized":"2023-10-09T06:39:53.089064+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:29:46.222421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"196'6","reported_seq":16,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.102794+0000","last_change":"2023-10-09T06:39:52.074833+0000","last_active":"2023-10-09T06:39:53.102794+0000","last_peered":"2023-10-09T06:39:53.102794+0000","last_clean":"2023-10-09T06:39:53.102794+0000","last_became_active":"2023-10-09T06:39:52.074588+0000","last_became_peered":"2023-10-09T06:39:52.074588+0000","last_unstale":"2023-10-09T06:39:53.102794+0000","last_undegraded":"2023-10-09T06:39:53.102794+0000","last_fullsized":"2023-10-09T06:39:53.102794+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:19:27.971639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.10","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.085978+0000","last_change":"2023-10-09T06:39:53.081740+0000","last_active":"2023-10-09T06:39:53.085978+0000","last_peered":"2023-10-09T06:39:53.085978+0000","last_clean":"2023-10-09T06:39:53.085978+0000","last_became_active":"2023-10-09T06:39:53.081489+0000","last_became_peered":"2023-10-09T06:39:53.081489+0000","last_unstale":"2023-10-09T06:39:53.085978+0000","last_undegraded":"2023-10-09T06:39:53.085978+0000","last_fullsized":"2023-10-09T06:39:53.085978+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:58:55.339669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"196'2","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.098774+0000","last_change":"2023-10-09T06:39:52.078081+0000","last_active":"2023-10-09T06:39:53.098774+0000","last_peered":"2023-10-09T06:39:53.098774+0000","last_clean":"2023-10-09T06:39:53.098774+0000","last_became_active":"2023-10-09T06:39:52.077809+0000","last_became_peered":"2023-10-09T06:39:52.077809+0000","last_unstale":"2023-10-09T06:39:53.098774+0000","last_undegraded":"2023-10-09T06:39:53.098774+0000","last_fullsized":"2023-10-09T06:39:53.098774+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:12:50.712857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.091329+0000","last_change":"2023-10-09T06:39:53.085476+0000","last_active":"2023-10-09T06:39:53.091329+0000","last_peered":"2023-10-09T06:39:53.091329+0000","last_clean":"2023-10-09T06:39:53.091329+0000","last_became_active":"2023-10-09T06:39:53.084884+0000","last_became_peered":"2023-10-09T06:39:53.084884+0000","last_unstale":"2023-10-09T06:39:53.091329+0000","last_undegraded":"2023-10-09T06:39:53.091329+0000","last_fullsized":"2023-10-09T06:39:53.091329+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:52:02.602342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.068418+0000","last_change":"2023-10-09T06:39:52.079034+0000","last_active":"2023-10-09T06:39:53.068418+0000","last_peered":"2023-10-09T06:39:53.068418+0000","last_clean":"2023-10-09T06:39:53.068418+0000","last_became_active":"2023-10-09T06:39:52.078754+0000","last_became_peered":"2023-10-09T06:39:52.078754+0000","last_unstale":"2023-10-09T06:39:53.068418+0000","last_undegraded":"2023-10-09T06:39:53.068418+0000","last_fullsized":"2023-10-09T06:39:53.068418+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:16:09.431033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.082996+0000","last_change":"2023-10-09T06:39:53.079455+0000","last_active":"2023-10-09T06:39:53.082996+0000","last_peered":"2023-10-09T06:39:53.082996+0000","last_clean":"2023-10-09T06:39:53.082996+0000","last_became_active":"2023-10-09T06:39:53.079217+0000","last_became_peered":"2023-10-09T06:39:53.079217+0000","last_unstale":"2023-10-09T06:39:53.082996+0000","last_undegraded":"2023-10-09T06:39:53.082996+0000","last_fullsized":"2023-10-09T06:39:53.082996+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:31:52.134120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.068259+0000","last_change":"2023-10-09T06:39:52.081786+0000","last_active":"2023-10-09T06:39:53.068259+0000","last_peered":"2023-10-09T06:39:53.068259+0000","last_clean":"2023-10-09T06:39:53.068259+0000","last_became_active":"2023-10-09T06:39:52.081540+0000","last_became_peered":"2023-10-09T06:39:52.081540+0000","last_unstale":"2023-10-09T06:39:53.068259+0000","last_undegraded":"2023-10-09T06:39:53.068259+0000","last_fullsized":"2023-10-09T06:39:53.068259+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:46:35.686631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.13","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.082968+0000","last_change":"2023-10-09T06:39:53.079904+0000","last_active":"2023-10-09T06:39:53.082968+0000","last_peered":"2023-10-09T06:39:53.082968+0000","last_clean":"2023-10-09T06:39:53.082968+0000","last_became_active":"2023-10-09T06:39:53.079644+0000","last_became_peered":"2023-10-09T06:39:53.079644+0000","last_unstale":"2023-10-09T06:39:53.082968+0000","last_undegraded":"2023-10-09T06:39:53.082968+0000","last_fullsized":"2023-10-09T06:39:53.082968+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:46:36.690844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.10","version":"196'8","reported_seq":18,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.100671+0000","last_change":"2023-10-09T06:39:52.078042+0000","last_active":"2023-10-09T06:39:53.100671+0000","last_peered":"2023-10-09T06:39:53.100671+0000","last_clean":"2023-10-09T06:39:53.100671+0000","last_became_active":"2023-10-09T06:39:52.077784+0000","last_became_peered":"2023-10-09T06:39:52.077784+0000","last_unstale":"2023-10-09T06:39:53.100671+0000","last_undegraded":"2023-10-09T06:39:53.100671+0000","last_fullsized":"2023-10-09T06:39:53.100671+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:31:51.129907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.088793+0000","last_change":"2023-10-09T06:39:53.084069+0000","last_active":"2023-10-09T06:39:53.088793+0000","last_peered":"2023-10-09T06:39:53.088793+0000","last_clean":"2023-10-09T06:39:53.088793+0000","last_became_active":"2023-10-09T06:39:53.083711+0000","last_became_peered":"2023-10-09T06:39:53.083711+0000","last_unstale":"2023-10-09T06:39:53.088793+0000","last_undegraded":"2023-10-09T06:39:53.088793+0000","last_fullsized":"2023-10-09T06:39:53.088793+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:41:09.798707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"196'1","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.100195+0000","last_change":"2023-10-09T06:39:52.083381+0000","last_active":"2023-10-09T06:39:53.100195+0000","last_peered":"2023-10-09T06:39:53.100195+0000","last_clean":"2023-10-09T06:39:53.100195+0000","last_became_active":"2023-10-09T06:39:52.083088+0000","last_became_peered":"2023-10-09T06:39:52.083088+0000","last_unstale":"2023-10-09T06:39:53.100195+0000","last_undegraded":"2023-10-09T06:39:53.100195+0000","last_fullsized":"2023-10-09T06:39:53.100195+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:30:10.166133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.088398+0000","last_change":"2023-10-09T06:39:53.083385+0000","last_active":"2023-10-09T06:39:53.088398+0000","last_peered":"2023-10-09T06:39:53.088398+0000","last_clean":"2023-10-09T06:39:53.088398+0000","last_became_active":"2023-10-09T06:39:53.083122+0000","last_became_peered":"2023-10-09T06:39:53.083122+0000","last_unstale":"2023-10-09T06:39:53.088398+0000","last_undegraded":"2023-10-09T06:39:53.088398+0000","last_fullsized":"2023-10-09T06:39:53.088398+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:04:13.567267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.e","version":"196'2","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.095093+0000","last_change":"2023-10-09T06:39:52.081771+0000","last_active":"2023-10-09T06:39:53.095093+0000","last_peered":"2023-10-09T06:39:53.095093+0000","last_clean":"2023-10-09T06:39:53.095093+0000","last_became_active":"2023-10-09T06:39:52.081496+0000","last_became_peered":"2023-10-09T06:39:52.081496+0000","last_unstale":"2023-10-09T06:39:53.095093+0000","last_undegraded":"2023-10-09T06:39:53.095093+0000","last_fullsized":"2023-10-09T06:39:53.095093+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:10:17.630559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.084736+0000","last_change":"2023-10-09T06:39:53.078433+0000","last_active":"2023-10-09T06:39:53.084736+0000","last_peered":"2023-10-09T06:39:53.084736+0000","last_clean":"2023-10-09T06:39:53.084736+0000","last_became_active":"2023-10-09T06:39:53.078191+0000","last_became_peered":"2023-10-09T06:39:53.078191+0000","last_unstale":"2023-10-09T06:39:53.084736+0000","last_undegraded":"2023-10-09T06:39:53.084736+0000","last_fullsized":"2023-10-09T06:39:53.084736+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:23:39.439786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.d","version":"196'1","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.099638+0000","last_change":"2023-10-09T06:39:52.087273+0000","last_active":"2023-10-09T06:39:53.099638+0000","last_peered":"2023-10-09T06:39:53.099638+0000","last_clean":"2023-10-09T06:39:53.099638+0000","last_became_active":"2023-10-09T06:39:52.086958+0000","last_became_peered":"2023-10-09T06:39:52.086958+0000","last_unstale":"2023-10-09T06:39:53.099638+0000","last_undegraded":"2023-10-09T06:39:53.099638+0000","last_fullsized":"2023-10-09T06:39:53.099638+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:01:09.088075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.089252+0000","last_change":"2023-10-09T06:39:53.086046+0000","last_active":"2023-10-09T06:39:53.089252+0000","last_peered":"2023-10-09T06:39:53.089252+0000","last_clean":"2023-10-09T06:39:53.089252+0000","last_became_active":"2023-10-09T06:39:53.084655+0000","last_became_peered":"2023-10-09T06:39:53.084655+0000","last_unstale":"2023-10-09T06:39:53.089252+0000","last_undegraded":"2023-10-09T06:39:53.089252+0000","last_fullsized":"2023-10-09T06:39:53.089252+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:53:31.682242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.071791+0000","last_change":"2023-10-09T06:39:52.081850+0000","last_active":"2023-10-09T06:39:53.071791+0000","last_peered":"2023-10-09T06:39:53.071791+0000","last_clean":"2023-10-09T06:39:53.071791+0000","last_became_active":"2023-10-09T06:39:52.081576+0000","last_became_peered":"2023-10-09T06:39:52.081576+0000","last_unstale":"2023-10-09T06:39:53.071791+0000","last_undegraded":"2023-10-09T06:39:53.071791+0000","last_fullsized":"2023-10-09T06:39:53.071791+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:23:46.437369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.086156+0000","last_change":"2023-10-09T06:39:53.082189+0000","last_active":"2023-10-09T06:39:53.086156+0000","last_peered":"2023-10-09T06:39:53.086156+0000","last_clean":"2023-10-09T06:39:53.086156+0000","last_became_active":"2023-10-09T06:39:53.081956+0000","last_became_peered":"2023-10-09T06:39:53.081956+0000","last_unstale":"2023-10-09T06:39:53.086156+0000","last_undegraded":"2023-10-09T06:39:53.086156+0000","last_fullsized":"2023-10-09T06:39:53.086156+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:08:59.720890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.b","version":"196'2","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.098797+0000","last_change":"2023-10-09T06:39:52.078643+0000","last_active":"2023-10-09T06:39:53.098797+0000","last_peered":"2023-10-09T06:39:53.098797+0000","last_clean":"2023-10-09T06:39:53.098797+0000","last_became_active":"2023-10-09T06:39:52.078338+0000","last_became_peered":"2023-10-09T06:39:52.078338+0000","last_unstale":"2023-10-09T06:39:53.098797+0000","last_undegraded":"2023-10-09T06:39:53.098797+0000","last_fullsized":"2023-10-09T06:39:53.098797+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:36:17.416403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.091345+0000","last_change":"2023-10-09T06:39:53.085656+0000","last_active":"2023-10-09T06:39:53.091345+0000","last_peered":"2023-10-09T06:39:53.091345+0000","last_clean":"2023-10-09T06:39:53.091345+0000","last_became_active":"2023-10-09T06:39:53.085219+0000","last_became_peered":"2023-10-09T06:39:53.085219+0000","last_unstale":"2023-10-09T06:39:53.091345+0000","last_undegraded":"2023-10-09T06:39:53.091345+0000","last_fullsized":"2023-10-09T06:39:53.091345+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:42:31.778156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.075064+0000","last_change":"2023-10-09T06:39:52.079690+0000","last_active":"2023-10-09T06:39:53.075064+0000","last_peered":"2023-10-09T06:39:53.075064+0000","last_clean":"2023-10-09T06:39:53.075064+0000","last_became_active":"2023-10-09T06:39:52.079438+0000","last_became_peered":"2023-10-09T06:39:52.079438+0000","last_unstale":"2023-10-09T06:39:53.075064+0000","last_undegraded":"2023-10-09T06:39:53.075064+0000","last_fullsized":"2023-10-09T06:39:53.075064+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:35:41.388946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.085473+0000","last_change":"2023-10-09T06:39:53.082971+0000","last_active":"2023-10-09T06:39:53.085473+0000","last_peered":"2023-10-09T06:39:53.085473+0000","last_clean":"2023-10-09T06:39:53.085473+0000","last_became_active":"2023-10-09T06:39:53.082716+0000","last_became_peered":"2023-10-09T06:39:53.082716+0000","last_unstale":"2023-10-09T06:39:53.085473+0000","last_undegraded":"2023-10-09T06:39:53.085473+0000","last_fullsized":"2023-10-09T06:39:53.085473+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:49:58.700962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.072685+0000","last_change":"2023-10-09T06:39:52.081467+0000","last_active":"2023-10-09T06:39:53.072685+0000","last_peered":"2023-10-09T06:39:53.072685+0000","last_clean":"2023-10-09T06:39:53.072685+0000","last_became_active":"2023-10-09T06:39:52.081208+0000","last_became_peered":"2023-10-09T06:39:52.081208+0000","last_unstale":"2023-10-09T06:39:53.072685+0000","last_undegraded":"2023-10-09T06:39:53.072685+0000","last_fullsized":"2023-10-09T06:39:53.072685+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:30:47.216775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.083475+0000","last_change":"2023-10-09T06:39:53.079349+0000","last_active":"2023-10-09T06:39:53.083475+0000","last_peered":"2023-10-09T06:39:53.083475+0000","last_clean":"2023-10-09T06:39:53.083475+0000","last_became_active":"2023-10-09T06:39:53.078726+0000","last_became_peered":"2023-10-09T06:39:53.078726+0000","last_unstale":"2023-10-09T06:39:53.083475+0000","last_undegraded":"2023-10-09T06:39:53.083475+0000","last_fullsized":"2023-10-09T06:39:53.083475+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:35:29.957992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.068712+0000","last_change":"2023-10-09T06:39:52.076805+0000","last_active":"2023-10-09T06:39:53.068712+0000","last_peered":"2023-10-09T06:39:53.068712+0000","last_clean":"2023-10-09T06:39:53.068712+0000","last_became_active":"2023-10-09T06:39:52.076346+0000","last_became_peered":"2023-10-09T06:39:52.076346+0000","last_unstale":"2023-10-09T06:39:53.068712+0000","last_undegraded":"2023-10-09T06:39:53.068712+0000","last_fullsized":"2023-10-09T06:39:53.068712+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:14:34.306264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.092809+0000","last_change":"2023-10-09T06:39:53.090665+0000","last_active":"2023-10-09T06:39:53.092809+0000","last_peered":"2023-10-09T06:39:53.092809+0000","last_clean":"2023-10-09T06:39:53.092809+0000","last_became_active":"2023-10-09T06:39:53.090019+0000","last_became_peered":"2023-10-09T06:39:53.090019+0000","last_unstale":"2023-10-09T06:39:53.092809+0000","last_undegraded":"2023-10-09T06:39:53.092809+0000","last_fullsized":"2023-10-09T06:39:53.092809+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:09:42.222904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.7","version":"196'2","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.098935+0000","last_change":"2023-10-09T06:39:52.077134+0000","last_active":"2023-10-09T06:39:53.098935+0000","last_peered":"2023-10-09T06:39:53.098935+0000","last_clean":"2023-10-09T06:39:53.098935+0000","last_became_active":"2023-10-09T06:39:52.076120+0000","last_became_peered":"2023-10-09T06:39:52.076120+0000","last_unstale":"2023-10-09T06:39:53.098935+0000","last_undegraded":"2023-10-09T06:39:53.098935+0000","last_fullsized":"2023-10-09T06:39:53.098935+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:35:28.953779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.089521+0000","last_change":"2023-10-09T06:39:53.085769+0000","last_active":"2023-10-09T06:39:53.089521+0000","last_peered":"2023-10-09T06:39:53.089521+0000","last_clean":"2023-10-09T06:39:53.089521+0000","last_became_active":"2023-10-09T06:39:53.085483+0000","last_became_peered":"2023-10-09T06:39:53.085483+0000","last_unstale":"2023-10-09T06:39:53.089521+0000","last_undegraded":"2023-10-09T06:39:53.089521+0000","last_fullsized":"2023-10-09T06:39:53.089521+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:34:14.228003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.073429+0000","last_change":"2023-10-09T06:39:52.076909+0000","last_active":"2023-10-09T06:39:53.073429+0000","last_peered":"2023-10-09T06:39:53.073429+0000","last_clean":"2023-10-09T06:39:53.073429+0000","last_became_active":"2023-10-09T06:39:52.076643+0000","last_became_peered":"2023-10-09T06:39:52.076643+0000","last_unstale":"2023-10-09T06:39:53.073429+0000","last_undegraded":"2023-10-09T06:39:53.073429+0000","last_fullsized":"2023-10-09T06:39:53.073429+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:23:45.416793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.088757+0000","last_change":"2023-10-09T06:39:53.085178+0000","last_active":"2023-10-09T06:39:53.088757+0000","last_peered":"2023-10-09T06:39:53.088757+0000","last_clean":"2023-10-09T06:39:53.088757+0000","last_became_active":"2023-10-09T06:39:53.084894+0000","last_became_peered":"2023-10-09T06:39:53.084894+0000","last_unstale":"2023-10-09T06:39:53.088757+0000","last_undegraded":"2023-10-09T06:39:53.088757+0000","last_fullsized":"2023-10-09T06:39:53.088757+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:35:46.409202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.079054+0000","last_change":"2023-10-09T06:39:52.082785+0000","last_active":"2023-10-09T06:39:53.079054+0000","last_peered":"2023-10-09T06:39:53.079054+0000","last_clean":"2023-10-09T06:39:53.079054+0000","last_became_active":"2023-10-09T06:39:52.082532+0000","last_became_peered":"2023-10-09T06:39:52.082532+0000","last_unstale":"2023-10-09T06:39:53.079054+0000","last_undegraded":"2023-10-09T06:39:53.079054+0000","last_fullsized":"2023-10-09T06:39:53.079054+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:07:23.025013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.7","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.085225+0000","last_change":"2023-10-09T06:39:53.082346+0000","last_active":"2023-10-09T06:39:53.085225+0000","last_peered":"2023-10-09T06:39:53.085225+0000","last_clean":"2023-10-09T06:39:53.085225+0000","last_became_active":"2023-10-09T06:39:53.082050+0000","last_became_peered":"2023-10-09T06:39:53.082050+0000","last_unstale":"2023-10-09T06:39:53.085225+0000","last_undegraded":"2023-10-09T06:39:53.085225+0000","last_fullsized":"2023-10-09T06:39:53.085225+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:25:09.337293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.078532+0000","last_change":"2023-10-09T06:39:52.087119+0000","last_active":"2023-10-09T06:39:53.078532+0000","last_peered":"2023-10-09T06:39:53.078532+0000","last_clean":"2023-10-09T06:39:53.078532+0000","last_became_active":"2023-10-09T06:39:52.086876+0000","last_became_peered":"2023-10-09T06:39:52.086876+0000","last_unstale":"2023-10-09T06:39:53.078532+0000","last_undegraded":"2023-10-09T06:39:53.078532+0000","last_fullsized":"2023-10-09T06:39:53.078532+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:22:02.138355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.084681+0000","last_change":"2023-10-09T06:39:53.077077+0000","last_active":"2023-10-09T06:39:53.084681+0000","last_peered":"2023-10-09T06:39:53.084681+0000","last_clean":"2023-10-09T06:39:53.084681+0000","last_became_active":"2023-10-09T06:39:53.076811+0000","last_became_peered":"2023-10-09T06:39:53.076811+0000","last_unstale":"2023-10-09T06:39:53.084681+0000","last_undegraded":"2023-10-09T06:39:53.084681+0000","last_fullsized":"2023-10-09T06:39:53.084681+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:03:00.699198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.071606+0000","last_change":"2023-10-09T06:39:52.079543+0000","last_active":"2023-10-09T06:39:53.071606+0000","last_peered":"2023-10-09T06:39:53.071606+0000","last_clean":"2023-10-09T06:39:53.071606+0000","last_became_active":"2023-10-09T06:39:52.079279+0000","last_became_peered":"2023-10-09T06:39:52.079279+0000","last_unstale":"2023-10-09T06:39:53.071606+0000","last_undegraded":"2023-10-09T06:39:53.071606+0000","last_fullsized":"2023-10-09T06:39:53.071606+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:04:19.759255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.087193+0000","last_change":"2023-10-09T06:39:53.083876+0000","last_active":"2023-10-09T06:39:53.087193+0000","last_peered":"2023-10-09T06:39:53.087193+0000","last_clean":"2023-10-09T06:39:53.087193+0000","last_became_active":"2023-10-09T06:39:53.083594+0000","last_became_peered":"2023-10-09T06:39:53.083594+0000","last_unstale":"2023-10-09T06:39:53.087193+0000","last_undegraded":"2023-10-09T06:39:53.087193+0000","last_fullsized":"2023-10-09T06:39:53.087193+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:33:59.174559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.071582+0000","last_change":"2023-10-09T06:39:52.078285+0000","last_active":"2023-10-09T06:39:53.071582+0000","last_peered":"2023-10-09T06:39:53.071582+0000","last_clean":"2023-10-09T06:39:53.071582+0000","last_became_active":"2023-10-09T06:39:52.078011+0000","last_became_peered":"2023-10-09T06:39:52.078011+0000","last_unstale":"2023-10-09T06:39:53.071582+0000","last_undegraded":"2023-10-09T06:39:53.071582+0000","last_fullsized":"2023-10-09T06:39:53.071582+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:39:21.726799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.083929+0000","last_change":"2023-10-09T06:39:53.078203+0000","last_active":"2023-10-09T06:39:53.083929+0000","last_peered":"2023-10-09T06:39:53.083929+0000","last_clean":"2023-10-09T06:39:53.083929+0000","last_became_active":"2023-10-09T06:39:53.077948+0000","last_became_peered":"2023-10-09T06:39:53.077948+0000","last_unstale":"2023-10-09T06:39:53.083929+0000","last_undegraded":"2023-10-09T06:39:53.083929+0000","last_fullsized":"2023-10-09T06:39:53.083929+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:46:34.854718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.078204+0000","last_change":"2023-10-09T06:39:52.083232+0000","last_active":"2023-10-09T06:39:53.078204+0000","last_peered":"2023-10-09T06:39:53.078204+0000","last_clean":"2023-10-09T06:39:53.078204+0000","last_became_active":"2023-10-09T06:39:52.082987+0000","last_became_peered":"2023-10-09T06:39:52.082987+0000","last_unstale":"2023-10-09T06:39:53.078204+0000","last_undegraded":"2023-10-09T06:39:53.078204+0000","last_fullsized":"2023-10-09T06:39:53.078204+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:24:59.257776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.086379+0000","last_change":"2023-10-09T06:39:53.077883+0000","last_active":"2023-10-09T06:39:53.086379+0000","last_peered":"2023-10-09T06:39:53.086379+0000","last_clean":"2023-10-09T06:39:53.086379+0000","last_became_active":"2023-10-09T06:39:53.077639+0000","last_became_peered":"2023-10-09T06:39:53.077639+0000","last_unstale":"2023-10-09T06:39:53.086379+0000","last_undegraded":"2023-10-09T06:39:53.086379+0000","last_fullsized":"2023-10-09T06:39:53.086379+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:52:02.602342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.071395+0000","last_change":"2023-10-09T06:39:52.077364+0000","last_active":"2023-10-09T06:39:53.071395+0000","last_peered":"2023-10-09T06:39:53.071395+0000","last_clean":"2023-10-09T06:39:53.071395+0000","last_became_active":"2023-10-09T06:39:52.077098+0000","last_became_peered":"2023-10-09T06:39:52.077098+0000","last_unstale":"2023-10-09T06:39:53.071395+0000","last_undegraded":"2023-10-09T06:39:53.071395+0000","last_fullsized":"2023-10-09T06:39:53.071395+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:34:55.758128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.23","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.087129+0000","last_change":"2023-10-09T06:39:53.084392+0000","last_active":"2023-10-09T06:39:53.087129+0000","last_peered":"2023-10-09T06:39:53.087129+0000","last_clean":"2023-10-09T06:39:53.087129+0000","last_became_active":"2023-10-09T06:39:53.084132+0000","last_became_peered":"2023-10-09T06:39:53.084132+0000","last_unstale":"2023-10-09T06:39:53.087129+0000","last_undegraded":"2023-10-09T06:39:53.087129+0000","last_fullsized":"2023-10-09T06:39:53.087129+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:45:42.010880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":308,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.077723+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:39:53.077723+0000","last_peered":"2023-10-09T06:39:53.077723+0000","last_clean":"2023-10-09T06:39:53.077723+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:39:53.077723+0000","last_undegraded":"2023-10-09T06:39:53.077723+0000","last_fullsized":"2023-10-09T06:39:53.077723+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":34,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":33,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574965,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304420,"kb_used_data":3036,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067420,"statfs":{"total":96636764160,"available":96325038080,"internally_reserved":0,"allocated":3108864,"data_stored":1716694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.55600000000000005,"15min":0.52300000000000002},"min":{"1min":0.442,"5min":0.308,"15min":0.308},"max":{"1min":3.778,"5min":3.778,"15min":3.778},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.51400000000000001,"15min":0.51500000000000001},"min":{"1min":0.35599999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":0.63300000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.36599999999999999,"5min":0.33600000000000002,"15min":0.27100000000000002},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.54500000000000004,"15min":0.53200000000000003},"min":{"1min":0.39000000000000001,"5min":0.32900000000000001,"15min":0.29699999999999999},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.496}]},{"osd":2,"last update":"Mon Oct 9 06:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.55800000000000005,"15min":0.52900000000000003},"min":{"1min":0.38300000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":3.7210000000000001,"5min":3.7210000000000001,"15min":3.7210000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.315},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.60499999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.57899999999999996,"15min":0.56200000000000006},"min":{"1min":0.38600000000000001,"5min":0.35899999999999999,"15min":0.33500000000000002},"max":{"1min":3.8090000000000002,"5min":3.8090000000000002,"15min":3.8090000000000002},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56200000000000006,"15min":0.58499999999999996},"min":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.36499999999999999},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.56299999999999994}]},{"osd":4,"last update":"Mon Oct 9 06:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.51400000000000001,"15min":0.51900000000000002},"min":{"1min":0.28899999999999998,"5min":0.25600000000000001,"15min":0.24399999999999999},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.628,"15min":0.61499999999999999},"min":{"1min":0.39700000000000002,"5min":0.35199999999999998,"15min":0.30299999999999999},"max":{"1min":4.8239999999999998,"5min":4.8239999999999998,"15min":4.8239999999999998},"last":0.76000000000000001}]},{"osd":5,"last update":"Mon Oct 9 06:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54900000000000004,"15min":0.56399999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.307},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.64200000000000002,"15min":0.626},"min":{"1min":0.35299999999999998,"5min":0.34300000000000003,"15min":0.34200000000000003},"max":{"1min":3.6909999999999998,"5min":3.6909999999999998,"15min":3.6909999999999998},"last":0.747}]},{"osd":6,"last update":"Mon Oct 9 06:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.36399999999999999,"5min":0.318,"15min":0.28699999999999998},"max":{"1min":3.5529999999999999,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.63700000000000001,"15min":0.621},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.315},"max":{"1min":4.7789999999999999,"5min":4.7789999999999999,"15min":4.7789999999999999},"last":0.53800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574964,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303836,"kb_used_data":2452,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068004,"statfs":{"total":96636764160,"available":96325636096,"internally_reserved":0,"allocated":2510848,"data_stored":1125784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52500000000000002,"15min":0.50700000000000001},"min":{"1min":0.41299999999999998,"5min":0.35399999999999998,"15min":0.30399999999999999},"max":{"1min":0.80500000000000005,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.752},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52400000000000002,"15min":0.51500000000000001},"min":{"1min":0.33000000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.67400000000000004}]},{"osd":1,"last update":"Mon Oct 9 06:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.50900000000000001,"15min":0.51000000000000001},"min":{"1min":0.41299999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.65900000000000003,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54600000000000004,"15min":0.53500000000000003},"min":{"1min":0.42999999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73699999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.73799999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.375,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71199999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52700000000000002,"15min":0.53000000000000003},"min":{"1min":0.438,"5min":0.313,"15min":0.313},"max":{"1min":0.71699999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.62}]},{"osd":3,"last update":"Mon Oct 9 06:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.39400000000000002,"5min":0.38900000000000001,"15min":0.33500000000000002},"max":{"1min":0.76500000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55900000000000005,"15min":0.65900000000000003},"min":{"1min":0.46400000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74099999999999999,"5min":0.76900000000000002,"15min":4.7510000000000003},"last":0.72699999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.46999999999999997,"15min":0.47599999999999998},"min":{"1min":0.29299999999999998,"5min":0.28199999999999997,"15min":0.27900000000000003},"max":{"1min":0.72799999999999998,"5min":0.74299999999999999,"15min":0.76400000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57099999999999995,"15min":0.58199999999999996},"min":{"1min":0.38700000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.78600000000000003,"5min":0.80800000000000005,"15min":0.83699999999999997},"last":0.76900000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52300000000000002,"15min":0.53100000000000003},"min":{"1min":0.375,"5min":0.246,"15min":0.246},"max":{"1min":0.745,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57399999999999995,"15min":0.59099999999999997},"min":{"1min":0.40999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76200000000000001,"5min":0.78000000000000003,"15min":0.86599999999999999},"last":0.68300000000000005}]},{"osd":7,"last update":"Mon Oct 9 06:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.57799999999999996},"min":{"1min":0.34799999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.81499999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.38400000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.84399999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.69599999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303840,"kb_used_data":2456,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068000,"statfs":{"total":96636764160,"available":96325632000,"internally_reserved":0,"allocated":2514944,"data_stored":1125874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.315,"15min":0.31},"min":{"1min":0.22900000000000001,"5min":0.192,"15min":0.192},"max":{"1min":0.46200000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35399999999999998,"15min":0.35799999999999998},"min":{"1min":0.26500000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.48599999999999999,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.46200000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.34300000000000003,"15min":0.34799999999999998},"min":{"1min":0.24099999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.44,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.378},{"interface":"front","average":{"1min":0.373,"5min":0.38100000000000001,"15min":0.38400000000000001},"min":{"1min":0.27100000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.49399999999999999,"5min":0.60399999999999998,"15min":0.60899999999999999},"last":0.48799999999999999}]},{"osd":3,"last update":"Mon Oct 9 06:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.372,"15min":0.36499999999999999},"min":{"1min":0.254,"5min":0.22,"15min":0.22},"max":{"1min":0.52700000000000002,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.34599999999999997,"15min":0.49399999999999999},"min":{"1min":0.23799999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.54800000000000004,"5min":0.58199999999999996,"15min":5.8529999999999998},"last":0.50600000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57799999999999996,"15min":0.57699999999999996},"min":{"1min":0.371,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.873,"5min":1.01,"15min":1.01},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.42599999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.79400000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.68100000000000005}]},{"osd":5,"last update":"Mon Oct 9 06:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57199999999999995,"15min":0.70399999999999996},"min":{"1min":0.39500000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73699999999999999,"5min":0.92500000000000004,"15min":5.9459999999999997},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59999999999999998,"15min":0.60699999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.34300000000000003},"max":{"1min":0.76700000000000002,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.71299999999999997}]},{"osd":6,"last update":"Mon Oct 9 06:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.54700000000000004,"15min":0.67600000000000005},"min":{"1min":0.39500000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.80800000000000005,"5min":0.81399999999999995,"15min":5.9160000000000004},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57499999999999996,"15min":0.57799999999999996},"min":{"1min":0.42399999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83899999999999997,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.68700000000000006}]},{"osd":7,"last update":"Mon Oct 9 06:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56299999999999994,"15min":0.58099999999999996},"min":{"1min":0.42999999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.73299999999999998,"5min":0.91000000000000003,"15min":0.97099999999999997},"last":0.628},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56200000000000006,"15min":0.70699999999999996},"min":{"1min":0.43099999999999999,"5min":0.40400000000000003,"15min":0.38100000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":5.9729999999999999},"last":0.69799999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574966,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304416,"kb_used_data":3032,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067424,"statfs":{"total":96636764160,"available":96325042176,"internally_reserved":0,"allocated":3104768,"data_stored":1716188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.32500000000000001,"15min":0.31900000000000001},"min":{"1min":0.253,"5min":0.214,"15min":0.21299999999999999},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34100000000000003,"15min":0.34399999999999997},"min":{"1min":0.24199999999999999,"5min":0.222,"15min":0.214},"max":{"1min":0.51300000000000001,"5min":0.53600000000000003,"15min":0.94499999999999995},"last":0.42599999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.371,"15min":0.36399999999999999},"min":{"1min":0.22,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.495,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38900000000000001,"15min":0.38800000000000001},"min":{"1min":0.33200000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.49199999999999999,"5min":0.69399999999999995,"15min":0.91700000000000004},"last":0.46400000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.375},"min":{"1min":0.23999999999999999,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":1.554},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.373,"5min":0.375,"15min":0.39300000000000002},"min":{"1min":0.28100000000000003,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.47299999999999998,"5min":0.58099999999999996,"15min":0.79000000000000004},"last":0.40899999999999997}]},{"osd":4,"last update":"Mon Oct 9 06:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56200000000000006,"15min":0.55700000000000005},"min":{"1min":0.41499999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":1.212},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56799999999999995,"15min":0.68799999999999994},"min":{"1min":0.41599999999999998,"5min":0.37,"15min":0.34799999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":5.484},"last":0.69699999999999995}]},{"osd":5,"last update":"Mon Oct 9 06:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56599999999999995,"15min":0.69099999999999995},"min":{"1min":0.40300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":5.5449999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57799999999999996,"15min":0.57699999999999996},"min":{"1min":0.42199999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.998,"5min":0.998,"15min":1.113},"last":0.56299999999999994}]},{"osd":6,"last update":"Mon Oct 9 06:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.44800000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":1.0529999999999999},"last":0.628},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.45800000000000002,"5min":0.42799999999999999,"15min":0.41599999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":1.169},"last":0.68000000000000005}]},{"osd":7,"last update":"Mon Oct 9 06:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.435,"5min":0.39700000000000002,"15min":0.371},"max":{"1min":0.878,"5min":0.878,"15min":1.2390000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57299999999999995,"15min":0.71199999999999997},"min":{"1min":0.442,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":5.5800000000000001},"last":0.66400000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574967,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303852,"kb_used_data":2468,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067988,"statfs":{"total":96636764160,"available":96325619712,"internally_reserved":0,"allocated":2527232,"data_stored":1127862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:39:52 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.318,"15min":0.34499999999999997},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.19900000000000001},"max":{"1min":0.45700000000000002,"5min":0.66000000000000003,"15min":1.4199999999999999},"last":0.20000000000000001},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.33400000000000002,"15min":0.50900000000000001},"min":{"1min":0.221,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.433,"5min":0.55600000000000005,"15min":7.2699999999999996},"last":0.221}]},{"osd":1,"last update":"Mon Oct 9 06:39:52 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.31900000000000001,"15min":0.33900000000000002},"min":{"1min":0.24199999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.442,"5min":0.55200000000000005,"15min":0.65400000000000003},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.372,"5min":0.376,"15min":0.56000000000000005},"min":{"1min":0.253,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.51600000000000001,"5min":0.57999999999999996,"15min":7.327},"last":0.29599999999999999}]},{"osd":3,"last update":"Mon Oct 9 06:39:52 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.379,"15min":0.38900000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.49299999999999999,"5min":0.64600000000000002,"15min":0.68600000000000005},"last":0.307},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.373,"15min":0.38700000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.23999999999999999},"max":{"1min":0.5,"5min":0.629,"15min":0.72699999999999998},"last":0.252}]},{"osd":4,"last update":"Mon Oct 9 06:39:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54600000000000004,"15min":0.54300000000000004},"min":{"1min":0.40000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.72999999999999998,"5min":0.79700000000000004,"15min":0.80300000000000005},"last":0.438},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56000000000000005,"15min":0.73399999999999999},"min":{"1min":0.38800000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71599999999999997,"5min":0.83199999999999996,"15min":7.2389999999999999},"last":0.38800000000000001}]},{"osd":5,"last update":"Mon Oct 9 06:39:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56999999999999995,"15min":0.83999999999999997},"min":{"1min":0.38100000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77100000000000002,"5min":0.82699999999999996,"15min":10.890000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59799999999999998,"15min":0.60099999999999998},"min":{"1min":0.39800000000000002,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":0.78200000000000003,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.39800000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:39:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56699999999999995,"15min":0.74199999999999999},"min":{"1min":0.35999999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.68999999999999995,"5min":0.77300000000000002,"15min":7.2999999999999998},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57899999999999996,"15min":0.57599999999999996},"min":{"1min":0.436,"5min":0.377,"15min":0.377},"max":{"1min":0.83099999999999996,"5min":0.875,"15min":0.875},"last":0.50700000000000001}]},{"osd":7,"last update":"Mon Oct 9 06:39:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58999999999999997,"15min":0.77100000000000002},"min":{"1min":0.42099999999999999,"5min":0.38600000000000001,"15min":0.37},"max":{"1min":0.81299999999999994,"5min":0.91300000000000003,"15min":7.1879999999999997},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60099999999999998,"15min":0.621},"min":{"1min":0.47699999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.86699999999999999,"5min":0.89800000000000002,"15min":0.90500000000000003},"last":0.56899999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574966,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303836,"kb_used_data":2452,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068004,"statfs":{"total":96636764160,"available":96325636096,"internally_reserved":0,"allocated":2510848,"data_stored":1125840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:39:50 2023","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.32600000000000001,"15min":0.33300000000000002},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.438,"5min":0.61699999999999999,"15min":5.8700000000000001},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.34899999999999998,"15min":0.36499999999999999},"min":{"1min":0.23799999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.45900000000000002,"5min":0.57199999999999995,"15min":5.6769999999999996},"last":0.28299999999999997}]},{"osd":1,"last update":"Mon Oct 9 06:39:50 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.34799999999999998,"15min":0.35799999999999998},"min":{"1min":0.23999999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.47199999999999998,"5min":0.59499999999999997,"15min":4.0469999999999997},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.377,"15min":0.38800000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.51400000000000001,"5min":0.68600000000000005,"15min":4.1509999999999998},"last":0.27600000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:39:50 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.35099999999999998,"15min":0.36799999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.222},"max":{"1min":0.41399999999999998,"5min":0.56799999999999995,"15min":3.1139999999999999},"last":0.254},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.36899999999999999,"15min":0.38300000000000001},"min":{"1min":0.26000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.51500000000000001,"5min":0.66900000000000004,"15min":3.3180000000000001},"last":0.29799999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:39:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55300000000000005,"15min":0.56799999999999995},"min":{"1min":0.38700000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72399999999999998,"5min":0.81699999999999995,"15min":3.0899999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.40699999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71999999999999997,"5min":0.77400000000000002,"15min":3.0710000000000002},"last":0.46700000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:39:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57299999999999995,"15min":0.58599999999999997},"min":{"1min":0.38900000000000001,"5min":0.33600000000000002,"15min":0.32700000000000001},"max":{"1min":0.69299999999999995,"5min":0.877,"15min":6.2300000000000004},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.58399999999999996,"15min":0.59199999999999997},"min":{"1min":0.41799999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":3.0219999999999998},"last":0.38}]},{"osd":6,"last update":"Mon Oct 9 06:39:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58999999999999997,"15min":0.59299999999999997},"min":{"1min":0.42699999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.73799999999999999,"5min":0.89200000000000002,"15min":2.6850000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63700000000000001,"15min":0.64900000000000002},"min":{"1min":0.45700000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.77400000000000002,"5min":0.94299999999999995,"15min":6.5970000000000004},"last":0.51600000000000001}]},{"osd":7,"last update":"Mon Oct 9 06:39:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.45100000000000001,"5min":0.41699999999999998,"15min":0.38200000000000001},"max":{"1min":0.78300000000000003,"5min":0.86099999999999999,"15min":4.2629999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59099999999999997,"15min":0.61099999999999999},"min":{"1min":0.438,"5min":0.374,"15min":0.374},"max":{"1min":0.75700000000000001,"5min":0.82899999999999996,"15min":7.3650000000000002},"last":0.48899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574965,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303844,"kb_used_data":2460,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067996,"statfs":{"total":96636764160,"available":96325627904,"internally_reserved":0,"allocated":2519040,"data_stored":1127286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51500000000000001,"15min":0.54600000000000004},"min":{"1min":0.34499999999999997,"5min":0.28100000000000003,"15min":0.27000000000000002},"max":{"1min":0.79000000000000004,"5min":0.88900000000000001,"15min":1.083},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51700000000000002,"15min":0.51900000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.65600000000000003,"5min":0.97799999999999998,"15min":1.054},"last":0.58599999999999997}]},{"osd":1,"last update":"Mon Oct 9 06:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.53100000000000003,"15min":0.54500000000000004},"min":{"1min":0.34599999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67100000000000004,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.502,"15min":0.498},"min":{"1min":0.38900000000000001,"5min":0.31900000000000001,"15min":0.31},"max":{"1min":0.69999999999999996,"5min":0.76300000000000001,"15min":0.79100000000000004},"last":0.45800000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52200000000000002,"15min":0.53600000000000003},"min":{"1min":0.33100000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84299999999999997,"5min":0.84899999999999998,"15min":1.0249999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54500000000000004,"15min":0.58199999999999996},"min":{"1min":0.41399999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.67900000000000005,"5min":0.90500000000000003,"15min":0.93799999999999994},"last":0.50900000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52200000000000002,"15min":0.54600000000000004},"min":{"1min":0.33700000000000002,"5min":0.32200000000000001,"15min":0.29199999999999998},"max":{"1min":0.69199999999999995,"5min":0.86899999999999999,"15min":0.98499999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57099999999999995,"15min":0.58499999999999996},"min":{"1min":0.42299999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76300000000000001,"5min":0.92900000000000005,"15min":1.1319999999999999},"last":0.47999999999999998}]},{"osd":5,"last update":"Mon Oct 9 06:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.52900000000000003,"15min":0.54700000000000004},"min":{"1min":0.38,"5min":0.30399999999999999,"15min":0.27300000000000002},"max":{"1min":0.73999999999999999,"5min":0.81100000000000005,"15min":0.998},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.58499999999999996,"15min":0.60299999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35099999999999998},"max":{"1min":0.85599999999999998,"5min":0.95599999999999996,"15min":0.97999999999999998},"last":0.498}]},{"osd":6,"last update":"Mon Oct 9 06:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.36899999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.86299999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.47199999999999998,"5min":0.38700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78500000000000003,"5min":1.01,"15min":1.01},"last":0.54900000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.55500000000000005,"15min":0.56699999999999995},"min":{"1min":0.42899999999999999,"5min":0.33600000000000002,"15min":0.28299999999999997},"max":{"1min":0.81399999999999995,"5min":0.83199999999999996,"15min":0.90000000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61899999999999999,"15min":0.63800000000000001},"min":{"1min":0.437,"5min":0.34999999999999998,"15min":0.32200000000000001},"max":{"1min":0.93799999999999994,"5min":0.98999999999999999,"15min":1.175},"last":0.59999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574965,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303836,"kb_used_data":2452,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068004,"statfs":{"total":96636764160,"available":96325636096,"internally_reserved":0,"allocated":2510848,"data_stored":1125820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:39:49 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51100000000000001,"15min":0.50800000000000001},"min":{"1min":0.38400000000000001,"5min":0.30599999999999999,"15min":0.28599999999999998},"max":{"1min":0.64700000000000002,"5min":0.79700000000000004,"15min":0.86599999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.41099999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.65400000000000003}]},{"osd":1,"last update":"Mon Oct 9 06:39:49 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.50800000000000001,"15min":0.50900000000000001},"min":{"1min":0.34499999999999997,"5min":0.33000000000000002,"15min":0.311},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.78200000000000003},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.52000000000000002,"15min":0.52800000000000002},"min":{"1min":0.36899999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63,"5min":0.76600000000000001,"15min":0.81599999999999995},"last":0.55900000000000005}]},{"osd":2,"last update":"Mon Oct 9 06:39:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.55800000000000005,"15min":0.55000000000000004},"min":{"1min":0.34699999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.623},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57199999999999995,"15min":0.58299999999999996},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78200000000000003,"5min":0.79300000000000004,"15min":0.79900000000000004},"last":0.73799999999999999}]},{"osd":3,"last update":"Mon Oct 9 06:39:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55700000000000005,"15min":0.54700000000000004},"min":{"1min":0.39200000000000002,"5min":0.317,"15min":0.317},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54500000000000004,"15min":0.55800000000000005},"min":{"1min":0.42599999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.68700000000000006,"5min":0.71099999999999997,"15min":0.81100000000000005},"last":0.67500000000000004}]},{"osd":4,"last update":"Mon Oct 9 06:39:49 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.504,"15min":0.5},"min":{"1min":0.28399999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.74399999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54900000000000004,"15min":0.54800000000000004},"min":{"1min":0.39600000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.70299999999999996}]},{"osd":6,"last update":"Mon Oct 9 06:39:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.56899999999999995,"15min":0.54700000000000004},"min":{"1min":0.34799999999999998,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.71499999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.63600000000000001}]},{"osd":7,"last update":"Mon Oct 9 06:39:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60099999999999998,"15min":0.60799999999999998},"min":{"1min":0.373,"5min":0.35199999999999998,"15min":0.314},"max":{"1min":0.71099999999999997,"5min":0.84499999999999997,"15min":0.90600000000000003},"last":0.77200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:39:57.141 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:39:57.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.287+0000 7f4006209700 1 Processor -- start 2023-10-09T06:39:57.288 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.288+0000 7f4006209700 1 -- start start 2023-10-09T06:39:57.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.288+0000 7f4006209700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f400006bcd0 0x7f40000caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:57.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.288+0000 7f4006209700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f400015df50 0x7f400015e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:57.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.288+0000 7f4006209700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f400015d7e0 con 0x7f400006c0a0 2023-10-09T06:39:57.296 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.288+0000 7f4006209700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f400015dac0 con 0x7f400015df50 2023-10-09T06:39:57.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.288+0000 7f4006209700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f400015aa30 con 0x7f400006bcd0 2023-10-09T06:39:57.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.288+0000 7f3ffffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f400015df50 0x7f400015e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:57.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.288+0000 7f3ff6ffd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f400006c0a0 0x7f400015a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:36482/0 (socket says 172.21.15.163:36482) 2023-10-09T06:39:57.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.289+0000 7f3ff6ffd700 1 -- 172.21.15.163:0/338986648 learned_addr learned my addr 172.21.15.163:0/338986648 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:57.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.289+0000 7f3fff7fe700 1 --2- 172.21.15.163:0/338986648 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f400006bcd0 0x7f40000caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:57.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.289+0000 7f3ffffff700 1 -- 172.21.15.163:0/338986648 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f400006bcd0 msgr2=0x7f40000caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:57.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.289+0000 7f3ffffff700 1 --2- 172.21.15.163:0/338986648 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f400006bcd0 0x7f40000caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:57.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.289+0000 7f3ffffff700 1 -- 172.21.15.163:0/338986648 >> v1:172.21.15.163:6789/0 conn(0x7f400006c0a0 legacy=0x7f400015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:57.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.289+0000 7f3ffffff700 1 -- 172.21.15.163:0/338986648 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400015ad10 con 0x7f400015df50 2023-10-09T06:39:57.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.289+0000 7f3ffeffd700 1 -- 172.21.15.163:0/338986648 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1415559594 0 0) 0x7f400015d7e0 con 0x7f400006c0a0 2023-10-09T06:39:57.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.289+0000 7f3ffffff700 1 --2- 172.21.15.163:0/338986648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f400015df50 0x7f400015e3b0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009e60 tx=0x7f3ff8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b150369c4d24a1cf server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:57.297 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.289+0000 7f3ffeffd700 1 -- 172.21.15.163:0/338986648 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff800e070 con 0x7f400015df50 2023-10-09T06:39:57.298 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.289+0000 7f3ffeffd700 1 -- 172.21.15.163:0/338986648 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff800bc30 con 0x7f400015df50 2023-10-09T06:39:57.298 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.289+0000 7f3ffeffd700 1 -- 172.21.15.163:0/338986648 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8012430 con 0x7f400015df50 2023-10-09T06:39:57.298 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.290+0000 7f4006209700 1 -- 172.21.15.163:0/338986648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f400015df50 msgr2=0x7f400015e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:57.298 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.290+0000 7f4006209700 1 --2- 172.21.15.163:0/338986648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f400015df50 0x7f400015e3b0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009e60 tx=0x7f3ff8009780 comp rx=0 tx=0).stop 2023-10-09T06:39:57.298 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.290+0000 7f4006209700 1 -- 172.21.15.163:0/338986648 shutdown_connections 2023-10-09T06:39:57.298 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.290+0000 7f4006209700 1 --2- 172.21.15.163:0/338986648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f400015df50 0x7f400015e3b0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:57.298 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.290+0000 7f4006209700 1 --2- 172.21.15.163:0/338986648 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f400006bcd0 0x7f40000caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:57.298 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.290+0000 7f4006209700 1 -- 172.21.15.163:0/338986648 >> 172.21.15.163:0/338986648 conn(0x7f40000ce600 msgr2=0x7f40000cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:57.298 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.290+0000 7f4006209700 1 -- 172.21.15.163:0/338986648 shutdown_connections 2023-10-09T06:39:57.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.290+0000 7f4006209700 1 -- 172.21.15.163:0/338986648 wait complete. 2023-10-09T06:39:57.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.291+0000 7f4006209700 1 Processor -- start 2023-10-09T06:39:57.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.291+0000 7f4006209700 1 -- start start 2023-10-09T06:39:57.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.292+0000 7f4006209700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f400006bcd0 0x7f40000c6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:57.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.292+0000 7f4006209700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f400006c0a0 0x7f40001598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:57.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.292+0000 7f4006209700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f400015df50 0x7f40001491e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:57.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.292+0000 7f4006209700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40000c7360 con 0x7f400015df50 2023-10-09T06:39:57.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.292+0000 7f4006209700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4000159e30 con 0x7f400006c0a0 2023-10-09T06:39:57.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.292+0000 7f4006209700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40000c6750 con 0x7f400006bcd0 2023-10-09T06:39:57.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.292+0000 7f3ffffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f400015df50 0x7f40001491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:57.299 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.292+0000 7f3ffffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f400015df50 0x7f40001491e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49182/0 (socket says 172.21.15.163:49182) 2023-10-09T06:39:57.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.292+0000 7f3ffffff700 1 -- 172.21.15.163:0/1869693053 learned_addr learned my addr 172.21.15.163:0/1869693053 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:57.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.292+0000 7f3fff7fe700 1 --2- 172.21.15.163:0/1869693053 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f400006bcd0 0x7f40000c6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:57.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.292+0000 7f3ff6ffd700 1 --2- 172.21.15.163:0/1869693053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f400006c0a0 0x7f40001598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:57.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.292+0000 7f3ffffff700 1 -- 172.21.15.163:0/1869693053 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f400006bcd0 msgr2=0x7f40000c6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:57.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.292+0000 7f3ffffff700 1 --2- 172.21.15.163:0/1869693053 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f400006bcd0 0x7f40000c6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:57.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.292+0000 7f3ffffff700 1 -- 172.21.15.163:0/1869693053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f400006c0a0 msgr2=0x7f40001598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:57.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.292+0000 7f3ffffff700 1 --2- 172.21.15.163:0/1869693053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f400006c0a0 0x7f40001598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:57.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.292+0000 7f3ffffff700 1 -- 172.21.15.163:0/1869693053 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff8009a20 con 0x7f400015df50 2023-10-09T06:39:57.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.292+0000 7f3ffffff700 1 --2- 172.21.15.163:0/1869693053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f400015df50 0x7f40001491e0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800b6c0 tx=0x7f3ff8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:57.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.292+0000 7f3ff7fff700 1 -- 172.21.15.163:0/1869693053 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8007040 con 0x7f400015df50 2023-10-09T06:39:57.300 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.292+0000 7f3ff7fff700 1 -- 172.21.15.163:0/1869693053 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff80263f0 con 0x7f400015df50 2023-10-09T06:39:57.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.292+0000 7f4006209700 1 -- 172.21.15.163:0/1869693053 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40000c6a30 con 0x7f400015df50 2023-10-09T06:39:57.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.293+0000 7f3ff7fff700 1 -- 172.21.15.163:0/1869693053 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff80267b0 con 0x7f400015df50 2023-10-09T06:39:57.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.293+0000 7f4006209700 1 -- 172.21.15.163:0/1869693053 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40000c6ff0 con 0x7f400015df50 2023-10-09T06:39:57.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.293+0000 7f4006209700 1 -- 172.21.15.163:0/1869693053 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40000c74d0 con 0x7f400015df50 2023-10-09T06:39:57.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.294+0000 7f3ff7fff700 1 -- 172.21.15.163:0/1869693053 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3ff8026ba0 con 0x7f400015df50 2023-10-09T06:39:57.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.294+0000 7f3ff7fff700 1 --2- 172.21.15.163:0/1869693053 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3fe0067320 0x7f3fe00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:57.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.294+0000 7f3ff7fff700 1 -- 172.21.15.163:0/1869693053 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3ff8086cb0 con 0x7f400015df50 2023-10-09T06:39:57.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.294+0000 7f3fff7fe700 1 --2- 172.21.15.163:0/1869693053 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3fe0067320 0x7f3fe00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:57.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.295+0000 7f3fff7fe700 1 --2- 172.21.15.163:0/1869693053 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3fe0067320 0x7f3fe00697d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0006fd0 tx=0x7f3ff0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:57.307 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.298+0000 7f3ff7fff700 1 -- 172.21.15.163:0/1869693053 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ff8053830 con 0x7f400015df50 2023-10-09T06:39:57.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.476+0000 7f4006209700 1 -- 172.21.15.163:0/1869693053 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f400004c3c0 con 0x7f3fe0067320 2023-10-09T06:39:57.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.484+0000 7f3ff7fff700 1 -- 172.21.15.163:0/1869693053 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289885 (secure 0 0 0) 0x7f400004c3c0 con 0x7f3fe0067320 2023-10-09T06:39:57.488 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:57.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.487+0000 7f4006209700 1 -- 172.21.15.163:0/1869693053 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3fe0067320 msgr2=0x7f3fe00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:57.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.487+0000 7f4006209700 1 --2- 172.21.15.163:0/1869693053 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3fe0067320 0x7f3fe00697d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0006fd0 tx=0x7f3ff0008040 comp rx=0 tx=0).stop 2023-10-09T06:39:57.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.487+0000 7f4006209700 1 -- 172.21.15.163:0/1869693053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f400015df50 msgr2=0x7f40001491e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:57.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.487+0000 7f4006209700 1 --2- 172.21.15.163:0/1869693053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f400015df50 0x7f40001491e0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800b6c0 tx=0x7f3ff8019000 comp rx=0 tx=0).stop 2023-10-09T06:39:57.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.487+0000 7f4006209700 1 -- 172.21.15.163:0/1869693053 shutdown_connections 2023-10-09T06:39:57.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.487+0000 7f4006209700 1 --2- 172.21.15.163:0/1869693053 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3fe0067320 0x7f3fe00697d0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:57.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.487+0000 7f4006209700 1 --2- 172.21.15.163:0/1869693053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f400015df50 0x7f40001491e0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:57.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.487+0000 7f4006209700 1 --2- 172.21.15.163:0/1869693053 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f400006bcd0 0x7f40000c6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:57.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.487+0000 7f4006209700 1 --2- 172.21.15.163:0/1869693053 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f400006c0a0 0x7f40001598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:57.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.487+0000 7f4006209700 1 -- 172.21.15.163:0/1869693053 >> 172.21.15.163:0/1869693053 conn(0x7f40000ce600 msgr2=0x7f400006cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:57.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.488+0000 7f4006209700 1 -- 172.21.15.163:0/1869693053 shutdown_connections 2023-10-09T06:39:57.491 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.488+0000 7f4006209700 1 -- 172.21.15.163:0/1869693053 wait complete. 2023-10-09T06:39:57.491 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:39:57.500 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":473,"stamp":"2023-10-09T06:39:56.433167+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2431880,"kb_used_data":20808,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752542840,"statfs":{"total":773094113280,"available":770603868160,"internally_reserved":0,"allocated":21307392,"data_stored":10191348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.374054"},"pg_stats":[{"pgid":"34.3f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.089234+0000","last_change":"2023-10-09T06:39:53.085861+0000","last_active":"2023-10-09T06:39:53.089234+0000","last_peered":"2023-10-09T06:39:53.089234+0000","last_clean":"2023-10-09T06:39:53.089234+0000","last_became_active":"2023-10-09T06:39:53.084847+0000","last_became_peered":"2023-10-09T06:39:53.084847+0000","last_unstale":"2023-10-09T06:39:53.089234+0000","last_undegraded":"2023-10-09T06:39:53.089234+0000","last_fullsized":"2023-10-09T06:39:53.089234+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:27:34.543589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.3e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.083972+0000","last_change":"2023-10-09T06:39:53.080593+0000","last_active":"2023-10-09T06:39:53.083972+0000","last_peered":"2023-10-09T06:39:53.083972+0000","last_clean":"2023-10-09T06:39:53.083972+0000","last_became_active":"2023-10-09T06:39:53.080325+0000","last_became_peered":"2023-10-09T06:39:53.080325+0000","last_unstale":"2023-10-09T06:39:53.083972+0000","last_undegraded":"2023-10-09T06:39:53.083972+0000","last_fullsized":"2023-10-09T06:39:53.083972+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:14:35.310477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.091828+0000","last_change":"2023-10-09T06:39:53.085906+0000","last_active":"2023-10-09T06:39:53.091828+0000","last_peered":"2023-10-09T06:39:53.091828+0000","last_clean":"2023-10-09T06:39:53.091828+0000","last_became_active":"2023-10-09T06:39:53.085659+0000","last_became_peered":"2023-10-09T06:39:53.085659+0000","last_unstale":"2023-10-09T06:39:53.091828+0000","last_undegraded":"2023-10-09T06:39:53.091828+0000","last_fullsized":"2023-10-09T06:39:53.091828+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:16:03.156409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.3c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.088003+0000","last_change":"2023-10-09T06:39:53.085913+0000","last_active":"2023-10-09T06:39:53.088003+0000","last_peered":"2023-10-09T06:39:53.088003+0000","last_clean":"2023-10-09T06:39:53.088003+0000","last_became_active":"2023-10-09T06:39:53.085631+0000","last_became_peered":"2023-10-09T06:39:53.085631+0000","last_unstale":"2023-10-09T06:39:53.088003+0000","last_undegraded":"2023-10-09T06:39:53.088003+0000","last_fullsized":"2023-10-09T06:39:53.088003+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:27:58.872044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.3b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.081904+0000","last_change":"2023-10-09T06:39:53.077555+0000","last_active":"2023-10-09T06:39:53.081904+0000","last_peered":"2023-10-09T06:39:53.081904+0000","last_clean":"2023-10-09T06:39:53.081904+0000","last_became_active":"2023-10-09T06:39:53.077294+0000","last_became_peered":"2023-10-09T06:39:53.077294+0000","last_unstale":"2023-10-09T06:39:53.081904+0000","last_undegraded":"2023-10-09T06:39:53.081904+0000","last_fullsized":"2023-10-09T06:39:53.081904+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:18:51.600579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3a","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.085929+0000","last_change":"2023-10-09T06:39:53.083376+0000","last_active":"2023-10-09T06:39:53.085929+0000","last_peered":"2023-10-09T06:39:53.085929+0000","last_clean":"2023-10-09T06:39:53.085929+0000","last_became_active":"2023-10-09T06:39:53.083112+0000","last_became_peered":"2023-10-09T06:39:53.083112+0000","last_unstale":"2023-10-09T06:39:53.085929+0000","last_undegraded":"2023-10-09T06:39:53.085929+0000","last_fullsized":"2023-10-09T06:39:53.085929+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:12:21.943169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.39","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.092240+0000","last_change":"2023-10-09T06:39:53.087854+0000","last_active":"2023-10-09T06:39:53.092240+0000","last_peered":"2023-10-09T06:39:53.092240+0000","last_clean":"2023-10-09T06:39:53.092240+0000","last_became_active":"2023-10-09T06:39:53.087576+0000","last_became_peered":"2023-10-09T06:39:53.087576+0000","last_unstale":"2023-10-09T06:39:53.092240+0000","last_undegraded":"2023-10-09T06:39:53.092240+0000","last_fullsized":"2023-10-09T06:39:53.092240+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:29:10.172886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.38","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.084701+0000","last_change":"2023-10-09T06:39:53.077764+0000","last_active":"2023-10-09T06:39:53.084701+0000","last_peered":"2023-10-09T06:39:53.084701+0000","last_clean":"2023-10-09T06:39:53.084701+0000","last_became_active":"2023-10-09T06:39:53.077514+0000","last_became_peered":"2023-10-09T06:39:53.077514+0000","last_unstale":"2023-10-09T06:39:53.084701+0000","last_undegraded":"2023-10-09T06:39:53.084701+0000","last_fullsized":"2023-10-09T06:39:53.084701+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:02:38.457232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.37","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.089697+0000","last_change":"2023-10-09T06:39:53.087859+0000","last_active":"2023-10-09T06:39:53.089697+0000","last_peered":"2023-10-09T06:39:53.089697+0000","last_clean":"2023-10-09T06:39:53.089697+0000","last_became_active":"2023-10-09T06:39:53.087272+0000","last_became_peered":"2023-10-09T06:39:53.087272+0000","last_unstale":"2023-10-09T06:39:53.089697+0000","last_undegraded":"2023-10-09T06:39:53.089697+0000","last_fullsized":"2023-10-09T06:39:53.089697+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:02:11.161904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.36","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.091854+0000","last_change":"2023-10-09T06:39:53.086702+0000","last_active":"2023-10-09T06:39:53.091854+0000","last_peered":"2023-10-09T06:39:53.091854+0000","last_clean":"2023-10-09T06:39:53.091854+0000","last_became_active":"2023-10-09T06:39:53.086381+0000","last_became_peered":"2023-10-09T06:39:53.086381+0000","last_unstale":"2023-10-09T06:39:53.091854+0000","last_undegraded":"2023-10-09T06:39:53.091854+0000","last_fullsized":"2023-10-09T06:39:53.091854+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:30:34.855906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.35","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.082750+0000","last_change":"2023-10-09T06:39:53.081502+0000","last_active":"2023-10-09T06:39:53.082750+0000","last_peered":"2023-10-09T06:39:53.082750+0000","last_clean":"2023-10-09T06:39:53.082750+0000","last_became_active":"2023-10-09T06:39:53.080997+0000","last_became_peered":"2023-10-09T06:39:53.080997+0000","last_unstale":"2023-10-09T06:39:53.082750+0000","last_undegraded":"2023-10-09T06:39:53.082750+0000","last_fullsized":"2023-10-09T06:39:53.082750+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:49:45.615239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.34","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.092819+0000","last_change":"2023-10-09T06:39:53.090829+0000","last_active":"2023-10-09T06:39:53.092819+0000","last_peered":"2023-10-09T06:39:53.092819+0000","last_clean":"2023-10-09T06:39:53.092819+0000","last_became_active":"2023-10-09T06:39:53.090294+0000","last_became_peered":"2023-10-09T06:39:53.090294+0000","last_unstale":"2023-10-09T06:39:53.092819+0000","last_undegraded":"2023-10-09T06:39:53.092819+0000","last_fullsized":"2023-10-09T06:39:53.092819+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:39:27.333849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.33","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.087111+0000","last_change":"2023-10-09T06:39:53.084305+0000","last_active":"2023-10-09T06:39:53.087111+0000","last_peered":"2023-10-09T06:39:53.087111+0000","last_clean":"2023-10-09T06:39:53.087111+0000","last_became_active":"2023-10-09T06:39:53.084022+0000","last_became_peered":"2023-10-09T06:39:53.084022+0000","last_unstale":"2023-10-09T06:39:53.087111+0000","last_undegraded":"2023-10-09T06:39:53.087111+0000","last_fullsized":"2023-10-09T06:39:53.087111+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:02:19.682732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.32","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.083947+0000","last_change":"2023-10-09T06:39:53.079249+0000","last_active":"2023-10-09T06:39:53.083947+0000","last_peered":"2023-10-09T06:39:53.083947+0000","last_clean":"2023-10-09T06:39:53.083947+0000","last_became_active":"2023-10-09T06:39:53.078043+0000","last_became_peered":"2023-10-09T06:39:53.078043+0000","last_unstale":"2023-10-09T06:39:53.083947+0000","last_undegraded":"2023-10-09T06:39:53.083947+0000","last_fullsized":"2023-10-09T06:39:53.083947+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:58:55.339669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.31","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.087170+0000","last_change":"2023-10-09T06:39:53.083004+0000","last_active":"2023-10-09T06:39:53.087170+0000","last_peered":"2023-10-09T06:39:53.087170+0000","last_clean":"2023-10-09T06:39:53.087170+0000","last_became_active":"2023-10-09T06:39:53.082727+0000","last_became_peered":"2023-10-09T06:39:53.082727+0000","last_unstale":"2023-10-09T06:39:53.087170+0000","last_undegraded":"2023-10-09T06:39:53.087170+0000","last_fullsized":"2023-10-09T06:39:53.087170+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:18:51.600579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.30","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.089491+0000","last_change":"2023-10-09T06:39:53.082325+0000","last_active":"2023-10-09T06:39:53.089491+0000","last_peered":"2023-10-09T06:39:53.089491+0000","last_clean":"2023-10-09T06:39:53.089491+0000","last_became_active":"2023-10-09T06:39:53.082098+0000","last_became_peered":"2023-10-09T06:39:53.082098+0000","last_unstale":"2023-10-09T06:39:53.089491+0000","last_undegraded":"2023-10-09T06:39:53.089491+0000","last_fullsized":"2023-10-09T06:39:53.089491+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:48:58.176885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.089434+0000","last_change":"2023-10-09T06:39:53.080816+0000","last_active":"2023-10-09T06:39:53.089434+0000","last_peered":"2023-10-09T06:39:53.089434+0000","last_clean":"2023-10-09T06:39:53.089434+0000","last_became_active":"2023-10-09T06:39:53.080465+0000","last_became_peered":"2023-10-09T06:39:53.080465+0000","last_unstale":"2023-10-09T06:39:53.089434+0000","last_undegraded":"2023-10-09T06:39:53.089434+0000","last_fullsized":"2023-10-09T06:39:53.089434+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:27:34.543589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.083613+0000","last_change":"2023-10-09T06:39:53.080867+0000","last_active":"2023-10-09T06:39:53.083613+0000","last_peered":"2023-10-09T06:39:53.083613+0000","last_clean":"2023-10-09T06:39:53.083613+0000","last_became_active":"2023-10-09T06:39:53.080615+0000","last_became_peered":"2023-10-09T06:39:53.080615+0000","last_unstale":"2023-10-09T06:39:53.083613+0000","last_undegraded":"2023-10-09T06:39:53.083613+0000","last_fullsized":"2023-10-09T06:39:53.083613+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:21:17.222488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.089450+0000","last_change":"2023-10-09T06:39:53.080919+0000","last_active":"2023-10-09T06:39:53.089450+0000","last_peered":"2023-10-09T06:39:53.089450+0000","last_clean":"2023-10-09T06:39:53.089450+0000","last_became_active":"2023-10-09T06:39:53.080650+0000","last_became_peered":"2023-10-09T06:39:53.080650+0000","last_unstale":"2023-10-09T06:39:53.089450+0000","last_undegraded":"2023-10-09T06:39:53.089450+0000","last_fullsized":"2023-10-09T06:39:53.089450+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:30:11.170346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.092797+0000","last_change":"2023-10-09T06:39:53.090488+0000","last_active":"2023-10-09T06:39:53.092797+0000","last_peered":"2023-10-09T06:39:53.092797+0000","last_clean":"2023-10-09T06:39:53.092797+0000","last_became_active":"2023-10-09T06:39:53.089700+0000","last_became_peered":"2023-10-09T06:39:53.089700+0000","last_unstale":"2023-10-09T06:39:53.092797+0000","last_undegraded":"2023-10-09T06:39:53.092797+0000","last_fullsized":"2023-10-09T06:39:53.092797+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:08:01.872171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.2b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.081871+0000","last_change":"2023-10-09T06:39:53.077357+0000","last_active":"2023-10-09T06:39:53.081871+0000","last_peered":"2023-10-09T06:39:53.081871+0000","last_clean":"2023-10-09T06:39:53.081871+0000","last_became_active":"2023-10-09T06:39:53.077094+0000","last_became_peered":"2023-10-09T06:39:53.077094+0000","last_unstale":"2023-10-09T06:39:53.081871+0000","last_undegraded":"2023-10-09T06:39:53.081871+0000","last_fullsized":"2023-10-09T06:39:53.081871+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:08:59.720890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.2a","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.089158+0000","last_change":"2023-10-09T06:39:53.085482+0000","last_active":"2023-10-09T06:39:53.089158+0000","last_peered":"2023-10-09T06:39:53.089158+0000","last_clean":"2023-10-09T06:39:53.089158+0000","last_became_active":"2023-10-09T06:39:53.084983+0000","last_became_peered":"2023-10-09T06:39:53.084983+0000","last_unstale":"2023-10-09T06:39:53.089158+0000","last_undegraded":"2023-10-09T06:39:53.089158+0000","last_fullsized":"2023-10-09T06:39:53.089158+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:51:23.445071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.29","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.083823+0000","last_change":"2023-10-09T06:39:53.080735+0000","last_active":"2023-10-09T06:39:53.083823+0000","last_peered":"2023-10-09T06:39:53.083823+0000","last_clean":"2023-10-09T06:39:53.083823+0000","last_became_active":"2023-10-09T06:39:53.080452+0000","last_became_peered":"2023-10-09T06:39:53.080452+0000","last_unstale":"2023-10-09T06:39:53.083823+0000","last_undegraded":"2023-10-09T06:39:53.083823+0000","last_fullsized":"2023-10-09T06:39:53.083823+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:35:42.393159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.28","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.082815+0000","last_change":"2023-10-09T06:39:53.081539+0000","last_active":"2023-10-09T06:39:53.082815+0000","last_peered":"2023-10-09T06:39:53.082815+0000","last_clean":"2023-10-09T06:39:53.082815+0000","last_became_active":"2023-10-09T06:39:53.081258+0000","last_became_peered":"2023-10-09T06:39:53.081258+0000","last_unstale":"2023-10-09T06:39:53.082815+0000","last_undegraded":"2023-10-09T06:39:53.082815+0000","last_fullsized":"2023-10-09T06:39:53.082815+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:56:08.251031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.27","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.089126+0000","last_change":"2023-10-09T06:39:53.084208+0000","last_active":"2023-10-09T06:39:53.089126+0000","last_peered":"2023-10-09T06:39:53.089126+0000","last_clean":"2023-10-09T06:39:53.089126+0000","last_became_active":"2023-10-09T06:39:53.083953+0000","last_became_peered":"2023-10-09T06:39:53.083953+0000","last_unstale":"2023-10-09T06:39:53.089126+0000","last_undegraded":"2023-10-09T06:39:53.089126+0000","last_fullsized":"2023-10-09T06:39:53.089126+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:50:05.704255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.26","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.087203+0000","last_change":"2023-10-09T06:39:53.083801+0000","last_active":"2023-10-09T06:39:53.087203+0000","last_peered":"2023-10-09T06:39:53.087203+0000","last_clean":"2023-10-09T06:39:53.087203+0000","last_became_active":"2023-10-09T06:39:53.083472+0000","last_became_peered":"2023-10-09T06:39:53.083472+0000","last_unstale":"2023-10-09T06:39:53.087203+0000","last_undegraded":"2023-10-09T06:39:53.087203+0000","last_fullsized":"2023-10-09T06:39:53.087203+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:21:01.957723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.25","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.091324+0000","last_change":"2023-10-09T06:39:53.085009+0000","last_active":"2023-10-09T06:39:53.091324+0000","last_peered":"2023-10-09T06:39:53.091324+0000","last_clean":"2023-10-09T06:39:53.091324+0000","last_became_active":"2023-10-09T06:39:53.084715+0000","last_became_peered":"2023-10-09T06:39:53.084715+0000","last_unstale":"2023-10-09T06:39:53.091324+0000","last_undegraded":"2023-10-09T06:39:53.091324+0000","last_fullsized":"2023-10-09T06:39:53.091324+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:30:34.855906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.24","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.089200+0000","last_change":"2023-10-09T06:39:53.085818+0000","last_active":"2023-10-09T06:39:53.089200+0000","last_peered":"2023-10-09T06:39:53.089200+0000","last_clean":"2023-10-09T06:39:53.089200+0000","last_became_active":"2023-10-09T06:39:53.084411+0000","last_became_peered":"2023-10-09T06:39:53.084411+0000","last_unstale":"2023-10-09T06:39:53.089200+0000","last_undegraded":"2023-10-09T06:39:53.089200+0000","last_fullsized":"2023-10-09T06:39:53.089200+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:30:11.170346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.22","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.087983+0000","last_change":"2023-10-09T06:39:53.084816+0000","last_active":"2023-10-09T06:39:53.087983+0000","last_peered":"2023-10-09T06:39:53.087983+0000","last_clean":"2023-10-09T06:39:53.087983+0000","last_became_active":"2023-10-09T06:39:53.084535+0000","last_became_peered":"2023-10-09T06:39:53.084535+0000","last_unstale":"2023-10-09T06:39:53.087983+0000","last_undegraded":"2023-10-09T06:39:53.087983+0000","last_fullsized":"2023-10-09T06:39:53.087983+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:30:48.220988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.21","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.083291+0000","last_change":"2023-10-09T06:39:53.078286+0000","last_active":"2023-10-09T06:39:53.083291+0000","last_peered":"2023-10-09T06:39:53.083291+0000","last_clean":"2023-10-09T06:39:53.083291+0000","last_became_active":"2023-10-09T06:39:53.077850+0000","last_became_peered":"2023-10-09T06:39:53.077850+0000","last_unstale":"2023-10-09T06:39:53.083291+0000","last_undegraded":"2023-10-09T06:39:53.083291+0000","last_fullsized":"2023-10-09T06:39:53.083291+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:31:52.134120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.20","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.083507+0000","last_change":"2023-10-09T06:39:53.080068+0000","last_active":"2023-10-09T06:39:53.083507+0000","last_peered":"2023-10-09T06:39:53.083507+0000","last_clean":"2023-10-09T06:39:53.083507+0000","last_became_active":"2023-10-09T06:39:53.079104+0000","last_became_peered":"2023-10-09T06:39:53.079104+0000","last_unstale":"2023-10-09T06:39:53.083507+0000","last_undegraded":"2023-10-09T06:39:53.083507+0000","last_fullsized":"2023-10-09T06:39:53.083507+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:16:10.435246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.1c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.091980+0000","last_change":"2023-10-09T06:39:53.087891+0000","last_active":"2023-10-09T06:39:53.091980+0000","last_peered":"2023-10-09T06:39:53.091980+0000","last_clean":"2023-10-09T06:39:53.091980+0000","last_became_active":"2023-10-09T06:39:53.087616+0000","last_became_peered":"2023-10-09T06:39:53.087616+0000","last_unstale":"2023-10-09T06:39:53.091980+0000","last_undegraded":"2023-10-09T06:39:53.091980+0000","last_fullsized":"2023-10-09T06:39:53.091980+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:02:42.734000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1f","version":"196'4","reported_seq":14,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.098093+0000","last_change":"2023-10-09T06:39:52.081149+0000","last_active":"2023-10-09T06:39:53.098093+0000","last_peered":"2023-10-09T06:39:53.098093+0000","last_clean":"2023-10-09T06:39:53.098093+0000","last_became_active":"2023-10-09T06:39:52.080868+0000","last_became_peered":"2023-10-09T06:39:52.080868+0000","last_unstale":"2023-10-09T06:39:53.098093+0000","last_undegraded":"2023-10-09T06:39:53.098093+0000","last_fullsized":"2023-10-09T06:39:53.098093+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:14:56.789141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.083080+0000","last_change":"2023-10-09T06:39:53.080497+0000","last_active":"2023-10-09T06:39:53.083080+0000","last_peered":"2023-10-09T06:39:53.083080+0000","last_clean":"2023-10-09T06:39:53.083080+0000","last_became_active":"2023-10-09T06:39:53.080249+0000","last_became_peered":"2023-10-09T06:39:53.080249+0000","last_unstale":"2023-10-09T06:39:53.083080+0000","last_undegraded":"2023-10-09T06:39:53.083080+0000","last_fullsized":"2023-10-09T06:39:53.083080+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:22:03.142568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"196'1","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.096566+0000","last_change":"2023-10-09T06:39:52.081470+0000","last_active":"2023-10-09T06:39:53.096566+0000","last_peered":"2023-10-09T06:39:53.096566+0000","last_clean":"2023-10-09T06:39:53.096566+0000","last_became_active":"2023-10-09T06:39:52.081170+0000","last_became_peered":"2023-10-09T06:39:52.081170+0000","last_unstale":"2023-10-09T06:39:53.096566+0000","last_undegraded":"2023-10-09T06:39:53.096566+0000","last_fullsized":"2023-10-09T06:39:53.096566+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:34:37.947853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.1e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.083585+0000","last_change":"2023-10-09T06:39:53.078610+0000","last_active":"2023-10-09T06:39:53.083585+0000","last_peered":"2023-10-09T06:39:53.083585+0000","last_clean":"2023-10-09T06:39:53.083585+0000","last_became_active":"2023-10-09T06:39:53.078292+0000","last_became_peered":"2023-10-09T06:39:53.078292+0000","last_unstale":"2023-10-09T06:39:53.083585+0000","last_undegraded":"2023-10-09T06:39:53.083585+0000","last_fullsized":"2023-10-09T06:39:53.083585+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:01:10.092288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1d","version":"196'2","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.098126+0000","last_change":"2023-10-09T06:39:52.083228+0000","last_active":"2023-10-09T06:39:53.098126+0000","last_peered":"2023-10-09T06:39:53.098126+0000","last_clean":"2023-10-09T06:39:53.098126+0000","last_became_active":"2023-10-09T06:39:52.082704+0000","last_became_peered":"2023-10-09T06:39:52.082704+0000","last_unstale":"2023-10-09T06:39:53.098126+0000","last_undegraded":"2023-10-09T06:39:53.098126+0000","last_fullsized":"2023-10-09T06:39:53.098126+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:55:53.294887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.1f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.088667+0000","last_change":"2023-10-09T06:39:53.086019+0000","last_active":"2023-10-09T06:39:53.088667+0000","last_peered":"2023-10-09T06:39:53.088667+0000","last_clean":"2023-10-09T06:39:53.088667+0000","last_became_active":"2023-10-09T06:39:53.085692+0000","last_became_peered":"2023-10-09T06:39:53.085692+0000","last_unstale":"2023-10-09T06:39:53.088667+0000","last_undegraded":"2023-10-09T06:39:53.088667+0000","last_fullsized":"2023-10-09T06:39:53.088667+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:46:36.690844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"196'2","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.099256+0000","last_change":"2023-10-09T06:39:52.078914+0000","last_active":"2023-10-09T06:39:53.099256+0000","last_peered":"2023-10-09T06:39:53.099256+0000","last_clean":"2023-10-09T06:39:53.099256+0000","last_became_active":"2023-10-09T06:39:52.078619+0000","last_became_peered":"2023-10-09T06:39:52.078619+0000","last_unstale":"2023-10-09T06:39:53.099256+0000","last_undegraded":"2023-10-09T06:39:53.099256+0000","last_fullsized":"2023-10-09T06:39:53.099256+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:17:04.704861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.086176+0000","last_change":"2023-10-09T06:39:53.082694+0000","last_active":"2023-10-09T06:39:53.086176+0000","last_peered":"2023-10-09T06:39:53.086176+0000","last_clean":"2023-10-09T06:39:53.086176+0000","last_became_active":"2023-10-09T06:39:53.082415+0000","last_became_peered":"2023-10-09T06:39:53.082415+0000","last_unstale":"2023-10-09T06:39:53.086176+0000","last_undegraded":"2023-10-09T06:39:53.086176+0000","last_fullsized":"2023-10-09T06:39:53.086176+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:36:18.420616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1b","version":"196'1","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.098688+0000","last_change":"2023-10-09T06:39:52.076258+0000","last_active":"2023-10-09T06:39:53.098688+0000","last_peered":"2023-10-09T06:39:53.098688+0000","last_clean":"2023-10-09T06:39:53.098688+0000","last_became_active":"2023-10-09T06:39:52.075552+0000","last_became_peered":"2023-10-09T06:39:52.075552+0000","last_unstale":"2023-10-09T06:39:53.098688+0000","last_undegraded":"2023-10-09T06:39:53.098688+0000","last_fullsized":"2023-10-09T06:39:53.098688+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:21:00.953510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.083321+0000","last_change":"2023-10-09T06:39:53.078827+0000","last_active":"2023-10-09T06:39:53.083321+0000","last_peered":"2023-10-09T06:39:53.083321+0000","last_clean":"2023-10-09T06:39:53.083321+0000","last_became_active":"2023-10-09T06:39:53.078562+0000","last_became_peered":"2023-10-09T06:39:53.078562+0000","last_unstale":"2023-10-09T06:39:53.083321+0000","last_undegraded":"2023-10-09T06:39:53.083321+0000","last_fullsized":"2023-10-09T06:39:53.083321+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:24:17.586832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"196'1","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.098216+0000","last_change":"2023-10-09T06:39:52.082928+0000","last_active":"2023-10-09T06:39:53.098216+0000","last_peered":"2023-10-09T06:39:53.098216+0000","last_clean":"2023-10-09T06:39:53.098216+0000","last_became_active":"2023-10-09T06:39:52.082707+0000","last_became_peered":"2023-10-09T06:39:52.082707+0000","last_unstale":"2023-10-09T06:39:53.098216+0000","last_undegraded":"2023-10-09T06:39:53.098216+0000","last_fullsized":"2023-10-09T06:39:53.098216+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:39:26.329636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.1a","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.092279+0000","last_change":"2023-10-09T06:39:53.088119+0000","last_active":"2023-10-09T06:39:53.092279+0000","last_peered":"2023-10-09T06:39:53.092279+0000","last_clean":"2023-10-09T06:39:53.092279+0000","last_became_active":"2023-10-09T06:39:53.087858+0000","last_became_peered":"2023-10-09T06:39:53.087858+0000","last_unstale":"2023-10-09T06:39:53.092279+0000","last_undegraded":"2023-10-09T06:39:53.092279+0000","last_fullsized":"2023-10-09T06:39:53.092279+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:16:55.231556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.075316+0000","last_change":"2023-10-09T06:39:52.075825+0000","last_active":"2023-10-09T06:39:53.075316+0000","last_peered":"2023-10-09T06:39:53.075316+0000","last_clean":"2023-10-09T06:39:53.075316+0000","last_became_active":"2023-10-09T06:39:52.075568+0000","last_became_peered":"2023-10-09T06:39:52.075568+0000","last_unstale":"2023-10-09T06:39:53.075316+0000","last_undegraded":"2023-10-09T06:39:53.075316+0000","last_fullsized":"2023-10-09T06:39:53.075316+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:42:30.773943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.1b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.084398+0000","last_change":"2023-10-09T06:39:53.077739+0000","last_active":"2023-10-09T06:39:53.084398+0000","last_peered":"2023-10-09T06:39:53.084398+0000","last_clean":"2023-10-09T06:39:53.084398+0000","last_became_active":"2023-10-09T06:39:53.077483+0000","last_became_peered":"2023-10-09T06:39:53.077483+0000","last_unstale":"2023-10-09T06:39:53.084398+0000","last_undegraded":"2023-10-09T06:39:53.084398+0000","last_fullsized":"2023-10-09T06:39:53.084398+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:17:04.774066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.072490+0000","last_change":"2023-10-09T06:39:52.079426+0000","last_active":"2023-10-09T06:39:53.072490+0000","last_peered":"2023-10-09T06:39:53.072490+0000","last_clean":"2023-10-09T06:39:53.072490+0000","last_became_active":"2023-10-09T06:39:52.079159+0000","last_became_peered":"2023-10-09T06:39:52.079159+0000","last_unstale":"2023-10-09T06:39:53.072490+0000","last_undegraded":"2023-10-09T06:39:53.072490+0000","last_fullsized":"2023-10-09T06:39:53.072490+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:27:11.297409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.14","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.088025+0000","last_change":"2023-10-09T06:39:53.086083+0000","last_active":"2023-10-09T06:39:53.088025+0000","last_peered":"2023-10-09T06:39:53.088025+0000","last_clean":"2023-10-09T06:39:53.088025+0000","last_became_active":"2023-10-09T06:39:53.085770+0000","last_became_peered":"2023-10-09T06:39:53.085770+0000","last_unstale":"2023-10-09T06:39:53.088025+0000","last_undegraded":"2023-10-09T06:39:53.088025+0000","last_fullsized":"2023-10-09T06:39:53.088025+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:53:10.470109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.068622+0000","last_change":"2023-10-09T06:39:52.081093+0000","last_active":"2023-10-09T06:39:53.068622+0000","last_peered":"2023-10-09T06:39:53.068622+0000","last_clean":"2023-10-09T06:39:53.068622+0000","last_became_active":"2023-10-09T06:39:52.080822+0000","last_became_peered":"2023-10-09T06:39:52.080822+0000","last_unstale":"2023-10-09T06:39:53.068622+0000","last_undegraded":"2023-10-09T06:39:53.068622+0000","last_fullsized":"2023-10-09T06:39:53.068622+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:46:33.850505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.15","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.086582+0000","last_change":"2023-10-09T06:39:53.082263+0000","last_active":"2023-10-09T06:39:53.086582+0000","last_peered":"2023-10-09T06:39:53.086582+0000","last_clean":"2023-10-09T06:39:53.086582+0000","last_became_active":"2023-10-09T06:39:53.081999+0000","last_became_peered":"2023-10-09T06:39:53.081999+0000","last_unstale":"2023-10-09T06:39:53.086582+0000","last_undegraded":"2023-10-09T06:39:53.086582+0000","last_fullsized":"2023-10-09T06:39:53.086582+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:25:00.261989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.075411+0000","last_change":"2023-10-09T06:39:52.076723+0000","last_active":"2023-10-09T06:39:53.075411+0000","last_peered":"2023-10-09T06:39:53.075411+0000","last_clean":"2023-10-09T06:39:53.075411+0000","last_became_active":"2023-10-09T06:39:52.076455+0000","last_became_peered":"2023-10-09T06:39:52.076455+0000","last_unstale":"2023-10-09T06:39:53.075411+0000","last_undegraded":"2023-10-09T06:39:53.075411+0000","last_fullsized":"2023-10-09T06:39:53.075411+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T06:58:29.869457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.088436+0000","last_change":"2023-10-09T06:39:53.084322+0000","last_active":"2023-10-09T06:39:53.088436+0000","last_peered":"2023-10-09T06:39:53.088436+0000","last_clean":"2023-10-09T06:39:53.088436+0000","last_became_active":"2023-10-09T06:39:53.084117+0000","last_became_peered":"2023-10-09T06:39:53.084117+0000","last_unstale":"2023-10-09T06:39:53.088436+0000","last_undegraded":"2023-10-09T06:39:53.088436+0000","last_fullsized":"2023-10-09T06:39:53.088436+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:52:10.706737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.15","version":"196'3","reported_seq":13,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.099243+0000","last_change":"2023-10-09T06:39:52.078205+0000","last_active":"2023-10-09T06:39:53.099243+0000","last_peered":"2023-10-09T06:39:53.099243+0000","last_clean":"2023-10-09T06:39:53.099243+0000","last_became_active":"2023-10-09T06:39:52.077945+0000","last_became_peered":"2023-10-09T06:39:52.077945+0000","last_unstale":"2023-10-09T06:39:53.099243+0000","last_undegraded":"2023-10-09T06:39:53.099243+0000","last_fullsized":"2023-10-09T06:39:53.099243+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:24:16.582618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.089064+0000","last_change":"2023-10-09T06:39:53.084391+0000","last_active":"2023-10-09T06:39:53.089064+0000","last_peered":"2023-10-09T06:39:53.089064+0000","last_clean":"2023-10-09T06:39:53.089064+0000","last_became_active":"2023-10-09T06:39:53.084044+0000","last_became_peered":"2023-10-09T06:39:53.084044+0000","last_unstale":"2023-10-09T06:39:53.089064+0000","last_undegraded":"2023-10-09T06:39:53.089064+0000","last_fullsized":"2023-10-09T06:39:53.089064+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:29:46.222421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"196'6","reported_seq":16,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.102794+0000","last_change":"2023-10-09T06:39:52.074833+0000","last_active":"2023-10-09T06:39:53.102794+0000","last_peered":"2023-10-09T06:39:53.102794+0000","last_clean":"2023-10-09T06:39:53.102794+0000","last_became_active":"2023-10-09T06:39:52.074588+0000","last_became_peered":"2023-10-09T06:39:52.074588+0000","last_unstale":"2023-10-09T06:39:53.102794+0000","last_undegraded":"2023-10-09T06:39:53.102794+0000","last_fullsized":"2023-10-09T06:39:53.102794+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:19:27.971639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.10","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.085978+0000","last_change":"2023-10-09T06:39:53.081740+0000","last_active":"2023-10-09T06:39:53.085978+0000","last_peered":"2023-10-09T06:39:53.085978+0000","last_clean":"2023-10-09T06:39:53.085978+0000","last_became_active":"2023-10-09T06:39:53.081489+0000","last_became_peered":"2023-10-09T06:39:53.081489+0000","last_unstale":"2023-10-09T06:39:53.085978+0000","last_undegraded":"2023-10-09T06:39:53.085978+0000","last_fullsized":"2023-10-09T06:39:53.085978+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:58:55.339669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"196'2","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.098774+0000","last_change":"2023-10-09T06:39:52.078081+0000","last_active":"2023-10-09T06:39:53.098774+0000","last_peered":"2023-10-09T06:39:53.098774+0000","last_clean":"2023-10-09T06:39:53.098774+0000","last_became_active":"2023-10-09T06:39:52.077809+0000","last_became_peered":"2023-10-09T06:39:52.077809+0000","last_unstale":"2023-10-09T06:39:53.098774+0000","last_undegraded":"2023-10-09T06:39:53.098774+0000","last_fullsized":"2023-10-09T06:39:53.098774+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:12:50.712857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.091329+0000","last_change":"2023-10-09T06:39:53.085476+0000","last_active":"2023-10-09T06:39:53.091329+0000","last_peered":"2023-10-09T06:39:53.091329+0000","last_clean":"2023-10-09T06:39:53.091329+0000","last_became_active":"2023-10-09T06:39:53.084884+0000","last_became_peered":"2023-10-09T06:39:53.084884+0000","last_unstale":"2023-10-09T06:39:53.091329+0000","last_undegraded":"2023-10-09T06:39:53.091329+0000","last_fullsized":"2023-10-09T06:39:53.091329+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:52:02.602342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.068418+0000","last_change":"2023-10-09T06:39:52.079034+0000","last_active":"2023-10-09T06:39:53.068418+0000","last_peered":"2023-10-09T06:39:53.068418+0000","last_clean":"2023-10-09T06:39:53.068418+0000","last_became_active":"2023-10-09T06:39:52.078754+0000","last_became_peered":"2023-10-09T06:39:52.078754+0000","last_unstale":"2023-10-09T06:39:53.068418+0000","last_undegraded":"2023-10-09T06:39:53.068418+0000","last_fullsized":"2023-10-09T06:39:53.068418+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T18:16:09.431033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.082996+0000","last_change":"2023-10-09T06:39:53.079455+0000","last_active":"2023-10-09T06:39:53.082996+0000","last_peered":"2023-10-09T06:39:53.082996+0000","last_clean":"2023-10-09T06:39:53.082996+0000","last_became_active":"2023-10-09T06:39:53.079217+0000","last_became_peered":"2023-10-09T06:39:53.079217+0000","last_unstale":"2023-10-09T06:39:53.082996+0000","last_undegraded":"2023-10-09T06:39:53.082996+0000","last_fullsized":"2023-10-09T06:39:53.082996+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:31:52.134120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.068259+0000","last_change":"2023-10-09T06:39:52.081786+0000","last_active":"2023-10-09T06:39:53.068259+0000","last_peered":"2023-10-09T06:39:53.068259+0000","last_clean":"2023-10-09T06:39:53.068259+0000","last_became_active":"2023-10-09T06:39:52.081540+0000","last_became_peered":"2023-10-09T06:39:52.081540+0000","last_unstale":"2023-10-09T06:39:53.068259+0000","last_undegraded":"2023-10-09T06:39:53.068259+0000","last_fullsized":"2023-10-09T06:39:53.068259+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:46:35.686631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.13","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.082968+0000","last_change":"2023-10-09T06:39:53.079904+0000","last_active":"2023-10-09T06:39:53.082968+0000","last_peered":"2023-10-09T06:39:53.082968+0000","last_clean":"2023-10-09T06:39:53.082968+0000","last_became_active":"2023-10-09T06:39:53.079644+0000","last_became_peered":"2023-10-09T06:39:53.079644+0000","last_unstale":"2023-10-09T06:39:53.082968+0000","last_undegraded":"2023-10-09T06:39:53.082968+0000","last_fullsized":"2023-10-09T06:39:53.082968+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:46:36.690844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.10","version":"196'8","reported_seq":18,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.100671+0000","last_change":"2023-10-09T06:39:52.078042+0000","last_active":"2023-10-09T06:39:53.100671+0000","last_peered":"2023-10-09T06:39:53.100671+0000","last_clean":"2023-10-09T06:39:53.100671+0000","last_became_active":"2023-10-09T06:39:52.077784+0000","last_became_peered":"2023-10-09T06:39:52.077784+0000","last_unstale":"2023-10-09T06:39:53.100671+0000","last_undegraded":"2023-10-09T06:39:53.100671+0000","last_fullsized":"2023-10-09T06:39:53.100671+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:31:51.129907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.088793+0000","last_change":"2023-10-09T06:39:53.084069+0000","last_active":"2023-10-09T06:39:53.088793+0000","last_peered":"2023-10-09T06:39:53.088793+0000","last_clean":"2023-10-09T06:39:53.088793+0000","last_became_active":"2023-10-09T06:39:53.083711+0000","last_became_peered":"2023-10-09T06:39:53.083711+0000","last_unstale":"2023-10-09T06:39:53.088793+0000","last_undegraded":"2023-10-09T06:39:53.088793+0000","last_fullsized":"2023-10-09T06:39:53.088793+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:41:09.798707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"196'1","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.100195+0000","last_change":"2023-10-09T06:39:52.083381+0000","last_active":"2023-10-09T06:39:53.100195+0000","last_peered":"2023-10-09T06:39:53.100195+0000","last_clean":"2023-10-09T06:39:53.100195+0000","last_became_active":"2023-10-09T06:39:52.083088+0000","last_became_peered":"2023-10-09T06:39:52.083088+0000","last_unstale":"2023-10-09T06:39:53.100195+0000","last_undegraded":"2023-10-09T06:39:53.100195+0000","last_fullsized":"2023-10-09T06:39:53.100195+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:30:10.166133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.088398+0000","last_change":"2023-10-09T06:39:53.083385+0000","last_active":"2023-10-09T06:39:53.088398+0000","last_peered":"2023-10-09T06:39:53.088398+0000","last_clean":"2023-10-09T06:39:53.088398+0000","last_became_active":"2023-10-09T06:39:53.083122+0000","last_became_peered":"2023-10-09T06:39:53.083122+0000","last_unstale":"2023-10-09T06:39:53.088398+0000","last_undegraded":"2023-10-09T06:39:53.088398+0000","last_fullsized":"2023-10-09T06:39:53.088398+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T12:04:13.567267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.e","version":"196'2","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.095093+0000","last_change":"2023-10-09T06:39:52.081771+0000","last_active":"2023-10-09T06:39:53.095093+0000","last_peered":"2023-10-09T06:39:53.095093+0000","last_clean":"2023-10-09T06:39:53.095093+0000","last_became_active":"2023-10-09T06:39:52.081496+0000","last_became_peered":"2023-10-09T06:39:52.081496+0000","last_unstale":"2023-10-09T06:39:53.095093+0000","last_undegraded":"2023-10-09T06:39:53.095093+0000","last_fullsized":"2023-10-09T06:39:53.095093+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:10:17.630559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.084736+0000","last_change":"2023-10-09T06:39:53.078433+0000","last_active":"2023-10-09T06:39:53.084736+0000","last_peered":"2023-10-09T06:39:53.084736+0000","last_clean":"2023-10-09T06:39:53.084736+0000","last_became_active":"2023-10-09T06:39:53.078191+0000","last_became_peered":"2023-10-09T06:39:53.078191+0000","last_unstale":"2023-10-09T06:39:53.084736+0000","last_undegraded":"2023-10-09T06:39:53.084736+0000","last_fullsized":"2023-10-09T06:39:53.084736+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T15:23:39.439786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.d","version":"196'1","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.099638+0000","last_change":"2023-10-09T06:39:52.087273+0000","last_active":"2023-10-09T06:39:53.099638+0000","last_peered":"2023-10-09T06:39:53.099638+0000","last_clean":"2023-10-09T06:39:53.099638+0000","last_became_active":"2023-10-09T06:39:52.086958+0000","last_became_peered":"2023-10-09T06:39:52.086958+0000","last_unstale":"2023-10-09T06:39:53.099638+0000","last_undegraded":"2023-10-09T06:39:53.099638+0000","last_fullsized":"2023-10-09T06:39:53.099638+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:01:09.088075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.089252+0000","last_change":"2023-10-09T06:39:53.086046+0000","last_active":"2023-10-09T06:39:53.089252+0000","last_peered":"2023-10-09T06:39:53.089252+0000","last_clean":"2023-10-09T06:39:53.089252+0000","last_became_active":"2023-10-09T06:39:53.084655+0000","last_became_peered":"2023-10-09T06:39:53.084655+0000","last_unstale":"2023-10-09T06:39:53.089252+0000","last_undegraded":"2023-10-09T06:39:53.089252+0000","last_fullsized":"2023-10-09T06:39:53.089252+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:53:31.682242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.071791+0000","last_change":"2023-10-09T06:39:52.081850+0000","last_active":"2023-10-09T06:39:53.071791+0000","last_peered":"2023-10-09T06:39:53.071791+0000","last_clean":"2023-10-09T06:39:53.071791+0000","last_became_active":"2023-10-09T06:39:52.081576+0000","last_became_peered":"2023-10-09T06:39:52.081576+0000","last_unstale":"2023-10-09T06:39:53.071791+0000","last_undegraded":"2023-10-09T06:39:53.071791+0000","last_fullsized":"2023-10-09T06:39:53.071791+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:23:46.437369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.086156+0000","last_change":"2023-10-09T06:39:53.082189+0000","last_active":"2023-10-09T06:39:53.086156+0000","last_peered":"2023-10-09T06:39:53.086156+0000","last_clean":"2023-10-09T06:39:53.086156+0000","last_became_active":"2023-10-09T06:39:53.081956+0000","last_became_peered":"2023-10-09T06:39:53.081956+0000","last_unstale":"2023-10-09T06:39:53.086156+0000","last_undegraded":"2023-10-09T06:39:53.086156+0000","last_fullsized":"2023-10-09T06:39:53.086156+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:08:59.720890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.b","version":"196'2","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.098797+0000","last_change":"2023-10-09T06:39:52.078643+0000","last_active":"2023-10-09T06:39:53.098797+0000","last_peered":"2023-10-09T06:39:53.098797+0000","last_clean":"2023-10-09T06:39:53.098797+0000","last_became_active":"2023-10-09T06:39:52.078338+0000","last_became_peered":"2023-10-09T06:39:52.078338+0000","last_unstale":"2023-10-09T06:39:53.098797+0000","last_undegraded":"2023-10-09T06:39:53.098797+0000","last_fullsized":"2023-10-09T06:39:53.098797+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:36:17.416403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.091345+0000","last_change":"2023-10-09T06:39:53.085656+0000","last_active":"2023-10-09T06:39:53.091345+0000","last_peered":"2023-10-09T06:39:53.091345+0000","last_clean":"2023-10-09T06:39:53.091345+0000","last_became_active":"2023-10-09T06:39:53.085219+0000","last_became_peered":"2023-10-09T06:39:53.085219+0000","last_unstale":"2023-10-09T06:39:53.091345+0000","last_undegraded":"2023-10-09T06:39:53.091345+0000","last_fullsized":"2023-10-09T06:39:53.091345+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:42:31.778156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.075064+0000","last_change":"2023-10-09T06:39:52.079690+0000","last_active":"2023-10-09T06:39:53.075064+0000","last_peered":"2023-10-09T06:39:53.075064+0000","last_clean":"2023-10-09T06:39:53.075064+0000","last_became_active":"2023-10-09T06:39:52.079438+0000","last_became_peered":"2023-10-09T06:39:52.079438+0000","last_unstale":"2023-10-09T06:39:53.075064+0000","last_undegraded":"2023-10-09T06:39:53.075064+0000","last_fullsized":"2023-10-09T06:39:53.075064+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:35:41.388946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.085473+0000","last_change":"2023-10-09T06:39:53.082971+0000","last_active":"2023-10-09T06:39:53.085473+0000","last_peered":"2023-10-09T06:39:53.085473+0000","last_clean":"2023-10-09T06:39:53.085473+0000","last_became_active":"2023-10-09T06:39:53.082716+0000","last_became_peered":"2023-10-09T06:39:53.082716+0000","last_unstale":"2023-10-09T06:39:53.085473+0000","last_undegraded":"2023-10-09T06:39:53.085473+0000","last_fullsized":"2023-10-09T06:39:53.085473+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:49:58.700962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.072685+0000","last_change":"2023-10-09T06:39:52.081467+0000","last_active":"2023-10-09T06:39:53.072685+0000","last_peered":"2023-10-09T06:39:53.072685+0000","last_clean":"2023-10-09T06:39:53.072685+0000","last_became_active":"2023-10-09T06:39:52.081208+0000","last_became_peered":"2023-10-09T06:39:52.081208+0000","last_unstale":"2023-10-09T06:39:53.072685+0000","last_undegraded":"2023-10-09T06:39:53.072685+0000","last_fullsized":"2023-10-09T06:39:53.072685+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:30:47.216775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.083475+0000","last_change":"2023-10-09T06:39:53.079349+0000","last_active":"2023-10-09T06:39:53.083475+0000","last_peered":"2023-10-09T06:39:53.083475+0000","last_clean":"2023-10-09T06:39:53.083475+0000","last_became_active":"2023-10-09T06:39:53.078726+0000","last_became_peered":"2023-10-09T06:39:53.078726+0000","last_unstale":"2023-10-09T06:39:53.083475+0000","last_undegraded":"2023-10-09T06:39:53.083475+0000","last_fullsized":"2023-10-09T06:39:53.083475+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:35:29.957992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.068712+0000","last_change":"2023-10-09T06:39:52.076805+0000","last_active":"2023-10-09T06:39:53.068712+0000","last_peered":"2023-10-09T06:39:53.068712+0000","last_clean":"2023-10-09T06:39:53.068712+0000","last_became_active":"2023-10-09T06:39:52.076346+0000","last_became_peered":"2023-10-09T06:39:52.076346+0000","last_unstale":"2023-10-09T06:39:53.068712+0000","last_undegraded":"2023-10-09T06:39:53.068712+0000","last_fullsized":"2023-10-09T06:39:53.068712+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:14:34.306264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.092809+0000","last_change":"2023-10-09T06:39:53.090665+0000","last_active":"2023-10-09T06:39:53.092809+0000","last_peered":"2023-10-09T06:39:53.092809+0000","last_clean":"2023-10-09T06:39:53.092809+0000","last_became_active":"2023-10-09T06:39:53.090019+0000","last_became_peered":"2023-10-09T06:39:53.090019+0000","last_unstale":"2023-10-09T06:39:53.092809+0000","last_undegraded":"2023-10-09T06:39:53.092809+0000","last_fullsized":"2023-10-09T06:39:53.092809+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:09:42.222904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.7","version":"196'2","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.098935+0000","last_change":"2023-10-09T06:39:52.077134+0000","last_active":"2023-10-09T06:39:53.098935+0000","last_peered":"2023-10-09T06:39:53.098935+0000","last_clean":"2023-10-09T06:39:53.098935+0000","last_became_active":"2023-10-09T06:39:52.076120+0000","last_became_peered":"2023-10-09T06:39:52.076120+0000","last_unstale":"2023-10-09T06:39:53.098935+0000","last_undegraded":"2023-10-09T06:39:53.098935+0000","last_fullsized":"2023-10-09T06:39:53.098935+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T14:35:28.953779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.089521+0000","last_change":"2023-10-09T06:39:53.085769+0000","last_active":"2023-10-09T06:39:53.089521+0000","last_peered":"2023-10-09T06:39:53.089521+0000","last_clean":"2023-10-09T06:39:53.089521+0000","last_became_active":"2023-10-09T06:39:53.085483+0000","last_became_peered":"2023-10-09T06:39:53.085483+0000","last_unstale":"2023-10-09T06:39:53.089521+0000","last_undegraded":"2023-10-09T06:39:53.089521+0000","last_fullsized":"2023-10-09T06:39:53.089521+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:34:14.228003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.073429+0000","last_change":"2023-10-09T06:39:52.076909+0000","last_active":"2023-10-09T06:39:53.073429+0000","last_peered":"2023-10-09T06:39:53.073429+0000","last_clean":"2023-10-09T06:39:53.073429+0000","last_became_active":"2023-10-09T06:39:52.076643+0000","last_became_peered":"2023-10-09T06:39:52.076643+0000","last_unstale":"2023-10-09T06:39:53.073429+0000","last_undegraded":"2023-10-09T06:39:53.073429+0000","last_fullsized":"2023-10-09T06:39:53.073429+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:23:45.416793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.088757+0000","last_change":"2023-10-09T06:39:53.085178+0000","last_active":"2023-10-09T06:39:53.088757+0000","last_peered":"2023-10-09T06:39:53.088757+0000","last_clean":"2023-10-09T06:39:53.088757+0000","last_became_active":"2023-10-09T06:39:53.084894+0000","last_became_peered":"2023-10-09T06:39:53.084894+0000","last_unstale":"2023-10-09T06:39:53.088757+0000","last_undegraded":"2023-10-09T06:39:53.088757+0000","last_fullsized":"2023-10-09T06:39:53.088757+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:35:46.409202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.079054+0000","last_change":"2023-10-09T06:39:52.082785+0000","last_active":"2023-10-09T06:39:53.079054+0000","last_peered":"2023-10-09T06:39:53.079054+0000","last_clean":"2023-10-09T06:39:53.079054+0000","last_became_active":"2023-10-09T06:39:52.082532+0000","last_became_peered":"2023-10-09T06:39:52.082532+0000","last_unstale":"2023-10-09T06:39:53.079054+0000","last_undegraded":"2023-10-09T06:39:53.079054+0000","last_fullsized":"2023-10-09T06:39:53.079054+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:07:23.025013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.7","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.085225+0000","last_change":"2023-10-09T06:39:53.082346+0000","last_active":"2023-10-09T06:39:53.085225+0000","last_peered":"2023-10-09T06:39:53.085225+0000","last_clean":"2023-10-09T06:39:53.085225+0000","last_became_active":"2023-10-09T06:39:53.082050+0000","last_became_peered":"2023-10-09T06:39:53.082050+0000","last_unstale":"2023-10-09T06:39:53.085225+0000","last_undegraded":"2023-10-09T06:39:53.085225+0000","last_fullsized":"2023-10-09T06:39:53.085225+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:25:09.337293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.078532+0000","last_change":"2023-10-09T06:39:52.087119+0000","last_active":"2023-10-09T06:39:53.078532+0000","last_peered":"2023-10-09T06:39:53.078532+0000","last_clean":"2023-10-09T06:39:53.078532+0000","last_became_active":"2023-10-09T06:39:52.086876+0000","last_became_peered":"2023-10-09T06:39:52.086876+0000","last_unstale":"2023-10-09T06:39:53.078532+0000","last_undegraded":"2023-10-09T06:39:53.078532+0000","last_fullsized":"2023-10-09T06:39:53.078532+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T11:22:02.138355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.084681+0000","last_change":"2023-10-09T06:39:53.077077+0000","last_active":"2023-10-09T06:39:53.084681+0000","last_peered":"2023-10-09T06:39:53.084681+0000","last_clean":"2023-10-09T06:39:53.084681+0000","last_became_active":"2023-10-09T06:39:53.076811+0000","last_became_peered":"2023-10-09T06:39:53.076811+0000","last_unstale":"2023-10-09T06:39:53.084681+0000","last_undegraded":"2023-10-09T06:39:53.084681+0000","last_fullsized":"2023-10-09T06:39:53.084681+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T16:03:00.699198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.071606+0000","last_change":"2023-10-09T06:39:52.079543+0000","last_active":"2023-10-09T06:39:53.071606+0000","last_peered":"2023-10-09T06:39:53.071606+0000","last_clean":"2023-10-09T06:39:53.071606+0000","last_became_active":"2023-10-09T06:39:52.079279+0000","last_became_peered":"2023-10-09T06:39:52.079279+0000","last_unstale":"2023-10-09T06:39:53.071606+0000","last_undegraded":"2023-10-09T06:39:53.071606+0000","last_fullsized":"2023-10-09T06:39:53.071606+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:04:19.759255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.087193+0000","last_change":"2023-10-09T06:39:53.083876+0000","last_active":"2023-10-09T06:39:53.087193+0000","last_peered":"2023-10-09T06:39:53.087193+0000","last_clean":"2023-10-09T06:39:53.087193+0000","last_became_active":"2023-10-09T06:39:53.083594+0000","last_became_peered":"2023-10-09T06:39:53.083594+0000","last_unstale":"2023-10-09T06:39:53.087193+0000","last_undegraded":"2023-10-09T06:39:53.087193+0000","last_fullsized":"2023-10-09T06:39:53.087193+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:33:59.174559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.071582+0000","last_change":"2023-10-09T06:39:52.078285+0000","last_active":"2023-10-09T06:39:53.071582+0000","last_peered":"2023-10-09T06:39:53.071582+0000","last_clean":"2023-10-09T06:39:53.071582+0000","last_became_active":"2023-10-09T06:39:52.078011+0000","last_became_peered":"2023-10-09T06:39:52.078011+0000","last_unstale":"2023-10-09T06:39:53.071582+0000","last_undegraded":"2023-10-09T06:39:53.071582+0000","last_fullsized":"2023-10-09T06:39:53.071582+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T13:39:21.726799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.083929+0000","last_change":"2023-10-09T06:39:53.078203+0000","last_active":"2023-10-09T06:39:53.083929+0000","last_peered":"2023-10-09T06:39:53.083929+0000","last_clean":"2023-10-09T06:39:53.083929+0000","last_became_active":"2023-10-09T06:39:53.077948+0000","last_became_peered":"2023-10-09T06:39:53.077948+0000","last_unstale":"2023-10-09T06:39:53.083929+0000","last_undegraded":"2023-10-09T06:39:53.083929+0000","last_fullsized":"2023-10-09T06:39:53.083929+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:46:34.854718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.078204+0000","last_change":"2023-10-09T06:39:52.083232+0000","last_active":"2023-10-09T06:39:53.078204+0000","last_peered":"2023-10-09T06:39:53.078204+0000","last_clean":"2023-10-09T06:39:53.078204+0000","last_became_active":"2023-10-09T06:39:52.082987+0000","last_became_peered":"2023-10-09T06:39:52.082987+0000","last_unstale":"2023-10-09T06:39:53.078204+0000","last_undegraded":"2023-10-09T06:39:53.078204+0000","last_fullsized":"2023-10-09T06:39:53.078204+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T09:24:59.257776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.086379+0000","last_change":"2023-10-09T06:39:53.077883+0000","last_active":"2023-10-09T06:39:53.086379+0000","last_peered":"2023-10-09T06:39:53.086379+0000","last_clean":"2023-10-09T06:39:53.086379+0000","last_became_active":"2023-10-09T06:39:53.077639+0000","last_became_peered":"2023-10-09T06:39:53.077639+0000","last_unstale":"2023-10-09T06:39:53.086379+0000","last_undegraded":"2023-10-09T06:39:53.086379+0000","last_fullsized":"2023-10-09T06:39:53.086379+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:52:02.602342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.071395+0000","last_change":"2023-10-09T06:39:52.077364+0000","last_active":"2023-10-09T06:39:53.071395+0000","last_peered":"2023-10-09T06:39:53.071395+0000","last_clean":"2023-10-09T06:39:53.071395+0000","last_became_active":"2023-10-09T06:39:52.077098+0000","last_became_peered":"2023-10-09T06:39:52.077098+0000","last_unstale":"2023-10-09T06:39:53.071395+0000","last_undegraded":"2023-10-09T06:39:53.071395+0000","last_fullsized":"2023-10-09T06:39:53.071395+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:51.053679+0000","last_clean_scrub_stamp":"2023-10-09T06:39:51.053679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T17:34:55.758128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.23","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.087129+0000","last_change":"2023-10-09T06:39:53.084392+0000","last_active":"2023-10-09T06:39:53.087129+0000","last_peered":"2023-10-09T06:39:53.087129+0000","last_clean":"2023-10-09T06:39:53.087129+0000","last_became_active":"2023-10-09T06:39:53.084132+0000","last_became_peered":"2023-10-09T06:39:53.084132+0000","last_unstale":"2023-10-09T06:39:53.087129+0000","last_undegraded":"2023-10-09T06:39:53.087129+0000","last_fullsized":"2023-10-09T06:39:53.087129+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:39:52.057892+0000","last_clean_scrub_stamp":"2023-10-09T06:39:52.057892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T07:45:42.010880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":308,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-09T06:39:53.077723+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:39:53.077723+0000","last_peered":"2023-10-09T06:39:53.077723+0000","last_clean":"2023-10-09T06:39:53.077723+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:39:53.077723+0000","last_undegraded":"2023-10-09T06:39:53.077723+0000","last_fullsized":"2023-10-09T06:39:53.077723+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":34,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":33,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574965,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304420,"kb_used_data":3036,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067420,"statfs":{"total":96636764160,"available":96325038080,"internally_reserved":0,"allocated":3108864,"data_stored":1716694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.55600000000000005,"15min":0.52300000000000002},"min":{"1min":0.442,"5min":0.308,"15min":0.308},"max":{"1min":3.778,"5min":3.778,"15min":3.778},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.51400000000000001,"15min":0.51500000000000001},"min":{"1min":0.35599999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":0.63300000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.51600000000000001,"15min":0.52000000000000002},"min":{"1min":0.36599999999999999,"5min":0.33600000000000002,"15min":0.27100000000000002},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.54500000000000004,"15min":0.53200000000000003},"min":{"1min":0.39000000000000001,"5min":0.32900000000000001,"15min":0.29699999999999999},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.496}]},{"osd":2,"last update":"Mon Oct 9 06:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.55800000000000005,"15min":0.52900000000000003},"min":{"1min":0.38300000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":3.7210000000000001,"5min":3.7210000000000001,"15min":3.7210000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.315},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.60499999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.57899999999999996,"15min":0.56200000000000006},"min":{"1min":0.38600000000000001,"5min":0.35899999999999999,"15min":0.33500000000000002},"max":{"1min":3.8090000000000002,"5min":3.8090000000000002,"15min":3.8090000000000002},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56200000000000006,"15min":0.58499999999999996},"min":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.36499999999999999},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.56299999999999994}]},{"osd":4,"last update":"Mon Oct 9 06:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.51400000000000001,"15min":0.51900000000000002},"min":{"1min":0.28899999999999998,"5min":0.25600000000000001,"15min":0.24399999999999999},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.628,"15min":0.61499999999999999},"min":{"1min":0.39700000000000002,"5min":0.35199999999999998,"15min":0.30299999999999999},"max":{"1min":4.8239999999999998,"5min":4.8239999999999998,"15min":4.8239999999999998},"last":0.76000000000000001}]},{"osd":5,"last update":"Mon Oct 9 06:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54900000000000004,"15min":0.56399999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.307},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.64200000000000002,"15min":0.626},"min":{"1min":0.35299999999999998,"5min":0.34300000000000003,"15min":0.34200000000000003},"max":{"1min":3.6909999999999998,"5min":3.6909999999999998,"15min":3.6909999999999998},"last":0.747}]},{"osd":6,"last update":"Mon Oct 9 06:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.36399999999999999,"5min":0.318,"15min":0.28699999999999998},"max":{"1min":3.5529999999999999,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.63700000000000001,"15min":0.621},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.315},"max":{"1min":4.7789999999999999,"5min":4.7789999999999999,"15min":4.7789999999999999},"last":0.53800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574964,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303836,"kb_used_data":2452,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068004,"statfs":{"total":96636764160,"available":96325636096,"internally_reserved":0,"allocated":2510848,"data_stored":1125784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52500000000000002,"15min":0.50700000000000001},"min":{"1min":0.41299999999999998,"5min":0.35399999999999998,"15min":0.30399999999999999},"max":{"1min":0.80500000000000005,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.752},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52400000000000002,"15min":0.51500000000000001},"min":{"1min":0.33000000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.67400000000000004}]},{"osd":1,"last update":"Mon Oct 9 06:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.50900000000000001,"15min":0.51000000000000001},"min":{"1min":0.41299999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.65900000000000003,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54600000000000004,"15min":0.53500000000000003},"min":{"1min":0.42999999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73699999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.73799999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.375,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71199999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52700000000000002,"15min":0.53000000000000003},"min":{"1min":0.438,"5min":0.313,"15min":0.313},"max":{"1min":0.71699999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.62}]},{"osd":3,"last update":"Mon Oct 9 06:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54900000000000004,"15min":0.54500000000000004},"min":{"1min":0.39400000000000002,"5min":0.38900000000000001,"15min":0.33500000000000002},"max":{"1min":0.76500000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55900000000000005,"15min":0.65900000000000003},"min":{"1min":0.46400000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74099999999999999,"5min":0.76900000000000002,"15min":4.7510000000000003},"last":0.72699999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.46999999999999997,"15min":0.47599999999999998},"min":{"1min":0.29299999999999998,"5min":0.28199999999999997,"15min":0.27900000000000003},"max":{"1min":0.72799999999999998,"5min":0.74299999999999999,"15min":0.76400000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57099999999999995,"15min":0.58199999999999996},"min":{"1min":0.38700000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.78600000000000003,"5min":0.80800000000000005,"15min":0.83699999999999997},"last":0.76900000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52300000000000002,"15min":0.53100000000000003},"min":{"1min":0.375,"5min":0.246,"15min":0.246},"max":{"1min":0.745,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57399999999999995,"15min":0.59099999999999997},"min":{"1min":0.40999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76200000000000001,"5min":0.78000000000000003,"15min":0.86599999999999999},"last":0.68300000000000005}]},{"osd":7,"last update":"Mon Oct 9 06:39:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.57799999999999996},"min":{"1min":0.34799999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.81499999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.38400000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.84399999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.69599999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303840,"kb_used_data":2456,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068000,"statfs":{"total":96636764160,"available":96325632000,"internally_reserved":0,"allocated":2514944,"data_stored":1125874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.29899999999999999,"5min":0.315,"15min":0.31},"min":{"1min":0.22900000000000001,"5min":0.192,"15min":0.192},"max":{"1min":0.46200000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35399999999999998,"15min":0.35799999999999998},"min":{"1min":0.26500000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.48599999999999999,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.46200000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.34300000000000003,"15min":0.34799999999999998},"min":{"1min":0.24099999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.44,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.378},{"interface":"front","average":{"1min":0.373,"5min":0.38100000000000001,"15min":0.38400000000000001},"min":{"1min":0.27100000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.49399999999999999,"5min":0.60399999999999998,"15min":0.60899999999999999},"last":0.48799999999999999}]},{"osd":3,"last update":"Mon Oct 9 06:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.372,"15min":0.36499999999999999},"min":{"1min":0.254,"5min":0.22,"15min":0.22},"max":{"1min":0.52700000000000002,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.34599999999999997,"15min":0.49399999999999999},"min":{"1min":0.23799999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.54800000000000004,"5min":0.58199999999999996,"15min":5.8529999999999998},"last":0.50600000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57799999999999996,"15min":0.57699999999999996},"min":{"1min":0.371,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.873,"5min":1.01,"15min":1.01},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59199999999999997,"15min":0.58799999999999997},"min":{"1min":0.42599999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.79400000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.68100000000000005}]},{"osd":5,"last update":"Mon Oct 9 06:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57199999999999995,"15min":0.70399999999999996},"min":{"1min":0.39500000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73699999999999999,"5min":0.92500000000000004,"15min":5.9459999999999997},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59999999999999998,"15min":0.60699999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.34300000000000003},"max":{"1min":0.76700000000000002,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.71299999999999997}]},{"osd":6,"last update":"Mon Oct 9 06:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.54700000000000004,"15min":0.67600000000000005},"min":{"1min":0.39500000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.80800000000000005,"5min":0.81399999999999995,"15min":5.9160000000000004},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57499999999999996,"15min":0.57799999999999996},"min":{"1min":0.42399999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83899999999999997,"5min":0.91000000000000003,"15min":0.91400000000000003},"last":0.68700000000000006}]},{"osd":7,"last update":"Mon Oct 9 06:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56299999999999994,"15min":0.58099999999999996},"min":{"1min":0.42999999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.73299999999999998,"5min":0.91000000000000003,"15min":0.97099999999999997},"last":0.628},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56200000000000006,"15min":0.70699999999999996},"min":{"1min":0.43099999999999999,"5min":0.40400000000000003,"15min":0.38100000000000001},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":5.9729999999999999},"last":0.69799999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574966,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304416,"kb_used_data":3032,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067424,"statfs":{"total":96636764160,"available":96325042176,"internally_reserved":0,"allocated":3104768,"data_stored":1716188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.32500000000000001,"15min":0.31900000000000001},"min":{"1min":0.253,"5min":0.214,"15min":0.21299999999999999},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34100000000000003,"15min":0.34399999999999997},"min":{"1min":0.24199999999999999,"5min":0.222,"15min":0.214},"max":{"1min":0.51300000000000001,"5min":0.53600000000000003,"15min":0.94499999999999995},"last":0.42599999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.371,"15min":0.36399999999999999},"min":{"1min":0.22,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.495,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38900000000000001,"15min":0.38800000000000001},"min":{"1min":0.33200000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.49199999999999999,"5min":0.69399999999999995,"15min":0.91700000000000004},"last":0.46400000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.375},"min":{"1min":0.23999999999999999,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":1.554},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.373,"5min":0.375,"15min":0.39300000000000002},"min":{"1min":0.28100000000000003,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.47299999999999998,"5min":0.58099999999999996,"15min":0.79000000000000004},"last":0.40899999999999997}]},{"osd":4,"last update":"Mon Oct 9 06:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56200000000000006,"15min":0.55700000000000005},"min":{"1min":0.41499999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":1.212},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56799999999999995,"15min":0.68799999999999994},"min":{"1min":0.41599999999999998,"5min":0.37,"15min":0.34799999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":5.484},"last":0.69699999999999995}]},{"osd":5,"last update":"Mon Oct 9 06:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56599999999999995,"15min":0.69099999999999995},"min":{"1min":0.40300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":5.5449999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57799999999999996,"15min":0.57699999999999996},"min":{"1min":0.42199999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.998,"5min":0.998,"15min":1.113},"last":0.56299999999999994}]},{"osd":6,"last update":"Mon Oct 9 06:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.44800000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":1.0529999999999999},"last":0.628},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60699999999999998,"15min":0.60899999999999999},"min":{"1min":0.45800000000000002,"5min":0.42799999999999999,"15min":0.41599999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":1.169},"last":0.68000000000000005}]},{"osd":7,"last update":"Mon Oct 9 06:39:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.435,"5min":0.39700000000000002,"15min":0.371},"max":{"1min":0.878,"5min":0.878,"15min":1.2390000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57299999999999995,"15min":0.71199999999999997},"min":{"1min":0.442,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":5.5800000000000001},"last":0.66400000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574967,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303852,"kb_used_data":2468,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067988,"statfs":{"total":96636764160,"available":96325619712,"internally_reserved":0,"allocated":2527232,"data_stored":1127862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:39:52 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.318,"15min":0.34499999999999997},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.19900000000000001},"max":{"1min":0.45700000000000002,"5min":0.66000000000000003,"15min":1.4199999999999999},"last":0.20000000000000001},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.33400000000000002,"15min":0.50900000000000001},"min":{"1min":0.221,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.433,"5min":0.55600000000000005,"15min":7.2699999999999996},"last":0.221}]},{"osd":1,"last update":"Mon Oct 9 06:39:52 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.31900000000000001,"15min":0.33900000000000002},"min":{"1min":0.24199999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.442,"5min":0.55200000000000005,"15min":0.65400000000000003},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.372,"5min":0.376,"15min":0.56000000000000005},"min":{"1min":0.253,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.51600000000000001,"5min":0.57999999999999996,"15min":7.327},"last":0.29599999999999999}]},{"osd":3,"last update":"Mon Oct 9 06:39:52 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.379,"15min":0.38900000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.49299999999999999,"5min":0.64600000000000002,"15min":0.68600000000000005},"last":0.307},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.373,"15min":0.38700000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.23999999999999999},"max":{"1min":0.5,"5min":0.629,"15min":0.72699999999999998},"last":0.252}]},{"osd":4,"last update":"Mon Oct 9 06:39:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54600000000000004,"15min":0.54300000000000004},"min":{"1min":0.40000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.72999999999999998,"5min":0.79700000000000004,"15min":0.80300000000000005},"last":0.438},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56000000000000005,"15min":0.73399999999999999},"min":{"1min":0.38800000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71599999999999997,"5min":0.83199999999999996,"15min":7.2389999999999999},"last":0.38800000000000001}]},{"osd":5,"last update":"Mon Oct 9 06:39:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56999999999999995,"15min":0.83999999999999997},"min":{"1min":0.38100000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77100000000000002,"5min":0.82699999999999996,"15min":10.890000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59799999999999998,"15min":0.60099999999999998},"min":{"1min":0.39800000000000002,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":0.78200000000000003,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.39800000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:39:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56699999999999995,"15min":0.74199999999999999},"min":{"1min":0.35999999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.68999999999999995,"5min":0.77300000000000002,"15min":7.2999999999999998},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57899999999999996,"15min":0.57599999999999996},"min":{"1min":0.436,"5min":0.377,"15min":0.377},"max":{"1min":0.83099999999999996,"5min":0.875,"15min":0.875},"last":0.50700000000000001}]},{"osd":7,"last update":"Mon Oct 9 06:39:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58999999999999997,"15min":0.77100000000000002},"min":{"1min":0.42099999999999999,"5min":0.38600000000000001,"15min":0.37},"max":{"1min":0.81299999999999994,"5min":0.91300000000000003,"15min":7.1879999999999997},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60099999999999998,"15min":0.621},"min":{"1min":0.47699999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.86699999999999999,"5min":0.89800000000000002,"15min":0.90500000000000003},"last":0.56899999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574966,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303836,"kb_used_data":2452,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068004,"statfs":{"total":96636764160,"available":96325636096,"internally_reserved":0,"allocated":2510848,"data_stored":1125840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:39:50 2023","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.32600000000000001,"15min":0.33300000000000002},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.438,"5min":0.61699999999999999,"15min":5.8700000000000001},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.34899999999999998,"15min":0.36499999999999999},"min":{"1min":0.23799999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.45900000000000002,"5min":0.57199999999999995,"15min":5.6769999999999996},"last":0.28299999999999997}]},{"osd":1,"last update":"Mon Oct 9 06:39:50 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.34799999999999998,"15min":0.35799999999999998},"min":{"1min":0.23999999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.47199999999999998,"5min":0.59499999999999997,"15min":4.0469999999999997},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.377,"15min":0.38800000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.51400000000000001,"5min":0.68600000000000005,"15min":4.1509999999999998},"last":0.27600000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:39:50 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.35099999999999998,"15min":0.36799999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.222},"max":{"1min":0.41399999999999998,"5min":0.56799999999999995,"15min":3.1139999999999999},"last":0.254},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.36899999999999999,"15min":0.38300000000000001},"min":{"1min":0.26000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.51500000000000001,"5min":0.66900000000000004,"15min":3.3180000000000001},"last":0.29799999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:39:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55300000000000005,"15min":0.56799999999999995},"min":{"1min":0.38700000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72399999999999998,"5min":0.81699999999999995,"15min":3.0899999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.40699999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.71999999999999997,"5min":0.77400000000000002,"15min":3.0710000000000002},"last":0.46700000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:39:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57299999999999995,"15min":0.58599999999999997},"min":{"1min":0.38900000000000001,"5min":0.33600000000000002,"15min":0.32700000000000001},"max":{"1min":0.69299999999999995,"5min":0.877,"15min":6.2300000000000004},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.58399999999999996,"15min":0.59199999999999997},"min":{"1min":0.41799999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":3.0219999999999998},"last":0.38}]},{"osd":6,"last update":"Mon Oct 9 06:39:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58999999999999997,"15min":0.59299999999999997},"min":{"1min":0.42699999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.73799999999999999,"5min":0.89200000000000002,"15min":2.6850000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63700000000000001,"15min":0.64900000000000002},"min":{"1min":0.45700000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.77400000000000002,"5min":0.94299999999999995,"15min":6.5970000000000004},"last":0.51600000000000001}]},{"osd":7,"last update":"Mon Oct 9 06:39:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.60599999999999998,"15min":0.61199999999999999},"min":{"1min":0.45100000000000001,"5min":0.41699999999999998,"15min":0.38200000000000001},"max":{"1min":0.78300000000000003,"5min":0.86099999999999999,"15min":4.2629999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59099999999999997,"15min":0.61099999999999999},"min":{"1min":0.438,"5min":0.374,"15min":0.374},"max":{"1min":0.75700000000000001,"5min":0.82899999999999996,"15min":7.3650000000000002},"last":0.48899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574965,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303844,"kb_used_data":2460,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067996,"statfs":{"total":96636764160,"available":96325627904,"internally_reserved":0,"allocated":2519040,"data_stored":1127286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51500000000000001,"15min":0.54600000000000004},"min":{"1min":0.34499999999999997,"5min":0.28100000000000003,"15min":0.27000000000000002},"max":{"1min":0.79000000000000004,"5min":0.88900000000000001,"15min":1.083},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51700000000000002,"15min":0.51900000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.65600000000000003,"5min":0.97799999999999998,"15min":1.054},"last":0.58599999999999997}]},{"osd":1,"last update":"Mon Oct 9 06:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.53100000000000003,"15min":0.54500000000000004},"min":{"1min":0.34599999999999997,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67100000000000004,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.502,"15min":0.498},"min":{"1min":0.38900000000000001,"5min":0.31900000000000001,"15min":0.31},"max":{"1min":0.69999999999999996,"5min":0.76300000000000001,"15min":0.79100000000000004},"last":0.45800000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52200000000000002,"15min":0.53600000000000003},"min":{"1min":0.33100000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84299999999999997,"5min":0.84899999999999998,"15min":1.0249999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54500000000000004,"15min":0.58199999999999996},"min":{"1min":0.41399999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.67900000000000005,"5min":0.90500000000000003,"15min":0.93799999999999994},"last":0.50900000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52200000000000002,"15min":0.54600000000000004},"min":{"1min":0.33700000000000002,"5min":0.32200000000000001,"15min":0.29199999999999998},"max":{"1min":0.69199999999999995,"5min":0.86899999999999999,"15min":0.98499999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57099999999999995,"15min":0.58499999999999996},"min":{"1min":0.42299999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76300000000000001,"5min":0.92900000000000005,"15min":1.1319999999999999},"last":0.47999999999999998}]},{"osd":5,"last update":"Mon Oct 9 06:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.52900000000000003,"15min":0.54700000000000004},"min":{"1min":0.38,"5min":0.30399999999999999,"15min":0.27300000000000002},"max":{"1min":0.73999999999999999,"5min":0.81100000000000005,"15min":0.998},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.58499999999999996,"15min":0.60299999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35099999999999998},"max":{"1min":0.85599999999999998,"5min":0.95599999999999996,"15min":0.97999999999999998},"last":0.498}]},{"osd":6,"last update":"Mon Oct 9 06:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.36899999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.86299999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.47199999999999998,"5min":0.38700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78500000000000003,"5min":1.01,"15min":1.01},"last":0.54900000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:39:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.55500000000000005,"15min":0.56699999999999995},"min":{"1min":0.42899999999999999,"5min":0.33600000000000002,"15min":0.28299999999999997},"max":{"1min":0.81399999999999995,"5min":0.83199999999999996,"15min":0.90000000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61899999999999999,"15min":0.63800000000000001},"min":{"1min":0.437,"5min":0.34999999999999998,"15min":0.32200000000000001},"max":{"1min":0.93799999999999994,"5min":0.98999999999999999,"15min":1.175},"last":0.59999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574965,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303836,"kb_used_data":2452,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068004,"statfs":{"total":96636764160,"available":96325636096,"internally_reserved":0,"allocated":2510848,"data_stored":1125820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:39:49 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51100000000000001,"15min":0.50800000000000001},"min":{"1min":0.38400000000000001,"5min":0.30599999999999999,"15min":0.28599999999999998},"max":{"1min":0.64700000000000002,"5min":0.79700000000000004,"15min":0.86599999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.41099999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.65400000000000003}]},{"osd":1,"last update":"Mon Oct 9 06:39:49 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.50800000000000001,"15min":0.50900000000000001},"min":{"1min":0.34499999999999997,"5min":0.33000000000000002,"15min":0.311},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.78200000000000003},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.52000000000000002,"15min":0.52800000000000002},"min":{"1min":0.36899999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63,"5min":0.76600000000000001,"15min":0.81599999999999995},"last":0.55900000000000005}]},{"osd":2,"last update":"Mon Oct 9 06:39:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.55800000000000005,"15min":0.55000000000000004},"min":{"1min":0.34699999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.623},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.57199999999999995,"15min":0.58299999999999996},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78200000000000003,"5min":0.79300000000000004,"15min":0.79900000000000004},"last":0.73799999999999999}]},{"osd":3,"last update":"Mon Oct 9 06:39:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55700000000000005,"15min":0.54700000000000004},"min":{"1min":0.39200000000000002,"5min":0.317,"15min":0.317},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54500000000000004,"15min":0.55800000000000005},"min":{"1min":0.42599999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.68700000000000006,"5min":0.71099999999999997,"15min":0.81100000000000005},"last":0.67500000000000004}]},{"osd":4,"last update":"Mon Oct 9 06:39:49 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.504,"15min":0.5},"min":{"1min":0.28399999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.74399999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54900000000000004,"15min":0.54800000000000004},"min":{"1min":0.39600000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.70299999999999996}]},{"osd":6,"last update":"Mon Oct 9 06:39:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.56899999999999995,"15min":0.54700000000000004},"min":{"1min":0.34799999999999998,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.71499999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.63600000000000001}]},{"osd":7,"last update":"Mon Oct 9 06:39:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60099999999999998,"15min":0.60799999999999998},"min":{"1min":0.373,"5min":0.35199999999999998,"15min":0.314},"max":{"1min":0.71099999999999997,"5min":0.84499999999999997,"15min":0.90600000000000003},"last":0.77200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:39:57.507 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-09T06:39:57.507 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-09T06:39:57.662 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.662+0000 7f0d995b6700 1 Processor -- start 2023-10-09T06:39:57.662 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.662+0000 7f0d995b6700 1 -- start start 2023-10-09T06:39:57.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.663+0000 7f0d995b6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0d94069c70 0x7f0d940cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:57.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.663+0000 7f0d995b6700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d940ce380 con 0x7f0d940af320 2023-10-09T06:39:57.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.663+0000 7f0d995b6700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d940cec20 con 0x7f0d94159950 2023-10-09T06:39:57.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.663+0000 7f0d995b6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d940afa90 con 0x7f0d94069c70 2023-10-09T06:39:57.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.663+0000 7f0d937fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f0d940af320 0x7f0d940cf0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:36490/0 (socket says 172.21.15.163:36490) 2023-10-09T06:39:57.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.663+0000 7f0d937fe700 1 -- 172.21.15.163:0/2331847292 learned_addr learned my addr 172.21.15.163:0/2331847292 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:39:57.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.663+0000 7f0d8a7fc700 1 --2- 172.21.15.163:0/2331847292 >> v2:172.21.15.181:3300/0 conn(0x7f0d94069c70 0x7f0d940cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:57.671 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.663+0000 7f0d927fc700 1 -- 172.21.15.163:0/2331847292 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1159498056 0 0) 0x7f0d940ce380 con 0x7f0d940af320 2023-10-09T06:39:57.671 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.663+0000 7f0d927fc700 1 -- 172.21.15.163:0/2331847292 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d80003570 con 0x7f0d940af320 2023-10-09T06:39:57.671 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.663+0000 7f0d8a7fc700 1 -- 172.21.15.163:0/2331847292 >> v1:172.21.15.181:6789/0 conn(0x7f0d94159950 legacy=0x7f0d94069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:57.671 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.663+0000 7f0d927fc700 1 -- 172.21.15.163:0/2331847292 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3529247669 0 0) 0x7f0d940cec20 con 0x7f0d94159950 2023-10-09T06:39:57.671 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.663+0000 7f0d8a7fc700 1 -- 172.21.15.163:0/2331847292 >> v1:172.21.15.163:6789/0 conn(0x7f0d940af320 legacy=0x7f0d940cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:57.671 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.663+0000 7f0d8a7fc700 1 -- 172.21.15.163:0/2331847292 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d940ce5d0 con 0x7f0d94069c70 2023-10-09T06:39:57.671 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.664+0000 7f0d8a7fc700 1 --2- 172.21.15.163:0/2331847292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d94069c70 0x7f0d940cbb90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c0098e0 tx=0x7f0d7c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e5f570b511acf1a server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:57.671 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.664+0000 7f0d927fc700 1 -- 172.21.15.163:0/2331847292 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c0017f0 con 0x7f0d94069c70 2023-10-09T06:39:57.671 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.664+0000 7f0d927fc700 1 -- 172.21.15.163:0/2331847292 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7c011420 con 0x7f0d94069c70 2023-10-09T06:39:57.671 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.664+0000 7f0d927fc700 1 -- 172.21.15.163:0/2331847292 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c0117a0 con 0x7f0d94069c70 2023-10-09T06:39:57.671 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.664+0000 7f0d995b6700 1 -- 172.21.15.163:0/2331847292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d94069c70 msgr2=0x7f0d940cbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:57.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.664+0000 7f0d995b6700 1 --2- 172.21.15.163:0/2331847292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d94069c70 0x7f0d940cbb90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c0098e0 tx=0x7f0d7c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:57.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.664+0000 7f0d995b6700 1 -- 172.21.15.163:0/2331847292 shutdown_connections 2023-10-09T06:39:57.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.664+0000 7f0d995b6700 1 --2- 172.21.15.163:0/2331847292 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d94069c70 0x7f0d940cbb90 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:57.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.664+0000 7f0d995b6700 1 -- 172.21.15.163:0/2331847292 >> 172.21.15.163:0/2331847292 conn(0x7f0d94068e00 msgr2=0x7f0d940d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:57.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.664+0000 7f0d995b6700 1 -- 172.21.15.163:0/2331847292 shutdown_connections 2023-10-09T06:39:57.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.664+0000 7f0d995b6700 1 -- 172.21.15.163:0/2331847292 wait complete. 2023-10-09T06:39:57.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.666+0000 7f0d995b6700 1 Processor -- start 2023-10-09T06:39:57.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.666+0000 7f0d995b6700 1 -- start start 2023-10-09T06:39:57.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.666+0000 7f0d995b6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d94069c70 0x7f0d940b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:57.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.666+0000 7f0d995b6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d940af320 0x7f0d94151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:57.672 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.666+0000 7f0d995b6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d94159950 0x7f0d941589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:57.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.666+0000 7f0d995b6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d940b60b0 con 0x7f0d940af320 2023-10-09T06:39:57.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.666+0000 7f0d995b6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d940b56e0 con 0x7f0d94069c70 2023-10-09T06:39:57.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.666+0000 7f0d995b6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d940b4ad0 con 0x7f0d94159950 2023-10-09T06:39:57.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.666+0000 7f0d8a7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d940af320 0x7f0d94151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:57.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.666+0000 7f0d8a7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d940af320 0x7f0d94151e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49198/0 (socket says 172.21.15.163:49198) 2023-10-09T06:39:57.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.666+0000 7f0d8a7fc700 1 -- 172.21.15.163:0/1820301200 learned_addr learned my addr 172.21.15.163:0/1820301200 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:57.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.667+0000 7f0d92ffd700 1 --2- 172.21.15.163:0/1820301200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d94069c70 0x7f0d940b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:57.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.667+0000 7f0d937fe700 1 --2- 172.21.15.163:0/1820301200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d94159950 0x7f0d941589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:57.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.667+0000 7f0d8a7fc700 1 -- 172.21.15.163:0/1820301200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d94159950 msgr2=0x7f0d941589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:57.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.667+0000 7f0d8a7fc700 1 --2- 172.21.15.163:0/1820301200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d94159950 0x7f0d941589f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:57.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.667+0000 7f0d8a7fc700 1 -- 172.21.15.163:0/1820301200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d94069c70 msgr2=0x7f0d940b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:57.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.667+0000 7f0d8a7fc700 1 --2- 172.21.15.163:0/1820301200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d94069c70 0x7f0d940b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:57.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.667+0000 7f0d8a7fc700 1 -- 172.21.15.163:0/1820301200 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c009420 con 0x7f0d940af320 2023-10-09T06:39:57.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.667+0000 7f0d92ffd700 1 --2- 172.21.15.163:0/1820301200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d94069c70 0x7f0d940b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:57.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.667+0000 7f0d8a7fc700 1 --2- 172.21.15.163:0/1820301200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d940af320 0x7f0d94151e90 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f0d94070e20 tx=0x7f0d7c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:57.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.667+0000 7f0d8b7fe700 1 -- 172.21.15.163:0/1820301200 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c010cb0 con 0x7f0d940af320 2023-10-09T06:39:57.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.667+0000 7f0d8b7fe700 1 -- 172.21.15.163:0/1820301200 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7c010e10 con 0x7f0d940af320 2023-10-09T06:39:57.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.667+0000 7f0d995b6700 1 -- 172.21.15.163:0/1820301200 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d940b4db0 con 0x7f0d940af320 2023-10-09T06:39:57.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.667+0000 7f0d8b7fe700 1 -- 172.21.15.163:0/1820301200 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c01d680 con 0x7f0d940af320 2023-10-09T06:39:57.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.667+0000 7f0d995b6700 1 -- 172.21.15.163:0/1820301200 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d940b5370 con 0x7f0d940af320 2023-10-09T06:39:57.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.668+0000 7f0d995b6700 1 -- 172.21.15.163:0/1820301200 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d940b5890 con 0x7f0d940af320 2023-10-09T06:39:57.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.669+0000 7f0d8b7fe700 1 -- 172.21.15.163:0/1820301200 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0d7c01dab0 con 0x7f0d940af320 2023-10-09T06:39:57.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.670+0000 7f0d8b7fe700 1 --2- 172.21.15.163:0/1820301200 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0d74067320 0x7f0d740697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:57.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.670+0000 7f0d8b7fe700 1 -- 172.21.15.163:0/1820301200 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0d7c087070 con 0x7f0d940af320 2023-10-09T06:39:57.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.670+0000 7f0d92ffd700 1 --2- 172.21.15.163:0/1820301200 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0d74067320 0x7f0d740697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:57.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.670+0000 7f0d92ffd700 1 --2- 172.21.15.163:0/1820301200 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0d74067320 0x7f0d740697d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f0d940ca300 tx=0x7f0d84009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:57.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.673+0000 7f0d8b7fe700 1 -- 172.21.15.163:0/1820301200 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d7c053d80 con 0x7f0d940af320 2023-10-09T06:39:57.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.933+0000 7f0d995b6700 1 -- 172.21.15.163:0/1820301200 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f0d940b5890 con 0x7f0d940af320 2023-10-09T06:39:57.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.936+0000 7f0d8b7fe700 1 -- 172.21.15.163:0/1820301200 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v25) v1 ==== 190+0+0 (secure 0 0 0) 0x7f0d940b5890 con 0x7f0d940af320 2023-10-09T06:39:57.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.938+0000 7f0d995b6700 1 -- 172.21.15.163:0/1820301200 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0d74067320 msgr2=0x7f0d740697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:57.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.938+0000 7f0d995b6700 1 --2- 172.21.15.163:0/1820301200 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0d74067320 0x7f0d740697d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f0d940ca300 tx=0x7f0d84009040 comp rx=0 tx=0).stop 2023-10-09T06:39:57.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.938+0000 7f0d995b6700 1 -- 172.21.15.163:0/1820301200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d940af320 msgr2=0x7f0d94151e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:57.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.938+0000 7f0d995b6700 1 --2- 172.21.15.163:0/1820301200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d940af320 0x7f0d94151e90 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f0d94070e20 tx=0x7f0d7c017000 comp rx=0 tx=0).stop 2023-10-09T06:39:57.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.938+0000 7f0d995b6700 1 -- 172.21.15.163:0/1820301200 shutdown_connections 2023-10-09T06:39:57.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.938+0000 7f0d995b6700 1 --2- 172.21.15.163:0/1820301200 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0d74067320 0x7f0d740697d0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:57.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.938+0000 7f0d995b6700 1 --2- 172.21.15.163:0/1820301200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d94159950 0x7f0d941589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:57.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.938+0000 7f0d995b6700 1 --2- 172.21.15.163:0/1820301200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d94069c70 0x7f0d940b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:57.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.938+0000 7f0d995b6700 1 --2- 172.21.15.163:0/1820301200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d940af320 0x7f0d94151e90 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:57.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.938+0000 7f0d995b6700 1 -- 172.21.15.163:0/1820301200 >> 172.21.15.163:0/1820301200 conn(0x7f0d94068e00 msgr2=0x7f0d940b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:57.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.939+0000 7f0d995b6700 1 -- 172.21.15.163:0/1820301200 shutdown_connections 2023-10-09T06:39:57.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:57.939+0000 7f0d995b6700 1 -- 172.21.15.163:0/1820301200 wait complete. 2023-10-09T06:39:57.940 INFO:teuthology.orchestra.run.smithi163.stderr:updated caps for client.0 2023-10-09T06:39:57.950 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:39:58.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.065+0000 7f50d697c700 1 Processor -- start 2023-10-09T06:39:58.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.065+0000 7f50d697c700 1 -- start start 2023-10-09T06:39:58.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.066+0000 7f50d697c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50d0159890 0x7f50d015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:58.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.066+0000 7f50d697c700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f50d0159c60 0x7f50d015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:58.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.066+0000 7f50d697c700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d015a160 con 0x7f50d015db10 2023-10-09T06:39:58.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.066+0000 7f50d697c700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50d00d13a0 con 0x7f50d0159c60 2023-10-09T06:39:58.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.066+0000 7f50d697c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d015e140 con 0x7f50d0159890 2023-10-09T06:39:58.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.066+0000 7f50cffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50d0159890 0x7f50d015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:58.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.067+0000 7f50cf7fe700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f50d0159c60 0x7f50d015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:58.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.067+0000 7f50cffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50d0159890 0x7f50d015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.163:47284/0 (socket says 172.21.15.163:47284) 2023-10-09T06:39:58.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.067+0000 7f50cffff700 1 -- 172.21.15.163:0/2991157884 learned_addr learned my addr 172.21.15.163:0/2991157884 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:58.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.067+0000 7f50cffff700 1 -- 172.21.15.163:0/2991157884 >> v2:172.21.15.181:3300/0 conn(0x7f50d0159c60 msgr2=0x7f50d015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:58.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.067+0000 7f50cffff700 1 --2- 172.21.15.163:0/2991157884 >> v2:172.21.15.181:3300/0 conn(0x7f50d0159c60 0x7f50d015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.067+0000 7f50cffff700 1 -- 172.21.15.163:0/2991157884 >> v1:172.21.15.181:6789/0 conn(0x7f50d015db10 legacy=0x7f50d0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:58.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.067+0000 7f50cffff700 1 -- 172.21.15.163:0/2991157884 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d015a3d0 con 0x7f50d0159890 2023-10-09T06:39:58.070 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.067+0000 7f50cf7fe700 1 --2- 172.21.15.163:0/2991157884 >> v2:172.21.15.181:3300/0 conn(0x7f50d0159c60 0x7f50d015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:39:58.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.067+0000 7f50cffff700 1 --2- 172.21.15.163:0/2991157884 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50d0159890 0x7f50d015abd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f50c00099b0 tx=0x7f50c0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2db55da1055a399 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:58.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.067+0000 7f50ceffd700 1 -- 172.21.15.163:0/2991157884 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0009ea0 con 0x7f50d0159890 2023-10-09T06:39:58.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.067+0000 7f50ceffd700 1 -- 172.21.15.163:0/2991157884 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c000b860 con 0x7f50d0159890 2023-10-09T06:39:58.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.067+0000 7f50ceffd700 1 -- 172.21.15.163:0/2991157884 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c000bc00 con 0x7f50d0159890 2023-10-09T06:39:58.071 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.068+0000 7f50d697c700 1 -- 172.21.15.163:0/2991157884 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50d0159890 msgr2=0x7f50d015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:58.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.068+0000 7f50d697c700 1 --2- 172.21.15.163:0/2991157884 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50d0159890 0x7f50d015abd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f50c00099b0 tx=0x7f50c0009250 comp rx=0 tx=0).stop 2023-10-09T06:39:58.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.068+0000 7f50d697c700 1 -- 172.21.15.163:0/2991157884 shutdown_connections 2023-10-09T06:39:58.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.068+0000 7f50d697c700 1 --2- 172.21.15.163:0/2991157884 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50d0159890 0x7f50d015abd0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.068+0000 7f50d697c700 1 --2- 172.21.15.163:0/2991157884 >> v2:172.21.15.181:3300/0 conn(0x7f50d0159c60 0x7f50d015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.068+0000 7f50d697c700 1 -- 172.21.15.163:0/2991157884 >> 172.21.15.163:0/2991157884 conn(0x7f50d0068e20 msgr2=0x7f50d00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:58.072 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.068+0000 7f50d697c700 1 -- 172.21.15.163:0/2991157884 shutdown_connections 2023-10-09T06:39:58.073 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.068+0000 7f50d697c700 1 -- 172.21.15.163:0/2991157884 wait complete. 2023-10-09T06:39:58.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.069+0000 7f50d697c700 1 Processor -- start 2023-10-09T06:39:58.075 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.069+0000 7f50d697c700 1 -- start start 2023-10-09T06:39:58.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.070+0000 7f50d697c700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50d0159890 0x7f50d00c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:58.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.070+0000 7f50d697c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50d0159c60 0x7f50d00c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:58.077 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.070+0000 7f50d697c700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50d015db10 0x7f50d015cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:58.078 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.070+0000 7f50d697c700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d00b3e30 con 0x7f50d0159c60 2023-10-09T06:39:58.078 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.070+0000 7f50d697c700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d00c8210 con 0x7f50d015db10 2023-10-09T06:39:58.078 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.070+0000 7f50d697c700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d00c84f0 con 0x7f50d0159890 2023-10-09T06:39:58.078 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.070+0000 7f50cf7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50d0159c60 0x7f50d00c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:58.078 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.070+0000 7f50cf7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50d0159c60 0x7f50d00c9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:49204/0 (socket says 172.21.15.163:49204) 2023-10-09T06:39:58.079 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.070+0000 7f50cf7fe700 1 -- 172.21.15.163:0/782489966 learned_addr learned my addr 172.21.15.163:0/782489966 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:58.079 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.070+0000 7f50cffff700 1 --2- 172.21.15.163:0/782489966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50d0159890 0x7f50d00c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:58.079 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.070+0000 7f50d4f19700 1 --2- 172.21.15.163:0/782489966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50d015db10 0x7f50d015cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:58.079 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.070+0000 7f50cf7fe700 1 -- 172.21.15.163:0/782489966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50d0159890 msgr2=0x7f50d00c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:58.079 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.070+0000 7f50cf7fe700 1 --2- 172.21.15.163:0/782489966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50d0159890 0x7f50d00c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.079 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.070+0000 7f50cf7fe700 1 -- 172.21.15.163:0/782489966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50d015db10 msgr2=0x7f50d015cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:58.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.070+0000 7f50cf7fe700 1 --2- 172.21.15.163:0/782489966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50d015db10 0x7f50d015cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.070+0000 7f50cf7fe700 1 -- 172.21.15.163:0/782489966 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c4009420 con 0x7f50d0159c60 2023-10-09T06:39:58.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.070+0000 7f50cf7fe700 1 --2- 172.21.15.163:0/782489966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50d0159c60 0x7f50d00c9620 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f50c400ec80 tx=0x7f50c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:58.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.071+0000 7f50b7fff700 1 -- 172.21.15.163:0/782489966 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c400c9d0 con 0x7f50d0159c60 2023-10-09T06:39:58.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.071+0000 7f50b7fff700 1 -- 172.21.15.163:0/782489966 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c400cb30 con 0x7f50d0159c60 2023-10-09T06:39:58.080 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.071+0000 7f50d697c700 1 -- 172.21.15.163:0/782489966 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c00094f0 con 0x7f50d0159c60 2023-10-09T06:39:58.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.071+0000 7f50b7fff700 1 -- 172.21.15.163:0/782489966 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c4013470 con 0x7f50d0159c60 2023-10-09T06:39:58.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.071+0000 7f50d697c700 1 -- 172.21.15.163:0/782489966 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d00b40d0 con 0x7f50d0159c60 2023-10-09T06:39:58.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.072+0000 7f50d697c700 1 -- 172.21.15.163:0/782489966 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d004c3c0 con 0x7f50d0159c60 2023-10-09T06:39:58.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.072+0000 7f50b7fff700 1 -- 172.21.15.163:0/782489966 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f50c4013820 con 0x7f50d0159c60 2023-10-09T06:39:58.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.073+0000 7f50b7fff700 1 --2- 172.21.15.163:0/782489966 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f50bc067320 0x7f50bc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:58.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.073+0000 7f50b7fff700 1 -- 172.21.15.163:0/782489966 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f50c4084f50 con 0x7f50d0159c60 2023-10-09T06:39:58.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.073+0000 7f50cffff700 1 --2- 172.21.15.163:0/782489966 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f50bc067320 0x7f50bc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:58.082 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.073+0000 7f50cffff700 1 --2- 172.21.15.163:0/782489966 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f50bc067320 0x7f50bc0697d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f50c00099b0 tx=0x7f50c0017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:58.083 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.077+0000 7f50b7fff700 1 -- 172.21.15.163:0/782489966 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f50c4051be0 con 0x7f50d0159c60 2023-10-09T06:39:58.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.301+0000 7f50d697c700 1 -- 172.21.15.163:0/782489966 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50d006a5b0 con 0x7f50d0159c60 2023-10-09T06:39:58.302 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.301+0000 7f50b7fff700 1 -- 172.21.15.163:0/782489966 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) v1 ==== 95+0+4619 (secure 0 0 0) 0x7f50c4021090 con 0x7f50d0159c60 2023-10-09T06:39:58.302 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:39:58.302 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":132,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7046,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3374277959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3374277959},{"type":"v1","addr":"172.21.15.181:6837","nonce":3374277959}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3834979450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3834979450},{"type":"v1","addr":"172.21.15.181:6836","nonce":3834979450}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":8991,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2681808199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2681808199},{"type":"v1","addr":"172.21.15.163:6834","nonce":2681808199}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128}],"filesystems":[{"mdsmap":{"epoch":132,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:39:52.456713+0000","modified":"2023-10-09T06:39:56.092448+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8994":{"gid":8994,"name":"a","rank":0,"incarnation":129,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/2564044136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2564044136},{"type":"v1","addr":"172.21.15.163:6835","nonce":2564044136}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-09T06:39:58.304 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.304+0000 7f50d697c700 1 -- 172.21.15.163:0/782489966 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f50bc067320 msgr2=0x7f50bc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:58.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.304+0000 7f50d697c700 1 --2- 172.21.15.163:0/782489966 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f50bc067320 0x7f50bc0697d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f50c00099b0 tx=0x7f50c0017000 comp rx=0 tx=0).stop 2023-10-09T06:39:58.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.304+0000 7f50d697c700 1 -- 172.21.15.163:0/782489966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50d0159c60 msgr2=0x7f50d00c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:58.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.304+0000 7f50d697c700 1 --2- 172.21.15.163:0/782489966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50d0159c60 0x7f50d00c9620 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f50c400ec80 tx=0x7f50c40091a0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.304+0000 7f50d697c700 1 -- 172.21.15.163:0/782489966 shutdown_connections 2023-10-09T06:39:58.305 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.304+0000 7f50d697c700 1 --2- 172.21.15.163:0/782489966 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f50bc067320 0x7f50bc0697d0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.304+0000 7f50d697c700 1 --2- 172.21.15.163:0/782489966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50d015db10 0x7f50d015cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.304+0000 7f50d697c700 1 --2- 172.21.15.163:0/782489966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50d0159890 0x7f50d00c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.304+0000 7f50d697c700 1 --2- 172.21.15.163:0/782489966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50d0159c60 0x7f50d00c9620 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.304+0000 7f50d697c700 1 -- 172.21.15.163:0/782489966 >> 172.21.15.163:0/782489966 conn(0x7f50d0068e20 msgr2=0x7f50d00c6da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:58.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.304+0000 7f50d697c700 1 -- 172.21.15.163:0/782489966 shutdown_connections 2023-10-09T06:39:58.306 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.304+0000 7f50d697c700 1 -- 172.21.15.163:0/782489966 wait complete. 2023-10-09T06:39:58.307 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 132 2023-10-09T06:39:58.367 INFO:tasks.ceph.mds.a.smithi163.stderr:2023-10-09T06:39:58.367+0000 7f7e1b350700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-09T06:39:58.720 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 132, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-09T06:39:52.456713+0000', 'modified': '2023-10-09T06:39:56.092448+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8994}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8994': {'gid': 8994, 'name': 'a', 'rank': 0, 'incarnation': 129, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.163:6835/2564044136', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6833', 'nonce': 2564044136}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 2564044136}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-09T06:39:58.720 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-09T06:39:58.721 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-09T06:39:58.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.838+0000 7f76518e9700 1 Processor -- start 2023-10-09T06:39:58.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.838+0000 7f76518e9700 1 -- start start 2023-10-09T06:39:58.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.839+0000 7f76518e9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f764c06bcd0 0x7f764c140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:58.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.839+0000 7f76518e9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f764c06c0a0 0x7f764c141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:58.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.839+0000 7f76518e9700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f764c0c8e00 0x7f764c0c9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:58.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.839+0000 7f76518e9700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f764c15c2a0 con 0x7f764c06bcd0 2023-10-09T06:39:58.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.839+0000 7f76518e9700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f764c15b690 con 0x7f764c0c8e00 2023-10-09T06:39:58.847 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.839+0000 7f76518e9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764c15b970 con 0x7f764c06c0a0 2023-10-09T06:39:58.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.839+0000 7f764affd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f764c06bcd0 0x7f764c140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:58.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.839+0000 7f764affd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f764c06bcd0 0x7f764c140df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51910/0 (socket says 172.21.15.163:51910) 2023-10-09T06:39:58.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.839+0000 7f764affd700 1 -- 172.21.15.163:0/3718470690 learned_addr learned my addr 172.21.15.163:0/3718470690 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:58.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.840+0000 7f76427fc700 1 --2- 172.21.15.163:0/3718470690 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f764c06c0a0 0x7f764c141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:58.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.840+0000 7f764b7fe700 1 --2- 172.21.15.163:0/3718470690 >> v2:172.21.15.181:3300/0 conn(0x7f764c0c8e00 0x7f764c0c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:58.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.840+0000 7f764affd700 1 -- 172.21.15.163:0/3718470690 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f764c06c0a0 msgr2=0x7f764c141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:58.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.840+0000 7f764affd700 1 --2- 172.21.15.163:0/3718470690 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f764c06c0a0 0x7f764c141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.840+0000 7f764affd700 1 -- 172.21.15.163:0/3718470690 >> v2:172.21.15.181:3300/0 conn(0x7f764c0c8e00 msgr2=0x7f764c0c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:58.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.840+0000 7f764affd700 1 --2- 172.21.15.163:0/3718470690 >> v2:172.21.15.181:3300/0 conn(0x7f764c0c8e00 0x7f764c0c9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.840+0000 7f764affd700 1 -- 172.21.15.163:0/3718470690 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c15bc50 con 0x7f764c06bcd0 2023-10-09T06:39:58.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.840+0000 7f764b7fe700 1 --2- 172.21.15.163:0/3718470690 >> v2:172.21.15.181:3300/0 conn(0x7f764c0c8e00 0x7f764c0c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:58.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.840+0000 7f764affd700 1 --2- 172.21.15.163:0/3718470690 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f764c06bcd0 0x7f764c140df0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f764c15b530 tx=0x7f763c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81628f8855f26592 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:58.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.840+0000 7f764a7fc700 1 -- 172.21.15.163:0/3718470690 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c00b860 con 0x7f764c06bcd0 2023-10-09T06:39:58.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.840+0000 7f764a7fc700 1 -- 172.21.15.163:0/3718470690 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763c00b9c0 con 0x7f764c06bcd0 2023-10-09T06:39:58.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.840+0000 7f764a7fc700 1 -- 172.21.15.163:0/3718470690 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c00bd60 con 0x7f764c06bcd0 2023-10-09T06:39:58.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.841+0000 7f76518e9700 1 -- 172.21.15.163:0/3718470690 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f764c06bcd0 msgr2=0x7f764c140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:58.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.841+0000 7f76518e9700 1 --2- 172.21.15.163:0/3718470690 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f764c06bcd0 0x7f764c140df0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f764c15b530 tx=0x7f763c009250 comp rx=0 tx=0).stop 2023-10-09T06:39:58.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.841+0000 7f76518e9700 1 -- 172.21.15.163:0/3718470690 shutdown_connections 2023-10-09T06:39:58.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.841+0000 7f76518e9700 1 --2- 172.21.15.163:0/3718470690 >> v2:172.21.15.181:3300/0 conn(0x7f764c0c8e00 0x7f764c0c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.841+0000 7f76518e9700 1 --2- 172.21.15.163:0/3718470690 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f764c06bcd0 0x7f764c140df0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.841+0000 7f76518e9700 1 --2- 172.21.15.163:0/3718470690 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f764c06c0a0 0x7f764c141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.841+0000 7f76518e9700 1 -- 172.21.15.163:0/3718470690 >> 172.21.15.163:0/3718470690 conn(0x7f764c0ce600 msgr2=0x7f764c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:58.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.841+0000 7f76518e9700 1 -- 172.21.15.163:0/3718470690 shutdown_connections 2023-10-09T06:39:58.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.841+0000 7f76518e9700 1 -- 172.21.15.163:0/3718470690 wait complete. 2023-10-09T06:39:58.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.842+0000 7f76518e9700 1 Processor -- start 2023-10-09T06:39:58.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.842+0000 7f76518e9700 1 -- start start 2023-10-09T06:39:58.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.843+0000 7f76518e9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f764c06bcd0 0x7f764c149270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:58.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.843+0000 7f76518e9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f764c06c0a0 0x7f764c0b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:58.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.843+0000 7f76518e9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f764c0c8e00 0x7f764c15d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:58.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.843+0000 7f76518e9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c13ccf0 con 0x7f764c06bcd0 2023-10-09T06:39:58.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.843+0000 7f76518e9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c0c6cd0 con 0x7f764c06c0a0 2023-10-09T06:39:58.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.843+0000 7f76518e9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764c0c63a0 con 0x7f764c0c8e00 2023-10-09T06:39:58.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.843+0000 7f764affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f764c06bcd0 0x7f764c149270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:58.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.843+0000 7f764affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f764c06bcd0 0x7f764c149270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51926/0 (socket says 172.21.15.163:51926) 2023-10-09T06:39:58.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.843+0000 7f764affd700 1 -- 172.21.15.163:0/3175224185 learned_addr learned my addr 172.21.15.163:0/3175224185 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:58.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.843+0000 7f764b7fe700 1 --2- 172.21.15.163:0/3175224185 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f764c0c8e00 0x7f764c15d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:58.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.843+0000 7f76427fc700 1 --2- 172.21.15.163:0/3175224185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f764c06c0a0 0x7f764c0b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:58.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.843+0000 7f764affd700 1 -- 172.21.15.163:0/3175224185 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f764c0c8e00 msgr2=0x7f764c15d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:58.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.843+0000 7f764affd700 1 --2- 172.21.15.163:0/3175224185 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f764c0c8e00 0x7f764c15d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.843+0000 7f764affd700 1 -- 172.21.15.163:0/3175224185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f764c06c0a0 msgr2=0x7f764c0b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:58.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.843+0000 7f764affd700 1 --2- 172.21.15.163:0/3175224185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f764c06c0a0 0x7f764c0b2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.843+0000 7f764affd700 1 -- 172.21.15.163:0/3175224185 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763c0094f0 con 0x7f764c06bcd0 2023-10-09T06:39:58.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.843+0000 7f764affd700 1 --2- 172.21.15.163:0/3175224185 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f764c06bcd0 0x7f764c149270 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f764c069220 tx=0x7f763c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:58.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.843+0000 7f76437fe700 1 -- 172.21.15.163:0/3175224185 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c024440 con 0x7f764c06bcd0 2023-10-09T06:39:58.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.843+0000 7f76437fe700 1 -- 172.21.15.163:0/3175224185 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763c0245a0 con 0x7f764c06bcd0 2023-10-09T06:39:58.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.843+0000 7f76518e9700 1 -- 172.21.15.163:0/3175224185 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764c0c6680 con 0x7f764c06bcd0 2023-10-09T06:39:58.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.843+0000 7f76437fe700 1 -- 172.21.15.163:0/3175224185 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c024960 con 0x7f764c06bcd0 2023-10-09T06:39:58.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.843+0000 7f76518e9700 1 -- 172.21.15.163:0/3175224185 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764c149830 con 0x7f764c06bcd0 2023-10-09T06:39:58.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.845+0000 7f76437fe700 1 -- 172.21.15.163:0/3175224185 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f763c01a420 con 0x7f764c06bcd0 2023-10-09T06:39:58.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.845+0000 7f76437fe700 1 --2- 172.21.15.163:0/3175224185 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f762c067320 0x7f762c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:58.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.845+0000 7f76437fe700 1 -- 172.21.15.163:0/3175224185 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f763c085e60 con 0x7f764c06bcd0 2023-10-09T06:39:58.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.845+0000 7f76427fc700 1 --2- 172.21.15.163:0/3175224185 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f762c067320 0x7f762c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:58.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.846+0000 7f76427fc700 1 --2- 172.21.15.163:0/3175224185 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f762c067320 0x7f762c0697d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f764c0b4250 tx=0x7f7634008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:58.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.847+0000 7f765abe1b80 1 Processor -- start 2023-10-09T06:39:58.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.847+0000 7f765abe1b80 1 -- start start 2023-10-09T06:39:58.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.848+0000 7f765abe1b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d24242aa50 0x55d24242ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:58.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.848+0000 7f765abe1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d242474d00 0x55d242453a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:58.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.848+0000 7f765abe1b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d242453f60 0x55d242426cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:58.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.848+0000 7f765abe1b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d242381a70 con 0x55d24242aa50 2023-10-09T06:39:58.860 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.848+0000 7f765abe1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24237f270 con 0x55d242474d00 2023-10-09T06:39:58.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.848+0000 7f765abe1b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d24237b060 con 0x55d242453f60 2023-10-09T06:39:58.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.848+0000 7f764affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d24242aa50 0x55d24242ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:58.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.848+0000 7f764affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d24242aa50 0x55d24242ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51928/0 (socket says 172.21.15.163:51928) 2023-10-09T06:39:58.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.848+0000 7f764affd700 1 -- 172.21.15.163:0/17400779 learned_addr learned my addr 172.21.15.163:0/17400779 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:58.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.848+0000 7f76427fc700 1 --2- 172.21.15.163:0/17400779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d242474d00 0x55d242453a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:58.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.848+0000 7f764b7fe700 1 --2- 172.21.15.163:0/17400779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d242453f60 0x55d242426cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:58.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.848+0000 7f764affd700 1 -- 172.21.15.163:0/17400779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d242453f60 msgr2=0x55d242426cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:58.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.848+0000 7f764affd700 1 --2- 172.21.15.163:0/17400779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d242453f60 0x55d242426cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.848+0000 7f764affd700 1 -- 172.21.15.163:0/17400779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d242474d00 msgr2=0x55d242453a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:58.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.848+0000 7f764affd700 1 --2- 172.21.15.163:0/17400779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d242474d00 0x55d242453a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.848+0000 7f764affd700 1 -- 172.21.15.163:0/17400779 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d242215270 con 0x55d24242aa50 2023-10-09T06:39:58.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.849+0000 7f764affd700 1 --2- 172.21.15.163:0/17400779 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d24242aa50 0x55d24242ae20 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f763c083d60 tx=0x7f763c006f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=442f82838586510d server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:58.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.849+0000 7f762bfff700 1 -- 172.21.15.163:0/17400779 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c056eb0 con 0x55d24242aa50 2023-10-09T06:39:58.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.849+0000 7f762bfff700 1 -- 172.21.15.163:0/17400779 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763c0567b0 con 0x55d24242aa50 2023-10-09T06:39:58.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.849+0000 7f762bfff700 1 -- 172.21.15.163:0/17400779 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c059a00 con 0x55d24242aa50 2023-10-09T06:39:58.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.849+0000 7f765abe1b80 1 -- 172.21.15.163:0/17400779 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d24242aa50 msgr2=0x55d24242ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:58.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.849+0000 7f765abe1b80 1 --2- 172.21.15.163:0/17400779 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d24242aa50 0x55d24242ae20 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f763c083d60 tx=0x7f763c006f40 comp rx=0 tx=0).stop 2023-10-09T06:39:58.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.849+0000 7f765abe1b80 1 -- 172.21.15.163:0/17400779 shutdown_connections 2023-10-09T06:39:58.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.849+0000 7f765abe1b80 1 --2- 172.21.15.163:0/17400779 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d24242aa50 0x55d24242ae20 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.849+0000 7f765abe1b80 1 --2- 172.21.15.163:0/17400779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d242453f60 0x55d242426cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.849+0000 7f765abe1b80 1 --2- 172.21.15.163:0/17400779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d242474d00 0x55d242453a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.849+0000 7f765abe1b80 1 -- 172.21.15.163:0/17400779 >> 172.21.15.163:0/17400779 conn(0x55d24242c370 msgr2=0x55d24242c740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:58.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.850+0000 7f765abe1b80 1 -- 172.21.15.163:0/17400779 shutdown_connections 2023-10-09T06:39:58.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.850+0000 7f765abe1b80 1 -- 172.21.15.163:0/17400779 wait complete. 2023-10-09T06:39:58.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.850+0000 7f765abe1b80 1 -- start start 2023-10-09T06:39:58.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.850+0000 7f765abe1b80 1 Processor -- start 2023-10-09T06:39:58.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.851+0000 7f765abe1b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d24242aa50 0x55d242450480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:58.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.851+0000 7f765abe1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d242453f60 0x55d242459c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:58.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.851+0000 7f765abe1b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d242474d00 0x55d24245a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:58.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.851+0000 7f765abe1b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d242326890 con 0x55d24242aa50 2023-10-09T06:39:58.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.851+0000 7f765abe1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d242341ea0 con 0x55d242453f60 2023-10-09T06:39:58.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.851+0000 7f765abe1b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d2421fde30 con 0x55d242474d00 2023-10-09T06:39:58.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.851+0000 7f765abe1b80 10 client.0 _ll_register_callbacks cb 0x55d2423ee110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-09T06:39:58.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.851+0000 7f764affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d24242aa50 0x55d242450480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:58.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.851+0000 7f764affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d24242aa50 0x55d242450480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51930/0 (socket says 172.21.15.163:51930) 2023-10-09T06:39:58.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.851+0000 7f764affd700 1 -- 172.21.15.163:0/1431698029 learned_addr learned my addr 172.21.15.163:0/1431698029 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:58.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.851+0000 7f76427fc700 1 --2- 172.21.15.163:0/1431698029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d242453f60 0x55d242459c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:58.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.851+0000 7f764b7fe700 1 --2- 172.21.15.163:0/1431698029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d242474d00 0x55d24245a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:58.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.851+0000 7f764affd700 1 -- 172.21.15.163:0/1431698029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d242474d00 msgr2=0x55d24245a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:58.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.851+0000 7f764affd700 1 --2- 172.21.15.163:0/1431698029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d242474d00 0x55d24245a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.851+0000 7f764affd700 1 -- 172.21.15.163:0/1431698029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d242453f60 msgr2=0x55d242459c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:58.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.851+0000 7f764affd700 1 --2- 172.21.15.163:0/1431698029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d242453f60 0x55d242459c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.851+0000 7f764affd700 1 -- 172.21.15.163:0/1431698029 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764c149830 con 0x55d24242aa50 2023-10-09T06:39:58.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.851+0000 7f764b7fe700 1 --2- 172.21.15.163:0/1431698029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d242474d00 0x55d24245a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:58.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.851+0000 7f76427fc700 1 --2- 172.21.15.163:0/1431698029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d242453f60 0x55d242459c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:39:58.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.851+0000 7f764affd700 1 --2- 172.21.15.163:0/1431698029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d24242aa50 0x55d242450480 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f763c008010 tx=0x7f763c06a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f252b2975d4ef0fe server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:58.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.851+0000 7f76297fa700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-10-09T06:39:58.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.852+0000 7f76297fa700 1 -- 172.21.15.163:0/1431698029 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c059a00 con 0x55d24242aa50 2023-10-09T06:39:58.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.852+0000 7f76297fa700 1 -- 172.21.15.163:0/1431698029 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(197..197 src has 1..197) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f763c054b50 con 0x55d24242aa50 2023-10-09T06:39:58.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.852+0000 7f765abe1b80 1 -- 172.21.15.163:0/1431698029 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d24241fb70 con 0x55d24242aa50 2023-10-09T06:39:58.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.852+0000 7f76297fa700 1 -- 172.21.15.163:0/1431698029 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=132) v2 ==== 24+0+0 (secure 0 0 0) 0x7f763c072420 con 0x55d24242aa50 2023-10-09T06:39:58.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.852+0000 7f765abe1b80 10 client.9141 fetch_fsmap learned FSMap version 132 2023-10-09T06:39:58.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.852+0000 7f765abe1b80 1 -- 172.21.15.163:0/1431698029 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=132,monmap=2+}) v3 -- 0x55d2421b7fe0 con 0x55d24242aa50 2023-10-09T06:39:58.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.852+0000 7f76297fa700 1 -- 172.21.15.163:0/1431698029 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d24241fb70 con 0x55d24242aa50 2023-10-09T06:39:58.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.852+0000 7f76297fa700 1 -- 172.21.15.163:0/1431698029 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 132) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f763c055ca0 con 0x55d24242aa50 2023-10-09T06:39:58.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.852+0000 7f765abe1b80 10 client.9141 fetch_fsmap finished waiting for FSMap version 132 2023-10-09T06:39:58.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.852+0000 7f765abe1b80 10 client.9141 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-09T06:39:58.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.852+0000 7f765abe1b80 20 client.9141 populate_metadata read hostname 'smithi163' 2023-10-09T06:39:58.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.852+0000 7f765abe1b80 1 --2- 172.21.15.163:0/1431698029 >> [v2:172.21.15.163:6833/2564044136,v1:172.21.15.163:6835/2564044136] conn(0x55d242468ca0 0x55d242469070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:58.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.852+0000 7f765abe1b80 4 client.9141 mds_command: new command op to 8994 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-09T06:39:58.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.852+0000 7f765abe1b80 1 -- 172.21.15.163:0/1431698029 --> [v2:172.21.15.163:6833/2564044136,v1:172.21.15.163:6835/2564044136] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d2423fcaa0 con 0x55d242468ca0 2023-10-09T06:39:58.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.852+0000 7f76427fc700 1 --2- 172.21.15.163:0/1431698029 >> [v2:172.21.15.163:6833/2564044136,v1:172.21.15.163:6835/2564044136] conn(0x55d242468ca0 0x55d242469070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:58.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.852+0000 7f76297fa700 10 client.9141 ms_handle_connect on v2:172.21.15.163:6833/2564044136 2023-10-09T06:39:58.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.852+0000 7f76427fc700 1 --2- 172.21.15.163:0/1431698029 >> [v2:172.21.15.163:6833/2564044136,v1:172.21.15.163:6835/2564044136] conn(0x55d242468ca0 0x55d242469070 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63349c791948d3fb server_cookie=f3cbc6183f4998cb in_seq=0 out_seq=0 2023-10-09T06:39:58.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.853+0000 7f76297fa700 1 -- 172.21.15.163:0/1431698029 <== mds.0 v2:172.21.15.163:6833/2564044136 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55d2423fcaa0 con 0x55d242468ca0 2023-10-09T06:39:58.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.853+0000 7f76297fa700 10 client.9141 handle_command_reply: tid=0 2023-10-09T06:39:58.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.853+0000 7f765abe1b80 1 client.9141 shutdown 2023-10-09T06:39:58.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.853+0000 7f765abe1b80 1 -- 172.21.15.163:0/1431698029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d24242aa50 msgr2=0x55d242450480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:58.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.853+0000 7f765abe1b80 1 --2- 172.21.15.163:0/1431698029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d24242aa50 0x55d242450480 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f763c008010 tx=0x7f763c06a000 comp rx=0 tx=0).stop 2023-10-09T06:39:58.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.853+0000 7f765abe1b80 1 -- 172.21.15.163:0/1431698029 shutdown_connections 2023-10-09T06:39:58.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.853+0000 7f765abe1b80 1 --2- 172.21.15.163:0/1431698029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d24242aa50 0x55d242450480 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.853+0000 7f765abe1b80 1 --2- 172.21.15.163:0/1431698029 >> [v2:172.21.15.163:6833/2564044136,v1:172.21.15.163:6835/2564044136] conn(0x55d242468ca0 0x55d242469070 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.853+0000 7f765abe1b80 1 --2- 172.21.15.163:0/1431698029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d242474d00 0x55d24245a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.853+0000 7f765abe1b80 1 --2- 172.21.15.163:0/1431698029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d242453f60 0x55d242459c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.853+0000 7f76297fa700 0 client.9141 ms_handle_reset on v2:172.21.15.163:6833/2564044136 2023-10-09T06:39:58.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.853+0000 7f765abe1b80 1 -- 172.21.15.163:0/1431698029 >> 172.21.15.163:0/1431698029 conn(0x55d24242c370 msgr2=0x55d242470550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:58.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.853+0000 7f765abe1b80 1 -- 172.21.15.163:0/1431698029 shutdown_connections 2023-10-09T06:39:58.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.853+0000 7f765abe1b80 1 -- 172.21.15.163:0/1431698029 wait complete. 2023-10-09T06:39:58.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.853+0000 7f765abe1b80 20 client.9141 trim_cache size 0 max 16384 2023-10-09T06:39:58.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.870+0000 7f765abe1b80 1 Processor -- start 2023-10-09T06:39:58.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.870+0000 7f765abe1b80 1 -- start start 2023-10-09T06:39:58.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.871+0000 7f765abe1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d24242aa50 0x55d242425df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:58.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.871+0000 7f765abe1b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d242453f60 0x55d2424737a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:58.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.871+0000 7f765abe1b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d242474d00 0x55d242473ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:58.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.871+0000 7f765abe1b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d242426c40 con 0x55d242474d00 2023-10-09T06:39:58.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.871+0000 7f765abe1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24246f850 con 0x55d24242aa50 2023-10-09T06:39:58.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.871+0000 7f765abe1b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d2424034b0 con 0x55d242453f60 2023-10-09T06:39:58.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.871+0000 7f764b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d242474d00 0x55d242473ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:58.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.871+0000 7f764b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d242474d00 0x55d242473ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51938/0 (socket says 172.21.15.163:51938) 2023-10-09T06:39:58.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.871+0000 7f764b7fe700 1 -- 172.21.15.163:0/2623630295 learned_addr learned my addr 172.21.15.163:0/2623630295 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:58.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.871+0000 7f764b7fe700 1 -- 172.21.15.163:0/2623630295 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d242453f60 msgr2=0x55d2424737a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:58.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.871+0000 7f764b7fe700 1 --2- 172.21.15.163:0/2623630295 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d242453f60 0x55d2424737a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.871+0000 7f764b7fe700 1 -- 172.21.15.163:0/2623630295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d24242aa50 msgr2=0x55d242425df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:58.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.871+0000 7f764affd700 1 --2- 172.21.15.163:0/2623630295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d24242aa50 0x55d242425df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:58.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.871+0000 7f764b7fe700 1 --2- 172.21.15.163:0/2623630295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d24242aa50 0x55d242425df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.871+0000 7f764b7fe700 1 -- 172.21.15.163:0/2623630295 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d242349780 con 0x55d242474d00 2023-10-09T06:39:58.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.871+0000 7f764b7fe700 1 --2- 172.21.15.163:0/2623630295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d242474d00 0x55d242473ce0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f764400f4f0 tx=0x7f7644007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5c6f1d5d96a609 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:58.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.871+0000 7f7640ff9700 1 -- 172.21.15.163:0/2623630295 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7644014070 con 0x55d242474d00 2023-10-09T06:39:58.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.871+0000 7f7640ff9700 1 -- 172.21.15.163:0/2623630295 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764400b5e0 con 0x55d242474d00 2023-10-09T06:39:58.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.871+0000 7f7640ff9700 1 -- 172.21.15.163:0/2623630295 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764400b980 con 0x55d242474d00 2023-10-09T06:39:58.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.872+0000 7f765abe1b80 1 -- 172.21.15.163:0/2623630295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d242474d00 msgr2=0x55d242473ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:58.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.872+0000 7f765abe1b80 1 --2- 172.21.15.163:0/2623630295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d242474d00 0x55d242473ce0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f764400f4f0 tx=0x7f7644007870 comp rx=0 tx=0).stop 2023-10-09T06:39:58.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.872+0000 7f765abe1b80 1 -- 172.21.15.163:0/2623630295 shutdown_connections 2023-10-09T06:39:58.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.872+0000 7f765abe1b80 1 --2- 172.21.15.163:0/2623630295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d242474d00 0x55d242473ce0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.872+0000 7f765abe1b80 1 --2- 172.21.15.163:0/2623630295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d24242aa50 0x55d242425df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.872+0000 7f765abe1b80 1 --2- 172.21.15.163:0/2623630295 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d242453f60 0x55d2424737a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.872+0000 7f765abe1b80 1 -- 172.21.15.163:0/2623630295 >> 172.21.15.163:0/2623630295 conn(0x55d24242c370 msgr2=0x55d2424234b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:58.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.872+0000 7f765abe1b80 1 -- 172.21.15.163:0/2623630295 shutdown_connections 2023-10-09T06:39:58.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.872+0000 7f765abe1b80 1 -- 172.21.15.163:0/2623630295 wait complete. 2023-10-09T06:39:58.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.872+0000 7f765abe1b80 1 -- start start 2023-10-09T06:39:58.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.872+0000 7f765abe1b80 1 Processor -- start 2023-10-09T06:39:58.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.873+0000 7f765abe1b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d24242aa50 0x55d24242ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:58.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.873+0000 7f765abe1b80 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d242453f60 0x55d24242b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:58.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.873+0000 7f765abe1b80 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d242474d00 0x55d24244e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:58.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.873+0000 7f765abe1b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24244f8d0 con 0x55d24242aa50 2023-10-09T06:39:58.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.873+0000 7f765abe1b80 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24248ce30 con 0x55d242474d00 2023-10-09T06:39:58.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.873+0000 7f765abe1b80 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d242492380 con 0x55d242453f60 2023-10-09T06:39:58.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.873+0000 7f765abe1b80 10 client.0 _ll_register_callbacks cb 0x55d2423ee110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-09T06:39:58.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.873+0000 7f764affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d24242aa50 0x55d24242ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:58.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.873+0000 7f764affd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d24242aa50 0x55d24242ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51954/0 (socket says 172.21.15.163:51954) 2023-10-09T06:39:58.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.873+0000 7f764affd700 1 -- 172.21.15.163:0/1804945280 learned_addr learned my addr 172.21.15.163:0/1804945280 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:58.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.873+0000 7f764b7fe700 1 --2- 172.21.15.163:0/1804945280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d242474d00 0x55d24244e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:58.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.874+0000 7f76427fc700 1 --2- 172.21.15.163:0/1804945280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d242453f60 0x55d24242b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:58.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.874+0000 7f764affd700 1 -- 172.21.15.163:0/1804945280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d242453f60 msgr2=0x55d24242b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:58.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.874+0000 7f764affd700 1 --2- 172.21.15.163:0/1804945280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d242453f60 0x55d24242b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.874+0000 7f764affd700 1 -- 172.21.15.163:0/1804945280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d242474d00 msgr2=0x55d24244e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:58.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.874+0000 7f764affd700 1 --2- 172.21.15.163:0/1804945280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d242474d00 0x55d24244e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.874+0000 7f764affd700 1 -- 172.21.15.163:0/1804945280 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7644007e20 con 0x55d24242aa50 2023-10-09T06:39:58.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.874+0000 7f76427fc700 1 --2- 172.21.15.163:0/1804945280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d242453f60 0x55d24242b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:39:58.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.874+0000 7f764affd700 1 --2- 172.21.15.163:0/1804945280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d24242aa50 0x55d24242ae20 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f763c007fe0 tx=0x7f763c06e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d724dc7e0d929da server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:58.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.874+0000 7f76297fa700 10 client.0 ms_handle_connect on v2:172.21.15.163:3300/0 2023-10-09T06:39:58.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.874+0000 7f76297fa700 1 -- 172.21.15.163:0/1804945280 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c072420 con 0x55d24242aa50 2023-10-09T06:39:58.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.874+0000 7f76297fa700 1 -- 172.21.15.163:0/1804945280 <== mon.0 v2:172.21.15.163:3300/0 2 ==== osd_map(197..197 src has 1..197) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f763c054b50 con 0x55d24242aa50 2023-10-09T06:39:58.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.874+0000 7f765abe1b80 1 -- 172.21.15.163:0/1804945280 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f763c0567b0 con 0x55d24242aa50 2023-10-09T06:39:58.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.874+0000 7f76297fa700 1 -- 172.21.15.163:0/1804945280 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_get_version_reply(handle=1 version=132) v2 ==== 24+0+0 (secure 0 0 0) 0x7f763c059a00 con 0x55d24242aa50 2023-10-09T06:39:58.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.874+0000 7f765abe1b80 10 client.9147 fetch_fsmap learned FSMap version 132 2023-10-09T06:39:58.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.874+0000 7f765abe1b80 1 -- 172.21.15.163:0/1804945280 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,fsmap=132,monmap=2+}) v3 -- 0x55d242330470 con 0x55d24242aa50 2023-10-09T06:39:58.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.874+0000 7f76297fa700 1 -- 172.21.15.163:0/1804945280 <== mon.0 v2:172.21.15.163:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763c0567b0 con 0x55d24242aa50 2023-10-09T06:39:58.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.874+0000 7f76297fa700 1 -- 172.21.15.163:0/1804945280 <== mon.0 v2:172.21.15.163:3300/0 5 ==== fsmap(e 132) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f763c055710 con 0x55d24242aa50 2023-10-09T06:39:58.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.874+0000 7f765abe1b80 10 client.9147 fetch_fsmap finished waiting for FSMap version 132 2023-10-09T06:39:58.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.874+0000 7f765abe1b80 10 client.9147 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-09T06:39:58.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.874+0000 7f765abe1b80 20 client.9147 populate_metadata read hostname 'smithi163' 2023-10-09T06:39:58.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.874+0000 7f765abe1b80 1 --2- 172.21.15.163:0/1804945280 >> [v2:172.21.15.163:6833/2564044136,v1:172.21.15.163:6835/2564044136] conn(0x55d242469070 0x55d24244ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:58.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.874+0000 7f765abe1b80 4 client.9147 mds_command: new command op to 8994 tid=0[{"prefix": "status"}] 2023-10-09T06:39:58.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.874+0000 7f765abe1b80 1 -- 172.21.15.163:0/1804945280 --> [v2:172.21.15.163:6833/2564044136,v1:172.21.15.163:6835/2564044136] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d24231de00 con 0x55d242469070 2023-10-09T06:39:58.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.874+0000 7f76427fc700 1 --2- 172.21.15.163:0/1804945280 >> [v2:172.21.15.163:6833/2564044136,v1:172.21.15.163:6835/2564044136] conn(0x55d242469070 0x55d24244ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:58.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.875+0000 7f76297fa700 10 client.9147 ms_handle_connect on v2:172.21.15.163:6833/2564044136 2023-10-09T06:39:58.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.875+0000 7f76427fc700 1 --2- 172.21.15.163:0/1804945280 >> [v2:172.21.15.163:6833/2564044136,v1:172.21.15.163:6835/2564044136] conn(0x55d242469070 0x55d24244ede0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7f9ef35f3aa811a server_cookie=287fc291157f1dc1 in_seq=0 out_seq=0 2023-10-09T06:39:58.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.875+0000 7f76297fa700 1 -- 172.21.15.163:0/1804945280 <== mds.0 v2:172.21.15.163:6833/2564044136 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55d24231de00 con 0x55d242469070 2023-10-09T06:39:58.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.875+0000 7f76297fa700 10 client.9147 handle_command_reply: tid=0 2023-10-09T06:39:58.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.875+0000 7f765abe1b80 1 client.9147 shutdown 2023-10-09T06:39:58.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.875+0000 7f765abe1b80 1 -- 172.21.15.163:0/1804945280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d24242aa50 msgr2=0x55d24242ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:58.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.875+0000 7f765abe1b80 1 --2- 172.21.15.163:0/1804945280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d24242aa50 0x55d24242ae20 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f763c007fe0 tx=0x7f763c06e070 comp rx=0 tx=0).stop 2023-10-09T06:39:58.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.875+0000 7f765abe1b80 1 -- 172.21.15.163:0/1804945280 shutdown_connections 2023-10-09T06:39:58.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.875+0000 7f765abe1b80 1 --2- 172.21.15.163:0/1804945280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d24242aa50 0x55d24242ae20 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.875+0000 7f765abe1b80 1 --2- 172.21.15.163:0/1804945280 >> [v2:172.21.15.163:6833/2564044136,v1:172.21.15.163:6835/2564044136] conn(0x55d242469070 0x55d24244ede0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.875+0000 7f765abe1b80 1 --2- 172.21.15.163:0/1804945280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55d242474d00 0x55d24244e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.875+0000 7f765abe1b80 1 --2- 172.21.15.163:0/1804945280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55d242453f60 0x55d24242b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.875+0000 7f76297fa700 0 client.9147 ms_handle_reset on v2:172.21.15.163:6833/2564044136 2023-10-09T06:39:58.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.875+0000 7f765abe1b80 1 -- 172.21.15.163:0/1804945280 >> 172.21.15.163:0/1804945280 conn(0x55d24242c370 msgr2=0x55d242459c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:58.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.875+0000 7f765abe1b80 1 -- 172.21.15.163:0/1804945280 shutdown_connections 2023-10-09T06:39:58.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.875+0000 7f765abe1b80 1 -- 172.21.15.163:0/1804945280 wait complete. 2023-10-09T06:39:58.885 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.875+0000 7f765abe1b80 20 client.9147 trim_cache size 0 max 16384 2023-10-09T06:39:58.885 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:39:58.885 INFO:teuthology.orchestra.run.smithi163.stdout: "cluster_fsid": "2520b8f7-539f-4bae-9f17-1fce66f66ddd", 2023-10-09T06:39:58.885 INFO:teuthology.orchestra.run.smithi163.stdout: "whoami": 0, 2023-10-09T06:39:58.886 INFO:teuthology.orchestra.run.smithi163.stdout: "id": 8994, 2023-10-09T06:39:58.886 INFO:teuthology.orchestra.run.smithi163.stdout: "want_state": "up:active", 2023-10-09T06:39:58.886 INFO:teuthology.orchestra.run.smithi163.stdout: "state": "up:active", 2023-10-09T06:39:58.886 INFO:teuthology.orchestra.run.smithi163.stdout: "fs_name": "cephfs", 2023-10-09T06:39:58.886 INFO:teuthology.orchestra.run.smithi163.stdout: "rank_uptime": 5.7889466479999996, 2023-10-09T06:39:58.886 INFO:teuthology.orchestra.run.smithi163.stdout: "mdsmap_epoch": 132, 2023-10-09T06:39:58.886 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch": 197, 2023-10-09T06:39:58.886 INFO:teuthology.orchestra.run.smithi163.stdout: "osdmap_epoch_barrier": 197, 2023-10-09T06:39:58.886 INFO:teuthology.orchestra.run.smithi163.stdout: "uptime": 10.922971201999999 2023-10-09T06:39:58.886 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:39:58.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.876+0000 7f76518e9700 1 -- 172.21.15.163:0/3175224185 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f762c067320 msgr2=0x7f762c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:58.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.876+0000 7f76518e9700 1 --2- 172.21.15.163:0/3175224185 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f762c067320 0x7f762c0697d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f764c0b4250 tx=0x7f7634008040 comp rx=0 tx=0).stop 2023-10-09T06:39:58.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.876+0000 7f76518e9700 1 -- 172.21.15.163:0/3175224185 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f764c06bcd0 msgr2=0x7f764c149270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:58.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.876+0000 7f76518e9700 1 --2- 172.21.15.163:0/3175224185 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f764c06bcd0 0x7f764c149270 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f764c069220 tx=0x7f763c017000 comp rx=0 tx=0).stop 2023-10-09T06:39:58.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.876+0000 7f76518e9700 1 -- 172.21.15.163:0/3175224185 shutdown_connections 2023-10-09T06:39:58.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.876+0000 7f76518e9700 1 --2- 172.21.15.163:0/3175224185 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f762c067320 0x7f762c0697d0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.876+0000 7f76518e9700 1 --2- 172.21.15.163:0/3175224185 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f764c06bcd0 0x7f764c149270 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.876+0000 7f76518e9700 1 --2- 172.21.15.163:0/3175224185 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f764c0c8e00 0x7f764c15d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.876+0000 7f76518e9700 1 --2- 172.21.15.163:0/3175224185 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f764c06c0a0 0x7f764c0b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:58.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.876+0000 7f76518e9700 1 -- 172.21.15.163:0/3175224185 >> 172.21.15.163:0/3175224185 conn(0x7f764c0ce600 msgr2=0x7f764c0cd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:58.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.876+0000 7f76518e9700 1 -- 172.21.15.163:0/3175224185 shutdown_connections 2023-10-09T06:39:58.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:58.876+0000 7f76518e9700 1 -- 172.21.15.163:0/3175224185 wait complete. 2023-10-09T06:39:58.889 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-09T06:39:59.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.004+0000 7fc49046d700 1 Processor -- start 2023-10-09T06:39:59.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.004+0000 7fc49046d700 1 -- start start 2023-10-09T06:39:59.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.005+0000 7fc49046d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc488159c60 0x7fc48815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:59.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.005+0000 7fc49046d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc48815db10 0x7fc488069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:59.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.005+0000 7fc49046d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc48815a160 con 0x7fc488159890 2023-10-09T06:39:59.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.005+0000 7fc49046d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc48815df50 con 0x7fc488159c60 2023-10-09T06:39:59.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.005+0000 7fc49046d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4880d13a0 con 0x7fc48815db10 2023-10-09T06:39:59.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.005+0000 7fc48da08700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc488159c60 0x7fc48815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:59.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.005+0000 7fc48da08700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc488159c60 0x7fc48815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51958/0 (socket says 172.21.15.163:51958) 2023-10-09T06:39:59.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.005+0000 7fc48da08700 1 -- 172.21.15.163:0/3963073937 learned_addr learned my addr 172.21.15.163:0/3963073937 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:59.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.005+0000 7fc48ea0a700 1 --2- 172.21.15.163:0/3963073937 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc48815db10 0x7fc488069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:59.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.006+0000 7fc48da08700 1 -- 172.21.15.163:0/3963073937 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc48815db10 msgr2=0x7fc488069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:39:59.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.006+0000 7fc48da08700 1 --2- 172.21.15.163:0/3963073937 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc48815db10 0x7fc488069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:59.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.006+0000 7fc48da08700 1 -- 172.21.15.163:0/3963073937 >> v1:172.21.15.181:6789/0 conn(0x7fc488159890 legacy=0x7fc48815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:59.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.006+0000 7fc48da08700 1 -- 172.21.15.163:0/3963073937 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48815e140 con 0x7fc488159c60 2023-10-09T06:39:59.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.006+0000 7fc48da08700 1 --2- 172.21.15.163:0/3963073937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc488159c60 0x7fc48815d5d0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fc4780099b0 tx=0x7fc478009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab81252a295bf940 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:59.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.006+0000 7fc48d207700 1 -- 172.21.15.163:0/3963073937 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc478009ea0 con 0x7fc488159c60 2023-10-09T06:39:59.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.006+0000 7fc48d207700 1 -- 172.21.15.163:0/3963073937 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47800b860 con 0x7fc488159c60 2023-10-09T06:39:59.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.006+0000 7fc48d207700 1 -- 172.21.15.163:0/3963073937 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47800bc00 con 0x7fc488159c60 2023-10-09T06:39:59.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.006+0000 7fc49046d700 1 -- 172.21.15.163:0/3963073937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc488159c60 msgr2=0x7fc48815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:59.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.006+0000 7fc49046d700 1 --2- 172.21.15.163:0/3963073937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc488159c60 0x7fc48815d5d0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fc4780099b0 tx=0x7fc478009250 comp rx=0 tx=0).stop 2023-10-09T06:39:59.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.007+0000 7fc49046d700 1 -- 172.21.15.163:0/3963073937 shutdown_connections 2023-10-09T06:39:59.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.007+0000 7fc49046d700 1 --2- 172.21.15.163:0/3963073937 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc48815db10 0x7fc488069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:59.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.007+0000 7fc49046d700 1 --2- 172.21.15.163:0/3963073937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc488159c60 0x7fc48815d5d0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:59.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.007+0000 7fc49046d700 1 -- 172.21.15.163:0/3963073937 >> 172.21.15.163:0/3963073937 conn(0x7fc488068e20 msgr2=0x7fc4880d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:59.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.007+0000 7fc49046d700 1 -- 172.21.15.163:0/3963073937 shutdown_connections 2023-10-09T06:39:59.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.007+0000 7fc49046d700 1 -- 172.21.15.163:0/3963073937 wait complete. 2023-10-09T06:39:59.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.008+0000 7fc49046d700 1 Processor -- start 2023-10-09T06:39:59.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.008+0000 7fc49046d700 1 -- start start 2023-10-09T06:39:59.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.008+0000 7fc49046d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc488159890 0x7fc4880b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:59.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.008+0000 7fc49046d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc488159c60 0x7fc4880cd770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:59.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.008+0000 7fc49046d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc48815db10 0x7fc488169ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:59.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.008+0000 7fc49046d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4880b0eb0 con 0x7fc488159890 2023-10-09T06:39:59.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.008+0000 7fc49046d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4880cdcb0 con 0x7fc48815db10 2023-10-09T06:39:59.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.008+0000 7fc49046d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc488150b00 con 0x7fc488159c60 2023-10-09T06:39:59.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.008+0000 7fc48e209700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc488159890 0x7fc4880b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:59.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.008+0000 7fc48e209700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc488159890 0x7fc4880b6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51968/0 (socket says 172.21.15.163:51968) 2023-10-09T06:39:59.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.008+0000 7fc48e209700 1 -- 172.21.15.163:0/1632917709 learned_addr learned my addr 172.21.15.163:0/1632917709 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:39:59.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.008+0000 7fc48da08700 1 --2- 172.21.15.163:0/1632917709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc488159c60 0x7fc4880cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:59.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.009+0000 7fc48ea0a700 1 --2- 172.21.15.163:0/1632917709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc48815db10 0x7fc488169ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:59.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.009+0000 7fc48e209700 1 -- 172.21.15.163:0/1632917709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc488159c60 msgr2=0x7fc4880cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:59.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.009+0000 7fc48e209700 1 --2- 172.21.15.163:0/1632917709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc488159c60 0x7fc4880cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:59.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.009+0000 7fc48e209700 1 -- 172.21.15.163:0/1632917709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc48815db10 msgr2=0x7fc488169ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:59.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.009+0000 7fc48e209700 1 --2- 172.21.15.163:0/1632917709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc48815db10 0x7fc488169ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:59.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.009+0000 7fc48e209700 1 -- 172.21.15.163:0/1632917709 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4780094f0 con 0x7fc488159890 2023-10-09T06:39:59.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.009+0000 7fc48e209700 1 --2- 172.21.15.163:0/1632917709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc488159890 0x7fc4880b6090 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fc48400cb10 tx=0x7fc4840017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:59.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.009+0000 7fc47e7fc700 1 -- 172.21.15.163:0/1632917709 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48400ce20 con 0x7fc488159890 2023-10-09T06:39:59.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.009+0000 7fc47e7fc700 1 -- 172.21.15.163:0/1632917709 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48400a7e0 con 0x7fc488159890 2023-10-09T06:39:59.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.009+0000 7fc47e7fc700 1 -- 172.21.15.163:0/1632917709 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48400aba0 con 0x7fc488159890 2023-10-09T06:39:59.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.009+0000 7fc49046d700 1 -- 172.21.15.163:0/1632917709 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4881507b0 con 0x7fc488159890 2023-10-09T06:39:59.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.009+0000 7fc49046d700 1 -- 172.21.15.163:0/1632917709 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc48816aa40 con 0x7fc488159890 2023-10-09T06:39:59.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.010+0000 7fc49046d700 1 -- 172.21.15.163:0/1632917709 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc48804c3c0 con 0x7fc488159890 2023-10-09T06:39:59.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.010+0000 7fc47e7fc700 1 -- 172.21.15.163:0/1632917709 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc484003030 con 0x7fc488159890 2023-10-09T06:39:59.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.011+0000 7fc47e7fc700 1 --2- 172.21.15.163:0/1632917709 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc46c067320 0x7fc46c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:39:59.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.011+0000 7fc47e7fc700 1 -- 172.21.15.163:0/1632917709 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc484084c50 con 0x7fc488159890 2023-10-09T06:39:59.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.011+0000 7fc48da08700 1 --2- 172.21.15.163:0/1632917709 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc46c067320 0x7fc46c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:39:59.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.012+0000 7fc48da08700 1 --2- 172.21.15.163:0/1632917709 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc46c067320 0x7fc46c0697d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc4780099b0 tx=0x7fc478017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:39:59.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.015+0000 7fc47e7fc700 1 -- 172.21.15.163:0/1632917709 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc4840518e0 con 0x7fc488159890 2023-10-09T06:39:59.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.243+0000 7fc49046d700 1 -- 172.21.15.163:0/1632917709 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc48806a5b0 con 0x7fc488159890 2023-10-09T06:39:59.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.243+0000 7fc47e7fc700 1 -- 172.21.15.163:0/1632917709 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v132) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc484054f00 con 0x7fc488159890 2023-10-09T06:39:59.243 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-09T06:39:59.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.245+0000 7fc49046d700 1 -- 172.21.15.163:0/1632917709 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc46c067320 msgr2=0x7fc46c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:59.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.245+0000 7fc49046d700 1 --2- 172.21.15.163:0/1632917709 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc46c067320 0x7fc46c0697d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc4780099b0 tx=0x7fc478017000 comp rx=0 tx=0).stop 2023-10-09T06:39:59.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.245+0000 7fc49046d700 1 -- 172.21.15.163:0/1632917709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc488159890 msgr2=0x7fc4880b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:39:59.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.245+0000 7fc49046d700 1 --2- 172.21.15.163:0/1632917709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc488159890 0x7fc4880b6090 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fc48400cb10 tx=0x7fc4840017b0 comp rx=0 tx=0).stop 2023-10-09T06:39:59.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.245+0000 7fc49046d700 1 -- 172.21.15.163:0/1632917709 shutdown_connections 2023-10-09T06:39:59.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.245+0000 7fc49046d700 1 --2- 172.21.15.163:0/1632917709 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc46c067320 0x7fc46c0697d0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:59.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.245+0000 7fc49046d700 1 --2- 172.21.15.163:0/1632917709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc488159890 0x7fc4880b6090 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:59.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.245+0000 7fc49046d700 1 --2- 172.21.15.163:0/1632917709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc48815db10 0x7fc488169ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:59.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.245+0000 7fc49046d700 1 --2- 172.21.15.163:0/1632917709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc488159c60 0x7fc4880cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:39:59.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.245+0000 7fc49046d700 1 -- 172.21.15.163:0/1632917709 >> 172.21.15.163:0/1632917709 conn(0x7fc488068e20 msgr2=0x7fc488153290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:39:59.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.245+0000 7fc49046d700 1 -- 172.21.15.163:0/1632917709 shutdown_connections 2023-10-09T06:39:59.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:39:59.245+0000 7fc49046d700 1 -- 172.21.15.163:0/1632917709 wait complete. 2023-10-09T06:39:59.257 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-09T06:39:59.257 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-09T06:39:59.257 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi163.front.sepia.ceph.com 2023-10-09T06:39:59.257 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-09T06:39:59.257 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-09T06:39:59.257 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-09T06:39:59.257 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-09T06:39:59.257 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-09T06:39:59.257 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:39:59.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:39:59.258 DEBUG:teuthology.orchestra.run.smithi163:> ip addr 2023-10-09T06:39:59.273 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:39:59.273 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:39:59.273 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:39:59.273 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:39:59.273 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2023-10-09T06:39:59.273 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:39:59.273 INFO:teuthology.orchestra.run.smithi163.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:39:59.273 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:06 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:39:59.274 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f0 2023-10-09T06:39:59.274 INFO:teuthology.orchestra.run.smithi163.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:39:59.274 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:07 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:39:59.274 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f1 2023-10-09T06:39:59.274 INFO:teuthology.orchestra.run.smithi163.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:39:59.274 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5a brd ff:ff:ff:ff:ff:ff 2023-10-09T06:39:59.274 INFO:teuthology.orchestra.run.smithi163.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-09T06:39:59.274 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5b brd ff:ff:ff:ff:ff:ff 2023-10-09T06:39:59.274 INFO:teuthology.orchestra.run.smithi163.stdout: inet 172.21.15.163/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-09T06:39:59.275 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft 41924sec preferred_lft 41924sec 2023-10-09T06:39:59.275 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::ec4:7aff:fe88:7d5b/64 scope link 2023-10-09T06:39:59.275 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:39:59.275 INFO:teuthology.orchestra.run.smithi163.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-09T06:39:59.275 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 2e:01:84:b9:f4:8e brd ff:ff:ff:ff:ff:ff 2023-10-09T06:39:59.276 INFO:teuthology.orchestra.run.smithi163.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-09T06:39:59.276 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:39:59.277 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::d03c:93ff:fe9e:c8cb/64 scope link 2023-10-09T06:39:59.277 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:39:59.277 INFO:teuthology.orchestra.run.smithi163.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:39:59.277 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 2e:01:84:b9:f4:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-09T06:39:59.277 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::2c01:84ff:feb9:f48e/64 scope link 2023-10-09T06:39:59.277 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:39:59.277 INFO:teuthology.orchestra.run.smithi163.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:39:59.277 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether c6:88:3e:d7:c1:56 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-09T06:39:59.278 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::c488:3eff:fed7:c156/64 scope link 2023-10-09T06:39:59.278 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:39:59.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:39:59.278 DEBUG:teuthology.orchestra.run.smithi163:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-09T06:39:59.336 INFO:teuthology.orchestra.run.smithi163.stdout:1 2023-10-09T06:39:59.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:39:59.337 DEBUG:teuthology.orchestra.run.smithi163:> route 2023-10-09T06:39:59.393 INFO:teuthology.orchestra.run.smithi163.stdout:Kernel IP routing table 2023-10-09T06:39:59.394 INFO:teuthology.orchestra.run.smithi163.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-09T06:39:59.394 INFO:teuthology.orchestra.run.smithi163.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-09T06:39:59.394 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-09T06:39:59.394 INFO:teuthology.orchestra.run.smithi163.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-09T06:39:59.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:39:59.394 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-09T06:39:59.394 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-09T06:39:59.394 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-09T06:39:59.395 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-09T06:39:59.395 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-09T06:39:59.395 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-09T06:39:59.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:39:59.494 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2023-10-09T06:39:59.508 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-09T06:39:59.508 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-09T06:39:59.509 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:39:59.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:39:59.509 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set brx.0 up 2023-10-09T06:39:59.573 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:39:59.573 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-09T06:39:59.628 INFO:teuthology.orchestra.run.smithi163.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-09T06:39:59.628 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:39:59.629 DEBUG:teuthology.orchestra.run.smithi163:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-09T06:39:59.683 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-09T06:39:59.683 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-09T06:39:59.683 DEBUG:teuthology.orchestra.run.smithi163:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:39:59.763 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:39:59.763 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:39:59.782 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:39:59.782 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:39:59.782 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:39:59.782 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with new device syntax: 0@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:39:59.782 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:39:59.782 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:39:59.782 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:39:59.782 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:39:59.804 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:39:59.804 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:39:59.804 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:39:59.805 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with new device syntax: 0@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:39:59.805 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:39:59.806 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:39:59.806 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:39:59.806 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:39:59.806 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790" 2023-10-09T06:39:59.806 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790:/ 2023-10-09T06:39:59.806 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=2520b8f7-539f-4bae-9f17-1fce66f66ddd" will pass to kernel 2023-10-09T06:39:59.807 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-09T06:39:59.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:39:59.807 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-09T06:39:59.873 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-09T06:40:00.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.029+0000 7fc8fd82b700 1 Processor -- start 2023-10-09T06:40:00.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.029+0000 7fc8fd82b700 1 -- start start 2023-10-09T06:40:00.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.030+0000 7fc8fd82b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8f8159420 0x7fc8f80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:00.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.030+0000 7fc8fd82b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc8f815d230 0x7fc8f815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:00.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.030+0000 7fc8fd82b700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f815dbd0 con 0x7fc8f81597f0 2023-10-09T06:40:00.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.030+0000 7fc8fd82b700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f80b2980 con 0x7fc8f815d230 2023-10-09T06:40:00.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.030+0000 7fc8fd82b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f80b11d0 con 0x7fc8f8159420 2023-10-09T06:40:00.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.030+0000 7fc8fd02a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc8f815d230 0x7fc8f815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:00.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.030+0000 7fc8fd02a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc8f815d230 0x7fc8f815d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51976/0 (socket says 172.21.15.163:51976) 2023-10-09T06:40:00.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.030+0000 7fc8fd02a700 1 -- 172.21.15.163:0/2023753839 learned_addr learned my addr 172.21.15.163:0/2023753839 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:40:00.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.030+0000 7fc8fc829700 1 --2- 172.21.15.163:0/2023753839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8f8159420 0x7fc8f80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:00.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.030+0000 7fc8fd02a700 1 -- 172.21.15.163:0/2023753839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8f8159420 msgr2=0x7fc8f80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:40:00.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.030+0000 7fc8fd02a700 1 --2- 172.21.15.163:0/2023753839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8f8159420 0x7fc8f80b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:00.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.030+0000 7fc8fd02a700 1 -- 172.21.15.163:0/2023753839 >> v1:172.21.15.181:6789/0 conn(0x7fc8f81597f0 legacy=0x7fc8f80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:00.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.030+0000 7fc8fd02a700 1 -- 172.21.15.163:0/2023753839 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f80b14b0 con 0x7fc8f815d230 2023-10-09T06:40:00.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.031+0000 7fc8fd02a700 1 --2- 172.21.15.163:0/2023753839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f815d230 0x7fc8f815d690 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00099b0 tx=0x7fc8f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46d79d2acffbb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:40:00.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.031+0000 7fc8ef7fe700 1 -- 172.21.15.163:0/2023753839 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f000b860 con 0x7fc8f815d230 2023-10-09T06:40:00.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.031+0000 7fc8ef7fe700 1 -- 172.21.15.163:0/2023753839 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f000b9c0 con 0x7fc8f815d230 2023-10-09T06:40:00.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.031+0000 7fc8ef7fe700 1 -- 172.21.15.163:0/2023753839 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f000bd60 con 0x7fc8f815d230 2023-10-09T06:40:00.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.031+0000 7fc8fd82b700 1 -- 172.21.15.163:0/2023753839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f815d230 msgr2=0x7fc8f815d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:00.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.031+0000 7fc8fd82b700 1 --2- 172.21.15.163:0/2023753839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f815d230 0x7fc8f815d690 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00099b0 tx=0x7fc8f0009250 comp rx=0 tx=0).stop 2023-10-09T06:40:00.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.031+0000 7fc8fd82b700 1 -- 172.21.15.163:0/2023753839 shutdown_connections 2023-10-09T06:40:00.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.031+0000 7fc8fd82b700 1 --2- 172.21.15.163:0/2023753839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f815d230 0x7fc8f815d690 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:00.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.031+0000 7fc8fd82b700 1 --2- 172.21.15.163:0/2023753839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8f8159420 0x7fc8f80b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:00.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.031+0000 7fc8fd82b700 1 -- 172.21.15.163:0/2023753839 >> 172.21.15.163:0/2023753839 conn(0x7fc8f80d0660 msgr2=0x7fc8f80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:40:00.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.031+0000 7fc8fd82b700 1 -- 172.21.15.163:0/2023753839 shutdown_connections 2023-10-09T06:40:00.033 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.031+0000 7fc8fd82b700 1 -- 172.21.15.163:0/2023753839 wait complete. 2023-10-09T06:40:00.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.032+0000 7fc8fd82b700 1 Processor -- start 2023-10-09T06:40:00.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.033+0000 7fc8fd82b700 1 -- start start 2023-10-09T06:40:00.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.033+0000 7fc8fd82b700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8f8159420 0x7fc8f80ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:00.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.033+0000 7fc8fd82b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f81597f0 0x7fc8f80cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:00.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.033+0000 7fc8fd82b700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8f815d230 0x7fc8f816a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:00.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.033+0000 7fc8fd82b700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f80caf70 con 0x7fc8f81597f0 2023-10-09T06:40:00.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.033+0000 7fc8fd82b700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f80cb250 con 0x7fc8f815d230 2023-10-09T06:40:00.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.033+0000 7fc8fd82b700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f80cb530 con 0x7fc8f8159420 2023-10-09T06:40:00.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.033+0000 7fc8effff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f81597f0 0x7fc8f80cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:00.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.033+0000 7fc8effff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f81597f0 0x7fc8f80cc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51982/0 (socket says 172.21.15.163:51982) 2023-10-09T06:40:00.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.033+0000 7fc8effff700 1 -- 172.21.15.163:0/535521988 learned_addr learned my addr 172.21.15.163:0/535521988 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:40:00.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.034+0000 7fc8fc829700 1 --2- 172.21.15.163:0/535521988 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8f8159420 0x7fc8f80ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:00.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.034+0000 7fc8fd02a700 1 --2- 172.21.15.163:0/535521988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8f815d230 0x7fc8f816a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:00.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.034+0000 7fc8effff700 1 -- 172.21.15.163:0/535521988 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8f8159420 msgr2=0x7fc8f80ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:00.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.034+0000 7fc8effff700 1 --2- 172.21.15.163:0/535521988 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8f8159420 0x7fc8f80ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:00.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.034+0000 7fc8effff700 1 -- 172.21.15.163:0/535521988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8f815d230 msgr2=0x7fc8f816a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:00.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.034+0000 7fc8effff700 1 --2- 172.21.15.163:0/535521988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8f815d230 0x7fc8f816a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:00.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.034+0000 7fc8effff700 1 -- 172.21.15.163:0/535521988 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f00094f0 con 0x7fc8f81597f0 2023-10-09T06:40:00.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.034+0000 7fc8fd02a700 1 --2- 172.21.15.163:0/535521988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8f815d230 0x7fc8f816a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:40:00.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.034+0000 7fc8effff700 1 --2- 172.21.15.163:0/535521988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f81597f0 0x7fc8f80cc010 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800c590 tx=0x7fc8e800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:40:00.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.034+0000 7fc8ecff9700 1 -- 172.21.15.163:0/535521988 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8011590 con 0x7fc8f81597f0 2023-10-09T06:40:00.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.034+0000 7fc8ecff9700 1 -- 172.21.15.163:0/535521988 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e80116f0 con 0x7fc8f81597f0 2023-10-09T06:40:00.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.034+0000 7fc8fd82b700 1 -- 172.21.15.163:0/535521988 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f80cb810 con 0x7fc8f81597f0 2023-10-09T06:40:00.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.034+0000 7fc8ecff9700 1 -- 172.21.15.163:0/535521988 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8011ab0 con 0x7fc8f81597f0 2023-10-09T06:40:00.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.034+0000 7fc8fd82b700 1 -- 172.21.15.163:0/535521988 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f8155920 con 0x7fc8f81597f0 2023-10-09T06:40:00.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.035+0000 7fc8fd82b700 1 -- 172.21.15.163:0/535521988 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f804c3c0 con 0x7fc8f81597f0 2023-10-09T06:40:00.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.035+0000 7fc8ecff9700 1 -- 172.21.15.163:0/535521988 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc8e801b3b0 con 0x7fc8f81597f0 2023-10-09T06:40:00.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.036+0000 7fc8ecff9700 1 --2- 172.21.15.163:0/535521988 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc8e4067320 0x7fc8e40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:00.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.036+0000 7fc8ecff9700 1 -- 172.21.15.163:0/535521988 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc8e80840a0 con 0x7fc8f81597f0 2023-10-09T06:40:00.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.036+0000 7fc8fc829700 1 --2- 172.21.15.163:0/535521988 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc8e4067320 0x7fc8e40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:00.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.036+0000 7fc8fc829700 1 --2- 172.21.15.163:0/535521988 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc8e4067320 0x7fc8e40697d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8156080 tx=0x7fc8f40097c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:40:00.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.040+0000 7fc8ecff9700 1 -- 172.21.15.163:0/535521988 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc8e8050d30 con 0x7fc8f81597f0 2023-10-09T06:40:00.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.271+0000 7fc8fd82b700 1 -- 172.21.15.163:0/535521988 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc8f815dbd0 con 0x7fc8f81597f0 2023-10-09T06:40:00.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.272+0000 7fc8ecff9700 1 -- 172.21.15.163:0/535521988 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v132) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc8f815dbd0 con 0x7fc8f81597f0 2023-10-09T06:40:00.272 INFO:teuthology.orchestra.run.smithi163.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-09T06:40:00.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.274+0000 7fc8fd82b700 1 -- 172.21.15.163:0/535521988 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc8e4067320 msgr2=0x7fc8e40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:00.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.274+0000 7fc8fd82b700 1 --2- 172.21.15.163:0/535521988 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc8e4067320 0x7fc8e40697d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8156080 tx=0x7fc8f40097c0 comp rx=0 tx=0).stop 2023-10-09T06:40:00.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.274+0000 7fc8fd82b700 1 -- 172.21.15.163:0/535521988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f81597f0 msgr2=0x7fc8f80cc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:00.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.274+0000 7fc8fd82b700 1 --2- 172.21.15.163:0/535521988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f81597f0 0x7fc8f80cc010 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800c590 tx=0x7fc8e800c950 comp rx=0 tx=0).stop 2023-10-09T06:40:00.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.274+0000 7fc8fd82b700 1 -- 172.21.15.163:0/535521988 shutdown_connections 2023-10-09T06:40:00.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.274+0000 7fc8fd82b700 1 --2- 172.21.15.163:0/535521988 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fc8e4067320 0x7fc8e40697d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:00.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.274+0000 7fc8fd82b700 1 --2- 172.21.15.163:0/535521988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8f815d230 0x7fc8f816a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:00.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.274+0000 7fc8fd82b700 1 --2- 172.21.15.163:0/535521988 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8f8159420 0x7fc8f80ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:00.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.274+0000 7fc8fd82b700 1 --2- 172.21.15.163:0/535521988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8f81597f0 0x7fc8f80cc010 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:00.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.274+0000 7fc8fd82b700 1 -- 172.21.15.163:0/535521988 >> 172.21.15.163:0/535521988 conn(0x7fc8f80d0660 msgr2=0x7fc8f81564f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:40:00.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.274+0000 7fc8fd82b700 1 -- 172.21.15.163:0/535521988 shutdown_connections 2023-10-09T06:40:00.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:00.274+0000 7fc8fd82b700 1 -- 172.21.15.163:0/535521988 wait complete. 2023-10-09T06:40:00.286 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-09T06:40:00.286 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-09T06:40:00.286 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-09T06:40:00.286 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-09T06:40:00.286 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-09T06:40:00.286 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-09T06:40:00.286 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-09T06:40:00.286 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-09T06:40:00.287 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:40:00.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:40:00.287 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-09T06:40:00.307 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-09T06:40:00.308 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-09T06:40:00.308 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-09T06:40:00.308 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:40:00.308 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-09T06:40:00.308 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:40:00.308 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:40:00.308 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:40:00.308 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2023-10-09T06:40:00.308 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:40:00.308 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-09T06:40:00.308 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2023-10-09T06:40:00.309 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-09T06:40:00.309 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-09T06:40:00.309 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-09T06:40:00.309 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-09T06:40:00.309 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-09T06:40:00.309 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 41900sec preferred_lft 41900sec 2023-10-09T06:40:00.309 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2023-10-09T06:40:00.309 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:40:00.309 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-09T06:40:00.309 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 72:62:e6:72:7c:2e brd ff:ff:ff:ff:ff:ff 2023-10-09T06:40:00.309 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-09T06:40:00.310 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:40:00.310 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::e89a:4cff:fefb:7198/64 scope link 2023-10-09T06:40:00.310 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:40:00.310 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:40:00.310 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 72:62:e6:72:7c:2e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-09T06:40:00.310 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::7062:e6ff:fe72:7c2e/64 scope link 2023-10-09T06:40:00.310 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:40:00.310 INFO:teuthology.orchestra.run.smithi181.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-09T06:40:00.310 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether fa:4e:a3:9d:5a:5f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-09T06:40:00.310 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::f84e:a3ff:fe9d:5a5f/64 scope link 2023-10-09T06:40:00.310 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-09T06:40:00.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:40:00.311 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-09T06:40:00.375 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-09T06:40:00.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:40:00.377 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-09T06:40:00.436 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-09T06:40:00.436 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-09T06:40:00.436 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-09T06:40:00.436 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-09T06:40:00.436 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-09T06:40:00.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:40:00.437 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-09T06:40:00.437 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-09T06:40:00.437 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-09T06:40:00.437 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-09T06:40:00.437 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-09T06:40:00.437 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-09T06:40:00.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:40:00.542 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-09T06:40:00.556 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-09T06:40:00.557 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-09T06:40:00.557 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:40:00.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:40:00.557 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-09T06:40:00.621 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:40:00.622 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-09T06:40:00.676 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-09T06:40:00.677 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-09T06:40:00.677 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-09T06:40:00.731 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-09T06:40:00.731 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-09T06:40:00.731 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:40:00.811 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:40:00.811 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:40:00.829 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:40:00.830 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:40:00.830 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:40:00.830 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 1@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:40:00.830 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:40:00.830 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:40:00.830 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:40:00.830 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:40:00.852 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-09T06:40:00.852 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-09T06:40:00.852 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-09T06:40:00.852 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 1@2520b8f7-539f-4bae-9f17-1fce66f66ddd.cephfs=/ 2023-10-09T06:40:00.852 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.163:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-09T06:40:00.852 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: v2 mount support check returned 2 2023-10-09T06:40:00.852 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-09T06:40:00.852 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-09T06:40:00.852 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790" 2023-10-09T06:40:00.852 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.163:6789,172.21.15.181:6789,172.21.15.181:6790:/ 2023-10-09T06:40:00.853 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=2520b8f7-539f-4bae-9f17-1fce66f66ddd" will pass to kernel 2023-10-09T06:40:00.853 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-09T06:40:00.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:40:00.854 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-09T06:40:00.918 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-09T06:40:01.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.042+0000 7f04f83bb700 1 Processor -- start 2023-10-09T06:40:01.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.042+0000 7f04f83bb700 1 -- start start 2023-10-09T06:40:01.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.043+0000 7f04f83bb700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f04f0154320 0x7f04f006d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:01.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.043+0000 7f04f83bb700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f04f015a190 0x7f04f006d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:01.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.043+0000 7f04f83bb700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04f00b4570 con 0x7f04f0153f50 2023-10-09T06:40:01.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.043+0000 7f04f83bb700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04f00b2710 con 0x7f04f015a190 2023-10-09T06:40:01.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.043+0000 7f04f83bb700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04f015a730 con 0x7f04f0154320 2023-10-09T06:40:01.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.043+0000 7f04f6958700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f04f015a190 0x7f04f006d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:01.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.043+0000 7f04f6157700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f04f0153f50 0x7f04f00b2000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:46034/0 (socket says 172.21.15.163:46034) 2023-10-09T06:40:01.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.043+0000 7f04f6157700 1 -- 172.21.15.163:0/3161751006 learned_addr learned my addr 172.21.15.163:0/3161751006 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:40:01.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.044+0000 7f04f5956700 1 --2- 172.21.15.163:0/3161751006 >> v2:172.21.15.181:3300/0 conn(0x7f04f0154320 0x7f04f006d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:01.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.044+0000 7f04f6958700 1 -- 172.21.15.163:0/3161751006 >> v2:172.21.15.181:3300/0 conn(0x7f04f0154320 msgr2=0x7f04f006d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:40:01.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.044+0000 7f04f6958700 1 --2- 172.21.15.163:0/3161751006 >> v2:172.21.15.181:3300/0 conn(0x7f04f0154320 0x7f04f006d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:01.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.044+0000 7f04f6958700 1 -- 172.21.15.163:0/3161751006 >> v1:172.21.15.163:6789/0 conn(0x7f04f0153f50 legacy=0x7f04f00b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:01.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.044+0000 7f04f6958700 1 -- 172.21.15.163:0/3161751006 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f015aa10 con 0x7f04f015a190 2023-10-09T06:40:01.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.044+0000 7f04f5155700 1 -- 172.21.15.163:0/3161751006 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3413904225 0 0) 0x7f04f00b4570 con 0x7f04f0153f50 2023-10-09T06:40:01.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.044+0000 7f04f5956700 1 --2- 172.21.15.163:0/3161751006 >> v2:172.21.15.181:3300/0 conn(0x7f04f0154320 0x7f04f006d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:40:01.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.044+0000 7f04f6958700 1 --2- 172.21.15.163:0/3161751006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04f015a190 0x7f04f006d670 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f04f00b36a0 tx=0x7f04e8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145cf29af303c858 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:40:01.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.044+0000 7f04f5155700 1 -- 172.21.15.163:0/3161751006 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e800e070 con 0x7f04f015a190 2023-10-09T06:40:01.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.044+0000 7f04f5155700 1 -- 172.21.15.163:0/3161751006 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e800bc30 con 0x7f04f015a190 2023-10-09T06:40:01.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.044+0000 7f04f5155700 1 -- 172.21.15.163:0/3161751006 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e8012430 con 0x7f04f015a190 2023-10-09T06:40:01.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.044+0000 7f04f83bb700 1 -- 172.21.15.163:0/3161751006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04f015a190 msgr2=0x7f04f006d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:01.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.044+0000 7f04f83bb700 1 --2- 172.21.15.163:0/3161751006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04f015a190 0x7f04f006d670 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f04f00b36a0 tx=0x7f04e8009780 comp rx=0 tx=0).stop 2023-10-09T06:40:01.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.045+0000 7f04f83bb700 1 -- 172.21.15.163:0/3161751006 shutdown_connections 2023-10-09T06:40:01.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.045+0000 7f04f83bb700 1 --2- 172.21.15.163:0/3161751006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04f015a190 0x7f04f006d670 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:01.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.045+0000 7f04f83bb700 1 --2- 172.21.15.163:0/3161751006 >> v2:172.21.15.181:3300/0 conn(0x7f04f0154320 0x7f04f006d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:01.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.045+0000 7f04f83bb700 1 -- 172.21.15.163:0/3161751006 >> 172.21.15.163:0/3161751006 conn(0x7f04f006baf0 msgr2=0x7f04f006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:40:01.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.045+0000 7f04f83bb700 1 -- 172.21.15.163:0/3161751006 shutdown_connections 2023-10-09T06:40:01.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.045+0000 7f04f83bb700 1 -- 172.21.15.163:0/3161751006 wait complete. 2023-10-09T06:40:01.049 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.046+0000 7f04f83bb700 1 Processor -- start 2023-10-09T06:40:01.049 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.046+0000 7f04f83bb700 1 -- start start 2023-10-09T06:40:01.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.047+0000 7f04f83bb700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04f0153f50 0x7f04f00cb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:01.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.047+0000 7f04f83bb700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04f0154320 0x7f04f00ce720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:01.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.047+0000 7f04f83bb700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04f015a190 0x7f04f0169e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:01.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.047+0000 7f04f83bb700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f00cca10 con 0x7f04f015a190 2023-10-09T06:40:01.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.047+0000 7f04f83bb700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f00ccea0 con 0x7f04f0154320 2023-10-09T06:40:01.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.047+0000 7f04f83bb700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04f00cec60 con 0x7f04f0153f50 2023-10-09T06:40:01.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.047+0000 7f04f6958700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04f015a190 0x7f04f0169e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:01.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.047+0000 7f04f6958700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04f015a190 0x7f04f0169e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:52006/0 (socket says 172.21.15.163:52006) 2023-10-09T06:40:01.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.047+0000 7f04f6958700 1 -- 172.21.15.163:0/643951282 learned_addr learned my addr 172.21.15.163:0/643951282 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:40:01.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.047+0000 7f04f6157700 1 --2- 172.21.15.163:0/643951282 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04f0153f50 0x7f04f00cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:01.050 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.047+0000 7f04f5956700 1 --2- 172.21.15.163:0/643951282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04f0154320 0x7f04f00ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:01.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.047+0000 7f04f6958700 1 -- 172.21.15.163:0/643951282 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04f0153f50 msgr2=0x7f04f00cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:01.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.047+0000 7f04f6958700 1 --2- 172.21.15.163:0/643951282 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04f0153f50 0x7f04f00cb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:01.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.047+0000 7f04f6958700 1 -- 172.21.15.163:0/643951282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04f0154320 msgr2=0x7f04f00ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:01.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.047+0000 7f04f6958700 1 --2- 172.21.15.163:0/643951282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04f0154320 0x7f04f00ce720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:01.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.047+0000 7f04f6958700 1 -- 172.21.15.163:0/643951282 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e8009a20 con 0x7f04f015a190 2023-10-09T06:40:01.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.047+0000 7f04f6157700 1 --2- 172.21.15.163:0/643951282 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04f0153f50 0x7f04f00cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:40:01.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.047+0000 7f04f5956700 1 --2- 172.21.15.163:0/643951282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04f0154320 0x7f04f00ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:40:01.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.047+0000 7f04f6958700 1 --2- 172.21.15.163:0/643951282 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04f015a190 0x7f04f0169e80 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f04f006f610 tx=0x7f04e8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:40:01.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.047+0000 7f04e67fc700 1 -- 172.21.15.163:0/643951282 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e8012e60 con 0x7f04f015a190 2023-10-09T06:40:01.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.047+0000 7f04e67fc700 1 -- 172.21.15.163:0/643951282 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e801d630 con 0x7f04f015a190 2023-10-09T06:40:01.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.048+0000 7f04f83bb700 1 -- 172.21.15.163:0/643951282 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04f00cbe00 con 0x7f04f015a190 2023-10-09T06:40:01.051 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.048+0000 7f04e67fc700 1 -- 172.21.15.163:0/643951282 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e801d9f0 con 0x7f04f015a190 2023-10-09T06:40:01.052 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.048+0000 7f04f83bb700 1 -- 172.21.15.163:0/643951282 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04f00cc3c0 con 0x7f04f015a190 2023-10-09T06:40:01.052 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.048+0000 7f04f83bb700 1 -- 172.21.15.163:0/643951282 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04f00718b0 con 0x7f04f015a190 2023-10-09T06:40:01.052 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.049+0000 7f04e67fc700 1 -- 172.21.15.163:0/643951282 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f04e8024070 con 0x7f04f015a190 2023-10-09T06:40:01.052 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.049+0000 7f04e67fc700 1 --2- 172.21.15.163:0/643951282 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f04d4067320 0x7f04d40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:01.052 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.049+0000 7f04e67fc700 1 -- 172.21.15.163:0/643951282 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f04e80874c0 con 0x7f04f015a190 2023-10-09T06:40:01.052 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.049+0000 7f04f6157700 1 --2- 172.21.15.163:0/643951282 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f04d4067320 0x7f04d40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:01.052 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.050+0000 7f04f6157700 1 --2- 172.21.15.163:0/643951282 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f04d4067320 0x7f04d40697d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f04f015f3b0 tx=0x7f04ec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:40:01.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.053+0000 7f04e67fc700 1 -- 172.21.15.163:0/643951282 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f04e8054200 con 0x7f04f015a190 2023-10-09T06:40:01.285 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:01.285+0000 7f04f83bb700 1 -- 172.21.15.163:0/643951282 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f04f00ccb50 con 0x7f04f015a190 2023-10-09T06:40:02.127 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.127+0000 7f04e67fc700 1 -- 172.21.15.163:0/643951282 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v133) v1 ==== 136+0+0 (secure 0 0 0) 0x7f04e8057820 con 0x7f04f015a190 2023-10-09T06:40:02.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.130+0000 7f04f83bb700 1 -- 172.21.15.163:0/643951282 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f04d4067320 msgr2=0x7f04d40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:02.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.130+0000 7f04f83bb700 1 --2- 172.21.15.163:0/643951282 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f04d4067320 0x7f04d40697d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f04f015f3b0 tx=0x7f04ec009040 comp rx=0 tx=0).stop 2023-10-09T06:40:02.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.130+0000 7f04f83bb700 1 -- 172.21.15.163:0/643951282 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04f015a190 msgr2=0x7f04f0169e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:02.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.130+0000 7f04f83bb700 1 --2- 172.21.15.163:0/643951282 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04f015a190 0x7f04f0169e80 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f04f006f610 tx=0x7f04e8019000 comp rx=0 tx=0).stop 2023-10-09T06:40:02.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.130+0000 7f04f83bb700 1 -- 172.21.15.163:0/643951282 shutdown_connections 2023-10-09T06:40:02.130 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.130+0000 7f04f83bb700 1 --2- 172.21.15.163:0/643951282 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f04d4067320 0x7f04d40697d0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:02.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.130+0000 7f04f83bb700 1 --2- 172.21.15.163:0/643951282 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04f015a190 0x7f04f0169e80 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:02.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.130+0000 7f04f83bb700 1 --2- 172.21.15.163:0/643951282 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04f0153f50 0x7f04f00cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:02.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.130+0000 7f04f83bb700 1 --2- 172.21.15.163:0/643951282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04f0154320 0x7f04f00ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:02.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.130+0000 7f04f83bb700 1 -- 172.21.15.163:0/643951282 >> 172.21.15.163:0/643951282 conn(0x7f04f006baf0 msgr2=0x7f04f01530c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:40:02.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.130+0000 7f04f83bb700 1 -- 172.21.15.163:0/643951282 shutdown_connections 2023-10-09T06:40:02.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.130+0000 7f04f83bb700 1 -- 172.21.15.163:0/643951282 wait complete. 2023-10-09T06:40:02.144 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-09T06:40:02.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.263+0000 7f50a6133700 1 Processor -- start 2023-10-09T06:40:02.264 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.263+0000 7f50a6133700 1 -- start start 2023-10-09T06:40:02.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.264+0000 7f50a6133700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f50a006bcd0 0x7f50a00caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:02.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.264+0000 7f50a6133700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50a006c0a0 0x7f50a015a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:02.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.264+0000 7f50a6133700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f50a015df50 0x7f50a015e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:02.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.264+0000 7f50a6133700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50a015d7e0 con 0x7f50a006bcd0 2023-10-09T06:40:02.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.264+0000 7f50a6133700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50a015dac0 con 0x7f50a015df50 2023-10-09T06:40:02.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.264+0000 7f50a6133700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a015aa30 con 0x7f50a006c0a0 2023-10-09T06:40:02.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.264+0000 7f509f7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f50a006bcd0 0x7f50a00caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:02.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.264+0000 7f509f7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f50a006bcd0 0x7f50a00caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:52014/0 (socket says 172.21.15.163:52014) 2023-10-09T06:40:02.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.264+0000 7f509f7fe700 1 -- 172.21.15.163:0/3459240681 learned_addr learned my addr 172.21.15.163:0/3459240681 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:40:02.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.264+0000 7f509ffff700 1 --2- 172.21.15.163:0/3459240681 >> v2:172.21.15.181:3300/0 conn(0x7f50a015df50 0x7f50a015e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:02.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.264+0000 7f509effd700 1 --2- 172.21.15.163:0/3459240681 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50a006c0a0 0x7f50a015a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:02.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.265+0000 7f509f7fe700 1 -- 172.21.15.163:0/3459240681 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50a006c0a0 msgr2=0x7f50a015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:40:02.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.265+0000 7f509f7fe700 1 --2- 172.21.15.163:0/3459240681 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50a006c0a0 0x7f50a015a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:02.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.265+0000 7f509f7fe700 1 -- 172.21.15.163:0/3459240681 >> v2:172.21.15.181:3300/0 conn(0x7f50a015df50 msgr2=0x7f50a015e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:40:02.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.265+0000 7f509f7fe700 1 --2- 172.21.15.163:0/3459240681 >> v2:172.21.15.181:3300/0 conn(0x7f50a015df50 0x7f50a015e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:02.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.265+0000 7f509f7fe700 1 -- 172.21.15.163:0/3459240681 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a015ad10 con 0x7f50a006bcd0 2023-10-09T06:40:02.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.265+0000 7f509f7fe700 1 --2- 172.21.15.163:0/3459240681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50a006bcd0 0x7f50a00caa90 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f50a00b4600 tx=0x7f5090009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e98a43d542b15c42 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:40:02.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.265+0000 7f509e7fc700 1 -- 172.21.15.163:0/3459240681 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509000e070 con 0x7f50a006bcd0 2023-10-09T06:40:02.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.265+0000 7f509e7fc700 1 -- 172.21.15.163:0/3459240681 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509000ba50 con 0x7f50a006bcd0 2023-10-09T06:40:02.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.265+0000 7f509e7fc700 1 -- 172.21.15.163:0/3459240681 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50900092e0 con 0x7f50a006bcd0 2023-10-09T06:40:02.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.265+0000 7f50a6133700 1 -- 172.21.15.163:0/3459240681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50a006bcd0 msgr2=0x7f50a00caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:02.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.265+0000 7f50a6133700 1 --2- 172.21.15.163:0/3459240681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50a006bcd0 0x7f50a00caa90 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f50a00b4600 tx=0x7f5090009840 comp rx=0 tx=0).stop 2023-10-09T06:40:02.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.265+0000 7f50a6133700 1 -- 172.21.15.163:0/3459240681 shutdown_connections 2023-10-09T06:40:02.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.265+0000 7f50a6133700 1 --2- 172.21.15.163:0/3459240681 >> v2:172.21.15.181:3300/0 conn(0x7f50a015df50 0x7f50a015e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:02.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.265+0000 7f50a6133700 1 --2- 172.21.15.163:0/3459240681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50a006bcd0 0x7f50a00caa90 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:02.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.265+0000 7f50a6133700 1 --2- 172.21.15.163:0/3459240681 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50a006c0a0 0x7f50a015a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:02.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.265+0000 7f50a6133700 1 -- 172.21.15.163:0/3459240681 >> 172.21.15.163:0/3459240681 conn(0x7f50a00ce600 msgr2=0x7f50a00cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:40:02.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.265+0000 7f50a6133700 1 -- 172.21.15.163:0/3459240681 shutdown_connections 2023-10-09T06:40:02.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.265+0000 7f50a6133700 1 -- 172.21.15.163:0/3459240681 wait complete. 2023-10-09T06:40:02.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.267+0000 7f50a6133700 1 Processor -- start 2023-10-09T06:40:02.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.267+0000 7f50a6133700 1 -- start start 2023-10-09T06:40:02.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.267+0000 7f50a6133700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50a006bcd0 0x7f50a00c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:02.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.267+0000 7f50a6133700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50a006c0a0 0x7f50a00c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:02.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.267+0000 7f50a6133700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50a015df50 0x7f50a00c6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:02.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.267+0000 7f50a6133700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a016ac80 con 0x7f50a006bcd0 2023-10-09T06:40:02.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.267+0000 7f50a6133700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a016b010 con 0x7f50a006c0a0 2023-10-09T06:40:02.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.267+0000 7f50a6133700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a016b2f0 con 0x7f50a015df50 2023-10-09T06:40:02.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.267+0000 7f509f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50a006bcd0 0x7f50a00c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:02.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.267+0000 7f509f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50a006bcd0 0x7f50a00c9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:52022/0 (socket says 172.21.15.163:52022) 2023-10-09T06:40:02.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.268+0000 7f509f7fe700 1 -- 172.21.15.163:0/3878270669 learned_addr learned my addr 172.21.15.163:0/3878270669 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:40:02.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.268+0000 7f509ffff700 1 --2- 172.21.15.163:0/3878270669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50a015df50 0x7f50a00c6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:02.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.268+0000 7f509effd700 1 --2- 172.21.15.163:0/3878270669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50a006c0a0 0x7f50a00c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:02.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.268+0000 7f509f7fe700 1 -- 172.21.15.163:0/3878270669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50a015df50 msgr2=0x7f50a00c6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:02.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.268+0000 7f509f7fe700 1 --2- 172.21.15.163:0/3878270669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50a015df50 0x7f50a00c6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:02.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.268+0000 7f509f7fe700 1 -- 172.21.15.163:0/3878270669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50a006c0a0 msgr2=0x7f50a00c67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:02.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.268+0000 7f509f7fe700 1 --2- 172.21.15.163:0/3878270669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50a006c0a0 0x7f50a00c67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:02.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.268+0000 7f509f7fe700 1 -- 172.21.15.163:0/3878270669 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5090009ae0 con 0x7f50a006bcd0 2023-10-09T06:40:02.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.268+0000 7f509f7fe700 1 --2- 172.21.15.163:0/3878270669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50a006bcd0 0x7f50a00c9110 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f5090009fd0 tx=0x7f5090018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:40:02.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.268+0000 7f50877fe700 1 -- 172.21.15.163:0/3878270669 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5090007190 con 0x7f50a006bcd0 2023-10-09T06:40:02.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.268+0000 7f50877fe700 1 -- 172.21.15.163:0/3878270669 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5090007820 con 0x7f50a006bcd0 2023-10-09T06:40:02.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.268+0000 7f50a6133700 1 -- 172.21.15.163:0/3878270669 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a016b5d0 con 0x7f50a006bcd0 2023-10-09T06:40:02.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.268+0000 7f50877fe700 1 -- 172.21.15.163:0/3878270669 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5090007be0 con 0x7f50a006bcd0 2023-10-09T06:40:02.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.268+0000 7f50a6133700 1 -- 172.21.15.163:0/3878270669 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a00c9650 con 0x7f50a006bcd0 2023-10-09T06:40:02.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.269+0000 7f50a6133700 1 -- 172.21.15.163:0/3878270669 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a004c3c0 con 0x7f50a006bcd0 2023-10-09T06:40:02.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.269+0000 7f50877fe700 1 -- 172.21.15.163:0/3878270669 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5090025460 con 0x7f50a006bcd0 2023-10-09T06:40:02.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.270+0000 7f50877fe700 1 --2- 172.21.15.163:0/3878270669 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f508c067320 0x7f508c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:02.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.270+0000 7f50877fe700 1 -- 172.21.15.163:0/3878270669 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5090086670 con 0x7f50a006bcd0 2023-10-09T06:40:02.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.270+0000 7f509effd700 1 --2- 172.21.15.163:0/3878270669 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f508c067320 0x7f508c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:02.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.270+0000 7f509effd700 1 --2- 172.21.15.163:0/3878270669 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f508c067320 0x7f508c0697d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f50a0064e30 tx=0x7f5094008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:40:02.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.274+0000 7f50877fe700 1 -- 172.21.15.163:0/3878270669 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f50900531f0 con 0x7f50a006bcd0 2023-10-09T06:40:02.482 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.482+0000 7f50a6133700 1 -- 172.21.15.163:0/3878270669 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f50a015aff0 con 0x7f508c067320 2023-10-09T06:40:02.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.484+0000 7f50877fe700 1 -- 172.21.15.163:0/3878270669 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f50a015aff0 con 0x7f508c067320 2023-10-09T06:40:02.484 INFO:teuthology.orchestra.run.smithi163.stdout:[ 2023-10-09T06:40:02.484 INFO:teuthology.orchestra.run.smithi163.stdout: { 2023-10-09T06:40:02.485 INFO:teuthology.orchestra.run.smithi163.stdout: "name": "cephfs" 2023-10-09T06:40:02.485 INFO:teuthology.orchestra.run.smithi163.stdout: } 2023-10-09T06:40:02.485 INFO:teuthology.orchestra.run.smithi163.stdout:] 2023-10-09T06:40:02.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.486+0000 7f50a6133700 1 -- 172.21.15.163:0/3878270669 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f508c067320 msgr2=0x7f508c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:02.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.486+0000 7f50a6133700 1 --2- 172.21.15.163:0/3878270669 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f508c067320 0x7f508c0697d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f50a0064e30 tx=0x7f5094008040 comp rx=0 tx=0).stop 2023-10-09T06:40:02.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.486+0000 7f50a6133700 1 -- 172.21.15.163:0/3878270669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50a006bcd0 msgr2=0x7f50a00c9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:02.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.486+0000 7f50a6133700 1 --2- 172.21.15.163:0/3878270669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50a006bcd0 0x7f50a00c9110 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f5090009fd0 tx=0x7f5090018000 comp rx=0 tx=0).stop 2023-10-09T06:40:02.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.486+0000 7f50a6133700 1 -- 172.21.15.163:0/3878270669 shutdown_connections 2023-10-09T06:40:02.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.486+0000 7f50a6133700 1 --2- 172.21.15.163:0/3878270669 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f508c067320 0x7f508c0697d0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:02.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.486+0000 7f50a6133700 1 --2- 172.21.15.163:0/3878270669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50a006bcd0 0x7f50a00c9110 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:02.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.486+0000 7f50a6133700 1 --2- 172.21.15.163:0/3878270669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50a015df50 0x7f50a00c6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:02.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.486+0000 7f50a6133700 1 --2- 172.21.15.163:0/3878270669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50a006c0a0 0x7f50a00c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:02.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.486+0000 7f50a6133700 1 -- 172.21.15.163:0/3878270669 >> 172.21.15.163:0/3878270669 conn(0x7f50a00ce600 msgr2=0x7f50a00c33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:40:02.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.487+0000 7f50a6133700 1 -- 172.21.15.163:0/3878270669 shutdown_connections 2023-10-09T06:40:02.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.487+0000 7f50a6133700 1 -- 172.21.15.163:0/3878270669 wait complete. 2023-10-09T06:40:02.499 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-09T06:40:02.616 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.616+0000 7faecd839700 1 Processor -- start 2023-10-09T06:40:02.617 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.616+0000 7faecd839700 1 -- start start 2023-10-09T06:40:02.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.617+0000 7faecd839700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faec80caa90 0x7faec80c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:02.619 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.617+0000 7faecd839700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faec815d7e0 con 0x7faec806c0a0 2023-10-09T06:40:02.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.617+0000 7faecd839700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faec815dac0 con 0x7faec806bcd0 2023-10-09T06:40:02.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.617+0000 7faecd839700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec80caef0 con 0x7faec80caa90 2023-10-09T06:40:02.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.617+0000 7faebe7fc700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7faec806c0a0 0x7faec815df50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:46042/0 (socket says 172.21.15.163:46042) 2023-10-09T06:40:02.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.617+0000 7faebe7fc700 1 -- 172.21.15.163:0/3232961326 learned_addr learned my addr 172.21.15.163:0/3232961326 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:40:02.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.618+0000 7faec77fe700 1 --2- 172.21.15.163:0/3232961326 >> v2:172.21.15.181:3300/0 conn(0x7faec80caa90 0x7faec80c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:02.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.618+0000 7faec67fc700 1 -- 172.21.15.163:0/3232961326 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1671781428 0 0) 0x7faec815d7e0 con 0x7faec806c0a0 2023-10-09T06:40:02.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.618+0000 7faec67fc700 1 -- 172.21.15.163:0/3232961326 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faeb4003570 con 0x7faec806c0a0 2023-10-09T06:40:02.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.618+0000 7faec67fc700 1 -- 172.21.15.163:0/3232961326 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3482027120 0 0) 0x7faeb4003570 con 0x7faec806c0a0 2023-10-09T06:40:02.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.618+0000 7faec67fc700 1 -- 172.21.15.163:0/3232961326 >> v2:172.21.15.181:3300/0 conn(0x7faec80caa90 msgr2=0x7faec80c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:40:02.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.618+0000 7faec67fc700 1 --2- 172.21.15.163:0/3232961326 >> v2:172.21.15.181:3300/0 conn(0x7faec80caa90 0x7faec80c8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:02.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.618+0000 7faec67fc700 1 -- 172.21.15.163:0/3232961326 >> v1:172.21.15.181:6789/0 conn(0x7faec806bcd0 legacy=0x7faec815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:02.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.618+0000 7faec67fc700 1 -- 172.21.15.163:0/3232961326 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec815aa30 con 0x7faec806c0a0 2023-10-09T06:40:02.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.618+0000 7faec67fc700 1 -- 172.21.15.163:0/3232961326 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7faeb0001470 con 0x7faec806c0a0 2023-10-09T06:40:02.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.618+0000 7faec67fc700 1 -- 172.21.15.163:0/3232961326 >> v1:172.21.15.163:6789/0 conn(0x7faec806c0a0 legacy=0x7faec815df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:02.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.618+0000 7faec67fc700 1 --2- 172.21.15.163:0/3232961326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faeb4003a60 0x7faeb4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:02.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.618+0000 7faec67fc700 1 --2- 172.21.15.163:0/3232961326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb4004810 0x7faeb4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:02.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.618+0000 7faec67fc700 1 --2- 172.21.15.163:0/3232961326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faeb40052e0 0x7faeb400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:02.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.618+0000 7faec67fc700 1 -- 172.21.15.163:0/3232961326 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec80caef0 con 0x7faeb4004810 2023-10-09T06:40:02.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.618+0000 7faec67fc700 1 -- 172.21.15.163:0/3232961326 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb400c2a0 con 0x7faeb4003a60 2023-10-09T06:40:02.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.618+0000 7faec67fc700 1 -- 172.21.15.163:0/3232961326 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb400c460 con 0x7faeb40052e0 2023-10-09T06:40:02.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.618+0000 7faebe7fc700 1 --2- 172.21.15.163:0/3232961326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb4004810 0x7faeb4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:02.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.618+0000 7faec6ffd700 1 --2- 172.21.15.163:0/3232961326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faeb4003a60 0x7faeb4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:02.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.618+0000 7faec77fe700 1 --2- 172.21.15.163:0/3232961326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faeb40052e0 0x7faeb400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:02.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.618+0000 7faebe7fc700 1 -- 172.21.15.163:0/3232961326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faeb40052e0 msgr2=0x7faeb400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:40:02.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.618+0000 7faebe7fc700 1 --2- 172.21.15.163:0/3232961326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faeb40052e0 0x7faeb400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:02.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.618+0000 7faebe7fc700 1 -- 172.21.15.163:0/3232961326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faeb4003a60 msgr2=0x7faeb4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:40:02.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.618+0000 7faebe7fc700 1 --2- 172.21.15.163:0/3232961326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faeb4003a60 0x7faeb4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:02.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.618+0000 7faebe7fc700 1 -- 172.21.15.163:0/3232961326 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb4003570 con 0x7faeb4004810 2023-10-09T06:40:02.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.618+0000 7faec77fe700 1 -- 172.21.15.163:0/3232961326 reap_dead start 2023-10-09T06:40:02.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.619+0000 7faebe7fc700 1 --2- 172.21.15.163:0/3232961326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb4004810 0x7faeb4004c80 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7faeb000c390 tx=0x7faec815da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1908de2e46426046 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:40:02.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.619+0000 7faec67fc700 1 -- 172.21.15.163:0/3232961326 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb000cbe0 con 0x7faeb4004810 2023-10-09T06:40:02.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.619+0000 7faec67fc700 1 -- 172.21.15.163:0/3232961326 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb000cd40 con 0x7faeb4004810 2023-10-09T06:40:02.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.619+0000 7faec67fc700 1 -- 172.21.15.163:0/3232961326 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb0005b70 con 0x7faeb4004810 2023-10-09T06:40:02.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.619+0000 7faecd839700 1 -- 172.21.15.163:0/3232961326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb4004810 msgr2=0x7faeb4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:02.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.619+0000 7faecd839700 1 --2- 172.21.15.163:0/3232961326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb4004810 0x7faeb4004c80 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7faeb000c390 tx=0x7faec815da30 comp rx=0 tx=0).stop 2023-10-09T06:40:02.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.619+0000 7faecd839700 1 -- 172.21.15.163:0/3232961326 shutdown_connections 2023-10-09T06:40:02.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.619+0000 7faecd839700 1 --2- 172.21.15.163:0/3232961326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb4004810 0x7faeb4004c80 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:02.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.619+0000 7faecd839700 1 -- 172.21.15.163:0/3232961326 >> 172.21.15.163:0/3232961326 conn(0x7faec80ce600 msgr2=0x7faec80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:40:02.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.619+0000 7faecd839700 1 -- 172.21.15.163:0/3232961326 shutdown_connections 2023-10-09T06:40:02.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.619+0000 7faecd839700 1 -- 172.21.15.163:0/3232961326 wait complete. 2023-10-09T06:40:02.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.621+0000 7faecd839700 1 Processor -- start 2023-10-09T06:40:02.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.621+0000 7faecd839700 1 -- start start 2023-10-09T06:40:02.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.621+0000 7faecd839700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb4004810 0x7faec80c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:02.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.621+0000 7faecd839700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faec80b13a0 0x7faec816a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:02.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.621+0000 7faecd839700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faec80b1810 0x7faec816aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:02.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.621+0000 7faecd839700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec80b3250 con 0x7faeb4004810 2023-10-09T06:40:02.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.621+0000 7faecd839700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec80b1d10 con 0x7faec80b1810 2023-10-09T06:40:02.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.621+0000 7faecd839700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec80b1ff0 con 0x7faec80b13a0 2023-10-09T06:40:02.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.621+0000 7faec6ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb4004810 0x7faec80c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:02.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.621+0000 7faec6ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb4004810 0x7faec80c56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:52026/0 (socket says 172.21.15.163:52026) 2023-10-09T06:40:02.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.621+0000 7faec6ffd700 1 -- 172.21.15.163:0/2286325580 learned_addr learned my addr 172.21.15.163:0/2286325580 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:40:02.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.622+0000 7faebe7fc700 1 --2- 172.21.15.163:0/2286325580 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faec80b13a0 0x7faec816a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:02.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.622+0000 7faec77fe700 1 --2- 172.21.15.163:0/2286325580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faec80b1810 0x7faec816aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:02.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.622+0000 7faec6ffd700 1 -- 172.21.15.163:0/2286325580 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faec80b13a0 msgr2=0x7faec816a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:02.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.622+0000 7faec6ffd700 1 --2- 172.21.15.163:0/2286325580 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faec80b13a0 0x7faec816a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:02.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.622+0000 7faec6ffd700 1 -- 172.21.15.163:0/2286325580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faec80b1810 msgr2=0x7faec816aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:02.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.622+0000 7faec6ffd700 1 --2- 172.21.15.163:0/2286325580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faec80b1810 0x7faec816aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:02.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.622+0000 7faec6ffd700 1 -- 172.21.15.163:0/2286325580 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec80b22d0 con 0x7faeb4004810 2023-10-09T06:40:02.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.622+0000 7faebe7fc700 1 --2- 172.21.15.163:0/2286325580 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faec80b13a0 0x7faec816a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:40:02.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.622+0000 7faec6ffd700 1 --2- 172.21.15.163:0/2286325580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb4004810 0x7faec80c56b0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7faeb8002950 tx=0x7faeb8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:40:02.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.622+0000 7faebf7fe700 1 -- 172.21.15.163:0/2286325580 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb800f850 con 0x7faeb4004810 2023-10-09T06:40:02.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.622+0000 7faebf7fe700 1 -- 172.21.15.163:0/2286325580 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb800f9b0 con 0x7faeb4004810 2023-10-09T06:40:02.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.622+0000 7faebf7fe700 1 -- 172.21.15.163:0/2286325580 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb800fd50 con 0x7faeb4004810 2023-10-09T06:40:02.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.622+0000 7faecd839700 1 -- 172.21.15.163:0/2286325580 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec815bbf0 con 0x7faeb4004810 2023-10-09T06:40:02.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.622+0000 7faecd839700 1 -- 172.21.15.163:0/2286325580 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec815c060 con 0x7faeb4004810 2023-10-09T06:40:02.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.623+0000 7faecd839700 1 -- 172.21.15.163:0/2286325580 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec804c3c0 con 0x7faeb4004810 2023-10-09T06:40:02.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.623+0000 7faebf7fe700 1 -- 172.21.15.163:0/2286325580 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7faeb8013600 con 0x7faeb4004810 2023-10-09T06:40:02.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.624+0000 7faebf7fe700 1 --2- 172.21.15.163:0/2286325580 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faea8067350 0x7faea8069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:02.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.624+0000 7faebf7fe700 1 -- 172.21.15.163:0/2286325580 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5705+0+0 (secure 0 0 0) 0x7faeb8085110 con 0x7faeb4004810 2023-10-09T06:40:02.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.624+0000 7faebe7fc700 1 --2- 172.21.15.163:0/2286325580 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faea8067350 0x7faea8069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:02.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.624+0000 7faebe7fc700 1 --2- 172.21.15.163:0/2286325580 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faea8067350 0x7faea8069800 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7faec815c7a0 tx=0x7faeb0015000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:40:02.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.628+0000 7faebf7fe700 1 -- 172.21.15.163:0/2286325580 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faeb8051da0 con 0x7faeb4004810 2023-10-09T06:40:02.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.808+0000 7faecd839700 1 -- 172.21.15.163:0/2286325580 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7faec815ad10 con 0x7faeb4004810 2023-10-09T06:40:02.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.809+0000 7faebf7fe700 1 -- 172.21.15.163:0/2286325580 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7faeb80553c0 con 0x7faeb4004810 2023-10-09T06:40:02.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.811+0000 7faecd839700 1 -- 172.21.15.163:0/2286325580 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faea8067350 msgr2=0x7faea8069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:02.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.811+0000 7faecd839700 1 --2- 172.21.15.163:0/2286325580 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faea8067350 0x7faea8069800 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7faec815c7a0 tx=0x7faeb0015000 comp rx=0 tx=0).stop 2023-10-09T06:40:02.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.811+0000 7faecd839700 1 -- 172.21.15.163:0/2286325580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb4004810 msgr2=0x7faec80c56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:02.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.811+0000 7faecd839700 1 --2- 172.21.15.163:0/2286325580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb4004810 0x7faec80c56b0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7faeb8002950 tx=0x7faeb8007d30 comp rx=0 tx=0).stop 2023-10-09T06:40:02.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.811+0000 7faecd839700 1 -- 172.21.15.163:0/2286325580 shutdown_connections 2023-10-09T06:40:02.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.811+0000 7faecd839700 1 --2- 172.21.15.163:0/2286325580 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faea8067350 0x7faea8069800 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:02.811 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.811+0000 7faecd839700 1 --2- 172.21.15.163:0/2286325580 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb4004810 0x7faec80c56b0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:02.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.811+0000 7faecd839700 1 --2- 172.21.15.163:0/2286325580 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faec80b1810 0x7faec816aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:02.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.811+0000 7faecd839700 1 --2- 172.21.15.163:0/2286325580 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faec80b13a0 0x7faec816a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:02.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.811+0000 7faecd839700 1 -- 172.21.15.163:0/2286325580 >> 172.21.15.163:0/2286325580 conn(0x7faec80ce600 msgr2=0x7faec8159e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:40:02.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.811+0000 7faecd839700 1 -- 172.21.15.163:0/2286325580 shutdown_connections 2023-10-09T06:40:02.812 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:02.811+0000 7faecd839700 1 -- 172.21.15.163:0/2286325580 wait complete. 2023-10-09T06:40:02.823 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-09T06:40:02.823 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:40:02.823 DEBUG:teuthology.orchestra.run.smithi163:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-09T06:40:02.860 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi163.front.sepia.ceph.com 2023-10-09T06:40:02.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:40:02.860 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-09T06:40:02.915 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-09T06:40:02.915 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-09T06:40:02.916 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-09T06:40:02.955 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-09T06:40:02.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:40:02.955 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-09T06:40:03.012 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi163.front.sepia.ceph.com 2023-10-09T06:40:03.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:40:03.012 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-09T06:40:03.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-09T06:40:03.027 INFO:teuthology.orchestra.run.smithi163.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-09T06:40:03.027 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-09T06:40:03.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:40:03.027 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-09T06:40:03.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-09T06:40:03.068 INFO:teuthology.orchestra.run.smithi181.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-09T06:40:03.069 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs 2023-10-09T06:40:03.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.188+0000 7fbaa5176700 1 Processor -- start 2023-10-09T06:40:03.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.188+0000 7fbaa5176700 1 -- start start 2023-10-09T06:40:03.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.189+0000 7fbaa5176700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbaa00b4e70 0x7fbaa00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:03.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.189+0000 7fbaa5176700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaa00cfb00 con 0x7fbaa00b16d0 2023-10-09T06:40:03.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.189+0000 7fbaa5176700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaa015e060 con 0x7fbaa00b1b40 2023-10-09T06:40:03.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.189+0000 7fbaa5176700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaa00b5c40 con 0x7fbaa00b4e70 2023-10-09T06:40:03.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.189+0000 7fba9ffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbaa00b4e70 0x7fbaa00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:03.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.189+0000 7fba9f7fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fbaa00b16d0 0x7fbaa015d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:46056/0 (socket says 172.21.15.163:46056) 2023-10-09T06:40:03.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.189+0000 7fba9f7fe700 1 -- 172.21.15.163:0/3236479581 learned_addr learned my addr 172.21.15.163:0/3236479581 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:40:03.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.189+0000 7fba9ffff700 1 -- 172.21.15.163:0/3236479581 >> v1:172.21.15.181:6789/0 conn(0x7fbaa00b1b40 legacy=0x7fbaa015d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:40:03.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.189+0000 7fba9ffff700 1 -- 172.21.15.163:0/3236479581 >> v1:172.21.15.163:6789/0 conn(0x7fbaa00b16d0 legacy=0x7fbaa015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:03.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.189+0000 7fba9ffff700 1 -- 172.21.15.163:0/3236479581 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa00b5f20 con 0x7fbaa00b4e70 2023-10-09T06:40:03.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.189+0000 7fba9effd700 1 -- 172.21.15.163:0/3236479581 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4031279200 0 0) 0x7fbaa00cfb00 con 0x7fbaa00b16d0 2023-10-09T06:40:03.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.190+0000 7fba9ffff700 1 --2- 172.21.15.163:0/3236479581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaa00b4e70 0x7fbaa00ccff0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fba900098e0 tx=0x7fba900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f43239a24a224f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:40:03.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.190+0000 7fba9effd700 1 -- 172.21.15.163:0/3236479581 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba90009e80 con 0x7fbaa00b4e70 2023-10-09T06:40:03.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.190+0000 7fba9effd700 1 -- 172.21.15.163:0/3236479581 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba9000b820 con 0x7fbaa00b4e70 2023-10-09T06:40:03.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.190+0000 7fba9effd700 1 -- 172.21.15.163:0/3236479581 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9000bba0 con 0x7fbaa00b4e70 2023-10-09T06:40:03.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.190+0000 7fbaa5176700 1 -- 172.21.15.163:0/3236479581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaa00b4e70 msgr2=0x7fbaa00ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:03.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.190+0000 7fbaa5176700 1 --2- 172.21.15.163:0/3236479581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaa00b4e70 0x7fbaa00ccff0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fba900098e0 tx=0x7fba900091a0 comp rx=0 tx=0).stop 2023-10-09T06:40:03.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.190+0000 7fbaa5176700 1 -- 172.21.15.163:0/3236479581 shutdown_connections 2023-10-09T06:40:03.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.190+0000 7fbaa5176700 1 --2- 172.21.15.163:0/3236479581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaa00b4e70 0x7fbaa00ccff0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:03.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.190+0000 7fbaa5176700 1 -- 172.21.15.163:0/3236479581 >> 172.21.15.163:0/3236479581 conn(0x7fbaa00d10c0 msgr2=0x7fbaa00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:40:03.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.190+0000 7fbaa5176700 1 -- 172.21.15.163:0/3236479581 shutdown_connections 2023-10-09T06:40:03.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.190+0000 7fbaa5176700 1 -- 172.21.15.163:0/3236479581 wait complete. 2023-10-09T06:40:03.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.191+0000 7fbaa5176700 1 Processor -- start 2023-10-09T06:40:03.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.192+0000 7fbaa5176700 1 -- start start 2023-10-09T06:40:03.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.192+0000 7fbaa5176700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbaa00b16d0 0x7fbaa016ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:03.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.192+0000 7fbaa5176700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaa00b1b40 0x7fbaa0068b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:03.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.192+0000 7fbaa5176700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaa00b4e70 0x7fbaa0069040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:03.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.192+0000 7fbaa5176700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa016d410 con 0x7fbaa00b1b40 2023-10-09T06:40:03.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.192+0000 7fbaa5176700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa016d810 con 0x7fbaa00b4e70 2023-10-09T06:40:03.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.192+0000 7fbaa5176700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa016daf0 con 0x7fbaa00b16d0 2023-10-09T06:40:03.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.192+0000 7fba9f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaa00b1b40 0x7fbaa0068b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:03.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.192+0000 7fba9f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaa00b1b40 0x7fbaa0068b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:52054/0 (socket says 172.21.15.163:52054) 2023-10-09T06:40:03.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.192+0000 7fba9f7fe700 1 -- 172.21.15.163:0/49602414 learned_addr learned my addr 172.21.15.163:0/49602414 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:40:03.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.193+0000 7fba9f7fe700 1 -- 172.21.15.163:0/49602414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbaa00b16d0 msgr2=0x7fbaa016ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:03.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.193+0000 7fba9f7fe700 1 --2- 172.21.15.163:0/49602414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbaa00b16d0 0x7fbaa016ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:03.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.193+0000 7fba9f7fe700 1 -- 172.21.15.163:0/49602414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaa00b4e70 msgr2=0x7fbaa0069040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:03.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.193+0000 7fba9f7fe700 1 --2- 172.21.15.163:0/49602414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaa00b4e70 0x7fbaa0069040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:03.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.193+0000 7fba9f7fe700 1 -- 172.21.15.163:0/49602414 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba90009420 con 0x7fbaa00b1b40 2023-10-09T06:40:03.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.193+0000 7fba9f7fe700 1 --2- 172.21.15.163:0/49602414 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaa00b1b40 0x7fbaa0068b00 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fba9400c7e0 tx=0x7fba94007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:40:03.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.193+0000 7fba87fff700 1 -- 172.21.15.163:0/49602414 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba94005b30 con 0x7fbaa00b1b40 2023-10-09T06:40:03.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.193+0000 7fba87fff700 1 -- 172.21.15.163:0/49602414 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba94005c90 con 0x7fbaa00b1b40 2023-10-09T06:40:03.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.193+0000 7fbaa5176700 1 -- 172.21.15.163:0/49602414 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaa016ddd0 con 0x7fbaa00b1b40 2023-10-09T06:40:03.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.193+0000 7fba87fff700 1 -- 172.21.15.163:0/49602414 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba94007300 con 0x7fbaa00b1b40 2023-10-09T06:40:03.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.193+0000 7fbaa5176700 1 -- 172.21.15.163:0/49602414 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa016e390 con 0x7fbaa00b1b40 2023-10-09T06:40:03.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.194+0000 7fbaa5176700 1 -- 172.21.15.163:0/49602414 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaa0088990 con 0x7fbaa00b1b40 2023-10-09T06:40:03.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.194+0000 7fba87fff700 1 -- 172.21.15.163:0/49602414 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fba9401b400 con 0x7fbaa00b1b40 2023-10-09T06:40:03.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.195+0000 7fba87fff700 1 --2- 172.21.15.163:0/49602414 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fba8c067320 0x7fba8c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:03.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.195+0000 7fba87fff700 1 -- 172.21.15.163:0/49602414 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fba940847f0 con 0x7fbaa00b1b40 2023-10-09T06:40:03.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.195+0000 7fba9ffff700 1 --2- 172.21.15.163:0/49602414 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fba8c067320 0x7fba8c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:03.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.195+0000 7fba9ffff700 1 --2- 172.21.15.163:0/49602414 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fba8c067320 0x7fba8c0697d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fba900091e0 tx=0x7fba90016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:40:03.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.199+0000 7fba87fff700 1 -- 172.21.15.163:0/49602414 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba94051500 con 0x7fbaa00b1b40 2023-10-09T06:40:03.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.414+0000 7fbaa5176700 1 -- 172.21.15.163:0/49602414 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) v1 -- 0x7fbaa0157690 con 0x7fba8c067320 2023-10-09T06:40:03.416 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-09T06:40:03.416+0000 7ff9594e4700 -1 mgr.server reply reply (1) Operation not permitted WARNING: this will *PERMANENTLY DESTROY* all data stored in the filesystem 'cephfs'. If you are *ABSOLUTELY CERTAIN* that is what you want, re-issue the command followed by --yes-i-really-mean-it. 2023-10-09T06:40:03.416 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.415+0000 7fba87fff700 1 -- 172.21.15.163:0/49602414 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: -1 WARNING: this will *PERMANENTLY DESTROY* all data stored in the filesystem 'cephfs'. If you are *ABSOLUTELY CERTAIN* that is what you want, re-issue the command followed by --yes-i-really-mean-it.) v1 ==== 204+0+0 (secure 0 0 0) 0x7fbaa0157690 con 0x7fba8c067320 2023-10-09T06:40:03.418 INFO:teuthology.orchestra.run.smithi163.stderr:Error EPERM: WARNING: this will *PERMANENTLY DESTROY* all data stored in the filesystem 'cephfs'. If you are *ABSOLUTELY CERTAIN* that is what you want, re-issue the command followed by --yes-i-really-mean-it. 2023-10-09T06:40:03.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-09T06:40:03.422 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it 2023-10-09T06:40:03.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.539+0000 7faa329a7700 1 Processor -- start 2023-10-09T06:40:03.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.539+0000 7faa329a7700 1 -- start start 2023-10-09T06:40:03.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.540+0000 7faa329a7700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faa2c0b16d0 0x7faa2c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:03.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.540+0000 7faa329a7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7faa2c0b33b0 0x7faa2c0b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:03.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.540+0000 7faa329a7700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa2c0b2080 con 0x7faa2c0b4e70 2023-10-09T06:40:03.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.540+0000 7faa329a7700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa2c0cfb00 con 0x7faa2c0b33b0 2023-10-09T06:40:03.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.540+0000 7faa329a7700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa2c0b5c40 con 0x7faa2c0b16d0 2023-10-09T06:40:03.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.540+0000 7faa321a6700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7faa2c0b33b0 0x7faa2c0b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:03.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.540+0000 7faa319a5700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7faa2c0b4e70 0x7faa2c0ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:46060/0 (socket says 172.21.15.163:46060) 2023-10-09T06:40:03.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.540+0000 7faa319a5700 1 -- 172.21.15.163:0/2821054758 learned_addr learned my addr 172.21.15.163:0/2821054758 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:40:03.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.540+0000 7faa311a4700 1 --2- 172.21.15.163:0/2821054758 >> v2:172.21.15.181:3300/0 conn(0x7faa2c0b16d0 0x7faa2c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:03.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.541+0000 7faa321a6700 1 -- 172.21.15.163:0/2821054758 >> v2:172.21.15.181:3300/0 conn(0x7faa2c0b16d0 msgr2=0x7faa2c0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:40:03.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.541+0000 7faa321a6700 1 --2- 172.21.15.163:0/2821054758 >> v2:172.21.15.181:3300/0 conn(0x7faa2c0b16d0 0x7faa2c0b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:03.543 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.541+0000 7faa321a6700 1 -- 172.21.15.163:0/2821054758 >> v1:172.21.15.163:6789/0 conn(0x7faa2c0b4e70 legacy=0x7faa2c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:03.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.541+0000 7faa309a3700 1 -- 172.21.15.163:0/2821054758 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3784799059 0 0) 0x7faa2c0b2080 con 0x7faa2c0b4e70 2023-10-09T06:40:03.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.541+0000 7faa321a6700 1 -- 172.21.15.163:0/2821054758 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2c0b5f20 con 0x7faa2c0b33b0 2023-10-09T06:40:03.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.541+0000 7faa321a6700 1 --2- 172.21.15.163:0/2821054758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa2c0b33b0 0x7faa2c0b37d0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7faa2c089830 tx=0x7faa240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907b115cbd77b9fb server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:40:03.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.541+0000 7faa309a3700 1 -- 172.21.15.163:0/2821054758 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2400b6b0 con 0x7faa2c0b33b0 2023-10-09T06:40:03.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.541+0000 7faa309a3700 1 -- 172.21.15.163:0/2821054758 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa2400b810 con 0x7faa2c0b33b0 2023-10-09T06:40:03.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.541+0000 7faa309a3700 1 -- 172.21.15.163:0/2821054758 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2400bbb0 con 0x7faa2c0b33b0 2023-10-09T06:40:03.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.541+0000 7faa329a7700 1 -- 172.21.15.163:0/2821054758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa2c0b33b0 msgr2=0x7faa2c0b37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:03.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.541+0000 7faa329a7700 1 --2- 172.21.15.163:0/2821054758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa2c0b33b0 0x7faa2c0b37d0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7faa2c089830 tx=0x7faa240091a0 comp rx=0 tx=0).stop 2023-10-09T06:40:03.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.541+0000 7faa329a7700 1 -- 172.21.15.163:0/2821054758 shutdown_connections 2023-10-09T06:40:03.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.541+0000 7faa329a7700 1 --2- 172.21.15.163:0/2821054758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa2c0b33b0 0x7faa2c0b37d0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:03.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.541+0000 7faa329a7700 1 --2- 172.21.15.163:0/2821054758 >> v2:172.21.15.181:3300/0 conn(0x7faa2c0b16d0 0x7faa2c0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:03.544 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.541+0000 7faa329a7700 1 -- 172.21.15.163:0/2821054758 >> 172.21.15.163:0/2821054758 conn(0x7faa2c0d10c0 msgr2=0x7faa2c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:40:03.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.542+0000 7faa329a7700 1 -- 172.21.15.163:0/2821054758 shutdown_connections 2023-10-09T06:40:03.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.542+0000 7faa329a7700 1 -- 172.21.15.163:0/2821054758 wait complete. 2023-10-09T06:40:03.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.543+0000 7faa329a7700 1 Processor -- start 2023-10-09T06:40:03.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.543+0000 7faa329a7700 1 -- start start 2023-10-09T06:40:03.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.543+0000 7faa329a7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa2c0b16d0 0x7faa2c0c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:03.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.543+0000 7faa329a7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa2c0b33b0 0x7faa2c0c6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:03.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.543+0000 7faa329a7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa2c0b4e70 0x7faa2c0cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:03.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.543+0000 7faa329a7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c0cc230 con 0x7faa2c0b4e70 2023-10-09T06:40:03.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.543+0000 7faa329a7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c0cb620 con 0x7faa2c0b16d0 2023-10-09T06:40:03.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.543+0000 7faa329a7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c0cb900 con 0x7faa2c0b33b0 2023-10-09T06:40:03.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.543+0000 7faa321a6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa2c0b4e70 0x7faa2c0cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:03.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.544+0000 7faa321a6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa2c0b4e70 0x7faa2c0cadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:52078/0 (socket says 172.21.15.163:52078) 2023-10-09T06:40:03.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.544+0000 7faa321a6700 1 -- 172.21.15.163:0/78947825 learned_addr learned my addr 172.21.15.163:0/78947825 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:40:03.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.544+0000 7faa319a5700 1 --2- 172.21.15.163:0/78947825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa2c0b16d0 0x7faa2c0c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:03.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.544+0000 7faa311a4700 1 --2- 172.21.15.163:0/78947825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa2c0b33b0 0x7faa2c0c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:03.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.544+0000 7faa321a6700 1 -- 172.21.15.163:0/78947825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa2c0b33b0 msgr2=0x7faa2c0c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:03.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.544+0000 7faa321a6700 1 --2- 172.21.15.163:0/78947825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa2c0b33b0 0x7faa2c0c6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:03.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.544+0000 7faa321a6700 1 -- 172.21.15.163:0/78947825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa2c0b16d0 msgr2=0x7faa2c0c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:40:03.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.544+0000 7faa321a6700 1 --2- 172.21.15.163:0/78947825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa2c0b16d0 0x7faa2c0c95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:40:03.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.544+0000 7faa321a6700 1 -- 172.21.15.163:0/78947825 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa24009420 con 0x7faa2c0b4e70 2023-10-09T06:40:03.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.544+0000 7faa321a6700 1 --2- 172.21.15.163:0/78947825 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa2c0b4e70 0x7faa2c0cadf0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7faa2c070dd0 tx=0x7faa24017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:40:03.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.544+0000 7faa21ffb700 1 -- 172.21.15.163:0/78947825 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa24006af0 con 0x7faa2c0b4e70 2023-10-09T06:40:03.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.544+0000 7faa21ffb700 1 -- 172.21.15.163:0/78947825 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa2400bb90 con 0x7faa2c0b4e70 2023-10-09T06:40:03.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.544+0000 7faa329a7700 1 -- 172.21.15.163:0/78947825 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa2c0cbbe0 con 0x7faa2c0b4e70 2023-10-09T06:40:03.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.544+0000 7faa21ffb700 1 -- 172.21.15.163:0/78947825 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa24011630 con 0x7faa2c0b4e70 2023-10-09T06:40:03.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.544+0000 7faa329a7700 1 -- 172.21.15.163:0/78947825 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa2c0cb330 con 0x7faa2c0b4e70 2023-10-09T06:40:03.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.545+0000 7faa329a7700 1 -- 172.21.15.163:0/78947825 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa2c088990 con 0x7faa2c0b4e70 2023-10-09T06:40:03.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.546+0000 7faa21ffb700 1 -- 172.21.15.163:0/78947825 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7faa24011a60 con 0x7faa2c0b4e70 2023-10-09T06:40:03.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.546+0000 7faa21ffb700 1 --2- 172.21.15.163:0/78947825 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faa10067320 0x7faa100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:40:03.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.546+0000 7faa21ffb700 1 -- 172.21.15.163:0/78947825 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5705+0+0 (secure 0 0 0) 0x7faa24086630 con 0x7faa2c0b4e70 2023-10-09T06:40:03.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.546+0000 7faa319a5700 1 --2- 172.21.15.163:0/78947825 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faa10067320 0x7faa100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:40:03.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.546+0000 7faa319a5700 1 --2- 172.21.15.163:0/78947825 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faa10067320 0x7faa100697d0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7faa28002230 tx=0x7faa28001a20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:40:03.551 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.550+0000 7faa21ffb700 1 -- 172.21.15.163:0/78947825 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa240532c0 con 0x7faa2c0b4e70 2023-10-09T06:40:03.761 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:40:03.760+0000 7faa329a7700 1 -- 172.21.15.163:0/78947825 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7faa2c04bb80 con 0x7faa10067320 2023-10-09T06:40:14.390 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:40:14.392 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:40:44.471 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:40:44.475 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:41:14.568 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:41:14.571 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:41:44.639 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:41:44.642 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:42:03.451 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-10-09T06:42:03.452 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-09T06:42:03.453 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:42:03.495 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-09T06:42:03.545 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-09T06:42:03.546 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:42:03.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.669+0000 7effb27e0700 1 Processor -- start 2023-10-09T06:42:03.670 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.669+0000 7effb27e0700 1 -- start start 2023-10-09T06:42:03.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.669+0000 7effb27e0700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7effac140350 0x7effac069c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:03.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.669+0000 7effb27e0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effac068aa0 0x7effac0c8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:03.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.669+0000 7effb27e0700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effac0c7de0 con 0x7effac06a1c0 2023-10-09T06:42:03.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.669+0000 7effb27e0700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effac140c10 con 0x7effac140350 2023-10-09T06:42:03.673 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.669+0000 7effb27e0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effac141490 con 0x7effac068aa0 2023-10-09T06:42:03.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.670+0000 7effab7fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7effac06a1c0 0x7effac068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:54028/0 (socket says 172.21.15.163:54028) 2023-10-09T06:42:03.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.670+0000 7effab7fe700 1 -- 172.21.15.163:0/2531721741 learned_addr learned my addr 172.21.15.163:0/2531721741 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:42:03.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.670+0000 7effb0d7d700 1 --2- 172.21.15.163:0/2531721741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effac068aa0 0x7effac0c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:03.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.670+0000 7effabfff700 1 --2- 172.21.15.163:0/2531721741 >> v2:172.21.15.181:3300/0 conn(0x7effac140350 0x7effac069c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:03.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.670+0000 7effaaffd700 1 -- 172.21.15.163:0/2531721741 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1687908849 0 0) 0x7effac0c7de0 con 0x7effac06a1c0 2023-10-09T06:42:03.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.670+0000 7effaaffd700 1 -- 172.21.15.163:0/2531721741 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff94003570 con 0x7effac06a1c0 2023-10-09T06:42:03.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.670+0000 7effaaffd700 1 -- 172.21.15.163:0/2531721741 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1776437519 0 0) 0x7eff94003570 con 0x7effac06a1c0 2023-10-09T06:42:03.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.670+0000 7effaaffd700 1 -- 172.21.15.163:0/2531721741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effac068aa0 msgr2=0x7effac0c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:03.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.670+0000 7effaaffd700 1 --2- 172.21.15.163:0/2531721741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effac068aa0 0x7effac0c8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:03.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.670+0000 7effaaffd700 1 -- 172.21.15.163:0/2531721741 >> v2:172.21.15.181:3300/0 conn(0x7effac140350 msgr2=0x7effac069c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:03.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.670+0000 7effaaffd700 1 --2- 172.21.15.163:0/2531721741 >> v2:172.21.15.181:3300/0 conn(0x7effac140350 0x7effac069c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:03.674 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.670+0000 7effaaffd700 1 -- 172.21.15.163:0/2531721741 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effac140ed0 con 0x7effac06a1c0 2023-10-09T06:42:03.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.670+0000 7effaaffd700 1 -- 172.21.15.163:0/2531721741 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7effa0001470 con 0x7effac06a1c0 2023-10-09T06:42:03.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.670+0000 7effaaffd700 1 -- 172.21.15.163:0/2531721741 >> v1:172.21.15.163:6789/0 conn(0x7effac06a1c0 legacy=0x7effac068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:03.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.670+0000 7effaaffd700 1 --2- 172.21.15.163:0/2531721741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff94003a60 0x7eff94003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:03.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.670+0000 7effaaffd700 1 --2- 172.21.15.163:0/2531721741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effac068aa0 0x7eff94004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:03.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.670+0000 7effaaffd700 1 --2- 172.21.15.163:0/2531721741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff94004d50 0x7eff940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:03.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.670+0000 7effaaffd700 1 -- 172.21.15.163:0/2531721741 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effac140c10 con 0x7eff94004d50 2023-10-09T06:42:03.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.670+0000 7effaaffd700 1 -- 172.21.15.163:0/2531721741 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effac141490 con 0x7eff94003a60 2023-10-09T06:42:03.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.670+0000 7effaaffd700 1 -- 172.21.15.163:0/2531721741 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff9400ba60 con 0x7effac068aa0 2023-10-09T06:42:03.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.670+0000 7effb0d7d700 1 --2- 172.21.15.163:0/2531721741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff94004d50 0x7eff940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:03.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.670+0000 7effab7fe700 1 --2- 172.21.15.163:0/2531721741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effac068aa0 0x7eff94004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:03.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.671+0000 7effabfff700 1 --2- 172.21.15.163:0/2531721741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff94003a60 0x7eff94003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:03.675 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.671+0000 7effb0d7d700 1 -- 172.21.15.163:0/2531721741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effac068aa0 msgr2=0x7eff94004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:03.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.671+0000 7effb0d7d700 1 --2- 172.21.15.163:0/2531721741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effac068aa0 0x7eff94004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:03.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.671+0000 7effb0d7d700 1 -- 172.21.15.163:0/2531721741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff94003a60 msgr2=0x7eff94003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:03.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.671+0000 7effb0d7d700 1 --2- 172.21.15.163:0/2531721741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff94003a60 0x7eff94003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:03.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.671+0000 7effb0d7d700 1 -- 172.21.15.163:0/2531721741 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff94003570 con 0x7eff94004d50 2023-10-09T06:42:03.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.671+0000 7effb0d7d700 1 --2- 172.21.15.163:0/2531721741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff94004d50 0x7eff940051c0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7effac068ea0 tx=0x7effa400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb4b3fe4a5757481 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:03.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.671+0000 7effaaffd700 1 -- 172.21.15.163:0/2531721741 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa400e9a0 con 0x7eff94004d50 2023-10-09T06:42:03.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.671+0000 7effaaffd700 1 -- 172.21.15.163:0/2531721741 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa400f040 con 0x7eff94004d50 2023-10-09T06:42:03.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.671+0000 7effaaffd700 1 -- 172.21.15.163:0/2531721741 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa400ed50 con 0x7eff94004d50 2023-10-09T06:42:03.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.671+0000 7effb27e0700 1 -- 172.21.15.163:0/2531721741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff94004d50 msgr2=0x7eff940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:03.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.671+0000 7effb27e0700 1 --2- 172.21.15.163:0/2531721741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff94004d50 0x7eff940051c0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7effac068ea0 tx=0x7effa400b040 comp rx=0 tx=0).stop 2023-10-09T06:42:03.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.672+0000 7effb0d7d700 1 -- 172.21.15.163:0/2531721741 reap_dead start 2023-10-09T06:42:03.676 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.672+0000 7effb27e0700 1 -- 172.21.15.163:0/2531721741 shutdown_connections 2023-10-09T06:42:03.677 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.672+0000 7effb27e0700 1 -- 172.21.15.163:0/2531721741 >> 172.21.15.163:0/2531721741 conn(0x7effac0cd670 msgr2=0x7effac0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:03.677 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.672+0000 7effb27e0700 1 -- 172.21.15.163:0/2531721741 shutdown_connections 2023-10-09T06:42:03.677 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.672+0000 7effb27e0700 1 -- 172.21.15.163:0/2531721741 wait complete. 2023-10-09T06:42:03.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.673+0000 7effb27e0700 1 Processor -- start 2023-10-09T06:42:03.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.673+0000 7effb27e0700 1 -- start start 2023-10-09T06:42:03.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.674+0000 7effb27e0700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effac1575e0 0x7effac158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:03.679 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.674+0000 7effb27e0700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effac1579b0 0x7effac0c6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:03.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.674+0000 7effb27e0700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effac0b2560 0x7effac0b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:03.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.674+0000 7effb27e0700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effac0b2f10 con 0x7effac0b2560 2023-10-09T06:42:03.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.674+0000 7effb27e0700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effac159ac0 con 0x7effac1575e0 2023-10-09T06:42:03.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.674+0000 7effb27e0700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effac0c71a0 con 0x7effac1579b0 2023-10-09T06:42:03.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.674+0000 7effb0d7d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effac0b2560 0x7effac0b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:03.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.674+0000 7effb0d7d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effac0b2560 0x7effac0b29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:52212/0 (socket says 172.21.15.163:52212) 2023-10-09T06:42:03.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.674+0000 7effb0d7d700 1 -- 172.21.15.163:0/1400790638 learned_addr learned my addr 172.21.15.163:0/1400790638 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:03.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.674+0000 7effabfff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effac1575e0 0x7effac158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:03.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.674+0000 7effab7fe700 1 --2- 172.21.15.163:0/1400790638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effac1579b0 0x7effac0c6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:03.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.674+0000 7effb0d7d700 1 -- 172.21.15.163:0/1400790638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effac1579b0 msgr2=0x7effac0c6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:03.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.674+0000 7effb0d7d700 1 --2- 172.21.15.163:0/1400790638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effac1579b0 0x7effac0c6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:03.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.674+0000 7effb0d7d700 1 -- 172.21.15.163:0/1400790638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effac1575e0 msgr2=0x7effac158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:03.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.674+0000 7effb0d7d700 1 --2- 172.21.15.163:0/1400790638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effac1575e0 0x7effac158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:03.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.674+0000 7effb0d7d700 1 -- 172.21.15.163:0/1400790638 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effac158eb0 con 0x7effac0b2560 2023-10-09T06:42:03.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.674+0000 7effb0d7d700 1 --2- 172.21.15.163:0/1400790638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effac0b2560 0x7effac0b29d0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7effa4016000 tx=0x7effa401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:03.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.674+0000 7eff93fff700 1 -- 172.21.15.163:0/1400790638 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa400ee40 con 0x7effac0b2560 2023-10-09T06:42:03.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.674+0000 7eff93fff700 1 -- 172.21.15.163:0/1400790638 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa401d630 con 0x7effac0b2560 2023-10-09T06:42:03.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.674+0000 7effb27e0700 1 -- 172.21.15.163:0/1400790638 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effac159190 con 0x7effac0b2560 2023-10-09T06:42:03.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.674+0000 7eff93fff700 1 -- 172.21.15.163:0/1400790638 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa401d9f0 con 0x7effac0b2560 2023-10-09T06:42:03.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.674+0000 7effb27e0700 1 -- 172.21.15.163:0/1400790638 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effac159750 con 0x7effac0b2560 2023-10-09T06:42:03.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.675+0000 7effb27e0700 1 -- 172.21.15.163:0/1400790638 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effac159c70 con 0x7effac0b2560 2023-10-09T06:42:03.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.676+0000 7eff93fff700 1 -- 172.21.15.163:0/1400790638 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7effa40273e0 con 0x7effac0b2560 2023-10-09T06:42:03.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.676+0000 7eff93fff700 1 --2- 172.21.15.163:0/1400790638 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7eff98067320 0x7eff980697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:03.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.676+0000 7eff93fff700 1 -- 172.21.15.163:0/1400790638 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5792+0+0 (secure 0 0 0) 0x7effa4085eb0 con 0x7effac0b2560 2023-10-09T06:42:03.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.676+0000 7effabfff700 1 --2- 172.21.15.163:0/1400790638 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7eff98067320 0x7eff980697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:03.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.677+0000 7effabfff700 1 --2- 172.21.15.163:0/1400790638 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7eff98067320 0x7eff980697d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0090f0 tx=0x7eff9c009120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:03.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.680+0000 7eff93fff700 1 -- 172.21.15.163:0/1400790638 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7effac159c70 con 0x7effac0b2560 2023-10-09T06:42:03.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.907+0000 7effb27e0700 1 -- 172.21.15.163:0/1400790638 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effac06e5e0 con 0x7effac0b2560 2023-10-09T06:42:03.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.907+0000 7eff93fff700 1 -- 172.21.15.163:0/1400790638 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+4619 (secure 0 0 0) 0x7effa4052b20 con 0x7effac0b2560 2023-10-09T06:42:03.908 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:42:03.908 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":133,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7046,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3374277959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3374277959},{"type":"v1","addr":"172.21.15.181:6837","nonce":3374277959}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3834979450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3834979450},{"type":"v1","addr":"172.21.15.181:6836","nonce":3834979450}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":8991,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2681808199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2681808199},{"type":"v1","addr":"172.21.15.163:6834","nonce":2681808199}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128}],"filesystems":[{"mdsmap":{"epoch":132,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:39:52.456713+0000","modified":"2023-10-09T06:39:56.092448+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8994":{"gid":8994,"name":"a","rank":0,"incarnation":129,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/2564044136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2564044136},{"type":"v1","addr":"172.21.15.163:6835","nonce":2564044136}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-09T06:42:03.910 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.909+0000 7effb27e0700 1 -- 172.21.15.163:0/1400790638 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7eff98067320 msgr2=0x7eff980697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:03.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.909+0000 7effb27e0700 1 --2- 172.21.15.163:0/1400790638 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7eff98067320 0x7eff980697d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0090f0 tx=0x7eff9c009120 comp rx=0 tx=0).stop 2023-10-09T06:42:03.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.909+0000 7effb27e0700 1 -- 172.21.15.163:0/1400790638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effac0b2560 msgr2=0x7effac0b29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:03.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.909+0000 7effb27e0700 1 --2- 172.21.15.163:0/1400790638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effac0b2560 0x7effac0b29d0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7effa4016000 tx=0x7effa401a000 comp rx=0 tx=0).stop 2023-10-09T06:42:03.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.909+0000 7effb27e0700 1 -- 172.21.15.163:0/1400790638 shutdown_connections 2023-10-09T06:42:03.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.909+0000 7effb27e0700 1 --2- 172.21.15.163:0/1400790638 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7eff98067320 0x7eff980697d0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:03.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.909+0000 7effb27e0700 1 --2- 172.21.15.163:0/1400790638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effac0b2560 0x7effac0b29d0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:03.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.909+0000 7effb27e0700 1 --2- 172.21.15.163:0/1400790638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7effac1575e0 0x7effac158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:03.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.909+0000 7effb27e0700 1 --2- 172.21.15.163:0/1400790638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7effac1579b0 0x7effac0c6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:03.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.909+0000 7effb27e0700 1 -- 172.21.15.163:0/1400790638 >> 172.21.15.163:0/1400790638 conn(0x7effac0cd670 msgr2=0x7effac13d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:03.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.909+0000 7effb27e0700 1 -- 172.21.15.163:0/1400790638 shutdown_connections 2023-10-09T06:42:03.911 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:03.909+0000 7effb27e0700 1 -- 172.21.15.163:0/1400790638 wait complete. 2023-10-09T06:42:03.912 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 133 2023-10-09T06:42:03.923 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:42:04.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.037+0000 7f6cd0ac1700 1 Processor -- start 2023-10-09T06:42:04.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.037+0000 7f6cd0ac1700 1 -- start start 2023-10-09T06:42:04.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.038+0000 7f6cd0ac1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6ccc159890 0x7f6ccc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:04.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.038+0000 7f6cd0ac1700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6ccc069560 0x7f6ccc0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:04.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.038+0000 7f6cd0ac1700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ccc069ee0 con 0x7f6ccc159c60 2023-10-09T06:42:04.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.038+0000 7f6cd0ac1700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ccc15a160 con 0x7f6ccc159890 2023-10-09T06:42:04.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.038+0000 7f6cd0ac1700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ccc0d13a0 con 0x7f6ccc069560 2023-10-09T06:42:04.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.038+0000 7f6cca59c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6ccc159890 0x7f6ccc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:04.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.038+0000 7f6cca59c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6ccc159890 0x7f6ccc15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:52214/0 (socket says 172.21.15.163:52214) 2023-10-09T06:42:04.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.038+0000 7f6cc9d9b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f6ccc159c60 0x7f6ccc15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:54036/0 (socket says 172.21.15.163:54036) 2023-10-09T06:42:04.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.038+0000 7f6cca59c700 1 -- 172.21.15.163:0/3110287489 learned_addr learned my addr 172.21.15.163:0/3110287489 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:04.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.039+0000 7f6ccad9d700 1 --2- 172.21.15.163:0/3110287489 >> v2:172.21.15.181:3300/0 conn(0x7f6ccc069560 0x7f6ccc0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:04.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.039+0000 7f6cca59c700 1 -- 172.21.15.163:0/3110287489 >> v2:172.21.15.181:3300/0 conn(0x7f6ccc069560 msgr2=0x7f6ccc0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:04.041 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.039+0000 7f6cca59c700 1 --2- 172.21.15.163:0/3110287489 >> v2:172.21.15.181:3300/0 conn(0x7f6ccc069560 0x7f6ccc0699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:04.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.039+0000 7f6cca59c700 1 -- 172.21.15.163:0/3110287489 >> v1:172.21.15.163:6789/0 conn(0x7f6ccc159c60 legacy=0x7f6ccc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:04.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.039+0000 7f6cca59c700 1 -- 172.21.15.163:0/3110287489 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ccc15e140 con 0x7f6ccc159890 2023-10-09T06:42:04.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.039+0000 7f6cc959a700 1 -- 172.21.15.163:0/3110287489 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 231351 0 0) 0x7f6ccc069ee0 con 0x7f6ccc159c60 2023-10-09T06:42:04.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.039+0000 7f6cca59c700 1 --2- 172.21.15.163:0/3110287489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ccc159890 0x7f6ccc15abd0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc06a890 tx=0x7f6cbc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90353276e835dadd server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:04.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.039+0000 7f6cc959a700 1 -- 172.21.15.163:0/3110287489 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc00e070 con 0x7f6ccc159890 2023-10-09T06:42:04.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.039+0000 7f6cc959a700 1 -- 172.21.15.163:0/3110287489 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cbc00ba50 con 0x7f6ccc159890 2023-10-09T06:42:04.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.039+0000 7f6cc959a700 1 -- 172.21.15.163:0/3110287489 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc0092e0 con 0x7f6ccc159890 2023-10-09T06:42:04.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.040+0000 7f6cd0ac1700 1 -- 172.21.15.163:0/3110287489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ccc159890 msgr2=0x7f6ccc15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:04.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.040+0000 7f6cd0ac1700 1 --2- 172.21.15.163:0/3110287489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ccc159890 0x7f6ccc15abd0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc06a890 tx=0x7f6cbc009840 comp rx=0 tx=0).stop 2023-10-09T06:42:04.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.040+0000 7f6cd0ac1700 1 -- 172.21.15.163:0/3110287489 shutdown_connections 2023-10-09T06:42:04.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.040+0000 7f6cd0ac1700 1 --2- 172.21.15.163:0/3110287489 >> v2:172.21.15.181:3300/0 conn(0x7f6ccc069560 0x7f6ccc0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:04.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.040+0000 7f6cd0ac1700 1 --2- 172.21.15.163:0/3110287489 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ccc159890 0x7f6ccc15abd0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:04.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.040+0000 7f6cd0ac1700 1 -- 172.21.15.163:0/3110287489 >> 172.21.15.163:0/3110287489 conn(0x7f6ccc068e20 msgr2=0x7f6ccc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:04.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.040+0000 7f6cd0ac1700 1 -- 172.21.15.163:0/3110287489 shutdown_connections 2023-10-09T06:42:04.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.040+0000 7f6cd0ac1700 1 -- 172.21.15.163:0/3110287489 wait complete. 2023-10-09T06:42:04.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.041+0000 7f6cd0ac1700 1 Processor -- start 2023-10-09T06:42:04.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.041+0000 7f6cd0ac1700 1 -- start start 2023-10-09T06:42:04.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.041+0000 7f6cd0ac1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ccc069560 0x7f6ccc14bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:04.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.041+0000 7f6cd0ac1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ccc159890 0x7f6ccc0cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:04.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.041+0000 7f6cd0ac1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ccc159c60 0x7f6ccc0c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:04.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.041+0000 7f6cd0ac1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc16ce90 con 0x7f6ccc069560 2023-10-09T06:42:04.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.041+0000 7f6cd0ac1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc0cdb60 con 0x7f6ccc159890 2023-10-09T06:42:04.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.041+0000 7f6cd0ac1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc0cde40 con 0x7f6ccc159c60 2023-10-09T06:42:04.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.042+0000 7f6cca59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ccc069560 0x7f6ccc14bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:04.046 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.042+0000 7f6cca59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ccc069560 0x7f6ccc14bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:52216/0 (socket says 172.21.15.163:52216) 2023-10-09T06:42:04.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.042+0000 7f6cca59c700 1 -- 172.21.15.163:0/4200024067 learned_addr learned my addr 172.21.15.163:0/4200024067 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:04.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.042+0000 7f6cc9d9b700 1 --2- 172.21.15.163:0/4200024067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ccc159890 0x7f6ccc0cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:04.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.042+0000 7f6ccad9d700 1 --2- 172.21.15.163:0/4200024067 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ccc159c60 0x7f6ccc0c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:04.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.042+0000 7f6cca59c700 1 -- 172.21.15.163:0/4200024067 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ccc159c60 msgr2=0x7f6ccc0c5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:04.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.042+0000 7f6cca59c700 1 --2- 172.21.15.163:0/4200024067 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ccc159c60 0x7f6ccc0c5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:04.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.042+0000 7f6cca59c700 1 -- 172.21.15.163:0/4200024067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ccc159890 msgr2=0x7f6ccc0cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:04.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.042+0000 7f6cca59c700 1 --2- 172.21.15.163:0/4200024067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ccc159890 0x7f6ccc0cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:04.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.042+0000 7f6cca59c700 1 -- 172.21.15.163:0/4200024067 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cbc009ae0 con 0x7f6ccc069560 2023-10-09T06:42:04.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.042+0000 7f6cc9d9b700 1 --2- 172.21.15.163:0/4200024067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ccc159890 0x7f6ccc0cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:42:04.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.042+0000 7f6cca59c700 1 --2- 172.21.15.163:0/4200024067 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ccc069560 0x7f6ccc14bc20 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc009ab0 tx=0x7f6cbc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:04.047 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.042+0000 7f6cba7fc700 1 -- 172.21.15.163:0/4200024067 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc008b70 con 0x7f6ccc069560 2023-10-09T06:42:04.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.042+0000 7f6cba7fc700 1 -- 172.21.15.163:0/4200024067 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cbc008cf0 con 0x7f6ccc069560 2023-10-09T06:42:04.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.042+0000 7f6cd0ac1700 1 -- 172.21.15.163:0/4200024067 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ccc16d130 con 0x7f6ccc069560 2023-10-09T06:42:04.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.042+0000 7f6cba7fc700 1 -- 172.21.15.163:0/4200024067 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc0266b0 con 0x7f6ccc069560 2023-10-09T06:42:04.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.042+0000 7f6cd0ac1700 1 -- 172.21.15.163:0/4200024067 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ccc14c7c0 con 0x7f6ccc069560 2023-10-09T06:42:04.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.043+0000 7f6cd0ac1700 1 -- 172.21.15.163:0/4200024067 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ccc16d650 con 0x7f6ccc069560 2023-10-09T06:42:04.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.044+0000 7f6cba7fc700 1 -- 172.21.15.163:0/4200024067 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6cbc003040 con 0x7f6ccc069560 2023-10-09T06:42:04.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.044+0000 7f6cba7fc700 1 --2- 172.21.15.163:0/4200024067 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6cac067320 0x7f6cac0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:04.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.044+0000 7f6cba7fc700 1 -- 172.21.15.163:0/4200024067 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5792+0+0 (secure 0 0 0) 0x7f6cbc086a10 con 0x7f6ccc069560 2023-10-09T06:42:04.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.044+0000 7f6cc9d9b700 1 --2- 172.21.15.163:0/4200024067 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6cac067320 0x7f6cac0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:04.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.045+0000 7f6cc9d9b700 1 --2- 172.21.15.163:0/4200024067 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6cac067320 0x7f6cac0697d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc1590b0 tx=0x7f6cc0007da0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:04.049 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.049+0000 7f6cba7fc700 1 -- 172.21.15.163:0/4200024067 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ccc16d650 con 0x7f6ccc069560 2023-10-09T06:42:04.231 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.231+0000 7f6cd0ac1700 1 -- 172.21.15.163:0/4200024067 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ccc16d650 con 0x7f6ccc069560 2023-10-09T06:42:04.232 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.232+0000 7f6cba7fc700 1 -- 172.21.15.163:0/4200024067 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v199) v1 ==== 74+0+15558 (secure 0 0 0) 0x7f6ccc16d650 con 0x7f6ccc069560 2023-10-09T06:42:04.233 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:42:04.233 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":199,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:40:26.611732+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:39:50.786303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":34,"pool_name":"cephfs_data","create_time":"2023-10-09T06:39:51.411021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":198,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":198,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"34.24","mappings":[{"from":0,"to":2}]},{"pgid":"34.25","mappings":[{"from":0,"to":1}]},{"pgid":"34.30","mappings":[{"from":0,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:42:04.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.234+0000 7f6cd0ac1700 1 -- 172.21.15.163:0/4200024067 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6cac067320 msgr2=0x7f6cac0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:04.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.234+0000 7f6cd0ac1700 1 --2- 172.21.15.163:0/4200024067 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6cac067320 0x7f6cac0697d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc1590b0 tx=0x7f6cc0007da0 comp rx=0 tx=0).stop 2023-10-09T06:42:04.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.234+0000 7f6cd0ac1700 1 -- 172.21.15.163:0/4200024067 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ccc069560 msgr2=0x7f6ccc14bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:04.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.234+0000 7f6cd0ac1700 1 --2- 172.21.15.163:0/4200024067 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ccc069560 0x7f6ccc14bc20 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc009ab0 tx=0x7f6cbc018000 comp rx=0 tx=0).stop 2023-10-09T06:42:04.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.234+0000 7f6cd0ac1700 1 -- 172.21.15.163:0/4200024067 shutdown_connections 2023-10-09T06:42:04.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.234+0000 7f6cd0ac1700 1 --2- 172.21.15.163:0/4200024067 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f6cac067320 0x7f6cac0697d0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:04.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.234+0000 7f6cd0ac1700 1 --2- 172.21.15.163:0/4200024067 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ccc069560 0x7f6ccc14bc20 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:04.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.234+0000 7f6cd0ac1700 1 --2- 172.21.15.163:0/4200024067 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ccc159c60 0x7f6ccc0c5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:04.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.234+0000 7f6cd0ac1700 1 --2- 172.21.15.163:0/4200024067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ccc159890 0x7f6ccc0cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:04.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.234+0000 7f6cd0ac1700 1 -- 172.21.15.163:0/4200024067 >> 172.21.15.163:0/4200024067 conn(0x7f6ccc068e20 msgr2=0x7f6ccc0b0890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:04.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.234+0000 7f6cd0ac1700 1 -- 172.21.15.163:0/4200024067 shutdown_connections 2023-10-09T06:42:04.236 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.234+0000 7f6cd0ac1700 1 -- 172.21.15.163:0/4200024067 wait complete. 2023-10-09T06:42:04.247 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-09T06:30:30.334762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 33, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-09T06:39:50.786303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '196', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 34, 'pool_name': 'cephfs_data', 'create_time': '2023-10-09T06:39:51.411021+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '197', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-09T06:42:04.248 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-09T06:42:04.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.365+0000 7f65ca59f700 1 Processor -- start 2023-10-09T06:42:04.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.366+0000 7f65ca59f700 1 -- start start 2023-10-09T06:42:04.375 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.366+0000 7f65ca59f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f65c40b16d0 0x7f65c40b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:04.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.366+0000 7f65ca59f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65c40b33b0 0x7f65c40b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:04.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.366+0000 7f65ca59f700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65c40b2080 con 0x7f65c40b4e70 2023-10-09T06:42:04.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.366+0000 7f65ca59f700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65c40cfb00 con 0x7f65c40b16d0 2023-10-09T06:42:04.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.366+0000 7f65ca59f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c40b5c40 con 0x7f65c40b33b0 2023-10-09T06:42:04.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.366+0000 7f65c8d9c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f65c40b16d0 0x7f65c40b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:04.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.366+0000 7f65c8d9c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f65c40b16d0 0x7f65c40b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:52226/0 (socket says 172.21.15.163:52226) 2023-10-09T06:42:04.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.366+0000 7f65c8d9c700 1 -- 172.21.15.163:0/2217914105 learned_addr learned my addr 172.21.15.163:0/2217914105 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:04.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.366+0000 7f65c9d9e700 1 --2- 172.21.15.163:0/2217914105 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65c40b33b0 0x7f65c40b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:04.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.366+0000 7f65c8d9c700 1 -- 172.21.15.163:0/2217914105 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65c40b33b0 msgr2=0x7f65c40b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:04.376 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.366+0000 7f65c8d9c700 1 --2- 172.21.15.163:0/2217914105 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65c40b33b0 0x7f65c40b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:04.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.366+0000 7f65c8d9c700 1 -- 172.21.15.163:0/2217914105 >> v1:172.21.15.181:6789/0 conn(0x7f65c40b4e70 legacy=0x7f65c40ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:04.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.366+0000 7f65c8d9c700 1 -- 172.21.15.163:0/2217914105 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c40b5f20 con 0x7f65c40b16d0 2023-10-09T06:42:04.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.366+0000 7f65c8d9c700 1 --2- 172.21.15.163:0/2217914105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65c40b16d0 0x7f65c40b1b40 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f65b40099b0 tx=0x7f65b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f160d81dd4999c67 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:04.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.366+0000 7f65bbfff700 1 -- 172.21.15.163:0/2217914105 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b400b860 con 0x7f65c40b16d0 2023-10-09T06:42:04.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.366+0000 7f65bbfff700 1 -- 172.21.15.163:0/2217914105 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b400b9c0 con 0x7f65c40b16d0 2023-10-09T06:42:04.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.366+0000 7f65bbfff700 1 -- 172.21.15.163:0/2217914105 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b400bd80 con 0x7f65c40b16d0 2023-10-09T06:42:04.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.367+0000 7f65ca59f700 1 -- 172.21.15.163:0/2217914105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65c40b16d0 msgr2=0x7f65c40b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:04.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.367+0000 7f65ca59f700 1 --2- 172.21.15.163:0/2217914105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65c40b16d0 0x7f65c40b1b40 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f65b40099b0 tx=0x7f65b4009250 comp rx=0 tx=0).stop 2023-10-09T06:42:04.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.367+0000 7f65ca59f700 1 -- 172.21.15.163:0/2217914105 shutdown_connections 2023-10-09T06:42:04.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.367+0000 7f65ca59f700 1 --2- 172.21.15.163:0/2217914105 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65c40b33b0 0x7f65c40b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:04.377 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.367+0000 7f65ca59f700 1 --2- 172.21.15.163:0/2217914105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65c40b16d0 0x7f65c40b1b40 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:04.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.367+0000 7f65ca59f700 1 -- 172.21.15.163:0/2217914105 >> 172.21.15.163:0/2217914105 conn(0x7f65c40d10c0 msgr2=0x7f65c40affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:04.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.367+0000 7f65ca59f700 1 -- 172.21.15.163:0/2217914105 shutdown_connections 2023-10-09T06:42:04.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.367+0000 7f65ca59f700 1 -- 172.21.15.163:0/2217914105 wait complete. 2023-10-09T06:42:04.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.368+0000 7f65ca59f700 1 Processor -- start 2023-10-09T06:42:04.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.368+0000 7f65ca59f700 1 -- start start 2023-10-09T06:42:04.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.370+0000 7f65ca59f700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65c40b16d0 0x7f65c40a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:04.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.370+0000 7f65ca59f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65c40b33b0 0x7f65c40c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:04.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.370+0000 7f65ca59f700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65c40b4e70 0x7f65c4169ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:04.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.370+0000 7f65ca59f700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c416ae60 con 0x7f65c40b16d0 2023-10-09T06:42:04.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.370+0000 7f65ca59f700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c40a7440 con 0x7f65c40b4e70 2023-10-09T06:42:04.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.370+0000 7f65ca59f700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c40cac20 con 0x7f65c40b33b0 2023-10-09T06:42:04.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.370+0000 7f65c959d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65c40b16d0 0x7f65c40a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:04.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.370+0000 7f65c959d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65c40b16d0 0x7f65c40a6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:52236/0 (socket says 172.21.15.163:52236) 2023-10-09T06:42:04.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.370+0000 7f65c959d700 1 -- 172.21.15.163:0/2018703567 learned_addr learned my addr 172.21.15.163:0/2018703567 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:04.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.370+0000 7f65c9d9e700 1 --2- 172.21.15.163:0/2018703567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65c40b4e70 0x7f65c4169ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:04.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.370+0000 7f65c8d9c700 1 --2- 172.21.15.163:0/2018703567 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65c40b33b0 0x7f65c40c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:04.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.370+0000 7f65c959d700 1 -- 172.21.15.163:0/2018703567 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65c40b33b0 msgr2=0x7f65c40c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:04.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.370+0000 7f65c959d700 1 --2- 172.21.15.163:0/2018703567 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65c40b33b0 0x7f65c40c91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:04.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.370+0000 7f65c959d700 1 -- 172.21.15.163:0/2018703567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65c40b4e70 msgr2=0x7f65c4169ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:04.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.370+0000 7f65c959d700 1 --2- 172.21.15.163:0/2018703567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65c40b4e70 0x7f65c4169ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:04.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.370+0000 7f65c959d700 1 -- 172.21.15.163:0/2018703567 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b40094f0 con 0x7f65c40b16d0 2023-10-09T06:42:04.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.370+0000 7f65c959d700 1 --2- 172.21.15.163:0/2018703567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65c40b16d0 0x7f65c40a6f00 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f65c000cb10 tx=0x7f65c00017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:04.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.370+0000 7f65b97fa700 1 -- 172.21.15.163:0/2018703567 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c000a760 con 0x7f65c40b16d0 2023-10-09T06:42:04.379 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.370+0000 7f65b97fa700 1 -- 172.21.15.163:0/2018703567 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c000a8e0 con 0x7f65c40b16d0 2023-10-09T06:42:04.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.370+0000 7f65ca59f700 1 -- 172.21.15.163:0/2018703567 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c40caf00 con 0x7f65c40b16d0 2023-10-09T06:42:04.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.370+0000 7f65b97fa700 1 -- 172.21.15.163:0/2018703567 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c000aca0 con 0x7f65c40b16d0 2023-10-09T06:42:04.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.370+0000 7f65ca59f700 1 -- 172.21.15.163:0/2018703567 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c40cb290 con 0x7f65c40b16d0 2023-10-09T06:42:04.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.371+0000 7f65ca59f700 1 -- 172.21.15.163:0/2018703567 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c4088990 con 0x7f65c40b16d0 2023-10-09T06:42:04.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.371+0000 7f65b97fa700 1 -- 172.21.15.163:0/2018703567 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f65c0003030 con 0x7f65c40b16d0 2023-10-09T06:42:04.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.371+0000 7f65b97fa700 1 --2- 172.21.15.163:0/2018703567 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f65a8067320 0x7f65a80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:04.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.372+0000 7f65b97fa700 1 -- 172.21.15.163:0/2018703567 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5792+0+0 (secure 0 0 0) 0x7f65c0084d80 con 0x7f65c40b16d0 2023-10-09T06:42:04.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.372+0000 7f65c8d9c700 1 --2- 172.21.15.163:0/2018703567 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f65a8067320 0x7f65a80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:04.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.372+0000 7f65c8d9c700 1 --2- 172.21.15.163:0/2018703567 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f65a8067320 0x7f65a80697d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f65b40099b0 tx=0x7f65b4017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:04.381 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.376+0000 7f65b97fa700 1 -- 172.21.15.163:0/2018703567 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f65c00519c0 con 0x7f65c40b16d0 2023-10-09T06:42:04.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.559+0000 7f65ca59f700 1 -- 172.21.15.163:0/2018703567 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f65c4069b20 con 0x7f65c40b16d0 2023-10-09T06:42:04.560 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.560+0000 7f65b97fa700 1 -- 172.21.15.163:0/2018703567 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v199) v1 ==== 93+0+10 (secure 0 0 0) 0x7f65c0054fe0 con 0x7f65c40b16d0 2023-10-09T06:42:04.561 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 1 2023-10-09T06:42:04.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.561+0000 7f65ca59f700 1 -- 172.21.15.163:0/2018703567 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f65a8067320 msgr2=0x7f65a80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:04.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.561+0000 7f65ca59f700 1 --2- 172.21.15.163:0/2018703567 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f65a8067320 0x7f65a80697d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f65b40099b0 tx=0x7f65b4017000 comp rx=0 tx=0).stop 2023-10-09T06:42:04.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.561+0000 7f65ca59f700 1 -- 172.21.15.163:0/2018703567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65c40b16d0 msgr2=0x7f65c40a6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:04.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.561+0000 7f65ca59f700 1 --2- 172.21.15.163:0/2018703567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65c40b16d0 0x7f65c40a6f00 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f65c000cb10 tx=0x7f65c00017b0 comp rx=0 tx=0).stop 2023-10-09T06:42:04.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.562+0000 7f65ca59f700 1 -- 172.21.15.163:0/2018703567 shutdown_connections 2023-10-09T06:42:04.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.562+0000 7f65ca59f700 1 --2- 172.21.15.163:0/2018703567 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f65a8067320 0x7f65a80697d0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:04.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.562+0000 7f65ca59f700 1 --2- 172.21.15.163:0/2018703567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65c40b16d0 0x7f65c40a6f00 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:04.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.562+0000 7f65ca59f700 1 --2- 172.21.15.163:0/2018703567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65c40b4e70 0x7f65c4169ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:04.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.562+0000 7f65ca59f700 1 --2- 172.21.15.163:0/2018703567 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65c40b33b0 0x7f65c40c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:04.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.562+0000 7f65ca59f700 1 -- 172.21.15.163:0/2018703567 >> 172.21.15.163:0/2018703567 conn(0x7f65c40d10c0 msgr2=0x7f65c41590c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:04.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.562+0000 7f65ca59f700 1 -- 172.21.15.163:0/2018703567 shutdown_connections 2023-10-09T06:42:04.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.562+0000 7f65ca59f700 1 -- 172.21.15.163:0/2018703567 wait complete. 2023-10-09T06:42:04.575 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-09T06:42:04.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.694+0000 7f1638a6d700 1 Processor -- start 2023-10-09T06:42:04.695 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.694+0000 7f1638a6d700 1 -- start start 2023-10-09T06:42:04.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.695+0000 7f1638a6d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f16340b16d0 0x7f16340b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:04.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.695+0000 7f1638a6d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f16340b33b0 0x7f16340b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:04.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.695+0000 7f1638a6d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16340b2080 con 0x7f16340b4e70 2023-10-09T06:42:04.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.695+0000 7f1638a6d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16340cfb00 con 0x7f16340b16d0 2023-10-09T06:42:04.697 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.695+0000 7f1638a6d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16340b5c40 con 0x7f16340b33b0 2023-10-09T06:42:04.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.695+0000 7f1632ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f16340b16d0 0x7f16340b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:04.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.695+0000 7f1632ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f16340b16d0 0x7f16340b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:52252/0 (socket says 172.21.15.163:52252) 2023-10-09T06:42:04.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.695+0000 7f1632ffd700 1 -- 172.21.15.163:0/1536000731 learned_addr learned my addr 172.21.15.163:0/1536000731 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:04.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.695+0000 7f1633fff700 1 --2- 172.21.15.163:0/1536000731 >> v2:172.21.15.181:3300/0 conn(0x7f16340b33b0 0x7f16340b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:04.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.695+0000 7f1632ffd700 1 -- 172.21.15.163:0/1536000731 >> v2:172.21.15.181:3300/0 conn(0x7f16340b33b0 msgr2=0x7f16340b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:04.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.695+0000 7f1632ffd700 1 --2- 172.21.15.163:0/1536000731 >> v2:172.21.15.181:3300/0 conn(0x7f16340b33b0 0x7f16340b37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:04.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.695+0000 7f1632ffd700 1 -- 172.21.15.163:0/1536000731 >> v1:172.21.15.181:6789/0 conn(0x7f16340b4e70 legacy=0x7f16340ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:04.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.695+0000 7f1632ffd700 1 -- 172.21.15.163:0/1536000731 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16340b5f20 con 0x7f16340b16d0 2023-10-09T06:42:04.698 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.695+0000 7f1632ffd700 1 --2- 172.21.15.163:0/1536000731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16340b16d0 0x7f16340b1b40 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f1634071380 tx=0x7f161c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33a8b0f84188a23 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:04.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.695+0000 7f16327fc700 1 -- 172.21.15.163:0/1536000731 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c00b860 con 0x7f16340b16d0 2023-10-09T06:42:04.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.696+0000 7f16327fc700 1 -- 172.21.15.163:0/1536000731 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161c00b9c0 con 0x7f16340b16d0 2023-10-09T06:42:04.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.696+0000 7f16327fc700 1 -- 172.21.15.163:0/1536000731 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c00bd80 con 0x7f16340b16d0 2023-10-09T06:42:04.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.696+0000 7f1638a6d700 1 -- 172.21.15.163:0/1536000731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16340b16d0 msgr2=0x7f16340b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:04.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.696+0000 7f1638a6d700 1 --2- 172.21.15.163:0/1536000731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16340b16d0 0x7f16340b1b40 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f1634071380 tx=0x7f161c009250 comp rx=0 tx=0).stop 2023-10-09T06:42:04.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.696+0000 7f1638a6d700 1 -- 172.21.15.163:0/1536000731 shutdown_connections 2023-10-09T06:42:04.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.696+0000 7f1638a6d700 1 --2- 172.21.15.163:0/1536000731 >> v2:172.21.15.181:3300/0 conn(0x7f16340b33b0 0x7f16340b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:04.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.696+0000 7f1638a6d700 1 --2- 172.21.15.163:0/1536000731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16340b16d0 0x7f16340b1b40 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:04.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.696+0000 7f1638a6d700 1 -- 172.21.15.163:0/1536000731 >> 172.21.15.163:0/1536000731 conn(0x7f16340d10c0 msgr2=0x7f16340affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:04.699 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.696+0000 7f1638a6d700 1 -- 172.21.15.163:0/1536000731 shutdown_connections 2023-10-09T06:42:04.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.696+0000 7f1638a6d700 1 -- 172.21.15.163:0/1536000731 wait complete. 2023-10-09T06:42:04.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.697+0000 7f1638a6d700 1 Processor -- start 2023-10-09T06:42:04.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.697+0000 7f1638a6d700 1 -- start start 2023-10-09T06:42:04.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.698+0000 7f1638a6d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16340b16d0 0x7f16340cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:04.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.698+0000 7f1638a6d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16340b33b0 0x7f16340cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:04.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.698+0000 7f1638a6d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16340b4e70 0x7f163414a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:04.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.698+0000 7f1638a6d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16341488d0 con 0x7f16340b33b0 2023-10-09T06:42:04.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.698+0000 7f1638a6d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1634148bb0 con 0x7f16340b16d0 2023-10-09T06:42:04.703 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.698+0000 7f1638a6d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1634148e90 con 0x7f16340b4e70 2023-10-09T06:42:04.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.698+0000 7f1632ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16340b33b0 0x7f16340cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:04.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.698+0000 7f1632ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16340b33b0 0x7f16340cbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:52260/0 (socket says 172.21.15.163:52260) 2023-10-09T06:42:04.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.698+0000 7f1632ffd700 1 -- 172.21.15.163:0/3614232895 learned_addr learned my addr 172.21.15.163:0/3614232895 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:04.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.698+0000 7f1633fff700 1 --2- 172.21.15.163:0/3614232895 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16340b4e70 0x7f163414a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:04.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.698+0000 7f1632ffd700 1 -- 172.21.15.163:0/3614232895 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16340b4e70 msgr2=0x7f163414a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:04.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.698+0000 7f16337fe700 1 --2- 172.21.15.163:0/3614232895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16340b16d0 0x7f16340cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:04.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.698+0000 7f1632ffd700 1 --2- 172.21.15.163:0/3614232895 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16340b4e70 0x7f163414a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:04.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.698+0000 7f1632ffd700 1 -- 172.21.15.163:0/3614232895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16340b16d0 msgr2=0x7f16340cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:04.704 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.698+0000 7f1632ffd700 1 --2- 172.21.15.163:0/3614232895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16340b16d0 0x7f16340cb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:04.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.698+0000 7f1632ffd700 1 -- 172.21.15.163:0/3614232895 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161c0094f0 con 0x7f16340b33b0 2023-10-09T06:42:04.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.699+0000 7f1632ffd700 1 --2- 172.21.15.163:0/3614232895 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16340b33b0 0x7f16340cbbf0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f1634070dd0 tx=0x7f161c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:04.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.699+0000 7f162b7fe700 1 -- 172.21.15.163:0/3614232895 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c01f070 con 0x7f16340b33b0 2023-10-09T06:42:04.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.699+0000 7f162b7fe700 1 -- 172.21.15.163:0/3614232895 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161c002f30 con 0x7f16340b33b0 2023-10-09T06:42:04.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.699+0000 7f1638a6d700 1 -- 172.21.15.163:0/3614232895 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1634149170 con 0x7f16340b33b0 2023-10-09T06:42:04.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.699+0000 7f162b7fe700 1 -- 172.21.15.163:0/3614232895 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c011c80 con 0x7f16340b33b0 2023-10-09T06:42:04.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.699+0000 7f1638a6d700 1 -- 172.21.15.163:0/3614232895 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1634149970 con 0x7f16340b33b0 2023-10-09T06:42:04.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.700+0000 7f1638a6d700 1 -- 172.21.15.163:0/3614232895 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1634149690 con 0x7f16340b33b0 2023-10-09T06:42:04.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.700+0000 7f162b7fe700 1 -- 172.21.15.163:0/3614232895 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f161c023070 con 0x7f16340b33b0 2023-10-09T06:42:04.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.701+0000 7f162b7fe700 1 --2- 172.21.15.163:0/3614232895 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1614067320 0x7f16140697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:04.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.701+0000 7f162b7fe700 1 -- 172.21.15.163:0/3614232895 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5792+0+0 (secure 0 0 0) 0x7f161c0860d0 con 0x7f16340b33b0 2023-10-09T06:42:04.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.701+0000 7f16337fe700 1 --2- 172.21.15.163:0/3614232895 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1614067320 0x7f16140697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:04.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.701+0000 7f16337fe700 1 --2- 172.21.15.163:0/3614232895 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1614067320 0x7f16140697d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1624007fd0 tx=0x7f1624009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:04.706 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.705+0000 7f162b7fe700 1 -- 172.21.15.163:0/3614232895 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f161c0b75a0 con 0x7f16340b33b0 2023-10-09T06:42:04.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.888+0000 7f1638a6d700 1 -- 172.21.15.163:0/3614232895 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f163404bb80 con 0x7f16340b33b0 2023-10-09T06:42:04.888 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.888+0000 7f162b7fe700 1 -- 172.21.15.163:0/3614232895 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v199) v1 ==== 104+0+11 (secure 0 0 0) 0x7f161c0150a0 con 0x7f16340b33b0 2023-10-09T06:42:04.888 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 32 2023-10-09T06:42:04.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.890+0000 7f1638a6d700 1 -- 172.21.15.163:0/3614232895 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1614067320 msgr2=0x7f16140697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:04.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.890+0000 7f1638a6d700 1 --2- 172.21.15.163:0/3614232895 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1614067320 0x7f16140697d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1624007fd0 tx=0x7f1624009040 comp rx=0 tx=0).stop 2023-10-09T06:42:04.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.890+0000 7f1638a6d700 1 -- 172.21.15.163:0/3614232895 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16340b33b0 msgr2=0x7f16340cbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:04.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.890+0000 7f1638a6d700 1 --2- 172.21.15.163:0/3614232895 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16340b33b0 0x7f16340cbbf0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f1634070dd0 tx=0x7f161c017000 comp rx=0 tx=0).stop 2023-10-09T06:42:04.890 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.890+0000 7f1638a6d700 1 -- 172.21.15.163:0/3614232895 shutdown_connections 2023-10-09T06:42:04.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.890+0000 7f1638a6d700 1 --2- 172.21.15.163:0/3614232895 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1614067320 0x7f16140697d0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:04.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.890+0000 7f1638a6d700 1 --2- 172.21.15.163:0/3614232895 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16340b4e70 0x7f163414a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:04.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.890+0000 7f1638a6d700 1 --2- 172.21.15.163:0/3614232895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16340b16d0 0x7f16340cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:04.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.890+0000 7f1638a6d700 1 --2- 172.21.15.163:0/3614232895 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16340b33b0 0x7f16340cbbf0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:04.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.890+0000 7f1638a6d700 1 -- 172.21.15.163:0/3614232895 >> 172.21.15.163:0/3614232895 conn(0x7f16340d10c0 msgr2=0x7f16340695b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:04.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.890+0000 7f1638a6d700 1 -- 172.21.15.163:0/3614232895 shutdown_connections 2023-10-09T06:42:04.891 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:04.890+0000 7f1638a6d700 1 -- 172.21.15.163:0/3614232895 wait complete. 2023-10-09T06:42:04.902 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-09T06:42:05.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.020+0000 7f3d9d7b4700 1 Processor -- start 2023-10-09T06:42:05.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.020+0000 7f3d9d7b4700 1 -- start start 2023-10-09T06:42:05.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.021+0000 7f3d9d7b4700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3d98159c60 0x7f3d9815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:05.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.021+0000 7f3d9d7b4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d9815db10 0x7f3d98069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:05.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.021+0000 7f3d9d7b4700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d9815a160 con 0x7f3d98159890 2023-10-09T06:42:05.022 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.021+0000 7f3d9d7b4700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d9815df50 con 0x7f3d98159c60 2023-10-09T06:42:05.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.021+0000 7f3d9d7b4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d980d13a0 con 0x7f3d9815db10 2023-10-09T06:42:05.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.021+0000 7f3d967fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3d98159c60 0x7f3d9815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:05.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.021+0000 7f3d967fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3d98159c60 0x7f3d9815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:52270/0 (socket says 172.21.15.163:52270) 2023-10-09T06:42:05.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.021+0000 7f3d967fc700 1 -- 172.21.15.163:0/1211859836 learned_addr learned my addr 172.21.15.163:0/1211859836 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:05.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.021+0000 7f3d977fe700 1 --2- 172.21.15.163:0/1211859836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d9815db10 0x7f3d98069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:05.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.021+0000 7f3d967fc700 1 -- 172.21.15.163:0/1211859836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d9815db10 msgr2=0x7f3d98069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:05.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.021+0000 7f3d967fc700 1 --2- 172.21.15.163:0/1211859836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d9815db10 0x7f3d98069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:05.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.021+0000 7f3d967fc700 1 -- 172.21.15.163:0/1211859836 >> v1:172.21.15.181:6789/0 conn(0x7f3d98159890 legacy=0x7f3d9815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:05.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.021+0000 7f3d967fc700 1 -- 172.21.15.163:0/1211859836 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9815e140 con 0x7f3d98159c60 2023-10-09T06:42:05.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.021+0000 7f3d967fc700 1 --2- 172.21.15.163:0/1211859836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d98159c60 0x7f3d9815d5d0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c0099b0 tx=0x7f3d8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f9e7a113c4aba1 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:05.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.021+0000 7f3d95ffb700 1 -- 172.21.15.163:0/1211859836 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c00b860 con 0x7f3d98159c60 2023-10-09T06:42:05.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.021+0000 7f3d95ffb700 1 -- 172.21.15.163:0/1211859836 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8c00b9c0 con 0x7f3d98159c60 2023-10-09T06:42:05.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.021+0000 7f3d95ffb700 1 -- 172.21.15.163:0/1211859836 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c00bd60 con 0x7f3d98159c60 2023-10-09T06:42:05.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.022+0000 7f3d9d7b4700 1 -- 172.21.15.163:0/1211859836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d98159c60 msgr2=0x7f3d9815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:05.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.022+0000 7f3d9d7b4700 1 --2- 172.21.15.163:0/1211859836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d98159c60 0x7f3d9815d5d0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c0099b0 tx=0x7f3d8c009250 comp rx=0 tx=0).stop 2023-10-09T06:42:05.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.022+0000 7f3d9d7b4700 1 -- 172.21.15.163:0/1211859836 shutdown_connections 2023-10-09T06:42:05.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.022+0000 7f3d9d7b4700 1 --2- 172.21.15.163:0/1211859836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d9815db10 0x7f3d98069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:05.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.022+0000 7f3d9d7b4700 1 --2- 172.21.15.163:0/1211859836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d98159c60 0x7f3d9815d5d0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:05.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.022+0000 7f3d9d7b4700 1 -- 172.21.15.163:0/1211859836 >> 172.21.15.163:0/1211859836 conn(0x7f3d98068e20 msgr2=0x7f3d980d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:05.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.022+0000 7f3d9d7b4700 1 -- 172.21.15.163:0/1211859836 shutdown_connections 2023-10-09T06:42:05.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.022+0000 7f3d9d7b4700 1 -- 172.21.15.163:0/1211859836 wait complete. 2023-10-09T06:42:05.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.023+0000 7f3d9d7b4700 1 Processor -- start 2023-10-09T06:42:05.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.023+0000 7f3d9d7b4700 1 -- start start 2023-10-09T06:42:05.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.024+0000 7f3d9d7b4700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d98159890 0x7f3d980b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:05.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.024+0000 7f3d9d7b4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d98159c60 0x7f3d980cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:05.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.024+0000 7f3d9d7b4700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d9815db10 0x7f3d9814c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:05.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.024+0000 7f3d9d7b4700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d980b4830 con 0x7f3d98159c60 2023-10-09T06:42:05.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.024+0000 7f3d9d7b4700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d980b3c20 con 0x7f3d9815db10 2023-10-09T06:42:05.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.024+0000 7f3d9d7b4700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d980b3f00 con 0x7f3d98159890 2023-10-09T06:42:05.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.024+0000 7f3d967fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d98159c60 0x7f3d980cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:05.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.024+0000 7f3d967fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d98159c60 0x7f3d980cfa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:52278/0 (socket says 172.21.15.163:52278) 2023-10-09T06:42:05.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.024+0000 7f3d967fc700 1 -- 172.21.15.163:0/649330761 learned_addr learned my addr 172.21.15.163:0/649330761 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:05.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.024+0000 7f3d967fc700 1 -- 172.21.15.163:0/649330761 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d98159890 msgr2=0x7f3d980b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:05.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.024+0000 7f3d977fe700 1 --2- 172.21.15.163:0/649330761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d9815db10 0x7f3d9814c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:05.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.024+0000 7f3d96ffd700 1 --2- 172.21.15.163:0/649330761 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d98159890 0x7f3d980b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:05.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.024+0000 7f3d967fc700 1 --2- 172.21.15.163:0/649330761 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d98159890 0x7f3d980b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:05.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.024+0000 7f3d967fc700 1 -- 172.21.15.163:0/649330761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d9815db10 msgr2=0x7f3d9814c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:05.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.024+0000 7f3d967fc700 1 --2- 172.21.15.163:0/649330761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d9815db10 0x7f3d9814c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:05.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.024+0000 7f3d967fc700 1 -- 172.21.15.163:0/649330761 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8c0094f0 con 0x7f3d98159c60 2023-10-09T06:42:05.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.024+0000 7f3d977fe700 1 --2- 172.21.15.163:0/649330761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d9815db10 0x7f3d9814c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:42:05.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.024+0000 7f3d96ffd700 1 --2- 172.21.15.163:0/649330761 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d98159890 0x7f3d980b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:42:05.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.024+0000 7f3d967fc700 1 --2- 172.21.15.163:0/649330761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d98159c60 0x7f3d980cfa50 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f3d980899c0 tx=0x7f3d8c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:05.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.025+0000 7f3d7effd700 1 -- 172.21.15.163:0/649330761 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c024440 con 0x7f3d98159c60 2023-10-09T06:42:05.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.025+0000 7f3d7effd700 1 -- 172.21.15.163:0/649330761 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8c0245a0 con 0x7f3d98159c60 2023-10-09T06:42:05.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.025+0000 7f3d9d7b4700 1 -- 172.21.15.163:0/649330761 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d980b41e0 con 0x7f3d98159c60 2023-10-09T06:42:05.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.025+0000 7f3d7effd700 1 -- 172.21.15.163:0/649330761 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c024960 con 0x7f3d98159c60 2023-10-09T06:42:05.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.025+0000 7f3d9d7b4700 1 -- 172.21.15.163:0/649330761 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d9814cbb0 con 0x7f3d98159c60 2023-10-09T06:42:05.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.026+0000 7f3d9d7b4700 1 -- 172.21.15.163:0/649330761 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d980b49a0 con 0x7f3d98159c60 2023-10-09T06:42:05.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.026+0000 7f3d7effd700 1 -- 172.21.15.163:0/649330761 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3d8c01a420 con 0x7f3d98159c60 2023-10-09T06:42:05.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.026+0000 7f3d7effd700 1 --2- 172.21.15.163:0/649330761 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3d84067320 0x7f3d840697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:05.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.026+0000 7f3d7effd700 1 -- 172.21.15.163:0/649330761 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5792+0+0 (secure 0 0 0) 0x7f3d8c086ab0 con 0x7f3d98159c60 2023-10-09T06:42:05.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.027+0000 7f3d96ffd700 1 --2- 172.21.15.163:0/649330761 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3d84067320 0x7f3d840697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:05.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.027+0000 7f3d96ffd700 1 --2- 172.21.15.163:0/649330761 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3d84067320 0x7f3d840697d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f3d90001840 tx=0x7f3d90009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:05.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.030+0000 7f3d7effd700 1 -- 172.21.15.163:0/649330761 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d8c0536f0 con 0x7f3d98159c60 2023-10-09T06:42:05.211 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.211+0000 7f3d9d7b4700 1 -- 172.21.15.163:0/649330761 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3d9814cdd0 con 0x7f3d98159c60 2023-10-09T06:42:05.212 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.212+0000 7f3d7effd700 1 -- 172.21.15.163:0/649330761 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v199) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3d8c056d10 con 0x7f3d98159c60 2023-10-09T06:42:05.212 INFO:teuthology.orchestra.run.smithi163.stdout:pg_num: 64 2023-10-09T06:42:05.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.214+0000 7f3d9d7b4700 1 -- 172.21.15.163:0/649330761 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3d84067320 msgr2=0x7f3d840697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:05.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.214+0000 7f3d9d7b4700 1 --2- 172.21.15.163:0/649330761 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3d84067320 0x7f3d840697d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f3d90001840 tx=0x7f3d90009040 comp rx=0 tx=0).stop 2023-10-09T06:42:05.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.214+0000 7f3d9d7b4700 1 -- 172.21.15.163:0/649330761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d98159c60 msgr2=0x7f3d980cfa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:05.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.214+0000 7f3d9d7b4700 1 --2- 172.21.15.163:0/649330761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d98159c60 0x7f3d980cfa50 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f3d980899c0 tx=0x7f3d8c017000 comp rx=0 tx=0).stop 2023-10-09T06:42:05.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.214+0000 7f3d9d7b4700 1 -- 172.21.15.163:0/649330761 shutdown_connections 2023-10-09T06:42:05.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.214+0000 7f3d9d7b4700 1 --2- 172.21.15.163:0/649330761 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3d84067320 0x7f3d840697d0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:05.214 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.214+0000 7f3d9d7b4700 1 --2- 172.21.15.163:0/649330761 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d9815db10 0x7f3d9814c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:05.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.214+0000 7f3d9d7b4700 1 --2- 172.21.15.163:0/649330761 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d98159890 0x7f3d980b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:05.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.214+0000 7f3d9d7b4700 1 --2- 172.21.15.163:0/649330761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d98159c60 0x7f3d980cfa50 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:05.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.214+0000 7f3d9d7b4700 1 -- 172.21.15.163:0/649330761 >> 172.21.15.163:0/649330761 conn(0x7f3d98068e20 msgr2=0x7f3d9814d660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:05.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.214+0000 7f3d9d7b4700 1 -- 172.21.15.163:0/649330761 shutdown_connections 2023-10-09T06:42:05.215 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.214+0000 7f3d9d7b4700 1 -- 172.21.15.163:0/649330761 wait complete. 2023-10-09T06:42:05.226 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:42:05.341 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.341+0000 7f78872ae700 1 Processor -- start 2023-10-09T06:42:05.341 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.341+0000 7f78872ae700 1 -- start start 2023-10-09T06:42:05.343 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.342+0000 7f78872ae700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7880157a50 0x7f788006e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:05.344 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.342+0000 7f78872ae700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78800b00f0 con 0x7f78800af5c0 2023-10-09T06:42:05.344 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.342+0000 7f78872ae700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f788006ebe0 con 0x7f7880157680 2023-10-09T06:42:05.344 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.342+0000 7f78872ae700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f788006eec0 con 0x7f7880157a50 2023-10-09T06:42:05.344 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.342+0000 7f7885aab700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7880157a50 0x7f788006e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:05.344 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.342+0000 7f7886aad700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f78800af5c0 0x7f78800b0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:54050/0 (socket says 172.21.15.163:54050) 2023-10-09T06:42:05.344 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.342+0000 7f7886aad700 1 -- 172.21.15.163:0/988563154 learned_addr learned my addr 172.21.15.163:0/988563154 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:42:05.344 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.342+0000 7f78852aa700 1 -- 172.21.15.163:0/988563154 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 234927448 0 0) 0x7f78800b00f0 con 0x7f78800af5c0 2023-10-09T06:42:05.344 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.342+0000 7f78852aa700 1 -- 172.21.15.163:0/988563154 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f786c003570 con 0x7f78800af5c0 2023-10-09T06:42:05.344 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.342+0000 7f7885aab700 1 -- 172.21.15.163:0/988563154 >> v1:172.21.15.181:6789/0 conn(0x7f7880157680 legacy=0x7f78800aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:05.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.342+0000 7f7885aab700 1 -- 172.21.15.163:0/988563154 >> v1:172.21.15.163:6789/0 conn(0x7f78800af5c0 legacy=0x7f78800b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:05.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.342+0000 7f7885aab700 1 -- 172.21.15.163:0/988563154 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788006f1a0 con 0x7f7880157a50 2023-10-09T06:42:05.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.343+0000 7f7885aab700 1 --2- 172.21.15.163:0/988563154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7880157a50 0x7f788006e570 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f78700098d0 tx=0x7f7870009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4249f1e1019c36 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:05.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.343+0000 7f78852aa700 1 -- 172.21.15.163:0/988563154 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870005c10 con 0x7f7880157a50 2023-10-09T06:42:05.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.343+0000 7f78852aa700 1 -- 172.21.15.163:0/988563154 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7870005d70 con 0x7f7880157a50 2023-10-09T06:42:05.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.343+0000 7f78852aa700 1 -- 172.21.15.163:0/988563154 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870006280 con 0x7f7880157a50 2023-10-09T06:42:05.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.343+0000 7f78872ae700 1 -- 172.21.15.163:0/988563154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7880157a50 msgr2=0x7f788006e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:05.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.343+0000 7f78872ae700 1 --2- 172.21.15.163:0/988563154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7880157a50 0x7f788006e570 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f78700098d0 tx=0x7f7870009120 comp rx=0 tx=0).stop 2023-10-09T06:42:05.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.343+0000 7f78872ae700 1 -- 172.21.15.163:0/988563154 shutdown_connections 2023-10-09T06:42:05.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.343+0000 7f78872ae700 1 --2- 172.21.15.163:0/988563154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7880157a50 0x7f788006e570 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:05.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.343+0000 7f78872ae700 1 -- 172.21.15.163:0/988563154 >> 172.21.15.163:0/988563154 conn(0x7f788006b450 msgr2=0x7f78800cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:05.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.343+0000 7f78872ae700 1 -- 172.21.15.163:0/988563154 shutdown_connections 2023-10-09T06:42:05.347 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.343+0000 7f78872ae700 1 -- 172.21.15.163:0/988563154 wait complete. 2023-10-09T06:42:05.349 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.344+0000 7f78872ae700 1 Processor -- start 2023-10-09T06:42:05.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.345+0000 7f78872ae700 1 -- start start 2023-10-09T06:42:05.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.345+0000 7f78872ae700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78800af5c0 0x7f78800a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:05.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.345+0000 7f78872ae700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7880157680 0x7f7880167f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:05.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.345+0000 7f78872ae700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7880157a50 0x7f78801669e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:05.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.345+0000 7f78872ae700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78801684a0 con 0x7f7880157680 2023-10-09T06:42:05.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.345+0000 7f78872ae700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78800a5e90 con 0x7f78800af5c0 2023-10-09T06:42:05.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.345+0000 7f78872ae700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78800a6170 con 0x7f7880157a50 2023-10-09T06:42:05.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.345+0000 7f7885aab700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7880157680 0x7f7880167f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:05.350 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.345+0000 7f7885aab700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7880157680 0x7f7880167f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:52288/0 (socket says 172.21.15.163:52288) 2023-10-09T06:42:05.351 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.345+0000 7f7885aab700 1 -- 172.21.15.163:0/2002002431 learned_addr learned my addr 172.21.15.163:0/2002002431 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:05.351 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.346+0000 7f7886aad700 1 --2- 172.21.15.163:0/2002002431 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7880157a50 0x7f78801669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:05.351 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.346+0000 7f78862ac700 1 --2- 172.21.15.163:0/2002002431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78800af5c0 0x7f78800a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:05.351 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.346+0000 7f7885aab700 1 -- 172.21.15.163:0/2002002431 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7880157a50 msgr2=0x7f78801669e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:05.351 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.346+0000 7f7885aab700 1 --2- 172.21.15.163:0/2002002431 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7880157a50 0x7f78801669e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:05.351 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.346+0000 7f7885aab700 1 -- 172.21.15.163:0/2002002431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78800af5c0 msgr2=0x7f78800a58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:05.351 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.346+0000 7f7885aab700 1 --2- 172.21.15.163:0/2002002431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78800af5c0 0x7f78800a58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:05.351 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.346+0000 7f7885aab700 1 -- 172.21.15.163:0/2002002431 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78700093c0 con 0x7f7880157680 2023-10-09T06:42:05.351 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.346+0000 7f7885aab700 1 --2- 172.21.15.163:0/2002002431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7880157680 0x7f7880167f60 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f7870005fd0 tx=0x7f7870016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:05.352 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.346+0000 7f78767fc700 1 -- 172.21.15.163:0/2002002431 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787001d880 con 0x7f7880157680 2023-10-09T06:42:05.352 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.346+0000 7f78767fc700 1 -- 172.21.15.163:0/2002002431 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787001da20 con 0x7f7880157680 2023-10-09T06:42:05.352 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.346+0000 7f78872ae700 1 -- 172.21.15.163:0/2002002431 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78800a6450 con 0x7f7880157680 2023-10-09T06:42:05.352 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.346+0000 7f78767fc700 1 -- 172.21.15.163:0/2002002431 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787001dde0 con 0x7f7880157680 2023-10-09T06:42:05.352 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.346+0000 7f78872ae700 1 -- 172.21.15.163:0/2002002431 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78800a6a10 con 0x7f7880157680 2023-10-09T06:42:05.352 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.347+0000 7f78872ae700 1 -- 172.21.15.163:0/2002002431 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78800a6c00 con 0x7f7880157680 2023-10-09T06:42:05.352 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.347+0000 7f78767fc700 1 -- 172.21.15.163:0/2002002431 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f787001b690 con 0x7f7880157680 2023-10-09T06:42:05.352 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.348+0000 7f78767fc700 1 --2- 172.21.15.163:0/2002002431 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7864067320 0x7f78640697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:05.352 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.348+0000 7f78767fc700 1 -- 172.21.15.163:0/2002002431 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5792+0+0 (secure 0 0 0) 0x7f78700869c0 con 0x7f7880157680 2023-10-09T06:42:05.352 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.348+0000 7f78862ac700 1 --2- 172.21.15.163:0/2002002431 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7864067320 0x7f78640697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:05.353 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.348+0000 7f78862ac700 1 --2- 172.21.15.163:0/2002002431 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7864067320 0x7f78640697d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f787c003860 tx=0x7f787c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:05.353 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.352+0000 7f78767fc700 1 -- 172.21.15.163:0/2002002431 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7870053600 con 0x7f7880157680 2023-10-09T06:42:05.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.578+0000 7f78872ae700 1 -- 172.21.15.163:0/2002002431 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f788004c3c0 con 0x7f7880157680 2023-10-09T06:42:05.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.579+0000 7f78767fc700 1 -- 172.21.15.163:0/2002002431 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+4619 (secure 0 0 0) 0x7f7870056c20 con 0x7f7880157680 2023-10-09T06:42:05.579 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:42:05.579 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":133,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7046,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3374277959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3374277959},{"type":"v1","addr":"172.21.15.181:6837","nonce":3374277959}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3834979450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3834979450},{"type":"v1","addr":"172.21.15.181:6836","nonce":3834979450}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":8991,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2681808199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2681808199},{"type":"v1","addr":"172.21.15.163:6834","nonce":2681808199}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128}],"filesystems":[{"mdsmap":{"epoch":132,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:39:52.456713+0000","modified":"2023-10-09T06:39:56.092448+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8994":{"gid":8994,"name":"a","rank":0,"incarnation":129,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/2564044136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2564044136},{"type":"v1","addr":"172.21.15.163:6835","nonce":2564044136}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-09T06:42:05.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.581+0000 7f78872ae700 1 -- 172.21.15.163:0/2002002431 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7864067320 msgr2=0x7f78640697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:05.581 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.581+0000 7f78872ae700 1 --2- 172.21.15.163:0/2002002431 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7864067320 0x7f78640697d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f787c003860 tx=0x7f787c009040 comp rx=0 tx=0).stop 2023-10-09T06:42:05.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.581+0000 7f78872ae700 1 -- 172.21.15.163:0/2002002431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7880157680 msgr2=0x7f7880167f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:05.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.581+0000 7f78872ae700 1 --2- 172.21.15.163:0/2002002431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7880157680 0x7f7880167f60 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f7870005fd0 tx=0x7f7870016000 comp rx=0 tx=0).stop 2023-10-09T06:42:05.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.581+0000 7f78872ae700 1 -- 172.21.15.163:0/2002002431 shutdown_connections 2023-10-09T06:42:05.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.581+0000 7f78872ae700 1 --2- 172.21.15.163:0/2002002431 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f7864067320 0x7f78640697d0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:05.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.581+0000 7f78872ae700 1 --2- 172.21.15.163:0/2002002431 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7880157a50 0x7f78801669e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:05.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.581+0000 7f78872ae700 1 --2- 172.21.15.163:0/2002002431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78800af5c0 0x7f78800a58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:05.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.581+0000 7f78872ae700 1 --2- 172.21.15.163:0/2002002431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7880157680 0x7f7880167f60 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:05.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.581+0000 7f78872ae700 1 -- 172.21.15.163:0/2002002431 >> 172.21.15.163:0/2002002431 conn(0x7f788006b450 msgr2=0x7f7880159ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:05.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.581+0000 7f78872ae700 1 -- 172.21.15.163:0/2002002431 shutdown_connections 2023-10-09T06:42:05.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.581+0000 7f78872ae700 1 -- 172.21.15.163:0/2002002431 wait complete. 2023-10-09T06:42:05.583 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 133 2023-10-09T06:42:05.593 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:42:05.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.707+0000 7f0c59701700 1 Processor -- start 2023-10-09T06:42:05.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.707+0000 7f0c59701700 1 -- start start 2023-10-09T06:42:05.709 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.708+0000 7f0c59701700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0c54159890 0x7f0c5415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:05.710 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.708+0000 7f0c59701700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c54069560 0x7f0c540699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:05.710 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.708+0000 7f0c59701700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c54069ee0 con 0x7f0c54159c60 2023-10-09T06:42:05.710 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.708+0000 7f0c59701700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c5415a160 con 0x7f0c54159890 2023-10-09T06:42:05.710 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.708+0000 7f0c59701700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c540d13a0 con 0x7f0c54069560 2023-10-09T06:42:05.710 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.708+0000 7f0c52ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0c54159890 0x7f0c5415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:05.710 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.708+0000 7f0c52ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0c54159890 0x7f0c5415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:52298/0 (socket says 172.21.15.163:52298) 2023-10-09T06:42:05.710 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.708+0000 7f0c52ffd700 1 -- 172.21.15.163:0/2196717044 learned_addr learned my addr 172.21.15.163:0/2196717044 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:05.710 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.708+0000 7f0c537fe700 1 --2- 172.21.15.163:0/2196717044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c54069560 0x7f0c540699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:05.710 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.708+0000 7f0c52ffd700 1 -- 172.21.15.163:0/2196717044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c54069560 msgr2=0x7f0c540699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:05.710 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.708+0000 7f0c52ffd700 1 --2- 172.21.15.163:0/2196717044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c54069560 0x7f0c540699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:05.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.708+0000 7f0c52ffd700 1 -- 172.21.15.163:0/2196717044 >> v1:172.21.15.181:6789/0 conn(0x7f0c54159c60 legacy=0x7f0c5415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:05.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.709+0000 7f0c52ffd700 1 -- 172.21.15.163:0/2196717044 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5415e140 con 0x7f0c54159890 2023-10-09T06:42:05.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.709+0000 7f0c52ffd700 1 --2- 172.21.15.163:0/2196717044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c54159890 0x7f0c5415abd0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f0c440099b0 tx=0x7f0c44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90cf414fea807aa server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:05.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.709+0000 7f0c527fc700 1 -- 172.21.15.163:0/2196717044 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4400b860 con 0x7f0c54159890 2023-10-09T06:42:05.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.709+0000 7f0c527fc700 1 -- 172.21.15.163:0/2196717044 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c4400b9c0 con 0x7f0c54159890 2023-10-09T06:42:05.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.709+0000 7f0c527fc700 1 -- 172.21.15.163:0/2196717044 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4400bd60 con 0x7f0c54159890 2023-10-09T06:42:05.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.709+0000 7f0c59701700 1 -- 172.21.15.163:0/2196717044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c54159890 msgr2=0x7f0c5415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:05.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.709+0000 7f0c59701700 1 --2- 172.21.15.163:0/2196717044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c54159890 0x7f0c5415abd0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f0c440099b0 tx=0x7f0c44009250 comp rx=0 tx=0).stop 2023-10-09T06:42:05.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.709+0000 7f0c59701700 1 -- 172.21.15.163:0/2196717044 shutdown_connections 2023-10-09T06:42:05.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.709+0000 7f0c59701700 1 --2- 172.21.15.163:0/2196717044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c54069560 0x7f0c540699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:05.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.709+0000 7f0c59701700 1 --2- 172.21.15.163:0/2196717044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c54159890 0x7f0c5415abd0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:05.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.709+0000 7f0c59701700 1 -- 172.21.15.163:0/2196717044 >> 172.21.15.163:0/2196717044 conn(0x7f0c54068e20 msgr2=0x7f0c540d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:05.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.709+0000 7f0c59701700 1 -- 172.21.15.163:0/2196717044 shutdown_connections 2023-10-09T06:42:05.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.709+0000 7f0c59701700 1 -- 172.21.15.163:0/2196717044 wait complete. 2023-10-09T06:42:05.714 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.711+0000 7f0c59701700 1 Processor -- start 2023-10-09T06:42:05.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.711+0000 7f0c59701700 1 -- start start 2023-10-09T06:42:05.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.711+0000 7f0c59701700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c54069560 0x7f0c5414cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:05.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.711+0000 7f0c59701700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c54159890 0x7f0c540c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:05.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.711+0000 7f0c59701700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c54159c60 0x7f0c541524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:05.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.711+0000 7f0c59701700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5414d330 con 0x7f0c54159c60 2023-10-09T06:42:05.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.711+0000 7f0c59701700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c540ca380 con 0x7f0c54069560 2023-10-09T06:42:05.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.711+0000 7f0c59701700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c540c8b00 con 0x7f0c54159890 2023-10-09T06:42:05.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.712+0000 7f0c537fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c54159c60 0x7f0c541524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:05.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.712+0000 7f0c537fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c54159c60 0x7f0c541524b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:52314/0 (socket says 172.21.15.163:52314) 2023-10-09T06:42:05.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.712+0000 7f0c537fe700 1 -- 172.21.15.163:0/1809409831 learned_addr learned my addr 172.21.15.163:0/1809409831 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:05.715 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.712+0000 7f0c52ffd700 1 --2- 172.21.15.163:0/1809409831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c54069560 0x7f0c5414cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:05.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.712+0000 7f0c4a7fc700 1 --2- 172.21.15.163:0/1809409831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c54159890 0x7f0c540c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:05.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.712+0000 7f0c537fe700 1 -- 172.21.15.163:0/1809409831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c54159890 msgr2=0x7f0c540c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:05.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.712+0000 7f0c537fe700 1 --2- 172.21.15.163:0/1809409831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c54159890 0x7f0c540c8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:05.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.712+0000 7f0c537fe700 1 -- 172.21.15.163:0/1809409831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c54069560 msgr2=0x7f0c5414cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:05.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.712+0000 7f0c537fe700 1 --2- 172.21.15.163:0/1809409831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c54069560 0x7f0c5414cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:05.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.712+0000 7f0c537fe700 1 -- 172.21.15.163:0/1809409831 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c440094f0 con 0x7f0c54159c60 2023-10-09T06:42:05.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.712+0000 7f0c537fe700 1 --2- 172.21.15.163:0/1809409831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c54159c60 0x7f0c541524b0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c007f20 tx=0x7f0c4c009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:05.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.712+0000 7f0c4b7fe700 1 -- 172.21.15.163:0/1809409831 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c00eb90 con 0x7f0c54159c60 2023-10-09T06:42:05.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.712+0000 7f0c4b7fe700 1 -- 172.21.15.163:0/1809409831 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c4c00ed10 con 0x7f0c54159c60 2023-10-09T06:42:05.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.712+0000 7f0c4b7fe700 1 -- 172.21.15.163:0/1809409831 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c014670 con 0x7f0c54159c60 2023-10-09T06:42:05.716 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.712+0000 7f0c59701700 1 -- 172.21.15.163:0/1809409831 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c540c9770 con 0x7f0c54159c60 2023-10-09T06:42:05.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.712+0000 7f0c59701700 1 -- 172.21.15.163:0/1809409831 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c540c9d30 con 0x7f0c54159c60 2023-10-09T06:42:05.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.713+0000 7f0c59701700 1 -- 172.21.15.163:0/1809409831 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c540ca530 con 0x7f0c54159c60 2023-10-09T06:42:05.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.714+0000 7f0c4b7fe700 1 -- 172.21.15.163:0/1809409831 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0c4c0149e0 con 0x7f0c54159c60 2023-10-09T06:42:05.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.714+0000 7f0c4b7fe700 1 --2- 172.21.15.163:0/1809409831 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0c34067320 0x7f0c340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:05.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.714+0000 7f0c4b7fe700 1 -- 172.21.15.163:0/1809409831 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5792+0+0 (secure 0 0 0) 0x7f0c4c085290 con 0x7f0c54159c60 2023-10-09T06:42:05.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.714+0000 7f0c52ffd700 1 --2- 172.21.15.163:0/1809409831 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0c34067320 0x7f0c340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:05.717 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.715+0000 7f0c52ffd700 1 --2- 172.21.15.163:0/1809409831 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0c34067320 0x7f0c340697d0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f0c440099b0 tx=0x7f0c44017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:05.718 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.718+0000 7f0c4b7fe700 1 -- 172.21.15.163:0/1809409831 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c540ca530 con 0x7f0c54159c60 2023-10-09T06:42:05.903 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.903+0000 7f0c59701700 1 -- 172.21.15.163:0/1809409831 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c540ca710 con 0x7f0c54159c60 2023-10-09T06:42:05.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.904+0000 7f0c4b7fe700 1 -- 172.21.15.163:0/1809409831 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v199) v1 ==== 74+0+15558 (secure 0 0 0) 0x7f0c4c051ed0 con 0x7f0c54159c60 2023-10-09T06:42:05.904 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:42:05.905 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":199,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:40:26.611732+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:39:50.786303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":34,"pool_name":"cephfs_data","create_time":"2023-10-09T06:39:51.411021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":198,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":198,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"34.24","mappings":[{"from":0,"to":2}]},{"pgid":"34.25","mappings":[{"from":0,"to":1}]},{"pgid":"34.30","mappings":[{"from":0,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:42:05.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.907+0000 7f0c59701700 1 -- 172.21.15.163:0/1809409831 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0c34067320 msgr2=0x7f0c340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:05.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.907+0000 7f0c59701700 1 --2- 172.21.15.163:0/1809409831 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0c34067320 0x7f0c340697d0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f0c440099b0 tx=0x7f0c44017000 comp rx=0 tx=0).stop 2023-10-09T06:42:05.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.907+0000 7f0c59701700 1 -- 172.21.15.163:0/1809409831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c54159c60 msgr2=0x7f0c541524b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:05.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.907+0000 7f0c59701700 1 --2- 172.21.15.163:0/1809409831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c54159c60 0x7f0c541524b0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c007f20 tx=0x7f0c4c009860 comp rx=0 tx=0).stop 2023-10-09T06:42:05.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.907+0000 7f0c59701700 1 -- 172.21.15.163:0/1809409831 shutdown_connections 2023-10-09T06:42:05.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.907+0000 7f0c59701700 1 --2- 172.21.15.163:0/1809409831 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f0c34067320 0x7f0c340697d0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:05.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.907+0000 7f0c59701700 1 --2- 172.21.15.163:0/1809409831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c54159c60 0x7f0c541524b0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:05.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.907+0000 7f0c59701700 1 --2- 172.21.15.163:0/1809409831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c54069560 0x7f0c5414cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:05.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.907+0000 7f0c59701700 1 --2- 172.21.15.163:0/1809409831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c54159890 0x7f0c540c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:05.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.907+0000 7f0c59701700 1 -- 172.21.15.163:0/1809409831 >> 172.21.15.163:0/1809409831 conn(0x7f0c54068e20 msgr2=0x7f0c5415c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:05.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.907+0000 7f0c59701700 1 -- 172.21.15.163:0/1809409831 shutdown_connections 2023-10-09T06:42:05.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:05.907+0000 7f0c59701700 1 -- 172.21.15.163:0/1809409831 wait complete. 2023-10-09T06:42:05.919 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-09T06:42:05.919 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:42:06.034 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.034+0000 7f727fff3700 1 Processor -- start 2023-10-09T06:42:06.034 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.034+0000 7f727fff3700 1 -- start start 2023-10-09T06:42:06.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.035+0000 7f727fff3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f727806bcd0 0x7f72780caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:06.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.035+0000 7f727fff3700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f727806c0a0 0x7f727815a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:06.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.035+0000 7f727fff3700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f727815d7e0 con 0x7f727815df50 2023-10-09T06:42:06.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.035+0000 7f727fff3700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f727815dac0 con 0x7f727806c0a0 2023-10-09T06:42:06.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.035+0000 7f727fff3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f727815aa30 con 0x7f727806bcd0 2023-10-09T06:42:06.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.035+0000 7f727e590700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f727815df50 0x7f72780c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:54062/0 (socket says 172.21.15.163:54062) 2023-10-09T06:42:06.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.035+0000 7f727e590700 1 -- 172.21.15.163:0/4193423500 learned_addr learned my addr 172.21.15.163:0/4193423500 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:42:06.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.035+0000 7f727dd8f700 1 --2- 172.21.15.163:0/4193423500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f727806bcd0 0x7f72780caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:06.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.035+0000 7f727d58e700 1 --2- 172.21.15.163:0/4193423500 >> v2:172.21.15.181:3300/0 conn(0x7f727806c0a0 0x7f727815a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:06.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.035+0000 7f727cd8d700 1 -- 172.21.15.163:0/4193423500 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 970980690 0 0) 0x7f727815d7e0 con 0x7f727815df50 2023-10-09T06:42:06.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.035+0000 7f727cd8d700 1 -- 172.21.15.163:0/4193423500 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7264003570 con 0x7f727815df50 2023-10-09T06:42:06.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.036+0000 7f727dd8f700 1 -- 172.21.15.163:0/4193423500 >> v2:172.21.15.181:3300/0 conn(0x7f727806c0a0 msgr2=0x7f727815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:06.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.036+0000 7f727dd8f700 1 --2- 172.21.15.163:0/4193423500 >> v2:172.21.15.181:3300/0 conn(0x7f727806c0a0 0x7f727815a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.036+0000 7f727dd8f700 1 -- 172.21.15.163:0/4193423500 >> v1:172.21.15.163:6789/0 conn(0x7f727815df50 legacy=0x7f72780c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:06.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.036+0000 7f727dd8f700 1 -- 172.21.15.163:0/4193423500 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727815ad10 con 0x7f727806bcd0 2023-10-09T06:42:06.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.036+0000 7f727d58e700 1 --2- 172.21.15.163:0/4193423500 >> v2:172.21.15.181:3300/0 conn(0x7f727806c0a0 0x7f727815a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:42:06.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.036+0000 7f727dd8f700 1 --2- 172.21.15.163:0/4193423500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f727806bcd0 0x7f72780caa90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7274009e60 tx=0x7f7274009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d986ae97cc49af8e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:06.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.036+0000 7f727cd8d700 1 -- 172.21.15.163:0/4193423500 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727400e070 con 0x7f727806bcd0 2023-10-09T06:42:06.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.036+0000 7f727cd8d700 1 -- 172.21.15.163:0/4193423500 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727400bab0 con 0x7f727806bcd0 2023-10-09T06:42:06.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.036+0000 7f727cd8d700 1 -- 172.21.15.163:0/4193423500 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727400be50 con 0x7f727806bcd0 2023-10-09T06:42:06.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.036+0000 7f727fff3700 1 -- 172.21.15.163:0/4193423500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f727806bcd0 msgr2=0x7f72780caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:06.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.036+0000 7f727fff3700 1 --2- 172.21.15.163:0/4193423500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f727806bcd0 0x7f72780caa90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7274009e60 tx=0x7f7274009780 comp rx=0 tx=0).stop 2023-10-09T06:42:06.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.037+0000 7f727fff3700 1 -- 172.21.15.163:0/4193423500 shutdown_connections 2023-10-09T06:42:06.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.037+0000 7f727fff3700 1 --2- 172.21.15.163:0/4193423500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f727806bcd0 0x7f72780caa90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.037+0000 7f727fff3700 1 --2- 172.21.15.163:0/4193423500 >> v2:172.21.15.181:3300/0 conn(0x7f727806c0a0 0x7f727815a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.037+0000 7f727fff3700 1 -- 172.21.15.163:0/4193423500 >> 172.21.15.163:0/4193423500 conn(0x7f72780ce600 msgr2=0x7f72780cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:06.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.037+0000 7f727fff3700 1 -- 172.21.15.163:0/4193423500 shutdown_connections 2023-10-09T06:42:06.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.037+0000 7f727fff3700 1 -- 172.21.15.163:0/4193423500 wait complete. 2023-10-09T06:42:06.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.038+0000 7f727fff3700 1 Processor -- start 2023-10-09T06:42:06.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.038+0000 7f727fff3700 1 -- start start 2023-10-09T06:42:06.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.038+0000 7f727fff3700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f727806bcd0 0x7f7278154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:06.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.038+0000 7f727fff3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f727806c0a0 0x7f72780b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:06.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.038+0000 7f727fff3700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f727815df50 0x7f727806cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:06.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.038+0000 7f727fff3700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72780c7b70 con 0x7f727806c0a0 2023-10-09T06:42:06.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.038+0000 7f727fff3700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72780b1d80 con 0x7f727815df50 2023-10-09T06:42:06.042 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.038+0000 7f727fff3700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72780b2060 con 0x7f727806bcd0 2023-10-09T06:42:06.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.039+0000 7f727d58e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f727806c0a0 0x7f72780b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:06.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.039+0000 7f727d58e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f727806c0a0 0x7f72780b13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:52322/0 (socket says 172.21.15.163:52322) 2023-10-09T06:42:06.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.039+0000 7f727d58e700 1 -- 172.21.15.163:0/3759084344 learned_addr learned my addr 172.21.15.163:0/3759084344 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:06.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.039+0000 7f727dd8f700 1 --2- 172.21.15.163:0/3759084344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f727806bcd0 0x7f7278154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:06.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.039+0000 7f727e590700 1 --2- 172.21.15.163:0/3759084344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f727815df50 0x7f727806cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:06.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.039+0000 7f727d58e700 1 -- 172.21.15.163:0/3759084344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f727806bcd0 msgr2=0x7f7278154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:06.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.039+0000 7f727d58e700 1 --2- 172.21.15.163:0/3759084344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f727806bcd0 0x7f7278154b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.039+0000 7f727d58e700 1 -- 172.21.15.163:0/3759084344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f727815df50 msgr2=0x7f727806cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:06.043 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.039+0000 7f727d58e700 1 --2- 172.21.15.163:0/3759084344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f727815df50 0x7f727806cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.039+0000 7f727d58e700 1 -- 172.21.15.163:0/3759084344 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7268009420 con 0x7f727806c0a0 2023-10-09T06:42:06.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.039+0000 7f727d58e700 1 --2- 172.21.15.163:0/3759084344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f727806c0a0 0x7f72780b13a0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f726800ec80 tx=0x7f72680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:06.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.039+0000 7f726dffb700 1 -- 172.21.15.163:0/3759084344 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726800c850 con 0x7f727806c0a0 2023-10-09T06:42:06.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.039+0000 7f726dffb700 1 -- 172.21.15.163:0/3759084344 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f726800c9b0 con 0x7f727806c0a0 2023-10-09T06:42:06.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.039+0000 7f727fff3700 1 -- 172.21.15.163:0/3759084344 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7274009a20 con 0x7f727806c0a0 2023-10-09T06:42:06.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.039+0000 7f726dffb700 1 -- 172.21.15.163:0/3759084344 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726800cd30 con 0x7f727806c0a0 2023-10-09T06:42:06.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.039+0000 7f727fff3700 1 -- 172.21.15.163:0/3759084344 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72780b2620 con 0x7f727806c0a0 2023-10-09T06:42:06.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.040+0000 7f727fff3700 1 -- 172.21.15.163:0/3759084344 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f727804c3c0 con 0x7f727806c0a0 2023-10-09T06:42:06.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.041+0000 7f726dffb700 1 -- 172.21.15.163:0/3759084344 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7268013680 con 0x7f727806c0a0 2023-10-09T06:42:06.044 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.041+0000 7f726dffb700 1 --2- 172.21.15.163:0/3759084344 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f725c067320 0x7f725c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:06.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.041+0000 7f726dffb700 1 -- 172.21.15.163:0/3759084344 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5792+0+0 (secure 0 0 0) 0x7f7268085200 con 0x7f727806c0a0 2023-10-09T06:42:06.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.041+0000 7f727dd8f700 1 --2- 172.21.15.163:0/3759084344 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f725c067320 0x7f725c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:06.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.042+0000 7f727dd8f700 1 --2- 172.21.15.163:0/3759084344 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f725c067320 0x7f725c0697d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f72780b4620 tx=0x7f7274019000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:06.045 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.045+0000 7f726dffb700 1 -- 172.21.15.163:0/3759084344 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72680558d0 con 0x7f727806c0a0 2023-10-09T06:42:06.268 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.268+0000 7f727fff3700 1 -- 172.21.15.163:0/3759084344 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72780c7cb0 con 0x7f727806c0a0 2023-10-09T06:42:06.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.269+0000 7f726dffb700 1 -- 172.21.15.163:0/3759084344 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+4619 (secure 0 0 0) 0x7f7268055460 con 0x7f727806c0a0 2023-10-09T06:42:06.270 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:42:06.270 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":133,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7046,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3374277959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3374277959},{"type":"v1","addr":"172.21.15.181:6837","nonce":3374277959}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3834979450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3834979450},{"type":"v1","addr":"172.21.15.181:6836","nonce":3834979450}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":8991,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2681808199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2681808199},{"type":"v1","addr":"172.21.15.163:6834","nonce":2681808199}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128}],"filesystems":[{"mdsmap":{"epoch":132,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:39:52.456713+0000","modified":"2023-10-09T06:39:56.092448+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8994":{"gid":8994,"name":"a","rank":0,"incarnation":129,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/2564044136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2564044136},{"type":"v1","addr":"172.21.15.163:6835","nonce":2564044136}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-09T06:42:06.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.271+0000 7f727fff3700 1 -- 172.21.15.163:0/3759084344 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f725c067320 msgr2=0x7f725c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:06.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.271+0000 7f727fff3700 1 --2- 172.21.15.163:0/3759084344 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f725c067320 0x7f725c0697d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f72780b4620 tx=0x7f7274019000 comp rx=0 tx=0).stop 2023-10-09T06:42:06.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.271+0000 7f727fff3700 1 -- 172.21.15.163:0/3759084344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f727806c0a0 msgr2=0x7f72780b13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:06.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.271+0000 7f727fff3700 1 --2- 172.21.15.163:0/3759084344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f727806c0a0 0x7f72780b13a0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f726800ec80 tx=0x7f72680091a0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.271+0000 7f727fff3700 1 -- 172.21.15.163:0/3759084344 shutdown_connections 2023-10-09T06:42:06.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.271+0000 7f727fff3700 1 --2- 172.21.15.163:0/3759084344 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f725c067320 0x7f725c0697d0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.271+0000 7f727fff3700 1 --2- 172.21.15.163:0/3759084344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f727815df50 0x7f727806cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.271+0000 7f727fff3700 1 --2- 172.21.15.163:0/3759084344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f727806bcd0 0x7f7278154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.271+0000 7f727fff3700 1 --2- 172.21.15.163:0/3759084344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f727806c0a0 0x7f72780b13a0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.271+0000 7f727fff3700 1 -- 172.21.15.163:0/3759084344 >> 172.21.15.163:0/3759084344 conn(0x7f72780ce600 msgr2=0x7f727815a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:06.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.271+0000 7f727fff3700 1 -- 172.21.15.163:0/3759084344 shutdown_connections 2023-10-09T06:42:06.275 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.271+0000 7f727fff3700 1 -- 172.21.15.163:0/3759084344 wait complete. 2023-10-09T06:42:06.275 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 133 2023-10-09T06:42:06.283 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:42:06.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.398+0000 7fe8e10a1700 1 Processor -- start 2023-10-09T06:42:06.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.398+0000 7fe8e10a1700 1 -- start start 2023-10-09T06:42:06.401 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.399+0000 7fe8e10a1700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe8dc159c60 0x7fe8dc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:06.402 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.399+0000 7fe8e10a1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc15db10 0x7fe8dc069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:06.402 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.399+0000 7fe8e10a1700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8dc15a160 con 0x7fe8dc159890 2023-10-09T06:42:06.402 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.399+0000 7fe8e10a1700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8dc15df50 con 0x7fe8dc159c60 2023-10-09T06:42:06.403 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.399+0000 7fe8e10a1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc0d13a0 con 0x7fe8dc15db10 2023-10-09T06:42:06.404 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.399+0000 7fe8d259c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe8dc159c60 0x7fe8dc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:06.404 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.399+0000 7fe8dad9d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fe8dc159890 0x7fe8dc15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:54078/0 (socket says 172.21.15.163:54078) 2023-10-09T06:42:06.404 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.399+0000 7fe8dad9d700 1 -- 172.21.15.163:0/1966178051 learned_addr learned my addr 172.21.15.163:0/1966178051 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:42:06.404 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.399+0000 7fe8d259c700 1 -- 172.21.15.163:0/1966178051 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc15db10 msgr2=0x7fe8dc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:06.404 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.399+0000 7fe8db59e700 1 --2- 172.21.15.163:0/1966178051 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc15db10 0x7fe8dc069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:06.404 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.399+0000 7fe8d259c700 1 --2- 172.21.15.163:0/1966178051 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc15db10 0x7fe8dc069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.404 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.399+0000 7fe8da59c700 1 -- 172.21.15.163:0/1966178051 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1197422520 0 0) 0x7fe8dc15a160 con 0x7fe8dc159890 2023-10-09T06:42:06.404 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.399+0000 7fe8d259c700 1 -- 172.21.15.163:0/1966178051 >> v1:172.21.15.163:6789/0 conn(0x7fe8dc159890 legacy=0x7fe8dc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:06.405 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.399+0000 7fe8d259c700 1 -- 172.21.15.163:0/1966178051 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8dc15e140 con 0x7fe8dc159c60 2023-10-09T06:42:06.405 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.400+0000 7fe8db59e700 1 --2- 172.21.15.163:0/1966178051 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc15db10 0x7fe8dc069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:42:06.405 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.400+0000 7fe8d259c700 1 --2- 172.21.15.163:0/1966178051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8dc159c60 0x7fe8dc15d5d0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40098e0 tx=0x7fe8c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0f185119cec246e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:06.405 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.400+0000 7fe8da59c700 1 -- 172.21.15.163:0/1966178051 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4009e80 con 0x7fe8dc159c60 2023-10-09T06:42:06.405 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.400+0000 7fe8da59c700 1 -- 172.21.15.163:0/1966178051 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c400b860 con 0x7fe8dc159c60 2023-10-09T06:42:06.405 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.400+0000 7fe8da59c700 1 -- 172.21.15.163:0/1966178051 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c400bbe0 con 0x7fe8dc159c60 2023-10-09T06:42:06.405 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.400+0000 7fe8e10a1700 1 -- 172.21.15.163:0/1966178051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8dc159c60 msgr2=0x7fe8dc15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:06.405 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.400+0000 7fe8e10a1700 1 --2- 172.21.15.163:0/1966178051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8dc159c60 0x7fe8dc15d5d0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40098e0 tx=0x7fe8c40091a0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.405 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.400+0000 7fe8e10a1700 1 -- 172.21.15.163:0/1966178051 shutdown_connections 2023-10-09T06:42:06.405 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.400+0000 7fe8e10a1700 1 --2- 172.21.15.163:0/1966178051 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc15db10 0x7fe8dc069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.407 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.400+0000 7fe8e10a1700 1 --2- 172.21.15.163:0/1966178051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8dc159c60 0x7fe8dc15d5d0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.407 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.400+0000 7fe8e10a1700 1 -- 172.21.15.163:0/1966178051 >> 172.21.15.163:0/1966178051 conn(0x7fe8dc068e20 msgr2=0x7fe8dc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:06.407 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.400+0000 7fe8e10a1700 1 -- 172.21.15.163:0/1966178051 shutdown_connections 2023-10-09T06:42:06.407 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.400+0000 7fe8e10a1700 1 -- 172.21.15.163:0/1966178051 wait complete. 2023-10-09T06:42:06.410 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.401+0000 7fe8e10a1700 1 Processor -- start 2023-10-09T06:42:06.410 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.402+0000 7fe8e10a1700 1 -- start start 2023-10-09T06:42:06.410 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.402+0000 7fe8e10a1700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc159890 0x7fe8dc14a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:06.410 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.402+0000 7fe8e10a1700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8dc159c60 0x7fe8dc15d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:06.410 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.402+0000 7fe8e10a1700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8dc15db10 0x7fe8dc16a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:06.410 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.402+0000 7fe8e10a1700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc15d5f0 con 0x7fe8dc15db10 2023-10-09T06:42:06.410 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.402+0000 7fe8e10a1700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc16abd0 con 0x7fe8dc159c60 2023-10-09T06:42:06.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.402+0000 7fe8e10a1700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc16a200 con 0x7fe8dc159890 2023-10-09T06:42:06.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.402+0000 7fe8db59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8dc15db10 0x7fe8dc16a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:06.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.402+0000 7fe8db59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8dc15db10 0x7fe8dc16a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:52350/0 (socket says 172.21.15.163:52350) 2023-10-09T06:42:06.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.402+0000 7fe8db59e700 1 -- 172.21.15.163:0/2895227086 learned_addr learned my addr 172.21.15.163:0/2895227086 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:06.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.403+0000 7fe8d259c700 1 --2- 172.21.15.163:0/2895227086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8dc159c60 0x7fe8dc15d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:06.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.403+0000 7fe8dad9d700 1 --2- 172.21.15.163:0/2895227086 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc159890 0x7fe8dc14a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:06.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.403+0000 7fe8db59e700 1 -- 172.21.15.163:0/2895227086 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc159890 msgr2=0x7fe8dc14a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:06.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.403+0000 7fe8db59e700 1 --2- 172.21.15.163:0/2895227086 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc159890 0x7fe8dc14a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.403+0000 7fe8db59e700 1 -- 172.21.15.163:0/2895227086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8dc159c60 msgr2=0x7fe8dc15d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:06.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.403+0000 7fe8db59e700 1 --2- 172.21.15.163:0/2895227086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8dc159c60 0x7fe8dc15d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.403+0000 7fe8db59e700 1 -- 172.21.15.163:0/2895227086 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c4009420 con 0x7fe8dc15db10 2023-10-09T06:42:06.411 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.403+0000 7fe8d259c700 1 --2- 172.21.15.163:0/2895227086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8dc159c60 0x7fe8dc15d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:42:06.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.403+0000 7fe8dad9d700 1 --2- 172.21.15.163:0/2895227086 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc159890 0x7fe8dc14a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:42:06.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.403+0000 7fe8db59e700 1 --2- 172.21.15.163:0/2895227086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8dc15db10 0x7fe8dc16a690 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fe8d400cf50 tx=0x7fe8d4007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:06.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.403+0000 7fe8d37fe700 1 -- 172.21.15.163:0/2895227086 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d400ae20 con 0x7fe8dc15db10 2023-10-09T06:42:06.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.403+0000 7fe8d37fe700 1 -- 172.21.15.163:0/2895227086 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d4016420 con 0x7fe8dc15db10 2023-10-09T06:42:06.412 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.403+0000 7fe8e10a1700 1 -- 172.21.15.163:0/2895227086 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8dc169bd0 con 0x7fe8dc15db10 2023-10-09T06:42:06.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.403+0000 7fe8d37fe700 1 -- 172.21.15.163:0/2895227086 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d40167e0 con 0x7fe8dc15db10 2023-10-09T06:42:06.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.403+0000 7fe8e10a1700 1 -- 172.21.15.163:0/2895227086 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8dc0b1340 con 0x7fe8dc15db10 2023-10-09T06:42:06.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.404+0000 7fe8e10a1700 1 -- 172.21.15.163:0/2895227086 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8dc0b1530 con 0x7fe8dc15db10 2023-10-09T06:42:06.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.404+0000 7fe8d37fe700 1 -- 172.21.15.163:0/2895227086 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe8d4016b90 con 0x7fe8dc15db10 2023-10-09T06:42:06.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.405+0000 7fe8d37fe700 1 --2- 172.21.15.163:0/2895227086 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe8bc067320 0x7fe8bc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:06.413 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.405+0000 7fe8d37fe700 1 -- 172.21.15.163:0/2895227086 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5792+0+0 (secure 0 0 0) 0x7fe8d4012070 con 0x7fe8dc15db10 2023-10-09T06:42:06.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.405+0000 7fe8dad9d700 1 --2- 172.21.15.163:0/2895227086 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe8bc067320 0x7fe8bc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:06.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.405+0000 7fe8dad9d700 1 --2- 172.21.15.163:0/2895227086 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe8bc067320 0x7fe8bc0697d0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc0018d0 tx=0x7fe8cc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:06.414 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.409+0000 7fe8d37fe700 1 -- 172.21.15.163:0/2895227086 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe8d404e5a0 con 0x7fe8dc15db10 2023-10-09T06:42:06.634 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.633+0000 7fe8e10a1700 1 -- 172.21.15.163:0/2895227086 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8dc04c3c0 con 0x7fe8dc15db10 2023-10-09T06:42:06.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.633+0000 7fe8d37fe700 1 -- 172.21.15.163:0/2895227086 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+4619 (secure 0 0 0) 0x7fe8d40522f0 con 0x7fe8dc15db10 2023-10-09T06:42:06.636 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:42:06.636 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":133,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7046,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3374277959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3374277959},{"type":"v1","addr":"172.21.15.181:6837","nonce":3374277959}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3834979450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3834979450},{"type":"v1","addr":"172.21.15.181:6836","nonce":3834979450}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":8991,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2681808199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2681808199},{"type":"v1","addr":"172.21.15.163:6834","nonce":2681808199}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128}],"filesystems":[{"mdsmap":{"epoch":132,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-09T06:39:52.456713+0000","modified":"2023-10-09T06:39:56.092448+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8994":{"gid":8994,"name":"a","rank":0,"incarnation":129,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6835/2564044136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2564044136},{"type":"v1","addr":"172.21.15.163:6835","nonce":2564044136}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-09T06:42:06.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.635+0000 7fe8e10a1700 1 -- 172.21.15.163:0/2895227086 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe8bc067320 msgr2=0x7fe8bc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:06.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.635+0000 7fe8e10a1700 1 --2- 172.21.15.163:0/2895227086 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe8bc067320 0x7fe8bc0697d0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc0018d0 tx=0x7fe8cc009040 comp rx=0 tx=0).stop 2023-10-09T06:42:06.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.635+0000 7fe8e10a1700 1 -- 172.21.15.163:0/2895227086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8dc15db10 msgr2=0x7fe8dc16a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:06.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.635+0000 7fe8e10a1700 1 --2- 172.21.15.163:0/2895227086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8dc15db10 0x7fe8dc16a690 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fe8d400cf50 tx=0x7fe8d4007860 comp rx=0 tx=0).stop 2023-10-09T06:42:06.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.635+0000 7fe8e10a1700 1 -- 172.21.15.163:0/2895227086 shutdown_connections 2023-10-09T06:42:06.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.635+0000 7fe8e10a1700 1 --2- 172.21.15.163:0/2895227086 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fe8bc067320 0x7fe8bc0697d0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.635+0000 7fe8e10a1700 1 --2- 172.21.15.163:0/2895227086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8dc15db10 0x7fe8dc16a690 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.635+0000 7fe8e10a1700 1 --2- 172.21.15.163:0/2895227086 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc159890 0x7fe8dc14a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.635+0000 7fe8e10a1700 1 --2- 172.21.15.163:0/2895227086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8dc159c60 0x7fe8dc15d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.635+0000 7fe8e10a1700 1 -- 172.21.15.163:0/2895227086 >> 172.21.15.163:0/2895227086 conn(0x7fe8dc068e20 msgr2=0x7fe8dc0b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:06.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.635+0000 7fe8e10a1700 1 -- 172.21.15.163:0/2895227086 shutdown_connections 2023-10-09T06:42:06.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.636+0000 7fe8e10a1700 1 -- 172.21.15.163:0/2895227086 wait complete. 2023-10-09T06:42:06.639 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 133 2023-10-09T06:42:06.648 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:42:06.763 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.763+0000 7f29952e6700 1 Processor -- start 2023-10-09T06:42:06.764 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.763+0000 7f29952e6700 1 -- start start 2023-10-09T06:42:06.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.764+0000 7f29952e6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2990159890 0x7f299015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:06.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.764+0000 7f29952e6700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2990159c60 0x7f299015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:06.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.764+0000 7f29952e6700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f299015a160 con 0x7f299015db10 2023-10-09T06:42:06.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.764+0000 7f29952e6700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f299015df50 con 0x7f2990159c60 2023-10-09T06:42:06.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.764+0000 7f29952e6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29900d13a0 con 0x7f2990159890 2023-10-09T06:42:06.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.764+0000 7f29867fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2990159c60 0x7f299015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:06.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.764+0000 7f298f7fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f299015db10 0x7f2990069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:54082/0 (socket says 172.21.15.163:54082) 2023-10-09T06:42:06.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.764+0000 7f298f7fe700 1 -- 172.21.15.163:0/2684402028 learned_addr learned my addr 172.21.15.163:0/2684402028 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:42:06.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.765+0000 7f298effd700 1 --2- 172.21.15.163:0/2684402028 >> v2:172.21.15.181:3300/0 conn(0x7f2990159890 0x7f299015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:06.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.765+0000 7f29867fc700 1 -- 172.21.15.163:0/2684402028 >> v2:172.21.15.181:3300/0 conn(0x7f2990159890 msgr2=0x7f299015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:06.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.765+0000 7f29867fc700 1 --2- 172.21.15.163:0/2684402028 >> v2:172.21.15.181:3300/0 conn(0x7f2990159890 0x7f299015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.765+0000 7f29867fc700 1 -- 172.21.15.163:0/2684402028 >> v1:172.21.15.163:6789/0 conn(0x7f299015db10 legacy=0x7f2990069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:06.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.765+0000 7f298e7fc700 1 -- 172.21.15.163:0/2684402028 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2678556744 0 0) 0x7f299015a160 con 0x7f299015db10 2023-10-09T06:42:06.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.765+0000 7f29867fc700 1 -- 172.21.15.163:0/2684402028 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299015e140 con 0x7f2990159c60 2023-10-09T06:42:06.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.765+0000 7f29867fc700 1 --2- 172.21.15.163:0/2684402028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2990159c60 0x7f299015d5d0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f29900d1090 tx=0x7f29780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3be2735630acd97 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:06.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.765+0000 7f298e7fc700 1 -- 172.21.15.163:0/2684402028 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297800b6b0 con 0x7f2990159c60 2023-10-09T06:42:06.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.765+0000 7f298e7fc700 1 -- 172.21.15.163:0/2684402028 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297800b810 con 0x7f2990159c60 2023-10-09T06:42:06.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.765+0000 7f298e7fc700 1 -- 172.21.15.163:0/2684402028 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297800bbb0 con 0x7f2990159c60 2023-10-09T06:42:06.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.765+0000 7f29952e6700 1 -- 172.21.15.163:0/2684402028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2990159c60 msgr2=0x7f299015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:06.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.766+0000 7f29952e6700 1 --2- 172.21.15.163:0/2684402028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2990159c60 0x7f299015d5d0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f29900d1090 tx=0x7f29780091a0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.766+0000 7f29952e6700 1 -- 172.21.15.163:0/2684402028 shutdown_connections 2023-10-09T06:42:06.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.766+0000 7f29952e6700 1 --2- 172.21.15.163:0/2684402028 >> v2:172.21.15.181:3300/0 conn(0x7f2990159890 0x7f299015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.766+0000 7f29952e6700 1 --2- 172.21.15.163:0/2684402028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2990159c60 0x7f299015d5d0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.766+0000 7f29952e6700 1 -- 172.21.15.163:0/2684402028 >> 172.21.15.163:0/2684402028 conn(0x7f2990068e20 msgr2=0x7f29900d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:06.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.766+0000 7f29952e6700 1 -- 172.21.15.163:0/2684402028 shutdown_connections 2023-10-09T06:42:06.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.766+0000 7f29952e6700 1 -- 172.21.15.163:0/2684402028 wait complete. 2023-10-09T06:42:06.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.767+0000 7f29952e6700 1 Processor -- start 2023-10-09T06:42:06.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.767+0000 7f29952e6700 1 -- start start 2023-10-09T06:42:06.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.767+0000 7f29952e6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2990159890 0x7f29900b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:06.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.768+0000 7f29952e6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2990159c60 0x7f2990140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:06.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.768+0000 7f29952e6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f299015db10 0x7f299016dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:06.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.768+0000 7f29952e6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29900b38c0 con 0x7f2990159890 2023-10-09T06:42:06.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.768+0000 7f29952e6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299016e3e0 con 0x7f299015db10 2023-10-09T06:42:06.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.768+0000 7f29952e6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29900b5990 con 0x7f2990159c60 2023-10-09T06:42:06.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.768+0000 7f298effd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2990159890 0x7f29900b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:06.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.768+0000 7f298effd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2990159890 0x7f29900b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:52356/0 (socket says 172.21.15.163:52356) 2023-10-09T06:42:06.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.768+0000 7f298effd700 1 -- 172.21.15.163:0/2053600092 learned_addr learned my addr 172.21.15.163:0/2053600092 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:06.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.768+0000 7f298f7fe700 1 --2- 172.21.15.163:0/2053600092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f299015db10 0x7f299016dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:06.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.768+0000 7f29867fc700 1 --2- 172.21.15.163:0/2053600092 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2990159c60 0x7f2990140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:06.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.768+0000 7f298effd700 1 -- 172.21.15.163:0/2053600092 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2990159c60 msgr2=0x7f2990140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:06.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.768+0000 7f298effd700 1 --2- 172.21.15.163:0/2053600092 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2990159c60 0x7f2990140ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.768+0000 7f298effd700 1 -- 172.21.15.163:0/2053600092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f299015db10 msgr2=0x7f299016dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:06.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.768+0000 7f298effd700 1 --2- 172.21.15.163:0/2053600092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f299015db10 0x7f299016dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.768+0000 7f298effd700 1 -- 172.21.15.163:0/2053600092 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2978009420 con 0x7f2990159890 2023-10-09T06:42:06.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.768+0000 7f298effd700 1 --2- 172.21.15.163:0/2053600092 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2990159890 0x7f29900b5390 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f298000d890 tx=0x7f2980008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:06.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.768+0000 7f29877fe700 1 -- 172.21.15.163:0/2053600092 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29800125f0 con 0x7f2990159890 2023-10-09T06:42:06.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.768+0000 7f29877fe700 1 -- 172.21.15.163:0/2053600092 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2980012750 con 0x7f2990159890 2023-10-09T06:42:06.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.768+0000 7f29952e6700 1 -- 172.21.15.163:0/2053600092 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29900b5c70 con 0x7f2990159890 2023-10-09T06:42:06.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.768+0000 7f29877fe700 1 -- 172.21.15.163:0/2053600092 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2980012af0 con 0x7f2990159890 2023-10-09T06:42:06.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.768+0000 7f29952e6700 1 -- 172.21.15.163:0/2053600092 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29900b6230 con 0x7f2990159890 2023-10-09T06:42:06.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.769+0000 7f29952e6700 1 -- 172.21.15.163:0/2053600092 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29900cd4b0 con 0x7f2990159890 2023-10-09T06:42:06.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.770+0000 7f29877fe700 1 -- 172.21.15.163:0/2053600092 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f298001e460 con 0x7f2990159890 2023-10-09T06:42:06.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.770+0000 7f29877fe700 1 --2- 172.21.15.163:0/2053600092 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2970067320 0x7f29700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:06.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.770+0000 7f29877fe700 1 -- 172.21.15.163:0/2053600092 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5792+0+0 (secure 0 0 0) 0x7f2980085230 con 0x7f2990159890 2023-10-09T06:42:06.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.770+0000 7f29867fc700 1 --2- 172.21.15.163:0/2053600092 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2970067320 0x7f29700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:06.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.771+0000 7f29867fc700 1 --2- 172.21.15.163:0/2053600092 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2970067320 0x7f29700697d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f29780093f0 tx=0x7f2978017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:06.776 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.774+0000 7f29877fe700 1 -- 172.21.15.163:0/2053600092 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2980051e70 con 0x7f2990159890 2023-10-09T06:42:06.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.958+0000 7f29952e6700 1 -- 172.21.15.163:0/2053600092 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f299004c3c0 con 0x7f2990159890 2023-10-09T06:42:06.959 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.959+0000 7f29877fe700 1 -- 172.21.15.163:0/2053600092 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v199) v1 ==== 74+0+15558 (secure 0 0 0) 0x7f2980055490 con 0x7f2990159890 2023-10-09T06:42:06.959 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:42:06.960 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":199,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:40:26.611732+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2023-10-09T06:39:50.786303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":34,"pool_name":"cephfs_data","create_time":"2023-10-09T06:39:51.411021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":198,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":198,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"34.24","mappings":[{"from":0,"to":2}]},{"pgid":"34.25","mappings":[{"from":0,"to":1}]},{"pgid":"34.30","mappings":[{"from":0,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:42:06.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.961+0000 7f29952e6700 1 -- 172.21.15.163:0/2053600092 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2970067320 msgr2=0x7f29700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:06.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.961+0000 7f29952e6700 1 --2- 172.21.15.163:0/2053600092 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2970067320 0x7f29700697d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f29780093f0 tx=0x7f2978017000 comp rx=0 tx=0).stop 2023-10-09T06:42:06.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.961+0000 7f29952e6700 1 -- 172.21.15.163:0/2053600092 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2990159890 msgr2=0x7f29900b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:06.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.961+0000 7f29952e6700 1 --2- 172.21.15.163:0/2053600092 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2990159890 0x7f29900b5390 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f298000d890 tx=0x7f2980008040 comp rx=0 tx=0).stop 2023-10-09T06:42:06.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.961+0000 7f29952e6700 1 -- 172.21.15.163:0/2053600092 shutdown_connections 2023-10-09T06:42:06.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.961+0000 7f29952e6700 1 --2- 172.21.15.163:0/2053600092 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f2970067320 0x7f29700697d0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.961+0000 7f29952e6700 1 --2- 172.21.15.163:0/2053600092 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2990159890 0x7f29900b5390 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.961+0000 7f29952e6700 1 --2- 172.21.15.163:0/2053600092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f299015db10 0x7f299016dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.961+0000 7f29952e6700 1 --2- 172.21.15.163:0/2053600092 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2990159c60 0x7f2990140ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:06.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.961+0000 7f29952e6700 1 -- 172.21.15.163:0/2053600092 >> 172.21.15.163:0/2053600092 conn(0x7f2990068e20 msgr2=0x7f29901542a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:06.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.961+0000 7f29952e6700 1 -- 172.21.15.163:0/2053600092 shutdown_connections 2023-10-09T06:42:06.963 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:06.961+0000 7f29952e6700 1 -- 172.21.15.163:0/2053600092 wait complete. 2023-10-09T06:42:06.974 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-09T06:42:07.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.089+0000 7f8117d01700 1 Processor -- start 2023-10-09T06:42:07.089 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.089+0000 7f8117d01700 1 -- start start 2023-10-09T06:42:07.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.090+0000 7f8117d01700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8110159890 0x7f811015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:07.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.090+0000 7f8117d01700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f811015a160 con 0x7f8110069560 2023-10-09T06:42:07.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.090+0000 7f8117d01700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81100d13a0 con 0x7f8110159c60 2023-10-09T06:42:07.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.090+0000 7f8117d01700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f811015e140 con 0x7f8110159890 2023-10-09T06:42:07.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.090+0000 7f8115a9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8110159890 0x7f811015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:07.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.090+0000 7f811629e700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f8110069560 0x7f81100ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:54096/0 (socket says 172.21.15.163:54096) 2023-10-09T06:42:07.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.090+0000 7f811629e700 1 -- 172.21.15.163:0/4254871728 learned_addr learned my addr 172.21.15.163:0/4254871728 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:42:07.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.091+0000 7f8115a9d700 1 -- 172.21.15.163:0/4254871728 >> v1:172.21.15.181:6789/0 conn(0x7f8110159c60 legacy=0x7f811015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:07.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.091+0000 7f8115a9d700 1 -- 172.21.15.163:0/4254871728 >> v1:172.21.15.163:6789/0 conn(0x7f8110069560 legacy=0x7f81100ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:07.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.091+0000 7f8115a9d700 1 -- 172.21.15.163:0/4254871728 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811015a3d0 con 0x7f8110159890 2023-10-09T06:42:07.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.091+0000 7f8115a9d700 1 --2- 172.21.15.163:0/4254871728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8110159890 0x7f811015abd0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f810c0098e0 tx=0x7f810c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864094d6dad71ec1 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:07.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.091+0000 7f8114a9b700 1 -- 172.21.15.163:0/4254871728 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c009e80 con 0x7f8110159890 2023-10-09T06:42:07.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.091+0000 7f8114a9b700 1 -- 172.21.15.163:0/4254871728 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f810c00b820 con 0x7f8110159890 2023-10-09T06:42:07.092 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.091+0000 7f8114a9b700 1 -- 172.21.15.163:0/4254871728 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c00bba0 con 0x7f8110159890 2023-10-09T06:42:07.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.091+0000 7f8117d01700 1 -- 172.21.15.163:0/4254871728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8110159890 msgr2=0x7f811015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:07.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.091+0000 7f8117d01700 1 --2- 172.21.15.163:0/4254871728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8110159890 0x7f811015abd0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f810c0098e0 tx=0x7f810c0091a0 comp rx=0 tx=0).stop 2023-10-09T06:42:07.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.091+0000 7f8117d01700 1 -- 172.21.15.163:0/4254871728 shutdown_connections 2023-10-09T06:42:07.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.091+0000 7f8117d01700 1 --2- 172.21.15.163:0/4254871728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8110159890 0x7f811015abd0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:07.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.092+0000 7f8117d01700 1 -- 172.21.15.163:0/4254871728 >> 172.21.15.163:0/4254871728 conn(0x7f8110068e20 msgr2=0x7f81100d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:07.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.092+0000 7f8117d01700 1 -- 172.21.15.163:0/4254871728 shutdown_connections 2023-10-09T06:42:07.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.092+0000 7f8117d01700 1 -- 172.21.15.163:0/4254871728 wait complete. 2023-10-09T06:42:07.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.093+0000 7f8117d01700 1 Processor -- start 2023-10-09T06:42:07.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.093+0000 7f8117d01700 1 -- start start 2023-10-09T06:42:07.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.093+0000 7f8117d01700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8110069560 0x7f81101460b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:07.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.093+0000 7f8117d01700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8110159890 0x7f81100c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:07.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.093+0000 7f8117d01700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8110159c60 0x7f81100ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:07.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.093+0000 7f8117d01700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81100b4f20 con 0x7f8110069560 2023-10-09T06:42:07.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.093+0000 7f8117d01700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81100c8020 con 0x7f8110159890 2023-10-09T06:42:07.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.093+0000 7f8117d01700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81100b4c40 con 0x7f8110159c60 2023-10-09T06:42:07.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.093+0000 7f8115a9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8110069560 0x7f81101460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:07.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.093+0000 7f8115a9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8110069560 0x7f81101460b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:52384/0 (socket says 172.21.15.163:52384) 2023-10-09T06:42:07.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.093+0000 7f8115a9d700 1 -- 172.21.15.163:0/1432719090 learned_addr learned my addr 172.21.15.163:0/1432719090 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:07.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.094+0000 7f811629e700 1 --2- 172.21.15.163:0/1432719090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8110159c60 0x7f81100ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:07.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.094+0000 7f811529c700 1 --2- 172.21.15.163:0/1432719090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8110159890 0x7f81100c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:07.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.094+0000 7f8115a9d700 1 -- 172.21.15.163:0/1432719090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8110159c60 msgr2=0x7f81100ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:07.097 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.094+0000 7f8115a9d700 1 --2- 172.21.15.163:0/1432719090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8110159c60 0x7f81100ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:07.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.094+0000 7f8115a9d700 1 -- 172.21.15.163:0/1432719090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8110159890 msgr2=0x7f81100c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:07.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.094+0000 7f8115a9d700 1 --2- 172.21.15.163:0/1432719090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8110159890 0x7f81100c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:07.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.094+0000 7f8115a9d700 1 -- 172.21.15.163:0/1432719090 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f810c009420 con 0x7f8110069560 2023-10-09T06:42:07.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.094+0000 7f8115a9d700 1 --2- 172.21.15.163:0/1432719090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8110069560 0x7f81101460b0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f810c0030c0 tx=0x7f810c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:07.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.094+0000 7f8105ffb700 1 -- 172.21.15.163:0/1432719090 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c00bae0 con 0x7f8110069560 2023-10-09T06:42:07.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.094+0000 7f8105ffb700 1 -- 172.21.15.163:0/1432719090 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f810c020070 con 0x7f8110069560 2023-10-09T06:42:07.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.094+0000 7f8117d01700 1 -- 172.21.15.163:0/1432719090 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81100c79d0 con 0x7f8110069560 2023-10-09T06:42:07.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.094+0000 7f8105ffb700 1 -- 172.21.15.163:0/1432719090 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c01b6a0 con 0x7f8110069560 2023-10-09T06:42:07.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.094+0000 7f8117d01700 1 -- 172.21.15.163:0/1432719090 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81101465f0 con 0x7f8110069560 2023-10-09T06:42:07.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.095+0000 7f8117d01700 1 -- 172.21.15.163:0/1432719090 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f811004c3c0 con 0x7f8110069560 2023-10-09T06:42:07.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.095+0000 7f8105ffb700 1 -- 172.21.15.163:0/1432719090 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f810c022030 con 0x7f8110069560 2023-10-09T06:42:07.098 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.096+0000 7f8105ffb700 1 --2- 172.21.15.163:0/1432719090 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f80f4067320 0x7f80f40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:07.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.096+0000 7f8105ffb700 1 -- 172.21.15.163:0/1432719090 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5792+0+0 (secure 0 0 0) 0x7f810c086b80 con 0x7f8110069560 2023-10-09T06:42:07.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.096+0000 7f811529c700 1 --2- 172.21.15.163:0/1432719090 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f80f4067320 0x7f80f40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:07.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.096+0000 7f811529c700 1 --2- 172.21.15.163:0/1432719090 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f80f4067320 0x7f80f40697d0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f81100c9830 tx=0x7f8100009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:07.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.100+0000 7f8105ffb700 1 -- 172.21.15.163:0/1432719090 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f810c0538d0 con 0x7f8110069560 2023-10-09T06:42:07.324 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.325+0000 7f8117d01700 1 -- 172.21.15.163:0/1432719090 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f811004c3c0 con 0x7f8110069560 2023-10-09T06:42:07.325 INFO:tasks.ceph.mon.a.smithi163.stderr:2023-10-09T06:42:07.325+0000 7ff251c04700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-09T06:42:07.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.329+0000 7f8105ffb700 1 -- 172.21.15.163:0/1432719090 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v134) v1 ==== 161+0+0 (secure 0 0 0) 0x7f810c056ef0 con 0x7f8110069560 2023-10-09T06:42:07.331 INFO:tasks.ceph.mds.a.smithi163.stderr: -3251> 2023-10-09T06:39:58.367+0000 7f7e1b350700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-09T06:42:07.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.332+0000 7f8117d01700 1 -- 172.21.15.163:0/1432719090 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f80f4067320 msgr2=0x7f80f40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:07.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.332+0000 7f8117d01700 1 --2- 172.21.15.163:0/1432719090 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f80f4067320 0x7f80f40697d0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f81100c9830 tx=0x7f8100009040 comp rx=0 tx=0).stop 2023-10-09T06:42:07.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.332+0000 7f8117d01700 1 -- 172.21.15.163:0/1432719090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8110069560 msgr2=0x7f81101460b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:07.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.332+0000 7f8117d01700 1 --2- 172.21.15.163:0/1432719090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8110069560 0x7f81101460b0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f810c0030c0 tx=0x7f810c016000 comp rx=0 tx=0).stop 2023-10-09T06:42:07.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.332+0000 7f8117d01700 1 -- 172.21.15.163:0/1432719090 shutdown_connections 2023-10-09T06:42:07.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.332+0000 7f8117d01700 1 --2- 172.21.15.163:0/1432719090 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f80f4067320 0x7f80f40697d0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:07.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.332+0000 7f8117d01700 1 --2- 172.21.15.163:0/1432719090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8110069560 0x7f81101460b0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:07.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.332+0000 7f8117d01700 1 --2- 172.21.15.163:0/1432719090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8110159c60 0x7f81100ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:07.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.332+0000 7f8117d01700 1 --2- 172.21.15.163:0/1432719090 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8110159890 0x7f81100c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:07.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.332+0000 7f8117d01700 1 -- 172.21.15.163:0/1432719090 >> 172.21.15.163:0/1432719090 conn(0x7f8110068e20 msgr2=0x7f81100b1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:07.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.332+0000 7f8117d01700 1 -- 172.21.15.163:0/1432719090 shutdown_connections 2023-10-09T06:42:07.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.332+0000 7f8117d01700 1 -- 172.21.15.163:0/1432719090 wait complete. 2023-10-09T06:42:07.333 INFO:teuthology.orchestra.run.smithi163.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-09T06:42:07.345 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-09T06:42:07.355 INFO:tasks.ceph.mds.a.smithi163.stdout:starting mds.a at 2023-10-09T06:42:07.462 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.462+0000 7fba94238700 1 Processor -- start 2023-10-09T06:42:07.463 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.463+0000 7fba94238700 1 -- start start 2023-10-09T06:42:07.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.463+0000 7fba94238700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fba8c159c60 0x7fba8c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:07.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.463+0000 7fba94238700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fba8c15db10 0x7fba8c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:07.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.463+0000 7fba94238700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba8c15a160 con 0x7fba8c159890 2023-10-09T06:42:07.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.463+0000 7fba94238700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba8c15df50 con 0x7fba8c159c60 2023-10-09T06:42:07.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.463+0000 7fba94238700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba8c0d13a0 con 0x7fba8c15db10 2023-10-09T06:42:07.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.464+0000 7fba917d3700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fba8c159c60 0x7fba8c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:07.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.464+0000 7fba91fd4700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fba8c159890 0x7fba8c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:54108/0 (socket says 172.21.15.163:54108) 2023-10-09T06:42:07.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.464+0000 7fba91fd4700 1 -- 172.21.15.163:0/4016491169 learned_addr learned my addr 172.21.15.163:0/4016491169 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:42:07.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.464+0000 7fba927d5700 1 --2- 172.21.15.163:0/4016491169 >> v2:172.21.15.181:3300/0 conn(0x7fba8c15db10 0x7fba8c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:07.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.464+0000 7fba917d3700 1 -- 172.21.15.163:0/4016491169 >> v2:172.21.15.181:3300/0 conn(0x7fba8c15db10 msgr2=0x7fba8c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:07.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.464+0000 7fba917d3700 1 --2- 172.21.15.163:0/4016491169 >> v2:172.21.15.181:3300/0 conn(0x7fba8c15db10 0x7fba8c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:07.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.464+0000 7fba917d3700 1 -- 172.21.15.163:0/4016491169 >> v1:172.21.15.163:6789/0 conn(0x7fba8c159890 legacy=0x7fba8c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:07.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.464+0000 7fba917d3700 1 -- 172.21.15.163:0/4016491169 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba8c15e140 con 0x7fba8c159c60 2023-10-09T06:42:07.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.464+0000 7fba90fd2700 1 -- 172.21.15.163:0/4016491169 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3933131717 0 0) 0x7fba8c15a160 con 0x7fba8c159890 2023-10-09T06:42:07.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.464+0000 7fba927d5700 1 --2- 172.21.15.163:0/4016491169 >> v2:172.21.15.181:3300/0 conn(0x7fba8c15db10 0x7fba8c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:42:07.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.464+0000 7fba917d3700 1 --2- 172.21.15.163:0/4016491169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba8c159c60 0x7fba8c15d5d0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fba8c06a890 tx=0x7fba800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3364c34b6b1299f server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:07.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.464+0000 7fba90fd2700 1 -- 172.21.15.163:0/4016491169 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8000b6b0 con 0x7fba8c159c60 2023-10-09T06:42:07.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.464+0000 7fba90fd2700 1 -- 172.21.15.163:0/4016491169 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba8000b810 con 0x7fba8c159c60 2023-10-09T06:42:07.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.464+0000 7fba90fd2700 1 -- 172.21.15.163:0/4016491169 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8000bbb0 con 0x7fba8c159c60 2023-10-09T06:42:07.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.465+0000 7fba94238700 1 -- 172.21.15.163:0/4016491169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba8c159c60 msgr2=0x7fba8c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:07.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.465+0000 7fba94238700 1 --2- 172.21.15.163:0/4016491169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba8c159c60 0x7fba8c15d5d0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fba8c06a890 tx=0x7fba800091a0 comp rx=0 tx=0).stop 2023-10-09T06:42:07.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.465+0000 7fba94238700 1 -- 172.21.15.163:0/4016491169 shutdown_connections 2023-10-09T06:42:07.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.465+0000 7fba94238700 1 --2- 172.21.15.163:0/4016491169 >> v2:172.21.15.181:3300/0 conn(0x7fba8c15db10 0x7fba8c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:07.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.465+0000 7fba94238700 1 --2- 172.21.15.163:0/4016491169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba8c159c60 0x7fba8c15d5d0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:07.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.465+0000 7fba94238700 1 -- 172.21.15.163:0/4016491169 >> 172.21.15.163:0/4016491169 conn(0x7fba8c068e20 msgr2=0x7fba8c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:07.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.465+0000 7fba94238700 1 -- 172.21.15.163:0/4016491169 shutdown_connections 2023-10-09T06:42:07.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.465+0000 7fba94238700 1 -- 172.21.15.163:0/4016491169 wait complete. 2023-10-09T06:42:07.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.466+0000 7fba94238700 1 Processor -- start 2023-10-09T06:42:07.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.466+0000 7fba94238700 1 -- start start 2023-10-09T06:42:07.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.467+0000 7fba94238700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba8c159890 0x7fba8c0b5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:07.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.467+0000 7fba94238700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba8c159c60 0x7fba8c169d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:07.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.467+0000 7fba94238700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba8c15db10 0x7fba8c151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:07.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.467+0000 7fba94238700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c149c00 con 0x7fba8c159890 2023-10-09T06:42:07.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.467+0000 7fba94238700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c0b50b0 con 0x7fba8c159c60 2023-10-09T06:42:07.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.467+0000 7fba94238700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c16a250 con 0x7fba8c15db10 2023-10-09T06:42:07.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.467+0000 7fba91fd4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba8c159890 0x7fba8c0b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:07.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.467+0000 7fba91fd4700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba8c159890 0x7fba8c0b5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:52420/0 (socket says 172.21.15.163:52420) 2023-10-09T06:42:07.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.467+0000 7fba91fd4700 1 -- 172.21.15.163:0/1719937009 learned_addr learned my addr 172.21.15.163:0/1719937009 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:07.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.467+0000 7fba917d3700 1 --2- 172.21.15.163:0/1719937009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba8c159c60 0x7fba8c169d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:07.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.467+0000 7fba927d5700 1 --2- 172.21.15.163:0/1719937009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba8c15db10 0x7fba8c151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:07.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.467+0000 7fba91fd4700 1 -- 172.21.15.163:0/1719937009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba8c15db10 msgr2=0x7fba8c151a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:07.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.467+0000 7fba91fd4700 1 --2- 172.21.15.163:0/1719937009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba8c15db10 0x7fba8c151a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:07.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.467+0000 7fba91fd4700 1 -- 172.21.15.163:0/1719937009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba8c159c60 msgr2=0x7fba8c169d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:07.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.467+0000 7fba91fd4700 1 --2- 172.21.15.163:0/1719937009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba8c159c60 0x7fba8c169d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:07.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.467+0000 7fba91fd4700 1 -- 172.21.15.163:0/1719937009 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba80009420 con 0x7fba8c159890 2023-10-09T06:42:07.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.467+0000 7fba91fd4700 1 --2- 172.21.15.163:0/1719937009 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba8c159890 0x7fba8c0b5540 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00cd80 tx=0x7fba7c001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:07.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.467+0000 7fba89ffb700 1 -- 172.21.15.163:0/1719937009 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c00ab10 con 0x7fba8c159890 2023-10-09T06:42:07.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.467+0000 7fba89ffb700 1 -- 172.21.15.163:0/1719937009 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7c00ac70 con 0x7fba8c159890 2023-10-09T06:42:07.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.467+0000 7fba89ffb700 1 -- 172.21.15.163:0/1719937009 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c0070f0 con 0x7fba8c159890 2023-10-09T06:42:07.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.467+0000 7fba94238700 1 -- 172.21.15.163:0/1719937009 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba8c0b44a0 con 0x7fba8c159890 2023-10-09T06:42:07.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.468+0000 7fba94238700 1 -- 172.21.15.163:0/1719937009 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba8c0b4a60 con 0x7fba8c159890 2023-10-09T06:42:07.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.468+0000 7fba94238700 1 -- 172.21.15.163:0/1719937009 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba8c0b5390 con 0x7fba8c159890 2023-10-09T06:42:07.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.469+0000 7fba89ffb700 1 -- 172.21.15.163:0/1719937009 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fba7c0074e0 con 0x7fba8c159890 2023-10-09T06:42:07.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.469+0000 7fba89ffb700 1 --2- 172.21.15.163:0/1719937009 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fba78067320 0x7fba780697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:07.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.469+0000 7fba89ffb700 1 -- 172.21.15.163:0/1719937009 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5878+0+0 (secure 0 0 0) 0x7fba7c0852b0 con 0x7fba8c159890 2023-10-09T06:42:07.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.469+0000 7fba917d3700 1 --2- 172.21.15.163:0/1719937009 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fba78067320 0x7fba780697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:07.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.470+0000 7fba917d3700 1 --2- 172.21.15.163:0/1719937009 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fba78067320 0x7fba780697d0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fba800093f0 tx=0x7fba80017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:07.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.473+0000 7fba89ffb700 1 -- 172.21.15.163:0/1719937009 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba7c051f10 con 0x7fba8c159890 2023-10-09T06:42:07.700 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:07.700+0000 7fba94238700 1 -- 172.21.15.163:0/1719937009 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fba8c15aeb0 con 0x7fba8c159890 2023-10-09T06:42:08.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.329+0000 7fba89ffb700 1 -- 172.21.15.163:0/1719937009 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v135) v1 ==== 104+0+0 (secure 0 0 0) 0x7fba7c055530 con 0x7fba8c159890 2023-10-09T06:42:08.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.332+0000 7fba94238700 1 -- 172.21.15.163:0/1719937009 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fba78067320 msgr2=0x7fba780697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:08.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.332+0000 7fba94238700 1 --2- 172.21.15.163:0/1719937009 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fba78067320 0x7fba780697d0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fba800093f0 tx=0x7fba80017000 comp rx=0 tx=0).stop 2023-10-09T06:42:08.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.332+0000 7fba94238700 1 -- 172.21.15.163:0/1719937009 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba8c159890 msgr2=0x7fba8c0b5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:08.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.332+0000 7fba94238700 1 --2- 172.21.15.163:0/1719937009 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba8c159890 0x7fba8c0b5540 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00cd80 tx=0x7fba7c001ae0 comp rx=0 tx=0).stop 2023-10-09T06:42:08.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.332+0000 7fba94238700 1 -- 172.21.15.163:0/1719937009 shutdown_connections 2023-10-09T06:42:08.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.332+0000 7fba94238700 1 --2- 172.21.15.163:0/1719937009 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fba78067320 0x7fba780697d0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:08.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.332+0000 7fba94238700 1 --2- 172.21.15.163:0/1719937009 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba8c159890 0x7fba8c0b5540 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:08.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.332+0000 7fba94238700 1 --2- 172.21.15.163:0/1719937009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba8c15db10 0x7fba8c151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:08.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.332+0000 7fba94238700 1 --2- 172.21.15.163:0/1719937009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba8c159c60 0x7fba8c169d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:08.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.332+0000 7fba94238700 1 -- 172.21.15.163:0/1719937009 >> 172.21.15.163:0/1719937009 conn(0x7fba8c068e20 msgr2=0x7fba8c1570b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:08.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.332+0000 7fba94238700 1 -- 172.21.15.163:0/1719937009 shutdown_connections 2023-10-09T06:42:08.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.332+0000 7fba94238700 1 -- 172.21.15.163:0/1719937009 wait complete. 2023-10-09T06:42:08.345 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-09T06:42:08.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.505+0000 7f0614e09700 1 Processor -- start 2023-10-09T06:42:08.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.505+0000 7f0614e09700 1 -- start start 2023-10-09T06:42:08.507 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.506+0000 7f0614e09700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06100b03c0 0x7f06100cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:08.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.506+0000 7f0614e09700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f06100b0790 0x7f06100b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:08.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.506+0000 7f0614e09700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06100cfa00 con 0x7f06101591e0 2023-10-09T06:42:08.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.506+0000 7f0614e09700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06100cb580 con 0x7f06100b0790 2023-10-09T06:42:08.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.506+0000 7f0614e09700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06100b2de0 con 0x7f06100b03c0 2023-10-09T06:42:08.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.506+0000 7f060effd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f06100b0790 0x7f06100b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:08.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.506+0000 7f060ffff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f06101591e0 0x7f0610159620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60550/0 (socket says 172.21.15.163:60550) 2023-10-09T06:42:08.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.506+0000 7f060ffff700 1 -- 172.21.15.163:0/1435046548 learned_addr learned my addr 172.21.15.163:0/1435046548 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:42:08.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.506+0000 7f060f7fe700 1 --2- 172.21.15.163:0/1435046548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06100b03c0 0x7f06100cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:08.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.506+0000 7f060effd700 1 -- 172.21.15.163:0/1435046548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06100b03c0 msgr2=0x7f06100cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:08.508 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.506+0000 7f060effd700 1 --2- 172.21.15.163:0/1435046548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06100b03c0 0x7f06100cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:08.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.506+0000 7f060effd700 1 -- 172.21.15.163:0/1435046548 >> v1:172.21.15.163:6789/0 conn(0x7f06101591e0 legacy=0x7f0610159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:08.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.506+0000 7f060effd700 1 -- 172.21.15.163:0/1435046548 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06100cf160 con 0x7f06100b0790 2023-10-09T06:42:08.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.506+0000 7f060e7fc700 1 -- 172.21.15.163:0/1435046548 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 364629918 0 0) 0x7f06100cfa00 con 0x7f06101591e0 2023-10-09T06:42:08.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.506+0000 7f060f7fe700 1 --2- 172.21.15.163:0/1435046548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06100b03c0 0x7f06100cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:42:08.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.507+0000 7f060effd700 1 --2- 172.21.15.163:0/1435046548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06100b0790 0x7f06100b28a0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f06040098e0 tx=0x7f06040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b900f07af55ce3d server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:08.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.507+0000 7f060e7fc700 1 -- 172.21.15.163:0/1435046548 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0604009e80 con 0x7f06100b0790 2023-10-09T06:42:08.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.507+0000 7f060e7fc700 1 -- 172.21.15.163:0/1435046548 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060400b860 con 0x7f06100b0790 2023-10-09T06:42:08.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.507+0000 7f060e7fc700 1 -- 172.21.15.163:0/1435046548 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060400bbe0 con 0x7f06100b0790 2023-10-09T06:42:08.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.507+0000 7f0614e09700 1 -- 172.21.15.163:0/1435046548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06100b0790 msgr2=0x7f06100b28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:08.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.507+0000 7f0614e09700 1 --2- 172.21.15.163:0/1435046548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06100b0790 0x7f06100b28a0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f06040098e0 tx=0x7f06040091a0 comp rx=0 tx=0).stop 2023-10-09T06:42:08.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.507+0000 7f0614e09700 1 -- 172.21.15.163:0/1435046548 shutdown_connections 2023-10-09T06:42:08.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.507+0000 7f0614e09700 1 --2- 172.21.15.163:0/1435046548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06100b03c0 0x7f06100cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:08.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.507+0000 7f0614e09700 1 --2- 172.21.15.163:0/1435046548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06100b0790 0x7f06100b28a0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:08.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.507+0000 7f0614e09700 1 -- 172.21.15.163:0/1435046548 >> 172.21.15.163:0/1435046548 conn(0x7f0610063230 msgr2=0x7f0610063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:08.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.507+0000 7f0614e09700 1 -- 172.21.15.163:0/1435046548 shutdown_connections 2023-10-09T06:42:08.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.507+0000 7f0614e09700 1 -- 172.21.15.163:0/1435046548 wait complete. 2023-10-09T06:42:08.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.509+0000 7f0614e09700 1 Processor -- start 2023-10-09T06:42:08.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.509+0000 7f0614e09700 1 -- start start 2023-10-09T06:42:08.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.509+0000 7f0614e09700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06100b03c0 0x7f06100c5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:08.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.509+0000 7f0614e09700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06100b0790 0x7f0610159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:08.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.509+0000 7f0614e09700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06101591e0 0x7f061015a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:08.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.509+0000 7f0614e09700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061015a840 con 0x7f06100b03c0 2023-10-09T06:42:08.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.509+0000 7f0614e09700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06100c7640 con 0x7f06100b0790 2023-10-09T06:42:08.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.509+0000 7f0614e09700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06100c78e0 con 0x7f06101591e0 2023-10-09T06:42:08.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.509+0000 7f060f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06100b03c0 0x7f06100c5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:08.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.509+0000 7f060f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06100b03c0 0x7f06100c5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:54930/0 (socket says 172.21.15.163:54930) 2023-10-09T06:42:08.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.509+0000 7f060f7fe700 1 -- 172.21.15.163:0/3672963680 learned_addr learned my addr 172.21.15.163:0/3672963680 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:08.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.510+0000 7f060effd700 1 --2- 172.21.15.163:0/3672963680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06100b0790 0x7f0610159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:08.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.510+0000 7f060ffff700 1 --2- 172.21.15.163:0/3672963680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06101591e0 0x7f061015a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:08.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.510+0000 7f060f7fe700 1 -- 172.21.15.163:0/3672963680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06101591e0 msgr2=0x7f061015a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:08.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.510+0000 7f060f7fe700 1 --2- 172.21.15.163:0/3672963680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06101591e0 0x7f061015a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:08.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.510+0000 7f060f7fe700 1 -- 172.21.15.163:0/3672963680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06100b0790 msgr2=0x7f0610159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:08.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.510+0000 7f060f7fe700 1 --2- 172.21.15.163:0/3672963680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06100b0790 0x7f0610159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:08.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.510+0000 7f060f7fe700 1 -- 172.21.15.163:0/3672963680 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0604009420 con 0x7f06100b03c0 2023-10-09T06:42:08.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.510+0000 7f060effd700 1 --2- 172.21.15.163:0/3672963680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06100b0790 0x7f0610159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:42:08.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.510+0000 7f060f7fe700 1 --2- 172.21.15.163:0/3672963680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06100b03c0 0x7f06100c5520 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f060000cfb0 tx=0x7f0600007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:08.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.510+0000 7f05f77fe700 1 -- 172.21.15.163:0/3672963680 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0600016420 con 0x7f06100b03c0 2023-10-09T06:42:08.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.510+0000 7f05f77fe700 1 -- 172.21.15.163:0/3672963680 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0600016580 con 0x7f06100b03c0 2023-10-09T06:42:08.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.510+0000 7f0614e09700 1 -- 172.21.15.163:0/3672963680 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06100c7bc0 con 0x7f06100b03c0 2023-10-09T06:42:08.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.510+0000 7f05f77fe700 1 -- 172.21.15.163:0/3672963680 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0600016940 con 0x7f06100b03c0 2023-10-09T06:42:08.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.510+0000 7f0614e09700 1 -- 172.21.15.163:0/3672963680 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061016a1f0 con 0x7f06100b03c0 2023-10-09T06:42:08.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.511+0000 7f0614e09700 1 -- 172.21.15.163:0/3672963680 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f061004c3c0 con 0x7f06100b03c0 2023-10-09T06:42:08.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.511+0000 7f05f77fe700 1 -- 172.21.15.163:0/3672963680 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0600016d30 con 0x7f06100b03c0 2023-10-09T06:42:08.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.512+0000 7f05f77fe700 1 --2- 172.21.15.163:0/3672963680 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f05fc067320 0x7f05fc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:08.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.512+0000 7f05f77fe700 1 -- 172.21.15.163:0/3672963680 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5878+0+0 (secure 0 0 0) 0x7f0600012070 con 0x7f06100b03c0 2023-10-09T06:42:08.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.512+0000 7f060effd700 1 --2- 172.21.15.163:0/3672963680 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f05fc067320 0x7f05fc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:08.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.512+0000 7f060effd700 1 --2- 172.21.15.163:0/3672963680 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f05fc067320 0x7f05fc0697d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f06100c6470 tx=0x7f0604017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:08.516 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.516+0000 7f05f77fe700 1 -- 172.21.15.163:0/3672963680 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f060004e5a0 con 0x7f06100b03c0 2023-10-09T06:42:08.702 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:08.702+0000 7f0614e09700 1 -- 172.21.15.163:0/3672963680 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f06100cad50 con 0x7f06100b03c0 2023-10-09T06:42:09.596 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.596+0000 7f05f77fe700 1 -- 172.21.15.163:0/3672963680 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v201) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0600055d00 con 0x7f06100b03c0 2023-10-09T06:42:09.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.600+0000 7f0614e09700 1 -- 172.21.15.163:0/3672963680 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f05fc067320 msgr2=0x7f05fc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:09.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.600+0000 7f0614e09700 1 --2- 172.21.15.163:0/3672963680 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f05fc067320 0x7f05fc0697d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f06100c6470 tx=0x7f0604017000 comp rx=0 tx=0).stop 2023-10-09T06:42:09.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.600+0000 7f0614e09700 1 -- 172.21.15.163:0/3672963680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06100b03c0 msgr2=0x7f06100c5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:09.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.600+0000 7f0614e09700 1 --2- 172.21.15.163:0/3672963680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06100b03c0 0x7f06100c5520 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f060000cfb0 tx=0x7f0600007870 comp rx=0 tx=0).stop 2023-10-09T06:42:09.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.600+0000 7f0614e09700 1 -- 172.21.15.163:0/3672963680 shutdown_connections 2023-10-09T06:42:09.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.600+0000 7f0614e09700 1 --2- 172.21.15.163:0/3672963680 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f05fc067320 0x7f05fc0697d0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:09.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.600+0000 7f0614e09700 1 --2- 172.21.15.163:0/3672963680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06100b03c0 0x7f06100c5520 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:09.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.600+0000 7f0614e09700 1 --2- 172.21.15.163:0/3672963680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06101591e0 0x7f061015a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:09.600 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.600+0000 7f0614e09700 1 --2- 172.21.15.163:0/3672963680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06100b0790 0x7f0610159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:09.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.600+0000 7f0614e09700 1 -- 172.21.15.163:0/3672963680 >> 172.21.15.163:0/3672963680 conn(0x7f0610063230 msgr2=0x7f06100cc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:09.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.600+0000 7f0614e09700 1 -- 172.21.15.163:0/3672963680 shutdown_connections 2023-10-09T06:42:09.601 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.600+0000 7f0614e09700 1 -- 172.21.15.163:0/3672963680 wait complete. 2023-10-09T06:42:09.601 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_metadata' removed 2023-10-09T06:42:09.613 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-09T06:42:09.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.737+0000 7f35f147a700 1 Processor -- start 2023-10-09T06:42:09.737 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.737+0000 7f35f147a700 1 -- start start 2023-10-09T06:42:09.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.738+0000 7f35f147a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f35ec0b03c0 0x7f35ec0cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:09.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.738+0000 7f35f147a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f35ec0b0790 0x7f35ec0b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:09.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.738+0000 7f35f147a700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35ec0cfa00 con 0x7f35ec1591e0 2023-10-09T06:42:09.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.738+0000 7f35f147a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35ec0cb580 con 0x7f35ec0b03c0 2023-10-09T06:42:09.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.738+0000 7f35f147a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35ec0b2de0 con 0x7f35ec0b0790 2023-10-09T06:42:09.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.738+0000 7f35ebfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f35ec0b03c0 0x7f35ec0cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:09.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.738+0000 7f35f0c79700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f35ec1591e0 0x7f35ec159600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60554/0 (socket says 172.21.15.163:60554) 2023-10-09T06:42:09.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.738+0000 7f35f0c79700 1 -- 172.21.15.163:0/556397842 learned_addr learned my addr 172.21.15.163:0/556397842 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:42:09.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.738+0000 7f35ebfff700 1 -- 172.21.15.163:0/556397842 >> v2:172.21.15.181:3300/0 conn(0x7f35ec0b0790 msgr2=0x7f35ec0b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:09.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.738+0000 7f35eb7fe700 1 --2- 172.21.15.163:0/556397842 >> v2:172.21.15.181:3300/0 conn(0x7f35ec0b0790 0x7f35ec0b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:09.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.738+0000 7f35ebfff700 1 --2- 172.21.15.163:0/556397842 >> v2:172.21.15.181:3300/0 conn(0x7f35ec0b0790 0x7f35ec0b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:09.740 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.738+0000 7f35eaffd700 1 -- 172.21.15.163:0/556397842 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 170803767 0 0) 0x7f35ec0cfa00 con 0x7f35ec1591e0 2023-10-09T06:42:09.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.738+0000 7f35ebfff700 1 -- 172.21.15.163:0/556397842 >> v1:172.21.15.163:6789/0 conn(0x7f35ec1591e0 legacy=0x7f35ec159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:09.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.738+0000 7f35ebfff700 1 -- 172.21.15.163:0/556397842 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ec0cf160 con 0x7f35ec0b03c0 2023-10-09T06:42:09.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.739+0000 7f35ebfff700 1 --2- 172.21.15.163:0/556397842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35ec0b03c0 0x7f35ec0cea70 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0098e0 tx=0x7f35dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d195b471b09fa4c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:09.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.739+0000 7f35eaffd700 1 -- 172.21.15.163:0/556397842 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc009e80 con 0x7f35ec0b03c0 2023-10-09T06:42:09.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.739+0000 7f35eaffd700 1 -- 172.21.15.163:0/556397842 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35dc00b820 con 0x7f35ec0b03c0 2023-10-09T06:42:09.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.739+0000 7f35eaffd700 1 -- 172.21.15.163:0/556397842 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc00bba0 con 0x7f35ec0b03c0 2023-10-09T06:42:09.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.739+0000 7f35f147a700 1 -- 172.21.15.163:0/556397842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35ec0b03c0 msgr2=0x7f35ec0cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:09.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.739+0000 7f35f147a700 1 --2- 172.21.15.163:0/556397842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35ec0b03c0 0x7f35ec0cea70 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0098e0 tx=0x7f35dc0091a0 comp rx=0 tx=0).stop 2023-10-09T06:42:09.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.739+0000 7f35f147a700 1 -- 172.21.15.163:0/556397842 shutdown_connections 2023-10-09T06:42:09.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.739+0000 7f35f147a700 1 --2- 172.21.15.163:0/556397842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35ec0b03c0 0x7f35ec0cea70 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:09.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.739+0000 7f35f147a700 1 --2- 172.21.15.163:0/556397842 >> v2:172.21.15.181:3300/0 conn(0x7f35ec0b0790 0x7f35ec0b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:09.741 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.739+0000 7f35f147a700 1 -- 172.21.15.163:0/556397842 >> 172.21.15.163:0/556397842 conn(0x7f35ec063230 msgr2=0x7f35ec063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:09.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.739+0000 7f35f147a700 1 -- 172.21.15.163:0/556397842 shutdown_connections 2023-10-09T06:42:09.742 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.739+0000 7f35f147a700 1 -- 172.21.15.163:0/556397842 wait complete. 2023-10-09T06:42:09.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.740+0000 7f35f147a700 1 Processor -- start 2023-10-09T06:42:09.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.741+0000 7f35f147a700 1 -- start start 2023-10-09T06:42:09.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.741+0000 7f35f147a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35ec0b03c0 0x7f35ec1575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:09.744 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.741+0000 7f35f147a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35ec0b0790 0x7f35ec0c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:09.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.741+0000 7f35f147a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35ec1591e0 0x7f35ec1585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:09.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.741+0000 7f35f147a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec0c8350 con 0x7f35ec0b03c0 2023-10-09T06:42:09.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.741+0000 7f35f147a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec0c7740 con 0x7f35ec0b0790 2023-10-09T06:42:09.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.741+0000 7f35f147a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec0c7a20 con 0x7f35ec1591e0 2023-10-09T06:42:09.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.741+0000 7f35ebfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35ec0b03c0 0x7f35ec1575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:09.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.741+0000 7f35ebfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35ec0b03c0 0x7f35ec1575b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:54942/0 (socket says 172.21.15.163:54942) 2023-10-09T06:42:09.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.741+0000 7f35ebfff700 1 -- 172.21.15.163:0/251208277 learned_addr learned my addr 172.21.15.163:0/251208277 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:09.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.742+0000 7f35eb7fe700 1 --2- 172.21.15.163:0/251208277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35ec0b0790 0x7f35ec0c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:09.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.742+0000 7f35f0c79700 1 --2- 172.21.15.163:0/251208277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35ec1591e0 0x7f35ec1585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:09.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.742+0000 7f35ebfff700 1 -- 172.21.15.163:0/251208277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35ec1591e0 msgr2=0x7f35ec1585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:09.745 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.742+0000 7f35ebfff700 1 --2- 172.21.15.163:0/251208277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35ec1591e0 0x7f35ec1585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:09.746 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.742+0000 7f35ebfff700 1 -- 172.21.15.163:0/251208277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35ec0b0790 msgr2=0x7f35ec0c87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:09.746 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.742+0000 7f35ebfff700 1 --2- 172.21.15.163:0/251208277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35ec0b0790 0x7f35ec0c87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:09.746 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.742+0000 7f35ebfff700 1 -- 172.21.15.163:0/251208277 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35dc009420 con 0x7f35ec0b03c0 2023-10-09T06:42:09.746 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.742+0000 7f35ebfff700 1 --2- 172.21.15.163:0/251208277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35ec0b03c0 0x7f35ec1575b0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f35ec0cfe40 tx=0x7f35dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:09.746 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.742+0000 7f35d3fff700 1 -- 172.21.15.163:0/251208277 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc00b990 con 0x7f35ec0b03c0 2023-10-09T06:42:09.746 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.742+0000 7f35d3fff700 1 -- 172.21.15.163:0/251208277 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35dc00baf0 con 0x7f35ec0b03c0 2023-10-09T06:42:09.746 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.742+0000 7f35f147a700 1 -- 172.21.15.163:0/251208277 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35ec0c7d00 con 0x7f35ec0b03c0 2023-10-09T06:42:09.746 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.742+0000 7f35d3fff700 1 -- 172.21.15.163:0/251208277 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc025670 con 0x7f35ec0b03c0 2023-10-09T06:42:09.746 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.742+0000 7f35f147a700 1 -- 172.21.15.163:0/251208277 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ec157af0 con 0x7f35ec0b03c0 2023-10-09T06:42:09.746 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.743+0000 7f35f147a700 1 -- 172.21.15.163:0/251208277 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35ec04c3c0 con 0x7f35ec0b03c0 2023-10-09T06:42:09.746 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.743+0000 7f35d3fff700 1 -- 172.21.15.163:0/251208277 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f35dc023070 con 0x7f35ec0b03c0 2023-10-09T06:42:09.746 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.744+0000 7f35d3fff700 1 --2- 172.21.15.163:0/251208277 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f35d8067320 0x7f35d80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:09.747 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.744+0000 7f35d3fff700 1 -- 172.21.15.163:0/251208277 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5440+0+0 (secure 0 0 0) 0x7f35dc0861c0 con 0x7f35ec0b03c0 2023-10-09T06:42:09.747 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.744+0000 7f35eb7fe700 1 --2- 172.21.15.163:0/251208277 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f35d8067320 0x7f35d80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:09.747 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.744+0000 7f35eb7fe700 1 --2- 172.21.15.163:0/251208277 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f35d8067320 0x7f35d80697d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f35ec0693d0 tx=0x7f35e0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:09.748 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.748+0000 7f35d3fff700 1 -- 172.21.15.163:0/251208277 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f35dc052f60 con 0x7f35ec0b03c0 2023-10-09T06:42:09.932 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:09.932+0000 7f35f147a700 1 -- 172.21.15.163:0/251208277 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f35ec0cad50 con 0x7f35ec0b03c0 2023-10-09T06:42:10.803 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.803+0000 7f35d3fff700 1 -- 172.21.15.163:0/251208277 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v202) v1 ==== 169+0+0 (secure 0 0 0) 0x7f35dc056580 con 0x7f35ec0b03c0 2023-10-09T06:42:10.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.806+0000 7f35f147a700 1 -- 172.21.15.163:0/251208277 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f35d8067320 msgr2=0x7f35d80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:10.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.806+0000 7f35f147a700 1 --2- 172.21.15.163:0/251208277 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f35d8067320 0x7f35d80697d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f35ec0693d0 tx=0x7f35e0008040 comp rx=0 tx=0).stop 2023-10-09T06:42:10.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.806+0000 7f35f147a700 1 -- 172.21.15.163:0/251208277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35ec0b03c0 msgr2=0x7f35ec1575b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:10.806 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.806+0000 7f35f147a700 1 --2- 172.21.15.163:0/251208277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35ec0b03c0 0x7f35ec1575b0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f35ec0cfe40 tx=0x7f35dc017000 comp rx=0 tx=0).stop 2023-10-09T06:42:10.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.806+0000 7f35f147a700 1 -- 172.21.15.163:0/251208277 shutdown_connections 2023-10-09T06:42:10.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.806+0000 7f35f147a700 1 --2- 172.21.15.163:0/251208277 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f35d8067320 0x7f35d80697d0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:10.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.806+0000 7f35f147a700 1 --2- 172.21.15.163:0/251208277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35ec0b03c0 0x7f35ec1575b0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:10.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.806+0000 7f35f147a700 1 --2- 172.21.15.163:0/251208277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35ec1591e0 0x7f35ec1585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:10.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.806+0000 7f35f147a700 1 --2- 172.21.15.163:0/251208277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f35ec0b0790 0x7f35ec0c87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:10.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.806+0000 7f35f147a700 1 -- 172.21.15.163:0/251208277 >> 172.21.15.163:0/251208277 conn(0x7f35ec063230 msgr2=0x7f35ec0d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:10.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.806+0000 7f35f147a700 1 -- 172.21.15.163:0/251208277 shutdown_connections 2023-10-09T06:42:10.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.806+0000 7f35f147a700 1 -- 172.21.15.163:0/251208277 wait complete. 2023-10-09T06:42:10.807 INFO:teuthology.orchestra.run.smithi163.stderr:pool 'cephfs_data' removed 2023-10-09T06:42:10.820 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-09T06:42:10.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.939+0000 7ff61ffff700 1 Processor -- start 2023-10-09T06:42:10.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.939+0000 7ff61ffff700 1 -- start start 2023-10-09T06:42:10.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.940+0000 7ff61ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6200b4e70 0x7ff6200ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:10.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.940+0000 7ff61ffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff6200b1b40 0x7ff62015d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:10.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.940+0000 7ff61ffff700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6200cfb00 con 0x7ff6200b16d0 2023-10-09T06:42:10.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.940+0000 7ff61ffff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6200b5c40 con 0x7ff6200b1b40 2023-10-09T06:42:10.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.940+0000 7ff61ffff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6200b5f20 con 0x7ff6200b4e70 2023-10-09T06:42:10.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.940+0000 7ff61f7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff6200b1b40 0x7ff62015d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:10.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.940+0000 7ff61f7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff6200b1b40 0x7ff62015d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:54958/0 (socket says 172.21.15.163:54958) 2023-10-09T06:42:10.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.940+0000 7ff61f7fe700 1 -- 172.21.15.163:0/420837199 learned_addr learned my addr 172.21.15.163:0/420837199 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:10.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.940+0000 7ff61effd700 1 --2- 172.21.15.163:0/420837199 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6200b4e70 0x7ff6200ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:10.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.941+0000 7ff61f7fe700 1 -- 172.21.15.163:0/420837199 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6200b4e70 msgr2=0x7ff6200ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:10.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.941+0000 7ff61f7fe700 1 --2- 172.21.15.163:0/420837199 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6200b4e70 0x7ff6200ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:10.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.941+0000 7ff61f7fe700 1 -- 172.21.15.163:0/420837199 >> v1:172.21.15.181:6789/0 conn(0x7ff6200b16d0 legacy=0x7ff62015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:10.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.941+0000 7ff61f7fe700 1 -- 172.21.15.163:0/420837199 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6200b6200 con 0x7ff6200b1b40 2023-10-09T06:42:10.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.941+0000 7ff61f7fe700 1 --2- 172.21.15.163:0/420837199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6200b1b40 0x7ff62015d950 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7ff6180099b0 tx=0x7ff618009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=659592b56750b7c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:10.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.941+0000 7ff61dffb700 1 -- 172.21.15.163:0/420837199 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff618009ea0 con 0x7ff6200b1b40 2023-10-09T06:42:10.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.941+0000 7ff61dffb700 1 -- 172.21.15.163:0/420837199 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61800b860 con 0x7ff6200b1b40 2023-10-09T06:42:10.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.941+0000 7ff61dffb700 1 -- 172.21.15.163:0/420837199 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61800bc00 con 0x7ff6200b1b40 2023-10-09T06:42:10.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.941+0000 7ff61ffff700 1 -- 172.21.15.163:0/420837199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6200b1b40 msgr2=0x7ff62015d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:10.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.941+0000 7ff61ffff700 1 --2- 172.21.15.163:0/420837199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6200b1b40 0x7ff62015d950 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7ff6180099b0 tx=0x7ff618009250 comp rx=0 tx=0).stop 2023-10-09T06:42:10.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.941+0000 7ff61ffff700 1 -- 172.21.15.163:0/420837199 shutdown_connections 2023-10-09T06:42:10.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.941+0000 7ff61ffff700 1 --2- 172.21.15.163:0/420837199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6200b1b40 0x7ff62015d950 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:10.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.941+0000 7ff61ffff700 1 --2- 172.21.15.163:0/420837199 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6200b4e70 0x7ff6200ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:10.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.941+0000 7ff61ffff700 1 -- 172.21.15.163:0/420837199 >> 172.21.15.163:0/420837199 conn(0x7ff6200d10c0 msgr2=0x7ff6200affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:10.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.941+0000 7ff61ffff700 1 -- 172.21.15.163:0/420837199 shutdown_connections 2023-10-09T06:42:10.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.941+0000 7ff61ffff700 1 -- 172.21.15.163:0/420837199 wait complete. 2023-10-09T06:42:10.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.943+0000 7ff61ffff700 1 Processor -- start 2023-10-09T06:42:10.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.943+0000 7ff61ffff700 1 -- start start 2023-10-09T06:42:10.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.943+0000 7ff61ffff700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6200b16d0 0x7ff6200c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:10.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.943+0000 7ff61ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6200b1b40 0x7ff620156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:10.947 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.943+0000 7ff61ffff700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6200b4e70 0x7ff6200cba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:10.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.943+0000 7ff61ffff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff620144be0 con 0x7ff6200b1b40 2023-10-09T06:42:10.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.943+0000 7ff61ffff700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6200c7060 con 0x7ff6200b16d0 2023-10-09T06:42:10.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.943+0000 7ff61ffff700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6200c6730 con 0x7ff6200b4e70 2023-10-09T06:42:10.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.943+0000 7ff61e7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6200b1b40 0x7ff620156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:10.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.943+0000 7ff61e7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6200b1b40 0x7ff620156ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:54960/0 (socket says 172.21.15.163:54960) 2023-10-09T06:42:10.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.943+0000 7ff61e7fc700 1 -- 172.21.15.163:0/1784214616 learned_addr learned my addr 172.21.15.163:0/1784214616 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:10.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.944+0000 7ff61f7fe700 1 --2- 172.21.15.163:0/1784214616 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6200b4e70 0x7ff6200cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:10.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.944+0000 7ff61effd700 1 --2- 172.21.15.163:0/1784214616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6200b16d0 0x7ff6200c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:10.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.944+0000 7ff61e7fc700 1 -- 172.21.15.163:0/1784214616 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6200b4e70 msgr2=0x7ff6200cba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:10.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.944+0000 7ff61e7fc700 1 --2- 172.21.15.163:0/1784214616 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6200b4e70 0x7ff6200cba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:10.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.944+0000 7ff61e7fc700 1 -- 172.21.15.163:0/1784214616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6200b16d0 msgr2=0x7ff6200c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:10.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.944+0000 7ff61e7fc700 1 --2- 172.21.15.163:0/1784214616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6200b16d0 0x7ff6200c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:10.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.944+0000 7ff61e7fc700 1 -- 172.21.15.163:0/1784214616 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6180094f0 con 0x7ff6200b1b40 2023-10-09T06:42:10.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.944+0000 7ff61e7fc700 1 --2- 172.21.15.163:0/1784214616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6200b1b40 0x7ff620156ec0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7ff61400c590 tx=0x7ff61400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:10.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.944+0000 7ff606ffd700 1 -- 172.21.15.163:0/1784214616 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff614011590 con 0x7ff6200b1b40 2023-10-09T06:42:10.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.944+0000 7ff606ffd700 1 -- 172.21.15.163:0/1784214616 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6140116f0 con 0x7ff6200b1b40 2023-10-09T06:42:10.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.944+0000 7ff61ffff700 1 -- 172.21.15.163:0/1784214616 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6200c6a10 con 0x7ff6200b1b40 2023-10-09T06:42:10.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.944+0000 7ff606ffd700 1 -- 172.21.15.163:0/1784214616 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff614011ab0 con 0x7ff6200b1b40 2023-10-09T06:42:10.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.944+0000 7ff61ffff700 1 -- 172.21.15.163:0/1784214616 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6200cbfb0 con 0x7ff6200b1b40 2023-10-09T06:42:10.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.945+0000 7ff61ffff700 1 -- 172.21.15.163:0/1784214616 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6200c7210 con 0x7ff6200b1b40 2023-10-09T06:42:10.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.945+0000 7ff606ffd700 1 -- 172.21.15.163:0/1784214616 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff61401c440 con 0x7ff6200b1b40 2023-10-09T06:42:10.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.946+0000 7ff606ffd700 1 --2- 172.21.15.163:0/1784214616 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff60c067320 0x7ff60c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:10.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.946+0000 7ff606ffd700 1 -- 172.21.15.163:0/1784214616 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff614083ec0 con 0x7ff6200b1b40 2023-10-09T06:42:10.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.946+0000 7ff61effd700 1 --2- 172.21.15.163:0/1784214616 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff60c067320 0x7ff60c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:10.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.946+0000 7ff61effd700 1 --2- 172.21.15.163:0/1784214616 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff60c067320 0x7ff60c0697d0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff610007620 tx=0x7ff61000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:10.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:10.950+0000 7ff606ffd700 1 -- 172.21.15.163:0/1784214616 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6200c7210 con 0x7ff6200b1b40 2023-10-09T06:42:11.131 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.131+0000 7ff61ffff700 1 -- 172.21.15.163:0/1784214616 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff6200c71d0 con 0x7ff6200b1b40 2023-10-09T06:42:11.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.135+0000 7ff606ffd700 1 -- 172.21.15.163:0/1784214616 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff614017020 con 0x7ff6200b1b40 2023-10-09T06:42:11.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.137+0000 7ff61ffff700 1 -- 172.21.15.163:0/1784214616 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff60c067320 msgr2=0x7ff60c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:11.137 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.137+0000 7ff61ffff700 1 --2- 172.21.15.163:0/1784214616 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff60c067320 0x7ff60c0697d0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff610007620 tx=0x7ff61000b040 comp rx=0 tx=0).stop 2023-10-09T06:42:11.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.137+0000 7ff61ffff700 1 -- 172.21.15.163:0/1784214616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6200b1b40 msgr2=0x7ff620156ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:11.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.137+0000 7ff61ffff700 1 --2- 172.21.15.163:0/1784214616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6200b1b40 0x7ff620156ec0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7ff61400c590 tx=0x7ff61400c950 comp rx=0 tx=0).stop 2023-10-09T06:42:11.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.137+0000 7ff61ffff700 1 -- 172.21.15.163:0/1784214616 shutdown_connections 2023-10-09T06:42:11.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.137+0000 7ff61ffff700 1 --2- 172.21.15.163:0/1784214616 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff60c067320 0x7ff60c0697d0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:11.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.137+0000 7ff61ffff700 1 --2- 172.21.15.163:0/1784214616 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6200b4e70 0x7ff6200cba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:11.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.137+0000 7ff61ffff700 1 --2- 172.21.15.163:0/1784214616 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6200b16d0 0x7ff6200c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:11.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.137+0000 7ff61ffff700 1 --2- 172.21.15.163:0/1784214616 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6200b1b40 0x7ff620156ec0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:11.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.137+0000 7ff61ffff700 1 -- 172.21.15.163:0/1784214616 >> 172.21.15.163:0/1784214616 conn(0x7ff6200d10c0 msgr2=0x7ff620159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:11.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.137+0000 7ff61ffff700 1 -- 172.21.15.163:0/1784214616 shutdown_connections 2023-10-09T06:42:11.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.137+0000 7ff61ffff700 1 -- 172.21.15.163:0/1784214616 wait complete. 2023-10-09T06:42:11.150 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rm' 2023-10-09T06:42:11.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.265+0000 7f3803056700 1 Processor -- start 2023-10-09T06:42:11.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.265+0000 7f3803056700 1 -- start start 2023-10-09T06:42:11.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.266+0000 7f3803056700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37fc159890 0x7f37fc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:11.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.266+0000 7f3803056700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f37fc069560 0x7f37fc0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:11.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.266+0000 7f3803056700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37fc15a160 con 0x7f37fc159c60 2023-10-09T06:42:11.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.266+0000 7f3803056700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37fc069f20 con 0x7f37fc069560 2023-10-09T06:42:11.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.266+0000 7f3803056700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc0d13a0 con 0x7f37fc159890 2023-10-09T06:42:11.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.266+0000 7f37fbfff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f37fc159c60 0x7f37fc15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60564/0 (socket says 172.21.15.163:60564) 2023-10-09T06:42:11.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.266+0000 7f37fbfff700 1 -- 172.21.15.163:0/273471147 learned_addr learned my addr 172.21.15.163:0/273471147 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:42:11.269 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.266+0000 7f38015f3700 1 --2- 172.21.15.163:0/273471147 >> v2:172.21.15.181:3300/0 conn(0x7f37fc069560 0x7f37fc0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:11.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.266+0000 7f3800df2700 1 --2- 172.21.15.163:0/273471147 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37fc159890 0x7f37fc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:11.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.266+0000 7f37fb7fe700 1 -- 172.21.15.163:0/273471147 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 278962522 0 0) 0x7f37fc15a160 con 0x7f37fc159c60 2023-10-09T06:42:11.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.266+0000 7f37fb7fe700 1 -- 172.21.15.163:0/273471147 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37e4003570 con 0x7f37fc159c60 2023-10-09T06:42:11.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.267+0000 7f37fb7fe700 1 -- 172.21.15.163:0/273471147 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1579112701 0 0) 0x7f37e4003570 con 0x7f37fc159c60 2023-10-09T06:42:11.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.267+0000 7f37fb7fe700 1 -- 172.21.15.163:0/273471147 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37fc159890 msgr2=0x7f37fc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:11.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.267+0000 7f37fb7fe700 1 --2- 172.21.15.163:0/273471147 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37fc159890 0x7f37fc15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:11.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.267+0000 7f37fb7fe700 1 -- 172.21.15.163:0/273471147 >> v2:172.21.15.181:3300/0 conn(0x7f37fc069560 msgr2=0x7f37fc0699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:11.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.267+0000 7f37fb7fe700 1 --2- 172.21.15.163:0/273471147 >> v2:172.21.15.181:3300/0 conn(0x7f37fc069560 0x7f37fc0699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:11.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.267+0000 7f37fb7fe700 1 -- 172.21.15.163:0/273471147 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37fc15e140 con 0x7f37fc159c60 2023-10-09T06:42:11.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.267+0000 7f37fb7fe700 1 -- 172.21.15.163:0/273471147 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f37f0001470 con 0x7f37fc159c60 2023-10-09T06:42:11.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.267+0000 7f37fb7fe700 1 -- 172.21.15.163:0/273471147 >> v1:172.21.15.163:6789/0 conn(0x7f37fc159c60 legacy=0x7f37fc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:11.270 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.267+0000 7f37fb7fe700 1 --2- 172.21.15.163:0/273471147 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37fc159890 0x7f37e4003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:11.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.267+0000 7f37fb7fe700 1 --2- 172.21.15.163:0/273471147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37e4003fe0 0x7f37e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:11.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.267+0000 7f37fb7fe700 1 --2- 172.21.15.163:0/273471147 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37e4004d50 0x7f37e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:11.271 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.267+0000 7f37fb7fe700 1 -- 172.21.15.163:0/273471147 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc069f20 con 0x7f37e4004d50 2023-10-09T06:42:11.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.267+0000 7f37fb7fe700 1 -- 172.21.15.163:0/273471147 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc0d13a0 con 0x7f37e4003fe0 2023-10-09T06:42:11.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.267+0000 7f37fb7fe700 1 -- 172.21.15.163:0/273471147 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e400ba90 con 0x7f37fc159890 2023-10-09T06:42:11.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.267+0000 7f38015f3700 1 --2- 172.21.15.163:0/273471147 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37e4004d50 0x7f37e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:11.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.267+0000 7f3800df2700 1 --2- 172.21.15.163:0/273471147 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37fc159890 0x7f37e4003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:11.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.267+0000 7f37fbfff700 1 --2- 172.21.15.163:0/273471147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37e4003fe0 0x7f37e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:11.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.267+0000 7f38015f3700 1 -- 172.21.15.163:0/273471147 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37fc159890 msgr2=0x7f37e4003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:11.272 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.267+0000 7f38015f3700 1 --2- 172.21.15.163:0/273471147 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37fc159890 0x7f37e4003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:11.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.267+0000 7f38015f3700 1 -- 172.21.15.163:0/273471147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37e4003fe0 msgr2=0x7f37e4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:11.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.267+0000 7f38015f3700 1 --2- 172.21.15.163:0/273471147 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37e4003fe0 0x7f37e4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:11.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.267+0000 7f38015f3700 1 -- 172.21.15.163:0/273471147 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e4003570 con 0x7f37e4004d50 2023-10-09T06:42:11.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.267+0000 7f38015f3700 1 --2- 172.21.15.163:0/273471147 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37e4004d50 0x7f37e40051c0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f37f4009c00 tx=0x7f37f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75402391b6bfbeb8 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:11.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.267+0000 7f37fb7fe700 1 -- 172.21.15.163:0/273471147 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f400e6b0 con 0x7f37e4004d50 2023-10-09T06:42:11.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.267+0000 7f37fb7fe700 1 -- 172.21.15.163:0/273471147 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f400e810 con 0x7f37e4004d50 2023-10-09T06:42:11.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.267+0000 7f37fb7fe700 1 -- 172.21.15.163:0/273471147 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f400ebd0 con 0x7f37e4004d50 2023-10-09T06:42:11.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.268+0000 7f3803056700 1 -- 172.21.15.163:0/273471147 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37e4004d50 msgr2=0x7f37e40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:11.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.268+0000 7f3803056700 1 --2- 172.21.15.163:0/273471147 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37e4004d50 0x7f37e40051c0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f37f4009c00 tx=0x7f37f40091a0 comp rx=0 tx=0).stop 2023-10-09T06:42:11.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.268+0000 7f38015f3700 1 -- 172.21.15.163:0/273471147 reap_dead start 2023-10-09T06:42:11.273 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.268+0000 7f3803056700 1 -- 172.21.15.163:0/273471147 shutdown_connections 2023-10-09T06:42:11.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.268+0000 7f3803056700 1 -- 172.21.15.163:0/273471147 >> 172.21.15.163:0/273471147 conn(0x7f37fc068e20 msgr2=0x7f37fc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:11.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.268+0000 7f3803056700 1 -- 172.21.15.163:0/273471147 shutdown_connections 2023-10-09T06:42:11.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.268+0000 7f3803056700 1 -- 172.21.15.163:0/273471147 wait complete. 2023-10-09T06:42:11.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.269+0000 7f3803056700 1 Processor -- start 2023-10-09T06:42:11.274 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.269+0000 7f3803056700 1 -- start start 2023-10-09T06:42:11.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.270+0000 7f3803056700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37fc149e50 0x7f37fc0ce280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:11.276 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.270+0000 7f3803056700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37fc156d20 0x7f37fc0b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:11.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.270+0000 7f3803056700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37fc0b1ee0 0x7f37fc155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:11.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.270+0000 7f3803056700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc0cd710 con 0x7f37fc156d20 2023-10-09T06:42:11.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.270+0000 7f3803056700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc0b2380 con 0x7f37fc0b1ee0 2023-10-09T06:42:11.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.270+0000 7f3803056700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc157ea0 con 0x7f37fc149e50 2023-10-09T06:42:11.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.270+0000 7f37fbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37fc156d20 0x7f37fc0b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:11.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.270+0000 7f37fbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37fc156d20 0x7f37fc0b19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:54982/0 (socket says 172.21.15.163:54982) 2023-10-09T06:42:11.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.270+0000 7f37fbfff700 1 -- 172.21.15.163:0/3449606852 learned_addr learned my addr 172.21.15.163:0/3449606852 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:11.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.270+0000 7f38015f3700 1 --2- 172.21.15.163:0/3449606852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37fc0b1ee0 0x7f37fc155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:11.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.270+0000 7f3800df2700 1 --2- 172.21.15.163:0/3449606852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37fc149e50 0x7f37fc0ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:11.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.270+0000 7f37fbfff700 1 -- 172.21.15.163:0/3449606852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37fc149e50 msgr2=0x7f37fc0ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:11.277 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.270+0000 7f37fbfff700 1 --2- 172.21.15.163:0/3449606852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37fc149e50 0x7f37fc0ce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:11.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.270+0000 7f37fbfff700 1 -- 172.21.15.163:0/3449606852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37fc0b1ee0 msgr2=0x7f37fc155a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:11.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.270+0000 7f37fbfff700 1 --2- 172.21.15.163:0/3449606852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37fc0b1ee0 0x7f37fc155a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:11.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.270+0000 7f37fbfff700 1 -- 172.21.15.163:0/3449606852 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37fc0ce7c0 con 0x7f37fc156d20 2023-10-09T06:42:11.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.270+0000 7f38015f3700 1 --2- 172.21.15.163:0/3449606852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37fc0b1ee0 0x7f37fc155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:42:11.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.271+0000 7f37fbfff700 1 --2- 172.21.15.163:0/3449606852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37fc156d20 0x7f37fc0b19a0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f37f00074c0 tx=0x7f37f0001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:11.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.271+0000 7f37f8ff9700 1 -- 172.21.15.163:0/3449606852 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f0018030 con 0x7f37fc156d20 2023-10-09T06:42:11.278 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.271+0000 7f37f8ff9700 1 -- 172.21.15.163:0/3449606852 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f000fa00 con 0x7f37fc156d20 2023-10-09T06:42:11.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.271+0000 7f3803056700 1 -- 172.21.15.163:0/3449606852 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37fc14a220 con 0x7f37fc156d20 2023-10-09T06:42:11.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.271+0000 7f37f8ff9700 1 -- 172.21.15.163:0/3449606852 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f000fda0 con 0x7f37fc156d20 2023-10-09T06:42:11.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.271+0000 7f3803056700 1 -- 172.21.15.163:0/3449606852 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37fc157570 con 0x7f37fc156d20 2023-10-09T06:42:11.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.272+0000 7f3803056700 1 -- 172.21.15.163:0/3449606852 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37fc04c3c0 con 0x7f37fc156d20 2023-10-09T06:42:11.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.272+0000 7f37f8ff9700 1 -- 172.21.15.163:0/3449606852 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f37f001b070 con 0x7f37fc156d20 2023-10-09T06:42:11.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.272+0000 7f37f8ff9700 1 --2- 172.21.15.163:0/3449606852 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f37e8067320 0x7f37e80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:11.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.273+0000 7f37f8ff9700 1 -- 172.21.15.163:0/3449606852 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f37f0019070 con 0x7f37fc156d20 2023-10-09T06:42:11.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.273+0000 7f3800df2700 1 --2- 172.21.15.163:0/3449606852 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f37e8067320 0x7f37e80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:11.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.273+0000 7f3800df2700 1 --2- 172.21.15.163:0/3449606852 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f37e8067320 0x7f37e80697d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f37ec009fd0 tx=0x7f37ec00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:11.279 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.277+0000 7f37f8ff9700 1 -- 172.21.15.163:0/3449606852 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37f0051b10 con 0x7f37fc156d20 2023-10-09T06:42:11.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.532+0000 7f3803056700 1 -- 172.21.15.163:0/3449606852 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rm"]} v 0) v1 -- 0x7f37fc157fe0 con 0x7f37fc156d20 2023-10-09T06:42:11.802 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.802+0000 7f37f8ff9700 1 -- 172.21.15.163:0/3449606852 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rm"]}]=0 v559) v1 ==== 131+0+0 (secure 0 0 0) 0x7f37f0055130 con 0x7f37fc156d20 2023-10-09T06:42:11.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.804+0000 7f3803056700 1 -- 172.21.15.163:0/3449606852 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f37e8067320 msgr2=0x7f37e80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:11.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.804+0000 7f3803056700 1 --2- 172.21.15.163:0/3449606852 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f37e8067320 0x7f37e80697d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f37ec009fd0 tx=0x7f37ec00b040 comp rx=0 tx=0).stop 2023-10-09T06:42:11.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.804+0000 7f3803056700 1 -- 172.21.15.163:0/3449606852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37fc156d20 msgr2=0x7f37fc0b19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:11.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.804+0000 7f3803056700 1 --2- 172.21.15.163:0/3449606852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37fc156d20 0x7f37fc0b19a0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f37f00074c0 tx=0x7f37f0001e90 comp rx=0 tx=0).stop 2023-10-09T06:42:11.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.804+0000 7f3803056700 1 -- 172.21.15.163:0/3449606852 shutdown_connections 2023-10-09T06:42:11.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.804+0000 7f3803056700 1 --2- 172.21.15.163:0/3449606852 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f37e8067320 0x7f37e80697d0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:11.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.804+0000 7f3803056700 1 --2- 172.21.15.163:0/3449606852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37fc0b1ee0 0x7f37fc155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:11.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.804+0000 7f3803056700 1 --2- 172.21.15.163:0/3449606852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37fc149e50 0x7f37fc0ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:11.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.804+0000 7f3803056700 1 --2- 172.21.15.163:0/3449606852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37fc156d20 0x7f37fc0b19a0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:11.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.804+0000 7f3803056700 1 -- 172.21.15.163:0/3449606852 >> 172.21.15.163:0/3449606852 conn(0x7f37fc068e20 msgr2=0x7f37fc154dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:11.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.805+0000 7f3803056700 1 -- 172.21.15.163:0/3449606852 shutdown_connections 2023-10-09T06:42:11.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:11.805+0000 7f3803056700 1 -- 172.21.15.163:0/3449606852 wait complete. 2023-10-09T06:42:11.862 INFO:tasks.cephfs_test_runner:That the volume can only be removed when --yes-i-really-mean-it is used ... ERROR 2023-10-09T06:42:11.863 INFO:tasks.cephfs_test_runner: 2023-10-09T06:42:11.863 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-09T06:42:11.863 INFO:tasks.cephfs_test_runner:ERROR: test_volume_rm (tasks.cephfs.test_volumes.TestVolumes) 2023-10-09T06:42:11.863 INFO:tasks.cephfs_test_runner:That the volume can only be removed when --yes-i-really-mean-it is used 2023-10-09T06:42:11.863 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-09T06:42:11.863 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-09T06:42:11.863 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_vshankar_ceph_62a27f0d8453e2104f759ed08ba781e24c5ed056/qa/tasks/cephfs/test_volumes.py", line 497, in test_volume_rm 2023-10-09T06:42:11.863 INFO:tasks.cephfs_test_runner: self._fs_cmd("volume", "rm", self.volname) 2023-10-09T06:42:11.863 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_vshankar_ceph_62a27f0d8453e2104f759ed08ba781e24c5ed056/qa/tasks/cephfs/test_volumes.py", line 38, in _fs_cmd 2023-10-09T06:42:11.863 INFO:tasks.cephfs_test_runner: return self.mgr_cluster.mon_manager.raw_cluster_cmd("fs", *args) 2023-10-09T06:42:11.864 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_vshankar_ceph_62a27f0d8453e2104f759ed08ba781e24c5ed056/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd 2023-10-09T06:42:11.864 INFO:tasks.cephfs_test_runner: return self.run_cluster_cmd(**kwargs).stdout.getvalue() 2023-10-09T06:42:11.864 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_vshankar_ceph_62a27f0d8453e2104f759ed08ba781e24c5ed056/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd 2023-10-09T06:42:11.864 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-10-09T06:42:11.864 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run 2023-10-09T06:42:11.864 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-10-09T06:42:11.864 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run 2023-10-09T06:42:11.864 INFO:tasks.cephfs_test_runner: r.wait() 2023-10-09T06:42:11.864 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait 2023-10-09T06:42:11.864 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-09T06:42:11.864 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-09T06:42:11.865 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-09T06:42:11.865 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi163 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs' 2023-10-09T06:42:11.865 INFO:tasks.cephfs_test_runner: 2023-10-09T06:42:11.865 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2023-10-09T06:42:11.865 INFO:tasks.cephfs_test_runner: 2023-10-09T06:42:11.865 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-09T06:42:11.865 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_vshankar_ceph_62a27f0d8453e2104f759ed08ba781e24c5ed056/qa/tasks/cephfs/test_volumes.py", line 503, in test_volume_rm 2023-10-09T06:42:11.865 INFO:tasks.cephfs_test_runner: self._fs_cmd("volume", "rm", self.volname, "--yes-i-really-mean-it") 2023-10-09T06:42:11.865 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_vshankar_ceph_62a27f0d8453e2104f759ed08ba781e24c5ed056/qa/tasks/cephfs/test_volumes.py", line 38, in _fs_cmd 2023-10-09T06:42:11.866 INFO:tasks.cephfs_test_runner: return self.mgr_cluster.mon_manager.raw_cluster_cmd("fs", *args) 2023-10-09T06:42:11.866 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_vshankar_ceph_62a27f0d8453e2104f759ed08ba781e24c5ed056/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd 2023-10-09T06:42:11.866 INFO:tasks.cephfs_test_runner: return self.run_cluster_cmd(**kwargs).stdout.getvalue() 2023-10-09T06:42:11.866 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_vshankar_ceph_62a27f0d8453e2104f759ed08ba781e24c5ed056/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd 2023-10-09T06:42:11.866 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-10-09T06:42:11.866 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run 2023-10-09T06:42:11.866 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-10-09T06:42:11.866 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run 2023-10-09T06:42:11.866 INFO:tasks.cephfs_test_runner: r.wait() 2023-10-09T06:42:11.866 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait 2023-10-09T06:42:11.866 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-09T06:42:11.866 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-09T06:42:11.867 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-09T06:42:11.867 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi163 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it' 2023-10-09T06:42:11.867 INFO:tasks.cephfs_test_runner: 2023-10-09T06:42:11.867 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-09T06:42:11.867 INFO:tasks.cephfs_test_runner:Ran 12 tests in 660.055s 2023-10-09T06:42:11.867 INFO:tasks.cephfs_test_runner: 2023-10-09T06:42:11.867 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-10-09T06:42:11.867 INFO:tasks.cephfs_test_runner: 2023-10-09T06:42:11.867 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-09T06:42:11.868 INFO:tasks.cephfs_test_runner:ERROR: test_volume_rm (tasks.cephfs.test_volumes.TestVolumes) 2023-10-09T06:42:11.868 INFO:tasks.cephfs_test_runner:That the volume can only be removed when --yes-i-really-mean-it is used 2023-10-09T06:42:11.868 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-09T06:42:11.868 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-09T06:42:11.868 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_vshankar_ceph_62a27f0d8453e2104f759ed08ba781e24c5ed056/qa/tasks/cephfs/test_volumes.py", line 497, in test_volume_rm 2023-10-09T06:42:11.868 INFO:tasks.cephfs_test_runner: self._fs_cmd("volume", "rm", self.volname) 2023-10-09T06:42:11.868 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_vshankar_ceph_62a27f0d8453e2104f759ed08ba781e24c5ed056/qa/tasks/cephfs/test_volumes.py", line 38, in _fs_cmd 2023-10-09T06:42:11.868 INFO:tasks.cephfs_test_runner: return self.mgr_cluster.mon_manager.raw_cluster_cmd("fs", *args) 2023-10-09T06:42:11.868 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_vshankar_ceph_62a27f0d8453e2104f759ed08ba781e24c5ed056/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd 2023-10-09T06:42:11.868 INFO:tasks.cephfs_test_runner: return self.run_cluster_cmd(**kwargs).stdout.getvalue() 2023-10-09T06:42:11.868 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_vshankar_ceph_62a27f0d8453e2104f759ed08ba781e24c5ed056/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd 2023-10-09T06:42:11.869 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-10-09T06:42:11.869 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run 2023-10-09T06:42:11.869 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-10-09T06:42:11.869 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run 2023-10-09T06:42:11.869 INFO:tasks.cephfs_test_runner: r.wait() 2023-10-09T06:42:11.869 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait 2023-10-09T06:42:11.869 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-09T06:42:11.869 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-09T06:42:11.869 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-09T06:42:11.869 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi163 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs' 2023-10-09T06:42:11.869 INFO:tasks.cephfs_test_runner: 2023-10-09T06:42:11.869 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2023-10-09T06:42:11.870 INFO:tasks.cephfs_test_runner: 2023-10-09T06:42:11.870 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-09T06:42:11.870 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_vshankar_ceph_62a27f0d8453e2104f759ed08ba781e24c5ed056/qa/tasks/cephfs/test_volumes.py", line 503, in test_volume_rm 2023-10-09T06:42:11.870 INFO:tasks.cephfs_test_runner: self._fs_cmd("volume", "rm", self.volname, "--yes-i-really-mean-it") 2023-10-09T06:42:11.870 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_vshankar_ceph_62a27f0d8453e2104f759ed08ba781e24c5ed056/qa/tasks/cephfs/test_volumes.py", line 38, in _fs_cmd 2023-10-09T06:42:11.870 INFO:tasks.cephfs_test_runner: return self.mgr_cluster.mon_manager.raw_cluster_cmd("fs", *args) 2023-10-09T06:42:11.870 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_vshankar_ceph_62a27f0d8453e2104f759ed08ba781e24c5ed056/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd 2023-10-09T06:42:11.870 INFO:tasks.cephfs_test_runner: return self.run_cluster_cmd(**kwargs).stdout.getvalue() 2023-10-09T06:42:11.870 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_vshankar_ceph_62a27f0d8453e2104f759ed08ba781e24c5ed056/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd 2023-10-09T06:42:11.870 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-10-09T06:42:11.870 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run 2023-10-09T06:42:11.870 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-10-09T06:42:11.871 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run 2023-10-09T06:42:11.871 INFO:tasks.cephfs_test_runner: r.wait() 2023-10-09T06:42:11.871 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait 2023-10-09T06:42:11.871 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-09T06:42:11.871 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-09T06:42:11.871 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-09T06:42:11.871 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi163 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it' 2023-10-09T06:42:11.871 INFO:tasks.cephfs_test_runner: 2023-10-09T06:42:11.871 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_vshankar_ceph_62a27f0d8453e2104f759ed08ba781e24c5ed056/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_volume_rm (tasks.cephfs.test_volumes.TestVolumes) 2023-10-09T06:42:11.947 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=e68320d3c7b5416b846fa78dd8c23c89 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_vshankar_ceph_62a27f0d8453e2104f759ed08ba781e24c5ed056/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_volume_rm (tasks.cephfs.test_volumes.TestVolumes) 2023-10-09T06:42:11.948 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-09T06:42:11.958 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-09T06:42:12.027 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-09T06:42:12.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:42:12.028 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2023-10-09T06:42:12.046 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-09T06:42:12.047 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-09T06:42:12.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:42:12.047 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-09T06:42:12.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:42:12.120 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-09T06:42:12.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:42:12.195 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link delete ceph-brx 2023-10-09T06:42:12.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:42:12.295 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2023-10-09T06:42:12.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:42:12.311 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link delete ceph-brx 2023-10-09T06:42:12.378 INFO:teuthology.orchestra.run.smithi163.stderr:Cannot find device "ceph-brx" 2023-10-09T06:42:12.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-09T06:42:12.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:42:12.380 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-09T06:42:12.398 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-09T06:42:12.399 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-09T06:42:12.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:42:12.400 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-09T06:42:12.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:42:12.478 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-09T06:42:12.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:42:12.556 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link delete ceph-brx 2023-10-09T06:42:12.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:42:12.648 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-09T06:42:12.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-09T06:42:12.664 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link delete ceph-brx 2023-10-09T06:42:12.733 INFO:teuthology.orchestra.run.smithi181.stderr:Cannot find device "ceph-brx" 2023-10-09T06:42:12.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-09T06:42:12.735 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-09T06:42:12.749 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-10-09T06:42:12.749 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:42:12.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.866+0000 7f867e21a700 1 Processor -- start 2023-10-09T06:42:12.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.866+0000 7f867e21a700 1 -- start start 2023-10-09T06:42:12.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.866+0000 7f867e21a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86780ccf90 0x7f86780ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:12.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.866+0000 7f867e21a700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86780cc300 con 0x7f86780cbf30 2023-10-09T06:42:12.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.866+0000 7f867e21a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86780ad370 con 0x7f86780b1420 2023-10-09T06:42:12.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.866+0000 7f867e21a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8678156bf0 con 0x7f86780ccf90 2023-10-09T06:42:12.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.867+0000 7f86777fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f86780cbf30 0x7f86780b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60580/0 (socket says 172.21.15.163:60580) 2023-10-09T06:42:12.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.867+0000 7f86777fe700 1 -- 172.21.15.163:0/1162036001 learned_addr learned my addr 172.21.15.163:0/1162036001 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:42:12.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.867+0000 7f8677fff700 1 --2- 172.21.15.163:0/1162036001 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86780ccf90 0x7f86780ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:12.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.867+0000 7f86767fc700 1 -- 172.21.15.163:0/1162036001 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2581661262 0 0) 0x7f86780cc300 con 0x7f86780cbf30 2023-10-09T06:42:12.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.867+0000 7f86767fc700 1 -- 172.21.15.163:0/1162036001 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8664003570 con 0x7f86780cbf30 2023-10-09T06:42:12.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.867+0000 7f86767fc700 1 -- 172.21.15.163:0/1162036001 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3727504438 0 0) 0x7f86780ad370 con 0x7f86780b1420 2023-10-09T06:42:12.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.867+0000 7f86767fc700 1 -- 172.21.15.163:0/1162036001 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86780cc300 con 0x7f86780b1420 2023-10-09T06:42:12.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.867+0000 7f8677fff700 1 -- 172.21.15.163:0/1162036001 >> v1:172.21.15.181:6789/0 conn(0x7f86780b1420 legacy=0x7f86780c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:12.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.867+0000 7f86767fc700 1 -- 172.21.15.163:0/1162036001 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 99382057 0 0) 0x7f8664003570 con 0x7f86780cbf30 2023-10-09T06:42:12.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.867+0000 7f8677fff700 1 -- 172.21.15.163:0/1162036001 >> v1:172.21.15.163:6789/0 conn(0x7f86780cbf30 legacy=0x7f86780b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:12.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.867+0000 7f8677fff700 1 -- 172.21.15.163:0/1162036001 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86780ad650 con 0x7f86780ccf90 2023-10-09T06:42:12.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.867+0000 7f8677fff700 1 --2- 172.21.15.163:0/1162036001 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86780ccf90 0x7f86780ae410 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8668009910 tx=0x7f86680091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1ae6a08dceb22 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:12.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.867+0000 7f86767fc700 1 -- 172.21.15.163:0/1162036001 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86700017f0 con 0x7f86780ccf90 2023-10-09T06:42:12.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.867+0000 7f86767fc700 1 -- 172.21.15.163:0/1162036001 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8668011420 con 0x7f86780ccf90 2023-10-09T06:42:12.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.867+0000 7f86767fc700 1 -- 172.21.15.163:0/1162036001 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86680117c0 con 0x7f86780ccf90 2023-10-09T06:42:12.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.868+0000 7f867e21a700 1 -- 172.21.15.163:0/1162036001 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86780ccf90 msgr2=0x7f86780ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:12.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.868+0000 7f867e21a700 1 --2- 172.21.15.163:0/1162036001 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86780ccf90 0x7f86780ae410 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8668009910 tx=0x7f86680091a0 comp rx=0 tx=0).stop 2023-10-09T06:42:12.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.868+0000 7f867e21a700 1 -- 172.21.15.163:0/1162036001 shutdown_connections 2023-10-09T06:42:12.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.868+0000 7f867e21a700 1 --2- 172.21.15.163:0/1162036001 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86780ccf90 0x7f86780ae410 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:12.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.868+0000 7f867e21a700 1 -- 172.21.15.163:0/1162036001 >> 172.21.15.163:0/1162036001 conn(0x7f867806a320 msgr2=0x7f86780cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:12.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.868+0000 7f867e21a700 1 -- 172.21.15.163:0/1162036001 shutdown_connections 2023-10-09T06:42:12.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.868+0000 7f867e21a700 1 -- 172.21.15.163:0/1162036001 wait complete. 2023-10-09T06:42:12.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.869+0000 7f867e21a700 1 Processor -- start 2023-10-09T06:42:12.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.869+0000 7f867e21a700 1 -- start start 2023-10-09T06:42:12.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.870+0000 7f867e21a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86780b1420 0x7f867806ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:12.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.870+0000 7f867e21a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86780cbf30 0x7f86780c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:12.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.870+0000 7f867e21a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86780ccf90 0x7f86781530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:12.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.870+0000 7f867e21a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8678169a80 con 0x7f86780cbf30 2023-10-09T06:42:12.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.870+0000 7f867e21a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86780c4850 con 0x7f86780ccf90 2023-10-09T06:42:12.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.870+0000 7f867e21a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8678169150 con 0x7f86780b1420 2023-10-09T06:42:12.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.870+0000 7f8676ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86780cbf30 0x7f86780c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:12.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.870+0000 7f8676ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86780cbf30 0x7f86780c3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:54986/0 (socket says 172.21.15.163:54986) 2023-10-09T06:42:12.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.870+0000 7f8676ffd700 1 -- 172.21.15.163:0/778287852 learned_addr learned my addr 172.21.15.163:0/778287852 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:12.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.870+0000 7f86777fe700 1 --2- 172.21.15.163:0/778287852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86780b1420 0x7f867806ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:12.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.870+0000 7f8677fff700 1 --2- 172.21.15.163:0/778287852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86780ccf90 0x7f86781530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:12.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.870+0000 7f8676ffd700 1 -- 172.21.15.163:0/778287852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86780b1420 msgr2=0x7f867806ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:12.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.870+0000 7f8676ffd700 1 --2- 172.21.15.163:0/778287852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86780b1420 0x7f867806ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:12.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.870+0000 7f8676ffd700 1 -- 172.21.15.163:0/778287852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86780ccf90 msgr2=0x7f86781530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:12.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.870+0000 7f8676ffd700 1 --2- 172.21.15.163:0/778287852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86780ccf90 0x7f86781530a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:12.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.870+0000 7f8676ffd700 1 -- 172.21.15.163:0/778287852 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8668009420 con 0x7f86780cbf30 2023-10-09T06:42:12.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.870+0000 7f86777fe700 1 --2- 172.21.15.163:0/778287852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86780b1420 0x7f867806ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:42:12.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.870+0000 7f8677fff700 1 --2- 172.21.15.163:0/778287852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86780ccf90 0x7f86781530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:42:12.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.871+0000 7f8676ffd700 1 --2- 172.21.15.163:0/778287852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86780cbf30 0x7f86780c3dd0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f866000c590 tx=0x7f8660007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:12.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.871+0000 7f866f7fe700 1 -- 172.21.15.163:0/778287852 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8660010510 con 0x7f86780cbf30 2023-10-09T06:42:12.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.871+0000 7f866f7fe700 1 -- 172.21.15.163:0/778287852 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8660010690 con 0x7f86780cbf30 2023-10-09T06:42:12.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.871+0000 7f867e21a700 1 -- 172.21.15.163:0/778287852 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8678169430 con 0x7f86780cbf30 2023-10-09T06:42:12.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.871+0000 7f866f7fe700 1 -- 172.21.15.163:0/778287852 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8660010a50 con 0x7f86780cbf30 2023-10-09T06:42:12.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.871+0000 7f867e21a700 1 -- 172.21.15.163:0/778287852 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8678169f10 con 0x7f86780cbf30 2023-10-09T06:42:12.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.872+0000 7f867e21a700 1 -- 172.21.15.163:0/778287852 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8678169c30 con 0x7f86780cbf30 2023-10-09T06:42:12.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.872+0000 7f866f7fe700 1 -- 172.21.15.163:0/778287852 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f866001b400 con 0x7f86780cbf30 2023-10-09T06:42:12.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.872+0000 7f866f7fe700 1 --2- 172.21.15.163:0/778287852 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8658067320 0x7f86580697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:12.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.873+0000 7f866f7fe700 1 -- 172.21.15.163:0/778287852 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f86600845e0 con 0x7f86780cbf30 2023-10-09T06:42:12.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.873+0000 7f86777fe700 1 --2- 172.21.15.163:0/778287852 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8658067320 0x7f86580697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:12.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.873+0000 7f86777fe700 1 --2- 172.21.15.163:0/778287852 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8658067320 0x7f86580697d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f8670001920 tx=0x7f8670009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:12.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:12.877+0000 7f866f7fe700 1 -- 172.21.15.163:0/778287852 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8678169c30 con 0x7f86780cbf30 2023-10-09T06:42:13.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.059+0000 7f867e21a700 1 -- 172.21.15.163:0/778287852 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f867804c3c0 con 0x7f86780cbf30 2023-10-09T06:42:13.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.060+0000 7f866f7fe700 1 -- 172.21.15.163:0/778287852 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v202) v1 ==== 74+0+12323 (secure 0 0 0) 0x7f8660051570 con 0x7f86780cbf30 2023-10-09T06:42:13.060 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:42:13.061 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":202,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:42:10.791998+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":198,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":198,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6835/2564044136":"2023-10-10T06:42:07.325638+0000","172.21.15.163:6833/2564044136":"2023-10-10T06:42:07.325638+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:42:13.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.062+0000 7f867e21a700 1 -- 172.21.15.163:0/778287852 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8658067320 msgr2=0x7f86580697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:13.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.062+0000 7f867e21a700 1 --2- 172.21.15.163:0/778287852 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8658067320 0x7f86580697d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f8670001920 tx=0x7f8670009040 comp rx=0 tx=0).stop 2023-10-09T06:42:13.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.062+0000 7f867e21a700 1 -- 172.21.15.163:0/778287852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86780cbf30 msgr2=0x7f86780c3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:13.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.062+0000 7f867e21a700 1 --2- 172.21.15.163:0/778287852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86780cbf30 0x7f86780c3dd0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f866000c590 tx=0x7f8660007db0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.062+0000 7f867e21a700 1 -- 172.21.15.163:0/778287852 shutdown_connections 2023-10-09T06:42:13.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.062+0000 7f867e21a700 1 --2- 172.21.15.163:0/778287852 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f8658067320 0x7f86580697d0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.062+0000 7f867e21a700 1 --2- 172.21.15.163:0/778287852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86780ccf90 0x7f86781530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.062+0000 7f867e21a700 1 --2- 172.21.15.163:0/778287852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86780b1420 0x7f867806ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.062+0000 7f867e21a700 1 --2- 172.21.15.163:0/778287852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f86780cbf30 0x7f86780c3dd0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.062+0000 7f867e21a700 1 -- 172.21.15.163:0/778287852 >> 172.21.15.163:0/778287852 conn(0x7f867806a320 msgr2=0x7f86780afc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:13.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.062+0000 7f867e21a700 1 -- 172.21.15.163:0/778287852 shutdown_connections 2023-10-09T06:42:13.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.062+0000 7f867e21a700 1 -- 172.21.15.163:0/778287852 wait complete. 2023-10-09T06:42:13.075 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-09T06:42:13.075 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:42:13.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.189+0000 7ff550481700 1 Processor -- start 2023-10-09T06:42:13.189 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.189+0000 7ff550481700 1 -- start start 2023-10-09T06:42:13.192 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.190+0000 7ff550481700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff548159890 0x7ff54815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:13.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.190+0000 7ff550481700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff54815a160 con 0x7ff548159c60 2023-10-09T06:42:13.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.190+0000 7ff550481700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5480d13a0 con 0x7ff548069560 2023-10-09T06:42:13.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.190+0000 7ff550481700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54815e140 con 0x7ff548159890 2023-10-09T06:42:13.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.190+0000 7ff54da1c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff548159c60 0x7ff54815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60592/0 (socket says 172.21.15.163:60592) 2023-10-09T06:42:13.193 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.190+0000 7ff54da1c700 1 -- 172.21.15.163:0/2676879634 learned_addr learned my addr 172.21.15.163:0/2676879634 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:42:13.194 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.191+0000 7ff54e21d700 1 --2- 172.21.15.163:0/2676879634 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff548159890 0x7ff54815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:13.194 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.191+0000 7ff54d21b700 1 -- 172.21.15.163:0/2676879634 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3815228498 0 0) 0x7ff54815a160 con 0x7ff548159c60 2023-10-09T06:42:13.194 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.191+0000 7ff54d21b700 1 -- 172.21.15.163:0/2676879634 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff530003570 con 0x7ff548159c60 2023-10-09T06:42:13.194 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.191+0000 7ff54d21b700 1 -- 172.21.15.163:0/2676879634 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3920283197 0 0) 0x7ff530003570 con 0x7ff548159c60 2023-10-09T06:42:13.194 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.191+0000 7ff54d21b700 1 -- 172.21.15.163:0/2676879634 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff548159890 msgr2=0x7ff54815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:13.194 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.191+0000 7ff54d21b700 1 --2- 172.21.15.163:0/2676879634 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff548159890 0x7ff54815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.194 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.191+0000 7ff54d21b700 1 -- 172.21.15.163:0/2676879634 >> v1:172.21.15.181:6789/0 conn(0x7ff548069560 legacy=0x7ff5480ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:13.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.191+0000 7ff54d21b700 1 -- 172.21.15.163:0/2676879634 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54815a3d0 con 0x7ff548159c60 2023-10-09T06:42:13.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.191+0000 7ff54d21b700 1 -- 172.21.15.163:0/2676879634 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7ff53c001470 con 0x7ff548159c60 2023-10-09T06:42:13.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.191+0000 7ff54d21b700 1 -- 172.21.15.163:0/2676879634 >> v1:172.21.15.163:6789/0 conn(0x7ff548159c60 legacy=0x7ff54815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:13.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.191+0000 7ff54d21b700 1 --2- 172.21.15.163:0/2676879634 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff548159890 0x7ff530003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:13.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.191+0000 7ff54d21b700 1 --2- 172.21.15.163:0/2676879634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff530003fd0 0x7ff530004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:13.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.191+0000 7ff54d21b700 1 --2- 172.21.15.163:0/2676879634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff530004d50 0x7ff5300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:13.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.191+0000 7ff54d21b700 1 -- 172.21.15.163:0/2676879634 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54815e140 con 0x7ff530003fd0 2023-10-09T06:42:13.195 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.191+0000 7ff54d21b700 1 -- 172.21.15.163:0/2676879634 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53000bb40 con 0x7ff530004d50 2023-10-09T06:42:13.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.191+0000 7ff54d21b700 1 -- 172.21.15.163:0/2676879634 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff53000bd00 con 0x7ff548159890 2023-10-09T06:42:13.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.191+0000 7ff54da1c700 1 --2- 172.21.15.163:0/2676879634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff530003fd0 0x7ff530004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:13.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.191+0000 7ff54e21d700 1 --2- 172.21.15.163:0/2676879634 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff548159890 0x7ff530003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:13.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.191+0000 7ff54ea1e700 1 --2- 172.21.15.163:0/2676879634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff530004d50 0x7ff5300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:13.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.191+0000 7ff54da1c700 1 -- 172.21.15.163:0/2676879634 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff548159890 msgr2=0x7ff530003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:13.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.191+0000 7ff54da1c700 1 --2- 172.21.15.163:0/2676879634 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff548159890 0x7ff530003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.191+0000 7ff54da1c700 1 -- 172.21.15.163:0/2676879634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff530004d50 msgr2=0x7ff5300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:13.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.191+0000 7ff54da1c700 1 --2- 172.21.15.163:0/2676879634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff530004d50 0x7ff5300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.191+0000 7ff54da1c700 1 -- 172.21.15.163:0/2676879634 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff530003570 con 0x7ff530003fd0 2023-10-09T06:42:13.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.192+0000 7ff54da1c700 1 --2- 172.21.15.163:0/2676879634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff530003fd0 0x7ff530004810 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00c390 tx=0x7ff53c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1cda67581a2f1a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:13.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.192+0000 7ff54d21b700 1 -- 172.21.15.163:0/2676879634 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c00cc60 con 0x7ff530003fd0 2023-10-09T06:42:13.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.192+0000 7ff54d21b700 1 -- 172.21.15.163:0/2676879634 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53c00cdc0 con 0x7ff530003fd0 2023-10-09T06:42:13.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.192+0000 7ff54d21b700 1 -- 172.21.15.163:0/2676879634 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c005b70 con 0x7ff530003fd0 2023-10-09T06:42:13.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.192+0000 7ff550481700 1 -- 172.21.15.163:0/2676879634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff530003fd0 msgr2=0x7ff530004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:13.197 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.192+0000 7ff550481700 1 --2- 172.21.15.163:0/2676879634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff530003fd0 0x7ff530004810 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00c390 tx=0x7ff53c001e70 comp rx=0 tx=0).stop 2023-10-09T06:42:13.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.192+0000 7ff54ea1e700 1 -- 172.21.15.163:0/2676879634 reap_dead start 2023-10-09T06:42:13.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.192+0000 7ff550481700 1 -- 172.21.15.163:0/2676879634 shutdown_connections 2023-10-09T06:42:13.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.192+0000 7ff550481700 1 -- 172.21.15.163:0/2676879634 >> 172.21.15.163:0/2676879634 conn(0x7ff548068e20 msgr2=0x7ff5480d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:13.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.192+0000 7ff550481700 1 -- 172.21.15.163:0/2676879634 shutdown_connections 2023-10-09T06:42:13.198 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.192+0000 7ff550481700 1 -- 172.21.15.163:0/2676879634 wait complete. 2023-10-09T06:42:13.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.193+0000 7ff550481700 1 Processor -- start 2023-10-09T06:42:13.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.194+0000 7ff550481700 1 -- start start 2023-10-09T06:42:13.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.194+0000 7ff550481700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff530003fd0 0x7ff5481585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:13.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.194+0000 7ff550481700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5480b5220 0x7ff548151d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:13.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.194+0000 7ff550481700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5480cecc0 0x7ff5480cf110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:13.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.194+0000 7ff550481700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5480cf650 con 0x7ff5480b5220 2023-10-09T06:42:13.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.194+0000 7ff550481700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5481522b0 con 0x7ff5480cecc0 2023-10-09T06:42:13.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.194+0000 7ff550481700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5480b5690 con 0x7ff530003fd0 2023-10-09T06:42:13.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.194+0000 7ff54da1c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5480b5220 0x7ff548151d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:13.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.194+0000 7ff54da1c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5480b5220 0x7ff548151d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55002/0 (socket says 172.21.15.163:55002) 2023-10-09T06:42:13.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.194+0000 7ff54da1c700 1 -- 172.21.15.163:0/3800701179 learned_addr learned my addr 172.21.15.163:0/3800701179 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:13.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.194+0000 7ff54e21d700 1 --2- 172.21.15.163:0/3800701179 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff530003fd0 0x7ff5481585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:13.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.194+0000 7ff54ea1e700 1 --2- 172.21.15.163:0/3800701179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5480cecc0 0x7ff5480cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:13.202 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.195+0000 7ff54da1c700 1 -- 172.21.15.163:0/3800701179 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff530003fd0 msgr2=0x7ff5481585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:13.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.195+0000 7ff54da1c700 1 --2- 172.21.15.163:0/3800701179 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff530003fd0 0x7ff5481585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.195+0000 7ff54da1c700 1 -- 172.21.15.163:0/3800701179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5480cecc0 msgr2=0x7ff5480cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:13.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.195+0000 7ff54da1c700 1 --2- 172.21.15.163:0/3800701179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5480cecc0 0x7ff5480cf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.195+0000 7ff54da1c700 1 -- 172.21.15.163:0/3800701179 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5480b5950 con 0x7ff5480b5220 2023-10-09T06:42:13.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.195+0000 7ff54ea1e700 1 --2- 172.21.15.163:0/3800701179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5480cecc0 0x7ff5480cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:42:13.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.195+0000 7ff54da1c700 1 --2- 172.21.15.163:0/3800701179 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5480b5220 0x7ff548151d70 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7ff53000bb10 tx=0x7ff53c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:13.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.195+0000 7ff5467fc700 1 -- 172.21.15.163:0/3800701179 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c00cc60 con 0x7ff5480b5220 2023-10-09T06:42:13.203 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.195+0000 7ff5467fc700 1 -- 172.21.15.163:0/3800701179 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53c000fe0 con 0x7ff5480b5220 2023-10-09T06:42:13.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.195+0000 7ff550481700 1 -- 172.21.15.163:0/3800701179 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5480cf650 con 0x7ff5480b5220 2023-10-09T06:42:13.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.195+0000 7ff5467fc700 1 -- 172.21.15.163:0/3800701179 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c007530 con 0x7ff5480b5220 2023-10-09T06:42:13.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.195+0000 7ff550481700 1 -- 172.21.15.163:0/3800701179 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5480b5f10 con 0x7ff5480b5220 2023-10-09T06:42:13.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.196+0000 7ff550481700 1 -- 172.21.15.163:0/3800701179 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5481560d0 con 0x7ff5480b5220 2023-10-09T06:42:13.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.196+0000 7ff5467fc700 1 -- 172.21.15.163:0/3800701179 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff53c019030 con 0x7ff5480b5220 2023-10-09T06:42:13.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.197+0000 7ff5467fc700 1 --2- 172.21.15.163:0/3800701179 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff534067320 0x7ff5340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:13.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.197+0000 7ff5467fc700 1 -- 172.21.15.163:0/3800701179 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff53c0858e0 con 0x7ff5480b5220 2023-10-09T06:42:13.204 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.197+0000 7ff54e21d700 1 --2- 172.21.15.163:0/3800701179 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff534067320 0x7ff5340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:13.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.197+0000 7ff54e21d700 1 --2- 172.21.15.163:0/3800701179 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff534067320 0x7ff5340697d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff5400094d0 tx=0x7ff54000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:13.205 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.201+0000 7ff5467fc700 1 -- 172.21.15.163:0/3800701179 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff53c052870 con 0x7ff5480b5220 2023-10-09T06:42:13.378 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.378+0000 7ff550481700 1 -- 172.21.15.163:0/3800701179 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff54804c3c0 con 0x7ff534067320 2023-10-09T06:42:13.381 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.381+0000 7ff5467fc700 1 -- 172.21.15.163:0/3800701179 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51015 (secure 0 0 0) 0x7ff54804c3c0 con 0x7ff534067320 2023-10-09T06:42:13.382 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:42:13.384 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.383+0000 7ff550481700 1 -- 172.21.15.163:0/3800701179 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff534067320 msgr2=0x7ff5340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:13.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.383+0000 7ff550481700 1 --2- 172.21.15.163:0/3800701179 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff534067320 0x7ff5340697d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff5400094d0 tx=0x7ff54000b040 comp rx=0 tx=0).stop 2023-10-09T06:42:13.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.383+0000 7ff550481700 1 -- 172.21.15.163:0/3800701179 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5480b5220 msgr2=0x7ff548151d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:13.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.383+0000 7ff550481700 1 --2- 172.21.15.163:0/3800701179 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5480b5220 0x7ff548151d70 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7ff53000bb10 tx=0x7ff53c015000 comp rx=0 tx=0).stop 2023-10-09T06:42:13.385 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.383+0000 7ff550481700 1 -- 172.21.15.163:0/3800701179 shutdown_connections 2023-10-09T06:42:13.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.383+0000 7ff550481700 1 --2- 172.21.15.163:0/3800701179 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff534067320 0x7ff5340697d0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.383+0000 7ff550481700 1 --2- 172.21.15.163:0/3800701179 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5480cecc0 0x7ff5480cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.383+0000 7ff550481700 1 --2- 172.21.15.163:0/3800701179 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff530003fd0 0x7ff5481585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.383+0000 7ff550481700 1 --2- 172.21.15.163:0/3800701179 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5480b5220 0x7ff548151d70 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.383+0000 7ff550481700 1 -- 172.21.15.163:0/3800701179 >> 172.21.15.163:0/3800701179 conn(0x7ff548068e20 msgr2=0x7ff548152b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:13.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.383+0000 7ff550481700 1 -- 172.21.15.163:0/3800701179 shutdown_connections 2023-10-09T06:42:13.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.383+0000 7ff550481700 1 -- 172.21.15.163:0/3800701179 wait complete. 2023-10-09T06:42:13.387 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:42:13.395 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":546,"stamp":"2023-10-09T06:42:12.468249+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2432236,"kb_used_data":21164,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752542484,"statfs":{"total":773094113280,"available":770603503616,"internally_reserved":0,"allocated":21671936,"data_stored":10378971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4067,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-49,"num_write_kb":-18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001511"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":312,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-09T06:42:08.646970+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:42:08.646970+0000","last_peered":"2023-10-09T06:42:08.646970+0000","last_clean":"2023-10-09T06:42:08.646970+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:42:08.646970+0000","last_undegraded":"2023-10-09T06:42:08.646970+0000","last_fullsized":"2023-10-09T06:42:08.646970+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574992,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304464,"kb_used_data":3080,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067376,"statfs":{"total":96636764160,"available":96324993024,"internally_reserved":0,"allocated":3153920,"data_stored":1740423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58199999999999996,"15min":0.53100000000000003},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.70599999999999996,"5min":3.778,"15min":3.778},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54300000000000004,"15min":0.52200000000000002},"min":{"1min":0.38400000000000001,"5min":0.33400000000000002,"15min":0.29699999999999999},"max":{"1min":0.77100000000000002,"5min":1.768,"15min":1.768},"last":0.50900000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53400000000000003,"15min":0.52200000000000002},"min":{"1min":0.32100000000000001,"5min":0.31900000000000001,"15min":0.27100000000000002},"max":{"1min":0.72999999999999998,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.38},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.53700000000000003},"min":{"1min":0.435,"5min":0.378,"15min":0.29699999999999999},"max":{"1min":0.68100000000000005,"5min":1.806,"15min":1.806},"last":0.40300000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59099999999999997,"15min":0.54100000000000004},"min":{"1min":0.42599999999999999,"5min":0.36399999999999999,"15min":0.32200000000000001},"max":{"1min":0.76000000000000001,"5min":3.7210000000000001,"15min":3.7210000000000001},"last":0.432},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56699999999999995,"15min":0.54900000000000004},"min":{"1min":0.44600000000000001,"5min":0.35399999999999998,"15min":0.315},"max":{"1min":0.69799999999999995,"5min":1.51,"15min":1.51},"last":0.47899999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60799999999999998,"15min":0.56699999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.33500000000000002},"max":{"1min":0.73299999999999998,"5min":3.8090000000000002,"15min":3.8090000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.47499999999999998,"5min":0.38400000000000001,"15min":0.36499999999999999},"max":{"1min":0.74299999999999999,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.442}]},{"osd":4,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.505,"15min":0.50600000000000001},"min":{"1min":0.31,"5min":0.25600000000000001,"15min":0.24399999999999999},"max":{"1min":0.55100000000000005,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.628,"15min":0.60599999999999998},"min":{"1min":0.35599999999999998,"5min":0.35199999999999998,"15min":0.30299999999999999},"max":{"1min":0.751,"5min":4.8239999999999998,"15min":4.8239999999999998},"last":0.53300000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53800000000000003,"15min":0.55100000000000005},"min":{"1min":0.34599999999999997,"5min":0.313,"15min":0.307},"max":{"1min":0.64800000000000002,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.63200000000000001,"15min":0.61799999999999999},"min":{"1min":0.38900000000000001,"5min":0.34300000000000003,"15min":0.34200000000000003},"max":{"1min":0.71699999999999997,"5min":3.6909999999999998,"15min":3.6909999999999998},"last":0.61599999999999999}]},{"osd":6,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.59499999999999997,"15min":0.58699999999999997},"min":{"1min":0.39100000000000001,"5min":0.318,"15min":0.28699999999999998},"max":{"1min":0.66600000000000004,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.63400000000000001,"15min":0.60999999999999999},"min":{"1min":0.375,"5min":0.33000000000000002,"15min":0.315},"max":{"1min":0.72499999999999998,"5min":4.7789999999999999,"15min":4.7789999999999999},"last":0.56999999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574991,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303884,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067956,"statfs":{"total":96636764160,"available":96325586944,"internally_reserved":0,"allocated":2560000,"data_stored":1150055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54200000000000004,"15min":0.51600000000000001},"min":{"1min":0.40600000000000003,"5min":0.35499999999999998,"15min":0.30399999999999999},"max":{"1min":0.75700000000000001,"5min":0.80500000000000005,"15min":0.82299999999999995},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55200000000000005,"15min":0.52300000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.79000000000000004,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.41099999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52900000000000003,"15min":0.51600000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.73499999999999999,"5min":0.74199999999999999,"15min":1.1359999999999999},"last":0.433},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.56899999999999995,"15min":0.54200000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.28799999999999998},"max":{"1min":0.754,"5min":0.78700000000000003,"15min":1.1539999999999999},"last":0.35799999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.55900000000000005,"15min":0.53000000000000003},"min":{"1min":0.45400000000000001,"5min":0.34300000000000003,"15min":0.29899999999999999},"max":{"1min":0.74199999999999999,"5min":0.77000000000000002,"15min":0.78400000000000003},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54400000000000004,"15min":0.53200000000000003},"min":{"1min":0.34899999999999998,"5min":0.34499999999999997,"15min":0.313},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.48299999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56499999999999995,"15min":0.55000000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.75700000000000001,"5min":0.79000000000000004,"15min":0.86799999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.58799999999999997,"15min":0.63700000000000001},"min":{"1min":0.49299999999999999,"5min":0.371,"15min":0.34300000000000003},"max":{"1min":0.71699999999999997,"5min":0.76900000000000002,"15min":4.7510000000000003},"last":0.51500000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47099999999999997,"15min":0.47399999999999998},"min":{"1min":0.30299999999999999,"5min":0.28999999999999998,"15min":0.27900000000000003},"max":{"1min":0.69699999999999995,"5min":0.74299999999999999,"15min":0.76400000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59099999999999997,"15min":0.58299999999999996},"min":{"1min":0.435,"5min":0.38700000000000001,"15min":0.25800000000000001},"max":{"1min":0.77700000000000002,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.53000000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.36399999999999999,"5min":0.31900000000000001,"15min":0.246},"max":{"1min":0.75,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.41999999999999998,"5min":0.40999999999999998,"15min":0.35699999999999998},"max":{"1min":0.73399999999999999,"5min":0.76200000000000001,"15min":0.86599999999999999},"last":0.625}]},{"osd":7,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56000000000000005,"15min":0.57199999999999995},"min":{"1min":0.39100000000000001,"5min":0.33400000000000002,"15min":0.29899999999999999},"max":{"1min":0.74199999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.499},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59599999999999997,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.38400000000000001,"15min":0.33000000000000002},"max":{"1min":0.72099999999999997,"5min":0.84399999999999997,"15min":0.88300000000000001},"last":0.57299999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574994,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303884,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067956,"statfs":{"total":96636764160,"available":96325586944,"internally_reserved":0,"allocated":2560000,"data_stored":1150055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.32800000000000001,"15min":0.316},"min":{"1min":0.26900000000000002,"5min":0.192,"15min":0.192},"max":{"1min":0.50900000000000001,"5min":0.55600000000000005,"15min":0.80700000000000005},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.432,"5min":0.38100000000000001,"15min":0.36399999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.21199999999999999},"max":{"1min":0.54600000000000004,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.47299999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.36099999999999999,"15min":0.35399999999999998},"min":{"1min":0.317,"5min":0.215,"15min":0.20399999999999999},"max":{"1min":0.53100000000000003,"5min":0.53700000000000003,"15min":0.68300000000000005},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.40899999999999997,"15min":0.39300000000000002},"min":{"1min":0.36799999999999999,"5min":0.25800000000000001,"15min":0.251},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.438}]},{"osd":3,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.39800000000000002,"15min":0.376},"min":{"1min":0.32600000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.375,"15min":0.46200000000000002},"min":{"1min":0.28599999999999998,"5min":0.23400000000000001,"15min":0.22700000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":5.8529999999999998},"last":0.39800000000000002}]},{"osd":4,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.59999999999999998,"15min":0.58499999999999996},"min":{"1min":0.45300000000000001,"5min":0.36199999999999999,"15min":0.32200000000000001},"max":{"1min":0.79800000000000004,"5min":1.01,"15min":1.01},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61799999999999999,"15min":0.59799999999999998},"min":{"1min":0.47299999999999998,"5min":0.40000000000000002,"15min":0.37},"max":{"1min":0.79300000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.67200000000000004}]},{"osd":5,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.59799999999999998,"15min":0.67500000000000004},"min":{"1min":0.40899999999999997,"5min":0.36499999999999999,"15min":0.34599999999999997},"max":{"1min":0.85599999999999998,"5min":0.92500000000000004,"15min":5.9459999999999997},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.621,"15min":0.61399999999999999},"min":{"1min":0.55700000000000005,"5min":0.38100000000000001,"15min":0.34300000000000003},"max":{"1min":0.82599999999999996,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.60599999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.57799999999999996,"15min":0.65000000000000002},"min":{"1min":0.44600000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":5.9160000000000004},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.51700000000000002,"5min":0.379,"15min":0.35699999999999998},"max":{"1min":0.86699999999999999,"5min":0.88600000000000001,"15min":0.91400000000000003},"last":0.68500000000000005}]},{"osd":7,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.51400000000000001,"5min":0.39000000000000001,"15min":0.379},"max":{"1min":0.76800000000000002,"5min":0.91000000000000003,"15min":0.97099999999999997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.59099999999999997,"15min":0.67600000000000005},"min":{"1min":0.48399999999999999,"5min":0.41399999999999998,"15min":0.38100000000000001},"max":{"1min":0.83399999999999996,"5min":0.86099999999999999,"15min":5.9729999999999999},"last":0.625}]}]},{"osd":0,"up_from":13,"seq":55834574993,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304464,"kb_used_data":3080,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067376,"statfs":{"total":96636764160,"available":96324993024,"internally_reserved":0,"allocated":3153920,"data_stored":1740423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.34000000000000002,"15min":0.32200000000000001},"min":{"1min":0.252,"5min":0.216,"15min":0.21299999999999999},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.35799999999999998,"15min":0.34699999999999998},"min":{"1min":0.24399999999999999,"5min":0.222,"15min":0.214},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.94499999999999995},"last":0.315}]},{"osd":2,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.38300000000000001,"15min":0.36799999999999999},"min":{"1min":0.27100000000000002,"5min":0.22,"15min":0.20200000000000001},"max":{"1min":0.53000000000000003,"5min":0.54000000000000004,"15min":0.75600000000000001},"last":0.247},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.41399999999999998,"15min":0.39500000000000002},"min":{"1min":0.34599999999999997,"5min":0.29099999999999998,"15min":0.28000000000000003},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.91700000000000004},"last":0.35199999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.36499999999999999,"15min":0.372},"min":{"1min":0.318,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":1.554},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.40200000000000002,"15min":0.39500000000000002},"min":{"1min":0.32400000000000001,"5min":0.27700000000000002,"15min":0.25800000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.79000000000000004},"last":0.41899999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58699999999999997,"15min":0.56299999999999994},"min":{"1min":0.46300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85699999999999998,"5min":0.92600000000000005,"15min":1.212},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58699999999999997,"15min":0.65700000000000003},"min":{"1min":0.43099999999999999,"5min":0.35799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85599999999999998,"5min":0.90800000000000003,"15min":5.484},"last":0.59199999999999997}]},{"osd":5,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58499999999999996,"15min":0.65900000000000003},"min":{"1min":0.437,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83099999999999996,"5min":0.97399999999999998,"15min":5.5449999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.47499999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.875,"5min":0.998,"15min":1.113},"last":0.63800000000000001}]},{"osd":6,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.42399999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86899999999999999,"5min":0.871,"15min":1.0529999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.627,"15min":0.61399999999999999},"min":{"1min":0.505,"5min":0.42899999999999999,"15min":0.41599999999999998},"max":{"1min":0.88400000000000001,"5min":0.96099999999999997,"15min":1.169},"last":0.61899999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60399999999999998,"15min":0.59399999999999997},"min":{"1min":0.47299999999999998,"5min":0.40200000000000002,"15min":0.371},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":1.2390000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59899999999999998,"15min":0.67800000000000005},"min":{"1min":0.48399999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83199999999999996,"5min":0.94299999999999995,"15min":5.5800000000000001},"last":0.55300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574994,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303888,"kb_used_data":2504,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067952,"statfs":{"total":96636764160,"available":96325582848,"internally_reserved":0,"allocated":2564096,"data_stored":1147850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33400000000000002,"15min":0.34000000000000002},"min":{"1min":0.252,"5min":0.20000000000000001,"15min":0.19900000000000001},"max":{"1min":0.51800000000000002,"5min":0.66000000000000003,"15min":1.4199999999999999},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.34499999999999997,"15min":0.46100000000000002},"min":{"1min":0.28000000000000003,"5min":0.221,"15min":0.19900000000000001},"max":{"1min":0.53100000000000003,"5min":0.55600000000000005,"15min":7.2699999999999996},"last":0.22900000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.34200000000000003,"15min":0.34100000000000003},"min":{"1min":0.26200000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.497,"5min":0.56399999999999995,"15min":0.65400000000000003},"last":0.21199999999999999},{"interface":"front","average":{"1min":0.434,"5min":0.39600000000000002,"15min":0.51400000000000001},"min":{"1min":0.33500000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.56799999999999995,"5min":0.60399999999999998,"15min":7.327},"last":0.27800000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.41099999999999998,"15min":0.39400000000000002},"min":{"1min":0.39600000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.56899999999999995,"5min":0.64600000000000002,"15min":0.68600000000000005},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.39000000000000001,"15min":0.38700000000000001},"min":{"1min":0.32600000000000001,"5min":0.252,"15min":0.23999999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.72699999999999998},"last":0.311}]},{"osd":4,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56299999999999994,"15min":0.54800000000000004},"min":{"1min":0.45800000000000002,"5min":0.33600000000000002,"15min":0.317},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.58199999999999996,"15min":0.68999999999999995},"min":{"1min":0.46899999999999997,"5min":0.35999999999999999,"15min":0.33600000000000002},"max":{"1min":0.82599999999999996,"5min":0.91100000000000003,"15min":7.2389999999999999},"last":0.54600000000000004}]},{"osd":5,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59499999999999997,"15min":0.77300000000000002},"min":{"1min":0.495,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.872,"5min":0.89800000000000002,"15min":10.890000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.60899999999999999,"15min":0.60299999999999998},"min":{"1min":0.45600000000000002,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":0.80000000000000004,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.61699999999999999}]},{"osd":6,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58099999999999996,"15min":0.69599999999999995},"min":{"1min":0.50800000000000001,"5min":0.35999999999999999,"15min":0.33500000000000002},"max":{"1min":0.77600000000000002,"5min":0.82499999999999996,"15min":7.2999999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60499999999999998,"15min":0.58299999999999996},"min":{"1min":0.44700000000000001,"5min":0.40300000000000002,"15min":0.377},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.749}]},{"osd":7,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.61099999999999999,"15min":0.72599999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.37},"max":{"1min":0.80100000000000005,"5min":0.91300000000000003,"15min":7.1879999999999997},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63,"15min":0.624},"min":{"1min":0.53300000000000003,"5min":0.437,"15min":0.38},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.70199999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574993,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303884,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067956,"statfs":{"total":96636764160,"available":96325586944,"internally_reserved":0,"allocated":2560000,"data_stored":1150055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.34200000000000003,"15min":0.33800000000000002},"min":{"1min":0.23799999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.55400000000000005,"5min":0.61699999999999999,"15min":5.8700000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.36299999999999999,"15min":0.36799999999999999},"min":{"1min":0.255,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.55400000000000005,"5min":0.58499999999999996,"15min":5.6769999999999996},"last":0.28799999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.36599999999999999,"15min":0.36399999999999999},"min":{"1min":0.28599999999999998,"5min":0.21299999999999999,"15min":0.21199999999999999},"max":{"1min":0.46700000000000003,"5min":0.61199999999999999,"15min":4.0469999999999997},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.40200000000000002,"15min":0.39500000000000002},"min":{"1min":0.32900000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.60199999999999998,"5min":0.68600000000000005,"15min":4.1509999999999998},"last":0.31900000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.371,"15min":0.373},"min":{"1min":0.27300000000000002,"5min":0.23200000000000001,"15min":0.222},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":3.1139999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.39500000000000002,"15min":0.38900000000000001},"min":{"1min":0.28799999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.54700000000000004,"5min":0.66900000000000004,"15min":3.3180000000000001},"last":0.34699999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56999999999999995,"15min":0.57299999999999995},"min":{"1min":0.42799999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.75700000000000001,"5min":0.86699999999999999,"15min":3.0899999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.432,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79300000000000004,"5min":0.84499999999999997,"15min":3.0710000000000002},"last":0.60499999999999998}]},{"osd":5,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.58099999999999996,"15min":0.59099999999999997},"min":{"1min":0.45300000000000001,"5min":0.33600000000000002,"15min":0.32700000000000001},"max":{"1min":0.86399999999999999,"5min":0.877,"15min":6.2300000000000004},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.58699999999999997,"15min":0.59699999999999998},"min":{"1min":0.498,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78800000000000003,"5min":0.83599999999999997,"15min":3.0219999999999998},"last":0.59499999999999997}]},{"osd":6,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.61899999999999999,"15min":0.60499999999999998},"min":{"1min":0.47899999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.147,"5min":1.147,"15min":2.6850000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.65000000000000002,"15min":0.65700000000000003},"min":{"1min":0.5,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":6.5970000000000004},"last":0.68999999999999995}]},{"osd":7,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.61199999999999999,"15min":0.61699999999999999},"min":{"1min":0.46600000000000003,"5min":0.41499999999999998,"15min":0.38200000000000001},"max":{"1min":0.81799999999999995,"5min":0.86099999999999999,"15min":4.2629999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60799999999999998,"15min":0.61899999999999999},"min":{"1min":0.47899999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.89200000000000002,"5min":0.89900000000000002,"15min":7.3650000000000002},"last":0.57499999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574992,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303884,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067956,"statfs":{"total":96636764160,"available":96325586944,"internally_reserved":0,"allocated":2560000,"data_stored":1150055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54800000000000004,"15min":0.54500000000000004},"min":{"1min":0.38900000000000001,"5min":0.307,"15min":0.27000000000000002},"max":{"1min":0.71899999999999997,"5min":0.88900000000000001,"15min":1.083},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.55200000000000005,"15min":0.53000000000000003},"min":{"1min":0.50800000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":1.054},"last":0.58599999999999997}]},{"osd":1,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56599999999999995,"15min":0.55100000000000005},"min":{"1min":0.52400000000000002,"5min":0.28399999999999997,"15min":0.27100000000000002},"max":{"1min":0.77300000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53100000000000003,"15min":0.50800000000000001},"min":{"1min":0.38500000000000001,"5min":0.33800000000000002,"15min":0.31},"max":{"1min":0.75,"5min":0.76300000000000001,"15min":0.79100000000000004},"last":0.78500000000000003}]},{"osd":2,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.54700000000000004,"15min":0.54000000000000004},"min":{"1min":0.443,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76500000000000001,"5min":0.84899999999999998,"15min":1.0249999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.499,"5min":0.38100000000000001,"15min":0.36299999999999999},"max":{"1min":0.73399999999999999,"5min":0.90500000000000003,"15min":0.93799999999999994},"last":0.748}]},{"osd":3,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55300000000000005,"15min":0.54800000000000004},"min":{"1min":0.40699999999999997,"5min":0.33300000000000002,"15min":0.29199999999999998},"max":{"1min":0.747,"5min":0.86899999999999999,"15min":0.98499999999999999},"last":0.62},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.47799999999999998,"5min":0.38700000000000001,"15min":0.34100000000000003},"max":{"1min":0.68200000000000005,"5min":0.92900000000000005,"15min":1.1319999999999999},"last":0.68300000000000005}]},{"osd":5,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.50600000000000001,"15min":0.53300000000000003},"min":{"1min":0.35499999999999998,"5min":0.307,"15min":0.27300000000000002},"max":{"1min":0.55900000000000005,"5min":0.83299999999999996,"15min":0.998},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57699999999999996,"15min":0.59399999999999997},"min":{"1min":0.40000000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71999999999999997,"5min":0.95599999999999996,"15min":0.97999999999999998},"last":0.50900000000000001}]},{"osd":6,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.54600000000000004,"15min":0.55600000000000005},"min":{"1min":0.36899999999999999,"5min":0.35499999999999998,"15min":0.29999999999999999},"max":{"1min":0.67500000000000004,"5min":0.86299999999999999,"15min":1.0189999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.56999999999999995,"15min":0.58399999999999996},"min":{"1min":0.38900000000000001,"5min":0.36399999999999999,"15min":0.32700000000000001},"max":{"1min":0.72699999999999998,"5min":0.85599999999999998,"15min":1.01},"last":0.79900000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55700000000000005,"15min":0.56200000000000006},"min":{"1min":0.42399999999999999,"5min":0.35299999999999998,"15min":0.28299999999999997},"max":{"1min":0.65400000000000003,"5min":0.84399999999999997,"15min":0.90000000000000002},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.60599999999999998,"15min":0.625},"min":{"1min":0.374,"5min":0.34999999999999998,"15min":0.32200000000000001},"max":{"1min":0.71299999999999997,"5min":0.93799999999999994,"15min":1.175},"last":0.63600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574992,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303884,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067956,"statfs":{"total":96636764160,"available":96325586944,"internally_reserved":0,"allocated":2560000,"data_stored":1150055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54200000000000004,"15min":0.51700000000000002},"min":{"1min":0.36399999999999999,"5min":0.29499999999999998,"15min":0.28599999999999998},"max":{"1min":0.76400000000000001,"5min":0.79700000000000004,"15min":0.86599999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55400000000000005,"15min":0.52800000000000002},"min":{"1min":0.45000000000000001,"5min":0.36399999999999999,"15min":0.28799999999999998},"max":{"1min":0.83599999999999997,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.63100000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52700000000000002,"15min":0.51500000000000001},"min":{"1min":0.38800000000000001,"5min":0.34100000000000003,"15min":0.311},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.54400000000000004,"15min":0.53400000000000003},"min":{"1min":0.41499999999999998,"5min":0.36899999999999999,"15min":0.30199999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.81599999999999995},"last":0.40899999999999997}]},{"osd":2,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58199999999999996,"15min":0.55800000000000005},"min":{"1min":0.39300000000000002,"5min":0.34699999999999998,"15min":0.28199999999999997},"max":{"1min":0.79800000000000004,"5min":1.139,"15min":1.139},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.52600000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78100000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.56899999999999995}]},{"osd":3,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57099999999999995,"15min":0.55200000000000005},"min":{"1min":0.43099999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.75,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.45900000000000002,"5min":0.40699999999999997,"15min":0.312},"max":{"1min":0.80200000000000005,"5min":0.80600000000000005,"15min":0.81100000000000005},"last":0.48099999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.33600000000000002,"5min":0.28399999999999997,"15min":0.26700000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53600000000000003,"15min":0.54200000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.27700000000000002},"max":{"1min":0.751,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.501}]},{"osd":6,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.35599999999999998,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":0.73499999999999999,"5min":1.752,"15min":1.752},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55600000000000005,"15min":0.55900000000000005},"min":{"1min":0.39100000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.377}]},{"osd":7,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56399999999999995,"15min":0.57599999999999996},"min":{"1min":0.375,"5min":0.311,"15min":0.311},"max":{"1min":0.79900000000000004,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58599999999999997,"15min":0.60299999999999998},"min":{"1min":0.40799999999999997,"5min":0.372,"15min":0.314},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.90600000000000003},"last":0.52000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:42:13.400 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:42:13.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.514+0000 7fda78dd7700 1 Processor -- start 2023-10-09T06:42:13.515 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.514+0000 7fda78dd7700 1 -- start start 2023-10-09T06:42:13.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.515+0000 7fda78dd7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda740aca80 0x7fda7406aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:13.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.515+0000 7fda78dd7700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fda740c83c0 0x7fda740cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:13.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.515+0000 7fda78dd7700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda7406bdb0 con 0x7fda74069d60 2023-10-09T06:42:13.521 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.515+0000 7fda78dd7700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda740cb370 con 0x7fda740c83c0 2023-10-09T06:42:13.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.515+0000 7fda78dd7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda74159f60 con 0x7fda740aca80 2023-10-09T06:42:13.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.515+0000 7fda72d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fda740c83c0 0x7fda740cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:13.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.515+0000 7fda72d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fda740c83c0 0x7fda740cb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55008/0 (socket says 172.21.15.163:55008) 2023-10-09T06:42:13.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.515+0000 7fda72d9d700 1 -- 172.21.15.163:0/2513105477 learned_addr learned my addr 172.21.15.163:0/2513105477 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:13.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.515+0000 7fda7259c700 1 --2- 172.21.15.163:0/2513105477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda740aca80 0x7fda7406aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:13.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.515+0000 7fda72d9d700 1 -- 172.21.15.163:0/2513105477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda740aca80 msgr2=0x7fda7406aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:13.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.515+0000 7fda72d9d700 1 --2- 172.21.15.163:0/2513105477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda740aca80 0x7fda7406aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.515+0000 7fda72d9d700 1 -- 172.21.15.163:0/2513105477 >> v1:172.21.15.181:6789/0 conn(0x7fda74069d60 legacy=0x7fda740c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:13.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.516+0000 7fda72d9d700 1 -- 172.21.15.163:0/2513105477 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda7406b510 con 0x7fda740c83c0 2023-10-09T06:42:13.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.516+0000 7fda72d9d700 1 --2- 172.21.15.163:0/2513105477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda740c83c0 0x7fda740cb800 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009f20 tx=0x7fda6c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6758a7a441197e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:13.523 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.516+0000 7fda71d9b700 1 -- 172.21.15.163:0/2513105477 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c00e070 con 0x7fda740c83c0 2023-10-09T06:42:13.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.516+0000 7fda71d9b700 1 -- 172.21.15.163:0/2513105477 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6c00b9d0 con 0x7fda740c83c0 2023-10-09T06:42:13.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.516+0000 7fda71d9b700 1 -- 172.21.15.163:0/2513105477 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c0092e0 con 0x7fda740c83c0 2023-10-09T06:42:13.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.516+0000 7fda78dd7700 1 -- 172.21.15.163:0/2513105477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda740c83c0 msgr2=0x7fda740cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:13.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.516+0000 7fda78dd7700 1 --2- 172.21.15.163:0/2513105477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda740c83c0 0x7fda740cb800 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009f20 tx=0x7fda6c009840 comp rx=0 tx=0).stop 2023-10-09T06:42:13.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.516+0000 7fda78dd7700 1 -- 172.21.15.163:0/2513105477 shutdown_connections 2023-10-09T06:42:13.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.516+0000 7fda78dd7700 1 --2- 172.21.15.163:0/2513105477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda740c83c0 0x7fda740cb800 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.516+0000 7fda78dd7700 1 --2- 172.21.15.163:0/2513105477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda740aca80 0x7fda7406aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.516+0000 7fda78dd7700 1 -- 172.21.15.163:0/2513105477 >> 172.21.15.163:0/2513105477 conn(0x7fda740cd670 msgr2=0x7fda740cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:13.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.516+0000 7fda78dd7700 1 -- 172.21.15.163:0/2513105477 shutdown_connections 2023-10-09T06:42:13.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.516+0000 7fda78dd7700 1 -- 172.21.15.163:0/2513105477 wait complete. 2023-10-09T06:42:13.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.518+0000 7fda78dd7700 1 Processor -- start 2023-10-09T06:42:13.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.518+0000 7fda78dd7700 1 -- start start 2023-10-09T06:42:13.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.518+0000 7fda78dd7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda74069d60 0x7fda740c4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:13.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.518+0000 7fda78dd7700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda740aca80 0x7fda7416a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:13.527 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.518+0000 7fda78dd7700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda740c83c0 0x7fda74145f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:13.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.518+0000 7fda78dd7700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda740b2240 con 0x7fda74069d60 2023-10-09T06:42:13.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.518+0000 7fda78dd7700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7416acc0 con 0x7fda740aca80 2023-10-09T06:42:13.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.518+0000 7fda78dd7700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda74158c70 con 0x7fda740c83c0 2023-10-09T06:42:13.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.518+0000 7fda7259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda74069d60 0x7fda740c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:13.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.518+0000 7fda7259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda74069d60 0x7fda740c4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55010/0 (socket says 172.21.15.163:55010) 2023-10-09T06:42:13.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.518+0000 7fda7259c700 1 -- 172.21.15.163:0/745239368 learned_addr learned my addr 172.21.15.163:0/745239368 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:13.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.519+0000 7fda69d9b700 1 --2- 172.21.15.163:0/745239368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda740aca80 0x7fda7416a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:13.529 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.519+0000 7fda72d9d700 1 --2- 172.21.15.163:0/745239368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda740c83c0 0x7fda74145f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:13.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.519+0000 7fda7259c700 1 -- 172.21.15.163:0/745239368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda740c83c0 msgr2=0x7fda74145f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:13.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.519+0000 7fda7259c700 1 --2- 172.21.15.163:0/745239368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda740c83c0 0x7fda74145f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.519+0000 7fda7259c700 1 -- 172.21.15.163:0/745239368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda740aca80 msgr2=0x7fda7416a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:13.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.519+0000 7fda7259c700 1 --2- 172.21.15.163:0/745239368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda740aca80 0x7fda7416a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.519+0000 7fda7259c700 1 -- 172.21.15.163:0/745239368 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6c009ae0 con 0x7fda74069d60 2023-10-09T06:42:13.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.519+0000 7fda69d9b700 1 --2- 172.21.15.163:0/745239368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda740aca80 0x7fda7416a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:42:13.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.519+0000 7fda7259c700 1 --2- 172.21.15.163:0/745239368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda74069d60 0x7fda740c4720 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fda6400ecb0 tx=0x7fda64009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:13.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.519+0000 7fda6affd700 1 -- 172.21.15.163:0/745239368 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6400c9d0 con 0x7fda74069d60 2023-10-09T06:42:13.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.519+0000 7fda6affd700 1 -- 172.21.15.163:0/745239368 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6400cb30 con 0x7fda74069d60 2023-10-09T06:42:13.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.519+0000 7fda6affd700 1 -- 172.21.15.163:0/745239368 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6400ceb0 con 0x7fda74069d60 2023-10-09T06:42:13.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.519+0000 7fda78dd7700 1 -- 172.21.15.163:0/745239368 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda741464d0 con 0x7fda74069d60 2023-10-09T06:42:13.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.519+0000 7fda78dd7700 1 -- 172.21.15.163:0/745239368 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda7416db90 con 0x7fda74069d60 2023-10-09T06:42:13.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.520+0000 7fda78dd7700 1 -- 172.21.15.163:0/745239368 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda7404c3c0 con 0x7fda74069d60 2023-10-09T06:42:13.533 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.520+0000 7fda6affd700 1 -- 172.21.15.163:0/745239368 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fda64013680 con 0x7fda74069d60 2023-10-09T06:42:13.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.521+0000 7fda6affd700 1 --2- 172.21.15.163:0/745239368 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fda54067320 0x7fda540697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:13.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.521+0000 7fda6affd700 1 -- 172.21.15.163:0/745239368 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fda640850b0 con 0x7fda74069d60 2023-10-09T06:42:13.534 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.521+0000 7fda69d9b700 1 --2- 172.21.15.163:0/745239368 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fda54067320 0x7fda540697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:13.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.521+0000 7fda69d9b700 1 --2- 172.21.15.163:0/745239368 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fda54067320 0x7fda540697d0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fda740b1cf0 tx=0x7fda5c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:13.535 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.525+0000 7fda6affd700 1 -- 172.21.15.163:0/745239368 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda64052000 con 0x7fda74069d60 2023-10-09T06:42:13.705 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.705+0000 7fda78dd7700 1 -- 172.21.15.163:0/745239368 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fda740b2520 con 0x7fda54067320 2023-10-09T06:42:13.707 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.708+0000 7fda6affd700 1 -- 172.21.15.163:0/745239368 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51015 (secure 0 0 0) 0x7fda740b2520 con 0x7fda54067320 2023-10-09T06:42:13.708 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:42:13.710 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.710+0000 7fda78dd7700 1 -- 172.21.15.163:0/745239368 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fda54067320 msgr2=0x7fda540697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:13.710 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.710+0000 7fda78dd7700 1 --2- 172.21.15.163:0/745239368 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fda54067320 0x7fda540697d0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fda740b1cf0 tx=0x7fda5c009040 comp rx=0 tx=0).stop 2023-10-09T06:42:13.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.710+0000 7fda78dd7700 1 -- 172.21.15.163:0/745239368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda74069d60 msgr2=0x7fda740c4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:13.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.710+0000 7fda78dd7700 1 --2- 172.21.15.163:0/745239368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda74069d60 0x7fda740c4720 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fda6400ecb0 tx=0x7fda64009120 comp rx=0 tx=0).stop 2023-10-09T06:42:13.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.710+0000 7fda78dd7700 1 -- 172.21.15.163:0/745239368 shutdown_connections 2023-10-09T06:42:13.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.710+0000 7fda78dd7700 1 --2- 172.21.15.163:0/745239368 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fda54067320 0x7fda540697d0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.710+0000 7fda78dd7700 1 --2- 172.21.15.163:0/745239368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda74069d60 0x7fda740c4720 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.710+0000 7fda78dd7700 1 --2- 172.21.15.163:0/745239368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda740c83c0 0x7fda74145f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.711 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.710+0000 7fda78dd7700 1 --2- 172.21.15.163:0/745239368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda740aca80 0x7fda7416a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.710+0000 7fda78dd7700 1 -- 172.21.15.163:0/745239368 >> 172.21.15.163:0/745239368 conn(0x7fda740cd670 msgr2=0x7fda740c2410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:13.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.710+0000 7fda78dd7700 1 -- 172.21.15.163:0/745239368 shutdown_connections 2023-10-09T06:42:13.712 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.710+0000 7fda78dd7700 1 -- 172.21.15.163:0/745239368 wait complete. 2023-10-09T06:42:13.712 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:42:13.722 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":546,"stamp":"2023-10-09T06:42:12.468249+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2432236,"kb_used_data":21164,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752542484,"statfs":{"total":773094113280,"available":770603503616,"internally_reserved":0,"allocated":21671936,"data_stored":10378971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4067,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-49,"num_write_kb":-18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001511"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":312,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-09T06:42:08.646970+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:42:08.646970+0000","last_peered":"2023-10-09T06:42:08.646970+0000","last_clean":"2023-10-09T06:42:08.646970+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:42:08.646970+0000","last_undegraded":"2023-10-09T06:42:08.646970+0000","last_fullsized":"2023-10-09T06:42:08.646970+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574992,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304464,"kb_used_data":3080,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067376,"statfs":{"total":96636764160,"available":96324993024,"internally_reserved":0,"allocated":3153920,"data_stored":1740423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58199999999999996,"15min":0.53100000000000003},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.70599999999999996,"5min":3.778,"15min":3.778},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54300000000000004,"15min":0.52200000000000002},"min":{"1min":0.38400000000000001,"5min":0.33400000000000002,"15min":0.29699999999999999},"max":{"1min":0.77100000000000002,"5min":1.768,"15min":1.768},"last":0.50900000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53400000000000003,"15min":0.52200000000000002},"min":{"1min":0.32100000000000001,"5min":0.31900000000000001,"15min":0.27100000000000002},"max":{"1min":0.72999999999999998,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.38},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.53700000000000003},"min":{"1min":0.435,"5min":0.378,"15min":0.29699999999999999},"max":{"1min":0.68100000000000005,"5min":1.806,"15min":1.806},"last":0.40300000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59099999999999997,"15min":0.54100000000000004},"min":{"1min":0.42599999999999999,"5min":0.36399999999999999,"15min":0.32200000000000001},"max":{"1min":0.76000000000000001,"5min":3.7210000000000001,"15min":3.7210000000000001},"last":0.432},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56699999999999995,"15min":0.54900000000000004},"min":{"1min":0.44600000000000001,"5min":0.35399999999999998,"15min":0.315},"max":{"1min":0.69799999999999995,"5min":1.51,"15min":1.51},"last":0.47899999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60799999999999998,"15min":0.56699999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.33500000000000002},"max":{"1min":0.73299999999999998,"5min":3.8090000000000002,"15min":3.8090000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.47499999999999998,"5min":0.38400000000000001,"15min":0.36499999999999999},"max":{"1min":0.74299999999999999,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.442}]},{"osd":4,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.505,"15min":0.50600000000000001},"min":{"1min":0.31,"5min":0.25600000000000001,"15min":0.24399999999999999},"max":{"1min":0.55100000000000005,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.628,"15min":0.60599999999999998},"min":{"1min":0.35599999999999998,"5min":0.35199999999999998,"15min":0.30299999999999999},"max":{"1min":0.751,"5min":4.8239999999999998,"15min":4.8239999999999998},"last":0.53300000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53800000000000003,"15min":0.55100000000000005},"min":{"1min":0.34599999999999997,"5min":0.313,"15min":0.307},"max":{"1min":0.64800000000000002,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.63200000000000001,"15min":0.61799999999999999},"min":{"1min":0.38900000000000001,"5min":0.34300000000000003,"15min":0.34200000000000003},"max":{"1min":0.71699999999999997,"5min":3.6909999999999998,"15min":3.6909999999999998},"last":0.61599999999999999}]},{"osd":6,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.59499999999999997,"15min":0.58699999999999997},"min":{"1min":0.39100000000000001,"5min":0.318,"15min":0.28699999999999998},"max":{"1min":0.66600000000000004,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.63400000000000001,"15min":0.60999999999999999},"min":{"1min":0.375,"5min":0.33000000000000002,"15min":0.315},"max":{"1min":0.72499999999999998,"5min":4.7789999999999999,"15min":4.7789999999999999},"last":0.56999999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574991,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303884,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067956,"statfs":{"total":96636764160,"available":96325586944,"internally_reserved":0,"allocated":2560000,"data_stored":1150055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54200000000000004,"15min":0.51600000000000001},"min":{"1min":0.40600000000000003,"5min":0.35499999999999998,"15min":0.30399999999999999},"max":{"1min":0.75700000000000001,"5min":0.80500000000000005,"15min":0.82299999999999995},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55200000000000005,"15min":0.52300000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.79000000000000004,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.41099999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52900000000000003,"15min":0.51600000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.73499999999999999,"5min":0.74199999999999999,"15min":1.1359999999999999},"last":0.433},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.56899999999999995,"15min":0.54200000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.28799999999999998},"max":{"1min":0.754,"5min":0.78700000000000003,"15min":1.1539999999999999},"last":0.35799999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.55900000000000005,"15min":0.53000000000000003},"min":{"1min":0.45400000000000001,"5min":0.34300000000000003,"15min":0.29899999999999999},"max":{"1min":0.74199999999999999,"5min":0.77000000000000002,"15min":0.78400000000000003},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54400000000000004,"15min":0.53200000000000003},"min":{"1min":0.34899999999999998,"5min":0.34499999999999997,"15min":0.313},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.48299999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56499999999999995,"15min":0.55000000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.75700000000000001,"5min":0.79000000000000004,"15min":0.86799999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.58799999999999997,"15min":0.63700000000000001},"min":{"1min":0.49299999999999999,"5min":0.371,"15min":0.34300000000000003},"max":{"1min":0.71699999999999997,"5min":0.76900000000000002,"15min":4.7510000000000003},"last":0.51500000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47099999999999997,"15min":0.47399999999999998},"min":{"1min":0.30299999999999999,"5min":0.28999999999999998,"15min":0.27900000000000003},"max":{"1min":0.69699999999999995,"5min":0.74299999999999999,"15min":0.76400000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59099999999999997,"15min":0.58299999999999996},"min":{"1min":0.435,"5min":0.38700000000000001,"15min":0.25800000000000001},"max":{"1min":0.77700000000000002,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.53000000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.36399999999999999,"5min":0.31900000000000001,"15min":0.246},"max":{"1min":0.75,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.41999999999999998,"5min":0.40999999999999998,"15min":0.35699999999999998},"max":{"1min":0.73399999999999999,"5min":0.76200000000000001,"15min":0.86599999999999999},"last":0.625}]},{"osd":7,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56000000000000005,"15min":0.57199999999999995},"min":{"1min":0.39100000000000001,"5min":0.33400000000000002,"15min":0.29899999999999999},"max":{"1min":0.74199999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.499},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59599999999999997,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.38400000000000001,"15min":0.33000000000000002},"max":{"1min":0.72099999999999997,"5min":0.84399999999999997,"15min":0.88300000000000001},"last":0.57299999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574994,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303884,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067956,"statfs":{"total":96636764160,"available":96325586944,"internally_reserved":0,"allocated":2560000,"data_stored":1150055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.32800000000000001,"15min":0.316},"min":{"1min":0.26900000000000002,"5min":0.192,"15min":0.192},"max":{"1min":0.50900000000000001,"5min":0.55600000000000005,"15min":0.80700000000000005},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.432,"5min":0.38100000000000001,"15min":0.36399999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.21199999999999999},"max":{"1min":0.54600000000000004,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.47299999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.36099999999999999,"15min":0.35399999999999998},"min":{"1min":0.317,"5min":0.215,"15min":0.20399999999999999},"max":{"1min":0.53100000000000003,"5min":0.53700000000000003,"15min":0.68300000000000005},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.40899999999999997,"15min":0.39300000000000002},"min":{"1min":0.36799999999999999,"5min":0.25800000000000001,"15min":0.251},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.438}]},{"osd":3,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.39800000000000002,"15min":0.376},"min":{"1min":0.32600000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.375,"15min":0.46200000000000002},"min":{"1min":0.28599999999999998,"5min":0.23400000000000001,"15min":0.22700000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":5.8529999999999998},"last":0.39800000000000002}]},{"osd":4,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.59999999999999998,"15min":0.58499999999999996},"min":{"1min":0.45300000000000001,"5min":0.36199999999999999,"15min":0.32200000000000001},"max":{"1min":0.79800000000000004,"5min":1.01,"15min":1.01},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61799999999999999,"15min":0.59799999999999998},"min":{"1min":0.47299999999999998,"5min":0.40000000000000002,"15min":0.37},"max":{"1min":0.79300000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.67200000000000004}]},{"osd":5,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.59799999999999998,"15min":0.67500000000000004},"min":{"1min":0.40899999999999997,"5min":0.36499999999999999,"15min":0.34599999999999997},"max":{"1min":0.85599999999999998,"5min":0.92500000000000004,"15min":5.9459999999999997},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.621,"15min":0.61399999999999999},"min":{"1min":0.55700000000000005,"5min":0.38100000000000001,"15min":0.34300000000000003},"max":{"1min":0.82599999999999996,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.60599999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.57799999999999996,"15min":0.65000000000000002},"min":{"1min":0.44600000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":5.9160000000000004},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.51700000000000002,"5min":0.379,"15min":0.35699999999999998},"max":{"1min":0.86699999999999999,"5min":0.88600000000000001,"15min":0.91400000000000003},"last":0.68500000000000005}]},{"osd":7,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.51400000000000001,"5min":0.39000000000000001,"15min":0.379},"max":{"1min":0.76800000000000002,"5min":0.91000000000000003,"15min":0.97099999999999997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.59099999999999997,"15min":0.67600000000000005},"min":{"1min":0.48399999999999999,"5min":0.41399999999999998,"15min":0.38100000000000001},"max":{"1min":0.83399999999999996,"5min":0.86099999999999999,"15min":5.9729999999999999},"last":0.625}]}]},{"osd":0,"up_from":13,"seq":55834574993,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304464,"kb_used_data":3080,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067376,"statfs":{"total":96636764160,"available":96324993024,"internally_reserved":0,"allocated":3153920,"data_stored":1740423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.34000000000000002,"15min":0.32200000000000001},"min":{"1min":0.252,"5min":0.216,"15min":0.21299999999999999},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.35799999999999998,"15min":0.34699999999999998},"min":{"1min":0.24399999999999999,"5min":0.222,"15min":0.214},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.94499999999999995},"last":0.315}]},{"osd":2,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.38300000000000001,"15min":0.36799999999999999},"min":{"1min":0.27100000000000002,"5min":0.22,"15min":0.20200000000000001},"max":{"1min":0.53000000000000003,"5min":0.54000000000000004,"15min":0.75600000000000001},"last":0.247},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.41399999999999998,"15min":0.39500000000000002},"min":{"1min":0.34599999999999997,"5min":0.29099999999999998,"15min":0.28000000000000003},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.91700000000000004},"last":0.35199999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.36499999999999999,"15min":0.372},"min":{"1min":0.318,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":1.554},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.40200000000000002,"15min":0.39500000000000002},"min":{"1min":0.32400000000000001,"5min":0.27700000000000002,"15min":0.25800000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.79000000000000004},"last":0.41899999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58699999999999997,"15min":0.56299999999999994},"min":{"1min":0.46300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85699999999999998,"5min":0.92600000000000005,"15min":1.212},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58699999999999997,"15min":0.65700000000000003},"min":{"1min":0.43099999999999999,"5min":0.35799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85599999999999998,"5min":0.90800000000000003,"15min":5.484},"last":0.59199999999999997}]},{"osd":5,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58499999999999996,"15min":0.65900000000000003},"min":{"1min":0.437,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83099999999999996,"5min":0.97399999999999998,"15min":5.5449999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.47499999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.875,"5min":0.998,"15min":1.113},"last":0.63800000000000001}]},{"osd":6,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.42399999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86899999999999999,"5min":0.871,"15min":1.0529999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.627,"15min":0.61399999999999999},"min":{"1min":0.505,"5min":0.42899999999999999,"15min":0.41599999999999998},"max":{"1min":0.88400000000000001,"5min":0.96099999999999997,"15min":1.169},"last":0.61899999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60399999999999998,"15min":0.59399999999999997},"min":{"1min":0.47299999999999998,"5min":0.40200000000000002,"15min":0.371},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":1.2390000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59899999999999998,"15min":0.67800000000000005},"min":{"1min":0.48399999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83199999999999996,"5min":0.94299999999999995,"15min":5.5800000000000001},"last":0.55300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574994,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303888,"kb_used_data":2504,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067952,"statfs":{"total":96636764160,"available":96325582848,"internally_reserved":0,"allocated":2564096,"data_stored":1147850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33400000000000002,"15min":0.34000000000000002},"min":{"1min":0.252,"5min":0.20000000000000001,"15min":0.19900000000000001},"max":{"1min":0.51800000000000002,"5min":0.66000000000000003,"15min":1.4199999999999999},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.34499999999999997,"15min":0.46100000000000002},"min":{"1min":0.28000000000000003,"5min":0.221,"15min":0.19900000000000001},"max":{"1min":0.53100000000000003,"5min":0.55600000000000005,"15min":7.2699999999999996},"last":0.22900000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.34200000000000003,"15min":0.34100000000000003},"min":{"1min":0.26200000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.497,"5min":0.56399999999999995,"15min":0.65400000000000003},"last":0.21199999999999999},{"interface":"front","average":{"1min":0.434,"5min":0.39600000000000002,"15min":0.51400000000000001},"min":{"1min":0.33500000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.56799999999999995,"5min":0.60399999999999998,"15min":7.327},"last":0.27800000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.41099999999999998,"15min":0.39400000000000002},"min":{"1min":0.39600000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.56899999999999995,"5min":0.64600000000000002,"15min":0.68600000000000005},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.39000000000000001,"15min":0.38700000000000001},"min":{"1min":0.32600000000000001,"5min":0.252,"15min":0.23999999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.72699999999999998},"last":0.311}]},{"osd":4,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56299999999999994,"15min":0.54800000000000004},"min":{"1min":0.45800000000000002,"5min":0.33600000000000002,"15min":0.317},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.58199999999999996,"15min":0.68999999999999995},"min":{"1min":0.46899999999999997,"5min":0.35999999999999999,"15min":0.33600000000000002},"max":{"1min":0.82599999999999996,"5min":0.91100000000000003,"15min":7.2389999999999999},"last":0.54600000000000004}]},{"osd":5,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59499999999999997,"15min":0.77300000000000002},"min":{"1min":0.495,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.872,"5min":0.89800000000000002,"15min":10.890000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.60899999999999999,"15min":0.60299999999999998},"min":{"1min":0.45600000000000002,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":0.80000000000000004,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.61699999999999999}]},{"osd":6,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58099999999999996,"15min":0.69599999999999995},"min":{"1min":0.50800000000000001,"5min":0.35999999999999999,"15min":0.33500000000000002},"max":{"1min":0.77600000000000002,"5min":0.82499999999999996,"15min":7.2999999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60499999999999998,"15min":0.58299999999999996},"min":{"1min":0.44700000000000001,"5min":0.40300000000000002,"15min":0.377},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.749}]},{"osd":7,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.61099999999999999,"15min":0.72599999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.37},"max":{"1min":0.80100000000000005,"5min":0.91300000000000003,"15min":7.1879999999999997},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63,"15min":0.624},"min":{"1min":0.53300000000000003,"5min":0.437,"15min":0.38},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.70199999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574993,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303884,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067956,"statfs":{"total":96636764160,"available":96325586944,"internally_reserved":0,"allocated":2560000,"data_stored":1150055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.34200000000000003,"15min":0.33800000000000002},"min":{"1min":0.23799999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.55400000000000005,"5min":0.61699999999999999,"15min":5.8700000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.36299999999999999,"15min":0.36799999999999999},"min":{"1min":0.255,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.55400000000000005,"5min":0.58499999999999996,"15min":5.6769999999999996},"last":0.28799999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.36599999999999999,"15min":0.36399999999999999},"min":{"1min":0.28599999999999998,"5min":0.21299999999999999,"15min":0.21199999999999999},"max":{"1min":0.46700000000000003,"5min":0.61199999999999999,"15min":4.0469999999999997},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.40200000000000002,"15min":0.39500000000000002},"min":{"1min":0.32900000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.60199999999999998,"5min":0.68600000000000005,"15min":4.1509999999999998},"last":0.31900000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.371,"15min":0.373},"min":{"1min":0.27300000000000002,"5min":0.23200000000000001,"15min":0.222},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":3.1139999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.39500000000000002,"15min":0.38900000000000001},"min":{"1min":0.28799999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.54700000000000004,"5min":0.66900000000000004,"15min":3.3180000000000001},"last":0.34699999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56999999999999995,"15min":0.57299999999999995},"min":{"1min":0.42799999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.75700000000000001,"5min":0.86699999999999999,"15min":3.0899999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.432,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79300000000000004,"5min":0.84499999999999997,"15min":3.0710000000000002},"last":0.60499999999999998}]},{"osd":5,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.58099999999999996,"15min":0.59099999999999997},"min":{"1min":0.45300000000000001,"5min":0.33600000000000002,"15min":0.32700000000000001},"max":{"1min":0.86399999999999999,"5min":0.877,"15min":6.2300000000000004},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.58699999999999997,"15min":0.59699999999999998},"min":{"1min":0.498,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78800000000000003,"5min":0.83599999999999997,"15min":3.0219999999999998},"last":0.59499999999999997}]},{"osd":6,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.61899999999999999,"15min":0.60499999999999998},"min":{"1min":0.47899999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.147,"5min":1.147,"15min":2.6850000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.65000000000000002,"15min":0.65700000000000003},"min":{"1min":0.5,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":6.5970000000000004},"last":0.68999999999999995}]},{"osd":7,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.61199999999999999,"15min":0.61699999999999999},"min":{"1min":0.46600000000000003,"5min":0.41499999999999998,"15min":0.38200000000000001},"max":{"1min":0.81799999999999995,"5min":0.86099999999999999,"15min":4.2629999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60799999999999998,"15min":0.61899999999999999},"min":{"1min":0.47899999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.89200000000000002,"5min":0.89900000000000002,"15min":7.3650000000000002},"last":0.57499999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574992,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303884,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067956,"statfs":{"total":96636764160,"available":96325586944,"internally_reserved":0,"allocated":2560000,"data_stored":1150055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54800000000000004,"15min":0.54500000000000004},"min":{"1min":0.38900000000000001,"5min":0.307,"15min":0.27000000000000002},"max":{"1min":0.71899999999999997,"5min":0.88900000000000001,"15min":1.083},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.55200000000000005,"15min":0.53000000000000003},"min":{"1min":0.50800000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":1.054},"last":0.58599999999999997}]},{"osd":1,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56599999999999995,"15min":0.55100000000000005},"min":{"1min":0.52400000000000002,"5min":0.28399999999999997,"15min":0.27100000000000002},"max":{"1min":0.77300000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53100000000000003,"15min":0.50800000000000001},"min":{"1min":0.38500000000000001,"5min":0.33800000000000002,"15min":0.31},"max":{"1min":0.75,"5min":0.76300000000000001,"15min":0.79100000000000004},"last":0.78500000000000003}]},{"osd":2,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.54700000000000004,"15min":0.54000000000000004},"min":{"1min":0.443,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76500000000000001,"5min":0.84899999999999998,"15min":1.0249999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.499,"5min":0.38100000000000001,"15min":0.36299999999999999},"max":{"1min":0.73399999999999999,"5min":0.90500000000000003,"15min":0.93799999999999994},"last":0.748}]},{"osd":3,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55300000000000005,"15min":0.54800000000000004},"min":{"1min":0.40699999999999997,"5min":0.33300000000000002,"15min":0.29199999999999998},"max":{"1min":0.747,"5min":0.86899999999999999,"15min":0.98499999999999999},"last":0.62},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.47799999999999998,"5min":0.38700000000000001,"15min":0.34100000000000003},"max":{"1min":0.68200000000000005,"5min":0.92900000000000005,"15min":1.1319999999999999},"last":0.68300000000000005}]},{"osd":5,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.50600000000000001,"15min":0.53300000000000003},"min":{"1min":0.35499999999999998,"5min":0.307,"15min":0.27300000000000002},"max":{"1min":0.55900000000000005,"5min":0.83299999999999996,"15min":0.998},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57699999999999996,"15min":0.59399999999999997},"min":{"1min":0.40000000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71999999999999997,"5min":0.95599999999999996,"15min":0.97999999999999998},"last":0.50900000000000001}]},{"osd":6,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.54600000000000004,"15min":0.55600000000000005},"min":{"1min":0.36899999999999999,"5min":0.35499999999999998,"15min":0.29999999999999999},"max":{"1min":0.67500000000000004,"5min":0.86299999999999999,"15min":1.0189999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.56999999999999995,"15min":0.58399999999999996},"min":{"1min":0.38900000000000001,"5min":0.36399999999999999,"15min":0.32700000000000001},"max":{"1min":0.72699999999999998,"5min":0.85599999999999998,"15min":1.01},"last":0.79900000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55700000000000005,"15min":0.56200000000000006},"min":{"1min":0.42399999999999999,"5min":0.35299999999999998,"15min":0.28299999999999997},"max":{"1min":0.65400000000000003,"5min":0.84399999999999997,"15min":0.90000000000000002},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.60599999999999998,"15min":0.625},"min":{"1min":0.374,"5min":0.34999999999999998,"15min":0.32200000000000001},"max":{"1min":0.71299999999999997,"5min":0.93799999999999994,"15min":1.175},"last":0.63600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574992,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303884,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067956,"statfs":{"total":96636764160,"available":96325586944,"internally_reserved":0,"allocated":2560000,"data_stored":1150055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54200000000000004,"15min":0.51700000000000002},"min":{"1min":0.36399999999999999,"5min":0.29499999999999998,"15min":0.28599999999999998},"max":{"1min":0.76400000000000001,"5min":0.79700000000000004,"15min":0.86599999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55400000000000005,"15min":0.52800000000000002},"min":{"1min":0.45000000000000001,"5min":0.36399999999999999,"15min":0.28799999999999998},"max":{"1min":0.83599999999999997,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.63100000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52700000000000002,"15min":0.51500000000000001},"min":{"1min":0.38800000000000001,"5min":0.34100000000000003,"15min":0.311},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.54400000000000004,"15min":0.53400000000000003},"min":{"1min":0.41499999999999998,"5min":0.36899999999999999,"15min":0.30199999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.81599999999999995},"last":0.40899999999999997}]},{"osd":2,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58199999999999996,"15min":0.55800000000000005},"min":{"1min":0.39300000000000002,"5min":0.34699999999999998,"15min":0.28199999999999997},"max":{"1min":0.79800000000000004,"5min":1.139,"15min":1.139},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.52600000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78100000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.56899999999999995}]},{"osd":3,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57099999999999995,"15min":0.55200000000000005},"min":{"1min":0.43099999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.75,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.45900000000000002,"5min":0.40699999999999997,"15min":0.312},"max":{"1min":0.80200000000000005,"5min":0.80600000000000005,"15min":0.81100000000000005},"last":0.48099999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.33600000000000002,"5min":0.28399999999999997,"15min":0.26700000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53600000000000003,"15min":0.54200000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.27700000000000002},"max":{"1min":0.751,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.501}]},{"osd":6,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.35599999999999998,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":0.73499999999999999,"5min":1.752,"15min":1.752},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55600000000000005,"15min":0.55900000000000005},"min":{"1min":0.39100000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.377}]},{"osd":7,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56399999999999995,"15min":0.57599999999999996},"min":{"1min":0.375,"5min":0.311,"15min":0.311},"max":{"1min":0.79900000000000004,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58599999999999997,"15min":0.60299999999999998},"min":{"1min":0.40799999999999997,"5min":0.372,"15min":0.314},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.90600000000000003},"last":0.52000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:42:13.724 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-09T06:42:13.724 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:42:13.839 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.839+0000 7f306dc5e700 1 Processor -- start 2023-10-09T06:42:13.840 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.839+0000 7f306dc5e700 1 -- start start 2023-10-09T06:42:13.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.840+0000 7f306dc5e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3068159890 0x7f306815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:13.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.840+0000 7f306dc5e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3068159c60 0x7f306815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:13.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.840+0000 7f306dc5e700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f306815db10 0x7f3068069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:13.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.840+0000 7f306dc5e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f306815a160 con 0x7f3068159c60 2023-10-09T06:42:13.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.840+0000 7f306dc5e700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30680d13a0 con 0x7f306815db10 2023-10-09T06:42:13.842 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.840+0000 7f306dc5e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f306815e140 con 0x7f3068159890 2023-10-09T06:42:13.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.840+0000 7f3066ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3068159c60 0x7f306815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:13.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.840+0000 7f3066ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3068159c60 0x7f306815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55014/0 (socket says 172.21.15.163:55014) 2023-10-09T06:42:13.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.840+0000 7f3066ffd700 1 -- 172.21.15.163:0/384502013 learned_addr learned my addr 172.21.15.163:0/384502013 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:13.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.840+0000 7f3067fff700 1 --2- 172.21.15.163:0/384502013 >> v2:172.21.15.181:3300/0 conn(0x7f306815db10 0x7f3068069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:13.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.841+0000 7f30677fe700 1 --2- 172.21.15.163:0/384502013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3068159890 0x7f306815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:13.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.841+0000 7f3066ffd700 1 -- 172.21.15.163:0/384502013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3068159890 msgr2=0x7f306815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:13.843 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.841+0000 7f3066ffd700 1 --2- 172.21.15.163:0/384502013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3068159890 0x7f306815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.841+0000 7f3066ffd700 1 -- 172.21.15.163:0/384502013 >> v2:172.21.15.181:3300/0 conn(0x7f306815db10 msgr2=0x7f3068069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:13.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.841+0000 7f3066ffd700 1 --2- 172.21.15.163:0/384502013 >> v2:172.21.15.181:3300/0 conn(0x7f306815db10 0x7f3068069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.841+0000 7f3066ffd700 1 -- 172.21.15.163:0/384502013 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306815a3d0 con 0x7f3068159c60 2023-10-09T06:42:13.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.841+0000 7f3066ffd700 1 --2- 172.21.15.163:0/384502013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3068159c60 0x7f306815d5d0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f3068089b50 tx=0x7f305c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467b12eada42c62d server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:13.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.841+0000 7f30667fc700 1 -- 172.21.15.163:0/384502013 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c00b860 con 0x7f3068159c60 2023-10-09T06:42:13.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.841+0000 7f30667fc700 1 -- 172.21.15.163:0/384502013 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305c00b9c0 con 0x7f3068159c60 2023-10-09T06:42:13.844 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.841+0000 7f30667fc700 1 -- 172.21.15.163:0/384502013 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c00bd60 con 0x7f3068159c60 2023-10-09T06:42:13.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.841+0000 7f306dc5e700 1 -- 172.21.15.163:0/384502013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3068159c60 msgr2=0x7f306815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:13.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.841+0000 7f306dc5e700 1 --2- 172.21.15.163:0/384502013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3068159c60 0x7f306815d5d0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f3068089b50 tx=0x7f305c009250 comp rx=0 tx=0).stop 2023-10-09T06:42:13.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.841+0000 7f306dc5e700 1 -- 172.21.15.163:0/384502013 shutdown_connections 2023-10-09T06:42:13.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.841+0000 7f306dc5e700 1 --2- 172.21.15.163:0/384502013 >> v2:172.21.15.181:3300/0 conn(0x7f306815db10 0x7f3068069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.841+0000 7f306dc5e700 1 --2- 172.21.15.163:0/384502013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3068159890 0x7f306815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.841+0000 7f306dc5e700 1 --2- 172.21.15.163:0/384502013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3068159c60 0x7f306815d5d0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.842+0000 7f306dc5e700 1 -- 172.21.15.163:0/384502013 >> 172.21.15.163:0/384502013 conn(0x7f3068068e20 msgr2=0x7f30680d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:13.845 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.842+0000 7f306dc5e700 1 -- 172.21.15.163:0/384502013 shutdown_connections 2023-10-09T06:42:13.846 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.842+0000 7f306dc5e700 1 -- 172.21.15.163:0/384502013 wait complete. 2023-10-09T06:42:13.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.843+0000 7f306dc5e700 1 Processor -- start 2023-10-09T06:42:13.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.843+0000 7f306dc5e700 1 -- start start 2023-10-09T06:42:13.848 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.843+0000 7f306dc5e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3068159890 0x7f30680cf830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:13.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.843+0000 7f306dc5e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3068159c60 0x7f30680b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:13.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.843+0000 7f306dc5e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f306815db10 0x7f30680b5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:13.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.843+0000 7f306dc5e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3068152800 con 0x7f3068159c60 2023-10-09T06:42:13.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.843+0000 7f306dc5e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30680c8fd0 con 0x7f306815db10 2023-10-09T06:42:13.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.843+0000 7f306dc5e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30680c83c0 con 0x7f3068159890 2023-10-09T06:42:13.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.844+0000 7f3066ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3068159c60 0x7f30680b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:13.849 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.844+0000 7f3066ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3068159c60 0x7f30680b4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55028/0 (socket says 172.21.15.163:55028) 2023-10-09T06:42:13.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.844+0000 7f3066ffd700 1 -- 172.21.15.163:0/1801801802 learned_addr learned my addr 172.21.15.163:0/1801801802 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:13.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.844+0000 7f3067fff700 1 --2- 172.21.15.163:0/1801801802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f306815db10 0x7f30680b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:13.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.844+0000 7f30677fe700 1 --2- 172.21.15.163:0/1801801802 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3068159890 0x7f30680cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:13.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.844+0000 7f3066ffd700 1 -- 172.21.15.163:0/1801801802 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3068159890 msgr2=0x7f30680cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:13.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.844+0000 7f3066ffd700 1 --2- 172.21.15.163:0/1801801802 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3068159890 0x7f30680cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.844+0000 7f3066ffd700 1 -- 172.21.15.163:0/1801801802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f306815db10 msgr2=0x7f30680b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:13.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.844+0000 7f3066ffd700 1 --2- 172.21.15.163:0/1801801802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f306815db10 0x7f30680b5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:13.850 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.844+0000 7f3066ffd700 1 -- 172.21.15.163:0/1801801802 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305c0094f0 con 0x7f3068159c60 2023-10-09T06:42:13.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.844+0000 7f3066ffd700 1 --2- 172.21.15.163:0/1801801802 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3068159c60 0x7f30680b4d00 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f3068070e20 tx=0x7f305c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:13.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.844+0000 7f304f7fe700 1 -- 172.21.15.163:0/1801801802 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c011dc0 con 0x7f3068159c60 2023-10-09T06:42:13.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.844+0000 7f304f7fe700 1 -- 172.21.15.163:0/1801801802 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305c024440 con 0x7f3068159c60 2023-10-09T06:42:13.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.844+0000 7f306dc5e700 1 -- 172.21.15.163:0/1801801802 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30680c86a0 con 0x7f3068159c60 2023-10-09T06:42:13.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.844+0000 7f304f7fe700 1 -- 172.21.15.163:0/1801801802 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c024800 con 0x7f3068159c60 2023-10-09T06:42:13.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.844+0000 7f306dc5e700 1 -- 172.21.15.163:0/1801801802 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30680c8c60 con 0x7f3068159c60 2023-10-09T06:42:13.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.845+0000 7f306dc5e700 1 -- 172.21.15.163:0/1801801802 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30680c9140 con 0x7f3068159c60 2023-10-09T06:42:13.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.846+0000 7f304f7fe700 1 -- 172.21.15.163:0/1801801802 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f305c01a420 con 0x7f3068159c60 2023-10-09T06:42:13.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.846+0000 7f304f7fe700 1 --2- 172.21.15.163:0/1801801802 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3054067320 0x7f30540697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:13.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.846+0000 7f304f7fe700 1 -- 172.21.15.163:0/1801801802 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f305c086790 con 0x7f3068159c60 2023-10-09T06:42:13.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.846+0000 7f30677fe700 1 --2- 172.21.15.163:0/1801801802 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3054067320 0x7f30540697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:13.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.847+0000 7f30677fe700 1 --2- 172.21.15.163:0/1801801802 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3054067320 0x7f30540697d0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f3058006fd0 tx=0x7f3058008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:13.852 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:13.850+0000 7f304f7fe700 1 -- 172.21.15.163:0/1801801802 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f305c053720 con 0x7f3068159c60 2023-10-09T06:42:14.032 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.032+0000 7f306dc5e700 1 -- 172.21.15.163:0/1801801802 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f306804c3c0 con 0x7f3054067320 2023-10-09T06:42:14.034 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.035+0000 7f304f7fe700 1 -- 172.21.15.163:0/1801801802 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51015 (secure 0 0 0) 0x7f306804c3c0 con 0x7f3054067320 2023-10-09T06:42:14.035 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:42:14.036 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.036+0000 7f306dc5e700 1 -- 172.21.15.163:0/1801801802 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3054067320 msgr2=0x7f30540697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:14.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.036+0000 7f306dc5e700 1 --2- 172.21.15.163:0/1801801802 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3054067320 0x7f30540697d0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f3058006fd0 tx=0x7f3058008040 comp rx=0 tx=0).stop 2023-10-09T06:42:14.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.036+0000 7f306dc5e700 1 -- 172.21.15.163:0/1801801802 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3068159c60 msgr2=0x7f30680b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:14.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.036+0000 7f306dc5e700 1 --2- 172.21.15.163:0/1801801802 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3068159c60 0x7f30680b4d00 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f3068070e20 tx=0x7f305c017000 comp rx=0 tx=0).stop 2023-10-09T06:42:14.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.036+0000 7f306dc5e700 1 -- 172.21.15.163:0/1801801802 shutdown_connections 2023-10-09T06:42:14.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.037+0000 7f306dc5e700 1 --2- 172.21.15.163:0/1801801802 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3054067320 0x7f30540697d0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:14.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.037+0000 7f306dc5e700 1 --2- 172.21.15.163:0/1801801802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f306815db10 0x7f30680b5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:14.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.037+0000 7f306dc5e700 1 --2- 172.21.15.163:0/1801801802 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3068159890 0x7f30680cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:14.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.037+0000 7f306dc5e700 1 --2- 172.21.15.163:0/1801801802 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3068159c60 0x7f30680b4d00 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:14.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.037+0000 7f306dc5e700 1 -- 172.21.15.163:0/1801801802 >> 172.21.15.163:0/1801801802 conn(0x7f3068068e20 msgr2=0x7f306815c330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:14.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.037+0000 7f306dc5e700 1 -- 172.21.15.163:0/1801801802 shutdown_connections 2023-10-09T06:42:14.038 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.037+0000 7f306dc5e700 1 -- 172.21.15.163:0/1801801802 wait complete. 2023-10-09T06:42:14.039 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:42:14.049 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":546,"stamp":"2023-10-09T06:42:12.468249+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2432236,"kb_used_data":21164,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752542484,"statfs":{"total":773094113280,"available":770603503616,"internally_reserved":0,"allocated":21671936,"data_stored":10378971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4067,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-49,"num_write_kb":-18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001511"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":312,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-09T06:42:08.646970+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:42:08.646970+0000","last_peered":"2023-10-09T06:42:08.646970+0000","last_clean":"2023-10-09T06:42:08.646970+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:42:08.646970+0000","last_undegraded":"2023-10-09T06:42:08.646970+0000","last_fullsized":"2023-10-09T06:42:08.646970+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574992,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304464,"kb_used_data":3080,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067376,"statfs":{"total":96636764160,"available":96324993024,"internally_reserved":0,"allocated":3153920,"data_stored":1740423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58199999999999996,"15min":0.53100000000000003},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.70599999999999996,"5min":3.778,"15min":3.778},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54300000000000004,"15min":0.52200000000000002},"min":{"1min":0.38400000000000001,"5min":0.33400000000000002,"15min":0.29699999999999999},"max":{"1min":0.77100000000000002,"5min":1.768,"15min":1.768},"last":0.50900000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53400000000000003,"15min":0.52200000000000002},"min":{"1min":0.32100000000000001,"5min":0.31900000000000001,"15min":0.27100000000000002},"max":{"1min":0.72999999999999998,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.38},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.53700000000000003},"min":{"1min":0.435,"5min":0.378,"15min":0.29699999999999999},"max":{"1min":0.68100000000000005,"5min":1.806,"15min":1.806},"last":0.40300000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59099999999999997,"15min":0.54100000000000004},"min":{"1min":0.42599999999999999,"5min":0.36399999999999999,"15min":0.32200000000000001},"max":{"1min":0.76000000000000001,"5min":3.7210000000000001,"15min":3.7210000000000001},"last":0.432},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56699999999999995,"15min":0.54900000000000004},"min":{"1min":0.44600000000000001,"5min":0.35399999999999998,"15min":0.315},"max":{"1min":0.69799999999999995,"5min":1.51,"15min":1.51},"last":0.47899999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60799999999999998,"15min":0.56699999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.33500000000000002},"max":{"1min":0.73299999999999998,"5min":3.8090000000000002,"15min":3.8090000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.47499999999999998,"5min":0.38400000000000001,"15min":0.36499999999999999},"max":{"1min":0.74299999999999999,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.442}]},{"osd":4,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.505,"15min":0.50600000000000001},"min":{"1min":0.31,"5min":0.25600000000000001,"15min":0.24399999999999999},"max":{"1min":0.55100000000000005,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.628,"15min":0.60599999999999998},"min":{"1min":0.35599999999999998,"5min":0.35199999999999998,"15min":0.30299999999999999},"max":{"1min":0.751,"5min":4.8239999999999998,"15min":4.8239999999999998},"last":0.53300000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53800000000000003,"15min":0.55100000000000005},"min":{"1min":0.34599999999999997,"5min":0.313,"15min":0.307},"max":{"1min":0.64800000000000002,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.63200000000000001,"15min":0.61799999999999999},"min":{"1min":0.38900000000000001,"5min":0.34300000000000003,"15min":0.34200000000000003},"max":{"1min":0.71699999999999997,"5min":3.6909999999999998,"15min":3.6909999999999998},"last":0.61599999999999999}]},{"osd":6,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.59499999999999997,"15min":0.58699999999999997},"min":{"1min":0.39100000000000001,"5min":0.318,"15min":0.28699999999999998},"max":{"1min":0.66600000000000004,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.63400000000000001,"15min":0.60999999999999999},"min":{"1min":0.375,"5min":0.33000000000000002,"15min":0.315},"max":{"1min":0.72499999999999998,"5min":4.7789999999999999,"15min":4.7789999999999999},"last":0.56999999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574991,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303884,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067956,"statfs":{"total":96636764160,"available":96325586944,"internally_reserved":0,"allocated":2560000,"data_stored":1150055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54200000000000004,"15min":0.51600000000000001},"min":{"1min":0.40600000000000003,"5min":0.35499999999999998,"15min":0.30399999999999999},"max":{"1min":0.75700000000000001,"5min":0.80500000000000005,"15min":0.82299999999999995},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55200000000000005,"15min":0.52300000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.79000000000000004,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.41099999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52900000000000003,"15min":0.51600000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.73499999999999999,"5min":0.74199999999999999,"15min":1.1359999999999999},"last":0.433},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.56899999999999995,"15min":0.54200000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.28799999999999998},"max":{"1min":0.754,"5min":0.78700000000000003,"15min":1.1539999999999999},"last":0.35799999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.55900000000000005,"15min":0.53000000000000003},"min":{"1min":0.45400000000000001,"5min":0.34300000000000003,"15min":0.29899999999999999},"max":{"1min":0.74199999999999999,"5min":0.77000000000000002,"15min":0.78400000000000003},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54400000000000004,"15min":0.53200000000000003},"min":{"1min":0.34899999999999998,"5min":0.34499999999999997,"15min":0.313},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.48299999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56499999999999995,"15min":0.55000000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.75700000000000001,"5min":0.79000000000000004,"15min":0.86799999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.58799999999999997,"15min":0.63700000000000001},"min":{"1min":0.49299999999999999,"5min":0.371,"15min":0.34300000000000003},"max":{"1min":0.71699999999999997,"5min":0.76900000000000002,"15min":4.7510000000000003},"last":0.51500000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47099999999999997,"15min":0.47399999999999998},"min":{"1min":0.30299999999999999,"5min":0.28999999999999998,"15min":0.27900000000000003},"max":{"1min":0.69699999999999995,"5min":0.74299999999999999,"15min":0.76400000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59099999999999997,"15min":0.58299999999999996},"min":{"1min":0.435,"5min":0.38700000000000001,"15min":0.25800000000000001},"max":{"1min":0.77700000000000002,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.53000000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.36399999999999999,"5min":0.31900000000000001,"15min":0.246},"max":{"1min":0.75,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.41999999999999998,"5min":0.40999999999999998,"15min":0.35699999999999998},"max":{"1min":0.73399999999999999,"5min":0.76200000000000001,"15min":0.86599999999999999},"last":0.625}]},{"osd":7,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56000000000000005,"15min":0.57199999999999995},"min":{"1min":0.39100000000000001,"5min":0.33400000000000002,"15min":0.29899999999999999},"max":{"1min":0.74199999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.499},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59599999999999997,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.38400000000000001,"15min":0.33000000000000002},"max":{"1min":0.72099999999999997,"5min":0.84399999999999997,"15min":0.88300000000000001},"last":0.57299999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574994,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303884,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067956,"statfs":{"total":96636764160,"available":96325586944,"internally_reserved":0,"allocated":2560000,"data_stored":1150055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.32800000000000001,"15min":0.316},"min":{"1min":0.26900000000000002,"5min":0.192,"15min":0.192},"max":{"1min":0.50900000000000001,"5min":0.55600000000000005,"15min":0.80700000000000005},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.432,"5min":0.38100000000000001,"15min":0.36399999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.21199999999999999},"max":{"1min":0.54600000000000004,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.47299999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.36099999999999999,"15min":0.35399999999999998},"min":{"1min":0.317,"5min":0.215,"15min":0.20399999999999999},"max":{"1min":0.53100000000000003,"5min":0.53700000000000003,"15min":0.68300000000000005},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.40899999999999997,"15min":0.39300000000000002},"min":{"1min":0.36799999999999999,"5min":0.25800000000000001,"15min":0.251},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.438}]},{"osd":3,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.39800000000000002,"15min":0.376},"min":{"1min":0.32600000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.375,"15min":0.46200000000000002},"min":{"1min":0.28599999999999998,"5min":0.23400000000000001,"15min":0.22700000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":5.8529999999999998},"last":0.39800000000000002}]},{"osd":4,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.59999999999999998,"15min":0.58499999999999996},"min":{"1min":0.45300000000000001,"5min":0.36199999999999999,"15min":0.32200000000000001},"max":{"1min":0.79800000000000004,"5min":1.01,"15min":1.01},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61799999999999999,"15min":0.59799999999999998},"min":{"1min":0.47299999999999998,"5min":0.40000000000000002,"15min":0.37},"max":{"1min":0.79300000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.67200000000000004}]},{"osd":5,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.59799999999999998,"15min":0.67500000000000004},"min":{"1min":0.40899999999999997,"5min":0.36499999999999999,"15min":0.34599999999999997},"max":{"1min":0.85599999999999998,"5min":0.92500000000000004,"15min":5.9459999999999997},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.621,"15min":0.61399999999999999},"min":{"1min":0.55700000000000005,"5min":0.38100000000000001,"15min":0.34300000000000003},"max":{"1min":0.82599999999999996,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.60599999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.57799999999999996,"15min":0.65000000000000002},"min":{"1min":0.44600000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":5.9160000000000004},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.51700000000000002,"5min":0.379,"15min":0.35699999999999998},"max":{"1min":0.86699999999999999,"5min":0.88600000000000001,"15min":0.91400000000000003},"last":0.68500000000000005}]},{"osd":7,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.51400000000000001,"5min":0.39000000000000001,"15min":0.379},"max":{"1min":0.76800000000000002,"5min":0.91000000000000003,"15min":0.97099999999999997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.59099999999999997,"15min":0.67600000000000005},"min":{"1min":0.48399999999999999,"5min":0.41399999999999998,"15min":0.38100000000000001},"max":{"1min":0.83399999999999996,"5min":0.86099999999999999,"15min":5.9729999999999999},"last":0.625}]}]},{"osd":0,"up_from":13,"seq":55834574993,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304464,"kb_used_data":3080,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067376,"statfs":{"total":96636764160,"available":96324993024,"internally_reserved":0,"allocated":3153920,"data_stored":1740423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.34000000000000002,"15min":0.32200000000000001},"min":{"1min":0.252,"5min":0.216,"15min":0.21299999999999999},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.35799999999999998,"15min":0.34699999999999998},"min":{"1min":0.24399999999999999,"5min":0.222,"15min":0.214},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.94499999999999995},"last":0.315}]},{"osd":2,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.38300000000000001,"15min":0.36799999999999999},"min":{"1min":0.27100000000000002,"5min":0.22,"15min":0.20200000000000001},"max":{"1min":0.53000000000000003,"5min":0.54000000000000004,"15min":0.75600000000000001},"last":0.247},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.41399999999999998,"15min":0.39500000000000002},"min":{"1min":0.34599999999999997,"5min":0.29099999999999998,"15min":0.28000000000000003},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.91700000000000004},"last":0.35199999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.36499999999999999,"15min":0.372},"min":{"1min":0.318,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":1.554},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.40200000000000002,"15min":0.39500000000000002},"min":{"1min":0.32400000000000001,"5min":0.27700000000000002,"15min":0.25800000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.79000000000000004},"last":0.41899999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58699999999999997,"15min":0.56299999999999994},"min":{"1min":0.46300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85699999999999998,"5min":0.92600000000000005,"15min":1.212},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58699999999999997,"15min":0.65700000000000003},"min":{"1min":0.43099999999999999,"5min":0.35799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85599999999999998,"5min":0.90800000000000003,"15min":5.484},"last":0.59199999999999997}]},{"osd":5,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58499999999999996,"15min":0.65900000000000003},"min":{"1min":0.437,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83099999999999996,"5min":0.97399999999999998,"15min":5.5449999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.47499999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.875,"5min":0.998,"15min":1.113},"last":0.63800000000000001}]},{"osd":6,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.42399999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86899999999999999,"5min":0.871,"15min":1.0529999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.627,"15min":0.61399999999999999},"min":{"1min":0.505,"5min":0.42899999999999999,"15min":0.41599999999999998},"max":{"1min":0.88400000000000001,"5min":0.96099999999999997,"15min":1.169},"last":0.61899999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60399999999999998,"15min":0.59399999999999997},"min":{"1min":0.47299999999999998,"5min":0.40200000000000002,"15min":0.371},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":1.2390000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59899999999999998,"15min":0.67800000000000005},"min":{"1min":0.48399999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83199999999999996,"5min":0.94299999999999995,"15min":5.5800000000000001},"last":0.55300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574994,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303888,"kb_used_data":2504,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067952,"statfs":{"total":96636764160,"available":96325582848,"internally_reserved":0,"allocated":2564096,"data_stored":1147850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33400000000000002,"15min":0.34000000000000002},"min":{"1min":0.252,"5min":0.20000000000000001,"15min":0.19900000000000001},"max":{"1min":0.51800000000000002,"5min":0.66000000000000003,"15min":1.4199999999999999},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.34499999999999997,"15min":0.46100000000000002},"min":{"1min":0.28000000000000003,"5min":0.221,"15min":0.19900000000000001},"max":{"1min":0.53100000000000003,"5min":0.55600000000000005,"15min":7.2699999999999996},"last":0.22900000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.34200000000000003,"15min":0.34100000000000003},"min":{"1min":0.26200000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.497,"5min":0.56399999999999995,"15min":0.65400000000000003},"last":0.21199999999999999},{"interface":"front","average":{"1min":0.434,"5min":0.39600000000000002,"15min":0.51400000000000001},"min":{"1min":0.33500000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.56799999999999995,"5min":0.60399999999999998,"15min":7.327},"last":0.27800000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.41099999999999998,"15min":0.39400000000000002},"min":{"1min":0.39600000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.56899999999999995,"5min":0.64600000000000002,"15min":0.68600000000000005},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.39000000000000001,"15min":0.38700000000000001},"min":{"1min":0.32600000000000001,"5min":0.252,"15min":0.23999999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.72699999999999998},"last":0.311}]},{"osd":4,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56299999999999994,"15min":0.54800000000000004},"min":{"1min":0.45800000000000002,"5min":0.33600000000000002,"15min":0.317},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.58199999999999996,"15min":0.68999999999999995},"min":{"1min":0.46899999999999997,"5min":0.35999999999999999,"15min":0.33600000000000002},"max":{"1min":0.82599999999999996,"5min":0.91100000000000003,"15min":7.2389999999999999},"last":0.54600000000000004}]},{"osd":5,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59499999999999997,"15min":0.77300000000000002},"min":{"1min":0.495,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.872,"5min":0.89800000000000002,"15min":10.890000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.60899999999999999,"15min":0.60299999999999998},"min":{"1min":0.45600000000000002,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":0.80000000000000004,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.61699999999999999}]},{"osd":6,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58099999999999996,"15min":0.69599999999999995},"min":{"1min":0.50800000000000001,"5min":0.35999999999999999,"15min":0.33500000000000002},"max":{"1min":0.77600000000000002,"5min":0.82499999999999996,"15min":7.2999999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60499999999999998,"15min":0.58299999999999996},"min":{"1min":0.44700000000000001,"5min":0.40300000000000002,"15min":0.377},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.749}]},{"osd":7,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.61099999999999999,"15min":0.72599999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.37},"max":{"1min":0.80100000000000005,"5min":0.91300000000000003,"15min":7.1879999999999997},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63,"15min":0.624},"min":{"1min":0.53300000000000003,"5min":0.437,"15min":0.38},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.70199999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574993,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303884,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067956,"statfs":{"total":96636764160,"available":96325586944,"internally_reserved":0,"allocated":2560000,"data_stored":1150055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.34200000000000003,"15min":0.33800000000000002},"min":{"1min":0.23799999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.55400000000000005,"5min":0.61699999999999999,"15min":5.8700000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.36299999999999999,"15min":0.36799999999999999},"min":{"1min":0.255,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.55400000000000005,"5min":0.58499999999999996,"15min":5.6769999999999996},"last":0.28799999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.36599999999999999,"15min":0.36399999999999999},"min":{"1min":0.28599999999999998,"5min":0.21299999999999999,"15min":0.21199999999999999},"max":{"1min":0.46700000000000003,"5min":0.61199999999999999,"15min":4.0469999999999997},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.40200000000000002,"15min":0.39500000000000002},"min":{"1min":0.32900000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.60199999999999998,"5min":0.68600000000000005,"15min":4.1509999999999998},"last":0.31900000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.371,"15min":0.373},"min":{"1min":0.27300000000000002,"5min":0.23200000000000001,"15min":0.222},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":3.1139999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.39500000000000002,"15min":0.38900000000000001},"min":{"1min":0.28799999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.54700000000000004,"5min":0.66900000000000004,"15min":3.3180000000000001},"last":0.34699999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56999999999999995,"15min":0.57299999999999995},"min":{"1min":0.42799999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.75700000000000001,"5min":0.86699999999999999,"15min":3.0899999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.432,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79300000000000004,"5min":0.84499999999999997,"15min":3.0710000000000002},"last":0.60499999999999998}]},{"osd":5,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.58099999999999996,"15min":0.59099999999999997},"min":{"1min":0.45300000000000001,"5min":0.33600000000000002,"15min":0.32700000000000001},"max":{"1min":0.86399999999999999,"5min":0.877,"15min":6.2300000000000004},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.58699999999999997,"15min":0.59699999999999998},"min":{"1min":0.498,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78800000000000003,"5min":0.83599999999999997,"15min":3.0219999999999998},"last":0.59499999999999997}]},{"osd":6,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.61899999999999999,"15min":0.60499999999999998},"min":{"1min":0.47899999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.147,"5min":1.147,"15min":2.6850000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.65000000000000002,"15min":0.65700000000000003},"min":{"1min":0.5,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":6.5970000000000004},"last":0.68999999999999995}]},{"osd":7,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.61199999999999999,"15min":0.61699999999999999},"min":{"1min":0.46600000000000003,"5min":0.41499999999999998,"15min":0.38200000000000001},"max":{"1min":0.81799999999999995,"5min":0.86099999999999999,"15min":4.2629999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60799999999999998,"15min":0.61899999999999999},"min":{"1min":0.47899999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.89200000000000002,"5min":0.89900000000000002,"15min":7.3650000000000002},"last":0.57499999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574992,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303884,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067956,"statfs":{"total":96636764160,"available":96325586944,"internally_reserved":0,"allocated":2560000,"data_stored":1150055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54800000000000004,"15min":0.54500000000000004},"min":{"1min":0.38900000000000001,"5min":0.307,"15min":0.27000000000000002},"max":{"1min":0.71899999999999997,"5min":0.88900000000000001,"15min":1.083},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.55200000000000005,"15min":0.53000000000000003},"min":{"1min":0.50800000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":1.054},"last":0.58599999999999997}]},{"osd":1,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56599999999999995,"15min":0.55100000000000005},"min":{"1min":0.52400000000000002,"5min":0.28399999999999997,"15min":0.27100000000000002},"max":{"1min":0.77300000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53100000000000003,"15min":0.50800000000000001},"min":{"1min":0.38500000000000001,"5min":0.33800000000000002,"15min":0.31},"max":{"1min":0.75,"5min":0.76300000000000001,"15min":0.79100000000000004},"last":0.78500000000000003}]},{"osd":2,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.54700000000000004,"15min":0.54000000000000004},"min":{"1min":0.443,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76500000000000001,"5min":0.84899999999999998,"15min":1.0249999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.499,"5min":0.38100000000000001,"15min":0.36299999999999999},"max":{"1min":0.73399999999999999,"5min":0.90500000000000003,"15min":0.93799999999999994},"last":0.748}]},{"osd":3,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55300000000000005,"15min":0.54800000000000004},"min":{"1min":0.40699999999999997,"5min":0.33300000000000002,"15min":0.29199999999999998},"max":{"1min":0.747,"5min":0.86899999999999999,"15min":0.98499999999999999},"last":0.62},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.47799999999999998,"5min":0.38700000000000001,"15min":0.34100000000000003},"max":{"1min":0.68200000000000005,"5min":0.92900000000000005,"15min":1.1319999999999999},"last":0.68300000000000005}]},{"osd":5,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.50600000000000001,"15min":0.53300000000000003},"min":{"1min":0.35499999999999998,"5min":0.307,"15min":0.27300000000000002},"max":{"1min":0.55900000000000005,"5min":0.83299999999999996,"15min":0.998},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57699999999999996,"15min":0.59399999999999997},"min":{"1min":0.40000000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71999999999999997,"5min":0.95599999999999996,"15min":0.97999999999999998},"last":0.50900000000000001}]},{"osd":6,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.54600000000000004,"15min":0.55600000000000005},"min":{"1min":0.36899999999999999,"5min":0.35499999999999998,"15min":0.29999999999999999},"max":{"1min":0.67500000000000004,"5min":0.86299999999999999,"15min":1.0189999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.56999999999999995,"15min":0.58399999999999996},"min":{"1min":0.38900000000000001,"5min":0.36399999999999999,"15min":0.32700000000000001},"max":{"1min":0.72699999999999998,"5min":0.85599999999999998,"15min":1.01},"last":0.79900000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55700000000000005,"15min":0.56200000000000006},"min":{"1min":0.42399999999999999,"5min":0.35299999999999998,"15min":0.28299999999999997},"max":{"1min":0.65400000000000003,"5min":0.84399999999999997,"15min":0.90000000000000002},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.60599999999999998,"15min":0.625},"min":{"1min":0.374,"5min":0.34999999999999998,"15min":0.32200000000000001},"max":{"1min":0.71299999999999997,"5min":0.93799999999999994,"15min":1.175},"last":0.63600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574992,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303884,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067956,"statfs":{"total":96636764160,"available":96325586944,"internally_reserved":0,"allocated":2560000,"data_stored":1150055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54200000000000004,"15min":0.51700000000000002},"min":{"1min":0.36399999999999999,"5min":0.29499999999999998,"15min":0.28599999999999998},"max":{"1min":0.76400000000000001,"5min":0.79700000000000004,"15min":0.86599999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55400000000000005,"15min":0.52800000000000002},"min":{"1min":0.45000000000000001,"5min":0.36399999999999999,"15min":0.28799999999999998},"max":{"1min":0.83599999999999997,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.63100000000000001}]},{"osd":1,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52700000000000002,"15min":0.51500000000000001},"min":{"1min":0.38800000000000001,"5min":0.34100000000000003,"15min":0.311},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.54400000000000004,"15min":0.53400000000000003},"min":{"1min":0.41499999999999998,"5min":0.36899999999999999,"15min":0.30199999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.81599999999999995},"last":0.40899999999999997}]},{"osd":2,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58199999999999996,"15min":0.55800000000000005},"min":{"1min":0.39300000000000002,"5min":0.34699999999999998,"15min":0.28199999999999997},"max":{"1min":0.79800000000000004,"5min":1.139,"15min":1.139},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.52600000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78100000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.56899999999999995}]},{"osd":3,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57099999999999995,"15min":0.55200000000000005},"min":{"1min":0.43099999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.75,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.45900000000000002,"5min":0.40699999999999997,"15min":0.312},"max":{"1min":0.80200000000000005,"5min":0.80600000000000005,"15min":0.81100000000000005},"last":0.48099999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.33600000000000002,"5min":0.28399999999999997,"15min":0.26700000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53600000000000003,"15min":0.54200000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.27700000000000002},"max":{"1min":0.751,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.501}]},{"osd":6,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.35599999999999998,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":0.73499999999999999,"5min":1.752,"15min":1.752},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55600000000000005,"15min":0.55900000000000005},"min":{"1min":0.39100000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.377}]},{"osd":7,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56399999999999995,"15min":0.57599999999999996},"min":{"1min":0.375,"5min":0.311,"15min":0.311},"max":{"1min":0.79900000000000004,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58599999999999997,"15min":0.60299999999999998},"min":{"1min":0.40799999999999997,"5min":0.372,"15min":0.314},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.90600000000000003},"last":0.52000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:42:14.051 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-09T06:42:14.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.164+0000 7ff25f59e700 1 Processor -- start 2023-10-09T06:42:14.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.164+0000 7ff25f59e700 1 -- start start 2023-10-09T06:42:14.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.165+0000 7ff25f59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff260159420 0x7ff2600b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:14.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.165+0000 7ff25f59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2600b2980 con 0x7ff2601597f0 2023-10-09T06:42:14.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.165+0000 7ff25f59e700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2600b11d0 con 0x7ff26015d230 2023-10-09T06:42:14.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.165+0000 7ff25f59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2600b14b0 con 0x7ff260159420 2023-10-09T06:42:14.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.165+0000 7ff255d9b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff2601597f0 0x7ff2600b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60598/0 (socket says 172.21.15.163:60598) 2023-10-09T06:42:14.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.165+0000 7ff255d9b700 1 -- 172.21.15.163:0/1126174666 learned_addr learned my addr 172.21.15.163:0/1126174666 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:42:14.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.165+0000 7ff25dd9b700 1 -- 172.21.15.163:0/1126174666 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3533841703 0 0) 0x7ff2600b2980 con 0x7ff2601597f0 2023-10-09T06:42:14.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.165+0000 7ff25dd9b700 1 -- 172.21.15.163:0/1126174666 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff24c003570 con 0x7ff2601597f0 2023-10-09T06:42:14.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.165+0000 7ff25e59c700 1 --2- 172.21.15.163:0/1126174666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff260159420 0x7ff2600b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:14.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.165+0000 7ff25dd9b700 1 -- 172.21.15.163:0/1126174666 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1551081752 0 0) 0x7ff24c003570 con 0x7ff2601597f0 2023-10-09T06:42:14.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.165+0000 7ff25dd9b700 1 -- 172.21.15.163:0/1126174666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff260159420 msgr2=0x7ff2600b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:14.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.165+0000 7ff25dd9b700 1 --2- 172.21.15.163:0/1126174666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff260159420 0x7ff2600b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:14.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.165+0000 7ff25dd9b700 1 -- 172.21.15.163:0/1126174666 >> v1:172.21.15.181:6789/0 conn(0x7ff26015d230 legacy=0x7ff2600b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:14.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.165+0000 7ff25dd9b700 1 -- 172.21.15.163:0/1126174666 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2600b1790 con 0x7ff2601597f0 2023-10-09T06:42:14.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.166+0000 7ff25dd9b700 1 -- 172.21.15.163:0/1126174666 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7ff248001470 con 0x7ff2601597f0 2023-10-09T06:42:14.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.166+0000 7ff25e59c700 1 --2- 172.21.15.163:0/1126174666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff260159420 0x7ff2600b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:42:14.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.166+0000 7ff25dd9b700 1 -- 172.21.15.163:0/1126174666 >> v1:172.21.15.163:6789/0 conn(0x7ff2601597f0 legacy=0x7ff2600b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:14.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.166+0000 7ff25dd9b700 1 --2- 172.21.15.163:0/1126174666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff260159420 0x7ff24c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:14.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.166+0000 7ff25dd9b700 1 --2- 172.21.15.163:0/1126174666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff24c004080 0x7ff24c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:14.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.166+0000 7ff25dd9b700 1 --2- 172.21.15.163:0/1126174666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff24c004d50 0x7ff24c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:14.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.166+0000 7ff25dd9b700 1 -- 172.21.15.163:0/1126174666 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2600b14b0 con 0x7ff24c004080 2023-10-09T06:42:14.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.166+0000 7ff25dd9b700 1 -- 172.21.15.163:0/1126174666 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c00bca0 con 0x7ff24c004d50 2023-10-09T06:42:14.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.166+0000 7ff25dd9b700 1 -- 172.21.15.163:0/1126174666 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c00be60 con 0x7ff260159420 2023-10-09T06:42:14.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.166+0000 7ff25e59c700 1 --2- 172.21.15.163:0/1126174666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff260159420 0x7ff24c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:14.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.166+0000 7ff255d9b700 1 --2- 172.21.15.163:0/1126174666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff24c004080 0x7ff24c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:14.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.166+0000 7ff25ed9d700 1 --2- 172.21.15.163:0/1126174666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff24c004d50 0x7ff24c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:14.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.166+0000 7ff255d9b700 1 -- 172.21.15.163:0/1126174666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff260159420 msgr2=0x7ff24c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:14.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.166+0000 7ff255d9b700 1 --2- 172.21.15.163:0/1126174666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff260159420 0x7ff24c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:14.173 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.166+0000 7ff255d9b700 1 -- 172.21.15.163:0/1126174666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff24c004d50 msgr2=0x7ff24c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:14.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.166+0000 7ff255d9b700 1 --2- 172.21.15.163:0/1126174666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff24c004d50 0x7ff24c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:14.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.166+0000 7ff255d9b700 1 -- 172.21.15.163:0/1126174666 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24c003570 con 0x7ff24c004080 2023-10-09T06:42:14.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.166+0000 7ff25e59c700 1 --2- 172.21.15.163:0/1126174666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff260159420 0x7ff24c003a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:42:14.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.166+0000 7ff255d9b700 1 --2- 172.21.15.163:0/1126174666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff24c004080 0x7ff24c004810 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7ff24800c390 tx=0x7ff248001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cfa40dbef88b990 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:14.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.166+0000 7ff25dd9b700 1 -- 172.21.15.163:0/1126174666 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24800cc60 con 0x7ff24c004080 2023-10-09T06:42:14.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.166+0000 7ff25dd9b700 1 -- 172.21.15.163:0/1126174666 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff24800cdc0 con 0x7ff24c004080 2023-10-09T06:42:14.174 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.166+0000 7ff25dd9b700 1 -- 172.21.15.163:0/1126174666 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff248005b50 con 0x7ff24c004080 2023-10-09T06:42:14.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.167+0000 7ff25f59e700 1 -- 172.21.15.163:0/1126174666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff24c004080 msgr2=0x7ff24c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:14.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.167+0000 7ff25f59e700 1 --2- 172.21.15.163:0/1126174666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff24c004080 0x7ff24c004810 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7ff24800c390 tx=0x7ff248001e50 comp rx=0 tx=0).stop 2023-10-09T06:42:14.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.167+0000 7ff25ed9d700 1 -- 172.21.15.163:0/1126174666 reap_dead start 2023-10-09T06:42:14.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.167+0000 7ff25f59e700 1 -- 172.21.15.163:0/1126174666 shutdown_connections 2023-10-09T06:42:14.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.167+0000 7ff25f59e700 1 -- 172.21.15.163:0/1126174666 >> 172.21.15.163:0/1126174666 conn(0x7ff2600d0660 msgr2=0x7ff2600d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:14.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.167+0000 7ff25f59e700 1 -- 172.21.15.163:0/1126174666 shutdown_connections 2023-10-09T06:42:14.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.167+0000 7ff25f59e700 1 -- 172.21.15.163:0/1126174666 wait complete. 2023-10-09T06:42:14.175 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.168+0000 7ff25f59e700 1 Processor -- start 2023-10-09T06:42:14.176 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.168+0000 7ff25f59e700 1 -- start start 2023-10-09T06:42:14.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.169+0000 7ff25f59e700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff24c004080 0x7ff26015c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:14.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.169+0000 7ff25f59e700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff260150390 0x7ff2600cf480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:14.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.169+0000 7ff25f59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff26014ccd0 0x7ff26014d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:14.178 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.169+0000 7ff25f59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff260151040 con 0x7ff26014ccd0 2023-10-09T06:42:14.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.169+0000 7ff25f59e700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26014d680 con 0x7ff24c004080 2023-10-09T06:42:14.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.169+0000 7ff25f59e700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26016aa30 con 0x7ff260150390 2023-10-09T06:42:14.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.169+0000 7ff25ed9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff26014ccd0 0x7ff26014d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:14.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.169+0000 7ff25ed9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff26014ccd0 0x7ff26014d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55060/0 (socket says 172.21.15.163:55060) 2023-10-09T06:42:14.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.169+0000 7ff25ed9d700 1 -- 172.21.15.163:0/1706735637 learned_addr learned my addr 172.21.15.163:0/1706735637 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:14.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.169+0000 7ff255d9b700 1 --2- 172.21.15.163:0/1706735637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff260150390 0x7ff2600cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:14.179 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.169+0000 7ff25e59c700 1 --2- 172.21.15.163:0/1706735637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff24c004080 0x7ff26015c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:14.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.169+0000 7ff25ed9d700 1 -- 172.21.15.163:0/1706735637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff260150390 msgr2=0x7ff2600cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:14.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.169+0000 7ff25ed9d700 1 --2- 172.21.15.163:0/1706735637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff260150390 0x7ff2600cf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:14.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.169+0000 7ff25ed9d700 1 -- 172.21.15.163:0/1706735637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff24c004080 msgr2=0x7ff26015c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:14.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.169+0000 7ff25ed9d700 1 --2- 172.21.15.163:0/1706735637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff24c004080 0x7ff26015c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:14.180 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.169+0000 7ff25ed9d700 1 -- 172.21.15.163:0/1706735637 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26016aec0 con 0x7ff26014ccd0 2023-10-09T06:42:14.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.169+0000 7ff25e59c700 1 --2- 172.21.15.163:0/1706735637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff24c004080 0x7ff26015c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:42:14.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.169+0000 7ff25ed9d700 1 --2- 172.21.15.163:0/1706735637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff26014ccd0 0x7ff26014d140 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7ff2600b3180 tx=0x7ff258001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:14.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.170+0000 7ff256ffd700 1 -- 172.21.15.163:0/1706735637 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff258019070 con 0x7ff26014ccd0 2023-10-09T06:42:14.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.170+0000 7ff256ffd700 1 -- 172.21.15.163:0/1706735637 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff258006ba0 con 0x7ff26014ccd0 2023-10-09T06:42:14.181 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.170+0000 7ff25f59e700 1 -- 172.21.15.163:0/1706735637 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff26016a100 con 0x7ff26014ccd0 2023-10-09T06:42:14.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.170+0000 7ff256ffd700 1 -- 172.21.15.163:0/1706735637 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25800c7b0 con 0x7ff26014ccd0 2023-10-09T06:42:14.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.170+0000 7ff25f59e700 1 -- 172.21.15.163:0/1706735637 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26016a6c0 con 0x7ff26014ccd0 2023-10-09T06:42:14.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.170+0000 7ff25f59e700 1 -- 172.21.15.163:0/1706735637 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff26004c3c0 con 0x7ff26014ccd0 2023-10-09T06:42:14.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.171+0000 7ff256ffd700 1 -- 172.21.15.163:0/1706735637 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff25800cb60 con 0x7ff26014ccd0 2023-10-09T06:42:14.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.171+0000 7ff256ffd700 1 --2- 172.21.15.163:0/1706735637 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff240067350 0x7ff240069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:14.182 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.172+0000 7ff256ffd700 1 -- 172.21.15.163:0/1706735637 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff258084580 con 0x7ff26014ccd0 2023-10-09T06:42:14.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.172+0000 7ff25e59c700 1 --2- 172.21.15.163:0/1706735637 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff240067350 0x7ff240069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:14.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.172+0000 7ff25e59c700 1 --2- 172.21.15.163:0/1706735637 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff240067350 0x7ff240069800 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7ff250006fd0 tx=0x7ff250008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:14.183 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.176+0000 7ff256ffd700 1 -- 172.21.15.163:0/1706735637 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff2580514d0 con 0x7ff26014ccd0 2023-10-09T06:42:14.362 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.361+0000 7ff25f59e700 1 -- 172.21.15.163:0/1706735637 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff2600b1a70 con 0x7ff26014ccd0 2023-10-09T06:42:14.363 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.362+0000 7ff256ffd700 1 -- 172.21.15.163:0/1706735637 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v202) v1 ==== 74+0+12323 (secure 0 0 0) 0x7ff258054af0 con 0x7ff26014ccd0 2023-10-09T06:42:14.363 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:42:14.363 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":202,"fsid":"2520b8f7-539f-4bae-9f17-1fce66f66ddd","created":"2023-10-09T06:30:18.715024+0000","modified":"2023-10-09T06:42:10.791998+0000","last_up_change":"2023-10-09T06:30:29.513823+0000","last_in_change":"2023-10-09T06:30:21.097413+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-09T06:30:30.334762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"51bfc982-10fe-4bfd-9541-07653c290e84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":90867},{"type":"v1","addr":"172.21.15.163:6825","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":90867},{"type":"v1","addr":"172.21.15.163:6827","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":90867},{"type":"v1","addr":"172.21.15.163:6831","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":90867},{"type":"v1","addr":"172.21.15.163:6829","nonce":90867}]},"public_addr":"172.21.15.163:6825/90867","cluster_addr":"172.21.15.163:6827/90867","heartbeat_back_addr":"172.21.15.163:6831/90867","heartbeat_front_addr":"172.21.15.163:6829/90867","state":["exists","up"]},{"osd":1,"uuid":"ae88b544-638e-469d-87a3-9646eed2b725","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":90868},{"type":"v1","addr":"172.21.15.163:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":90868},{"type":"v1","addr":"172.21.15.163:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6817","nonce":90868},{"type":"v1","addr":"172.21.15.163:6819","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6813","nonce":90868},{"type":"v1","addr":"172.21.15.163:6815","nonce":90868}]},"public_addr":"172.21.15.163:6809/90868","cluster_addr":"172.21.15.163:6811/90868","heartbeat_back_addr":"172.21.15.163:6819/90868","heartbeat_front_addr":"172.21.15.163:6815/90868","state":["exists","up"]},{"osd":2,"uuid":"e5d77274-5154-4ca7-8201-2cbf958775b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":90866},{"type":"v1","addr":"172.21.15.163:6801","nonce":90866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":90866},{"type":"v1","addr":"172.21.15.163:6803","nonce":90866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":90866},{"type":"v1","addr":"172.21.15.163:6807","nonce":90866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":90866},{"type":"v1","addr":"172.21.15.163:6805","nonce":90866}]},"public_addr":"172.21.15.163:6801/90866","cluster_addr":"172.21.15.163:6803/90866","heartbeat_back_addr":"172.21.15.163:6807/90866","heartbeat_front_addr":"172.21.15.163:6805/90866","state":["exists","up"]},{"osd":3,"uuid":"9cd7d3f6-562c-4c52-b3ad-ea5f37031a7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":90865},{"type":"v1","addr":"172.21.15.163:6814","nonce":90865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":90865},{"type":"v1","addr":"172.21.15.163:6818","nonce":90865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":90865},{"type":"v1","addr":"172.21.15.163:6823","nonce":90865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":90865},{"type":"v1","addr":"172.21.15.163:6821","nonce":90865}]},"public_addr":"172.21.15.163:6814/90865","cluster_addr":"172.21.15.163:6818/90865","heartbeat_back_addr":"172.21.15.163:6823/90865","heartbeat_front_addr":"172.21.15.163:6821/90865","state":["exists","up"]},{"osd":4,"uuid":"4440f08d-6323-44ad-a0db-fd7e5ddf7424","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":198,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":91143},{"type":"v1","addr":"172.21.15.181:6825","nonce":91143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":91143},{"type":"v1","addr":"172.21.15.181:6827","nonce":91143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":91143},{"type":"v1","addr":"172.21.15.181:6831","nonce":91143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":91143},{"type":"v1","addr":"172.21.15.181:6829","nonce":91143}]},"public_addr":"172.21.15.181:6825/91143","cluster_addr":"172.21.15.181:6827/91143","heartbeat_back_addr":"172.21.15.181:6831/91143","heartbeat_front_addr":"172.21.15.181:6829/91143","state":["exists","up"]},{"osd":5,"uuid":"38a04aeb-2406-4cbd-953c-f7a762102715","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":91130},{"type":"v1","addr":"172.21.15.181:6801","nonce":91130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":91130},{"type":"v1","addr":"172.21.15.181:6803","nonce":91130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":91130},{"type":"v1","addr":"172.21.15.181:6807","nonce":91130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":91130},{"type":"v1","addr":"172.21.15.181:6805","nonce":91130}]},"public_addr":"172.21.15.181:6801/91130","cluster_addr":"172.21.15.181:6803/91130","heartbeat_back_addr":"172.21.15.181:6807/91130","heartbeat_front_addr":"172.21.15.181:6805/91130","state":["exists","up"]},{"osd":6,"uuid":"4023515b-10ff-428d-ab92-acad13005e9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":198,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":91131},{"type":"v1","addr":"172.21.15.181:6809","nonce":91131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":91131},{"type":"v1","addr":"172.21.15.181:6811","nonce":91131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":91131},{"type":"v1","addr":"172.21.15.181:6815","nonce":91131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":91131},{"type":"v1","addr":"172.21.15.181:6813","nonce":91131}]},"public_addr":"172.21.15.181:6809/91131","cluster_addr":"172.21.15.181:6811/91131","heartbeat_back_addr":"172.21.15.181:6815/91131","heartbeat_front_addr":"172.21.15.181:6813/91131","state":["exists","up"]},{"osd":7,"uuid":"82d7b464-b236-4762-b546-511dfc0ac7ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":91132},{"type":"v1","addr":"172.21.15.181:6817","nonce":91132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":91132},{"type":"v1","addr":"172.21.15.181:6819","nonce":91132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":91132},{"type":"v1","addr":"172.21.15.181:6823","nonce":91132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":91132},{"type":"v1","addr":"172.21.15.181:6821","nonce":91132}]},"public_addr":"172.21.15.181:6817/91132","cluster_addr":"172.21.15.181:6819/91132","heartbeat_back_addr":"172.21.15.181:6823/91132","heartbeat_front_addr":"172.21.15.181:6821/91132","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.307767+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.269796+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.294381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.195647+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.268272+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:25.987907+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.258358+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-09T06:30:26.203114+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6835/2564044136":"2023-10-10T06:42:07.325638+0000","172.21.15.163:6833/2564044136":"2023-10-10T06:42:07.325638+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-09T06:42:14.365 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.364+0000 7ff25f59e700 1 -- 172.21.15.163:0/1706735637 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff240067350 msgr2=0x7ff240069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:14.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.364+0000 7ff25f59e700 1 --2- 172.21.15.163:0/1706735637 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff240067350 0x7ff240069800 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7ff250006fd0 tx=0x7ff250008040 comp rx=0 tx=0).stop 2023-10-09T06:42:14.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.364+0000 7ff25f59e700 1 -- 172.21.15.163:0/1706735637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff26014ccd0 msgr2=0x7ff26014d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:14.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.364+0000 7ff25f59e700 1 --2- 172.21.15.163:0/1706735637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff26014ccd0 0x7ff26014d140 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7ff2600b3180 tx=0x7ff258001640 comp rx=0 tx=0).stop 2023-10-09T06:42:14.366 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.364+0000 7ff25f59e700 1 -- 172.21.15.163:0/1706735637 shutdown_connections 2023-10-09T06:42:14.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.364+0000 7ff25f59e700 1 --2- 172.21.15.163:0/1706735637 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff240067350 0x7ff240069800 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:14.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.364+0000 7ff25f59e700 1 --2- 172.21.15.163:0/1706735637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff26014ccd0 0x7ff26014d140 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:14.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.364+0000 7ff25f59e700 1 --2- 172.21.15.163:0/1706735637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff24c004080 0x7ff26015c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:14.367 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.364+0000 7ff25f59e700 1 --2- 172.21.15.163:0/1706735637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff260150390 0x7ff2600cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:14.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.364+0000 7ff25f59e700 1 -- 172.21.15.163:0/1706735637 >> 172.21.15.163:0/1706735637 conn(0x7ff2600d0660 msgr2=0x7ff2600b23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:14.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.365+0000 7ff25f59e700 1 -- 172.21.15.163:0/1706735637 shutdown_connections 2023-10-09T06:42:14.368 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:14.365+0000 7ff25f59e700 1 -- 172.21.15.163:0/1706735637 wait complete. 2023-10-09T06:42:14.712 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:42:14.715 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:42:15.379 INFO:tasks.ceph:Scrubbing osd.0 2023-10-09T06:42:15.379 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-09T06:42:15.500 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.499+0000 7fd82b7a9700 1 Processor -- start 2023-10-09T06:42:15.501 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.500+0000 7fd82b7a9700 1 -- start start 2023-10-09T06:42:15.502 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.500+0000 7fd82b7a9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd824159ec0 0x7fd8240b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:15.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.500+0000 7fd82b7a9700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd8240b53d0 0x7fd8240cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:15.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.500+0000 7fd82b7a9700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8240b5b00 con 0x7fd8240cce40 2023-10-09T06:42:15.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.500+0000 7fd82b7a9700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8240b5da0 con 0x7fd8240b53d0 2023-10-09T06:42:15.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.500+0000 7fd82b7a9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8240b6080 con 0x7fd824159ec0 2023-10-09T06:42:15.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.501+0000 7fd828d44700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd8240b53d0 0x7fd8240cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:15.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.501+0000 7fd829d46700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fd8240cce40 0x7fd8240cd280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60600/0 (socket says 172.21.15.163:60600) 2023-10-09T06:42:15.503 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.501+0000 7fd828d44700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd8240b53d0 0x7fd8240cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55072/0 (socket says 172.21.15.163:55072) 2023-10-09T06:42:15.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.501+0000 7fd829d46700 1 -- 172.21.15.163:0/658546952 learned_addr learned my addr 172.21.15.163:0/658546952 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:42:15.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.501+0000 7fd829545700 1 --2- 172.21.15.163:0/658546952 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd824159ec0 0x7fd8240b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:15.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.501+0000 7fd828d44700 1 -- 172.21.15.163:0/658546952 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd824159ec0 msgr2=0x7fd8240b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:15.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.501+0000 7fd828d44700 1 --2- 172.21.15.163:0/658546952 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd824159ec0 0x7fd8240b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:15.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.501+0000 7fd828d44700 1 -- 172.21.15.163:0/658546952 >> v1:172.21.15.163:6789/0 conn(0x7fd8240cce40 legacy=0x7fd8240cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:15.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.501+0000 7fd81bfff700 1 -- 172.21.15.163:0/658546952 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 558745959 0 0) 0x7fd8240b5b00 con 0x7fd8240cce40 2023-10-09T06:42:15.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.501+0000 7fd828d44700 1 -- 172.21.15.163:0/658546952 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8240cc100 con 0x7fd8240b53d0 2023-10-09T06:42:15.504 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.501+0000 7fd828d44700 1 --2- 172.21.15.163:0/658546952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8240b53d0 0x7fd8240cc900 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fd8140098e0 tx=0x7fd8140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bdfee7c85c70082 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:15.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.501+0000 7fd81bfff700 1 -- 172.21.15.163:0/658546952 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd814009e80 con 0x7fd8240b53d0 2023-10-09T06:42:15.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.501+0000 7fd81bfff700 1 -- 172.21.15.163:0/658546952 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd81400b860 con 0x7fd8240b53d0 2023-10-09T06:42:15.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.501+0000 7fd81bfff700 1 -- 172.21.15.163:0/658546952 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81400bbe0 con 0x7fd8240b53d0 2023-10-09T06:42:15.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.502+0000 7fd82b7a9700 1 -- 172.21.15.163:0/658546952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8240b53d0 msgr2=0x7fd8240cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:15.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.502+0000 7fd82b7a9700 1 --2- 172.21.15.163:0/658546952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8240b53d0 0x7fd8240cc900 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fd8140098e0 tx=0x7fd8140091a0 comp rx=0 tx=0).stop 2023-10-09T06:42:15.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.502+0000 7fd82b7a9700 1 -- 172.21.15.163:0/658546952 shutdown_connections 2023-10-09T06:42:15.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.502+0000 7fd82b7a9700 1 --2- 172.21.15.163:0/658546952 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd824159ec0 0x7fd8240b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:15.505 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.502+0000 7fd82b7a9700 1 --2- 172.21.15.163:0/658546952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8240b53d0 0x7fd8240cc900 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:15.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.502+0000 7fd82b7a9700 1 -- 172.21.15.163:0/658546952 >> 172.21.15.163:0/658546952 conn(0x7fd8240999f0 msgr2=0x7fd8240b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:15.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.502+0000 7fd82b7a9700 1 -- 172.21.15.163:0/658546952 shutdown_connections 2023-10-09T06:42:15.506 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.502+0000 7fd82b7a9700 1 -- 172.21.15.163:0/658546952 wait complete. 2023-10-09T06:42:15.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.503+0000 7fd82b7a9700 1 Processor -- start 2023-10-09T06:42:15.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.503+0000 7fd82b7a9700 1 -- start start 2023-10-09T06:42:15.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.504+0000 7fd82b7a9700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8240b53d0 0x7fd824144b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:15.509 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.504+0000 7fd82b7a9700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8240cce40 0x7fd824146890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:15.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.504+0000 7fd82b7a9700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd824159ec0 0x7fd824146dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:15.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.504+0000 7fd82b7a9700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd824147660 con 0x7fd8240b53d0 2023-10-09T06:42:15.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.504+0000 7fd82b7a9700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd824147940 con 0x7fd8240cce40 2023-10-09T06:42:15.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.504+0000 7fd82b7a9700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd824147c20 con 0x7fd824159ec0 2023-10-09T06:42:15.510 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.504+0000 7fd829545700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8240b53d0 0x7fd824144b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:15.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.504+0000 7fd829545700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8240b53d0 0x7fd824144b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55088/0 (socket says 172.21.15.163:55088) 2023-10-09T06:42:15.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.504+0000 7fd829545700 1 -- 172.21.15.163:0/1244898670 learned_addr learned my addr 172.21.15.163:0/1244898670 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:15.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.504+0000 7fd829545700 1 -- 172.21.15.163:0/1244898670 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd824159ec0 msgr2=0x7fd824146dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:15.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.504+0000 7fd829545700 1 --2- 172.21.15.163:0/1244898670 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd824159ec0 0x7fd824146dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:15.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.504+0000 7fd829545700 1 -- 172.21.15.163:0/1244898670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8240cce40 msgr2=0x7fd824146890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:15.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.504+0000 7fd829545700 1 --2- 172.21.15.163:0/1244898670 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8240cce40 0x7fd824146890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:15.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.504+0000 7fd829545700 1 -- 172.21.15.163:0/1244898670 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd814009420 con 0x7fd8240b53d0 2023-10-09T06:42:15.511 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.504+0000 7fd829545700 1 --2- 172.21.15.163:0/1244898670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8240b53d0 0x7fd824144b40 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fd82000e0e0 tx=0x7fd8200098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:15.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.504+0000 7fd8197fa700 1 -- 172.21.15.163:0/1244898670 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82000ed80 con 0x7fd8240b53d0 2023-10-09T06:42:15.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.505+0000 7fd8197fa700 1 -- 172.21.15.163:0/1244898670 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd820014420 con 0x7fd8240b53d0 2023-10-09T06:42:15.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.505+0000 7fd82b7a9700 1 -- 172.21.15.163:0/1244898670 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd824147f00 con 0x7fd8240b53d0 2023-10-09T06:42:15.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.505+0000 7fd8197fa700 1 -- 172.21.15.163:0/1244898670 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8200147e0 con 0x7fd8240b53d0 2023-10-09T06:42:15.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.505+0000 7fd82b7a9700 1 -- 172.21.15.163:0/1244898670 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd824145080 con 0x7fd8240b53d0 2023-10-09T06:42:15.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.505+0000 7fd82b7a9700 1 -- 172.21.15.163:0/1244898670 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd824063c90 con 0x7fd8240b53d0 2023-10-09T06:42:15.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.506+0000 7fd8197fa700 1 -- 172.21.15.163:0/1244898670 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd820014b50 con 0x7fd8240b53d0 2023-10-09T06:42:15.512 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.506+0000 7fd8197fa700 1 --2- 172.21.15.163:0/1244898670 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd808067320 0x7fd8080697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:15.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.506+0000 7fd8197fa700 1 -- 172.21.15.163:0/1244898670 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fd820085170 con 0x7fd8240b53d0 2023-10-09T06:42:15.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.507+0000 7fd8197fa700 1 --2- 172.21.15.163:0/1244898670 >> [v2:172.21.15.163:6824/90867,v1:172.21.15.163:6825/90867] conn(0x7fd80806d170 0x7fd80806f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:15.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.507+0000 7fd8197fa700 1 -- 172.21.15.163:0/1244898670 --> [v2:172.21.15.163:6824/90867,v1:172.21.15.163:6825/90867] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd80806fc50 con 0x7fd80806d170 2023-10-09T06:42:15.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.507+0000 7fd8197fa700 1 -- 172.21.15.163:0/1244898670 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=202) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8200876d0 con 0x7fd8240b53d0 2023-10-09T06:42:15.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.507+0000 7fd828d44700 1 --2- 172.21.15.163:0/1244898670 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd808067320 0x7fd8080697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:15.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.507+0000 7fd829d46700 1 --2- 172.21.15.163:0/1244898670 >> [v2:172.21.15.163:6824/90867,v1:172.21.15.163:6825/90867] conn(0x7fd80806d170 0x7fd80806f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:15.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.507+0000 7fd829d46700 1 --2- 172.21.15.163:0/1244898670 >> [v2:172.21.15.163:6824/90867,v1:172.21.15.163:6825/90867] conn(0x7fd80806d170 0x7fd80806f580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:15.513 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.507+0000 7fd828d44700 1 --2- 172.21.15.163:0/1244898670 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd808067320 0x7fd8080697d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fd8240b5bb0 tx=0x7fd814017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:15.514 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.507+0000 7fd8197fa700 1 -- 172.21.15.163:0/1244898670 <== osd.0 v2:172.21.15.163:6824/90867 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fd80806fc50 con 0x7fd80806d170 2023-10-09T06:42:15.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.521+0000 7fd82b7a9700 1 -- 172.21.15.163:0/1244898670 --> [v2:172.21.15.163:6824/90867,v1:172.21.15.163:6825/90867] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd8240718b0 con 0x7fd80806d170 2023-10-09T06:42:15.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.521+0000 7fd8197fa700 1 -- 172.21.15.163:0/1244898670 <== osd.0 v2:172.21.15.163:6824/90867 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fd8240718b0 con 0x7fd80806d170 2023-10-09T06:42:15.523 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:42:15.523 INFO:teuthology.orchestra.run.smithi163.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-09T06:42:15.523 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:42:15.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.522+0000 7fd82b7a9700 1 -- 172.21.15.163:0/1244898670 >> [v2:172.21.15.163:6824/90867,v1:172.21.15.163:6825/90867] conn(0x7fd80806d170 msgr2=0x7fd80806f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:15.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.522+0000 7fd82b7a9700 1 --2- 172.21.15.163:0/1244898670 >> [v2:172.21.15.163:6824/90867,v1:172.21.15.163:6825/90867] conn(0x7fd80806d170 0x7fd80806f580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:15.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.522+0000 7fd82b7a9700 1 -- 172.21.15.163:0/1244898670 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd808067320 msgr2=0x7fd8080697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:15.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.522+0000 7fd82b7a9700 1 --2- 172.21.15.163:0/1244898670 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fd808067320 0x7fd8080697d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fd8240b5bb0 tx=0x7fd814017000 comp rx=0 tx=0).stop 2023-10-09T06:42:15.524 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.522+0000 7fd82b7a9700 1 -- 172.21.15.163:0/1244898670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8240b53d0 msgr2=0x7fd824144b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:15.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.522+0000 7fd82b7a9700 1 --2- 172.21.15.163:0/1244898670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8240b53d0 0x7fd824144b40 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fd82000e0e0 tx=0x7fd8200098d0 comp rx=0 tx=0).stop 2023-10-09T06:42:15.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.522+0000 7fd829d46700 1 -- 172.21.15.163:0/1244898670 reap_dead start 2023-10-09T06:42:15.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.522+0000 7fd82b7a9700 1 -- 172.21.15.163:0/1244898670 shutdown_connections 2023-10-09T06:42:15.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.522+0000 7fd82b7a9700 1 -- 172.21.15.163:0/1244898670 >> 172.21.15.163:0/1244898670 conn(0x7fd8240999f0 msgr2=0x7fd8240ce320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:15.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.522+0000 7fd82b7a9700 1 -- 172.21.15.163:0/1244898670 shutdown_connections 2023-10-09T06:42:15.525 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.522+0000 7fd82b7a9700 1 -- 172.21.15.163:0/1244898670 wait complete. 2023-10-09T06:42:15.535 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-09T06:42:15.651 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.650+0000 7f07ece9d700 1 Processor -- start 2023-10-09T06:42:15.652 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.651+0000 7f07ece9d700 1 -- start start 2023-10-09T06:42:15.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.651+0000 7f07ece9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f07e813f4b0 0x7f07e80af260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:15.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.651+0000 7f07ece9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07e813f880 0x7f07e80af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:15.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.651+0000 7f07ece9d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e80aff10 con 0x7f07e8063230 2023-10-09T06:42:15.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.651+0000 7f07ece9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e81563e0 con 0x7f07e813f4b0 2023-10-09T06:42:15.654 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.651+0000 7f07ece9d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e80b22f0 con 0x7f07e813f880 2023-10-09T06:42:15.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.652+0000 7f07e77fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f07e813f4b0 0x7f07e80af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:15.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.652+0000 7f07e77fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f07e813f4b0 0x7f07e80af260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55104/0 (socket says 172.21.15.163:55104) 2023-10-09T06:42:15.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.652+0000 7f07e77fe700 1 -- 172.21.15.163:0/4259648434 learned_addr learned my addr 172.21.15.163:0/4259648434 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:15.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.652+0000 7f07e6ffd700 1 --2- 172.21.15.163:0/4259648434 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07e813f880 0x7f07e80af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:15.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.652+0000 7f07e77fe700 1 -- 172.21.15.163:0/4259648434 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07e813f880 msgr2=0x7f07e80af9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:15.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.652+0000 7f07e77fe700 1 --2- 172.21.15.163:0/4259648434 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07e813f880 0x7f07e80af9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:15.655 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.652+0000 7f07e77fe700 1 -- 172.21.15.163:0/4259648434 >> v1:172.21.15.181:6789/0 conn(0x7f07e8063230 legacy=0x7f07e80c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:15.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.652+0000 7f07e77fe700 1 -- 172.21.15.163:0/4259648434 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e80b25d0 con 0x7f07e813f4b0 2023-10-09T06:42:15.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.652+0000 7f07e77fe700 1 --2- 172.21.15.163:0/4259648434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07e813f4b0 0x7f07e80af260 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f07d8009f20 tx=0x7f07d8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7149c50e7f2b1e24 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:15.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.652+0000 7f07e67fc700 1 -- 172.21.15.163:0/4259648434 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d800e070 con 0x7f07e813f4b0 2023-10-09T06:42:15.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.652+0000 7f07e67fc700 1 -- 172.21.15.163:0/4259648434 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d800b9d0 con 0x7f07e813f4b0 2023-10-09T06:42:15.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.652+0000 7f07e67fc700 1 -- 172.21.15.163:0/4259648434 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d80092e0 con 0x7f07e813f4b0 2023-10-09T06:42:15.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.653+0000 7f07ece9d700 1 -- 172.21.15.163:0/4259648434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07e813f4b0 msgr2=0x7f07e80af260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:15.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.653+0000 7f07ece9d700 1 --2- 172.21.15.163:0/4259648434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07e813f4b0 0x7f07e80af260 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f07d8009f20 tx=0x7f07d8009840 comp rx=0 tx=0).stop 2023-10-09T06:42:15.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.653+0000 7f07ece9d700 1 -- 172.21.15.163:0/4259648434 shutdown_connections 2023-10-09T06:42:15.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.653+0000 7f07ece9d700 1 --2- 172.21.15.163:0/4259648434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07e813f4b0 0x7f07e80af260 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:15.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.653+0000 7f07ece9d700 1 --2- 172.21.15.163:0/4259648434 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07e813f880 0x7f07e80af9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:15.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.653+0000 7f07ece9d700 1 -- 172.21.15.163:0/4259648434 >> 172.21.15.163:0/4259648434 conn(0x7f07e806ddb0 msgr2=0x7f07e806e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:15.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.653+0000 7f07ece9d700 1 -- 172.21.15.163:0/4259648434 shutdown_connections 2023-10-09T06:42:15.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.653+0000 7f07ece9d700 1 -- 172.21.15.163:0/4259648434 wait complete. 2023-10-09T06:42:15.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.654+0000 7f07ece9d700 1 Processor -- start 2023-10-09T06:42:15.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.654+0000 7f07ece9d700 1 -- start start 2023-10-09T06:42:15.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.655+0000 7f07ece9d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07e8063230 0x7f07e813c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:15.660 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.655+0000 7f07ece9d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07e813f4b0 0x7f07e8156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:15.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.655+0000 7f07ece9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07e813f880 0x7f07e8156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:15.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.655+0000 7f07ece9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e806afb0 con 0x7f07e813f880 2023-10-09T06:42:15.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.655+0000 7f07ece9d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e806b440 con 0x7f07e813f4b0 2023-10-09T06:42:15.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.655+0000 7f07ece9d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e813c780 con 0x7f07e8063230 2023-10-09T06:42:15.661 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.655+0000 7f07e7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07e813f880 0x7f07e8156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:15.662 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.655+0000 7f07e7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07e813f880 0x7f07e8156d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55116/0 (socket says 172.21.15.163:55116) 2023-10-09T06:42:15.662 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.655+0000 7f07e7fff700 1 -- 172.21.15.163:0/2407117587 learned_addr learned my addr 172.21.15.163:0/2407117587 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:15.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.655+0000 7f07e77fe700 1 --2- 172.21.15.163:0/2407117587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07e8063230 0x7f07e813c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:15.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.655+0000 7f07e6ffd700 1 --2- 172.21.15.163:0/2407117587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07e813f4b0 0x7f07e8156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:15.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.655+0000 7f07e7fff700 1 -- 172.21.15.163:0/2407117587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07e8063230 msgr2=0x7f07e813c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:15.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.655+0000 7f07e7fff700 1 --2- 172.21.15.163:0/2407117587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07e8063230 0x7f07e813c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:15.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.655+0000 7f07e7fff700 1 -- 172.21.15.163:0/2407117587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07e813f4b0 msgr2=0x7f07e8156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:15.663 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.655+0000 7f07e7fff700 1 --2- 172.21.15.163:0/2407117587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07e813f4b0 0x7f07e8156830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:15.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.655+0000 7f07e7fff700 1 -- 172.21.15.163:0/2407117587 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d8009ae0 con 0x7f07e813f880 2023-10-09T06:42:15.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.655+0000 7f07e7fff700 1 --2- 172.21.15.163:0/2407117587 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07e813f880 0x7f07e8156d70 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f07e000c560 tx=0x7f07e0007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:15.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.655+0000 7f07cf7fe700 1 -- 172.21.15.163:0/2407117587 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e0001120 con 0x7f07e813f880 2023-10-09T06:42:15.664 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.655+0000 7f07cf7fe700 1 -- 172.21.15.163:0/2407117587 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e0011590 con 0x7f07e813f880 2023-10-09T06:42:15.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.655+0000 7f07cf7fe700 1 -- 172.21.15.163:0/2407117587 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e0011950 con 0x7f07e813f880 2023-10-09T06:42:15.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.655+0000 7f07ece9d700 1 -- 172.21.15.163:0/2407117587 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e806a3a0 con 0x7f07e813f880 2023-10-09T06:42:15.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.656+0000 7f07ece9d700 1 -- 172.21.15.163:0/2407117587 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e806a960 con 0x7f07e813f880 2023-10-09T06:42:15.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.656+0000 7f07ece9d700 1 -- 172.21.15.163:0/2407117587 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e804c3c0 con 0x7f07e813f880 2023-10-09T06:42:15.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.657+0000 7f07cf7fe700 1 -- 172.21.15.163:0/2407117587 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f07e0011d00 con 0x7f07e813f880 2023-10-09T06:42:15.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.657+0000 7f07cf7fe700 1 --2- 172.21.15.163:0/2407117587 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f07d4067320 0x7f07d40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:15.665 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.657+0000 7f07cf7fe700 1 -- 172.21.15.163:0/2407117587 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f07e0083c30 con 0x7f07e813f880 2023-10-09T06:42:15.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.657+0000 7f07e77fe700 1 --2- 172.21.15.163:0/2407117587 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f07d4067320 0x7f07d40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:15.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.658+0000 7f07e77fe700 1 --2- 172.21.15.163:0/2407117587 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f07d4067320 0x7f07d40697d0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f07d8009880 tx=0x7f07d8018000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:15.666 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.661+0000 7f07cf7fe700 1 -- 172.21.15.163:0/2407117587 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f07e0050bc0 con 0x7f07e813f880 2023-10-09T06:42:15.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.850+0000 7f07ece9d700 1 -- 172.21.15.163:0/2407117587 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f07e80aead0 con 0x7f07d4067320 2023-10-09T06:42:15.851 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.851+0000 7f07cf7fe700 1 -- 172.21.15.163:0/2407117587 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f07e80aead0 con 0x7f07d4067320 2023-10-09T06:42:15.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.852+0000 7f07ece9d700 1 -- 172.21.15.163:0/2407117587 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f07d4067320 msgr2=0x7f07d40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:15.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.852+0000 7f07ece9d700 1 --2- 172.21.15.163:0/2407117587 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f07d4067320 0x7f07d40697d0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f07d8009880 tx=0x7f07d8018000 comp rx=0 tx=0).stop 2023-10-09T06:42:15.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.852+0000 7f07ece9d700 1 -- 172.21.15.163:0/2407117587 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07e813f880 msgr2=0x7f07e8156d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:15.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.852+0000 7f07ece9d700 1 --2- 172.21.15.163:0/2407117587 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07e813f880 0x7f07e8156d70 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f07e000c560 tx=0x7f07e0007ef0 comp rx=0 tx=0).stop 2023-10-09T06:42:15.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.853+0000 7f07ece9d700 1 -- 172.21.15.163:0/2407117587 shutdown_connections 2023-10-09T06:42:15.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.853+0000 7f07ece9d700 1 --2- 172.21.15.163:0/2407117587 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f07d4067320 0x7f07d40697d0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:15.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.853+0000 7f07ece9d700 1 --2- 172.21.15.163:0/2407117587 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07e813f880 0x7f07e8156d70 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:15.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.853+0000 7f07ece9d700 1 --2- 172.21.15.163:0/2407117587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07e8063230 0x7f07e813c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:15.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.853+0000 7f07ece9d700 1 --2- 172.21.15.163:0/2407117587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07e813f4b0 0x7f07e8156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:15.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.853+0000 7f07ece9d700 1 -- 172.21.15.163:0/2407117587 >> 172.21.15.163:0/2407117587 conn(0x7f07e806ddb0 msgr2=0x7f07e80c85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:15.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.853+0000 7f07ece9d700 1 -- 172.21.15.163:0/2407117587 shutdown_connections 2023-10-09T06:42:15.855 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.853+0000 7f07ece9d700 1 -- 172.21.15.163:0/2407117587 wait complete. 2023-10-09T06:42:15.855 INFO:teuthology.orchestra.run.smithi163.stderr:instructed osd(s) 0 to deep-scrub 2023-10-09T06:42:15.866 INFO:tasks.ceph:Scrubbing osd.1 2023-10-09T06:42:15.866 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-09T06:42:15.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.983+0000 7f1e7b2bc700 1 Processor -- start 2023-10-09T06:42:15.985 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.984+0000 7f1e7b2bc700 1 -- start start 2023-10-09T06:42:15.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.984+0000 7f1e7b2bc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e74159ec0 0x7f1e740b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:15.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.984+0000 7f1e7b2bc700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1e740cf350 0x7f1e740cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:15.987 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.984+0000 7f1e7b2bc700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e740b5b00 con 0x7f1e740b53d0 2023-10-09T06:42:15.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.984+0000 7f1e7b2bc700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e740b5da0 con 0x7f1e740cf350 2023-10-09T06:42:15.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.984+0000 7f1e7b2bc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e740b6080 con 0x7f1e74159ec0 2023-10-09T06:42:15.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.985+0000 7f1e78857700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f1e740b53d0 0x7f1e740cc900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60602/0 (socket says 172.21.15.163:60602) 2023-10-09T06:42:15.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.985+0000 7f1e78857700 1 -- 172.21.15.163:0/544707376 learned_addr learned my addr 172.21.15.163:0/544707376 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:42:15.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.985+0000 7f1e79058700 1 --2- 172.21.15.163:0/544707376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e74159ec0 0x7f1e740b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:15.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.985+0000 7f1e79859700 1 --2- 172.21.15.163:0/544707376 >> v2:172.21.15.181:3300/0 conn(0x7f1e740cf350 0x7f1e740cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:15.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.985+0000 7f1e6bfff700 1 -- 172.21.15.163:0/544707376 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1524179819 0 0) 0x7f1e740b5b00 con 0x7f1e740b53d0 2023-10-09T06:42:15.988 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.985+0000 7f1e6bfff700 1 -- 172.21.15.163:0/544707376 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e60003570 con 0x7f1e740b53d0 2023-10-09T06:42:15.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.985+0000 7f1e6bfff700 1 -- 172.21.15.163:0/544707376 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1513627896 0 0) 0x7f1e60003570 con 0x7f1e740b53d0 2023-10-09T06:42:15.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.985+0000 7f1e79058700 1 -- 172.21.15.163:0/544707376 >> v2:172.21.15.181:3300/0 conn(0x7f1e740cf350 msgr2=0x7f1e740cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:15.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.985+0000 7f1e79058700 1 --2- 172.21.15.163:0/544707376 >> v2:172.21.15.181:3300/0 conn(0x7f1e740cf350 0x7f1e740cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:15.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.985+0000 7f1e79058700 1 -- 172.21.15.163:0/544707376 >> v1:172.21.15.163:6789/0 conn(0x7f1e740b53d0 legacy=0x7f1e740cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:15.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.985+0000 7f1e79058700 1 -- 172.21.15.163:0/544707376 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e740cc100 con 0x7f1e74159ec0 2023-10-09T06:42:15.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.985+0000 7f1e79859700 1 --2- 172.21.15.163:0/544707376 >> v2:172.21.15.181:3300/0 conn(0x7f1e740cf350 0x7f1e740cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-09T06:42:15.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.985+0000 7f1e79058700 1 --2- 172.21.15.163:0/544707376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e74159ec0 0x7f1e740b4e90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1e700098d0 tx=0x7f1e70009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdbf4f37beda649b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:15.989 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.985+0000 7f1e6bfff700 1 -- 172.21.15.163:0/544707376 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e64001470 con 0x7f1e74159ec0 2023-10-09T06:42:15.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.985+0000 7f1e6bfff700 1 -- 172.21.15.163:0/544707376 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e70011420 con 0x7f1e74159ec0 2023-10-09T06:42:15.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.985+0000 7f1e6bfff700 1 -- 172.21.15.163:0/544707376 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e700117c0 con 0x7f1e74159ec0 2023-10-09T06:42:15.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.986+0000 7f1e7b2bc700 1 -- 172.21.15.163:0/544707376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e74159ec0 msgr2=0x7f1e740b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:15.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.986+0000 7f1e7b2bc700 1 --2- 172.21.15.163:0/544707376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e74159ec0 0x7f1e740b4e90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1e700098d0 tx=0x7f1e70009120 comp rx=0 tx=0).stop 2023-10-09T06:42:15.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.986+0000 7f1e7b2bc700 1 -- 172.21.15.163:0/544707376 shutdown_connections 2023-10-09T06:42:15.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.986+0000 7f1e7b2bc700 1 --2- 172.21.15.163:0/544707376 >> v2:172.21.15.181:3300/0 conn(0x7f1e740cf350 0x7f1e740cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:15.990 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.986+0000 7f1e7b2bc700 1 --2- 172.21.15.163:0/544707376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e74159ec0 0x7f1e740b4e90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:15.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.986+0000 7f1e7b2bc700 1 -- 172.21.15.163:0/544707376 >> 172.21.15.163:0/544707376 conn(0x7f1e740999f0 msgr2=0x7f1e740b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:15.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.986+0000 7f1e7b2bc700 1 -- 172.21.15.163:0/544707376 shutdown_connections 2023-10-09T06:42:15.991 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.986+0000 7f1e7b2bc700 1 -- 172.21.15.163:0/544707376 wait complete. 2023-10-09T06:42:15.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.987+0000 7f1e7b2bc700 1 Processor -- start 2023-10-09T06:42:15.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.987+0000 7f1e7b2bc700 1 -- start start 2023-10-09T06:42:15.994 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.988+0000 7f1e7b2bc700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e740b53d0 0x7f1e74158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:15.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.988+0000 7f1e7b2bc700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e740cf350 0x7f1e74068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:15.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.988+0000 7f1e7b2bc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e74159ec0 0x7f1e74153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:15.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.988+0000 7f1e7b2bc700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e740688d0 con 0x7f1e74159ec0 2023-10-09T06:42:15.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.988+0000 7f1e7b2bc700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e740693a0 con 0x7f1e740cf350 2023-10-09T06:42:15.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.988+0000 7f1e7b2bc700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e74068b00 con 0x7f1e740b53d0 2023-10-09T06:42:15.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.988+0000 7f1e79859700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e74159ec0 0x7f1e74153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:15.995 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.988+0000 7f1e79859700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e74159ec0 0x7f1e74153390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55130/0 (socket says 172.21.15.163:55130) 2023-10-09T06:42:15.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.988+0000 7f1e79859700 1 -- 172.21.15.163:0/3888790479 learned_addr learned my addr 172.21.15.163:0/3888790479 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:15.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.988+0000 7f1e79058700 1 --2- 172.21.15.163:0/3888790479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e740b53d0 0x7f1e74158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:15.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.988+0000 7f1e78857700 1 --2- 172.21.15.163:0/3888790479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e740cf350 0x7f1e74068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:15.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.989+0000 7f1e79859700 1 -- 172.21.15.163:0/3888790479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e740b53d0 msgr2=0x7f1e74158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:15.996 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.989+0000 7f1e79859700 1 --2- 172.21.15.163:0/3888790479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e740b53d0 0x7f1e74158cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:15.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.989+0000 7f1e79859700 1 -- 172.21.15.163:0/3888790479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e740cf350 msgr2=0x7f1e74068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:15.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.989+0000 7f1e79859700 1 --2- 172.21.15.163:0/3888790479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e740cf350 0x7f1e74068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:15.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.989+0000 7f1e79859700 1 -- 172.21.15.163:0/3888790479 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e6c009a20 con 0x7f1e74159ec0 2023-10-09T06:42:15.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.989+0000 7f1e79859700 1 --2- 172.21.15.163:0/3888790479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e74159ec0 0x7f1e74153390 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c009fc0 tx=0x7f1e6c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:15.997 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.989+0000 7f1e697fa700 1 -- 172.21.15.163:0/3888790479 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c00cbd0 con 0x7f1e74159ec0 2023-10-09T06:42:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.989+0000 7f1e697fa700 1 -- 172.21.15.163:0/3888790479 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6c00f040 con 0x7f1e74159ec0 2023-10-09T06:42:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.989+0000 7f1e697fa700 1 -- 172.21.15.163:0/3888790479 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c014480 con 0x7f1e74159ec0 2023-10-09T06:42:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.989+0000 7f1e7b2bc700 1 -- 172.21.15.163:0/3888790479 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e700093c0 con 0x7f1e74159ec0 2023-10-09T06:42:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.989+0000 7f1e7b2bc700 1 -- 172.21.15.163:0/3888790479 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e740690c0 con 0x7f1e74159ec0 2023-10-09T06:42:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.990+0000 7f1e7b2bc700 1 -- 172.21.15.163:0/3888790479 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1e74063c90 con 0x7f1e74159ec0 2023-10-09T06:42:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.990+0000 7f1e697fa700 1 -- 172.21.15.163:0/3888790479 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1e6c003020 con 0x7f1e74159ec0 2023-10-09T06:42:15.998 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.991+0000 7f1e697fa700 1 --2- 172.21.15.163:0/3888790479 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1e58067320 0x7f1e580697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:15.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.991+0000 7f1e697fa700 1 -- 172.21.15.163:0/3888790479 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f1e6c0853e0 con 0x7f1e74159ec0 2023-10-09T06:42:15.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.991+0000 7f1e697fa700 1 --2- 172.21.15.163:0/3888790479 >> [v2:172.21.15.163:6808/90868,v1:172.21.15.163:6809/90868] conn(0x7f1e5806d170 0x7f1e5806f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:15.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.991+0000 7f1e79058700 1 --2- 172.21.15.163:0/3888790479 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1e58067320 0x7f1e580697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:15.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.991+0000 7f1e697fa700 1 -- 172.21.15.163:0/3888790479 --> [v2:172.21.15.163:6808/90868,v1:172.21.15.163:6809/90868] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e5806fc50 con 0x7f1e5806d170 2023-10-09T06:42:15.999 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.991+0000 7f1e697fa700 1 -- 172.21.15.163:0/3888790479 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=202) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e6c087940 con 0x7f1e74159ec0 2023-10-09T06:42:16.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.991+0000 7f1e78857700 1 --2- 172.21.15.163:0/3888790479 >> [v2:172.21.15.163:6808/90868,v1:172.21.15.163:6809/90868] conn(0x7f1e5806d170 0x7f1e5806f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:16.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.991+0000 7f1e79058700 1 --2- 172.21.15.163:0/3888790479 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1e58067320 0x7f1e580697d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f1e70009390 tx=0x7f1e70016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:16.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.991+0000 7f1e78857700 1 --2- 172.21.15.163:0/3888790479 >> [v2:172.21.15.163:6808/90868,v1:172.21.15.163:6809/90868] conn(0x7f1e5806d170 0x7f1e5806f580 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:16.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:15.992+0000 7f1e697fa700 1 -- 172.21.15.163:0/3888790479 <== osd.1 v2:172.21.15.163:6808/90868 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f1e5806fc50 con 0x7f1e5806d170 2023-10-09T06:42:16.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.005+0000 7f1e7b2bc700 1 -- 172.21.15.163:0/3888790479 --> [v2:172.21.15.163:6808/90868,v1:172.21.15.163:6809/90868] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1e7415ada0 con 0x7f1e5806d170 2023-10-09T06:42:16.007 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.005+0000 7f1e697fa700 1 -- 172.21.15.163:0/3888790479 <== osd.1 v2:172.21.15.163:6808/90868 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f1e7415ada0 con 0x7f1e5806d170 2023-10-09T06:42:16.007 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:42:16.007 INFO:teuthology.orchestra.run.smithi163.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-09T06:42:16.008 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:42:16.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.006+0000 7f1e7b2bc700 1 -- 172.21.15.163:0/3888790479 >> [v2:172.21.15.163:6808/90868,v1:172.21.15.163:6809/90868] conn(0x7f1e5806d170 msgr2=0x7f1e5806f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:16.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.006+0000 7f1e7b2bc700 1 --2- 172.21.15.163:0/3888790479 >> [v2:172.21.15.163:6808/90868,v1:172.21.15.163:6809/90868] conn(0x7f1e5806d170 0x7f1e5806f580 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.006+0000 7f1e7b2bc700 1 -- 172.21.15.163:0/3888790479 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1e58067320 msgr2=0x7f1e580697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:16.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.006+0000 7f1e7b2bc700 1 --2- 172.21.15.163:0/3888790479 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1e58067320 0x7f1e580697d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f1e70009390 tx=0x7f1e70016000 comp rx=0 tx=0).stop 2023-10-09T06:42:16.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.006+0000 7f1e7b2bc700 1 -- 172.21.15.163:0/3888790479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e74159ec0 msgr2=0x7f1e74153390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:16.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.006+0000 7f1e7b2bc700 1 --2- 172.21.15.163:0/3888790479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e74159ec0 0x7f1e74153390 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c009fc0 tx=0x7f1e6c00b040 comp rx=0 tx=0).stop 2023-10-09T06:42:16.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.006+0000 7f1e79859700 1 -- 172.21.15.163:0/3888790479 reap_dead start 2023-10-09T06:42:16.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.006+0000 7f1e7b2bc700 1 -- 172.21.15.163:0/3888790479 shutdown_connections 2023-10-09T06:42:16.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.006+0000 7f1e7b2bc700 1 -- 172.21.15.163:0/3888790479 >> 172.21.15.163:0/3888790479 conn(0x7f1e740999f0 msgr2=0x7f1e740c60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:16.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.006+0000 7f1e7b2bc700 1 -- 172.21.15.163:0/3888790479 shutdown_connections 2023-10-09T06:42:16.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.006+0000 7f1e7b2bc700 1 -- 172.21.15.163:0/3888790479 wait complete. 2023-10-09T06:42:16.019 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-09T06:42:16.134 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.133+0000 7fee39fc2700 1 Processor -- start 2023-10-09T06:42:16.135 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.133+0000 7fee39fc2700 1 -- start start 2023-10-09T06:42:16.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.134+0000 7fee39fc2700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fee34068aa0 0x7fee34162820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:16.138 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.134+0000 7fee39fc2700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee340c7de0 con 0x7fee34140720 2023-10-09T06:42:16.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.134+0000 7fee39fc2700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee34140c10 con 0x7fee34140350 2023-10-09T06:42:16.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.134+0000 7fee39fc2700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee34141490 con 0x7fee34068aa0 2023-10-09T06:42:16.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.134+0000 7fee32ffd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fee34140720 0x7fee34068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60616/0 (socket says 172.21.15.163:60616) 2023-10-09T06:42:16.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.134+0000 7fee32ffd700 1 -- 172.21.15.163:0/2166358944 learned_addr learned my addr 172.21.15.163:0/2166358944 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:42:16.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.135+0000 7fee33fff700 1 --2- 172.21.15.163:0/2166358944 >> v2:172.21.15.181:3300/0 conn(0x7fee34068aa0 0x7fee34162820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:16.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.135+0000 7fee327fc700 1 -- 172.21.15.163:0/2166358944 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 507298387 0 0) 0x7fee340c7de0 con 0x7fee34140720 2023-10-09T06:42:16.139 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.135+0000 7fee327fc700 1 -- 172.21.15.163:0/2166358944 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee1c003570 con 0x7fee34140720 2023-10-09T06:42:16.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.135+0000 7fee327fc700 1 -- 172.21.15.163:0/2166358944 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2650327638 0 0) 0x7fee1c003570 con 0x7fee34140720 2023-10-09T06:42:16.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.135+0000 7fee327fc700 1 -- 172.21.15.163:0/2166358944 >> v2:172.21.15.181:3300/0 conn(0x7fee34068aa0 msgr2=0x7fee34162820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:16.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.135+0000 7fee327fc700 1 --2- 172.21.15.163:0/2166358944 >> v2:172.21.15.181:3300/0 conn(0x7fee34068aa0 0x7fee34162820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.135+0000 7fee327fc700 1 -- 172.21.15.163:0/2166358944 >> v1:172.21.15.181:6789/0 conn(0x7fee34140350 legacy=0x7fee34069c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:16.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.135+0000 7fee327fc700 1 -- 172.21.15.163:0/2166358944 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee34140ed0 con 0x7fee34140720 2023-10-09T06:42:16.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.135+0000 7fee327fc700 1 -- 172.21.15.163:0/2166358944 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fee28001470 con 0x7fee34140720 2023-10-09T06:42:16.140 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.135+0000 7fee327fc700 1 -- 172.21.15.163:0/2166358944 >> v1:172.21.15.163:6789/0 conn(0x7fee34140720 legacy=0x7fee34068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:16.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.135+0000 7fee327fc700 1 --2- 172.21.15.163:0/2166358944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee1c003a60 0x7fee1c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:16.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.135+0000 7fee327fc700 1 --2- 172.21.15.163:0/2166358944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee1c004810 0x7fee1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:16.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.135+0000 7fee327fc700 1 --2- 172.21.15.163:0/2166358944 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee1c0051c0 0x7fee1c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:16.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.135+0000 7fee327fc700 1 -- 172.21.15.163:0/2166358944 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee34141490 con 0x7fee1c003a60 2023-10-09T06:42:16.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.135+0000 7fee327fc700 1 -- 172.21.15.163:0/2166358944 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c00c0d0 con 0x7fee1c004810 2023-10-09T06:42:16.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.135+0000 7fee327fc700 1 -- 172.21.15.163:0/2166358944 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c00c290 con 0x7fee1c0051c0 2023-10-09T06:42:16.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.135+0000 7fee337fe700 1 --2- 172.21.15.163:0/2166358944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee1c003a60 0x7fee1c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:16.141 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.135+0000 7fee32ffd700 1 --2- 172.21.15.163:0/2166358944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee1c004810 0x7fee1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:16.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.135+0000 7fee33fff700 1 --2- 172.21.15.163:0/2166358944 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee1c0051c0 0x7fee1c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:16.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.135+0000 7fee337fe700 1 -- 172.21.15.163:0/2166358944 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee1c0051c0 msgr2=0x7fee1c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:16.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.135+0000 7fee337fe700 1 --2- 172.21.15.163:0/2166358944 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee1c0051c0 0x7fee1c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.135+0000 7fee337fe700 1 -- 172.21.15.163:0/2166358944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee1c004810 msgr2=0x7fee1c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:16.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.135+0000 7fee337fe700 1 --2- 172.21.15.163:0/2166358944 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee1c004810 0x7fee1c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.135+0000 7fee337fe700 1 -- 172.21.15.163:0/2166358944 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1c003570 con 0x7fee1c003a60 2023-10-09T06:42:16.142 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.135+0000 7fee33fff700 1 --2- 172.21.15.163:0/2166358944 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee1c0051c0 0x7fee1c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:42:16.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.135+0000 7fee33fff700 1 -- 172.21.15.163:0/2166358944 reap_dead start 2023-10-09T06:42:16.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.136+0000 7fee337fe700 1 --2- 172.21.15.163:0/2166358944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee1c003a60 0x7fee1c003e70 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fee2400c2d0 tx=0x7fee24001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843e78113181bb1b server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:16.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.136+0000 7fee327fc700 1 -- 172.21.15.163:0/2166358944 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee24001d80 con 0x7fee1c003a60 2023-10-09T06:42:16.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.136+0000 7fee327fc700 1 -- 172.21.15.163:0/2166358944 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2400cba0 con 0x7fee1c003a60 2023-10-09T06:42:16.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.136+0000 7fee327fc700 1 -- 172.21.15.163:0/2166358944 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee24012400 con 0x7fee1c003a60 2023-10-09T06:42:16.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.136+0000 7fee39fc2700 1 -- 172.21.15.163:0/2166358944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee1c003a60 msgr2=0x7fee1c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:16.143 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.136+0000 7fee39fc2700 1 --2- 172.21.15.163:0/2166358944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee1c003a60 0x7fee1c003e70 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fee2400c2d0 tx=0x7fee24001a70 comp rx=0 tx=0).stop 2023-10-09T06:42:16.144 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.136+0000 7fee39fc2700 1 -- 172.21.15.163:0/2166358944 shutdown_connections 2023-10-09T06:42:16.144 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.136+0000 7fee39fc2700 1 --2- 172.21.15.163:0/2166358944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee1c003a60 0x7fee1c003e70 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.144 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.136+0000 7fee39fc2700 1 -- 172.21.15.163:0/2166358944 >> 172.21.15.163:0/2166358944 conn(0x7fee340cd670 msgr2=0x7fee340cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:16.144 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.136+0000 7fee39fc2700 1 -- 172.21.15.163:0/2166358944 shutdown_connections 2023-10-09T06:42:16.144 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.136+0000 7fee39fc2700 1 -- 172.21.15.163:0/2166358944 wait complete. 2023-10-09T06:42:16.144 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.137+0000 7fee39fc2700 1 Processor -- start 2023-10-09T06:42:16.144 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.137+0000 7fee39fc2700 1 -- start start 2023-10-09T06:42:16.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.138+0000 7fee39fc2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee1c003a60 0x7fee3413b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:16.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.138+0000 7fee39fc2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee34168a80 0x7fee340c4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:16.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.138+0000 7fee39fc2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee3413eea0 0x7fee3413f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:16.147 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.138+0000 7fee39fc2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3413f850 con 0x7fee3413eea0 2023-10-09T06:42:16.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.138+0000 7fee39fc2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3413bf20 con 0x7fee34168a80 2023-10-09T06:42:16.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.138+0000 7fee39fc2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee340c4c60 con 0x7fee1c003a60 2023-10-09T06:42:16.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.138+0000 7fee33fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee3413eea0 0x7fee3413f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:16.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.138+0000 7fee33fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee3413eea0 0x7fee3413f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55138/0 (socket says 172.21.15.163:55138) 2023-10-09T06:42:16.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.138+0000 7fee33fff700 1 -- 172.21.15.163:0/3349115429 learned_addr learned my addr 172.21.15.163:0/3349115429 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:16.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.138+0000 7fee337fe700 1 --2- 172.21.15.163:0/3349115429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee1c003a60 0x7fee3413b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:16.148 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.138+0000 7fee32ffd700 1 --2- 172.21.15.163:0/3349115429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee34168a80 0x7fee340c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:16.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.138+0000 7fee33fff700 1 -- 172.21.15.163:0/3349115429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee1c003a60 msgr2=0x7fee3413b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:16.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.138+0000 7fee33fff700 1 --2- 172.21.15.163:0/3349115429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee1c003a60 0x7fee3413b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.138+0000 7fee33fff700 1 -- 172.21.15.163:0/3349115429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee34168a80 msgr2=0x7fee340c4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:16.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.138+0000 7fee33fff700 1 --2- 172.21.15.163:0/3349115429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee34168a80 0x7fee340c4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.139+0000 7fee33fff700 1 -- 172.21.15.163:0/3349115429 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3416b5c0 con 0x7fee3413eea0 2023-10-09T06:42:16.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.139+0000 7fee32ffd700 1 --2- 172.21.15.163:0/3349115429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee34168a80 0x7fee340c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:42:16.149 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.139+0000 7fee33fff700 1 --2- 172.21.15.163:0/3349115429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee3413eea0 0x7fee3413f310 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fee3406d2f0 tx=0x7fee2c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:16.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.139+0000 7fee1b7fe700 1 -- 172.21.15.163:0/3349115429 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c015670 con 0x7fee3413eea0 2023-10-09T06:42:16.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.139+0000 7fee1b7fe700 1 -- 172.21.15.163:0/3349115429 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2c0157d0 con 0x7fee3413eea0 2023-10-09T06:42:16.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.139+0000 7fee1b7fe700 1 -- 172.21.15.163:0/3349115429 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c015b90 con 0x7fee3413eea0 2023-10-09T06:42:16.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.139+0000 7fee39fc2700 1 -- 172.21.15.163:0/3349115429 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee3416b8a0 con 0x7fee3413eea0 2023-10-09T06:42:16.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.139+0000 7fee39fc2700 1 -- 172.21.15.163:0/3349115429 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee3416be60 con 0x7fee3413eea0 2023-10-09T06:42:16.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.140+0000 7fee39fc2700 1 -- 172.21.15.163:0/3349115429 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee3404c3c0 con 0x7fee3413eea0 2023-10-09T06:42:16.150 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.140+0000 7fee1b7fe700 1 -- 172.21.15.163:0/3349115429 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fee2c020460 con 0x7fee3413eea0 2023-10-09T06:42:16.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.141+0000 7fee1b7fe700 1 --2- 172.21.15.163:0/3349115429 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fee20067350 0x7fee20069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:16.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.141+0000 7fee1b7fe700 1 -- 172.21.15.163:0/3349115429 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fee2c084a30 con 0x7fee3413eea0 2023-10-09T06:42:16.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.141+0000 7fee337fe700 1 --2- 172.21.15.163:0/3349115429 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fee20067350 0x7fee20069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:16.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.141+0000 7fee337fe700 1 --2- 172.21.15.163:0/3349115429 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fee20067350 0x7fee20069800 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fee240013e0 tx=0x7fee24016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:16.151 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.145+0000 7fee1b7fe700 1 -- 172.21.15.163:0/3349115429 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee2c0519c0 con 0x7fee3413eea0 2023-10-09T06:42:16.328 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.327+0000 7fee39fc2700 1 -- 172.21.15.163:0/3349115429 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fee341411b0 con 0x7fee20067350 2023-10-09T06:42:16.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.328+0000 7fee1b7fe700 1 -- 172.21.15.163:0/3349115429 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fee341411b0 con 0x7fee20067350 2023-10-09T06:42:16.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.330+0000 7fee39fc2700 1 -- 172.21.15.163:0/3349115429 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fee20067350 msgr2=0x7fee20069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:16.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.330+0000 7fee39fc2700 1 --2- 172.21.15.163:0/3349115429 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fee20067350 0x7fee20069800 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fee240013e0 tx=0x7fee24016000 comp rx=0 tx=0).stop 2023-10-09T06:42:16.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.330+0000 7fee39fc2700 1 -- 172.21.15.163:0/3349115429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee3413eea0 msgr2=0x7fee3413f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:16.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.330+0000 7fee39fc2700 1 --2- 172.21.15.163:0/3349115429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee3413eea0 0x7fee3413f310 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fee3406d2f0 tx=0x7fee2c008040 comp rx=0 tx=0).stop 2023-10-09T06:42:16.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.330+0000 7fee39fc2700 1 -- 172.21.15.163:0/3349115429 shutdown_connections 2023-10-09T06:42:16.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.330+0000 7fee39fc2700 1 --2- 172.21.15.163:0/3349115429 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fee20067350 0x7fee20069800 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.330+0000 7fee39fc2700 1 --2- 172.21.15.163:0/3349115429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee3413eea0 0x7fee3413f310 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.330+0000 7fee39fc2700 1 --2- 172.21.15.163:0/3349115429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee1c003a60 0x7fee3413b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.330+0000 7fee39fc2700 1 --2- 172.21.15.163:0/3349115429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee34168a80 0x7fee340c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.330+0000 7fee39fc2700 1 -- 172.21.15.163:0/3349115429 >> 172.21.15.163:0/3349115429 conn(0x7fee340cd670 msgr2=0x7fee340ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:16.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.330+0000 7fee39fc2700 1 -- 172.21.15.163:0/3349115429 shutdown_connections 2023-10-09T06:42:16.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.330+0000 7fee39fc2700 1 -- 172.21.15.163:0/3349115429 wait complete. 2023-10-09T06:42:16.334 INFO:teuthology.orchestra.run.smithi163.stderr:instructed osd(s) 1 to deep-scrub 2023-10-09T06:42:16.344 INFO:tasks.ceph:Scrubbing osd.2 2023-10-09T06:42:16.344 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-09T06:42:16.463 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.462+0000 7fb3b71e2700 1 Processor -- start 2023-10-09T06:42:16.464 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.462+0000 7fb3b71e2700 1 -- start start 2023-10-09T06:42:16.466 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.463+0000 7fb3b71e2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3b01430c0 0x7fb3b00cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:16.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.463+0000 7fb3b71e2700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb3b0143490 0x7fb3b00694a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:16.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.463+0000 7fb3b71e2700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3b00cf410 con 0x7fb3b00cee50 2023-10-09T06:42:16.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.463+0000 7fb3b71e2700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3b00cf6f0 con 0x7fb3b0143490 2023-10-09T06:42:16.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.463+0000 7fb3b71e2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b00cf9d0 con 0x7fb3b01430c0 2023-10-09T06:42:16.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.463+0000 7fb3affff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb3b0143490 0x7fb3b00694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:16.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.463+0000 7fb3b577f700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb3b00cee50 0x7fb3b00b3570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60626/0 (socket says 172.21.15.163:60626) 2023-10-09T06:42:16.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.463+0000 7fb3b577f700 1 -- 172.21.15.163:0/2500851845 learned_addr learned my addr 172.21.15.163:0/2500851845 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:42:16.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.463+0000 7fb3b4f7e700 1 --2- 172.21.15.163:0/2500851845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3b01430c0 0x7fb3b00cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:16.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.463+0000 7fb3affff700 1 -- 172.21.15.163:0/2500851845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3b01430c0 msgr2=0x7fb3b00cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:16.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.463+0000 7fb3affff700 1 --2- 172.21.15.163:0/2500851845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3b01430c0 0x7fb3b00cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.463+0000 7fb3affff700 1 -- 172.21.15.163:0/2500851845 >> v1:172.21.15.163:6789/0 conn(0x7fb3b00cee50 legacy=0x7fb3b00b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:16.470 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.463+0000 7fb3affff700 1 -- 172.21.15.163:0/2500851845 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b00cfcb0 con 0x7fb3b0143490 2023-10-09T06:42:16.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.463+0000 7fb3af7fe700 1 -- 172.21.15.163:0/2500851845 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3195054446 0 0) 0x7fb3b00cf410 con 0x7fb3b00cee50 2023-10-09T06:42:16.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.464+0000 7fb3affff700 1 --2- 172.21.15.163:0/2500851845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3b0143490 0x7fb3b00694a0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fb3980098e0 tx=0x7fb3980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0e9c42002fa298 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:16.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.464+0000 7fb3af7fe700 1 -- 172.21.15.163:0/2500851845 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39800b640 con 0x7fb3b0143490 2023-10-09T06:42:16.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.464+0000 7fb3af7fe700 1 -- 172.21.15.163:0/2500851845 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb39800b7a0 con 0x7fb3b0143490 2023-10-09T06:42:16.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.464+0000 7fb3af7fe700 1 -- 172.21.15.163:0/2500851845 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39800bb40 con 0x7fb3b0143490 2023-10-09T06:42:16.471 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.464+0000 7fb3b71e2700 1 -- 172.21.15.163:0/2500851845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3b0143490 msgr2=0x7fb3b00694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:16.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.464+0000 7fb3b71e2700 1 --2- 172.21.15.163:0/2500851845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3b0143490 0x7fb3b00694a0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fb3980098e0 tx=0x7fb3980091a0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.464+0000 7fb3b71e2700 1 -- 172.21.15.163:0/2500851845 shutdown_connections 2023-10-09T06:42:16.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.464+0000 7fb3b71e2700 1 --2- 172.21.15.163:0/2500851845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3b01430c0 0x7fb3b00cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.464+0000 7fb3b71e2700 1 --2- 172.21.15.163:0/2500851845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3b0143490 0x7fb3b00694a0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.464+0000 7fb3b71e2700 1 -- 172.21.15.163:0/2500851845 >> 172.21.15.163:0/2500851845 conn(0x7fb3b0068e60 msgr2=0x7fb3b00b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:16.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.464+0000 7fb3b71e2700 1 -- 172.21.15.163:0/2500851845 shutdown_connections 2023-10-09T06:42:16.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.464+0000 7fb3b71e2700 1 -- 172.21.15.163:0/2500851845 wait complete. 2023-10-09T06:42:16.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.465+0000 7fb3b71e2700 1 Processor -- start 2023-10-09T06:42:16.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.466+0000 7fb3b71e2700 1 -- start start 2023-10-09T06:42:16.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.466+0000 7fb3b71e2700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3b00cee50 0x7fb3b0140c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:16.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.466+0000 7fb3b71e2700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3b01430c0 0x7fb3b00c7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:16.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.466+0000 7fb3b71e2700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3b0143490 0x7fb3b00c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:16.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.466+0000 7fb3b71e2700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b015d430 con 0x7fb3b01430c0 2023-10-09T06:42:16.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.466+0000 7fb3b71e2700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b0141180 con 0x7fb3b00cee50 2023-10-09T06:42:16.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.466+0000 7fb3b71e2700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b00c62f0 con 0x7fb3b0143490 2023-10-09T06:42:16.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.466+0000 7fb3affff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3b01430c0 0x7fb3b00c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:16.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.466+0000 7fb3affff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3b01430c0 0x7fb3b00c7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55148/0 (socket says 172.21.15.163:55148) 2023-10-09T06:42:16.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.466+0000 7fb3affff700 1 -- 172.21.15.163:0/4098270171 learned_addr learned my addr 172.21.15.163:0/4098270171 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:16.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.467+0000 7fb3b577f700 1 --2- 172.21.15.163:0/4098270171 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3b0143490 0x7fb3b00c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:16.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.467+0000 7fb3affff700 1 -- 172.21.15.163:0/4098270171 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3b0143490 msgr2=0x7fb3b00c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:16.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.467+0000 7fb3affff700 1 --2- 172.21.15.163:0/4098270171 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3b0143490 0x7fb3b00c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.477 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.467+0000 7fb3affff700 1 -- 172.21.15.163:0/4098270171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3b00cee50 msgr2=0x7fb3b0140c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:16.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.467+0000 7fb3affff700 1 --2- 172.21.15.163:0/4098270171 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3b00cee50 0x7fb3b0140c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.467+0000 7fb3affff700 1 -- 172.21.15.163:0/4098270171 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb398009420 con 0x7fb3b01430c0 2023-10-09T06:42:16.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.467+0000 7fb3affff700 1 --2- 172.21.15.163:0/4098270171 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3b01430c0 0x7fb3b00c7870 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0067c20 tx=0x7fb398017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:16.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.467+0000 7fb3acff9700 1 -- 172.21.15.163:0/4098270171 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39800bb20 con 0x7fb3b01430c0 2023-10-09T06:42:16.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.467+0000 7fb3acff9700 1 -- 172.21.15.163:0/4098270171 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb398011420 con 0x7fb3b01430c0 2023-10-09T06:42:16.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.467+0000 7fb3acff9700 1 -- 172.21.15.163:0/4098270171 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3980117a0 con 0x7fb3b01430c0 2023-10-09T06:42:16.478 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.467+0000 7fb3b71e2700 1 -- 172.21.15.163:0/4098270171 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b00c65d0 con 0x7fb3b01430c0 2023-10-09T06:42:16.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.467+0000 7fb3b71e2700 1 -- 172.21.15.163:0/4098270171 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b00c6b90 con 0x7fb3b01430c0 2023-10-09T06:42:16.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.468+0000 7fb3b71e2700 1 -- 172.21.15.163:0/4098270171 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb3b0064550 con 0x7fb3b01430c0 2023-10-09T06:42:16.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.468+0000 7fb3acff9700 1 -- 172.21.15.163:0/4098270171 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb398011b50 con 0x7fb3b01430c0 2023-10-09T06:42:16.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.469+0000 7fb3acff9700 1 --2- 172.21.15.163:0/4098270171 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb390067320 0x7fb3900697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:16.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.469+0000 7fb3acff9700 1 -- 172.21.15.163:0/4098270171 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fb398086020 con 0x7fb3b01430c0 2023-10-09T06:42:16.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.469+0000 7fb3b4f7e700 1 --2- 172.21.15.163:0/4098270171 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb390067320 0x7fb3900697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:16.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.469+0000 7fb3acff9700 1 --2- 172.21.15.163:0/4098270171 >> [v2:172.21.15.163:6800/90866,v1:172.21.15.163:6801/90866] conn(0x7fb39006d170 0x7fb39006f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:16.479 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.469+0000 7fb3acff9700 1 -- 172.21.15.163:0/4098270171 --> [v2:172.21.15.163:6800/90866,v1:172.21.15.163:6801/90866] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb39006fc50 con 0x7fb39006d170 2023-10-09T06:42:16.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.469+0000 7fb3acff9700 1 -- 172.21.15.163:0/4098270171 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=202) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb398088580 con 0x7fb3b01430c0 2023-10-09T06:42:16.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.469+0000 7fb3b577f700 1 --2- 172.21.15.163:0/4098270171 >> [v2:172.21.15.163:6800/90866,v1:172.21.15.163:6801/90866] conn(0x7fb39006d170 0x7fb39006f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:16.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.469+0000 7fb3b4f7e700 1 --2- 172.21.15.163:0/4098270171 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb390067320 0x7fb3900697d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb3b015f7e0 tx=0x7fb3a000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:16.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.469+0000 7fb3b577f700 1 --2- 172.21.15.163:0/4098270171 >> [v2:172.21.15.163:6800/90866,v1:172.21.15.163:6801/90866] conn(0x7fb39006d170 0x7fb39006f580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:16.480 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.470+0000 7fb3acff9700 1 -- 172.21.15.163:0/4098270171 <== osd.2 v2:172.21.15.163:6800/90866 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fb39006fc50 con 0x7fb39006d170 2023-10-09T06:42:16.484 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.484+0000 7fb3b71e2700 1 -- 172.21.15.163:0/4098270171 --> [v2:172.21.15.163:6800/90866,v1:172.21.15.163:6801/90866] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb3b004c3c0 con 0x7fb39006d170 2023-10-09T06:42:16.485 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.484+0000 7fb3acff9700 1 -- 172.21.15.163:0/4098270171 <== osd.2 v2:172.21.15.163:6800/90866 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb3b004c3c0 con 0x7fb39006d170 2023-10-09T06:42:16.485 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:42:16.485 INFO:teuthology.orchestra.run.smithi163.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-09T06:42:16.485 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:42:16.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.484+0000 7fb3b71e2700 1 -- 172.21.15.163:0/4098270171 >> [v2:172.21.15.163:6800/90866,v1:172.21.15.163:6801/90866] conn(0x7fb39006d170 msgr2=0x7fb39006f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:16.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.484+0000 7fb3b71e2700 1 --2- 172.21.15.163:0/4098270171 >> [v2:172.21.15.163:6800/90866,v1:172.21.15.163:6801/90866] conn(0x7fb39006d170 0x7fb39006f580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.484+0000 7fb3b71e2700 1 -- 172.21.15.163:0/4098270171 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb390067320 msgr2=0x7fb3900697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:16.487 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.484+0000 7fb3b71e2700 1 --2- 172.21.15.163:0/4098270171 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb390067320 0x7fb3900697d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb3b015f7e0 tx=0x7fb3a000b040 comp rx=0 tx=0).stop 2023-10-09T06:42:16.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.484+0000 7fb3b71e2700 1 -- 172.21.15.163:0/4098270171 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3b01430c0 msgr2=0x7fb3b00c7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:16.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.484+0000 7fb3b71e2700 1 --2- 172.21.15.163:0/4098270171 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3b01430c0 0x7fb3b00c7870 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0067c20 tx=0x7fb398017000 comp rx=0 tx=0).stop 2023-10-09T06:42:16.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.484+0000 7fb3b577f700 1 -- 172.21.15.163:0/4098270171 reap_dead start 2023-10-09T06:42:16.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.484+0000 7fb3b71e2700 1 -- 172.21.15.163:0/4098270171 shutdown_connections 2023-10-09T06:42:16.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.484+0000 7fb3b71e2700 1 -- 172.21.15.163:0/4098270171 >> 172.21.15.163:0/4098270171 conn(0x7fb3b0068e60 msgr2=0x7fb3b00704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:16.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.484+0000 7fb3b71e2700 1 -- 172.21.15.163:0/4098270171 shutdown_connections 2023-10-09T06:42:16.488 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.484+0000 7fb3b71e2700 1 -- 172.21.15.163:0/4098270171 wait complete. 2023-10-09T06:42:16.498 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-09T06:42:16.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.612+0000 7f4d45788700 1 Processor -- start 2023-10-09T06:42:16.613 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.612+0000 7f4d45788700 1 -- start start 2023-10-09T06:42:16.620 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.613+0000 7f4d45788700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4d40159890 0x7f4d4015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:16.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.613+0000 7f4d45788700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4d40159c60 0x7f4d4015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:16.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.613+0000 7f4d45788700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d4015db10 0x7f4d40069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:16.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.613+0000 7f4d45788700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d4015df50 con 0x7f4d40159c60 2023-10-09T06:42:16.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.613+0000 7f4d45788700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d4015a160 con 0x7f4d40159890 2023-10-09T06:42:16.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.613+0000 7f4d45788700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d400d13a0 con 0x7f4d4015db10 2023-10-09T06:42:16.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.613+0000 7f4d3e7fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4d40159c60 0x7f4d4015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:16.621 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.613+0000 7f4d3e7fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4d40159c60 0x7f4d4015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55160/0 (socket says 172.21.15.163:55160) 2023-10-09T06:42:16.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.613+0000 7f4d3e7fc700 1 -- 172.21.15.163:0/1385135745 learned_addr learned my addr 172.21.15.163:0/1385135745 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:16.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.613+0000 7f4d3f7fe700 1 --2- 172.21.15.163:0/1385135745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d4015db10 0x7f4d40069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:16.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.613+0000 7f4d3effd700 1 --2- 172.21.15.163:0/1385135745 >> v2:172.21.15.181:3300/0 conn(0x7f4d40159890 0x7f4d4015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:16.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.613+0000 7f4d3e7fc700 1 -- 172.21.15.163:0/1385135745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d4015db10 msgr2=0x7f4d40069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:16.622 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.613+0000 7f4d3e7fc700 1 --2- 172.21.15.163:0/1385135745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d4015db10 0x7f4d40069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.613+0000 7f4d3e7fc700 1 -- 172.21.15.163:0/1385135745 >> v2:172.21.15.181:3300/0 conn(0x7f4d40159890 msgr2=0x7f4d4015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:16.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.613+0000 7f4d3e7fc700 1 --2- 172.21.15.163:0/1385135745 >> v2:172.21.15.181:3300/0 conn(0x7f4d40159890 0x7f4d4015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.613+0000 7f4d3e7fc700 1 -- 172.21.15.163:0/1385135745 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4015e140 con 0x7f4d40159c60 2023-10-09T06:42:16.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.613+0000 7f4d3e7fc700 1 --2- 172.21.15.163:0/1385135745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d40159c60 0x7f4d4015d5d0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f4d400d1090 tx=0x7f4d34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ab910822d0a67c server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:16.623 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.613+0000 7f4d3dffb700 1 -- 172.21.15.163:0/1385135745 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3400b640 con 0x7f4d40159c60 2023-10-09T06:42:16.624 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.613+0000 7f4d3dffb700 1 -- 172.21.15.163:0/1385135745 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d3400b7a0 con 0x7f4d40159c60 2023-10-09T06:42:16.624 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.613+0000 7f4d3dffb700 1 -- 172.21.15.163:0/1385135745 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3400bb60 con 0x7f4d40159c60 2023-10-09T06:42:16.624 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.614+0000 7f4d45788700 1 -- 172.21.15.163:0/1385135745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d40159c60 msgr2=0x7f4d4015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:16.624 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.614+0000 7f4d45788700 1 --2- 172.21.15.163:0/1385135745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d40159c60 0x7f4d4015d5d0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f4d400d1090 tx=0x7f4d34009250 comp rx=0 tx=0).stop 2023-10-09T06:42:16.624 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.614+0000 7f4d45788700 1 -- 172.21.15.163:0/1385135745 shutdown_connections 2023-10-09T06:42:16.624 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.614+0000 7f4d45788700 1 --2- 172.21.15.163:0/1385135745 >> v2:172.21.15.181:3300/0 conn(0x7f4d40159890 0x7f4d4015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.624 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.614+0000 7f4d45788700 1 --2- 172.21.15.163:0/1385135745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d4015db10 0x7f4d40069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.614+0000 7f4d45788700 1 --2- 172.21.15.163:0/1385135745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d40159c60 0x7f4d4015d5d0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.614+0000 7f4d45788700 1 -- 172.21.15.163:0/1385135745 >> 172.21.15.163:0/1385135745 conn(0x7f4d40068e20 msgr2=0x7f4d400d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:16.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.614+0000 7f4d45788700 1 -- 172.21.15.163:0/1385135745 shutdown_connections 2023-10-09T06:42:16.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.614+0000 7f4d45788700 1 -- 172.21.15.163:0/1385135745 wait complete. 2023-10-09T06:42:16.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.615+0000 7f4d45788700 1 Processor -- start 2023-10-09T06:42:16.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.615+0000 7f4d45788700 1 -- start start 2023-10-09T06:42:16.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.616+0000 7f4d45788700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d40159890 0x7f4d400b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:16.625 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.616+0000 7f4d45788700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d40159c60 0x7f4d400b2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:16.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.616+0000 7f4d45788700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d4015db10 0x7f4d400b5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:16.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.616+0000 7f4d45788700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d400b05d0 con 0x7f4d40159890 2023-10-09T06:42:16.626 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.616+0000 7f4d45788700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d400b46a0 con 0x7f4d4015db10 2023-10-09T06:42:16.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.616+0000 7f4d45788700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d400b4980 con 0x7f4d40159c60 2023-10-09T06:42:16.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.616+0000 7f4d3effd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d40159890 0x7f4d400b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:16.627 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.616+0000 7f4d3effd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d40159890 0x7f4d400b3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55176/0 (socket says 172.21.15.163:55176) 2023-10-09T06:42:16.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.616+0000 7f4d3effd700 1 -- 172.21.15.163:0/2819620961 learned_addr learned my addr 172.21.15.163:0/2819620961 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:16.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.616+0000 7f4d3e7fc700 1 --2- 172.21.15.163:0/2819620961 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d40159c60 0x7f4d400b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:16.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.616+0000 7f4d3f7fe700 1 --2- 172.21.15.163:0/2819620961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d4015db10 0x7f4d400b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:16.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.616+0000 7f4d3effd700 1 -- 172.21.15.163:0/2819620961 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d40159c60 msgr2=0x7f4d400b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:16.628 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.616+0000 7f4d3effd700 1 --2- 172.21.15.163:0/2819620961 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d40159c60 0x7f4d400b2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.616+0000 7f4d3effd700 1 -- 172.21.15.163:0/2819620961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d4015db10 msgr2=0x7f4d400b5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:16.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.616+0000 7f4d3effd700 1 --2- 172.21.15.163:0/2819620961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d4015db10 0x7f4d400b5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.616+0000 7f4d3effd700 1 -- 172.21.15.163:0/2819620961 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d340094f0 con 0x7f4d40159890 2023-10-09T06:42:16.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.616+0000 7f4d3e7fc700 1 --2- 172.21.15.163:0/2819620961 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d40159c60 0x7f4d400b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:42:16.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.616+0000 7f4d3f7fe700 1 --2- 172.21.15.163:0/2819620961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d4015db10 0x7f4d400b5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:42:16.629 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.617+0000 7f4d3effd700 1 --2- 172.21.15.163:0/2819620961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d40159890 0x7f4d400b3fa0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f4d3000e930 tx=0x7f4d300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:16.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.617+0000 7f4d26ffd700 1 -- 172.21.15.163:0/2819620961 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3000eed0 con 0x7f4d40159890 2023-10-09T06:42:16.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.617+0000 7f4d26ffd700 1 -- 172.21.15.163:0/2819620961 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d3000c850 con 0x7f4d40159890 2023-10-09T06:42:16.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.617+0000 7f4d45788700 1 -- 172.21.15.163:0/2819620961 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d400b07e0 con 0x7f4d40159890 2023-10-09T06:42:16.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.617+0000 7f4d26ffd700 1 -- 172.21.15.163:0/2819620961 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3000cbf0 con 0x7f4d40159890 2023-10-09T06:42:16.630 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.617+0000 7f4d45788700 1 -- 172.21.15.163:0/2819620961 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d400c8560 con 0x7f4d40159890 2023-10-09T06:42:16.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.618+0000 7f4d45788700 1 -- 172.21.15.163:0/2819620961 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d400b4ba0 con 0x7f4d40159890 2023-10-09T06:42:16.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.618+0000 7f4d26ffd700 1 -- 172.21.15.163:0/2819620961 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4d30013470 con 0x7f4d40159890 2023-10-09T06:42:16.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.618+0000 7f4d26ffd700 1 --2- 172.21.15.163:0/2819620961 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4d2c067320 0x7f4d2c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:16.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.618+0000 7f4d26ffd700 1 -- 172.21.15.163:0/2819620961 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f4d30084ba0 con 0x7f4d40159890 2023-10-09T06:42:16.631 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.619+0000 7f4d3e7fc700 1 --2- 172.21.15.163:0/2819620961 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4d2c067320 0x7f4d2c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:16.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.619+0000 7f4d3e7fc700 1 --2- 172.21.15.163:0/2819620961 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4d2c067320 0x7f4d2c0697d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f4d400d1090 tx=0x7f4d34017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:16.632 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.623+0000 7f4d26ffd700 1 -- 172.21.15.163:0/2819620961 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d30051b30 con 0x7f4d40159890 2023-10-09T06:42:16.804 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.803+0000 7f4d45788700 1 -- 172.21.15.163:0/2819620961 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d4004c3c0 con 0x7f4d2c067320 2023-10-09T06:42:16.805 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.804+0000 7f4d26ffd700 1 -- 172.21.15.163:0/2819620961 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4d4004c3c0 con 0x7f4d2c067320 2023-10-09T06:42:16.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.805+0000 7f4d45788700 1 -- 172.21.15.163:0/2819620961 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4d2c067320 msgr2=0x7f4d2c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:16.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.805+0000 7f4d45788700 1 --2- 172.21.15.163:0/2819620961 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4d2c067320 0x7f4d2c0697d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f4d400d1090 tx=0x7f4d34017000 comp rx=0 tx=0).stop 2023-10-09T06:42:16.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.805+0000 7f4d45788700 1 -- 172.21.15.163:0/2819620961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d40159890 msgr2=0x7f4d400b3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:16.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.805+0000 7f4d45788700 1 --2- 172.21.15.163:0/2819620961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d40159890 0x7f4d400b3fa0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f4d3000e930 tx=0x7f4d300091a0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.807 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.806+0000 7f4d45788700 1 -- 172.21.15.163:0/2819620961 shutdown_connections 2023-10-09T06:42:16.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.806+0000 7f4d45788700 1 --2- 172.21.15.163:0/2819620961 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4d2c067320 0x7f4d2c0697d0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.806+0000 7f4d45788700 1 --2- 172.21.15.163:0/2819620961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d40159890 0x7f4d400b3fa0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.806+0000 7f4d45788700 1 --2- 172.21.15.163:0/2819620961 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d4015db10 0x7f4d400b5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.806+0000 7f4d45788700 1 --2- 172.21.15.163:0/2819620961 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d40159c60 0x7f4d400b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.806+0000 7f4d45788700 1 -- 172.21.15.163:0/2819620961 >> 172.21.15.163:0/2819620961 conn(0x7f4d40068e20 msgr2=0x7f4d400cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:16.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.806+0000 7f4d45788700 1 -- 172.21.15.163:0/2819620961 shutdown_connections 2023-10-09T06:42:16.808 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.806+0000 7f4d45788700 1 -- 172.21.15.163:0/2819620961 wait complete. 2023-10-09T06:42:16.809 INFO:teuthology.orchestra.run.smithi163.stderr:instructed osd(s) 2 to deep-scrub 2023-10-09T06:42:16.820 INFO:tasks.ceph:Scrubbing osd.3 2023-10-09T06:42:16.820 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-09T06:42:16.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.939+0000 7f401ef6a700 1 Processor -- start 2023-10-09T06:42:16.940 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.939+0000 7f401ef6a700 1 -- start start 2023-10-09T06:42:16.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.940+0000 7f401ef6a700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f40180cd910 0x7f40180ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:16.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.940+0000 7f401ef6a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f40180b55a0 0x7f40180ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:16.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.940+0000 7f401ef6a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40180b5b00 con 0x7f4018159ec0 2023-10-09T06:42:16.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.940+0000 7f401ef6a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40180b5da0 con 0x7f40180b55a0 2023-10-09T06:42:16.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.940+0000 7f401ef6a700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40180b6080 con 0x7f40180cd910 2023-10-09T06:42:16.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.940+0000 7f401d507700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f40180b55a0 0x7f40180ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:16.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.940+0000 7f401d507700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f40180b55a0 0x7f40180ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55190/0 (socket says 172.21.15.163:55190) 2023-10-09T06:42:16.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.940+0000 7f401d507700 1 -- 172.21.15.163:0/1337446848 learned_addr learned my addr 172.21.15.163:0/1337446848 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:16.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.940+0000 7f401d507700 1 -- 172.21.15.163:0/1337446848 >> v2:172.21.15.181:3300/0 conn(0x7f40180cd910 msgr2=0x7f40180ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:16.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.940+0000 7f401d507700 1 --2- 172.21.15.163:0/1337446848 >> v2:172.21.15.181:3300/0 conn(0x7f40180cd910 0x7f40180ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.940+0000 7f401d507700 1 -- 172.21.15.163:0/1337446848 >> v1:172.21.15.181:6789/0 conn(0x7f4018159ec0 legacy=0x7f40180b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:16.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.940+0000 7f401d507700 1 -- 172.21.15.163:0/1337446848 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40180cc100 con 0x7f40180b55a0 2023-10-09T06:42:16.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.940+0000 7f401d507700 1 --2- 172.21.15.163:0/1337446848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40180b55a0 0x7f40180ce880 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f40000099b0 tx=0x7f4000009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa2bcdf6bb15935 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:16.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.940+0000 7f40177fe700 1 -- 172.21.15.163:0/1337446848 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400000b7e0 con 0x7f40180b55a0 2023-10-09T06:42:16.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.941+0000 7f40177fe700 1 -- 172.21.15.163:0/1337446848 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400000b940 con 0x7f40180b55a0 2023-10-09T06:42:16.944 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.941+0000 7f40177fe700 1 -- 172.21.15.163:0/1337446848 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400000bce0 con 0x7f40180b55a0 2023-10-09T06:42:16.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.941+0000 7f401ef6a700 1 -- 172.21.15.163:0/1337446848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40180b55a0 msgr2=0x7f40180ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:16.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.941+0000 7f401ef6a700 1 --2- 172.21.15.163:0/1337446848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40180b55a0 0x7f40180ce880 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f40000099b0 tx=0x7f4000009250 comp rx=0 tx=0).stop 2023-10-09T06:42:16.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.941+0000 7f401ef6a700 1 -- 172.21.15.163:0/1337446848 shutdown_connections 2023-10-09T06:42:16.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.941+0000 7f401ef6a700 1 --2- 172.21.15.163:0/1337446848 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40180b55a0 0x7f40180ce880 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.941+0000 7f401ef6a700 1 --2- 172.21.15.163:0/1337446848 >> v2:172.21.15.181:3300/0 conn(0x7f40180cd910 0x7f40180ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.941+0000 7f401ef6a700 1 -- 172.21.15.163:0/1337446848 >> 172.21.15.163:0/1337446848 conn(0x7f40180999f0 msgr2=0x7f40180b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:16.945 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.941+0000 7f401ef6a700 1 -- 172.21.15.163:0/1337446848 shutdown_connections 2023-10-09T06:42:16.946 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.941+0000 7f401ef6a700 1 -- 172.21.15.163:0/1337446848 wait complete. 2023-10-09T06:42:16.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.942+0000 7f401ef6a700 1 Processor -- start 2023-10-09T06:42:16.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.942+0000 7f401ef6a700 1 -- start start 2023-10-09T06:42:16.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.943+0000 7f401ef6a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40180b55a0 0x7f4018156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:16.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.943+0000 7f401ef6a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4018159ec0 0x7f4018148200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:16.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.943+0000 7f401ef6a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4018156100 0x7f401815ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:16.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.943+0000 7f401ef6a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4018156f70 con 0x7f4018159ec0 2023-10-09T06:42:16.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.943+0000 7f401ef6a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401816e2b0 con 0x7f40180b55a0 2023-10-09T06:42:16.949 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.943+0000 7f401ef6a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4018148740 con 0x7f4018156100 2023-10-09T06:42:16.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.943+0000 7f4017fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4018159ec0 0x7f4018148200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:16.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.943+0000 7f4017fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4018159ec0 0x7f4018148200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55192/0 (socket says 172.21.15.163:55192) 2023-10-09T06:42:16.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.943+0000 7f4017fff700 1 -- 172.21.15.163:0/3653564906 learned_addr learned my addr 172.21.15.163:0/3653564906 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:16.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.943+0000 7f401d507700 1 --2- 172.21.15.163:0/3653564906 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4018156100 0x7f401815ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:16.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.943+0000 7f401cd06700 1 --2- 172.21.15.163:0/3653564906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40180b55a0 0x7f4018156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:16.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.943+0000 7f4017fff700 1 -- 172.21.15.163:0/3653564906 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4018156100 msgr2=0x7f401815ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:16.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.943+0000 7f4017fff700 1 --2- 172.21.15.163:0/3653564906 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4018156100 0x7f401815ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.950 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.943+0000 7f4017fff700 1 -- 172.21.15.163:0/3653564906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40180b55a0 msgr2=0x7f4018156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:16.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.943+0000 7f4017fff700 1 --2- 172.21.15.163:0/3653564906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40180b55a0 0x7f4018156600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.943+0000 7f4017fff700 1 -- 172.21.15.163:0/3653564906 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40000094f0 con 0x7f4018159ec0 2023-10-09T06:42:16.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.943+0000 7f401cd06700 1 --2- 172.21.15.163:0/3653564906 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40180b55a0 0x7f4018156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:42:16.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.944+0000 7f4017fff700 1 --2- 172.21.15.163:0/3653564906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4018159ec0 0x7f4018148200 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f400800b810 tx=0x7f4008009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:16.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.944+0000 7f4014ff9700 1 -- 172.21.15.163:0/3653564906 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400800f5f0 con 0x7f4018159ec0 2023-10-09T06:42:16.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.944+0000 7f4014ff9700 1 -- 172.21.15.163:0/3653564906 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400800f750 con 0x7f4018159ec0 2023-10-09T06:42:16.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.944+0000 7f401ef6a700 1 -- 172.21.15.163:0/3653564906 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f401816dd00 con 0x7f4018159ec0 2023-10-09T06:42:16.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.944+0000 7f4014ff9700 1 -- 172.21.15.163:0/3653564906 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400800fb10 con 0x7f4018159ec0 2023-10-09T06:42:16.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.944+0000 7f401ef6a700 1 -- 172.21.15.163:0/3653564906 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f401815a450 con 0x7f4018159ec0 2023-10-09T06:42:16.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.945+0000 7f401ef6a700 1 -- 172.21.15.163:0/3653564906 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4018063c90 con 0x7f4018159ec0 2023-10-09T06:42:16.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.945+0000 7f4014ff9700 1 -- 172.21.15.163:0/3653564906 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4008010480 con 0x7f4018159ec0 2023-10-09T06:42:16.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.946+0000 7f4014ff9700 1 --2- 172.21.15.163:0/3653564906 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3ff8067320 0x7f3ff80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:16.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.946+0000 7f4014ff9700 1 -- 172.21.15.163:0/3653564906 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f4008084880 con 0x7f4018159ec0 2023-10-09T06:42:16.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.946+0000 7f4014ff9700 1 --2- 172.21.15.163:0/3653564906 >> [v2:172.21.15.163:6812/90865,v1:172.21.15.163:6814/90865] conn(0x7f3ff806d170 0x7f3ff806f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:16.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.946+0000 7f401cd06700 1 --2- 172.21.15.163:0/3653564906 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3ff8067320 0x7f3ff80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:16.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.946+0000 7f4014ff9700 1 -- 172.21.15.163:0/3653564906 --> [v2:172.21.15.163:6812/90865,v1:172.21.15.163:6814/90865] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3ff806fc50 con 0x7f3ff806d170 2023-10-09T06:42:16.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.946+0000 7f4014ff9700 1 -- 172.21.15.163:0/3653564906 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=202) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4008086de0 con 0x7f4018159ec0 2023-10-09T06:42:16.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.946+0000 7f401d507700 1 --2- 172.21.15.163:0/3653564906 >> [v2:172.21.15.163:6812/90865,v1:172.21.15.163:6814/90865] conn(0x7f3ff806d170 0x7f3ff806f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:16.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.946+0000 7f401cd06700 1 --2- 172.21.15.163:0/3653564906 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3ff8067320 0x7f3ff80697d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f401808aad0 tx=0x7f4010007510 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:16.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.946+0000 7f401d507700 1 --2- 172.21.15.163:0/3653564906 >> [v2:172.21.15.163:6812/90865,v1:172.21.15.163:6814/90865] conn(0x7f3ff806d170 0x7f3ff806f580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:16.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.947+0000 7f4014ff9700 1 -- 172.21.15.163:0/3653564906 <== osd.3 v2:172.21.15.163:6812/90865 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f3ff806fc50 con 0x7f3ff806d170 2023-10-09T06:42:16.961 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.960+0000 7f401ef6a700 1 -- 172.21.15.163:0/3653564906 --> [v2:172.21.15.163:6812/90865,v1:172.21.15.163:6814/90865] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f40180718b0 con 0x7f3ff806d170 2023-10-09T06:42:16.962 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.960+0000 7f4014ff9700 1 -- 172.21.15.163:0/3653564906 <== osd.3 v2:172.21.15.163:6812/90865 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f40180718b0 con 0x7f3ff806d170 2023-10-09T06:42:16.962 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:42:16.962 INFO:teuthology.orchestra.run.smithi163.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-09T06:42:16.963 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:42:16.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.961+0000 7f401ef6a700 1 -- 172.21.15.163:0/3653564906 >> [v2:172.21.15.163:6812/90865,v1:172.21.15.163:6814/90865] conn(0x7f3ff806d170 msgr2=0x7f3ff806f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:16.964 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.961+0000 7f401ef6a700 1 --2- 172.21.15.163:0/3653564906 >> [v2:172.21.15.163:6812/90865,v1:172.21.15.163:6814/90865] conn(0x7f3ff806d170 0x7f3ff806f580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:16.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.961+0000 7f401ef6a700 1 -- 172.21.15.163:0/3653564906 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3ff8067320 msgr2=0x7f3ff80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:16.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.961+0000 7f401ef6a700 1 --2- 172.21.15.163:0/3653564906 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3ff8067320 0x7f3ff80697d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f401808aad0 tx=0x7f4010007510 comp rx=0 tx=0).stop 2023-10-09T06:42:16.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.961+0000 7f401ef6a700 1 -- 172.21.15.163:0/3653564906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4018159ec0 msgr2=0x7f4018148200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:16.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.961+0000 7f401ef6a700 1 --2- 172.21.15.163:0/3653564906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4018159ec0 0x7f4018148200 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f400800b810 tx=0x7f4008009220 comp rx=0 tx=0).stop 2023-10-09T06:42:16.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.961+0000 7f401d507700 1 -- 172.21.15.163:0/3653564906 reap_dead start 2023-10-09T06:42:16.965 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.961+0000 7f401ef6a700 1 -- 172.21.15.163:0/3653564906 shutdown_connections 2023-10-09T06:42:16.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.961+0000 7f401ef6a700 1 -- 172.21.15.163:0/3653564906 >> 172.21.15.163:0/3653564906 conn(0x7f40180999f0 msgr2=0x7f40180cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:16.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.961+0000 7f401ef6a700 1 -- 172.21.15.163:0/3653564906 shutdown_connections 2023-10-09T06:42:16.966 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:16.961+0000 7f401ef6a700 1 -- 172.21.15.163:0/3653564906 wait complete. 2023-10-09T06:42:16.973 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-09T06:42:17.090 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.089+0000 7ff31b392700 1 Processor -- start 2023-10-09T06:42:17.091 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.089+0000 7ff31b392700 1 -- start start 2023-10-09T06:42:17.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.090+0000 7ff31b392700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff31406bcd0 0x7ff3140caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.090+0000 7ff31b392700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff31415df50 0x7ff31415e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.090+0000 7ff31b392700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff31415d7e0 con 0x7ff31406c0a0 2023-10-09T06:42:17.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.090+0000 7ff31b392700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff31415dac0 con 0x7ff31415df50 2023-10-09T06:42:17.093 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.090+0000 7ff31b392700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff31415aa30 con 0x7ff31406bcd0 2023-10-09T06:42:17.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.090+0000 7ff31992f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff31415df50 0x7ff31415e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.090+0000 7ff31992f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff31415df50 0x7ff31415e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55202/0 (socket says 172.21.15.163:55202) 2023-10-09T06:42:17.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.090+0000 7ff31992f700 1 -- 172.21.15.163:0/1316825160 learned_addr learned my addr 172.21.15.163:0/1316825160 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:17.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.090+0000 7ff31912e700 1 --2- 172.21.15.163:0/1316825160 >> v2:172.21.15.181:3300/0 conn(0x7ff31406bcd0 0x7ff3140caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.091+0000 7ff31992f700 1 -- 172.21.15.163:0/1316825160 >> v2:172.21.15.181:3300/0 conn(0x7ff31406bcd0 msgr2=0x7ff3140caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:17.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.091+0000 7ff31992f700 1 --2- 172.21.15.163:0/1316825160 >> v2:172.21.15.181:3300/0 conn(0x7ff31406bcd0 0x7ff3140caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.094 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.091+0000 7ff31992f700 1 -- 172.21.15.163:0/1316825160 >> v1:172.21.15.181:6789/0 conn(0x7ff31406c0a0 legacy=0x7ff31415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:17.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.091+0000 7ff31992f700 1 -- 172.21.15.163:0/1316825160 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31415ad10 con 0x7ff31415df50 2023-10-09T06:42:17.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.091+0000 7ff31912e700 1 --2- 172.21.15.163:0/1316825160 >> v2:172.21.15.181:3300/0 conn(0x7ff31406bcd0 0x7ff3140caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:42:17.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.091+0000 7ff31992f700 1 --2- 172.21.15.163:0/1316825160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff31415df50 0x7ff31415e3b0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7ff3140691a0 tx=0x7ff30c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3850b03991b528e server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:17.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.091+0000 7ff30bfff700 1 -- 172.21.15.163:0/1316825160 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c00e070 con 0x7ff31415df50 2023-10-09T06:42:17.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.091+0000 7ff30bfff700 1 -- 172.21.15.163:0/1316825160 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30c00ba50 con 0x7ff31415df50 2023-10-09T06:42:17.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.091+0000 7ff30bfff700 1 -- 172.21.15.163:0/1316825160 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c0092e0 con 0x7ff31415df50 2023-10-09T06:42:17.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.091+0000 7ff31b392700 1 -- 172.21.15.163:0/1316825160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff31415df50 msgr2=0x7ff31415e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:17.095 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.091+0000 7ff31b392700 1 --2- 172.21.15.163:0/1316825160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff31415df50 0x7ff31415e3b0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7ff3140691a0 tx=0x7ff30c009840 comp rx=0 tx=0).stop 2023-10-09T06:42:17.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.092+0000 7ff31b392700 1 -- 172.21.15.163:0/1316825160 shutdown_connections 2023-10-09T06:42:17.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.092+0000 7ff31b392700 1 --2- 172.21.15.163:0/1316825160 >> v2:172.21.15.181:3300/0 conn(0x7ff31406bcd0 0x7ff3140caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.092+0000 7ff31b392700 1 --2- 172.21.15.163:0/1316825160 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff31415df50 0x7ff31415e3b0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.092+0000 7ff31b392700 1 -- 172.21.15.163:0/1316825160 >> 172.21.15.163:0/1316825160 conn(0x7ff3140ce600 msgr2=0x7ff3140cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:17.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.092+0000 7ff31b392700 1 -- 172.21.15.163:0/1316825160 shutdown_connections 2023-10-09T06:42:17.096 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.092+0000 7ff31b392700 1 -- 172.21.15.163:0/1316825160 wait complete. 2023-10-09T06:42:17.099 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.093+0000 7ff31b392700 1 Processor -- start 2023-10-09T06:42:17.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.093+0000 7ff31b392700 1 -- start start 2023-10-09T06:42:17.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.094+0000 7ff31b392700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff31406bcd0 0x7ff3141549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.094+0000 7ff31b392700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff31406c0a0 0x7ff3140b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.094+0000 7ff31b392700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff31415df50 0x7ff3140b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.094+0000 7ff31b392700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3140b2b20 con 0x7ff31406bcd0 2023-10-09T06:42:17.100 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.094+0000 7ff31b392700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3140b1f10 con 0x7ff31415df50 2023-10-09T06:42:17.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.094+0000 7ff31b392700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3140b21f0 con 0x7ff31406c0a0 2023-10-09T06:42:17.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.094+0000 7ff31912e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff31406bcd0 0x7ff3141549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.094+0000 7ff31912e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff31406bcd0 0x7ff3141549f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55214/0 (socket says 172.21.15.163:55214) 2023-10-09T06:42:17.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.094+0000 7ff31912e700 1 -- 172.21.15.163:0/1686022790 learned_addr learned my addr 172.21.15.163:0/1686022790 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:17.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.094+0000 7ff31992f700 1 --2- 172.21.15.163:0/1686022790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff31415df50 0x7ff3140b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.094+0000 7ff31892d700 1 --2- 172.21.15.163:0/1686022790 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff31406c0a0 0x7ff3140b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.101 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.094+0000 7ff31912e700 1 -- 172.21.15.163:0/1686022790 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff31406c0a0 msgr2=0x7ff3140b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:17.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.094+0000 7ff31912e700 1 --2- 172.21.15.163:0/1686022790 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff31406c0a0 0x7ff3140b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.094+0000 7ff31912e700 1 -- 172.21.15.163:0/1686022790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff31415df50 msgr2=0x7ff3140b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:17.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.094+0000 7ff31912e700 1 --2- 172.21.15.163:0/1686022790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff31415df50 0x7ff3140b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.094+0000 7ff31912e700 1 -- 172.21.15.163:0/1686022790 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30c009ae0 con 0x7ff31406bcd0 2023-10-09T06:42:17.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.094+0000 7ff31912e700 1 --2- 172.21.15.163:0/1686022790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff31406bcd0 0x7ff3141549f0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7ff31000d860 tx=0x7ff310008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:17.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.094+0000 7ff3097fa700 1 -- 172.21.15.163:0/1686022790 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3100125f0 con 0x7ff31406bcd0 2023-10-09T06:42:17.102 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.094+0000 7ff3097fa700 1 -- 172.21.15.163:0/1686022790 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff310012750 con 0x7ff31406bcd0 2023-10-09T06:42:17.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.094+0000 7ff31b392700 1 -- 172.21.15.163:0/1686022790 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3140b24d0 con 0x7ff31406bcd0 2023-10-09T06:42:17.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.094+0000 7ff3097fa700 1 -- 172.21.15.163:0/1686022790 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff310012af0 con 0x7ff31406bcd0 2023-10-09T06:42:17.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.094+0000 7ff31b392700 1 -- 172.21.15.163:0/1686022790 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff314154f30 con 0x7ff31406bcd0 2023-10-09T06:42:17.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.095+0000 7ff31b392700 1 -- 172.21.15.163:0/1686022790 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3140b2c90 con 0x7ff31406bcd0 2023-10-09T06:42:17.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.096+0000 7ff3097fa700 1 -- 172.21.15.163:0/1686022790 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff310013480 con 0x7ff31406bcd0 2023-10-09T06:42:17.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.096+0000 7ff3097fa700 1 --2- 172.21.15.163:0/1686022790 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff2f8067320 0x7ff2f80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.096+0000 7ff3097fa700 1 -- 172.21.15.163:0/1686022790 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff310084ab0 con 0x7ff31406bcd0 2023-10-09T06:42:17.103 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.096+0000 7ff31892d700 1 --2- 172.21.15.163:0/1686022790 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff2f8067320 0x7ff2f80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.097+0000 7ff31892d700 1 --2- 172.21.15.163:0/1686022790 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff2f8067320 0x7ff2f80697d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff31406ad00 tx=0x7ff304007ec0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:17.104 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.100+0000 7ff3097fa700 1 -- 172.21.15.163:0/1686022790 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff310051a40 con 0x7ff31406bcd0 2023-10-09T06:42:17.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.282+0000 7ff31b392700 1 -- 172.21.15.163:0/1686022790 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff31404c3c0 con 0x7ff2f8067320 2023-10-09T06:42:17.283 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.283+0000 7ff3097fa700 1 -- 172.21.15.163:0/1686022790 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff31404c3c0 con 0x7ff2f8067320 2023-10-09T06:42:17.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.284+0000 7ff31b392700 1 -- 172.21.15.163:0/1686022790 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff2f8067320 msgr2=0x7ff2f80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:17.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.284+0000 7ff31b392700 1 --2- 172.21.15.163:0/1686022790 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff2f8067320 0x7ff2f80697d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff31406ad00 tx=0x7ff304007ec0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.284+0000 7ff31b392700 1 -- 172.21.15.163:0/1686022790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff31406bcd0 msgr2=0x7ff3141549f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:17.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.284+0000 7ff31b392700 1 --2- 172.21.15.163:0/1686022790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff31406bcd0 0x7ff3141549f0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7ff31000d860 tx=0x7ff310008040 comp rx=0 tx=0).stop 2023-10-09T06:42:17.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.285+0000 7ff31b392700 1 -- 172.21.15.163:0/1686022790 shutdown_connections 2023-10-09T06:42:17.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.285+0000 7ff31b392700 1 --2- 172.21.15.163:0/1686022790 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff2f8067320 0x7ff2f80697d0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.285+0000 7ff31b392700 1 --2- 172.21.15.163:0/1686022790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff31406bcd0 0x7ff3141549f0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.285+0000 7ff31b392700 1 --2- 172.21.15.163:0/1686022790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff31415df50 0x7ff3140b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.285+0000 7ff31b392700 1 --2- 172.21.15.163:0/1686022790 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff31406c0a0 0x7ff3140b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.285+0000 7ff31b392700 1 -- 172.21.15.163:0/1686022790 >> 172.21.15.163:0/1686022790 conn(0x7ff3140ce600 msgr2=0x7ff3140704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:17.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.285+0000 7ff31b392700 1 -- 172.21.15.163:0/1686022790 shutdown_connections 2023-10-09T06:42:17.287 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.285+0000 7ff31b392700 1 -- 172.21.15.163:0/1686022790 wait complete. 2023-10-09T06:42:17.288 INFO:teuthology.orchestra.run.smithi163.stderr:instructed osd(s) 3 to deep-scrub 2023-10-09T06:42:17.298 INFO:tasks.ceph:Scrubbing osd.4 2023-10-09T06:42:17.298 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-09T06:42:17.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.416+0000 7f79c544d700 1 Processor -- start 2023-10-09T06:42:17.417 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.416+0000 7f79c544d700 1 -- start start 2023-10-09T06:42:17.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.417+0000 7f79c544d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f79c0159ec0 0x7f79c00b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.421 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.417+0000 7f79c544d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79c00b53d0 0x7f79c00cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.417+0000 7f79c544d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79c00b5b00 con 0x7f79c00cce40 2023-10-09T06:42:17.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.417+0000 7f79c544d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79c00b5850 con 0x7f79c0159ec0 2023-10-09T06:42:17.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.417+0000 7f79c544d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c00b5da0 con 0x7f79c00b53d0 2023-10-09T06:42:17.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.417+0000 7f79bf7fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f79c00cce40 0x7f79c00cd2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60638/0 (socket says 172.21.15.163:60638) 2023-10-09T06:42:17.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.417+0000 7f79bf7fe700 1 -- 172.21.15.163:0/2704206214 learned_addr learned my addr 172.21.15.163:0/2704206214 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:42:17.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.417+0000 7f79bdffb700 1 -- 172.21.15.163:0/2704206214 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1669075282 0 0) 0x7f79c00b5b00 con 0x7f79c00cce40 2023-10-09T06:42:17.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.418+0000 7f79bdffb700 1 -- 172.21.15.163:0/2704206214 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79a8003570 con 0x7f79c00cce40 2023-10-09T06:42:17.422 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.417+0000 7f79be7fc700 1 --2- 172.21.15.163:0/2704206214 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79c00b53d0 0x7f79c00cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.418+0000 7f79beffd700 1 --2- 172.21.15.163:0/2704206214 >> v2:172.21.15.181:3300/0 conn(0x7f79c0159ec0 0x7f79c00b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.418+0000 7f79bdffb700 1 -- 172.21.15.163:0/2704206214 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2045109611 0 0) 0x7f79a8003570 con 0x7f79c00cce40 2023-10-09T06:42:17.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.418+0000 7f79bdffb700 1 -- 172.21.15.163:0/2704206214 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79c00b53d0 msgr2=0x7f79c00cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:17.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.418+0000 7f79bdffb700 1 --2- 172.21.15.163:0/2704206214 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79c00b53d0 0x7f79c00cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.418+0000 7f79bdffb700 1 -- 172.21.15.163:0/2704206214 >> v2:172.21.15.181:3300/0 conn(0x7f79c0159ec0 msgr2=0x7f79c00b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:17.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.418+0000 7f79bdffb700 1 --2- 172.21.15.163:0/2704206214 >> v2:172.21.15.181:3300/0 conn(0x7f79c0159ec0 0x7f79c00b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.418+0000 7f79bdffb700 1 -- 172.21.15.163:0/2704206214 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c00b6080 con 0x7f79c00cce40 2023-10-09T06:42:17.423 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.418+0000 7f79bdffb700 1 -- 172.21.15.163:0/2704206214 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7f79b8001470 con 0x7f79c00cce40 2023-10-09T06:42:17.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.418+0000 7f79bdffb700 1 -- 172.21.15.163:0/2704206214 >> v1:172.21.15.163:6789/0 conn(0x7f79c00cce40 legacy=0x7f79c00cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:17.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.418+0000 7f79bdffb700 1 --2- 172.21.15.163:0/2704206214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79a8003a60 0x7f79a8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.418+0000 7f79bdffb700 1 --2- 172.21.15.163:0/2704206214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79a8004810 0x7f79a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.424 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.418+0000 7f79bdffb700 1 --2- 172.21.15.163:0/2704206214 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79c00b53d0 0x7f79a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.418+0000 7f79bdffb700 1 -- 172.21.15.163:0/2704206214 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c00b5850 con 0x7f79a8003a60 2023-10-09T06:42:17.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.418+0000 7f79bdffb700 1 -- 172.21.15.163:0/2704206214 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c00b5da0 con 0x7f79a8004810 2023-10-09T06:42:17.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.418+0000 7f79bdffb700 1 -- 172.21.15.163:0/2704206214 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a800ba90 con 0x7f79c00b53d0 2023-10-09T06:42:17.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.418+0000 7f79beffd700 1 --2- 172.21.15.163:0/2704206214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79a8003a60 0x7f79a8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.418+0000 7f79be7fc700 1 --2- 172.21.15.163:0/2704206214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79a8004810 0x7f79a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.418+0000 7f79beffd700 1 -- 172.21.15.163:0/2704206214 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79c00b53d0 msgr2=0x7f79a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:17.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.418+0000 7f79bf7fe700 1 --2- 172.21.15.163:0/2704206214 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79c00b53d0 0x7f79a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.425 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.418+0000 7f79beffd700 1 --2- 172.21.15.163:0/2704206214 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79c00b53d0 0x7f79a80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.418+0000 7f79beffd700 1 -- 172.21.15.163:0/2704206214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79a8004810 msgr2=0x7f79a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:17.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.418+0000 7f79beffd700 1 --2- 172.21.15.163:0/2704206214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79a8004810 0x7f79a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.418+0000 7f79beffd700 1 -- 172.21.15.163:0/2704206214 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a8003570 con 0x7f79a8003a60 2023-10-09T06:42:17.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.418+0000 7f79be7fc700 1 --2- 172.21.15.163:0/2704206214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79a8004810 0x7f79a8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:42:17.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.418+0000 7f79bf7fe700 1 --2- 172.21.15.163:0/2704206214 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79c00b53d0 0x7f79a80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:42:17.426 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.418+0000 7f79beffd700 1 --2- 172.21.15.163:0/2704206214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79a8003a60 0x7f79a8003e70 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f79b000e2d0 tx=0x7f79b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3bec1569216f2a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:17.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.419+0000 7f79bdffb700 1 -- 172.21.15.163:0/2704206214 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b000eac0 con 0x7f79a8003a60 2023-10-09T06:42:17.428 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.419+0000 7f79bdffb700 1 -- 172.21.15.163:0/2704206214 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b000f040 con 0x7f79a8003a60 2023-10-09T06:42:17.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.419+0000 7f79bdffb700 1 -- 172.21.15.163:0/2704206214 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b000ee70 con 0x7f79a8003a60 2023-10-09T06:42:17.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.419+0000 7f79c544d700 1 -- 172.21.15.163:0/2704206214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79a8003a60 msgr2=0x7f79a8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:17.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.419+0000 7f79c544d700 1 --2- 172.21.15.163:0/2704206214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79a8003a60 0x7f79a8003e70 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f79b000e2d0 tx=0x7f79b000b040 comp rx=0 tx=0).stop 2023-10-09T06:42:17.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.419+0000 7f79bf7fe700 1 -- 172.21.15.163:0/2704206214 reap_dead start 2023-10-09T06:42:17.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.419+0000 7f79c544d700 1 -- 172.21.15.163:0/2704206214 shutdown_connections 2023-10-09T06:42:17.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.419+0000 7f79c544d700 1 -- 172.21.15.163:0/2704206214 >> 172.21.15.163:0/2704206214 conn(0x7f79c00999f0 msgr2=0x7f79c00b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:17.429 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.419+0000 7f79c544d700 1 -- 172.21.15.163:0/2704206214 shutdown_connections 2023-10-09T06:42:17.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.419+0000 7f79c544d700 1 -- 172.21.15.163:0/2704206214 wait complete. 2023-10-09T06:42:17.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.420+0000 7f79c544d700 1 Processor -- start 2023-10-09T06:42:17.430 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.420+0000 7f79c544d700 1 -- start start 2023-10-09T06:42:17.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.421+0000 7f79c544d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79a8003a60 0x7f79c0154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.421+0000 7f79c544d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79c016a1d0 0x7f79c00ce420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.433 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.421+0000 7f79c544d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79c016a640 0x7f79c00ce960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.421+0000 7f79c544d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c006a450 con 0x7f79a8003a60 2023-10-09T06:42:17.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.421+0000 7f79c544d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c0069580 con 0x7f79c016a1d0 2023-10-09T06:42:17.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.421+0000 7f79c544d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c0069840 con 0x7f79c016a640 2023-10-09T06:42:17.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.421+0000 7f79beffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79a8003a60 0x7f79c0154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.421+0000 7f79beffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79a8003a60 0x7f79c0154e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55224/0 (socket says 172.21.15.163:55224) 2023-10-09T06:42:17.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.421+0000 7f79beffd700 1 -- 172.21.15.163:0/3363900845 learned_addr learned my addr 172.21.15.163:0/3363900845 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:17.434 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.421+0000 7f79beffd700 1 -- 172.21.15.163:0/3363900845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79c016a640 msgr2=0x7f79c00ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:17.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.421+0000 7f79bf7fe700 1 --2- 172.21.15.163:0/3363900845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79c016a640 0x7f79c00ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.421+0000 7f79be7fc700 1 --2- 172.21.15.163:0/3363900845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79c016a1d0 0x7f79c00ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.422+0000 7f79beffd700 1 --2- 172.21.15.163:0/3363900845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79c016a640 0x7f79c00ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.422+0000 7f79beffd700 1 -- 172.21.15.163:0/3363900845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79c016a1d0 msgr2=0x7f79c00ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:17.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.422+0000 7f79beffd700 1 --2- 172.21.15.163:0/3363900845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79c016a1d0 0x7f79c00ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.435 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.422+0000 7f79beffd700 1 -- 172.21.15.163:0/3363900845 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c0069b20 con 0x7f79a8003a60 2023-10-09T06:42:17.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.422+0000 7f79bf7fe700 1 --2- 172.21.15.163:0/3363900845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79c016a640 0x7f79c00ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:42:17.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.422+0000 7f79be7fc700 1 --2- 172.21.15.163:0/3363900845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79c016a1d0 0x7f79c00ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:42:17.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.422+0000 7f79beffd700 1 --2- 172.21.15.163:0/3363900845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79a8003a60 0x7f79c0154e30 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f79a8004730 tx=0x7f79b001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:17.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.422+0000 7f79a6ffd700 1 -- 172.21.15.163:0/3363900845 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b001d420 con 0x7f79a8003a60 2023-10-09T06:42:17.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.422+0000 7f79a6ffd700 1 -- 172.21.15.163:0/3363900845 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b001d580 con 0x7f79a8003a60 2023-10-09T06:42:17.436 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.422+0000 7f79c544d700 1 -- 172.21.15.163:0/3363900845 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c0069e00 con 0x7f79a8003a60 2023-10-09T06:42:17.437 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.422+0000 7f79a6ffd700 1 -- 172.21.15.163:0/3363900845 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b001d980 con 0x7f79a8003a60 2023-10-09T06:42:17.437 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.422+0000 7f79c544d700 1 -- 172.21.15.163:0/3363900845 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c0152820 con 0x7f79a8003a60 2023-10-09T06:42:17.437 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.423+0000 7f79c544d700 1 -- 172.21.15.163:0/3363900845 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f79c0063c90 con 0x7f79a8003a60 2023-10-09T06:42:17.437 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.423+0000 7f79a6ffd700 1 -- 172.21.15.163:0/3363900845 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f79b0003040 con 0x7f79a8003a60 2023-10-09T06:42:17.438 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.424+0000 7f79a6ffd700 1 --2- 172.21.15.163:0/3363900845 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f79ac067320 0x7f79ac0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.438 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.424+0000 7f79a6ffd700 1 -- 172.21.15.163:0/3363900845 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f79b0087590 con 0x7f79a8003a60 2023-10-09T06:42:17.438 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.424+0000 7f79a6ffd700 1 --2- 172.21.15.163:0/3363900845 >> [v2:172.21.15.181:6824/91143,v1:172.21.15.181:6825/91143] conn(0x7f79ac06d170 0x7f79ac06f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.424+0000 7f79a6ffd700 1 -- 172.21.15.163:0/3363900845 --> [v2:172.21.15.181:6824/91143,v1:172.21.15.181:6825/91143] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f79ac06fc50 con 0x7f79ac06d170 2023-10-09T06:42:17.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.424+0000 7f79a6ffd700 1 -- 172.21.15.163:0/3363900845 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=202) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79b0087830 con 0x7f79a8003a60 2023-10-09T06:42:17.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.424+0000 7f79be7fc700 1 --2- 172.21.15.163:0/3363900845 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f79ac067320 0x7f79ac0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.439 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.424+0000 7f79bf7fe700 1 --2- 172.21.15.163:0/3363900845 >> [v2:172.21.15.181:6824/91143,v1:172.21.15.181:6825/91143] conn(0x7f79ac06d170 0x7f79ac06f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.424+0000 7f79be7fc700 1 --2- 172.21.15.163:0/3363900845 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f79ac067320 0x7f79ac0697d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f79b4006f00 tx=0x7f79b4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:17.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.424+0000 7f79bf7fe700 1 --2- 172.21.15.163:0/3363900845 >> [v2:172.21.15.181:6824/91143,v1:172.21.15.181:6825/91143] conn(0x7f79ac06d170 0x7f79ac06f580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:17.440 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.425+0000 7f79a6ffd700 1 -- 172.21.15.163:0/3363900845 <== osd.4 v2:172.21.15.181:6824/91143 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f79ac06fc50 con 0x7f79ac06d170 2023-10-09T06:42:17.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.439+0000 7f79c544d700 1 -- 172.21.15.163:0/3363900845 --> [v2:172.21.15.181:6824/91143,v1:172.21.15.181:6825/91143] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f79c00718b0 con 0x7f79ac06d170 2023-10-09T06:42:17.441 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.439+0000 7f79a6ffd700 1 -- 172.21.15.163:0/3363900845 <== osd.4 v2:172.21.15.181:6824/91143 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f79c00718b0 con 0x7f79ac06d170 2023-10-09T06:42:17.442 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:42:17.442 INFO:teuthology.orchestra.run.smithi163.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-09T06:42:17.442 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:42:17.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.439+0000 7f79c544d700 1 -- 172.21.15.163:0/3363900845 >> [v2:172.21.15.181:6824/91143,v1:172.21.15.181:6825/91143] conn(0x7f79ac06d170 msgr2=0x7f79ac06f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:17.444 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.439+0000 7f79c544d700 1 --2- 172.21.15.163:0/3363900845 >> [v2:172.21.15.181:6824/91143,v1:172.21.15.181:6825/91143] conn(0x7f79ac06d170 0x7f79ac06f580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.439+0000 7f79c544d700 1 -- 172.21.15.163:0/3363900845 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f79ac067320 msgr2=0x7f79ac0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:17.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.439+0000 7f79c544d700 1 --2- 172.21.15.163:0/3363900845 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f79ac067320 0x7f79ac0697d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f79b4006f00 tx=0x7f79b4008040 comp rx=0 tx=0).stop 2023-10-09T06:42:17.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.439+0000 7f79c544d700 1 -- 172.21.15.163:0/3363900845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79a8003a60 msgr2=0x7f79c0154e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:17.445 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.439+0000 7f79c544d700 1 --2- 172.21.15.163:0/3363900845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79a8003a60 0x7f79c0154e30 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f79a8004730 tx=0x7f79b001a000 comp rx=0 tx=0).stop 2023-10-09T06:42:17.446 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.439+0000 7f79bf7fe700 1 -- 172.21.15.163:0/3363900845 reap_dead start 2023-10-09T06:42:17.451 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.439+0000 7f79c544d700 1 -- 172.21.15.163:0/3363900845 shutdown_connections 2023-10-09T06:42:17.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.439+0000 7f79c544d700 1 -- 172.21.15.163:0/3363900845 >> 172.21.15.163:0/3363900845 conn(0x7f79c00999f0 msgr2=0x7f79c0158fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:17.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.439+0000 7f79c544d700 1 -- 172.21.15.163:0/3363900845 shutdown_connections 2023-10-09T06:42:17.452 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.439+0000 7f79c544d700 1 -- 172.21.15.163:0/3363900845 wait complete. 2023-10-09T06:42:17.454 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-09T06:42:17.569 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.568+0000 7f224f298700 1 Processor -- start 2023-10-09T06:42:17.569 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.568+0000 7f224f298700 1 -- start start 2023-10-09T06:42:17.572 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.569+0000 7f224f298700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2248159890 0x7f224815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.572 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.569+0000 7f224f298700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2248159c60 0x7f224815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.569+0000 7f224f298700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f224815db10 0x7f2248069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.569+0000 7f224f298700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f224815a160 con 0x7f2248159890 2023-10-09T06:42:17.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.569+0000 7f224f298700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22480d13a0 con 0x7f224815db10 2023-10-09T06:42:17.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.569+0000 7f224f298700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f224815e140 con 0x7f2248159c60 2023-10-09T06:42:17.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.569+0000 7f224d034700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2248159890 0x7f224815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.569+0000 7f224d034700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2248159890 0x7f224815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55234/0 (socket says 172.21.15.163:55234) 2023-10-09T06:42:17.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.569+0000 7f224d034700 1 -- 172.21.15.163:0/1447848115 learned_addr learned my addr 172.21.15.163:0/1447848115 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:17.575 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.569+0000 7f224c833700 1 --2- 172.21.15.163:0/1447848115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2248159c60 0x7f224815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.569+0000 7f224d835700 1 --2- 172.21.15.163:0/1447848115 >> v2:172.21.15.181:3300/0 conn(0x7f224815db10 0x7f2248069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.570+0000 7f224d034700 1 -- 172.21.15.163:0/1447848115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2248159c60 msgr2=0x7f224815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:17.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.570+0000 7f224d034700 1 --2- 172.21.15.163:0/1447848115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2248159c60 0x7f224815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.570+0000 7f224d034700 1 -- 172.21.15.163:0/1447848115 >> v2:172.21.15.181:3300/0 conn(0x7f224815db10 msgr2=0x7f2248069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:17.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.570+0000 7f224d034700 1 --2- 172.21.15.163:0/1447848115 >> v2:172.21.15.181:3300/0 conn(0x7f224815db10 0x7f2248069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.570+0000 7f224d034700 1 -- 172.21.15.163:0/1447848115 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f224815a3d0 con 0x7f2248159890 2023-10-09T06:42:17.576 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.570+0000 7f224c833700 1 --2- 172.21.15.163:0/1447848115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2248159c60 0x7f224815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:42:17.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.570+0000 7f224d034700 1 --2- 172.21.15.163:0/1447848115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2248159890 0x7f224815abd0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f2248089b50 tx=0x7f223c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f64d798ad7ede7 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:17.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.570+0000 7f2247fff700 1 -- 172.21.15.163:0/1447848115 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c00e070 con 0x7f2248159890 2023-10-09T06:42:17.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.570+0000 7f2247fff700 1 -- 172.21.15.163:0/1447848115 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223c00ba50 con 0x7f2248159890 2023-10-09T06:42:17.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.570+0000 7f2247fff700 1 -- 172.21.15.163:0/1447848115 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c0092e0 con 0x7f2248159890 2023-10-09T06:42:17.577 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.570+0000 7f224f298700 1 -- 172.21.15.163:0/1447848115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2248159890 msgr2=0x7f224815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:17.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.570+0000 7f224f298700 1 --2- 172.21.15.163:0/1447848115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2248159890 0x7f224815abd0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f2248089b50 tx=0x7f223c009840 comp rx=0 tx=0).stop 2023-10-09T06:42:17.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.570+0000 7f224f298700 1 -- 172.21.15.163:0/1447848115 shutdown_connections 2023-10-09T06:42:17.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.570+0000 7f224f298700 1 --2- 172.21.15.163:0/1447848115 >> v2:172.21.15.181:3300/0 conn(0x7f224815db10 0x7f2248069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.570+0000 7f224f298700 1 --2- 172.21.15.163:0/1447848115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2248159890 0x7f224815abd0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.570+0000 7f224f298700 1 --2- 172.21.15.163:0/1447848115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2248159c60 0x7f224815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.570+0000 7f224f298700 1 -- 172.21.15.163:0/1447848115 >> 172.21.15.163:0/1447848115 conn(0x7f2248068e20 msgr2=0x7f22480d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:17.578 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.571+0000 7f224f298700 1 -- 172.21.15.163:0/1447848115 shutdown_connections 2023-10-09T06:42:17.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.571+0000 7f224f298700 1 -- 172.21.15.163:0/1447848115 wait complete. 2023-10-09T06:42:17.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.572+0000 7f224f298700 1 Processor -- start 2023-10-09T06:42:17.579 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.572+0000 7f224f298700 1 -- start start 2023-10-09T06:42:17.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.572+0000 7f224f298700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2248159890 0x7f22481441c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.572+0000 7f224f298700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2248159c60 0x7f22480b5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.582 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.572+0000 7f224f298700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f224815db10 0x7f22480ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.572+0000 7f224f298700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22480b6040 con 0x7f2248159c60 2023-10-09T06:42:17.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.572+0000 7f224f298700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2248169d60 con 0x7f224815db10 2023-10-09T06:42:17.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.572+0000 7f224f298700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22480b5d60 con 0x7f2248159890 2023-10-09T06:42:17.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.573+0000 7f224c833700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2248159c60 0x7f22480b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.583 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.573+0000 7f224c833700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2248159c60 0x7f22480b5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55240/0 (socket says 172.21.15.163:55240) 2023-10-09T06:42:17.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.573+0000 7f224c833700 1 -- 172.21.15.163:0/1815852069 learned_addr learned my addr 172.21.15.163:0/1815852069 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:17.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.573+0000 7f224d034700 1 --2- 172.21.15.163:0/1815852069 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2248159890 0x7f22481441c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.573+0000 7f224d835700 1 --2- 172.21.15.163:0/1815852069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f224815db10 0x7f22480ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.573+0000 7f224c833700 1 -- 172.21.15.163:0/1815852069 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2248159890 msgr2=0x7f22481441c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:17.584 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.573+0000 7f224c833700 1 --2- 172.21.15.163:0/1815852069 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2248159890 0x7f22481441c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.573+0000 7f224c833700 1 -- 172.21.15.163:0/1815852069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f224815db10 msgr2=0x7f22480ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:17.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.573+0000 7f224c833700 1 --2- 172.21.15.163:0/1815852069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f224815db10 0x7f22480ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.573+0000 7f224c833700 1 -- 172.21.15.163:0/1815852069 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223c009ae0 con 0x7f2248159c60 2023-10-09T06:42:17.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.573+0000 7f224c833700 1 --2- 172.21.15.163:0/1815852069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2248159c60 0x7f22480b5380 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f223400e8a0 tx=0x7f2234008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:17.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.573+0000 7f22457fa700 1 -- 172.21.15.163:0/1815852069 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2234013410 con 0x7f2248159c60 2023-10-09T06:42:17.585 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.573+0000 7f22457fa700 1 -- 172.21.15.163:0/1815852069 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2234013570 con 0x7f2248159c60 2023-10-09T06:42:17.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.573+0000 7f22457fa700 1 -- 172.21.15.163:0/1815852069 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2234013930 con 0x7f2248159c60 2023-10-09T06:42:17.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.573+0000 7f224f298700 1 -- 172.21.15.163:0/1815852069 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f224816a000 con 0x7f2248159c60 2023-10-09T06:42:17.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.573+0000 7f224f298700 1 -- 172.21.15.163:0/1815852069 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2248144700 con 0x7f2248159c60 2023-10-09T06:42:17.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.574+0000 7f224f298700 1 -- 172.21.15.163:0/1815852069 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f224816a4a0 con 0x7f2248159c60 2023-10-09T06:42:17.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.574+0000 7f22457fa700 1 -- 172.21.15.163:0/1815852069 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2234013ca0 con 0x7f2248159c60 2023-10-09T06:42:17.586 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.575+0000 7f22457fa700 1 --2- 172.21.15.163:0/1815852069 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f222c067320 0x7f222c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.575+0000 7f22457fa700 1 -- 172.21.15.163:0/1815852069 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f22340846b0 con 0x7f2248159c60 2023-10-09T06:42:17.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.575+0000 7f224d034700 1 --2- 172.21.15.163:0/1815852069 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f222c067320 0x7f222c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.575+0000 7f224d034700 1 --2- 172.21.15.163:0/1815852069 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f222c067320 0x7f222c0697d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f223c009880 tx=0x7f223c018000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:17.587 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.579+0000 7f22457fa700 1 -- 172.21.15.163:0/1815852069 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2234054fc0 con 0x7f2248159c60 2023-10-09T06:42:17.765 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.764+0000 7f224f298700 1 -- 172.21.15.163:0/1815852069 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f2248169bc0 con 0x7f222c067320 2023-10-09T06:42:17.766 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.765+0000 7f22457fa700 1 -- 172.21.15.163:0/1815852069 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2248169bc0 con 0x7f222c067320 2023-10-09T06:42:17.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.767+0000 7f224f298700 1 -- 172.21.15.163:0/1815852069 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f222c067320 msgr2=0x7f222c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:17.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.767+0000 7f224f298700 1 --2- 172.21.15.163:0/1815852069 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f222c067320 0x7f222c0697d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f223c009880 tx=0x7f223c018000 comp rx=0 tx=0).stop 2023-10-09T06:42:17.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.767+0000 7f224f298700 1 -- 172.21.15.163:0/1815852069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2248159c60 msgr2=0x7f22480b5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:17.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.767+0000 7f224f298700 1 --2- 172.21.15.163:0/1815852069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2248159c60 0x7f22480b5380 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f223400e8a0 tx=0x7f2234008040 comp rx=0 tx=0).stop 2023-10-09T06:42:17.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.767+0000 7f224f298700 1 -- 172.21.15.163:0/1815852069 shutdown_connections 2023-10-09T06:42:17.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.767+0000 7f224f298700 1 --2- 172.21.15.163:0/1815852069 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f222c067320 0x7f222c0697d0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.767+0000 7f224f298700 1 --2- 172.21.15.163:0/1815852069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f224815db10 0x7f22480ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.767+0000 7f224f298700 1 --2- 172.21.15.163:0/1815852069 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2248159890 0x7f22481441c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.770 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.767+0000 7f224f298700 1 --2- 172.21.15.163:0/1815852069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2248159c60 0x7f22480b5380 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.767+0000 7f224f298700 1 -- 172.21.15.163:0/1815852069 >> 172.21.15.163:0/1815852069 conn(0x7f2248068e20 msgr2=0x7f224815bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:17.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.767+0000 7f224f298700 1 -- 172.21.15.163:0/1815852069 shutdown_connections 2023-10-09T06:42:17.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.767+0000 7f224f298700 1 -- 172.21.15.163:0/1815852069 wait complete. 2023-10-09T06:42:17.771 INFO:teuthology.orchestra.run.smithi163.stderr:instructed osd(s) 4 to deep-scrub 2023-10-09T06:42:17.780 INFO:tasks.ceph:Scrubbing osd.5 2023-10-09T06:42:17.781 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-09T06:42:17.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.900+0000 7f1d96861700 1 Processor -- start 2023-10-09T06:42:17.901 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.900+0000 7f1d96861700 1 -- start start 2023-10-09T06:42:17.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.901+0000 7f1d96861700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1d900b0790 0x7f1d900cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.904 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.901+0000 7f1d96861700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1d901591e0 0x7f1d90159600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.901+0000 7f1d96861700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d900cfa00 con 0x7f1d900b03c0 2023-10-09T06:42:17.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.901+0000 7f1d96861700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d900cb580 con 0x7f1d901591e0 2023-10-09T06:42:17.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.901+0000 7f1d96861700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d900cf160 con 0x7f1d900b0790 2023-10-09T06:42:17.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.901+0000 7f1d96060700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1d901591e0 0x7f1d90159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.901+0000 7f1d9585f700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f1d900b03c0 0x7f1d900b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60646/0 (socket says 172.21.15.163:60646) 2023-10-09T06:42:17.905 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.901+0000 7f1d9585f700 1 -- 172.21.15.163:0/311892465 learned_addr learned my addr 172.21.15.163:0/311892465 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:42:17.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.901+0000 7f1d9505e700 1 --2- 172.21.15.163:0/311892465 >> v2:172.21.15.181:3300/0 conn(0x7f1d900b0790 0x7f1d900cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.901+0000 7f1d96060700 1 -- 172.21.15.163:0/311892465 >> v2:172.21.15.181:3300/0 conn(0x7f1d900b0790 msgr2=0x7f1d900cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:17.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.901+0000 7f1d96060700 1 --2- 172.21.15.163:0/311892465 >> v2:172.21.15.181:3300/0 conn(0x7f1d900b0790 0x7f1d900cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.901+0000 7f1d96060700 1 -- 172.21.15.163:0/311892465 >> v1:172.21.15.163:6789/0 conn(0x7f1d900b03c0 legacy=0x7f1d900b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:17.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.901+0000 7f1d96060700 1 -- 172.21.15.163:0/311892465 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d900cf440 con 0x7f1d901591e0 2023-10-09T06:42:17.906 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.901+0000 7f1d9485d700 1 -- 172.21.15.163:0/311892465 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3711682909 0 0) 0x7f1d900cfa00 con 0x7f1d900b03c0 2023-10-09T06:42:17.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.902+0000 7f1d96060700 1 --2- 172.21.15.163:0/311892465 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d901591e0 0x7f1d90159600 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009e60 tx=0x7f1d88009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308273ed4708c036 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:17.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.902+0000 7f1d9485d700 1 -- 172.21.15.163:0/311892465 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8800e070 con 0x7f1d901591e0 2023-10-09T06:42:17.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.902+0000 7f1d9485d700 1 -- 172.21.15.163:0/311892465 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8800bbf0 con 0x7f1d901591e0 2023-10-09T06:42:17.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.902+0000 7f1d9485d700 1 -- 172.21.15.163:0/311892465 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d88012430 con 0x7f1d901591e0 2023-10-09T06:42:17.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.902+0000 7f1d96861700 1 -- 172.21.15.163:0/311892465 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d901591e0 msgr2=0x7f1d90159600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:17.907 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.902+0000 7f1d96861700 1 --2- 172.21.15.163:0/311892465 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d901591e0 0x7f1d90159600 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009e60 tx=0x7f1d88009780 comp rx=0 tx=0).stop 2023-10-09T06:42:17.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.902+0000 7f1d96861700 1 -- 172.21.15.163:0/311892465 shutdown_connections 2023-10-09T06:42:17.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.902+0000 7f1d96861700 1 --2- 172.21.15.163:0/311892465 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d901591e0 0x7f1d90159600 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.902+0000 7f1d96861700 1 --2- 172.21.15.163:0/311892465 >> v2:172.21.15.181:3300/0 conn(0x7f1d900b0790 0x7f1d900cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.902+0000 7f1d96861700 1 -- 172.21.15.163:0/311892465 >> 172.21.15.163:0/311892465 conn(0x7f1d90063230 msgr2=0x7f1d90063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:17.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.902+0000 7f1d96861700 1 -- 172.21.15.163:0/311892465 shutdown_connections 2023-10-09T06:42:17.908 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.902+0000 7f1d96861700 1 -- 172.21.15.163:0/311892465 wait complete. 2023-10-09T06:42:17.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.903+0000 7f1d96861700 1 Processor -- start 2023-10-09T06:42:17.909 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.903+0000 7f1d96861700 1 -- start start 2023-10-09T06:42:17.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.904+0000 7f1d96861700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d900b03c0 0x7f1d9015a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.912 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.904+0000 7f1d96861700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d900b0790 0x7f1d900c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.904+0000 7f1d96861700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d901591e0 0x7f1d90157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.904+0000 7f1d96861700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9016d9e0 con 0x7f1d900b0790 2023-10-09T06:42:17.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.904+0000 7f1d96861700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d900c5700 con 0x7f1d900b03c0 2023-10-09T06:42:17.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.904+0000 7f1d96861700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9015c780 con 0x7f1d901591e0 2023-10-09T06:42:17.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.904+0000 7f1d9505e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d900b0790 0x7f1d900c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.904+0000 7f1d9505e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d900b0790 0x7f1d900c8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55268/0 (socket says 172.21.15.163:55268) 2023-10-09T06:42:17.913 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.904+0000 7f1d9505e700 1 -- 172.21.15.163:0/3663779394 learned_addr learned my addr 172.21.15.163:0/3663779394 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:17.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.905+0000 7f1d96060700 1 --2- 172.21.15.163:0/3663779394 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d901591e0 0x7f1d90157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.905+0000 7f1d9585f700 1 --2- 172.21.15.163:0/3663779394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d900b03c0 0x7f1d9015a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.905+0000 7f1d9505e700 1 -- 172.21.15.163:0/3663779394 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d901591e0 msgr2=0x7f1d90157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:17.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.905+0000 7f1d9505e700 1 --2- 172.21.15.163:0/3663779394 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d901591e0 0x7f1d90157c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.905+0000 7f1d9505e700 1 -- 172.21.15.163:0/3663779394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d900b03c0 msgr2=0x7f1d9015a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:17.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.905+0000 7f1d9505e700 1 --2- 172.21.15.163:0/3663779394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d900b03c0 0x7f1d9015a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.905+0000 7f1d9505e700 1 -- 172.21.15.163:0/3663779394 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d88009a20 con 0x7f1d900b0790 2023-10-09T06:42:17.914 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.905+0000 7f1d9585f700 1 --2- 172.21.15.163:0/3663779394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d900b03c0 0x7f1d9015a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:42:17.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.905+0000 7f1d9505e700 1 --2- 172.21.15.163:0/3663779394 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d900b0790 0x7f1d900c8c40 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000c7d0 tx=0x7f1d80008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:17.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.905+0000 7f1d85ffb700 1 -- 172.21.15.163:0/3663779394 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d80011600 con 0x7f1d900b0790 2023-10-09T06:42:17.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.905+0000 7f1d85ffb700 1 -- 172.21.15.163:0/3663779394 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d80011760 con 0x7f1d900b0790 2023-10-09T06:42:17.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.905+0000 7f1d96861700 1 -- 172.21.15.163:0/3663779394 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9016ce60 con 0x7f1d900b0790 2023-10-09T06:42:17.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.905+0000 7f1d85ffb700 1 -- 172.21.15.163:0/3663779394 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d80011b20 con 0x7f1d900b0790 2023-10-09T06:42:17.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.905+0000 7f1d96861700 1 -- 172.21.15.163:0/3663779394 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9016d1f0 con 0x7f1d900b0790 2023-10-09T06:42:17.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.906+0000 7f1d96861700 1 -- 172.21.15.163:0/3663779394 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1d9004c3c0 con 0x7f1d900b0790 2023-10-09T06:42:17.915 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.906+0000 7f1d85ffb700 1 -- 172.21.15.163:0/3663779394 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1d800193f0 con 0x7f1d900b0790 2023-10-09T06:42:17.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.907+0000 7f1d85ffb700 1 --2- 172.21.15.163:0/3663779394 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1d74067320 0x7f1d740697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.907+0000 7f1d85ffb700 1 -- 172.21.15.163:0/3663779394 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f1d80084ef0 con 0x7f1d900b0790 2023-10-09T06:42:17.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.907+0000 7f1d85ffb700 1 --2- 172.21.15.163:0/3663779394 >> [v2:172.21.15.181:6800/91130,v1:172.21.15.181:6801/91130] conn(0x7f1d7406d170 0x7f1d7406f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:17.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.907+0000 7f1d85ffb700 1 -- 172.21.15.163:0/3663779394 --> [v2:172.21.15.181:6800/91130,v1:172.21.15.181:6801/91130] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1d7406fc50 con 0x7f1d7406d170 2023-10-09T06:42:17.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.907+0000 7f1d85ffb700 1 -- 172.21.15.163:0/3663779394 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=202) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d80087450 con 0x7f1d900b0790 2023-10-09T06:42:17.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.907+0000 7f1d9585f700 1 --2- 172.21.15.163:0/3663779394 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1d74067320 0x7f1d740697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.907+0000 7f1d96060700 1 --2- 172.21.15.163:0/3663779394 >> [v2:172.21.15.181:6800/91130,v1:172.21.15.181:6801/91130] conn(0x7f1d7406d170 0x7f1d7406f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:17.916 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.907+0000 7f1d9585f700 1 --2- 172.21.15.163:0/3663779394 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1d74067320 0x7f1d740697d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f1d900cde20 tx=0x7f1d8c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:17.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.908+0000 7f1d96060700 1 --2- 172.21.15.163:0/3663779394 >> [v2:172.21.15.181:6800/91130,v1:172.21.15.181:6801/91130] conn(0x7f1d7406d170 0x7f1d7406f580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:17.917 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.908+0000 7f1d85ffb700 1 -- 172.21.15.163:0/3663779394 <== osd.5 v2:172.21.15.181:6800/91130 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f1d7406fc50 con 0x7f1d7406d170 2023-10-09T06:42:17.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.922+0000 7f1d96861700 1 -- 172.21.15.163:0/3663779394 --> [v2:172.21.15.181:6800/91130,v1:172.21.15.181:6801/91130] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1d9016d7c0 con 0x7f1d7406d170 2023-10-09T06:42:17.924 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.922+0000 7f1d85ffb700 1 -- 172.21.15.163:0/3663779394 <== osd.5 v2:172.21.15.181:6800/91130 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f1d9016d7c0 con 0x7f1d7406d170 2023-10-09T06:42:17.924 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:42:17.924 INFO:teuthology.orchestra.run.smithi163.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-09T06:42:17.925 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:42:17.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.923+0000 7f1d96861700 1 -- 172.21.15.163:0/3663779394 >> [v2:172.21.15.181:6800/91130,v1:172.21.15.181:6801/91130] conn(0x7f1d7406d170 msgr2=0x7f1d7406f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:17.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.923+0000 7f1d96861700 1 --2- 172.21.15.163:0/3663779394 >> [v2:172.21.15.181:6800/91130,v1:172.21.15.181:6801/91130] conn(0x7f1d7406d170 0x7f1d7406f580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:17.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.923+0000 7f1d96861700 1 -- 172.21.15.163:0/3663779394 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1d74067320 msgr2=0x7f1d740697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:17.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.923+0000 7f1d96861700 1 --2- 172.21.15.163:0/3663779394 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f1d74067320 0x7f1d740697d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f1d900cde20 tx=0x7f1d8c009040 comp rx=0 tx=0).stop 2023-10-09T06:42:17.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.923+0000 7f1d96861700 1 -- 172.21.15.163:0/3663779394 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d900b0790 msgr2=0x7f1d900c8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:17.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.923+0000 7f1d96861700 1 --2- 172.21.15.163:0/3663779394 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d900b0790 0x7f1d900c8c40 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000c7d0 tx=0x7f1d80008040 comp rx=0 tx=0).stop 2023-10-09T06:42:17.926 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.923+0000 7f1d96060700 1 -- 172.21.15.163:0/3663779394 reap_dead start 2023-10-09T06:42:17.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.923+0000 7f1d96861700 1 -- 172.21.15.163:0/3663779394 shutdown_connections 2023-10-09T06:42:17.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.923+0000 7f1d96861700 1 -- 172.21.15.163:0/3663779394 >> 172.21.15.163:0/3663779394 conn(0x7f1d90063230 msgr2=0x7f1d900ca0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:17.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.923+0000 7f1d96861700 1 -- 172.21.15.163:0/3663779394 shutdown_connections 2023-10-09T06:42:17.927 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:17.923+0000 7f1d96861700 1 -- 172.21.15.163:0/3663779394 wait complete. 2023-10-09T06:42:17.936 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-09T06:42:18.052 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.051+0000 7faf3cf8a700 1 Processor -- start 2023-10-09T06:42:18.053 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.051+0000 7faf3cf8a700 1 -- start start 2023-10-09T06:42:18.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.052+0000 7faf3cf8a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf38159890 0x7faf3815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.058 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.052+0000 7faf3cf8a700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf3815a160 con 0x7faf38069560 2023-10-09T06:42:18.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.052+0000 7faf3cf8a700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf380d13a0 con 0x7faf38159c60 2023-10-09T06:42:18.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.052+0000 7faf3cf8a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3815e140 con 0x7faf38159890 2023-10-09T06:42:18.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.052+0000 7faf36d9d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7faf38069560 0x7faf380ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60662/0 (socket says 172.21.15.163:60662) 2023-10-09T06:42:18.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.052+0000 7faf36d9d700 1 -- 172.21.15.163:0/2519994467 learned_addr learned my addr 172.21.15.163:0/2519994467 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:42:18.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.053+0000 7faf3659c700 1 --2- 172.21.15.163:0/2519994467 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf38159890 0x7faf3815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:18.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.053+0000 7faf35d9b700 1 -- 172.21.15.163:0/2519994467 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1175928315 0 0) 0x7faf3815a160 con 0x7faf38069560 2023-10-09T06:42:18.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.053+0000 7faf35d9b700 1 -- 172.21.15.163:0/2519994467 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf24003570 con 0x7faf38069560 2023-10-09T06:42:18.059 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.053+0000 7faf35d9b700 1 -- 172.21.15.163:0/2519994467 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 330132460 0 0) 0x7faf24003570 con 0x7faf38069560 2023-10-09T06:42:18.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.053+0000 7faf35d9b700 1 -- 172.21.15.163:0/2519994467 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf38159890 msgr2=0x7faf3815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:18.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.053+0000 7faf35d9b700 1 --2- 172.21.15.163:0/2519994467 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf38159890 0x7faf3815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.053+0000 7faf35d9b700 1 -- 172.21.15.163:0/2519994467 >> v1:172.21.15.181:6789/0 conn(0x7faf38159c60 legacy=0x7faf3815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:18.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.053+0000 7faf35d9b700 1 -- 172.21.15.163:0/2519994467 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3815a3d0 con 0x7faf38069560 2023-10-09T06:42:18.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.053+0000 7faf35d9b700 1 -- 172.21.15.163:0/2519994467 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7faf300017f0 con 0x7faf38069560 2023-10-09T06:42:18.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.053+0000 7faf35d9b700 1 -- 172.21.15.163:0/2519994467 >> v1:172.21.15.163:6789/0 conn(0x7faf38069560 legacy=0x7faf380ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:18.060 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.053+0000 7faf35d9b700 1 --2- 172.21.15.163:0/2519994467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf24003a90 0x7faf24003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.053+0000 7faf35d9b700 1 --2- 172.21.15.163:0/2519994467 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf38159890 0x7faf24004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.053+0000 7faf35d9b700 1 --2- 172.21.15.163:0/2519994467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf24004d50 0x7faf240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.053+0000 7faf35d9b700 1 -- 172.21.15.163:0/2519994467 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3815e140 con 0x7faf24004d50 2023-10-09T06:42:18.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.053+0000 7faf35d9b700 1 -- 172.21.15.163:0/2519994467 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2400bb10 con 0x7faf24003a90 2023-10-09T06:42:18.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.053+0000 7faf35d9b700 1 -- 172.21.15.163:0/2519994467 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf2400bcd0 con 0x7faf38159890 2023-10-09T06:42:18.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.053+0000 7faf36d9d700 1 --2- 172.21.15.163:0/2519994467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf24004d50 0x7faf240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:18.061 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.053+0000 7faf3659c700 1 --2- 172.21.15.163:0/2519994467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf24003a90 0x7faf24003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:18.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.053+0000 7faf2dd9b700 1 --2- 172.21.15.163:0/2519994467 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf38159890 0x7faf24004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:18.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.053+0000 7faf36d9d700 1 -- 172.21.15.163:0/2519994467 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf38159890 msgr2=0x7faf24004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:18.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.053+0000 7faf36d9d700 1 --2- 172.21.15.163:0/2519994467 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf38159890 0x7faf24004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.053+0000 7faf36d9d700 1 -- 172.21.15.163:0/2519994467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf24003a90 msgr2=0x7faf24003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:18.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.053+0000 7faf36d9d700 1 --2- 172.21.15.163:0/2519994467 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf24003a90 0x7faf24003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.053+0000 7faf36d9d700 1 -- 172.21.15.163:0/2519994467 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf24003570 con 0x7faf24004d50 2023-10-09T06:42:18.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.054+0000 7faf36d9d700 1 --2- 172.21.15.163:0/2519994467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf24004d50 0x7faf240051c0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7faf3000a950 tx=0x7faf300021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b25d0f8f2de92f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:18.062 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.054+0000 7faf35d9b700 1 -- 172.21.15.163:0/2519994467 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf300070d0 con 0x7faf24004d50 2023-10-09T06:42:18.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.054+0000 7faf35d9b700 1 -- 172.21.15.163:0/2519994467 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf30007230 con 0x7faf24004d50 2023-10-09T06:42:18.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.054+0000 7faf35d9b700 1 -- 172.21.15.163:0/2519994467 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf300075f0 con 0x7faf24004d50 2023-10-09T06:42:18.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.054+0000 7faf3cf8a700 1 -- 172.21.15.163:0/2519994467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf24004d50 msgr2=0x7faf240051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:18.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.054+0000 7faf3cf8a700 1 --2- 172.21.15.163:0/2519994467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf24004d50 0x7faf240051c0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7faf3000a950 tx=0x7faf300021f0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.054+0000 7faf36d9d700 1 -- 172.21.15.163:0/2519994467 reap_dead start 2023-10-09T06:42:18.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.054+0000 7faf3cf8a700 1 -- 172.21.15.163:0/2519994467 shutdown_connections 2023-10-09T06:42:18.063 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.054+0000 7faf3cf8a700 1 -- 172.21.15.163:0/2519994467 >> 172.21.15.163:0/2519994467 conn(0x7faf38068e20 msgr2=0x7faf380d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:18.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.054+0000 7faf3cf8a700 1 -- 172.21.15.163:0/2519994467 shutdown_connections 2023-10-09T06:42:18.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.054+0000 7faf3cf8a700 1 -- 172.21.15.163:0/2519994467 wait complete. 2023-10-09T06:42:18.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.055+0000 7faf3cf8a700 1 Processor -- start 2023-10-09T06:42:18.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.055+0000 7faf3cf8a700 1 -- start start 2023-10-09T06:42:18.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.056+0000 7faf3cf8a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf24004d50 0x7faf3816d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.056+0000 7faf3cf8a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf380ce610 0x7faf38149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.056+0000 7faf3cf8a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf3815c930 0x7faf3815cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.064 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.056+0000 7faf3cf8a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3816de90 con 0x7faf380ce610 2023-10-09T06:42:18.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.056+0000 7faf3cf8a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3815d2e0 con 0x7faf3815c930 2023-10-09T06:42:18.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.056+0000 7faf3cf8a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf380cf9e0 con 0x7faf24004d50 2023-10-09T06:42:18.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.056+0000 7faf2dd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf380ce610 0x7faf38149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:18.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.056+0000 7faf2dd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf380ce610 0x7faf38149dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55274/0 (socket says 172.21.15.163:55274) 2023-10-09T06:42:18.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.056+0000 7faf2dd9b700 1 -- 172.21.15.163:0/2458470941 learned_addr learned my addr 172.21.15.163:0/2458470941 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:18.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.056+0000 7faf36d9d700 1 --2- 172.21.15.163:0/2458470941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf3815c930 0x7faf3815cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:18.065 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.056+0000 7faf3659c700 1 --2- 172.21.15.163:0/2458470941 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf24004d50 0x7faf3816d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:18.066 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.056+0000 7faf2dd9b700 1 -- 172.21.15.163:0/2458470941 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf24004d50 msgr2=0x7faf3816d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:18.066 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.056+0000 7faf2dd9b700 1 --2- 172.21.15.163:0/2458470941 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf24004d50 0x7faf3816d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.066 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.056+0000 7faf2dd9b700 1 -- 172.21.15.163:0/2458470941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf3815c930 msgr2=0x7faf3815cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:18.066 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.056+0000 7faf2dd9b700 1 --2- 172.21.15.163:0/2458470941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf3815c930 0x7faf3815cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.066 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.056+0000 7faf2dd9b700 1 -- 172.21.15.163:0/2458470941 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3814a300 con 0x7faf380ce610 2023-10-09T06:42:18.066 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.056+0000 7faf3659c700 1 --2- 172.21.15.163:0/2458470941 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf24004d50 0x7faf3816d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:42:18.066 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.057+0000 7faf2dd9b700 1 --2- 172.21.15.163:0/2458470941 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf380ce610 0x7faf38149dc0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7faf20002ab0 tx=0x7faf20007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:18.066 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.057+0000 7faf2effd700 1 -- 172.21.15.163:0/2458470941 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2000f870 con 0x7faf380ce610 2023-10-09T06:42:18.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.057+0000 7faf2effd700 1 -- 172.21.15.163:0/2458470941 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2000f9f0 con 0x7faf380ce610 2023-10-09T06:42:18.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.057+0000 7faf2effd700 1 -- 172.21.15.163:0/2458470941 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2000fdb0 con 0x7faf380ce610 2023-10-09T06:42:18.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.057+0000 7faf3cf8a700 1 -- 172.21.15.163:0/2458470941 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf380cedd0 con 0x7faf380ce610 2023-10-09T06:42:18.067 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.057+0000 7faf3cf8a700 1 -- 172.21.15.163:0/2458470941 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf380cf390 con 0x7faf380ce610 2023-10-09T06:42:18.068 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.058+0000 7faf3cf8a700 1 -- 172.21.15.163:0/2458470941 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf380cfb20 con 0x7faf380ce610 2023-10-09T06:42:18.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.058+0000 7faf2effd700 1 -- 172.21.15.163:0/2458470941 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7faf20013600 con 0x7faf380ce610 2023-10-09T06:42:18.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.059+0000 7faf2effd700 1 --2- 172.21.15.163:0/2458470941 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faf18067320 0x7faf180697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.059+0000 7faf2effd700 1 -- 172.21.15.163:0/2458470941 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4939+0+0 (secure 0 0 0) 0x7faf20084f70 con 0x7faf380ce610 2023-10-09T06:42:18.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.059+0000 7faf3659c700 1 --2- 172.21.15.163:0/2458470941 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faf18067320 0x7faf180697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:18.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.059+0000 7faf3659c700 1 --2- 172.21.15.163:0/2458470941 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faf18067320 0x7faf180697d0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7faf28006f00 tx=0x7faf28008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:18.069 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.063+0000 7faf2effd700 1 -- 172.21.15.163:0/2458470941 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faf380cfb20 con 0x7faf380ce610 2023-10-09T06:42:18.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.244+0000 7faf3cf8a700 1 -- 172.21.15.163:0/2458470941 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7faf380cfb20 con 0x7faf18067320 2023-10-09T06:42:18.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.244+0000 7faf2effd700 1 -- 172.21.15.163:0/2458470941 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7faf380cfb20 con 0x7faf18067320 2023-10-09T06:42:18.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.246+0000 7faf3cf8a700 1 -- 172.21.15.163:0/2458470941 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faf18067320 msgr2=0x7faf180697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:18.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.246+0000 7faf3cf8a700 1 --2- 172.21.15.163:0/2458470941 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faf18067320 0x7faf180697d0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7faf28006f00 tx=0x7faf28008040 comp rx=0 tx=0).stop 2023-10-09T06:42:18.247 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.246+0000 7faf3cf8a700 1 -- 172.21.15.163:0/2458470941 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf380ce610 msgr2=0x7faf38149dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:18.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.246+0000 7faf3cf8a700 1 --2- 172.21.15.163:0/2458470941 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf380ce610 0x7faf38149dc0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7faf20002ab0 tx=0x7faf20007d10 comp rx=0 tx=0).stop 2023-10-09T06:42:18.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.246+0000 7faf3cf8a700 1 -- 172.21.15.163:0/2458470941 shutdown_connections 2023-10-09T06:42:18.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.246+0000 7faf3cf8a700 1 --2- 172.21.15.163:0/2458470941 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7faf18067320 0x7faf180697d0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.246+0000 7faf3cf8a700 1 --2- 172.21.15.163:0/2458470941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf3815c930 0x7faf3815cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.246+0000 7faf3cf8a700 1 --2- 172.21.15.163:0/2458470941 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf24004d50 0x7faf3816d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.246+0000 7faf3cf8a700 1 --2- 172.21.15.163:0/2458470941 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf380ce610 0x7faf38149dc0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.248 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.246+0000 7faf3cf8a700 1 -- 172.21.15.163:0/2458470941 >> 172.21.15.163:0/2458470941 conn(0x7faf38068e20 msgr2=0x7faf380c6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:18.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.246+0000 7faf3cf8a700 1 -- 172.21.15.163:0/2458470941 shutdown_connections 2023-10-09T06:42:18.249 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.246+0000 7faf3cf8a700 1 -- 172.21.15.163:0/2458470941 wait complete. 2023-10-09T06:42:18.249 INFO:teuthology.orchestra.run.smithi163.stderr:instructed osd(s) 5 to deep-scrub 2023-10-09T06:42:18.259 INFO:tasks.ceph:Scrubbing osd.6 2023-10-09T06:42:18.259 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-09T06:42:18.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.379+0000 7fb897ec6700 1 Processor -- start 2023-10-09T06:42:18.380 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.379+0000 7fb897ec6700 1 -- start start 2023-10-09T06:42:18.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.380+0000 7fb897ec6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb890159ec0 0x7fb8900b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.380+0000 7fb897ec6700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb8900cf350 0x7fb8900cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.380+0000 7fb897ec6700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8900b5b00 con 0x7fb8900b53d0 2023-10-09T06:42:18.386 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.380+0000 7fb897ec6700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8900b5da0 con 0x7fb8900cf350 2023-10-09T06:42:18.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.380+0000 7fb897ec6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8900b6080 con 0x7fb890159ec0 2023-10-09T06:42:18.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.380+0000 7fb895461700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb8900b53d0 0x7fb8900cc900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:60674/0 (socket says 172.21.15.163:60674) 2023-10-09T06:42:18.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.380+0000 7fb895461700 1 -- 172.21.15.163:0/1948758826 learned_addr learned my addr 172.21.15.163:0/1948758826 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:42:18.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.380+0000 7fb896463700 1 --2- 172.21.15.163:0/1948758826 >> v2:172.21.15.181:3300/0 conn(0x7fb8900cf350 0x7fb8900cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:18.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.380+0000 7fb895c62700 1 --2- 172.21.15.163:0/1948758826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb890159ec0 0x7fb8900b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:18.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.380+0000 7fb894c60700 1 -- 172.21.15.163:0/1948758826 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1078291820 0 0) 0x7fb8900b5b00 con 0x7fb8900b53d0 2023-10-09T06:42:18.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.380+0000 7fb894c60700 1 -- 172.21.15.163:0/1948758826 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb87c003570 con 0x7fb8900b53d0 2023-10-09T06:42:18.387 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.380+0000 7fb894c60700 1 -- 172.21.15.163:0/1948758826 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 397663313 0 0) 0x7fb87c003570 con 0x7fb8900b53d0 2023-10-09T06:42:18.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.380+0000 7fb894c60700 1 -- 172.21.15.163:0/1948758826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb890159ec0 msgr2=0x7fb8900b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:18.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.380+0000 7fb894c60700 1 --2- 172.21.15.163:0/1948758826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb890159ec0 0x7fb8900b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.380+0000 7fb894c60700 1 -- 172.21.15.163:0/1948758826 >> v2:172.21.15.181:3300/0 conn(0x7fb8900cf350 msgr2=0x7fb8900cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:18.388 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.380+0000 7fb894c60700 1 --2- 172.21.15.163:0/1948758826 >> v2:172.21.15.181:3300/0 conn(0x7fb8900cf350 0x7fb8900cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.380+0000 7fb894c60700 1 -- 172.21.15.163:0/1948758826 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8900cc100 con 0x7fb8900b53d0 2023-10-09T06:42:18.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.380+0000 7fb894c60700 1 -- 172.21.15.163:0/1948758826 <== mon.0 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1090058798 0 0) 0x7fb880001470 con 0x7fb8900b53d0 2023-10-09T06:42:18.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.380+0000 7fb894c60700 1 -- 172.21.15.163:0/1948758826 >> v1:172.21.15.163:6789/0 conn(0x7fb8900b53d0 legacy=0x7fb8900cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:18.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.380+0000 7fb894c60700 1 --2- 172.21.15.163:0/1948758826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb890159ec0 0x7fb87c003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.380+0000 7fb894c60700 1 --2- 172.21.15.163:0/1948758826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb87c0040b0 0x7fb87c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.389 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.380+0000 7fb894c60700 1 --2- 172.21.15.163:0/1948758826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb87c004d50 0x7fb87c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.380+0000 7fb894c60700 1 -- 172.21.15.163:0/1948758826 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8900b5da0 con 0x7fb87c004d50 2023-10-09T06:42:18.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.380+0000 7fb894c60700 1 -- 172.21.15.163:0/1948758826 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8900b6080 con 0x7fb87c0040b0 2023-10-09T06:42:18.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.380+0000 7fb894c60700 1 -- 172.21.15.163:0/1948758826 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c00bb60 con 0x7fb890159ec0 2023-10-09T06:42:18.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.381+0000 7fb896463700 1 --2- 172.21.15.163:0/1948758826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb87c004d50 0x7fb87c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:18.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.381+0000 7fb895461700 1 --2- 172.21.15.163:0/1948758826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb87c0040b0 0x7fb87c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:18.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.381+0000 7fb895c62700 1 --2- 172.21.15.163:0/1948758826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb890159ec0 0x7fb87c003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:18.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.381+0000 7fb896463700 1 -- 172.21.15.163:0/1948758826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb890159ec0 msgr2=0x7fb87c003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:18.390 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.381+0000 7fb896463700 1 --2- 172.21.15.163:0/1948758826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb890159ec0 0x7fb87c003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.381+0000 7fb896463700 1 -- 172.21.15.163:0/1948758826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb87c0040b0 msgr2=0x7fb87c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:18.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.381+0000 7fb896463700 1 --2- 172.21.15.163:0/1948758826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb87c0040b0 0x7fb87c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.381+0000 7fb896463700 1 -- 172.21.15.163:0/1948758826 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87c003570 con 0x7fb87c004d50 2023-10-09T06:42:18.391 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.381+0000 7fb895461700 1 --2- 172.21.15.163:0/1948758826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb87c0040b0 0x7fb87c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:42:18.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.381+0000 7fb896463700 1 --2- 172.21.15.163:0/1948758826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb87c004d50 0x7fb87c0051c0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fb88c0094d0 tx=0x7fb88c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9697f3cd49e6e22 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:18.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.381+0000 7fb894c60700 1 -- 172.21.15.163:0/1948758826 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c00ea90 con 0x7fb87c004d50 2023-10-09T06:42:18.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.381+0000 7fb894c60700 1 -- 172.21.15.163:0/1948758826 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88c00f040 con 0x7fb87c004d50 2023-10-09T06:42:18.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.381+0000 7fb894c60700 1 -- 172.21.15.163:0/1948758826 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c00ee20 con 0x7fb87c004d50 2023-10-09T06:42:18.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.382+0000 7fb897ec6700 1 -- 172.21.15.163:0/1948758826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb87c004d50 msgr2=0x7fb87c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:18.392 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.382+0000 7fb897ec6700 1 --2- 172.21.15.163:0/1948758826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb87c004d50 0x7fb87c0051c0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fb88c0094d0 tx=0x7fb88c00b040 comp rx=0 tx=0).stop 2023-10-09T06:42:18.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.382+0000 7fb896463700 1 -- 172.21.15.163:0/1948758826 reap_dead start 2023-10-09T06:42:18.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.382+0000 7fb897ec6700 1 -- 172.21.15.163:0/1948758826 shutdown_connections 2023-10-09T06:42:18.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.382+0000 7fb897ec6700 1 -- 172.21.15.163:0/1948758826 >> 172.21.15.163:0/1948758826 conn(0x7fb8900999f0 msgr2=0x7fb8900b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:18.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.382+0000 7fb897ec6700 1 -- 172.21.15.163:0/1948758826 shutdown_connections 2023-10-09T06:42:18.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.382+0000 7fb897ec6700 1 -- 172.21.15.163:0/1948758826 wait complete. 2023-10-09T06:42:18.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.383+0000 7fb897ec6700 1 Processor -- start 2023-10-09T06:42:18.393 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.383+0000 7fb897ec6700 1 -- start start 2023-10-09T06:42:18.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.383+0000 7fb897ec6700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb89013ed10 0x7fb890154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.383+0000 7fb897ec6700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb890154f90 0x7fb890068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.383+0000 7fb897ec6700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb890069390 0x7fb89016d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.383+0000 7fb897ec6700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8900697e0 con 0x7fb890069390 2023-10-09T06:42:18.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.383+0000 7fb897ec6700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89013f0e0 con 0x7fb89013ed10 2023-10-09T06:42:18.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.383+0000 7fb897ec6700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8901529e0 con 0x7fb890154f90 2023-10-09T06:42:18.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.384+0000 7fb896463700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb890069390 0x7fb89016d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:18.394 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.384+0000 7fb896463700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb890069390 0x7fb89016d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55294/0 (socket says 172.21.15.163:55294) 2023-10-09T06:42:18.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.384+0000 7fb896463700 1 -- 172.21.15.163:0/2242169366 learned_addr learned my addr 172.21.15.163:0/2242169366 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:18.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.384+0000 7fb895c62700 1 --2- 172.21.15.163:0/2242169366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb89013ed10 0x7fb890154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:18.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.384+0000 7fb895461700 1 --2- 172.21.15.163:0/2242169366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb890154f90 0x7fb890068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:18.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.384+0000 7fb896463700 1 -- 172.21.15.163:0/2242169366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb890154f90 msgr2=0x7fb890068e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:18.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.384+0000 7fb896463700 1 --2- 172.21.15.163:0/2242169366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb890154f90 0x7fb890068e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.384+0000 7fb896463700 1 -- 172.21.15.163:0/2242169366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb89013ed10 msgr2=0x7fb890154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:18.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.384+0000 7fb896463700 1 --2- 172.21.15.163:0/2242169366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb89013ed10 0x7fb890154a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.395 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.384+0000 7fb896463700 1 -- 172.21.15.163:0/2242169366 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8901545c0 con 0x7fb890069390 2023-10-09T06:42:18.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.384+0000 7fb895461700 1 --2- 172.21.15.163:0/2242169366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb890154f90 0x7fb890068e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:42:18.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.384+0000 7fb896463700 1 --2- 172.21.15.163:0/2242169366 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb890069390 0x7fb89016d080 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fb88c014c90 tx=0x7fb88c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:18.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.384+0000 7fb885ffb700 1 -- 172.21.15.163:0/2242169366 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c00ee90 con 0x7fb890069390 2023-10-09T06:42:18.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.384+0000 7fb885ffb700 1 -- 172.21.15.163:0/2242169366 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88c01d630 con 0x7fb890069390 2023-10-09T06:42:18.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.384+0000 7fb897ec6700 1 -- 172.21.15.163:0/2242169366 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb890151800 con 0x7fb890069390 2023-10-09T06:42:18.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.384+0000 7fb885ffb700 1 -- 172.21.15.163:0/2242169366 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c01d9f0 con 0x7fb890069390 2023-10-09T06:42:18.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.384+0000 7fb897ec6700 1 -- 172.21.15.163:0/2242169366 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb89016dc40 con 0x7fb890069390 2023-10-09T06:42:18.396 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.385+0000 7fb897ec6700 1 -- 172.21.15.163:0/2242169366 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb890153e10 con 0x7fb890069390 2023-10-09T06:42:18.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.386+0000 7fb885ffb700 1 -- 172.21.15.163:0/2242169366 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb88c003040 con 0x7fb890069390 2023-10-09T06:42:18.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.386+0000 7fb885ffb700 1 --2- 172.21.15.163:0/2242169366 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb874067320 0x7fb8740697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.386+0000 7fb885ffb700 1 -- 172.21.15.163:0/2242169366 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fb88c0860d0 con 0x7fb890069390 2023-10-09T06:42:18.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.386+0000 7fb895c62700 1 --2- 172.21.15.163:0/2242169366 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb874067320 0x7fb8740697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:18.398 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.386+0000 7fb885ffb700 1 --2- 172.21.15.163:0/2242169366 >> [v2:172.21.15.181:6808/91131,v1:172.21.15.181:6809/91131] conn(0x7fb87406d170 0x7fb87406f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.386+0000 7fb885ffb700 1 -- 172.21.15.163:0/2242169366 --> [v2:172.21.15.181:6808/91131,v1:172.21.15.181:6809/91131] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb87406fc50 con 0x7fb87406d170 2023-10-09T06:42:18.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.386+0000 7fb885ffb700 1 -- 172.21.15.163:0/2242169366 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=202) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb88c088630 con 0x7fb890069390 2023-10-09T06:42:18.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.386+0000 7fb895461700 1 --2- 172.21.15.163:0/2242169366 >> [v2:172.21.15.181:6808/91131,v1:172.21.15.181:6809/91131] conn(0x7fb87406d170 0x7fb87406f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:18.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.387+0000 7fb895c62700 1 --2- 172.21.15.163:0/2242169366 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb874067320 0x7fb8740697d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fb888006e50 tx=0x7fb888009120 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:18.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.387+0000 7fb895461700 1 --2- 172.21.15.163:0/2242169366 >> [v2:172.21.15.181:6808/91131,v1:172.21.15.181:6809/91131] conn(0x7fb87406d170 0x7fb87406f580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:18.399 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.388+0000 7fb885ffb700 1 -- 172.21.15.163:0/2242169366 <== osd.6 v2:172.21.15.181:6808/91131 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fb87406fc50 con 0x7fb87406d170 2023-10-09T06:42:18.403 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.401+0000 7fb897ec6700 1 -- 172.21.15.163:0/2242169366 --> [v2:172.21.15.181:6808/91131,v1:172.21.15.181:6809/91131] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb890063c90 con 0x7fb87406d170 2023-10-09T06:42:18.403 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.402+0000 7fb885ffb700 1 -- 172.21.15.163:0/2242169366 <== osd.6 v2:172.21.15.181:6808/91131 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb890063c90 con 0x7fb87406d170 2023-10-09T06:42:18.404 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:42:18.404 INFO:teuthology.orchestra.run.smithi163.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-09T06:42:18.404 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:42:18.405 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.402+0000 7fb897ec6700 1 -- 172.21.15.163:0/2242169366 >> [v2:172.21.15.181:6808/91131,v1:172.21.15.181:6809/91131] conn(0x7fb87406d170 msgr2=0x7fb87406f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:18.405 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.402+0000 7fb897ec6700 1 --2- 172.21.15.163:0/2242169366 >> [v2:172.21.15.181:6808/91131,v1:172.21.15.181:6809/91131] conn(0x7fb87406d170 0x7fb87406f580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.402+0000 7fb897ec6700 1 -- 172.21.15.163:0/2242169366 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb874067320 msgr2=0x7fb8740697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:18.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.402+0000 7fb897ec6700 1 --2- 172.21.15.163:0/2242169366 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7fb874067320 0x7fb8740697d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fb888006e50 tx=0x7fb888009120 comp rx=0 tx=0).stop 2023-10-09T06:42:18.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.402+0000 7fb897ec6700 1 -- 172.21.15.163:0/2242169366 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb890069390 msgr2=0x7fb89016d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:18.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.402+0000 7fb897ec6700 1 --2- 172.21.15.163:0/2242169366 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb890069390 0x7fb89016d080 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fb88c014c90 tx=0x7fb88c01a000 comp rx=0 tx=0).stop 2023-10-09T06:42:18.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.402+0000 7fb896463700 1 -- 172.21.15.163:0/2242169366 reap_dead start 2023-10-09T06:42:18.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.403+0000 7fb897ec6700 1 -- 172.21.15.163:0/2242169366 shutdown_connections 2023-10-09T06:42:18.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.403+0000 7fb897ec6700 1 -- 172.21.15.163:0/2242169366 >> 172.21.15.163:0/2242169366 conn(0x7fb8900999f0 msgr2=0x7fb8900c5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:18.406 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.403+0000 7fb897ec6700 1 -- 172.21.15.163:0/2242169366 shutdown_connections 2023-10-09T06:42:18.407 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.403+0000 7fb897ec6700 1 -- 172.21.15.163:0/2242169366 wait complete. 2023-10-09T06:42:18.415 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-09T06:42:18.538 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.534+0000 7ff5b0b21700 1 Processor -- start 2023-10-09T06:42:18.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.534+0000 7ff5b0b21700 1 -- start start 2023-10-09T06:42:18.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.535+0000 7ff5b0b21700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff5ac159c60 0x7ff5ac15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.535+0000 7ff5b0b21700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5ac15a160 con 0x7ff5ac159890 2023-10-09T06:42:18.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.535+0000 7ff5b0b21700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5ac0d13a0 con 0x7ff5ac15db10 2023-10-09T06:42:18.539 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.535+0000 7ff5b0b21700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5ac15df50 con 0x7ff5ac159c60 2023-10-09T06:42:18.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.535+0000 7ff5aa59c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff5ac159890 0x7ff5ac15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:36900/0 (socket says 172.21.15.163:36900) 2023-10-09T06:42:18.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.535+0000 7ff5a9d9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff5ac159c60 0x7ff5ac15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:18.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.535+0000 7ff5aa59c700 1 -- 172.21.15.163:0/4032145853 learned_addr learned my addr 172.21.15.163:0/4032145853 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:42:18.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.535+0000 7ff5a959a700 1 -- 172.21.15.163:0/4032145853 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3647389936 0 0) 0x7ff5ac15a160 con 0x7ff5ac159890 2023-10-09T06:42:18.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.535+0000 7ff5a959a700 1 -- 172.21.15.163:0/4032145853 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff598003570 con 0x7ff5ac159890 2023-10-09T06:42:18.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.535+0000 7ff5a9d9b700 1 -- 172.21.15.163:0/4032145853 >> v1:172.21.15.181:6789/0 conn(0x7ff5ac15db10 legacy=0x7ff5ac069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:18.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.535+0000 7ff5a9d9b700 1 -- 172.21.15.163:0/4032145853 >> v1:172.21.15.163:6789/0 conn(0x7ff5ac159890 legacy=0x7ff5ac15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:18.540 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.535+0000 7ff5a9d9b700 1 -- 172.21.15.163:0/4032145853 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ac15e140 con 0x7ff5ac159c60 2023-10-09T06:42:18.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.535+0000 7ff5a9d9b700 1 --2- 172.21.15.163:0/4032145853 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5ac159c60 0x7ff5ac15d5d0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7ff5940098e0 tx=0x7ff5940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe66a9621263561 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:18.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.536+0000 7ff5a959a700 1 -- 172.21.15.163:0/4032145853 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff594011630 con 0x7ff5ac159c60 2023-10-09T06:42:18.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.536+0000 7ff5a959a700 1 -- 172.21.15.163:0/4032145853 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff594011790 con 0x7ff5ac159c60 2023-10-09T06:42:18.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.536+0000 7ff5a959a700 1 -- 172.21.15.163:0/4032145853 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff594011b30 con 0x7ff5ac159c60 2023-10-09T06:42:18.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.536+0000 7ff5b0b21700 1 -- 172.21.15.163:0/4032145853 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5ac159c60 msgr2=0x7ff5ac15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:18.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.536+0000 7ff5b0b21700 1 --2- 172.21.15.163:0/4032145853 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5ac159c60 0x7ff5ac15d5d0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7ff5940098e0 tx=0x7ff5940091a0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.541 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.536+0000 7ff5b0b21700 1 -- 172.21.15.163:0/4032145853 shutdown_connections 2023-10-09T06:42:18.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.536+0000 7ff5b0b21700 1 --2- 172.21.15.163:0/4032145853 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5ac159c60 0x7ff5ac15d5d0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.536+0000 7ff5b0b21700 1 -- 172.21.15.163:0/4032145853 >> 172.21.15.163:0/4032145853 conn(0x7ff5ac068e20 msgr2=0x7ff5ac0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:18.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.536+0000 7ff5b0b21700 1 -- 172.21.15.163:0/4032145853 shutdown_connections 2023-10-09T06:42:18.542 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.536+0000 7ff5b0b21700 1 -- 172.21.15.163:0/4032145853 wait complete. 2023-10-09T06:42:18.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.537+0000 7ff5b0b21700 1 Processor -- start 2023-10-09T06:42:18.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.537+0000 7ff5b0b21700 1 -- start start 2023-10-09T06:42:18.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.538+0000 7ff5b0b21700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5ac159890 0x7ff5ac155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.545 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.538+0000 7ff5b0b21700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5ac159c60 0x7ff5ac0b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.538+0000 7ff5b0b21700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5ac15db10 0x7ff5ac0b6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.538+0000 7ff5b0b21700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac16a300 con 0x7ff5ac159c60 2023-10-09T06:42:18.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.538+0000 7ff5b0b21700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac16a5e0 con 0x7ff5ac159890 2023-10-09T06:42:18.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.538+0000 7ff5b0b21700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac16a8c0 con 0x7ff5ac15db10 2023-10-09T06:42:18.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.538+0000 7ff5a9d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5ac159c60 0x7ff5ac0b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:18.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.538+0000 7ff5a9d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5ac159c60 0x7ff5ac0b5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55016/0 (socket says 172.21.15.163:55016) 2023-10-09T06:42:18.546 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.538+0000 7ff5a9d9b700 1 -- 172.21.15.163:0/2371240340 learned_addr learned my addr 172.21.15.163:0/2371240340 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:18.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.538+0000 7ff5a9d9b700 1 -- 172.21.15.163:0/2371240340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5ac15db10 msgr2=0x7ff5ac0b6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:18.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.538+0000 7ff5a9d9b700 1 --2- 172.21.15.163:0/2371240340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5ac15db10 0x7ff5ac0b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.538+0000 7ff5a9d9b700 1 -- 172.21.15.163:0/2371240340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5ac159890 msgr2=0x7ff5ac155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:18.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.538+0000 7ff5a9d9b700 1 --2- 172.21.15.163:0/2371240340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5ac159890 0x7ff5ac155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.538+0000 7ff5a9d9b700 1 -- 172.21.15.163:0/2371240340 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff594009420 con 0x7ff5ac159c60 2023-10-09T06:42:18.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.539+0000 7ff5a9d9b700 1 --2- 172.21.15.163:0/2371240340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5ac159c60 0x7ff5ac0b5b40 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7ff5940030c0 tx=0x7ff594016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:18.547 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.539+0000 7ff5a27fc700 1 -- 172.21.15.163:0/2371240340 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff594000d10 con 0x7ff5ac159c60 2023-10-09T06:42:18.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.539+0000 7ff5a27fc700 1 -- 172.21.15.163:0/2371240340 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff594020070 con 0x7ff5ac159c60 2023-10-09T06:42:18.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.539+0000 7ff5a27fc700 1 -- 172.21.15.163:0/2371240340 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5940276b0 con 0x7ff5ac159c60 2023-10-09T06:42:18.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.539+0000 7ff5b0b21700 1 -- 172.21.15.163:0/2371240340 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5ac16aba0 con 0x7ff5ac159c60 2023-10-09T06:42:18.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.539+0000 7ff5b0b21700 1 -- 172.21.15.163:0/2371240340 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5ac169f90 con 0x7ff5ac159c60 2023-10-09T06:42:18.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.540+0000 7ff5b0b21700 1 -- 172.21.15.163:0/2371240340 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5ac16b1c0 con 0x7ff5ac159c60 2023-10-09T06:42:18.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.540+0000 7ff5a27fc700 1 -- 172.21.15.163:0/2371240340 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff59401c450 con 0x7ff5ac159c60 2023-10-09T06:42:18.548 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.541+0000 7ff5a27fc700 1 --2- 172.21.15.163:0/2371240340 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff58c067320 0x7ff58c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.541+0000 7ff5a27fc700 1 -- 172.21.15.163:0/2371240340 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff594022070 con 0x7ff5ac159c60 2023-10-09T06:42:18.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.541+0000 7ff5aa59c700 1 --2- 172.21.15.163:0/2371240340 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff58c067320 0x7ff58c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:18.549 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.541+0000 7ff5aa59c700 1 --2- 172.21.15.163:0/2371240340 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff58c067320 0x7ff58c0697d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4005c50 tx=0x7ff5a40016e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:18.550 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.545+0000 7ff5a27fc700 1 -- 172.21.15.163:0/2371240340 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff594053b20 con 0x7ff5ac159c60 2023-10-09T06:42:18.727 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.726+0000 7ff5b0b21700 1 -- 172.21.15.163:0/2371240340 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5ac04c3c0 con 0x7ff58c067320 2023-10-09T06:42:18.728 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.727+0000 7ff5a27fc700 1 -- 172.21.15.163:0/2371240340 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff5ac04c3c0 con 0x7ff58c067320 2023-10-09T06:42:18.730 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.729+0000 7ff5b0b21700 1 -- 172.21.15.163:0/2371240340 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff58c067320 msgr2=0x7ff58c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:18.730 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.729+0000 7ff5b0b21700 1 --2- 172.21.15.163:0/2371240340 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff58c067320 0x7ff58c0697d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4005c50 tx=0x7ff5a40016e0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.731 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.729+0000 7ff5b0b21700 1 -- 172.21.15.163:0/2371240340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5ac159c60 msgr2=0x7ff5ac0b5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:18.731 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.729+0000 7ff5b0b21700 1 --2- 172.21.15.163:0/2371240340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5ac159c60 0x7ff5ac0b5b40 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7ff5940030c0 tx=0x7ff594016000 comp rx=0 tx=0).stop 2023-10-09T06:42:18.731 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.729+0000 7ff5b0b21700 1 -- 172.21.15.163:0/2371240340 shutdown_connections 2023-10-09T06:42:18.731 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.729+0000 7ff5b0b21700 1 --2- 172.21.15.163:0/2371240340 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7ff58c067320 0x7ff58c0697d0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.731 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.729+0000 7ff5b0b21700 1 --2- 172.21.15.163:0/2371240340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5ac15db10 0x7ff5ac0b6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.729+0000 7ff5b0b21700 1 --2- 172.21.15.163:0/2371240340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5ac159890 0x7ff5ac155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.729+0000 7ff5b0b21700 1 --2- 172.21.15.163:0/2371240340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5ac159c60 0x7ff5ac0b5b40 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.729+0000 7ff5b0b21700 1 -- 172.21.15.163:0/2371240340 >> 172.21.15.163:0/2371240340 conn(0x7ff5ac068e20 msgr2=0x7ff5ac153d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:18.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.729+0000 7ff5b0b21700 1 -- 172.21.15.163:0/2371240340 shutdown_connections 2023-10-09T06:42:18.732 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.729+0000 7ff5b0b21700 1 -- 172.21.15.163:0/2371240340 wait complete. 2023-10-09T06:42:18.733 INFO:teuthology.orchestra.run.smithi163.stderr:instructed osd(s) 6 to deep-scrub 2023-10-09T06:42:18.742 INFO:tasks.ceph:Scrubbing osd.7 2023-10-09T06:42:18.742 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-09T06:42:18.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.863+0000 7f593f51a700 1 Processor -- start 2023-10-09T06:42:18.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.864+0000 7f593f51a700 1 -- start start 2023-10-09T06:42:18.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.864+0000 7f593f51a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5938159ec0 0x7f59380b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.864+0000 7f593f51a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f59380b53d0 0x7f59380cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.864+0000 7f593f51a700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59380b5b00 con 0x7f59380cce40 2023-10-09T06:42:18.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.864+0000 7f593f51a700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59380b5da0 con 0x7f59380b53d0 2023-10-09T06:42:18.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.864+0000 7f593f51a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59380b6080 con 0x7f5938159ec0 2023-10-09T06:42:18.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.865+0000 7f593cab5700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f59380b53d0 0x7f59380cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:18.867 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.865+0000 7f593dab7700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f59380cce40 0x7f59380cd280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:36904/0 (socket says 172.21.15.163:36904) 2023-10-09T06:42:18.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.865+0000 7f593dab7700 1 -- 172.21.15.163:0/1763205757 learned_addr learned my addr 172.21.15.163:0/1763205757 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:42:18.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.865+0000 7f593d2b6700 1 --2- 172.21.15.163:0/1763205757 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5938159ec0 0x7f59380b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:18.868 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.865+0000 7f593cab5700 1 -- 172.21.15.163:0/1763205757 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5938159ec0 msgr2=0x7f59380b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:18.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.865+0000 7f593cab5700 1 --2- 172.21.15.163:0/1763205757 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5938159ec0 0x7f59380b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.865+0000 7f593cab5700 1 -- 172.21.15.163:0/1763205757 >> v1:172.21.15.163:6789/0 conn(0x7f59380cce40 legacy=0x7f59380cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:18.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.865+0000 7f592ffff700 1 -- 172.21.15.163:0/1763205757 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 374406328 0 0) 0x7f59380b5b00 con 0x7f59380cce40 2023-10-09T06:42:18.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.865+0000 7f593cab5700 1 -- 172.21.15.163:0/1763205757 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59380cc100 con 0x7f59380b53d0 2023-10-09T06:42:18.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.865+0000 7f593d2b6700 1 --2- 172.21.15.163:0/1763205757 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5938159ec0 0x7f59380b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:42:18.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.865+0000 7f593cab5700 1 --2- 172.21.15.163:0/1763205757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59380b53d0 0x7f59380cc900 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f59280098e0 tx=0x7f59280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7bd1c72023bec53 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:18.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.865+0000 7f592ffff700 1 -- 172.21.15.163:0/1763205757 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5928009e80 con 0x7f59380b53d0 2023-10-09T06:42:18.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.865+0000 7f592ffff700 1 -- 172.21.15.163:0/1763205757 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f592800b860 con 0x7f59380b53d0 2023-10-09T06:42:18.870 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.865+0000 7f592ffff700 1 -- 172.21.15.163:0/1763205757 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592800bbe0 con 0x7f59380b53d0 2023-10-09T06:42:18.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.866+0000 7f593f51a700 1 -- 172.21.15.163:0/1763205757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59380b53d0 msgr2=0x7f59380cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:18.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.866+0000 7f593f51a700 1 --2- 172.21.15.163:0/1763205757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59380b53d0 0x7f59380cc900 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f59280098e0 tx=0x7f59280091a0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.866+0000 7f593f51a700 1 -- 172.21.15.163:0/1763205757 shutdown_connections 2023-10-09T06:42:18.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.866+0000 7f593f51a700 1 --2- 172.21.15.163:0/1763205757 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5938159ec0 0x7f59380b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.866+0000 7f593f51a700 1 --2- 172.21.15.163:0/1763205757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59380b53d0 0x7f59380cc900 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.866+0000 7f593f51a700 1 -- 172.21.15.163:0/1763205757 >> 172.21.15.163:0/1763205757 conn(0x7f59380999f0 msgr2=0x7f59380b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:18.877 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.866+0000 7f593f51a700 1 -- 172.21.15.163:0/1763205757 shutdown_connections 2023-10-09T06:42:18.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.866+0000 7f593f51a700 1 -- 172.21.15.163:0/1763205757 wait complete. 2023-10-09T06:42:18.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.867+0000 7f593f51a700 1 Processor -- start 2023-10-09T06:42:18.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.867+0000 7f593f51a700 1 -- start start 2023-10-09T06:42:18.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.868+0000 7f593f51a700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59380b53d0 0x7f5938157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.868+0000 7f593f51a700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59380cce40 0x7f5938169da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.868+0000 7f593f51a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5938159ec0 0x7f593809c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.868+0000 7f593f51a700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5938069f70 con 0x7f5938159ec0 2023-10-09T06:42:18.878 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.868+0000 7f593f51a700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5938157570 con 0x7f59380cce40 2023-10-09T06:42:18.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.868+0000 7f593f51a700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5938069360 con 0x7f59380b53d0 2023-10-09T06:42:18.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.868+0000 7f593dab7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5938159ec0 0x7f593809c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:18.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.868+0000 7f593dab7700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5938159ec0 0x7f593809c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55036/0 (socket says 172.21.15.163:55036) 2023-10-09T06:42:18.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.868+0000 7f593dab7700 1 -- 172.21.15.163:0/1675128795 learned_addr learned my addr 172.21.15.163:0/1675128795 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:18.879 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.868+0000 7f593d2b6700 1 --2- 172.21.15.163:0/1675128795 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59380b53d0 0x7f5938157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:18.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.868+0000 7f593cab5700 1 --2- 172.21.15.163:0/1675128795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59380cce40 0x7f5938169da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:18.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.868+0000 7f593dab7700 1 -- 172.21.15.163:0/1675128795 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59380b53d0 msgr2=0x7f5938157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:18.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.868+0000 7f593dab7700 1 --2- 172.21.15.163:0/1675128795 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59380b53d0 0x7f5938157030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.868+0000 7f593dab7700 1 -- 172.21.15.163:0/1675128795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59380cce40 msgr2=0x7f5938169da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:18.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.868+0000 7f593dab7700 1 --2- 172.21.15.163:0/1675128795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59380cce40 0x7f5938169da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.868+0000 7f593dab7700 1 -- 172.21.15.163:0/1675128795 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5928009420 con 0x7f5938159ec0 2023-10-09T06:42:18.880 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.868+0000 7f593cab5700 1 --2- 172.21.15.163:0/1675128795 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59380cce40 0x7f5938169da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:42:18.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.868+0000 7f593dab7700 1 --2- 172.21.15.163:0/1675128795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5938159ec0 0x7f593809c550 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f593000cc10 tx=0x7f5930001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:18.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.868+0000 7f592d7fa700 1 -- 172.21.15.163:0/1675128795 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593000a880 con 0x7f5938159ec0 2023-10-09T06:42:18.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.868+0000 7f592d7fa700 1 -- 172.21.15.163:0/1675128795 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593000aa00 con 0x7f5938159ec0 2023-10-09T06:42:18.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.868+0000 7f593f51a700 1 -- 172.21.15.163:0/1675128795 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5938069640 con 0x7f5938159ec0 2023-10-09T06:42:18.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.868+0000 7f592d7fa700 1 -- 172.21.15.163:0/1675128795 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593000adc0 con 0x7f5938159ec0 2023-10-09T06:42:18.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.869+0000 7f593f51a700 1 -- 172.21.15.163:0/1675128795 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5938069c00 con 0x7f5938159ec0 2023-10-09T06:42:18.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.869+0000 7f593f51a700 1 -- 172.21.15.163:0/1675128795 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5938063c90 con 0x7f5938159ec0 2023-10-09T06:42:18.881 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.870+0000 7f592d7fa700 1 -- 172.21.15.163:0/1675128795 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5930007300 con 0x7f5938159ec0 2023-10-09T06:42:18.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.870+0000 7f592d7fa700 1 --2- 172.21.15.163:0/1675128795 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f591c067320 0x7f591c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.870+0000 7f592d7fa700 1 -- 172.21.15.163:0/1675128795 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f5930085720 con 0x7f5938159ec0 2023-10-09T06:42:18.882 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.870+0000 7f592d7fa700 1 --2- 172.21.15.163:0/1675128795 >> [v2:172.21.15.181:6816/91132,v1:172.21.15.181:6817/91132] conn(0x7f591c06d170 0x7f591c06f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:18.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.870+0000 7f592d7fa700 1 -- 172.21.15.163:0/1675128795 --> [v2:172.21.15.181:6816/91132,v1:172.21.15.181:6817/91132] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f591c06fc50 con 0x7f591c06d170 2023-10-09T06:42:18.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.870+0000 7f592d7fa700 1 -- 172.21.15.163:0/1675128795 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=202) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5930085a30 con 0x7f5938159ec0 2023-10-09T06:42:18.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.870+0000 7f593d2b6700 1 --2- 172.21.15.163:0/1675128795 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f591c067320 0x7f591c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:18.883 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.871+0000 7f593cab5700 1 --2- 172.21.15.163:0/1675128795 >> [v2:172.21.15.181:6816/91132,v1:172.21.15.181:6817/91132] conn(0x7f591c06d170 0x7f591c06f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:18.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.871+0000 7f593d2b6700 1 --2- 172.21.15.163:0/1675128795 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f591c067320 0x7f591c0697d0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f593400ad40 tx=0x7f5934008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:18.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.871+0000 7f593cab5700 1 --2- 172.21.15.163:0/1675128795 >> [v2:172.21.15.181:6816/91132,v1:172.21.15.181:6817/91132] conn(0x7f591c06d170 0x7f591c06f580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:18.884 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.872+0000 7f592d7fa700 1 -- 172.21.15.163:0/1675128795 <== osd.7 v2:172.21.15.181:6816/91132 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f591c06fc50 con 0x7f591c06d170 2023-10-09T06:42:18.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.885+0000 7f593f51a700 1 -- 172.21.15.163:0/1675128795 --> [v2:172.21.15.181:6816/91132,v1:172.21.15.181:6817/91132] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f59380718b0 con 0x7f591c06d170 2023-10-09T06:42:18.889 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.886+0000 7f592d7fa700 1 -- 172.21.15.163:0/1675128795 <== osd.7 v2:172.21.15.181:6816/91132 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f59380718b0 con 0x7f591c06d170 2023-10-09T06:42:18.890 INFO:teuthology.orchestra.run.smithi163.stdout:{ 2023-10-09T06:42:18.890 INFO:teuthology.orchestra.run.smithi163.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-09T06:42:18.890 INFO:teuthology.orchestra.run.smithi163.stdout:} 2023-10-09T06:42:18.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.886+0000 7f593f51a700 1 -- 172.21.15.163:0/1675128795 >> [v2:172.21.15.181:6816/91132,v1:172.21.15.181:6817/91132] conn(0x7f591c06d170 msgr2=0x7f591c06f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:18.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.886+0000 7f593f51a700 1 --2- 172.21.15.163:0/1675128795 >> [v2:172.21.15.181:6816/91132,v1:172.21.15.181:6817/91132] conn(0x7f591c06d170 0x7f591c06f580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:18.892 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.886+0000 7f593f51a700 1 -- 172.21.15.163:0/1675128795 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f591c067320 msgr2=0x7f591c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:18.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.886+0000 7f593f51a700 1 --2- 172.21.15.163:0/1675128795 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f591c067320 0x7f591c0697d0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f593400ad40 tx=0x7f5934008040 comp rx=0 tx=0).stop 2023-10-09T06:42:18.893 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.886+0000 7f593f51a700 1 -- 172.21.15.163:0/1675128795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5938159ec0 msgr2=0x7f593809c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:18.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.886+0000 7f593f51a700 1 --2- 172.21.15.163:0/1675128795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5938159ec0 0x7f593809c550 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f593000cc10 tx=0x7f5930001970 comp rx=0 tx=0).stop 2023-10-09T06:42:18.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.886+0000 7f593dab7700 1 -- 172.21.15.163:0/1675128795 reap_dead start 2023-10-09T06:42:18.894 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.886+0000 7f593f51a700 1 -- 172.21.15.163:0/1675128795 shutdown_connections 2023-10-09T06:42:18.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.886+0000 7f593f51a700 1 -- 172.21.15.163:0/1675128795 >> 172.21.15.163:0/1675128795 conn(0x7f59380999f0 msgr2=0x7f59380c9910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:18.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.886+0000 7f593f51a700 1 -- 172.21.15.163:0/1675128795 shutdown_connections 2023-10-09T06:42:18.895 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:18.886+0000 7f593f51a700 1 -- 172.21.15.163:0/1675128795 wait complete. 2023-10-09T06:42:18.899 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-09T06:42:19.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.013+0000 7f8380270700 1 Processor -- start 2023-10-09T06:42:19.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.013+0000 7f8380270700 1 -- start start 2023-10-09T06:42:19.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.014+0000 7f8380270700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8378159890 0x7f837815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:19.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.014+0000 7f8380270700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f837815a160 con 0x7f8378159c60 2023-10-09T06:42:19.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.014+0000 7f8380270700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83780d13a0 con 0x7f8378069560 2023-10-09T06:42:19.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.014+0000 7f8380270700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f837815e140 con 0x7f8378159890 2023-10-09T06:42:19.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.014+0000 7f837d80b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f8378159c60 0x7f837815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:36906/0 (socket says 172.21.15.163:36906) 2023-10-09T06:42:19.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.014+0000 7f837d80b700 1 -- 172.21.15.163:0/1367388726 learned_addr learned my addr 172.21.15.163:0/1367388726 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:42:19.018 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.014+0000 7f837e00c700 1 --2- 172.21.15.163:0/1367388726 >> v2:172.21.15.181:3300/0 conn(0x7f8378159890 0x7f837815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:19.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.015+0000 7f837d00a700 1 -- 172.21.15.163:0/1367388726 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2414107728 0 0) 0x7f837815a160 con 0x7f8378159c60 2023-10-09T06:42:19.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.015+0000 7f837d00a700 1 -- 172.21.15.163:0/1367388726 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8364003570 con 0x7f8378159c60 2023-10-09T06:42:19.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.015+0000 7f837e00c700 1 -- 172.21.15.163:0/1367388726 >> v1:172.21.15.181:6789/0 conn(0x7f8378069560 legacy=0x7f83780ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:19.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.015+0000 7f837d00a700 1 -- 172.21.15.163:0/1367388726 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3636852070 0 0) 0x7f8364003570 con 0x7f8378159c60 2023-10-09T06:42:19.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.015+0000 7f837e00c700 1 -- 172.21.15.163:0/1367388726 >> v1:172.21.15.163:6789/0 conn(0x7f8378159c60 legacy=0x7f837815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:19.019 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.015+0000 7f837e00c700 1 -- 172.21.15.163:0/1367388726 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f837815a3d0 con 0x7f8378159890 2023-10-09T06:42:19.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.015+0000 7f837e00c700 1 --2- 172.21.15.163:0/1367388726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8378159890 0x7f837815abd0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f8374009e60 tx=0x7f8374009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd1a46f73bec4b45 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:19.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.015+0000 7f837d00a700 1 -- 172.21.15.163:0/1367388726 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837400e070 con 0x7f8378159890 2023-10-09T06:42:19.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.015+0000 7f837d00a700 1 -- 172.21.15.163:0/1367388726 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837400b850 con 0x7f8378159890 2023-10-09T06:42:19.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.015+0000 7f837d00a700 1 -- 172.21.15.163:0/1367388726 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8368001470 con 0x7f8378159890 2023-10-09T06:42:19.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.016+0000 7f8380270700 1 -- 172.21.15.163:0/1367388726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8378159890 msgr2=0x7f837815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:19.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.016+0000 7f8380270700 1 --2- 172.21.15.163:0/1367388726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8378159890 0x7f837815abd0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f8374009e60 tx=0x7f8374009780 comp rx=0 tx=0).stop 2023-10-09T06:42:19.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.016+0000 7f8380270700 1 -- 172.21.15.163:0/1367388726 shutdown_connections 2023-10-09T06:42:19.020 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.016+0000 7f8380270700 1 --2- 172.21.15.163:0/1367388726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8378159890 0x7f837815abd0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:19.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.016+0000 7f8380270700 1 -- 172.21.15.163:0/1367388726 >> 172.21.15.163:0/1367388726 conn(0x7f8378068e20 msgr2=0x7f83780d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:19.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.016+0000 7f8380270700 1 -- 172.21.15.163:0/1367388726 shutdown_connections 2023-10-09T06:42:19.021 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.016+0000 7f8380270700 1 -- 172.21.15.163:0/1367388726 wait complete. 2023-10-09T06:42:19.023 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.017+0000 7f8380270700 1 Processor -- start 2023-10-09T06:42:19.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.017+0000 7f8380270700 1 -- start start 2023-10-09T06:42:19.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.018+0000 7f8380270700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8378069560 0x7f837814c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:19.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.018+0000 7f8380270700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8378159890 0x7f8378155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:19.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.018+0000 7f8380270700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8378159c60 0x7f83780cf250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:19.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.018+0000 7f8380270700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837815bc50 con 0x7f8378069560 2023-10-09T06:42:19.024 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.018+0000 7f8380270700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837815bf30 con 0x7f8378159c60 2023-10-09T06:42:19.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.018+0000 7f8380270700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f837815c210 con 0x7f8378159890 2023-10-09T06:42:19.025 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.018+0000 7f837e00c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8378069560 0x7f837814c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:19.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.018+0000 7f837e00c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8378069560 0x7f837814c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55048/0 (socket says 172.21.15.163:55048) 2023-10-09T06:42:19.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.018+0000 7f837e00c700 1 -- 172.21.15.163:0/781416395 learned_addr learned my addr 172.21.15.163:0/781416395 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:19.026 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.018+0000 7f837e80d700 1 --2- 172.21.15.163:0/781416395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8378159c60 0x7f83780cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:19.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.018+0000 7f837d80b700 1 --2- 172.21.15.163:0/781416395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8378159890 0x7f8378155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:19.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.018+0000 7f837e00c700 1 -- 172.21.15.163:0/781416395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8378159890 msgr2=0x7f8378155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:19.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.018+0000 7f837e00c700 1 --2- 172.21.15.163:0/781416395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8378159890 0x7f8378155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:19.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.018+0000 7f837e00c700 1 -- 172.21.15.163:0/781416395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8378159c60 msgr2=0x7f83780cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:19.027 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.018+0000 7f837e00c700 1 --2- 172.21.15.163:0/781416395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8378159c60 0x7f83780cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:19.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.018+0000 7f837e00c700 1 -- 172.21.15.163:0/781416395 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8374009a20 con 0x7f8378069560 2023-10-09T06:42:19.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.018+0000 7f837d80b700 1 --2- 172.21.15.163:0/781416395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8378159890 0x7f8378155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-09T06:42:19.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.018+0000 7f837e00c700 1 --2- 172.21.15.163:0/781416395 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8378069560 0x7f837814c570 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f8378070e20 tx=0x7f8374017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:19.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.018+0000 7f836e7fc700 1 -- 172.21.15.163:0/781416395 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8374001090 con 0x7f8378069560 2023-10-09T06:42:19.028 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.018+0000 7f836e7fc700 1 -- 172.21.15.163:0/781416395 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8374012c20 con 0x7f8378069560 2023-10-09T06:42:19.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.018+0000 7f836e7fc700 1 -- 172.21.15.163:0/781416395 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837401e450 con 0x7f8378069560 2023-10-09T06:42:19.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.018+0000 7f8380270700 1 -- 172.21.15.163:0/781416395 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f837815c4f0 con 0x7f8378069560 2023-10-09T06:42:19.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.018+0000 7f8380270700 1 -- 172.21.15.163:0/781416395 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f837814cab0 con 0x7f8378069560 2023-10-09T06:42:19.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.019+0000 7f8380270700 1 -- 172.21.15.163:0/781416395 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f837804c3c0 con 0x7f8378069560 2023-10-09T06:42:19.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.020+0000 7f836e7fc700 1 -- 172.21.15.163:0/781416395 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f837401e800 con 0x7f8378069560 2023-10-09T06:42:19.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.020+0000 7f836e7fc700 1 --2- 172.21.15.163:0/781416395 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f835c067320 0x7f835c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:19.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.020+0000 7f836e7fc700 1 -- 172.21.15.163:0/781416395 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f8374086790 con 0x7f8378069560 2023-10-09T06:42:19.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.020+0000 7f837d80b700 1 --2- 172.21.15.163:0/781416395 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f835c067320 0x7f835c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:19.030 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.021+0000 7f837d80b700 1 --2- 172.21.15.163:0/781416395 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f835c067320 0x7f835c0697d0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f83780b19f0 tx=0x7f8368000bc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:19.031 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.025+0000 7f836e7fc700 1 -- 172.21.15.163:0/781416395 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8374053610 con 0x7f8378069560 2023-10-09T06:42:19.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.205+0000 7f8380270700 1 -- 172.21.15.163:0/781416395 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f83780b1740 con 0x7f835c067320 2023-10-09T06:42:19.206 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.205+0000 7f836e7fc700 1 -- 172.21.15.163:0/781416395 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f83780b1740 con 0x7f835c067320 2023-10-09T06:42:19.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.207+0000 7f8380270700 1 -- 172.21.15.163:0/781416395 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f835c067320 msgr2=0x7f835c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:19.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.207+0000 7f8380270700 1 --2- 172.21.15.163:0/781416395 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f835c067320 0x7f835c0697d0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f83780b19f0 tx=0x7f8368000bc0 comp rx=0 tx=0).stop 2023-10-09T06:42:19.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.207+0000 7f8380270700 1 -- 172.21.15.163:0/781416395 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8378069560 msgr2=0x7f837814c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:19.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.207+0000 7f8380270700 1 --2- 172.21.15.163:0/781416395 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8378069560 0x7f837814c570 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f8378070e20 tx=0x7f8374017000 comp rx=0 tx=0).stop 2023-10-09T06:42:19.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.207+0000 7f8380270700 1 -- 172.21.15.163:0/781416395 shutdown_connections 2023-10-09T06:42:19.209 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.207+0000 7f8380270700 1 --2- 172.21.15.163:0/781416395 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f835c067320 0x7f835c0697d0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:19.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.207+0000 7f8380270700 1 --2- 172.21.15.163:0/781416395 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8378069560 0x7f837814c570 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:19.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.207+0000 7f8380270700 1 --2- 172.21.15.163:0/781416395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8378159c60 0x7f83780cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:19.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.207+0000 7f8380270700 1 --2- 172.21.15.163:0/781416395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8378159890 0x7f8378155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:19.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.207+0000 7f8380270700 1 -- 172.21.15.163:0/781416395 >> 172.21.15.163:0/781416395 conn(0x7f8378068e20 msgr2=0x7f83780b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:19.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.207+0000 7f8380270700 1 -- 172.21.15.163:0/781416395 shutdown_connections 2023-10-09T06:42:19.210 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.207+0000 7f8380270700 1 -- 172.21.15.163:0/781416395 wait complete. 2023-10-09T06:42:19.210 INFO:teuthology.orchestra.run.smithi163.stderr:instructed osd(s) 7 to deep-scrub 2023-10-09T06:42:19.222 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:42:19.336 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.335+0000 7f445b38d700 1 Processor -- start 2023-10-09T06:42:19.336 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.335+0000 7f445b38d700 1 -- start start 2023-10-09T06:42:19.339 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.336+0000 7f445b38d700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4454159c60 0x7f445415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:19.339 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.336+0000 7f445b38d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f445415a160 con 0x7f4454159890 2023-10-09T06:42:19.339 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.336+0000 7f445b38d700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44540d13a0 con 0x7f445415db10 2023-10-09T06:42:19.339 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.336+0000 7f445b38d700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f445415df50 con 0x7f4454159c60 2023-10-09T06:42:19.339 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.336+0000 7f4459129700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f4454159890 0x7f445415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:36920/0 (socket says 172.21.15.163:36920) 2023-10-09T06:42:19.339 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.336+0000 7f4459129700 1 -- 172.21.15.163:0/2026782821 learned_addr learned my addr 172.21.15.163:0/2026782821 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-09T06:42:19.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.337+0000 7f4458928700 1 --2- 172.21.15.163:0/2026782821 >> v2:172.21.15.181:3300/0 conn(0x7f4454159c60 0x7f445415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:19.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.337+0000 7f444bfff700 1 -- 172.21.15.163:0/2026782821 <== mon.0 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1606893850 0 0) 0x7f445415a160 con 0x7f4454159890 2023-10-09T06:42:19.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.337+0000 7f444bfff700 1 -- 172.21.15.163:0/2026782821 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4440003570 con 0x7f4454159890 2023-10-09T06:42:19.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.337+0000 7f4458928700 1 -- 172.21.15.163:0/2026782821 >> v1:172.21.15.181:6789/0 conn(0x7f445415db10 legacy=0x7f4454069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:19.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.337+0000 7f444bfff700 1 -- 172.21.15.163:0/2026782821 <== mon.0 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2996915899 0 0) 0x7f4440003570 con 0x7f4454159890 2023-10-09T06:42:19.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.337+0000 7f4458928700 1 -- 172.21.15.163:0/2026782821 >> v1:172.21.15.163:6789/0 conn(0x7f4454159890 legacy=0x7f445415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:19.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.337+0000 7f4458928700 1 -- 172.21.15.163:0/2026782821 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445415e140 con 0x7f4454159c60 2023-10-09T06:42:19.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.337+0000 7f4458928700 1 --2- 172.21.15.163:0/2026782821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4454159c60 0x7f445415d5d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f44440098e0 tx=0x7f44440091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a1141e95db8ecfb server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:19.341 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.337+0000 7f444bfff700 1 -- 172.21.15.163:0/2026782821 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44500017f0 con 0x7f4454159c60 2023-10-09T06:42:19.341 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.337+0000 7f444bfff700 1 -- 172.21.15.163:0/2026782821 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4444011420 con 0x7f4454159c60 2023-10-09T06:42:19.341 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.337+0000 7f444bfff700 1 -- 172.21.15.163:0/2026782821 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44440117c0 con 0x7f4454159c60 2023-10-09T06:42:19.341 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.338+0000 7f445b38d700 1 -- 172.21.15.163:0/2026782821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4454159c60 msgr2=0x7f445415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:19.341 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.338+0000 7f445b38d700 1 --2- 172.21.15.163:0/2026782821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4454159c60 0x7f445415d5d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f44440098e0 tx=0x7f44440091a0 comp rx=0 tx=0).stop 2023-10-09T06:42:19.341 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.338+0000 7f445b38d700 1 -- 172.21.15.163:0/2026782821 shutdown_connections 2023-10-09T06:42:19.341 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.338+0000 7f445b38d700 1 --2- 172.21.15.163:0/2026782821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4454159c60 0x7f445415d5d0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:19.341 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.338+0000 7f445b38d700 1 -- 172.21.15.163:0/2026782821 >> 172.21.15.163:0/2026782821 conn(0x7f4454068e20 msgr2=0x7f44540d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:19.342 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.338+0000 7f445b38d700 1 -- 172.21.15.163:0/2026782821 shutdown_connections 2023-10-09T06:42:19.342 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.338+0000 7f445b38d700 1 -- 172.21.15.163:0/2026782821 wait complete. 2023-10-09T06:42:19.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.339+0000 7f445b38d700 1 Processor -- start 2023-10-09T06:42:19.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.339+0000 7f445b38d700 1 -- start start 2023-10-09T06:42:19.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.340+0000 7f445b38d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4454159890 0x7f44540cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:19.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.340+0000 7f445b38d700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4454159c60 0x7f445416d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:19.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.340+0000 7f445b38d700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f445415db10 0x7f4454150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:19.345 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.340+0000 7f445b38d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4454151610 con 0x7f4454159890 2023-10-09T06:42:19.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.340+0000 7f445b38d700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4454150d00 con 0x7f445415db10 2023-10-09T06:42:19.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.340+0000 7f445b38d700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44540cdea0 con 0x7f4454159c60 2023-10-09T06:42:19.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.340+0000 7f4459129700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4454159890 0x7f44540cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:19.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.340+0000 7f4459129700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4454159890 0x7f44540cd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:55052/0 (socket says 172.21.15.163:55052) 2023-10-09T06:42:19.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.340+0000 7f4459129700 1 -- 172.21.15.163:0/2463128370 learned_addr learned my addr 172.21.15.163:0/2463128370 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:19.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.340+0000 7f445992a700 1 --2- 172.21.15.163:0/2463128370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f445415db10 0x7f4454150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:19.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.340+0000 7f4459129700 1 -- 172.21.15.163:0/2463128370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4454159c60 msgr2=0x7f445416d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:19.346 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.340+0000 7f4459129700 1 --2- 172.21.15.163:0/2463128370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4454159c60 0x7f445416d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:19.347 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.340+0000 7f4459129700 1 -- 172.21.15.163:0/2463128370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f445415db10 msgr2=0x7f4454150130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:19.347 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.340+0000 7f4459129700 1 --2- 172.21.15.163:0/2463128370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f445415db10 0x7f4454150130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:19.347 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.340+0000 7f4459129700 1 -- 172.21.15.163:0/2463128370 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4444009420 con 0x7f4454159890 2023-10-09T06:42:19.347 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.340+0000 7f4459129700 1 --2- 172.21.15.163:0/2463128370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4454159890 0x7f44540cd100 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f445000ccc0 tx=0x7f4450007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:19.347 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.341+0000 7f44497fa700 1 -- 172.21.15.163:0/2463128370 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445000ce70 con 0x7f4454159890 2023-10-09T06:42:19.347 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.341+0000 7f44497fa700 1 -- 172.21.15.163:0/2463128370 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4450004dc0 con 0x7f4454159890 2023-10-09T06:42:19.347 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.341+0000 7f44497fa700 1 -- 172.21.15.163:0/2463128370 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445000ab90 con 0x7f4454159890 2023-10-09T06:42:19.348 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.341+0000 7f445b38d700 1 -- 172.21.15.163:0/2463128370 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4454150fc0 con 0x7f4454159890 2023-10-09T06:42:19.348 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.341+0000 7f445b38d700 1 -- 172.21.15.163:0/2463128370 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44540cdac0 con 0x7f4454159890 2023-10-09T06:42:19.348 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.342+0000 7f445b38d700 1 -- 172.21.15.163:0/2463128370 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f445404c3c0 con 0x7f4454159890 2023-10-09T06:42:19.348 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.342+0000 7f44497fa700 1 -- 172.21.15.163:0/2463128370 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4450007150 con 0x7f4454159890 2023-10-09T06:42:19.348 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.342+0000 7f44497fa700 1 --2- 172.21.15.163:0/2463128370 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4438067320 0x7f44380697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:19.348 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.342+0000 7f44497fa700 1 -- 172.21.15.163:0/2463128370 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f4450084200 con 0x7f4454159890 2023-10-09T06:42:19.348 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.343+0000 7f4458928700 1 --2- 172.21.15.163:0/2463128370 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4438067320 0x7f44380697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:19.348 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.343+0000 7f4458928700 1 --2- 172.21.15.163:0/2463128370 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4438067320 0x7f44380697d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f445416d650 tx=0x7f4444017000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:19.349 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.347+0000 7f44497fa700 1 -- 172.21.15.163:0/2463128370 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4450051210 con 0x7f4454159890 2023-10-09T06:42:19.526 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.525+0000 7f445b38d700 1 -- 172.21.15.163:0/2463128370 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4454151750 con 0x7f4438067320 2023-10-09T06:42:19.528 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.528+0000 7f44497fa700 1 -- 172.21.15.163:0/2463128370 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+51057 (secure 0 0 0) 0x7f44440116a0 con 0x7f4438067320 2023-10-09T06:42:19.529 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:42:19.530 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.529+0000 7f445b38d700 1 -- 172.21.15.163:0/2463128370 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4438067320 msgr2=0x7f44380697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:19.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.529+0000 7f445b38d700 1 --2- 172.21.15.163:0/2463128370 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4438067320 0x7f44380697d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f445416d650 tx=0x7f4444017000 comp rx=0 tx=0).stop 2023-10-09T06:42:19.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.529+0000 7f445b38d700 1 -- 172.21.15.163:0/2463128370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4454159890 msgr2=0x7f44540cd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:19.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.529+0000 7f445b38d700 1 --2- 172.21.15.163:0/2463128370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4454159890 0x7f44540cd100 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f445000ccc0 tx=0x7f4450007d20 comp rx=0 tx=0).stop 2023-10-09T06:42:19.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.530+0000 7f445b38d700 1 -- 172.21.15.163:0/2463128370 shutdown_connections 2023-10-09T06:42:19.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.530+0000 7f445b38d700 1 --2- 172.21.15.163:0/2463128370 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f4438067320 0x7f44380697d0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:19.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.530+0000 7f445b38d700 1 --2- 172.21.15.163:0/2463128370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4454159890 0x7f44540cd100 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:19.531 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.530+0000 7f445b38d700 1 --2- 172.21.15.163:0/2463128370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f445415db10 0x7f4454150130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:19.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.530+0000 7f445b38d700 1 --2- 172.21.15.163:0/2463128370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4454159c60 0x7f445416d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:19.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.530+0000 7f445b38d700 1 -- 172.21.15.163:0/2463128370 >> 172.21.15.163:0/2463128370 conn(0x7f4454068e20 msgr2=0x7f4454158420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:19.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.530+0000 7f445b38d700 1 -- 172.21.15.163:0/2463128370 shutdown_connections 2023-10-09T06:42:19.532 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:19.530+0000 7f445b38d700 1 -- 172.21.15.163:0/2463128370 wait complete. 2023-10-09T06:42:19.532 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:42:19.543 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":549,"stamp":"2023-10-09T06:42:18.469417+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2432360,"kb_used_data":21288,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752542360,"statfs":{"total":773094113280,"available":770603376640,"internally_reserved":0,"allocated":21798912,"data_stored":10426008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.875801"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":313,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-10-09T06:42:09.663493+0000","last_change":"2023-10-09T06:30:32.218234+0000","last_active":"2023-10-09T06:42:09.663493+0000","last_peered":"2023-10-09T06:42:09.663493+0000","last_clean":"2023-10-09T06:42:09.663493+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:42:09.663493+0000","last_undegraded":"2023-10-09T06:42:09.663493+0000","last_fullsized":"2023-10-09T06:42:09.663493+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-09T06:30:30.762468+0000","last_clean_scrub_stamp":"2023-10-09T06:30:30.762468+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T10:42:45.375556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574993,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304480,"kb_used_data":3096,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067360,"statfs":{"total":96636764160,"available":96324976640,"internally_reserved":0,"allocated":3170304,"data_stored":1746027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58199999999999996,"15min":0.53100000000000003},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.70599999999999996,"5min":3.778,"15min":3.778},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54300000000000004,"15min":0.52200000000000002},"min":{"1min":0.38400000000000001,"5min":0.33400000000000002,"15min":0.29699999999999999},"max":{"1min":0.77100000000000002,"5min":1.768,"15min":1.768},"last":0.49099999999999999}]},{"osd":1,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53400000000000003,"15min":0.52200000000000002},"min":{"1min":0.32100000000000001,"5min":0.31900000000000001,"15min":0.27100000000000002},"max":{"1min":0.72999999999999998,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.53700000000000003},"min":{"1min":0.435,"5min":0.378,"15min":0.29699999999999999},"max":{"1min":0.68100000000000005,"5min":1.806,"15min":1.806},"last":0.41199999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59099999999999997,"15min":0.54100000000000004},"min":{"1min":0.42599999999999999,"5min":0.36399999999999999,"15min":0.32200000000000001},"max":{"1min":0.76000000000000001,"5min":3.7210000000000001,"15min":3.7210000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56699999999999995,"15min":0.54900000000000004},"min":{"1min":0.44600000000000001,"5min":0.35399999999999998,"15min":0.315},"max":{"1min":0.69799999999999995,"5min":1.51,"15min":1.51},"last":0.51200000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60799999999999998,"15min":0.56699999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.33500000000000002},"max":{"1min":0.73299999999999998,"5min":3.8090000000000002,"15min":3.8090000000000002},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.47499999999999998,"5min":0.38400000000000001,"15min":0.36499999999999999},"max":{"1min":0.74299999999999999,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.54800000000000004}]},{"osd":4,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.505,"15min":0.50600000000000001},"min":{"1min":0.31,"5min":0.25600000000000001,"15min":0.24399999999999999},"max":{"1min":0.55100000000000005,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.628,"15min":0.60599999999999998},"min":{"1min":0.35599999999999998,"5min":0.35199999999999998,"15min":0.30299999999999999},"max":{"1min":0.751,"5min":4.8239999999999998,"15min":4.8239999999999998},"last":0.69799999999999995}]},{"osd":5,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53800000000000003,"15min":0.55100000000000005},"min":{"1min":0.34599999999999997,"5min":0.313,"15min":0.307},"max":{"1min":0.64800000000000002,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.63200000000000001,"15min":0.61799999999999999},"min":{"1min":0.38900000000000001,"5min":0.34300000000000003,"15min":0.34200000000000003},"max":{"1min":0.71699999999999997,"5min":3.6909999999999998,"15min":3.6909999999999998},"last":0.65500000000000003}]},{"osd":6,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.59499999999999997,"15min":0.58699999999999997},"min":{"1min":0.39100000000000001,"5min":0.318,"15min":0.28699999999999998},"max":{"1min":0.66600000000000004,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":0.627},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.63400000000000001,"15min":0.60999999999999999},"min":{"1min":0.375,"5min":0.33000000000000002,"15min":0.315},"max":{"1min":0.72499999999999998,"5min":4.7789999999999999,"15min":4.7789999999999999},"last":0.67400000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574992,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303900,"kb_used_data":2516,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067940,"statfs":{"total":96636764160,"available":96325570560,"internally_reserved":0,"allocated":2576384,"data_stored":1155659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54200000000000004,"15min":0.51600000000000001},"min":{"1min":0.40600000000000003,"5min":0.35499999999999998,"15min":0.30399999999999999},"max":{"1min":0.75700000000000001,"5min":0.80500000000000005,"15min":0.82299999999999995},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55200000000000005,"15min":0.52300000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.79000000000000004,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.36099999999999999}]},{"osd":1,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52900000000000003,"15min":0.51600000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.73499999999999999,"5min":0.74199999999999999,"15min":1.1359999999999999},"last":0.504},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.56899999999999995,"15min":0.54200000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.28799999999999998},"max":{"1min":0.754,"5min":0.78700000000000003,"15min":1.1539999999999999},"last":0.49299999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.55900000000000005,"15min":0.53000000000000003},"min":{"1min":0.45400000000000001,"5min":0.34300000000000003,"15min":0.29899999999999999},"max":{"1min":0.74199999999999999,"5min":0.77000000000000002,"15min":0.78400000000000003},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54400000000000004,"15min":0.53200000000000003},"min":{"1min":0.34899999999999998,"5min":0.34499999999999997,"15min":0.313},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.55000000000000004}]},{"osd":3,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56499999999999995,"15min":0.55000000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.75700000000000001,"5min":0.79000000000000004,"15min":0.86799999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.58799999999999997,"15min":0.63700000000000001},"min":{"1min":0.49299999999999999,"5min":0.371,"15min":0.34300000000000003},"max":{"1min":0.71699999999999997,"5min":0.76900000000000002,"15min":4.7510000000000003},"last":0.52300000000000002}]},{"osd":4,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47099999999999997,"15min":0.47399999999999998},"min":{"1min":0.30299999999999999,"5min":0.28999999999999998,"15min":0.27900000000000003},"max":{"1min":0.69699999999999995,"5min":0.74299999999999999,"15min":0.76400000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59099999999999997,"15min":0.58299999999999996},"min":{"1min":0.435,"5min":0.38700000000000001,"15min":0.25800000000000001},"max":{"1min":0.77700000000000002,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.57599999999999996}]},{"osd":5,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.36399999999999999,"5min":0.31900000000000001,"15min":0.246},"max":{"1min":0.75,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.625},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.41999999999999998,"5min":0.40999999999999998,"15min":0.35699999999999998},"max":{"1min":0.73399999999999999,"5min":0.76200000000000001,"15min":0.86599999999999999},"last":0.61499999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56000000000000005,"15min":0.57199999999999995},"min":{"1min":0.39100000000000001,"5min":0.33400000000000002,"15min":0.29899999999999999},"max":{"1min":0.74199999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59599999999999997,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.38400000000000001,"15min":0.33000000000000002},"max":{"1min":0.72099999999999997,"5min":0.84399999999999997,"15min":0.88300000000000001},"last":0.60099999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303900,"kb_used_data":2516,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067940,"statfs":{"total":96636764160,"available":96325570560,"internally_reserved":0,"allocated":2576384,"data_stored":1155659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.32800000000000001,"15min":0.316},"min":{"1min":0.26900000000000002,"5min":0.192,"15min":0.192},"max":{"1min":0.50900000000000001,"5min":0.55600000000000005,"15min":0.80700000000000005},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.432,"5min":0.38100000000000001,"15min":0.36399999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.21199999999999999},"max":{"1min":0.54600000000000004,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.31900000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.36099999999999999,"15min":0.35399999999999998},"min":{"1min":0.317,"5min":0.215,"15min":0.20399999999999999},"max":{"1min":0.53100000000000003,"5min":0.53700000000000003,"15min":0.68300000000000005},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.40899999999999997,"15min":0.39300000000000002},"min":{"1min":0.36799999999999999,"5min":0.25800000000000001,"15min":0.251},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.35299999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.39800000000000002,"15min":0.376},"min":{"1min":0.32600000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.373},{"interface":"front","average":{"1min":0.442,"5min":0.375,"15min":0.46200000000000002},"min":{"1min":0.28599999999999998,"5min":0.23400000000000001,"15min":0.22700000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":5.8529999999999998},"last":0.36099999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.59999999999999998,"15min":0.58499999999999996},"min":{"1min":0.45300000000000001,"5min":0.36199999999999999,"15min":0.32200000000000001},"max":{"1min":0.79800000000000004,"5min":1.01,"15min":1.01},"last":0.753},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61799999999999999,"15min":0.59799999999999998},"min":{"1min":0.47299999999999998,"5min":0.40000000000000002,"15min":0.37},"max":{"1min":0.79300000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.77000000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.59799999999999998,"15min":0.67500000000000004},"min":{"1min":0.40899999999999997,"5min":0.36499999999999999,"15min":0.34599999999999997},"max":{"1min":0.85599999999999998,"5min":0.92500000000000004,"15min":5.9459999999999997},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.621,"15min":0.61399999999999999},"min":{"1min":0.55700000000000005,"5min":0.38100000000000001,"15min":0.34300000000000003},"max":{"1min":0.82599999999999996,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.81399999999999995}]},{"osd":6,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.57799999999999996,"15min":0.65000000000000002},"min":{"1min":0.44600000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":5.9160000000000004},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.51700000000000002,"5min":0.379,"15min":0.35699999999999998},"max":{"1min":0.86699999999999999,"5min":0.88600000000000001,"15min":0.91400000000000003},"last":0.78400000000000003}]},{"osd":7,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.51400000000000001,"5min":0.39000000000000001,"15min":0.379},"max":{"1min":0.76800000000000002,"5min":0.91000000000000003,"15min":0.97099999999999997},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.59099999999999997,"15min":0.67600000000000005},"min":{"1min":0.48399999999999999,"5min":0.41399999999999998,"15min":0.38100000000000001},"max":{"1min":0.83399999999999996,"5min":0.86099999999999999,"15min":5.9729999999999999},"last":0.79700000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574994,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304480,"kb_used_data":3096,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067360,"statfs":{"total":96636764160,"available":96324976640,"internally_reserved":0,"allocated":3170304,"data_stored":1746027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.34000000000000002,"15min":0.32200000000000001},"min":{"1min":0.252,"5min":0.216,"15min":0.21299999999999999},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.35799999999999998,"15min":0.34699999999999998},"min":{"1min":0.24399999999999999,"5min":0.222,"15min":0.214},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.94499999999999995},"last":0.29899999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.38300000000000001,"15min":0.36799999999999999},"min":{"1min":0.27100000000000002,"5min":0.22,"15min":0.20200000000000001},"max":{"1min":0.53000000000000003,"5min":0.54000000000000004,"15min":0.75600000000000001},"last":0.441},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.41399999999999998,"15min":0.39500000000000002},"min":{"1min":0.34599999999999997,"5min":0.29099999999999998,"15min":0.28000000000000003},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.91700000000000004},"last":0.34000000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.36499999999999999,"15min":0.372},"min":{"1min":0.318,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":1.554},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.40200000000000002,"15min":0.39500000000000002},"min":{"1min":0.32400000000000001,"5min":0.27700000000000002,"15min":0.25800000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.79000000000000004},"last":0.39400000000000002}]},{"osd":4,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58699999999999997,"15min":0.56299999999999994},"min":{"1min":0.46300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85699999999999998,"5min":0.92600000000000005,"15min":1.212},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58699999999999997,"15min":0.65700000000000003},"min":{"1min":0.43099999999999999,"5min":0.35799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85599999999999998,"5min":0.90800000000000003,"15min":5.484},"last":0.53000000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58499999999999996,"15min":0.65900000000000003},"min":{"1min":0.437,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83099999999999996,"5min":0.97399999999999998,"15min":5.5449999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.47499999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.875,"5min":0.998,"15min":1.113},"last":0.53900000000000003}]},{"osd":6,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.42399999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86899999999999999,"5min":0.871,"15min":1.0529999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.627,"15min":0.61399999999999999},"min":{"1min":0.505,"5min":0.42899999999999999,"15min":0.41599999999999998},"max":{"1min":0.88400000000000001,"5min":0.96099999999999997,"15min":1.169},"last":0.55400000000000005}]},{"osd":7,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60399999999999998,"15min":0.59399999999999997},"min":{"1min":0.47299999999999998,"5min":0.40200000000000002,"15min":0.371},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":1.2390000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59899999999999998,"15min":0.67800000000000005},"min":{"1min":0.48399999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83199999999999996,"5min":0.94299999999999995,"15min":5.5800000000000001},"last":0.43099999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303900,"kb_used_data":2516,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067940,"statfs":{"total":96636764160,"available":96325570560,"internally_reserved":0,"allocated":2576384,"data_stored":1155659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33400000000000002,"15min":0.34000000000000002},"min":{"1min":0.252,"5min":0.20000000000000001,"15min":0.19900000000000001},"max":{"1min":0.51800000000000002,"5min":0.66000000000000003,"15min":1.4199999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.34499999999999997,"15min":0.46100000000000002},"min":{"1min":0.28000000000000003,"5min":0.221,"15min":0.19900000000000001},"max":{"1min":0.53100000000000003,"5min":0.55600000000000005,"15min":7.2699999999999996},"last":0.29499999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.34200000000000003,"15min":0.34100000000000003},"min":{"1min":0.26200000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.497,"5min":0.56399999999999995,"15min":0.65400000000000003},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.434,"5min":0.39600000000000002,"15min":0.51400000000000001},"min":{"1min":0.33500000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.56799999999999995,"5min":0.60399999999999998,"15min":7.327},"last":0.42799999999999999}]},{"osd":3,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.41099999999999998,"15min":0.39400000000000002},"min":{"1min":0.39600000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.56899999999999995,"5min":0.64600000000000002,"15min":0.68600000000000005},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.39000000000000001,"15min":0.38700000000000001},"min":{"1min":0.32600000000000001,"5min":0.252,"15min":0.23999999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.72699999999999998},"last":0.28000000000000003}]},{"osd":4,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56299999999999994,"15min":0.54800000000000004},"min":{"1min":0.45800000000000002,"5min":0.33600000000000002,"15min":0.317},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.58199999999999996,"15min":0.68999999999999995},"min":{"1min":0.46899999999999997,"5min":0.35999999999999999,"15min":0.33600000000000002},"max":{"1min":0.82599999999999996,"5min":0.91100000000000003,"15min":7.2389999999999999},"last":0.53800000000000003}]},{"osd":5,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59499999999999997,"15min":0.77300000000000002},"min":{"1min":0.495,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.872,"5min":0.89800000000000002,"15min":10.890000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.60899999999999999,"15min":0.60299999999999998},"min":{"1min":0.45600000000000002,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":0.80000000000000004,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.67300000000000004}]},{"osd":6,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58099999999999996,"15min":0.69599999999999995},"min":{"1min":0.50800000000000001,"5min":0.35999999999999999,"15min":0.33500000000000002},"max":{"1min":0.77600000000000002,"5min":0.82499999999999996,"15min":7.2999999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60499999999999998,"15min":0.58299999999999996},"min":{"1min":0.44700000000000001,"5min":0.40300000000000002,"15min":0.377},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.60099999999999998}]},{"osd":7,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.61099999999999999,"15min":0.72599999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.37},"max":{"1min":0.80100000000000005,"5min":0.91300000000000003,"15min":7.1879999999999997},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63,"15min":0.624},"min":{"1min":0.53300000000000003,"5min":0.437,"15min":0.38},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.56200000000000006}]}]},{"osd":3,"up_from":13,"seq":55834574994,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303900,"kb_used_data":2516,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067940,"statfs":{"total":96636764160,"available":96325570560,"internally_reserved":0,"allocated":2576384,"data_stored":1155659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.34200000000000003,"15min":0.33800000000000002},"min":{"1min":0.23799999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.55400000000000005,"5min":0.61699999999999999,"15min":5.8700000000000001},"last":0.375},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.36299999999999999,"15min":0.36799999999999999},"min":{"1min":0.255,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.55400000000000005,"5min":0.58499999999999996,"15min":5.6769999999999996},"last":0.46300000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.36599999999999999,"15min":0.36399999999999999},"min":{"1min":0.28599999999999998,"5min":0.21299999999999999,"15min":0.21199999999999999},"max":{"1min":0.46700000000000003,"5min":0.61199999999999999,"15min":4.0469999999999997},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.40200000000000002,"15min":0.39500000000000002},"min":{"1min":0.32900000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.60199999999999998,"5min":0.68600000000000005,"15min":4.1509999999999998},"last":0.438}]},{"osd":2,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.371,"15min":0.373},"min":{"1min":0.27300000000000002,"5min":0.23200000000000001,"15min":0.222},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":3.1139999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.39500000000000002,"15min":0.38900000000000001},"min":{"1min":0.28799999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.54700000000000004,"5min":0.66900000000000004,"15min":3.3180000000000001},"last":0.45400000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56999999999999995,"15min":0.57299999999999995},"min":{"1min":0.42799999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.75700000000000001,"5min":0.86699999999999999,"15min":3.0899999999999999},"last":0.501},{"interface":"front","average":{"1min":0.62,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.432,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79300000000000004,"5min":0.84499999999999997,"15min":3.0710000000000002},"last":0.49199999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.58099999999999996,"15min":0.59099999999999997},"min":{"1min":0.45300000000000001,"5min":0.33600000000000002,"15min":0.32700000000000001},"max":{"1min":0.86399999999999999,"5min":0.877,"15min":6.2300000000000004},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.58699999999999997,"15min":0.59699999999999998},"min":{"1min":0.498,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78800000000000003,"5min":0.83599999999999997,"15min":3.0219999999999998},"last":0.52300000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.61899999999999999,"15min":0.60499999999999998},"min":{"1min":0.47899999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.147,"5min":1.147,"15min":2.6850000000000001},"last":0.628},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.65000000000000002,"15min":0.65700000000000003},"min":{"1min":0.5,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":6.5970000000000004},"last":0.61799999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.61199999999999999,"15min":0.61699999999999999},"min":{"1min":0.46600000000000003,"5min":0.41499999999999998,"15min":0.38200000000000001},"max":{"1min":0.81799999999999995,"5min":0.86099999999999999,"15min":4.2629999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60799999999999998,"15min":0.61899999999999999},"min":{"1min":0.47899999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.89200000000000002,"5min":0.89900000000000002,"15min":7.3650000000000002},"last":0.60699999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574993,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303900,"kb_used_data":2516,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067940,"statfs":{"total":96636764160,"available":96325570560,"internally_reserved":0,"allocated":2576384,"data_stored":1155659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54800000000000004,"15min":0.54500000000000004},"min":{"1min":0.38900000000000001,"5min":0.307,"15min":0.27000000000000002},"max":{"1min":0.71899999999999997,"5min":0.88900000000000001,"15min":1.083},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.55200000000000005,"15min":0.53000000000000003},"min":{"1min":0.50800000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":1.054},"last":0.56999999999999995}]},{"osd":1,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56599999999999995,"15min":0.55100000000000005},"min":{"1min":0.52400000000000002,"5min":0.28399999999999997,"15min":0.27100000000000002},"max":{"1min":0.77300000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53100000000000003,"15min":0.50800000000000001},"min":{"1min":0.38500000000000001,"5min":0.33800000000000002,"15min":0.31},"max":{"1min":0.75,"5min":0.76300000000000001,"15min":0.79100000000000004},"last":0.52300000000000002}]},{"osd":2,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.54700000000000004,"15min":0.54000000000000004},"min":{"1min":0.443,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76500000000000001,"5min":0.84899999999999998,"15min":1.0249999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.499,"5min":0.38100000000000001,"15min":0.36299999999999999},"max":{"1min":0.73399999999999999,"5min":0.90500000000000003,"15min":0.93799999999999994},"last":0.63800000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55300000000000005,"15min":0.54800000000000004},"min":{"1min":0.40699999999999997,"5min":0.33300000000000002,"15min":0.29199999999999998},"max":{"1min":0.747,"5min":0.86899999999999999,"15min":0.98499999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.47799999999999998,"5min":0.38700000000000001,"15min":0.34100000000000003},"max":{"1min":0.68200000000000005,"5min":0.92900000000000005,"15min":1.1319999999999999},"last":0.42499999999999999}]},{"osd":5,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.50600000000000001,"15min":0.53300000000000003},"min":{"1min":0.35499999999999998,"5min":0.307,"15min":0.27300000000000002},"max":{"1min":0.55900000000000005,"5min":0.83299999999999996,"15min":0.998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57699999999999996,"15min":0.59399999999999997},"min":{"1min":0.40000000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71999999999999997,"5min":0.95599999999999996,"15min":0.97999999999999998},"last":0.623}]},{"osd":6,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.54600000000000004,"15min":0.55600000000000005},"min":{"1min":0.36899999999999999,"5min":0.35499999999999998,"15min":0.29999999999999999},"max":{"1min":0.67500000000000004,"5min":0.86299999999999999,"15min":1.0189999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.56999999999999995,"15min":0.58399999999999996},"min":{"1min":0.38900000000000001,"5min":0.36399999999999999,"15min":0.32700000000000001},"max":{"1min":0.72699999999999998,"5min":0.85599999999999998,"15min":1.01},"last":0.67000000000000004}]},{"osd":7,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55700000000000005,"15min":0.56200000000000006},"min":{"1min":0.42399999999999999,"5min":0.35299999999999998,"15min":0.28299999999999997},"max":{"1min":0.65400000000000003,"5min":0.84399999999999997,"15min":0.90000000000000002},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.60599999999999998,"15min":0.625},"min":{"1min":0.374,"5min":0.34999999999999998,"15min":0.32200000000000001},"max":{"1min":0.71299999999999997,"5min":0.93799999999999994,"15min":1.175},"last":0.65000000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574993,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303900,"kb_used_data":2516,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067940,"statfs":{"total":96636764160,"available":96325570560,"internally_reserved":0,"allocated":2576384,"data_stored":1155659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54200000000000004,"15min":0.51700000000000002},"min":{"1min":0.36399999999999999,"5min":0.29499999999999998,"15min":0.28599999999999998},"max":{"1min":0.76400000000000001,"5min":0.79700000000000004,"15min":0.86599999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55400000000000005,"15min":0.52800000000000002},"min":{"1min":0.45000000000000001,"5min":0.36399999999999999,"15min":0.28799999999999998},"max":{"1min":0.83599999999999997,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.377}]},{"osd":1,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52700000000000002,"15min":0.51500000000000001},"min":{"1min":0.38800000000000001,"5min":0.34100000000000003,"15min":0.311},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.54400000000000004,"15min":0.53400000000000003},"min":{"1min":0.41499999999999998,"5min":0.36899999999999999,"15min":0.30199999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.81599999999999995},"last":0.54200000000000004}]},{"osd":2,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58199999999999996,"15min":0.55800000000000005},"min":{"1min":0.39300000000000002,"5min":0.34699999999999998,"15min":0.28199999999999997},"max":{"1min":0.79800000000000004,"5min":1.139,"15min":1.139},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.52600000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78100000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.48699999999999999}]},{"osd":3,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57099999999999995,"15min":0.55200000000000005},"min":{"1min":0.43099999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.75,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.436},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.45900000000000002,"5min":0.40699999999999997,"15min":0.312},"max":{"1min":0.80200000000000005,"5min":0.80600000000000005,"15min":0.81100000000000005},"last":0.63500000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.33600000000000002,"5min":0.28399999999999997,"15min":0.26700000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53600000000000003,"15min":0.54200000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.27700000000000002},"max":{"1min":0.751,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.52500000000000002}]},{"osd":6,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.35599999999999998,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":0.73499999999999999,"5min":1.752,"15min":1.752},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55600000000000005,"15min":0.55900000000000005},"min":{"1min":0.39100000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.60899999999999999}]},{"osd":7,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56399999999999995,"15min":0.57599999999999996},"min":{"1min":0.375,"5min":0.311,"15min":0.311},"max":{"1min":0.79900000000000004,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58599999999999997,"15min":0.60299999999999998},"min":{"1min":0.40799999999999997,"5min":0.372,"15min":0.314},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.90600000000000003},"last":0.72599999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:42:19.547 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-09T06:30:30.762468+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=9, tm_hour=6, tm_min=30, tm_sec=30, tm_wday=0, tm_yday=282, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=9, tm_hour=6, tm_min=42, tm_sec=14, tm_wday=0, tm_yday=282, tm_isdst=0) 2023-10-09T06:42:19.547 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-09T06:42:39.549 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-09T06:42:39.677 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.676+0000 7f9255a91700 1 Processor -- start 2023-10-09T06:42:39.677 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.676+0000 7f9255a91700 1 -- start start 2023-10-09T06:42:39.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.677+0000 7f9255a91700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9250156310 0x7f92500c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:39.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.677+0000 7f9255a91700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f92501566e0 0x7f92500ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:39.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.677+0000 7f9255a91700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92500ca300 0x7f92500ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:39.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.677+0000 7f9255a91700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92500ca860 con 0x7f9250156310 2023-10-09T06:42:39.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.677+0000 7f9255a91700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92500cab40 con 0x7f92501566e0 2023-10-09T06:42:39.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.677+0000 7f9255a91700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92500cae20 con 0x7f92500ca300 2023-10-09T06:42:39.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.677+0000 7f9254a8f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9250156310 0x7f92500c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:39.680 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.677+0000 7f9254a8f700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9250156310 0x7f92500c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:52960/0 (socket says 172.21.15.163:52960) 2023-10-09T06:42:39.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.677+0000 7f9254a8f700 1 -- 172.21.15.163:0/1596701949 learned_addr learned my addr 172.21.15.163:0/1596701949 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:39.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.677+0000 7f9254a8f700 1 -- 172.21.15.163:0/1596701949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92500ca300 msgr2=0x7f92500ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:39.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.677+0000 7f9254a8f700 1 --2- 172.21.15.163:0/1596701949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92500ca300 0x7f92500ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:39.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.677+0000 7f9254a8f700 1 -- 172.21.15.163:0/1596701949 >> v2:172.21.15.181:3300/0 conn(0x7f92501566e0 msgr2=0x7f92500ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:39.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.677+0000 7f9247fff700 1 --2- 172.21.15.163:0/1596701949 >> v2:172.21.15.181:3300/0 conn(0x7f92501566e0 0x7f92500ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:39.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.678+0000 7f9254a8f700 1 --2- 172.21.15.163:0/1596701949 >> v2:172.21.15.181:3300/0 conn(0x7f92501566e0 0x7f92500ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:39.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.678+0000 7f9254a8f700 1 -- 172.21.15.163:0/1596701949 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92500cb100 con 0x7f9250156310 2023-10-09T06:42:39.681 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.678+0000 7f9254a8f700 1 --2- 172.21.15.163:0/1596701949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9250156310 0x7f92500c9dc0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f925006c000 tx=0x7f924c00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7c34c5c0eb60bd server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:39.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.678+0000 7f92477fe700 1 -- 172.21.15.163:0/1596701949 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c010070 con 0x7f9250156310 2023-10-09T06:42:39.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.678+0000 7f92477fe700 1 -- 172.21.15.163:0/1596701949 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f924c00de90 con 0x7f9250156310 2023-10-09T06:42:39.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.678+0000 7f92477fe700 1 -- 172.21.15.163:0/1596701949 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c00b320 con 0x7f9250156310 2023-10-09T06:42:39.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.678+0000 7f9255a91700 1 -- 172.21.15.163:0/1596701949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9250156310 msgr2=0x7f92500c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:39.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.678+0000 7f9255a91700 1 --2- 172.21.15.163:0/1596701949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9250156310 0x7f92500c9dc0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f925006c000 tx=0x7f924c00ba50 comp rx=0 tx=0).stop 2023-10-09T06:42:39.682 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.678+0000 7f9255a91700 1 -- 172.21.15.163:0/1596701949 shutdown_connections 2023-10-09T06:42:39.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.678+0000 7f9255a91700 1 --2- 172.21.15.163:0/1596701949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92500ca300 0x7f92500ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:39.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.678+0000 7f9255a91700 1 --2- 172.21.15.163:0/1596701949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9250156310 0x7f92500c9dc0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:39.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.678+0000 7f9255a91700 1 --2- 172.21.15.163:0/1596701949 >> v2:172.21.15.181:3300/0 conn(0x7f92501566e0 0x7f92500ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:39.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.678+0000 7f9255a91700 1 -- 172.21.15.163:0/1596701949 >> 172.21.15.163:0/1596701949 conn(0x7f92500cc1f0 msgr2=0x7f92500cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:39.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.679+0000 7f9255a91700 1 -- 172.21.15.163:0/1596701949 shutdown_connections 2023-10-09T06:42:39.683 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.679+0000 7f9255a91700 1 -- 172.21.15.163:0/1596701949 wait complete. 2023-10-09T06:42:39.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.680+0000 7f9255a91700 1 Processor -- start 2023-10-09T06:42:39.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.680+0000 7f9255a91700 1 -- start start 2023-10-09T06:42:39.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.681+0000 7f9255a91700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92500ca300 0x7f9250144c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:39.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.681+0000 7f9255a91700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9250156310 0x7f92501596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:39.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.681+0000 7f9255a91700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92501566e0 0x7f92501519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:39.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.681+0000 7f9255a91700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925015a060 con 0x7f92501566e0 2023-10-09T06:42:39.686 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.681+0000 7f9255a91700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925015a340 con 0x7f9250156310 2023-10-09T06:42:39.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.681+0000 7f9255a91700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f925015a620 con 0x7f92500ca300 2023-10-09T06:42:39.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.681+0000 7f9255290700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92501566e0 0x7f92501519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:39.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.681+0000 7f9255290700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92501566e0 0x7f92501519f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:52972/0 (socket says 172.21.15.163:52972) 2023-10-09T06:42:39.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.681+0000 7f9254a8f700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92500ca300 0x7f9250144c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:39.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.681+0000 7f9255290700 1 -- 172.21.15.163:0/2292501701 learned_addr learned my addr 172.21.15.163:0/2292501701 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:39.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.681+0000 7f9247fff700 1 --2- 172.21.15.163:0/2292501701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9250156310 0x7f92501596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:39.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.681+0000 7f9255290700 1 -- 172.21.15.163:0/2292501701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92500ca300 msgr2=0x7f9250144c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:39.687 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.681+0000 7f9255290700 1 --2- 172.21.15.163:0/2292501701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92500ca300 0x7f9250144c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:39.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.681+0000 7f9255290700 1 -- 172.21.15.163:0/2292501701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9250156310 msgr2=0x7f92501596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:39.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.681+0000 7f9255290700 1 --2- 172.21.15.163:0/2292501701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9250156310 0x7f92501596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:39.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.681+0000 7f9255290700 1 -- 172.21.15.163:0/2292501701 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924c00bcf0 con 0x7f92501566e0 2023-10-09T06:42:39.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.681+0000 7f9255290700 1 --2- 172.21.15.163:0/2292501701 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92501566e0 0x7f92501519f0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f924800b810 tx=0x7f92480091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:39.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.681+0000 7f9244ff9700 1 -- 172.21.15.163:0/2292501701 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924800f510 con 0x7f92501566e0 2023-10-09T06:42:39.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.681+0000 7f9244ff9700 1 -- 172.21.15.163:0/2292501701 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f924800f670 con 0x7f92501566e0 2023-10-09T06:42:39.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.681+0000 7f9255a91700 1 -- 172.21.15.163:0/2292501701 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f925015a900 con 0x7f92501566e0 2023-10-09T06:42:39.688 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.681+0000 7f9244ff9700 1 -- 172.21.15.163:0/2292501701 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924800fa30 con 0x7f92501566e0 2023-10-09T06:42:39.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.681+0000 7f9255a91700 1 -- 172.21.15.163:0/2292501701 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9250145150 con 0x7f92501566e0 2023-10-09T06:42:39.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.682+0000 7f9255a91700 1 -- 172.21.15.163:0/2292501701 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f925015ae20 con 0x7f92501566e0 2023-10-09T06:42:39.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.683+0000 7f9244ff9700 1 -- 172.21.15.163:0/2292501701 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f924801a400 con 0x7f92501566e0 2023-10-09T06:42:39.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.683+0000 7f9244ff9700 1 --2- 172.21.15.163:0/2292501701 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f923c067320 0x7f923c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:39.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.683+0000 7f9244ff9700 1 -- 172.21.15.163:0/2292501701 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f92480842c0 con 0x7f92501566e0 2023-10-09T06:42:39.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.683+0000 7f9254a8f700 1 --2- 172.21.15.163:0/2292501701 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f923c067320 0x7f923c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:39.689 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.684+0000 7f9254a8f700 1 --2- 172.21.15.163:0/2292501701 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f923c067320 0x7f923c0697d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f924c00ba90 tx=0x7f924c01a000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:39.690 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.687+0000 7f9244ff9700 1 -- 172.21.15.163:0/2292501701 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f92480513d0 con 0x7f92501566e0 2023-10-09T06:42:39.866 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.865+0000 7f9255a91700 1 -- 172.21.15.163:0/2292501701 --> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f925015a4b0 con 0x7f923c067320 2023-10-09T06:42:39.869 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.868+0000 7f9244ff9700 1 -- 172.21.15.163:0/2292501701 <== mgr.4103 v2:172.21.15.181:6832/90545 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50973 (secure 0 0 0) 0x7f925015a4b0 con 0x7f923c067320 2023-10-09T06:42:39.870 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:42:39.871 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.870+0000 7f9255a91700 1 -- 172.21.15.163:0/2292501701 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f923c067320 msgr2=0x7f923c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:39.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.870+0000 7f9255a91700 1 --2- 172.21.15.163:0/2292501701 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f923c067320 0x7f923c0697d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f924c00ba90 tx=0x7f924c01a000 comp rx=0 tx=0).stop 2023-10-09T06:42:39.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.870+0000 7f9255a91700 1 -- 172.21.15.163:0/2292501701 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92501566e0 msgr2=0x7f92501519f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:39.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.870+0000 7f9255a91700 1 --2- 172.21.15.163:0/2292501701 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92501566e0 0x7f92501519f0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f924800b810 tx=0x7f92480091d0 comp rx=0 tx=0).stop 2023-10-09T06:42:39.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.870+0000 7f9255a91700 1 -- 172.21.15.163:0/2292501701 shutdown_connections 2023-10-09T06:42:39.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.870+0000 7f9255a91700 1 --2- 172.21.15.163:0/2292501701 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f923c067320 0x7f923c0697d0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:39.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.870+0000 7f9255a91700 1 --2- 172.21.15.163:0/2292501701 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92501566e0 0x7f92501519f0 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:39.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.870+0000 7f9255a91700 1 --2- 172.21.15.163:0/2292501701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92500ca300 0x7f9250144c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:39.872 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.870+0000 7f9255a91700 1 --2- 172.21.15.163:0/2292501701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9250156310 0x7f92501596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:39.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.870+0000 7f9255a91700 1 -- 172.21.15.163:0/2292501701 >> 172.21.15.163:0/2292501701 conn(0x7f92500cc1f0 msgr2=0x7f92500afcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:39.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.870+0000 7f9255a91700 1 -- 172.21.15.163:0/2292501701 shutdown_connections 2023-10-09T06:42:39.873 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.870+0000 7f9255a91700 1 -- 172.21.15.163:0/2292501701 wait complete. 2023-10-09T06:42:39.873 INFO:teuthology.orchestra.run.smithi163.stderr:dumped all 2023-10-09T06:42:39.883 INFO:teuthology.orchestra.run.smithi163.stdout:{"pg_ready":true,"pg_map":{"version":559,"stamp":"2023-10-09T06:42:38.473713+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2432360,"kb_used_data":21288,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752542360,"statfs":{"total":773094113280,"available":770603376640,"internally_reserved":0,"allocated":21798912,"data_stored":10426008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002613"},"pg_stats":[{"pgid":"1.0","version":"202'119","reported_seq":326,"reported_epoch":202,"state":"active+clean","last_fresh":"2023-10-09T06:42:19.569712+0000","last_change":"2023-10-09T06:42:19.569683+0000","last_active":"2023-10-09T06:42:19.569712+0000","last_peered":"2023-10-09T06:42:19.569712+0000","last_clean":"2023-10-09T06:42:19.569712+0000","last_became_active":"2023-10-09T06:30:32.217265+0000","last_became_peered":"2023-10-09T06:30:32.217265+0000","last_unstale":"2023-10-09T06:42:19.569712+0000","last_undegraded":"2023-10-09T06:42:19.569712+0000","last_fullsized":"2023-10-09T06:42:19.569712+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"202'119","last_scrub_stamp":"2023-10-09T06:42:19.569589+0000","last_deep_scrub":"202'119","last_deep_scrub_stamp":"2023-10-09T06:42:19.569589+0000","last_clean_scrub_stamp":"2023-10-09T06:42:19.569589+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T08:35:21.858655+0000","scrub_duration":0.0049514889999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574997,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304480,"kb_used_data":3096,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067360,"statfs":{"total":96636764160,"available":96324976640,"internally_reserved":0,"allocated":3170304,"data_stored":1746027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58199999999999996,"15min":0.53100000000000003},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.70599999999999996,"5min":3.778,"15min":3.778},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54300000000000004,"15min":0.52200000000000002},"min":{"1min":0.38400000000000001,"5min":0.33400000000000002,"15min":0.29699999999999999},"max":{"1min":0.77100000000000002,"5min":1.768,"15min":1.768},"last":0.54000000000000004}]},{"osd":1,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53400000000000003,"15min":0.52200000000000002},"min":{"1min":0.32100000000000001,"5min":0.31900000000000001,"15min":0.27100000000000002},"max":{"1min":0.72999999999999998,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.53700000000000003},"min":{"1min":0.435,"5min":0.378,"15min":0.29699999999999999},"max":{"1min":0.68100000000000005,"5min":1.806,"15min":1.806},"last":0.50900000000000001}]},{"osd":2,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59099999999999997,"15min":0.54100000000000004},"min":{"1min":0.42599999999999999,"5min":0.36399999999999999,"15min":0.32200000000000001},"max":{"1min":0.76000000000000001,"5min":3.7210000000000001,"15min":3.7210000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56699999999999995,"15min":0.54900000000000004},"min":{"1min":0.44600000000000001,"5min":0.35399999999999998,"15min":0.315},"max":{"1min":0.69799999999999995,"5min":1.51,"15min":1.51},"last":0.56399999999999995}]},{"osd":3,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60799999999999998,"15min":0.56699999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.33500000000000002},"max":{"1min":0.73299999999999998,"5min":3.8090000000000002,"15min":3.8090000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57499999999999996,"15min":0.58199999999999996},"min":{"1min":0.47499999999999998,"5min":0.38400000000000001,"15min":0.36499999999999999},"max":{"1min":0.74299999999999999,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.60699999999999998}]},{"osd":4,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.505,"15min":0.50600000000000001},"min":{"1min":0.31,"5min":0.25600000000000001,"15min":0.24399999999999999},"max":{"1min":0.55100000000000005,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.628,"15min":0.60599999999999998},"min":{"1min":0.35599999999999998,"5min":0.35199999999999998,"15min":0.30299999999999999},"max":{"1min":0.751,"5min":4.8239999999999998,"15min":4.8239999999999998},"last":0.629}]},{"osd":5,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53800000000000003,"15min":0.55100000000000005},"min":{"1min":0.34599999999999997,"5min":0.313,"15min":0.307},"max":{"1min":0.64800000000000002,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.63200000000000001,"15min":0.61799999999999999},"min":{"1min":0.38900000000000001,"5min":0.34300000000000003,"15min":0.34200000000000003},"max":{"1min":0.71699999999999997,"5min":3.6909999999999998,"15min":3.6909999999999998},"last":0.61799999999999999}]},{"osd":6,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.59499999999999997,"15min":0.58699999999999997},"min":{"1min":0.39100000000000001,"5min":0.318,"15min":0.28699999999999998},"max":{"1min":0.66600000000000004,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.63400000000000001,"15min":0.60999999999999999},"min":{"1min":0.375,"5min":0.33000000000000002,"15min":0.315},"max":{"1min":0.72499999999999998,"5min":4.7789999999999999,"15min":4.7789999999999999},"last":0.66100000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574996,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303900,"kb_used_data":2516,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067940,"statfs":{"total":96636764160,"available":96325570560,"internally_reserved":0,"allocated":2576384,"data_stored":1155659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54200000000000004,"15min":0.51600000000000001},"min":{"1min":0.40600000000000003,"5min":0.35499999999999998,"15min":0.30399999999999999},"max":{"1min":0.75700000000000001,"5min":0.80500000000000005,"15min":0.82299999999999995},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55200000000000005,"15min":0.52300000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.79000000000000004,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.77000000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52900000000000003,"15min":0.51600000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.73499999999999999,"5min":0.74199999999999999,"15min":1.1359999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.56899999999999995,"15min":0.54200000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.28799999999999998},"max":{"1min":0.754,"5min":0.78700000000000003,"15min":1.1539999999999999},"last":0.80800000000000005}]},{"osd":2,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.55900000000000005,"15min":0.53000000000000003},"min":{"1min":0.45400000000000001,"5min":0.34300000000000003,"15min":0.29899999999999999},"max":{"1min":0.74199999999999999,"5min":0.77000000000000002,"15min":0.78400000000000003},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54400000000000004,"15min":0.53200000000000003},"min":{"1min":0.34899999999999998,"5min":0.34499999999999997,"15min":0.313},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.68899999999999995}]},{"osd":3,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56499999999999995,"15min":0.55000000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.75700000000000001,"5min":0.79000000000000004,"15min":0.86799999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.58799999999999997,"15min":0.63700000000000001},"min":{"1min":0.49299999999999999,"5min":0.371,"15min":0.34300000000000003},"max":{"1min":0.71699999999999997,"5min":0.76900000000000002,"15min":4.7510000000000003},"last":0.76000000000000001}]},{"osd":4,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47099999999999997,"15min":0.47399999999999998},"min":{"1min":0.30299999999999999,"5min":0.28999999999999998,"15min":0.27900000000000003},"max":{"1min":0.69699999999999995,"5min":0.74299999999999999,"15min":0.76400000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59099999999999997,"15min":0.58299999999999996},"min":{"1min":0.435,"5min":0.38700000000000001,"15min":0.25800000000000001},"max":{"1min":0.77700000000000002,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.624}]},{"osd":5,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.36399999999999999,"5min":0.31900000000000001,"15min":0.246},"max":{"1min":0.75,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.502},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.41999999999999998,"5min":0.40999999999999998,"15min":0.35699999999999998},"max":{"1min":0.73399999999999999,"5min":0.76200000000000001,"15min":0.86599999999999999},"last":0.60099999999999998}]},{"osd":7,"last update":"Mon Oct 9 06:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56000000000000005,"15min":0.57199999999999995},"min":{"1min":0.39100000000000001,"5min":0.33400000000000002,"15min":0.29899999999999999},"max":{"1min":0.74199999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59599999999999997,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.38400000000000001,"15min":0.33000000000000002},"max":{"1min":0.72099999999999997,"5min":0.84399999999999997,"15min":0.88300000000000001},"last":0.58599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303900,"kb_used_data":2516,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067940,"statfs":{"total":96636764160,"available":96325570560,"internally_reserved":0,"allocated":2576384,"data_stored":1155659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.32800000000000001,"15min":0.316},"min":{"1min":0.26900000000000002,"5min":0.192,"15min":0.192},"max":{"1min":0.50900000000000001,"5min":0.55600000000000005,"15min":0.80700000000000005},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.432,"5min":0.38100000000000001,"15min":0.36399999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.21199999999999999},"max":{"1min":0.54600000000000004,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.35299999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.36099999999999999,"15min":0.35399999999999998},"min":{"1min":0.317,"5min":0.215,"15min":0.20399999999999999},"max":{"1min":0.53100000000000003,"5min":0.53700000000000003,"15min":0.68300000000000005},"last":0.379},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.40899999999999997,"15min":0.39300000000000002},"min":{"1min":0.36799999999999999,"5min":0.25800000000000001,"15min":0.251},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.41199999999999998}]},{"osd":3,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.39800000000000002,"15min":0.376},"min":{"1min":0.32600000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.442,"5min":0.375,"15min":0.46200000000000002},"min":{"1min":0.28599999999999998,"5min":0.23400000000000001,"15min":0.22700000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":5.8529999999999998},"last":0.33200000000000002}]},{"osd":4,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.59999999999999998,"15min":0.58499999999999996},"min":{"1min":0.45300000000000001,"5min":0.36199999999999999,"15min":0.32200000000000001},"max":{"1min":0.79800000000000004,"5min":1.01,"15min":1.01},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61799999999999999,"15min":0.59799999999999998},"min":{"1min":0.47299999999999998,"5min":0.40000000000000002,"15min":0.37},"max":{"1min":0.79300000000000004,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.64500000000000002}]},{"osd":5,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.59799999999999998,"15min":0.67500000000000004},"min":{"1min":0.40899999999999997,"5min":0.36499999999999999,"15min":0.34599999999999997},"max":{"1min":0.85599999999999998,"5min":0.92500000000000004,"15min":5.9459999999999997},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.621,"15min":0.61399999999999999},"min":{"1min":0.55700000000000005,"5min":0.38100000000000001,"15min":0.34300000000000003},"max":{"1min":0.82599999999999996,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.61699999999999999}]},{"osd":6,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.57799999999999996,"15min":0.65000000000000002},"min":{"1min":0.44600000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":5.9160000000000004},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.51700000000000002,"5min":0.379,"15min":0.35699999999999998},"max":{"1min":0.86699999999999999,"5min":0.88600000000000001,"15min":0.91400000000000003},"last":0.57399999999999995}]},{"osd":7,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.51400000000000001,"5min":0.39000000000000001,"15min":0.379},"max":{"1min":0.76800000000000002,"5min":0.91000000000000003,"15min":0.97099999999999997},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.59099999999999997,"15min":0.67600000000000005},"min":{"1min":0.48399999999999999,"5min":0.41399999999999998,"15min":0.38100000000000001},"max":{"1min":0.83399999999999996,"5min":0.86099999999999999,"15min":5.9729999999999999},"last":0.59199999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574998,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304480,"kb_used_data":3096,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067360,"statfs":{"total":96636764160,"available":96324976640,"internally_reserved":0,"allocated":3170304,"data_stored":1746027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.34000000000000002,"15min":0.32200000000000001},"min":{"1min":0.252,"5min":0.216,"15min":0.21299999999999999},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.35799999999999998,"15min":0.34699999999999998},"min":{"1min":0.24399999999999999,"5min":0.222,"15min":0.214},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.94499999999999995},"last":0.35199999999999998}]},{"osd":2,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.38300000000000001,"15min":0.36799999999999999},"min":{"1min":0.27100000000000002,"5min":0.22,"15min":0.20200000000000001},"max":{"1min":0.53000000000000003,"5min":0.54000000000000004,"15min":0.75600000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.41399999999999998,"15min":0.39500000000000002},"min":{"1min":0.34599999999999997,"5min":0.29099999999999998,"15min":0.28000000000000003},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.91700000000000004},"last":0.40100000000000002}]},{"osd":3,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.36499999999999999,"15min":0.372},"min":{"1min":0.318,"5min":0.23699999999999999,"15min":0.23300000000000001},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":1.554},"last":0.432},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.40200000000000002,"15min":0.39500000000000002},"min":{"1min":0.32400000000000001,"5min":0.27700000000000002,"15min":0.25800000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.79000000000000004},"last":0.33500000000000002}]},{"osd":4,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58699999999999997,"15min":0.56299999999999994},"min":{"1min":0.46300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85699999999999998,"5min":0.92600000000000005,"15min":1.212},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58699999999999997,"15min":0.65700000000000003},"min":{"1min":0.43099999999999999,"5min":0.35799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85599999999999998,"5min":0.90800000000000003,"15min":5.484},"last":0.58199999999999996}]},{"osd":5,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58499999999999996,"15min":0.65900000000000003},"min":{"1min":0.437,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.83099999999999996,"5min":0.97399999999999998,"15min":5.5449999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.47499999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.875,"5min":0.998,"15min":1.113},"last":0.56499999999999995}]},{"osd":6,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.42399999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86899999999999999,"5min":0.871,"15min":1.0529999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.627,"15min":0.61399999999999999},"min":{"1min":0.505,"5min":0.42899999999999999,"15min":0.41599999999999998},"max":{"1min":0.88400000000000001,"5min":0.96099999999999997,"15min":1.169},"last":0.55100000000000005}]},{"osd":7,"last update":"Mon Oct 9 06:41:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60399999999999998,"15min":0.59399999999999997},"min":{"1min":0.47299999999999998,"5min":0.40200000000000002,"15min":0.371},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":1.2390000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59899999999999998,"15min":0.67800000000000005},"min":{"1min":0.48399999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83199999999999996,"5min":0.94299999999999995,"15min":5.5800000000000001},"last":0.54200000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303900,"kb_used_data":2516,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067940,"statfs":{"total":96636764160,"available":96325570560,"internally_reserved":0,"allocated":2576384,"data_stored":1155659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33400000000000002,"15min":0.34000000000000002},"min":{"1min":0.252,"5min":0.20000000000000001,"15min":0.19900000000000001},"max":{"1min":0.51800000000000002,"5min":0.66000000000000003,"15min":1.4199999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.34499999999999997,"15min":0.46100000000000002},"min":{"1min":0.28000000000000003,"5min":0.221,"15min":0.19900000000000001},"max":{"1min":0.53100000000000003,"5min":0.55600000000000005,"15min":7.2699999999999996},"last":0.33300000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.34200000000000003,"15min":0.34100000000000003},"min":{"1min":0.26200000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.497,"5min":0.56399999999999995,"15min":0.65400000000000003},"last":0.307},{"interface":"front","average":{"1min":0.434,"5min":0.39600000000000002,"15min":0.51400000000000001},"min":{"1min":0.33500000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.56799999999999995,"5min":0.60399999999999998,"15min":7.327},"last":0.375}]},{"osd":3,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.41099999999999998,"15min":0.39400000000000002},"min":{"1min":0.39600000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.56899999999999995,"5min":0.64600000000000002,"15min":0.68600000000000005},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.39000000000000001,"15min":0.38700000000000001},"min":{"1min":0.32600000000000001,"5min":0.252,"15min":0.23999999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.72699999999999998},"last":0.36499999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56299999999999994,"15min":0.54800000000000004},"min":{"1min":0.45800000000000002,"5min":0.33600000000000002,"15min":0.317},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.434},{"interface":"front","average":{"1min":0.625,"5min":0.58199999999999996,"15min":0.68999999999999995},"min":{"1min":0.46899999999999997,"5min":0.35999999999999999,"15min":0.33600000000000002},"max":{"1min":0.82599999999999996,"5min":0.91100000000000003,"15min":7.2389999999999999},"last":0.55200000000000005}]},{"osd":5,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.59499999999999997,"15min":0.77300000000000002},"min":{"1min":0.495,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.872,"5min":0.89800000000000002,"15min":10.890000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.60899999999999999,"15min":0.60299999999999998},"min":{"1min":0.45600000000000002,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":0.80000000000000004,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.53200000000000003}]},{"osd":6,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58099999999999996,"15min":0.69599999999999995},"min":{"1min":0.50800000000000001,"5min":0.35999999999999999,"15min":0.33500000000000002},"max":{"1min":0.77600000000000002,"5min":0.82499999999999996,"15min":7.2999999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60499999999999998,"15min":0.58299999999999996},"min":{"1min":0.44700000000000001,"5min":0.40300000000000002,"15min":0.377},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.502}]},{"osd":7,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.61099999999999999,"15min":0.72599999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.37},"max":{"1min":0.80100000000000005,"5min":0.91300000000000003,"15min":7.1879999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63,"15min":0.624},"min":{"1min":0.53300000000000003,"5min":0.437,"15min":0.38},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.56399999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303900,"kb_used_data":2516,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067940,"statfs":{"total":96636764160,"available":96325570560,"internally_reserved":0,"allocated":2576384,"data_stored":1155659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.34200000000000003,"15min":0.33800000000000002},"min":{"1min":0.23799999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.55400000000000005,"5min":0.61699999999999999,"15min":5.8700000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.36299999999999999,"15min":0.36799999999999999},"min":{"1min":0.255,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.55400000000000005,"5min":0.58499999999999996,"15min":5.6769999999999996},"last":0.52600000000000002}]},{"osd":1,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.36599999999999999,"15min":0.36399999999999999},"min":{"1min":0.28599999999999998,"5min":0.21299999999999999,"15min":0.21199999999999999},"max":{"1min":0.46700000000000003,"5min":0.61199999999999999,"15min":4.0469999999999997},"last":0.5},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.40200000000000002,"15min":0.39500000000000002},"min":{"1min":0.32900000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.60199999999999998,"5min":0.68600000000000005,"15min":4.1509999999999998},"last":0.48699999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.371,"15min":0.373},"min":{"1min":0.27300000000000002,"5min":0.23200000000000001,"15min":0.222},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":3.1139999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.39500000000000002,"15min":0.38900000000000001},"min":{"1min":0.28799999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.54700000000000004,"5min":0.66900000000000004,"15min":3.3180000000000001},"last":0.42899999999999999}]},{"osd":4,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56999999999999995,"15min":0.57299999999999995},"min":{"1min":0.42799999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.75700000000000001,"5min":0.86699999999999999,"15min":3.0899999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.432,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79300000000000004,"5min":0.84499999999999997,"15min":3.0710000000000002},"last":0.75600000000000001}]},{"osd":5,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.58099999999999996,"15min":0.59099999999999997},"min":{"1min":0.45300000000000001,"5min":0.33600000000000002,"15min":0.32700000000000001},"max":{"1min":0.86399999999999999,"5min":0.877,"15min":6.2300000000000004},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.58699999999999997,"15min":0.59699999999999998},"min":{"1min":0.498,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78800000000000003,"5min":0.83599999999999997,"15min":3.0219999999999998},"last":0.82099999999999995}]},{"osd":6,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.61899999999999999,"15min":0.60499999999999998},"min":{"1min":0.47899999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.147,"5min":1.147,"15min":2.6850000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.65000000000000002,"15min":0.65700000000000003},"min":{"1min":0.5,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":6.5970000000000004},"last":0.77100000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:41:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.61199999999999999,"15min":0.61699999999999999},"min":{"1min":0.46600000000000003,"5min":0.41499999999999998,"15min":0.38200000000000001},"max":{"1min":0.81799999999999995,"5min":0.86099999999999999,"15min":4.2629999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60799999999999998,"15min":0.61899999999999999},"min":{"1min":0.47899999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.89200000000000002,"5min":0.89900000000000002,"15min":7.3650000000000002},"last":0.66700000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303900,"kb_used_data":2516,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067940,"statfs":{"total":96636764160,"available":96325570560,"internally_reserved":0,"allocated":2576384,"data_stored":1155659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54800000000000004,"15min":0.54500000000000004},"min":{"1min":0.38900000000000001,"5min":0.307,"15min":0.27000000000000002},"max":{"1min":0.71899999999999997,"5min":0.88900000000000001,"15min":1.083},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.55200000000000005,"15min":0.53000000000000003},"min":{"1min":0.50800000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":1.054},"last":0.625}]},{"osd":1,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56599999999999995,"15min":0.55100000000000005},"min":{"1min":0.52400000000000002,"5min":0.28399999999999997,"15min":0.27100000000000002},"max":{"1min":0.77300000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53100000000000003,"15min":0.50800000000000001},"min":{"1min":0.38500000000000001,"5min":0.33800000000000002,"15min":0.31},"max":{"1min":0.75,"5min":0.76300000000000001,"15min":0.79100000000000004},"last":0.65400000000000003}]},{"osd":2,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.54700000000000004,"15min":0.54000000000000004},"min":{"1min":0.443,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76500000000000001,"5min":0.84899999999999998,"15min":1.0249999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.499,"5min":0.38100000000000001,"15min":0.36299999999999999},"max":{"1min":0.73399999999999999,"5min":0.90500000000000003,"15min":0.93799999999999994},"last":0.59099999999999997}]},{"osd":3,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55300000000000005,"15min":0.54800000000000004},"min":{"1min":0.40699999999999997,"5min":0.33300000000000002,"15min":0.29199999999999998},"max":{"1min":0.747,"5min":0.86899999999999999,"15min":0.98499999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.47799999999999998,"5min":0.38700000000000001,"15min":0.34100000000000003},"max":{"1min":0.68200000000000005,"5min":0.92900000000000005,"15min":1.1319999999999999},"last":0.66700000000000004}]},{"osd":5,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.50600000000000001,"15min":0.53300000000000003},"min":{"1min":0.35499999999999998,"5min":0.307,"15min":0.27300000000000002},"max":{"1min":0.55900000000000005,"5min":0.83299999999999996,"15min":0.998},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57699999999999996,"15min":0.59399999999999997},"min":{"1min":0.40000000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71999999999999997,"5min":0.95599999999999996,"15min":0.97999999999999998},"last":0.45400000000000001}]},{"osd":6,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.54600000000000004,"15min":0.55600000000000005},"min":{"1min":0.36899999999999999,"5min":0.35499999999999998,"15min":0.29999999999999999},"max":{"1min":0.67500000000000004,"5min":0.86299999999999999,"15min":1.0189999999999999},"last":0.438},{"interface":"front","average":{"1min":0.501,"5min":0.56999999999999995,"15min":0.58399999999999996},"min":{"1min":0.38900000000000001,"5min":0.36399999999999999,"15min":0.32700000000000001},"max":{"1min":0.72699999999999998,"5min":0.85599999999999998,"15min":1.01},"last":0.47199999999999998}]},{"osd":7,"last update":"Mon Oct 9 06:41:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55700000000000005,"15min":0.56200000000000006},"min":{"1min":0.42399999999999999,"5min":0.35299999999999998,"15min":0.28299999999999997},"max":{"1min":0.65400000000000003,"5min":0.84399999999999997,"15min":0.90000000000000002},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.60599999999999998,"15min":0.625},"min":{"1min":0.374,"5min":0.34999999999999998,"15min":0.32200000000000001},"max":{"1min":0.71299999999999997,"5min":0.93799999999999994,"15min":1.175},"last":0.56599999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303900,"kb_used_data":2516,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067940,"statfs":{"total":96636764160,"available":96325570560,"internally_reserved":0,"allocated":2576384,"data_stored":1155659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54200000000000004,"15min":0.51700000000000002},"min":{"1min":0.36399999999999999,"5min":0.29499999999999998,"15min":0.28599999999999998},"max":{"1min":0.76400000000000001,"5min":0.79700000000000004,"15min":0.86599999999999999},"last":0.438},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55400000000000005,"15min":0.52800000000000002},"min":{"1min":0.45000000000000001,"5min":0.36399999999999999,"15min":0.28799999999999998},"max":{"1min":0.83599999999999997,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.47199999999999998}]},{"osd":1,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52700000000000002,"15min":0.51500000000000001},"min":{"1min":0.38800000000000001,"5min":0.34100000000000003,"15min":0.311},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.54400000000000004,"15min":0.53400000000000003},"min":{"1min":0.41499999999999998,"5min":0.36899999999999999,"15min":0.30199999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.81599999999999995},"last":0.48599999999999999}]},{"osd":2,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58199999999999996,"15min":0.55800000000000005},"min":{"1min":0.39300000000000002,"5min":0.34699999999999998,"15min":0.28199999999999997},"max":{"1min":0.79800000000000004,"5min":1.139,"15min":1.139},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.52600000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78100000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.44900000000000001}]},{"osd":3,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57099999999999995,"15min":0.55200000000000005},"min":{"1min":0.43099999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.75,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.45900000000000002,"5min":0.40699999999999997,"15min":0.312},"max":{"1min":0.80200000000000005,"5min":0.80600000000000005,"15min":0.81100000000000005},"last":0.53500000000000003}]},{"osd":4,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.33600000000000002,"5min":0.28399999999999997,"15min":0.26700000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53600000000000003,"15min":0.54200000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.27700000000000002},"max":{"1min":0.751,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.41199999999999998}]},{"osd":6,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.35599999999999998,"5min":0.33700000000000002,"15min":0.32600000000000001},"max":{"1min":0.73499999999999999,"5min":1.752,"15min":1.752},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55600000000000005,"15min":0.55900000000000005},"min":{"1min":0.39100000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.52500000000000002}]},{"osd":7,"last update":"Mon Oct 9 06:41:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56399999999999995,"15min":0.57599999999999996},"min":{"1min":0.375,"5min":0.311,"15min":0.311},"max":{"1min":0.79900000000000004,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58599999999999997,"15min":0.60299999999999998},"min":{"1min":0.40799999999999997,"5min":0.372,"15min":0.314},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.90600000000000003},"last":0.63400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-09T06:42:39.885 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-09T06:42:40.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.999+0000 7f3a94ddd700 1 Processor -- start 2023-10-09T06:42:40.000 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:39.999+0000 7f3a94ddd700 1 -- start start 2023-10-09T06:42:40.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.000+0000 7f3a94ddd700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3a90159890 0x7f3a9015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:40.002 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.000+0000 7f3a94ddd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3a90159c60 0x7f3a9015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:40.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.000+0000 7f3a94ddd700 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a9015a160 con 0x7f3a9015db10 2023-10-09T06:42:40.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.000+0000 7f3a94ddd700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a9015df50 con 0x7f3a90159c60 2023-10-09T06:42:40.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.000+0000 7f3a94ddd700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a900d13a0 con 0x7f3a90159890 2023-10-09T06:42:40.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.000+0000 7f3a8dd9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3a90159c60 0x7f3a9015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:40.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.000+0000 7f3a8dd9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3a90159c60 0x7f3a9015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:52986/0 (socket says 172.21.15.163:52986) 2023-10-09T06:42:40.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.000+0000 7f3a8dd9b700 1 -- 172.21.15.163:0/222797211 learned_addr learned my addr 172.21.15.163:0/222797211 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:40.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.001+0000 7f3a8e59c700 1 --2- 172.21.15.163:0/222797211 >> v2:172.21.15.181:3300/0 conn(0x7f3a90159890 0x7f3a9015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:40.003 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.001+0000 7f3a8dd9b700 1 -- 172.21.15.163:0/222797211 >> v2:172.21.15.181:3300/0 conn(0x7f3a90159890 msgr2=0x7f3a9015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:40.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.001+0000 7f3a8dd9b700 1 --2- 172.21.15.163:0/222797211 >> v2:172.21.15.181:3300/0 conn(0x7f3a90159890 0x7f3a9015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:40.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.001+0000 7f3a8dd9b700 1 -- 172.21.15.163:0/222797211 >> v1:172.21.15.181:6789/0 conn(0x7f3a9015db10 legacy=0x7f3a90069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:40.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.001+0000 7f3a8dd9b700 1 -- 172.21.15.163:0/222797211 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9015e140 con 0x7f3a90159c60 2023-10-09T06:42:40.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.001+0000 7f3a8dd9b700 1 --2- 172.21.15.163:0/222797211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a90159c60 0x7f3a9015d5d0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f3a900d1090 tx=0x7f3a78009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67329595b72c6fd0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:40.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.001+0000 7f3a8d59a700 1 -- 172.21.15.163:0/222797211 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7800b820 con 0x7f3a90159c60 2023-10-09T06:42:40.004 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.001+0000 7f3a8d59a700 1 -- 172.21.15.163:0/222797211 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7800b980 con 0x7f3a90159c60 2023-10-09T06:42:40.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.001+0000 7f3a8d59a700 1 -- 172.21.15.163:0/222797211 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7800bd20 con 0x7f3a90159c60 2023-10-09T06:42:40.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.001+0000 7f3a94ddd700 1 -- 172.21.15.163:0/222797211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a90159c60 msgr2=0x7f3a9015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:40.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.001+0000 7f3a94ddd700 1 --2- 172.21.15.163:0/222797211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a90159c60 0x7f3a9015d5d0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f3a900d1090 tx=0x7f3a78009250 comp rx=0 tx=0).stop 2023-10-09T06:42:40.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.002+0000 7f3a94ddd700 1 -- 172.21.15.163:0/222797211 shutdown_connections 2023-10-09T06:42:40.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.002+0000 7f3a94ddd700 1 --2- 172.21.15.163:0/222797211 >> v2:172.21.15.181:3300/0 conn(0x7f3a90159890 0x7f3a9015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:40.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.002+0000 7f3a94ddd700 1 --2- 172.21.15.163:0/222797211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a90159c60 0x7f3a9015d5d0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:40.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.002+0000 7f3a94ddd700 1 -- 172.21.15.163:0/222797211 >> 172.21.15.163:0/222797211 conn(0x7f3a90068e20 msgr2=0x7f3a900d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:40.005 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.002+0000 7f3a94ddd700 1 -- 172.21.15.163:0/222797211 shutdown_connections 2023-10-09T06:42:40.006 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.002+0000 7f3a94ddd700 1 -- 172.21.15.163:0/222797211 wait complete. 2023-10-09T06:42:40.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.003+0000 7f3a94ddd700 1 Processor -- start 2023-10-09T06:42:40.008 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.003+0000 7f3a94ddd700 1 -- start start 2023-10-09T06:42:40.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.003+0000 7f3a94ddd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a90159890 0x7f3a900b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:40.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.003+0000 7f3a94ddd700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a90159c60 0x7f3a90148ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:40.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.003+0000 7f3a94ddd700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a9015db10 0x7f3a900cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:40.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.003+0000 7f3a94ddd700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a90151820 con 0x7f3a90159890 2023-10-09T06:42:40.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.003+0000 7f3a94ddd700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a90151b00 con 0x7f3a9015db10 2023-10-09T06:42:40.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.003+0000 7f3a94ddd700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a90151de0 con 0x7f3a90159c60 2023-10-09T06:42:40.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.003+0000 7f3a8e59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a90159890 0x7f3a900b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:40.009 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.004+0000 7f3a8e59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a90159890 0x7f3a900b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:52996/0 (socket says 172.21.15.163:52996) 2023-10-09T06:42:40.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.004+0000 7f3a8e59c700 1 -- 172.21.15.163:0/1323059525 learned_addr learned my addr 172.21.15.163:0/1323059525 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:40.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.004+0000 7f3a8dd9b700 1 --2- 172.21.15.163:0/1323059525 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a90159c60 0x7f3a90148ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:40.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.004+0000 7f3a8ed9d700 1 --2- 172.21.15.163:0/1323059525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a9015db10 0x7f3a900cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:40.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.004+0000 7f3a8e59c700 1 -- 172.21.15.163:0/1323059525 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a90159c60 msgr2=0x7f3a90148ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:40.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.004+0000 7f3a8e59c700 1 --2- 172.21.15.163:0/1323059525 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a90159c60 0x7f3a90148ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:40.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.004+0000 7f3a8e59c700 1 -- 172.21.15.163:0/1323059525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a9015db10 msgr2=0x7f3a900cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:40.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.004+0000 7f3a8e59c700 1 --2- 172.21.15.163:0/1323059525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a9015db10 0x7f3a900cf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:40.010 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.004+0000 7f3a8e59c700 1 -- 172.21.15.163:0/1323059525 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a780094f0 con 0x7f3a90159890 2023-10-09T06:42:40.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.004+0000 7f3a8e59c700 1 --2- 172.21.15.163:0/1323059525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a90159890 0x7f3a900b3590 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000e190 tx=0x7f3a80009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:40.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.004+0000 7f3a867fc700 1 -- 172.21.15.163:0/1323059525 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8000ecd0 con 0x7f3a90159890 2023-10-09T06:42:40.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.004+0000 7f3a867fc700 1 -- 172.21.15.163:0/1323059525 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8000ee30 con 0x7f3a90159890 2023-10-09T06:42:40.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.004+0000 7f3a94ddd700 1 -- 172.21.15.163:0/1323059525 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a901520c0 con 0x7f3a90159890 2023-10-09T06:42:40.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.004+0000 7f3a867fc700 1 -- 172.21.15.163:0/1323059525 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a80014670 con 0x7f3a90159890 2023-10-09T06:42:40.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.004+0000 7f3a94ddd700 1 -- 172.21.15.163:0/1323059525 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a90149400 con 0x7f3a90159890 2023-10-09T06:42:40.011 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.005+0000 7f3a94ddd700 1 -- 172.21.15.163:0/1323059525 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a9004c3c0 con 0x7f3a90159890 2023-10-09T06:42:40.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.005+0000 7f3a867fc700 1 -- 172.21.15.163:0/1323059525 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3a80014a20 con 0x7f3a90159890 2023-10-09T06:42:40.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.006+0000 7f3a867fc700 1 --2- 172.21.15.163:0/1323059525 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3a70067320 0x7f3a700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:40.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.006+0000 7f3a867fc700 1 -- 172.21.15.163:0/1323059525 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f3a80085030 con 0x7f3a90159890 2023-10-09T06:42:40.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.006+0000 7f3a8dd9b700 1 --2- 172.21.15.163:0/1323059525 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3a70067320 0x7f3a700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:40.012 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.007+0000 7f3a8dd9b700 1 --2- 172.21.15.163:0/1323059525 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3a70067320 0x7f3a700697d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f3a900d1090 tx=0x7f3a78016000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:40.013 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.010+0000 7f3a867fc700 1 -- 172.21.15.163:0/1323059525 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3a80051fc0 con 0x7f3a90159890 2023-10-09T06:42:40.191 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.190+0000 7f3a94ddd700 1 -- 172.21.15.163:0/1323059525 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f3a9006a5b0 con 0x7f3a90159890 2023-10-09T06:42:40.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.195+0000 7f3a867fc700 1 -- 172.21.15.163:0/1323059525 <== mon.0 v2:172.21.15.163:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f3a80051fc0 con 0x7f3a90159890 2023-10-09T06:42:40.196 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.195+0000 7f3a867fc700 1 -- 172.21.15.163:0/1323059525 <== mon.0 v2:172.21.15.163:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v34)=0 v34) v1 ==== 123+0+0 (secure 0 0 0) 0x7f3a80009a60 con 0x7f3a90159890 2023-10-09T06:42:40.199 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.198+0000 7f3a94ddd700 1 -- 172.21.15.163:0/1323059525 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3a70067320 msgr2=0x7f3a700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:40.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.198+0000 7f3a94ddd700 1 --2- 172.21.15.163:0/1323059525 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3a70067320 0x7f3a700697d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f3a900d1090 tx=0x7f3a78016000 comp rx=0 tx=0).stop 2023-10-09T06:42:40.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.198+0000 7f3a94ddd700 1 -- 172.21.15.163:0/1323059525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a90159890 msgr2=0x7f3a900b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:40.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.198+0000 7f3a94ddd700 1 --2- 172.21.15.163:0/1323059525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a90159890 0x7f3a900b3590 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000e190 tx=0x7f3a80009920 comp rx=0 tx=0).stop 2023-10-09T06:42:40.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.198+0000 7f3a94ddd700 1 -- 172.21.15.163:0/1323059525 shutdown_connections 2023-10-09T06:42:40.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.198+0000 7f3a94ddd700 1 --2- 172.21.15.163:0/1323059525 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f3a70067320 0x7f3a700697d0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:40.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.198+0000 7f3a94ddd700 1 --2- 172.21.15.163:0/1323059525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a90159890 0x7f3a900b3590 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:40.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.198+0000 7f3a94ddd700 1 --2- 172.21.15.163:0/1323059525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a9015db10 0x7f3a900cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:40.200 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.198+0000 7f3a94ddd700 1 --2- 172.21.15.163:0/1323059525 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a90159c60 0x7f3a90148ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:40.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.198+0000 7f3a94ddd700 1 -- 172.21.15.163:0/1323059525 >> 172.21.15.163:0/1323059525 conn(0x7f3a90068e20 msgr2=0x7f3a90153f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:40.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.198+0000 7f3a94ddd700 1 -- 172.21.15.163:0/1323059525 shutdown_connections 2023-10-09T06:42:40.201 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.198+0000 7f3a94ddd700 1 -- 172.21.15.163:0/1323059525 wait complete. 2023-10-09T06:42:40.211 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-09T06:42:40.211 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-09T06:42:40.326 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.325+0000 7f77ee536700 1 Processor -- start 2023-10-09T06:42:40.327 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.326+0000 7f77ee536700 1 -- start start 2023-10-09T06:42:40.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.326+0000 7f77ee536700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f77e806bcd0 0x7f77e80caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:40.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.326+0000 7f77ee536700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77e806c0a0 0x7f77e815a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:40.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.326+0000 7f77ee536700 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f77e815df50 0x7f77e815e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:40.329 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.326+0000 7f77ee536700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77e815d7e0 con 0x7f77e806bcd0 2023-10-09T06:42:40.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.326+0000 7f77ee536700 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77e815dac0 con 0x7f77e815df50 2023-10-09T06:42:40.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.326+0000 7f77ee536700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e815aa30 con 0x7f77e806c0a0 2023-10-09T06:42:40.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.326+0000 7f77e7fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f77e806bcd0 0x7f77e80caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:40.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.326+0000 7f77e7fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f77e806bcd0 0x7f77e80caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53010/0 (socket says 172.21.15.163:53010) 2023-10-09T06:42:40.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.327+0000 7f77e7fff700 1 -- 172.21.15.163:0/388143603 learned_addr learned my addr 172.21.15.163:0/388143603 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:40.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.327+0000 7f77e7fff700 1 -- 172.21.15.163:0/388143603 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77e806c0a0 msgr2=0x7f77e815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:40.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.327+0000 7f77e7fff700 1 --2- 172.21.15.163:0/388143603 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77e806c0a0 0x7f77e815a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:40.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.327+0000 7f77e7fff700 1 -- 172.21.15.163:0/388143603 >> v2:172.21.15.181:3300/0 conn(0x7f77e815df50 msgr2=0x7f77e815e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-09T06:42:40.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.327+0000 7f77ecad3700 1 --2- 172.21.15.163:0/388143603 >> v2:172.21.15.181:3300/0 conn(0x7f77e815df50 0x7f77e815e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:40.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.327+0000 7f77e7fff700 1 --2- 172.21.15.163:0/388143603 >> v2:172.21.15.181:3300/0 conn(0x7f77e815df50 0x7f77e815e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:40.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.327+0000 7f77e7fff700 1 -- 172.21.15.163:0/388143603 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e815ad10 con 0x7f77e806bcd0 2023-10-09T06:42:40.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.327+0000 7f77ecad3700 1 --2- 172.21.15.163:0/388143603 >> v2:172.21.15.181:3300/0 conn(0x7f77e815df50 0x7f77e815e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-09T06:42:40.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.327+0000 7f77e7fff700 1 --2- 172.21.15.163:0/388143603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77e806bcd0 0x7f77e80caa90 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f77e80b4600 tx=0x7f77d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2289672d4b8480bb server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:40.331 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.327+0000 7f77e6ffd700 1 -- 172.21.15.163:0/388143603 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d800b860 con 0x7f77e806bcd0 2023-10-09T06:42:40.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.327+0000 7f77e6ffd700 1 -- 172.21.15.163:0/388143603 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f77d800b9c0 con 0x7f77e806bcd0 2023-10-09T06:42:40.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.327+0000 7f77e6ffd700 1 -- 172.21.15.163:0/388143603 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d800bda0 con 0x7f77e806bcd0 2023-10-09T06:42:40.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.328+0000 7f77ee536700 1 -- 172.21.15.163:0/388143603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77e806bcd0 msgr2=0x7f77e80caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:40.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.328+0000 7f77ee536700 1 --2- 172.21.15.163:0/388143603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77e806bcd0 0x7f77e80caa90 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f77e80b4600 tx=0x7f77d8009250 comp rx=0 tx=0).stop 2023-10-09T06:42:40.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.328+0000 7f77ee536700 1 -- 172.21.15.163:0/388143603 shutdown_connections 2023-10-09T06:42:40.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.328+0000 7f77ee536700 1 --2- 172.21.15.163:0/388143603 >> v2:172.21.15.181:3300/0 conn(0x7f77e815df50 0x7f77e815e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:40.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.328+0000 7f77ee536700 1 --2- 172.21.15.163:0/388143603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77e806bcd0 0x7f77e80caa90 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:40.332 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.328+0000 7f77ee536700 1 --2- 172.21.15.163:0/388143603 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77e806c0a0 0x7f77e815a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:40.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.328+0000 7f77ee536700 1 -- 172.21.15.163:0/388143603 >> 172.21.15.163:0/388143603 conn(0x7f77e80ce600 msgr2=0x7f77e80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:40.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.328+0000 7f77ee536700 1 -- 172.21.15.163:0/388143603 shutdown_connections 2023-10-09T06:42:40.333 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.328+0000 7f77ee536700 1 -- 172.21.15.163:0/388143603 wait complete. 2023-10-09T06:42:40.335 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.329+0000 7f77ee536700 1 Processor -- start 2023-10-09T06:42:40.336 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.329+0000 7f77ee536700 1 -- start start 2023-10-09T06:42:40.336 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.330+0000 7f77ee536700 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77e806bcd0 0x7f77e80c9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:40.336 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.330+0000 7f77ee536700 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77e806c0a0 0x7f77e80c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:40.336 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.330+0000 7f77ee536700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77e815df50 0x7f77e80c3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:40.336 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.330+0000 7f77ee536700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e80c6360 con 0x7f77e815df50 2023-10-09T06:42:40.336 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.330+0000 7f77ee536700 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e80c5750 con 0x7f77e806c0a0 2023-10-09T06:42:40.337 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.330+0000 7f77ee536700 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e80c5a30 con 0x7f77e806bcd0 2023-10-09T06:42:40.337 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.330+0000 7f77ecad3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77e815df50 0x7f77e80c3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:40.337 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.330+0000 7f77ecad3700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77e815df50 0x7f77e80c3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:53020/0 (socket says 172.21.15.163:53020) 2023-10-09T06:42:40.337 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.330+0000 7f77ecad3700 1 -- 172.21.15.163:0/960248920 learned_addr learned my addr 172.21.15.163:0/960248920 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-09T06:42:40.337 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.330+0000 7f77e7fff700 1 --2- 172.21.15.163:0/960248920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77e806bcd0 0x7f77e80c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:40.337 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.330+0000 7f77e77fe700 1 --2- 172.21.15.163:0/960248920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77e806c0a0 0x7f77e80c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:40.337 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.330+0000 7f77ecad3700 1 -- 172.21.15.163:0/960248920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77e806bcd0 msgr2=0x7f77e80c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:40.337 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.330+0000 7f77ecad3700 1 --2- 172.21.15.163:0/960248920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77e806bcd0 0x7f77e80c9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:40.338 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.330+0000 7f77ecad3700 1 -- 172.21.15.163:0/960248920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77e806c0a0 msgr2=0x7f77e80c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:40.338 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.330+0000 7f77ecad3700 1 --2- 172.21.15.163:0/960248920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77e806c0a0 0x7f77e80c39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:40.338 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.330+0000 7f77ecad3700 1 -- 172.21.15.163:0/960248920 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d80094f0 con 0x7f77e815df50 2023-10-09T06:42:40.338 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.330+0000 7f77ecad3700 1 --2- 172.21.15.163:0/960248920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77e815df50 0x7f77e80c3f10 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f77e000cef0 tx=0x7f77e0007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:40.338 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.330+0000 7f77cffff700 1 -- 172.21.15.163:0/960248920 <== mon.0 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e000ad20 con 0x7f77e815df50 2023-10-09T06:42:40.338 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.330+0000 7f77cffff700 1 -- 172.21.15.163:0/960248920 <== mon.0 v2:172.21.15.163:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f77e0016420 con 0x7f77e815df50 2023-10-09T06:42:40.338 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.330+0000 7f77cffff700 1 -- 172.21.15.163:0/960248920 <== mon.0 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e000aea0 con 0x7f77e815df50 2023-10-09T06:42:40.339 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.330+0000 7f77ee536700 1 -- 172.21.15.163:0/960248920 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e80c5d10 con 0x7f77e815df50 2023-10-09T06:42:40.339 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.330+0000 7f77ee536700 1 -- 172.21.15.163:0/960248920 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e80c9990 con 0x7f77e815df50 2023-10-09T06:42:40.339 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.331+0000 7f77ee536700 1 -- 172.21.15.163:0/960248920 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e804c3c0 con 0x7f77e815df50 2023-10-09T06:42:40.339 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.332+0000 7f77cffff700 1 -- 172.21.15.163:0/960248920 <== mon.0 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f77e0016a90 con 0x7f77e815df50 2023-10-09T06:42:40.339 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.332+0000 7f77cffff700 1 --2- 172.21.15.163:0/960248920 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f77d4067320 0x7f77d40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-09T06:42:40.339 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.332+0000 7f77cffff700 1 -- 172.21.15.163:0/960248920 <== mon.0 v2:172.21.15.163:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f77e0012070 con 0x7f77e815df50 2023-10-09T06:42:40.339 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.332+0000 7f77e7fff700 1 --2- 172.21.15.163:0/960248920 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f77d4067320 0x7f77d40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-09T06:42:40.339 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.333+0000 7f77e7fff700 1 --2- 172.21.15.163:0/960248920 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f77d4067320 0x7f77d40697d0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f77e80b4600 tx=0x7f77d8009250 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-09T06:42:40.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.336+0000 7f77cffff700 1 -- 172.21.15.163:0/960248920 <== mon.0 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f77e004e5a0 con 0x7f77e815df50 2023-10-09T06:42:40.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.561+0000 7f77ee536700 1 -- 172.21.15.163:0/960248920 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77e80c64a0 con 0x7f77e815df50 2023-10-09T06:42:40.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.562+0000 7f77cffff700 1 -- 172.21.15.163:0/960248920 <== mon.0 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 135 v135) v1 ==== 95+0+3546 (secure 0 0 0) 0x7f77e0059e60 con 0x7f77e815df50 2023-10-09T06:42:40.563 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-09T06:42:40.563 INFO:teuthology.orchestra.run.smithi163.stdout:{"epoch":135,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7046,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3374277959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3374277959},{"type":"v1","addr":"172.21.15.181:6837","nonce":3374277959}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":7049,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3834979450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3834979450},{"type":"v1","addr":"172.21.15.181:6836","nonce":3834979450}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":8991,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2681808199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2681808199},{"type":"v1","addr":"172.21.15.163:6834","nonce":2681808199}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":128},{"gid":9291,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1190443154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1190443154},{"type":"v1","addr":"172.21.15.163:6835","nonce":1190443154}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":135}],"filesystems":[]} 2023-10-09T06:42:40.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.564+0000 7f77ee536700 1 -- 172.21.15.163:0/960248920 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f77d4067320 msgr2=0x7f77d40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:40.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.564+0000 7f77ee536700 1 --2- 172.21.15.163:0/960248920 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f77d4067320 0x7f77d40697d0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f77e80b4600 tx=0x7f77d8009250 comp rx=0 tx=0).stop 2023-10-09T06:42:40.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.564+0000 7f77ee536700 1 -- 172.21.15.163:0/960248920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77e815df50 msgr2=0x7f77e80c3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-09T06:42:40.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.564+0000 7f77ee536700 1 --2- 172.21.15.163:0/960248920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77e815df50 0x7f77e80c3f10 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f77e000cef0 tx=0x7f77e0007860 comp rx=0 tx=0).stop 2023-10-09T06:42:40.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.564+0000 7f77ee536700 1 -- 172.21.15.163:0/960248920 shutdown_connections 2023-10-09T06:42:40.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.564+0000 7f77ee536700 1 --2- 172.21.15.163:0/960248920 >> [v2:172.21.15.181:6832/90545,v1:172.21.15.181:6833/90545] conn(0x7f77d4067320 0x7f77d40697d0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:40.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.564+0000 7f77ee536700 1 --2- 172.21.15.163:0/960248920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77e815df50 0x7f77e80c3f10 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:40.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.564+0000 7f77ee536700 1 --2- 172.21.15.163:0/960248920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77e806bcd0 0x7f77e80c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:40.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.564+0000 7f77ee536700 1 --2- 172.21.15.163:0/960248920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77e806c0a0 0x7f77e80c39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-09T06:42:40.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.564+0000 7f77ee536700 1 -- 172.21.15.163:0/960248920 >> 172.21.15.163:0/960248920 conn(0x7f77e80ce600 msgr2=0x7f77e8156730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-09T06:42:40.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.564+0000 7f77ee536700 1 -- 172.21.15.163:0/960248920 shutdown_connections 2023-10-09T06:42:40.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-09T06:42:40.564+0000 7f77ee536700 1 -- 172.21.15.163:0/960248920 wait complete. 2023-10-09T06:42:40.567 INFO:teuthology.orchestra.run.smithi163.stderr:dumped fsmap epoch 135 2023-10-09T06:42:40.577 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-09T06:42:40.577 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-09T06:42:40.578 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-09T06:42:40.578 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:42:40.586 INFO:tasks.ceph.mds.a:Stopped 2023-10-09T06:42:40.586 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-09T06:42:40.586 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:42:40.636 INFO:tasks.ceph.mds.c:Stopped 2023-10-09T06:42:40.636 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-09T06:42:40.636 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:42:40.648 INFO:tasks.ceph.mds.b:Stopped 2023-10-09T06:42:40.648 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-09T06:42:40.648 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:42:40.702 INFO:tasks.ceph.mds.d:Stopped 2023-10-09T06:42:40.702 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-09T06:42:40.702 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-09T06:42:40.702 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:42:40.735 INFO:tasks.ceph.osd.0:Stopped 2023-10-09T06:42:40.736 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-09T06:42:40.736 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:42:40.807 INFO:tasks.ceph.osd.1:Stopped 2023-10-09T06:42:40.807 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-09T06:42:40.807 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:42:40.875 INFO:tasks.ceph.osd.2:Stopped 2023-10-09T06:42:40.876 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-09T06:42:40.876 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:42:40.945 INFO:tasks.ceph.osd.3:Stopped 2023-10-09T06:42:40.945 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-09T06:42:40.945 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:42:40.986 INFO:tasks.ceph.osd.4:Stopped 2023-10-09T06:42:40.987 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-09T06:42:40.987 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:42:41.017 INFO:tasks.ceph.osd.5:Stopped 2023-10-09T06:42:41.018 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-09T06:42:41.018 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:42:41.094 INFO:tasks.ceph.osd.6:Stopped 2023-10-09T06:42:41.094 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-09T06:42:41.094 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:42:41.169 INFO:tasks.ceph.osd.7:Stopped 2023-10-09T06:42:41.169 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-09T06:42:41.169 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-09T06:42:41.170 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:42:41.192 INFO:tasks.ceph.mgr.y:Stopped 2023-10-09T06:42:41.193 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-09T06:42:41.193 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:42:41.237 INFO:tasks.ceph.mgr.x:Stopped 2023-10-09T06:42:41.237 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-09T06:42:41.237 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-09T06:42:41.237 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:42:41.248 INFO:tasks.ceph.mon.a:Stopped 2023-10-09T06:42:41.249 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-09T06:42:41.249 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:42:41.288 INFO:tasks.ceph.mon.b:Stopped 2023-10-09T06:42:41.289 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-09T06:42:41.289 INFO:teuthology.orchestra.run:waiting for 300 2023-10-09T06:42:41.340 INFO:tasks.ceph.mon.c:Stopped 2023-10-09T06:42:41.340 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-09T06:42:41.341 DEBUG:teuthology.orchestra.run.smithi163:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-10-09T06:42:41.370 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi163.front.sepia.ceph.com 2023-10-09T06:42:41.370 DEBUG:teuthology.orchestra.run.smithi163:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-09T06:42:41.568 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi163.front.sepia.ceph.com 2023-10-09T06:42:41.568 DEBUG:teuthology.orchestra.run.smithi163:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-09T06:42:41.655 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi163.front.sepia.ceph.com 2023-10-09T06:42:41.655 DEBUG:teuthology.orchestra.run.smithi163:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-09T06:42:41.718 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi163.front.sepia.ceph.com 2023-10-09T06:42:41.718 DEBUG:teuthology.orchestra.run.smithi163:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-09T06:42:41.778 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi181.front.sepia.ceph.com 2023-10-09T06:42:41.778 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-09T06:42:41.984 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi181.front.sepia.ceph.com 2023-10-09T06:42:41.985 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-09T06:42:42.059 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi181.front.sepia.ceph.com 2023-10-09T06:42:42.059 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-09T06:42:42.149 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi181.front.sepia.ceph.com 2023-10-09T06:42:42.150 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-09T06:42:42.224 INFO:tasks.ceph:Archiving mon data... 2023-10-09T06:42:42.225 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/vshankar-2023-10-09_05:36:44-fs-wip-yuri-testing-2023-10-06-0949-quincy-testing-default-smithi/7418377/data/mon.a.tgz 2023-10-09T06:42:42.226 DEBUG:teuthology.orchestra.run.smithi163:> mktemp 2023-10-09T06:42:42.241 INFO:teuthology.orchestra.run.smithi163.stdout:/tmp/tmp.x7B6mGOz0E 2023-10-09T06:42:42.241 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.x7B6mGOz0E 2023-10-09T06:42:42.416 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 0666 /tmp/tmp.x7B6mGOz0E 2023-10-09T06:42:42.498 DEBUG:teuthology.orchestra.remote:smithi163:/tmp/tmp.x7B6mGOz0E is 583KB 2023-10-09T06:42:42.600 DEBUG:teuthology.orchestra.run.smithi163:> rm -fr /tmp/tmp.x7B6mGOz0E 2023-10-09T06:42:42.614 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/vshankar-2023-10-09_05:36:44-fs-wip-yuri-testing-2023-10-06-0949-quincy-testing-default-smithi/7418377/data/mon.b.tgz 2023-10-09T06:42:42.615 DEBUG:teuthology.orchestra.run.smithi181:> mktemp 2023-10-09T06:42:42.631 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.ipZoL9ps8P 2023-10-09T06:42:42.631 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.ipZoL9ps8P 2023-10-09T06:42:42.813 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.ipZoL9ps8P 2023-10-09T06:42:42.895 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.ipZoL9ps8P is 593KB 2023-10-09T06:42:42.995 DEBUG:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.ipZoL9ps8P 2023-10-09T06:42:43.009 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/vshankar-2023-10-09_05:36:44-fs-wip-yuri-testing-2023-10-06-0949-quincy-testing-default-smithi/7418377/data/mon.c.tgz 2023-10-09T06:42:43.009 DEBUG:teuthology.orchestra.run.smithi181:> mktemp 2023-10-09T06:42:43.062 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.vPcIWOiYSi 2023-10-09T06:42:43.063 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.vPcIWOiYSi 2023-10-09T06:42:43.228 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.vPcIWOiYSi 2023-10-09T06:42:43.309 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.vPcIWOiYSi is 593KB 2023-10-09T06:42:43.409 DEBUG:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.vPcIWOiYSi 2023-10-09T06:42:43.423 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-09T06:42:43.423 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-09T06:42:43.426 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-09T06:42:43.487 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-09T06:42:43.487 DEBUG:teuthology.orchestra.run.smithi163:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-09T06:42:43.493 DEBUG:teuthology.orchestra.run.smithi181:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-09T06:42:43.517 INFO:teuthology.orchestra.run.smithi163.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-09T06:42:43.551 INFO:teuthology.orchestra.run.smithi181.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-09T06:42:43.557 INFO:tasks.ceph:Archiving crash dumps... 2023-10-09T06:42:43.559 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-10-09_05:36:44-fs-wip-yuri-testing-2023-10-06-0949-quincy-testing-default-smithi/7418377/remote/smithi163/crash 2023-10-09T06:42:43.560 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-09T06:42:43.589 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-10-09_05:36:44-fs-wip-yuri-testing-2023-10-06-0949-quincy-testing-default-smithi/7418377/remote/smithi181/crash 2023-10-09T06:42:43.590 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-09T06:42:43.620 INFO:tasks.ceph:Shutting down logrotate 2023-10-09T06:42:43.620 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:42:43.634 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-09T06:42:43.741 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-09T06:42:43.766 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-09T06:42:43.790 INFO:tasks.ceph:Compressing logs... 2023-10-09T06:42:43.790 DEBUG:teuthology.orchestra.run.smithi163:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-09T06:42:43.808 DEBUG:teuthology.orchestra.run.smithi181:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-09T06:42:43.881 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90904.log 2023-10-09T06:42:43.881 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.101454.log 2023-10-09T06:42:43.881 INFO:teuthology.orchestra.run.smithi181.stderr:gzip /var/log/ceph/ceph-client.admin.90904.log: -5 --verbose -- /var/log/ceph/ceph-client.1.102642.log 2023-10-09T06:42:43.881 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.101454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.102240.log 2023-10-09T06:42:43.882 INFO:teuthology.orchestra.run.smithi181.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.90904.log.gz 2023-10-09T06:42:43.882 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.101454.log.gz 2023-10-09T06:42:43.883 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.102642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.101067.log 2023-10-09T06:42:43.883 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.102642.log.gz 2023-10-09T06:42:43.883 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.102240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.100675.log 2023-10-09T06:42:43.883 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.102240.log.gz 2023-10-09T06:42:43.883 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.101067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.99000.log 2023-10-09T06:42:43.883 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.101067.log.gz 2023-10-09T06:42:43.884 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.100675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90778.log 2023-10-09T06:42:43.884 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.100675.log.gz 2023-10-09T06:42:43.884 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.99000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.99908.log 2023-10-09T06:42:43.884 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.99000.log.gz 2023-10-09T06:42:43.884 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.90778.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-09T06:42:43.884 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.99908.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.90778.log.gz 2023-10-09T06:42:43.884 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.98182.log 2023-10-09T06:42:43.884 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.99908.log.gz 2023-10-09T06:42:43.885 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.98186.log 2023-10-09T06:42:43.885 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.98182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.97771.log 2023-10-09T06:42:43.885 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.98182.log.gz 2023-10-09T06:42:43.885 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.98186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.102244.log 2023-10-09T06:42:43.885 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.98186.log.gz 2023-10-09T06:42:43.885 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.3.97771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.97401.log 2023-10-09T06:42:43.885 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.97771.log.gz 2023-10-09T06:42:43.885 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.102244.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-10-09T06:42:43.886 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.102244.log.gz 2023-10-09T06:42:43.886 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.97401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.98996.log 2023-10-09T06:42:43.886 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.97401.log.gz 2023-10-09T06:42:43.886 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.101827.log 2023-10-09T06:42:43.886 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.98996.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-09T06:42:43.886 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.98996.log.gz 2023-10-09T06:42:43.886 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.101827.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.101827.log.gz 2023-10-09T06:42:43.886 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90738.log 2023-10-09T06:42:43.887 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.97767.log 2023-10-09T06:42:43.887 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.90738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.97409.log 2023-10-09T06:42:43.887 INFO:teuthology.orchestra.run.smithi181.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.90738.log.gz 2023-10-09T06:42:43.887 INFO:teuthology.orchestra.run.smithi181.stderr: 92.1% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-09T06:42:43.887 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.3.97767.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-09T06:42:43.887 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.97767.log.gz 2023-10-09T06:42:43.887 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.97409.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-10-09T06:42:43.887 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.97409.log.gz 2023-10-09T06:42:43.888 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-10-09T06:42:43.888 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.100289.log 2023-10-09T06:42:43.888 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90862.log 2023-10-09T06:42:43.888 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.100289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.99528.log 2023-10-09T06:42:43.888 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.100289.log.gz 2023-10-09T06:42:43.888 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.90862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.101823.log 2023-10-09T06:42:43.888 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.99528.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.90862.log.gz 2023-10-09T06:42:43.888 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.99528.log.gz 2023-10-09T06:42:43.889 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.101823.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.101823.log.gz 2023-10-09T06:42:43.889 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.98600.log 2023-10-09T06:42:43.889 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-10-09T06:42:43.889 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.98600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90949.log 2023-10-09T06:42:43.889 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.98600.log.gz 2023-10-09T06:42:43.889 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.100671.log 2023-10-09T06:42:43.889 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.90949.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.90949.log.gz 2023-10-09T06:42:43.889 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.100671.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.100671.log.gz 2023-10-09T06:42:43.890 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.99524.log 2023-10-09T06:42:43.890 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-10-09T06:42:43.890 INFO:teuthology.orchestra.run.smithi181.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-09T06:42:43.890 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.99524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.101063.log 2023-10-09T06:42:43.890 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.99524.log.gz 2023-10-09T06:42:43.890 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-client.1.101063.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.101063.log.gz 2023-10-09T06:42:43.890 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91034.log 2023-10-09T06:42:43.891 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.99912.log 2023-10-09T06:42:43.891 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.91034.log: /var/log/ceph/ceph-client.1.99912.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.91034.log.gz 2023-10-09T06:42:43.891 INFO:teuthology.orchestra.run.smithi181.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.99912.log.gz 2023-10-09T06:42:43.891 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-10-09T06:42:43.891 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90991.log 2023-10-09T06:42:43.891 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.admin.90991.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.90991.log.gz 2023-10-09T06:42:43.891 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-10-09T06:42:43.894 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.98604.log 2023-10-09T06:42:43.901 INFO:teuthology.orchestra.run.smithi181.stderr: 90.3% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-10-09T06:42:43.901 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.98604.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.98604.log.gz 2023-10-09T06:42:43.901 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.101450.log 2023-10-09T06:42:43.901 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.101450.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.101450.log.gz 2023-10-09T06:42:43.901 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90819.log 2023-10-09T06:42:43.902 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.90819.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.90819.log.gz 2023-10-09T06:42:43.902 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.102646.log 2023-10-09T06:42:43.906 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.102646.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.102646.log.gz 2023-10-09T06:42:43.906 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-10-09T06:42:43.919 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.100285.log 2023-10-09T06:42:43.919 INFO:teuthology.orchestra.run.smithi181.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-10-09T06:42:43.919 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.100285.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.100285.log.gz 2023-10-09T06:42:44.055 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103711.log 2023-10-09T06:42:44.055 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101157.log 2023-10-09T06:42:44.055 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.103711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97963.log 2023-10-09T06:42:44.055 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.101157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164387.log 2023-10-09T06:42:44.055 INFO:teuthology.orchestra.run.smithi163.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103711.log.gz 2023-10-09T06:42:44.056 INFO:teuthology.orchestra.run.smithi163.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.101157.log.gz 2023-10-09T06:42:44.056 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.97963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178664.log 2023-10-09T06:42:44.056 INFO:teuthology.orchestra.run.smithi163.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97963.log.gz 2023-10-09T06:42:44.056 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.164387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164511.log 2023-10-09T06:42:44.056 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.178664.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.164387.log.gz 2023-10-09T06:42:44.056 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107446.log 2023-10-09T06:42:44.056 INFO:teuthology.orchestra.run.smithi163.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.178664.log.gz 2023-10-09T06:42:44.057 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.164511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165686.log 2023-10-09T06:42:44.057 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.107446.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.164511.log.gz 2023-10-09T06:42:44.057 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173316.log 2023-10-09T06:42:44.057 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.165686.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107446.log.gz 2023-10-09T06:42:44.057 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.100081.log 2023-10-09T06:42:44.057 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.173316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170401.log 2023-10-09T06:42:44.057 INFO:teuthology.orchestra.run.smithi163.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.165686.log.gz 2023-10-09T06:42:44.058 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.2.100081.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.173316.log.gz 2023-10-09T06:42:44.058 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174591.log 2023-10-09T06:42:44.058 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.100081.log.gz 2023-10-09T06:42:44.058 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.170401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170149.log 2023-10-09T06:42:44.058 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.174591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170275.log 2023-10-09T06:42:44.058 INFO:teuthology.orchestra.run.smithi163.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.170401.log.gz 2023-10-09T06:42:44.058 INFO:teuthology.orchestra.run.smithi163.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.174591.log.gz 2023-10-09T06:42:44.058 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.170149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.178173.log 2023-10-09T06:42:44.059 INFO:teuthology.orchestra.run.smithi163.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.170149.log.gz 2023-10-09T06:42:44.059 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.170275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98294.log 2023-10-09T06:42:44.059 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.0.178173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169800.log 2023-10-09T06:42:44.059 INFO:teuthology.orchestra.run.smithi163.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.170275.log.gz 2023-10-09T06:42:44.059 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.178173.log.gz 2023-10-09T06:42:44.059 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.98294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167570.log 2023-10-09T06:42:44.059 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.169800.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98294.log.gz 2023-10-09T06:42:44.059 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97964.log 2023-10-09T06:42:44.060 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.167570.log: 88.7%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.169800.log.gz--verbose 2023-10-09T06:42:44.060 INFO:teuthology.orchestra.run.smithi163.stderr:-- /var/log/ceph/ceph-client.admin.162860.log 2023-10-09T06:42:44.060 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.97964.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.167570.log.gz 2023-10-09T06:42:44.060 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173180.log 2023-10-09T06:42:44.060 INFO:teuthology.orchestra.run.smithi163.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.97964.log.gz 2023-10-09T06:42:44.060 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.162860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157545.log 2023-10-09T06:42:44.060 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.173180.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.162860.log.gz 2023-10-09T06:42:44.061 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167738.log 2023-10-09T06:42:44.061 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.157545.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.173180.log.gz 2023-10-09T06:42:44.061 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171751.log 2023-10-09T06:42:44.061 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.167738.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.157545.log.gz 2023-10-09T06:42:44.061 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162411.log 2023-10-09T06:42:44.061 INFO:teuthology.orchestra.run.smithi163.stderr: 88.5%/var/log/ceph/ceph-client.admin.171751.log: -- replaced with /var/log/ceph/ceph-client.admin.167738.log.gz 2023-10-09T06:42:44.062 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107404.log 2023-10-09T06:42:44.062 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.162411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102749.log 2023-10-09T06:42:44.062 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.171751.log.gz 2023-10-09T06:42:44.062 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.107404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161277.log 2023-10-09T06:42:44.062 INFO:teuthology.orchestra.run.smithi163.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.162411.log.gz 2023-10-09T06:42:44.062 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.102749.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107404.log.gz 2023-10-09T06:42:44.062 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.99681.log 2023-10-09T06:42:44.063 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.161277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159302.log 2023-10-09T06:42:44.063 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.0.99681.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.161277.log.gz 2023-10-09T06:42:44.063 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159260.log 2023-10-09T06:42:44.063 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.99681.log.gz 2023-10-09T06:42:44.063 INFO:teuthology.orchestra.run.smithi163.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.102749.log.gz 2023-10-09T06:42:44.063 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.159302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162199.log 2023-10-09T06:42:44.063 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.159260.log: 89.2%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.159302.log.gz--verbose 2023-10-09T06:42:44.064 INFO:teuthology.orchestra.run.smithi163.stderr:-- /var/log/ceph/ceph-client.0.170744.log 2023-10-09T06:42:44.064 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.162199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178916.log 2023-10-09T06:42:44.064 INFO:teuthology.orchestra.run.smithi163.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.159260.log.gz 2023-10-09T06:42:44.064 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.0.170744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101451.log 2023-10-09T06:42:44.064 INFO:teuthology.orchestra.run.smithi163.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.162199.log.gz 2023-10-09T06:42:44.064 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.170744.log.gz 2023-10-09T06:42:44.064 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.178916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101241.log 2023-10-09T06:42:44.065 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.101451.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.178916.log.gz 2023-10-09T06:42:44.065 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169181.log 2023-10-09T06:42:44.065 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.101241.log: 88.2%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.101451.log.gz--verbose 2023-10-09T06:42:44.065 INFO:teuthology.orchestra.run.smithi163.stderr:-- /var/log/ceph/ceph-client.admin.179552.log 2023-10-09T06:42:44.065 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.169181.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.101241.log.gz 2023-10-09T06:42:44.065 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158229.log 2023-10-09T06:42:44.065 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.179552.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.169181.log.gz 2023-10-09T06:42:44.066 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162818.log 2023-10-09T06:42:44.066 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.158229.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.179552.log.gz 2023-10-09T06:42:44.066 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172582.log 2023-10-09T06:42:44.066 INFO:teuthology.orchestra.run.smithi163.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.158229.log.gz 2023-10-09T06:42:44.066 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.162818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161926.log 2023-10-09T06:42:44.066 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.172582.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.162818.log.gz 2023-10-09T06:42:44.066 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160328.log 2023-10-09T06:42:44.067 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.161926.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.172582.log.gz 2023-10-09T06:42:44.067 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.84611.log 2023-10-09T06:42:44.067 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.160328.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.161926.log.gz 2023-10-09T06:42:44.067 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.178169.log 2023-10-09T06:42:44.067 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.160328.log.gz 2023-10-09T06:42:44.067 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph.tmp-client.admin.84611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179102.log 2023-10-09T06:42:44.067 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.84611.log.gz 2023-10-09T06:42:44.067 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.0.178169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105239.log 2023-10-09T06:42:44.068 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.178169.log.gz 2023-10-09T06:42:44.068 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.179102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175175.log 2023-10-09T06:42:44.068 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.105239.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.179102.log.gz 2023-10-09T06:42:44.068 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159050.log 2023-10-09T06:42:44.068 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.175175.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.105239.log.gz 2023-10-09T06:42:44.068 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107790.log 2023-10-09T06:42:44.068 INFO:teuthology.orchestra.run.smithi163.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.175175.log.gz/var/log/ceph/ceph-client.admin.159050.log: 2023-10-09T06:42:44.068 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175773.log 2023-10-09T06:42:44.069 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.107790.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.159050.log.gz 2023-10-09T06:42:44.069 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180073.log 2023-10-09T06:42:44.069 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.175773.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107790.log.gz 2023-10-09T06:42:44.069 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164163.log 2023-10-09T06:42:44.069 INFO:teuthology.orchestra.run.smithi163.stderr: 89.7%/var/log/ceph/ceph-client.admin.180073.log: -- replaced with /var/log/ceph/ceph-client.admin.175773.log.gz 2023-10-09T06:42:44.069 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98168.log 2023-10-09T06:42:44.069 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.164163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100126.log 2023-10-09T06:42:44.069 INFO:teuthology.orchestra.run.smithi163.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.180073.log.gz 2023-10-09T06:42:44.070 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.98168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173924.log 2023-10-09T06:42:44.070 INFO:teuthology.orchestra.run.smithi163.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.164163.log.gz 2023-10-09T06:42:44.070 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.100126.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98168.log.gz 2023-10-09T06:42:44.070 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167864.log 2023-10-09T06:42:44.070 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.173924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164247.log 2023-10-09T06:42:44.070 INFO:teuthology.orchestra.run.smithi163.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.100126.log.gz 2023-10-09T06:42:44.070 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.173924.log.gz 2023-10-09T06:42:44.071 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.167864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172116.log 2023-10-09T06:42:44.071 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.164247.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.167864.log.gz 2023-10-09T06:42:44.071 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172200.log 2023-10-09T06:42:44.071 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.172116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.168169.log 2023-10-09T06:42:44.071 INFO:teuthology.orchestra.run.smithi163.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.164247.log.gz 2023-10-09T06:42:44.071 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-10-09T06:42:44.071 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.172200.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.172116.log.gz 2023-10-09T06:42:44.071 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.0.168169.log: gzip 87.2%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.172200.log.gz-- 2023-10-09T06:42:44.072 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0%/var/log/ceph/ceph-client.admin.157669.log 2023-10-09T06:42:44.072 INFO:teuthology.orchestra.run.smithi163.stderr: -- replaced with /var/log/ceph/ceph-client.0.168169.log.gz 2023-10-09T06:42:44.072 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98840.log 2023-10-09T06:42:44.072 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.157669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104861.log 2023-10-09T06:42:44.072 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.98840.log: 88.8%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.157669.log.gz--verbose 2023-10-09T06:42:44.072 INFO:teuthology.orchestra.run.smithi163.stderr:-- /var/log/ceph/ceph-client.admin.100421.log 2023-10-09T06:42:44.072 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.104861.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.98840.log.gz 2023-10-09T06:42:44.072 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101199.log 2023-10-09T06:42:44.073 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.100421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170873.log 2023-10-09T06:42:44.073 INFO:teuthology.orchestra.run.smithi163.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.104861.log.gz 2023-10-09T06:42:44.073 INFO:teuthology.orchestra.run.smithi163.stderr: 89.0%/var/log/ceph/ceph-client.admin.101199.log: -- replaced with /var/log/ceph/ceph-client.admin.100421.log.gz 2023-10-09T06:42:44.073 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105281.log 2023-10-09T06:42:44.073 INFO:teuthology.orchestra.run.smithi163.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.101199.log.gz 2023-10-09T06:42:44.073 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.170873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157795.log 2023-10-09T06:42:44.073 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.105281.log: gzip 88.2%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.170873.log.gz-- 2023-10-09T06:42:44.074 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.171136.log 2023-10-09T06:42:44.074 INFO:teuthology.orchestra.run.smithi163.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105281.log.gz 2023-10-09T06:42:44.074 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.157795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178874.log 2023-10-09T06:42:44.074 INFO:teuthology.orchestra.run.smithi163.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.157795.log.gz 2023-10-09T06:42:44.074 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.171136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100590.log 2023-10-09T06:42:44.074 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.178874.log: gzip 88.2%-5 -- replaced with /var/log/ceph/ceph-client.admin.171136.log.gz--verbose 2023-10-09T06:42:44.074 INFO:teuthology.orchestra.run.smithi163.stderr:-- /var/log/ceph/ceph-client.admin.169506.log 2023-10-09T06:42:44.074 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.178874.log.gz 2023-10-09T06:42:44.075 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.100590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173798.log 2023-10-09T06:42:44.075 INFO:teuthology.orchestra.run.smithi163.stderr: 88.0%/var/log/ceph/ceph-client.admin.169506.log: -- replaced with /var/log/ceph/ceph-client.admin.100590.log.gz 2023-10-09T06:42:44.075 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166911.log 2023-10-09T06:42:44.075 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.173798.log: gzip -5 88.0%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.169506.log.gz 2023-10-09T06:42:44.075 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.180115.log 2023-10-09T06:42:44.075 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.166911.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.173798.log.gz 2023-10-09T06:42:44.075 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158869.log 2023-10-09T06:42:44.076 INFO:teuthology.orchestra.run.smithi163.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.166911.log.gz 2023-10-09T06:42:44.076 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.180115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100379.log 2023-10-09T06:42:44.076 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.158869.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.180115.log.gz 2023-10-09T06:42:44.076 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163010.log 2023-10-09T06:42:44.076 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.100379.log: gzip 87.2%-5 -- replaced with /var/log/ceph/ceph-client.admin.158869.log.gz--verbose 2023-10-09T06:42:44.076 INFO:teuthology.orchestra.run.smithi163.stderr:-- /var/log/ceph/ceph-client.admin.157711.log 2023-10-09T06:42:44.076 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100379.log.gz 2023-10-09T06:42:44.076 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.163010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180283.log 2023-10-09T06:42:44.076 INFO:teuthology.orchestra.run.smithi163.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.163010.log.gz 2023-10-09T06:42:44.077 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.157711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178214.log 2023-10-09T06:42:44.077 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.180283.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.157711.log.gz 2023-10-09T06:42:44.077 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179737.log 2023-10-09T06:42:44.077 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.178214.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.180283.log.gz 2023-10-09T06:42:44.077 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97118.log 2023-10-09T06:42:44.077 INFO:teuthology.orchestra.run.smithi163.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.178214.log.gz 2023-10-09T06:42:44.077 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.179737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106768.log 2023-10-09T06:42:44.078 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.97118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.158618.log 2023-10-09T06:42:44.078 INFO:teuthology.orchestra.run.smithi163.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.179737.log.gz 2023-10-09T06:42:44.078 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.106768.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97118.log.gz 2023-10-09T06:42:44.078 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169714.log 2023-10-09T06:42:44.078 INFO:teuthology.orchestra.run.smithi163.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.106768.log.gz 2023-10-09T06:42:44.078 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.0.158618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158701.log 2023-10-09T06:42:44.078 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.158618.log.gz 2023-10-09T06:42:44.078 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.169714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171540.log 2023-10-09T06:42:44.079 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.158701.log: gzip -5 --verbose 89.0%-- -- replaced with /var/log/ceph/ceph-client.admin.169714.log.gz/var/log/ceph/ceph-client.admin.162776.log 2023-10-09T06:42:44.079 INFO:teuthology.orchestra.run.smithi163.stderr: 2023-10-09T06:42:44.079 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.171540.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.158701.log.gz 2023-10-09T06:42:44.079 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.163207.log 2023-10-09T06:42:44.079 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.162776.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.171540.log.gz 2023-10-09T06:42:44.079 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158017.log 2023-10-09T06:42:44.079 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.0.163207.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.162776.log.gz 2023-10-09T06:42:44.079 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159176.log 2023-10-09T06:42:44.080 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.163207.log.gz 2023-10-09T06:42:44.080 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.158017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176876.log 2023-10-09T06:42:44.080 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.159176.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.158017.log.gz 2023-10-09T06:42:44.080 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170317.log 2023-10-09T06:42:44.080 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.176876.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.159176.log.gz 2023-10-09T06:42:44.080 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160582.log 2023-10-09T06:42:44.080 INFO:teuthology.orchestra.run.smithi163.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.176876.log.gz 2023-10-09T06:42:44.081 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.170317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105365.log 2023-10-09T06:42:44.081 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.160582.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.170317.log.gz 2023-10-09T06:42:44.081 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101577.log 2023-10-09T06:42:44.081 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.105365.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.160582.log.gz 2023-10-09T06:42:44.081 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177784.log 2023-10-09T06:42:44.081 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.101577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.158614.log 2023-10-09T06:42:44.081 INFO:teuthology.orchestra.run.smithi163.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.105365.log.gz 2023-10-09T06:42:44.081 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.177784.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101577.log.gz 2023-10-09T06:42:44.082 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166728.log 2023-10-09T06:42:44.082 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176229.log 2023-10-09T06:42:44.082 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.0.158614.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.177784.log.gz 2023-10-09T06:42:44.082 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.158614.log.gz 2023-10-09T06:42:44.082 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.166728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166407.log 2023-10-09T06:42:44.082 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.176229.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.166728.log.gz 2023-10-09T06:42:44.082 INFO:teuthology.orchestra.run.smithi163.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.179653.log 2023-10-09T06:42:44.082 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.166407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106202.log 88.1% 2023-10-09T06:42:44.083 INFO:teuthology.orchestra.run.smithi163.stderr: -- replaced with /var/log/ceph/ceph-client.admin.176229.log.gz 2023-10-09T06:42:44.083 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.179653.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.166407.log.gz 2023-10-09T06:42:44.083 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168470.log 2023-10-09T06:42:44.083 INFO:teuthology.orchestra.run.smithi163.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.179653.log.gz 2023-10-09T06:42:44.083 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.106202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98672.log 2023-10-09T06:42:44.083 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.168470.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106202.log.gz 2023-10-09T06:42:44.083 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169936.log 2023-10-09T06:42:44.083 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.98672.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.168470.log.gz 2023-10-09T06:42:44.084 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157075.log 2023-10-09T06:42:44.084 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98672.log.gz 2023-10-09T06:42:44.084 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.169936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97961.log 2023-10-09T06:42:44.084 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.157075.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.169936.log.gz 2023-10-09T06:42:44.084 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96452.log 2023-10-09T06:42:44.084 INFO:teuthology.orchestra.run.smithi163.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.157075.log.gz 2023-10-09T06:42:44.084 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.97961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96887.log 2023-10-09T06:42:44.085 INFO:teuthology.orchestra.run.smithi163.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.97961.log.gz 2023-10-09T06:42:44.085 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.96452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158355.log 2023-10-09T06:42:44.085 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.96887.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96452.log.gz 2023-10-09T06:42:44.085 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165830.log 2023-10-09T06:42:44.085 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.158355.log: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.96887.log.gz-5 2023-10-09T06:42:44.085 INFO:teuthology.orchestra.run.smithi163.stderr:--verbose -- /var/log/ceph/ceph-client.admin.103367.log 2023-10-09T06:42:44.085 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.165830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170107.log 2023-10-09T06:42:44.085 INFO:teuthology.orchestra.run.smithi163.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.158355.log.gz 2023-10-09T06:42:44.086 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.103367.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.165830.log.gz 2023-10-09T06:42:44.086 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103627.log 2023-10-09T06:42:44.086 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160106.log 2023-10-09T06:42:44.086 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103367.log.gz 2023-10-09T06:42:44.086 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.170107.log: /var/log/ceph/ceph-client.admin.103627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161655.log 2023-10-09T06:42:44.086 INFO:teuthology.orchestra.run.smithi163.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.170107.log.gz 2023-10-09T06:42:44.086 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.160106.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.103627.log.gz 2023-10-09T06:42:44.086 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177438.log 2023-10-09T06:42:44.087 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.161655.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.160106.log.gz 2023-10-09T06:42:44.087 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104131.log 2023-10-09T06:42:44.087 INFO:teuthology.orchestra.run.smithi163.stderr: 88.1%/var/log/ceph/ceph-client.admin.177438.log: -- replaced with /var/log/ceph/ceph-client.admin.161655.log.gz 2023-10-09T06:42:44.087 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172372.log 2023-10-09T06:42:44.087 INFO:teuthology.orchestra.run.smithi163.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.177438.log.gz 2023-10-09T06:42:44.087 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.104131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101778.log 2023-10-09T06:42:44.087 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.172372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105992.log 2023-10-09T06:42:44.088 INFO:teuthology.orchestra.run.smithi163.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104131.log.gz 2023-10-09T06:42:44.088 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.101778.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.172372.log.gz 2023-10-09T06:42:44.088 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103145.log 2023-10-09T06:42:44.088 INFO:teuthology.orchestra.run.smithi163.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101778.log.gz 2023-10-09T06:42:44.088 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.105992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107184.log 2023-10-09T06:42:44.088 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.103145.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.105992.log.gz 2023-10-09T06:42:44.088 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.170748.log 2023-10-09T06:42:44.088 INFO:teuthology.orchestra.run.smithi163.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103145.log.gz 2023-10-09T06:42:44.089 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.107184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162564.log 2023-10-09T06:42:44.089 INFO:teuthology.orchestra.run.smithi163.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107184.log.gz 2023-10-09T06:42:44.089 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.0.170748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176397.log 2023-10-09T06:42:44.089 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.170748.log.gz 2023-10-09T06:42:44.089 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.162564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107054.log 2023-10-09T06:42:44.089 INFO:teuthology.orchestra.run.smithi163.stderr: 87.6%/var/log/ceph/ceph-client.admin.176397.log: -- replaced with /var/log/ceph/ceph-client.admin.162564.log.gz 2023-10-09T06:42:44.089 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169978.log 2023-10-09T06:42:44.089 INFO:teuthology.orchestra.run.smithi163.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.176397.log.gz 2023-10-09T06:42:44.090 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.107054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101905.log 2023-10-09T06:42:44.090 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.169978.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107054.log.gz 2023-10-09T06:42:44.090 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171667.log 2023-10-09T06:42:44.090 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.101905.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.169978.log.gz 2023-10-09T06:42:44.090 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163374.log 2023-10-09T06:42:44.090 INFO:teuthology.orchestra.run.smithi163.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.101905.log.gz 2023-10-09T06:42:44.090 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.171667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174445.log 2023-10-09T06:42:44.091 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.163374.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.171667.log.gz 2023-10-09T06:42:44.091 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158421.log 2023-10-09T06:42:44.091 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.174445.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.163374.log.gz 2023-10-09T06:42:44.091 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177350.log 2023-10-09T06:42:44.091 INFO:teuthology.orchestra.run.smithi163.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.174445.log.gz 2023-10-09T06:42:44.091 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.158421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176313.log 2023-10-09T06:42:44.091 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.177350.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.158421.log.gz 2023-10-09T06:42:44.091 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103411.log 2023-10-09T06:42:44.092 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.176313.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.177350.log.gz 2023-10-09T06:42:44.092 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104173.log 2023-10-09T06:42:44.092 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.176313.log.gz 2023-10-09T06:42:44.092 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.103411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98504.log 2023-10-09T06:42:44.092 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.104173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173484.log 2023-10-09T06:42:44.092 INFO:teuthology.orchestra.run.smithi163.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103411.log.gz 2023-10-09T06:42:44.092 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.98504.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.104173.log.gz 2023-10-09T06:42:44.092 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.160929.log 2023-10-09T06:42:44.093 INFO:teuthology.orchestra.run.smithi163.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98504.log.gz 2023-10-09T06:42:44.093 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.173484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160244.log 2023-10-09T06:42:44.093 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.0.160929.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.173484.log.gzgzip 2023-10-09T06:42:44.093 INFO:teuthology.orchestra.run.smithi163.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.166770.log 2023-10-09T06:42:44.093 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.160929.log.gz 2023-10-09T06:42:44.093 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.160244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97370.log 2023-10-09T06:42:44.093 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.166770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103879.log 2023-10-09T06:42:44.093 INFO:teuthology.orchestra.run.smithi163.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.160244.log.gz 2023-10-09T06:42:44.094 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.97370.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.166770.log.gz 2023-10-09T06:42:44.094 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176033.log 2023-10-09T06:42:44.094 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.103879.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97370.log.gz 2023-10-09T06:42:44.094 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106970.log 2023-10-09T06:42:44.094 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103879.log.gz 2023-10-09T06:42:44.094 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.176033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98798.log 2023-10-09T06:42:44.094 INFO:teuthology.orchestra.run.smithi163.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.176033.log.gz 2023-10-09T06:42:44.095 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.106970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164595.log 2023-10-09T06:42:44.095 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.98798.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106970.log.gz 2023-10-09T06:42:44.095 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176481.log 2023-10-09T06:42:44.095 INFO:teuthology.orchestra.run.smithi163.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98798.log.gz 2023-10-09T06:42:44.095 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.164595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158785.log 2023-10-09T06:42:44.095 INFO:teuthology.orchestra.run.smithi163.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.164595.log.gz 2023-10-09T06:42:44.095 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.176481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162117.log 2023-10-09T06:42:44.095 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.158785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171220.log 2023-10-09T06:42:44.096 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.176481.log.gz 2023-10-09T06:42:44.096 INFO:teuthology.orchestra.run.smithi163.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.158785.log.gz 2023-10-09T06:42:44.096 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.162117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107276.log 2023-10-09T06:42:44.096 INFO:teuthology.orchestra.run.smithi163.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.162117.log.gz 2023-10-09T06:42:44.096 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.171220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168784.log 2023-10-09T06:42:44.096 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.107276.log: 88.3%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.171220.log.gz--verbose 2023-10-09T06:42:44.096 INFO:teuthology.orchestra.run.smithi163.stderr:-- /var/log/ceph/ceph-client.admin.170831.log 2023-10-09T06:42:44.097 INFO:teuthology.orchestra.run.smithi163.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.107276.log.gz 2023-10-09T06:42:44.097 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.168784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156907.log 2023-10-09T06:42:44.097 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.168784.log.gz 2023-10-09T06:42:44.097 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.170831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178832.log 2023-10-09T06:42:44.097 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.156907.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.170831.log.gz 2023-10-09T06:42:44.097 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158187.log 2023-10-09T06:42:44.097 INFO:teuthology.orchestra.run.smithi163.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.156907.log.gz 2023-10-09T06:42:44.097 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.178832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97286.log 2023-10-09T06:42:44.098 INFO:teuthology.orchestra.run.smithi163.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.178832.log.gz 2023-10-09T06:42:44.098 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.158187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102665.log 2023-10-09T06:42:44.098 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.97286.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.158187.log.gz 2023-10-09T06:42:44.098 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103187.log 2023-10-09T06:42:44.098 INFO:teuthology.orchestra.run.smithi163.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.97286.log.gz 2023-10-09T06:42:44.098 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.102665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171094.log 2023-10-09T06:42:44.098 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102665.log.gz 2023-10-09T06:42:44.098 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.103187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162367.log 2023-10-09T06:42:44.099 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.171094.log: 89.5%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.103187.log.gz--verbose 2023-10-09T06:42:44.099 INFO:teuthology.orchestra.run.smithi163.stderr:-- /var/log/ceph/ceph-client.admin.157220.log 2023-10-09T06:42:44.099 INFO:teuthology.orchestra.run.smithi163.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.171094.log.gz 2023-10-09T06:42:44.099 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.162367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104903.log 2023-10-09T06:42:44.099 INFO:teuthology.orchestra.run.smithi163.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.162367.log.gz 2023-10-09T06:42:44.099 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.157220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103753.log 2023-10-09T06:42:44.099 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.104903.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.157220.log.gz 2023-10-09T06:42:44.099 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106034.log 2023-10-09T06:42:44.100 INFO:teuthology.orchestra.run.smithi163.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.104903.log.gz 2023-10-09T06:42:44.100 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.103753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97624.log 2023-10-09T06:42:44.100 INFO:teuthology.orchestra.run.smithi163.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103753.log.gz 2023-10-09T06:42:44.100 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.106034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180462.log 2023-10-09T06:42:44.100 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.97624.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.106034.log.gz 2023-10-09T06:42:44.101 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177022.log 2023-10-09T06:42:44.101 INFO:teuthology.orchestra.run.smithi163.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97624.log.gz 2023-10-09T06:42:44.101 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.180462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163555.log 2023-10-09T06:42:44.101 INFO:teuthology.orchestra.run.smithi163.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.180462.log.gz 2023-10-09T06:42:44.101 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.177022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161235.log 2023-10-09T06:42:44.101 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.163555.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.177022.log.gz 2023-10-09T06:42:44.102 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97962.log 2023-10-09T06:42:44.102 INFO:teuthology.orchestra.run.smithi163.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.163555.log.gz 2023-10-09T06:42:44.102 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.161235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168554.log 2023-10-09T06:42:44.102 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.161235.log.gz 2023-10-09T06:42:44.102 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.97962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107362.log 2023-10-09T06:42:44.102 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.168554.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97962.log.gz 2023-10-09T06:42:44.102 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100842.log 2023-10-09T06:42:44.103 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.168554.log.gz 2023-10-09T06:42:44.103 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.107362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159839.log 2023-10-09T06:42:44.103 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107362.log.gz 2023-10-09T06:42:44.103 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.100842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104691.log 2023-10-09T06:42:44.103 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.159839.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.100842.log.gz 2023-10-09T06:42:44.103 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167528.log 2023-10-09T06:42:44.104 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.159839.log.gz 2023-10-09T06:42:44.104 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.104691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176075.log 2023-10-09T06:42:44.104 INFO:teuthology.orchestra.run.smithi163.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.104691.log.gz 2023-10-09T06:42:44.104 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.167528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96761.log 2023-10-09T06:42:44.105 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.176075.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.167528.log.gz 2023-10-09T06:42:44.105 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176523.log 2023-10-09T06:42:44.105 INFO:teuthology.orchestra.run.smithi163.stderr: 88.9%/var/log/ceph/ceph-client.admin.96761.log: -- replaced with /var/log/ceph/ceph-client.admin.176075.log.gz 2023-10-09T06:42:44.105 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-09T06:42:44.105 INFO:teuthology.orchestra.run.smithi163.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96761.log.gz 2023-10-09T06:42:44.105 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 /var/log/ceph/ceph-client.admin.176523.log: --verbose -- /var/log/ceph/ceph-client.admin.168336.log 2023-10-09T06:42:44.105 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101409.log 2023-10-09T06:42:44.106 INFO:teuthology.orchestra.run.smithi163.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.176523.log.gz 2023-10-09T06:42:44.106 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.168336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172158.log 2023-10-09T06:42:44.106 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.101409.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.168336.log.gz 2023-10-09T06:42:44.106 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178748.log 2023-10-09T06:42:44.106 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101409.log.gz 2023-10-09T06:42:44.106 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.172158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160372.log 2023-10-09T06:42:44.106 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.178748.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.172158.log.gz 2023-10-09T06:42:44.107 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180199.log 2023-10-09T06:42:44.107 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.160372.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.178748.log.gz 2023-10-09T06:42:44.107 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173610.log 2023-10-09T06:42:44.107 INFO:teuthology.orchestra.run.smithi163.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.160372.log.gz 2023-10-09T06:42:44.107 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180031.log 2023-10-09T06:42:44.107 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.180199.log: /var/log/ceph/ceph-client.admin.173610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158827.log 2023-10-09T06:42:44.107 INFO:teuthology.orchestra.run.smithi163.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.180199.log.gz 2023-10-09T06:42:44.107 INFO:teuthology.orchestra.run.smithi163.stderr: 86.0% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-09T06:42:44.108 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.180031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104735.log 2023-10-09T06:42:44.108 INFO:teuthology.orchestra.run.smithi163.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.173610.log.gz 2023-10-09T06:42:44.108 INFO:teuthology.orchestra.run.smithi163.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.180031.log.gz 2023-10-09T06:42:44.108 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.158827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166197.log 2023-10-09T06:42:44.108 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.104735.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.158827.log.gz 2023-10-09T06:42:44.108 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104521.log 2023-10-09T06:42:44.108 INFO:teuthology.orchestra.run.smithi163.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104735.log.gz 2023-10-09T06:42:44.108 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.166197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160064.log 2023-10-09T06:42:44.108 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.104521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.100085.log 2023-10-09T06:42:44.109 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.166197.log.gz 2023-10-09T06:42:44.109 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.160064.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104521.log.gz 2023-10-09T06:42:44.109 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104319.log 2023-10-09T06:42:44.109 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.160064.log.gz 2023-10-09T06:42:44.109 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.2.100085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170957.log 2023-10-09T06:42:44.109 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.100085.log.gz 2023-10-09T06:42:44.109 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.104319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166113.log 2023-10-09T06:42:44.110 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.170957.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.104319.log.gz 2023-10-09T06:42:44.110 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174669.log 2023-10-09T06:42:44.110 INFO:teuthology.orchestra.run.smithi163.stderr: 85.6%/var/log/ceph/ceph-client.admin.166113.log: -- replaced with /var/log/ceph/ceph-client.admin.170957.log.gz 2023-10-09T06:42:44.110 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97623.log 2023-10-09T06:42:44.110 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.174669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156697.log 88.6% 2023-10-09T06:42:44.110 INFO:teuthology.orchestra.run.smithi163.stderr: -- replaced with /var/log/ceph/ceph-client.admin.166113.log.gz 2023-10-09T06:42:44.110 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.97623.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.174669.log.gz 2023-10-09T06:42:44.111 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162039.log 2023-10-09T06:42:44.111 INFO:teuthology.orchestra.run.smithi163.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.97623.log.gz 2023-10-09T06:42:44.111 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.156697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107748.log 2023-10-09T06:42:44.111 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.162039.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.156697.log.gz 2023-10-09T06:42:44.111 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179468.log 2023-10-09T06:42:44.111 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.107748.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.162039.log.gz 2023-10-09T06:42:44.111 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164987.log 2023-10-09T06:42:44.111 INFO:teuthology.orchestra.run.smithi163.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.107748.log.gz 2023-10-09T06:42:44.112 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.179468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106454.log 2023-10-09T06:42:44.112 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.164987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169630.log 2023-10-09T06:42:44.112 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.179468.log.gz 2023-10-09T06:42:44.112 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.106454.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.164987.log.gz 2023-10-09T06:42:44.112 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97627.log 2023-10-09T06:42:44.112 INFO:teuthology.orchestra.run.smithi163.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106454.log.gz 2023-10-09T06:42:44.112 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.169630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.103008.log 2023-10-09T06:42:44.112 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.97627.log: 88.2%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.169630.log.gz--verbose 2023-10-09T06:42:44.113 INFO:teuthology.orchestra.run.smithi163.stderr:-- /var/log/ceph/ceph-client.admin.102497.log 2023-10-09T06:42:44.113 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.0.103008.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.97627.log.gz 2023-10-09T06:42:44.113 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.103008.log.gz 2023-10-09T06:42:44.113 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179989.log 2023-10-09T06:42:44.113 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.102497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174008.log 2023-10-09T06:42:44.113 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.179989.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102497.log.gz 2023-10-09T06:42:44.113 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102623.log 2023-10-09T06:42:44.113 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.174008.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.179989.log.gz 2023-10-09T06:42:44.114 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98588.log 2023-10-09T06:42:44.114 INFO:teuthology.orchestra.run.smithi163.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.174008.log.gz 2023-10-09T06:42:44.114 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.102623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106118.log 2023-10-09T06:42:44.114 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.98588.log: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.102623.log.gz-5 2023-10-09T06:42:44.114 INFO:teuthology.orchestra.run.smithi163.stderr:--verbose -- /var/log/ceph/ceph-client.admin.106622.log 2023-10-09T06:42:44.114 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.106118.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.98588.log.gz 2023-10-09T06:42:44.114 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175991.log 2023-10-09T06:42:44.115 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106118.log.gz 2023-10-09T06:42:44.115 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.106622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163765.log 2023-10-09T06:42:44.115 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.175991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158313.log 2023-10-09T06:42:44.115 INFO:teuthology.orchestra.run.smithi163.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.106622.log.gz 2023-10-09T06:42:44.115 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.163765.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.175991.log.gz 2023-10-09T06:42:44.115 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166029.log 2023-10-09T06:42:44.115 INFO:teuthology.orchestra.run.smithi163.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.163765.log.gz 2023-10-09T06:42:44.115 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.158313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.105628.log 2023-10-09T06:42:44.116 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.166029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167612.log 2023-10-09T06:42:44.116 INFO:teuthology.orchestra.run.smithi163.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.158313.log.gz 2023-10-09T06:42:44.116 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.0.105628.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.166029.log.gz 2023-10-09T06:42:44.116 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.105628.log.gz 2023-10-09T06:42:44.116 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103543.log 2023-10-09T06:42:44.116 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.167612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.108091.log 2023-10-09T06:42:44.116 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.103543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167780.log 2023-10-09T06:42:44.116 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.167612.log.gz 2023-10-09T06:42:44.117 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.0.108091.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103543.log.gz 2023-10-09T06:42:44.117 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.108091.log.gz 2023-10-09T06:42:44.117 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166602.log 2023-10-09T06:42:44.117 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.167780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97328.log 2023-10-09T06:42:44.117 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.166602.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.167780.log.gz 2023-10-09T06:42:44.117 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179187.log 2023-10-09T06:42:44.117 INFO:teuthology.orchestra.run.smithi163.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.166602.log.gz 2023-10-09T06:42:44.117 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.97328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164637.log 2023-10-09T06:42:44.118 INFO:teuthology.orchestra.run.smithi163.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97328.log.gz 2023-10-09T06:42:44.118 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.179187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177182.log 2023-10-09T06:42:44.118 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.164637.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.179187.log.gz 2023-10-09T06:42:44.118 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157975.log 2023-10-09T06:42:44.118 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.177182.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.164637.log.gz 2023-10-09T06:42:44.118 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156508.log 2023-10-09T06:42:44.118 INFO:teuthology.orchestra.run.smithi163.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.177182.log.gz 2023-10-09T06:42:44.119 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.157975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97034.log 2023-10-09T06:42:44.119 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.156508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173714.log 2023-10-09T06:42:44.119 INFO:teuthology.orchestra.run.smithi163.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.157975.log.gz 2023-10-09T06:42:44.119 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.97034.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.156508.log.gz 2023-10-09T06:42:44.119 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169428.log 2023-10-09T06:42:44.119 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97034.log.gz 2023-10-09T06:42:44.119 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.173714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171178.log 2023-10-09T06:42:44.119 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.169428.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.173714.log.gz 2023-10-09T06:42:44.120 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167136.log 2023-10-09T06:42:44.120 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.171178.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.169428.log.gz 2023-10-09T06:42:44.120 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175469.log 2023-10-09T06:42:44.120 INFO:teuthology.orchestra.run.smithi163.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.171178.log.gz 2023-10-09T06:42:44.120 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.167136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108279.log 2023-10-09T06:42:44.120 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.175469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176834.log 87.6% 2023-10-09T06:42:44.120 INFO:teuthology.orchestra.run.smithi163.stderr: -- replaced with /var/log/ceph/ceph-client.admin.167136.log.gz 2023-10-09T06:42:44.120 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.108279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96634.log 2023-10-09T06:42:44.121 INFO:teuthology.orchestra.run.smithi163.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.108279.log.gz 2023-10-09T06:42:44.121 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.176834.log: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.175469.log.gz 2023-10-09T06:42:44.121 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172456.log 2023-10-09T06:42:44.121 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.176834.log.gz 2023-10-09T06:42:44.121 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.96634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103921.log 2023-10-09T06:42:44.121 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.172456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162522.log 2023-10-09T06:42:44.121 INFO:teuthology.orchestra.run.smithi163.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.96634.log.gz 2023-10-09T06:42:44.121 INFO:teuthology.orchestra.run.smithi163.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.172456.log.gz 2023-10-09T06:42:44.122 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.103921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174134.log 2023-10-09T06:42:44.122 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.162522.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103921.log.gz 2023-10-09T06:42:44.122 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161403.log 2023-10-09T06:42:44.122 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.174134.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.162522.log.gz 2023-10-09T06:42:44.122 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161445.log 2023-10-09T06:42:44.122 INFO:teuthology.orchestra.run.smithi163.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.174134.log.gz 2023-10-09T06:42:44.122 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.161403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172750.log 2023-10-09T06:42:44.123 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.161445.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.161403.log.gz 2023-10-09T06:42:44.123 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164903.log 2023-10-09T06:42:44.123 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.172750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97160.log 2023-10-09T06:42:44.123 INFO:teuthology.orchestra.run.smithi163.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.161445.log.gz 2023-10-09T06:42:44.123 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.164903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102413.log 2023-10-09T06:42:44.123 INFO:teuthology.orchestra.run.smithi163.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.172750.log.gz 2023-10-09T06:42:44.123 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.97160.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.164903.log.gz 2023-10-09T06:42:44.123 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170233.log 2023-10-09T06:42:44.124 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159134.log /var/log/ceph/ceph-client.admin.102413.log: 2023-10-09T06:42:44.124 INFO:teuthology.orchestra.run.smithi163.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.97160.log.gz 2023-10-09T06:42:44.124 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.170233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163597.log 2023-10-09T06:42:44.124 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102413.log.gz 2023-10-09T06:42:44.124 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.159134.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.170233.log.gz 2023-10-09T06:42:44.124 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159446.log 2023-10-09T06:42:44.124 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.163597.log: 90.9%gzip -- replaced with /var/log/ceph/ceph-mds.a.log.gz 89.0% 2023-10-09T06:42:44.125 INFO:teuthology.orchestra.run.smithi163.stderr:-5 -- replaced with /var/log/ceph/ceph-client.admin.159134.log.gz--verbose 2023-10-09T06:42:44.125 INFO:teuthology.orchestra.run.smithi163.stderr:-- /var/log/ceph/ceph-client.admin.166560.log 2023-10-09T06:42:44.125 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.159446.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.163597.log.gz 2023-10-09T06:42:44.125 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159573.log 2023-10-09T06:42:44.125 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.159446.log.gz 2023-10-09T06:42:44.125 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.166560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178342.log 2023-10-09T06:42:44.125 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.159573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163332.log 2023-10-09T06:42:44.125 INFO:teuthology.orchestra.run.smithi163.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.166560.log.gz 2023-10-09T06:42:44.126 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.178342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170485.log 2023-10-09T06:42:44.126 INFO:teuthology.orchestra.run.smithi163.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.159573.log.gz 2023-10-09T06:42:44.126 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.163332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101283.log 2023-10-09T06:42:44.126 INFO:teuthology.orchestra.run.smithi163.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.178342.log.gz 2023-10-09T06:42:44.126 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.170485.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.163332.log.gz 2023-10-09T06:42:44.126 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164037.log 2023-10-09T06:42:44.126 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.101283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174487.log 2023-10-09T06:42:44.126 INFO:teuthology.orchestra.run.smithi163.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101283.log.gz 2023-10-09T06:42:44.127 INFO:teuthology.orchestra.run.smithi163.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.170485.log.gz 2023-10-09T06:42:44.127 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.164037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164309.log 2023-10-09T06:42:44.127 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.174487.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-09T06:42:44.127 INFO:teuthology.orchestra.run.smithi163.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.164037.log.gz 2023-10-09T06:42:44.127 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.164309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172032.log 2023-10-09T06:42:44.127 INFO:teuthology.orchestra.run.smithi163.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.174487.log.gz 2023-10-09T06:42:44.127 INFO:teuthology.orchestra.run.smithi163.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.164309.log.gz 2023-10-09T06:42:44.128 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106370.log 2023-10-09T06:42:44.128 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.172032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175007.log 2023-10-09T06:42:44.128 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.106370.log: gzip 89.0%-5 -- replaced with /var/log/ceph/ceph-client.admin.172032.log.gz--verbose 2023-10-09T06:42:44.128 INFO:teuthology.orchestra.run.smithi163.stderr:-- /var/log/ceph/ceph-client.0.173009.log 2023-10-09T06:42:44.128 INFO:teuthology.orchestra.run.smithi163.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.106370.log.gz 2023-10-09T06:42:44.128 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.175007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172288.log 2023-10-09T06:42:44.128 INFO:teuthology.orchestra.run.smithi163.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.175007.log.gz 2023-10-09T06:42:44.129 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.0.173009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100295.log 2023-10-09T06:42:44.129 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.173009.log.gz 2023-10-09T06:42:44.129 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.172288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106244.log 2023-10-09T06:42:44.129 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.172288.log.gz 2023-10-09T06:42:44.129 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.100295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171625.log 2023-10-09T06:42:44.129 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.106244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102371.log 2023-10-09T06:42:44.129 INFO:teuthology.orchestra.run.smithi163.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100295.log.gz 2023-10-09T06:42:44.129 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.171625.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106244.log.gzgzip 2023-10-09T06:42:44.130 INFO:teuthology.orchestra.run.smithi163.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.174529.log 2023-10-09T06:42:44.130 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.102371.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.171625.log.gz 2023-10-09T06:42:44.130 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172708.log 2023-10-09T06:42:44.130 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102371.log.gz 2023-10-09T06:42:44.130 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.174529.log: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-09T06:42:44.130 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96719.log 2023-10-09T06:42:44.130 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.172708.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.174529.log.gz 2023-10-09T06:42:44.130 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157279.log 2023-10-09T06:42:44.131 INFO:teuthology.orchestra.run.smithi163.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.172708.log.gz 2023-10-09T06:42:44.131 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.96719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160022.log 2023-10-09T06:42:44.131 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.157279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175385.log 2023-10-09T06:42:44.131 INFO:teuthology.orchestra.run.smithi163.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.96719.log.gz 2023-10-09T06:42:44.131 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.160022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108321.log 2023-10-09T06:42:44.131 INFO:teuthology.orchestra.run.smithi163.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.157279.log.gz 2023-10-09T06:42:44.131 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.175385.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.160022.log.gz 2023-10-09T06:42:44.131 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173358.log 2023-10-09T06:42:44.132 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.108321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99268.log 2023-10-09T06:42:44.132 INFO:teuthology.orchestra.run.smithi163.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.175385.log.gz 2023-10-09T06:42:44.132 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.173358.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.108321.log.gz 2023-10-09T06:42:44.132 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171262.log 2023-10-09T06:42:44.132 INFO:teuthology.orchestra.run.smithi163.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.173358.log.gz 2023-10-09T06:42:44.132 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.99268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99050.log 2023-10-09T06:42:44.132 INFO:teuthology.orchestra.run.smithi163.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99268.log.gz 2023-10-09T06:42:44.133 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.171262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.165473.log 2023-10-09T06:42:44.133 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101989.log 2023-10-09T06:42:44.133 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.99050.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.171262.log.gz 2023-10-09T06:42:44.133 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.0.165473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174176.log 2023-10-09T06:42:44.133 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.165473.log.gz 2023-10-09T06:42:44.133 INFO:teuthology.orchestra.run.smithi163.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.99050.log.gz 2023-10-09T06:42:44.133 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.101989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171052.log 2023-10-09T06:42:44.133 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.174176.log: gzip -5 87.9%--verbose -- /var/log/ceph/ceph-client.admin.101947.log -- replaced with /var/log/ceph/ceph-client.admin.101989.log.gz 2023-10-09T06:42:44.134 INFO:teuthology.orchestra.run.smithi163.stderr: 2023-10-09T06:42:44.134 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.171052.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.174176.log.gz 2023-10-09T06:42:44.134 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175301.log 2023-10-09T06:42:44.134 INFO:teuthology.orchestra.run.smithi163.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.171052.log.gz 2023-10-09T06:42:44.134 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.101947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.168165.log 2023-10-09T06:42:44.134 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.175301.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101947.log.gz 2023-10-09T06:42:44.134 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161976.log 2023-10-09T06:42:44.135 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.0.168165.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.175301.log.gz 2023-10-09T06:42:44.135 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose 0.0%-- /var/log/ceph/ceph-client.admin.105113.log -- replaced with /var/log/ceph/ceph-client.0.168165.log.gz 2023-10-09T06:42:44.135 INFO:teuthology.orchestra.run.smithi163.stderr: 2023-10-09T06:42:44.135 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.161976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.108095.log 2023-10-09T06:42:44.135 INFO:teuthology.orchestra.run.smithi163.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.161976.log.gz 2023-10-09T06:42:44.135 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.105113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171950.log 2023-10-09T06:42:44.135 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.0.108095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162325.log 2023-10-09T06:42:44.135 INFO:teuthology.orchestra.run.smithi163.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.105113.log.gz 2023-10-09T06:42:44.136 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.108095.log.gz 2023-10-09T06:42:44.136 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.171950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96676.log 2023-10-09T06:42:44.136 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.171950.log.gz 2023-10-09T06:42:44.136 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.162325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100674.log 2023-10-09T06:42:44.136 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.96676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107832.log 2023-10-09T06:42:44.136 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.162325.log.gz 2023-10-09T06:42:44.136 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7%/var/log/ceph/ceph-client.admin.100674.log: -- replaced with /var/log/ceph/ceph-client.admin.96676.log.gz 2023-10-09T06:42:44.136 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159489.log 2023-10-09T06:42:44.137 INFO:teuthology.orchestra.run.smithi163.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100674.log.gz 2023-10-09T06:42:44.137 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.107832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171872.log 2023-10-09T06:42:44.137 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.159489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104215.log 2023-10-09T06:42:44.137 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.171872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177224.log 2023-10-09T06:42:44.137 INFO:teuthology.orchestra.run.smithi163.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.159489.log.gz 2023-10-09T06:42:44.137 INFO:teuthology.orchestra.run.smithi163.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.107832.log.gz 2023-10-09T06:42:44.137 INFO:teuthology.orchestra.run.smithi163.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.171872.log.gz 2023-10-09T06:42:44.138 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.104215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180325.log 2023-10-09T06:42:44.138 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.177224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174302.log 2023-10-09T06:42:44.138 INFO:teuthology.orchestra.run.smithi163.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.104215.log.gz 2023-10-09T06:42:44.138 INFO:teuthology.orchestra.run.smithi163.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.177224.log.gz 2023-10-09T06:42:44.138 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.180325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102581.log 2023-10-09T06:42:44.138 INFO:teuthology.orchestra.run.smithi163.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.180325.log.gz 2023-10-09T06:42:44.138 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.174302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179271.log 2023-10-09T06:42:44.138 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.102581.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-10-09T06:42:44.139 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.174302.log.gz 2023-10-09T06:42:44.139 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.179271.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.102581.log.gz 2023-10-09T06:42:44.139 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157363.log 2023-10-09T06:42:44.139 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-mds.c.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.179271.log.gz 2023-10-09T06:42:44.139 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168742.log 2023-10-09T06:42:44.139 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.157363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166449.log 2023-10-09T06:42:44.139 INFO:teuthology.orchestra.run.smithi163.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.157363.log.gz 2023-10-09T06:42:44.139 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.168742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96803.log 2023-10-09T06:42:44.140 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.166449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173442.log 2023-10-09T06:42:44.140 INFO:teuthology.orchestra.run.smithi163.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.168742.log.gz 2023-10-09T06:42:44.140 INFO:teuthology.orchestra.run.smithi163.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.166449.log.gz 2023-10-09T06:42:44.140 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.96803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102157.log 2023-10-09T06:42:44.140 INFO:teuthology.orchestra.run.smithi163.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.96803.log.gz 2023-10-09T06:42:44.140 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.173442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156781.log 2023-10-09T06:42:44.140 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.102157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180157.log 2023-10-09T06:42:44.141 INFO:teuthology.orchestra.run.smithi163.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.173442.log.gz 2023-10-09T06:42:44.141 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102157.log.gz 2023-10-09T06:42:44.141 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.156781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174963.log 2023-10-09T06:42:44.141 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102115.log 2023-10-09T06:42:44.141 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.180157.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.156781.log.gz 2023-10-09T06:42:44.141 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.174963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156647.log 2023-10-09T06:42:44.141 INFO:teuthology.orchestra.run.smithi163.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.180157.log.gz 2023-10-09T06:42:44.142 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.102115.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.174963.log.gz 2023-10-09T06:42:44.142 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107706.log 2023-10-09T06:42:44.142 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.156647.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.102115.log.gz 2023-10-09T06:42:44.142 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177100.log 2023-10-09T06:42:44.142 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.107706.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.156647.log.gz 2023-10-09T06:42:44.142 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170359.log 2023-10-09T06:42:44.142 INFO:teuthology.orchestra.run.smithi163.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.107706.log.gz 2023-10-09T06:42:44.143 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.177100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104005.log 2023-10-09T06:42:44.143 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.170359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106664.log 2023-10-09T06:42:44.143 INFO:teuthology.orchestra.run.smithi163.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.177100.log.gz 2023-10-09T06:42:44.143 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.104005.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.170359.log.gz 2023-10-09T06:42:44.143 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91000.log 2023-10-09T06:42:44.143 INFO:teuthology.orchestra.run.smithi163.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104005.log.gz 2023-10-09T06:42:44.143 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.106664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159980.log 2023-10-09T06:42:44.144 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.91000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169844.log 2023-10-09T06:42:44.144 INFO:teuthology.orchestra.run.smithi163.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.106664.log.gz 2023-10-09T06:42:44.144 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.159980.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.91000.log.gz 2023-10-09T06:42:44.144 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162650.log 2023-10-09T06:42:44.144 INFO:teuthology.orchestra.run.smithi163.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.159980.log.gz 2023-10-09T06:42:44.144 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.169844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103229.log 2023-10-09T06:42:44.144 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.162650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180380.log 2023-10-09T06:42:44.145 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.169844.log.gz 2023-10-09T06:42:44.145 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.103229.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.162650.log.gz 2023-10-09T06:42:44.145 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103283.log 2023-10-09T06:42:44.145 INFO:teuthology.orchestra.run.smithi163.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.103229.log.gz 2023-10-09T06:42:44.145 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.180380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101325.log 2023-10-09T06:42:44.145 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.103283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167400.log 2023-10-09T06:42:44.145 INFO:teuthology.orchestra.run.smithi163.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.180380.log.gz 2023-10-09T06:42:44.145 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.101325.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.103283.log.gz 2023-10-09T06:42:44.146 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176565.log 2023-10-09T06:42:44.146 INFO:teuthology.orchestra.run.smithi163.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.101325.log.gz 2023-10-09T06:42:44.146 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.167400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97244.log 2023-10-09T06:42:44.146 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.176565.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.167400.log.gz 2023-10-09T06:42:44.146 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160498.log 2023-10-09T06:42:44.146 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.97244.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.176565.log.gz 2023-10-09T06:42:44.146 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160970.log 2023-10-09T06:42:44.146 INFO:teuthology.orchestra.run.smithi163.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.97244.log.gz 2023-10-09T06:42:44.147 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.160498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104605.log 2023-10-09T06:42:44.147 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.160970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174260.log 88.4% 2023-10-09T06:42:44.147 INFO:teuthology.orchestra.run.smithi163.stderr: -- replaced with /var/log/ceph/ceph-client.admin.160498.log.gz 2023-10-09T06:42:44.147 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.104605.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.160970.log.gz 2023-10-09T06:42:44.147 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177976.log 2023-10-09T06:42:44.147 INFO:teuthology.orchestra.run.smithi163.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.104605.log.gz 2023-10-09T06:42:44.147 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.174260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172816.log 2023-10-09T06:42:44.148 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.177976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104257.log 2023-10-09T06:42:44.148 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.174260.log.gz 2023-10-09T06:42:44.148 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.172816.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.177976.log.gz 2023-10-09T06:42:44.148 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176271.log 2023-10-09T06:42:44.148 INFO:teuthology.orchestra.run.smithi163.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.172816.log.gz 2023-10-09T06:42:44.148 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.104257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100632.log 2023-10-09T06:42:44.148 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.176271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174092.log 2023-10-09T06:42:44.149 INFO:teuthology.orchestra.run.smithi163.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.104257.log.gz 2023-10-09T06:42:44.149 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.100632.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.176271.log.gz 2023-10-09T06:42:44.149 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167035.log 2023-10-09T06:42:44.149 INFO:teuthology.orchestra.run.smithi163.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100632.log.gz 2023-10-09T06:42:44.149 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.174092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171793.log 2023-10-09T06:42:44.149 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.167035.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.174092.log.gz 2023-10-09T06:42:44.149 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159699.log 2023-10-09T06:42:44.149 INFO:teuthology.orchestra.run.smithi163.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.167035.log.gz 2023-10-09T06:42:44.150 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.171793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162902.log 2023-10-09T06:42:44.150 INFO:teuthology.orchestra.run.smithi163.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.171793.log.gz 2023-10-09T06:42:44.150 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.159699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166239.log 2023-10-09T06:42:44.150 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.162902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159938.log 2023-10-09T06:42:44.150 INFO:teuthology.orchestra.run.smithi163.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.159699.log.gz 2023-10-09T06:42:44.150 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.166239.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.162902.log.gz 2023-10-09T06:42:44.150 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105770.log 2023-10-09T06:42:44.151 INFO:teuthology.orchestra.run.smithi163.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.166239.log.gz 2023-10-09T06:42:44.151 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.159938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172666.log 2023-10-09T06:42:44.151 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.105770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161054.log 2023-10-09T06:42:44.151 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.159938.log.gz 2023-10-09T06:42:44.151 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.172666.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105770.log.gz 2023-10-09T06:42:44.151 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103095.log 2023-10-09T06:42:44.151 INFO:teuthology.orchestra.run.smithi163.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.172666.log.gz 2023-10-09T06:42:44.151 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.161054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159386.log 2023-10-09T06:42:44.152 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.103095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176439.log 2023-10-09T06:42:44.152 INFO:teuthology.orchestra.run.smithi163.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.161054.log.gz 2023-10-09T06:42:44.152 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.159386.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.103095.log.gz 2023-10-09T06:42:44.152 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161613.log 2023-10-09T06:42:44.152 INFO:teuthology.orchestra.run.smithi163.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.159386.log.gz 2023-10-09T06:42:44.152 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.176439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97076.log 2023-10-09T06:42:44.152 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.161613.log: 87.5%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.176439.log.gz--verbose 2023-10-09T06:42:44.153 INFO:teuthology.orchestra.run.smithi163.stderr:-- /var/log/ceph/ceph-client.admin.170443.log 2023-10-09T06:42:44.153 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.97076.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.161613.log.gz 2023-10-09T06:42:44.153 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171480.log 2023-10-09T06:42:44.153 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97076.log.gz 2023-10-09T06:42:44.153 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.170443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156991.log 2023-10-09T06:42:44.153 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.171480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104047.log 2023-10-09T06:42:44.153 INFO:teuthology.orchestra.run.smithi163.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.170443.log.gz 2023-10-09T06:42:44.153 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.156991.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.171480.log.gz 2023-10-09T06:42:44.154 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104987.log 2023-10-09T06:42:44.154 INFO:teuthology.orchestra.run.smithi163.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.156991.log.gz 2023-10-09T06:42:44.154 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.104047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.160925.log 2023-10-09T06:42:44.154 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.104987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171583.log 2023-10-09T06:42:44.154 INFO:teuthology.orchestra.run.smithi163.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104047.log.gz 2023-10-09T06:42:44.154 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.0.160925.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104987.log.gz 2023-10-09T06:42:44.154 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.160925.log.gz 2023-10-09T06:42:44.155 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104647.log 2023-10-09T06:42:44.155 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.171583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179000.log 2023-10-09T06:42:44.155 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.104647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99008.log 2023-10-09T06:42:44.155 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.171583.log.gz 2023-10-09T06:42:44.155 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.179000.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.104647.log.gz 2023-10-09T06:42:44.155 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179947.log 2023-10-09T06:42:44.155 INFO:teuthology.orchestra.run.smithi163.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.179000.log.gz 2023-10-09T06:42:44.155 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.99008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.105624.log 2023-10-09T06:42:44.156 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.179947.log: gzip 88.6%-5 -- replaced with /var/log/ceph/ceph-client.admin.99008.log.gz--verbose 2023-10-09T06:42:44.156 INFO:teuthology.orchestra.run.smithi163.stderr:-- /var/log/ceph/ceph-client.admin.158061.log 2023-10-09T06:42:44.156 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.0.105624.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.179947.log.gz 2023-10-09T06:42:44.156 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.105624.log.gz 2023-10-09T06:42:44.156 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160666.log 2023-10-09T06:42:44.156 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.158061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159531.log 2023-10-09T06:42:44.156 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.160666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100505.log 2023-10-09T06:42:44.157 INFO:teuthology.orchestra.run.smithi163.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.158061.log.gz 2023-10-09T06:42:44.157 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.159531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101862.log 2023-10-09T06:42:44.157 INFO:teuthology.orchestra.run.smithi163.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.159531.log.gz 2023-10-09T06:42:44.157 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.100505.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.160666.log.gz 2023-10-09T06:42:44.157 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156865.log 2023-10-09T06:42:44.157 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.101862.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100505.log.gz 2023-10-09T06:42:44.157 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164079.log 2023-10-09T06:42:44.157 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.156865.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101862.log.gz 2023-10-09T06:42:44.158 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166365.log 2023-10-09T06:42:44.158 INFO:teuthology.orchestra.run.smithi163.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.156865.log.gz 2023-10-09T06:42:44.158 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.164079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104089.log 2023-10-09T06:42:44.158 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.166365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97629.log 2023-10-09T06:42:44.158 INFO:teuthology.orchestra.run.smithi163.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.164079.log.gz 2023-10-09T06:42:44.158 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.104089.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.166365.log.gz 2023-10-09T06:42:44.158 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.165477.log 2023-10-09T06:42:44.159 INFO:teuthology.orchestra.run.smithi163.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.104089.log.gz 2023-10-09T06:42:44.159 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.97629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166071.log 2023-10-09T06:42:44.159 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.0.165477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165933.log 2023-10-09T06:42:44.159 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.165477.log.gz 2023-10-09T06:42:44.159 INFO:teuthology.orchestra.run.smithi163.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97629.log.gz 2023-10-09T06:42:44.159 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.166071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157135.log 2023-10-09T06:42:44.160 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.166071.log.gz 2023-10-09T06:42:44.160 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.165933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169366.log 2023-10-09T06:42:44.160 INFO:teuthology.orchestra.run.smithi163.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.165933.log.gz 2023-10-09T06:42:44.160 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164775.log 2023-10-09T06:42:44.160 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.157135.log: /var/log/ceph/ceph-client.admin.169366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96545.log 2023-10-09T06:42:44.160 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.157135.log.gz 2023-10-09T06:42:44.160 INFO:teuthology.orchestra.run.smithi163.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.169366.log.gz 2023-10-09T06:42:44.161 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.164775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175427.log 2023-10-09T06:42:44.161 INFO:teuthology.orchestra.run.smithi163.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.164775.log.gz 2023-10-09T06:42:44.161 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.96545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105812.log 2023-10-09T06:42:44.161 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.175427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170915.log 2023-10-09T06:42:44.161 INFO:teuthology.orchestra.run.smithi163.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.96545.log.gz 2023-10-09T06:42:44.161 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.175427.log.gz 2023-10-09T06:42:44.161 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.105812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172330.log 2023-10-09T06:42:44.162 INFO:teuthology.orchestra.run.smithi163.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.105812.log.gz/var/log/ceph/ceph-client.admin.170915.log: 2023-10-09T06:42:44.162 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172074.log 2023-10-09T06:42:44.162 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.172330.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.170915.log.gz 2023-10-09T06:42:44.162 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176649.log 2023-10-09T06:42:44.162 INFO:teuthology.orchestra.run.smithi163.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.172330.log.gz 2023-10-09T06:42:44.162 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.172074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158145.log 2023-10-09T06:42:44.162 INFO:teuthology.orchestra.run.smithi163.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.172074.log.gz 2023-10-09T06:42:44.163 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.176649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165518.log 2023-10-09T06:42:44.163 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.158145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100337.log 2023-10-09T06:42:44.163 INFO:teuthology.orchestra.run.smithi163.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.176649.log.gz 2023-10-09T06:42:44.163 INFO:teuthology.orchestra.run.smithi163.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.158145.log.gz 2023-10-09T06:42:44.163 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.165518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157753.log 2023-10-09T06:42:44.163 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.165518.log.gz 2023-10-09T06:42:44.163 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.100337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105155.log 2023-10-09T06:42:44.164 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.157753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173400.log 2023-10-09T06:42:44.164 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100337.log.gz 2023-10-09T06:42:44.164 INFO:teuthology.orchestra.run.smithi163.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.157753.log.gz 2023-10-09T06:42:44.164 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.105155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174919.log 2023-10-09T06:42:44.164 INFO:teuthology.orchestra.run.smithi163.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.105155.log.gz 2023-10-09T06:42:44.164 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.173400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100169.log 2023-10-09T06:42:44.165 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.174919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99092.log 2023-10-09T06:42:44.165 INFO:teuthology.orchestra.run.smithi163.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.173400.log.gz 2023-10-09T06:42:44.165 INFO:teuthology.orchestra.run.smithi163.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.174919.log.gz 2023-10-09T06:42:44.165 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.100169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167316.log 2023-10-09T06:42:44.165 INFO:teuthology.orchestra.run.smithi163.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100169.log.gz 2023-10-09T06:42:44.165 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.99092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177910.log 2023-10-09T06:42:44.165 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.167316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103669.log 2023-10-09T06:42:44.166 INFO:teuthology.orchestra.run.smithi163.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99092.log.gz 2023-10-09T06:42:44.166 INFO:teuthology.orchestra.run.smithi163.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.167316.log.gz 2023-10-09T06:42:44.166 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.177910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177826.log 2023-10-09T06:42:44.166 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.103669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157627.log 2023-10-09T06:42:44.166 INFO:teuthology.orchestra.run.smithi163.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.177910.log.gz 2023-10-09T06:42:44.166 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.177826.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103669.log.gz 2023-10-09T06:42:44.166 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96931.log 2023-10-09T06:42:44.167 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169894.log 2023-10-09T06:42:44.167 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.157627.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.177826.log.gz 2023-10-09T06:42:44.167 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.96931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166644.log 2023-10-09T06:42:44.167 INFO:teuthology.orchestra.run.smithi163.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.157627.log.gz 2023-10-09T06:42:44.167 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.169894.log: gzip -5 88.8%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.96931.log.gz/var/log/ceph/ceph-client.admin.101820.log 2023-10-09T06:42:44.167 INFO:teuthology.orchestra.run.smithi163.stderr: 2023-10-09T06:42:44.168 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.166644.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.169894.log.gz 2023-10-09T06:42:44.168 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163891.log 2023-10-09T06:42:44.168 INFO:teuthology.orchestra.run.smithi163.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.166644.log.gz 2023-10-09T06:42:44.168 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.101820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179229.log 2023-10-09T06:42:44.168 INFO:teuthology.orchestra.run.smithi163.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.101820.log.gz 2023-10-09T06:42:44.168 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.163891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100926.log 2023-10-09T06:42:44.169 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.179229.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.163891.log.gz 2023-10-09T06:42:44.169 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157883.log 2023-10-09T06:42:44.169 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.179229.log.gz 2023-10-09T06:42:44.169 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.100926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102073.log 2023-10-09T06:42:44.169 INFO:teuthology.orchestra.run.smithi163.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.100926.log.gz 2023-10-09T06:42:44.169 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.157883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106496.log 2023-10-09T06:42:44.170 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.102073.log: gzip -5 --verbose -- 88.7%/var/log/ceph/ceph-client.admin.106928.log 2023-10-09T06:42:44.170 INFO:teuthology.orchestra.run.smithi163.stderr: -- replaced with /var/log/ceph/ceph-client.admin.157883.log.gz 2023-10-09T06:42:44.170 INFO:teuthology.orchestra.run.smithi163.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.102073.log.gz 2023-10-09T06:42:44.170 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.106496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103795.log 2023-10-09T06:42:44.170 INFO:teuthology.orchestra.run.smithi163.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.106496.log.gz 2023-10-09T06:42:44.170 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.106928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166993.log 2023-10-09T06:42:44.170 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.103795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100253.log 2023-10-09T06:42:44.170 INFO:teuthology.orchestra.run.smithi163.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.106928.log.gz 2023-10-09T06:42:44.171 INFO:teuthology.orchestra.run.smithi163.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103795.log.gz 2023-10-09T06:42:44.171 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.166993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102707.log 2023-10-09T06:42:44.171 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.166993.log.gz 2023-10-09T06:42:44.171 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.100253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90864.log 2023-10-09T06:42:44.171 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.102707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106580.log 2023-10-09T06:42:44.171 INFO:teuthology.orchestra.run.smithi163.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100253.log.gz 2023-10-09T06:42:44.171 INFO:teuthology.orchestra.run.smithi163.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.102707.log.gz 2023-10-09T06:42:44.172 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.90864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170023.log 2023-10-09T06:42:44.172 INFO:teuthology.orchestra.run.smithi163.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.90864.log.gz 2023-10-09T06:42:44.172 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.106580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165113.log 2023-10-09T06:42:44.172 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.170023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160286.log 2023-10-09T06:42:44.172 INFO:teuthology.orchestra.run.smithi163.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.106580.log.gz 2023-10-09T06:42:44.172 INFO:teuthology.orchestra.run.smithi163.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.170023.log.gz 2023-10-09T06:42:44.172 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.165113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97965.log 2023-10-09T06:42:44.173 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8%/var/log/ceph/ceph-client.admin.160286.log: -- replaced with /var/log/ceph/ceph-client.admin.165113.log.gz 2023-10-09T06:42:44.173 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177530.log 2023-10-09T06:42:44.173 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.97965.log: gzip 88.9%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.160286.log.gz-- 2023-10-09T06:42:44.173 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.163416.log 2023-10-09T06:42:44.173 INFO:teuthology.orchestra.run.smithi163.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97965.log.gz 2023-10-09T06:42:44.173 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.177530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105029.log 2023-10-09T06:42:44.173 INFO:teuthology.orchestra.run.smithi163.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.177530.log.gz 2023-10-09T06:42:44.173 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.163416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168658.log 2023-10-09T06:42:44.174 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.105029.log: gzip 88.4%-5 -- replaced with /var/log/ceph/ceph-client.admin.163416.log.gz--verbose 2023-10-09T06:42:44.174 INFO:teuthology.orchestra.run.smithi163.stderr:-- /var/log/ceph/ceph-client.admin.168994.log 2023-10-09T06:42:44.174 INFO:teuthology.orchestra.run.smithi163.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.105029.log.gz 2023-10-09T06:42:44.174 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.168658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163807.log 2023-10-09T06:42:44.174 INFO:teuthology.orchestra.run.smithi163.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.168658.log.gz 2023-10-09T06:42:44.174 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.168994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.163203.log 2023-10-09T06:42:44.174 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.163807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160194.log 2023-10-09T06:42:44.174 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.168994.log.gz 2023-10-09T06:42:44.175 INFO:teuthology.orchestra.run.smithi163.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.163807.log.gz 2023-10-09T06:42:44.175 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.0.163203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108195.log 2023-10-09T06:42:44.175 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.163203.log.gz 2023-10-09T06:42:44.175 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.160194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160624.log 2023-10-09T06:42:44.175 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.108195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176125.log 2023-10-09T06:42:44.175 INFO:teuthology.orchestra.run.smithi163.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.160194.log.gz 2023-10-09T06:42:44.175 INFO:teuthology.orchestra.run.smithi163.stderr: 89.0%/var/log/ceph/ceph-client.admin.160624.log: -- replaced with /var/log/ceph/ceph-client.admin.108195.log.gz 2023-10-09T06:42:44.176 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.175728.log 2023-10-09T06:42:44.176 INFO:teuthology.orchestra.run.smithi163.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.160624.log.gz 2023-10-09T06:42:44.176 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.176125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165728.log 2023-10-09T06:42:44.176 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.0.175728.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.175728.log.gz 2023-10-09T06:42:44.176 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173568.log 2023-10-09T06:42:44.176 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.176125.log.gz 2023-10-09T06:42:44.176 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.165728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166323.log 2023-10-09T06:42:44.176 INFO:teuthology.orchestra.run.smithi163.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.165728.log.gz 2023-10-09T06:42:44.177 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.173568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108237.log 2023-10-09T06:42:44.177 INFO:teuthology.orchestra.run.smithi163.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.173568.log.gz 2023-10-09T06:42:44.177 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.166323.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-10-09T06:42:44.177 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.108237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174403.log 2023-10-09T06:42:44.177 INFO:teuthology.orchestra.run.smithi163.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.166323.log.gz 2023-10-09T06:42:44.177 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-osd.3.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.108237.log.gz 2023-10-09T06:42:44.177 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107012.log 2023-10-09T06:42:44.177 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.174403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98924.log 2023-10-09T06:42:44.178 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.107012.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.174403.log.gz 2023-10-09T06:42:44.178 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162692.log 2023-10-09T06:42:44.178 INFO:teuthology.orchestra.run.smithi163.stderr:gzip /var/log/ceph/ceph-client.admin.98924.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.165782.log 2023-10-09T06:42:44.178 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.107012.log.gz 2023-10-09T06:42:44.178 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.162692.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.98924.log.gz 2023-10-09T06:42:44.178 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107656.log 2023-10-09T06:42:44.178 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.162692.log.gz 2023-10-09T06:42:44.178 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-09T06:42:44.179 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.165782.log: /var/log/ceph/ceph-client.admin.107656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100716.log 2023-10-09T06:42:44.179 INFO:teuthology.orchestra.run.smithi163.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.165782.log.gz 2023-10-09T06:42:44.179 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-osd.1.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107656.log.gz 2023-10-09T06:42:44.179 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163993.log 2023-10-09T06:42:44.179 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.100716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173756.log 2023-10-09T06:42:44.179 INFO:teuthology.orchestra.run.smithi163.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100716.log.gz 2023-10-09T06:42:44.179 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.163993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161884.log 2023-10-09T06:42:44.180 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.173756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98301.log 2023-10-09T06:42:44.180 INFO:teuthology.orchestra.run.smithi163.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.163993.log.gz 2023-10-09T06:42:44.180 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.161884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161361.log 2023-10-09T06:42:44.180 INFO:teuthology.orchestra.run.smithi163.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.173756.log.gz 2023-10-09T06:42:44.180 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.98301.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.161884.log.gz 2023-10-09T06:42:44.180 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159092.log 2023-10-09T06:42:44.180 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172414.log 2023-10-09T06:42:44.180 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.161361.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.98301.log.gz 2023-10-09T06:42:44.181 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.159092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98756.log 2023-10-09T06:42:44.181 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.161361.log.gz 2023-10-09T06:42:44.181 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.172414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97454.log 2023-10-09T06:42:44.181 INFO:teuthology.orchestra.run.smithi163.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.159092.log.gz 2023-10-09T06:42:44.181 INFO:teuthology.orchestra.run.smithi163.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.172414.log.gz 2023-10-09T06:42:44.181 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.98756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164681.log 2023-10-09T06:42:44.181 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168512.log 2023-10-09T06:42:44.181 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.97454.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98756.log.gz 2023-10-09T06:42:44.182 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.164681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107488.log 2023-10-09T06:42:44.182 INFO:teuthology.orchestra.run.smithi163.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97454.log.gz 2023-10-09T06:42:44.182 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.168512.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.164681.log.gz 2023-10-09T06:42:44.182 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177658.log 2023-10-09T06:42:44.182 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.107488.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.168512.log.gz 2023-10-09T06:42:44.182 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160456.log 2023-10-09T06:42:44.182 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.177658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.173013.log 2023-10-09T06:42:44.182 INFO:teuthology.orchestra.run.smithi163.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.107488.log.gz 2023-10-09T06:42:44.183 INFO:teuthology.orchestra.run.smithi163.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.177658.log.gz 2023-10-09T06:42:44.183 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.160456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161571.log 2023-10-09T06:42:44.183 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169756.log 2023-10-09T06:42:44.183 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.0.173013.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.160456.log.gz 2023-10-09T06:42:44.183 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.173013.log.gz 2023-10-09T06:42:44.183 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.161571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168868.log 2023-10-09T06:42:44.183 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.169756.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.161571.log.gz 2023-10-09T06:42:44.185 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99134.log 2023-10-09T06:42:44.185 INFO:teuthology.orchestra.run.smithi163.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.169756.log.gz 2023-10-09T06:42:44.185 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.168868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165280.log 2023-10-09T06:42:44.185 INFO:teuthology.orchestra.run.smithi163.stderr:gzip /var/log/ceph/ceph-client.admin.99134.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.170065.log 2023-10-09T06:42:44.185 INFO:teuthology.orchestra.run.smithi163.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.168868.log.gz 2023-10-09T06:42:44.185 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.165280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165872.log 2023-10-09T06:42:44.186 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.170065.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.165280.log.gz 2023-10-09T06:42:44.186 INFO:teuthology.orchestra.run.smithi163.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.99134.log.gz 2023-10-09T06:42:44.186 INFO:teuthology.orchestra.run.smithi163.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.170065.log.gz 2023-10-09T06:42:44.186 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.165872.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.165872.log.gz 2023-10-09T06:42:44.186 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168428.log 2023-10-09T06:42:44.186 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169307.log 2023-10-09T06:42:44.186 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.168428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100548.log 2023-10-09T06:42:44.187 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.169307.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.168428.log.gz 2023-10-09T06:42:44.187 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156550.log 2023-10-09T06:42:44.187 INFO:teuthology.orchestra.run.smithi163.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.169307.log.gz 2023-10-09T06:42:44.187 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.100548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173054.log 2023-10-09T06:42:44.187 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172540.log 2023-10-09T06:42:44.187 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.156550.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100548.log.gz 2023-10-09T06:42:44.187 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.173054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163290.log 2023-10-09T06:42:44.187 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.156550.log.gz 2023-10-09T06:42:44.188 INFO:teuthology.orchestra.run.smithi163.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.173054.log.gz 2023-10-09T06:42:44.188 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.172540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169223.log 2023-10-09T06:42:44.188 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170191.log 2023-10-09T06:42:44.188 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.163290.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.172540.log.gz 2023-10-09T06:42:44.188 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.169223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98296.log 2023-10-09T06:42:44.188 INFO:teuthology.orchestra.run.smithi163.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.163290.log.gz 2023-10-09T06:42:44.188 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.170191.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.169223.log.gz 2023-10-09T06:42:44.188 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164859.log 2023-10-09T06:42:44.189 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.98296.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.170191.log.gz 2023-10-09T06:42:44.189 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158743.log 2023-10-09T06:42:44.189 INFO:teuthology.orchestra.run.smithi163.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.98296.log.gz 2023-10-09T06:42:44.189 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.164859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98630.log 2023-10-09T06:42:44.189 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.158743.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.164859.log.gz 2023-10-09T06:42:44.189 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178706.log 2023-10-09T06:42:44.189 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.98630.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.158743.log.gz 2023-10-09T06:42:44.190 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.99673.log 2023-10-09T06:42:44.190 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.178706.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.98630.log.gzgzip 2023-10-09T06:42:44.190 INFO:teuthology.orchestra.run.smithi163.stderr:-5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-09T06:42:44.190 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.0.99673.log: 0.0% 88.9% -- replaced with /var/log/ceph/ceph-client.0.99673.log.gz -- replaced with /var/log/ceph/ceph-client.admin.178706.log.gz 2023-10-09T06:42:44.190 INFO:teuthology.orchestra.run.smithi163.stderr: 2023-10-09T06:42:44.190 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161138.log 2023-10-09T06:42:44.190 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168910.log 2023-10-09T06:42:44.191 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.161138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104397.log 2023-10-09T06:42:44.191 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.168910.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.161138.log.gz 2023-10-09T06:42:44.191 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178300.log 2023-10-09T06:42:44.191 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.104397.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.168910.log.gz 2023-10-09T06:42:44.191 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102287.log 2023-10-09T06:42:44.191 INFO:teuthology.orchestra.run.smithi163.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.104397.log.gz 2023-10-09T06:42:44.191 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.178300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158103.log 2023-10-09T06:42:44.192 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.102287.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.178300.log.gz 2023-10-09T06:42:44.192 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102287.log.gz 2023-10-09T06:42:44.192 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.158103.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.158103.log.gz 2023-10-09T06:42:44.192 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103325.log 2023-10-09T06:42:44.192 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167822.log 2023-10-09T06:42:44.192 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.103325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174751.log 2023-10-09T06:42:44.192 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103325.log.gz 2023-10-09T06:42:44.193 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.167822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175535.log 2023-10-09T06:42:44.193 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.174751.log: gzip 88.8%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.167822.log.gz-- 2023-10-09T06:42:44.193 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.168210.log 2023-10-09T06:42:44.193 INFO:teuthology.orchestra.run.smithi163.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.174751.log.gz 2023-10-09T06:42:44.193 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.175535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97618.log 2023-10-09T06:42:44.193 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.168210.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.175535.log.gz 2023-10-09T06:42:44.194 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.97618.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.168210.log.gz 2023-10-09T06:42:44.194 INFO:teuthology.orchestra.run.smithi163.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.97618.log.gz 2023-10-09T06:42:44.194 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157033.log 2023-10-09T06:42:44.194 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165560.log 2023-10-09T06:42:44.194 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.157033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162944.log 2023-10-09T06:42:44.194 INFO:teuthology.orchestra.run.smithi163.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.157033.log.gz 2023-10-09T06:42:44.194 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.165560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107096.log 2023-10-09T06:42:44.195 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163933.log 2023-10-09T06:42:44.195 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.162944.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.165560.log.gz 2023-10-09T06:42:44.195 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.107096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104479.log 2023-10-09T06:42:44.195 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.163933.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.107096.log.gz 2023-10-09T06:42:44.195 INFO:teuthology.orchestra.run.smithi163.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.162944.log.gz 2023-10-09T06:42:44.195 INFO:teuthology.orchestra.run.smithi163.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.163933.log.gz 2023-10-09T06:42:44.195 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.104479.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.104479.log.gz 2023-10-09T06:42:44.195 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158966.log 2023-10-09T06:42:44.196 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163849.log 2023-10-09T06:42:44.196 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.158966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159657.log 2023-10-09T06:42:44.196 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.163849.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.158966.log.gz 2023-10-09T06:42:44.197 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.163849.log.gz 2023-10-09T06:42:44.197 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.159657.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.159657.log.gz 2023-10-09T06:42:44.197 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106412.log 2023-10-09T06:42:44.197 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174218.log 2023-10-09T06:42:44.197 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.106412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161487.log 2023-10-09T06:42:44.197 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.174218.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106412.log.gz 2023-10-09T06:42:44.197 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.174218.log.gz 2023-10-09T06:42:44.197 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.161487.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.161487.log.gz 2023-10-09T06:42:44.198 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107318.log 2023-10-09T06:42:44.198 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105669.log 2023-10-09T06:42:44.198 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.107318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167486.log 2023-10-09T06:42:44.198 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.105669.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107318.log.gz 2023-10-09T06:42:44.198 INFO:teuthology.orchestra.run.smithi163.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.105669.log.gz 2023-10-09T06:42:44.198 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.167486.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.167486.log.gz 2023-10-09T06:42:44.198 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173966.log 2023-10-09T06:42:44.199 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179426.log 2023-10-09T06:42:44.199 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.173966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175049.log 2023-10-09T06:42:44.199 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.179426.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.173966.log.gz 2023-10-09T06:42:44.199 INFO:teuthology.orchestra.run.smithi163.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.179426.log.gz 2023-10-09T06:42:44.199 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.175049.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.175049.log.gz 2023-10-09T06:42:44.199 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175259.log 2023-10-09T06:42:44.199 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172244.log 2023-10-09T06:42:44.199 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.175259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103837.log 2023-10-09T06:42:44.200 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.172244.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.175259.log.gz 2023-10-09T06:42:44.200 INFO:teuthology.orchestra.run.smithi163.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.172244.log.gz 2023-10-09T06:42:44.200 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.103837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177266.log 2023-10-09T06:42:44.200 INFO:teuthology.orchestra.run.smithi163.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.103837.log.gz 2023-10-09T06:42:44.200 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161758.log 2023-10-09T06:42:44.200 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.177266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177308.log 2023-10-09T06:42:44.200 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.161758.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.177266.log.gz 2023-10-09T06:42:44.200 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.177308.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.161758.log.gz 2023-10-09T06:42:44.201 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.177308.log.gz 2023-10-09T06:42:44.201 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179863.log 2023-10-09T06:42:44.201 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159344.log 2023-10-09T06:42:44.201 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.179863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173652.log 2023-10-09T06:42:44.201 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.159344.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.179863.log.gz 2023-10-09T06:42:44.201 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.159344.log.gz 2023-10-09T06:42:44.201 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.173652.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.173652.log.gz 2023-10-09T06:42:44.201 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175857.log 2023-10-09T06:42:44.202 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98882.log 2023-10-09T06:42:44.202 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.175857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106328.log 2023-10-09T06:42:44.202 INFO:teuthology.orchestra.run.smithi163.stderr: 87.3%/var/log/ceph/ceph-client.admin.98882.log: -- replaced with /var/log/ceph/ceph-client.admin.175857.log.gz 2023-10-09T06:42:44.202 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98882.log.gz 2023-10-09T06:42:44.202 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.106328.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.106328.log.gz 2023-10-09T06:42:44.202 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171354.log 2023-10-09T06:42:44.202 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179905.log 2023-10-09T06:42:44.203 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.171354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160540.log 2023-10-09T06:42:44.203 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.179905.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.171354.log.gz 2023-10-09T06:42:44.203 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.179905.log.gz 2023-10-09T06:42:44.203 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.160540.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.160540.log.gz 2023-10-09T06:42:44.203 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173526.log 2023-10-09T06:42:44.203 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171709.log 2023-10-09T06:42:44.203 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.173526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101367.log 2023-10-09T06:42:44.203 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.171709.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.173526.log.gz 2023-10-09T06:42:44.204 INFO:teuthology.orchestra.run.smithi163.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.171709.log.gz 2023-10-09T06:42:44.204 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.101367.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.101367.log.gz 2023-10-09T06:42:44.204 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176960.log 2023-10-09T06:42:44.204 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164469.log 2023-10-09T06:42:44.204 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.176960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103053.log 2023-10-09T06:42:44.204 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.164469.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.176960.log.gz 2023-10-09T06:42:44.204 INFO:teuthology.orchestra.run.smithi163.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.164469.log.gz 2023-10-09T06:42:44.205 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.103053.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.103053.log.gz 2023-10-09T06:42:44.205 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97619.log 2023-10-09T06:42:44.205 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102329.log 2023-10-09T06:42:44.205 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.97619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103585.log 2023-10-09T06:42:44.205 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.102329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170551.log 88.2% 2023-10-09T06:42:44.205 INFO:teuthology.orchestra.run.smithi163.stderr: -- replaced with /var/log/ceph/ceph-client.admin.97619.log.gz 2023-10-09T06:42:44.205 INFO:teuthology.orchestra.run.smithi163.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102329.log.gz 2023-10-09T06:42:44.205 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.103585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108363.log 2023-10-09T06:42:44.206 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105854.log 2023-10-09T06:42:44.206 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.170551.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103585.log.gz 2023-10-09T06:42:44.206 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.108363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179594.log 2023-10-09T06:42:44.206 INFO:teuthology.orchestra.run.smithi163.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.170551.log.gz 2023-10-09T06:42:44.206 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.105854.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108363.log.gz 2023-10-09T06:42:44.206 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180241.log 2023-10-09T06:42:44.206 INFO:teuthology.orchestra.run.smithi163.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.105854.log.gz 2023-10-09T06:42:44.207 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.179594.log: /var/log/ceph/ceph-client.admin.180241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168294.log 2023-10-09T06:42:44.207 INFO:teuthology.orchestra.run.smithi163.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.179594.log.gz 2023-10-09T06:42:44.207 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176775.log 2023-10-09T06:42:44.207 INFO:teuthology.orchestra.run.smithi163.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.180241.log.gz 2023-10-09T06:42:44.207 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.168294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158271.log 2023-10-09T06:42:44.207 INFO:teuthology.orchestra.run.smithi163.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.168294.log.gz 2023-10-09T06:42:44.207 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.176775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164553.log 2023-10-09T06:42:44.207 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102815.log 2023-10-09T06:42:44.208 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.158271.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.176775.log.gz 2023-10-09T06:42:44.208 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.164553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168952.log 2023-10-09T06:42:44.208 INFO:teuthology.orchestra.run.smithi163.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.158271.log.gz 2023-10-09T06:42:44.208 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.102815.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.164553.log.gz 2023-10-09T06:42:44.208 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161842.log 2023-10-09T06:42:44.208 INFO:teuthology.orchestra.run.smithi163.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.102815.log.gz 2023-10-09T06:42:44.208 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.168952.log: /var/log/ceph/ceph-client.admin.161842.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.168952.log.gz 2023-10-09T06:42:44.208 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.161842.log.gz 2023-10-09T06:42:44.209 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101535.log 2023-10-09T06:42:44.209 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172498.log 2023-10-09T06:42:44.209 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.101535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176733.log 2023-10-09T06:42:44.209 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.172498.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101535.log.gz 2023-10-09T06:42:44.209 INFO:teuthology.orchestra.run.smithi163.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.172498.log.gz 2023-10-09T06:42:44.209 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.176733.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.176733.log.gz 2023-10-09T06:42:44.209 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160414.log 2023-10-09T06:42:44.209 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167222.log 2023-10-09T06:42:44.210 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.160414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179821.log 2023-10-09T06:42:44.210 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.167222.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.160414.log.gz 2023-10-09T06:42:44.210 INFO:teuthology.orchestra.run.smithi163.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.167222.log.gz 2023-10-09T06:42:44.210 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.179821.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.179821.log.gz 2023-10-09T06:42:44.210 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107140.log 2023-10-09T06:42:44.210 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163513.log 2023-10-09T06:42:44.210 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.107140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178958.log 2023-10-09T06:42:44.211 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.163513.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107140.log.gz 2023-10-09T06:42:44.211 INFO:teuthology.orchestra.run.smithi163.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.163513.log.gz 2023-10-09T06:42:44.211 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.178958.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.178958.log.gz 2023-10-09T06:42:44.211 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175343.log 2023-10-09T06:42:44.211 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102243.log 2023-10-09T06:42:44.211 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.175343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167444.log 2023-10-09T06:42:44.211 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.102243.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.175343.log.gz 2023-10-09T06:42:44.211 INFO:teuthology.orchestra.run.smithi163.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102243.log.gz 2023-10-09T06:42:44.212 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.167444.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.167444.log.gz 2023-10-09T06:42:44.212 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168826.log 2023-10-09T06:42:44.212 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.103012.log 2023-10-09T06:42:44.212 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.168826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107572.log 2023-10-09T06:42:44.212 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.0.103012.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.168826.log.gz 2023-10-09T06:42:44.212 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.103012.log.gz 2023-10-09T06:42:44.212 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.107572.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.107572.log.gz 2023-10-09T06:42:44.212 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178790.log 2023-10-09T06:42:44.213 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166281.log 2023-10-09T06:42:44.213 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.178790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164121.log 2023-10-09T06:42:44.213 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.166281.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.178790.log.gz 2023-10-09T06:42:44.213 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.164121.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.166281.log.gz 2023-10-09T06:42:44.213 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98714.log 2023-10-09T06:42:44.213 INFO:teuthology.orchestra.run.smithi163.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.164121.log.gz 2023-10-09T06:42:44.213 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175815.log 2023-10-09T06:42:44.213 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.98714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176607.log 2023-10-09T06:42:44.214 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.175815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172624.log 2023-10-09T06:42:44.214 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98714.log.gz 2023-10-09T06:42:44.214 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.175815.log.gz 2023-10-09T06:42:44.214 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.176607.log: /var/log/ceph/ceph-client.admin.172624.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.176607.log.gz 2023-10-09T06:42:44.214 INFO:teuthology.orchestra.run.smithi163.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.172624.log.gz 2023-10-09T06:42:44.214 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174344.log 2023-10-09T06:42:44.214 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167972.log 2023-10-09T06:42:44.215 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.174344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106706.log 2023-10-09T06:42:44.215 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.167972.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.174344.log.gz 2023-10-09T06:42:44.215 INFO:teuthology.orchestra.run.smithi163.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.167972.log.gz 2023-10-09T06:42:44.215 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.106706.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106706.log.gz 2023-10-09T06:42:44.215 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174793.log 2023-10-09T06:42:44.215 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162606.log 2023-10-09T06:42:44.215 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.174793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168252.log 2023-10-09T06:42:44.216 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.162606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162734.log 2023-10-09T06:42:44.216 INFO:teuthology.orchestra.run.smithi163.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.174793.log.gz 2023-10-09T06:42:44.216 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.168252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97628.log 2023-10-09T06:42:44.216 INFO:teuthology.orchestra.run.smithi163.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.162606.log.gz 2023-10-09T06:42:44.216 INFO:teuthology.orchestra.run.smithi163.stderr:gzip /var/log/ceph/ceph-client.admin.162734.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.102455.log 2023-10-09T06:42:44.216 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.168252.log.gz 2023-10-09T06:42:44.216 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.97628.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.162734.log.gz 2023-10-09T06:42:44.216 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104819.log 2023-10-09T06:42:44.217 INFO:teuthology.orchestra.run.smithi163.stderr: 87.5%/var/log/ceph/ceph-client.admin.102455.log: -- replaced with /var/log/ceph/ceph-client.admin.97628.log.gz 2023-10-09T06:42:44.217 INFO:teuthology.orchestra.run.smithi163.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102455.log.gz 2023-10-09T06:42:44.217 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.104819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169036.log 2023-10-09T06:42:44.217 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104819.log.gz 2023-10-09T06:42:44.217 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165214.log 2023-10-09T06:42:44.217 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.169036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167696.log 2023-10-09T06:42:44.217 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.165214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101073.log 2023-10-09T06:42:44.217 INFO:teuthology.orchestra.run.smithi163.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.169036.log.gz 2023-10-09T06:42:44.218 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.167696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156823.log 2023-10-09T06:42:44.218 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.101073.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.167696.log.gz 2023-10-09T06:42:44.218 INFO:teuthology.orchestra.run.smithi163.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.165214.log.gz 2023-10-09T06:42:44.218 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.156823.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101073.log.gz 2023-10-09T06:42:44.218 INFO:teuthology.orchestra.run.smithi163.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.156823.log.gz 2023-10-09T06:42:44.218 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157933.log 2023-10-09T06:42:44.218 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177700.log 2023-10-09T06:42:44.219 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.157933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167358.log 2023-10-09T06:42:44.219 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.177700.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.157933.log.gz 2023-10-09T06:42:44.219 INFO:teuthology.orchestra.run.smithi163.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.177700.log.gz 2023-10-09T06:42:44.219 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.167358.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.167358.log.gz 2023-10-09T06:42:44.219 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168700.log 2023-10-09T06:42:44.219 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179695.log 2023-10-09T06:42:44.289 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.168700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179779.log 2023-10-09T06:42:44.289 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.179695.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.168700.log.gz 2023-10-09T06:42:44.289 INFO:teuthology.orchestra.run.smithi163.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.179695.log.gz 2023-10-09T06:42:44.289 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.179779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167077.log 2023-10-09T06:42:44.289 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.179779.log.gz 2023-10-09T06:42:44.289 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105728.log 2023-10-09T06:42:44.290 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.167077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96845.log 2023-10-09T06:42:44.290 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.167077.log.gz 2023-10-09T06:42:44.290 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.105728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169265.log 2023-10-09T06:42:44.290 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175091.log 2023-10-09T06:42:44.290 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.96845.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.105728.log.gz 2023-10-09T06:42:44.290 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.169265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156949.log 2023-10-09T06:42:44.290 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96845.log.gz 2023-10-09T06:42:44.290 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.175091.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.169265.log.gz 2023-10-09T06:42:44.291 INFO:teuthology.orchestra.run.smithi163.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.175091.log.gz 2023-10-09T06:42:44.291 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.156949.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.156949.log.gz 2023-10-09T06:42:44.291 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-09T06:42:44.291 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162455.log 2023-10-09T06:42:44.291 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174835.log 2023-10-09T06:42:44.291 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.162455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179145.log 2023-10-09T06:42:44.291 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.174835.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.162455.log.gz 2023-10-09T06:42:44.292 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.179145.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.174835.log.gz 2023-10-09T06:42:44.292 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.179145.log.gz 2023-10-09T06:42:44.292 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165029.log 2023-10-09T06:42:44.292 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104945.log 2023-10-09T06:42:44.292 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.165029.log: /var/log/ceph/ceph-client.admin.104945.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.165029.log.gz 2023-10-09T06:42:44.292 INFO:teuthology.orchestra.run.smithi163.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.104945.log.gz 2023-10-09T06:42:44.292 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161800.log 2023-10-09T06:42:44.292 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166833.log 2023-10-09T06:42:44.293 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.161800.log: /var/log/ceph/ceph-client.admin.166833.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.161800.log.gz 2023-10-09T06:42:44.293 INFO:teuthology.orchestra.run.smithi163.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.166833.log.gz 2023-10-09T06:42:44.293 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176918.log 2023-10-09T06:42:44.293 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97202.log 2023-10-09T06:42:44.293 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.176918.log: /var/log/ceph/ceph-client.admin.97202.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.176918.log.gz 2023-10-09T06:42:44.293 INFO:teuthology.orchestra.run.smithi163.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.97202.log.gz 2023-10-09T06:42:44.293 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98300.log 2023-10-09T06:42:44.293 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168616.log 2023-10-09T06:42:44.294 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.98300.log: /var/log/ceph/ceph-client.admin.168616.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98300.log.gz 2023-10-09T06:42:44.294 INFO:teuthology.orchestra.run.smithi163.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.168616.log.gz 2023-10-09T06:42:44.294 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178622.log 2023-10-09T06:42:44.294 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102539.log 2023-10-09T06:42:44.294 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.178622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160732.log 2023-10-09T06:42:44.294 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.102539.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.178622.log.gz 2023-10-09T06:42:44.294 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.160732.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.102539.log.gz 2023-10-09T06:42:44.294 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.160732.log.gz 2023-10-09T06:42:44.295 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177488.log 2023-10-09T06:42:44.295 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105323.log 2023-10-09T06:42:44.295 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.177488.log: /var/log/ceph/ceph-client.admin.105323.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.177488.log.gz 2023-10-09T06:42:44.295 INFO:teuthology.orchestra.run.smithi163.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.105323.log.gz 2023-10-09T06:42:44.295 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177742.log 2023-10-09T06:42:44.295 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106538.log 2023-10-09T06:42:44.295 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.177742.log: /var/log/ceph/ceph-client.admin.106538.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.177742.log.gz 2023-10-09T06:42:44.295 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106538.log.gz 2023-10-09T06:42:44.296 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169139.log 2023-10-09T06:42:44.296 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100463.log 2023-10-09T06:42:44.296 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.169139.log: /var/log/ceph/ceph-client.admin.100463.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.169139.log.gz 2023-10-09T06:42:44.296 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100463.log.gz 2023-10-09T06:42:44.296 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170789.log 2023-10-09T06:42:44.296 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171438.log 2023-10-09T06:42:44.296 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.170789.log: /var/log/ceph/ceph-client.admin.171438.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.170789.log.gz 2023-10-09T06:42:44.297 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.171438.log.gz 2023-10-09T06:42:44.297 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166509.log 2023-10-09T06:42:44.297 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177394.log 2023-10-09T06:42:44.297 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.166509.log: /var/log/ceph/ceph-client.admin.177394.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.166509.log.gz 2023-10-09T06:42:44.297 INFO:teuthology.orchestra.run.smithi163.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.177394.log.gz 2023-10-09T06:42:44.297 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178258.log 2023-10-09T06:42:44.297 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101493.log 2023-10-09T06:42:44.297 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.178258.log: /var/log/ceph/ceph-client.admin.101493.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.178258.log.gz 2023-10-09T06:42:44.298 INFO:teuthology.orchestra.run.smithi163.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101493.log.gz 2023-10-09T06:42:44.298 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101619.log 2023-10-09T06:42:44.298 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176355.log 2023-10-09T06:42:44.298 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.101619.log: /var/log/ceph/ceph-client.admin.176355.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101619.log.gz 2023-10-09T06:42:44.298 INFO:teuthology.orchestra.run.smithi163.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.176355.log.gz 2023-10-09T06:42:44.298 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165602.log 2023-10-09T06:42:44.298 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107614.log 2023-10-09T06:42:44.298 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.165602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179042.log 2023-10-09T06:42:44.299 INFO:teuthology.orchestra.run.smithi163.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.165602.log.gz/var/log/ceph/ceph-client.admin.107614.log: 2023-10-09T06:42:44.299 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.179042.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.107614.log.gz 2023-10-09T06:42:44.299 INFO:teuthology.orchestra.run.smithi163.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.179042.log.gz 2023-10-09T06:42:44.299 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157467.log 2023-10-09T06:42:44.299 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165155.log 2023-10-09T06:42:44.299 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.157467.log: /var/log/ceph/ceph-client.admin.165155.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.157467.log.gz 2023-10-09T06:42:44.299 INFO:teuthology.orchestra.run.smithi163.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.165155.log.gz 2023-10-09T06:42:44.299 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178475.log 2023-10-09T06:42:44.300 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108136.log 2023-10-09T06:42:44.300 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.178475.log: /var/log/ceph/ceph-client.admin.108136.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.178475.log.gz 2023-10-09T06:42:44.300 INFO:teuthology.orchestra.run.smithi163.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108136.log.gz 2023-10-09T06:42:44.300 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100800.log 2023-10-09T06:42:44.300 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.175732.log 2023-10-09T06:42:44.300 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.100800.log: /var/log/ceph/ceph-client.0.175732.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100800.log.gz 2023-10-09T06:42:44.300 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.175732.log.gz 2023-10-09T06:42:44.301 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157839.log 2023-10-09T06:42:44.301 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97412.log 2023-10-09T06:42:44.301 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.157839.log: /var/log/ceph/ceph-client.admin.97412.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.157839.log.gz 2023-10-09T06:42:44.301 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97412.log.gz 2023-10-09T06:42:44.301 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102031.log 2023-10-09T06:42:44.301 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162283.log 2023-10-09T06:42:44.301 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.102031.log: /var/log/ceph/ceph-client.admin.162283.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102031.log.gz 2023-10-09T06:42:44.301 INFO:teuthology.orchestra.run.smithi163.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.162283.log.gz 2023-10-09T06:42:44.302 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166686.log 2023-10-09T06:42:44.302 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-10-09T06:42:44.302 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.166686.log: /var/log/ceph/ceph-mgr.y.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.166686.log.gz 2023-10-09T06:42:44.302 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96371.log 2023-10-09T06:42:44.302 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100758.log 2023-10-09T06:42:44.302 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.96371.log: 88.8%/var/log/ceph/ceph-client.admin.100758.log: -- replaced with /var/log/ceph/ceph-client.admin.96371.log.gz 2023-10-09T06:42:44.302 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100758.log.gz 2023-10-09T06:42:44.302 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106286.log 2023-10-09T06:42:44.303 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98297.log 2023-10-09T06:42:44.303 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.106286.log: /var/log/ceph/ceph-client.admin.98297.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.106286.log.gz 2023-10-09T06:42:44.303 INFO:teuthology.orchestra.run.smithi163.stderr: 94.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-10-09T06:42:44.303 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98297.log.gz 2023-10-09T06:42:44.303 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167654.log 2023-10-09T06:42:44.303 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167266.log 2023-10-09T06:42:44.303 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.167654.log: /var/log/ceph/ceph-client.admin.167266.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.167654.log.gz 2023-10-09T06:42:44.303 INFO:teuthology.orchestra.run.smithi163.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.167266.log.gz 2023-10-09T06:42:44.304 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175217.log 2023-10-09T06:42:44.304 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105896.log 2023-10-09T06:42:44.304 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.175217.log: /var/log/ceph/ceph-client.admin.105896.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.175217.log.gz 2023-10-09T06:42:44.304 INFO:teuthology.orchestra.run.smithi163.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.105896.log.gz 2023-10-09T06:42:44.304 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97959.log 2023-10-09T06:42:44.304 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98966.log 2023-10-09T06:42:44.304 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.97959.log: /var/log/ceph/ceph-client.admin.98966.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97959.log.gz 2023-10-09T06:42:44.304 INFO:teuthology.orchestra.run.smithi163.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98966.log.gz 2023-10-09T06:42:44.305 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97957.log 2023-10-09T06:42:44.305 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173138.log 2023-10-09T06:42:44.305 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.97957.log: /var/log/ceph/ceph-client.admin.173138.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97957.log.gz 2023-10-09T06:42:44.305 INFO:teuthology.orchestra.run.smithi163.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.173138.log.gz 2023-10-09T06:42:44.305 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164817.log 2023-10-09T06:42:44.305 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176691.log 2023-10-09T06:42:44.306 INFO:teuthology.orchestra.run.smithi181.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-10-09T06:42:44.325 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.164817.log: /var/log/ceph/ceph-client.admin.176691.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.164817.log.gz 2023-10-09T06:42:44.325 INFO:teuthology.orchestra.run.smithi163.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.176691.log.gz 2023-10-09T06:42:44.325 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175899.log 2023-10-09T06:42:44.325 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102199.log 2023-10-09T06:42:44.326 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.175899.log: /var/log/ceph/ceph-client.admin.102199.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.175899.log.gz 2023-10-09T06:42:44.326 INFO:teuthology.orchestra.run.smithi163.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.102199.log.gz 2023-10-09T06:42:44.326 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158659.log 2023-10-09T06:42:44.326 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169588.log 2023-10-09T06:42:44.326 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.158659.log: /var/log/ceph/ceph-client.admin.169588.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.158659.log.gz 2023-10-09T06:42:44.326 INFO:teuthology.orchestra.run.smithi163.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.169588.log.gz 2023-10-09T06:42:44.326 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166155.log 2023-10-09T06:42:44.326 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159615.log 2023-10-09T06:42:44.327 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.166155.log: /var/log/ceph/ceph-client.admin.159615.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.166155.log.gz 2023-10-09T06:42:44.327 INFO:teuthology.orchestra.run.smithi163.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.159615.log.gz 2023-10-09T06:42:44.327 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178434.log 2023-10-09T06:42:44.327 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171396.log 2023-10-09T06:42:44.327 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.178434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157405.log 2023-10-09T06:42:44.327 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.171396.log: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.178434.log.gz 2023-10-09T06:42:44.327 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.157405.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.171396.log.gz 2023-10-09T06:42:44.328 INFO:teuthology.orchestra.run.smithi163.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.157405.log.gz 2023-10-09T06:42:44.328 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163681.log 2023-10-09T06:42:44.328 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156739.log 2023-10-09T06:42:44.328 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.163681.log: /var/log/ceph/ceph-client.admin.156739.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.163681.log.gz 2023-10-09T06:42:44.328 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.156739.log.gz 2023-10-09T06:42:44.328 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169672.log 2023-10-09T06:42:44.328 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103963.log 2023-10-09T06:42:44.328 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.169672.log: /var/log/ceph/ceph-client.admin.103963.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.169672.log.gz 2023-10-09T06:42:44.329 INFO:teuthology.orchestra.run.smithi163.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.103963.log.gz 2023-10-09T06:42:44.329 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159008.log 2023-10-09T06:42:44.329 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107898.log 2023-10-09T06:42:44.329 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.159008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96975.log 2023-10-09T06:42:44.329 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.107898.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.159008.log.gz 2023-10-09T06:42:44.329 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.96975.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.107898.log.gz 2023-10-09T06:42:44.329 INFO:teuthology.orchestra.run.smithi163.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96975.log.gz 2023-10-09T06:42:44.329 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167178.log 2023-10-09T06:42:44.330 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106160.log 2023-10-09T06:42:44.330 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.167178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164945.log 2023-10-09T06:42:44.330 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.106160.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.167178.log.gz 2023-10-09T06:42:44.330 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.164945.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.106160.log.gz 2023-10-09T06:42:44.330 INFO:teuthology.orchestra.run.smithi163.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.164945.log.gz 2023-10-09T06:42:44.330 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106846.log 2023-10-09T06:42:44.330 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99722.log 2023-10-09T06:42:44.331 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.106846.log: /var/log/ceph/ceph-client.admin.99722.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106846.log.gz 2023-10-09T06:42:44.331 INFO:teuthology.orchestra.run.smithi163.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99722.log.gz 2023-10-09T06:42:44.331 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180420.log 2023-10-09T06:42:44.331 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176167.log 2023-10-09T06:42:44.331 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.180420.log: /var/log/ceph/ceph-client.admin.176167.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.180420.log.gz 2023-10-09T06:42:44.331 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.176167.log.gz 2023-10-09T06:42:44.331 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162241.log 2023-10-09T06:42:44.331 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173272.log 2023-10-09T06:42:44.332 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.162241.log: /var/log/ceph/ceph-client.admin.173272.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.162241.log.gz 2023-10-09T06:42:44.332 INFO:teuthology.orchestra.run.smithi163.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.173272.log.gz 2023-10-09T06:42:44.332 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173882.log 2023-10-09T06:42:44.332 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157321.log 2023-10-09T06:42:44.332 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.173882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160150.log 2023-10-09T06:42:44.332 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.157321.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.173882.log.gz 2023-10-09T06:42:44.332 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.160150.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.157321.log.gz 2023-10-09T06:42:44.332 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.160150.log.gz 2023-10-09T06:42:44.333 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161319.log 2023-10-09T06:42:44.333 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106076.log 2023-10-09T06:42:44.333 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.161319.log: /var/log/ceph/ceph-client.admin.106076.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.161319.log.gz 2023-10-09T06:42:44.333 INFO:teuthology.orchestra.run.smithi163.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.106076.log.gz 2023-10-09T06:42:44.333 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177868.log 2023-10-09T06:42:44.333 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159218.log 2023-10-09T06:42:44.333 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.177868.log: /var/log/ceph/ceph-client.admin.159218.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.177868.log.gz 2023-10-09T06:42:44.333 INFO:teuthology.orchestra.run.smithi163.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.159218.log.gz 2023-10-09T06:42:44.334 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164205.log 2023-10-09T06:42:44.334 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97952.log 2023-10-09T06:42:44.334 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.164205.log: /var/log/ceph/ceph-client.admin.97952.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.164205.log.gz 2023-10-09T06:42:44.334 INFO:teuthology.orchestra.run.smithi163.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.97952.log.gz 2023-10-09T06:42:44.334 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104563.log 2023-10-09T06:42:44.334 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159761.log 2023-10-09T06:42:44.334 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.104563.log: /var/log/ceph/ceph-client.admin.159761.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104563.log.gz 2023-10-09T06:42:44.335 INFO:teuthology.orchestra.run.smithi163.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.159761.log.gz 2023-10-09T06:42:44.335 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90594.log 2023-10-09T06:42:44.335 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165644.log 2023-10-09T06:42:44.335 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.90594.log: /var/log/ceph/ceph-client.admin.165644.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.90594.log.gz 2023-10-09T06:42:44.335 INFO:teuthology.orchestra.run.smithi163.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.165644.log.gz 2023-10-09T06:42:44.335 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163248.log 2023-10-09T06:42:44.335 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165071.log 2023-10-09T06:42:44.335 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.163248.log: /var/log/ceph/ceph-client.admin.165071.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.163248.log.gz 2023-10-09T06:42:44.336 INFO:teuthology.orchestra.run.smithi163.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.165071.log.gz 2023-10-09T06:42:44.336 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98546.log 2023-10-09T06:42:44.336 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101115.log 2023-10-09T06:42:44.336 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.98546.log: /var/log/ceph/ceph-client.admin.101115.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98546.log.gz 2023-10-09T06:42:44.336 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101115.log.gz 2023-10-09T06:42:44.336 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175133.log 2023-10-09T06:42:44.336 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100211.log 2023-10-09T06:42:44.337 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.175133.log: /var/log/ceph/ceph-client.admin.100211.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.175133.log.gz 2023-10-09T06:42:44.337 INFO:teuthology.orchestra.run.smithi163.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100211.log.gz 2023-10-09T06:42:44.337 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174877.log 2023-10-09T06:42:44.337 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161012.log 2023-10-09T06:42:44.337 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.174877.log: /var/log/ceph/ceph-client.admin.161012.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.174877.log.gz 2023-10-09T06:42:44.337 INFO:teuthology.orchestra.run.smithi163.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.161012.log.gz 2023-10-09T06:42:44.337 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167906.log 2023-10-09T06:42:44.337 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161715.log 2023-10-09T06:42:44.338 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.167906.log: /var/log/ceph/ceph-client.admin.161715.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.161715.log.gz 2023-10-09T06:42:44.338 INFO:teuthology.orchestra.run.smithi163.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.167906.log.gz 2023-10-09T06:42:44.338 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105071.log 2023-10-09T06:42:44.338 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161096.log 2023-10-09T06:42:44.338 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.105071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177616.log 2023-10-09T06:42:44.338 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.161096.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.105071.log.gz 2023-10-09T06:42:44.338 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.177616.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.161096.log.gz 2023-10-09T06:42:44.339 INFO:teuthology.orchestra.run.smithi163.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.177616.log.gz 2023-10-09T06:42:44.339 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100884.log 2023-10-09T06:42:44.339 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98299.log 2023-10-09T06:42:44.339 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.100884.log: /var/log/ceph/ceph-client.admin.98299.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100884.log.gz 2023-10-09T06:42:44.339 INFO:teuthology.orchestra.run.smithi163.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98299.log.gz 2023-10-09T06:42:44.339 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98298.log 2023-10-09T06:42:44.339 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164725.log 2023-10-09T06:42:44.339 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.98298.log: /var/log/ceph/ceph-client.admin.164725.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.98298.log.gz 2023-10-09T06:42:44.340 INFO:teuthology.orchestra.run.smithi163.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.164725.log.gz 2023-10-09T06:42:44.340 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163639.log 2023-10-09T06:42:44.340 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157178.log 2023-10-09T06:42:44.340 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.163639.log: /var/log/ceph/ceph-client.admin.157178.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.163639.log.gz 2023-10-09T06:42:44.340 INFO:teuthology.orchestra.run.smithi163.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.157178.log.gz 2023-10-09T06:42:44.340 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105197.log 2023-10-09T06:42:44.340 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107234.log 2023-10-09T06:42:44.340 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.105197.log: /var/log/ceph/ceph-client.admin.107234.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105197.log.gz 2023-10-09T06:42:44.341 INFO:teuthology.orchestra.run.smithi163.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.107234.log.gz 2023-10-09T06:42:44.341 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179510.log 2023-10-09T06:42:44.341 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177574.log 2023-10-09T06:42:44.341 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.179510.log: /var/log/ceph/ceph-client.admin.177574.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.179510.log.gz 2023-10-09T06:42:44.341 INFO:teuthology.orchestra.run.smithi163.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.177574.log.gz 2023-10-09T06:42:44.341 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171312.log 2023-10-09T06:42:44.341 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163723.log 2023-10-09T06:42:44.341 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.171312.log: /var/log/ceph/ceph-client.admin.163723.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.171312.log.gz 2023-10-09T06:42:44.342 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.163723.log.gz 2023-10-09T06:42:44.342 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161529.log 2023-10-09T06:42:44.342 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97626.log 2023-10-09T06:42:44.342 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.161529.log: /var/log/ceph/ceph-client.admin.97626.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.161529.log.gz 2023-10-09T06:42:44.342 INFO:teuthology.orchestra.run.smithi163.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97626.log.gz 2023-10-09T06:42:44.342 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101697.log 2023-10-09T06:42:44.342 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173096.log 2023-10-09T06:42:44.343 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.101697.log: /var/log/ceph/ceph-client.admin.173096.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101697.log.gz 2023-10-09T06:42:44.343 INFO:teuthology.orchestra.run.smithi163.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.173096.log.gz 2023-10-09T06:42:44.343 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105431.log 2023-10-09T06:42:44.343 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-09T06:42:44.343 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.105431.log: /var/log/ceph/ceph-osd.2.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105431.log.gz 2023-10-09T06:42:44.343 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104777.log 2023-10-09T06:42:44.343 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169096.log 2023-10-09T06:42:44.344 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.104777.log: /var/log/ceph/ceph-client.admin.169096.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.104777.log.gz 2023-10-09T06:42:44.344 INFO:teuthology.orchestra.run.smithi163.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.169096.log.gz 2023-10-09T06:42:44.344 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174050.log 2023-10-09T06:42:44.344 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173840.log 2023-10-09T06:42:44.344 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.174050.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.174050.log.gz 2023-10-09T06:42:44.344 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.173840.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.173840.log.gz 2023-10-09T06:42:44.344 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107530.log 2023-10-09T06:42:44.344 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.107530.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.107530.log.gz 2023-10-09T06:42:44.430 INFO:teuthology.orchestra.run.smithi181.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-10-09T06:42:44.433 INFO:teuthology.orchestra.run.smithi181.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-10-09T06:42:44.627 INFO:teuthology.orchestra.run.smithi163.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-09T06:42:44.662 INFO:teuthology.orchestra.run.smithi181.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-10-09T06:42:44.684 INFO:teuthology.orchestra.run.smithi181.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-10-09T06:42:44.706 INFO:teuthology.orchestra.run.smithi181.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-10-09T06:42:44.774 INFO:teuthology.orchestra.run.smithi181.stderr: 93.9% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-09T06:42:44.776 INFO:teuthology.orchestra.run.smithi181.stderr: 2023-10-09T06:42:44.776 INFO:teuthology.orchestra.run.smithi181.stderr:real 0m0.928s 2023-10-09T06:42:44.776 INFO:teuthology.orchestra.run.smithi181.stderr:user 0m4.744s 2023-10-09T06:42:44.777 INFO:teuthology.orchestra.run.smithi181.stderr:sys 0m0.191s 2023-10-09T06:42:44.789 INFO:teuthology.orchestra.run.smithi163.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-09T06:42:44.847 INFO:teuthology.orchestra.run.smithi163.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-10-09T06:42:44.906 INFO:teuthology.orchestra.run.smithi163.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-09T06:42:44.910 INFO:teuthology.orchestra.run.smithi163.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-09T06:42:48.703 INFO:teuthology.orchestra.run.smithi163.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-10-09T06:42:48.705 INFO:teuthology.orchestra.run.smithi163.stderr: 2023-10-09T06:42:48.705 INFO:teuthology.orchestra.run.smithi163.stderr:real 0m4.886s 2023-10-09T06:42:48.705 INFO:teuthology.orchestra.run.smithi163.stderr:user 0m8.953s 2023-10-09T06:42:48.705 INFO:teuthology.orchestra.run.smithi163.stderr:sys 0m0.674s 2023-10-09T06:42:48.706 INFO:tasks.ceph:Archiving logs... 2023-10-09T06:42:48.706 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-10-09_05:36:44-fs-wip-yuri-testing-2023-10-06-0949-quincy-testing-default-smithi/7418377/remote/smithi163/log 2023-10-09T06:42:48.707 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-09T06:42:50.719 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-10-09_05:36:44-fs-wip-yuri-testing-2023-10-06-0949-quincy-testing-default-smithi/7418377/remote/smithi181/log 2023-10-09T06:42:50.720 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-09T06:42:51.356 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-09T06:42:51.366 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-09T06:42:51.367 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-09T06:42:51.369 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-09T06:42:51.451 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-09T06:42:51.460 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-09T06:42:51.460 DEBUG:teuthology.orchestra.run.smithi163:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-09T06:42:51.463 DEBUG:teuthology.orchestra.run.smithi181:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-09T06:42:51.475 INFO:teuthology.orchestra.run.smithi163.stderr:bash: ntpq: command not found 2023-10-09T06:42:51.477 INFO:teuthology.orchestra.run.smithi181.stderr:bash: ntpq: command not found 2023-10-09T06:42:51.478 INFO:teuthology.orchestra.run.smithi163.stdout:210 Number of sources = 4 2023-10-09T06:42:51.479 INFO:teuthology.orchestra.run.smithi163.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-09T06:42:51.479 INFO:teuthology.orchestra.run.smithi163.stdout:=============================================================================== 2023-10-09T06:42:51.479 INFO:teuthology.orchestra.run.smithi163.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 2 -680us[ -680us] +/- 83ms 2023-10-09T06:42:51.479 INFO:teuthology.orchestra.run.smithi163.stdout:^* hv02.front.sepia.ceph.com 3 6 377 4 +245us[ +246us] +/- 66ms 2023-10-09T06:42:51.479 INFO:teuthology.orchestra.run.smithi163.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 +360us[ +360us] +/- 80ms 2023-10-09T06:42:51.479 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-09T06:42:51.481 INFO:teuthology.orchestra.run.smithi181.stdout:210 Number of sources = 4 2023-10-09T06:42:51.481 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-09T06:42:51.481 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2023-10-09T06:42:51.481 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 5 -680us[ -678us] +/- 83ms 2023-10-09T06:42:51.481 INFO:teuthology.orchestra.run.smithi181.stdout:^* hv02.front.sepia.ceph.com 3 6 377 4 +257us[ +259us] +/- 66ms 2023-10-09T06:42:51.481 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 +359us[ +359us] +/- 80ms 2023-10-09T06:42:51.482 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-09T06:42:51.482 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-09T06:42:51.492 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-09T06:42:51.493 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-09T06:42:51.506 DEBUG:teuthology.orchestra.run.smithi163:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-09T06:42:51.572 DEBUG:teuthology.orchestra.run.smithi181:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-09T06:42:51.670 DEBUG:teuthology.orchestra.run.smithi163:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-09T06:42:51.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-09T06:42:51.698 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-09T06:42:51.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-09T06:42:51.729 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-09T06:42:51.743 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-09T06:42:51.765 INFO:teuthology.task.internal:Duration was 1404.597280 seconds 2023-10-09T06:42:51.766 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-09T06:42:51.779 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-09T06:42:51.779 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-09T06:42:51.786 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-09T06:42:51.824 INFO:teuthology.orchestra.run.smithi163.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-09T06:42:51.830 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-09T06:42:52.161 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-09T06:42:52.161 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi163.front.sepia.ceph.com 2023-10-09T06:42:52.161 DEBUG:teuthology.orchestra.run.smithi163:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-09T06:42:52.219 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi181.front.sepia.ceph.com 2023-10-09T06:42:52.220 DEBUG:teuthology.orchestra.run.smithi181:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-09T06:42:52.240 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-09T06:42:52.240 DEBUG:teuthology.orchestra.run.smithi163:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-09T06:42:52.262 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-09T06:42:52.319 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-09T06:42:52.320 DEBUG:teuthology.orchestra.run.smithi163:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-09T06:42:52.347 DEBUG:teuthology.orchestra.run.smithi181:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-09T06:42:52.441 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-09T06:42:52.453 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-09T06:42:52.453 DEBUG:teuthology.orchestra.run.smithi163:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-09T06:42:52.478 DEBUG:teuthology.orchestra.run.smithi181:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-09T06:42:52.502 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-09T06:42:52.512 DEBUG:teuthology.orchestra.run.smithi163:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-09T06:42:52.520 DEBUG:teuthology.orchestra.run.smithi181:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-09T06:42:52.541 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern = core 2023-10-09T06:42:52.566 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = core 2023-10-09T06:42:52.580 DEBUG:teuthology.orchestra.run.smithi163:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-09T06:42:52.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-09T06:42:52.610 DEBUG:teuthology.orchestra.run.smithi181:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-09T06:42:52.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-09T06:42:52.634 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-09T06:42:52.646 INFO:teuthology.task.internal:Transferring archived files... 2023-10-09T06:42:52.647 DEBUG:teuthology.misc:Transferring archived files from smithi163:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-10-09_05:36:44-fs-wip-yuri-testing-2023-10-06-0949-quincy-testing-default-smithi/7418377/remote/smithi163 2023-10-09T06:42:52.647 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-09T06:42:52.683 DEBUG:teuthology.misc:Transferring archived files from smithi181:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-10-09_05:36:44-fs-wip-yuri-testing-2023-10-06-0949-quincy-testing-default-smithi/7418377/remote/smithi181 2023-10-09T06:42:52.683 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-09T06:42:52.719 INFO:teuthology.task.internal:Removing archive directory... 2023-10-09T06:42:52.719 DEBUG:teuthology.orchestra.run.smithi163:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-09T06:42:52.721 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-09T06:42:52.768 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-09T06:42:52.779 INFO:teuthology.task.internal:Not uploading archives. 2023-10-09T06:42:52.779 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-09T06:42:52.793 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-09T06:42:52.793 DEBUG:teuthology.orchestra.run.smithi163:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-09T06:42:52.796 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-09T06:42:52.808 INFO:teuthology.orchestra.run.smithi163.stdout: 262153 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 9 06:42 /home/ubuntu/cephtest 2023-10-09T06:42:52.822 INFO:teuthology.orchestra.run.smithi181.stdout: 262153 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 9 06:42 /home/ubuntu/cephtest 2023-10-09T06:42:52.824 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-09T06:42:52.835 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-09T06:42:52.876 INFO:teuthology.nuke:Checking targets against current locks 2023-10-09T06:42:52.902 DEBUG:teuthology.nuke:shortname: smithi163 2023-10-09T06:42:52.903 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-09T06:42:52.925 DEBUG:teuthology.nuke:shortname: smithi181 2023-10-09T06:42:52.926 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-09T06:42:52.931 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi163.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-10-09_05:36:44-fs-wip-yuri-testing-2023-10-06-0949-quincy-testing-default-smithi/7418377', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-09 06:10:37.184420', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN9GGg0ezV7qRelQs7pwRE+MZtSu6jseUyPPtZsCZIeNOgmh5+zpfPVNBkjVEpqTmyPSjfzG3lVrFWglAlauRCs='} 2023-10-09T06:42:52.956 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-10-09_05:36:44-fs-wip-yuri-testing-2023-10-06-0949-quincy-testing-default-smithi/7418377', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-09 06:10:37.185594', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFYiqfrDjs8ACW9S/FUJkeTfDNLufZAP2urZo97qL+CxrOpH2qyVALjYHlaRDJXbPBxrqZmj0/7St9RDFNpBooo='} 2023-10-09T06:42:52.976 INFO:teuthology.orchestra.console.smithi163:Power off 2023-10-09T06:42:52.976 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-09T06:42:53.002 DEBUG:teuthology.orchestra.console.smithi163:power off output: Chassis Power Control: Down/Off 2023-10-09T06:42:53.002 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-09T06:42:53.019 INFO:teuthology.orchestra.console.smithi181:Power off 2023-10-09T06:42:53.019 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-09T06:42:53.032 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on 2023-10-09T06:42:53.046 DEBUG:teuthology.orchestra.console.smithi181:power off output: Chassis Power Control: Down/Off 2023-10-09T06:42:53.046 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-09T06:42:53.070 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2023-10-09T06:42:57.033 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-09T06:42:57.072 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-09T06:42:57.147 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on 2023-10-09T06:42:57.186 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2023-10-09T06:43:01.150 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-09T06:43:01.187 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-09T06:43:01.261 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is off 2023-10-09T06:43:01.298 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2023-10-09T06:43:01.363 INFO:teuthology.orchestra.console.smithi163:Power off completed 2023-10-09T06:43:05.302 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-09T06:43:05.414 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off 2023-10-09T06:43:05.514 INFO:teuthology.orchestra.console.smithi181:Power off completed 2023-10-09T06:43:05.619 INFO:teuthology.run:Summary data: description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} duration: 1404.5972797870636 failure_reason: 'Test failure: test_volume_rm (tasks.cephfs.test_volumes.TestVolumes)' flavor: default owner: scheduled_vshankar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=e68320d3c7b5416b846fa78dd8c23c89 status: fail success: false 2023-10-09T06:43:05.620 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-09T06:43:05.698 INFO:teuthology.run:FAIL